2024-09-16T15:28:12.993 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-16T15:28:13.014 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-16T15:28:13.106 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/avan-2024-09-16_15:21:40-fs:volumes-wip-athakkar-testing-2024-09-16-1454-distro-default-smithi/7907354 branch: wip-athakkar-testing-2024-09-16-1454 description: fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/clone-progress}} email: athakkar@redhat.com first_in_suite: false job_id: '7907354' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: avan-2024-09-16_15:21:40-fs:volumes-wip-athakkar-testing-2024-09-16-1454-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-athakkar-testing-2024-09-16-1454 ceph: cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mon: mon op complaint time: 120 mon warn on pool no app: false osd: osd op complaint time: 180 conf: client: pid file: '' global: ms die on skipped message: false mgr: client mount timeout: 30 debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - fs.*is degraded - filesystem is degraded - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - filesystem is offline - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - insufficient standby MDS daemons available - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: 33f95d2ae1bcbe7eb04404303f179eeeaa6982df ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 33f95d2ae1bcbe7eb04404303f179eeeaa6982df kclient: syntax: v1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-athakkar-testing-2024-09-16-1454 sha1: 33f95d2ae1bcbe7eb04404303f179eeeaa6982df owner: scheduled_avan@teuthology priority: 80 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 3077 sha1: 33f95d2ae1bcbe7eb04404303f179eeeaa6982df sleep_before_teardown: 0 subset: 1/20 suite: fs:volumes suite_branch: wip-athakkar-testing-2024-09-16-1454 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_33f95d2ae1bcbe7eb04404303f179eeeaa6982df/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 33f95d2ae1bcbe7eb04404303f179eeeaa6982df syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi151.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGck6mysMoarx5v9Lx4Mg0nzqIYwd1rFOpGPdwdxW5dZXHGVgwj2CNE3PMzxjDmPNqX7z5lAeDgx+WoqYDG+CWs= smithi167.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFDumuaBHu91r2fvqaNRnjzggIEpbYvy1Y8kGjCZvrcHhG8Fkz6KsCl4REHXM6pmIevcZGtrmFJDG/DpItnhXGY= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volumes.TestCloneProgressReporter teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-16_15:21:40 tube: smithi user: avan verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-16T15:28:13.106 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_33f95d2ae1bcbe7eb04404303f179eeeaa6982df/qa; will attempt to use it 2024-09-16T15:28:13.107 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_33f95d2ae1bcbe7eb04404303f179eeeaa6982df/qa/tasks 2024-09-16T15:28:13.107 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-16T15:28:13.108 INFO:teuthology.task.internal:Checking packages... 2024-09-16T15:28:13.129 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '33f95d2ae1bcbe7eb04404303f179eeeaa6982df' 2024-09-16T15:28:13.129 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-16T15:28:13.129 INFO:teuthology.packaging:ref: None 2024-09-16T15:28:13.129 INFO:teuthology.packaging:tag: None 2024-09-16T15:28:13.129 INFO:teuthology.packaging:branch: wip-athakkar-testing-2024-09-16-1454 2024-09-16T15:28:13.129 INFO:teuthology.packaging:sha1: 33f95d2ae1bcbe7eb04404303f179eeeaa6982df 2024-09-16T15:28:13.129 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-athakkar-testing-2024-09-16-1454 2024-09-16T15:28:13.276 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4938.g33f95d2a 2024-09-16T15:28:13.277 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-16T15:28:13.284 INFO:teuthology.task.internal:no buildpackages task found 2024-09-16T15:28:13.284 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-16T15:28:13.356 INFO:teuthology.task.internal:Saving configuration 2024-09-16T15:28:13.372 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-16T15:28:13.378 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-16T15:28:13.403 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi151.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/avan-2024-09-16_15:21:40-fs:volumes-wip-athakkar-testing-2024-09-16-1454-distro-default-smithi/7907354', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 15:22:49.285621', 'locked_by': 'scheduled_avan@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGck6mysMoarx5v9Lx4Mg0nzqIYwd1rFOpGPdwdxW5dZXHGVgwj2CNE3PMzxjDmPNqX7z5lAeDgx+WoqYDG+CWs='} 2024-09-16T15:28:13.426 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi167.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/avan-2024-09-16_15:21:40-fs:volumes-wip-athakkar-testing-2024-09-16-1454-distro-default-smithi/7907354', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 15:22:49.284094', 'locked_by': 'scheduled_avan@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFDumuaBHu91r2fvqaNRnjzggIEpbYvy1Y8kGjCZvrcHhG8Fkz6KsCl4REHXM6pmIevcZGtrmFJDG/DpItnhXGY='} 2024-09-16T15:28:13.427 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-16T15:28:13.432 INFO:teuthology.task.internal:roles: ubuntu@smithi151.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-09-16T15:28:13.432 INFO:teuthology.task.internal:roles: ubuntu@smithi167.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-09-16T15:28:13.432 INFO:teuthology.run_tasks:Running task console_log... 2024-09-16T15:28:13.531 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f0d0ae5fbe0>, signals=[15]) 2024-09-16T15:28:13.532 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-16T15:28:13.538 INFO:teuthology.task.internal:Opening connections... 2024-09-16T15:28:13.538 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi151.front.sepia.ceph.com 2024-09-16T15:28:13.539 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T15:28:13.612 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi167.front.sepia.ceph.com 2024-09-16T15:28:13.613 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T15:28:13.683 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-16T15:28:13.689 DEBUG:teuthology.orchestra.run.smithi151:> uname -m 2024-09-16T15:28:13.708 INFO:teuthology.orchestra.run.smithi151.stdout:x86_64 2024-09-16T15:28:13.708 DEBUG:teuthology.orchestra.run.smithi151:> cat /etc/os-release 2024-09-16T15:28:13.765 INFO:teuthology.orchestra.run.smithi151.stdout:NAME="CentOS Stream" 2024-09-16T15:28:13.766 INFO:teuthology.orchestra.run.smithi151.stdout:VERSION="9" 2024-09-16T15:28:13.766 INFO:teuthology.orchestra.run.smithi151.stdout:ID="centos" 2024-09-16T15:28:13.766 INFO:teuthology.orchestra.run.smithi151.stdout:ID_LIKE="rhel fedora" 2024-09-16T15:28:13.766 INFO:teuthology.orchestra.run.smithi151.stdout:VERSION_ID="9" 2024-09-16T15:28:13.766 INFO:teuthology.orchestra.run.smithi151.stdout:PLATFORM_ID="platform:el9" 2024-09-16T15:28:13.766 INFO:teuthology.orchestra.run.smithi151.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-16T15:28:13.766 INFO:teuthology.orchestra.run.smithi151.stdout:ANSI_COLOR="0;31" 2024-09-16T15:28:13.766 INFO:teuthology.orchestra.run.smithi151.stdout:LOGO="fedora-logo-icon" 2024-09-16T15:28:13.766 INFO:teuthology.orchestra.run.smithi151.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-16T15:28:13.766 INFO:teuthology.orchestra.run.smithi151.stdout:HOME_URL="https://centos.org/" 2024-09-16T15:28:13.766 INFO:teuthology.orchestra.run.smithi151.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-16T15:28:13.766 INFO:teuthology.orchestra.run.smithi151.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-16T15:28:13.766 INFO:teuthology.orchestra.run.smithi151.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-16T15:28:13.767 INFO:teuthology.lock.ops:Updating smithi151.front.sepia.ceph.com on lock server 2024-09-16T15:28:13.805 DEBUG:teuthology.orchestra.run.smithi167:> uname -m 2024-09-16T15:28:13.823 INFO:teuthology.orchestra.run.smithi167.stdout:x86_64 2024-09-16T15:28:13.823 DEBUG:teuthology.orchestra.run.smithi167:> cat /etc/os-release 2024-09-16T15:28:13.881 INFO:teuthology.orchestra.run.smithi167.stdout:NAME="CentOS Stream" 2024-09-16T15:28:13.881 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION="9" 2024-09-16T15:28:13.881 INFO:teuthology.orchestra.run.smithi167.stdout:ID="centos" 2024-09-16T15:28:13.881 INFO:teuthology.orchestra.run.smithi167.stdout:ID_LIKE="rhel fedora" 2024-09-16T15:28:13.882 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION_ID="9" 2024-09-16T15:28:13.882 INFO:teuthology.orchestra.run.smithi167.stdout:PLATFORM_ID="platform:el9" 2024-09-16T15:28:13.882 INFO:teuthology.orchestra.run.smithi167.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-16T15:28:13.882 INFO:teuthology.orchestra.run.smithi167.stdout:ANSI_COLOR="0;31" 2024-09-16T15:28:13.882 INFO:teuthology.orchestra.run.smithi167.stdout:LOGO="fedora-logo-icon" 2024-09-16T15:28:13.882 INFO:teuthology.orchestra.run.smithi167.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-16T15:28:13.882 INFO:teuthology.orchestra.run.smithi167.stdout:HOME_URL="https://centos.org/" 2024-09-16T15:28:13.882 INFO:teuthology.orchestra.run.smithi167.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-16T15:28:13.882 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-16T15:28:13.882 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-16T15:28:13.882 INFO:teuthology.lock.ops:Updating smithi167.front.sepia.ceph.com on lock server 2024-09-16T15:28:13.912 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-16T15:28:13.922 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-16T15:28:13.973 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-16T15:28:13.973 DEBUG:teuthology.orchestra.run.smithi151:> test '!' -e /home/ubuntu/cephtest 2024-09-16T15:28:13.976 DEBUG:teuthology.orchestra.run.smithi167:> test '!' -e /home/ubuntu/cephtest 2024-09-16T15:28:13.993 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-16T15:28:13.999 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-16T15:28:13.999 DEBUG:teuthology.orchestra.run.smithi151:> test -z $(ls -A /var/lib/ceph) 2024-09-16T15:28:14.033 DEBUG:teuthology.orchestra.run.smithi167:> test -z $(ls -A /var/lib/ceph) 2024-09-16T15:28:14.342 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-16T15:28:14.399 INFO:teuthology.run_tasks:Running task kernel... 2024-09-16T15:28:14.413 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-16T15:28:14.413 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2024-09-16T15:28:14.414 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2024-09-16T15:28:14.414 DEBUG:teuthology.orchestra.run.smithi151:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T15:28:14.414 DEBUG:teuthology.orchestra.run.smithi167:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T15:28:14.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T15:28:14.432 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-09-16T15:28:14.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T15:28:14.435 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-09-16T15:28:14.618 DEBUG:teuthology.task.kernel:sha1 for client.0 is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-16T15:28:14.618 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-16T15:28:14.618 DEBUG:teuthology.orchestra.run.smithi151:> uname -r 2024-09-16T15:28:14.620 DEBUG:teuthology.task.kernel:sha1 for client.1 is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-16T15:28:14.620 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-16T15:28:14.620 DEBUG:teuthology.orchestra.run.smithi167:> uname -r 2024-09-16T15:28:14.636 INFO:teuthology.orchestra.run.smithi151.stdout:5.14.0-503.el9.x86_64 2024-09-16T15:28:14.636 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-503.el9.x86_64 vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-16T15:28:14.637 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-16T15:28:14.637 DEBUG:teuthology.orchestra.run.smithi151:> sudo yum upgrade -y linux-firmware 2024-09-16T15:28:14.639 INFO:teuthology.orchestra.run.smithi167.stdout:5.14.0-503.el9.x86_64 2024-09-16T15:28:14.639 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-503.el9.x86_64 vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-16T15:28:14.639 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-16T15:28:14.639 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum upgrade -y linux-firmware 2024-09-16T15:28:16.956 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.2 MB 00:00 2024-09-16T15:28:18.216 INFO:teuthology.orchestra.run.smithi151.stdout:CentOS Stream 9 - BaseOS 4.6 MB/s | 8.2 MB 00:01 2024-09-16T15:28:18.966 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - AppStream 23 MB/s | 20 MB 00:00 2024-09-16T15:28:20.150 INFO:teuthology.orchestra.run.smithi151.stdout:CentOS Stream 9 - AppStream 26 MB/s | 20 MB 00:00 2024-09-16T15:28:24.125 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - CRB 4.7 MB/s | 6.5 MB 00:01 2024-09-16T15:28:25.217 INFO:teuthology.orchestra.run.smithi151.stdout:CentOS Stream 9 - CRB 5.1 MB/s | 6.5 MB 00:01 2024-09-16T15:28:25.567 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - Extras packages 81 kB/s | 19 kB 00:00 2024-09-16T15:28:26.667 INFO:teuthology.orchestra.run.smithi151.stdout:CentOS Stream 9 - Extras packages 76 kB/s | 19 kB 00:00 2024-09-16T15:28:26.830 INFO:teuthology.orchestra.run.smithi167.stdout:Extra Packages for Enterprise Linux 20 MB/s | 23 MB 00:01 2024-09-16T15:28:28.247 INFO:teuthology.orchestra.run.smithi151.stdout:Extra Packages for Enterprise Linux 16 MB/s | 23 MB 00:01 2024-09-16T15:28:32.261 INFO:teuthology.orchestra.run.smithi167.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-16T15:28:33.769 INFO:teuthology.orchestra.run.smithi151.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-09-16T15:28:34.321 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2024-09-16T15:28:34.322 INFO:teuthology.orchestra.run.smithi167.stdout:Nothing to do. 2024-09-16T15:28:34.322 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2024-09-16T15:28:34.428 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.1... 2024-09-16T15:28:34.428 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-16T15:28:34.574 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ 2024-09-16T15:28:34.575 DEBUG:teuthology.orchestra.run.smithi167:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ --input-file=- 2024-09-16T15:28:34.577 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi167.front.sepia.ceph.com to complete... 2024-09-16T15:28:35.894 INFO:teuthology.orchestra.run.smithi151.stdout:Dependencies resolved. 2024-09-16T15:28:35.895 INFO:teuthology.orchestra.run.smithi151.stdout:Nothing to do. 2024-09-16T15:28:35.895 INFO:teuthology.orchestra.run.smithi151.stdout:Complete! 2024-09-16T15:28:35.993 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-16T15:28:35.993 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-16T15:28:36.143 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ 2024-09-16T15:28:36.143 DEBUG:teuthology.orchestra.run.smithi151:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ --input-file=- 2024-09-16T15:28:36.146 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi151.front.sepia.ceph.com to complete... 2024-09-16T15:28:43.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16 15:28:43 URL:https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm [359633503/359633503] -> "/tmp/kernel.x86_64.rpm" [1] 2024-09-16T15:28:43.853 INFO:teuthology.orchestra.run.smithi167.stderr:FINISHED --2024-09-16 15:28:43-- 2024-09-16T15:28:43.853 INFO:teuthology.orchestra.run.smithi167.stderr:Total wall clock time: 9.3s 2024-09-16T15:28:43.853 INFO:teuthology.orchestra.run.smithi167.stderr:Downloaded: 1 files, 343M in 9.0s (38.3 MB/s) 2024-09-16T15:28:43.856 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.1... 2024-09-16T15:28:43.856 DEBUG:teuthology.orchestra.run.smithi167:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-09-16T15:28:46.013 INFO:teuthology.orchestra.run.smithi167.stdout:Verifying... ######################################## 2024-09-16T15:28:46.187 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing... ######################################## 2024-09-16T15:28:46.300 INFO:teuthology.orchestra.run.smithi167.stdout:Updating / installing... 2024-09-16T15:28:46.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16 15:28:46 URL:https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm [359633503/359633503] -> "/tmp/kernel.x86_64.rpm" [1] 2024-09-16T15:28:46.786 INFO:teuthology.orchestra.run.smithi151.stderr:FINISHED --2024-09-16 15:28:46-- 2024-09-16T15:28:46.786 INFO:teuthology.orchestra.run.smithi151.stderr:Total wall clock time: 11s 2024-09-16T15:28:46.786 INFO:teuthology.orchestra.run.smithi151.stderr:Downloaded: 1 files, 343M in 10s (33.6 MB/s) 2024-09-16T15:28:46.788 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-16T15:28:46.789 DEBUG:teuthology.orchestra.run.smithi151:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-09-16T15:28:48.902 INFO:teuthology.orchestra.run.smithi151.stdout:Verifying... ######################################## 2024-09-16T15:28:49.055 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing... ######################################## 2024-09-16T15:28:49.154 INFO:teuthology.orchestra.run.smithi151.stdout:Updating / installing... 2024-09-16T15:28:54.308 INFO:teuthology.orchestra.run.smithi167.stdout:kernel-6.11.0_rc5_g2ca44a2f35ca-1 ######################################## 2024-09-16T15:28:57.126 INFO:teuthology.orchestra.run.smithi151.stdout:kernel-6.11.0_rc5_g2ca44a2f35ca-1 ######################################## 2024-09-16T15:29:34.890 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi167.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-09-16T15:29:34.891 DEBUG:teuthology.orchestra.run.smithi167:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-09-16T15:29:34.977 INFO:teuthology.orchestra.run.smithi167.stdout:/boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-16T15:29:34.978 INFO:teuthology.orchestra.run.smithi167.stdout:/boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-16T15:29:34.978 INFO:teuthology.orchestra.run.smithi167.stdout:/boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-16T15:29:34.978 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca 2024-09-16T15:29:34.978 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/System.map 2024-09-16T15:29:34.978 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/config 2024-09-16T15:29:34.978 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel 2024-09-16T15:29:34.978 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch 2024-09-16T15:29:34.978 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86 2024-09-16T15:29:34.978 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto 2024-09-16T15:29:34.978 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-16T15:29:34.978 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-16T15:29:34.979 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-16T15:29:34.979 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-16T15:29:34.979 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-16T15:29:34.979 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-16T15:29:34.979 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-16T15:29:34.979 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32c-intel.ko 2024-09-16T15:29:34.979 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-16T15:29:34.979 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-16T15:29:34.979 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-16T15:29:34.979 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-16T15:29:34.979 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-16T15:29:34.979 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-16T15:29:34.980 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-16T15:29:34.980 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-16T15:29:34.980 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-16T15:29:34.980 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-16T15:29:34.980 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-16T15:29:34.980 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-16T15:29:34.980 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel 2024-09-16T15:29:34.980 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu 2024-09-16T15:29:34.980 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce 2024-09-16T15:29:34.980 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-16T15:29:34.980 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm 2024-09-16T15:29:34.980 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-16T15:29:34.981 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-16T15:29:34.981 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-16T15:29:34.981 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto 2024-09-16T15:29:34.981 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-16T15:29:34.981 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-16T15:29:34.981 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-16T15:29:34.981 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx 2024-09-16T15:29:34.981 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-16T15:29:34.981 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-16T15:29:34.981 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-16T15:29:34.981 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-16T15:29:34.981 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-16T15:29:34.981 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-16T15:29:34.981 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-16T15:29:34.982 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-16T15:29:34.982 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-16T15:29:34.982 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-16T15:29:34.982 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-16T15:29:34.982 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-16T15:29:34.982 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-16T15:29:34.982 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-16T15:29:34.982 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-16T15:29:34.982 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-16T15:29:34.982 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-16T15:29:34.982 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-16T15:29:34.982 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-16T15:29:34.983 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-16T15:29:34.983 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-16T15:29:34.983 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-16T15:29:34.983 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-16T15:29:34.983 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecc.ko 2024-09-16T15:29:34.983 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecdh_generic.ko 2024-09-16T15:29:34.983 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-16T15:29:34.983 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-16T15:29:34.983 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-16T15:29:34.983 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-16T15:29:34.983 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-16T15:29:34.983 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-16T15:29:34.983 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-16T15:29:34.984 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-16T15:29:34.984 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-16T15:29:34.984 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-16T15:29:34.984 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-16T15:29:34.984 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-16T15:29:34.984 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-16T15:29:34.984 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-16T15:29:34.984 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-16T15:29:34.984 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-16T15:29:34.984 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-16T15:29:34.984 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-16T15:29:34.984 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-16T15:29:34.984 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-16T15:29:34.984 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-16T15:29:34.984 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-16T15:29:34.985 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers 2024-09-16T15:29:34.985 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi 2024-09-16T15:29:34.985 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-16T15:29:34.985 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-16T15:29:34.985 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-16T15:29:34.985 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-16T15:29:34.985 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-16T15:29:34.985 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-16T15:29:34.985 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-16T15:29:34.985 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata 2024-09-16T15:29:34.985 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-16T15:29:34.985 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-16T15:29:34.985 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ata_generic.ko 2024-09-16T15:29:34.985 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-16T15:29:34.986 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-16T15:29:34.986 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-16T15:29:34.986 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-16T15:29:34.986 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-16T15:29:34.986 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-16T15:29:34.986 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-16T15:29:34.986 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-16T15:29:34.986 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-16T15:29:34.986 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-16T15:29:34.986 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-16T15:29:34.986 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-16T15:29:34.986 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-16T15:29:34.986 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-16T15:29:34.986 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-16T15:29:34.986 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-16T15:29:34.986 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-16T15:29:34.987 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-16T15:29:34.987 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-16T15:29:34.987 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-16T15:29:34.987 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-16T15:29:34.987 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-16T15:29:34.987 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-16T15:29:34.987 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-16T15:29:34.987 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-16T15:29:34.987 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-16T15:29:34.987 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-16T15:29:34.987 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-16T15:29:34.987 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-16T15:29:34.987 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-16T15:29:34.988 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-16T15:29:34.988 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-16T15:29:34.988 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-16T15:29:34.988 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-16T15:29:34.988 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-16T15:29:34.988 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sis.ko 2024-09-16T15:29:34.988 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-16T15:29:34.988 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-16T15:29:34.988 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-16T15:29:34.988 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-16T15:29:34.988 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-16T15:29:34.988 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-16T15:29:34.988 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-16T15:29:34.989 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-16T15:29:34.989 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-16T15:29:34.989 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-16T15:29:34.989 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-16T15:29:34.989 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-16T15:29:34.989 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-16T15:29:34.989 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-16T15:29:34.989 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-16T15:29:34.989 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-16T15:29:34.989 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-16T15:29:34.989 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay 2024-09-16T15:29:34.989 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-16T15:29:34.989 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base 2024-09-16T15:29:34.989 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap 2024-09-16T15:29:34.990 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-i2c.ko 2024-09-16T15:29:34.990 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-mmio.ko 2024-09-16T15:29:34.990 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma 2024-09-16T15:29:34.990 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-16T15:29:34.990 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block 2024-09-16T15:29:34.990 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe 2024-09-16T15:29:34.990 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-16T15:29:34.990 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/brd.ko 2024-09-16T15:29:34.990 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd 2024-09-16T15:29:34.990 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-16T15:29:34.990 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-16T15:29:34.990 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/loop.ko 2024-09-16T15:29:34.990 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx 2024-09-16T15:29:34.990 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-16T15:29:34.990 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-16T15:29:34.991 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk 2024-09-16T15:29:34.991 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-16T15:29:34.991 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-16T15:29:34.991 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-16T15:29:34.991 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/virtio_blk.ko 2024-09-16T15:29:34.991 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback 2024-09-16T15:29:34.991 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-16T15:29:34.991 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkfront.ko 2024-09-16T15:29:34.991 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram 2024-09-16T15:29:34.991 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-16T15:29:34.991 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char 2024-09-16T15:29:34.991 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-16T15:29:34.991 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random 2024-09-16T15:29:34.991 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-16T15:29:34.991 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-16T15:29:34.992 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-16T15:29:34.992 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-16T15:29:34.992 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-16T15:29:34.992 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi 2024-09-16T15:29:34.992 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-16T15:29:34.992 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-16T15:29:34.992 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-16T15:29:34.992 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-16T15:29:34.992 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-16T15:29:34.992 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-16T15:29:34.992 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-16T15:29:34.992 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave 2024-09-16T15:29:34.992 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-16T15:29:34.992 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-16T15:29:34.992 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-16T15:29:34.993 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm 2024-09-16T15:29:34.993 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm.ko 2024-09-16T15:29:34.993 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-16T15:29:34.993 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-16T15:29:34.993 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-16T15:29:34.993 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis.ko 2024-09-16T15:29:34.993 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-09-16T15:29:34.993 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/uv_mmtimer.ko 2024-09-16T15:29:34.993 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/virtio_console.ko 2024-09-16T15:29:34.993 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq 2024-09-16T15:29:34.993 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-09-16T15:29:34.993 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-16T15:29:34.993 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-16T15:29:34.993 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-09-16T15:29:34.993 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/powernow-k8.ko 2024-09-16T15:29:34.994 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-16T15:29:34.994 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto 2024-09-16T15:29:34.994 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp 2024-09-16T15:29:34.994 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-16T15:29:34.994 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-16T15:29:34.994 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel 2024-09-16T15:29:34.994 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat 2024-09-16T15:29:34.994 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common 2024-09-16T15:29:34.994 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-16T15:29:34.994 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-09-16T15:29:34.994 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-16T15:29:34.994 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-16T15:29:34.994 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-16T15:29:34.994 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio 2024-09-16T15:29:34.994 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-16T15:29:34.994 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca 2024-09-16T15:29:34.995 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-16T15:29:34.995 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq 2024-09-16T15:29:34.995 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-09-16T15:29:34.995 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma 2024-09-16T15:29:34.995 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw 2024-09-16T15:29:34.995 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-16T15:29:34.995 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat 2024-09-16T15:29:34.995 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-16T15:29:34.995 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac 2024-09-16T15:29:34.995 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-16T15:29:34.995 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-16T15:29:34.995 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-16T15:29:34.995 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-16T15:29:34.995 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-16T15:29:34.995 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-16T15:29:34.996 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-16T15:29:34.996 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-16T15:29:34.996 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-16T15:29:34.996 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-16T15:29:34.996 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-16T15:29:34.996 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-16T15:29:34.996 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-16T15:29:34.996 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware 2024-09-16T15:29:34.996 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/edd.ko 2024-09-16T15:29:34.996 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-16T15:29:34.996 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio 2024-09-16T15:29:34.996 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-16T15:29:34.996 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-16T15:29:34.997 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid 2024-09-16T15:29:34.997 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-16T15:29:34.997 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-16T15:29:34.997 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-16T15:29:34.997 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-16T15:29:34.997 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-16T15:29:34.997 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-16T15:29:34.997 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-16T15:29:34.997 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-16T15:29:34.997 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-16T15:29:34.997 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-16T15:29:34.997 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-16T15:29:34.997 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-16T15:29:34.997 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-16T15:29:34.998 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-16T15:29:34.998 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-16T15:29:34.998 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-16T15:29:34.998 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-16T15:29:34.998 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-16T15:29:34.998 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-16T15:29:34.998 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-16T15:29:34.998 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-16T15:29:34.998 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-16T15:29:34.998 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-16T15:29:34.998 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-16T15:29:34.998 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-16T15:29:34.998 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-16T15:29:34.999 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-16T15:29:34.999 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-16T15:29:34.999 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-16T15:29:34.999 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-16T15:29:34.999 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-16T15:29:34.999 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-16T15:29:34.999 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-16T15:29:34.999 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-16T15:29:34.999 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-16T15:29:34.999 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-16T15:29:34.999 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-16T15:29:34.999 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-16T15:29:34.999 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-16T15:29:34.999 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-16T15:29:35.000 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-16T15:29:35.000 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-16T15:29:35.000 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-16T15:29:35.000 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-16T15:29:35.000 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-16T15:29:35.000 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-16T15:29:35.000 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-16T15:29:35.000 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-16T15:29:35.000 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-16T15:29:35.000 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-16T15:29:35.000 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-16T15:29:35.000 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-16T15:29:35.000 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-16T15:29:35.000 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-16T15:29:35.001 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-16T15:29:35.001 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-16T15:29:35.001 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-16T15:29:35.001 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-16T15:29:35.001 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-16T15:29:35.001 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-16T15:29:35.001 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-16T15:29:35.001 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-16T15:29:35.001 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-16T15:29:35.001 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-16T15:29:35.001 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv 2024-09-16T15:29:35.001 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-16T15:29:35.001 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-16T15:29:35.001 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-16T15:29:35.001 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon 2024-09-16T15:29:35.002 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-16T15:29:35.002 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-16T15:29:35.002 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-16T15:29:35.002 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-16T15:29:35.002 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-16T15:29:35.002 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-16T15:29:35.002 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-16T15:29:35.002 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-16T15:29:35.002 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-16T15:29:35.002 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-16T15:29:35.002 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-16T15:29:35.002 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-16T15:29:35.002 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-16T15:29:35.002 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-16T15:29:35.002 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-16T15:29:35.003 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-16T15:29:35.003 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-16T15:29:35.003 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-16T15:29:35.003 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-16T15:29:35.003 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-16T15:29:35.003 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-16T15:29:35.003 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-16T15:29:35.003 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-16T15:29:35.003 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-16T15:29:35.003 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-16T15:29:35.003 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-16T15:29:35.003 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-16T15:29:35.003 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-16T15:29:35.003 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-16T15:29:35.004 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-16T15:29:35.004 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-16T15:29:35.004 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-16T15:29:35.004 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-16T15:29:35.004 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-16T15:29:35.004 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-16T15:29:35.004 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-16T15:29:35.004 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-16T15:29:35.004 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-16T15:29:35.004 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-16T15:29:35.004 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-16T15:29:35.004 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-16T15:29:35.004 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-16T15:29:35.005 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-16T15:29:35.005 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-16T15:29:35.005 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-16T15:29:35.005 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-16T15:29:35.005 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-16T15:29:35.005 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-16T15:29:35.005 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-16T15:29:35.005 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-16T15:29:35.005 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-16T15:29:35.005 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-16T15:29:35.005 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-16T15:29:35.005 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-16T15:29:35.005 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-16T15:29:35.005 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-16T15:29:35.006 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-16T15:29:35.006 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-16T15:29:35.006 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-16T15:29:35.006 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-16T15:29:35.006 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-16T15:29:35.006 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-16T15:29:35.006 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-16T15:29:35.006 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-16T15:29:35.006 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-16T15:29:35.006 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-16T15:29:35.006 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-16T15:29:35.006 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-16T15:29:35.006 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-16T15:29:35.006 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-16T15:29:35.007 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-16T15:29:35.007 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-16T15:29:35.007 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-16T15:29:35.007 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-16T15:29:35.007 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-16T15:29:35.007 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-16T15:29:35.007 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-16T15:29:35.007 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-16T15:29:35.007 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-16T15:29:35.007 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-16T15:29:35.007 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-16T15:29:35.007 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-16T15:29:35.007 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-16T15:29:35.007 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-16T15:29:35.008 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-16T15:29:35.008 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-16T15:29:35.008 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-16T15:29:35.008 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-16T15:29:35.008 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus 2024-09-16T15:29:35.008 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-16T15:29:35.008 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-16T15:29:35.008 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-16T15:29:35.008 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-16T15:29:35.008 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-16T15:29:35.008 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-16T15:29:35.008 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-16T15:29:35.008 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-16T15:29:35.009 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-16T15:29:35.009 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-16T15:29:35.009 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-16T15:29:35.009 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-16T15:29:35.009 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-16T15:29:35.009 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-16T15:29:35.009 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-16T15:29:35.009 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-16T15:29:35.009 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-16T15:29:35.009 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-16T15:29:35.009 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-16T15:29:35.009 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-16T15:29:35.009 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-16T15:29:35.010 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-16T15:29:35.010 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-16T15:29:35.010 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-16T15:29:35.010 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-16T15:29:35.010 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-16T15:29:35.010 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-16T15:29:35.010 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-16T15:29:35.010 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-16T15:29:35.010 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-16T15:29:35.010 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-16T15:29:35.010 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-16T15:29:35.010 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-16T15:29:35.010 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-16T15:29:35.010 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-16T15:29:35.011 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-16T15:29:35.011 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-16T15:29:35.011 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-16T15:29:35.011 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-16T15:29:35.011 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-16T15:29:35.011 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-16T15:29:35.011 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c 2024-09-16T15:29:35.011 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos 2024-09-16T15:29:35.011 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-16T15:29:35.011 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-16T15:29:35.011 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses 2024-09-16T15:29:35.011 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-16T15:29:35.011 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-16T15:29:35.012 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-16T15:29:35.012 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-16T15:29:35.012 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-16T15:29:35.012 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-16T15:29:35.012 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-16T15:29:35.012 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-16T15:29:35.012 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-16T15:29:35.012 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-16T15:29:35.012 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-16T15:29:35.012 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-16T15:29:35.012 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-16T15:29:35.012 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-16T15:29:35.012 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-16T15:29:35.012 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-16T15:29:35.012 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-16T15:29:35.013 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-16T15:29:35.013 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-16T15:29:35.013 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-16T15:29:35.013 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-16T15:29:35.013 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-16T15:29:35.013 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-16T15:29:35.013 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-dev.ko 2024-09-16T15:29:35.013 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-16T15:29:35.013 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-09-16T15:29:35.013 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-16T15:29:35.013 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-16T15:29:35.013 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband 2024-09-16T15:29:35.013 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core 2024-09-16T15:29:35.013 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-16T15:29:35.013 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-16T15:29:35.014 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-16T15:29:35.014 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-16T15:29:35.014 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-16T15:29:35.014 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-16T15:29:35.014 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-16T15:29:35.014 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw 2024-09-16T15:29:35.014 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4 2024-09-16T15:29:35.014 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-16T15:29:35.014 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4 2024-09-16T15:29:35.014 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-16T15:29:35.014 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5 2024-09-16T15:29:35.014 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-16T15:29:35.014 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca 2024-09-16T15:29:35.014 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-16T15:29:35.015 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp 2024-09-16T15:29:35.015 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib 2024-09-16T15:29:35.015 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-16T15:29:35.015 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser 2024-09-16T15:29:35.015 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-16T15:29:35.015 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert 2024-09-16T15:29:35.015 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-16T15:29:35.015 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp 2024-09-16T15:29:35.015 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-16T15:29:35.015 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt 2024-09-16T15:29:35.015 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-16T15:29:35.015 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input 2024-09-16T15:29:35.015 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport 2024-09-16T15:29:35.015 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-16T15:29:35.015 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-16T15:29:35.016 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-16T15:29:35.016 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-16T15:29:35.016 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard 2024-09-16T15:29:35.016 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-16T15:29:35.016 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse 2024-09-16T15:29:35.016 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-16T15:29:35.016 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-16T15:29:35.016 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-16T15:29:35.016 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-16T15:29:35.016 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-16T15:29:35.016 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-16T15:29:35.016 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-16T15:29:35.016 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-16T15:29:35.016 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4 2024-09-16T15:29:35.016 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-16T15:29:35.017 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio 2024-09-16T15:29:35.017 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-16T15:29:35.017 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-16T15:29:35.017 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-16T15:29:35.017 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-16T15:29:35.017 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-16T15:29:35.017 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds 2024-09-16T15:29:35.017 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-16T15:29:35.017 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-16T15:29:35.017 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-16T15:29:35.017 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-16T15:29:35.017 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger 2024-09-16T15:29:35.017 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-16T15:29:35.018 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-16T15:29:35.018 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-16T15:29:35.018 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-16T15:29:35.018 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-16T15:29:35.018 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-16T15:29:35.018 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-16T15:29:35.018 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-16T15:29:35.018 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md 2024-09-16T15:29:35.018 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache 2024-09-16T15:29:35.018 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-16T15:29:35.018 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-16T15:29:35.018 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-16T15:29:35.018 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-16T15:29:35.018 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-16T15:29:35.018 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-16T15:29:35.019 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-16T15:29:35.019 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-16T15:29:35.019 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-16T15:29:35.019 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-16T15:29:35.019 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-16T15:29:35.019 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-16T15:29:35.019 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-16T15:29:35.019 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-16T15:29:35.019 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-16T15:29:35.019 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-16T15:29:35.019 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data 2024-09-16T15:29:35.019 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-16T15:29:35.019 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-16T15:29:35.020 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-16T15:29:35.020 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-16T15:29:35.020 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-16T15:29:35.020 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message 2024-09-16T15:29:35.020 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion 2024-09-16T15:29:35.020 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-16T15:29:35.020 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-16T15:29:35.020 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-16T15:29:35.020 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-16T15:29:35.020 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-16T15:29:35.020 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-16T15:29:35.020 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-16T15:29:35.020 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd 2024-09-16T15:29:35.021 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-16T15:29:35.021 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-16T15:29:35.021 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mfd-core.ko 2024-09-16T15:29:35.021 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-16T15:29:35.021 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-16T15:29:35.021 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-16T15:29:35.021 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-16T15:29:35.021 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc 2024-09-16T15:29:35.021 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl 2024-09-16T15:29:35.021 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-16T15:29:35.021 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-16T15:29:35.021 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-16T15:29:35.021 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-16T15:29:35.022 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710 2024-09-16T15:29:35.022 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-16T15:29:35.022 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom 2024-09-16T15:29:35.022 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-16T15:29:35.022 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-16T15:29:35.022 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-16T15:29:35.022 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-16T15:29:35.022 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-16T15:29:35.022 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-16T15:29:35.022 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-16T15:29:35.022 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d 2024-09-16T15:29:35.022 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-16T15:29:35.022 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-16T15:29:35.023 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei 2024-09-16T15:29:35.023 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-16T15:29:35.023 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-16T15:29:35.023 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-16T15:29:35.023 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru 2024-09-16T15:29:35.023 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru/gru.ko 2024-09-16T15:29:35.023 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp 2024-09-16T15:29:35.023 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xp.ko 2024-09-16T15:29:35.023 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpc.ko 2024-09-16T15:29:35.023 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-09-16T15:29:35.023 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-16T15:29:35.023 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-16T15:29:35.023 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-16T15:29:35.023 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-16T15:29:35.024 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci 2024-09-16T15:29:35.024 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-16T15:29:35.024 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net 2024-09-16T15:29:35.024 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding 2024-09-16T15:29:35.024 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-16T15:29:35.024 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa 2024-09-16T15:29:35.024 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53 2024-09-16T15:29:35.024 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-16T15:29:35.024 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-16T15:29:35.024 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-16T15:29:35.024 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx 2024-09-16T15:29:35.024 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-16T15:29:35.024 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-16T15:29:35.025 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-16T15:29:35.025 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet 2024-09-16T15:29:35.025 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com 2024-09-16T15:29:35.025 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-16T15:29:35.025 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-16T15:29:35.025 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-16T15:29:35.025 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-16T15:29:35.025 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390 2024-09-16T15:29:35.025 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-16T15:29:35.025 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-16T15:29:35.025 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-16T15:29:35.025 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-16T15:29:35.025 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec 2024-09-16T15:29:35.025 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-16T15:29:35.026 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere 2024-09-16T15:29:35.026 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-16T15:29:35.026 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon 2024-09-16T15:29:35.026 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-16T15:29:35.026 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera 2024-09-16T15:29:35.026 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-16T15:29:35.026 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd 2024-09-16T15:29:35.026 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-16T15:29:35.026 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-16T15:29:35.026 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-16T15:29:35.026 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros 2024-09-16T15:29:35.026 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx 2024-09-16T15:29:35.026 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-16T15:29:35.026 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c 2024-09-16T15:29:35.027 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-16T15:29:35.027 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e 2024-09-16T15:29:35.027 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-16T15:29:35.027 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx 2024-09-16T15:29:35.027 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-16T15:29:35.027 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-16T15:29:35.027 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom 2024-09-16T15:29:35.027 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-16T15:29:35.027 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-16T15:29:35.027 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-09-16T15:29:35.027 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-16T15:29:35.027 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-16T15:29:35.027 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet 2024-09-16T15:29:35.027 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-16T15:29:35.028 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-16T15:29:35.028 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade 2024-09-16T15:29:35.028 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna 2024-09-16T15:29:35.028 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-16T15:29:35.028 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio 2024-09-16T15:29:35.028 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb 2024-09-16T15:29:35.028 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-16T15:29:35.028 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-09-16T15:29:35.028 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-16T15:29:35.028 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-09-16T15:29:35.028 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-16T15:29:35.028 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-09-16T15:29:35.028 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-16T15:29:35.028 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-09-16T15:29:35.029 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-16T15:29:35.029 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco 2024-09-16T15:29:35.029 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic 2024-09-16T15:29:35.029 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-16T15:29:35.029 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec 2024-09-16T15:29:35.029 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip 2024-09-16T15:29:35.029 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-16T15:29:35.029 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-16T15:29:35.029 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-16T15:29:35.029 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-16T15:29:35.029 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-16T15:29:35.029 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-16T15:29:35.029 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink 2024-09-16T15:29:35.029 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-16T15:29:35.030 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-16T15:29:35.030 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-16T15:29:35.030 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex 2024-09-16T15:29:35.030 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet 2024-09-16T15:29:35.030 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-16T15:29:35.030 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-16T15:29:35.030 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-16T15:29:35.030 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel 2024-09-16T15:29:35.030 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-16T15:29:35.030 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000 2024-09-16T15:29:35.030 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-16T15:29:35.030 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e 2024-09-16T15:29:35.030 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-16T15:29:35.030 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k 2024-09-16T15:29:35.031 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-16T15:29:35.031 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e 2024-09-16T15:29:35.031 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-16T15:29:35.031 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf 2024-09-16T15:29:35.031 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-16T15:29:35.031 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb 2024-09-16T15:29:35.031 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-16T15:29:35.031 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf 2024-09-16T15:29:35.031 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-16T15:29:35.031 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe 2024-09-16T15:29:35.031 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-16T15:29:35.031 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf 2024-09-16T15:29:35.031 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-16T15:29:35.031 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth 2024-09-16T15:29:35.031 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-16T15:29:35.032 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie 2024-09-16T15:29:35.032 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-16T15:29:35.032 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-16T15:29:35.032 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell 2024-09-16T15:29:35.032 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-16T15:29:35.032 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-16T15:29:35.032 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-16T15:29:35.032 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox 2024-09-16T15:29:35.032 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4 2024-09-16T15:29:35.032 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-16T15:29:35.032 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-16T15:29:35.032 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5 2024-09-16T15:29:35.032 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-09-16T15:29:35.033 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-16T15:29:35.033 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel 2024-09-16T15:29:35.033 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-16T15:29:35.033 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom 2024-09-16T15:29:35.033 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge 2024-09-16T15:29:35.033 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-16T15:29:35.033 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi 2024-09-16T15:29:35.033 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-16T15:29:35.033 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-16T15:29:35.033 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion 2024-09-16T15:29:35.033 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-16T15:29:35.033 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia 2024-09-16T15:29:35.033 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-16T15:29:35.033 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines 2024-09-16T15:29:35.033 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-16T15:29:35.034 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-16T15:29:35.034 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic 2024-09-16T15:29:35.034 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen 2024-09-16T15:29:35.034 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-16T15:29:35.034 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-16T15:29:35.034 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-09-16T15:29:35.034 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-16T15:29:35.034 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc 2024-09-16T15:29:35.034 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-16T15:29:35.034 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek 2024-09-16T15:29:35.034 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-16T15:29:35.034 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-16T15:29:35.034 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-16T15:29:35.034 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-16T15:29:35.034 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker 2024-09-16T15:29:35.035 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-09-16T15:29:35.035 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc 2024-09-16T15:29:35.035 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-16T15:29:35.035 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan 2024-09-16T15:29:35.035 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-16T15:29:35.035 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis 2024-09-16T15:29:35.035 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-16T15:29:35.035 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-16T15:29:35.035 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc 2024-09-16T15:29:35.035 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-16T15:29:35.035 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-16T15:29:35.035 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-16T15:29:35.035 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-16T15:29:35.036 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro 2024-09-16T15:29:35.036 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac 2024-09-16T15:29:35.036 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-16T15:29:35.036 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-16T15:29:35.036 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun 2024-09-16T15:29:35.036 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-16T15:29:35.036 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-16T15:29:35.036 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-16T15:29:35.036 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-16T15:29:35.036 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti 2024-09-16T15:29:35.036 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-16T15:29:35.036 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti 2024-09-16T15:29:35.036 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-16T15:29:35.036 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via 2024-09-16T15:29:35.037 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-16T15:29:35.037 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-16T15:29:35.037 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet 2024-09-16T15:29:35.037 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-16T15:29:35.037 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-16T15:29:35.037 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom 2024-09-16T15:29:35.037 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-16T15:29:35.037 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-16T15:29:35.037 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv 2024-09-16T15:29:35.037 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-16T15:29:35.037 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154 2024-09-16T15:29:35.037 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-16T15:29:35.037 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-16T15:29:35.037 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan 2024-09-16T15:29:35.038 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-16T15:29:35.038 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-16T15:29:35.038 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-16T15:29:35.038 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio 2024-09-16T15:29:35.038 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-16T15:29:35.038 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-16T15:29:35.038 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-16T15:29:35.038 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-16T15:29:35.038 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-16T15:29:35.038 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/net_failover.ko 2024-09-16T15:29:35.038 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-16T15:29:35.038 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-16T15:29:35.038 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs 2024-09-16T15:29:35.039 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-16T15:29:35.039 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-16T15:29:35.039 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy 2024-09-16T15:29:35.039 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-16T15:29:35.039 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-16T15:29:35.039 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-09-16T15:29:35.039 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-16T15:29:35.039 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-16T15:29:35.039 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-16T15:29:35.039 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-16T15:29:35.039 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-16T15:29:35.039 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/dp83640.ko 2024-09-16T15:29:35.039 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-16T15:29:35.039 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-16T15:29:35.040 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-16T15:29:35.040 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-16T15:29:35.040 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-16T15:29:35.040 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-16T15:29:35.040 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-16T15:29:35.040 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-16T15:29:35.040 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-16T15:29:35.040 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-16T15:29:35.040 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-16T15:29:35.040 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-16T15:29:35.040 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp 2024-09-16T15:29:35.040 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-16T15:29:35.040 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-16T15:29:35.040 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-16T15:29:35.040 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_generic.ko 2024-09-16T15:29:35.041 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-16T15:29:35.041 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-16T15:29:35.041 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-16T15:29:35.041 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-16T15:29:35.041 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-16T15:29:35.041 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip 2024-09-16T15:29:35.041 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slhc.ko 2024-09-16T15:29:35.041 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-16T15:29:35.041 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-16T15:29:35.041 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-16T15:29:35.041 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team 2024-09-16T15:29:35.041 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-16T15:29:35.041 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-16T15:29:35.041 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-16T15:29:35.042 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-16T15:29:35.042 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-16T15:29:35.042 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-16T15:29:35.042 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tun.ko 2024-09-16T15:29:35.042 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-16T15:29:35.042 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/virtio_net.ko 2024-09-16T15:29:35.042 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3 2024-09-16T15:29:35.042 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-16T15:29:35.042 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan 2024-09-16T15:29:35.042 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-16T15:29:35.042 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback 2024-09-16T15:29:35.042 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-16T15:29:35.042 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netfront.ko 2024-09-16T15:29:35.042 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb 2024-09-16T15:29:35.042 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-16T15:29:35.043 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme 2024-09-16T15:29:35.043 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host 2024-09-16T15:29:35.043 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-16T15:29:35.043 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-16T15:29:35.043 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport 2024-09-16T15:29:35.043 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-16T15:29:35.043 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-16T15:29:35.043 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-16T15:29:35.043 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-16T15:29:35.043 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci 2024-09-16T15:29:35.043 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug 2024-09-16T15:29:35.043 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-16T15:29:35.043 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie 2024-09-16T15:29:35.043 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie/aer_inject.ko 2024-09-16T15:29:35.044 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-16T15:29:35.044 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia 2024-09-16T15:29:35.044 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-16T15:29:35.044 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-16T15:29:35.044 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-16T15:29:35.044 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl 2024-09-16T15:29:35.044 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel 2024-09-16T15:29:35.044 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-16T15:29:35.044 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform 2024-09-16T15:29:35.044 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86 2024-09-16T15:29:35.044 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-16T15:29:35.044 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-16T15:29:35.044 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-16T15:29:35.044 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-16T15:29:35.044 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-16T15:29:35.044 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-16T15:29:35.045 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-16T15:29:35.045 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-16T15:29:35.045 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-16T15:29:35.045 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-16T15:29:35.045 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-16T15:29:35.045 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-16T15:29:35.045 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel 2024-09-16T15:29:35.045 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-16T15:29:35.045 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-16T15:29:35.045 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-16T15:29:35.045 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-16T15:29:35.045 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-16T15:29:35.045 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-16T15:29:35.046 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-16T15:29:35.046 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-16T15:29:35.046 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-16T15:29:35.046 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-16T15:29:35.046 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-16T15:29:35.046 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-16T15:29:35.046 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap 2024-09-16T15:29:35.046 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-16T15:29:35.046 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-16T15:29:35.046 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps 2024-09-16T15:29:35.046 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients 2024-09-16T15:29:35.046 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-16T15:29:35.046 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-16T15:29:35.046 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-16T15:29:35.046 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-16T15:29:35.046 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp 2024-09-16T15:29:35.047 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-16T15:29:35.047 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-16T15:29:35.047 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras 2024-09-16T15:29:35.047 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd 2024-09-16T15:29:35.047 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl 2024-09-16T15:29:35.047 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-16T15:29:35.047 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-16T15:29:35.047 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc 2024-09-16T15:29:35.047 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-16T15:29:35.047 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-16T15:29:35.047 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-16T15:29:35.047 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-16T15:29:35.047 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-16T15:29:35.047 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-16T15:29:35.047 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-16T15:29:35.048 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-16T15:29:35.048 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-16T15:29:35.048 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-16T15:29:35.048 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-16T15:29:35.048 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-16T15:29:35.048 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-16T15:29:35.048 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-16T15:29:35.048 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-16T15:29:35.048 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-16T15:29:35.048 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-16T15:29:35.048 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-16T15:29:35.048 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-16T15:29:35.048 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-16T15:29:35.048 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-16T15:29:35.048 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-16T15:29:35.048 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-16T15:29:35.049 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-16T15:29:35.049 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-16T15:29:35.049 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-16T15:29:35.049 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-16T15:29:35.049 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-16T15:29:35.049 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-16T15:29:35.049 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-16T15:29:35.049 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-16T15:29:35.049 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi 2024-09-16T15:29:35.049 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-16T15:29:35.049 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-16T15:29:35.049 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-16T15:29:35.049 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-16T15:29:35.049 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-16T15:29:35.049 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid 2024-09-16T15:29:35.050 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-16T15:29:35.050 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-16T15:29:35.050 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx 2024-09-16T15:29:35.050 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-16T15:29:35.050 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-16T15:29:35.050 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-16T15:29:35.050 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr 2024-09-16T15:29:35.050 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-16T15:29:35.050 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-16T15:29:35.050 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi 2024-09-16T15:29:35.050 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-16T15:29:35.050 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa 2024-09-16T15:29:35.050 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-16T15:29:35.050 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc 2024-09-16T15:29:35.050 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-16T15:29:35.050 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i 2024-09-16T15:29:35.051 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-16T15:29:35.051 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-16T15:29:35.051 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor 2024-09-16T15:29:35.051 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-16T15:29:35.051 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi 2024-09-16T15:29:35.051 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i 2024-09-16T15:29:35.051 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-16T15:29:35.051 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i 2024-09-16T15:29:35.051 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-16T15:29:35.051 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-16T15:29:35.051 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-16T15:29:35.051 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler 2024-09-16T15:29:35.051 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-09-16T15:29:35.051 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-09-16T15:29:35.052 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-09-16T15:29:35.052 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-09-16T15:29:35.052 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-16T15:29:35.052 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r 2024-09-16T15:29:35.052 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-16T15:29:35.052 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-16T15:29:35.052 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe 2024-09-16T15:29:35.052 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-16T15:29:35.052 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-16T15:29:35.052 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic 2024-09-16T15:29:35.052 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-16T15:29:35.052 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-16T15:29:35.052 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-16T15:29:35.052 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-16T15:29:35.053 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-16T15:29:35.053 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-16T15:29:35.053 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-16T15:29:35.053 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci 2024-09-16T15:29:35.053 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-16T15:29:35.053 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-16T15:29:35.053 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-16T15:29:35.053 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc 2024-09-16T15:29:35.053 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-16T15:29:35.053 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-16T15:29:35.053 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-16T15:29:35.053 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas 2024-09-16T15:29:35.053 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-16T15:29:35.053 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc 2024-09-16T15:29:35.053 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-16T15:29:35.054 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid 2024-09-16T15:29:35.054 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-16T15:29:35.054 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-16T15:29:35.054 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-16T15:29:35.054 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-16T15:29:35.054 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas 2024-09-16T15:29:35.054 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-16T15:29:35.054 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas 2024-09-16T15:29:35.054 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-16T15:29:35.054 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-16T15:29:35.054 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001 2024-09-16T15:29:35.054 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-16T15:29:35.054 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-16T15:29:35.054 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-16T15:29:35.054 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx 2024-09-16T15:29:35.054 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-16T15:29:35.055 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-16T15:29:35.055 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx 2024-09-16T15:29:35.055 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-16T15:29:35.055 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-16T15:29:35.055 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-16T15:29:35.055 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-16T15:29:35.055 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-16T15:29:35.055 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-16T15:29:35.055 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-16T15:29:35.055 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-16T15:29:35.055 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-16T15:29:35.055 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-16T15:29:35.055 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-16T15:29:35.055 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2 2024-09-16T15:29:35.055 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-16T15:29:35.055 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-16T15:29:35.056 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-16T15:29:35.056 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-16T15:29:35.056 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb 2024-09-16T15:29:35.056 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-16T15:29:35.056 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target 2024-09-16T15:29:35.056 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi 2024-09-16T15:29:35.056 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-16T15:29:35.056 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback 2024-09-16T15:29:35.056 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-16T15:29:35.056 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-16T15:29:35.056 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-16T15:29:35.056 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-16T15:29:35.056 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-16T15:29:35.056 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-16T15:29:35.056 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc 2024-09-16T15:29:35.057 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-16T15:29:35.057 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal 2024-09-16T15:29:35.057 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel 2024-09-16T15:29:35.057 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal 2024-09-16T15:29:35.057 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-16T15:29:35.057 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-16T15:29:35.057 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-16T15:29:35.057 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-16T15:29:35.057 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-16T15:29:35.057 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-16T15:29:35.057 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-16T15:29:35.057 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-16T15:29:35.057 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-16T15:29:35.057 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-16T15:29:35.058 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-16T15:29:35.058 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-16T15:29:35.058 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-16T15:29:35.058 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-16T15:29:35.058 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-16T15:29:35.058 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-16T15:29:35.058 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-16T15:29:35.058 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-16T15:29:35.058 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty 2024-09-16T15:29:35.058 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless 2024-09-16T15:29:35.058 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-16T15:29:35.058 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-16T15:29:35.058 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-16T15:29:35.058 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-16T15:29:35.059 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial 2024-09-16T15:29:35.059 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250 2024-09-16T15:29:35.059 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-16T15:29:35.059 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-16T15:29:35.059 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm 2024-09-16T15:29:35.059 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-16T15:29:35.059 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs 2024-09-16T15:29:35.059 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core 2024-09-16T15:29:35.059 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-16T15:29:35.059 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host 2024-09-16T15:29:35.059 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-16T15:29:35.059 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio 2024-09-16T15:29:35.059 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-16T15:29:35.059 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-16T15:29:35.060 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-16T15:29:35.060 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-16T15:29:35.060 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-16T15:29:35.060 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb 2024-09-16T15:29:35.060 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class 2024-09-16T15:29:35.060 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-16T15:29:35.060 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-16T15:29:35.060 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-16T15:29:35.060 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-16T15:29:35.060 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host 2024-09-16T15:29:35.060 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-16T15:29:35.060 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image 2024-09-16T15:29:35.060 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-16T15:29:35.060 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-16T15:29:35.061 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc 2024-09-16T15:29:35.061 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-16T15:29:35.061 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-16T15:29:35.061 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-16T15:29:35.061 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-16T15:29:35.061 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-16T15:29:35.061 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-16T15:29:35.061 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-16T15:29:35.061 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-16T15:29:35.061 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-16T15:29:35.061 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-16T15:29:35.061 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga 2024-09-16T15:29:35.061 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-16T15:29:35.061 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-16T15:29:35.061 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-16T15:29:35.062 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-16T15:29:35.062 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-16T15:29:35.062 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-16T15:29:35.062 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-16T15:29:35.062 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy 2024-09-16T15:29:35.062 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-16T15:29:35.062 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial 2024-09-16T15:29:35.062 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-16T15:29:35.062 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-16T15:29:35.062 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-16T15:29:35.062 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-16T15:29:35.062 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-16T15:29:35.062 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-16T15:29:35.062 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-16T15:29:35.063 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-16T15:29:35.063 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-16T15:29:35.063 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-16T15:29:35.063 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-16T15:29:35.063 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-16T15:29:35.063 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-16T15:29:35.063 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-16T15:29:35.063 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-16T15:29:35.063 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-16T15:29:35.063 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-16T15:29:35.063 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-16T15:29:35.063 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-16T15:29:35.063 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-16T15:29:35.063 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-16T15:29:35.064 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-16T15:29:35.064 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-16T15:29:35.064 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-16T15:29:35.064 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-16T15:29:35.064 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-16T15:29:35.064 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-16T15:29:35.064 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-16T15:29:35.064 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-16T15:29:35.064 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-16T15:29:35.064 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-16T15:29:35.064 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-16T15:29:35.064 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-16T15:29:35.064 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-16T15:29:35.064 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-16T15:29:35.065 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-16T15:29:35.065 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-16T15:29:35.065 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-16T15:29:35.065 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-16T15:29:35.065 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-16T15:29:35.065 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-16T15:29:35.065 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-16T15:29:35.065 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-16T15:29:35.065 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-16T15:29:35.065 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-16T15:29:35.065 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage 2024-09-16T15:29:35.065 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-16T15:29:35.065 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-16T15:29:35.065 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-16T15:29:35.066 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-16T15:29:35.066 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-16T15:29:35.066 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-16T15:29:35.066 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-16T15:29:35.066 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-16T15:29:35.066 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-16T15:29:35.066 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-16T15:29:35.066 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-16T15:29:35.066 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-16T15:29:35.066 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-16T15:29:35.066 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-16T15:29:35.066 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-16T15:29:35.066 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip 2024-09-16T15:29:35.066 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-16T15:29:35.067 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-16T15:29:35.067 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-16T15:29:35.067 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio 2024-09-16T15:29:35.067 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci 2024-09-16T15:29:35.067 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-16T15:29:35.067 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-16T15:29:35.067 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-16T15:29:35.067 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-16T15:29:35.067 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost 2024-09-16T15:29:35.067 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-16T15:29:35.067 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-16T15:29:35.067 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-16T15:29:35.067 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-16T15:29:35.067 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video 2024-09-16T15:29:35.068 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight 2024-09-16T15:29:35.068 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-16T15:29:35.068 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-16T15:29:35.068 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-16T15:29:35.068 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio 2024-09-16T15:29:35.068 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio.ko 2024-09-16T15:29:35.068 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_balloon.ko 2024-09-16T15:29:35.068 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_mmio.ko 2024-09-16T15:29:35.068 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci.ko 2024-09-16T15:29:35.068 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-09-16T15:29:35.068 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-09-16T15:29:35.068 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_ring.ko 2024-09-16T15:29:35.068 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1 2024-09-16T15:29:35.068 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters 2024-09-16T15:29:35.069 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-16T15:29:35.069 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-16T15:29:35.069 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves 2024-09-16T15:29:35.069 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-16T15:29:35.069 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-16T15:29:35.069 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-16T15:29:35.069 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-16T15:29:35.069 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-16T15:29:35.069 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-16T15:29:35.069 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-16T15:29:35.069 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-16T15:29:35.069 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-16T15:29:35.069 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-16T15:29:35.069 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-16T15:29:35.070 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-16T15:29:35.070 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog 2024-09-16T15:29:35.070 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-16T15:29:35.070 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-16T15:29:35.070 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-16T15:29:35.070 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-16T15:29:35.070 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-16T15:29:35.070 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-16T15:29:35.070 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-16T15:29:35.070 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-16T15:29:35.070 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-16T15:29:35.070 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-16T15:29:35.070 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-16T15:29:35.070 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-16T15:29:35.071 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-16T15:29:35.071 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-16T15:29:35.071 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-16T15:29:35.071 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-16T15:29:35.071 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-16T15:29:35.071 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-16T15:29:35.071 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-16T15:29:35.071 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-16T15:29:35.071 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-16T15:29:35.071 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-16T15:29:35.071 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-16T15:29:35.071 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-16T15:29:35.071 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-16T15:29:35.071 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-16T15:29:35.071 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen 2024-09-16T15:29:35.072 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-acpi-processor.ko 2024-09-16T15:29:35.072 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-16T15:29:35.072 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-16T15:29:35.072 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-16T15:29:35.072 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback 2024-09-16T15:29:35.072 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-16T15:29:35.072 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-16T15:29:35.072 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs 2024-09-16T15:29:35.072 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-16T15:29:35.072 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs 2024-09-16T15:29:35.072 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p 2024-09-16T15:29:35.072 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-16T15:29:35.072 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs 2024-09-16T15:29:35.072 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-16T15:29:35.073 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs 2024-09-16T15:29:35.073 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-16T15:29:35.073 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-16T15:29:35.073 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs 2024-09-16T15:29:35.073 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-16T15:29:35.073 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles 2024-09-16T15:29:35.073 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-16T15:29:35.073 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph 2024-09-16T15:29:35.073 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-16T15:29:35.073 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda 2024-09-16T15:29:35.073 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-16T15:29:35.073 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs 2024-09-16T15:29:35.073 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-16T15:29:35.074 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm 2024-09-16T15:29:35.074 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-16T15:29:35.074 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs 2024-09-16T15:29:35.074 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs/ecryptfs.ko 2024-09-16T15:29:35.074 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs 2024-09-16T15:29:35.074 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-16T15:29:35.074 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat 2024-09-16T15:29:35.074 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/fat.ko 2024-09-16T15:29:35.074 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-16T15:29:35.074 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/vfat.ko 2024-09-16T15:29:35.074 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse 2024-09-16T15:29:35.074 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-16T15:29:35.074 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/fuse.ko 2024-09-16T15:29:35.074 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2 2024-09-16T15:29:35.075 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-16T15:29:35.075 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs 2024-09-16T15:29:35.075 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-16T15:29:35.075 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus 2024-09-16T15:29:35.075 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-16T15:29:35.075 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs 2024-09-16T15:29:35.075 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-16T15:29:35.075 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs 2024-09-16T15:29:35.075 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-16T15:29:35.075 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd 2024-09-16T15:29:35.075 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-16T15:29:35.075 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix 2024-09-16T15:29:35.075 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-16T15:29:35.075 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs 2024-09-16T15:29:35.075 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-16T15:29:35.075 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs 2024-09-16T15:29:35.076 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout 2024-09-16T15:29:35.076 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-16T15:29:35.076 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout 2024-09-16T15:29:35.076 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-16T15:29:35.076 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout 2024-09-16T15:29:35.076 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-16T15:29:35.076 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-16T15:29:35.076 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-16T15:29:35.076 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-16T15:29:35.076 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common 2024-09-16T15:29:35.076 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-16T15:29:35.076 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-16T15:29:35.076 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd 2024-09-16T15:29:35.076 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-16T15:29:35.077 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2 2024-09-16T15:29:35.077 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-16T15:29:35.077 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls 2024-09-16T15:29:35.077 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-16T15:29:35.077 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-16T15:29:35.077 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-16T15:29:35.077 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-16T15:29:35.077 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-16T15:29:35.077 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-16T15:29:35.077 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-16T15:29:35.077 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-16T15:29:35.077 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-16T15:29:35.077 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-16T15:29:35.077 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-16T15:29:35.078 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-16T15:29:35.078 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-16T15:29:35.078 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-16T15:29:35.078 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-16T15:29:35.078 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-16T15:29:35.078 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-16T15:29:35.078 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-16T15:29:35.078 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-16T15:29:35.078 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-16T15:29:35.078 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-16T15:29:35.078 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-16T15:29:35.078 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-16T15:29:35.078 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-16T15:29:35.078 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-16T15:29:35.078 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-16T15:29:35.079 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-16T15:29:35.079 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-16T15:29:35.079 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-16T15:29:35.079 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-16T15:29:35.079 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-16T15:29:35.079 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-16T15:29:35.079 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-16T15:29:35.079 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-16T15:29:35.079 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-16T15:29:35.079 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-16T15:29:35.079 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-16T15:29:35.079 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-16T15:29:35.079 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-16T15:29:35.079 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-16T15:29:35.080 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-16T15:29:35.080 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-16T15:29:35.080 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-16T15:29:35.080 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-16T15:29:35.080 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-16T15:29:35.080 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-16T15:29:35.080 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-16T15:29:35.080 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-16T15:29:35.080 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-16T15:29:35.080 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-16T15:29:35.080 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2 2024-09-16T15:29:35.080 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster 2024-09-16T15:29:35.080 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-16T15:29:35.080 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm 2024-09-16T15:29:35.081 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-16T15:29:35.081 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs 2024-09-16T15:29:35.081 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-16T15:29:35.081 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-16T15:29:35.081 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-16T15:29:35.081 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-16T15:29:35.081 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-16T15:29:35.081 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs 2024-09-16T15:29:35.081 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-16T15:29:35.081 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore 2024-09-16T15:29:35.081 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-16T15:29:35.081 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs 2024-09-16T15:29:35.081 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-16T15:29:35.082 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs 2024-09-16T15:29:35.082 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-16T15:29:35.082 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb 2024-09-16T15:29:35.082 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client 2024-09-16T15:29:35.082 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-16T15:29:35.082 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common 2024-09-16T15:29:35.082 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-16T15:29:35.082 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-16T15:29:35.082 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs 2024-09-16T15:29:35.082 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-16T15:29:35.082 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv 2024-09-16T15:29:35.082 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-16T15:29:35.082 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf 2024-09-16T15:29:35.082 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-16T15:29:35.083 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs 2024-09-16T15:29:35.083 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-16T15:29:35.083 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs 2024-09-16T15:29:35.083 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-16T15:29:35.083 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel 2024-09-16T15:29:35.083 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu 2024-09-16T15:29:35.083 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu/rcutorture.ko 2024-09-16T15:29:35.083 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-16T15:29:35.083 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace 2024-09-16T15:29:35.083 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-09-16T15:29:35.083 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib 2024-09-16T15:29:35.083 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/asn1_encoder.ko 2024-09-16T15:29:35.083 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-16T15:29:35.083 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-16T15:29:35.083 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto 2024-09-16T15:29:35.084 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libaescfb.ko 2024-09-16T15:29:35.084 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-16T15:29:35.084 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-16T15:29:35.084 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-16T15:29:35.084 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-16T15:29:35.084 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4 2024-09-16T15:29:35.084 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-16T15:29:35.084 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-16T15:29:35.084 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math 2024-09-16T15:29:35.084 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-16T15:29:35.084 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6 2024-09-16T15:29:35.084 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-16T15:29:35.084 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon 2024-09-16T15:29:35.084 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-16T15:29:35.085 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-16T15:29:35.085 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-16T15:29:35.085 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-16T15:29:35.085 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd 2024-09-16T15:29:35.085 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd/zstd_compress.ko 2024-09-16T15:29:35.085 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm 2024-09-16T15:29:35.085 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-16T15:29:35.085 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net 2024-09-16T15:29:35.085 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan 2024-09-16T15:29:35.085 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-16T15:29:35.085 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-16T15:29:35.085 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-16T15:29:35.085 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-16T15:29:35.085 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-16T15:29:35.086 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-16T15:29:35.086 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-16T15:29:35.086 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-16T15:29:35.086 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802 2024-09-16T15:29:35.086 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-16T15:29:35.086 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-16T15:29:35.086 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-16T15:29:35.086 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-16T15:29:35.086 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-16T15:29:35.086 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q 2024-09-16T15:29:35.086 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-16T15:29:35.086 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p 2024-09-16T15:29:35.086 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-16T15:29:35.086 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-16T15:29:35.086 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-16T15:29:35.087 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-16T15:29:35.087 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv 2024-09-16T15:29:35.087 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-16T15:29:35.087 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge 2024-09-16T15:29:35.087 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-16T15:29:35.087 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-16T15:29:35.087 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter 2024-09-16T15:29:35.087 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-16T15:29:35.087 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-16T15:29:35.087 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-16T15:29:35.087 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-16T15:29:35.087 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-16T15:29:35.087 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-16T15:29:35.088 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-16T15:29:35.088 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-16T15:29:35.088 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-16T15:29:35.088 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-16T15:29:35.088 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-16T15:29:35.088 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-16T15:29:35.088 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-16T15:29:35.088 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-16T15:29:35.088 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-16T15:29:35.088 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-16T15:29:35.088 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-16T15:29:35.088 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-16T15:29:35.088 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-16T15:29:35.089 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-16T15:29:35.089 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-16T15:29:35.089 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-09-16T15:29:35.089 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-16T15:29:35.089 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-16T15:29:35.089 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph 2024-09-16T15:29:35.089 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-16T15:29:35.089 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core 2024-09-16T15:29:35.089 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/failover.ko 2024-09-16T15:29:35.089 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-16T15:29:35.089 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp 2024-09-16T15:29:35.089 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-16T15:29:35.090 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-16T15:29:35.090 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-16T15:29:35.090 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-16T15:29:35.090 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver 2024-09-16T15:29:35.090 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver/dns_resolver.ko 2024-09-16T15:29:35.090 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa 2024-09-16T15:29:35.090 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-16T15:29:35.090 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-16T15:29:35.090 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-16T15:29:35.090 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-16T15:29:35.090 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-16T15:29:35.090 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154 2024-09-16T15:29:35.090 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan 2024-09-16T15:29:35.090 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-16T15:29:35.091 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-16T15:29:35.091 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-16T15:29:35.091 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4 2024-09-16T15:29:35.091 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-16T15:29:35.091 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-16T15:29:35.091 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-16T15:29:35.091 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-16T15:29:35.091 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-16T15:29:35.091 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-16T15:29:35.091 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-16T15:29:35.091 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-16T15:29:35.091 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-16T15:29:35.091 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-16T15:29:35.091 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter 2024-09-16T15:29:35.092 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-16T15:29:35.092 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-16T15:29:35.092 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-16T15:29:35.092 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-16T15:29:35.092 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-16T15:29:35.092 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-16T15:29:35.092 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-16T15:29:35.092 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-16T15:29:35.092 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-16T15:29:35.092 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-16T15:29:35.092 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-16T15:29:35.092 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-16T15:29:35.092 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-16T15:29:35.093 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-16T15:29:35.093 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-16T15:29:35.093 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-16T15:29:35.093 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-16T15:29:35.093 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-16T15:29:35.093 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-16T15:29:35.093 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-16T15:29:35.093 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-16T15:29:35.093 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-16T15:29:35.093 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-16T15:29:35.093 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-16T15:29:35.093 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-16T15:29:35.093 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-16T15:29:35.094 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-16T15:29:35.094 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-16T15:29:35.094 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-16T15:29:35.094 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-16T15:29:35.094 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-16T15:29:35.094 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-16T15:29:35.094 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-16T15:29:35.094 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-16T15:29:35.094 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-16T15:29:35.094 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-16T15:29:35.094 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-16T15:29:35.094 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-16T15:29:35.094 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6 2024-09-16T15:29:35.094 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-16T15:29:35.094 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-16T15:29:35.095 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-16T15:29:35.095 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-16T15:29:35.095 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-16T15:29:35.095 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-16T15:29:35.095 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-16T15:29:35.095 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter 2024-09-16T15:29:35.095 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-16T15:29:35.095 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-16T15:29:35.095 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-16T15:29:35.095 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-16T15:29:35.095 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-16T15:29:35.095 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-16T15:29:35.095 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-16T15:29:35.095 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-16T15:29:35.096 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-16T15:29:35.096 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-16T15:29:35.096 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-16T15:29:35.096 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-16T15:29:35.096 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-16T15:29:35.096 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-16T15:29:35.096 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-16T15:29:35.096 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-16T15:29:35.096 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-16T15:29:35.096 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-16T15:29:35.096 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-16T15:29:35.096 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-16T15:29:35.096 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-16T15:29:35.096 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-16T15:29:35.097 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-16T15:29:35.097 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-16T15:29:35.097 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-16T15:29:35.097 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-16T15:29:35.097 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-16T15:29:35.097 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key 2024-09-16T15:29:35.097 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-16T15:29:35.097 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp 2024-09-16T15:29:35.097 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-16T15:29:35.097 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-16T15:29:35.097 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-16T15:29:35.097 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-16T15:29:35.097 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-16T15:29:35.097 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-16T15:29:35.098 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-16T15:29:35.098 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc 2024-09-16T15:29:35.098 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-16T15:29:35.098 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154 2024-09-16T15:29:35.098 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-16T15:29:35.098 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls 2024-09-16T15:29:35.098 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-16T15:29:35.098 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter 2024-09-16T15:29:35.098 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset 2024-09-16T15:29:35.098 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-16T15:29:35.098 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-16T15:29:35.098 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-16T15:29:35.098 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-16T15:29:35.098 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-16T15:29:35.099 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-16T15:29:35.099 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-16T15:29:35.099 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-16T15:29:35.099 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-16T15:29:35.099 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-16T15:29:35.099 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-16T15:29:35.099 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-16T15:29:35.099 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-16T15:29:35.099 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-16T15:29:35.099 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-16T15:29:35.099 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-16T15:29:35.099 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs 2024-09-16T15:29:35.099 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-16T15:29:35.100 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-16T15:29:35.100 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-16T15:29:35.100 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-16T15:29:35.100 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-16T15:29:35.100 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-16T15:29:35.100 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-16T15:29:35.100 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-16T15:29:35.100 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-16T15:29:35.100 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-16T15:29:35.100 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-16T15:29:35.100 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-16T15:29:35.100 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-16T15:29:35.100 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-16T15:29:35.100 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-16T15:29:35.100 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-16T15:29:35.101 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-16T15:29:35.101 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-16T15:29:35.101 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-16T15:29:35.101 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-16T15:29:35.101 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-16T15:29:35.101 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-16T15:29:35.101 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-16T15:29:35.101 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-16T15:29:35.101 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-16T15:29:35.101 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-16T15:29:35.101 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-16T15:29:35.101 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-16T15:29:35.101 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-16T15:29:35.101 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-16T15:29:35.101 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-16T15:29:35.102 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-16T15:29:35.102 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-16T15:29:35.102 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-16T15:29:35.102 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-16T15:29:35.102 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-16T15:29:35.102 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-16T15:29:35.102 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-16T15:29:35.102 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-16T15:29:35.102 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-16T15:29:35.102 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-16T15:29:35.102 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-16T15:29:35.102 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-16T15:29:35.102 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-16T15:29:35.102 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-16T15:29:35.102 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-16T15:29:35.103 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-16T15:29:35.103 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-16T15:29:35.103 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-16T15:29:35.103 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-16T15:29:35.103 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-16T15:29:35.103 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-16T15:29:35.103 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-16T15:29:35.103 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-16T15:29:35.103 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-16T15:29:35.103 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-16T15:29:35.103 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-16T15:29:35.103 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-16T15:29:35.103 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_quota.ko 2024-09-16T15:29:35.103 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-16T15:29:35.103 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-16T15:29:35.103 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-16T15:29:35.104 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-16T15:29:35.104 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tunnel.ko 2024-09-16T15:29:35.104 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-16T15:29:35.104 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-16T15:29:35.104 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-16T15:29:35.104 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-16T15:29:35.104 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-16T15:29:35.104 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-16T15:29:35.104 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-16T15:29:35.104 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-16T15:29:35.104 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-16T15:29:35.104 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-16T15:29:35.104 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-16T15:29:35.104 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-16T15:29:35.105 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-16T15:29:35.105 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-16T15:29:35.105 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-16T15:29:35.105 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-16T15:29:35.105 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-16T15:29:35.105 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-16T15:29:35.105 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-16T15:29:35.105 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-16T15:29:35.105 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-16T15:29:35.105 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-16T15:29:35.105 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-16T15:29:35.105 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-16T15:29:35.105 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-16T15:29:35.105 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-16T15:29:35.106 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-16T15:29:35.106 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-16T15:29:35.106 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-16T15:29:35.106 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-16T15:29:35.106 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-16T15:29:35.106 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-16T15:29:35.106 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-16T15:29:35.106 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-16T15:29:35.106 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-16T15:29:35.106 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-16T15:29:35.106 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-16T15:29:35.106 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-16T15:29:35.106 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-16T15:29:35.106 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-16T15:29:35.107 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-16T15:29:35.107 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-16T15:29:35.107 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-16T15:29:35.107 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-16T15:29:35.107 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-16T15:29:35.107 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-16T15:29:35.107 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-16T15:29:35.107 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-16T15:29:35.107 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-16T15:29:35.107 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-16T15:29:35.107 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-16T15:29:35.107 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-16T15:29:35.107 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-16T15:29:35.107 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-16T15:29:35.108 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-16T15:29:35.108 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-16T15:29:35.108 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-16T15:29:35.108 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-16T15:29:35.108 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-16T15:29:35.108 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-16T15:29:35.108 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-16T15:29:35.108 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-16T15:29:35.108 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-16T15:29:35.108 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-16T15:29:35.108 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-16T15:29:35.108 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-16T15:29:35.108 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-16T15:29:35.108 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-16T15:29:35.108 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-16T15:29:35.109 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-16T15:29:35.109 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-16T15:29:35.109 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink 2024-09-16T15:29:35.109 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-16T15:29:35.109 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh 2024-09-16T15:29:35.109 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-16T15:29:35.109 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch 2024-09-16T15:29:35.109 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-16T15:29:35.109 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-16T15:29:35.109 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-16T15:29:35.109 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-16T15:29:35.109 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet 2024-09-16T15:29:35.109 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-16T15:29:35.109 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds 2024-09-16T15:29:35.110 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-16T15:29:35.110 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-16T15:29:35.110 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-16T15:29:35.110 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched 2024-09-16T15:29:35.110 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-16T15:29:35.110 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-16T15:29:35.110 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-16T15:29:35.110 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-16T15:29:35.110 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-16T15:29:35.110 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-16T15:29:35.110 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-16T15:29:35.110 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-16T15:29:35.110 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-16T15:29:35.110 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-16T15:29:35.111 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-16T15:29:35.111 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-16T15:29:35.111 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-16T15:29:35.111 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-16T15:29:35.111 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-16T15:29:35.111 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-16T15:29:35.111 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-16T15:29:35.111 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-16T15:29:35.111 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-16T15:29:35.111 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-16T15:29:35.111 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-16T15:29:35.111 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-16T15:29:35.111 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-16T15:29:35.111 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-16T15:29:35.111 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-16T15:29:35.111 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-16T15:29:35.112 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-16T15:29:35.112 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-16T15:29:35.112 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-16T15:29:35.112 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-16T15:29:35.112 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-16T15:29:35.112 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-16T15:29:35.112 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-16T15:29:35.112 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-16T15:29:35.112 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-16T15:29:35.112 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-16T15:29:35.112 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-16T15:29:35.112 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-16T15:29:35.112 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-16T15:29:35.112 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-16T15:29:35.112 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-16T15:29:35.113 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-16T15:29:35.113 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-16T15:29:35.113 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp 2024-09-16T15:29:35.113 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-16T15:29:35.113 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-16T15:29:35.113 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc 2024-09-16T15:29:35.113 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss 2024-09-16T15:29:35.113 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-16T15:29:35.113 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-16T15:29:35.113 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-16T15:29:35.113 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma 2024-09-16T15:29:35.113 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-16T15:29:35.113 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc 2024-09-16T15:29:35.113 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-16T15:29:35.114 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-16T15:29:35.114 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix 2024-09-16T15:29:35.114 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-16T15:29:35.114 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock 2024-09-16T15:29:35.114 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-16T15:29:35.114 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-16T15:29:35.114 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-16T15:29:35.114 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-16T15:29:35.114 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-16T15:29:35.114 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm 2024-09-16T15:29:35.114 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-16T15:29:35.114 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security 2024-09-16T15:29:35.114 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys 2024-09-16T15:29:35.115 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys 2024-09-16T15:29:35.115 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-09-16T15:29:35.115 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys 2024-09-16T15:29:35.115 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys/trusted.ko 2024-09-16T15:29:35.115 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-16T15:29:35.115 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-16T15:29:35.115 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-16T15:29:35.115 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-16T15:29:35.115 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-16T15:29:35.115 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-16T15:29:35.115 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-16T15:29:35.115 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-16T15:29:35.115 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-16T15:29:35.115 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-16T15:29:35.115 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-16T15:29:35.115 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-16T15:29:35.116 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-16T15:29:35.116 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.weakdep 2024-09-16T15:29:35.116 INFO:teuthology.orchestra.run.smithi167.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-16T15:29:35.119 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-16T15:29:35.120 DEBUG:teuthology.orchestra.run.smithi167:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-09-16T15:29:35.142 INFO:teuthology.orchestra.run.smithi167.stdout:postinstall scriptlet (using /bin/sh): 2024-09-16T15:29:35.142 INFO:teuthology.orchestra.run.smithi167.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-16T15:29:35.143 INFO:teuthology.orchestra.run.smithi167.stdout: /usr/bin/kernel-install add 6.11.0-rc5-g2ca44a2f35ca /lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-16T15:29:35.143 INFO:teuthology.orchestra.run.smithi167.stdout:fi 2024-09-16T15:29:35.143 INFO:teuthology.orchestra.run.smithi167.stdout:for file in vmlinuz System.map config; do 2024-09-16T15:29:35.143 INFO:teuthology.orchestra.run.smithi167.stdout: if ! cmp --silent "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca"; then 2024-09-16T15:29:35.143 INFO:teuthology.orchestra.run.smithi167.stdout: cp "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca" 2024-09-16T15:29:35.143 INFO:teuthology.orchestra.run.smithi167.stdout: fi 2024-09-16T15:29:35.143 INFO:teuthology.orchestra.run.smithi167.stdout:done 2024-09-16T15:29:35.143 INFO:teuthology.orchestra.run.smithi167.stdout:if [ -d "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" ] && \ 2024-09-16T15:29:35.143 INFO:teuthology.orchestra.run.smithi167.stdout: ! diff -rq "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" >/dev/null 2>&1; then 2024-09-16T15:29:35.143 INFO:teuthology.orchestra.run.smithi167.stdout: rm -rf "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-16T15:29:35.143 INFO:teuthology.orchestra.run.smithi167.stdout: cp -r "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-16T15:29:35.143 INFO:teuthology.orchestra.run.smithi167.stdout:fi 2024-09-16T15:29:35.143 INFO:teuthology.orchestra.run.smithi167.stdout:if [ ! -e "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep" ]; then 2024-09-16T15:29:35.143 INFO:teuthology.orchestra.run.smithi167.stdout: /usr/sbin/depmod 6.11.0-rc5-g2ca44a2f35ca 2024-09-16T15:29:35.144 INFO:teuthology.orchestra.run.smithi167.stdout:fi 2024-09-16T15:29:35.144 INFO:teuthology.orchestra.run.smithi167.stdout:preuninstall scriptlet (using /bin/sh): 2024-09-16T15:29:35.144 INFO:teuthology.orchestra.run.smithi167.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-16T15:29:35.144 INFO:teuthology.orchestra.run.smithi167.stdout:kernel-install remove 6.11.0-rc5-g2ca44a2f35ca 2024-09-16T15:29:35.144 INFO:teuthology.orchestra.run.smithi167.stdout:fi 2024-09-16T15:29:35.145 DEBUG:teuthology.orchestra.run.smithi167:> sudo rpm -qi grub2-tools 2024-09-16T15:29:35.233 INFO:teuthology.orchestra.run.smithi167.stdout:Name : grub2-tools 2024-09-16T15:29:35.233 INFO:teuthology.orchestra.run.smithi167.stdout:Epoch : 1 2024-09-16T15:29:35.233 INFO:teuthology.orchestra.run.smithi167.stdout:Version : 2.06 2024-09-16T15:29:35.233 INFO:teuthology.orchestra.run.smithi167.stdout:Release : 82.el9 2024-09-16T15:29:35.233 INFO:teuthology.orchestra.run.smithi167.stdout:Architecture: x86_64 2024-09-16T15:29:35.233 INFO:teuthology.orchestra.run.smithi167.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-16T15:29:35.233 INFO:teuthology.orchestra.run.smithi167.stdout:Group : Unspecified 2024-09-16T15:29:35.233 INFO:teuthology.orchestra.run.smithi167.stdout:Size : 8274670 2024-09-16T15:29:35.233 INFO:teuthology.orchestra.run.smithi167.stdout:License : GPLv3+ 2024-09-16T15:29:35.233 INFO:teuthology.orchestra.run.smithi167.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-16T15:29:35.233 INFO:teuthology.orchestra.run.smithi167.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-16T15:29:35.233 INFO:teuthology.orchestra.run.smithi167.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-16T15:29:35.233 INFO:teuthology.orchestra.run.smithi167.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-16T15:29:35.234 INFO:teuthology.orchestra.run.smithi167.stdout:Packager : builder@centos.org 2024-09-16T15:29:35.234 INFO:teuthology.orchestra.run.smithi167.stdout:Vendor : CentOS 2024-09-16T15:29:35.234 INFO:teuthology.orchestra.run.smithi167.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-16T15:29:35.234 INFO:teuthology.orchestra.run.smithi167.stdout:Summary : Support tools for GRUB. 2024-09-16T15:29:35.234 INFO:teuthology.orchestra.run.smithi167.stdout:Description : 2024-09-16T15:29:35.234 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-16T15:29:35.234 INFO:teuthology.orchestra.run.smithi167.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-16T15:29:35.234 INFO:teuthology.orchestra.run.smithi167.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-16T15:29:35.234 INFO:teuthology.orchestra.run.smithi167.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-16T15:29:35.234 INFO:teuthology.orchestra.run.smithi167.stdout:hardware devices. 2024-09-16T15:29:35.234 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-16T15:29:35.234 INFO:teuthology.orchestra.run.smithi167.stdout:This subpackage provides tools for support of all platforms. 2024-09-16T15:29:35.235 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-16T15:29:35.235 INFO:teuthology.task.kernel:Updating grub on smithi167 to boot 6.11.0-rc5-g2ca44a2f35ca 2024-09-16T15:29:35.235 DEBUG:teuthology.orchestra.run.smithi167:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-16T15:29:35.819 INFO:teuthology.orchestra.run.smithi167.stderr:Generating grub configuration file ... 2024-09-16T15:29:37.248 INFO:teuthology.orchestra.run.smithi167.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-16T15:29:37.278 INFO:teuthology.orchestra.run.smithi167.stderr:done 2024-09-16T15:29:37.281 DEBUG:teuthology.orchestra.run.smithi167:> mktemp 2024-09-16T15:29:37.296 INFO:teuthology.orchestra.run.smithi167.stdout:/tmp/tmp.uMnzRzWufe 2024-09-16T15:29:37.296 DEBUG:teuthology.orchestra.run.smithi167:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.uMnzRzWufe 2024-09-16T15:29:37.378 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 0666 /tmp/tmp.uMnzRzWufe 2024-09-16T15:29:37.518 DEBUG:teuthology.orchestra.remote:smithi167:/tmp/tmp.uMnzRzWufe is 6KB 2024-09-16T15:29:37.567 DEBUG:teuthology.orchestra.run.smithi167:> rm -fr /tmp/tmp.uMnzRzWufe 2024-09-16T15:29:37.582 DEBUG:teuthology.orchestra.run.smithi167:> sudo /bin/ls /boot/loader/entries || true 2024-09-16T15:29:37.651 INFO:teuthology.orchestra.run.smithi167.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-16T15:29:37.652 INFO:teuthology.orchestra.run.smithi167.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-16T15:29:37.652 INFO:teuthology.orchestra.run.smithi167.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-16T15:29:37.652 INFO:teuthology.orchestra.run.smithi167.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca.conf 2024-09-16T15:29:37.653 DEBUG:teuthology.orchestra.run.smithi167:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca 2024-09-16T15:29:37.806 DEBUG:teuthology.orchestra.run.smithi167:> sudo shutdown -r now 2024-09-16T15:29:37.972 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi151.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-09-16T15:29:37.972 DEBUG:teuthology.orchestra.run.smithi151:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-09-16T15:29:38.055 INFO:teuthology.orchestra.run.smithi151.stdout:/boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-16T15:29:38.055 INFO:teuthology.orchestra.run.smithi151.stdout:/boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-16T15:29:38.055 INFO:teuthology.orchestra.run.smithi151.stdout:/boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-16T15:29:38.055 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca 2024-09-16T15:29:38.055 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/System.map 2024-09-16T15:29:38.056 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/config 2024-09-16T15:29:38.056 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel 2024-09-16T15:29:38.056 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch 2024-09-16T15:29:38.056 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86 2024-09-16T15:29:38.056 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto 2024-09-16T15:29:38.056 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-16T15:29:38.056 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-16T15:29:38.056 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-16T15:29:38.056 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-16T15:29:38.056 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-16T15:29:38.056 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-16T15:29:38.056 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-16T15:29:38.056 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32c-intel.ko 2024-09-16T15:29:38.056 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-16T15:29:38.057 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-16T15:29:38.057 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-16T15:29:38.057 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-16T15:29:38.057 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-16T15:29:38.057 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-16T15:29:38.057 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-16T15:29:38.057 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-16T15:29:38.057 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-16T15:29:38.057 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-16T15:29:38.057 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-16T15:29:38.057 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-16T15:29:38.057 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel 2024-09-16T15:29:38.057 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu 2024-09-16T15:29:38.057 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce 2024-09-16T15:29:38.058 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-16T15:29:38.058 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm 2024-09-16T15:29:38.058 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-16T15:29:38.058 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-16T15:29:38.058 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-16T15:29:38.058 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto 2024-09-16T15:29:38.058 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-16T15:29:38.058 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-16T15:29:38.058 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-16T15:29:38.058 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx 2024-09-16T15:29:38.058 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-16T15:29:38.058 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-16T15:29:38.058 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-16T15:29:38.058 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-16T15:29:38.059 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-16T15:29:38.059 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-16T15:29:38.059 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-16T15:29:38.059 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-16T15:29:38.059 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-16T15:29:38.059 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-16T15:29:38.059 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-16T15:29:38.059 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-16T15:29:38.059 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-16T15:29:38.059 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-16T15:29:38.059 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-16T15:29:38.059 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-16T15:29:38.059 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-16T15:29:38.059 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-16T15:29:38.060 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-16T15:29:38.060 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-16T15:29:38.060 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-16T15:29:38.060 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-16T15:29:38.060 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-16T15:29:38.060 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecc.ko 2024-09-16T15:29:38.060 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecdh_generic.ko 2024-09-16T15:29:38.060 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-16T15:29:38.060 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-16T15:29:38.060 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-16T15:29:38.060 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-16T15:29:38.060 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-16T15:29:38.060 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-16T15:29:38.060 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-16T15:29:38.061 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-16T15:29:38.061 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-16T15:29:38.061 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-16T15:29:38.061 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-16T15:29:38.061 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-16T15:29:38.061 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-16T15:29:38.061 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-16T15:29:38.061 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-16T15:29:38.061 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-16T15:29:38.061 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-16T15:29:38.061 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-16T15:29:38.061 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-16T15:29:38.061 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-16T15:29:38.061 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-16T15:29:38.062 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-16T15:29:38.062 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers 2024-09-16T15:29:38.062 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi 2024-09-16T15:29:38.062 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-16T15:29:38.062 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-16T15:29:38.062 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-16T15:29:38.062 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-16T15:29:38.062 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-16T15:29:38.062 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-16T15:29:38.062 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-16T15:29:38.062 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata 2024-09-16T15:29:38.062 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-16T15:29:38.062 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-16T15:29:38.063 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ata_generic.ko 2024-09-16T15:29:38.063 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-16T15:29:38.063 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-16T15:29:38.063 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-16T15:29:38.063 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-16T15:29:38.063 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-16T15:29:38.063 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-16T15:29:38.063 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-16T15:29:38.063 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-16T15:29:38.063 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-16T15:29:38.063 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-16T15:29:38.063 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-16T15:29:38.063 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-16T15:29:38.063 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-16T15:29:38.063 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-16T15:29:38.064 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-16T15:29:38.064 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-16T15:29:38.064 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-16T15:29:38.064 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-16T15:29:38.064 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-16T15:29:38.064 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-16T15:29:38.064 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-16T15:29:38.064 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-16T15:29:38.064 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-16T15:29:38.064 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-16T15:29:38.064 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-16T15:29:38.064 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-16T15:29:38.064 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-16T15:29:38.065 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-16T15:29:38.065 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-16T15:29:38.065 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-16T15:29:38.065 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-16T15:29:38.065 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-16T15:29:38.065 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-16T15:29:38.065 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-16T15:29:38.065 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-16T15:29:38.065 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sis.ko 2024-09-16T15:29:38.065 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-16T15:29:38.065 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-16T15:29:38.065 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-16T15:29:38.065 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-16T15:29:38.065 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-16T15:29:38.066 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-16T15:29:38.066 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-16T15:29:38.066 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-16T15:29:38.066 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-16T15:29:38.066 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-16T15:29:38.066 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-16T15:29:38.066 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-16T15:29:38.066 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-16T15:29:38.066 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-16T15:29:38.066 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-16T15:29:38.066 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-16T15:29:38.066 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-16T15:29:38.066 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay 2024-09-16T15:29:38.066 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-16T15:29:38.067 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base 2024-09-16T15:29:38.067 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap 2024-09-16T15:29:38.067 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-i2c.ko 2024-09-16T15:29:38.067 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-mmio.ko 2024-09-16T15:29:38.067 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma 2024-09-16T15:29:38.067 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-16T15:29:38.067 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block 2024-09-16T15:29:38.067 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe 2024-09-16T15:29:38.067 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-16T15:29:38.067 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/brd.ko 2024-09-16T15:29:38.067 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd 2024-09-16T15:29:38.067 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-16T15:29:38.067 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-16T15:29:38.067 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/loop.ko 2024-09-16T15:29:38.068 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx 2024-09-16T15:29:38.068 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-16T15:29:38.068 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-16T15:29:38.068 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk 2024-09-16T15:29:38.068 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-16T15:29:38.068 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-16T15:29:38.068 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-16T15:29:38.068 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/virtio_blk.ko 2024-09-16T15:29:38.068 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback 2024-09-16T15:29:38.068 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-16T15:29:38.068 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkfront.ko 2024-09-16T15:29:38.068 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram 2024-09-16T15:29:38.068 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-16T15:29:38.069 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char 2024-09-16T15:29:38.069 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-16T15:29:38.069 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random 2024-09-16T15:29:38.069 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-16T15:29:38.069 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-16T15:29:38.069 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-16T15:29:38.069 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-16T15:29:38.069 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-16T15:29:38.069 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi 2024-09-16T15:29:38.069 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-16T15:29:38.069 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-16T15:29:38.069 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-16T15:29:38.069 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-16T15:29:38.069 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-16T15:29:38.070 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-16T15:29:38.070 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-16T15:29:38.070 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave 2024-09-16T15:29:38.070 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-16T15:29:38.070 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-16T15:29:38.070 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-16T15:29:38.070 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm 2024-09-16T15:29:38.070 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm.ko 2024-09-16T15:29:38.070 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-16T15:29:38.070 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-16T15:29:38.070 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-16T15:29:38.070 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis.ko 2024-09-16T15:29:38.070 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-09-16T15:29:38.070 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/uv_mmtimer.ko 2024-09-16T15:29:38.071 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/virtio_console.ko 2024-09-16T15:29:38.071 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq 2024-09-16T15:29:38.071 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-09-16T15:29:38.071 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-16T15:29:38.071 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-16T15:29:38.071 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-09-16T15:29:38.071 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/powernow-k8.ko 2024-09-16T15:29:38.071 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-16T15:29:38.071 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto 2024-09-16T15:29:38.071 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp 2024-09-16T15:29:38.071 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-16T15:29:38.071 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-16T15:29:38.071 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel 2024-09-16T15:29:38.071 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat 2024-09-16T15:29:38.072 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common 2024-09-16T15:29:38.072 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-16T15:29:38.072 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-09-16T15:29:38.072 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-16T15:29:38.072 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-16T15:29:38.072 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-16T15:29:38.072 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio 2024-09-16T15:29:38.072 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-16T15:29:38.072 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca 2024-09-16T15:29:38.072 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-16T15:29:38.072 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq 2024-09-16T15:29:38.072 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-09-16T15:29:38.072 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma 2024-09-16T15:29:38.073 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw 2024-09-16T15:29:38.073 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-16T15:29:38.073 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat 2024-09-16T15:29:38.073 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-16T15:29:38.073 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac 2024-09-16T15:29:38.073 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-16T15:29:38.073 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-16T15:29:38.073 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-16T15:29:38.073 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-16T15:29:38.073 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-16T15:29:38.073 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-16T15:29:38.073 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-16T15:29:38.073 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-16T15:29:38.074 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-16T15:29:38.074 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-16T15:29:38.074 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-16T15:29:38.074 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-16T15:29:38.074 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-16T15:29:38.074 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware 2024-09-16T15:29:38.074 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/edd.ko 2024-09-16T15:29:38.074 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-16T15:29:38.074 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio 2024-09-16T15:29:38.074 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-16T15:29:38.074 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-16T15:29:38.074 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid 2024-09-16T15:29:38.074 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-16T15:29:38.074 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-16T15:29:38.075 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-16T15:29:38.075 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-16T15:29:38.075 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-16T15:29:38.075 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-16T15:29:38.075 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-16T15:29:38.075 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-16T15:29:38.075 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-16T15:29:38.075 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-16T15:29:38.075 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-16T15:29:38.075 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-16T15:29:38.075 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-16T15:29:38.075 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-16T15:29:38.075 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-16T15:29:38.075 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-16T15:29:38.076 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-16T15:29:38.076 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-16T15:29:38.076 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-16T15:29:38.076 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-16T15:29:38.076 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-16T15:29:38.076 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-16T15:29:38.076 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-16T15:29:38.076 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-16T15:29:38.076 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-16T15:29:38.076 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-16T15:29:38.076 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-16T15:29:38.076 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-16T15:29:38.076 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-16T15:29:38.076 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-16T15:29:38.076 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-16T15:29:38.077 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-16T15:29:38.077 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-16T15:29:38.077 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-16T15:29:38.077 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-16T15:29:38.077 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-16T15:29:38.077 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-16T15:29:38.077 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-16T15:29:38.077 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-16T15:29:38.077 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-16T15:29:38.077 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-16T15:29:38.077 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-16T15:29:38.077 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-16T15:29:38.077 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-16T15:29:38.078 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-16T15:29:38.078 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-16T15:29:38.078 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-16T15:29:38.078 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-16T15:29:38.078 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-16T15:29:38.078 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-16T15:29:38.078 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-16T15:29:38.078 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-16T15:29:38.078 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-16T15:29:38.078 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-16T15:29:38.078 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-16T15:29:38.079 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-16T15:29:38.079 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-16T15:29:38.079 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-16T15:29:38.079 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-16T15:29:38.079 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-16T15:29:38.079 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-16T15:29:38.079 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-16T15:29:38.079 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-16T15:29:38.079 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-16T15:29:38.079 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv 2024-09-16T15:29:38.079 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-16T15:29:38.079 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-16T15:29:38.079 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-16T15:29:38.080 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon 2024-09-16T15:29:38.080 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-16T15:29:38.080 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-16T15:29:38.080 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-16T15:29:38.080 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-16T15:29:38.080 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-16T15:29:38.080 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-16T15:29:38.080 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-16T15:29:38.080 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-16T15:29:38.080 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-16T15:29:38.080 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-16T15:29:38.080 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-16T15:29:38.080 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-16T15:29:38.081 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-16T15:29:38.081 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-16T15:29:38.081 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-16T15:29:38.081 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-16T15:29:38.081 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-16T15:29:38.081 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-16T15:29:38.081 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-16T15:29:38.081 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-16T15:29:38.081 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-16T15:29:38.081 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-16T15:29:38.081 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-16T15:29:38.081 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-16T15:29:38.081 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-16T15:29:38.081 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-16T15:29:38.082 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-16T15:29:38.082 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-16T15:29:38.082 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-16T15:29:38.082 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-16T15:29:38.082 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-16T15:29:38.082 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-16T15:29:38.082 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-16T15:29:38.082 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-16T15:29:38.082 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-16T15:29:38.082 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-16T15:29:38.082 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-16T15:29:38.082 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-16T15:29:38.082 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-16T15:29:38.082 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-16T15:29:38.083 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-16T15:29:38.083 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-16T15:29:38.083 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-16T15:29:38.083 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-16T15:29:38.083 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-16T15:29:38.083 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-16T15:29:38.083 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-16T15:29:38.083 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-16T15:29:38.083 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-16T15:29:38.083 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-16T15:29:38.083 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-16T15:29:38.083 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-16T15:29:38.083 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-16T15:29:38.083 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-16T15:29:38.083 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-16T15:29:38.084 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-16T15:29:38.084 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-16T15:29:38.084 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-16T15:29:38.084 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-16T15:29:38.084 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-16T15:29:38.084 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-16T15:29:38.084 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-16T15:29:38.084 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-16T15:29:38.084 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-16T15:29:38.084 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-16T15:29:38.084 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-16T15:29:38.084 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-16T15:29:38.084 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-16T15:29:38.085 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-16T15:29:38.085 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-16T15:29:38.085 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-16T15:29:38.085 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-16T15:29:38.085 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-16T15:29:38.085 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-16T15:29:38.085 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-16T15:29:38.085 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-16T15:29:38.085 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-16T15:29:38.085 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-16T15:29:38.085 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-16T15:29:38.085 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-16T15:29:38.085 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-16T15:29:38.085 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-16T15:29:38.085 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-16T15:29:38.086 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-16T15:29:38.086 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-16T15:29:38.086 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-16T15:29:38.086 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-16T15:29:38.086 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-16T15:29:38.086 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus 2024-09-16T15:29:38.086 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-16T15:29:38.086 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-16T15:29:38.086 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-16T15:29:38.086 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-16T15:29:38.086 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-16T15:29:38.086 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-16T15:29:38.086 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-16T15:29:38.086 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-16T15:29:38.087 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-16T15:29:38.087 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-16T15:29:38.087 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-16T15:29:38.087 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-16T15:29:38.087 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-16T15:29:38.087 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-16T15:29:38.087 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-16T15:29:38.087 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-16T15:29:38.087 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-16T15:29:38.087 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-16T15:29:38.087 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-16T15:29:38.087 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-16T15:29:38.087 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-16T15:29:38.087 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-16T15:29:38.088 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-16T15:29:38.088 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-16T15:29:38.088 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-16T15:29:38.088 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-16T15:29:38.088 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-16T15:29:38.088 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-16T15:29:38.088 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-16T15:29:38.088 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-16T15:29:38.088 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-16T15:29:38.088 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-16T15:29:38.088 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-16T15:29:38.088 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-16T15:29:38.088 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-16T15:29:38.089 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-16T15:29:38.089 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-16T15:29:38.089 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-16T15:29:38.089 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-16T15:29:38.089 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-16T15:29:38.089 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-16T15:29:38.089 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c 2024-09-16T15:29:38.089 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos 2024-09-16T15:29:38.089 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-16T15:29:38.089 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-16T15:29:38.089 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses 2024-09-16T15:29:38.089 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-16T15:29:38.089 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-16T15:29:38.089 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-16T15:29:38.090 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-16T15:29:38.090 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-16T15:29:38.090 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-16T15:29:38.090 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-16T15:29:38.090 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-16T15:29:38.090 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-16T15:29:38.090 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-16T15:29:38.090 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-16T15:29:38.090 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-16T15:29:38.090 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-16T15:29:38.090 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-16T15:29:38.090 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-16T15:29:38.090 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-16T15:29:38.091 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-16T15:29:38.091 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-16T15:29:38.091 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-16T15:29:38.091 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-16T15:29:38.091 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-16T15:29:38.091 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-16T15:29:38.091 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-16T15:29:38.091 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-dev.ko 2024-09-16T15:29:38.091 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-16T15:29:38.091 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-09-16T15:29:38.091 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-16T15:29:38.091 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-16T15:29:38.091 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband 2024-09-16T15:29:38.091 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core 2024-09-16T15:29:38.092 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-16T15:29:38.092 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-16T15:29:38.092 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-16T15:29:38.092 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-16T15:29:38.092 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-16T15:29:38.092 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-16T15:29:38.092 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-16T15:29:38.092 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw 2024-09-16T15:29:38.092 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4 2024-09-16T15:29:38.092 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-16T15:29:38.092 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4 2024-09-16T15:29:38.092 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-16T15:29:38.092 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5 2024-09-16T15:29:38.092 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-16T15:29:38.093 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca 2024-09-16T15:29:38.093 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-16T15:29:38.093 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp 2024-09-16T15:29:38.093 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib 2024-09-16T15:29:38.093 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-16T15:29:38.093 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser 2024-09-16T15:29:38.093 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-16T15:29:38.093 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert 2024-09-16T15:29:38.093 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-16T15:29:38.093 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp 2024-09-16T15:29:38.093 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-16T15:29:38.093 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt 2024-09-16T15:29:38.093 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-16T15:29:38.093 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input 2024-09-16T15:29:38.094 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport 2024-09-16T15:29:38.094 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-16T15:29:38.094 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-16T15:29:38.094 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-16T15:29:38.094 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-16T15:29:38.094 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard 2024-09-16T15:29:38.094 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-16T15:29:38.094 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse 2024-09-16T15:29:38.094 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-16T15:29:38.094 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-16T15:29:38.094 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-16T15:29:38.094 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-16T15:29:38.094 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-16T15:29:38.094 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-16T15:29:38.094 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-16T15:29:38.095 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-16T15:29:38.095 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4 2024-09-16T15:29:38.095 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-16T15:29:38.095 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio 2024-09-16T15:29:38.095 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-16T15:29:38.095 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-16T15:29:38.095 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-16T15:29:38.095 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-16T15:29:38.095 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-16T15:29:38.095 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds 2024-09-16T15:29:38.095 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-16T15:29:38.095 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-16T15:29:38.095 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-16T15:29:38.095 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-16T15:29:38.096 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger 2024-09-16T15:29:38.096 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-16T15:29:38.096 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-16T15:29:38.096 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-16T15:29:38.096 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-16T15:29:38.096 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-16T15:29:38.096 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-16T15:29:38.096 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-16T15:29:38.096 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-16T15:29:38.096 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md 2024-09-16T15:29:38.096 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache 2024-09-16T15:29:38.096 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-16T15:29:38.096 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-16T15:29:38.096 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-16T15:29:38.096 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-16T15:29:38.097 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-16T15:29:38.097 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-16T15:29:38.097 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-16T15:29:38.097 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-16T15:29:38.097 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-16T15:29:38.097 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-16T15:29:38.097 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-16T15:29:38.097 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-16T15:29:38.097 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-16T15:29:38.097 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-16T15:29:38.097 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-16T15:29:38.097 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-16T15:29:38.098 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data 2024-09-16T15:29:38.098 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-16T15:29:38.098 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-16T15:29:38.098 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-16T15:29:38.098 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-16T15:29:38.098 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-16T15:29:38.098 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message 2024-09-16T15:29:38.098 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion 2024-09-16T15:29:38.098 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-16T15:29:38.098 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-16T15:29:38.098 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-16T15:29:38.098 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-16T15:29:38.098 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-16T15:29:38.098 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-16T15:29:38.099 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-16T15:29:38.099 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd 2024-09-16T15:29:38.099 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-16T15:29:38.099 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-16T15:29:38.099 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mfd-core.ko 2024-09-16T15:29:38.099 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-16T15:29:38.099 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-16T15:29:38.099 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-16T15:29:38.099 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-16T15:29:38.099 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc 2024-09-16T15:29:38.099 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl 2024-09-16T15:29:38.099 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-16T15:29:38.099 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-16T15:29:38.099 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-16T15:29:38.100 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-16T15:29:38.100 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710 2024-09-16T15:29:38.100 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-16T15:29:38.100 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom 2024-09-16T15:29:38.100 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-16T15:29:38.100 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-16T15:29:38.100 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-16T15:29:38.100 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-16T15:29:38.100 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-16T15:29:38.100 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-16T15:29:38.100 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-16T15:29:38.100 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d 2024-09-16T15:29:38.100 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-16T15:29:38.100 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-16T15:29:38.101 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei 2024-09-16T15:29:38.101 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-16T15:29:38.101 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-16T15:29:38.101 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-16T15:29:38.101 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru 2024-09-16T15:29:38.101 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru/gru.ko 2024-09-16T15:29:38.101 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp 2024-09-16T15:29:38.101 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xp.ko 2024-09-16T15:29:38.101 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpc.ko 2024-09-16T15:29:38.101 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-09-16T15:29:38.101 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-16T15:29:38.101 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-16T15:29:38.101 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-16T15:29:38.101 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-16T15:29:38.102 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci 2024-09-16T15:29:38.102 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-16T15:29:38.102 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net 2024-09-16T15:29:38.102 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding 2024-09-16T15:29:38.102 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-16T15:29:38.102 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa 2024-09-16T15:29:38.102 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53 2024-09-16T15:29:38.102 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-16T15:29:38.102 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-16T15:29:38.102 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-16T15:29:38.102 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx 2024-09-16T15:29:38.102 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-16T15:29:38.102 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-16T15:29:38.102 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-16T15:29:38.102 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet 2024-09-16T15:29:38.103 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com 2024-09-16T15:29:38.103 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-16T15:29:38.103 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-16T15:29:38.103 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-16T15:29:38.103 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-16T15:29:38.103 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390 2024-09-16T15:29:38.103 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-16T15:29:38.103 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-16T15:29:38.103 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-16T15:29:38.103 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-16T15:29:38.103 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec 2024-09-16T15:29:38.103 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-16T15:29:38.103 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere 2024-09-16T15:29:38.103 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-16T15:29:38.103 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon 2024-09-16T15:29:38.104 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-16T15:29:38.104 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera 2024-09-16T15:29:38.104 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-16T15:29:38.104 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd 2024-09-16T15:29:38.104 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-16T15:29:38.104 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-16T15:29:38.104 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-16T15:29:38.104 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros 2024-09-16T15:29:38.104 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx 2024-09-16T15:29:38.104 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-16T15:29:38.104 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c 2024-09-16T15:29:38.104 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-16T15:29:38.104 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e 2024-09-16T15:29:38.104 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-16T15:29:38.105 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx 2024-09-16T15:29:38.105 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-16T15:29:38.105 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-16T15:29:38.105 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom 2024-09-16T15:29:38.105 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-16T15:29:38.105 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-16T15:29:38.105 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-09-16T15:29:38.105 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-16T15:29:38.105 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-16T15:29:38.105 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet 2024-09-16T15:29:38.105 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-16T15:29:38.105 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-16T15:29:38.105 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade 2024-09-16T15:29:38.105 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna 2024-09-16T15:29:38.105 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-16T15:29:38.106 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio 2024-09-16T15:29:38.106 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb 2024-09-16T15:29:38.106 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-16T15:29:38.106 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-09-16T15:29:38.106 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-16T15:29:38.106 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-09-16T15:29:38.106 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-16T15:29:38.106 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-09-16T15:29:38.106 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-16T15:29:38.106 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-09-16T15:29:38.106 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-16T15:29:38.106 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco 2024-09-16T15:29:38.106 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic 2024-09-16T15:29:38.106 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-16T15:29:38.106 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec 2024-09-16T15:29:38.107 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip 2024-09-16T15:29:38.107 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-16T15:29:38.107 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-16T15:29:38.107 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-16T15:29:38.107 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-16T15:29:38.107 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-16T15:29:38.107 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-16T15:29:38.107 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink 2024-09-16T15:29:38.107 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-16T15:29:38.107 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-16T15:29:38.107 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-16T15:29:38.107 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex 2024-09-16T15:29:38.107 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet 2024-09-16T15:29:38.107 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-16T15:29:38.108 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-16T15:29:38.108 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-16T15:29:38.108 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel 2024-09-16T15:29:38.108 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-16T15:29:38.108 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000 2024-09-16T15:29:38.108 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-16T15:29:38.108 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e 2024-09-16T15:29:38.108 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-16T15:29:38.108 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k 2024-09-16T15:29:38.108 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-16T15:29:38.108 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e 2024-09-16T15:29:38.108 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-16T15:29:38.108 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf 2024-09-16T15:29:38.108 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-16T15:29:38.109 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb 2024-09-16T15:29:38.109 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-16T15:29:38.109 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf 2024-09-16T15:29:38.109 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-16T15:29:38.109 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe 2024-09-16T15:29:38.109 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-16T15:29:38.109 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf 2024-09-16T15:29:38.109 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-16T15:29:38.109 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth 2024-09-16T15:29:38.109 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-16T15:29:38.109 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie 2024-09-16T15:29:38.109 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-16T15:29:38.109 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-16T15:29:38.109 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell 2024-09-16T15:29:38.110 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-16T15:29:38.110 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-16T15:29:38.110 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-16T15:29:38.110 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox 2024-09-16T15:29:38.110 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4 2024-09-16T15:29:38.110 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-16T15:29:38.110 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-16T15:29:38.110 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5 2024-09-16T15:29:38.110 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-09-16T15:29:38.110 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-16T15:29:38.110 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel 2024-09-16T15:29:38.110 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-16T15:29:38.110 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom 2024-09-16T15:29:38.110 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge 2024-09-16T15:29:38.110 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-16T15:29:38.111 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi 2024-09-16T15:29:38.111 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-16T15:29:38.111 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-16T15:29:38.111 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion 2024-09-16T15:29:38.111 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-16T15:29:38.111 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia 2024-09-16T15:29:38.111 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-16T15:29:38.111 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines 2024-09-16T15:29:38.111 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-16T15:29:38.111 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-16T15:29:38.111 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic 2024-09-16T15:29:38.111 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen 2024-09-16T15:29:38.111 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-16T15:29:38.111 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-16T15:29:38.111 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-09-16T15:29:38.112 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-16T15:29:38.112 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc 2024-09-16T15:29:38.112 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-16T15:29:38.112 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek 2024-09-16T15:29:38.112 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-16T15:29:38.112 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-16T15:29:38.112 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-16T15:29:38.112 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-16T15:29:38.112 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker 2024-09-16T15:29:38.112 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-09-16T15:29:38.112 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc 2024-09-16T15:29:38.112 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-16T15:29:38.112 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan 2024-09-16T15:29:38.112 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-16T15:29:38.113 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis 2024-09-16T15:29:38.113 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-16T15:29:38.113 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-16T15:29:38.113 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc 2024-09-16T15:29:38.113 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-16T15:29:38.113 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-16T15:29:38.113 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-16T15:29:38.113 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-16T15:29:38.113 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro 2024-09-16T15:29:38.113 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac 2024-09-16T15:29:38.113 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-16T15:29:38.113 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-16T15:29:38.113 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun 2024-09-16T15:29:38.114 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-16T15:29:38.114 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-16T15:29:38.114 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-16T15:29:38.114 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-16T15:29:38.114 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti 2024-09-16T15:29:38.114 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-16T15:29:38.114 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti 2024-09-16T15:29:38.114 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-16T15:29:38.114 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via 2024-09-16T15:29:38.114 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-16T15:29:38.114 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-16T15:29:38.114 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet 2024-09-16T15:29:38.114 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-16T15:29:38.114 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-16T15:29:38.115 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom 2024-09-16T15:29:38.115 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-16T15:29:38.115 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-16T15:29:38.115 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv 2024-09-16T15:29:38.115 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-16T15:29:38.115 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154 2024-09-16T15:29:38.115 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-16T15:29:38.115 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-16T15:29:38.115 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan 2024-09-16T15:29:38.115 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-16T15:29:38.115 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-16T15:29:38.115 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-16T15:29:38.115 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio 2024-09-16T15:29:38.115 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-16T15:29:38.115 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-16T15:29:38.116 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-16T15:29:38.116 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-16T15:29:38.116 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-16T15:29:38.116 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/net_failover.ko 2024-09-16T15:29:38.116 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-16T15:29:38.116 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-16T15:29:38.116 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs 2024-09-16T15:29:38.116 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-16T15:29:38.116 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-16T15:29:38.116 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy 2024-09-16T15:29:38.116 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-16T15:29:38.116 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-16T15:29:38.116 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-09-16T15:29:38.116 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-16T15:29:38.116 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-16T15:29:38.117 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-16T15:29:38.117 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-16T15:29:38.117 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-16T15:29:38.117 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/dp83640.ko 2024-09-16T15:29:38.117 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-16T15:29:38.117 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-16T15:29:38.117 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-16T15:29:38.117 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-16T15:29:38.117 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-16T15:29:38.117 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-16T15:29:38.117 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-16T15:29:38.117 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-16T15:29:38.117 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-16T15:29:38.117 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-16T15:29:38.118 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-16T15:29:38.118 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-16T15:29:38.118 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp 2024-09-16T15:29:38.118 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-16T15:29:38.118 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-16T15:29:38.118 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-16T15:29:38.118 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_generic.ko 2024-09-16T15:29:38.118 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-16T15:29:38.118 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-16T15:29:38.118 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-16T15:29:38.118 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-16T15:29:38.118 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-16T15:29:38.118 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip 2024-09-16T15:29:38.118 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slhc.ko 2024-09-16T15:29:38.118 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-16T15:29:38.119 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-16T15:29:38.119 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-16T15:29:38.119 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team 2024-09-16T15:29:38.119 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-16T15:29:38.119 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-16T15:29:38.119 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-16T15:29:38.119 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-16T15:29:38.119 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-16T15:29:38.119 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-16T15:29:38.119 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tun.ko 2024-09-16T15:29:38.119 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-16T15:29:38.119 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/virtio_net.ko 2024-09-16T15:29:38.119 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3 2024-09-16T15:29:38.119 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-16T15:29:38.119 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan 2024-09-16T15:29:38.120 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-16T15:29:38.120 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback 2024-09-16T15:29:38.120 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-16T15:29:38.120 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netfront.ko 2024-09-16T15:29:38.120 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb 2024-09-16T15:29:38.120 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-16T15:29:38.120 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme 2024-09-16T15:29:38.120 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host 2024-09-16T15:29:38.120 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-16T15:29:38.120 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-16T15:29:38.120 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport 2024-09-16T15:29:38.120 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-16T15:29:38.120 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-16T15:29:38.120 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-16T15:29:38.121 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-16T15:29:38.121 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci 2024-09-16T15:29:38.121 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug 2024-09-16T15:29:38.121 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-16T15:29:38.121 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie 2024-09-16T15:29:38.121 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie/aer_inject.ko 2024-09-16T15:29:38.121 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-16T15:29:38.121 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia 2024-09-16T15:29:38.121 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-16T15:29:38.121 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-16T15:29:38.121 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-16T15:29:38.121 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl 2024-09-16T15:29:38.121 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel 2024-09-16T15:29:38.121 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-16T15:29:38.121 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform 2024-09-16T15:29:38.122 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86 2024-09-16T15:29:38.122 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-16T15:29:38.122 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-16T15:29:38.122 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-16T15:29:38.122 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-16T15:29:38.122 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-16T15:29:38.122 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-16T15:29:38.122 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-16T15:29:38.122 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-16T15:29:38.122 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-16T15:29:38.122 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-16T15:29:38.122 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-16T15:29:38.122 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-16T15:29:38.122 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel 2024-09-16T15:29:38.122 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-16T15:29:38.122 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-16T15:29:38.123 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-16T15:29:38.123 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-16T15:29:38.123 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-16T15:29:38.123 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-16T15:29:38.123 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-16T15:29:38.123 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-16T15:29:38.123 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-16T15:29:38.123 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-16T15:29:38.123 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-16T15:29:38.123 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-16T15:29:38.123 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap 2024-09-16T15:29:38.123 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-16T15:29:38.123 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-16T15:29:38.123 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps 2024-09-16T15:29:38.123 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients 2024-09-16T15:29:38.124 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-16T15:29:38.124 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-16T15:29:38.124 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-16T15:29:38.124 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-16T15:29:38.124 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp 2024-09-16T15:29:38.124 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-16T15:29:38.124 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-16T15:29:38.124 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras 2024-09-16T15:29:38.124 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd 2024-09-16T15:29:38.124 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl 2024-09-16T15:29:38.124 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-16T15:29:38.124 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-16T15:29:38.124 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc 2024-09-16T15:29:38.124 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-16T15:29:38.124 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-16T15:29:38.125 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-16T15:29:38.125 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-16T15:29:38.125 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-16T15:29:38.125 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-16T15:29:38.125 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-16T15:29:38.125 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-16T15:29:38.125 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-16T15:29:38.125 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-16T15:29:38.125 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-16T15:29:38.125 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-16T15:29:38.125 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-16T15:29:38.125 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-16T15:29:38.125 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-16T15:29:38.125 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-16T15:29:38.125 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-16T15:29:38.126 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-16T15:29:38.126 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-16T15:29:38.126 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-16T15:29:38.126 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-16T15:29:38.126 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-16T15:29:38.126 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-16T15:29:38.126 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-16T15:29:38.126 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-16T15:29:38.126 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-16T15:29:38.126 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-16T15:29:38.126 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-16T15:29:38.126 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-16T15:29:38.126 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-16T15:29:38.126 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-16T15:29:38.127 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi 2024-09-16T15:29:38.127 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-16T15:29:38.127 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-16T15:29:38.127 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-16T15:29:38.127 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-16T15:29:38.127 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-16T15:29:38.127 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid 2024-09-16T15:29:38.127 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-16T15:29:38.127 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-16T15:29:38.127 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx 2024-09-16T15:29:38.127 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-16T15:29:38.127 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-16T15:29:38.127 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-16T15:29:38.127 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr 2024-09-16T15:29:38.128 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-16T15:29:38.128 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-16T15:29:38.128 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi 2024-09-16T15:29:38.128 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-16T15:29:38.128 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa 2024-09-16T15:29:38.128 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-16T15:29:38.128 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc 2024-09-16T15:29:38.128 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-16T15:29:38.128 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i 2024-09-16T15:29:38.128 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-16T15:29:38.128 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-16T15:29:38.128 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor 2024-09-16T15:29:38.128 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-16T15:29:38.128 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi 2024-09-16T15:29:38.129 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i 2024-09-16T15:29:38.129 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-16T15:29:38.129 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i 2024-09-16T15:29:38.129 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-16T15:29:38.129 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-16T15:29:38.129 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-16T15:29:38.129 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler 2024-09-16T15:29:38.129 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-09-16T15:29:38.129 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-09-16T15:29:38.129 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-09-16T15:29:38.129 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-09-16T15:29:38.129 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-16T15:29:38.129 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r 2024-09-16T15:29:38.129 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-16T15:29:38.129 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-16T15:29:38.130 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe 2024-09-16T15:29:38.130 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-16T15:29:38.130 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-16T15:29:38.130 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic 2024-09-16T15:29:38.130 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-16T15:29:38.130 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-16T15:29:38.130 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-16T15:29:38.130 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-16T15:29:38.130 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-16T15:29:38.130 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-16T15:29:38.130 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-16T15:29:38.130 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci 2024-09-16T15:29:38.130 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-16T15:29:38.130 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-16T15:29:38.131 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-16T15:29:38.131 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc 2024-09-16T15:29:38.131 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-16T15:29:38.131 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-16T15:29:38.131 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-16T15:29:38.131 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas 2024-09-16T15:29:38.131 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-16T15:29:38.131 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc 2024-09-16T15:29:38.131 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-16T15:29:38.131 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid 2024-09-16T15:29:38.131 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-16T15:29:38.131 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-16T15:29:38.131 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-16T15:29:38.131 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-16T15:29:38.131 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas 2024-09-16T15:29:38.132 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-16T15:29:38.132 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas 2024-09-16T15:29:38.132 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-16T15:29:38.132 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-16T15:29:38.132 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001 2024-09-16T15:29:38.132 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-16T15:29:38.132 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-16T15:29:38.132 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-16T15:29:38.132 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx 2024-09-16T15:29:38.132 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-16T15:29:38.132 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-16T15:29:38.132 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx 2024-09-16T15:29:38.132 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-16T15:29:38.132 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-16T15:29:38.133 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-16T15:29:38.133 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-16T15:29:38.133 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-16T15:29:38.133 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-16T15:29:38.133 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-16T15:29:38.133 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-16T15:29:38.133 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-16T15:29:38.133 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-16T15:29:38.133 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-16T15:29:38.133 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2 2024-09-16T15:29:38.133 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-16T15:29:38.133 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-16T15:29:38.133 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-16T15:29:38.133 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-16T15:29:38.134 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb 2024-09-16T15:29:38.134 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-16T15:29:38.134 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target 2024-09-16T15:29:38.134 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi 2024-09-16T15:29:38.134 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-16T15:29:38.134 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback 2024-09-16T15:29:38.134 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-16T15:29:38.134 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-16T15:29:38.134 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-16T15:29:38.134 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-16T15:29:38.134 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-16T15:29:38.134 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-16T15:29:38.134 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc 2024-09-16T15:29:38.134 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-16T15:29:38.134 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal 2024-09-16T15:29:38.134 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel 2024-09-16T15:29:38.135 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal 2024-09-16T15:29:38.135 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-16T15:29:38.135 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-16T15:29:38.135 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-16T15:29:38.135 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-16T15:29:38.135 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-16T15:29:38.135 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-16T15:29:38.135 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-16T15:29:38.135 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-16T15:29:38.135 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-16T15:29:38.135 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-16T15:29:38.135 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-16T15:29:38.135 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-16T15:29:38.135 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-16T15:29:38.135 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-16T15:29:38.136 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-16T15:29:38.136 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-16T15:29:38.136 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-16T15:29:38.136 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-16T15:29:38.136 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty 2024-09-16T15:29:38.136 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless 2024-09-16T15:29:38.136 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-16T15:29:38.136 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-16T15:29:38.136 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-16T15:29:38.136 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-16T15:29:38.136 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial 2024-09-16T15:29:38.136 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250 2024-09-16T15:29:38.136 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-16T15:29:38.136 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-16T15:29:38.137 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm 2024-09-16T15:29:38.137 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-16T15:29:38.137 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs 2024-09-16T15:29:38.137 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core 2024-09-16T15:29:38.137 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-16T15:29:38.137 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host 2024-09-16T15:29:38.137 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-16T15:29:38.137 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio 2024-09-16T15:29:38.137 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-16T15:29:38.137 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-16T15:29:38.137 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-16T15:29:38.137 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-16T15:29:38.137 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-16T15:29:38.137 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb 2024-09-16T15:29:38.137 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class 2024-09-16T15:29:38.138 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-16T15:29:38.138 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-16T15:29:38.138 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-16T15:29:38.138 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-16T15:29:38.138 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host 2024-09-16T15:29:38.138 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-16T15:29:38.138 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image 2024-09-16T15:29:38.138 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-16T15:29:38.138 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-16T15:29:38.138 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc 2024-09-16T15:29:38.138 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-16T15:29:38.138 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-16T15:29:38.138 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-16T15:29:38.138 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-16T15:29:38.138 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-16T15:29:38.139 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-16T15:29:38.139 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-16T15:29:38.139 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-16T15:29:38.139 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-16T15:29:38.139 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-16T15:29:38.139 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga 2024-09-16T15:29:38.139 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-16T15:29:38.139 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-16T15:29:38.139 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-16T15:29:38.139 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-16T15:29:38.139 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-16T15:29:38.139 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-16T15:29:38.139 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-16T15:29:38.139 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy 2024-09-16T15:29:38.139 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-16T15:29:38.140 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial 2024-09-16T15:29:38.140 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-16T15:29:38.140 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-16T15:29:38.140 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-16T15:29:38.140 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-16T15:29:38.140 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-16T15:29:38.140 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-16T15:29:38.140 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-16T15:29:38.140 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-16T15:29:38.140 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-16T15:29:38.140 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-16T15:29:38.140 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-16T15:29:38.140 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-16T15:29:38.140 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-16T15:29:38.140 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-16T15:29:38.141 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-16T15:29:38.141 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-16T15:29:38.141 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-16T15:29:38.141 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-16T15:29:38.141 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-16T15:29:38.141 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-16T15:29:38.141 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-16T15:29:38.141 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-16T15:29:38.141 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-16T15:29:38.141 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-16T15:29:38.141 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-16T15:29:38.141 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-16T15:29:38.141 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-16T15:29:38.141 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-16T15:29:38.141 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-16T15:29:38.141 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-16T15:29:38.142 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-16T15:29:38.142 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-16T15:29:38.142 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-16T15:29:38.142 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-16T15:29:38.142 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-16T15:29:38.142 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-16T15:29:38.142 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-16T15:29:38.142 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-16T15:29:38.142 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-16T15:29:38.142 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-16T15:29:38.142 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-16T15:29:38.142 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-16T15:29:38.142 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-16T15:29:38.142 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-16T15:29:38.142 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-16T15:29:38.143 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage 2024-09-16T15:29:38.143 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-16T15:29:38.143 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-16T15:29:38.143 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-16T15:29:38.143 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-16T15:29:38.143 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-16T15:29:38.143 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-16T15:29:38.143 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-16T15:29:38.143 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-16T15:29:38.143 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-16T15:29:38.143 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-16T15:29:38.143 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-16T15:29:38.143 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-16T15:29:38.143 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-16T15:29:38.144 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-16T15:29:38.144 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-16T15:29:38.144 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip 2024-09-16T15:29:38.144 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-16T15:29:38.144 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-16T15:29:38.144 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-16T15:29:38.144 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio 2024-09-16T15:29:38.144 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci 2024-09-16T15:29:38.144 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-16T15:29:38.144 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-16T15:29:38.144 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-16T15:29:38.144 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-16T15:29:38.144 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost 2024-09-16T15:29:38.144 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-16T15:29:38.144 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-16T15:29:38.145 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-16T15:29:38.145 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-16T15:29:38.145 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video 2024-09-16T15:29:38.145 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight 2024-09-16T15:29:38.145 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-16T15:29:38.145 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-16T15:29:38.145 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-16T15:29:38.145 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio 2024-09-16T15:29:38.145 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio.ko 2024-09-16T15:29:38.145 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_balloon.ko 2024-09-16T15:29:38.145 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_mmio.ko 2024-09-16T15:29:38.145 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci.ko 2024-09-16T15:29:38.145 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-09-16T15:29:38.145 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-09-16T15:29:38.145 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_ring.ko 2024-09-16T15:29:38.146 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1 2024-09-16T15:29:38.146 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters 2024-09-16T15:29:38.146 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-16T15:29:38.146 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-16T15:29:38.146 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves 2024-09-16T15:29:38.146 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-16T15:29:38.146 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-16T15:29:38.146 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-16T15:29:38.146 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-16T15:29:38.146 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-16T15:29:38.146 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-16T15:29:38.146 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-16T15:29:38.146 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-16T15:29:38.146 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-16T15:29:38.146 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-16T15:29:38.146 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-16T15:29:38.147 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-16T15:29:38.147 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog 2024-09-16T15:29:38.147 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-16T15:29:38.147 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-16T15:29:38.147 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-16T15:29:38.147 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-16T15:29:38.147 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-16T15:29:38.147 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-16T15:29:38.147 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-16T15:29:38.147 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-16T15:29:38.147 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-16T15:29:38.147 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-16T15:29:38.147 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-16T15:29:38.147 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-16T15:29:38.147 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-16T15:29:38.148 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-16T15:29:38.148 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-16T15:29:38.148 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-16T15:29:38.148 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-16T15:29:38.148 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-16T15:29:38.148 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-16T15:29:38.148 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-16T15:29:38.148 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-16T15:29:38.148 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-16T15:29:38.148 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-16T15:29:38.148 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-16T15:29:38.148 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-16T15:29:38.148 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-16T15:29:38.148 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen 2024-09-16T15:29:38.148 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-acpi-processor.ko 2024-09-16T15:29:38.148 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-16T15:29:38.149 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-16T15:29:38.149 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-16T15:29:38.149 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback 2024-09-16T15:29:38.149 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-16T15:29:38.149 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-16T15:29:38.149 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs 2024-09-16T15:29:38.149 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-16T15:29:38.149 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs 2024-09-16T15:29:38.149 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p 2024-09-16T15:29:38.149 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-16T15:29:38.149 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs 2024-09-16T15:29:38.149 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-16T15:29:38.149 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs 2024-09-16T15:29:38.149 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-16T15:29:38.149 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-16T15:29:38.150 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs 2024-09-16T15:29:38.150 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-16T15:29:38.150 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles 2024-09-16T15:29:38.150 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-16T15:29:38.150 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph 2024-09-16T15:29:38.150 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-16T15:29:38.150 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda 2024-09-16T15:29:38.150 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-16T15:29:38.150 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs 2024-09-16T15:29:38.150 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-16T15:29:38.150 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm 2024-09-16T15:29:38.150 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-16T15:29:38.150 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs 2024-09-16T15:29:38.150 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs/ecryptfs.ko 2024-09-16T15:29:38.150 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs 2024-09-16T15:29:38.150 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-16T15:29:38.151 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat 2024-09-16T15:29:38.151 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/fat.ko 2024-09-16T15:29:38.151 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-16T15:29:38.151 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/vfat.ko 2024-09-16T15:29:38.151 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse 2024-09-16T15:29:38.151 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-16T15:29:38.151 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/fuse.ko 2024-09-16T15:29:38.151 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2 2024-09-16T15:29:38.151 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-16T15:29:38.151 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs 2024-09-16T15:29:38.151 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-16T15:29:38.151 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus 2024-09-16T15:29:38.151 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-16T15:29:38.151 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs 2024-09-16T15:29:38.152 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-16T15:29:38.152 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs 2024-09-16T15:29:38.152 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-16T15:29:38.152 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd 2024-09-16T15:29:38.152 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-16T15:29:38.152 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix 2024-09-16T15:29:38.152 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-16T15:29:38.152 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs 2024-09-16T15:29:38.152 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-16T15:29:38.152 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs 2024-09-16T15:29:38.152 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout 2024-09-16T15:29:38.152 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-16T15:29:38.152 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout 2024-09-16T15:29:38.152 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-16T15:29:38.153 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout 2024-09-16T15:29:38.153 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-16T15:29:38.153 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-16T15:29:38.153 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-16T15:29:38.153 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-16T15:29:38.153 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common 2024-09-16T15:29:38.153 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-16T15:29:38.153 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-16T15:29:38.153 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd 2024-09-16T15:29:38.153 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-16T15:29:38.153 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2 2024-09-16T15:29:38.153 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-16T15:29:38.153 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls 2024-09-16T15:29:38.153 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-16T15:29:38.153 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-16T15:29:38.154 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-16T15:29:38.154 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-16T15:29:38.154 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-16T15:29:38.154 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-16T15:29:38.154 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-16T15:29:38.154 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-16T15:29:38.154 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-16T15:29:38.154 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-16T15:29:38.154 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-16T15:29:38.154 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-16T15:29:38.154 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-16T15:29:38.154 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-16T15:29:38.154 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-16T15:29:38.154 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-16T15:29:38.155 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-16T15:29:38.155 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-16T15:29:38.155 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-16T15:29:38.155 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-16T15:29:38.155 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-16T15:29:38.155 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-16T15:29:38.155 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-16T15:29:38.155 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-16T15:29:38.155 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-16T15:29:38.155 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-16T15:29:38.155 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-16T15:29:38.155 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-16T15:29:38.155 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-16T15:29:38.155 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-16T15:29:38.155 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-16T15:29:38.156 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-16T15:29:38.156 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-16T15:29:38.156 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-16T15:29:38.156 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-16T15:29:38.156 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-16T15:29:38.156 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-16T15:29:38.156 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-16T15:29:38.156 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-16T15:29:38.156 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-16T15:29:38.156 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-16T15:29:38.156 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-16T15:29:38.156 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-16T15:29:38.156 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-16T15:29:38.156 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-16T15:29:38.156 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-16T15:29:38.157 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-16T15:29:38.157 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-16T15:29:38.157 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-16T15:29:38.157 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-16T15:29:38.157 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2 2024-09-16T15:29:38.157 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster 2024-09-16T15:29:38.157 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-16T15:29:38.157 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm 2024-09-16T15:29:38.157 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-16T15:29:38.157 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs 2024-09-16T15:29:38.157 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-16T15:29:38.157 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-16T15:29:38.157 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-16T15:29:38.157 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-16T15:29:38.157 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-16T15:29:38.157 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs 2024-09-16T15:29:38.158 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-16T15:29:38.158 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore 2024-09-16T15:29:38.158 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-16T15:29:38.158 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs 2024-09-16T15:29:38.158 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-16T15:29:38.158 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs 2024-09-16T15:29:38.158 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-16T15:29:38.158 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb 2024-09-16T15:29:38.158 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client 2024-09-16T15:29:38.158 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-16T15:29:38.158 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common 2024-09-16T15:29:38.158 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-16T15:29:38.158 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-16T15:29:38.158 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs 2024-09-16T15:29:38.158 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-16T15:29:38.159 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv 2024-09-16T15:29:38.159 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-16T15:29:38.159 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf 2024-09-16T15:29:38.159 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-16T15:29:38.159 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs 2024-09-16T15:29:38.159 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-16T15:29:38.159 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs 2024-09-16T15:29:38.159 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-16T15:29:38.159 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel 2024-09-16T15:29:38.159 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu 2024-09-16T15:29:38.159 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu/rcutorture.ko 2024-09-16T15:29:38.159 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-16T15:29:38.159 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace 2024-09-16T15:29:38.160 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-09-16T15:29:38.160 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib 2024-09-16T15:29:38.160 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/asn1_encoder.ko 2024-09-16T15:29:38.160 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-16T15:29:38.160 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-16T15:29:38.160 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto 2024-09-16T15:29:38.160 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libaescfb.ko 2024-09-16T15:29:38.160 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-16T15:29:38.160 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-16T15:29:38.160 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-16T15:29:38.160 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-16T15:29:38.160 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4 2024-09-16T15:29:38.160 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-16T15:29:38.160 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-16T15:29:38.160 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math 2024-09-16T15:29:38.161 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-16T15:29:38.161 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6 2024-09-16T15:29:38.161 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-16T15:29:38.161 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon 2024-09-16T15:29:38.161 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-16T15:29:38.161 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-16T15:29:38.161 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-16T15:29:38.161 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-16T15:29:38.161 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd 2024-09-16T15:29:38.161 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd/zstd_compress.ko 2024-09-16T15:29:38.161 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm 2024-09-16T15:29:38.161 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-16T15:29:38.161 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net 2024-09-16T15:29:38.161 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan 2024-09-16T15:29:38.162 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-16T15:29:38.162 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-16T15:29:38.162 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-16T15:29:38.162 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-16T15:29:38.162 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-16T15:29:38.162 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-16T15:29:38.162 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-16T15:29:38.162 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-16T15:29:38.162 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802 2024-09-16T15:29:38.162 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-16T15:29:38.162 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-16T15:29:38.162 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-16T15:29:38.162 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-16T15:29:38.162 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-16T15:29:38.162 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q 2024-09-16T15:29:38.163 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-16T15:29:38.163 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p 2024-09-16T15:29:38.163 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-16T15:29:38.163 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-16T15:29:38.163 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-16T15:29:38.163 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-16T15:29:38.163 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv 2024-09-16T15:29:38.163 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-16T15:29:38.163 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge 2024-09-16T15:29:38.163 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-16T15:29:38.163 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-16T15:29:38.163 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter 2024-09-16T15:29:38.163 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-16T15:29:38.163 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-16T15:29:38.164 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-16T15:29:38.164 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-16T15:29:38.164 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-16T15:29:38.164 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-16T15:29:38.164 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-16T15:29:38.164 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-16T15:29:38.164 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-16T15:29:38.164 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-16T15:29:38.164 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-16T15:29:38.164 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-16T15:29:38.164 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-16T15:29:38.164 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-16T15:29:38.164 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-16T15:29:38.164 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-16T15:29:38.164 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-16T15:29:38.165 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-16T15:29:38.165 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-16T15:29:38.165 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-16T15:29:38.165 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-16T15:29:38.165 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-09-16T15:29:38.165 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-16T15:29:38.165 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-16T15:29:38.165 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph 2024-09-16T15:29:38.165 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-16T15:29:38.165 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core 2024-09-16T15:29:38.165 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/failover.ko 2024-09-16T15:29:38.165 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-16T15:29:38.165 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp 2024-09-16T15:29:38.165 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-16T15:29:38.165 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-16T15:29:38.166 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-16T15:29:38.166 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-16T15:29:38.166 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver 2024-09-16T15:29:38.166 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver/dns_resolver.ko 2024-09-16T15:29:38.166 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa 2024-09-16T15:29:38.166 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-16T15:29:38.166 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-16T15:29:38.166 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-16T15:29:38.166 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-16T15:29:38.166 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-16T15:29:38.166 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154 2024-09-16T15:29:38.166 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan 2024-09-16T15:29:38.166 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-16T15:29:38.166 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-16T15:29:38.166 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-16T15:29:38.167 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4 2024-09-16T15:29:38.167 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-16T15:29:38.167 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-16T15:29:38.167 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-16T15:29:38.167 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-16T15:29:38.167 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-16T15:29:38.167 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-16T15:29:38.167 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-16T15:29:38.167 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-16T15:29:38.167 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-16T15:29:38.167 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-16T15:29:38.167 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter 2024-09-16T15:29:38.167 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-16T15:29:38.167 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-16T15:29:38.168 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-16T15:29:38.168 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-16T15:29:38.168 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-16T15:29:38.168 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-16T15:29:38.168 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-16T15:29:38.168 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-16T15:29:38.168 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-16T15:29:38.168 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-16T15:29:38.168 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-16T15:29:38.168 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-16T15:29:38.168 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-16T15:29:38.168 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-16T15:29:38.168 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-16T15:29:38.168 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-16T15:29:38.169 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-16T15:29:38.169 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-16T15:29:38.169 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-16T15:29:38.169 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-16T15:29:38.169 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-16T15:29:38.169 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-16T15:29:38.169 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-16T15:29:38.169 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-16T15:29:38.169 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-16T15:29:38.169 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-16T15:29:38.169 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-16T15:29:38.169 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-16T15:29:38.169 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-16T15:29:38.169 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-16T15:29:38.170 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-16T15:29:38.170 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-16T15:29:38.170 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-16T15:29:38.170 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-16T15:29:38.170 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-16T15:29:38.170 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-16T15:29:38.170 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-16T15:29:38.170 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-16T15:29:38.170 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6 2024-09-16T15:29:38.170 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-16T15:29:38.170 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-16T15:29:38.170 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-16T15:29:38.170 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-16T15:29:38.170 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-16T15:29:38.170 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-16T15:29:38.171 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-16T15:29:38.171 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter 2024-09-16T15:29:38.171 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-16T15:29:38.171 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-16T15:29:38.171 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-16T15:29:38.171 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-16T15:29:38.171 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-16T15:29:38.171 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-16T15:29:38.171 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-16T15:29:38.171 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-16T15:29:38.171 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-16T15:29:38.171 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-16T15:29:38.171 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-16T15:29:38.171 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-16T15:29:38.172 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-16T15:29:38.172 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-16T15:29:38.172 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-16T15:29:38.172 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-16T15:29:38.172 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-16T15:29:38.172 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-16T15:29:38.172 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-16T15:29:38.172 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-16T15:29:38.172 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-16T15:29:38.172 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-16T15:29:38.172 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-16T15:29:38.172 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-16T15:29:38.172 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-16T15:29:38.173 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-16T15:29:38.173 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-16T15:29:38.173 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key 2024-09-16T15:29:38.173 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-16T15:29:38.173 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp 2024-09-16T15:29:38.173 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-16T15:29:38.173 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-16T15:29:38.173 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-16T15:29:38.173 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-16T15:29:38.173 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-16T15:29:38.173 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-16T15:29:38.173 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-16T15:29:38.173 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc 2024-09-16T15:29:38.173 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-16T15:29:38.173 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154 2024-09-16T15:29:38.174 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-16T15:29:38.174 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls 2024-09-16T15:29:38.174 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-16T15:29:38.174 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter 2024-09-16T15:29:38.174 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset 2024-09-16T15:29:38.174 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-16T15:29:38.174 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-16T15:29:38.174 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-16T15:29:38.174 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-16T15:29:38.174 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-16T15:29:38.174 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-16T15:29:38.174 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-16T15:29:38.174 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-16T15:29:38.174 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-16T15:29:38.174 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-16T15:29:38.174 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-16T15:29:38.175 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-16T15:29:38.175 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-16T15:29:38.175 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-16T15:29:38.175 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-16T15:29:38.175 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-16T15:29:38.175 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs 2024-09-16T15:29:38.175 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-16T15:29:38.175 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-16T15:29:38.175 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-16T15:29:38.175 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-16T15:29:38.175 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-16T15:29:38.175 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-16T15:29:38.175 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-16T15:29:38.175 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-16T15:29:38.176 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-16T15:29:38.176 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-16T15:29:38.176 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-16T15:29:38.176 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-16T15:29:38.176 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-16T15:29:38.176 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-16T15:29:38.176 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-16T15:29:38.176 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-16T15:29:38.176 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-16T15:29:38.176 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-16T15:29:38.176 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-16T15:29:38.176 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-16T15:29:38.176 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-16T15:29:38.176 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-16T15:29:38.177 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-16T15:29:38.177 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-16T15:29:38.177 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-16T15:29:38.177 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-16T15:29:38.177 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-16T15:29:38.177 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-16T15:29:38.177 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-16T15:29:38.177 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-16T15:29:38.177 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-16T15:29:38.177 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-16T15:29:38.177 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-16T15:29:38.177 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-16T15:29:38.177 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-16T15:29:38.177 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-16T15:29:38.177 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-16T15:29:38.178 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-16T15:29:38.178 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-16T15:29:38.178 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-16T15:29:38.178 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-16T15:29:38.178 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-16T15:29:38.178 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-16T15:29:38.178 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-16T15:29:38.178 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-16T15:29:38.178 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-16T15:29:38.178 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-16T15:29:38.178 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-16T15:29:38.178 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-16T15:29:38.178 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-16T15:29:38.178 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-16T15:29:38.179 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-16T15:29:38.179 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-16T15:29:38.179 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-16T15:29:38.179 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-16T15:29:38.179 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-16T15:29:38.179 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-16T15:29:38.179 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-16T15:29:38.179 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_quota.ko 2024-09-16T15:29:38.179 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-16T15:29:38.179 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-16T15:29:38.179 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-16T15:29:38.179 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-16T15:29:38.179 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tunnel.ko 2024-09-16T15:29:38.179 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-16T15:29:38.179 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-16T15:29:38.180 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-16T15:29:38.180 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-16T15:29:38.180 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-16T15:29:38.180 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-16T15:29:38.180 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-16T15:29:38.180 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-16T15:29:38.180 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-16T15:29:38.180 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-16T15:29:38.180 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-16T15:29:38.180 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-16T15:29:38.180 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-16T15:29:38.180 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-16T15:29:38.180 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-16T15:29:38.180 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-16T15:29:38.181 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-16T15:29:38.181 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-16T15:29:38.181 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-16T15:29:38.181 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-16T15:29:38.181 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-16T15:29:38.181 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-16T15:29:38.181 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-16T15:29:38.181 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-16T15:29:38.181 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-16T15:29:38.181 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-16T15:29:38.181 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-16T15:29:38.181 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-16T15:29:38.181 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-16T15:29:38.181 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-16T15:29:38.182 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-16T15:29:38.182 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-16T15:29:38.182 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-16T15:29:38.182 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-16T15:29:38.182 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-16T15:29:38.182 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-16T15:29:38.182 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-16T15:29:38.182 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-16T15:29:38.182 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-16T15:29:38.182 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-16T15:29:38.182 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-16T15:29:38.182 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-16T15:29:38.182 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-16T15:29:38.182 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-16T15:29:38.183 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-16T15:29:38.183 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-16T15:29:38.183 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-16T15:29:38.183 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-16T15:29:38.183 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-16T15:29:38.183 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-16T15:29:38.183 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-16T15:29:38.183 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-16T15:29:38.183 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-16T15:29:38.183 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-16T15:29:38.183 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-16T15:29:38.183 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-16T15:29:38.183 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-16T15:29:38.183 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-16T15:29:38.183 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-16T15:29:38.183 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-16T15:29:38.184 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-16T15:29:38.184 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-16T15:29:38.184 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-16T15:29:38.184 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-16T15:29:38.184 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-16T15:29:38.184 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-16T15:29:38.184 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-16T15:29:38.184 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-16T15:29:38.184 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-16T15:29:38.184 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-16T15:29:38.184 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-16T15:29:38.184 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink 2024-09-16T15:29:38.184 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-16T15:29:38.184 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh 2024-09-16T15:29:38.184 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-16T15:29:38.185 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch 2024-09-16T15:29:38.185 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-16T15:29:38.185 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-16T15:29:38.185 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-16T15:29:38.185 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-16T15:29:38.185 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet 2024-09-16T15:29:38.185 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-16T15:29:38.185 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds 2024-09-16T15:29:38.185 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-16T15:29:38.185 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-16T15:29:38.185 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-16T15:29:38.185 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched 2024-09-16T15:29:38.185 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-16T15:29:38.185 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-16T15:29:38.185 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-16T15:29:38.186 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-16T15:29:38.186 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-16T15:29:38.186 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-16T15:29:38.186 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-16T15:29:38.186 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-16T15:29:38.186 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-16T15:29:38.186 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-16T15:29:38.186 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-16T15:29:38.186 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-16T15:29:38.186 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-16T15:29:38.186 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-16T15:29:38.186 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-16T15:29:38.186 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-16T15:29:38.186 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-16T15:29:38.186 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-16T15:29:38.187 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-16T15:29:38.187 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-16T15:29:38.187 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-16T15:29:38.187 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-16T15:29:38.187 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-16T15:29:38.187 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-16T15:29:38.187 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-16T15:29:38.187 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-16T15:29:38.187 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-16T15:29:38.187 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-16T15:29:38.187 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-16T15:29:38.187 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-16T15:29:38.187 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-16T15:29:38.187 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-16T15:29:38.188 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-16T15:29:38.188 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-16T15:29:38.188 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-16T15:29:38.188 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-16T15:29:38.188 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-16T15:29:38.188 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-16T15:29:38.188 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-16T15:29:38.188 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-16T15:29:38.188 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-16T15:29:38.188 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-16T15:29:38.188 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-16T15:29:38.188 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp 2024-09-16T15:29:38.188 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-16T15:29:38.189 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-16T15:29:38.189 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc 2024-09-16T15:29:38.189 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss 2024-09-16T15:29:38.189 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-16T15:29:38.189 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-16T15:29:38.189 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-16T15:29:38.189 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma 2024-09-16T15:29:38.189 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-16T15:29:38.189 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc 2024-09-16T15:29:38.189 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-16T15:29:38.189 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-16T15:29:38.189 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix 2024-09-16T15:29:38.189 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-16T15:29:38.189 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock 2024-09-16T15:29:38.189 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-16T15:29:38.190 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-16T15:29:38.190 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-16T15:29:38.190 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-16T15:29:38.190 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-16T15:29:38.190 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm 2024-09-16T15:29:38.190 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-16T15:29:38.190 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security 2024-09-16T15:29:38.190 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys 2024-09-16T15:29:38.190 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys 2024-09-16T15:29:38.190 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-09-16T15:29:38.190 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys 2024-09-16T15:29:38.190 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys/trusted.ko 2024-09-16T15:29:38.190 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-16T15:29:38.190 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-16T15:29:38.191 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-16T15:29:38.191 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-16T15:29:38.191 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-16T15:29:38.191 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-16T15:29:38.191 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-16T15:29:38.191 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-16T15:29:38.191 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-16T15:29:38.191 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-16T15:29:38.191 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-16T15:29:38.191 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-16T15:29:38.191 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-16T15:29:38.191 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.weakdep 2024-09-16T15:29:38.191 INFO:teuthology.orchestra.run.smithi151.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-16T15:29:38.195 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-16T15:29:38.195 DEBUG:teuthology.orchestra.run.smithi151:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-09-16T15:29:38.218 INFO:teuthology.orchestra.run.smithi151.stdout:postinstall scriptlet (using /bin/sh): 2024-09-16T15:29:38.219 INFO:teuthology.orchestra.run.smithi151.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-16T15:29:38.219 INFO:teuthology.orchestra.run.smithi151.stdout: /usr/bin/kernel-install add 6.11.0-rc5-g2ca44a2f35ca /lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-16T15:29:38.219 INFO:teuthology.orchestra.run.smithi151.stdout:fi 2024-09-16T15:29:38.219 INFO:teuthology.orchestra.run.smithi151.stdout:for file in vmlinuz System.map config; do 2024-09-16T15:29:38.219 INFO:teuthology.orchestra.run.smithi151.stdout: if ! cmp --silent "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca"; then 2024-09-16T15:29:38.219 INFO:teuthology.orchestra.run.smithi151.stdout: cp "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca" 2024-09-16T15:29:38.219 INFO:teuthology.orchestra.run.smithi151.stdout: fi 2024-09-16T15:29:38.219 INFO:teuthology.orchestra.run.smithi151.stdout:done 2024-09-16T15:29:38.219 INFO:teuthology.orchestra.run.smithi151.stdout:if [ -d "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" ] && \ 2024-09-16T15:29:38.219 INFO:teuthology.orchestra.run.smithi151.stdout: ! diff -rq "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" >/dev/null 2>&1; then 2024-09-16T15:29:38.219 INFO:teuthology.orchestra.run.smithi151.stdout: rm -rf "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-16T15:29:38.219 INFO:teuthology.orchestra.run.smithi151.stdout: cp -r "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-16T15:29:38.219 INFO:teuthology.orchestra.run.smithi151.stdout:fi 2024-09-16T15:29:38.219 INFO:teuthology.orchestra.run.smithi151.stdout:if [ ! -e "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep" ]; then 2024-09-16T15:29:38.220 INFO:teuthology.orchestra.run.smithi151.stdout: /usr/sbin/depmod 6.11.0-rc5-g2ca44a2f35ca 2024-09-16T15:29:38.220 INFO:teuthology.orchestra.run.smithi151.stdout:fi 2024-09-16T15:29:38.220 INFO:teuthology.orchestra.run.smithi151.stdout:preuninstall scriptlet (using /bin/sh): 2024-09-16T15:29:38.220 INFO:teuthology.orchestra.run.smithi151.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-16T15:29:38.220 INFO:teuthology.orchestra.run.smithi151.stdout:kernel-install remove 6.11.0-rc5-g2ca44a2f35ca 2024-09-16T15:29:38.220 INFO:teuthology.orchestra.run.smithi151.stdout:fi 2024-09-16T15:29:38.221 DEBUG:teuthology.orchestra.run.smithi151:> sudo rpm -qi grub2-tools 2024-09-16T15:29:38.294 INFO:teuthology.orchestra.run.smithi151.stdout:Name : grub2-tools 2024-09-16T15:29:38.294 INFO:teuthology.orchestra.run.smithi151.stdout:Epoch : 1 2024-09-16T15:29:38.294 INFO:teuthology.orchestra.run.smithi151.stdout:Version : 2.06 2024-09-16T15:29:38.294 INFO:teuthology.orchestra.run.smithi151.stdout:Release : 82.el9 2024-09-16T15:29:38.295 INFO:teuthology.orchestra.run.smithi151.stdout:Architecture: x86_64 2024-09-16T15:29:38.295 INFO:teuthology.orchestra.run.smithi151.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-16T15:29:38.295 INFO:teuthology.orchestra.run.smithi151.stdout:Group : Unspecified 2024-09-16T15:29:38.295 INFO:teuthology.orchestra.run.smithi151.stdout:Size : 8274670 2024-09-16T15:29:38.295 INFO:teuthology.orchestra.run.smithi151.stdout:License : GPLv3+ 2024-09-16T15:29:38.295 INFO:teuthology.orchestra.run.smithi151.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-16T15:29:38.295 INFO:teuthology.orchestra.run.smithi151.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-16T15:29:38.295 INFO:teuthology.orchestra.run.smithi151.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-16T15:29:38.295 INFO:teuthology.orchestra.run.smithi151.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-16T15:29:38.295 INFO:teuthology.orchestra.run.smithi151.stdout:Packager : builder@centos.org 2024-09-16T15:29:38.295 INFO:teuthology.orchestra.run.smithi151.stdout:Vendor : CentOS 2024-09-16T15:29:38.295 INFO:teuthology.orchestra.run.smithi151.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-16T15:29:38.295 INFO:teuthology.orchestra.run.smithi151.stdout:Summary : Support tools for GRUB. 2024-09-16T15:29:38.295 INFO:teuthology.orchestra.run.smithi151.stdout:Description : 2024-09-16T15:29:38.295 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:29:38.296 INFO:teuthology.orchestra.run.smithi151.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-16T15:29:38.296 INFO:teuthology.orchestra.run.smithi151.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-16T15:29:38.296 INFO:teuthology.orchestra.run.smithi151.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-16T15:29:38.296 INFO:teuthology.orchestra.run.smithi151.stdout:hardware devices. 2024-09-16T15:29:38.296 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:29:38.296 INFO:teuthology.orchestra.run.smithi151.stdout:This subpackage provides tools for support of all platforms. 2024-09-16T15:29:38.297 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-16T15:29:38.297 INFO:teuthology.task.kernel:Updating grub on smithi151 to boot 6.11.0-rc5-g2ca44a2f35ca 2024-09-16T15:29:38.297 DEBUG:teuthology.orchestra.run.smithi151:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-16T15:29:38.881 INFO:teuthology.orchestra.run.smithi151.stderr:Generating grub configuration file ... 2024-09-16T15:29:40.185 INFO:teuthology.orchestra.run.smithi151.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-16T15:29:40.212 INFO:teuthology.orchestra.run.smithi151.stderr:done 2024-09-16T15:29:40.214 DEBUG:teuthology.orchestra.run.smithi151:> mktemp 2024-09-16T15:29:40.230 INFO:teuthology.orchestra.run.smithi151.stdout:/tmp/tmp.OkTF0WOnOe 2024-09-16T15:29:40.230 DEBUG:teuthology.orchestra.run.smithi151:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.OkTF0WOnOe 2024-09-16T15:29:40.297 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod 0666 /tmp/tmp.OkTF0WOnOe 2024-09-16T15:29:40.450 DEBUG:teuthology.orchestra.remote:smithi151:/tmp/tmp.OkTF0WOnOe is 6KB 2024-09-16T15:29:40.470 DEBUG:teuthology.orchestra.run.smithi151:> rm -fr /tmp/tmp.OkTF0WOnOe 2024-09-16T15:29:40.485 DEBUG:teuthology.orchestra.run.smithi151:> sudo /bin/ls /boot/loader/entries || true 2024-09-16T15:29:40.554 INFO:teuthology.orchestra.run.smithi151.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-16T15:29:40.554 INFO:teuthology.orchestra.run.smithi151.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-16T15:29:40.554 INFO:teuthology.orchestra.run.smithi151.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-16T15:29:40.554 INFO:teuthology.orchestra.run.smithi151.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca.conf 2024-09-16T15:29:40.555 DEBUG:teuthology.orchestra.run.smithi151:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca 2024-09-16T15:29:40.721 DEBUG:teuthology.orchestra.run.smithi151:> sudo shutdown -r now 2024-09-16T15:30:07.823 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-09-16T15:30:07.824 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-09-16T15:30:07.824 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T15:30:10.725 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-16T15:30:10.725 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi151.front.sepia.ceph.com' 2024-09-16T15:30:10.725 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T15:30:42.365 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.167 2024-09-16T15:30:45.181 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.151 2024-09-16T15:30:51.372 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-09-16T15:30:51.373 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T15:30:54.185 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi151.front.sepia.ceph.com' 2024-09-16T15:30:54.185 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T15:31:09.757 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.167 2024-09-16T15:31:12.573 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.151 2024-09-16T15:31:21.765 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-09-16T15:31:21.766 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T15:31:23.932 DEBUG:teuthology.orchestra.run.smithi167:> true 2024-09-16T15:31:24.463 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-09-16T15:31:24.463 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-16T15:31:24.463 DEBUG:teuthology.orchestra.run.smithi167:> uname -r 2024-09-16T15:31:24.498 INFO:teuthology.orchestra.run.smithi167.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-16T15:31:24.498 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-16T15:31:24.498 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-16T15:31:24.498 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-16T15:31:24.574 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi151.front.sepia.ceph.com' 2024-09-16T15:31:24.574 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T15:31:25.500 DEBUG:teuthology.parallel:result is None 2024-09-16T15:31:27.645 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.151 2024-09-16T15:31:42.646 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi151.front.sepia.ceph.com' 2024-09-16T15:31:42.646 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T15:31:42.988 DEBUG:teuthology.orchestra.run.smithi151:> true 2024-09-16T15:31:43.722 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi151.front.sepia.ceph.com' 2024-09-16T15:31:43.722 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-16T15:31:43.722 DEBUG:teuthology.orchestra.run.smithi151:> uname -r 2024-09-16T15:31:43.757 INFO:teuthology.orchestra.run.smithi151.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-16T15:31:43.757 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-16T15:31:43.758 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-16T15:31:43.758 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-16T15:31:44.760 DEBUG:teuthology.parallel:result is None 2024-09-16T15:31:44.760 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-16T15:31:44.768 INFO:teuthology.task.internal:Creating test directory... 2024-09-16T15:31:44.769 DEBUG:teuthology.orchestra.run.smithi151:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T15:31:44.773 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T15:31:44.813 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-16T15:31:44.842 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-16T15:31:44.881 INFO:teuthology.task.internal:Creating archive directory... 2024-09-16T15:31:44.882 DEBUG:teuthology.orchestra.run.smithi151:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T15:31:44.886 DEBUG:teuthology.orchestra.run.smithi167:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T15:31:45.140 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-16T15:31:45.146 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-16T15:31:45.147 DEBUG:teuthology.orchestra.run.smithi151:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T15:31:45.151 DEBUG:teuthology.orchestra.run.smithi167:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T15:31:45.248 INFO:teuthology.orchestra.run.smithi151.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T15:31:45.294 INFO:teuthology.orchestra.run.smithi167.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T15:31:45.320 INFO:teuthology.orchestra.run.smithi167.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T15:31:45.425 INFO:teuthology.orchestra.run.smithi151.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T15:31:45.428 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-16T15:31:45.435 INFO:teuthology.task.internal:Configuring sudo... 2024-09-16T15:31:45.436 DEBUG:teuthology.orchestra.run.smithi151:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T15:31:45.440 DEBUG:teuthology.orchestra.run.smithi167:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T15:31:45.496 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-16T15:31:45.506 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-16T15:31:45.507 DEBUG:teuthology.orchestra.run.smithi151:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T15:31:45.511 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T15:31:45.545 DEBUG:teuthology.orchestra.run.smithi151:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T15:31:45.649 DEBUG:teuthology.orchestra.run.smithi151:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T15:31:45.738 DEBUG:teuthology.orchestra.run.smithi151:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T15:31:45.802 DEBUG:teuthology.orchestra.run.smithi151:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T15:31:45.873 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:31:45.873 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T15:31:45.927 DEBUG:teuthology.orchestra.run.smithi167:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T15:31:45.992 DEBUG:teuthology.orchestra.run.smithi167:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T15:31:46.080 DEBUG:teuthology.orchestra.run.smithi167:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T15:31:46.143 DEBUG:teuthology.orchestra.run.smithi167:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T15:31:46.213 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-16T15:31:46.213 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T15:31:46.268 DEBUG:teuthology.orchestra.run.smithi151:> sudo service rsyslog restart 2024-09-16T15:31:46.272 DEBUG:teuthology.orchestra.run.smithi167:> sudo service rsyslog restart 2024-09-16T15:31:46.344 INFO:teuthology.orchestra.run.smithi151.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T15:31:46.393 INFO:teuthology.orchestra.run.smithi167.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T15:31:46.909 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-16T15:31:46.917 INFO:teuthology.task.internal:Starting timer... 2024-09-16T15:31:46.917 INFO:teuthology.run_tasks:Running task pcp... 2024-09-16T15:31:46.995 INFO:teuthology.run_tasks:Running task selinux... 2024-09-16T15:31:47.005 DEBUG:teuthology.orchestra.run.smithi151:> sudo service auditd rotate 2024-09-16T15:31:47.085 INFO:teuthology.orchestra.run.smithi151.stdout:Rotating logs: 2024-09-16T15:31:47.088 DEBUG:teuthology.orchestra.run.smithi167:> sudo service auditd rotate 2024-09-16T15:31:47.214 INFO:teuthology.orchestra.run.smithi167.stdout:Rotating logs: 2024-09-16T15:31:47.217 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-16T15:31:47.217 DEBUG:teuthology.orchestra.run.smithi151:> /usr/sbin/getenforce 2024-09-16T15:31:47.264 INFO:teuthology.orchestra.run.smithi151.stdout:Permissive 2024-09-16T15:31:47.264 DEBUG:teuthology.orchestra.run.smithi167:> /usr/sbin/getenforce 2024-09-16T15:31:47.305 INFO:teuthology.orchestra.run.smithi167.stdout:Permissive 2024-09-16T15:31:47.306 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi151.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi167.front.sepia.ceph.com': 'permissive'} 2024-09-16T15:31:47.306 DEBUG:teuthology.orchestra.run.smithi151:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-16T15:31:47.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T15:31:47.363 DEBUG:teuthology.orchestra.run.smithi167:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-16T15:31:47.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T15:31:47.419 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-16T15:31:47.420 DEBUG:teuthology.orchestra.run.smithi151:> sudo /usr/sbin/setenforce permissive 2024-09-16T15:31:47.495 DEBUG:teuthology.orchestra.run.smithi167:> sudo /usr/sbin/setenforce permissive 2024-09-16T15:31:47.562 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-16T15:31:47.572 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-16T15:31:47.583 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-16T15:31:47.584 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi151.front.sepia.ceph.com,smithi167.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-16T15:36:26.594 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi151.front.sepia.ceph.com'), Remote(name='ubuntu@smithi167.front.sepia.ceph.com')] 2024-09-16T15:36:26.595 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi151.front.sepia.ceph.com' 2024-09-16T15:36:26.597 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T15:36:26.686 DEBUG:teuthology.orchestra.run.smithi151:> true 2024-09-16T15:36:26.785 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi151.front.sepia.ceph.com' 2024-09-16T15:36:26.786 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-09-16T15:36:26.786 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T15:36:26.872 DEBUG:teuthology.orchestra.run.smithi167:> true 2024-09-16T15:36:26.982 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-09-16T15:36:26.983 INFO:teuthology.run_tasks:Running task clock... 2024-09-16T15:36:26.995 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-16T15:36:26.995 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T15:36:26.995 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T15:36:27.000 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T15:36:27.000 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T15:36:27.056 INFO:teuthology.orchestra.run.smithi151.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-16T15:36:27.083 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-16T15:36:27.085 INFO:teuthology.orchestra.run.smithi151.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-16T15:36:27.113 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-16T15:36:27.137 INFO:teuthology.orchestra.run.smithi151.stderr:sudo: ntpd: command not found 2024-09-16T15:36:27.161 INFO:teuthology.orchestra.run.smithi151.stdout:506 Cannot talk to daemon 2024-09-16T15:36:27.165 INFO:teuthology.orchestra.run.smithi167.stderr:sudo: ntpd: command not found 2024-09-16T15:36:27.189 INFO:teuthology.orchestra.run.smithi167.stdout:506 Cannot talk to daemon 2024-09-16T15:36:27.190 INFO:teuthology.orchestra.run.smithi151.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-16T15:36:27.218 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-16T15:36:27.220 INFO:teuthology.orchestra.run.smithi151.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-16T15:36:27.248 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-16T15:36:27.311 INFO:teuthology.orchestra.run.smithi151.stderr:bash: line 1: ntpq: command not found 2024-09-16T15:36:27.318 INFO:teuthology.orchestra.run.smithi151.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T15:36:27.318 INFO:teuthology.orchestra.run.smithi151.stdout:=============================================================================== 2024-09-16T15:36:27.318 INFO:teuthology.orchestra.run.smithi151.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T15:36:27.318 INFO:teuthology.orchestra.run.smithi151.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T15:36:27.318 INFO:teuthology.orchestra.run.smithi151.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T15:36:27.318 INFO:teuthology.orchestra.run.smithi151.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T15:36:27.357 INFO:teuthology.orchestra.run.smithi167.stderr:bash: line 1: ntpq: command not found 2024-09-16T15:36:27.365 INFO:teuthology.orchestra.run.smithi167.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T15:36:27.365 INFO:teuthology.orchestra.run.smithi167.stdout:=============================================================================== 2024-09-16T15:36:27.365 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T15:36:27.365 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T15:36:27.365 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T15:36:27.365 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T15:36:27.366 INFO:teuthology.run_tasks:Running task install... 2024-09-16T15:36:27.376 DEBUG:teuthology.task.install:project ceph 2024-09-16T15:36:27.376 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '33f95d2ae1bcbe7eb04404303f179eeeaa6982df'}} 2024-09-16T15:36:27.376 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '33f95d2ae1bcbe7eb04404303f179eeeaa6982df'} 2024-09-16T15:36:27.376 INFO:teuthology.task.install:Using flavor: default 2024-09-16T15:36:27.381 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-16T15:36:27.381 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-16T15:36:27.382 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '33f95d2ae1bcbe7eb04404303f179eeeaa6982df', 'tag': None, 'wait_for_package': False} 2024-09-16T15:36:27.382 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=33f95d2ae1bcbe7eb04404303f179eeeaa6982df 2024-09-16T15:36:27.384 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '33f95d2ae1bcbe7eb04404303f179eeeaa6982df', 'tag': None, 'wait_for_package': False} 2024-09-16T15:36:27.384 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=33f95d2ae1bcbe7eb04404303f179eeeaa6982df 2024-09-16T15:36:27.527 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-1454/33f95d2ae1bcbe7eb04404303f179eeeaa6982df/centos/9/flavors/default/ 2024-09-16T15:36:27.527 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4938.g33f95d2a 2024-09-16T15:36:27.530 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-1454/33f95d2ae1bcbe7eb04404303f179eeeaa6982df/centos/9/flavors/default/ 2024-09-16T15:36:27.530 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4938.g33f95d2a 2024-09-16T15:36:27.639 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-1454/33f95d2ae1bcbe7eb04404303f179eeeaa6982df/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-1454/33f95d2ae1bcbe7eb04404303f179eeeaa6982df/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-1454/33f95d2ae1bcbe7eb04404303f179eeeaa6982df/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-16T15:36:27.639 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-16T15:36:27.639 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-16T15:36:27.646 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-1454/33f95d2ae1bcbe7eb04404303f179eeeaa6982df/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-1454/33f95d2ae1bcbe7eb04404303f179eeeaa6982df/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-16-1454/33f95d2ae1bcbe7eb04404303f179eeeaa6982df/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-16T15:36:27.646 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:36:27.646 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-16T15:36:27.695 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-16T15:36:27.695 DEBUG:teuthology.orchestra.run.smithi167:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/33f95d2ae1bcbe7eb04404303f179eeeaa6982df/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-16T15:36:27.702 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-16T15:36:27.702 DEBUG:teuthology.orchestra.run.smithi151:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/33f95d2ae1bcbe7eb04404303f179eeeaa6982df/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-16T15:36:27.792 DEBUG:teuthology.orchestra.run.smithi167:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-16T15:36:27.799 DEBUG:teuthology.orchestra.run.smithi151:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-16T15:36:27.870 DEBUG:teuthology.orchestra.run.smithi167:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-16T15:36:27.877 DEBUG:teuthology.orchestra.run.smithi151:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-16T15:36:27.925 INFO:teuthology.orchestra.run.smithi167.stdout:check_obsoletes = 1 2024-09-16T15:36:27.929 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum clean all 2024-09-16T15:36:27.932 INFO:teuthology.orchestra.run.smithi151.stdout:check_obsoletes = 1 2024-09-16T15:36:27.935 DEBUG:teuthology.orchestra.run.smithi151:> sudo yum clean all 2024-09-16T15:36:28.193 INFO:teuthology.orchestra.run.smithi167.stdout:45 files removed 2024-09-16T15:36:28.211 INFO:teuthology.orchestra.run.smithi151.stdout:45 files removed 2024-09-16T15:36:28.226 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-16T15:36:28.244 DEBUG:teuthology.orchestra.run.smithi151:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-16T15:36:28.910 INFO:teuthology.orchestra.run.smithi167.stdout:ceph packages for x86_64 295 kB/s | 84 kB 00:00 2024-09-16T15:36:28.965 INFO:teuthology.orchestra.run.smithi151.stdout:ceph packages for x86_64 267 kB/s | 84 kB 00:00 2024-09-16T15:36:29.228 INFO:teuthology.orchestra.run.smithi167.stdout:ceph noarch packages 88 kB/s | 24 kB 00:00 2024-09-16T15:36:29.273 INFO:teuthology.orchestra.run.smithi151.stdout:ceph noarch packages 93 kB/s | 24 kB 00:00 2024-09-16T15:36:29.495 INFO:teuthology.orchestra.run.smithi167.stdout:ceph source packages 8.5 kB/s | 1.9 kB 00:00 2024-09-16T15:36:29.566 INFO:teuthology.orchestra.run.smithi151.stdout:ceph source packages 7.6 kB/s | 1.9 kB 00:00 2024-09-16T15:36:30.245 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.2 MB 00:00 2024-09-16T15:36:30.275 INFO:teuthology.orchestra.run.smithi151.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.2 MB 00:00 2024-09-16T15:36:32.188 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - AppStream 25 MB/s | 20 MB 00:00 2024-09-16T15:36:32.502 INFO:teuthology.orchestra.run.smithi151.stdout:CentOS Stream 9 - AppStream 19 MB/s | 20 MB 00:01 2024-09-16T15:36:37.365 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - CRB 5.6 MB/s | 6.5 MB 00:01 2024-09-16T15:36:38.786 INFO:teuthology.orchestra.run.smithi151.stdout:CentOS Stream 9 - CRB 2.9 MB/s | 6.5 MB 00:02 2024-09-16T15:36:38.874 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - Extras packages 98 kB/s | 19 kB 00:00 2024-09-16T15:36:40.220 INFO:teuthology.orchestra.run.smithi167.stdout:Extra Packages for Enterprise Linux 19 MB/s | 23 MB 00:01 2024-09-16T15:36:40.412 INFO:teuthology.orchestra.run.smithi151.stdout:CentOS Stream 9 - Extras packages 58 kB/s | 19 kB 00:00 2024-09-16T15:36:41.808 INFO:teuthology.orchestra.run.smithi151.stdout:Extra Packages for Enterprise Linux 18 MB/s | 23 MB 00:01 2024-09-16T15:36:46.509 INFO:teuthology.orchestra.run.smithi167.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-16T15:36:48.235 INFO:teuthology.orchestra.run.smithi151.stdout:lab-extras 17 kB/s | 1.7 kB 00:00 2024-09-16T15:36:49.024 INFO:teuthology.orchestra.run.smithi167.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-16T15:36:49.025 INFO:teuthology.orchestra.run.smithi167.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-16T15:36:49.038 INFO:teuthology.orchestra.run.smithi167.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-16T15:36:49.039 INFO:teuthology.orchestra.run.smithi167.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-16T15:36:49.041 INFO:teuthology.orchestra.run.smithi167.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-16T15:36:49.041 INFO:teuthology.orchestra.run.smithi167.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-16T15:36:49.043 INFO:teuthology.orchestra.run.smithi167.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-16T15:36:49.046 INFO:teuthology.orchestra.run.smithi167.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-16T15:36:49.047 INFO:teuthology.orchestra.run.smithi167.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-16T15:36:49.114 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2024-09-16T15:36:49.133 INFO:teuthology.orchestra.run.smithi167.stdout:======================================================================================= 2024-09-16T15:36:49.133 INFO:teuthology.orchestra.run.smithi167.stdout: Package Arch Version Repository Size 2024-09-16T15:36:49.133 INFO:teuthology.orchestra.run.smithi167.stdout:======================================================================================= 2024-09-16T15:36:49.133 INFO:teuthology.orchestra.run.smithi167.stdout:Installing: 2024-09-16T15:36:49.134 INFO:teuthology.orchestra.run.smithi167.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-16T15:36:49.134 INFO:teuthology.orchestra.run.smithi167.stdout: ceph x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 6.6 k 2024-09-16T15:36:49.134 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-base x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 5.5 M 2024-09-16T15:36:49.134 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-fuse x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 900 k 2024-09-16T15:36:49.134 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 145 k 2024-09-16T15:36:49.134 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 1.5 M 2024-09-16T15:36:49.134 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4938.g33f95d2a.el9 ceph-noarch 154 k 2024-09-16T15:36:49.134 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4938.g33f95d2a.el9 ceph-noarch 14 M 2024-09-16T15:36:49.134 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4938.g33f95d2a.el9 ceph-noarch 7.4 M 2024-09-16T15:36:49.134 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-rook noarch 2:19.3.0-4938.g33f95d2a.el9 ceph-noarch 49 k 2024-09-16T15:36:49.134 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-radosgw x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 11 M 2024-09-16T15:36:49.134 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-test x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 46 M 2024-09-16T15:36:49.134 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-volume noarch 2:19.3.0-4938.g33f95d2a.el9 ceph-noarch 285 k 2024-09-16T15:36:49.134 INFO:teuthology.orchestra.run.smithi167.stdout: cephadm noarch 2:19.3.0-4938.g33f95d2a.el9 ceph-noarch 784 k 2024-09-16T15:36:49.135 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-mirror x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 225 k 2024-09-16T15:36:49.135 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-shell x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 27 k 2024-09-16T15:36:49.135 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-top noarch 2:19.3.0-4938.g33f95d2a.el9 ceph-noarch 18 k 2024-09-16T15:36:49.135 INFO:teuthology.orchestra.run.smithi167.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-16T15:36:49.135 INFO:teuthology.orchestra.run.smithi167.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-16T15:36:49.135 INFO:teuthology.orchestra.run.smithi167.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-16T15:36:49.135 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs-devel x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 32 k 2024-09-16T15:36:49.135 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs2 x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 756 k 2024-09-16T15:36:49.135 INFO:teuthology.orchestra.run.smithi167.stdout: librados-devel x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 127 k 2024-09-16T15:36:49.135 INFO:teuthology.orchestra.run.smithi167.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-16T15:36:49.135 INFO:teuthology.orchestra.run.smithi167.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-16T15:36:49.135 INFO:teuthology.orchestra.run.smithi167.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-16T15:36:49.135 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-16T15:36:49.135 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-16T15:36:49.135 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-16T15:36:49.136 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cephfs x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 161 k 2024-09-16T15:36:49.136 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rados x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 321 k 2024-09-16T15:36:49.136 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rbd x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 306 k 2024-09-16T15:36:49.136 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rgw x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 99 k 2024-09-16T15:36:49.136 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-fuse x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 85 k 2024-09-16T15:36:49.136 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-mirror x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 3.2 M 2024-09-16T15:36:49.136 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-nbd x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 170 k 2024-09-16T15:36:49.136 INFO:teuthology.orchestra.run.smithi167.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-16T15:36:49.136 INFO:teuthology.orchestra.run.smithi167.stdout:Upgrading: 2024-09-16T15:36:49.136 INFO:teuthology.orchestra.run.smithi167.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-16T15:36:49.136 INFO:teuthology.orchestra.run.smithi167.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-16T15:36:49.136 INFO:teuthology.orchestra.run.smithi167.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-16T15:36:49.136 INFO:teuthology.orchestra.run.smithi167.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-16T15:36:49.137 INFO:teuthology.orchestra.run.smithi167.stdout: librados2 x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 3.5 M 2024-09-16T15:36:49.137 INFO:teuthology.orchestra.run.smithi167.stdout: librbd1 x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 3.2 M 2024-09-16T15:36:49.137 INFO:teuthology.orchestra.run.smithi167.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-16T15:36:49.137 INFO:teuthology.orchestra.run.smithi167.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-16T15:36:49.137 INFO:teuthology.orchestra.run.smithi167.stdout:Installing dependencies: 2024-09-16T15:36:49.137 INFO:teuthology.orchestra.run.smithi167.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-16T15:36:49.137 INFO:teuthology.orchestra.run.smithi167.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-16T15:36:49.137 INFO:teuthology.orchestra.run.smithi167.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-16T15:36:49.137 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-common x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 22 M 2024-09-16T15:36:49.137 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4938.g33f95d2a.el9 ceph-noarch 34 k 2024-09-16T15:36:49.137 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mds x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 2.4 M 2024-09-16T15:36:49.137 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4938.g33f95d2a.el9 ceph-noarch 291 k 2024-09-16T15:36:49.137 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mon x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 4.8 M 2024-09-16T15:36:49.138 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-osd x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 17 M 2024-09-16T15:36:49.138 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4938.g33f95d2a.el9 ceph-noarch 16 k 2024-09-16T15:36:49.138 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-selinux x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 25 k 2024-09-16T15:36:49.138 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-16T15:36:49.138 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-16T15:36:49.138 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-16T15:36:49.138 INFO:teuthology.orchestra.run.smithi167.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-16T15:36:49.138 INFO:teuthology.orchestra.run.smithi167.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-16T15:36:49.138 INFO:teuthology.orchestra.run.smithi167.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-16T15:36:49.138 INFO:teuthology.orchestra.run.smithi167.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-16T15:36:49.138 INFO:teuthology.orchestra.run.smithi167.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-16T15:36:49.138 INFO:teuthology.orchestra.run.smithi167.stdout: libcephsqlite x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 161 k 2024-09-16T15:36:49.138 INFO:teuthology.orchestra.run.smithi167.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-16T15:36:49.138 INFO:teuthology.orchestra.run.smithi167.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-16T15:36:49.138 INFO:teuthology.orchestra.run.smithi167.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-16T15:36:49.139 INFO:teuthology.orchestra.run.smithi167.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-16T15:36:49.139 INFO:teuthology.orchestra.run.smithi167.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-16T15:36:49.139 INFO:teuthology.orchestra.run.smithi167.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-16T15:36:49.139 INFO:teuthology.orchestra.run.smithi167.stdout: libradosstriper1 x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 508 k 2024-09-16T15:36:49.139 INFO:teuthology.orchestra.run.smithi167.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-16T15:36:49.139 INFO:teuthology.orchestra.run.smithi167.stdout: librgw2 x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 5.5 M 2024-09-16T15:36:49.139 INFO:teuthology.orchestra.run.smithi167.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-16T15:36:49.139 INFO:teuthology.orchestra.run.smithi167.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-16T15:36:49.139 INFO:teuthology.orchestra.run.smithi167.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-16T15:36:49.139 INFO:teuthology.orchestra.run.smithi167.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-16T15:36:49.139 INFO:teuthology.orchestra.run.smithi167.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-16T15:36:49.139 INFO:teuthology.orchestra.run.smithi167.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-16T15:36:49.139 INFO:teuthology.orchestra.run.smithi167.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-16T15:36:49.139 INFO:teuthology.orchestra.run.smithi167.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-16T15:36:49.139 INFO:teuthology.orchestra.run.smithi167.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-16T15:36:49.140 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-16T15:36:49.140 INFO:teuthology.orchestra.run.smithi167.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-16T15:36:49.140 INFO:teuthology.orchestra.run.smithi167.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-16T15:36:49.140 INFO:teuthology.orchestra.run.smithi167.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-16T15:36:49.140 INFO:teuthology.orchestra.run.smithi167.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-16T15:36:49.140 INFO:teuthology.orchestra.run.smithi167.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-16T15:36:49.140 INFO:teuthology.orchestra.run.smithi167.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-16T15:36:49.140 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-16T15:36:49.140 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-argparse x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 45 k 2024-09-16T15:36:49.140 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-common x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 148 k 2024-09-16T15:36:49.140 INFO:teuthology.orchestra.run.smithi167.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-16T15:36:49.140 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-16T15:36:49.140 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-16T15:36:49.140 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-16T15:36:49.141 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-16T15:36:49.141 INFO:teuthology.orchestra.run.smithi167.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-16T15:36:49.141 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-16T15:36:49.141 INFO:teuthology.orchestra.run.smithi167.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-16T15:36:49.141 INFO:teuthology.orchestra.run.smithi167.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-16T15:36:49.141 INFO:teuthology.orchestra.run.smithi167.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-16T15:36:49.141 INFO:teuthology.orchestra.run.smithi167.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-16T15:36:49.141 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-16T15:36:49.141 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-16T15:36:49.141 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-16T15:36:49.141 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-16T15:36:49.141 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-16T15:36:49.141 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-16T15:36:49.141 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-16T15:36:49.141 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-16T15:36:49.142 INFO:teuthology.orchestra.run.smithi167.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-16T15:36:49.142 INFO:teuthology.orchestra.run.smithi167.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-16T15:36:49.142 INFO:teuthology.orchestra.run.smithi167.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-16T15:36:49.142 INFO:teuthology.orchestra.run.smithi167.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-16T15:36:49.142 INFO:teuthology.orchestra.run.smithi167.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-16T15:36:49.142 INFO:teuthology.orchestra.run.smithi167.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-16T15:36:49.142 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-16T15:36:49.142 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-16T15:36:49.142 INFO:teuthology.orchestra.run.smithi167.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-16T15:36:49.142 INFO:teuthology.orchestra.run.smithi167.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-16T15:36:49.142 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-16T15:36:49.142 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-16T15:36:49.142 INFO:teuthology.orchestra.run.smithi167.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-16T15:36:49.142 INFO:teuthology.orchestra.run.smithi167.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-16T15:36:49.142 INFO:teuthology.orchestra.run.smithi167.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-16T15:36:49.143 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-16T15:36:49.143 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-16T15:36:49.143 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-16T15:36:49.143 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-16T15:36:49.143 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-16T15:36:49.143 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-16T15:36:49.143 INFO:teuthology.orchestra.run.smithi167.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-16T15:36:49.143 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-16T15:36:49.143 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-16T15:36:49.143 INFO:teuthology.orchestra.run.smithi167.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-16T15:36:49.143 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-16T15:36:49.143 INFO:teuthology.orchestra.run.smithi167.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-16T15:36:49.143 INFO:teuthology.orchestra.run.smithi167.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-16T15:36:49.143 INFO:teuthology.orchestra.run.smithi167.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-16T15:36:49.143 INFO:teuthology.orchestra.run.smithi167.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-16T15:36:49.144 INFO:teuthology.orchestra.run.smithi167.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-16T15:36:49.144 INFO:teuthology.orchestra.run.smithi167.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-16T15:36:49.144 INFO:teuthology.orchestra.run.smithi167.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-16T15:36:49.144 INFO:teuthology.orchestra.run.smithi167.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-16T15:36:49.144 INFO:teuthology.orchestra.run.smithi167.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-16T15:36:49.144 INFO:teuthology.orchestra.run.smithi167.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-16T15:36:49.144 INFO:teuthology.orchestra.run.smithi167.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-16T15:36:49.144 INFO:teuthology.orchestra.run.smithi167.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-16T15:36:49.144 INFO:teuthology.orchestra.run.smithi167.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-16T15:36:49.144 INFO:teuthology.orchestra.run.smithi167.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-16T15:36:49.144 INFO:teuthology.orchestra.run.smithi167.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-16T15:36:49.144 INFO:teuthology.orchestra.run.smithi167.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-16T15:36:49.144 INFO:teuthology.orchestra.run.smithi167.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-16T15:36:49.144 INFO:teuthology.orchestra.run.smithi167.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-16T15:36:49.144 INFO:teuthology.orchestra.run.smithi167.stdout:Installing weak dependencies: 2024-09-16T15:36:49.145 INFO:teuthology.orchestra.run.smithi167.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-16T15:36:49.145 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-16T15:36:49.145 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2024-09-16T15:36:49.145 INFO:teuthology.orchestra.run.smithi167.stdout:======================================================================================= 2024-09-16T15:36:49.145 INFO:teuthology.orchestra.run.smithi167.stdout:Install 149 Packages 2024-09-16T15:36:49.145 INFO:teuthology.orchestra.run.smithi167.stdout:Upgrade 8 Packages 2024-09-16T15:36:49.145 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-16T15:36:49.145 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 235 M 2024-09-16T15:36:49.145 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2024-09-16T15:36:49.777 INFO:teuthology.orchestra.run.smithi167.stdout:(1/157): ceph-19.3.0-4938.g33f95d2a.el9.x86_64. 51 kB/s | 6.6 kB 00:00 2024-09-16T15:36:49.953 INFO:teuthology.orchestra.run.smithi167.stdout:(2/157): ceph-fuse-19.3.0-4938.g33f95d2a.el9.x8 5.0 MB/s | 900 kB 00:00 2024-09-16T15:36:50.003 INFO:teuthology.orchestra.run.smithi167.stdout:(3/157): ceph-immutable-object-cache-19.3.0-493 2.8 MB/s | 145 kB 00:00 2024-09-16T15:36:50.203 INFO:teuthology.orchestra.run.smithi167.stdout:(4/157): ceph-mds-19.3.0-4938.g33f95d2a.el9.x86 12 MB/s | 2.4 MB 00:00 2024-09-16T15:36:50.279 INFO:teuthology.orchestra.run.smithi167.stdout:(5/157): ceph-base-19.3.0-4938.g33f95d2a.el9.x8 8.7 MB/s | 5.5 MB 00:00 2024-09-16T15:36:50.387 INFO:teuthology.orchestra.run.smithi167.stdout:(6/157): ceph-mgr-19.3.0-4938.g33f95d2a.el9.x86 8.3 MB/s | 1.5 MB 00:00 2024-09-16T15:36:50.604 INFO:teuthology.orchestra.run.smithi167.stdout:(7/157): ceph-mon-19.3.0-4938.g33f95d2a.el9.x86 15 MB/s | 4.8 MB 00:00 2024-09-16T15:36:50.739 INFO:teuthology.orchestra.run.smithi151.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-16T15:36:50.740 INFO:teuthology.orchestra.run.smithi151.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-16T15:36:50.754 INFO:teuthology.orchestra.run.smithi151.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-16T15:36:50.755 INFO:teuthology.orchestra.run.smithi151.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-16T15:36:50.756 INFO:teuthology.orchestra.run.smithi151.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-16T15:36:50.757 INFO:teuthology.orchestra.run.smithi151.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-16T15:36:50.759 INFO:teuthology.orchestra.run.smithi151.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-16T15:36:50.762 INFO:teuthology.orchestra.run.smithi151.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-16T15:36:50.763 INFO:teuthology.orchestra.run.smithi151.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-16T15:36:50.830 INFO:teuthology.orchestra.run.smithi151.stdout:Dependencies resolved. 2024-09-16T15:36:50.869 INFO:teuthology.orchestra.run.smithi151.stdout:======================================================================================= 2024-09-16T15:36:50.869 INFO:teuthology.orchestra.run.smithi151.stdout: Package Arch Version Repository Size 2024-09-16T15:36:50.869 INFO:teuthology.orchestra.run.smithi151.stdout:======================================================================================= 2024-09-16T15:36:50.869 INFO:teuthology.orchestra.run.smithi151.stdout:Installing: 2024-09-16T15:36:50.869 INFO:teuthology.orchestra.run.smithi151.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-16T15:36:50.869 INFO:teuthology.orchestra.run.smithi151.stdout: ceph x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 6.6 k 2024-09-16T15:36:50.869 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-base x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 5.5 M 2024-09-16T15:36:50.869 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-fuse x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 900 k 2024-09-16T15:36:50.869 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 145 k 2024-09-16T15:36:50.869 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 1.5 M 2024-09-16T15:36:50.869 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4938.g33f95d2a.el9 ceph-noarch 154 k 2024-09-16T15:36:50.870 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4938.g33f95d2a.el9 ceph-noarch 14 M 2024-09-16T15:36:50.870 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4938.g33f95d2a.el9 ceph-noarch 7.4 M 2024-09-16T15:36:50.870 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-rook noarch 2:19.3.0-4938.g33f95d2a.el9 ceph-noarch 49 k 2024-09-16T15:36:50.870 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-radosgw x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 11 M 2024-09-16T15:36:50.870 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-test x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 46 M 2024-09-16T15:36:50.870 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-volume noarch 2:19.3.0-4938.g33f95d2a.el9 ceph-noarch 285 k 2024-09-16T15:36:50.870 INFO:teuthology.orchestra.run.smithi151.stdout: cephadm noarch 2:19.3.0-4938.g33f95d2a.el9 ceph-noarch 784 k 2024-09-16T15:36:50.870 INFO:teuthology.orchestra.run.smithi151.stdout: cephfs-mirror x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 225 k 2024-09-16T15:36:50.870 INFO:teuthology.orchestra.run.smithi151.stdout: cephfs-shell x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 27 k 2024-09-16T15:36:50.870 INFO:teuthology.orchestra.run.smithi151.stdout: cephfs-top noarch 2:19.3.0-4938.g33f95d2a.el9 ceph-noarch 18 k 2024-09-16T15:36:50.870 INFO:teuthology.orchestra.run.smithi151.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-16T15:36:50.870 INFO:teuthology.orchestra.run.smithi151.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-16T15:36:50.870 INFO:teuthology.orchestra.run.smithi151.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-16T15:36:50.870 INFO:teuthology.orchestra.run.smithi151.stdout: libcephfs-devel x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 32 k 2024-09-16T15:36:50.871 INFO:teuthology.orchestra.run.smithi151.stdout: libcephfs2 x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 756 k 2024-09-16T15:36:50.871 INFO:teuthology.orchestra.run.smithi151.stdout: librados-devel x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 127 k 2024-09-16T15:36:50.871 INFO:teuthology.orchestra.run.smithi151.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-16T15:36:50.871 INFO:teuthology.orchestra.run.smithi151.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-16T15:36:50.871 INFO:teuthology.orchestra.run.smithi151.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-16T15:36:50.871 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-16T15:36:50.871 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-16T15:36:50.871 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-16T15:36:50.871 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cephfs x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 161 k 2024-09-16T15:36:50.871 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rados x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 321 k 2024-09-16T15:36:50.871 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rbd x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 306 k 2024-09-16T15:36:50.871 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rgw x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 99 k 2024-09-16T15:36:50.871 INFO:teuthology.orchestra.run.smithi151.stdout: rbd-fuse x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 85 k 2024-09-16T15:36:50.871 INFO:teuthology.orchestra.run.smithi151.stdout: rbd-mirror x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 3.2 M 2024-09-16T15:36:50.871 INFO:teuthology.orchestra.run.smithi151.stdout: rbd-nbd x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 170 k 2024-09-16T15:36:50.872 INFO:teuthology.orchestra.run.smithi151.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-16T15:36:50.872 INFO:teuthology.orchestra.run.smithi151.stdout:Upgrading: 2024-09-16T15:36:50.872 INFO:teuthology.orchestra.run.smithi151.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-16T15:36:50.872 INFO:teuthology.orchestra.run.smithi151.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-16T15:36:50.872 INFO:teuthology.orchestra.run.smithi151.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-16T15:36:50.872 INFO:teuthology.orchestra.run.smithi151.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-16T15:36:50.872 INFO:teuthology.orchestra.run.smithi151.stdout: librados2 x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 3.5 M 2024-09-16T15:36:50.872 INFO:teuthology.orchestra.run.smithi151.stdout: librbd1 x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 3.2 M 2024-09-16T15:36:50.872 INFO:teuthology.orchestra.run.smithi151.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-16T15:36:50.872 INFO:teuthology.orchestra.run.smithi151.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-16T15:36:50.872 INFO:teuthology.orchestra.run.smithi151.stdout:Installing dependencies: 2024-09-16T15:36:50.872 INFO:teuthology.orchestra.run.smithi151.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-16T15:36:50.872 INFO:teuthology.orchestra.run.smithi151.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-16T15:36:50.872 INFO:teuthology.orchestra.run.smithi151.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-16T15:36:50.873 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-common x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 22 M 2024-09-16T15:36:50.873 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4938.g33f95d2a.el9 ceph-noarch 34 k 2024-09-16T15:36:50.873 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mds x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 2.4 M 2024-09-16T15:36:50.873 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4938.g33f95d2a.el9 ceph-noarch 291 k 2024-09-16T15:36:50.873 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mon x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 4.8 M 2024-09-16T15:36:50.873 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-osd x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 17 M 2024-09-16T15:36:50.873 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4938.g33f95d2a.el9 ceph-noarch 16 k 2024-09-16T15:36:50.873 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-selinux x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 25 k 2024-09-16T15:36:50.873 INFO:teuthology.orchestra.run.smithi151.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-16T15:36:50.873 INFO:teuthology.orchestra.run.smithi151.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-16T15:36:50.873 INFO:teuthology.orchestra.run.smithi151.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-16T15:36:50.873 INFO:teuthology.orchestra.run.smithi151.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-16T15:36:50.873 INFO:teuthology.orchestra.run.smithi151.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-16T15:36:50.874 INFO:teuthology.orchestra.run.smithi151.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-16T15:36:50.874 INFO:teuthology.orchestra.run.smithi151.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-16T15:36:50.874 INFO:teuthology.orchestra.run.smithi151.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-16T15:36:50.874 INFO:teuthology.orchestra.run.smithi151.stdout: libcephsqlite x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 161 k 2024-09-16T15:36:50.874 INFO:teuthology.orchestra.run.smithi151.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-16T15:36:50.874 INFO:teuthology.orchestra.run.smithi151.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-16T15:36:50.874 INFO:teuthology.orchestra.run.smithi151.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-16T15:36:50.874 INFO:teuthology.orchestra.run.smithi151.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-16T15:36:50.874 INFO:teuthology.orchestra.run.smithi151.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-16T15:36:50.874 INFO:teuthology.orchestra.run.smithi151.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-16T15:36:50.874 INFO:teuthology.orchestra.run.smithi151.stdout: libradosstriper1 x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 508 k 2024-09-16T15:36:50.874 INFO:teuthology.orchestra.run.smithi151.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-16T15:36:50.874 INFO:teuthology.orchestra.run.smithi151.stdout: librgw2 x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 5.5 M 2024-09-16T15:36:50.874 INFO:teuthology.orchestra.run.smithi151.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-16T15:36:50.875 INFO:teuthology.orchestra.run.smithi151.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-16T15:36:50.875 INFO:teuthology.orchestra.run.smithi151.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-16T15:36:50.875 INFO:teuthology.orchestra.run.smithi151.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-16T15:36:50.875 INFO:teuthology.orchestra.run.smithi151.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-16T15:36:50.875 INFO:teuthology.orchestra.run.smithi151.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-16T15:36:50.875 INFO:teuthology.orchestra.run.smithi151.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-16T15:36:50.875 INFO:teuthology.orchestra.run.smithi151.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-16T15:36:50.875 INFO:teuthology.orchestra.run.smithi151.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-16T15:36:50.875 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-16T15:36:50.875 INFO:teuthology.orchestra.run.smithi151.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-16T15:36:50.875 INFO:teuthology.orchestra.run.smithi151.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-16T15:36:50.875 INFO:teuthology.orchestra.run.smithi151.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-16T15:36:50.875 INFO:teuthology.orchestra.run.smithi151.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-16T15:36:50.875 INFO:teuthology.orchestra.run.smithi151.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-16T15:36:50.875 INFO:teuthology.orchestra.run.smithi151.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-16T15:36:50.876 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-16T15:36:50.876 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ceph-argparse x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 45 k 2024-09-16T15:36:50.876 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ceph-common x86_64 2:19.3.0-4938.g33f95d2a.el9 ceph 148 k 2024-09-16T15:36:50.876 INFO:teuthology.orchestra.run.smithi151.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-16T15:36:50.876 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-16T15:36:50.876 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-16T15:36:50.876 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-16T15:36:50.876 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-16T15:36:50.876 INFO:teuthology.orchestra.run.smithi151.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-16T15:36:50.876 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-16T15:36:50.876 INFO:teuthology.orchestra.run.smithi151.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-16T15:36:50.876 INFO:teuthology.orchestra.run.smithi151.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-16T15:36:50.876 INFO:teuthology.orchestra.run.smithi151.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-16T15:36:50.877 INFO:teuthology.orchestra.run.smithi151.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-16T15:36:50.877 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-16T15:36:50.877 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-16T15:36:50.877 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-16T15:36:50.877 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-16T15:36:50.877 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-16T15:36:50.877 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-16T15:36:50.877 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-16T15:36:50.877 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-16T15:36:50.877 INFO:teuthology.orchestra.run.smithi151.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-16T15:36:50.877 INFO:teuthology.orchestra.run.smithi151.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-16T15:36:50.877 INFO:teuthology.orchestra.run.smithi151.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-16T15:36:50.877 INFO:teuthology.orchestra.run.smithi151.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-16T15:36:50.877 INFO:teuthology.orchestra.run.smithi151.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-16T15:36:50.878 INFO:teuthology.orchestra.run.smithi151.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-16T15:36:50.878 INFO:teuthology.orchestra.run.smithi151.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-16T15:36:50.878 INFO:teuthology.orchestra.run.smithi151.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-16T15:36:50.878 INFO:teuthology.orchestra.run.smithi151.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-16T15:36:50.878 INFO:teuthology.orchestra.run.smithi151.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-16T15:36:50.878 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-16T15:36:50.878 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-16T15:36:50.878 INFO:teuthology.orchestra.run.smithi151.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-16T15:36:50.878 INFO:teuthology.orchestra.run.smithi151.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-16T15:36:50.878 INFO:teuthology.orchestra.run.smithi151.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-16T15:36:50.878 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-16T15:36:50.878 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-16T15:36:50.878 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-16T15:36:50.878 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-16T15:36:50.878 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-16T15:36:50.879 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-16T15:36:50.879 INFO:teuthology.orchestra.run.smithi151.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-16T15:36:50.879 INFO:teuthology.orchestra.run.smithi151.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-16T15:36:50.879 INFO:teuthology.orchestra.run.smithi151.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-16T15:36:50.879 INFO:teuthology.orchestra.run.smithi151.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-16T15:36:50.879 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-16T15:36:50.879 INFO:teuthology.orchestra.run.smithi151.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-16T15:36:50.879 INFO:teuthology.orchestra.run.smithi151.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-16T15:36:50.879 INFO:teuthology.orchestra.run.smithi151.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-16T15:36:50.879 INFO:teuthology.orchestra.run.smithi151.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-16T15:36:50.879 INFO:teuthology.orchestra.run.smithi151.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-16T15:36:50.879 INFO:teuthology.orchestra.run.smithi151.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-16T15:36:50.879 INFO:teuthology.orchestra.run.smithi151.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-16T15:36:50.879 INFO:teuthology.orchestra.run.smithi151.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-16T15:36:50.880 INFO:teuthology.orchestra.run.smithi151.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-16T15:36:50.880 INFO:teuthology.orchestra.run.smithi151.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-16T15:36:50.880 INFO:teuthology.orchestra.run.smithi151.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-16T15:36:50.880 INFO:teuthology.orchestra.run.smithi151.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-16T15:36:50.880 INFO:teuthology.orchestra.run.smithi151.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-16T15:36:50.880 INFO:teuthology.orchestra.run.smithi151.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-16T15:36:50.880 INFO:teuthology.orchestra.run.smithi151.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-16T15:36:50.880 INFO:teuthology.orchestra.run.smithi151.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-16T15:36:50.880 INFO:teuthology.orchestra.run.smithi151.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-16T15:36:50.880 INFO:teuthology.orchestra.run.smithi151.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-16T15:36:50.880 INFO:teuthology.orchestra.run.smithi151.stdout:Installing weak dependencies: 2024-09-16T15:36:50.880 INFO:teuthology.orchestra.run.smithi151.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-16T15:36:50.880 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:36:50.880 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction Summary 2024-09-16T15:36:50.880 INFO:teuthology.orchestra.run.smithi151.stdout:======================================================================================= 2024-09-16T15:36:50.881 INFO:teuthology.orchestra.run.smithi151.stdout:Install 149 Packages 2024-09-16T15:36:50.881 INFO:teuthology.orchestra.run.smithi151.stdout:Upgrade 8 Packages 2024-09-16T15:36:50.881 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:36:50.881 INFO:teuthology.orchestra.run.smithi151.stdout:Total download size: 235 M 2024-09-16T15:36:50.881 INFO:teuthology.orchestra.run.smithi151.stdout:Downloading Packages: 2024-09-16T15:36:51.213 INFO:teuthology.orchestra.run.smithi167.stdout:(8/157): ceph-common-19.3.0-4938.g33f95d2a.el9. 14 MB/s | 22 MB 00:01 2024-09-16T15:36:51.263 INFO:teuthology.orchestra.run.smithi167.stdout:(9/157): ceph-selinux-19.3.0-4938.g33f95d2a.el9 507 kB/s | 25 kB 00:00 2024-09-16T15:36:51.422 INFO:teuthology.orchestra.run.smithi167.stdout:(10/157): ceph-radosgw-19.3.0-4938.g33f95d2a.el 13 MB/s | 11 MB 00:00 2024-09-16T15:36:51.472 INFO:teuthology.orchestra.run.smithi167.stdout:(11/157): cephfs-mirror-19.3.0-4938.g33f95d2a.e 4.4 MB/s | 225 kB 00:00 2024-09-16T15:36:51.592 INFO:teuthology.orchestra.run.smithi151.stdout:(1/157): ceph-19.3.0-4938.g33f95d2a.el9.x86_64. 39 kB/s | 6.6 kB 00:00 2024-09-16T15:36:51.714 INFO:teuthology.orchestra.run.smithi167.stdout:(12/157): ceph-osd-19.3.0-4938.g33f95d2a.el9.x8 13 MB/s | 17 MB 00:01 2024-09-16T15:36:51.739 INFO:teuthology.orchestra.run.smithi167.stdout:(13/157): cephfs-shell-19.3.0-4938.g33f95d2a.el 101 kB/s | 27 kB 00:00 2024-09-16T15:36:51.767 INFO:teuthology.orchestra.run.smithi151.stdout:(2/157): ceph-fuse-19.3.0-4938.g33f95d2a.el9.x8 5.0 MB/s | 900 kB 00:00 2024-09-16T15:36:51.773 INFO:teuthology.orchestra.run.smithi167.stdout:(14/157): libcephfs-devel-19.3.0-4938.g33f95d2a 543 kB/s | 32 kB 00:00 2024-09-16T15:36:51.815 INFO:teuthology.orchestra.run.smithi167.stdout:(15/157): libcephfs2-19.3.0-4938.g33f95d2a.el9. 9.8 MB/s | 756 kB 00:00 2024-09-16T15:36:51.817 INFO:teuthology.orchestra.run.smithi151.stdout:(3/157): ceph-immutable-object-cache-19.3.0-493 2.8 MB/s | 145 kB 00:00 2024-09-16T15:36:51.849 INFO:teuthology.orchestra.run.smithi167.stdout:(16/157): libcephsqlite-19.3.0-4938.g33f95d2a.e 2.1 MB/s | 161 kB 00:00 2024-09-16T15:36:51.874 INFO:teuthology.orchestra.run.smithi167.stdout:(17/157): librados-devel-19.3.0-4938.g33f95d2a. 2.1 MB/s | 127 kB 00:00 2024-09-16T15:36:51.916 INFO:teuthology.orchestra.run.smithi167.stdout:(18/157): libradosstriper1-19.3.0-4938.g33f95d2 7.4 MB/s | 508 kB 00:00 2024-09-16T15:36:51.999 INFO:teuthology.orchestra.run.smithi167.stdout:(19/157): python3-ceph-argparse-19.3.0-4938.g33 543 kB/s | 45 kB 00:00 2024-09-16T15:36:52.100 INFO:teuthology.orchestra.run.smithi167.stdout:(20/157): python3-ceph-common-19.3.0-4938.g33f9 1.5 MB/s | 148 kB 00:00 2024-09-16T15:36:52.126 INFO:teuthology.orchestra.run.smithi151.stdout:(4/157): ceph-base-19.3.0-4938.g33f95d2a.el9.x8 7.8 MB/s | 5.5 MB 00:00 2024-09-16T15:36:52.192 INFO:teuthology.orchestra.run.smithi167.stdout:(21/157): python3-cephfs-19.3.0-4938.g33f95d2a. 1.7 MB/s | 161 kB 00:00 2024-09-16T15:36:52.193 INFO:teuthology.orchestra.run.smithi151.stdout:(5/157): ceph-mds-19.3.0-4938.g33f95d2a.el9.x86 6.3 MB/s | 2.4 MB 00:00 2024-09-16T15:36:52.309 INFO:teuthology.orchestra.run.smithi167.stdout:(22/157): librgw2-19.3.0-4938.g33f95d2a.el9.x86 13 MB/s | 5.5 MB 00:00 2024-09-16T15:36:52.342 INFO:teuthology.orchestra.run.smithi167.stdout:(23/157): python3-rados-19.3.0-4938.g33f95d2a.e 2.1 MB/s | 321 kB 00:00 2024-09-16T15:36:52.352 INFO:teuthology.orchestra.run.smithi151.stdout:(6/157): ceph-mgr-19.3.0-4938.g33f95d2a.el9.x86 6.8 MB/s | 1.5 MB 00:00 2024-09-16T15:36:52.401 INFO:teuthology.orchestra.run.smithi167.stdout:(24/157): python3-rbd-19.3.0-4938.g33f95d2a.el9 3.3 MB/s | 306 kB 00:00 2024-09-16T15:36:52.426 INFO:teuthology.orchestra.run.smithi167.stdout:(25/157): python3-rgw-19.3.0-4938.g33f95d2a.el9 1.2 MB/s | 99 kB 00:00 2024-09-16T15:36:52.493 INFO:teuthology.orchestra.run.smithi167.stdout:(26/157): rbd-fuse-19.3.0-4938.g33f95d2a.el9.x8 929 kB/s | 85 kB 00:00 2024-09-16T15:36:52.827 INFO:teuthology.orchestra.run.smithi167.stdout:(27/157): rbd-mirror-19.3.0-4938.g33f95d2a.el9. 8.0 MB/s | 3.2 MB 00:00 2024-09-16T15:36:52.869 INFO:teuthology.orchestra.run.smithi151.stdout:(7/157): ceph-mon-19.3.0-4938.g33f95d2a.el9.x86 7.1 MB/s | 4.8 MB 00:00 2024-09-16T15:36:52.902 INFO:teuthology.orchestra.run.smithi167.stdout:(28/157): ceph-grafana-dashboards-19.3.0-4938.g 458 kB/s | 34 kB 00:00 2024-09-16T15:36:52.994 INFO:teuthology.orchestra.run.smithi167.stdout:(29/157): ceph-mgr-cephadm-19.3.0-4938.g33f95d2 1.6 MB/s | 154 kB 00:00 2024-09-16T15:36:53.128 INFO:teuthology.orchestra.run.smithi167.stdout:(30/157): rbd-nbd-19.3.0-4938.g33f95d2a.el9.x86 269 kB/s | 170 kB 00:00 2024-09-16T15:36:53.853 INFO:teuthology.orchestra.run.smithi151.stdout:(8/157): ceph-common-19.3.0-4938.g33f95d2a.el9. 8.9 MB/s | 22 MB 00:02 2024-09-16T15:36:53.928 INFO:teuthology.orchestra.run.smithi151.stdout:(9/157): ceph-selinux-19.3.0-4938.g33f95d2a.el9 336 kB/s | 25 kB 00:00 2024-09-16T15:36:53.995 INFO:teuthology.orchestra.run.smithi167.stdout:(31/157): ceph-test-19.3.0-4938.g33f95d2a.el9.x 17 MB/s | 46 MB 00:02 2024-09-16T15:36:54.045 INFO:teuthology.orchestra.run.smithi167.stdout:(32/157): ceph-mgr-modules-core-19.3.0-4938.g33 5.8 MB/s | 291 kB 00:00 2024-09-16T15:36:54.112 INFO:teuthology.orchestra.run.smithi167.stdout:(33/157): ceph-mgr-rook-19.3.0-4938.g33f95d2a.e 744 kB/s | 49 kB 00:00 2024-09-16T15:36:54.171 INFO:teuthology.orchestra.run.smithi167.stdout:(34/157): ceph-prometheus-alerts-19.3.0-4938.g3 283 kB/s | 16 kB 00:00 2024-09-16T15:36:54.203 INFO:teuthology.orchestra.run.smithi151.stdout:(10/157): ceph-osd-19.3.0-4938.g33f95d2a.el9.x8 9.3 MB/s | 17 MB 00:01 2024-09-16T15:36:54.238 INFO:teuthology.orchestra.run.smithi167.stdout:(35/157): ceph-volume-19.3.0-4938.g33f95d2a.el9 4.2 MB/s | 285 kB 00:00 2024-09-16T15:36:54.321 INFO:teuthology.orchestra.run.smithi151.stdout:(11/157): ceph-radosgw-19.3.0-4938.g33f95d2a.el 7.5 MB/s | 11 MB 00:01 2024-09-16T15:36:54.354 INFO:teuthology.orchestra.run.smithi151.stdout:(12/157): cephfs-mirror-19.3.0-4938.g33f95d2a.e 1.5 MB/s | 225 kB 00:00 2024-09-16T15:36:54.396 INFO:teuthology.orchestra.run.smithi151.stdout:(13/157): cephfs-shell-19.3.0-4938.g33f95d2a.el 360 kB/s | 27 kB 00:00 2024-09-16T15:36:54.422 INFO:teuthology.orchestra.run.smithi167.stdout:(36/157): ceph-mgr-dashboard-19.3.0-4938.g33f95 9.9 MB/s | 14 MB 00:01 2024-09-16T15:36:54.438 INFO:teuthology.orchestra.run.smithi151.stdout:(14/157): libcephfs-devel-19.3.0-4938.g33f95d2a 379 kB/s | 32 kB 00:00 2024-09-16T15:36:54.455 INFO:teuthology.orchestra.run.smithi167.stdout:(37/157): cephadm-19.3.0-4938.g33f95d2a.el9.noa 3.5 MB/s | 784 kB 00:00 2024-09-16T15:36:54.489 INFO:teuthology.orchestra.run.smithi151.stdout:(15/157): libcephsqlite-19.3.0-4938.g33f95d2a.e 3.1 MB/s | 161 kB 00:00 2024-09-16T15:36:54.497 INFO:teuthology.orchestra.run.smithi167.stdout:(38/157): cephfs-top-19.3.0-4938.g33f95d2a.el9. 244 kB/s | 18 kB 00:00 2024-09-16T15:36:54.531 INFO:teuthology.orchestra.run.smithi151.stdout:(16/157): libcephfs2-19.3.0-4938.g33f95d2a.el9. 5.5 MB/s | 756 kB 00:00 2024-09-16T15:36:54.556 INFO:teuthology.orchestra.run.smithi151.stdout:(17/157): librados-devel-19.3.0-4938.g33f95d2a. 1.9 MB/s | 127 kB 00:00 2024-09-16T15:36:54.590 INFO:teuthology.orchestra.run.smithi151.stdout:(18/157): libradosstriper1-19.3.0-4938.g33f95d2 8.5 MB/s | 508 kB 00:00 2024-09-16T15:36:54.623 INFO:teuthology.orchestra.run.smithi167.stdout:(39/157): fuse-2.9.9-16.el9.x86_64.rpm 478 kB/s | 80 kB 00:00 2024-09-16T15:36:54.674 INFO:teuthology.orchestra.run.smithi151.stdout:(19/157): python3-ceph-argparse-19.3.0-4938.g33 541 kB/s | 45 kB 00:00 2024-09-16T15:36:54.724 INFO:teuthology.orchestra.run.smithi151.stdout:(20/157): python3-ceph-common-19.3.0-4938.g33f9 2.9 MB/s | 148 kB 00:00 2024-09-16T15:36:54.740 INFO:teuthology.orchestra.run.smithi167.stdout:(40/157): libgfortran-11.5.0-2.el9.x86_64.rpm 3.2 MB/s | 801 kB 00:00 2024-09-16T15:36:54.773 INFO:teuthology.orchestra.run.smithi167.stdout:(41/157): libquadmath-11.5.0-2.el9.x86_64.rpm 1.2 MB/s | 191 kB 00:00 2024-09-16T15:36:54.782 INFO:teuthology.orchestra.run.smithi151.stdout:(21/157): python3-cephfs-19.3.0-4938.g33f95d2a. 2.7 MB/s | 161 kB 00:00 2024-09-16T15:36:54.807 INFO:teuthology.orchestra.run.smithi167.stdout:(42/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.7 MB/s | 253 kB 00:00 2024-09-16T15:36:54.849 INFO:teuthology.orchestra.run.smithi151.stdout:(22/157): python3-rados-19.3.0-4938.g33f95d2a.e 4.7 MB/s | 321 kB 00:00 2024-09-16T15:36:54.866 INFO:teuthology.orchestra.run.smithi167.stdout:(43/157): python3-cryptography-36.0.1-4.el9.x86 13 MB/s | 1.2 MB 00:00 2024-09-16T15:36:54.908 INFO:teuthology.orchestra.run.smithi167.stdout:(44/157): python3-ply-3.11-14.el9.noarch.rpm 1.0 MB/s | 106 kB 00:00 2024-09-16T15:36:54.933 INFO:teuthology.orchestra.run.smithi167.stdout:(45/157): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-09-16T15:36:54.959 INFO:teuthology.orchestra.run.smithi167.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-16T15:36:54.959 INFO:teuthology.orchestra.run.smithi151.stdout:(23/157): librgw2-19.3.0-4938.g33f95d2a.el9.x86 14 MB/s | 5.5 MB 00:00 2024-09-16T15:36:54.984 INFO:teuthology.orchestra.run.smithi167.stdout:(47/157): python3-urllib3-1.26.5-6.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-09-16T15:36:54.984 INFO:teuthology.orchestra.run.smithi151.stdout:(24/157): python3-rbd-19.3.0-4938.g33f95d2a.el9 2.2 MB/s | 306 kB 00:00 2024-09-16T15:36:55.034 INFO:teuthology.orchestra.run.smithi151.stdout:(25/157): python3-rgw-19.3.0-4938.g33f95d2a.el9 1.3 MB/s | 99 kB 00:00 2024-09-16T15:36:55.051 INFO:teuthology.orchestra.run.smithi167.stdout:(48/157): bison-3.7.4-5.el9.x86_64.rpm 14 MB/s | 943 kB 00:00 2024-09-16T15:36:55.059 INFO:teuthology.orchestra.run.smithi151.stdout:(26/157): rbd-fuse-19.3.0-4938.g33f95d2a.el9.x8 1.1 MB/s | 85 kB 00:00 2024-09-16T15:36:55.076 INFO:teuthology.orchestra.run.smithi167.stdout:(49/157): automake-1.16.2-8.el9.noarch.rpm 5.7 MB/s | 689 kB 00:00 2024-09-16T15:36:55.110 INFO:teuthology.orchestra.run.smithi167.stdout:(50/157): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-09-16T15:36:55.110 INFO:teuthology.orchestra.run.smithi151.stdout:(27/157): rbd-nbd-19.3.0-4938.g33f95d2a.el9.x86 3.3 MB/s | 170 kB 00:00 2024-09-16T15:36:55.135 INFO:teuthology.orchestra.run.smithi167.stdout:(51/157): elfutils-libelf-devel-0.191-4.el9.x86 372 kB/s | 22 kB 00:00 2024-09-16T15:36:55.185 INFO:teuthology.orchestra.run.smithi151.stdout:(28/157): ceph-grafana-dashboards-19.3.0-4938.g 458 kB/s | 34 kB 00:00 2024-09-16T15:36:55.252 INFO:teuthology.orchestra.run.smithi167.stdout:(52/157): ceph-mgr-diskprediction-local-19.3.0- 3.5 MB/s | 7.4 MB 00:02 2024-09-16T15:36:55.277 INFO:teuthology.orchestra.run.smithi151.stdout:(29/157): rbd-mirror-19.3.0-4938.g33f95d2a.el9. 13 MB/s | 3.2 MB 00:00 2024-09-16T15:36:55.302 INFO:teuthology.orchestra.run.smithi167.stdout:(53/157): flexiblas-3.0.4-8.el9.x86_64.rpm 196 kB/s | 33 kB 00:00 2024-09-16T15:36:55.303 INFO:teuthology.orchestra.run.smithi151.stdout:(30/157): ceph-mgr-cephadm-19.3.0-4938.g33f95d2 1.3 MB/s | 154 kB 00:00 2024-09-16T15:36:55.328 INFO:teuthology.orchestra.run.smithi167.stdout:(54/157): flex-2.6.4-9.el9.x86_64.rpm 1.4 MB/s | 313 kB 00:00 2024-09-16T15:36:55.353 INFO:teuthology.orchestra.run.smithi167.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 362 kB/s | 18 kB 00:00 2024-09-16T15:36:55.378 INFO:teuthology.orchestra.run.smithi167.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 221 kB/s | 11 kB 00:00 2024-09-16T15:36:55.404 INFO:teuthology.orchestra.run.smithi167.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 3.3 MB/s | 171 kB 00:00 2024-09-16T15:36:55.429 INFO:teuthology.orchestra.run.smithi167.stdout:(58/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-09-16T15:36:55.454 INFO:teuthology.orchestra.run.smithi167.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 908 kB/s | 45 kB 00:00 2024-09-16T15:36:55.488 INFO:teuthology.orchestra.run.smithi167.stdout:(60/157): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-09-16T15:36:55.530 INFO:teuthology.orchestra.run.smithi167.stdout:(61/157): libtool-2.4.6-46.el9.x86_64.rpm 7.5 MB/s | 581 kB 00:00 2024-09-16T15:36:55.555 INFO:teuthology.orchestra.run.smithi167.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 367 kB/s | 24 kB 00:00 2024-09-16T15:36:55.580 INFO:teuthology.orchestra.run.smithi167.stdout:(63/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 950 kB/s | 47 kB 00:00 2024-09-16T15:36:55.606 INFO:teuthology.orchestra.run.smithi167.stdout:(64/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-09-16T15:36:55.631 INFO:teuthology.orchestra.run.smithi167.stdout:(65/157): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-09-16T15:36:55.657 INFO:teuthology.orchestra.run.smithi167.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 761 kB/s | 38 kB 00:00 2024-09-16T15:36:55.749 INFO:teuthology.orchestra.run.smithi167.stdout:(67/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.0 MB/s | 3.0 MB 00:00 2024-09-16T15:36:55.833 INFO:teuthology.orchestra.run.smithi167.stdout:(68/157): postgresql-13.16-1.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-09-16T15:36:55.895 INFO:teuthology.orchestra.run.smithi151.stdout:(31/157): ceph-mgr-diskprediction-local-19.3.0- 13 MB/s | 7.4 MB 00:00 2024-09-16T15:36:55.908 INFO:teuthology.orchestra.run.smithi167.stdout:(69/157): openblas-openmp-0.3.26-2.el9.x86_64.r 18 MB/s | 4.9 MB 00:00 2024-09-16T15:36:55.945 INFO:teuthology.orchestra.run.smithi151.stdout:(32/157): ceph-mgr-modules-core-19.3.0-4938.g33 5.7 MB/s | 291 kB 00:00 2024-09-16T15:36:55.983 INFO:teuthology.orchestra.run.smithi167.stdout:(70/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-09-16T15:36:56.008 INFO:teuthology.orchestra.run.smithi167.stdout:(71/157): postgresql-private-libs-13.16-1.el9.x 1.3 MB/s | 137 kB 00:00 2024-09-16T15:36:56.020 INFO:teuthology.orchestra.run.smithi151.stdout:(33/157): ceph-mgr-rook-19.3.0-4938.g33f95d2a.e 657 kB/s | 49 kB 00:00 2024-09-16T15:36:56.050 INFO:teuthology.orchestra.run.smithi167.stdout:(72/157): postgresql-contrib-13.16-1.el9.x86_64 4.1 MB/s | 910 kB 00:00 2024-09-16T15:36:56.125 INFO:teuthology.orchestra.run.smithi167.stdout:(73/157): protobuf-3.14.0-13.el9.x86_64.rpm 8.7 MB/s | 1.0 MB 00:00 2024-09-16T15:36:56.159 INFO:teuthology.orchestra.run.smithi167.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 807 kB/s | 87 kB 00:00 2024-09-16T15:36:56.196 INFO:teuthology.orchestra.run.smithi151.stdout:(34/157): ceph-mgr-dashboard-19.3.0-4938.g33f95 15 MB/s | 14 MB 00:00 2024-09-16T15:36:56.201 INFO:teuthology.orchestra.run.smithi167.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-09-16T15:36:56.238 INFO:teuthology.orchestra.run.smithi151.stdout:(35/157): ceph-prometheus-alerts-19.3.0-4938.g3 76 kB/s | 16 kB 00:00 2024-09-16T15:36:56.263 INFO:teuthology.orchestra.run.smithi151.stdout:(36/157): ceph-volume-19.3.0-4938.g33f95d2a.el9 4.2 MB/s | 285 kB 00:00 2024-09-16T15:36:56.302 INFO:teuthology.orchestra.run.smithi167.stdout:(76/157): postgresql-server-13.16-1.el9.x86_64. 18 MB/s | 5.8 MB 00:00 2024-09-16T15:36:56.322 INFO:teuthology.orchestra.run.smithi151.stdout:(37/157): cephfs-top-19.3.0-4938.g33f95d2a.el9. 310 kB/s | 18 kB 00:00 2024-09-16T15:36:56.327 INFO:teuthology.orchestra.run.smithi167.stdout:(77/157): python3-devel-3.9.19-8.el9.x86_64.rpm 1.9 MB/s | 245 kB 00:00 2024-09-16T15:36:56.352 INFO:teuthology.orchestra.run.smithi167.stdout:(78/157): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-09-16T15:36:56.356 INFO:teuthology.orchestra.run.smithi151.stdout:(38/157): cephadm-19.3.0-4938.g33f95d2a.el9.noa 6.5 MB/s | 784 kB 00:00 2024-09-16T15:36:56.369 INFO:teuthology.orchestra.run.smithi167.stdout:(79/157): python3-jsonpatch-1.21-16.el9.noarch. 610 kB/s | 26 kB 00:00 2024-09-16T15:36:56.386 INFO:teuthology.orchestra.run.smithi167.stdout:(80/157): python3-jsonpointer-2.0-4.el9.noarch. 572 kB/s | 19 kB 00:00 2024-09-16T15:36:56.431 INFO:teuthology.orchestra.run.smithi151.stdout:(39/157): fuse-2.9.9-16.el9.x86_64.rpm 736 kB/s | 80 kB 00:00 2024-09-16T15:36:56.478 INFO:teuthology.orchestra.run.smithi167.stdout:(81/157): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 6.0 MB 00:00 2024-09-16T15:36:56.495 INFO:teuthology.orchestra.run.smithi167.stdout:(82/157): python3-markupsafe-1.1.1-12.el9.x86_6 320 kB/s | 35 kB 00:00 2024-09-16T15:36:56.554 INFO:teuthology.orchestra.run.smithi167.stdout:(83/157): python3-mako-1.1.4-6.el9.noarch.rpm 936 kB/s | 172 kB 00:00 2024-09-16T15:36:56.579 INFO:teuthology.orchestra.run.smithi167.stdout:(84/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.9 MB/s | 250 kB 00:00 2024-09-16T15:36:56.604 INFO:teuthology.orchestra.run.smithi167.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-09-16T15:36:56.629 INFO:teuthology.orchestra.run.smithi167.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-16T15:36:56.655 INFO:teuthology.orchestra.run.smithi167.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 840 kB/s | 42 kB 00:00 2024-09-16T15:36:56.680 INFO:teuthology.orchestra.run.smithi167.stdout:(88/157): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-09-16T15:36:56.706 INFO:teuthology.orchestra.run.smithi167.stdout:(89/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-16T15:36:56.731 INFO:teuthology.orchestra.run.smithi167.stdout:(90/157): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-09-16T15:36:56.756 INFO:teuthology.orchestra.run.smithi167.stdout:(91/157): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-16T15:36:56.782 INFO:teuthology.orchestra.run.smithi167.stdout:(92/157): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-16T15:36:56.832 INFO:teuthology.orchestra.run.smithi167.stdout:(93/157): python3-toml-0.10.2-6.el9.noarch.rpm 838 kB/s | 42 kB 00:00 2024-09-16T15:36:56.907 INFO:teuthology.orchestra.run.smithi151.stdout:(40/157): ceph-test-19.3.0-4938.g33f95d2a.el9.x 15 MB/s | 46 MB 00:02 2024-09-16T15:36:57.002 INFO:teuthology.orchestra.run.smithi151.stdout:(41/157): libquadmath-11.5.0-2.el9.x86_64.rpm 375 kB/s | 191 kB 00:00 2024-09-16T15:36:57.002 INFO:teuthology.orchestra.run.smithi151.stdout:(42/157): libgfortran-11.5.0-2.el9.x86_64.rpm 1.3 MB/s | 801 kB 00:00 2024-09-16T15:36:57.002 INFO:teuthology.orchestra.run.smithi167.stdout:(94/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 11 MB/s | 5.1 MB 00:00 2024-09-16T15:36:57.003 INFO:teuthology.orchestra.run.smithi167.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 331 kB/s | 44 kB 00:00 2024-09-16T15:36:57.003 INFO:teuthology.orchestra.run.smithi167.stdout:(96/157): qatlib-24.02.0-1.el9.x86_64.rpm 4.2 MB/s | 221 kB 00:00 2024-09-16T15:36:57.017 INFO:teuthology.orchestra.run.smithi167.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 712 kB/s | 36 kB 00:00 2024-09-16T15:36:57.039 INFO:teuthology.orchestra.run.smithi167.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.0 MB/s | 47 kB 00:00 2024-09-16T15:36:57.058 INFO:teuthology.orchestra.run.smithi151.stdout:(43/157): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 106 kB 00:00 2024-09-16T15:36:57.064 INFO:teuthology.orchestra.run.smithi167.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 6.3 MB/s | 304 kB 00:00 2024-09-16T15:36:57.089 INFO:teuthology.orchestra.run.smithi167.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-16T15:36:57.092 INFO:teuthology.orchestra.run.smithi151.stdout:(44/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.3 MB/s | 253 kB 00:00 2024-09-16T15:36:57.123 INFO:teuthology.orchestra.run.smithi167.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.6 MB/s | 96 kB 00:00 2024-09-16T15:36:57.142 INFO:teuthology.orchestra.run.smithi151.stdout:(45/157): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 135 kB 00:00 2024-09-16T15:36:57.148 INFO:teuthology.orchestra.run.smithi167.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-09-16T15:36:57.173 INFO:teuthology.orchestra.run.smithi167.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 891 kB/s | 45 kB 00:00 2024-09-16T15:36:57.176 INFO:teuthology.orchestra.run.smithi151.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-09-16T15:36:57.199 INFO:teuthology.orchestra.run.smithi167.stdout:(104/157): lua-devel-5.4.4-4.el9.x86_64.rpm 445 kB/s | 22 kB 00:00 2024-09-16T15:36:57.249 INFO:teuthology.orchestra.run.smithi167.stdout:(105/157): protobuf-compiler-3.14.0-13.el9.x86_ 11 MB/s | 863 kB 00:00 2024-09-16T15:36:57.251 INFO:teuthology.orchestra.run.smithi151.stdout:(47/157): python3-urllib3-1.26.5-6.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-09-16T15:36:57.318 INFO:teuthology.orchestra.run.smithi151.stdout:(48/157): automake-1.16.2-8.el9.noarch.rpm 4.8 MB/s | 689 kB 00:00 2024-09-16T15:36:57.358 INFO:teuthology.orchestra.run.smithi167.stdout:(106/157): gperftools-libs-2.9.1-3.el9.x86_64.r 2.8 MB/s | 308 kB 00:00 2024-09-16T15:36:57.360 INFO:teuthology.orchestra.run.smithi151.stdout:(49/157): boost-program-options-1.75.0-8.el9.x8 2.5 MB/s | 107 kB 00:00 2024-09-16T15:36:57.383 INFO:teuthology.orchestra.run.smithi167.stdout:(107/157): abseil-cpp-20211102.0-3.el9.x86_64.r 2.9 MB/s | 548 kB 00:00 2024-09-16T15:36:57.394 INFO:teuthology.orchestra.run.smithi151.stdout:(50/157): elfutils-libelf-devel-0.191-4.el9.x86 654 kB/s | 22 kB 00:00 2024-09-16T15:36:57.409 INFO:teuthology.orchestra.run.smithi167.stdout:(108/157): grpc-data-1.46.7-10.el9.noarch.rpm 389 kB/s | 19 kB 00:00 2024-09-16T15:36:57.428 INFO:teuthology.orchestra.run.smithi151.stdout:(51/157): python3-cryptography-36.0.1-4.el9.x86 2.5 MB/s | 1.2 MB 00:00 2024-09-16T15:36:57.434 INFO:teuthology.orchestra.run.smithi167.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-16T15:36:57.467 INFO:teuthology.orchestra.run.smithi167.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 1.4 MB/s | 49 kB 00:00 2024-09-16T15:36:57.469 INFO:teuthology.orchestra.run.smithi151.stdout:(52/157): bison-3.7.4-5.el9.x86_64.rpm 4.2 MB/s | 943 kB 00:00 2024-09-16T15:36:57.495 INFO:teuthology.orchestra.run.smithi151.stdout:(53/157): flex-2.6.4-9.el9.x86_64.rpm 3.0 MB/s | 313 kB 00:00 2024-09-16T15:36:57.501 INFO:teuthology.orchestra.run.smithi167.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2024-09-16T15:36:57.520 INFO:teuthology.orchestra.run.smithi151.stdout:(54/157): flexiblas-3.0.4-8.el9.x86_64.rpm 356 kB/s | 33 kB 00:00 2024-09-16T15:36:57.543 INFO:teuthology.orchestra.run.smithi167.stdout:(112/157): luarocks-3.9.2-5.el9.noarch.rpm 3.6 MB/s | 151 kB 00:00 2024-09-16T15:36:57.545 INFO:teuthology.orchestra.run.smithi151.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 364 kB/s | 18 kB 00:00 2024-09-16T15:36:57.571 INFO:teuthology.orchestra.run.smithi151.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-09-16T15:36:57.596 INFO:teuthology.orchestra.run.smithi151.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 3.3 MB/s | 171 kB 00:00 2024-09-16T15:36:57.680 INFO:teuthology.orchestra.run.smithi151.stdout:(58/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 14 MB/s | 3.0 MB 00:00 2024-09-16T15:36:57.714 INFO:teuthology.orchestra.run.smithi151.stdout:(59/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.1 MB/s | 160 kB 00:00 2024-09-16T15:36:57.739 INFO:teuthology.orchestra.run.smithi151.stdout:(60/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 317 kB/s | 45 kB 00:00 2024-09-16T15:36:57.753 INFO:teuthology.orchestra.run.smithi167.stdout:(113/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 16 MB/s | 16 MB 00:00 2024-09-16T15:36:57.781 INFO:teuthology.orchestra.run.smithi151.stdout:(61/157): libtool-2.4.6-46.el9.x86_64.rpm 8.5 MB/s | 581 kB 00:00 2024-09-16T15:36:57.807 INFO:teuthology.orchestra.run.smithi151.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 366 kB/s | 24 kB 00:00 2024-09-16T15:36:57.832 INFO:teuthology.orchestra.run.smithi151.stdout:(63/157): librdkafka-1.6.1-102.el9.x86_64.rpm 4.3 MB/s | 662 kB 00:00 2024-09-16T15:36:57.844 INFO:teuthology.orchestra.run.smithi167.stdout:(114/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 2.7 MB/s | 839 kB 00:00 2024-09-16T15:36:57.857 INFO:teuthology.orchestra.run.smithi151.stdout:(64/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 628 kB/s | 47 kB 00:00 2024-09-16T15:36:57.878 INFO:teuthology.orchestra.run.smithi167.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.3 MB/s | 43 kB 00:00 2024-09-16T15:36:57.883 INFO:teuthology.orchestra.run.smithi151.stdout:(65/157): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-09-16T15:36:57.908 INFO:teuthology.orchestra.run.smithi151.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 768 kB/s | 38 kB 00:00 2024-09-16T15:36:57.911 INFO:teuthology.orchestra.run.smithi167.stdout:(116/157): python3-asyncssh-2.13.2-5.el9.noarch 3.4 MB/s | 548 kB 00:00 2024-09-16T15:36:57.933 INFO:teuthology.orchestra.run.smithi151.stdout:(67/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.3 MB/s | 292 kB 00:00 2024-09-16T15:36:57.936 INFO:teuthology.orchestra.run.smithi167.stdout:(117/157): python3-cachetools-4.2.4-1.el9.noarc 550 kB/s | 32 kB 00:00 2024-09-16T15:36:57.962 INFO:teuthology.orchestra.run.smithi167.stdout:(118/157): python3-certifi-2023.05.07-4.el9.noa 282 kB/s | 14 kB 00:00 2024-09-16T15:36:57.987 INFO:teuthology.orchestra.run.smithi167.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-16T15:36:58.076 INFO:teuthology.orchestra.run.smithi151.stdout:(68/157): openblas-openmp-0.3.26-2.el9.x86_64.r 26 MB/s | 4.9 MB 00:00 2024-09-16T15:36:58.079 INFO:teuthology.orchestra.run.smithi167.stdout:(120/157): libarrow-9.0.0-11.el9.x86_64.rpm 6.3 MB/s | 4.4 MB 00:00 2024-09-16T15:36:58.104 INFO:teuthology.orchestra.run.smithi167.stdout:(121/157): python3-cherrypy-18.6.1-2.el9.noarch 2.5 MB/s | 358 kB 00:00 2024-09-16T15:36:58.129 INFO:teuthology.orchestra.run.smithi167.stdout:(122/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.1 MB/s | 303 kB 00:00 2024-09-16T15:36:58.134 INFO:teuthology.orchestra.run.smithi151.stdout:(69/157): postgresql-13.16-1.el9.x86_64.rpm 7.9 MB/s | 1.6 MB 00:00 2024-09-16T15:36:58.172 INFO:teuthology.orchestra.run.smithi167.stdout:(123/157): python3-google-auth-2.34.0-1.el9.noa 2.3 MB/s | 220 kB 00:00 2024-09-16T15:36:58.197 INFO:teuthology.orchestra.run.smithi167.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 430 kB/s | 11 kB 00:00 2024-09-16T15:36:58.218 INFO:teuthology.orchestra.run.smithi151.stdout:(70/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-09-16T15:36:58.222 INFO:teuthology.orchestra.run.smithi167.stdout:(125/157): python3-grpcio-tools-1.46.7-10.el9.x 1.5 MB/s | 144 kB 00:00 2024-09-16T15:36:58.243 INFO:teuthology.orchestra.run.smithi151.stdout:(71/157): postgresql-private-libs-13.16-1.el9.x 1.2 MB/s | 137 kB 00:00 2024-09-16T15:36:58.248 INFO:teuthology.orchestra.run.smithi167.stdout:(126/157): python3-jaraco-classes-3.2.1-5.el9.n 351 kB/s | 18 kB 00:00 2024-09-16T15:36:58.273 INFO:teuthology.orchestra.run.smithi167.stdout:(127/157): python3-jaraco-collections-3.0.0-8.e 463 kB/s | 23 kB 00:00 2024-09-16T15:36:58.285 INFO:teuthology.orchestra.run.smithi151.stdout:(72/157): postgresql-contrib-13.16-1.el9.x86_64 4.2 MB/s | 910 kB 00:00 2024-09-16T15:36:58.298 INFO:teuthology.orchestra.run.smithi167.stdout:(128/157): python3-jaraco-functools-3.5.0-2.el9 388 kB/s | 19 kB 00:00 2024-09-16T15:36:58.323 INFO:teuthology.orchestra.run.smithi167.stdout:(129/157): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-09-16T15:36:58.336 INFO:teuthology.orchestra.run.smithi151.stdout:(73/157): protobuf-3.14.0-13.el9.x86_64.rpm 11 MB/s | 1.0 MB 00:00 2024-09-16T15:36:58.357 INFO:teuthology.orchestra.run.smithi167.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.4 MB/s | 46 kB 00:00 2024-09-16T15:36:58.361 INFO:teuthology.orchestra.run.smithi151.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 1.1 MB/s | 87 kB 00:00 2024-09-16T15:36:58.390 INFO:teuthology.orchestra.run.smithi167.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-09-16T15:36:58.433 INFO:teuthology.orchestra.run.smithi167.stdout:(132/157): python3-kubernetes-26.1.0-2.el9.noar 7.6 MB/s | 1.0 MB 00:00 2024-09-16T15:36:58.458 INFO:teuthology.orchestra.run.smithi167.stdout:(133/157): python3-natsort-7.1.1-5.el9.noarch.r 858 kB/s | 58 kB 00:00 2024-09-16T15:36:58.483 INFO:teuthology.orchestra.run.smithi167.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 661 kB/s | 16 kB 00:00 2024-09-16T15:36:58.494 INFO:teuthology.orchestra.run.smithi151.stdout:(75/157): postgresql-server-13.16-1.el9.x86_64. 21 MB/s | 5.8 MB 00:00 2024-09-16T15:36:58.509 INFO:teuthology.orchestra.run.smithi167.stdout:(135/157): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-09-16T15:36:58.520 INFO:teuthology.orchestra.run.smithi151.stdout:(76/157): python3-colorama-0.4.6-3.el9.noarch.r 386 kB/s | 61 kB 00:00 2024-09-16T15:36:58.543 INFO:teuthology.orchestra.run.smithi167.stdout:(136/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-09-16T15:36:58.554 INFO:teuthology.orchestra.run.smithi151.stdout:(77/157): python3-devel-3.9.19-8.el9.x86_64.rpm 4.0 MB/s | 245 kB 00:00 2024-09-16T15:36:58.579 INFO:teuthology.orchestra.run.smithi151.stdout:(78/157): python3-jinja2-2.11.3-6.el9.noarch.rp 4.1 MB/s | 248 kB 00:00 2024-09-16T15:36:58.584 INFO:teuthology.orchestra.run.smithi167.stdout:(137/157): python3-pyperclip-1.8.2-2.el9.noarch 375 kB/s | 28 kB 00:00 2024-09-16T15:36:58.596 INFO:teuthology.orchestra.run.smithi151.stdout:(79/157): python3-jsonpatch-1.21-16.el9.noarch. 614 kB/s | 26 kB 00:00 2024-09-16T15:36:58.601 INFO:teuthology.orchestra.run.smithi167.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 525 kB/s | 31 kB 00:00 2024-09-16T15:36:58.613 INFO:teuthology.orchestra.run.smithi151.stdout:(80/157): python3-jsonpointer-2.0-4.el9.noarch. 567 kB/s | 19 kB 00:00 2024-09-16T15:36:58.635 INFO:teuthology.orchestra.run.smithi167.stdout:(139/157): python3-routes-2.5.1-5.el9.noarch.rp 3.6 MB/s | 188 kB 00:00 2024-09-16T15:36:58.647 INFO:teuthology.orchestra.run.smithi151.stdout:(81/157): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-09-16T15:36:58.677 INFO:teuthology.orchestra.run.smithi167.stdout:(140/157): python3-grpcio-1.46.7-10.el9.x86_64. 3.6 MB/s | 2.0 MB 00:00 2024-09-16T15:36:58.681 INFO:teuthology.orchestra.run.smithi151.stdout:(82/157): python3-markupsafe-1.1.1-12.el9.x86_6 518 kB/s | 35 kB 00:00 2024-09-16T15:36:58.702 INFO:teuthology.orchestra.run.smithi167.stdout:(141/157): python3-rsa-4.9-2.el9.noarch.rpm 588 kB/s | 59 kB 00:00 2024-09-16T15:36:58.728 INFO:teuthology.orchestra.run.smithi167.stdout:(142/157): python3-tempora-5.0.0-2.el9.noarch.r 387 kB/s | 36 kB 00:00 2024-09-16T15:36:58.731 INFO:teuthology.orchestra.run.smithi151.stdout:(83/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.9 MB/s | 250 kB 00:00 2024-09-16T15:36:58.753 INFO:teuthology.orchestra.run.smithi167.stdout:(143/157): python3-typing-extensions-4.12.2-2.e 990 kB/s | 74 kB 00:00 2024-09-16T15:36:58.778 INFO:teuthology.orchestra.run.smithi167.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-16T15:36:58.812 INFO:teuthology.orchestra.run.smithi167.stdout:(145/157): python3-webob-1.8.8-2.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-09-16T15:36:58.837 INFO:teuthology.orchestra.run.smithi167.stdout:(146/157): python3-zc-lockfile-2.0-10.el9.noarc 342 kB/s | 20 kB 00:00 2024-09-16T15:36:58.840 INFO:teuthology.orchestra.run.smithi151.stdout:(84/157): python3-babel-2.9.1-2.el9.noarch.rpm 12 MB/s | 6.0 MB 00:00 2024-09-16T15:36:58.865 INFO:teuthology.orchestra.run.smithi151.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 1.6 MB/s | 222 kB 00:00 2024-09-16T15:36:58.871 INFO:teuthology.orchestra.run.smithi167.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-16T15:36:58.890 INFO:teuthology.orchestra.run.smithi151.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-16T15:36:58.896 INFO:teuthology.orchestra.run.smithi167.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 2.9 MB/s | 427 kB 00:00 2024-09-16T15:36:58.916 INFO:teuthology.orchestra.run.smithi151.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 841 kB/s | 42 kB 00:00 2024-09-16T15:36:58.941 INFO:teuthology.orchestra.run.smithi151.stdout:(88/157): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-09-16T15:36:58.967 INFO:teuthology.orchestra.run.smithi151.stdout:(89/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-16T15:36:59.022 INFO:teuthology.orchestra.run.smithi167.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 8.6 MB/s | 1.6 MB 00:00 2024-09-16T15:36:59.059 INFO:teuthology.orchestra.run.smithi151.stdout:(90/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-09-16T15:36:59.084 INFO:teuthology.orchestra.run.smithi151.stdout:(91/157): python3-pytz-2021.1-5.el9.noarch.rpm 435 kB/s | 51 kB 00:00 2024-09-16T15:36:59.109 INFO:teuthology.orchestra.run.smithi151.stdout:(92/157): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-16T15:36:59.113 INFO:teuthology.orchestra.run.smithi167.stdout:(150/157): NetworkManager-1.49.90-1.el9.x86_64. 25 MB/s | 2.3 MB 00:00 2024-09-16T15:36:59.135 INFO:teuthology.orchestra.run.smithi151.stdout:(93/157): python3-pyasn1-modules-0.4.8-6.el9.no 1.4 MB/s | 279 kB 00:00 2024-09-16T15:36:59.160 INFO:teuthology.orchestra.run.smithi151.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 825 kB/s | 42 kB 00:00 2024-09-16T15:36:59.185 INFO:teuthology.orchestra.run.smithi151.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 877 kB/s | 44 kB 00:00 2024-09-16T15:36:59.211 INFO:teuthology.orchestra.run.smithi151.stdout:(96/157): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-16T15:36:59.214 INFO:teuthology.orchestra.run.smithi167.stdout:(151/157): librados2-19.3.0-4938.g33f95d2a.el9. 10 MB/s | 3.5 MB 00:00 2024-09-16T15:36:59.236 INFO:teuthology.orchestra.run.smithi151.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 714 kB/s | 36 kB 00:00 2024-09-16T15:36:59.256 INFO:teuthology.orchestra.run.smithi167.stdout:(152/157): NetworkManager-libnm-1.49.90-1.el9.x 13 MB/s | 1.8 MB 00:00 2024-09-16T15:36:59.262 INFO:teuthology.orchestra.run.smithi151.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 940 kB/s | 47 kB 00:00 2024-09-16T15:36:59.281 INFO:teuthology.orchestra.run.smithi167.stdout:(153/157): NetworkManager-team-1.49.90-1.el9.x8 610 kB/s | 41 kB 00:00 2024-09-16T15:36:59.304 INFO:teuthology.orchestra.run.smithi151.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 4.4 MB/s | 304 kB 00:00 2024-09-16T15:36:59.315 INFO:teuthology.orchestra.run.smithi167.stdout:(154/157): NetworkManager-tui-1.49.90-1.el9.x86 4.1 MB/s | 249 kB 00:00 2024-09-16T15:36:59.321 INFO:teuthology.orchestra.run.smithi151.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 987 kB/s | 58 kB 00:00 2024-09-16T15:36:59.366 INFO:teuthology.orchestra.run.smithi151.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.5 MB/s | 96 kB 00:00 2024-09-16T15:36:59.388 INFO:teuthology.orchestra.run.smithi151.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 952 kB/s | 64 kB 00:00 2024-09-16T15:36:59.388 INFO:teuthology.orchestra.run.smithi167.stdout:(155/157): openssl-3.2.2-6.el9.x86_64.rpm 13 MB/s | 1.4 MB 00:00 2024-09-16T15:36:59.405 INFO:teuthology.orchestra.run.smithi151.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-09-16T15:36:59.465 INFO:teuthology.orchestra.run.smithi167.stdout:(156/157): librbd1-19.3.0-4938.g33f95d2a.el9.x8 5.7 MB/s | 3.2 MB 00:00 2024-09-16T15:36:59.516 INFO:teuthology.orchestra.run.smithi167.stdout:(157/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 12 MB/s | 2.4 MB 00:00 2024-09-16T15:36:59.521 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2024-09-16T15:36:59.522 INFO:teuthology.orchestra.run.smithi167.stdout:Total 23 MB/s | 235 MB 00:10 2024-09-16T15:36:59.547 INFO:teuthology.orchestra.run.smithi151.stdout:(104/157): lua-devel-5.4.4-4.el9.x86_64.rpm 140 kB/s | 22 kB 00:00 2024-09-16T15:36:59.764 INFO:teuthology.orchestra.run.smithi151.stdout:(105/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 24 MB/s | 16 MB 00:00 2024-09-16T15:36:59.823 INFO:teuthology.orchestra.run.smithi151.stdout:(106/157): protobuf-compiler-3.14.0-13.el9.x86_ 2.0 MB/s | 863 kB 00:00 2024-09-16T15:36:59.882 INFO:teuthology.orchestra.run.smithi151.stdout:(107/157): abseil-cpp-20211102.0-3.el9.x86_64.r 1.6 MB/s | 548 kB 00:00 2024-09-16T15:36:59.907 INFO:teuthology.orchestra.run.smithi151.stdout:(108/157): grpc-data-1.46.7-10.el9.noarch.rpm 238 kB/s | 19 kB 00:00 2024-09-16T15:36:59.941 INFO:teuthology.orchestra.run.smithi151.stdout:(109/157): gperftools-libs-2.9.1-3.el9.x86_64.r 1.8 MB/s | 308 kB 00:00 2024-09-16T15:36:59.966 INFO:teuthology.orchestra.run.smithi151.stdout:(110/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 443 kB/s | 26 kB 00:00 2024-09-16T15:36:59.991 INFO:teuthology.orchestra.run.smithi151.stdout:(111/157): liboath-2.6.7-2.el9.x86_64.rpm 967 kB/s | 49 kB 00:00 2024-09-16T15:37:00.017 INFO:teuthology.orchestra.run.smithi151.stdout:(112/157): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-16T15:37:00.050 INFO:teuthology.orchestra.run.smithi151.stdout:(113/157): luarocks-3.9.2-5.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-09-16T15:37:00.143 INFO:teuthology.orchestra.run.smithi151.stdout:(114/157): python3-asyncssh-2.13.2-5.el9.noarch 5.8 MB/s | 548 kB 00:00 2024-09-16T15:37:00.196 INFO:teuthology.orchestra.run.smithi151.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 815 kB/s | 43 kB 00:00 2024-09-16T15:37:00.235 INFO:teuthology.orchestra.run.smithi151.stdout:(116/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.8 MB/s | 839 kB 00:00 2024-09-16T15:37:00.260 INFO:teuthology.orchestra.run.smithi151.stdout:(117/157): python3-cachetools-4.2.4-1.el9.noarc 508 kB/s | 32 kB 00:00 2024-09-16T15:37:00.285 INFO:teuthology.orchestra.run.smithi151.stdout:(118/157): python3-certifi-2023.05.07-4.el9.noa 281 kB/s | 14 kB 00:00 2024-09-16T15:37:00.311 INFO:teuthology.orchestra.run.smithi151.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-16T15:37:00.353 INFO:teuthology.orchestra.run.smithi151.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-09-16T15:37:00.378 INFO:teuthology.orchestra.run.smithi151.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-09-16T15:37:00.417 INFO:teuthology.orchestra.run.smithi151.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 3.4 MB/s | 220 kB 00:00 2024-09-16T15:37:00.510 INFO:teuthology.orchestra.run.smithi151.stdout:(123/157): libarrow-9.0.0-11.el9.x86_64.rpm 7.0 MB/s | 4.4 MB 00:00 2024-09-16T15:37:00.535 INFO:teuthology.orchestra.run.smithi151.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 429 kB/s | 11 kB 00:00 2024-09-16T15:37:00.560 INFO:teuthology.orchestra.run.smithi151.stdout:(125/157): python3-grpcio-tools-1.46.7-10.el9.x 1.0 MB/s | 144 kB 00:00 2024-09-16T15:37:00.586 INFO:teuthology.orchestra.run.smithi151.stdout:(126/157): python3-jaraco-classes-3.2.1-5.el9.n 351 kB/s | 18 kB 00:00 2024-09-16T15:37:00.603 INFO:teuthology.orchestra.run.smithi151.stdout:(127/157): python3-jaraco-collections-3.0.0-8.e 550 kB/s | 23 kB 00:00 2024-09-16T15:37:00.628 INFO:teuthology.orchestra.run.smithi151.stdout:(128/157): python3-jaraco-functools-3.5.0-2.el9 464 kB/s | 19 kB 00:00 2024-09-16T15:37:00.645 INFO:teuthology.orchestra.run.smithi151.stdout:(129/157): python3-jaraco-text-3.2.0-6.el9.noar 469 kB/s | 20 kB 00:00 2024-09-16T15:37:00.670 INFO:teuthology.orchestra.run.smithi151.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-16T15:37:00.704 INFO:teuthology.orchestra.run.smithi151.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-09-16T15:37:00.738 INFO:teuthology.orchestra.run.smithi151.stdout:(132/157): python3-kubernetes-26.1.0-2.el9.noar 9.3 MB/s | 1.0 MB 00:00 2024-09-16T15:37:00.763 INFO:teuthology.orchestra.run.smithi151.stdout:(133/157): python3-natsort-7.1.1-5.el9.noarch.r 980 kB/s | 58 kB 00:00 2024-09-16T15:37:00.788 INFO:teuthology.orchestra.run.smithi151.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 664 kB/s | 16 kB 00:00 2024-09-16T15:37:00.814 INFO:teuthology.orchestra.run.smithi151.stdout:(135/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-09-16T15:37:00.847 INFO:teuthology.orchestra.run.smithi151.stdout:(136/157): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2024-09-16T15:37:00.873 INFO:teuthology.orchestra.run.smithi151.stdout:(137/157): python3-pyperclip-1.8.2-2.el9.noarch 481 kB/s | 28 kB 00:00 2024-09-16T15:37:00.898 INFO:teuthology.orchestra.run.smithi151.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 608 kB/s | 31 kB 00:00 2024-09-16T15:37:00.949 INFO:teuthology.orchestra.run.smithi151.stdout:(139/157): python3-grpcio-1.46.7-10.el9.x86_64. 3.6 MB/s | 2.0 MB 00:00 2024-09-16T15:37:01.041 INFO:teuthology.orchestra.run.smithi151.stdout:(140/157): python3-rsa-4.9-2.el9.noarch.rpm 415 kB/s | 59 kB 00:00 2024-09-16T15:37:01.066 INFO:teuthology.orchestra.run.smithi151.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 305 kB/s | 36 kB 00:00 2024-09-16T15:37:01.091 INFO:teuthology.orchestra.run.smithi151.stdout:(142/157): python3-routes-2.5.1-5.el9.noarch.rp 863 kB/s | 188 kB 00:00 2024-09-16T15:37:01.125 INFO:teuthology.orchestra.run.smithi151.stdout:(143/157): python3-typing-extensions-4.12.2-2.e 889 kB/s | 74 kB 00:00 2024-09-16T15:37:01.151 INFO:teuthology.orchestra.run.smithi151.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-09-16T15:37:01.176 INFO:teuthology.orchestra.run.smithi151.stdout:(145/157): python3-webob-1.8.8-2.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-09-16T15:37:01.201 INFO:teuthology.orchestra.run.smithi151.stdout:(146/157): python3-zc-lockfile-2.0-10.el9.noarc 396 kB/s | 20 kB 00:00 2024-09-16T15:37:01.227 INFO:teuthology.orchestra.run.smithi151.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-16T15:37:01.260 INFO:teuthology.orchestra.run.smithi151.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-09-16T15:37:01.353 INFO:teuthology.orchestra.run.smithi151.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-09-16T15:37:01.528 INFO:teuthology.orchestra.run.smithi151.stdout:(150/157): librados2-19.3.0-4938.g33f95d2a.el9. 12 MB/s | 3.5 MB 00:00 2024-09-16T15:37:01.670 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2024-09-16T15:37:01.695 INFO:teuthology.orchestra.run.smithi151.stdout:(151/157): librbd1-19.3.0-4938.g33f95d2a.el9.x8 7.5 MB/s | 3.2 MB 00:00 2024-09-16T15:37:01.804 INFO:teuthology.orchestra.run.smithi151.stdout:(152/157): NetworkManager-team-1.49.90-1.el9.x8 376 kB/s | 41 kB 00:00 2024-09-16T15:37:01.859 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2024-09-16T15:37:01.860 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2024-09-16T15:37:01.946 INFO:teuthology.orchestra.run.smithi151.stdout:(153/157): NetworkManager-tui-1.49.90-1.el9.x86 1.7 MB/s | 249 kB 00:00 2024-09-16T15:37:02.029 INFO:teuthology.orchestra.run.smithi151.stdout:(154/157): NetworkManager-1.49.90-1.el9.x86_64. 3.4 MB/s | 2.3 MB 00:00 2024-09-16T15:37:02.072 INFO:teuthology.orchestra.run.smithi151.stdout:(155/157): NetworkManager-libnm-1.49.90-1.el9.x 3.3 MB/s | 1.8 MB 00:00 2024-09-16T15:37:02.364 INFO:teuthology.orchestra.run.smithi151.stdout:(156/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 7.2 MB/s | 2.4 MB 00:00 2024-09-16T15:37:02.397 INFO:teuthology.orchestra.run.smithi151.stdout:(157/157): openssl-3.2.2-6.el9.x86_64.rpm 3.0 MB/s | 1.4 MB 00:00 2024-09-16T15:37:02.403 INFO:teuthology.orchestra.run.smithi151.stdout:-------------------------------------------------------------------------------- 2024-09-16T15:37:02.403 INFO:teuthology.orchestra.run.smithi151.stdout:Total 20 MB/s | 235 MB 00:11 2024-09-16T15:37:04.289 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2024-09-16T15:37:04.290 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2024-09-16T15:37:04.490 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction check 2024-09-16T15:37:04.675 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction check succeeded. 2024-09-16T15:37:04.675 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction test 2024-09-16T15:37:07.099 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction test succeeded. 2024-09-16T15:37:07.099 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction 2024-09-16T15:37:11.486 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2024-09-16T15:37:11.713 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-16T15:37:11.763 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-16T15:37:11.847 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-16T15:37:11.917 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-16T15:37:11.955 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-16T15:37:12.017 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-16T15:37:12.379 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-16T15:37:12.498 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : librados2-2:19.3.0-4938.g33f95d2a.el9.x86_64 8/165 2024-09-16T15:37:12.643 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librados2-2:19.3.0-4938.g33f95d2a.el9.x86_64 8/165 2024-09-16T15:37:12.674 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libcephfs2-2:19.3.0-4938.g33f95d2a.el9.x86_64 9/165 2024-09-16T15:37:12.796 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libcephfs2-2:19.3.0-4938.g33f95d2a.el9.x86_64 9/165 2024-09-16T15:37:12.830 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rados-2:19.3.0-4938.g33f95d2a.el9.x86_64 10/165 2024-09-16T15:37:12.870 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-16T15:37:13.091 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-16T15:37:13.161 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-16T15:37:13.196 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libcephsqlite-2:19.3.0-4938.g33f95d2a.el9.x86_64 14/165 2024-09-16T15:37:13.322 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4938.g33f95d2a.el9.x86_64 14/165 2024-09-16T15:37:13.356 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libradosstriper1-2:19.3.0-4938.g33f95d2a.el9.x86 15/165 2024-09-16T15:37:13.421 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4938.g33f95d2a.el9.x86 15/165 2024-09-16T15:37:13.457 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-16T15:37:13.501 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-16T15:37:13.743 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-16T15:37:13.925 INFO:teuthology.orchestra.run.smithi151.stdout: Preparing : 1/1 2024-09-16T15:37:13.947 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-16T15:37:14.078 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-16T15:37:14.114 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 18/165 2024-09-16T15:37:14.129 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-16T15:37:14.195 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-16T15:37:14.247 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-16T15:37:14.256 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-16T15:37:14.288 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-16T15:37:14.345 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-16T15:37:14.374 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-16T15:37:14.657 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-16T15:37:14.712 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-16T15:37:14.803 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-16T15:37:14.814 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : librados2-2:19.3.0-4938.g33f95d2a.el9.x86_64 8/165 2024-09-16T15:37:14.838 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-16T15:37:14.882 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-16T15:37:14.943 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: librados2-2:19.3.0-4938.g33f95d2a.el9.x86_64 8/165 2024-09-16T15:37:14.964 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-16T15:37:14.974 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libcephfs2-2:19.3.0-4938.g33f95d2a.el9.x86_64 9/165 2024-09-16T15:37:15.019 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-16T15:37:15.057 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-16T15:37:15.079 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: libcephfs2-2:19.3.0-4938.g33f95d2a.el9.x86_64 9/165 2024-09-16T15:37:15.112 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-16T15:37:15.119 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-rados-2:19.3.0-4938.g33f95d2a.el9.x86_64 10/165 2024-09-16T15:37:15.162 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-16T15:37:15.167 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-16T15:37:15.209 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-16T15:37:15.298 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-16T15:37:15.362 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-16T15:37:15.377 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-16T15:37:15.397 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-16T15:37:15.443 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-16T15:37:15.445 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-16T15:37:15.472 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libcephsqlite-2:19.3.0-4938.g33f95d2a.el9.x86_64 14/165 2024-09-16T15:37:15.529 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-16T15:37:15.567 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-ceph-common-2:19.3.0-4938.g33f95d2a.el9. 34/165 2024-09-16T15:37:15.598 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4938.g33f95d2a.el9.x86_64 14/165 2024-09-16T15:37:15.611 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-ceph-argparse-2:19.3.0-4938.g33f95d2a.el 35/165 2024-09-16T15:37:15.624 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libradosstriper1-2:19.3.0-4938.g33f95d2a.el9.x86 15/165 2024-09-16T15:37:15.639 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cephfs-2:19.3.0-4938.g33f95d2a.el9.x86_6 36/165 2024-09-16T15:37:15.714 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4938.g33f95d2a.el9.x86 15/165 2024-09-16T15:37:15.738 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-16T15:37:15.741 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-16T15:37:15.785 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-16T15:37:15.800 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-16T15:37:15.890 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-16T15:37:16.019 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-16T15:37:16.075 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-16T15:37:16.145 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-16T15:37:16.211 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-16T15:37:16.408 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 18/165 2024-09-16T15:37:16.540 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-16T15:37:16.643 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-16T15:37:16.785 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-16T15:37:16.888 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-16T15:37:16.944 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-16T15:37:17.104 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-16T15:37:17.158 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-16T15:37:17.202 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-16T15:37:17.284 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-16T15:37:17.300 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-16T15:37:17.338 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-16T15:37:17.343 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-16T15:37:17.377 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-16T15:37:17.382 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librados-devel-2:19.3.0-4938.g33f95d2a.el9.x86_6 46/165 2024-09-16T15:37:17.432 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-16T15:37:17.435 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-16T15:37:17.471 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-16T15:37:17.505 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-16T15:37:17.593 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-16T15:37:17.638 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-16T15:37:17.649 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-16T15:37:17.693 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-16T15:37:17.730 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-16T15:37:17.812 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-16T15:37:17.845 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-16T15:37:17.916 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-16T15:37:18.000 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-16T15:37:18.040 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-16T15:37:18.109 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-16T15:37:18.178 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-16T15:37:18.211 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-16T15:37:18.226 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-16T15:37:18.247 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-ceph-common-2:19.3.0-4938.g33f95d2a.el9. 34/165 2024-09-16T15:37:18.272 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-16T15:37:18.290 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-ceph-argparse-2:19.3.0-4938.g33f95d2a.el 35/165 2024-09-16T15:37:18.319 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cephfs-2:19.3.0-4938.g33f95d2a.el9.x86_6 36/165 2024-09-16T15:37:18.322 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-16T15:37:18.410 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-16T15:37:18.464 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-16T15:37:18.496 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-16T15:37:18.547 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-16T15:37:18.554 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-16T15:37:18.590 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-16T15:37:18.706 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-16T15:37:18.721 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-16T15:37:18.756 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-16T15:37:18.792 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-16T15:37:18.804 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-16T15:37:18.848 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-16T15:37:18.880 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-16T15:37:19.023 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-16T15:37:19.187 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-16T15:37:19.227 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-16T15:37:19.271 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-16T15:37:19.347 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-16T15:37:19.389 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-16T15:37:19.428 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-16T15:37:19.456 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-16T15:37:19.474 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-16T15:37:19.509 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-16T15:37:19.542 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-16T15:37:19.585 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-16T15:37:19.626 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-16T15:37:19.627 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-16T15:37:19.680 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-16T15:37:19.724 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-16T15:37:19.726 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-16T15:37:19.762 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : librados-devel-2:19.3.0-4938.g33f95d2a.el9.x86_6 46/165 2024-09-16T15:37:19.786 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-16T15:37:19.822 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-16T15:37:19.823 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-16T15:37:19.952 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-16T15:37:20.020 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-16T15:37:20.091 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-16T15:37:20.199 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-16T15:37:20.279 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-16T15:37:20.344 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-16T15:37:20.363 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-16T15:37:20.411 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-16T15:37:20.455 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-16T15:37:20.539 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-16T15:37:20.578 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-16T15:37:20.624 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-16T15:37:20.674 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-16T15:37:20.848 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-16T15:37:20.900 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-16T15:37:20.943 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-16T15:37:21.070 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-16T15:37:21.126 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-16T15:37:21.169 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-16T15:37:21.198 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-16T15:37:21.223 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-16T15:37:21.229 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-16T15:37:21.288 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-16T15:37:21.321 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-16T15:37:21.347 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-16T15:37:21.466 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-16T15:37:21.529 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-16T15:37:21.538 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-16T15:37:21.575 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-16T15:37:21.599 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-16T15:37:21.611 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-16T15:37:21.656 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-16T15:37:21.695 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-16T15:37:21.703 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-16T15:37:21.741 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-16T15:37:21.742 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-16T15:37:21.780 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-16T15:37:21.815 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-16T15:37:21.848 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-16T15:37:21.901 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-16T15:37:21.938 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-16T15:37:22.004 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-16T15:37:22.038 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-16T15:37:22.056 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-16T15:37:22.124 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-16T15:37:22.136 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-16T15:37:22.176 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-16T15:37:22.299 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-16T15:37:22.365 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-16T15:37:22.966 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-16T15:37:23.109 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-16T15:37:23.255 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-16T15:37:23.310 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-16T15:37:23.343 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-16T15:37:23.458 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-16T15:37:23.527 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-16T15:37:23.581 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-16T15:37:23.676 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-16T15:37:23.722 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-16T15:37:23.962 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-16T15:37:24.003 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-16T15:37:24.099 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-16T15:37:24.691 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-16T15:37:24.795 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-16T15:37:24.830 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-16T15:37:24.864 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-16T15:37:24.915 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-16T15:37:25.051 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-16T15:37:25.211 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-16T15:37:25.267 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : librbd1-2:19.3.0-4938.g33f95d2a.el9.x86_64 102/165 2024-09-16T15:37:25.379 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librbd1-2:19.3.0-4938.g33f95d2a.el9.x86_64 102/165 2024-09-16T15:37:25.423 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rbd-2:19.3.0-4938.g33f95d2a.el9.x86_64 103/165 2024-09-16T15:37:25.472 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-16T15:37:26.016 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-16T15:37:26.054 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librgw2-2:19.3.0-4938.g33f95d2a.el9.x86_64 106/165 2024-09-16T15:37:26.145 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librgw2-2:19.3.0-4938.g33f95d2a.el9.x86_64 106/165 2024-09-16T15:37:26.189 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rgw-2:19.3.0-4938.g33f95d2a.el9.x86_64 107/165 2024-09-16T15:37:26.610 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-16T15:37:26.714 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-16T15:37:26.750 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-16T15:37:26.783 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-16T15:37:27.132 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-16T15:37:27.187 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : librbd1-2:19.3.0-4938.g33f95d2a.el9.x86_64 102/165 2024-09-16T15:37:27.283 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: librbd1-2:19.3.0-4938.g33f95d2a.el9.x86_64 102/165 2024-09-16T15:37:27.327 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-rbd-2:19.3.0-4938.g33f95d2a.el9.x86_64 103/165 2024-09-16T15:37:27.382 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-16T15:37:27.919 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-16T15:37:27.974 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : librgw2-2:19.3.0-4938.g33f95d2a.el9.x86_64 106/165 2024-09-16T15:37:28.065 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: librgw2-2:19.3.0-4938.g33f95d2a.el9.x86_64 106/165 2024-09-16T15:37:28.093 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-rgw-2:19.3.0-4938.g33f95d2a.el9.x86_64 107/165 2024-09-16T15:37:28.781 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-common-2:19.3.0-4938.g33f95d2a.el9.x86_64 108/165 2024-09-16T15:37:28.828 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-common-2:19.3.0-4938.g33f95d2a.el9.x86_64 108/165 2024-09-16T15:37:29.467 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-common-2:19.3.0-4938.g33f95d2a.el9.x86_64 108/165 2024-09-16T15:37:29.523 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-base-2:19.3.0-4938.g33f95d2a.el9.x86_64 109/165 2024-09-16T15:37:29.671 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-base-2:19.3.0-4938.g33f95d2a.el9.x86_64 109/165 2024-09-16T15:37:29.671 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-16T15:37:29.671 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-16T15:37:29.672 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-16T15:37:29.716 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-selinux-2:19.3.0-4938.g33f95d2a.el9.x86_64 110/165 2024-09-16T15:37:30.644 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-common-2:19.3.0-4938.g33f95d2a.el9.x86_64 108/165 2024-09-16T15:37:30.708 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-common-2:19.3.0-4938.g33f95d2a.el9.x86_64 108/165 2024-09-16T15:37:31.343 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-common-2:19.3.0-4938.g33f95d2a.el9.x86_64 108/165 2024-09-16T15:37:31.412 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-base-2:19.3.0-4938.g33f95d2a.el9.x86_64 109/165 2024-09-16T15:37:31.540 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-base-2:19.3.0-4938.g33f95d2a.el9.x86_64 109/165 2024-09-16T15:37:31.541 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-16T15:37:31.541 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-16T15:37:31.541 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:37:31.581 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-selinux-2:19.3.0-4938.g33f95d2a.el9.x86_64 110/165 2024-09-16T15:37:42.554 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4938.g33f95d2a.el9.x86_64 110/165 2024-09-16T15:37:42.554 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /sys 2024-09-16T15:37:42.554 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /proc 2024-09-16T15:37:42.554 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /mnt 2024-09-16T15:37:42.554 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /var/tmp 2024-09-16T15:37:42.554 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /home 2024-09-16T15:37:42.554 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /root 2024-09-16T15:37:42.554 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /tmp 2024-09-16T15:37:42.554 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-16T15:37:42.787 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-volume-2:19.3.0-4938.g33f95d2a.el9.noarch 111/165 2024-09-16T15:37:42.829 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-volume-2:19.3.0-4938.g33f95d2a.el9.noarch 111/165 2024-09-16T15:37:42.829 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T15:37:42.829 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-16T15:37:42.830 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-16T15:37:44.413 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-osd-2:19.3.0-4938.g33f95d2a.el9.x86_64 112/165 2024-09-16T15:37:44.471 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-osd-2:19.3.0-4938.g33f95d2a.el9.x86_64 112/165 2024-09-16T15:37:44.472 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T15:37:44.472 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-16T15:37:44.472 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T15:37:44.472 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T15:37:44.472 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-16T15:37:44.495 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4938.g33f95d2a.el9.x86_64 110/165 2024-09-16T15:37:44.495 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /sys 2024-09-16T15:37:44.495 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /proc 2024-09-16T15:37:44.495 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /mnt 2024-09-16T15:37:44.495 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /var/tmp 2024-09-16T15:37:44.495 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /home 2024-09-16T15:37:44.496 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /root 2024-09-16T15:37:44.496 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /tmp 2024-09-16T15:37:44.496 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:37:44.722 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mds-2:19.3.0-4938.g33f95d2a.el9.x86_64 113/165 2024-09-16T15:37:44.722 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-volume-2:19.3.0-4938.g33f95d2a.el9.noarch 111/165 2024-09-16T15:37:44.759 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-volume-2:19.3.0-4938.g33f95d2a.el9.noarch 111/165 2024-09-16T15:37:44.759 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T15:37:44.759 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-16T15:37:44.759 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:37:44.773 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mds-2:19.3.0-4938.g33f95d2a.el9.x86_64 113/165 2024-09-16T15:37:44.773 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T15:37:44.773 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-16T15:37:44.773 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T15:37:44.773 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T15:37:44.773 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-16T15:37:45.264 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mon-2:19.3.0-4938.g33f95d2a.el9.x86_64 114/165 2024-09-16T15:37:45.315 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mon-2:19.3.0-4938.g33f95d2a.el9.x86_64 114/165 2024-09-16T15:37:45.315 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T15:37:45.315 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-16T15:37:45.315 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T15:37:45.315 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T15:37:45.315 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-16T15:37:45.447 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-16T15:37:45.517 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-16T15:37:45.614 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-16T15:37:45.848 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-16T15:37:45.919 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-16T15:37:46.038 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-16T15:37:46.166 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-16T15:37:46.331 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-16T15:37:46.341 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-osd-2:19.3.0-4938.g33f95d2a.el9.x86_64 112/165 2024-09-16T15:37:46.399 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-osd-2:19.3.0-4938.g33f95d2a.el9.x86_64 112/165 2024-09-16T15:37:46.399 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T15:37:46.399 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-16T15:37:46.399 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T15:37:46.400 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T15:37:46.400 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:37:46.411 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-16T15:37:46.451 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-16T15:37:46.484 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-16T15:37:46.650 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mds-2:19.3.0-4938.g33f95d2a.el9.x86_64 113/165 2024-09-16T15:37:46.701 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mds-2:19.3.0-4938.g33f95d2a.el9.x86_64 113/165 2024-09-16T15:37:46.702 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T15:37:46.702 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-16T15:37:46.702 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T15:37:46.702 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T15:37:46.702 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:37:47.019 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: cephadm-2:19.3.0-4938.g33f95d2a.el9.noarch 126/165 2024-09-16T15:37:47.050 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cephadm-2:19.3.0-4938.g33f95d2a.el9.noarch 126/165 2024-09-16T15:37:47.109 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4938.g33f95d2a.e 127/165 2024-09-16T15:37:47.193 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mon-2:19.3.0-4938.g33f95d2a.el9.x86_64 114/165 2024-09-16T15:37:47.198 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4938.g33f95d2a. 128/165 2024-09-16T15:37:47.231 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4938.g33f95d2a.el9.noa 129/165 2024-09-16T15:37:47.244 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mon-2:19.3.0-4938.g33f95d2a.el9.x86_64 114/165 2024-09-16T15:37:47.244 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T15:37:47.244 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-16T15:37:47.244 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T15:37:47.244 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T15:37:47.244 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:37:47.377 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-16T15:37:47.462 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-16T15:37:47.527 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-16T15:37:47.753 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-16T15:37:47.807 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-16T15:37:47.892 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-16T15:37:48.002 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-16T15:37:48.177 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-16T15:37:48.237 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-16T15:37:48.289 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-16T15:37:48.315 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-16T15:37:48.849 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: cephadm-2:19.3.0-4938.g33f95d2a.el9.noarch 126/165 2024-09-16T15:37:48.883 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : cephadm-2:19.3.0-4938.g33f95d2a.el9.noarch 126/165 2024-09-16T15:37:48.925 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4938.g33f95d2a.e 127/165 2024-09-16T15:37:49.032 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4938.g33f95d2a. 128/165 2024-09-16T15:37:49.092 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4938.g33f95d2a.el9.noa 129/165 2024-09-16T15:37:50.874 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4938.g33f95d2a.el9.noa 129/165 2024-09-16T15:37:50.938 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4938.g33f95d2a.el9.n 130/165 2024-09-16T15:37:52.029 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4938.g33f95d2a.el9.n 130/165 2024-09-16T15:37:52.063 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4938.g33f 131/165 2024-09-16T15:37:52.217 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4938.g33f 131/165 2024-09-16T15:37:52.396 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4938.g33f95d2a.el 132/165 2024-09-16T15:37:52.457 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-2:19.3.0-4938.g33f95d2a.el9.x86_64 133/165 2024-09-16T15:37:52.510 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4938.g33f95d2a.el9.x86_64 133/165 2024-09-16T15:37:52.510 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T15:37:52.510 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-16T15:37:52.510 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T15:37:52.510 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T15:37:52.510 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-16T15:37:52.566 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-rook-2:19.3.0-4938.g33f95d2a.el9.noarch 134/165 2024-09-16T15:37:52.592 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4938.g33f95d2a.el9.noarch 134/165 2024-09-16T15:37:52.699 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-2:19.3.0-4938.g33f95d2a.el9.x86_64 135/165 2024-09-16T15:37:52.733 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4938.g33f95d2a.el9.noa 129/165 2024-09-16T15:37:52.759 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-fuse-2:19.3.0-4938.g33f95d2a.el9.x86_64 136/165 2024-09-16T15:37:52.806 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4938.g33f95d2a.el9.n 130/165 2024-09-16T15:37:53.797 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-16T15:37:53.827 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-radosgw-2:19.3.0-4938.g33f95d2a.el9.x86_64 138/165 2024-09-16T15:37:53.879 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4938.g33f95d2a.el9.x86_64 138/165 2024-09-16T15:37:53.880 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T15:37:53.880 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-16T15:37:53.880 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T15:37:53.880 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T15:37:53.880 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-16T15:37:53.893 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4938.g33f95d2a.el9.n 130/165 2024-09-16T15:37:53.931 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4938.g33f 131/165 2024-09-16T15:37:53.945 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4938.g33f95 139/165 2024-09-16T15:37:53.995 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4938.g33f95 139/165 2024-09-16T15:37:53.995 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T15:37:53.996 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-16T15:37:53.996 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-16T15:37:54.070 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cephfs-mirror-2:19.3.0-4938.g33f95d2a.el9.x86_64 140/165 2024-09-16T15:37:54.083 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4938.g33f 131/165 2024-09-16T15:37:54.122 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-4938.g33f95d2a.el9.x86_64 140/165 2024-09-16T15:37:54.122 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T15:37:54.122 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-16T15:37:54.122 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-16T15:37:54.122 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-16T15:37:54.122 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-16T15:37:54.263 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4938.g33f95d2a.el 132/165 2024-09-16T15:37:54.308 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mgr-2:19.3.0-4938.g33f95d2a.el9.x86_64 133/165 2024-09-16T15:37:54.360 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4938.g33f95d2a.el9.x86_64 133/165 2024-09-16T15:37:54.360 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T15:37:54.360 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-16T15:37:54.360 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T15:37:54.360 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T15:37:54.360 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:37:54.418 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mgr-rook-2:19.3.0-4938.g33f95d2a.el9.noarch 134/165 2024-09-16T15:37:54.444 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4938.g33f95d2a.el9.noarch 134/165 2024-09-16T15:37:54.471 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : rbd-mirror-2:19.3.0-4938.g33f95d2a.el9.x86_64 141/165 2024-09-16T15:37:54.523 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4938.g33f95d2a.el9.x86_64 141/165 2024-09-16T15:37:54.523 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T15:37:54.523 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-16T15:37:54.523 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T15:37:54.523 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T15:37:54.523 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-16T15:37:54.551 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-2:19.3.0-4938.g33f95d2a.el9.x86_64 135/165 2024-09-16T15:37:54.611 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-fuse-2:19.3.0-4938.g33f95d2a.el9.x86_64 136/165 2024-09-16T15:37:55.660 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-16T15:37:55.696 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-radosgw-2:19.3.0-4938.g33f95d2a.el9.x86_64 138/165 2024-09-16T15:37:55.748 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4938.g33f95d2a.el9.x86_64 138/165 2024-09-16T15:37:55.748 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T15:37:55.748 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-16T15:37:55.748 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T15:37:55.748 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T15:37:55.748 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:37:55.797 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4938.g33f95 139/165 2024-09-16T15:37:55.847 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4938.g33f95 139/165 2024-09-16T15:37:55.847 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T15:37:55.848 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-16T15:37:55.848 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:37:55.914 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : cephfs-mirror-2:19.3.0-4938.g33f95d2a.el9.x86_64 140/165 2024-09-16T15:37:55.965 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-4938.g33f95d2a.el9.x86_64 140/165 2024-09-16T15:37:55.965 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T15:37:55.965 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-16T15:37:55.965 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-16T15:37:55.965 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-16T15:37:55.965 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:37:56.324 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : rbd-mirror-2:19.3.0-4938.g33f95d2a.el9.x86_64 141/165 2024-09-16T15:37:56.375 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4938.g33f95d2a.el9.x86_64 141/165 2024-09-16T15:37:56.375 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T15:37:56.375 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-16T15:37:56.375 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T15:37:56.375 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T15:37:56.375 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:37:59.210 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-test-2:19.3.0-4938.g33f95d2a.el9.x86_64 142/165 2024-09-16T15:37:59.285 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : rbd-fuse-2:19.3.0-4938.g33f95d2a.el9.x86_64 143/165 2024-09-16T15:37:59.329 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : rbd-nbd-2:19.3.0-4938.g33f95d2a.el9.x86_64 144/165 2024-09-16T15:37:59.358 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-16T15:37:59.393 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-16T15:37:59.617 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cephfs-shell-2:19.3.0-4938.g33f95d2a.el9.x86_64 147/165 2024-09-16T15:37:59.699 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-16T15:37:59.729 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libcephfs-devel-2:19.3.0-4938.g33f95d2a.el9.x86_ 149/165 2024-09-16T15:37:59.761 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 150/165 2024-09-16T15:37:59.804 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 151/165 2024-09-16T15:38:00.547 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-16T15:38:00.595 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-16T15:38:00.637 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-16T15:38:01.056 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-test-2:19.3.0-4938.g33f95d2a.el9.x86_64 142/165 2024-09-16T15:38:01.097 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : rbd-fuse-2:19.3.0-4938.g33f95d2a.el9.x86_64 143/165 2024-09-16T15:38:01.141 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : rbd-nbd-2:19.3.0-4938.g33f95d2a.el9.x86_64 144/165 2024-09-16T15:38:01.170 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-16T15:38:01.206 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-16T15:38:01.435 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : cephfs-shell-2:19.3.0-4938.g33f95d2a.el9.x86_64 147/165 2024-09-16T15:38:01.488 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-16T15:38:01.517 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libcephfs-devel-2:19.3.0-4938.g33f95d2a.el9.x86_ 149/165 2024-09-16T15:38:01.549 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 150/165 2024-09-16T15:38:01.618 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 151/165 2024-09-16T15:38:01.911 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cephfs-top-2:19.3.0-4938.g33f95d2a.el9.noarch 153/165 2024-09-16T15:38:02.009 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-16T15:38:02.070 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-16T15:38:02.198 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-16T15:38:02.229 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-16T15:38:02.255 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-16T15:38:02.291 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-16T15:38:02.291 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-16T15:38:02.353 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-16T15:38:02.409 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-16T15:38:02.415 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-16T15:38:02.415 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-16T15:38:02.451 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-16T15:38:02.549 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-16T15:38:02.573 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-16T15:38:02.590 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-16T15:38:02.590 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-16T15:38:02.834 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-16T15:38:02.835 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-16T15:38:02.969 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-16T15:38:03.042 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-16T15:38:03.262 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-16T15:38:03.262 INFO:teuthology.orchestra.run.smithi167.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-09-16T15:38:03.262 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-16T15:38:03.755 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : cephfs-top-2:19.3.0-4938.g33f95d2a.el9.noarch 153/165 2024-09-16T15:38:03.869 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-16T15:38:03.925 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-16T15:38:04.037 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-16T15:38:04.073 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-16T15:38:04.109 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-16T15:38:04.168 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-16T15:38:04.168 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-16T15:38:04.276 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-16T15:38:04.276 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-16T15:38:04.377 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-16T15:38:04.401 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-16T15:38:04.417 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-16T15:38:04.417 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-16T15:38:04.662 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-16T15:38:04.662 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-16T15:38:04.756 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-16T15:38:04.831 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-16T15:38:05.058 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-16T15:38:05.058 INFO:teuthology.orchestra.run.smithi151.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-09-16T15:38:05.058 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:38:09.811 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-2:19.3.0-4938.g33f95d2a.el9.x86_64 1/165 2024-09-16T15:38:09.811 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-base-2:19.3.0-4938.g33f95d2a.el9.x86_64 2/165 2024-09-16T15:38:09.811 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-common-2:19.3.0-4938.g33f95d2a.el9.x86_64 3/165 2024-09-16T15:38:09.811 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-fuse-2:19.3.0-4938.g33f95d2a.el9.x86_64 4/165 2024-09-16T15:38:09.811 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4938.g33f95 5/165 2024-09-16T15:38:09.811 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mds-2:19.3.0-4938.g33f95d2a.el9.x86_64 6/165 2024-09-16T15:38:09.811 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-2:19.3.0-4938.g33f95d2a.el9.x86_64 7/165 2024-09-16T15:38:09.811 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mon-2:19.3.0-4938.g33f95d2a.el9.x86_64 8/165 2024-09-16T15:38:09.811 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-osd-2:19.3.0-4938.g33f95d2a.el9.x86_64 9/165 2024-09-16T15:38:09.812 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-radosgw-2:19.3.0-4938.g33f95d2a.el9.x86_64 10/165 2024-09-16T15:38:09.812 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-selinux-2:19.3.0-4938.g33f95d2a.el9.x86_64 11/165 2024-09-16T15:38:09.812 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-test-2:19.3.0-4938.g33f95d2a.el9.x86_64 12/165 2024-09-16T15:38:09.812 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cephfs-mirror-2:19.3.0-4938.g33f95d2a.el9.x86_64 13/165 2024-09-16T15:38:09.812 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cephfs-shell-2:19.3.0-4938.g33f95d2a.el9.x86_64 14/165 2024-09-16T15:38:09.812 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libcephfs-devel-2:19.3.0-4938.g33f95d2a.el9.x86_ 15/165 2024-09-16T15:38:09.812 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libcephfs2-2:19.3.0-4938.g33f95d2a.el9.x86_64 16/165 2024-09-16T15:38:09.812 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libcephsqlite-2:19.3.0-4938.g33f95d2a.el9.x86_64 17/165 2024-09-16T15:38:09.812 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librados-devel-2:19.3.0-4938.g33f95d2a.el9.x86_6 18/165 2024-09-16T15:38:09.812 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libradosstriper1-2:19.3.0-4938.g33f95d2a.el9.x86 19/165 2024-09-16T15:38:09.812 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librgw2-2:19.3.0-4938.g33f95d2a.el9.x86_64 20/165 2024-09-16T15:38:09.812 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4938.g33f95d2a.el 21/165 2024-09-16T15:38:09.812 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-ceph-common-2:19.3.0-4938.g33f95d2a.el9. 22/165 2024-09-16T15:38:09.812 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cephfs-2:19.3.0-4938.g33f95d2a.el9.x86_6 23/165 2024-09-16T15:38:09.812 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rados-2:19.3.0-4938.g33f95d2a.el9.x86_64 24/165 2024-09-16T15:38:09.813 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rbd-2:19.3.0-4938.g33f95d2a.el9.x86_64 25/165 2024-09-16T15:38:09.813 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rgw-2:19.3.0-4938.g33f95d2a.el9.x86_64 26/165 2024-09-16T15:38:09.813 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : rbd-fuse-2:19.3.0-4938.g33f95d2a.el9.x86_64 27/165 2024-09-16T15:38:09.824 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : rbd-mirror-2:19.3.0-4938.g33f95d2a.el9.x86_64 28/165 2024-09-16T15:38:09.824 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : rbd-nbd-2:19.3.0-4938.g33f95d2a.el9.x86_64 29/165 2024-09-16T15:38:09.824 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4938.g33f95d2a. 30/165 2024-09-16T15:38:09.825 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4938.g33f95d2a.el9.noa 31/165 2024-09-16T15:38:09.825 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4938.g33f95d2a.el9.n 32/165 2024-09-16T15:38:09.825 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4938.g33f 33/165 2024-09-16T15:38:09.825 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4938.g33f95d2a.el 34/165 2024-09-16T15:38:09.825 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4938.g33f95d2a.el9.noarch 35/165 2024-09-16T15:38:09.825 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4938.g33f95d2a.e 36/165 2024-09-16T15:38:09.825 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-volume-2:19.3.0-4938.g33f95d2a.el9.noarch 37/165 2024-09-16T15:38:09.825 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cephadm-2:19.3.0-4938.g33f95d2a.el9.noarch 38/165 2024-09-16T15:38:09.825 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cephfs-top-2:19.3.0-4938.g33f95d2a.el9.noarch 39/165 2024-09-16T15:38:09.825 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-16T15:38:09.825 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-16T15:38:09.825 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-16T15:38:09.825 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-16T15:38:09.826 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-16T15:38:09.826 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-16T15:38:09.826 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-16T15:38:09.826 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-16T15:38:09.826 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-16T15:38:09.826 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-16T15:38:09.826 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-16T15:38:09.826 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-16T15:38:09.826 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-16T15:38:09.826 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-16T15:38:09.826 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-16T15:38:09.826 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-16T15:38:09.826 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-16T15:38:09.826 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-16T15:38:09.827 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-16T15:38:09.827 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-16T15:38:09.827 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-16T15:38:09.827 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-16T15:38:09.827 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-16T15:38:09.827 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-16T15:38:09.827 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-16T15:38:09.827 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-16T15:38:09.827 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-16T15:38:09.827 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-16T15:38:09.827 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-16T15:38:09.827 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-16T15:38:09.828 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-16T15:38:09.828 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-16T15:38:09.828 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-16T15:38:09.828 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-16T15:38:09.828 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-16T15:38:09.828 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-16T15:38:09.828 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-16T15:38:09.828 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-16T15:38:09.828 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-16T15:38:09.828 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-16T15:38:09.828 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-16T15:38:09.828 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-16T15:38:09.828 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-16T15:38:09.828 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-16T15:38:09.828 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-16T15:38:09.829 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-16T15:38:09.829 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-16T15:38:09.829 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-16T15:38:09.829 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-16T15:38:09.829 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-16T15:38:09.829 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-16T15:38:09.829 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-16T15:38:09.829 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-16T15:38:09.829 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-16T15:38:09.829 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-16T15:38:09.829 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-16T15:38:09.829 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-16T15:38:09.829 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-16T15:38:09.829 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-16T15:38:09.829 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-16T15:38:09.830 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-16T15:38:09.830 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-16T15:38:09.830 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-16T15:38:09.830 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-16T15:38:09.830 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-16T15:38:09.830 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-16T15:38:09.830 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-16T15:38:09.830 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-16T15:38:09.830 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-16T15:38:09.830 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-16T15:38:09.830 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-16T15:38:09.830 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-16T15:38:09.830 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-16T15:38:09.830 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-16T15:38:09.830 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-16T15:38:09.830 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-16T15:38:09.831 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-16T15:38:09.831 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-16T15:38:09.831 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-16T15:38:09.831 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-16T15:38:09.831 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-16T15:38:09.831 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-16T15:38:09.831 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-16T15:38:09.831 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-16T15:38:09.831 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-16T15:38:09.831 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-16T15:38:09.831 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-16T15:38:09.831 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-16T15:38:09.831 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-16T15:38:09.831 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-16T15:38:09.831 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-16T15:38:09.831 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-16T15:38:09.832 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-16T15:38:09.832 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-16T15:38:09.832 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-16T15:38:09.832 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-16T15:38:09.832 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-16T15:38:09.832 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-16T15:38:09.832 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-16T15:38:09.832 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-16T15:38:09.832 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-16T15:38:09.832 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-16T15:38:09.832 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-16T15:38:09.832 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-16T15:38:09.832 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-16T15:38:09.832 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-16T15:38:09.832 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-16T15:38:09.833 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-16T15:38:09.833 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-16T15:38:09.833 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-16T15:38:09.833 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librados2-2:19.3.0-4938.g33f95d2a.el9.x86_64 150/165 2024-09-16T15:38:09.833 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-16T15:38:09.833 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librbd1-2:19.3.0-4938.g33f95d2a.el9.x86_64 152/165 2024-09-16T15:38:09.833 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-16T15:38:09.833 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 154/165 2024-09-16T15:38:09.833 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-16T15:38:09.833 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 156/165 2024-09-16T15:38:09.833 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-16T15:38:09.833 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 158/165 2024-09-16T15:38:09.833 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-16T15:38:09.833 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 160/165 2024-09-16T15:38:09.833 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-16T15:38:09.834 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-16T15:38:09.834 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-16T15:38:09.834 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-16T15:38:11.485 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-2:19.3.0-4938.g33f95d2a.el9.x86_64 1/165 2024-09-16T15:38:11.485 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-base-2:19.3.0-4938.g33f95d2a.el9.x86_64 2/165 2024-09-16T15:38:11.485 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-common-2:19.3.0-4938.g33f95d2a.el9.x86_64 3/165 2024-09-16T15:38:11.485 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-fuse-2:19.3.0-4938.g33f95d2a.el9.x86_64 4/165 2024-09-16T15:38:11.485 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4938.g33f95 5/165 2024-09-16T15:38:11.485 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mds-2:19.3.0-4938.g33f95d2a.el9.x86_64 6/165 2024-09-16T15:38:11.485 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mgr-2:19.3.0-4938.g33f95d2a.el9.x86_64 7/165 2024-09-16T15:38:11.486 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mon-2:19.3.0-4938.g33f95d2a.el9.x86_64 8/165 2024-09-16T15:38:11.486 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-osd-2:19.3.0-4938.g33f95d2a.el9.x86_64 9/165 2024-09-16T15:38:11.486 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-radosgw-2:19.3.0-4938.g33f95d2a.el9.x86_64 10/165 2024-09-16T15:38:11.486 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-selinux-2:19.3.0-4938.g33f95d2a.el9.x86_64 11/165 2024-09-16T15:38:11.486 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-test-2:19.3.0-4938.g33f95d2a.el9.x86_64 12/165 2024-09-16T15:38:11.486 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : cephfs-mirror-2:19.3.0-4938.g33f95d2a.el9.x86_64 13/165 2024-09-16T15:38:11.486 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : cephfs-shell-2:19.3.0-4938.g33f95d2a.el9.x86_64 14/165 2024-09-16T15:38:11.486 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libcephfs-devel-2:19.3.0-4938.g33f95d2a.el9.x86_ 15/165 2024-09-16T15:38:11.486 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libcephfs2-2:19.3.0-4938.g33f95d2a.el9.x86_64 16/165 2024-09-16T15:38:11.486 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libcephsqlite-2:19.3.0-4938.g33f95d2a.el9.x86_64 17/165 2024-09-16T15:38:11.486 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librados-devel-2:19.3.0-4938.g33f95d2a.el9.x86_6 18/165 2024-09-16T15:38:11.486 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libradosstriper1-2:19.3.0-4938.g33f95d2a.el9.x86 19/165 2024-09-16T15:38:11.486 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librgw2-2:19.3.0-4938.g33f95d2a.el9.x86_64 20/165 2024-09-16T15:38:11.486 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4938.g33f95d2a.el 21/165 2024-09-16T15:38:11.487 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-ceph-common-2:19.3.0-4938.g33f95d2a.el9. 22/165 2024-09-16T15:38:11.487 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cephfs-2:19.3.0-4938.g33f95d2a.el9.x86_6 23/165 2024-09-16T15:38:11.487 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-rados-2:19.3.0-4938.g33f95d2a.el9.x86_64 24/165 2024-09-16T15:38:11.487 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-rbd-2:19.3.0-4938.g33f95d2a.el9.x86_64 25/165 2024-09-16T15:38:11.487 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-rgw-2:19.3.0-4938.g33f95d2a.el9.x86_64 26/165 2024-09-16T15:38:11.487 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : rbd-fuse-2:19.3.0-4938.g33f95d2a.el9.x86_64 27/165 2024-09-16T15:38:11.487 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : rbd-mirror-2:19.3.0-4938.g33f95d2a.el9.x86_64 28/165 2024-09-16T15:38:11.487 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : rbd-nbd-2:19.3.0-4938.g33f95d2a.el9.x86_64 29/165 2024-09-16T15:38:11.487 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4938.g33f95d2a. 30/165 2024-09-16T15:38:11.499 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4938.g33f95d2a.el9.noa 31/165 2024-09-16T15:38:11.499 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4938.g33f95d2a.el9.n 32/165 2024-09-16T15:38:11.499 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4938.g33f 33/165 2024-09-16T15:38:11.499 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4938.g33f95d2a.el 34/165 2024-09-16T15:38:11.499 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4938.g33f95d2a.el9.noarch 35/165 2024-09-16T15:38:11.499 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4938.g33f95d2a.e 36/165 2024-09-16T15:38:11.499 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-volume-2:19.3.0-4938.g33f95d2a.el9.noarch 37/165 2024-09-16T15:38:11.499 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : cephadm-2:19.3.0-4938.g33f95d2a.el9.noarch 38/165 2024-09-16T15:38:11.499 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : cephfs-top-2:19.3.0-4938.g33f95d2a.el9.noarch 39/165 2024-09-16T15:38:11.499 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-16T15:38:11.499 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-16T15:38:11.500 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-16T15:38:11.500 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-16T15:38:11.500 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-16T15:38:11.500 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-16T15:38:11.500 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-16T15:38:11.500 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-16T15:38:11.500 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-16T15:38:11.500 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-16T15:38:11.500 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-16T15:38:11.500 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-16T15:38:11.500 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-16T15:38:11.500 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-16T15:38:11.500 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-16T15:38:11.500 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-16T15:38:11.501 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-16T15:38:11.501 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-16T15:38:11.501 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-16T15:38:11.501 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-16T15:38:11.501 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-16T15:38:11.501 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-16T15:38:11.501 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-16T15:38:11.501 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-16T15:38:11.501 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-16T15:38:11.501 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-16T15:38:11.501 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-16T15:38:11.501 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-16T15:38:11.501 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-16T15:38:11.501 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-16T15:38:11.502 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-16T15:38:11.502 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-16T15:38:11.502 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-16T15:38:11.502 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-16T15:38:11.502 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-16T15:38:11.502 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-16T15:38:11.502 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-16T15:38:11.502 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-16T15:38:11.502 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-16T15:38:11.502 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-16T15:38:11.502 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-16T15:38:11.502 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-16T15:38:11.502 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-16T15:38:11.502 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-16T15:38:11.503 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-16T15:38:11.503 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-16T15:38:11.503 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-16T15:38:11.503 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-16T15:38:11.503 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-16T15:38:11.503 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-16T15:38:11.503 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-16T15:38:11.503 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-16T15:38:11.503 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-16T15:38:11.503 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-16T15:38:11.503 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-16T15:38:11.503 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-16T15:38:11.503 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-16T15:38:11.503 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-16T15:38:11.503 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-16T15:38:11.504 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-16T15:38:11.504 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-16T15:38:11.504 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-16T15:38:11.504 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-16T15:38:11.504 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-16T15:38:11.504 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-16T15:38:11.504 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-16T15:38:11.504 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-16T15:38:11.504 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-16T15:38:11.504 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-16T15:38:11.504 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-16T15:38:11.504 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-16T15:38:11.504 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-16T15:38:11.505 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-16T15:38:11.505 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-16T15:38:11.505 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-16T15:38:11.505 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-16T15:38:11.505 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-16T15:38:11.505 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-16T15:38:11.505 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-16T15:38:11.505 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-16T15:38:11.505 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-16T15:38:11.505 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-16T15:38:11.505 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-16T15:38:11.505 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-16T15:38:11.505 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-16T15:38:11.505 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-16T15:38:11.505 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-16T15:38:11.506 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-16T15:38:11.506 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-16T15:38:11.506 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-16T15:38:11.506 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-16T15:38:11.506 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-16T15:38:11.506 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-16T15:38:11.506 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-16T15:38:11.506 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-16T15:38:11.506 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-16T15:38:11.506 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-16T15:38:11.506 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-16T15:38:11.506 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-16T15:38:11.506 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-16T15:38:11.506 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-16T15:38:11.507 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-16T15:38:11.507 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-16T15:38:11.507 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-16T15:38:11.507 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-16T15:38:11.507 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-16T15:38:11.507 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-16T15:38:11.507 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-16T15:38:11.507 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-16T15:38:11.507 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-16T15:38:11.507 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librados2-2:19.3.0-4938.g33f95d2a.el9.x86_64 150/165 2024-09-16T15:38:11.507 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-16T15:38:11.507 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librbd1-2:19.3.0-4938.g33f95d2a.el9.x86_64 152/165 2024-09-16T15:38:11.507 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-16T15:38:11.507 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 154/165 2024-09-16T15:38:11.507 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-16T15:38:11.508 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 156/165 2024-09-16T15:38:11.508 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-16T15:38:11.508 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 158/165 2024-09-16T15:38:11.508 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-16T15:38:11.508 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 160/165 2024-09-16T15:38:11.508 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-16T15:38:11.508 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-16T15:38:11.508 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-16T15:38:11.508 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-16T15:38:12.111 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-16T15:38:12.112 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-16T15:38:12.112 INFO:teuthology.orchestra.run.smithi167.stdout:Upgraded: 2024-09-16T15:38:12.112 INFO:teuthology.orchestra.run.smithi167.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-16T15:38:12.112 INFO:teuthology.orchestra.run.smithi167.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-16T15:38:12.112 INFO:teuthology.orchestra.run.smithi167.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-16T15:38:12.112 INFO:teuthology.orchestra.run.smithi167.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-16T15:38:12.112 INFO:teuthology.orchestra.run.smithi167.stdout: librados2-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:38:12.112 INFO:teuthology.orchestra.run.smithi167.stdout: librbd1-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:38:12.112 INFO:teuthology.orchestra.run.smithi167.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-16T15:38:12.112 INFO:teuthology.orchestra.run.smithi167.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-16T15:38:12.112 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2024-09-16T15:38:12.112 INFO:teuthology.orchestra.run.smithi167.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-16T15:38:12.112 INFO:teuthology.orchestra.run.smithi167.stdout: automake-1.16.2-8.el9.noarch 2024-09-16T15:38:12.113 INFO:teuthology.orchestra.run.smithi167.stdout: bison-3.7.4-5.el9.x86_64 2024-09-16T15:38:12.113 INFO:teuthology.orchestra.run.smithi167.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-16T15:38:12.113 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:38:12.113 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-base-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:38:12.113 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-common-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:38:12.113 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-fuse-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:38:12.113 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-grafana-dashboards-2:19.3.0-4938.g33f95d2a.el9.noarch 2024-09-16T15:38:12.113 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-immutable-object-cache-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:38:12.113 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mds-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:38:12.113 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:38:12.113 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-cephadm-2:19.3.0-4938.g33f95d2a.el9.noarch 2024-09-16T15:38:12.113 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-dashboard-2:19.3.0-4938.g33f95d2a.el9.noarch 2024-09-16T15:38:12.113 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4938.g33f95d2a.el9.noarch 2024-09-16T15:38:12.113 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-modules-core-2:19.3.0-4938.g33f95d2a.el9.noarch 2024-09-16T15:38:12.113 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-rook-2:19.3.0-4938.g33f95d2a.el9.noarch 2024-09-16T15:38:12.114 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mon-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:38:12.114 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-osd-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:38:12.114 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-prometheus-alerts-2:19.3.0-4938.g33f95d2a.el9.noarch 2024-09-16T15:38:12.114 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-radosgw-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:38:12.114 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-selinux-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:38:12.114 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-test-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:38:12.114 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-volume-2:19.3.0-4938.g33f95d2a.el9.noarch 2024-09-16T15:38:12.114 INFO:teuthology.orchestra.run.smithi167.stdout: cephadm-2:19.3.0-4938.g33f95d2a.el9.noarch 2024-09-16T15:38:12.114 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-mirror-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:38:12.114 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-shell-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:38:12.114 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-top-2:19.3.0-4938.g33f95d2a.el9.noarch 2024-09-16T15:38:12.114 INFO:teuthology.orchestra.run.smithi167.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-16T15:38:12.114 INFO:teuthology.orchestra.run.smithi167.stdout: flex-2.6.4-9.el9.x86_64 2024-09-16T15:38:12.115 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-16T15:38:12.115 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-16T15:38:12.115 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-16T15:38:12.115 INFO:teuthology.orchestra.run.smithi167.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-16T15:38:12.115 INFO:teuthology.orchestra.run.smithi167.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-16T15:38:12.115 INFO:teuthology.orchestra.run.smithi167.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-16T15:38:12.115 INFO:teuthology.orchestra.run.smithi167.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-16T15:38:12.115 INFO:teuthology.orchestra.run.smithi167.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-16T15:38:12.115 INFO:teuthology.orchestra.run.smithi167.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-16T15:38:12.115 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs-devel-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:38:12.115 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs2-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:38:12.115 INFO:teuthology.orchestra.run.smithi167.stdout: libcephsqlite-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:38:12.116 INFO:teuthology.orchestra.run.smithi167.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-16T15:38:12.116 INFO:teuthology.orchestra.run.smithi167.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-16T15:38:12.116 INFO:teuthology.orchestra.run.smithi167.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-16T15:38:12.116 INFO:teuthology.orchestra.run.smithi167.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-16T15:38:12.116 INFO:teuthology.orchestra.run.smithi167.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-16T15:38:12.116 INFO:teuthology.orchestra.run.smithi167.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-16T15:38:12.116 INFO:teuthology.orchestra.run.smithi167.stdout: librados-devel-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:38:12.116 INFO:teuthology.orchestra.run.smithi167.stdout: libradosstriper1-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:38:12.116 INFO:teuthology.orchestra.run.smithi167.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-16T15:38:12.116 INFO:teuthology.orchestra.run.smithi167.stdout: librgw2-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:38:12.116 INFO:teuthology.orchestra.run.smithi167.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-16T15:38:12.116 INFO:teuthology.orchestra.run.smithi167.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-16T15:38:12.116 INFO:teuthology.orchestra.run.smithi167.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-16T15:38:12.117 INFO:teuthology.orchestra.run.smithi167.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-16T15:38:12.117 INFO:teuthology.orchestra.run.smithi167.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-16T15:38:12.117 INFO:teuthology.orchestra.run.smithi167.stdout: lua-5.4.4-4.el9.x86_64 2024-09-16T15:38:12.117 INFO:teuthology.orchestra.run.smithi167.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-16T15:38:12.117 INFO:teuthology.orchestra.run.smithi167.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-16T15:38:12.117 INFO:teuthology.orchestra.run.smithi167.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-16T15:38:12.117 INFO:teuthology.orchestra.run.smithi167.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-16T15:38:12.117 INFO:teuthology.orchestra.run.smithi167.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-16T15:38:12.117 INFO:teuthology.orchestra.run.smithi167.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-16T15:38:12.117 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-13.16-1.el9.x86_64 2024-09-16T15:38:12.117 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-16T15:38:12.117 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-16T15:38:12.117 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-16T15:38:12.118 INFO:teuthology.orchestra.run.smithi167.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-16T15:38:12.118 INFO:teuthology.orchestra.run.smithi167.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-16T15:38:12.118 INFO:teuthology.orchestra.run.smithi167.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-16T15:38:12.118 INFO:teuthology.orchestra.run.smithi167.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-16T15:38:12.118 INFO:teuthology.orchestra.run.smithi167.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-16T15:38:12.118 INFO:teuthology.orchestra.run.smithi167.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-16T15:38:12.118 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-16T15:38:12.118 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-argparse-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:38:12.118 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-common-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:38:12.118 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cephfs-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:38:12.118 INFO:teuthology.orchestra.run.smithi167.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-16T15:38:12.118 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-16T15:38:12.118 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-16T15:38:12.118 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-16T15:38:12.119 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-16T15:38:12.119 INFO:teuthology.orchestra.run.smithi167.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-16T15:38:12.119 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-16T15:38:12.119 INFO:teuthology.orchestra.run.smithi167.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-16T15:38:12.119 INFO:teuthology.orchestra.run.smithi167.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-16T15:38:12.119 INFO:teuthology.orchestra.run.smithi167.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-16T15:38:12.119 INFO:teuthology.orchestra.run.smithi167.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-16T15:38:12.119 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-16T15:38:12.119 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-16T15:38:12.119 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-16T15:38:12.119 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-16T15:38:12.119 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-16T15:38:12.119 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-16T15:38:12.119 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-16T15:38:12.120 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-16T15:38:12.120 INFO:teuthology.orchestra.run.smithi167.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-16T15:38:12.120 INFO:teuthology.orchestra.run.smithi167.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-16T15:38:12.120 INFO:teuthology.orchestra.run.smithi167.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-16T15:38:12.120 INFO:teuthology.orchestra.run.smithi167.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-16T15:38:12.120 INFO:teuthology.orchestra.run.smithi167.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-16T15:38:12.120 INFO:teuthology.orchestra.run.smithi167.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-16T15:38:12.120 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-16T15:38:12.120 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-16T15:38:12.120 INFO:teuthology.orchestra.run.smithi167.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-16T15:38:12.120 INFO:teuthology.orchestra.run.smithi167.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-16T15:38:12.120 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-16T15:38:12.120 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ply-3.11-14.el9.noarch 2024-09-16T15:38:12.120 INFO:teuthology.orchestra.run.smithi167.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-16T15:38:12.121 INFO:teuthology.orchestra.run.smithi167.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-16T15:38:12.121 INFO:teuthology.orchestra.run.smithi167.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-16T15:38:12.121 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-16T15:38:12.121 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-16T15:38:12.121 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-16T15:38:12.121 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-16T15:38:12.121 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-16T15:38:12.121 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-16T15:38:12.121 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rados-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:38:12.121 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rbd-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:38:12.121 INFO:teuthology.orchestra.run.smithi167.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-16T15:38:12.121 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-16T15:38:12.121 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-16T15:38:12.121 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rgw-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:38:12.122 INFO:teuthology.orchestra.run.smithi167.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-16T15:38:12.122 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-16T15:38:12.122 INFO:teuthology.orchestra.run.smithi167.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-16T15:38:12.122 INFO:teuthology.orchestra.run.smithi167.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-16T15:38:12.122 INFO:teuthology.orchestra.run.smithi167.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-16T15:38:12.122 INFO:teuthology.orchestra.run.smithi167.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-16T15:38:12.122 INFO:teuthology.orchestra.run.smithi167.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-16T15:38:12.122 INFO:teuthology.orchestra.run.smithi167.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-16T15:38:12.122 INFO:teuthology.orchestra.run.smithi167.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-16T15:38:12.122 INFO:teuthology.orchestra.run.smithi167.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-16T15:38:12.122 INFO:teuthology.orchestra.run.smithi167.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-16T15:38:12.122 INFO:teuthology.orchestra.run.smithi167.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-16T15:38:12.122 INFO:teuthology.orchestra.run.smithi167.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-16T15:38:12.122 INFO:teuthology.orchestra.run.smithi167.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-16T15:38:12.123 INFO:teuthology.orchestra.run.smithi167.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-16T15:38:12.123 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-fuse-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:38:12.123 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-mirror-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:38:12.123 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-nbd-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:38:12.123 INFO:teuthology.orchestra.run.smithi167.stdout: re2-1:20211101-20.el9.x86_64 2024-09-16T15:38:12.123 INFO:teuthology.orchestra.run.smithi167.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-16T15:38:12.123 INFO:teuthology.orchestra.run.smithi167.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-16T15:38:12.123 INFO:teuthology.orchestra.run.smithi167.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-16T15:38:12.123 INFO:teuthology.orchestra.run.smithi167.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-16T15:38:12.123 INFO:teuthology.orchestra.run.smithi167.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-16T15:38:12.123 INFO:teuthology.orchestra.run.smithi167.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-16T15:38:12.124 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-16T15:38:12.124 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2024-09-16T15:38:12.833 DEBUG:teuthology.parallel:result is None 2024-09-16T15:38:13.692 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-16T15:38:13.692 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:38:13.692 INFO:teuthology.orchestra.run.smithi151.stdout:Upgraded: 2024-09-16T15:38:13.692 INFO:teuthology.orchestra.run.smithi151.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-16T15:38:13.692 INFO:teuthology.orchestra.run.smithi151.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-16T15:38:13.692 INFO:teuthology.orchestra.run.smithi151.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-16T15:38:13.692 INFO:teuthology.orchestra.run.smithi151.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-16T15:38:13.692 INFO:teuthology.orchestra.run.smithi151.stdout: librados2-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:38:13.692 INFO:teuthology.orchestra.run.smithi151.stdout: librbd1-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:38:13.692 INFO:teuthology.orchestra.run.smithi151.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-16T15:38:13.693 INFO:teuthology.orchestra.run.smithi151.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-16T15:38:13.693 INFO:teuthology.orchestra.run.smithi151.stdout:Installed: 2024-09-16T15:38:13.693 INFO:teuthology.orchestra.run.smithi151.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-16T15:38:13.693 INFO:teuthology.orchestra.run.smithi151.stdout: automake-1.16.2-8.el9.noarch 2024-09-16T15:38:13.693 INFO:teuthology.orchestra.run.smithi151.stdout: bison-3.7.4-5.el9.x86_64 2024-09-16T15:38:13.693 INFO:teuthology.orchestra.run.smithi151.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-16T15:38:13.693 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:38:13.693 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-base-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:38:13.693 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-common-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:38:13.693 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-fuse-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:38:13.693 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-grafana-dashboards-2:19.3.0-4938.g33f95d2a.el9.noarch 2024-09-16T15:38:13.693 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-immutable-object-cache-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:38:13.693 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mds-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:38:13.694 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:38:13.694 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-cephadm-2:19.3.0-4938.g33f95d2a.el9.noarch 2024-09-16T15:38:13.694 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-dashboard-2:19.3.0-4938.g33f95d2a.el9.noarch 2024-09-16T15:38:13.694 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4938.g33f95d2a.el9.noarch 2024-09-16T15:38:13.694 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-modules-core-2:19.3.0-4938.g33f95d2a.el9.noarch 2024-09-16T15:38:13.694 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-rook-2:19.3.0-4938.g33f95d2a.el9.noarch 2024-09-16T15:38:13.694 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mon-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:38:13.694 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-osd-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:38:13.694 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-prometheus-alerts-2:19.3.0-4938.g33f95d2a.el9.noarch 2024-09-16T15:38:13.694 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-radosgw-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:38:13.694 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-selinux-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:38:13.694 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-test-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:38:13.694 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-volume-2:19.3.0-4938.g33f95d2a.el9.noarch 2024-09-16T15:38:13.695 INFO:teuthology.orchestra.run.smithi151.stdout: cephadm-2:19.3.0-4938.g33f95d2a.el9.noarch 2024-09-16T15:38:13.695 INFO:teuthology.orchestra.run.smithi151.stdout: cephfs-mirror-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:38:13.695 INFO:teuthology.orchestra.run.smithi151.stdout: cephfs-shell-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:38:13.695 INFO:teuthology.orchestra.run.smithi151.stdout: cephfs-top-2:19.3.0-4938.g33f95d2a.el9.noarch 2024-09-16T15:38:13.695 INFO:teuthology.orchestra.run.smithi151.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-16T15:38:13.695 INFO:teuthology.orchestra.run.smithi151.stdout: flex-2.6.4-9.el9.x86_64 2024-09-16T15:38:13.695 INFO:teuthology.orchestra.run.smithi151.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-16T15:38:13.695 INFO:teuthology.orchestra.run.smithi151.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-16T15:38:13.695 INFO:teuthology.orchestra.run.smithi151.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-16T15:38:13.695 INFO:teuthology.orchestra.run.smithi151.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-16T15:38:13.695 INFO:teuthology.orchestra.run.smithi151.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-16T15:38:13.695 INFO:teuthology.orchestra.run.smithi151.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-16T15:38:13.695 INFO:teuthology.orchestra.run.smithi151.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-16T15:38:13.695 INFO:teuthology.orchestra.run.smithi151.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-16T15:38:13.696 INFO:teuthology.orchestra.run.smithi151.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-16T15:38:13.696 INFO:teuthology.orchestra.run.smithi151.stdout: libcephfs-devel-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:38:13.696 INFO:teuthology.orchestra.run.smithi151.stdout: libcephfs2-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:38:13.696 INFO:teuthology.orchestra.run.smithi151.stdout: libcephsqlite-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:38:13.696 INFO:teuthology.orchestra.run.smithi151.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-16T15:38:13.696 INFO:teuthology.orchestra.run.smithi151.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-16T15:38:13.696 INFO:teuthology.orchestra.run.smithi151.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-16T15:38:13.696 INFO:teuthology.orchestra.run.smithi151.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-16T15:38:13.696 INFO:teuthology.orchestra.run.smithi151.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-16T15:38:13.696 INFO:teuthology.orchestra.run.smithi151.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-16T15:38:13.696 INFO:teuthology.orchestra.run.smithi151.stdout: librados-devel-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:38:13.696 INFO:teuthology.orchestra.run.smithi151.stdout: libradosstriper1-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:38:13.696 INFO:teuthology.orchestra.run.smithi151.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-16T15:38:13.696 INFO:teuthology.orchestra.run.smithi151.stdout: librgw2-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:38:13.697 INFO:teuthology.orchestra.run.smithi151.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-16T15:38:13.697 INFO:teuthology.orchestra.run.smithi151.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-16T15:38:13.697 INFO:teuthology.orchestra.run.smithi151.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-16T15:38:13.697 INFO:teuthology.orchestra.run.smithi151.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-16T15:38:13.697 INFO:teuthology.orchestra.run.smithi151.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-16T15:38:13.697 INFO:teuthology.orchestra.run.smithi151.stdout: lua-5.4.4-4.el9.x86_64 2024-09-16T15:38:13.697 INFO:teuthology.orchestra.run.smithi151.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-16T15:38:13.697 INFO:teuthology.orchestra.run.smithi151.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-16T15:38:13.697 INFO:teuthology.orchestra.run.smithi151.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-16T15:38:13.697 INFO:teuthology.orchestra.run.smithi151.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-16T15:38:13.697 INFO:teuthology.orchestra.run.smithi151.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-16T15:38:13.697 INFO:teuthology.orchestra.run.smithi151.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-16T15:38:13.697 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-13.16-1.el9.x86_64 2024-09-16T15:38:13.697 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-16T15:38:13.698 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-16T15:38:13.698 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-16T15:38:13.698 INFO:teuthology.orchestra.run.smithi151.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-16T15:38:13.698 INFO:teuthology.orchestra.run.smithi151.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-16T15:38:13.698 INFO:teuthology.orchestra.run.smithi151.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-16T15:38:13.698 INFO:teuthology.orchestra.run.smithi151.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-16T15:38:13.698 INFO:teuthology.orchestra.run.smithi151.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-16T15:38:13.698 INFO:teuthology.orchestra.run.smithi151.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-16T15:38:13.698 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-16T15:38:13.698 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ceph-argparse-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:38:13.698 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ceph-common-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:38:13.698 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cephfs-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:38:13.698 INFO:teuthology.orchestra.run.smithi151.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-16T15:38:13.698 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-16T15:38:13.699 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-16T15:38:13.699 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-16T15:38:13.699 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-16T15:38:13.699 INFO:teuthology.orchestra.run.smithi151.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-16T15:38:13.699 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-16T15:38:13.699 INFO:teuthology.orchestra.run.smithi151.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-16T15:38:13.699 INFO:teuthology.orchestra.run.smithi151.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-16T15:38:13.699 INFO:teuthology.orchestra.run.smithi151.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-16T15:38:13.699 INFO:teuthology.orchestra.run.smithi151.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-16T15:38:13.699 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-16T15:38:13.699 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-16T15:38:13.699 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-16T15:38:13.699 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-16T15:38:13.699 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-16T15:38:13.700 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-16T15:38:13.700 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-16T15:38:13.700 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-16T15:38:13.700 INFO:teuthology.orchestra.run.smithi151.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-16T15:38:13.700 INFO:teuthology.orchestra.run.smithi151.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-16T15:38:13.700 INFO:teuthology.orchestra.run.smithi151.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-16T15:38:13.700 INFO:teuthology.orchestra.run.smithi151.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-16T15:38:13.700 INFO:teuthology.orchestra.run.smithi151.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-16T15:38:13.700 INFO:teuthology.orchestra.run.smithi151.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-16T15:38:13.700 INFO:teuthology.orchestra.run.smithi151.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-16T15:38:13.700 INFO:teuthology.orchestra.run.smithi151.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-16T15:38:13.700 INFO:teuthology.orchestra.run.smithi151.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-16T15:38:13.700 INFO:teuthology.orchestra.run.smithi151.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-16T15:38:13.700 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-16T15:38:13.700 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ply-3.11-14.el9.noarch 2024-09-16T15:38:13.701 INFO:teuthology.orchestra.run.smithi151.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-16T15:38:13.701 INFO:teuthology.orchestra.run.smithi151.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-16T15:38:13.701 INFO:teuthology.orchestra.run.smithi151.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-16T15:38:13.701 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-16T15:38:13.701 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-16T15:38:13.701 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-16T15:38:13.701 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-16T15:38:13.701 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-16T15:38:13.701 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-16T15:38:13.701 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rados-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:38:13.701 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rbd-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:38:13.701 INFO:teuthology.orchestra.run.smithi151.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-16T15:38:13.701 INFO:teuthology.orchestra.run.smithi151.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-16T15:38:13.701 INFO:teuthology.orchestra.run.smithi151.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-16T15:38:13.702 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rgw-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:38:13.702 INFO:teuthology.orchestra.run.smithi151.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-16T15:38:13.702 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-16T15:38:13.702 INFO:teuthology.orchestra.run.smithi151.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-16T15:38:13.702 INFO:teuthology.orchestra.run.smithi151.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-16T15:38:13.702 INFO:teuthology.orchestra.run.smithi151.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-16T15:38:13.702 INFO:teuthology.orchestra.run.smithi151.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-16T15:38:13.702 INFO:teuthology.orchestra.run.smithi151.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-16T15:38:13.702 INFO:teuthology.orchestra.run.smithi151.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-16T15:38:13.702 INFO:teuthology.orchestra.run.smithi151.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-16T15:38:13.702 INFO:teuthology.orchestra.run.smithi151.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-16T15:38:13.702 INFO:teuthology.orchestra.run.smithi151.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-16T15:38:13.702 INFO:teuthology.orchestra.run.smithi151.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-16T15:38:13.702 INFO:teuthology.orchestra.run.smithi151.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-16T15:38:13.703 INFO:teuthology.orchestra.run.smithi151.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-16T15:38:13.703 INFO:teuthology.orchestra.run.smithi151.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-16T15:38:13.703 INFO:teuthology.orchestra.run.smithi151.stdout: rbd-fuse-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:38:13.703 INFO:teuthology.orchestra.run.smithi151.stdout: rbd-mirror-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:38:13.703 INFO:teuthology.orchestra.run.smithi151.stdout: rbd-nbd-2:19.3.0-4938.g33f95d2a.el9.x86_64 2024-09-16T15:38:13.703 INFO:teuthology.orchestra.run.smithi151.stdout: re2-1:20211101-20.el9.x86_64 2024-09-16T15:38:13.703 INFO:teuthology.orchestra.run.smithi151.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-16T15:38:13.703 INFO:teuthology.orchestra.run.smithi151.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-16T15:38:13.703 INFO:teuthology.orchestra.run.smithi151.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-16T15:38:13.703 INFO:teuthology.orchestra.run.smithi151.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-16T15:38:13.703 INFO:teuthology.orchestra.run.smithi151.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-16T15:38:13.704 INFO:teuthology.orchestra.run.smithi151.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-16T15:38:13.704 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:38:13.704 INFO:teuthology.orchestra.run.smithi151.stdout:Complete! 2024-09-16T15:38:14.351 DEBUG:teuthology.parallel:result is None 2024-09-16T15:38:14.352 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=33f95d2ae1bcbe7eb04404303f179eeeaa6982df 2024-09-16T15:38:14.497 DEBUG:teuthology.orchestra.run.smithi151:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-16T15:38:14.542 INFO:teuthology.orchestra.run.smithi151.stdout:19.3.0-4938.g33f95d2a.el9 2024-09-16T15:38:14.543 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4938.g33f95d2a.el9 2024-09-16T15:38:14.543 INFO:teuthology.task.install:The correct ceph version 19.3.0-4938.g33f95d2a is installed. 2024-09-16T15:38:14.545 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=33f95d2ae1bcbe7eb04404303f179eeeaa6982df 2024-09-16T15:38:14.685 DEBUG:teuthology.orchestra.run.smithi167:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-16T15:38:14.730 INFO:teuthology.orchestra.run.smithi167.stdout:19.3.0-4938.g33f95d2a.el9 2024-09-16T15:38:14.730 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4938.g33f95d2a.el9 2024-09-16T15:38:14.730 INFO:teuthology.task.install:The correct ceph version 19.3.0-4938.g33f95d2a is installed. 2024-09-16T15:38:14.732 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-16T15:38:14.732 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:38:14.732 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-16T15:38:14.794 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-16T15:38:14.794 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-16T15:38:14.852 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-16T15:38:14.852 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:38:14.852 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/usr/bin/daemon-helper 2024-09-16T15:38:14.911 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-16T15:38:15.008 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-16T15:38:15.008 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/usr/bin/daemon-helper 2024-09-16T15:38:15.064 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-16T15:38:15.161 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-16T15:38:15.162 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:38:15.162 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-16T15:38:15.226 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-16T15:38:15.324 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-16T15:38:15.324 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-16T15:38:15.379 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-16T15:38:15.475 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-16T15:38:15.475 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:38:15.475 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/usr/bin/stdin-killer 2024-09-16T15:38:15.533 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-16T15:38:15.630 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-16T15:38:15.630 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/usr/bin/stdin-killer 2024-09-16T15:38:15.716 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-16T15:38:15.811 INFO:teuthology.run_tasks:Running task ceph... 2024-09-16T15:38:15.890 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-16T15:38:15.890 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod 777 /var/log/ceph 2024-09-16T15:38:15.894 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 777 /var/log/ceph 2024-09-16T15:38:15.950 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-16T15:38:15.950 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-16T15:38:15.954 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-16T15:38:16.008 INFO:tasks.ceph:Creating extra log directories... 2024-09-16T15:38:16.008 DEBUG:teuthology.orchestra.run.smithi151:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-16T15:38:16.012 DEBUG:teuthology.orchestra.run.smithi167:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-16T15:38:16.074 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-09-16T15:38:16.075 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-09-16T15:38:16.075 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-09-16T15:38:16.075 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:38:16.075 DEBUG:teuthology.orchestra.run.smithi151:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-16T15:38:16.112 DEBUG:teuthology.orchestra.run.smithi151:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-16T15:38:16.287 DEBUG:teuthology.orchestra.run.smithi151:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-09-16T15:38:16.344 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-16T15:38:16.344 DEBUG:teuthology.orchestra.run.smithi167:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-16T15:38:16.379 DEBUG:teuthology.orchestra.run.smithi167:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-16T15:38:16.545 DEBUG:teuthology.orchestra.run.smithi167:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-09-16T15:38:16.606 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-16T15:38:16.606 INFO:tasks.ceph:config {'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-16T15:38:16.607 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/avan-2024-09-16_15:21:40-fs:volumes-wip-athakkar-testing-2024-09-16-1454-distro-default-smithi/7907354', 'branch': 'wip-athakkar-testing-2024-09-16-1454', 'description': 'fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/clone-progress}}', 'email': 'athakkar@redhat.com', 'first_in_suite': False, 'job_id': '7907354', 'kernel': {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'avan-2024-09-16_15:21:40-fs:volumes-wip-athakkar-testing-2024-09-16-1454-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-athakkar-testing-2024-09-16-1454'}, 'ceph': {'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon': {'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '33f95d2ae1bcbe7eb04404303f179eeeaa6982df'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '33f95d2ae1bcbe7eb04404303f179eeeaa6982df'}}, 'kclient': {'syntax': 'v1'}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-athakkar-testing-2024-09-16-1454', 'sha1': '33f95d2ae1bcbe7eb04404303f179eeeaa6982df'}}, 'owner': 'scheduled_avan@teuthology', 'priority': 80, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 3077, 'sha1': '33f95d2ae1bcbe7eb04404303f179eeeaa6982df', 'sleep_before_teardown': 0, 'subset': '1/20', 'suite': 'fs:volumes', 'suite_branch': 'wip-athakkar-testing-2024-09-16-1454', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_33f95d2ae1bcbe7eb04404303f179eeeaa6982df/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '33f95d2ae1bcbe7eb04404303f179eeeaa6982df', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi151.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGck6mysMoarx5v9Lx4Mg0nzqIYwd1rFOpGPdwdxW5dZXHGVgwj2CNE3PMzxjDmPNqX7z5lAeDgx+WoqYDG+CWs=', 'smithi167.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFDumuaBHu91r2fvqaNRnjzggIEpbYvy1Y8kGjCZvrcHhG8Fkz6KsCl4REHXM6pmIevcZGtrmFJDG/DpItnhXGY='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '33f95d2ae1bcbe7eb04404303f179eeeaa6982df'}}, {'ceph': {'create_rbd_pool': False, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon': {'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '33f95d2ae1bcbe7eb04404303f179eeeaa6982df', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volumes.TestCloneProgressReporter']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-16_15:21:40', 'tube': 'smithi', 'user': 'avan', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-16T15:38:16.607 DEBUG:teuthology.orchestra.run.smithi151:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-16T15:38:16.611 DEBUG:teuthology.orchestra.run.smithi167:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-16T15:38:16.646 DEBUG:teuthology.orchestra.run.smithi151:> sudo install -d -m0777 -- /var/run/ceph 2024-09-16T15:38:16.689 DEBUG:teuthology.orchestra.run.smithi167:> sudo install -d -m0777 -- /var/run/ceph 2024-09-16T15:38:16.744 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:38:16.744 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T15:38:16.780 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T15:38:16.781 DEBUG:teuthology.orchestra.run.smithi151:> stat /dev/vg_nvme/lv_1 2024-09-16T15:38:16.857 INFO:teuthology.orchestra.run.smithi151.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T15:38:16.857 INFO:teuthology.orchestra.run.smithi151.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T15:38:16.857 INFO:teuthology.orchestra.run.smithi151.stdout:Device: 6h/6d Inode: 633 Links: 1 2024-09-16T15:38:16.858 INFO:teuthology.orchestra.run.smithi151.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T15:38:16.858 INFO:teuthology.orchestra.run.smithi151.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T15:38:16.858 INFO:teuthology.orchestra.run.smithi151.stdout:Access: 2024-09-16 15:38:05.414345902 +0000 2024-09-16T15:38:16.858 INFO:teuthology.orchestra.run.smithi151.stdout:Modify: 2024-09-16 15:36:02.931036718 +0000 2024-09-16T15:38:16.858 INFO:teuthology.orchestra.run.smithi151.stdout:Change: 2024-09-16 15:36:02.931036718 +0000 2024-09-16T15:38:16.858 INFO:teuthology.orchestra.run.smithi151.stdout: Birth: 2024-09-16 15:36:02.931036718 +0000 2024-09-16T15:38:16.858 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T15:38:16.953 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records in 2024-09-16T15:38:16.953 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records out 2024-09-16T15:38:16.953 INFO:teuthology.orchestra.run.smithi151.stderr:512 bytes copied, 0.000331194 s, 1.5 MB/s 2024-09-16T15:38:16.955 DEBUG:teuthology.orchestra.run.smithi151:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T15:38:16.993 DEBUG:teuthology.orchestra.run.smithi151:> stat /dev/vg_nvme/lv_2 2024-09-16T15:38:17.069 INFO:teuthology.orchestra.run.smithi151.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T15:38:17.069 INFO:teuthology.orchestra.run.smithi151.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T15:38:17.069 INFO:teuthology.orchestra.run.smithi151.stdout:Device: 6h/6d Inode: 650 Links: 1 2024-09-16T15:38:17.069 INFO:teuthology.orchestra.run.smithi151.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T15:38:17.069 INFO:teuthology.orchestra.run.smithi151.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T15:38:17.069 INFO:teuthology.orchestra.run.smithi151.stdout:Access: 2024-09-16 15:38:05.415345890 +0000 2024-09-16T15:38:17.069 INFO:teuthology.orchestra.run.smithi151.stdout:Modify: 2024-09-16 15:36:03.432030652 +0000 2024-09-16T15:38:17.069 INFO:teuthology.orchestra.run.smithi151.stdout:Change: 2024-09-16 15:36:03.432030652 +0000 2024-09-16T15:38:17.069 INFO:teuthology.orchestra.run.smithi151.stdout: Birth: 2024-09-16 15:36:03.432030652 +0000 2024-09-16T15:38:17.070 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T15:38:17.172 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records in 2024-09-16T15:38:17.172 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records out 2024-09-16T15:38:17.172 INFO:teuthology.orchestra.run.smithi151.stderr:512 bytes copied, 0.000268376 s, 1.9 MB/s 2024-09-16T15:38:17.175 DEBUG:teuthology.orchestra.run.smithi151:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T15:38:17.212 DEBUG:teuthology.orchestra.run.smithi151:> stat /dev/vg_nvme/lv_3 2024-09-16T15:38:17.288 INFO:teuthology.orchestra.run.smithi151.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T15:38:17.288 INFO:teuthology.orchestra.run.smithi151.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T15:38:17.289 INFO:teuthology.orchestra.run.smithi151.stdout:Device: 6h/6d Inode: 672 Links: 1 2024-09-16T15:38:17.289 INFO:teuthology.orchestra.run.smithi151.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T15:38:17.289 INFO:teuthology.orchestra.run.smithi151.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T15:38:17.289 INFO:teuthology.orchestra.run.smithi151.stdout:Access: 2024-09-16 15:38:05.416345878 +0000 2024-09-16T15:38:17.289 INFO:teuthology.orchestra.run.smithi151.stdout:Modify: 2024-09-16 15:36:03.901024973 +0000 2024-09-16T15:38:17.289 INFO:teuthology.orchestra.run.smithi151.stdout:Change: 2024-09-16 15:36:03.901024973 +0000 2024-09-16T15:38:17.289 INFO:teuthology.orchestra.run.smithi151.stdout: Birth: 2024-09-16 15:36:03.901024973 +0000 2024-09-16T15:38:17.289 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T15:38:17.382 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records in 2024-09-16T15:38:17.382 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records out 2024-09-16T15:38:17.382 INFO:teuthology.orchestra.run.smithi151.stderr:512 bytes copied, 0.000272899 s, 1.9 MB/s 2024-09-16T15:38:17.385 DEBUG:teuthology.orchestra.run.smithi151:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T15:38:17.421 DEBUG:teuthology.orchestra.run.smithi151:> stat /dev/vg_nvme/lv_4 2024-09-16T15:38:17.499 INFO:teuthology.orchestra.run.smithi151.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T15:38:17.499 INFO:teuthology.orchestra.run.smithi151.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T15:38:17.499 INFO:teuthology.orchestra.run.smithi151.stdout:Device: 6h/6d Inode: 688 Links: 1 2024-09-16T15:38:17.499 INFO:teuthology.orchestra.run.smithi151.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T15:38:17.499 INFO:teuthology.orchestra.run.smithi151.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T15:38:17.499 INFO:teuthology.orchestra.run.smithi151.stdout:Access: 2024-09-16 15:38:05.416345878 +0000 2024-09-16T15:38:17.500 INFO:teuthology.orchestra.run.smithi151.stdout:Modify: 2024-09-16 15:36:04.374019245 +0000 2024-09-16T15:38:17.500 INFO:teuthology.orchestra.run.smithi151.stdout:Change: 2024-09-16 15:36:04.374019245 +0000 2024-09-16T15:38:17.500 INFO:teuthology.orchestra.run.smithi151.stdout: Birth: 2024-09-16 15:36:04.374019245 +0000 2024-09-16T15:38:17.500 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T15:38:17.596 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records in 2024-09-16T15:38:17.596 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records out 2024-09-16T15:38:17.597 INFO:teuthology.orchestra.run.smithi151.stderr:512 bytes copied, 0.000251193 s, 2.0 MB/s 2024-09-16T15:38:17.599 DEBUG:teuthology.orchestra.run.smithi151:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T15:38:17.636 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-16T15:38:17.636 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-16T15:38:17.636 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T15:38:17.674 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T15:38:17.674 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_1 2024-09-16T15:38:17.753 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T15:38:17.753 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T15:38:17.753 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 6h/6d Inode: 632 Links: 1 2024-09-16T15:38:17.753 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T15:38:17.753 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T15:38:17.753 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2024-09-16 15:38:03.618559293 +0000 2024-09-16T15:38:17.753 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2024-09-16 15:36:02.918908255 +0000 2024-09-16T15:38:17.753 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2024-09-16 15:36:02.918908255 +0000 2024-09-16T15:38:17.754 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: 2024-09-16 15:36:02.918908255 +0000 2024-09-16T15:38:17.754 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T15:38:17.847 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2024-09-16T15:38:17.847 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2024-09-16T15:38:17.848 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000287798 s, 1.8 MB/s 2024-09-16T15:38:17.850 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T15:38:17.887 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_2 2024-09-16T15:38:17.962 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T15:38:17.962 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T15:38:17.962 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 6h/6d Inode: 651 Links: 1 2024-09-16T15:38:17.962 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T15:38:17.962 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T15:38:17.963 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2024-09-16 15:38:03.619559275 +0000 2024-09-16T15:38:17.963 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2024-09-16 15:36:03.416899379 +0000 2024-09-16T15:38:17.963 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2024-09-16 15:36:03.416899379 +0000 2024-09-16T15:38:17.963 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: 2024-09-16 15:36:03.416899379 +0000 2024-09-16T15:38:17.963 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T15:38:18.055 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2024-09-16T15:38:18.055 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2024-09-16T15:38:18.055 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.00027662 s, 1.9 MB/s 2024-09-16T15:38:18.058 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T15:38:18.095 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_3 2024-09-16T15:38:18.170 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T15:38:18.170 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T15:38:18.170 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 6h/6d Inode: 669 Links: 1 2024-09-16T15:38:18.170 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T15:38:18.170 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T15:38:18.171 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2024-09-16 15:38:03.620559257 +0000 2024-09-16T15:38:18.171 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2024-09-16 15:36:03.898890787 +0000 2024-09-16T15:38:18.171 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2024-09-16 15:36:03.898890787 +0000 2024-09-16T15:38:18.171 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: 2024-09-16 15:36:03.898890787 +0000 2024-09-16T15:38:18.171 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T15:38:18.263 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2024-09-16T15:38:18.263 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2024-09-16T15:38:18.263 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000288038 s, 1.8 MB/s 2024-09-16T15:38:18.266 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T15:38:18.302 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_4 2024-09-16T15:38:18.378 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T15:38:18.378 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T15:38:18.378 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 6h/6d Inode: 691 Links: 1 2024-09-16T15:38:18.378 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T15:38:18.378 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T15:38:18.378 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2024-09-16 15:38:03.620559257 +0000 2024-09-16T15:38:18.378 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2024-09-16 15:36:04.371882355 +0000 2024-09-16T15:38:18.378 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2024-09-16 15:36:04.371882355 +0000 2024-09-16T15:38:18.379 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: 2024-09-16 15:36:04.371882355 +0000 2024-09-16T15:38:18.379 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T15:38:18.471 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2024-09-16T15:38:18.471 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2024-09-16T15:38:18.471 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000288599 s, 1.8 MB/s 2024-09-16T15:38:18.473 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T15:38:18.510 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-16T15:38:18.510 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi151.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi167.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-16T15:38:18.510 INFO:tasks.ceph:Generating config... 2024-09-16T15:38:18.511 INFO:tasks.ceph:[client] pid file = 2024-09-16T15:38:18.511 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-09-16T15:38:18.511 INFO:tasks.ceph:[mgr] client mount timeout = 30 2024-09-16T15:38:18.511 INFO:tasks.ceph:[mgr] debug client = 20 2024-09-16T15:38:18.511 INFO:tasks.ceph:[mgr] debug finisher = 20 2024-09-16T15:38:18.511 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-16T15:38:18.511 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-16T15:38:18.512 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-16T15:38:18.512 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-16T15:38:18.512 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-16T15:38:18.512 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-09-16T15:38:18.512 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-16T15:38:18.512 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-16T15:38:18.512 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-09-16T15:38:18.512 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-16T15:38:18.512 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-16T15:38:18.512 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-16T15:38:18.512 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-16T15:38:18.512 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-16T15:38:18.512 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-16T15:38:18.512 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-16T15:38:18.513 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-16T15:38:18.513 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-16T15:38:18.513 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-16T15:38:18.513 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-16T15:38:18.513 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-16T15:38:18.513 INFO:tasks.ceph:Setting up mon.a... 2024-09-16T15:38:18.513 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-16T15:38:18.613 INFO:teuthology.orchestra.run.smithi151.stdout:creating /etc/ceph/ceph.keyring 2024-09-16T15:38:18.619 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-16T15:38:18.723 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-16T15:38:18.778 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.151'), ('mon.b', '172.21.15.167'), ('mon.c', '[v2:172.21.15.167:3301,v1:172.21.15.167:6790]')] 2024-09-16T15:38:18.778 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.151,172.21.15.167,[v2:172.21.15.167:3301,v1:172.21.15.167:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'client mount timeout': 30, 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'pid file': ''}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}} 2024-09-16T15:38:18.779 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:38:18.779 DEBUG:teuthology.orchestra.run.smithi151:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-16T15:38:18.814 DEBUG:teuthology.orchestra.run.smithi151:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.151 --add b 172.21.15.167 --addv c '[v2:172.21.15.167:3301,v1:172.21.15.167:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-16T15:38:18.931 INFO:teuthology.orchestra.run.smithi151.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-16T15:38:18.931 INFO:teuthology.orchestra.run.smithi151.stdout:monmaptool: generated fsid 1757be18-e41a-4446-bb4f-ee4ac81c39df 2024-09-16T15:38:18.931 INFO:teuthology.orchestra.run.smithi151.stdout:setting min_mon_release = quincy 2024-09-16T15:38:18.931 INFO:teuthology.orchestra.run.smithi151.stdout:epoch 0 2024-09-16T15:38:18.931 INFO:teuthology.orchestra.run.smithi151.stdout:fsid 1757be18-e41a-4446-bb4f-ee4ac81c39df 2024-09-16T15:38:18.931 INFO:teuthology.orchestra.run.smithi151.stdout:last_changed 2024-09-16T15:38:18.930471+0000 2024-09-16T15:38:18.931 INFO:teuthology.orchestra.run.smithi151.stdout:created 2024-09-16T15:38:18.930471+0000 2024-09-16T15:38:18.931 INFO:teuthology.orchestra.run.smithi151.stdout:min_mon_release 17 (quincy) 2024-09-16T15:38:18.931 INFO:teuthology.orchestra.run.smithi151.stdout:election_strategy: 1 2024-09-16T15:38:18.931 INFO:teuthology.orchestra.run.smithi151.stdout:0: [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon.a 2024-09-16T15:38:18.931 INFO:teuthology.orchestra.run.smithi151.stdout:1: [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon.b 2024-09-16T15:38:18.931 INFO:teuthology.orchestra.run.smithi151.stdout:2: [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] mon.c 2024-09-16T15:38:18.932 INFO:teuthology.orchestra.run.smithi151.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-16T15:38:18.934 DEBUG:teuthology.orchestra.run.smithi151:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-16T15:38:18.968 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 1757be18-e41a-4446-bb4f-ee4ac81c39df... 2024-09-16T15:38:18.969 DEBUG:teuthology.orchestra.run.smithi151:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-16T15:38:19.012 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-16T15:38:19.105 INFO:teuthology.orchestra.run.smithi151.stdout:[global] 2024-09-16T15:38:19.105 INFO:teuthology.orchestra.run.smithi151.stdout: chdir = "" 2024-09-16T15:38:19.105 INFO:teuthology.orchestra.run.smithi151.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-16T15:38:19.105 INFO:teuthology.orchestra.run.smithi151.stdout: auth supported = cephx 2024-09-16T15:38:19.105 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:38:19.105 INFO:teuthology.orchestra.run.smithi151.stdout: filestore xattr use omap = true 2024-09-16T15:38:19.105 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:38:19.105 INFO:teuthology.orchestra.run.smithi151.stdout: mon clock drift allowed = 1.000 2024-09-16T15:38:19.105 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:38:19.106 INFO:teuthology.orchestra.run.smithi151.stdout: osd crush chooseleaf type = 0 2024-09-16T15:38:19.106 INFO:teuthology.orchestra.run.smithi151.stdout: auth debug = true 2024-09-16T15:38:19.106 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:38:19.106 INFO:teuthology.orchestra.run.smithi151.stdout: ms die on old message = true 2024-09-16T15:38:19.106 INFO:teuthology.orchestra.run.smithi151.stdout: ms die on bug = true 2024-09-16T15:38:19.106 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:38:19.106 INFO:teuthology.orchestra.run.smithi151.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-16T15:38:19.106 INFO:teuthology.orchestra.run.smithi151.stdout: mon pg warn max object skew = 0 2024-09-16T15:38:19.106 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:38:19.106 INFO:teuthology.orchestra.run.smithi151.stdout: # disable pg_autoscaler by default for new pools 2024-09-16T15:38:19.106 INFO:teuthology.orchestra.run.smithi151.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-16T15:38:19.106 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:38:19.106 INFO:teuthology.orchestra.run.smithi151.stdout: osd pool default size = 2 2024-09-16T15:38:19.106 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:38:19.106 INFO:teuthology.orchestra.run.smithi151.stdout: mon osd allow primary affinity = true 2024-09-16T15:38:19.106 INFO:teuthology.orchestra.run.smithi151.stdout: mon osd allow pg remap = true 2024-09-16T15:38:19.107 INFO:teuthology.orchestra.run.smithi151.stdout: mon warn on legacy crush tunables = false 2024-09-16T15:38:19.107 INFO:teuthology.orchestra.run.smithi151.stdout: mon warn on crush straw calc version zero = false 2024-09-16T15:38:19.107 INFO:teuthology.orchestra.run.smithi151.stdout: mon warn on no sortbitwise = false 2024-09-16T15:38:19.107 INFO:teuthology.orchestra.run.smithi151.stdout: mon warn on osd down out interval zero = false 2024-09-16T15:38:19.107 INFO:teuthology.orchestra.run.smithi151.stdout: mon warn on too few osds = false 2024-09-16T15:38:19.107 INFO:teuthology.orchestra.run.smithi151.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-16T15:38:19.107 INFO:teuthology.orchestra.run.smithi151.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-16T15:38:19.107 INFO:teuthology.orchestra.run.smithi151.stdout: mon_allow_pool_size_one = true 2024-09-16T15:38:19.107 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:38:19.107 INFO:teuthology.orchestra.run.smithi151.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-16T15:38:19.107 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:38:19.107 INFO:teuthology.orchestra.run.smithi151.stdout: osd default data pool replay window = 5 2024-09-16T15:38:19.107 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:38:19.107 INFO:teuthology.orchestra.run.smithi151.stdout: mon allow pool delete = true 2024-09-16T15:38:19.108 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:38:19.108 INFO:teuthology.orchestra.run.smithi151.stdout: mon cluster log file level = debug 2024-09-16T15:38:19.108 INFO:teuthology.orchestra.run.smithi151.stdout: debug asserts on shutdown = true 2024-09-16T15:38:19.108 INFO:teuthology.orchestra.run.smithi151.stdout: mon health detail to clog = false 2024-09-16T15:38:19.108 INFO:teuthology.orchestra.run.smithi151.stdout: mon host = "172.21.15.151,172.21.15.167,[v2:172.21.15.167:3301,v1:172.21.15.167:6790]" 2024-09-16T15:38:19.108 INFO:teuthology.orchestra.run.smithi151.stdout: ms die on skipped message = False 2024-09-16T15:38:19.108 INFO:teuthology.orchestra.run.smithi151.stdout: fsid = 1757be18-e41a-4446-bb4f-ee4ac81c39df 2024-09-16T15:38:19.108 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:38:19.108 INFO:teuthology.orchestra.run.smithi151.stdout:[osd] 2024-09-16T15:38:19.108 INFO:teuthology.orchestra.run.smithi151.stdout: osd journal size = 100 2024-09-16T15:38:19.108 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:38:19.109 INFO:teuthology.orchestra.run.smithi151.stdout: osd scrub load threshold = 5.0 2024-09-16T15:38:19.109 INFO:teuthology.orchestra.run.smithi151.stdout: osd scrub max interval = 600 2024-09-16T15:38:19.109 INFO:teuthology.orchestra.run.smithi151.stdout: osd mclock profile = high_recovery_ops 2024-09-16T15:38:19.109 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:38:19.109 INFO:teuthology.orchestra.run.smithi151.stdout: osd recover clone overlap = true 2024-09-16T15:38:19.109 INFO:teuthology.orchestra.run.smithi151.stdout: osd recovery max chunk = 1048576 2024-09-16T15:38:19.109 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:38:19.109 INFO:teuthology.orchestra.run.smithi151.stdout: osd debug shutdown = true 2024-09-16T15:38:19.109 INFO:teuthology.orchestra.run.smithi151.stdout: osd debug op order = true 2024-09-16T15:38:19.109 INFO:teuthology.orchestra.run.smithi151.stdout: osd debug verify stray on activate = true 2024-09-16T15:38:19.109 INFO:teuthology.orchestra.run.smithi151.stdout: osd debug trim objects = true 2024-09-16T15:38:19.109 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:38:19.109 INFO:teuthology.orchestra.run.smithi151.stdout: osd open classes on start = true 2024-09-16T15:38:19.109 INFO:teuthology.orchestra.run.smithi151.stdout: osd debug pg log writeout = true 2024-09-16T15:38:19.110 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:38:19.110 INFO:teuthology.orchestra.run.smithi151.stdout: osd deep scrub update digest min age = 30 2024-09-16T15:38:19.110 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:38:19.110 INFO:teuthology.orchestra.run.smithi151.stdout: osd map max advance = 10 2024-09-16T15:38:19.110 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:38:19.110 INFO:teuthology.orchestra.run.smithi151.stdout: journal zero on create = true 2024-09-16T15:38:19.110 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:38:19.110 INFO:teuthology.orchestra.run.smithi151.stdout: filestore ondisk finisher threads = 3 2024-09-16T15:38:19.110 INFO:teuthology.orchestra.run.smithi151.stdout: filestore apply finisher threads = 3 2024-09-16T15:38:19.110 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:38:19.110 INFO:teuthology.orchestra.run.smithi151.stdout: bdev debug aio = true 2024-09-16T15:38:19.110 INFO:teuthology.orchestra.run.smithi151.stdout: osd debug misdirected ops = true 2024-09-16T15:38:19.110 INFO:teuthology.orchestra.run.smithi151.stdout: bdev async discard = True 2024-09-16T15:38:19.110 INFO:teuthology.orchestra.run.smithi151.stdout: bdev enable discard = True 2024-09-16T15:38:19.111 INFO:teuthology.orchestra.run.smithi151.stdout: bluestore allocator = bitmap 2024-09-16T15:38:19.111 INFO:teuthology.orchestra.run.smithi151.stdout: bluestore block size = 96636764160 2024-09-16T15:38:19.111 INFO:teuthology.orchestra.run.smithi151.stdout: bluestore fsck on mount = True 2024-09-16T15:38:19.111 INFO:teuthology.orchestra.run.smithi151.stdout: debug bluefs = 1/20 2024-09-16T15:38:19.111 INFO:teuthology.orchestra.run.smithi151.stdout: debug bluestore = 1/20 2024-09-16T15:38:19.111 INFO:teuthology.orchestra.run.smithi151.stdout: debug ms = 1 2024-09-16T15:38:19.111 INFO:teuthology.orchestra.run.smithi151.stdout: debug osd = 20 2024-09-16T15:38:19.111 INFO:teuthology.orchestra.run.smithi151.stdout: debug rocksdb = 4/10 2024-09-16T15:38:19.111 INFO:teuthology.orchestra.run.smithi151.stdout: mon osd backfillfull_ratio = 0.85 2024-09-16T15:38:19.111 INFO:teuthology.orchestra.run.smithi151.stdout: mon osd full ratio = 0.9 2024-09-16T15:38:19.111 INFO:teuthology.orchestra.run.smithi151.stdout: mon osd nearfull ratio = 0.8 2024-09-16T15:38:19.111 INFO:teuthology.orchestra.run.smithi151.stdout: osd failsafe full ratio = 0.95 2024-09-16T15:38:19.111 INFO:teuthology.orchestra.run.smithi151.stdout: osd objectstore = bluestore 2024-09-16T15:38:19.111 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:38:19.111 INFO:teuthology.orchestra.run.smithi151.stdout:[mgr] 2024-09-16T15:38:19.112 INFO:teuthology.orchestra.run.smithi151.stdout: debug ms = 1 2024-09-16T15:38:19.112 INFO:teuthology.orchestra.run.smithi151.stdout: debug mgr = 20 2024-09-16T15:38:19.112 INFO:teuthology.orchestra.run.smithi151.stdout: debug mon = 20 2024-09-16T15:38:19.112 INFO:teuthology.orchestra.run.smithi151.stdout: debug auth = 20 2024-09-16T15:38:19.112 INFO:teuthology.orchestra.run.smithi151.stdout: mon reweight min pgs per osd = 4 2024-09-16T15:38:19.112 INFO:teuthology.orchestra.run.smithi151.stdout: mon reweight min bytes per osd = 10 2024-09-16T15:38:19.112 INFO:teuthology.orchestra.run.smithi151.stdout: mgr/telemetry/nag = false 2024-09-16T15:38:19.112 INFO:teuthology.orchestra.run.smithi151.stdout: client mount timeout = 30 2024-09-16T15:38:19.112 INFO:teuthology.orchestra.run.smithi151.stdout: debug client = 20 2024-09-16T15:38:19.112 INFO:teuthology.orchestra.run.smithi151.stdout: debug finisher = 20 2024-09-16T15:38:19.112 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:38:19.112 INFO:teuthology.orchestra.run.smithi151.stdout:[mon] 2024-09-16T15:38:19.112 INFO:teuthology.orchestra.run.smithi151.stdout: debug ms = 1 2024-09-16T15:38:19.112 INFO:teuthology.orchestra.run.smithi151.stdout: debug mon = 20 2024-09-16T15:38:19.113 INFO:teuthology.orchestra.run.smithi151.stdout: debug paxos = 20 2024-09-16T15:38:19.113 INFO:teuthology.orchestra.run.smithi151.stdout: debug auth = 20 2024-09-16T15:38:19.113 INFO:teuthology.orchestra.run.smithi151.stdout: mon data avail warn = 5 2024-09-16T15:38:19.113 INFO:teuthology.orchestra.run.smithi151.stdout: mon mgr mkfs grace = 240 2024-09-16T15:38:19.113 INFO:teuthology.orchestra.run.smithi151.stdout: mon reweight min pgs per osd = 4 2024-09-16T15:38:19.113 INFO:teuthology.orchestra.run.smithi151.stdout: mon osd reporter subtree level = osd 2024-09-16T15:38:19.113 INFO:teuthology.orchestra.run.smithi151.stdout: mon osd prime pg temp = true 2024-09-16T15:38:19.113 INFO:teuthology.orchestra.run.smithi151.stdout: mon reweight min bytes per osd = 10 2024-09-16T15:38:19.113 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:38:19.113 INFO:teuthology.orchestra.run.smithi151.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-16T15:38:19.113 INFO:teuthology.orchestra.run.smithi151.stdout: auth mon ticket ttl = 660 # 11m 2024-09-16T15:38:19.113 INFO:teuthology.orchestra.run.smithi151.stdout: auth service ticket ttl = 240 # 4m 2024-09-16T15:38:19.113 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:38:19.113 INFO:teuthology.orchestra.run.smithi151.stdout: # don't complain about insecure global_id in the test suite 2024-09-16T15:38:19.114 INFO:teuthology.orchestra.run.smithi151.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-16T15:38:19.114 INFO:teuthology.orchestra.run.smithi151.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-16T15:38:19.114 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:38:19.114 INFO:teuthology.orchestra.run.smithi151.stdout: # 1m isn't quite enough 2024-09-16T15:38:19.114 INFO:teuthology.orchestra.run.smithi151.stdout: mon_down_mkfs_grace = 2m 2024-09-16T15:38:19.114 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:38:19.114 INFO:teuthology.orchestra.run.smithi151.stdout: mon_warn_on_filestore_osds = false 2024-09-16T15:38:19.114 INFO:teuthology.orchestra.run.smithi151.stdout: mon down mkfs grace = 300 2024-09-16T15:38:19.114 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:38:19.114 INFO:teuthology.orchestra.run.smithi151.stdout:[client] 2024-09-16T15:38:19.114 INFO:teuthology.orchestra.run.smithi151.stdout: rgw cache enabled = true 2024-09-16T15:38:19.114 INFO:teuthology.orchestra.run.smithi151.stdout: rgw enable ops log = true 2024-09-16T15:38:19.114 INFO:teuthology.orchestra.run.smithi151.stdout: rgw enable usage log = true 2024-09-16T15:38:19.114 INFO:teuthology.orchestra.run.smithi151.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-16T15:38:19.115 INFO:teuthology.orchestra.run.smithi151.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-16T15:38:19.115 INFO:teuthology.orchestra.run.smithi151.stdout: pid file = "" 2024-09-16T15:38:19.115 INFO:teuthology.orchestra.run.smithi151.stdout:[mon.a] 2024-09-16T15:38:19.115 INFO:teuthology.orchestra.run.smithi151.stdout:[mon.b] 2024-09-16T15:38:19.115 INFO:teuthology.orchestra.run.smithi151.stdout:[mon.c] 2024-09-16T15:38:19.115 INFO:teuthology.orchestra.run.smithi151.stdout:[mds.a] 2024-09-16T15:38:19.115 INFO:teuthology.orchestra.run.smithi151.stdout:[mds.c] 2024-09-16T15:38:19.115 INFO:teuthology.orchestra.run.smithi151.stdout:[mds.b] 2024-09-16T15:38:19.115 INFO:teuthology.orchestra.run.smithi151.stdout:[mds.d] 2024-09-16T15:38:19.115 INFO:teuthology.orchestra.run.smithi167.stdout:[global] 2024-09-16T15:38:19.116 INFO:teuthology.orchestra.run.smithi167.stdout: chdir = "" 2024-09-16T15:38:19.116 INFO:teuthology.orchestra.run.smithi167.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-16T15:38:19.116 INFO:teuthology.orchestra.run.smithi167.stdout: auth supported = cephx 2024-09-16T15:38:19.116 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-16T15:38:19.116 INFO:teuthology.orchestra.run.smithi167.stdout: filestore xattr use omap = true 2024-09-16T15:38:19.116 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-16T15:38:19.116 INFO:teuthology.orchestra.run.smithi167.stdout: mon clock drift allowed = 1.000 2024-09-16T15:38:19.116 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-16T15:38:19.116 INFO:teuthology.orchestra.run.smithi167.stdout: osd crush chooseleaf type = 0 2024-09-16T15:38:19.116 INFO:teuthology.orchestra.run.smithi167.stdout: auth debug = true 2024-09-16T15:38:19.116 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-16T15:38:19.116 INFO:teuthology.orchestra.run.smithi167.stdout: ms die on old message = true 2024-09-16T15:38:19.116 INFO:teuthology.orchestra.run.smithi167.stdout: ms die on bug = true 2024-09-16T15:38:19.116 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-16T15:38:19.117 INFO:teuthology.orchestra.run.smithi167.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-16T15:38:19.117 INFO:teuthology.orchestra.run.smithi167.stdout: mon pg warn max object skew = 0 2024-09-16T15:38:19.117 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-16T15:38:19.117 INFO:teuthology.orchestra.run.smithi167.stdout: # disable pg_autoscaler by default for new pools 2024-09-16T15:38:19.117 INFO:teuthology.orchestra.run.smithi167.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-16T15:38:19.117 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-16T15:38:19.117 INFO:teuthology.orchestra.run.smithi167.stdout: osd pool default size = 2 2024-09-16T15:38:19.117 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-16T15:38:19.117 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd allow primary affinity = true 2024-09-16T15:38:19.117 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd allow pg remap = true 2024-09-16T15:38:19.117 INFO:teuthology.orchestra.run.smithi167.stdout: mon warn on legacy crush tunables = false 2024-09-16T15:38:19.117 INFO:teuthology.orchestra.run.smithi167.stdout: mon warn on crush straw calc version zero = false 2024-09-16T15:38:19.117 INFO:teuthology.orchestra.run.smithi167.stdout: mon warn on no sortbitwise = false 2024-09-16T15:38:19.117 INFO:teuthology.orchestra.run.smithi167.stdout: mon warn on osd down out interval zero = false 2024-09-16T15:38:19.118 INFO:teuthology.orchestra.run.smithi167.stdout: mon warn on too few osds = false 2024-09-16T15:38:19.118 INFO:teuthology.orchestra.run.smithi167.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-16T15:38:19.118 INFO:teuthology.orchestra.run.smithi167.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-16T15:38:19.118 INFO:teuthology.orchestra.run.smithi167.stdout: mon_allow_pool_size_one = true 2024-09-16T15:38:19.118 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-16T15:38:19.118 INFO:teuthology.orchestra.run.smithi167.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-16T15:38:19.118 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-16T15:38:19.118 INFO:teuthology.orchestra.run.smithi167.stdout: osd default data pool replay window = 5 2024-09-16T15:38:19.118 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-16T15:38:19.118 INFO:teuthology.orchestra.run.smithi167.stdout: mon allow pool delete = true 2024-09-16T15:38:19.118 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-16T15:38:19.118 INFO:teuthology.orchestra.run.smithi167.stdout: mon cluster log file level = debug 2024-09-16T15:38:19.118 INFO:teuthology.orchestra.run.smithi167.stdout: debug asserts on shutdown = true 2024-09-16T15:38:19.118 INFO:teuthology.orchestra.run.smithi167.stdout: mon health detail to clog = false 2024-09-16T15:38:19.118 INFO:teuthology.orchestra.run.smithi167.stdout: mon host = "172.21.15.151,172.21.15.167,[v2:172.21.15.167:3301,v1:172.21.15.167:6790]" 2024-09-16T15:38:19.119 INFO:teuthology.orchestra.run.smithi167.stdout: ms die on skipped message = False 2024-09-16T15:38:19.119 INFO:teuthology.orchestra.run.smithi167.stdout: fsid = 1757be18-e41a-4446-bb4f-ee4ac81c39df 2024-09-16T15:38:19.119 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-16T15:38:19.119 INFO:teuthology.orchestra.run.smithi167.stdout:[osd] 2024-09-16T15:38:19.119 INFO:teuthology.orchestra.run.smithi167.stdout: osd journal size = 100 2024-09-16T15:38:19.119 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-16T15:38:19.119 INFO:teuthology.orchestra.run.smithi167.stdout: osd scrub load threshold = 5.0 2024-09-16T15:38:19.119 INFO:teuthology.orchestra.run.smithi167.stdout: osd scrub max interval = 600 2024-09-16T15:38:19.119 INFO:teuthology.orchestra.run.smithi167.stdout: osd mclock profile = high_recovery_ops 2024-09-16T15:38:19.119 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-16T15:38:19.119 INFO:teuthology.orchestra.run.smithi167.stdout: osd recover clone overlap = true 2024-09-16T15:38:19.119 INFO:teuthology.orchestra.run.smithi167.stdout: osd recovery max chunk = 1048576 2024-09-16T15:38:19.119 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-16T15:38:19.119 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug shutdown = true 2024-09-16T15:38:19.120 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug op order = true 2024-09-16T15:38:19.120 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug verify stray on activate = true 2024-09-16T15:38:19.120 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug trim objects = true 2024-09-16T15:38:19.120 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-16T15:38:19.120 INFO:teuthology.orchestra.run.smithi167.stdout: osd open classes on start = true 2024-09-16T15:38:19.120 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug pg log writeout = true 2024-09-16T15:38:19.120 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-16T15:38:19.120 INFO:teuthology.orchestra.run.smithi167.stdout: osd deep scrub update digest min age = 30 2024-09-16T15:38:19.120 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-16T15:38:19.120 INFO:teuthology.orchestra.run.smithi167.stdout: osd map max advance = 10 2024-09-16T15:38:19.120 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-16T15:38:19.120 INFO:teuthology.orchestra.run.smithi167.stdout: journal zero on create = true 2024-09-16T15:38:19.120 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-16T15:38:19.120 INFO:teuthology.orchestra.run.smithi167.stdout: filestore ondisk finisher threads = 3 2024-09-16T15:38:19.120 INFO:teuthology.orchestra.run.smithi167.stdout: filestore apply finisher threads = 3 2024-09-16T15:38:19.120 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-16T15:38:19.121 INFO:teuthology.orchestra.run.smithi167.stdout: bdev debug aio = true 2024-09-16T15:38:19.121 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug misdirected ops = true 2024-09-16T15:38:19.121 INFO:teuthology.orchestra.run.smithi167.stdout: bdev async discard = True 2024-09-16T15:38:19.121 INFO:teuthology.orchestra.run.smithi167.stdout: bdev enable discard = True 2024-09-16T15:38:19.121 INFO:teuthology.orchestra.run.smithi167.stdout: bluestore allocator = bitmap 2024-09-16T15:38:19.121 INFO:teuthology.orchestra.run.smithi167.stdout: bluestore block size = 96636764160 2024-09-16T15:38:19.121 INFO:teuthology.orchestra.run.smithi167.stdout: bluestore fsck on mount = True 2024-09-16T15:38:19.121 INFO:teuthology.orchestra.run.smithi167.stdout: debug bluefs = 1/20 2024-09-16T15:38:19.121 INFO:teuthology.orchestra.run.smithi167.stdout: debug bluestore = 1/20 2024-09-16T15:38:19.121 INFO:teuthology.orchestra.run.smithi167.stdout: debug ms = 1 2024-09-16T15:38:19.121 INFO:teuthology.orchestra.run.smithi167.stdout: debug osd = 20 2024-09-16T15:38:19.121 INFO:teuthology.orchestra.run.smithi167.stdout: debug rocksdb = 4/10 2024-09-16T15:38:19.121 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd backfillfull_ratio = 0.85 2024-09-16T15:38:19.121 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd full ratio = 0.9 2024-09-16T15:38:19.122 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd nearfull ratio = 0.8 2024-09-16T15:38:19.122 INFO:teuthology.orchestra.run.smithi167.stdout: osd failsafe full ratio = 0.95 2024-09-16T15:38:19.122 INFO:teuthology.orchestra.run.smithi167.stdout: osd objectstore = bluestore 2024-09-16T15:38:19.122 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-16T15:38:19.122 INFO:teuthology.orchestra.run.smithi167.stdout:[mgr] 2024-09-16T15:38:19.122 INFO:teuthology.orchestra.run.smithi167.stdout: debug ms = 1 2024-09-16T15:38:19.122 INFO:teuthology.orchestra.run.smithi167.stdout: debug mgr = 20 2024-09-16T15:38:19.122 INFO:teuthology.orchestra.run.smithi167.stdout: debug mon = 20 2024-09-16T15:38:19.122 INFO:teuthology.orchestra.run.smithi167.stdout: debug auth = 20 2024-09-16T15:38:19.122 INFO:teuthology.orchestra.run.smithi167.stdout: mon reweight min pgs per osd = 4 2024-09-16T15:38:19.122 INFO:teuthology.orchestra.run.smithi167.stdout: mon reweight min bytes per osd = 10 2024-09-16T15:38:19.122 INFO:teuthology.orchestra.run.smithi167.stdout: mgr/telemetry/nag = false 2024-09-16T15:38:19.122 INFO:teuthology.orchestra.run.smithi167.stdout: client mount timeout = 30 2024-09-16T15:38:19.122 INFO:teuthology.orchestra.run.smithi167.stdout: debug client = 20 2024-09-16T15:38:19.123 INFO:teuthology.orchestra.run.smithi167.stdout: debug finisher = 20 2024-09-16T15:38:19.123 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-16T15:38:19.123 INFO:teuthology.orchestra.run.smithi167.stdout:[mon] 2024-09-16T15:38:19.123 INFO:teuthology.orchestra.run.smithi167.stdout: debug ms = 1 2024-09-16T15:38:19.123 INFO:teuthology.orchestra.run.smithi167.stdout: debug mon = 20 2024-09-16T15:38:19.123 INFO:teuthology.orchestra.run.smithi167.stdout: debug paxos = 20 2024-09-16T15:38:19.123 INFO:teuthology.orchestra.run.smithi167.stdout: debug auth = 20 2024-09-16T15:38:19.123 INFO:teuthology.orchestra.run.smithi167.stdout: mon data avail warn = 5 2024-09-16T15:38:19.123 INFO:teuthology.orchestra.run.smithi167.stdout: mon mgr mkfs grace = 240 2024-09-16T15:38:19.123 INFO:teuthology.orchestra.run.smithi167.stdout: mon reweight min pgs per osd = 4 2024-09-16T15:38:19.123 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd reporter subtree level = osd 2024-09-16T15:38:19.123 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd prime pg temp = true 2024-09-16T15:38:19.123 INFO:teuthology.orchestra.run.smithi167.stdout: mon reweight min bytes per osd = 10 2024-09-16T15:38:19.123 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-16T15:38:19.123 INFO:teuthology.orchestra.run.smithi167.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-16T15:38:19.124 INFO:teuthology.orchestra.run.smithi167.stdout: auth mon ticket ttl = 660 # 11m 2024-09-16T15:38:19.124 INFO:teuthology.orchestra.run.smithi167.stdout: auth service ticket ttl = 240 # 4m 2024-09-16T15:38:19.124 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-16T15:38:19.124 INFO:teuthology.orchestra.run.smithi167.stdout: # don't complain about insecure global_id in the test suite 2024-09-16T15:38:19.124 INFO:teuthology.orchestra.run.smithi167.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-16T15:38:19.124 INFO:teuthology.orchestra.run.smithi167.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-16T15:38:19.124 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-16T15:38:19.124 INFO:teuthology.orchestra.run.smithi167.stdout: # 1m isn't quite enough 2024-09-16T15:38:19.124 INFO:teuthology.orchestra.run.smithi167.stdout: mon_down_mkfs_grace = 2m 2024-09-16T15:38:19.124 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-16T15:38:19.124 INFO:teuthology.orchestra.run.smithi167.stdout: mon_warn_on_filestore_osds = false 2024-09-16T15:38:19.124 INFO:teuthology.orchestra.run.smithi167.stdout: mon down mkfs grace = 300 2024-09-16T15:38:19.124 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-16T15:38:19.124 INFO:teuthology.orchestra.run.smithi167.stdout:[client] 2024-09-16T15:38:19.124 INFO:teuthology.orchestra.run.smithi167.stdout: rgw cache enabled = true 2024-09-16T15:38:19.124 INFO:teuthology.orchestra.run.smithi167.stdout: rgw enable ops log = true 2024-09-16T15:38:19.125 INFO:teuthology.orchestra.run.smithi167.stdout: rgw enable usage log = true 2024-09-16T15:38:19.125 INFO:teuthology.orchestra.run.smithi167.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-16T15:38:19.125 INFO:teuthology.orchestra.run.smithi167.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-16T15:38:19.125 INFO:teuthology.orchestra.run.smithi167.stdout: pid file = "" 2024-09-16T15:38:19.125 INFO:teuthology.orchestra.run.smithi167.stdout:[mon.a] 2024-09-16T15:38:19.125 INFO:teuthology.orchestra.run.smithi167.stdout:[mon.b] 2024-09-16T15:38:19.125 INFO:teuthology.orchestra.run.smithi167.stdout:[mon.c] 2024-09-16T15:38:19.125 INFO:teuthology.orchestra.run.smithi167.stdout:[mds.a] 2024-09-16T15:38:19.125 INFO:teuthology.orchestra.run.smithi167.stdout:[mds.c] 2024-09-16T15:38:19.125 INFO:teuthology.orchestra.run.smithi167.stdout:[mds.b] 2024-09-16T15:38:19.125 INFO:teuthology.orchestra.run.smithi167.stdout:[mds.d] 2024-09-16T15:38:19.135 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-16T15:38:19.136 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-16T15:38:19.233 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-16T15:38:19.233 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:38:19.233 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-16T15:38:19.268 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:38:19.268 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-16T15:38:19.343 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi151.front.sepia.ceph.com 2024-09-16T15:38:19.343 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:38:19.343 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-16T15:38:19.344 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-16T15:38:19.461 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:38:19.461 DEBUG:teuthology.orchestra.run.smithi151:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-16T15:38:19.537 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi167.front.sepia.ceph.com 2024-09-16T15:38:19.537 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-16T15:38:19.537 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-16T15:38:19.537 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-16T15:38:19.615 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-16T15:38:19.615 DEBUG:teuthology.orchestra.run.smithi167:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-16T15:38:19.690 INFO:tasks.ceph:Setting up mon nodes... 2024-09-16T15:38:19.690 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-16T15:38:19.690 DEBUG:teuthology.orchestra.run.smithi151:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-16T15:38:19.793 INFO:teuthology.orchestra.run.smithi151.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-16T15:38:19.799 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-16T15:38:19.935 INFO:teuthology.orchestra.run.smithi167.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-16T15:38:19.942 INFO:tasks.ceph:Setting up mds nodes... 2024-09-16T15:38:19.942 DEBUG:teuthology.orchestra.run.smithi151:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-09-16T15:38:20.046 INFO:teuthology.orchestra.run.smithi151.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-09-16T15:38:20.051 DEBUG:teuthology.orchestra.run.smithi151:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-09-16T15:38:20.107 DEBUG:teuthology.orchestra.run.smithi151:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-09-16T15:38:20.210 INFO:teuthology.orchestra.run.smithi151.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-09-16T15:38:20.216 DEBUG:teuthology.orchestra.run.smithi151:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-09-16T15:38:20.271 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-09-16T15:38:20.377 INFO:teuthology.orchestra.run.smithi167.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-09-16T15:38:20.382 DEBUG:teuthology.orchestra.run.smithi167:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-09-16T15:38:20.437 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-09-16T15:38:20.540 INFO:teuthology.orchestra.run.smithi167.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-09-16T15:38:20.546 DEBUG:teuthology.orchestra.run.smithi167:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-09-16T15:38:20.601 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-16T15:38:20.602 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-16T15:38:20.683 INFO:teuthology.orchestra.run.smithi151.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-16T15:38:20.711 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-09-16T15:38:20.792 INFO:teuthology.orchestra.run.smithi151.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-09-16T15:38:20.819 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-16T15:38:20.900 INFO:teuthology.orchestra.run.smithi167.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-16T15:38:20.928 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-09-16T15:38:21.009 INFO:teuthology.orchestra.run.smithi167.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-09-16T15:38:21.036 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-16T15:38:21.037 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi151.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi167.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-16T15:38:21.037 DEBUG:teuthology.orchestra.run.smithi151:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-16T15:38:21.093 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-16T15:38:21.093 INFO:tasks.ceph:role: osd.0 2024-09-16T15:38:21.093 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi151.front.sepia.ceph.com 2024-09-16T15:38:21.093 DEBUG:teuthology.orchestra.run.smithi151:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-16T15:38:21.155 INFO:teuthology.orchestra.run.smithi151.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T15:38:21.155 INFO:teuthology.orchestra.run.smithi151.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T15:38:21.156 INFO:teuthology.orchestra.run.smithi151.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T15:38:21.156 INFO:teuthology.orchestra.run.smithi151.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T15:38:21.156 INFO:teuthology.orchestra.run.smithi151.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T15:38:21.156 INFO:teuthology.orchestra.run.smithi151.stdout: = sunit=0 swidth=0 blks 2024-09-16T15:38:21.156 INFO:teuthology.orchestra.run.smithi151.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T15:38:21.156 INFO:teuthology.orchestra.run.smithi151.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T15:38:21.156 INFO:teuthology.orchestra.run.smithi151.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T15:38:21.156 INFO:teuthology.orchestra.run.smithi151.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T15:38:21.569 INFO:teuthology.orchestra.run.smithi151.stdout:Discarding blocks...Done. 2024-09-16T15:38:21.573 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi151.front.sepia.ceph.com -o noatime 2024-09-16T15:38:21.573 DEBUG:teuthology.orchestra.run.smithi151:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-16T15:38:21.707 DEBUG:teuthology.orchestra.run.smithi151:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-16T15:38:21.770 DEBUG:teuthology.orchestra.run.smithi151:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-16T15:38:21.825 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-16T15:38:21.826 INFO:tasks.ceph:role: osd.1 2024-09-16T15:38:21.826 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi151.front.sepia.ceph.com 2024-09-16T15:38:21.826 DEBUG:teuthology.orchestra.run.smithi151:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-16T15:38:21.888 INFO:teuthology.orchestra.run.smithi151.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T15:38:21.888 INFO:teuthology.orchestra.run.smithi151.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T15:38:21.888 INFO:teuthology.orchestra.run.smithi151.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T15:38:21.888 INFO:teuthology.orchestra.run.smithi151.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T15:38:21.888 INFO:teuthology.orchestra.run.smithi151.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T15:38:21.888 INFO:teuthology.orchestra.run.smithi151.stdout: = sunit=0 swidth=0 blks 2024-09-16T15:38:21.888 INFO:teuthology.orchestra.run.smithi151.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T15:38:21.888 INFO:teuthology.orchestra.run.smithi151.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T15:38:21.889 INFO:teuthology.orchestra.run.smithi151.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T15:38:21.889 INFO:teuthology.orchestra.run.smithi151.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T15:38:22.247 INFO:teuthology.orchestra.run.smithi151.stdout:Discarding blocks...Done. 2024-09-16T15:38:22.251 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi151.front.sepia.ceph.com -o noatime 2024-09-16T15:38:22.251 DEBUG:teuthology.orchestra.run.smithi151:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-16T15:38:22.336 DEBUG:teuthology.orchestra.run.smithi151:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-16T15:38:22.398 DEBUG:teuthology.orchestra.run.smithi151:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-16T15:38:22.453 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-16T15:38:22.454 INFO:tasks.ceph:role: osd.2 2024-09-16T15:38:22.454 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi151.front.sepia.ceph.com 2024-09-16T15:38:22.454 DEBUG:teuthology.orchestra.run.smithi151:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-16T15:38:22.517 INFO:teuthology.orchestra.run.smithi151.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T15:38:22.517 INFO:teuthology.orchestra.run.smithi151.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T15:38:22.517 INFO:teuthology.orchestra.run.smithi151.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T15:38:22.517 INFO:teuthology.orchestra.run.smithi151.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T15:38:22.517 INFO:teuthology.orchestra.run.smithi151.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T15:38:22.517 INFO:teuthology.orchestra.run.smithi151.stdout: = sunit=0 swidth=0 blks 2024-09-16T15:38:22.517 INFO:teuthology.orchestra.run.smithi151.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T15:38:22.517 INFO:teuthology.orchestra.run.smithi151.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T15:38:22.517 INFO:teuthology.orchestra.run.smithi151.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T15:38:22.517 INFO:teuthology.orchestra.run.smithi151.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T15:38:22.907 INFO:teuthology.orchestra.run.smithi151.stdout:Discarding blocks...Done. 2024-09-16T15:38:22.910 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi151.front.sepia.ceph.com -o noatime 2024-09-16T15:38:22.911 DEBUG:teuthology.orchestra.run.smithi151:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-16T15:38:23.011 DEBUG:teuthology.orchestra.run.smithi151:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-16T15:38:23.074 DEBUG:teuthology.orchestra.run.smithi151:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-16T15:38:23.129 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-16T15:38:23.129 INFO:tasks.ceph:role: osd.3 2024-09-16T15:38:23.129 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi151.front.sepia.ceph.com 2024-09-16T15:38:23.130 DEBUG:teuthology.orchestra.run.smithi151:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-16T15:38:23.192 INFO:teuthology.orchestra.run.smithi151.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T15:38:23.192 INFO:teuthology.orchestra.run.smithi151.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T15:38:23.192 INFO:teuthology.orchestra.run.smithi151.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T15:38:23.192 INFO:teuthology.orchestra.run.smithi151.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T15:38:23.192 INFO:teuthology.orchestra.run.smithi151.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T15:38:23.192 INFO:teuthology.orchestra.run.smithi151.stdout: = sunit=0 swidth=0 blks 2024-09-16T15:38:23.193 INFO:teuthology.orchestra.run.smithi151.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T15:38:23.193 INFO:teuthology.orchestra.run.smithi151.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T15:38:23.193 INFO:teuthology.orchestra.run.smithi151.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T15:38:23.193 INFO:teuthology.orchestra.run.smithi151.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T15:38:23.618 INFO:teuthology.orchestra.run.smithi151.stdout:Discarding blocks...Done. 2024-09-16T15:38:23.622 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi151.front.sepia.ceph.com -o noatime 2024-09-16T15:38:23.622 DEBUG:teuthology.orchestra.run.smithi151:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-16T15:38:23.769 DEBUG:teuthology.orchestra.run.smithi151:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-16T15:38:23.834 DEBUG:teuthology.orchestra.run.smithi151:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T15:38:23.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:23.934+0000 7f47fc42a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-16T15:38:23.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:23.936+0000 7f47fc42a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-16T15:38:23.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:23.936+0000 7f47fc42a640 -1 bdev(0x555e9e8dd800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-16T15:38:23.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:23.936+0000 7f47fc42a640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-16T15:38:26.967 DEBUG:teuthology.orchestra.run.smithi151:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-16T15:38:27.023 DEBUG:teuthology.orchestra.run.smithi151:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T15:38:27.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:27.113+0000 7f0c9f264640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-16T15:38:27.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:27.114+0000 7f0c9f264640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-16T15:38:27.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:27.114+0000 7f0c9f264640 -1 bdev(0x55f711f29800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-16T15:38:27.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:27.114+0000 7f0c9f264640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-16T15:38:30.147 DEBUG:teuthology.orchestra.run.smithi151:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-16T15:38:30.203 DEBUG:teuthology.orchestra.run.smithi151:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T15:38:30.294 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:30.293+0000 7f74b9a0f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-16T15:38:30.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:30.293+0000 7f74b9a0f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-16T15:38:30.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:30.293+0000 7f74b9a0f640 -1 bdev(0x556ed3fa1800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-16T15:38:30.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:30.294+0000 7f74b9a0f640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-16T15:38:33.288 DEBUG:teuthology.orchestra.run.smithi151:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-16T15:38:33.344 DEBUG:teuthology.orchestra.run.smithi151:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T15:38:33.434 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:33.433+0000 7f04adc0c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-16T15:38:33.434 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:33.433+0000 7f04adc0c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-16T15:38:33.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:33.433+0000 7f04adc0c640 -1 bdev(0x5644ce9b3800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-16T15:38:33.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:33.433+0000 7f04adc0c640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-16T15:38:36.455 DEBUG:teuthology.orchestra.run.smithi151:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-16T15:38:36.512 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-16T15:38:36.574 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-16T15:38:36.574 INFO:tasks.ceph:role: osd.4 2024-09-16T15:38:36.574 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi167.front.sepia.ceph.com 2024-09-16T15:38:36.574 DEBUG:teuthology.orchestra.run.smithi167:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-16T15:38:36.638 INFO:teuthology.orchestra.run.smithi167.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T15:38:36.638 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T15:38:36.638 INFO:teuthology.orchestra.run.smithi167.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T15:38:36.638 INFO:teuthology.orchestra.run.smithi167.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T15:38:36.638 INFO:teuthology.orchestra.run.smithi167.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T15:38:36.638 INFO:teuthology.orchestra.run.smithi167.stdout: = sunit=0 swidth=0 blks 2024-09-16T15:38:36.639 INFO:teuthology.orchestra.run.smithi167.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T15:38:36.639 INFO:teuthology.orchestra.run.smithi167.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T15:38:36.639 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T15:38:36.639 INFO:teuthology.orchestra.run.smithi167.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T15:38:37.047 INFO:teuthology.orchestra.run.smithi167.stdout:Discarding blocks...Done. 2024-09-16T15:38:37.051 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi167.front.sepia.ceph.com -o noatime 2024-09-16T15:38:37.051 DEBUG:teuthology.orchestra.run.smithi167:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-16T15:38:37.197 DEBUG:teuthology.orchestra.run.smithi167:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-16T15:38:37.264 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-16T15:38:37.323 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-16T15:38:37.323 INFO:tasks.ceph:role: osd.5 2024-09-16T15:38:37.323 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi167.front.sepia.ceph.com 2024-09-16T15:38:37.323 DEBUG:teuthology.orchestra.run.smithi167:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-16T15:38:37.388 INFO:teuthology.orchestra.run.smithi167.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T15:38:37.388 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T15:38:37.388 INFO:teuthology.orchestra.run.smithi167.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T15:38:37.388 INFO:teuthology.orchestra.run.smithi167.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T15:38:37.388 INFO:teuthology.orchestra.run.smithi167.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T15:38:37.388 INFO:teuthology.orchestra.run.smithi167.stdout: = sunit=0 swidth=0 blks 2024-09-16T15:38:37.388 INFO:teuthology.orchestra.run.smithi167.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T15:38:37.388 INFO:teuthology.orchestra.run.smithi167.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T15:38:37.388 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T15:38:37.388 INFO:teuthology.orchestra.run.smithi167.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T15:38:37.741 INFO:teuthology.orchestra.run.smithi167.stdout:Discarding blocks...Done. 2024-09-16T15:38:37.747 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi167.front.sepia.ceph.com -o noatime 2024-09-16T15:38:37.747 DEBUG:teuthology.orchestra.run.smithi167:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-16T15:38:37.832 DEBUG:teuthology.orchestra.run.smithi167:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-16T15:38:37.894 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-16T15:38:37.951 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-16T15:38:37.951 INFO:tasks.ceph:role: osd.6 2024-09-16T15:38:37.951 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi167.front.sepia.ceph.com 2024-09-16T15:38:37.951 DEBUG:teuthology.orchestra.run.smithi167:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-16T15:38:38.015 INFO:teuthology.orchestra.run.smithi167.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T15:38:38.015 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T15:38:38.015 INFO:teuthology.orchestra.run.smithi167.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T15:38:38.015 INFO:teuthology.orchestra.run.smithi167.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T15:38:38.015 INFO:teuthology.orchestra.run.smithi167.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T15:38:38.015 INFO:teuthology.orchestra.run.smithi167.stdout: = sunit=0 swidth=0 blks 2024-09-16T15:38:38.015 INFO:teuthology.orchestra.run.smithi167.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T15:38:38.015 INFO:teuthology.orchestra.run.smithi167.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T15:38:38.015 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T15:38:38.015 INFO:teuthology.orchestra.run.smithi167.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T15:38:38.402 INFO:teuthology.orchestra.run.smithi167.stdout:Discarding blocks...Done. 2024-09-16T15:38:38.406 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi167.front.sepia.ceph.com -o noatime 2024-09-16T15:38:38.406 DEBUG:teuthology.orchestra.run.smithi167:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-16T15:38:38.506 DEBUG:teuthology.orchestra.run.smithi167:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-16T15:38:38.572 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-16T15:38:38.628 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-16T15:38:38.628 INFO:tasks.ceph:role: osd.7 2024-09-16T15:38:38.628 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi167.front.sepia.ceph.com 2024-09-16T15:38:38.628 DEBUG:teuthology.orchestra.run.smithi167:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-16T15:38:38.691 INFO:teuthology.orchestra.run.smithi167.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T15:38:38.691 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T15:38:38.691 INFO:teuthology.orchestra.run.smithi167.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T15:38:38.691 INFO:teuthology.orchestra.run.smithi167.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T15:38:38.691 INFO:teuthology.orchestra.run.smithi167.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T15:38:38.691 INFO:teuthology.orchestra.run.smithi167.stdout: = sunit=0 swidth=0 blks 2024-09-16T15:38:38.691 INFO:teuthology.orchestra.run.smithi167.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T15:38:38.691 INFO:teuthology.orchestra.run.smithi167.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T15:38:38.692 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T15:38:38.692 INFO:teuthology.orchestra.run.smithi167.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T15:38:39.114 INFO:teuthology.orchestra.run.smithi167.stdout:Discarding blocks...Done. 2024-09-16T15:38:39.118 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi167.front.sepia.ceph.com -o noatime 2024-09-16T15:38:39.118 DEBUG:teuthology.orchestra.run.smithi167:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-16T15:38:39.257 DEBUG:teuthology.orchestra.run.smithi167:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-16T15:38:39.320 DEBUG:teuthology.orchestra.run.smithi167:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T15:38:39.435 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:39.432+0000 7fc932416640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-16T15:38:39.436 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:39.434+0000 7fc932416640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-16T15:38:39.436 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:39.434+0000 7fc932416640 -1 bdev(0x557b4484f800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-16T15:38:39.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:39.434+0000 7fc932416640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-16T15:38:42.454 DEBUG:teuthology.orchestra.run.smithi167:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-16T15:38:42.513 DEBUG:teuthology.orchestra.run.smithi167:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T15:38:42.604 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:42.602+0000 7f8b1a45e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-16T15:38:42.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:42.602+0000 7f8b1a45e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-16T15:38:42.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:42.602+0000 7f8b1a45e640 -1 bdev(0x561bc7d61800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-16T15:38:42.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:42.602+0000 7f8b1a45e640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-16T15:38:45.620 DEBUG:teuthology.orchestra.run.smithi167:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-16T15:38:45.676 DEBUG:teuthology.orchestra.run.smithi167:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T15:38:45.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:45.767+0000 7fdccbb58640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-16T15:38:45.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:45.767+0000 7fdccbb58640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-16T15:38:45.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:45.767+0000 7fdccbb58640 -1 bdev(0x558e2408b800 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-16T15:38:45.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:45.767+0000 7fdccbb58640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-16T15:38:46.609 DEBUG:teuthology.orchestra.run.smithi151:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:38:46.613 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:38:48.786 DEBUG:teuthology.orchestra.run.smithi167:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-16T15:38:48.842 DEBUG:teuthology.orchestra.run.smithi167:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T15:38:48.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:48.930+0000 7f5bef614640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-16T15:38:48.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:48.931+0000 7f5bef614640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-16T15:38:48.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:48.931+0000 7f5bef614640 -1 bdev(0x55e32ae1f800 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-16T15:38:48.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:48.932+0000 7f5bef614640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-16T15:38:51.949 DEBUG:teuthology.orchestra.run.smithi167:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-16T15:38:52.006 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-16T15:38:52.007 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:38:52.007 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-16T15:38:52.069 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:38:52.069 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-09-16T15:38:52.169 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:38:52.169 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-09-16T15:38:52.266 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:38:52.266 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-16T15:38:52.361 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:38:52.361 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-16T15:38:52.456 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:38:52.456 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-16T15:38:52.550 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:38:52.550 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-16T15:38:52.646 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-16T15:38:52.646 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-16T15:38:52.708 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-16T15:38:52.708 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-09-16T15:38:52.806 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-16T15:38:52.806 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-09-16T15:38:52.902 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-16T15:38:52.902 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-16T15:38:52.997 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-16T15:38:52.997 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-16T15:38:53.092 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-16T15:38:53.092 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-16T15:38:53.187 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-16T15:38:53.187 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-16T15:38:53.282 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:38:53.282 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-16T15:38:53.317 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:38:53.317 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-09-16T15:38:53.392 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-16T15:38:53.392 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-16T15:38:53.427 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-16T15:38:53.428 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-09-16T15:38:53.502 INFO:tasks.ceph:Adding keys to all mons... 2024-09-16T15:38:53.502 DEBUG:teuthology.orchestra.run.smithi151:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-16T15:38:53.506 DEBUG:teuthology.orchestra.run.smithi167:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-16T15:38:53.555 INFO:teuthology.orchestra.run.smithi151.stdout:[mgr.y] 2024-09-16T15:38:53.555 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQDrUOhm1uZOLxAATgjA79vYf0psqQVo03sdDA== 2024-09-16T15:38:53.555 INFO:teuthology.orchestra.run.smithi151.stdout:[mds.a] 2024-09-16T15:38:53.555 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQDsUOhmJVK6AhAANfmTPS6kKn41LSD/MuyW9g== 2024-09-16T15:38:53.555 INFO:teuthology.orchestra.run.smithi151.stdout:[mds.c] 2024-09-16T15:38:53.555 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQDsUOhmYGCKDBAAY2EFNmiFNf1x2so9wxp4YQ== 2024-09-16T15:38:53.555 INFO:teuthology.orchestra.run.smithi151.stdout:[osd.0] 2024-09-16T15:38:53.555 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQDvUOhmKji5NxAAn/E3YQR7jeMnBTLBXc1G6A== 2024-09-16T15:38:53.555 INFO:teuthology.orchestra.run.smithi151.stdout:[osd.1] 2024-09-16T15:38:53.555 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQDzUOhmmBrNBhAA0Uwh4Fam2bmKKvz5jAQBWw== 2024-09-16T15:38:53.556 INFO:teuthology.orchestra.run.smithi151.stdout:[osd.2] 2024-09-16T15:38:53.556 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQD2UOhmOV6GERAAjD+PQLTZrCFR1669UEEIAQ== 2024-09-16T15:38:53.556 INFO:teuthology.orchestra.run.smithi151.stdout:[osd.3] 2024-09-16T15:38:53.556 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQD5UOhmCRrdGRAAQC0bqWX6N1jniVGEQ2aAvg== 2024-09-16T15:38:53.556 INFO:teuthology.orchestra.run.smithi151.stdout:[mgr.x] 2024-09-16T15:38:53.556 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQDrUOhmloW3NxAA8UoLUfrw3wf1nzKbmtK/jw== 2024-09-16T15:38:53.556 INFO:teuthology.orchestra.run.smithi151.stdout:[mds.b] 2024-09-16T15:38:53.556 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQDsUOhmX1R3FhAAUQgUsE/6wUyokXcZuBuIWg== 2024-09-16T15:38:53.556 INFO:teuthology.orchestra.run.smithi151.stdout:[mds.d] 2024-09-16T15:38:53.556 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQDsUOhmfCk4IBAAx12Gw2QudQ2nrPb8VlE0SA== 2024-09-16T15:38:53.556 INFO:teuthology.orchestra.run.smithi151.stdout:[osd.4] 2024-09-16T15:38:53.556 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQD/UOhmXI/nGRAA7t6XveKvrWgqy6rHuMQSZQ== 2024-09-16T15:38:53.556 INFO:teuthology.orchestra.run.smithi151.stdout:[osd.5] 2024-09-16T15:38:53.556 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQACUehmKM//IxAAsoFpiiTXNs//v3YXeUf/Iw== 2024-09-16T15:38:53.557 INFO:teuthology.orchestra.run.smithi151.stdout:[osd.6] 2024-09-16T15:38:53.557 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQAFUehmlo/FLRAAp5mvUSWCaBbm85QyTZY0eA== 2024-09-16T15:38:53.557 INFO:teuthology.orchestra.run.smithi151.stdout:[osd.7] 2024-09-16T15:38:53.557 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQAIUehmR32JNxAA27antOIKot1nScQbLZDf5g== 2024-09-16T15:38:53.557 INFO:teuthology.orchestra.run.smithi151.stdout:[client.0] 2024-09-16T15:38:53.557 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQDsUOhmf5PBKBAAKE4Y4zbA4h07IBNjPkyptw== 2024-09-16T15:38:53.557 INFO:teuthology.orchestra.run.smithi151.stdout:[client.2] 2024-09-16T15:38:53.557 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQDsUOhm6zUyLxAAkzeiIQTuNZkMM+Oit9rC5A== 2024-09-16T15:38:53.557 INFO:teuthology.orchestra.run.smithi151.stdout:[client.1] 2024-09-16T15:38:53.557 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQDsUOhm15emNRAAwhK7QmA2c9IYGn04ERvY0g== 2024-09-16T15:38:53.557 INFO:teuthology.orchestra.run.smithi151.stdout:[client.3] 2024-09-16T15:38:53.557 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQDtUOhmD82EABAAeDf82/hPxpPfAUk8oj5h/Q== 2024-09-16T15:38:53.594 INFO:teuthology.orchestra.run.smithi167.stdout:[mgr.y] 2024-09-16T15:38:53.594 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQDrUOhm1uZOLxAATgjA79vYf0psqQVo03sdDA== 2024-09-16T15:38:53.594 INFO:teuthology.orchestra.run.smithi167.stdout:[mds.a] 2024-09-16T15:38:53.594 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQDsUOhmJVK6AhAANfmTPS6kKn41LSD/MuyW9g== 2024-09-16T15:38:53.595 INFO:teuthology.orchestra.run.smithi167.stdout:[mds.c] 2024-09-16T15:38:53.595 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQDsUOhmYGCKDBAAY2EFNmiFNf1x2so9wxp4YQ== 2024-09-16T15:38:53.595 INFO:teuthology.orchestra.run.smithi167.stdout:[osd.0] 2024-09-16T15:38:53.595 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQDvUOhmKji5NxAAn/E3YQR7jeMnBTLBXc1G6A== 2024-09-16T15:38:53.595 INFO:teuthology.orchestra.run.smithi167.stdout:[osd.1] 2024-09-16T15:38:53.595 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQDzUOhmmBrNBhAA0Uwh4Fam2bmKKvz5jAQBWw== 2024-09-16T15:38:53.595 INFO:teuthology.orchestra.run.smithi167.stdout:[osd.2] 2024-09-16T15:38:53.595 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQD2UOhmOV6GERAAjD+PQLTZrCFR1669UEEIAQ== 2024-09-16T15:38:53.595 INFO:teuthology.orchestra.run.smithi167.stdout:[osd.3] 2024-09-16T15:38:53.595 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQD5UOhmCRrdGRAAQC0bqWX6N1jniVGEQ2aAvg== 2024-09-16T15:38:53.595 INFO:teuthology.orchestra.run.smithi167.stdout:[mgr.x] 2024-09-16T15:38:53.595 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQDrUOhmloW3NxAA8UoLUfrw3wf1nzKbmtK/jw== 2024-09-16T15:38:53.595 INFO:teuthology.orchestra.run.smithi167.stdout:[mds.b] 2024-09-16T15:38:53.595 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQDsUOhmX1R3FhAAUQgUsE/6wUyokXcZuBuIWg== 2024-09-16T15:38:53.595 INFO:teuthology.orchestra.run.smithi167.stdout:[mds.d] 2024-09-16T15:38:53.596 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQDsUOhmfCk4IBAAx12Gw2QudQ2nrPb8VlE0SA== 2024-09-16T15:38:53.596 INFO:teuthology.orchestra.run.smithi167.stdout:[osd.4] 2024-09-16T15:38:53.596 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQD/UOhmXI/nGRAA7t6XveKvrWgqy6rHuMQSZQ== 2024-09-16T15:38:53.596 INFO:teuthology.orchestra.run.smithi167.stdout:[osd.5] 2024-09-16T15:38:53.596 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQACUehmKM//IxAAsoFpiiTXNs//v3YXeUf/Iw== 2024-09-16T15:38:53.596 INFO:teuthology.orchestra.run.smithi167.stdout:[osd.6] 2024-09-16T15:38:53.596 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQAFUehmlo/FLRAAp5mvUSWCaBbm85QyTZY0eA== 2024-09-16T15:38:53.596 INFO:teuthology.orchestra.run.smithi167.stdout:[osd.7] 2024-09-16T15:38:53.596 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQAIUehmR32JNxAA27antOIKot1nScQbLZDf5g== 2024-09-16T15:38:53.596 INFO:teuthology.orchestra.run.smithi167.stdout:[client.0] 2024-09-16T15:38:53.596 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQDsUOhmf5PBKBAAKE4Y4zbA4h07IBNjPkyptw== 2024-09-16T15:38:53.596 INFO:teuthology.orchestra.run.smithi167.stdout:[client.2] 2024-09-16T15:38:53.596 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQDsUOhm6zUyLxAAkzeiIQTuNZkMM+Oit9rC5A== 2024-09-16T15:38:53.596 INFO:teuthology.orchestra.run.smithi167.stdout:[client.1] 2024-09-16T15:38:53.597 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQDsUOhm15emNRAAwhK7QmA2c9IYGn04ERvY0g== 2024-09-16T15:38:53.597 INFO:teuthology.orchestra.run.smithi167.stdout:[client.3] 2024-09-16T15:38:53.597 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQDtUOhmD82EABAAeDf82/hPxpPfAUk8oj5h/Q== 2024-09-16T15:38:53.600 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-16T15:38:53.603 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-16T15:38:53.732 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-16T15:38:53.737 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-16T15:38:53.834 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-16T15:38:53.838 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-16T15:38:53.935 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T15:38:53.939 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T15:38:54.033 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T15:38:54.037 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T15:38:54.132 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T15:38:54.136 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T15:38:54.230 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T15:38:54.234 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T15:38:54.327 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-16T15:38:54.331 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-16T15:38:54.426 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-16T15:38:54.430 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-16T15:38:54.527 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-16T15:38:54.531 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-16T15:38:54.629 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T15:38:54.633 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T15:38:54.727 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T15:38:54.731 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T15:38:54.828 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T15:38:54.832 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T15:38:54.926 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T15:38:54.929 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T15:38:55.024 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-16T15:38:55.028 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-16T15:38:55.125 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-16T15:38:55.129 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-16T15:38:55.226 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-16T15:38:55.230 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-16T15:38:55.325 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-16T15:38:55.329 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-16T15:38:55.427 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-16T15:38:55.427 DEBUG:teuthology.orchestra.run.smithi151:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-16T15:38:55.482 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-16T15:38:55.591 DEBUG:teuthology.orchestra.run.smithi151:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-16T15:38:55.647 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-16T15:38:55.703 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-16T15:38:55.813 DEBUG:teuthology.orchestra.run.smithi167:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-16T15:38:55.869 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-16T15:38:55.924 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-16T15:38:56.032 DEBUG:teuthology.orchestra.run.smithi167:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-16T15:38:56.088 DEBUG:teuthology.orchestra.run.smithi151:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-16T15:38:56.092 DEBUG:teuthology.orchestra.run.smithi167:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-16T15:38:56.126 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-16T15:38:56.126 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-16T15:38:56.127 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-16T15:38:56.169 INFO:tasks.ceph.mon.a:Started 2024-09-16T15:38:56.169 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-16T15:38:56.169 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-16T15:38:56.173 INFO:tasks.ceph.mon.b:Started 2024-09-16T15:38:56.173 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-16T15:38:56.173 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-16T15:38:56.177 INFO:tasks.ceph.mon.c:Started 2024-09-16T15:38:56.177 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-16T15:38:56.177 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-16T15:38:56.177 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-16T15:38:56.181 INFO:tasks.ceph.mgr.y:Started 2024-09-16T15:38:56.181 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-16T15:38:56.181 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-16T15:38:56.185 INFO:tasks.ceph.mgr.x:Started 2024-09-16T15:38:56.185 INFO:tasks.ceph:setting config [client] client_mount_timeout = 600 2024-09-16T15:38:56.185 DEBUG:teuthology.orchestra.run.smithi151:> sudo ceph --cluster ceph config set client client_mount_timeout 600 2024-09-16T15:38:56.189 INFO:tasks.ceph:setting config [client] debug_client = 20 2024-09-16T15:38:56.189 DEBUG:teuthology.orchestra.run.smithi151:> sudo ceph --cluster ceph config set client debug_client 20 2024-09-16T15:38:56.234 INFO:tasks.ceph:setting config [client] debug_ms = 1 2024-09-16T15:38:56.234 DEBUG:teuthology.orchestra.run.smithi151:> sudo ceph --cluster ceph config set client debug_ms 1 2024-09-16T15:38:56.281 INFO:tasks.ceph:setting config [client] rados_mon_op_timeout = 900 2024-09-16T15:38:56.281 DEBUG:teuthology.orchestra.run.smithi151:> sudo ceph --cluster ceph config set client rados_mon_op_timeout 900 2024-09-16T15:38:56.326 INFO:tasks.ceph:setting config [client] rados_osd_op_timeout = 900 2024-09-16T15:38:56.326 DEBUG:teuthology.orchestra.run.smithi151:> sudo ceph --cluster ceph config set client rados_osd_op_timeout 900 2024-09-16T15:38:56.371 INFO:tasks.ceph:setting config [mds] debug_mds = 20 2024-09-16T15:38:56.371 DEBUG:teuthology.orchestra.run.smithi151:> sudo ceph --cluster ceph config set mds debug_mds 20 2024-09-16T15:38:56.417 INFO:tasks.ceph:setting config [mds] debug_mds_balancer = 20 2024-09-16T15:38:56.417 DEBUG:teuthology.orchestra.run.smithi151:> sudo ceph --cluster ceph config set mds debug_mds_balancer 20 2024-09-16T15:38:56.464 INFO:tasks.ceph:setting config [mds] debug_ms = 2 2024-09-16T15:38:56.465 DEBUG:teuthology.orchestra.run.smithi151:> sudo ceph --cluster ceph config set mds debug_ms 2 2024-09-16T15:38:56.513 INFO:tasks.ceph:setting config [mds] mds_debug_frag = True 2024-09-16T15:38:56.513 DEBUG:teuthology.orchestra.run.smithi151:> sudo ceph --cluster ceph config set mds mds_debug_frag True 2024-09-16T15:38:56.521 INFO:tasks.ceph:setting config [mds] mds_debug_scatterstat = True 2024-09-16T15:38:56.521 DEBUG:teuthology.orchestra.run.smithi151:> sudo ceph --cluster ceph config set mds mds_debug_scatterstat True 2024-09-16T15:38:56.569 INFO:tasks.ceph:setting config [mds] mds_op_complaint_time = 180 2024-09-16T15:38:56.569 DEBUG:teuthology.orchestra.run.smithi151:> sudo ceph --cluster ceph config set mds mds_op_complaint_time 180 2024-09-16T15:38:56.621 INFO:tasks.ceph:setting config [mds] mds_verify_scatter = True 2024-09-16T15:38:56.621 DEBUG:teuthology.orchestra.run.smithi151:> sudo ceph --cluster ceph config set mds mds_verify_scatter True 2024-09-16T15:38:56.670 INFO:tasks.ceph:setting config [mds] osd_op_complaint_time = 180 2024-09-16T15:38:56.671 DEBUG:teuthology.orchestra.run.smithi151:> sudo ceph --cluster ceph config set mds osd_op_complaint_time 180 2024-09-16T15:38:56.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.709+0000 7fac14400640 1 -- 172.21.15.151:0/414353472 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac0c14b790 msgr2=0x7fac0c15f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:56.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.709+0000 7fac14400640 1 --2- 172.21.15.151:0/414353472 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac0c14b790 0x7fac0c15f270 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fac0400c680 tx=0x7fac0400cb40 comp rx=0 tx=0).stop 2024-09-16T15:38:56.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.709+0000 7fac14400640 1 -- 172.21.15.151:0/414353472 shutdown_connections 2024-09-16T15:38:56.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.709+0000 7fac14400640 1 --2- 172.21.15.151:0/414353472 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac0c14b790 0x7fac0c15f270 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:56.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.709+0000 7fac14400640 1 --2- 172.21.15.151:0/414353472 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac0c0d22a0 0x7fac0c14b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:56.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.709+0000 7fac14400640 1 --2- 172.21.15.151:0/414353472 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac0c0cfaf0 0x7fac0c1fce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:56.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.709+0000 7fac14400640 1 -- 172.21.15.151:0/414353472 >> 172.21.15.151:0/414353472 conn(0x7fac0c0d3fb0 msgr2=0x7fac0c0b7770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:56.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.710+0000 7fac14400640 1 -- 172.21.15.151:0/414353472 shutdown_connections 2024-09-16T15:38:56.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.710+0000 7fac14400640 1 -- 172.21.15.151:0/414353472 wait complete. 2024-09-16T15:38:56.718 INFO:tasks.ceph:setting config [mds] rados_mon_op_timeout = 900 2024-09-16T15:38:56.718 DEBUG:teuthology.orchestra.run.smithi151:> sudo ceph --cluster ceph config set mds rados_mon_op_timeout 900 2024-09-16T15:38:56.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.724+0000 7f7407e00640 1 -- 172.21.15.151:0/1880537754 <== mon.1 v2:172.21.15.167:3300/0 9 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7f7400010840 con 0x7f74100b5630 2024-09-16T15:38:56.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.724+0000 7f7407e00640 1 -- 172.21.15.151:0/1880537754 <== mon.1 v2:172.21.15.167:3300/0 10 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v5)=0 v5) ==== 123+0+0 (secure 0 0 0) 0x7f74101569e0 con 0x7f74100b5630 2024-09-16T15:38:56.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.723+0000 7efd33400640 1 -- 172.21.15.151:0/800847774 <== mon.1 v2:172.21.15.167:3300/0 8 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7efd2c0177c0 con 0x7efd3c0b2b40 2024-09-16T15:38:56.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.727+0000 7f7416c00640 1 -- 172.21.15.151:0/1880537754 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74100b5630 msgr2=0x7f7410073720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:56.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.727+0000 7f7416c00640 1 --2- 172.21.15.151:0/1880537754 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74100b5630 0x7f7410073720 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f740000c540 tx=0x7f740000ca00 comp rx=0 tx=0).stop 2024-09-16T15:38:56.750 INFO:tasks.ceph:setting config [mds] rados_osd_op_timeout = 900 2024-09-16T15:38:56.750 DEBUG:teuthology.orchestra.run.smithi151:> sudo ceph --cluster ceph config set mds rados_osd_op_timeout 900 2024-09-16T15:38:56.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.731+0000 7f7416c00640 1 -- 172.21.15.151:0/1880537754 shutdown_connections 2024-09-16T15:38:56.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.731+0000 7f7416c00640 1 --2- 172.21.15.151:0/1880537754 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74100d22a0 0x7f741015a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:56.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.731+0000 7f7416c00640 1 --2- 172.21.15.151:0/1880537754 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f74100cfaf0 0x7f741015a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:56.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.731+0000 7f7416c00640 1 --2- 172.21.15.151:0/1880537754 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74100b5630 0x7f7410073720 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:56.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.731+0000 7f7416c00640 1 -- 172.21.15.151:0/1880537754 >> 172.21.15.151:0/1880537754 conn(0x7f74100d3fb0 msgr2=0x7f74100d4390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:56.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.733+0000 7f7416c00640 1 -- 172.21.15.151:0/1880537754 shutdown_connections 2024-09-16T15:38:56.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.733+0000 7f7416c00640 1 -- 172.21.15.151:0/1880537754 wait complete. 2024-09-16T15:38:56.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.728+0000 7f7fc5600640 1 -- 172.21.15.151:0/3310943945 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f98005290 con 0x7f7fc00b5270 2024-09-16T15:38:56.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.734+0000 7f7fb7e00640 1 -- 172.21.15.151:0/3310943945 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7fac018020 con 0x7f7fc00b5270 2024-09-16T15:38:56.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.734+0000 7f7fb7e00640 1 -- 172.21.15.151:0/3310943945 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7fac00f3e0 con 0x7f7fc00b5270 2024-09-16T15:38:56.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.740+0000 7f7fb7e00640 1 -- 172.21.15.151:0/3310943945 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f7fac00f700 con 0x7f7fc00b5270 2024-09-16T15:38:56.780 INFO:tasks.ceph:setting config [mon] mon_op_complaint_time = 120 2024-09-16T15:38:56.780 DEBUG:teuthology.orchestra.run.smithi151:> sudo ceph --cluster ceph config set mon mon_op_complaint_time 120 2024-09-16T15:38:56.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.780+0000 7ff3ac400640 1 -- 172.21.15.151:0/672937681 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3a40d0cc0 msgr2=0x7ff3a40d10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:56.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.780+0000 7ff3ac400640 1 --2- 172.21.15.151:0/672937681 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3a40d0cc0 0x7ff3a40d10a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4069a30 tx=0x7ff39002aa30 comp rx=0 tx=0).stop 2024-09-16T15:38:56.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.780+0000 7ff3ac400640 1 -- 172.21.15.151:0/672937681 shutdown_connections 2024-09-16T15:38:56.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.780+0000 7ff3ac400640 1 --2- 172.21.15.151:0/672937681 >> v2:172.21.15.151:3300/0 conn(0x7ff3a40b4e90 0x7ff3a40d2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:56.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.780+0000 7ff3ac400640 1 --2- 172.21.15.151:0/672937681 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff3a40d4bd0 0x7ff3a40d1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:56.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.780+0000 7ff3ac400640 1 --2- 172.21.15.151:0/672937681 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3a40d0cc0 0x7ff3a40d10a0 secure :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4069a30 tx=0x7ff39002aa30 comp rx=0 tx=0).stop 2024-09-16T15:38:56.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.780+0000 7ff3ac400640 1 -- 172.21.15.151:0/672937681 >> 172.21.15.151:0/672937681 conn(0x7ff3a40d4040 msgr2=0x7ff3a40d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:56.792 INFO:tasks.ceph:setting config [mon] mon_warn_on_pool_no_app = False 2024-09-16T15:38:56.792 DEBUG:teuthology.orchestra.run.smithi151:> sudo ceph --cluster ceph config set mon mon_warn_on_pool_no_app False 2024-09-16T15:38:56.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.792+0000 7ff3ac400640 1 -- 172.21.15.151:0/672937681 shutdown_connections 2024-09-16T15:38:56.811 INFO:tasks.ceph:setting config [osd] osd_op_complaint_time = 180 2024-09-16T15:38:56.811 DEBUG:teuthology.orchestra.run.smithi151:> sudo ceph --cluster ceph config set osd osd_op_complaint_time 180 2024-09-16T15:38:56.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.806+0000 7efd42200640 1 -- 172.21.15.151:0/800847774 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7efd10005310 con 0x7efd3c0b2b40 2024-09-16T15:38:56.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.794+0000 7ff3ac400640 1 -- 172.21.15.151:0/672937681 wait complete. 2024-09-16T15:38:56.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.796+0000 7ff3ac400640 1 Processor -- start 2024-09-16T15:38:56.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.809+0000 7ff3ac400640 1 -- start start 2024-09-16T15:38:56.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.811+0000 7ff3ac400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff3a40b4e90 0x7ff3a415d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:56.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.811+0000 7ff3ac400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff3a40d4bd0 0x7ff3a4156390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:56.827 DEBUG:tasks.ceph:set 18 configs 2024-09-16T15:38:56.827 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi151') 2024-09-16T15:38:56.828 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi151') 2024-09-16T15:38:56.828 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi151') 2024-09-16T15:38:56.828 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi151') 2024-09-16T15:38:56.828 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi151') 2024-09-16T15:38:56.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.818+0000 7efd33400640 1 -- 172.21.15.151:0/800847774 <== mon.1 v2:172.21.15.167:3300/0 9 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efd2c017a90 con 0x7efd3c0b2b40 2024-09-16T15:38:56.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.819+0000 7efd33400640 1 -- 172.21.15.151:0/800847774 <== mon.1 v2:172.21.15.167:3300/0 10 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v6)=0 v6) ==== 123+0+0 (secure 0 0 0) 0x7efd3c145e20 con 0x7efd3c0b2b40 2024-09-16T15:38:56.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.817+0000 7f7fb7e00640 1 -- 172.21.15.151:0/3310943945 <== mon.0 v2:172.21.15.151:3300/0 7 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7fac00f8e0 con 0x7f7fc00b5270 2024-09-16T15:38:56.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.811+0000 7ff3ac400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3a40c92a0 0x7ff3a41568d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:56.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.811+0000 7ff3ac400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a40d2c30 con 0x7ff3a40d4bd0 2024-09-16T15:38:56.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.811+0000 7ff3ac400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a40d2ab0 con 0x7ff3a40c92a0 2024-09-16T15:38:56.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.811+0000 7ff3ac400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7ff3a41621a0 con 0x7ff3a40b4e90 2024-09-16T15:38:56.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.811+0000 7ff3a9a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff3a40b4e90 0x7ff3a415d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:56.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.811+0000 7ff3a9a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff3a40b4e90 0x7ff3a415d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.151:47892/0 (socket says 172.21.15.151:47892) 2024-09-16T15:38:56.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.811+0000 7ff3a9a00640 1 -- 172.21.15.151:0/958452665 learned_addr learned my addr 172.21.15.151:0/958452665 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:38:56.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.812+0000 7ff3a9a00640 1 -- 172.21.15.151:0/958452665 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3a40c92a0 msgr2=0x7ff3a41568d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:38:56.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.812+0000 7ff3a9a00640 1 --2- 172.21.15.151:0/958452665 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3a40c92a0 0x7ff3a41568d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:56.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.812+0000 7ff3a9a00640 1 -- 172.21.15.151:0/958452665 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff3a40d4bd0 msgr2=0x7ff3a4156390 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:38:56.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.812+0000 7ff3a9a00640 1 --2- 172.21.15.151:0/958452665 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff3a40d4bd0 0x7ff3a4156390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:56.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.812+0000 7ff3a9a00640 1 -- 172.21.15.151:0/958452665 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3a41fc770 con 0x7ff3a40b4e90 2024-09-16T15:38:56.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.812+0000 7ff3a9a00640 1 --2- 172.21.15.151:0/958452665 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff3a40b4e90 0x7ff3a415d240 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff39002afa0 tx=0x7ff390003ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:56.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.813+0000 7ff3a1600640 1 -- 172.21.15.151:0/958452665 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff390047070 con 0x7ff3a40b4e90 2024-09-16T15:38:56.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.813+0000 7ff3a1600640 1 -- 172.21.15.151:0/958452665 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7ff390004250 con 0x7ff3a40b4e90 2024-09-16T15:38:56.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.813+0000 7ff3a1600640 1 -- 172.21.15.151:0/958452665 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff390047070 con 0x7ff3a40b4e90 2024-09-16T15:38:56.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.819+0000 7ff3a1600640 1 -- 172.21.15.151:0/958452665 <== mon.2 v2:172.21.15.167:3301/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff390004440 con 0x7ff3a40b4e90 2024-09-16T15:38:56.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.824+0000 7efd42200640 1 -- 172.21.15.151:0/800847774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd3c0b2b40 msgr2=0x7efd3c1458e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:56.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.824+0000 7efd42200640 1 --2- 172.21.15.151:0/800847774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd3c0b2b40 0x7efd3c1458e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7efd2c00b590 tx=0x7efd2c00ba50 comp rx=0 tx=0).stop 2024-09-16T15:38:56.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.825+0000 7efd42200640 1 -- 172.21.15.151:0/800847774 shutdown_connections 2024-09-16T15:38:56.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.825+0000 7efd42200640 1 --2- 172.21.15.151:0/800847774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd3c0b2b40 0x7efd3c1458e0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:56.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.825+0000 7efd42200640 1 --2- 172.21.15.151:0/800847774 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efd3c0af170 0x7efd3c14c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:56.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.825+0000 7efd42200640 1 --2- 172.21.15.151:0/800847774 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efd3c06fba0 0x7efd3c1501a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:56.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.825+0000 7efd42200640 1 -- 172.21.15.151:0/800847774 >> 172.21.15.151:0/800847774 conn(0x7efd3c0b1840 msgr2=0x7efd3c14d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:56.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.826+0000 7efd42200640 1 -- 172.21.15.151:0/800847774 shutdown_connections 2024-09-16T15:38:56.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.826+0000 7efd42200640 1 -- 172.21.15.151:0/800847774 wait complete. 2024-09-16T15:38:56.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.829+0000 7ff3ac400640 1 -- 172.21.15.151:0/958452665 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3a41fca10 con 0x7ff3a40b4e90 2024-09-16T15:38:56.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.829+0000 7ff3ac400640 1 -- 172.21.15.151:0/958452665 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3a41fcfd0 con 0x7ff3a40b4e90 2024-09-16T15:38:56.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.832+0000 7ff3ac400640 1 -- 172.21.15.151:0/958452665 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3a40a2030 con 0x7ff3a40b4e90 2024-09-16T15:38:56.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.839+0000 7ff3a1600640 1 -- 172.21.15.151:0/958452665 <== mon.2 v2:172.21.15.167:3301/0 5 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff390036070 con 0x7ff3a40b4e90 2024-09-16T15:38:56.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.839+0000 7ff3a1600640 1 -- 172.21.15.151:0/958452665 <== mon.2 v2:172.21.15.167:3301/0 6 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff390030020 con 0x7ff3a40b4e90 2024-09-16T15:38:56.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.843+0000 7ff3a1600640 1 -- 172.21.15.151:0/958452665 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7ff3a40a2030 con 0x7ff3a40b4e90 2024-09-16T15:38:56.869 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi151') 2024-09-16T15:38:56.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.878+0000 7f54f7e00640 1 -- 172.21.15.151:0/721315958 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f54dc003c30 msgr2=0x7f54dc004030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:56.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.878+0000 7f54f7e00640 1 --2- 172.21.15.151:0/721315958 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f54dc003c30 0x7f54dc004030 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f54e4009bf0 tx=0x7f54dc005830 comp rx=0 tx=0).stop 2024-09-16T15:38:56.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.879+0000 7f54f7e00640 1 -- 172.21.15.151:0/721315958 shutdown_connections 2024-09-16T15:38:56.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.879+0000 7f54f7e00640 1 --2- 172.21.15.151:0/721315958 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f54dc003c30 0x7f54dc004030 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:56.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.879+0000 7f54f7e00640 1 -- 172.21.15.151:0/721315958 >> 172.21.15.151:0/721315958 conn(0x7f54f80cf9f0 msgr2=0x7f54f80cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:56.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.879+0000 7f54f7e00640 1 -- 172.21.15.151:0/721315958 shutdown_connections 2024-09-16T15:38:56.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.879+0000 7f54f7e00640 1 -- 172.21.15.151:0/721315958 wait complete. 2024-09-16T15:38:56.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.887+0000 7f54f7e00640 1 Processor -- start 2024-09-16T15:38:56.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.887+0000 7f54f7e00640 1 -- start start 2024-09-16T15:38:56.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.889+0000 7f54f7e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f54dc003c30 0x7f54f80c5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:56.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.893+0000 7f54f7e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f54f80c4ff0 0x7f54f8072d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:56.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.893+0000 7f54f6a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f54dc003c30 0x7f54f80c5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:56.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.893+0000 7f54f6a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f54dc003c30 0x7f54f80c5c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.151:44970/0 (socket says 172.21.15.151:44970) 2024-09-16T15:38:56.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.893+0000 7f54f6a00640 1 -- 172.21.15.151:0/104284297 learned_addr learned my addr 172.21.15.151:0/104284297 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:38:56.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.894+0000 7f54f7e00640 1 --2- 172.21.15.151:0/104284297 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f54f8156690 0x7f54f8156b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:56.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.894+0000 7f54f6000640 1 --2- 172.21.15.151:0/104284297 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f54f80c4ff0 0x7f54f8072d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:56.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.894+0000 7f54f7e00640 1 -- 172.21.15.151:0/104284297 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f54f80ca040 con 0x7f54f8156690 2024-09-16T15:38:56.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.894+0000 7f54f7e00640 1 -- 172.21.15.151:0/104284297 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f54f80d0770 con 0x7f54dc003c30 2024-09-16T15:38:56.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.894+0000 7f54f7e00640 1 -- 172.21.15.151:0/104284297 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f54f80ca1c0 con 0x7f54f80c4ff0 2024-09-16T15:38:56.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.899+0000 7f54f7400640 1 --2- 172.21.15.151:0/104284297 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f54f8156690 0x7f54f8156b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:56.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.901+0000 7f54f7400640 1 -- 172.21.15.151:0/104284297 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f54f80c4ff0 msgr2=0x7f54f8072d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:56.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.901+0000 7f54f7400640 1 --2- 172.21.15.151:0/104284297 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f54f80c4ff0 0x7f54f8072d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:56.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.901+0000 7f54f7400640 1 -- 172.21.15.151:0/104284297 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f54dc003c30 msgr2=0x7f54f80c5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:56.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.901+0000 7f54f7400640 1 --2- 172.21.15.151:0/104284297 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f54dc003c30 0x7f54f80c5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:56.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.901+0000 7f54f7400640 1 -- 172.21.15.151:0/104284297 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54f81f0990 con 0x7f54f8156690 2024-09-16T15:38:56.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.902+0000 7f54f7400640 1 --2- 172.21.15.151:0/104284297 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f54f8156690 0x7f54f8156b10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f54f815e070 tx=0x7f54f000dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:56.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.903+0000 7f54eea00640 1 -- 172.21.15.151:0/104284297 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f0018070 con 0x7f54f8156690 2024-09-16T15:38:56.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.903+0000 7f54eea00640 1 -- 172.21.15.151:0/104284297 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f54f0003b80 con 0x7f54f8156690 2024-09-16T15:38:56.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.903+0000 7f54f7e00640 1 -- 172.21.15.151:0/104284297 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54f81f1710 con 0x7f54f8156690 2024-09-16T15:38:56.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.903+0000 7f54f7e00640 1 -- 172.21.15.151:0/104284297 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54f81f0c30 con 0x7f54f8156690 2024-09-16T15:38:56.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.905+0000 7f54f7e00640 1 -- 172.21.15.151:0/104284297 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54f80ca6b0 con 0x7f54f8156690 2024-09-16T15:38:56.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.907+0000 7f54eea00640 1 -- 172.21.15.151:0/104284297 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f00050d0 con 0x7f54f8156690 2024-09-16T15:38:56.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.911+0000 7f54eea00640 1 -- 172.21.15.151:0/104284297 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f54f001b070 con 0x7f54f8156690 2024-09-16T15:38:56.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.911+0000 7f54eea00640 1 -- 172.21.15.151:0/104284297 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f54f0013480 con 0x7f54f8156690 2024-09-16T15:38:56.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.916+0000 7f54eea00640 1 -- 172.21.15.151:0/104284297 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f54f0022020 con 0x7f54f8156690 2024-09-16T15:38:56.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.992+0000 7f0ff3200640 1 -- 172.21.15.151:0/943862117 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ff40cb840 msgr2=0x7f0ff40cbc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:56.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.992+0000 7f0ff3200640 1 --2- 172.21.15.151:0/943862117 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ff40cb840 0x7f0ff40cbc20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4071ef0 tx=0x7f0fe402a4d0 comp rx=0 tx=0).stop 2024-09-16T15:38:56.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.992+0000 7f0ff3200640 1 -- 172.21.15.151:0/943862117 shutdown_connections 2024-09-16T15:38:56.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.992+0000 7f0ff3200640 1 --2- 172.21.15.151:0/943862117 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ff40ca040 0x7f0ff40ca4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:56.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.992+0000 7f0ff3200640 1 --2- 172.21.15.151:0/943862117 >> v2:172.21.15.167:3300/0 conn(0x7f0ff406e5a0 0x7f0ff406e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:56.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.992+0000 7f0ff3200640 1 --2- 172.21.15.151:0/943862117 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ff40cb840 0x7f0ff40cbc20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:56.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.992+0000 7f0ff3200640 1 -- 172.21.15.151:0/943862117 >> 172.21.15.151:0/943862117 conn(0x7f0ff40cf9f0 msgr2=0x7f0ff40cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:56.996 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.993+0000 7f0ff3200640 1 -- 172.21.15.151:0/943862117 shutdown_connections 2024-09-16T15:38:56.996 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.993+0000 7f0ff3200640 1 -- 172.21.15.151:0/943862117 wait complete. 2024-09-16T15:38:57.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:56.995+0000 7f0ff3200640 1 Processor -- start 2024-09-16T15:38:57.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.002+0000 7f0ff3200640 1 -- start start 2024-09-16T15:38:57.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.004+0000 7f0ff3200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ff406e5a0 0x7f0ff4150810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:57.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.005+0000 7f7fc5600640 1 -- 172.21.15.151:0/3310943945 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) -- 0x7f7f98005580 con 0x7f7fc00b5270 2024-09-16T15:38:57.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.005+0000 7f0ff3200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ff40ca040 0x7f0ff40c6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:57.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.005+0000 7f0ff3200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ff40cb840 0x7f0ff41532b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:57.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.005+0000 7f0ff1400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ff40ca040 0x7f0ff40c6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:57.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.005+0000 7f0ff1400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ff40ca040 0x7f0ff40c6110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.151:45002/0 (socket says 172.21.15.151:45002) 2024-09-16T15:38:57.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.006+0000 7f0ff2800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ff40cb840 0x7f0ff41532b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:57.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.006+0000 7f0ff2800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ff40cb840 0x7f0ff41532b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43758/0 (socket says 172.21.15.151:43758) 2024-09-16T15:38:57.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.006+0000 7f0ff1400640 1 -- 172.21.15.151:0/3772388281 learned_addr learned my addr 172.21.15.151:0/3772388281 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:38:57.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.006+0000 7f0ff3200640 1 -- 172.21.15.151:0/3772388281 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff40cada0 con 0x7f0ff40cb840 2024-09-16T15:38:57.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.006+0000 7f0ff3200640 1 -- 172.21.15.151:0/3772388281 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff40cac20 con 0x7f0ff40ca040 2024-09-16T15:38:57.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.006+0000 7f0ff3200640 1 -- 172.21.15.151:0/3772388281 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f0ff406fd80 con 0x7f0ff406e5a0 2024-09-16T15:38:57.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.007+0000 7f0ff2800640 1 -- 172.21.15.151:0/3772388281 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ff406e5a0 msgr2=0x7f0ff4150810 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T15:38:57.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.007+0000 7f0ff2800640 1 --2- 172.21.15.151:0/3772388281 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ff406e5a0 0x7f0ff4150810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.007+0000 7f0ff2800640 1 -- 172.21.15.151:0/3772388281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ff40ca040 msgr2=0x7f0ff40c6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:57.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.007+0000 7f0ff2800640 1 --2- 172.21.15.151:0/3772388281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ff40ca040 0x7f0ff40c6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.007+0000 7f0ff2800640 1 -- 172.21.15.151:0/3772388281 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ff40c9df0 con 0x7f0ff40cb840 2024-09-16T15:38:57.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.009+0000 7f0ff1400640 1 --2- 172.21.15.151:0/3772388281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ff40ca040 0x7f0ff40c6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:38:57.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.011+0000 7f0ff2800640 1 --2- 172.21.15.151:0/3772388281 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ff40cb840 0x7f0ff41532b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0fec007ed0 tx=0x7f0fec00a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:57.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.012+0000 7f0fea000640 1 -- 172.21.15.151:0/3772388281 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fec003b70 con 0x7f0ff40cb840 2024-09-16T15:38:57.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.012+0000 7f0fea000640 1 -- 172.21.15.151:0/3772388281 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0fec003d10 con 0x7f0ff40cb840 2024-09-16T15:38:57.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.012+0000 7f0fea000640 1 -- 172.21.15.151:0/3772388281 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fec004280 con 0x7f0ff40cb840 2024-09-16T15:38:57.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.020+0000 7f7fb7e00640 1 -- 172.21.15.151:0/3310943945 <== mon.0 v2:172.21.15.151:3300/0 8 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v7)=0 v7) ==== 108+0+0 (secure 0 0 0) 0x7f7fac00fa90 con 0x7f7fc00b5270 2024-09-16T15:38:57.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.018+0000 7f0ff3200640 1 -- 172.21.15.151:0/3772388281 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ff40c6650 con 0x7f0ff40cb840 2024-09-16T15:38:57.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.018+0000 7f0ff3200640 1 -- 172.21.15.151:0/3772388281 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ff41510b0 con 0x7f0ff40cb840 2024-09-16T15:38:57.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.025+0000 7f7fc5600640 1 -- 172.21.15.151:0/3310943945 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7fc00b5270 msgr2=0x7f7fc01f8600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:57.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.026+0000 7f7fc5600640 1 --2- 172.21.15.151:0/3310943945 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7fc00b5270 0x7f7fc01f8600 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7fac001110 tx=0x7f7fac0015d0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.025+0000 7f0fdf400640 1 -- 172.21.15.151:0/3772388281 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fc8005290 con 0x7f0ff40cb840 2024-09-16T15:38:57.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.028+0000 7f7fc5600640 1 -- 172.21.15.151:0/3310943945 shutdown_connections 2024-09-16T15:38:57.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.028+0000 7f7fc5600640 1 --2- 172.21.15.151:0/3310943945 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7fc0161460 0x7f7fc01f3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.028+0000 7f7fc5600640 1 --2- 172.21.15.151:0/3310943945 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7fc00b5270 0x7f7fc01f8600 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.028+0000 7f7fc5600640 1 --2- 172.21.15.151:0/3310943945 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7fc00b4d40 0x7f7fc0158940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.028+0000 7f7fc5600640 1 -- 172.21.15.151:0/3310943945 >> 172.21.15.151:0/3310943945 conn(0x7f7fc009d0b0 msgr2=0x7f7fc006d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:57.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.028+0000 7f7fc5600640 1 -- 172.21.15.151:0/3310943945 shutdown_connections 2024-09-16T15:38:57.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.028+0000 7f7fc5600640 1 -- 172.21.15.151:0/3310943945 wait complete. 2024-09-16T15:38:57.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.034+0000 7f0fea000640 1 -- 172.21.15.151:0/3772388281 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0fec00e450 con 0x7f0ff40cb840 2024-09-16T15:38:57.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.035+0000 7f0fea000640 1 -- 172.21.15.151:0/3772388281 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0fec01f850 con 0x7f0ff40cb840 2024-09-16T15:38:57.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.035+0000 7f0fea000640 1 -- 172.21.15.151:0/3772388281 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f0fec01fc50 con 0x7f0ff40cb840 2024-09-16T15:38:57.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.037+0000 7f8b57c00640 1 -- 172.21.15.151:0/1384555904 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b480e4090 msgr2=0x7f8b480513f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:57.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.037+0000 7f8b57c00640 1 --2- 172.21.15.151:0/1384555904 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b480e4090 0x7f8b480513f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8b44005fd0 tx=0x7f8b4402a560 comp rx=0 tx=0).stop 2024-09-16T15:38:57.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.039+0000 7f8b57c00640 1 -- 172.21.15.151:0/1384555904 shutdown_connections 2024-09-16T15:38:57.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.039+0000 7f8b57c00640 1 --2- 172.21.15.151:0/1384555904 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b48052040 0x7f8b48036140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.039+0000 7f8b57c00640 1 --2- 172.21.15.151:0/1384555904 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b480e4090 0x7f8b480513f0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.039+0000 7f8b57c00640 1 -- 172.21.15.151:0/1384555904 >> 172.21.15.151:0/1384555904 conn(0x7f8b48056770 msgr2=0x7f8b48056b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:57.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.040+0000 7f8b57c00640 1 -- 172.21.15.151:0/1384555904 shutdown_connections 2024-09-16T15:38:57.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.040+0000 7f8b57c00640 1 -- 172.21.15.151:0/1384555904 wait complete. 2024-09-16T15:38:57.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.043+0000 7ff3ac400640 1 -- 172.21.15.151:0/958452665 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) -- 0x7ff3a40d10a0 con 0x7ff3a40b4e90 2024-09-16T15:38:57.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.054+0000 7ff3a1600640 1 -- 172.21.15.151:0/958452665 <== mon.2 v2:172.21.15.167:3301/0 8 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v8)=0 v8) ==== 117+0+0 (secure 0 0 0) 0x7ff390004630 con 0x7ff3a40b4e90 2024-09-16T15:38:57.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.063+0000 7ff3ac400640 1 -- 172.21.15.151:0/958452665 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff3a40b4e90 msgr2=0x7ff3a415d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:57.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.063+0000 7ff3ac400640 1 --2- 172.21.15.151:0/958452665 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff3a40b4e90 0x7ff3a415d240 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff39002afa0 tx=0x7ff390003ef0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.063+0000 7ff3ac400640 1 -- 172.21.15.151:0/958452665 shutdown_connections 2024-09-16T15:38:57.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.063+0000 7ff3ac400640 1 --2- 172.21.15.151:0/958452665 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3a40c92a0 0x7ff3a41568d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.063+0000 7ff3ac400640 1 --2- 172.21.15.151:0/958452665 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff3a40d4bd0 0x7ff3a4156390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.063+0000 7ff3ac400640 1 --2- 172.21.15.151:0/958452665 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff3a40b4e90 0x7ff3a415d240 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.063+0000 7ff3ac400640 1 -- 172.21.15.151:0/958452665 >> 172.21.15.151:0/958452665 conn(0x7ff3a40d4040 msgr2=0x7ff3a40b0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:57.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.064+0000 7ff3ac400640 1 -- 172.21.15.151:0/958452665 shutdown_connections 2024-09-16T15:38:57.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.064+0000 7ff3ac400640 1 -- 172.21.15.151:0/958452665 wait complete. 2024-09-16T15:38:57.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.069+0000 7ffa66600640 1 -- 172.21.15.151:0/2613676130 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa60074330 msgr2=0x7ffa600cb0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:57.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.069+0000 7ffa66600640 1 --2- 172.21.15.151:0/2613676130 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa60074330 0x7ffa600cb0b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ffa58005fd0 tx=0x7ffa5802a820 comp rx=0 tx=0).stop 2024-09-16T15:38:57.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.070+0000 7ffa66600640 1 -- 172.21.15.151:0/2613676130 shutdown_connections 2024-09-16T15:38:57.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.070+0000 7ffa66600640 1 --2- 172.21.15.151:0/2613676130 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa60074330 0x7ffa600cb0b0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.073 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.070+0000 7ffa66600640 1 --2- 172.21.15.151:0/2613676130 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffa6015e290 0x7ffa600cab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.073 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.070+0000 7ffa66600640 1 -- 172.21.15.151:0/2613676130 >> 172.21.15.151:0/2613676130 conn(0x7ffa600cf9f0 msgr2=0x7ffa600cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:57.074 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi151') 2024-09-16T15:38:57.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.072+0000 7ffa66600640 1 -- 172.21.15.151:0/2613676130 shutdown_connections 2024-09-16T15:38:57.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.072+0000 7ffa66600640 1 -- 172.21.15.151:0/2613676130 wait complete. 2024-09-16T15:38:57.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.074+0000 7f8b57c00640 1 Processor -- start 2024-09-16T15:38:57.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.074+0000 7f8b57c00640 1 -- start start 2024-09-16T15:38:57.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.080+0000 7ffa66600640 1 Processor -- start 2024-09-16T15:38:57.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.080+0000 7f8b57c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b48035630 0x7f8b48181c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:57.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.080+0000 7f8b57c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b48052040 0x7f8b4804c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:57.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.080+0000 7f8b57c00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b480e4090 0x7f8b4803bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:57.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.081+0000 7f8b57c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8b48036a10 con 0x7f8b48035630 2024-09-16T15:38:57.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.081+0000 7f8b55c00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b480e4090 0x7f8b4803bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:57.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.081+0000 7f8b55c00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b480e4090 0x7f8b4803bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.151:47954/0 (socket says 172.21.15.151:47954) 2024-09-16T15:38:57.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.081+0000 7f8b55c00640 1 -- 172.21.15.151:0/1198397358 learned_addr learned my addr 172.21.15.151:0/1198397358 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:38:57.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.083+0000 7f8b55200640 1 --2- 172.21.15.151:0/1198397358 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b48035630 0x7f8b48181c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:57.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.083+0000 7f8b4fe00640 1 --2- 172.21.15.151:0/1198397358 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b48052040 0x7f8b4804c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:57.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.080+0000 7ffa66600640 1 -- start start 2024-09-16T15:38:57.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.081+0000 7f8b57c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8b48036890 con 0x7f8b48052040 2024-09-16T15:38:57.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.085+0000 7f8b57c00640 1 -- 172.21.15.151:0/1198397358 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f8b48036b90 con 0x7f8b480e4090 2024-09-16T15:38:57.090 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.087+0000 7ffa66600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa60074330 0x7ffa60201300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:57.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.088+0000 7f8b55c00640 1 -- 172.21.15.151:0/1198397358 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b48052040 msgr2=0x7f8b4804c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:57.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.088+0000 7f8b55c00640 1 --2- 172.21.15.151:0/1198397358 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b48052040 0x7f8b4804c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.088+0000 7f8b55c00640 1 -- 172.21.15.151:0/1198397358 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b48035630 msgr2=0x7f8b48181c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:57.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.090+0000 7f8b55c00640 1 --2- 172.21.15.151:0/1198397358 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b48035630 0x7f8b48181c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.090+0000 7f8b55c00640 1 -- 172.21.15.151:0/1198397358 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b4804b7d0 con 0x7f8b480e4090 2024-09-16T15:38:57.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.090+0000 7f8b55200640 1 --2- 172.21.15.151:0/1198397358 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b48035630 0x7f8b48181c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:38:57.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.091+0000 7ffa66600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa6015e290 0x7ffa600b4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:57.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.092+0000 7ffa66600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffa6015e670 0x7ffa600c9500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:57.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.092+0000 7f8b55c00640 1 --2- 172.21.15.151:0/1198397358 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b480e4090 0x7f8b4803bf80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8b500527e0 tx=0x7f8b5006a370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:57.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.092+0000 7ffa5fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa6015e290 0x7ffa600b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:57.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.092+0000 7ffa5fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa6015e290 0x7ffa600b4ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.151:45010/0 (socket says 172.21.15.151:45010) 2024-09-16T15:38:57.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.094+0000 7ffa5fe00640 1 -- 172.21.15.151:0/2266242900 learned_addr learned my addr 172.21.15.151:0/2266242900 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:38:57.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.094+0000 7ffa66600640 1 -- 172.21.15.151:0/2266242900 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ffa600aeb20 con 0x7ffa60074330 2024-09-16T15:38:57.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.095+0000 7ffa65200640 1 --2- 172.21.15.151:0/2266242900 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa60074330 0x7ffa60201300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:57.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.095+0000 7f8b4cc00640 1 -- 172.21.15.151:0/1198397358 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b5008f070 con 0x7f8b480e4090 2024-09-16T15:38:57.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.095+0000 7ffa65c00640 1 --2- 172.21.15.151:0/2266242900 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffa6015e670 0x7ffa600c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:57.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.096+0000 7ffa66600640 1 -- 172.21.15.151:0/2266242900 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ffa600ae9a0 con 0x7ffa6015e290 2024-09-16T15:38:57.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.096+0000 7ffa66600640 1 -- 172.21.15.151:0/2266242900 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7ffa600aeca0 con 0x7ffa6015e670 2024-09-16T15:38:57.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.096+0000 7f8b57c00640 1 -- 172.21.15.151:0/1198397358 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b4804bab0 con 0x7f8b480e4090 2024-09-16T15:38:57.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.096+0000 7f8b57c00640 1 -- 172.21.15.151:0/1198397358 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b4804c070 con 0x7f8b480e4090 2024-09-16T15:38:57.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.101+0000 7f8b57c00640 1 -- 172.21.15.151:0/1198397358 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b48037180 con 0x7f8b480e4090 2024-09-16T15:38:57.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.100+0000 7ffa65c00640 1 -- 172.21.15.151:0/2266242900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa6015e290 msgr2=0x7ffa600b4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:57.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.100+0000 7ffa65c00640 1 --2- 172.21.15.151:0/2266242900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa6015e290 0x7ffa600b4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.100+0000 7ffa65c00640 1 -- 172.21.15.151:0/2266242900 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa60074330 msgr2=0x7ffa60201300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:57.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.100+0000 7ffa65c00640 1 --2- 172.21.15.151:0/2266242900 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa60074330 0x7ffa60201300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.100+0000 7ffa65c00640 1 -- 172.21.15.151:0/2266242900 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa600b5220 con 0x7ffa6015e670 2024-09-16T15:38:57.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.100+0000 7ffa65c00640 1 --2- 172.21.15.151:0/2266242900 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffa6015e670 0x7ffa600c9500 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ffa58009000 tx=0x7ffa58033ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:57.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.101+0000 7ffa5cc00640 1 -- 172.21.15.151:0/2266242900 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa58037050 con 0x7ffa6015e670 2024-09-16T15:38:57.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.101+0000 7ffa5cc00640 1 -- 172.21.15.151:0/2266242900 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffa58043070 con 0x7ffa6015e670 2024-09-16T15:38:57.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.101+0000 7ffa5cc00640 1 -- 172.21.15.151:0/2266242900 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa580044b0 con 0x7ffa6015e670 2024-09-16T15:38:57.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.101+0000 7f2d61600640 1 -- 172.21.15.151:0/90380086 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d54006550 con 0x7f2d48004800 2024-09-16T15:38:57.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.106+0000 7f2d63e00640 1 -- 172.21.15.151:0/90380086 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d48004800 msgr2=0x7f2d48004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:57.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.106+0000 7f2d63e00640 1 --2- 172.21.15.151:0/90380086 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d48004800 0x7f2d48004c80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2d5400c310 tx=0x7f2d48005830 comp rx=0 tx=0).stop 2024-09-16T15:38:57.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.102+0000 7ffa66600640 1 -- 172.21.15.151:0/2266242900 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa600b3c40 con 0x7ffa6015e670 2024-09-16T15:38:57.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.102+0000 7ffa66600640 1 -- 172.21.15.151:0/2266242900 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa600b4200 con 0x7ffa6015e670 2024-09-16T15:38:57.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.102+0000 7ffa65200640 1 --2- 172.21.15.151:0/2266242900 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa60074330 0x7ffa60201300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:38:57.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.106+0000 7f8b4cc00640 1 -- 172.21.15.151:0/1198397358 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8b5006de50 con 0x7f8b480e4090 2024-09-16T15:38:57.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.106+0000 7f8b4cc00640 1 -- 172.21.15.151:0/1198397358 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b5008f070 con 0x7f8b480e4090 2024-09-16T15:38:57.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.107+0000 7f8b4cc00640 1 -- 172.21.15.151:0/1198397358 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8b50072630 con 0x7f8b480e4090 2024-09-16T15:38:57.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.107+0000 7f8b4cc00640 1 -- 172.21.15.151:0/1198397358 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8b50064020 con 0x7f8b480e4090 2024-09-16T15:38:57.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.108+0000 7ffa66600640 1 -- 172.21.15.151:0/2266242900 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa6015f5a0 con 0x7ffa6015e670 2024-09-16T15:38:57.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.111+0000 7f8b4cc00640 1 -- 172.21.15.151:0/1198397358 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f8b50063d00 con 0x7f8b480e4090 2024-09-16T15:38:57.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.112+0000 7f2d63400640 1 -- 172.21.15.151:0/90380086 reap_dead start 2024-09-16T15:38:57.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.113+0000 7f2d63400640 1 -- 172.21.15.151:0/90380086 reap_dead start 2024-09-16T15:38:57.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.114+0000 7f2d63e00640 1 -- 172.21.15.151:0/90380086 shutdown_connections 2024-09-16T15:38:57.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.114+0000 7f2d63e00640 1 -- 172.21.15.151:0/90380086 >> 172.21.15.151:0/90380086 conn(0x7f2d640cf9f0 msgr2=0x7f2d640cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:57.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.116+0000 7f2d63e00640 1 -- 172.21.15.151:0/90380086 shutdown_connections 2024-09-16T15:38:57.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.116+0000 7f2d63e00640 1 -- 172.21.15.151:0/90380086 wait complete. 2024-09-16T15:38:57.121 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi151') 2024-09-16T15:38:57.122 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.117+0000 7ffa5cc00640 1 -- 172.21.15.151:0/2266242900 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ffa58004650 con 0x7ffa6015e670 2024-09-16T15:38:57.122 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.117+0000 7ffa5cc00640 1 -- 172.21.15.151:0/2266242900 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ffa580325e0 con 0x7ffa6015e670 2024-09-16T15:38:57.122 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.119+0000 7ffa5cc00640 1 -- 172.21.15.151:0/2266242900 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7ffa5804c020 con 0x7ffa6015e670 2024-09-16T15:38:57.122 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.120+0000 7f2d63e00640 1 Processor -- start 2024-09-16T15:38:57.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.120+0000 7f2d63e00640 1 -- start start 2024-09-16T15:38:57.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.122+0000 7f2d63e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d6415b640 0x7f2d640c4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:57.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.122+0000 7f2d63e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d64158930 0x7f2d640c54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:57.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.122+0000 7f2d63e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d6415ba20 0x7f2d641fc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:57.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.122+0000 7f2d63e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2d64162950 con 0x7f2d64158930 2024-09-16T15:38:57.125 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.122+0000 7f2d63e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2d641627d0 con 0x7f2d6415ba20 2024-09-16T15:38:57.125 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.122+0000 7f2d63e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f2d640cb0d0 con 0x7f2d6415b640 2024-09-16T15:38:57.125 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.123+0000 7f2d62a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d6415b640 0x7f2d640c4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:57.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.123+0000 7f2d62000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d64158930 0x7f2d640c54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:57.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.123+0000 7f2d62000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d64158930 0x7f2d640c54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43806/0 (socket says 172.21.15.151:43806) 2024-09-16T15:38:57.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.123+0000 7f2d62000640 1 -- 172.21.15.151:0/2532260370 learned_addr learned my addr 172.21.15.151:0/2532260370 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:38:57.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.124+0000 7f2d62000640 1 -- 172.21.15.151:0/2532260370 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d6415b640 msgr2=0x7f2d640c4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:57.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.123+0000 7f2d63400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d6415ba20 0x7f2d641fc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:57.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.124+0000 7f2d62000640 1 --2- 172.21.15.151:0/2532260370 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d6415b640 0x7f2d640c4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.130 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.124+0000 7f2d62000640 1 -- 172.21.15.151:0/2532260370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d6415ba20 msgr2=0x7f2d641fc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:57.130 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.124+0000 7f2d62000640 1 --2- 172.21.15.151:0/2532260370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d6415ba20 0x7f2d641fc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.130 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.124+0000 7f2d62000640 1 -- 172.21.15.151:0/2532260370 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d640c5a00 con 0x7f2d64158930 2024-09-16T15:38:57.130 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.125+0000 7f2d63400640 1 --2- 172.21.15.151:0/2532260370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d6415ba20 0x7f2d641fc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:38:57.130 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.125+0000 7f2d62a00640 1 --2- 172.21.15.151:0/2532260370 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d6415b640 0x7f2d640c4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:38:57.130 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.125+0000 7f2d62000640 1 --2- 172.21.15.151:0/2532260370 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d64158930 0x7f2d640c54c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2d5400c430 tx=0x7f2d54001780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:57.130 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.126+0000 7f2d5b400640 1 -- 172.21.15.151:0/2532260370 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d540038d0 con 0x7f2d64158930 2024-09-16T15:38:57.130 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.126+0000 7f2d5b400640 1 -- 172.21.15.151:0/2532260370 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2d54003b60 con 0x7f2d64158930 2024-09-16T15:38:57.130 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.126+0000 7f2d5b400640 1 -- 172.21.15.151:0/2532260370 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d5402b590 con 0x7f2d64158930 2024-09-16T15:38:57.130 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.127+0000 7f2d63e00640 1 -- 172.21.15.151:0/2532260370 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d640c7f20 con 0x7f2d64158930 2024-09-16T15:38:57.131 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.128+0000 7f2d63e00640 1 -- 172.21.15.151:0/2532260370 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d640c81a0 con 0x7f2d64158930 2024-09-16T15:38:57.131 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.128+0000 7f2d5b400640 1 -- 172.21.15.151:0/2532260370 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2d5402bac0 con 0x7f2d64158930 2024-09-16T15:38:57.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.128+0000 7f2d5b400640 1 -- 172.21.15.151:0/2532260370 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2d540407c0 con 0x7f2d64158930 2024-09-16T15:38:57.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.136+0000 7f2d63e00640 1 -- 172.21.15.151:0/2532260370 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d64070090 con 0x7f2d64158930 2024-09-16T15:38:57.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.142+0000 7f2d5b400640 1 -- 172.21.15.151:0/2532260370 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f2d640c7f20 con 0x7f2d64158930 2024-09-16T15:38:57.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.145+0000 7fac27200640 1 -- 172.21.15.151:0/3429576133 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac1c0e5ea0 msgr2=0x7fac1c006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:57.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.145+0000 7fac24a00640 1 -- 172.21.15.151:0/3429576133 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac100016d0 con 0x7fac1c0e5ea0 2024-09-16T15:38:57.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.145+0000 7fac27200640 1 --2- 172.21.15.151:0/3429576133 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac1c0e5ea0 0x7fac1c006e50 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fac1c035e00 tx=0x7fac1402a4a0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.146+0000 7fac27200640 1 -- 172.21.15.151:0/3429576133 shutdown_connections 2024-09-16T15:38:57.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.146+0000 7fac27200640 1 --2- 172.21.15.151:0/3429576133 >> v2:172.21.15.151:3300/0 conn(0x7fac1c0e6e50 0x7fac1c036110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.146+0000 7fac27200640 1 --2- 172.21.15.151:0/3429576133 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac1c0e5ea0 0x7fac1c006e50 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.146+0000 7fac27200640 1 -- 172.21.15.151:0/3429576133 >> 172.21.15.151:0/3429576133 conn(0x7fac1c01d770 msgr2=0x7fac1c054560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:57.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.152+0000 7fac27200640 1 -- 172.21.15.151:0/3429576133 shutdown_connections 2024-09-16T15:38:57.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.152+0000 7fac27200640 1 -- 172.21.15.151:0/3429576133 wait complete. 2024-09-16T15:38:57.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.154+0000 7fac27200640 1 Processor -- start 2024-09-16T15:38:57.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.154+0000 7fac27200640 1 -- start start 2024-09-16T15:38:57.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.156+0000 7fac27200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac1c0e5ea0 0x7fac1c17d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:57.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.156+0000 7fac27200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac1c0e6280 0x7fac1c179f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:57.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.160+0000 7fac25e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac1c0e5ea0 0x7fac1c17d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:57.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.160+0000 7fac25e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac1c0e5ea0 0x7fac1c17d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.151:45034/0 (socket says 172.21.15.151:45034) 2024-09-16T15:38:57.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.160+0000 7fac25e00640 1 -- 172.21.15.151:0/1893246683 learned_addr learned my addr 172.21.15.151:0/1893246683 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:38:57.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.160+0000 7fac27200640 1 --2- 172.21.15.151:0/1893246683 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac1c0e6e50 0x7fac1c0d37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:57.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.160+0000 7fac27200640 1 -- 172.21.15.151:0/1893246683 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fac1c055290 con 0x7fac1c0e6280 2024-09-16T15:38:57.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.160+0000 7fac27200640 1 -- 172.21.15.151:0/1893246683 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fac1c036650 con 0x7fac1c0e5ea0 2024-09-16T15:38:57.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.161+0000 7fac27200640 1 -- 172.21.15.151:0/1893246683 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fac1c179a80 con 0x7fac1c0e6e50 2024-09-16T15:38:57.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.161+0000 7fac25400640 1 --2- 172.21.15.151:0/1893246683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac1c0e6280 0x7fac1c179f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:57.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.162+0000 7fac25400640 1 -- 172.21.15.151:0/1893246683 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac1c0e6e50 msgr2=0x7fac1c0d37c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T15:38:57.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.162+0000 7fac25400640 1 --2- 172.21.15.151:0/1893246683 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac1c0e6e50 0x7fac1c0d37c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.162+0000 7fac25400640 1 -- 172.21.15.151:0/1893246683 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac1c0e5ea0 msgr2=0x7fac1c17d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:57.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.162+0000 7fac25400640 1 --2- 172.21.15.151:0/1893246683 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac1c0e5ea0 0x7fac1c17d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.162+0000 7fac25400640 1 -- 172.21.15.151:0/1893246683 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac1c17dab0 con 0x7fac1c0e6280 2024-09-16T15:38:57.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.163+0000 7fac25400640 1 --2- 172.21.15.151:0/1893246683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac1c0e6280 0x7fac1c179f10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fac10001eb0 tx=0x7fac1000cad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:57.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.164+0000 7fac1a000640 1 -- 172.21.15.151:0/1893246683 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac10016020 con 0x7fac1c0e6280 2024-09-16T15:38:57.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.164+0000 7fac1a000640 1 -- 172.21.15.151:0/1893246683 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fac10012920 con 0x7fac1c0e6280 2024-09-16T15:38:57.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.164+0000 7fac25e00640 1 --2- 172.21.15.151:0/1893246683 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac1c0e5ea0 0x7fac1c17d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:38:57.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.164+0000 7fac27200640 1 -- 172.21.15.151:0/1893246683 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac1c04ca10 con 0x7fac1c0e6280 2024-09-16T15:38:57.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.165+0000 7fac27200640 1 -- 172.21.15.151:0/1893246683 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac1c179710 con 0x7fac1c0e6280 2024-09-16T15:38:57.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.165+0000 7fac1a000640 1 -- 172.21.15.151:0/1893246683 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac10012d50 con 0x7fac1c0e6280 2024-09-16T15:38:57.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.167+0000 7fac27200640 1 -- 172.21.15.151:0/1893246683 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabfc005290 con 0x7fac1c0e6280 2024-09-16T15:38:57.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.171+0000 7fac1a000640 1 -- 172.21.15.151:0/1893246683 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fac10019020 con 0x7fac1c0e6280 2024-09-16T15:38:57.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.171+0000 7fac1a000640 1 -- 172.21.15.151:0/1893246683 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fac1001b8e0 con 0x7fac1c0e6280 2024-09-16T15:38:57.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.175+0000 7fac1a000640 1 -- 172.21.15.151:0/1893246683 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fac10020020 con 0x7fac1c0e6280 2024-09-16T15:38:57.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.184+0000 7fa398a00640 1 -- 172.21.15.151:0/338094188 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa38c0e69b0 msgr2=0x7fa38c006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:57.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.184+0000 7fa398a00640 1 --2- 172.21.15.151:0/338094188 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa38c0e69b0 0x7fa38c006e50 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa38c035e00 tx=0x7fa3940b6d50 comp rx=0 tx=0).stop 2024-09-16T15:38:57.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.185+0000 7fa398a00640 1 -- 172.21.15.151:0/338094188 shutdown_connections 2024-09-16T15:38:57.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.185+0000 7fa398a00640 1 --2- 172.21.15.151:0/338094188 >> v2:172.21.15.167:3300/0 conn(0x7fa38c0e6d90 0x7fa38c036820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.185+0000 7fa398a00640 1 --2- 172.21.15.151:0/338094188 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa38c0e69b0 0x7fa38c006e50 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.185+0000 7fa398a00640 1 -- 172.21.15.151:0/338094188 >> 172.21.15.151:0/338094188 conn(0x7fa38c01d770 msgr2=0x7fa38c054560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:57.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.189+0000 7fbc2fc00640 1 -- 172.21.15.151:0/3291878940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc10004800 msgr2=0x7fbc10004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:57.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.189+0000 7fbc2fc00640 1 --2- 172.21.15.151:0/3291878940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc10004800 0x7fbc10004c80 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c001780 tx=0x7fbc10005830 comp rx=0 tx=0).stop 2024-09-16T15:38:57.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.189+0000 7fbc2fc00640 1 -- 172.21.15.151:0/3291878940 shutdown_connections 2024-09-16T15:38:57.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.189+0000 7fbc2fc00640 1 --2- 172.21.15.151:0/3291878940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc10004800 0x7fbc10004c80 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.189+0000 7fbc2fc00640 1 -- 172.21.15.151:0/3291878940 >> 172.21.15.151:0/3291878940 conn(0x7fbc280d3fb0 msgr2=0x7fbc280d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:57.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.190+0000 7fbc2fc00640 1 -- 172.21.15.151:0/3291878940 shutdown_connections 2024-09-16T15:38:57.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.190+0000 7fbc2fc00640 1 -- 172.21.15.151:0/3291878940 wait complete. 2024-09-16T15:38:57.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.191+0000 7f75cbe00640 1 -- 172.21.15.151:0/1672986409 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75cc15e200 msgr2=0x7f75cc072ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:57.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.191+0000 7f75cbe00640 1 --2- 172.21.15.151:0/1672986409 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75cc15e200 0x7f75cc072ab0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f75b8005fd0 tx=0x7f75b802a670 comp rx=0 tx=0).stop 2024-09-16T15:38:57.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.196+0000 7fa398a00640 1 -- 172.21.15.151:0/338094188 shutdown_connections 2024-09-16T15:38:57.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.196+0000 7fa398a00640 1 -- 172.21.15.151:0/338094188 wait complete. 2024-09-16T15:38:57.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.197+0000 7f75cbe00640 1 -- 172.21.15.151:0/1672986409 shutdown_connections 2024-09-16T15:38:57.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.197+0000 7f75cbe00640 1 --2- 172.21.15.151:0/1672986409 >> v2:172.21.15.167:3300/0 conn(0x7f75cc0c31b0 0x7f75cc072ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.197+0000 7f75cbe00640 1 --2- 172.21.15.151:0/1672986409 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75cc15e200 0x7f75cc072ab0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.197+0000 7f75cbe00640 1 -- 172.21.15.151:0/1672986409 >> 172.21.15.151:0/1672986409 conn(0x7f75cc0cfb00 msgr2=0x7f75cc0cff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:57.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.197+0000 7f75cbe00640 1 -- 172.21.15.151:0/1672986409 shutdown_connections 2024-09-16T15:38:57.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.199+0000 7f75cbe00640 1 -- 172.21.15.151:0/1672986409 wait complete. 2024-09-16T15:38:57.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.192+0000 7fbc2fc00640 1 Processor -- start 2024-09-16T15:38:57.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.200+0000 7fbc2fc00640 1 -- start start 2024-09-16T15:38:57.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.198+0000 7fa398a00640 1 Processor -- start 2024-09-16T15:38:57.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.202+0000 7fa398a00640 1 -- start start 2024-09-16T15:38:57.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.206+0000 7fbc2fc00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc10004800 0x7fbc281f78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:57.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.206+0000 7fbc2fc00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbc2809dca0 0x7fbc281f54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:57.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.207+0000 7fbc2d200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc10004800 0x7fbc281f78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:57.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.207+0000 7fbc2d200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc10004800 0x7fbc281f78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.151:45070/0 (socket says 172.21.15.151:45070) 2024-09-16T15:38:57.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.207+0000 7fbc2d200640 1 -- 172.21.15.151:0/3284986911 learned_addr learned my addr 172.21.15.151:0/3284986911 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:38:57.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.207+0000 7fbc27e00640 1 --2- 172.21.15.151:0/3284986911 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbc2809dca0 0x7fbc281f54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:57.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.207+0000 7fbc2fc00640 1 --2- 172.21.15.151:0/3284986911 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc280ccb70 0x7fbc280ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:57.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.211+0000 7fa398a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa38c0e69b0 0x7fa38c0da470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:57.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.211+0000 7fa398a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa38c0e6d90 0x7fa38c049e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:57.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.211+0000 7fa398a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa38c0d2c00 0x7fa38c0d2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:57.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.211+0000 7fbc2fc00640 1 -- 172.21.15.151:0/3284986911 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbc280b7190 con 0x7fbc280ccb70 2024-09-16T15:38:57.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.212+0000 7fbc2dc00640 1 --2- 172.21.15.151:0/3284986911 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc280ccb70 0x7fbc280ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:57.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.212+0000 7fbc2fc00640 1 -- 172.21.15.151:0/3284986911 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbc280b7010 con 0x7fbc10004800 2024-09-16T15:38:57.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.212+0000 7fbc2fc00640 1 -- 172.21.15.151:0/3284986911 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fbc280cf5b0 con 0x7fbc2809dca0 2024-09-16T15:38:57.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.213+0000 7f75cbe00640 1 Processor -- start 2024-09-16T15:38:57.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.213+0000 7fbc2dc00640 1 -- 172.21.15.151:0/3284986911 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbc2809dca0 msgr2=0x7fbc281f54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:57.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.213+0000 7fbc2dc00640 1 --2- 172.21.15.151:0/3284986911 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbc2809dca0 0x7fbc281f54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.213+0000 7fbc2dc00640 1 -- 172.21.15.151:0/3284986911 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc10004800 msgr2=0x7fbc281f78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:57.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.212+0000 7fa393e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa38c0d2c00 0x7fa38c0d2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:57.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.212+0000 7fa393e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa38c0d2c00 0x7fa38c0d2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.151:48010/0 (socket says 172.21.15.151:48010) 2024-09-16T15:38:57.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.212+0000 7fa393e00640 1 -- 172.21.15.151:0/2132673185 learned_addr learned my addr 172.21.15.151:0/2132673185 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:38:57.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.215+0000 7fa398a00640 1 -- 172.21.15.151:0/2132673185 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa38c055200 con 0x7fa38c0e69b0 2024-09-16T15:38:57.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.215+0000 7fa398a00640 1 -- 172.21.15.151:0/2132673185 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa38c055080 con 0x7fa38c0e6d90 2024-09-16T15:38:57.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.213+0000 7fbc2dc00640 1 --2- 172.21.15.151:0/3284986911 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc10004800 0x7fbc281f78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.215+0000 7fa398a00640 1 -- 172.21.15.151:0/2132673185 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fa38c055380 con 0x7fa38c0d2c00 2024-09-16T15:38:57.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.213+0000 7fbc2dc00640 1 -- 172.21.15.151:0/3284986911 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc2809e1d0 con 0x7fbc280ccb70 2024-09-16T15:38:57.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.215+0000 7fa392a00640 1 --2- 172.21.15.151:0/2132673185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa38c0e6d90 0x7fa38c049e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:57.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.214+0000 7fbc2dc00640 1 --2- 172.21.15.151:0/3284986911 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc280ccb70 0x7fbc280ccff0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fbc28069890 tx=0x7fbc2000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:57.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.215+0000 7fa393400640 1 --2- 172.21.15.151:0/2132673185 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa38c0e69b0 0x7fa38c0da470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:57.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.215+0000 7fbc24c00640 1 -- 172.21.15.151:0/3284986911 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc20003d80 con 0x7fbc280ccb70 2024-09-16T15:38:57.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.215+0000 7fbc24c00640 1 -- 172.21.15.151:0/3284986911 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbc2000b950 con 0x7fbc280ccb70 2024-09-16T15:38:57.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.215+0000 7fbc24c00640 1 -- 172.21.15.151:0/3284986911 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2000bda0 con 0x7fbc280ccb70 2024-09-16T15:38:57.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.216+0000 7fa392a00640 1 -- 172.21.15.151:0/2132673185 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa38c0d2c00 msgr2=0x7fa38c0d2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:57.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.216+0000 7fbc2fc00640 1 -- 172.21.15.151:0/3284986911 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc2809e4b0 con 0x7fbc280ccb70 2024-09-16T15:38:57.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.216+0000 7fa392a00640 1 --2- 172.21.15.151:0/2132673185 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa38c0d2c00 0x7fa38c0d2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.216+0000 7fa392a00640 1 -- 172.21.15.151:0/2132673185 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa38c0e69b0 msgr2=0x7fa38c0da470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:57.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.218+0000 7fbc2fc00640 1 -- 172.21.15.151:0/3284986911 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc2809ea70 con 0x7fbc280ccb70 2024-09-16T15:38:57.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.218+0000 7fbc24c00640 1 -- 172.21.15.151:0/3284986911 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbc20006400 con 0x7fbc280ccb70 2024-09-16T15:38:57.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.217+0000 7fa393e00640 1 --2- 172.21.15.151:0/2132673185 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa38c0d2c00 0x7fa38c0d2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:38:57.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.217+0000 7fa392a00640 1 --2- 172.21.15.151:0/2132673185 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa38c0e69b0 0x7fa38c0da470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.217+0000 7fa392a00640 1 -- 172.21.15.151:0/2132673185 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa38c179200 con 0x7fa38c0e6d90 2024-09-16T15:38:57.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.217+0000 7fa393400640 1 --2- 172.21.15.151:0/2132673185 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa38c0e69b0 0x7fa38c0da470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:38:57.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.218+0000 7fa392a00640 1 --2- 172.21.15.151:0/2132673185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa38c0e6d90 0x7fa38c049e30 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa38c035420 tx=0x7fa38000c870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:57.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.219+0000 7fa387400640 1 -- 172.21.15.151:0/2132673185 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa380017070 con 0x7fa38c0e6d90 2024-09-16T15:38:57.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.219+0000 7fa398a00640 1 -- 172.21.15.151:0/2132673185 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa38c1794e0 con 0x7fa38c0e6d90 2024-09-16T15:38:57.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.220+0000 7fa398a00640 1 -- 172.21.15.151:0/2132673185 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa38c179aa0 con 0x7fa38c0e6d90 2024-09-16T15:38:57.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.220+0000 7fa387400640 1 -- 172.21.15.151:0/2132673185 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa380003610 con 0x7fa38c0e6d90 2024-09-16T15:38:57.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.220+0000 7fa387400640 1 -- 172.21.15.151:0/2132673185 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa380017070 con 0x7fa38c0e6d90 2024-09-16T15:38:57.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.221+0000 7fa387400640 1 -- 172.21.15.151:0/2132673185 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa38001a070 con 0x7fa38c0e6d90 2024-09-16T15:38:57.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.221+0000 7fa387400640 1 -- 172.21.15.151:0/2132673185 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa38001cc80 con 0x7fa38c0e6d90 2024-09-16T15:38:57.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.217+0000 7f54f7e00640 1 -- 172.21.15.151:0/104284297 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7f54f81597f0 con 0x7f54f8156690 2024-09-16T15:38:57.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.218+0000 7fbc24c00640 1 -- 172.21.15.151:0/3284986911 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fbc200237b0 con 0x7fbc280ccb70 2024-09-16T15:38:57.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.213+0000 7f75cbe00640 1 -- start start 2024-09-16T15:38:57.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.215+0000 7f75cbe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75cc0c31b0 0x7f75cc0c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:57.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.215+0000 7f75cbe00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f75cc0cdab0 0x7f75cc0c9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:57.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.215+0000 7f75cbe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75cc15e200 0x7f75cc158b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:57.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.215+0000 7f75cbe00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f75cc0cf070 con 0x7f75cc15e200 2024-09-16T15:38:57.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.215+0000 7f75cbe00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f75cc15e7f0 con 0x7f75cc0c31b0 2024-09-16T15:38:57.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.215+0000 7f75cbe00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f75cc0cf1f0 con 0x7f75cc0cdab0 2024-09-16T15:38:57.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.215+0000 7f75caa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75cc0c31b0 0x7f75cc0c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:57.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.215+0000 7f75caa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75cc0c31b0 0x7f75cc0c7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.151:45082/0 (socket says 172.21.15.151:45082) 2024-09-16T15:38:57.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.215+0000 7f75caa00640 1 -- 172.21.15.151:0/619974434 learned_addr learned my addr 172.21.15.151:0/619974434 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:38:57.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.216+0000 7f75caa00640 1 -- 172.21.15.151:0/619974434 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f75cc0cdab0 msgr2=0x7f75cc0c9790 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:38:57.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.216+0000 7f75caa00640 1 --2- 172.21.15.151:0/619974434 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f75cc0cdab0 0x7f75cc0c9790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.216+0000 7f75caa00640 1 -- 172.21.15.151:0/619974434 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75cc15e200 msgr2=0x7f75cc158b20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:38:57.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.216+0000 7f75caa00640 1 --2- 172.21.15.151:0/619974434 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75cc15e200 0x7f75cc158b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.216+0000 7f75caa00640 1 -- 172.21.15.151:0/619974434 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75cc0c9cd0 con 0x7f75cc0c31b0 2024-09-16T15:38:57.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.217+0000 7f75caa00640 1 --2- 172.21.15.151:0/619974434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75cc0c31b0 0x7f75cc0c7db0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f75b8006010 tx=0x7f75b8003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:57.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.218+0000 7f75c3400640 1 -- 172.21.15.151:0/619974434 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b803e030 con 0x7f75cc0c31b0 2024-09-16T15:38:57.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.218+0000 7f75c3400640 1 -- 172.21.15.151:0/619974434 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f75b8003c90 con 0x7f75cc0c31b0 2024-09-16T15:38:57.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.218+0000 7f75c3400640 1 -- 172.21.15.151:0/619974434 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b803e030 con 0x7f75cc0c31b0 2024-09-16T15:38:57.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.221+0000 7f75cbe00640 1 -- 172.21.15.151:0/619974434 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75cc0c37b0 con 0x7f75cc0c31b0 2024-09-16T15:38:57.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.221+0000 7f75cbe00640 1 -- 172.21.15.151:0/619974434 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75cc0c3d50 con 0x7f75cc0c31b0 2024-09-16T15:38:57.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.223+0000 7f75c0c00640 1 -- 172.21.15.151:0/619974434 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75a0005290 con 0x7f75cc0c31b0 2024-09-16T15:38:57.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.225+0000 7f54eea00640 1 -- 172.21.15.151:0/104284297 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v9)=0 v9) ==== 106+0+0 (secure 0 0 0) 0x7f54f000b7c0 con 0x7f54f8156690 2024-09-16T15:38:57.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.225+0000 7f75c3400640 1 -- 172.21.15.151:0/619974434 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f75b8041030 con 0x7f75cc0c31b0 2024-09-16T15:38:57.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.227+0000 7f54f7e00640 1 -- 172.21.15.151:0/104284297 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f54f8156690 msgr2=0x7f54f8156b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:57.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.227+0000 7f54f7e00640 1 --2- 172.21.15.151:0/104284297 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f54f8156690 0x7f54f8156b10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f54f815e070 tx=0x7f54f000dcd0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.227+0000 7f75c3400640 1 -- 172.21.15.151:0/619974434 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f75b80437a0 con 0x7f75cc0c31b0 2024-09-16T15:38:57.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.227+0000 7f54f7e00640 1 -- 172.21.15.151:0/104284297 shutdown_connections 2024-09-16T15:38:57.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.227+0000 7f54f7e00640 1 --2- 172.21.15.151:0/104284297 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f54f8156690 0x7f54f8156b10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.227+0000 7f54f7e00640 1 --2- 172.21.15.151:0/104284297 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f54f80c4ff0 0x7f54f8072d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.227+0000 7f54f7e00640 1 --2- 172.21.15.151:0/104284297 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f54dc003c30 0x7f54f80c5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.227+0000 7f54f7e00640 1 -- 172.21.15.151:0/104284297 >> 172.21.15.151:0/104284297 conn(0x7f54f80cf9f0 msgr2=0x7f54f81589d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:57.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.226+0000 7fa398a00640 1 -- 172.21.15.151:0/2132673185 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa370005290 con 0x7fa38c0e6d90 2024-09-16T15:38:57.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.228+0000 7fbc27e00640 1 --2- 172.21.15.151:0/3284986911 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbc2809dca0 0x7fbc281f54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:38:57.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.228+0000 7fbc2fc00640 1 -- 172.21.15.151:0/3284986911 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc04005290 con 0x7fbc280ccb70 2024-09-16T15:38:57.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.232+0000 7f75c3400640 1 -- 172.21.15.151:0/619974434 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f75b8043aa0 con 0x7f75cc0c31b0 2024-09-16T15:38:57.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.232+0000 7f54f7e00640 1 -- 172.21.15.151:0/104284297 shutdown_connections 2024-09-16T15:38:57.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.232+0000 7f54f7e00640 1 -- 172.21.15.151:0/104284297 wait complete. 2024-09-16T15:38:57.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.232+0000 7fbc24c00640 1 -- 172.21.15.151:0/3284986911 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fbc20029060 con 0x7fbc280ccb70 2024-09-16T15:38:57.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.240+0000 7fa387400640 1 -- 172.21.15.151:0/2132673185 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fa38001c440 con 0x7fa38c0e6d90 2024-09-16T15:38:57.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.253+0000 7f0fdf400640 1 -- 172.21.15.151:0/3772388281 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) -- 0x7f0fc8005580 con 0x7f0ff40cb840 2024-09-16T15:38:57.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.258+0000 7f0fea000640 1 -- 172.21.15.151:0/3772388281 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v10)=0 v10) ==== 115+0+0 (secure 0 0 0) 0x7f0fec004b40 con 0x7f0ff40cb840 2024-09-16T15:38:57.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.260+0000 7f0fdf400640 1 -- 172.21.15.151:0/3772388281 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ff40cb840 msgr2=0x7f0ff41532b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:57.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.260+0000 7f0fdf400640 1 --2- 172.21.15.151:0/3772388281 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ff40cb840 0x7f0ff41532b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0fec007ed0 tx=0x7f0fec00a6c0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.260+0000 7f0fdf400640 1 -- 172.21.15.151:0/3772388281 shutdown_connections 2024-09-16T15:38:57.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.260+0000 7f0fdf400640 1 --2- 172.21.15.151:0/3772388281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ff40ca040 0x7f0ff40c6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.260+0000 7f0fdf400640 1 --2- 172.21.15.151:0/3772388281 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ff40cb840 0x7f0ff41532b0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.260+0000 7f0fdf400640 1 --2- 172.21.15.151:0/3772388281 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ff406e5a0 0x7f0ff4150810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.260+0000 7f0fdf400640 1 -- 172.21.15.151:0/3772388281 >> 172.21.15.151:0/3772388281 conn(0x7f0ff40cf9f0 msgr2=0x7f0ff40cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:57.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.261+0000 7f0fdf400640 1 -- 172.21.15.151:0/3772388281 shutdown_connections 2024-09-16T15:38:57.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.261+0000 7f0fdf400640 1 -- 172.21.15.151:0/3772388281 wait complete. 2024-09-16T15:38:57.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.261+0000 7fed77200640 1 -- 172.21.15.151:0/637259427 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fed7815a0c0 msgr2=0x7fed780ca040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:57.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.261+0000 7fed77200640 1 --2- 172.21.15.151:0/637259427 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fed7815a0c0 0x7fed780ca040 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fed68009880 tx=0x7fed6802b530 comp rx=0 tx=0).stop 2024-09-16T15:38:57.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.261+0000 7fed77200640 1 -- 172.21.15.151:0/637259427 shutdown_connections 2024-09-16T15:38:57.264 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi151') 2024-09-16T15:38:57.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.261+0000 7fed77200640 1 --2- 172.21.15.151:0/637259427 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fed7815a0c0 0x7fed780ca040 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.262+0000 7fed77200640 1 -- 172.21.15.151:0/637259427 >> 172.21.15.151:0/637259427 conn(0x7fed780cf9f0 msgr2=0x7fed780cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:57.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.262+0000 7fed77200640 1 -- 172.21.15.151:0/637259427 shutdown_connections 2024-09-16T15:38:57.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.262+0000 7fed77200640 1 -- 172.21.15.151:0/637259427 wait complete. 2024-09-16T15:38:57.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.266+0000 7fed77200640 1 Processor -- start 2024-09-16T15:38:57.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.267+0000 7fed77200640 1 -- start start 2024-09-16T15:38:57.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.271+0000 7fed77200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed780cf540 0x7fed780c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:57.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.272+0000 7fed77200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fed7815a0c0 0x7fed7814e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:57.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.272+0000 7fed77200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fed7815dc90 0x7fed781fd040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:57.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.272+0000 7fed77200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fed780d0690 con 0x7fed7815a0c0 2024-09-16T15:38:57.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.272+0000 7fed77200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fed780d0510 con 0x7fed780cf540 2024-09-16T15:38:57.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.272+0000 7fed77200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fed7815b2c0 con 0x7fed7815dc90 2024-09-16T15:38:57.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.272+0000 7fed75e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed780cf540 0x7fed780c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:57.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.272+0000 7fed75400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fed7815a0c0 0x7fed7814e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:57.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.272+0000 7fed75400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fed7815a0c0 0x7fed7814e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43904/0 (socket says 172.21.15.151:43904) 2024-09-16T15:38:57.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.272+0000 7fed75400640 1 -- 172.21.15.151:0/2322896019 learned_addr learned my addr 172.21.15.151:0/2322896019 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:38:57.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.273+0000 7fca87e00640 1 -- 172.21.15.151:0/2854331042 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca800b5630 msgr2=0x7fca800cf5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:57.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.273+0000 7fca87e00640 1 --2- 172.21.15.151:0/2854331042 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca800b5630 0x7fca800cf5b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fca74009880 tx=0x7fca7402a560 comp rx=0 tx=0).stop 2024-09-16T15:38:57.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.273+0000 7fed76800640 1 --2- 172.21.15.151:0/2322896019 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fed7815dc90 0x7fed781fd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:57.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.273+0000 7fca87e00640 1 -- 172.21.15.151:0/2854331042 shutdown_connections 2024-09-16T15:38:57.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.273+0000 7fca87e00640 1 --2- 172.21.15.151:0/2854331042 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca800b5630 0x7fca800cf5b0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.273+0000 7fca87e00640 1 --2- 172.21.15.151:0/2854331042 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca80161590 0x7fca800b6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.273+0000 7fca87e00640 1 -- 172.21.15.151:0/2854331042 >> 172.21.15.151:0/2854331042 conn(0x7fca800d3fb0 msgr2=0x7fca800d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:57.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.273+0000 7fed75400640 1 -- 172.21.15.151:0/2322896019 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fed7815dc90 msgr2=0x7fed781fd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:57.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.274+0000 7fca87e00640 1 -- 172.21.15.151:0/2854331042 shutdown_connections 2024-09-16T15:38:57.278 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.273+0000 7fed75400640 1 --2- 172.21.15.151:0/2322896019 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fed7815dc90 0x7fed781fd040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.278 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.273+0000 7fed75400640 1 -- 172.21.15.151:0/2322896019 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed780cf540 msgr2=0x7fed780c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:57.278 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.273+0000 7fed75400640 1 --2- 172.21.15.151:0/2322896019 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed780cf540 0x7fed780c6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.278 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.273+0000 7fed75400640 1 -- 172.21.15.151:0/2322896019 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed781f94a0 con 0x7fed7815a0c0 2024-09-16T15:38:57.278 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.274+0000 7fca87e00640 1 -- 172.21.15.151:0/2854331042 wait complete. 2024-09-16T15:38:57.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.276+0000 7fca87e00640 1 Processor -- start 2024-09-16T15:38:57.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.277+0000 7fed75400640 1 --2- 172.21.15.151:0/2322896019 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fed7815a0c0 0x7fed7814e590 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fed6400c700 tx=0x7fed6400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:57.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.280+0000 7fed6e000640 1 -- 172.21.15.151:0/2322896019 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed64017070 con 0x7fed7815a0c0 2024-09-16T15:38:57.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.280+0000 7fed6e000640 1 -- 172.21.15.151:0/2322896019 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fed64005360 con 0x7fed7815a0c0 2024-09-16T15:38:57.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.280+0000 7fed6e000640 1 -- 172.21.15.151:0/2322896019 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6400a8a0 con 0x7fed7815a0c0 2024-09-16T15:38:57.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.280+0000 7fed77200640 1 -- 172.21.15.151:0/2322896019 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed781f9740 con 0x7fed7815a0c0 2024-09-16T15:38:57.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.280+0000 7fed77200640 1 -- 172.21.15.151:0/2322896019 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed781f9d00 con 0x7fed7815a0c0 2024-09-16T15:38:57.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.282+0000 7fed6e000640 1 -- 172.21.15.151:0/2322896019 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fed6401a070 con 0x7fed7815a0c0 2024-09-16T15:38:57.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.282+0000 7fed6e000640 1 -- 172.21.15.151:0/2322896019 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fed6401c440 con 0x7fed7815a0c0 2024-09-16T15:38:57.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.282+0000 7fed77200640 1 -- 172.21.15.151:0/2322896019 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed780ca040 con 0x7fed7815a0c0 2024-09-16T15:38:57.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.280+0000 7fca87e00640 1 -- start start 2024-09-16T15:38:57.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.282+0000 7fca87e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca800b5630 0x7fca8015f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:57.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.282+0000 7fca87e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca800b7340 0x7fca80204a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:57.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.282+0000 7fca87e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca80161590 0x7fca801f5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:57.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.282+0000 7fca87e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fca800cfe80 con 0x7fca800b7340 2024-09-16T15:38:57.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.282+0000 7fca87e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fca800cfd00 con 0x7fca80161590 2024-09-16T15:38:57.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.282+0000 7fca87e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fca80162430 con 0x7fca800b5630 2024-09-16T15:38:57.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.287+0000 7fca85400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca800b5630 0x7fca8015f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:57.290 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.287+0000 7fca85400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca800b5630 0x7fca8015f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.151:48030/0 (socket says 172.21.15.151:48030) 2024-09-16T15:38:57.290 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.287+0000 7fca85400640 1 -- 172.21.15.151:0/2427696498 learned_addr learned my addr 172.21.15.151:0/2427696498 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:38:57.290 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.287+0000 7fca85e00640 1 --2- 172.21.15.151:0/2427696498 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca80161590 0x7fca801f5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:57.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.288+0000 7fca85e00640 1 -- 172.21.15.151:0/2427696498 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca800b5630 msgr2=0x7fca8015f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:57.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.288+0000 7fca85e00640 1 --2- 172.21.15.151:0/2427696498 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca800b5630 0x7fca8015f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.288+0000 7fca85e00640 1 -- 172.21.15.151:0/2427696498 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca800b7340 msgr2=0x7fca80204a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:57.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.288+0000 7fca85e00640 1 --2- 172.21.15.151:0/2427696498 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca800b7340 0x7fca80204a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.288+0000 7fca85e00640 1 -- 172.21.15.151:0/2427696498 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca80205330 con 0x7fca80161590 2024-09-16T15:38:57.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.288+0000 7fca85400640 1 --2- 172.21.15.151:0/2427696498 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca800b5630 0x7fca8015f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:38:57.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.289+0000 7fca85e00640 1 --2- 172.21.15.151:0/2427696498 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca80161590 0x7fca801f5140 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fca7800ca70 tx=0x7fca7800cf30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:57.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.289+0000 7fca7d600640 1 -- 172.21.15.151:0/2427696498 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca78016020 con 0x7fca80161590 2024-09-16T15:38:57.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.290+0000 7fca7d600640 1 -- 172.21.15.151:0/2427696498 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fca7800acc0 con 0x7fca80161590 2024-09-16T15:38:57.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.290+0000 7fca7d600640 1 -- 172.21.15.151:0/2427696498 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca78012680 con 0x7fca80161590 2024-09-16T15:38:57.294 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi151') 2024-09-16T15:38:57.294 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.292+0000 7fed6e000640 1 -- 172.21.15.151:0/2322896019 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fed64021020 con 0x7fed7815a0c0 2024-09-16T15:38:57.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.295+0000 7fca87e00640 1 -- 172.21.15.151:0/2427696498 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca80205610 con 0x7fca80161590 2024-09-16T15:38:57.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.295+0000 7fca87e00640 1 -- 172.21.15.151:0/2427696498 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca8015f810 con 0x7fca80161590 2024-09-16T15:38:57.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.295+0000 7fca7d600640 1 -- 172.21.15.151:0/2427696498 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fca78019020 con 0x7fca80161590 2024-09-16T15:38:57.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.297+0000 7fca87e00640 1 -- 172.21.15.151:0/2427696498 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca801628b0 con 0x7fca80161590 2024-09-16T15:38:57.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.299+0000 7fca7d600640 1 -- 172.21.15.151:0/2427696498 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fca78005320 con 0x7fca80161590 2024-09-16T15:38:57.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.303+0000 7fca7d600640 1 -- 172.21.15.151:0/2427696498 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fca78020030 con 0x7fca80161590 2024-09-16T15:38:57.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.322+0000 7f8b57c00640 1 -- 172.21.15.151:0/1198397358 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) -- 0x7f8b480497e0 con 0x7f8b480e4090 2024-09-16T15:38:57.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.324+0000 7f2d63e00640 1 -- 172.21.15.151:0/2532260370 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) -- 0x7f2d640d0900 con 0x7f2d64158930 2024-09-16T15:38:57.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.332+0000 7f2d5b400640 1 -- 172.21.15.151:0/2532260370 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v11)=0 v11) ==== 119+0+0 (secure 0 0 0) 0x7f2d640c7f20 con 0x7f2d64158930 2024-09-16T15:38:57.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.337+0000 7f2d63e00640 1 -- 172.21.15.151:0/2532260370 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d64158930 msgr2=0x7f2d640c54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:57.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.337+0000 7f2d63e00640 1 --2- 172.21.15.151:0/2532260370 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d64158930 0x7f2d640c54c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2d5400c430 tx=0x7f2d54001780 comp rx=0 tx=0).stop 2024-09-16T15:38:57.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.337+0000 7f2d63e00640 1 -- 172.21.15.151:0/2532260370 shutdown_connections 2024-09-16T15:38:57.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.337+0000 7f2d63e00640 1 --2- 172.21.15.151:0/2532260370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d6415ba20 0x7f2d641fc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.337+0000 7f2d63e00640 1 --2- 172.21.15.151:0/2532260370 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d64158930 0x7f2d640c54c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.337+0000 7f2d63e00640 1 --2- 172.21.15.151:0/2532260370 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d6415b640 0x7f2d640c4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.337+0000 7f2d63e00640 1 -- 172.21.15.151:0/2532260370 >> 172.21.15.151:0/2532260370 conn(0x7f2d640cf9f0 msgr2=0x7f2d640cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:57.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.337+0000 7f2d63e00640 1 -- 172.21.15.151:0/2532260370 shutdown_connections 2024-09-16T15:38:57.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.337+0000 7f2d63e00640 1 -- 172.21.15.151:0/2532260370 wait complete. 2024-09-16T15:38:57.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.338+0000 7f8b4cc00640 1 -- 172.21.15.151:0/1198397358 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v12)=0 v12) ==== 122+0+0 (secure 0 0 0) 0x7f8b50063d00 con 0x7f8b480e4090 2024-09-16T15:38:57.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.340+0000 7f8b57c00640 1 -- 172.21.15.151:0/1198397358 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b480e4090 msgr2=0x7f8b4803bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:57.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.340+0000 7f8b57c00640 1 --2- 172.21.15.151:0/1198397358 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b480e4090 0x7f8b4803bf80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8b500527e0 tx=0x7f8b5006a370 comp rx=0 tx=0).stop 2024-09-16T15:38:57.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.342+0000 7f8b57c00640 1 -- 172.21.15.151:0/1198397358 shutdown_connections 2024-09-16T15:38:57.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.342+0000 7f8b57c00640 1 --2- 172.21.15.151:0/1198397358 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b48052040 0x7f8b4804c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.342+0000 7f8b57c00640 1 --2- 172.21.15.151:0/1198397358 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b480e4090 0x7f8b4803bf80 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.342+0000 7f8b57c00640 1 --2- 172.21.15.151:0/1198397358 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b48035630 0x7f8b48181c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.342+0000 7f8b57c00640 1 -- 172.21.15.151:0/1198397358 >> 172.21.15.151:0/1198397358 conn(0x7f8b48056770 msgr2=0x7f8b4800d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:57.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.342+0000 7f8b57c00640 1 -- 172.21.15.151:0/1198397358 shutdown_connections 2024-09-16T15:38:57.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.344+0000 7f8b57c00640 1 -- 172.21.15.151:0/1198397358 wait complete. 2024-09-16T15:38:57.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.363+0000 7ffa66600640 1 -- 172.21.15.151:0/2266242900 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7ffa600d0900 con 0x7ffa6015e670 2024-09-16T15:38:57.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.373+0000 7ffa5cc00640 1 -- 172.21.15.151:0/2266242900 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v13)=0 v13) ==== 121+0+0 (secure 0 0 0) 0x7ffa5803e450 con 0x7ffa6015e670 2024-09-16T15:38:57.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.376+0000 7ffa66600640 1 -- 172.21.15.151:0/2266242900 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffa6015e670 msgr2=0x7ffa600c9500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:57.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.376+0000 7ffa66600640 1 --2- 172.21.15.151:0/2266242900 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffa6015e670 0x7ffa600c9500 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ffa58009000 tx=0x7ffa58033ca0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.377+0000 7ffa66600640 1 -- 172.21.15.151:0/2266242900 shutdown_connections 2024-09-16T15:38:57.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.377+0000 7ffa66600640 1 --2- 172.21.15.151:0/2266242900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa6015e290 0x7ffa600b4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.377+0000 7ffa66600640 1 --2- 172.21.15.151:0/2266242900 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffa6015e670 0x7ffa600c9500 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.377+0000 7ffa66600640 1 --2- 172.21.15.151:0/2266242900 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa60074330 0x7ffa60201300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.377+0000 7ffa66600640 1 -- 172.21.15.151:0/2266242900 >> 172.21.15.151:0/2266242900 conn(0x7ffa600cf9f0 msgr2=0x7ffa600ce090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:57.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.377+0000 7ffa66600640 1 -- 172.21.15.151:0/2266242900 shutdown_connections 2024-09-16T15:38:57.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.377+0000 7ffa66600640 1 -- 172.21.15.151:0/2266242900 wait complete. 2024-09-16T15:38:57.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.383+0000 7fac27200640 1 -- 172.21.15.151:0/1893246683 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7fabfc005580 con 0x7fac1c0e6280 2024-09-16T15:38:57.388 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi151') 2024-09-16T15:38:57.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.394+0000 7fac1a000640 1 -- 172.21.15.151:0/1893246683 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v14)=0 v14) ==== 120+0+0 (secure 0 0 0) 0x7fac1c17dab0 con 0x7fac1c0e6280 2024-09-16T15:38:57.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.396+0000 7fac27200640 1 -- 172.21.15.151:0/1893246683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac1c0e6280 msgr2=0x7fac1c179f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:57.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.397+0000 7fac27200640 1 --2- 172.21.15.151:0/1893246683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac1c0e6280 0x7fac1c179f10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fac10001eb0 tx=0x7fac1000cad0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.397+0000 7fac27200640 1 -- 172.21.15.151:0/1893246683 shutdown_connections 2024-09-16T15:38:57.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.397+0000 7fac27200640 1 --2- 172.21.15.151:0/1893246683 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac1c0e6e50 0x7fac1c0d37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.397+0000 7fac27200640 1 --2- 172.21.15.151:0/1893246683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac1c0e6280 0x7fac1c179f10 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.397+0000 7fac27200640 1 --2- 172.21.15.151:0/1893246683 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac1c0e5ea0 0x7fac1c17d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.397+0000 7fac27200640 1 -- 172.21.15.151:0/1893246683 >> 172.21.15.151:0/1893246683 conn(0x7fac1c01d770 msgr2=0x7fac1c0099b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:57.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.398+0000 7fac27200640 1 -- 172.21.15.151:0/1893246683 shutdown_connections 2024-09-16T15:38:57.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.398+0000 7fac27200640 1 -- 172.21.15.151:0/1893246683 wait complete. 2024-09-16T15:38:57.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.402+0000 7f75c0c00640 1 -- 172.21.15.151:0/619974434 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7f75a00055f0 con 0x7f75cc0c31b0 2024-09-16T15:38:57.410 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi151') 2024-09-16T15:38:57.410 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi151') 2024-09-16T15:38:57.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.409+0000 7f75c3400640 1 -- 172.21.15.151:0/619974434 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v15)=0 v15) ==== 121+0+0 (secure 0 0 0) 0x7f75a00055f0 con 0x7f75cc0c31b0 2024-09-16T15:38:57.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.411+0000 7f75c0c00640 1 -- 172.21.15.151:0/619974434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75cc0c31b0 msgr2=0x7f75cc0c7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:57.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.411+0000 7f75c0c00640 1 --2- 172.21.15.151:0/619974434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75cc0c31b0 0x7f75cc0c7db0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f75b8006010 tx=0x7f75b8003820 comp rx=0 tx=0).stop 2024-09-16T15:38:57.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.412+0000 7f75c0c00640 1 -- 172.21.15.151:0/619974434 shutdown_connections 2024-09-16T15:38:57.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.412+0000 7f75c0c00640 1 --2- 172.21.15.151:0/619974434 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75cc15e200 0x7f75cc158b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.412+0000 7f75c0c00640 1 --2- 172.21.15.151:0/619974434 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f75cc0cdab0 0x7f75cc0c9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.412+0000 7f75c0c00640 1 --2- 172.21.15.151:0/619974434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75cc0c31b0 0x7f75cc0c7db0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.412+0000 7f75c0c00640 1 -- 172.21.15.151:0/619974434 >> 172.21.15.151:0/619974434 conn(0x7f75cc0cfb00 msgr2=0x7f75cc0af640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:57.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.412+0000 7f75c0c00640 1 -- 172.21.15.151:0/619974434 shutdown_connections 2024-09-16T15:38:57.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.412+0000 7f75c0c00640 1 -- 172.21.15.151:0/619974434 wait complete. 2024-09-16T15:38:57.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.413+0000 7fbc2fc00640 1 -- 172.21.15.151:0/3284986911 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7fbc04005850 con 0x7fbc280ccb70 2024-09-16T15:38:57.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.420+0000 7fbc24c00640 1 -- 172.21.15.151:0/3284986911 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v16)=0 v16) ==== 120+0+0 (secure 0 0 0) 0x7fbc200066f0 con 0x7fbc280ccb70 2024-09-16T15:38:57.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.422+0000 7fbc2fc00640 1 -- 172.21.15.151:0/3284986911 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc280ccb70 msgr2=0x7fbc280ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:57.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.422+0000 7fbc2fc00640 1 --2- 172.21.15.151:0/3284986911 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc280ccb70 0x7fbc280ccff0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fbc28069890 tx=0x7fbc2000de10 comp rx=0 tx=0).stop 2024-09-16T15:38:57.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.422+0000 7fbc2fc00640 1 -- 172.21.15.151:0/3284986911 shutdown_connections 2024-09-16T15:38:57.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.422+0000 7fbc2fc00640 1 --2- 172.21.15.151:0/3284986911 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc280ccb70 0x7fbc280ccff0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.422+0000 7fbc2fc00640 1 --2- 172.21.15.151:0/3284986911 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbc2809dca0 0x7fbc281f54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.422+0000 7fbc2fc00640 1 --2- 172.21.15.151:0/3284986911 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc10004800 0x7fbc281f78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.422+0000 7fbc2fc00640 1 -- 172.21.15.151:0/3284986911 >> 172.21.15.151:0/3284986911 conn(0x7fbc280d3fb0 msgr2=0x7fbc28161bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:57.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.423+0000 7fbc2fc00640 1 -- 172.21.15.151:0/3284986911 shutdown_connections 2024-09-16T15:38:57.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.424+0000 7fbc2fc00640 1 -- 172.21.15.151:0/3284986911 wait complete. 2024-09-16T15:38:57.437 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.435+0000 7fa398a00640 1 -- 172.21.15.151:0/2132673185 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7fa370005580 con 0x7fa38c0e6d90 2024-09-16T15:38:57.442 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.441+0000 7fa387400640 1 -- 172.21.15.151:0/2132673185 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v17)=0 v17) ==== 121+0+0 (secure 0 0 0) 0x7fa380003830 con 0x7fa38c0e6d90 2024-09-16T15:38:57.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.442+0000 7fa398a00640 1 -- 172.21.15.151:0/2132673185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa38c0e6d90 msgr2=0x7fa38c049e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:57.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.442+0000 7fa398a00640 1 --2- 172.21.15.151:0/2132673185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa38c0e6d90 0x7fa38c049e30 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa38c035420 tx=0x7fa38000c870 comp rx=0 tx=0).stop 2024-09-16T15:38:57.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.442+0000 7fa398a00640 1 -- 172.21.15.151:0/2132673185 shutdown_connections 2024-09-16T15:38:57.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.442+0000 7fa398a00640 1 --2- 172.21.15.151:0/2132673185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa38c0e6d90 0x7fa38c049e30 secure :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa38c035420 tx=0x7fa38000c870 comp rx=0 tx=0).stop 2024-09-16T15:38:57.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.442+0000 7fa398a00640 1 --2- 172.21.15.151:0/2132673185 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa38c0d2c00 0x7fa38c0d2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.442+0000 7fa398a00640 1 --2- 172.21.15.151:0/2132673185 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa38c0e69b0 0x7fa38c0da470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.442+0000 7fa398a00640 1 -- 172.21.15.151:0/2132673185 >> 172.21.15.151:0/2132673185 conn(0x7fa38c01d770 msgr2=0x7fa38c0e0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:57.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.442+0000 7fa398a00640 1 -- 172.21.15.151:0/2132673185 shutdown_connections 2024-09-16T15:38:57.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.442+0000 7fa398a00640 1 -- 172.21.15.151:0/2132673185 wait complete. 2024-09-16T15:38:57.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.447+0000 7fed77200640 1 -- 172.21.15.151:0/2322896019 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) -- 0x7fed780703d0 con 0x7fed7815a0c0 2024-09-16T15:38:57.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.452+0000 7fed6e000640 1 -- 172.21.15.151:0/2322896019 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v18)=0 v18) ==== 125+0+0 (secure 0 0 0) 0x7fed640035f0 con 0x7fed7815a0c0 2024-09-16T15:38:57.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.453+0000 7fed77200640 1 -- 172.21.15.151:0/2322896019 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fed7815a0c0 msgr2=0x7fed7814e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:57.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.453+0000 7fed77200640 1 --2- 172.21.15.151:0/2322896019 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fed7815a0c0 0x7fed7814e590 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fed6400c700 tx=0x7fed6400cbc0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.454+0000 7fed77200640 1 -- 172.21.15.151:0/2322896019 shutdown_connections 2024-09-16T15:38:57.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.454+0000 7fed77200640 1 --2- 172.21.15.151:0/2322896019 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fed7815dc90 0x7fed781fd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.454+0000 7fed77200640 1 --2- 172.21.15.151:0/2322896019 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fed7815a0c0 0x7fed7814e590 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.454+0000 7fed77200640 1 --2- 172.21.15.151:0/2322896019 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed780cf540 0x7fed780c6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.454+0000 7fed77200640 1 -- 172.21.15.151:0/2322896019 >> 172.21.15.151:0/2322896019 conn(0x7fed780cf9f0 msgr2=0x7fed7815d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:57.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.454+0000 7fed77200640 1 -- 172.21.15.151:0/2322896019 shutdown_connections 2024-09-16T15:38:57.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.454+0000 7fed77200640 1 -- 172.21.15.151:0/2322896019 wait complete. 2024-09-16T15:38:57.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.457+0000 7fca87e00640 1 -- 172.21.15.151:0/2427696498 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7fca800b9a10 con 0x7fca80161590 2024-09-16T15:38:57.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.461+0000 7fca7d600640 1 -- 172.21.15.151:0/2427696498 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v19)=0 v19) ==== 121+0+0 (secure 0 0 0) 0x7fca78005520 con 0x7fca80161590 2024-09-16T15:38:57.465 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi151') 2024-09-16T15:38:57.465 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi151') 2024-09-16T15:38:57.465 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi151') 2024-09-16T15:38:57.465 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mon', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi151') 2024-09-16T15:38:57.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.462+0000 7fca87e00640 1 -- 172.21.15.151:0/2427696498 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca80161590 msgr2=0x7fca801f5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:57.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.463+0000 7fca87e00640 1 --2- 172.21.15.151:0/2427696498 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca80161590 0x7fca801f5140 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fca7800ca70 tx=0x7fca7800cf30 comp rx=0 tx=0).stop 2024-09-16T15:38:57.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.463+0000 7fca87e00640 1 -- 172.21.15.151:0/2427696498 shutdown_connections 2024-09-16T15:38:57.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.463+0000 7fca87e00640 1 --2- 172.21.15.151:0/2427696498 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca80161590 0x7fca801f5140 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.463+0000 7fca87e00640 1 --2- 172.21.15.151:0/2427696498 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca800b7340 0x7fca80204a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.463+0000 7fca87e00640 1 --2- 172.21.15.151:0/2427696498 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca800b5630 0x7fca8015f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.463+0000 7fca87e00640 1 -- 172.21.15.151:0/2427696498 >> 172.21.15.151:0/2427696498 conn(0x7fca800d3fb0 msgr2=0x7fca800b8d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:57.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.463+0000 7fca87e00640 1 -- 172.21.15.151:0/2427696498 shutdown_connections 2024-09-16T15:38:57.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.463+0000 7fca87e00640 1 -- 172.21.15.151:0/2427696498 wait complete. 2024-09-16T15:38:57.474 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi151') 2024-09-16T15:38:57.480 INFO:tasks.ceph:Setting crush tunables to default 2024-09-16T15:38:57.481 DEBUG:teuthology.orchestra.run.smithi151:> sudo ceph --cluster ceph osd crush tunables default 2024-09-16T15:38:57.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.674+0000 7f4337200640 1 -- 172.21.15.151:0/2894940694 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4338161460 msgr2=0x7f43380d22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:57.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.674+0000 7f4337200640 1 --2- 172.21.15.151:0/2894940694 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4338161460 0x7f43380d22f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4328009860 tx=0x7f432802a510 comp rx=0 tx=0).stop 2024-09-16T15:38:57.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.674+0000 7f4337200640 1 -- 172.21.15.151:0/2894940694 shutdown_connections 2024-09-16T15:38:57.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.674+0000 7f4337200640 1 --2- 172.21.15.151:0/2894940694 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4338161460 0x7f43380d22f0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.674+0000 7f4337200640 1 -- 172.21.15.151:0/2894940694 >> 172.21.15.151:0/2894940694 conn(0x7f433809d0b0 msgr2=0x7f43380d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:57.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.674+0000 7f4337200640 1 -- 172.21.15.151:0/2894940694 shutdown_connections 2024-09-16T15:38:57.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.674+0000 7f4337200640 1 -- 172.21.15.151:0/2894940694 wait complete. 2024-09-16T15:38:57.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.676+0000 7f4337200640 1 Processor -- start 2024-09-16T15:38:57.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.676+0000 7f4337200640 1 -- start start 2024-09-16T15:38:57.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.677+0000 7f4337200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f43380b43c0 0x7f433815ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:57.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.677+0000 7f4337200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43380b5480 0x7f433815ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:57.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.677+0000 7f4337200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4338161460 0x7f433815b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:57.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.677+0000 7f4337200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f43380d2bc0 con 0x7f43380b43c0 2024-09-16T15:38:57.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.677+0000 7f4337200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f43380d2a40 con 0x7f43380b5480 2024-09-16T15:38:57.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.677+0000 7f4337200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f43380b51e0 con 0x7f4338161460 2024-09-16T15:38:57.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.677+0000 7f4335e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f43380b43c0 0x7f433815ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:57.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.677+0000 7f4335e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f43380b43c0 0x7f433815ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43924/0 (socket says 172.21.15.151:43924) 2024-09-16T15:38:57.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.677+0000 7f4335e00640 1 -- 172.21.15.151:0/2677715798 learned_addr learned my addr 172.21.15.151:0/2677715798 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:38:57.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.677+0000 7f4335400640 1 --2- 172.21.15.151:0/2677715798 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43380b5480 0x7f433815ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:57.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.677+0000 7f4336800640 1 --2- 172.21.15.151:0/2677715798 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4338161460 0x7f433815b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:57.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.677+0000 7f4335e00640 1 -- 172.21.15.151:0/2677715798 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4338161460 msgr2=0x7f433815b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:57.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.677+0000 7f4335e00640 1 --2- 172.21.15.151:0/2677715798 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4338161460 0x7f433815b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.677+0000 7f4335e00640 1 -- 172.21.15.151:0/2677715798 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43380b5480 msgr2=0x7f433815ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:57.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.677+0000 7f4335e00640 1 --2- 172.21.15.151:0/2677715798 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43380b5480 0x7f433815ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.677+0000 7f4335e00640 1 -- 172.21.15.151:0/2677715798 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43381f5a30 con 0x7f43380b43c0 2024-09-16T15:38:57.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.677+0000 7f4335400640 1 --2- 172.21.15.151:0/2677715798 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43380b5480 0x7f433815ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:38:57.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.678+0000 7f4335e00640 1 --2- 172.21.15.151:0/2677715798 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f43380b43c0 0x7f433815ed30 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4328009980 tx=0x7f4328033ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:57.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.678+0000 7f432e000640 1 -- 172.21.15.151:0/2677715798 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4328031b50 con 0x7f43380b43c0 2024-09-16T15:38:57.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.678+0000 7f432e000640 1 -- 172.21.15.151:0/2677715798 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4328031de0 con 0x7f43380b43c0 2024-09-16T15:38:57.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.678+0000 7f432e000640 1 -- 172.21.15.151:0/2677715798 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43280332d0 con 0x7f43380b43c0 2024-09-16T15:38:57.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.678+0000 7f4337200640 1 -- 172.21.15.151:0/2677715798 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4338158c40 con 0x7f43380b43c0 2024-09-16T15:38:57.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.678+0000 7f4337200640 1 -- 172.21.15.151:0/2677715798 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4338159200 con 0x7f43380b43c0 2024-09-16T15:38:57.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.678+0000 7f432e000640 1 -- 172.21.15.151:0/2677715798 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4328033470 con 0x7f43380b43c0 2024-09-16T15:38:57.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.678+0000 7f432e000640 1 -- 172.21.15.151:0/2677715798 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f432803b410 con 0x7f43380b43c0 2024-09-16T15:38:57.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.679+0000 7f4337200640 1 -- 172.21.15.151:0/2677715798 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f433806d850 con 0x7f43380b43c0 2024-09-16T15:38:57.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.681+0000 7f432e000640 1 -- 172.21.15.151:0/2677715798 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f4328052080 con 0x7f43380b43c0 2024-09-16T15:38:57.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.747+0000 7f4337200640 1 -- 172.21.15.151:0/2677715798 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f433806d570 con 0x7f43380b43c0 2024-09-16T15:38:57.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.749+0000 7f432e000640 1 -- 172.21.15.151:0/2677715798 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f432803f030 con 0x7f43380b43c0 2024-09-16T15:38:57.751 INFO:teuthology.orchestra.run.smithi151.stderr:adjusted tunables profile to default 2024-09-16T15:38:57.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.750+0000 7f4337200640 1 -- 172.21.15.151:0/2677715798 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f43380b43c0 msgr2=0x7f433815ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:57.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.750+0000 7f4337200640 1 --2- 172.21.15.151:0/2677715798 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f43380b43c0 0x7f433815ed30 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4328009980 tx=0x7f4328033ba0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.750+0000 7f4337200640 1 -- 172.21.15.151:0/2677715798 shutdown_connections 2024-09-16T15:38:57.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.750+0000 7f4337200640 1 --2- 172.21.15.151:0/2677715798 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43380b5480 0x7f433815ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.750+0000 7f4337200640 1 --2- 172.21.15.151:0/2677715798 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4338161460 0x7f433815b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.750+0000 7f4337200640 1 --2- 172.21.15.151:0/2677715798 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f43380b43c0 0x7f433815ed30 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:57.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.750+0000 7f4337200640 1 -- 172.21.15.151:0/2677715798 >> 172.21.15.151:0/2677715798 conn(0x7f433809d0b0 msgr2=0x7f43380d07c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:57.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.750+0000 7f4337200640 1 -- 172.21.15.151:0/2677715798 shutdown_connections 2024-09-16T15:38:57.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:38:57.750+0000 7f4337200640 1 -- 172.21.15.151:0/2677715798 wait complete. 2024-09-16T15:38:57.766 INFO:tasks.ceph:check_enable_crimson: False 2024-09-16T15:38:57.767 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-16T15:38:57.767 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:38:57.767 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-16T15:38:57.823 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:38:57.823 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-16T15:38:57.919 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:38:57.919 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-16T15:38:58.015 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:38:58.016 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-16T15:38:58.112 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-16T15:38:58.112 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-16T15:38:58.168 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-16T15:38:58.168 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-16T15:38:58.268 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-16T15:38:58.268 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-16T15:38:58.367 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-16T15:38:58.367 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-16T15:38:58.463 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph --cluster ceph osd new d5524076-a903-4d0a-af6a-caf474c4765d 0 2024-09-16T15:38:58.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.655+0000 7f87ff000640 1 -- 172.21.15.167:0/2415809597 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87f8161630 msgr2=0x7f87f80b37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:58.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.655+0000 7f87ff000640 1 --2- 172.21.15.167:0/2415809597 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87f8161630 0x7f87f80b37b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f87e8009860 tx=0x7f87e802a5a0 comp rx=0 tx=0).stop 2024-09-16T15:38:58.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.655+0000 7f87ff000640 1 -- 172.21.15.167:0/2415809597 shutdown_connections 2024-09-16T15:38:58.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.655+0000 7f87ff000640 1 --2- 172.21.15.167:0/2415809597 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87f8161630 0x7f87f80b37b0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:58.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.655+0000 7f87ff000640 1 -- 172.21.15.167:0/2415809597 >> 172.21.15.167:0/2415809597 conn(0x7f87f80d3fe0 msgr2=0x7f87f80d43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:58.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.656+0000 7f87ff000640 1 -- 172.21.15.167:0/2415809597 shutdown_connections 2024-09-16T15:38:58.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.656+0000 7f87ff000640 1 -- 172.21.15.167:0/2415809597 wait complete. 2024-09-16T15:38:58.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.657+0000 7f87ff000640 1 Processor -- start 2024-09-16T15:38:58.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.657+0000 7f87ff000640 1 -- start start 2024-09-16T15:38:58.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.658+0000 7f87ff000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87f80b2d10 0x7f87f80b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:58.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.658+0000 7f87ff000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87f8161630 0x7f87f80b96f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:58.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.658+0000 7f87ff000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87f8161a10 0x7f87f80c9b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:58.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.658+0000 7f87ff000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f87f80d25f0 con 0x7f87f8161a10 2024-09-16T15:38:58.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.658+0000 7f87ff000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f87f80b3f00 con 0x7f87f80b2d10 2024-09-16T15:38:58.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.658+0000 7f87ff000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f87f80d2770 con 0x7f87f8161630 2024-09-16T15:38:58.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.658+0000 7f87f7e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87f80b2d10 0x7f87f80b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:58.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.658+0000 7f87f7400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87f8161630 0x7f87f80b96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:58.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.658+0000 7f87f7400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87f8161630 0x7f87f80b96f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.167:54670/0 (socket says 172.21.15.167:54670) 2024-09-16T15:38:58.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.658+0000 7f87f7400640 1 -- 172.21.15.167:0/3369312817 learned_addr learned my addr 172.21.15.167:0/3369312817 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-16T15:38:58.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.658+0000 7f87f7e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87f80b2d10 0x7f87f80b7820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:51456/0 (socket says 172.21.15.167:51456) 2024-09-16T15:38:58.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.659+0000 7f87fd000640 1 --2- 172.21.15.167:0/3369312817 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87f8161a10 0x7f87f80c9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:58.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.659+0000 7f87f7400640 1 -- 172.21.15.167:0/3369312817 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87f80b2d10 msgr2=0x7f87f80b7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:58.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.659+0000 7f87f7400640 1 --2- 172.21.15.167:0/3369312817 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87f80b2d10 0x7f87f80b7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:58.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.659+0000 7f87f7400640 1 -- 172.21.15.167:0/3369312817 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87f8161a10 msgr2=0x7f87f80c9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:58.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.659+0000 7f87f7400640 1 --2- 172.21.15.167:0/3369312817 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87f8161a10 0x7f87f80c9b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:58.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.659+0000 7f87f7400640 1 -- 172.21.15.167:0/3369312817 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87f80b7e00 con 0x7f87f8161630 2024-09-16T15:38:58.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.659+0000 7f87f7400640 1 --2- 172.21.15.167:0/3369312817 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87f8161630 0x7f87f80b96f0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f87e400c370 tx=0x7f87e400c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:58.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.659+0000 7f87efe00640 1 -- 172.21.15.167:0/3369312817 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e4017070 con 0x7f87f8161630 2024-09-16T15:38:58.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.659+0000 7f87efe00640 1 -- 172.21.15.167:0/3369312817 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f87e40053c0 con 0x7f87f8161630 2024-09-16T15:38:58.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.659+0000 7f87efe00640 1 -- 172.21.15.167:0/3369312817 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e4003850 con 0x7f87f8161630 2024-09-16T15:38:58.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.660+0000 7f87ff000640 1 -- 172.21.15.167:0/3369312817 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87f80b80e0 con 0x7f87f8161630 2024-09-16T15:38:58.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.660+0000 7f87ff000640 1 -- 172.21.15.167:0/3369312817 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f87f80b86a0 con 0x7f87f8161630 2024-09-16T15:38:58.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.660+0000 7f87efe00640 1 -- 172.21.15.167:0/3369312817 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f87e401a070 con 0x7f87f8161630 2024-09-16T15:38:58.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.660+0000 7f87efe00640 1 -- 172.21.15.167:0/3369312817 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f87e401c440 con 0x7f87f8161630 2024-09-16T15:38:58.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.661+0000 7f87ff000640 1 -- 172.21.15.167:0/3369312817 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87f80d0f10 con 0x7f87f8161630 2024-09-16T15:38:58.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.663+0000 7f87efe00640 1 -- 172.21.15.167:0/3369312817 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f87e4021020 con 0x7f87f8161630 2024-09-16T15:38:58.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.736+0000 7f87ff000640 1 -- 172.21.15.167:0/3369312817 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "osd new", "uuid": "d5524076-a903-4d0a-af6a-caf474c4765d", "id": 0} v 0) -- 0x7f87f80b63d0 con 0x7f87f8161630 2024-09-16T15:38:58.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.741+0000 7f87efe00640 1 -- 172.21.15.167:0/3369312817 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d5524076-a903-4d0a-af6a-caf474c4765d", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f87e4011590 con 0x7f87f8161630 2024-09-16T15:38:58.743 INFO:teuthology.orchestra.run.smithi167.stdout:0 2024-09-16T15:38:58.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.743+0000 7f87ff000640 1 -- 172.21.15.167:0/3369312817 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87f8161630 msgr2=0x7f87f80b96f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:58.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.743+0000 7f87ff000640 1 --2- 172.21.15.167:0/3369312817 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87f8161630 0x7f87f80b96f0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f87e400c370 tx=0x7f87e400c830 comp rx=0 tx=0).stop 2024-09-16T15:38:58.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.743+0000 7f87ff000640 1 -- 172.21.15.167:0/3369312817 shutdown_connections 2024-09-16T15:38:58.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.743+0000 7f87ff000640 1 --2- 172.21.15.167:0/3369312817 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87f8161a10 0x7f87f80c9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:58.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.743+0000 7f87ff000640 1 --2- 172.21.15.167:0/3369312817 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f87f8161630 0x7f87f80b96f0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:58.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.743+0000 7f87ff000640 1 --2- 172.21.15.167:0/3369312817 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87f80b2d10 0x7f87f80b7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:58.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.743+0000 7f87ff000640 1 -- 172.21.15.167:0/3369312817 >> 172.21.15.167:0/3369312817 conn(0x7f87f80d3fe0 msgr2=0x7f87f80d1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:58.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.743+0000 7f87ff000640 1 -- 172.21.15.167:0/3369312817 shutdown_connections 2024-09-16T15:38:58.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.743+0000 7f87ff000640 1 -- 172.21.15.167:0/3369312817 wait complete. 2024-09-16T15:38:58.760 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph --cluster ceph osd new dd25925f-0ae1-494e-8529-4dc205759db3 1 2024-09-16T15:38:58.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.914+0000 7f897ee00640 1 -- 172.21.15.167:0/3120625679 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89780d2400 msgr2=0x7f89780cf780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:58.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.914+0000 7f897ee00640 1 --2- 172.21.15.167:0/3120625679 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89780d2400 0x7f89780cf780 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f896c009830 tx=0x7f896c02a5c0 comp rx=0 tx=0).stop 2024-09-16T15:38:58.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.914+0000 7f897ee00640 1 -- 172.21.15.167:0/3120625679 shutdown_connections 2024-09-16T15:38:58.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.914+0000 7f897ee00640 1 --2- 172.21.15.167:0/3120625679 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89780d2400 0x7f89780cf780 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:58.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.914+0000 7f897ee00640 1 -- 172.21.15.167:0/3120625679 >> 172.21.15.167:0/3120625679 conn(0x7f897809d040 msgr2=0x7f89780d3dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:58.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.914+0000 7f897ee00640 1 -- 172.21.15.167:0/3120625679 shutdown_connections 2024-09-16T15:38:58.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.914+0000 7f897ee00640 1 -- 172.21.15.167:0/3120625679 wait complete. 2024-09-16T15:38:58.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.916+0000 7f897ee00640 1 Processor -- start 2024-09-16T15:38:58.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.916+0000 7f897ee00640 1 -- start start 2024-09-16T15:38:58.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.917+0000 7f897ee00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89780b5290 0x7f89781f5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:58.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.917+0000 7f897ee00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89780d2400 0x7f89781605f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:58.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.917+0000 7f897ee00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8978161070 0x7f89780cae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:58.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.917+0000 7f897ee00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f89780b38d0 con 0x7f89780b5290 2024-09-16T15:38:58.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.917+0000 7f897ee00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f89780d2b00 con 0x7f89780d2400 2024-09-16T15:38:58.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.917+0000 7f897ee00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f89780b3a50 con 0x7f8978161070 2024-09-16T15:38:58.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.917+0000 7f897d000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89780d2400 0x7f89781605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:58.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.917+0000 7f897d000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89780d2400 0x7f89781605f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:51466/0 (socket says 172.21.15.167:51466) 2024-09-16T15:38:58.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.917+0000 7f897d000640 1 -- 172.21.15.167:0/733692256 learned_addr learned my addr 172.21.15.167:0/733692256 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-16T15:38:58.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.917+0000 7f897e400640 1 --2- 172.21.15.167:0/733692256 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8978161070 0x7f89780cae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:58.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.917+0000 7f897da00640 1 --2- 172.21.15.167:0/733692256 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89780b5290 0x7f89781f5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:58.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.917+0000 7f897d000640 1 -- 172.21.15.167:0/733692256 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8978161070 msgr2=0x7f89780cae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:58.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.917+0000 7f897d000640 1 --2- 172.21.15.167:0/733692256 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8978161070 0x7f89780cae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:58.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.917+0000 7f897d000640 1 -- 172.21.15.167:0/733692256 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89780b5290 msgr2=0x7f89781f5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:58.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.917+0000 7f897d000640 1 --2- 172.21.15.167:0/733692256 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89780b5290 0x7f89781f5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:58.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.917+0000 7f897d000640 1 -- 172.21.15.167:0/733692256 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89780cc800 con 0x7f89780d2400 2024-09-16T15:38:58.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.918+0000 7f897d000640 1 --2- 172.21.15.167:0/733692256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89780d2400 0x7f89781605f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f896400c490 tx=0x7f896400c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:58.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.918+0000 7f8975600640 1 -- 172.21.15.167:0/733692256 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89640052d0 con 0x7f89780d2400 2024-09-16T15:38:58.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.918+0000 7f8975600640 1 -- 172.21.15.167:0/733692256 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8964005490 con 0x7f89780d2400 2024-09-16T15:38:58.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.918+0000 7f8975600640 1 -- 172.21.15.167:0/733692256 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8964012890 con 0x7f89780d2400 2024-09-16T15:38:58.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.918+0000 7f897ee00640 1 -- 172.21.15.167:0/733692256 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89780ccac0 con 0x7f89780d2400 2024-09-16T15:38:58.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.918+0000 7f8975600640 1 -- 172.21.15.167:0/733692256 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8964012a30 con 0x7f89780d2400 2024-09-16T15:38:58.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.919+0000 7f897ee00640 1 -- 172.21.15.167:0/733692256 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89780cd080 con 0x7f89780d2400 2024-09-16T15:38:58.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.919+0000 7f8975600640 1 -- 172.21.15.167:0/733692256 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f8964003650 con 0x7f89780d2400 2024-09-16T15:38:58.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.920+0000 7f897ee00640 1 -- 172.21.15.167:0/733692256 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89780d2f70 con 0x7f89780d2400 2024-09-16T15:38:58.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.922+0000 7f8975600640 1 -- 172.21.15.167:0/733692256 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f8964020020 con 0x7f89780d2400 2024-09-16T15:38:58.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.995+0000 7f897ee00640 1 -- 172.21.15.167:0/733692256 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd new", "uuid": "dd25925f-0ae1-494e-8529-4dc205759db3", "id": 1} v 0) -- 0x7f89780cf760 con 0x7f89780d2400 2024-09-16T15:38:59.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:58.999+0000 7f8975600640 1 -- 172.21.15.167:0/733692256 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "dd25925f-0ae1-494e-8529-4dc205759db3", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f8964003920 con 0x7f89780d2400 2024-09-16T15:38:59.001 INFO:teuthology.orchestra.run.smithi167.stdout:1 2024-09-16T15:38:59.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.000+0000 7f897ee00640 1 -- 172.21.15.167:0/733692256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89780d2400 msgr2=0x7f89781605f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:59.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.000+0000 7f897ee00640 1 --2- 172.21.15.167:0/733692256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89780d2400 0x7f89781605f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f896400c490 tx=0x7f896400c950 comp rx=0 tx=0).stop 2024-09-16T15:38:59.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.000+0000 7f897ee00640 1 -- 172.21.15.167:0/733692256 shutdown_connections 2024-09-16T15:38:59.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.000+0000 7f897ee00640 1 --2- 172.21.15.167:0/733692256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89780d2400 0x7f89781605f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:59.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.000+0000 7f897ee00640 1 --2- 172.21.15.167:0/733692256 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8978161070 0x7f89780cae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:59.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.000+0000 7f897ee00640 1 --2- 172.21.15.167:0/733692256 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89780b5290 0x7f89781f5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:59.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.000+0000 7f897ee00640 1 -- 172.21.15.167:0/733692256 >> 172.21.15.167:0/733692256 conn(0x7f897809d040 msgr2=0x7f897806ea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:59.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.000+0000 7f897ee00640 1 -- 172.21.15.167:0/733692256 shutdown_connections 2024-09-16T15:38:59.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.000+0000 7f897ee00640 1 -- 172.21.15.167:0/733692256 wait complete. 2024-09-16T15:38:59.017 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph --cluster ceph osd new c2f18115-013d-4da6-a0fe-cb978f4494b3 2 2024-09-16T15:38:59.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.170+0000 7f53bb200640 1 -- 172.21.15.167:0/4261924143 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f53bc0cf830 msgr2=0x7f53bc0b33b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:59.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.170+0000 7f53bb200640 1 --2- 172.21.15.167:0/4261924143 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f53bc0cf830 0x7f53bc0b33b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f53a8009860 tx=0x7f53a802a4b0 comp rx=0 tx=0).stop 2024-09-16T15:38:59.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.170+0000 7f53bb200640 1 -- 172.21.15.167:0/4261924143 shutdown_connections 2024-09-16T15:38:59.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.170+0000 7f53bb200640 1 --2- 172.21.15.167:0/4261924143 >> v2:172.21.15.151:3300/0 conn(0x7f53bc0d2a00 0x7f53bc0b38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:59.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.170+0000 7f53bb200640 1 --2- 172.21.15.167:0/4261924143 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f53bc0cf830 0x7f53bc0b33b0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:59.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.170+0000 7f53bb200640 1 -- 172.21.15.167:0/4261924143 >> 172.21.15.167:0/4261924143 conn(0x7f53bc09d0b0 msgr2=0x7f53bc0d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:59.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.170+0000 7f53bb200640 1 -- 172.21.15.167:0/4261924143 shutdown_connections 2024-09-16T15:38:59.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.170+0000 7f53bb200640 1 -- 172.21.15.167:0/4261924143 wait complete. 2024-09-16T15:38:59.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.171+0000 7f53bb200640 1 Processor -- start 2024-09-16T15:38:59.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.172+0000 7f53bb200640 1 -- start start 2024-09-16T15:38:59.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.172+0000 7f53bb200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53bc0cf830 0x7f53bc156f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:59.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.172+0000 7f53bb200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53bc0d2a00 0x7f53bc1f8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:59.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.173+0000 7f53bb200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f53bc161460 0x7f53bc0b2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:59.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.173+0000 7f53bb200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f53bc0b56d0 con 0x7f53bc0cf830 2024-09-16T15:38:59.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.173+0000 7f53bb200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f53bc0b5150 con 0x7f53bc0d2a00 2024-09-16T15:38:59.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.173+0000 7f53bb200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f53bc0b3e30 con 0x7f53bc161460 2024-09-16T15:38:59.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.173+0000 7f53b9e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53bc0cf830 0x7f53bc156f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:59.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.173+0000 7f53b9400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53bc0d2a00 0x7f53bc1f8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:59.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.173+0000 7f53ba800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f53bc161460 0x7f53bc0b2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:59.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.173+0000 7f53b9400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53bc0d2a00 0x7f53bc1f8360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:51482/0 (socket says 172.21.15.167:51482) 2024-09-16T15:38:59.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.173+0000 7f53b9e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53bc0cf830 0x7f53bc156f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.167:48824/0 (socket says 172.21.15.167:48824) 2024-09-16T15:38:59.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.173+0000 7f53b9400640 1 -- 172.21.15.167:0/3884058411 learned_addr learned my addr 172.21.15.167:0/3884058411 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-16T15:38:59.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.173+0000 7f53ba800640 1 -- 172.21.15.167:0/3884058411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53bc0d2a00 msgr2=0x7f53bc1f8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:59.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.173+0000 7f53ba800640 1 --2- 172.21.15.167:0/3884058411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53bc0d2a00 0x7f53bc1f8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:59.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.173+0000 7f53ba800640 1 -- 172.21.15.167:0/3884058411 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53bc0cf830 msgr2=0x7f53bc156f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:59.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.173+0000 7f53ba800640 1 --2- 172.21.15.167:0/3884058411 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53bc0cf830 0x7f53bc156f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:59.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.173+0000 7f53ba800640 1 -- 172.21.15.167:0/3884058411 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53bc15abf0 con 0x7f53bc161460 2024-09-16T15:38:59.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.174+0000 7f53ba800640 1 --2- 172.21.15.167:0/3884058411 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f53bc161460 0x7f53bc0b2240 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f53bc08d200 tx=0x7f53b400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:59.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.174+0000 7f53b2000640 1 -- 172.21.15.167:0/3884058411 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b400cdf0 con 0x7f53bc161460 2024-09-16T15:38:59.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.174+0000 7f53b2000640 1 -- 172.21.15.167:0/3884058411 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f53b4004c10 con 0x7f53bc161460 2024-09-16T15:38:59.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.174+0000 7f53b2000640 1 -- 172.21.15.167:0/3884058411 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b4005020 con 0x7f53bc161460 2024-09-16T15:38:59.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.174+0000 7f53bb200640 1 -- 172.21.15.167:0/3884058411 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53bc15ae90 con 0x7f53bc161460 2024-09-16T15:38:59.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.174+0000 7f53bb200640 1 -- 172.21.15.167:0/3884058411 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f53bc1f7e40 con 0x7f53bc161460 2024-09-16T15:38:59.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.174+0000 7f53b2000640 1 -- 172.21.15.167:0/3884058411 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f53b4005550 con 0x7f53bc161460 2024-09-16T15:38:59.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.174+0000 7f53b2000640 1 -- 172.21.15.167:0/3884058411 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f53b4018470 con 0x7f53bc161460 2024-09-16T15:38:59.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.175+0000 7f53bb200640 1 -- 172.21.15.167:0/3884058411 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53bc0b42f0 con 0x7f53bc161460 2024-09-16T15:38:59.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.177+0000 7f53b2000640 1 -- 172.21.15.167:0/3884058411 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f53b4005840 con 0x7f53bc161460 2024-09-16T15:38:59.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.249+0000 7f53bb200640 1 -- 172.21.15.167:0/3884058411 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "osd new", "uuid": "c2f18115-013d-4da6-a0fe-cb978f4494b3", "id": 2} v 0) -- 0x7f53bc1f8010 con 0x7f53bc161460 2024-09-16T15:38:59.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.253+0000 7f53b2000640 1 -- 172.21.15.167:0/3884058411 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c2f18115-013d-4da6-a0fe-cb978f4494b3", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f53b4018790 con 0x7f53bc161460 2024-09-16T15:38:59.255 INFO:teuthology.orchestra.run.smithi167.stdout:2 2024-09-16T15:38:59.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.255+0000 7f53bb200640 1 -- 172.21.15.167:0/3884058411 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f53bc161460 msgr2=0x7f53bc0b2240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:59.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.255+0000 7f53bb200640 1 --2- 172.21.15.167:0/3884058411 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f53bc161460 0x7f53bc0b2240 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f53bc08d200 tx=0x7f53b400ef10 comp rx=0 tx=0).stop 2024-09-16T15:38:59.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.255+0000 7f53bb200640 1 -- 172.21.15.167:0/3884058411 shutdown_connections 2024-09-16T15:38:59.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.255+0000 7f53bb200640 1 --2- 172.21.15.167:0/3884058411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53bc0d2a00 0x7f53bc1f8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:59.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.255+0000 7f53bb200640 1 --2- 172.21.15.167:0/3884058411 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f53bc161460 0x7f53bc0b2240 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:59.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.255+0000 7f53bb200640 1 --2- 172.21.15.167:0/3884058411 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53bc0cf830 0x7f53bc156f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:59.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.255+0000 7f53bb200640 1 -- 172.21.15.167:0/3884058411 >> 172.21.15.167:0/3884058411 conn(0x7f53bc09d0b0 msgr2=0x7f53bc0b3180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:59.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.255+0000 7f53bb200640 1 -- 172.21.15.167:0/3884058411 shutdown_connections 2024-09-16T15:38:59.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.255+0000 7f53bb200640 1 -- 172.21.15.167:0/3884058411 wait complete. 2024-09-16T15:38:59.270 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph --cluster ceph osd new 36edb718-2fcb-4d88-8ddc-c321eed4279d 3 2024-09-16T15:38:59.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.424+0000 7f3852200640 1 -- 172.21.15.167:0/3052553417 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f384c0cf830 msgr2=0x7f384c0b33b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:59.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.424+0000 7f3852200640 1 --2- 172.21.15.167:0/3052553417 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f384c0cf830 0x7f384c0b33b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3838009830 tx=0x7f383802a530 comp rx=0 tx=0).stop 2024-09-16T15:38:59.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.424+0000 7f3852200640 1 -- 172.21.15.167:0/3052553417 shutdown_connections 2024-09-16T15:38:59.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.424+0000 7f3852200640 1 --2- 172.21.15.167:0/3052553417 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f384c0cf830 0x7f384c0b33b0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:59.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.424+0000 7f3852200640 1 -- 172.21.15.167:0/3052553417 >> 172.21.15.167:0/3052553417 conn(0x7f384c09d0b0 msgr2=0x7f384c0d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:59.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.424+0000 7f3852200640 1 -- 172.21.15.167:0/3052553417 shutdown_connections 2024-09-16T15:38:59.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.424+0000 7f3852200640 1 -- 172.21.15.167:0/3052553417 wait complete. 2024-09-16T15:38:59.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.426+0000 7f3852200640 1 Processor -- start 2024-09-16T15:38:59.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.426+0000 7f3852200640 1 -- start start 2024-09-16T15:38:59.430 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.427+0000 7f3852200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f384c0cf830 0x7f384c0c81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:59.430 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.427+0000 7f3852200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f384c0d2a00 0x7f384c0ca250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:59.430 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.427+0000 7f3852200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f384c161460 0x7f384c0ca790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:59.430 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.427+0000 7f3852200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f384c0b50c0 con 0x7f384c161460 2024-09-16T15:38:59.430 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.427+0000 7f3852200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f384c0b4f40 con 0x7f384c0d2a00 2024-09-16T15:38:59.430 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.427+0000 7f3852200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f384c0b4000 con 0x7f384c0cf830 2024-09-16T15:38:59.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.427+0000 7f3850e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f384c0cf830 0x7f384c0c81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:59.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.427+0000 7f384be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f384c0d2a00 0x7f384c0ca250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:59.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.427+0000 7f384be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f384c0d2a00 0x7f384c0ca250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:51492/0 (socket says 172.21.15.167:51492) 2024-09-16T15:38:59.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.427+0000 7f3850e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f384c0cf830 0x7f384c0c81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.167:54696/0 (socket says 172.21.15.167:54696) 2024-09-16T15:38:59.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.427+0000 7f384be00640 1 -- 172.21.15.167:0/3513718516 learned_addr learned my addr 172.21.15.167:0/3513718516 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-16T15:38:59.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.427+0000 7f3851800640 1 --2- 172.21.15.167:0/3513718516 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f384c161460 0x7f384c0ca790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:59.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.428+0000 7f3850e00640 1 -- 172.21.15.167:0/3513718516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f384c0d2a00 msgr2=0x7f384c0ca250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:59.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.428+0000 7f3850e00640 1 --2- 172.21.15.167:0/3513718516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f384c0d2a00 0x7f384c0ca250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:59.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.428+0000 7f3850e00640 1 -- 172.21.15.167:0/3513718516 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f384c161460 msgr2=0x7f384c0ca790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:59.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.428+0000 7f3850e00640 1 --2- 172.21.15.167:0/3513718516 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f384c161460 0x7f384c0ca790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:59.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.428+0000 7f3850e00640 1 -- 172.21.15.167:0/3513718516 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f384c0b2010 con 0x7f384c0cf830 2024-09-16T15:38:59.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.428+0000 7f3850e00640 1 --2- 172.21.15.167:0/3513718516 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f384c0cf830 0x7f384c0c81f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f384000c3c0 tx=0x7f384000c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:59.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.428+0000 7f3848c00640 1 -- 172.21.15.167:0/3513718516 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3840017070 con 0x7f384c0cf830 2024-09-16T15:38:59.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.428+0000 7f3848c00640 1 -- 172.21.15.167:0/3513718516 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f38400053c0 con 0x7f384c0cf830 2024-09-16T15:38:59.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.428+0000 7f3852200640 1 -- 172.21.15.167:0/3513718516 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f384c0b22f0 con 0x7f384c0cf830 2024-09-16T15:38:59.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.428+0000 7f3848c00640 1 -- 172.21.15.167:0/3513718516 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3840003850 con 0x7f384c0cf830 2024-09-16T15:38:59.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.429+0000 7f3852200640 1 -- 172.21.15.167:0/3513718516 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f384c0b28b0 con 0x7f384c0cf830 2024-09-16T15:38:59.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.429+0000 7f3848c00640 1 -- 172.21.15.167:0/3513718516 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f384001a070 con 0x7f384c0cf830 2024-09-16T15:38:59.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.429+0000 7f3848c00640 1 -- 172.21.15.167:0/3513718516 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f38400039f0 con 0x7f384c0cf830 2024-09-16T15:38:59.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.430+0000 7f3852200640 1 -- 172.21.15.167:0/3513718516 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f384c0b2c90 con 0x7f384c0cf830 2024-09-16T15:38:59.433 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.432+0000 7f3848c00640 1 -- 172.21.15.167:0/3513718516 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f384c0b2c90 con 0x7f384c0cf830 2024-09-16T15:38:59.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.503+0000 7f3852200640 1 -- 172.21.15.167:0/3513718516 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "osd new", "uuid": "36edb718-2fcb-4d88-8ddc-c321eed4279d", "id": 3} v 0) -- 0x7f384c0b4500 con 0x7f384c0cf830 2024-09-16T15:38:59.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.507+0000 7f3848c00640 1 -- 172.21.15.167:0/3513718516 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "36edb718-2fcb-4d88-8ddc-c321eed4279d", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f3840011df0 con 0x7f384c0cf830 2024-09-16T15:38:59.509 INFO:teuthology.orchestra.run.smithi167.stdout:3 2024-09-16T15:38:59.510 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.508+0000 7f3852200640 1 -- 172.21.15.167:0/3513718516 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f384c0cf830 msgr2=0x7f384c0c81f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:59.510 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.508+0000 7f3852200640 1 --2- 172.21.15.167:0/3513718516 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f384c0cf830 0x7f384c0c81f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f384000c3c0 tx=0x7f384000c880 comp rx=0 tx=0).stop 2024-09-16T15:38:59.510 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.509+0000 7f3852200640 1 -- 172.21.15.167:0/3513718516 shutdown_connections 2024-09-16T15:38:59.510 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.509+0000 7f3852200640 1 --2- 172.21.15.167:0/3513718516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f384c0d2a00 0x7f384c0ca250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:59.510 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.509+0000 7f3852200640 1 --2- 172.21.15.167:0/3513718516 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f384c161460 0x7f384c0ca790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:59.510 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.509+0000 7f3852200640 1 --2- 172.21.15.167:0/3513718516 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f384c0cf830 0x7f384c0c81f0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:59.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.509+0000 7f3852200640 1 -- 172.21.15.167:0/3513718516 >> 172.21.15.167:0/3513718516 conn(0x7f384c09d0b0 msgr2=0x7f384c1606b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:59.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.509+0000 7f3852200640 1 -- 172.21.15.167:0/3513718516 shutdown_connections 2024-09-16T15:38:59.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.509+0000 7f3852200640 1 -- 172.21.15.167:0/3513718516 wait complete. 2024-09-16T15:38:59.524 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph --cluster ceph osd new 64d33708-cd0a-4236-abe3-1b646e82ab54 4 2024-09-16T15:38:59.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.676+0000 7fd5ce400640 1 -- 172.21.15.167:0/3516079991 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5c8161460 msgr2=0x7fd5c80d22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:59.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.676+0000 7fd5ce400640 1 --2- 172.21.15.167:0/3516079991 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5c8161460 0x7fd5c80d22f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc005fd0 tx=0x7fd5bc02a540 comp rx=0 tx=0).stop 2024-09-16T15:38:59.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.676+0000 7fd5ce400640 1 -- 172.21.15.167:0/3516079991 shutdown_connections 2024-09-16T15:38:59.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.676+0000 7fd5ce400640 1 --2- 172.21.15.167:0/3516079991 >> v2:172.21.15.151:3300/0 conn(0x7fd5c80b43c0 0x7fd5c80d2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:59.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.676+0000 7fd5ce400640 1 --2- 172.21.15.167:0/3516079991 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5c8161460 0x7fd5c80d22f0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:59.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.676+0000 7fd5ce400640 1 -- 172.21.15.167:0/3516079991 >> 172.21.15.167:0/3516079991 conn(0x7fd5c809d0b0 msgr2=0x7fd5c80d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:59.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.676+0000 7fd5ce400640 1 -- 172.21.15.167:0/3516079991 shutdown_connections 2024-09-16T15:38:59.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.676+0000 7fd5ce400640 1 -- 172.21.15.167:0/3516079991 wait complete. 2024-09-16T15:38:59.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.678+0000 7fd5ce400640 1 Processor -- start 2024-09-16T15:38:59.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.678+0000 7fd5ce400640 1 -- start start 2024-09-16T15:38:59.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.678+0000 7fd5ce400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5c80b43c0 0x7fd5c815dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:59.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.678+0000 7fd5ce400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5c80b5480 0x7fd5c81572c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:59.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.679+0000 7fd5ce400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5c8161460 0x7fd5c81f50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:59.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.679+0000 7fd5ce400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c80cf0b0 con 0x7fd5c8161460 2024-09-16T15:38:59.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.679+0000 7fd5ce400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c80cef30 con 0x7fd5c80b43c0 2024-09-16T15:38:59.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.679+0000 7fd5ce400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fd5c80cf230 con 0x7fd5c80b5480 2024-09-16T15:38:59.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.679+0000 7fd5c7e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5c80b5480 0x7fd5c81572c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:59.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.679+0000 7fd5cd000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5c80b43c0 0x7fd5c815dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:59.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.679+0000 7fd5cd000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5c80b43c0 0x7fd5c815dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:51498/0 (socket says 172.21.15.167:51498) 2024-09-16T15:38:59.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.679+0000 7fd5cd000640 1 -- 172.21.15.167:0/801826104 learned_addr learned my addr 172.21.15.167:0/801826104 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-16T15:38:59.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.679+0000 7fd5cda00640 1 --2- 172.21.15.167:0/801826104 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5c8161460 0x7fd5c81f50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:59.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.679+0000 7fd5cd000640 1 -- 172.21.15.167:0/801826104 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5c80b5480 msgr2=0x7fd5c81572c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:59.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.679+0000 7fd5cd000640 1 --2- 172.21.15.167:0/801826104 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5c80b5480 0x7fd5c81572c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:59.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.679+0000 7fd5cd000640 1 -- 172.21.15.167:0/801826104 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5c8161460 msgr2=0x7fd5c81f50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:59.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.679+0000 7fd5cd000640 1 --2- 172.21.15.167:0/801826104 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5c8161460 0x7fd5c81f50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:59.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.679+0000 7fd5cd000640 1 -- 172.21.15.167:0/801826104 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5c8157800 con 0x7fd5c80b43c0 2024-09-16T15:38:59.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.679+0000 7fd5c7e00640 1 --2- 172.21.15.167:0/801826104 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5c80b5480 0x7fd5c81572c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:38:59.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.680+0000 7fd5cd000640 1 --2- 172.21.15.167:0/801826104 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5c80b43c0 0x7fd5c815dec0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc034040 tx=0x7fd5bc0038a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:59.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.680+0000 7fd5c4c00640 1 -- 172.21.15.167:0/801826104 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5bc03e030 con 0x7fd5c80b43c0 2024-09-16T15:38:59.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.680+0000 7fd5c4c00640 1 -- 172.21.15.167:0/801826104 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd5bc003d90 con 0x7fd5c80b43c0 2024-09-16T15:38:59.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.680+0000 7fd5c4c00640 1 -- 172.21.15.167:0/801826104 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5bc035040 con 0x7fd5c80b43c0 2024-09-16T15:38:59.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.680+0000 7fd5ce400640 1 -- 172.21.15.167:0/801826104 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5c815e4c0 con 0x7fd5c80b43c0 2024-09-16T15:38:59.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.680+0000 7fd5ce400640 1 -- 172.21.15.167:0/801826104 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5c81f48c0 con 0x7fd5c80b43c0 2024-09-16T15:38:59.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.680+0000 7fd5c4c00640 1 -- 172.21.15.167:0/801826104 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd5bc041030 con 0x7fd5c80b43c0 2024-09-16T15:38:59.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.680+0000 7fd5c4c00640 1 -- 172.21.15.167:0/801826104 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7fd5bc043c90 con 0x7fd5c80b43c0 2024-09-16T15:38:59.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.681+0000 7fd5ce400640 1 -- 172.21.15.167:0/801826104 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5c81f46d0 con 0x7fd5c80b43c0 2024-09-16T15:38:59.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.683+0000 7fd5c4c00640 1 -- 172.21.15.167:0/801826104 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fd5bc043780 con 0x7fd5c80b43c0 2024-09-16T15:38:59.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.757+0000 7fd5ce400640 1 -- 172.21.15.167:0/801826104 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd new", "uuid": "64d33708-cd0a-4236-abe3-1b646e82ab54", "id": 4} v 0) -- 0x7fd5c80cf750 con 0x7fd5c80b43c0 2024-09-16T15:38:59.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.760+0000 7fd5c4c00640 1 -- 172.21.15.167:0/801826104 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "64d33708-cd0a-4236-abe3-1b646e82ab54", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7fd5bc043960 con 0x7fd5c80b43c0 2024-09-16T15:38:59.762 INFO:teuthology.orchestra.run.smithi167.stdout:4 2024-09-16T15:38:59.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.762+0000 7fd5ce400640 1 -- 172.21.15.167:0/801826104 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5c80b43c0 msgr2=0x7fd5c815dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:59.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.762+0000 7fd5ce400640 1 --2- 172.21.15.167:0/801826104 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5c80b43c0 0x7fd5c815dec0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc034040 tx=0x7fd5bc0038a0 comp rx=0 tx=0).stop 2024-09-16T15:38:59.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.762+0000 7fd5ce400640 1 -- 172.21.15.167:0/801826104 shutdown_connections 2024-09-16T15:38:59.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.762+0000 7fd5ce400640 1 --2- 172.21.15.167:0/801826104 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5c8161460 0x7fd5c81f50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:59.764 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.762+0000 7fd5ce400640 1 --2- 172.21.15.167:0/801826104 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5c80b5480 0x7fd5c81572c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:59.764 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.762+0000 7fd5ce400640 1 --2- 172.21.15.167:0/801826104 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5c80b43c0 0x7fd5c815dec0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:59.764 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.762+0000 7fd5ce400640 1 -- 172.21.15.167:0/801826104 >> 172.21.15.167:0/801826104 conn(0x7fd5c809d0b0 msgr2=0x7fd5c80d20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:59.764 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.762+0000 7fd5ce400640 1 -- 172.21.15.167:0/801826104 shutdown_connections 2024-09-16T15:38:59.764 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.762+0000 7fd5ce400640 1 -- 172.21.15.167:0/801826104 wait complete. 2024-09-16T15:38:59.777 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph --cluster ceph osd new f14b44a9-5910-458c-b518-09a8d0de94de 5 2024-09-16T15:38:59.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.930+0000 7f1500a00640 1 -- 172.21.15.167:0/3865202408 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14fc0d2a00 msgr2=0x7f14fc0b3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:59.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.930+0000 7f1500a00640 1 --2- 172.21.15.167:0/3865202408 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14fc0d2a00 0x7f14fc0b3ac0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f14f4009be0 tx=0x7f14f402a890 comp rx=0 tx=0).stop 2024-09-16T15:38:59.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.930+0000 7f1500a00640 1 -- 172.21.15.167:0/3865202408 shutdown_connections 2024-09-16T15:38:59.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.930+0000 7f1500a00640 1 --2- 172.21.15.167:0/3865202408 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14fc0d2a00 0x7f14fc0b3ac0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:59.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.930+0000 7f1500a00640 1 -- 172.21.15.167:0/3865202408 >> 172.21.15.167:0/3865202408 conn(0x7f14fc09d0b0 msgr2=0x7f14fc0d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:38:59.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.931+0000 7f1500a00640 1 -- 172.21.15.167:0/3865202408 shutdown_connections 2024-09-16T15:38:59.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.931+0000 7f1500a00640 1 -- 172.21.15.167:0/3865202408 wait complete. 2024-09-16T15:38:59.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.932+0000 7f1500a00640 1 Processor -- start 2024-09-16T15:38:59.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.932+0000 7f1500a00640 1 -- start start 2024-09-16T15:38:59.936 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.933+0000 7f1500a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14fc0cf830 0x7f14fc0b8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:59.936 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.933+0000 7f1500a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14fc0d2a00 0x7f14fc156c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:59.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.933+0000 7f1500a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14fc161460 0x7f14fc0c9330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:38:59.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.933+0000 7f1500a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc0b56c0 con 0x7f14fc161460 2024-09-16T15:38:59.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.933+0000 7f1500a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc0b5150 con 0x7f14fc0cf830 2024-09-16T15:38:59.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.933+0000 7f1500a00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f14fc0b4000 con 0x7f14fc0d2a00 2024-09-16T15:38:59.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.934+0000 7f14fa800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14fc0cf830 0x7f14fc0b8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:59.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.934+0000 7f14fa800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14fc0cf830 0x7f14fc0b8200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:51510/0 (socket says 172.21.15.167:51510) 2024-09-16T15:38:59.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.934+0000 7f14fb200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14fc161460 0x7f14fc0c9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:59.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.934+0000 7f14fa800640 1 -- 172.21.15.167:0/3390967544 learned_addr learned my addr 172.21.15.167:0/3390967544 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-16T15:38:59.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.934+0000 7f14f3e00640 1 --2- 172.21.15.167:0/3390967544 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14fc0d2a00 0x7f14fc156c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:38:59.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.934+0000 7f14fa800640 1 -- 172.21.15.167:0/3390967544 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14fc0d2a00 msgr2=0x7f14fc156c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:59.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.934+0000 7f14fa800640 1 --2- 172.21.15.167:0/3390967544 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14fc0d2a00 0x7f14fc156c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:59.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.934+0000 7f14fa800640 1 -- 172.21.15.167:0/3390967544 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14fc161460 msgr2=0x7f14fc0c9330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:38:59.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.934+0000 7f14fa800640 1 --2- 172.21.15.167:0/3390967544 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14fc161460 0x7f14fc0c9330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:38:59.938 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.934+0000 7f14fa800640 1 -- 172.21.15.167:0/3390967544 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14fc0b78c0 con 0x7f14fc0cf830 2024-09-16T15:38:59.938 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.934+0000 7f14f3e00640 1 --2- 172.21.15.167:0/3390967544 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14fc0d2a00 0x7f14fc156c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:38:59.938 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.934+0000 7f14fa800640 1 --2- 172.21.15.167:0/3390967544 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14fc0cf830 0x7f14fc0b8200 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f14ec00c400 tx=0x7f14ec00c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:38:59.938 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.935+0000 7f14f2a00640 1 -- 172.21.15.167:0/3390967544 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ec017070 con 0x7f14fc0cf830 2024-09-16T15:38:59.938 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.935+0000 7f14f2a00640 1 -- 172.21.15.167:0/3390967544 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f14ec0053c0 con 0x7f14fc0cf830 2024-09-16T15:38:59.938 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.935+0000 7f1500a00640 1 -- 172.21.15.167:0/3390967544 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14fc0b7ba0 con 0x7f14fc0cf830 2024-09-16T15:38:59.938 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.935+0000 7f14f2a00640 1 -- 172.21.15.167:0/3390967544 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ec003850 con 0x7f14fc0cf830 2024-09-16T15:38:59.938 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.935+0000 7f1500a00640 1 -- 172.21.15.167:0/3390967544 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14fc0b8740 con 0x7f14fc0cf830 2024-09-16T15:38:59.938 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.935+0000 7f14f2a00640 1 -- 172.21.15.167:0/3390967544 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f14ec01a070 con 0x7f14fc0cf830 2024-09-16T15:38:59.938 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.935+0000 7f14f2a00640 1 -- 172.21.15.167:0/3390967544 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f14ec011c70 con 0x7f14fc0cf830 2024-09-16T15:38:59.938 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.936+0000 7f1500a00640 1 -- 172.21.15.167:0/3390967544 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14fc0b4520 con 0x7f14fc0cf830 2024-09-16T15:38:59.941 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:38:59.940+0000 7f14f2a00640 1 -- 172.21.15.167:0/3390967544 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f14ec021020 con 0x7f14fc0cf830 2024-09-16T15:39:00.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.013+0000 7f1500a00640 1 -- 172.21.15.167:0/3390967544 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f14b44a9-5910-458c-b518-09a8d0de94de", "id": 5} v 0) -- 0x7f14fc0cab60 con 0x7f14fc0cf830 2024-09-16T15:39:00.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.018+0000 7f14f2a00640 1 -- 172.21.15.167:0/3390967544 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f14b44a9-5910-458c-b518-09a8d0de94de", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f14ec011590 con 0x7f14fc0cf830 2024-09-16T15:39:00.019 INFO:teuthology.orchestra.run.smithi167.stdout:5 2024-09-16T15:39:00.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.019+0000 7f1500a00640 1 -- 172.21.15.167:0/3390967544 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14fc0cf830 msgr2=0x7f14fc0b8200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:00.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.019+0000 7f1500a00640 1 --2- 172.21.15.167:0/3390967544 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14fc0cf830 0x7f14fc0b8200 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f14ec00c400 tx=0x7f14ec00c8c0 comp rx=0 tx=0).stop 2024-09-16T15:39:00.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.019+0000 7f1500a00640 1 -- 172.21.15.167:0/3390967544 shutdown_connections 2024-09-16T15:39:00.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.019+0000 7f1500a00640 1 --2- 172.21.15.167:0/3390967544 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14fc161460 0x7f14fc0c9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:00.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.019+0000 7f1500a00640 1 --2- 172.21.15.167:0/3390967544 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14fc0d2a00 0x7f14fc156c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:00.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.019+0000 7f1500a00640 1 --2- 172.21.15.167:0/3390967544 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14fc0cf830 0x7f14fc0b8200 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:00.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.019+0000 7f1500a00640 1 -- 172.21.15.167:0/3390967544 >> 172.21.15.167:0/3390967544 conn(0x7f14fc09d0b0 msgr2=0x7f14fc160720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:00.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.019+0000 7f1500a00640 1 -- 172.21.15.167:0/3390967544 shutdown_connections 2024-09-16T15:39:00.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.019+0000 7f1500a00640 1 -- 172.21.15.167:0/3390967544 wait complete. 2024-09-16T15:39:00.034 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph --cluster ceph osd new 39335545-2654-4710-84b0-a9aa47cfcbba 6 2024-09-16T15:39:00.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.188+0000 7ff28d600640 1 -- 172.21.15.167:0/1211727992 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2880b5480 msgr2=0x7ff2880d2830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:00.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.188+0000 7ff28d600640 1 --2- 172.21.15.167:0/1211727992 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2880b5480 0x7ff2880d2830 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff270005fd0 tx=0x7ff27002a540 comp rx=0 tx=0).stop 2024-09-16T15:39:00.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.188+0000 7ff28d600640 1 -- 172.21.15.167:0/1211727992 shutdown_connections 2024-09-16T15:39:00.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.188+0000 7ff28d600640 1 --2- 172.21.15.167:0/1211727992 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2880b5480 0x7ff2880d2830 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:00.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.188+0000 7ff28d600640 1 --2- 172.21.15.167:0/1211727992 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff288161460 0x7ff2880d22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:00.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.188+0000 7ff28d600640 1 -- 172.21.15.167:0/1211727992 >> 172.21.15.167:0/1211727992 conn(0x7ff28809d0b0 msgr2=0x7ff2880d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:00.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.188+0000 7ff28d600640 1 -- 172.21.15.167:0/1211727992 shutdown_connections 2024-09-16T15:39:00.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.189+0000 7ff28d600640 1 -- 172.21.15.167:0/1211727992 wait complete. 2024-09-16T15:39:00.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.190+0000 7ff28d600640 1 Processor -- start 2024-09-16T15:39:00.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.190+0000 7ff28d600640 1 -- start start 2024-09-16T15:39:00.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.191+0000 7ff28d600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff2880b4f40 0x7ff2881f5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:00.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.191+0000 7ff28d600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff2880b5480 0x7ff2881572c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:00.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.191+0000 7ff28d600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff288161460 0x7ff28806d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:00.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.191+0000 7ff28d600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff2880cf0b0 con 0x7ff2880b4f40 2024-09-16T15:39:00.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.191+0000 7ff28d600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff2880cef30 con 0x7ff288161460 2024-09-16T15:39:00.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.191+0000 7ff28d600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7ff2880cf230 con 0x7ff2880b5480 2024-09-16T15:39:00.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.191+0000 7ff28cc00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff288161460 0x7ff28806d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:00.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.191+0000 7ff27f400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff2880b5480 0x7ff2881572c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:00.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.191+0000 7ff27f400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff2880b5480 0x7ff2881572c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.167:54754/0 (socket says 172.21.15.167:54754) 2024-09-16T15:39:00.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.191+0000 7ff27f400640 1 -- 172.21.15.167:0/2211637661 learned_addr learned my addr 172.21.15.167:0/2211637661 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-16T15:39:00.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.191+0000 7ff28cc00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff288161460 0x7ff28806d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:51532/0 (socket says 172.21.15.167:51532) 2024-09-16T15:39:00.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.191+0000 7ff287e00640 1 --2- 172.21.15.167:0/2211637661 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff2880b4f40 0x7ff2881f5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:00.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.192+0000 7ff28cc00640 1 -- 172.21.15.167:0/2211637661 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff2880b5480 msgr2=0x7ff2881572c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:00.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.192+0000 7ff28cc00640 1 --2- 172.21.15.167:0/2211637661 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff2880b5480 0x7ff2881572c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:00.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.192+0000 7ff28cc00640 1 -- 172.21.15.167:0/2211637661 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff2880b4f40 msgr2=0x7ff2881f5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:00.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.192+0000 7ff28cc00640 1 --2- 172.21.15.167:0/2211637661 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff2880b4f40 0x7ff2881f5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:00.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.192+0000 7ff28cc00640 1 -- 172.21.15.167:0/2211637661 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2881f5790 con 0x7ff288161460 2024-09-16T15:39:00.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.192+0000 7ff27f400640 1 --2- 172.21.15.167:0/2211637661 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff2880b5480 0x7ff2881572c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:39:00.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.192+0000 7ff28cc00640 1 --2- 172.21.15.167:0/2211637661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff288161460 0x7ff28806d7b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff28000c700 tx=0x7ff28000cbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:00.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.192+0000 7ff284c00640 1 -- 172.21.15.167:0/2211637661 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff280003b70 con 0x7ff288161460 2024-09-16T15:39:00.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.192+0000 7ff284c00640 1 -- 172.21.15.167:0/2211637661 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff280003d10 con 0x7ff288161460 2024-09-16T15:39:00.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.193+0000 7ff284c00640 1 -- 172.21.15.167:0/2211637661 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28000aab0 con 0x7ff288161460 2024-09-16T15:39:00.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.193+0000 7ff28d600640 1 -- 172.21.15.167:0/2211637661 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff288157800 con 0x7ff288161460 2024-09-16T15:39:00.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.193+0000 7ff28d600640 1 -- 172.21.15.167:0/2211637661 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2880b0cd0 con 0x7ff288161460 2024-09-16T15:39:00.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.193+0000 7ff284c00640 1 -- 172.21.15.167:0/2211637661 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff28000ac50 con 0x7ff288161460 2024-09-16T15:39:00.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.193+0000 7ff284c00640 1 -- 172.21.15.167:0/2211637661 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7ff280012510 con 0x7ff288161460 2024-09-16T15:39:00.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.194+0000 7ff28d600640 1 -- 172.21.15.167:0/2211637661 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2880b0ad0 con 0x7ff288161460 2024-09-16T15:39:00.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.196+0000 7ff284c00640 1 -- 172.21.15.167:0/2211637661 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7ff2800050d0 con 0x7ff288161460 2024-09-16T15:39:00.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.282+0000 7ff28d600640 1 -- 172.21.15.167:0/2211637661 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd new", "uuid": "39335545-2654-4710-84b0-a9aa47cfcbba", "id": 6} v 0) -- 0x7ff2880cf750 con 0x7ff288161460 2024-09-16T15:39:00.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.286+0000 7ff284c00640 1 -- 172.21.15.167:0/2211637661 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "39335545-2654-4710-84b0-a9aa47cfcbba", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7ff2800052b0 con 0x7ff288161460 2024-09-16T15:39:00.287 INFO:teuthology.orchestra.run.smithi167.stdout:6 2024-09-16T15:39:00.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.287+0000 7ff28d600640 1 -- 172.21.15.167:0/2211637661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff288161460 msgr2=0x7ff28806d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:00.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.287+0000 7ff28d600640 1 --2- 172.21.15.167:0/2211637661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff288161460 0x7ff28806d7b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff28000c700 tx=0x7ff28000cbc0 comp rx=0 tx=0).stop 2024-09-16T15:39:00.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.287+0000 7ff28d600640 1 -- 172.21.15.167:0/2211637661 shutdown_connections 2024-09-16T15:39:00.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.287+0000 7ff28d600640 1 --2- 172.21.15.167:0/2211637661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff288161460 0x7ff28806d7b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:00.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.287+0000 7ff28d600640 1 --2- 172.21.15.167:0/2211637661 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff2880b5480 0x7ff2881572c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:00.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.288+0000 7ff28d600640 1 --2- 172.21.15.167:0/2211637661 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff2880b4f40 0x7ff2881f5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:00.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.288+0000 7ff28d600640 1 -- 172.21.15.167:0/2211637661 >> 172.21.15.167:0/2211637661 conn(0x7ff28809d0b0 msgr2=0x7ff288075890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:00.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.288+0000 7ff28d600640 1 -- 172.21.15.167:0/2211637661 shutdown_connections 2024-09-16T15:39:00.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.288+0000 7ff28d600640 1 -- 172.21.15.167:0/2211637661 wait complete. 2024-09-16T15:39:00.309 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph --cluster ceph osd new 4f8bfc81-53dd-4339-ac22-1a4007d9f963 7 2024-09-16T15:39:00.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.463+0000 7efffe600640 1 -- 172.21.15.167:0/2331567910 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efff80b5480 msgr2=0x7efff80d2830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:00.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.463+0000 7efffe600640 1 --2- 172.21.15.167:0/2331567910 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efff80b5480 0x7efff80d2830 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7effe4009830 tx=0x7effe402a480 comp rx=0 tx=0).stop 2024-09-16T15:39:00.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.463+0000 7efffe600640 1 -- 172.21.15.167:0/2331567910 shutdown_connections 2024-09-16T15:39:00.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.463+0000 7efffe600640 1 --2- 172.21.15.167:0/2331567910 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efff80b5480 0x7efff80d2830 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:00.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.463+0000 7efffe600640 1 --2- 172.21.15.167:0/2331567910 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efff8161460 0x7efff80d22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:00.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.463+0000 7efffe600640 1 -- 172.21.15.167:0/2331567910 >> 172.21.15.167:0/2331567910 conn(0x7efff809d0b0 msgr2=0x7efff80d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:00.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.464+0000 7efffe600640 1 -- 172.21.15.167:0/2331567910 shutdown_connections 2024-09-16T15:39:00.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.464+0000 7efffe600640 1 -- 172.21.15.167:0/2331567910 wait complete. 2024-09-16T15:39:00.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.465+0000 7efffe600640 1 Processor -- start 2024-09-16T15:39:00.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.465+0000 7efffe600640 1 -- start start 2024-09-16T15:39:00.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.466+0000 7efffe600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efff80b4f40 0x7efff80a0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:00.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.466+0000 7efffe600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efff80b5480 0x7efff80ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:00.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.466+0000 7efffe600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efff8161460 0x7efff815cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:00.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.466+0000 7efffe600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efff80cf0b0 con 0x7efff80b5480 2024-09-16T15:39:00.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.466+0000 7efffe600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efff80cef30 con 0x7efff80b4f40 2024-09-16T15:39:00.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.466+0000 7efffe600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7efff80cf230 con 0x7efff8161460 2024-09-16T15:39:00.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.466+0000 7efffd200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efff80b4f40 0x7efff80a0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:00.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.466+0000 7efffd200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efff80b4f40 0x7efff80a0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:51550/0 (socket says 172.21.15.167:51550) 2024-09-16T15:39:00.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.466+0000 7efffd200640 1 -- 172.21.15.167:0/3266473819 learned_addr learned my addr 172.21.15.167:0/3266473819 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-16T15:39:00.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.466+0000 7efffdc00640 1 --2- 172.21.15.167:0/3266473819 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efff8161460 0x7efff815cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:00.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.467+0000 7efff7e00640 1 --2- 172.21.15.167:0/3266473819 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efff80b5480 0x7efff80ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:00.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.467+0000 7efffd200640 1 -- 172.21.15.167:0/3266473819 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efff8161460 msgr2=0x7efff815cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:00.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.467+0000 7efffd200640 1 --2- 172.21.15.167:0/3266473819 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efff8161460 0x7efff815cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:00.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.467+0000 7efffd200640 1 -- 172.21.15.167:0/3266473819 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efff80b5480 msgr2=0x7efff80ccaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:00.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.467+0000 7efffd200640 1 --2- 172.21.15.167:0/3266473819 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efff80b5480 0x7efff80ccaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:00.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.467+0000 7efffd200640 1 -- 172.21.15.167:0/3266473819 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efff80b1e80 con 0x7efff80b4f40 2024-09-16T15:39:00.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.467+0000 7efff7e00640 1 --2- 172.21.15.167:0/3266473819 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efff80b5480 0x7efff80ccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:39:00.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.467+0000 7efffd200640 1 --2- 172.21.15.167:0/3266473819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efff80b4f40 0x7efff80a0280 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7effec00e9c0 tx=0x7effec00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:00.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.467+0000 7efff4c00640 1 -- 172.21.15.167:0/3266473819 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effec00cd80 con 0x7efff80b4f40 2024-09-16T15:39:00.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.468+0000 7efff4c00640 1 -- 172.21.15.167:0/3266473819 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7effec004020 con 0x7efff80b4f40 2024-09-16T15:39:00.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.468+0000 7efffe600640 1 -- 172.21.15.167:0/3266473819 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efff81f5290 con 0x7efff80b4f40 2024-09-16T15:39:00.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.468+0000 7efff4c00640 1 -- 172.21.15.167:0/3266473819 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effec004450 con 0x7efff80b4f40 2024-09-16T15:39:00.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.468+0000 7efffe600640 1 -- 172.21.15.167:0/3266473819 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efff809e890 con 0x7efff80b4f40 2024-09-16T15:39:00.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.468+0000 7efff4c00640 1 -- 172.21.15.167:0/3266473819 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7effec0053b0 con 0x7efff80b4f40 2024-09-16T15:39:00.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.468+0000 7efff4c00640 1 -- 172.21.15.167:0/3266473819 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7effec009460 con 0x7efff80b4f40 2024-09-16T15:39:00.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.469+0000 7efffe600640 1 -- 172.21.15.167:0/3266473819 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efff80cf750 con 0x7efff80b4f40 2024-09-16T15:39:00.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.471+0000 7efff4c00640 1 -- 172.21.15.167:0/3266473819 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7effec0047c0 con 0x7efff80b4f40 2024-09-16T15:39:00.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.542+0000 7efffe600640 1 -- 172.21.15.167:0/3266473819 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd new", "uuid": "4f8bfc81-53dd-4339-ac22-1a4007d9f963", "id": 7} v 0) -- 0x7efff8160bf0 con 0x7efff80b4f40 2024-09-16T15:39:00.548 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.546+0000 7efff4c00640 1 -- 172.21.15.167:0/3266473819 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4f8bfc81-53dd-4339-ac22-1a4007d9f963", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7effec0049a0 con 0x7efff80b4f40 2024-09-16T15:39:00.548 INFO:teuthology.orchestra.run.smithi167.stdout:7 2024-09-16T15:39:00.549 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.547+0000 7efffe600640 1 -- 172.21.15.167:0/3266473819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efff80b4f40 msgr2=0x7efff80a0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:00.549 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.547+0000 7efffe600640 1 --2- 172.21.15.167:0/3266473819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efff80b4f40 0x7efff80a0280 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7effec00e9c0 tx=0x7effec00ee80 comp rx=0 tx=0).stop 2024-09-16T15:39:00.549 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.547+0000 7efffe600640 1 -- 172.21.15.167:0/3266473819 shutdown_connections 2024-09-16T15:39:00.549 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.547+0000 7efffe600640 1 --2- 172.21.15.167:0/3266473819 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efff8161460 0x7efff815cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:00.549 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.547+0000 7efffe600640 1 --2- 172.21.15.167:0/3266473819 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efff80b5480 0x7efff80ccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:00.549 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.547+0000 7efffe600640 1 --2- 172.21.15.167:0/3266473819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efff80b4f40 0x7efff80a0280 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:00.549 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.547+0000 7efffe600640 1 -- 172.21.15.167:0/3266473819 >> 172.21.15.167:0/3266473819 conn(0x7efff809d0b0 msgr2=0x7efff80cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:00.549 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.547+0000 7efffe600640 1 -- 172.21.15.167:0/3266473819 shutdown_connections 2024-09-16T15:39:00.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:00.547+0000 7efffe600640 1 -- 172.21.15.167:0/3266473819 wait complete. 2024-09-16T15:39:00.563 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-16T15:39:00.564 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-16T15:39:00.567 INFO:tasks.ceph.osd.0:Started 2024-09-16T15:39:00.568 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-16T15:39:00.568 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-16T15:39:00.571 INFO:tasks.ceph.osd.1:Started 2024-09-16T15:39:00.572 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-16T15:39:00.572 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-16T15:39:00.575 INFO:tasks.ceph.osd.2:Started 2024-09-16T15:39:00.576 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-16T15:39:00.576 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-16T15:39:00.580 INFO:tasks.ceph.osd.3:Started 2024-09-16T15:39:00.580 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-16T15:39:00.580 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-16T15:39:00.584 INFO:tasks.ceph.osd.4:Started 2024-09-16T15:39:00.584 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-16T15:39:00.584 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-16T15:39:00.588 INFO:tasks.ceph.osd.5:Started 2024-09-16T15:39:00.588 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-16T15:39:00.588 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-16T15:39:00.593 INFO:tasks.ceph.osd.6:Started 2024-09-16T15:39:00.593 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-16T15:39:00.593 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-16T15:39:00.597 INFO:tasks.ceph.osd.7:Started 2024-09-16T15:39:00.598 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:39:00.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:00.810+0000 7fcda9800640 1 -- 172.21.15.151:0/1649708841 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcda4161440 msgr2=0x7fcda40b5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:00.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:00.810+0000 7fcda9800640 1 --2- 172.21.15.151:0/1649708841 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcda4161440 0x7fcda40b5370 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcda40b6930 tx=0x7fcd9402a820 comp rx=0 tx=0).stop 2024-09-16T15:39:00.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:00.810+0000 7fcda9800640 1 -- 172.21.15.151:0/1649708841 shutdown_connections 2024-09-16T15:39:00.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:00.810+0000 7fcda9800640 1 --2- 172.21.15.151:0/1649708841 >> v2:172.21.15.151:3300/0 conn(0x7fcda40b4050 0x7fcda40b1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:00.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:00.810+0000 7fcda9800640 1 --2- 172.21.15.151:0/1649708841 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcda4161440 0x7fcda40b5370 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:00.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:00.810+0000 7fcda9800640 1 -- 172.21.15.151:0/1649708841 >> 172.21.15.151:0/1649708841 conn(0x7fcda40d3df0 msgr2=0x7fcda40d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:00.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:00.810+0000 7fcda9800640 1 -- 172.21.15.151:0/1649708841 shutdown_connections 2024-09-16T15:39:00.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:00.810+0000 7fcda9800640 1 -- 172.21.15.151:0/1649708841 wait complete. 2024-09-16T15:39:00.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:00.812+0000 7fcda9800640 1 Processor -- start 2024-09-16T15:39:00.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:00.812+0000 7fcda9800640 1 -- start start 2024-09-16T15:39:00.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:00.813+0000 7fcda9800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcda40b34b0 0x7fcda40b8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:00.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:00.813+0000 7fcda9800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcda40b4050 0x7fcda40c9610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:00.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:00.813+0000 7fcda9800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcda4161440 0x7fcda40c9b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:00.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:00.813+0000 7fcda9800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcda40b2040 con 0x7fcda40b34b0 2024-09-16T15:39:00.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:00.813+0000 7fcda9800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcda40b1ec0 con 0x7fcda40b4050 2024-09-16T15:39:00.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:00.813+0000 7fcda9800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fcda40b21c0 con 0x7fcda4161440 2024-09-16T15:39:00.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:00.813+0000 7fcda3e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcda40b34b0 0x7fcda40b8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:00.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:00.813+0000 7fcda3e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcda40b34b0 0x7fcda40b8400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43970/0 (socket says 172.21.15.151:43970) 2024-09-16T15:39:00.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:00.813+0000 7fcda3e00640 1 -- 172.21.15.151:0/3925659045 learned_addr learned my addr 172.21.15.151:0/3925659045 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:00.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:00.813+0000 7fcd9b400640 1 --2- 172.21.15.151:0/3925659045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcda40b4050 0x7fcda40c9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:00.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:00.813+0000 7fcda8e00640 1 --2- 172.21.15.151:0/3925659045 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcda4161440 0x7fcda40c9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:00.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:00.814+0000 7fcda3e00640 1 -- 172.21.15.151:0/3925659045 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcda4161440 msgr2=0x7fcda40c9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:00.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:00.814+0000 7fcda3e00640 1 --2- 172.21.15.151:0/3925659045 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcda4161440 0x7fcda40c9b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:00.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:00.814+0000 7fcda3e00640 1 -- 172.21.15.151:0/3925659045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcda40b4050 msgr2=0x7fcda40c9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:00.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:00.814+0000 7fcda3e00640 1 --2- 172.21.15.151:0/3925659045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcda40b4050 0x7fcda40c9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:00.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:00.814+0000 7fcda3e00640 1 -- 172.21.15.151:0/3925659045 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcda41f4510 con 0x7fcda40b34b0 2024-09-16T15:39:00.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:00.814+0000 7fcd9b400640 1 --2- 172.21.15.151:0/3925659045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcda40b4050 0x7fcda40c9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:39:00.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:00.814+0000 7fcda3e00640 1 --2- 172.21.15.151:0/3925659045 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcda40b34b0 0x7fcda40b8400 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fcd94035040 tx=0x7fcd94032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:00.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:00.814+0000 7fcda0c00640 1 -- 172.21.15.151:0/3925659045 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd94037040 con 0x7fcda40b34b0 2024-09-16T15:39:00.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:00.814+0000 7fcda0c00640 1 -- 172.21.15.151:0/3925659045 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd94042070 con 0x7fcda40b34b0 2024-09-16T15:39:00.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:00.814+0000 7fcda0c00640 1 -- 172.21.15.151:0/3925659045 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd94003e60 con 0x7fcda40b34b0 2024-09-16T15:39:00.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:00.814+0000 7fcda9800640 1 -- 172.21.15.151:0/3925659045 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcda40b8940 con 0x7fcda40b34b0 2024-09-16T15:39:00.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:00.814+0000 7fcda9800640 1 -- 172.21.15.151:0/3925659045 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcda41f47a0 con 0x7fcda40b34b0 2024-09-16T15:39:00.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:00.815+0000 7fcda0c00640 1 -- 172.21.15.151:0/3925659045 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fcd94004280 con 0x7fcda40b34b0 2024-09-16T15:39:00.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:00.815+0000 7fcda0c00640 1 -- 172.21.15.151:0/3925659045 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fcd94032450 con 0x7fcda40b34b0 2024-09-16T15:39:00.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:00.816+0000 7fcda9800640 1 -- 172.21.15.151:0/3925659045 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcda40b1770 con 0x7fcda40b34b0 2024-09-16T15:39:00.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:00.818+0000 7fcda0c00640 1 -- 172.21.15.151:0/3925659045 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fcd94004570 con 0x7fcda40b34b0 2024-09-16T15:39:00.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:00.881+0000 7fcda9800640 1 -- 172.21.15.151:0/3925659045 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcda40b6390 con 0x7fcda40b34b0 2024-09-16T15:39:00.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:00.881+0000 7fcda0c00640 1 -- 172.21.15.151:0/3925659045 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fcd94004750 con 0x7fcda40b34b0 2024-09-16T15:39:00.884 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:39:00.884 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":10,"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","created":"2024-09-16T15:38:56.348535+0000","modified":"2024-09-16T15:39:00.544477+0000","last_up_change":"0.000000","last_in_change":"2024-09-16T15:39:00.544477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"d5524076-a903-4d0a-af6a-caf474c4765d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"dd25925f-0ae1-494e-8529-4dc205759db3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c2f18115-013d-4da6-a0fe-cb978f4494b3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"36edb718-2fcb-4d88-8ddc-c321eed4279d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"64d33708-cd0a-4236-abe3-1b646e82ab54","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"f14b44a9-5910-458c-b518-09a8d0de94de","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"39335545-2654-4710-84b0-a9aa47cfcbba","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"4f8bfc81-53dd-4339-ac22-1a4007d9f963","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:39:00.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:00.882+0000 7fcda9800640 1 -- 172.21.15.151:0/3925659045 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcda40b34b0 msgr2=0x7fcda40b8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:00.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:00.882+0000 7fcda9800640 1 --2- 172.21.15.151:0/3925659045 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcda40b34b0 0x7fcda40b8400 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fcd94035040 tx=0x7fcd94032d10 comp rx=0 tx=0).stop 2024-09-16T15:39:00.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:00.882+0000 7fcda9800640 1 -- 172.21.15.151:0/3925659045 shutdown_connections 2024-09-16T15:39:00.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:00.882+0000 7fcda9800640 1 --2- 172.21.15.151:0/3925659045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcda40b4050 0x7fcda40c9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:00.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:00.882+0000 7fcda9800640 1 --2- 172.21.15.151:0/3925659045 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcda4161440 0x7fcda40c9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:00.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:00.882+0000 7fcda9800640 1 --2- 172.21.15.151:0/3925659045 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcda40b34b0 0x7fcda40b8400 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:00.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:00.882+0000 7fcda9800640 1 -- 172.21.15.151:0/3925659045 >> 172.21.15.151:0/3925659045 conn(0x7fcda40d3df0 msgr2=0x7fcda40b3920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:00.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:00.883+0000 7fcda9800640 1 -- 172.21.15.151:0/3925659045 shutdown_connections 2024-09-16T15:39:00.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:00.883+0000 7fcda9800640 1 -- 172.21.15.151:0/3925659045 wait complete. 2024-09-16T15:39:00.899 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-16T15:39:00.899 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-16T15:39:01.703 DEBUG:teuthology.orchestra.run.smithi151:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-16T15:39:01.848 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:01.845+0000 7f7f56000640 1 -- 172.21.15.151:0/1955245193 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7f500cf830 msgr2=0x7f7f500b4030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:01.848 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:01.845+0000 7f7f56000640 1 --2- 172.21.15.151:0/1955245193 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7f500cf830 0x7f7f500b4030 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7f48005fd0 tx=0x7f7f4802aaa0 comp rx=0 tx=0).stop 2024-09-16T15:39:01.848 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:01.845+0000 7f7f56000640 1 -- 172.21.15.151:0/1955245193 shutdown_connections 2024-09-16T15:39:01.848 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:01.845+0000 7f7f56000640 1 --2- 172.21.15.151:0/1955245193 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7f500cf830 0x7f7f500b4030 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:01.848 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:01.845+0000 7f7f56000640 1 --2- 172.21.15.151:0/1955245193 >> v2:172.21.15.167:3300/0 conn(0x7f7f500d0790 0x7f7f500d0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:01.848 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:01.845+0000 7f7f56000640 1 -- 172.21.15.151:0/1955245193 >> 172.21.15.151:0/1955245193 conn(0x7f7f5009d110 msgr2=0x7f7f500d3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:01.848 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:01.845+0000 7f7f56000640 1 -- 172.21.15.151:0/1955245193 shutdown_connections 2024-09-16T15:39:01.848 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:01.845+0000 7f7f56000640 1 -- 172.21.15.151:0/1955245193 wait complete. 2024-09-16T15:39:01.849 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:01.847+0000 7f7f56000640 1 Processor -- start 2024-09-16T15:39:01.849 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:01.847+0000 7f7f56000640 1 -- start start 2024-09-16T15:39:01.851 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:01.848+0000 7f7f56000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f500b2de0 0x7f7f501f4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:01.851 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:01.848+0000 7f7f56000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7f500cf830 0x7f7f500cdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:01.851 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:01.848+0000 7f7f56000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7f500d0790 0x7f7f50160db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:01.851 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:01.848+0000 7f7f56000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7f500b46f0 con 0x7f7f500cf830 2024-09-16T15:39:01.851 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:01.848+0000 7f7f56000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7f500b4570 con 0x7f7f500b2de0 2024-09-16T15:39:01.851 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:01.848+0000 7f7f56000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f7f500b4870 con 0x7f7f500d0790 2024-09-16T15:39:01.851 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:01.848+0000 7f7f47e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7f500cf830 0x7f7f500cdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:01.851 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:01.848+0000 7f7f54c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f500b2de0 0x7f7f501f4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:01.851 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:01.848+0000 7f7f54c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f500b2de0 0x7f7f501f4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.151:45226/0 (socket says 172.21.15.151:45226) 2024-09-16T15:39:01.852 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:01.848+0000 7f7f54c00640 1 -- 172.21.15.151:0/963319113 learned_addr learned my addr 172.21.15.151:0/963319113 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:01.852 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:01.848+0000 7f7f55600640 1 --2- 172.21.15.151:0/963319113 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7f500d0790 0x7f7f50160db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:01.852 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:01.849+0000 7f7f54c00640 1 -- 172.21.15.151:0/963319113 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7f500d0790 msgr2=0x7f7f50160db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:01.852 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:01.849+0000 7f7f54c00640 1 --2- 172.21.15.151:0/963319113 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7f500d0790 0x7f7f50160db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:01.852 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:01.849+0000 7f7f54c00640 1 -- 172.21.15.151:0/963319113 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7f500cf830 msgr2=0x7f7f500cdcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:01.852 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:01.849+0000 7f7f54c00640 1 --2- 172.21.15.151:0/963319113 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7f500cf830 0x7f7f500cdcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:01.852 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:01.849+0000 7f7f54c00640 1 -- 172.21.15.151:0/963319113 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f500ce1f0 con 0x7f7f500b2de0 2024-09-16T15:39:01.852 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:01.849+0000 7f7f54c00640 1 --2- 172.21.15.151:0/963319113 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f500b2de0 0x7f7f501f4b80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7f4000e790 tx=0x7f7f4000ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:01.853 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:01.849+0000 7f7f4d600640 1 -- 172.21.15.151:0/963319113 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4000eea0 con 0x7f7f500b2de0 2024-09-16T15:39:01.853 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:01.849+0000 7f7f4d600640 1 -- 172.21.15.151:0/963319113 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7f4000cdb0 con 0x7f7f500b2de0 2024-09-16T15:39:01.853 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:01.849+0000 7f7f56000640 1 -- 172.21.15.151:0/963319113 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f501f5210 con 0x7f7f500b2de0 2024-09-16T15:39:01.853 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:01.849+0000 7f7f4d600640 1 -- 172.21.15.151:0/963319113 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f40010690 con 0x7f7f500b2de0 2024-09-16T15:39:01.853 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:01.849+0000 7f7f56000640 1 -- 172.21.15.151:0/963319113 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f501f57d0 con 0x7f7f500b2de0 2024-09-16T15:39:01.853 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:01.850+0000 7f7f4d600640 1 -- 172.21.15.151:0/963319113 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 2) ==== 50665+0+0 (secure 0 0 0) 0x7f7f40010830 con 0x7f7f500b2de0 2024-09-16T15:39:01.853 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:01.850+0000 7f7f56000640 1 -- 172.21.15.151:0/963319113 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f500d0b70 con 0x7f7f500b2de0 2024-09-16T15:39:01.853 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:01.851+0000 7f7f4d600640 1 -- 172.21.15.151:0/963319113 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f7f40052c50 con 0x7f7f500b2de0 2024-09-16T15:39:01.855 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:01.853+0000 7f7f4d600640 1 -- 172.21.15.151:0/963319113 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f7f500ce1f0 con 0x7f7f500b2de0 2024-09-16T15:39:01.918 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:01.917+0000 7f7f56000640 1 -- 172.21.15.151:0/963319113 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7f5015f6c0 con 0x7f7f500b2de0 2024-09-16T15:39:01.919 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:01.917+0000 7f7f4d600640 1 -- 172.21.15.151:0/963319113 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f7f40020360 con 0x7f7f500b2de0 2024-09-16T15:39:01.919 INFO:teuthology.misc.health.smithi151.stdout: 2024-09-16T15:39:01.919 INFO:teuthology.misc.health.smithi151.stdout:{"epoch":10,"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","created":"2024-09-16T15:38:56.348535+0000","modified":"2024-09-16T15:39:00.544477+0000","last_up_change":"0.000000","last_in_change":"2024-09-16T15:39:00.544477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"d5524076-a903-4d0a-af6a-caf474c4765d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"dd25925f-0ae1-494e-8529-4dc205759db3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c2f18115-013d-4da6-a0fe-cb978f4494b3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"36edb718-2fcb-4d88-8ddc-c321eed4279d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"64d33708-cd0a-4236-abe3-1b646e82ab54","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"f14b44a9-5910-458c-b518-09a8d0de94de","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"39335545-2654-4710-84b0-a9aa47cfcbba","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"4f8bfc81-53dd-4339-ac22-1a4007d9f963","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:39:01.920 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:01.918+0000 7f7f56000640 1 -- 172.21.15.151:0/963319113 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f500b2de0 msgr2=0x7f7f501f4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:01.920 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:01.918+0000 7f7f56000640 1 --2- 172.21.15.151:0/963319113 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f500b2de0 0x7f7f501f4b80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7f4000e790 tx=0x7f7f4000ec50 comp rx=0 tx=0).stop 2024-09-16T15:39:01.920 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:01.918+0000 7f7f56000640 1 -- 172.21.15.151:0/963319113 shutdown_connections 2024-09-16T15:39:01.920 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:01.918+0000 7f7f56000640 1 --2- 172.21.15.151:0/963319113 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7f500d0790 0x7f7f50160db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:01.920 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:01.918+0000 7f7f56000640 1 --2- 172.21.15.151:0/963319113 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7f500cf830 0x7f7f500cdcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:01.920 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:01.918+0000 7f7f56000640 1 --2- 172.21.15.151:0/963319113 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f500b2de0 0x7f7f501f4b80 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:01.920 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:01.918+0000 7f7f56000640 1 -- 172.21.15.151:0/963319113 >> 172.21.15.151:0/963319113 conn(0x7f7f5009d110 msgr2=0x7f7f500b16a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:01.921 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:01.918+0000 7f7f56000640 1 -- 172.21.15.151:0/963319113 shutdown_connections 2024-09-16T15:39:01.921 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:01.918+0000 7f7f56000640 1 -- 172.21.15.151:0/963319113 wait complete. 2024-09-16T15:39:01.932 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-16T15:39:02.140 INFO:tasks.ceph.osd.7.smithi167.stderr:2024-09-16T15:39:02.139+0000 7f928abc5640 -1 Falling back to public interface 2024-09-16T15:39:02.148 INFO:tasks.ceph.osd.4.smithi167.stderr:2024-09-16T15:39:02.147+0000 7f2c98a18640 -1 Falling back to public interface 2024-09-16T15:39:02.156 INFO:tasks.ceph.osd.1.smithi151.stderr:2024-09-16T15:39:02.154+0000 7ff1d6f2e640 -1 Falling back to public interface 2024-09-16T15:39:02.180 INFO:tasks.ceph.osd.0.smithi151.stderr:2024-09-16T15:39:02.179+0000 7fd1f66b0640 -1 Falling back to public interface 2024-09-16T15:39:02.206 INFO:tasks.ceph.osd.6.smithi167.stderr:2024-09-16T15:39:02.205+0000 7f8090fb2640 -1 Falling back to public interface 2024-09-16T15:39:02.223 INFO:tasks.ceph.osd.3.smithi151.stderr:2024-09-16T15:39:02.222+0000 7fae1780f640 -1 Falling back to public interface 2024-09-16T15:39:02.228 INFO:tasks.ceph.osd.2.smithi151.stderr:2024-09-16T15:39:02.228+0000 7f348621d640 -1 Falling back to public interface 2024-09-16T15:39:02.439 INFO:tasks.ceph.osd.5.smithi167.stderr:2024-09-16T15:39:02.438+0000 7f2a78e16640 -1 Falling back to public interface 2024-09-16T15:39:02.585 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:39:02.584+0000 7f491a600640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-16T15:39:02.586 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:39:02.584+0000 7f491a600640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-16T15:39:05.805 INFO:tasks.ceph.osd.7.smithi167.stderr:2024-09-16T15:39:05.804+0000 7f928abc5640 -1 osd.7 0 log_to_monitors true 2024-09-16T15:39:05.964 INFO:tasks.ceph.osd.6.smithi167.stderr:2024-09-16T15:39:05.963+0000 7f8090fb2640 -1 osd.6 0 log_to_monitors true 2024-09-16T15:39:06.092 INFO:tasks.ceph.osd.1.smithi151.stderr:2024-09-16T15:39:06.091+0000 7ff1d6f2e640 -1 osd.1 0 log_to_monitors true 2024-09-16T15:39:06.099 INFO:tasks.ceph.osd.0.smithi151.stderr:2024-09-16T15:39:06.098+0000 7fd1f66b0640 -1 osd.0 0 log_to_monitors true 2024-09-16T15:39:06.351 INFO:tasks.ceph.osd.4.smithi167.stderr:2024-09-16T15:39:06.350+0000 7f2c98a18640 -1 osd.4 0 log_to_monitors true 2024-09-16T15:39:06.361 INFO:tasks.ceph.osd.3.smithi151.stderr:2024-09-16T15:39:06.360+0000 7fae1780f640 -1 osd.3 0 log_to_monitors true 2024-09-16T15:39:06.441 INFO:tasks.ceph.osd.5.smithi167.stderr:2024-09-16T15:39:06.439+0000 7f2a78e16640 -1 osd.5 0 log_to_monitors true 2024-09-16T15:39:06.688 INFO:tasks.ceph.osd.2.smithi151.stderr:2024-09-16T15:39:06.687+0000 7f348621d640 -1 osd.2 0 log_to_monitors true 2024-09-16T15:39:07.970 INFO:tasks.ceph.osd.7.smithi167.stderr:2024-09-16T15:39:07.969+0000 7f9286400640 -1 osd.7 0 waiting for initial osdmap 2024-09-16T15:39:07.985 INFO:tasks.ceph.osd.7.smithi167.stderr:2024-09-16T15:39:07.984+0000 7f927ba00640 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T15:39:08.739 DEBUG:teuthology.orchestra.run.smithi151:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-16T15:39:08.917 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:08.916+0000 7f20f1c00640 1 -- 172.21.15.151:0/444677998 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f20ec0cec40 msgr2=0x7f20ec0cf020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:08.917 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:08.916+0000 7f20f1c00640 1 --2- 172.21.15.151:0/444677998 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f20ec0cec40 0x7f20ec0cf020 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f20dc009830 tx=0x7f20dc02a530 comp rx=0 tx=0).stop 2024-09-16T15:39:08.917 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:08.916+0000 7f20f1c00640 1 -- 172.21.15.151:0/444677998 shutdown_connections 2024-09-16T15:39:08.918 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:08.916+0000 7f20f1c00640 1 --2- 172.21.15.151:0/444677998 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f20ec0cec40 0x7f20ec0cf020 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:08.918 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:08.916+0000 7f20f1c00640 1 -- 172.21.15.151:0/444677998 >> 172.21.15.151:0/444677998 conn(0x7f20ec0cfb80 msgr2=0x7f20ec0cff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:08.918 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:08.916+0000 7f20f1c00640 1 -- 172.21.15.151:0/444677998 shutdown_connections 2024-09-16T15:39:08.918 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:08.916+0000 7f20f1c00640 1 -- 172.21.15.151:0/444677998 wait complete. 2024-09-16T15:39:08.919 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:08.918+0000 7f20f1c00640 1 Processor -- start 2024-09-16T15:39:08.919 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:08.918+0000 7f20f1c00640 1 -- start start 2024-09-16T15:39:08.921 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:08.919+0000 7f20f1c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20ec0ad8e0 0x7f20ec1f7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:08.921 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:08.919+0000 7f20f1c00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f20ec0cec40 0x7f20ec1f5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:08.921 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:08.919+0000 7f20f1c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f20ec162920 0x7f20ec0c54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:08.921 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:08.919+0000 7f20f1c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f20ec0ae400 con 0x7f20ec162920 2024-09-16T15:39:08.921 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:08.919+0000 7f20f1c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f20ec15c0c0 con 0x7f20ec0ad8e0 2024-09-16T15:39:08.921 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:08.919+0000 7f20f1c00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f20ec0ae580 con 0x7f20ec0cec40 2024-09-16T15:39:08.922 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:08.919+0000 7f20eb400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f20ec162920 0x7f20ec0c54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:08.922 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:08.919+0000 7f20eaa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20ec0ad8e0 0x7f20ec1f7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:08.922 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:08.919+0000 7f20ea000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f20ec0cec40 0x7f20ec1f5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:08.922 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:08.919+0000 7f20eaa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20ec0ad8e0 0x7f20ec1f7d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.151:50332/0 (socket says 172.21.15.151:50332) 2024-09-16T15:39:08.922 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:08.919+0000 7f20eaa00640 1 -- 172.21.15.151:0/721190908 learned_addr learned my addr 172.21.15.151:0/721190908 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:08.922 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:08.920+0000 7f20eb400640 1 -- 172.21.15.151:0/721190908 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f20ec0cec40 msgr2=0x7f20ec1f5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:08.922 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:08.920+0000 7f20eb400640 1 --2- 172.21.15.151:0/721190908 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f20ec0cec40 0x7f20ec1f5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:08.922 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:08.920+0000 7f20eb400640 1 -- 172.21.15.151:0/721190908 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20ec0ad8e0 msgr2=0x7f20ec1f7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:08.922 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:08.920+0000 7f20eb400640 1 --2- 172.21.15.151:0/721190908 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20ec0ad8e0 0x7f20ec1f7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:08.922 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:08.920+0000 7f20eb400640 1 -- 172.21.15.151:0/721190908 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20ec1f6380 con 0x7f20ec162920 2024-09-16T15:39:08.922 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:08.920+0000 7f20eb400640 1 --2- 172.21.15.151:0/721190908 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f20ec162920 0x7f20ec0c54b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f20e400cb40 tx=0x7f20e400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:08.923 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:08.921+0000 7f20e2a00640 1 -- 172.21.15.151:0/721190908 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e4016020 con 0x7f20ec162920 2024-09-16T15:39:08.923 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:08.921+0000 7f20e2a00640 1 -- 172.21.15.151:0/721190908 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f20e400ad90 con 0x7f20ec162920 2024-09-16T15:39:08.923 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:08.921+0000 7f20f1c00640 1 -- 172.21.15.151:0/721190908 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20ec0b29d0 con 0x7f20ec162920 2024-09-16T15:39:08.923 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:08.921+0000 7f20e2a00640 1 -- 172.21.15.151:0/721190908 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e4012680 con 0x7f20ec162920 2024-09-16T15:39:08.923 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:08.921+0000 7f20f1c00640 1 -- 172.21.15.151:0/721190908 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20ec0c5ca0 con 0x7f20ec162920 2024-09-16T15:39:08.923 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:08.922+0000 7f20f1c00640 1 -- 172.21.15.151:0/721190908 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20ec0aeb10 con 0x7f20ec162920 2024-09-16T15:39:08.924 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:08.923+0000 7f20e2a00640 1 -- 172.21.15.151:0/721190908 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 4) ==== 101231+0+0 (secure 0 0 0) 0x7f20e4019020 con 0x7f20ec162920 2024-09-16T15:39:08.925 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:08.924+0000 7f20e2a00640 1 --2- 172.21.15.151:0/721190908 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f20cc0787c0 0x7f20cc07ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:08.925 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:08.924+0000 7f20e2a00640 1 -- 172.21.15.151:0/721190908 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2231+0+0 (secure 0 0 0) 0x7f20e4099cd0 con 0x7f20ec162920 2024-09-16T15:39:08.925 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:08.924+0000 7f20eaa00640 1 --2- 172.21.15.151:0/721190908 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f20cc0787c0 0x7f20cc07ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:08.926 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:08.925+0000 7f20eaa00640 1 --2- 172.21.15.151:0/721190908 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f20cc0787c0 0x7f20cc07ac80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f20dc009830 tx=0x7f20dc004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:08.931 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:08.930+0000 7f20e2a00640 1 -- 172.21.15.151:0/721190908 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f20e4020030 con 0x7f20ec162920 2024-09-16T15:39:09.175 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:09.174+0000 7f20f1c00640 1 -- 172.21.15.151:0/721190908 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f20ec160a90 con 0x7f20ec162920 2024-09-16T15:39:09.176 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:09.175+0000 7f20e2a00640 1 -- 172.21.15.151:0/721190908 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) ==== 74+0+7555 (secure 0 0 0) 0x7f20e4062ad0 con 0x7f20ec162920 2024-09-16T15:39:09.176 INFO:teuthology.misc.health.smithi151.stdout: 2024-09-16T15:39:09.176 INFO:teuthology.misc.health.smithi151.stdout:{"epoch":14,"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","created":"2024-09-16T15:38:56.348535+0000","modified":"2024-09-16T15:39:08.887489+0000","last_up_change":"2024-09-16T15:39:08.887489+0000","last_in_change":"2024-09-16T15:39:00.544477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"d5524076-a903-4d0a-af6a-caf474c4765d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"dd25925f-0ae1-494e-8529-4dc205759db3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c2f18115-013d-4da6-a0fe-cb978f4494b3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"36edb718-2fcb-4d88-8ddc-c321eed4279d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"64d33708-cd0a-4236-abe3-1b646e82ab54","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"f14b44a9-5910-458c-b518-09a8d0de94de","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"39335545-2654-4710-84b0-a9aa47cfcbba","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"4f8bfc81-53dd-4339-ac22-1a4007d9f963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6803","nonce":265072227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6805","nonce":265072227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6809","nonce":265072227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6807","nonce":265072227}]},"public_addr":"172.21.15.167:6803/265072227","cluster_addr":"172.21.15.167:6805/265072227","heartbeat_back_addr":"172.21.15.167:6809/265072227","heartbeat_front_addr":"172.21.15.167:6807/265072227","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:39:09.179 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:09.177+0000 7f20f1c00640 1 -- 172.21.15.151:0/721190908 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f20cc0787c0 msgr2=0x7f20cc07ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:09.179 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:09.177+0000 7f20f1c00640 1 --2- 172.21.15.151:0/721190908 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f20cc0787c0 0x7f20cc07ac80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f20dc009830 tx=0x7f20dc004730 comp rx=0 tx=0).stop 2024-09-16T15:39:09.179 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:09.177+0000 7f20f1c00640 1 -- 172.21.15.151:0/721190908 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f20ec162920 msgr2=0x7f20ec0c54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:09.179 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:09.177+0000 7f20f1c00640 1 --2- 172.21.15.151:0/721190908 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f20ec162920 0x7f20ec0c54b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f20e400cb40 tx=0x7f20e400a060 comp rx=0 tx=0).stop 2024-09-16T15:39:09.179 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:09.177+0000 7f20f1c00640 1 -- 172.21.15.151:0/721190908 shutdown_connections 2024-09-16T15:39:09.179 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:09.177+0000 7f20f1c00640 1 --2- 172.21.15.151:0/721190908 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f20cc0787c0 0x7f20cc07ac80 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:09.179 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:09.177+0000 7f20f1c00640 1 --2- 172.21.15.151:0/721190908 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f20ec162920 0x7f20ec0c54b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:09.179 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:09.177+0000 7f20f1c00640 1 --2- 172.21.15.151:0/721190908 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f20ec0cec40 0x7f20ec1f5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:09.179 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:09.177+0000 7f20f1c00640 1 --2- 172.21.15.151:0/721190908 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f20ec0ad8e0 0x7f20ec1f7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:09.180 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:09.178+0000 7f20f1c00640 1 -- 172.21.15.151:0/721190908 >> 172.21.15.151:0/721190908 conn(0x7f20ec0cfb80 msgr2=0x7f20ec1614c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:09.180 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:09.178+0000 7f20f1c00640 1 -- 172.21.15.151:0/721190908 shutdown_connections 2024-09-16T15:39:09.180 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:09.178+0000 7f20f1c00640 1 -- 172.21.15.151:0/721190908 wait complete. 2024-09-16T15:39:09.195 DEBUG:teuthology.misc:1 of 8 OSDs are up 2024-09-16T15:39:09.676 INFO:tasks.ceph.osd.6.smithi167.stderr:2024-09-16T15:39:09.675+0000 7f808c800640 -1 osd.6 0 waiting for initial osdmap 2024-09-16T15:39:09.682 INFO:tasks.ceph.osd.6.smithi167.stderr:2024-09-16T15:39:09.681+0000 7f8081e00640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T15:39:09.728 INFO:tasks.ceph.osd.5.smithi167.stderr:2024-09-16T15:39:09.727+0000 7f2a73600640 -1 osd.5 0 waiting for initial osdmap 2024-09-16T15:39:09.735 INFO:tasks.ceph.osd.5.smithi167.stderr:2024-09-16T15:39:09.734+0000 7f2a69600640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T15:39:09.744 INFO:tasks.ceph.osd.4.smithi167.stderr:2024-09-16T15:39:09.743+0000 7f2c93200640 -1 osd.4 0 waiting for initial osdmap 2024-09-16T15:39:09.757 INFO:tasks.ceph.osd.4.smithi167.stderr:2024-09-16T15:39:09.756+0000 7f2c89200640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T15:39:10.201 INFO:tasks.ceph.osd.1.smithi151.stderr:2024-09-16T15:39:10.200+0000 7ff1d2800640 -1 osd.1 0 waiting for initial osdmap 2024-09-16T15:39:10.210 INFO:tasks.ceph.osd.1.smithi151.stderr:2024-09-16T15:39:10.210+0000 7ff1c7e00640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T15:39:10.221 INFO:tasks.ceph.osd.0.smithi151.stderr:2024-09-16T15:39:10.220+0000 7fd1f2000640 -1 osd.0 0 waiting for initial osdmap 2024-09-16T15:39:10.227 INFO:tasks.ceph.osd.3.smithi151.stderr:2024-09-16T15:39:10.226+0000 7fae12000640 -1 osd.3 0 waiting for initial osdmap 2024-09-16T15:39:10.231 INFO:tasks.ceph.osd.0.smithi151.stderr:2024-09-16T15:39:10.228+0000 7fd1e7600640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T15:39:10.240 INFO:tasks.ceph.osd.3.smithi151.stderr:2024-09-16T15:39:10.239+0000 7fae08000640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T15:39:10.241 INFO:tasks.ceph.osd.2.smithi151.stderr:2024-09-16T15:39:10.240+0000 7f3480c00640 -1 osd.2 0 waiting for initial osdmap 2024-09-16T15:39:10.250 INFO:tasks.ceph.osd.2.smithi151.stderr:2024-09-16T15:39:10.249+0000 7f3476c00640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T15:39:16.002 DEBUG:teuthology.orchestra.run.smithi151:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-16T15:39:16.150 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:16.149+0000 7f6e63200640 1 -- 172.21.15.151:0/4201711455 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e640ce920 msgr2=0x7f6e640cd5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:16.150 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:16.149+0000 7f6e63200640 1 --2- 172.21.15.151:0/4201711455 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e640ce920 0x7f6e640cd5b0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6e50009860 tx=0x7f6e5002a5a0 comp rx=0 tx=0).stop 2024-09-16T15:39:16.150 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:16.149+0000 7f6e63200640 1 -- 172.21.15.151:0/4201711455 shutdown_connections 2024-09-16T15:39:16.151 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:16.149+0000 7f6e63200640 1 --2- 172.21.15.151:0/4201711455 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e640ce920 0x7f6e640cd5b0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:16.151 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:16.149+0000 7f6e63200640 1 -- 172.21.15.151:0/4201711455 >> 172.21.15.151:0/4201711455 conn(0x7f6e640cc460 msgr2=0x7f6e640cc840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:16.151 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:16.149+0000 7f6e63200640 1 -- 172.21.15.151:0/4201711455 shutdown_connections 2024-09-16T15:39:16.151 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:16.149+0000 7f6e63200640 1 -- 172.21.15.151:0/4201711455 wait complete. 2024-09-16T15:39:16.151 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:16.150+0000 7f6e63200640 1 Processor -- start 2024-09-16T15:39:16.151 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:16.151+0000 7f6e63200640 1 -- start start 2024-09-16T15:39:16.154 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:16.151+0000 7f6e63200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6e640ce920 0x7f6e6414cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:16.155 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:16.151+0000 7f6e63200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e6415ba60 0x7f6e640c5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:16.155 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:16.151+0000 7f6e63200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e6415be40 0x7f6e641f5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:16.155 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:16.151+0000 7f6e63200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6e640cf430 con 0x7f6e640ce920 2024-09-16T15:39:16.155 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:16.152+0000 7f6e63200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6e640cf2b0 con 0x7f6e6415be40 2024-09-16T15:39:16.155 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:16.152+0000 7f6e63200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f6e640cf5b0 con 0x7f6e6415ba60 2024-09-16T15:39:16.155 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:16.152+0000 7f6e61e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6e640ce920 0x7f6e6414cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:16.155 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:16.152+0000 7f6e61e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6e640ce920 0x7f6e6414cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:35156/0 (socket says 172.21.15.151:35156) 2024-09-16T15:39:16.155 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:16.152+0000 7f6e61e00640 1 -- 172.21.15.151:0/2774350050 learned_addr learned my addr 172.21.15.151:0/2774350050 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:16.155 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:16.152+0000 7f6e61400640 1 --2- 172.21.15.151:0/2774350050 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e6415ba60 0x7f6e640c5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:16.155 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:16.152+0000 7f6e62800640 1 --2- 172.21.15.151:0/2774350050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e6415be40 0x7f6e641f5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:16.155 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:16.152+0000 7f6e61e00640 1 -- 172.21.15.151:0/2774350050 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e6415ba60 msgr2=0x7f6e640c5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:16.155 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:16.152+0000 7f6e61e00640 1 --2- 172.21.15.151:0/2774350050 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e6415ba60 0x7f6e640c5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:16.155 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:16.152+0000 7f6e61e00640 1 -- 172.21.15.151:0/2774350050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e6415be40 msgr2=0x7f6e641f5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:16.156 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:16.152+0000 7f6e61e00640 1 --2- 172.21.15.151:0/2774350050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e6415be40 0x7f6e641f5f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:16.156 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:16.152+0000 7f6e61e00640 1 -- 172.21.15.151:0/2774350050 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e640c9650 con 0x7f6e640ce920 2024-09-16T15:39:16.156 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:16.152+0000 7f6e61e00640 1 --2- 172.21.15.151:0/2774350050 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6e640ce920 0x7f6e6414cdf0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6e5400c830 tx=0x7f6e5400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:16.156 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:16.153+0000 7f6e5a000640 1 -- 172.21.15.151:0/2774350050 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e54003b70 con 0x7f6e640ce920 2024-09-16T15:39:16.156 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:16.153+0000 7f6e5a000640 1 -- 172.21.15.151:0/2774350050 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6e54003d30 con 0x7f6e640ce920 2024-09-16T15:39:16.156 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:16.153+0000 7f6e63200640 1 -- 172.21.15.151:0/2774350050 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e640c8db0 con 0x7f6e640ce920 2024-09-16T15:39:16.156 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:16.153+0000 7f6e5a000640 1 -- 172.21.15.151:0/2774350050 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e5400aab0 con 0x7f6e640ce920 2024-09-16T15:39:16.156 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:16.153+0000 7f6e63200640 1 -- 172.21.15.151:0/2774350050 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e640c9370 con 0x7f6e640ce920 2024-09-16T15:39:16.157 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:16.154+0000 7f6e63200640 1 -- 172.21.15.151:0/2774350050 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e64074400 con 0x7f6e640ce920 2024-09-16T15:39:16.157 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:16.155+0000 7f6e5a000640 1 -- 172.21.15.151:0/2774350050 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f6e5400ac50 con 0x7f6e640ce920 2024-09-16T15:39:16.157 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:16.156+0000 7f6e5a000640 1 --2- 172.21.15.151:0/2774350050 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6e44078850 0x7f6e4407ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:16.157 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:16.156+0000 7f6e5a000640 1 -- 172.21.15.151:0/2774350050 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f6e5409a550 con 0x7f6e640ce920 2024-09-16T15:39:16.157 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:16.156+0000 7f6e61400640 1 --2- 172.21.15.151:0/2774350050 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6e44078850 0x7f6e4407ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:16.158 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:16.157+0000 7f6e61400640 1 --2- 172.21.15.151:0/2774350050 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6e44078850 0x7f6e4407ad10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6e50034000 tx=0x7f6e50004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:16.161 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:16.161+0000 7f6e5a000640 1 -- 172.21.15.151:0/2774350050 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6e54062f40 con 0x7f6e640ce920 2024-09-16T15:39:16.351 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:16.350+0000 7f6e63200640 1 -- 172.21.15.151:0/2774350050 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6e640b4e20 con 0x7f6e640ce920 2024-09-16T15:39:16.352 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:16.350+0000 7f6e5a000640 1 -- 172.21.15.151:0/2774350050 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12956 (secure 0 0 0) 0x7f6e54062840 con 0x7f6e640ce920 2024-09-16T15:39:16.352 INFO:teuthology.misc.health.smithi151.stdout: 2024-09-16T15:39:16.352 INFO:teuthology.misc.health.smithi151.stdout:{"epoch":18,"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","created":"2024-09-16T15:38:56.348535+0000","modified":"2024-09-16T15:39:13.690317+0000","last_up_change":"2024-09-16T15:39:10.681384+0000","last_in_change":"2024-09-16T15:39:00.544477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:39:11.602252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5524076-a903-4d0a-af6a-caf474c4765d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6809","nonce":3111342741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6811","nonce":3111342741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6815","nonce":3111342741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6813","nonce":3111342741}]},"public_addr":"172.21.15.151:6809/3111342741","cluster_addr":"172.21.15.151:6811/3111342741","heartbeat_back_addr":"172.21.15.151:6815/3111342741","heartbeat_front_addr":"172.21.15.151:6813/3111342741","state":["exists","up"]},{"osd":1,"uuid":"dd25925f-0ae1-494e-8529-4dc205759db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6801","nonce":3368304087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6803","nonce":3368304087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6807","nonce":3368304087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6805","nonce":3368304087}]},"public_addr":"172.21.15.151:6801/3368304087","cluster_addr":"172.21.15.151:6803/3368304087","heartbeat_back_addr":"172.21.15.151:6807/3368304087","heartbeat_front_addr":"172.21.15.151:6805/3368304087","state":["exists","up"]},{"osd":2,"uuid":"c2f18115-013d-4da6-a0fe-cb978f4494b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6823","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6825","nonce":3079373271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6827","nonce":3079373271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6831","nonce":3079373271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6829","nonce":3079373271}]},"public_addr":"172.21.15.151:6825/3079373271","cluster_addr":"172.21.15.151:6827/3079373271","heartbeat_back_addr":"172.21.15.151:6831/3079373271","heartbeat_front_addr":"172.21.15.151:6829/3079373271","state":["exists","up"]},{"osd":3,"uuid":"36edb718-2fcb-4d88-8ddc-c321eed4279d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6817","nonce":1142509041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6819","nonce":1142509041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6824","nonce":1142509041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6821","nonce":1142509041}]},"public_addr":"172.21.15.151:6817/1142509041","cluster_addr":"172.21.15.151:6819/1142509041","heartbeat_back_addr":"172.21.15.151:6824/1142509041","heartbeat_front_addr":"172.21.15.151:6821/1142509041","state":["exists","up"]},{"osd":4,"uuid":"64d33708-cd0a-4236-abe3-1b646e82ab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6811","nonce":2735495271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6813","nonce":2735495271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6817","nonce":2735495271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6815","nonce":2735495271}]},"public_addr":"172.21.15.167:6811/2735495271","cluster_addr":"172.21.15.167:6813/2735495271","heartbeat_back_addr":"172.21.15.167:6817/2735495271","heartbeat_front_addr":"172.21.15.167:6815/2735495271","state":["exists","up"]},{"osd":5,"uuid":"f14b44a9-5910-458c-b518-09a8d0de94de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6827","nonce":1494916572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6829","nonce":1494916572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6833","nonce":1494916572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6831","nonce":1494916572}]},"public_addr":"172.21.15.167:6827/1494916572","cluster_addr":"172.21.15.167:6829/1494916572","heartbeat_back_addr":"172.21.15.167:6833/1494916572","heartbeat_front_addr":"172.21.15.167:6831/1494916572","state":["exists","up"]},{"osd":6,"uuid":"39335545-2654-4710-84b0-a9aa47cfcbba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6819","nonce":2538273273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6821","nonce":2538273273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6825","nonce":2538273273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6823","nonce":2538273273}]},"public_addr":"172.21.15.167:6819/2538273273","cluster_addr":"172.21.15.167:6821/2538273273","heartbeat_back_addr":"172.21.15.167:6825/2538273273","heartbeat_front_addr":"172.21.15.167:6823/2538273273","state":["exists","up"]},{"osd":7,"uuid":"4f8bfc81-53dd-4339-ac22-1a4007d9f963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6803","nonce":265072227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6805","nonce":265072227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6809","nonce":265072227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6807","nonce":265072227}]},"public_addr":"172.21.15.167:6803/265072227","cluster_addr":"172.21.15.167:6805/265072227","heartbeat_back_addr":"172.21.15.167:6809/265072227","heartbeat_front_addr":"172.21.15.167:6807/265072227","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.150359+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.084358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.722368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.332809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.320709+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.395153+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.935250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.801443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:39:16.354 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:16.353+0000 7f6e63200640 1 -- 172.21.15.151:0/2774350050 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6e44078850 msgr2=0x7f6e4407ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:16.354 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:16.353+0000 7f6e63200640 1 --2- 172.21.15.151:0/2774350050 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6e44078850 0x7f6e4407ad10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6e50034000 tx=0x7f6e50004730 comp rx=0 tx=0).stop 2024-09-16T15:39:16.354 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:16.353+0000 7f6e63200640 1 -- 172.21.15.151:0/2774350050 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6e640ce920 msgr2=0x7f6e6414cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:16.355 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:16.353+0000 7f6e63200640 1 --2- 172.21.15.151:0/2774350050 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6e640ce920 0x7f6e6414cdf0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6e5400c830 tx=0x7f6e5400ccf0 comp rx=0 tx=0).stop 2024-09-16T15:39:16.355 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:16.353+0000 7f6e63200640 1 -- 172.21.15.151:0/2774350050 shutdown_connections 2024-09-16T15:39:16.355 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:16.353+0000 7f6e63200640 1 --2- 172.21.15.151:0/2774350050 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6e44078850 0x7f6e4407ad10 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:16.355 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:16.353+0000 7f6e63200640 1 --2- 172.21.15.151:0/2774350050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e6415be40 0x7f6e641f5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:16.355 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:16.353+0000 7f6e63200640 1 --2- 172.21.15.151:0/2774350050 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e6415ba60 0x7f6e640c5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:16.355 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:16.353+0000 7f6e63200640 1 --2- 172.21.15.151:0/2774350050 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6e640ce920 0x7f6e6414cdf0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:16.355 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:16.353+0000 7f6e63200640 1 -- 172.21.15.151:0/2774350050 >> 172.21.15.151:0/2774350050 conn(0x7f6e640cc460 msgr2=0x7f6e64156f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:16.355 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:16.353+0000 7f6e63200640 1 -- 172.21.15.151:0/2774350050 shutdown_connections 2024-09-16T15:39:16.355 INFO:teuthology.misc.health.smithi151.stderr:2024-09-16T15:39:16.353+0000 7f6e63200640 1 -- 172.21.15.151:0/2774350050 wait complete. 2024-09-16T15:39:16.367 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-16T15:39:16.367 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-16T15:39:16.368 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-16T15:39:16.368 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-16T15:39:16.372 INFO:tasks.ceph.mds.a:Started 2024-09-16T15:39:16.372 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-16T15:39:16.372 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-16T15:39:16.376 INFO:tasks.ceph.mds.c:Started 2024-09-16T15:39:16.376 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-16T15:39:16.376 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-16T15:39:16.380 INFO:tasks.ceph.mds.b:Started 2024-09-16T15:39:16.380 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-16T15:39:16.381 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-16T15:39:16.385 INFO:tasks.ceph.mds.d:Started 2024-09-16T15:39:16.385 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-09-16T15:39:16.385 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:39:16.503 INFO:tasks.ceph.mds.a.smithi151.stdout:starting mds.a at 2024-09-16T15:39:16.503 INFO:tasks.ceph.mds.c.smithi151.stdout:starting mds.c at 2024-09-16T15:39:16.519 INFO:tasks.ceph.mds.b.smithi167.stdout:starting mds.b at 2024-09-16T15:39:16.519 INFO:tasks.ceph.mds.d.smithi167.stdout:starting mds.d at 2024-09-16T15:39:16.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:16.551+0000 7f5f3c800640 1 -- 172.21.15.151:0/906334105 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5f34070920 msgr2=0x7f5f341438b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:16.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:16.551+0000 7f5f3c800640 1 --2- 172.21.15.151:0/906334105 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5f34070920 0x7f5f341438b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5f3415b6d0 tx=0x7f5f2802a5f0 comp rx=0 tx=0).stop 2024-09-16T15:39:16.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:16.551+0000 7f5f3c800640 1 -- 172.21.15.151:0/906334105 shutdown_connections 2024-09-16T15:39:16.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:16.551+0000 7f5f3c800640 1 --2- 172.21.15.151:0/906334105 >> v2:172.21.15.167:3300/0 conn(0x7f5f340b3f00 0x7f5f340b4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:16.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:16.551+0000 7f5f3c800640 1 --2- 172.21.15.151:0/906334105 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5f34070920 0x7f5f341438b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:16.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:16.551+0000 7f5f3c800640 1 -- 172.21.15.151:0/906334105 >> 172.21.15.151:0/906334105 conn(0x7f5f340b1950 msgr2=0x7f5f340b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:16.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:16.551+0000 7f5f3c800640 1 -- 172.21.15.151:0/906334105 shutdown_connections 2024-09-16T15:39:16.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:16.551+0000 7f5f3c800640 1 -- 172.21.15.151:0/906334105 wait complete. 2024-09-16T15:39:16.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:16.553+0000 7f5f3c800640 1 Processor -- start 2024-09-16T15:39:16.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:16.553+0000 7f5f3c800640 1 -- start start 2024-09-16T15:39:16.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:16.554+0000 7f5f3c800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5f3406f850 0x7f5f3415a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:16.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:16.554+0000 7f5f3c800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5f34070920 0x7f5f34159180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:16.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:16.554+0000 7f5f3c800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f340b3f00 0x7f5f3409d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:16.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:16.554+0000 7f5f3c800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5f340b4d40 con 0x7f5f34070920 2024-09-16T15:39:16.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:16.554+0000 7f5f3c800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5f340b4bc0 con 0x7f5f340b3f00 2024-09-16T15:39:16.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:16.554+0000 7f5f3c800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f5f340b4ec0 con 0x7f5f3406f850 2024-09-16T15:39:16.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:16.554+0000 7f5f39400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5f34070920 0x7f5f34159180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:16.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:16.554+0000 7f5f39400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5f34070920 0x7f5f34159180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:35222/0 (socket says 172.21.15.151:35222) 2024-09-16T15:39:16.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:16.554+0000 7f5f39400640 1 -- 172.21.15.151:0/2636565400 learned_addr learned my addr 172.21.15.151:0/2636565400 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:16.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:16.554+0000 7f5f3a800640 1 --2- 172.21.15.151:0/2636565400 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f340b3f00 0x7f5f3409d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:16.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:16.554+0000 7f5f39e00640 1 --2- 172.21.15.151:0/2636565400 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5f3406f850 0x7f5f3415a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:16.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:16.554+0000 7f5f39400640 1 -- 172.21.15.151:0/2636565400 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5f3406f850 msgr2=0x7f5f3415a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:16.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:16.554+0000 7f5f39400640 1 --2- 172.21.15.151:0/2636565400 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5f3406f850 0x7f5f3415a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:16.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:16.555+0000 7f5f39400640 1 -- 172.21.15.151:0/2636565400 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f340b3f00 msgr2=0x7f5f3409d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:16.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:16.555+0000 7f5f39400640 1 --2- 172.21.15.151:0/2636565400 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f340b3f00 0x7f5f3409d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:16.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:16.555+0000 7f5f39400640 1 -- 172.21.15.151:0/2636565400 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f341596c0 con 0x7f5f34070920 2024-09-16T15:39:16.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:16.555+0000 7f5f39400640 1 --2- 172.21.15.151:0/2636565400 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5f34070920 0x7f5f34159180 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5f2400c730 tx=0x7f5f2400cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:16.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:16.555+0000 7f5f2e000640 1 -- 172.21.15.151:0/2636565400 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f240052d0 con 0x7f5f34070920 2024-09-16T15:39:16.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:16.555+0000 7f5f2e000640 1 -- 172.21.15.151:0/2636565400 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f24005490 con 0x7f5f34070920 2024-09-16T15:39:16.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:16.555+0000 7f5f2e000640 1 -- 172.21.15.151:0/2636565400 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2400a8a0 con 0x7f5f34070920 2024-09-16T15:39:16.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:16.555+0000 7f5f3c800640 1 -- 172.21.15.151:0/2636565400 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f341599e0 con 0x7f5f34070920 2024-09-16T15:39:16.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:16.555+0000 7f5f3c800640 1 -- 172.21.15.151:0/2636565400 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f34159fa0 con 0x7f5f34070920 2024-09-16T15:39:16.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:16.556+0000 7f5f3c800640 1 -- 172.21.15.151:0/2636565400 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f341438b0 con 0x7f5f34070920 2024-09-16T15:39:16.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:16.557+0000 7f5f2e000640 1 -- 172.21.15.151:0/2636565400 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f5f2400aa40 con 0x7f5f34070920 2024-09-16T15:39:16.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:16.558+0000 7f5f2e000640 1 --2- 172.21.15.151:0/2636565400 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5f18078850 0x7f5f1807ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:16.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:16.558+0000 7f5f2e000640 1 -- 172.21.15.151:0/2636565400 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f5f2409a080 con 0x7f5f34070920 2024-09-16T15:39:16.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:16.558+0000 7f5f39e00640 1 --2- 172.21.15.151:0/2636565400 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5f18078850 0x7f5f1807ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:16.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:16.559+0000 7f5f39e00640 1 --2- 172.21.15.151:0/2636565400 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5f18078850 0x7f5f1807ad10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5f28033000 tx=0x7f5f28004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:16.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:16.562+0000 7f5f2e000640 1 -- 172.21.15.151:0/2636565400 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f5f340cdae0 con 0x7f5f34070920 2024-09-16T15:39:16.730 DEBUG:teuthology.orchestra.run.smithi151:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:39:16.735 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:39:16.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:16.797+0000 7f5f3c800640 1 -- 172.21.15.151:0/2636565400 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5f340cd430 con 0x7f5f34070920 2024-09-16T15:39:16.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:16.797+0000 7f5f2e000640 1 -- 172.21.15.151:0/2636565400 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1437 (secure 0 0 0) 0x7f5f24062a30 con 0x7f5f34070920 2024-09-16T15:39:16.798 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:39:16.798 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":2,"btime":"2024-09-16T15:39:16:510678+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4341,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/3198287696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":3198287696},{"type":"v1","addr":"172.21.15.151:6835","nonce":3198287696}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-16T15:39:16.799 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 2 2024-09-16T15:39:16.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:16.800+0000 7f5f3c800640 1 -- 172.21.15.151:0/2636565400 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5f18078850 msgr2=0x7f5f1807ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:16.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:16.800+0000 7f5f3c800640 1 --2- 172.21.15.151:0/2636565400 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5f18078850 0x7f5f1807ad10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5f28033000 tx=0x7f5f28004730 comp rx=0 tx=0).stop 2024-09-16T15:39:16.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:16.800+0000 7f5f3c800640 1 -- 172.21.15.151:0/2636565400 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5f34070920 msgr2=0x7f5f34159180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:16.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:16.800+0000 7f5f3c800640 1 --2- 172.21.15.151:0/2636565400 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5f34070920 0x7f5f34159180 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5f2400c730 tx=0x7f5f2400cb30 comp rx=0 tx=0).stop 2024-09-16T15:39:16.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:16.800+0000 7f5f3c800640 1 -- 172.21.15.151:0/2636565400 shutdown_connections 2024-09-16T15:39:16.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:16.800+0000 7f5f3c800640 1 --2- 172.21.15.151:0/2636565400 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5f18078850 0x7f5f1807ad10 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:16.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:16.800+0000 7f5f3c800640 1 --2- 172.21.15.151:0/2636565400 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f340b3f00 0x7f5f3409d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:16.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:16.800+0000 7f5f3c800640 1 --2- 172.21.15.151:0/2636565400 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5f34070920 0x7f5f34159180 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:16.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:16.800+0000 7f5f3c800640 1 --2- 172.21.15.151:0/2636565400 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5f3406f850 0x7f5f3415a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:16.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:16.800+0000 7f5f3c800640 1 -- 172.21.15.151:0/2636565400 >> 172.21.15.151:0/2636565400 conn(0x7f5f340b1950 msgr2=0x7f5f34075940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:16.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:16.800+0000 7f5f3c800640 1 -- 172.21.15.151:0/2636565400 shutdown_connections 2024-09-16T15:39:16.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:16.800+0000 7f5f3c800640 1 -- 172.21.15.151:0/2636565400 wait complete. 2024-09-16T15:39:18.818 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:39:18.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:18.976+0000 7f2c2e400640 1 -- 172.21.15.151:0/644833334 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c280ccf00 msgr2=0x7f2c280acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:18.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:18.976+0000 7f2c2e400640 1 --2- 172.21.15.151:0/644833334 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c280ccf00 0x7f2c280acd30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2c28076720 tx=0x7f2c1402a4d0 comp rx=0 tx=0).stop 2024-09-16T15:39:18.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:18.976+0000 7f2c2e400640 1 -- 172.21.15.151:0/644833334 shutdown_connections 2024-09-16T15:39:18.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:18.976+0000 7f2c2e400640 1 --2- 172.21.15.151:0/644833334 >> v2:172.21.15.167:3300/0 conn(0x7f2c280ad270 0x7f2c28073fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:18.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:18.976+0000 7f2c2e400640 1 --2- 172.21.15.151:0/644833334 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c280ccf00 0x7f2c280acd30 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:18.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:18.976+0000 7f2c2e400640 1 --2- 172.21.15.151:0/644833334 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c2815f5a0 0x7f2c280b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:18.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:18.976+0000 7f2c2e400640 1 -- 172.21.15.151:0/644833334 >> 172.21.15.151:0/644833334 conn(0x7f2c280b1950 msgr2=0x7f2c280b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:18.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:18.976+0000 7f2c2e400640 1 -- 172.21.15.151:0/644833334 shutdown_connections 2024-09-16T15:39:18.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:18.976+0000 7f2c2e400640 1 -- 172.21.15.151:0/644833334 wait complete. 2024-09-16T15:39:18.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:18.977+0000 7f2c2e400640 1 Processor -- start 2024-09-16T15:39:18.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:18.977+0000 7f2c2e400640 1 -- start start 2024-09-16T15:39:18.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:18.978+0000 7f2c2e400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c280ad270 0x7f2c28145870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:18.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:18.978+0000 7f2c2e400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c280ccf00 0x7f2c280caf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:18.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:18.978+0000 7f2c2e400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c2815f5a0 0x7f2c280cb490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:18.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:18.978+0000 7f2c2e400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2806f160 con 0x7f2c280ad270 2024-09-16T15:39:18.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:18.978+0000 7f2c2e400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2c28074780 con 0x7f2c280ccf00 2024-09-16T15:39:18.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:18.978+0000 7f2c2e400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f2c2806f2e0 con 0x7f2c2815f5a0 2024-09-16T15:39:18.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:18.979+0000 7f2c27400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c280ad270 0x7f2c28145870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:18.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:18.979+0000 7f2c27400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c280ad270 0x7f2c28145870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:35244/0 (socket says 172.21.15.151:35244) 2024-09-16T15:39:18.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:18.979+0000 7f2c27400640 1 -- 172.21.15.151:0/1262627683 learned_addr learned my addr 172.21.15.151:0/1262627683 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:18.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:18.979+0000 7f2c26a00640 1 --2- 172.21.15.151:0/1262627683 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c280ccf00 0x7f2c280caf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:18.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:18.979+0000 7f2c27e00640 1 --2- 172.21.15.151:0/1262627683 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c2815f5a0 0x7f2c280cb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:18.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:18.979+0000 7f2c27400640 1 -- 172.21.15.151:0/1262627683 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c2815f5a0 msgr2=0x7f2c280cb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:18.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:18.979+0000 7f2c27400640 1 --2- 172.21.15.151:0/1262627683 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c2815f5a0 0x7f2c280cb490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:18.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:18.979+0000 7f2c27400640 1 -- 172.21.15.151:0/1262627683 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c280ccf00 msgr2=0x7f2c280caf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:18.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:18.979+0000 7f2c27400640 1 --2- 172.21.15.151:0/1262627683 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c280ccf00 0x7f2c280caf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:18.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:18.979+0000 7f2c27400640 1 -- 172.21.15.151:0/1262627683 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c280c6b40 con 0x7f2c280ad270 2024-09-16T15:39:18.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:18.979+0000 7f2c26a00640 1 --2- 172.21.15.151:0/1262627683 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c280ccf00 0x7f2c280caf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:39:18.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:18.979+0000 7f2c27400640 1 --2- 172.21.15.151:0/1262627683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c280ad270 0x7f2c28145870 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2c1800e7c0 tx=0x7f2c1800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:18.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:18.979+0000 7f2c1f400640 1 -- 172.21.15.151:0/1262627683 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c1800ccc0 con 0x7f2c280ad270 2024-09-16T15:39:18.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:18.979+0000 7f2c1f400640 1 -- 172.21.15.151:0/1262627683 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2c1800ce80 con 0x7f2c280ad270 2024-09-16T15:39:18.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:18.979+0000 7f2c1f400640 1 -- 172.21.15.151:0/1262627683 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c18005280 con 0x7f2c280ad270 2024-09-16T15:39:18.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:18.980+0000 7f2c2e400640 1 -- 172.21.15.151:0/1262627683 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c280c6e20 con 0x7f2c280ad270 2024-09-16T15:39:18.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:18.980+0000 7f2c2e400640 1 -- 172.21.15.151:0/1262627683 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c280c73e0 con 0x7f2c280ad270 2024-09-16T15:39:18.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:18.981+0000 7f2c2e400640 1 -- 172.21.15.151:0/1262627683 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c2806f810 con 0x7f2c280ad270 2024-09-16T15:39:18.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:18.982+0000 7f2c1f400640 1 -- 172.21.15.151:0/1262627683 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f2c18005420 con 0x7f2c280ad270 2024-09-16T15:39:18.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:18.983+0000 7f2c1f400640 1 --2- 172.21.15.151:0/1262627683 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2c08078850 0x7f2c0807ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:18.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:18.983+0000 7f2c1f400640 1 -- 172.21.15.151:0/1262627683 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f2c1809aff0 con 0x7f2c280ad270 2024-09-16T15:39:18.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:18.983+0000 7f2c26a00640 1 --2- 172.21.15.151:0/1262627683 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2c08078850 0x7f2c0807ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:18.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:18.984+0000 7f2c26a00640 1 --2- 172.21.15.151:0/1262627683 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2c08078850 0x7f2c0807ad10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2c14033000 tx=0x7f2c14004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:18.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:18.987+0000 7f2c1f400640 1 -- 172.21.15.151:0/1262627683 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2c18063950 con 0x7f2c280ad270 2024-09-16T15:39:19.207 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.206+0000 7f2c2e400640 1 -- 172.21.15.151:0/1262627683 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2c280b0dc0 con 0x7f2c280ad270 2024-09-16T15:39:19.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.206+0000 7f2c1f400640 1 -- 172.21.15.151:0/1262627683 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3918 (secure 0 0 0) 0x7f2c18063250 con 0x7f2c280ad270 2024-09-16T15:39:19.208 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:39:19.208 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":3,"btime":"2024-09-16T15:39:17:513738+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4301,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/2005239755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2005239755},{"type":"v1","addr":"172.21.15.167:6836","nonce":2005239755}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4338,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/3568704323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3568704323},{"type":"v1","addr":"172.21.15.151:6834","nonce":3568704323}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4341,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/3198287696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":3198287696},{"type":"v1","addr":"172.21.15.151:6835","nonce":3198287696}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4350,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3014501393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3014501393},{"type":"v1","addr":"172.21.15.167:6837","nonce":3014501393}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-16T15:39:19.208 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 3 2024-09-16T15:39:19.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.209+0000 7f2c2e400640 1 -- 172.21.15.151:0/1262627683 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2c08078850 msgr2=0x7f2c0807ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:19.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.209+0000 7f2c2e400640 1 --2- 172.21.15.151:0/1262627683 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2c08078850 0x7f2c0807ad10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2c14033000 tx=0x7f2c14004730 comp rx=0 tx=0).stop 2024-09-16T15:39:19.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.209+0000 7f2c2e400640 1 -- 172.21.15.151:0/1262627683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c280ad270 msgr2=0x7f2c28145870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:19.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.209+0000 7f2c2e400640 1 --2- 172.21.15.151:0/1262627683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c280ad270 0x7f2c28145870 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2c1800e7c0 tx=0x7f2c1800ec80 comp rx=0 tx=0).stop 2024-09-16T15:39:19.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.209+0000 7f2c2e400640 1 -- 172.21.15.151:0/1262627683 shutdown_connections 2024-09-16T15:39:19.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.209+0000 7f2c2e400640 1 --2- 172.21.15.151:0/1262627683 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2c08078850 0x7f2c0807ad10 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:19.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.209+0000 7f2c2e400640 1 --2- 172.21.15.151:0/1262627683 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c280ccf00 0x7f2c280caf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:19.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.209+0000 7f2c2e400640 1 --2- 172.21.15.151:0/1262627683 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c2815f5a0 0x7f2c280cb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:19.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.209+0000 7f2c2e400640 1 --2- 172.21.15.151:0/1262627683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c280ad270 0x7f2c28145870 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:19.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.209+0000 7f2c2e400640 1 -- 172.21.15.151:0/1262627683 >> 172.21.15.151:0/1262627683 conn(0x7f2c280b1950 msgr2=0x7f2c2809c610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:19.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.209+0000 7f2c2e400640 1 -- 172.21.15.151:0/1262627683 shutdown_connections 2024-09-16T15:39:19.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.209+0000 7f2c2e400640 1 -- 172.21.15.151:0/1262627683 wait complete. 2024-09-16T15:39:19.225 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-16T15:39:19.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.382+0000 7fd5ca600640 1 -- 172.21.15.151:0/2651479847 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5c40b34b0 msgr2=0x7fd5c40b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:19.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.382+0000 7fd5ca600640 1 --2- 172.21.15.151:0/2651479847 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5c40b34b0 0x7fd5c40b3940 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd5c40b6930 tx=0x7fd5b002a4d0 comp rx=0 tx=0).stop 2024-09-16T15:39:19.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.383+0000 7fd5ca600640 1 -- 172.21.15.151:0/2651479847 shutdown_connections 2024-09-16T15:39:19.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.383+0000 7fd5ca600640 1 --2- 172.21.15.151:0/2651479847 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5c40b34b0 0x7fd5c40b3940 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:19.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.383+0000 7fd5ca600640 1 --2- 172.21.15.151:0/2651479847 >> v2:172.21.15.167:3300/0 conn(0x7fd5c4161440 0x7fd5c40b5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:19.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.383+0000 7fd5ca600640 1 -- 172.21.15.151:0/2651479847 >> 172.21.15.151:0/2651479847 conn(0x7fd5c40d3df0 msgr2=0x7fd5c40d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:19.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.383+0000 7fd5ca600640 1 -- 172.21.15.151:0/2651479847 shutdown_connections 2024-09-16T15:39:19.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.383+0000 7fd5ca600640 1 -- 172.21.15.151:0/2651479847 wait complete. 2024-09-16T15:39:19.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.384+0000 7fd5ca600640 1 Processor -- start 2024-09-16T15:39:19.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.384+0000 7fd5ca600640 1 -- start start 2024-09-16T15:39:19.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.385+0000 7fd5ca600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5c40b34b0 0x7fd5c40d11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:19.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.385+0000 7fd5ca600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5c40b3e80 0x7fd5c41557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:19.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.385+0000 7fd5ca600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5c4161440 0x7fd5c415ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:19.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.385+0000 7fd5ca600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c40b2900 con 0x7fd5c40b34b0 2024-09-16T15:39:19.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.385+0000 7fd5ca600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c40b2780 con 0x7fd5c4161440 2024-09-16T15:39:19.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.385+0000 7fd5ca600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fd5c40b2a80 con 0x7fd5c40b3e80 2024-09-16T15:39:19.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.386+0000 7fd5c9200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5c40b34b0 0x7fd5c40d11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:19.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.386+0000 7fd5c9200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5c40b34b0 0x7fd5c40d11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:35264/0 (socket says 172.21.15.151:35264) 2024-09-16T15:39:19.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.386+0000 7fd5c9200640 1 -- 172.21.15.151:0/1711190457 learned_addr learned my addr 172.21.15.151:0/1711190457 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:19.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.386+0000 7fd5c3e00640 1 --2- 172.21.15.151:0/1711190457 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5c40b3e80 0x7fd5c41557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:19.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.386+0000 7fd5c9c00640 1 --2- 172.21.15.151:0/1711190457 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5c4161440 0x7fd5c415ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:19.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.386+0000 7fd5c9200640 1 -- 172.21.15.151:0/1711190457 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5c40b3e80 msgr2=0x7fd5c41557f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:19.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.386+0000 7fd5c9200640 1 --2- 172.21.15.151:0/1711190457 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5c40b3e80 0x7fd5c41557f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:19.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.386+0000 7fd5c9200640 1 -- 172.21.15.151:0/1711190457 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5c4161440 msgr2=0x7fd5c415ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:19.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.386+0000 7fd5c9200640 1 --2- 172.21.15.151:0/1711190457 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5c4161440 0x7fd5c415ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:19.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.386+0000 7fd5c9200640 1 -- 172.21.15.151:0/1711190457 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5c415b500 con 0x7fd5c40b34b0 2024-09-16T15:39:19.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.386+0000 7fd5c9200640 1 --2- 172.21.15.151:0/1711190457 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5c40b34b0 0x7fd5c40d11c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd5b800d6d0 tx=0x7fd5b800db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:19.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.386+0000 7fd5c0c00640 1 -- 172.21.15.151:0/1711190457 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b8017070 con 0x7fd5c40b34b0 2024-09-16T15:39:19.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.387+0000 7fd5c0c00640 1 -- 172.21.15.151:0/1711190457 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd5b8005330 con 0x7fd5c40b34b0 2024-09-16T15:39:19.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.387+0000 7fd5ca600640 1 -- 172.21.15.151:0/1711190457 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5c415b7e0 con 0x7fd5c40b34b0 2024-09-16T15:39:19.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.387+0000 7fd5c0c00640 1 -- 172.21.15.151:0/1711190457 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b8009820 con 0x7fd5c40b34b0 2024-09-16T15:39:19.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.387+0000 7fd5ca600640 1 -- 172.21.15.151:0/1711190457 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5c40cfc60 con 0x7fd5c40b34b0 2024-09-16T15:39:19.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.388+0000 7fd5ca600640 1 -- 172.21.15.151:0/1711190457 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5c40b3940 con 0x7fd5c40b34b0 2024-09-16T15:39:19.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.389+0000 7fd5c0c00640 1 -- 172.21.15.151:0/1711190457 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fd5b8009bd0 con 0x7fd5c40b34b0 2024-09-16T15:39:19.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.390+0000 7fd5c0c00640 1 --2- 172.21.15.151:0/1711190457 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd5a4078850 0x7fd5a407ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:19.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.390+0000 7fd5c0c00640 1 -- 172.21.15.151:0/1711190457 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7fd5b809a8e0 con 0x7fd5c40b34b0 2024-09-16T15:39:19.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.391+0000 7fd5c3e00640 1 --2- 172.21.15.151:0/1711190457 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd5a4078850 0x7fd5a407ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:19.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.391+0000 7fd5c3e00640 1 --2- 172.21.15.151:0/1711190457 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd5a4078850 0x7fd5a407ad10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0033000 tx=0x7fd5b0004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:19.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.394+0000 7fd5c0c00640 1 -- 172.21.15.151:0/1711190457 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd5b8063290 con 0x7fd5c40b34b0 2024-09-16T15:39:19.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.585+0000 7fd5ca600640 1 -- 172.21.15.151:0/1711190457 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fd5c406dcf0 con 0x7fd5c40b34b0 2024-09-16T15:39:19.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.586+0000 7fd5c0c00640 1 -- 172.21.15.151:0/1711190457 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v18) ==== 84+0+65 (secure 0 0 0) 0x7fd5b8062b90 con 0x7fd5c40b34b0 2024-09-16T15:39:19.587 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:39:19.587 INFO:teuthology.orchestra.run.smithi151.stdout:[ 2024-09-16T15:39:19.587 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:39:19.587 INFO:teuthology.orchestra.run.smithi151.stdout: "poolnum": 1, 2024-09-16T15:39:19.587 INFO:teuthology.orchestra.run.smithi151.stdout: "poolname": ".mgr" 2024-09-16T15:39:19.587 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:39:19.587 INFO:teuthology.orchestra.run.smithi151.stdout:] 2024-09-16T15:39:19.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.589+0000 7fd5ca600640 1 -- 172.21.15.151:0/1711190457 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd5a4078850 msgr2=0x7fd5a407ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:19.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.589+0000 7fd5ca600640 1 --2- 172.21.15.151:0/1711190457 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd5a4078850 0x7fd5a407ad10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0033000 tx=0x7fd5b0004730 comp rx=0 tx=0).stop 2024-09-16T15:39:19.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.589+0000 7fd5ca600640 1 -- 172.21.15.151:0/1711190457 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5c40b34b0 msgr2=0x7fd5c40d11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:19.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.589+0000 7fd5ca600640 1 --2- 172.21.15.151:0/1711190457 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5c40b34b0 0x7fd5c40d11c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd5b800d6d0 tx=0x7fd5b800db90 comp rx=0 tx=0).stop 2024-09-16T15:39:19.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.589+0000 7fd5ca600640 1 -- 172.21.15.151:0/1711190457 shutdown_connections 2024-09-16T15:39:19.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.589+0000 7fd5ca600640 1 --2- 172.21.15.151:0/1711190457 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd5a4078850 0x7fd5a407ad10 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:19.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.589+0000 7fd5ca600640 1 --2- 172.21.15.151:0/1711190457 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5c4161440 0x7fd5c415ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:19.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.589+0000 7fd5ca600640 1 --2- 172.21.15.151:0/1711190457 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5c40b3e80 0x7fd5c41557f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:19.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.589+0000 7fd5ca600640 1 --2- 172.21.15.151:0/1711190457 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5c40b34b0 0x7fd5c40d11c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:19.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.589+0000 7fd5ca600640 1 -- 172.21.15.151:0/1711190457 >> 172.21.15.151:0/1711190457 conn(0x7fd5c40d3df0 msgr2=0x7fd5c4158780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:19.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.589+0000 7fd5ca600640 1 -- 172.21.15.151:0/1711190457 shutdown_connections 2024-09-16T15:39:19.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.589+0000 7fd5ca600640 1 -- 172.21.15.151:0/1711190457 wait complete. 2024-09-16T15:39:19.606 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-16T15:39:19.606 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-16T15:39:19.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.765+0000 7fa221000640 1 -- 172.21.15.151:0/3801072086 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa21c15c350 msgr2=0x7fa21c15c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:19.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.765+0000 7fa221000640 1 --2- 172.21.15.151:0/3801072086 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa21c15c350 0x7fa21c15c800 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa21c069890 tx=0x7fa21402aac0 comp rx=0 tx=0).stop 2024-09-16T15:39:19.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.765+0000 7fa221000640 1 -- 172.21.15.151:0/3801072086 shutdown_connections 2024-09-16T15:39:19.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.765+0000 7fa221000640 1 --2- 172.21.15.151:0/3801072086 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa21c15c350 0x7fa21c15c800 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:19.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.765+0000 7fa221000640 1 --2- 172.21.15.151:0/3801072086 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa21c0733e0 0x7fa21c0cce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:19.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.765+0000 7fa221000640 1 --2- 172.21.15.151:0/3801072086 >> v2:172.21.15.167:3300/0 conn(0x7fa21c15f910 0x7fa21c15fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:19.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.765+0000 7fa221000640 1 -- 172.21.15.151:0/3801072086 >> 172.21.15.151:0/3801072086 conn(0x7fa21c0b18e0 msgr2=0x7fa21c0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:19.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.765+0000 7fa221000640 1 -- 172.21.15.151:0/3801072086 shutdown_connections 2024-09-16T15:39:19.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.765+0000 7fa221000640 1 -- 172.21.15.151:0/3801072086 wait complete. 2024-09-16T15:39:19.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.767+0000 7fa221000640 1 Processor -- start 2024-09-16T15:39:19.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.767+0000 7fa221000640 1 -- start start 2024-09-16T15:39:19.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.768+0000 7fa221000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa21c15c350 0x7fa21c0aea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:19.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.768+0000 7fa221000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa21c15f910 0x7fa21c155850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:19.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.768+0000 7fa221000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa21c157230 0x7fa21c0ceb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:19.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.768+0000 7fa221000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa21c15cd40 con 0x7fa21c15f910 2024-09-16T15:39:19.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.768+0000 7fa221000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa21c0cd5a0 con 0x7fa21c157230 2024-09-16T15:39:19.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.768+0000 7fa221000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fa21c15cec0 con 0x7fa21c15c350 2024-09-16T15:39:19.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.768+0000 7fa219400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa21c15f910 0x7fa21c155850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:19.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.768+0000 7fa219400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa21c15f910 0x7fa21c155850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:35288/0 (socket says 172.21.15.151:35288) 2024-09-16T15:39:19.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.768+0000 7fa219400640 1 -- 172.21.15.151:0/1856348411 learned_addr learned my addr 172.21.15.151:0/1856348411 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:19.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.768+0000 7fa219e00640 1 --2- 172.21.15.151:0/1856348411 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa21c15c350 0x7fa21c0aea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:19.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.768+0000 7fa21a800640 1 --2- 172.21.15.151:0/1856348411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa21c157230 0x7fa21c0ceb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:19.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.769+0000 7fa219400640 1 -- 172.21.15.151:0/1856348411 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa21c15c350 msgr2=0x7fa21c0aea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:19.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.769+0000 7fa219400640 1 --2- 172.21.15.151:0/1856348411 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa21c15c350 0x7fa21c0aea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:19.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.769+0000 7fa219400640 1 -- 172.21.15.151:0/1856348411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa21c157230 msgr2=0x7fa21c0ceb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:19.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.769+0000 7fa219400640 1 --2- 172.21.15.151:0/1856348411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa21c157230 0x7fa21c0ceb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:19.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.769+0000 7fa219400640 1 -- 172.21.15.151:0/1856348411 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa21c1f8710 con 0x7fa21c15f910 2024-09-16T15:39:19.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.769+0000 7fa21a800640 1 --2- 172.21.15.151:0/1856348411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa21c157230 0x7fa21c0ceb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:39:19.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.769+0000 7fa219400640 1 --2- 172.21.15.151:0/1856348411 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa21c15f910 0x7fa21c155850 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa20800d770 tx=0x7fa20800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:19.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.769+0000 7fa212000640 1 -- 172.21.15.151:0/1856348411 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa208017070 con 0x7fa21c15f910 2024-09-16T15:39:19.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.769+0000 7fa212000640 1 -- 172.21.15.151:0/1856348411 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa208009d20 con 0x7fa21c15f910 2024-09-16T15:39:19.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.769+0000 7fa212000640 1 -- 172.21.15.151:0/1856348411 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa208005280 con 0x7fa21c15f910 2024-09-16T15:39:19.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.769+0000 7fa221000640 1 -- 172.21.15.151:0/1856348411 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa21c1f89f0 con 0x7fa21c15f910 2024-09-16T15:39:19.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.769+0000 7fa221000640 1 -- 172.21.15.151:0/1856348411 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa21c1f8fb0 con 0x7fa21c15f910 2024-09-16T15:39:19.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.770+0000 7fa221000640 1 -- 172.21.15.151:0/1856348411 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa21c069360 con 0x7fa21c15f910 2024-09-16T15:39:19.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.771+0000 7fa212000640 1 -- 172.21.15.151:0/1856348411 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fa208005420 con 0x7fa21c15f910 2024-09-16T15:39:19.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.772+0000 7fa212000640 1 --2- 172.21.15.151:0/1856348411 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa1fc078850 0x7fa1fc07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:19.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.772+0000 7fa212000640 1 -- 172.21.15.151:0/1856348411 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7fa20809a6f0 con 0x7fa21c15f910 2024-09-16T15:39:19.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.773+0000 7fa219e00640 1 --2- 172.21.15.151:0/1856348411 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa1fc078850 0x7fa1fc07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:19.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.773+0000 7fa219e00640 1 --2- 172.21.15.151:0/1856348411 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa1fc078850 0x7fa1fc07ad10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa20c00bad0 tx=0x7fa20c00b2e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:19.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.777+0000 7fa212000640 1 -- 172.21.15.151:0/1856348411 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa208062a50 con 0x7fa21c15f910 2024-09-16T15:39:19.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:19.961+0000 7fa221000640 1 -- 172.21.15.151:0/1856348411 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fa21c15c800 con 0x7fa21c15f910 2024-09-16T15:39:20.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:20.530+0000 7fa212000640 1 -- 172.21.15.151:0/1856348411 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v19) ==== 138+0+0 (secure 0 0 0) 0x7fa208067480 con 0x7fa21c15f910 2024-09-16T15:39:20.532 INFO:teuthology.orchestra.run.smithi151.stderr:pool 'cephfs_metadata' created 2024-09-16T15:39:20.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:20.533+0000 7fa221000640 1 -- 172.21.15.151:0/1856348411 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa1fc078850 msgr2=0x7fa1fc07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:20.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:20.533+0000 7fa221000640 1 --2- 172.21.15.151:0/1856348411 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa1fc078850 0x7fa1fc07ad10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa20c00bad0 tx=0x7fa20c00b2e0 comp rx=0 tx=0).stop 2024-09-16T15:39:20.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:20.533+0000 7fa221000640 1 -- 172.21.15.151:0/1856348411 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa21c15f910 msgr2=0x7fa21c155850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:20.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:20.533+0000 7fa221000640 1 --2- 172.21.15.151:0/1856348411 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa21c15f910 0x7fa21c155850 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa20800d770 tx=0x7fa20800dc30 comp rx=0 tx=0).stop 2024-09-16T15:39:20.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:20.533+0000 7fa221000640 1 -- 172.21.15.151:0/1856348411 shutdown_connections 2024-09-16T15:39:20.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:20.533+0000 7fa221000640 1 --2- 172.21.15.151:0/1856348411 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa1fc078850 0x7fa1fc07ad10 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:20.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:20.533+0000 7fa221000640 1 --2- 172.21.15.151:0/1856348411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa21c157230 0x7fa21c0ceb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:20.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:20.533+0000 7fa221000640 1 --2- 172.21.15.151:0/1856348411 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa21c15f910 0x7fa21c155850 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:20.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:20.533+0000 7fa221000640 1 --2- 172.21.15.151:0/1856348411 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa21c15c350 0x7fa21c0aea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:20.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:20.533+0000 7fa221000640 1 -- 172.21.15.151:0/1856348411 >> 172.21.15.151:0/1856348411 conn(0x7fa21c0b18e0 msgr2=0x7fa21c0cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:20.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:20.533+0000 7fa221000640 1 -- 172.21.15.151:0/1856348411 shutdown_connections 2024-09-16T15:39:20.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:20.533+0000 7fa221000640 1 -- 172.21.15.151:0/1856348411 wait complete. 2024-09-16T15:39:20.555 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-16T15:39:20.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:20.718+0000 7f2430000640 1 -- 172.21.15.151:0/1718947298 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f24280cccd0 msgr2=0x7f2428073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:20.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:20.718+0000 7f2430000640 1 --2- 172.21.15.151:0/1718947298 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f24280cccd0 0x7f2428073440 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2414009830 tx=0x7f241402a480 comp rx=0 tx=0).stop 2024-09-16T15:39:20.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:20.718+0000 7f2430000640 1 -- 172.21.15.151:0/1718947298 shutdown_connections 2024-09-16T15:39:20.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:20.718+0000 7f2430000640 1 --2- 172.21.15.151:0/1718947298 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f242806efa0 0x7f24280cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:20.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:20.718+0000 7f2430000640 1 --2- 172.21.15.151:0/1718947298 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f24280cccd0 0x7f2428073440 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:20.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:20.718+0000 7f2430000640 1 -- 172.21.15.151:0/1718947298 >> 172.21.15.151:0/1718947298 conn(0x7f24280b1840 msgr2=0x7f24280b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:20.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:20.718+0000 7f2430000640 1 -- 172.21.15.151:0/1718947298 shutdown_connections 2024-09-16T15:39:20.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:20.719+0000 7f2430000640 1 -- 172.21.15.151:0/1718947298 wait complete. 2024-09-16T15:39:20.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:20.720+0000 7f2430000640 1 Processor -- start 2024-09-16T15:39:20.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:20.720+0000 7f2430000640 1 -- start start 2024-09-16T15:39:20.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:20.721+0000 7f2430000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f242806efa0 0x7f24280b2c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:20.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:20.721+0000 7f2430000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24280cccd0 0x7f24280b42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:20.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:20.721+0000 7f2430000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2428145750 0x7f24281f7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:20.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:20.721+0000 7f2430000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f24280ce480 con 0x7f2428145750 2024-09-16T15:39:20.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:20.721+0000 7f2430000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f24280ce300 con 0x7f242806efa0 2024-09-16T15:39:20.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:20.721+0000 7f2430000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f24280ce600 con 0x7f24280cccd0 2024-09-16T15:39:20.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:20.721+0000 7f242e000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2428145750 0x7f24281f7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:20.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:20.721+0000 7f242e000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2428145750 0x7f24281f7e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:35302/0 (socket says 172.21.15.151:35302) 2024-09-16T15:39:20.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:20.721+0000 7f242e000640 1 -- 172.21.15.151:0/3452062705 learned_addr learned my addr 172.21.15.151:0/3452062705 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:20.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:20.722+0000 7f242d600640 1 --2- 172.21.15.151:0/3452062705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f242806efa0 0x7f24280b2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:20.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:20.722+0000 7f242cc00640 1 --2- 172.21.15.151:0/3452062705 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24280cccd0 0x7f24280b42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:20.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:20.722+0000 7f242e000640 1 -- 172.21.15.151:0/3452062705 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24280cccd0 msgr2=0x7f24280b42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:20.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:20.722+0000 7f242e000640 1 --2- 172.21.15.151:0/3452062705 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24280cccd0 0x7f24280b42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:20.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:20.722+0000 7f242e000640 1 -- 172.21.15.151:0/3452062705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f242806efa0 msgr2=0x7f24280b2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:20.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:20.722+0000 7f242e000640 1 --2- 172.21.15.151:0/3452062705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f242806efa0 0x7f24280b2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:20.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:20.722+0000 7f242e000640 1 -- 172.21.15.151:0/3452062705 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24280b3220 con 0x7f2428145750 2024-09-16T15:39:20.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:20.722+0000 7f242e000640 1 --2- 172.21.15.151:0/3452062705 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2428145750 0x7f24281f7e90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f242000d740 tx=0x7f242000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:20.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:20.722+0000 7f2425600640 1 -- 172.21.15.151:0/3452062705 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2420017070 con 0x7f2428145750 2024-09-16T15:39:20.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:20.722+0000 7f2425600640 1 -- 172.21.15.151:0/3452062705 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2420009d20 con 0x7f2428145750 2024-09-16T15:39:20.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:20.722+0000 7f2425600640 1 -- 172.21.15.151:0/3452062705 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2420004280 con 0x7f2428145750 2024-09-16T15:39:20.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:20.722+0000 7f2430000640 1 -- 172.21.15.151:0/3452062705 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24280b3500 con 0x7f2428145750 2024-09-16T15:39:20.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:20.723+0000 7f2430000640 1 -- 172.21.15.151:0/3452062705 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24280b3da0 con 0x7f2428145750 2024-09-16T15:39:20.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:20.724+0000 7f2430000640 1 -- 172.21.15.151:0/3452062705 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24280b38f0 con 0x7f2428145750 2024-09-16T15:39:20.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:20.725+0000 7f2425600640 1 -- 172.21.15.151:0/3452062705 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f24200035f0 con 0x7f2428145750 2024-09-16T15:39:20.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:20.726+0000 7f2425600640 1 --2- 172.21.15.151:0/3452062705 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2408078850 0x7f240807ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:20.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:20.726+0000 7f2425600640 1 -- 172.21.15.151:0/3452062705 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5227+0+0 (secure 0 0 0) 0x7f242009a7a0 con 0x7f2428145750 2024-09-16T15:39:20.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:20.727+0000 7f242d600640 1 --2- 172.21.15.151:0/3452062705 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2408078850 0x7f240807ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:20.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:20.727+0000 7f242d600640 1 --2- 172.21.15.151:0/3452062705 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2408078850 0x7f240807ad10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f241c0017a0 tx=0x7f241c001730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:20.732 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:20.731+0000 7f2425600640 1 -- 172.21.15.151:0/3452062705 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2420062990 con 0x7f2428145750 2024-09-16T15:39:20.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:20.920+0000 7f2430000640 1 -- 172.21.15.151:0/3452062705 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f24280cdbb0 con 0x7f2428145750 2024-09-16T15:39:21.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:21.538+0000 7f2425600640 1 -- 172.21.15.151:0/3452062705 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v20) ==== 170+0+0 (secure 0 0 0) 0x7f24200673c0 con 0x7f2428145750 2024-09-16T15:39:21.539 INFO:teuthology.orchestra.run.smithi151.stderr:pool 'cephfs_data' created 2024-09-16T15:39:21.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:21.542+0000 7f2430000640 1 -- 172.21.15.151:0/3452062705 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2408078850 msgr2=0x7f240807ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:21.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:21.542+0000 7f2430000640 1 --2- 172.21.15.151:0/3452062705 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2408078850 0x7f240807ad10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f241c0017a0 tx=0x7f241c001730 comp rx=0 tx=0).stop 2024-09-16T15:39:21.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:21.542+0000 7f2430000640 1 -- 172.21.15.151:0/3452062705 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2428145750 msgr2=0x7f24281f7e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:21.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:21.542+0000 7f2430000640 1 --2- 172.21.15.151:0/3452062705 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2428145750 0x7f24281f7e90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f242000d740 tx=0x7f242000dc00 comp rx=0 tx=0).stop 2024-09-16T15:39:21.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:21.542+0000 7f2430000640 1 -- 172.21.15.151:0/3452062705 shutdown_connections 2024-09-16T15:39:21.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:21.542+0000 7f2430000640 1 --2- 172.21.15.151:0/3452062705 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2408078850 0x7f240807ad10 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:21.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:21.542+0000 7f2430000640 1 --2- 172.21.15.151:0/3452062705 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2428145750 0x7f24281f7e90 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:21.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:21.542+0000 7f2430000640 1 --2- 172.21.15.151:0/3452062705 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24280cccd0 0x7f24280b42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:21.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:21.542+0000 7f2430000640 1 --2- 172.21.15.151:0/3452062705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f242806efa0 0x7f24280b2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:21.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:21.542+0000 7f2430000640 1 -- 172.21.15.151:0/3452062705 >> 172.21.15.151:0/3452062705 conn(0x7f24280b1840 msgr2=0x7f24280a9460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:21.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:21.544+0000 7f2430000640 1 -- 172.21.15.151:0/3452062705 shutdown_connections 2024-09-16T15:39:21.546 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:21.545+0000 7f2430000640 1 -- 172.21.15.151:0/3452062705 wait complete. 2024-09-16T15:39:21.576 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-16T15:39:21.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:21.735+0000 7f995c200640 1 -- 172.21.15.151:0/1168663250 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99540cccd0 msgr2=0x7f9954073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:21.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:21.735+0000 7f995c200640 1 --2- 172.21.15.151:0/1168663250 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99540cccd0 0x7f9954073440 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9940009830 tx=0x7f994002a480 comp rx=0 tx=0).stop 2024-09-16T15:39:21.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:21.735+0000 7f995c200640 1 -- 172.21.15.151:0/1168663250 shutdown_connections 2024-09-16T15:39:21.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:21.735+0000 7f995c200640 1 --2- 172.21.15.151:0/1168663250 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f995406efa0 0x7f99540cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:21.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:21.735+0000 7f995c200640 1 --2- 172.21.15.151:0/1168663250 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99540cccd0 0x7f9954073440 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:21.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:21.735+0000 7f995c200640 1 -- 172.21.15.151:0/1168663250 >> 172.21.15.151:0/1168663250 conn(0x7f99540b1840 msgr2=0x7f99540b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:21.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:21.735+0000 7f995c200640 1 -- 172.21.15.151:0/1168663250 shutdown_connections 2024-09-16T15:39:21.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:21.735+0000 7f995c200640 1 -- 172.21.15.151:0/1168663250 wait complete. 2024-09-16T15:39:21.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:21.736+0000 7f995c200640 1 Processor -- start 2024-09-16T15:39:21.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:21.736+0000 7f995c200640 1 -- start start 2024-09-16T15:39:21.740 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:21.737+0000 7f995c200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f995406efa0 0x7f995415edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:21.740 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:21.737+0000 7f995c200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99540cccd0 0x7f995413bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:21.740 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:21.737+0000 7f995c200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9954145750 0x7f99540aabb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:21.740 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:21.737+0000 7f995c200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f99540ce480 con 0x7f995406efa0 2024-09-16T15:39:21.740 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:21.737+0000 7f995c200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f99540ce300 con 0x7f99540cccd0 2024-09-16T15:39:21.740 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:21.737+0000 7f995c200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f99540ce600 con 0x7f9954145750 2024-09-16T15:39:21.740 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:21.738+0000 7f9959800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f995406efa0 0x7f995415edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:21.740 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:21.738+0000 7f9959800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f995406efa0 0x7f995415edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:35314/0 (socket says 172.21.15.151:35314) 2024-09-16T15:39:21.740 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:21.738+0000 7f9959800640 1 -- 172.21.15.151:0/519876001 learned_addr learned my addr 172.21.15.151:0/519876001 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:21.740 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:21.738+0000 7f9958e00640 1 --2- 172.21.15.151:0/519876001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99540cccd0 0x7f995413bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:21.740 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:21.738+0000 7f995a200640 1 --2- 172.21.15.151:0/519876001 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9954145750 0x7f99540aabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:21.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:21.738+0000 7f9959800640 1 -- 172.21.15.151:0/519876001 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9954145750 msgr2=0x7f99540aabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:21.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:21.738+0000 7f9959800640 1 --2- 172.21.15.151:0/519876001 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9954145750 0x7f99540aabb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:21.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:21.738+0000 7f9959800640 1 -- 172.21.15.151:0/519876001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99540cccd0 msgr2=0x7f995413bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:21.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:21.738+0000 7f9959800640 1 --2- 172.21.15.151:0/519876001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99540cccd0 0x7f995413bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:21.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:21.738+0000 7f9959800640 1 -- 172.21.15.151:0/519876001 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9954156bb0 con 0x7f995406efa0 2024-09-16T15:39:21.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:21.738+0000 7f9959800640 1 --2- 172.21.15.151:0/519876001 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f995406efa0 0x7f995415edb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f994800c730 tx=0x7f994800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:21.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:21.738+0000 7f9951600640 1 -- 172.21.15.151:0/519876001 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9948003b70 con 0x7f995406efa0 2024-09-16T15:39:21.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:21.738+0000 7f9951600640 1 -- 172.21.15.151:0/519876001 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9948003d30 con 0x7f995406efa0 2024-09-16T15:39:21.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:21.739+0000 7f995c200640 1 -- 172.21.15.151:0/519876001 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99540ab0f0 con 0x7f995406efa0 2024-09-16T15:39:21.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:21.739+0000 7f9951600640 1 -- 172.21.15.151:0/519876001 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f994800aab0 con 0x7f995406efa0 2024-09-16T15:39:21.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:21.739+0000 7f995c200640 1 -- 172.21.15.151:0/519876001 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99540ab670 con 0x7f995406efa0 2024-09-16T15:39:21.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:21.740+0000 7f995c200640 1 -- 172.21.15.151:0/519876001 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99540cdbb0 con 0x7f995406efa0 2024-09-16T15:39:21.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:21.741+0000 7f9951600640 1 -- 172.21.15.151:0/519876001 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f994800ac50 con 0x7f995406efa0 2024-09-16T15:39:21.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:21.742+0000 7f9951600640 1 --2- 172.21.15.151:0/519876001 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9934078850 0x7f993407ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:21.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:21.743+0000 7f9951600640 1 -- 172.21.15.151:0/519876001 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5609+0+0 (secure 0 0 0) 0x7f994809a7b0 con 0x7f995406efa0 2024-09-16T15:39:21.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:21.743+0000 7f9958e00640 1 --2- 172.21.15.151:0/519876001 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9934078850 0x7f993407ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:21.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:21.743+0000 7f9958e00640 1 --2- 172.21.15.151:0/519876001 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9934078850 0x7f993407ad10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9940034000 tx=0x7f9940004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:21.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:21.747+0000 7f9951600640 1 -- 172.21.15.151:0/519876001 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9948062df0 con 0x7f995406efa0 2024-09-16T15:39:21.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:21.968+0000 7f995c200640 1 -- 172.21.15.151:0/519876001 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f99541436d0 con 0x7f995406efa0 2024-09-16T15:39:22.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:22.540+0000 7f9951600640 1 -- 172.21.15.151:0/519876001 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f99480626f0 con 0x7f995406efa0 2024-09-16T15:39:22.541 INFO:teuthology.orchestra.run.smithi151.stderr:new fs with metadata pool 2 and data pool 3 2024-09-16T15:39:22.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:22.544+0000 7f995c200640 1 -- 172.21.15.151:0/519876001 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9934078850 msgr2=0x7f993407ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:22.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:22.544+0000 7f995c200640 1 --2- 172.21.15.151:0/519876001 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9934078850 0x7f993407ad10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9940034000 tx=0x7f9940004730 comp rx=0 tx=0).stop 2024-09-16T15:39:22.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:22.544+0000 7f995c200640 1 -- 172.21.15.151:0/519876001 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f995406efa0 msgr2=0x7f995415edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:22.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:22.544+0000 7f995c200640 1 --2- 172.21.15.151:0/519876001 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f995406efa0 0x7f995415edb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f994800c730 tx=0x7f994800cbf0 comp rx=0 tx=0).stop 2024-09-16T15:39:22.546 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:22.544+0000 7f995c200640 1 -- 172.21.15.151:0/519876001 shutdown_connections 2024-09-16T15:39:22.546 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:22.544+0000 7f995c200640 1 --2- 172.21.15.151:0/519876001 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9934078850 0x7f993407ad10 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:22.546 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:22.544+0000 7f995c200640 1 --2- 172.21.15.151:0/519876001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99540cccd0 0x7f995413bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:22.546 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:22.544+0000 7f995c200640 1 --2- 172.21.15.151:0/519876001 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9954145750 0x7f99540aabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:22.546 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:22.544+0000 7f995c200640 1 --2- 172.21.15.151:0/519876001 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f995406efa0 0x7f995415edb0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:22.546 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:22.544+0000 7f995c200640 1 -- 172.21.15.151:0/519876001 >> 172.21.15.151:0/519876001 conn(0x7f99540b1840 msgr2=0x7f995406f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:22.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:22.545+0000 7f995c200640 1 -- 172.21.15.151:0/519876001 shutdown_connections 2024-09-16T15:39:22.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:22.546+0000 7f995c200640 1 -- 172.21.15.151:0/519876001 wait complete. 2024-09-16T15:39:22.588 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:39:22.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:22.744+0000 7f5b15000640 1 -- 172.21.15.151:0/1852941355 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5b100ca4d0 msgr2=0x7f5b1015e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:22.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:22.744+0000 7f5b15000640 1 --2- 172.21.15.151:0/1852941355 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5b100ca4d0 0x7f5b1015e390 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5afc005fd0 tx=0x7f5afc02a5d0 comp rx=0 tx=0).stop 2024-09-16T15:39:22.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:22.744+0000 7f5b15000640 1 -- 172.21.15.151:0/1852941355 shutdown_connections 2024-09-16T15:39:22.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:22.744+0000 7f5b15000640 1 --2- 172.21.15.151:0/1852941355 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5b100ca4d0 0x7f5b1015e390 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:22.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:22.744+0000 7f5b15000640 1 --2- 172.21.15.151:0/1852941355 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b100709e0 0x7f5b10073af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:22.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:22.744+0000 7f5b15000640 1 -- 172.21.15.151:0/1852941355 >> 172.21.15.151:0/1852941355 conn(0x7f5b100b2b90 msgr2=0x7f5b100b2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:22.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:22.744+0000 7f5b15000640 1 -- 172.21.15.151:0/1852941355 shutdown_connections 2024-09-16T15:39:22.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:22.744+0000 7f5b15000640 1 -- 172.21.15.151:0/1852941355 wait complete. 2024-09-16T15:39:22.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:22.745+0000 7f5b15000640 1 Processor -- start 2024-09-16T15:39:22.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:22.746+0000 7f5b15000640 1 -- start start 2024-09-16T15:39:22.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:22.746+0000 7f5b15000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5b1006cb70 0x7f5b100cb550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:22.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:22.746+0000 7f5b15000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b100709e0 0x7f5b1015b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:22.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:22.747+0000 7f5b15000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b100ca4d0 0x7f5b10140890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:22.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:22.747+0000 7f5b15000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5b100d0250 con 0x7f5b1006cb70 2024-09-16T15:39:22.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:22.747+0000 7f5b15000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5b100d00d0 con 0x7f5b100ca4d0 2024-09-16T15:39:22.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:22.747+0000 7f5b15000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f5b100d03d0 con 0x7f5b100709e0 2024-09-16T15:39:22.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:22.747+0000 7f5b0de00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5b1006cb70 0x7f5b100cb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:22.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:22.747+0000 7f5b0de00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5b1006cb70 0x7f5b100cb550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:35330/0 (socket says 172.21.15.151:35330) 2024-09-16T15:39:22.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:22.747+0000 7f5b0de00640 1 -- 172.21.15.151:0/3764170642 learned_addr learned my addr 172.21.15.151:0/3764170642 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:22.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:22.747+0000 7f5b0e800640 1 --2- 172.21.15.151:0/3764170642 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b100ca4d0 0x7f5b10140890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:22.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:22.747+0000 7f5b0d400640 1 --2- 172.21.15.151:0/3764170642 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b100709e0 0x7f5b1015b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:22.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:22.747+0000 7f5b0de00640 1 -- 172.21.15.151:0/3764170642 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b100709e0 msgr2=0x7f5b1015b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:22.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:22.747+0000 7f5b0de00640 1 --2- 172.21.15.151:0/3764170642 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b100709e0 0x7f5b1015b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:22.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:22.747+0000 7f5b0de00640 1 -- 172.21.15.151:0/3764170642 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b100ca4d0 msgr2=0x7f5b10140890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:22.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:22.747+0000 7f5b0de00640 1 --2- 172.21.15.151:0/3764170642 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b100ca4d0 0x7f5b10140890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:22.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:22.747+0000 7f5b0de00640 1 -- 172.21.15.151:0/3764170642 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b1015bb90 con 0x7f5b1006cb70 2024-09-16T15:39:22.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:22.747+0000 7f5b0de00640 1 --2- 172.21.15.151:0/3764170642 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5b1006cb70 0x7f5b100cb550 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5b0000d950 tx=0x7f5b0000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:22.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:22.748+0000 7f5b06000640 1 -- 172.21.15.151:0/3764170642 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b00017030 con 0x7f5b1006cb70 2024-09-16T15:39:22.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:22.748+0000 7f5b06000640 1 -- 172.21.15.151:0/3764170642 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5b00009b20 con 0x7f5b1006cb70 2024-09-16T15:39:22.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:22.748+0000 7f5b06000640 1 -- 172.21.15.151:0/3764170642 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b00004020 con 0x7f5b1006cb70 2024-09-16T15:39:22.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:22.748+0000 7f5b15000640 1 -- 172.21.15.151:0/3764170642 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b100cdb10 con 0x7f5b1006cb70 2024-09-16T15:39:22.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:22.748+0000 7f5b15000640 1 -- 172.21.15.151:0/3764170642 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b100ce0d0 con 0x7f5b1006cb70 2024-09-16T15:39:22.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:22.749+0000 7f5b15000640 1 -- 172.21.15.151:0/3764170642 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b1006d2f0 con 0x7f5b1006cb70 2024-09-16T15:39:22.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:22.750+0000 7f5b06000640 1 -- 172.21.15.151:0/3764170642 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f5b000041c0 con 0x7f5b1006cb70 2024-09-16T15:39:22.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:22.751+0000 7f5b06000640 1 --2- 172.21.15.151:0/3764170642 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5af0078850 0x7f5af007ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:22.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:22.751+0000 7f5b06000640 1 -- 172.21.15.151:0/3764170642 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f5b0009a9a0 con 0x7f5b1006cb70 2024-09-16T15:39:22.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:22.751+0000 7f5b0d400640 1 --2- 172.21.15.151:0/3764170642 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5af0078850 0x7f5af007ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:22.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:22.752+0000 7f5b0d400640 1 --2- 172.21.15.151:0/3764170642 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5af0078850 0x7f5af007ad10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5afc033000 tx=0x7f5afc004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:22.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:22.755+0000 7f5b06000640 1 -- 172.21.15.151:0/3764170642 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f5b00062920 con 0x7f5b1006cb70 2024-09-16T15:39:22.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:22.943+0000 7f5b15000640 1 -- 172.21.15.151:0/3764170642 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5b10152010 con 0x7f5b1006cb70 2024-09-16T15:39:22.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:22.944+0000 7f5b06000640 1 -- 172.21.15.151:0/3764170642 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16742 (secure 0 0 0) 0x7f5b00067350 con 0x7f5b1006cb70 2024-09-16T15:39:22.946 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:39:22.946 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":21,"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","created":"2024-09-16T15:38:56.348535+0000","modified":"2024-09-16T15:39:21.969442+0000","last_up_change":"2024-09-16T15:39:10.681384+0000","last_in_change":"2024-09-16T15:39:00.544477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:39:11.602252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:39:19.970182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T15:39:20.929376+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5524076-a903-4d0a-af6a-caf474c4765d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6809","nonce":3111342741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6811","nonce":3111342741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6815","nonce":3111342741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6813","nonce":3111342741}]},"public_addr":"172.21.15.151:6809/3111342741","cluster_addr":"172.21.15.151:6811/3111342741","heartbeat_back_addr":"172.21.15.151:6815/3111342741","heartbeat_front_addr":"172.21.15.151:6813/3111342741","state":["exists","up"]},{"osd":1,"uuid":"dd25925f-0ae1-494e-8529-4dc205759db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6801","nonce":3368304087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6803","nonce":3368304087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6807","nonce":3368304087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6805","nonce":3368304087}]},"public_addr":"172.21.15.151:6801/3368304087","cluster_addr":"172.21.15.151:6803/3368304087","heartbeat_back_addr":"172.21.15.151:6807/3368304087","heartbeat_front_addr":"172.21.15.151:6805/3368304087","state":["exists","up"]},{"osd":2,"uuid":"c2f18115-013d-4da6-a0fe-cb978f4494b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6823","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6825","nonce":3079373271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6827","nonce":3079373271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6831","nonce":3079373271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6829","nonce":3079373271}]},"public_addr":"172.21.15.151:6825/3079373271","cluster_addr":"172.21.15.151:6827/3079373271","heartbeat_back_addr":"172.21.15.151:6831/3079373271","heartbeat_front_addr":"172.21.15.151:6829/3079373271","state":["exists","up"]},{"osd":3,"uuid":"36edb718-2fcb-4d88-8ddc-c321eed4279d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6817","nonce":1142509041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6819","nonce":1142509041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6824","nonce":1142509041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6821","nonce":1142509041}]},"public_addr":"172.21.15.151:6817/1142509041","cluster_addr":"172.21.15.151:6819/1142509041","heartbeat_back_addr":"172.21.15.151:6824/1142509041","heartbeat_front_addr":"172.21.15.151:6821/1142509041","state":["exists","up"]},{"osd":4,"uuid":"64d33708-cd0a-4236-abe3-1b646e82ab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6811","nonce":2735495271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6813","nonce":2735495271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6817","nonce":2735495271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6815","nonce":2735495271}]},"public_addr":"172.21.15.167:6811/2735495271","cluster_addr":"172.21.15.167:6813/2735495271","heartbeat_back_addr":"172.21.15.167:6817/2735495271","heartbeat_front_addr":"172.21.15.167:6815/2735495271","state":["exists","up"]},{"osd":5,"uuid":"f14b44a9-5910-458c-b518-09a8d0de94de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6827","nonce":1494916572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6829","nonce":1494916572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6833","nonce":1494916572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6831","nonce":1494916572}]},"public_addr":"172.21.15.167:6827/1494916572","cluster_addr":"172.21.15.167:6829/1494916572","heartbeat_back_addr":"172.21.15.167:6833/1494916572","heartbeat_front_addr":"172.21.15.167:6831/1494916572","state":["exists","up"]},{"osd":6,"uuid":"39335545-2654-4710-84b0-a9aa47cfcbba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6819","nonce":2538273273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6821","nonce":2538273273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6825","nonce":2538273273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6823","nonce":2538273273}]},"public_addr":"172.21.15.167:6819/2538273273","cluster_addr":"172.21.15.167:6821/2538273273","heartbeat_back_addr":"172.21.15.167:6825/2538273273","heartbeat_front_addr":"172.21.15.167:6823/2538273273","state":["exists","up"]},{"osd":7,"uuid":"4f8bfc81-53dd-4339-ac22-1a4007d9f963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6803","nonce":265072227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6805","nonce":265072227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6809","nonce":265072227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6807","nonce":265072227}]},"public_addr":"172.21.15.167:6803/265072227","cluster_addr":"172.21.15.167:6805/265072227","heartbeat_back_addr":"172.21.15.167:6809/265072227","heartbeat_front_addr":"172.21.15.167:6807/265072227","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.150359+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.084358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.722368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.332809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.320709+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.395153+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.935250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.801443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:39:22.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:22.946+0000 7f5b15000640 1 -- 172.21.15.151:0/3764170642 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5af0078850 msgr2=0x7f5af007ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:22.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:22.946+0000 7f5b15000640 1 --2- 172.21.15.151:0/3764170642 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5af0078850 0x7f5af007ad10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5afc033000 tx=0x7f5afc004730 comp rx=0 tx=0).stop 2024-09-16T15:39:22.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:22.947+0000 7f5b15000640 1 -- 172.21.15.151:0/3764170642 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5b1006cb70 msgr2=0x7f5b100cb550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:22.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:22.947+0000 7f5b15000640 1 --2- 172.21.15.151:0/3764170642 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5b1006cb70 0x7f5b100cb550 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5b0000d950 tx=0x7f5b0000de10 comp rx=0 tx=0).stop 2024-09-16T15:39:22.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:22.947+0000 7f5b15000640 1 -- 172.21.15.151:0/3764170642 shutdown_connections 2024-09-16T15:39:22.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:22.947+0000 7f5b15000640 1 --2- 172.21.15.151:0/3764170642 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5af0078850 0x7f5af007ad10 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:22.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:22.947+0000 7f5b15000640 1 --2- 172.21.15.151:0/3764170642 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b100ca4d0 0x7f5b10140890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:22.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:22.947+0000 7f5b15000640 1 --2- 172.21.15.151:0/3764170642 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5b100709e0 0x7f5b1015b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:22.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:22.947+0000 7f5b15000640 1 --2- 172.21.15.151:0/3764170642 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5b1006cb70 0x7f5b100cb550 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:22.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:22.947+0000 7f5b15000640 1 -- 172.21.15.151:0/3764170642 >> 172.21.15.151:0/3764170642 conn(0x7f5b100b2b90 msgr2=0x7f5b1015cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:22.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:22.947+0000 7f5b15000640 1 -- 172.21.15.151:0/3764170642 shutdown_connections 2024-09-16T15:39:22.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:22.947+0000 7f5b15000640 1 -- 172.21.15.151:0/3764170642 wait complete. 2024-09-16T15:39:22.963 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:39:23.122 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.121+0000 7f82fb400640 1 -- 172.21.15.151:0/1852921618 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f82d8004d40 msgr2=0x7f82d80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:23.122 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.121+0000 7f82fb400640 1 --2- 172.21.15.151:0/1852921618 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f82d8004d40 0x7f82d80051c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f82ec001d60 tx=0x7f82d8005830 comp rx=0 tx=0).stop 2024-09-16T15:39:23.122 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.121+0000 7f82f9400640 1 -- 172.21.15.151:0/1852921618 reap_dead start 2024-09-16T15:39:23.122 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.121+0000 7f82fb400640 1 -- 172.21.15.151:0/1852921618 shutdown_connections 2024-09-16T15:39:23.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.121+0000 7f82fb400640 1 -- 172.21.15.151:0/1852921618 >> 172.21.15.151:0/1852921618 conn(0x7f82f40b1950 msgr2=0x7f82f40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:23.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.121+0000 7f82fb400640 1 -- 172.21.15.151:0/1852921618 shutdown_connections 2024-09-16T15:39:23.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.121+0000 7f82fb400640 1 -- 172.21.15.151:0/1852921618 wait complete. 2024-09-16T15:39:23.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.123+0000 7f82fb400640 1 Processor -- start 2024-09-16T15:39:23.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.123+0000 7f82fb400640 1 -- start start 2024-09-16T15:39:23.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.124+0000 7f82fb400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f82f41f8200 0x7f82f41f49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:23.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.124+0000 7f82fb400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f82f41f85e0 0x7f82f41f4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:23.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.124+0000 7f82fb400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82f415d710 0x7f82f415dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:23.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.124+0000 7f82fb400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f82f406f9f0 con 0x7f82f41f85e0 2024-09-16T15:39:23.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.124+0000 7f82fb400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f82f406f870 con 0x7f82f415d710 2024-09-16T15:39:23.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.124+0000 7f82fb400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f82f40b2440 con 0x7f82f41f8200 2024-09-16T15:39:23.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.124+0000 7f82f3e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f82f41f85e0 0x7f82f41f4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:23.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.124+0000 7f82f3e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f82f41f85e0 0x7f82f41f4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47160/0 (socket says 172.21.15.151:47160) 2024-09-16T15:39:23.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.124+0000 7f82f3e00640 1 -- 172.21.15.151:0/3375067262 learned_addr learned my addr 172.21.15.151:0/3375067262 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:23.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.124+0000 7f82f9400640 1 --2- 172.21.15.151:0/3375067262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82f415d710 0x7f82f415dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:23.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.124+0000 7f82f8a00640 1 --2- 172.21.15.151:0/3375067262 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f82f41f8200 0x7f82f41f49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:23.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.124+0000 7f82f3e00640 1 -- 172.21.15.151:0/3375067262 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f82f41f8200 msgr2=0x7f82f41f49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:23.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.124+0000 7f82f3e00640 1 --2- 172.21.15.151:0/3375067262 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f82f41f8200 0x7f82f41f49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:23.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.124+0000 7f82f3e00640 1 -- 172.21.15.151:0/3375067262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82f415d710 msgr2=0x7f82f415dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:23.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.124+0000 7f82f3e00640 1 --2- 172.21.15.151:0/3375067262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82f415d710 0x7f82f415dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:23.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.124+0000 7f82f3e00640 1 -- 172.21.15.151:0/3375067262 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82f409acb0 con 0x7f82f41f85e0 2024-09-16T15:39:23.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.124+0000 7f82f8a00640 1 --2- 172.21.15.151:0/3375067262 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f82f41f8200 0x7f82f41f49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:39:23.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.125+0000 7f82f3e00640 1 --2- 172.21.15.151:0/3375067262 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f82f41f85e0 0x7f82f41f4ee0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f82e0011ae0 tx=0x7f82e0009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:23.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.125+0000 7f82f0c00640 1 -- 172.21.15.151:0/3375067262 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e001e070 con 0x7f82f41f85e0 2024-09-16T15:39:23.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.125+0000 7f82f0c00640 1 -- 172.21.15.151:0/3375067262 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f82e000fe70 con 0x7f82f41f85e0 2024-09-16T15:39:23.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.125+0000 7f82fb400640 1 -- 172.21.15.151:0/3375067262 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82f41f8df0 con 0x7f82f41f85e0 2024-09-16T15:39:23.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.125+0000 7f82f0c00640 1 -- 172.21.15.151:0/3375067262 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e0004290 con 0x7f82f41f85e0 2024-09-16T15:39:23.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.125+0000 7f82fb400640 1 -- 172.21.15.151:0/3375067262 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82f409af50 con 0x7f82f41f85e0 2024-09-16T15:39:23.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.126+0000 7f82fb400640 1 -- 172.21.15.151:0/3375067262 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82f4074440 con 0x7f82f41f85e0 2024-09-16T15:39:23.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.127+0000 7f82f0c00640 1 -- 172.21.15.151:0/3375067262 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f82e0021070 con 0x7f82f41f85e0 2024-09-16T15:39:23.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.128+0000 7f82f0c00640 1 --2- 172.21.15.151:0/3375067262 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f82d4078850 0x7f82d407ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:23.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.128+0000 7f82f0c00640 1 -- 172.21.15.151:0/3375067262 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f82e009b3e0 con 0x7f82f41f85e0 2024-09-16T15:39:23.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.128+0000 7f82f8a00640 1 --2- 172.21.15.151:0/3375067262 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f82d4078850 0x7f82d407ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:23.130 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.129+0000 7f82f8a00640 1 --2- 172.21.15.151:0/3375067262 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f82d4078850 0x7f82d407ad10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f82e4000fb0 tx=0x7f82e40067a0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:23.133 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.132+0000 7f82f0c00640 1 -- 172.21.15.151:0/3375067262 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f82e0063420 con 0x7f82f41f85e0 2024-09-16T15:39:23.318 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.317+0000 7f82fb400640 1 -- 172.21.15.151:0/3375067262 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f82f40aced0 con 0x7f82f41f85e0 2024-09-16T15:39:23.320 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.318+0000 7f82f0c00640 1 -- 172.21.15.151:0/3375067262 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16742 (secure 0 0 0) 0x7f82e0067e50 con 0x7f82f41f85e0 2024-09-16T15:39:23.320 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:39:23.320 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":21,"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","created":"2024-09-16T15:38:56.348535+0000","modified":"2024-09-16T15:39:21.969442+0000","last_up_change":"2024-09-16T15:39:10.681384+0000","last_in_change":"2024-09-16T15:39:00.544477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:39:11.602252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:39:19.970182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T15:39:20.929376+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5524076-a903-4d0a-af6a-caf474c4765d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6809","nonce":3111342741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6811","nonce":3111342741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6815","nonce":3111342741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6813","nonce":3111342741}]},"public_addr":"172.21.15.151:6809/3111342741","cluster_addr":"172.21.15.151:6811/3111342741","heartbeat_back_addr":"172.21.15.151:6815/3111342741","heartbeat_front_addr":"172.21.15.151:6813/3111342741","state":["exists","up"]},{"osd":1,"uuid":"dd25925f-0ae1-494e-8529-4dc205759db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6801","nonce":3368304087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6803","nonce":3368304087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6807","nonce":3368304087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6805","nonce":3368304087}]},"public_addr":"172.21.15.151:6801/3368304087","cluster_addr":"172.21.15.151:6803/3368304087","heartbeat_back_addr":"172.21.15.151:6807/3368304087","heartbeat_front_addr":"172.21.15.151:6805/3368304087","state":["exists","up"]},{"osd":2,"uuid":"c2f18115-013d-4da6-a0fe-cb978f4494b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6823","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6825","nonce":3079373271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6827","nonce":3079373271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6831","nonce":3079373271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6829","nonce":3079373271}]},"public_addr":"172.21.15.151:6825/3079373271","cluster_addr":"172.21.15.151:6827/3079373271","heartbeat_back_addr":"172.21.15.151:6831/3079373271","heartbeat_front_addr":"172.21.15.151:6829/3079373271","state":["exists","up"]},{"osd":3,"uuid":"36edb718-2fcb-4d88-8ddc-c321eed4279d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6817","nonce":1142509041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6819","nonce":1142509041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6824","nonce":1142509041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6821","nonce":1142509041}]},"public_addr":"172.21.15.151:6817/1142509041","cluster_addr":"172.21.15.151:6819/1142509041","heartbeat_back_addr":"172.21.15.151:6824/1142509041","heartbeat_front_addr":"172.21.15.151:6821/1142509041","state":["exists","up"]},{"osd":4,"uuid":"64d33708-cd0a-4236-abe3-1b646e82ab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6811","nonce":2735495271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6813","nonce":2735495271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6817","nonce":2735495271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6815","nonce":2735495271}]},"public_addr":"172.21.15.167:6811/2735495271","cluster_addr":"172.21.15.167:6813/2735495271","heartbeat_back_addr":"172.21.15.167:6817/2735495271","heartbeat_front_addr":"172.21.15.167:6815/2735495271","state":["exists","up"]},{"osd":5,"uuid":"f14b44a9-5910-458c-b518-09a8d0de94de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6827","nonce":1494916572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6829","nonce":1494916572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6833","nonce":1494916572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6831","nonce":1494916572}]},"public_addr":"172.21.15.167:6827/1494916572","cluster_addr":"172.21.15.167:6829/1494916572","heartbeat_back_addr":"172.21.15.167:6833/1494916572","heartbeat_front_addr":"172.21.15.167:6831/1494916572","state":["exists","up"]},{"osd":6,"uuid":"39335545-2654-4710-84b0-a9aa47cfcbba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6819","nonce":2538273273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6821","nonce":2538273273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6825","nonce":2538273273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6823","nonce":2538273273}]},"public_addr":"172.21.15.167:6819/2538273273","cluster_addr":"172.21.15.167:6821/2538273273","heartbeat_back_addr":"172.21.15.167:6825/2538273273","heartbeat_front_addr":"172.21.15.167:6823/2538273273","state":["exists","up"]},{"osd":7,"uuid":"4f8bfc81-53dd-4339-ac22-1a4007d9f963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6803","nonce":265072227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6805","nonce":265072227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6809","nonce":265072227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6807","nonce":265072227}]},"public_addr":"172.21.15.167:6803/265072227","cluster_addr":"172.21.15.167:6805/265072227","heartbeat_back_addr":"172.21.15.167:6809/265072227","heartbeat_front_addr":"172.21.15.167:6807/265072227","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.150359+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.084358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.722368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.332809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.320709+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.395153+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.935250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.801443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:39:23.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.321+0000 7f82fb400640 1 -- 172.21.15.151:0/3375067262 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f82d4078850 msgr2=0x7f82d407ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:23.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.321+0000 7f82fb400640 1 --2- 172.21.15.151:0/3375067262 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f82d4078850 0x7f82d407ad10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f82e4000fb0 tx=0x7f82e40067a0 comp rx=0 tx=0).stop 2024-09-16T15:39:23.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.321+0000 7f82fb400640 1 -- 172.21.15.151:0/3375067262 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f82f41f85e0 msgr2=0x7f82f41f4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:23.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.321+0000 7f82fb400640 1 --2- 172.21.15.151:0/3375067262 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f82f41f85e0 0x7f82f41f4ee0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f82e0011ae0 tx=0x7f82e0009e90 comp rx=0 tx=0).stop 2024-09-16T15:39:23.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.321+0000 7f82fb400640 1 -- 172.21.15.151:0/3375067262 shutdown_connections 2024-09-16T15:39:23.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.321+0000 7f82fb400640 1 --2- 172.21.15.151:0/3375067262 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f82d4078850 0x7f82d407ad10 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:23.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.321+0000 7f82fb400640 1 --2- 172.21.15.151:0/3375067262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82f415d710 0x7f82f415dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:23.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.321+0000 7f82fb400640 1 --2- 172.21.15.151:0/3375067262 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f82f41f85e0 0x7f82f41f4ee0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:23.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.321+0000 7f82fb400640 1 --2- 172.21.15.151:0/3375067262 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f82f41f8200 0x7f82f41f49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:23.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.321+0000 7f82fb400640 1 -- 172.21.15.151:0/3375067262 >> 172.21.15.151:0/3375067262 conn(0x7f82f40b1950 msgr2=0x7f82f4154cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:23.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.321+0000 7f82fb400640 1 -- 172.21.15.151:0/3375067262 shutdown_connections 2024-09-16T15:39:23.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.321+0000 7f82fb400640 1 -- 172.21.15.151:0/3375067262 wait complete. 2024-09-16T15:39:23.339 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-16T15:39:23.501 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.500+0000 7f03f9c00640 1 -- 172.21.15.151:0/2164801752 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f03f406efa0 msgr2=0x7f03f4073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:23.501 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.500+0000 7f03f9c00640 1 --2- 172.21.15.151:0/2164801752 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f03f406efa0 0x7f03f4073440 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f03ec005fd0 tx=0x7f03ec02a820 comp rx=0 tx=0).stop 2024-09-16T15:39:23.501 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.500+0000 7f03f9c00640 1 -- 172.21.15.151:0/2164801752 shutdown_connections 2024-09-16T15:39:23.501 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.500+0000 7f03f9c00640 1 --2- 172.21.15.151:0/2164801752 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f03f406efa0 0x7f03f4073440 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:23.501 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.500+0000 7f03f9c00640 1 -- 172.21.15.151:0/2164801752 >> 172.21.15.151:0/2164801752 conn(0x7f03f40b1840 msgr2=0x7f03f40b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:23.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.500+0000 7f03f9c00640 1 -- 172.21.15.151:0/2164801752 shutdown_connections 2024-09-16T15:39:23.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.500+0000 7f03f9c00640 1 -- 172.21.15.151:0/2164801752 wait complete. 2024-09-16T15:39:23.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.501+0000 7f03f9c00640 1 Processor -- start 2024-09-16T15:39:23.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.502+0000 7f03f9c00640 1 -- start start 2024-09-16T15:39:23.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.502+0000 7f03f9c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03f406efa0 0x7f03f4140c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:23.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.503+0000 7f03f9c00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03f40cccd0 0x7f03f413f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:23.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.503+0000 7f03f9c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f03f4145750 0x7f03f41f3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:23.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.503+0000 7f03f9c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f03f40ce6d0 con 0x7f03f4145750 2024-09-16T15:39:23.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.503+0000 7f03f9c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f03f40ce550 con 0x7f03f406efa0 2024-09-16T15:39:23.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.503+0000 7f03f9c00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f03f40ce850 con 0x7f03f40cccd0 2024-09-16T15:39:23.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.503+0000 7f03f3400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f03f4145750 0x7f03f41f3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:23.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.503+0000 7f03f3400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f03f4145750 0x7f03f41f3020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47190/0 (socket says 172.21.15.151:47190) 2024-09-16T15:39:23.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.503+0000 7f03f3400640 1 -- 172.21.15.151:0/2236639554 learned_addr learned my addr 172.21.15.151:0/2236639554 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:23.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.503+0000 7f03f2a00640 1 --2- 172.21.15.151:0/2236639554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03f406efa0 0x7f03f4140c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:23.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.503+0000 7f03f2000640 1 --2- 172.21.15.151:0/2236639554 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03f40cccd0 0x7f03f413f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:23.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.503+0000 7f03f3400640 1 -- 172.21.15.151:0/2236639554 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03f40cccd0 msgr2=0x7f03f413f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:23.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.503+0000 7f03f3400640 1 --2- 172.21.15.151:0/2236639554 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03f40cccd0 0x7f03f413f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:23.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.503+0000 7f03f3400640 1 -- 172.21.15.151:0/2236639554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03f406efa0 msgr2=0x7f03f4140c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:23.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.503+0000 7f03f3400640 1 --2- 172.21.15.151:0/2236639554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03f406efa0 0x7f03f4140c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:23.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.503+0000 7f03f3400640 1 -- 172.21.15.151:0/2236639554 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03f413fed0 con 0x7f03f4145750 2024-09-16T15:39:23.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.503+0000 7f03f2000640 1 --2- 172.21.15.151:0/2236639554 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03f40cccd0 0x7f03f413f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:39:23.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.503+0000 7f03f2a00640 1 --2- 172.21.15.151:0/2236639554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03f406efa0 0x7f03f4140c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:39:23.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.504+0000 7f03f3400640 1 --2- 172.21.15.151:0/2236639554 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f03f4145750 0x7f03f41f3020 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f03ec009b30 tx=0x7f03ec032c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:23.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.504+0000 7f03eaa00640 1 -- 172.21.15.151:0/2236639554 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ec037040 con 0x7f03f4145750 2024-09-16T15:39:23.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.504+0000 7f03eaa00640 1 -- 172.21.15.151:0/2236639554 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f03ec042070 con 0x7f03f4145750 2024-09-16T15:39:23.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.504+0000 7f03eaa00640 1 -- 172.21.15.151:0/2236639554 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ec0042a0 con 0x7f03f4145750 2024-09-16T15:39:23.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.504+0000 7f03f9c00640 1 -- 172.21.15.151:0/2236639554 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03f41401b0 con 0x7f03f4145750 2024-09-16T15:39:23.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.504+0000 7f03f9c00640 1 -- 172.21.15.151:0/2236639554 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03f4140770 con 0x7f03f4145750 2024-09-16T15:39:23.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.505+0000 7f03f9c00640 1 -- 172.21.15.151:0/2236639554 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03f40ced00 con 0x7f03f4145750 2024-09-16T15:39:23.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.506+0000 7f03eaa00640 1 -- 172.21.15.151:0/2236639554 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f03ec004440 con 0x7f03f4145750 2024-09-16T15:39:23.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.507+0000 7f03eaa00640 1 --2- 172.21.15.151:0/2236639554 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f03d4078850 0x7f03d407ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:23.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.507+0000 7f03eaa00640 1 -- 172.21.15.151:0/2236639554 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f03ec0bb620 con 0x7f03f4145750 2024-09-16T15:39:23.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.507+0000 7f03f2a00640 1 --2- 172.21.15.151:0/2236639554 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f03d4078850 0x7f03d407ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:23.509 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.508+0000 7f03f2a00640 1 --2- 172.21.15.151:0/2236639554 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f03d4078850 0x7f03d407ad10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f03e4006fd0 tx=0x7f03e4008040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:23.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.511+0000 7f03eaa00640 1 -- 172.21.15.151:0/2236639554 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f03ec083c80 con 0x7f03f4145750 2024-09-16T15:39:23.738 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:23.737+0000 7f03f9c00640 1 -- 172.21.15.151:0/2236639554 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f03f4073520 con 0x7f03f4145750 2024-09-16T15:39:24.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:24.568+0000 7f03eaa00640 1 -- 172.21.15.151:0/2236639554 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) ==== 118+0+0 (secure 0 0 0) 0x7f03ec083580 con 0x7f03f4145750 2024-09-16T15:39:24.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:24.571+0000 7f03f9c00640 1 -- 172.21.15.151:0/2236639554 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f03d4078850 msgr2=0x7f03d407ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:24.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:24.571+0000 7f03f9c00640 1 --2- 172.21.15.151:0/2236639554 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f03d4078850 0x7f03d407ad10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f03e4006fd0 tx=0x7f03e4008040 comp rx=0 tx=0).stop 2024-09-16T15:39:24.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:24.571+0000 7f03f9c00640 1 -- 172.21.15.151:0/2236639554 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f03f4145750 msgr2=0x7f03f41f3020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:24.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:24.571+0000 7f03f9c00640 1 --2- 172.21.15.151:0/2236639554 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f03f4145750 0x7f03f41f3020 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f03ec009b30 tx=0x7f03ec032c90 comp rx=0 tx=0).stop 2024-09-16T15:39:24.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:24.571+0000 7f03f9c00640 1 -- 172.21.15.151:0/2236639554 shutdown_connections 2024-09-16T15:39:24.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:24.571+0000 7f03f9c00640 1 --2- 172.21.15.151:0/2236639554 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f03d4078850 0x7f03d407ad10 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:24.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:24.571+0000 7f03f9c00640 1 --2- 172.21.15.151:0/2236639554 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f03f4145750 0x7f03f41f3020 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:24.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:24.571+0000 7f03f9c00640 1 --2- 172.21.15.151:0/2236639554 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f03f40cccd0 0x7f03f413f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:24.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:24.571+0000 7f03f9c00640 1 --2- 172.21.15.151:0/2236639554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03f406efa0 0x7f03f4140c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:24.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:24.571+0000 7f03f9c00640 1 -- 172.21.15.151:0/2236639554 >> 172.21.15.151:0/2236639554 conn(0x7f03f40b1840 msgr2=0x7f03f40ae4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:24.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:24.571+0000 7f03f9c00640 1 -- 172.21.15.151:0/2236639554 shutdown_connections 2024-09-16T15:39:24.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:24.571+0000 7f03f9c00640 1 -- 172.21.15.151:0/2236639554 wait complete. 2024-09-16T15:39:24.589 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-16T15:39:24.589 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-16T15:39:24.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:24.749+0000 7fba59400640 1 -- 172.21.15.151:0/3597682599 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba5406efa0 msgr2=0x7fba540cc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:24.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:24.749+0000 7fba59400640 1 --2- 172.21.15.151:0/3597682599 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba5406efa0 0x7fba540cc5c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fba40005fd0 tx=0x7fba4002a4a0 comp rx=0 tx=0).stop 2024-09-16T15:39:24.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:24.749+0000 7fba59400640 1 -- 172.21.15.151:0/3597682599 shutdown_connections 2024-09-16T15:39:24.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:24.749+0000 7fba59400640 1 --2- 172.21.15.151:0/3597682599 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba5406efa0 0x7fba540cc5c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:24.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:24.749+0000 7fba59400640 1 --2- 172.21.15.151:0/3597682599 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fba54145750 0x7fba54073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:24.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:24.749+0000 7fba59400640 1 -- 172.21.15.151:0/3597682599 >> 172.21.15.151:0/3597682599 conn(0x7fba540b1840 msgr2=0x7fba540b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:24.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:24.749+0000 7fba59400640 1 -- 172.21.15.151:0/3597682599 shutdown_connections 2024-09-16T15:39:24.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:24.749+0000 7fba59400640 1 -- 172.21.15.151:0/3597682599 wait complete. 2024-09-16T15:39:24.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:24.751+0000 7fba59400640 1 Processor -- start 2024-09-16T15:39:24.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:24.751+0000 7fba59400640 1 -- start start 2024-09-16T15:39:24.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:24.752+0000 7fba59400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba5406efa0 0x7fba541345f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:24.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:24.752+0000 7fba59400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fba540ccb00 0x7fba540c9860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:24.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:24.752+0000 7fba59400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba54145750 0x7fba540c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:24.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:24.752+0000 7fba59400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fba540b26b0 con 0x7fba54145750 2024-09-16T15:39:24.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:24.752+0000 7fba59400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fba540cedd0 con 0x7fba5406efa0 2024-09-16T15:39:24.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:24.752+0000 7fba59400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fba54160100 con 0x7fba540ccb00 2024-09-16T15:39:24.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:24.752+0000 7fba53200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba54145750 0x7fba540c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:24.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:24.752+0000 7fba53200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba54145750 0x7fba540c9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47210/0 (socket says 172.21.15.151:47210) 2024-09-16T15:39:24.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:24.752+0000 7fba53200640 1 -- 172.21.15.151:0/1712833732 learned_addr learned my addr 172.21.15.151:0/1712833732 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:24.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:24.752+0000 7fba51e00640 1 --2- 172.21.15.151:0/1712833732 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fba540ccb00 0x7fba540c9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:24.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:24.752+0000 7fba52800640 1 --2- 172.21.15.151:0/1712833732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba5406efa0 0x7fba541345f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:24.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:24.753+0000 7fba53200640 1 -- 172.21.15.151:0/1712833732 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fba540ccb00 msgr2=0x7fba540c9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:24.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:24.753+0000 7fba53200640 1 --2- 172.21.15.151:0/1712833732 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fba540ccb00 0x7fba540c9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:24.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:24.753+0000 7fba53200640 1 -- 172.21.15.151:0/1712833732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba5406efa0 msgr2=0x7fba541345f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:24.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:24.753+0000 7fba53200640 1 --2- 172.21.15.151:0/1712833732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba5406efa0 0x7fba541345f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:24.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:24.753+0000 7fba53200640 1 -- 172.21.15.151:0/1712833732 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba541374d0 con 0x7fba54145750 2024-09-16T15:39:24.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:24.753+0000 7fba53200640 1 --2- 172.21.15.151:0/1712833732 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba54145750 0x7fba540c9da0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fba4c00c440 tx=0x7fba4c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:24.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:24.753+0000 7fba4aa00640 1 -- 172.21.15.151:0/1712833732 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba4c011790 con 0x7fba54145750 2024-09-16T15:39:24.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:24.753+0000 7fba4aa00640 1 -- 172.21.15.151:0/1712833732 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fba4c017070 con 0x7fba54145750 2024-09-16T15:39:24.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:24.753+0000 7fba4aa00640 1 -- 172.21.15.151:0/1712833732 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba4c011cc0 con 0x7fba54145750 2024-09-16T15:39:24.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:24.753+0000 7fba59400640 1 -- 172.21.15.151:0/1712833732 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba540c4ab0 con 0x7fba54145750 2024-09-16T15:39:24.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:24.753+0000 7fba59400640 1 -- 172.21.15.151:0/1712833732 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba540c5070 con 0x7fba54145750 2024-09-16T15:39:24.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:24.755+0000 7fba59400640 1 -- 172.21.15.151:0/1712833732 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba541605b0 con 0x7fba54145750 2024-09-16T15:39:24.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:24.756+0000 7fba4aa00640 1 -- 172.21.15.151:0/1712833732 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fba4c01c460 con 0x7fba54145750 2024-09-16T15:39:24.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:24.757+0000 7fba4aa00640 1 --2- 172.21.15.151:0/1712833732 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fba34078850 0x7fba3407ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:24.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:24.757+0000 7fba4aa00640 1 -- 172.21.15.151:0/1712833732 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fba4c09a610 con 0x7fba54145750 2024-09-16T15:39:24.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:24.757+0000 7fba52800640 1 --2- 172.21.15.151:0/1712833732 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fba34078850 0x7fba3407ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:24.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:24.758+0000 7fba52800640 1 --2- 172.21.15.151:0/1712833732 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fba34078850 0x7fba3407ad10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fba540c68e0 tx=0x7fba440095e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:24.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:24.761+0000 7fba4aa00640 1 -- 172.21.15.151:0/1712833732 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fba4c062630 con 0x7fba54145750 2024-09-16T15:39:24.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:24.982+0000 7fba59400640 1 -- 172.21.15.151:0/1712833732 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fba54073390 con 0x7fba54145750 2024-09-16T15:39:25.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:25.570+0000 7fba4aa00640 1 -- 172.21.15.151:0/1712833732 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) ==== 122+0+0 (secure 0 0 0) 0x7fba4c067060 con 0x7fba54145750 2024-09-16T15:39:25.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:25.572+0000 7fba59400640 1 -- 172.21.15.151:0/1712833732 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fba34078850 msgr2=0x7fba3407ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:25.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:25.572+0000 7fba59400640 1 --2- 172.21.15.151:0/1712833732 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fba34078850 0x7fba3407ad10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fba540c68e0 tx=0x7fba440095e0 comp rx=0 tx=0).stop 2024-09-16T15:39:25.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:25.572+0000 7fba59400640 1 -- 172.21.15.151:0/1712833732 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba54145750 msgr2=0x7fba540c9da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:25.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:25.572+0000 7fba59400640 1 --2- 172.21.15.151:0/1712833732 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba54145750 0x7fba540c9da0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fba4c00c440 tx=0x7fba4c00c900 comp rx=0 tx=0).stop 2024-09-16T15:39:25.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:25.572+0000 7fba59400640 1 -- 172.21.15.151:0/1712833732 shutdown_connections 2024-09-16T15:39:25.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:25.573+0000 7fba59400640 1 --2- 172.21.15.151:0/1712833732 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fba34078850 0x7fba3407ad10 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:25.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:25.573+0000 7fba59400640 1 --2- 172.21.15.151:0/1712833732 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba54145750 0x7fba540c9da0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:25.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:25.573+0000 7fba59400640 1 --2- 172.21.15.151:0/1712833732 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fba540ccb00 0x7fba540c9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:25.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:25.573+0000 7fba59400640 1 --2- 172.21.15.151:0/1712833732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba5406efa0 0x7fba541345f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:25.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:25.573+0000 7fba59400640 1 -- 172.21.15.151:0/1712833732 >> 172.21.15.151:0/1712833732 conn(0x7fba540b1840 msgr2=0x7fba540a9b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:25.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:25.573+0000 7fba59400640 1 -- 172.21.15.151:0/1712833732 shutdown_connections 2024-09-16T15:39:25.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:25.573+0000 7fba59400640 1 -- 172.21.15.151:0/1712833732 wait complete. 2024-09-16T15:39:25.592 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-16T15:39:25.593 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:39:25.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:25.753+0000 7f689fe00640 1 -- 172.21.15.151:0/70069328 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6884003aa0 msgr2=0x7f6884003ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:25.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:25.753+0000 7f689fe00640 1 --2- 172.21.15.151:0/70069328 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6884003aa0 0x7f6884003ee0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6890001dc0 tx=0x7f6884005830 comp rx=0 tx=0).stop 2024-09-16T15:39:25.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:25.753+0000 7f689fe00640 1 -- 172.21.15.151:0/70069328 shutdown_connections 2024-09-16T15:39:25.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:25.753+0000 7f689fe00640 1 --2- 172.21.15.151:0/70069328 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6884003aa0 0x7f6884003ee0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:25.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:25.753+0000 7f689fe00640 1 -- 172.21.15.151:0/70069328 >> 172.21.15.151:0/70069328 conn(0x7f68a00d3df0 msgr2=0x7f68a00d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:25.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:25.753+0000 7f689fe00640 1 -- 172.21.15.151:0/70069328 shutdown_connections 2024-09-16T15:39:25.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:25.753+0000 7f689fe00640 1 -- 172.21.15.151:0/70069328 wait complete. 2024-09-16T15:39:25.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:25.754+0000 7f689fe00640 1 Processor -- start 2024-09-16T15:39:25.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:25.755+0000 7f689fe00640 1 -- start start 2024-09-16T15:39:25.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:25.755+0000 7f689fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6884003aa0 0x7f68a00b8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:25.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:25.755+0000 7f689fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68a00c8c00 0x7f68a00d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:25.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:25.755+0000 7f689fe00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68a00b8f50 0x7f68a00c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:25.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:25.755+0000 7f689fe00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f68a00b3840 con 0x7f6884003aa0 2024-09-16T15:39:25.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:25.755+0000 7f689fe00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f68a00b36c0 con 0x7f68a00c8c00 2024-09-16T15:39:25.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:25.755+0000 7f689fe00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f68a00b39c0 con 0x7f68a00b8f50 2024-09-16T15:39:25.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:25.756+0000 7f689ea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6884003aa0 0x7f68a00b8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:25.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:25.756+0000 7f689ea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6884003aa0 0x7f68a00b8630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47224/0 (socket says 172.21.15.151:47224) 2024-09-16T15:39:25.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:25.756+0000 7f689ea00640 1 -- 172.21.15.151:0/1097325546 learned_addr learned my addr 172.21.15.151:0/1097325546 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:25.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:25.756+0000 7f689e000640 1 --2- 172.21.15.151:0/1097325546 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68a00c8c00 0x7f68a00d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:25.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:25.756+0000 7f689f400640 1 --2- 172.21.15.151:0/1097325546 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68a00b8f50 0x7f68a00c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:25.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:25.756+0000 7f689ea00640 1 -- 172.21.15.151:0/1097325546 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68a00b8f50 msgr2=0x7f68a00c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:25.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:25.756+0000 7f689ea00640 1 --2- 172.21.15.151:0/1097325546 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68a00b8f50 0x7f68a00c92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:25.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:25.756+0000 7f689ea00640 1 -- 172.21.15.151:0/1097325546 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68a00c8c00 msgr2=0x7f68a00d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:25.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:25.756+0000 7f689ea00640 1 --2- 172.21.15.151:0/1097325546 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68a00c8c00 0x7f68a00d0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:25.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:25.756+0000 7f689ea00640 1 -- 172.21.15.151:0/1097325546 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68a01f7c40 con 0x7f6884003aa0 2024-09-16T15:39:25.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:25.756+0000 7f689e000640 1 --2- 172.21.15.151:0/1097325546 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68a00c8c00 0x7f68a00d0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:39:25.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:25.756+0000 7f689ea00640 1 --2- 172.21.15.151:0/1097325546 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6884003aa0 0x7f68a00b8630 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6890000f80 tx=0x7f6890003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:25.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:25.757+0000 7f6897400640 1 -- 172.21.15.151:0/1097325546 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6890006460 con 0x7f6884003aa0 2024-09-16T15:39:25.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:25.757+0000 7f6897400640 1 -- 172.21.15.151:0/1097325546 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6890006710 con 0x7f6884003aa0 2024-09-16T15:39:25.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:25.757+0000 7f6897400640 1 -- 172.21.15.151:0/1097325546 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68900336e0 con 0x7f6884003aa0 2024-09-16T15:39:25.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:25.757+0000 7f689fe00640 1 -- 172.21.15.151:0/1097325546 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68a01f7f20 con 0x7f6884003aa0 2024-09-16T15:39:25.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:25.757+0000 7f689fe00640 1 -- 172.21.15.151:0/1097325546 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68a01f84e0 con 0x7f6884003aa0 2024-09-16T15:39:25.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:25.758+0000 7f689fe00640 1 -- 172.21.15.151:0/1097325546 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68a00d1d60 con 0x7f6884003aa0 2024-09-16T15:39:25.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:25.759+0000 7f6897400640 1 -- 172.21.15.151:0/1097325546 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f6890033880 con 0x7f6884003aa0 2024-09-16T15:39:25.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:25.760+0000 7f6897400640 1 --2- 172.21.15.151:0/1097325546 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6880078880 0x7f688007ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:25.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:25.760+0000 7f6897400640 1 -- 172.21.15.151:0/1097325546 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f68900ba780 con 0x7f6884003aa0 2024-09-16T15:39:25.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:25.760+0000 7f689e000640 1 --2- 172.21.15.151:0/1097325546 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6880078880 0x7f688007ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:25.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:25.761+0000 7f689e000640 1 --2- 172.21.15.151:0/1097325546 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6880078880 0x7f688007ad40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f68a00b8ee0 tx=0x7f688c005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:25.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:25.764+0000 7f6897400640 1 -- 172.21.15.151:0/1097325546 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6890082de0 con 0x7f6884003aa0 2024-09-16T15:39:25.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:25.986+0000 7f689fe00640 1 -- 172.21.15.151:0/1097325546 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f68a015ce70 con 0x7f6884003aa0 2024-09-16T15:39:25.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:25.986+0000 7f6897400640 1 -- 172.21.15.151:0/1097325546 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5229 (secure 0 0 0) 0x7f68900826e0 con 0x7f6884003aa0 2024-09-16T15:39:25.988 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:39:25.988 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":8,"btime":"2024-09-16T15:39:25:569070+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4301,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/2005239755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2005239755},{"type":"v1","addr":"172.21.15.167:6836","nonce":2005239755}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4338,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/3568704323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3568704323},{"type":"v1","addr":"172.21.15.151:6834","nonce":3568704323}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4341,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/3198287696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":3198287696},{"type":"v1","addr":"172.21.15.151:6835","nonce":3198287696}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:39:21.969407+0000","modified":"2024-09-16T15:39:24.983141+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4350},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4350":{"gid":4350,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6837/3014501393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3014501393},{"type":"v1","addr":"172.21.15.167:6837","nonce":3014501393}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4350,"qdb_cluster":[4350]},"id":1}]} 2024-09-16T15:39:25.988 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 8 2024-09-16T15:39:25.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:25.989+0000 7f689fe00640 1 -- 172.21.15.151:0/1097325546 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6880078880 msgr2=0x7f688007ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:25.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:25.989+0000 7f689fe00640 1 --2- 172.21.15.151:0/1097325546 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6880078880 0x7f688007ad40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f68a00b8ee0 tx=0x7f688c005f90 comp rx=0 tx=0).stop 2024-09-16T15:39:25.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:25.989+0000 7f689fe00640 1 -- 172.21.15.151:0/1097325546 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6884003aa0 msgr2=0x7f68a00b8630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:25.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:25.989+0000 7f689fe00640 1 --2- 172.21.15.151:0/1097325546 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6884003aa0 0x7f68a00b8630 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6890000f80 tx=0x7f6890003d40 comp rx=0 tx=0).stop 2024-09-16T15:39:25.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:25.989+0000 7f689fe00640 1 -- 172.21.15.151:0/1097325546 shutdown_connections 2024-09-16T15:39:25.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:25.989+0000 7f689fe00640 1 --2- 172.21.15.151:0/1097325546 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6880078880 0x7f688007ad40 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:25.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:25.989+0000 7f689fe00640 1 --2- 172.21.15.151:0/1097325546 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f68a00c8c00 0x7f68a00d0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:25.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:25.989+0000 7f689fe00640 1 --2- 172.21.15.151:0/1097325546 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f68a00b8f50 0x7f68a00c92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:25.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:25.989+0000 7f689fe00640 1 --2- 172.21.15.151:0/1097325546 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6884003aa0 0x7f68a00b8630 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:25.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:25.989+0000 7f689fe00640 1 -- 172.21.15.151:0/1097325546 >> 172.21.15.151:0/1097325546 conn(0x7f68a00d3df0 msgr2=0x7f68a00d41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:25.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:25.989+0000 7f689fe00640 1 -- 172.21.15.151:0/1097325546 shutdown_connections 2024-09-16T15:39:25.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:25.989+0000 7f689fe00640 1 -- 172.21.15.151:0/1097325546 wait complete. 2024-09-16T15:39:26.005 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:39:26.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.162+0000 7f75b3200640 1 -- 172.21.15.151:0/3912264519 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75b40b34b0 msgr2=0x7f75b40b3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:26.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.162+0000 7f75b3200640 1 --2- 172.21.15.151:0/3912264519 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75b40b34b0 0x7f75b40b3910 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f75b40d4ea0 tx=0x7f75a002a490 comp rx=0 tx=0).stop 2024-09-16T15:39:26.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.162+0000 7f75b3200640 1 -- 172.21.15.151:0/3912264519 shutdown_connections 2024-09-16T15:39:26.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.162+0000 7f75b3200640 1 --2- 172.21.15.151:0/3912264519 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75b40b34b0 0x7f75b40b3910 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:26.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.162+0000 7f75b3200640 1 --2- 172.21.15.151:0/3912264519 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f75b4161440 0x7f75b40b5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:26.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.163+0000 7f75b3200640 1 -- 172.21.15.151:0/3912264519 >> 172.21.15.151:0/3912264519 conn(0x7f75b40d3df0 msgr2=0x7f75b40d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:26.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.163+0000 7f75b3200640 1 -- 172.21.15.151:0/3912264519 shutdown_connections 2024-09-16T15:39:26.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.163+0000 7f75b3200640 1 -- 172.21.15.151:0/3912264519 wait complete. 2024-09-16T15:39:26.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.164+0000 7f75b3200640 1 Processor -- start 2024-09-16T15:39:26.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.164+0000 7f75b3200640 1 -- start start 2024-09-16T15:39:26.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.165+0000 7f75b3200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75b40b34b0 0x7f75b40b8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:26.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.165+0000 7f75b3200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f75b40b3e50 0x7f75b41f4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:26.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.165+0000 7f75b3200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75b4161440 0x7f75b41579e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:26.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.165+0000 7f75b3200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f75b41619c0 con 0x7f75b4161440 2024-09-16T15:39:26.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.165+0000 7f75b3200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f75b40b2990 con 0x7f75b40b34b0 2024-09-16T15:39:26.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.165+0000 7f75b3200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f75b40d2470 con 0x7f75b40b3e50 2024-09-16T15:39:26.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.165+0000 7f75b2800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75b4161440 0x7f75b41579e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:26.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.165+0000 7f75b2800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75b4161440 0x7f75b41579e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47228/0 (socket says 172.21.15.151:47228) 2024-09-16T15:39:26.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.165+0000 7f75b2800640 1 -- 172.21.15.151:0/2414578499 learned_addr learned my addr 172.21.15.151:0/2414578499 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:26.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.166+0000 7f75b1e00640 1 --2- 172.21.15.151:0/2414578499 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75b40b34b0 0x7f75b40b8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:26.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.166+0000 7f75b1400640 1 --2- 172.21.15.151:0/2414578499 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f75b40b3e50 0x7f75b41f4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:26.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.166+0000 7f75b2800640 1 -- 172.21.15.151:0/2414578499 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f75b40b3e50 msgr2=0x7f75b41f4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:26.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.166+0000 7f75b2800640 1 --2- 172.21.15.151:0/2414578499 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f75b40b3e50 0x7f75b41f4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:26.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.166+0000 7f75b2800640 1 -- 172.21.15.151:0/2414578499 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75b40b34b0 msgr2=0x7f75b40b8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:26.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.166+0000 7f75b2800640 1 --2- 172.21.15.151:0/2414578499 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75b40b34b0 0x7f75b40b8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:26.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.166+0000 7f75b2800640 1 -- 172.21.15.151:0/2414578499 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75b41f49d0 con 0x7f75b4161440 2024-09-16T15:39:26.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.166+0000 7f75b2800640 1 --2- 172.21.15.151:0/2414578499 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75b4161440 0x7f75b41579e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f75ac00c780 tx=0x7f75ac00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:26.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.166+0000 7f75aa000640 1 -- 172.21.15.151:0/2414578499 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ac017070 con 0x7f75b4161440 2024-09-16T15:39:26.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.166+0000 7f75aa000640 1 -- 172.21.15.151:0/2414578499 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f75ac00ce30 con 0x7f75b4161440 2024-09-16T15:39:26.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.166+0000 7f75aa000640 1 -- 172.21.15.151:0/2414578499 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ac005290 con 0x7f75b4161440 2024-09-16T15:39:26.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.166+0000 7f75b3200640 1 -- 172.21.15.151:0/2414578499 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75b41f4ca0 con 0x7f75b4161440 2024-09-16T15:39:26.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.167+0000 7f75b3200640 1 -- 172.21.15.151:0/2414578499 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75b41f5260 con 0x7f75b4161440 2024-09-16T15:39:26.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.168+0000 7f75b3200640 1 -- 172.21.15.151:0/2414578499 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75b40d29a0 con 0x7f75b4161440 2024-09-16T15:39:26.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.169+0000 7f75aa000640 1 -- 172.21.15.151:0/2414578499 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f75ac01a070 con 0x7f75b4161440 2024-09-16T15:39:26.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.169+0000 7f75aa000640 1 --2- 172.21.15.151:0/2414578499 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7594078880 0x7f759407ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:26.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.170+0000 7f75aa000640 1 -- 172.21.15.151:0/2414578499 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f75ac099e90 con 0x7f75b4161440 2024-09-16T15:39:26.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.170+0000 7f75b1e00640 1 --2- 172.21.15.151:0/2414578499 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7594078880 0x7f759407ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:26.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.171+0000 7f75b1e00640 1 --2- 172.21.15.151:0/2414578499 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7594078880 0x7f759407ad40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f75a40098e0 tx=0x7f75a4009060 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:26.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.174+0000 7f75aa000640 1 -- 172.21.15.151:0/2414578499 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f75ac061ea0 con 0x7f75b4161440 2024-09-16T15:39:26.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.360+0000 7f75b3200640 1 -- 172.21.15.151:0/2414578499 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f75b40760b0 con 0x7f75b4161440 2024-09-16T15:39:26.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.361+0000 7f75aa000640 1 -- 172.21.15.151:0/2414578499 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16729 (secure 0 0 0) 0x7f75ac09b020 con 0x7f75b4161440 2024-09-16T15:39:26.363 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:39:26.363 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":22,"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","created":"2024-09-16T15:38:56.348535+0000","modified":"2024-09-16T15:39:23.550834+0000","last_up_change":"2024-09-16T15:39:10.681384+0000","last_in_change":"2024-09-16T15:39:00.544477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:39:11.602252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:39:19.970182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T15:39:20.929376+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5524076-a903-4d0a-af6a-caf474c4765d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6809","nonce":3111342741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6811","nonce":3111342741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6815","nonce":3111342741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6813","nonce":3111342741}]},"public_addr":"172.21.15.151:6809/3111342741","cluster_addr":"172.21.15.151:6811/3111342741","heartbeat_back_addr":"172.21.15.151:6815/3111342741","heartbeat_front_addr":"172.21.15.151:6813/3111342741","state":["exists","up"]},{"osd":1,"uuid":"dd25925f-0ae1-494e-8529-4dc205759db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6801","nonce":3368304087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6803","nonce":3368304087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6807","nonce":3368304087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6805","nonce":3368304087}]},"public_addr":"172.21.15.151:6801/3368304087","cluster_addr":"172.21.15.151:6803/3368304087","heartbeat_back_addr":"172.21.15.151:6807/3368304087","heartbeat_front_addr":"172.21.15.151:6805/3368304087","state":["exists","up"]},{"osd":2,"uuid":"c2f18115-013d-4da6-a0fe-cb978f4494b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6823","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6825","nonce":3079373271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6827","nonce":3079373271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6831","nonce":3079373271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6829","nonce":3079373271}]},"public_addr":"172.21.15.151:6825/3079373271","cluster_addr":"172.21.15.151:6827/3079373271","heartbeat_back_addr":"172.21.15.151:6831/3079373271","heartbeat_front_addr":"172.21.15.151:6829/3079373271","state":["exists","up"]},{"osd":3,"uuid":"36edb718-2fcb-4d88-8ddc-c321eed4279d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6817","nonce":1142509041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6819","nonce":1142509041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6824","nonce":1142509041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6821","nonce":1142509041}]},"public_addr":"172.21.15.151:6817/1142509041","cluster_addr":"172.21.15.151:6819/1142509041","heartbeat_back_addr":"172.21.15.151:6824/1142509041","heartbeat_front_addr":"172.21.15.151:6821/1142509041","state":["exists","up"]},{"osd":4,"uuid":"64d33708-cd0a-4236-abe3-1b646e82ab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6811","nonce":2735495271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6813","nonce":2735495271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6817","nonce":2735495271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6815","nonce":2735495271}]},"public_addr":"172.21.15.167:6811/2735495271","cluster_addr":"172.21.15.167:6813/2735495271","heartbeat_back_addr":"172.21.15.167:6817/2735495271","heartbeat_front_addr":"172.21.15.167:6815/2735495271","state":["exists","up"]},{"osd":5,"uuid":"f14b44a9-5910-458c-b518-09a8d0de94de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6827","nonce":1494916572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6829","nonce":1494916572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6833","nonce":1494916572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6831","nonce":1494916572}]},"public_addr":"172.21.15.167:6827/1494916572","cluster_addr":"172.21.15.167:6829/1494916572","heartbeat_back_addr":"172.21.15.167:6833/1494916572","heartbeat_front_addr":"172.21.15.167:6831/1494916572","state":["exists","up"]},{"osd":6,"uuid":"39335545-2654-4710-84b0-a9aa47cfcbba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6819","nonce":2538273273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6821","nonce":2538273273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6825","nonce":2538273273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6823","nonce":2538273273}]},"public_addr":"172.21.15.167:6819/2538273273","cluster_addr":"172.21.15.167:6821/2538273273","heartbeat_back_addr":"172.21.15.167:6825/2538273273","heartbeat_front_addr":"172.21.15.167:6823/2538273273","state":["exists","up"]},{"osd":7,"uuid":"4f8bfc81-53dd-4339-ac22-1a4007d9f963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6803","nonce":265072227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6805","nonce":265072227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6809","nonce":265072227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6807","nonce":265072227}]},"public_addr":"172.21.15.167:6803/265072227","cluster_addr":"172.21.15.167:6805/265072227","heartbeat_back_addr":"172.21.15.167:6809/265072227","heartbeat_front_addr":"172.21.15.167:6807/265072227","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.150359+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.084358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.722368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.332809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.320709+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.395153+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.935250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.801443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:39:26.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.363+0000 7f75b3200640 1 -- 172.21.15.151:0/2414578499 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7594078880 msgr2=0x7f759407ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:26.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.363+0000 7f75b3200640 1 --2- 172.21.15.151:0/2414578499 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7594078880 0x7f759407ad40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f75a40098e0 tx=0x7f75a4009060 comp rx=0 tx=0).stop 2024-09-16T15:39:26.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.363+0000 7f75b3200640 1 -- 172.21.15.151:0/2414578499 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75b4161440 msgr2=0x7f75b41579e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:26.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.363+0000 7f75b3200640 1 --2- 172.21.15.151:0/2414578499 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75b4161440 0x7f75b41579e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f75ac00c780 tx=0x7f75ac00cc40 comp rx=0 tx=0).stop 2024-09-16T15:39:26.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.364+0000 7f75b3200640 1 -- 172.21.15.151:0/2414578499 shutdown_connections 2024-09-16T15:39:26.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.364+0000 7f75b3200640 1 --2- 172.21.15.151:0/2414578499 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7594078880 0x7f759407ad40 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:26.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.364+0000 7f75b3200640 1 --2- 172.21.15.151:0/2414578499 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75b4161440 0x7f75b41579e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:26.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.364+0000 7f75b3200640 1 --2- 172.21.15.151:0/2414578499 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f75b40b3e50 0x7f75b41f4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:26.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.364+0000 7f75b3200640 1 --2- 172.21.15.151:0/2414578499 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75b40b34b0 0x7f75b40b8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:26.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.364+0000 7f75b3200640 1 -- 172.21.15.151:0/2414578499 >> 172.21.15.151:0/2414578499 conn(0x7f75b40d3df0 msgr2=0x7f75b40d41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:26.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.364+0000 7f75b3200640 1 -- 172.21.15.151:0/2414578499 shutdown_connections 2024-09-16T15:39:26.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.364+0000 7f75b3200640 1 -- 172.21.15.151:0/2414578499 wait complete. 2024-09-16T15:39:26.380 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T15:39:26.381 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:39:26.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.536+0000 7fefd2c00640 1 -- 172.21.15.151:0/4251048239 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fefcc073fb0 msgr2=0x7fefcc074440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:26.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.536+0000 7fefd2c00640 1 --2- 172.21.15.151:0/4251048239 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fefcc073fb0 0x7fefcc074440 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fefc4005fd0 tx=0x7fefc402aa10 comp rx=0 tx=0).stop 2024-09-16T15:39:26.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.536+0000 7fefd2c00640 1 -- 172.21.15.151:0/4251048239 shutdown_connections 2024-09-16T15:39:26.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.536+0000 7fefd2c00640 1 --2- 172.21.15.151:0/4251048239 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fefcc073fb0 0x7fefcc074440 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:26.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.536+0000 7fefd2c00640 1 --2- 172.21.15.151:0/4251048239 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fefcc0ccf00 0x7fefcc0b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:26.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.536+0000 7fefd2c00640 1 -- 172.21.15.151:0/4251048239 >> 172.21.15.151:0/4251048239 conn(0x7fefcc0b1950 msgr2=0x7fefcc0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:26.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.536+0000 7fefd2c00640 1 -- 172.21.15.151:0/4251048239 shutdown_connections 2024-09-16T15:39:26.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.536+0000 7fefd2c00640 1 -- 172.21.15.151:0/4251048239 wait complete. 2024-09-16T15:39:26.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.538+0000 7fefd2c00640 1 Processor -- start 2024-09-16T15:39:26.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.538+0000 7fefd2c00640 1 -- start start 2024-09-16T15:39:26.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.539+0000 7fefd2c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fefcc073fb0 0x7fefcc146660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:26.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.539+0000 7fefd2c00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fefcc0ccf00 0x7fefcc1f8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:26.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.539+0000 7fefd2c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fefcc15f5a0 0x7fefcc151e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:26.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.539+0000 7fefd2c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fefcc06f2e0 con 0x7fefcc15f5a0 2024-09-16T15:39:26.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.539+0000 7fefd2c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fefcc06f160 con 0x7fefcc073fb0 2024-09-16T15:39:26.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.539+0000 7fefd2c00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fefcc06f460 con 0x7fefcc0ccf00 2024-09-16T15:39:26.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.539+0000 7fefd0c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fefcc15f5a0 0x7fefcc151e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:26.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.539+0000 7fefd0c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fefcc15f5a0 0x7fefcc151e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47252/0 (socket says 172.21.15.151:47252) 2024-09-16T15:39:26.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.539+0000 7fefd0c00640 1 -- 172.21.15.151:0/4261821052 learned_addr learned my addr 172.21.15.151:0/4261821052 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:26.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.539+0000 7fefcbe00640 1 --2- 172.21.15.151:0/4261821052 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fefcc073fb0 0x7fefcc146660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:26.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.539+0000 7fefcb400640 1 --2- 172.21.15.151:0/4261821052 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fefcc0ccf00 0x7fefcc1f8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:26.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.539+0000 7fefd0c00640 1 -- 172.21.15.151:0/4261821052 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fefcc0ccf00 msgr2=0x7fefcc1f8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:26.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.539+0000 7fefd0c00640 1 --2- 172.21.15.151:0/4261821052 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fefcc0ccf00 0x7fefcc1f8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:26.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.539+0000 7fefd0c00640 1 -- 172.21.15.151:0/4261821052 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fefcc073fb0 msgr2=0x7fefcc146660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:26.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.539+0000 7fefd0c00640 1 --2- 172.21.15.151:0/4261821052 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fefcc073fb0 0x7fefcc146660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:26.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.539+0000 7fefd0c00640 1 -- 172.21.15.151:0/4261821052 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefcc1f89f0 con 0x7fefcc15f5a0 2024-09-16T15:39:26.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.540+0000 7fefd0c00640 1 --2- 172.21.15.151:0/4261821052 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fefcc15f5a0 0x7fefcc151e60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fefc4009d90 tx=0x7fefc4003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:26.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.540+0000 7fefc3e00640 1 -- 172.21.15.151:0/4261821052 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc4047070 con 0x7fefcc15f5a0 2024-09-16T15:39:26.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.540+0000 7fefc3e00640 1 -- 172.21.15.151:0/4261821052 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fefc4004250 con 0x7fefcc15f5a0 2024-09-16T15:39:26.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.540+0000 7fefc3e00640 1 -- 172.21.15.151:0/4261821052 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc4033050 con 0x7fefcc15f5a0 2024-09-16T15:39:26.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.540+0000 7fefd2c00640 1 -- 172.21.15.151:0/4261821052 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefcc1f8cd0 con 0x7fefcc15f5a0 2024-09-16T15:39:26.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.540+0000 7fefd2c00640 1 -- 172.21.15.151:0/4261821052 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefcc0b2d50 con 0x7fefcc15f5a0 2024-09-16T15:39:26.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.541+0000 7fefd2c00640 1 -- 172.21.15.151:0/4261821052 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefcc06f9f0 con 0x7fefcc15f5a0 2024-09-16T15:39:26.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.542+0000 7fefc3e00640 1 -- 172.21.15.151:0/4261821052 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fefc404a070 con 0x7fefcc15f5a0 2024-09-16T15:39:26.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.543+0000 7fefc3e00640 1 --2- 172.21.15.151:0/4261821052 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fefac078850 0x7fefac07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:26.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.543+0000 7fefc3e00640 1 -- 172.21.15.151:0/4261821052 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fefc40bbc40 con 0x7fefcc15f5a0 2024-09-16T15:39:26.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.544+0000 7fefcbe00640 1 --2- 172.21.15.151:0/4261821052 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fefac078850 0x7fefac07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:26.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.544+0000 7fefcbe00640 1 --2- 172.21.15.151:0/4261821052 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fefac078850 0x7fefac07ad10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fefcc09b860 tx=0x7fefbc009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:26.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.547+0000 7fefc3e00640 1 -- 172.21.15.151:0/4261821052 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fefc4051080 con 0x7fefcc15f5a0 2024-09-16T15:39:26.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.730+0000 7fefd2c00640 1 -- 172.21.15.151:0/4261821052 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fefcc09b5b0 con 0x7fefac078850 2024-09-16T15:39:26.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.736+0000 7fefc3e00640 1 -- 172.21.15.151:0/4261821052 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240885 (secure 0 0 0) 0x7fefcc09b5b0 con 0x7fefac078850 2024-09-16T15:39:26.740 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:39:26.741 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-09-16T15:39:26.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.740+0000 7fefd2c00640 1 -- 172.21.15.151:0/4261821052 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fefac078850 msgr2=0x7fefac07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:26.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.740+0000 7fefd2c00640 1 --2- 172.21.15.151:0/4261821052 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fefac078850 0x7fefac07ad10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fefcc09b860 tx=0x7fefbc009040 comp rx=0 tx=0).stop 2024-09-16T15:39:26.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.740+0000 7fefd2c00640 1 -- 172.21.15.151:0/4261821052 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fefcc15f5a0 msgr2=0x7fefcc151e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:26.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.740+0000 7fefd2c00640 1 --2- 172.21.15.151:0/4261821052 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fefcc15f5a0 0x7fefcc151e60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fefc4009d90 tx=0x7fefc4003f10 comp rx=0 tx=0).stop 2024-09-16T15:39:26.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.740+0000 7fefd2c00640 1 -- 172.21.15.151:0/4261821052 shutdown_connections 2024-09-16T15:39:26.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.740+0000 7fefd2c00640 1 --2- 172.21.15.151:0/4261821052 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fefac078850 0x7fefac07ad10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:26.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.740+0000 7fefd2c00640 1 --2- 172.21.15.151:0/4261821052 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fefcc15f5a0 0x7fefcc151e60 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:26.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.740+0000 7fefd2c00640 1 --2- 172.21.15.151:0/4261821052 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fefcc0ccf00 0x7fefcc1f8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:26.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.740+0000 7fefd2c00640 1 --2- 172.21.15.151:0/4261821052 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fefcc073fb0 0x7fefcc146660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:26.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.740+0000 7fefd2c00640 1 -- 172.21.15.151:0/4261821052 >> 172.21.15.151:0/4261821052 conn(0x7fefcc0b1950 msgr2=0x7fefcc09aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:26.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.741+0000 7fefd2c00640 1 -- 172.21.15.151:0/4261821052 shutdown_connections 2024-09-16T15:39:26.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.741+0000 7fefd2c00640 1 -- 172.21.15.151:0/4261821052 wait complete. 2024-09-16T15:39:26.757 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-16T15:39:25.577163+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590492,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":108,"num_write_kb":1636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77,"ondisk_log_size":77,"up":72,"acting":72,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":27,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219316,"kb_used_data":2572,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755404,"statfs":{"total":773094113280,"available":772869533696,"internally_reserved":0,"allocated":2633728,"data_stored":1620780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12738,"internal_metadata":221236798},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":124,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.044880"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.714434+0000","last_change":"2024-09-16T15:39:22.579035+0000","last_active":"2024-09-16T15:39:23.714434+0000","last_peered":"2024-09-16T15:39:23.714434+0000","last_clean":"2024-09-16T15:39:23.714434+0000","last_became_active":"2024-09-16T15:39:22.578734+0000","last_became_peered":"2024-09-16T15:39:22.578734+0000","last_unstale":"2024-09-16T15:39:23.714434+0000","last_undegraded":"2024-09-16T15:39:23.714434+0000","last_fullsized":"2024-09-16T15:39:23.714434+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:13:19.369864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.714354+0000","last_change":"2024-09-16T15:39:22.566051+0000","last_active":"2024-09-16T15:39:23.714354+0000","last_peered":"2024-09-16T15:39:23.714354+0000","last_clean":"2024-09-16T15:39:23.714354+0000","last_became_active":"2024-09-16T15:39:22.565764+0000","last_became_peered":"2024-09-16T15:39:22.565764+0000","last_unstale":"2024-09-16T15:39:23.714354+0000","last_undegraded":"2024-09-16T15:39:23.714354+0000","last_fullsized":"2024-09-16T15:39:23.714354+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:42:34.417720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.536611+0000","last_change":"2024-09-16T15:39:20.536611+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.536611+0000","last_undegraded":"2024-09-16T15:39:20.536611+0000","last_fullsized":"2024-09-16T15:39:20.536611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.544697+0000","last_change":"2024-09-16T15:39:20.544697+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.544697+0000","last_undegraded":"2024-09-16T15:39:20.544697+0000","last_fullsized":"2024-09-16T15:39:20.544697+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.541664+0000","last_change":"2024-09-16T15:39:20.541664+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.541664+0000","last_undegraded":"2024-09-16T15:39:20.541664+0000","last_fullsized":"2024-09-16T15:39:20.541664+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.544178+0000","last_change":"2024-09-16T15:39:20.544178+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.544178+0000","last_undegraded":"2024-09-16T15:39:20.544178+0000","last_fullsized":"2024-09-16T15:39:20.544178+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.544198+0000","last_change":"2024-09-16T15:39:20.544198+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.544198+0000","last_undegraded":"2024-09-16T15:39:20.544198+0000","last_fullsized":"2024-09-16T15:39:20.544198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.538592+0000","last_change":"2024-09-16T15:39:20.538592+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.538592+0000","last_undegraded":"2024-09-16T15:39:20.538592+0000","last_fullsized":"2024-09-16T15:39:20.538592+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.714227+0000","last_change":"2024-09-16T15:39:21.561906+0000","last_active":"2024-09-16T15:39:23.714227+0000","last_peered":"2024-09-16T15:39:23.714227+0000","last_clean":"2024-09-16T15:39:23.714227+0000","last_became_active":"2024-09-16T15:39:21.561566+0000","last_became_peered":"2024-09-16T15:39:21.561566+0000","last_unstale":"2024-09-16T15:39:23.714227+0000","last_undegraded":"2024-09-16T15:39:23.714227+0000","last_fullsized":"2024-09-16T15:39:23.714227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:12:53.853659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.535462+0000","last_change":"2024-09-16T15:39:20.535462+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.535462+0000","last_undegraded":"2024-09-16T15:39:20.535462+0000","last_fullsized":"2024-09-16T15:39:20.535462+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.714255+0000","last_change":"2024-09-16T15:39:21.567091+0000","last_active":"2024-09-16T15:39:23.714255+0000","last_peered":"2024-09-16T15:39:23.714255+0000","last_clean":"2024-09-16T15:39:23.714255+0000","last_became_active":"2024-09-16T15:39:21.566668+0000","last_became_peered":"2024-09-16T15:39:21.566668+0000","last_unstale":"2024-09-16T15:39:23.714255+0000","last_undegraded":"2024-09-16T15:39:23.714255+0000","last_fullsized":"2024-09-16T15:39:23.714255+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:08:06.242307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.544145+0000","last_change":"2024-09-16T15:39:20.544145+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.544145+0000","last_undegraded":"2024-09-16T15:39:20.544145+0000","last_fullsized":"2024-09-16T15:39:20.544145+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.537994+0000","last_change":"2024-09-16T15:39:20.537994+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.537994+0000","last_undegraded":"2024-09-16T15:39:20.537994+0000","last_fullsized":"2024-09-16T15:39:20.537994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.534947+0000","last_change":"2024-09-16T15:39:20.534947+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.534947+0000","last_undegraded":"2024-09-16T15:39:20.534947+0000","last_fullsized":"2024-09-16T15:39:20.534947+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.537954+0000","last_change":"2024-09-16T15:39:20.537954+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.537954+0000","last_undegraded":"2024-09-16T15:39:20.537954+0000","last_fullsized":"2024-09-16T15:39:20.537954+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.535736+0000","last_change":"2024-09-16T15:39:20.535736+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.535736+0000","last_undegraded":"2024-09-16T15:39:20.535736+0000","last_fullsized":"2024-09-16T15:39:20.535736+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.536294+0000","last_change":"2024-09-16T15:39:20.536294+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.536294+0000","last_undegraded":"2024-09-16T15:39:20.536294+0000","last_fullsized":"2024-09-16T15:39:20.536294+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.541319+0000","last_change":"2024-09-16T15:39:20.541319+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.541319+0000","last_undegraded":"2024-09-16T15:39:20.541319+0000","last_fullsized":"2024-09-16T15:39:20.541319+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":105,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-16T15:39:20.545567+0000","last_change":"2024-09-16T15:39:12.698813+0000","last_active":"2024-09-16T15:39:20.545567+0000","last_peered":"2024-09-16T15:39:20.545567+0000","last_clean":"2024-09-16T15:39:20.545567+0000","last_became_active":"2024-09-16T15:39:12.697695+0000","last_became_peered":"2024-09-16T15:39:12.697695+0000","last_unstale":"2024-09-16T15:39:20.545567+0000","last_undegraded":"2024-09-16T15:39:20.545567+0000","last_fullsized":"2024-09-16T15:39:20.545567+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_clean_scrub_stamp":"2024-09-16T15:39:11.684967+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:02.061081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.543255+0000","last_change":"2024-09-16T15:39:20.543255+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.543255+0000","last_undegraded":"2024-09-16T15:39:20.543255+0000","last_fullsized":"2024-09-16T15:39:20.543255+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.543264+0000","last_change":"2024-09-16T15:39:20.543264+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.543264+0000","last_undegraded":"2024-09-16T15:39:20.543264+0000","last_fullsized":"2024-09-16T15:39:20.543264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.714355+0000","last_change":"2024-09-16T15:39:21.563752+0000","last_active":"2024-09-16T15:39:23.714355+0000","last_peered":"2024-09-16T15:39:23.714355+0000","last_clean":"2024-09-16T15:39:23.714355+0000","last_became_active":"2024-09-16T15:39:21.563430+0000","last_became_peered":"2024-09-16T15:39:21.563430+0000","last_unstale":"2024-09-16T15:39:23.714355+0000","last_undegraded":"2024-09-16T15:39:23.714355+0000","last_fullsized":"2024-09-16T15:39:23.714355+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:44:53.582663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.543244+0000","last_change":"2024-09-16T15:39:20.543244+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.543244+0000","last_undegraded":"2024-09-16T15:39:20.543244+0000","last_fullsized":"2024-09-16T15:39:20.543244+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.535059+0000","last_change":"2024-09-16T15:39:20.535059+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.535059+0000","last_undegraded":"2024-09-16T15:39:20.535059+0000","last_fullsized":"2024-09-16T15:39:20.535059+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.543480+0000","last_change":"2024-09-16T15:39:20.543480+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.543480+0000","last_undegraded":"2024-09-16T15:39:20.543480+0000","last_fullsized":"2024-09-16T15:39:20.543480+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.535043+0000","last_change":"2024-09-16T15:39:20.535043+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.535043+0000","last_undegraded":"2024-09-16T15:39:20.535043+0000","last_fullsized":"2024-09-16T15:39:20.535043+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.543478+0000","last_change":"2024-09-16T15:39:20.543478+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.543478+0000","last_undegraded":"2024-09-16T15:39:20.543478+0000","last_fullsized":"2024-09-16T15:39:20.543478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.538101+0000","last_change":"2024-09-16T15:39:20.538101+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.538101+0000","last_undegraded":"2024-09-16T15:39:20.538101+0000","last_fullsized":"2024-09-16T15:39:20.538101+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.534937+0000","last_change":"2024-09-16T15:39:20.534937+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.534937+0000","last_undegraded":"2024-09-16T15:39:20.534937+0000","last_fullsized":"2024-09-16T15:39:20.534937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.541272+0000","last_change":"2024-09-16T15:39:20.541272+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.541272+0000","last_undegraded":"2024-09-16T15:39:20.541272+0000","last_fullsized":"2024-09-16T15:39:20.541272+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.714500+0000","last_change":"2024-09-16T15:39:22.576232+0000","last_active":"2024-09-16T15:39:23.714500+0000","last_peered":"2024-09-16T15:39:23.714500+0000","last_clean":"2024-09-16T15:39:23.714500+0000","last_became_active":"2024-09-16T15:39:22.575962+0000","last_became_peered":"2024-09-16T15:39:22.575962+0000","last_unstale":"2024-09-16T15:39:23.714500+0000","last_undegraded":"2024-09-16T15:39:23.714500+0000","last_fullsized":"2024-09-16T15:39:23.714500+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:43:58.162669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.544248+0000","last_change":"2024-09-16T15:39:20.544248+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.544248+0000","last_undegraded":"2024-09-16T15:39:20.544248+0000","last_fullsized":"2024-09-16T15:39:20.544248+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.537887+0000","last_change":"2024-09-16T15:39:20.537887+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.537887+0000","last_undegraded":"2024-09-16T15:39:20.537887+0000","last_fullsized":"2024-09-16T15:39:20.537887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.538384+0000","last_change":"2024-09-16T15:39:20.538384+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.538384+0000","last_undegraded":"2024-09-16T15:39:20.538384+0000","last_fullsized":"2024-09-16T15:39:20.538384+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.534421+0000","last_change":"2024-09-16T15:39:20.534421+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.534421+0000","last_undegraded":"2024-09-16T15:39:20.534421+0000","last_fullsized":"2024-09-16T15:39:20.534421+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.535936+0000","last_change":"2024-09-16T15:39:20.535936+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.535936+0000","last_undegraded":"2024-09-16T15:39:20.535936+0000","last_fullsized":"2024-09-16T15:39:20.535936+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":6,"acting":6,"num_store_stats":1},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":124,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":172032,"data_stored":51117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28024,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343816,"statfs":{"total":96636764160,"available":96608067584,"internally_reserved":0,"allocated":765952,"data_stored":641485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":172032,"data_stored":51117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509444,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":172032,"data_stored":51117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":56934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28024,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343816,"statfs":{"total":96636764160,"available":96608067584,"internally_reserved":0,"allocated":765952,"data_stored":641485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":172032,"data_stored":51117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542151,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":76408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":27654604},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:39:26.761 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:39:26.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.953+0000 7f0cb9400640 1 -- 172.21.15.151:0/3768232581 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0cb4073fb0 msgr2=0x7f0cb4074420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:26.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.953+0000 7f0cb9400640 1 --2- 172.21.15.151:0/3768232581 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0cb4073fb0 0x7f0cb4074420 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0cb40cfbb0 tx=0x7f0cac02a4d0 comp rx=0 tx=0).stop 2024-09-16T15:39:26.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.953+0000 7f0cb9400640 1 -- 172.21.15.151:0/3768232581 shutdown_connections 2024-09-16T15:39:26.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.953+0000 7f0cb9400640 1 --2- 172.21.15.151:0/3768232581 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0cb4073fb0 0x7f0cb4074420 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:26.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.953+0000 7f0cb9400640 1 --2- 172.21.15.151:0/3768232581 >> v2:172.21.15.167:3300/0 conn(0x7f0cb40ccf00 0x7f0cb40b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:26.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.953+0000 7f0cb9400640 1 -- 172.21.15.151:0/3768232581 >> 172.21.15.151:0/3768232581 conn(0x7f0cb40b1950 msgr2=0x7f0cb40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:26.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.953+0000 7f0cb9400640 1 -- 172.21.15.151:0/3768232581 shutdown_connections 2024-09-16T15:39:26.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.953+0000 7f0cb9400640 1 -- 172.21.15.151:0/3768232581 wait complete. 2024-09-16T15:39:26.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.955+0000 7f0cb9400640 1 Processor -- start 2024-09-16T15:39:26.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.955+0000 7f0cb9400640 1 -- start start 2024-09-16T15:39:26.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.956+0000 7f0cb9400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0cb4073fb0 0x7f0cb41f87d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:26.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.956+0000 7f0cb9400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0cb40ccf00 0x7f0cb40b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:26.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.956+0000 7f0cb9400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0cb415f5a0 0x7f0cb41f41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:26.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.956+0000 7f0cb9400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb406f2e0 con 0x7f0cb4073fb0 2024-09-16T15:39:26.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.956+0000 7f0cb9400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb406f160 con 0x7f0cb415f5a0 2024-09-16T15:39:26.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.956+0000 7f0cb9400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f0cb406f460 con 0x7f0cb40ccf00 2024-09-16T15:39:26.959 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.956+0000 7f0cb2800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0cb4073fb0 0x7f0cb41f87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:26.959 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.956+0000 7f0cb2800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0cb4073fb0 0x7f0cb41f87d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47272/0 (socket says 172.21.15.151:47272) 2024-09-16T15:39:26.959 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.956+0000 7f0cb2800640 1 -- 172.21.15.151:0/966164401 learned_addr learned my addr 172.21.15.151:0/966164401 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:26.959 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.956+0000 7f0cb3200640 1 --2- 172.21.15.151:0/966164401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0cb415f5a0 0x7f0cb41f41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:26.959 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.956+0000 7f0cb1e00640 1 --2- 172.21.15.151:0/966164401 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0cb40ccf00 0x7f0cb40b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:26.959 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.957+0000 7f0cb2800640 1 -- 172.21.15.151:0/966164401 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0cb40ccf00 msgr2=0x7f0cb40b48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:26.959 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.957+0000 7f0cb2800640 1 --2- 172.21.15.151:0/966164401 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0cb40ccf00 0x7f0cb40b48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:26.959 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.957+0000 7f0cb2800640 1 -- 172.21.15.151:0/966164401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0cb415f5a0 msgr2=0x7f0cb41f41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:26.959 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.957+0000 7f0cb2800640 1 --2- 172.21.15.151:0/966164401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0cb415f5a0 0x7f0cb41f41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:26.959 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.957+0000 7f0cb2800640 1 -- 172.21.15.151:0/966164401 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cb40b2ef0 con 0x7f0cb4073fb0 2024-09-16T15:39:26.959 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.957+0000 7f0cb2800640 1 --2- 172.21.15.151:0/966164401 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0cb4073fb0 0x7f0cb41f87d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0ca400ba50 tx=0x7f0ca400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:26.959 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.957+0000 7f0caaa00640 1 -- 172.21.15.151:0/966164401 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca4019070 con 0x7f0cb4073fb0 2024-09-16T15:39:26.959 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.957+0000 7f0caaa00640 1 -- 172.21.15.151:0/966164401 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0ca400cc40 con 0x7f0cb4073fb0 2024-09-16T15:39:26.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.957+0000 7f0caaa00640 1 -- 172.21.15.151:0/966164401 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca40146a0 con 0x7f0cb4073fb0 2024-09-16T15:39:26.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.957+0000 7f0cb9400640 1 -- 172.21.15.151:0/966164401 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cb40b31d0 con 0x7f0cb4073fb0 2024-09-16T15:39:26.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.957+0000 7f0cb9400640 1 -- 172.21.15.151:0/966164401 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cb40b3cb0 con 0x7f0cb4073fb0 2024-09-16T15:39:26.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.958+0000 7f0cb9400640 1 -- 172.21.15.151:0/966164401 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cb406fa10 con 0x7f0cb4073fb0 2024-09-16T15:39:26.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.960+0000 7f0caaa00640 1 -- 172.21.15.151:0/966164401 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f0ca401c070 con 0x7f0cb4073fb0 2024-09-16T15:39:26.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.960+0000 7f0caaa00640 1 --2- 172.21.15.151:0/966164401 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0c94078850 0x7f0c9407ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:26.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.961+0000 7f0caaa00640 1 -- 172.21.15.151:0/966164401 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f0ca409a660 con 0x7f0cb4073fb0 2024-09-16T15:39:26.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.961+0000 7f0cb1e00640 1 --2- 172.21.15.151:0/966164401 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0c94078850 0x7f0c9407ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:26.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.962+0000 7f0cb1e00640 1 --2- 172.21.15.151:0/966164401 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0c94078850 0x7f0c9407ad10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0cb415c8e0 tx=0x7f0ca0009110 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:26.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:26.964+0000 7f0caaa00640 1 -- 172.21.15.151:0/966164401 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0ca40625e0 con 0x7f0cb4073fb0 2024-09-16T15:39:27.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.146+0000 7f0cb9400640 1 -- 172.21.15.151:0/966164401 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0cb40cbef0 con 0x7f0c94078850 2024-09-16T15:39:27.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.151+0000 7f0caaa00640 1 -- 172.21.15.151:0/966164401 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240885 (secure 0 0 0) 0x7f0cb40cbef0 con 0x7f0c94078850 2024-09-16T15:39:27.154 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:39:27.155 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-09-16T15:39:27.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.155+0000 7f0cb9400640 1 -- 172.21.15.151:0/966164401 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0c94078850 msgr2=0x7f0c9407ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:27.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.155+0000 7f0cb9400640 1 --2- 172.21.15.151:0/966164401 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0c94078850 0x7f0c9407ad10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0cb415c8e0 tx=0x7f0ca0009110 comp rx=0 tx=0).stop 2024-09-16T15:39:27.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.155+0000 7f0cb9400640 1 -- 172.21.15.151:0/966164401 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0cb4073fb0 msgr2=0x7f0cb41f87d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:27.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.155+0000 7f0cb9400640 1 --2- 172.21.15.151:0/966164401 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0cb4073fb0 0x7f0cb41f87d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0ca400ba50 tx=0x7f0ca400be50 comp rx=0 tx=0).stop 2024-09-16T15:39:27.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.155+0000 7f0cb9400640 1 -- 172.21.15.151:0/966164401 shutdown_connections 2024-09-16T15:39:27.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.155+0000 7f0cb9400640 1 --2- 172.21.15.151:0/966164401 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0c94078850 0x7f0c9407ad10 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:27.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.155+0000 7f0cb9400640 1 --2- 172.21.15.151:0/966164401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0cb415f5a0 0x7f0cb41f41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:27.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.155+0000 7f0cb9400640 1 --2- 172.21.15.151:0/966164401 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0cb40ccf00 0x7f0cb40b48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:27.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.155+0000 7f0cb9400640 1 --2- 172.21.15.151:0/966164401 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0cb4073fb0 0x7f0cb41f87d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:27.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.155+0000 7f0cb9400640 1 -- 172.21.15.151:0/966164401 >> 172.21.15.151:0/966164401 conn(0x7f0cb40b1950 msgr2=0x7f0cb409cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:27.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.155+0000 7f0cb9400640 1 -- 172.21.15.151:0/966164401 shutdown_connections 2024-09-16T15:39:27.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.155+0000 7f0cb9400640 1 -- 172.21.15.151:0/966164401 wait complete. 2024-09-16T15:39:27.171 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-16T15:39:25.577163+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590492,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":108,"num_write_kb":1636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77,"ondisk_log_size":77,"up":72,"acting":72,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":27,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219316,"kb_used_data":2572,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755404,"statfs":{"total":773094113280,"available":772869533696,"internally_reserved":0,"allocated":2633728,"data_stored":1620780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12738,"internal_metadata":221236798},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":124,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.044880"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.714434+0000","last_change":"2024-09-16T15:39:22.579035+0000","last_active":"2024-09-16T15:39:23.714434+0000","last_peered":"2024-09-16T15:39:23.714434+0000","last_clean":"2024-09-16T15:39:23.714434+0000","last_became_active":"2024-09-16T15:39:22.578734+0000","last_became_peered":"2024-09-16T15:39:22.578734+0000","last_unstale":"2024-09-16T15:39:23.714434+0000","last_undegraded":"2024-09-16T15:39:23.714434+0000","last_fullsized":"2024-09-16T15:39:23.714434+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:13:19.369864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.714354+0000","last_change":"2024-09-16T15:39:22.566051+0000","last_active":"2024-09-16T15:39:23.714354+0000","last_peered":"2024-09-16T15:39:23.714354+0000","last_clean":"2024-09-16T15:39:23.714354+0000","last_became_active":"2024-09-16T15:39:22.565764+0000","last_became_peered":"2024-09-16T15:39:22.565764+0000","last_unstale":"2024-09-16T15:39:23.714354+0000","last_undegraded":"2024-09-16T15:39:23.714354+0000","last_fullsized":"2024-09-16T15:39:23.714354+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:42:34.417720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.536611+0000","last_change":"2024-09-16T15:39:20.536611+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.536611+0000","last_undegraded":"2024-09-16T15:39:20.536611+0000","last_fullsized":"2024-09-16T15:39:20.536611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.544697+0000","last_change":"2024-09-16T15:39:20.544697+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.544697+0000","last_undegraded":"2024-09-16T15:39:20.544697+0000","last_fullsized":"2024-09-16T15:39:20.544697+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.541664+0000","last_change":"2024-09-16T15:39:20.541664+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.541664+0000","last_undegraded":"2024-09-16T15:39:20.541664+0000","last_fullsized":"2024-09-16T15:39:20.541664+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.544178+0000","last_change":"2024-09-16T15:39:20.544178+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.544178+0000","last_undegraded":"2024-09-16T15:39:20.544178+0000","last_fullsized":"2024-09-16T15:39:20.544178+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.544198+0000","last_change":"2024-09-16T15:39:20.544198+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.544198+0000","last_undegraded":"2024-09-16T15:39:20.544198+0000","last_fullsized":"2024-09-16T15:39:20.544198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.538592+0000","last_change":"2024-09-16T15:39:20.538592+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.538592+0000","last_undegraded":"2024-09-16T15:39:20.538592+0000","last_fullsized":"2024-09-16T15:39:20.538592+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.714227+0000","last_change":"2024-09-16T15:39:21.561906+0000","last_active":"2024-09-16T15:39:23.714227+0000","last_peered":"2024-09-16T15:39:23.714227+0000","last_clean":"2024-09-16T15:39:23.714227+0000","last_became_active":"2024-09-16T15:39:21.561566+0000","last_became_peered":"2024-09-16T15:39:21.561566+0000","last_unstale":"2024-09-16T15:39:23.714227+0000","last_undegraded":"2024-09-16T15:39:23.714227+0000","last_fullsized":"2024-09-16T15:39:23.714227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:12:53.853659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.535462+0000","last_change":"2024-09-16T15:39:20.535462+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.535462+0000","last_undegraded":"2024-09-16T15:39:20.535462+0000","last_fullsized":"2024-09-16T15:39:20.535462+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.714255+0000","last_change":"2024-09-16T15:39:21.567091+0000","last_active":"2024-09-16T15:39:23.714255+0000","last_peered":"2024-09-16T15:39:23.714255+0000","last_clean":"2024-09-16T15:39:23.714255+0000","last_became_active":"2024-09-16T15:39:21.566668+0000","last_became_peered":"2024-09-16T15:39:21.566668+0000","last_unstale":"2024-09-16T15:39:23.714255+0000","last_undegraded":"2024-09-16T15:39:23.714255+0000","last_fullsized":"2024-09-16T15:39:23.714255+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:08:06.242307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.544145+0000","last_change":"2024-09-16T15:39:20.544145+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.544145+0000","last_undegraded":"2024-09-16T15:39:20.544145+0000","last_fullsized":"2024-09-16T15:39:20.544145+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.537994+0000","last_change":"2024-09-16T15:39:20.537994+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.537994+0000","last_undegraded":"2024-09-16T15:39:20.537994+0000","last_fullsized":"2024-09-16T15:39:20.537994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.534947+0000","last_change":"2024-09-16T15:39:20.534947+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.534947+0000","last_undegraded":"2024-09-16T15:39:20.534947+0000","last_fullsized":"2024-09-16T15:39:20.534947+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.537954+0000","last_change":"2024-09-16T15:39:20.537954+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.537954+0000","last_undegraded":"2024-09-16T15:39:20.537954+0000","last_fullsized":"2024-09-16T15:39:20.537954+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.535736+0000","last_change":"2024-09-16T15:39:20.535736+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.535736+0000","last_undegraded":"2024-09-16T15:39:20.535736+0000","last_fullsized":"2024-09-16T15:39:20.535736+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.536294+0000","last_change":"2024-09-16T15:39:20.536294+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.536294+0000","last_undegraded":"2024-09-16T15:39:20.536294+0000","last_fullsized":"2024-09-16T15:39:20.536294+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.541319+0000","last_change":"2024-09-16T15:39:20.541319+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.541319+0000","last_undegraded":"2024-09-16T15:39:20.541319+0000","last_fullsized":"2024-09-16T15:39:20.541319+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":105,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-16T15:39:20.545567+0000","last_change":"2024-09-16T15:39:12.698813+0000","last_active":"2024-09-16T15:39:20.545567+0000","last_peered":"2024-09-16T15:39:20.545567+0000","last_clean":"2024-09-16T15:39:20.545567+0000","last_became_active":"2024-09-16T15:39:12.697695+0000","last_became_peered":"2024-09-16T15:39:12.697695+0000","last_unstale":"2024-09-16T15:39:20.545567+0000","last_undegraded":"2024-09-16T15:39:20.545567+0000","last_fullsized":"2024-09-16T15:39:20.545567+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_clean_scrub_stamp":"2024-09-16T15:39:11.684967+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:02.061081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.543255+0000","last_change":"2024-09-16T15:39:20.543255+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.543255+0000","last_undegraded":"2024-09-16T15:39:20.543255+0000","last_fullsized":"2024-09-16T15:39:20.543255+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.543264+0000","last_change":"2024-09-16T15:39:20.543264+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.543264+0000","last_undegraded":"2024-09-16T15:39:20.543264+0000","last_fullsized":"2024-09-16T15:39:20.543264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.714355+0000","last_change":"2024-09-16T15:39:21.563752+0000","last_active":"2024-09-16T15:39:23.714355+0000","last_peered":"2024-09-16T15:39:23.714355+0000","last_clean":"2024-09-16T15:39:23.714355+0000","last_became_active":"2024-09-16T15:39:21.563430+0000","last_became_peered":"2024-09-16T15:39:21.563430+0000","last_unstale":"2024-09-16T15:39:23.714355+0000","last_undegraded":"2024-09-16T15:39:23.714355+0000","last_fullsized":"2024-09-16T15:39:23.714355+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:44:53.582663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.543244+0000","last_change":"2024-09-16T15:39:20.543244+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.543244+0000","last_undegraded":"2024-09-16T15:39:20.543244+0000","last_fullsized":"2024-09-16T15:39:20.543244+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.535059+0000","last_change":"2024-09-16T15:39:20.535059+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.535059+0000","last_undegraded":"2024-09-16T15:39:20.535059+0000","last_fullsized":"2024-09-16T15:39:20.535059+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.543480+0000","last_change":"2024-09-16T15:39:20.543480+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.543480+0000","last_undegraded":"2024-09-16T15:39:20.543480+0000","last_fullsized":"2024-09-16T15:39:20.543480+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.535043+0000","last_change":"2024-09-16T15:39:20.535043+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.535043+0000","last_undegraded":"2024-09-16T15:39:20.535043+0000","last_fullsized":"2024-09-16T15:39:20.535043+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.543478+0000","last_change":"2024-09-16T15:39:20.543478+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.543478+0000","last_undegraded":"2024-09-16T15:39:20.543478+0000","last_fullsized":"2024-09-16T15:39:20.543478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.538101+0000","last_change":"2024-09-16T15:39:20.538101+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.538101+0000","last_undegraded":"2024-09-16T15:39:20.538101+0000","last_fullsized":"2024-09-16T15:39:20.538101+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.534937+0000","last_change":"2024-09-16T15:39:20.534937+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.534937+0000","last_undegraded":"2024-09-16T15:39:20.534937+0000","last_fullsized":"2024-09-16T15:39:20.534937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.541272+0000","last_change":"2024-09-16T15:39:20.541272+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.541272+0000","last_undegraded":"2024-09-16T15:39:20.541272+0000","last_fullsized":"2024-09-16T15:39:20.541272+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.714500+0000","last_change":"2024-09-16T15:39:22.576232+0000","last_active":"2024-09-16T15:39:23.714500+0000","last_peered":"2024-09-16T15:39:23.714500+0000","last_clean":"2024-09-16T15:39:23.714500+0000","last_became_active":"2024-09-16T15:39:22.575962+0000","last_became_peered":"2024-09-16T15:39:22.575962+0000","last_unstale":"2024-09-16T15:39:23.714500+0000","last_undegraded":"2024-09-16T15:39:23.714500+0000","last_fullsized":"2024-09-16T15:39:23.714500+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:43:58.162669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.544248+0000","last_change":"2024-09-16T15:39:20.544248+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.544248+0000","last_undegraded":"2024-09-16T15:39:20.544248+0000","last_fullsized":"2024-09-16T15:39:20.544248+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.537887+0000","last_change":"2024-09-16T15:39:20.537887+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.537887+0000","last_undegraded":"2024-09-16T15:39:20.537887+0000","last_fullsized":"2024-09-16T15:39:20.537887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.538384+0000","last_change":"2024-09-16T15:39:20.538384+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.538384+0000","last_undegraded":"2024-09-16T15:39:20.538384+0000","last_fullsized":"2024-09-16T15:39:20.538384+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.534421+0000","last_change":"2024-09-16T15:39:20.534421+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.534421+0000","last_undegraded":"2024-09-16T15:39:20.534421+0000","last_fullsized":"2024-09-16T15:39:20.534421+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.535936+0000","last_change":"2024-09-16T15:39:20.535936+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.535936+0000","last_undegraded":"2024-09-16T15:39:20.535936+0000","last_fullsized":"2024-09-16T15:39:20.535936+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":6,"acting":6,"num_store_stats":1},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":124,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":172032,"data_stored":51117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28024,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343816,"statfs":{"total":96636764160,"available":96608067584,"internally_reserved":0,"allocated":765952,"data_stored":641485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":172032,"data_stored":51117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509444,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":172032,"data_stored":51117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":56934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28024,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343816,"statfs":{"total":96636764160,"available":96608067584,"internally_reserved":0,"allocated":765952,"data_stored":641485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":172032,"data_stored":51117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542151,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":76408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":27654604},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:39:27.175 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:39:27.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.370+0000 7f6ce3200640 1 -- 172.21.15.151:0/3586744451 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ce40b34b0 msgr2=0x7f6ce40b3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:27.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.370+0000 7f6ce3200640 1 --2- 172.21.15.151:0/3586744451 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ce40b34b0 0x7f6ce40b3910 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6ce40d4ea0 tx=0x7f6cd002a520 comp rx=0 tx=0).stop 2024-09-16T15:39:27.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.370+0000 7f6ce3200640 1 -- 172.21.15.151:0/3586744451 shutdown_connections 2024-09-16T15:39:27.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.370+0000 7f6ce3200640 1 --2- 172.21.15.151:0/3586744451 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ce40b34b0 0x7f6ce40b3910 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:27.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.370+0000 7f6ce3200640 1 --2- 172.21.15.151:0/3586744451 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ce4161440 0x7f6ce40b5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:27.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.370+0000 7f6ce3200640 1 -- 172.21.15.151:0/3586744451 >> 172.21.15.151:0/3586744451 conn(0x7f6ce40d3df0 msgr2=0x7f6ce40d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:27.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.370+0000 7f6ce3200640 1 -- 172.21.15.151:0/3586744451 shutdown_connections 2024-09-16T15:39:27.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.370+0000 7f6ce3200640 1 -- 172.21.15.151:0/3586744451 wait complete. 2024-09-16T15:39:27.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.371+0000 7f6ce3200640 1 Processor -- start 2024-09-16T15:39:27.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.371+0000 7f6ce3200640 1 -- start start 2024-09-16T15:39:27.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.372+0000 7f6ce3200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ce40b34b0 0x7f6ce41f8890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:27.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.372+0000 7f6ce3200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ce40b3e50 0x7f6ce41f4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:27.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.372+0000 7f6ce3200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ce4161440 0x7f6ce415fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:27.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.372+0000 7f6ce3200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce41619c0 con 0x7f6ce40b34b0 2024-09-16T15:39:27.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.372+0000 7f6ce3200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce40b2990 con 0x7f6ce4161440 2024-09-16T15:39:27.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.372+0000 7f6ce3200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f6ce40d2470 con 0x7f6ce40b3e50 2024-09-16T15:39:27.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.372+0000 7f6ce1e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ce40b34b0 0x7f6ce41f8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:27.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.373+0000 7f6ce1e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ce40b34b0 0x7f6ce41f8890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47296/0 (socket says 172.21.15.151:47296) 2024-09-16T15:39:27.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.373+0000 7f6ce1e00640 1 -- 172.21.15.151:0/418808146 learned_addr learned my addr 172.21.15.151:0/418808146 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:27.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.373+0000 7f6ce1400640 1 --2- 172.21.15.151:0/418808146 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ce40b3e50 0x7f6ce41f4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:27.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.373+0000 7f6ce2800640 1 --2- 172.21.15.151:0/418808146 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ce4161440 0x7f6ce415fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:27.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.373+0000 7f6ce1e00640 1 -- 172.21.15.151:0/418808146 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ce40b3e50 msgr2=0x7f6ce41f4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:27.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.373+0000 7f6ce1e00640 1 --2- 172.21.15.151:0/418808146 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ce40b3e50 0x7f6ce41f4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:27.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.373+0000 7f6ce1e00640 1 -- 172.21.15.151:0/418808146 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ce4161440 msgr2=0x7f6ce415fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:27.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.373+0000 7f6ce1e00640 1 --2- 172.21.15.151:0/418808146 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ce4161440 0x7f6ce415fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:27.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.373+0000 7f6ce1e00640 1 -- 172.21.15.151:0/418808146 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ce41f49a0 con 0x7f6ce40b34b0 2024-09-16T15:39:27.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.373+0000 7f6ce1e00640 1 --2- 172.21.15.151:0/418808146 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ce40b34b0 0x7f6ce41f8890 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6cd400d6d0 tx=0x7f6cd400db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:27.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.374+0000 7f6cda000640 1 -- 172.21.15.151:0/418808146 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd4017070 con 0x7f6ce40b34b0 2024-09-16T15:39:27.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.374+0000 7f6cda000640 1 -- 172.21.15.151:0/418808146 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6cd4005330 con 0x7f6ce40b34b0 2024-09-16T15:39:27.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.374+0000 7f6cda000640 1 -- 172.21.15.151:0/418808146 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd4009820 con 0x7f6ce40b34b0 2024-09-16T15:39:27.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.374+0000 7f6ce3200640 1 -- 172.21.15.151:0/418808146 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ce41f4c80 con 0x7f6ce40b34b0 2024-09-16T15:39:27.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.374+0000 7f6ce3200640 1 -- 172.21.15.151:0/418808146 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ce41f5240 con 0x7f6ce40b34b0 2024-09-16T15:39:27.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.375+0000 7f6ce3200640 1 -- 172.21.15.151:0/418808146 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ce40d10a0 con 0x7f6ce40b34b0 2024-09-16T15:39:27.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.376+0000 7f6cda000640 1 -- 172.21.15.151:0/418808146 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f6cd4009bd0 con 0x7f6ce40b34b0 2024-09-16T15:39:27.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.377+0000 7f6cda000640 1 --2- 172.21.15.151:0/418808146 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6cc4078850 0x7f6cc407ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:27.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.377+0000 7f6cda000640 1 -- 172.21.15.151:0/418808146 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f6cd409abb0 con 0x7f6ce40b34b0 2024-09-16T15:39:27.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.377+0000 7f6ce1400640 1 --2- 172.21.15.151:0/418808146 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6cc4078850 0x7f6cc407ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:27.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.378+0000 7f6ce1400640 1 --2- 172.21.15.151:0/418808146 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6cc4078850 0x7f6cc407ad10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6cd0033000 tx=0x7f6cd0004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:27.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.381+0000 7f6cda000640 1 -- 172.21.15.151:0/418808146 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6cd4062bc0 con 0x7f6ce40b34b0 2024-09-16T15:39:27.524 INFO:tasks.ceph.mds.d.smithi167.stderr:2024-09-16T15:39:27.523+0000 7f533f800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-16T15:39:27.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.564+0000 7f6ce3200640 1 -- 172.21.15.151:0/418808146 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6ce40b3910 con 0x7f6cc4078850 2024-09-16T15:39:27.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.568+0000 7f6cda000640 1 -- 172.21.15.151:0/418808146 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240885 (secure 0 0 0) 0x7f6ce40b3910 con 0x7f6cc4078850 2024-09-16T15:39:27.572 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:39:27.573 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-09-16T15:39:27.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.572+0000 7f6ce3200640 1 -- 172.21.15.151:0/418808146 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6cc4078850 msgr2=0x7f6cc407ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:27.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.572+0000 7f6ce3200640 1 --2- 172.21.15.151:0/418808146 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6cc4078850 0x7f6cc407ad10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6cd0033000 tx=0x7f6cd0004730 comp rx=0 tx=0).stop 2024-09-16T15:39:27.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.572+0000 7f6ce3200640 1 -- 172.21.15.151:0/418808146 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ce40b34b0 msgr2=0x7f6ce41f8890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:27.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.572+0000 7f6ce3200640 1 --2- 172.21.15.151:0/418808146 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ce40b34b0 0x7f6ce41f8890 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6cd400d6d0 tx=0x7f6cd400db90 comp rx=0 tx=0).stop 2024-09-16T15:39:27.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.572+0000 7f6ce3200640 1 -- 172.21.15.151:0/418808146 shutdown_connections 2024-09-16T15:39:27.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.572+0000 7f6ce3200640 1 --2- 172.21.15.151:0/418808146 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6cc4078850 0x7f6cc407ad10 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:27.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.572+0000 7f6ce3200640 1 --2- 172.21.15.151:0/418808146 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ce4161440 0x7f6ce415fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:27.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.572+0000 7f6ce3200640 1 --2- 172.21.15.151:0/418808146 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ce40b3e50 0x7f6ce41f4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:27.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.572+0000 7f6ce3200640 1 --2- 172.21.15.151:0/418808146 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ce40b34b0 0x7f6ce41f8890 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:27.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.572+0000 7f6ce3200640 1 -- 172.21.15.151:0/418808146 >> 172.21.15.151:0/418808146 conn(0x7f6ce40d3df0 msgr2=0x7f6ce40b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:27.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.572+0000 7f6ce3200640 1 -- 172.21.15.151:0/418808146 shutdown_connections 2024-09-16T15:39:27.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.572+0000 7f6ce3200640 1 -- 172.21.15.151:0/418808146 wait complete. 2024-09-16T15:39:27.590 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-16T15:39:25.577163+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590492,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":108,"num_write_kb":1636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77,"ondisk_log_size":77,"up":72,"acting":72,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":27,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219316,"kb_used_data":2572,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755404,"statfs":{"total":773094113280,"available":772869533696,"internally_reserved":0,"allocated":2633728,"data_stored":1620780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12738,"internal_metadata":221236798},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":124,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.044880"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.714434+0000","last_change":"2024-09-16T15:39:22.579035+0000","last_active":"2024-09-16T15:39:23.714434+0000","last_peered":"2024-09-16T15:39:23.714434+0000","last_clean":"2024-09-16T15:39:23.714434+0000","last_became_active":"2024-09-16T15:39:22.578734+0000","last_became_peered":"2024-09-16T15:39:22.578734+0000","last_unstale":"2024-09-16T15:39:23.714434+0000","last_undegraded":"2024-09-16T15:39:23.714434+0000","last_fullsized":"2024-09-16T15:39:23.714434+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:13:19.369864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.714354+0000","last_change":"2024-09-16T15:39:22.566051+0000","last_active":"2024-09-16T15:39:23.714354+0000","last_peered":"2024-09-16T15:39:23.714354+0000","last_clean":"2024-09-16T15:39:23.714354+0000","last_became_active":"2024-09-16T15:39:22.565764+0000","last_became_peered":"2024-09-16T15:39:22.565764+0000","last_unstale":"2024-09-16T15:39:23.714354+0000","last_undegraded":"2024-09-16T15:39:23.714354+0000","last_fullsized":"2024-09-16T15:39:23.714354+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:42:34.417720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.536611+0000","last_change":"2024-09-16T15:39:20.536611+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.536611+0000","last_undegraded":"2024-09-16T15:39:20.536611+0000","last_fullsized":"2024-09-16T15:39:20.536611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.544697+0000","last_change":"2024-09-16T15:39:20.544697+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.544697+0000","last_undegraded":"2024-09-16T15:39:20.544697+0000","last_fullsized":"2024-09-16T15:39:20.544697+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.541664+0000","last_change":"2024-09-16T15:39:20.541664+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.541664+0000","last_undegraded":"2024-09-16T15:39:20.541664+0000","last_fullsized":"2024-09-16T15:39:20.541664+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.544178+0000","last_change":"2024-09-16T15:39:20.544178+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.544178+0000","last_undegraded":"2024-09-16T15:39:20.544178+0000","last_fullsized":"2024-09-16T15:39:20.544178+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.544198+0000","last_change":"2024-09-16T15:39:20.544198+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.544198+0000","last_undegraded":"2024-09-16T15:39:20.544198+0000","last_fullsized":"2024-09-16T15:39:20.544198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.538592+0000","last_change":"2024-09-16T15:39:20.538592+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.538592+0000","last_undegraded":"2024-09-16T15:39:20.538592+0000","last_fullsized":"2024-09-16T15:39:20.538592+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.714227+0000","last_change":"2024-09-16T15:39:21.561906+0000","last_active":"2024-09-16T15:39:23.714227+0000","last_peered":"2024-09-16T15:39:23.714227+0000","last_clean":"2024-09-16T15:39:23.714227+0000","last_became_active":"2024-09-16T15:39:21.561566+0000","last_became_peered":"2024-09-16T15:39:21.561566+0000","last_unstale":"2024-09-16T15:39:23.714227+0000","last_undegraded":"2024-09-16T15:39:23.714227+0000","last_fullsized":"2024-09-16T15:39:23.714227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:12:53.853659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.535462+0000","last_change":"2024-09-16T15:39:20.535462+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.535462+0000","last_undegraded":"2024-09-16T15:39:20.535462+0000","last_fullsized":"2024-09-16T15:39:20.535462+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.714255+0000","last_change":"2024-09-16T15:39:21.567091+0000","last_active":"2024-09-16T15:39:23.714255+0000","last_peered":"2024-09-16T15:39:23.714255+0000","last_clean":"2024-09-16T15:39:23.714255+0000","last_became_active":"2024-09-16T15:39:21.566668+0000","last_became_peered":"2024-09-16T15:39:21.566668+0000","last_unstale":"2024-09-16T15:39:23.714255+0000","last_undegraded":"2024-09-16T15:39:23.714255+0000","last_fullsized":"2024-09-16T15:39:23.714255+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:08:06.242307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.544145+0000","last_change":"2024-09-16T15:39:20.544145+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.544145+0000","last_undegraded":"2024-09-16T15:39:20.544145+0000","last_fullsized":"2024-09-16T15:39:20.544145+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.537994+0000","last_change":"2024-09-16T15:39:20.537994+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.537994+0000","last_undegraded":"2024-09-16T15:39:20.537994+0000","last_fullsized":"2024-09-16T15:39:20.537994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.534947+0000","last_change":"2024-09-16T15:39:20.534947+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.534947+0000","last_undegraded":"2024-09-16T15:39:20.534947+0000","last_fullsized":"2024-09-16T15:39:20.534947+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.537954+0000","last_change":"2024-09-16T15:39:20.537954+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.537954+0000","last_undegraded":"2024-09-16T15:39:20.537954+0000","last_fullsized":"2024-09-16T15:39:20.537954+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.535736+0000","last_change":"2024-09-16T15:39:20.535736+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.535736+0000","last_undegraded":"2024-09-16T15:39:20.535736+0000","last_fullsized":"2024-09-16T15:39:20.535736+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.536294+0000","last_change":"2024-09-16T15:39:20.536294+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.536294+0000","last_undegraded":"2024-09-16T15:39:20.536294+0000","last_fullsized":"2024-09-16T15:39:20.536294+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.541319+0000","last_change":"2024-09-16T15:39:20.541319+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.541319+0000","last_undegraded":"2024-09-16T15:39:20.541319+0000","last_fullsized":"2024-09-16T15:39:20.541319+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":105,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-16T15:39:20.545567+0000","last_change":"2024-09-16T15:39:12.698813+0000","last_active":"2024-09-16T15:39:20.545567+0000","last_peered":"2024-09-16T15:39:20.545567+0000","last_clean":"2024-09-16T15:39:20.545567+0000","last_became_active":"2024-09-16T15:39:12.697695+0000","last_became_peered":"2024-09-16T15:39:12.697695+0000","last_unstale":"2024-09-16T15:39:20.545567+0000","last_undegraded":"2024-09-16T15:39:20.545567+0000","last_fullsized":"2024-09-16T15:39:20.545567+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_clean_scrub_stamp":"2024-09-16T15:39:11.684967+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:02.061081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.543255+0000","last_change":"2024-09-16T15:39:20.543255+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.543255+0000","last_undegraded":"2024-09-16T15:39:20.543255+0000","last_fullsized":"2024-09-16T15:39:20.543255+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.543264+0000","last_change":"2024-09-16T15:39:20.543264+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.543264+0000","last_undegraded":"2024-09-16T15:39:20.543264+0000","last_fullsized":"2024-09-16T15:39:20.543264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.714355+0000","last_change":"2024-09-16T15:39:21.563752+0000","last_active":"2024-09-16T15:39:23.714355+0000","last_peered":"2024-09-16T15:39:23.714355+0000","last_clean":"2024-09-16T15:39:23.714355+0000","last_became_active":"2024-09-16T15:39:21.563430+0000","last_became_peered":"2024-09-16T15:39:21.563430+0000","last_unstale":"2024-09-16T15:39:23.714355+0000","last_undegraded":"2024-09-16T15:39:23.714355+0000","last_fullsized":"2024-09-16T15:39:23.714355+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:44:53.582663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.543244+0000","last_change":"2024-09-16T15:39:20.543244+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.543244+0000","last_undegraded":"2024-09-16T15:39:20.543244+0000","last_fullsized":"2024-09-16T15:39:20.543244+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.535059+0000","last_change":"2024-09-16T15:39:20.535059+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.535059+0000","last_undegraded":"2024-09-16T15:39:20.535059+0000","last_fullsized":"2024-09-16T15:39:20.535059+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.543480+0000","last_change":"2024-09-16T15:39:20.543480+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.543480+0000","last_undegraded":"2024-09-16T15:39:20.543480+0000","last_fullsized":"2024-09-16T15:39:20.543480+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.535043+0000","last_change":"2024-09-16T15:39:20.535043+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.535043+0000","last_undegraded":"2024-09-16T15:39:20.535043+0000","last_fullsized":"2024-09-16T15:39:20.535043+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.543478+0000","last_change":"2024-09-16T15:39:20.543478+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.543478+0000","last_undegraded":"2024-09-16T15:39:20.543478+0000","last_fullsized":"2024-09-16T15:39:20.543478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.538101+0000","last_change":"2024-09-16T15:39:20.538101+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.538101+0000","last_undegraded":"2024-09-16T15:39:20.538101+0000","last_fullsized":"2024-09-16T15:39:20.538101+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.534937+0000","last_change":"2024-09-16T15:39:20.534937+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.534937+0000","last_undegraded":"2024-09-16T15:39:20.534937+0000","last_fullsized":"2024-09-16T15:39:20.534937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.541272+0000","last_change":"2024-09-16T15:39:20.541272+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.541272+0000","last_undegraded":"2024-09-16T15:39:20.541272+0000","last_fullsized":"2024-09-16T15:39:20.541272+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.714500+0000","last_change":"2024-09-16T15:39:22.576232+0000","last_active":"2024-09-16T15:39:23.714500+0000","last_peered":"2024-09-16T15:39:23.714500+0000","last_clean":"2024-09-16T15:39:23.714500+0000","last_became_active":"2024-09-16T15:39:22.575962+0000","last_became_peered":"2024-09-16T15:39:22.575962+0000","last_unstale":"2024-09-16T15:39:23.714500+0000","last_undegraded":"2024-09-16T15:39:23.714500+0000","last_fullsized":"2024-09-16T15:39:23.714500+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:43:58.162669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.544248+0000","last_change":"2024-09-16T15:39:20.544248+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.544248+0000","last_undegraded":"2024-09-16T15:39:20.544248+0000","last_fullsized":"2024-09-16T15:39:20.544248+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.537887+0000","last_change":"2024-09-16T15:39:20.537887+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.537887+0000","last_undegraded":"2024-09-16T15:39:20.537887+0000","last_fullsized":"2024-09-16T15:39:20.537887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.538384+0000","last_change":"2024-09-16T15:39:20.538384+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.538384+0000","last_undegraded":"2024-09-16T15:39:20.538384+0000","last_fullsized":"2024-09-16T15:39:20.538384+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.534421+0000","last_change":"2024-09-16T15:39:20.534421+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.534421+0000","last_undegraded":"2024-09-16T15:39:20.534421+0000","last_fullsized":"2024-09-16T15:39:20.534421+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:39:21.531232+0000","last_change":"2024-09-16T15:39:21.531232+0000","last_active":"2024-09-16T15:39:21.531232+0000","last_peered":"2024-09-16T15:39:21.531232+0000","last_clean":"2024-09-16T15:39:21.531232+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:21.531232+0000","last_undegraded":"2024-09-16T15:39:21.531232+0000","last_fullsized":"2024-09-16T15:39:21.531232+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T15:39:20.535936+0000","last_change":"2024-09-16T15:39:20.535936+0000","last_active":"2024-09-16T15:39:20.523277+0000","last_peered":"2024-09-16T15:39:20.523277+0000","last_clean":"2024-09-16T15:39:20.523277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:39:20.535936+0000","last_undegraded":"2024-09-16T15:39:20.535936+0000","last_fullsized":"2024-09-16T15:39:20.535936+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":6,"acting":6,"num_store_stats":1},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":124,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":172032,"data_stored":51117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28024,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343816,"statfs":{"total":96636764160,"available":96608067584,"internally_reserved":0,"allocated":765952,"data_stored":641485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":172032,"data_stored":51117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509444,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":172032,"data_stored":51117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":56934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28024,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343816,"statfs":{"total":96636764160,"available":96608067584,"internally_reserved":0,"allocated":765952,"data_stored":641485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":172032,"data_stored":51117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542151,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":76408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":27654604},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:39:27.595 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3f is not active+clean 2024-09-16T15:39:27.595 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:39:21.531232+0000', 'last_change': '2024-09-16T15:39:21.531232+0000', 'last_active': '2024-09-16T15:39:21.531232+0000', 'last_peered': '2024-09-16T15:39:21.531232+0000', 'last_clean': '2024-09-16T15:39:21.531232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:21.531232+0000', 'last_undegraded': '2024-09-16T15:39:21.531232+0000', 'last_fullsized': '2024-09-16T15:39:21.531232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:39:27.595 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3e is not active+clean 2024-09-16T15:39:27.596 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:39:21.531232+0000', 'last_change': '2024-09-16T15:39:21.531232+0000', 'last_active': '2024-09-16T15:39:21.531232+0000', 'last_peered': '2024-09-16T15:39:21.531232+0000', 'last_clean': '2024-09-16T15:39:21.531232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:21.531232+0000', 'last_undegraded': '2024-09-16T15:39:21.531232+0000', 'last_fullsized': '2024-09-16T15:39:21.531232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:39:27.596 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3d is not active+clean 2024-09-16T15:39:27.596 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:39:21.531232+0000', 'last_change': '2024-09-16T15:39:21.531232+0000', 'last_active': '2024-09-16T15:39:21.531232+0000', 'last_peered': '2024-09-16T15:39:21.531232+0000', 'last_clean': '2024-09-16T15:39:21.531232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:21.531232+0000', 'last_undegraded': '2024-09-16T15:39:21.531232+0000', 'last_fullsized': '2024-09-16T15:39:21.531232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:39:27.596 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3c is not active+clean 2024-09-16T15:39:27.596 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:39:21.531232+0000', 'last_change': '2024-09-16T15:39:21.531232+0000', 'last_active': '2024-09-16T15:39:21.531232+0000', 'last_peered': '2024-09-16T15:39:21.531232+0000', 'last_clean': '2024-09-16T15:39:21.531232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:21.531232+0000', 'last_undegraded': '2024-09-16T15:39:21.531232+0000', 'last_fullsized': '2024-09-16T15:39:21.531232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:39:27.596 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3b is not active+clean 2024-09-16T15:39:27.596 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:39:21.531232+0000', 'last_change': '2024-09-16T15:39:21.531232+0000', 'last_active': '2024-09-16T15:39:21.531232+0000', 'last_peered': '2024-09-16T15:39:21.531232+0000', 'last_clean': '2024-09-16T15:39:21.531232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:21.531232+0000', 'last_undegraded': '2024-09-16T15:39:21.531232+0000', 'last_fullsized': '2024-09-16T15:39:21.531232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:39:27.596 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3a is not active+clean 2024-09-16T15:39:27.596 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:39:21.531232+0000', 'last_change': '2024-09-16T15:39:21.531232+0000', 'last_active': '2024-09-16T15:39:21.531232+0000', 'last_peered': '2024-09-16T15:39:21.531232+0000', 'last_clean': '2024-09-16T15:39:21.531232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:21.531232+0000', 'last_undegraded': '2024-09-16T15:39:21.531232+0000', 'last_fullsized': '2024-09-16T15:39:21.531232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:39:27.596 INFO:tasks.ceph.ceph_manager.ceph:PG 3.39 is not active+clean 2024-09-16T15:39:27.596 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:39:21.531232+0000', 'last_change': '2024-09-16T15:39:21.531232+0000', 'last_active': '2024-09-16T15:39:21.531232+0000', 'last_peered': '2024-09-16T15:39:21.531232+0000', 'last_clean': '2024-09-16T15:39:21.531232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:21.531232+0000', 'last_undegraded': '2024-09-16T15:39:21.531232+0000', 'last_fullsized': '2024-09-16T15:39:21.531232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:39:27.596 INFO:tasks.ceph.ceph_manager.ceph:PG 3.38 is not active+clean 2024-09-16T15:39:27.597 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:39:21.531232+0000', 'last_change': '2024-09-16T15:39:21.531232+0000', 'last_active': '2024-09-16T15:39:21.531232+0000', 'last_peered': '2024-09-16T15:39:21.531232+0000', 'last_clean': '2024-09-16T15:39:21.531232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:21.531232+0000', 'last_undegraded': '2024-09-16T15:39:21.531232+0000', 'last_fullsized': '2024-09-16T15:39:21.531232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:39:27.597 INFO:tasks.ceph.ceph_manager.ceph:PG 3.37 is not active+clean 2024-09-16T15:39:27.597 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:39:21.531232+0000', 'last_change': '2024-09-16T15:39:21.531232+0000', 'last_active': '2024-09-16T15:39:21.531232+0000', 'last_peered': '2024-09-16T15:39:21.531232+0000', 'last_clean': '2024-09-16T15:39:21.531232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:21.531232+0000', 'last_undegraded': '2024-09-16T15:39:21.531232+0000', 'last_fullsized': '2024-09-16T15:39:21.531232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:39:27.597 INFO:tasks.ceph.ceph_manager.ceph:PG 3.36 is not active+clean 2024-09-16T15:39:27.597 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:39:21.531232+0000', 'last_change': '2024-09-16T15:39:21.531232+0000', 'last_active': '2024-09-16T15:39:21.531232+0000', 'last_peered': '2024-09-16T15:39:21.531232+0000', 'last_clean': '2024-09-16T15:39:21.531232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:21.531232+0000', 'last_undegraded': '2024-09-16T15:39:21.531232+0000', 'last_fullsized': '2024-09-16T15:39:21.531232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:39:27.597 INFO:tasks.ceph.ceph_manager.ceph:PG 3.35 is not active+clean 2024-09-16T15:39:27.597 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:39:21.531232+0000', 'last_change': '2024-09-16T15:39:21.531232+0000', 'last_active': '2024-09-16T15:39:21.531232+0000', 'last_peered': '2024-09-16T15:39:21.531232+0000', 'last_clean': '2024-09-16T15:39:21.531232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:21.531232+0000', 'last_undegraded': '2024-09-16T15:39:21.531232+0000', 'last_fullsized': '2024-09-16T15:39:21.531232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:39:27.597 INFO:tasks.ceph.ceph_manager.ceph:PG 3.34 is not active+clean 2024-09-16T15:39:27.597 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:39:21.531232+0000', 'last_change': '2024-09-16T15:39:21.531232+0000', 'last_active': '2024-09-16T15:39:21.531232+0000', 'last_peered': '2024-09-16T15:39:21.531232+0000', 'last_clean': '2024-09-16T15:39:21.531232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:21.531232+0000', 'last_undegraded': '2024-09-16T15:39:21.531232+0000', 'last_fullsized': '2024-09-16T15:39:21.531232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:39:27.597 INFO:tasks.ceph.ceph_manager.ceph:PG 3.33 is not active+clean 2024-09-16T15:39:27.597 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:39:21.531232+0000', 'last_change': '2024-09-16T15:39:21.531232+0000', 'last_active': '2024-09-16T15:39:21.531232+0000', 'last_peered': '2024-09-16T15:39:21.531232+0000', 'last_clean': '2024-09-16T15:39:21.531232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:21.531232+0000', 'last_undegraded': '2024-09-16T15:39:21.531232+0000', 'last_fullsized': '2024-09-16T15:39:21.531232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:39:27.597 INFO:tasks.ceph.ceph_manager.ceph:PG 3.32 is not active+clean 2024-09-16T15:39:27.598 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:39:21.531232+0000', 'last_change': '2024-09-16T15:39:21.531232+0000', 'last_active': '2024-09-16T15:39:21.531232+0000', 'last_peered': '2024-09-16T15:39:21.531232+0000', 'last_clean': '2024-09-16T15:39:21.531232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:21.531232+0000', 'last_undegraded': '2024-09-16T15:39:21.531232+0000', 'last_fullsized': '2024-09-16T15:39:21.531232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:39:27.598 INFO:tasks.ceph.ceph_manager.ceph:PG 3.31 is not active+clean 2024-09-16T15:39:27.598 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:39:21.531232+0000', 'last_change': '2024-09-16T15:39:21.531232+0000', 'last_active': '2024-09-16T15:39:21.531232+0000', 'last_peered': '2024-09-16T15:39:21.531232+0000', 'last_clean': '2024-09-16T15:39:21.531232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:21.531232+0000', 'last_undegraded': '2024-09-16T15:39:21.531232+0000', 'last_fullsized': '2024-09-16T15:39:21.531232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:39:27.598 INFO:tasks.ceph.ceph_manager.ceph:PG 3.30 is not active+clean 2024-09-16T15:39:27.598 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:39:21.531232+0000', 'last_change': '2024-09-16T15:39:21.531232+0000', 'last_active': '2024-09-16T15:39:21.531232+0000', 'last_peered': '2024-09-16T15:39:21.531232+0000', 'last_clean': '2024-09-16T15:39:21.531232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:21.531232+0000', 'last_undegraded': '2024-09-16T15:39:21.531232+0000', 'last_fullsized': '2024-09-16T15:39:21.531232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:39:27.598 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2f is not active+clean 2024-09-16T15:39:27.598 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:39:21.531232+0000', 'last_change': '2024-09-16T15:39:21.531232+0000', 'last_active': '2024-09-16T15:39:21.531232+0000', 'last_peered': '2024-09-16T15:39:21.531232+0000', 'last_clean': '2024-09-16T15:39:21.531232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:21.531232+0000', 'last_undegraded': '2024-09-16T15:39:21.531232+0000', 'last_fullsized': '2024-09-16T15:39:21.531232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:39:27.598 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2d is not active+clean 2024-09-16T15:39:27.598 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:39:21.531232+0000', 'last_change': '2024-09-16T15:39:21.531232+0000', 'last_active': '2024-09-16T15:39:21.531232+0000', 'last_peered': '2024-09-16T15:39:21.531232+0000', 'last_clean': '2024-09-16T15:39:21.531232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:21.531232+0000', 'last_undegraded': '2024-09-16T15:39:21.531232+0000', 'last_fullsized': '2024-09-16T15:39:21.531232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:39:27.598 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2c is not active+clean 2024-09-16T15:39:27.598 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:39:21.531232+0000', 'last_change': '2024-09-16T15:39:21.531232+0000', 'last_active': '2024-09-16T15:39:21.531232+0000', 'last_peered': '2024-09-16T15:39:21.531232+0000', 'last_clean': '2024-09-16T15:39:21.531232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:21.531232+0000', 'last_undegraded': '2024-09-16T15:39:21.531232+0000', 'last_fullsized': '2024-09-16T15:39:21.531232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:39:27.598 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2b is not active+clean 2024-09-16T15:39:27.599 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:39:21.531232+0000', 'last_change': '2024-09-16T15:39:21.531232+0000', 'last_active': '2024-09-16T15:39:21.531232+0000', 'last_peered': '2024-09-16T15:39:21.531232+0000', 'last_clean': '2024-09-16T15:39:21.531232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:21.531232+0000', 'last_undegraded': '2024-09-16T15:39:21.531232+0000', 'last_fullsized': '2024-09-16T15:39:21.531232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:39:27.599 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2a is not active+clean 2024-09-16T15:39:27.599 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:39:21.531232+0000', 'last_change': '2024-09-16T15:39:21.531232+0000', 'last_active': '2024-09-16T15:39:21.531232+0000', 'last_peered': '2024-09-16T15:39:21.531232+0000', 'last_clean': '2024-09-16T15:39:21.531232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:21.531232+0000', 'last_undegraded': '2024-09-16T15:39:21.531232+0000', 'last_fullsized': '2024-09-16T15:39:21.531232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:39:27.599 INFO:tasks.ceph.ceph_manager.ceph:PG 3.29 is not active+clean 2024-09-16T15:39:27.599 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:39:21.531232+0000', 'last_change': '2024-09-16T15:39:21.531232+0000', 'last_active': '2024-09-16T15:39:21.531232+0000', 'last_peered': '2024-09-16T15:39:21.531232+0000', 'last_clean': '2024-09-16T15:39:21.531232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:21.531232+0000', 'last_undegraded': '2024-09-16T15:39:21.531232+0000', 'last_fullsized': '2024-09-16T15:39:21.531232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:39:27.599 INFO:tasks.ceph.ceph_manager.ceph:PG 3.28 is not active+clean 2024-09-16T15:39:27.599 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:39:21.531232+0000', 'last_change': '2024-09-16T15:39:21.531232+0000', 'last_active': '2024-09-16T15:39:21.531232+0000', 'last_peered': '2024-09-16T15:39:21.531232+0000', 'last_clean': '2024-09-16T15:39:21.531232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:21.531232+0000', 'last_undegraded': '2024-09-16T15:39:21.531232+0000', 'last_fullsized': '2024-09-16T15:39:21.531232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:39:27.599 INFO:tasks.ceph.ceph_manager.ceph:PG 3.27 is not active+clean 2024-09-16T15:39:27.599 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:39:21.531232+0000', 'last_change': '2024-09-16T15:39:21.531232+0000', 'last_active': '2024-09-16T15:39:21.531232+0000', 'last_peered': '2024-09-16T15:39:21.531232+0000', 'last_clean': '2024-09-16T15:39:21.531232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:21.531232+0000', 'last_undegraded': '2024-09-16T15:39:21.531232+0000', 'last_fullsized': '2024-09-16T15:39:21.531232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:39:27.599 INFO:tasks.ceph.ceph_manager.ceph:PG 3.26 is not active+clean 2024-09-16T15:39:27.599 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:39:21.531232+0000', 'last_change': '2024-09-16T15:39:21.531232+0000', 'last_active': '2024-09-16T15:39:21.531232+0000', 'last_peered': '2024-09-16T15:39:21.531232+0000', 'last_clean': '2024-09-16T15:39:21.531232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:21.531232+0000', 'last_undegraded': '2024-09-16T15:39:21.531232+0000', 'last_fullsized': '2024-09-16T15:39:21.531232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:39:27.599 INFO:tasks.ceph.ceph_manager.ceph:PG 3.25 is not active+clean 2024-09-16T15:39:27.600 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:39:21.531232+0000', 'last_change': '2024-09-16T15:39:21.531232+0000', 'last_active': '2024-09-16T15:39:21.531232+0000', 'last_peered': '2024-09-16T15:39:21.531232+0000', 'last_clean': '2024-09-16T15:39:21.531232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:21.531232+0000', 'last_undegraded': '2024-09-16T15:39:21.531232+0000', 'last_fullsized': '2024-09-16T15:39:21.531232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:39:27.600 INFO:tasks.ceph.ceph_manager.ceph:PG 3.24 is not active+clean 2024-09-16T15:39:27.600 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:39:21.531232+0000', 'last_change': '2024-09-16T15:39:21.531232+0000', 'last_active': '2024-09-16T15:39:21.531232+0000', 'last_peered': '2024-09-16T15:39:21.531232+0000', 'last_clean': '2024-09-16T15:39:21.531232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:21.531232+0000', 'last_undegraded': '2024-09-16T15:39:21.531232+0000', 'last_fullsized': '2024-09-16T15:39:21.531232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:39:27.600 INFO:tasks.ceph.ceph_manager.ceph:PG 3.23 is not active+clean 2024-09-16T15:39:27.600 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:39:21.531232+0000', 'last_change': '2024-09-16T15:39:21.531232+0000', 'last_active': '2024-09-16T15:39:21.531232+0000', 'last_peered': '2024-09-16T15:39:21.531232+0000', 'last_clean': '2024-09-16T15:39:21.531232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:21.531232+0000', 'last_undegraded': '2024-09-16T15:39:21.531232+0000', 'last_fullsized': '2024-09-16T15:39:21.531232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:39:27.600 INFO:tasks.ceph.ceph_manager.ceph:PG 3.22 is not active+clean 2024-09-16T15:39:27.600 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:39:21.531232+0000', 'last_change': '2024-09-16T15:39:21.531232+0000', 'last_active': '2024-09-16T15:39:21.531232+0000', 'last_peered': '2024-09-16T15:39:21.531232+0000', 'last_clean': '2024-09-16T15:39:21.531232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:21.531232+0000', 'last_undegraded': '2024-09-16T15:39:21.531232+0000', 'last_fullsized': '2024-09-16T15:39:21.531232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:39:27.600 INFO:tasks.ceph.ceph_manager.ceph:PG 3.20 is not active+clean 2024-09-16T15:39:27.600 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:39:21.531232+0000', 'last_change': '2024-09-16T15:39:21.531232+0000', 'last_active': '2024-09-16T15:39:21.531232+0000', 'last_peered': '2024-09-16T15:39:21.531232+0000', 'last_clean': '2024-09-16T15:39:21.531232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:21.531232+0000', 'last_undegraded': '2024-09-16T15:39:21.531232+0000', 'last_fullsized': '2024-09-16T15:39:21.531232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:39:27.600 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1a is not active+clean 2024-09-16T15:39:27.600 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:39:21.531232+0000', 'last_change': '2024-09-16T15:39:21.531232+0000', 'last_active': '2024-09-16T15:39:21.531232+0000', 'last_peered': '2024-09-16T15:39:21.531232+0000', 'last_clean': '2024-09-16T15:39:21.531232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:21.531232+0000', 'last_undegraded': '2024-09-16T15:39:21.531232+0000', 'last_fullsized': '2024-09-16T15:39:21.531232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:39:27.600 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1b is not active+clean 2024-09-16T15:39:27.601 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:20.536611+0000', 'last_change': '2024-09-16T15:39:20.536611+0000', 'last_active': '2024-09-16T15:39:20.523277+0000', 'last_peered': '2024-09-16T15:39:20.523277+0000', 'last_clean': '2024-09-16T15:39:20.523277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:20.536611+0000', 'last_undegraded': '2024-09-16T15:39:20.536611+0000', 'last_fullsized': '2024-09-16T15:39:20.536611+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-16T15:39:27.601 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1b is not active+clean 2024-09-16T15:39:27.601 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:39:21.531232+0000', 'last_change': '2024-09-16T15:39:21.531232+0000', 'last_active': '2024-09-16T15:39:21.531232+0000', 'last_peered': '2024-09-16T15:39:21.531232+0000', 'last_clean': '2024-09-16T15:39:21.531232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:21.531232+0000', 'last_undegraded': '2024-09-16T15:39:21.531232+0000', 'last_fullsized': '2024-09-16T15:39:21.531232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:39:27.601 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1a is not active+clean 2024-09-16T15:39:27.601 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:20.544697+0000', 'last_change': '2024-09-16T15:39:20.544697+0000', 'last_active': '2024-09-16T15:39:20.523277+0000', 'last_peered': '2024-09-16T15:39:20.523277+0000', 'last_clean': '2024-09-16T15:39:20.523277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:20.544697+0000', 'last_undegraded': '2024-09-16T15:39:20.544697+0000', 'last_fullsized': '2024-09-16T15:39:20.544697+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T15:39:27.601 INFO:tasks.ceph.ceph_manager.ceph:PG 2.19 is not active+clean 2024-09-16T15:39:27.601 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:20.541664+0000', 'last_change': '2024-09-16T15:39:20.541664+0000', 'last_active': '2024-09-16T15:39:20.523277+0000', 'last_peered': '2024-09-16T15:39:20.523277+0000', 'last_clean': '2024-09-16T15:39:20.523277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:20.541664+0000', 'last_undegraded': '2024-09-16T15:39:20.541664+0000', 'last_fullsized': '2024-09-16T15:39:20.541664+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-16T15:39:27.601 INFO:tasks.ceph.ceph_manager.ceph:PG 3.18 is not active+clean 2024-09-16T15:39:27.601 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:39:21.531232+0000', 'last_change': '2024-09-16T15:39:21.531232+0000', 'last_active': '2024-09-16T15:39:21.531232+0000', 'last_peered': '2024-09-16T15:39:21.531232+0000', 'last_clean': '2024-09-16T15:39:21.531232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:21.531232+0000', 'last_undegraded': '2024-09-16T15:39:21.531232+0000', 'last_fullsized': '2024-09-16T15:39:21.531232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:39:27.601 INFO:tasks.ceph.ceph_manager.ceph:PG 2.18 is not active+clean 2024-09-16T15:39:27.601 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:20.544178+0000', 'last_change': '2024-09-16T15:39:20.544178+0000', 'last_active': '2024-09-16T15:39:20.523277+0000', 'last_peered': '2024-09-16T15:39:20.523277+0000', 'last_clean': '2024-09-16T15:39:20.523277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:20.544178+0000', 'last_undegraded': '2024-09-16T15:39:20.544178+0000', 'last_fullsized': '2024-09-16T15:39:20.544178+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T15:39:27.602 INFO:tasks.ceph.ceph_manager.ceph:PG 3.19 is not active+clean 2024-09-16T15:39:27.602 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:39:21.531232+0000', 'last_change': '2024-09-16T15:39:21.531232+0000', 'last_active': '2024-09-16T15:39:21.531232+0000', 'last_peered': '2024-09-16T15:39:21.531232+0000', 'last_clean': '2024-09-16T15:39:21.531232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:21.531232+0000', 'last_undegraded': '2024-09-16T15:39:21.531232+0000', 'last_fullsized': '2024-09-16T15:39:21.531232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:39:27.602 INFO:tasks.ceph.ceph_manager.ceph:PG 2.17 is not active+clean 2024-09-16T15:39:27.602 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:20.544198+0000', 'last_change': '2024-09-16T15:39:20.544198+0000', 'last_active': '2024-09-16T15:39:20.523277+0000', 'last_peered': '2024-09-16T15:39:20.523277+0000', 'last_clean': '2024-09-16T15:39:20.523277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:20.544198+0000', 'last_undegraded': '2024-09-16T15:39:20.544198+0000', 'last_fullsized': '2024-09-16T15:39:20.544198+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-16T15:39:27.602 INFO:tasks.ceph.ceph_manager.ceph:PG 3.16 is not active+clean 2024-09-16T15:39:27.602 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:39:21.531232+0000', 'last_change': '2024-09-16T15:39:21.531232+0000', 'last_active': '2024-09-16T15:39:21.531232+0000', 'last_peered': '2024-09-16T15:39:21.531232+0000', 'last_clean': '2024-09-16T15:39:21.531232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:21.531232+0000', 'last_undegraded': '2024-09-16T15:39:21.531232+0000', 'last_fullsized': '2024-09-16T15:39:21.531232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:39:27.602 INFO:tasks.ceph.ceph_manager.ceph:PG 2.16 is not active+clean 2024-09-16T15:39:27.602 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:20.538592+0000', 'last_change': '2024-09-16T15:39:20.538592+0000', 'last_active': '2024-09-16T15:39:20.523277+0000', 'last_peered': '2024-09-16T15:39:20.523277+0000', 'last_clean': '2024-09-16T15:39:20.523277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:20.538592+0000', 'last_undegraded': '2024-09-16T15:39:20.538592+0000', 'last_fullsized': '2024-09-16T15:39:20.538592+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T15:39:27.602 INFO:tasks.ceph.ceph_manager.ceph:PG 3.17 is not active+clean 2024-09-16T15:39:27.602 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:39:21.531232+0000', 'last_change': '2024-09-16T15:39:21.531232+0000', 'last_active': '2024-09-16T15:39:21.531232+0000', 'last_peered': '2024-09-16T15:39:21.531232+0000', 'last_clean': '2024-09-16T15:39:21.531232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:21.531232+0000', 'last_undegraded': '2024-09-16T15:39:21.531232+0000', 'last_fullsized': '2024-09-16T15:39:21.531232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:39:27.602 INFO:tasks.ceph.ceph_manager.ceph:PG 3.14 is not active+clean 2024-09-16T15:39:27.602 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:39:21.531232+0000', 'last_change': '2024-09-16T15:39:21.531232+0000', 'last_active': '2024-09-16T15:39:21.531232+0000', 'last_peered': '2024-09-16T15:39:21.531232+0000', 'last_clean': '2024-09-16T15:39:21.531232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:21.531232+0000', 'last_undegraded': '2024-09-16T15:39:21.531232+0000', 'last_fullsized': '2024-09-16T15:39:21.531232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:39:27.602 INFO:tasks.ceph.ceph_manager.ceph:PG 2.14 is not active+clean 2024-09-16T15:39:27.603 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:20.535462+0000', 'last_change': '2024-09-16T15:39:20.535462+0000', 'last_active': '2024-09-16T15:39:20.523277+0000', 'last_peered': '2024-09-16T15:39:20.523277+0000', 'last_clean': '2024-09-16T15:39:20.523277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:20.535462+0000', 'last_undegraded': '2024-09-16T15:39:20.535462+0000', 'last_fullsized': '2024-09-16T15:39:20.535462+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T15:39:27.603 INFO:tasks.ceph.ceph_manager.ceph:PG 3.15 is not active+clean 2024-09-16T15:39:27.603 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:39:21.531232+0000', 'last_change': '2024-09-16T15:39:21.531232+0000', 'last_active': '2024-09-16T15:39:21.531232+0000', 'last_peered': '2024-09-16T15:39:21.531232+0000', 'last_clean': '2024-09-16T15:39:21.531232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:21.531232+0000', 'last_undegraded': '2024-09-16T15:39:21.531232+0000', 'last_fullsized': '2024-09-16T15:39:21.531232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:39:27.603 INFO:tasks.ceph.ceph_manager.ceph:PG 3.12 is not active+clean 2024-09-16T15:39:27.603 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:39:21.531232+0000', 'last_change': '2024-09-16T15:39:21.531232+0000', 'last_active': '2024-09-16T15:39:21.531232+0000', 'last_peered': '2024-09-16T15:39:21.531232+0000', 'last_clean': '2024-09-16T15:39:21.531232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:21.531232+0000', 'last_undegraded': '2024-09-16T15:39:21.531232+0000', 'last_fullsized': '2024-09-16T15:39:21.531232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:39:27.603 INFO:tasks.ceph.ceph_manager.ceph:PG 2.12 is not active+clean 2024-09-16T15:39:27.603 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:20.544145+0000', 'last_change': '2024-09-16T15:39:20.544145+0000', 'last_active': '2024-09-16T15:39:20.523277+0000', 'last_peered': '2024-09-16T15:39:20.523277+0000', 'last_clean': '2024-09-16T15:39:20.523277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:20.544145+0000', 'last_undegraded': '2024-09-16T15:39:20.544145+0000', 'last_fullsized': '2024-09-16T15:39:20.544145+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T15:39:27.603 INFO:tasks.ceph.ceph_manager.ceph:PG 3.13 is not active+clean 2024-09-16T15:39:27.603 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:39:21.531232+0000', 'last_change': '2024-09-16T15:39:21.531232+0000', 'last_active': '2024-09-16T15:39:21.531232+0000', 'last_peered': '2024-09-16T15:39:21.531232+0000', 'last_clean': '2024-09-16T15:39:21.531232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:21.531232+0000', 'last_undegraded': '2024-09-16T15:39:21.531232+0000', 'last_fullsized': '2024-09-16T15:39:21.531232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:39:27.603 INFO:tasks.ceph.ceph_manager.ceph:PG 2.11 is not active+clean 2024-09-16T15:39:27.603 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:20.537994+0000', 'last_change': '2024-09-16T15:39:20.537994+0000', 'last_active': '2024-09-16T15:39:20.523277+0000', 'last_peered': '2024-09-16T15:39:20.523277+0000', 'last_clean': '2024-09-16T15:39:20.523277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:20.537994+0000', 'last_undegraded': '2024-09-16T15:39:20.537994+0000', 'last_fullsized': '2024-09-16T15:39:20.537994+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T15:39:27.603 INFO:tasks.ceph.ceph_manager.ceph:PG 3.10 is not active+clean 2024-09-16T15:39:27.603 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:39:21.531232+0000', 'last_change': '2024-09-16T15:39:21.531232+0000', 'last_active': '2024-09-16T15:39:21.531232+0000', 'last_peered': '2024-09-16T15:39:21.531232+0000', 'last_clean': '2024-09-16T15:39:21.531232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:21.531232+0000', 'last_undegraded': '2024-09-16T15:39:21.531232+0000', 'last_fullsized': '2024-09-16T15:39:21.531232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:39:27.604 INFO:tasks.ceph.ceph_manager.ceph:PG 2.10 is not active+clean 2024-09-16T15:39:27.604 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:20.534947+0000', 'last_change': '2024-09-16T15:39:20.534947+0000', 'last_active': '2024-09-16T15:39:20.523277+0000', 'last_peered': '2024-09-16T15:39:20.523277+0000', 'last_clean': '2024-09-16T15:39:20.523277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:20.534947+0000', 'last_undegraded': '2024-09-16T15:39:20.534947+0000', 'last_fullsized': '2024-09-16T15:39:20.534947+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-16T15:39:27.604 INFO:tasks.ceph.ceph_manager.ceph:PG 3.11 is not active+clean 2024-09-16T15:39:27.604 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:39:21.531232+0000', 'last_change': '2024-09-16T15:39:21.531232+0000', 'last_active': '2024-09-16T15:39:21.531232+0000', 'last_peered': '2024-09-16T15:39:21.531232+0000', 'last_clean': '2024-09-16T15:39:21.531232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:21.531232+0000', 'last_undegraded': '2024-09-16T15:39:21.531232+0000', 'last_fullsized': '2024-09-16T15:39:21.531232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:39:27.604 INFO:tasks.ceph.ceph_manager.ceph:PG 2.f is not active+clean 2024-09-16T15:39:27.604 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:20.537954+0000', 'last_change': '2024-09-16T15:39:20.537954+0000', 'last_active': '2024-09-16T15:39:20.523277+0000', 'last_peered': '2024-09-16T15:39:20.523277+0000', 'last_clean': '2024-09-16T15:39:20.523277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:20.537954+0000', 'last_undegraded': '2024-09-16T15:39:20.537954+0000', 'last_fullsized': '2024-09-16T15:39:20.537954+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T15:39:27.604 INFO:tasks.ceph.ceph_manager.ceph:PG 3.e is not active+clean 2024-09-16T15:39:27.604 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:39:21.531232+0000', 'last_change': '2024-09-16T15:39:21.531232+0000', 'last_active': '2024-09-16T15:39:21.531232+0000', 'last_peered': '2024-09-16T15:39:21.531232+0000', 'last_clean': '2024-09-16T15:39:21.531232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:21.531232+0000', 'last_undegraded': '2024-09-16T15:39:21.531232+0000', 'last_fullsized': '2024-09-16T15:39:21.531232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:39:27.604 INFO:tasks.ceph.ceph_manager.ceph:PG 2.e is not active+clean 2024-09-16T15:39:27.604 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:20.535736+0000', 'last_change': '2024-09-16T15:39:20.535736+0000', 'last_active': '2024-09-16T15:39:20.523277+0000', 'last_peered': '2024-09-16T15:39:20.523277+0000', 'last_clean': '2024-09-16T15:39:20.523277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:20.535736+0000', 'last_undegraded': '2024-09-16T15:39:20.535736+0000', 'last_fullsized': '2024-09-16T15:39:20.535736+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-16T15:39:27.604 INFO:tasks.ceph.ceph_manager.ceph:PG 3.f is not active+clean 2024-09-16T15:39:27.604 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:39:21.531232+0000', 'last_change': '2024-09-16T15:39:21.531232+0000', 'last_active': '2024-09-16T15:39:21.531232+0000', 'last_peered': '2024-09-16T15:39:21.531232+0000', 'last_clean': '2024-09-16T15:39:21.531232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:21.531232+0000', 'last_undegraded': '2024-09-16T15:39:21.531232+0000', 'last_fullsized': '2024-09-16T15:39:21.531232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:39:27.605 INFO:tasks.ceph.ceph_manager.ceph:PG 2.d is not active+clean 2024-09-16T15:39:27.605 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:20.536294+0000', 'last_change': '2024-09-16T15:39:20.536294+0000', 'last_active': '2024-09-16T15:39:20.523277+0000', 'last_peered': '2024-09-16T15:39:20.523277+0000', 'last_clean': '2024-09-16T15:39:20.523277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:20.536294+0000', 'last_undegraded': '2024-09-16T15:39:20.536294+0000', 'last_fullsized': '2024-09-16T15:39:20.536294+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-16T15:39:27.605 INFO:tasks.ceph.ceph_manager.ceph:PG 3.c is not active+clean 2024-09-16T15:39:27.605 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:39:21.531232+0000', 'last_change': '2024-09-16T15:39:21.531232+0000', 'last_active': '2024-09-16T15:39:21.531232+0000', 'last_peered': '2024-09-16T15:39:21.531232+0000', 'last_clean': '2024-09-16T15:39:21.531232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:21.531232+0000', 'last_undegraded': '2024-09-16T15:39:21.531232+0000', 'last_fullsized': '2024-09-16T15:39:21.531232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:39:27.605 INFO:tasks.ceph.ceph_manager.ceph:PG 2.c is not active+clean 2024-09-16T15:39:27.605 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:20.541319+0000', 'last_change': '2024-09-16T15:39:20.541319+0000', 'last_active': '2024-09-16T15:39:20.523277+0000', 'last_peered': '2024-09-16T15:39:20.523277+0000', 'last_clean': '2024-09-16T15:39:20.523277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:20.541319+0000', 'last_undegraded': '2024-09-16T15:39:20.541319+0000', 'last_fullsized': '2024-09-16T15:39:20.541319+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-16T15:39:27.605 INFO:tasks.ceph.ceph_manager.ceph:PG 3.d is not active+clean 2024-09-16T15:39:27.605 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:39:21.531232+0000', 'last_change': '2024-09-16T15:39:21.531232+0000', 'last_active': '2024-09-16T15:39:21.531232+0000', 'last_peered': '2024-09-16T15:39:21.531232+0000', 'last_clean': '2024-09-16T15:39:21.531232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:21.531232+0000', 'last_undegraded': '2024-09-16T15:39:21.531232+0000', 'last_fullsized': '2024-09-16T15:39:21.531232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:39:27.605 INFO:tasks.ceph.ceph_manager.ceph:PG 2.3 is not active+clean 2024-09-16T15:39:27.605 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:20.543255+0000', 'last_change': '2024-09-16T15:39:20.543255+0000', 'last_active': '2024-09-16T15:39:20.523277+0000', 'last_peered': '2024-09-16T15:39:20.523277+0000', 'last_clean': '2024-09-16T15:39:20.523277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:20.543255+0000', 'last_undegraded': '2024-09-16T15:39:20.543255+0000', 'last_fullsized': '2024-09-16T15:39:20.543255+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T15:39:27.605 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2 is not active+clean 2024-09-16T15:39:27.605 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:39:21.531232+0000', 'last_change': '2024-09-16T15:39:21.531232+0000', 'last_active': '2024-09-16T15:39:21.531232+0000', 'last_peered': '2024-09-16T15:39:21.531232+0000', 'last_clean': '2024-09-16T15:39:21.531232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:21.531232+0000', 'last_undegraded': '2024-09-16T15:39:21.531232+0000', 'last_fullsized': '2024-09-16T15:39:21.531232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:39:27.606 INFO:tasks.ceph.ceph_manager.ceph:PG 2.0 is not active+clean 2024-09-16T15:39:27.606 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:20.543264+0000', 'last_change': '2024-09-16T15:39:20.543264+0000', 'last_active': '2024-09-16T15:39:20.523277+0000', 'last_peered': '2024-09-16T15:39:20.523277+0000', 'last_clean': '2024-09-16T15:39:20.523277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:20.543264+0000', 'last_undegraded': '2024-09-16T15:39:20.543264+0000', 'last_fullsized': '2024-09-16T15:39:20.543264+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T15:39:27.606 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1 is not active+clean 2024-09-16T15:39:27.606 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:39:21.531232+0000', 'last_change': '2024-09-16T15:39:21.531232+0000', 'last_active': '2024-09-16T15:39:21.531232+0000', 'last_peered': '2024-09-16T15:39:21.531232+0000', 'last_clean': '2024-09-16T15:39:21.531232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:21.531232+0000', 'last_undegraded': '2024-09-16T15:39:21.531232+0000', 'last_fullsized': '2024-09-16T15:39:21.531232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:39:27.606 INFO:tasks.ceph.ceph_manager.ceph:PG 3.0 is not active+clean 2024-09-16T15:39:27.606 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:39:21.531232+0000', 'last_change': '2024-09-16T15:39:21.531232+0000', 'last_active': '2024-09-16T15:39:21.531232+0000', 'last_peered': '2024-09-16T15:39:21.531232+0000', 'last_clean': '2024-09-16T15:39:21.531232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:21.531232+0000', 'last_undegraded': '2024-09-16T15:39:21.531232+0000', 'last_fullsized': '2024-09-16T15:39:21.531232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:39:27.606 INFO:tasks.ceph.ceph_manager.ceph:PG 2.2 is not active+clean 2024-09-16T15:39:27.606 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:20.543244+0000', 'last_change': '2024-09-16T15:39:20.543244+0000', 'last_active': '2024-09-16T15:39:20.523277+0000', 'last_peered': '2024-09-16T15:39:20.523277+0000', 'last_clean': '2024-09-16T15:39:20.523277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:20.543244+0000', 'last_undegraded': '2024-09-16T15:39:20.543244+0000', 'last_fullsized': '2024-09-16T15:39:20.543244+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T15:39:27.606 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3 is not active+clean 2024-09-16T15:39:27.606 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:39:21.531232+0000', 'last_change': '2024-09-16T15:39:21.531232+0000', 'last_active': '2024-09-16T15:39:21.531232+0000', 'last_peered': '2024-09-16T15:39:21.531232+0000', 'last_clean': '2024-09-16T15:39:21.531232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:21.531232+0000', 'last_undegraded': '2024-09-16T15:39:21.531232+0000', 'last_fullsized': '2024-09-16T15:39:21.531232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:39:27.606 INFO:tasks.ceph.ceph_manager.ceph:PG 2.4 is not active+clean 2024-09-16T15:39:27.606 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:20.535059+0000', 'last_change': '2024-09-16T15:39:20.535059+0000', 'last_active': '2024-09-16T15:39:20.523277+0000', 'last_peered': '2024-09-16T15:39:20.523277+0000', 'last_clean': '2024-09-16T15:39:20.523277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:20.535059+0000', 'last_undegraded': '2024-09-16T15:39:20.535059+0000', 'last_fullsized': '2024-09-16T15:39:20.535059+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-16T15:39:27.607 INFO:tasks.ceph.ceph_manager.ceph:PG 3.5 is not active+clean 2024-09-16T15:39:27.607 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:39:21.531232+0000', 'last_change': '2024-09-16T15:39:21.531232+0000', 'last_active': '2024-09-16T15:39:21.531232+0000', 'last_peered': '2024-09-16T15:39:21.531232+0000', 'last_clean': '2024-09-16T15:39:21.531232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:21.531232+0000', 'last_undegraded': '2024-09-16T15:39:21.531232+0000', 'last_fullsized': '2024-09-16T15:39:21.531232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:39:27.607 INFO:tasks.ceph.ceph_manager.ceph:PG 2.5 is not active+clean 2024-09-16T15:39:27.607 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:20.543480+0000', 'last_change': '2024-09-16T15:39:20.543480+0000', 'last_active': '2024-09-16T15:39:20.523277+0000', 'last_peered': '2024-09-16T15:39:20.523277+0000', 'last_clean': '2024-09-16T15:39:20.523277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:20.543480+0000', 'last_undegraded': '2024-09-16T15:39:20.543480+0000', 'last_fullsized': '2024-09-16T15:39:20.543480+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T15:39:27.607 INFO:tasks.ceph.ceph_manager.ceph:PG 3.4 is not active+clean 2024-09-16T15:39:27.607 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:39:21.531232+0000', 'last_change': '2024-09-16T15:39:21.531232+0000', 'last_active': '2024-09-16T15:39:21.531232+0000', 'last_peered': '2024-09-16T15:39:21.531232+0000', 'last_clean': '2024-09-16T15:39:21.531232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:21.531232+0000', 'last_undegraded': '2024-09-16T15:39:21.531232+0000', 'last_fullsized': '2024-09-16T15:39:21.531232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:39:27.607 INFO:tasks.ceph.ceph_manager.ceph:PG 2.6 is not active+clean 2024-09-16T15:39:27.607 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:20.535043+0000', 'last_change': '2024-09-16T15:39:20.535043+0000', 'last_active': '2024-09-16T15:39:20.523277+0000', 'last_peered': '2024-09-16T15:39:20.523277+0000', 'last_clean': '2024-09-16T15:39:20.523277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:20.535043+0000', 'last_undegraded': '2024-09-16T15:39:20.535043+0000', 'last_fullsized': '2024-09-16T15:39:20.535043+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-16T15:39:27.607 INFO:tasks.ceph.ceph_manager.ceph:PG 3.7 is not active+clean 2024-09-16T15:39:27.607 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:39:21.531232+0000', 'last_change': '2024-09-16T15:39:21.531232+0000', 'last_active': '2024-09-16T15:39:21.531232+0000', 'last_peered': '2024-09-16T15:39:21.531232+0000', 'last_clean': '2024-09-16T15:39:21.531232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:21.531232+0000', 'last_undegraded': '2024-09-16T15:39:21.531232+0000', 'last_fullsized': '2024-09-16T15:39:21.531232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:39:27.607 INFO:tasks.ceph.ceph_manager.ceph:PG 2.7 is not active+clean 2024-09-16T15:39:27.607 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:20.543478+0000', 'last_change': '2024-09-16T15:39:20.543478+0000', 'last_active': '2024-09-16T15:39:20.523277+0000', 'last_peered': '2024-09-16T15:39:20.523277+0000', 'last_clean': '2024-09-16T15:39:20.523277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:20.543478+0000', 'last_undegraded': '2024-09-16T15:39:20.543478+0000', 'last_fullsized': '2024-09-16T15:39:20.543478+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T15:39:27.608 INFO:tasks.ceph.ceph_manager.ceph:PG 3.6 is not active+clean 2024-09-16T15:39:27.608 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:39:21.531232+0000', 'last_change': '2024-09-16T15:39:21.531232+0000', 'last_active': '2024-09-16T15:39:21.531232+0000', 'last_peered': '2024-09-16T15:39:21.531232+0000', 'last_clean': '2024-09-16T15:39:21.531232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:21.531232+0000', 'last_undegraded': '2024-09-16T15:39:21.531232+0000', 'last_fullsized': '2024-09-16T15:39:21.531232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:39:27.608 INFO:tasks.ceph.ceph_manager.ceph:PG 2.8 is not active+clean 2024-09-16T15:39:27.608 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:20.538101+0000', 'last_change': '2024-09-16T15:39:20.538101+0000', 'last_active': '2024-09-16T15:39:20.523277+0000', 'last_peered': '2024-09-16T15:39:20.523277+0000', 'last_clean': '2024-09-16T15:39:20.523277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:20.538101+0000', 'last_undegraded': '2024-09-16T15:39:20.538101+0000', 'last_fullsized': '2024-09-16T15:39:20.538101+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T15:39:27.608 INFO:tasks.ceph.ceph_manager.ceph:PG 3.9 is not active+clean 2024-09-16T15:39:27.608 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:39:21.531232+0000', 'last_change': '2024-09-16T15:39:21.531232+0000', 'last_active': '2024-09-16T15:39:21.531232+0000', 'last_peered': '2024-09-16T15:39:21.531232+0000', 'last_clean': '2024-09-16T15:39:21.531232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:21.531232+0000', 'last_undegraded': '2024-09-16T15:39:21.531232+0000', 'last_fullsized': '2024-09-16T15:39:21.531232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:39:27.608 INFO:tasks.ceph.ceph_manager.ceph:PG 2.9 is not active+clean 2024-09-16T15:39:27.608 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:20.534937+0000', 'last_change': '2024-09-16T15:39:20.534937+0000', 'last_active': '2024-09-16T15:39:20.523277+0000', 'last_peered': '2024-09-16T15:39:20.523277+0000', 'last_clean': '2024-09-16T15:39:20.523277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:20.534937+0000', 'last_undegraded': '2024-09-16T15:39:20.534937+0000', 'last_fullsized': '2024-09-16T15:39:20.534937+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-16T15:39:27.608 INFO:tasks.ceph.ceph_manager.ceph:PG 3.8 is not active+clean 2024-09-16T15:39:27.608 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:39:21.531232+0000', 'last_change': '2024-09-16T15:39:21.531232+0000', 'last_active': '2024-09-16T15:39:21.531232+0000', 'last_peered': '2024-09-16T15:39:21.531232+0000', 'last_clean': '2024-09-16T15:39:21.531232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:21.531232+0000', 'last_undegraded': '2024-09-16T15:39:21.531232+0000', 'last_fullsized': '2024-09-16T15:39:21.531232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:39:27.608 INFO:tasks.ceph.ceph_manager.ceph:PG 2.a is not active+clean 2024-09-16T15:39:27.608 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:20.541272+0000', 'last_change': '2024-09-16T15:39:20.541272+0000', 'last_active': '2024-09-16T15:39:20.523277+0000', 'last_peered': '2024-09-16T15:39:20.523277+0000', 'last_clean': '2024-09-16T15:39:20.523277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:20.541272+0000', 'last_undegraded': '2024-09-16T15:39:20.541272+0000', 'last_fullsized': '2024-09-16T15:39:20.541272+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-16T15:39:27.609 INFO:tasks.ceph.ceph_manager.ceph:PG 2.b is not active+clean 2024-09-16T15:39:27.609 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:20.544248+0000', 'last_change': '2024-09-16T15:39:20.544248+0000', 'last_active': '2024-09-16T15:39:20.523277+0000', 'last_peered': '2024-09-16T15:39:20.523277+0000', 'last_clean': '2024-09-16T15:39:20.523277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:20.544248+0000', 'last_undegraded': '2024-09-16T15:39:20.544248+0000', 'last_fullsized': '2024-09-16T15:39:20.544248+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T15:39:27.609 INFO:tasks.ceph.ceph_manager.ceph:PG 3.a is not active+clean 2024-09-16T15:39:27.609 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:39:21.531232+0000', 'last_change': '2024-09-16T15:39:21.531232+0000', 'last_active': '2024-09-16T15:39:21.531232+0000', 'last_peered': '2024-09-16T15:39:21.531232+0000', 'last_clean': '2024-09-16T15:39:21.531232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:21.531232+0000', 'last_undegraded': '2024-09-16T15:39:21.531232+0000', 'last_fullsized': '2024-09-16T15:39:21.531232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:39:27.609 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1d is not active+clean 2024-09-16T15:39:27.609 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:39:21.531232+0000', 'last_change': '2024-09-16T15:39:21.531232+0000', 'last_active': '2024-09-16T15:39:21.531232+0000', 'last_peered': '2024-09-16T15:39:21.531232+0000', 'last_clean': '2024-09-16T15:39:21.531232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:21.531232+0000', 'last_undegraded': '2024-09-16T15:39:21.531232+0000', 'last_fullsized': '2024-09-16T15:39:21.531232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:39:27.609 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1c is not active+clean 2024-09-16T15:39:27.609 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:20.537887+0000', 'last_change': '2024-09-16T15:39:20.537887+0000', 'last_active': '2024-09-16T15:39:20.523277+0000', 'last_peered': '2024-09-16T15:39:20.523277+0000', 'last_clean': '2024-09-16T15:39:20.523277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:20.537887+0000', 'last_undegraded': '2024-09-16T15:39:20.537887+0000', 'last_fullsized': '2024-09-16T15:39:20.537887+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T15:39:27.609 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1c is not active+clean 2024-09-16T15:39:27.609 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:39:21.531232+0000', 'last_change': '2024-09-16T15:39:21.531232+0000', 'last_active': '2024-09-16T15:39:21.531232+0000', 'last_peered': '2024-09-16T15:39:21.531232+0000', 'last_clean': '2024-09-16T15:39:21.531232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:21.531232+0000', 'last_undegraded': '2024-09-16T15:39:21.531232+0000', 'last_fullsized': '2024-09-16T15:39:21.531232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:39:27.610 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1d is not active+clean 2024-09-16T15:39:27.610 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:20.538384+0000', 'last_change': '2024-09-16T15:39:20.538384+0000', 'last_active': '2024-09-16T15:39:20.523277+0000', 'last_peered': '2024-09-16T15:39:20.523277+0000', 'last_clean': '2024-09-16T15:39:20.523277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:20.538384+0000', 'last_undegraded': '2024-09-16T15:39:20.538384+0000', 'last_fullsized': '2024-09-16T15:39:20.538384+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T15:39:27.610 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1f is not active+clean 2024-09-16T15:39:27.610 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:39:21.531232+0000', 'last_change': '2024-09-16T15:39:21.531232+0000', 'last_active': '2024-09-16T15:39:21.531232+0000', 'last_peered': '2024-09-16T15:39:21.531232+0000', 'last_clean': '2024-09-16T15:39:21.531232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:21.531232+0000', 'last_undegraded': '2024-09-16T15:39:21.531232+0000', 'last_fullsized': '2024-09-16T15:39:21.531232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:39:27.610 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1e is not active+clean 2024-09-16T15:39:27.610 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:20.534421+0000', 'last_change': '2024-09-16T15:39:20.534421+0000', 'last_active': '2024-09-16T15:39:20.523277+0000', 'last_peered': '2024-09-16T15:39:20.523277+0000', 'last_clean': '2024-09-16T15:39:20.523277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:20.534421+0000', 'last_undegraded': '2024-09-16T15:39:20.534421+0000', 'last_fullsized': '2024-09-16T15:39:20.534421+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-16T15:39:27.610 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1e is not active+clean 2024-09-16T15:39:27.610 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:39:21.531232+0000', 'last_change': '2024-09-16T15:39:21.531232+0000', 'last_active': '2024-09-16T15:39:21.531232+0000', 'last_peered': '2024-09-16T15:39:21.531232+0000', 'last_clean': '2024-09-16T15:39:21.531232+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:21.531232+0000', 'last_undegraded': '2024-09-16T15:39:21.531232+0000', 'last_fullsized': '2024-09-16T15:39:21.531232+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:21.531232+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:39:27.610 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1f is not active+clean 2024-09-16T15:39:27.610 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:39:20.535936+0000', 'last_change': '2024-09-16T15:39:20.535936+0000', 'last_active': '2024-09-16T15:39:20.523277+0000', 'last_peered': '2024-09-16T15:39:20.523277+0000', 'last_clean': '2024-09-16T15:39:20.523277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:39:20.535936+0000', 'last_undegraded': '2024-09-16T15:39:20.535936+0000', 'last_fullsized': '2024-09-16T15:39:20.535936+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'last_clean_scrub_stamp': '2024-09-16T15:39:20.523277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-16T15:39:27.611 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T15:39:27.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.787+0000 7fc602400640 1 -- 172.21.15.151:0/3985857471 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc5e0005230 msgr2=0x7fc5e000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:27.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.787+0000 7fc602400640 1 --2- 172.21.15.151:0/3985857471 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc5e0005230 0x7fc5e000b9c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc5f400c640 tx=0x7fc5e0005830 comp rx=0 tx=0).stop 2024-09-16T15:39:27.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.788+0000 7fc602400640 1 -- 172.21.15.151:0/3985857471 shutdown_connections 2024-09-16T15:39:27.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.788+0000 7fc602400640 1 --2- 172.21.15.151:0/3985857471 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc5e0005230 0x7fc5e000b9c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:27.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.788+0000 7fc602400640 1 -- 172.21.15.151:0/3985857471 >> 172.21.15.151:0/3985857471 conn(0x7fc5fc0b18e0 msgr2=0x7fc5fc0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:27.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.788+0000 7fc602400640 1 -- 172.21.15.151:0/3985857471 shutdown_connections 2024-09-16T15:39:27.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.788+0000 7fc602400640 1 -- 172.21.15.151:0/3985857471 wait complete. 2024-09-16T15:39:27.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.789+0000 7fc602400640 1 Processor -- start 2024-09-16T15:39:27.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.789+0000 7fc602400640 1 -- start start 2024-09-16T15:39:27.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.790+0000 7fc602400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5e0005230 0x7fc5fc09d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:27.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.790+0000 7fc602400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5fc1f67b0 0x7fc5fc147c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:27.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.790+0000 7fc602400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc5fc0ae0b0 0x7fc5fc1f5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:27.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.790+0000 7fc602400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc5fc15cad0 con 0x7fc5fc0ae0b0 2024-09-16T15:39:27.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.790+0000 7fc602400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc5fc0cc980 con 0x7fc5e0005230 2024-09-16T15:39:27.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.790+0000 7fc602400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fc5fc15cc50 con 0x7fc5fc1f67b0 2024-09-16T15:39:27.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.791+0000 7fc601a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc5fc0ae0b0 0x7fc5fc1f5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:27.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.791+0000 7fc601a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc5fc0ae0b0 0x7fc5fc1f5af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47314/0 (socket says 172.21.15.151:47314) 2024-09-16T15:39:27.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.791+0000 7fc601a00640 1 -- 172.21.15.151:0/615848416 learned_addr learned my addr 172.21.15.151:0/615848416 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:27.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.791+0000 7fc601000640 1 --2- 172.21.15.151:0/615848416 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5e0005230 0x7fc5fc09d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:27.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.791+0000 7fc5f3e00640 1 --2- 172.21.15.151:0/615848416 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5fc1f67b0 0x7fc5fc147c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:27.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.791+0000 7fc601a00640 1 -- 172.21.15.151:0/615848416 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5fc1f67b0 msgr2=0x7fc5fc147c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:27.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.791+0000 7fc601a00640 1 --2- 172.21.15.151:0/615848416 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5fc1f67b0 0x7fc5fc147c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:27.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.791+0000 7fc601a00640 1 -- 172.21.15.151:0/615848416 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5e0005230 msgr2=0x7fc5fc09d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:27.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.791+0000 7fc601a00640 1 --2- 172.21.15.151:0/615848416 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5e0005230 0x7fc5fc09d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:27.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.791+0000 7fc601a00640 1 -- 172.21.15.151:0/615848416 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5fc148190 con 0x7fc5fc0ae0b0 2024-09-16T15:39:27.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.791+0000 7fc601a00640 1 --2- 172.21.15.151:0/615848416 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc5fc0ae0b0 0x7fc5fc1f5af0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc5f402b350 tx=0x7fc5f4001b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:27.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.792+0000 7fc5f9600640 1 -- 172.21.15.151:0/615848416 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f4039070 con 0x7fc5fc0ae0b0 2024-09-16T15:39:27.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.792+0000 7fc5f9600640 1 -- 172.21.15.151:0/615848416 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc5f4006680 con 0x7fc5fc0ae0b0 2024-09-16T15:39:27.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.792+0000 7fc5f9600640 1 -- 172.21.15.151:0/615848416 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f402b6a0 con 0x7fc5fc0ae0b0 2024-09-16T15:39:27.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.792+0000 7fc602400640 1 -- 172.21.15.151:0/615848416 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5fc09c830 con 0x7fc5fc0ae0b0 2024-09-16T15:39:27.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.792+0000 7fc602400640 1 -- 172.21.15.151:0/615848416 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5fc09cdf0 con 0x7fc5fc0ae0b0 2024-09-16T15:39:27.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.793+0000 7fc602400640 1 -- 172.21.15.151:0/615848416 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5fc09de10 con 0x7fc5fc0ae0b0 2024-09-16T15:39:27.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.794+0000 7fc5f9600640 1 -- 172.21.15.151:0/615848416 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fc5f403c020 con 0x7fc5fc0ae0b0 2024-09-16T15:39:27.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.795+0000 7fc5f9600640 1 --2- 172.21.15.151:0/615848416 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc5d80788b0 0x7fc5d807ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:27.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.795+0000 7fc5f9600640 1 -- 172.21.15.151:0/615848416 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fc5f40b9ef0 con 0x7fc5fc0ae0b0 2024-09-16T15:39:27.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.795+0000 7fc601000640 1 --2- 172.21.15.151:0/615848416 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc5d80788b0 0x7fc5d807ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:27.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.796+0000 7fc601000640 1 --2- 172.21.15.151:0/615848416 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc5d80788b0 0x7fc5d807ad70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec006580 tx=0x7fc5ec006b10 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:27.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:27.799+0000 7fc5f9600640 1 -- 172.21.15.151:0/615848416 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc5f4045030 con 0x7fc5fc0ae0b0 2024-09-16T15:39:28.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:28.051+0000 7fc602400640 1 -- 172.21.15.151:0/615848416 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fc5fc070ae0 con 0x7fc5fc0ae0b0 2024-09-16T15:39:28.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:28.052+0000 7fc5f9600640 1 -- 172.21.15.151:0/615848416 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1213 (secure 0 0 0) 0x7fc5fc148190 con 0x7fc5fc0ae0b0 2024-09-16T15:39:28.053 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:39:28.053 INFO:teuthology.orchestra.run.smithi151.stdout:{"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":31,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":22,"num_osds":8,"num_up_osds":8,"osd_up_since":1726501150,"num_in_osds":8,"osd_in_since":1726501140,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":61},{"state_name":"creating+peering","count":29},{"state_name":"active+clean","count":7}],"num_pgs":97,"num_pools":3,"num_objects":6,"data_bytes":590492,"bytes_used":224579584,"bytes_avail":772869533696,"bytes_total":773094113280,"unknown_pgs_ratio":0.62886595726013184,"inactive_pgs_ratio":0.29896906018257141,"write_bytes_sec":1623,"read_op_per_sec":0,"write_op_per_sec":1},"fsmap":{"epoch":8,"btime":"2024-09-16T15:39:25:569070+0000","id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"d","status":"up:active","gid":4350}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-09-16T15:38:56.350493+0000","services":{}},"progress_events":{}} 2024-09-16T15:39:28.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:28.054+0000 7fc602400640 1 -- 172.21.15.151:0/615848416 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc5d80788b0 msgr2=0x7fc5d807ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:28.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:28.054+0000 7fc602400640 1 --2- 172.21.15.151:0/615848416 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc5d80788b0 0x7fc5d807ad70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec006580 tx=0x7fc5ec006b10 comp rx=0 tx=0).stop 2024-09-16T15:39:28.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:28.054+0000 7fc602400640 1 -- 172.21.15.151:0/615848416 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc5fc0ae0b0 msgr2=0x7fc5fc1f5af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:28.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:28.054+0000 7fc602400640 1 --2- 172.21.15.151:0/615848416 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc5fc0ae0b0 0x7fc5fc1f5af0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc5f402b350 tx=0x7fc5f4001b20 comp rx=0 tx=0).stop 2024-09-16T15:39:28.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:28.054+0000 7fc602400640 1 -- 172.21.15.151:0/615848416 shutdown_connections 2024-09-16T15:39:28.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:28.054+0000 7fc602400640 1 --2- 172.21.15.151:0/615848416 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc5d80788b0 0x7fc5d807ad70 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:28.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:28.054+0000 7fc602400640 1 --2- 172.21.15.151:0/615848416 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc5fc0ae0b0 0x7fc5fc1f5af0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:28.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:28.054+0000 7fc602400640 1 --2- 172.21.15.151:0/615848416 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5fc1f67b0 0x7fc5fc147c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:28.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:28.054+0000 7fc602400640 1 --2- 172.21.15.151:0/615848416 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5e0005230 0x7fc5fc09d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:28.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:28.054+0000 7fc602400640 1 -- 172.21.15.151:0/615848416 >> 172.21.15.151:0/615848416 conn(0x7fc5fc0b18e0 msgr2=0x7fc5fc15e270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:28.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:28.054+0000 7fc602400640 1 -- 172.21.15.151:0/615848416 shutdown_connections 2024-09-16T15:39:28.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:28.054+0000 7fc602400640 1 -- 172.21.15.151:0/615848416 wait complete. 2024-09-16T15:39:28.072 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-16T15:39:28.072 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:39:28.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:28.234+0000 7f99ce000640 1 -- 172.21.15.151:0/1174978468 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99c8161430 msgr2=0x7f99c80b5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:28.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:28.234+0000 7f99ce000640 1 --2- 172.21.15.151:0/1174978468 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99c8161430 0x7f99c80b5540 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f99c808e060 tx=0x7f99bc02a420 comp rx=0 tx=0).stop 2024-09-16T15:39:28.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:28.234+0000 7f99ce000640 1 -- 172.21.15.151:0/1174978468 shutdown_connections 2024-09-16T15:39:28.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:28.234+0000 7f99ce000640 1 --2- 172.21.15.151:0/1174978468 >> v2:172.21.15.167:3300/0 conn(0x7f99c80cef40 0x7f99c80cf360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:28.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:28.234+0000 7f99ce000640 1 --2- 172.21.15.151:0/1174978468 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99c80b4aa0 0x7f99c80b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:28.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:28.234+0000 7f99ce000640 1 --2- 172.21.15.151:0/1174978468 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99c8161430 0x7f99c80b5540 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:28.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:28.234+0000 7f99ce000640 1 -- 172.21.15.151:0/1174978468 >> 172.21.15.151:0/1174978468 conn(0x7f99c80d3dc0 msgr2=0x7f99c80d41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:28.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:28.235+0000 7f99ce000640 1 -- 172.21.15.151:0/1174978468 shutdown_connections 2024-09-16T15:39:28.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:28.235+0000 7f99ce000640 1 -- 172.21.15.151:0/1174978468 wait complete. 2024-09-16T15:39:28.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:28.236+0000 7f99ce000640 1 Processor -- start 2024-09-16T15:39:28.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:28.236+0000 7f99ce000640 1 -- start start 2024-09-16T15:39:28.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:28.237+0000 7f99ce000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99c80b4aa0 0x7f99c80cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:28.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:28.237+0000 7f99ce000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99c80cef40 0x7f99c80cc390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:28.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:28.237+0000 7f99ce000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99c8161430 0x7f99c81f79f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:28.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:28.237+0000 7f99ce000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f99c80cfc30 con 0x7f99c80b4aa0 2024-09-16T15:39:28.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:28.237+0000 7f99ce000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f99c80cfab0 con 0x7f99c80cef40 2024-09-16T15:39:28.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:28.237+0000 7f99ce000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f99c80cfdb0 con 0x7f99c8161430 2024-09-16T15:39:28.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:28.237+0000 7f99ccc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99c80b4aa0 0x7f99c80cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:28.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:28.237+0000 7f99ccc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99c80b4aa0 0x7f99c80cbe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47332/0 (socket says 172.21.15.151:47332) 2024-09-16T15:39:28.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:28.237+0000 7f99ccc00640 1 -- 172.21.15.151:0/2377994393 learned_addr learned my addr 172.21.15.151:0/2377994393 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:28.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:28.238+0000 7f99c7e00640 1 --2- 172.21.15.151:0/2377994393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99c80cef40 0x7f99c80cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:28.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:28.238+0000 7f99cd600640 1 --2- 172.21.15.151:0/2377994393 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99c8161430 0x7f99c81f79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:28.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:28.238+0000 7f99ccc00640 1 -- 172.21.15.151:0/2377994393 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99c8161430 msgr2=0x7f99c81f79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:28.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:28.238+0000 7f99ccc00640 1 --2- 172.21.15.151:0/2377994393 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99c8161430 0x7f99c81f79f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:28.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:28.238+0000 7f99ccc00640 1 -- 172.21.15.151:0/2377994393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99c80cef40 msgr2=0x7f99c80cc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:28.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:28.238+0000 7f99ccc00640 1 --2- 172.21.15.151:0/2377994393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99c80cef40 0x7f99c80cc390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:28.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:28.238+0000 7f99ccc00640 1 -- 172.21.15.151:0/2377994393 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99c8153ba0 con 0x7f99c80b4aa0 2024-09-16T15:39:28.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:28.238+0000 7f99ccc00640 1 --2- 172.21.15.151:0/2377994393 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99c80b4aa0 0x7f99c80cbe50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f99bc034040 tx=0x7f99bc0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:28.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:28.238+0000 7f99c4c00640 1 -- 172.21.15.151:0/2377994393 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99bc03e030 con 0x7f99c80b4aa0 2024-09-16T15:39:28.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:28.238+0000 7f99c4c00640 1 -- 172.21.15.151:0/2377994393 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f99bc0047a0 con 0x7f99c80b4aa0 2024-09-16T15:39:28.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:28.238+0000 7f99ce000640 1 -- 172.21.15.151:0/2377994393 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99c80cc9c0 con 0x7f99c80b4aa0 2024-09-16T15:39:28.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:28.238+0000 7f99c4c00640 1 -- 172.21.15.151:0/2377994393 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99bc035040 con 0x7f99c80b4aa0 2024-09-16T15:39:28.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:28.239+0000 7f99ce000640 1 -- 172.21.15.151:0/2377994393 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99c80ccf80 con 0x7f99c80b4aa0 2024-09-16T15:39:28.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:28.239+0000 7f99ce000640 1 -- 172.21.15.151:0/2377994393 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99c80cf360 con 0x7f99c80b4aa0 2024-09-16T15:39:28.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:28.241+0000 7f99c4c00640 1 -- 172.21.15.151:0/2377994393 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f99bc041030 con 0x7f99c80b4aa0 2024-09-16T15:39:28.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:28.241+0000 7f99c4c00640 1 --2- 172.21.15.151:0/2377994393 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f99a8078850 0x7f99a807ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:28.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:28.241+0000 7f99c4c00640 1 -- 172.21.15.151:0/2377994393 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f99bc0bb210 con 0x7f99c80b4aa0 2024-09-16T15:39:28.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:28.242+0000 7f99c7e00640 1 --2- 172.21.15.151:0/2377994393 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f99a8078850 0x7f99a807ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:28.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:28.242+0000 7f99c7e00640 1 --2- 172.21.15.151:0/2377994393 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f99a8078850 0x7f99a807ad10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f99b4005fd0 tx=0x7f99b400a4e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:28.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:28.245+0000 7f99c4c00640 1 -- 172.21.15.151:0/2377994393 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f99bc0837e0 con 0x7f99c80b4aa0 2024-09-16T15:39:28.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:28.425+0000 7f99ce000640 1 -- 172.21.15.151:0/2377994393 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f99c80760b0 con 0x7f99a8078850 2024-09-16T15:39:28.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:28.430+0000 7f99c4c00640 1 -- 172.21.15.151:0/2377994393 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251931 (secure 0 0 0) 0x7f99c80760b0 con 0x7f99a8078850 2024-09-16T15:39:28.434 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:39:28.435 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-09-16T15:39:28.437 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:28.434+0000 7f99ce000640 1 -- 172.21.15.151:0/2377994393 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f99a8078850 msgr2=0x7f99a807ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:28.437 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:28.434+0000 7f99ce000640 1 --2- 172.21.15.151:0/2377994393 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f99a8078850 0x7f99a807ad10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f99b4005fd0 tx=0x7f99b400a4e0 comp rx=0 tx=0).stop 2024-09-16T15:39:28.437 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:28.434+0000 7f99ce000640 1 -- 172.21.15.151:0/2377994393 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99c80b4aa0 msgr2=0x7f99c80cbe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:28.437 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:28.434+0000 7f99ce000640 1 --2- 172.21.15.151:0/2377994393 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99c80b4aa0 0x7f99c80cbe50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f99bc034040 tx=0x7f99bc0039e0 comp rx=0 tx=0).stop 2024-09-16T15:39:28.438 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:28.434+0000 7f99ce000640 1 -- 172.21.15.151:0/2377994393 shutdown_connections 2024-09-16T15:39:28.438 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:28.434+0000 7f99ce000640 1 --2- 172.21.15.151:0/2377994393 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f99a8078850 0x7f99a807ad10 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:28.438 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:28.434+0000 7f99ce000640 1 --2- 172.21.15.151:0/2377994393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99c80cef40 0x7f99c80cc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:28.438 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:28.434+0000 7f99ce000640 1 --2- 172.21.15.151:0/2377994393 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99c8161430 0x7f99c81f79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:28.438 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:28.434+0000 7f99ce000640 1 --2- 172.21.15.151:0/2377994393 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99c80b4aa0 0x7f99c80cbe50 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:28.438 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:28.434+0000 7f99ce000640 1 -- 172.21.15.151:0/2377994393 >> 172.21.15.151:0/2377994393 conn(0x7f99c80d3dc0 msgr2=0x7f99c80b5440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:28.438 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:28.434+0000 7f99ce000640 1 -- 172.21.15.151:0/2377994393 shutdown_connections 2024-09-16T15:39:28.438 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:28.434+0000 7f99ce000640 1 -- 172.21.15.151:0/2377994393 wait complete. 2024-09-16T15:39:28.451 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-16T15:39:27.577556+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":150,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219184,"kb_used_data":2952,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755536,"statfs":{"total":773094113280,"available":772869668864,"internally_reserved":0,"allocated":3022848,"data_stored":1795828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12738,"internal_metadata":221236798},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.037349"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.362052+0000","last_change":"2024-09-16T15:39:22.577878+0000","last_active":"2024-09-16T15:39:24.362052+0000","last_peered":"2024-09-16T15:39:24.362052+0000","last_clean":"2024-09-16T15:39:24.362052+0000","last_became_active":"2024-09-16T15:39:22.570201+0000","last_became_peered":"2024-09-16T15:39:22.570201+0000","last_unstale":"2024-09-16T15:39:24.362052+0000","last_undegraded":"2024-09-16T15:39:24.362052+0000","last_fullsized":"2024-09-16T15:39:24.362052+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:39:49.705970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.563074+0000","last_change":"2024-09-16T15:39:22.568365+0000","last_active":"2024-09-16T15:39:23.563074+0000","last_peered":"2024-09-16T15:39:23.563074+0000","last_clean":"2024-09-16T15:39:23.563074+0000","last_became_active":"2024-09-16T15:39:22.564695+0000","last_became_peered":"2024-09-16T15:39:22.564695+0000","last_unstale":"2024-09-16T15:39:23.563074+0000","last_undegraded":"2024-09-16T15:39:23.563074+0000","last_fullsized":"2024-09-16T15:39:23.563074+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:18:39.431491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.563192+0000","last_change":"2024-09-16T15:39:22.575183+0000","last_active":"2024-09-16T15:39:23.563192+0000","last_peered":"2024-09-16T15:39:23.563192+0000","last_clean":"2024-09-16T15:39:23.563192+0000","last_became_active":"2024-09-16T15:39:22.574916+0000","last_became_peered":"2024-09-16T15:39:22.574916+0000","last_unstale":"2024-09-16T15:39:23.563192+0000","last_undegraded":"2024-09-16T15:39:23.563192+0000","last_fullsized":"2024-09-16T15:39:23.563192+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:06:34.579134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.362360+0000","last_change":"2024-09-16T15:39:22.580108+0000","last_active":"2024-09-16T15:39:24.362360+0000","last_peered":"2024-09-16T15:39:24.362360+0000","last_clean":"2024-09-16T15:39:24.362360+0000","last_became_active":"2024-09-16T15:39:22.579859+0000","last_became_peered":"2024-09-16T15:39:22.579859+0000","last_unstale":"2024-09-16T15:39:24.362360+0000","last_undegraded":"2024-09-16T15:39:24.362360+0000","last_fullsized":"2024-09-16T15:39:24.362360+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:32:44.877620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361299+0000","last_change":"2024-09-16T15:39:22.566468+0000","last_active":"2024-09-16T15:39:24.361299+0000","last_peered":"2024-09-16T15:39:24.361299+0000","last_clean":"2024-09-16T15:39:24.361299+0000","last_became_active":"2024-09-16T15:39:22.559581+0000","last_became_peered":"2024-09-16T15:39:22.559581+0000","last_unstale":"2024-09-16T15:39:24.361299+0000","last_undegraded":"2024-09-16T15:39:24.361299+0000","last_fullsized":"2024-09-16T15:39:24.361299+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:35:04.935435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361543+0000","last_change":"2024-09-16T15:39:22.579964+0000","last_active":"2024-09-16T15:39:24.361543+0000","last_peered":"2024-09-16T15:39:24.361543+0000","last_clean":"2024-09-16T15:39:24.361543+0000","last_became_active":"2024-09-16T15:39:22.579714+0000","last_became_peered":"2024-09-16T15:39:22.579714+0000","last_unstale":"2024-09-16T15:39:24.361543+0000","last_undegraded":"2024-09-16T15:39:24.361543+0000","last_fullsized":"2024-09-16T15:39:24.361543+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:54:55.307273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360552+0000","last_change":"2024-09-16T15:39:22.578008+0000","last_active":"2024-09-16T15:39:24.360552+0000","last_peered":"2024-09-16T15:39:24.360552+0000","last_clean":"2024-09-16T15:39:24.360552+0000","last_became_active":"2024-09-16T15:39:22.577770+0000","last_became_peered":"2024-09-16T15:39:22.577770+0000","last_unstale":"2024-09-16T15:39:24.360552+0000","last_undegraded":"2024-09-16T15:39:24.360552+0000","last_fullsized":"2024-09-16T15:39:24.360552+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:03:05.655450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.562837+0000","last_change":"2024-09-16T15:39:22.565821+0000","last_active":"2024-09-16T15:39:23.562837+0000","last_peered":"2024-09-16T15:39:23.562837+0000","last_clean":"2024-09-16T15:39:23.562837+0000","last_became_active":"2024-09-16T15:39:22.564114+0000","last_became_peered":"2024-09-16T15:39:22.564114+0000","last_unstale":"2024-09-16T15:39:23.562837+0000","last_undegraded":"2024-09-16T15:39:23.562837+0000","last_fullsized":"2024-09-16T15:39:23.562837+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:03:28.558231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.561256+0000","last_change":"2024-09-16T15:39:22.557568+0000","last_active":"2024-09-16T15:39:23.561256+0000","last_peered":"2024-09-16T15:39:23.561256+0000","last_clean":"2024-09-16T15:39:23.561256+0000","last_became_active":"2024-09-16T15:39:22.556305+0000","last_became_peered":"2024-09-16T15:39:22.556305+0000","last_unstale":"2024-09-16T15:39:23.561256+0000","last_undegraded":"2024-09-16T15:39:23.561256+0000","last_fullsized":"2024-09-16T15:39:23.561256+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:19:39.018320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361933+0000","last_change":"2024-09-16T15:39:22.573944+0000","last_active":"2024-09-16T15:39:24.361933+0000","last_peered":"2024-09-16T15:39:24.361933+0000","last_clean":"2024-09-16T15:39:24.361933+0000","last_became_active":"2024-09-16T15:39:22.573688+0000","last_became_peered":"2024-09-16T15:39:22.573688+0000","last_unstale":"2024-09-16T15:39:24.361933+0000","last_undegraded":"2024-09-16T15:39:24.361933+0000","last_fullsized":"2024-09-16T15:39:24.361933+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:00:05.185794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360929+0000","last_change":"2024-09-16T15:39:22.580084+0000","last_active":"2024-09-16T15:39:24.360929+0000","last_peered":"2024-09-16T15:39:24.360929+0000","last_clean":"2024-09-16T15:39:24.360929+0000","last_became_active":"2024-09-16T15:39:22.571878+0000","last_became_peered":"2024-09-16T15:39:22.571878+0000","last_unstale":"2024-09-16T15:39:24.360929+0000","last_undegraded":"2024-09-16T15:39:24.360929+0000","last_fullsized":"2024-09-16T15:39:24.360929+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:46:31.911518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361633+0000","last_change":"2024-09-16T15:39:22.575436+0000","last_active":"2024-09-16T15:39:24.361633+0000","last_peered":"2024-09-16T15:39:24.361633+0000","last_clean":"2024-09-16T15:39:24.361633+0000","last_became_active":"2024-09-16T15:39:22.568528+0000","last_became_peered":"2024-09-16T15:39:22.568528+0000","last_unstale":"2024-09-16T15:39:24.361633+0000","last_undegraded":"2024-09-16T15:39:24.361633+0000","last_fullsized":"2024-09-16T15:39:24.361633+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:05:28.120874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360645+0000","last_change":"2024-09-16T15:39:22.566575+0000","last_active":"2024-09-16T15:39:24.360645+0000","last_peered":"2024-09-16T15:39:24.360645+0000","last_clean":"2024-09-16T15:39:24.360645+0000","last_became_active":"2024-09-16T15:39:22.566295+0000","last_became_peered":"2024-09-16T15:39:22.566295+0000","last_unstale":"2024-09-16T15:39:24.360645+0000","last_undegraded":"2024-09-16T15:39:24.360645+0000","last_fullsized":"2024-09-16T15:39:24.360645+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:10:23.402092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360395+0000","last_change":"2024-09-16T15:39:22.571476+0000","last_active":"2024-09-16T15:39:24.360395+0000","last_peered":"2024-09-16T15:39:24.360395+0000","last_clean":"2024-09-16T15:39:24.360395+0000","last_became_active":"2024-09-16T15:39:22.571216+0000","last_became_peered":"2024-09-16T15:39:22.571216+0000","last_unstale":"2024-09-16T15:39:24.360395+0000","last_undegraded":"2024-09-16T15:39:24.360395+0000","last_fullsized":"2024-09-16T15:39:24.360395+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:40:58.311998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.563103+0000","last_change":"2024-09-16T15:39:22.564630+0000","last_active":"2024-09-16T15:39:23.563103+0000","last_peered":"2024-09-16T15:39:23.563103+0000","last_clean":"2024-09-16T15:39:23.563103+0000","last_became_active":"2024-09-16T15:39:22.564326+0000","last_became_peered":"2024-09-16T15:39:22.564326+0000","last_unstale":"2024-09-16T15:39:23.563103+0000","last_undegraded":"2024-09-16T15:39:23.563103+0000","last_fullsized":"2024-09-16T15:39:23.563103+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:58:23.963120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360193+0000","last_change":"2024-09-16T15:39:22.574288+0000","last_active":"2024-09-16T15:39:24.360193+0000","last_peered":"2024-09-16T15:39:24.360193+0000","last_clean":"2024-09-16T15:39:24.360193+0000","last_became_active":"2024-09-16T15:39:22.573979+0000","last_became_peered":"2024-09-16T15:39:22.573979+0000","last_unstale":"2024-09-16T15:39:24.360193+0000","last_undegraded":"2024-09-16T15:39:24.360193+0000","last_fullsized":"2024-09-16T15:39:24.360193+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:10:15.420114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361109+0000","last_change":"2024-09-16T15:39:22.561784+0000","last_active":"2024-09-16T15:39:24.361109+0000","last_peered":"2024-09-16T15:39:24.361109+0000","last_clean":"2024-09-16T15:39:24.361109+0000","last_became_active":"2024-09-16T15:39:22.561487+0000","last_became_peered":"2024-09-16T15:39:22.561487+0000","last_unstale":"2024-09-16T15:39:24.361109+0000","last_undegraded":"2024-09-16T15:39:24.361109+0000","last_fullsized":"2024-09-16T15:39:24.361109+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:26:52.642854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.714434+0000","last_change":"2024-09-16T15:39:22.579035+0000","last_active":"2024-09-16T15:39:23.714434+0000","last_peered":"2024-09-16T15:39:23.714434+0000","last_clean":"2024-09-16T15:39:23.714434+0000","last_became_active":"2024-09-16T15:39:22.578734+0000","last_became_peered":"2024-09-16T15:39:22.578734+0000","last_unstale":"2024-09-16T15:39:23.714434+0000","last_undegraded":"2024-09-16T15:39:23.714434+0000","last_fullsized":"2024-09-16T15:39:23.714434+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:13:19.369864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361928+0000","last_change":"2024-09-16T15:39:22.577483+0000","last_active":"2024-09-16T15:39:24.361928+0000","last_peered":"2024-09-16T15:39:24.361928+0000","last_clean":"2024-09-16T15:39:24.361928+0000","last_became_active":"2024-09-16T15:39:22.577242+0000","last_became_peered":"2024-09-16T15:39:22.577242+0000","last_unstale":"2024-09-16T15:39:24.361928+0000","last_undegraded":"2024-09-16T15:39:24.361928+0000","last_fullsized":"2024-09-16T15:39:24.361928+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:29:43.879706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360912+0000","last_change":"2024-09-16T15:39:22.561587+0000","last_active":"2024-09-16T15:39:24.360912+0000","last_peered":"2024-09-16T15:39:24.360912+0000","last_clean":"2024-09-16T15:39:24.360912+0000","last_became_active":"2024-09-16T15:39:22.561213+0000","last_became_peered":"2024-09-16T15:39:22.561213+0000","last_unstale":"2024-09-16T15:39:24.360912+0000","last_undegraded":"2024-09-16T15:39:24.360912+0000","last_fullsized":"2024-09-16T15:39:24.360912+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:00:48.581056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.562380+0000","last_change":"2024-09-16T15:39:22.579126+0000","last_active":"2024-09-16T15:39:23.562380+0000","last_peered":"2024-09-16T15:39:23.562380+0000","last_clean":"2024-09-16T15:39:23.562380+0000","last_became_active":"2024-09-16T15:39:22.577915+0000","last_became_peered":"2024-09-16T15:39:22.577915+0000","last_unstale":"2024-09-16T15:39:23.562380+0000","last_undegraded":"2024-09-16T15:39:23.562380+0000","last_fullsized":"2024-09-16T15:39:23.562380+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:18:15.980220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.560783+0000","last_change":"2024-09-16T15:39:22.560525+0000","last_active":"2024-09-16T15:39:23.560783+0000","last_peered":"2024-09-16T15:39:23.560783+0000","last_clean":"2024-09-16T15:39:23.560783+0000","last_became_active":"2024-09-16T15:39:22.560279+0000","last_became_peered":"2024-09-16T15:39:22.560279+0000","last_unstale":"2024-09-16T15:39:23.560783+0000","last_undegraded":"2024-09-16T15:39:23.560783+0000","last_fullsized":"2024-09-16T15:39:23.560783+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:22:24.884519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.562668+0000","last_change":"2024-09-16T15:39:22.567598+0000","last_active":"2024-09-16T15:39:23.562668+0000","last_peered":"2024-09-16T15:39:23.562668+0000","last_clean":"2024-09-16T15:39:23.562668+0000","last_became_active":"2024-09-16T15:39:22.567356+0000","last_became_peered":"2024-09-16T15:39:22.567356+0000","last_unstale":"2024-09-16T15:39:23.562668+0000","last_undegraded":"2024-09-16T15:39:23.562668+0000","last_fullsized":"2024-09-16T15:39:23.562668+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:22:34.502460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360199+0000","last_change":"2024-09-16T15:39:22.558806+0000","last_active":"2024-09-16T15:39:24.360199+0000","last_peered":"2024-09-16T15:39:24.360199+0000","last_clean":"2024-09-16T15:39:24.360199+0000","last_became_active":"2024-09-16T15:39:22.558473+0000","last_became_peered":"2024-09-16T15:39:22.558473+0000","last_unstale":"2024-09-16T15:39:24.360199+0000","last_undegraded":"2024-09-16T15:39:24.360199+0000","last_fullsized":"2024-09-16T15:39:24.360199+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:05:28.120874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.563124+0000","last_change":"2024-09-16T15:39:22.564434+0000","last_active":"2024-09-16T15:39:23.563124+0000","last_peered":"2024-09-16T15:39:23.563124+0000","last_clean":"2024-09-16T15:39:23.563124+0000","last_became_active":"2024-09-16T15:39:22.563970+0000","last_became_peered":"2024-09-16T15:39:22.563970+0000","last_unstale":"2024-09-16T15:39:23.563124+0000","last_undegraded":"2024-09-16T15:39:23.563124+0000","last_fullsized":"2024-09-16T15:39:23.563124+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:43:27.368913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.362535+0000","last_change":"2024-09-16T15:39:22.584858+0000","last_active":"2024-09-16T15:39:24.362535+0000","last_peered":"2024-09-16T15:39:24.362535+0000","last_clean":"2024-09-16T15:39:24.362535+0000","last_became_active":"2024-09-16T15:39:22.584152+0000","last_became_peered":"2024-09-16T15:39:22.584152+0000","last_unstale":"2024-09-16T15:39:24.362535+0000","last_undegraded":"2024-09-16T15:39:24.362535+0000","last_fullsized":"2024-09-16T15:39:24.362535+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:08:07.935905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360764+0000","last_change":"2024-09-16T15:39:22.573867+0000","last_active":"2024-09-16T15:39:24.360764+0000","last_peered":"2024-09-16T15:39:24.360764+0000","last_clean":"2024-09-16T15:39:24.360764+0000","last_became_active":"2024-09-16T15:39:22.573567+0000","last_became_peered":"2024-09-16T15:39:22.573567+0000","last_unstale":"2024-09-16T15:39:24.360764+0000","last_undegraded":"2024-09-16T15:39:24.360764+0000","last_fullsized":"2024-09-16T15:39:24.360764+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:18:15.980220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360545+0000","last_change":"2024-09-16T15:39:22.574268+0000","last_active":"2024-09-16T15:39:24.360545+0000","last_peered":"2024-09-16T15:39:24.360545+0000","last_clean":"2024-09-16T15:39:24.360545+0000","last_became_active":"2024-09-16T15:39:22.572891+0000","last_became_peered":"2024-09-16T15:39:22.572891+0000","last_unstale":"2024-09-16T15:39:24.360545+0000","last_undegraded":"2024-09-16T15:39:24.360545+0000","last_fullsized":"2024-09-16T15:39:24.360545+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:29:11.296912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.562547+0000","last_change":"2024-09-16T15:39:22.563793+0000","last_active":"2024-09-16T15:39:23.562547+0000","last_peered":"2024-09-16T15:39:23.562547+0000","last_clean":"2024-09-16T15:39:23.562547+0000","last_became_active":"2024-09-16T15:39:22.563504+0000","last_became_peered":"2024-09-16T15:39:22.563504+0000","last_unstale":"2024-09-16T15:39:23.562547+0000","last_undegraded":"2024-09-16T15:39:23.562547+0000","last_fullsized":"2024-09-16T15:39:23.562547+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:56:07.665096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361245+0000","last_change":"2024-09-16T15:39:22.569041+0000","last_active":"2024-09-16T15:39:24.361245+0000","last_peered":"2024-09-16T15:39:24.361245+0000","last_clean":"2024-09-16T15:39:24.361245+0000","last_became_active":"2024-09-16T15:39:22.568799+0000","last_became_peered":"2024-09-16T15:39:22.568799+0000","last_unstale":"2024-09-16T15:39:24.361245+0000","last_undegraded":"2024-09-16T15:39:24.361245+0000","last_fullsized":"2024-09-16T15:39:24.361245+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:58:26.472440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.714354+0000","last_change":"2024-09-16T15:39:22.566051+0000","last_active":"2024-09-16T15:39:23.714354+0000","last_peered":"2024-09-16T15:39:23.714354+0000","last_clean":"2024-09-16T15:39:23.714354+0000","last_became_active":"2024-09-16T15:39:22.565764+0000","last_became_peered":"2024-09-16T15:39:22.565764+0000","last_unstale":"2024-09-16T15:39:23.714354+0000","last_undegraded":"2024-09-16T15:39:23.714354+0000","last_fullsized":"2024-09-16T15:39:23.714354+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:42:34.417720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361771+0000","last_change":"2024-09-16T15:39:22.574296+0000","last_active":"2024-09-16T15:39:24.361771+0000","last_peered":"2024-09-16T15:39:24.361771+0000","last_clean":"2024-09-16T15:39:24.361771+0000","last_became_active":"2024-09-16T15:39:22.574004+0000","last_became_peered":"2024-09-16T15:39:22.574004+0000","last_unstale":"2024-09-16T15:39:24.361771+0000","last_undegraded":"2024-09-16T15:39:24.361771+0000","last_fullsized":"2024-09-16T15:39:24.361771+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:36:56.913296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.560837+0000","last_change":"2024-09-16T15:39:22.559868+0000","last_active":"2024-09-16T15:39:23.560837+0000","last_peered":"2024-09-16T15:39:23.560837+0000","last_clean":"2024-09-16T15:39:23.560837+0000","last_became_active":"2024-09-16T15:39:22.559506+0000","last_became_peered":"2024-09-16T15:39:22.559506+0000","last_unstale":"2024-09-16T15:39:23.560837+0000","last_undegraded":"2024-09-16T15:39:23.560837+0000","last_fullsized":"2024-09-16T15:39:23.560837+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:33:53.332507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.562441+0000","last_change":"2024-09-16T15:39:21.562538+0000","last_active":"2024-09-16T15:39:23.562441+0000","last_peered":"2024-09-16T15:39:23.562441+0000","last_clean":"2024-09-16T15:39:23.562441+0000","last_became_active":"2024-09-16T15:39:21.559240+0000","last_became_peered":"2024-09-16T15:39:21.559240+0000","last_unstale":"2024-09-16T15:39:23.562441+0000","last_undegraded":"2024-09-16T15:39:23.562441+0000","last_fullsized":"2024-09-16T15:39:23.562441+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:43:58.109261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.562593+0000","last_change":"2024-09-16T15:39:22.582803+0000","last_active":"2024-09-16T15:39:23.562593+0000","last_peered":"2024-09-16T15:39:23.562593+0000","last_clean":"2024-09-16T15:39:23.562593+0000","last_became_active":"2024-09-16T15:39:22.582364+0000","last_became_peered":"2024-09-16T15:39:22.582364+0000","last_unstale":"2024-09-16T15:39:23.562593+0000","last_undegraded":"2024-09-16T15:39:23.562593+0000","last_fullsized":"2024-09-16T15:39:23.562593+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:40:27.419876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360960+0000","last_change":"2024-09-16T15:39:21.567667+0000","last_active":"2024-09-16T15:39:24.360960+0000","last_peered":"2024-09-16T15:39:24.360960+0000","last_clean":"2024-09-16T15:39:24.360960+0000","last_became_active":"2024-09-16T15:39:21.567399+0000","last_became_peered":"2024-09-16T15:39:21.567399+0000","last_unstale":"2024-09-16T15:39:24.360960+0000","last_undegraded":"2024-09-16T15:39:24.360960+0000","last_fullsized":"2024-09-16T15:39:24.360960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:22:23.876565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360132+0000","last_change":"2024-09-16T15:39:21.566105+0000","last_active":"2024-09-16T15:39:24.360132+0000","last_peered":"2024-09-16T15:39:24.360132+0000","last_clean":"2024-09-16T15:39:24.360132+0000","last_became_active":"2024-09-16T15:39:21.565719+0000","last_became_peered":"2024-09-16T15:39:21.565719+0000","last_unstale":"2024-09-16T15:39:24.360132+0000","last_undegraded":"2024-09-16T15:39:24.360132+0000","last_fullsized":"2024-09-16T15:39:24.360132+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:43:58.109261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.562255+0000","last_change":"2024-09-16T15:39:22.575843+0000","last_active":"2024-09-16T15:39:23.562255+0000","last_peered":"2024-09-16T15:39:23.562255+0000","last_clean":"2024-09-16T15:39:23.562255+0000","last_became_active":"2024-09-16T15:39:22.575543+0000","last_became_peered":"2024-09-16T15:39:22.575543+0000","last_unstale":"2024-09-16T15:39:23.562255+0000","last_undegraded":"2024-09-16T15:39:23.562255+0000","last_fullsized":"2024-09-16T15:39:23.562255+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:40:27.419876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360365+0000","last_change":"2024-09-16T15:39:21.562879+0000","last_active":"2024-09-16T15:39:24.360365+0000","last_peered":"2024-09-16T15:39:24.360365+0000","last_clean":"2024-09-16T15:39:24.360365+0000","last_became_active":"2024-09-16T15:39:21.562554+0000","last_became_peered":"2024-09-16T15:39:21.562554+0000","last_unstale":"2024-09-16T15:39:24.360365+0000","last_undegraded":"2024-09-16T15:39:24.360365+0000","last_fullsized":"2024-09-16T15:39:24.360365+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:33:52.324553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.562038+0000","last_change":"2024-09-16T15:39:22.571218+0000","last_active":"2024-09-16T15:39:23.562038+0000","last_peered":"2024-09-16T15:39:23.562038+0000","last_clean":"2024-09-16T15:39:23.562038+0000","last_became_active":"2024-09-16T15:39:22.570931+0000","last_became_peered":"2024-09-16T15:39:22.570931+0000","last_unstale":"2024-09-16T15:39:23.562038+0000","last_undegraded":"2024-09-16T15:39:23.562038+0000","last_fullsized":"2024-09-16T15:39:23.562038+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:57:08.823794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.561161+0000","last_change":"2024-09-16T15:39:21.558263+0000","last_active":"2024-09-16T15:39:23.561161+0000","last_peered":"2024-09-16T15:39:23.561161+0000","last_clean":"2024-09-16T15:39:23.561161+0000","last_became_active":"2024-09-16T15:39:21.557615+0000","last_became_peered":"2024-09-16T15:39:21.557615+0000","last_unstale":"2024-09-16T15:39:23.561161+0000","last_undegraded":"2024-09-16T15:39:23.561161+0000","last_fullsized":"2024-09-16T15:39:23.561161+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:10:40.788797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.362565+0000","last_change":"2024-09-16T15:39:22.586182+0000","last_active":"2024-09-16T15:39:24.362565+0000","last_peered":"2024-09-16T15:39:24.362565+0000","last_clean":"2024-09-16T15:39:24.362565+0000","last_became_active":"2024-09-16T15:39:22.584486+0000","last_became_peered":"2024-09-16T15:39:22.584486+0000","last_unstale":"2024-09-16T15:39:24.362565+0000","last_undegraded":"2024-09-16T15:39:24.362565+0000","last_fullsized":"2024-09-16T15:39:24.362565+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:25:52.095325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361757+0000","last_change":"2024-09-16T15:39:21.558934+0000","last_active":"2024-09-16T15:39:24.361757+0000","last_peered":"2024-09-16T15:39:24.361757+0000","last_clean":"2024-09-16T15:39:24.361757+0000","last_became_active":"2024-09-16T15:39:21.557612+0000","last_became_peered":"2024-09-16T15:39:21.557612+0000","last_unstale":"2024-09-16T15:39:24.361757+0000","last_undegraded":"2024-09-16T15:39:24.361757+0000","last_fullsized":"2024-09-16T15:39:24.361757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:21:44.910826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.563494+0000","last_change":"2024-09-16T15:39:22.563449+0000","last_active":"2024-09-16T15:39:23.563494+0000","last_peered":"2024-09-16T15:39:23.563494+0000","last_clean":"2024-09-16T15:39:23.563494+0000","last_became_active":"2024-09-16T15:39:22.558597+0000","last_became_peered":"2024-09-16T15:39:22.558597+0000","last_unstale":"2024-09-16T15:39:23.563494+0000","last_undegraded":"2024-09-16T15:39:23.563494+0000","last_fullsized":"2024-09-16T15:39:23.563494+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:08:47.835110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.714227+0000","last_change":"2024-09-16T15:39:21.561906+0000","last_active":"2024-09-16T15:39:23.714227+0000","last_peered":"2024-09-16T15:39:23.714227+0000","last_clean":"2024-09-16T15:39:23.714227+0000","last_became_active":"2024-09-16T15:39:21.561566+0000","last_became_peered":"2024-09-16T15:39:21.561566+0000","last_unstale":"2024-09-16T15:39:23.714227+0000","last_undegraded":"2024-09-16T15:39:23.714227+0000","last_fullsized":"2024-09-16T15:39:23.714227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:12:53.853659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360840+0000","last_change":"2024-09-16T15:39:22.581196+0000","last_active":"2024-09-16T15:39:24.360840+0000","last_peered":"2024-09-16T15:39:24.360840+0000","last_clean":"2024-09-16T15:39:24.360840+0000","last_became_active":"2024-09-16T15:39:22.579631+0000","last_became_peered":"2024-09-16T15:39:22.579631+0000","last_unstale":"2024-09-16T15:39:24.360840+0000","last_undegraded":"2024-09-16T15:39:24.360840+0000","last_fullsized":"2024-09-16T15:39:24.360840+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:15:37.944346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.362202+0000","last_change":"2024-09-16T15:39:21.560090+0000","last_active":"2024-09-16T15:39:24.362202+0000","last_peered":"2024-09-16T15:39:24.362202+0000","last_clean":"2024-09-16T15:39:24.362202+0000","last_became_active":"2024-09-16T15:39:21.558972+0000","last_became_peered":"2024-09-16T15:39:21.558972+0000","last_unstale":"2024-09-16T15:39:24.362202+0000","last_undegraded":"2024-09-16T15:39:24.362202+0000","last_fullsized":"2024-09-16T15:39:24.362202+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:12:19.666037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361423+0000","last_change":"2024-09-16T15:39:22.560439+0000","last_active":"2024-09-16T15:39:24.361423+0000","last_peered":"2024-09-16T15:39:24.361423+0000","last_clean":"2024-09-16T15:39:24.361423+0000","last_became_active":"2024-09-16T15:39:22.560172+0000","last_became_peered":"2024-09-16T15:39:22.560172+0000","last_unstale":"2024-09-16T15:39:24.361423+0000","last_undegraded":"2024-09-16T15:39:24.361423+0000","last_fullsized":"2024-09-16T15:39:24.361423+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:01:25.170048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.714255+0000","last_change":"2024-09-16T15:39:21.567091+0000","last_active":"2024-09-16T15:39:23.714255+0000","last_peered":"2024-09-16T15:39:23.714255+0000","last_clean":"2024-09-16T15:39:23.714255+0000","last_became_active":"2024-09-16T15:39:21.566668+0000","last_became_peered":"2024-09-16T15:39:21.566668+0000","last_unstale":"2024-09-16T15:39:23.714255+0000","last_undegraded":"2024-09-16T15:39:23.714255+0000","last_fullsized":"2024-09-16T15:39:23.714255+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:08:06.242307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360373+0000","last_change":"2024-09-16T15:39:22.569735+0000","last_active":"2024-09-16T15:39:24.360373+0000","last_peered":"2024-09-16T15:39:24.360373+0000","last_clean":"2024-09-16T15:39:24.360373+0000","last_became_active":"2024-09-16T15:39:22.569217+0000","last_became_peered":"2024-09-16T15:39:22.569217+0000","last_unstale":"2024-09-16T15:39:24.360373+0000","last_undegraded":"2024-09-16T15:39:24.360373+0000","last_fullsized":"2024-09-16T15:39:24.360373+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:34:12.122166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361075+0000","last_change":"2024-09-16T15:39:21.560934+0000","last_active":"2024-09-16T15:39:24.361075+0000","last_peered":"2024-09-16T15:39:24.361075+0000","last_clean":"2024-09-16T15:39:24.361075+0000","last_became_active":"2024-09-16T15:39:21.560690+0000","last_became_peered":"2024-09-16T15:39:21.560690+0000","last_unstale":"2024-09-16T15:39:24.361075+0000","last_undegraded":"2024-09-16T15:39:24.361075+0000","last_fullsized":"2024-09-16T15:39:24.361075+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:10:22.394137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360730+0000","last_change":"2024-09-16T15:39:22.569919+0000","last_active":"2024-09-16T15:39:24.360730+0000","last_peered":"2024-09-16T15:39:24.360730+0000","last_clean":"2024-09-16T15:39:24.360730+0000","last_became_active":"2024-09-16T15:39:22.569624+0000","last_became_peered":"2024-09-16T15:39:22.569624+0000","last_unstale":"2024-09-16T15:39:24.360730+0000","last_undegraded":"2024-09-16T15:39:24.360730+0000","last_fullsized":"2024-09-16T15:39:24.360730+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:42:11.842370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360437+0000","last_change":"2024-09-16T15:39:21.565143+0000","last_active":"2024-09-16T15:39:24.360437+0000","last_peered":"2024-09-16T15:39:24.360437+0000","last_clean":"2024-09-16T15:39:24.360437+0000","last_became_active":"2024-09-16T15:39:21.564887+0000","last_became_peered":"2024-09-16T15:39:21.564887+0000","last_unstale":"2024-09-16T15:39:24.360437+0000","last_undegraded":"2024-09-16T15:39:24.360437+0000","last_fullsized":"2024-09-16T15:39:24.360437+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:54:54.299318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.560542+0000","last_change":"2024-09-16T15:39:22.560025+0000","last_active":"2024-09-16T15:39:23.560542+0000","last_peered":"2024-09-16T15:39:23.560542+0000","last_clean":"2024-09-16T15:39:23.560542+0000","last_became_active":"2024-09-16T15:39:22.559563+0000","last_became_peered":"2024-09-16T15:39:22.559563+0000","last_unstale":"2024-09-16T15:39:23.560542+0000","last_undegraded":"2024-09-16T15:39:23.560542+0000","last_fullsized":"2024-09-16T15:39:23.560542+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:17:35.609828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.562702+0000","last_change":"2024-09-16T15:39:21.556186+0000","last_active":"2024-09-16T15:39:23.562702+0000","last_peered":"2024-09-16T15:39:23.562702+0000","last_clean":"2024-09-16T15:39:23.562702+0000","last_became_active":"2024-09-16T15:39:21.555847+0000","last_became_peered":"2024-09-16T15:39:21.555847+0000","last_unstale":"2024-09-16T15:39:23.562702+0000","last_undegraded":"2024-09-16T15:39:23.562702+0000","last_fullsized":"2024-09-16T15:39:23.562702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:43:58.109261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360215+0000","last_change":"2024-09-16T15:39:22.581265+0000","last_active":"2024-09-16T15:39:24.360215+0000","last_peered":"2024-09-16T15:39:24.360215+0000","last_clean":"2024-09-16T15:39:24.360215+0000","last_became_active":"2024-09-16T15:39:22.580808+0000","last_became_peered":"2024-09-16T15:39:22.580808+0000","last_unstale":"2024-09-16T15:39:24.360215+0000","last_undegraded":"2024-09-16T15:39:24.360215+0000","last_fullsized":"2024-09-16T15:39:24.360215+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:40:04.044401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.362115+0000","last_change":"2024-09-16T15:39:21.565669+0000","last_active":"2024-09-16T15:39:24.362115+0000","last_peered":"2024-09-16T15:39:24.362115+0000","last_clean":"2024-09-16T15:39:24.362115+0000","last_became_active":"2024-09-16T15:39:21.564058+0000","last_became_peered":"2024-09-16T15:39:21.564058+0000","last_unstale":"2024-09-16T15:39:24.362115+0000","last_undegraded":"2024-09-16T15:39:24.362115+0000","last_fullsized":"2024-09-16T15:39:24.362115+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:36:55.905341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.362680+0000","last_change":"2024-09-16T15:39:22.586068+0000","last_active":"2024-09-16T15:39:24.362680+0000","last_peered":"2024-09-16T15:39:24.362680+0000","last_clean":"2024-09-16T15:39:24.362680+0000","last_became_active":"2024-09-16T15:39:22.585745+0000","last_became_peered":"2024-09-16T15:39:22.585745+0000","last_unstale":"2024-09-16T15:39:24.362680+0000","last_undegraded":"2024-09-16T15:39:24.362680+0000","last_fullsized":"2024-09-16T15:39:24.362680+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:46:44.196294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.562702+0000","last_change":"2024-09-16T15:39:21.564692+0000","last_active":"2024-09-16T15:39:23.562702+0000","last_peered":"2024-09-16T15:39:23.562702+0000","last_clean":"2024-09-16T15:39:23.562702+0000","last_became_active":"2024-09-16T15:39:21.564414+0000","last_became_peered":"2024-09-16T15:39:21.564414+0000","last_unstale":"2024-09-16T15:39:23.562702+0000","last_undegraded":"2024-09-16T15:39:23.562702+0000","last_fullsized":"2024-09-16T15:39:23.562702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:46:18.513631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.362100+0000","last_change":"2024-09-16T15:39:22.560891+0000","last_active":"2024-09-16T15:39:24.362100+0000","last_peered":"2024-09-16T15:39:24.362100+0000","last_clean":"2024-09-16T15:39:24.362100+0000","last_became_active":"2024-09-16T15:39:22.560598+0000","last_became_peered":"2024-09-16T15:39:22.560598+0000","last_unstale":"2024-09-16T15:39:24.362100+0000","last_undegraded":"2024-09-16T15:39:24.362100+0000","last_fullsized":"2024-09-16T15:39:24.362100+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:57:51.336962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.563158+0000","last_change":"2024-09-16T15:39:21.559845+0000","last_active":"2024-09-16T15:39:23.563158+0000","last_peered":"2024-09-16T15:39:23.563158+0000","last_clean":"2024-09-16T15:39:23.563158+0000","last_became_active":"2024-09-16T15:39:21.559089+0000","last_became_peered":"2024-09-16T15:39:21.559089+0000","last_unstale":"2024-09-16T15:39:23.563158+0000","last_undegraded":"2024-09-16T15:39:23.563158+0000","last_fullsized":"2024-09-16T15:39:23.563158+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:40:57.508356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361096+0000","last_change":"2024-09-16T15:39:22.575906+0000","last_active":"2024-09-16T15:39:24.361096+0000","last_peered":"2024-09-16T15:39:24.361096+0000","last_clean":"2024-09-16T15:39:24.361096+0000","last_became_active":"2024-09-16T15:39:22.574839+0000","last_became_peered":"2024-09-16T15:39:22.574839+0000","last_unstale":"2024-09-16T15:39:24.361096+0000","last_undegraded":"2024-09-16T15:39:24.361096+0000","last_fullsized":"2024-09-16T15:39:24.361096+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:13:11.232766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360712+0000","last_change":"2024-09-16T15:39:21.565376+0000","last_active":"2024-09-16T15:39:24.360712+0000","last_peered":"2024-09-16T15:39:24.360712+0000","last_clean":"2024-09-16T15:39:24.360712+0000","last_became_active":"2024-09-16T15:39:21.565007+0000","last_became_peered":"2024-09-16T15:39:21.565007+0000","last_unstale":"2024-09-16T15:39:24.360712+0000","last_undegraded":"2024-09-16T15:39:24.360712+0000","last_fullsized":"2024-09-16T15:39:24.360712+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:39:19.838119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361711+0000","last_change":"2024-09-16T15:39:22.565594+0000","last_active":"2024-09-16T15:39:24.361711+0000","last_peered":"2024-09-16T15:39:24.361711+0000","last_clean":"2024-09-16T15:39:24.361711+0000","last_became_active":"2024-09-16T15:39:22.565255+0000","last_became_peered":"2024-09-16T15:39:22.565255+0000","last_unstale":"2024-09-16T15:39:24.361711+0000","last_undegraded":"2024-09-16T15:39:24.361711+0000","last_fullsized":"2024-09-16T15:39:24.361711+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:32.663886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.560438+0000","last_change":"2024-09-16T15:39:12.698813+0000","last_active":"2024-09-16T15:39:23.560438+0000","last_peered":"2024-09-16T15:39:23.560438+0000","last_clean":"2024-09-16T15:39:23.560438+0000","last_became_active":"2024-09-16T15:39:12.697695+0000","last_became_peered":"2024-09-16T15:39:12.697695+0000","last_unstale":"2024-09-16T15:39:23.560438+0000","last_undegraded":"2024-09-16T15:39:23.560438+0000","last_fullsized":"2024-09-16T15:39:23.560438+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_clean_scrub_stamp":"2024-09-16T15:39:11.684967+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:02.061081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360791+0000","last_change":"2024-09-16T15:39:21.565360+0000","last_active":"2024-09-16T15:39:24.360791+0000","last_peered":"2024-09-16T15:39:24.360791+0000","last_clean":"2024-09-16T15:39:24.360791+0000","last_became_active":"2024-09-16T15:39:21.565114+0000","last_became_peered":"2024-09-16T15:39:21.565114+0000","last_unstale":"2024-09-16T15:39:24.360791+0000","last_undegraded":"2024-09-16T15:39:24.360791+0000","last_fullsized":"2024-09-16T15:39:24.360791+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:40:57.304044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.562889+0000","last_change":"2024-09-16T15:39:22.579056+0000","last_active":"2024-09-16T15:39:23.562889+0000","last_peered":"2024-09-16T15:39:23.562889+0000","last_clean":"2024-09-16T15:39:23.562889+0000","last_became_active":"2024-09-16T15:39:22.576443+0000","last_became_peered":"2024-09-16T15:39:22.576443+0000","last_unstale":"2024-09-16T15:39:23.562889+0000","last_undegraded":"2024-09-16T15:39:23.562889+0000","last_fullsized":"2024-09-16T15:39:23.562889+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:56:07.665096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360279+0000","last_change":"2024-09-16T15:39:21.568792+0000","last_active":"2024-09-16T15:39:24.360279+0000","last_peered":"2024-09-16T15:39:24.360279+0000","last_clean":"2024-09-16T15:39:24.360279+0000","last_became_active":"2024-09-16T15:39:21.568534+0000","last_became_peered":"2024-09-16T15:39:21.568534+0000","last_unstale":"2024-09-16T15:39:24.360279+0000","last_undegraded":"2024-09-16T15:39:24.360279+0000","last_fullsized":"2024-09-16T15:39:24.360279+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:16:01.495612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360315+0000","last_change":"2024-09-16T15:39:22.572844+0000","last_active":"2024-09-16T15:39:24.360315+0000","last_peered":"2024-09-16T15:39:24.360315+0000","last_clean":"2024-09-16T15:39:24.360315+0000","last_became_active":"2024-09-16T15:39:22.570080+0000","last_became_peered":"2024-09-16T15:39:22.570080+0000","last_unstale":"2024-09-16T15:39:24.360315+0000","last_undegraded":"2024-09-16T15:39:24.360315+0000","last_fullsized":"2024-09-16T15:39:24.360315+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:19:39.018320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.714355+0000","last_change":"2024-09-16T15:39:21.563752+0000","last_active":"2024-09-16T15:39:23.714355+0000","last_peered":"2024-09-16T15:39:23.714355+0000","last_clean":"2024-09-16T15:39:23.714355+0000","last_became_active":"2024-09-16T15:39:21.563430+0000","last_became_peered":"2024-09-16T15:39:21.563430+0000","last_unstale":"2024-09-16T15:39:23.714355+0000","last_undegraded":"2024-09-16T15:39:23.714355+0000","last_fullsized":"2024-09-16T15:39:23.714355+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:44:53.582663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.562026+0000","last_change":"2024-09-16T15:39:22.560753+0000","last_active":"2024-09-16T15:39:23.562026+0000","last_peered":"2024-09-16T15:39:23.562026+0000","last_clean":"2024-09-16T15:39:23.562026+0000","last_became_active":"2024-09-16T15:39:22.560416+0000","last_became_peered":"2024-09-16T15:39:22.560416+0000","last_unstale":"2024-09-16T15:39:23.562026+0000","last_undegraded":"2024-09-16T15:39:23.562026+0000","last_fullsized":"2024-09-16T15:39:23.562026+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:30:53.438955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361167+0000","last_change":"2024-09-16T15:39:21.568766+0000","last_active":"2024-09-16T15:39:24.361167+0000","last_peered":"2024-09-16T15:39:24.361167+0000","last_clean":"2024-09-16T15:39:24.361167+0000","last_became_active":"2024-09-16T15:39:21.568515+0000","last_became_peered":"2024-09-16T15:39:21.568515+0000","last_unstale":"2024-09-16T15:39:24.361167+0000","last_undegraded":"2024-09-16T15:39:24.361167+0000","last_fullsized":"2024-09-16T15:39:24.361167+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:29:10.288957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361313+0000","last_change":"2024-09-16T15:39:22.581762+0000","last_active":"2024-09-16T15:39:24.361313+0000","last_peered":"2024-09-16T15:39:24.361313+0000","last_clean":"2024-09-16T15:39:24.361313+0000","last_became_active":"2024-09-16T15:39:22.572639+0000","last_became_peered":"2024-09-16T15:39:22.572639+0000","last_unstale":"2024-09-16T15:39:24.361313+0000","last_undegraded":"2024-09-16T15:39:24.361313+0000","last_fullsized":"2024-09-16T15:39:24.361313+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:06:34.579134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.562849+0000","last_change":"2024-09-16T15:39:21.561768+0000","last_active":"2024-09-16T15:39:23.562849+0000","last_peered":"2024-09-16T15:39:23.562849+0000","last_clean":"2024-09-16T15:39:23.562849+0000","last_became_active":"2024-09-16T15:39:21.561188+0000","last_became_peered":"2024-09-16T15:39:21.561188+0000","last_unstale":"2024-09-16T15:39:23.562849+0000","last_undegraded":"2024-09-16T15:39:23.562849+0000","last_fullsized":"2024-09-16T15:39:23.562849+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:39:19.838119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.362299+0000","last_change":"2024-09-16T15:39:22.578044+0000","last_active":"2024-09-16T15:39:24.362299+0000","last_peered":"2024-09-16T15:39:24.362299+0000","last_clean":"2024-09-16T15:39:24.362299+0000","last_became_active":"2024-09-16T15:39:22.577752+0000","last_became_peered":"2024-09-16T15:39:22.577752+0000","last_unstale":"2024-09-16T15:39:24.362299+0000","last_undegraded":"2024-09-16T15:39:24.362299+0000","last_fullsized":"2024-09-16T15:39:24.362299+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:36:16.988647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361983+0000","last_change":"2024-09-16T15:39:21.560508+0000","last_active":"2024-09-16T15:39:24.361983+0000","last_peered":"2024-09-16T15:39:24.361983+0000","last_clean":"2024-09-16T15:39:24.361983+0000","last_became_active":"2024-09-16T15:39:21.560082+0000","last_became_peered":"2024-09-16T15:39:21.560082+0000","last_unstale":"2024-09-16T15:39:24.361983+0000","last_undegraded":"2024-09-16T15:39:24.361983+0000","last_fullsized":"2024-09-16T15:39:24.361983+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:17:34.601873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360646+0000","last_change":"2024-09-16T15:39:22.581816+0000","last_active":"2024-09-16T15:39:24.360646+0000","last_peered":"2024-09-16T15:39:24.360646+0000","last_clean":"2024-09-16T15:39:24.360646+0000","last_became_active":"2024-09-16T15:39:22.581523+0000","last_became_peered":"2024-09-16T15:39:22.581523+0000","last_unstale":"2024-09-16T15:39:24.360646+0000","last_undegraded":"2024-09-16T15:39:24.360646+0000","last_fullsized":"2024-09-16T15:39:24.360646+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:57:08.823794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.562963+0000","last_change":"2024-09-16T15:39:21.564493+0000","last_active":"2024-09-16T15:39:23.562963+0000","last_peered":"2024-09-16T15:39:23.562963+0000","last_clean":"2024-09-16T15:39:23.562963+0000","last_became_active":"2024-09-16T15:39:21.564207+0000","last_became_peered":"2024-09-16T15:39:21.564207+0000","last_unstale":"2024-09-16T15:39:23.562963+0000","last_undegraded":"2024-09-16T15:39:23.562963+0000","last_fullsized":"2024-09-16T15:39:23.562963+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:17:34.601873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.562454+0000","last_change":"2024-09-16T15:39:22.577107+0000","last_active":"2024-09-16T15:39:23.562454+0000","last_peered":"2024-09-16T15:39:23.562454+0000","last_clean":"2024-09-16T15:39:23.562454+0000","last_became_active":"2024-09-16T15:39:22.576840+0000","last_became_peered":"2024-09-16T15:39:22.576840+0000","last_unstale":"2024-09-16T15:39:23.562454+0000","last_undegraded":"2024-09-16T15:39:23.562454+0000","last_fullsized":"2024-09-16T15:39:23.562454+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:30:53.438955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361083+0000","last_change":"2024-09-16T15:39:21.570398+0000","last_active":"2024-09-16T15:39:24.361083+0000","last_peered":"2024-09-16T15:39:24.361083+0000","last_clean":"2024-09-16T15:39:24.361083+0000","last_became_active":"2024-09-16T15:39:21.570020+0000","last_became_peered":"2024-09-16T15:39:21.570020+0000","last_unstale":"2024-09-16T15:39:24.361083+0000","last_undegraded":"2024-09-16T15:39:24.361083+0000","last_fullsized":"2024-09-16T15:39:24.361083+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:19:38.010365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.563030+0000","last_change":"2024-09-16T15:39:22.572370+0000","last_active":"2024-09-16T15:39:23.563030+0000","last_peered":"2024-09-16T15:39:23.563030+0000","last_clean":"2024-09-16T15:39:23.563030+0000","last_became_active":"2024-09-16T15:39:22.572114+0000","last_became_peered":"2024-09-16T15:39:22.572114+0000","last_unstale":"2024-09-16T15:39:23.563030+0000","last_undegraded":"2024-09-16T15:39:23.563030+0000","last_fullsized":"2024-09-16T15:39:23.563030+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:22:34.502460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360218+0000","last_change":"2024-09-16T15:39:21.561676+0000","last_active":"2024-09-16T15:39:24.360218+0000","last_peered":"2024-09-16T15:39:24.360218+0000","last_clean":"2024-09-16T15:39:24.360218+0000","last_became_active":"2024-09-16T15:39:21.560798+0000","last_became_peered":"2024-09-16T15:39:21.560798+0000","last_unstale":"2024-09-16T15:39:24.360218+0000","last_undegraded":"2024-09-16T15:39:24.360218+0000","last_fullsized":"2024-09-16T15:39:24.360218+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:12:34.622210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360594+0000","last_change":"2024-09-16T15:39:22.567438+0000","last_active":"2024-09-16T15:39:24.360594+0000","last_peered":"2024-09-16T15:39:24.360594+0000","last_clean":"2024-09-16T15:39:24.360594+0000","last_became_active":"2024-09-16T15:39:22.567191+0000","last_became_peered":"2024-09-16T15:39:22.567191+0000","last_unstale":"2024-09-16T15:39:24.360594+0000","last_undegraded":"2024-09-16T15:39:24.360594+0000","last_fullsized":"2024-09-16T15:39:24.360594+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:42:25.111846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.561951+0000","last_change":"2024-09-16T15:39:21.566726+0000","last_active":"2024-09-16T15:39:23.561951+0000","last_peered":"2024-09-16T15:39:23.561951+0000","last_clean":"2024-09-16T15:39:23.561951+0000","last_became_active":"2024-09-16T15:39:21.566406+0000","last_became_peered":"2024-09-16T15:39:21.566406+0000","last_unstale":"2024-09-16T15:39:23.561951+0000","last_undegraded":"2024-09-16T15:39:23.561951+0000","last_fullsized":"2024-09-16T15:39:23.561951+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:40:57.508356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.561742+0000","last_change":"2024-09-16T15:39:22.575174+0000","last_active":"2024-09-16T15:39:23.561742+0000","last_peered":"2024-09-16T15:39:23.561742+0000","last_clean":"2024-09-16T15:39:23.561742+0000","last_became_active":"2024-09-16T15:39:22.574865+0000","last_became_peered":"2024-09-16T15:39:22.574865+0000","last_unstale":"2024-09-16T15:39:23.561742+0000","last_undegraded":"2024-09-16T15:39:23.561742+0000","last_fullsized":"2024-09-16T15:39:23.561742+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:10:23.402092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360483+0000","last_change":"2024-09-16T15:39:21.561384+0000","last_active":"2024-09-16T15:39:24.360483+0000","last_peered":"2024-09-16T15:39:24.360483+0000","last_clean":"2024-09-16T15:39:24.360483+0000","last_became_active":"2024-09-16T15:39:21.561078+0000","last_became_peered":"2024-09-16T15:39:21.561078+0000","last_unstale":"2024-09-16T15:39:24.360483+0000","last_undegraded":"2024-09-16T15:39:24.360483+0000","last_fullsized":"2024-09-16T15:39:24.360483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:10:40.788797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.714500+0000","last_change":"2024-09-16T15:39:22.576232+0000","last_active":"2024-09-16T15:39:23.714500+0000","last_peered":"2024-09-16T15:39:23.714500+0000","last_clean":"2024-09-16T15:39:23.714500+0000","last_became_active":"2024-09-16T15:39:22.575962+0000","last_became_peered":"2024-09-16T15:39:22.575962+0000","last_unstale":"2024-09-16T15:39:23.714500+0000","last_undegraded":"2024-09-16T15:39:23.714500+0000","last_fullsized":"2024-09-16T15:39:23.714500+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:43:58.162669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360704+0000","last_change":"2024-09-16T15:39:21.566471+0000","last_active":"2024-09-16T15:39:24.360704+0000","last_peered":"2024-09-16T15:39:24.360704+0000","last_clean":"2024-09-16T15:39:24.360704+0000","last_became_active":"2024-09-16T15:39:21.566220+0000","last_became_peered":"2024-09-16T15:39:21.566220+0000","last_unstale":"2024-09-16T15:39:24.360704+0000","last_undegraded":"2024-09-16T15:39:24.360704+0000","last_fullsized":"2024-09-16T15:39:24.360704+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:34:11.114212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360894+0000","last_change":"2024-09-16T15:39:22.565878+0000","last_active":"2024-09-16T15:39:24.360894+0000","last_peered":"2024-09-16T15:39:24.360894+0000","last_clean":"2024-09-16T15:39:24.360894+0000","last_became_active":"2024-09-16T15:39:22.565599+0000","last_became_peered":"2024-09-16T15:39:22.565599+0000","last_unstale":"2024-09-16T15:39:24.360894+0000","last_undegraded":"2024-09-16T15:39:24.360894+0000","last_fullsized":"2024-09-16T15:39:24.360894+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:04:51.595786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361501+0000","last_change":"2024-09-16T15:39:22.563299+0000","last_active":"2024-09-16T15:39:24.361501+0000","last_peered":"2024-09-16T15:39:24.361501+0000","last_clean":"2024-09-16T15:39:24.361501+0000","last_became_active":"2024-09-16T15:39:22.560515+0000","last_became_peered":"2024-09-16T15:39:22.560515+0000","last_unstale":"2024-09-16T15:39:24.361501+0000","last_undegraded":"2024-09-16T15:39:24.361501+0000","last_fullsized":"2024-09-16T15:39:24.361501+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:45:21.853848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361156+0000","last_change":"2024-09-16T15:39:21.561779+0000","last_active":"2024-09-16T15:39:24.361156+0000","last_peered":"2024-09-16T15:39:24.361156+0000","last_clean":"2024-09-16T15:39:24.361156+0000","last_became_active":"2024-09-16T15:39:21.561471+0000","last_became_peered":"2024-09-16T15:39:21.561471+0000","last_unstale":"2024-09-16T15:39:24.361156+0000","last_undegraded":"2024-09-16T15:39:24.361156+0000","last_fullsized":"2024-09-16T15:39:24.361156+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:29:24.601704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.560906+0000","last_change":"2024-09-16T15:39:22.565324+0000","last_active":"2024-09-16T15:39:23.560906+0000","last_peered":"2024-09-16T15:39:23.560906+0000","last_clean":"2024-09-16T15:39:23.560906+0000","last_became_active":"2024-09-16T15:39:22.564275+0000","last_became_peered":"2024-09-16T15:39:22.564275+0000","last_unstale":"2024-09-16T15:39:23.560906+0000","last_undegraded":"2024-09-16T15:39:23.560906+0000","last_fullsized":"2024-09-16T15:39:23.560906+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:16:02.503567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361469+0000","last_change":"2024-09-16T15:39:21.563348+0000","last_active":"2024-09-16T15:39:24.361469+0000","last_peered":"2024-09-16T15:39:24.361469+0000","last_clean":"2024-09-16T15:39:24.361469+0000","last_became_active":"2024-09-16T15:39:21.560699+0000","last_became_peered":"2024-09-16T15:39:21.560699+0000","last_unstale":"2024-09-16T15:39:24.361469+0000","last_undegraded":"2024-09-16T15:39:24.361469+0000","last_fullsized":"2024-09-16T15:39:24.361469+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:36:15.980692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.563361+0000","last_change":"2024-09-16T15:39:22.558509+0000","last_active":"2024-09-16T15:39:23.563361+0000","last_peered":"2024-09-16T15:39:23.563361+0000","last_clean":"2024-09-16T15:39:23.563361+0000","last_became_active":"2024-09-16T15:39:22.558197+0000","last_became_peered":"2024-09-16T15:39:22.558197+0000","last_unstale":"2024-09-16T15:39:23.563361+0000","last_undegraded":"2024-09-16T15:39:23.563361+0000","last_fullsized":"2024-09-16T15:39:23.563361+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:10:15.420114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.563141+0000","last_change":"2024-09-16T15:39:21.561287+0000","last_active":"2024-09-16T15:39:23.563141+0000","last_peered":"2024-09-16T15:39:23.563141+0000","last_clean":"2024-09-16T15:39:23.563141+0000","last_became_active":"2024-09-16T15:39:21.559935+0000","last_became_peered":"2024-09-16T15:39:21.559935+0000","last_unstale":"2024-09-16T15:39:23.563141+0000","last_undegraded":"2024-09-16T15:39:23.563141+0000","last_fullsized":"2024-09-16T15:39:23.563141+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:19:57.274084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.563208+0000","last_change":"2024-09-16T15:39:22.576005+0000","last_active":"2024-09-16T15:39:23.563208+0000","last_peered":"2024-09-16T15:39:23.563208+0000","last_clean":"2024-09-16T15:39:23.563208+0000","last_became_active":"2024-09-16T15:39:22.575434+0000","last_became_peered":"2024-09-16T15:39:22.575434+0000","last_unstale":"2024-09-16T15:39:23.563208+0000","last_undegraded":"2024-09-16T15:39:23.563208+0000","last_fullsized":"2024-09-16T15:39:23.563208+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:03:51.340088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.562531+0000","last_change":"2024-09-16T15:39:21.568944+0000","last_active":"2024-09-16T15:39:23.562531+0000","last_peered":"2024-09-16T15:39:23.562531+0000","last_clean":"2024-09-16T15:39:23.562531+0000","last_became_active":"2024-09-16T15:39:21.568706+0000","last_became_peered":"2024-09-16T15:39:21.568706+0000","last_unstale":"2024-09-16T15:39:23.562531+0000","last_undegraded":"2024-09-16T15:39:23.562531+0000","last_fullsized":"2024-09-16T15:39:23.562531+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:19:57.274084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509447,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":76284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":823296,"data_stored":666744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":241664,"data_stored":78568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":225280,"data_stored":76866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509445,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":233472,"data_stored":77896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509446,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":819200,"data_stored":666722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509444,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":76340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542151,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":76408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":27654604},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:39:31.455 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:39:31.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:31.624+0000 7ff86b600640 1 -- 172.21.15.151:0/3575998720 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff8640af3d0 msgr2=0x7ff86406f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:31.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:31.624+0000 7ff86b600640 1 --2- 172.21.15.151:0/3575998720 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff8640af3d0 0x7ff86406f660 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff8640b4890 tx=0x7ff85002a4a0 comp rx=0 tx=0).stop 2024-09-16T15:39:31.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:31.624+0000 7ff86b600640 1 -- 172.21.15.151:0/3575998720 shutdown_connections 2024-09-16T15:39:31.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:31.624+0000 7ff86b600640 1 --2- 172.21.15.151:0/3575998720 >> v2:172.21.15.167:3300/0 conn(0x7ff864072df0 0x7ff864073240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:31.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:31.624+0000 7ff86b600640 1 --2- 172.21.15.151:0/3575998720 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff8640af3d0 0x7ff86406f660 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:31.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:31.624+0000 7ff86b600640 1 -- 172.21.15.151:0/3575998720 >> 172.21.15.151:0/3575998720 conn(0x7ff8640b1950 msgr2=0x7ff8640b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:31.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:31.624+0000 7ff86b600640 1 -- 172.21.15.151:0/3575998720 shutdown_connections 2024-09-16T15:39:31.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:31.624+0000 7ff86b600640 1 -- 172.21.15.151:0/3575998720 wait complete. 2024-09-16T15:39:31.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:31.626+0000 7ff86b600640 1 Processor -- start 2024-09-16T15:39:31.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:31.626+0000 7ff86b600640 1 -- start start 2024-09-16T15:39:31.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:31.627+0000 7ff86b600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff864072df0 0x7ff8641f5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:31.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:31.627+0000 7ff86b600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff8640af3d0 0x7ff8641f7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:31.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:31.627+0000 7ff86b600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff864161550 0x7ff864146d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:31.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:31.627+0000 7ff86b600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff864073780 con 0x7ff864072df0 2024-09-16T15:39:31.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:31.627+0000 7ff86b600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff86406fdb0 con 0x7ff864161550 2024-09-16T15:39:31.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:31.627+0000 7ff86b600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7ff864073900 con 0x7ff8640af3d0 2024-09-16T15:39:31.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:31.627+0000 7ff868c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff864072df0 0x7ff8641f5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:31.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:31.627+0000 7ff868c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff864072df0 0x7ff8641f5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47342/0 (socket says 172.21.15.151:47342) 2024-09-16T15:39:31.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:31.627+0000 7ff868c00640 1 -- 172.21.15.151:0/4090830203 learned_addr learned my addr 172.21.15.151:0/4090830203 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:31.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:31.627+0000 7ff863e00640 1 --2- 172.21.15.151:0/4090830203 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff8640af3d0 0x7ff8641f7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:31.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:31.627+0000 7ff869600640 1 --2- 172.21.15.151:0/4090830203 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff864161550 0x7ff864146d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:31.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:31.628+0000 7ff868c00640 1 -- 172.21.15.151:0/4090830203 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff8640af3d0 msgr2=0x7ff8641f7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:31.630 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:31.628+0000 7ff868c00640 1 --2- 172.21.15.151:0/4090830203 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff8640af3d0 0x7ff8641f7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:31.630 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:31.628+0000 7ff868c00640 1 -- 172.21.15.151:0/4090830203 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff864161550 msgr2=0x7ff864146d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:31.630 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:31.628+0000 7ff868c00640 1 --2- 172.21.15.151:0/4090830203 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff864161550 0x7ff864146d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:31.630 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:31.628+0000 7ff868c00640 1 -- 172.21.15.151:0/4090830203 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8641f8130 con 0x7ff864072df0 2024-09-16T15:39:31.630 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:31.628+0000 7ff868c00640 1 --2- 172.21.15.151:0/4090830203 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff864072df0 0x7ff8641f5fa0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff85400c7a0 tx=0x7ff85400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:31.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:31.628+0000 7ff860c00640 1 -- 172.21.15.151:0/4090830203 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff854017070 con 0x7ff864072df0 2024-09-16T15:39:31.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:31.628+0000 7ff860c00640 1 -- 172.21.15.151:0/4090830203 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff854005070 con 0x7ff864072df0 2024-09-16T15:39:31.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:31.628+0000 7ff860c00640 1 -- 172.21.15.151:0/4090830203 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8540054a0 con 0x7ff864072df0 2024-09-16T15:39:31.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:31.628+0000 7ff86b600640 1 -- 172.21.15.151:0/4090830203 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff86415b170 con 0x7ff864072df0 2024-09-16T15:39:31.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:31.628+0000 7ff86b600640 1 -- 172.21.15.151:0/4090830203 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8641552d0 con 0x7ff864072df0 2024-09-16T15:39:31.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:31.629+0000 7ff86b600640 1 -- 172.21.15.151:0/4090830203 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8640b3310 con 0x7ff864072df0 2024-09-16T15:39:31.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:31.630+0000 7ff860c00640 1 -- 172.21.15.151:0/4090830203 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7ff85401a070 con 0x7ff864072df0 2024-09-16T15:39:31.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:31.631+0000 7ff860c00640 1 --2- 172.21.15.151:0/4090830203 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff844078880 0x7ff84407ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:31.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:31.631+0000 7ff860c00640 1 -- 172.21.15.151:0/4090830203 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7ff854099f40 con 0x7ff864072df0 2024-09-16T15:39:31.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:31.632+0000 7ff863e00640 1 --2- 172.21.15.151:0/4090830203 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff844078880 0x7ff84407ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:31.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:31.632+0000 7ff863e00640 1 --2- 172.21.15.151:0/4090830203 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff844078880 0x7ff84407ad40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff850034000 tx=0x7ff850004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:31.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:31.635+0000 7ff860c00640 1 -- 172.21.15.151:0/4090830203 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff8640b3310 con 0x7ff864072df0 2024-09-16T15:39:31.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:31.816+0000 7ff86b600640 1 -- 172.21.15.151:0/4090830203 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff8640cd4f0 con 0x7ff844078880 2024-09-16T15:39:31.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:31.821+0000 7ff860c00640 1 -- 172.21.15.151:0/4090830203 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251821 (secure 0 0 0) 0x7ff8640cd4f0 con 0x7ff844078880 2024-09-16T15:39:31.825 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:39:31.826 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-09-16T15:39:31.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:31.825+0000 7ff86b600640 1 -- 172.21.15.151:0/4090830203 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff844078880 msgr2=0x7ff84407ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:31.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:31.825+0000 7ff86b600640 1 --2- 172.21.15.151:0/4090830203 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff844078880 0x7ff84407ad40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff850034000 tx=0x7ff850004730 comp rx=0 tx=0).stop 2024-09-16T15:39:31.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:31.825+0000 7ff86b600640 1 -- 172.21.15.151:0/4090830203 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff864072df0 msgr2=0x7ff8641f5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:31.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:31.825+0000 7ff86b600640 1 --2- 172.21.15.151:0/4090830203 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff864072df0 0x7ff8641f5fa0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff85400c7a0 tx=0x7ff85400cc60 comp rx=0 tx=0).stop 2024-09-16T15:39:31.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:31.825+0000 7ff86b600640 1 -- 172.21.15.151:0/4090830203 shutdown_connections 2024-09-16T15:39:31.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:31.825+0000 7ff86b600640 1 --2- 172.21.15.151:0/4090830203 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff844078880 0x7ff84407ad40 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:31.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:31.825+0000 7ff86b600640 1 --2- 172.21.15.151:0/4090830203 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff864161550 0x7ff864146d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:31.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:31.825+0000 7ff86b600640 1 --2- 172.21.15.151:0/4090830203 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff8640af3d0 0x7ff8641f7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:31.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:31.825+0000 7ff86b600640 1 --2- 172.21.15.151:0/4090830203 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff864072df0 0x7ff8641f5fa0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:31.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:31.825+0000 7ff86b600640 1 -- 172.21.15.151:0/4090830203 >> 172.21.15.151:0/4090830203 conn(0x7ff8640b1950 msgr2=0x7ff8640ad4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:31.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:31.825+0000 7ff86b600640 1 -- 172.21.15.151:0/4090830203 shutdown_connections 2024-09-16T15:39:31.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:31.825+0000 7ff86b600640 1 -- 172.21.15.151:0/4090830203 wait complete. 2024-09-16T15:39:31.842 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-09-16T15:39:31.578619+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219184,"kb_used_data":2952,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755536,"statfs":{"total":773094113280,"available":772869668864,"internally_reserved":0,"allocated":3022848,"data_stored":1795828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12738,"internal_metadata":221236798},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.035193"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.362052+0000","last_change":"2024-09-16T15:39:22.577878+0000","last_active":"2024-09-16T15:39:24.362052+0000","last_peered":"2024-09-16T15:39:24.362052+0000","last_clean":"2024-09-16T15:39:24.362052+0000","last_became_active":"2024-09-16T15:39:22.570201+0000","last_became_peered":"2024-09-16T15:39:22.570201+0000","last_unstale":"2024-09-16T15:39:24.362052+0000","last_undegraded":"2024-09-16T15:39:24.362052+0000","last_fullsized":"2024-09-16T15:39:24.362052+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:39:49.705970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.563074+0000","last_change":"2024-09-16T15:39:22.568365+0000","last_active":"2024-09-16T15:39:23.563074+0000","last_peered":"2024-09-16T15:39:23.563074+0000","last_clean":"2024-09-16T15:39:23.563074+0000","last_became_active":"2024-09-16T15:39:22.564695+0000","last_became_peered":"2024-09-16T15:39:22.564695+0000","last_unstale":"2024-09-16T15:39:23.563074+0000","last_undegraded":"2024-09-16T15:39:23.563074+0000","last_fullsized":"2024-09-16T15:39:23.563074+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:18:39.431491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.563192+0000","last_change":"2024-09-16T15:39:22.575183+0000","last_active":"2024-09-16T15:39:23.563192+0000","last_peered":"2024-09-16T15:39:23.563192+0000","last_clean":"2024-09-16T15:39:23.563192+0000","last_became_active":"2024-09-16T15:39:22.574916+0000","last_became_peered":"2024-09-16T15:39:22.574916+0000","last_unstale":"2024-09-16T15:39:23.563192+0000","last_undegraded":"2024-09-16T15:39:23.563192+0000","last_fullsized":"2024-09-16T15:39:23.563192+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:06:34.579134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.362360+0000","last_change":"2024-09-16T15:39:22.580108+0000","last_active":"2024-09-16T15:39:24.362360+0000","last_peered":"2024-09-16T15:39:24.362360+0000","last_clean":"2024-09-16T15:39:24.362360+0000","last_became_active":"2024-09-16T15:39:22.579859+0000","last_became_peered":"2024-09-16T15:39:22.579859+0000","last_unstale":"2024-09-16T15:39:24.362360+0000","last_undegraded":"2024-09-16T15:39:24.362360+0000","last_fullsized":"2024-09-16T15:39:24.362360+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:32:44.877620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361299+0000","last_change":"2024-09-16T15:39:22.566468+0000","last_active":"2024-09-16T15:39:24.361299+0000","last_peered":"2024-09-16T15:39:24.361299+0000","last_clean":"2024-09-16T15:39:24.361299+0000","last_became_active":"2024-09-16T15:39:22.559581+0000","last_became_peered":"2024-09-16T15:39:22.559581+0000","last_unstale":"2024-09-16T15:39:24.361299+0000","last_undegraded":"2024-09-16T15:39:24.361299+0000","last_fullsized":"2024-09-16T15:39:24.361299+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:35:04.935435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361543+0000","last_change":"2024-09-16T15:39:22.579964+0000","last_active":"2024-09-16T15:39:24.361543+0000","last_peered":"2024-09-16T15:39:24.361543+0000","last_clean":"2024-09-16T15:39:24.361543+0000","last_became_active":"2024-09-16T15:39:22.579714+0000","last_became_peered":"2024-09-16T15:39:22.579714+0000","last_unstale":"2024-09-16T15:39:24.361543+0000","last_undegraded":"2024-09-16T15:39:24.361543+0000","last_fullsized":"2024-09-16T15:39:24.361543+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:54:55.307273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360552+0000","last_change":"2024-09-16T15:39:22.578008+0000","last_active":"2024-09-16T15:39:24.360552+0000","last_peered":"2024-09-16T15:39:24.360552+0000","last_clean":"2024-09-16T15:39:24.360552+0000","last_became_active":"2024-09-16T15:39:22.577770+0000","last_became_peered":"2024-09-16T15:39:22.577770+0000","last_unstale":"2024-09-16T15:39:24.360552+0000","last_undegraded":"2024-09-16T15:39:24.360552+0000","last_fullsized":"2024-09-16T15:39:24.360552+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:03:05.655450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.562837+0000","last_change":"2024-09-16T15:39:22.565821+0000","last_active":"2024-09-16T15:39:23.562837+0000","last_peered":"2024-09-16T15:39:23.562837+0000","last_clean":"2024-09-16T15:39:23.562837+0000","last_became_active":"2024-09-16T15:39:22.564114+0000","last_became_peered":"2024-09-16T15:39:22.564114+0000","last_unstale":"2024-09-16T15:39:23.562837+0000","last_undegraded":"2024-09-16T15:39:23.562837+0000","last_fullsized":"2024-09-16T15:39:23.562837+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:03:28.558231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.561256+0000","last_change":"2024-09-16T15:39:22.557568+0000","last_active":"2024-09-16T15:39:23.561256+0000","last_peered":"2024-09-16T15:39:23.561256+0000","last_clean":"2024-09-16T15:39:23.561256+0000","last_became_active":"2024-09-16T15:39:22.556305+0000","last_became_peered":"2024-09-16T15:39:22.556305+0000","last_unstale":"2024-09-16T15:39:23.561256+0000","last_undegraded":"2024-09-16T15:39:23.561256+0000","last_fullsized":"2024-09-16T15:39:23.561256+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:19:39.018320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361933+0000","last_change":"2024-09-16T15:39:22.573944+0000","last_active":"2024-09-16T15:39:24.361933+0000","last_peered":"2024-09-16T15:39:24.361933+0000","last_clean":"2024-09-16T15:39:24.361933+0000","last_became_active":"2024-09-16T15:39:22.573688+0000","last_became_peered":"2024-09-16T15:39:22.573688+0000","last_unstale":"2024-09-16T15:39:24.361933+0000","last_undegraded":"2024-09-16T15:39:24.361933+0000","last_fullsized":"2024-09-16T15:39:24.361933+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:00:05.185794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360929+0000","last_change":"2024-09-16T15:39:22.580084+0000","last_active":"2024-09-16T15:39:24.360929+0000","last_peered":"2024-09-16T15:39:24.360929+0000","last_clean":"2024-09-16T15:39:24.360929+0000","last_became_active":"2024-09-16T15:39:22.571878+0000","last_became_peered":"2024-09-16T15:39:22.571878+0000","last_unstale":"2024-09-16T15:39:24.360929+0000","last_undegraded":"2024-09-16T15:39:24.360929+0000","last_fullsized":"2024-09-16T15:39:24.360929+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:46:31.911518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361633+0000","last_change":"2024-09-16T15:39:22.575436+0000","last_active":"2024-09-16T15:39:24.361633+0000","last_peered":"2024-09-16T15:39:24.361633+0000","last_clean":"2024-09-16T15:39:24.361633+0000","last_became_active":"2024-09-16T15:39:22.568528+0000","last_became_peered":"2024-09-16T15:39:22.568528+0000","last_unstale":"2024-09-16T15:39:24.361633+0000","last_undegraded":"2024-09-16T15:39:24.361633+0000","last_fullsized":"2024-09-16T15:39:24.361633+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:05:28.120874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360645+0000","last_change":"2024-09-16T15:39:22.566575+0000","last_active":"2024-09-16T15:39:24.360645+0000","last_peered":"2024-09-16T15:39:24.360645+0000","last_clean":"2024-09-16T15:39:24.360645+0000","last_became_active":"2024-09-16T15:39:22.566295+0000","last_became_peered":"2024-09-16T15:39:22.566295+0000","last_unstale":"2024-09-16T15:39:24.360645+0000","last_undegraded":"2024-09-16T15:39:24.360645+0000","last_fullsized":"2024-09-16T15:39:24.360645+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:10:23.402092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360395+0000","last_change":"2024-09-16T15:39:22.571476+0000","last_active":"2024-09-16T15:39:24.360395+0000","last_peered":"2024-09-16T15:39:24.360395+0000","last_clean":"2024-09-16T15:39:24.360395+0000","last_became_active":"2024-09-16T15:39:22.571216+0000","last_became_peered":"2024-09-16T15:39:22.571216+0000","last_unstale":"2024-09-16T15:39:24.360395+0000","last_undegraded":"2024-09-16T15:39:24.360395+0000","last_fullsized":"2024-09-16T15:39:24.360395+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:40:58.311998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.563103+0000","last_change":"2024-09-16T15:39:22.564630+0000","last_active":"2024-09-16T15:39:23.563103+0000","last_peered":"2024-09-16T15:39:23.563103+0000","last_clean":"2024-09-16T15:39:23.563103+0000","last_became_active":"2024-09-16T15:39:22.564326+0000","last_became_peered":"2024-09-16T15:39:22.564326+0000","last_unstale":"2024-09-16T15:39:23.563103+0000","last_undegraded":"2024-09-16T15:39:23.563103+0000","last_fullsized":"2024-09-16T15:39:23.563103+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:58:23.963120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360193+0000","last_change":"2024-09-16T15:39:22.574288+0000","last_active":"2024-09-16T15:39:24.360193+0000","last_peered":"2024-09-16T15:39:24.360193+0000","last_clean":"2024-09-16T15:39:24.360193+0000","last_became_active":"2024-09-16T15:39:22.573979+0000","last_became_peered":"2024-09-16T15:39:22.573979+0000","last_unstale":"2024-09-16T15:39:24.360193+0000","last_undegraded":"2024-09-16T15:39:24.360193+0000","last_fullsized":"2024-09-16T15:39:24.360193+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:10:15.420114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361109+0000","last_change":"2024-09-16T15:39:22.561784+0000","last_active":"2024-09-16T15:39:24.361109+0000","last_peered":"2024-09-16T15:39:24.361109+0000","last_clean":"2024-09-16T15:39:24.361109+0000","last_became_active":"2024-09-16T15:39:22.561487+0000","last_became_peered":"2024-09-16T15:39:22.561487+0000","last_unstale":"2024-09-16T15:39:24.361109+0000","last_undegraded":"2024-09-16T15:39:24.361109+0000","last_fullsized":"2024-09-16T15:39:24.361109+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:26:52.642854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.714434+0000","last_change":"2024-09-16T15:39:22.579035+0000","last_active":"2024-09-16T15:39:23.714434+0000","last_peered":"2024-09-16T15:39:23.714434+0000","last_clean":"2024-09-16T15:39:23.714434+0000","last_became_active":"2024-09-16T15:39:22.578734+0000","last_became_peered":"2024-09-16T15:39:22.578734+0000","last_unstale":"2024-09-16T15:39:23.714434+0000","last_undegraded":"2024-09-16T15:39:23.714434+0000","last_fullsized":"2024-09-16T15:39:23.714434+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:13:19.369864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361928+0000","last_change":"2024-09-16T15:39:22.577483+0000","last_active":"2024-09-16T15:39:24.361928+0000","last_peered":"2024-09-16T15:39:24.361928+0000","last_clean":"2024-09-16T15:39:24.361928+0000","last_became_active":"2024-09-16T15:39:22.577242+0000","last_became_peered":"2024-09-16T15:39:22.577242+0000","last_unstale":"2024-09-16T15:39:24.361928+0000","last_undegraded":"2024-09-16T15:39:24.361928+0000","last_fullsized":"2024-09-16T15:39:24.361928+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:29:43.879706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360912+0000","last_change":"2024-09-16T15:39:22.561587+0000","last_active":"2024-09-16T15:39:24.360912+0000","last_peered":"2024-09-16T15:39:24.360912+0000","last_clean":"2024-09-16T15:39:24.360912+0000","last_became_active":"2024-09-16T15:39:22.561213+0000","last_became_peered":"2024-09-16T15:39:22.561213+0000","last_unstale":"2024-09-16T15:39:24.360912+0000","last_undegraded":"2024-09-16T15:39:24.360912+0000","last_fullsized":"2024-09-16T15:39:24.360912+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:00:48.581056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.562380+0000","last_change":"2024-09-16T15:39:22.579126+0000","last_active":"2024-09-16T15:39:23.562380+0000","last_peered":"2024-09-16T15:39:23.562380+0000","last_clean":"2024-09-16T15:39:23.562380+0000","last_became_active":"2024-09-16T15:39:22.577915+0000","last_became_peered":"2024-09-16T15:39:22.577915+0000","last_unstale":"2024-09-16T15:39:23.562380+0000","last_undegraded":"2024-09-16T15:39:23.562380+0000","last_fullsized":"2024-09-16T15:39:23.562380+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:18:15.980220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.560783+0000","last_change":"2024-09-16T15:39:22.560525+0000","last_active":"2024-09-16T15:39:23.560783+0000","last_peered":"2024-09-16T15:39:23.560783+0000","last_clean":"2024-09-16T15:39:23.560783+0000","last_became_active":"2024-09-16T15:39:22.560279+0000","last_became_peered":"2024-09-16T15:39:22.560279+0000","last_unstale":"2024-09-16T15:39:23.560783+0000","last_undegraded":"2024-09-16T15:39:23.560783+0000","last_fullsized":"2024-09-16T15:39:23.560783+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:22:24.884519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.562668+0000","last_change":"2024-09-16T15:39:22.567598+0000","last_active":"2024-09-16T15:39:23.562668+0000","last_peered":"2024-09-16T15:39:23.562668+0000","last_clean":"2024-09-16T15:39:23.562668+0000","last_became_active":"2024-09-16T15:39:22.567356+0000","last_became_peered":"2024-09-16T15:39:22.567356+0000","last_unstale":"2024-09-16T15:39:23.562668+0000","last_undegraded":"2024-09-16T15:39:23.562668+0000","last_fullsized":"2024-09-16T15:39:23.562668+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:22:34.502460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360199+0000","last_change":"2024-09-16T15:39:22.558806+0000","last_active":"2024-09-16T15:39:24.360199+0000","last_peered":"2024-09-16T15:39:24.360199+0000","last_clean":"2024-09-16T15:39:24.360199+0000","last_became_active":"2024-09-16T15:39:22.558473+0000","last_became_peered":"2024-09-16T15:39:22.558473+0000","last_unstale":"2024-09-16T15:39:24.360199+0000","last_undegraded":"2024-09-16T15:39:24.360199+0000","last_fullsized":"2024-09-16T15:39:24.360199+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:05:28.120874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.563124+0000","last_change":"2024-09-16T15:39:22.564434+0000","last_active":"2024-09-16T15:39:23.563124+0000","last_peered":"2024-09-16T15:39:23.563124+0000","last_clean":"2024-09-16T15:39:23.563124+0000","last_became_active":"2024-09-16T15:39:22.563970+0000","last_became_peered":"2024-09-16T15:39:22.563970+0000","last_unstale":"2024-09-16T15:39:23.563124+0000","last_undegraded":"2024-09-16T15:39:23.563124+0000","last_fullsized":"2024-09-16T15:39:23.563124+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:43:27.368913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.362535+0000","last_change":"2024-09-16T15:39:22.584858+0000","last_active":"2024-09-16T15:39:24.362535+0000","last_peered":"2024-09-16T15:39:24.362535+0000","last_clean":"2024-09-16T15:39:24.362535+0000","last_became_active":"2024-09-16T15:39:22.584152+0000","last_became_peered":"2024-09-16T15:39:22.584152+0000","last_unstale":"2024-09-16T15:39:24.362535+0000","last_undegraded":"2024-09-16T15:39:24.362535+0000","last_fullsized":"2024-09-16T15:39:24.362535+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:08:07.935905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360764+0000","last_change":"2024-09-16T15:39:22.573867+0000","last_active":"2024-09-16T15:39:24.360764+0000","last_peered":"2024-09-16T15:39:24.360764+0000","last_clean":"2024-09-16T15:39:24.360764+0000","last_became_active":"2024-09-16T15:39:22.573567+0000","last_became_peered":"2024-09-16T15:39:22.573567+0000","last_unstale":"2024-09-16T15:39:24.360764+0000","last_undegraded":"2024-09-16T15:39:24.360764+0000","last_fullsized":"2024-09-16T15:39:24.360764+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:18:15.980220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360545+0000","last_change":"2024-09-16T15:39:22.574268+0000","last_active":"2024-09-16T15:39:24.360545+0000","last_peered":"2024-09-16T15:39:24.360545+0000","last_clean":"2024-09-16T15:39:24.360545+0000","last_became_active":"2024-09-16T15:39:22.572891+0000","last_became_peered":"2024-09-16T15:39:22.572891+0000","last_unstale":"2024-09-16T15:39:24.360545+0000","last_undegraded":"2024-09-16T15:39:24.360545+0000","last_fullsized":"2024-09-16T15:39:24.360545+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:29:11.296912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.562547+0000","last_change":"2024-09-16T15:39:22.563793+0000","last_active":"2024-09-16T15:39:23.562547+0000","last_peered":"2024-09-16T15:39:23.562547+0000","last_clean":"2024-09-16T15:39:23.562547+0000","last_became_active":"2024-09-16T15:39:22.563504+0000","last_became_peered":"2024-09-16T15:39:22.563504+0000","last_unstale":"2024-09-16T15:39:23.562547+0000","last_undegraded":"2024-09-16T15:39:23.562547+0000","last_fullsized":"2024-09-16T15:39:23.562547+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:56:07.665096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361245+0000","last_change":"2024-09-16T15:39:22.569041+0000","last_active":"2024-09-16T15:39:24.361245+0000","last_peered":"2024-09-16T15:39:24.361245+0000","last_clean":"2024-09-16T15:39:24.361245+0000","last_became_active":"2024-09-16T15:39:22.568799+0000","last_became_peered":"2024-09-16T15:39:22.568799+0000","last_unstale":"2024-09-16T15:39:24.361245+0000","last_undegraded":"2024-09-16T15:39:24.361245+0000","last_fullsized":"2024-09-16T15:39:24.361245+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:58:26.472440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.714354+0000","last_change":"2024-09-16T15:39:22.566051+0000","last_active":"2024-09-16T15:39:23.714354+0000","last_peered":"2024-09-16T15:39:23.714354+0000","last_clean":"2024-09-16T15:39:23.714354+0000","last_became_active":"2024-09-16T15:39:22.565764+0000","last_became_peered":"2024-09-16T15:39:22.565764+0000","last_unstale":"2024-09-16T15:39:23.714354+0000","last_undegraded":"2024-09-16T15:39:23.714354+0000","last_fullsized":"2024-09-16T15:39:23.714354+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:42:34.417720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361771+0000","last_change":"2024-09-16T15:39:22.574296+0000","last_active":"2024-09-16T15:39:24.361771+0000","last_peered":"2024-09-16T15:39:24.361771+0000","last_clean":"2024-09-16T15:39:24.361771+0000","last_became_active":"2024-09-16T15:39:22.574004+0000","last_became_peered":"2024-09-16T15:39:22.574004+0000","last_unstale":"2024-09-16T15:39:24.361771+0000","last_undegraded":"2024-09-16T15:39:24.361771+0000","last_fullsized":"2024-09-16T15:39:24.361771+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:36:56.913296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.560837+0000","last_change":"2024-09-16T15:39:22.559868+0000","last_active":"2024-09-16T15:39:23.560837+0000","last_peered":"2024-09-16T15:39:23.560837+0000","last_clean":"2024-09-16T15:39:23.560837+0000","last_became_active":"2024-09-16T15:39:22.559506+0000","last_became_peered":"2024-09-16T15:39:22.559506+0000","last_unstale":"2024-09-16T15:39:23.560837+0000","last_undegraded":"2024-09-16T15:39:23.560837+0000","last_fullsized":"2024-09-16T15:39:23.560837+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:33:53.332507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.562441+0000","last_change":"2024-09-16T15:39:21.562538+0000","last_active":"2024-09-16T15:39:23.562441+0000","last_peered":"2024-09-16T15:39:23.562441+0000","last_clean":"2024-09-16T15:39:23.562441+0000","last_became_active":"2024-09-16T15:39:21.559240+0000","last_became_peered":"2024-09-16T15:39:21.559240+0000","last_unstale":"2024-09-16T15:39:23.562441+0000","last_undegraded":"2024-09-16T15:39:23.562441+0000","last_fullsized":"2024-09-16T15:39:23.562441+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:43:58.109261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.562593+0000","last_change":"2024-09-16T15:39:22.582803+0000","last_active":"2024-09-16T15:39:23.562593+0000","last_peered":"2024-09-16T15:39:23.562593+0000","last_clean":"2024-09-16T15:39:23.562593+0000","last_became_active":"2024-09-16T15:39:22.582364+0000","last_became_peered":"2024-09-16T15:39:22.582364+0000","last_unstale":"2024-09-16T15:39:23.562593+0000","last_undegraded":"2024-09-16T15:39:23.562593+0000","last_fullsized":"2024-09-16T15:39:23.562593+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:40:27.419876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360960+0000","last_change":"2024-09-16T15:39:21.567667+0000","last_active":"2024-09-16T15:39:24.360960+0000","last_peered":"2024-09-16T15:39:24.360960+0000","last_clean":"2024-09-16T15:39:24.360960+0000","last_became_active":"2024-09-16T15:39:21.567399+0000","last_became_peered":"2024-09-16T15:39:21.567399+0000","last_unstale":"2024-09-16T15:39:24.360960+0000","last_undegraded":"2024-09-16T15:39:24.360960+0000","last_fullsized":"2024-09-16T15:39:24.360960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:22:23.876565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360132+0000","last_change":"2024-09-16T15:39:21.566105+0000","last_active":"2024-09-16T15:39:24.360132+0000","last_peered":"2024-09-16T15:39:24.360132+0000","last_clean":"2024-09-16T15:39:24.360132+0000","last_became_active":"2024-09-16T15:39:21.565719+0000","last_became_peered":"2024-09-16T15:39:21.565719+0000","last_unstale":"2024-09-16T15:39:24.360132+0000","last_undegraded":"2024-09-16T15:39:24.360132+0000","last_fullsized":"2024-09-16T15:39:24.360132+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:43:58.109261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.562255+0000","last_change":"2024-09-16T15:39:22.575843+0000","last_active":"2024-09-16T15:39:23.562255+0000","last_peered":"2024-09-16T15:39:23.562255+0000","last_clean":"2024-09-16T15:39:23.562255+0000","last_became_active":"2024-09-16T15:39:22.575543+0000","last_became_peered":"2024-09-16T15:39:22.575543+0000","last_unstale":"2024-09-16T15:39:23.562255+0000","last_undegraded":"2024-09-16T15:39:23.562255+0000","last_fullsized":"2024-09-16T15:39:23.562255+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:40:27.419876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360365+0000","last_change":"2024-09-16T15:39:21.562879+0000","last_active":"2024-09-16T15:39:24.360365+0000","last_peered":"2024-09-16T15:39:24.360365+0000","last_clean":"2024-09-16T15:39:24.360365+0000","last_became_active":"2024-09-16T15:39:21.562554+0000","last_became_peered":"2024-09-16T15:39:21.562554+0000","last_unstale":"2024-09-16T15:39:24.360365+0000","last_undegraded":"2024-09-16T15:39:24.360365+0000","last_fullsized":"2024-09-16T15:39:24.360365+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:33:52.324553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.562038+0000","last_change":"2024-09-16T15:39:22.571218+0000","last_active":"2024-09-16T15:39:23.562038+0000","last_peered":"2024-09-16T15:39:23.562038+0000","last_clean":"2024-09-16T15:39:23.562038+0000","last_became_active":"2024-09-16T15:39:22.570931+0000","last_became_peered":"2024-09-16T15:39:22.570931+0000","last_unstale":"2024-09-16T15:39:23.562038+0000","last_undegraded":"2024-09-16T15:39:23.562038+0000","last_fullsized":"2024-09-16T15:39:23.562038+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:57:08.823794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.561161+0000","last_change":"2024-09-16T15:39:21.558263+0000","last_active":"2024-09-16T15:39:23.561161+0000","last_peered":"2024-09-16T15:39:23.561161+0000","last_clean":"2024-09-16T15:39:23.561161+0000","last_became_active":"2024-09-16T15:39:21.557615+0000","last_became_peered":"2024-09-16T15:39:21.557615+0000","last_unstale":"2024-09-16T15:39:23.561161+0000","last_undegraded":"2024-09-16T15:39:23.561161+0000","last_fullsized":"2024-09-16T15:39:23.561161+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:10:40.788797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.362565+0000","last_change":"2024-09-16T15:39:22.586182+0000","last_active":"2024-09-16T15:39:24.362565+0000","last_peered":"2024-09-16T15:39:24.362565+0000","last_clean":"2024-09-16T15:39:24.362565+0000","last_became_active":"2024-09-16T15:39:22.584486+0000","last_became_peered":"2024-09-16T15:39:22.584486+0000","last_unstale":"2024-09-16T15:39:24.362565+0000","last_undegraded":"2024-09-16T15:39:24.362565+0000","last_fullsized":"2024-09-16T15:39:24.362565+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:25:52.095325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361757+0000","last_change":"2024-09-16T15:39:21.558934+0000","last_active":"2024-09-16T15:39:24.361757+0000","last_peered":"2024-09-16T15:39:24.361757+0000","last_clean":"2024-09-16T15:39:24.361757+0000","last_became_active":"2024-09-16T15:39:21.557612+0000","last_became_peered":"2024-09-16T15:39:21.557612+0000","last_unstale":"2024-09-16T15:39:24.361757+0000","last_undegraded":"2024-09-16T15:39:24.361757+0000","last_fullsized":"2024-09-16T15:39:24.361757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:21:44.910826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.563494+0000","last_change":"2024-09-16T15:39:22.563449+0000","last_active":"2024-09-16T15:39:23.563494+0000","last_peered":"2024-09-16T15:39:23.563494+0000","last_clean":"2024-09-16T15:39:23.563494+0000","last_became_active":"2024-09-16T15:39:22.558597+0000","last_became_peered":"2024-09-16T15:39:22.558597+0000","last_unstale":"2024-09-16T15:39:23.563494+0000","last_undegraded":"2024-09-16T15:39:23.563494+0000","last_fullsized":"2024-09-16T15:39:23.563494+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:08:47.835110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.714227+0000","last_change":"2024-09-16T15:39:21.561906+0000","last_active":"2024-09-16T15:39:23.714227+0000","last_peered":"2024-09-16T15:39:23.714227+0000","last_clean":"2024-09-16T15:39:23.714227+0000","last_became_active":"2024-09-16T15:39:21.561566+0000","last_became_peered":"2024-09-16T15:39:21.561566+0000","last_unstale":"2024-09-16T15:39:23.714227+0000","last_undegraded":"2024-09-16T15:39:23.714227+0000","last_fullsized":"2024-09-16T15:39:23.714227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:12:53.853659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360840+0000","last_change":"2024-09-16T15:39:22.581196+0000","last_active":"2024-09-16T15:39:24.360840+0000","last_peered":"2024-09-16T15:39:24.360840+0000","last_clean":"2024-09-16T15:39:24.360840+0000","last_became_active":"2024-09-16T15:39:22.579631+0000","last_became_peered":"2024-09-16T15:39:22.579631+0000","last_unstale":"2024-09-16T15:39:24.360840+0000","last_undegraded":"2024-09-16T15:39:24.360840+0000","last_fullsized":"2024-09-16T15:39:24.360840+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:15:37.944346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.362202+0000","last_change":"2024-09-16T15:39:21.560090+0000","last_active":"2024-09-16T15:39:24.362202+0000","last_peered":"2024-09-16T15:39:24.362202+0000","last_clean":"2024-09-16T15:39:24.362202+0000","last_became_active":"2024-09-16T15:39:21.558972+0000","last_became_peered":"2024-09-16T15:39:21.558972+0000","last_unstale":"2024-09-16T15:39:24.362202+0000","last_undegraded":"2024-09-16T15:39:24.362202+0000","last_fullsized":"2024-09-16T15:39:24.362202+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:12:19.666037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361423+0000","last_change":"2024-09-16T15:39:22.560439+0000","last_active":"2024-09-16T15:39:24.361423+0000","last_peered":"2024-09-16T15:39:24.361423+0000","last_clean":"2024-09-16T15:39:24.361423+0000","last_became_active":"2024-09-16T15:39:22.560172+0000","last_became_peered":"2024-09-16T15:39:22.560172+0000","last_unstale":"2024-09-16T15:39:24.361423+0000","last_undegraded":"2024-09-16T15:39:24.361423+0000","last_fullsized":"2024-09-16T15:39:24.361423+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:01:25.170048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.714255+0000","last_change":"2024-09-16T15:39:21.567091+0000","last_active":"2024-09-16T15:39:23.714255+0000","last_peered":"2024-09-16T15:39:23.714255+0000","last_clean":"2024-09-16T15:39:23.714255+0000","last_became_active":"2024-09-16T15:39:21.566668+0000","last_became_peered":"2024-09-16T15:39:21.566668+0000","last_unstale":"2024-09-16T15:39:23.714255+0000","last_undegraded":"2024-09-16T15:39:23.714255+0000","last_fullsized":"2024-09-16T15:39:23.714255+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:08:06.242307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360373+0000","last_change":"2024-09-16T15:39:22.569735+0000","last_active":"2024-09-16T15:39:24.360373+0000","last_peered":"2024-09-16T15:39:24.360373+0000","last_clean":"2024-09-16T15:39:24.360373+0000","last_became_active":"2024-09-16T15:39:22.569217+0000","last_became_peered":"2024-09-16T15:39:22.569217+0000","last_unstale":"2024-09-16T15:39:24.360373+0000","last_undegraded":"2024-09-16T15:39:24.360373+0000","last_fullsized":"2024-09-16T15:39:24.360373+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:34:12.122166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361075+0000","last_change":"2024-09-16T15:39:21.560934+0000","last_active":"2024-09-16T15:39:24.361075+0000","last_peered":"2024-09-16T15:39:24.361075+0000","last_clean":"2024-09-16T15:39:24.361075+0000","last_became_active":"2024-09-16T15:39:21.560690+0000","last_became_peered":"2024-09-16T15:39:21.560690+0000","last_unstale":"2024-09-16T15:39:24.361075+0000","last_undegraded":"2024-09-16T15:39:24.361075+0000","last_fullsized":"2024-09-16T15:39:24.361075+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:10:22.394137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360730+0000","last_change":"2024-09-16T15:39:22.569919+0000","last_active":"2024-09-16T15:39:24.360730+0000","last_peered":"2024-09-16T15:39:24.360730+0000","last_clean":"2024-09-16T15:39:24.360730+0000","last_became_active":"2024-09-16T15:39:22.569624+0000","last_became_peered":"2024-09-16T15:39:22.569624+0000","last_unstale":"2024-09-16T15:39:24.360730+0000","last_undegraded":"2024-09-16T15:39:24.360730+0000","last_fullsized":"2024-09-16T15:39:24.360730+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:42:11.842370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360437+0000","last_change":"2024-09-16T15:39:21.565143+0000","last_active":"2024-09-16T15:39:24.360437+0000","last_peered":"2024-09-16T15:39:24.360437+0000","last_clean":"2024-09-16T15:39:24.360437+0000","last_became_active":"2024-09-16T15:39:21.564887+0000","last_became_peered":"2024-09-16T15:39:21.564887+0000","last_unstale":"2024-09-16T15:39:24.360437+0000","last_undegraded":"2024-09-16T15:39:24.360437+0000","last_fullsized":"2024-09-16T15:39:24.360437+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:54:54.299318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.560542+0000","last_change":"2024-09-16T15:39:22.560025+0000","last_active":"2024-09-16T15:39:23.560542+0000","last_peered":"2024-09-16T15:39:23.560542+0000","last_clean":"2024-09-16T15:39:23.560542+0000","last_became_active":"2024-09-16T15:39:22.559563+0000","last_became_peered":"2024-09-16T15:39:22.559563+0000","last_unstale":"2024-09-16T15:39:23.560542+0000","last_undegraded":"2024-09-16T15:39:23.560542+0000","last_fullsized":"2024-09-16T15:39:23.560542+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:17:35.609828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.562702+0000","last_change":"2024-09-16T15:39:21.556186+0000","last_active":"2024-09-16T15:39:23.562702+0000","last_peered":"2024-09-16T15:39:23.562702+0000","last_clean":"2024-09-16T15:39:23.562702+0000","last_became_active":"2024-09-16T15:39:21.555847+0000","last_became_peered":"2024-09-16T15:39:21.555847+0000","last_unstale":"2024-09-16T15:39:23.562702+0000","last_undegraded":"2024-09-16T15:39:23.562702+0000","last_fullsized":"2024-09-16T15:39:23.562702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:43:58.109261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360215+0000","last_change":"2024-09-16T15:39:22.581265+0000","last_active":"2024-09-16T15:39:24.360215+0000","last_peered":"2024-09-16T15:39:24.360215+0000","last_clean":"2024-09-16T15:39:24.360215+0000","last_became_active":"2024-09-16T15:39:22.580808+0000","last_became_peered":"2024-09-16T15:39:22.580808+0000","last_unstale":"2024-09-16T15:39:24.360215+0000","last_undegraded":"2024-09-16T15:39:24.360215+0000","last_fullsized":"2024-09-16T15:39:24.360215+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:40:04.044401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.362115+0000","last_change":"2024-09-16T15:39:21.565669+0000","last_active":"2024-09-16T15:39:24.362115+0000","last_peered":"2024-09-16T15:39:24.362115+0000","last_clean":"2024-09-16T15:39:24.362115+0000","last_became_active":"2024-09-16T15:39:21.564058+0000","last_became_peered":"2024-09-16T15:39:21.564058+0000","last_unstale":"2024-09-16T15:39:24.362115+0000","last_undegraded":"2024-09-16T15:39:24.362115+0000","last_fullsized":"2024-09-16T15:39:24.362115+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:36:55.905341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.362680+0000","last_change":"2024-09-16T15:39:22.586068+0000","last_active":"2024-09-16T15:39:24.362680+0000","last_peered":"2024-09-16T15:39:24.362680+0000","last_clean":"2024-09-16T15:39:24.362680+0000","last_became_active":"2024-09-16T15:39:22.585745+0000","last_became_peered":"2024-09-16T15:39:22.585745+0000","last_unstale":"2024-09-16T15:39:24.362680+0000","last_undegraded":"2024-09-16T15:39:24.362680+0000","last_fullsized":"2024-09-16T15:39:24.362680+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:46:44.196294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.562702+0000","last_change":"2024-09-16T15:39:21.564692+0000","last_active":"2024-09-16T15:39:23.562702+0000","last_peered":"2024-09-16T15:39:23.562702+0000","last_clean":"2024-09-16T15:39:23.562702+0000","last_became_active":"2024-09-16T15:39:21.564414+0000","last_became_peered":"2024-09-16T15:39:21.564414+0000","last_unstale":"2024-09-16T15:39:23.562702+0000","last_undegraded":"2024-09-16T15:39:23.562702+0000","last_fullsized":"2024-09-16T15:39:23.562702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:46:18.513631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.362100+0000","last_change":"2024-09-16T15:39:22.560891+0000","last_active":"2024-09-16T15:39:24.362100+0000","last_peered":"2024-09-16T15:39:24.362100+0000","last_clean":"2024-09-16T15:39:24.362100+0000","last_became_active":"2024-09-16T15:39:22.560598+0000","last_became_peered":"2024-09-16T15:39:22.560598+0000","last_unstale":"2024-09-16T15:39:24.362100+0000","last_undegraded":"2024-09-16T15:39:24.362100+0000","last_fullsized":"2024-09-16T15:39:24.362100+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:57:51.336962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.563158+0000","last_change":"2024-09-16T15:39:21.559845+0000","last_active":"2024-09-16T15:39:23.563158+0000","last_peered":"2024-09-16T15:39:23.563158+0000","last_clean":"2024-09-16T15:39:23.563158+0000","last_became_active":"2024-09-16T15:39:21.559089+0000","last_became_peered":"2024-09-16T15:39:21.559089+0000","last_unstale":"2024-09-16T15:39:23.563158+0000","last_undegraded":"2024-09-16T15:39:23.563158+0000","last_fullsized":"2024-09-16T15:39:23.563158+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:40:57.508356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361096+0000","last_change":"2024-09-16T15:39:22.575906+0000","last_active":"2024-09-16T15:39:24.361096+0000","last_peered":"2024-09-16T15:39:24.361096+0000","last_clean":"2024-09-16T15:39:24.361096+0000","last_became_active":"2024-09-16T15:39:22.574839+0000","last_became_peered":"2024-09-16T15:39:22.574839+0000","last_unstale":"2024-09-16T15:39:24.361096+0000","last_undegraded":"2024-09-16T15:39:24.361096+0000","last_fullsized":"2024-09-16T15:39:24.361096+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:13:11.232766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360712+0000","last_change":"2024-09-16T15:39:21.565376+0000","last_active":"2024-09-16T15:39:24.360712+0000","last_peered":"2024-09-16T15:39:24.360712+0000","last_clean":"2024-09-16T15:39:24.360712+0000","last_became_active":"2024-09-16T15:39:21.565007+0000","last_became_peered":"2024-09-16T15:39:21.565007+0000","last_unstale":"2024-09-16T15:39:24.360712+0000","last_undegraded":"2024-09-16T15:39:24.360712+0000","last_fullsized":"2024-09-16T15:39:24.360712+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:39:19.838119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361711+0000","last_change":"2024-09-16T15:39:22.565594+0000","last_active":"2024-09-16T15:39:24.361711+0000","last_peered":"2024-09-16T15:39:24.361711+0000","last_clean":"2024-09-16T15:39:24.361711+0000","last_became_active":"2024-09-16T15:39:22.565255+0000","last_became_peered":"2024-09-16T15:39:22.565255+0000","last_unstale":"2024-09-16T15:39:24.361711+0000","last_undegraded":"2024-09-16T15:39:24.361711+0000","last_fullsized":"2024-09-16T15:39:24.361711+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:32.663886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.560438+0000","last_change":"2024-09-16T15:39:12.698813+0000","last_active":"2024-09-16T15:39:23.560438+0000","last_peered":"2024-09-16T15:39:23.560438+0000","last_clean":"2024-09-16T15:39:23.560438+0000","last_became_active":"2024-09-16T15:39:12.697695+0000","last_became_peered":"2024-09-16T15:39:12.697695+0000","last_unstale":"2024-09-16T15:39:23.560438+0000","last_undegraded":"2024-09-16T15:39:23.560438+0000","last_fullsized":"2024-09-16T15:39:23.560438+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_clean_scrub_stamp":"2024-09-16T15:39:11.684967+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:02.061081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360791+0000","last_change":"2024-09-16T15:39:21.565360+0000","last_active":"2024-09-16T15:39:24.360791+0000","last_peered":"2024-09-16T15:39:24.360791+0000","last_clean":"2024-09-16T15:39:24.360791+0000","last_became_active":"2024-09-16T15:39:21.565114+0000","last_became_peered":"2024-09-16T15:39:21.565114+0000","last_unstale":"2024-09-16T15:39:24.360791+0000","last_undegraded":"2024-09-16T15:39:24.360791+0000","last_fullsized":"2024-09-16T15:39:24.360791+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:40:57.304044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.562889+0000","last_change":"2024-09-16T15:39:22.579056+0000","last_active":"2024-09-16T15:39:23.562889+0000","last_peered":"2024-09-16T15:39:23.562889+0000","last_clean":"2024-09-16T15:39:23.562889+0000","last_became_active":"2024-09-16T15:39:22.576443+0000","last_became_peered":"2024-09-16T15:39:22.576443+0000","last_unstale":"2024-09-16T15:39:23.562889+0000","last_undegraded":"2024-09-16T15:39:23.562889+0000","last_fullsized":"2024-09-16T15:39:23.562889+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:56:07.665096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360279+0000","last_change":"2024-09-16T15:39:21.568792+0000","last_active":"2024-09-16T15:39:24.360279+0000","last_peered":"2024-09-16T15:39:24.360279+0000","last_clean":"2024-09-16T15:39:24.360279+0000","last_became_active":"2024-09-16T15:39:21.568534+0000","last_became_peered":"2024-09-16T15:39:21.568534+0000","last_unstale":"2024-09-16T15:39:24.360279+0000","last_undegraded":"2024-09-16T15:39:24.360279+0000","last_fullsized":"2024-09-16T15:39:24.360279+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:16:01.495612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360315+0000","last_change":"2024-09-16T15:39:22.572844+0000","last_active":"2024-09-16T15:39:24.360315+0000","last_peered":"2024-09-16T15:39:24.360315+0000","last_clean":"2024-09-16T15:39:24.360315+0000","last_became_active":"2024-09-16T15:39:22.570080+0000","last_became_peered":"2024-09-16T15:39:22.570080+0000","last_unstale":"2024-09-16T15:39:24.360315+0000","last_undegraded":"2024-09-16T15:39:24.360315+0000","last_fullsized":"2024-09-16T15:39:24.360315+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:19:39.018320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.714355+0000","last_change":"2024-09-16T15:39:21.563752+0000","last_active":"2024-09-16T15:39:23.714355+0000","last_peered":"2024-09-16T15:39:23.714355+0000","last_clean":"2024-09-16T15:39:23.714355+0000","last_became_active":"2024-09-16T15:39:21.563430+0000","last_became_peered":"2024-09-16T15:39:21.563430+0000","last_unstale":"2024-09-16T15:39:23.714355+0000","last_undegraded":"2024-09-16T15:39:23.714355+0000","last_fullsized":"2024-09-16T15:39:23.714355+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:44:53.582663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.562026+0000","last_change":"2024-09-16T15:39:22.560753+0000","last_active":"2024-09-16T15:39:23.562026+0000","last_peered":"2024-09-16T15:39:23.562026+0000","last_clean":"2024-09-16T15:39:23.562026+0000","last_became_active":"2024-09-16T15:39:22.560416+0000","last_became_peered":"2024-09-16T15:39:22.560416+0000","last_unstale":"2024-09-16T15:39:23.562026+0000","last_undegraded":"2024-09-16T15:39:23.562026+0000","last_fullsized":"2024-09-16T15:39:23.562026+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:30:53.438955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361167+0000","last_change":"2024-09-16T15:39:21.568766+0000","last_active":"2024-09-16T15:39:24.361167+0000","last_peered":"2024-09-16T15:39:24.361167+0000","last_clean":"2024-09-16T15:39:24.361167+0000","last_became_active":"2024-09-16T15:39:21.568515+0000","last_became_peered":"2024-09-16T15:39:21.568515+0000","last_unstale":"2024-09-16T15:39:24.361167+0000","last_undegraded":"2024-09-16T15:39:24.361167+0000","last_fullsized":"2024-09-16T15:39:24.361167+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:29:10.288957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361313+0000","last_change":"2024-09-16T15:39:22.581762+0000","last_active":"2024-09-16T15:39:24.361313+0000","last_peered":"2024-09-16T15:39:24.361313+0000","last_clean":"2024-09-16T15:39:24.361313+0000","last_became_active":"2024-09-16T15:39:22.572639+0000","last_became_peered":"2024-09-16T15:39:22.572639+0000","last_unstale":"2024-09-16T15:39:24.361313+0000","last_undegraded":"2024-09-16T15:39:24.361313+0000","last_fullsized":"2024-09-16T15:39:24.361313+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:06:34.579134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.562849+0000","last_change":"2024-09-16T15:39:21.561768+0000","last_active":"2024-09-16T15:39:23.562849+0000","last_peered":"2024-09-16T15:39:23.562849+0000","last_clean":"2024-09-16T15:39:23.562849+0000","last_became_active":"2024-09-16T15:39:21.561188+0000","last_became_peered":"2024-09-16T15:39:21.561188+0000","last_unstale":"2024-09-16T15:39:23.562849+0000","last_undegraded":"2024-09-16T15:39:23.562849+0000","last_fullsized":"2024-09-16T15:39:23.562849+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:39:19.838119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.362299+0000","last_change":"2024-09-16T15:39:22.578044+0000","last_active":"2024-09-16T15:39:24.362299+0000","last_peered":"2024-09-16T15:39:24.362299+0000","last_clean":"2024-09-16T15:39:24.362299+0000","last_became_active":"2024-09-16T15:39:22.577752+0000","last_became_peered":"2024-09-16T15:39:22.577752+0000","last_unstale":"2024-09-16T15:39:24.362299+0000","last_undegraded":"2024-09-16T15:39:24.362299+0000","last_fullsized":"2024-09-16T15:39:24.362299+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:36:16.988647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361983+0000","last_change":"2024-09-16T15:39:21.560508+0000","last_active":"2024-09-16T15:39:24.361983+0000","last_peered":"2024-09-16T15:39:24.361983+0000","last_clean":"2024-09-16T15:39:24.361983+0000","last_became_active":"2024-09-16T15:39:21.560082+0000","last_became_peered":"2024-09-16T15:39:21.560082+0000","last_unstale":"2024-09-16T15:39:24.361983+0000","last_undegraded":"2024-09-16T15:39:24.361983+0000","last_fullsized":"2024-09-16T15:39:24.361983+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:17:34.601873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360646+0000","last_change":"2024-09-16T15:39:22.581816+0000","last_active":"2024-09-16T15:39:24.360646+0000","last_peered":"2024-09-16T15:39:24.360646+0000","last_clean":"2024-09-16T15:39:24.360646+0000","last_became_active":"2024-09-16T15:39:22.581523+0000","last_became_peered":"2024-09-16T15:39:22.581523+0000","last_unstale":"2024-09-16T15:39:24.360646+0000","last_undegraded":"2024-09-16T15:39:24.360646+0000","last_fullsized":"2024-09-16T15:39:24.360646+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:57:08.823794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.562963+0000","last_change":"2024-09-16T15:39:21.564493+0000","last_active":"2024-09-16T15:39:23.562963+0000","last_peered":"2024-09-16T15:39:23.562963+0000","last_clean":"2024-09-16T15:39:23.562963+0000","last_became_active":"2024-09-16T15:39:21.564207+0000","last_became_peered":"2024-09-16T15:39:21.564207+0000","last_unstale":"2024-09-16T15:39:23.562963+0000","last_undegraded":"2024-09-16T15:39:23.562963+0000","last_fullsized":"2024-09-16T15:39:23.562963+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:17:34.601873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.562454+0000","last_change":"2024-09-16T15:39:22.577107+0000","last_active":"2024-09-16T15:39:23.562454+0000","last_peered":"2024-09-16T15:39:23.562454+0000","last_clean":"2024-09-16T15:39:23.562454+0000","last_became_active":"2024-09-16T15:39:22.576840+0000","last_became_peered":"2024-09-16T15:39:22.576840+0000","last_unstale":"2024-09-16T15:39:23.562454+0000","last_undegraded":"2024-09-16T15:39:23.562454+0000","last_fullsized":"2024-09-16T15:39:23.562454+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:30:53.438955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361083+0000","last_change":"2024-09-16T15:39:21.570398+0000","last_active":"2024-09-16T15:39:24.361083+0000","last_peered":"2024-09-16T15:39:24.361083+0000","last_clean":"2024-09-16T15:39:24.361083+0000","last_became_active":"2024-09-16T15:39:21.570020+0000","last_became_peered":"2024-09-16T15:39:21.570020+0000","last_unstale":"2024-09-16T15:39:24.361083+0000","last_undegraded":"2024-09-16T15:39:24.361083+0000","last_fullsized":"2024-09-16T15:39:24.361083+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:19:38.010365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.563030+0000","last_change":"2024-09-16T15:39:22.572370+0000","last_active":"2024-09-16T15:39:23.563030+0000","last_peered":"2024-09-16T15:39:23.563030+0000","last_clean":"2024-09-16T15:39:23.563030+0000","last_became_active":"2024-09-16T15:39:22.572114+0000","last_became_peered":"2024-09-16T15:39:22.572114+0000","last_unstale":"2024-09-16T15:39:23.563030+0000","last_undegraded":"2024-09-16T15:39:23.563030+0000","last_fullsized":"2024-09-16T15:39:23.563030+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:22:34.502460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360218+0000","last_change":"2024-09-16T15:39:21.561676+0000","last_active":"2024-09-16T15:39:24.360218+0000","last_peered":"2024-09-16T15:39:24.360218+0000","last_clean":"2024-09-16T15:39:24.360218+0000","last_became_active":"2024-09-16T15:39:21.560798+0000","last_became_peered":"2024-09-16T15:39:21.560798+0000","last_unstale":"2024-09-16T15:39:24.360218+0000","last_undegraded":"2024-09-16T15:39:24.360218+0000","last_fullsized":"2024-09-16T15:39:24.360218+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:12:34.622210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360594+0000","last_change":"2024-09-16T15:39:22.567438+0000","last_active":"2024-09-16T15:39:24.360594+0000","last_peered":"2024-09-16T15:39:24.360594+0000","last_clean":"2024-09-16T15:39:24.360594+0000","last_became_active":"2024-09-16T15:39:22.567191+0000","last_became_peered":"2024-09-16T15:39:22.567191+0000","last_unstale":"2024-09-16T15:39:24.360594+0000","last_undegraded":"2024-09-16T15:39:24.360594+0000","last_fullsized":"2024-09-16T15:39:24.360594+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:42:25.111846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.561951+0000","last_change":"2024-09-16T15:39:21.566726+0000","last_active":"2024-09-16T15:39:23.561951+0000","last_peered":"2024-09-16T15:39:23.561951+0000","last_clean":"2024-09-16T15:39:23.561951+0000","last_became_active":"2024-09-16T15:39:21.566406+0000","last_became_peered":"2024-09-16T15:39:21.566406+0000","last_unstale":"2024-09-16T15:39:23.561951+0000","last_undegraded":"2024-09-16T15:39:23.561951+0000","last_fullsized":"2024-09-16T15:39:23.561951+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:40:57.508356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.561742+0000","last_change":"2024-09-16T15:39:22.575174+0000","last_active":"2024-09-16T15:39:23.561742+0000","last_peered":"2024-09-16T15:39:23.561742+0000","last_clean":"2024-09-16T15:39:23.561742+0000","last_became_active":"2024-09-16T15:39:22.574865+0000","last_became_peered":"2024-09-16T15:39:22.574865+0000","last_unstale":"2024-09-16T15:39:23.561742+0000","last_undegraded":"2024-09-16T15:39:23.561742+0000","last_fullsized":"2024-09-16T15:39:23.561742+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:10:23.402092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360483+0000","last_change":"2024-09-16T15:39:21.561384+0000","last_active":"2024-09-16T15:39:24.360483+0000","last_peered":"2024-09-16T15:39:24.360483+0000","last_clean":"2024-09-16T15:39:24.360483+0000","last_became_active":"2024-09-16T15:39:21.561078+0000","last_became_peered":"2024-09-16T15:39:21.561078+0000","last_unstale":"2024-09-16T15:39:24.360483+0000","last_undegraded":"2024-09-16T15:39:24.360483+0000","last_fullsized":"2024-09-16T15:39:24.360483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:10:40.788797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.714500+0000","last_change":"2024-09-16T15:39:22.576232+0000","last_active":"2024-09-16T15:39:23.714500+0000","last_peered":"2024-09-16T15:39:23.714500+0000","last_clean":"2024-09-16T15:39:23.714500+0000","last_became_active":"2024-09-16T15:39:22.575962+0000","last_became_peered":"2024-09-16T15:39:22.575962+0000","last_unstale":"2024-09-16T15:39:23.714500+0000","last_undegraded":"2024-09-16T15:39:23.714500+0000","last_fullsized":"2024-09-16T15:39:23.714500+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:43:58.162669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360704+0000","last_change":"2024-09-16T15:39:21.566471+0000","last_active":"2024-09-16T15:39:24.360704+0000","last_peered":"2024-09-16T15:39:24.360704+0000","last_clean":"2024-09-16T15:39:24.360704+0000","last_became_active":"2024-09-16T15:39:21.566220+0000","last_became_peered":"2024-09-16T15:39:21.566220+0000","last_unstale":"2024-09-16T15:39:24.360704+0000","last_undegraded":"2024-09-16T15:39:24.360704+0000","last_fullsized":"2024-09-16T15:39:24.360704+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:34:11.114212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360894+0000","last_change":"2024-09-16T15:39:22.565878+0000","last_active":"2024-09-16T15:39:24.360894+0000","last_peered":"2024-09-16T15:39:24.360894+0000","last_clean":"2024-09-16T15:39:24.360894+0000","last_became_active":"2024-09-16T15:39:22.565599+0000","last_became_peered":"2024-09-16T15:39:22.565599+0000","last_unstale":"2024-09-16T15:39:24.360894+0000","last_undegraded":"2024-09-16T15:39:24.360894+0000","last_fullsized":"2024-09-16T15:39:24.360894+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:04:51.595786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361501+0000","last_change":"2024-09-16T15:39:22.563299+0000","last_active":"2024-09-16T15:39:24.361501+0000","last_peered":"2024-09-16T15:39:24.361501+0000","last_clean":"2024-09-16T15:39:24.361501+0000","last_became_active":"2024-09-16T15:39:22.560515+0000","last_became_peered":"2024-09-16T15:39:22.560515+0000","last_unstale":"2024-09-16T15:39:24.361501+0000","last_undegraded":"2024-09-16T15:39:24.361501+0000","last_fullsized":"2024-09-16T15:39:24.361501+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:45:21.853848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361156+0000","last_change":"2024-09-16T15:39:21.561779+0000","last_active":"2024-09-16T15:39:24.361156+0000","last_peered":"2024-09-16T15:39:24.361156+0000","last_clean":"2024-09-16T15:39:24.361156+0000","last_became_active":"2024-09-16T15:39:21.561471+0000","last_became_peered":"2024-09-16T15:39:21.561471+0000","last_unstale":"2024-09-16T15:39:24.361156+0000","last_undegraded":"2024-09-16T15:39:24.361156+0000","last_fullsized":"2024-09-16T15:39:24.361156+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:29:24.601704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.560906+0000","last_change":"2024-09-16T15:39:22.565324+0000","last_active":"2024-09-16T15:39:23.560906+0000","last_peered":"2024-09-16T15:39:23.560906+0000","last_clean":"2024-09-16T15:39:23.560906+0000","last_became_active":"2024-09-16T15:39:22.564275+0000","last_became_peered":"2024-09-16T15:39:22.564275+0000","last_unstale":"2024-09-16T15:39:23.560906+0000","last_undegraded":"2024-09-16T15:39:23.560906+0000","last_fullsized":"2024-09-16T15:39:23.560906+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:16:02.503567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361469+0000","last_change":"2024-09-16T15:39:21.563348+0000","last_active":"2024-09-16T15:39:24.361469+0000","last_peered":"2024-09-16T15:39:24.361469+0000","last_clean":"2024-09-16T15:39:24.361469+0000","last_became_active":"2024-09-16T15:39:21.560699+0000","last_became_peered":"2024-09-16T15:39:21.560699+0000","last_unstale":"2024-09-16T15:39:24.361469+0000","last_undegraded":"2024-09-16T15:39:24.361469+0000","last_fullsized":"2024-09-16T15:39:24.361469+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:36:15.980692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.563361+0000","last_change":"2024-09-16T15:39:22.558509+0000","last_active":"2024-09-16T15:39:23.563361+0000","last_peered":"2024-09-16T15:39:23.563361+0000","last_clean":"2024-09-16T15:39:23.563361+0000","last_became_active":"2024-09-16T15:39:22.558197+0000","last_became_peered":"2024-09-16T15:39:22.558197+0000","last_unstale":"2024-09-16T15:39:23.563361+0000","last_undegraded":"2024-09-16T15:39:23.563361+0000","last_fullsized":"2024-09-16T15:39:23.563361+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:10:15.420114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.563141+0000","last_change":"2024-09-16T15:39:21.561287+0000","last_active":"2024-09-16T15:39:23.563141+0000","last_peered":"2024-09-16T15:39:23.563141+0000","last_clean":"2024-09-16T15:39:23.563141+0000","last_became_active":"2024-09-16T15:39:21.559935+0000","last_became_peered":"2024-09-16T15:39:21.559935+0000","last_unstale":"2024-09-16T15:39:23.563141+0000","last_undegraded":"2024-09-16T15:39:23.563141+0000","last_fullsized":"2024-09-16T15:39:23.563141+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:19:57.274084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.563208+0000","last_change":"2024-09-16T15:39:22.576005+0000","last_active":"2024-09-16T15:39:23.563208+0000","last_peered":"2024-09-16T15:39:23.563208+0000","last_clean":"2024-09-16T15:39:23.563208+0000","last_became_active":"2024-09-16T15:39:22.575434+0000","last_became_peered":"2024-09-16T15:39:22.575434+0000","last_unstale":"2024-09-16T15:39:23.563208+0000","last_undegraded":"2024-09-16T15:39:23.563208+0000","last_fullsized":"2024-09-16T15:39:23.563208+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:03:51.340088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.562531+0000","last_change":"2024-09-16T15:39:21.568944+0000","last_active":"2024-09-16T15:39:23.562531+0000","last_peered":"2024-09-16T15:39:23.562531+0000","last_clean":"2024-09-16T15:39:23.562531+0000","last_became_active":"2024-09-16T15:39:21.568706+0000","last_became_peered":"2024-09-16T15:39:21.568706+0000","last_unstale":"2024-09-16T15:39:23.562531+0000","last_undegraded":"2024-09-16T15:39:23.562531+0000","last_fullsized":"2024-09-16T15:39:23.562531+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:19:57.274084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509448,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":76284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509447,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":823296,"data_stored":666744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509446,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":241664,"data_stored":78568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509446,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":225280,"data_stored":76866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":233472,"data_stored":77896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509447,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":819200,"data_stored":666722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509445,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":76340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542152,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":76408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":27654604},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:39:31.846 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T15:39:31.846 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-09-16T15:39:31.847 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-16T15:39:32.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.039+0000 7fd972600640 1 -- 172.21.15.151:0/2643300120 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd96c0b2780 msgr2=0x7fd96c168c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:32.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.039+0000 7fd972600640 1 --2- 172.21.15.151:0/2643300120 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd96c0b2780 0x7fd96c168c30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd96c0d4ea0 tx=0x7fd96402a890 comp rx=0 tx=0).stop 2024-09-16T15:39:32.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.039+0000 7fd972600640 1 -- 172.21.15.151:0/2643300120 shutdown_connections 2024-09-16T15:39:32.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.039+0000 7fd972600640 1 --2- 172.21.15.151:0/2643300120 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd96c0b2780 0x7fd96c168c30 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:32.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.039+0000 7fd972600640 1 -- 172.21.15.151:0/2643300120 >> 172.21.15.151:0/2643300120 conn(0x7fd96c0d3df0 msgr2=0x7fd96c0d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:32.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.039+0000 7fd972600640 1 -- 172.21.15.151:0/2643300120 shutdown_connections 2024-09-16T15:39:32.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.039+0000 7fd972600640 1 -- 172.21.15.151:0/2643300120 wait complete. 2024-09-16T15:39:32.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.041+0000 7fd972600640 1 Processor -- start 2024-09-16T15:39:32.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.041+0000 7fd972600640 1 -- start start 2024-09-16T15:39:32.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.042+0000 7fd972600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd96c0b2780 0x7fd96c1f87b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:32.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.042+0000 7fd972600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd96c0b5370 0x7fd96c0b9050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:32.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.042+0000 7fd972600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd96c161440 0x7fd96c0d19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:32.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.042+0000 7fd972600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd96c1692f0 con 0x7fd96c161440 2024-09-16T15:39:32.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.042+0000 7fd972600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd96c169170 con 0x7fd96c0b2780 2024-09-16T15:39:32.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.042+0000 7fd972600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fd96c169470 con 0x7fd96c0b5370 2024-09-16T15:39:32.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.042+0000 7fd971c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd96c161440 0x7fd96c0d19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:32.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.042+0000 7fd971200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd96c0b2780 0x7fd96c1f87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:32.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.042+0000 7fd971c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd96c161440 0x7fd96c0d19a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47354/0 (socket says 172.21.15.151:47354) 2024-09-16T15:39:32.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.042+0000 7fd971c00640 1 -- 172.21.15.151:0/2978671429 learned_addr learned my addr 172.21.15.151:0/2978671429 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:32.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.042+0000 7fd96be00640 1 --2- 172.21.15.151:0/2978671429 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd96c0b5370 0x7fd96c0b9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:32.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.042+0000 7fd971c00640 1 -- 172.21.15.151:0/2978671429 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd96c0b5370 msgr2=0x7fd96c0b9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:32.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.042+0000 7fd971c00640 1 --2- 172.21.15.151:0/2978671429 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd96c0b5370 0x7fd96c0b9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:32.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.042+0000 7fd971c00640 1 -- 172.21.15.151:0/2978671429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd96c0b2780 msgr2=0x7fd96c1f87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:32.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.042+0000 7fd971c00640 1 --2- 172.21.15.151:0/2978671429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd96c0b2780 0x7fd96c1f87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:32.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.042+0000 7fd971c00640 1 -- 172.21.15.151:0/2978671429 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd96c0b8290 con 0x7fd96c161440 2024-09-16T15:39:32.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.042+0000 7fd971200640 1 --2- 172.21.15.151:0/2978671429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd96c0b2780 0x7fd96c1f87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:39:32.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.043+0000 7fd971c00640 1 --2- 172.21.15.151:0/2978671429 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd96c161440 0x7fd96c0d19a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd96402ae80 tx=0x7fd964003d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:32.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.043+0000 7fd968c00640 1 -- 172.21.15.151:0/2978671429 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd964031e40 con 0x7fd96c161440 2024-09-16T15:39:32.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.043+0000 7fd968c00640 1 -- 172.21.15.151:0/2978671429 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd964004130 con 0x7fd96c161440 2024-09-16T15:39:32.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.043+0000 7fd972600640 1 -- 172.21.15.151:0/2978671429 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd96c0b8570 con 0x7fd96c161440 2024-09-16T15:39:32.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.043+0000 7fd968c00640 1 -- 172.21.15.151:0/2978671429 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd964004580 con 0x7fd96c161440 2024-09-16T15:39:32.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.043+0000 7fd972600640 1 -- 172.21.15.151:0/2978671429 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd96c0b8b30 con 0x7fd96c161440 2024-09-16T15:39:32.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.044+0000 7fd972600640 1 -- 172.21.15.151:0/2978671429 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd96c0b34b0 con 0x7fd96c161440 2024-09-16T15:39:32.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.045+0000 7fd968c00640 1 -- 172.21.15.151:0/2978671429 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fd964039040 con 0x7fd96c161440 2024-09-16T15:39:32.047 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.046+0000 7fd968c00640 1 --2- 172.21.15.151:0/2978671429 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd94c078880 0x7fd94c07ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:32.047 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.046+0000 7fd968c00640 1 -- 172.21.15.151:0/2978671429 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fd9640bb970 con 0x7fd96c161440 2024-09-16T15:39:32.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.047+0000 7fd971200640 1 --2- 172.21.15.151:0/2978671429 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd94c078880 0x7fd94c07ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:32.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.048+0000 7fd971200640 1 --2- 172.21.15.151:0/2978671429 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd94c078880 0x7fd94c07ad40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd960007f20 tx=0x7fd960007eb0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:32.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.050+0000 7fd968c00640 1 -- 172.21.15.151:0/2978671429 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd964040030 con 0x7fd96c161440 2024-09-16T15:39:32.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.237+0000 7fd972600640 1 -- 172.21.15.151:0/2978671429 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7fd96c151880 con 0x7fd96c161440 2024-09-16T15:39:32.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.237+0000 7fd968c00640 1 -- 172.21.15.151:0/2978671429 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v27) ==== 86+0+160 (secure 0 0 0) 0x7fd964031790 con 0x7fd96c161440 2024-09-16T15:39:32.239 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:39:32.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.240+0000 7fd972600640 1 -- 172.21.15.151:0/2978671429 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd94c078880 msgr2=0x7fd94c07ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:32.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.240+0000 7fd972600640 1 --2- 172.21.15.151:0/2978671429 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd94c078880 0x7fd94c07ad40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd960007f20 tx=0x7fd960007eb0 comp rx=0 tx=0).stop 2024-09-16T15:39:32.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.240+0000 7fd972600640 1 -- 172.21.15.151:0/2978671429 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd96c161440 msgr2=0x7fd96c0d19a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:32.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.240+0000 7fd972600640 1 --2- 172.21.15.151:0/2978671429 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd96c161440 0x7fd96c0d19a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd96402ae80 tx=0x7fd964003d60 comp rx=0 tx=0).stop 2024-09-16T15:39:32.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.240+0000 7fd972600640 1 -- 172.21.15.151:0/2978671429 shutdown_connections 2024-09-16T15:39:32.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.240+0000 7fd972600640 1 --2- 172.21.15.151:0/2978671429 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd94c078880 0x7fd94c07ad40 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:32.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.240+0000 7fd972600640 1 --2- 172.21.15.151:0/2978671429 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd96c161440 0x7fd96c0d19a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:32.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.240+0000 7fd972600640 1 --2- 172.21.15.151:0/2978671429 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd96c0b5370 0x7fd96c0b9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:32.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.240+0000 7fd972600640 1 --2- 172.21.15.151:0/2978671429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd96c0b2780 0x7fd96c1f87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:32.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.240+0000 7fd972600640 1 -- 172.21.15.151:0/2978671429 >> 172.21.15.151:0/2978671429 conn(0x7fd96c0d3df0 msgr2=0x7fd96c0b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:32.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.240+0000 7fd972600640 1 -- 172.21.15.151:0/2978671429 shutdown_connections 2024-09-16T15:39:32.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.240+0000 7fd972600640 1 -- 172.21.15.151:0/2978671429 wait complete. 2024-09-16T15:39:32.257 INFO:teuthology.orchestra.run.smithi151.stdout:[{"version":27,"timestamp":"2024-09-16T15:39:10.241573+0000","name":"","changes":[{"name":"osd.2/osd_mclock_max_capacity_iops_ssd","new_value":"3817.533640"}]}] 2024-09-16T15:39:32.257 INFO:tasks.ceph_manager:config epoch is 27 2024-09-16T15:39:32.257 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-16T15:39:32.257 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-16T15:39:32.258 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-16T15:39:32.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.414+0000 7fdcb4400640 1 -- 172.21.15.151:0/2940307743 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdcac0ccf00 msgr2=0x7fdcac0b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:32.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.414+0000 7fdcb4400640 1 --2- 172.21.15.151:0/2940307743 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdcac0ccf00 0x7fdcac0b2440 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fdc98009830 tx=0x7fdc9802a530 comp rx=0 tx=0).stop 2024-09-16T15:39:32.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.414+0000 7fdcb4400640 1 -- 172.21.15.151:0/2940307743 shutdown_connections 2024-09-16T15:39:32.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.414+0000 7fdcb4400640 1 --2- 172.21.15.151:0/2940307743 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdcac0ccf00 0x7fdcac0b2440 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:32.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.414+0000 7fdcb4400640 1 -- 172.21.15.151:0/2940307743 >> 172.21.15.151:0/2940307743 conn(0x7fdcac0b1950 msgr2=0x7fdcac0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:32.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.414+0000 7fdcb4400640 1 -- 172.21.15.151:0/2940307743 shutdown_connections 2024-09-16T15:39:32.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.414+0000 7fdcb4400640 1 -- 172.21.15.151:0/2940307743 wait complete. 2024-09-16T15:39:32.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.416+0000 7fdcb4400640 1 Processor -- start 2024-09-16T15:39:32.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.416+0000 7fdcb4400640 1 -- start start 2024-09-16T15:39:32.419 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.417+0000 7fdcb4400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdcac073fb0 0x7fdcac1f6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:32.419 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.417+0000 7fdcb4400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdcac0ccf00 0x7fdcac1574d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:32.419 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.417+0000 7fdcb4400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdcac15f5a0 0x7fdcac1f1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:32.419 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.417+0000 7fdcb4400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdcac06f9f0 con 0x7fdcac073fb0 2024-09-16T15:39:32.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.417+0000 7fdcb4400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdcac06f870 con 0x7fdcac0ccf00 2024-09-16T15:39:32.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.417+0000 7fdcb4400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fdcac074420 con 0x7fdcac15f5a0 2024-09-16T15:39:32.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.417+0000 7fdcb1a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdcac073fb0 0x7fdcac1f6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:32.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.417+0000 7fdcb1a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdcac073fb0 0x7fdcac1f6720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47372/0 (socket says 172.21.15.151:47372) 2024-09-16T15:39:32.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.417+0000 7fdcb1a00640 1 -- 172.21.15.151:0/2127845576 learned_addr learned my addr 172.21.15.151:0/2127845576 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:32.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.417+0000 7fdcb1000640 1 --2- 172.21.15.151:0/2127845576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdcac0ccf00 0x7fdcac1574d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:32.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.417+0000 7fdcb2400640 1 --2- 172.21.15.151:0/2127845576 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdcac15f5a0 0x7fdcac1f1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:32.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.417+0000 7fdcb1a00640 1 -- 172.21.15.151:0/2127845576 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdcac15f5a0 msgr2=0x7fdcac1f1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:32.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.417+0000 7fdcb1a00640 1 --2- 172.21.15.151:0/2127845576 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdcac15f5a0 0x7fdcac1f1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:32.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.417+0000 7fdcb1a00640 1 -- 172.21.15.151:0/2127845576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdcac0ccf00 msgr2=0x7fdcac1574d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:32.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.417+0000 7fdcb1a00640 1 --2- 172.21.15.151:0/2127845576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdcac0ccf00 0x7fdcac1574d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:32.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.417+0000 7fdcb1a00640 1 -- 172.21.15.151:0/2127845576 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcac1f6c60 con 0x7fdcac073fb0 2024-09-16T15:39:32.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.417+0000 7fdcb2400640 1 --2- 172.21.15.151:0/2127845576 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdcac15f5a0 0x7fdcac1f1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:39:32.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.418+0000 7fdcb1a00640 1 --2- 172.21.15.151:0/2127845576 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdcac073fb0 0x7fdcac1f6720 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdca000c400 tx=0x7fdca000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:32.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.418+0000 7fdca9600640 1 -- 172.21.15.151:0/2127845576 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca0011790 con 0x7fdcac073fb0 2024-09-16T15:39:32.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.418+0000 7fdca9600640 1 -- 172.21.15.151:0/2127845576 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdca0017070 con 0x7fdcac073fb0 2024-09-16T15:39:32.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.418+0000 7fdca9600640 1 -- 172.21.15.151:0/2127845576 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca0011cc0 con 0x7fdcac073fb0 2024-09-16T15:39:32.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.418+0000 7fdcb4400640 1 -- 172.21.15.151:0/2127845576 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcac157a10 con 0x7fdcac073fb0 2024-09-16T15:39:32.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.418+0000 7fdcb4400640 1 -- 172.21.15.151:0/2127845576 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcac156710 con 0x7fdcac073fb0 2024-09-16T15:39:32.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.419+0000 7fdcb4400640 1 -- 172.21.15.151:0/2127845576 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcac0b2440 con 0x7fdcac073fb0 2024-09-16T15:39:32.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.420+0000 7fdca9600640 1 -- 172.21.15.151:0/2127845576 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fdca001c460 con 0x7fdcac073fb0 2024-09-16T15:39:32.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.421+0000 7fdca9600640 1 --2- 172.21.15.151:0/2127845576 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdc8c078850 0x7fdc8c07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:32.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.421+0000 7fdca9600640 1 -- 172.21.15.151:0/2127845576 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fdca009a720 con 0x7fdcac073fb0 2024-09-16T15:39:32.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.422+0000 7fdcb1000640 1 --2- 172.21.15.151:0/2127845576 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdc8c078850 0x7fdc8c07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:32.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.422+0000 7fdcb1000640 1 --2- 172.21.15.151:0/2127845576 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdc8c078850 0x7fdc8c07ad10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fdc98034000 tx=0x7fdc98004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:32.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.425+0000 7fdca9600640 1 -- 172.21.15.151:0/2127845576 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fdcac1f6c60 con 0x7fdcac073fb0 2024-09-16T15:39:32.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.657+0000 7fdcb4400640 1 -- 172.21.15.151:0/2127845576 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fdcac0b12d0 con 0x7fdcac073fb0 2024-09-16T15:39:32.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.661+0000 7fdca9600640 1 -- 172.21.15.151:0/2127845576 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+194343 (secure 0 0 0) 0x7fdca0062d80 con 0x7fdcac073fb0 2024-09-16T15:39:32.664 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:39:32.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.664+0000 7fdcb4400640 1 -- 172.21.15.151:0/2127845576 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdc8c078850 msgr2=0x7fdc8c07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:32.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.664+0000 7fdcb4400640 1 --2- 172.21.15.151:0/2127845576 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdc8c078850 0x7fdc8c07ad10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fdc98034000 tx=0x7fdc98004730 comp rx=0 tx=0).stop 2024-09-16T15:39:32.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.664+0000 7fdcb4400640 1 -- 172.21.15.151:0/2127845576 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdcac073fb0 msgr2=0x7fdcac1f6720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:32.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.664+0000 7fdcb4400640 1 --2- 172.21.15.151:0/2127845576 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdcac073fb0 0x7fdcac1f6720 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdca000c400 tx=0x7fdca000c8c0 comp rx=0 tx=0).stop 2024-09-16T15:39:32.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.664+0000 7fdcb4400640 1 -- 172.21.15.151:0/2127845576 shutdown_connections 2024-09-16T15:39:32.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.664+0000 7fdcb4400640 1 --2- 172.21.15.151:0/2127845576 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdc8c078850 0x7fdc8c07ad10 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:32.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.664+0000 7fdcb4400640 1 --2- 172.21.15.151:0/2127845576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdcac0ccf00 0x7fdcac1574d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:32.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.664+0000 7fdcb4400640 1 --2- 172.21.15.151:0/2127845576 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdcac15f5a0 0x7fdcac1f1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:32.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.664+0000 7fdcb4400640 1 --2- 172.21.15.151:0/2127845576 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdcac073fb0 0x7fdcac1f6720 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:32.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.664+0000 7fdcb4400640 1 -- 172.21.15.151:0/2127845576 >> 172.21.15.151:0/2127845576 conn(0x7fdcac0b1950 msgr2=0x7fdcac15cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:32.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.665+0000 7fdcb4400640 1 -- 172.21.15.151:0/2127845576 shutdown_connections 2024-09-16T15:39:32.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.665+0000 7fdcb4400640 1 -- 172.21.15.151:0/2127845576 wait complete. 2024-09-16T15:39:32.681 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":5,"flags":0,"active_gid":4123,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":501171453},{"type":"v1","addr":"172.21.15.167:6801","nonce":501171453}]},"active_addr":"172.21.15.167:6801/501171453","active_change":"2024-09-16T15:39:01.565106+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":4115,"name":"y","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.167:0","nonce":863323752}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.167:0","nonce":2738812568}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.167:0","nonce":236308730}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.167:0","nonce":2262229134}]}]} 2024-09-16T15:39:32.685 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-16T15:39:32.685 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-16T15:39:32.685 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:39:32.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.842+0000 7fd8d2c00640 1 -- 172.21.15.151:0/3610046138 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8cc0b2780 msgr2=0x7fd8cc168c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:32.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.842+0000 7fd8d2c00640 1 --2- 172.21.15.151:0/3610046138 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8cc0b2780 0x7fd8cc168c30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc0d4ea0 tx=0x7fd8c402a890 comp rx=0 tx=0).stop 2024-09-16T15:39:32.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.842+0000 7fd8d2c00640 1 -- 172.21.15.151:0/3610046138 shutdown_connections 2024-09-16T15:39:32.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.842+0000 7fd8d2c00640 1 --2- 172.21.15.151:0/3610046138 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8cc0b2780 0x7fd8cc168c30 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:32.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.842+0000 7fd8d2c00640 1 -- 172.21.15.151:0/3610046138 >> 172.21.15.151:0/3610046138 conn(0x7fd8cc0d3df0 msgr2=0x7fd8cc0d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:32.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.842+0000 7fd8d2c00640 1 -- 172.21.15.151:0/3610046138 shutdown_connections 2024-09-16T15:39:32.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.842+0000 7fd8d2c00640 1 -- 172.21.15.151:0/3610046138 wait complete. 2024-09-16T15:39:32.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.843+0000 7fd8d2c00640 1 Processor -- start 2024-09-16T15:39:32.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.844+0000 7fd8d2c00640 1 -- start start 2024-09-16T15:39:32.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.844+0000 7fd8d2c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8cc0b2780 0x7fd8cc1520f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:32.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.844+0000 7fd8d2c00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8cc0b5370 0x7fd8cc15ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:32.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.845+0000 7fd8d2c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8cc161440 0x7fd8cc14f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:32.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.845+0000 7fd8d2c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd8cc1692f0 con 0x7fd8cc161440 2024-09-16T15:39:32.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.845+0000 7fd8d2c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd8cc169170 con 0x7fd8cc0b2780 2024-09-16T15:39:32.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.845+0000 7fd8d2c00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fd8cc169470 con 0x7fd8cc0b5370 2024-09-16T15:39:32.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.845+0000 7fd8d2200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8cc161440 0x7fd8cc14f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:32.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.845+0000 7fd8d2200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8cc161440 0x7fd8cc14f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:56978/0 (socket says 172.21.15.151:56978) 2024-09-16T15:39:32.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.845+0000 7fd8d2200640 1 -- 172.21.15.151:0/2046955922 learned_addr learned my addr 172.21.15.151:0/2046955922 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:32.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.845+0000 7fd8d0e00640 1 --2- 172.21.15.151:0/2046955922 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8cc0b5370 0x7fd8cc15ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:32.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.845+0000 7fd8d1800640 1 --2- 172.21.15.151:0/2046955922 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8cc0b2780 0x7fd8cc1520f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:32.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.845+0000 7fd8d2200640 1 -- 172.21.15.151:0/2046955922 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8cc0b5370 msgr2=0x7fd8cc15ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:32.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.845+0000 7fd8d2200640 1 --2- 172.21.15.151:0/2046955922 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8cc0b5370 0x7fd8cc15ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:32.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.845+0000 7fd8d2200640 1 -- 172.21.15.151:0/2046955922 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8cc0b2780 msgr2=0x7fd8cc1520f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:32.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.845+0000 7fd8d2200640 1 --2- 172.21.15.151:0/2046955922 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8cc0b2780 0x7fd8cc1520f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:32.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.845+0000 7fd8d2200640 1 -- 172.21.15.151:0/2046955922 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8cc15bfe0 con 0x7fd8cc161440 2024-09-16T15:39:32.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.846+0000 7fd8d2200640 1 --2- 172.21.15.151:0/2046955922 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8cc161440 0x7fd8cc14f2d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd8c402ae80 tx=0x7fd8c4003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:32.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.846+0000 7fd8c9600640 1 -- 172.21.15.151:0/2046955922 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c4031d80 con 0x7fd8cc161440 2024-09-16T15:39:32.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.846+0000 7fd8c9600640 1 -- 172.21.15.151:0/2046955922 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd8c4004130 con 0x7fd8cc161440 2024-09-16T15:39:32.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.846+0000 7fd8d2c00640 1 -- 172.21.15.151:0/2046955922 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8cc159de0 con 0x7fd8cc161440 2024-09-16T15:39:32.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.846+0000 7fd8c9600640 1 -- 172.21.15.151:0/2046955922 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c4004560 con 0x7fd8cc161440 2024-09-16T15:39:32.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.846+0000 7fd8d2c00640 1 -- 172.21.15.151:0/2046955922 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8cc15a3a0 con 0x7fd8cc161440 2024-09-16T15:39:32.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.847+0000 7fd8d2c00640 1 -- 172.21.15.151:0/2046955922 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8cc0b34b0 con 0x7fd8cc161440 2024-09-16T15:39:32.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.848+0000 7fd8c9600640 1 -- 172.21.15.151:0/2046955922 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fd8c4039040 con 0x7fd8cc161440 2024-09-16T15:39:32.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.849+0000 7fd8c9600640 1 --2- 172.21.15.151:0/2046955922 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd8ac078850 0x7fd8ac07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:32.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.849+0000 7fd8c9600640 1 -- 172.21.15.151:0/2046955922 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fd8c40bb9e0 con 0x7fd8cc161440 2024-09-16T15:39:32.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.850+0000 7fd8d1800640 1 --2- 172.21.15.151:0/2046955922 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd8ac078850 0x7fd8ac07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:32.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.850+0000 7fd8d1800640 1 --2- 172.21.15.151:0/2046955922 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd8ac078850 0x7fd8ac07ad10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0000f60 tx=0x7fd8c0009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:32.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:32.853+0000 7fd8c9600640 1 -- 172.21.15.151:0/2046955922 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd8cc15bfe0 con 0x7fd8cc161440 2024-09-16T15:39:33.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.047+0000 7fd8d2c00640 1 -- 172.21.15.151:0/2046955922 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd8cc0760b0 con 0x7fd8cc161440 2024-09-16T15:39:33.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.048+0000 7fd8c9600640 1 -- 172.21.15.151:0/2046955922 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16729 (secure 0 0 0) 0x7fd8c4040030 con 0x7fd8cc161440 2024-09-16T15:39:33.049 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:39:33.049 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":22,"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","created":"2024-09-16T15:38:56.348535+0000","modified":"2024-09-16T15:39:23.550834+0000","last_up_change":"2024-09-16T15:39:10.681384+0000","last_in_change":"2024-09-16T15:39:00.544477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:39:11.602252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:39:19.970182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T15:39:20.929376+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5524076-a903-4d0a-af6a-caf474c4765d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6809","nonce":3111342741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6811","nonce":3111342741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6815","nonce":3111342741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6813","nonce":3111342741}]},"public_addr":"172.21.15.151:6809/3111342741","cluster_addr":"172.21.15.151:6811/3111342741","heartbeat_back_addr":"172.21.15.151:6815/3111342741","heartbeat_front_addr":"172.21.15.151:6813/3111342741","state":["exists","up"]},{"osd":1,"uuid":"dd25925f-0ae1-494e-8529-4dc205759db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6801","nonce":3368304087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6803","nonce":3368304087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6807","nonce":3368304087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6805","nonce":3368304087}]},"public_addr":"172.21.15.151:6801/3368304087","cluster_addr":"172.21.15.151:6803/3368304087","heartbeat_back_addr":"172.21.15.151:6807/3368304087","heartbeat_front_addr":"172.21.15.151:6805/3368304087","state":["exists","up"]},{"osd":2,"uuid":"c2f18115-013d-4da6-a0fe-cb978f4494b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6823","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6825","nonce":3079373271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6827","nonce":3079373271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6831","nonce":3079373271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6829","nonce":3079373271}]},"public_addr":"172.21.15.151:6825/3079373271","cluster_addr":"172.21.15.151:6827/3079373271","heartbeat_back_addr":"172.21.15.151:6831/3079373271","heartbeat_front_addr":"172.21.15.151:6829/3079373271","state":["exists","up"]},{"osd":3,"uuid":"36edb718-2fcb-4d88-8ddc-c321eed4279d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6817","nonce":1142509041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6819","nonce":1142509041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6824","nonce":1142509041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6821","nonce":1142509041}]},"public_addr":"172.21.15.151:6817/1142509041","cluster_addr":"172.21.15.151:6819/1142509041","heartbeat_back_addr":"172.21.15.151:6824/1142509041","heartbeat_front_addr":"172.21.15.151:6821/1142509041","state":["exists","up"]},{"osd":4,"uuid":"64d33708-cd0a-4236-abe3-1b646e82ab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6811","nonce":2735495271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6813","nonce":2735495271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6817","nonce":2735495271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6815","nonce":2735495271}]},"public_addr":"172.21.15.167:6811/2735495271","cluster_addr":"172.21.15.167:6813/2735495271","heartbeat_back_addr":"172.21.15.167:6817/2735495271","heartbeat_front_addr":"172.21.15.167:6815/2735495271","state":["exists","up"]},{"osd":5,"uuid":"f14b44a9-5910-458c-b518-09a8d0de94de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6827","nonce":1494916572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6829","nonce":1494916572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6833","nonce":1494916572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6831","nonce":1494916572}]},"public_addr":"172.21.15.167:6827/1494916572","cluster_addr":"172.21.15.167:6829/1494916572","heartbeat_back_addr":"172.21.15.167:6833/1494916572","heartbeat_front_addr":"172.21.15.167:6831/1494916572","state":["exists","up"]},{"osd":6,"uuid":"39335545-2654-4710-84b0-a9aa47cfcbba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6819","nonce":2538273273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6821","nonce":2538273273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6825","nonce":2538273273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6823","nonce":2538273273}]},"public_addr":"172.21.15.167:6819/2538273273","cluster_addr":"172.21.15.167:6821/2538273273","heartbeat_back_addr":"172.21.15.167:6825/2538273273","heartbeat_front_addr":"172.21.15.167:6823/2538273273","state":["exists","up"]},{"osd":7,"uuid":"4f8bfc81-53dd-4339-ac22-1a4007d9f963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6803","nonce":265072227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6805","nonce":265072227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6809","nonce":265072227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6807","nonce":265072227}]},"public_addr":"172.21.15.167:6803/265072227","cluster_addr":"172.21.15.167:6805/265072227","heartbeat_back_addr":"172.21.15.167:6809/265072227","heartbeat_front_addr":"172.21.15.167:6807/265072227","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.150359+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.084358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.722368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.332809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.320709+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.395153+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.935250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.801443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:39:33.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.050+0000 7fd8d2c00640 1 -- 172.21.15.151:0/2046955922 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd8ac078850 msgr2=0x7fd8ac07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:33.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.050+0000 7fd8d2c00640 1 --2- 172.21.15.151:0/2046955922 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd8ac078850 0x7fd8ac07ad10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0000f60 tx=0x7fd8c0009040 comp rx=0 tx=0).stop 2024-09-16T15:39:33.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.050+0000 7fd8d2c00640 1 -- 172.21.15.151:0/2046955922 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8cc161440 msgr2=0x7fd8cc14f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:33.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.050+0000 7fd8d2c00640 1 --2- 172.21.15.151:0/2046955922 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8cc161440 0x7fd8cc14f2d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd8c402ae80 tx=0x7fd8c4003f10 comp rx=0 tx=0).stop 2024-09-16T15:39:33.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.050+0000 7fd8d2c00640 1 -- 172.21.15.151:0/2046955922 shutdown_connections 2024-09-16T15:39:33.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.050+0000 7fd8d2c00640 1 --2- 172.21.15.151:0/2046955922 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd8ac078850 0x7fd8ac07ad10 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.050+0000 7fd8d2c00640 1 --2- 172.21.15.151:0/2046955922 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8cc161440 0x7fd8cc14f2d0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.050+0000 7fd8d2c00640 1 --2- 172.21.15.151:0/2046955922 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8cc0b5370 0x7fd8cc15ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.050+0000 7fd8d2c00640 1 --2- 172.21.15.151:0/2046955922 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8cc0b2780 0x7fd8cc1520f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.050+0000 7fd8d2c00640 1 -- 172.21.15.151:0/2046955922 >> 172.21.15.151:0/2046955922 conn(0x7fd8cc0d3df0 msgr2=0x7fd8cc1601d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:33.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.050+0000 7fd8d2c00640 1 -- 172.21.15.151:0/2046955922 shutdown_connections 2024-09-16T15:39:33.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.050+0000 7fd8d2c00640 1 -- 172.21.15.151:0/2046955922 wait complete. 2024-09-16T15:39:33.067 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-16T15:39:33.068 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:39:33.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.223+0000 7f5d79000640 1 -- 172.21.15.151:0/927010074 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d740ccf00 msgr2=0x7f5d740b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:33.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.223+0000 7f5d79000640 1 --2- 172.21.15.151:0/927010074 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d740ccf00 0x7f5d740b2440 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5d60005fd0 tx=0x7f5d6002a540 comp rx=0 tx=0).stop 2024-09-16T15:39:33.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.223+0000 7f5d79000640 1 -- 172.21.15.151:0/927010074 shutdown_connections 2024-09-16T15:39:33.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.223+0000 7f5d79000640 1 --2- 172.21.15.151:0/927010074 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d74073fb0 0x7f5d74074440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.223+0000 7f5d79000640 1 --2- 172.21.15.151:0/927010074 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d740ccf00 0x7f5d740b2440 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.223+0000 7f5d79000640 1 -- 172.21.15.151:0/927010074 >> 172.21.15.151:0/927010074 conn(0x7f5d740b1950 msgr2=0x7f5d740b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:33.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.224+0000 7f5d79000640 1 -- 172.21.15.151:0/927010074 shutdown_connections 2024-09-16T15:39:33.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.224+0000 7f5d79000640 1 -- 172.21.15.151:0/927010074 wait complete. 2024-09-16T15:39:33.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.225+0000 7f5d79000640 1 Processor -- start 2024-09-16T15:39:33.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.225+0000 7f5d79000640 1 -- start start 2024-09-16T15:39:33.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.226+0000 7f5d79000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d74073fb0 0x7f5d7415bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:33.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.226+0000 7f5d79000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5d740ccf00 0x7f5d740cf8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:33.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.226+0000 7f5d79000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d7415f5a0 0x7f5d740cfe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:33.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.226+0000 7f5d79000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5d7406f2e0 con 0x7f5d740ccf00 2024-09-16T15:39:33.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.226+0000 7f5d79000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5d7406f160 con 0x7f5d74073fb0 2024-09-16T15:39:33.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.226+0000 7f5d79000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f5d7406f460 con 0x7f5d7415f5a0 2024-09-16T15:39:33.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.226+0000 7f5d71400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5d740ccf00 0x7f5d740cf8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:33.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.226+0000 7f5d71400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5d740ccf00 0x7f5d740cf8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:56990/0 (socket says 172.21.15.151:56990) 2024-09-16T15:39:33.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.226+0000 7f5d71400640 1 -- 172.21.15.151:0/1982276247 learned_addr learned my addr 172.21.15.151:0/1982276247 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:33.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.227+0000 7f5d71e00640 1 --2- 172.21.15.151:0/1982276247 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d74073fb0 0x7f5d7415bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:33.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.227+0000 7f5d72800640 1 --2- 172.21.15.151:0/1982276247 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d7415f5a0 0x7f5d740cfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:33.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.227+0000 7f5d71400640 1 -- 172.21.15.151:0/1982276247 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d7415f5a0 msgr2=0x7f5d740cfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:33.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.227+0000 7f5d71400640 1 --2- 172.21.15.151:0/1982276247 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d7415f5a0 0x7f5d740cfe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.227+0000 7f5d71400640 1 -- 172.21.15.151:0/1982276247 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d74073fb0 msgr2=0x7f5d7415bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:33.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.227+0000 7f5d71400640 1 --2- 172.21.15.151:0/1982276247 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d74073fb0 0x7f5d7415bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.227+0000 7f5d71400640 1 -- 172.21.15.151:0/1982276247 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d741593d0 con 0x7f5d740ccf00 2024-09-16T15:39:33.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.227+0000 7f5d71400640 1 --2- 172.21.15.151:0/1982276247 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5d740ccf00 0x7f5d740cf8f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5d60009850 tx=0x7f5d600038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:33.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.228+0000 7f5d6a000640 1 -- 172.21.15.151:0/1982276247 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d6003f070 con 0x7f5d740ccf00 2024-09-16T15:39:33.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.228+0000 7f5d6a000640 1 -- 172.21.15.151:0/1982276247 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5d600047a0 con 0x7f5d740ccf00 2024-09-16T15:39:33.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.228+0000 7f5d6a000640 1 -- 172.21.15.151:0/1982276247 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d60035030 con 0x7f5d740ccf00 2024-09-16T15:39:33.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.228+0000 7f5d79000640 1 -- 172.21.15.151:0/1982276247 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d740ce390 con 0x7f5d740ccf00 2024-09-16T15:39:33.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.228+0000 7f5d79000640 1 -- 172.21.15.151:0/1982276247 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d740ce950 con 0x7f5d740ccf00 2024-09-16T15:39:33.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.229+0000 7f5d79000640 1 -- 172.21.15.151:0/1982276247 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d7406f980 con 0x7f5d740ccf00 2024-09-16T15:39:33.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.230+0000 7f5d6a000640 1 -- 172.21.15.151:0/1982276247 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f5d60042070 con 0x7f5d740ccf00 2024-09-16T15:39:33.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.231+0000 7f5d6a000640 1 --2- 172.21.15.151:0/1982276247 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5d54078850 0x7f5d5407ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:33.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.231+0000 7f5d6a000640 1 -- 172.21.15.151:0/1982276247 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f5d600bbaf0 con 0x7f5d740ccf00 2024-09-16T15:39:33.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.231+0000 7f5d71e00640 1 --2- 172.21.15.151:0/1982276247 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5d54078850 0x7f5d5407ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:33.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.232+0000 7f5d71e00640 1 --2- 172.21.15.151:0/1982276247 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5d54078850 0x7f5d5407ad10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5d64001e20 tx=0x7f5d64009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:33.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.235+0000 7f5d6a000640 1 -- 172.21.15.151:0/1982276247 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f5d60083a70 con 0x7f5d740ccf00 2024-09-16T15:39:33.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.420+0000 7f5d79000640 1 -- 172.21.15.151:0/1982276247 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5d740b25e0 con 0x7f5d740ccf00 2024-09-16T15:39:33.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.421+0000 7f5d6a000640 1 -- 172.21.15.151:0/1982276247 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16729 (secure 0 0 0) 0x7f5d600884a0 con 0x7f5d740ccf00 2024-09-16T15:39:33.422 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:39:33.423 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":22,"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","created":"2024-09-16T15:38:56.348535+0000","modified":"2024-09-16T15:39:23.550834+0000","last_up_change":"2024-09-16T15:39:10.681384+0000","last_in_change":"2024-09-16T15:39:00.544477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:39:11.602252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:39:19.970182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T15:39:20.929376+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5524076-a903-4d0a-af6a-caf474c4765d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6809","nonce":3111342741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6811","nonce":3111342741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6815","nonce":3111342741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6813","nonce":3111342741}]},"public_addr":"172.21.15.151:6809/3111342741","cluster_addr":"172.21.15.151:6811/3111342741","heartbeat_back_addr":"172.21.15.151:6815/3111342741","heartbeat_front_addr":"172.21.15.151:6813/3111342741","state":["exists","up"]},{"osd":1,"uuid":"dd25925f-0ae1-494e-8529-4dc205759db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6801","nonce":3368304087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6803","nonce":3368304087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6807","nonce":3368304087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6805","nonce":3368304087}]},"public_addr":"172.21.15.151:6801/3368304087","cluster_addr":"172.21.15.151:6803/3368304087","heartbeat_back_addr":"172.21.15.151:6807/3368304087","heartbeat_front_addr":"172.21.15.151:6805/3368304087","state":["exists","up"]},{"osd":2,"uuid":"c2f18115-013d-4da6-a0fe-cb978f4494b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6823","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6825","nonce":3079373271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6827","nonce":3079373271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6831","nonce":3079373271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6829","nonce":3079373271}]},"public_addr":"172.21.15.151:6825/3079373271","cluster_addr":"172.21.15.151:6827/3079373271","heartbeat_back_addr":"172.21.15.151:6831/3079373271","heartbeat_front_addr":"172.21.15.151:6829/3079373271","state":["exists","up"]},{"osd":3,"uuid":"36edb718-2fcb-4d88-8ddc-c321eed4279d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6817","nonce":1142509041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6819","nonce":1142509041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6824","nonce":1142509041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6821","nonce":1142509041}]},"public_addr":"172.21.15.151:6817/1142509041","cluster_addr":"172.21.15.151:6819/1142509041","heartbeat_back_addr":"172.21.15.151:6824/1142509041","heartbeat_front_addr":"172.21.15.151:6821/1142509041","state":["exists","up"]},{"osd":4,"uuid":"64d33708-cd0a-4236-abe3-1b646e82ab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6811","nonce":2735495271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6813","nonce":2735495271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6817","nonce":2735495271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6815","nonce":2735495271}]},"public_addr":"172.21.15.167:6811/2735495271","cluster_addr":"172.21.15.167:6813/2735495271","heartbeat_back_addr":"172.21.15.167:6817/2735495271","heartbeat_front_addr":"172.21.15.167:6815/2735495271","state":["exists","up"]},{"osd":5,"uuid":"f14b44a9-5910-458c-b518-09a8d0de94de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6827","nonce":1494916572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6829","nonce":1494916572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6833","nonce":1494916572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6831","nonce":1494916572}]},"public_addr":"172.21.15.167:6827/1494916572","cluster_addr":"172.21.15.167:6829/1494916572","heartbeat_back_addr":"172.21.15.167:6833/1494916572","heartbeat_front_addr":"172.21.15.167:6831/1494916572","state":["exists","up"]},{"osd":6,"uuid":"39335545-2654-4710-84b0-a9aa47cfcbba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6819","nonce":2538273273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6821","nonce":2538273273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6825","nonce":2538273273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6823","nonce":2538273273}]},"public_addr":"172.21.15.167:6819/2538273273","cluster_addr":"172.21.15.167:6821/2538273273","heartbeat_back_addr":"172.21.15.167:6825/2538273273","heartbeat_front_addr":"172.21.15.167:6823/2538273273","state":["exists","up"]},{"osd":7,"uuid":"4f8bfc81-53dd-4339-ac22-1a4007d9f963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6803","nonce":265072227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6805","nonce":265072227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6809","nonce":265072227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6807","nonce":265072227}]},"public_addr":"172.21.15.167:6803/265072227","cluster_addr":"172.21.15.167:6805/265072227","heartbeat_back_addr":"172.21.15.167:6809/265072227","heartbeat_front_addr":"172.21.15.167:6807/265072227","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.150359+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.084358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.722368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.332809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.320709+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.395153+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.935250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.801443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:39:33.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.424+0000 7f5d79000640 1 -- 172.21.15.151:0/1982276247 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5d54078850 msgr2=0x7f5d5407ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:33.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.424+0000 7f5d79000640 1 --2- 172.21.15.151:0/1982276247 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5d54078850 0x7f5d5407ad10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5d64001e20 tx=0x7f5d64009040 comp rx=0 tx=0).stop 2024-09-16T15:39:33.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.424+0000 7f5d79000640 1 -- 172.21.15.151:0/1982276247 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5d740ccf00 msgr2=0x7f5d740cf8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:33.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.424+0000 7f5d79000640 1 --2- 172.21.15.151:0/1982276247 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5d740ccf00 0x7f5d740cf8f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5d60009850 tx=0x7f5d600038a0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.424+0000 7f5d79000640 1 -- 172.21.15.151:0/1982276247 shutdown_connections 2024-09-16T15:39:33.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.424+0000 7f5d79000640 1 --2- 172.21.15.151:0/1982276247 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5d54078850 0x7f5d5407ad10 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.424+0000 7f5d79000640 1 --2- 172.21.15.151:0/1982276247 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5d7415f5a0 0x7f5d740cfe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.424+0000 7f5d79000640 1 --2- 172.21.15.151:0/1982276247 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5d740ccf00 0x7f5d740cf8f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.424+0000 7f5d79000640 1 --2- 172.21.15.151:0/1982276247 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d74073fb0 0x7f5d7415bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.424+0000 7f5d79000640 1 -- 172.21.15.151:0/1982276247 >> 172.21.15.151:0/1982276247 conn(0x7f5d740b1950 msgr2=0x7f5d740b0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:33.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.424+0000 7f5d79000640 1 -- 172.21.15.151:0/1982276247 shutdown_connections 2024-09-16T15:39:33.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.424+0000 7f5d79000640 1 -- 172.21.15.151:0/1982276247 wait complete. 2024-09-16T15:39:33.441 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-16T15:39:33.441 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-16T15:39:33.442 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-16T15:39:33.442 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-16T15:39:33.443 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-16T15:39:33.443 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-16T15:39:33.443 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-16T15:39:33.444 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-16T15:39:33.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.717+0000 7f0796600640 1 -- 172.21.15.151:0/58413831 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f07900cdfb0 msgr2=0x7f07900cfc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:33.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.717+0000 7f0796600640 1 --2- 172.21.15.151:0/58413831 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f07900cdfb0 0x7f07900cfc40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0780009830 tx=0x7f078002a440 comp rx=0 tx=0).stop 2024-09-16T15:39:33.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.719+0000 7f0796600640 1 -- 172.21.15.151:0/58413831 shutdown_connections 2024-09-16T15:39:33.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.719+0000 7f0796600640 1 --2- 172.21.15.151:0/58413831 >> v2:172.21.15.151:3300/0 conn(0x7f079015b0c0 0x7f079015b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.719+0000 7f0796600640 1 --2- 172.21.15.151:0/58413831 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f07900cdfb0 0x7f07900cfc40 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.719+0000 7f0796600640 1 -- 172.21.15.151:0/58413831 >> 172.21.15.151:0/58413831 conn(0x7f07900b2b90 msgr2=0x7f07900b2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:33.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.719+0000 7f0796600640 1 -- 172.21.15.151:0/58413831 shutdown_connections 2024-09-16T15:39:33.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.719+0000 7f0796600640 1 -- 172.21.15.151:0/58413831 wait complete. 2024-09-16T15:39:33.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.722+0000 7f0796600640 1 Processor -- start 2024-09-16T15:39:33.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.722+0000 7f0796600640 1 -- start start 2024-09-16T15:39:33.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.723+0000 7f0796600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07900cdfb0 0x7f079014ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:33.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.723+0000 7f0796600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f079015b0c0 0x7f07902047b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:33.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.724+0000 7f0796600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f079015f220 0x7f0790204cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:33.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.724+0000 7f0796600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0790070970 con 0x7f079015f220 2024-09-16T15:39:33.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.724+0000 7f0796600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f07900707f0 con 0x7f07900cdfb0 2024-09-16T15:39:33.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.724+0000 7f0796600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f0790070af0 con 0x7f079015b0c0 2024-09-16T15:39:33.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.724+0000 7f078f400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07900cdfb0 0x7f079014ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:33.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.724+0000 7f078f400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07900cdfb0 0x7f079014ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.151:42494/0 (socket says 172.21.15.151:42494) 2024-09-16T15:39:33.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.725+0000 7f078f400640 1 -- 172.21.15.151:0/1513321303 learned_addr learned my addr 172.21.15.151:0/1513321303 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:33.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.725+0000 7f078fe00640 1 --2- 172.21.15.151:0/1513321303 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f079015f220 0x7f0790204cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:33.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.725+0000 7f078ea00640 1 --2- 172.21.15.151:0/1513321303 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f079015b0c0 0x7f07902047b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:33.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.726+0000 7f078ea00640 1 -- 172.21.15.151:0/1513321303 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07900cdfb0 msgr2=0x7f079014ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:33.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.726+0000 7f078ea00640 1 --2- 172.21.15.151:0/1513321303 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07900cdfb0 0x7f079014ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.726+0000 7f078ea00640 1 -- 172.21.15.151:0/1513321303 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f079015f220 msgr2=0x7f0790204cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:33.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.726+0000 7f078ea00640 1 --2- 172.21.15.151:0/1513321303 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f079015f220 0x7f0790204cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.726+0000 7f078ea00640 1 -- 172.21.15.151:0/1513321303 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0790153c00 con 0x7f079015b0c0 2024-09-16T15:39:33.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.727+0000 7f078fe00640 1 --2- 172.21.15.151:0/1513321303 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f079015f220 0x7f0790204cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:39:33.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.728+0000 7f078ea00640 1 --2- 172.21.15.151:0/1513321303 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f079015b0c0 0x7f07902047b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0780031000 tx=0x7f0780033c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:33.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.728+0000 7f0787400640 1 -- 172.21.15.151:0/1513321303 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f078003f020 con 0x7f079015b0c0 2024-09-16T15:39:33.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.728+0000 7f0787400640 1 -- 172.21.15.151:0/1513321303 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0780033e10 con 0x7f079015b0c0 2024-09-16T15:39:33.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.728+0000 7f0787400640 1 -- 172.21.15.151:0/1513321303 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0780037040 con 0x7f079015b0c0 2024-09-16T15:39:33.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.729+0000 7f0796600640 1 -- 172.21.15.151:0/1513321303 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0790152e40 con 0x7f079015b0c0 2024-09-16T15:39:33.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.730+0000 7f0796600640 1 -- 172.21.15.151:0/1513321303 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0790153400 con 0x7f079015b0c0 2024-09-16T15:39:33.732 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.731+0000 7f0796600640 1 -- 172.21.15.151:0/1513321303 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0790152bf0 con 0x7f079015b0c0 2024-09-16T15:39:33.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.732+0000 7ff8c1200640 1 -- 172.21.15.151:0/3478542837 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8b40e27a0 msgr2=0x7ff8b40e6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:33.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.732+0000 7f570e600640 1 -- 172.21.15.151:0/2515867304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57000e2a60 msgr2=0x7f5700037940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:33.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.732+0000 7f570e600640 1 --2- 172.21.15.151:0/2515867304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57000e2a60 0x7f5700037940 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5700038100 tx=0x7f56f402a6f0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.732+0000 7ff8c1200640 1 --2- 172.21.15.151:0/3478542837 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8b40e27a0 0x7ff8b40e6430 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff8b400f2c0 tx=0x7ff8ac02a420 comp rx=0 tx=0).stop 2024-09-16T15:39:33.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.732+0000 7f570e600640 1 -- 172.21.15.151:0/2515867304 shutdown_connections 2024-09-16T15:39:33.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.732+0000 7ff8c1200640 1 -- 172.21.15.151:0/3478542837 shutdown_connections 2024-09-16T15:39:33.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.732+0000 7ff8c1200640 1 --2- 172.21.15.151:0/3478542837 >> v2:172.21.15.151:3300/0 conn(0x7ff8b4037f10 0x7ff8b40e6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.732+0000 7ff8c1200640 1 --2- 172.21.15.151:0/3478542837 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8b40e27a0 0x7ff8b40e6430 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.732+0000 7ff8c1200640 1 -- 172.21.15.151:0/3478542837 >> 172.21.15.151:0/3478542837 conn(0x7ff8b403a3d0 msgr2=0x7ff8b403a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:33.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.732+0000 7f570e600640 1 --2- 172.21.15.151:0/2515867304 >> v2:172.21.15.151:3300/0 conn(0x7f57000e39e0 0x7f570003a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.732+0000 7f570e600640 1 --2- 172.21.15.151:0/2515867304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57000e2a60 0x7f5700037940 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.732+0000 7f570e600640 1 -- 172.21.15.151:0/2515867304 >> 172.21.15.151:0/2515867304 conn(0x7f5700055540 msgr2=0x7f5700055920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:33.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.733+0000 7ff8c1200640 1 -- 172.21.15.151:0/3478542837 shutdown_connections 2024-09-16T15:39:33.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.734+0000 7f570e600640 1 -- 172.21.15.151:0/2515867304 shutdown_connections 2024-09-16T15:39:33.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.735+0000 7f0787400640 1 -- 172.21.15.151:0/1513321303 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f0780042020 con 0x7f079015b0c0 2024-09-16T15:39:33.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.733+0000 7ff8c1200640 1 -- 172.21.15.151:0/3478542837 wait complete. 2024-09-16T15:39:33.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.734+0000 7f570e600640 1 -- 172.21.15.151:0/2515867304 wait complete. 2024-09-16T15:39:33.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.735+0000 7f4eae200640 1 -- 172.21.15.151:0/2496695953 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e8c003a00 msgr2=0x7f4e8c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:33.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.735+0000 7f4eae200640 1 --2- 172.21.15.151:0/2496695953 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e8c003a00 0x7f4e8c003e00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4e9800c2e0 tx=0x7f4e8c005830 comp rx=0 tx=0).stop 2024-09-16T15:39:33.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.735+0000 7f4eae200640 1 -- 172.21.15.151:0/2496695953 shutdown_connections 2024-09-16T15:39:33.738 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.735+0000 7f4eae200640 1 --2- 172.21.15.151:0/2496695953 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e8c003a00 0x7f4e8c003e00 secure :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4e9800c2e0 tx=0x7f4e8c005830 comp rx=0 tx=0).stop 2024-09-16T15:39:33.738 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.735+0000 7f4eae200640 1 -- 172.21.15.151:0/2496695953 >> 172.21.15.151:0/2496695953 conn(0x7f4ea80d3fe0 msgr2=0x7f4ea80d43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:33.738 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.736+0000 7f4eae200640 1 -- 172.21.15.151:0/2496695953 shutdown_connections 2024-09-16T15:39:33.738 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.736+0000 7ff8c1200640 1 Processor -- start 2024-09-16T15:39:33.738 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.736+0000 7f4eae200640 1 -- 172.21.15.151:0/2496695953 wait complete. 2024-09-16T15:39:33.738 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.736+0000 7ff8c1200640 1 -- start start 2024-09-16T15:39:33.739 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.737+0000 7f0787400640 1 --2- 172.21.15.151:0/1513321303 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f07700788d0 0x7f077007ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:33.739 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.737+0000 7f570e600640 1 Processor -- start 2024-09-16T15:39:33.739 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.737+0000 7f0787400640 1 -- 172.21.15.151:0/1513321303 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f07800ba470 con 0x7f079015b0c0 2024-09-16T15:39:33.739 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.737+0000 7f0787400640 1 --2- 172.21.15.151:0/1513321303 >> [v2:172.21.15.151:6808/3111342741,v1:172.21.15.151:6809/3111342741] conn(0x7f0770080b50 0x7f0770082f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:33.739 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.737+0000 7f0787400640 1 -- 172.21.15.151:0/1513321303 --> [v2:172.21.15.151:6808/3111342741,v1:172.21.15.151:6809/3111342741] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f078002cea0 con 0x7f0770080b50 2024-09-16T15:39:33.739 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.737+0000 7f0787400640 1 -- 172.21.15.151:0/1513321303 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f07800ba950 con 0x7f079015b0c0 2024-09-16T15:39:33.740 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.737+0000 7f570e600640 1 -- start start 2024-09-16T15:39:33.740 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.738+0000 7f078fe00640 1 --2- 172.21.15.151:0/1513321303 >> [v2:172.21.15.151:6808/3111342741,v1:172.21.15.151:6809/3111342741] conn(0x7f0770080b50 0x7f0770082f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:33.740 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.738+0000 7f078f400640 1 --2- 172.21.15.151:0/1513321303 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f07700788d0 0x7f077007ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:33.740 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.739+0000 7ff8c1200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff8b4037f10 0x7ff8b40da0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:33.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.739+0000 7ff8c1200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff8b403b940 0x7ff8b40db760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:33.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.739+0000 7ff8c1200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8b40e27a0 0x7ff8b404e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:33.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.739+0000 7ff8c1200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b4058010 con 0x7ff8b403b940 2024-09-16T15:39:33.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.739+0000 7ff8c1200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b4057e90 con 0x7ff8b40e27a0 2024-09-16T15:39:33.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.739+0000 7ff8c1200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7ff8b4006e00 con 0x7ff8b4037f10 2024-09-16T15:39:33.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.739+0000 7f570e600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f570003a940 0x7f5700050330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:33.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.739+0000 7f570d200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f570003a940 0x7f5700050330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:33.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.739+0000 7f570d200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f570003a940 0x7f5700050330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.151:42566/0 (socket says 172.21.15.151:42566) 2024-09-16T15:39:33.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.739+0000 7f078fe00640 1 --2- 172.21.15.151:0/1513321303 >> [v2:172.21.15.151:6808/3111342741,v1:172.21.15.151:6809/3111342741] conn(0x7f0770080b50 0x7f0770082f70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:33.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.739+0000 7f078f400640 1 --2- 172.21.15.151:0/1513321303 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f07700788d0 0x7f077007ad90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f077c000f60 tx=0x7f077c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:33.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.739+0000 7ff8b9e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff8b4037f10 0x7ff8b40da0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:33.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.739+0000 7f570e600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f57000e2a60 0x7f570002fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:33.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.739+0000 7ff8b9e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff8b4037f10 0x7ff8b40da0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.151:48476/0 (socket says 172.21.15.151:48476) 2024-09-16T15:39:33.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.739+0000 7ff8b9e00640 1 -- 172.21.15.151:0/3311710651 learned_addr learned my addr 172.21.15.151:0/3311710651 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:33.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.739+0000 7fc957400640 1 -- 172.21.15.151:0/3116236591 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc948004640 con 0x7fc93c0051c0 2024-09-16T15:39:33.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.739+0000 7f570e600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f57000e39e0 0x7f57000dbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:33.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.740+0000 7f570e600640 1 -- 172.21.15.151:0/703642271 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f57000e6c60 con 0x7f57000e39e0 2024-09-16T15:39:33.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.739+0000 7f113a800640 1 -- 172.21.15.151:0/1666797341 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f111c0053c0 msgr2=0x7f111c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:33.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.739+0000 7f113a800640 1 --2- 172.21.15.151:0/1666797341 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f111c0053c0 0x7f111c00b9c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f112c009ed0 tx=0x7f111c005830 comp rx=0 tx=0).stop 2024-09-16T15:39:33.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.740+0000 7ff8b9e00640 1 -- 172.21.15.151:0/3311710651 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8b40e27a0 msgr2=0x7ff8b404e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:33.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.740+0000 7f570e600640 1 -- 172.21.15.151:0/703642271 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f57000e6de0 con 0x7f570003a940 2024-09-16T15:39:33.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.740+0000 7f570e600640 1 -- 172.21.15.151:0/703642271 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f57000e6f60 con 0x7f57000e2a60 2024-09-16T15:39:33.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.740+0000 7fc95fa00640 1 -- 172.21.15.151:0/3116236591 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc93c0051c0 msgr2=0x7fc93c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:33.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.740+0000 7fc95fa00640 1 --2- 172.21.15.151:0/3116236591 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc93c0051c0 0x7fc93c00b9c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc948001740 tx=0x7fc93c005830 comp rx=0 tx=0).stop 2024-09-16T15:39:33.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.739+0000 7f570d200640 1 -- 172.21.15.151:0/703642271 learned_addr learned my addr 172.21.15.151:0/703642271 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:33.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.740+0000 7f5707e00640 1 --2- 172.21.15.151:0/703642271 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f57000e2a60 0x7f570002fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:33.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.740+0000 7fc95fa00640 1 -- 172.21.15.151:0/3116236591 shutdown_connections 2024-09-16T15:39:33.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.740+0000 7fc95fa00640 1 --2- 172.21.15.151:0/3116236591 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc93c0051c0 0x7fc93c00b9c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.740+0000 7fc95fa00640 1 -- 172.21.15.151:0/3116236591 >> 172.21.15.151:0/3116236591 conn(0x7fc9580b1950 msgr2=0x7fc9580b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:33.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.740+0000 7f570dc00640 1 --2- 172.21.15.151:0/703642271 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f57000e39e0 0x7f57000dbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:33.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.740+0000 7ff8b9400640 1 --2- 172.21.15.151:0/3311710651 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff8b403b940 0x7ff8b40db760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:33.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.740+0000 7ff8b9e00640 1 --2- 172.21.15.151:0/3311710651 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8b40e27a0 0x7ff8b404e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.740+0000 7f4eae200640 1 Processor -- start 2024-09-16T15:39:33.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.740+0000 7ff8b9e00640 1 -- 172.21.15.151:0/3311710651 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff8b403b940 msgr2=0x7ff8b40db760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:33.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.740+0000 7f113a800640 1 -- 172.21.15.151:0/1666797341 shutdown_connections 2024-09-16T15:39:33.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.740+0000 7f113a800640 1 --2- 172.21.15.151:0/1666797341 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f111c0053c0 0x7f111c00b9c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.740+0000 7fc95fa00640 1 -- 172.21.15.151:0/3116236591 shutdown_connections 2024-09-16T15:39:33.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.740+0000 7f113a800640 1 -- 172.21.15.151:0/1666797341 >> 172.21.15.151:0/1666797341 conn(0x7f11340d3dc0 msgr2=0x7f11340d41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:33.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.741+0000 7ff8b9e00640 1 --2- 172.21.15.151:0/3311710651 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff8b403b940 0x7ff8b40db760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.741+0000 7f570d200640 1 -- 172.21.15.151:0/703642271 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f57000e2a60 msgr2=0x7f570002fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:33.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.741+0000 7f570d200640 1 --2- 172.21.15.151:0/703642271 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f57000e2a60 0x7f570002fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.741+0000 7f570d200640 1 -- 172.21.15.151:0/703642271 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f57000e39e0 msgr2=0x7f57000dbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:33.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.741+0000 7f570d200640 1 --2- 172.21.15.151:0/703642271 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f57000e39e0 0x7f57000dbbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.741+0000 7f570d200640 1 -- 172.21.15.151:0/703642271 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f570004f230 con 0x7f570003a940 2024-09-16T15:39:33.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.741+0000 7f5707e00640 1 --2- 172.21.15.151:0/703642271 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f57000e2a60 0x7f570002fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:39:33.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.741+0000 7f113a800640 1 -- 172.21.15.151:0/1666797341 shutdown_connections 2024-09-16T15:39:33.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.741+0000 7f113a800640 1 -- 172.21.15.151:0/1666797341 wait complete. 2024-09-16T15:39:33.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.741+0000 7f0787400640 1 -- 172.21.15.151:0/1513321303 <== osd.0 v2:172.21.15.151:6808/3111342741 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f078002cea0 con 0x7f0770080b50 2024-09-16T15:39:33.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.741+0000 7ff8b9e00640 1 -- 172.21.15.151:0/3311710651 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8b40dbca0 con 0x7ff8b4037f10 2024-09-16T15:39:33.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.741+0000 7ff8b9400640 1 --2- 172.21.15.151:0/3311710651 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff8b403b940 0x7ff8b40db760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:39:33.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.741+0000 7fce62800640 1 -- 172.21.15.151:0/1044505418 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce5c0cc5f0 msgr2=0x7fce5c0cca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:33.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.741+0000 7fce62800640 1 --2- 172.21.15.151:0/1044505418 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce5c0cc5f0 0x7fce5c0cca80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fce5c0af2c0 tx=0x7fce4802a5d0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.741+0000 7ff8b9e00640 1 --2- 172.21.15.151:0/3311710651 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff8b4037f10 0x7ff8b40da0c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac009850 tx=0x7ff8ac0045b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:33.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.741+0000 7fc95fa00640 1 -- 172.21.15.151:0/3116236591 wait complete. 2024-09-16T15:39:33.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.741+0000 7fce62800640 1 -- 172.21.15.151:0/1044505418 shutdown_connections 2024-09-16T15:39:33.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.741+0000 7fce62800640 1 --2- 172.21.15.151:0/1044505418 >> v2:172.21.15.167:3300/0 conn(0x7fce5c06f850 0x7fce5c06fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.741+0000 7fce62800640 1 --2- 172.21.15.151:0/1044505418 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce5c0cc5f0 0x7fce5c0cca80 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.741+0000 7fce62800640 1 -- 172.21.15.151:0/1044505418 >> 172.21.15.151:0/1044505418 conn(0x7fce5c0b1950 msgr2=0x7fce5c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:33.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.741+0000 7f570d200640 1 --2- 172.21.15.151:0/703642271 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f570003a940 0x7f5700050330 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f56f4009850 tx=0x7f56f4005770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:33.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.742+0000 7fce62800640 1 -- 172.21.15.151:0/1044505418 shutdown_connections 2024-09-16T15:39:33.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.742+0000 7f4eae200640 1 -- start start 2024-09-16T15:39:33.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.742+0000 7fce62800640 1 -- 172.21.15.151:0/1044505418 wait complete. 2024-09-16T15:39:33.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.742+0000 7ff8b2000640 1 -- 172.21.15.151:0/3311710651 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac005020 con 0x7ff8b4037f10 2024-09-16T15:39:33.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.742+0000 7f5704c00640 1 -- 172.21.15.151:0/703642271 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f403f030 con 0x7f570003a940 2024-09-16T15:39:33.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.742+0000 7f5704c00640 1 -- 172.21.15.151:0/703642271 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f56f4004020 con 0x7f570003a940 2024-09-16T15:39:33.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.742+0000 7f5704c00640 1 -- 172.21.15.151:0/703642271 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f4035070 con 0x7f570003a940 2024-09-16T15:39:33.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.742+0000 7f570e600640 1 -- 172.21.15.151:0/703642271 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57000509c0 con 0x7f570003a940 2024-09-16T15:39:33.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.742+0000 7ff8b2000640 1 -- 172.21.15.151:0/3311710651 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff8ac0052b0 con 0x7ff8b4037f10 2024-09-16T15:39:33.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.743+0000 7ff8b2000640 1 -- 172.21.15.151:0/3311710651 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac0054a0 con 0x7ff8b4037f10 2024-09-16T15:39:33.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.743+0000 7ff8c1200640 1 -- 172.21.15.151:0/3311710651 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8b40da9a0 con 0x7ff8b4037f10 2024-09-16T15:39:33.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.743+0000 7ff8c1200640 1 -- 172.21.15.151:0/3311710651 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8b40daf60 con 0x7ff8b4037f10 2024-09-16T15:39:33.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.743+0000 7f4eae200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4ea8153820 0x7f4ea81f7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:33.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.743+0000 7f4eae200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ea8153c00 0x7f4ea81f7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:33.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.743+0000 7f4eae200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ea81f84d0 0x7f4ea81f5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:33.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.743+0000 7f4eae200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea80cf0a0 con 0x7f4ea81f84d0 2024-09-16T15:39:33.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.743+0000 7f4eae200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea80cef20 con 0x7f4ea8153c00 2024-09-16T15:39:33.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.743+0000 7f4eae200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f4ea80cf220 con 0x7f4ea8153820 2024-09-16T15:39:33.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.743+0000 7f113a800640 1 Processor -- start 2024-09-16T15:39:33.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.743+0000 7fc95fa00640 1 Processor -- start 2024-09-16T15:39:33.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.744+0000 7f4ea6a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ea8153c00 0x7f4ea81f7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:33.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.744+0000 7fc95fa00640 1 -- start start 2024-09-16T15:39:33.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.744+0000 7f113a800640 1 -- start start 2024-09-16T15:39:33.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.743+0000 7f570e600640 1 -- 172.21.15.151:0/703642271 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5700050f80 con 0x7f570003a940 2024-09-16T15:39:33.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.744+0000 7f4ea6a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ea8153c00 0x7f4ea81f7f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.151:42574/0 (socket says 172.21.15.151:42574) 2024-09-16T15:39:33.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.744+0000 7f4ea6a00640 1 -- 172.21.15.151:0/3126668769 learned_addr learned my addr 172.21.15.151:0/3126668769 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:33.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.744+0000 7f4ea7400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4ea8153820 0x7f4ea81f7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:33.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.744+0000 7fce62800640 1 Processor -- start 2024-09-16T15:39:33.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.745+0000 7f4ea6a00640 1 -- 172.21.15.151:0/3126668769 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4ea8153820 msgr2=0x7f4ea81f7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:33.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.745+0000 7f4ea6a00640 1 --2- 172.21.15.151:0/3126668769 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4ea8153820 0x7f4ea81f7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.745+0000 7f4ea6a00640 1 -- 172.21.15.151:0/3126668769 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ea81f84d0 msgr2=0x7f4ea81f5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:33.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.745+0000 7f4ea6a00640 1 --2- 172.21.15.151:0/3126668769 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ea81f84d0 0x7f4ea81f5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.745+0000 7f4ea6a00640 1 -- 172.21.15.151:0/3126668769 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ea80b7e20 con 0x7f4ea8153c00 2024-09-16T15:39:33.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.745+0000 7f570e600640 1 -- 172.21.15.151:0/703642271 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f57000e1530 con 0x7f570003a940 2024-09-16T15:39:33.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.745+0000 7f4ea6a00640 1 --2- 172.21.15.151:0/3126668769 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ea8153c00 0x7f4ea81f7f90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4e9400c3c0 tx=0x7f4e9400c7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:33.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.746+0000 7ff8c1200640 1 -- 172.21.15.151:0/3311710651 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff8b40072b0 con 0x7ff8b4037f10 2024-09-16T15:39:33.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.746+0000 7fc95fa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc93c0051c0 0x7fc95809bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:33.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.746+0000 7fc95fa00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc95815e690 0x7fc958156e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:33.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.746+0000 7fc95fa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc95809b170 0x7fc95809b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:33.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.746+0000 7fc95fa00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc9580cd010 con 0x7fc95809b170 2024-09-16T15:39:33.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.746+0000 7fc95fa00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc958161e20 con 0x7fc93c0051c0 2024-09-16T15:39:33.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.746+0000 7fc95fa00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fc9580cd190 con 0x7fc95815e690 2024-09-16T15:39:33.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.746+0000 7f113a800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f111c0053c0 0x7f11340cae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:33.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.746+0000 7f113a800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f113415c7a0 0x7f11341f9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:33.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.746+0000 7f113a800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f11340ccc80 0x7f11340b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:33.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.746+0000 7f113a800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1134161bc0 con 0x7f11340ccc80 2024-09-16T15:39:33.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.746+0000 7f113a800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1134161990 con 0x7f113415c7a0 2024-09-16T15:39:33.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.746+0000 7f113a800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f1134161d40 con 0x7f111c0053c0 2024-09-16T15:39:33.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.746+0000 7f1138a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f113415c7a0 0x7f11341f9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:33.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.746+0000 7fc95da00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc95809b170 0x7fc95809b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:33.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.746+0000 7f4e9f400640 1 -- 172.21.15.151:0/3126668769 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e94011790 con 0x7f4ea8153c00 2024-09-16T15:39:33.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.746+0000 7f4e9f400640 1 -- 172.21.15.151:0/3126668769 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e94017070 con 0x7f4ea8153c00 2024-09-16T15:39:33.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.746+0000 7f4e9f400640 1 -- 172.21.15.151:0/3126668769 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e94011d00 con 0x7f4ea8153c00 2024-09-16T15:39:33.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.747+0000 7f4eae200640 1 -- 172.21.15.151:0/3126668769 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ea80b8100 con 0x7f4ea8153c00 2024-09-16T15:39:33.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.745+0000 7fce62800640 1 -- start start 2024-09-16T15:39:33.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.746+0000 7fce62800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce5c06f850 0x7fce5c13e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:33.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.746+0000 7fce62800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce5c070920 0x7fce5c0b0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:33.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.746+0000 7fce62800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce5c0cc5f0 0x7fce5c0b05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:33.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.746+0000 7fce62800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fce5c0b4290 con 0x7fce5c06f850 2024-09-16T15:39:33.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.746+0000 7fce62800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fce5c0b4110 con 0x7fce5c0cc5f0 2024-09-16T15:39:33.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.746+0000 7fce62800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fce5c0b4410 con 0x7fce5c070920 2024-09-16T15:39:33.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.746+0000 7f1138a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f113415c7a0 0x7f11341f9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.151:42584/0 (socket says 172.21.15.151:42584) 2024-09-16T15:39:33.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.746+0000 7f1138a00640 1 -- 172.21.15.151:0/267414007 learned_addr learned my addr 172.21.15.151:0/267414007 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:33.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.747+0000 7f1139e00640 1 --2- 172.21.15.151:0/267414007 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f11340ccc80 0x7f11340b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:33.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.747+0000 7f1139400640 1 --2- 172.21.15.151:0/267414007 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f111c0053c0 0x7f11340cae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:33.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.747+0000 7fce5be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce5c0cc5f0 0x7fce5c0b05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:33.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.747+0000 7fc957e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc95815e690 0x7fc958156e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:33.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.747+0000 7fc957e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc95815e690 0x7fc958156e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.151:48512/0 (socket says 172.21.15.151:48512) 2024-09-16T15:39:33.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.747+0000 7fc957e00640 1 -- 172.21.15.151:0/716538852 learned_addr learned my addr 172.21.15.151:0/716538852 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:33.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.747+0000 7fc95d000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc93c0051c0 0x7fc95809bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:33.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.747+0000 7fce5be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce5c0cc5f0 0x7fce5c0b05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.151:42598/0 (socket says 172.21.15.151:42598) 2024-09-16T15:39:33.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.747+0000 7fce5be00640 1 -- 172.21.15.151:0/1268495649 learned_addr learned my addr 172.21.15.151:0/1268495649 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:33.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.747+0000 7fce5b400640 1 --2- 172.21.15.151:0/1268495649 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce5c06f850 0x7fce5c13e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:33.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.747+0000 7f1138a00640 1 -- 172.21.15.151:0/267414007 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f111c0053c0 msgr2=0x7f11340cae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:33.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.747+0000 7f1138a00640 1 --2- 172.21.15.151:0/267414007 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f111c0053c0 0x7f11340cae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.747+0000 7f1138a00640 1 -- 172.21.15.151:0/267414007 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f11340ccc80 msgr2=0x7f11340b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:33.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.747+0000 7f1138a00640 1 --2- 172.21.15.151:0/267414007 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f11340ccc80 0x7f11340b3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.747+0000 7f1138a00640 1 -- 172.21.15.151:0/267414007 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11340cb3e0 con 0x7f113415c7a0 2024-09-16T15:39:33.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.747+0000 7f1139e00640 1 --2- 172.21.15.151:0/267414007 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f11340ccc80 0x7f11340b3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:39:33.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.748+0000 7f4eae200640 1 -- 172.21.15.151:0/3126668769 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ea80b86c0 con 0x7f4ea8153c00 2024-09-16T15:39:33.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.748+0000 7fce5be00640 1 -- 172.21.15.151:0/1268495649 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce5c070920 msgr2=0x7fce5c0b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:33.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.748+0000 7fce5be00640 1 --2- 172.21.15.151:0/1268495649 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce5c070920 0x7fce5c0b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.748+0000 7fce5be00640 1 -- 172.21.15.151:0/1268495649 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce5c06f850 msgr2=0x7fce5c13e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:33.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.748+0000 7fce5be00640 1 --2- 172.21.15.151:0/1268495649 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce5c06f850 0x7fce5c13e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.748+0000 7fce5be00640 1 -- 172.21.15.151:0/1268495649 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce5c13e850 con 0x7fce5c0cc5f0 2024-09-16T15:39:33.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.748+0000 7fc957e00640 1 -- 172.21.15.151:0/716538852 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc93c0051c0 msgr2=0x7fc95809bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:33.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.748+0000 7fc957e00640 1 --2- 172.21.15.151:0/716538852 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc93c0051c0 0x7fc95809bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.748+0000 7fc957e00640 1 -- 172.21.15.151:0/716538852 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc95809b170 msgr2=0x7fc95809b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:33.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.748+0000 7fc957e00640 1 --2- 172.21.15.151:0/716538852 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc95809b170 0x7fc95809b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.748+0000 7fc957e00640 1 -- 172.21.15.151:0/716538852 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9580cdd80 con 0x7fc95815e690 2024-09-16T15:39:33.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.748+0000 7f1138a00640 1 --2- 172.21.15.151:0/267414007 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f113415c7a0 0x7f11341f9d30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f112000c500 tx=0x7f112000c9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:33.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.748+0000 7fce5be00640 1 --2- 172.21.15.151:0/1268495649 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce5c0cc5f0 0x7fce5c0b05d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fce4c00e970 tx=0x7fce4c00ed70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:33.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.748+0000 7fc957e00640 1 --2- 172.21.15.151:0/716538852 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc95815e690 0x7fc958156e60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc950010d40 tx=0x7fc95000e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:33.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.748+0000 7f5704c00640 1 -- 172.21.15.151:0/703642271 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f56f40327a0 con 0x7f570003a940 2024-09-16T15:39:33.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.749+0000 7f1131600640 1 -- 172.21.15.151:0/267414007 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1120017070 con 0x7f113415c7a0 2024-09-16T15:39:33.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.749+0000 7f1131600640 1 -- 172.21.15.151:0/267414007 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f11200035f0 con 0x7f113415c7a0 2024-09-16T15:39:33.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.749+0000 7f1131600640 1 -- 172.21.15.151:0/267414007 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1120003a40 con 0x7f113415c7a0 2024-09-16T15:39:33.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.749+0000 7f4eae200640 1 -- 172.21.15.151:0/3126668769 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4ea80cf780 con 0x7f4ea8153c00 2024-09-16T15:39:33.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.749+0000 7fc954c00640 1 -- 172.21.15.151:0/716538852 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc950017070 con 0x7fc95815e690 2024-09-16T15:39:33.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.749+0000 7fc954c00640 1 -- 172.21.15.151:0/716538852 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc950015070 con 0x7fc95815e690 2024-09-16T15:39:33.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.749+0000 7fc954c00640 1 -- 172.21.15.151:0/716538852 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc950003df0 con 0x7fc95815e690 2024-09-16T15:39:33.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.749+0000 7fc95fa00640 1 -- 172.21.15.151:0/716538852 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc95815ebe0 con 0x7fc95815e690 2024-09-16T15:39:33.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.749+0000 7fc95fa00640 1 -- 172.21.15.151:0/716538852 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc95815f1a0 con 0x7fc95815e690 2024-09-16T15:39:33.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.748+0000 7fce53400640 1 -- 172.21.15.151:0/1268495649 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce4c00ccc0 con 0x7fce5c0cc5f0 2024-09-16T15:39:33.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.749+0000 7fce53400640 1 -- 172.21.15.151:0/1268495649 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fce4c00ce60 con 0x7fce5c0cc5f0 2024-09-16T15:39:33.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.749+0000 7fce53400640 1 -- 172.21.15.151:0/1268495649 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce4c005280 con 0x7fce5c0cc5f0 2024-09-16T15:39:33.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.750+0000 7f113a800640 1 -- 172.21.15.151:0/267414007 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11340cb6c0 con 0x7f113415c7a0 2024-09-16T15:39:33.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.750+0000 7f113a800640 1 -- 172.21.15.151:0/267414007 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11340cbc80 con 0x7f113415c7a0 2024-09-16T15:39:33.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.750+0000 7fce62800640 1 -- 172.21.15.151:0/1268495649 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce5c13d2e0 con 0x7fce5c0cc5f0 2024-09-16T15:39:33.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.751+0000 7f5704c00640 1 --2- 172.21.15.151:0/703642271 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f56e8078920 0x7f56e807ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:33.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.751+0000 7f5704c00640 1 -- 172.21.15.151:0/703642271 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f56f40bb930 con 0x7f570003a940 2024-09-16T15:39:33.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.751+0000 7f5704c00640 1 --2- 172.21.15.151:0/703642271 >> [v2:172.21.15.151:6816/1142509041,v1:172.21.15.151:6817/1142509041] conn(0x7f56e8080b40 0x7f56e8082f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:33.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.751+0000 7f5704c00640 1 -- 172.21.15.151:0/703642271 --> [v2:172.21.15.151:6816/1142509041,v1:172.21.15.151:6817/1142509041] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f56f403ce50 con 0x7f56e8080b40 2024-09-16T15:39:33.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.751+0000 7f5704c00640 1 -- 172.21.15.151:0/703642271 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f56f40bbd20 con 0x7f570003a940 2024-09-16T15:39:33.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.752+0000 7f4e9f400640 1 -- 172.21.15.151:0/3126668769 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f4e9401c460 con 0x7f4ea8153c00 2024-09-16T15:39:33.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.752+0000 7fce62800640 1 -- 172.21.15.151:0/1268495649 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce5c09d7d0 con 0x7fce5c0cc5f0 2024-09-16T15:39:33.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.752+0000 7f2482800640 1 -- 172.21.15.151:0/4293022251 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f247c0aec20 msgr2=0x7f247c15aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:33.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.752+0000 7f2482800640 1 --2- 172.21.15.151:0/4293022251 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f247c0aec20 0x7f247c15aa00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f246c005fd0 tx=0x7f246c02a560 comp rx=0 tx=0).stop 2024-09-16T15:39:33.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.750+0000 7ff8b2000640 1 -- 172.21.15.151:0/3311710651 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7ff8ac041030 con 0x7ff8b4037f10 2024-09-16T15:39:33.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.751+0000 7ff8b2000640 1 --2- 172.21.15.151:0/3311710651 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff89c078850 0x7ff89c07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:33.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.751+0000 7ff8b2000640 1 -- 172.21.15.151:0/3311710651 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7ff8ac0ba580 con 0x7ff8b4037f10 2024-09-16T15:39:33.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.752+0000 7ff8b2000640 1 --2- 172.21.15.151:0/3311710651 >> [v2:172.21.15.151:6800/3368304087,v1:172.21.15.151:6801/3368304087] conn(0x7ff89c0809f0 0x7ff89c082e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:33.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.752+0000 7ff8b2000640 1 -- 172.21.15.151:0/3311710651 --> [v2:172.21.15.151:6800/3368304087,v1:172.21.15.151:6801/3368304087] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff8ac009080 con 0x7ff89c0809f0 2024-09-16T15:39:33.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.750+0000 7fc95fa00640 1 -- 172.21.15.151:0/716538852 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc930001030 con 0x7fc95815e690 2024-09-16T15:39:33.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.752+0000 7f2482800640 1 -- 172.21.15.151:0/4293022251 shutdown_connections 2024-09-16T15:39:33.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.752+0000 7f2482800640 1 --2- 172.21.15.151:0/4293022251 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f247c0aec20 0x7f247c15aa00 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.752+0000 7f2482800640 1 --2- 172.21.15.151:0/4293022251 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f247c15a620 0x7f247c0cbdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.752+0000 7f2482800640 1 -- 172.21.15.151:0/4293022251 >> 172.21.15.151:0/4293022251 conn(0x7f247c0cfa20 msgr2=0x7f247c0cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:33.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.753+0000 7f2482800640 1 -- 172.21.15.151:0/4293022251 shutdown_connections 2024-09-16T15:39:33.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.753+0000 7f2482800640 1 -- 172.21.15.151:0/4293022251 wait complete. 2024-09-16T15:39:33.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.753+0000 7ff8b2000640 1 -- 172.21.15.151:0/3311710651 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7ff8ac0c1050 con 0x7ff8b4037f10 2024-09-16T15:39:33.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.753+0000 7ff8b9400640 1 --2- 172.21.15.151:0/3311710651 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff89c078850 0x7ff89c07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:33.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.752+0000 7f570dc00640 1 --2- 172.21.15.151:0/703642271 >> [v2:172.21.15.151:6816/1142509041,v1:172.21.15.151:6817/1142509041] conn(0x7f56e8080b40 0x7f56e8082f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:33.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.753+0000 7f4e9f400640 1 --2- 172.21.15.151:0/3126668769 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4e88078880 0x7f4e8807ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:33.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.753+0000 7f113a800640 1 -- 172.21.15.151:0/267414007 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f11340b6c10 con 0x7f113415c7a0 2024-09-16T15:39:33.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.753+0000 7f4e9f400640 1 -- 172.21.15.151:0/3126668769 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f4e9409a8e0 con 0x7f4ea8153c00 2024-09-16T15:39:33.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.753+0000 7f5707e00640 1 --2- 172.21.15.151:0/703642271 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f56e8078920 0x7f56e807ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:33.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.754+0000 7f4e9f400640 1 --2- 172.21.15.151:0/3126668769 >> [v2:172.21.15.167:6810/2735495271,v1:172.21.15.167:6811/2735495271] conn(0x7f4e88080a20 0x7f4e88082e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:33.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.754+0000 7f4e9f400640 1 -- 172.21.15.151:0/3126668769 --> [v2:172.21.15.167:6810/2735495271,v1:172.21.15.167:6811/2735495271] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4e880834f0 con 0x7f4e88080a20 2024-09-16T15:39:33.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.754+0000 7f4e9f400640 1 -- 172.21.15.151:0/3126668769 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f4e9409ad70 con 0x7f4ea8153c00 2024-09-16T15:39:33.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.754+0000 7f1131600640 1 -- 172.21.15.151:0/267414007 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f112001a070 con 0x7f113415c7a0 2024-09-16T15:39:33.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.754+0000 7f4ea7400640 1 --2- 172.21.15.151:0/3126668769 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4e88078880 0x7f4e8807ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:33.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.754+0000 7f570dc00640 1 --2- 172.21.15.151:0/703642271 >> [v2:172.21.15.151:6816/1142509041,v1:172.21.15.151:6817/1142509041] conn(0x7f56e8080b40 0x7f56e8082f60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:33.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.754+0000 7f5707e00640 1 --2- 172.21.15.151:0/703642271 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f56e8078920 0x7f56e807ade0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f56fc0095f0 tx=0x7f56fc005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:33.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.754+0000 7ff8ba800640 1 --2- 172.21.15.151:0/3311710651 >> [v2:172.21.15.151:6800/3368304087,v1:172.21.15.151:6801/3368304087] conn(0x7ff89c0809f0 0x7ff89c082e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:33.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.754+0000 7f4ea7e00640 1 --2- 172.21.15.151:0/3126668769 >> [v2:172.21.15.167:6810/2735495271,v1:172.21.15.167:6811/2735495271] conn(0x7f4e88080a20 0x7f4e88082e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:33.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.753+0000 7fc954c00640 1 -- 172.21.15.151:0/716538852 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fc950004030 con 0x7fc95815e690 2024-09-16T15:39:33.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.754+0000 7fce62800640 1 -- 172.21.15.151:0/1268495649 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fce5c06fce0 con 0x7fce5c0cc5f0 2024-09-16T15:39:33.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.755+0000 7f2482800640 1 Processor -- start 2024-09-16T15:39:33.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.755+0000 7ff8ba800640 1 --2- 172.21.15.151:0/3311710651 >> [v2:172.21.15.151:6800/3368304087,v1:172.21.15.151:6801/3368304087] conn(0x7ff89c0809f0 0x7ff89c082e10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:33.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.755+0000 7fc954c00640 1 --2- 172.21.15.151:0/716538852 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc9380788d0 0x7fc93807ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:33.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.755+0000 7f2482800640 1 -- start start 2024-09-16T15:39:33.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.756+0000 7fce53400640 1 -- 172.21.15.151:0/1268495649 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fce4c005420 con 0x7fce5c0cc5f0 2024-09-16T15:39:33.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.756+0000 7f1131600640 1 --2- 172.21.15.151:0/267414007 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f11140788d0 0x7f111407ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:33.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.756+0000 7f5704c00640 1 -- 172.21.15.151:0/703642271 <== osd.3 v2:172.21.15.151:6816/1142509041 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f56f403ce50 con 0x7f56e8080b40 2024-09-16T15:39:33.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.756+0000 7f4ea7400640 1 --2- 172.21.15.151:0/3126668769 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4e88078880 0x7f4e8807ad40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c0057e0 tx=0x7f4e980019f0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:33.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.756+0000 7ff8b9400640 1 --2- 172.21.15.151:0/3311710651 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff89c078850 0x7ff89c07ad10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff8b403da40 tx=0x7ff8a8005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:33.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.756+0000 7fc954c00640 1 -- 172.21.15.151:0/716538852 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fc95009b4b0 con 0x7fc95815e690 2024-09-16T15:39:33.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.756+0000 7fc954c00640 1 --2- 172.21.15.151:0/716538852 >> [v2:172.21.15.151:6823/3079373271,v1:172.21.15.151:6825/3079373271] conn(0x7fc938080af0 0x7fc938082f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:33.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.756+0000 7fc954c00640 1 -- 172.21.15.151:0/716538852 --> [v2:172.21.15.151:6823/3079373271,v1:172.21.15.151:6825/3079373271] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc950023e80 con 0x7fc938080af0 2024-09-16T15:39:33.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.756+0000 7fc954c00640 1 -- 172.21.15.151:0/716538852 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fc9500a2050 con 0x7fc95815e690 2024-09-16T15:39:33.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.756+0000 7f1131600640 1 -- 172.21.15.151:0/267414007 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f1120099d60 con 0x7f113415c7a0 2024-09-16T15:39:33.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.756+0000 7f1139400640 1 --2- 172.21.15.151:0/267414007 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f11140788d0 0x7f111407ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:33.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.757+0000 7f4ea7e00640 1 --2- 172.21.15.151:0/3126668769 >> [v2:172.21.15.167:6810/2735495271,v1:172.21.15.167:6811/2735495271] conn(0x7f4e88080a20 0x7f4e88082e40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:33.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.756+0000 7fc95d000640 1 --2- 172.21.15.151:0/716538852 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc9380788d0 0x7fc93807ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:33.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.757+0000 7ff8b2000640 1 -- 172.21.15.151:0/3311710651 <== osd.1 v2:172.21.15.151:6800/3368304087 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff8ac009080 con 0x7ff89c0809f0 2024-09-16T15:39:33.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.757+0000 7f1131600640 1 --2- 172.21.15.151:0/267414007 >> [v2:172.21.15.167:6818/2538273273,v1:172.21.15.167:6819/2538273273] conn(0x7f1114080b50 0x7f1114082f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:33.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.757+0000 7f1131600640 1 -- 172.21.15.151:0/267414007 --> [v2:172.21.15.167:6818/2538273273,v1:172.21.15.167:6819/2538273273] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1114083620 con 0x7f1114080b50 2024-09-16T15:39:33.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.757+0000 7f1131600640 1 -- 172.21.15.151:0/267414007 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f112009c380 con 0x7f113415c7a0 2024-09-16T15:39:33.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.757+0000 7fc95da00640 1 --2- 172.21.15.151:0/716538852 >> [v2:172.21.15.151:6823/3079373271,v1:172.21.15.151:6825/3079373271] conn(0x7fc938080af0 0x7fc938082f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:33.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.757+0000 7f2482800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f247c0aec20 0x7f247c0c5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:33.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.757+0000 7f2482800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f247c0af040 0x7f247c1fc600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:33.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.757+0000 7f2482800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f247c15a620 0x7f247c1fcb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:33.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.757+0000 7f2482800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f247c15b0c0 con 0x7f247c15a620 2024-09-16T15:39:33.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.757+0000 7f2482800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f247c15af40 con 0x7f247c0aec20 2024-09-16T15:39:33.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.757+0000 7f2482800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f247c15b240 con 0x7f247c0af040 2024-09-16T15:39:33.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.757+0000 7f1139e00640 1 --2- 172.21.15.151:0/267414007 >> [v2:172.21.15.167:6818/2538273273,v1:172.21.15.167:6819/2538273273] conn(0x7f1114080b50 0x7f1114082f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:33.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.758+0000 7f1139400640 1 --2- 172.21.15.151:0/267414007 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f11140788d0 0x7f111407ad90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1128001680 tx=0x7f1128009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:33.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.758+0000 7f2481400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f247c0aec20 0x7f247c0c5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:33.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.758+0000 7f2481400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f247c0aec20 0x7f247c0c5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.151:42608/0 (socket says 172.21.15.151:42608) 2024-09-16T15:39:33.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.758+0000 7f2481400640 1 -- 172.21.15.151:0/1912303792 learned_addr learned my addr 172.21.15.151:0/1912303792 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:33.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.758+0000 7f2480a00640 1 --2- 172.21.15.151:0/1912303792 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f247c0af040 0x7f247c1fc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:33.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.758+0000 7f2481e00640 1 --2- 172.21.15.151:0/1912303792 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f247c15a620 0x7f247c1fcb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:33.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.758+0000 7f4e9f400640 1 -- 172.21.15.151:0/3126668769 <== osd.4 v2:172.21.15.167:6810/2735495271 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4e880834f0 con 0x7f4e88080a20 2024-09-16T15:39:33.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.758+0000 7fc95da00640 1 --2- 172.21.15.151:0/716538852 >> [v2:172.21.15.151:6823/3079373271,v1:172.21.15.151:6825/3079373271] conn(0x7fc938080af0 0x7fc938082f10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:33.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.759+0000 7f1139e00640 1 --2- 172.21.15.151:0/267414007 >> [v2:172.21.15.167:6818/2538273273,v1:172.21.15.167:6819/2538273273] conn(0x7f1114080b50 0x7f1114082f70 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:33.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.758+0000 7f2480a00640 1 -- 172.21.15.151:0/1912303792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f247c0aec20 msgr2=0x7f247c0c5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:33.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.758+0000 7f2480a00640 1 --2- 172.21.15.151:0/1912303792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f247c0aec20 0x7f247c0c5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.758+0000 7f2480a00640 1 -- 172.21.15.151:0/1912303792 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f247c15a620 msgr2=0x7f247c1fcb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:33.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.758+0000 7fce53400640 1 --2- 172.21.15.151:0/1268495649 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fce3c078850 0x7fce3c07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:33.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.758+0000 7fce53400640 1 -- 172.21.15.151:0/1268495649 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fce4c09b2c0 con 0x7fce5c0cc5f0 2024-09-16T15:39:33.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.759+0000 7fce5b400640 1 --2- 172.21.15.151:0/1268495649 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fce3c078850 0x7fce3c07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:33.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.759+0000 7fce53400640 1 --2- 172.21.15.151:0/1268495649 >> [v2:172.21.15.167:6826/1494916572,v1:172.21.15.167:6827/1494916572] conn(0x7fce3c0809f0 0x7fce3c082e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:33.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.759+0000 7fce53400640 1 -- 172.21.15.151:0/1268495649 --> [v2:172.21.15.167:6826/1494916572,v1:172.21.15.167:6827/1494916572] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fce4c019e70 con 0x7fce3c0809f0 2024-09-16T15:39:33.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.759+0000 7fce53400640 1 -- 172.21.15.151:0/1268495649 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fce4c0a2050 con 0x7fce5c0cc5f0 2024-09-16T15:39:33.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.759+0000 7f2480a00640 1 --2- 172.21.15.151:0/1912303792 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f247c15a620 0x7f247c1fcb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.759+0000 7fc95d000640 1 --2- 172.21.15.151:0/716538852 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc9380788d0 0x7fc93807ad90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc944001920 tx=0x7fc9440018b0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:33.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.759+0000 7f2480a00640 1 -- 172.21.15.151:0/1912303792 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f247c0c3b20 con 0x7f247c0af040 2024-09-16T15:39:33.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.759+0000 7fce5aa00640 1 --2- 172.21.15.151:0/1268495649 >> [v2:172.21.15.167:6826/1494916572,v1:172.21.15.167:6827/1494916572] conn(0x7fce3c0809f0 0x7fce3c082e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:33.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.759+0000 7f2481e00640 1 --2- 172.21.15.151:0/1912303792 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f247c15a620 0x7f247c1fcb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:39:33.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.760+0000 7fc954c00640 1 -- 172.21.15.151:0/716538852 <== osd.2 v2:172.21.15.151:6823/3079373271 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc950023e80 con 0x7fc938080af0 2024-09-16T15:39:33.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.760+0000 7f1131600640 1 -- 172.21.15.151:0/267414007 <== osd.6 v2:172.21.15.167:6818/2538273273 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1114083620 con 0x7f1114080b50 2024-09-16T15:39:33.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.760+0000 7f2480a00640 1 --2- 172.21.15.151:0/1912303792 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f247c0af040 0x7f247c1fc600 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f246c031000 tx=0x7f246c003960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:33.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.760+0000 7fce5b400640 1 --2- 172.21.15.151:0/1268495649 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fce3c078850 0x7fce3c07ad10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fce54001840 tx=0x7fce540017d0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:33.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.760+0000 7f2479600640 1 -- 172.21.15.151:0/1912303792 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246c03f070 con 0x7f247c0af040 2024-09-16T15:39:33.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.760+0000 7fce5aa00640 1 --2- 172.21.15.151:0/1268495649 >> [v2:172.21.15.167:6826/1494916572,v1:172.21.15.167:6827/1494916572] conn(0x7fce3c0809f0 0x7fce3c082e10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:33.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.761+0000 7fce53400640 1 -- 172.21.15.151:0/1268495649 <== osd.5 v2:172.21.15.167:6826/1494916572 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fce4c019e70 con 0x7fce3c0809f0 2024-09-16T15:39:33.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.760+0000 7f2479600640 1 -- 172.21.15.151:0/1912303792 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f246c004890 con 0x7f247c0af040 2024-09-16T15:39:33.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.760+0000 7f2479600640 1 -- 172.21.15.151:0/1912303792 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246c035030 con 0x7f247c0af040 2024-09-16T15:39:33.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.761+0000 7f2482800640 1 -- 172.21.15.151:0/1912303792 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f247c0c3280 con 0x7f247c0af040 2024-09-16T15:39:33.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.762+0000 7f2482800640 1 -- 172.21.15.151:0/1912303792 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f247c0c3840 con 0x7f247c0af040 2024-09-16T15:39:33.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.762+0000 7f2482800640 1 -- 172.21.15.151:0/1912303792 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f247c15aa00 con 0x7f247c0af040 2024-09-16T15:39:33.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.765+0000 7f2479600640 1 -- 172.21.15.151:0/1912303792 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f246c042070 con 0x7f247c0af040 2024-09-16T15:39:33.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.767+0000 7f2479600640 1 --2- 172.21.15.151:0/1912303792 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f24600788a0 0x7f246007ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:33.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.767+0000 7f2479600640 1 -- 172.21.15.151:0/1912303792 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f246c07fee0 con 0x7f247c0af040 2024-09-16T15:39:33.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.767+0000 7f2481400640 1 --2- 172.21.15.151:0/1912303792 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f24600788a0 0x7f246007ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:33.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.768+0000 7f2479600640 1 --2- 172.21.15.151:0/1912303792 >> [v2:172.21.15.167:6802/265072227,v1:172.21.15.167:6803/265072227] conn(0x7f2460081790 0x7f2460083c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:33.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.768+0000 7f2479600640 1 -- 172.21.15.151:0/1912303792 --> [v2:172.21.15.167:6802/265072227,v1:172.21.15.167:6803/265072227] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2460084300 con 0x7f2460081790 2024-09-16T15:39:33.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.768+0000 7f2479600640 1 -- 172.21.15.151:0/1912303792 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f246c080460 con 0x7f247c0af040 2024-09-16T15:39:33.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.768+0000 7f2481e00640 1 --2- 172.21.15.151:0/1912303792 >> [v2:172.21.15.167:6802/265072227,v1:172.21.15.167:6803/265072227] conn(0x7f2460081790 0x7f2460083c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:33.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.768+0000 7f2481400640 1 --2- 172.21.15.151:0/1912303792 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f24600788a0 0x7f246007ad60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f24700099f0 tx=0x7f24700090e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:33.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.769+0000 7f2481e00640 1 --2- 172.21.15.151:0/1912303792 >> [v2:172.21.15.167:6802/265072227,v1:172.21.15.167:6803/265072227] conn(0x7f2460081790 0x7f2460083c30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:33.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.770+0000 7f2479600640 1 -- 172.21.15.151:0/1912303792 <== osd.7 v2:172.21.15.167:6802/265072227 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2460084300 con 0x7f2460081790 2024-09-16T15:39:33.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.776+0000 7f0796600640 1 -- 172.21.15.151:0/1513321303 --> [v2:172.21.15.151:6808/3111342741,v1:172.21.15.151:6809/3111342741] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0790071010 con 0x7f0770080b50 2024-09-16T15:39:33.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.777+0000 7f0787400640 1 -- 172.21.15.151:0/1513321303 <== osd.0 v2:172.21.15.151:6808/3111342741 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0790071010 con 0x7f0770080b50 2024-09-16T15:39:33.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.778+0000 7ff8c1200640 1 -- 172.21.15.151:0/3311710651 --> [v2:172.21.15.151:6800/3368304087,v1:172.21.15.151:6801/3368304087] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff8b4052be0 con 0x7ff89c0809f0 2024-09-16T15:39:33.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.779+0000 7f0796600640 1 -- 172.21.15.151:0/1513321303 >> [v2:172.21.15.151:6808/3111342741,v1:172.21.15.151:6809/3111342741] conn(0x7f0770080b50 msgr2=0x7f0770082f70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:33.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.779+0000 7f0796600640 1 --2- 172.21.15.151:0/1513321303 >> [v2:172.21.15.151:6808/3111342741,v1:172.21.15.151:6809/3111342741] conn(0x7f0770080b50 0x7f0770082f70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.779+0000 7ff8b2000640 1 -- 172.21.15.151:0/3311710651 <== osd.1 v2:172.21.15.151:6800/3368304087 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff8b4052be0 con 0x7ff89c0809f0 2024-09-16T15:39:33.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.779+0000 7f0796600640 1 -- 172.21.15.151:0/1513321303 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f07700788d0 msgr2=0x7f077007ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:33.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.779+0000 7f0796600640 1 --2- 172.21.15.151:0/1513321303 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f07700788d0 0x7f077007ad90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f077c000f60 tx=0x7f077c009040 comp rx=0 tx=0).stop 2024-09-16T15:39:33.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.779+0000 7f0796600640 1 -- 172.21.15.151:0/1513321303 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f079015b0c0 msgr2=0x7f07902047b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:33.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.779+0000 7f0796600640 1 --2- 172.21.15.151:0/1513321303 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f079015b0c0 0x7f07902047b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0780031000 tx=0x7f0780033c20 comp rx=0 tx=0).stop 2024-09-16T15:39:33.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.779+0000 7f078fe00640 1 -- 172.21.15.151:0/1513321303 reap_dead start 2024-09-16T15:39:33.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.779+0000 7f0796600640 1 -- 172.21.15.151:0/1513321303 shutdown_connections 2024-09-16T15:39:33.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.779+0000 7f0796600640 1 -- 172.21.15.151:0/1513321303 >> 172.21.15.151:0/1513321303 conn(0x7f07900b2b90 msgr2=0x7f07900cbdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:33.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.780+0000 7f0796600640 1 -- 172.21.15.151:0/1513321303 shutdown_connections 2024-09-16T15:39:33.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.780+0000 7ff8c1200640 1 -- 172.21.15.151:0/3311710651 >> [v2:172.21.15.151:6800/3368304087,v1:172.21.15.151:6801/3368304087] conn(0x7ff89c0809f0 msgr2=0x7ff89c082e10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:33.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.780+0000 7ff8c1200640 1 --2- 172.21.15.151:0/3311710651 >> [v2:172.21.15.151:6800/3368304087,v1:172.21.15.151:6801/3368304087] conn(0x7ff89c0809f0 0x7ff89c082e10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.780+0000 7ff8c1200640 1 -- 172.21.15.151:0/3311710651 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff89c078850 msgr2=0x7ff89c07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:33.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.780+0000 7ff8c1200640 1 --2- 172.21.15.151:0/3311710651 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff89c078850 0x7ff89c07ad10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff8b403da40 tx=0x7ff8a8005f90 comp rx=0 tx=0).stop 2024-09-16T15:39:33.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.780+0000 7ff8c1200640 1 -- 172.21.15.151:0/3311710651 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff8b4037f10 msgr2=0x7ff8b40da0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:33.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.780+0000 7ff8c1200640 1 --2- 172.21.15.151:0/3311710651 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff8b4037f10 0x7ff8b40da0c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac009850 tx=0x7ff8ac0045b0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.780+0000 7ff8ba800640 1 -- 172.21.15.151:0/3311710651 reap_dead start 2024-09-16T15:39:33.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.780+0000 7f0796600640 1 -- 172.21.15.151:0/1513321303 wait complete. 2024-09-16T15:39:33.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.786+0000 7ff8c1200640 1 -- 172.21.15.151:0/3311710651 shutdown_connections 2024-09-16T15:39:33.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.786+0000 7ff8c1200640 1 -- 172.21.15.151:0/3311710651 >> 172.21.15.151:0/3311710651 conn(0x7ff8b403a3d0 msgr2=0x7ff8b403c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:33.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.786+0000 7ff8c1200640 1 -- 172.21.15.151:0/3311710651 shutdown_connections 2024-09-16T15:39:33.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.786+0000 7ff8c1200640 1 -- 172.21.15.151:0/3311710651 wait complete. 2024-09-16T15:39:33.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.791+0000 7f570e600640 1 -- 172.21.15.151:0/703642271 --> [v2:172.21.15.151:6816/1142509041,v1:172.21.15.151:6817/1142509041] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5700039e20 con 0x7f56e8080b40 2024-09-16T15:39:33.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.792+0000 7f5704c00640 1 -- 172.21.15.151:0/703642271 <== osd.3 v2:172.21.15.151:6816/1142509041 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5700039e20 con 0x7f56e8080b40 2024-09-16T15:39:33.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.793+0000 7fc95fa00640 1 -- 172.21.15.151:0/716538852 --> [v2:172.21.15.151:6823/3079373271,v1:172.21.15.151:6825/3079373271] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc930002dd0 con 0x7fc938080af0 2024-09-16T15:39:33.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.793+0000 7fce62800640 1 -- 172.21.15.151:0/1268495649 --> [v2:172.21.15.167:6826/1494916572,v1:172.21.15.167:6827/1494916572] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fce5c0cd430 con 0x7fce3c0809f0 2024-09-16T15:39:33.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.793+0000 7fc954c00640 1 -- 172.21.15.151:0/716538852 <== osd.2 v2:172.21.15.151:6823/3079373271 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc930002dd0 con 0x7fc938080af0 2024-09-16T15:39:33.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.793+0000 7f113a800640 1 -- 172.21.15.151:0/267414007 --> [v2:172.21.15.167:6818/2538273273,v1:172.21.15.167:6819/2538273273] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f11341622a0 con 0x7f1114080b50 2024-09-16T15:39:33.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.794+0000 7f570e600640 1 -- 172.21.15.151:0/703642271 >> [v2:172.21.15.151:6816/1142509041,v1:172.21.15.151:6817/1142509041] conn(0x7f56e8080b40 msgr2=0x7f56e8082f60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:33.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.794+0000 7fce53400640 1 -- 172.21.15.151:0/1268495649 <== osd.5 v2:172.21.15.167:6826/1494916572 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fce5c0cd430 con 0x7fce3c0809f0 2024-09-16T15:39:33.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.794+0000 7f570e600640 1 --2- 172.21.15.151:0/703642271 >> [v2:172.21.15.151:6816/1142509041,v1:172.21.15.151:6817/1142509041] conn(0x7f56e8080b40 0x7f56e8082f60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.794+0000 7f570e600640 1 -- 172.21.15.151:0/703642271 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f56e8078920 msgr2=0x7f56e807ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:33.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.794+0000 7f1131600640 1 -- 172.21.15.151:0/267414007 <== osd.6 v2:172.21.15.167:6818/2538273273 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f11341622a0 con 0x7f1114080b50 2024-09-16T15:39:33.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.794+0000 7f4eae200640 1 -- 172.21.15.151:0/3126668769 --> [v2:172.21.15.167:6810/2735495271,v1:172.21.15.167:6811/2735495271] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4ea80b8890 con 0x7f4e88080a20 2024-09-16T15:39:33.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.794+0000 7f570e600640 1 --2- 172.21.15.151:0/703642271 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f56e8078920 0x7f56e807ade0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f56fc0095f0 tx=0x7f56fc005f90 comp rx=0 tx=0).stop 2024-09-16T15:39:33.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.795+0000 7f4e9f400640 1 -- 172.21.15.151:0/3126668769 <== osd.4 v2:172.21.15.167:6810/2735495271 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4ea80b8890 con 0x7f4e88080a20 2024-09-16T15:39:33.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.794+0000 7f570e600640 1 -- 172.21.15.151:0/703642271 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f570003a940 msgr2=0x7f5700050330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:33.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.794+0000 7f570e600640 1 --2- 172.21.15.151:0/703642271 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f570003a940 0x7f5700050330 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f56f4009850 tx=0x7f56f4005770 comp rx=0 tx=0).stop 2024-09-16T15:39:33.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.794+0000 7f570dc00640 1 -- 172.21.15.151:0/703642271 reap_dead start 2024-09-16T15:39:33.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.794+0000 7f570e600640 1 -- 172.21.15.151:0/703642271 shutdown_connections 2024-09-16T15:39:33.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.795+0000 7fc95fa00640 1 -- 172.21.15.151:0/716538852 >> [v2:172.21.15.151:6823/3079373271,v1:172.21.15.151:6825/3079373271] conn(0x7fc938080af0 msgr2=0x7fc938082f10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:33.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.795+0000 7fc95fa00640 1 --2- 172.21.15.151:0/716538852 >> [v2:172.21.15.151:6823/3079373271,v1:172.21.15.151:6825/3079373271] conn(0x7fc938080af0 0x7fc938082f10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.794+0000 7f570e600640 1 -- 172.21.15.151:0/703642271 >> 172.21.15.151:0/703642271 conn(0x7f5700055540 msgr2=0x7f57000e5d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:33.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.795+0000 7f113a800640 1 -- 172.21.15.151:0/267414007 >> [v2:172.21.15.167:6818/2538273273,v1:172.21.15.167:6819/2538273273] conn(0x7f1114080b50 msgr2=0x7f1114082f70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:33.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.795+0000 7f113a800640 1 --2- 172.21.15.151:0/267414007 >> [v2:172.21.15.167:6818/2538273273,v1:172.21.15.167:6819/2538273273] conn(0x7f1114080b50 0x7f1114082f70 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.795+0000 7f570e600640 1 -- 172.21.15.151:0/703642271 shutdown_connections 2024-09-16T15:39:33.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.795+0000 7f570e600640 1 -- 172.21.15.151:0/703642271 wait complete. 2024-09-16T15:39:33.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.795+0000 7fc95fa00640 1 -- 172.21.15.151:0/716538852 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc9380788d0 msgr2=0x7fc93807ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:33.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.795+0000 7fc95fa00640 1 --2- 172.21.15.151:0/716538852 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc9380788d0 0x7fc93807ad90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc944001920 tx=0x7fc9440018b0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.795+0000 7fc95fa00640 1 -- 172.21.15.151:0/716538852 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc95815e690 msgr2=0x7fc958156e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:33.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.795+0000 7fc95fa00640 1 --2- 172.21.15.151:0/716538852 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc95815e690 0x7fc958156e60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc950010d40 tx=0x7fc95000e720 comp rx=0 tx=0).stop 2024-09-16T15:39:33.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.795+0000 7fc95da00640 1 -- 172.21.15.151:0/716538852 reap_dead start 2024-09-16T15:39:33.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.795+0000 7fc95fa00640 1 -- 172.21.15.151:0/716538852 shutdown_connections 2024-09-16T15:39:33.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.795+0000 7fc95fa00640 1 -- 172.21.15.151:0/716538852 >> 172.21.15.151:0/716538852 conn(0x7fc9580b1950 msgr2=0x7fc95815a160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:33.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.795+0000 7f113a800640 1 -- 172.21.15.151:0/267414007 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f11140788d0 msgr2=0x7f111407ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:33.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.795+0000 7f113a800640 1 --2- 172.21.15.151:0/267414007 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f11140788d0 0x7f111407ad90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1128001680 tx=0x7f1128009040 comp rx=0 tx=0).stop 2024-09-16T15:39:33.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.795+0000 7f113a800640 1 -- 172.21.15.151:0/267414007 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f113415c7a0 msgr2=0x7f11341f9d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:33.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.795+0000 7f113a800640 1 --2- 172.21.15.151:0/267414007 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f113415c7a0 0x7f11341f9d30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f112000c500 tx=0x7f112000c9c0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.795+0000 7fce50c00640 1 -- 172.21.15.151:0/1268495649 >> [v2:172.21.15.167:6826/1494916572,v1:172.21.15.167:6827/1494916572] conn(0x7fce3c0809f0 msgr2=0x7fce3c082e10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:33.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.795+0000 7fce50c00640 1 --2- 172.21.15.151:0/1268495649 >> [v2:172.21.15.167:6826/1494916572,v1:172.21.15.167:6827/1494916572] conn(0x7fce3c0809f0 0x7fce3c082e10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.795+0000 7f1139e00640 1 -- 172.21.15.151:0/267414007 reap_dead start 2024-09-16T15:39:33.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.795+0000 7fc95fa00640 1 -- 172.21.15.151:0/716538852 shutdown_connections 2024-09-16T15:39:33.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.795+0000 7fc95fa00640 1 -- 172.21.15.151:0/716538852 wait complete. 2024-09-16T15:39:33.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.796+0000 7fce50c00640 1 -- 172.21.15.151:0/1268495649 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fce3c078850 msgr2=0x7fce3c07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:33.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.796+0000 7fce50c00640 1 --2- 172.21.15.151:0/1268495649 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fce3c078850 0x7fce3c07ad10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fce54001840 tx=0x7fce540017d0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.796+0000 7fce50c00640 1 -- 172.21.15.151:0/1268495649 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce5c0cc5f0 msgr2=0x7fce5c0b05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:33.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.796+0000 7fce50c00640 1 --2- 172.21.15.151:0/1268495649 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce5c0cc5f0 0x7fce5c0b05d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fce4c00e970 tx=0x7fce4c00ed70 comp rx=0 tx=0).stop 2024-09-16T15:39:33.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.796+0000 7fce5be00640 1 -- 172.21.15.151:0/1268495649 reap_dead start 2024-09-16T15:39:33.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.796+0000 7f113a800640 1 -- 172.21.15.151:0/267414007 shutdown_connections 2024-09-16T15:39:33.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.796+0000 7f113a800640 1 -- 172.21.15.151:0/267414007 >> 172.21.15.151:0/267414007 conn(0x7f11340d3dc0 msgr2=0x7f11340d41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:33.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.796+0000 7f4eae200640 1 -- 172.21.15.151:0/3126668769 >> [v2:172.21.15.167:6810/2735495271,v1:172.21.15.167:6811/2735495271] conn(0x7f4e88080a20 msgr2=0x7f4e88082e40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:33.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.796+0000 7f4eae200640 1 --2- 172.21.15.151:0/3126668769 >> [v2:172.21.15.167:6810/2735495271,v1:172.21.15.167:6811/2735495271] conn(0x7f4e88080a20 0x7f4e88082e40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.796+0000 7fce50c00640 1 -- 172.21.15.151:0/1268495649 shutdown_connections 2024-09-16T15:39:33.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.796+0000 7fce50c00640 1 -- 172.21.15.151:0/1268495649 >> 172.21.15.151:0/1268495649 conn(0x7fce5c0b1950 msgr2=0x7fce5c0cdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:33.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.796+0000 7f113a800640 1 -- 172.21.15.151:0/267414007 shutdown_connections 2024-09-16T15:39:33.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.796+0000 7f113a800640 1 -- 172.21.15.151:0/267414007 wait complete. 2024-09-16T15:39:33.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.796+0000 7fce50c00640 1 -- 172.21.15.151:0/1268495649 shutdown_connections 2024-09-16T15:39:33.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.797+0000 7fce50c00640 1 -- 172.21.15.151:0/1268495649 wait complete. 2024-09-16T15:39:33.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.796+0000 7f4eae200640 1 -- 172.21.15.151:0/3126668769 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4e88078880 msgr2=0x7f4e8807ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:33.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.796+0000 7f4eae200640 1 --2- 172.21.15.151:0/3126668769 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4e88078880 0x7f4e8807ad40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c0057e0 tx=0x7f4e980019f0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.796+0000 7f4eae200640 1 -- 172.21.15.151:0/3126668769 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ea8153c00 msgr2=0x7f4ea81f7f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:33.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.796+0000 7f4eae200640 1 --2- 172.21.15.151:0/3126668769 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ea8153c00 0x7f4ea81f7f90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4e9400c3c0 tx=0x7f4e9400c7c0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.797+0000 7f4ea7e00640 1 -- 172.21.15.151:0/3126668769 reap_dead start 2024-09-16T15:39:33.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.801+0000 7f4eae200640 1 -- 172.21.15.151:0/3126668769 shutdown_connections 2024-09-16T15:39:33.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.801+0000 7f4eae200640 1 -- 172.21.15.151:0/3126668769 >> 172.21.15.151:0/3126668769 conn(0x7f4ea80d3fe0 msgr2=0x7f4ea80d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:33.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.802+0000 7f4eae200640 1 -- 172.21.15.151:0/3126668769 shutdown_connections 2024-09-16T15:39:33.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.802+0000 7f4eae200640 1 -- 172.21.15.151:0/3126668769 wait complete. 2024-09-16T15:39:33.810 INFO:teuthology.orchestra.run.smithi151.stdout:64424509446 2024-09-16T15:39:33.811 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-16T15:39:33.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.806+0000 7f2482800640 1 -- 172.21.15.151:0/1912303792 --> [v2:172.21.15.167:6802/265072227,v1:172.21.15.167:6803/265072227] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f247c06f910 con 0x7f2460081790 2024-09-16T15:39:33.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.807+0000 7f2479600640 1 -- 172.21.15.151:0/1912303792 <== osd.7 v2:172.21.15.167:6802/265072227 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f247c06f910 con 0x7f2460081790 2024-09-16T15:39:33.812 INFO:teuthology.orchestra.run.smithi151.stdout:64424509448 2024-09-16T15:39:33.813 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-16T15:39:33.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.808+0000 7f2482800640 1 -- 172.21.15.151:0/1912303792 >> [v2:172.21.15.167:6802/265072227,v1:172.21.15.167:6803/265072227] conn(0x7f2460081790 msgr2=0x7f2460083c30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:33.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.808+0000 7f2482800640 1 --2- 172.21.15.151:0/1912303792 >> [v2:172.21.15.167:6802/265072227,v1:172.21.15.167:6803/265072227] conn(0x7f2460081790 0x7f2460083c30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.808+0000 7f2482800640 1 -- 172.21.15.151:0/1912303792 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f24600788a0 msgr2=0x7f246007ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:33.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.808+0000 7f2482800640 1 --2- 172.21.15.151:0/1912303792 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f24600788a0 0x7f246007ad60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f24700099f0 tx=0x7f24700090e0 comp rx=0 tx=0).stop 2024-09-16T15:39:33.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.808+0000 7f2482800640 1 -- 172.21.15.151:0/1912303792 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f247c0af040 msgr2=0x7f247c1fc600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:33.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.808+0000 7f2482800640 1 --2- 172.21.15.151:0/1912303792 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f247c0af040 0x7f247c1fc600 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f246c031000 tx=0x7f246c003960 comp rx=0 tx=0).stop 2024-09-16T15:39:33.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.808+0000 7f2481e00640 1 -- 172.21.15.151:0/1912303792 reap_dead start 2024-09-16T15:39:33.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.808+0000 7f2482800640 1 -- 172.21.15.151:0/1912303792 shutdown_connections 2024-09-16T15:39:33.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.808+0000 7f2482800640 1 -- 172.21.15.151:0/1912303792 >> 172.21.15.151:0/1912303792 conn(0x7f247c0cfa20 msgr2=0x7f247c15be80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:33.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.809+0000 7f2482800640 1 -- 172.21.15.151:0/1912303792 shutdown_connections 2024-09-16T15:39:33.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:33.809+0000 7f2482800640 1 -- 172.21.15.151:0/1912303792 wait complete. 2024-09-16T15:39:33.821 INFO:teuthology.orchestra.run.smithi151.stdout:64424509449 2024-09-16T15:39:33.822 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-16T15:39:33.825 INFO:teuthology.orchestra.run.smithi151.stdout:64424509447 2024-09-16T15:39:33.826 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-16T15:39:33.827 INFO:teuthology.orchestra.run.smithi151.stdout:64424509447 2024-09-16T15:39:33.827 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-16T15:39:33.831 INFO:teuthology.orchestra.run.smithi151.stdout:64424509448 2024-09-16T15:39:33.832 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-16T15:39:33.835 INFO:teuthology.orchestra.run.smithi151.stdout:64424509447 2024-09-16T15:39:33.835 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-16T15:39:33.839 INFO:teuthology.orchestra.run.smithi151.stdout:60129542153 2024-09-16T15:39:33.840 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-16T15:39:34.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.097+0000 7fa7c4c00640 1 -- 172.21.15.151:0/2656764732 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7c0161440 msgr2=0x7fa7c00b5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:34.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.097+0000 7fa7c4c00640 1 --2- 172.21.15.151:0/2656764732 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7c0161440 0x7fa7c00b5370 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa7c00d4ea0 tx=0x7fa7ac02a470 comp rx=0 tx=0).stop 2024-09-16T15:39:34.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.098+0000 7fa7c4c00640 1 -- 172.21.15.151:0/2656764732 shutdown_connections 2024-09-16T15:39:34.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.098+0000 7fa7c4c00640 1 --2- 172.21.15.151:0/2656764732 >> v2:172.21.15.167:3300/0 conn(0x7fa7c00b4020 0x7fa7c00b1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.098+0000 7fa7c4c00640 1 --2- 172.21.15.151:0/2656764732 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7c0161440 0x7fa7c00b5370 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.098+0000 7fa7c4c00640 1 -- 172.21.15.151:0/2656764732 >> 172.21.15.151:0/2656764732 conn(0x7fa7c00d3df0 msgr2=0x7fa7c00d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:34.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.098+0000 7fa7c4c00640 1 -- 172.21.15.151:0/2656764732 shutdown_connections 2024-09-16T15:39:34.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.098+0000 7fa7c4c00640 1 -- 172.21.15.151:0/2656764732 wait complete. 2024-09-16T15:39:34.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.098+0000 7fdc22a00640 1 -- 172.21.15.151:0/519662286 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc140036d0 con 0x7fdc08004d30 2024-09-16T15:39:34.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.099+0000 7fdc29600640 1 -- 172.21.15.151:0/519662286 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc08004d30 msgr2=0x7fdc08005130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:34.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.099+0000 7fdc29600640 1 --2- 172.21.15.151:0/519662286 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc08004d30 0x7fdc08005130 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdc14001970 tx=0x7fdc08005830 comp rx=0 tx=0).stop 2024-09-16T15:39:34.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.099+0000 7fdc28c00640 1 -- 172.21.15.151:0/519662286 reap_dead start 2024-09-16T15:39:34.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.099+0000 7fdc29600640 1 -- 172.21.15.151:0/519662286 shutdown_connections 2024-09-16T15:39:34.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.100+0000 7fdc29600640 1 -- 172.21.15.151:0/519662286 >> 172.21.15.151:0/519662286 conn(0x7fdc240d3dc0 msgr2=0x7fdc240d41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:34.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.100+0000 7f666c400640 1 -- 172.21.15.151:0/1893480390 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f665c01c880 msgr2=0x7f665c01cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:34.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.100+0000 7f2c67e00640 1 -- 172.21.15.151:0/4257487518 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c600ce860 msgr2=0x7f2c600cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:34.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.100+0000 7f2c67e00640 1 --2- 172.21.15.151:0/4257487518 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c600ce860 0x7f2c600cecc0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c009860 tx=0x7f2c4c02a560 comp rx=0 tx=0).stop 2024-09-16T15:39:34.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.101+0000 7f666c400640 1 --2- 172.21.15.151:0/1893480390 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f665c01c880 0x7f665c01cc60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f66580098e0 tx=0x7f665802a530 comp rx=0 tx=0).stop 2024-09-16T15:39:34.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.101+0000 7f666c400640 1 -- 172.21.15.151:0/1893480390 shutdown_connections 2024-09-16T15:39:34.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.101+0000 7f666c400640 1 --2- 172.21.15.151:0/1893480390 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f665c005a50 0x7f665c035fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.101+0000 7f666c400640 1 --2- 172.21.15.151:0/1893480390 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f665c01c880 0x7f665c01cc60 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.101+0000 7f666c400640 1 -- 172.21.15.151:0/1893480390 >> 172.21.15.151:0/1893480390 conn(0x7f665c054550 msgr2=0x7f665c054930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:34.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.101+0000 7fa7c4c00640 1 Processor -- start 2024-09-16T15:39:34.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.101+0000 7f666c400640 1 -- 172.21.15.151:0/1893480390 shutdown_connections 2024-09-16T15:39:34.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.101+0000 7f666c400640 1 -- 172.21.15.151:0/1893480390 wait complete. 2024-09-16T15:39:34.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.101+0000 7fa7c4c00640 1 -- start start 2024-09-16T15:39:34.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.102+0000 7f2c67e00640 1 -- 172.21.15.151:0/4257487518 shutdown_connections 2024-09-16T15:39:34.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.102+0000 7f2c67e00640 1 --2- 172.21.15.151:0/4257487518 >> v2:172.21.15.151:3300/0 conn(0x7f2c600b24d0 0x7f2c600b2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.102+0000 7f2c67e00640 1 --2- 172.21.15.151:0/4257487518 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c600ce860 0x7f2c600cecc0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.102+0000 7f2c67e00640 1 -- 172.21.15.151:0/4257487518 >> 172.21.15.151:0/4257487518 conn(0x7f2c600d3de0 msgr2=0x7f2c600d41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:34.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.102+0000 7f2c67e00640 1 -- 172.21.15.151:0/4257487518 shutdown_connections 2024-09-16T15:39:34.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.102+0000 7f2c67e00640 1 -- 172.21.15.151:0/4257487518 wait complete. 2024-09-16T15:39:34.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.103+0000 7fdc29600640 1 -- 172.21.15.151:0/519662286 shutdown_connections 2024-09-16T15:39:34.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.103+0000 7fdc29600640 1 -- 172.21.15.151:0/519662286 wait complete. 2024-09-16T15:39:34.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.103+0000 7f666c400640 1 Processor -- start 2024-09-16T15:39:34.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.102+0000 7fa7c4c00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7c00b34b0 0x7fa7c01f9310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:34.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.102+0000 7fa7c4c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa7c00b4020 0x7fa7c01fc720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:34.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.102+0000 7fa7c4c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7c0161440 0x7fa7c006e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:34.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.102+0000 7fa7c4c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c00b2040 con 0x7fa7c00b4020 2024-09-16T15:39:34.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.103+0000 7fa7c4c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c00b1ec0 con 0x7fa7c0161440 2024-09-16T15:39:34.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.103+0000 7fa7c4c00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fa7c00b21c0 con 0x7fa7c00b34b0 2024-09-16T15:39:34.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.103+0000 7fa7bfe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7c0161440 0x7fa7c006e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:34.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.103+0000 7fa7bfe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7c0161440 0x7fa7c006e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.151:42628/0 (socket says 172.21.15.151:42628) 2024-09-16T15:39:34.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.103+0000 7fa7bfe00640 1 -- 172.21.15.151:0/3738310755 learned_addr learned my addr 172.21.15.151:0/3738310755 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:34.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.104+0000 7f666c400640 1 -- start start 2024-09-16T15:39:34.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.104+0000 7fa7bfe00640 1 -- 172.21.15.151:0/3738310755 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7c00b34b0 msgr2=0x7fa7c01f9310 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T15:39:34.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.104+0000 7fa7bfe00640 1 --2- 172.21.15.151:0/3738310755 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7c00b34b0 0x7fa7c01f9310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.104+0000 7fa7bfe00640 1 -- 172.21.15.151:0/3738310755 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa7c00b4020 msgr2=0x7fa7c01fc720 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:39:34.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.104+0000 7fa7bea00640 1 --2- 172.21.15.151:0/3738310755 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa7c00b4020 0x7fa7c01fc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:34.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.104+0000 7fa7bfe00640 1 --2- 172.21.15.151:0/3738310755 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa7c00b4020 0x7fa7c01fc720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.104+0000 7fa7bfe00640 1 -- 172.21.15.151:0/3738310755 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7c01fcc60 con 0x7fa7c0161440 2024-09-16T15:39:34.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.104+0000 7fa7bea00640 1 --2- 172.21.15.151:0/3738310755 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa7c00b4020 0x7fa7c01fc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:39:34.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.105+0000 7fa7bfe00640 1 --2- 172.21.15.151:0/3738310755 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7c0161440 0x7fa7c006e5e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa7b800efd0 tx=0x7fa7b800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:34.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.106+0000 7f666c400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f665c005a50 0x7f665c0de630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:34.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.106+0000 7fa7b7400640 1 -- 172.21.15.151:0/3738310755 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b8003b70 con 0x7fa7c0161440 2024-09-16T15:39:34.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.106+0000 7f666c400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f665c01c880 0x7f665c0494d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:34.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.106+0000 7f666c400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f665c036510 0x7f665c04f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:34.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.106+0000 7fa7b7400640 1 -- 172.21.15.151:0/3738310755 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa7b8003e00 con 0x7fa7c0161440 2024-09-16T15:39:34.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.106+0000 7fa7b7400640 1 -- 172.21.15.151:0/3738310755 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b8004280 con 0x7fa7c0161440 2024-09-16T15:39:34.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.106+0000 7f666c400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f665c0e2c80 con 0x7f665c036510 2024-09-16T15:39:34.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.106+0000 7f666c400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f665c00ac60 con 0x7f665c005a50 2024-09-16T15:39:34.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.106+0000 7f666c400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f665c0e2e00 con 0x7f665c01c880 2024-09-16T15:39:34.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.107+0000 7f666a400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f665c036510 0x7f665c04f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:34.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.106+0000 7fdc29600640 1 Processor -- start 2024-09-16T15:39:34.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.107+0000 7fa7c4c00640 1 -- 172.21.15.151:0/3738310755 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7c00cd9e0 con 0x7fa7c0161440 2024-09-16T15:39:34.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.107+0000 7f666a400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f665c036510 0x7f665c04f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:57202/0 (socket says 172.21.15.151:57202) 2024-09-16T15:39:34.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.107+0000 7f666a400640 1 -- 172.21.15.151:0/4056835422 learned_addr learned my addr 172.21.15.151:0/4056835422 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:34.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.107+0000 7f6669a00640 1 --2- 172.21.15.151:0/4056835422 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f665c005a50 0x7f665c0de630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:34.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.107+0000 7fa7c4c00640 1 -- 172.21.15.151:0/3738310755 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7c01f8b10 con 0x7fa7c0161440 2024-09-16T15:39:34.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.107+0000 7fdc29600640 1 -- start start 2024-09-16T15:39:34.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.107+0000 7f6669000640 1 --2- 172.21.15.151:0/4056835422 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f665c01c880 0x7f665c0494d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:34.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.108+0000 7f666a400640 1 -- 172.21.15.151:0/4056835422 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f665c01c880 msgr2=0x7f665c0494d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:34.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.108+0000 7f666a400640 1 --2- 172.21.15.151:0/4056835422 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f665c01c880 0x7f665c0494d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.108+0000 7f666a400640 1 -- 172.21.15.151:0/4056835422 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f665c005a50 msgr2=0x7f665c0de630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:34.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.108+0000 7f666a400640 1 --2- 172.21.15.151:0/4056835422 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f665c005a50 0x7f665c0de630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.108+0000 7f666a400640 1 -- 172.21.15.151:0/4056835422 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f665c0df1f0 con 0x7f665c036510 2024-09-16T15:39:34.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.108+0000 7f6669a00640 1 --2- 172.21.15.151:0/4056835422 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f665c005a50 0x7f665c0de630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:39:34.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.108+0000 7f666a400640 1 --2- 172.21.15.151:0/4056835422 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f665c036510 0x7f665c04f810 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f666406ba70 tx=0x7f666406df50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:34.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.109+0000 7fdc29600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc08004d30 0x7fdc240ca2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:34.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.109+0000 7f6661600640 1 -- 172.21.15.151:0/4056835422 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f666408f070 con 0x7f665c036510 2024-09-16T15:39:34.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.109+0000 7f6661600640 1 -- 172.21.15.151:0/4056835422 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f66640746e0 con 0x7f665c036510 2024-09-16T15:39:34.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.109+0000 7f6661600640 1 -- 172.21.15.151:0/4056835422 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6664074b70 con 0x7f665c036510 2024-09-16T15:39:34.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.109+0000 7fdc29600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdc241fa160 0x7fdc240ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:34.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.109+0000 7fdc29600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc241fbdf0 0x7fdc241fc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:34.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.109+0000 7fdc29600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdc24162450 con 0x7fdc241fa160 2024-09-16T15:39:34.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.109+0000 7fdc29600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdc241622d0 con 0x7fdc08004d30 2024-09-16T15:39:34.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.109+0000 7fdc29600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fdc241625d0 con 0x7fdc241fbdf0 2024-09-16T15:39:34.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.109+0000 7fa7c4c00640 1 -- 172.21.15.151:0/3738310755 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7c0150ae0 con 0x7fa7c0161440 2024-09-16T15:39:34.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.109+0000 7fdc23e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc08004d30 0x7fdc240ca2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:34.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.109+0000 7fdc23e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc08004d30 0x7fdc240ca2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.151:42648/0 (socket says 172.21.15.151:42648) 2024-09-16T15:39:34.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.109+0000 7fdc23e00640 1 -- 172.21.15.151:0/3247007243 learned_addr learned my addr 172.21.15.151:0/3247007243 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:34.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.109+0000 7f666c400640 1 -- 172.21.15.151:0/4056835422 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f665c0df4d0 con 0x7f665c036510 2024-09-16T15:39:34.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.110+0000 7fdc28c00640 1 --2- 172.21.15.151:0/3247007243 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc241fbdf0 0x7fdc241fc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:34.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.110+0000 7f2c67e00640 1 Processor -- start 2024-09-16T15:39:34.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.110+0000 7fdc28c00640 1 -- 172.21.15.151:0/3247007243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc08004d30 msgr2=0x7fdc240ca2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:34.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.110+0000 7fdc28c00640 1 --2- 172.21.15.151:0/3247007243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc08004d30 0x7fdc240ca2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.110+0000 7fdc28c00640 1 -- 172.21.15.151:0/3247007243 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdc241fa160 msgr2=0x7fdc240ccc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:34.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.110+0000 7fdc28c00640 1 --2- 172.21.15.151:0/3247007243 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdc241fa160 0x7fdc240ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.110+0000 7fdc28c00640 1 -- 172.21.15.151:0/3247007243 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc240cd150 con 0x7fdc241fbdf0 2024-09-16T15:39:34.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.110+0000 7f666c400640 1 -- 172.21.15.151:0/4056835422 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f665c0dfa90 con 0x7f665c036510 2024-09-16T15:39:34.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.111+0000 7fdc28c00640 1 --2- 172.21.15.151:0/3247007243 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc241fbdf0 0x7fdc241fc640 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdc2408c2a0 tx=0x7fdc1c010f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:34.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.110+0000 7f2c67e00640 1 -- start start 2024-09-16T15:39:34.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.111+0000 7fec07000640 1 -- 172.21.15.151:0/1927482534 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec000729e0 msgr2=0x7fec00072e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:34.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.111+0000 7fec07000640 1 --2- 172.21.15.151:0/1927482534 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec000729e0 0x7fec00072e70 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7febf8005fd0 tx=0x7febf802a7a0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.112+0000 7fdc1be00640 1 -- 172.21.15.151:0/3247007243 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c017070 con 0x7fdc241fbdf0 2024-09-16T15:39:34.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.112+0000 7fec07000640 1 -- 172.21.15.151:0/1927482534 shutdown_connections 2024-09-16T15:39:34.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.112+0000 7f666c400640 1 -- 172.21.15.151:0/4056835422 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f665c0e32b0 con 0x7f665c036510 2024-09-16T15:39:34.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.112+0000 7fec07000640 1 --2- 172.21.15.151:0/1927482534 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec000729e0 0x7fec00072e70 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.112+0000 7fec07000640 1 --2- 172.21.15.151:0/1927482534 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fec0006fa60 0x7fec0008b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.113+0000 7fec07000640 1 -- 172.21.15.151:0/1927482534 >> 172.21.15.151:0/1927482534 conn(0x7fec000b1950 msgr2=0x7fec000b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:34.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.112+0000 7fdc1be00640 1 -- 172.21.15.151:0/3247007243 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdc1c003970 con 0x7fdc241fbdf0 2024-09-16T15:39:34.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.112+0000 7fdc1be00640 1 -- 172.21.15.151:0/3247007243 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c003dc0 con 0x7fdc241fbdf0 2024-09-16T15:39:34.122 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.112+0000 7fa7b7400640 1 -- 172.21.15.151:0/3738310755 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fa7b8004b40 con 0x7fa7c0161440 2024-09-16T15:39:34.122 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.113+0000 7fa7b7400640 1 --2- 172.21.15.151:0/3738310755 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa7a00788a0 0x7fa7a007ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:34.122 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.113+0000 7fa7b7400640 1 -- 172.21.15.151:0/3738310755 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fa7b805ff60 con 0x7fa7c0161440 2024-09-16T15:39:34.122 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.113+0000 7f2c67e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c600b24d0 0x7f2c601f5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:34.122 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.113+0000 7fdc29600640 1 -- 172.21.15.151:0/3247007243 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc240cd6d0 con 0x7fdc241fbdf0 2024-09-16T15:39:34.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.113+0000 7fa7bf400640 1 --2- 172.21.15.151:0/3738310755 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa7a00788a0 0x7fa7a007ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:34.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.113+0000 7fdc29600640 1 -- 172.21.15.151:0/3247007243 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc240c7900 con 0x7fdc241fbdf0 2024-09-16T15:39:34.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.114+0000 7fec07000640 1 -- 172.21.15.151:0/1927482534 shutdown_connections 2024-09-16T15:39:34.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.114+0000 7f2c67e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c600ce860 0x7f2c600cae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:34.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.114+0000 7f2c67e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c6015dff0 0x7f2c600c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:34.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.114+0000 7f2c67e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2c600b1080 con 0x7f2c600b24d0 2024-09-16T15:39:34.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.114+0000 7f2c67e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2c600b2ec0 con 0x7f2c6015dff0 2024-09-16T15:39:34.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.114+0000 7f2c67e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f2c600c7c00 con 0x7f2c600ce860 2024-09-16T15:39:34.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.114+0000 7f2c65e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c6015dff0 0x7f2c600c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:34.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.114+0000 7f2c65e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c6015dff0 0x7f2c600c8fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.151:42666/0 (socket says 172.21.15.151:42666) 2024-09-16T15:39:34.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.114+0000 7f2c65e00640 1 -- 172.21.15.151:0/1467485888 learned_addr learned my addr 172.21.15.151:0/1467485888 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:34.125 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.115+0000 7fa7bf400640 1 --2- 172.21.15.151:0/3738310755 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa7a00788a0 0x7fa7a007ad60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac034000 tx=0x7fa7ac004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:34.125 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.114+0000 7fec07000640 1 -- 172.21.15.151:0/1927482534 wait complete. 2024-09-16T15:39:34.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.115+0000 7f2c64a00640 1 --2- 172.21.15.151:0/1467485888 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c600ce860 0x7f2c600cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:34.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.115+0000 7f2c65400640 1 --2- 172.21.15.151:0/1467485888 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c600b24d0 0x7f2c601f5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:34.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.115+0000 7f2c65e00640 1 -- 172.21.15.151:0/1467485888 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c600ce860 msgr2=0x7f2c600cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:34.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.115+0000 7f2c65e00640 1 --2- 172.21.15.151:0/1467485888 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c600ce860 0x7f2c600cae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.115+0000 7f2c65e00640 1 -- 172.21.15.151:0/1467485888 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c600b24d0 msgr2=0x7f2c601f5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:34.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.115+0000 7f2c65e00640 1 --2- 172.21.15.151:0/1467485888 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c600b24d0 0x7f2c601f5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.115+0000 7f2c65e00640 1 -- 172.21.15.151:0/1467485888 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c600cd850 con 0x7f2c6015dff0 2024-09-16T15:39:34.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.115+0000 7f2c65e00640 1 --2- 172.21.15.151:0/1467485888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c6015dff0 0x7f2c600c8fa0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2c58009cf0 tx=0x7f2c5800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:34.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.116+0000 7f2c5d600640 1 -- 172.21.15.151:0/1467485888 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c58003b70 con 0x7f2c6015dff0 2024-09-16T15:39:34.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.117+0000 7fa7b7400640 1 -- 172.21.15.151:0/3738310755 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa7c0150ae0 con 0x7fa7c0161440 2024-09-16T15:39:34.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.117+0000 7f2c5d600640 1 -- 172.21.15.151:0/1467485888 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2c58003d10 con 0x7f2c6015dff0 2024-09-16T15:39:34.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.117+0000 7f2c5d600640 1 -- 172.21.15.151:0/1467485888 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c58004280 con 0x7f2c6015dff0 2024-09-16T15:39:34.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.117+0000 7fdc29600640 1 -- 172.21.15.151:0/3247007243 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc240d1b70 con 0x7fdc241fbdf0 2024-09-16T15:39:34.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.116+0000 7f6661600640 1 -- 172.21.15.151:0/4056835422 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f6664092070 con 0x7f665c036510 2024-09-16T15:39:34.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.118+0000 7f6661600640 1 --2- 172.21.15.151:0/4056835422 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f66440788a0 0x7f664407ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:34.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.118+0000 7f6661600640 1 -- 172.21.15.151:0/4056835422 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f66641159e0 con 0x7f665c036510 2024-09-16T15:39:34.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.118+0000 7f2c67e00640 1 -- 172.21.15.151:0/1467485888 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c600cd100 con 0x7f2c6015dff0 2024-09-16T15:39:34.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.118+0000 7f2c67e00640 1 -- 172.21.15.151:0/1467485888 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c600cddf0 con 0x7f2c6015dff0 2024-09-16T15:39:34.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.120+0000 7f2c67e00640 1 -- 172.21.15.151:0/1467485888 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c600b3370 con 0x7f2c6015dff0 2024-09-16T15:39:34.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.120+0000 7fec07000640 1 Processor -- start 2024-09-16T15:39:34.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.118+0000 7fdc1be00640 1 -- 172.21.15.151:0/3247007243 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fdc1c003f60 con 0x7fdc241fbdf0 2024-09-16T15:39:34.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.120+0000 7fdc1be00640 1 --2- 172.21.15.151:0/3247007243 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdc040788d0 0x7fdc0407ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:34.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.120+0000 7fdc1be00640 1 -- 172.21.15.151:0/3247007243 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fdc1c09b330 con 0x7fdc241fbdf0 2024-09-16T15:39:34.130 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.120+0000 7fec07000640 1 -- start start 2024-09-16T15:39:34.130 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.121+0000 7fdc23e00640 1 --2- 172.21.15.151:0/3247007243 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdc040788d0 0x7fdc0407ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:34.130 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.120+0000 7f6669a00640 1 --2- 172.21.15.151:0/4056835422 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f66440788a0 0x7f664407ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:34.130 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.122+0000 7fdc23e00640 1 --2- 172.21.15.151:0/3247007243 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdc040788d0 0x7fdc0407ad90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fdc14000ff0 tx=0x7fdc14001380 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:34.131 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.122+0000 7fec07000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec0006e760 0x7fec0015f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:34.131 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.122+0000 7fec07000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec0006fa60 0x7fec00208ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:34.131 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.122+0000 7fec07000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fec000729e0 0x7fec002061c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:34.131 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.122+0000 7fec07000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fec000733b0 con 0x7fec0006fa60 2024-09-16T15:39:34.131 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.122+0000 7fec07000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fec0006f820 con 0x7fec0006e760 2024-09-16T15:39:34.131 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.122+0000 7fec07000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fec00073530 con 0x7fec000729e0 2024-09-16T15:39:34.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.122+0000 7f2c5d600640 1 -- 172.21.15.151:0/1467485888 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f2c58004b20 con 0x7f2c6015dff0 2024-09-16T15:39:34.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.123+0000 7febffe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec0006e760 0x7fec0015f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:34.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.123+0000 7febffe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec0006e760 0x7fec0015f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.151:42694/0 (socket says 172.21.15.151:42694) 2024-09-16T15:39:34.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.123+0000 7febffe00640 1 -- 172.21.15.151:0/2016375307 learned_addr learned my addr 172.21.15.151:0/2016375307 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:34.133 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.123+0000 7febff400640 1 --2- 172.21.15.151:0/2016375307 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec0006fa60 0x7fec00208ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:34.133 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.122+0000 7f6669a00640 1 --2- 172.21.15.151:0/4056835422 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f66440788a0 0x7f664407ad60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6658034000 tx=0x7f6658004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:34.133 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.123+0000 7f2c5d600640 1 --2- 172.21.15.151:0/1467485888 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2c40078920 0x7f2c4007ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:34.133 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.123+0000 7fec05000640 1 --2- 172.21.15.151:0/2016375307 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fec000729e0 0x7fec002061c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:34.134 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.123+0000 7f2c5d600640 1 -- 172.21.15.151:0/1467485888 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f2c5809baa0 con 0x7f2c6015dff0 2024-09-16T15:39:34.134 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.124+0000 7f2c65400640 1 --2- 172.21.15.151:0/1467485888 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2c40078920 0x7f2c4007ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:34.135 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.124+0000 7febffe00640 1 -- 172.21.15.151:0/2016375307 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fec000729e0 msgr2=0x7fec002061c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:34.135 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.124+0000 7febffe00640 1 --2- 172.21.15.151:0/2016375307 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fec000729e0 0x7fec002061c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.135 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.124+0000 7febffe00640 1 -- 172.21.15.151:0/2016375307 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec0006fa60 msgr2=0x7fec00208ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:34.135 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.124+0000 7febffe00640 1 --2- 172.21.15.151:0/2016375307 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec0006fa60 0x7fec00208ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.135 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.124+0000 7febffe00640 1 -- 172.21.15.151:0/2016375307 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec002097c0 con 0x7fec0006e760 2024-09-16T15:39:34.135 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.124+0000 7febff400640 1 --2- 172.21.15.151:0/2016375307 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec0006fa60 0x7fec00208ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:39:34.135 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.125+0000 7febffe00640 1 --2- 172.21.15.151:0/2016375307 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec0006e760 0x7fec0015f730 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7febf000c730 tx=0x7febf000cbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:34.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.125+0000 7f2c65400640 1 --2- 172.21.15.151:0/1467485888 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2c40078920 0x7f2c4007ade0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2c54000f60 tx=0x7f2c54009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:34.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.126+0000 7f6661600640 1 -- 172.21.15.151:0/4056835422 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f66640de0c0 con 0x7f665c036510 2024-09-16T15:39:34.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.126+0000 7febf7e00640 1 -- 172.21.15.151:0/2016375307 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf0017070 con 0x7fec0006e760 2024-09-16T15:39:34.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.126+0000 7febf7e00640 1 -- 172.21.15.151:0/2016375307 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7febf0005380 con 0x7fec0006e760 2024-09-16T15:39:34.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.126+0000 7febf7e00640 1 -- 172.21.15.151:0/2016375307 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf000a8a0 con 0x7fec0006e760 2024-09-16T15:39:34.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.126+0000 7fe6ccc00640 1 -- 172.21.15.151:0/479076834 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe6c806fd90 msgr2=0x7fe6c80b3f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:34.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.126+0000 7fe6ccc00640 1 --2- 172.21.15.151:0/479076834 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe6c806fd90 0x7fe6c80b3f00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8070850 tx=0x7fe6c002a6b0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.127+0000 7fe6ccc00640 1 -- 172.21.15.151:0/479076834 shutdown_connections 2024-09-16T15:39:34.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.127+0000 7fe6ccc00640 1 --2- 172.21.15.151:0/479076834 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe6c806fd90 0x7fe6c80b3f00 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.127+0000 7fe6ccc00640 1 --2- 172.21.15.151:0/479076834 >> v2:172.21.15.167:3300/0 conn(0x7fe6c80ccd00 0x7fe6c806f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.127+0000 7fe6ccc00640 1 -- 172.21.15.151:0/479076834 >> 172.21.15.151:0/479076834 conn(0x7fe6c80b1950 msgr2=0x7fe6c80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:34.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.127+0000 7fe6ccc00640 1 -- 172.21.15.151:0/479076834 shutdown_connections 2024-09-16T15:39:34.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.128+0000 7fec07000640 1 -- 172.21.15.151:0/2016375307 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec00209aa0 con 0x7fec0006e760 2024-09-16T15:39:34.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.128+0000 7f3b31c00640 1 -- 172.21.15.151:0/3459654024 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3b2c0b50c0 msgr2=0x7f3b2c0d2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:34.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.128+0000 7f3b31c00640 1 --2- 172.21.15.151:0/3459654024 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3b2c0b50c0 0x7f3b2c0d2ae0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3b18009900 tx=0x7f3b1802a5f0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.128+0000 7fec07000640 1 -- 172.21.15.151:0/2016375307 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec002059e0 con 0x7fec0006e760 2024-09-16T15:39:34.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.128+0000 7f3b31c00640 1 -- 172.21.15.151:0/3459654024 shutdown_connections 2024-09-16T15:39:34.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.128+0000 7f3b31c00640 1 --2- 172.21.15.151:0/3459654024 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3b2c0b50c0 0x7f3b2c0d2ae0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.128+0000 7f3b31c00640 1 --2- 172.21.15.151:0/3459654024 >> v2:172.21.15.167:3300/0 conn(0x7f3b2c0d21c0 0x7f3b2c0d25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.128+0000 7f3b31c00640 1 -- 172.21.15.151:0/3459654024 >> 172.21.15.151:0/3459654024 conn(0x7f3b2c0d3fe0 msgr2=0x7f3b2c0d43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:34.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.128+0000 7f3b31c00640 1 -- 172.21.15.151:0/3459654024 shutdown_connections 2024-09-16T15:39:34.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.128+0000 7f3b31c00640 1 -- 172.21.15.151:0/3459654024 wait complete. 2024-09-16T15:39:34.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.127+0000 7fe6ccc00640 1 -- 172.21.15.151:0/479076834 wait complete. 2024-09-16T15:39:34.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.129+0000 7f2c5d600640 1 -- 172.21.15.151:0/1467485888 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2c580640f0 con 0x7f2c6015dff0 2024-09-16T15:39:34.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.130+0000 7fdc1be00640 1 -- 172.21.15.151:0/3247007243 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fdc1c063a00 con 0x7fdc241fbdf0 2024-09-16T15:39:34.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.129+0000 7febf5600640 1 -- 172.21.15.151:0/2016375307 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febd4005290 con 0x7fec0006e760 2024-09-16T15:39:34.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.130+0000 7fe6ccc00640 1 Processor -- start 2024-09-16T15:39:34.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.130+0000 7fe6ccc00640 1 -- start start 2024-09-16T15:39:34.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.131+0000 7febf7e00640 1 -- 172.21.15.151:0/2016375307 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7febf001a070 con 0x7fec0006e760 2024-09-16T15:39:34.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.131+0000 7f3b31c00640 1 Processor -- start 2024-09-16T15:39:34.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.132+0000 7f3b31c00640 1 -- start start 2024-09-16T15:39:34.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.132+0000 7febf7e00640 1 --2- 172.21.15.151:0/2016375307 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7febe0078920 0x7febe007ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:34.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.132+0000 7febf7e00640 1 -- 172.21.15.151:0/2016375307 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7febf0099db0 con 0x7fec0006e760 2024-09-16T15:39:34.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.133+0000 7febff400640 1 --2- 172.21.15.151:0/2016375307 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7febe0078920 0x7febe007ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:34.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.132+0000 7fe6ccc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe6c806fd90 0x7fe6c815aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:34.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.133+0000 7fe6ccc00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6c80ccd00 0x7fe6c8073880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:34.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.133+0000 7fe6ccc00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6c81438b0 0x7fe6c8073dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:34.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.133+0000 7f3b31c00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3b2c0b50c0 0x7f3b2c14a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:34.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.134+0000 7febff400640 1 --2- 172.21.15.151:0/2016375307 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7febe0078920 0x7febe007ade0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fec0009bc40 tx=0x7febec009260 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:34.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.134+0000 7f3b31c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b2c0d21c0 0x7f3b2c1f7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:34.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.133+0000 7fe6ccc00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c8070aa0 con 0x7fe6c806fd90 2024-09-16T15:39:34.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.133+0000 7fe6ccc00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c8070920 con 0x7fe6c80ccd00 2024-09-16T15:39:34.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.134+0000 7fe6c6800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6c81438b0 0x7fe6c8073dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:34.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.134+0000 7fe6c6800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6c81438b0 0x7fe6c8073dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.151:48650/0 (socket says 172.21.15.151:48650) 2024-09-16T15:39:34.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.134+0000 7fe6c6800640 1 -- 172.21.15.151:0/2002038756 learned_addr learned my addr 172.21.15.151:0/2002038756 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:34.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.134+0000 7fe6ccc00640 1 -- 172.21.15.151:0/2002038756 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fe6c80b4440 con 0x7fe6c81438b0 2024-09-16T15:39:34.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.134+0000 7fe6c5e00640 1 --2- 172.21.15.151:0/2002038756 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe6c806fd90 0x7fe6c815aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:34.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.134+0000 7fe6c5400640 1 --2- 172.21.15.151:0/2002038756 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6c80ccd00 0x7fe6c8073880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:34.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.134+0000 7fe6c6800640 1 -- 172.21.15.151:0/2002038756 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6c80ccd00 msgr2=0x7fe6c8073880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:34.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.134+0000 7fe6c6800640 1 --2- 172.21.15.151:0/2002038756 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6c80ccd00 0x7fe6c8073880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.134+0000 7fe6c6800640 1 -- 172.21.15.151:0/2002038756 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe6c806fd90 msgr2=0x7fe6c815aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:34.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.134+0000 7f3b2aa00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3b2c0b50c0 0x7f3b2c14a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:34.146 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.135+0000 7fe6c6800640 1 --2- 172.21.15.151:0/2002038756 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe6c806fd90 0x7fe6c815aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.146 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.135+0000 7fe6c6800640 1 -- 172.21.15.151:0/2002038756 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6c81f1850 con 0x7fe6c81438b0 2024-09-16T15:39:34.146 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.135+0000 7fe6c5e00640 1 --2- 172.21.15.151:0/2002038756 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe6c806fd90 0x7fe6c815aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:39:34.146 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.135+0000 7f3b2aa00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3b2c0b50c0 0x7f3b2c14a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.151:48664/0 (socket says 172.21.15.151:48664) 2024-09-16T15:39:34.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.135+0000 7fe6c6800640 1 --2- 172.21.15.151:0/2002038756 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6c81438b0 0x7fe6c8073dc0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe6c0005fd0 tx=0x7fe6c0031770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:34.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.135+0000 7f3b2aa00640 1 -- 172.21.15.151:0/2525880307 learned_addr learned my addr 172.21.15.151:0/2525880307 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:34.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.136+0000 7f3b31c00640 1 --2- 172.21.15.151:0/2525880307 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b2c161230 0x7f3b2c159080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:34.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.136+0000 7f3b2b400640 1 --2- 172.21.15.151:0/2525880307 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b2c161230 0x7f3b2c159080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:34.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.136+0000 7febf7e00640 1 -- 172.21.15.151:0/2016375307 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7febf0061db0 con 0x7fec0006e760 2024-09-16T15:39:34.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.136+0000 7f3b2a000640 1 --2- 172.21.15.151:0/2525880307 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b2c0d21c0 0x7f3b2c1f7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:34.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.137+0000 7f3b31c00640 1 -- 172.21.15.151:0/2525880307 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2c161b30 con 0x7f3b2c161230 2024-09-16T15:39:34.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.137+0000 7f3b31c00640 1 -- 172.21.15.151:0/2525880307 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2c1619b0 con 0x7f3b2c0d21c0 2024-09-16T15:39:34.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.137+0000 7f3b31c00640 1 -- 172.21.15.151:0/2525880307 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f3b2c0b4050 con 0x7f3b2c0b50c0 2024-09-16T15:39:34.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.138+0000 7f3b2b400640 1 -- 172.21.15.151:0/2525880307 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3b2c0b50c0 msgr2=0x7f3b2c14a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:34.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.138+0000 7f3b2b400640 1 --2- 172.21.15.151:0/2525880307 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3b2c0b50c0 0x7f3b2c14a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.139+0000 7fe6be000640 1 -- 172.21.15.151:0/2002038756 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c003f070 con 0x7fe6c81438b0 2024-09-16T15:39:34.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.139+0000 7fe6be000640 1 -- 172.21.15.151:0/2002038756 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe6c0037040 con 0x7fe6c81438b0 2024-09-16T15:39:34.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.139+0000 7fe6be000640 1 -- 172.21.15.151:0/2002038756 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c0036070 con 0x7fe6c81438b0 2024-09-16T15:39:34.150 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.139+0000 7f3b2b400640 1 -- 172.21.15.151:0/2525880307 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b2c0d21c0 msgr2=0x7f3b2c1f7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:34.150 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.139+0000 7f3b2b400640 1 --2- 172.21.15.151:0/2525880307 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b2c0d21c0 0x7f3b2c1f7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.150 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.139+0000 7f3b2b400640 1 -- 172.21.15.151:0/2525880307 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b2c1f8df0 con 0x7f3b2c161230 2024-09-16T15:39:34.150 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.139+0000 7f3b2aa00640 1 --2- 172.21.15.151:0/2525880307 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3b2c0b50c0 0x7f3b2c14a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:39:34.150 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.139+0000 7f3b2a000640 1 --2- 172.21.15.151:0/2525880307 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b2c0d21c0 0x7f3b2c1f7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:39:34.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.139+0000 7fe6ccc00640 1 -- 172.21.15.151:0/2002038756 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6c80af5f0 con 0x7fe6c81438b0 2024-09-16T15:39:34.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.139+0000 7fe6ccc00640 1 -- 172.21.15.151:0/2002038756 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6c80afbb0 con 0x7fe6c81438b0 2024-09-16T15:39:34.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.140+0000 7f3b2b400640 1 --2- 172.21.15.151:0/2525880307 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b2c161230 0x7f3b2c159080 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3b2400be50 tx=0x7f3b2400bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:34.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.141+0000 7f3b22a00640 1 -- 172.21.15.151:0/2525880307 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b24019070 con 0x7f3b2c161230 2024-09-16T15:39:34.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.141+0000 7f3b22a00640 1 -- 172.21.15.151:0/2525880307 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3b24014440 con 0x7f3b2c161230 2024-09-16T15:39:34.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.141+0000 7f3b31c00640 1 -- 172.21.15.151:0/2525880307 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b2c1f8830 con 0x7f3b2c161230 2024-09-16T15:39:34.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.142+0000 7f3b31c00640 1 -- 172.21.15.151:0/2525880307 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b2c149fc0 con 0x7f3b2c161230 2024-09-16T15:39:34.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.142+0000 7fe6ccc00640 1 -- 172.21.15.151:0/2002038756 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6c80b3f00 con 0x7fe6c81438b0 2024-09-16T15:39:34.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.142+0000 7f3b22a00640 1 -- 172.21.15.151:0/2525880307 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b24014870 con 0x7f3b2c161230 2024-09-16T15:39:34.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.147+0000 7f3b22a00640 1 -- 172.21.15.151:0/2525880307 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f3b2401c070 con 0x7f3b2c161230 2024-09-16T15:39:34.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.148+0000 7fe6be000640 1 -- 172.21.15.151:0/2002038756 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fe6c000a0d0 con 0x7fe6c81438b0 2024-09-16T15:39:34.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.148+0000 7f3b31c00640 1 -- 172.21.15.151:0/2525880307 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b04005290 con 0x7f3b2c161230 2024-09-16T15:39:34.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.150+0000 7fe6be000640 1 --2- 172.21.15.151:0/2002038756 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe6a8078920 0x7fe6a807ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:34.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.150+0000 7fe6be000640 1 -- 172.21.15.151:0/2002038756 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fe6c00bab40 con 0x7fe6c81438b0 2024-09-16T15:39:34.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.151+0000 7fe6be000640 1 -- 172.21.15.151:0/2002038756 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe6c0083220 con 0x7fe6c81438b0 2024-09-16T15:39:34.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.152+0000 7fe6c5e00640 1 --2- 172.21.15.151:0/2002038756 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe6a8078920 0x7fe6a807ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:34.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.153+0000 7fe6c5e00640 1 --2- 172.21.15.151:0/2002038756 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe6a8078920 0x7fe6a807ade0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe6b80046b0 tx=0x7fe6b8009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:34.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.149+0000 7f3b22a00640 1 --2- 172.21.15.151:0/2525880307 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3b0c078920 0x7f3b0c07ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:34.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.149+0000 7f3b22a00640 1 -- 172.21.15.151:0/2525880307 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f3b2409a760 con 0x7f3b2c161230 2024-09-16T15:39:34.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.156+0000 7f3b2aa00640 1 --2- 172.21.15.151:0/2525880307 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3b0c078920 0x7f3b0c07ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:34.161 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.158+0000 7f3b2aa00640 1 --2- 172.21.15.151:0/2525880307 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3b0c078920 0x7f3b0c07ade0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c0099f0 tx=0x7f3b1c008040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:34.161 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.160+0000 7f3b22a00640 1 -- 172.21.15.151:0/2525880307 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f3b24062760 con 0x7f3b2c161230 2024-09-16T15:39:34.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.179+0000 7f6a47200640 1 -- 172.21.15.151:0/1106428868 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a40070150 msgr2=0x7f6a4006fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:34.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.179+0000 7f6a47200640 1 --2- 172.21.15.151:0/1106428868 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a40070150 0x7f6a4006fbf0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6a38005fd0 tx=0x7f6a3802a9c0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.180+0000 7f6a47200640 1 -- 172.21.15.151:0/1106428868 shutdown_connections 2024-09-16T15:39:34.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.180+0000 7f6a47200640 1 --2- 172.21.15.151:0/1106428868 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a40070150 0x7f6a4006fbf0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.180+0000 7f6a47200640 1 --2- 172.21.15.151:0/1106428868 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6a40161ee0 0x7f6a40073010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.180+0000 7f6a47200640 1 -- 172.21.15.151:0/1106428868 >> 172.21.15.151:0/1106428868 conn(0x7f6a4006cb70 msgr2=0x7f6a4006cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:34.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.180+0000 7f6a47200640 1 -- 172.21.15.151:0/1106428868 shutdown_connections 2024-09-16T15:39:34.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.180+0000 7f6a47200640 1 -- 172.21.15.151:0/1106428868 wait complete. 2024-09-16T15:39:34.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.183+0000 7f6a47200640 1 Processor -- start 2024-09-16T15:39:34.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.183+0000 7f6a47200640 1 -- start start 2024-09-16T15:39:34.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.184+0000 7f6a47200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a40070150 0x7f6a401fbb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:34.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.185+0000 7f6a47200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a40161b00 0x7f6a401fc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:34.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.185+0000 7f6a3fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a40070150 0x7f6a401fbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:34.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.185+0000 7f6a3fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a40070150 0x7f6a401fbb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.151:42748/0 (socket says 172.21.15.151:42748) 2024-09-16T15:39:34.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.185+0000 7f6a3f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a40161b00 0x7f6a401fc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:34.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.185+0000 7f6a3fe00640 1 -- 172.21.15.151:0/2233998867 learned_addr learned my addr 172.21.15.151:0/2233998867 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:34.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.185+0000 7f6a47200640 1 --2- 172.21.15.151:0/2233998867 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6a40161ee0 0x7f6a401fc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:34.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.185+0000 7f6a47200640 1 -- 172.21.15.151:0/2233998867 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6a4015f0b0 con 0x7f6a40161b00 2024-09-16T15:39:34.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.185+0000 7f6a47200640 1 -- 172.21.15.151:0/2233998867 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6a4015ef30 con 0x7f6a40070150 2024-09-16T15:39:34.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.185+0000 7f6a47200640 1 -- 172.21.15.151:0/2233998867 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f6a400b2ba0 con 0x7f6a40161ee0 2024-09-16T15:39:34.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.185+0000 7f6a3fe00640 1 -- 172.21.15.151:0/2233998867 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6a40161ee0 msgr2=0x7f6a401fc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:34.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.185+0000 7f6a3fe00640 1 --2- 172.21.15.151:0/2233998867 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6a40161ee0 0x7f6a401fc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.185+0000 7f6a3fe00640 1 -- 172.21.15.151:0/2233998867 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a40161b00 msgr2=0x7f6a401fc2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:34.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.185+0000 7f6a3fe00640 1 --2- 172.21.15.151:0/2233998867 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a40161b00 0x7f6a401fc2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.186+0000 7f6a3fe00640 1 -- 172.21.15.151:0/2233998867 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a401496e0 con 0x7f6a40070150 2024-09-16T15:39:34.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.187+0000 7f6a3fe00640 1 --2- 172.21.15.151:0/2233998867 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a40070150 0x7f6a401fbb90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c00c560 tx=0x7f6a2c00ca20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:34.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.187+0000 7f6a37e00640 1 -- 172.21.15.151:0/2233998867 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c0052d0 con 0x7f6a40070150 2024-09-16T15:39:34.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.188+0000 7f6a47200640 1 -- 172.21.15.151:0/2233998867 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a40149980 con 0x7f6a40070150 2024-09-16T15:39:34.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.188+0000 7f6a37e00640 1 -- 172.21.15.151:0/2233998867 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6a2c005470 con 0x7f6a40070150 2024-09-16T15:39:34.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.188+0000 7f6a37e00640 1 -- 172.21.15.151:0/2233998867 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c0052d0 con 0x7f6a40070150 2024-09-16T15:39:34.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.188+0000 7f6a47200640 1 -- 172.21.15.151:0/2233998867 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a40149f40 con 0x7f6a40070150 2024-09-16T15:39:34.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.190+0000 7f6a47200640 1 -- 172.21.15.151:0/2233998867 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a4006fbf0 con 0x7f6a40070150 2024-09-16T15:39:34.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.194+0000 7f6a37e00640 1 -- 172.21.15.151:0/2233998867 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f6a2c012630 con 0x7f6a40070150 2024-09-16T15:39:34.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.195+0000 7f6a37e00640 1 --2- 172.21.15.151:0/2233998867 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6a20078a50 0x7f6a2007af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:34.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.195+0000 7f6a37e00640 1 -- 172.21.15.151:0/2233998867 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f6a2c09aad0 con 0x7f6a40070150 2024-09-16T15:39:34.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.196+0000 7f6a37e00640 1 -- 172.21.15.151:0/2233998867 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6a2c063120 con 0x7f6a40070150 2024-09-16T15:39:34.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.200+0000 7f6a3f400640 1 --2- 172.21.15.151:0/2233998867 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6a20078a50 0x7f6a2007af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:34.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.201+0000 7f6a3f400640 1 --2- 172.21.15.151:0/2233998867 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6a20078a50 0x7f6a2007af10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6a40160d30 tx=0x7f6a30005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:34.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.546+0000 7fa7c4c00640 1 -- 172.21.15.151:0/3738310755 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fa7c00b1770 con 0x7fa7c0161440 2024-09-16T15:39:34.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.546+0000 7fa7b7400640 1 -- 172.21.15.151:0/3738310755 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa7c01fcc60 con 0x7fa7c0161440 2024-09-16T15:39:34.548 INFO:teuthology.orchestra.run.smithi151.stdout:64424509447 2024-09-16T15:39:34.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.550+0000 7fa7c4c00640 1 -- 172.21.15.151:0/3738310755 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa7a00788a0 msgr2=0x7fa7a007ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:34.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.550+0000 7f2c67e00640 1 -- 172.21.15.151:0/1467485888 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f2c600cd630 con 0x7f2c6015dff0 2024-09-16T15:39:34.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.550+0000 7fa7c4c00640 1 --2- 172.21.15.151:0/3738310755 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa7a00788a0 0x7fa7a007ad60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac034000 tx=0x7fa7ac004730 comp rx=0 tx=0).stop 2024-09-16T15:39:34.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.550+0000 7fa7c4c00640 1 -- 172.21.15.151:0/3738310755 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7c0161440 msgr2=0x7fa7c006e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:34.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.550+0000 7fa7c4c00640 1 --2- 172.21.15.151:0/3738310755 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7c0161440 0x7fa7c006e5e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa7b800efd0 tx=0x7fa7b800c6a0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.551+0000 7f2c5d600640 1 -- 172.21.15.151:0/1467485888 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2c580639f0 con 0x7f2c6015dff0 2024-09-16T15:39:34.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.551+0000 7fa7c4c00640 1 -- 172.21.15.151:0/3738310755 shutdown_connections 2024-09-16T15:39:34.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.551+0000 7fa7c4c00640 1 --2- 172.21.15.151:0/3738310755 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa7a00788a0 0x7fa7a007ad60 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.551+0000 7fa7c4c00640 1 --2- 172.21.15.151:0/3738310755 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7c0161440 0x7fa7c006e5e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.551+0000 7fa7c4c00640 1 --2- 172.21.15.151:0/3738310755 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa7c00b4020 0x7fa7c01fc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.551+0000 7fa7c4c00640 1 --2- 172.21.15.151:0/3738310755 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7c00b34b0 0x7fa7c01f9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.551+0000 7fa7c4c00640 1 -- 172.21.15.151:0/3738310755 >> 172.21.15.151:0/3738310755 conn(0x7fa7c00d3df0 msgr2=0x7fa7c015eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:34.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.551+0000 7fa7c4c00640 1 -- 172.21.15.151:0/3738310755 shutdown_connections 2024-09-16T15:39:34.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.552+0000 7fa7c4c00640 1 -- 172.21.15.151:0/3738310755 wait complete. 2024-09-16T15:39:34.556 INFO:teuthology.orchestra.run.smithi151.stdout:64424509448 2024-09-16T15:39:34.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.555+0000 7f2c67e00640 1 -- 172.21.15.151:0/1467485888 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2c40078920 msgr2=0x7f2c4007ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:34.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.555+0000 7f2c67e00640 1 --2- 172.21.15.151:0/1467485888 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2c40078920 0x7f2c4007ade0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2c54000f60 tx=0x7f2c54009040 comp rx=0 tx=0).stop 2024-09-16T15:39:34.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.556+0000 7f2c67e00640 1 -- 172.21.15.151:0/1467485888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c6015dff0 msgr2=0x7f2c600c8fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:34.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.556+0000 7f2c67e00640 1 --2- 172.21.15.151:0/1467485888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c6015dff0 0x7f2c600c8fa0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2c58009cf0 tx=0x7f2c5800c6a0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.556+0000 7f2c67e00640 1 -- 172.21.15.151:0/1467485888 shutdown_connections 2024-09-16T15:39:34.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.556+0000 7f2c67e00640 1 --2- 172.21.15.151:0/1467485888 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2c40078920 0x7f2c4007ade0 secure :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2c54000f60 tx=0x7f2c54009040 comp rx=0 tx=0).stop 2024-09-16T15:39:34.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.556+0000 7f2c67e00640 1 --2- 172.21.15.151:0/1467485888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c6015dff0 0x7f2c600c8fa0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.556+0000 7f2c67e00640 1 --2- 172.21.15.151:0/1467485888 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c600ce860 0x7f2c600cae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.556+0000 7f2c67e00640 1 --2- 172.21.15.151:0/1467485888 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c600b24d0 0x7f2c601f5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.556+0000 7f2c67e00640 1 -- 172.21.15.151:0/1467485888 >> 172.21.15.151:0/1467485888 conn(0x7f2c600d3de0 msgr2=0x7f2c600d2140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:34.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.557+0000 7f2c67e00640 1 -- 172.21.15.151:0/1467485888 shutdown_connections 2024-09-16T15:39:34.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.557+0000 7f2c67e00640 1 -- 172.21.15.151:0/1467485888 wait complete. 2024-09-16T15:39:34.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.563+0000 7f666c400640 1 -- 172.21.15.151:0/4056835422 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f665c054a60 con 0x7f665c036510 2024-09-16T15:39:34.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.564+0000 7fdc29600640 1 -- 172.21.15.151:0/3247007243 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fdc240b9d40 con 0x7fdc241fbdf0 2024-09-16T15:39:34.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.564+0000 7f6661600640 1 -- 172.21.15.151:0/4056835422 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f66640dd9c0 con 0x7f665c036510 2024-09-16T15:39:34.566 INFO:teuthology.orchestra.run.smithi151.stdout:64424509446 2024-09-16T15:39:34.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.565+0000 7fdc1be00640 1 -- 172.21.15.151:0/3247007243 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fdc1c063300 con 0x7fdc241fbdf0 2024-09-16T15:39:34.569 INFO:teuthology.orchestra.run.smithi151.stdout:64424509445 2024-09-16T15:39:34.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.569+0000 7f666c400640 1 -- 172.21.15.151:0/4056835422 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f66440788a0 msgr2=0x7f664407ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:34.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.569+0000 7f666c400640 1 --2- 172.21.15.151:0/4056835422 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f66440788a0 0x7f664407ad60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6658034000 tx=0x7f6658004730 comp rx=0 tx=0).stop 2024-09-16T15:39:34.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.569+0000 7f666c400640 1 -- 172.21.15.151:0/4056835422 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f665c036510 msgr2=0x7f665c04f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:34.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.569+0000 7f666c400640 1 --2- 172.21.15.151:0/4056835422 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f665c036510 0x7f665c04f810 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f666406ba70 tx=0x7f666406df50 comp rx=0 tx=0).stop 2024-09-16T15:39:34.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.569+0000 7fdc19600640 1 -- 172.21.15.151:0/3247007243 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdc040788d0 msgr2=0x7fdc0407ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:34.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.569+0000 7fdc19600640 1 --2- 172.21.15.151:0/3247007243 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdc040788d0 0x7fdc0407ad90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fdc14000ff0 tx=0x7fdc14001380 comp rx=0 tx=0).stop 2024-09-16T15:39:34.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.569+0000 7fdc19600640 1 -- 172.21.15.151:0/3247007243 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc241fbdf0 msgr2=0x7fdc241fc640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:34.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.569+0000 7fdc19600640 1 --2- 172.21.15.151:0/3247007243 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc241fbdf0 0x7fdc241fc640 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdc2408c2a0 tx=0x7fdc1c010f40 comp rx=0 tx=0).stop 2024-09-16T15:39:34.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.570+0000 7f666c400640 1 -- 172.21.15.151:0/4056835422 shutdown_connections 2024-09-16T15:39:34.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.570+0000 7f666c400640 1 --2- 172.21.15.151:0/4056835422 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f66440788a0 0x7f664407ad60 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.570+0000 7f666c400640 1 --2- 172.21.15.151:0/4056835422 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f665c036510 0x7f665c04f810 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.570+0000 7f666c400640 1 --2- 172.21.15.151:0/4056835422 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f665c01c880 0x7f665c0494d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.570+0000 7f666c400640 1 --2- 172.21.15.151:0/4056835422 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f665c005a50 0x7f665c0de630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.570+0000 7f666c400640 1 -- 172.21.15.151:0/4056835422 >> 172.21.15.151:0/4056835422 conn(0x7f665c054550 msgr2=0x7f665c050090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:34.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.570+0000 7fdc19600640 1 -- 172.21.15.151:0/3247007243 shutdown_connections 2024-09-16T15:39:34.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.570+0000 7fdc19600640 1 --2- 172.21.15.151:0/3247007243 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdc040788d0 0x7fdc0407ad90 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.570+0000 7f666c400640 1 -- 172.21.15.151:0/4056835422 shutdown_connections 2024-09-16T15:39:34.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.570+0000 7f666c400640 1 -- 172.21.15.151:0/4056835422 wait complete. 2024-09-16T15:39:34.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.570+0000 7fdc19600640 1 --2- 172.21.15.151:0/3247007243 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc241fbdf0 0x7fdc241fc640 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.570+0000 7fdc19600640 1 --2- 172.21.15.151:0/3247007243 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdc241fa160 0x7fdc240ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.570+0000 7fdc19600640 1 --2- 172.21.15.151:0/3247007243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc08004d30 0x7fdc240ca2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.570+0000 7fdc19600640 1 -- 172.21.15.151:0/3247007243 >> 172.21.15.151:0/3247007243 conn(0x7fdc240d3dc0 msgr2=0x7fdc2406d570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:34.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.571+0000 7fdc19600640 1 -- 172.21.15.151:0/3247007243 shutdown_connections 2024-09-16T15:39:34.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.571+0000 7fdc19600640 1 -- 172.21.15.151:0/3247007243 wait complete. 2024-09-16T15:39:34.580 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509447 for osd.1 2024-09-16T15:39:34.587 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509448 for osd.6 2024-09-16T15:39:34.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.587+0000 7f3b31c00640 1 -- 172.21.15.151:0/2525880307 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f3b04005580 con 0x7f3b2c161230 2024-09-16T15:39:34.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.588+0000 7f3b22a00640 1 -- 172.21.15.151:0/2525880307 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3b24067190 con 0x7f3b2c161230 2024-09-16T15:39:34.589 INFO:teuthology.orchestra.run.smithi151.stdout:64424509446 2024-09-16T15:39:34.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.588+0000 7febf5600640 1 -- 172.21.15.151:0/2016375307 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7febd4005580 con 0x7fec0006e760 2024-09-16T15:39:34.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.588+0000 7febf7e00640 1 -- 172.21.15.151:0/2016375307 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7febf00667e0 con 0x7fec0006e760 2024-09-16T15:39:34.590 INFO:teuthology.orchestra.run.smithi151.stdout:64424509446 2024-09-16T15:39:34.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.590+0000 7f3b31c00640 1 -- 172.21.15.151:0/2525880307 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3b0c078920 msgr2=0x7f3b0c07ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:34.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.590+0000 7f3b31c00640 1 --2- 172.21.15.151:0/2525880307 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3b0c078920 0x7f3b0c07ade0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c0099f0 tx=0x7f3b1c008040 comp rx=0 tx=0).stop 2024-09-16T15:39:34.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.590+0000 7f3b31c00640 1 -- 172.21.15.151:0/2525880307 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b2c161230 msgr2=0x7f3b2c159080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:34.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.590+0000 7f3b31c00640 1 --2- 172.21.15.151:0/2525880307 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b2c161230 0x7f3b2c159080 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3b2400be50 tx=0x7f3b2400bf40 comp rx=0 tx=0).stop 2024-09-16T15:39:34.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.591+0000 7f3b31c00640 1 -- 172.21.15.151:0/2525880307 shutdown_connections 2024-09-16T15:39:34.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.591+0000 7f3b31c00640 1 --2- 172.21.15.151:0/2525880307 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3b0c078920 0x7f3b0c07ade0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.591+0000 7f3b31c00640 1 --2- 172.21.15.151:0/2525880307 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b2c0d21c0 0x7f3b2c1f7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.591+0000 7f3b31c00640 1 --2- 172.21.15.151:0/2525880307 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b2c161230 0x7f3b2c159080 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.591+0000 7f3b31c00640 1 --2- 172.21.15.151:0/2525880307 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3b2c0b50c0 0x7f3b2c14a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.591+0000 7f3b31c00640 1 -- 172.21.15.151:0/2525880307 >> 172.21.15.151:0/2525880307 conn(0x7f3b2c0d3fe0 msgr2=0x7f3b2c06d7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:34.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.591+0000 7f3b31c00640 1 -- 172.21.15.151:0/2525880307 shutdown_connections 2024-09-16T15:39:34.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.591+0000 7f3b31c00640 1 -- 172.21.15.151:0/2525880307 wait complete. 2024-09-16T15:39:34.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.592+0000 7febf5600640 1 -- 172.21.15.151:0/2016375307 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7febe0078920 msgr2=0x7febe007ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:34.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.592+0000 7febf5600640 1 --2- 172.21.15.151:0/2016375307 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7febe0078920 0x7febe007ade0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fec0009bc40 tx=0x7febec009260 comp rx=0 tx=0).stop 2024-09-16T15:39:34.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.592+0000 7febf5600640 1 -- 172.21.15.151:0/2016375307 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec0006e760 msgr2=0x7fec0015f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:34.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.592+0000 7febf5600640 1 --2- 172.21.15.151:0/2016375307 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec0006e760 0x7fec0015f730 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7febf000c730 tx=0x7febf000cbf0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.592+0000 7febf5600640 1 -- 172.21.15.151:0/2016375307 shutdown_connections 2024-09-16T15:39:34.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.593+0000 7febf5600640 1 --2- 172.21.15.151:0/2016375307 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7febe0078920 0x7febe007ade0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.593+0000 7febf5600640 1 --2- 172.21.15.151:0/2016375307 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fec000729e0 0x7fec002061c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.593+0000 7febf5600640 1 --2- 172.21.15.151:0/2016375307 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec0006fa60 0x7fec00208ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.593+0000 7febf5600640 1 --2- 172.21.15.151:0/2016375307 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec0006e760 0x7fec0015f730 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.593+0000 7febf5600640 1 -- 172.21.15.151:0/2016375307 >> 172.21.15.151:0/2016375307 conn(0x7fec000b1950 msgr2=0x7fec000d0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:34.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.593+0000 7febf5600640 1 -- 172.21.15.151:0/2016375307 shutdown_connections 2024-09-16T15:39:34.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.593+0000 7febf5600640 1 -- 172.21.15.151:0/2016375307 wait complete. 2024-09-16T15:39:34.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.595+0000 7fe6ccc00640 1 -- 172.21.15.151:0/2002038756 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fe6c80cd430 con 0x7fe6c81438b0 2024-09-16T15:39:34.596 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.3 2024-09-16T15:39:34.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.596+0000 7fe6be000640 1 -- 172.21.15.151:0/2002038756 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe6c0082b20 con 0x7fe6c81438b0 2024-09-16T15:39:34.597 INFO:teuthology.orchestra.run.smithi151.stdout:64424509447 2024-09-16T15:39:34.599 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.0 2024-09-16T15:39:34.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.599+0000 7fe6ccc00640 1 -- 172.21.15.151:0/2002038756 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe6a8078920 msgr2=0x7fe6a807ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:34.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.599+0000 7fe6ccc00640 1 --2- 172.21.15.151:0/2002038756 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe6a8078920 0x7fe6a807ade0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe6b80046b0 tx=0x7fe6b8009040 comp rx=0 tx=0).stop 2024-09-16T15:39:34.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.599+0000 7fe6ccc00640 1 -- 172.21.15.151:0/2002038756 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6c81438b0 msgr2=0x7fe6c8073dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:34.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.599+0000 7fe6ccc00640 1 --2- 172.21.15.151:0/2002038756 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6c81438b0 0x7fe6c8073dc0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe6c0005fd0 tx=0x7fe6c0031770 comp rx=0 tx=0).stop 2024-09-16T15:39:34.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.599+0000 7fe6ccc00640 1 -- 172.21.15.151:0/2002038756 shutdown_connections 2024-09-16T15:39:34.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.599+0000 7fe6ccc00640 1 --2- 172.21.15.151:0/2002038756 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe6a8078920 0x7fe6a807ade0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.599+0000 7fe6ccc00640 1 --2- 172.21.15.151:0/2002038756 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6c80ccd00 0x7fe6c8073880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.599+0000 7fe6ccc00640 1 --2- 172.21.15.151:0/2002038756 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe6c81438b0 0x7fe6c8073dc0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.599+0000 7fe6ccc00640 1 --2- 172.21.15.151:0/2002038756 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe6c806fd90 0x7fe6c815aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.599+0000 7fe6ccc00640 1 -- 172.21.15.151:0/2002038756 >> 172.21.15.151:0/2002038756 conn(0x7fe6c80b1950 msgr2=0x7fe6c80b4950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:34.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.600+0000 7fe6ccc00640 1 -- 172.21.15.151:0/2002038756 shutdown_connections 2024-09-16T15:39:34.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.600+0000 7fe6ccc00640 1 -- 172.21.15.151:0/2002038756 wait complete. 2024-09-16T15:39:34.613 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.4 2024-09-16T15:39:34.614 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.613+0000 7f6a47200640 1 -- 172.21.15.151:0/2233998867 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f6a4014b120 con 0x7f6a40070150 2024-09-16T15:39:34.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.614+0000 7f6a37e00640 1 -- 172.21.15.151:0/2233998867 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6a2c062a20 con 0x7f6a40070150 2024-09-16T15:39:34.615 INFO:teuthology.orchestra.run.smithi151.stdout:60129542152 2024-09-16T15:39:34.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.616+0000 7f6a47200640 1 -- 172.21.15.151:0/2233998867 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6a20078a50 msgr2=0x7f6a2007af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:34.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.616+0000 7f6a47200640 1 --2- 172.21.15.151:0/2233998867 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6a20078a50 0x7f6a2007af10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6a40160d30 tx=0x7f6a30005f90 comp rx=0 tx=0).stop 2024-09-16T15:39:34.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.616+0000 7f6a47200640 1 -- 172.21.15.151:0/2233998867 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a40070150 msgr2=0x7f6a401fbb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:34.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.616+0000 7f6a47200640 1 --2- 172.21.15.151:0/2233998867 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a40070150 0x7f6a401fbb90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c00c560 tx=0x7f6a2c00ca20 comp rx=0 tx=0).stop 2024-09-16T15:39:34.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.616+0000 7f6a47200640 1 -- 172.21.15.151:0/2233998867 shutdown_connections 2024-09-16T15:39:34.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.616+0000 7f6a47200640 1 --2- 172.21.15.151:0/2233998867 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6a20078a50 0x7f6a2007af10 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.616+0000 7f6a47200640 1 --2- 172.21.15.151:0/2233998867 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6a40161ee0 0x7f6a401fc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.616+0000 7f6a47200640 1 --2- 172.21.15.151:0/2233998867 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a40161b00 0x7f6a401fc2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.616+0000 7f6a47200640 1 --2- 172.21.15.151:0/2233998867 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a40070150 0x7f6a401fbb90 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:34.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.616+0000 7f6a47200640 1 -- 172.21.15.151:0/2233998867 >> 172.21.15.151:0/2233998867 conn(0x7f6a4006cb70 msgr2=0x7f6a4014e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:34.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.617+0000 7f6a47200640 1 -- 172.21.15.151:0/2233998867 shutdown_connections 2024-09-16T15:39:34.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:34.617+0000 7f6a47200640 1 -- 172.21.15.151:0/2233998867 wait complete. 2024-09-16T15:39:34.623 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509447 for osd.5 2024-09-16T15:39:34.625 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.2 2024-09-16T15:39:34.637 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542153 got 60129542152 for osd.7 2024-09-16T15:39:35.580 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-16T15:39:35.588 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-16T15:39:35.598 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-16T15:39:35.600 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-16T15:39:35.614 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-16T15:39:35.624 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-16T15:39:35.626 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-16T15:39:35.638 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-16T15:39:35.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.819+0000 7faf71200640 1 -- 172.21.15.151:0/1461645281 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf6c0b0d30 msgr2=0x7faf6c06dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:35.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.819+0000 7faf71200640 1 --2- 172.21.15.151:0/1461645281 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf6c0b0d30 0x7faf6c06dc80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7faf6c0d1040 tx=0x7faf6402a980 comp rx=0 tx=0).stop 2024-09-16T15:39:35.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.819+0000 7faf71200640 1 -- 172.21.15.151:0/1461645281 shutdown_connections 2024-09-16T15:39:35.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.820+0000 7faf71200640 1 --2- 172.21.15.151:0/1461645281 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf6c0b0d30 0x7faf6c06dc80 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:35.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.820+0000 7faf71200640 1 --2- 172.21.15.151:0/1461645281 >> v2:172.21.15.167:3300/0 conn(0x7faf6c0cfa50 0x7faf6c071800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:35.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.820+0000 7faf71200640 1 --2- 172.21.15.151:0/1461645281 >> v2:172.21.15.151:3300/0 conn(0x7faf6c0d1990 0x7faf6c0d1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:35.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.820+0000 7faf71200640 1 -- 172.21.15.151:0/1461645281 >> 172.21.15.151:0/1461645281 conn(0x7faf6c0688b0 msgr2=0x7faf6c068cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:35.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.820+0000 7faf71200640 1 -- 172.21.15.151:0/1461645281 shutdown_connections 2024-09-16T15:39:35.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.821+0000 7faf71200640 1 -- 172.21.15.151:0/1461645281 wait complete. 2024-09-16T15:39:35.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.824+0000 7faf71200640 1 Processor -- start 2024-09-16T15:39:35.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.824+0000 7faf71200640 1 -- start start 2024-09-16T15:39:35.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.825+0000 7faf71200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf6c0b0d30 0x7faf6c0b3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:35.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.825+0000 7faf71200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf6c0d1990 0x7faf6c0ce1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:35.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.825+0000 7faf71200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf6c0ce6e0 0x7faf6c155780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:35.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.826+0000 7faf71200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7faf6c0b1320 con 0x7faf6c0ce6e0 2024-09-16T15:39:35.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.826+0000 7faf71200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7faf6c0b11a0 con 0x7faf6c0b0d30 2024-09-16T15:39:35.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.826+0000 7faf71200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7faf6c0b14a0 con 0x7faf6c0d1990 2024-09-16T15:39:35.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.826+0000 7faf69400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf6c0d1990 0x7faf6c0ce1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:35.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.826+0000 7faf69e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf6c0b0d30 0x7faf6c0b3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:35.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.826+0000 7faf69400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf6c0d1990 0x7faf6c0ce1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.151:48688/0 (socket says 172.21.15.151:48688) 2024-09-16T15:39:35.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.826+0000 7faf69400640 1 -- 172.21.15.151:0/2615718658 learned_addr learned my addr 172.21.15.151:0/2615718658 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:35.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.827+0000 7faf6a800640 1 --2- 172.21.15.151:0/2615718658 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf6c0ce6e0 0x7faf6c155780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:35.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.827+0000 7faf69400640 1 -- 172.21.15.151:0/2615718658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf6c0b0d30 msgr2=0x7faf6c0b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:35.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.827+0000 7faf69400640 1 --2- 172.21.15.151:0/2615718658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf6c0b0d30 0x7faf6c0b3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:35.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.827+0000 7faf69400640 1 -- 172.21.15.151:0/2615718658 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf6c0ce6e0 msgr2=0x7faf6c155780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:35.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.827+0000 7faf69400640 1 --2- 172.21.15.151:0/2615718658 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf6c0ce6e0 0x7faf6c155780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:35.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.827+0000 7faf69400640 1 -- 172.21.15.151:0/2615718658 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf6c0cbb30 con 0x7faf6c0d1990 2024-09-16T15:39:35.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.827+0000 7faf6a800640 1 --2- 172.21.15.151:0/2615718658 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf6c0ce6e0 0x7faf6c155780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:39:35.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.828+0000 7faf69400640 1 --2- 172.21.15.151:0/2615718658 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf6c0d1990 0x7faf6c0ce1a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7faf5800c920 tx=0x7faf5800cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:35.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.829+0000 7faf62000640 1 -- 172.21.15.151:0/2615718658 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf58007c70 con 0x7faf6c0d1990 2024-09-16T15:39:35.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.829+0000 7faf62000640 1 -- 172.21.15.151:0/2615718658 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faf5800f450 con 0x7faf6c0d1990 2024-09-16T15:39:35.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.829+0000 7faf71200640 1 -- 172.21.15.151:0/2615718658 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf6c0c9920 con 0x7faf6c0d1990 2024-09-16T15:39:35.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.829+0000 7faf71200640 1 -- 172.21.15.151:0/2615718658 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7faf6c0c9e70 con 0x7faf6c0d1990 2024-09-16T15:39:35.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.830+0000 7faf62000640 1 -- 172.21.15.151:0/2615718658 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf5800f8a0 con 0x7faf6c0d1990 2024-09-16T15:39:35.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.831+0000 7faf71200640 1 -- 172.21.15.151:0/2615718658 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf44005290 con 0x7faf6c0d1990 2024-09-16T15:39:35.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.834+0000 7faf62000640 1 -- 172.21.15.151:0/2615718658 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7faf5800fa40 con 0x7faf6c0d1990 2024-09-16T15:39:35.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.837+0000 7faf62000640 1 --2- 172.21.15.151:0/2615718658 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7faf4c078850 0x7faf4c07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:35.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.837+0000 7faf62000640 1 -- 172.21.15.151:0/2615718658 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7faf5809b1f0 con 0x7faf6c0d1990 2024-09-16T15:39:35.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.838+0000 7faf69e00640 1 --2- 172.21.15.151:0/2615718658 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7faf4c078850 0x7faf4c07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:35.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.839+0000 7faf69e00640 1 --2- 172.21.15.151:0/2615718658 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7faf4c078850 0x7faf4c07ad10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7faf5c00bad0 tx=0x7faf5c00b2e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:35.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.843+0000 7faf62000640 1 -- 172.21.15.151:0/2615718658 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7faf58014030 con 0x7faf6c0d1990 2024-09-16T15:39:35.899 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.898+0000 7f7cb1400640 1 -- 172.21.15.151:0/1704553772 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca4006b60 con 0x7f7c98003c10 2024-09-16T15:39:35.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.899+0000 7f7cb9400640 1 -- 172.21.15.151:0/1704553772 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c98003c10 msgr2=0x7f7c98003ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:35.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.899+0000 7f7cb9400640 1 --2- 172.21.15.151:0/1704553772 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c98003c10 0x7f7c98003ff0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7ca4006f40 tx=0x7f7c98005830 comp rx=0 tx=0).stop 2024-09-16T15:39:35.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.900+0000 7f7cb9400640 1 -- 172.21.15.151:0/1704553772 shutdown_connections 2024-09-16T15:39:35.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.900+0000 7f7cb9400640 1 --2- 172.21.15.151:0/1704553772 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c98003c10 0x7f7c98003ff0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:35.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.901+0000 7f7cb9400640 1 -- 172.21.15.151:0/1704553772 >> 172.21.15.151:0/1704553772 conn(0x7f7cb40d3e00 msgr2=0x7f7cb40d4210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:35.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.901+0000 7f7cb9400640 1 -- 172.21.15.151:0/1704553772 shutdown_connections 2024-09-16T15:39:35.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.902+0000 7f7cb9400640 1 -- 172.21.15.151:0/1704553772 wait complete. 2024-09-16T15:39:35.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.905+0000 7f7cb9400640 1 Processor -- start 2024-09-16T15:39:35.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.905+0000 7f7cb9400640 1 -- start start 2024-09-16T15:39:35.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.906+0000 7f795a800640 1 -- 172.21.15.151:0/484934560 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79380051c0 msgr2=0x7f793800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:35.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.906+0000 7f795a800640 1 --2- 172.21.15.151:0/484934560 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79380051c0 0x7f793800b9c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f794c005fd0 tx=0x7f7938005830 comp rx=0 tx=0).stop 2024-09-16T15:39:35.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.906+0000 7f7953e00640 1 -- 172.21.15.151:0/484934560 reap_dead start 2024-09-16T15:39:35.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.907+0000 7f7cb9400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c98003c10 0x7f7cb415fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:35.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.907+0000 7f7cb9400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7cb41553d0 0x7f7cb40b8f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:35.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.907+0000 7f7cb9400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7cb4155810 0x7f7cb41581f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:35.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.907+0000 7f795a800640 1 -- 172.21.15.151:0/484934560 shutdown_connections 2024-09-16T15:39:35.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.907+0000 7f795a800640 1 -- 172.21.15.151:0/484934560 >> 172.21.15.151:0/484934560 conn(0x7f795406cb70 msgr2=0x7f795406cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:35.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.907+0000 7f795a800640 1 -- 172.21.15.151:0/484934560 shutdown_connections 2024-09-16T15:39:35.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.908+0000 7f795a800640 1 -- 172.21.15.151:0/484934560 wait complete. 2024-09-16T15:39:35.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.908+0000 7f7cb3200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7cb4155810 0x7f7cb41581f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:35.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.908+0000 7f7cb1e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7cb41553d0 0x7f7cb40b8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:35.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.908+0000 7f7cb9400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb40d2770 con 0x7f7cb4155810 2024-09-16T15:39:35.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.908+0000 7f7cb9400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb40d25f0 con 0x7f7c98003c10 2024-09-16T15:39:35.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.908+0000 7f7cb9400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f7cb40d1ab0 con 0x7f7cb41553d0 2024-09-16T15:39:35.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.908+0000 7f7cb1e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7cb41553d0 0x7f7cb40b8f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.151:48726/0 (socket says 172.21.15.151:48726) 2024-09-16T15:39:35.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.908+0000 7f7cb1e00640 1 -- 172.21.15.151:0/2771051288 learned_addr learned my addr 172.21.15.151:0/2771051288 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:35.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.908+0000 7f7cb2800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c98003c10 0x7f7cb415fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:35.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.909+0000 7f7cb2800640 1 -- 172.21.15.151:0/2771051288 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7cb41553d0 msgr2=0x7f7cb40b8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:35.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.909+0000 7f7cb2800640 1 --2- 172.21.15.151:0/2771051288 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7cb41553d0 0x7f7cb40b8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:35.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.909+0000 7f7cb2800640 1 -- 172.21.15.151:0/2771051288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7cb4155810 msgr2=0x7f7cb41581f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:35.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.909+0000 7f7cb2800640 1 --2- 172.21.15.151:0/2771051288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7cb4155810 0x7f7cb41581f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:35.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.909+0000 7f7cb2800640 1 -- 172.21.15.151:0/2771051288 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cb4156090 con 0x7f7c98003c10 2024-09-16T15:39:35.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.910+0000 7f7cb2800640 1 --2- 172.21.15.151:0/2771051288 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c98003c10 0x7f7cb415fec0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7c9800c210 tx=0x7f7ca4003ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:35.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.910+0000 7f516dc00640 1 -- 172.21.15.151:0/2136843059 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51680d2050 msgr2=0x7f51680d10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:35.912 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.910+0000 7f795a800640 1 Processor -- start 2024-09-16T15:39:35.912 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.910+0000 7fe208e00640 1 -- 172.21.15.151:0/2968563439 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe2041621b0 msgr2=0x7fe20406eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:35.912 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.911+0000 7fe208e00640 1 --2- 172.21.15.151:0/2968563439 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe2041621b0 0x7fe20406eeb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0005fd0 tx=0x7fe1f002a540 comp rx=0 tx=0).stop 2024-09-16T15:39:35.912 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.910+0000 7f7caaa00640 1 -- 172.21.15.151:0/2771051288 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca4006b60 con 0x7f7c98003c10 2024-09-16T15:39:35.912 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.910+0000 7f7caaa00640 1 -- 172.21.15.151:0/2771051288 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ca403e030 con 0x7f7c98003c10 2024-09-16T15:39:35.912 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.910+0000 7f7caaa00640 1 -- 172.21.15.151:0/2771051288 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca4001be0 con 0x7f7c98003c10 2024-09-16T15:39:35.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.911+0000 7f795a800640 1 -- start start 2024-09-16T15:39:35.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.910+0000 7f516dc00640 1 --2- 172.21.15.151:0/2136843059 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51680d2050 0x7f51680d10d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5154009860 tx=0x7f515402a510 comp rx=0 tx=0).stop 2024-09-16T15:39:35.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.911+0000 7fe208e00640 1 -- 172.21.15.151:0/2968563439 shutdown_connections 2024-09-16T15:39:35.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.911+0000 7fe208e00640 1 --2- 172.21.15.151:0/2968563439 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe2041621b0 0x7fe20406eeb0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:35.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.911+0000 7fe208e00640 1 --2- 172.21.15.151:0/2968563439 >> v2:172.21.15.167:3300/0 conn(0x7fe2040b4cf0 0x7fe20406e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:35.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.911+0000 7fe208e00640 1 -- 172.21.15.151:0/2968563439 >> 172.21.15.151:0/2968563439 conn(0x7fe2040b1950 msgr2=0x7fe2040b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:35.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.911+0000 7fe208e00640 1 -- 172.21.15.151:0/2968563439 shutdown_connections 2024-09-16T15:39:35.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.911+0000 7fe208e00640 1 -- 172.21.15.151:0/2968563439 wait complete. 2024-09-16T15:39:35.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.911+0000 7f516dc00640 1 -- 172.21.15.151:0/2136843059 shutdown_connections 2024-09-16T15:39:35.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.911+0000 7f516dc00640 1 --2- 172.21.15.151:0/2136843059 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51680d2050 0x7f51680d10d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:35.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.911+0000 7f516dc00640 1 -- 172.21.15.151:0/2136843059 >> 172.21.15.151:0/2136843059 conn(0x7f51680d3fe0 msgr2=0x7f51680d43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:35.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.912+0000 7f7cb9400640 1 -- 172.21.15.151:0/2771051288 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cb4155db0 con 0x7f7c98003c10 2024-09-16T15:39:35.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.912+0000 7f7cb9400640 1 -- 172.21.15.151:0/2771051288 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cb40b8710 con 0x7f7c98003c10 2024-09-16T15:39:35.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.912+0000 7f516dc00640 1 -- 172.21.15.151:0/2136843059 shutdown_connections 2024-09-16T15:39:35.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.912+0000 7f795a800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79380051c0 0x7f795414eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:35.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.913+0000 7f795a800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79541fc260 0x7f79541fc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:35.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.913+0000 7f795a800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f795414bfd0 0x7f795414c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:35.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.913+0000 7f795a800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f795415eba0 con 0x7f795414bfd0 2024-09-16T15:39:35.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.913+0000 7f795a800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f795415ea20 con 0x7f79380051c0 2024-09-16T15:39:35.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.913+0000 7f795a800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f795415ed20 con 0x7f79541fc260 2024-09-16T15:39:35.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.913+0000 7f7953400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79380051c0 0x7f795414eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:35.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.913+0000 7f7953400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79380051c0 0x7f795414eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.151:42836/0 (socket says 172.21.15.151:42836) 2024-09-16T15:39:35.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.913+0000 7f7953400640 1 -- 172.21.15.151:0/3643638411 learned_addr learned my addr 172.21.15.151:0/3643638411 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:35.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.913+0000 7f7952a00640 1 --2- 172.21.15.151:0/3643638411 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79541fc260 0x7f79541fc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:35.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.914+0000 7f7c9fe00640 1 -- 172.21.15.151:0/2771051288 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cb40d1f50 con 0x7f7c98003c10 2024-09-16T15:39:35.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.914+0000 7fe208e00640 1 Processor -- start 2024-09-16T15:39:35.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.915+0000 7fe208e00640 1 -- start start 2024-09-16T15:39:35.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.915+0000 7f7952a00640 1 -- 172.21.15.151:0/3643638411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79380051c0 msgr2=0x7f795414eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:35.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.915+0000 7f7952a00640 1 --2- 172.21.15.151:0/3643638411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79380051c0 0x7f795414eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:35.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.915+0000 7f7952a00640 1 -- 172.21.15.151:0/3643638411 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f795414bfd0 msgr2=0x7f795414c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:35.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.915+0000 7f7952a00640 1 --2- 172.21.15.151:0/3643638411 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f795414bfd0 0x7f795414c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:35.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.915+0000 7f7952a00640 1 -- 172.21.15.151:0/3643638411 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79540b0870 con 0x7f79541fc260 2024-09-16T15:39:35.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.912+0000 7f516dc00640 1 -- 172.21.15.151:0/2136843059 wait complete. 2024-09-16T15:39:35.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.916+0000 7f7952a00640 1 --2- 172.21.15.151:0/3643638411 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79541fc260 0x7f79541fc6c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7940000fb0 tx=0x7f7940003a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:35.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.916+0000 7fe208e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe2040b4cf0 0x7fe20414acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:35.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.916+0000 7fe208e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe204161dd0 0x7fe20414dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:35.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.916+0000 7fe208e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2041621b0 0x7fe20414cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:35.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.916+0000 7fe208e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe20406f780 con 0x7fe2040b4cf0 2024-09-16T15:39:35.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.916+0000 7fe208e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe20406f600 con 0x7fe2041621b0 2024-09-16T15:39:35.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.916+0000 7fe208e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fe20415e570 con 0x7fe204161dd0 2024-09-16T15:39:35.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.916+0000 7f794b400640 1 -- 172.21.15.151:0/3643638411 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7940015070 con 0x7f79541fc260 2024-09-16T15:39:35.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.916+0000 7fe201e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe2040b4cf0 0x7fe20414acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:35.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.916+0000 7fe201e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe2040b4cf0 0x7fe20414acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:57346/0 (socket says 172.21.15.151:57346) 2024-09-16T15:39:35.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.916+0000 7fe201e00640 1 -- 172.21.15.151:0/4038336150 learned_addr learned my addr 172.21.15.151:0/4038336150 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:35.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.916+0000 7f516dc00640 1 Processor -- start 2024-09-16T15:39:35.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.917+0000 7f794b400640 1 -- 172.21.15.151:0/3643638411 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7940011900 con 0x7f79541fc260 2024-09-16T15:39:35.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.917+0000 7fe201400640 1 --2- 172.21.15.151:0/4038336150 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe204161dd0 0x7fe20414dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:35.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.917+0000 7fe202800640 1 --2- 172.21.15.151:0/4038336150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2041621b0 0x7fe20414cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:35.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.917+0000 7f795a800640 1 -- 172.21.15.151:0/3643638411 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79540b0b50 con 0x7f79541fc260 2024-09-16T15:39:35.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.917+0000 7fe201400640 1 -- 172.21.15.151:0/4038336150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2041621b0 msgr2=0x7fe20414cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:35.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.917+0000 7fe201400640 1 --2- 172.21.15.151:0/4038336150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2041621b0 0x7fe20414cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:35.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.917+0000 7fe201400640 1 -- 172.21.15.151:0/4038336150 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe2040b4cf0 msgr2=0x7fe20414acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:35.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.917+0000 7fe201400640 1 --2- 172.21.15.151:0/4038336150 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe2040b4cf0 0x7fe20414acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:35.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.917+0000 7fe201400640 1 -- 172.21.15.151:0/4038336150 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe20409ade0 con 0x7fe204161dd0 2024-09-16T15:39:35.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.917+0000 7fe201400640 1 --2- 172.21.15.151:0/4038336150 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe204161dd0 0x7fe20414dd60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc00ecb0 tx=0x7fe1fc00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:35.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.917+0000 7f516dc00640 1 -- start start 2024-09-16T15:39:35.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.918+0000 7fe1fa000640 1 -- 172.21.15.151:0/4038336150 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc003b70 con 0x7fe204161dd0 2024-09-16T15:39:35.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.919+0000 7fe1fa000640 1 -- 172.21.15.151:0/4038336150 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe1fc003e00 con 0x7fe204161dd0 2024-09-16T15:39:35.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.919+0000 7fe208e00640 1 -- 172.21.15.151:0/4038336150 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe20414c020 con 0x7fe204161dd0 2024-09-16T15:39:35.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.919+0000 7fe208e00640 1 -- 172.21.15.151:0/4038336150 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe20409b0c0 con 0x7fe204161dd0 2024-09-16T15:39:35.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.919+0000 7fe1fa000640 1 -- 172.21.15.151:0/4038336150 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc004260 con 0x7fe204161dd0 2024-09-16T15:39:35.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.919+0000 7ffb6a800640 1 -- 172.21.15.151:0/1915749310 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb640d2050 msgr2=0x7ffb640d14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:35.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.919+0000 7ffb6a800640 1 --2- 172.21.15.151:0/1915749310 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb640d2050 0x7ffb640d14b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ffb500097e0 tx=0x7ffb5002a4d0 comp rx=0 tx=0).stop 2024-09-16T15:39:35.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.917+0000 7f795a800640 1 -- 172.21.15.151:0/3643638411 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f79540b1110 con 0x7f79541fc260 2024-09-16T15:39:35.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.918+0000 7f794b400640 1 -- 172.21.15.151:0/3643638411 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7940011d50 con 0x7f79541fc260 2024-09-16T15:39:35.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.920+0000 7ffb6a800640 1 -- 172.21.15.151:0/1915749310 shutdown_connections 2024-09-16T15:39:35.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.920+0000 7ffb6a800640 1 --2- 172.21.15.151:0/1915749310 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb640d2050 0x7ffb640d14b0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:35.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.920+0000 7ffb6a800640 1 -- 172.21.15.151:0/1915749310 >> 172.21.15.151:0/1915749310 conn(0x7ffb640d3fe0 msgr2=0x7ffb640d43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:35.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.918+0000 7f516dc00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51680d1610 0x7f51680ca0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:35.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.918+0000 7f516dc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f51680d2050 0x7f51680b6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:35.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.918+0000 7f516dc00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f516815e0d0 0x7f51680c8b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:35.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.918+0000 7f516dc00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f51680cf840 con 0x7f51680d2050 2024-09-16T15:39:35.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.918+0000 7f516dc00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f51680cf6c0 con 0x7f51680d1610 2024-09-16T15:39:35.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.918+0000 7f516dc00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f51680cf9c0 con 0x7f516815e0d0 2024-09-16T15:39:35.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.919+0000 7f5166a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51680d1610 0x7f51680ca0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:35.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.919+0000 7f5166a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51680d1610 0x7f51680ca0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.151:42854/0 (socket says 172.21.15.151:42854) 2024-09-16T15:39:35.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.919+0000 7f5166a00640 1 -- 172.21.15.151:0/1564435853 learned_addr learned my addr 172.21.15.151:0/1564435853 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:35.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.919+0000 7f5166a00640 1 -- 172.21.15.151:0/1564435853 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f516815e0d0 msgr2=0x7f51680c8b40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:39:35.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.919+0000 7f5166a00640 1 --2- 172.21.15.151:0/1564435853 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f516815e0d0 0x7f51680c8b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:35.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.919+0000 7f5166a00640 1 -- 172.21.15.151:0/1564435853 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f51680d2050 msgr2=0x7f51680b6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:35.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.919+0000 7f5166000640 1 --2- 172.21.15.151:0/1564435853 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f51680d2050 0x7f51680b6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:35.926 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.920+0000 7f5166a00640 1 --2- 172.21.15.151:0/1564435853 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f51680d2050 0x7f51680b6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:35.926 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.920+0000 7f5166a00640 1 -- 172.21.15.151:0/1564435853 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51680ce050 con 0x7f51680d1610 2024-09-16T15:39:35.926 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.920+0000 7f795a800640 1 -- 172.21.15.151:0/3643638411 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f795415fa90 con 0x7f79541fc260 2024-09-16T15:39:35.926 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.920+0000 7f5166000640 1 --2- 172.21.15.151:0/1564435853 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f51680d2050 0x7f51680b6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:39:35.926 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.920+0000 7f5166a00640 1 --2- 172.21.15.151:0/1564435853 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51680d1610 0x7f51680ca0e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f515800c430 tx=0x7f515800c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:35.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.920+0000 7fe1ef400640 1 -- 172.21.15.151:0/4038336150 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1dc005290 con 0x7fe204161dd0 2024-09-16T15:39:35.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.920+0000 7ffb6a800640 1 -- 172.21.15.151:0/1915749310 shutdown_connections 2024-09-16T15:39:35.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.920+0000 7ffb6a800640 1 -- 172.21.15.151:0/1915749310 wait complete. 2024-09-16T15:39:35.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.921+0000 7f515ea00640 1 -- 172.21.15.151:0/1564435853 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5158017070 con 0x7f51680d1610 2024-09-16T15:39:35.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.921+0000 7f515ea00640 1 -- 172.21.15.151:0/1564435853 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f51580053c0 con 0x7f51680d1610 2024-09-16T15:39:35.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.921+0000 7f515ea00640 1 -- 172.21.15.151:0/1564435853 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5158003850 con 0x7f51680d1610 2024-09-16T15:39:35.928 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.922+0000 7f516dc00640 1 -- 172.21.15.151:0/1564435853 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51680cdd70 con 0x7f51680d1610 2024-09-16T15:39:35.928 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.922+0000 7ffb6a800640 1 Processor -- start 2024-09-16T15:39:35.928 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.922+0000 7fe1fa000640 1 -- 172.21.15.151:0/4038336150 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fe1fc004400 con 0x7fe204161dd0 2024-09-16T15:39:35.928 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.923+0000 7f516dc00640 1 -- 172.21.15.151:0/1564435853 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51680b9620 con 0x7f51680d1610 2024-09-16T15:39:35.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.923+0000 7ffb6a800640 1 -- start start 2024-09-16T15:39:35.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.923+0000 7f7caaa00640 1 -- 172.21.15.151:0/2771051288 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f7ca4006140 con 0x7f7c98003c10 2024-09-16T15:39:35.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.924+0000 7f1725c00640 1 -- 172.21.15.151:0/3136919099 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f17040039b0 msgr2=0x7f1704003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:35.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.924+0000 7f1725c00640 1 --2- 172.21.15.151:0/3136919099 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f17040039b0 0x7f1704003dd0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1710009b50 tx=0x7f1704005830 comp rx=0 tx=0).stop 2024-09-16T15:39:35.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.924+0000 7fe1fa000640 1 --2- 172.21.15.151:0/4038336150 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe1e40788a0 0x7fe1e407ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:35.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.924+0000 7ffb6a800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb640cfa20 0x7ffb640a1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:35.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.924+0000 7ffb6a800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb640d09c0 0x7ffb640cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:35.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.924+0000 7ffb6a800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb640d2050 0x7ffb6415e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:35.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.924+0000 7ffb6a800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ffb6416d8a0 con 0x7ffb640d2050 2024-09-16T15:39:35.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.924+0000 7ffb6a800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ffb6416d720 con 0x7ffb640d09c0 2024-09-16T15:39:35.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.924+0000 7ffb6a800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7ffb6416da20 con 0x7ffb640cfa20 2024-09-16T15:39:35.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.925+0000 7fe201e00640 1 --2- 172.21.15.151:0/4038336150 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe1e40788a0 0x7fe1e407ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:35.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.925+0000 7fe1fa000640 1 -- 172.21.15.151:0/4038336150 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fe1fc09b610 con 0x7fe204161dd0 2024-09-16T15:39:35.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.923+0000 7f794b400640 1 -- 172.21.15.151:0/3643638411 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f7940010420 con 0x7f79541fc260 2024-09-16T15:39:35.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.925+0000 7ffb63400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb640cfa20 0x7ffb640a1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:35.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.925+0000 7ffb62a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb640d09c0 0x7ffb640cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:35.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.925+0000 7ffb62a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb640d09c0 0x7ffb640cbaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.151:42864/0 (socket says 172.21.15.151:42864) 2024-09-16T15:39:35.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.925+0000 7ffb62a00640 1 -- 172.21.15.151:0/249649158 learned_addr learned my addr 172.21.15.151:0/249649158 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:35.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.925+0000 7f171f400640 1 -- 172.21.15.151:0/3136919099 reap_dead start 2024-09-16T15:39:35.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.924+0000 7f516dc00640 1 -- 172.21.15.151:0/1564435853 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51680cfee0 con 0x7f51680d1610 2024-09-16T15:39:35.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.925+0000 7f1725c00640 1 -- 172.21.15.151:0/3136919099 shutdown_connections 2024-09-16T15:39:35.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.925+0000 7f1725c00640 1 -- 172.21.15.151:0/3136919099 >> 172.21.15.151:0/3136919099 conn(0x7f1718055310 msgr2=0x7f1718055720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:35.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.925+0000 7ffb63e00640 1 --2- 172.21.15.151:0/249649158 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb640d2050 0x7ffb6415e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:35.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.925+0000 7ffb63400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb640cfa20 0x7ffb640a1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.151:48776/0 (socket says 172.21.15.151:48776) 2024-09-16T15:39:35.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.926+0000 7f1725c00640 1 -- 172.21.15.151:0/3136919099 shutdown_connections 2024-09-16T15:39:35.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.926+0000 7ffb62a00640 1 -- 172.21.15.151:0/249649158 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb640cfa20 msgr2=0x7ffb640a1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:35.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.926+0000 7ffb62a00640 1 --2- 172.21.15.151:0/249649158 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb640cfa20 0x7ffb640a1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:35.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.926+0000 7fe201e00640 1 --2- 172.21.15.151:0/4038336150 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe1e40788a0 0x7fe1e407ad60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe204151860 tx=0x7fe1f4009190 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:35.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.926+0000 7ffb62a00640 1 -- 172.21.15.151:0/249649158 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb640d2050 msgr2=0x7ffb6415e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:35.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.926+0000 7ffb62a00640 1 --2- 172.21.15.151:0/249649158 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb640d2050 0x7ffb6415e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:35.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.926+0000 7ffb62a00640 1 -- 172.21.15.151:0/249649158 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb640cc490 con 0x7ffb640d09c0 2024-09-16T15:39:35.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.926+0000 7f794b400640 1 --2- 172.21.15.151:0/3643638411 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f79340788d0 0x7f793407ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:35.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.925+0000 7f7caaa00640 1 --2- 172.21.15.151:0/2771051288 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7c94078950 0x7f7c9407ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:35.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.927+0000 7f794b400640 1 -- 172.21.15.151:0/3643638411 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f794009b500 con 0x7f79541fc260 2024-09-16T15:39:35.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.926+0000 7f1725c00640 1 -- 172.21.15.151:0/3136919099 wait complete. 2024-09-16T15:39:35.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.927+0000 7f7953400640 1 --2- 172.21.15.151:0/3643638411 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f79340788d0 0x7f793407ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:35.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.927+0000 7ffb62a00640 1 --2- 172.21.15.151:0/249649158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb640d09c0 0x7ffb640cbaf0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ffb5002aac0 tx=0x7ffb50003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:35.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.927+0000 7f7caaa00640 1 -- 172.21.15.151:0/2771051288 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f7ca40bb040 con 0x7f7c98003c10 2024-09-16T15:39:35.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.927+0000 7f7cb1e00640 1 --2- 172.21.15.151:0/2771051288 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7c94078950 0x7f7c9407ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:35.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.927+0000 7f7caaa00640 1 -- 172.21.15.151:0/2771051288 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7ca4048030 con 0x7f7c98003c10 2024-09-16T15:39:35.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.927+0000 7ffb5b400640 1 -- 172.21.15.151:0/249649158 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb50005c90 con 0x7ffb640d09c0 2024-09-16T15:39:35.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.928+0000 7f7953400640 1 --2- 172.21.15.151:0/3643638411 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f79340788d0 0x7f793407ad90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7944005bc0 tx=0x7f7944005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:35.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.928+0000 7ffb5b400640 1 -- 172.21.15.151:0/249649158 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffb50005110 con 0x7ffb640d09c0 2024-09-16T15:39:35.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.928+0000 7fe1fa000640 1 -- 172.21.15.151:0/4038336150 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe1fc063730 con 0x7fe204161dd0 2024-09-16T15:39:35.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.928+0000 7ffb6a800640 1 -- 172.21.15.151:0/249649158 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb640cc770 con 0x7ffb640d09c0 2024-09-16T15:39:35.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.928+0000 7ffb6a800640 1 -- 172.21.15.151:0/249649158 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb64158b50 con 0x7ffb640d09c0 2024-09-16T15:39:35.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.928+0000 7ffb5b400640 1 -- 172.21.15.151:0/249649158 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb50005500 con 0x7ffb640d09c0 2024-09-16T15:39:35.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.929+0000 7f7cb1e00640 1 --2- 172.21.15.151:0/2771051288 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7c94078950 0x7f7c9407ae10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7cb41ea0c0 tx=0x7f7ca0005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:35.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.928+0000 7f515ea00640 1 -- 172.21.15.151:0/1564435853 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f515801a070 con 0x7f51680d1610 2024-09-16T15:39:35.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.930+0000 7ffb6a800640 1 -- 172.21.15.151:0/249649158 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb3c005290 con 0x7ffb640d09c0 2024-09-16T15:39:35.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.928+0000 7f1725c00640 1 Processor -- start 2024-09-16T15:39:35.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.929+0000 7f1725c00640 1 -- start start 2024-09-16T15:39:35.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.930+0000 7f515ea00640 1 --2- 172.21.15.151:0/1564435853 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5148078920 0x7f514807ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:35.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.930+0000 7f515ea00640 1 -- 172.21.15.151:0/1564435853 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f5158099d10 con 0x7f51680d1610 2024-09-16T15:39:35.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.930+0000 7f5166000640 1 --2- 172.21.15.151:0/1564435853 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5148078920 0x7f514807ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:35.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.931+0000 7f1725c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17180d68c0 0x7f17180da560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:35.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.931+0000 7f1725c00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f17180e54f0 0x7f17180d9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:35.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.931+0000 7f5166000640 1 --2- 172.21.15.151:0/1564435853 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5148078920 0x7f514807ade0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5154034000 tx=0x7f5154004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:35.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.931+0000 7f1725c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f17180e5950 0x7f17180d9640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:35.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.931+0000 7f1725c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f171803c630 con 0x7f17180d68c0 2024-09-16T15:39:35.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.931+0000 7f1725c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f171803c4b0 con 0x7f17180e5950 2024-09-16T15:39:35.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.931+0000 7f1725c00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f171803c7b0 con 0x7f17180e54f0 2024-09-16T15:39:35.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.931+0000 7f171ea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17180d68c0 0x7f17180da560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:35.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.932+0000 7f171ea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17180d68c0 0x7f17180da560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:57374/0 (socket says 172.21.15.151:57374) 2024-09-16T15:39:35.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.932+0000 7f171ea00640 1 -- 172.21.15.151:0/2938170815 learned_addr learned my addr 172.21.15.151:0/2938170815 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:35.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.932+0000 7f171f400640 1 --2- 172.21.15.151:0/2938170815 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f17180e5950 0x7f17180d9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:35.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.932+0000 7f171e000640 1 --2- 172.21.15.151:0/2938170815 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f17180e54f0 0x7f17180d9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:35.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.932+0000 7f171ea00640 1 -- 172.21.15.151:0/2938170815 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f17180e54f0 msgr2=0x7f17180d9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:35.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.932+0000 7f171ea00640 1 --2- 172.21.15.151:0/2938170815 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f17180e54f0 0x7f17180d9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:35.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.932+0000 7f171ea00640 1 -- 172.21.15.151:0/2938170815 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f17180e5950 msgr2=0x7f17180d9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:35.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.932+0000 7f171ea00640 1 --2- 172.21.15.151:0/2938170815 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f17180e5950 0x7f17180d9640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:35.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.932+0000 7f171ea00640 1 -- 172.21.15.151:0/2938170815 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f171800c570 con 0x7f17180d68c0 2024-09-16T15:39:35.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.932+0000 7f171f400640 1 --2- 172.21.15.151:0/2938170815 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f17180e5950 0x7f17180d9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:39:35.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.933+0000 7f794b400640 1 -- 172.21.15.151:0/3643638411 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7940063590 con 0x7f79541fc260 2024-09-16T15:39:35.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.933+0000 7f171ea00640 1 --2- 172.21.15.151:0/2938170815 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17180d68c0 0x7f17180da560 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1710036040 tx=0x7f1710003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:35.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.934+0000 7f1716a00640 1 -- 172.21.15.151:0/2938170815 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1710040070 con 0x7f17180d68c0 2024-09-16T15:39:35.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.934+0000 7f515ea00640 1 -- 172.21.15.151:0/1564435853 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f51580623f0 con 0x7f51680d1610 2024-09-16T15:39:35.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.934+0000 7f1716a00640 1 -- 172.21.15.151:0/2938170815 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1710003ba0 con 0x7f17180d68c0 2024-09-16T15:39:35.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.934+0000 7f1716a00640 1 -- 172.21.15.151:0/2938170815 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1710038070 con 0x7f17180d68c0 2024-09-16T15:39:35.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.934+0000 7f1725c00640 1 -- 172.21.15.151:0/2938170815 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17180d73e0 con 0x7f17180d68c0 2024-09-16T15:39:35.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.934+0000 7f1725c00640 1 -- 172.21.15.151:0/2938170815 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17180d79a0 con 0x7f17180d68c0 2024-09-16T15:39:35.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.936+0000 7f1725c00640 1 -- 172.21.15.151:0/2938170815 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16f8005290 con 0x7f17180d68c0 2024-09-16T15:39:35.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.938+0000 7f1716a00640 1 -- 172.21.15.151:0/2938170815 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f1710043070 con 0x7f17180d68c0 2024-09-16T15:39:35.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.941+0000 7f1716a00640 1 --2- 172.21.15.151:0/2938170815 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1700078850 0x7f170007ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:35.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.942+0000 7f1716a00640 1 -- 172.21.15.151:0/2938170815 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f17100bbb80 con 0x7f17180d68c0 2024-09-16T15:39:35.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.942+0000 7f171e000640 1 --2- 172.21.15.151:0/2938170815 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1700078850 0x7f170007ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:35.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.942+0000 7ffb5b400640 1 -- 172.21.15.151:0/249649158 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7ffb500317c0 con 0x7ffb640d09c0 2024-09-16T15:39:35.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.944+0000 7ffb5b400640 1 --2- 172.21.15.151:0/249649158 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ffb44078920 0x7ffb4407ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:35.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.944+0000 7ffb5b400640 1 -- 172.21.15.151:0/249649158 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7ffb500bbba0 con 0x7ffb640d09c0 2024-09-16T15:39:35.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.945+0000 7ffb5b400640 1 -- 172.21.15.151:0/249649158 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ffb5003f030 con 0x7ffb640d09c0 2024-09-16T15:39:35.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.946+0000 7f171e000640 1 --2- 172.21.15.151:0/2938170815 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1700078850 0x7f170007ad10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f17180e4450 tx=0x7f170c005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:35.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.947+0000 7ffb63400640 1 --2- 172.21.15.151:0/249649158 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ffb44078920 0x7ffb4407ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:35.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.948+0000 7ffb63400640 1 --2- 172.21.15.151:0/249649158 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ffb44078920 0x7ffb4407ade0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ffb540017c0 tx=0x7ffb54009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:35.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.949+0000 7f1716a00640 1 -- 172.21.15.151:0/2938170815 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1710083b00 con 0x7f17180d68c0 2024-09-16T15:39:35.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.992+0000 7f48bc200640 1 -- 172.21.15.151:0/2036239998 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f48ac037f00 msgr2=0x7f48ac0e6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:35.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.992+0000 7f48bc200640 1 --2- 172.21.15.151:0/2036239998 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f48ac037f00 0x7f48ac0e6b30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f48a8009830 tx=0x7f48a802a5f0 comp rx=0 tx=0).stop 2024-09-16T15:39:35.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.993+0000 7f48bc200640 1 -- 172.21.15.151:0/2036239998 shutdown_connections 2024-09-16T15:39:35.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.993+0000 7f48bc200640 1 --2- 172.21.15.151:0/2036239998 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48ac03b930 0x7f48ac056c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:35.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.993+0000 7f48bc200640 1 --2- 172.21.15.151:0/2036239998 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f48ac037f00 0x7f48ac0e6b30 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:35.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.993+0000 7f48bc200640 1 -- 172.21.15.151:0/2036239998 >> 172.21.15.151:0/2036239998 conn(0x7f48ac03a3c0 msgr2=0x7f48ac03a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:35.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.993+0000 7f48bc200640 1 -- 172.21.15.151:0/2036239998 shutdown_connections 2024-09-16T15:39:35.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.993+0000 7f48bc200640 1 -- 172.21.15.151:0/2036239998 wait complete. 2024-09-16T15:39:35.997 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.995+0000 7f48bc200640 1 Processor -- start 2024-09-16T15:39:35.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.996+0000 7f48bc200640 1 -- start start 2024-09-16T15:39:35.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.997+0000 7f48bc200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f48ac037f00 0x7f48ac0dbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:35.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.997+0000 7f48bc200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48ac03b930 0x7f48ac179260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:35.999 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.998+0000 7f48b9800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f48ac037f00 0x7f48ac0dbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:35.999 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.998+0000 7f48b9800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f48ac037f00 0x7f48ac0dbfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:57390/0 (socket says 172.21.15.151:57390) 2024-09-16T15:39:35.999 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.998+0000 7f48b8e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48ac03b930 0x7f48ac179260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:36.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.998+0000 7f48bc200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48ac0e2790 0x7f48ac04c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:36.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.998+0000 7f48b8e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48ac03b930 0x7f48ac179260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.151:48806/0 (socket says 172.21.15.151:48806) 2024-09-16T15:39:36.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.998+0000 7f48b9800640 1 -- 172.21.15.151:0/3670060766 learned_addr learned my addr 172.21.15.151:0/3670060766 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:36.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.998+0000 7f48bc200640 1 -- 172.21.15.151:0/3670060766 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f48ac0575b0 con 0x7f48ac037f00 2024-09-16T15:39:36.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.998+0000 7f48bc200640 1 -- 172.21.15.151:0/3670060766 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f48ac057430 con 0x7f48ac0e2790 2024-09-16T15:39:36.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.998+0000 7f48bc200640 1 -- 172.21.15.151:0/3670060766 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f48ac057730 con 0x7f48ac03b930 2024-09-16T15:39:36.001 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.999+0000 7f48b8e00640 1 -- 172.21.15.151:0/3670060766 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48ac0e2790 msgr2=0x7f48ac04c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:36.001 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.999+0000 7f48b8e00640 1 --2- 172.21.15.151:0/3670060766 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48ac0e2790 0x7f48ac04c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:36.001 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.999+0000 7f48b8e00640 1 -- 172.21.15.151:0/3670060766 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f48ac037f00 msgr2=0x7f48ac0dbfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:36.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.999+0000 7f48b8e00640 1 --2- 172.21.15.151:0/3670060766 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f48ac037f00 0x7f48ac0dbfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:36.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:35.999+0000 7f48b8e00640 1 -- 172.21.15.151:0/3670060766 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48ac179bd0 con 0x7f48ac03b930 2024-09-16T15:39:36.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.000+0000 7f48b8e00640 1 --2- 172.21.15.151:0/3670060766 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48ac03b930 0x7f48ac179260 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f48ac03b360 tx=0x7f48a8032b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:36.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.001+0000 7f48b1600640 1 -- 172.21.15.151:0/3670060766 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a8040070 con 0x7f48ac03b930 2024-09-16T15:39:36.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.001+0000 7f48b1600640 1 -- 172.21.15.151:0/3670060766 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f48a80037f0 con 0x7f48ac03b930 2024-09-16T15:39:36.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.001+0000 7f48b1600640 1 -- 172.21.15.151:0/3670060766 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a8003c40 con 0x7f48ac03b930 2024-09-16T15:39:36.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.001+0000 7f48bc200640 1 -- 172.21.15.151:0/3670060766 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48ac179eb0 con 0x7f48ac03b930 2024-09-16T15:39:36.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.001+0000 7f48bc200640 1 -- 172.21.15.151:0/3670060766 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f48ac04c1d0 con 0x7f48ac03b930 2024-09-16T15:39:36.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.003+0000 7f48bc200640 1 -- 172.21.15.151:0/3670060766 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48ac0e6c50 con 0x7f48ac03b930 2024-09-16T15:39:36.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.004+0000 7f48b1600640 1 -- 172.21.15.151:0/3670060766 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f48a8039040 con 0x7f48ac03b930 2024-09-16T15:39:36.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.006+0000 7f48b1600640 1 --2- 172.21.15.151:0/3670060766 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f48940788d0 0x7f489407ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:36.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.006+0000 7f48b1600640 1 -- 172.21.15.151:0/3670060766 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f48a8003de0 con 0x7f48ac03b930 2024-09-16T15:39:36.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.007+0000 7f48b9800640 1 --2- 172.21.15.151:0/3670060766 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f48940788d0 0x7f489407ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:36.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.007+0000 7f48b9800640 1 --2- 172.21.15.151:0/3670060766 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f48940788d0 0x7f489407ad90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f48a00046b0 tx=0x7f48a0009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:36.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.012+0000 7f48b1600640 1 -- 172.21.15.151:0/3670060766 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f48a8084200 con 0x7f48ac03b930 2024-09-16T15:39:36.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.255+0000 7faf71200640 1 -- 172.21.15.151:0/2615718658 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7faf44005580 con 0x7faf6c0d1990 2024-09-16T15:39:36.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.256+0000 7faf62000640 1 -- 172.21.15.151:0/2615718658 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7faf580637c0 con 0x7faf6c0d1990 2024-09-16T15:39:36.261 INFO:teuthology.orchestra.run.smithi151.stdout:64424509448 2024-09-16T15:39:36.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.260+0000 7faf71200640 1 -- 172.21.15.151:0/2615718658 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7faf4c078850 msgr2=0x7faf4c07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:36.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.260+0000 7faf71200640 1 --2- 172.21.15.151:0/2615718658 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7faf4c078850 0x7faf4c07ad10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7faf5c00bad0 tx=0x7faf5c00b2e0 comp rx=0 tx=0).stop 2024-09-16T15:39:36.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.261+0000 7faf71200640 1 -- 172.21.15.151:0/2615718658 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf6c0d1990 msgr2=0x7faf6c0ce1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:36.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.261+0000 7faf71200640 1 --2- 172.21.15.151:0/2615718658 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf6c0d1990 0x7faf6c0ce1a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7faf5800c920 tx=0x7faf5800cde0 comp rx=0 tx=0).stop 2024-09-16T15:39:36.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.261+0000 7faf71200640 1 -- 172.21.15.151:0/2615718658 shutdown_connections 2024-09-16T15:39:36.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.261+0000 7faf71200640 1 --2- 172.21.15.151:0/2615718658 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7faf4c078850 0x7faf4c07ad10 secure :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7faf5c00bad0 tx=0x7faf5c00b2e0 comp rx=0 tx=0).stop 2024-09-16T15:39:36.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.261+0000 7faf71200640 1 --2- 172.21.15.151:0/2615718658 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf6c0ce6e0 0x7faf6c155780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:36.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.262+0000 7faf71200640 1 --2- 172.21.15.151:0/2615718658 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf6c0d1990 0x7faf6c0ce1a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:36.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.262+0000 7faf71200640 1 --2- 172.21.15.151:0/2615718658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf6c0b0d30 0x7faf6c0b3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:36.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.262+0000 7faf71200640 1 -- 172.21.15.151:0/2615718658 >> 172.21.15.151:0/2615718658 conn(0x7faf6c0688b0 msgr2=0x7faf6c15bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:36.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.262+0000 7faf71200640 1 -- 172.21.15.151:0/2615718658 shutdown_connections 2024-09-16T15:39:36.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.262+0000 7faf71200640 1 -- 172.21.15.151:0/2615718658 wait complete. 2024-09-16T15:39:36.289 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.1 2024-09-16T15:39:36.290 DEBUG:teuthology.parallel:result is None 2024-09-16T15:39:36.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.322+0000 7fe1ef400640 1 -- 172.21.15.151:0/4038336150 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fe1dc005580 con 0x7fe204161dd0 2024-09-16T15:39:36.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.323+0000 7fe1fa000640 1 -- 172.21.15.151:0/4038336150 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe1fc068160 con 0x7fe204161dd0 2024-09-16T15:39:36.324 INFO:teuthology.orchestra.run.smithi151.stdout:64424509446 2024-09-16T15:39:36.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.326+0000 7fe1ef400640 1 -- 172.21.15.151:0/4038336150 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe1e40788a0 msgr2=0x7fe1e407ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:36.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.326+0000 7fe1ef400640 1 --2- 172.21.15.151:0/4038336150 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe1e40788a0 0x7fe1e407ad60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe204151860 tx=0x7fe1f4009190 comp rx=0 tx=0).stop 2024-09-16T15:39:36.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.326+0000 7fe1ef400640 1 -- 172.21.15.151:0/4038336150 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe204161dd0 msgr2=0x7fe20414dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:36.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.326+0000 7fe1ef400640 1 --2- 172.21.15.151:0/4038336150 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe204161dd0 0x7fe20414dd60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc00ecb0 tx=0x7fe1fc00c6a0 comp rx=0 tx=0).stop 2024-09-16T15:39:36.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.326+0000 7fe1ef400640 1 -- 172.21.15.151:0/4038336150 shutdown_connections 2024-09-16T15:39:36.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.326+0000 7fe1ef400640 1 --2- 172.21.15.151:0/4038336150 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe1e40788a0 0x7fe1e407ad60 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:36.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.326+0000 7fe1ef400640 1 --2- 172.21.15.151:0/4038336150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2041621b0 0x7fe20414cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:36.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.326+0000 7fe1ef400640 1 --2- 172.21.15.151:0/4038336150 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe204161dd0 0x7fe20414dd60 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:36.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.326+0000 7fe1ef400640 1 --2- 172.21.15.151:0/4038336150 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe2040b4cf0 0x7fe20414acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:36.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.326+0000 7fe1ef400640 1 -- 172.21.15.151:0/4038336150 >> 172.21.15.151:0/4038336150 conn(0x7fe2040b1950 msgr2=0x7fe204151fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:36.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.326+0000 7fe1ef400640 1 -- 172.21.15.151:0/4038336150 shutdown_connections 2024-09-16T15:39:36.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.326+0000 7fe1ef400640 1 -- 172.21.15.151:0/4038336150 wait complete. 2024-09-16T15:39:36.344 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.0 2024-09-16T15:39:36.344 DEBUG:teuthology.parallel:result is None 2024-09-16T15:39:36.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.364+0000 7f795a800640 1 -- 172.21.15.151:0/3643638411 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f795415f7b0 con 0x7f79541fc260 2024-09-16T15:39:36.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.365+0000 7f794b400640 1 -- 172.21.15.151:0/3643638411 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7940067fc0 con 0x7f79541fc260 2024-09-16T15:39:36.366 INFO:teuthology.orchestra.run.smithi151.stdout:64424509447 2024-09-16T15:39:36.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.368+0000 7f795a800640 1 -- 172.21.15.151:0/3643638411 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f79340788d0 msgr2=0x7f793407ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:36.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.368+0000 7f795a800640 1 --2- 172.21.15.151:0/3643638411 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f79340788d0 0x7f793407ad90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7944005bc0 tx=0x7f7944005f90 comp rx=0 tx=0).stop 2024-09-16T15:39:36.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.368+0000 7f795a800640 1 -- 172.21.15.151:0/3643638411 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79541fc260 msgr2=0x7f79541fc6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:36.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.368+0000 7f795a800640 1 --2- 172.21.15.151:0/3643638411 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79541fc260 0x7f79541fc6c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7940000fb0 tx=0x7f7940003a80 comp rx=0 tx=0).stop 2024-09-16T15:39:36.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.368+0000 7f795a800640 1 -- 172.21.15.151:0/3643638411 shutdown_connections 2024-09-16T15:39:36.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.368+0000 7f795a800640 1 --2- 172.21.15.151:0/3643638411 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f79340788d0 0x7f793407ad90 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:36.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.368+0000 7f795a800640 1 --2- 172.21.15.151:0/3643638411 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f795414bfd0 0x7f795414c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:36.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.368+0000 7f795a800640 1 --2- 172.21.15.151:0/3643638411 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79541fc260 0x7f79541fc6c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:36.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.368+0000 7f795a800640 1 --2- 172.21.15.151:0/3643638411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79380051c0 0x7f795414eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:36.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.368+0000 7f795a800640 1 -- 172.21.15.151:0/3643638411 >> 172.21.15.151:0/3643638411 conn(0x7f795406cb70 msgr2=0x7f795414f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:36.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.369+0000 7f795a800640 1 -- 172.21.15.151:0/3643638411 shutdown_connections 2024-09-16T15:39:36.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.369+0000 7f795a800640 1 -- 172.21.15.151:0/3643638411 wait complete. 2024-09-16T15:39:36.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.373+0000 7f7c9fe00640 1 -- 172.21.15.151:0/2771051288 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f7cb40b1fd0 con 0x7f7c98003c10 2024-09-16T15:39:36.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.373+0000 7f7caaa00640 1 -- 172.21.15.151:0/2771051288 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7cb4156090 con 0x7f7c98003c10 2024-09-16T15:39:36.375 INFO:teuthology.orchestra.run.smithi151.stdout:64424509449 2024-09-16T15:39:36.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.377+0000 7f7c9fe00640 1 -- 172.21.15.151:0/2771051288 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7c94078950 msgr2=0x7f7c9407ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:36.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.377+0000 7f7c9fe00640 1 --2- 172.21.15.151:0/2771051288 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7c94078950 0x7f7c9407ae10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7cb41ea0c0 tx=0x7f7ca0005f90 comp rx=0 tx=0).stop 2024-09-16T15:39:36.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.377+0000 7f7c9fe00640 1 -- 172.21.15.151:0/2771051288 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c98003c10 msgr2=0x7f7cb415fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:36.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.377+0000 7f7c9fe00640 1 --2- 172.21.15.151:0/2771051288 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c98003c10 0x7f7cb415fec0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7c9800c210 tx=0x7f7ca4003ec0 comp rx=0 tx=0).stop 2024-09-16T15:39:36.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.377+0000 7f7c9fe00640 1 -- 172.21.15.151:0/2771051288 shutdown_connections 2024-09-16T15:39:36.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.377+0000 7f7c9fe00640 1 --2- 172.21.15.151:0/2771051288 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7c94078950 0x7f7c9407ae10 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:36.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.377+0000 7f7c9fe00640 1 --2- 172.21.15.151:0/2771051288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7cb4155810 0x7f7cb41581f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:36.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.377+0000 7f7c9fe00640 1 --2- 172.21.15.151:0/2771051288 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7cb41553d0 0x7f7cb40b8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:36.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.377+0000 7f7c9fe00640 1 --2- 172.21.15.151:0/2771051288 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c98003c10 0x7f7cb415fec0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:36.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.377+0000 7f7c9fe00640 1 -- 172.21.15.151:0/2771051288 >> 172.21.15.151:0/2771051288 conn(0x7f7cb40d3e00 msgr2=0x7f7cb40b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:36.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.377+0000 7f7c9fe00640 1 -- 172.21.15.151:0/2771051288 shutdown_connections 2024-09-16T15:39:36.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.378+0000 7f7c9fe00640 1 -- 172.21.15.151:0/2771051288 wait complete. 2024-09-16T15:39:36.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.378+0000 7f516dc00640 1 -- 172.21.15.151:0/1564435853 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f51680b97f0 con 0x7f51680d1610 2024-09-16T15:39:36.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.379+0000 7f515ea00640 1 -- 172.21.15.151:0/1564435853 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5158061cf0 con 0x7f51680d1610 2024-09-16T15:39:36.381 INFO:teuthology.orchestra.run.smithi151.stdout:64424509447 2024-09-16T15:39:36.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.380+0000 7f1725c00640 1 -- 172.21.15.151:0/2938170815 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f16f8005580 con 0x7f17180d68c0 2024-09-16T15:39:36.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.380+0000 7f1716a00640 1 -- 172.21.15.151:0/2938170815 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1710088530 con 0x7f17180d68c0 2024-09-16T15:39:36.382 INFO:teuthology.orchestra.run.smithi151.stdout:64424509447 2024-09-16T15:39:36.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.380+0000 7ffb6a800640 1 -- 172.21.15.151:0/249649158 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7ffb3c005850 con 0x7ffb640d09c0 2024-09-16T15:39:36.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.381+0000 7ffb5b400640 1 -- 172.21.15.151:0/249649158 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ffb50083c30 con 0x7ffb640d09c0 2024-09-16T15:39:36.382 INFO:teuthology.orchestra.run.smithi151.stdout:64424509448 2024-09-16T15:39:36.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.383+0000 7f516dc00640 1 -- 172.21.15.151:0/1564435853 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5148078920 msgr2=0x7f514807ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:36.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.383+0000 7f516dc00640 1 --2- 172.21.15.151:0/1564435853 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5148078920 0x7f514807ade0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5154034000 tx=0x7f5154004730 comp rx=0 tx=0).stop 2024-09-16T15:39:36.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.383+0000 7f516dc00640 1 -- 172.21.15.151:0/1564435853 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51680d1610 msgr2=0x7f51680ca0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:36.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.383+0000 7f516dc00640 1 --2- 172.21.15.151:0/1564435853 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51680d1610 0x7f51680ca0e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f515800c430 tx=0x7f515800c8f0 comp rx=0 tx=0).stop 2024-09-16T15:39:36.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.383+0000 7f516dc00640 1 -- 172.21.15.151:0/1564435853 shutdown_connections 2024-09-16T15:39:36.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.383+0000 7f516dc00640 1 --2- 172.21.15.151:0/1564435853 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5148078920 0x7f514807ade0 secure :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5154034000 tx=0x7f5154004730 comp rx=0 tx=0).stop 2024-09-16T15:39:36.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.383+0000 7f516dc00640 1 --2- 172.21.15.151:0/1564435853 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f516815e0d0 0x7f51680c8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:36.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.383+0000 7f516dc00640 1 --2- 172.21.15.151:0/1564435853 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f51680d2050 0x7f51680b6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:36.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.383+0000 7f516dc00640 1 --2- 172.21.15.151:0/1564435853 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51680d1610 0x7f51680ca0e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:36.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.383+0000 7f516dc00640 1 -- 172.21.15.151:0/1564435853 >> 172.21.15.151:0/1564435853 conn(0x7f51680d3fe0 msgr2=0x7f51680cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:36.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.384+0000 7f516dc00640 1 -- 172.21.15.151:0/1564435853 shutdown_connections 2024-09-16T15:39:36.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.384+0000 7f516dc00640 1 -- 172.21.15.151:0/1564435853 wait complete. 2024-09-16T15:39:36.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.384+0000 7f1725c00640 1 -- 172.21.15.151:0/2938170815 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1700078850 msgr2=0x7f170007ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:36.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.384+0000 7f1725c00640 1 --2- 172.21.15.151:0/2938170815 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1700078850 0x7f170007ad10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f17180e4450 tx=0x7f170c005f90 comp rx=0 tx=0).stop 2024-09-16T15:39:36.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.384+0000 7f1725c00640 1 -- 172.21.15.151:0/2938170815 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17180d68c0 msgr2=0x7f17180da560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:36.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.384+0000 7f1725c00640 1 --2- 172.21.15.151:0/2938170815 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17180d68c0 0x7f17180da560 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1710036040 tx=0x7f1710003820 comp rx=0 tx=0).stop 2024-09-16T15:39:36.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.384+0000 7f1725c00640 1 -- 172.21.15.151:0/2938170815 shutdown_connections 2024-09-16T15:39:36.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.384+0000 7f1725c00640 1 --2- 172.21.15.151:0/2938170815 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1700078850 0x7f170007ad10 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:36.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.384+0000 7f1725c00640 1 --2- 172.21.15.151:0/2938170815 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f17180e5950 0x7f17180d9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:36.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.384+0000 7f1725c00640 1 --2- 172.21.15.151:0/2938170815 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f17180e54f0 0x7f17180d9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:36.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.384+0000 7f1725c00640 1 --2- 172.21.15.151:0/2938170815 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17180d68c0 0x7f17180da560 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:36.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.384+0000 7f1725c00640 1 -- 172.21.15.151:0/2938170815 >> 172.21.15.151:0/2938170815 conn(0x7f1718055310 msgr2=0x7f1718051b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:36.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.384+0000 7f1725c00640 1 -- 172.21.15.151:0/2938170815 shutdown_connections 2024-09-16T15:39:36.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.384+0000 7f1725c00640 1 -- 172.21.15.151:0/2938170815 wait complete. 2024-09-16T15:39:36.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.385+0000 7ffb6a800640 1 -- 172.21.15.151:0/249649158 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ffb44078920 msgr2=0x7ffb4407ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:36.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.385+0000 7ffb6a800640 1 --2- 172.21.15.151:0/249649158 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ffb44078920 0x7ffb4407ade0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ffb540017c0 tx=0x7ffb54009040 comp rx=0 tx=0).stop 2024-09-16T15:39:36.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.385+0000 7ffb6a800640 1 -- 172.21.15.151:0/249649158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb640d09c0 msgr2=0x7ffb640cbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:36.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.385+0000 7ffb6a800640 1 --2- 172.21.15.151:0/249649158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb640d09c0 0x7ffb640cbaf0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ffb5002aac0 tx=0x7ffb50003c10 comp rx=0 tx=0).stop 2024-09-16T15:39:36.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.386+0000 7ffb6a800640 1 -- 172.21.15.151:0/249649158 shutdown_connections 2024-09-16T15:39:36.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.386+0000 7ffb6a800640 1 --2- 172.21.15.151:0/249649158 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ffb44078920 0x7ffb4407ade0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:36.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.386+0000 7ffb6a800640 1 --2- 172.21.15.151:0/249649158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb640d09c0 0x7ffb640cbaf0 secure :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ffb5002aac0 tx=0x7ffb50003c10 comp rx=0 tx=0).stop 2024-09-16T15:39:36.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.386+0000 7ffb6a800640 1 --2- 172.21.15.151:0/249649158 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb640d2050 0x7ffb6415e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:36.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.386+0000 7ffb6a800640 1 --2- 172.21.15.151:0/249649158 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb640cfa20 0x7ffb640a1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:36.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.386+0000 7ffb6a800640 1 -- 172.21.15.151:0/249649158 >> 172.21.15.151:0/249649158 conn(0x7ffb640d3fe0 msgr2=0x7ffb6415f610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:36.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.386+0000 7ffb6a800640 1 -- 172.21.15.151:0/249649158 shutdown_connections 2024-09-16T15:39:36.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.387+0000 7ffb6a800640 1 -- 172.21.15.151:0/249649158 wait complete. 2024-09-16T15:39:36.392 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.3 2024-09-16T15:39:36.392 DEBUG:teuthology.parallel:result is None 2024-09-16T15:39:36.406 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509449 for osd.6 2024-09-16T15:39:36.406 DEBUG:teuthology.parallel:result is None 2024-09-16T15:39:36.410 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.4 2024-09-16T15:39:36.410 DEBUG:teuthology.parallel:result is None 2024-09-16T15:39:36.412 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.2 2024-09-16T15:39:36.412 DEBUG:teuthology.parallel:result is None 2024-09-16T15:39:36.413 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.5 2024-09-16T15:39:36.413 DEBUG:teuthology.parallel:result is None 2024-09-16T15:39:36.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.416+0000 7f48bc200640 1 -- 172.21.15.151:0/3670060766 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f48ac0e6970 con 0x7f48ac03b930 2024-09-16T15:39:36.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.417+0000 7f48b1600640 1 -- 172.21.15.151:0/3670060766 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f48a80885e0 con 0x7f48ac03b930 2024-09-16T15:39:36.418 INFO:teuthology.orchestra.run.smithi151.stdout:60129542154 2024-09-16T15:39:36.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.419+0000 7f48bc200640 1 -- 172.21.15.151:0/3670060766 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f48940788d0 msgr2=0x7f489407ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:36.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.420+0000 7f48bc200640 1 --2- 172.21.15.151:0/3670060766 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f48940788d0 0x7f489407ad90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f48a00046b0 tx=0x7f48a0009040 comp rx=0 tx=0).stop 2024-09-16T15:39:36.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.420+0000 7f48bc200640 1 -- 172.21.15.151:0/3670060766 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48ac03b930 msgr2=0x7f48ac179260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:36.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.420+0000 7f48bc200640 1 --2- 172.21.15.151:0/3670060766 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48ac03b930 0x7f48ac179260 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f48ac03b360 tx=0x7f48a8032b90 comp rx=0 tx=0).stop 2024-09-16T15:39:36.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.420+0000 7f48bc200640 1 -- 172.21.15.151:0/3670060766 shutdown_connections 2024-09-16T15:39:36.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.420+0000 7f48bc200640 1 --2- 172.21.15.151:0/3670060766 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f48940788d0 0x7f489407ad90 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:36.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.420+0000 7f48bc200640 1 --2- 172.21.15.151:0/3670060766 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48ac0e2790 0x7f48ac04c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:36.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.420+0000 7f48bc200640 1 --2- 172.21.15.151:0/3670060766 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48ac03b930 0x7f48ac179260 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:36.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.420+0000 7f48bc200640 1 --2- 172.21.15.151:0/3670060766 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f48ac037f00 0x7f48ac0dbfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:36.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.420+0000 7f48bc200640 1 -- 172.21.15.151:0/3670060766 >> 172.21.15.151:0/3670060766 conn(0x7f48ac03a3c0 msgr2=0x7f48ac006570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:36.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.420+0000 7f48bc200640 1 -- 172.21.15.151:0/3670060766 shutdown_connections 2024-09-16T15:39:36.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.420+0000 7f48bc200640 1 -- 172.21.15.151:0/3670060766 wait complete. 2024-09-16T15:39:36.441 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542153 got 60129542154 for osd.7 2024-09-16T15:39:36.442 DEBUG:teuthology.parallel:result is None 2024-09-16T15:39:36.442 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T15:39:36.442 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:39:36.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.639+0000 7f22d0e00640 1 -- 172.21.15.151:0/2022889265 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f22cc0cd5f0 msgr2=0x7f22cc0cda80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:36.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.639+0000 7f22d0e00640 1 --2- 172.21.15.151:0/2022889265 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f22cc0cd5f0 0x7f22cc0cda80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f22cc158920 tx=0x7f22c402aa30 comp rx=0 tx=0).stop 2024-09-16T15:39:36.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.639+0000 7f22d0e00640 1 -- 172.21.15.151:0/2022889265 shutdown_connections 2024-09-16T15:39:36.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.639+0000 7f22d0e00640 1 --2- 172.21.15.151:0/2022889265 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f22cc0cd5f0 0x7f22cc0cda80 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:36.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.639+0000 7f22d0e00640 1 --2- 172.21.15.151:0/2022889265 >> v2:172.21.15.167:3300/0 conn(0x7f22cc0ad880 0x7f22cc0adcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:36.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.639+0000 7f22d0e00640 1 -- 172.21.15.151:0/2022889265 >> 172.21.15.151:0/2022889265 conn(0x7f22cc0cfa20 msgr2=0x7f22cc0cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:36.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.639+0000 7f22d0e00640 1 -- 172.21.15.151:0/2022889265 shutdown_connections 2024-09-16T15:39:36.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.639+0000 7f22d0e00640 1 -- 172.21.15.151:0/2022889265 wait complete. 2024-09-16T15:39:36.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.641+0000 7f22d0e00640 1 Processor -- start 2024-09-16T15:39:36.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.641+0000 7f22d0e00640 1 -- start start 2024-09-16T15:39:36.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.642+0000 7f22d0e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f22cc06fb90 0x7f22cc1f7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:36.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.642+0000 7f22d0e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f22cc0ad880 0x7f22cc1f8250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:36.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.642+0000 7f22d0e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22cc0cd5f0 0x7f22cc1f8790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:36.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.642+0000 7f22d0e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f22cc15edf0 con 0x7f22cc0ad880 2024-09-16T15:39:36.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.642+0000 7f22d0e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f22cc15ec70 con 0x7f22cc0cd5f0 2024-09-16T15:39:36.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.642+0000 7f22d0e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f22cc15ef70 con 0x7f22cc06fb90 2024-09-16T15:39:36.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.642+0000 7f22caa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f22cc0ad880 0x7f22cc1f8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:36.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.642+0000 7f22caa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f22cc0ad880 0x7f22cc1f8250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:57406/0 (socket says 172.21.15.151:57406) 2024-09-16T15:39:36.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.642+0000 7f22caa00640 1 -- 172.21.15.151:0/1242068003 learned_addr learned my addr 172.21.15.151:0/1242068003 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:36.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.642+0000 7f22cbe00640 1 --2- 172.21.15.151:0/1242068003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22cc0cd5f0 0x7f22cc1f8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:36.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.642+0000 7f22cb400640 1 --2- 172.21.15.151:0/1242068003 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f22cc06fb90 0x7f22cc1f7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:36.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.643+0000 7f22caa00640 1 -- 172.21.15.151:0/1242068003 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f22cc06fb90 msgr2=0x7f22cc1f7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:36.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.643+0000 7f22caa00640 1 --2- 172.21.15.151:0/1242068003 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f22cc06fb90 0x7f22cc1f7d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:36.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.643+0000 7f22caa00640 1 -- 172.21.15.151:0/1242068003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22cc0cd5f0 msgr2=0x7f22cc1f8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:36.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.643+0000 7f22caa00640 1 --2- 172.21.15.151:0/1242068003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22cc0cd5f0 0x7f22cc1f8790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:36.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.643+0000 7f22caa00640 1 -- 172.21.15.151:0/1242068003 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22cc15ac30 con 0x7f22cc0ad880 2024-09-16T15:39:36.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.643+0000 7f22cb400640 1 --2- 172.21.15.151:0/1242068003 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f22cc06fb90 0x7f22cc1f7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:39:36.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.643+0000 7f22caa00640 1 --2- 172.21.15.151:0/1242068003 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f22cc0ad880 0x7f22cc1f8250 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f22b800d770 tx=0x7f22b800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:36.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.643+0000 7f22c3400640 1 -- 172.21.15.151:0/1242068003 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22b8017070 con 0x7f22cc0ad880 2024-09-16T15:39:36.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.643+0000 7f22c3400640 1 -- 172.21.15.151:0/1242068003 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f22b8009d20 con 0x7f22cc0ad880 2024-09-16T15:39:36.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.643+0000 7f22d0e00640 1 -- 172.21.15.151:0/1242068003 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22cc15af10 con 0x7f22cc0ad880 2024-09-16T15:39:36.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.643+0000 7f22c3400640 1 -- 172.21.15.151:0/1242068003 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22b8005280 con 0x7f22cc0ad880 2024-09-16T15:39:36.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.643+0000 7f22d0e00640 1 -- 172.21.15.151:0/1242068003 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22cc15b4d0 con 0x7f22cc0ad880 2024-09-16T15:39:36.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.644+0000 7f22d0e00640 1 -- 172.21.15.151:0/1242068003 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22cc0adcd0 con 0x7f22cc0ad880 2024-09-16T15:39:36.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.645+0000 7f22c3400640 1 -- 172.21.15.151:0/1242068003 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f22b8005420 con 0x7f22cc0ad880 2024-09-16T15:39:36.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.646+0000 7f22c3400640 1 --2- 172.21.15.151:0/1242068003 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f22ac078850 0x7f22ac07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:36.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.646+0000 7f22c3400640 1 -- 172.21.15.151:0/1242068003 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f22b809aa40 con 0x7f22cc0ad880 2024-09-16T15:39:36.648 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.647+0000 7f22cb400640 1 --2- 172.21.15.151:0/1242068003 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f22ac078850 0x7f22ac07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:36.648 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.647+0000 7f22cb400640 1 --2- 172.21.15.151:0/1242068003 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f22ac078850 0x7f22ac07ad10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f22bc003eb0 tx=0x7f22bc009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:36.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.651+0000 7f22c3400640 1 -- 172.21.15.151:0/1242068003 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f22b8062a50 con 0x7f22cc0ad880 2024-09-16T15:39:36.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.834+0000 7f22d0e00640 1 -- 172.21.15.151:0/1242068003 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f22cc15bb90 con 0x7f22ac078850 2024-09-16T15:39:36.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.839+0000 7f22c3400640 1 -- 172.21.15.151:0/1242068003 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251822 (secure 0 0 0) 0x7f22bc0567c0 con 0x7f22ac078850 2024-09-16T15:39:36.843 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:39:36.844 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-09-16T15:39:36.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.842+0000 7f22d0e00640 1 -- 172.21.15.151:0/1242068003 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f22ac078850 msgr2=0x7f22ac07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:36.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.842+0000 7f22d0e00640 1 --2- 172.21.15.151:0/1242068003 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f22ac078850 0x7f22ac07ad10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f22bc003eb0 tx=0x7f22bc009040 comp rx=0 tx=0).stop 2024-09-16T15:39:36.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.842+0000 7f22d0e00640 1 -- 172.21.15.151:0/1242068003 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f22cc0ad880 msgr2=0x7f22cc1f8250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:36.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.842+0000 7f22d0e00640 1 --2- 172.21.15.151:0/1242068003 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f22cc0ad880 0x7f22cc1f8250 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f22b800d770 tx=0x7f22b800dc30 comp rx=0 tx=0).stop 2024-09-16T15:39:36.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.842+0000 7f22d0e00640 1 -- 172.21.15.151:0/1242068003 shutdown_connections 2024-09-16T15:39:36.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.842+0000 7f22d0e00640 1 --2- 172.21.15.151:0/1242068003 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f22ac078850 0x7f22ac07ad10 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:36.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.842+0000 7f22d0e00640 1 --2- 172.21.15.151:0/1242068003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22cc0cd5f0 0x7f22cc1f8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:36.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.842+0000 7f22d0e00640 1 --2- 172.21.15.151:0/1242068003 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f22cc0ad880 0x7f22cc1f8250 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:36.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.842+0000 7f22d0e00640 1 --2- 172.21.15.151:0/1242068003 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f22cc06fb90 0x7f22cc1f7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:36.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.843+0000 7f22d0e00640 1 -- 172.21.15.151:0/1242068003 >> 172.21.15.151:0/1242068003 conn(0x7f22cc0cfa20 msgr2=0x7f22cc1607f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:36.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.843+0000 7f22d0e00640 1 -- 172.21.15.151:0/1242068003 shutdown_connections 2024-09-16T15:39:36.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:36.843+0000 7f22d0e00640 1 -- 172.21.15.151:0/1242068003 wait complete. 2024-09-16T15:39:36.859 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2024-09-16T15:39:35.579882+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219184,"kb_used_data":2952,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755536,"statfs":{"total":773094113280,"available":772869668864,"internally_reserved":0,"allocated":3022848,"data_stored":1795828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12738,"internal_metadata":221236798},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003146"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.362052+0000","last_change":"2024-09-16T15:39:22.577878+0000","last_active":"2024-09-16T15:39:24.362052+0000","last_peered":"2024-09-16T15:39:24.362052+0000","last_clean":"2024-09-16T15:39:24.362052+0000","last_became_active":"2024-09-16T15:39:22.570201+0000","last_became_peered":"2024-09-16T15:39:22.570201+0000","last_unstale":"2024-09-16T15:39:24.362052+0000","last_undegraded":"2024-09-16T15:39:24.362052+0000","last_fullsized":"2024-09-16T15:39:24.362052+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:39:49.705970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.563074+0000","last_change":"2024-09-16T15:39:22.568365+0000","last_active":"2024-09-16T15:39:23.563074+0000","last_peered":"2024-09-16T15:39:23.563074+0000","last_clean":"2024-09-16T15:39:23.563074+0000","last_became_active":"2024-09-16T15:39:22.564695+0000","last_became_peered":"2024-09-16T15:39:22.564695+0000","last_unstale":"2024-09-16T15:39:23.563074+0000","last_undegraded":"2024-09-16T15:39:23.563074+0000","last_fullsized":"2024-09-16T15:39:23.563074+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:18:39.431491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.563192+0000","last_change":"2024-09-16T15:39:22.575183+0000","last_active":"2024-09-16T15:39:23.563192+0000","last_peered":"2024-09-16T15:39:23.563192+0000","last_clean":"2024-09-16T15:39:23.563192+0000","last_became_active":"2024-09-16T15:39:22.574916+0000","last_became_peered":"2024-09-16T15:39:22.574916+0000","last_unstale":"2024-09-16T15:39:23.563192+0000","last_undegraded":"2024-09-16T15:39:23.563192+0000","last_fullsized":"2024-09-16T15:39:23.563192+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:06:34.579134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.362360+0000","last_change":"2024-09-16T15:39:22.580108+0000","last_active":"2024-09-16T15:39:24.362360+0000","last_peered":"2024-09-16T15:39:24.362360+0000","last_clean":"2024-09-16T15:39:24.362360+0000","last_became_active":"2024-09-16T15:39:22.579859+0000","last_became_peered":"2024-09-16T15:39:22.579859+0000","last_unstale":"2024-09-16T15:39:24.362360+0000","last_undegraded":"2024-09-16T15:39:24.362360+0000","last_fullsized":"2024-09-16T15:39:24.362360+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:32:44.877620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361299+0000","last_change":"2024-09-16T15:39:22.566468+0000","last_active":"2024-09-16T15:39:24.361299+0000","last_peered":"2024-09-16T15:39:24.361299+0000","last_clean":"2024-09-16T15:39:24.361299+0000","last_became_active":"2024-09-16T15:39:22.559581+0000","last_became_peered":"2024-09-16T15:39:22.559581+0000","last_unstale":"2024-09-16T15:39:24.361299+0000","last_undegraded":"2024-09-16T15:39:24.361299+0000","last_fullsized":"2024-09-16T15:39:24.361299+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:35:04.935435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361543+0000","last_change":"2024-09-16T15:39:22.579964+0000","last_active":"2024-09-16T15:39:24.361543+0000","last_peered":"2024-09-16T15:39:24.361543+0000","last_clean":"2024-09-16T15:39:24.361543+0000","last_became_active":"2024-09-16T15:39:22.579714+0000","last_became_peered":"2024-09-16T15:39:22.579714+0000","last_unstale":"2024-09-16T15:39:24.361543+0000","last_undegraded":"2024-09-16T15:39:24.361543+0000","last_fullsized":"2024-09-16T15:39:24.361543+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:54:55.307273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360552+0000","last_change":"2024-09-16T15:39:22.578008+0000","last_active":"2024-09-16T15:39:24.360552+0000","last_peered":"2024-09-16T15:39:24.360552+0000","last_clean":"2024-09-16T15:39:24.360552+0000","last_became_active":"2024-09-16T15:39:22.577770+0000","last_became_peered":"2024-09-16T15:39:22.577770+0000","last_unstale":"2024-09-16T15:39:24.360552+0000","last_undegraded":"2024-09-16T15:39:24.360552+0000","last_fullsized":"2024-09-16T15:39:24.360552+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:03:05.655450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.562837+0000","last_change":"2024-09-16T15:39:22.565821+0000","last_active":"2024-09-16T15:39:23.562837+0000","last_peered":"2024-09-16T15:39:23.562837+0000","last_clean":"2024-09-16T15:39:23.562837+0000","last_became_active":"2024-09-16T15:39:22.564114+0000","last_became_peered":"2024-09-16T15:39:22.564114+0000","last_unstale":"2024-09-16T15:39:23.562837+0000","last_undegraded":"2024-09-16T15:39:23.562837+0000","last_fullsized":"2024-09-16T15:39:23.562837+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:03:28.558231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.561256+0000","last_change":"2024-09-16T15:39:22.557568+0000","last_active":"2024-09-16T15:39:23.561256+0000","last_peered":"2024-09-16T15:39:23.561256+0000","last_clean":"2024-09-16T15:39:23.561256+0000","last_became_active":"2024-09-16T15:39:22.556305+0000","last_became_peered":"2024-09-16T15:39:22.556305+0000","last_unstale":"2024-09-16T15:39:23.561256+0000","last_undegraded":"2024-09-16T15:39:23.561256+0000","last_fullsized":"2024-09-16T15:39:23.561256+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:19:39.018320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361933+0000","last_change":"2024-09-16T15:39:22.573944+0000","last_active":"2024-09-16T15:39:24.361933+0000","last_peered":"2024-09-16T15:39:24.361933+0000","last_clean":"2024-09-16T15:39:24.361933+0000","last_became_active":"2024-09-16T15:39:22.573688+0000","last_became_peered":"2024-09-16T15:39:22.573688+0000","last_unstale":"2024-09-16T15:39:24.361933+0000","last_undegraded":"2024-09-16T15:39:24.361933+0000","last_fullsized":"2024-09-16T15:39:24.361933+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:00:05.185794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360929+0000","last_change":"2024-09-16T15:39:22.580084+0000","last_active":"2024-09-16T15:39:24.360929+0000","last_peered":"2024-09-16T15:39:24.360929+0000","last_clean":"2024-09-16T15:39:24.360929+0000","last_became_active":"2024-09-16T15:39:22.571878+0000","last_became_peered":"2024-09-16T15:39:22.571878+0000","last_unstale":"2024-09-16T15:39:24.360929+0000","last_undegraded":"2024-09-16T15:39:24.360929+0000","last_fullsized":"2024-09-16T15:39:24.360929+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:46:31.911518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361633+0000","last_change":"2024-09-16T15:39:22.575436+0000","last_active":"2024-09-16T15:39:24.361633+0000","last_peered":"2024-09-16T15:39:24.361633+0000","last_clean":"2024-09-16T15:39:24.361633+0000","last_became_active":"2024-09-16T15:39:22.568528+0000","last_became_peered":"2024-09-16T15:39:22.568528+0000","last_unstale":"2024-09-16T15:39:24.361633+0000","last_undegraded":"2024-09-16T15:39:24.361633+0000","last_fullsized":"2024-09-16T15:39:24.361633+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:05:28.120874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360645+0000","last_change":"2024-09-16T15:39:22.566575+0000","last_active":"2024-09-16T15:39:24.360645+0000","last_peered":"2024-09-16T15:39:24.360645+0000","last_clean":"2024-09-16T15:39:24.360645+0000","last_became_active":"2024-09-16T15:39:22.566295+0000","last_became_peered":"2024-09-16T15:39:22.566295+0000","last_unstale":"2024-09-16T15:39:24.360645+0000","last_undegraded":"2024-09-16T15:39:24.360645+0000","last_fullsized":"2024-09-16T15:39:24.360645+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:10:23.402092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360395+0000","last_change":"2024-09-16T15:39:22.571476+0000","last_active":"2024-09-16T15:39:24.360395+0000","last_peered":"2024-09-16T15:39:24.360395+0000","last_clean":"2024-09-16T15:39:24.360395+0000","last_became_active":"2024-09-16T15:39:22.571216+0000","last_became_peered":"2024-09-16T15:39:22.571216+0000","last_unstale":"2024-09-16T15:39:24.360395+0000","last_undegraded":"2024-09-16T15:39:24.360395+0000","last_fullsized":"2024-09-16T15:39:24.360395+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:40:58.311998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.563103+0000","last_change":"2024-09-16T15:39:22.564630+0000","last_active":"2024-09-16T15:39:23.563103+0000","last_peered":"2024-09-16T15:39:23.563103+0000","last_clean":"2024-09-16T15:39:23.563103+0000","last_became_active":"2024-09-16T15:39:22.564326+0000","last_became_peered":"2024-09-16T15:39:22.564326+0000","last_unstale":"2024-09-16T15:39:23.563103+0000","last_undegraded":"2024-09-16T15:39:23.563103+0000","last_fullsized":"2024-09-16T15:39:23.563103+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:58:23.963120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360193+0000","last_change":"2024-09-16T15:39:22.574288+0000","last_active":"2024-09-16T15:39:24.360193+0000","last_peered":"2024-09-16T15:39:24.360193+0000","last_clean":"2024-09-16T15:39:24.360193+0000","last_became_active":"2024-09-16T15:39:22.573979+0000","last_became_peered":"2024-09-16T15:39:22.573979+0000","last_unstale":"2024-09-16T15:39:24.360193+0000","last_undegraded":"2024-09-16T15:39:24.360193+0000","last_fullsized":"2024-09-16T15:39:24.360193+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:10:15.420114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361109+0000","last_change":"2024-09-16T15:39:22.561784+0000","last_active":"2024-09-16T15:39:24.361109+0000","last_peered":"2024-09-16T15:39:24.361109+0000","last_clean":"2024-09-16T15:39:24.361109+0000","last_became_active":"2024-09-16T15:39:22.561487+0000","last_became_peered":"2024-09-16T15:39:22.561487+0000","last_unstale":"2024-09-16T15:39:24.361109+0000","last_undegraded":"2024-09-16T15:39:24.361109+0000","last_fullsized":"2024-09-16T15:39:24.361109+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:26:52.642854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.714434+0000","last_change":"2024-09-16T15:39:22.579035+0000","last_active":"2024-09-16T15:39:23.714434+0000","last_peered":"2024-09-16T15:39:23.714434+0000","last_clean":"2024-09-16T15:39:23.714434+0000","last_became_active":"2024-09-16T15:39:22.578734+0000","last_became_peered":"2024-09-16T15:39:22.578734+0000","last_unstale":"2024-09-16T15:39:23.714434+0000","last_undegraded":"2024-09-16T15:39:23.714434+0000","last_fullsized":"2024-09-16T15:39:23.714434+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:13:19.369864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361928+0000","last_change":"2024-09-16T15:39:22.577483+0000","last_active":"2024-09-16T15:39:24.361928+0000","last_peered":"2024-09-16T15:39:24.361928+0000","last_clean":"2024-09-16T15:39:24.361928+0000","last_became_active":"2024-09-16T15:39:22.577242+0000","last_became_peered":"2024-09-16T15:39:22.577242+0000","last_unstale":"2024-09-16T15:39:24.361928+0000","last_undegraded":"2024-09-16T15:39:24.361928+0000","last_fullsized":"2024-09-16T15:39:24.361928+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:29:43.879706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360912+0000","last_change":"2024-09-16T15:39:22.561587+0000","last_active":"2024-09-16T15:39:24.360912+0000","last_peered":"2024-09-16T15:39:24.360912+0000","last_clean":"2024-09-16T15:39:24.360912+0000","last_became_active":"2024-09-16T15:39:22.561213+0000","last_became_peered":"2024-09-16T15:39:22.561213+0000","last_unstale":"2024-09-16T15:39:24.360912+0000","last_undegraded":"2024-09-16T15:39:24.360912+0000","last_fullsized":"2024-09-16T15:39:24.360912+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:00:48.581056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.562380+0000","last_change":"2024-09-16T15:39:22.579126+0000","last_active":"2024-09-16T15:39:23.562380+0000","last_peered":"2024-09-16T15:39:23.562380+0000","last_clean":"2024-09-16T15:39:23.562380+0000","last_became_active":"2024-09-16T15:39:22.577915+0000","last_became_peered":"2024-09-16T15:39:22.577915+0000","last_unstale":"2024-09-16T15:39:23.562380+0000","last_undegraded":"2024-09-16T15:39:23.562380+0000","last_fullsized":"2024-09-16T15:39:23.562380+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:18:15.980220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.560783+0000","last_change":"2024-09-16T15:39:22.560525+0000","last_active":"2024-09-16T15:39:23.560783+0000","last_peered":"2024-09-16T15:39:23.560783+0000","last_clean":"2024-09-16T15:39:23.560783+0000","last_became_active":"2024-09-16T15:39:22.560279+0000","last_became_peered":"2024-09-16T15:39:22.560279+0000","last_unstale":"2024-09-16T15:39:23.560783+0000","last_undegraded":"2024-09-16T15:39:23.560783+0000","last_fullsized":"2024-09-16T15:39:23.560783+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:22:24.884519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.562668+0000","last_change":"2024-09-16T15:39:22.567598+0000","last_active":"2024-09-16T15:39:23.562668+0000","last_peered":"2024-09-16T15:39:23.562668+0000","last_clean":"2024-09-16T15:39:23.562668+0000","last_became_active":"2024-09-16T15:39:22.567356+0000","last_became_peered":"2024-09-16T15:39:22.567356+0000","last_unstale":"2024-09-16T15:39:23.562668+0000","last_undegraded":"2024-09-16T15:39:23.562668+0000","last_fullsized":"2024-09-16T15:39:23.562668+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:22:34.502460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360199+0000","last_change":"2024-09-16T15:39:22.558806+0000","last_active":"2024-09-16T15:39:24.360199+0000","last_peered":"2024-09-16T15:39:24.360199+0000","last_clean":"2024-09-16T15:39:24.360199+0000","last_became_active":"2024-09-16T15:39:22.558473+0000","last_became_peered":"2024-09-16T15:39:22.558473+0000","last_unstale":"2024-09-16T15:39:24.360199+0000","last_undegraded":"2024-09-16T15:39:24.360199+0000","last_fullsized":"2024-09-16T15:39:24.360199+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:05:28.120874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.563124+0000","last_change":"2024-09-16T15:39:22.564434+0000","last_active":"2024-09-16T15:39:23.563124+0000","last_peered":"2024-09-16T15:39:23.563124+0000","last_clean":"2024-09-16T15:39:23.563124+0000","last_became_active":"2024-09-16T15:39:22.563970+0000","last_became_peered":"2024-09-16T15:39:22.563970+0000","last_unstale":"2024-09-16T15:39:23.563124+0000","last_undegraded":"2024-09-16T15:39:23.563124+0000","last_fullsized":"2024-09-16T15:39:23.563124+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:43:27.368913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.362535+0000","last_change":"2024-09-16T15:39:22.584858+0000","last_active":"2024-09-16T15:39:24.362535+0000","last_peered":"2024-09-16T15:39:24.362535+0000","last_clean":"2024-09-16T15:39:24.362535+0000","last_became_active":"2024-09-16T15:39:22.584152+0000","last_became_peered":"2024-09-16T15:39:22.584152+0000","last_unstale":"2024-09-16T15:39:24.362535+0000","last_undegraded":"2024-09-16T15:39:24.362535+0000","last_fullsized":"2024-09-16T15:39:24.362535+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:08:07.935905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360764+0000","last_change":"2024-09-16T15:39:22.573867+0000","last_active":"2024-09-16T15:39:24.360764+0000","last_peered":"2024-09-16T15:39:24.360764+0000","last_clean":"2024-09-16T15:39:24.360764+0000","last_became_active":"2024-09-16T15:39:22.573567+0000","last_became_peered":"2024-09-16T15:39:22.573567+0000","last_unstale":"2024-09-16T15:39:24.360764+0000","last_undegraded":"2024-09-16T15:39:24.360764+0000","last_fullsized":"2024-09-16T15:39:24.360764+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:18:15.980220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360545+0000","last_change":"2024-09-16T15:39:22.574268+0000","last_active":"2024-09-16T15:39:24.360545+0000","last_peered":"2024-09-16T15:39:24.360545+0000","last_clean":"2024-09-16T15:39:24.360545+0000","last_became_active":"2024-09-16T15:39:22.572891+0000","last_became_peered":"2024-09-16T15:39:22.572891+0000","last_unstale":"2024-09-16T15:39:24.360545+0000","last_undegraded":"2024-09-16T15:39:24.360545+0000","last_fullsized":"2024-09-16T15:39:24.360545+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:29:11.296912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.562547+0000","last_change":"2024-09-16T15:39:22.563793+0000","last_active":"2024-09-16T15:39:23.562547+0000","last_peered":"2024-09-16T15:39:23.562547+0000","last_clean":"2024-09-16T15:39:23.562547+0000","last_became_active":"2024-09-16T15:39:22.563504+0000","last_became_peered":"2024-09-16T15:39:22.563504+0000","last_unstale":"2024-09-16T15:39:23.562547+0000","last_undegraded":"2024-09-16T15:39:23.562547+0000","last_fullsized":"2024-09-16T15:39:23.562547+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:56:07.665096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361245+0000","last_change":"2024-09-16T15:39:22.569041+0000","last_active":"2024-09-16T15:39:24.361245+0000","last_peered":"2024-09-16T15:39:24.361245+0000","last_clean":"2024-09-16T15:39:24.361245+0000","last_became_active":"2024-09-16T15:39:22.568799+0000","last_became_peered":"2024-09-16T15:39:22.568799+0000","last_unstale":"2024-09-16T15:39:24.361245+0000","last_undegraded":"2024-09-16T15:39:24.361245+0000","last_fullsized":"2024-09-16T15:39:24.361245+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:58:26.472440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.714354+0000","last_change":"2024-09-16T15:39:22.566051+0000","last_active":"2024-09-16T15:39:23.714354+0000","last_peered":"2024-09-16T15:39:23.714354+0000","last_clean":"2024-09-16T15:39:23.714354+0000","last_became_active":"2024-09-16T15:39:22.565764+0000","last_became_peered":"2024-09-16T15:39:22.565764+0000","last_unstale":"2024-09-16T15:39:23.714354+0000","last_undegraded":"2024-09-16T15:39:23.714354+0000","last_fullsized":"2024-09-16T15:39:23.714354+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:42:34.417720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361771+0000","last_change":"2024-09-16T15:39:22.574296+0000","last_active":"2024-09-16T15:39:24.361771+0000","last_peered":"2024-09-16T15:39:24.361771+0000","last_clean":"2024-09-16T15:39:24.361771+0000","last_became_active":"2024-09-16T15:39:22.574004+0000","last_became_peered":"2024-09-16T15:39:22.574004+0000","last_unstale":"2024-09-16T15:39:24.361771+0000","last_undegraded":"2024-09-16T15:39:24.361771+0000","last_fullsized":"2024-09-16T15:39:24.361771+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:36:56.913296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.560837+0000","last_change":"2024-09-16T15:39:22.559868+0000","last_active":"2024-09-16T15:39:23.560837+0000","last_peered":"2024-09-16T15:39:23.560837+0000","last_clean":"2024-09-16T15:39:23.560837+0000","last_became_active":"2024-09-16T15:39:22.559506+0000","last_became_peered":"2024-09-16T15:39:22.559506+0000","last_unstale":"2024-09-16T15:39:23.560837+0000","last_undegraded":"2024-09-16T15:39:23.560837+0000","last_fullsized":"2024-09-16T15:39:23.560837+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:33:53.332507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.562441+0000","last_change":"2024-09-16T15:39:21.562538+0000","last_active":"2024-09-16T15:39:23.562441+0000","last_peered":"2024-09-16T15:39:23.562441+0000","last_clean":"2024-09-16T15:39:23.562441+0000","last_became_active":"2024-09-16T15:39:21.559240+0000","last_became_peered":"2024-09-16T15:39:21.559240+0000","last_unstale":"2024-09-16T15:39:23.562441+0000","last_undegraded":"2024-09-16T15:39:23.562441+0000","last_fullsized":"2024-09-16T15:39:23.562441+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:43:58.109261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.562593+0000","last_change":"2024-09-16T15:39:22.582803+0000","last_active":"2024-09-16T15:39:23.562593+0000","last_peered":"2024-09-16T15:39:23.562593+0000","last_clean":"2024-09-16T15:39:23.562593+0000","last_became_active":"2024-09-16T15:39:22.582364+0000","last_became_peered":"2024-09-16T15:39:22.582364+0000","last_unstale":"2024-09-16T15:39:23.562593+0000","last_undegraded":"2024-09-16T15:39:23.562593+0000","last_fullsized":"2024-09-16T15:39:23.562593+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:40:27.419876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360960+0000","last_change":"2024-09-16T15:39:21.567667+0000","last_active":"2024-09-16T15:39:24.360960+0000","last_peered":"2024-09-16T15:39:24.360960+0000","last_clean":"2024-09-16T15:39:24.360960+0000","last_became_active":"2024-09-16T15:39:21.567399+0000","last_became_peered":"2024-09-16T15:39:21.567399+0000","last_unstale":"2024-09-16T15:39:24.360960+0000","last_undegraded":"2024-09-16T15:39:24.360960+0000","last_fullsized":"2024-09-16T15:39:24.360960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:22:23.876565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360132+0000","last_change":"2024-09-16T15:39:21.566105+0000","last_active":"2024-09-16T15:39:24.360132+0000","last_peered":"2024-09-16T15:39:24.360132+0000","last_clean":"2024-09-16T15:39:24.360132+0000","last_became_active":"2024-09-16T15:39:21.565719+0000","last_became_peered":"2024-09-16T15:39:21.565719+0000","last_unstale":"2024-09-16T15:39:24.360132+0000","last_undegraded":"2024-09-16T15:39:24.360132+0000","last_fullsized":"2024-09-16T15:39:24.360132+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:43:58.109261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.562255+0000","last_change":"2024-09-16T15:39:22.575843+0000","last_active":"2024-09-16T15:39:23.562255+0000","last_peered":"2024-09-16T15:39:23.562255+0000","last_clean":"2024-09-16T15:39:23.562255+0000","last_became_active":"2024-09-16T15:39:22.575543+0000","last_became_peered":"2024-09-16T15:39:22.575543+0000","last_unstale":"2024-09-16T15:39:23.562255+0000","last_undegraded":"2024-09-16T15:39:23.562255+0000","last_fullsized":"2024-09-16T15:39:23.562255+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:40:27.419876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360365+0000","last_change":"2024-09-16T15:39:21.562879+0000","last_active":"2024-09-16T15:39:24.360365+0000","last_peered":"2024-09-16T15:39:24.360365+0000","last_clean":"2024-09-16T15:39:24.360365+0000","last_became_active":"2024-09-16T15:39:21.562554+0000","last_became_peered":"2024-09-16T15:39:21.562554+0000","last_unstale":"2024-09-16T15:39:24.360365+0000","last_undegraded":"2024-09-16T15:39:24.360365+0000","last_fullsized":"2024-09-16T15:39:24.360365+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:33:52.324553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.562038+0000","last_change":"2024-09-16T15:39:22.571218+0000","last_active":"2024-09-16T15:39:23.562038+0000","last_peered":"2024-09-16T15:39:23.562038+0000","last_clean":"2024-09-16T15:39:23.562038+0000","last_became_active":"2024-09-16T15:39:22.570931+0000","last_became_peered":"2024-09-16T15:39:22.570931+0000","last_unstale":"2024-09-16T15:39:23.562038+0000","last_undegraded":"2024-09-16T15:39:23.562038+0000","last_fullsized":"2024-09-16T15:39:23.562038+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:57:08.823794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.561161+0000","last_change":"2024-09-16T15:39:21.558263+0000","last_active":"2024-09-16T15:39:23.561161+0000","last_peered":"2024-09-16T15:39:23.561161+0000","last_clean":"2024-09-16T15:39:23.561161+0000","last_became_active":"2024-09-16T15:39:21.557615+0000","last_became_peered":"2024-09-16T15:39:21.557615+0000","last_unstale":"2024-09-16T15:39:23.561161+0000","last_undegraded":"2024-09-16T15:39:23.561161+0000","last_fullsized":"2024-09-16T15:39:23.561161+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:10:40.788797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.362565+0000","last_change":"2024-09-16T15:39:22.586182+0000","last_active":"2024-09-16T15:39:24.362565+0000","last_peered":"2024-09-16T15:39:24.362565+0000","last_clean":"2024-09-16T15:39:24.362565+0000","last_became_active":"2024-09-16T15:39:22.584486+0000","last_became_peered":"2024-09-16T15:39:22.584486+0000","last_unstale":"2024-09-16T15:39:24.362565+0000","last_undegraded":"2024-09-16T15:39:24.362565+0000","last_fullsized":"2024-09-16T15:39:24.362565+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:25:52.095325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361757+0000","last_change":"2024-09-16T15:39:21.558934+0000","last_active":"2024-09-16T15:39:24.361757+0000","last_peered":"2024-09-16T15:39:24.361757+0000","last_clean":"2024-09-16T15:39:24.361757+0000","last_became_active":"2024-09-16T15:39:21.557612+0000","last_became_peered":"2024-09-16T15:39:21.557612+0000","last_unstale":"2024-09-16T15:39:24.361757+0000","last_undegraded":"2024-09-16T15:39:24.361757+0000","last_fullsized":"2024-09-16T15:39:24.361757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:21:44.910826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.563494+0000","last_change":"2024-09-16T15:39:22.563449+0000","last_active":"2024-09-16T15:39:23.563494+0000","last_peered":"2024-09-16T15:39:23.563494+0000","last_clean":"2024-09-16T15:39:23.563494+0000","last_became_active":"2024-09-16T15:39:22.558597+0000","last_became_peered":"2024-09-16T15:39:22.558597+0000","last_unstale":"2024-09-16T15:39:23.563494+0000","last_undegraded":"2024-09-16T15:39:23.563494+0000","last_fullsized":"2024-09-16T15:39:23.563494+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:08:47.835110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.714227+0000","last_change":"2024-09-16T15:39:21.561906+0000","last_active":"2024-09-16T15:39:23.714227+0000","last_peered":"2024-09-16T15:39:23.714227+0000","last_clean":"2024-09-16T15:39:23.714227+0000","last_became_active":"2024-09-16T15:39:21.561566+0000","last_became_peered":"2024-09-16T15:39:21.561566+0000","last_unstale":"2024-09-16T15:39:23.714227+0000","last_undegraded":"2024-09-16T15:39:23.714227+0000","last_fullsized":"2024-09-16T15:39:23.714227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:12:53.853659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360840+0000","last_change":"2024-09-16T15:39:22.581196+0000","last_active":"2024-09-16T15:39:24.360840+0000","last_peered":"2024-09-16T15:39:24.360840+0000","last_clean":"2024-09-16T15:39:24.360840+0000","last_became_active":"2024-09-16T15:39:22.579631+0000","last_became_peered":"2024-09-16T15:39:22.579631+0000","last_unstale":"2024-09-16T15:39:24.360840+0000","last_undegraded":"2024-09-16T15:39:24.360840+0000","last_fullsized":"2024-09-16T15:39:24.360840+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:15:37.944346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.362202+0000","last_change":"2024-09-16T15:39:21.560090+0000","last_active":"2024-09-16T15:39:24.362202+0000","last_peered":"2024-09-16T15:39:24.362202+0000","last_clean":"2024-09-16T15:39:24.362202+0000","last_became_active":"2024-09-16T15:39:21.558972+0000","last_became_peered":"2024-09-16T15:39:21.558972+0000","last_unstale":"2024-09-16T15:39:24.362202+0000","last_undegraded":"2024-09-16T15:39:24.362202+0000","last_fullsized":"2024-09-16T15:39:24.362202+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:12:19.666037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361423+0000","last_change":"2024-09-16T15:39:22.560439+0000","last_active":"2024-09-16T15:39:24.361423+0000","last_peered":"2024-09-16T15:39:24.361423+0000","last_clean":"2024-09-16T15:39:24.361423+0000","last_became_active":"2024-09-16T15:39:22.560172+0000","last_became_peered":"2024-09-16T15:39:22.560172+0000","last_unstale":"2024-09-16T15:39:24.361423+0000","last_undegraded":"2024-09-16T15:39:24.361423+0000","last_fullsized":"2024-09-16T15:39:24.361423+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:01:25.170048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.714255+0000","last_change":"2024-09-16T15:39:21.567091+0000","last_active":"2024-09-16T15:39:23.714255+0000","last_peered":"2024-09-16T15:39:23.714255+0000","last_clean":"2024-09-16T15:39:23.714255+0000","last_became_active":"2024-09-16T15:39:21.566668+0000","last_became_peered":"2024-09-16T15:39:21.566668+0000","last_unstale":"2024-09-16T15:39:23.714255+0000","last_undegraded":"2024-09-16T15:39:23.714255+0000","last_fullsized":"2024-09-16T15:39:23.714255+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:08:06.242307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360373+0000","last_change":"2024-09-16T15:39:22.569735+0000","last_active":"2024-09-16T15:39:24.360373+0000","last_peered":"2024-09-16T15:39:24.360373+0000","last_clean":"2024-09-16T15:39:24.360373+0000","last_became_active":"2024-09-16T15:39:22.569217+0000","last_became_peered":"2024-09-16T15:39:22.569217+0000","last_unstale":"2024-09-16T15:39:24.360373+0000","last_undegraded":"2024-09-16T15:39:24.360373+0000","last_fullsized":"2024-09-16T15:39:24.360373+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:34:12.122166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361075+0000","last_change":"2024-09-16T15:39:21.560934+0000","last_active":"2024-09-16T15:39:24.361075+0000","last_peered":"2024-09-16T15:39:24.361075+0000","last_clean":"2024-09-16T15:39:24.361075+0000","last_became_active":"2024-09-16T15:39:21.560690+0000","last_became_peered":"2024-09-16T15:39:21.560690+0000","last_unstale":"2024-09-16T15:39:24.361075+0000","last_undegraded":"2024-09-16T15:39:24.361075+0000","last_fullsized":"2024-09-16T15:39:24.361075+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:10:22.394137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360730+0000","last_change":"2024-09-16T15:39:22.569919+0000","last_active":"2024-09-16T15:39:24.360730+0000","last_peered":"2024-09-16T15:39:24.360730+0000","last_clean":"2024-09-16T15:39:24.360730+0000","last_became_active":"2024-09-16T15:39:22.569624+0000","last_became_peered":"2024-09-16T15:39:22.569624+0000","last_unstale":"2024-09-16T15:39:24.360730+0000","last_undegraded":"2024-09-16T15:39:24.360730+0000","last_fullsized":"2024-09-16T15:39:24.360730+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:42:11.842370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360437+0000","last_change":"2024-09-16T15:39:21.565143+0000","last_active":"2024-09-16T15:39:24.360437+0000","last_peered":"2024-09-16T15:39:24.360437+0000","last_clean":"2024-09-16T15:39:24.360437+0000","last_became_active":"2024-09-16T15:39:21.564887+0000","last_became_peered":"2024-09-16T15:39:21.564887+0000","last_unstale":"2024-09-16T15:39:24.360437+0000","last_undegraded":"2024-09-16T15:39:24.360437+0000","last_fullsized":"2024-09-16T15:39:24.360437+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:54:54.299318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.560542+0000","last_change":"2024-09-16T15:39:22.560025+0000","last_active":"2024-09-16T15:39:23.560542+0000","last_peered":"2024-09-16T15:39:23.560542+0000","last_clean":"2024-09-16T15:39:23.560542+0000","last_became_active":"2024-09-16T15:39:22.559563+0000","last_became_peered":"2024-09-16T15:39:22.559563+0000","last_unstale":"2024-09-16T15:39:23.560542+0000","last_undegraded":"2024-09-16T15:39:23.560542+0000","last_fullsized":"2024-09-16T15:39:23.560542+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:17:35.609828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.562702+0000","last_change":"2024-09-16T15:39:21.556186+0000","last_active":"2024-09-16T15:39:23.562702+0000","last_peered":"2024-09-16T15:39:23.562702+0000","last_clean":"2024-09-16T15:39:23.562702+0000","last_became_active":"2024-09-16T15:39:21.555847+0000","last_became_peered":"2024-09-16T15:39:21.555847+0000","last_unstale":"2024-09-16T15:39:23.562702+0000","last_undegraded":"2024-09-16T15:39:23.562702+0000","last_fullsized":"2024-09-16T15:39:23.562702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:43:58.109261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360215+0000","last_change":"2024-09-16T15:39:22.581265+0000","last_active":"2024-09-16T15:39:24.360215+0000","last_peered":"2024-09-16T15:39:24.360215+0000","last_clean":"2024-09-16T15:39:24.360215+0000","last_became_active":"2024-09-16T15:39:22.580808+0000","last_became_peered":"2024-09-16T15:39:22.580808+0000","last_unstale":"2024-09-16T15:39:24.360215+0000","last_undegraded":"2024-09-16T15:39:24.360215+0000","last_fullsized":"2024-09-16T15:39:24.360215+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:40:04.044401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.362115+0000","last_change":"2024-09-16T15:39:21.565669+0000","last_active":"2024-09-16T15:39:24.362115+0000","last_peered":"2024-09-16T15:39:24.362115+0000","last_clean":"2024-09-16T15:39:24.362115+0000","last_became_active":"2024-09-16T15:39:21.564058+0000","last_became_peered":"2024-09-16T15:39:21.564058+0000","last_unstale":"2024-09-16T15:39:24.362115+0000","last_undegraded":"2024-09-16T15:39:24.362115+0000","last_fullsized":"2024-09-16T15:39:24.362115+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:36:55.905341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.362680+0000","last_change":"2024-09-16T15:39:22.586068+0000","last_active":"2024-09-16T15:39:24.362680+0000","last_peered":"2024-09-16T15:39:24.362680+0000","last_clean":"2024-09-16T15:39:24.362680+0000","last_became_active":"2024-09-16T15:39:22.585745+0000","last_became_peered":"2024-09-16T15:39:22.585745+0000","last_unstale":"2024-09-16T15:39:24.362680+0000","last_undegraded":"2024-09-16T15:39:24.362680+0000","last_fullsized":"2024-09-16T15:39:24.362680+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:46:44.196294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.562702+0000","last_change":"2024-09-16T15:39:21.564692+0000","last_active":"2024-09-16T15:39:23.562702+0000","last_peered":"2024-09-16T15:39:23.562702+0000","last_clean":"2024-09-16T15:39:23.562702+0000","last_became_active":"2024-09-16T15:39:21.564414+0000","last_became_peered":"2024-09-16T15:39:21.564414+0000","last_unstale":"2024-09-16T15:39:23.562702+0000","last_undegraded":"2024-09-16T15:39:23.562702+0000","last_fullsized":"2024-09-16T15:39:23.562702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:46:18.513631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.362100+0000","last_change":"2024-09-16T15:39:22.560891+0000","last_active":"2024-09-16T15:39:24.362100+0000","last_peered":"2024-09-16T15:39:24.362100+0000","last_clean":"2024-09-16T15:39:24.362100+0000","last_became_active":"2024-09-16T15:39:22.560598+0000","last_became_peered":"2024-09-16T15:39:22.560598+0000","last_unstale":"2024-09-16T15:39:24.362100+0000","last_undegraded":"2024-09-16T15:39:24.362100+0000","last_fullsized":"2024-09-16T15:39:24.362100+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:57:51.336962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.563158+0000","last_change":"2024-09-16T15:39:21.559845+0000","last_active":"2024-09-16T15:39:23.563158+0000","last_peered":"2024-09-16T15:39:23.563158+0000","last_clean":"2024-09-16T15:39:23.563158+0000","last_became_active":"2024-09-16T15:39:21.559089+0000","last_became_peered":"2024-09-16T15:39:21.559089+0000","last_unstale":"2024-09-16T15:39:23.563158+0000","last_undegraded":"2024-09-16T15:39:23.563158+0000","last_fullsized":"2024-09-16T15:39:23.563158+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:40:57.508356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361096+0000","last_change":"2024-09-16T15:39:22.575906+0000","last_active":"2024-09-16T15:39:24.361096+0000","last_peered":"2024-09-16T15:39:24.361096+0000","last_clean":"2024-09-16T15:39:24.361096+0000","last_became_active":"2024-09-16T15:39:22.574839+0000","last_became_peered":"2024-09-16T15:39:22.574839+0000","last_unstale":"2024-09-16T15:39:24.361096+0000","last_undegraded":"2024-09-16T15:39:24.361096+0000","last_fullsized":"2024-09-16T15:39:24.361096+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:13:11.232766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360712+0000","last_change":"2024-09-16T15:39:21.565376+0000","last_active":"2024-09-16T15:39:24.360712+0000","last_peered":"2024-09-16T15:39:24.360712+0000","last_clean":"2024-09-16T15:39:24.360712+0000","last_became_active":"2024-09-16T15:39:21.565007+0000","last_became_peered":"2024-09-16T15:39:21.565007+0000","last_unstale":"2024-09-16T15:39:24.360712+0000","last_undegraded":"2024-09-16T15:39:24.360712+0000","last_fullsized":"2024-09-16T15:39:24.360712+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:39:19.838119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361711+0000","last_change":"2024-09-16T15:39:22.565594+0000","last_active":"2024-09-16T15:39:24.361711+0000","last_peered":"2024-09-16T15:39:24.361711+0000","last_clean":"2024-09-16T15:39:24.361711+0000","last_became_active":"2024-09-16T15:39:22.565255+0000","last_became_peered":"2024-09-16T15:39:22.565255+0000","last_unstale":"2024-09-16T15:39:24.361711+0000","last_undegraded":"2024-09-16T15:39:24.361711+0000","last_fullsized":"2024-09-16T15:39:24.361711+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:32.663886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.560438+0000","last_change":"2024-09-16T15:39:12.698813+0000","last_active":"2024-09-16T15:39:23.560438+0000","last_peered":"2024-09-16T15:39:23.560438+0000","last_clean":"2024-09-16T15:39:23.560438+0000","last_became_active":"2024-09-16T15:39:12.697695+0000","last_became_peered":"2024-09-16T15:39:12.697695+0000","last_unstale":"2024-09-16T15:39:23.560438+0000","last_undegraded":"2024-09-16T15:39:23.560438+0000","last_fullsized":"2024-09-16T15:39:23.560438+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_clean_scrub_stamp":"2024-09-16T15:39:11.684967+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:02.061081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360791+0000","last_change":"2024-09-16T15:39:21.565360+0000","last_active":"2024-09-16T15:39:24.360791+0000","last_peered":"2024-09-16T15:39:24.360791+0000","last_clean":"2024-09-16T15:39:24.360791+0000","last_became_active":"2024-09-16T15:39:21.565114+0000","last_became_peered":"2024-09-16T15:39:21.565114+0000","last_unstale":"2024-09-16T15:39:24.360791+0000","last_undegraded":"2024-09-16T15:39:24.360791+0000","last_fullsized":"2024-09-16T15:39:24.360791+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:40:57.304044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.562889+0000","last_change":"2024-09-16T15:39:22.579056+0000","last_active":"2024-09-16T15:39:23.562889+0000","last_peered":"2024-09-16T15:39:23.562889+0000","last_clean":"2024-09-16T15:39:23.562889+0000","last_became_active":"2024-09-16T15:39:22.576443+0000","last_became_peered":"2024-09-16T15:39:22.576443+0000","last_unstale":"2024-09-16T15:39:23.562889+0000","last_undegraded":"2024-09-16T15:39:23.562889+0000","last_fullsized":"2024-09-16T15:39:23.562889+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:56:07.665096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360279+0000","last_change":"2024-09-16T15:39:21.568792+0000","last_active":"2024-09-16T15:39:24.360279+0000","last_peered":"2024-09-16T15:39:24.360279+0000","last_clean":"2024-09-16T15:39:24.360279+0000","last_became_active":"2024-09-16T15:39:21.568534+0000","last_became_peered":"2024-09-16T15:39:21.568534+0000","last_unstale":"2024-09-16T15:39:24.360279+0000","last_undegraded":"2024-09-16T15:39:24.360279+0000","last_fullsized":"2024-09-16T15:39:24.360279+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:16:01.495612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360315+0000","last_change":"2024-09-16T15:39:22.572844+0000","last_active":"2024-09-16T15:39:24.360315+0000","last_peered":"2024-09-16T15:39:24.360315+0000","last_clean":"2024-09-16T15:39:24.360315+0000","last_became_active":"2024-09-16T15:39:22.570080+0000","last_became_peered":"2024-09-16T15:39:22.570080+0000","last_unstale":"2024-09-16T15:39:24.360315+0000","last_undegraded":"2024-09-16T15:39:24.360315+0000","last_fullsized":"2024-09-16T15:39:24.360315+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:19:39.018320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.714355+0000","last_change":"2024-09-16T15:39:21.563752+0000","last_active":"2024-09-16T15:39:23.714355+0000","last_peered":"2024-09-16T15:39:23.714355+0000","last_clean":"2024-09-16T15:39:23.714355+0000","last_became_active":"2024-09-16T15:39:21.563430+0000","last_became_peered":"2024-09-16T15:39:21.563430+0000","last_unstale":"2024-09-16T15:39:23.714355+0000","last_undegraded":"2024-09-16T15:39:23.714355+0000","last_fullsized":"2024-09-16T15:39:23.714355+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:44:53.582663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.562026+0000","last_change":"2024-09-16T15:39:22.560753+0000","last_active":"2024-09-16T15:39:23.562026+0000","last_peered":"2024-09-16T15:39:23.562026+0000","last_clean":"2024-09-16T15:39:23.562026+0000","last_became_active":"2024-09-16T15:39:22.560416+0000","last_became_peered":"2024-09-16T15:39:22.560416+0000","last_unstale":"2024-09-16T15:39:23.562026+0000","last_undegraded":"2024-09-16T15:39:23.562026+0000","last_fullsized":"2024-09-16T15:39:23.562026+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:30:53.438955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361167+0000","last_change":"2024-09-16T15:39:21.568766+0000","last_active":"2024-09-16T15:39:24.361167+0000","last_peered":"2024-09-16T15:39:24.361167+0000","last_clean":"2024-09-16T15:39:24.361167+0000","last_became_active":"2024-09-16T15:39:21.568515+0000","last_became_peered":"2024-09-16T15:39:21.568515+0000","last_unstale":"2024-09-16T15:39:24.361167+0000","last_undegraded":"2024-09-16T15:39:24.361167+0000","last_fullsized":"2024-09-16T15:39:24.361167+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:29:10.288957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361313+0000","last_change":"2024-09-16T15:39:22.581762+0000","last_active":"2024-09-16T15:39:24.361313+0000","last_peered":"2024-09-16T15:39:24.361313+0000","last_clean":"2024-09-16T15:39:24.361313+0000","last_became_active":"2024-09-16T15:39:22.572639+0000","last_became_peered":"2024-09-16T15:39:22.572639+0000","last_unstale":"2024-09-16T15:39:24.361313+0000","last_undegraded":"2024-09-16T15:39:24.361313+0000","last_fullsized":"2024-09-16T15:39:24.361313+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:06:34.579134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.562849+0000","last_change":"2024-09-16T15:39:21.561768+0000","last_active":"2024-09-16T15:39:23.562849+0000","last_peered":"2024-09-16T15:39:23.562849+0000","last_clean":"2024-09-16T15:39:23.562849+0000","last_became_active":"2024-09-16T15:39:21.561188+0000","last_became_peered":"2024-09-16T15:39:21.561188+0000","last_unstale":"2024-09-16T15:39:23.562849+0000","last_undegraded":"2024-09-16T15:39:23.562849+0000","last_fullsized":"2024-09-16T15:39:23.562849+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:39:19.838119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.362299+0000","last_change":"2024-09-16T15:39:22.578044+0000","last_active":"2024-09-16T15:39:24.362299+0000","last_peered":"2024-09-16T15:39:24.362299+0000","last_clean":"2024-09-16T15:39:24.362299+0000","last_became_active":"2024-09-16T15:39:22.577752+0000","last_became_peered":"2024-09-16T15:39:22.577752+0000","last_unstale":"2024-09-16T15:39:24.362299+0000","last_undegraded":"2024-09-16T15:39:24.362299+0000","last_fullsized":"2024-09-16T15:39:24.362299+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:36:16.988647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361983+0000","last_change":"2024-09-16T15:39:21.560508+0000","last_active":"2024-09-16T15:39:24.361983+0000","last_peered":"2024-09-16T15:39:24.361983+0000","last_clean":"2024-09-16T15:39:24.361983+0000","last_became_active":"2024-09-16T15:39:21.560082+0000","last_became_peered":"2024-09-16T15:39:21.560082+0000","last_unstale":"2024-09-16T15:39:24.361983+0000","last_undegraded":"2024-09-16T15:39:24.361983+0000","last_fullsized":"2024-09-16T15:39:24.361983+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:17:34.601873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360646+0000","last_change":"2024-09-16T15:39:22.581816+0000","last_active":"2024-09-16T15:39:24.360646+0000","last_peered":"2024-09-16T15:39:24.360646+0000","last_clean":"2024-09-16T15:39:24.360646+0000","last_became_active":"2024-09-16T15:39:22.581523+0000","last_became_peered":"2024-09-16T15:39:22.581523+0000","last_unstale":"2024-09-16T15:39:24.360646+0000","last_undegraded":"2024-09-16T15:39:24.360646+0000","last_fullsized":"2024-09-16T15:39:24.360646+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:57:08.823794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.562963+0000","last_change":"2024-09-16T15:39:21.564493+0000","last_active":"2024-09-16T15:39:23.562963+0000","last_peered":"2024-09-16T15:39:23.562963+0000","last_clean":"2024-09-16T15:39:23.562963+0000","last_became_active":"2024-09-16T15:39:21.564207+0000","last_became_peered":"2024-09-16T15:39:21.564207+0000","last_unstale":"2024-09-16T15:39:23.562963+0000","last_undegraded":"2024-09-16T15:39:23.562963+0000","last_fullsized":"2024-09-16T15:39:23.562963+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:17:34.601873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.562454+0000","last_change":"2024-09-16T15:39:22.577107+0000","last_active":"2024-09-16T15:39:23.562454+0000","last_peered":"2024-09-16T15:39:23.562454+0000","last_clean":"2024-09-16T15:39:23.562454+0000","last_became_active":"2024-09-16T15:39:22.576840+0000","last_became_peered":"2024-09-16T15:39:22.576840+0000","last_unstale":"2024-09-16T15:39:23.562454+0000","last_undegraded":"2024-09-16T15:39:23.562454+0000","last_fullsized":"2024-09-16T15:39:23.562454+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:30:53.438955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361083+0000","last_change":"2024-09-16T15:39:21.570398+0000","last_active":"2024-09-16T15:39:24.361083+0000","last_peered":"2024-09-16T15:39:24.361083+0000","last_clean":"2024-09-16T15:39:24.361083+0000","last_became_active":"2024-09-16T15:39:21.570020+0000","last_became_peered":"2024-09-16T15:39:21.570020+0000","last_unstale":"2024-09-16T15:39:24.361083+0000","last_undegraded":"2024-09-16T15:39:24.361083+0000","last_fullsized":"2024-09-16T15:39:24.361083+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:19:38.010365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.563030+0000","last_change":"2024-09-16T15:39:22.572370+0000","last_active":"2024-09-16T15:39:23.563030+0000","last_peered":"2024-09-16T15:39:23.563030+0000","last_clean":"2024-09-16T15:39:23.563030+0000","last_became_active":"2024-09-16T15:39:22.572114+0000","last_became_peered":"2024-09-16T15:39:22.572114+0000","last_unstale":"2024-09-16T15:39:23.563030+0000","last_undegraded":"2024-09-16T15:39:23.563030+0000","last_fullsized":"2024-09-16T15:39:23.563030+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:22:34.502460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360218+0000","last_change":"2024-09-16T15:39:21.561676+0000","last_active":"2024-09-16T15:39:24.360218+0000","last_peered":"2024-09-16T15:39:24.360218+0000","last_clean":"2024-09-16T15:39:24.360218+0000","last_became_active":"2024-09-16T15:39:21.560798+0000","last_became_peered":"2024-09-16T15:39:21.560798+0000","last_unstale":"2024-09-16T15:39:24.360218+0000","last_undegraded":"2024-09-16T15:39:24.360218+0000","last_fullsized":"2024-09-16T15:39:24.360218+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:12:34.622210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360594+0000","last_change":"2024-09-16T15:39:22.567438+0000","last_active":"2024-09-16T15:39:24.360594+0000","last_peered":"2024-09-16T15:39:24.360594+0000","last_clean":"2024-09-16T15:39:24.360594+0000","last_became_active":"2024-09-16T15:39:22.567191+0000","last_became_peered":"2024-09-16T15:39:22.567191+0000","last_unstale":"2024-09-16T15:39:24.360594+0000","last_undegraded":"2024-09-16T15:39:24.360594+0000","last_fullsized":"2024-09-16T15:39:24.360594+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:42:25.111846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.561951+0000","last_change":"2024-09-16T15:39:21.566726+0000","last_active":"2024-09-16T15:39:23.561951+0000","last_peered":"2024-09-16T15:39:23.561951+0000","last_clean":"2024-09-16T15:39:23.561951+0000","last_became_active":"2024-09-16T15:39:21.566406+0000","last_became_peered":"2024-09-16T15:39:21.566406+0000","last_unstale":"2024-09-16T15:39:23.561951+0000","last_undegraded":"2024-09-16T15:39:23.561951+0000","last_fullsized":"2024-09-16T15:39:23.561951+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:40:57.508356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.561742+0000","last_change":"2024-09-16T15:39:22.575174+0000","last_active":"2024-09-16T15:39:23.561742+0000","last_peered":"2024-09-16T15:39:23.561742+0000","last_clean":"2024-09-16T15:39:23.561742+0000","last_became_active":"2024-09-16T15:39:22.574865+0000","last_became_peered":"2024-09-16T15:39:22.574865+0000","last_unstale":"2024-09-16T15:39:23.561742+0000","last_undegraded":"2024-09-16T15:39:23.561742+0000","last_fullsized":"2024-09-16T15:39:23.561742+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:10:23.402092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360483+0000","last_change":"2024-09-16T15:39:21.561384+0000","last_active":"2024-09-16T15:39:24.360483+0000","last_peered":"2024-09-16T15:39:24.360483+0000","last_clean":"2024-09-16T15:39:24.360483+0000","last_became_active":"2024-09-16T15:39:21.561078+0000","last_became_peered":"2024-09-16T15:39:21.561078+0000","last_unstale":"2024-09-16T15:39:24.360483+0000","last_undegraded":"2024-09-16T15:39:24.360483+0000","last_fullsized":"2024-09-16T15:39:24.360483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:10:40.788797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.714500+0000","last_change":"2024-09-16T15:39:22.576232+0000","last_active":"2024-09-16T15:39:23.714500+0000","last_peered":"2024-09-16T15:39:23.714500+0000","last_clean":"2024-09-16T15:39:23.714500+0000","last_became_active":"2024-09-16T15:39:22.575962+0000","last_became_peered":"2024-09-16T15:39:22.575962+0000","last_unstale":"2024-09-16T15:39:23.714500+0000","last_undegraded":"2024-09-16T15:39:23.714500+0000","last_fullsized":"2024-09-16T15:39:23.714500+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:43:58.162669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360704+0000","last_change":"2024-09-16T15:39:21.566471+0000","last_active":"2024-09-16T15:39:24.360704+0000","last_peered":"2024-09-16T15:39:24.360704+0000","last_clean":"2024-09-16T15:39:24.360704+0000","last_became_active":"2024-09-16T15:39:21.566220+0000","last_became_peered":"2024-09-16T15:39:21.566220+0000","last_unstale":"2024-09-16T15:39:24.360704+0000","last_undegraded":"2024-09-16T15:39:24.360704+0000","last_fullsized":"2024-09-16T15:39:24.360704+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:34:11.114212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360894+0000","last_change":"2024-09-16T15:39:22.565878+0000","last_active":"2024-09-16T15:39:24.360894+0000","last_peered":"2024-09-16T15:39:24.360894+0000","last_clean":"2024-09-16T15:39:24.360894+0000","last_became_active":"2024-09-16T15:39:22.565599+0000","last_became_peered":"2024-09-16T15:39:22.565599+0000","last_unstale":"2024-09-16T15:39:24.360894+0000","last_undegraded":"2024-09-16T15:39:24.360894+0000","last_fullsized":"2024-09-16T15:39:24.360894+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:04:51.595786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361501+0000","last_change":"2024-09-16T15:39:22.563299+0000","last_active":"2024-09-16T15:39:24.361501+0000","last_peered":"2024-09-16T15:39:24.361501+0000","last_clean":"2024-09-16T15:39:24.361501+0000","last_became_active":"2024-09-16T15:39:22.560515+0000","last_became_peered":"2024-09-16T15:39:22.560515+0000","last_unstale":"2024-09-16T15:39:24.361501+0000","last_undegraded":"2024-09-16T15:39:24.361501+0000","last_fullsized":"2024-09-16T15:39:24.361501+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:45:21.853848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361156+0000","last_change":"2024-09-16T15:39:21.561779+0000","last_active":"2024-09-16T15:39:24.361156+0000","last_peered":"2024-09-16T15:39:24.361156+0000","last_clean":"2024-09-16T15:39:24.361156+0000","last_became_active":"2024-09-16T15:39:21.561471+0000","last_became_peered":"2024-09-16T15:39:21.561471+0000","last_unstale":"2024-09-16T15:39:24.361156+0000","last_undegraded":"2024-09-16T15:39:24.361156+0000","last_fullsized":"2024-09-16T15:39:24.361156+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:29:24.601704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.560906+0000","last_change":"2024-09-16T15:39:22.565324+0000","last_active":"2024-09-16T15:39:23.560906+0000","last_peered":"2024-09-16T15:39:23.560906+0000","last_clean":"2024-09-16T15:39:23.560906+0000","last_became_active":"2024-09-16T15:39:22.564275+0000","last_became_peered":"2024-09-16T15:39:22.564275+0000","last_unstale":"2024-09-16T15:39:23.560906+0000","last_undegraded":"2024-09-16T15:39:23.560906+0000","last_fullsized":"2024-09-16T15:39:23.560906+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:16:02.503567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361469+0000","last_change":"2024-09-16T15:39:21.563348+0000","last_active":"2024-09-16T15:39:24.361469+0000","last_peered":"2024-09-16T15:39:24.361469+0000","last_clean":"2024-09-16T15:39:24.361469+0000","last_became_active":"2024-09-16T15:39:21.560699+0000","last_became_peered":"2024-09-16T15:39:21.560699+0000","last_unstale":"2024-09-16T15:39:24.361469+0000","last_undegraded":"2024-09-16T15:39:24.361469+0000","last_fullsized":"2024-09-16T15:39:24.361469+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:36:15.980692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.563361+0000","last_change":"2024-09-16T15:39:22.558509+0000","last_active":"2024-09-16T15:39:23.563361+0000","last_peered":"2024-09-16T15:39:23.563361+0000","last_clean":"2024-09-16T15:39:23.563361+0000","last_became_active":"2024-09-16T15:39:22.558197+0000","last_became_peered":"2024-09-16T15:39:22.558197+0000","last_unstale":"2024-09-16T15:39:23.563361+0000","last_undegraded":"2024-09-16T15:39:23.563361+0000","last_fullsized":"2024-09-16T15:39:23.563361+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:10:15.420114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.563141+0000","last_change":"2024-09-16T15:39:21.561287+0000","last_active":"2024-09-16T15:39:23.563141+0000","last_peered":"2024-09-16T15:39:23.563141+0000","last_clean":"2024-09-16T15:39:23.563141+0000","last_became_active":"2024-09-16T15:39:21.559935+0000","last_became_peered":"2024-09-16T15:39:21.559935+0000","last_unstale":"2024-09-16T15:39:23.563141+0000","last_undegraded":"2024-09-16T15:39:23.563141+0000","last_fullsized":"2024-09-16T15:39:23.563141+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:19:57.274084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.563208+0000","last_change":"2024-09-16T15:39:22.576005+0000","last_active":"2024-09-16T15:39:23.563208+0000","last_peered":"2024-09-16T15:39:23.563208+0000","last_clean":"2024-09-16T15:39:23.563208+0000","last_became_active":"2024-09-16T15:39:22.575434+0000","last_became_peered":"2024-09-16T15:39:22.575434+0000","last_unstale":"2024-09-16T15:39:23.563208+0000","last_undegraded":"2024-09-16T15:39:23.563208+0000","last_fullsized":"2024-09-16T15:39:23.563208+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:03:51.340088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.562531+0000","last_change":"2024-09-16T15:39:21.568944+0000","last_active":"2024-09-16T15:39:23.562531+0000","last_peered":"2024-09-16T15:39:23.562531+0000","last_clean":"2024-09-16T15:39:23.562531+0000","last_became_active":"2024-09-16T15:39:21.568706+0000","last_became_peered":"2024-09-16T15:39:21.568706+0000","last_unstale":"2024-09-16T15:39:23.562531+0000","last_undegraded":"2024-09-16T15:39:23.562531+0000","last_fullsized":"2024-09-16T15:39:23.562531+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:19:57.274084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509449,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":76284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509448,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":823296,"data_stored":666744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":241664,"data_stored":78568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":225280,"data_stored":76866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509447,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":233472,"data_stored":77896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509448,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":819200,"data_stored":666722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509446,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":76340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542154,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":76408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":27654604},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:39:36.863 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:39:37.058 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.056+0000 7f9dc4a00640 1 -- 172.21.15.151:0/2515087651 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9dc00b4100 msgr2=0x7f9dc00b1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:37.058 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.056+0000 7f9dc4a00640 1 --2- 172.21.15.151:0/2515087651 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9dc00b4100 0x7f9dc00b1770 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9dc00ce3d0 tx=0x7f9db802a980 comp rx=0 tx=0).stop 2024-09-16T15:39:37.058 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.056+0000 7f9dc4a00640 1 -- 172.21.15.151:0/2515087651 shutdown_connections 2024-09-16T15:39:37.058 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.056+0000 7f9dc4a00640 1 --2- 172.21.15.151:0/2515087651 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9dc00b4100 0x7f9dc00b1770 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:37.058 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.056+0000 7f9dc4a00640 1 --2- 172.21.15.151:0/2515087651 >> v2:172.21.15.167:3300/0 conn(0x7f9dc00b5370 0x7f9dc00b3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:37.058 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.057+0000 7f9dc4a00640 1 -- 172.21.15.151:0/2515087651 >> 172.21.15.151:0/2515087651 conn(0x7f9dc00d3df0 msgr2=0x7f9dc00d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:37.058 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.057+0000 7f9dc4a00640 1 -- 172.21.15.151:0/2515087651 shutdown_connections 2024-09-16T15:39:37.058 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.057+0000 7f9dc4a00640 1 -- 172.21.15.151:0/2515087651 wait complete. 2024-09-16T15:39:37.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.058+0000 7f9dc4a00640 1 Processor -- start 2024-09-16T15:39:37.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.058+0000 7f9dc4a00640 1 -- start start 2024-09-16T15:39:37.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.059+0000 7f9dc4a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9dc00b4100 0x7f9dc01f8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:37.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.059+0000 7f9dc4a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9dc00b5370 0x7f9dc006e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:37.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.059+0000 7f9dc4a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9dc0161440 0x7f9dc014eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:37.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.059+0000 7f9dc4a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc00b2040 con 0x7f9dc00b5370 2024-09-16T15:39:37.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.059+0000 7f9dc4a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc00b1ec0 con 0x7f9dc0161440 2024-09-16T15:39:37.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.059+0000 7f9dc4a00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f9dc00b21c0 con 0x7f9dc00b4100 2024-09-16T15:39:37.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.059+0000 7f9dbea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9dc00b5370 0x7f9dc006e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:37.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.060+0000 7f9dbea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9dc00b5370 0x7f9dc006e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:57434/0 (socket says 172.21.15.151:57434) 2024-09-16T15:39:37.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.060+0000 7f9dbea00640 1 -- 172.21.15.151:0/2800597362 learned_addr learned my addr 172.21.15.151:0/2800597362 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:37.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.060+0000 7f9dbfe00640 1 --2- 172.21.15.151:0/2800597362 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9dc0161440 0x7f9dc014eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:37.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.060+0000 7f9dbea00640 1 -- 172.21.15.151:0/2800597362 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9dc00b4100 msgr2=0x7f9dc01f8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:37.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.060+0000 7f9dbf400640 1 --2- 172.21.15.151:0/2800597362 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9dc00b4100 0x7f9dc01f8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:37.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.060+0000 7f9dbea00640 1 --2- 172.21.15.151:0/2800597362 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9dc00b4100 0x7f9dc01f8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:37.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.060+0000 7f9dbea00640 1 -- 172.21.15.151:0/2800597362 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9dc0161440 msgr2=0x7f9dc014eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:37.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.060+0000 7f9dbea00640 1 --2- 172.21.15.151:0/2800597362 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9dc0161440 0x7f9dc014eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:37.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.060+0000 7f9dbea00640 1 -- 172.21.15.151:0/2800597362 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dc01f87a0 con 0x7f9dc00b5370 2024-09-16T15:39:37.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.060+0000 7f9dbfe00640 1 --2- 172.21.15.151:0/2800597362 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9dc0161440 0x7f9dc014eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:39:37.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.060+0000 7f9dbf400640 1 --2- 172.21.15.151:0/2800597362 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9dc00b4100 0x7f9dc01f8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:39:37.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.060+0000 7f9dbea00640 1 --2- 172.21.15.151:0/2800597362 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9dc00b5370 0x7f9dc006e5d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9dac00c970 tx=0x7f9dac00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:37.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.060+0000 7f9db7400640 1 -- 172.21.15.151:0/2800597362 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dac007bb0 con 0x7f9dc00b5370 2024-09-16T15:39:37.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.060+0000 7f9db7400640 1 -- 172.21.15.151:0/2800597362 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9dac005020 con 0x7f9dc00b5370 2024-09-16T15:39:37.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.060+0000 7f9db7400640 1 -- 172.21.15.151:0/2800597362 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dac007e60 con 0x7f9dc00b5370 2024-09-16T15:39:37.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.060+0000 7f9dc4a00640 1 -- 172.21.15.151:0/2800597362 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dc0159600 con 0x7f9dc00b5370 2024-09-16T15:39:37.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.061+0000 7f9dc4a00640 1 -- 172.21.15.151:0/2800597362 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dc0159bc0 con 0x7f9dc00b5370 2024-09-16T15:39:37.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.062+0000 7f9dc4a00640 1 -- 172.21.15.151:0/2800597362 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dc00b1770 con 0x7f9dc00b5370 2024-09-16T15:39:37.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.063+0000 7f9db7400640 1 -- 172.21.15.151:0/2800597362 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f9dac005470 con 0x7f9dc00b5370 2024-09-16T15:39:37.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.064+0000 7f9db7400640 1 --2- 172.21.15.151:0/2800597362 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9da0078850 0x7f9da007ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:37.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.064+0000 7f9db7400640 1 -- 172.21.15.151:0/2800597362 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f9dac09aec0 con 0x7f9dc00b5370 2024-09-16T15:39:37.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.064+0000 7f9dbf400640 1 --2- 172.21.15.151:0/2800597362 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9da0078850 0x7f9da007ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:37.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.065+0000 7f9dbf400640 1 --2- 172.21.15.151:0/2800597362 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9da0078850 0x7f9da007ad10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9dc01f7800 tx=0x7f9db0008040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:37.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.068+0000 7f9db7400640 1 -- 172.21.15.151:0/2800597362 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9dac014030 con 0x7f9dc00b5370 2024-09-16T15:39:37.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.251+0000 7f9dc4a00640 1 -- 172.21.15.151:0/2800597362 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9dc01605d0 con 0x7f9da0078850 2024-09-16T15:39:37.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.256+0000 7f9db7400640 1 -- 172.21.15.151:0/2800597362 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251822 (secure 0 0 0) 0x7f9dc01605d0 con 0x7f9da0078850 2024-09-16T15:39:37.260 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:39:37.261 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-09-16T15:39:37.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.260+0000 7f9dc4a00640 1 -- 172.21.15.151:0/2800597362 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9da0078850 msgr2=0x7f9da007ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:37.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.260+0000 7f9dc4a00640 1 --2- 172.21.15.151:0/2800597362 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9da0078850 0x7f9da007ad10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9dc01f7800 tx=0x7f9db0008040 comp rx=0 tx=0).stop 2024-09-16T15:39:37.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.260+0000 7f9dc4a00640 1 -- 172.21.15.151:0/2800597362 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9dc00b5370 msgr2=0x7f9dc006e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:37.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.260+0000 7f9dc4a00640 1 --2- 172.21.15.151:0/2800597362 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9dc00b5370 0x7f9dc006e5d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9dac00c970 tx=0x7f9dac00ce30 comp rx=0 tx=0).stop 2024-09-16T15:39:37.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.260+0000 7f9dc4a00640 1 -- 172.21.15.151:0/2800597362 shutdown_connections 2024-09-16T15:39:37.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.260+0000 7f9dc4a00640 1 --2- 172.21.15.151:0/2800597362 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9da0078850 0x7f9da007ad10 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:37.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.260+0000 7f9dc4a00640 1 --2- 172.21.15.151:0/2800597362 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9dc0161440 0x7f9dc014eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:37.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.260+0000 7f9dc4a00640 1 --2- 172.21.15.151:0/2800597362 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9dc00b5370 0x7f9dc006e5d0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:37.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.260+0000 7f9dc4a00640 1 --2- 172.21.15.151:0/2800597362 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9dc00b4100 0x7f9dc01f8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:37.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.260+0000 7f9dc4a00640 1 -- 172.21.15.151:0/2800597362 >> 172.21.15.151:0/2800597362 conn(0x7f9dc00d3df0 msgr2=0x7f9dc00d41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:37.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.260+0000 7f9dc4a00640 1 -- 172.21.15.151:0/2800597362 shutdown_connections 2024-09-16T15:39:37.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.260+0000 7f9dc4a00640 1 -- 172.21.15.151:0/2800597362 wait complete. 2024-09-16T15:39:37.277 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2024-09-16T15:39:35.579882+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219184,"kb_used_data":2952,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755536,"statfs":{"total":773094113280,"available":772869668864,"internally_reserved":0,"allocated":3022848,"data_stored":1795828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12738,"internal_metadata":221236798},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003146"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.362052+0000","last_change":"2024-09-16T15:39:22.577878+0000","last_active":"2024-09-16T15:39:24.362052+0000","last_peered":"2024-09-16T15:39:24.362052+0000","last_clean":"2024-09-16T15:39:24.362052+0000","last_became_active":"2024-09-16T15:39:22.570201+0000","last_became_peered":"2024-09-16T15:39:22.570201+0000","last_unstale":"2024-09-16T15:39:24.362052+0000","last_undegraded":"2024-09-16T15:39:24.362052+0000","last_fullsized":"2024-09-16T15:39:24.362052+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:39:49.705970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.563074+0000","last_change":"2024-09-16T15:39:22.568365+0000","last_active":"2024-09-16T15:39:23.563074+0000","last_peered":"2024-09-16T15:39:23.563074+0000","last_clean":"2024-09-16T15:39:23.563074+0000","last_became_active":"2024-09-16T15:39:22.564695+0000","last_became_peered":"2024-09-16T15:39:22.564695+0000","last_unstale":"2024-09-16T15:39:23.563074+0000","last_undegraded":"2024-09-16T15:39:23.563074+0000","last_fullsized":"2024-09-16T15:39:23.563074+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:18:39.431491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.563192+0000","last_change":"2024-09-16T15:39:22.575183+0000","last_active":"2024-09-16T15:39:23.563192+0000","last_peered":"2024-09-16T15:39:23.563192+0000","last_clean":"2024-09-16T15:39:23.563192+0000","last_became_active":"2024-09-16T15:39:22.574916+0000","last_became_peered":"2024-09-16T15:39:22.574916+0000","last_unstale":"2024-09-16T15:39:23.563192+0000","last_undegraded":"2024-09-16T15:39:23.563192+0000","last_fullsized":"2024-09-16T15:39:23.563192+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:06:34.579134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.362360+0000","last_change":"2024-09-16T15:39:22.580108+0000","last_active":"2024-09-16T15:39:24.362360+0000","last_peered":"2024-09-16T15:39:24.362360+0000","last_clean":"2024-09-16T15:39:24.362360+0000","last_became_active":"2024-09-16T15:39:22.579859+0000","last_became_peered":"2024-09-16T15:39:22.579859+0000","last_unstale":"2024-09-16T15:39:24.362360+0000","last_undegraded":"2024-09-16T15:39:24.362360+0000","last_fullsized":"2024-09-16T15:39:24.362360+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:32:44.877620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361299+0000","last_change":"2024-09-16T15:39:22.566468+0000","last_active":"2024-09-16T15:39:24.361299+0000","last_peered":"2024-09-16T15:39:24.361299+0000","last_clean":"2024-09-16T15:39:24.361299+0000","last_became_active":"2024-09-16T15:39:22.559581+0000","last_became_peered":"2024-09-16T15:39:22.559581+0000","last_unstale":"2024-09-16T15:39:24.361299+0000","last_undegraded":"2024-09-16T15:39:24.361299+0000","last_fullsized":"2024-09-16T15:39:24.361299+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:35:04.935435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361543+0000","last_change":"2024-09-16T15:39:22.579964+0000","last_active":"2024-09-16T15:39:24.361543+0000","last_peered":"2024-09-16T15:39:24.361543+0000","last_clean":"2024-09-16T15:39:24.361543+0000","last_became_active":"2024-09-16T15:39:22.579714+0000","last_became_peered":"2024-09-16T15:39:22.579714+0000","last_unstale":"2024-09-16T15:39:24.361543+0000","last_undegraded":"2024-09-16T15:39:24.361543+0000","last_fullsized":"2024-09-16T15:39:24.361543+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:54:55.307273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360552+0000","last_change":"2024-09-16T15:39:22.578008+0000","last_active":"2024-09-16T15:39:24.360552+0000","last_peered":"2024-09-16T15:39:24.360552+0000","last_clean":"2024-09-16T15:39:24.360552+0000","last_became_active":"2024-09-16T15:39:22.577770+0000","last_became_peered":"2024-09-16T15:39:22.577770+0000","last_unstale":"2024-09-16T15:39:24.360552+0000","last_undegraded":"2024-09-16T15:39:24.360552+0000","last_fullsized":"2024-09-16T15:39:24.360552+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:03:05.655450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.562837+0000","last_change":"2024-09-16T15:39:22.565821+0000","last_active":"2024-09-16T15:39:23.562837+0000","last_peered":"2024-09-16T15:39:23.562837+0000","last_clean":"2024-09-16T15:39:23.562837+0000","last_became_active":"2024-09-16T15:39:22.564114+0000","last_became_peered":"2024-09-16T15:39:22.564114+0000","last_unstale":"2024-09-16T15:39:23.562837+0000","last_undegraded":"2024-09-16T15:39:23.562837+0000","last_fullsized":"2024-09-16T15:39:23.562837+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:03:28.558231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.561256+0000","last_change":"2024-09-16T15:39:22.557568+0000","last_active":"2024-09-16T15:39:23.561256+0000","last_peered":"2024-09-16T15:39:23.561256+0000","last_clean":"2024-09-16T15:39:23.561256+0000","last_became_active":"2024-09-16T15:39:22.556305+0000","last_became_peered":"2024-09-16T15:39:22.556305+0000","last_unstale":"2024-09-16T15:39:23.561256+0000","last_undegraded":"2024-09-16T15:39:23.561256+0000","last_fullsized":"2024-09-16T15:39:23.561256+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:19:39.018320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361933+0000","last_change":"2024-09-16T15:39:22.573944+0000","last_active":"2024-09-16T15:39:24.361933+0000","last_peered":"2024-09-16T15:39:24.361933+0000","last_clean":"2024-09-16T15:39:24.361933+0000","last_became_active":"2024-09-16T15:39:22.573688+0000","last_became_peered":"2024-09-16T15:39:22.573688+0000","last_unstale":"2024-09-16T15:39:24.361933+0000","last_undegraded":"2024-09-16T15:39:24.361933+0000","last_fullsized":"2024-09-16T15:39:24.361933+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:00:05.185794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360929+0000","last_change":"2024-09-16T15:39:22.580084+0000","last_active":"2024-09-16T15:39:24.360929+0000","last_peered":"2024-09-16T15:39:24.360929+0000","last_clean":"2024-09-16T15:39:24.360929+0000","last_became_active":"2024-09-16T15:39:22.571878+0000","last_became_peered":"2024-09-16T15:39:22.571878+0000","last_unstale":"2024-09-16T15:39:24.360929+0000","last_undegraded":"2024-09-16T15:39:24.360929+0000","last_fullsized":"2024-09-16T15:39:24.360929+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:46:31.911518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361633+0000","last_change":"2024-09-16T15:39:22.575436+0000","last_active":"2024-09-16T15:39:24.361633+0000","last_peered":"2024-09-16T15:39:24.361633+0000","last_clean":"2024-09-16T15:39:24.361633+0000","last_became_active":"2024-09-16T15:39:22.568528+0000","last_became_peered":"2024-09-16T15:39:22.568528+0000","last_unstale":"2024-09-16T15:39:24.361633+0000","last_undegraded":"2024-09-16T15:39:24.361633+0000","last_fullsized":"2024-09-16T15:39:24.361633+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:05:28.120874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360645+0000","last_change":"2024-09-16T15:39:22.566575+0000","last_active":"2024-09-16T15:39:24.360645+0000","last_peered":"2024-09-16T15:39:24.360645+0000","last_clean":"2024-09-16T15:39:24.360645+0000","last_became_active":"2024-09-16T15:39:22.566295+0000","last_became_peered":"2024-09-16T15:39:22.566295+0000","last_unstale":"2024-09-16T15:39:24.360645+0000","last_undegraded":"2024-09-16T15:39:24.360645+0000","last_fullsized":"2024-09-16T15:39:24.360645+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:10:23.402092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360395+0000","last_change":"2024-09-16T15:39:22.571476+0000","last_active":"2024-09-16T15:39:24.360395+0000","last_peered":"2024-09-16T15:39:24.360395+0000","last_clean":"2024-09-16T15:39:24.360395+0000","last_became_active":"2024-09-16T15:39:22.571216+0000","last_became_peered":"2024-09-16T15:39:22.571216+0000","last_unstale":"2024-09-16T15:39:24.360395+0000","last_undegraded":"2024-09-16T15:39:24.360395+0000","last_fullsized":"2024-09-16T15:39:24.360395+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:40:58.311998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.563103+0000","last_change":"2024-09-16T15:39:22.564630+0000","last_active":"2024-09-16T15:39:23.563103+0000","last_peered":"2024-09-16T15:39:23.563103+0000","last_clean":"2024-09-16T15:39:23.563103+0000","last_became_active":"2024-09-16T15:39:22.564326+0000","last_became_peered":"2024-09-16T15:39:22.564326+0000","last_unstale":"2024-09-16T15:39:23.563103+0000","last_undegraded":"2024-09-16T15:39:23.563103+0000","last_fullsized":"2024-09-16T15:39:23.563103+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:58:23.963120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360193+0000","last_change":"2024-09-16T15:39:22.574288+0000","last_active":"2024-09-16T15:39:24.360193+0000","last_peered":"2024-09-16T15:39:24.360193+0000","last_clean":"2024-09-16T15:39:24.360193+0000","last_became_active":"2024-09-16T15:39:22.573979+0000","last_became_peered":"2024-09-16T15:39:22.573979+0000","last_unstale":"2024-09-16T15:39:24.360193+0000","last_undegraded":"2024-09-16T15:39:24.360193+0000","last_fullsized":"2024-09-16T15:39:24.360193+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:10:15.420114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361109+0000","last_change":"2024-09-16T15:39:22.561784+0000","last_active":"2024-09-16T15:39:24.361109+0000","last_peered":"2024-09-16T15:39:24.361109+0000","last_clean":"2024-09-16T15:39:24.361109+0000","last_became_active":"2024-09-16T15:39:22.561487+0000","last_became_peered":"2024-09-16T15:39:22.561487+0000","last_unstale":"2024-09-16T15:39:24.361109+0000","last_undegraded":"2024-09-16T15:39:24.361109+0000","last_fullsized":"2024-09-16T15:39:24.361109+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:26:52.642854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.714434+0000","last_change":"2024-09-16T15:39:22.579035+0000","last_active":"2024-09-16T15:39:23.714434+0000","last_peered":"2024-09-16T15:39:23.714434+0000","last_clean":"2024-09-16T15:39:23.714434+0000","last_became_active":"2024-09-16T15:39:22.578734+0000","last_became_peered":"2024-09-16T15:39:22.578734+0000","last_unstale":"2024-09-16T15:39:23.714434+0000","last_undegraded":"2024-09-16T15:39:23.714434+0000","last_fullsized":"2024-09-16T15:39:23.714434+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:13:19.369864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361928+0000","last_change":"2024-09-16T15:39:22.577483+0000","last_active":"2024-09-16T15:39:24.361928+0000","last_peered":"2024-09-16T15:39:24.361928+0000","last_clean":"2024-09-16T15:39:24.361928+0000","last_became_active":"2024-09-16T15:39:22.577242+0000","last_became_peered":"2024-09-16T15:39:22.577242+0000","last_unstale":"2024-09-16T15:39:24.361928+0000","last_undegraded":"2024-09-16T15:39:24.361928+0000","last_fullsized":"2024-09-16T15:39:24.361928+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:29:43.879706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360912+0000","last_change":"2024-09-16T15:39:22.561587+0000","last_active":"2024-09-16T15:39:24.360912+0000","last_peered":"2024-09-16T15:39:24.360912+0000","last_clean":"2024-09-16T15:39:24.360912+0000","last_became_active":"2024-09-16T15:39:22.561213+0000","last_became_peered":"2024-09-16T15:39:22.561213+0000","last_unstale":"2024-09-16T15:39:24.360912+0000","last_undegraded":"2024-09-16T15:39:24.360912+0000","last_fullsized":"2024-09-16T15:39:24.360912+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:00:48.581056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.562380+0000","last_change":"2024-09-16T15:39:22.579126+0000","last_active":"2024-09-16T15:39:23.562380+0000","last_peered":"2024-09-16T15:39:23.562380+0000","last_clean":"2024-09-16T15:39:23.562380+0000","last_became_active":"2024-09-16T15:39:22.577915+0000","last_became_peered":"2024-09-16T15:39:22.577915+0000","last_unstale":"2024-09-16T15:39:23.562380+0000","last_undegraded":"2024-09-16T15:39:23.562380+0000","last_fullsized":"2024-09-16T15:39:23.562380+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:18:15.980220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.560783+0000","last_change":"2024-09-16T15:39:22.560525+0000","last_active":"2024-09-16T15:39:23.560783+0000","last_peered":"2024-09-16T15:39:23.560783+0000","last_clean":"2024-09-16T15:39:23.560783+0000","last_became_active":"2024-09-16T15:39:22.560279+0000","last_became_peered":"2024-09-16T15:39:22.560279+0000","last_unstale":"2024-09-16T15:39:23.560783+0000","last_undegraded":"2024-09-16T15:39:23.560783+0000","last_fullsized":"2024-09-16T15:39:23.560783+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:22:24.884519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.562668+0000","last_change":"2024-09-16T15:39:22.567598+0000","last_active":"2024-09-16T15:39:23.562668+0000","last_peered":"2024-09-16T15:39:23.562668+0000","last_clean":"2024-09-16T15:39:23.562668+0000","last_became_active":"2024-09-16T15:39:22.567356+0000","last_became_peered":"2024-09-16T15:39:22.567356+0000","last_unstale":"2024-09-16T15:39:23.562668+0000","last_undegraded":"2024-09-16T15:39:23.562668+0000","last_fullsized":"2024-09-16T15:39:23.562668+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:22:34.502460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360199+0000","last_change":"2024-09-16T15:39:22.558806+0000","last_active":"2024-09-16T15:39:24.360199+0000","last_peered":"2024-09-16T15:39:24.360199+0000","last_clean":"2024-09-16T15:39:24.360199+0000","last_became_active":"2024-09-16T15:39:22.558473+0000","last_became_peered":"2024-09-16T15:39:22.558473+0000","last_unstale":"2024-09-16T15:39:24.360199+0000","last_undegraded":"2024-09-16T15:39:24.360199+0000","last_fullsized":"2024-09-16T15:39:24.360199+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:05:28.120874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.563124+0000","last_change":"2024-09-16T15:39:22.564434+0000","last_active":"2024-09-16T15:39:23.563124+0000","last_peered":"2024-09-16T15:39:23.563124+0000","last_clean":"2024-09-16T15:39:23.563124+0000","last_became_active":"2024-09-16T15:39:22.563970+0000","last_became_peered":"2024-09-16T15:39:22.563970+0000","last_unstale":"2024-09-16T15:39:23.563124+0000","last_undegraded":"2024-09-16T15:39:23.563124+0000","last_fullsized":"2024-09-16T15:39:23.563124+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:43:27.368913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.362535+0000","last_change":"2024-09-16T15:39:22.584858+0000","last_active":"2024-09-16T15:39:24.362535+0000","last_peered":"2024-09-16T15:39:24.362535+0000","last_clean":"2024-09-16T15:39:24.362535+0000","last_became_active":"2024-09-16T15:39:22.584152+0000","last_became_peered":"2024-09-16T15:39:22.584152+0000","last_unstale":"2024-09-16T15:39:24.362535+0000","last_undegraded":"2024-09-16T15:39:24.362535+0000","last_fullsized":"2024-09-16T15:39:24.362535+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:08:07.935905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360764+0000","last_change":"2024-09-16T15:39:22.573867+0000","last_active":"2024-09-16T15:39:24.360764+0000","last_peered":"2024-09-16T15:39:24.360764+0000","last_clean":"2024-09-16T15:39:24.360764+0000","last_became_active":"2024-09-16T15:39:22.573567+0000","last_became_peered":"2024-09-16T15:39:22.573567+0000","last_unstale":"2024-09-16T15:39:24.360764+0000","last_undegraded":"2024-09-16T15:39:24.360764+0000","last_fullsized":"2024-09-16T15:39:24.360764+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:18:15.980220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360545+0000","last_change":"2024-09-16T15:39:22.574268+0000","last_active":"2024-09-16T15:39:24.360545+0000","last_peered":"2024-09-16T15:39:24.360545+0000","last_clean":"2024-09-16T15:39:24.360545+0000","last_became_active":"2024-09-16T15:39:22.572891+0000","last_became_peered":"2024-09-16T15:39:22.572891+0000","last_unstale":"2024-09-16T15:39:24.360545+0000","last_undegraded":"2024-09-16T15:39:24.360545+0000","last_fullsized":"2024-09-16T15:39:24.360545+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:29:11.296912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.562547+0000","last_change":"2024-09-16T15:39:22.563793+0000","last_active":"2024-09-16T15:39:23.562547+0000","last_peered":"2024-09-16T15:39:23.562547+0000","last_clean":"2024-09-16T15:39:23.562547+0000","last_became_active":"2024-09-16T15:39:22.563504+0000","last_became_peered":"2024-09-16T15:39:22.563504+0000","last_unstale":"2024-09-16T15:39:23.562547+0000","last_undegraded":"2024-09-16T15:39:23.562547+0000","last_fullsized":"2024-09-16T15:39:23.562547+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:56:07.665096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361245+0000","last_change":"2024-09-16T15:39:22.569041+0000","last_active":"2024-09-16T15:39:24.361245+0000","last_peered":"2024-09-16T15:39:24.361245+0000","last_clean":"2024-09-16T15:39:24.361245+0000","last_became_active":"2024-09-16T15:39:22.568799+0000","last_became_peered":"2024-09-16T15:39:22.568799+0000","last_unstale":"2024-09-16T15:39:24.361245+0000","last_undegraded":"2024-09-16T15:39:24.361245+0000","last_fullsized":"2024-09-16T15:39:24.361245+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:58:26.472440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.714354+0000","last_change":"2024-09-16T15:39:22.566051+0000","last_active":"2024-09-16T15:39:23.714354+0000","last_peered":"2024-09-16T15:39:23.714354+0000","last_clean":"2024-09-16T15:39:23.714354+0000","last_became_active":"2024-09-16T15:39:22.565764+0000","last_became_peered":"2024-09-16T15:39:22.565764+0000","last_unstale":"2024-09-16T15:39:23.714354+0000","last_undegraded":"2024-09-16T15:39:23.714354+0000","last_fullsized":"2024-09-16T15:39:23.714354+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:42:34.417720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361771+0000","last_change":"2024-09-16T15:39:22.574296+0000","last_active":"2024-09-16T15:39:24.361771+0000","last_peered":"2024-09-16T15:39:24.361771+0000","last_clean":"2024-09-16T15:39:24.361771+0000","last_became_active":"2024-09-16T15:39:22.574004+0000","last_became_peered":"2024-09-16T15:39:22.574004+0000","last_unstale":"2024-09-16T15:39:24.361771+0000","last_undegraded":"2024-09-16T15:39:24.361771+0000","last_fullsized":"2024-09-16T15:39:24.361771+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:36:56.913296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.560837+0000","last_change":"2024-09-16T15:39:22.559868+0000","last_active":"2024-09-16T15:39:23.560837+0000","last_peered":"2024-09-16T15:39:23.560837+0000","last_clean":"2024-09-16T15:39:23.560837+0000","last_became_active":"2024-09-16T15:39:22.559506+0000","last_became_peered":"2024-09-16T15:39:22.559506+0000","last_unstale":"2024-09-16T15:39:23.560837+0000","last_undegraded":"2024-09-16T15:39:23.560837+0000","last_fullsized":"2024-09-16T15:39:23.560837+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:33:53.332507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.562441+0000","last_change":"2024-09-16T15:39:21.562538+0000","last_active":"2024-09-16T15:39:23.562441+0000","last_peered":"2024-09-16T15:39:23.562441+0000","last_clean":"2024-09-16T15:39:23.562441+0000","last_became_active":"2024-09-16T15:39:21.559240+0000","last_became_peered":"2024-09-16T15:39:21.559240+0000","last_unstale":"2024-09-16T15:39:23.562441+0000","last_undegraded":"2024-09-16T15:39:23.562441+0000","last_fullsized":"2024-09-16T15:39:23.562441+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:43:58.109261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.562593+0000","last_change":"2024-09-16T15:39:22.582803+0000","last_active":"2024-09-16T15:39:23.562593+0000","last_peered":"2024-09-16T15:39:23.562593+0000","last_clean":"2024-09-16T15:39:23.562593+0000","last_became_active":"2024-09-16T15:39:22.582364+0000","last_became_peered":"2024-09-16T15:39:22.582364+0000","last_unstale":"2024-09-16T15:39:23.562593+0000","last_undegraded":"2024-09-16T15:39:23.562593+0000","last_fullsized":"2024-09-16T15:39:23.562593+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:40:27.419876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360960+0000","last_change":"2024-09-16T15:39:21.567667+0000","last_active":"2024-09-16T15:39:24.360960+0000","last_peered":"2024-09-16T15:39:24.360960+0000","last_clean":"2024-09-16T15:39:24.360960+0000","last_became_active":"2024-09-16T15:39:21.567399+0000","last_became_peered":"2024-09-16T15:39:21.567399+0000","last_unstale":"2024-09-16T15:39:24.360960+0000","last_undegraded":"2024-09-16T15:39:24.360960+0000","last_fullsized":"2024-09-16T15:39:24.360960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:22:23.876565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360132+0000","last_change":"2024-09-16T15:39:21.566105+0000","last_active":"2024-09-16T15:39:24.360132+0000","last_peered":"2024-09-16T15:39:24.360132+0000","last_clean":"2024-09-16T15:39:24.360132+0000","last_became_active":"2024-09-16T15:39:21.565719+0000","last_became_peered":"2024-09-16T15:39:21.565719+0000","last_unstale":"2024-09-16T15:39:24.360132+0000","last_undegraded":"2024-09-16T15:39:24.360132+0000","last_fullsized":"2024-09-16T15:39:24.360132+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:43:58.109261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.562255+0000","last_change":"2024-09-16T15:39:22.575843+0000","last_active":"2024-09-16T15:39:23.562255+0000","last_peered":"2024-09-16T15:39:23.562255+0000","last_clean":"2024-09-16T15:39:23.562255+0000","last_became_active":"2024-09-16T15:39:22.575543+0000","last_became_peered":"2024-09-16T15:39:22.575543+0000","last_unstale":"2024-09-16T15:39:23.562255+0000","last_undegraded":"2024-09-16T15:39:23.562255+0000","last_fullsized":"2024-09-16T15:39:23.562255+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:40:27.419876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360365+0000","last_change":"2024-09-16T15:39:21.562879+0000","last_active":"2024-09-16T15:39:24.360365+0000","last_peered":"2024-09-16T15:39:24.360365+0000","last_clean":"2024-09-16T15:39:24.360365+0000","last_became_active":"2024-09-16T15:39:21.562554+0000","last_became_peered":"2024-09-16T15:39:21.562554+0000","last_unstale":"2024-09-16T15:39:24.360365+0000","last_undegraded":"2024-09-16T15:39:24.360365+0000","last_fullsized":"2024-09-16T15:39:24.360365+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:33:52.324553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.562038+0000","last_change":"2024-09-16T15:39:22.571218+0000","last_active":"2024-09-16T15:39:23.562038+0000","last_peered":"2024-09-16T15:39:23.562038+0000","last_clean":"2024-09-16T15:39:23.562038+0000","last_became_active":"2024-09-16T15:39:22.570931+0000","last_became_peered":"2024-09-16T15:39:22.570931+0000","last_unstale":"2024-09-16T15:39:23.562038+0000","last_undegraded":"2024-09-16T15:39:23.562038+0000","last_fullsized":"2024-09-16T15:39:23.562038+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:57:08.823794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.561161+0000","last_change":"2024-09-16T15:39:21.558263+0000","last_active":"2024-09-16T15:39:23.561161+0000","last_peered":"2024-09-16T15:39:23.561161+0000","last_clean":"2024-09-16T15:39:23.561161+0000","last_became_active":"2024-09-16T15:39:21.557615+0000","last_became_peered":"2024-09-16T15:39:21.557615+0000","last_unstale":"2024-09-16T15:39:23.561161+0000","last_undegraded":"2024-09-16T15:39:23.561161+0000","last_fullsized":"2024-09-16T15:39:23.561161+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:10:40.788797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.362565+0000","last_change":"2024-09-16T15:39:22.586182+0000","last_active":"2024-09-16T15:39:24.362565+0000","last_peered":"2024-09-16T15:39:24.362565+0000","last_clean":"2024-09-16T15:39:24.362565+0000","last_became_active":"2024-09-16T15:39:22.584486+0000","last_became_peered":"2024-09-16T15:39:22.584486+0000","last_unstale":"2024-09-16T15:39:24.362565+0000","last_undegraded":"2024-09-16T15:39:24.362565+0000","last_fullsized":"2024-09-16T15:39:24.362565+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:25:52.095325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361757+0000","last_change":"2024-09-16T15:39:21.558934+0000","last_active":"2024-09-16T15:39:24.361757+0000","last_peered":"2024-09-16T15:39:24.361757+0000","last_clean":"2024-09-16T15:39:24.361757+0000","last_became_active":"2024-09-16T15:39:21.557612+0000","last_became_peered":"2024-09-16T15:39:21.557612+0000","last_unstale":"2024-09-16T15:39:24.361757+0000","last_undegraded":"2024-09-16T15:39:24.361757+0000","last_fullsized":"2024-09-16T15:39:24.361757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:21:44.910826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.563494+0000","last_change":"2024-09-16T15:39:22.563449+0000","last_active":"2024-09-16T15:39:23.563494+0000","last_peered":"2024-09-16T15:39:23.563494+0000","last_clean":"2024-09-16T15:39:23.563494+0000","last_became_active":"2024-09-16T15:39:22.558597+0000","last_became_peered":"2024-09-16T15:39:22.558597+0000","last_unstale":"2024-09-16T15:39:23.563494+0000","last_undegraded":"2024-09-16T15:39:23.563494+0000","last_fullsized":"2024-09-16T15:39:23.563494+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:08:47.835110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.714227+0000","last_change":"2024-09-16T15:39:21.561906+0000","last_active":"2024-09-16T15:39:23.714227+0000","last_peered":"2024-09-16T15:39:23.714227+0000","last_clean":"2024-09-16T15:39:23.714227+0000","last_became_active":"2024-09-16T15:39:21.561566+0000","last_became_peered":"2024-09-16T15:39:21.561566+0000","last_unstale":"2024-09-16T15:39:23.714227+0000","last_undegraded":"2024-09-16T15:39:23.714227+0000","last_fullsized":"2024-09-16T15:39:23.714227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:12:53.853659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360840+0000","last_change":"2024-09-16T15:39:22.581196+0000","last_active":"2024-09-16T15:39:24.360840+0000","last_peered":"2024-09-16T15:39:24.360840+0000","last_clean":"2024-09-16T15:39:24.360840+0000","last_became_active":"2024-09-16T15:39:22.579631+0000","last_became_peered":"2024-09-16T15:39:22.579631+0000","last_unstale":"2024-09-16T15:39:24.360840+0000","last_undegraded":"2024-09-16T15:39:24.360840+0000","last_fullsized":"2024-09-16T15:39:24.360840+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:15:37.944346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.362202+0000","last_change":"2024-09-16T15:39:21.560090+0000","last_active":"2024-09-16T15:39:24.362202+0000","last_peered":"2024-09-16T15:39:24.362202+0000","last_clean":"2024-09-16T15:39:24.362202+0000","last_became_active":"2024-09-16T15:39:21.558972+0000","last_became_peered":"2024-09-16T15:39:21.558972+0000","last_unstale":"2024-09-16T15:39:24.362202+0000","last_undegraded":"2024-09-16T15:39:24.362202+0000","last_fullsized":"2024-09-16T15:39:24.362202+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:12:19.666037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361423+0000","last_change":"2024-09-16T15:39:22.560439+0000","last_active":"2024-09-16T15:39:24.361423+0000","last_peered":"2024-09-16T15:39:24.361423+0000","last_clean":"2024-09-16T15:39:24.361423+0000","last_became_active":"2024-09-16T15:39:22.560172+0000","last_became_peered":"2024-09-16T15:39:22.560172+0000","last_unstale":"2024-09-16T15:39:24.361423+0000","last_undegraded":"2024-09-16T15:39:24.361423+0000","last_fullsized":"2024-09-16T15:39:24.361423+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:01:25.170048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.714255+0000","last_change":"2024-09-16T15:39:21.567091+0000","last_active":"2024-09-16T15:39:23.714255+0000","last_peered":"2024-09-16T15:39:23.714255+0000","last_clean":"2024-09-16T15:39:23.714255+0000","last_became_active":"2024-09-16T15:39:21.566668+0000","last_became_peered":"2024-09-16T15:39:21.566668+0000","last_unstale":"2024-09-16T15:39:23.714255+0000","last_undegraded":"2024-09-16T15:39:23.714255+0000","last_fullsized":"2024-09-16T15:39:23.714255+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:08:06.242307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360373+0000","last_change":"2024-09-16T15:39:22.569735+0000","last_active":"2024-09-16T15:39:24.360373+0000","last_peered":"2024-09-16T15:39:24.360373+0000","last_clean":"2024-09-16T15:39:24.360373+0000","last_became_active":"2024-09-16T15:39:22.569217+0000","last_became_peered":"2024-09-16T15:39:22.569217+0000","last_unstale":"2024-09-16T15:39:24.360373+0000","last_undegraded":"2024-09-16T15:39:24.360373+0000","last_fullsized":"2024-09-16T15:39:24.360373+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:34:12.122166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361075+0000","last_change":"2024-09-16T15:39:21.560934+0000","last_active":"2024-09-16T15:39:24.361075+0000","last_peered":"2024-09-16T15:39:24.361075+0000","last_clean":"2024-09-16T15:39:24.361075+0000","last_became_active":"2024-09-16T15:39:21.560690+0000","last_became_peered":"2024-09-16T15:39:21.560690+0000","last_unstale":"2024-09-16T15:39:24.361075+0000","last_undegraded":"2024-09-16T15:39:24.361075+0000","last_fullsized":"2024-09-16T15:39:24.361075+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:10:22.394137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360730+0000","last_change":"2024-09-16T15:39:22.569919+0000","last_active":"2024-09-16T15:39:24.360730+0000","last_peered":"2024-09-16T15:39:24.360730+0000","last_clean":"2024-09-16T15:39:24.360730+0000","last_became_active":"2024-09-16T15:39:22.569624+0000","last_became_peered":"2024-09-16T15:39:22.569624+0000","last_unstale":"2024-09-16T15:39:24.360730+0000","last_undegraded":"2024-09-16T15:39:24.360730+0000","last_fullsized":"2024-09-16T15:39:24.360730+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:42:11.842370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360437+0000","last_change":"2024-09-16T15:39:21.565143+0000","last_active":"2024-09-16T15:39:24.360437+0000","last_peered":"2024-09-16T15:39:24.360437+0000","last_clean":"2024-09-16T15:39:24.360437+0000","last_became_active":"2024-09-16T15:39:21.564887+0000","last_became_peered":"2024-09-16T15:39:21.564887+0000","last_unstale":"2024-09-16T15:39:24.360437+0000","last_undegraded":"2024-09-16T15:39:24.360437+0000","last_fullsized":"2024-09-16T15:39:24.360437+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:54:54.299318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.560542+0000","last_change":"2024-09-16T15:39:22.560025+0000","last_active":"2024-09-16T15:39:23.560542+0000","last_peered":"2024-09-16T15:39:23.560542+0000","last_clean":"2024-09-16T15:39:23.560542+0000","last_became_active":"2024-09-16T15:39:22.559563+0000","last_became_peered":"2024-09-16T15:39:22.559563+0000","last_unstale":"2024-09-16T15:39:23.560542+0000","last_undegraded":"2024-09-16T15:39:23.560542+0000","last_fullsized":"2024-09-16T15:39:23.560542+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:17:35.609828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.562702+0000","last_change":"2024-09-16T15:39:21.556186+0000","last_active":"2024-09-16T15:39:23.562702+0000","last_peered":"2024-09-16T15:39:23.562702+0000","last_clean":"2024-09-16T15:39:23.562702+0000","last_became_active":"2024-09-16T15:39:21.555847+0000","last_became_peered":"2024-09-16T15:39:21.555847+0000","last_unstale":"2024-09-16T15:39:23.562702+0000","last_undegraded":"2024-09-16T15:39:23.562702+0000","last_fullsized":"2024-09-16T15:39:23.562702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:43:58.109261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360215+0000","last_change":"2024-09-16T15:39:22.581265+0000","last_active":"2024-09-16T15:39:24.360215+0000","last_peered":"2024-09-16T15:39:24.360215+0000","last_clean":"2024-09-16T15:39:24.360215+0000","last_became_active":"2024-09-16T15:39:22.580808+0000","last_became_peered":"2024-09-16T15:39:22.580808+0000","last_unstale":"2024-09-16T15:39:24.360215+0000","last_undegraded":"2024-09-16T15:39:24.360215+0000","last_fullsized":"2024-09-16T15:39:24.360215+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:40:04.044401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.362115+0000","last_change":"2024-09-16T15:39:21.565669+0000","last_active":"2024-09-16T15:39:24.362115+0000","last_peered":"2024-09-16T15:39:24.362115+0000","last_clean":"2024-09-16T15:39:24.362115+0000","last_became_active":"2024-09-16T15:39:21.564058+0000","last_became_peered":"2024-09-16T15:39:21.564058+0000","last_unstale":"2024-09-16T15:39:24.362115+0000","last_undegraded":"2024-09-16T15:39:24.362115+0000","last_fullsized":"2024-09-16T15:39:24.362115+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:36:55.905341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.362680+0000","last_change":"2024-09-16T15:39:22.586068+0000","last_active":"2024-09-16T15:39:24.362680+0000","last_peered":"2024-09-16T15:39:24.362680+0000","last_clean":"2024-09-16T15:39:24.362680+0000","last_became_active":"2024-09-16T15:39:22.585745+0000","last_became_peered":"2024-09-16T15:39:22.585745+0000","last_unstale":"2024-09-16T15:39:24.362680+0000","last_undegraded":"2024-09-16T15:39:24.362680+0000","last_fullsized":"2024-09-16T15:39:24.362680+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:46:44.196294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.562702+0000","last_change":"2024-09-16T15:39:21.564692+0000","last_active":"2024-09-16T15:39:23.562702+0000","last_peered":"2024-09-16T15:39:23.562702+0000","last_clean":"2024-09-16T15:39:23.562702+0000","last_became_active":"2024-09-16T15:39:21.564414+0000","last_became_peered":"2024-09-16T15:39:21.564414+0000","last_unstale":"2024-09-16T15:39:23.562702+0000","last_undegraded":"2024-09-16T15:39:23.562702+0000","last_fullsized":"2024-09-16T15:39:23.562702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:46:18.513631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.362100+0000","last_change":"2024-09-16T15:39:22.560891+0000","last_active":"2024-09-16T15:39:24.362100+0000","last_peered":"2024-09-16T15:39:24.362100+0000","last_clean":"2024-09-16T15:39:24.362100+0000","last_became_active":"2024-09-16T15:39:22.560598+0000","last_became_peered":"2024-09-16T15:39:22.560598+0000","last_unstale":"2024-09-16T15:39:24.362100+0000","last_undegraded":"2024-09-16T15:39:24.362100+0000","last_fullsized":"2024-09-16T15:39:24.362100+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:57:51.336962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.563158+0000","last_change":"2024-09-16T15:39:21.559845+0000","last_active":"2024-09-16T15:39:23.563158+0000","last_peered":"2024-09-16T15:39:23.563158+0000","last_clean":"2024-09-16T15:39:23.563158+0000","last_became_active":"2024-09-16T15:39:21.559089+0000","last_became_peered":"2024-09-16T15:39:21.559089+0000","last_unstale":"2024-09-16T15:39:23.563158+0000","last_undegraded":"2024-09-16T15:39:23.563158+0000","last_fullsized":"2024-09-16T15:39:23.563158+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:40:57.508356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361096+0000","last_change":"2024-09-16T15:39:22.575906+0000","last_active":"2024-09-16T15:39:24.361096+0000","last_peered":"2024-09-16T15:39:24.361096+0000","last_clean":"2024-09-16T15:39:24.361096+0000","last_became_active":"2024-09-16T15:39:22.574839+0000","last_became_peered":"2024-09-16T15:39:22.574839+0000","last_unstale":"2024-09-16T15:39:24.361096+0000","last_undegraded":"2024-09-16T15:39:24.361096+0000","last_fullsized":"2024-09-16T15:39:24.361096+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:13:11.232766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360712+0000","last_change":"2024-09-16T15:39:21.565376+0000","last_active":"2024-09-16T15:39:24.360712+0000","last_peered":"2024-09-16T15:39:24.360712+0000","last_clean":"2024-09-16T15:39:24.360712+0000","last_became_active":"2024-09-16T15:39:21.565007+0000","last_became_peered":"2024-09-16T15:39:21.565007+0000","last_unstale":"2024-09-16T15:39:24.360712+0000","last_undegraded":"2024-09-16T15:39:24.360712+0000","last_fullsized":"2024-09-16T15:39:24.360712+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:39:19.838119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361711+0000","last_change":"2024-09-16T15:39:22.565594+0000","last_active":"2024-09-16T15:39:24.361711+0000","last_peered":"2024-09-16T15:39:24.361711+0000","last_clean":"2024-09-16T15:39:24.361711+0000","last_became_active":"2024-09-16T15:39:22.565255+0000","last_became_peered":"2024-09-16T15:39:22.565255+0000","last_unstale":"2024-09-16T15:39:24.361711+0000","last_undegraded":"2024-09-16T15:39:24.361711+0000","last_fullsized":"2024-09-16T15:39:24.361711+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:32.663886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.560438+0000","last_change":"2024-09-16T15:39:12.698813+0000","last_active":"2024-09-16T15:39:23.560438+0000","last_peered":"2024-09-16T15:39:23.560438+0000","last_clean":"2024-09-16T15:39:23.560438+0000","last_became_active":"2024-09-16T15:39:12.697695+0000","last_became_peered":"2024-09-16T15:39:12.697695+0000","last_unstale":"2024-09-16T15:39:23.560438+0000","last_undegraded":"2024-09-16T15:39:23.560438+0000","last_fullsized":"2024-09-16T15:39:23.560438+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_clean_scrub_stamp":"2024-09-16T15:39:11.684967+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:02.061081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360791+0000","last_change":"2024-09-16T15:39:21.565360+0000","last_active":"2024-09-16T15:39:24.360791+0000","last_peered":"2024-09-16T15:39:24.360791+0000","last_clean":"2024-09-16T15:39:24.360791+0000","last_became_active":"2024-09-16T15:39:21.565114+0000","last_became_peered":"2024-09-16T15:39:21.565114+0000","last_unstale":"2024-09-16T15:39:24.360791+0000","last_undegraded":"2024-09-16T15:39:24.360791+0000","last_fullsized":"2024-09-16T15:39:24.360791+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:40:57.304044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.562889+0000","last_change":"2024-09-16T15:39:22.579056+0000","last_active":"2024-09-16T15:39:23.562889+0000","last_peered":"2024-09-16T15:39:23.562889+0000","last_clean":"2024-09-16T15:39:23.562889+0000","last_became_active":"2024-09-16T15:39:22.576443+0000","last_became_peered":"2024-09-16T15:39:22.576443+0000","last_unstale":"2024-09-16T15:39:23.562889+0000","last_undegraded":"2024-09-16T15:39:23.562889+0000","last_fullsized":"2024-09-16T15:39:23.562889+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:56:07.665096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360279+0000","last_change":"2024-09-16T15:39:21.568792+0000","last_active":"2024-09-16T15:39:24.360279+0000","last_peered":"2024-09-16T15:39:24.360279+0000","last_clean":"2024-09-16T15:39:24.360279+0000","last_became_active":"2024-09-16T15:39:21.568534+0000","last_became_peered":"2024-09-16T15:39:21.568534+0000","last_unstale":"2024-09-16T15:39:24.360279+0000","last_undegraded":"2024-09-16T15:39:24.360279+0000","last_fullsized":"2024-09-16T15:39:24.360279+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:16:01.495612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360315+0000","last_change":"2024-09-16T15:39:22.572844+0000","last_active":"2024-09-16T15:39:24.360315+0000","last_peered":"2024-09-16T15:39:24.360315+0000","last_clean":"2024-09-16T15:39:24.360315+0000","last_became_active":"2024-09-16T15:39:22.570080+0000","last_became_peered":"2024-09-16T15:39:22.570080+0000","last_unstale":"2024-09-16T15:39:24.360315+0000","last_undegraded":"2024-09-16T15:39:24.360315+0000","last_fullsized":"2024-09-16T15:39:24.360315+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:19:39.018320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.714355+0000","last_change":"2024-09-16T15:39:21.563752+0000","last_active":"2024-09-16T15:39:23.714355+0000","last_peered":"2024-09-16T15:39:23.714355+0000","last_clean":"2024-09-16T15:39:23.714355+0000","last_became_active":"2024-09-16T15:39:21.563430+0000","last_became_peered":"2024-09-16T15:39:21.563430+0000","last_unstale":"2024-09-16T15:39:23.714355+0000","last_undegraded":"2024-09-16T15:39:23.714355+0000","last_fullsized":"2024-09-16T15:39:23.714355+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:44:53.582663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.562026+0000","last_change":"2024-09-16T15:39:22.560753+0000","last_active":"2024-09-16T15:39:23.562026+0000","last_peered":"2024-09-16T15:39:23.562026+0000","last_clean":"2024-09-16T15:39:23.562026+0000","last_became_active":"2024-09-16T15:39:22.560416+0000","last_became_peered":"2024-09-16T15:39:22.560416+0000","last_unstale":"2024-09-16T15:39:23.562026+0000","last_undegraded":"2024-09-16T15:39:23.562026+0000","last_fullsized":"2024-09-16T15:39:23.562026+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:30:53.438955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361167+0000","last_change":"2024-09-16T15:39:21.568766+0000","last_active":"2024-09-16T15:39:24.361167+0000","last_peered":"2024-09-16T15:39:24.361167+0000","last_clean":"2024-09-16T15:39:24.361167+0000","last_became_active":"2024-09-16T15:39:21.568515+0000","last_became_peered":"2024-09-16T15:39:21.568515+0000","last_unstale":"2024-09-16T15:39:24.361167+0000","last_undegraded":"2024-09-16T15:39:24.361167+0000","last_fullsized":"2024-09-16T15:39:24.361167+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:29:10.288957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361313+0000","last_change":"2024-09-16T15:39:22.581762+0000","last_active":"2024-09-16T15:39:24.361313+0000","last_peered":"2024-09-16T15:39:24.361313+0000","last_clean":"2024-09-16T15:39:24.361313+0000","last_became_active":"2024-09-16T15:39:22.572639+0000","last_became_peered":"2024-09-16T15:39:22.572639+0000","last_unstale":"2024-09-16T15:39:24.361313+0000","last_undegraded":"2024-09-16T15:39:24.361313+0000","last_fullsized":"2024-09-16T15:39:24.361313+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:06:34.579134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.562849+0000","last_change":"2024-09-16T15:39:21.561768+0000","last_active":"2024-09-16T15:39:23.562849+0000","last_peered":"2024-09-16T15:39:23.562849+0000","last_clean":"2024-09-16T15:39:23.562849+0000","last_became_active":"2024-09-16T15:39:21.561188+0000","last_became_peered":"2024-09-16T15:39:21.561188+0000","last_unstale":"2024-09-16T15:39:23.562849+0000","last_undegraded":"2024-09-16T15:39:23.562849+0000","last_fullsized":"2024-09-16T15:39:23.562849+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:39:19.838119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.362299+0000","last_change":"2024-09-16T15:39:22.578044+0000","last_active":"2024-09-16T15:39:24.362299+0000","last_peered":"2024-09-16T15:39:24.362299+0000","last_clean":"2024-09-16T15:39:24.362299+0000","last_became_active":"2024-09-16T15:39:22.577752+0000","last_became_peered":"2024-09-16T15:39:22.577752+0000","last_unstale":"2024-09-16T15:39:24.362299+0000","last_undegraded":"2024-09-16T15:39:24.362299+0000","last_fullsized":"2024-09-16T15:39:24.362299+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:36:16.988647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361983+0000","last_change":"2024-09-16T15:39:21.560508+0000","last_active":"2024-09-16T15:39:24.361983+0000","last_peered":"2024-09-16T15:39:24.361983+0000","last_clean":"2024-09-16T15:39:24.361983+0000","last_became_active":"2024-09-16T15:39:21.560082+0000","last_became_peered":"2024-09-16T15:39:21.560082+0000","last_unstale":"2024-09-16T15:39:24.361983+0000","last_undegraded":"2024-09-16T15:39:24.361983+0000","last_fullsized":"2024-09-16T15:39:24.361983+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:17:34.601873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360646+0000","last_change":"2024-09-16T15:39:22.581816+0000","last_active":"2024-09-16T15:39:24.360646+0000","last_peered":"2024-09-16T15:39:24.360646+0000","last_clean":"2024-09-16T15:39:24.360646+0000","last_became_active":"2024-09-16T15:39:22.581523+0000","last_became_peered":"2024-09-16T15:39:22.581523+0000","last_unstale":"2024-09-16T15:39:24.360646+0000","last_undegraded":"2024-09-16T15:39:24.360646+0000","last_fullsized":"2024-09-16T15:39:24.360646+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:57:08.823794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.562963+0000","last_change":"2024-09-16T15:39:21.564493+0000","last_active":"2024-09-16T15:39:23.562963+0000","last_peered":"2024-09-16T15:39:23.562963+0000","last_clean":"2024-09-16T15:39:23.562963+0000","last_became_active":"2024-09-16T15:39:21.564207+0000","last_became_peered":"2024-09-16T15:39:21.564207+0000","last_unstale":"2024-09-16T15:39:23.562963+0000","last_undegraded":"2024-09-16T15:39:23.562963+0000","last_fullsized":"2024-09-16T15:39:23.562963+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:17:34.601873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.562454+0000","last_change":"2024-09-16T15:39:22.577107+0000","last_active":"2024-09-16T15:39:23.562454+0000","last_peered":"2024-09-16T15:39:23.562454+0000","last_clean":"2024-09-16T15:39:23.562454+0000","last_became_active":"2024-09-16T15:39:22.576840+0000","last_became_peered":"2024-09-16T15:39:22.576840+0000","last_unstale":"2024-09-16T15:39:23.562454+0000","last_undegraded":"2024-09-16T15:39:23.562454+0000","last_fullsized":"2024-09-16T15:39:23.562454+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:30:53.438955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361083+0000","last_change":"2024-09-16T15:39:21.570398+0000","last_active":"2024-09-16T15:39:24.361083+0000","last_peered":"2024-09-16T15:39:24.361083+0000","last_clean":"2024-09-16T15:39:24.361083+0000","last_became_active":"2024-09-16T15:39:21.570020+0000","last_became_peered":"2024-09-16T15:39:21.570020+0000","last_unstale":"2024-09-16T15:39:24.361083+0000","last_undegraded":"2024-09-16T15:39:24.361083+0000","last_fullsized":"2024-09-16T15:39:24.361083+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:19:38.010365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.563030+0000","last_change":"2024-09-16T15:39:22.572370+0000","last_active":"2024-09-16T15:39:23.563030+0000","last_peered":"2024-09-16T15:39:23.563030+0000","last_clean":"2024-09-16T15:39:23.563030+0000","last_became_active":"2024-09-16T15:39:22.572114+0000","last_became_peered":"2024-09-16T15:39:22.572114+0000","last_unstale":"2024-09-16T15:39:23.563030+0000","last_undegraded":"2024-09-16T15:39:23.563030+0000","last_fullsized":"2024-09-16T15:39:23.563030+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:22:34.502460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360218+0000","last_change":"2024-09-16T15:39:21.561676+0000","last_active":"2024-09-16T15:39:24.360218+0000","last_peered":"2024-09-16T15:39:24.360218+0000","last_clean":"2024-09-16T15:39:24.360218+0000","last_became_active":"2024-09-16T15:39:21.560798+0000","last_became_peered":"2024-09-16T15:39:21.560798+0000","last_unstale":"2024-09-16T15:39:24.360218+0000","last_undegraded":"2024-09-16T15:39:24.360218+0000","last_fullsized":"2024-09-16T15:39:24.360218+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:12:34.622210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360594+0000","last_change":"2024-09-16T15:39:22.567438+0000","last_active":"2024-09-16T15:39:24.360594+0000","last_peered":"2024-09-16T15:39:24.360594+0000","last_clean":"2024-09-16T15:39:24.360594+0000","last_became_active":"2024-09-16T15:39:22.567191+0000","last_became_peered":"2024-09-16T15:39:22.567191+0000","last_unstale":"2024-09-16T15:39:24.360594+0000","last_undegraded":"2024-09-16T15:39:24.360594+0000","last_fullsized":"2024-09-16T15:39:24.360594+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:42:25.111846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.561951+0000","last_change":"2024-09-16T15:39:21.566726+0000","last_active":"2024-09-16T15:39:23.561951+0000","last_peered":"2024-09-16T15:39:23.561951+0000","last_clean":"2024-09-16T15:39:23.561951+0000","last_became_active":"2024-09-16T15:39:21.566406+0000","last_became_peered":"2024-09-16T15:39:21.566406+0000","last_unstale":"2024-09-16T15:39:23.561951+0000","last_undegraded":"2024-09-16T15:39:23.561951+0000","last_fullsized":"2024-09-16T15:39:23.561951+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:40:57.508356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.561742+0000","last_change":"2024-09-16T15:39:22.575174+0000","last_active":"2024-09-16T15:39:23.561742+0000","last_peered":"2024-09-16T15:39:23.561742+0000","last_clean":"2024-09-16T15:39:23.561742+0000","last_became_active":"2024-09-16T15:39:22.574865+0000","last_became_peered":"2024-09-16T15:39:22.574865+0000","last_unstale":"2024-09-16T15:39:23.561742+0000","last_undegraded":"2024-09-16T15:39:23.561742+0000","last_fullsized":"2024-09-16T15:39:23.561742+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:10:23.402092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360483+0000","last_change":"2024-09-16T15:39:21.561384+0000","last_active":"2024-09-16T15:39:24.360483+0000","last_peered":"2024-09-16T15:39:24.360483+0000","last_clean":"2024-09-16T15:39:24.360483+0000","last_became_active":"2024-09-16T15:39:21.561078+0000","last_became_peered":"2024-09-16T15:39:21.561078+0000","last_unstale":"2024-09-16T15:39:24.360483+0000","last_undegraded":"2024-09-16T15:39:24.360483+0000","last_fullsized":"2024-09-16T15:39:24.360483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:10:40.788797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.714500+0000","last_change":"2024-09-16T15:39:22.576232+0000","last_active":"2024-09-16T15:39:23.714500+0000","last_peered":"2024-09-16T15:39:23.714500+0000","last_clean":"2024-09-16T15:39:23.714500+0000","last_became_active":"2024-09-16T15:39:22.575962+0000","last_became_peered":"2024-09-16T15:39:22.575962+0000","last_unstale":"2024-09-16T15:39:23.714500+0000","last_undegraded":"2024-09-16T15:39:23.714500+0000","last_fullsized":"2024-09-16T15:39:23.714500+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:43:58.162669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360704+0000","last_change":"2024-09-16T15:39:21.566471+0000","last_active":"2024-09-16T15:39:24.360704+0000","last_peered":"2024-09-16T15:39:24.360704+0000","last_clean":"2024-09-16T15:39:24.360704+0000","last_became_active":"2024-09-16T15:39:21.566220+0000","last_became_peered":"2024-09-16T15:39:21.566220+0000","last_unstale":"2024-09-16T15:39:24.360704+0000","last_undegraded":"2024-09-16T15:39:24.360704+0000","last_fullsized":"2024-09-16T15:39:24.360704+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:34:11.114212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.360894+0000","last_change":"2024-09-16T15:39:22.565878+0000","last_active":"2024-09-16T15:39:24.360894+0000","last_peered":"2024-09-16T15:39:24.360894+0000","last_clean":"2024-09-16T15:39:24.360894+0000","last_became_active":"2024-09-16T15:39:22.565599+0000","last_became_peered":"2024-09-16T15:39:22.565599+0000","last_unstale":"2024-09-16T15:39:24.360894+0000","last_undegraded":"2024-09-16T15:39:24.360894+0000","last_fullsized":"2024-09-16T15:39:24.360894+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:04:51.595786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361501+0000","last_change":"2024-09-16T15:39:22.563299+0000","last_active":"2024-09-16T15:39:24.361501+0000","last_peered":"2024-09-16T15:39:24.361501+0000","last_clean":"2024-09-16T15:39:24.361501+0000","last_became_active":"2024-09-16T15:39:22.560515+0000","last_became_peered":"2024-09-16T15:39:22.560515+0000","last_unstale":"2024-09-16T15:39:24.361501+0000","last_undegraded":"2024-09-16T15:39:24.361501+0000","last_fullsized":"2024-09-16T15:39:24.361501+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:45:21.853848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361156+0000","last_change":"2024-09-16T15:39:21.561779+0000","last_active":"2024-09-16T15:39:24.361156+0000","last_peered":"2024-09-16T15:39:24.361156+0000","last_clean":"2024-09-16T15:39:24.361156+0000","last_became_active":"2024-09-16T15:39:21.561471+0000","last_became_peered":"2024-09-16T15:39:21.561471+0000","last_unstale":"2024-09-16T15:39:24.361156+0000","last_undegraded":"2024-09-16T15:39:24.361156+0000","last_fullsized":"2024-09-16T15:39:24.361156+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:29:24.601704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.560906+0000","last_change":"2024-09-16T15:39:22.565324+0000","last_active":"2024-09-16T15:39:23.560906+0000","last_peered":"2024-09-16T15:39:23.560906+0000","last_clean":"2024-09-16T15:39:23.560906+0000","last_became_active":"2024-09-16T15:39:22.564275+0000","last_became_peered":"2024-09-16T15:39:22.564275+0000","last_unstale":"2024-09-16T15:39:23.560906+0000","last_undegraded":"2024-09-16T15:39:23.560906+0000","last_fullsized":"2024-09-16T15:39:23.560906+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:16:02.503567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:24.361469+0000","last_change":"2024-09-16T15:39:21.563348+0000","last_active":"2024-09-16T15:39:24.361469+0000","last_peered":"2024-09-16T15:39:24.361469+0000","last_clean":"2024-09-16T15:39:24.361469+0000","last_became_active":"2024-09-16T15:39:21.560699+0000","last_became_peered":"2024-09-16T15:39:21.560699+0000","last_unstale":"2024-09-16T15:39:24.361469+0000","last_undegraded":"2024-09-16T15:39:24.361469+0000","last_fullsized":"2024-09-16T15:39:24.361469+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:36:15.980692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.563361+0000","last_change":"2024-09-16T15:39:22.558509+0000","last_active":"2024-09-16T15:39:23.563361+0000","last_peered":"2024-09-16T15:39:23.563361+0000","last_clean":"2024-09-16T15:39:23.563361+0000","last_became_active":"2024-09-16T15:39:22.558197+0000","last_became_peered":"2024-09-16T15:39:22.558197+0000","last_unstale":"2024-09-16T15:39:23.563361+0000","last_undegraded":"2024-09-16T15:39:23.563361+0000","last_fullsized":"2024-09-16T15:39:23.563361+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:10:15.420114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.563141+0000","last_change":"2024-09-16T15:39:21.561287+0000","last_active":"2024-09-16T15:39:23.563141+0000","last_peered":"2024-09-16T15:39:23.563141+0000","last_clean":"2024-09-16T15:39:23.563141+0000","last_became_active":"2024-09-16T15:39:21.559935+0000","last_became_peered":"2024-09-16T15:39:21.559935+0000","last_unstale":"2024-09-16T15:39:23.563141+0000","last_undegraded":"2024-09-16T15:39:23.563141+0000","last_fullsized":"2024-09-16T15:39:23.563141+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:19:57.274084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.563208+0000","last_change":"2024-09-16T15:39:22.576005+0000","last_active":"2024-09-16T15:39:23.563208+0000","last_peered":"2024-09-16T15:39:23.563208+0000","last_clean":"2024-09-16T15:39:23.563208+0000","last_became_active":"2024-09-16T15:39:22.575434+0000","last_became_peered":"2024-09-16T15:39:22.575434+0000","last_unstale":"2024-09-16T15:39:23.563208+0000","last_undegraded":"2024-09-16T15:39:23.563208+0000","last_fullsized":"2024-09-16T15:39:23.563208+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:21.531232+0000","last_clean_scrub_stamp":"2024-09-16T15:39:21.531232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:03:51.340088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T15:39:23.562531+0000","last_change":"2024-09-16T15:39:21.568944+0000","last_active":"2024-09-16T15:39:23.562531+0000","last_peered":"2024-09-16T15:39:23.562531+0000","last_clean":"2024-09-16T15:39:23.562531+0000","last_became_active":"2024-09-16T15:39:21.568706+0000","last_became_peered":"2024-09-16T15:39:21.568706+0000","last_unstale":"2024-09-16T15:39:23.562531+0000","last_undegraded":"2024-09-16T15:39:23.562531+0000","last_fullsized":"2024-09-16T15:39:23.562531+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:20.523277+0000","last_clean_scrub_stamp":"2024-09-16T15:39:20.523277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:19:57.274084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509449,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":76284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509448,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":823296,"data_stored":666744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":241664,"data_stored":78568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":225280,"data_stored":76866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509447,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":233472,"data_stored":77896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509448,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":819200,"data_stored":666722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509446,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":76340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542154,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":76408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":27654604},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:39:37.281 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T15:39:37.281 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-16T15:39:37.281 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-16T15:39:37.281 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-16T15:39:37.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.472+0000 7f0e48000640 1 -- 172.21.15.151:0/3610953723 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0e4015cff0 msgr2=0x7f0e4006e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:37.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.472+0000 7f0e48000640 1 --2- 172.21.15.151:0/3610953723 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0e4015cff0 0x7f0e4006e6f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0e400b2680 tx=0x7f0e2c02a560 comp rx=0 tx=0).stop 2024-09-16T15:39:37.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.472+0000 7f0e48000640 1 -- 172.21.15.151:0/3610953723 shutdown_connections 2024-09-16T15:39:37.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.472+0000 7f0e48000640 1 --2- 172.21.15.151:0/3610953723 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0e4006f220 0x7f0e4006f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:37.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.472+0000 7f0e48000640 1 --2- 172.21.15.151:0/3610953723 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0e4015cff0 0x7f0e4006e6f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:37.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.472+0000 7f0e48000640 1 --2- 172.21.15.151:0/3610953723 >> v2:172.21.15.167:3300/0 conn(0x7f0e40159aa0 0x7f0e400741a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:37.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.472+0000 7f0e48000640 1 -- 172.21.15.151:0/3610953723 >> 172.21.15.151:0/3610953723 conn(0x7f0e400b18e0 msgr2=0x7f0e400b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:37.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.472+0000 7f0e48000640 1 -- 172.21.15.151:0/3610953723 shutdown_connections 2024-09-16T15:39:37.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.472+0000 7f0e48000640 1 -- 172.21.15.151:0/3610953723 wait complete. 2024-09-16T15:39:37.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.474+0000 7f0e48000640 1 Processor -- start 2024-09-16T15:39:37.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.474+0000 7f0e48000640 1 -- start start 2024-09-16T15:39:37.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.475+0000 7f0e48000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0e4006f220 0x7f0e401f5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:37.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.475+0000 7f0e48000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e40159aa0 0x7f0e401537b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:37.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.475+0000 7f0e48000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0e4015cff0 0x7f0e400cb460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:37.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.475+0000 7f0e48000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0e400cc310 con 0x7f0e4006f220 2024-09-16T15:39:37.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.475+0000 7f0e48000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0e400cc190 con 0x7f0e40159aa0 2024-09-16T15:39:37.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.475+0000 7f0e48000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f0e400cc490 con 0x7f0e4015cff0 2024-09-16T15:39:37.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.475+0000 7f0e45600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0e4006f220 0x7f0e401f5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:37.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.475+0000 7f0e45600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0e4006f220 0x7f0e401f5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:57444/0 (socket says 172.21.15.151:57444) 2024-09-16T15:39:37.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.475+0000 7f0e45600640 1 -- 172.21.15.151:0/2071655857 learned_addr learned my addr 172.21.15.151:0/2071655857 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:37.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.475+0000 7f0e44c00640 1 --2- 172.21.15.151:0/2071655857 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e40159aa0 0x7f0e401537b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:37.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.475+0000 7f0e46000640 1 --2- 172.21.15.151:0/2071655857 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0e4015cff0 0x7f0e400cb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:37.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.475+0000 7f0e45600640 1 -- 172.21.15.151:0/2071655857 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0e4015cff0 msgr2=0x7f0e400cb460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:37.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.476+0000 7f0e45600640 1 --2- 172.21.15.151:0/2071655857 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0e4015cff0 0x7f0e400cb460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:37.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.476+0000 7f0e45600640 1 -- 172.21.15.151:0/2071655857 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e40159aa0 msgr2=0x7f0e401537b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:37.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.476+0000 7f0e45600640 1 --2- 172.21.15.151:0/2071655857 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e40159aa0 0x7f0e401537b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:37.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.476+0000 7f0e45600640 1 -- 172.21.15.151:0/2071655857 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e400c9860 con 0x7f0e4006f220 2024-09-16T15:39:37.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.476+0000 7f0e46000640 1 --2- 172.21.15.151:0/2071655857 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0e4015cff0 0x7f0e400cb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:39:37.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.476+0000 7f0e45600640 1 --2- 172.21.15.151:0/2071655857 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0e4006f220 0x7f0e401f5180 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0e3400ebc0 tx=0x7f0e3400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:37.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.476+0000 7f0e3d600640 1 -- 172.21.15.151:0/2071655857 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e34010630 con 0x7f0e4006f220 2024-09-16T15:39:37.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.476+0000 7f0e3d600640 1 -- 172.21.15.151:0/2071655857 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e340108c0 con 0x7f0e4006f220 2024-09-16T15:39:37.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.476+0000 7f0e3d600640 1 -- 172.21.15.151:0/2071655857 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e34010d10 con 0x7f0e4006f220 2024-09-16T15:39:37.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.476+0000 7f0e48000640 1 -- 172.21.15.151:0/2071655857 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e400af310 con 0x7f0e4006f220 2024-09-16T15:39:37.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.476+0000 7f0e48000640 1 -- 172.21.15.151:0/2071655857 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e400c9b00 con 0x7f0e4006f220 2024-09-16T15:39:37.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.477+0000 7f0e48000640 1 -- 172.21.15.151:0/2071655857 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e40069360 con 0x7f0e4006f220 2024-09-16T15:39:37.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.479+0000 7f0e3d600640 1 -- 172.21.15.151:0/2071655857 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f0e34018420 con 0x7f0e4006f220 2024-09-16T15:39:37.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.480+0000 7f0e3d600640 1 --2- 172.21.15.151:0/2071655857 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0e20078850 0x7f0e2007ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:37.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.480+0000 7f0e3d600640 1 -- 172.21.15.151:0/2071655857 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f0e3409b630 con 0x7f0e4006f220 2024-09-16T15:39:37.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.480+0000 7f0e44c00640 1 --2- 172.21.15.151:0/2071655857 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0e20078850 0x7f0e2007ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:37.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.481+0000 7f0e44c00640 1 --2- 172.21.15.151:0/2071655857 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0e20078850 0x7f0e2007ad10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c033000 tx=0x7f0e2c004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:37.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.484+0000 7f0e3d600640 1 -- 172.21.15.151:0/2071655857 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0e340636e0 con 0x7f0e4006f220 2024-09-16T15:39:37.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.708+0000 7f0e48000640 1 -- 172.21.15.151:0/2071655857 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f0e400741a0 con 0x7f0e4006f220 2024-09-16T15:39:37.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.709+0000 7f0e3d600640 1 -- 172.21.15.151:0/2071655857 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f0e34068110 con 0x7f0e4006f220 2024-09-16T15:39:37.710 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:39:37.710 INFO:teuthology.orchestra.run.smithi151.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-16T15:39:37.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.711+0000 7f0e48000640 1 -- 172.21.15.151:0/2071655857 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0e20078850 msgr2=0x7f0e2007ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:37.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.711+0000 7f0e48000640 1 --2- 172.21.15.151:0/2071655857 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0e20078850 0x7f0e2007ad10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c033000 tx=0x7f0e2c004730 comp rx=0 tx=0).stop 2024-09-16T15:39:37.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.711+0000 7f0e48000640 1 -- 172.21.15.151:0/2071655857 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0e4006f220 msgr2=0x7f0e401f5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:37.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.711+0000 7f0e48000640 1 --2- 172.21.15.151:0/2071655857 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0e4006f220 0x7f0e401f5180 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0e3400ebc0 tx=0x7f0e3400c6a0 comp rx=0 tx=0).stop 2024-09-16T15:39:37.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.711+0000 7f0e48000640 1 -- 172.21.15.151:0/2071655857 shutdown_connections 2024-09-16T15:39:37.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.711+0000 7f0e48000640 1 --2- 172.21.15.151:0/2071655857 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0e20078850 0x7f0e2007ad10 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:37.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.711+0000 7f0e48000640 1 --2- 172.21.15.151:0/2071655857 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e40159aa0 0x7f0e401537b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:37.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.711+0000 7f0e48000640 1 --2- 172.21.15.151:0/2071655857 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0e4015cff0 0x7f0e400cb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:37.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.711+0000 7f0e48000640 1 --2- 172.21.15.151:0/2071655857 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0e4006f220 0x7f0e401f5180 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:37.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.711+0000 7f0e48000640 1 -- 172.21.15.151:0/2071655857 >> 172.21.15.151:0/2071655857 conn(0x7f0e400b18e0 msgr2=0x7f0e40154190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:37.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.711+0000 7f0e48000640 1 -- 172.21.15.151:0/2071655857 shutdown_connections 2024-09-16T15:39:37.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.711+0000 7f0e48000640 1 -- 172.21.15.151:0/2071655857 wait complete. 2024-09-16T15:39:37.728 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-16T15:39:37.728 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:39:37.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.883+0000 7f48b5000640 1 -- 172.21.15.151:0/2446044314 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f48b01612d0 msgr2=0x7f48b00b4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:37.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.883+0000 7f48b5000640 1 --2- 172.21.15.151:0/2446044314 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f48b01612d0 0x7f48b00b4e00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f48b00d1870 tx=0x7f48a002a4d0 comp rx=0 tx=0).stop 2024-09-16T15:39:37.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.883+0000 7f48b5000640 1 -- 172.21.15.151:0/2446044314 shutdown_connections 2024-09-16T15:39:37.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.883+0000 7f48b5000640 1 --2- 172.21.15.151:0/2446044314 >> v2:172.21.15.167:3300/0 conn(0x7f48b01616b0 0x7f48b00d2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:37.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.883+0000 7f48b5000640 1 --2- 172.21.15.151:0/2446044314 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f48b01612d0 0x7f48b00b4e00 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:37.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.883+0000 7f48b5000640 1 -- 172.21.15.151:0/2446044314 >> 172.21.15.151:0/2446044314 conn(0x7f48b00d3df0 msgr2=0x7f48b00d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:37.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.883+0000 7f48b5000640 1 -- 172.21.15.151:0/2446044314 shutdown_connections 2024-09-16T15:39:37.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.883+0000 7f48b5000640 1 -- 172.21.15.151:0/2446044314 wait complete. 2024-09-16T15:39:37.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.885+0000 7f48b5000640 1 Processor -- start 2024-09-16T15:39:37.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.885+0000 7f48b5000640 1 -- start start 2024-09-16T15:39:37.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.886+0000 7f48b5000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48b00d4980 0x7f48b014ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:37.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.886+0000 7f48b5000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f48b01612d0 0x7f48b01f4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:37.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.886+0000 7f48b5000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48b01616b0 0x7f48b00b8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:37.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.886+0000 7f48b5000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f48b00b1b60 con 0x7f48b01612d0 2024-09-16T15:39:37.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.886+0000 7f48b5000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f48b00b19e0 con 0x7f48b01616b0 2024-09-16T15:39:37.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.886+0000 7f48b5000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f48b00b1ce0 con 0x7f48b00d4980 2024-09-16T15:39:37.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.886+0000 7f48aea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f48b01612d0 0x7f48b01f4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:37.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.886+0000 7f48aea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f48b01612d0 0x7f48b01f4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:57460/0 (socket says 172.21.15.151:57460) 2024-09-16T15:39:37.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.886+0000 7f48aea00640 1 -- 172.21.15.151:0/3064987024 learned_addr learned my addr 172.21.15.151:0/3064987024 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:37.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.886+0000 7f48af400640 1 --2- 172.21.15.151:0/3064987024 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48b00d4980 0x7f48b014ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:37.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.886+0000 7f48afe00640 1 --2- 172.21.15.151:0/3064987024 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48b01616b0 0x7f48b00b8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:37.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.886+0000 7f48aea00640 1 -- 172.21.15.151:0/3064987024 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48b00d4980 msgr2=0x7f48b014ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:37.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.886+0000 7f48aea00640 1 --2- 172.21.15.151:0/3064987024 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48b00d4980 0x7f48b014ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:37.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.886+0000 7f48aea00640 1 -- 172.21.15.151:0/3064987024 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48b01616b0 msgr2=0x7f48b00b8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:37.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.886+0000 7f48aea00640 1 --2- 172.21.15.151:0/3064987024 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48b01616b0 0x7f48b00b8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:37.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.886+0000 7f48aea00640 1 -- 172.21.15.151:0/3064987024 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48b014bfb0 con 0x7f48b01612d0 2024-09-16T15:39:37.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.887+0000 7f48aea00640 1 --2- 172.21.15.151:0/3064987024 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f48b01612d0 0x7f48b01f4150 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f489c00c430 tx=0x7f489c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:37.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.887+0000 7f48a7400640 1 -- 172.21.15.151:0/3064987024 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f489c011790 con 0x7f48b01612d0 2024-09-16T15:39:37.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.887+0000 7f48a7400640 1 -- 172.21.15.151:0/3064987024 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f489c017070 con 0x7f48b01612d0 2024-09-16T15:39:37.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.887+0000 7f48a7400640 1 -- 172.21.15.151:0/3064987024 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f489c011d00 con 0x7f48b01612d0 2024-09-16T15:39:37.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.887+0000 7f48b5000640 1 -- 172.21.15.151:0/3064987024 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48b01f8dd0 con 0x7f48b01612d0 2024-09-16T15:39:37.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.887+0000 7f48b5000640 1 -- 172.21.15.151:0/3064987024 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48b01f8530 con 0x7f48b01612d0 2024-09-16T15:39:37.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.888+0000 7f48b5000640 1 -- 172.21.15.151:0/3064987024 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48b00b4e00 con 0x7f48b01612d0 2024-09-16T15:39:37.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.889+0000 7f48a7400640 1 -- 172.21.15.151:0/3064987024 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f489c01c460 con 0x7f48b01612d0 2024-09-16T15:39:37.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.890+0000 7f48a7400640 1 --2- 172.21.15.151:0/3064987024 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4890078850 0x7f489007ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:37.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.890+0000 7f48a7400640 1 -- 172.21.15.151:0/3064987024 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f489c09a760 con 0x7f48b01612d0 2024-09-16T15:39:37.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.890+0000 7f48af400640 1 --2- 172.21.15.151:0/3064987024 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4890078850 0x7f489007ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:37.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.891+0000 7f48af400640 1 --2- 172.21.15.151:0/3064987024 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4890078850 0x7f489007ad10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f48a0033000 tx=0x7f48a0004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:37.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:37.894+0000 7f48a7400640 1 -- 172.21.15.151:0/3064987024 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f48b014bfb0 con 0x7f48b01612d0 2024-09-16T15:39:38.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.115+0000 7f48b5000640 1 -- 172.21.15.151:0/3064987024 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f48b00ce3d0 con 0x7f48b01612d0 2024-09-16T15:39:38.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.116+0000 7f48a7400640 1 -- 172.21.15.151:0/3064987024 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5229 (secure 0 0 0) 0x7f489c062770 con 0x7f48b01612d0 2024-09-16T15:39:38.117 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:39:38.117 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":8,"btime":"2024-09-16T15:39:25:569070+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4301,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/2005239755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2005239755},{"type":"v1","addr":"172.21.15.167:6836","nonce":2005239755}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4338,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/3568704323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3568704323},{"type":"v1","addr":"172.21.15.151:6834","nonce":3568704323}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4341,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/3198287696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":3198287696},{"type":"v1","addr":"172.21.15.151:6835","nonce":3198287696}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:39:21.969407+0000","modified":"2024-09-16T15:39:24.983141+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4350},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4350":{"gid":4350,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6837/3014501393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3014501393},{"type":"v1","addr":"172.21.15.167:6837","nonce":3014501393}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4350,"qdb_cluster":[4350]},"id":1}]} 2024-09-16T15:39:38.117 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 8 2024-09-16T15:39:38.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.118+0000 7f48b5000640 1 -- 172.21.15.151:0/3064987024 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4890078850 msgr2=0x7f489007ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:38.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.118+0000 7f48b5000640 1 --2- 172.21.15.151:0/3064987024 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4890078850 0x7f489007ad10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f48a0033000 tx=0x7f48a0004730 comp rx=0 tx=0).stop 2024-09-16T15:39:38.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.118+0000 7f48b5000640 1 -- 172.21.15.151:0/3064987024 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f48b01612d0 msgr2=0x7f48b01f4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:38.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.118+0000 7f48b5000640 1 --2- 172.21.15.151:0/3064987024 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f48b01612d0 0x7f48b01f4150 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f489c00c430 tx=0x7f489c00c8f0 comp rx=0 tx=0).stop 2024-09-16T15:39:38.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.118+0000 7f48b5000640 1 -- 172.21.15.151:0/3064987024 shutdown_connections 2024-09-16T15:39:38.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.118+0000 7f48b5000640 1 --2- 172.21.15.151:0/3064987024 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4890078850 0x7f489007ad10 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:38.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.118+0000 7f48b5000640 1 --2- 172.21.15.151:0/3064987024 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48b01616b0 0x7f48b00b8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:38.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.118+0000 7f48b5000640 1 --2- 172.21.15.151:0/3064987024 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f48b01612d0 0x7f48b01f4150 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:38.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.118+0000 7f48b5000640 1 --2- 172.21.15.151:0/3064987024 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48b00d4980 0x7f48b014ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:38.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.118+0000 7f48b5000640 1 -- 172.21.15.151:0/3064987024 >> 172.21.15.151:0/3064987024 conn(0x7f48b00d3df0 msgr2=0x7f48b00cec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:38.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.119+0000 7f48b5000640 1 -- 172.21.15.151:0/3064987024 shutdown_connections 2024-09-16T15:39:38.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.119+0000 7f48b5000640 1 -- 172.21.15.151:0/3064987024 wait complete. 2024-09-16T15:39:38.136 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:39:38.293 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.291+0000 7fc017200640 1 -- 172.21.15.151:0/702296887 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc01015f5a0 msgr2=0x7fc0100b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:38.293 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.291+0000 7fc017200640 1 --2- 172.21.15.151:0/702296887 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc01015f5a0 0x7fc0100b2440 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc000009830 tx=0x7fc00002a480 comp rx=0 tx=0).stop 2024-09-16T15:39:38.293 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.291+0000 7fc017200640 1 -- 172.21.15.151:0/702296887 shutdown_connections 2024-09-16T15:39:38.293 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.291+0000 7fc017200640 1 --2- 172.21.15.151:0/702296887 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc01015f5a0 0x7fc0100b2440 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:38.293 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.291+0000 7fc017200640 1 -- 172.21.15.151:0/702296887 >> 172.21.15.151:0/702296887 conn(0x7fc0100b1950 msgr2=0x7fc0100b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:38.293 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.291+0000 7fc017200640 1 -- 172.21.15.151:0/702296887 shutdown_connections 2024-09-16T15:39:38.293 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.291+0000 7fc017200640 1 -- 172.21.15.151:0/702296887 wait complete. 2024-09-16T15:39:38.294 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.293+0000 7fc017200640 1 Processor -- start 2024-09-16T15:39:38.294 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.293+0000 7fc017200640 1 -- start start 2024-09-16T15:39:38.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.294+0000 7fc017200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc010073fb0 0x7fc0101e8880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:38.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.294+0000 7fc017200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0100ccf00 0x7fc010147d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:38.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.294+0000 7fc017200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc01015f5a0 0x7fc0101482d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:38.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.294+0000 7fc017200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc01006f9f0 con 0x7fc01015f5a0 2024-09-16T15:39:38.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.294+0000 7fc017200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc01006f870 con 0x7fc0100ccf00 2024-09-16T15:39:38.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.294+0000 7fc017200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fc010074420 con 0x7fc010073fb0 2024-09-16T15:39:38.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.294+0000 7fc015200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc01015f5a0 0x7fc0101482d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:38.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.294+0000 7fc015200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc01015f5a0 0x7fc0101482d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:57470/0 (socket says 172.21.15.151:57470) 2024-09-16T15:39:38.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.294+0000 7fc015200640 1 -- 172.21.15.151:0/1670218174 learned_addr learned my addr 172.21.15.151:0/1670218174 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:38.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.295+0000 7fc00fe00640 1 --2- 172.21.15.151:0/1670218174 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc010073fb0 0x7fc0101e8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:38.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.295+0000 7fc00f400640 1 --2- 172.21.15.151:0/1670218174 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0100ccf00 0x7fc010147d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:38.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.295+0000 7fc015200640 1 -- 172.21.15.151:0/1670218174 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc010073fb0 msgr2=0x7fc0101e8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:38.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.295+0000 7fc015200640 1 --2- 172.21.15.151:0/1670218174 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc010073fb0 0x7fc0101e8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:38.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.295+0000 7fc015200640 1 -- 172.21.15.151:0/1670218174 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0100ccf00 msgr2=0x7fc010147d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:38.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.295+0000 7fc015200640 1 --2- 172.21.15.151:0/1670218174 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0100ccf00 0x7fc010147d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:38.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.295+0000 7fc015200640 1 -- 172.21.15.151:0/1670218174 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0101e48b0 con 0x7fc01015f5a0 2024-09-16T15:39:38.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.295+0000 7fc00fe00640 1 --2- 172.21.15.151:0/1670218174 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc010073fb0 0x7fc0101e8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:39:38.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.295+0000 7fc00f400640 1 --2- 172.21.15.151:0/1670218174 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0100ccf00 0x7fc010147d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:39:38.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.295+0000 7fc015200640 1 --2- 172.21.15.151:0/1670218174 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc01015f5a0 0x7fc0101482d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc00800cbc0 tx=0x7fc00800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:38.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.295+0000 7fc007e00640 1 -- 172.21.15.151:0/1670218174 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc008016020 con 0x7fc01015f5a0 2024-09-16T15:39:38.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.295+0000 7fc007e00640 1 -- 172.21.15.151:0/1670218174 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc00800ae30 con 0x7fc01015f5a0 2024-09-16T15:39:38.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.295+0000 7fc007e00640 1 -- 172.21.15.151:0/1670218174 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc008012660 con 0x7fc01015f5a0 2024-09-16T15:39:38.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.295+0000 7fc017200640 1 -- 172.21.15.151:0/1670218174 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0101e4b90 con 0x7fc01015f5a0 2024-09-16T15:39:38.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.295+0000 7fc017200640 1 -- 172.21.15.151:0/1670218174 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0100ce2f0 con 0x7fc01015f5a0 2024-09-16T15:39:38.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.296+0000 7fc017200640 1 -- 172.21.15.151:0/1670218174 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0100d06d0 con 0x7fc01015f5a0 2024-09-16T15:39:38.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.297+0000 7fc007e00640 1 -- 172.21.15.151:0/1670218174 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fc008019020 con 0x7fc01015f5a0 2024-09-16T15:39:38.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.298+0000 7fc007e00640 1 --2- 172.21.15.151:0/1670218174 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbff0078850 0x7fbff007ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:38.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.298+0000 7fc007e00640 1 -- 172.21.15.151:0/1670218174 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fc00809ad60 con 0x7fc01015f5a0 2024-09-16T15:39:38.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.299+0000 7fc00fe00640 1 --2- 172.21.15.151:0/1670218174 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbff0078850 0x7fbff007ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:38.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.299+0000 7fc00fe00640 1 --2- 172.21.15.151:0/1670218174 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbff0078850 0x7fbff007ad10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc000034000 tx=0x7fc000004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:38.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.302+0000 7fc007e00640 1 -- 172.21.15.151:0/1670218174 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc0100d06d0 con 0x7fc01015f5a0 2024-09-16T15:39:38.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.523+0000 7fc017200640 1 -- 172.21.15.151:0/1670218174 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc0100b2440 con 0x7fc01015f5a0 2024-09-16T15:39:38.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.524+0000 7fc007e00640 1 -- 172.21.15.151:0/1670218174 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5229 (secure 0 0 0) 0x7fc008020030 con 0x7fc01015f5a0 2024-09-16T15:39:38.525 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:39:38.525 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":8,"btime":"2024-09-16T15:39:25:569070+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4301,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/2005239755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2005239755},{"type":"v1","addr":"172.21.15.167:6836","nonce":2005239755}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4338,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/3568704323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3568704323},{"type":"v1","addr":"172.21.15.151:6834","nonce":3568704323}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4341,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/3198287696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":3198287696},{"type":"v1","addr":"172.21.15.151:6835","nonce":3198287696}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:39:21.969407+0000","modified":"2024-09-16T15:39:24.983141+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4350},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4350":{"gid":4350,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6837/3014501393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3014501393},{"type":"v1","addr":"172.21.15.167:6837","nonce":3014501393}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4350,"qdb_cluster":[4350]},"id":1}]} 2024-09-16T15:39:38.525 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 8 2024-09-16T15:39:38.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.526+0000 7fc017200640 1 -- 172.21.15.151:0/1670218174 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbff0078850 msgr2=0x7fbff007ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:38.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.526+0000 7fc017200640 1 --2- 172.21.15.151:0/1670218174 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbff0078850 0x7fbff007ad10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc000034000 tx=0x7fc000004730 comp rx=0 tx=0).stop 2024-09-16T15:39:38.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.526+0000 7fc017200640 1 -- 172.21.15.151:0/1670218174 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc01015f5a0 msgr2=0x7fc0101482d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:38.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.526+0000 7fc017200640 1 --2- 172.21.15.151:0/1670218174 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc01015f5a0 0x7fc0101482d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc00800cbc0 tx=0x7fc00800a430 comp rx=0 tx=0).stop 2024-09-16T15:39:38.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.526+0000 7fc017200640 1 -- 172.21.15.151:0/1670218174 shutdown_connections 2024-09-16T15:39:38.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.526+0000 7fc017200640 1 --2- 172.21.15.151:0/1670218174 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbff0078850 0x7fbff007ad10 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:38.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.526+0000 7fc017200640 1 --2- 172.21.15.151:0/1670218174 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0100ccf00 0x7fc010147d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:38.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.526+0000 7fc017200640 1 --2- 172.21.15.151:0/1670218174 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc01015f5a0 0x7fc0101482d0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:38.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.526+0000 7fc017200640 1 --2- 172.21.15.151:0/1670218174 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc010073fb0 0x7fc0101e8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:38.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.526+0000 7fc017200640 1 -- 172.21.15.151:0/1670218174 >> 172.21.15.151:0/1670218174 conn(0x7fc0100b1950 msgr2=0x7fc0100b3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:38.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.526+0000 7fc017200640 1 -- 172.21.15.151:0/1670218174 shutdown_connections 2024-09-16T15:39:38.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.526+0000 7fc017200640 1 -- 172.21.15.151:0/1670218174 wait complete. 2024-09-16T15:39:38.543 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:39:38.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.702+0000 7fc50e400640 1 -- 172.21.15.151:0/3130840420 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc508073fb0 msgr2=0x7fc508074460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:38.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.702+0000 7fc50e400640 1 --2- 172.21.15.151:0/3130840420 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc508073fb0 0x7fc508074460 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc500005fd0 tx=0x7fc50002a820 comp rx=0 tx=0).stop 2024-09-16T15:39:38.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.702+0000 7fc50e400640 1 -- 172.21.15.151:0/3130840420 shutdown_connections 2024-09-16T15:39:38.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.702+0000 7fc50e400640 1 --2- 172.21.15.151:0/3130840420 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc508073fb0 0x7fc508074460 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:38.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.702+0000 7fc50e400640 1 --2- 172.21.15.151:0/3130840420 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc50815f5a0 0x7fc5080b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:38.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.702+0000 7fc50e400640 1 -- 172.21.15.151:0/3130840420 >> 172.21.15.151:0/3130840420 conn(0x7fc5080b1950 msgr2=0x7fc5080b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:38.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.702+0000 7fc50e400640 1 -- 172.21.15.151:0/3130840420 shutdown_connections 2024-09-16T15:39:38.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.702+0000 7fc50e400640 1 -- 172.21.15.151:0/3130840420 wait complete. 2024-09-16T15:39:38.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.704+0000 7fc50e400640 1 Processor -- start 2024-09-16T15:39:38.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.704+0000 7fc50e400640 1 -- start start 2024-09-16T15:39:38.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.705+0000 7fc50e400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc508073fb0 0x7fc5081f64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:38.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.705+0000 7fc50e400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5080ccf00 0x7fc508144ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:38.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.705+0000 7fc50e400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc50815f5a0 0x7fc508154aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:38.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.705+0000 7fc50e400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc50806f2e0 con 0x7fc50815f5a0 2024-09-16T15:39:38.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.705+0000 7fc50e400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc50806f160 con 0x7fc5080ccf00 2024-09-16T15:39:38.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.705+0000 7fc50e400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fc50806f460 con 0x7fc508073fb0 2024-09-16T15:39:38.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.705+0000 7fc507e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc50815f5a0 0x7fc508154aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:38.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.705+0000 7fc507e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc50815f5a0 0x7fc508154aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:57484/0 (socket says 172.21.15.151:57484) 2024-09-16T15:39:38.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.705+0000 7fc507e00640 1 -- 172.21.15.151:0/3735075209 learned_addr learned my addr 172.21.15.151:0/3735075209 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:38.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.706+0000 7fc506a00640 1 --2- 172.21.15.151:0/3735075209 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5080ccf00 0x7fc508144ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:38.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.706+0000 7fc507400640 1 --2- 172.21.15.151:0/3735075209 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc508073fb0 0x7fc5081f64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:38.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.706+0000 7fc507e00640 1 -- 172.21.15.151:0/3735075209 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc508073fb0 msgr2=0x7fc5081f64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:38.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.706+0000 7fc507e00640 1 --2- 172.21.15.151:0/3735075209 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc508073fb0 0x7fc5081f64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:38.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.706+0000 7fc507e00640 1 -- 172.21.15.151:0/3735075209 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5080ccf00 msgr2=0x7fc508144ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:38.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.706+0000 7fc507e00640 1 --2- 172.21.15.151:0/3735075209 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5080ccf00 0x7fc508144ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:38.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.706+0000 7fc507e00640 1 -- 172.21.15.151:0/3735075209 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc508145de0 con 0x7fc50815f5a0 2024-09-16T15:39:38.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.706+0000 7fc507e00640 1 --2- 172.21.15.151:0/3735075209 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc50815f5a0 0x7fc508154aa0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc500006010 tx=0x7fc500032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:38.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.706+0000 7fc4ff400640 1 -- 172.21.15.151:0/3735075209 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc500037040 con 0x7fc50815f5a0 2024-09-16T15:39:38.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.706+0000 7fc4ff400640 1 -- 172.21.15.151:0/3735075209 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc500042070 con 0x7fc50815f5a0 2024-09-16T15:39:38.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.706+0000 7fc4ff400640 1 -- 172.21.15.151:0/3735075209 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5000042a0 con 0x7fc50815f5a0 2024-09-16T15:39:38.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.706+0000 7fc50e400640 1 -- 172.21.15.151:0/3735075209 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc508145820 con 0x7fc50815f5a0 2024-09-16T15:39:38.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.707+0000 7fc50e400640 1 -- 172.21.15.151:0/3735075209 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5081f5ca0 con 0x7fc50815f5a0 2024-09-16T15:39:38.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.708+0000 7fc50e400640 1 -- 172.21.15.151:0/3735075209 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc50806f9f0 con 0x7fc50815f5a0 2024-09-16T15:39:38.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.709+0000 7fc4ff400640 1 -- 172.21.15.151:0/3735075209 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fc500004440 con 0x7fc50815f5a0 2024-09-16T15:39:38.711 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.710+0000 7fc4ff400640 1 --2- 172.21.15.151:0/3735075209 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc4e8078850 0x7fc4e807ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:38.711 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.710+0000 7fc4ff400640 1 -- 172.21.15.151:0/3735075209 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fc5000bb800 con 0x7fc50815f5a0 2024-09-16T15:39:38.711 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.710+0000 7fc507400640 1 --2- 172.21.15.151:0/3735075209 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc4e8078850 0x7fc4e807ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:38.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.711+0000 7fc507400640 1 --2- 172.21.15.151:0/3735075209 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc4e8078850 0x7fc4e807ad10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc4f800dae0 tx=0x7fc4f8005e30 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:38.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.714+0000 7fc4ff400640 1 -- 172.21.15.151:0/3735075209 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc500083dd0 con 0x7fc50815f5a0 2024-09-16T15:39:38.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.900+0000 7fc50e400640 1 -- 172.21.15.151:0/3735075209 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc5080ad070 con 0x7fc50815f5a0 2024-09-16T15:39:38.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.901+0000 7fc4ff400640 1 -- 172.21.15.151:0/3735075209 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16729 (secure 0 0 0) 0x7fc5000836d0 con 0x7fc50815f5a0 2024-09-16T15:39:38.903 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:39:38.903 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":22,"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","created":"2024-09-16T15:38:56.348535+0000","modified":"2024-09-16T15:39:23.550834+0000","last_up_change":"2024-09-16T15:39:10.681384+0000","last_in_change":"2024-09-16T15:39:00.544477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:39:11.602252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:39:19.970182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T15:39:20.929376+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5524076-a903-4d0a-af6a-caf474c4765d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6809","nonce":3111342741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6811","nonce":3111342741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6815","nonce":3111342741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6813","nonce":3111342741}]},"public_addr":"172.21.15.151:6809/3111342741","cluster_addr":"172.21.15.151:6811/3111342741","heartbeat_back_addr":"172.21.15.151:6815/3111342741","heartbeat_front_addr":"172.21.15.151:6813/3111342741","state":["exists","up"]},{"osd":1,"uuid":"dd25925f-0ae1-494e-8529-4dc205759db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6801","nonce":3368304087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6803","nonce":3368304087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6807","nonce":3368304087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6805","nonce":3368304087}]},"public_addr":"172.21.15.151:6801/3368304087","cluster_addr":"172.21.15.151:6803/3368304087","heartbeat_back_addr":"172.21.15.151:6807/3368304087","heartbeat_front_addr":"172.21.15.151:6805/3368304087","state":["exists","up"]},{"osd":2,"uuid":"c2f18115-013d-4da6-a0fe-cb978f4494b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6823","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6825","nonce":3079373271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6827","nonce":3079373271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6831","nonce":3079373271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6829","nonce":3079373271}]},"public_addr":"172.21.15.151:6825/3079373271","cluster_addr":"172.21.15.151:6827/3079373271","heartbeat_back_addr":"172.21.15.151:6831/3079373271","heartbeat_front_addr":"172.21.15.151:6829/3079373271","state":["exists","up"]},{"osd":3,"uuid":"36edb718-2fcb-4d88-8ddc-c321eed4279d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6817","nonce":1142509041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6819","nonce":1142509041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6824","nonce":1142509041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6821","nonce":1142509041}]},"public_addr":"172.21.15.151:6817/1142509041","cluster_addr":"172.21.15.151:6819/1142509041","heartbeat_back_addr":"172.21.15.151:6824/1142509041","heartbeat_front_addr":"172.21.15.151:6821/1142509041","state":["exists","up"]},{"osd":4,"uuid":"64d33708-cd0a-4236-abe3-1b646e82ab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6811","nonce":2735495271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6813","nonce":2735495271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6817","nonce":2735495271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6815","nonce":2735495271}]},"public_addr":"172.21.15.167:6811/2735495271","cluster_addr":"172.21.15.167:6813/2735495271","heartbeat_back_addr":"172.21.15.167:6817/2735495271","heartbeat_front_addr":"172.21.15.167:6815/2735495271","state":["exists","up"]},{"osd":5,"uuid":"f14b44a9-5910-458c-b518-09a8d0de94de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6827","nonce":1494916572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6829","nonce":1494916572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6833","nonce":1494916572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6831","nonce":1494916572}]},"public_addr":"172.21.15.167:6827/1494916572","cluster_addr":"172.21.15.167:6829/1494916572","heartbeat_back_addr":"172.21.15.167:6833/1494916572","heartbeat_front_addr":"172.21.15.167:6831/1494916572","state":["exists","up"]},{"osd":6,"uuid":"39335545-2654-4710-84b0-a9aa47cfcbba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6819","nonce":2538273273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6821","nonce":2538273273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6825","nonce":2538273273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6823","nonce":2538273273}]},"public_addr":"172.21.15.167:6819/2538273273","cluster_addr":"172.21.15.167:6821/2538273273","heartbeat_back_addr":"172.21.15.167:6825/2538273273","heartbeat_front_addr":"172.21.15.167:6823/2538273273","state":["exists","up"]},{"osd":7,"uuid":"4f8bfc81-53dd-4339-ac22-1a4007d9f963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6803","nonce":265072227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6805","nonce":265072227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6809","nonce":265072227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6807","nonce":265072227}]},"public_addr":"172.21.15.167:6803/265072227","cluster_addr":"172.21.15.167:6805/265072227","heartbeat_back_addr":"172.21.15.167:6809/265072227","heartbeat_front_addr":"172.21.15.167:6807/265072227","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.150359+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.084358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.722368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.332809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.320709+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.395153+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.935250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.801443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:39:38.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.904+0000 7fc50e400640 1 -- 172.21.15.151:0/3735075209 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc4e8078850 msgr2=0x7fc4e807ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:38.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.904+0000 7fc50e400640 1 --2- 172.21.15.151:0/3735075209 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc4e8078850 0x7fc4e807ad10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc4f800dae0 tx=0x7fc4f8005e30 comp rx=0 tx=0).stop 2024-09-16T15:39:38.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.904+0000 7fc50e400640 1 -- 172.21.15.151:0/3735075209 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc50815f5a0 msgr2=0x7fc508154aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:38.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.904+0000 7fc50e400640 1 --2- 172.21.15.151:0/3735075209 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc50815f5a0 0x7fc508154aa0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc500006010 tx=0x7fc500032d10 comp rx=0 tx=0).stop 2024-09-16T15:39:38.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.904+0000 7fc50e400640 1 -- 172.21.15.151:0/3735075209 shutdown_connections 2024-09-16T15:39:38.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.904+0000 7fc50e400640 1 --2- 172.21.15.151:0/3735075209 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc4e8078850 0x7fc4e807ad10 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:38.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.904+0000 7fc50e400640 1 --2- 172.21.15.151:0/3735075209 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5080ccf00 0x7fc508144ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:38.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.904+0000 7fc50e400640 1 --2- 172.21.15.151:0/3735075209 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc50815f5a0 0x7fc508154aa0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:38.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.904+0000 7fc50e400640 1 --2- 172.21.15.151:0/3735075209 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc508073fb0 0x7fc5081f64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:38.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.904+0000 7fc50e400640 1 -- 172.21.15.151:0/3735075209 >> 172.21.15.151:0/3735075209 conn(0x7fc5080b1950 msgr2=0x7fc50809ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:38.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.904+0000 7fc50e400640 1 -- 172.21.15.151:0/3735075209 shutdown_connections 2024-09-16T15:39:38.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:38.904+0000 7fc50e400640 1 -- 172.21.15.151:0/3735075209 wait complete. 2024-09-16T15:39:38.921 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:39:39.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.077+0000 7fcea7200640 1 -- 172.21.15.151:0/355255291 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcea00d0540 msgr2=0x7fcea00b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:39.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.077+0000 7fcea7200640 1 --2- 172.21.15.151:0/355255291 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcea00d0540 0x7fcea00b28f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fce9c005fd0 tx=0x7fce9c02a940 comp rx=0 tx=0).stop 2024-09-16T15:39:39.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.077+0000 7fcea7200640 1 -- 172.21.15.151:0/355255291 shutdown_connections 2024-09-16T15:39:39.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.077+0000 7fcea7200640 1 --2- 172.21.15.151:0/355255291 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcea00d0540 0x7fcea00b28f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:39.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.077+0000 7fcea7200640 1 -- 172.21.15.151:0/355255291 >> 172.21.15.151:0/355255291 conn(0x7fcea00cfa20 msgr2=0x7fcea00cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:39.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.077+0000 7fcea7200640 1 -- 172.21.15.151:0/355255291 shutdown_connections 2024-09-16T15:39:39.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.077+0000 7fcea7200640 1 -- 172.21.15.151:0/355255291 wait complete. 2024-09-16T15:39:39.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.079+0000 7fcea7200640 1 Processor -- start 2024-09-16T15:39:39.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.079+0000 7fcea7200640 1 -- start start 2024-09-16T15:39:39.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.080+0000 7fcea7200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcea00d0540 0x7fcea00ce660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:39.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.080+0000 7fcea7200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcea015bc30 0x7fcea0154670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:39.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.080+0000 7fcea7200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcea0156670 0x7fcea0150b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:39.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.080+0000 7fcea7200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcea0074660 con 0x7fcea015bc30 2024-09-16T15:39:39.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.080+0000 7fcea7200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcea006fa30 con 0x7fcea00d0540 2024-09-16T15:39:39.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.080+0000 7fcea7200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fcea00ceba0 con 0x7fcea0156670 2024-09-16T15:39:39.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.080+0000 7fcea5400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcea015bc30 0x7fcea0154670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:39.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.080+0000 7fcea5400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcea015bc30 0x7fcea0154670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:57492/0 (socket says 172.21.15.151:57492) 2024-09-16T15:39:39.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.080+0000 7fcea5400640 1 -- 172.21.15.151:0/1706553931 learned_addr learned my addr 172.21.15.151:0/1706553931 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:39.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.080+0000 7fcea5e00640 1 --2- 172.21.15.151:0/1706553931 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcea00d0540 0x7fcea00ce660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:39.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.080+0000 7fcea6800640 1 --2- 172.21.15.151:0/1706553931 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcea0156670 0x7fcea0150b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:39.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.081+0000 7fcea5400640 1 -- 172.21.15.151:0/1706553931 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcea0156670 msgr2=0x7fcea0150b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:39.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.081+0000 7fcea5400640 1 --2- 172.21.15.151:0/1706553931 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcea0156670 0x7fcea0150b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:39.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.081+0000 7fcea5400640 1 -- 172.21.15.151:0/1706553931 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcea00d0540 msgr2=0x7fcea00ce660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:39.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.081+0000 7fcea5400640 1 --2- 172.21.15.151:0/1706553931 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcea00d0540 0x7fcea00ce660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:39.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.081+0000 7fcea5400640 1 -- 172.21.15.151:0/1706553931 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcea00cedd0 con 0x7fcea015bc30 2024-09-16T15:39:39.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.081+0000 7fcea5400640 1 --2- 172.21.15.151:0/1706553931 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcea015bc30 0x7fcea0154670 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fce9000c400 tx=0x7fce9000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:39.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.081+0000 7fce9a000640 1 -- 172.21.15.151:0/1706553931 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce90011790 con 0x7fcea015bc30 2024-09-16T15:39:39.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.081+0000 7fce9a000640 1 -- 172.21.15.151:0/1706553931 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fce90017070 con 0x7fcea015bc30 2024-09-16T15:39:39.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.081+0000 7fce9a000640 1 -- 172.21.15.151:0/1706553931 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce90011d00 con 0x7fcea015bc30 2024-09-16T15:39:39.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.081+0000 7fcea7200640 1 -- 172.21.15.151:0/1706553931 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcea00cf0b0 con 0x7fcea015bc30 2024-09-16T15:39:39.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.081+0000 7fcea7200640 1 -- 172.21.15.151:0/1706553931 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcea00c7ce0 con 0x7fcea015bc30 2024-09-16T15:39:39.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.082+0000 7fcea7200640 1 -- 172.21.15.151:0/1706553931 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcea00cf5d0 con 0x7fcea015bc30 2024-09-16T15:39:39.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.084+0000 7fce9a000640 1 -- 172.21.15.151:0/1706553931 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fce9001c460 con 0x7fcea015bc30 2024-09-16T15:39:39.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.084+0000 7fce9a000640 1 --2- 172.21.15.151:0/1706553931 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fce84078850 0x7fce8407ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:39.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.085+0000 7fce9a000640 1 -- 172.21.15.151:0/1706553931 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fce9009a670 con 0x7fcea015bc30 2024-09-16T15:39:39.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.085+0000 7fcea5e00640 1 --2- 172.21.15.151:0/1706553931 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fce84078850 0x7fce8407ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:39.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.086+0000 7fcea5e00640 1 --2- 172.21.15.151:0/1706553931 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fce84078850 0x7fce8407ad10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fce94000c00 tx=0x7fce94009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:39.090 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.089+0000 7fce9a000640 1 -- 172.21.15.151:0/1706553931 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fcea00cf5d0 con 0x7fcea015bc30 2024-09-16T15:39:39.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.309+0000 7fcea7200640 1 -- 172.21.15.151:0/1706553931 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcea00b28f0 con 0x7fcea015bc30 2024-09-16T15:39:39.311 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.309+0000 7fce9a000640 1 -- 172.21.15.151:0/1706553931 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5229 (secure 0 0 0) 0x7fce90062600 con 0x7fcea015bc30 2024-09-16T15:39:39.311 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:39:39.311 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":8,"btime":"2024-09-16T15:39:25:569070+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4301,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/2005239755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2005239755},{"type":"v1","addr":"172.21.15.167:6836","nonce":2005239755}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4338,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/3568704323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3568704323},{"type":"v1","addr":"172.21.15.151:6834","nonce":3568704323}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4341,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/3198287696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":3198287696},{"type":"v1","addr":"172.21.15.151:6835","nonce":3198287696}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:39:21.969407+0000","modified":"2024-09-16T15:39:24.983141+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4350},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4350":{"gid":4350,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6837/3014501393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3014501393},{"type":"v1","addr":"172.21.15.167:6837","nonce":3014501393}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4350,"qdb_cluster":[4350]},"id":1}]} 2024-09-16T15:39:39.311 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 8 2024-09-16T15:39:39.313 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.312+0000 7fcea7200640 1 -- 172.21.15.151:0/1706553931 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fce84078850 msgr2=0x7fce8407ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:39.313 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.312+0000 7fcea7200640 1 --2- 172.21.15.151:0/1706553931 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fce84078850 0x7fce8407ad10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fce94000c00 tx=0x7fce94009040 comp rx=0 tx=0).stop 2024-09-16T15:39:39.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.312+0000 7fcea7200640 1 -- 172.21.15.151:0/1706553931 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcea015bc30 msgr2=0x7fcea0154670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:39.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.312+0000 7fcea7200640 1 --2- 172.21.15.151:0/1706553931 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcea015bc30 0x7fcea0154670 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fce9000c400 tx=0x7fce9000c8c0 comp rx=0 tx=0).stop 2024-09-16T15:39:39.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.312+0000 7fcea7200640 1 -- 172.21.15.151:0/1706553931 shutdown_connections 2024-09-16T15:39:39.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.312+0000 7fcea7200640 1 --2- 172.21.15.151:0/1706553931 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fce84078850 0x7fce8407ad10 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:39.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.312+0000 7fcea7200640 1 --2- 172.21.15.151:0/1706553931 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcea0156670 0x7fcea0150b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:39.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.312+0000 7fcea7200640 1 --2- 172.21.15.151:0/1706553931 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcea015bc30 0x7fcea0154670 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:39.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.312+0000 7fcea7200640 1 --2- 172.21.15.151:0/1706553931 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcea00d0540 0x7fcea00ce660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:39.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.312+0000 7fcea7200640 1 -- 172.21.15.151:0/1706553931 >> 172.21.15.151:0/1706553931 conn(0x7fcea00cfa20 msgr2=0x7fcea00cc430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:39.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.312+0000 7fcea7200640 1 -- 172.21.15.151:0/1706553931 shutdown_connections 2024-09-16T15:39:39.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.312+0000 7fcea7200640 1 -- 172.21.15.151:0/1706553931 wait complete. 2024-09-16T15:39:39.732 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-16T15:39:21.969407+0000', 'modified': '2024-09-16T15:39:24.983141+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4350}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4350': {'gid': 4350, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.167:6837/3014501393', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6835', 'nonce': 3014501393}, {'type': 'v1', 'addr': '172.21.15.167:6837', 'nonce': 3014501393}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4350, 'qdb_cluster': [4350]} 2024-09-16T15:39:39.732 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-16T15:39:39.732 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-16T15:39:39.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.890+0000 7f8f63200640 1 -- 172.21.15.151:0/2058891117 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f5c0af3d0 msgr2=0x7f8f5c06f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:39.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.890+0000 7f8f63200640 1 --2- 172.21.15.151:0/2058891117 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f5c0af3d0 0x7f8f5c06f660 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8f48005fd0 tx=0x7f8f4802a4a0 comp rx=0 tx=0).stop 2024-09-16T15:39:39.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.890+0000 7f8f63200640 1 -- 172.21.15.151:0/2058891117 shutdown_connections 2024-09-16T15:39:39.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.890+0000 7f8f63200640 1 --2- 172.21.15.151:0/2058891117 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8f5c072df0 0x7f8f5c073280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:39.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.890+0000 7f8f63200640 1 --2- 172.21.15.151:0/2058891117 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f5c0af3d0 0x7f8f5c06f660 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:39.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.890+0000 7f8f63200640 1 -- 172.21.15.151:0/2058891117 >> 172.21.15.151:0/2058891117 conn(0x7f8f5c0b1950 msgr2=0x7f8f5c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:39.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.890+0000 7f8f63200640 1 -- 172.21.15.151:0/2058891117 shutdown_connections 2024-09-16T15:39:39.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.890+0000 7f8f63200640 1 -- 172.21.15.151:0/2058891117 wait complete. 2024-09-16T15:39:39.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.892+0000 7f8f63200640 1 Processor -- start 2024-09-16T15:39:39.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.892+0000 7f8f63200640 1 -- start start 2024-09-16T15:39:39.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.893+0000 7f8f63200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f5c072df0 0x7f8f5c145e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:39.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.893+0000 7f8f63200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8f5c0af3d0 0x7f8f5c159410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:39.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.893+0000 7f8f63200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f5c161550 0x7f8f5c0b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:39.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.893+0000 7f8f63200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8f5c0737c0 con 0x7f8f5c072df0 2024-09-16T15:39:39.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.893+0000 7f8f63200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8f5c06fdb0 con 0x7f8f5c161550 2024-09-16T15:39:39.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.893+0000 7f8f63200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f8f5c073940 con 0x7f8f5c0af3d0 2024-09-16T15:39:39.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.893+0000 7f8f5be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f5c072df0 0x7f8f5c145e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:39.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.893+0000 7f8f5be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f5c072df0 0x7f8f5c145e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:57506/0 (socket says 172.21.15.151:57506) 2024-09-16T15:39:39.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.893+0000 7f8f5be00640 1 -- 172.21.15.151:0/2725637002 learned_addr learned my addr 172.21.15.151:0/2725637002 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:39.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.893+0000 7f8f61200640 1 --2- 172.21.15.151:0/2725637002 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f5c161550 0x7f8f5c0b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:39.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.893+0000 7f8f5b400640 1 --2- 172.21.15.151:0/2725637002 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8f5c0af3d0 0x7f8f5c159410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:39.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.894+0000 7f8f5be00640 1 -- 172.21.15.151:0/2725637002 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8f5c0af3d0 msgr2=0x7f8f5c159410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:39.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.894+0000 7f8f5be00640 1 --2- 172.21.15.151:0/2725637002 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8f5c0af3d0 0x7f8f5c159410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:39.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.894+0000 7f8f5be00640 1 -- 172.21.15.151:0/2725637002 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f5c161550 msgr2=0x7f8f5c0b08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:39.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.894+0000 7f8f5be00640 1 --2- 172.21.15.151:0/2725637002 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f5c161550 0x7f8f5c0b08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:39.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.894+0000 7f8f5be00640 1 -- 172.21.15.151:0/2725637002 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f5c1f8390 con 0x7f8f5c072df0 2024-09-16T15:39:39.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.894+0000 7f8f5b400640 1 --2- 172.21.15.151:0/2725637002 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8f5c0af3d0 0x7f8f5c159410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:39:39.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.894+0000 7f8f5be00640 1 --2- 172.21.15.151:0/2725637002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f5c072df0 0x7f8f5c145e70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c00ba50 tx=0x7f8f4c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:39.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.894+0000 7f8f53e00640 1 -- 172.21.15.151:0/2725637002 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f4c019070 con 0x7f8f5c072df0 2024-09-16T15:39:39.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.894+0000 7f8f53e00640 1 -- 172.21.15.151:0/2725637002 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8f4c00cd10 con 0x7f8f5c072df0 2024-09-16T15:39:39.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.894+0000 7f8f53e00640 1 -- 172.21.15.151:0/2725637002 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f4c0146a0 con 0x7f8f5c072df0 2024-09-16T15:39:39.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.894+0000 7f8f63200640 1 -- 172.21.15.151:0/2725637002 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f5c0d07d0 con 0x7f8f5c072df0 2024-09-16T15:39:39.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.894+0000 7f8f63200640 1 -- 172.21.15.151:0/2725637002 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f5c1f8630 con 0x7f8f5c072df0 2024-09-16T15:39:39.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.896+0000 7f8f53e00640 1 -- 172.21.15.151:0/2725637002 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f8f4c01c070 con 0x7f8f5c072df0 2024-09-16T15:39:39.899 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.897+0000 7f8f53e00640 1 --2- 172.21.15.151:0/2725637002 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8f3c078850 0x7f8f3c07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:39.899 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.898+0000 7f8f53e00640 1 -- 172.21.15.151:0/2725637002 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f8f4c09a610 con 0x7f8f5c072df0 2024-09-16T15:39:39.899 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.898+0000 7f8f65ff0740 1 Processor -- start 2024-09-16T15:39:39.899 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.898+0000 7f8f5b400640 1 --2- 172.21.15.151:0/2725637002 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8f3c078850 0x7f8f3c07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:39.899 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.898+0000 7f8f65ff0740 1 -- start start 2024-09-16T15:39:39.899 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.899+0000 7f8f5b400640 1 --2- 172.21.15.151:0/2725637002 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8f3c078850 0x7f8f3c07ad10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8f48033000 tx=0x7f8f48004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:39.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.899+0000 7f8f65ff0740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643f42408b0 0x5643f4240c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:39.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.899+0000 7f8f65ff0740 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5643f41fba20 0x5643f423f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:39.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.899+0000 7f8f65ff0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5643f41fbfa0 0x5643f423f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:39.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.899+0000 7f8f65ff0740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5643f417e430 con 0x5643f42408b0 2024-09-16T15:39:39.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.899+0000 7f8f65ff0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5643f416e5d0 con 0x5643f41fbfa0 2024-09-16T15:39:39.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.899+0000 7f8f65ff0740 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x5643f415da00 con 0x5643f41fba20 2024-09-16T15:39:39.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.899+0000 7f8f5be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643f42408b0 0x5643f4240c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:39.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.899+0000 7f8f5be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643f42408b0 0x5643f4240c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:57520/0 (socket says 172.21.15.151:57520) 2024-09-16T15:39:39.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.899+0000 7f8f5be00640 1 -- 172.21.15.151:0/3886891384 learned_addr learned my addr 172.21.15.151:0/3886891384 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:39.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.899+0000 7f8f5b400640 1 --2- 172.21.15.151:0/3886891384 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5643f41fba20 0x5643f423f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:39.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.899+0000 7f8f61200640 1 --2- 172.21.15.151:0/3886891384 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5643f41fbfa0 0x5643f423f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:39.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.900+0000 7f8f5be00640 1 -- 172.21.15.151:0/3886891384 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5643f41fba20 msgr2=0x5643f423f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:39:39.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.900+0000 7f8f5be00640 1 --2- 172.21.15.151:0/3886891384 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5643f41fba20 0x5643f423f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:39.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.900+0000 7f8f5be00640 1 -- 172.21.15.151:0/3886891384 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5643f41fbfa0 msgr2=0x5643f423f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:39:39.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.900+0000 7f8f5be00640 1 --2- 172.21.15.151:0/3886891384 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5643f41fbfa0 0x5643f423f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:39.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.900+0000 7f8f5be00640 1 -- 172.21.15.151:0/3886891384 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643f4126520 con 0x5643f42408b0 2024-09-16T15:39:39.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.900+0000 7f8f5be00640 1 --2- 172.21.15.151:0/3886891384 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643f42408b0 0x5643f4240c90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c0625b0 tx=0x7f8f4c061f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27a54d9f0c8178ed server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:39.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.900+0000 7f8f50c00640 1 -- 172.21.15.151:0/3886891384 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f4c062620 con 0x5643f42408b0 2024-09-16T15:39:39.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.900+0000 7f8f50c00640 1 -- 172.21.15.151:0/3886891384 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8f4c06bf00 con 0x5643f42408b0 2024-09-16T15:39:39.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.900+0000 7f8f50c00640 1 -- 172.21.15.151:0/3886891384 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f4c06ba70 con 0x5643f42408b0 2024-09-16T15:39:39.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.901+0000 7f8f65ff0740 1 -- 172.21.15.151:0/3886891384 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643f42408b0 msgr2=0x5643f4240c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:39.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.901+0000 7f8f65ff0740 1 --2- 172.21.15.151:0/3886891384 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643f42408b0 0x5643f4240c90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c0625b0 tx=0x7f8f4c061f70 comp rx=0 tx=0).stop 2024-09-16T15:39:39.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.901+0000 7f8f65ff0740 1 -- 172.21.15.151:0/3886891384 shutdown_connections 2024-09-16T15:39:39.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.901+0000 7f8f65ff0740 1 --2- 172.21.15.151:0/3886891384 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5643f41fbfa0 0x5643f423f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:39.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.901+0000 7f8f65ff0740 1 --2- 172.21.15.151:0/3886891384 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5643f41fba20 0x5643f423f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:39.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.901+0000 7f8f65ff0740 1 --2- 172.21.15.151:0/3886891384 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643f42408b0 0x5643f4240c90 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:39.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.901+0000 7f8f65ff0740 1 -- 172.21.15.151:0/3886891384 >> 172.21.15.151:0/3886891384 conn(0x5643f41f6ff0 msgr2=0x5643f41faa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:39.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.901+0000 7f8f65ff0740 1 -- 172.21.15.151:0/3886891384 shutdown_connections 2024-09-16T15:39:39.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.901+0000 7f8f65ff0740 1 -- 172.21.15.151:0/3886891384 wait complete. 2024-09-16T15:39:39.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.902+0000 7f8f65ff0740 1 -- start start 2024-09-16T15:39:39.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.902+0000 7f8f65ff0740 1 Processor -- start 2024-09-16T15:39:39.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.903+0000 7f8f65ff0740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643f41fbfa0 0x5643f423a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:39.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.903+0000 7f8f65ff0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5643f42408b0 0x5643f4252ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:39.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.903+0000 7f8f65ff0740 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5643f4239120 0x5643f4239520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:39.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.903+0000 7f8f65ff0740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5643f4104330 con 0x5643f41fbfa0 2024-09-16T15:39:39.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.903+0000 7f8f65ff0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5643f4161810 con 0x5643f42408b0 2024-09-16T15:39:39.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.903+0000 7f8f65ff0740 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x5643f3ece620 con 0x5643f4239120 2024-09-16T15:39:39.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.903+0000 7f8f65ff0740 10 client.0 _ll_register_callbacks cb 0x5643f4109420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T15:39:39.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.903+0000 7f8f5be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643f41fbfa0 0x5643f423a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:39.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.903+0000 7f8f5be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643f41fbfa0 0x5643f423a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:57526/0 (socket says 172.21.15.151:57526) 2024-09-16T15:39:39.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.903+0000 7f8f5be00640 1 -- 172.21.15.151:0/3124016339 learned_addr learned my addr 172.21.15.151:0/3124016339 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:39.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.903+0000 7f8f5b400640 1 --2- 172.21.15.151:0/3124016339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5643f42408b0 0x5643f4252ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:39.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.903+0000 7f8f61200640 1 --2- 172.21.15.151:0/3124016339 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5643f4239120 0x5643f4239520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:39.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.903+0000 7f8f5be00640 1 -- 172.21.15.151:0/3124016339 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5643f4239120 msgr2=0x5643f4239520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:39:39.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.903+0000 7f8f5be00640 1 --2- 172.21.15.151:0/3124016339 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5643f4239120 0x5643f4239520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:39.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.903+0000 7f8f5be00640 1 -- 172.21.15.151:0/3124016339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5643f42408b0 msgr2=0x5643f4252ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:39:39.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.903+0000 7f8f5be00640 1 --2- 172.21.15.151:0/3124016339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5643f42408b0 0x5643f4252ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:39.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.903+0000 7f8f5be00640 1 -- 172.21.15.151:0/3124016339 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643f3ebf850 con 0x5643f41fbfa0 2024-09-16T15:39:39.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.904+0000 7f8f5be00640 1 --2- 172.21.15.151:0/3124016339 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643f41fbfa0 0x5643f423a740 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c0625b0 tx=0x7f8f4c0099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0c7a5b543859015 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:39.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.904+0000 7f8f45600640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-16T15:39:39.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.904+0000 7f8f45600640 1 -- 172.21.15.151:0/3124016339 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f4c084b40 con 0x5643f41fbfa0 2024-09-16T15:39:39.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.904+0000 7f8f45600640 1 -- 172.21.15.151:0/3124016339 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f8f4c084ce0 con 0x5643f41fbfa0 2024-09-16T15:39:39.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.904+0000 7f8f65ff0740 1 -- 172.21.15.151:0/3124016339 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f4c06bf00 con 0x5643f41fbfa0 2024-09-16T15:39:39.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.904+0000 7f8f45600640 1 -- 172.21.15.151:0/3124016339 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f8f4c072050 con 0x5643f41fbfa0 2024-09-16T15:39:39.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.904+0000 7f8f65ff0740 10 client.4662 fetch_fsmap learned FSMap version 8 2024-09-16T15:39:39.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.904+0000 7f8f65ff0740 1 -- 172.21.15.151:0/3124016339 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x5643f3f366b0 con 0x5643f41fbfa0 2024-09-16T15:39:39.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.904+0000 7f8f45600640 1 -- 172.21.15.151:0/3124016339 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8f4c06bf00 con 0x5643f41fbfa0 2024-09-16T15:39:39.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.904+0000 7f8f45600640 1 -- 172.21.15.151:0/3124016339 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 8) ==== 3502+0+0 (secure 0 0 0) 0x7f8f4c075020 con 0x5643f41fbfa0 2024-09-16T15:39:39.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.904+0000 7f8f65ff0740 10 client.4662 fetch_fsmap finished waiting for FSMap version 8 2024-09-16T15:39:39.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.904+0000 7f8f65ff0740 10 client.4662 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-16T15:39:39.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.904+0000 7f8f65ff0740 20 client.4662 populate_metadata read hostname 'smithi151' 2024-09-16T15:39:39.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.904+0000 7f8f65ff0740 1 --2- 172.21.15.151:0/3124016339 >> [v2:172.21.15.167:6835/3014501393,v1:172.21.15.167:6837/3014501393] conn(0x5643f4221f40 0x5643f4222320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:39.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.904+0000 7f8f65ff0740 4 client.4662 mds_command: new command op to 4350 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:39:39.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.904+0000 7f8f65ff0740 1 -- 172.21.15.151:0/3124016339 --> [v2:172.21.15.167:6835/3014501393,v1:172.21.15.167:6837/3014501393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5643f4170520 con 0x5643f4221f40 2024-09-16T15:39:39.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.905+0000 7f8f5b400640 1 --2- 172.21.15.151:0/3124016339 >> [v2:172.21.15.167:6835/3014501393,v1:172.21.15.167:6837/3014501393] conn(0x5643f4221f40 0x5643f4222320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:39.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.905+0000 7f8f5b400640 1 --2- 172.21.15.151:0/3124016339 >> [v2:172.21.15.167:6835/3014501393,v1:172.21.15.167:6837/3014501393] conn(0x5643f4221f40 0x5643f4222320 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78fb128a7311ffe0 server_cookie=1b7f0d86f6a65be1 in_seq=0 out_seq=0 2024-09-16T15:39:39.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.905+0000 7f8f45600640 10 client.4662 ms_handle_connect on v2:172.21.15.167:6835/3014501393 2024-09-16T15:39:39.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.906+0000 7f8f45600640 1 -- 172.21.15.151:0/3124016339 <== mds.0 v2:172.21.15.167:6835/3014501393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5643f4170520 con 0x5643f4221f40 2024-09-16T15:39:39.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.906+0000 7f8f45600640 10 client.4662 handle_command_reply: tid=0 2024-09-16T15:39:39.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.906+0000 7f8f65ff0740 1 client.4662 shutdown 2024-09-16T15:39:39.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.907+0000 7f8f65ff0740 1 -- 172.21.15.151:0/3124016339 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643f41fbfa0 msgr2=0x5643f423a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:39.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.907+0000 7f8f65ff0740 1 --2- 172.21.15.151:0/3124016339 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643f41fbfa0 0x5643f423a740 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c0625b0 tx=0x7f8f4c0099b0 comp rx=0 tx=0).stop 2024-09-16T15:39:39.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.907+0000 7f8f65ff0740 1 -- 172.21.15.151:0/3124016339 shutdown_connections 2024-09-16T15:39:39.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.907+0000 7f8f65ff0740 1 --2- 172.21.15.151:0/3124016339 >> [v2:172.21.15.167:6835/3014501393,v1:172.21.15.167:6837/3014501393] conn(0x5643f4221f40 0x5643f4222320 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:39.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.907+0000 7f8f65ff0740 1 --2- 172.21.15.151:0/3124016339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5643f42408b0 0x5643f4252ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:39.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.907+0000 7f8f65ff0740 1 --2- 172.21.15.151:0/3124016339 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5643f4239120 0x5643f4239520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:39.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.907+0000 7f8f45600640 0 client.4662 ms_handle_reset on v2:172.21.15.167:6835/3014501393 2024-09-16T15:39:39.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.907+0000 7f8f65ff0740 1 --2- 172.21.15.151:0/3124016339 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643f41fbfa0 0x5643f423a740 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:39.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.907+0000 7f8f65ff0740 1 -- 172.21.15.151:0/3124016339 >> 172.21.15.151:0/3124016339 conn(0x5643f41fba20 msgr2=0x5643f42211f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:39.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.907+0000 7f8f65ff0740 1 -- 172.21.15.151:0/3124016339 shutdown_connections 2024-09-16T15:39:39.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.907+0000 7f8f65ff0740 1 -- 172.21.15.151:0/3124016339 wait complete. 2024-09-16T15:39:39.912 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.907+0000 7f8f65ff0740 20 client.4662 trim_cache size 0 max 16384 2024-09-16T15:39:39.926 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.925+0000 7f8f65ff0740 1 Processor -- start 2024-09-16T15:39:39.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.926+0000 7f8f65ff0740 1 -- start start 2024-09-16T15:39:39.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.926+0000 7f8f65ff0740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643f41fbfa0 0x5643f4222f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:39.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.926+0000 7f8f65ff0740 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5643f4239120 0x5643f41fa390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:39.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.926+0000 7f8f65ff0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5643f42408b0 0x5643f41fa8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:39.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.926+0000 7f8f65ff0740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8f5c2003a0 con 0x5643f41fbfa0 2024-09-16T15:39:39.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.926+0000 7f8f65ff0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8f5c1ff790 con 0x5643f42408b0 2024-09-16T15:39:39.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.926+0000 7f8f65ff0740 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f8f5c1ffd80 con 0x5643f4239120 2024-09-16T15:39:39.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.927+0000 7f8f5be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643f41fbfa0 0x5643f4222f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:39.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.927+0000 7f8f5be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643f41fbfa0 0x5643f4222f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:57538/0 (socket says 172.21.15.151:57538) 2024-09-16T15:39:39.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.927+0000 7f8f5be00640 1 -- 172.21.15.151:0/607647665 learned_addr learned my addr 172.21.15.151:0/607647665 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:39.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.927+0000 7f8f5b400640 1 --2- 172.21.15.151:0/607647665 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5643f4239120 0x5643f41fa390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:39.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.927+0000 7f8f61200640 1 --2- 172.21.15.151:0/607647665 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5643f42408b0 0x5643f41fa8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:39.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.927+0000 7f8f5be00640 1 -- 172.21.15.151:0/607647665 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5643f4239120 msgr2=0x5643f41fa390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:39:39.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.927+0000 7f8f5be00640 1 --2- 172.21.15.151:0/607647665 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5643f4239120 0x5643f41fa390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:39.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.927+0000 7f8f5be00640 1 -- 172.21.15.151:0/607647665 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5643f42408b0 msgr2=0x5643f41fa8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:39:39.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.927+0000 7f8f5be00640 1 --2- 172.21.15.151:0/607647665 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5643f42408b0 0x5643f41fa8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:39.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.927+0000 7f8f5be00640 1 -- 172.21.15.151:0/607647665 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643f4210790 con 0x5643f41fbfa0 2024-09-16T15:39:39.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.927+0000 7f8f61200640 1 --2- 172.21.15.151:0/607647665 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5643f42408b0 0x5643f41fa8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:39:39.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.927+0000 7f8f5be00640 1 --2- 172.21.15.151:0/607647665 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643f41fbfa0 0x5643f4222f90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c062320 tx=0x7f8f4c05e5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa35493bf7e39b20 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:39.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.927+0000 7f8f47e00640 1 -- 172.21.15.151:0/607647665 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f4c0780d0 con 0x5643f41fbfa0 2024-09-16T15:39:39.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.927+0000 7f8f47e00640 1 -- 172.21.15.151:0/607647665 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8f4c084b40 con 0x5643f41fbfa0 2024-09-16T15:39:39.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.927+0000 7f8f47e00640 1 -- 172.21.15.151:0/607647665 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f4c069260 con 0x5643f41fbfa0 2024-09-16T15:39:39.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.928+0000 7f8f65ff0740 1 -- 172.21.15.151:0/607647665 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643f41fbfa0 msgr2=0x5643f4222f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:39.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.928+0000 7f8f65ff0740 1 --2- 172.21.15.151:0/607647665 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643f41fbfa0 0x5643f4222f90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c062320 tx=0x7f8f4c05e5f0 comp rx=0 tx=0).stop 2024-09-16T15:39:39.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.928+0000 7f8f65ff0740 1 -- 172.21.15.151:0/607647665 shutdown_connections 2024-09-16T15:39:39.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.928+0000 7f8f65ff0740 1 --2- 172.21.15.151:0/607647665 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5643f42408b0 0x5643f41fa8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:39.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.928+0000 7f8f65ff0740 1 --2- 172.21.15.151:0/607647665 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5643f4239120 0x5643f41fa390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:39.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.928+0000 7f8f65ff0740 1 --2- 172.21.15.151:0/607647665 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643f41fbfa0 0x5643f4222f90 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:39.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.928+0000 7f8f65ff0740 1 -- 172.21.15.151:0/607647665 >> 172.21.15.151:0/607647665 conn(0x5643f41fba20 msgr2=0x5643f4238840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:39.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.928+0000 7f8f65ff0740 1 -- 172.21.15.151:0/607647665 shutdown_connections 2024-09-16T15:39:39.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.928+0000 7f8f65ff0740 1 -- 172.21.15.151:0/607647665 wait complete. 2024-09-16T15:39:39.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.929+0000 7f8f65ff0740 1 -- start start 2024-09-16T15:39:39.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.929+0000 7f8f65ff0740 1 Processor -- start 2024-09-16T15:39:39.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.930+0000 7f8f65ff0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5643f41fbfa0 0x5643f42b6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:39.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.930+0000 7f8f65ff0740 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5643f4239120 0x5643f41fa180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:39.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.930+0000 7f8f65ff0740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643f42408b0 0x5643f41fa6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:39.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.930+0000 7f8f65ff0740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5643f42110b0 con 0x5643f42408b0 2024-09-16T15:39:39.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.930+0000 7f8f65ff0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5643f4104330 con 0x5643f41fbfa0 2024-09-16T15:39:39.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.930+0000 7f8f65ff0740 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x5643f4224140 con 0x5643f4239120 2024-09-16T15:39:39.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.930+0000 7f8f65ff0740 10 client.0 _ll_register_callbacks cb 0x7f8f4c0623f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T15:39:39.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.930+0000 7f8f61200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643f42408b0 0x5643f41fa6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:39.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.930+0000 7f8f61200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643f42408b0 0x5643f41fa6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:57542/0 (socket says 172.21.15.151:57542) 2024-09-16T15:39:39.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.930+0000 7f8f61200640 1 -- 172.21.15.151:0/1300090576 learned_addr learned my addr 172.21.15.151:0/1300090576 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:39.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.930+0000 7f8f5b400640 1 --2- 172.21.15.151:0/1300090576 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5643f4239120 0x5643f41fa180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:39.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.930+0000 7f8f5be00640 1 --2- 172.21.15.151:0/1300090576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5643f41fbfa0 0x5643f42b6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:39.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.931+0000 7f8f61200640 1 -- 172.21.15.151:0/1300090576 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5643f4239120 msgr2=0x5643f41fa180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:39:39.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.931+0000 7f8f61200640 1 --2- 172.21.15.151:0/1300090576 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5643f4239120 0x5643f41fa180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:39.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.931+0000 7f8f61200640 1 -- 172.21.15.151:0/1300090576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5643f41fbfa0 msgr2=0x5643f42b6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:39:39.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.931+0000 7f8f61200640 1 --2- 172.21.15.151:0/1300090576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5643f41fbfa0 0x5643f42b6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:39.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.931+0000 7f8f61200640 1 -- 172.21.15.151:0/1300090576 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643f4235620 con 0x5643f42408b0 2024-09-16T15:39:39.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.931+0000 7f8f5be00640 1 --2- 172.21.15.151:0/1300090576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5643f41fbfa0 0x5643f42b6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:39:39.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.931+0000 7f8f61200640 1 --2- 172.21.15.151:0/1300090576 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643f42408b0 0x5643f41fa6c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8f54013ab0 tx=0x7f8f54013f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=460a908c2706bae9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:39.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.931+0000 7f8f45600640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-16T15:39:39.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.931+0000 7f8f45600640 1 -- 172.21.15.151:0/1300090576 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5400f8a0 con 0x5643f42408b0 2024-09-16T15:39:39.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.931+0000 7f8f45600640 1 -- 172.21.15.151:0/1300090576 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f8f5400fa40 con 0x5643f42408b0 2024-09-16T15:39:39.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.931+0000 7f8f65ff0740 1 -- 172.21.15.151:0/1300090576 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f4c084b40 con 0x5643f42408b0 2024-09-16T15:39:39.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.931+0000 7f8f45600640 1 -- 172.21.15.151:0/1300090576 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f8f54024050 con 0x5643f42408b0 2024-09-16T15:39:39.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.931+0000 7f8f65ff0740 10 client.4668 fetch_fsmap learned FSMap version 8 2024-09-16T15:39:39.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.931+0000 7f8f65ff0740 1 -- 172.21.15.151:0/1300090576 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x5643f4261f80 con 0x5643f42408b0 2024-09-16T15:39:39.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.932+0000 7f8f45600640 1 -- 172.21.15.151:0/1300090576 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8f4c084b40 con 0x5643f42408b0 2024-09-16T15:39:39.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.932+0000 7f8f45600640 1 -- 172.21.15.151:0/1300090576 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 8) ==== 3502+0+0 (secure 0 0 0) 0x7f8f54027020 con 0x5643f42408b0 2024-09-16T15:39:39.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.932+0000 7f8f65ff0740 10 client.4668 fetch_fsmap finished waiting for FSMap version 8 2024-09-16T15:39:39.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.932+0000 7f8f65ff0740 10 client.4668 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-16T15:39:39.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.932+0000 7f8f65ff0740 20 client.4668 populate_metadata read hostname 'smithi151' 2024-09-16T15:39:39.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.932+0000 7f8f65ff0740 1 --2- 172.21.15.151:0/1300090576 >> [v2:172.21.15.167:6835/3014501393,v1:172.21.15.167:6837/3014501393] conn(0x5643f42c0600 0x5643f42c3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:39.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.932+0000 7f8f65ff0740 4 client.4668 mds_command: new command op to 4350 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T15:39:39.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.932+0000 7f8f65ff0740 1 -- 172.21.15.151:0/1300090576 --> [v2:172.21.15.167:6835/3014501393,v1:172.21.15.167:6837/3014501393] -- command(tid 0: {"prefix": "status"}) -- 0x5643f416bfd0 con 0x5643f42c0600 2024-09-16T15:39:39.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.932+0000 7f8f5be00640 1 --2- 172.21.15.151:0/1300090576 >> [v2:172.21.15.167:6835/3014501393,v1:172.21.15.167:6837/3014501393] conn(0x5643f42c0600 0x5643f42c3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:39.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.933+0000 7f8f5be00640 1 --2- 172.21.15.151:0/1300090576 >> [v2:172.21.15.167:6835/3014501393,v1:172.21.15.167:6837/3014501393] conn(0x5643f42c0600 0x5643f42c3130 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6c67f6ce898652e server_cookie=2cc6688c78dacf57 in_seq=0 out_seq=0 2024-09-16T15:39:39.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.933+0000 7f8f45600640 10 client.4668 ms_handle_connect on v2:172.21.15.167:6835/3014501393 2024-09-16T15:39:39.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.933+0000 7f8f45600640 1 -- 172.21.15.151:0/1300090576 <== mds.0 v2:172.21.15.167:6835/3014501393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+323 (crc 0 0 0) 0x5643f416bfd0 con 0x5643f42c0600 2024-09-16T15:39:39.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.933+0000 7f8f45600640 10 client.4668 handle_command_reply: tid=0 2024-09-16T15:39:39.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.933+0000 7f8f65ff0740 1 client.4668 shutdown 2024-09-16T15:39:39.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.933+0000 7f8f65ff0740 1 -- 172.21.15.151:0/1300090576 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643f42408b0 msgr2=0x5643f41fa6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:39.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.933+0000 7f8f65ff0740 1 --2- 172.21.15.151:0/1300090576 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643f42408b0 0x5643f41fa6c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8f54013ab0 tx=0x7f8f54013f70 comp rx=0 tx=0).stop 2024-09-16T15:39:39.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.933+0000 7f8f65ff0740 1 -- 172.21.15.151:0/1300090576 shutdown_connections 2024-09-16T15:39:39.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.933+0000 7f8f65ff0740 1 --2- 172.21.15.151:0/1300090576 >> [v2:172.21.15.167:6835/3014501393,v1:172.21.15.167:6837/3014501393] conn(0x5643f42c0600 0x5643f42c3130 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:39.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.933+0000 7f8f65ff0740 1 --2- 172.21.15.151:0/1300090576 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643f42408b0 0x5643f41fa6c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:39.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.933+0000 7f8f65ff0740 1 --2- 172.21.15.151:0/1300090576 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5643f4239120 0x5643f41fa180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:39.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.933+0000 7f8f65ff0740 1 --2- 172.21.15.151:0/1300090576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5643f41fbfa0 0x5643f42b6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:39.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.933+0000 7f8f45600640 0 client.4668 ms_handle_reset on v2:172.21.15.167:6835/3014501393 2024-09-16T15:39:39.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.933+0000 7f8f65ff0740 1 -- 172.21.15.151:0/1300090576 >> 172.21.15.151:0/1300090576 conn(0x5643f41fba20 msgr2=0x5643f4238840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:39.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.933+0000 7f8f65ff0740 1 -- 172.21.15.151:0/1300090576 shutdown_connections 2024-09-16T15:39:39.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.933+0000 7f8f65ff0740 1 -- 172.21.15.151:0/1300090576 wait complete. 2024-09-16T15:39:39.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.933+0000 7f8f65ff0740 20 client.4668 trim_cache size 0 max 16384 2024-09-16T15:39:39.939 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-16T15:39:39.939 INFO:teuthology.orchestra.run.smithi151.stdout: "cluster_fsid": "1757be18-e41a-4446-bb4f-ee4ac81c39df", 2024-09-16T15:39:39.939 INFO:teuthology.orchestra.run.smithi151.stdout: "whoami": 0, 2024-09-16T15:39:39.939 INFO:teuthology.orchestra.run.smithi151.stdout: "id": 4350, 2024-09-16T15:39:39.939 INFO:teuthology.orchestra.run.smithi151.stdout: "want_state": "up:active", 2024-09-16T15:39:39.939 INFO:teuthology.orchestra.run.smithi151.stdout: "state": "up:active", 2024-09-16T15:39:39.939 INFO:teuthology.orchestra.run.smithi151.stdout: "fs_name": "cephfs", 2024-09-16T15:39:39.939 INFO:teuthology.orchestra.run.smithi151.stdout: "rank_uptime": 17.391826008999999, 2024-09-16T15:39:39.939 INFO:teuthology.orchestra.run.smithi151.stdout: "mdsmap_epoch": 8, 2024-09-16T15:39:39.939 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_epoch": 22, 2024-09-16T15:39:39.939 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_epoch_barrier": 22, 2024-09-16T15:39:39.939 INFO:teuthology.orchestra.run.smithi151.stdout: "uptime": 23.40999764 2024-09-16T15:39:39.940 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-16T15:39:39.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.934+0000 7f8f63200640 1 -- 172.21.15.151:0/2725637002 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8f3c078850 msgr2=0x7f8f3c07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:39.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.934+0000 7f8f63200640 1 --2- 172.21.15.151:0/2725637002 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8f3c078850 0x7f8f3c07ad10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8f48033000 tx=0x7f8f48004730 comp rx=0 tx=0).stop 2024-09-16T15:39:39.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.934+0000 7f8f63200640 1 -- 172.21.15.151:0/2725637002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f5c072df0 msgr2=0x7f8f5c145e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:39.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.934+0000 7f8f63200640 1 --2- 172.21.15.151:0/2725637002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f5c072df0 0x7f8f5c145e70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c00ba50 tx=0x7f8f4c00bf10 comp rx=0 tx=0).stop 2024-09-16T15:39:39.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.934+0000 7f8f63200640 1 -- 172.21.15.151:0/2725637002 shutdown_connections 2024-09-16T15:39:39.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.934+0000 7f8f63200640 1 --2- 172.21.15.151:0/2725637002 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8f3c078850 0x7f8f3c07ad10 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:39.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.934+0000 7f8f63200640 1 --2- 172.21.15.151:0/2725637002 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f5c161550 0x7f8f5c0b08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:39.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.934+0000 7f8f63200640 1 --2- 172.21.15.151:0/2725637002 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8f5c0af3d0 0x7f8f5c159410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:39.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.934+0000 7f8f63200640 1 --2- 172.21.15.151:0/2725637002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f5c072df0 0x7f8f5c145e70 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:39.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.934+0000 7f8f63200640 1 -- 172.21.15.151:0/2725637002 >> 172.21.15.151:0/2725637002 conn(0x7f8f5c0b1950 msgr2=0x7f8f5c1603a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:39.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.934+0000 7f8f63200640 1 -- 172.21.15.151:0/2725637002 shutdown_connections 2024-09-16T15:39:39.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:39.934+0000 7f8f63200640 1 -- 172.21.15.151:0/2725637002 wait complete. 2024-09-16T15:39:39.951 INFO:teuthology.run_tasks:Running task kclient... 2024-09-16T15:39:39.977 INFO:tasks.kclient:Mounting kernel clients... 2024-09-16T15:39:39.977 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-09-16T15:39:39.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:39.977 DEBUG:teuthology.orchestra.run.smithi151:> ip netns list 2024-09-16T15:39:40.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:40.014 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip link delete ceph-brx 2024-09-16T15:39:40.113 INFO:teuthology.orchestra.run.smithi151.stderr:Cannot find device "ceph-brx" 2024-09-16T15:39:40.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T15:39:40.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:40.116 DEBUG:teuthology.orchestra.run.smithi151:> ip netns list 2024-09-16T15:39:40.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:40.152 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip link delete ceph-brx 2024-09-16T15:39:40.249 INFO:teuthology.orchestra.run.smithi151.stderr:Cannot find device "ceph-brx" 2024-09-16T15:39:40.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T15:39:40.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:40.253 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list 2024-09-16T15:39:40.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:40.293 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link delete ceph-brx 2024-09-16T15:39:40.396 INFO:teuthology.orchestra.run.smithi167.stderr:Cannot find device "ceph-brx" 2024-09-16T15:39:40.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T15:39:40.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:40.399 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list 2024-09-16T15:39:40.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:40.435 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link delete ceph-brx 2024-09-16T15:39:40.532 INFO:teuthology.orchestra.run.smithi167.stderr:Cannot find device "ceph-brx" 2024-09-16T15:39:40.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T15:39:40.535 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-09-16T15:39:40.535 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-16T15:39:40.535 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-16T15:39:40.536 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-16T15:39:40.694 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:40.693+0000 7f73fe200640 1 -- 172.21.15.151:0/1527292367 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73f8161550 msgr2=0x7f73f806ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:40.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:40.693+0000 7f73fe200640 1 --2- 172.21.15.151:0/1527292367 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73f8161550 0x7f73f806ef50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f73f80cd5d0 tx=0x7f73e802a560 comp rx=0 tx=0).stop 2024-09-16T15:39:40.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:40.693+0000 7f73fe200640 1 -- 172.21.15.151:0/1527292367 shutdown_connections 2024-09-16T15:39:40.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:40.693+0000 7f73fe200640 1 --2- 172.21.15.151:0/1527292367 >> v2:172.21.15.167:3300/0 conn(0x7f73f8072df0 0x7f73f80732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:40.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:40.693+0000 7f73fe200640 1 --2- 172.21.15.151:0/1527292367 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73f8161550 0x7f73f806ef50 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:40.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:40.693+0000 7f73fe200640 1 -- 172.21.15.151:0/1527292367 >> 172.21.15.151:0/1527292367 conn(0x7f73f80b1950 msgr2=0x7f73f80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:40.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:40.693+0000 7f73fe200640 1 -- 172.21.15.151:0/1527292367 shutdown_connections 2024-09-16T15:39:40.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:40.693+0000 7f73fe200640 1 -- 172.21.15.151:0/1527292367 wait complete. 2024-09-16T15:39:40.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:40.695+0000 7f73fe200640 1 Processor -- start 2024-09-16T15:39:40.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:40.695+0000 7f73fe200640 1 -- start start 2024-09-16T15:39:40.697 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:40.696+0000 7f73fe200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73f8072df0 0x7f73f8147600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:40.697 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:40.696+0000 7f73fe200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73f80af3d0 0x7f73f80d02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:40.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:40.696+0000 7f73fe200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73f8161550 0x7f73f81f7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:40.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:40.696+0000 7f73fe200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f73f80737e0 con 0x7f73f8072df0 2024-09-16T15:39:40.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:40.696+0000 7f73fe200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f73f806fdb0 con 0x7f73f8161550 2024-09-16T15:39:40.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:40.696+0000 7f73fe200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f73f8073960 con 0x7f73f80af3d0 2024-09-16T15:39:40.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:40.696+0000 7f73f7400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73f8072df0 0x7f73f8147600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:40.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:40.696+0000 7f73f7400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73f8072df0 0x7f73f8147600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:57572/0 (socket says 172.21.15.151:57572) 2024-09-16T15:39:40.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:40.696+0000 7f73f7400640 1 -- 172.21.15.151:0/1163591017 learned_addr learned my addr 172.21.15.151:0/1163591017 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:40.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:40.696+0000 7f73f7e00640 1 --2- 172.21.15.151:0/1163591017 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73f8161550 0x7f73f81f7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:40.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:40.696+0000 7f73f6a00640 1 --2- 172.21.15.151:0/1163591017 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73f80af3d0 0x7f73f80d02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:40.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:40.696+0000 7f73f7400640 1 -- 172.21.15.151:0/1163591017 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73f80af3d0 msgr2=0x7f73f80d02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:40.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:40.696+0000 7f73f7400640 1 --2- 172.21.15.151:0/1163591017 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73f80af3d0 0x7f73f80d02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:40.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:40.696+0000 7f73f7400640 1 -- 172.21.15.151:0/1163591017 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73f8161550 msgr2=0x7f73f81f7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:40.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:40.696+0000 7f73f7400640 1 --2- 172.21.15.151:0/1163591017 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73f8161550 0x7f73f81f7d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:40.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:40.696+0000 7f73f7400640 1 -- 172.21.15.151:0/1163591017 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73f80ce4c0 con 0x7f73f8072df0 2024-09-16T15:39:40.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:40.697+0000 7f73f7400640 1 --2- 172.21.15.151:0/1163591017 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73f8072df0 0x7f73f8147600 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f73e8034040 tx=0x7f73e8003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:40.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:40.697+0000 7f73ef400640 1 -- 172.21.15.151:0/1163591017 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e803e030 con 0x7f73f8072df0 2024-09-16T15:39:40.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:40.697+0000 7f73ef400640 1 -- 172.21.15.151:0/1163591017 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f73e8003e00 con 0x7f73f8072df0 2024-09-16T15:39:40.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:40.697+0000 7f73fe200640 1 -- 172.21.15.151:0/1163591017 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73f80cfe10 con 0x7f73f8072df0 2024-09-16T15:39:40.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:40.697+0000 7f73ef400640 1 -- 172.21.15.151:0/1163591017 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e8035040 con 0x7f73f8072df0 2024-09-16T15:39:40.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:40.697+0000 7f73fe200640 1 -- 172.21.15.151:0/1163591017 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73f80cea20 con 0x7f73f8072df0 2024-09-16T15:39:40.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:40.698+0000 7f73fe200640 1 -- 172.21.15.151:0/1163591017 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73f8073de0 con 0x7f73f8072df0 2024-09-16T15:39:40.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:40.699+0000 7f73ef400640 1 -- 172.21.15.151:0/1163591017 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f73e8041030 con 0x7f73f8072df0 2024-09-16T15:39:40.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:40.700+0000 7f73ef400640 1 --2- 172.21.15.151:0/1163591017 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f73d8078850 0x7f73d807ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:40.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:40.700+0000 7f73ef400640 1 -- 172.21.15.151:0/1163591017 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f73e80baee0 con 0x7f73f8072df0 2024-09-16T15:39:40.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:40.700+0000 7f73f6a00640 1 --2- 172.21.15.151:0/1163591017 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f73d8078850 0x7f73d807ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:40.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:40.701+0000 7f73f6a00640 1 --2- 172.21.15.151:0/1163591017 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f73d8078850 0x7f73d807ad10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f73f80cd5d0 tx=0x7f73e4009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:40.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:40.704+0000 7f73ef400640 1 -- 172.21.15.151:0/1163591017 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f73f80ce4c0 con 0x7f73f8072df0 2024-09-16T15:39:40.926 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:40.925+0000 7f73fe200640 1 -- 172.21.15.151:0/1163591017 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f73f8147b40 con 0x7f73f8072df0 2024-09-16T15:39:40.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:40.926+0000 7f73ef400640 1 -- 172.21.15.151:0/1163591017 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f73e8082f00 con 0x7f73f8072df0 2024-09-16T15:39:40.927 INFO:teuthology.orchestra.run.smithi151.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-16T15:39:40.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:40.928+0000 7f73fe200640 1 -- 172.21.15.151:0/1163591017 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f73d8078850 msgr2=0x7f73d807ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:40.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:40.928+0000 7f73fe200640 1 --2- 172.21.15.151:0/1163591017 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f73d8078850 0x7f73d807ad10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f73f80cd5d0 tx=0x7f73e4009040 comp rx=0 tx=0).stop 2024-09-16T15:39:40.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:40.928+0000 7f73fe200640 1 -- 172.21.15.151:0/1163591017 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73f8072df0 msgr2=0x7f73f8147600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:40.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:40.928+0000 7f73fe200640 1 --2- 172.21.15.151:0/1163591017 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73f8072df0 0x7f73f8147600 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f73e8034040 tx=0x7f73e8003910 comp rx=0 tx=0).stop 2024-09-16T15:39:40.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:40.928+0000 7f73fe200640 1 -- 172.21.15.151:0/1163591017 shutdown_connections 2024-09-16T15:39:40.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:40.928+0000 7f73fe200640 1 --2- 172.21.15.151:0/1163591017 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f73d8078850 0x7f73d807ad10 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:40.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:40.928+0000 7f73fe200640 1 --2- 172.21.15.151:0/1163591017 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73f8161550 0x7f73f81f7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:40.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:40.928+0000 7f73fe200640 1 --2- 172.21.15.151:0/1163591017 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73f80af3d0 0x7f73f80d02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:40.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:40.928+0000 7f73fe200640 1 --2- 172.21.15.151:0/1163591017 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73f8072df0 0x7f73f8147600 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:40.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:40.928+0000 7f73fe200640 1 -- 172.21.15.151:0/1163591017 >> 172.21.15.151:0/1163591017 conn(0x7f73f80b1950 msgr2=0x7f73f806ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:40.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:40.928+0000 7f73fe200640 1 -- 172.21.15.151:0/1163591017 shutdown_connections 2024-09-16T15:39:40.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:40.928+0000 7f73fe200640 1 -- 172.21.15.151:0/1163591017 wait complete. 2024-09-16T15:39:40.944 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-16T15:39:40.944 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T15:39:40.944 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi151.front.sepia.ceph.com 2024-09-16T15:39:40.945 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-16T15:39:40.945 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-16T15:39:40.945 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-16T15:39:40.945 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-16T15:39:40.945 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T15:39:40.945 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T15:39:40.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:40.945 DEBUG:teuthology.orchestra.run.smithi151:> ip addr 2024-09-16T15:39:40.980 INFO:teuthology.orchestra.run.smithi151.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T15:39:40.980 INFO:teuthology.orchestra.run.smithi151.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T15:39:40.980 INFO:teuthology.orchestra.run.smithi151.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T15:39:40.980 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:39:40.980 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 ::1/128 scope host 2024-09-16T15:39:40.980 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:39:40.980 INFO:teuthology.orchestra.run.smithi151.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:39:40.980 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:da:6e:0c brd ff:ff:ff:ff:ff:ff 2024-09-16T15:39:40.980 INFO:teuthology.orchestra.run.smithi151.stdout: altname enp7s0f0 2024-09-16T15:39:40.981 INFO:teuthology.orchestra.run.smithi151.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:39:40.981 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:da:6e:0d brd ff:ff:ff:ff:ff:ff 2024-09-16T15:39:40.981 INFO:teuthology.orchestra.run.smithi151.stdout: altname enp7s0f1 2024-09-16T15:39:40.981 INFO:teuthology.orchestra.run.smithi151.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:39:40.981 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:88:6f:62 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:39:40.981 INFO:teuthology.orchestra.run.smithi151.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T15:39:40.981 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:88:6f:63 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:39:40.981 INFO:teuthology.orchestra.run.smithi151.stdout: inet 172.21.15.151/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-16T15:39:40.981 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft 42708sec preferred_lft 42708sec 2024-09-16T15:39:40.981 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 fe80::ec4:7aff:fe88:6f63/64 scope link noprefixroute 2024-09-16T15:39:40.981 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:39:40.982 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-16T15:39:40.982 DEBUG:teuthology.orchestra.run.smithi151:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T15:39:40.982 DEBUG:teuthology.orchestra.run.smithi151:> set -e 2024-09-16T15:39:40.983 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip link add name ceph-brx type bridge 2024-09-16T15:39:40.983 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip addr flush dev ceph-brx 2024-09-16T15:39:40.983 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip link set ceph-brx up 2024-09-16T15:39:40.983 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-16T15:39:40.983 DEBUG:teuthology.orchestra.run.smithi151:> ') 2024-09-16T15:39:41.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:39:41.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:39:41.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:41.395 DEBUG:teuthology.orchestra.run.smithi151:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T15:39:41.452 INFO:teuthology.orchestra.run.smithi151.stdout:1 2024-09-16T15:39:41.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:41.455 DEBUG:teuthology.orchestra.run.smithi151:> route 2024-09-16T15:39:41.501 INFO:teuthology.orchestra.run.smithi151.stdout:Kernel IP routing table 2024-09-16T15:39:41.501 INFO:teuthology.orchestra.run.smithi151.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T15:39:41.502 INFO:teuthology.orchestra.run.smithi151.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-16T15:39:41.502 INFO:teuthology.orchestra.run.smithi151.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-16T15:39:41.502 INFO:teuthology.orchestra.run.smithi151.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T15:39:41.502 DEBUG:teuthology.orchestra.run.smithi151:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T15:39:41.502 DEBUG:teuthology.orchestra.run.smithi151:> set -e 2024-09-16T15:39:41.502 DEBUG:teuthology.orchestra.run.smithi151:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-16T15:39:41.502 DEBUG:teuthology.orchestra.run.smithi151:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-16T15:39:41.502 DEBUG:teuthology.orchestra.run.smithi151:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-16T15:39:41.502 DEBUG:teuthology.orchestra.run.smithi151:> ') 2024-09-16T15:39:41.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:39:42.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:39:42.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:42.131 DEBUG:teuthology.orchestra.run.smithi151:> ip netns list 2024-09-16T15:39:42.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:42.167 DEBUG:teuthology.orchestra.run.smithi151:> ip netns list-id 2024-09-16T15:39:42.242 DEBUG:teuthology.orchestra.run.smithi151:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T15:39:42.242 DEBUG:teuthology.orchestra.run.smithi151:> set -e 2024-09-16T15:39:42.242 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T15:39:42.242 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-16T15:39:42.242 DEBUG:teuthology.orchestra.run.smithi151:> ') 2024-09-16T15:39:42.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:39:42.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:39:42.419 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-16T15:39:42.419 DEBUG:teuthology.orchestra.run.smithi151:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T15:39:42.419 DEBUG:teuthology.orchestra.run.smithi151:> set -e 2024-09-16T15:39:42.420 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-16T15:39:42.420 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-16T15:39:42.420 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-16T15:39:42.420 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-16T15:39:42.420 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-16T15:39:42.420 DEBUG:teuthology.orchestra.run.smithi151:> ') 2024-09-16T15:39:42.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:39:42.660 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:39:42.669 DEBUG:teuthology.orchestra.run.smithi151:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T15:39:42.669 DEBUG:teuthology.orchestra.run.smithi151:> set -e 2024-09-16T15:39:42.669 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip link set brx.0 up 2024-09-16T15:39:42.669 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip link set dev brx.0 master ceph-brx 2024-09-16T15:39:42.669 DEBUG:teuthology.orchestra.run.smithi151:> ') 2024-09-16T15:39:42.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:39:42.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:39:42.835 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:39:42.835 DEBUG:teuthology.orchestra.run.smithi151:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-16T15:39:42.871 INFO:teuthology.orchestra.run.smithi151.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-16T15:39:42.871 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:39:42.871 DEBUG:teuthology.orchestra.run.smithi151:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-16T15:39:42.946 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-16T15:39:42.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:42.946 DEBUG:teuthology.orchestra.run.smithi151:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-16T15:39:43.083 INFO:teuthology.orchestra.run.smithi151.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-16T15:39:43.083 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-16T15:39:43.083 INFO:teuthology.orchestra.run.smithi151.stdout:invalid new device string format 2024-09-16T15:39:43.674 INFO:teuthology.orchestra.run.smithi151.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-16T15:39:43.674 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-16T15:39:43.674 INFO:teuthology.orchestra.run.smithi151.stdout:invalid new device string format 2024-09-16T15:39:43.674 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: resolved to: "172.21.15.151:3300,172.21.15.167:3300,172.21.15.167:3301" 2024-09-16T15:39:43.674 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-16T15:39:43.701 INFO:teuthology.orchestra.run.smithi151.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-16T15:39:43.701 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-16T15:39:43.701 INFO:teuthology.orchestra.run.smithi151.stdout:invalid new device string format 2024-09-16T15:39:43.701 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: resolved to: "172.21.15.151:3300,172.21.15.167:3300,172.21.15.167:3301" 2024-09-16T15:39:43.701 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-16T15:39:43.701 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: resolved to: "172.21.15.151:6789,172.21.15.167:6789,172.21.15.167:6790" 2024-09-16T15:39:43.701 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.151:6789,172.21.15.167:6789,172.21.15.167:6790:/ 2024-09-16T15:39:43.701 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=1757be18-e41a-4446-bb4f-ee4ac81c39df" will pass to kernel 2024-09-16T15:39:43.703 INFO:teuthology.orchestra.run.smithi151.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-16T15:39:43.703 INFO:teuthology.orchestra.run.smithi151.stdout: You just mounted a file system that supports labels which does not 2024-09-16T15:39:43.703 INFO:teuthology.orchestra.run.smithi151.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-16T15:39:43.703 INFO:teuthology.orchestra.run.smithi151.stdout: applications will generate AVC messages and not be allowed access to 2024-09-16T15:39:43.703 INFO:teuthology.orchestra.run.smithi151.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-16T15:39:43.706 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-16T15:39:43.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:43.706 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-16T15:39:43.763 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T15:39:43.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:43 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:39:43.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:39:43.864 INFO:teuthology.orchestra.run.smithi151.stdout:client4677 2024-09-16T15:39:43.864 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:39:43.864 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:39:43.900 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T15:39:44.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:44 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:39:44.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:39:44.039 INFO:teuthology.orchestra.run.smithi151.stdout:client4677 2024-09-16T15:39:44.040 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T15:39:44.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:44 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:39:44.131 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:39:44.139 INFO:teuthology.orchestra.run.smithi151.stdout:client4677 2024-09-16T15:39:44.139 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/1757be18-e41a-4446-bb4f-ee4ac81c39df.client4677/status') 2024-09-16T15:39:44.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:39:44.246 INFO:teuthology.orchestra.run.smithi151.stdout:instance: client.4677 (3)192.168.0.1:0/1418446671 2024-09-16T15:39:44.246 INFO:teuthology.orchestra.run.smithi151.stdout:blocklisted: false 2024-09-16T15:39:44.246 INFO:teuthology.orchestra.run.smithi151.stderr:0+1 records in 2024-09-16T15:39:44.247 INFO:teuthology.orchestra.run.smithi151.stderr:0+1 records out 2024-09-16T15:39:44.247 INFO:teuthology.orchestra.run.smithi151.stderr:69 bytes copied, 6.5914e-05 s, 1.0 MB/s 2024-09-16T15:39:44.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:39:44.257 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2024-09-16T15:39:44.257 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-16T15:39:44.257 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-16T15:39:44.258 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-16T15:39:44.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:44.413+0000 7fd289600640 1 -- 172.21.15.151:0/4123907332 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd2840b2440 msgr2=0x7fd28406f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:44.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:44.413+0000 7fd289600640 1 --2- 172.21.15.151:0/4123907332 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd2840b2440 0x7fd28406f160 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd27c005fd0 tx=0x7fd27c02aaa0 comp rx=0 tx=0).stop 2024-09-16T15:39:44.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:44.413+0000 7fd289600640 1 -- 172.21.15.151:0/4123907332 shutdown_connections 2024-09-16T15:39:44.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:44.413+0000 7fd289600640 1 --2- 172.21.15.151:0/4123907332 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd2840b2440 0x7fd28406f160 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:44.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:44.413+0000 7fd289600640 1 -- 172.21.15.151:0/4123907332 >> 172.21.15.151:0/4123907332 conn(0x7fd2840b1950 msgr2=0x7fd2840b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:44.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:44.413+0000 7fd289600640 1 -- 172.21.15.151:0/4123907332 shutdown_connections 2024-09-16T15:39:44.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:44.413+0000 7fd289600640 1 -- 172.21.15.151:0/4123907332 wait complete. 2024-09-16T15:39:44.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:44.415+0000 7fd289600640 1 Processor -- start 2024-09-16T15:39:44.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:44.415+0000 7fd289600640 1 -- start start 2024-09-16T15:39:44.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:44.416+0000 7fd289600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd2840b2440 0x7fd28415e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:44.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:44.416+0000 7fd289600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2840ccf00 0x7fd2840cc570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:44.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:44.416+0000 7fd289600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd28415f5a0 0x7fd28415ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:44.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:44.416+0000 7fd289600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd284074840 con 0x7fd2840b2440 2024-09-16T15:39:44.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:44.416+0000 7fd289600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd2840746c0 con 0x7fd2840ccf00 2024-09-16T15:39:44.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:44.416+0000 7fd289600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fd28406f6a0 con 0x7fd28415f5a0 2024-09-16T15:39:44.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:44.416+0000 7fd282a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd2840b2440 0x7fd28415e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:44.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:44.416+0000 7fd282a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd2840b2440 0x7fd28415e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:57592/0 (socket says 172.21.15.151:57592) 2024-09-16T15:39:44.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:44.416+0000 7fd282a00640 1 -- 172.21.15.151:0/2070689691 learned_addr learned my addr 172.21.15.151:0/2070689691 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:44.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:44.416+0000 7fd282000640 1 --2- 172.21.15.151:0/2070689691 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2840ccf00 0x7fd2840cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:44.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:44.416+0000 7fd283400640 1 --2- 172.21.15.151:0/2070689691 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd28415f5a0 0x7fd28415ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:44.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:44.417+0000 7fd282a00640 1 -- 172.21.15.151:0/2070689691 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd28415f5a0 msgr2=0x7fd28415ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:44.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:44.417+0000 7fd282a00640 1 --2- 172.21.15.151:0/2070689691 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd28415f5a0 0x7fd28415ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:44.419 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:44.417+0000 7fd282a00640 1 -- 172.21.15.151:0/2070689691 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2840ccf00 msgr2=0x7fd2840cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:44.419 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:44.417+0000 7fd282a00640 1 --2- 172.21.15.151:0/2070689691 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2840ccf00 0x7fd2840cc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:44.419 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:44.417+0000 7fd282a00640 1 -- 172.21.15.151:0/2070689691 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd28414b0c0 con 0x7fd2840b2440 2024-09-16T15:39:44.419 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:44.417+0000 7fd282000640 1 --2- 172.21.15.151:0/2070689691 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2840ccf00 0x7fd2840cc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:39:44.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:44.417+0000 7fd282a00640 1 --2- 172.21.15.151:0/2070689691 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd2840b2440 0x7fd28415e760 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd27400c400 tx=0x7fd27400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:44.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:44.417+0000 7fd27aa00640 1 -- 172.21.15.151:0/2070689691 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd274017070 con 0x7fd2840b2440 2024-09-16T15:39:44.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:44.417+0000 7fd27aa00640 1 -- 172.21.15.151:0/2070689691 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd2740053c0 con 0x7fd2840b2440 2024-09-16T15:39:44.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:44.417+0000 7fd289600640 1 -- 172.21.15.151:0/2070689691 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd28414b3a0 con 0x7fd2840b2440 2024-09-16T15:39:44.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:44.417+0000 7fd27aa00640 1 -- 172.21.15.151:0/2070689691 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd274003850 con 0x7fd2840b2440 2024-09-16T15:39:44.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:44.417+0000 7fd289600640 1 -- 172.21.15.151:0/2070689691 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd28415df60 con 0x7fd2840b2440 2024-09-16T15:39:44.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:44.418+0000 7fd289600640 1 -- 172.21.15.151:0/2070689691 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd28406f160 con 0x7fd2840b2440 2024-09-16T15:39:44.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:44.420+0000 7fd27aa00640 1 -- 172.21.15.151:0/2070689691 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fd27401a070 con 0x7fd2840b2440 2024-09-16T15:39:44.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:44.421+0000 7fd27aa00640 1 --2- 172.21.15.151:0/2070689691 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd264078850 0x7fd26407ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:44.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:44.421+0000 7fd27aa00640 1 -- 172.21.15.151:0/2070689691 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fd274099cc0 con 0x7fd2840b2440 2024-09-16T15:39:44.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:44.421+0000 7fd282000640 1 --2- 172.21.15.151:0/2070689691 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd264078850 0x7fd26407ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:44.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:44.422+0000 7fd282000640 1 --2- 172.21.15.151:0/2070689691 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd264078850 0x7fd26407ad10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd28409aab0 tx=0x7fd270009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:44.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:44.425+0000 7fd27aa00640 1 -- 172.21.15.151:0/2070689691 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd2740a0080 con 0x7fd2840b2440 2024-09-16T15:39:44.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:44.650+0000 7fd289600640 1 -- 172.21.15.151:0/2070689691 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd2840b1030 con 0x7fd2840b2440 2024-09-16T15:39:44.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:44.650+0000 7fd27aa00640 1 -- 172.21.15.151:0/2070689691 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7fd274061c40 con 0x7fd2840b2440 2024-09-16T15:39:44.652 INFO:teuthology.orchestra.run.smithi151.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-16T15:39:44.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:44.653+0000 7fd289600640 1 -- 172.21.15.151:0/2070689691 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd264078850 msgr2=0x7fd26407ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:44.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:44.653+0000 7fd289600640 1 --2- 172.21.15.151:0/2070689691 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd264078850 0x7fd26407ad10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd28409aab0 tx=0x7fd270009040 comp rx=0 tx=0).stop 2024-09-16T15:39:44.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:44.653+0000 7fd289600640 1 -- 172.21.15.151:0/2070689691 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd2840b2440 msgr2=0x7fd28415e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:44.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:44.653+0000 7fd289600640 1 --2- 172.21.15.151:0/2070689691 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd2840b2440 0x7fd28415e760 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd27400c400 tx=0x7fd27400c8c0 comp rx=0 tx=0).stop 2024-09-16T15:39:44.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:44.653+0000 7fd289600640 1 -- 172.21.15.151:0/2070689691 shutdown_connections 2024-09-16T15:39:44.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:44.653+0000 7fd289600640 1 --2- 172.21.15.151:0/2070689691 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd264078850 0x7fd26407ad10 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:44.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:44.653+0000 7fd289600640 1 --2- 172.21.15.151:0/2070689691 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2840ccf00 0x7fd2840cc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:44.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:44.653+0000 7fd289600640 1 --2- 172.21.15.151:0/2070689691 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd28415f5a0 0x7fd28415ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:44.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:44.653+0000 7fd289600640 1 --2- 172.21.15.151:0/2070689691 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd2840b2440 0x7fd28415e760 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:44.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:44.653+0000 7fd289600640 1 -- 172.21.15.151:0/2070689691 >> 172.21.15.151:0/2070689691 conn(0x7fd2840b1950 msgr2=0x7fd2840cdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:44.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:44.653+0000 7fd289600640 1 -- 172.21.15.151:0/2070689691 shutdown_connections 2024-09-16T15:39:44.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:44.653+0000 7fd289600640 1 -- 172.21.15.151:0/2070689691 wait complete. 2024-09-16T15:39:44.670 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-16T15:39:44.670 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T15:39:44.670 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi151.front.sepia.ceph.com 2024-09-16T15:39:44.670 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-09-16T15:39:44.670 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-16T15:39:44.670 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-16T15:39:44.670 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-16T15:39:44.670 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T15:39:44.670 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-09-16T15:39:44.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:44.671 DEBUG:teuthology.orchestra.run.smithi151:> ip addr 2024-09-16T15:39:44.706 INFO:teuthology.orchestra.run.smithi151.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T15:39:44.706 INFO:teuthology.orchestra.run.smithi151.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T15:39:44.706 INFO:teuthology.orchestra.run.smithi151.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T15:39:44.706 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:39:44.706 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 ::1/128 scope host 2024-09-16T15:39:44.706 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:39:44.706 INFO:teuthology.orchestra.run.smithi151.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:39:44.706 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:da:6e:0c brd ff:ff:ff:ff:ff:ff 2024-09-16T15:39:44.706 INFO:teuthology.orchestra.run.smithi151.stdout: altname enp7s0f0 2024-09-16T15:39:44.706 INFO:teuthology.orchestra.run.smithi151.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:39:44.707 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:da:6e:0d brd ff:ff:ff:ff:ff:ff 2024-09-16T15:39:44.707 INFO:teuthology.orchestra.run.smithi151.stdout: altname enp7s0f1 2024-09-16T15:39:44.707 INFO:teuthology.orchestra.run.smithi151.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:39:44.707 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:88:6f:62 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:39:44.707 INFO:teuthology.orchestra.run.smithi151.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T15:39:44.707 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:88:6f:63 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:39:44.707 INFO:teuthology.orchestra.run.smithi151.stdout: inet 172.21.15.151/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-16T15:39:44.707 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft 42705sec preferred_lft 42705sec 2024-09-16T15:39:44.707 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 fe80::ec4:7aff:fe88:6f63/64 scope link noprefixroute 2024-09-16T15:39:44.707 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:39:44.707 INFO:teuthology.orchestra.run.smithi151.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-16T15:39:44.707 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 62:3f:3b:48:ad:f8 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:39:44.707 INFO:teuthology.orchestra.run.smithi151.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-16T15:39:44.707 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:39:44.707 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 fe80::40fa:7fff:fedf:2a00/64 scope link 2024-09-16T15:39:44.708 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:39:44.708 INFO:teuthology.orchestra.run.smithi151.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T15:39:44.708 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 62:3f:3b:48:ad:f8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T15:39:44.708 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 fe80::603f:3bff:fe48:adf8/64 scope link 2024-09-16T15:39:44.708 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:39:44.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:44.709 DEBUG:teuthology.orchestra.run.smithi151:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T15:39:44.802 INFO:teuthology.orchestra.run.smithi151.stdout:1 2024-09-16T15:39:44.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:44.805 DEBUG:teuthology.orchestra.run.smithi151:> route 2024-09-16T15:39:44.841 INFO:teuthology.orchestra.run.smithi151.stdout:Kernel IP routing table 2024-09-16T15:39:44.842 INFO:teuthology.orchestra.run.smithi151.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T15:39:44.842 INFO:teuthology.orchestra.run.smithi151.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-16T15:39:44.842 INFO:teuthology.orchestra.run.smithi151.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-16T15:39:44.842 INFO:teuthology.orchestra.run.smithi151.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T15:39:44.842 DEBUG:teuthology.orchestra.run.smithi151:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T15:39:44.842 DEBUG:teuthology.orchestra.run.smithi151:> set -e 2024-09-16T15:39:44.842 DEBUG:teuthology.orchestra.run.smithi151:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-16T15:39:44.842 DEBUG:teuthology.orchestra.run.smithi151:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-16T15:39:44.842 DEBUG:teuthology.orchestra.run.smithi151:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-16T15:39:44.842 DEBUG:teuthology.orchestra.run.smithi151:> ') 2024-09-16T15:39:44.959 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:39:45.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:39:45.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:45.039 DEBUG:teuthology.orchestra.run.smithi151:> ip netns list 2024-09-16T15:39:45.074 INFO:teuthology.orchestra.run.smithi151.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-16T15:39:45.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:45.074 DEBUG:teuthology.orchestra.run.smithi151:> ip netns list-id 2024-09-16T15:39:45.148 INFO:teuthology.orchestra.run.smithi151.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-09-16T15:39:45.149 DEBUG:teuthology.orchestra.run.smithi151:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T15:39:45.149 DEBUG:teuthology.orchestra.run.smithi151:> set -e 2024-09-16T15:39:45.149 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-16T15:39:45.149 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-09-16T15:39:45.149 DEBUG:teuthology.orchestra.run.smithi151:> ') 2024-09-16T15:39:45.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:39:45.316 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:39:45.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:45.324 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-16T15:39:45.380 INFO:teuthology.orchestra.run.smithi151.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T15:39:45.380 INFO:teuthology.orchestra.run.smithi151.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T15:39:45.380 INFO:teuthology.orchestra.run.smithi151.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T15:39:45.380 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:39:45.380 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 ::1/128 scope host 2024-09-16T15:39:45.380 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:39:45.380 INFO:teuthology.orchestra.run.smithi151.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-16T15:39:45.380 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 02:4c:5d:13:e7:51 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-16T15:39:45.380 INFO:teuthology.orchestra.run.smithi151.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-16T15:39:45.380 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:39:45.380 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 fe80::4c:5dff:fe13:e751/64 scope link 2024-09-16T15:39:45.380 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:39:45.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:45.383 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-16T15:39:45.438 INFO:teuthology.orchestra.run.smithi151.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T15:39:45.438 INFO:teuthology.orchestra.run.smithi151.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T15:39:45.438 INFO:teuthology.orchestra.run.smithi151.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T15:39:45.438 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:39:45.438 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 ::1/128 scope host 2024-09-16T15:39:45.438 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:39:45.438 INFO:teuthology.orchestra.run.smithi151.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-16T15:39:45.438 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 02:4c:5d:13:e7:51 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-16T15:39:45.439 INFO:teuthology.orchestra.run.smithi151.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-16T15:39:45.439 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:39:45.439 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 fe80::4c:5dff:fe13:e751/64 scope link 2024-09-16T15:39:45.439 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:39:45.441 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-09-16T15:39:45.441 DEBUG:teuthology.orchestra.run.smithi151:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T15:39:45.441 DEBUG:teuthology.orchestra.run.smithi151:> set -e 2024-09-16T15:39:45.441 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-09-16T15:39:45.441 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-16T15:39:45.441 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-09-16T15:39:45.442 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-09-16T15:39:45.442 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-09-16T15:39:45.442 DEBUG:teuthology.orchestra.run.smithi151:> ') 2024-09-16T15:39:45.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:39:45.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:39:45.661 DEBUG:teuthology.orchestra.run.smithi151:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T15:39:45.661 DEBUG:teuthology.orchestra.run.smithi151:> set -e 2024-09-16T15:39:45.661 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip link set brx.1 up 2024-09-16T15:39:45.661 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip link set dev brx.1 master ceph-brx 2024-09-16T15:39:45.661 DEBUG:teuthology.orchestra.run.smithi151:> ') 2024-09-16T15:39:45.739 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:39:45.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:39:45.828 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:39:45.828 DEBUG:teuthology.orchestra.run.smithi151:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-09-16T15:39:45.863 INFO:teuthology.orchestra.run.smithi151.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-09-16T15:39:45.863 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:39:45.863 DEBUG:teuthology.orchestra.run.smithi151:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-09-16T15:39:45.938 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-16T15:39:45.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:45.938 DEBUG:teuthology.orchestra.run.smithi151:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-09-16T15:39:46.067 INFO:teuthology.orchestra.run.smithi151.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-09-16T15:39:46.067 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-09-16T15:39:46.067 INFO:teuthology.orchestra.run.smithi151.stdout:invalid new device string format 2024-09-16T15:39:46.083 INFO:teuthology.orchestra.run.smithi151.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-09-16T15:39:46.083 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-09-16T15:39:46.083 INFO:teuthology.orchestra.run.smithi151.stdout:invalid new device string format 2024-09-16T15:39:46.083 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: resolved to: "172.21.15.151:3300,172.21.15.167:3300,172.21.15.167:3301" 2024-09-16T15:39:46.083 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-16T15:39:46.107 INFO:teuthology.orchestra.run.smithi151.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-09-16T15:39:46.107 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-09-16T15:39:46.107 INFO:teuthology.orchestra.run.smithi151.stdout:invalid new device string format 2024-09-16T15:39:46.107 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: resolved to: "172.21.15.151:3300,172.21.15.167:3300,172.21.15.167:3301" 2024-09-16T15:39:46.107 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-16T15:39:46.108 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: resolved to: "172.21.15.151:6789,172.21.15.167:6789,172.21.15.167:6790" 2024-09-16T15:39:46.108 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.151:6789,172.21.15.167:6789,172.21.15.167:6790:/ 2024-09-16T15:39:46.108 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=1757be18-e41a-4446-bb4f-ee4ac81c39df" will pass to kernel 2024-09-16T15:39:46.110 INFO:teuthology.orchestra.run.smithi151.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-09-16T15:39:46.110 INFO:teuthology.orchestra.run.smithi151.stdout: You just mounted a file system that supports labels which does not 2024-09-16T15:39:46.110 INFO:teuthology.orchestra.run.smithi151.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-16T15:39:46.110 INFO:teuthology.orchestra.run.smithi151.stdout: applications will generate AVC messages and not be allowed access to 2024-09-16T15:39:46.110 INFO:teuthology.orchestra.run.smithi151.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-16T15:39:46.113 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-16T15:39:46.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:46.113 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-09-16T15:39:46.168 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T15:39:46.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:46 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:39:46.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:39:46.258 INFO:teuthology.orchestra.run.smithi151.stdout:client4544 2024-09-16T15:39:46.259 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:39:46.259 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:39:46.294 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T15:39:46.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:46 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:39:46.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:39:46.424 INFO:teuthology.orchestra.run.smithi151.stdout:client4544 2024-09-16T15:39:46.425 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T15:39:46.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:46 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:39:46.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:39:46.514 INFO:teuthology.orchestra.run.smithi151.stdout:client4544 2024-09-16T15:39:46.514 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/1757be18-e41a-4446-bb4f-ee4ac81c39df.client4544/status') 2024-09-16T15:39:46.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:39:46.616 INFO:teuthology.orchestra.run.smithi151.stdout:instance: client.4544 (3)172.21.15.151:0/2341439364 2024-09-16T15:39:46.616 INFO:teuthology.orchestra.run.smithi151.stdout:blocklisted: false 2024-09-16T15:39:46.617 INFO:teuthology.orchestra.run.smithi151.stderr:0+1 records in 2024-09-16T15:39:46.617 INFO:teuthology.orchestra.run.smithi151.stderr:0+1 records out 2024-09-16T15:39:46.617 INFO:teuthology.orchestra.run.smithi151.stderr:71 bytes copied, 5.2289e-05 s, 1.4 MB/s 2024-09-16T15:39:46.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:39:46.627 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-09-16T15:39:46.627 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-16T15:39:46.627 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-16T15:39:46.627 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-16T15:39:46.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:46.783+0000 7f1574a00640 1 -- 172.21.15.151:0/1792337600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15700b2440 msgr2=0x7f157006f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:46.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:46.783+0000 7f1574a00640 1 --2- 172.21.15.151:0/1792337600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15700b2440 0x7f157006f160 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1568005fd0 tx=0x7f156802a8b0 comp rx=0 tx=0).stop 2024-09-16T15:39:46.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:46.784+0000 7f1574a00640 1 -- 172.21.15.151:0/1792337600 shutdown_connections 2024-09-16T15:39:46.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:46.784+0000 7f1574a00640 1 --2- 172.21.15.151:0/1792337600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15700b2440 0x7f157006f160 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:46.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:46.784+0000 7f1574a00640 1 -- 172.21.15.151:0/1792337600 >> 172.21.15.151:0/1792337600 conn(0x7f15700b1950 msgr2=0x7f15700b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:46.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:46.784+0000 7f1574a00640 1 -- 172.21.15.151:0/1792337600 shutdown_connections 2024-09-16T15:39:46.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:46.784+0000 7f1574a00640 1 -- 172.21.15.151:0/1792337600 wait complete. 2024-09-16T15:39:46.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:46.785+0000 7f1574a00640 1 Processor -- start 2024-09-16T15:39:46.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:46.785+0000 7f1574a00640 1 -- start start 2024-09-16T15:39:46.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:46.786+0000 7f1574a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15700b2440 0x7f157015e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:46.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:46.786+0000 7f1574a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15700ccf00 0x7f1570142210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:46.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:46.786+0000 7f1574a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f157015f5a0 0x7f1570142750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:46.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:46.786+0000 7f1574a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1570074840 con 0x7f15700b2440 2024-09-16T15:39:46.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:46.786+0000 7f1574a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f15700746c0 con 0x7f15700ccf00 2024-09-16T15:39:46.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:46.786+0000 7f1574a00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f157006f6a0 con 0x7f157015f5a0 2024-09-16T15:39:46.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:46.786+0000 7f156de00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15700b2440 0x7f157015e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:46.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:46.787+0000 7f156de00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15700b2440 0x7f157015e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:57600/0 (socket says 172.21.15.151:57600) 2024-09-16T15:39:46.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:46.787+0000 7f156de00640 1 -- 172.21.15.151:0/1280808900 learned_addr learned my addr 172.21.15.151:0/1280808900 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:46.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:46.787+0000 7f156e800640 1 --2- 172.21.15.151:0/1280808900 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f157015f5a0 0x7f1570142750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:46.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:46.787+0000 7f156d400640 1 --2- 172.21.15.151:0/1280808900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15700ccf00 0x7f1570142210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:46.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:46.787+0000 7f156de00640 1 -- 172.21.15.151:0/1280808900 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f157015f5a0 msgr2=0x7f1570142750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:46.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:46.787+0000 7f156de00640 1 --2- 172.21.15.151:0/1280808900 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f157015f5a0 0x7f1570142750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:46.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:46.787+0000 7f156de00640 1 -- 172.21.15.151:0/1280808900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15700ccf00 msgr2=0x7f1570142210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:46.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:46.787+0000 7f156de00640 1 --2- 172.21.15.151:0/1280808900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15700ccf00 0x7f1570142210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:46.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:46.787+0000 7f156de00640 1 -- 172.21.15.151:0/1280808900 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15701f53f0 con 0x7f15700b2440 2024-09-16T15:39:46.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:46.787+0000 7f156e800640 1 --2- 172.21.15.151:0/1280808900 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f157015f5a0 0x7f1570142750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:39:46.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:46.787+0000 7f156d400640 1 --2- 172.21.15.151:0/1280808900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15700ccf00 0x7f1570142210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:39:46.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:46.787+0000 7f156de00640 1 --2- 172.21.15.151:0/1280808900 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15700b2440 0x7f157015e1b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f156000c410 tx=0x7f156000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:46.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:46.787+0000 7f1566000640 1 -- 172.21.15.151:0/1280808900 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1560011790 con 0x7f15700b2440 2024-09-16T15:39:46.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:46.787+0000 7f1566000640 1 -- 172.21.15.151:0/1280808900 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1560017070 con 0x7f15700b2440 2024-09-16T15:39:46.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:46.788+0000 7f1566000640 1 -- 172.21.15.151:0/1280808900 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1560011d40 con 0x7f15700b2440 2024-09-16T15:39:46.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:46.788+0000 7f1574a00640 1 -- 172.21.15.151:0/1280808900 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f157015a6b0 con 0x7f15700b2440 2024-09-16T15:39:46.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:46.788+0000 7f1574a00640 1 -- 172.21.15.151:0/1280808900 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f157015ac70 con 0x7f15700b2440 2024-09-16T15:39:46.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:46.789+0000 7f1574a00640 1 -- 172.21.15.151:0/1280808900 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f157015a4a0 con 0x7f15700b2440 2024-09-16T15:39:46.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:46.790+0000 7f1566000640 1 -- 172.21.15.151:0/1280808900 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f156001c460 con 0x7f15700b2440 2024-09-16T15:39:46.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:46.791+0000 7f1566000640 1 --2- 172.21.15.151:0/1280808900 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1550078850 0x7f155007ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:46.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:46.791+0000 7f1566000640 1 -- 172.21.15.151:0/1280808900 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f156009a730 con 0x7f15700b2440 2024-09-16T15:39:46.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:46.791+0000 7f156d400640 1 --2- 172.21.15.151:0/1280808900 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1550078850 0x7f155007ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:46.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:46.792+0000 7f156d400640 1 --2- 172.21.15.151:0/1280808900 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1550078850 0x7f155007ad10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f155c000c00 tx=0x7f155c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:46.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:46.795+0000 7f1566000640 1 -- 172.21.15.151:0/1280808900 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1560062d90 con 0x7f15700b2440 2024-09-16T15:39:46.872 DEBUG:teuthology.orchestra.run.smithi151:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:39:46.877 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:39:47.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:47.027+0000 7f1574a00640 1 -- 172.21.15.151:0/1280808900 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1570153ec0 con 0x7f15700b2440 2024-09-16T15:39:47.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:47.028+0000 7f1566000640 1 -- 172.21.15.151:0/1280808900 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f1560062690 con 0x7f15700b2440 2024-09-16T15:39:47.029 INFO:teuthology.orchestra.run.smithi151.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-16T15:39:47.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:47.030+0000 7f1574a00640 1 -- 172.21.15.151:0/1280808900 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1550078850 msgr2=0x7f155007ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:47.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:47.030+0000 7f1574a00640 1 --2- 172.21.15.151:0/1280808900 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1550078850 0x7f155007ad10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f155c000c00 tx=0x7f155c009040 comp rx=0 tx=0).stop 2024-09-16T15:39:47.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:47.030+0000 7f1574a00640 1 -- 172.21.15.151:0/1280808900 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15700b2440 msgr2=0x7f157015e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:47.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:47.030+0000 7f1574a00640 1 --2- 172.21.15.151:0/1280808900 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15700b2440 0x7f157015e1b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f156000c410 tx=0x7f156000c8d0 comp rx=0 tx=0).stop 2024-09-16T15:39:47.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:47.030+0000 7f1574a00640 1 -- 172.21.15.151:0/1280808900 shutdown_connections 2024-09-16T15:39:47.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:47.030+0000 7f1574a00640 1 --2- 172.21.15.151:0/1280808900 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1550078850 0x7f155007ad10 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:47.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:47.030+0000 7f1574a00640 1 --2- 172.21.15.151:0/1280808900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15700ccf00 0x7f1570142210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:47.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:47.030+0000 7f1574a00640 1 --2- 172.21.15.151:0/1280808900 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f157015f5a0 0x7f1570142750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:47.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:47.030+0000 7f1574a00640 1 --2- 172.21.15.151:0/1280808900 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15700b2440 0x7f157015e1b0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:47.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:47.030+0000 7f1574a00640 1 -- 172.21.15.151:0/1280808900 >> 172.21.15.151:0/1280808900 conn(0x7f15700b1950 msgr2=0x7f157009cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:47.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:47.030+0000 7f1574a00640 1 -- 172.21.15.151:0/1280808900 shutdown_connections 2024-09-16T15:39:47.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:47.030+0000 7f1574a00640 1 -- 172.21.15.151:0/1280808900 wait complete. 2024-09-16T15:39:47.048 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-16T15:39:47.048 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T15:39:47.048 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi167.front.sepia.ceph.com 2024-09-16T15:39:47.048 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-16T15:39:47.048 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-16T15:39:47.049 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-16T15:39:47.049 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-16T15:39:47.049 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T15:39:47.049 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-16T15:39:47.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:47.049 DEBUG:teuthology.orchestra.run.smithi167:> ip addr 2024-09-16T15:39:47.086 INFO:teuthology.orchestra.run.smithi167.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T15:39:47.087 INFO:teuthology.orchestra.run.smithi167.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T15:39:47.087 INFO:teuthology.orchestra.run.smithi167.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T15:39:47.087 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:39:47.087 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 ::1/128 scope host 2024-09-16T15:39:47.087 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:39:47.087 INFO:teuthology.orchestra.run.smithi167.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:39:47.087 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e6 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:39:47.087 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f0 2024-09-16T15:39:47.087 INFO:teuthology.orchestra.run.smithi167.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:39:47.087 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e7 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:39:47.087 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f1 2024-09-16T15:39:47.087 INFO:teuthology.orchestra.run.smithi167.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:39:47.087 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:08 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:39:47.087 INFO:teuthology.orchestra.run.smithi167.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T15:39:47.088 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:09 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:39:47.088 INFO:teuthology.orchestra.run.smithi167.stdout: inet 172.21.15.167/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-16T15:39:47.088 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft 42698sec preferred_lft 42698sec 2024-09-16T15:39:47.088 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::ec4:7aff:fe88:7309/64 scope link noprefixroute 2024-09-16T15:39:47.088 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:39:47.089 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-16T15:39:47.089 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T15:39:47.089 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2024-09-16T15:39:47.089 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link add name ceph-brx type bridge 2024-09-16T15:39:47.089 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip addr flush dev ceph-brx 2024-09-16T15:39:47.089 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link set ceph-brx up 2024-09-16T15:39:47.089 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-16T15:39:47.089 DEBUG:teuthology.orchestra.run.smithi167:> ') 2024-09-16T15:39:47.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:39:47.508 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:39:47.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:47.517 DEBUG:teuthology.orchestra.run.smithi167:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T15:39:47.575 INFO:teuthology.orchestra.run.smithi167.stdout:1 2024-09-16T15:39:47.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:47.579 DEBUG:teuthology.orchestra.run.smithi167:> route 2024-09-16T15:39:47.624 INFO:teuthology.orchestra.run.smithi167.stdout:Kernel IP routing table 2024-09-16T15:39:47.624 INFO:teuthology.orchestra.run.smithi167.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T15:39:47.625 INFO:teuthology.orchestra.run.smithi167.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-16T15:39:47.625 INFO:teuthology.orchestra.run.smithi167.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-16T15:39:47.625 INFO:teuthology.orchestra.run.smithi167.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T15:39:47.625 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T15:39:47.625 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2024-09-16T15:39:47.625 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-16T15:39:47.625 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-16T15:39:47.625 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-16T15:39:47.625 DEBUG:teuthology.orchestra.run.smithi167:> ') 2024-09-16T15:39:47.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:39:48.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:39:48.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:48.292 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list 2024-09-16T15:39:48.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:48.329 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list-id 2024-09-16T15:39:48.405 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T15:39:48.405 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2024-09-16T15:39:48.405 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-16T15:39:48.405 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-09-16T15:39:48.405 DEBUG:teuthology.orchestra.run.smithi167:> ') 2024-09-16T15:39:48.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:39:48.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:39:48.586 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-09-16T15:39:48.587 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T15:39:48.587 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2024-09-16T15:39:48.587 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-09-16T15:39:48.587 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-16T15:39:48.587 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-09-16T15:39:48.587 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-09-16T15:39:48.587 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-09-16T15:39:48.587 DEBUG:teuthology.orchestra.run.smithi167:> ') 2024-09-16T15:39:48.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:39:48.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:39:48.835 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T15:39:48.835 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2024-09-16T15:39:48.835 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link set brx.0 up 2024-09-16T15:39:48.835 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link set dev brx.0 master ceph-brx 2024-09-16T15:39:48.835 DEBUG:teuthology.orchestra.run.smithi167:> ') 2024-09-16T15:39:48.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:39:48.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:39:49.004 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:39:49.004 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-16T15:39:49.040 INFO:teuthology.orchestra.run.smithi167.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-16T15:39:49.040 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:39:49.040 DEBUG:teuthology.orchestra.run.smithi167:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-16T15:39:49.115 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-16T15:39:49.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:49.115 DEBUG:teuthology.orchestra.run.smithi167:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-16T15:39:49.254 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-16T15:39:49.254 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-16T15:39:49.254 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2024-09-16T15:39:49.905 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-16T15:39:49.905 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-16T15:39:49.905 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2024-09-16T15:39:49.905 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.151:3300,172.21.15.167:3300,172.21.15.167:3301" 2024-09-16T15:39:49.905 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-16T15:39:49.931 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-16T15:39:49.931 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-16T15:39:49.931 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2024-09-16T15:39:49.932 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.151:3300,172.21.15.167:3300,172.21.15.167:3301" 2024-09-16T15:39:49.932 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-16T15:39:49.932 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.151:6789,172.21.15.167:6789,172.21.15.167:6790" 2024-09-16T15:39:49.932 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.151:6789,172.21.15.167:6789,172.21.15.167:6790:/ 2024-09-16T15:39:49.932 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=1757be18-e41a-4446-bb4f-ee4ac81c39df" will pass to kernel 2024-09-16T15:39:49.934 INFO:teuthology.orchestra.run.smithi167.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-16T15:39:49.934 INFO:teuthology.orchestra.run.smithi167.stdout: You just mounted a file system that supports labels which does not 2024-09-16T15:39:49.935 INFO:teuthology.orchestra.run.smithi167.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-16T15:39:49.935 INFO:teuthology.orchestra.run.smithi167.stdout: applications will generate AVC messages and not be allowed access to 2024-09-16T15:39:49.935 INFO:teuthology.orchestra.run.smithi167.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-16T15:39:49.937 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-16T15:39:49.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:49.937 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-16T15:39:49.994 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T15:39:50.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:50 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:39:50.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:39:50.086 INFO:teuthology.orchestra.run.smithi167.stdout:client4695 2024-09-16T15:39:50.086 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-16T15:39:50.086 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:39:50.122 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T15:39:50.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:50 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:39:50.243 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:39:50.250 INFO:teuthology.orchestra.run.smithi167.stdout:client4695 2024-09-16T15:39:50.251 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T15:39:50.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:50 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:39:50.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:39:50.338 INFO:teuthology.orchestra.run.smithi167.stdout:client4695 2024-09-16T15:39:50.339 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/1757be18-e41a-4446-bb4f-ee4ac81c39df.client4695/status') 2024-09-16T15:39:50.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:39:50.436 INFO:teuthology.orchestra.run.smithi167.stdout:instance: client.4695 (3)172.21.15.167:0/3064675131 2024-09-16T15:39:50.437 INFO:teuthology.orchestra.run.smithi167.stdout:blocklisted: false 2024-09-16T15:39:50.437 INFO:teuthology.orchestra.run.smithi167.stderr:0+1 records in 2024-09-16T15:39:50.437 INFO:teuthology.orchestra.run.smithi167.stderr:0+1 records out 2024-09-16T15:39:50.437 INFO:teuthology.orchestra.run.smithi167.stderr:71 bytes copied, 5.1956e-05 s, 1.4 MB/s 2024-09-16T15:39:50.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:39:50.447 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2024-09-16T15:39:50.447 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-16T15:39:50.447 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-16T15:39:50.448 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-16T15:39:50.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:50.614+0000 7fcf1de00640 1 -- 172.21.15.151:0/4208715655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf180ccf00 msgr2=0x7fcf180b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:50.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:50.614+0000 7fcf1de00640 1 --2- 172.21.15.151:0/4208715655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf180ccf00 0x7fcf180b2440 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fcf04005fd0 tx=0x7fcf0402a540 comp rx=0 tx=0).stop 2024-09-16T15:39:50.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:50.614+0000 7fcf1de00640 1 -- 172.21.15.151:0/4208715655 shutdown_connections 2024-09-16T15:39:50.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:50.614+0000 7fcf1de00640 1 --2- 172.21.15.151:0/4208715655 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcf18073fb0 0x7fcf18074440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:50.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:50.614+0000 7fcf1de00640 1 --2- 172.21.15.151:0/4208715655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf180ccf00 0x7fcf180b2440 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:50.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:50.614+0000 7fcf1de00640 1 -- 172.21.15.151:0/4208715655 >> 172.21.15.151:0/4208715655 conn(0x7fcf180b1950 msgr2=0x7fcf180b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:50.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:50.614+0000 7fcf1de00640 1 -- 172.21.15.151:0/4208715655 shutdown_connections 2024-09-16T15:39:50.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:50.614+0000 7fcf1de00640 1 -- 172.21.15.151:0/4208715655 wait complete. 2024-09-16T15:39:50.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:50.616+0000 7fcf1de00640 1 Processor -- start 2024-09-16T15:39:50.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:50.616+0000 7fcf1de00640 1 -- start start 2024-09-16T15:39:50.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:50.617+0000 7fcf1de00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf18073fb0 0x7fcf181f8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:50.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:50.617+0000 7fcf1de00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcf180ccf00 0x7fcf181f4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:50.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:50.617+0000 7fcf1de00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf1815f5a0 0x7fcf180c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:50.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:50.617+0000 7fcf1de00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcf1806f2e0 con 0x7fcf18073fb0 2024-09-16T15:39:50.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:50.617+0000 7fcf1de00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcf1806f160 con 0x7fcf1815f5a0 2024-09-16T15:39:50.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:50.617+0000 7fcf1de00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fcf1806f460 con 0x7fcf180ccf00 2024-09-16T15:39:50.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:50.618+0000 7fcf16a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf18073fb0 0x7fcf181f8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:50.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:50.618+0000 7fcf16a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf18073fb0 0x7fcf181f8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:57618/0 (socket says 172.21.15.151:57618) 2024-09-16T15:39:50.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:50.618+0000 7fcf16a00640 1 -- 172.21.15.151:0/1237358022 learned_addr learned my addr 172.21.15.151:0/1237358022 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:50.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:50.618+0000 7fcf17400640 1 --2- 172.21.15.151:0/1237358022 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf1815f5a0 0x7fcf180c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:50.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:50.618+0000 7fcf16000640 1 --2- 172.21.15.151:0/1237358022 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcf180ccf00 0x7fcf181f4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:50.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:50.618+0000 7fcf16a00640 1 -- 172.21.15.151:0/1237358022 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcf180ccf00 msgr2=0x7fcf181f4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:50.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:50.618+0000 7fcf16a00640 1 --2- 172.21.15.151:0/1237358022 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcf180ccf00 0x7fcf181f4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:50.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:50.618+0000 7fcf16a00640 1 -- 172.21.15.151:0/1237358022 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf1815f5a0 msgr2=0x7fcf180c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:50.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:50.618+0000 7fcf16a00640 1 --2- 172.21.15.151:0/1237358022 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf1815f5a0 0x7fcf180c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:50.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:50.618+0000 7fcf16a00640 1 -- 172.21.15.151:0/1237358022 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf18158f00 con 0x7fcf18073fb0 2024-09-16T15:39:50.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:50.618+0000 7fcf16000640 1 --2- 172.21.15.151:0/1237358022 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcf180ccf00 0x7fcf181f4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:39:50.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:50.618+0000 7fcf16a00640 1 --2- 172.21.15.151:0/1237358022 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf18073fb0 0x7fcf181f8580 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcf0800c3a0 tx=0x7fcf0800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:50.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:50.619+0000 7fcf0f400640 1 -- 172.21.15.151:0/1237358022 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf08017070 con 0x7fcf18073fb0 2024-09-16T15:39:50.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:50.619+0000 7fcf0f400640 1 -- 172.21.15.151:0/1237358022 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcf080053c0 con 0x7fcf18073fb0 2024-09-16T15:39:50.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:50.619+0000 7fcf0f400640 1 -- 172.21.15.151:0/1237358022 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf08003850 con 0x7fcf18073fb0 2024-09-16T15:39:50.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:50.619+0000 7fcf1de00640 1 -- 172.21.15.151:0/1237358022 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf181f8c10 con 0x7fcf18073fb0 2024-09-16T15:39:50.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:50.619+0000 7fcf1de00640 1 -- 172.21.15.151:0/1237358022 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf181f91d0 con 0x7fcf18073fb0 2024-09-16T15:39:50.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:50.620+0000 7fcf1de00640 1 -- 172.21.15.151:0/1237358022 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf1806f980 con 0x7fcf18073fb0 2024-09-16T15:39:50.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:50.621+0000 7fcf0f400640 1 -- 172.21.15.151:0/1237358022 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fcf0801a070 con 0x7fcf18073fb0 2024-09-16T15:39:50.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:50.622+0000 7fcf0f400640 1 --2- 172.21.15.151:0/1237358022 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcef8078850 0x7fcef807ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:50.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:50.622+0000 7fcf0f400640 1 -- 172.21.15.151:0/1237358022 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fcf08099c50 con 0x7fcf18073fb0 2024-09-16T15:39:50.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:50.622+0000 7fcf16000640 1 --2- 172.21.15.151:0/1237358022 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcef8078850 0x7fcef807ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:50.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:50.623+0000 7fcf16000640 1 --2- 172.21.15.151:0/1237358022 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcef8078850 0x7fcef807ad10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fcf04033000 tx=0x7fcf04004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:50.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:50.626+0000 7fcf0f400640 1 -- 172.21.15.151:0/1237358022 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fcf080622b0 con 0x7fcf18073fb0 2024-09-16T15:39:50.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:50.852+0000 7fcf1de00640 1 -- 172.21.15.151:0/1237358022 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fcf1809bbc0 con 0x7fcf18073fb0 2024-09-16T15:39:50.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:50.853+0000 7fcf0f400640 1 -- 172.21.15.151:0/1237358022 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7fcf08061bb0 con 0x7fcf18073fb0 2024-09-16T15:39:50.854 INFO:teuthology.orchestra.run.smithi151.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-16T15:39:50.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:50.855+0000 7fcf1de00640 1 -- 172.21.15.151:0/1237358022 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcef8078850 msgr2=0x7fcef807ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:50.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:50.855+0000 7fcf1de00640 1 --2- 172.21.15.151:0/1237358022 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcef8078850 0x7fcef807ad10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fcf04033000 tx=0x7fcf04004730 comp rx=0 tx=0).stop 2024-09-16T15:39:50.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:50.855+0000 7fcf1de00640 1 -- 172.21.15.151:0/1237358022 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf18073fb0 msgr2=0x7fcf181f8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:50.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:50.855+0000 7fcf1de00640 1 --2- 172.21.15.151:0/1237358022 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf18073fb0 0x7fcf181f8580 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcf0800c3a0 tx=0x7fcf0800c860 comp rx=0 tx=0).stop 2024-09-16T15:39:50.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:50.855+0000 7fcf1de00640 1 -- 172.21.15.151:0/1237358022 shutdown_connections 2024-09-16T15:39:50.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:50.855+0000 7fcf1de00640 1 --2- 172.21.15.151:0/1237358022 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcef8078850 0x7fcef807ad10 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:50.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:50.855+0000 7fcf1de00640 1 --2- 172.21.15.151:0/1237358022 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf1815f5a0 0x7fcf180c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:50.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:50.855+0000 7fcf1de00640 1 --2- 172.21.15.151:0/1237358022 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcf180ccf00 0x7fcf181f4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:50.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:50.855+0000 7fcf1de00640 1 --2- 172.21.15.151:0/1237358022 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf18073fb0 0x7fcf181f8580 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:50.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:50.855+0000 7fcf1de00640 1 -- 172.21.15.151:0/1237358022 >> 172.21.15.151:0/1237358022 conn(0x7fcf180b1950 msgr2=0x7fcf1809c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:50.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:50.855+0000 7fcf1de00640 1 -- 172.21.15.151:0/1237358022 shutdown_connections 2024-09-16T15:39:50.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:50.856+0000 7fcf1de00640 1 -- 172.21.15.151:0/1237358022 wait complete. 2024-09-16T15:39:50.873 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-16T15:39:50.873 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T15:39:50.873 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi167.front.sepia.ceph.com 2024-09-16T15:39:50.873 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-09-16T15:39:50.874 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-16T15:39:50.874 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-16T15:39:50.874 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-16T15:39:50.874 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T15:39:50.874 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-09-16T15:39:50.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:50.874 DEBUG:teuthology.orchestra.run.smithi167:> ip addr 2024-09-16T15:39:50.912 INFO:teuthology.orchestra.run.smithi167.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T15:39:50.912 INFO:teuthology.orchestra.run.smithi167.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T15:39:50.912 INFO:teuthology.orchestra.run.smithi167.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T15:39:50.912 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:39:50.912 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 ::1/128 scope host 2024-09-16T15:39:50.912 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:39:50.912 INFO:teuthology.orchestra.run.smithi167.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:39:50.912 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e6 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:39:50.912 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f0 2024-09-16T15:39:50.912 INFO:teuthology.orchestra.run.smithi167.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:39:50.913 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:d9:35:e7 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:39:50.913 INFO:teuthology.orchestra.run.smithi167.stdout: altname enp7s0f1 2024-09-16T15:39:50.913 INFO:teuthology.orchestra.run.smithi167.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:39:50.913 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:08 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:39:50.913 INFO:teuthology.orchestra.run.smithi167.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T15:39:50.913 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 0c:c4:7a:88:73:09 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:39:50.913 INFO:teuthology.orchestra.run.smithi167.stdout: inet 172.21.15.167/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-16T15:39:50.913 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft 42694sec preferred_lft 42694sec 2024-09-16T15:39:50.913 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::ec4:7aff:fe88:7309/64 scope link noprefixroute 2024-09-16T15:39:50.913 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:39:50.913 INFO:teuthology.orchestra.run.smithi167.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-16T15:39:50.913 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether de:95:34:7c:81:d1 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:39:50.914 INFO:teuthology.orchestra.run.smithi167.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-16T15:39:50.914 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:39:50.914 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::2431:17ff:fe63:f60f/64 scope link 2024-09-16T15:39:50.914 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:39:50.914 INFO:teuthology.orchestra.run.smithi167.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T15:39:50.914 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether de:95:34:7c:81:d1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-16T15:39:50.914 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::dc95:34ff:fe7c:81d1/64 scope link 2024-09-16T15:39:50.914 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:39:50.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:50.915 DEBUG:teuthology.orchestra.run.smithi167:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T15:39:51.008 INFO:teuthology.orchestra.run.smithi167.stdout:1 2024-09-16T15:39:51.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:51.012 DEBUG:teuthology.orchestra.run.smithi167:> route 2024-09-16T15:39:51.048 INFO:teuthology.orchestra.run.smithi167.stdout:Kernel IP routing table 2024-09-16T15:39:51.048 INFO:teuthology.orchestra.run.smithi167.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T15:39:51.048 INFO:teuthology.orchestra.run.smithi167.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-16T15:39:51.048 INFO:teuthology.orchestra.run.smithi167.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-16T15:39:51.048 INFO:teuthology.orchestra.run.smithi167.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T15:39:51.048 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T15:39:51.049 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2024-09-16T15:39:51.049 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-16T15:39:51.049 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-16T15:39:51.049 DEBUG:teuthology.orchestra.run.smithi167:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-16T15:39:51.049 DEBUG:teuthology.orchestra.run.smithi167:> ') 2024-09-16T15:39:51.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:39:51.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:39:51.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:51.245 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list 2024-09-16T15:39:51.280 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-16T15:39:51.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:51.281 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list-id 2024-09-16T15:39:51.355 INFO:teuthology.orchestra.run.smithi167.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-09-16T15:39:51.356 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T15:39:51.356 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2024-09-16T15:39:51.356 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-16T15:39:51.357 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-09-16T15:39:51.357 DEBUG:teuthology.orchestra.run.smithi167:> ') 2024-09-16T15:39:51.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:39:51.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:39:51.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:51.532 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-16T15:39:51.587 INFO:teuthology.orchestra.run.smithi167.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T15:39:51.587 INFO:teuthology.orchestra.run.smithi167.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T15:39:51.587 INFO:teuthology.orchestra.run.smithi167.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T15:39:51.587 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:39:51.587 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 ::1/128 scope host 2024-09-16T15:39:51.588 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:39:51.588 INFO:teuthology.orchestra.run.smithi167.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-16T15:39:51.588 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 9e:b8:86:a5:a5:27 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-16T15:39:51.588 INFO:teuthology.orchestra.run.smithi167.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-16T15:39:51.588 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:39:51.588 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::9cb8:86ff:fea5:a527/64 scope link 2024-09-16T15:39:51.588 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:39:51.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:51.590 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-16T15:39:51.646 INFO:teuthology.orchestra.run.smithi167.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T15:39:51.646 INFO:teuthology.orchestra.run.smithi167.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T15:39:51.646 INFO:teuthology.orchestra.run.smithi167.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T15:39:51.647 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:39:51.647 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 ::1/128 scope host 2024-09-16T15:39:51.647 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:39:51.647 INFO:teuthology.orchestra.run.smithi167.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-16T15:39:51.647 INFO:teuthology.orchestra.run.smithi167.stdout: link/ether 9e:b8:86:a5:a5:27 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-16T15:39:51.647 INFO:teuthology.orchestra.run.smithi167.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-16T15:39:51.647 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:39:51.647 INFO:teuthology.orchestra.run.smithi167.stdout: inet6 fe80::9cb8:86ff:fea5:a527/64 scope link 2024-09-16T15:39:51.647 INFO:teuthology.orchestra.run.smithi167.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:39:51.649 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-09-16T15:39:51.649 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T15:39:51.649 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2024-09-16T15:39:51.649 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-09-16T15:39:51.649 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-16T15:39:51.650 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-09-16T15:39:51.650 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-09-16T15:39:51.650 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-09-16T15:39:51.650 DEBUG:teuthology.orchestra.run.smithi167:> ') 2024-09-16T15:39:51.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:39:51.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:39:51.867 DEBUG:teuthology.orchestra.run.smithi167:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T15:39:51.867 DEBUG:teuthology.orchestra.run.smithi167:> set -e 2024-09-16T15:39:51.867 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link set brx.1 up 2024-09-16T15:39:51.867 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link set dev brx.1 master ceph-brx 2024-09-16T15:39:51.867 DEBUG:teuthology.orchestra.run.smithi167:> ') 2024-09-16T15:39:51.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:39:52.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:39:52.032 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:39:52.032 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-09-16T15:39:52.067 INFO:teuthology.orchestra.run.smithi167.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-09-16T15:39:52.068 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:39:52.068 DEBUG:teuthology.orchestra.run.smithi167:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-09-16T15:39:52.142 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-16T15:39:52.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:52.143 DEBUG:teuthology.orchestra.run.smithi167:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-09-16T15:39:52.271 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-09-16T15:39:52.271 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-09-16T15:39:52.271 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2024-09-16T15:39:52.287 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-09-16T15:39:52.287 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-09-16T15:39:52.287 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2024-09-16T15:39:52.287 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.151:3300,172.21.15.167:3300,172.21.15.167:3301" 2024-09-16T15:39:52.287 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-16T15:39:52.309 INFO:teuthology.orchestra.run.smithi167.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-09-16T15:39:52.309 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-09-16T15:39:52.309 INFO:teuthology.orchestra.run.smithi167.stdout:invalid new device string format 2024-09-16T15:39:52.309 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.151:3300,172.21.15.167:3300,172.21.15.167:3301" 2024-09-16T15:39:52.309 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-16T15:39:52.310 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: resolved to: "172.21.15.151:6789,172.21.15.167:6789,172.21.15.167:6790" 2024-09-16T15:39:52.310 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.151:6789,172.21.15.167:6789,172.21.15.167:6790:/ 2024-09-16T15:39:52.310 INFO:teuthology.orchestra.run.smithi167.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=1757be18-e41a-4446-bb4f-ee4ac81c39df" will pass to kernel 2024-09-16T15:39:52.311 INFO:teuthology.orchestra.run.smithi167.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-09-16T15:39:52.311 INFO:teuthology.orchestra.run.smithi167.stdout: You just mounted a file system that supports labels which does not 2024-09-16T15:39:52.311 INFO:teuthology.orchestra.run.smithi167.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-16T15:39:52.311 INFO:teuthology.orchestra.run.smithi167.stdout: applications will generate AVC messages and not be allowed access to 2024-09-16T15:39:52.311 INFO:teuthology.orchestra.run.smithi167.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-16T15:39:52.314 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-16T15:39:52.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:52.315 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-09-16T15:39:52.370 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T15:39:52.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:52 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:39:52.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:39:52.461 INFO:teuthology.orchestra.run.smithi167.stdout:client4550 2024-09-16T15:39:52.461 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-16T15:39:52.461 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:39:52.496 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T15:39:52.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:52 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:39:52.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:39:52.625 INFO:teuthology.orchestra.run.smithi167.stdout:client4550 2024-09-16T15:39:52.625 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T15:39:52.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:52 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:39:52.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:39:52.715 INFO:teuthology.orchestra.run.smithi167.stdout:client4550 2024-09-16T15:39:52.715 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/1757be18-e41a-4446-bb4f-ee4ac81c39df.client4550/status') 2024-09-16T15:39:52.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:39:52.813 INFO:teuthology.orchestra.run.smithi167.stdout:instance: client.4550 (3)192.168.0.2:0/2616634221 2024-09-16T15:39:52.813 INFO:teuthology.orchestra.run.smithi167.stdout:blocklisted: false 2024-09-16T15:39:52.813 INFO:teuthology.orchestra.run.smithi167.stderr:0+1 records in 2024-09-16T15:39:52.813 INFO:teuthology.orchestra.run.smithi167.stderr:0+1 records out 2024-09-16T15:39:52.814 INFO:teuthology.orchestra.run.smithi167.stderr:69 bytes copied, 5.273e-05 s, 1.3 MB/s 2024-09-16T15:39:52.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-16T15:39:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:39:52.824 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-09-16T15:39:52.907 INFO:tasks.cephfs_test_runner:Starting test: test_clone_to_diff_group_and_less_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) Initiate cloning where clone subvolume and source subvolume are located 2024-09-16T15:39:52.907 INFO:tasks.cephfs_test_runner:test_clone_to_diff_group_and_less_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-09-16T15:39:52.908 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:39:53.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.074+0000 7f8b5e400640 1 -- 172.21.15.151:0/422988574 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b580ad880 msgr2=0x7f8b580adcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:53.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.074+0000 7f8b5e400640 1 --2- 172.21.15.151:0/422988574 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b580ad880 0x7f8b580adcd0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8b44009830 tx=0x7f8b4402a510 comp rx=0 tx=0).stop 2024-09-16T15:39:53.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.074+0000 7f8b5e400640 1 -- 172.21.15.151:0/422988574 shutdown_connections 2024-09-16T15:39:53.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.074+0000 7f8b5e400640 1 --2- 172.21.15.151:0/422988574 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b580ad880 0x7f8b580adcd0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:53.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.074+0000 7f8b5e400640 1 --2- 172.21.15.151:0/422988574 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b5806fb90 0x7f8b5806ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:53.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.074+0000 7f8b5e400640 1 -- 172.21.15.151:0/422988574 >> 172.21.15.151:0/422988574 conn(0x7f8b580cfa20 msgr2=0x7f8b580cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:53.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.075+0000 7f8b5e400640 1 -- 172.21.15.151:0/422988574 shutdown_connections 2024-09-16T15:39:53.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.075+0000 7f8b5e400640 1 -- 172.21.15.151:0/422988574 wait complete. 2024-09-16T15:39:53.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.076+0000 7f8b5e400640 1 Processor -- start 2024-09-16T15:39:53.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.076+0000 7f8b5e400640 1 -- start start 2024-09-16T15:39:53.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.077+0000 7f8b5e400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b5806fb90 0x7f8b58155490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:53.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.077+0000 7f8b5e400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b580ad880 0x7f8b580b5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:53.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.077+0000 7f8b5e400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b580cd5f0 0x7f8b581f7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:53.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.077+0000 7f8b5e400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8b5815ed60 con 0x7f8b5806fb90 2024-09-16T15:39:53.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.077+0000 7f8b5e400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8b5815ebe0 con 0x7f8b580ad880 2024-09-16T15:39:53.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.077+0000 7f8b5e400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f8b5815eee0 con 0x7f8b580cd5f0 2024-09-16T15:39:53.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.077+0000 7f8b5d000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b5806fb90 0x7f8b58155490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:53.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.078+0000 7f8b5d000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b5806fb90 0x7f8b58155490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43434/0 (socket says 172.21.15.151:43434) 2024-09-16T15:39:53.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.078+0000 7f8b5d000640 1 -- 172.21.15.151:0/2070305332 learned_addr learned my addr 172.21.15.151:0/2070305332 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:53.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.078+0000 7f8b5da00640 1 --2- 172.21.15.151:0/2070305332 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b580cd5f0 0x7f8b581f7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:53.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.078+0000 7f8b57e00640 1 --2- 172.21.15.151:0/2070305332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b580ad880 0x7f8b580b5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:53.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.078+0000 7f8b5d000640 1 -- 172.21.15.151:0/2070305332 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b580cd5f0 msgr2=0x7f8b581f7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:53.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.078+0000 7f8b5d000640 1 --2- 172.21.15.151:0/2070305332 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b580cd5f0 0x7f8b581f7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:53.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.078+0000 7f8b5d000640 1 -- 172.21.15.151:0/2070305332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b580ad880 msgr2=0x7f8b580b5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:53.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.078+0000 7f8b5d000640 1 --2- 172.21.15.151:0/2070305332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b580ad880 0x7f8b580b5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:53.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.078+0000 7f8b5d000640 1 -- 172.21.15.151:0/2070305332 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b581579a0 con 0x7f8b5806fb90 2024-09-16T15:39:53.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.078+0000 7f8b5d000640 1 --2- 172.21.15.151:0/2070305332 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b5806fb90 0x7f8b58155490 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8b4800ebf0 tx=0x7f8b4800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:53.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.079+0000 7f8b54c00640 1 -- 172.21.15.151:0/2070305332 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b48010630 con 0x7f8b5806fb90 2024-09-16T15:39:53.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.079+0000 7f8b54c00640 1 -- 172.21.15.151:0/2070305332 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8b480108c0 con 0x7f8b5806fb90 2024-09-16T15:39:53.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.079+0000 7f8b5e400640 1 -- 172.21.15.151:0/2070305332 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b58157c80 con 0x7f8b5806fb90 2024-09-16T15:39:53.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.079+0000 7f8b54c00640 1 -- 172.21.15.151:0/2070305332 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b48010d10 con 0x7f8b5806fb90 2024-09-16T15:39:53.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.079+0000 7f8b5e400640 1 -- 172.21.15.151:0/2070305332 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b58158240 con 0x7f8b5806fb90 2024-09-16T15:39:53.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.080+0000 7f8b5e400640 1 -- 172.21.15.151:0/2070305332 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b580c68a0 con 0x7f8b5806fb90 2024-09-16T15:39:53.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.081+0000 7f8b54c00640 1 -- 172.21.15.151:0/2070305332 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f8b48018420 con 0x7f8b5806fb90 2024-09-16T15:39:53.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.082+0000 7f8b54c00640 1 --2- 172.21.15.151:0/2070305332 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8b38078850 0x7f8b3807ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:53.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.082+0000 7f8b54c00640 1 -- 172.21.15.151:0/2070305332 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f8b4809b720 con 0x7f8b5806fb90 2024-09-16T15:39:53.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.082+0000 7f8b57e00640 1 --2- 172.21.15.151:0/2070305332 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8b38078850 0x7f8b3807ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:53.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.083+0000 7f8b57e00640 1 --2- 172.21.15.151:0/2070305332 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8b38078850 0x7f8b3807ad10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8b44034000 tx=0x7f8b44004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:53.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.086+0000 7f8b54c00640 1 -- 172.21.15.151:0/2070305332 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8b580c68a0 con 0x7f8b5806fb90 2024-09-16T15:39:53.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.271+0000 7f8b5e400640 1 -- 172.21.15.151:0/2070305332 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8b5806ff70 con 0x7f8b5806fb90 2024-09-16T15:39:53.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.272+0000 7f8b54c00640 1 -- 172.21.15.151:0/2070305332 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16729 (secure 0 0 0) 0x7f8b480636a0 con 0x7f8b5806fb90 2024-09-16T15:39:53.274 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:39:53.274 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":22,"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","created":"2024-09-16T15:38:56.348535+0000","modified":"2024-09-16T15:39:23.550834+0000","last_up_change":"2024-09-16T15:39:10.681384+0000","last_in_change":"2024-09-16T15:39:00.544477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:39:11.602252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:39:19.970182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T15:39:20.929376+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5524076-a903-4d0a-af6a-caf474c4765d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6809","nonce":3111342741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6811","nonce":3111342741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6815","nonce":3111342741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6813","nonce":3111342741}]},"public_addr":"172.21.15.151:6809/3111342741","cluster_addr":"172.21.15.151:6811/3111342741","heartbeat_back_addr":"172.21.15.151:6815/3111342741","heartbeat_front_addr":"172.21.15.151:6813/3111342741","state":["exists","up"]},{"osd":1,"uuid":"dd25925f-0ae1-494e-8529-4dc205759db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6801","nonce":3368304087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6803","nonce":3368304087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6807","nonce":3368304087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6805","nonce":3368304087}]},"public_addr":"172.21.15.151:6801/3368304087","cluster_addr":"172.21.15.151:6803/3368304087","heartbeat_back_addr":"172.21.15.151:6807/3368304087","heartbeat_front_addr":"172.21.15.151:6805/3368304087","state":["exists","up"]},{"osd":2,"uuid":"c2f18115-013d-4da6-a0fe-cb978f4494b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6823","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6825","nonce":3079373271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6827","nonce":3079373271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6831","nonce":3079373271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6829","nonce":3079373271}]},"public_addr":"172.21.15.151:6825/3079373271","cluster_addr":"172.21.15.151:6827/3079373271","heartbeat_back_addr":"172.21.15.151:6831/3079373271","heartbeat_front_addr":"172.21.15.151:6829/3079373271","state":["exists","up"]},{"osd":3,"uuid":"36edb718-2fcb-4d88-8ddc-c321eed4279d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6817","nonce":1142509041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6819","nonce":1142509041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6824","nonce":1142509041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6821","nonce":1142509041}]},"public_addr":"172.21.15.151:6817/1142509041","cluster_addr":"172.21.15.151:6819/1142509041","heartbeat_back_addr":"172.21.15.151:6824/1142509041","heartbeat_front_addr":"172.21.15.151:6821/1142509041","state":["exists","up"]},{"osd":4,"uuid":"64d33708-cd0a-4236-abe3-1b646e82ab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6811","nonce":2735495271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6813","nonce":2735495271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6817","nonce":2735495271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6815","nonce":2735495271}]},"public_addr":"172.21.15.167:6811/2735495271","cluster_addr":"172.21.15.167:6813/2735495271","heartbeat_back_addr":"172.21.15.167:6817/2735495271","heartbeat_front_addr":"172.21.15.167:6815/2735495271","state":["exists","up"]},{"osd":5,"uuid":"f14b44a9-5910-458c-b518-09a8d0de94de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6827","nonce":1494916572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6829","nonce":1494916572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6833","nonce":1494916572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6831","nonce":1494916572}]},"public_addr":"172.21.15.167:6827/1494916572","cluster_addr":"172.21.15.167:6829/1494916572","heartbeat_back_addr":"172.21.15.167:6833/1494916572","heartbeat_front_addr":"172.21.15.167:6831/1494916572","state":["exists","up"]},{"osd":6,"uuid":"39335545-2654-4710-84b0-a9aa47cfcbba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6819","nonce":2538273273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6821","nonce":2538273273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6825","nonce":2538273273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6823","nonce":2538273273}]},"public_addr":"172.21.15.167:6819/2538273273","cluster_addr":"172.21.15.167:6821/2538273273","heartbeat_back_addr":"172.21.15.167:6825/2538273273","heartbeat_front_addr":"172.21.15.167:6823/2538273273","state":["exists","up"]},{"osd":7,"uuid":"4f8bfc81-53dd-4339-ac22-1a4007d9f963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6803","nonce":265072227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6805","nonce":265072227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6809","nonce":265072227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6807","nonce":265072227}]},"public_addr":"172.21.15.167:6803/265072227","cluster_addr":"172.21.15.167:6805/265072227","heartbeat_back_addr":"172.21.15.167:6809/265072227","heartbeat_front_addr":"172.21.15.167:6807/265072227","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.150359+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.084358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.722368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.332809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.320709+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.395153+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.935250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.801443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:39:53.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.275+0000 7f8b5e400640 1 -- 172.21.15.151:0/2070305332 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8b38078850 msgr2=0x7f8b3807ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:53.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.275+0000 7f8b5e400640 1 --2- 172.21.15.151:0/2070305332 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8b38078850 0x7f8b3807ad10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8b44034000 tx=0x7f8b44004730 comp rx=0 tx=0).stop 2024-09-16T15:39:53.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.275+0000 7f8b5e400640 1 -- 172.21.15.151:0/2070305332 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b5806fb90 msgr2=0x7f8b58155490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:53.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.275+0000 7f8b5e400640 1 --2- 172.21.15.151:0/2070305332 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b5806fb90 0x7f8b58155490 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8b4800ebf0 tx=0x7f8b4800c6a0 comp rx=0 tx=0).stop 2024-09-16T15:39:53.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.275+0000 7f8b5e400640 1 -- 172.21.15.151:0/2070305332 shutdown_connections 2024-09-16T15:39:53.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.275+0000 7f8b5e400640 1 --2- 172.21.15.151:0/2070305332 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8b38078850 0x7f8b3807ad10 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:53.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.275+0000 7f8b5e400640 1 --2- 172.21.15.151:0/2070305332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b580ad880 0x7f8b580b5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:53.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.275+0000 7f8b5e400640 1 --2- 172.21.15.151:0/2070305332 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b580cd5f0 0x7f8b581f7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:53.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.275+0000 7f8b5e400640 1 --2- 172.21.15.151:0/2070305332 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b5806fb90 0x7f8b58155490 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:53.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.275+0000 7f8b5e400640 1 -- 172.21.15.151:0/2070305332 >> 172.21.15.151:0/2070305332 conn(0x7f8b580cfa20 msgr2=0x7f8b580725b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:53.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.275+0000 7f8b5e400640 1 -- 172.21.15.151:0/2070305332 shutdown_connections 2024-09-16T15:39:53.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.275+0000 7f8b5e400640 1 -- 172.21.15.151:0/2070305332 wait complete. 2024-09-16T15:39:53.292 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T15:39:11.602252+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-16T15:39:19.970182+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-16T15:39:20.929376+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-16T15:39:53.292 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-16T15:39:53.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.451+0000 7f4681800640 1 -- 172.21.15.151:0/1431483635 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f467c06efa0 msgr2=0x7f467c073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:53.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.451+0000 7f4681800640 1 --2- 172.21.15.151:0/1431483635 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f467c06efa0 0x7f467c073440 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4674005fd0 tx=0x7f467402aaa0 comp rx=0 tx=0).stop 2024-09-16T15:39:53.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.451+0000 7f4681800640 1 -- 172.21.15.151:0/1431483635 shutdown_connections 2024-09-16T15:39:53.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.451+0000 7f4681800640 1 --2- 172.21.15.151:0/1431483635 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f467c06efa0 0x7f467c073440 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:53.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.451+0000 7f4681800640 1 -- 172.21.15.151:0/1431483635 >> 172.21.15.151:0/1431483635 conn(0x7f467c0b1840 msgr2=0x7f467c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:53.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.451+0000 7f4681800640 1 -- 172.21.15.151:0/1431483635 shutdown_connections 2024-09-16T15:39:53.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.452+0000 7f4681800640 1 -- 172.21.15.151:0/1431483635 wait complete. 2024-09-16T15:39:53.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.453+0000 7f4681800640 1 Processor -- start 2024-09-16T15:39:53.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.453+0000 7f4681800640 1 -- start start 2024-09-16T15:39:53.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.454+0000 7f4681800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f467c06efa0 0x7f467c1557a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:53.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.454+0000 7f4681800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f467c0cccd0 0x7f467c141a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:53.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.454+0000 7f4681800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f467c145750 0x7f467c152d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:53.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.454+0000 7f4681800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f467c0ce6d0 con 0x7f467c0cccd0 2024-09-16T15:39:53.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.454+0000 7f4681800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f467c0ce550 con 0x7f467c06efa0 2024-09-16T15:39:53.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.454+0000 7f4681800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f467c0ce850 con 0x7f467c145750 2024-09-16T15:39:53.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.454+0000 7f467a000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f467c0cccd0 0x7f467c141a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:53.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.455+0000 7f467a000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f467c0cccd0 0x7f467c141a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43452/0 (socket says 172.21.15.151:43452) 2024-09-16T15:39:53.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.455+0000 7f467a000640 1 -- 172.21.15.151:0/4143427543 learned_addr learned my addr 172.21.15.151:0/4143427543 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:53.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.455+0000 7f467b400640 1 --2- 172.21.15.151:0/4143427543 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f467c145750 0x7f467c152d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:53.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.455+0000 7f467aa00640 1 --2- 172.21.15.151:0/4143427543 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f467c06efa0 0x7f467c1557a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:53.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.455+0000 7f467a000640 1 -- 172.21.15.151:0/4143427543 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f467c145750 msgr2=0x7f467c152d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:53.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.455+0000 7f467a000640 1 --2- 172.21.15.151:0/4143427543 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f467c145750 0x7f467c152d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:53.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.455+0000 7f467a000640 1 -- 172.21.15.151:0/4143427543 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f467c06efa0 msgr2=0x7f467c1557a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:53.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.455+0000 7f467a000640 1 --2- 172.21.15.151:0/4143427543 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f467c06efa0 0x7f467c1557a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:53.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.455+0000 7f467a000640 1 -- 172.21.15.151:0/4143427543 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f467c15f610 con 0x7f467c0cccd0 2024-09-16T15:39:53.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.455+0000 7f467b400640 1 --2- 172.21.15.151:0/4143427543 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f467c145750 0x7f467c152d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:39:53.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.455+0000 7f467a000640 1 --2- 172.21.15.151:0/4143427543 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f467c0cccd0 0x7f467c141a20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f466800c500 tx=0x7f466800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:53.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.455+0000 7f4672a00640 1 -- 172.21.15.151:0/4143427543 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46680052d0 con 0x7f467c0cccd0 2024-09-16T15:39:53.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.456+0000 7f4681800640 1 -- 172.21.15.151:0/4143427543 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f467c0c6750 con 0x7f467c0cccd0 2024-09-16T15:39:53.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.455+0000 7f4672a00640 1 -- 172.21.15.151:0/4143427543 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4668005470 con 0x7f467c0cccd0 2024-09-16T15:39:53.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.456+0000 7f4672a00640 1 -- 172.21.15.151:0/4143427543 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4668012870 con 0x7f467c0cccd0 2024-09-16T15:39:53.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.456+0000 7f4681800640 1 -- 172.21.15.151:0/4143427543 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f467c0c69f0 con 0x7f467c0cccd0 2024-09-16T15:39:53.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.457+0000 7f4681800640 1 -- 172.21.15.151:0/4143427543 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f467c0ced00 con 0x7f467c0cccd0 2024-09-16T15:39:53.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.458+0000 7f4672a00640 1 -- 172.21.15.151:0/4143427543 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f46680035f0 con 0x7f467c0cccd0 2024-09-16T15:39:53.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.459+0000 7f4672a00640 1 --2- 172.21.15.151:0/4143427543 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f465c078850 0x7f465c07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:53.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.459+0000 7f4672a00640 1 -- 172.21.15.151:0/4143427543 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f466809a5f0 con 0x7f467c0cccd0 2024-09-16T15:39:53.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.459+0000 7f467aa00640 1 --2- 172.21.15.151:0/4143427543 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f465c078850 0x7f465c07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:53.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.460+0000 7f467aa00640 1 --2- 172.21.15.151:0/4143427543 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f465c078850 0x7f465c07ad10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f466c0015e0 tx=0x7f466c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:53.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.463+0000 7f4672a00640 1 -- 172.21.15.151:0/4143427543 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4668062bd0 con 0x7f467c0cccd0 2024-09-16T15:39:53.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.651+0000 7f4681800640 1 -- 172.21.15.151:0/4143427543 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f467c0afce0 con 0x7f467c0cccd0 2024-09-16T15:39:53.653 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.652+0000 7f4672a00640 1 -- 172.21.15.151:0/4143427543 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f46680624d0 con 0x7f467c0cccd0 2024-09-16T15:39:53.653 INFO:teuthology.orchestra.run.smithi151.stdout:pg_num: 1 2024-09-16T15:39:53.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.654+0000 7f4681800640 1 -- 172.21.15.151:0/4143427543 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f465c078850 msgr2=0x7f465c07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:53.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.654+0000 7f4681800640 1 --2- 172.21.15.151:0/4143427543 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f465c078850 0x7f465c07ad10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f466c0015e0 tx=0x7f466c009040 comp rx=0 tx=0).stop 2024-09-16T15:39:53.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.654+0000 7f4681800640 1 -- 172.21.15.151:0/4143427543 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f467c0cccd0 msgr2=0x7f467c141a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:53.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.654+0000 7f4681800640 1 --2- 172.21.15.151:0/4143427543 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f467c0cccd0 0x7f467c141a20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f466800c500 tx=0x7f466800c9c0 comp rx=0 tx=0).stop 2024-09-16T15:39:53.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.654+0000 7f4681800640 1 -- 172.21.15.151:0/4143427543 shutdown_connections 2024-09-16T15:39:53.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.654+0000 7f4681800640 1 --2- 172.21.15.151:0/4143427543 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f465c078850 0x7f465c07ad10 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:53.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.654+0000 7f4681800640 1 --2- 172.21.15.151:0/4143427543 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f467c145750 0x7f467c152d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:53.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.654+0000 7f4681800640 1 --2- 172.21.15.151:0/4143427543 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f467c0cccd0 0x7f467c141a20 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:53.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.654+0000 7f4681800640 1 --2- 172.21.15.151:0/4143427543 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f467c06efa0 0x7f467c1557a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:53.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.654+0000 7f4681800640 1 -- 172.21.15.151:0/4143427543 >> 172.21.15.151:0/4143427543 conn(0x7f467c0b1840 msgr2=0x7f467c0b0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:53.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.654+0000 7f4681800640 1 -- 172.21.15.151:0/4143427543 shutdown_connections 2024-09-16T15:39:53.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.654+0000 7f4681800640 1 -- 172.21.15.151:0/4143427543 wait complete. 2024-09-16T15:39:53.670 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-16T15:39:53.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.831+0000 7ff31d200640 1 -- 172.21.15.151:0/1437274021 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff31806efa0 msgr2=0x7ff3180cc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:53.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.831+0000 7ff31d200640 1 --2- 172.21.15.151:0/1437274021 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff31806efa0 0x7ff3180cc5c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff3180d0cd0 tx=0x7ff30402a4d0 comp rx=0 tx=0).stop 2024-09-16T15:39:53.833 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.831+0000 7ff31d200640 1 -- 172.21.15.151:0/1437274021 shutdown_connections 2024-09-16T15:39:53.833 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.831+0000 7ff31d200640 1 --2- 172.21.15.151:0/1437274021 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff31806efa0 0x7ff3180cc5c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:53.833 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.831+0000 7ff31d200640 1 --2- 172.21.15.151:0/1437274021 >> v2:172.21.15.167:3300/0 conn(0x7ff318145750 0x7ff318073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:53.833 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.831+0000 7ff31d200640 1 -- 172.21.15.151:0/1437274021 >> 172.21.15.151:0/1437274021 conn(0x7ff3180b1840 msgr2=0x7ff3180b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:53.833 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.831+0000 7ff31d200640 1 -- 172.21.15.151:0/1437274021 shutdown_connections 2024-09-16T15:39:53.833 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.831+0000 7ff31d200640 1 -- 172.21.15.151:0/1437274021 wait complete. 2024-09-16T15:39:53.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.833+0000 7ff31d200640 1 Processor -- start 2024-09-16T15:39:53.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.833+0000 7ff31d200640 1 -- start start 2024-09-16T15:39:53.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.834+0000 7ff31d200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff31806efa0 0x7ff3181f8df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:53.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.834+0000 7ff31d200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff3180ccb00 0x7ff31813e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:53.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.834+0000 7ff31d200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff318145750 0x7ff3181f5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:53.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.834+0000 7ff31d200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff3180ced40 con 0x7ff3180ccb00 2024-09-16T15:39:53.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.834+0000 7ff31d200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff3180cebc0 con 0x7ff31806efa0 2024-09-16T15:39:53.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.834+0000 7ff31d200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7ff3180ceec0 con 0x7ff318145750 2024-09-16T15:39:53.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.834+0000 7ff315400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff3180ccb00 0x7ff31813e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:53.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.834+0000 7ff315400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff3180ccb00 0x7ff31813e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43464/0 (socket says 172.21.15.151:43464) 2024-09-16T15:39:53.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.834+0000 7ff315400640 1 -- 172.21.15.151:0/829306905 learned_addr learned my addr 172.21.15.151:0/829306905 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:53.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.834+0000 7ff315e00640 1 --2- 172.21.15.151:0/829306905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff31806efa0 0x7ff3181f8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:53.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.834+0000 7ff316800640 1 --2- 172.21.15.151:0/829306905 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff318145750 0x7ff3181f5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:53.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.834+0000 7ff315400640 1 -- 172.21.15.151:0/829306905 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff318145750 msgr2=0x7ff3181f5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:53.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.834+0000 7ff315400640 1 --2- 172.21.15.151:0/829306905 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff318145750 0x7ff3181f5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:53.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.834+0000 7ff315400640 1 -- 172.21.15.151:0/829306905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff31806efa0 msgr2=0x7ff3181f8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:53.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.835+0000 7ff315400640 1 --2- 172.21.15.151:0/829306905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff31806efa0 0x7ff3181f8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:53.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.835+0000 7ff315400640 1 -- 172.21.15.151:0/829306905 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3181f4e30 con 0x7ff3180ccb00 2024-09-16T15:39:53.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.835+0000 7ff316800640 1 --2- 172.21.15.151:0/829306905 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff318145750 0x7ff3181f5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:39:53.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.835+0000 7ff315400640 1 --2- 172.21.15.151:0/829306905 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff3180ccb00 0x7ff31813e530 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff304009850 tx=0x7ff3040039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:53.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.835+0000 7ff30e000640 1 -- 172.21.15.151:0/829306905 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30403f070 con 0x7ff3180ccb00 2024-09-16T15:39:53.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.835+0000 7ff30e000640 1 -- 172.21.15.151:0/829306905 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff304004890 con 0x7ff3180ccb00 2024-09-16T15:39:53.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.835+0000 7ff31d200640 1 -- 172.21.15.151:0/829306905 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3181430f0 con 0x7ff3180ccb00 2024-09-16T15:39:53.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.835+0000 7ff30e000640 1 -- 172.21.15.151:0/829306905 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff304035030 con 0x7ff3180ccb00 2024-09-16T15:39:53.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.835+0000 7ff31d200640 1 -- 172.21.15.151:0/829306905 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3181436b0 con 0x7ff3180ccb00 2024-09-16T15:39:53.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.836+0000 7ff31d200640 1 -- 172.21.15.151:0/829306905 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3180cc5c0 con 0x7ff3180ccb00 2024-09-16T15:39:53.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.837+0000 7ff30e000640 1 -- 172.21.15.151:0/829306905 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7ff304042070 con 0x7ff3180ccb00 2024-09-16T15:39:53.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.838+0000 7ff30e000640 1 --2- 172.21.15.151:0/829306905 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff2f8078850 0x7ff2f807ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:53.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.838+0000 7ff30e000640 1 -- 172.21.15.151:0/829306905 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7ff3040bb7f0 con 0x7ff3180ccb00 2024-09-16T15:39:53.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.839+0000 7ff315e00640 1 --2- 172.21.15.151:0/829306905 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff2f8078850 0x7ff2f807ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:53.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.839+0000 7ff315e00640 1 --2- 172.21.15.151:0/829306905 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff2f8078850 0x7ff2f807ad10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff3080098e0 tx=0x7ff308009060 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:53.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:53.842+0000 7ff30e000640 1 -- 172.21.15.151:0/829306905 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff304083780 con 0x7ff3180ccb00 2024-09-16T15:39:54.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.026+0000 7ff31d200640 1 -- 172.21.15.151:0/829306905 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff3180cc330 con 0x7ff3180ccb00 2024-09-16T15:39:54.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.027+0000 7ff30e000640 1 -- 172.21.15.151:0/829306905 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7ff3040881b0 con 0x7ff3180ccb00 2024-09-16T15:39:54.028 INFO:teuthology.orchestra.run.smithi151.stdout:pg_num: 32 2024-09-16T15:39:54.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.029+0000 7ff31d200640 1 -- 172.21.15.151:0/829306905 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff2f8078850 msgr2=0x7ff2f807ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:54.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.029+0000 7ff31d200640 1 --2- 172.21.15.151:0/829306905 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff2f8078850 0x7ff2f807ad10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff3080098e0 tx=0x7ff308009060 comp rx=0 tx=0).stop 2024-09-16T15:39:54.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.029+0000 7ff31d200640 1 -- 172.21.15.151:0/829306905 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff3180ccb00 msgr2=0x7ff31813e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:54.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.029+0000 7ff31d200640 1 --2- 172.21.15.151:0/829306905 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff3180ccb00 0x7ff31813e530 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff304009850 tx=0x7ff3040039e0 comp rx=0 tx=0).stop 2024-09-16T15:39:54.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.029+0000 7ff31d200640 1 -- 172.21.15.151:0/829306905 shutdown_connections 2024-09-16T15:39:54.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.029+0000 7ff31d200640 1 --2- 172.21.15.151:0/829306905 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff2f8078850 0x7ff2f807ad10 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:54.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.029+0000 7ff31d200640 1 --2- 172.21.15.151:0/829306905 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff318145750 0x7ff3181f5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:54.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.029+0000 7ff31d200640 1 --2- 172.21.15.151:0/829306905 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff3180ccb00 0x7ff31813e530 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:54.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.029+0000 7ff31d200640 1 --2- 172.21.15.151:0/829306905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff31806efa0 0x7ff3181f8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:54.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.029+0000 7ff31d200640 1 -- 172.21.15.151:0/829306905 >> 172.21.15.151:0/829306905 conn(0x7ff3180b1840 msgr2=0x7ff3180a9460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:54.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.029+0000 7ff31d200640 1 -- 172.21.15.151:0/829306905 shutdown_connections 2024-09-16T15:39:54.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.029+0000 7ff31d200640 1 -- 172.21.15.151:0/829306905 wait complete. 2024-09-16T15:39:54.046 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-16T15:39:54.207 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.205+0000 7efd4f800640 1 -- 172.21.15.151:0/572722893 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efd480737d0 msgr2=0x7efd48073bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:54.207 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.205+0000 7efd4f800640 1 --2- 172.21.15.151:0/572722893 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efd480737d0 0x7efd48073bb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7efd38005fd0 tx=0x7efd3802a5c0 comp rx=0 tx=0).stop 2024-09-16T15:39:54.207 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.205+0000 7efd4f800640 1 -- 172.21.15.151:0/572722893 shutdown_connections 2024-09-16T15:39:54.207 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.205+0000 7efd4f800640 1 --2- 172.21.15.151:0/572722893 >> v2:172.21.15.167:3300/0 conn(0x7efd480aec50 0x7efd480af090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:54.207 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.205+0000 7efd4f800640 1 --2- 172.21.15.151:0/572722893 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efd480737d0 0x7efd48073bb0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:54.207 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.205+0000 7efd4f800640 1 -- 172.21.15.151:0/572722893 >> 172.21.15.151:0/572722893 conn(0x7efd480d0d40 msgr2=0x7efd480d1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:54.207 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.206+0000 7efd4f800640 1 -- 172.21.15.151:0/572722893 shutdown_connections 2024-09-16T15:39:54.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.206+0000 7efd4f800640 1 -- 172.21.15.151:0/572722893 wait complete. 2024-09-16T15:39:54.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.207+0000 7efd4f800640 1 Processor -- start 2024-09-16T15:39:54.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.207+0000 7efd4f800640 1 -- start start 2024-09-16T15:39:54.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.208+0000 7efd4f800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efd480737d0 0x7efd481f5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:54.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.208+0000 7efd4f800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efd480aec50 0x7efd481518f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:54.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.208+0000 7efd4f800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd4815e6d0 0x7efd48151e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:54.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.208+0000 7efd4f800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efd4815f3f0 con 0x7efd480aec50 2024-09-16T15:39:54.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.208+0000 7efd4f800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efd4815f270 con 0x7efd4815e6d0 2024-09-16T15:39:54.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.208+0000 7efd4f800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7efd480cc480 con 0x7efd480737d0 2024-09-16T15:39:54.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.209+0000 7efd3fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efd480aec50 0x7efd481518f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:54.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.209+0000 7efd4ce00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efd480737d0 0x7efd481f5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:54.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.209+0000 7efd3fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efd480aec50 0x7efd481518f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43474/0 (socket says 172.21.15.151:43474) 2024-09-16T15:39:54.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.209+0000 7efd4ce00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efd480737d0 0x7efd481f5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.151:42966/0 (socket says 172.21.15.151:42966) 2024-09-16T15:39:54.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.209+0000 7efd3fe00640 1 -- 172.21.15.151:0/3254000441 learned_addr learned my addr 172.21.15.151:0/3254000441 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:54.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.209+0000 7efd4d800640 1 --2- 172.21.15.151:0/3254000441 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd4815e6d0 0x7efd48151e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:54.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.209+0000 7efd3fe00640 1 -- 172.21.15.151:0/3254000441 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efd480737d0 msgr2=0x7efd481f5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:54.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.209+0000 7efd3fe00640 1 --2- 172.21.15.151:0/3254000441 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efd480737d0 0x7efd481f5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:54.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.209+0000 7efd3fe00640 1 -- 172.21.15.151:0/3254000441 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd4815e6d0 msgr2=0x7efd48151e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:54.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.209+0000 7efd3fe00640 1 --2- 172.21.15.151:0/3254000441 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd4815e6d0 0x7efd48151e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:54.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.209+0000 7efd3fe00640 1 -- 172.21.15.151:0/3254000441 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd481f6080 con 0x7efd480aec50 2024-09-16T15:39:54.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.209+0000 7efd4d800640 1 --2- 172.21.15.151:0/3254000441 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd4815e6d0 0x7efd48151e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:39:54.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.209+0000 7efd3fe00640 1 --2- 172.21.15.151:0/3254000441 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efd480aec50 0x7efd481518f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7efd3000e9c0 tx=0x7efd3000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:54.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.210+0000 7efd45600640 1 -- 172.21.15.151:0/3254000441 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3000cde0 con 0x7efd480aec50 2024-09-16T15:39:54.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.210+0000 7efd4f800640 1 -- 172.21.15.151:0/3254000441 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd4806f7f0 con 0x7efd480aec50 2024-09-16T15:39:54.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.210+0000 7efd45600640 1 -- 172.21.15.151:0/3254000441 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efd30010520 con 0x7efd480aec50 2024-09-16T15:39:54.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.210+0000 7efd45600640 1 -- 172.21.15.151:0/3254000441 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd30010970 con 0x7efd480aec50 2024-09-16T15:39:54.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.210+0000 7efd4f800640 1 -- 172.21.15.151:0/3254000441 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd48154820 con 0x7efd480aec50 2024-09-16T15:39:54.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.211+0000 7efd4f800640 1 -- 172.21.15.151:0/3254000441 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd48076cb0 con 0x7efd480aec50 2024-09-16T15:39:54.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.212+0000 7efd45600640 1 -- 172.21.15.151:0/3254000441 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7efd3001f020 con 0x7efd480aec50 2024-09-16T15:39:54.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.213+0000 7efd45600640 1 --2- 172.21.15.151:0/3254000441 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efd24078850 0x7efd2407ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:54.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.213+0000 7efd45600640 1 -- 172.21.15.151:0/3254000441 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7efd3009b190 con 0x7efd480aec50 2024-09-16T15:39:54.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.214+0000 7efd4ce00640 1 --2- 172.21.15.151:0/3254000441 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efd24078850 0x7efd2407ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:54.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.214+0000 7efd4ce00640 1 --2- 172.21.15.151:0/3254000441 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efd24078850 0x7efd2407ad10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7efd38033000 tx=0x7efd38004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:54.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.217+0000 7efd45600640 1 -- 172.21.15.151:0/3254000441 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7efd48076cb0 con 0x7efd480aec50 2024-09-16T15:39:54.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.405+0000 7efd4f800640 1 -- 172.21.15.151:0/3254000441 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7efd48073bb0 con 0x7efd480aec50 2024-09-16T15:39:54.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.405+0000 7efd45600640 1 -- 172.21.15.151:0/3254000441 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7efd30063770 con 0x7efd480aec50 2024-09-16T15:39:54.406 INFO:teuthology.orchestra.run.smithi151.stdout:pg_num: 64 2024-09-16T15:39:54.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.408+0000 7efd4f800640 1 -- 172.21.15.151:0/3254000441 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efd24078850 msgr2=0x7efd2407ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:54.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.408+0000 7efd4f800640 1 --2- 172.21.15.151:0/3254000441 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efd24078850 0x7efd2407ad10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7efd38033000 tx=0x7efd38004730 comp rx=0 tx=0).stop 2024-09-16T15:39:54.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.408+0000 7efd4f800640 1 -- 172.21.15.151:0/3254000441 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efd480aec50 msgr2=0x7efd481518f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:54.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.408+0000 7efd4f800640 1 --2- 172.21.15.151:0/3254000441 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efd480aec50 0x7efd481518f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7efd3000e9c0 tx=0x7efd3000ee80 comp rx=0 tx=0).stop 2024-09-16T15:39:54.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.408+0000 7efd4f800640 1 -- 172.21.15.151:0/3254000441 shutdown_connections 2024-09-16T15:39:54.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.408+0000 7efd4f800640 1 --2- 172.21.15.151:0/3254000441 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efd24078850 0x7efd2407ad10 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:54.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.408+0000 7efd4f800640 1 --2- 172.21.15.151:0/3254000441 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd4815e6d0 0x7efd48151e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:54.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.408+0000 7efd4f800640 1 --2- 172.21.15.151:0/3254000441 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efd480aec50 0x7efd481518f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:54.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.408+0000 7efd4f800640 1 --2- 172.21.15.151:0/3254000441 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efd480737d0 0x7efd481f5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:54.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.408+0000 7efd4f800640 1 -- 172.21.15.151:0/3254000441 >> 172.21.15.151:0/3254000441 conn(0x7efd480d0d40 msgr2=0x7efd480d01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:54.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.408+0000 7efd4f800640 1 -- 172.21.15.151:0/3254000441 shutdown_connections 2024-09-16T15:39:54.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.408+0000 7efd4f800640 1 -- 172.21.15.151:0/3254000441 wait complete. 2024-09-16T15:39:54.425 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clone_to_diff_group_and_less_than_cloner_threads' 2024-09-16T15:39:54.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.584+0000 7f39eca00640 1 -- 172.21.15.151:0/2238114179 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39e815bc30 msgr2=0x7f39e80b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:54.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.584+0000 7f39eca00640 1 --2- 172.21.15.151:0/2238114179 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39e815bc30 0x7f39e80b28f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f39d8005fd0 tx=0x7f39d802a5d0 comp rx=0 tx=0).stop 2024-09-16T15:39:54.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.584+0000 7f39eca00640 1 -- 172.21.15.151:0/2238114179 shutdown_connections 2024-09-16T15:39:54.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.584+0000 7f39eca00640 1 --2- 172.21.15.151:0/2238114179 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f39e80d0540 0x7f39e806f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:54.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.584+0000 7f39eca00640 1 --2- 172.21.15.151:0/2238114179 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39e815bc30 0x7f39e80b28f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:54.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.584+0000 7f39eca00640 1 -- 172.21.15.151:0/2238114179 >> 172.21.15.151:0/2238114179 conn(0x7f39e80cfa20 msgr2=0x7f39e80cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:54.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.584+0000 7f39eca00640 1 -- 172.21.15.151:0/2238114179 shutdown_connections 2024-09-16T15:39:54.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.584+0000 7f39eca00640 1 -- 172.21.15.151:0/2238114179 wait complete. 2024-09-16T15:39:54.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.586+0000 7f39eca00640 1 Processor -- start 2024-09-16T15:39:54.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.586+0000 7f39eca00640 1 -- start start 2024-09-16T15:39:54.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.587+0000 7f39eca00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39e80d0540 0x7f39e81f23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:54.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.587+0000 7f39eca00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39e815bc30 0x7f39e8151e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:54.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.587+0000 7f39eca00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f39e8153730 0x7f39e815cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:54.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.587+0000 7f39eca00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f39e8074660 con 0x7f39e80d0540 2024-09-16T15:39:54.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.587+0000 7f39eca00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f39e806f860 con 0x7f39e815bc30 2024-09-16T15:39:54.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.587+0000 7f39eca00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f39e81f3270 con 0x7f39e8153730 2024-09-16T15:39:54.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.587+0000 7f39e7400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39e80d0540 0x7f39e81f23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:54.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.588+0000 7f39e7400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39e80d0540 0x7f39e81f23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43494/0 (socket says 172.21.15.151:43494) 2024-09-16T15:39:54.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.588+0000 7f39e7400640 1 -- 172.21.15.151:0/141199853 learned_addr learned my addr 172.21.15.151:0/141199853 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:54.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.588+0000 7f39e7400640 1 -- 172.21.15.151:0/141199853 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f39e8153730 msgr2=0x7f39e815cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:54.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.588+0000 7f39e7e00640 1 --2- 172.21.15.151:0/141199853 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f39e8153730 0x7f39e815cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:54.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.588+0000 7f39e6a00640 1 --2- 172.21.15.151:0/141199853 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39e815bc30 0x7f39e8151e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:54.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.588+0000 7f39e7400640 1 --2- 172.21.15.151:0/141199853 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f39e8153730 0x7f39e815cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:54.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.588+0000 7f39e7400640 1 -- 172.21.15.151:0/141199853 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39e815bc30 msgr2=0x7f39e8151e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:54.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.588+0000 7f39e7400640 1 --2- 172.21.15.151:0/141199853 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39e815bc30 0x7f39e8151e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:54.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.588+0000 7f39e7400640 1 -- 172.21.15.151:0/141199853 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39e81f2cb0 con 0x7f39e80d0540 2024-09-16T15:39:54.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.588+0000 7f39e7e00640 1 --2- 172.21.15.151:0/141199853 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f39e8153730 0x7f39e815cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:39:54.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.588+0000 7f39e6a00640 1 --2- 172.21.15.151:0/141199853 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39e815bc30 0x7f39e8151e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:39:54.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.588+0000 7f39e7400640 1 --2- 172.21.15.151:0/141199853 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39e80d0540 0x7f39e81f23b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f39d8034040 tx=0x7f39d80035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:54.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.589+0000 7f39df400640 1 -- 172.21.15.151:0/141199853 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d803e030 con 0x7f39e80d0540 2024-09-16T15:39:54.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.589+0000 7f39df400640 1 -- 172.21.15.151:0/141199853 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f39d8003ae0 con 0x7f39e80d0540 2024-09-16T15:39:54.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.589+0000 7f39df400640 1 -- 172.21.15.151:0/141199853 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d8035040 con 0x7f39e80d0540 2024-09-16T15:39:54.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.589+0000 7f39eca00640 1 -- 172.21.15.151:0/141199853 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39e81f2f90 con 0x7f39e80d0540 2024-09-16T15:39:54.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.589+0000 7f39eca00640 1 -- 172.21.15.151:0/141199853 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39e81f28f0 con 0x7f39e80d0540 2024-09-16T15:39:54.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.590+0000 7f39eca00640 1 -- 172.21.15.151:0/141199853 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39e806fdf0 con 0x7f39e80d0540 2024-09-16T15:39:54.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.591+0000 7f39df400640 1 -- 172.21.15.151:0/141199853 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f39d8041030 con 0x7f39e80d0540 2024-09-16T15:39:54.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.592+0000 7f39df400640 1 --2- 172.21.15.151:0/141199853 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f39c8078850 0x7f39c807ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:54.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.592+0000 7f39df400640 1 -- 172.21.15.151:0/141199853 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f39d80bada0 con 0x7f39e80d0540 2024-09-16T15:39:54.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.592+0000 7f39e6a00640 1 --2- 172.21.15.151:0/141199853 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f39c8078850 0x7f39c807ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:54.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.593+0000 7f39e6a00640 1 --2- 172.21.15.151:0/141199853 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f39c8078850 0x7f39c807ad10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f39e8159060 tx=0x7f39d4009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:54.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.596+0000 7f39df400640 1 -- 172.21.15.151:0/141199853 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f39d8083370 con 0x7f39e80d0540 2024-09-16T15:39:54.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:54.852+0000 7f39eca00640 1 -- 172.21.15.151:0/141199853 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clone_to_diff_group_and_less_than_cloner_threads"]} v 0) -- 0x7f39e8159df0 con 0x7f39e80d0540 2024-09-16T15:39:55.690 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:55.688+0000 7f39df400640 1 -- 172.21.15.151:0/141199853 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clone_to_diff_group_and_less_than_cloner_threads"]}]=0 v55) ==== 187+0+0 (secure 0 0 0) 0x7f39d8082c70 con 0x7f39e80d0540 2024-09-16T15:39:55.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:55.692+0000 7f39eca00640 1 -- 172.21.15.151:0/141199853 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f39c8078850 msgr2=0x7f39c807ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:55.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:55.692+0000 7f39eca00640 1 --2- 172.21.15.151:0/141199853 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f39c8078850 0x7f39c807ad10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f39e8159060 tx=0x7f39d4009040 comp rx=0 tx=0).stop 2024-09-16T15:39:55.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:55.692+0000 7f39eca00640 1 -- 172.21.15.151:0/141199853 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39e80d0540 msgr2=0x7f39e81f23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:55.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:55.692+0000 7f39eca00640 1 --2- 172.21.15.151:0/141199853 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39e80d0540 0x7f39e81f23b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f39d8034040 tx=0x7f39d80035f0 comp rx=0 tx=0).stop 2024-09-16T15:39:55.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:55.692+0000 7f39eca00640 1 -- 172.21.15.151:0/141199853 shutdown_connections 2024-09-16T15:39:55.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:55.692+0000 7f39eca00640 1 --2- 172.21.15.151:0/141199853 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f39c8078850 0x7f39c807ad10 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:55.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:55.692+0000 7f39eca00640 1 --2- 172.21.15.151:0/141199853 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39e815bc30 0x7f39e8151e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:55.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:55.692+0000 7f39eca00640 1 --2- 172.21.15.151:0/141199853 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f39e8153730 0x7f39e815cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:55.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:55.692+0000 7f39eca00640 1 --2- 172.21.15.151:0/141199853 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39e80d0540 0x7f39e81f23b0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:55.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:55.692+0000 7f39eca00640 1 -- 172.21.15.151:0/141199853 >> 172.21.15.151:0/141199853 conn(0x7f39e80cfa20 msgr2=0x7f39e80b31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:55.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:55.693+0000 7f39eca00640 1 -- 172.21.15.151:0/141199853 shutdown_connections 2024-09-16T15:39:55.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:55.693+0000 7f39eca00640 1 -- 172.21.15.151:0/141199853 wait complete. 2024-09-16T15:39:55.713 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-16T15:39:55.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:55.871+0000 7f41ad400640 1 -- 172.21.15.151:0/1776294115 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f41a80ccf00 msgr2=0x7f41a80acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:55.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:55.871+0000 7f41ad400640 1 --2- 172.21.15.151:0/1776294115 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f41a80ccf00 0x7f41a80acd30 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4190005fd0 tx=0x7f419002a540 comp rx=0 tx=0).stop 2024-09-16T15:39:55.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:55.871+0000 7f41ad400640 1 -- 172.21.15.151:0/1776294115 shutdown_connections 2024-09-16T15:39:55.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:55.871+0000 7f41ad400640 1 --2- 172.21.15.151:0/1776294115 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f41a80ccf00 0x7f41a80acd30 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:55.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:55.871+0000 7f41ad400640 1 --2- 172.21.15.151:0/1776294115 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f41a815f5a0 0x7f41a80b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:55.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:55.871+0000 7f41ad400640 1 -- 172.21.15.151:0/1776294115 >> 172.21.15.151:0/1776294115 conn(0x7f41a80b1950 msgr2=0x7f41a80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:55.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:55.871+0000 7f41ad400640 1 -- 172.21.15.151:0/1776294115 shutdown_connections 2024-09-16T15:39:55.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:55.872+0000 7f41ad400640 1 -- 172.21.15.151:0/1776294115 wait complete. 2024-09-16T15:39:55.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:55.873+0000 7f41ad400640 1 Processor -- start 2024-09-16T15:39:55.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:55.873+0000 7f41ad400640 1 -- start start 2024-09-16T15:39:55.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:55.874+0000 7f41ad400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f41a80ad270 0x7f41a8146760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:55.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:55.874+0000 7f41ad400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41a80ccf00 0x7f41a81f49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:55.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:55.874+0000 7f41ad400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f41a815f5a0 0x7f41a815da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:55.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:55.874+0000 7f41ad400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f41a806f2e0 con 0x7f41a80ad270 2024-09-16T15:39:55.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:55.874+0000 7f41ad400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f41a806f160 con 0x7f41a80ccf00 2024-09-16T15:39:55.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:55.874+0000 7f41ad400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f41a806f460 con 0x7f41a815f5a0 2024-09-16T15:39:55.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:55.874+0000 7f41a6800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f41a80ad270 0x7f41a8146760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:55.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:55.874+0000 7f41a6800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f41a80ad270 0x7f41a8146760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43502/0 (socket says 172.21.15.151:43502) 2024-09-16T15:39:55.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:55.874+0000 7f41a6800640 1 -- 172.21.15.151:0/1645096729 learned_addr learned my addr 172.21.15.151:0/1645096729 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:55.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:55.875+0000 7f41a7200640 1 --2- 172.21.15.151:0/1645096729 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f41a815f5a0 0x7f41a815da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:55.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:55.875+0000 7f41a5e00640 1 --2- 172.21.15.151:0/1645096729 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41a80ccf00 0x7f41a81f49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:55.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:55.875+0000 7f41a6800640 1 -- 172.21.15.151:0/1645096729 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f41a815f5a0 msgr2=0x7f41a815da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:55.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:55.875+0000 7f41a6800640 1 --2- 172.21.15.151:0/1645096729 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f41a815f5a0 0x7f41a815da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:55.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:55.875+0000 7f41a6800640 1 -- 172.21.15.151:0/1645096729 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41a80ccf00 msgr2=0x7f41a81f49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:55.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:55.875+0000 7f41a6800640 1 --2- 172.21.15.151:0/1645096729 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41a80ccf00 0x7f41a81f49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:55.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:55.875+0000 7f41a6800640 1 -- 172.21.15.151:0/1645096729 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41a81f54a0 con 0x7f41a80ad270 2024-09-16T15:39:55.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:55.875+0000 7f41a6800640 1 --2- 172.21.15.151:0/1645096729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f41a80ad270 0x7f41a8146760 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f419800ea50 tx=0x7f419800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:55.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:55.875+0000 7f419ea00640 1 -- 172.21.15.151:0/1645096729 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f419800cdc0 con 0x7f41a80ad270 2024-09-16T15:39:55.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:55.875+0000 7f419ea00640 1 -- 172.21.15.151:0/1645096729 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4198011560 con 0x7f41a80ad270 2024-09-16T15:39:55.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:55.875+0000 7f419ea00640 1 -- 172.21.15.151:0/1645096729 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4198011990 con 0x7f41a80ad270 2024-09-16T15:39:55.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:55.875+0000 7f41ad400640 1 -- 172.21.15.151:0/1645096729 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41a81f5780 con 0x7f41a80ad270 2024-09-16T15:39:55.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:55.876+0000 7f41ad400640 1 -- 172.21.15.151:0/1645096729 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41a814e6e0 con 0x7f41a80ad270 2024-09-16T15:39:55.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:55.877+0000 7f41ad400640 1 -- 172.21.15.151:0/1645096729 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41a806f980 con 0x7f41a80ad270 2024-09-16T15:39:55.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:55.878+0000 7f419ea00640 1 -- 172.21.15.151:0/1645096729 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f4198011b30 con 0x7f41a80ad270 2024-09-16T15:39:55.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:55.878+0000 7f419ea00640 1 --2- 172.21.15.151:0/1645096729 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4184078880 0x7f418407ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:55.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:55.879+0000 7f419ea00640 1 -- 172.21.15.151:0/1645096729 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f419809b500 con 0x7f41a80ad270 2024-09-16T15:39:55.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:55.879+0000 7f41a5e00640 1 --2- 172.21.15.151:0/1645096729 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4184078880 0x7f418407ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:55.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:55.880+0000 7f41a5e00640 1 --2- 172.21.15.151:0/1645096729 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4184078880 0x7f418407ad40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4190033000 tx=0x7f4190004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:55.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:55.883+0000 7f419ea00640 1 -- 172.21.15.151:0/1645096729 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4198016030 con 0x7f41a80ad270 2024-09-16T15:39:56.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:56.066+0000 7f41ad400640 1 -- 172.21.15.151:0/1645096729 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f41a8069ac0 con 0x7f41a80ad270 2024-09-16T15:39:56.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:56.066+0000 7f419ea00640 1 -- 172.21.15.151:0/1645096729 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v27) ==== 59+0+1998 (secure 0 0 0) 0x7f4198011e20 con 0x7f41a80ad270 2024-09-16T15:39:56.069 INFO:teuthology.orchestra.run.smithi151.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-16T15:39:56.069 INFO:teuthology.orchestra.run.smithi151.stdout:mon advanced mon_op_complaint_time 120 2024-09-16T15:39:56.069 INFO:teuthology.orchestra.run.smithi151.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-16T15:39:56.069 INFO:teuthology.orchestra.run.smithi151.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-16T15:39:56.069 INFO:teuthology.orchestra.run.smithi151.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3795.338492 2024-09-16T15:39:56.069 INFO:teuthology.orchestra.run.smithi151.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 4030.355715 2024-09-16T15:39:56.069 INFO:teuthology.orchestra.run.smithi151.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3817.533640 2024-09-16T15:39:56.070 INFO:teuthology.orchestra.run.smithi151.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3950.656136 2024-09-16T15:39:56.070 INFO:teuthology.orchestra.run.smithi151.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4449.408052 2024-09-16T15:39:56.070 INFO:teuthology.orchestra.run.smithi151.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4439.932296 2024-09-16T15:39:56.070 INFO:teuthology.orchestra.run.smithi151.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4566.875579 2024-09-16T15:39:56.070 INFO:teuthology.orchestra.run.smithi151.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 6320.073632 2024-09-16T15:39:56.070 INFO:teuthology.orchestra.run.smithi151.stdout:mds advanced debug_mds 20/20 2024-09-16T15:39:56.070 INFO:teuthology.orchestra.run.smithi151.stdout:mds advanced debug_mds_balancer 20/20 2024-09-16T15:39:56.070 INFO:teuthology.orchestra.run.smithi151.stdout:mds advanced debug_ms 2/2 2024-09-16T15:39:56.070 INFO:teuthology.orchestra.run.smithi151.stdout:mds dev mds_debug_frag true 2024-09-16T15:39:56.070 INFO:teuthology.orchestra.run.smithi151.stdout:mds dev mds_debug_scatterstat true 2024-09-16T15:39:56.070 INFO:teuthology.orchestra.run.smithi151.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-16T15:39:56.070 INFO:teuthology.orchestra.run.smithi151.stdout:mds dev mds_verify_scatter true 2024-09-16T15:39:56.070 INFO:teuthology.orchestra.run.smithi151.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-16T15:39:56.071 INFO:teuthology.orchestra.run.smithi151.stdout:mds advanced rados_mon_op_timeout 900 2024-09-16T15:39:56.071 INFO:teuthology.orchestra.run.smithi151.stdout:mds advanced rados_osd_op_timeout 900 2024-09-16T15:39:56.071 INFO:teuthology.orchestra.run.smithi151.stdout:client advanced client_mount_timeout 600 2024-09-16T15:39:56.071 INFO:teuthology.orchestra.run.smithi151.stdout:client advanced debug_client 20/20 2024-09-16T15:39:56.071 INFO:teuthology.orchestra.run.smithi151.stdout:client advanced debug_ms 1/1 2024-09-16T15:39:56.071 INFO:teuthology.orchestra.run.smithi151.stdout:client advanced rados_mon_op_timeout 900 2024-09-16T15:39:56.071 INFO:teuthology.orchestra.run.smithi151.stdout:client advanced rados_osd_op_timeout 900 2024-09-16T15:39:56.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:56.069+0000 7f41ad400640 1 -- 172.21.15.151:0/1645096729 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4184078880 msgr2=0x7f418407ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:56.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:56.069+0000 7f41ad400640 1 --2- 172.21.15.151:0/1645096729 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4184078880 0x7f418407ad40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4190033000 tx=0x7f4190004730 comp rx=0 tx=0).stop 2024-09-16T15:39:56.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:56.069+0000 7f41ad400640 1 -- 172.21.15.151:0/1645096729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f41a80ad270 msgr2=0x7f41a8146760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:56.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:56.069+0000 7f41ad400640 1 --2- 172.21.15.151:0/1645096729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f41a80ad270 0x7f41a8146760 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f419800ea50 tx=0x7f419800ef10 comp rx=0 tx=0).stop 2024-09-16T15:39:56.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:56.069+0000 7f41ad400640 1 -- 172.21.15.151:0/1645096729 shutdown_connections 2024-09-16T15:39:56.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:56.069+0000 7f41ad400640 1 --2- 172.21.15.151:0/1645096729 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4184078880 0x7f418407ad40 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:56.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:56.069+0000 7f41ad400640 1 --2- 172.21.15.151:0/1645096729 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f41a80ccf00 0x7f41a81f49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:56.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:56.069+0000 7f41ad400640 1 --2- 172.21.15.151:0/1645096729 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f41a815f5a0 0x7f41a815da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:56.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:56.069+0000 7f41ad400640 1 --2- 172.21.15.151:0/1645096729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f41a80ad270 0x7f41a8146760 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:56.073 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:56.069+0000 7f41ad400640 1 -- 172.21.15.151:0/1645096729 >> 172.21.15.151:0/1645096729 conn(0x7f41a80b1950 msgr2=0x7f41a80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:56.073 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:56.069+0000 7f41ad400640 1 -- 172.21.15.151:0/1645096729 shutdown_connections 2024-09-16T15:39:56.073 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:56.069+0000 7f41ad400640 1 -- 172.21.15.151:0/1645096729 wait complete. 2024-09-16T15:39:56.088 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-16T15:39:56.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:56.248+0000 7ffb6aa00640 1 -- 172.21.15.151:0/1809542709 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb4c0039e0 msgr2=0x7ffb4c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:56.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:56.248+0000 7ffb6aa00640 1 --2- 172.21.15.151:0/1809542709 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb4c0039e0 0x7ffb4c003dc0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ffb6406cdc0 tx=0x7ffb4c005830 comp rx=0 tx=0).stop 2024-09-16T15:39:56.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:56.248+0000 7ffb6a000640 1 -- 172.21.15.151:0/1809542709 reap_dead start 2024-09-16T15:39:56.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:56.248+0000 7ffb6aa00640 1 -- 172.21.15.151:0/1809542709 shutdown_connections 2024-09-16T15:39:56.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:56.248+0000 7ffb6aa00640 1 -- 172.21.15.151:0/1809542709 >> 172.21.15.151:0/1809542709 conn(0x7ffb6409d0b0 msgr2=0x7ffb640d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:56.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:56.248+0000 7ffb6aa00640 1 -- 172.21.15.151:0/1809542709 shutdown_connections 2024-09-16T15:39:56.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:56.248+0000 7ffb6aa00640 1 -- 172.21.15.151:0/1809542709 wait complete. 2024-09-16T15:39:56.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:56.249+0000 7ffb6aa00640 1 Processor -- start 2024-09-16T15:39:56.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:56.249+0000 7ffb6aa00640 1 -- start start 2024-09-16T15:39:56.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:56.250+0000 7ffb6aa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb4c0039e0 0x7ffb6409fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:56.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:56.250+0000 7ffb6aa00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb6409dc80 0x7ffb6414e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:56.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:56.250+0000 7ffb6aa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb6409e1c0 0x7ffb640cd490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:56.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:56.250+0000 7ffb6aa00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ffb640cf7c0 con 0x7ffb6409e1c0 2024-09-16T15:39:56.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:56.250+0000 7ffb6aa00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ffb640cf640 con 0x7ffb4c0039e0 2024-09-16T15:39:56.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:56.250+0000 7ffb6aa00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7ffb640cf940 con 0x7ffb6409dc80 2024-09-16T15:39:56.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:56.251+0000 7ffb6a000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb6409e1c0 0x7ffb640cd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:56.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:56.251+0000 7ffb6a000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb6409e1c0 0x7ffb640cd490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43516/0 (socket says 172.21.15.151:43516) 2024-09-16T15:39:56.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:56.251+0000 7ffb6a000640 1 -- 172.21.15.151:0/2369887870 learned_addr learned my addr 172.21.15.151:0/2369887870 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:56.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:56.251+0000 7ffb69600640 1 --2- 172.21.15.151:0/2369887870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb4c0039e0 0x7ffb6409fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:56.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:56.251+0000 7ffb68c00640 1 --2- 172.21.15.151:0/2369887870 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb6409dc80 0x7ffb6414e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:56.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:56.251+0000 7ffb6a000640 1 -- 172.21.15.151:0/2369887870 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb6409dc80 msgr2=0x7ffb6414e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:56.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:56.251+0000 7ffb6a000640 1 --2- 172.21.15.151:0/2369887870 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb6409dc80 0x7ffb6414e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:56.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:56.251+0000 7ffb6a000640 1 -- 172.21.15.151:0/2369887870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb4c0039e0 msgr2=0x7ffb6409fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:56.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:56.251+0000 7ffb6a000640 1 --2- 172.21.15.151:0/2369887870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb4c0039e0 0x7ffb6409fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:56.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:56.251+0000 7ffb6a000640 1 -- 172.21.15.151:0/2369887870 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb64151110 con 0x7ffb6409e1c0 2024-09-16T15:39:56.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:56.251+0000 7ffb6a000640 1 --2- 172.21.15.151:0/2369887870 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb6409e1c0 0x7ffb640cd490 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ffb6408d200 tx=0x7ffb5c001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:56.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:56.252+0000 7ffb61600640 1 -- 172.21.15.151:0/2369887870 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb5c015070 con 0x7ffb6409e1c0 2024-09-16T15:39:56.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:56.252+0000 7ffb61600640 1 -- 172.21.15.151:0/2369887870 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffb5c011860 con 0x7ffb6409e1c0 2024-09-16T15:39:56.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:56.252+0000 7ffb61600640 1 -- 172.21.15.151:0/2369887870 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb5c011cb0 con 0x7ffb6409e1c0 2024-09-16T15:39:56.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:56.252+0000 7ffb6aa00640 1 -- 172.21.15.151:0/2369887870 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb641513f0 con 0x7ffb6409e1c0 2024-09-16T15:39:56.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:56.252+0000 7ffb6aa00640 1 -- 172.21.15.151:0/2369887870 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb641519b0 con 0x7ffb6409e1c0 2024-09-16T15:39:56.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:56.253+0000 7ffb6aa00640 1 -- 172.21.15.151:0/2369887870 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb640d2500 con 0x7ffb6409e1c0 2024-09-16T15:39:56.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:56.254+0000 7ffb61600640 1 -- 172.21.15.151:0/2369887870 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7ffb5c018070 con 0x7ffb6409e1c0 2024-09-16T15:39:56.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:56.255+0000 7ffb61600640 1 --2- 172.21.15.151:0/2369887870 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ffb44078850 0x7ffb4407ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:56.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:56.255+0000 7ffb61600640 1 -- 172.21.15.151:0/2369887870 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7ffb5c09b2d0 con 0x7ffb6409e1c0 2024-09-16T15:39:56.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:56.255+0000 7ffb69600640 1 --2- 172.21.15.151:0/2369887870 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ffb44078850 0x7ffb4407ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:56.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:56.256+0000 7ffb69600640 1 --2- 172.21.15.151:0/2369887870 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ffb44078850 0x7ffb4407ad10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ffb58035000 tx=0x7ffb58038040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:56.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:56.259+0000 7ffb61600640 1 -- 172.21.15.151:0/2369887870 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ffb5c0632e0 con 0x7ffb6409e1c0 2024-09-16T15:39:56.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:56.444+0000 7ffb6aa00640 1 -- 172.21.15.151:0/2369887870 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7ffb640cb180 con 0x7ffb6409e1c0 2024-09-16T15:39:56.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:56.448+0000 7ffb61600640 1 -- 172.21.15.151:0/2369887870 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) ==== 122+0+0 (secure 0 0 0) 0x7ffb5c067d10 con 0x7ffb6409e1c0 2024-09-16T15:39:56.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:56.450+0000 7ffb6aa00640 1 -- 172.21.15.151:0/2369887870 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ffb44078850 msgr2=0x7ffb4407ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:56.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:56.450+0000 7ffb6aa00640 1 --2- 172.21.15.151:0/2369887870 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ffb44078850 0x7ffb4407ad10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ffb58035000 tx=0x7ffb58038040 comp rx=0 tx=0).stop 2024-09-16T15:39:56.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:56.450+0000 7ffb6aa00640 1 -- 172.21.15.151:0/2369887870 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb6409e1c0 msgr2=0x7ffb640cd490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:56.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:56.450+0000 7ffb6aa00640 1 --2- 172.21.15.151:0/2369887870 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb6409e1c0 0x7ffb640cd490 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ffb6408d200 tx=0x7ffb5c001710 comp rx=0 tx=0).stop 2024-09-16T15:39:56.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:56.450+0000 7ffb6aa00640 1 -- 172.21.15.151:0/2369887870 shutdown_connections 2024-09-16T15:39:56.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:56.450+0000 7ffb6aa00640 1 --2- 172.21.15.151:0/2369887870 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ffb44078850 0x7ffb4407ad10 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:56.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:56.450+0000 7ffb6aa00640 1 --2- 172.21.15.151:0/2369887870 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb6409e1c0 0x7ffb640cd490 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:56.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:56.450+0000 7ffb6aa00640 1 --2- 172.21.15.151:0/2369887870 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffb6409dc80 0x7ffb6414e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:56.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:56.450+0000 7ffb6aa00640 1 --2- 172.21.15.151:0/2369887870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffb4c0039e0 0x7ffb6409fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:56.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:56.450+0000 7ffb6aa00640 1 -- 172.21.15.151:0/2369887870 >> 172.21.15.151:0/2369887870 conn(0x7ffb6409d0b0 msgr2=0x7ffb640cc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:56.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:56.451+0000 7ffb6aa00640 1 -- 172.21.15.151:0/2369887870 shutdown_connections 2024-09-16T15:39:56.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:56.451+0000 7ffb6aa00640 1 -- 172.21.15.151:0/2369887870 wait complete. 2024-09-16T15:39:56.468 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-16T15:39:56.468 DEBUG:teuthology.orchestra.run.smithi151:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:39:56.560 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:39:56.647 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-16T15:39:56.647 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:39:56.647 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:39:56.683 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:39:56.683 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:39:56.758 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:39:56.758 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:39:56.834 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-16T15:39:57.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.031+0000 7f6250a00640 1 -- 172.21.15.151:0/2131983507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f624c0b4100 msgr2=0x7f624c0b1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:57.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.031+0000 7f6250a00640 1 --2- 172.21.15.151:0/2131983507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f624c0b4100 0x7f624c0b1770 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f624c0d4ea0 tx=0x7f624402a9f0 comp rx=0 tx=0).stop 2024-09-16T15:39:57.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.031+0000 7f6250a00640 1 -- 172.21.15.151:0/2131983507 shutdown_connections 2024-09-16T15:39:57.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.031+0000 7f6250a00640 1 --2- 172.21.15.151:0/2131983507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f624c0b4100 0x7f624c0b1770 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:57.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.031+0000 7f6250a00640 1 --2- 172.21.15.151:0/2131983507 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f624c0b5370 0x7f624c0b3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:57.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.031+0000 7f6250a00640 1 -- 172.21.15.151:0/2131983507 >> 172.21.15.151:0/2131983507 conn(0x7f624c0d3df0 msgr2=0x7f624c0d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:57.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.032+0000 7f6250a00640 1 -- 172.21.15.151:0/2131983507 shutdown_connections 2024-09-16T15:39:57.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.032+0000 7f6250a00640 1 -- 172.21.15.151:0/2131983507 wait complete. 2024-09-16T15:39:57.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.033+0000 7f6250a00640 1 Processor -- start 2024-09-16T15:39:57.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.033+0000 7f6250a00640 1 -- start start 2024-09-16T15:39:57.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.034+0000 7f6250a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f624c0b4100 0x7f624c0c7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:57.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.034+0000 7f6250a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f624c0b5370 0x7f624c158220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:57.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.034+0000 7f6250a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f624c161440 0x7f624c0b8df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:57.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.034+0000 7f6250a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f624c0b2040 con 0x7f624c161440 2024-09-16T15:39:57.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.034+0000 7f6250a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f624c0b1ec0 con 0x7f624c0b4100 2024-09-16T15:39:57.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.034+0000 7f6250a00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f624c0b21c0 con 0x7f624c0b5370 2024-09-16T15:39:57.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.034+0000 7f624b200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f624c161440 0x7f624c0b8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:57.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.034+0000 7f624b200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f624c161440 0x7f624c0b8df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43540/0 (socket says 172.21.15.151:43540) 2024-09-16T15:39:57.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.034+0000 7f624b200640 1 -- 172.21.15.151:0/402867814 learned_addr learned my addr 172.21.15.151:0/402867814 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:57.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.035+0000 7f6249e00640 1 --2- 172.21.15.151:0/402867814 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f624c0b5370 0x7f624c158220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:57.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.035+0000 7f624a800640 1 --2- 172.21.15.151:0/402867814 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f624c0b4100 0x7f624c0c7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:57.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.035+0000 7f624b200640 1 -- 172.21.15.151:0/402867814 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f624c0b5370 msgr2=0x7f624c158220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:57.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.035+0000 7f624b200640 1 --2- 172.21.15.151:0/402867814 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f624c0b5370 0x7f624c158220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:57.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.035+0000 7f624b200640 1 -- 172.21.15.151:0/402867814 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f624c0b4100 msgr2=0x7f624c0c7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:57.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.035+0000 7f624b200640 1 --2- 172.21.15.151:0/402867814 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f624c0b4100 0x7f624c0c7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:57.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.035+0000 7f624b200640 1 -- 172.21.15.151:0/402867814 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f624c0c8930 con 0x7f624c161440 2024-09-16T15:39:57.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.035+0000 7f6249e00640 1 --2- 172.21.15.151:0/402867814 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f624c0b5370 0x7f624c158220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:39:57.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.035+0000 7f624b200640 1 --2- 172.21.15.151:0/402867814 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f624c161440 0x7f624c0b8df0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6244009d60 tx=0x7f6244003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:57.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.035+0000 7f6242a00640 1 -- 172.21.15.151:0/402867814 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6244047070 con 0x7f624c161440 2024-09-16T15:39:57.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.035+0000 7f6242a00640 1 -- 172.21.15.151:0/402867814 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6244004250 con 0x7f624c161440 2024-09-16T15:39:57.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.035+0000 7f6242a00640 1 -- 172.21.15.151:0/402867814 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6244033050 con 0x7f624c161440 2024-09-16T15:39:57.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.035+0000 7f6250a00640 1 -- 172.21.15.151:0/402867814 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f624c0c8090 con 0x7f624c161440 2024-09-16T15:39:57.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.036+0000 7f6250a00640 1 -- 172.21.15.151:0/402867814 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f624c0c8650 con 0x7f624c161440 2024-09-16T15:39:57.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.037+0000 7f6250a00640 1 -- 172.21.15.151:0/402867814 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f624c0b1770 con 0x7f624c161440 2024-09-16T15:39:57.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.038+0000 7f6242a00640 1 -- 172.21.15.151:0/402867814 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f624404a070 con 0x7f624c161440 2024-09-16T15:39:57.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.038+0000 7f6242a00640 1 --2- 172.21.15.151:0/402867814 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f622c078850 0x7f622c07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:57.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.039+0000 7f6242a00640 1 -- 172.21.15.151:0/402867814 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f62440bbb50 con 0x7f624c161440 2024-09-16T15:39:57.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.039+0000 7f624a800640 1 --2- 172.21.15.151:0/402867814 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f622c078850 0x7f622c07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:57.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.040+0000 7f624a800640 1 --2- 172.21.15.151:0/402867814 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f622c078850 0x7f622c07ad10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f624c0c8f50 tx=0x7f623c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:57.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.043+0000 7f6242a00640 1 -- 172.21.15.151:0/402867814 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f62440841b0 con 0x7f624c161440 2024-09-16T15:39:57.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.237+0000 7f6250a00640 1 -- 172.21.15.151:0/402867814 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f624c06e540 con 0x7f624c161440 2024-09-16T15:39:57.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.238+0000 7f6242a00640 1 -- 172.21.15.151:0/402867814 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f6244083ab0 con 0x7f624c161440 2024-09-16T15:39:57.240 INFO:teuthology.orchestra.run.smithi151.stderr:listed 0 entries 2024-09-16T15:39:57.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.240+0000 7f6250a00640 1 -- 172.21.15.151:0/402867814 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f622c078850 msgr2=0x7f622c07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:57.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.240+0000 7f6250a00640 1 --2- 172.21.15.151:0/402867814 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f622c078850 0x7f622c07ad10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f624c0c8f50 tx=0x7f623c009040 comp rx=0 tx=0).stop 2024-09-16T15:39:57.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.240+0000 7f6250a00640 1 -- 172.21.15.151:0/402867814 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f624c161440 msgr2=0x7f624c0b8df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:57.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.240+0000 7f6250a00640 1 --2- 172.21.15.151:0/402867814 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f624c161440 0x7f624c0b8df0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6244009d60 tx=0x7f6244003f10 comp rx=0 tx=0).stop 2024-09-16T15:39:57.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.240+0000 7f6250a00640 1 -- 172.21.15.151:0/402867814 shutdown_connections 2024-09-16T15:39:57.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.240+0000 7f6250a00640 1 --2- 172.21.15.151:0/402867814 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f622c078850 0x7f622c07ad10 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:57.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.240+0000 7f6250a00640 1 --2- 172.21.15.151:0/402867814 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f624c161440 0x7f624c0b8df0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:57.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.240+0000 7f6250a00640 1 --2- 172.21.15.151:0/402867814 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f624c0b5370 0x7f624c158220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:57.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.240+0000 7f6250a00640 1 --2- 172.21.15.151:0/402867814 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f624c0b4100 0x7f624c0c7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:57.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.240+0000 7f6250a00640 1 -- 172.21.15.151:0/402867814 >> 172.21.15.151:0/402867814 conn(0x7f624c0d3df0 msgr2=0x7f624c06ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:57.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.241+0000 7f6250a00640 1 -- 172.21.15.151:0/402867814 shutdown_connections 2024-09-16T15:39:57.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.241+0000 7f6250a00640 1 -- 172.21.15.151:0/402867814 wait complete. 2024-09-16T15:39:57.258 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-16T15:39:57.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:57.258 DEBUG:teuthology.orchestra.run.smithi151:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-16T15:39:57.319 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi151.front.sepia.ceph.com 2024-09-16T15:39:57.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:57.319 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-16T15:39:57.368 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-16T15:39:57.368 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:39:57.408 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-16T15:39:57.408 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:39:57.487 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-16T15:39:57.487 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:39:57.565 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-16T15:39:57.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.723+0000 7fa634200640 1 -- 172.21.15.151:0/2766227256 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa62c161550 msgr2=0x7fa62c06ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:57.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.723+0000 7fa634200640 1 --2- 172.21.15.151:0/2766227256 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa62c161550 0x7fa62c06ef50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa620005fd0 tx=0x7fa62002a4a0 comp rx=0 tx=0).stop 2024-09-16T15:39:57.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.723+0000 7fa634200640 1 -- 172.21.15.151:0/2766227256 shutdown_connections 2024-09-16T15:39:57.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.723+0000 7fa634200640 1 --2- 172.21.15.151:0/2766227256 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa62c072df0 0x7fa62c0732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:57.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.723+0000 7fa634200640 1 --2- 172.21.15.151:0/2766227256 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa62c161550 0x7fa62c06ef50 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:57.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.723+0000 7fa634200640 1 -- 172.21.15.151:0/2766227256 >> 172.21.15.151:0/2766227256 conn(0x7fa62c0b1950 msgr2=0x7fa62c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:57.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.723+0000 7fa634200640 1 -- 172.21.15.151:0/2766227256 shutdown_connections 2024-09-16T15:39:57.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.723+0000 7fa634200640 1 -- 172.21.15.151:0/2766227256 wait complete. 2024-09-16T15:39:57.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.725+0000 7fa634200640 1 Processor -- start 2024-09-16T15:39:57.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.725+0000 7fa634200640 1 -- start start 2024-09-16T15:39:57.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.726+0000 7fa634200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa62c072df0 0x7fa62c0cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:57.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.726+0000 7fa634200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa62c0af3d0 0x7fa62c1f8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:57.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.726+0000 7fa634200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa62c161550 0x7fa62c148ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:57.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.726+0000 7fa634200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa62c0737e0 con 0x7fa62c161550 2024-09-16T15:39:57.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.726+0000 7fa634200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa62c06fdb0 con 0x7fa62c072df0 2024-09-16T15:39:57.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.726+0000 7fa634200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fa62c073960 con 0x7fa62c0af3d0 2024-09-16T15:39:57.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.726+0000 7fa632200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa62c161550 0x7fa62c148ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:57.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.726+0000 7fa632200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa62c161550 0x7fa62c148ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43564/0 (socket says 172.21.15.151:43564) 2024-09-16T15:39:57.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.726+0000 7fa632200640 1 -- 172.21.15.151:0/3511272548 learned_addr learned my addr 172.21.15.151:0/3511272548 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:57.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.726+0000 7fa631800640 1 --2- 172.21.15.151:0/3511272548 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa62c072df0 0x7fa62c0cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:57.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.726+0000 7fa630e00640 1 --2- 172.21.15.151:0/3511272548 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa62c0af3d0 0x7fa62c1f8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:57.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.727+0000 7fa632200640 1 -- 172.21.15.151:0/3511272548 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa62c0af3d0 msgr2=0x7fa62c1f8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:57.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.727+0000 7fa632200640 1 --2- 172.21.15.151:0/3511272548 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa62c0af3d0 0x7fa62c1f8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:57.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.727+0000 7fa632200640 1 -- 172.21.15.151:0/3511272548 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa62c072df0 msgr2=0x7fa62c0cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:57.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.727+0000 7fa632200640 1 --2- 172.21.15.151:0/3511272548 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa62c072df0 0x7fa62c0cdff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:57.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.727+0000 7fa632200640 1 -- 172.21.15.151:0/3511272548 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa62c0c9850 con 0x7fa62c161550 2024-09-16T15:39:57.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.727+0000 7fa632200640 1 --2- 172.21.15.151:0/3511272548 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa62c161550 0x7fa62c148ef0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa62400b970 tx=0x7fa62400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:57.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.727+0000 7fa629600640 1 -- 172.21.15.151:0/3511272548 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa624019070 con 0x7fa62c161550 2024-09-16T15:39:57.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.727+0000 7fa629600640 1 -- 172.21.15.151:0/3511272548 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa62400cc50 con 0x7fa62c161550 2024-09-16T15:39:57.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.727+0000 7fa629600640 1 -- 172.21.15.151:0/3511272548 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6240146a0 con 0x7fa62c161550 2024-09-16T15:39:57.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.727+0000 7fa634200640 1 -- 172.21.15.151:0/3511272548 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa62c0c8fd0 con 0x7fa62c161550 2024-09-16T15:39:57.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.727+0000 7fa634200640 1 -- 172.21.15.151:0/3511272548 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa62c0c9570 con 0x7fa62c161550 2024-09-16T15:39:57.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.728+0000 7fa634200640 1 -- 172.21.15.151:0/3511272548 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa62c073ed0 con 0x7fa62c161550 2024-09-16T15:39:57.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.729+0000 7fa629600640 1 -- 172.21.15.151:0/3511272548 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fa62401c070 con 0x7fa62c161550 2024-09-16T15:39:57.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.730+0000 7fa629600640 1 --2- 172.21.15.151:0/3511272548 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa60c078850 0x7fa60c07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:57.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.731+0000 7fa629600640 1 -- 172.21.15.151:0/3511272548 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fa62409a6b0 con 0x7fa62c161550 2024-09-16T15:39:57.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.731+0000 7fa631800640 1 --2- 172.21.15.151:0/3511272548 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa60c078850 0x7fa60c07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:57.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.732+0000 7fa631800640 1 --2- 172.21.15.151:0/3511272548 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa60c078850 0x7fa60c07ad10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa620033000 tx=0x7fa620004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:57.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.734+0000 7fa629600640 1 -- 172.21.15.151:0/3511272548 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa62c0c9850 con 0x7fa62c161550 2024-09-16T15:39:57.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.928+0000 7fa634200640 1 -- 172.21.15.151:0/3511272548 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa62c15d440 con 0x7fa62c161550 2024-09-16T15:39:57.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.929+0000 7fa629600640 1 -- 172.21.15.151:0/3511272548 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7fa624062630 con 0x7fa62c161550 2024-09-16T15:39:57.931 INFO:teuthology.orchestra.run.smithi151.stderr:listed 0 entries 2024-09-16T15:39:57.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.931+0000 7fa634200640 1 -- 172.21.15.151:0/3511272548 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa60c078850 msgr2=0x7fa60c07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:57.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.931+0000 7fa634200640 1 --2- 172.21.15.151:0/3511272548 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa60c078850 0x7fa60c07ad10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa620033000 tx=0x7fa620004730 comp rx=0 tx=0).stop 2024-09-16T15:39:57.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.931+0000 7fa634200640 1 -- 172.21.15.151:0/3511272548 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa62c161550 msgr2=0x7fa62c148ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:57.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.931+0000 7fa634200640 1 --2- 172.21.15.151:0/3511272548 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa62c161550 0x7fa62c148ef0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa62400b970 tx=0x7fa62400be30 comp rx=0 tx=0).stop 2024-09-16T15:39:57.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.931+0000 7fa634200640 1 -- 172.21.15.151:0/3511272548 shutdown_connections 2024-09-16T15:39:57.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.931+0000 7fa634200640 1 --2- 172.21.15.151:0/3511272548 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa60c078850 0x7fa60c07ad10 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:57.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.931+0000 7fa634200640 1 --2- 172.21.15.151:0/3511272548 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa62c161550 0x7fa62c148ef0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:57.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.931+0000 7fa634200640 1 --2- 172.21.15.151:0/3511272548 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa62c0af3d0 0x7fa62c1f8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:57.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.931+0000 7fa634200640 1 --2- 172.21.15.151:0/3511272548 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa62c072df0 0x7fa62c0cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:57.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.931+0000 7fa634200640 1 -- 172.21.15.151:0/3511272548 >> 172.21.15.151:0/3511272548 conn(0x7fa62c0b1950 msgr2=0x7fa62c0b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:57.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.931+0000 7fa634200640 1 -- 172.21.15.151:0/3511272548 shutdown_connections 2024-09-16T15:39:57.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:57.932+0000 7fa634200640 1 -- 172.21.15.151:0/3511272548 wait complete. 2024-09-16T15:39:57.950 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-16T15:39:57.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:57.950 DEBUG:teuthology.orchestra.run.smithi167:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-09-16T15:39:58.014 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi167.front.sepia.ceph.com 2024-09-16T15:39:58.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:58.014 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-16T15:39:58.059 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:39:58.059 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:39:58.096 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:39:58.096 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:39:58.171 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:39:58.171 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:39:58.247 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-16T15:39:58.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:58.444+0000 7f5052400640 1 -- 172.21.15.151:0/3287752565 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f504c0cd5f0 msgr2=0x7f504c0cda80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:58.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:58.444+0000 7f5052400640 1 --2- 172.21.15.151:0/3287752565 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f504c0cd5f0 0x7f504c0cda80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f504c158920 tx=0x7f504402aac0 comp rx=0 tx=0).stop 2024-09-16T15:39:58.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:58.444+0000 7f5052400640 1 -- 172.21.15.151:0/3287752565 shutdown_connections 2024-09-16T15:39:58.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:58.444+0000 7f5052400640 1 --2- 172.21.15.151:0/3287752565 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f504c0cd5f0 0x7f504c0cda80 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:58.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:58.444+0000 7f5052400640 1 --2- 172.21.15.151:0/3287752565 >> v2:172.21.15.167:3300/0 conn(0x7f504c0ad880 0x7f504c0adcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:58.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:58.444+0000 7f5052400640 1 -- 172.21.15.151:0/3287752565 >> 172.21.15.151:0/3287752565 conn(0x7f504c0cfa20 msgr2=0x7f504c0cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:58.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:58.444+0000 7f5052400640 1 -- 172.21.15.151:0/3287752565 shutdown_connections 2024-09-16T15:39:58.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:58.444+0000 7f5052400640 1 -- 172.21.15.151:0/3287752565 wait complete. 2024-09-16T15:39:58.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:58.446+0000 7f5052400640 1 Processor -- start 2024-09-16T15:39:58.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:58.446+0000 7f5052400640 1 -- start start 2024-09-16T15:39:58.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:58.447+0000 7f5052400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f504c06fb90 0x7f504c0bb650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:58.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:58.447+0000 7f5052400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f504c0ad880 0x7f504c0bbb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:58.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:58.447+0000 7f5052400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f504c0cd5f0 0x7f504c1f7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:58.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:58.447+0000 7f5052400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f504c15eda0 con 0x7f504c0ad880 2024-09-16T15:39:58.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:58.447+0000 7f5052400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f504c15ec20 con 0x7f504c0cd5f0 2024-09-16T15:39:58.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:58.447+0000 7f5052400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f504c15ef20 con 0x7f504c06fb90 2024-09-16T15:39:58.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:58.447+0000 7f504be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f504c0ad880 0x7f504c0bbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:58.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:58.447+0000 7f504be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f504c0ad880 0x7f504c0bbb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43582/0 (socket says 172.21.15.151:43582) 2024-09-16T15:39:58.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:58.447+0000 7f504be00640 1 -- 172.21.15.151:0/3951997180 learned_addr learned my addr 172.21.15.151:0/3951997180 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:58.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:58.447+0000 7f5051a00640 1 --2- 172.21.15.151:0/3951997180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f504c0cd5f0 0x7f504c1f7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:58.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:58.447+0000 7f5051000640 1 --2- 172.21.15.151:0/3951997180 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f504c06fb90 0x7f504c0bb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:58.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:58.447+0000 7f504be00640 1 -- 172.21.15.151:0/3951997180 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f504c06fb90 msgr2=0x7f504c0bb650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:58.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:58.447+0000 7f504be00640 1 --2- 172.21.15.151:0/3951997180 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f504c06fb90 0x7f504c0bb650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:58.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:58.447+0000 7f504be00640 1 -- 172.21.15.151:0/3951997180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f504c0cd5f0 msgr2=0x7f504c1f7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:58.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:58.447+0000 7f504be00640 1 --2- 172.21.15.151:0/3951997180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f504c0cd5f0 0x7f504c1f7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:58.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:58.447+0000 7f504be00640 1 -- 172.21.15.151:0/3951997180 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f504c0ccac0 con 0x7f504c0ad880 2024-09-16T15:39:58.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:58.447+0000 7f5051000640 1 --2- 172.21.15.151:0/3951997180 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f504c06fb90 0x7f504c0bb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:39:58.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:58.448+0000 7f504be00640 1 --2- 172.21.15.151:0/3951997180 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f504c0ad880 0x7f504c0bbb90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f503800e9f0 tx=0x7f503800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:58.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:58.448+0000 7f5048c00640 1 -- 172.21.15.151:0/3951997180 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f503800cd80 con 0x7f504c0ad880 2024-09-16T15:39:58.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:58.448+0000 7f5048c00640 1 -- 172.21.15.151:0/3951997180 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5038010430 con 0x7f504c0ad880 2024-09-16T15:39:58.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:58.448+0000 7f5048c00640 1 -- 172.21.15.151:0/3951997180 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5038010840 con 0x7f504c0ad880 2024-09-16T15:39:58.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:58.448+0000 7f5052400640 1 -- 172.21.15.151:0/3951997180 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f504c0cc7e0 con 0x7f504c0ad880 2024-09-16T15:39:58.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:58.448+0000 7f5052400640 1 -- 172.21.15.151:0/3951997180 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f504c157a30 con 0x7f504c0ad880 2024-09-16T15:39:58.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:58.449+0000 7f5052400640 1 -- 172.21.15.151:0/3951997180 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f504c0adcd0 con 0x7f504c0ad880 2024-09-16T15:39:58.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:58.450+0000 7f5048c00640 1 -- 172.21.15.151:0/3951997180 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f50380109e0 con 0x7f504c0ad880 2024-09-16T15:39:58.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:58.451+0000 7f5048c00640 1 --2- 172.21.15.151:0/3951997180 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f502c078850 0x7f502c07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:58.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:58.452+0000 7f5048c00640 1 -- 172.21.15.151:0/3951997180 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f503809b3f0 con 0x7f504c0ad880 2024-09-16T15:39:58.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:58.452+0000 7f5051000640 1 --2- 172.21.15.151:0/3951997180 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f502c078850 0x7f502c07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:58.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:58.453+0000 7f5051000640 1 --2- 172.21.15.151:0/3951997180 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f502c078850 0x7f502c07ad10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5040007fd0 tx=0x7f5040009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:58.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:58.456+0000 7f5048c00640 1 -- 172.21.15.151:0/3951997180 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f50380634c0 con 0x7f504c0ad880 2024-09-16T15:39:58.654 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:58.651+0000 7f5052400640 1 -- 172.21.15.151:0/3951997180 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f504c06f440 con 0x7f504c0ad880 2024-09-16T15:39:58.654 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:58.652+0000 7f5048c00640 1 -- 172.21.15.151:0/3951997180 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f5038067ef0 con 0x7f504c0ad880 2024-09-16T15:39:58.654 INFO:teuthology.orchestra.run.smithi151.stderr:listed 0 entries 2024-09-16T15:39:58.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:58.654+0000 7f5052400640 1 -- 172.21.15.151:0/3951997180 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f502c078850 msgr2=0x7f502c07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:58.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:58.654+0000 7f5052400640 1 --2- 172.21.15.151:0/3951997180 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f502c078850 0x7f502c07ad10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5040007fd0 tx=0x7f5040009040 comp rx=0 tx=0).stop 2024-09-16T15:39:58.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:58.654+0000 7f5052400640 1 -- 172.21.15.151:0/3951997180 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f504c0ad880 msgr2=0x7f504c0bbb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:58.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:58.654+0000 7f5052400640 1 --2- 172.21.15.151:0/3951997180 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f504c0ad880 0x7f504c0bbb90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f503800e9f0 tx=0x7f503800eeb0 comp rx=0 tx=0).stop 2024-09-16T15:39:58.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:58.654+0000 7f5052400640 1 -- 172.21.15.151:0/3951997180 shutdown_connections 2024-09-16T15:39:58.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:58.654+0000 7f5052400640 1 --2- 172.21.15.151:0/3951997180 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f502c078850 0x7f502c07ad10 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:58.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:58.654+0000 7f5052400640 1 --2- 172.21.15.151:0/3951997180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f504c0cd5f0 0x7f504c1f7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:58.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:58.654+0000 7f5052400640 1 --2- 172.21.15.151:0/3951997180 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f504c0ad880 0x7f504c0bbb90 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:58.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:58.654+0000 7f5052400640 1 --2- 172.21.15.151:0/3951997180 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f504c06fb90 0x7f504c0bb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:58.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:58.654+0000 7f5052400640 1 -- 172.21.15.151:0/3951997180 >> 172.21.15.151:0/3951997180 conn(0x7f504c0cfa20 msgr2=0x7f504c160ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:58.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:58.654+0000 7f5052400640 1 -- 172.21.15.151:0/3951997180 shutdown_connections 2024-09-16T15:39:58.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:58.654+0000 7f5052400640 1 -- 172.21.15.151:0/3951997180 wait complete. 2024-09-16T15:39:58.671 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-09-16T15:39:58.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:58.672 DEBUG:teuthology.orchestra.run.smithi151:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-09-16T15:39:58.731 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi151.front.sepia.ceph.com 2024-09-16T15:39:58.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:58.731 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-16T15:39:58.767 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-16T15:39:58.767 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:39:58.806 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-16T15:39:58.806 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:39:58.883 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-16T15:39:58.883 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:39:58.959 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-16T15:39:59.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.116+0000 7faf6f200640 1 -- 172.21.15.151:0/2987449909 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf6815f5a0 msgr2=0x7faf680b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:59.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.116+0000 7faf6f200640 1 --2- 172.21.15.151:0/2987449909 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf6815f5a0 0x7faf680b2440 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7faf58005fd0 tx=0x7faf5802a540 comp rx=0 tx=0).stop 2024-09-16T15:39:59.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.116+0000 7faf6f200640 1 -- 172.21.15.151:0/2987449909 shutdown_connections 2024-09-16T15:39:59.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.116+0000 7faf6f200640 1 --2- 172.21.15.151:0/2987449909 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf680ccf00 0x7faf680acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:59.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.116+0000 7faf6f200640 1 --2- 172.21.15.151:0/2987449909 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf6815f5a0 0x7faf680b2440 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:59.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.116+0000 7faf6f200640 1 -- 172.21.15.151:0/2987449909 >> 172.21.15.151:0/2987449909 conn(0x7faf680b1950 msgr2=0x7faf680b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:59.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.116+0000 7faf6f200640 1 -- 172.21.15.151:0/2987449909 shutdown_connections 2024-09-16T15:39:59.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.116+0000 7faf6f200640 1 -- 172.21.15.151:0/2987449909 wait complete. 2024-09-16T15:39:59.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.118+0000 7faf6f200640 1 Processor -- start 2024-09-16T15:39:59.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.118+0000 7faf6f200640 1 -- start start 2024-09-16T15:39:59.122 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.119+0000 7faf6f200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf680ad270 0x7faf680cfd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:59.122 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.119+0000 7faf6f200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf680ccf00 0x7faf680cb450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:59.122 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.119+0000 7faf6f200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf6815f5a0 0x7faf68149280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:59.122 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.119+0000 7faf6f200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7faf6806f2e0 con 0x7faf680ad270 2024-09-16T15:39:59.122 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.119+0000 7faf6f200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7faf6806f160 con 0x7faf6815f5a0 2024-09-16T15:39:59.122 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.119+0000 7faf6f200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7faf6806f460 con 0x7faf680ccf00 2024-09-16T15:39:59.122 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.119+0000 7faf67e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf680ad270 0x7faf680cfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:59.122 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.119+0000 7faf67e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf680ad270 0x7faf680cfd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43612/0 (socket says 172.21.15.151:43612) 2024-09-16T15:39:59.122 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.119+0000 7faf67e00640 1 -- 172.21.15.151:0/19427019 learned_addr learned my addr 172.21.15.151:0/19427019 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:59.122 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.119+0000 7faf67400640 1 --2- 172.21.15.151:0/19427019 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf680ccf00 0x7faf680cb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:59.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.120+0000 7faf6d200640 1 --2- 172.21.15.151:0/19427019 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf6815f5a0 0x7faf68149280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:59.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.120+0000 7faf67e00640 1 -- 172.21.15.151:0/19427019 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf680ccf00 msgr2=0x7faf680cb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:59.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.120+0000 7faf67e00640 1 --2- 172.21.15.151:0/19427019 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf680ccf00 0x7faf680cb450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:59.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.120+0000 7faf67e00640 1 -- 172.21.15.151:0/19427019 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf6815f5a0 msgr2=0x7faf68149280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:59.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.120+0000 7faf67e00640 1 --2- 172.21.15.151:0/19427019 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf6815f5a0 0x7faf68149280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:59.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.120+0000 7faf67e00640 1 -- 172.21.15.151:0/19427019 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf6814fe70 con 0x7faf680ad270 2024-09-16T15:39:59.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.120+0000 7faf67e00640 1 --2- 172.21.15.151:0/19427019 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf680ad270 0x7faf680cfd80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7faf58000c00 tx=0x7faf580038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:59.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.120+0000 7faf5fe00640 1 -- 172.21.15.151:0/19427019 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf5803e030 con 0x7faf680ad270 2024-09-16T15:39:59.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.120+0000 7faf5fe00640 1 -- 172.21.15.151:0/19427019 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faf58003db0 con 0x7faf680ad270 2024-09-16T15:39:59.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.120+0000 7faf6f200640 1 -- 172.21.15.151:0/19427019 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf680ce080 con 0x7faf680ad270 2024-09-16T15:39:59.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.120+0000 7faf5fe00640 1 -- 172.21.15.151:0/19427019 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf58035040 con 0x7faf680ad270 2024-09-16T15:39:59.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.121+0000 7faf6f200640 1 -- 172.21.15.151:0/19427019 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf680ce640 con 0x7faf680ad270 2024-09-16T15:39:59.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.122+0000 7faf6f200640 1 -- 172.21.15.151:0/19427019 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf6806f9f0 con 0x7faf680ad270 2024-09-16T15:39:59.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.123+0000 7faf5fe00640 1 -- 172.21.15.151:0/19427019 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7faf58041030 con 0x7faf680ad270 2024-09-16T15:39:59.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.124+0000 7faf5fe00640 1 --2- 172.21.15.151:0/19427019 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7faf48078850 0x7faf4807ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:59.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.124+0000 7faf5fe00640 1 -- 172.21.15.151:0/19427019 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7faf580bae70 con 0x7faf680ad270 2024-09-16T15:39:59.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.124+0000 7faf67400640 1 --2- 172.21.15.151:0/19427019 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7faf48078850 0x7faf4807ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:59.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.125+0000 7faf67400640 1 --2- 172.21.15.151:0/19427019 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7faf48078850 0x7faf4807ad10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7faf681f80f0 tx=0x7faf54005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:59.130 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.128+0000 7faf5fe00640 1 -- 172.21.15.151:0/19427019 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7faf58082e80 con 0x7faf680ad270 2024-09-16T15:39:59.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.320+0000 7faf6f200640 1 -- 172.21.15.151:0/19427019 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7faf6815ed50 con 0x7faf680ad270 2024-09-16T15:39:59.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.321+0000 7faf5fe00640 1 -- 172.21.15.151:0/19427019 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7faf580878b0 con 0x7faf680ad270 2024-09-16T15:39:59.323 INFO:teuthology.orchestra.run.smithi151.stderr:listed 0 entries 2024-09-16T15:39:59.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.323+0000 7faf6f200640 1 -- 172.21.15.151:0/19427019 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7faf48078850 msgr2=0x7faf4807ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:59.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.323+0000 7faf6f200640 1 --2- 172.21.15.151:0/19427019 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7faf48078850 0x7faf4807ad10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7faf681f80f0 tx=0x7faf54005f90 comp rx=0 tx=0).stop 2024-09-16T15:39:59.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.323+0000 7faf6f200640 1 -- 172.21.15.151:0/19427019 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf680ad270 msgr2=0x7faf680cfd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:59.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.323+0000 7faf6f200640 1 --2- 172.21.15.151:0/19427019 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf680ad270 0x7faf680cfd80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7faf58000c00 tx=0x7faf580038a0 comp rx=0 tx=0).stop 2024-09-16T15:39:59.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.323+0000 7faf6f200640 1 -- 172.21.15.151:0/19427019 shutdown_connections 2024-09-16T15:39:59.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.323+0000 7faf6f200640 1 --2- 172.21.15.151:0/19427019 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7faf48078850 0x7faf4807ad10 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:59.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.323+0000 7faf6f200640 1 --2- 172.21.15.151:0/19427019 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf6815f5a0 0x7faf68149280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:59.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.323+0000 7faf6f200640 1 --2- 172.21.15.151:0/19427019 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf680ccf00 0x7faf680cb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:59.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.323+0000 7faf6f200640 1 --2- 172.21.15.151:0/19427019 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf680ad270 0x7faf680cfd80 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:59.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.323+0000 7faf6f200640 1 -- 172.21.15.151:0/19427019 >> 172.21.15.151:0/19427019 conn(0x7faf680b1950 msgr2=0x7faf6809d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:59.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.323+0000 7faf6f200640 1 -- 172.21.15.151:0/19427019 shutdown_connections 2024-09-16T15:39:59.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.323+0000 7faf6f200640 1 -- 172.21.15.151:0/19427019 wait complete. 2024-09-16T15:39:59.341 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-09-16T15:39:59.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:59.342 DEBUG:teuthology.orchestra.run.smithi167:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-09-16T15:39:59.403 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi167.front.sepia.ceph.com 2024-09-16T15:39:59.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:39:59.403 DEBUG:teuthology.orchestra.run.smithi167:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-16T15:39:59.439 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-16T15:39:59.439 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:39:59.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.596+0000 7f4aba600640 1 -- 172.21.15.151:0/3559988616 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ab415f5a0 msgr2=0x7f4ab40b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:59.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.596+0000 7f4aba600640 1 --2- 172.21.15.151:0/3559988616 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ab415f5a0 0x7f4ab40b2440 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4ab40cfbb0 tx=0x7f4aa402a4d0 comp rx=0 tx=0).stop 2024-09-16T15:39:59.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.596+0000 7f4aba600640 1 -- 172.21.15.151:0/3559988616 shutdown_connections 2024-09-16T15:39:59.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.596+0000 7f4aba600640 1 --2- 172.21.15.151:0/3559988616 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4ab40ad270 0x7f4ab4073fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:59.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.596+0000 7f4aba600640 1 --2- 172.21.15.151:0/3559988616 >> v2:172.21.15.167:3300/0 conn(0x7f4ab40ccf00 0x7f4ab40acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:59.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.596+0000 7f4aba600640 1 --2- 172.21.15.151:0/3559988616 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ab415f5a0 0x7f4ab40b2440 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:59.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.596+0000 7f4aba600640 1 -- 172.21.15.151:0/3559988616 >> 172.21.15.151:0/3559988616 conn(0x7f4ab40b1950 msgr2=0x7f4ab40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:59.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.596+0000 7f4aba600640 1 -- 172.21.15.151:0/3559988616 shutdown_connections 2024-09-16T15:39:59.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.596+0000 7f4aba600640 1 -- 172.21.15.151:0/3559988616 wait complete. 2024-09-16T15:39:59.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.597+0000 7f4aba600640 1 Processor -- start 2024-09-16T15:39:59.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.597+0000 7f4aba600640 1 -- start start 2024-09-16T15:39:59.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.598+0000 7f4aba600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4ab40ad270 0x7f4ab41e50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:59.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.598+0000 7f4aba600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ab40ccf00 0x7f4ab41f4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:59.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.598+0000 7f4aba600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ab415f5a0 0x7f4ab41f5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:59.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.598+0000 7f4aba600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab4074880 con 0x7f4ab40ccf00 2024-09-16T15:39:59.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.598+0000 7f4aba600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab4074700 con 0x7f4ab415f5a0 2024-09-16T15:39:59.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.598+0000 7f4aba600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f4ab406f160 con 0x7f4ab40ad270 2024-09-16T15:39:59.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.599+0000 7f4ab2a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ab40ccf00 0x7f4ab41f4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:59.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.599+0000 7f4ab2a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ab40ccf00 0x7f4ab41f4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43620/0 (socket says 172.21.15.151:43620) 2024-09-16T15:39:59.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.599+0000 7f4ab2a00640 1 -- 172.21.15.151:0/1390017213 learned_addr learned my addr 172.21.15.151:0/1390017213 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:39:59.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.599+0000 7f4ab3400640 1 --2- 172.21.15.151:0/1390017213 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4ab40ad270 0x7f4ab41e50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:59.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.599+0000 7f4ab3e00640 1 --2- 172.21.15.151:0/1390017213 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ab415f5a0 0x7f4ab41f5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:59.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.599+0000 7f4ab2a00640 1 -- 172.21.15.151:0/1390017213 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4ab40ad270 msgr2=0x7f4ab41e50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:59.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.599+0000 7f4ab2a00640 1 --2- 172.21.15.151:0/1390017213 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4ab40ad270 0x7f4ab41e50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:59.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.599+0000 7f4ab2a00640 1 -- 172.21.15.151:0/1390017213 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ab415f5a0 msgr2=0x7f4ab41f5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:59.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.599+0000 7f4ab2a00640 1 --2- 172.21.15.151:0/1390017213 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ab415f5a0 0x7f4ab41f5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:59.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.599+0000 7f4ab2a00640 1 -- 172.21.15.151:0/1390017213 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ab40c91b0 con 0x7f4ab40ccf00 2024-09-16T15:39:59.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.599+0000 7f4ab2a00640 1 --2- 172.21.15.151:0/1390017213 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ab40ccf00 0x7f4ab41f4ee0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4aa000d950 tx=0x7f4aa000dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:59.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.599+0000 7f4aab400640 1 -- 172.21.15.151:0/1390017213 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa0017070 con 0x7f4ab40ccf00 2024-09-16T15:39:59.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.600+0000 7f4aab400640 1 -- 172.21.15.151:0/1390017213 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4aa0009d20 con 0x7f4ab40ccf00 2024-09-16T15:39:59.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.600+0000 7f4aab400640 1 -- 172.21.15.151:0/1390017213 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa0004280 con 0x7f4ab40ccf00 2024-09-16T15:39:59.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.600+0000 7f4aba600640 1 -- 172.21.15.151:0/1390017213 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ab40c9490 con 0x7f4ab40ccf00 2024-09-16T15:39:59.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.600+0000 7f4aba600640 1 -- 172.21.15.151:0/1390017213 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ab41e5620 con 0x7f4ab40ccf00 2024-09-16T15:39:59.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.601+0000 7f4aba600640 1 -- 172.21.15.151:0/1390017213 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ab41e58e0 con 0x7f4ab40ccf00 2024-09-16T15:39:59.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.602+0000 7f4aab400640 1 -- 172.21.15.151:0/1390017213 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f4aa00035f0 con 0x7f4ab40ccf00 2024-09-16T15:39:59.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.603+0000 7f4aab400640 1 --2- 172.21.15.151:0/1390017213 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4a94078880 0x7f4a9407ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:39:59.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.603+0000 7f4aab400640 1 -- 172.21.15.151:0/1390017213 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f4aa009a980 con 0x7f4ab40ccf00 2024-09-16T15:39:59.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.603+0000 7f4ab3400640 1 --2- 172.21.15.151:0/1390017213 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4a94078880 0x7f4a9407ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:39:59.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.604+0000 7f4ab3400640 1 --2- 172.21.15.151:0/1390017213 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4a94078880 0x7f4a9407ad40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4aa4033000 tx=0x7f4aa4004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:39:59.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.607+0000 7f4aab400640 1 -- 172.21.15.151:0/1390017213 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4aa0062900 con 0x7f4ab40ccf00 2024-09-16T15:39:59.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.828+0000 7f4aba600640 1 -- 172.21.15.151:0/1390017213 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4ab4073fb0 con 0x7f4ab40ccf00 2024-09-16T15:39:59.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.829+0000 7f4aab400640 1 -- 172.21.15.151:0/1390017213 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5229 (secure 0 0 0) 0x7f4aa0067330 con 0x7f4ab40ccf00 2024-09-16T15:39:59.831 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:39:59.831 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":8,"btime":"2024-09-16T15:39:25:569070+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4301,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/2005239755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2005239755},{"type":"v1","addr":"172.21.15.167:6836","nonce":2005239755}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4338,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/3568704323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3568704323},{"type":"v1","addr":"172.21.15.151:6834","nonce":3568704323}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4341,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/3198287696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":3198287696},{"type":"v1","addr":"172.21.15.151:6835","nonce":3198287696}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:39:21.969407+0000","modified":"2024-09-16T15:39:24.983141+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4350},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4350":{"gid":4350,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6837/3014501393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3014501393},{"type":"v1","addr":"172.21.15.167:6837","nonce":3014501393}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4350,"qdb_cluster":[4350]},"id":1}]} 2024-09-16T15:39:59.831 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 8 2024-09-16T15:39:59.833 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.831+0000 7f4aba600640 1 -- 172.21.15.151:0/1390017213 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4a94078880 msgr2=0x7f4a9407ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:59.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.831+0000 7f4aba600640 1 --2- 172.21.15.151:0/1390017213 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4a94078880 0x7f4a9407ad40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4aa4033000 tx=0x7f4aa4004730 comp rx=0 tx=0).stop 2024-09-16T15:39:59.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.831+0000 7f4aba600640 1 -- 172.21.15.151:0/1390017213 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ab40ccf00 msgr2=0x7f4ab41f4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:39:59.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.831+0000 7f4aba600640 1 --2- 172.21.15.151:0/1390017213 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ab40ccf00 0x7f4ab41f4ee0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4aa000d950 tx=0x7f4aa000dd50 comp rx=0 tx=0).stop 2024-09-16T15:39:59.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.831+0000 7f4aba600640 1 -- 172.21.15.151:0/1390017213 shutdown_connections 2024-09-16T15:39:59.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.831+0000 7f4aba600640 1 --2- 172.21.15.151:0/1390017213 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4a94078880 0x7f4a9407ad40 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:59.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.831+0000 7f4aba600640 1 --2- 172.21.15.151:0/1390017213 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ab415f5a0 0x7f4ab41f5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:59.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.831+0000 7f4aba600640 1 --2- 172.21.15.151:0/1390017213 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ab40ccf00 0x7f4ab41f4ee0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:59.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.831+0000 7f4aba600640 1 --2- 172.21.15.151:0/1390017213 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4ab40ad270 0x7f4ab41e50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:39:59.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.831+0000 7f4aba600640 1 -- 172.21.15.151:0/1390017213 >> 172.21.15.151:0/1390017213 conn(0x7f4ab40b1950 msgr2=0x7f4ab40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:39:59.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.831+0000 7f4aba600640 1 -- 172.21.15.151:0/1390017213 shutdown_connections 2024-09-16T15:39:59.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:39:59.831+0000 7f4aba600640 1 -- 172.21.15.151:0/1390017213 wait complete. 2024-09-16T15:39:59.850 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:40:00.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.004+0000 7f50b8a00640 1 -- 172.21.15.151:0/2634214590 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f50b4161550 msgr2=0x7f50b406ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:00.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.004+0000 7f50b8a00640 1 --2- 172.21.15.151:0/2634214590 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f50b4161550 0x7f50b406ef50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f50a4005fd0 tx=0x7f50a402a540 comp rx=0 tx=0).stop 2024-09-16T15:40:00.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.004+0000 7f50b8a00640 1 -- 172.21.15.151:0/2634214590 shutdown_connections 2024-09-16T15:40:00.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.004+0000 7f50b8a00640 1 --2- 172.21.15.151:0/2634214590 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50b40af3d0 0x7f50b406f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:00.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.004+0000 7f50b8a00640 1 --2- 172.21.15.151:0/2634214590 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f50b4161550 0x7f50b406ef50 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:00.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.005+0000 7f50b8a00640 1 -- 172.21.15.151:0/2634214590 >> 172.21.15.151:0/2634214590 conn(0x7f50b40b1950 msgr2=0x7f50b40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:00.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.005+0000 7f50b8a00640 1 -- 172.21.15.151:0/2634214590 shutdown_connections 2024-09-16T15:40:00.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.005+0000 7f50b8a00640 1 -- 172.21.15.151:0/2634214590 wait complete. 2024-09-16T15:40:00.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.006+0000 7f50b8a00640 1 Processor -- start 2024-09-16T15:40:00.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.006+0000 7f50b8a00640 1 -- start start 2024-09-16T15:40:00.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.007+0000 7f50b8a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50b406f9d0 0x7f50b41568f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:00.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.007+0000 7f50b8a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f50b40af3d0 0x7f50b40ccec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:00.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.007+0000 7f50b8a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50b4161550 0x7f50b41f84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:00.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.007+0000 7f50b8a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f50b41623a0 con 0x7f50b40af3d0 2024-09-16T15:40:00.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.007+0000 7f50b8a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f50b4074010 con 0x7f50b4161550 2024-09-16T15:40:00.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.007+0000 7f50b8a00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f50b4162520 con 0x7f50b406f9d0 2024-09-16T15:40:00.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.007+0000 7f50b1400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f50b40af3d0 0x7f50b40ccec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:00.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.007+0000 7f50b1400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f50b40af3d0 0x7f50b40ccec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43644/0 (socket says 172.21.15.151:43644) 2024-09-16T15:40:00.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.008+0000 7f50b1400640 1 -- 172.21.15.151:0/3020196707 learned_addr learned my addr 172.21.15.151:0/3020196707 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:40:00.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.008+0000 7f50b1e00640 1 --2- 172.21.15.151:0/3020196707 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50b406f9d0 0x7f50b41568f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:00.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.008+0000 7f50b2800640 1 --2- 172.21.15.151:0/3020196707 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50b4161550 0x7f50b41f84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:00.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.008+0000 7f50b1400640 1 -- 172.21.15.151:0/3020196707 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50b406f9d0 msgr2=0x7f50b41568f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:00.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.008+0000 7f50b1400640 1 --2- 172.21.15.151:0/3020196707 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50b406f9d0 0x7f50b41568f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:00.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.008+0000 7f50b1400640 1 -- 172.21.15.151:0/3020196707 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50b4161550 msgr2=0x7f50b41f84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:00.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.008+0000 7f50b1400640 1 --2- 172.21.15.151:0/3020196707 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50b4161550 0x7f50b41f84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:00.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.008+0000 7f50b1400640 1 -- 172.21.15.151:0/3020196707 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50b41576d0 con 0x7f50b40af3d0 2024-09-16T15:40:00.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.008+0000 7f50b1400640 1 --2- 172.21.15.151:0/3020196707 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f50b40af3d0 0x7f50b40ccec0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f50a000d950 tx=0x7f50a000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:00.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.008+0000 7f50aa000640 1 -- 172.21.15.151:0/3020196707 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a0017070 con 0x7f50b40af3d0 2024-09-16T15:40:00.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.008+0000 7f50aa000640 1 -- 172.21.15.151:0/3020196707 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f50a0009d20 con 0x7f50b40af3d0 2024-09-16T15:40:00.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.008+0000 7f50b8a00640 1 -- 172.21.15.151:0/3020196707 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50b41e8f40 con 0x7f50b40af3d0 2024-09-16T15:40:00.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.009+0000 7f50aa000640 1 -- 172.21.15.151:0/3020196707 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a00106c0 con 0x7f50b40af3d0 2024-09-16T15:40:00.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.009+0000 7f50b8a00640 1 -- 172.21.15.151:0/3020196707 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50b41e9500 con 0x7f50b40af3d0 2024-09-16T15:40:00.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.010+0000 7f50b8a00640 1 -- 172.21.15.151:0/3020196707 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50b40760b0 con 0x7f50b40af3d0 2024-09-16T15:40:00.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.011+0000 7f50aa000640 1 -- 172.21.15.151:0/3020196707 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f50a001a070 con 0x7f50b40af3d0 2024-09-16T15:40:00.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.012+0000 7f50aa000640 1 --2- 172.21.15.151:0/3020196707 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5094078850 0x7f509407ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:00.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.012+0000 7f50aa000640 1 -- 172.21.15.151:0/3020196707 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f50a009ab20 con 0x7f50b40af3d0 2024-09-16T15:40:00.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.012+0000 7f50b1e00640 1 --2- 172.21.15.151:0/3020196707 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5094078850 0x7f509407ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:00.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.013+0000 7f50b1e00640 1 --2- 172.21.15.151:0/3020196707 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5094078850 0x7f509407ad10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f50a4033000 tx=0x7f50a4004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:00.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.016+0000 7f50aa000640 1 -- 172.21.15.151:0/3020196707 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f50a0062aa0 con 0x7f50b40af3d0 2024-09-16T15:40:00.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.237+0000 7f50b8a00640 1 -- 172.21.15.151:0/3020196707 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f50b4162ab0 con 0x7f50b40af3d0 2024-09-16T15:40:00.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.238+0000 7f50aa000640 1 -- 172.21.15.151:0/3020196707 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5229 (secure 0 0 0) 0x7f50a00674d0 con 0x7f50b40af3d0 2024-09-16T15:40:00.240 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:40:00.240 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":8,"btime":"2024-09-16T15:39:25:569070+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4301,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/2005239755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2005239755},{"type":"v1","addr":"172.21.15.167:6836","nonce":2005239755}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4338,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/3568704323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3568704323},{"type":"v1","addr":"172.21.15.151:6834","nonce":3568704323}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4341,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/3198287696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":3198287696},{"type":"v1","addr":"172.21.15.151:6835","nonce":3198287696}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:39:21.969407+0000","modified":"2024-09-16T15:39:24.983141+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4350},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4350":{"gid":4350,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6837/3014501393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3014501393},{"type":"v1","addr":"172.21.15.167:6837","nonce":3014501393}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4350,"qdb_cluster":[4350]},"id":1}]} 2024-09-16T15:40:00.240 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 8 2024-09-16T15:40:00.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.240+0000 7f50b8a00640 1 -- 172.21.15.151:0/3020196707 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5094078850 msgr2=0x7f509407ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:00.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.240+0000 7f50b8a00640 1 --2- 172.21.15.151:0/3020196707 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5094078850 0x7f509407ad10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f50a4033000 tx=0x7f50a4004730 comp rx=0 tx=0).stop 2024-09-16T15:40:00.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.240+0000 7f50b8a00640 1 -- 172.21.15.151:0/3020196707 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f50b40af3d0 msgr2=0x7f50b40ccec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:00.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.240+0000 7f50b8a00640 1 --2- 172.21.15.151:0/3020196707 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f50b40af3d0 0x7f50b40ccec0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f50a000d950 tx=0x7f50a000de10 comp rx=0 tx=0).stop 2024-09-16T15:40:00.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.240+0000 7f50b8a00640 1 -- 172.21.15.151:0/3020196707 shutdown_connections 2024-09-16T15:40:00.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.240+0000 7f50b8a00640 1 --2- 172.21.15.151:0/3020196707 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5094078850 0x7f509407ad10 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:00.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.240+0000 7f50b8a00640 1 --2- 172.21.15.151:0/3020196707 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50b4161550 0x7f50b41f84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:00.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.240+0000 7f50b8a00640 1 --2- 172.21.15.151:0/3020196707 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f50b40af3d0 0x7f50b40ccec0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:00.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.240+0000 7f50b8a00640 1 --2- 172.21.15.151:0/3020196707 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50b406f9d0 0x7f50b41568f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:00.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.240+0000 7f50b8a00640 1 -- 172.21.15.151:0/3020196707 >> 172.21.15.151:0/3020196707 conn(0x7f50b40b1950 msgr2=0x7f50b40af870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:00.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.240+0000 7f50b8a00640 1 -- 172.21.15.151:0/3020196707 shutdown_connections 2024-09-16T15:40:00.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.240+0000 7f50b8a00640 1 -- 172.21.15.151:0/3020196707 wait complete. 2024-09-16T15:40:00.259 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:40:00.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.417+0000 7fc006800640 1 -- 172.21.15.151:0/454847300 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc0001445d0 msgr2=0x7fc0000cebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:00.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.417+0000 7fc006800640 1 --2- 172.21.15.151:0/454847300 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc0001445d0 0x7fc0000cebf0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbff0005fd0 tx=0x7fbff002aae0 comp rx=0 tx=0).stop 2024-09-16T15:40:00.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.417+0000 7fc006800640 1 -- 172.21.15.151:0/454847300 shutdown_connections 2024-09-16T15:40:00.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.417+0000 7fc006800640 1 --2- 172.21.15.151:0/454847300 >> v2:172.21.15.167:3300/0 conn(0x7fc0000ccfd0 0x7fc0000cf130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:00.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.417+0000 7fc006800640 1 --2- 172.21.15.151:0/454847300 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc0001445d0 0x7fc0000cebf0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:00.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.417+0000 7fc006800640 1 -- 172.21.15.151:0/454847300 >> 172.21.15.151:0/454847300 conn(0x7fc0000b1950 msgr2=0x7fc0000b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:00.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.417+0000 7fc006800640 1 -- 172.21.15.151:0/454847300 shutdown_connections 2024-09-16T15:40:00.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.417+0000 7fc006800640 1 -- 172.21.15.151:0/454847300 wait complete. 2024-09-16T15:40:00.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.419+0000 7fc006800640 1 Processor -- start 2024-09-16T15:40:00.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.419+0000 7fc006800640 1 -- start start 2024-09-16T15:40:00.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.420+0000 7fc006800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc0000b4ca0 0x7fc0001f3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:00.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.420+0000 7fc006800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0000ccfd0 0x7fc0001f6a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:00.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.420+0000 7fc006800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0001445d0 0x7fc00009cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:00.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.420+0000 7fc006800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc00015f080 con 0x7fc0000b4ca0 2024-09-16T15:40:00.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.420+0000 7fc006800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc000073bc0 con 0x7fc0001445d0 2024-09-16T15:40:00.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.420+0000 7fc006800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fc00015f200 con 0x7fc0000ccfd0 2024-09-16T15:40:00.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.420+0000 7fbfff400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc0000b4ca0 0x7fc0001f3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:00.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.420+0000 7fbfff400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc0000b4ca0 0x7fc0001f3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43656/0 (socket says 172.21.15.151:43656) 2024-09-16T15:40:00.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.420+0000 7fbfff400640 1 -- 172.21.15.151:0/284346184 learned_addr learned my addr 172.21.15.151:0/284346184 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:40:00.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.420+0000 7fbfffe00640 1 --2- 172.21.15.151:0/284346184 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0001445d0 0x7fc00009cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:00.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.420+0000 7fbffea00640 1 --2- 172.21.15.151:0/284346184 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0000ccfd0 0x7fc0001f6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:00.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.421+0000 7fbfff400640 1 -- 172.21.15.151:0/284346184 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0000ccfd0 msgr2=0x7fc0001f6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:00.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.421+0000 7fbfff400640 1 --2- 172.21.15.151:0/284346184 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0000ccfd0 0x7fc0001f6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:00.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.421+0000 7fbfff400640 1 -- 172.21.15.151:0/284346184 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0001445d0 msgr2=0x7fc00009cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:00.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.421+0000 7fbfff400640 1 --2- 172.21.15.151:0/284346184 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0001445d0 0x7fc00009cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:00.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.421+0000 7fbfff400640 1 -- 172.21.15.151:0/284346184 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0001f6fa0 con 0x7fc0000b4ca0 2024-09-16T15:40:00.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.421+0000 7fbfff400640 1 --2- 172.21.15.151:0/284346184 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc0000b4ca0 0x7fc0001f3520 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbff0000f80 tx=0x7fbff0003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:00.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.421+0000 7fbff7400640 1 -- 172.21.15.151:0/284346184 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff0044070 con 0x7fc0000b4ca0 2024-09-16T15:40:00.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.421+0000 7fbff7400640 1 -- 172.21.15.151:0/284346184 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbff0004250 con 0x7fc0000b4ca0 2024-09-16T15:40:00.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.421+0000 7fc006800640 1 -- 172.21.15.151:0/284346184 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0001e5150 con 0x7fc0000b4ca0 2024-09-16T15:40:00.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.421+0000 7fbff7400640 1 -- 172.21.15.151:0/284346184 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff0004660 con 0x7fc0000b4ca0 2024-09-16T15:40:00.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.421+0000 7fc006800640 1 -- 172.21.15.151:0/284346184 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0001f2ab0 con 0x7fc0000b4ca0 2024-09-16T15:40:00.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.422+0000 7fc006800640 1 -- 172.21.15.151:0/284346184 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc00015f740 con 0x7fc0000b4ca0 2024-09-16T15:40:00.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.424+0000 7fbff7400640 1 -- 172.21.15.151:0/284346184 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fbff0047070 con 0x7fc0000b4ca0 2024-09-16T15:40:00.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.425+0000 7fbff7400640 1 --2- 172.21.15.151:0/284346184 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbfe0078850 0x7fbfe007ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:00.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.425+0000 7fbff7400640 1 -- 172.21.15.151:0/284346184 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fbff00bb6e0 con 0x7fc0000b4ca0 2024-09-16T15:40:00.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.425+0000 7fbffea00640 1 --2- 172.21.15.151:0/284346184 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbfe0078850 0x7fbfe007ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:00.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.426+0000 7fbffea00640 1 --2- 172.21.15.151:0/284346184 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbfe0078850 0x7fbfe007ad10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc0001e5020 tx=0x7fbfec00b390 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:00.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.429+0000 7fbff7400640 1 -- 172.21.15.151:0/284346184 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc0001f6fa0 con 0x7fc0000b4ca0 2024-09-16T15:40:00.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.614+0000 7fc006800640 1 -- 172.21.15.151:0/284346184 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc000070a20 con 0x7fc0000b4ca0 2024-09-16T15:40:00.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.615+0000 7fbff7400640 1 -- 172.21.15.151:0/284346184 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16729 (secure 0 0 0) 0x7fbff0032030 con 0x7fc0000b4ca0 2024-09-16T15:40:00.618 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:40:00.618 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":22,"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","created":"2024-09-16T15:38:56.348535+0000","modified":"2024-09-16T15:39:23.550834+0000","last_up_change":"2024-09-16T15:39:10.681384+0000","last_in_change":"2024-09-16T15:39:00.544477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:39:11.602252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:39:19.970182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T15:39:20.929376+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5524076-a903-4d0a-af6a-caf474c4765d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6809","nonce":3111342741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6811","nonce":3111342741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6815","nonce":3111342741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6813","nonce":3111342741}]},"public_addr":"172.21.15.151:6809/3111342741","cluster_addr":"172.21.15.151:6811/3111342741","heartbeat_back_addr":"172.21.15.151:6815/3111342741","heartbeat_front_addr":"172.21.15.151:6813/3111342741","state":["exists","up"]},{"osd":1,"uuid":"dd25925f-0ae1-494e-8529-4dc205759db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6801","nonce":3368304087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6803","nonce":3368304087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6807","nonce":3368304087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6805","nonce":3368304087}]},"public_addr":"172.21.15.151:6801/3368304087","cluster_addr":"172.21.15.151:6803/3368304087","heartbeat_back_addr":"172.21.15.151:6807/3368304087","heartbeat_front_addr":"172.21.15.151:6805/3368304087","state":["exists","up"]},{"osd":2,"uuid":"c2f18115-013d-4da6-a0fe-cb978f4494b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6823","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6825","nonce":3079373271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6827","nonce":3079373271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6831","nonce":3079373271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6829","nonce":3079373271}]},"public_addr":"172.21.15.151:6825/3079373271","cluster_addr":"172.21.15.151:6827/3079373271","heartbeat_back_addr":"172.21.15.151:6831/3079373271","heartbeat_front_addr":"172.21.15.151:6829/3079373271","state":["exists","up"]},{"osd":3,"uuid":"36edb718-2fcb-4d88-8ddc-c321eed4279d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6817","nonce":1142509041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6819","nonce":1142509041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6824","nonce":1142509041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6821","nonce":1142509041}]},"public_addr":"172.21.15.151:6817/1142509041","cluster_addr":"172.21.15.151:6819/1142509041","heartbeat_back_addr":"172.21.15.151:6824/1142509041","heartbeat_front_addr":"172.21.15.151:6821/1142509041","state":["exists","up"]},{"osd":4,"uuid":"64d33708-cd0a-4236-abe3-1b646e82ab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6811","nonce":2735495271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6813","nonce":2735495271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6817","nonce":2735495271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6815","nonce":2735495271}]},"public_addr":"172.21.15.167:6811/2735495271","cluster_addr":"172.21.15.167:6813/2735495271","heartbeat_back_addr":"172.21.15.167:6817/2735495271","heartbeat_front_addr":"172.21.15.167:6815/2735495271","state":["exists","up"]},{"osd":5,"uuid":"f14b44a9-5910-458c-b518-09a8d0de94de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6827","nonce":1494916572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6829","nonce":1494916572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6833","nonce":1494916572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6831","nonce":1494916572}]},"public_addr":"172.21.15.167:6827/1494916572","cluster_addr":"172.21.15.167:6829/1494916572","heartbeat_back_addr":"172.21.15.167:6833/1494916572","heartbeat_front_addr":"172.21.15.167:6831/1494916572","state":["exists","up"]},{"osd":6,"uuid":"39335545-2654-4710-84b0-a9aa47cfcbba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6819","nonce":2538273273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6821","nonce":2538273273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6825","nonce":2538273273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6823","nonce":2538273273}]},"public_addr":"172.21.15.167:6819/2538273273","cluster_addr":"172.21.15.167:6821/2538273273","heartbeat_back_addr":"172.21.15.167:6825/2538273273","heartbeat_front_addr":"172.21.15.167:6823/2538273273","state":["exists","up"]},{"osd":7,"uuid":"4f8bfc81-53dd-4339-ac22-1a4007d9f963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6803","nonce":265072227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6805","nonce":265072227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6809","nonce":265072227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6807","nonce":265072227}]},"public_addr":"172.21.15.167:6803/265072227","cluster_addr":"172.21.15.167:6805/265072227","heartbeat_back_addr":"172.21.15.167:6809/265072227","heartbeat_front_addr":"172.21.15.167:6807/265072227","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.150359+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.084358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.722368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.332809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.320709+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.395153+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.935250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.801443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:40:00.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.617+0000 7fc006800640 1 -- 172.21.15.151:0/284346184 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbfe0078850 msgr2=0x7fbfe007ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:00.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.617+0000 7fc006800640 1 --2- 172.21.15.151:0/284346184 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbfe0078850 0x7fbfe007ad10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc0001e5020 tx=0x7fbfec00b390 comp rx=0 tx=0).stop 2024-09-16T15:40:00.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.617+0000 7fc006800640 1 -- 172.21.15.151:0/284346184 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc0000b4ca0 msgr2=0x7fc0001f3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:00.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.617+0000 7fc006800640 1 --2- 172.21.15.151:0/284346184 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc0000b4ca0 0x7fc0001f3520 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbff0000f80 tx=0x7fbff0003f10 comp rx=0 tx=0).stop 2024-09-16T15:40:00.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.618+0000 7fc006800640 1 -- 172.21.15.151:0/284346184 shutdown_connections 2024-09-16T15:40:00.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.618+0000 7fc006800640 1 --2- 172.21.15.151:0/284346184 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbfe0078850 0x7fbfe007ad10 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:00.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.618+0000 7fc006800640 1 --2- 172.21.15.151:0/284346184 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0001445d0 0x7fc00009cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:00.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.618+0000 7fc006800640 1 --2- 172.21.15.151:0/284346184 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0000ccfd0 0x7fc0001f6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:00.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.618+0000 7fc006800640 1 --2- 172.21.15.151:0/284346184 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc0000b4ca0 0x7fc0001f3520 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:00.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.618+0000 7fc006800640 1 -- 172.21.15.151:0/284346184 >> 172.21.15.151:0/284346184 conn(0x7fc0000b1950 msgr2=0x7fc0000cf840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:00.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.618+0000 7fc006800640 1 -- 172.21.15.151:0/284346184 shutdown_connections 2024-09-16T15:40:00.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.618+0000 7fc006800640 1 -- 172.21.15.151:0/284346184 wait complete. 2024-09-16T15:40:00.637 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-16T15:40:00.638 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:40:00.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.795+0000 7f7906c00640 1 -- 172.21.15.151:0/512559202 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79000ccf00 msgr2=0x7f79000acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:00.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.795+0000 7f7906c00640 1 --2- 172.21.15.151:0/512559202 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79000ccf00 0x7f79000acd30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7900075130 tx=0x7f78ec02a4d0 comp rx=0 tx=0).stop 2024-09-16T15:40:00.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.795+0000 7f7906c00640 1 -- 172.21.15.151:0/512559202 shutdown_connections 2024-09-16T15:40:00.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.795+0000 7f7906c00640 1 --2- 172.21.15.151:0/512559202 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79000ad270 0x7f7900073fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:00.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.795+0000 7f7906c00640 1 --2- 172.21.15.151:0/512559202 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79000ccf00 0x7f79000acd30 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:00.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.795+0000 7f7906c00640 1 --2- 172.21.15.151:0/512559202 >> v2:172.21.15.167:3300/0 conn(0x7f790015f5a0 0x7f79000b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:00.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.795+0000 7f7906c00640 1 -- 172.21.15.151:0/512559202 >> 172.21.15.151:0/512559202 conn(0x7f79000b1950 msgr2=0x7f79000b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:00.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.795+0000 7f7906c00640 1 -- 172.21.15.151:0/512559202 shutdown_connections 2024-09-16T15:40:00.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.795+0000 7f7906c00640 1 -- 172.21.15.151:0/512559202 wait complete. 2024-09-16T15:40:00.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.797+0000 7f7906c00640 1 Processor -- start 2024-09-16T15:40:00.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.797+0000 7f7906c00640 1 -- start start 2024-09-16T15:40:00.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.798+0000 7f7906c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79000ad270 0x7f79000b4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:00.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.798+0000 7f7906c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79000ccf00 0x7f790009aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:00.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.798+0000 7f7906c00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f790015f5a0 0x7f79000cc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:00.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.798+0000 7f7906c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7900074880 con 0x7f79000ad270 2024-09-16T15:40:00.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.798+0000 7f7906c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7900074700 con 0x7f79000ccf00 2024-09-16T15:40:00.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.798+0000 7f7906c00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f790006f160 con 0x7f790015f5a0 2024-09-16T15:40:00.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.798+0000 7f78ffe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79000ad270 0x7f79000b4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:00.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.798+0000 7f78ffe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79000ad270 0x7f79000b4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43668/0 (socket says 172.21.15.151:43668) 2024-09-16T15:40:00.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.798+0000 7f78ffe00640 1 -- 172.21.15.151:0/463070055 learned_addr learned my addr 172.21.15.151:0/463070055 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:40:00.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.799+0000 7f78ff400640 1 --2- 172.21.15.151:0/463070055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79000ccf00 0x7f790009aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:00.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.799+0000 7f7904c00640 1 --2- 172.21.15.151:0/463070055 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f790015f5a0 0x7f79000cc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:00.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.799+0000 7f78ffe00640 1 -- 172.21.15.151:0/463070055 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f790015f5a0 msgr2=0x7f79000cc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:00.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.799+0000 7f78ffe00640 1 --2- 172.21.15.151:0/463070055 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f790015f5a0 0x7f79000cc7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:00.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.799+0000 7f78ffe00640 1 -- 172.21.15.151:0/463070055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79000ccf00 msgr2=0x7f790009aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:00.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.799+0000 7f78ffe00640 1 --2- 172.21.15.151:0/463070055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79000ccf00 0x7f790009aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:00.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.799+0000 7f78ffe00640 1 -- 172.21.15.151:0/463070055 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79000ccd10 con 0x7f79000ad270 2024-09-16T15:40:00.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.799+0000 7f78ffe00640 1 --2- 172.21.15.151:0/463070055 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79000ad270 0x7f79000b4a00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f78f000c9a0 tx=0x7f78f000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:00.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.799+0000 7f78f7e00640 1 -- 172.21.15.151:0/463070055 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f0007bb0 con 0x7f79000ad270 2024-09-16T15:40:00.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.799+0000 7f78f7e00640 1 -- 172.21.15.151:0/463070055 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f78f0007e40 con 0x7f79000ad270 2024-09-16T15:40:00.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.799+0000 7f78f7e00640 1 -- 172.21.15.151:0/463070055 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f000f690 con 0x7f79000ad270 2024-09-16T15:40:00.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.799+0000 7f7906c00640 1 -- 172.21.15.151:0/463070055 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f790009b090 con 0x7f79000ad270 2024-09-16T15:40:00.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.799+0000 7f7906c00640 1 -- 172.21.15.151:0/463070055 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f790009b650 con 0x7f79000ad270 2024-09-16T15:40:00.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.800+0000 7f7906c00640 1 -- 172.21.15.151:0/463070055 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7900073fb0 con 0x7f79000ad270 2024-09-16T15:40:00.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.802+0000 7f78f7e00640 1 -- 172.21.15.151:0/463070055 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f78f000f830 con 0x7f79000ad270 2024-09-16T15:40:00.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.802+0000 7f78f7e00640 1 --2- 172.21.15.151:0/463070055 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f78e0078850 0x7f78e007ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:00.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.803+0000 7f78f7e00640 1 -- 172.21.15.151:0/463070055 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f78f009b110 con 0x7f79000ad270 2024-09-16T15:40:00.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.803+0000 7f78ff400640 1 --2- 172.21.15.151:0/463070055 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f78e0078850 0x7f78e007ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:00.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.804+0000 7f78ff400640 1 --2- 172.21.15.151:0/463070055 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f78e0078850 0x7f78e007ad10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f78ec033000 tx=0x7f78ec004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:00.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:00.807+0000 7f78f7e00640 1 -- 172.21.15.151:0/463070055 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f78f0014030 con 0x7f79000ad270 2024-09-16T15:40:01.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.028+0000 7f7906c00640 1 -- 172.21.15.151:0/463070055 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f79000ae200 con 0x7f79000ad270 2024-09-16T15:40:01.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.028+0000 7f78f7e00640 1 -- 172.21.15.151:0/463070055 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5229 (secure 0 0 0) 0x7f78f00636e0 con 0x7f79000ad270 2024-09-16T15:40:01.030 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:40:01.030 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":8,"btime":"2024-09-16T15:39:25:569070+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4301,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/2005239755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2005239755},{"type":"v1","addr":"172.21.15.167:6836","nonce":2005239755}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4338,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/3568704323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3568704323},{"type":"v1","addr":"172.21.15.151:6834","nonce":3568704323}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4341,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/3198287696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":3198287696},{"type":"v1","addr":"172.21.15.151:6835","nonce":3198287696}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:39:21.969407+0000","modified":"2024-09-16T15:39:24.983141+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4350},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4350":{"gid":4350,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6837/3014501393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3014501393},{"type":"v1","addr":"172.21.15.167:6837","nonce":3014501393}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4350,"qdb_cluster":[4350]},"id":1}]} 2024-09-16T15:40:01.031 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 8 2024-09-16T15:40:01.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.030+0000 7f7906c00640 1 -- 172.21.15.151:0/463070055 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f78e0078850 msgr2=0x7f78e007ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:01.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.030+0000 7f7906c00640 1 --2- 172.21.15.151:0/463070055 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f78e0078850 0x7f78e007ad10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f78ec033000 tx=0x7f78ec004730 comp rx=0 tx=0).stop 2024-09-16T15:40:01.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.030+0000 7f7906c00640 1 -- 172.21.15.151:0/463070055 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79000ad270 msgr2=0x7f79000b4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:01.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.030+0000 7f7906c00640 1 --2- 172.21.15.151:0/463070055 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79000ad270 0x7f79000b4a00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f78f000c9a0 tx=0x7f78f000ce60 comp rx=0 tx=0).stop 2024-09-16T15:40:01.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.030+0000 7f7906c00640 1 -- 172.21.15.151:0/463070055 shutdown_connections 2024-09-16T15:40:01.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.030+0000 7f7906c00640 1 --2- 172.21.15.151:0/463070055 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f78e0078850 0x7f78e007ad10 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:01.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.030+0000 7f7906c00640 1 --2- 172.21.15.151:0/463070055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79000ccf00 0x7f790009aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:01.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.030+0000 7f7906c00640 1 --2- 172.21.15.151:0/463070055 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f790015f5a0 0x7f79000cc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:01.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.030+0000 7f7906c00640 1 --2- 172.21.15.151:0/463070055 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79000ad270 0x7f79000b4a00 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:01.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.030+0000 7f7906c00640 1 -- 172.21.15.151:0/463070055 >> 172.21.15.151:0/463070055 conn(0x7f79000b1950 msgr2=0x7f79000b0310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:01.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.031+0000 7f7906c00640 1 -- 172.21.15.151:0/463070055 shutdown_connections 2024-09-16T15:40:01.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.031+0000 7f7906c00640 1 -- 172.21.15.151:0/463070055 wait complete. 2024-09-16T15:40:01.048 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:40:01.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.202+0000 7f2751400640 1 -- 172.21.15.151:0/3763096352 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f274c15f5a0 msgr2=0x7f274c0b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:01.205 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.202+0000 7f2751400640 1 --2- 172.21.15.151:0/3763096352 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f274c15f5a0 0x7f274c0b2440 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f273c005fd0 tx=0x7f273c02a4a0 comp rx=0 tx=0).stop 2024-09-16T15:40:01.205 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.202+0000 7f2751400640 1 -- 172.21.15.151:0/3763096352 shutdown_connections 2024-09-16T15:40:01.205 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.202+0000 7f2751400640 1 --2- 172.21.15.151:0/3763096352 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f274c073fb0 0x7f274c074440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:01.205 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.202+0000 7f2751400640 1 --2- 172.21.15.151:0/3763096352 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f274c15f5a0 0x7f274c0b2440 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:01.205 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.202+0000 7f2751400640 1 -- 172.21.15.151:0/3763096352 >> 172.21.15.151:0/3763096352 conn(0x7f274c0b1950 msgr2=0x7f274c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:01.205 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.202+0000 7f2751400640 1 -- 172.21.15.151:0/3763096352 shutdown_connections 2024-09-16T15:40:01.205 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.202+0000 7f2751400640 1 -- 172.21.15.151:0/3763096352 wait complete. 2024-09-16T15:40:01.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.204+0000 7f2751400640 1 Processor -- start 2024-09-16T15:40:01.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.204+0000 7f2751400640 1 -- start start 2024-09-16T15:40:01.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.205+0000 7f2751400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f274c073fb0 0x7f274c150ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:01.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.205+0000 7f2751400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f274c0ccf00 0x7f274c146510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:01.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.205+0000 7f2751400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f274c15f5a0 0x7f274c152410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:01.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.205+0000 7f2751400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f274c06f2e0 con 0x7f274c15f5a0 2024-09-16T15:40:01.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.205+0000 7f2751400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f274c06f160 con 0x7f274c073fb0 2024-09-16T15:40:01.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.205+0000 7f2751400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f274c06f460 con 0x7f274c0ccf00 2024-09-16T15:40:01.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.205+0000 7f274b200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f274c15f5a0 0x7f274c152410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:01.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.205+0000 7f274b200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f274c15f5a0 0x7f274c152410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43692/0 (socket says 172.21.15.151:43692) 2024-09-16T15:40:01.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.205+0000 7f274b200640 1 -- 172.21.15.151:0/412865818 learned_addr learned my addr 172.21.15.151:0/412865818 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:40:01.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.205+0000 7f274a800640 1 --2- 172.21.15.151:0/412865818 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f274c073fb0 0x7f274c150ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:01.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.205+0000 7f2749e00640 1 --2- 172.21.15.151:0/412865818 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f274c0ccf00 0x7f274c146510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:01.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.205+0000 7f274b200640 1 -- 172.21.15.151:0/412865818 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f274c0ccf00 msgr2=0x7f274c146510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:01.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.205+0000 7f274b200640 1 --2- 172.21.15.151:0/412865818 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f274c0ccf00 0x7f274c146510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:01.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.205+0000 7f274b200640 1 -- 172.21.15.151:0/412865818 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f274c073fb0 msgr2=0x7f274c150ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:01.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.205+0000 7f274b200640 1 --2- 172.21.15.151:0/412865818 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f274c073fb0 0x7f274c150ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:01.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.205+0000 7f274b200640 1 -- 172.21.15.151:0/412865818 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f274c1506e0 con 0x7f274c15f5a0 2024-09-16T15:40:01.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.205+0000 7f2749e00640 1 --2- 172.21.15.151:0/412865818 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f274c0ccf00 0x7f274c146510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:40:01.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.205+0000 7f274a800640 1 --2- 172.21.15.151:0/412865818 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f274c073fb0 0x7f274c150ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:40:01.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.206+0000 7f274b200640 1 --2- 172.21.15.151:0/412865818 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f274c15f5a0 0x7f274c152410 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f274400ead0 tx=0x7f274400ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:01.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.206+0000 7f2742a00640 1 -- 172.21.15.151:0/412865818 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f274400cde0 con 0x7f274c15f5a0 2024-09-16T15:40:01.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.206+0000 7f2742a00640 1 -- 172.21.15.151:0/412865818 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2744004c10 con 0x7f274c15f5a0 2024-09-16T15:40:01.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.206+0000 7f2751400640 1 -- 172.21.15.151:0/412865818 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f274c153290 con 0x7f274c15f5a0 2024-09-16T15:40:01.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.206+0000 7f2742a00640 1 -- 172.21.15.151:0/412865818 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2744005020 con 0x7f274c15f5a0 2024-09-16T15:40:01.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.206+0000 7f2751400640 1 -- 172.21.15.151:0/412865818 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f274c146a50 con 0x7f274c15f5a0 2024-09-16T15:40:01.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.207+0000 7f2751400640 1 -- 172.21.15.151:0/412865818 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f274c154e10 con 0x7f274c15f5a0 2024-09-16T15:40:01.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.208+0000 7f2742a00640 1 -- 172.21.15.151:0/412865818 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f27440051c0 con 0x7f274c15f5a0 2024-09-16T15:40:01.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.209+0000 7f2742a00640 1 --2- 172.21.15.151:0/412865818 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f272c078850 0x7f272c07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:01.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.209+0000 7f2742a00640 1 -- 172.21.15.151:0/412865818 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f274409b2e0 con 0x7f274c15f5a0 2024-09-16T15:40:01.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.210+0000 7f274a800640 1 --2- 172.21.15.151:0/412865818 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f272c078850 0x7f272c07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:01.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.210+0000 7f274a800640 1 --2- 172.21.15.151:0/412865818 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f272c078850 0x7f272c07ad10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f273c033000 tx=0x7f273c004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:01.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.213+0000 7f2742a00640 1 -- 172.21.15.151:0/412865818 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2744016030 con 0x7f274c15f5a0 2024-09-16T15:40:01.434 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.432+0000 7f2751400640 1 -- 172.21.15.151:0/412865818 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f274c0cf6e0 con 0x7f274c15f5a0 2024-09-16T15:40:01.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.433+0000 7f2742a00640 1 -- 172.21.15.151:0/412865818 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5229 (secure 0 0 0) 0x7f2744063940 con 0x7f274c15f5a0 2024-09-16T15:40:01.435 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:40:01.435 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":8,"btime":"2024-09-16T15:39:25:569070+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4301,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/2005239755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2005239755},{"type":"v1","addr":"172.21.15.167:6836","nonce":2005239755}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4338,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/3568704323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3568704323},{"type":"v1","addr":"172.21.15.151:6834","nonce":3568704323}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4341,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/3198287696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":3198287696},{"type":"v1","addr":"172.21.15.151:6835","nonce":3198287696}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:39:21.969407+0000","modified":"2024-09-16T15:39:24.983141+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4350},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4350":{"gid":4350,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6837/3014501393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3014501393},{"type":"v1","addr":"172.21.15.167:6837","nonce":3014501393}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4350,"qdb_cluster":[4350]},"id":1}]} 2024-09-16T15:40:01.435 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 8 2024-09-16T15:40:01.437 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.435+0000 7f2751400640 1 -- 172.21.15.151:0/412865818 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f272c078850 msgr2=0x7f272c07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:01.438 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.435+0000 7f2751400640 1 --2- 172.21.15.151:0/412865818 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f272c078850 0x7f272c07ad10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f273c033000 tx=0x7f273c004730 comp rx=0 tx=0).stop 2024-09-16T15:40:01.438 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.435+0000 7f2751400640 1 -- 172.21.15.151:0/412865818 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f274c15f5a0 msgr2=0x7f274c152410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:01.438 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.435+0000 7f2751400640 1 --2- 172.21.15.151:0/412865818 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f274c15f5a0 0x7f274c152410 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f274400ead0 tx=0x7f274400ef90 comp rx=0 tx=0).stop 2024-09-16T15:40:01.438 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.435+0000 7f2751400640 1 -- 172.21.15.151:0/412865818 shutdown_connections 2024-09-16T15:40:01.438 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.435+0000 7f2751400640 1 --2- 172.21.15.151:0/412865818 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f272c078850 0x7f272c07ad10 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:01.438 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.435+0000 7f2751400640 1 --2- 172.21.15.151:0/412865818 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f274c15f5a0 0x7f274c152410 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:01.438 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.435+0000 7f2751400640 1 --2- 172.21.15.151:0/412865818 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f274c0ccf00 0x7f274c146510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:01.438 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.435+0000 7f2751400640 1 --2- 172.21.15.151:0/412865818 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f274c073fb0 0x7f274c150ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:01.438 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.435+0000 7f2751400640 1 -- 172.21.15.151:0/412865818 >> 172.21.15.151:0/412865818 conn(0x7f274c0b1950 msgr2=0x7f274c154030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:01.438 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.435+0000 7f2751400640 1 -- 172.21.15.151:0/412865818 shutdown_connections 2024-09-16T15:40:01.438 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.435+0000 7f2751400640 1 -- 172.21.15.151:0/412865818 wait complete. 2024-09-16T15:40:01.453 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:40:01.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.609+0000 7fe0d2000640 1 -- 172.21.15.151:0/1117442138 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0cc161440 msgr2=0x7fe0cc0b5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:01.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.609+0000 7fe0d2000640 1 --2- 172.21.15.151:0/1117442138 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0cc161440 0x7fe0cc0b5370 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc0b3440 tx=0x7fe0c002a4d0 comp rx=0 tx=0).stop 2024-09-16T15:40:01.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.609+0000 7fe0d2000640 1 -- 172.21.15.151:0/1117442138 shutdown_connections 2024-09-16T15:40:01.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.609+0000 7fe0d2000640 1 --2- 172.21.15.151:0/1117442138 >> v2:172.21.15.167:3300/0 conn(0x7fe0cc0b3e50 0x7fe0cc0d1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:01.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.609+0000 7fe0d2000640 1 --2- 172.21.15.151:0/1117442138 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe0cc0b34b0 0x7fe0cc0b3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:01.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.609+0000 7fe0d2000640 1 --2- 172.21.15.151:0/1117442138 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0cc161440 0x7fe0cc0b5370 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:01.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.610+0000 7fe0d2000640 1 -- 172.21.15.151:0/1117442138 >> 172.21.15.151:0/1117442138 conn(0x7fe0cc0d3df0 msgr2=0x7fe0cc0d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:01.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.610+0000 7fe0d2000640 1 -- 172.21.15.151:0/1117442138 shutdown_connections 2024-09-16T15:40:01.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.610+0000 7fe0d2000640 1 -- 172.21.15.151:0/1117442138 wait complete. 2024-09-16T15:40:01.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.611+0000 7fe0d2000640 1 Processor -- start 2024-09-16T15:40:01.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.611+0000 7fe0d2000640 1 -- start start 2024-09-16T15:40:01.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.612+0000 7fe0d2000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0cc0b34b0 0x7fe0cc1f4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:01.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.612+0000 7fe0d2000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe0cc0b3e50 0x7fe0cc0ca630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:01.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.612+0000 7fe0d2000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0cc161440 0x7fe0cc156220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:01.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.612+0000 7fe0d2000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe0cc0d2420 con 0x7fe0cc161440 2024-09-16T15:40:01.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.612+0000 7fe0d2000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe0cc0d22a0 con 0x7fe0cc0b34b0 2024-09-16T15:40:01.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.612+0000 7fe0d2000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fe0cc0d25a0 con 0x7fe0cc0b3e50 2024-09-16T15:40:01.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.613+0000 7fe0d1600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0cc161440 0x7fe0cc156220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:01.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.613+0000 7fe0d1600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0cc161440 0x7fe0cc156220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43716/0 (socket says 172.21.15.151:43716) 2024-09-16T15:40:01.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.613+0000 7fe0d1600640 1 -- 172.21.15.151:0/797718532 learned_addr learned my addr 172.21.15.151:0/797718532 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:40:01.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.613+0000 7fe0d0c00640 1 --2- 172.21.15.151:0/797718532 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0cc0b34b0 0x7fe0cc1f4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:01.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.613+0000 7fe0cbe00640 1 --2- 172.21.15.151:0/797718532 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe0cc0b3e50 0x7fe0cc0ca630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:01.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.613+0000 7fe0d1600640 1 -- 172.21.15.151:0/797718532 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe0cc0b3e50 msgr2=0x7fe0cc0ca630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:01.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.613+0000 7fe0d1600640 1 --2- 172.21.15.151:0/797718532 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe0cc0b3e50 0x7fe0cc0ca630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:01.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.613+0000 7fe0d1600640 1 -- 172.21.15.151:0/797718532 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0cc0b34b0 msgr2=0x7fe0cc1f4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:01.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.613+0000 7fe0d1600640 1 --2- 172.21.15.151:0/797718532 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0cc0b34b0 0x7fe0cc1f4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:01.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.613+0000 7fe0d1600640 1 -- 172.21.15.151:0/797718532 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0cc0cc160 con 0x7fe0cc161440 2024-09-16T15:40:01.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.613+0000 7fe0d1600640 1 --2- 172.21.15.151:0/797718532 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0cc161440 0x7fe0cc156220 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe0c400eaa0 tx=0x7fe0c400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:01.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.613+0000 7fe0c8c00640 1 -- 172.21.15.151:0/797718532 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c400cdb0 con 0x7fe0cc161440 2024-09-16T15:40:01.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.614+0000 7fe0c8c00640 1 -- 172.21.15.151:0/797718532 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe0c4004c10 con 0x7fe0cc161440 2024-09-16T15:40:01.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.614+0000 7fe0d2000640 1 -- 172.21.15.151:0/797718532 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0cc0cc440 con 0x7fe0cc161440 2024-09-16T15:40:01.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.614+0000 7fe0c8c00640 1 -- 172.21.15.151:0/797718532 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c4005020 con 0x7fe0cc161440 2024-09-16T15:40:01.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.614+0000 7fe0d2000640 1 -- 172.21.15.151:0/797718532 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0cc0cca00 con 0x7fe0cc161440 2024-09-16T15:40:01.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.615+0000 7fe0d2000640 1 -- 172.21.15.151:0/797718532 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0cc0ccc00 con 0x7fe0cc161440 2024-09-16T15:40:01.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.616+0000 7fe0c8c00640 1 -- 172.21.15.151:0/797718532 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fe0c40051c0 con 0x7fe0cc161440 2024-09-16T15:40:01.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.617+0000 7fe0c8c00640 1 --2- 172.21.15.151:0/797718532 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe0ac078850 0x7fe0ac07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:01.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.617+0000 7fe0c8c00640 1 -- 172.21.15.151:0/797718532 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fe0c409b2f0 con 0x7fe0cc161440 2024-09-16T15:40:01.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.618+0000 7fe0d0c00640 1 --2- 172.21.15.151:0/797718532 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe0ac078850 0x7fe0ac07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:01.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.618+0000 7fe0d0c00640 1 --2- 172.21.15.151:0/797718532 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe0ac078850 0x7fe0ac07ad10 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0033000 tx=0x7fe0c0004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:01.624 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.622+0000 7fe0c8c00640 1 -- 172.21.15.151:0/797718532 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe0cc0ccc00 con 0x7fe0cc161440 2024-09-16T15:40:01.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.808+0000 7fe0d2000640 1 -- 172.21.15.151:0/797718532 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe0cc0d2b30 con 0x7fe0cc161440 2024-09-16T15:40:01.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.809+0000 7fe0c8c00640 1 -- 172.21.15.151:0/797718532 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16778 (secure 0 0 0) 0x7fe0c4016030 con 0x7fe0cc161440 2024-09-16T15:40:01.811 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:40:01.812 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":23,"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","created":"2024-09-16T15:38:56.348535+0000","modified":"2024-09-16T15:40:01.704938+0000","last_up_change":"2024-09-16T15:39:10.681384+0000","last_in_change":"2024-09-16T15:39:00.544477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:39:11.602252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:39:19.970182+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T15:39:20.929376+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5524076-a903-4d0a-af6a-caf474c4765d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6809","nonce":3111342741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6811","nonce":3111342741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6815","nonce":3111342741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6813","nonce":3111342741}]},"public_addr":"172.21.15.151:6809/3111342741","cluster_addr":"172.21.15.151:6811/3111342741","heartbeat_back_addr":"172.21.15.151:6815/3111342741","heartbeat_front_addr":"172.21.15.151:6813/3111342741","state":["exists","up"]},{"osd":1,"uuid":"dd25925f-0ae1-494e-8529-4dc205759db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6801","nonce":3368304087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6803","nonce":3368304087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6807","nonce":3368304087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6805","nonce":3368304087}]},"public_addr":"172.21.15.151:6801/3368304087","cluster_addr":"172.21.15.151:6803/3368304087","heartbeat_back_addr":"172.21.15.151:6807/3368304087","heartbeat_front_addr":"172.21.15.151:6805/3368304087","state":["exists","up"]},{"osd":2,"uuid":"c2f18115-013d-4da6-a0fe-cb978f4494b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6823","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6825","nonce":3079373271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6827","nonce":3079373271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6831","nonce":3079373271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6829","nonce":3079373271}]},"public_addr":"172.21.15.151:6825/3079373271","cluster_addr":"172.21.15.151:6827/3079373271","heartbeat_back_addr":"172.21.15.151:6831/3079373271","heartbeat_front_addr":"172.21.15.151:6829/3079373271","state":["exists","up"]},{"osd":3,"uuid":"36edb718-2fcb-4d88-8ddc-c321eed4279d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6817","nonce":1142509041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6819","nonce":1142509041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6824","nonce":1142509041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6821","nonce":1142509041}]},"public_addr":"172.21.15.151:6817/1142509041","cluster_addr":"172.21.15.151:6819/1142509041","heartbeat_back_addr":"172.21.15.151:6824/1142509041","heartbeat_front_addr":"172.21.15.151:6821/1142509041","state":["exists","up"]},{"osd":4,"uuid":"64d33708-cd0a-4236-abe3-1b646e82ab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6811","nonce":2735495271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6813","nonce":2735495271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6817","nonce":2735495271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6815","nonce":2735495271}]},"public_addr":"172.21.15.167:6811/2735495271","cluster_addr":"172.21.15.167:6813/2735495271","heartbeat_back_addr":"172.21.15.167:6817/2735495271","heartbeat_front_addr":"172.21.15.167:6815/2735495271","state":["exists","up"]},{"osd":5,"uuid":"f14b44a9-5910-458c-b518-09a8d0de94de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6827","nonce":1494916572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6829","nonce":1494916572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6833","nonce":1494916572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6831","nonce":1494916572}]},"public_addr":"172.21.15.167:6827/1494916572","cluster_addr":"172.21.15.167:6829/1494916572","heartbeat_back_addr":"172.21.15.167:6833/1494916572","heartbeat_front_addr":"172.21.15.167:6831/1494916572","state":["exists","up"]},{"osd":6,"uuid":"39335545-2654-4710-84b0-a9aa47cfcbba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6819","nonce":2538273273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6821","nonce":2538273273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6825","nonce":2538273273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6823","nonce":2538273273}]},"public_addr":"172.21.15.167:6819/2538273273","cluster_addr":"172.21.15.167:6821/2538273273","heartbeat_back_addr":"172.21.15.167:6825/2538273273","heartbeat_front_addr":"172.21.15.167:6823/2538273273","state":["exists","up"]},{"osd":7,"uuid":"4f8bfc81-53dd-4339-ac22-1a4007d9f963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6803","nonce":265072227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6805","nonce":265072227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6809","nonce":265072227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6807","nonce":265072227}]},"public_addr":"172.21.15.167:6803/265072227","cluster_addr":"172.21.15.167:6805/265072227","heartbeat_back_addr":"172.21.15.167:6809/265072227","heartbeat_front_addr":"172.21.15.167:6807/265072227","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.150359+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.084358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.722368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.332809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.320709+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.395153+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.935250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.801443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.35","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:40:01.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.811+0000 7fe0d2000640 1 -- 172.21.15.151:0/797718532 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe0ac078850 msgr2=0x7fe0ac07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:01.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.811+0000 7fe0d2000640 1 --2- 172.21.15.151:0/797718532 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe0ac078850 0x7fe0ac07ad10 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0033000 tx=0x7fe0c0004730 comp rx=0 tx=0).stop 2024-09-16T15:40:01.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.811+0000 7fe0d2000640 1 -- 172.21.15.151:0/797718532 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0cc161440 msgr2=0x7fe0cc156220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:01.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.811+0000 7fe0d2000640 1 --2- 172.21.15.151:0/797718532 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0cc161440 0x7fe0cc156220 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe0c400eaa0 tx=0x7fe0c400ef60 comp rx=0 tx=0).stop 2024-09-16T15:40:01.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.811+0000 7fe0d2000640 1 -- 172.21.15.151:0/797718532 shutdown_connections 2024-09-16T15:40:01.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.811+0000 7fe0d2000640 1 --2- 172.21.15.151:0/797718532 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe0ac078850 0x7fe0ac07ad10 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:01.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.811+0000 7fe0d2000640 1 --2- 172.21.15.151:0/797718532 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0cc161440 0x7fe0cc156220 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:01.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.811+0000 7fe0d2000640 1 --2- 172.21.15.151:0/797718532 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe0cc0b3e50 0x7fe0cc0ca630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:01.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.811+0000 7fe0d2000640 1 --2- 172.21.15.151:0/797718532 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0cc0b34b0 0x7fe0cc1f4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:01.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.811+0000 7fe0d2000640 1 -- 172.21.15.151:0/797718532 >> 172.21.15.151:0/797718532 conn(0x7fe0cc0d3df0 msgr2=0x7fe0cc15fd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:01.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.812+0000 7fe0d2000640 1 -- 172.21.15.151:0/797718532 shutdown_connections 2024-09-16T15:40:01.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.812+0000 7fe0d2000640 1 -- 172.21.15.151:0/797718532 wait complete. 2024-09-16T15:40:01.830 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-16T15:40:01.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.987+0000 7f2da3200640 1 -- 172.21.15.151:0/726005121 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2da40b34b0 msgr2=0x7f2da40b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:01.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.987+0000 7f2da3200640 1 --2- 172.21.15.151:0/726005121 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2da40b34b0 0x7f2da40b3940 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2da40b6930 tx=0x7f2d9002a4d0 comp rx=0 tx=0).stop 2024-09-16T15:40:01.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.987+0000 7f2da3200640 1 -- 172.21.15.151:0/726005121 shutdown_connections 2024-09-16T15:40:01.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.987+0000 7f2da3200640 1 --2- 172.21.15.151:0/726005121 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2da40b3e80 0x7f2da40d1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:01.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.987+0000 7f2da3200640 1 --2- 172.21.15.151:0/726005121 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2da40b34b0 0x7f2da40b3940 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:01.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.987+0000 7f2da3200640 1 --2- 172.21.15.151:0/726005121 >> v2:172.21.15.167:3300/0 conn(0x7f2da4161440 0x7f2da40b5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:01.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.987+0000 7f2da3200640 1 -- 172.21.15.151:0/726005121 >> 172.21.15.151:0/726005121 conn(0x7f2da40d3df0 msgr2=0x7f2da40d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:01.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.987+0000 7f2da3200640 1 -- 172.21.15.151:0/726005121 shutdown_connections 2024-09-16T15:40:01.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.987+0000 7f2da3200640 1 -- 172.21.15.151:0/726005121 wait complete. 2024-09-16T15:40:01.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.988+0000 7f2da3200640 1 Processor -- start 2024-09-16T15:40:01.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.989+0000 7f2da3200640 1 -- start start 2024-09-16T15:40:01.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.989+0000 7f2da3200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2da40b34b0 0x7f2da41f8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:01.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.990+0000 7f2da3200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2da40b3e80 0x7f2da4157250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:01.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.990+0000 7f2da3200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2da4161440 0x7f2da41f5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:01.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.990+0000 7f2da3200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2da40d2420 con 0x7f2da40b34b0 2024-09-16T15:40:01.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.990+0000 7f2da3200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2da40d22a0 con 0x7f2da40b3e80 2024-09-16T15:40:01.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.990+0000 7f2da3200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f2da40d25a0 con 0x7f2da4161440 2024-09-16T15:40:01.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.990+0000 7f2da1e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2da40b34b0 0x7f2da41f8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:01.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.990+0000 7f2da1e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2da40b34b0 0x7f2da41f8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43726/0 (socket says 172.21.15.151:43726) 2024-09-16T15:40:01.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.990+0000 7f2da1e00640 1 -- 172.21.15.151:0/1316071969 learned_addr learned my addr 172.21.15.151:0/1316071969 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:40:01.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.990+0000 7f2da1400640 1 --2- 172.21.15.151:0/1316071969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2da40b3e80 0x7f2da4157250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:01.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.990+0000 7f2da2800640 1 --2- 172.21.15.151:0/1316071969 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2da4161440 0x7f2da41f5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:01.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.990+0000 7f2da1e00640 1 -- 172.21.15.151:0/1316071969 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2da4161440 msgr2=0x7f2da41f5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:01.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.990+0000 7f2da1e00640 1 --2- 172.21.15.151:0/1316071969 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2da4161440 0x7f2da41f5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:01.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.990+0000 7f2da1e00640 1 -- 172.21.15.151:0/1316071969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2da40b3e80 msgr2=0x7f2da4157250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:01.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.990+0000 7f2da1e00640 1 --2- 172.21.15.151:0/1316071969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2da40b3e80 0x7f2da4157250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:01.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.990+0000 7f2da1e00640 1 -- 172.21.15.151:0/1316071969 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2da41f8580 con 0x7f2da40b34b0 2024-09-16T15:40:01.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.991+0000 7f2da1e00640 1 --2- 172.21.15.151:0/1316071969 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2da40b34b0 0x7f2da41f8040 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2d9400d780 tx=0x7f2d9400dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:01.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.991+0000 7f2d9a000640 1 -- 172.21.15.151:0/1316071969 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d94017070 con 0x7f2da40b34b0 2024-09-16T15:40:01.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.991+0000 7f2d9a000640 1 -- 172.21.15.151:0/1316071969 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2d9400de90 con 0x7f2da40b34b0 2024-09-16T15:40:01.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.991+0000 7f2d9a000640 1 -- 172.21.15.151:0/1316071969 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d940099d0 con 0x7f2da40b34b0 2024-09-16T15:40:01.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.991+0000 7f2da3200640 1 -- 172.21.15.151:0/1316071969 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2da41f8850 con 0x7f2da40b34b0 2024-09-16T15:40:01.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.991+0000 7f2da3200640 1 -- 172.21.15.151:0/1316071969 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2da41f4b60 con 0x7f2da40b34b0 2024-09-16T15:40:01.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.992+0000 7f2da3200640 1 -- 172.21.15.151:0/1316071969 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2da40b2f60 con 0x7f2da40b34b0 2024-09-16T15:40:01.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.993+0000 7f2d9a000640 1 -- 172.21.15.151:0/1316071969 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f2d94009b70 con 0x7f2da40b34b0 2024-09-16T15:40:01.996 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.994+0000 7f2d9a000640 1 --2- 172.21.15.151:0/1316071969 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2d84078850 0x7f2d8407ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:01.996 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.994+0000 7f2d9a000640 1 -- 172.21.15.151:0/1316071969 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5738+0+0 (secure 0 0 0) 0x7f2d9409ab50 con 0x7f2da40b34b0 2024-09-16T15:40:01.996 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.995+0000 7f2da1400640 1 --2- 172.21.15.151:0/1316071969 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2d84078850 0x7f2d8407ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:01.997 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.995+0000 7f2da1400640 1 --2- 172.21.15.151:0/1316071969 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2d84078850 0x7f2d8407ad10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2d90033000 tx=0x7f2d90004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:02.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:01.998+0000 7f2d9a000640 1 -- 172.21.15.151:0/1316071969 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2da40b2f60 con 0x7f2da40b34b0 2024-09-16T15:40:02.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:02.222+0000 7f2da3200640 1 -- 172.21.15.151:0/1316071969 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f2da40d2aa0 con 0x7f2da40b34b0 2024-09-16T15:40:02.225 INFO:tasks.ceph.mon.a.smithi151.stderr:2024-09-16T15:40:02.223+0000 7f62c3200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T15:40:02.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:02.225+0000 7f2d9a000640 1 -- 172.21.15.151:0/1316071969 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) ==== 161+0+0 (secure 0 0 0) 0x7f2d94020020 con 0x7f2da40b34b0 2024-09-16T15:40:02.227 INFO:teuthology.orchestra.run.smithi151.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-16T15:40:02.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:02.229+0000 7f2da3200640 1 -- 172.21.15.151:0/1316071969 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2d84078850 msgr2=0x7f2d8407ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:02.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:02.229+0000 7f2da3200640 1 --2- 172.21.15.151:0/1316071969 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2d84078850 0x7f2d8407ad10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2d90033000 tx=0x7f2d90004730 comp rx=0 tx=0).stop 2024-09-16T15:40:02.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:02.229+0000 7f2da3200640 1 -- 172.21.15.151:0/1316071969 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2da40b34b0 msgr2=0x7f2da41f8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:02.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:02.229+0000 7f2da3200640 1 --2- 172.21.15.151:0/1316071969 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2da40b34b0 0x7f2da41f8040 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2d9400d780 tx=0x7f2d9400dc40 comp rx=0 tx=0).stop 2024-09-16T15:40:02.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:02.229+0000 7f2da3200640 1 -- 172.21.15.151:0/1316071969 shutdown_connections 2024-09-16T15:40:02.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:02.229+0000 7f2da3200640 1 --2- 172.21.15.151:0/1316071969 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2d84078850 0x7f2d8407ad10 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:02.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:02.229+0000 7f2da3200640 1 --2- 172.21.15.151:0/1316071969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2da40b3e80 0x7f2da4157250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:02.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:02.229+0000 7f2da3200640 1 --2- 172.21.15.151:0/1316071969 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2da4161440 0x7f2da41f5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:02.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:02.229+0000 7f2da3200640 1 --2- 172.21.15.151:0/1316071969 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2da40b34b0 0x7f2da41f8040 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:02.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:02.229+0000 7f2da3200640 1 -- 172.21.15.151:0/1316071969 >> 172.21.15.151:0/1316071969 conn(0x7f2da40d3df0 msgr2=0x7f2da415bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:02.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:02.229+0000 7f2da3200640 1 -- 172.21.15.151:0/1316071969 shutdown_connections 2024-09-16T15:40:02.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:02.229+0000 7f2da3200640 1 -- 172.21.15.151:0/1316071969 wait complete. 2024-09-16T15:40:02.242 INFO:tasks.ceph.mds.d.smithi167.stderr: -7277> 2024-09-16T15:39:27.523+0000 7f533f800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-16T15:40:02.248 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-16T15:40:02.297 INFO:tasks.ceph.mds.d.smithi167.stdout:starting mds.d at 2024-09-16T15:40:02.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:02.407+0000 7f5c82200640 1 -- 172.21.15.151:0/1998714078 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5c7c15c080 msgr2=0x7f5c7c0cf120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:02.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:02.407+0000 7f5c82200640 1 --2- 172.21.15.151:0/1998714078 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5c7c15c080 0x7f5c7c0cf120 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5c68005fd0 tx=0x7f5c6802a5d0 comp rx=0 tx=0).stop 2024-09-16T15:40:02.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:02.407+0000 7f5c82200640 1 -- 172.21.15.151:0/1998714078 shutdown_connections 2024-09-16T15:40:02.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:02.407+0000 7f5c82200640 1 --2- 172.21.15.151:0/1998714078 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5c7c15c080 0x7f5c7c0cf120 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:02.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:02.407+0000 7f5c82200640 1 --2- 172.21.15.151:0/1998714078 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5c7c0b3370 0x7f5c7c0cebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:02.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:02.407+0000 7f5c82200640 1 -- 172.21.15.151:0/1998714078 >> 172.21.15.151:0/1998714078 conn(0x7f5c7c0b1840 msgr2=0x7f5c7c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:02.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:02.407+0000 7f5c82200640 1 -- 172.21.15.151:0/1998714078 shutdown_connections 2024-09-16T15:40:02.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:02.407+0000 7f5c82200640 1 -- 172.21.15.151:0/1998714078 wait complete. 2024-09-16T15:40:02.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:02.409+0000 7f5c82200640 1 Processor -- start 2024-09-16T15:40:02.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:02.409+0000 7f5c82200640 1 -- start start 2024-09-16T15:40:02.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:02.410+0000 7f5c82200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5c7c0b3370 0x7f5c7c13b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:02.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:02.410+0000 7f5c82200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5c7c15c080 0x7f5c7c1f6b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:02.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:02.410+0000 7f5c82200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5c7c15e980 0x7f5c7c15cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:02.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:02.410+0000 7f5c82200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5c7c164e20 con 0x7f5c7c15e980 2024-09-16T15:40:02.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:02.410+0000 7f5c82200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5c7c164ca0 con 0x7f5c7c0b3370 2024-09-16T15:40:02.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:02.410+0000 7f5c82200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f5c7c164fa0 con 0x7f5c7c15c080 2024-09-16T15:40:02.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:02.410+0000 7f5c7be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5c7c15e980 0x7f5c7c15cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:02.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:02.410+0000 7f5c7be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5c7c15e980 0x7f5c7c15cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43752/0 (socket says 172.21.15.151:43752) 2024-09-16T15:40:02.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:02.410+0000 7f5c7be00640 1 -- 172.21.15.151:0/701981388 learned_addr learned my addr 172.21.15.151:0/701981388 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:40:02.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:02.410+0000 7f5c7aa00640 1 --2- 172.21.15.151:0/701981388 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5c7c15c080 0x7f5c7c1f6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:02.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:02.410+0000 7f5c7b400640 1 --2- 172.21.15.151:0/701981388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5c7c0b3370 0x7f5c7c13b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:02.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:02.411+0000 7f5c7be00640 1 -- 172.21.15.151:0/701981388 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5c7c15c080 msgr2=0x7f5c7c1f6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:02.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:02.411+0000 7f5c7be00640 1 --2- 172.21.15.151:0/701981388 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5c7c15c080 0x7f5c7c1f6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:02.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:02.411+0000 7f5c7be00640 1 -- 172.21.15.151:0/701981388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5c7c0b3370 msgr2=0x7f5c7c13b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:02.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:02.411+0000 7f5c7be00640 1 --2- 172.21.15.151:0/701981388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5c7c0b3370 0x7f5c7c13b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:02.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:02.411+0000 7f5c7be00640 1 -- 172.21.15.151:0/701981388 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c7c13bbe0 con 0x7f5c7c15e980 2024-09-16T15:40:02.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:02.411+0000 7f5c7aa00640 1 --2- 172.21.15.151:0/701981388 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5c7c15c080 0x7f5c7c1f6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:40:02.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:02.411+0000 7f5c7b400640 1 --2- 172.21.15.151:0/701981388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5c7c0b3370 0x7f5c7c13b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:40:02.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:02.411+0000 7f5c7be00640 1 --2- 172.21.15.151:0/701981388 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5c7c15e980 0x7f5c7c15cdc0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5c7400c3c0 tx=0x7f5c7400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:02.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:02.411+0000 7f5c73400640 1 -- 172.21.15.151:0/701981388 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c74017070 con 0x7f5c7c15e980 2024-09-16T15:40:02.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:02.411+0000 7f5c73400640 1 -- 172.21.15.151:0/701981388 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5c740053c0 con 0x7f5c7c15e980 2024-09-16T15:40:02.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:02.411+0000 7f5c73400640 1 -- 172.21.15.151:0/701981388 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c74003850 con 0x7f5c7c15e980 2024-09-16T15:40:02.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:02.411+0000 7f5c82200640 1 -- 172.21.15.151:0/701981388 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c7c1f2e60 con 0x7f5c7c15e980 2024-09-16T15:40:02.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:02.411+0000 7f5c82200640 1 -- 172.21.15.151:0/701981388 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c7c1f2b00 con 0x7f5c7c15e980 2024-09-16T15:40:02.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:02.412+0000 7f5c82200640 1 -- 172.21.15.151:0/701981388 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c7c0b3f70 con 0x7f5c7c15e980 2024-09-16T15:40:02.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:02.413+0000 7f5c73400640 1 -- 172.21.15.151:0/701981388 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f5c7401a070 con 0x7f5c7c15e980 2024-09-16T15:40:02.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:02.414+0000 7f5c73400640 1 --2- 172.21.15.151:0/701981388 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5c5c078850 0x7f5c5c07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:02.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:02.414+0000 7f5c73400640 1 -- 172.21.15.151:0/701981388 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5824+0+0 (secure 0 0 0) 0x7f5c74099c90 con 0x7f5c7c15e980 2024-09-16T15:40:02.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:02.415+0000 7f5c7b400640 1 --2- 172.21.15.151:0/701981388 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5c5c078850 0x7f5c5c07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:02.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:02.415+0000 7f5c7b400640 1 --2- 172.21.15.151:0/701981388 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5c5c078850 0x7f5c5c07ad10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c005fd0 tx=0x7f5c6c0093e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:02.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:02.418+0000 7f5c73400640 1 -- 172.21.15.151:0/701981388 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f5c7c0b3f70 con 0x7f5c7c15e980 2024-09-16T15:40:02.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:02.645+0000 7f5c82200640 1 -- 172.21.15.151:0/701981388 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f5c7c15ee10 con 0x7f5c7c15e980 2024-09-16T15:40:03.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:03.227+0000 7f5c73400640 1 -- 172.21.15.151:0/701981388 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) ==== 104+0+0 (secure 0 0 0) 0x7f5c7c15b320 con 0x7f5c7c15e980 2024-09-16T15:40:03.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:03.229+0000 7f5c82200640 1 -- 172.21.15.151:0/701981388 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5c5c078850 msgr2=0x7f5c5c07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:03.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:03.229+0000 7f5c82200640 1 --2- 172.21.15.151:0/701981388 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5c5c078850 0x7f5c5c07ad10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c005fd0 tx=0x7f5c6c0093e0 comp rx=0 tx=0).stop 2024-09-16T15:40:03.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:03.229+0000 7f5c82200640 1 -- 172.21.15.151:0/701981388 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5c7c15e980 msgr2=0x7f5c7c15cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:03.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:03.229+0000 7f5c82200640 1 --2- 172.21.15.151:0/701981388 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5c7c15e980 0x7f5c7c15cdc0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5c7400c3c0 tx=0x7f5c7400c880 comp rx=0 tx=0).stop 2024-09-16T15:40:03.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:03.229+0000 7f5c82200640 1 -- 172.21.15.151:0/701981388 shutdown_connections 2024-09-16T15:40:03.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:03.229+0000 7f5c82200640 1 --2- 172.21.15.151:0/701981388 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5c5c078850 0x7f5c5c07ad10 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:03.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:03.229+0000 7f5c82200640 1 --2- 172.21.15.151:0/701981388 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5c7c15e980 0x7f5c7c15cdc0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:03.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:03.229+0000 7f5c82200640 1 --2- 172.21.15.151:0/701981388 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5c7c15c080 0x7f5c7c1f6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:03.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:03.229+0000 7f5c82200640 1 --2- 172.21.15.151:0/701981388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5c7c0b3370 0x7f5c7c13b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:03.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:03.229+0000 7f5c82200640 1 -- 172.21.15.151:0/701981388 >> 172.21.15.151:0/701981388 conn(0x7f5c7c0b1840 msgr2=0x7f5c7c13d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:03.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:03.230+0000 7f5c82200640 1 -- 172.21.15.151:0/701981388 shutdown_connections 2024-09-16T15:40:03.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:03.230+0000 7f5c82200640 1 -- 172.21.15.151:0/701981388 wait complete. 2024-09-16T15:40:03.249 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-16T15:40:03.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:03.408+0000 7fe9b8c00640 1 -- 172.21.15.151:0/3266611289 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9b4159aa0 msgr2=0x7fe9b40741a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:03.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:03.408+0000 7fe9b8c00640 1 --2- 172.21.15.151:0/3266611289 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9b4159aa0 0x7fe9b40741a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe9a4009830 tx=0x7fe9a402a520 comp rx=0 tx=0).stop 2024-09-16T15:40:03.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:03.408+0000 7fe9b8c00640 1 -- 172.21.15.151:0/3266611289 shutdown_connections 2024-09-16T15:40:03.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:03.408+0000 7fe9b8c00640 1 --2- 172.21.15.151:0/3266611289 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe9b406f220 0x7fe9b406f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:03.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:03.408+0000 7fe9b8c00640 1 --2- 172.21.15.151:0/3266611289 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9b4159aa0 0x7fe9b40741a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:03.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:03.408+0000 7fe9b8c00640 1 -- 172.21.15.151:0/3266611289 >> 172.21.15.151:0/3266611289 conn(0x7fe9b40b18e0 msgr2=0x7fe9b40b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:03.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:03.409+0000 7fe9b8c00640 1 -- 172.21.15.151:0/3266611289 shutdown_connections 2024-09-16T15:40:03.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:03.409+0000 7fe9b8c00640 1 -- 172.21.15.151:0/3266611289 wait complete. 2024-09-16T15:40:03.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:03.410+0000 7fe9b8c00640 1 Processor -- start 2024-09-16T15:40:03.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:03.410+0000 7fe9b8c00640 1 -- start start 2024-09-16T15:40:03.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:03.411+0000 7fe9b8c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9b406f220 0x7fe9b40c9c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:03.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:03.411+0000 7fe9b8c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9b4159aa0 0x7fe9b41f7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:03.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:03.411+0000 7fe9b8c00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe9b415cff0 0x7fe9b40cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:03.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:03.411+0000 7fe9b8c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b40cc520 con 0x7fe9b4159aa0 2024-09-16T15:40:03.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:03.411+0000 7fe9b8c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b40cc3a0 con 0x7fe9b406f220 2024-09-16T15:40:03.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:03.411+0000 7fe9b8c00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fe9b40cc6a0 con 0x7fe9b415cff0 2024-09-16T15:40:03.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:03.411+0000 7fe9abe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9b4159aa0 0x7fe9b41f7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:03.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:03.411+0000 7fe9abe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9b4159aa0 0x7fe9b41f7d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:38412/0 (socket says 172.21.15.151:38412) 2024-09-16T15:40:03.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:03.411+0000 7fe9abe00640 1 -- 172.21.15.151:0/4294037775 learned_addr learned my addr 172.21.15.151:0/4294037775 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:40:03.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:03.412+0000 7fe9b2800640 1 --2- 172.21.15.151:0/4294037775 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe9b415cff0 0x7fe9b40cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:03.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:03.412+0000 7fe9b1e00640 1 --2- 172.21.15.151:0/4294037775 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9b406f220 0x7fe9b40c9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:03.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:03.412+0000 7fe9abe00640 1 -- 172.21.15.151:0/4294037775 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe9b415cff0 msgr2=0x7fe9b40cdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:03.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:03.412+0000 7fe9abe00640 1 --2- 172.21.15.151:0/4294037775 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe9b415cff0 0x7fe9b40cdad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:03.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:03.412+0000 7fe9abe00640 1 -- 172.21.15.151:0/4294037775 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9b406f220 msgr2=0x7fe9b40c9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:03.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:03.412+0000 7fe9abe00640 1 --2- 172.21.15.151:0/4294037775 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9b406f220 0x7fe9b40c9c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:03.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:03.412+0000 7fe9abe00640 1 -- 172.21.15.151:0/4294037775 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9b40d04c0 con 0x7fe9b4159aa0 2024-09-16T15:40:03.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:03.412+0000 7fe9b2800640 1 --2- 172.21.15.151:0/4294037775 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe9b415cff0 0x7fe9b40cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:40:03.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:03.412+0000 7fe9b1e00640 1 --2- 172.21.15.151:0/4294037775 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9b406f220 0x7fe9b40c9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:40:03.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:03.412+0000 7fe9abe00640 1 --2- 172.21.15.151:0/4294037775 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9b4159aa0 0x7fe9b41f7d80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe99c00c3a0 tx=0x7fe99c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:03.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:03.412+0000 7fe9aa000640 1 -- 172.21.15.151:0/4294037775 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99c011790 con 0x7fe9b4159aa0 2024-09-16T15:40:03.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:03.412+0000 7fe9aa000640 1 -- 172.21.15.151:0/4294037775 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe99c017070 con 0x7fe9b4159aa0 2024-09-16T15:40:03.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:03.412+0000 7fe9b8c00640 1 -- 172.21.15.151:0/4294037775 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9b40d07a0 con 0x7fe9b4159aa0 2024-09-16T15:40:03.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:03.412+0000 7fe9aa000640 1 -- 172.21.15.151:0/4294037775 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99c011cc0 con 0x7fe9b4159aa0 2024-09-16T15:40:03.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:03.413+0000 7fe9b8c00640 1 -- 172.21.15.151:0/4294037775 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9b40d0150 con 0x7fe9b4159aa0 2024-09-16T15:40:03.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:03.414+0000 7fe9b8c00640 1 -- 172.21.15.151:0/4294037775 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9b4069360 con 0x7fe9b4159aa0 2024-09-16T15:40:03.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:03.415+0000 7fe9aa000640 1 -- 172.21.15.151:0/4294037775 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fe99c01c460 con 0x7fe9b4159aa0 2024-09-16T15:40:03.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:03.416+0000 7fe9aa000640 1 --2- 172.21.15.151:0/4294037775 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe990078850 0x7fe99007ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:03.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:03.416+0000 7fe9aa000640 1 -- 172.21.15.151:0/4294037775 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5824+0+0 (secure 0 0 0) 0x7fe99c09a710 con 0x7fe9b4159aa0 2024-09-16T15:40:03.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:03.416+0000 7fe9b1e00640 1 --2- 172.21.15.151:0/4294037775 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe990078850 0x7fe99007ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:03.419 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:03.417+0000 7fe9b1e00640 1 --2- 172.21.15.151:0/4294037775 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe990078850 0x7fe99007ad10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe9a4034000 tx=0x7fe9a4004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:03.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:03.420+0000 7fe9aa000640 1 -- 172.21.15.151:0/4294037775 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe99c062d10 con 0x7fe9b4159aa0 2024-09-16T15:40:03.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:03.604+0000 7fe9b8c00640 1 -- 172.21.15.151:0/4294037775 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe9b40ccb70 con 0x7fe9b4159aa0 2024-09-16T15:40:04.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:04.240+0000 7fe9aa000640 1 -- 172.21.15.151:0/4294037775 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v26) ==== 181+0+0 (secure 0 0 0) 0x7fe99c062610 con 0x7fe9b4159aa0 2024-09-16T15:40:04.243 INFO:teuthology.orchestra.run.smithi151.stderr:pool 'cephfs_metadata' removed 2024-09-16T15:40:04.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:04.243+0000 7fe9b8c00640 1 -- 172.21.15.151:0/4294037775 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe990078850 msgr2=0x7fe99007ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:04.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:04.243+0000 7fe9b8c00640 1 --2- 172.21.15.151:0/4294037775 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe990078850 0x7fe99007ad10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe9a4034000 tx=0x7fe9a4004730 comp rx=0 tx=0).stop 2024-09-16T15:40:04.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:04.243+0000 7fe9b8c00640 1 -- 172.21.15.151:0/4294037775 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9b4159aa0 msgr2=0x7fe9b41f7d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:04.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:04.243+0000 7fe9b8c00640 1 --2- 172.21.15.151:0/4294037775 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9b4159aa0 0x7fe9b41f7d80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe99c00c3a0 tx=0x7fe99c00c860 comp rx=0 tx=0).stop 2024-09-16T15:40:04.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:04.243+0000 7fe9b8c00640 1 -- 172.21.15.151:0/4294037775 shutdown_connections 2024-09-16T15:40:04.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:04.243+0000 7fe9b8c00640 1 --2- 172.21.15.151:0/4294037775 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe990078850 0x7fe99007ad10 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:04.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:04.243+0000 7fe9b8c00640 1 --2- 172.21.15.151:0/4294037775 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe9b415cff0 0x7fe9b40cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:04.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:04.243+0000 7fe9b8c00640 1 --2- 172.21.15.151:0/4294037775 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9b4159aa0 0x7fe9b41f7d80 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:04.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:04.243+0000 7fe9b8c00640 1 --2- 172.21.15.151:0/4294037775 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9b406f220 0x7fe9b40c9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:04.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:04.243+0000 7fe9b8c00640 1 -- 172.21.15.151:0/4294037775 >> 172.21.15.151:0/4294037775 conn(0x7fe9b40b18e0 msgr2=0x7fe9b40ccd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:04.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:04.243+0000 7fe9b8c00640 1 -- 172.21.15.151:0/4294037775 shutdown_connections 2024-09-16T15:40:04.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:04.243+0000 7fe9b8c00640 1 -- 172.21.15.151:0/4294037775 wait complete. 2024-09-16T15:40:04.265 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-16T15:40:04.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:04.426+0000 7f6086600640 1 -- 172.21.15.151:0/1638840026 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6080161480 msgr2=0x7f6080070e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:04.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:04.426+0000 7f6086600640 1 --2- 172.21.15.151:0/1638840026 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6080161480 0x7f6080070e70 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6070009830 tx=0x7f607002a3f0 comp rx=0 tx=0).stop 2024-09-16T15:40:04.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:04.426+0000 7f6086600640 1 -- 172.21.15.151:0/1638840026 shutdown_connections 2024-09-16T15:40:04.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:04.426+0000 7f6086600640 1 --2- 172.21.15.151:0/1638840026 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6080161860 0x7f60800713b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:04.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:04.426+0000 7f6086600640 1 --2- 172.21.15.151:0/1638840026 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6080161480 0x7f6080070e70 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:04.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:04.426+0000 7f6086600640 1 -- 172.21.15.151:0/1638840026 >> 172.21.15.151:0/1638840026 conn(0x7f60800d1cd0 msgr2=0x7f60800d20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:04.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:04.426+0000 7f6086600640 1 -- 172.21.15.151:0/1638840026 shutdown_connections 2024-09-16T15:40:04.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:04.426+0000 7f6086600640 1 -- 172.21.15.151:0/1638840026 wait complete. 2024-09-16T15:40:04.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:04.427+0000 7f6086600640 1 Processor -- start 2024-09-16T15:40:04.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:04.428+0000 7f6086600640 1 -- start start 2024-09-16T15:40:04.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:04.429+0000 7f6086600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f608006f390 0x7f6080156e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:04.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:04.429+0000 7f6086600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6080161480 0x7f60800b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:04.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:04.429+0000 7f6086600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6080161860 0x7f60800b7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:04.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:04.429+0000 7f6086600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f608006f960 con 0x7f6080161480 2024-09-16T15:40:04.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:04.429+0000 7f6086600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f60800d12c0 con 0x7f608006f390 2024-09-16T15:40:04.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:04.429+0000 7f6086600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f608006fae0 con 0x7f6080161860 2024-09-16T15:40:04.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:04.429+0000 7f6077e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6080161480 0x7f60800b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:04.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:04.429+0000 7f6077e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6080161480 0x7f60800b4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:38424/0 (socket says 172.21.15.151:38424) 2024-09-16T15:40:04.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:04.429+0000 7f6077e00640 1 -- 172.21.15.151:0/1154845419 learned_addr learned my addr 172.21.15.151:0/1154845419 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:40:04.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:04.429+0000 7f6085200640 1 --2- 172.21.15.151:0/1154845419 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f608006f390 0x7f6080156e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:04.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:04.429+0000 7f6085c00640 1 --2- 172.21.15.151:0/1154845419 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6080161860 0x7f60800b7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:04.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:04.429+0000 7f6077e00640 1 -- 172.21.15.151:0/1154845419 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6080161860 msgr2=0x7f60800b7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:04.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:04.429+0000 7f6077e00640 1 --2- 172.21.15.151:0/1154845419 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6080161860 0x7f60800b7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:04.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:04.429+0000 7f6077e00640 1 -- 172.21.15.151:0/1154845419 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f608006f390 msgr2=0x7f6080156e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:04.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:04.429+0000 7f6077e00640 1 --2- 172.21.15.151:0/1154845419 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f608006f390 0x7f6080156e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:04.433 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:04.429+0000 7f6077e00640 1 -- 172.21.15.151:0/1154845419 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60800cba30 con 0x7f6080161480 2024-09-16T15:40:04.433 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:04.430+0000 7f6077e00640 1 --2- 172.21.15.151:0/1154845419 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6080161480 0x7f60800b4b00 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f606800c3c0 tx=0x7f606800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:04.433 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:04.430+0000 7f607d600640 1 -- 172.21.15.151:0/1154845419 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6068017070 con 0x7f6080161480 2024-09-16T15:40:04.433 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:04.430+0000 7f607d600640 1 -- 172.21.15.151:0/1154845419 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f60680035f0 con 0x7f6080161480 2024-09-16T15:40:04.433 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:04.430+0000 7f607d600640 1 -- 172.21.15.151:0/1154845419 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6068003a80 con 0x7f6080161480 2024-09-16T15:40:04.434 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:04.430+0000 7f6086600640 1 -- 172.21.15.151:0/1154845419 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60800cb470 con 0x7f6080161480 2024-09-16T15:40:04.434 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:04.430+0000 7f6086600640 1 -- 172.21.15.151:0/1154845419 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60800c8f40 con 0x7f6080161480 2024-09-16T15:40:04.434 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:04.431+0000 7f6086600640 1 -- 172.21.15.151:0/1154845419 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f608006dc80 con 0x7f6080161480 2024-09-16T15:40:04.434 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:04.432+0000 7f607d600640 1 -- 172.21.15.151:0/1154845419 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f606801a070 con 0x7f6080161480 2024-09-16T15:40:04.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:04.433+0000 7f607d600640 1 --2- 172.21.15.151:0/1154845419 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f605c078880 0x7f605c07ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:04.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:04.433+0000 7f607d600640 1 -- 172.21.15.151:0/1154845419 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 5386+0+0 (secure 0 0 0) 0x7f6068099b20 con 0x7f6080161480 2024-09-16T15:40:04.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:04.434+0000 7f6085200640 1 --2- 172.21.15.151:0/1154845419 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f605c078880 0x7f605c07ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:04.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:04.434+0000 7f6085200640 1 --2- 172.21.15.151:0/1154845419 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f605c078880 0x7f605c07ad40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6070034000 tx=0x7f6070004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:04.439 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:04.437+0000 7f607d600640 1 -- 172.21.15.151:0/1154845419 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f60800cba30 con 0x7f6080161480 2024-09-16T15:40:04.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:04.623+0000 7f6086600640 1 -- 172.21.15.151:0/1154845419 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f60800aec00 con 0x7f6080161480 2024-09-16T15:40:05.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:05.247+0000 7f607d600640 1 -- 172.21.15.151:0/1154845419 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v27) ==== 169+0+0 (secure 0 0 0) 0x7f6068062230 con 0x7f6080161480 2024-09-16T15:40:05.249 INFO:teuthology.orchestra.run.smithi151.stderr:pool 'cephfs_data' removed 2024-09-16T15:40:05.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:05.251+0000 7f6086600640 1 -- 172.21.15.151:0/1154845419 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f605c078880 msgr2=0x7f605c07ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:05.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:05.251+0000 7f6086600640 1 --2- 172.21.15.151:0/1154845419 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f605c078880 0x7f605c07ad40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6070034000 tx=0x7f6070004730 comp rx=0 tx=0).stop 2024-09-16T15:40:05.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:05.251+0000 7f6086600640 1 -- 172.21.15.151:0/1154845419 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6080161480 msgr2=0x7f60800b4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:05.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:05.251+0000 7f6086600640 1 --2- 172.21.15.151:0/1154845419 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6080161480 0x7f60800b4b00 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f606800c3c0 tx=0x7f606800c880 comp rx=0 tx=0).stop 2024-09-16T15:40:05.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:05.252+0000 7f6086600640 1 -- 172.21.15.151:0/1154845419 shutdown_connections 2024-09-16T15:40:05.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:05.252+0000 7f6086600640 1 --2- 172.21.15.151:0/1154845419 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f605c078880 0x7f605c07ad40 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:05.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:05.252+0000 7f6086600640 1 --2- 172.21.15.151:0/1154845419 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6080161860 0x7f60800b7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:05.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:05.252+0000 7f6086600640 1 --2- 172.21.15.151:0/1154845419 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6080161480 0x7f60800b4b00 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:05.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:05.252+0000 7f6086600640 1 --2- 172.21.15.151:0/1154845419 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f608006f390 0x7f6080156e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:05.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:05.252+0000 7f6086600640 1 -- 172.21.15.151:0/1154845419 >> 172.21.15.151:0/1154845419 conn(0x7f60800d1cd0 msgr2=0x7f60800b3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:05.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:05.252+0000 7f6086600640 1 -- 172.21.15.151:0/1154845419 shutdown_connections 2024-09-16T15:40:05.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:05.252+0000 7f6086600640 1 -- 172.21.15.151:0/1154845419 wait complete. 2024-09-16T15:40:05.272 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-16T15:40:05.272 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-16T15:40:05.272 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-16T15:40:05.273 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:40:05.273 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-16T15:40:05.273 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-16T15:40:05.273 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-16T15:40:05.273 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:40:05.273 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-16T15:40:05.274 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-16T15:40:05.274 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-16T15:40:05.274 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:40:05.274 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-16T15:40:05.274 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-16T15:40:05.274 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-16T15:40:05.274 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:40:05.288 INFO:tasks.ceph.mds.c:Stopped 2024-09-16T15:40:05.289 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-16T15:40:05.289 INFO:tasks.ceph.mds.a:Stopped 2024-09-16T15:40:05.290 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-16T15:40:05.291 INFO:tasks.ceph.mds.d:Stopped 2024-09-16T15:40:05.291 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-16T15:40:05.292 INFO:tasks.ceph.mds.b:Stopped 2024-09-16T15:40:05.292 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-16T15:40:05.333 INFO:tasks.ceph.mds.c:Started 2024-09-16T15:40:05.333 DEBUG:teuthology.parallel:result is None 2024-09-16T15:40:05.335 INFO:tasks.ceph.mds.a:Started 2024-09-16T15:40:05.335 DEBUG:teuthology.parallel:result is None 2024-09-16T15:40:05.337 INFO:tasks.ceph.mds.d:Started 2024-09-16T15:40:05.337 DEBUG:teuthology.parallel:result is None 2024-09-16T15:40:05.340 INFO:tasks.ceph.mds.b:Started 2024-09-16T15:40:05.340 DEBUG:teuthology.parallel:result is None 2024-09-16T15:40:05.341 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-16T15:40:05.468 INFO:tasks.ceph.mds.a.smithi151.stdout:starting mds.a at 2024-09-16T15:40:05.469 INFO:tasks.ceph.mds.c.smithi151.stdout:starting mds.c at 2024-09-16T15:40:05.474 INFO:tasks.ceph.mds.d.smithi167.stdout:starting mds.d at 2024-09-16T15:40:05.491 INFO:tasks.ceph.mds.b.smithi167.stdout:starting mds.b at 2024-09-16T15:40:05.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:05.508+0000 7fa445800640 1 -- 172.21.15.151:0/450124772 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4400cc750 msgr2=0x7fa4400ccc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:05.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:05.508+0000 7fa445800640 1 --2- 172.21.15.151:0/450124772 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4400cc750 0x7fa4400ccc00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa438005fd0 tx=0x7fa43802aa50 comp rx=0 tx=0).stop 2024-09-16T15:40:05.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:05.508+0000 7fa445800640 1 -- 172.21.15.151:0/450124772 shutdown_connections 2024-09-16T15:40:05.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:05.508+0000 7fa445800640 1 --2- 172.21.15.151:0/450124772 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4400cc750 0x7fa4400ccc00 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:05.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:05.508+0000 7fa445800640 1 --2- 172.21.15.151:0/450124772 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa4400cff60 0x7fa4400d03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:05.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:05.508+0000 7fa445800640 1 --2- 172.21.15.151:0/450124772 >> v2:172.21.15.167:3300/0 conn(0x7fa44015bd30 0x7fa4400d2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:05.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:05.508+0000 7fa445800640 1 -- 172.21.15.151:0/450124772 >> 172.21.15.151:0/450124772 conn(0x7fa4400d1c40 msgr2=0x7fa4400d2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:05.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:05.508+0000 7fa445800640 1 -- 172.21.15.151:0/450124772 shutdown_connections 2024-09-16T15:40:05.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:05.508+0000 7fa445800640 1 -- 172.21.15.151:0/450124772 wait complete. 2024-09-16T15:40:05.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:05.510+0000 7fa445800640 1 Processor -- start 2024-09-16T15:40:05.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:05.510+0000 7fa445800640 1 -- start start 2024-09-16T15:40:05.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:05.511+0000 7fa445800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4400cc750 0x7fa4400b5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:05.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:05.511+0000 7fa445800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa4400cff60 0x7fa440154200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:05.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:05.511+0000 7fa445800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa44015bd30 0x7fa4400b7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:05.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:05.511+0000 7fa445800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa4400cd140 con 0x7fa44015bd30 2024-09-16T15:40:05.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:05.511+0000 7fa445800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa4400cfa20 con 0x7fa4400cc750 2024-09-16T15:40:05.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:05.511+0000 7fa445800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fa4400cd2c0 con 0x7fa4400cff60 2024-09-16T15:40:05.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:05.511+0000 7fa444e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa44015bd30 0x7fa4400b7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:05.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:05.511+0000 7fa43fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4400cc750 0x7fa4400b5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:05.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:05.511+0000 7fa444e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa44015bd30 0x7fa4400b7260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:38488/0 (socket says 172.21.15.151:38488) 2024-09-16T15:40:05.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:05.511+0000 7fa444e00640 1 -- 172.21.15.151:0/1806879724 learned_addr learned my addr 172.21.15.151:0/1806879724 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:40:05.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:05.511+0000 7fa437400640 1 --2- 172.21.15.151:0/1806879724 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa4400cff60 0x7fa440154200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:05.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:05.512+0000 7fa43fe00640 1 -- 172.21.15.151:0/1806879724 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa4400cff60 msgr2=0x7fa440154200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:05.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:05.512+0000 7fa43fe00640 1 --2- 172.21.15.151:0/1806879724 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa4400cff60 0x7fa440154200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:05.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:05.512+0000 7fa43fe00640 1 -- 172.21.15.151:0/1806879724 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa44015bd30 msgr2=0x7fa4400b7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:05.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:05.512+0000 7fa43fe00640 1 --2- 172.21.15.151:0/1806879724 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa44015bd30 0x7fa4400b7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:05.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:05.512+0000 7fa43fe00640 1 -- 172.21.15.151:0/1806879724 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa440154740 con 0x7fa4400cc750 2024-09-16T15:40:05.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:05.512+0000 7fa444e00640 1 --2- 172.21.15.151:0/1806879724 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa44015bd30 0x7fa4400b7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:40:05.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:05.512+0000 7fa43fe00640 1 --2- 172.21.15.151:0/1806879724 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4400cc750 0x7fa4400b5ac0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa43000d680 tx=0x7fa43000db40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:05.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:05.512+0000 7fa43cc00640 1 -- 172.21.15.151:0/1806879724 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa430017070 con 0x7fa4400cc750 2024-09-16T15:40:05.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:05.512+0000 7fa43cc00640 1 -- 172.21.15.151:0/1806879724 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa430005310 con 0x7fa4400cc750 2024-09-16T15:40:05.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:05.512+0000 7fa445800640 1 -- 172.21.15.151:0/1806879724 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4400b61c0 con 0x7fa4400cc750 2024-09-16T15:40:05.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:05.512+0000 7fa43cc00640 1 -- 172.21.15.151:0/1806879724 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa430009820 con 0x7fa4400cc750 2024-09-16T15:40:05.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:05.512+0000 7fa445800640 1 -- 172.21.15.151:0/1806879724 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4400b6780 con 0x7fa4400cc750 2024-09-16T15:40:05.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:05.513+0000 7fa445800640 1 -- 172.21.15.151:0/1806879724 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4400d03a0 con 0x7fa4400cc750 2024-09-16T15:40:05.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:05.515+0000 7fa43cc00640 1 -- 172.21.15.151:0/1806879724 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fa4300099c0 con 0x7fa4400cc750 2024-09-16T15:40:05.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:05.516+0000 7fa43cc00640 1 --2- 172.21.15.151:0/1806879724 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa41c078850 0x7fa41c07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:05.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:05.516+0000 7fa43cc00640 1 -- 172.21.15.151:0/1806879724 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 4943+0+0 (secure 0 0 0) 0x7fa43009a550 con 0x7fa4400cc750 2024-09-16T15:40:05.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:05.516+0000 7fa437400640 1 --2- 172.21.15.151:0/1806879724 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa41c078850 0x7fa41c07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:05.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:05.517+0000 7fa437400640 1 --2- 172.21.15.151:0/1806879724 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa41c078850 0x7fa41c07ad10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa4401f5c30 tx=0x7fa42800b310 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:05.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:05.520+0000 7fa43cc00640 1 -- 172.21.15.151:0/1806879724 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa430062860 con 0x7fa4400cc750 2024-09-16T15:40:05.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:05.720+0000 7fa445800640 1 -- 172.21.15.151:0/1806879724 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fa44006eea0 con 0x7fa4400cc750 2024-09-16T15:40:06.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.262+0000 7fa43cc00640 1 -- 172.21.15.151:0/1806879724 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v29) ==== 97+0+0 (secure 0 0 0) 0x7fa430067290 con 0x7fa4400cc750 2024-09-16T15:40:06.265 INFO:teuthology.orchestra.run.smithi151.stderr: removed all blocklist entries 2024-09-16T15:40:06.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.265+0000 7fa445800640 1 -- 172.21.15.151:0/1806879724 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa41c078850 msgr2=0x7fa41c07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:06.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.265+0000 7fa445800640 1 --2- 172.21.15.151:0/1806879724 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa41c078850 0x7fa41c07ad10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa4401f5c30 tx=0x7fa42800b310 comp rx=0 tx=0).stop 2024-09-16T15:40:06.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.265+0000 7fa445800640 1 -- 172.21.15.151:0/1806879724 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4400cc750 msgr2=0x7fa4400b5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:06.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.265+0000 7fa445800640 1 --2- 172.21.15.151:0/1806879724 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4400cc750 0x7fa4400b5ac0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa43000d680 tx=0x7fa43000db40 comp rx=0 tx=0).stop 2024-09-16T15:40:06.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.265+0000 7fa445800640 1 -- 172.21.15.151:0/1806879724 shutdown_connections 2024-09-16T15:40:06.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.265+0000 7fa445800640 1 --2- 172.21.15.151:0/1806879724 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa41c078850 0x7fa41c07ad10 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:06.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.265+0000 7fa445800640 1 --2- 172.21.15.151:0/1806879724 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa44015bd30 0x7fa4400b7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:06.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.265+0000 7fa445800640 1 --2- 172.21.15.151:0/1806879724 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa4400cff60 0x7fa440154200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:06.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.265+0000 7fa445800640 1 --2- 172.21.15.151:0/1806879724 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4400cc750 0x7fa4400b5ac0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:06.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.265+0000 7fa445800640 1 -- 172.21.15.151:0/1806879724 >> 172.21.15.151:0/1806879724 conn(0x7fa4400d1c40 msgr2=0x7fa4400b0440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:06.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.265+0000 7fa445800640 1 -- 172.21.15.151:0/1806879724 shutdown_connections 2024-09-16T15:40:06.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.265+0000 7fa445800640 1 -- 172.21.15.151:0/1806879724 wait complete. 2024-09-16T15:40:06.286 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-16T15:40:06.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.444+0000 7f7337200640 1 -- 172.21.15.151:0/126545244 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73380b0f20 msgr2=0x7f731c003a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:06.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.444+0000 7f7337200640 1 --2- 172.21.15.151:0/126545244 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73380b0f20 0x7f731c003a30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7328009b50 tx=0x7f731c005830 comp rx=0 tx=0).stop 2024-09-16T15:40:06.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.444+0000 7f7336800640 1 -- 172.21.15.151:0/126545244 reap_dead start 2024-09-16T15:40:06.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.444+0000 7f7337200640 1 -- 172.21.15.151:0/126545244 shutdown_connections 2024-09-16T15:40:06.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.444+0000 7f7337200640 1 -- 172.21.15.151:0/126545244 >> 172.21.15.151:0/126545244 conn(0x7f73380b1e90 msgr2=0x7f73380b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:06.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.444+0000 7f7337200640 1 -- 172.21.15.151:0/126545244 shutdown_connections 2024-09-16T15:40:06.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.444+0000 7f7337200640 1 -- 172.21.15.151:0/126545244 wait complete. 2024-09-16T15:40:06.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.445+0000 7f7337200640 1 Processor -- start 2024-09-16T15:40:06.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.446+0000 7f7337200640 1 -- start start 2024-09-16T15:40:06.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.446+0000 7f7337200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73380b0f20 0x7f7338155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:06.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.446+0000 7f7337200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73380c66e0 0x7f73380c6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:06.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.446+0000 7f7337200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73380cf490 0x7f73380cf910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:06.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.446+0000 7f7337200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7338074fe0 con 0x7f73380c66e0 2024-09-16T15:40:06.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.446+0000 7f7337200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7338074e60 con 0x7f73380cf490 2024-09-16T15:40:06.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.446+0000 7f7337200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f73380c4000 con 0x7f73380b0f20 2024-09-16T15:40:06.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.447+0000 7f732fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73380c66e0 0x7f73380c6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:06.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.447+0000 7f732fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73380c66e0 0x7f73380c6b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:38504/0 (socket says 172.21.15.151:38504) 2024-09-16T15:40:06.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.447+0000 7f732fe00640 1 -- 172.21.15.151:0/138343478 learned_addr learned my addr 172.21.15.151:0/138343478 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:40:06.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.447+0000 7f7335e00640 1 --2- 172.21.15.151:0/138343478 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73380b0f20 0x7f7338155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:06.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.447+0000 7f7336800640 1 --2- 172.21.15.151:0/138343478 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73380cf490 0x7f73380cf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:06.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.447+0000 7f732fe00640 1 -- 172.21.15.151:0/138343478 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73380b0f20 msgr2=0x7f7338155fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:06.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.447+0000 7f732fe00640 1 --2- 172.21.15.151:0/138343478 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73380b0f20 0x7f7338155fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:06.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.447+0000 7f732fe00640 1 -- 172.21.15.151:0/138343478 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73380cf490 msgr2=0x7f73380cf910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:06.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.447+0000 7f732fe00640 1 --2- 172.21.15.151:0/138343478 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73380cf490 0x7f73380cf910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:06.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.447+0000 7f732fe00640 1 -- 172.21.15.151:0/138343478 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73380c4250 con 0x7f73380c66e0 2024-09-16T15:40:06.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.447+0000 7f732fe00640 1 --2- 172.21.15.151:0/138343478 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73380c66e0 0x7f73380c6b60 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7320004dc0 tx=0x7f7320001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:06.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.448+0000 7f732e000640 1 -- 172.21.15.151:0/138343478 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7320015020 con 0x7f73380c66e0 2024-09-16T15:40:06.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.448+0000 7f732e000640 1 -- 172.21.15.151:0/138343478 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7320011960 con 0x7f73380c66e0 2024-09-16T15:40:06.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.448+0000 7f732e000640 1 -- 172.21.15.151:0/138343478 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7320011d50 con 0x7f73380c66e0 2024-09-16T15:40:06.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.448+0000 7f7337200640 1 -- 172.21.15.151:0/138343478 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73380c4530 con 0x7f73380c66e0 2024-09-16T15:40:06.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.448+0000 7f7337200640 1 -- 172.21.15.151:0/138343478 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73380c4af0 con 0x7f73380c66e0 2024-09-16T15:40:06.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.449+0000 7f7337200640 1 -- 172.21.15.151:0/138343478 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73380cbbf0 con 0x7f73380c66e0 2024-09-16T15:40:06.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.450+0000 7f732e000640 1 -- 172.21.15.151:0/138343478 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f7320018020 con 0x7f73380c66e0 2024-09-16T15:40:06.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.451+0000 7f732e000640 1 --2- 172.21.15.151:0/138343478 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7314078880 0x7f731407ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:06.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.451+0000 7f732e000640 1 -- 172.21.15.151:0/138343478 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7f732009af20 con 0x7f73380c66e0 2024-09-16T15:40:06.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.451+0000 7f7335e00640 1 --2- 172.21.15.151:0/138343478 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7314078880 0x7f731407ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:06.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.452+0000 7f7335e00640 1 --2- 172.21.15.151:0/138343478 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7314078880 0x7f731407ad40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7328035000 tx=0x7f73280046c0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:06.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.455+0000 7f732e000640 1 -- 172.21.15.151:0/138343478 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f732001f020 con 0x7f73380c66e0 2024-09-16T15:40:06.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.707+0000 7f7337200640 1 -- 172.21.15.151:0/138343478 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f73380c4cc0 con 0x7f73380c66e0 2024-09-16T15:40:06.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.708+0000 7f732e000640 1 -- 172.21.15.151:0/138343478 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v2) ==== 80+0+6645 (secure 0 0 0) 0x7f73200631f0 con 0x7f73380c66e0 2024-09-16T15:40:06.710 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:40:06.710 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-16T15:40:06.710 INFO:teuthology.orchestra.run.smithi151.stdout: "auth_dump": [ 2024-09-16T15:40:06.711 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:40:06.711 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "mds.a", 2024-09-16T15:40:06.711 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDsUOhmJVK6AhAANfmTPS6kKn41LSD/MuyW9g==", 2024-09-16T15:40:06.711 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:40:06.711 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:40:06.711 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow *", 2024-09-16T15:40:06.711 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow *", 2024-09-16T15:40:06.711 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:40:06.711 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:40:06.711 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:40:06.711 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:40:06.711 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "mds.b", 2024-09-16T15:40:06.711 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDsUOhmX1R3FhAAUQgUsE/6wUyokXcZuBuIWg==", 2024-09-16T15:40:06.711 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:40:06.712 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:40:06.712 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow *", 2024-09-16T15:40:06.712 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow *", 2024-09-16T15:40:06.712 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:40:06.712 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:40:06.712 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:40:06.712 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:40:06.712 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "mds.c", 2024-09-16T15:40:06.712 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDsUOhmYGCKDBAAY2EFNmiFNf1x2so9wxp4YQ==", 2024-09-16T15:40:06.712 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:40:06.712 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:40:06.712 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow *", 2024-09-16T15:40:06.712 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow *", 2024-09-16T15:40:06.713 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:40:06.713 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:40:06.713 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:40:06.713 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:40:06.713 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "mds.d", 2024-09-16T15:40:06.713 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDsUOhmfCk4IBAAx12Gw2QudQ2nrPb8VlE0SA==", 2024-09-16T15:40:06.713 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:40:06.713 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:40:06.713 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow *", 2024-09-16T15:40:06.713 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow *", 2024-09-16T15:40:06.713 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:40:06.713 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:40:06.713 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:40:06.713 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:40:06.714 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.0", 2024-09-16T15:40:06.714 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDvUOhmKji5NxAAn/E3YQR7jeMnBTLBXc1G6A==", 2024-09-16T15:40:06.714 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:40:06.714 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:40:06.714 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:40:06.714 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:40:06.714 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:40:06.714 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:40:06.714 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:40:06.714 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.1", 2024-09-16T15:40:06.714 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDzUOhmmBrNBhAA0Uwh4Fam2bmKKvz5jAQBWw==", 2024-09-16T15:40:06.714 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:40:06.714 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:40:06.714 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:40:06.714 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:40:06.715 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:40:06.715 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:40:06.715 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:40:06.715 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.2", 2024-09-16T15:40:06.715 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQD2UOhmOV6GERAAjD+PQLTZrCFR1669UEEIAQ==", 2024-09-16T15:40:06.715 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:40:06.715 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:40:06.715 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:40:06.715 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:40:06.715 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:40:06.715 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:40:06.715 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:40:06.715 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.3", 2024-09-16T15:40:06.715 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQD5UOhmCRrdGRAAQC0bqWX6N1jniVGEQ2aAvg==", 2024-09-16T15:40:06.716 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:40:06.716 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:40:06.716 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:40:06.716 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:40:06.716 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:40:06.716 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:40:06.716 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:40:06.716 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.4", 2024-09-16T15:40:06.716 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQD/UOhmXI/nGRAA7t6XveKvrWgqy6rHuMQSZQ==", 2024-09-16T15:40:06.716 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:40:06.716 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:40:06.716 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:40:06.716 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:40:06.716 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:40:06.717 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:40:06.717 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:40:06.717 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.5", 2024-09-16T15:40:06.717 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQACUehmKM//IxAAsoFpiiTXNs//v3YXeUf/Iw==", 2024-09-16T15:40:06.717 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:40:06.717 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:40:06.717 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:40:06.717 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:40:06.717 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:40:06.717 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:40:06.717 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:40:06.717 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.6", 2024-09-16T15:40:06.717 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQAFUehmlo/FLRAAp5mvUSWCaBbm85QyTZY0eA==", 2024-09-16T15:40:06.717 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:40:06.717 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:40:06.718 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:40:06.718 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:40:06.718 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:40:06.718 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:40:06.718 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:40:06.718 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.7", 2024-09-16T15:40:06.718 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQAIUehmR32JNxAA27antOIKot1nScQbLZDf5g==", 2024-09-16T15:40:06.718 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:40:06.718 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:40:06.718 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:40:06.718 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:40:06.718 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:40:06.718 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:40:06.718 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:40:06.719 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "client.0", 2024-09-16T15:40:06.719 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDsUOhmf5PBKBAAKE4Y4zbA4h07IBNjPkyptw==", 2024-09-16T15:40:06.719 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:40:06.719 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:40:06.719 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow r", 2024-09-16T15:40:06.719 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow rw", 2024-09-16T15:40:06.719 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow rwx" 2024-09-16T15:40:06.719 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:40:06.719 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:40:06.719 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:40:06.720 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "client.1", 2024-09-16T15:40:06.720 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDsUOhm15emNRAAwhK7QmA2c9IYGn04ERvY0g==", 2024-09-16T15:40:06.720 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:40:06.720 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:40:06.720 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow r", 2024-09-16T15:40:06.720 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow rw", 2024-09-16T15:40:06.720 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow rwx" 2024-09-16T15:40:06.720 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:40:06.720 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:40:06.720 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:40:06.720 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "client.2", 2024-09-16T15:40:06.721 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDsUOhm6zUyLxAAkzeiIQTuNZkMM+Oit9rC5A==", 2024-09-16T15:40:06.721 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:40:06.721 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:40:06.721 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow r", 2024-09-16T15:40:06.721 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow rw", 2024-09-16T15:40:06.721 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow rwx" 2024-09-16T15:40:06.721 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:40:06.721 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:40:06.721 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:40:06.721 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "client.3", 2024-09-16T15:40:06.721 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDtUOhmD82EABAAeDf82/hPxpPfAUk8oj5h/Q==", 2024-09-16T15:40:06.721 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:40:06.721 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:40:06.721 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow r", 2024-09-16T15:40:06.722 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow rw", 2024-09-16T15:40:06.722 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow rwx" 2024-09-16T15:40:06.722 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:40:06.722 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:40:06.722 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:40:06.722 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "client.admin", 2024-09-16T15:40:06.722 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDrUOhmQFmVDRAA8NKysbAe475xvg4CPAcFyw==", 2024-09-16T15:40:06.722 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:40:06.722 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow *", 2024-09-16T15:40:06.722 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow *", 2024-09-16T15:40:06.722 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow *", 2024-09-16T15:40:06.722 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:40:06.722 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:40:06.722 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:40:06.723 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:40:06.723 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "client.bootstrap-mds", 2024-09-16T15:40:06.723 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQAQUehmo8fSFBAAa4kAnpWnqZyj7yyYMw73mA==", 2024-09-16T15:40:06.723 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:40:06.723 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile bootstrap-mds" 2024-09-16T15:40:06.723 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:40:06.723 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:40:06.723 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:40:06.723 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "client.bootstrap-mgr", 2024-09-16T15:40:06.723 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQAQUehm6dnSFBAA3D62Rs0WgHgfVyOAKAUxoA==", 2024-09-16T15:40:06.723 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:40:06.723 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile bootstrap-mgr" 2024-09-16T15:40:06.723 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:40:06.723 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:40:06.724 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:40:06.724 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "client.bootstrap-osd", 2024-09-16T15:40:06.724 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQAQUehmFuzSFBAAk6WX0Xd+ztmaBZdOAiy5ug==", 2024-09-16T15:40:06.724 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:40:06.724 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile bootstrap-osd" 2024-09-16T15:40:06.724 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:40:06.724 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:40:06.724 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:40:06.724 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "client.bootstrap-rbd", 2024-09-16T15:40:06.724 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQAQUehmcAHTFBAAbqA+BBvgkFMq8KfxtQCNEA==", 2024-09-16T15:40:06.724 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:40:06.724 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile bootstrap-rbd" 2024-09-16T15:40:06.724 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:40:06.724 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:40:06.725 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:40:06.725 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-09-16T15:40:06.725 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQAQUehm7hLTFBAA2BRxdOcX0+7SoG5HVRu+wA==", 2024-09-16T15:40:06.725 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:40:06.725 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-09-16T15:40:06.725 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:40:06.725 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:40:06.725 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:40:06.725 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "client.bootstrap-rgw", 2024-09-16T15:40:06.725 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQAQUehm4iPTFBAAVqneKPOXn9APLCNlGmm4lw==", 2024-09-16T15:40:06.725 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:40:06.725 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile bootstrap-rgw" 2024-09-16T15:40:06.725 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:40:06.725 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:40:06.726 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:40:06.726 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "mgr.x", 2024-09-16T15:40:06.726 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDrUOhmloW3NxAA8UoLUfrw3wf1nzKbmtK/jw==", 2024-09-16T15:40:06.726 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:40:06.726 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow *", 2024-09-16T15:40:06.726 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile mgr", 2024-09-16T15:40:06.726 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:40:06.726 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:40:06.726 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:40:06.726 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:40:06.726 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "mgr.y", 2024-09-16T15:40:06.726 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDrUOhm1uZOLxAATgjA79vYf0psqQVo03sdDA==", 2024-09-16T15:40:06.726 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:40:06.726 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow *", 2024-09-16T15:40:06.726 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile mgr", 2024-09-16T15:40:06.727 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:40:06.727 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:40:06.727 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:40:06.727 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-09-16T15:40:06.727 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-16T15:40:06.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.711+0000 7f7337200640 1 -- 172.21.15.151:0/138343478 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7314078880 msgr2=0x7f731407ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:06.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.711+0000 7f7337200640 1 --2- 172.21.15.151:0/138343478 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7314078880 0x7f731407ad40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7328035000 tx=0x7f73280046c0 comp rx=0 tx=0).stop 2024-09-16T15:40:06.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.711+0000 7f7337200640 1 -- 172.21.15.151:0/138343478 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73380c66e0 msgr2=0x7f73380c6b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:06.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.711+0000 7f7337200640 1 --2- 172.21.15.151:0/138343478 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73380c66e0 0x7f73380c6b60 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7320004dc0 tx=0x7f7320001710 comp rx=0 tx=0).stop 2024-09-16T15:40:06.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.711+0000 7f7337200640 1 -- 172.21.15.151:0/138343478 shutdown_connections 2024-09-16T15:40:06.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.711+0000 7f7337200640 1 --2- 172.21.15.151:0/138343478 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7314078880 0x7f731407ad40 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:06.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.711+0000 7f7337200640 1 --2- 172.21.15.151:0/138343478 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f73380cf490 0x7f73380cf910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:06.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.711+0000 7f7337200640 1 --2- 172.21.15.151:0/138343478 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73380c66e0 0x7f73380c6b60 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:06.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.711+0000 7f7337200640 1 --2- 172.21.15.151:0/138343478 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73380b0f20 0x7f7338155fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:06.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.711+0000 7f7337200640 1 -- 172.21.15.151:0/138343478 >> 172.21.15.151:0/138343478 conn(0x7f73380b1e90 msgr2=0x7f7338074290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:06.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.711+0000 7f7337200640 1 -- 172.21.15.151:0/138343478 shutdown_connections 2024-09-16T15:40:06.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.711+0000 7f7337200640 1 -- 172.21.15.151:0/138343478 wait complete. 2024-09-16T15:40:06.738 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-09-16T15:40:06.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.896+0000 7fab8fe00640 1 -- 172.21.15.151:0/462240997 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab90161440 msgr2=0x7fab900b5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:06.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.896+0000 7fab8fe00640 1 --2- 172.21.15.151:0/462240997 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab90161440 0x7fab900b5370 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fab900c7470 tx=0x7fab8002a560 comp rx=0 tx=0).stop 2024-09-16T15:40:06.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.896+0000 7fab8fe00640 1 -- 172.21.15.151:0/462240997 shutdown_connections 2024-09-16T15:40:06.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.896+0000 7fab8fe00640 1 --2- 172.21.15.151:0/462240997 >> v2:172.21.15.167:3300/0 conn(0x7fab900b34b0 0x7fab900b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:06.899 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.896+0000 7fab8fe00640 1 --2- 172.21.15.151:0/462240997 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab90161440 0x7fab900b5370 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:06.899 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.896+0000 7fab8fe00640 1 -- 172.21.15.151:0/462240997 >> 172.21.15.151:0/462240997 conn(0x7fab900d3df0 msgr2=0x7fab900d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:06.899 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.896+0000 7fab8fe00640 1 -- 172.21.15.151:0/462240997 shutdown_connections 2024-09-16T15:40:06.899 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.896+0000 7fab8fe00640 1 -- 172.21.15.151:0/462240997 wait complete. 2024-09-16T15:40:06.899 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.897+0000 7fab8fe00640 1 Processor -- start 2024-09-16T15:40:06.899 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.897+0000 7fab8fe00640 1 -- start start 2024-09-16T15:40:06.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.898+0000 7fab8fe00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab900b34b0 0x7fab901f4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:06.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.898+0000 7fab8fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab900b3e80 0x7fab900cfd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:06.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.898+0000 7fab8fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab90161440 0x7fab900d0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:06.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.898+0000 7fab8fe00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fab900b2900 con 0x7fab900b3e80 2024-09-16T15:40:06.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.898+0000 7fab8fe00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fab900b2780 con 0x7fab90161440 2024-09-16T15:40:06.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.898+0000 7fab8fe00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fab900b2a80 con 0x7fab900b34b0 2024-09-16T15:40:06.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.899+0000 7fab8e000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab900b3e80 0x7fab900cfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:06.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.899+0000 7fab8e000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab900b3e80 0x7fab900cfd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:38528/0 (socket says 172.21.15.151:38528) 2024-09-16T15:40:06.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.899+0000 7fab8e000640 1 -- 172.21.15.151:0/1437719881 learned_addr learned my addr 172.21.15.151:0/1437719881 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:40:06.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.899+0000 7fab8f400640 1 --2- 172.21.15.151:0/1437719881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab90161440 0x7fab900d0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:06.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.899+0000 7fab8ea00640 1 --2- 172.21.15.151:0/1437719881 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab900b34b0 0x7fab901f4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:06.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.899+0000 7fab8e000640 1 -- 172.21.15.151:0/1437719881 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab900b34b0 msgr2=0x7fab901f4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:06.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.899+0000 7fab8e000640 1 --2- 172.21.15.151:0/1437719881 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab900b34b0 0x7fab901f4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:06.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.899+0000 7fab8e000640 1 -- 172.21.15.151:0/1437719881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab90161440 msgr2=0x7fab900d0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:06.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.899+0000 7fab8e000640 1 --2- 172.21.15.151:0/1437719881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab90161440 0x7fab900d0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:06.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.899+0000 7fab8e000640 1 -- 172.21.15.151:0/1437719881 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab901f8140 con 0x7fab900b3e80 2024-09-16T15:40:06.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.899+0000 7fab8e000640 1 --2- 172.21.15.151:0/1437719881 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab900b3e80 0x7fab900cfd20 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fab7c00d770 tx=0x7fab7c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:06.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.899+0000 7fab86a00640 1 -- 172.21.15.151:0/1437719881 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7c017070 con 0x7fab900b3e80 2024-09-16T15:40:06.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.900+0000 7fab86a00640 1 -- 172.21.15.151:0/1437719881 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fab7c009d20 con 0x7fab900b3e80 2024-09-16T15:40:06.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.900+0000 7fab8fe00640 1 -- 172.21.15.151:0/1437719881 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab901f8420 con 0x7fab900b3e80 2024-09-16T15:40:06.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.900+0000 7fab86a00640 1 -- 172.21.15.151:0/1437719881 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7c005280 con 0x7fab900b3e80 2024-09-16T15:40:06.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.900+0000 7fab8fe00640 1 -- 172.21.15.151:0/1437719881 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab901f89e0 con 0x7fab900b3e80 2024-09-16T15:40:06.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.901+0000 7fab8fe00640 1 -- 172.21.15.151:0/1437719881 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab900b3940 con 0x7fab900b3e80 2024-09-16T15:40:06.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.902+0000 7fab86a00640 1 -- 172.21.15.151:0/1437719881 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fab7c005420 con 0x7fab900b3e80 2024-09-16T15:40:06.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.903+0000 7fab86a00640 1 --2- 172.21.15.151:0/1437719881 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fab70078850 0x7fab7007ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:06.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.903+0000 7fab86a00640 1 -- 172.21.15.151:0/1437719881 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7fab7c09a7a0 con 0x7fab900b3e80 2024-09-16T15:40:06.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.903+0000 7fab8ea00640 1 --2- 172.21.15.151:0/1437719881 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fab70078850 0x7fab7007ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:06.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.904+0000 7fab8ea00640 1 --2- 172.21.15.151:0/1437719881 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fab70078850 0x7fab7007ad10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fab80033000 tx=0x7fab80004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:06.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:06.908+0000 7fab86a00640 1 -- 172.21.15.151:0/1437719881 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fab7c063150 con 0x7fab900b3e80 2024-09-16T15:40:07.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.167+0000 7fab8fe00640 1 -- 172.21.15.151:0/1437719881 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7fab90158a70 con 0x7fab900b3e80 2024-09-16T15:40:07.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.169+0000 7fab86a00640 1 -- 172.21.15.151:0/1437719881 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v3) ==== 90+0+0 (secure 0 0 0) 0x7fab7c062a50 con 0x7fab900b3e80 2024-09-16T15:40:07.174 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.172+0000 7fab8fe00640 1 -- 172.21.15.151:0/1437719881 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fab70078850 msgr2=0x7fab7007ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:07.174 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.172+0000 7fab8fe00640 1 --2- 172.21.15.151:0/1437719881 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fab70078850 0x7fab7007ad10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fab80033000 tx=0x7fab80004730 comp rx=0 tx=0).stop 2024-09-16T15:40:07.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.172+0000 7fab8fe00640 1 -- 172.21.15.151:0/1437719881 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab900b3e80 msgr2=0x7fab900cfd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:07.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.172+0000 7fab8fe00640 1 --2- 172.21.15.151:0/1437719881 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab900b3e80 0x7fab900cfd20 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fab7c00d770 tx=0x7fab7c00dc30 comp rx=0 tx=0).stop 2024-09-16T15:40:07.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.172+0000 7fab8fe00640 1 -- 172.21.15.151:0/1437719881 shutdown_connections 2024-09-16T15:40:07.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.172+0000 7fab8fe00640 1 --2- 172.21.15.151:0/1437719881 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fab70078850 0x7fab7007ad10 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:07.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.172+0000 7fab8fe00640 1 --2- 172.21.15.151:0/1437719881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab90161440 0x7fab900d0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:07.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.172+0000 7fab8fe00640 1 --2- 172.21.15.151:0/1437719881 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab900b3e80 0x7fab900cfd20 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:07.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.172+0000 7fab8fe00640 1 --2- 172.21.15.151:0/1437719881 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab900b34b0 0x7fab901f4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:07.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.172+0000 7fab8fe00640 1 -- 172.21.15.151:0/1437719881 >> 172.21.15.151:0/1437719881 conn(0x7fab900d3df0 msgr2=0x7fab900b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:07.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.172+0000 7fab8fe00640 1 -- 172.21.15.151:0/1437719881 shutdown_connections 2024-09-16T15:40:07.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.172+0000 7fab8fe00640 1 -- 172.21.15.151:0/1437719881 wait complete. 2024-09-16T15:40:07.189 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-09-16T15:40:07.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.347+0000 7fdc67000640 1 -- 172.21.15.151:0/4254665394 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdc6015df20 msgr2=0x7fdc6015e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:07.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.347+0000 7fdc67000640 1 --2- 172.21.15.151:0/4254665394 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdc6015df20 0x7fdc6015e300 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fdc6006f7b0 tx=0x7fdc5002a820 comp rx=0 tx=0).stop 2024-09-16T15:40:07.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.347+0000 7fdc67000640 1 -- 172.21.15.151:0/4254665394 shutdown_connections 2024-09-16T15:40:07.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.347+0000 7fdc67000640 1 --2- 172.21.15.151:0/4254665394 >> v2:172.21.15.167:3300/0 conn(0x7fdc600cebf0 0x7fdc60162220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:07.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.347+0000 7fdc67000640 1 --2- 172.21.15.151:0/4254665394 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdc6015df20 0x7fdc6015e300 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:07.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.347+0000 7fdc67000640 1 -- 172.21.15.151:0/4254665394 >> 172.21.15.151:0/4254665394 conn(0x7fdc600d1da0 msgr2=0x7fdc600d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:07.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.348+0000 7fdc67000640 1 -- 172.21.15.151:0/4254665394 shutdown_connections 2024-09-16T15:40:07.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.348+0000 7fdc67000640 1 -- 172.21.15.151:0/4254665394 wait complete. 2024-09-16T15:40:07.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.349+0000 7fdc67000640 1 Processor -- start 2024-09-16T15:40:07.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.349+0000 7fdc67000640 1 -- start start 2024-09-16T15:40:07.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.350+0000 7fdc67000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdc600ce760 0x7fdc600b25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:07.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.350+0000 7fdc67000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc600cebf0 0x7fdc600c7130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:07.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.350+0000 7fdc67000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc6015df20 0x7fdc60153ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:07.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.350+0000 7fdc67000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdc6006ee10 con 0x7fdc600ce760 2024-09-16T15:40:07.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.350+0000 7fdc67000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdc6006ec90 con 0x7fdc6015df20 2024-09-16T15:40:07.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.350+0000 7fdc67000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fdc600cf510 con 0x7fdc600cebf0 2024-09-16T15:40:07.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.350+0000 7fdc5fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdc600ce760 0x7fdc600b25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:07.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.350+0000 7fdc5fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdc600ce760 0x7fdc600b25b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:38554/0 (socket says 172.21.15.151:38554) 2024-09-16T15:40:07.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.350+0000 7fdc5fe00640 1 -- 172.21.15.151:0/741242296 learned_addr learned my addr 172.21.15.151:0/741242296 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:40:07.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.351+0000 7fdc5fe00640 1 -- 172.21.15.151:0/741242296 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc600cebf0 msgr2=0x7fdc600c7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:07.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.351+0000 7fdc5fe00640 1 --2- 172.21.15.151:0/741242296 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc600cebf0 0x7fdc600c7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:07.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.351+0000 7fdc5fe00640 1 -- 172.21.15.151:0/741242296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc6015df20 msgr2=0x7fdc60153ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:07.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.351+0000 7fdc5fe00640 1 --2- 172.21.15.151:0/741242296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc6015df20 0x7fdc60153ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:07.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.351+0000 7fdc5fe00640 1 -- 172.21.15.151:0/741242296 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc6009bf30 con 0x7fdc600ce760 2024-09-16T15:40:07.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.351+0000 7fdc5fe00640 1 --2- 172.21.15.151:0/741242296 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdc600ce760 0x7fdc600b25b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fdc50035040 tx=0x7fdc50032c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:07.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.351+0000 7fdc57e00640 1 -- 172.21.15.151:0/741242296 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc50037040 con 0x7fdc600ce760 2024-09-16T15:40:07.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.351+0000 7fdc67000640 1 -- 172.21.15.151:0/741242296 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc6009c210 con 0x7fdc600ce760 2024-09-16T15:40:07.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.351+0000 7fdc57e00640 1 -- 172.21.15.151:0/741242296 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdc50042070 con 0x7fdc600ce760 2024-09-16T15:40:07.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.352+0000 7fdc67000640 1 -- 172.21.15.151:0/741242296 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc600af150 con 0x7fdc600ce760 2024-09-16T15:40:07.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.352+0000 7fdc57e00640 1 -- 172.21.15.151:0/741242296 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc50032e50 con 0x7fdc600ce760 2024-09-16T15:40:07.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.353+0000 7fdc67000640 1 -- 172.21.15.151:0/741242296 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc600b7c20 con 0x7fdc600ce760 2024-09-16T15:40:07.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.354+0000 7fdc57e00640 1 -- 172.21.15.151:0/741242296 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fdc50004040 con 0x7fdc600ce760 2024-09-16T15:40:07.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.355+0000 7fdc57e00640 1 --2- 172.21.15.151:0/741242296 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdc400789f0 0x7fdc4007aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:07.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.355+0000 7fdc57e00640 1 -- 172.21.15.151:0/741242296 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7fdc500bb320 con 0x7fdc600ce760 2024-09-16T15:40:07.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.355+0000 7fdc5f400640 1 --2- 172.21.15.151:0/741242296 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdc400789f0 0x7fdc4007aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:07.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.356+0000 7fdc5f400640 1 --2- 172.21.15.151:0/741242296 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdc400789f0 0x7fdc4007aeb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c007fd0 tx=0x7fdc4c007900 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:07.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.359+0000 7fdc57e00640 1 -- 172.21.15.151:0/741242296 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fdc50083bc0 con 0x7fdc600ce760 2024-09-16T15:40:07.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.609+0000 7fdc67000640 1 -- 172.21.15.151:0/741242296 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7fdc600b5430 con 0x7fdc600ce760 2024-09-16T15:40:07.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.614+0000 7fdc57e00640 1 -- 172.21.15.151:0/741242296 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v4) ==== 90+0+0 (secure 0 0 0) 0x7fdc500834c0 con 0x7fdc600ce760 2024-09-16T15:40:07.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.616+0000 7fdc67000640 1 -- 172.21.15.151:0/741242296 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdc400789f0 msgr2=0x7fdc4007aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:07.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.616+0000 7fdc67000640 1 --2- 172.21.15.151:0/741242296 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdc400789f0 0x7fdc4007aeb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c007fd0 tx=0x7fdc4c007900 comp rx=0 tx=0).stop 2024-09-16T15:40:07.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.616+0000 7fdc67000640 1 -- 172.21.15.151:0/741242296 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdc600ce760 msgr2=0x7fdc600b25b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:07.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.616+0000 7fdc67000640 1 --2- 172.21.15.151:0/741242296 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdc600ce760 0x7fdc600b25b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fdc50035040 tx=0x7fdc50032c60 comp rx=0 tx=0).stop 2024-09-16T15:40:07.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.616+0000 7fdc67000640 1 -- 172.21.15.151:0/741242296 shutdown_connections 2024-09-16T15:40:07.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.616+0000 7fdc67000640 1 --2- 172.21.15.151:0/741242296 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdc400789f0 0x7fdc4007aeb0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:07.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.616+0000 7fdc67000640 1 --2- 172.21.15.151:0/741242296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc6015df20 0x7fdc60153ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:07.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.616+0000 7fdc67000640 1 --2- 172.21.15.151:0/741242296 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc600cebf0 0x7fdc600c7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:07.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.617+0000 7fdc67000640 1 --2- 172.21.15.151:0/741242296 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdc600ce760 0x7fdc600b25b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:07.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.617+0000 7fdc67000640 1 -- 172.21.15.151:0/741242296 >> 172.21.15.151:0/741242296 conn(0x7fdc600d1da0 msgr2=0x7fdc600d2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:07.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.617+0000 7fdc67000640 1 -- 172.21.15.151:0/741242296 shutdown_connections 2024-09-16T15:40:07.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.617+0000 7fdc67000640 1 -- 172.21.15.151:0/741242296 wait complete. 2024-09-16T15:40:07.634 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-09-16T15:40:07.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.790+0000 7fa717e00640 1 -- 172.21.15.151:0/3634813329 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa71015f5a0 msgr2=0x7fa7100b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:07.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.790+0000 7fa717e00640 1 --2- 172.21.15.151:0/3634813329 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa71015f5a0 0x7fa7100b2440 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa71006dac0 tx=0x7fa70402a4b0 comp rx=0 tx=0).stop 2024-09-16T15:40:07.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.790+0000 7fa717e00640 1 -- 172.21.15.151:0/3634813329 shutdown_connections 2024-09-16T15:40:07.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.790+0000 7fa717e00640 1 --2- 172.21.15.151:0/3634813329 >> v2:172.21.15.167:3300/0 conn(0x7fa710073fb0 0x7fa710074420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:07.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.790+0000 7fa717e00640 1 --2- 172.21.15.151:0/3634813329 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa71015f5a0 0x7fa7100b2440 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:07.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.790+0000 7fa717e00640 1 -- 172.21.15.151:0/3634813329 >> 172.21.15.151:0/3634813329 conn(0x7fa7100b1950 msgr2=0x7fa7100b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:07.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.790+0000 7fa717e00640 1 -- 172.21.15.151:0/3634813329 shutdown_connections 2024-09-16T15:40:07.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.791+0000 7fa717e00640 1 -- 172.21.15.151:0/3634813329 wait complete. 2024-09-16T15:40:07.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.792+0000 7fa717e00640 1 Processor -- start 2024-09-16T15:40:07.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.792+0000 7fa717e00640 1 -- start start 2024-09-16T15:40:07.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.793+0000 7fa717e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa710073fb0 0x7fa710146530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:07.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.793+0000 7fa717e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7100ccf00 0x7fa7100b2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:07.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.793+0000 7fa717e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa71015f5a0 0x7fa710150cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:07.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.793+0000 7fa717e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa71006f2e0 con 0x7fa710073fb0 2024-09-16T15:40:07.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.793+0000 7fa717e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa71006f160 con 0x7fa71015f5a0 2024-09-16T15:40:07.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.793+0000 7fa717e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fa71006f460 con 0x7fa7100ccf00 2024-09-16T15:40:07.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.793+0000 7fa715400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa710073fb0 0x7fa710146530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:07.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.793+0000 7fa715400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa710073fb0 0x7fa710146530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:38558/0 (socket says 172.21.15.151:38558) 2024-09-16T15:40:07.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.793+0000 7fa715400640 1 -- 172.21.15.151:0/1048318116 learned_addr learned my addr 172.21.15.151:0/1048318116 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:40:07.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.794+0000 7fa715e00640 1 --2- 172.21.15.151:0/1048318116 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa71015f5a0 0x7fa710150cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:07.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.794+0000 7fa714a00640 1 --2- 172.21.15.151:0/1048318116 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7100ccf00 0x7fa7100b2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:07.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.794+0000 7fa715400640 1 -- 172.21.15.151:0/1048318116 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7100ccf00 msgr2=0x7fa7100b2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:07.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.794+0000 7fa715400640 1 --2- 172.21.15.151:0/1048318116 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7100ccf00 0x7fa7100b2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:07.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.794+0000 7fa715400640 1 -- 172.21.15.151:0/1048318116 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa71015f5a0 msgr2=0x7fa710150cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:07.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.794+0000 7fa715400640 1 --2- 172.21.15.151:0/1048318116 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa71015f5a0 0x7fa710150cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:07.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.794+0000 7fa715400640 1 -- 172.21.15.151:0/1048318116 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7100b3840 con 0x7fa710073fb0 2024-09-16T15:40:07.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.794+0000 7fa714a00640 1 --2- 172.21.15.151:0/1048318116 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7100ccf00 0x7fa7100b2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:40:07.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.794+0000 7fa715400640 1 --2- 172.21.15.151:0/1048318116 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa710073fb0 0x7fa710146530 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa704009780 tx=0x7fa704033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:07.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.794+0000 7fa70d600640 1 -- 172.21.15.151:0/1048318116 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa704031d60 con 0x7fa710073fb0 2024-09-16T15:40:07.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.794+0000 7fa70d600640 1 -- 172.21.15.151:0/1048318116 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa704033e10 con 0x7fa710073fb0 2024-09-16T15:40:07.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.794+0000 7fa70d600640 1 -- 172.21.15.151:0/1048318116 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa704005260 con 0x7fa710073fb0 2024-09-16T15:40:07.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.794+0000 7fa717e00640 1 -- 172.21.15.151:0/1048318116 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7100b3b20 con 0x7fa710073fb0 2024-09-16T15:40:07.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.795+0000 7fa717e00640 1 -- 172.21.15.151:0/1048318116 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7100b40e0 con 0x7fa710073fb0 2024-09-16T15:40:07.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.796+0000 7fa717e00640 1 -- 172.21.15.151:0/1048318116 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa71006f9f0 con 0x7fa710073fb0 2024-09-16T15:40:07.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.797+0000 7fa70d600640 1 -- 172.21.15.151:0/1048318116 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fa704005400 con 0x7fa710073fb0 2024-09-16T15:40:07.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.797+0000 7fa70d600640 1 --2- 172.21.15.151:0/1048318116 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa6f0078880 0x7fa6f007ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:07.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.798+0000 7fa70d600640 1 -- 172.21.15.151:0/1048318116 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7fa7040bb5f0 con 0x7fa710073fb0 2024-09-16T15:40:07.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.798+0000 7fa714a00640 1 --2- 172.21.15.151:0/1048318116 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa6f0078880 0x7fa6f007ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:07.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.799+0000 7fa714a00640 1 --2- 172.21.15.151:0/1048318116 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa6f0078880 0x7fa6f007ad40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa710074600 tx=0x7fa6fc009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:07.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:07.801+0000 7fa70d600640 1 -- 172.21.15.151:0/1048318116 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa70403f030 con 0x7fa710073fb0 2024-09-16T15:40:08.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.057+0000 7fa717e00640 1 -- 172.21.15.151:0/1048318116 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7fa7100cf720 con 0x7fa710073fb0 2024-09-16T15:40:08.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.061+0000 7fa70d600640 1 -- 172.21.15.151:0/1048318116 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7fa7040838c0 con 0x7fa710073fb0 2024-09-16T15:40:08.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.064+0000 7fa717e00640 1 -- 172.21.15.151:0/1048318116 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa6f0078880 msgr2=0x7fa6f007ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:08.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.064+0000 7fa717e00640 1 --2- 172.21.15.151:0/1048318116 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa6f0078880 0x7fa6f007ad40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa710074600 tx=0x7fa6fc009040 comp rx=0 tx=0).stop 2024-09-16T15:40:08.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.064+0000 7fa717e00640 1 -- 172.21.15.151:0/1048318116 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa710073fb0 msgr2=0x7fa710146530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:08.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.064+0000 7fa717e00640 1 --2- 172.21.15.151:0/1048318116 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa710073fb0 0x7fa710146530 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa704009780 tx=0x7fa704033c20 comp rx=0 tx=0).stop 2024-09-16T15:40:08.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.064+0000 7fa717e00640 1 -- 172.21.15.151:0/1048318116 shutdown_connections 2024-09-16T15:40:08.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.064+0000 7fa717e00640 1 --2- 172.21.15.151:0/1048318116 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa6f0078880 0x7fa6f007ad40 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:08.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.064+0000 7fa717e00640 1 --2- 172.21.15.151:0/1048318116 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa71015f5a0 0x7fa710150cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:08.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.064+0000 7fa717e00640 1 --2- 172.21.15.151:0/1048318116 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa7100ccf00 0x7fa7100b2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:08.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.064+0000 7fa717e00640 1 --2- 172.21.15.151:0/1048318116 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa710073fb0 0x7fa710146530 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:08.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.064+0000 7fa717e00640 1 -- 172.21.15.151:0/1048318116 >> 172.21.15.151:0/1048318116 conn(0x7fa7100b1950 msgr2=0x7fa710153f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:08.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.064+0000 7fa717e00640 1 -- 172.21.15.151:0/1048318116 shutdown_connections 2024-09-16T15:40:08.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.064+0000 7fa717e00640 1 -- 172.21.15.151:0/1048318116 wait complete. 2024-09-16T15:40:08.082 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-09-16T15:40:08.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.238+0000 7ff086800640 1 -- 172.21.15.151:0/4248777859 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0800ccfd0 msgr2=0x7ff080072ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:08.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.238+0000 7ff086800640 1 --2- 172.21.15.151:0/4248777859 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0800ccfd0 0x7ff080072ee0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff078009ca0 tx=0x7ff07802aac0 comp rx=0 tx=0).stop 2024-09-16T15:40:08.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.238+0000 7ff086800640 1 -- 172.21.15.151:0/4248777859 shutdown_connections 2024-09-16T15:40:08.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.238+0000 7ff086800640 1 --2- 172.21.15.151:0/4248777859 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0800ccfd0 0x7ff080072ee0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:08.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.238+0000 7ff086800640 1 --2- 172.21.15.151:0/4248777859 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff0800cf300 0x7ff0800729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:08.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.238+0000 7ff086800640 1 -- 172.21.15.151:0/4248777859 >> 172.21.15.151:0/4248777859 conn(0x7ff0800b1950 msgr2=0x7ff0800b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:08.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.238+0000 7ff086800640 1 -- 172.21.15.151:0/4248777859 shutdown_connections 2024-09-16T15:40:08.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.238+0000 7ff086800640 1 -- 172.21.15.151:0/4248777859 wait complete. 2024-09-16T15:40:08.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.240+0000 7ff086800640 1 Processor -- start 2024-09-16T15:40:08.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.240+0000 7ff086800640 1 -- start start 2024-09-16T15:40:08.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.241+0000 7ff086800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0800ccfd0 0x7ff080141250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:08.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.241+0000 7ff086800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff0800cf300 0x7ff08015f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:08.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.241+0000 7ff086800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0801445d0 0x7ff080142990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:08.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.241+0000 7ff086800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff080073420 con 0x7ff0800ccfd0 2024-09-16T15:40:08.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.241+0000 7ff086800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff0800b4eb0 con 0x7ff0801445d0 2024-09-16T15:40:08.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.241+0000 7ff086800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7ff0800735a0 con 0x7ff0800cf300 2024-09-16T15:40:08.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.241+0000 7ff07f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0800ccfd0 0x7ff080141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:08.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.241+0000 7ff07f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0800ccfd0 0x7ff080141250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:38576/0 (socket says 172.21.15.151:38576) 2024-09-16T15:40:08.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.241+0000 7ff07f400640 1 -- 172.21.15.151:0/3850750094 learned_addr learned my addr 172.21.15.151:0/3850750094 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:40:08.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.241+0000 7ff07ea00640 1 --2- 172.21.15.151:0/3850750094 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff0800cf300 0x7ff08015f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:08.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.241+0000 7ff07fe00640 1 --2- 172.21.15.151:0/3850750094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0801445d0 0x7ff080142990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:08.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.241+0000 7ff07f400640 1 -- 172.21.15.151:0/3850750094 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff0800cf300 msgr2=0x7ff08015f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:08.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.241+0000 7ff07f400640 1 --2- 172.21.15.151:0/3850750094 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff0800cf300 0x7ff08015f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:08.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.241+0000 7ff07f400640 1 -- 172.21.15.151:0/3850750094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0801445d0 msgr2=0x7ff080142990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:08.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.241+0000 7ff07f400640 1 --2- 172.21.15.151:0/3850750094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0801445d0 0x7ff080142990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:08.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.241+0000 7ff07f400640 1 -- 172.21.15.151:0/3850750094 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0801f84a0 con 0x7ff0800ccfd0 2024-09-16T15:40:08.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.242+0000 7ff07f400640 1 --2- 172.21.15.151:0/3850750094 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0800ccfd0 0x7ff080141250 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff07000b3f0 tx=0x7ff07000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:08.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.242+0000 7ff077400640 1 -- 172.21.15.151:0/3850750094 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff070015020 con 0x7ff0800ccfd0 2024-09-16T15:40:08.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.242+0000 7ff077400640 1 -- 172.21.15.151:0/3850750094 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff070011920 con 0x7ff0800ccfd0 2024-09-16T15:40:08.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.242+0000 7ff086800640 1 -- 172.21.15.151:0/3850750094 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0801f8740 con 0x7ff0800ccfd0 2024-09-16T15:40:08.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.242+0000 7ff077400640 1 -- 172.21.15.151:0/3850750094 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff070011d70 con 0x7ff0800ccfd0 2024-09-16T15:40:08.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.242+0000 7ff086800640 1 -- 172.21.15.151:0/3850750094 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0801f8d00 con 0x7ff0800ccfd0 2024-09-16T15:40:08.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.243+0000 7ff086800640 1 -- 172.21.15.151:0/3850750094 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff080073a80 con 0x7ff0800ccfd0 2024-09-16T15:40:08.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.244+0000 7ff077400640 1 -- 172.21.15.151:0/3850750094 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7ff070018020 con 0x7ff0800ccfd0 2024-09-16T15:40:08.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.245+0000 7ff077400640 1 --2- 172.21.15.151:0/3850750094 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff060078850 0x7ff06007ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:08.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.245+0000 7ff077400640 1 -- 172.21.15.151:0/3850750094 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7ff07009a240 con 0x7ff0800ccfd0 2024-09-16T15:40:08.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.246+0000 7ff07ea00640 1 --2- 172.21.15.151:0/3850750094 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff060078850 0x7ff06007ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:08.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.246+0000 7ff07ea00640 1 --2- 172.21.15.151:0/3850750094 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff060078850 0x7ff06007ad10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff06c005fd0 tx=0x7ff06c005850 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:08.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.249+0000 7ff077400640 1 -- 172.21.15.151:0/3850750094 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff07001f030 con 0x7ff0800ccfd0 2024-09-16T15:40:08.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.503+0000 7ff086800640 1 -- 172.21.15.151:0/3850750094 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7ff0800cc330 con 0x7ff0800ccfd0 2024-09-16T15:40:08.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.506+0000 7ff077400640 1 -- 172.21.15.151:0/3850750094 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7ff070062c00 con 0x7ff0800ccfd0 2024-09-16T15:40:08.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.508+0000 7ff086800640 1 -- 172.21.15.151:0/3850750094 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff060078850 msgr2=0x7ff06007ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:08.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.508+0000 7ff086800640 1 --2- 172.21.15.151:0/3850750094 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff060078850 0x7ff06007ad10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff06c005fd0 tx=0x7ff06c005850 comp rx=0 tx=0).stop 2024-09-16T15:40:08.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.508+0000 7ff086800640 1 -- 172.21.15.151:0/3850750094 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0800ccfd0 msgr2=0x7ff080141250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:08.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.508+0000 7ff086800640 1 --2- 172.21.15.151:0/3850750094 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0800ccfd0 0x7ff080141250 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff07000b3f0 tx=0x7ff07000b8b0 comp rx=0 tx=0).stop 2024-09-16T15:40:08.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.508+0000 7ff086800640 1 -- 172.21.15.151:0/3850750094 shutdown_connections 2024-09-16T15:40:08.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.508+0000 7ff086800640 1 --2- 172.21.15.151:0/3850750094 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff060078850 0x7ff06007ad10 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:08.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.508+0000 7ff086800640 1 --2- 172.21.15.151:0/3850750094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0801445d0 0x7ff080142990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:08.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.508+0000 7ff086800640 1 --2- 172.21.15.151:0/3850750094 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff0800cf300 0x7ff08015f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:08.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.508+0000 7ff086800640 1 --2- 172.21.15.151:0/3850750094 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0800ccfd0 0x7ff080141250 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:08.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.508+0000 7ff086800640 1 -- 172.21.15.151:0/3850750094 >> 172.21.15.151:0/3850750094 conn(0x7ff0800b1950 msgr2=0x7ff0800b10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:08.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.508+0000 7ff086800640 1 -- 172.21.15.151:0/3850750094 shutdown_connections 2024-09-16T15:40:08.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.508+0000 7ff086800640 1 -- 172.21.15.151:0/3850750094 wait complete. 2024-09-16T15:40:08.527 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-09-16T15:40:08.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.684+0000 7f57a2a00640 1 -- 172.21.15.151:0/173173790 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f579c0ccf00 msgr2=0x7f579c0acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:08.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.684+0000 7f57a2a00640 1 --2- 172.21.15.151:0/173173790 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f579c0ccf00 0x7f579c0acd30 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5788009830 tx=0x7f578802a480 comp rx=0 tx=0).stop 2024-09-16T15:40:08.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.685+0000 7f57a2a00640 1 -- 172.21.15.151:0/173173790 shutdown_connections 2024-09-16T15:40:08.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.685+0000 7f57a2a00640 1 --2- 172.21.15.151:0/173173790 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f579c0ccf00 0x7f579c0acd30 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:08.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.685+0000 7f57a2a00640 1 --2- 172.21.15.151:0/173173790 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f579c15f5a0 0x7f579c0b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:08.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.685+0000 7f57a2a00640 1 -- 172.21.15.151:0/173173790 >> 172.21.15.151:0/173173790 conn(0x7f579c0b1950 msgr2=0x7f579c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:08.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.685+0000 7f57a2a00640 1 -- 172.21.15.151:0/173173790 shutdown_connections 2024-09-16T15:40:08.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.685+0000 7f57a2a00640 1 -- 172.21.15.151:0/173173790 wait complete. 2024-09-16T15:40:08.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.686+0000 7f57a2a00640 1 Processor -- start 2024-09-16T15:40:08.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.686+0000 7f57a2a00640 1 -- start start 2024-09-16T15:40:08.690 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.687+0000 7f57a2a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f579c0ad270 0x7f579c158c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:08.690 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.687+0000 7f57a2a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f579c0ccf00 0x7f579c1f7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:08.690 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.687+0000 7f57a2a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f579c15f5a0 0x7f579c0b4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:08.690 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.687+0000 7f57a2a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f579c06f2e0 con 0x7f579c15f5a0 2024-09-16T15:40:08.690 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.687+0000 7f57a2a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f579c06f160 con 0x7f579c0ad270 2024-09-16T15:40:08.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.687+0000 7f57a2a00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f579c06f460 con 0x7f579c0ccf00 2024-09-16T15:40:08.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.688+0000 7f57a0a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f579c15f5a0 0x7f579c0b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:08.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.688+0000 7f57a0a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f579c15f5a0 0x7f579c0b4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:38598/0 (socket says 172.21.15.151:38598) 2024-09-16T15:40:08.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.688+0000 7f57a0a00640 1 -- 172.21.15.151:0/1342092811 learned_addr learned my addr 172.21.15.151:0/1342092811 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:40:08.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.688+0000 7f579b400640 1 --2- 172.21.15.151:0/1342092811 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f579c0ccf00 0x7f579c1f7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:08.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.688+0000 7f579be00640 1 --2- 172.21.15.151:0/1342092811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f579c0ad270 0x7f579c158c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:08.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.688+0000 7f57a0a00640 1 -- 172.21.15.151:0/1342092811 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f579c0ccf00 msgr2=0x7f579c1f7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:08.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.688+0000 7f57a0a00640 1 --2- 172.21.15.151:0/1342092811 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f579c0ccf00 0x7f579c1f7e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:08.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.688+0000 7f57a0a00640 1 -- 172.21.15.151:0/1342092811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f579c0ad270 msgr2=0x7f579c158c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:08.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.688+0000 7f57a0a00640 1 --2- 172.21.15.151:0/1342092811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f579c0ad270 0x7f579c158c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:08.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.688+0000 7f57a0a00640 1 -- 172.21.15.151:0/1342092811 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f579c1f8690 con 0x7f579c15f5a0 2024-09-16T15:40:08.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.689+0000 7f57a0a00640 1 --2- 172.21.15.151:0/1342092811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f579c15f5a0 0x7f579c0b4320 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f579400caf0 tx=0x7f579400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:08.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.689+0000 7f5793e00640 1 -- 172.21.15.151:0/1342092811 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5794016020 con 0x7f579c15f5a0 2024-09-16T15:40:08.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.689+0000 7f5793e00640 1 -- 172.21.15.151:0/1342092811 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f579400ad90 con 0x7f579c15f5a0 2024-09-16T15:40:08.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.689+0000 7f5793e00640 1 -- 172.21.15.151:0/1342092811 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5794012680 con 0x7f579c15f5a0 2024-09-16T15:40:08.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.689+0000 7f57a2a00640 1 -- 172.21.15.151:0/1342092811 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f579c1f8970 con 0x7f579c15f5a0 2024-09-16T15:40:08.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.689+0000 7f57a2a00640 1 -- 172.21.15.151:0/1342092811 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f579c1f4850 con 0x7f579c15f5a0 2024-09-16T15:40:08.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.690+0000 7f57a2a00640 1 -- 172.21.15.151:0/1342092811 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f579c06f980 con 0x7f579c15f5a0 2024-09-16T15:40:08.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.691+0000 7f5793e00640 1 -- 172.21.15.151:0/1342092811 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f5794019020 con 0x7f579c15f5a0 2024-09-16T15:40:08.694 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.692+0000 7f5793e00640 1 --2- 172.21.15.151:0/1342092811 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f577c078850 0x7f577c07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:08.694 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.692+0000 7f5793e00640 1 -- 172.21.15.151:0/1342092811 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7f579409aaa0 con 0x7f579c15f5a0 2024-09-16T15:40:08.694 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.693+0000 7f579be00640 1 --2- 172.21.15.151:0/1342092811 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f577c078850 0x7f577c07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:08.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.693+0000 7f579be00640 1 --2- 172.21.15.151:0/1342092811 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f577c078850 0x7f577c07ad10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f579c1541e0 tx=0x7f578c00b2e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:08.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.696+0000 7f5793e00640 1 -- 172.21.15.151:0/1342092811 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f5794020020 con 0x7f579c15f5a0 2024-09-16T15:40:08.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.948+0000 7f57a2a00640 1 -- 172.21.15.151:0/1342092811 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f579c1f8b40 con 0x7f579c15f5a0 2024-09-16T15:40:08.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.951+0000 7f5793e00640 1 -- 172.21.15.151:0/1342092811 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v7) ==== 97+0+0 (secure 0 0 0) 0x7f5794062e10 con 0x7f579c15f5a0 2024-09-16T15:40:08.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.953+0000 7f57a2a00640 1 -- 172.21.15.151:0/1342092811 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f577c078850 msgr2=0x7f577c07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:08.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.953+0000 7f57a2a00640 1 --2- 172.21.15.151:0/1342092811 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f577c078850 0x7f577c07ad10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f579c1541e0 tx=0x7f578c00b2e0 comp rx=0 tx=0).stop 2024-09-16T15:40:08.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.953+0000 7f57a2a00640 1 -- 172.21.15.151:0/1342092811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f579c15f5a0 msgr2=0x7f579c0b4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:08.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.953+0000 7f57a2a00640 1 --2- 172.21.15.151:0/1342092811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f579c15f5a0 0x7f579c0b4320 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f579400caf0 tx=0x7f579400a060 comp rx=0 tx=0).stop 2024-09-16T15:40:08.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.954+0000 7f57a2a00640 1 -- 172.21.15.151:0/1342092811 shutdown_connections 2024-09-16T15:40:08.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.954+0000 7f57a2a00640 1 --2- 172.21.15.151:0/1342092811 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f577c078850 0x7f577c07ad10 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:08.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.954+0000 7f57a2a00640 1 --2- 172.21.15.151:0/1342092811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f579c15f5a0 0x7f579c0b4320 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:08.957 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.954+0000 7f57a2a00640 1 --2- 172.21.15.151:0/1342092811 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f579c0ccf00 0x7f579c1f7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:08.957 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.954+0000 7f57a2a00640 1 --2- 172.21.15.151:0/1342092811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f579c0ad270 0x7f579c158c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:08.957 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.954+0000 7f57a2a00640 1 -- 172.21.15.151:0/1342092811 >> 172.21.15.151:0/1342092811 conn(0x7f579c0b1950 msgr2=0x7f579c0b3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:08.957 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.954+0000 7f57a2a00640 1 -- 172.21.15.151:0/1342092811 shutdown_connections 2024-09-16T15:40:08.957 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:08.954+0000 7f57a2a00640 1 -- 172.21.15.151:0/1342092811 wait complete. 2024-09-16T15:40:08.973 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-09-16T15:40:09.131 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.129+0000 7f0703c00640 1 -- 172.21.15.151:0/3062063500 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f06fc15f5a0 msgr2=0x7f06fc0b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:09.131 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.129+0000 7f0703c00640 1 --2- 172.21.15.151:0/3062063500 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f06fc15f5a0 0x7f06fc0b2440 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f06fc0cfbb0 tx=0x7f06ec02a820 comp rx=0 tx=0).stop 2024-09-16T15:40:09.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.129+0000 7f0703c00640 1 -- 172.21.15.151:0/3062063500 shutdown_connections 2024-09-16T15:40:09.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.129+0000 7f0703c00640 1 --2- 172.21.15.151:0/3062063500 >> v2:172.21.15.167:3300/0 conn(0x7f06fc0ccf00 0x7f06fc0acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:09.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.129+0000 7f0703c00640 1 --2- 172.21.15.151:0/3062063500 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f06fc15f5a0 0x7f06fc0b2440 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:09.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.129+0000 7f0703c00640 1 -- 172.21.15.151:0/3062063500 >> 172.21.15.151:0/3062063500 conn(0x7f06fc0b1950 msgr2=0x7f06fc0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:09.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.129+0000 7f0703c00640 1 -- 172.21.15.151:0/3062063500 shutdown_connections 2024-09-16T15:40:09.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.129+0000 7f0703c00640 1 -- 172.21.15.151:0/3062063500 wait complete. 2024-09-16T15:40:09.133 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.131+0000 7f0703c00640 1 Processor -- start 2024-09-16T15:40:09.133 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.131+0000 7f0703c00640 1 -- start start 2024-09-16T15:40:09.135 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.132+0000 7f0703c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06fc0ad270 0x7f06fc0c4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:09.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.132+0000 7f0703c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f06fc0ccf00 0x7f06fc0c6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:09.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.132+0000 7f0703c00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f06fc15f5a0 0x7f06fc1f5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:09.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.132+0000 7f0703c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f06fc06f160 con 0x7f06fc0ccf00 2024-09-16T15:40:09.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.132+0000 7f0703c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f06fc06f2e0 con 0x7f06fc0ad270 2024-09-16T15:40:09.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.132+0000 7f0703c00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f06fc06f460 con 0x7f06fc15f5a0 2024-09-16T15:40:09.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.132+0000 7f06fbe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f06fc0ccf00 0x7f06fc0c6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:09.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.132+0000 7f06fbe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f06fc0ccf00 0x7f06fc0c6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:38628/0 (socket says 172.21.15.151:38628) 2024-09-16T15:40:09.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.132+0000 7f06fbe00640 1 -- 172.21.15.151:0/2550120907 learned_addr learned my addr 172.21.15.151:0/2550120907 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:40:09.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.132+0000 7f0701200640 1 --2- 172.21.15.151:0/2550120907 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06fc0ad270 0x7f06fc0c4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:09.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.132+0000 7f0701c00640 1 --2- 172.21.15.151:0/2550120907 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f06fc15f5a0 0x7f06fc1f5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:09.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.132+0000 7f06fbe00640 1 -- 172.21.15.151:0/2550120907 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f06fc15f5a0 msgr2=0x7f06fc1f5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:09.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.132+0000 7f06fbe00640 1 --2- 172.21.15.151:0/2550120907 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f06fc15f5a0 0x7f06fc1f5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:09.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.132+0000 7f06fbe00640 1 -- 172.21.15.151:0/2550120907 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06fc0ad270 msgr2=0x7f06fc0c4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:09.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.132+0000 7f06fbe00640 1 --2- 172.21.15.151:0/2550120907 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06fc0ad270 0x7f06fc0c4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:09.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.132+0000 7f06fbe00640 1 -- 172.21.15.151:0/2550120907 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06fc0c51a0 con 0x7f06fc0ccf00 2024-09-16T15:40:09.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.132+0000 7f0701c00640 1 --2- 172.21.15.151:0/2550120907 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f06fc15f5a0 0x7f06fc1f5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:40:09.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.133+0000 7f06fbe00640 1 --2- 172.21.15.151:0/2550120907 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f06fc0ccf00 0x7f06fc0c6240 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f06e800d980 tx=0x7f06e800de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:09.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.133+0000 7f06f8c00640 1 -- 172.21.15.151:0/2550120907 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e8017070 con 0x7f06fc0ccf00 2024-09-16T15:40:09.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.133+0000 7f06f8c00640 1 -- 172.21.15.151:0/2550120907 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f06e8009d20 con 0x7f06fc0ccf00 2024-09-16T15:40:09.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.133+0000 7f06f8c00640 1 -- 172.21.15.151:0/2550120907 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e8004280 con 0x7f06fc0ccf00 2024-09-16T15:40:09.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.133+0000 7f0703c00640 1 -- 172.21.15.151:0/2550120907 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06fc0c5480 con 0x7f06fc0ccf00 2024-09-16T15:40:09.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.133+0000 7f0703c00640 1 -- 172.21.15.151:0/2550120907 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06fc0c5a40 con 0x7f06fc0ccf00 2024-09-16T15:40:09.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.134+0000 7f0703c00640 1 -- 172.21.15.151:0/2550120907 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06fc06f9f0 con 0x7f06fc0ccf00 2024-09-16T15:40:09.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.136+0000 7f06f8c00640 1 -- 172.21.15.151:0/2550120907 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f06e80035f0 con 0x7f06fc0ccf00 2024-09-16T15:40:09.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.136+0000 7f06f8c00640 1 --2- 172.21.15.151:0/2550120907 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f06dc078850 0x7f06dc07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:09.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.137+0000 7f06f8c00640 1 -- 172.21.15.151:0/2550120907 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7f06e809a660 con 0x7f06fc0ccf00 2024-09-16T15:40:09.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.137+0000 7f0701200640 1 --2- 172.21.15.151:0/2550120907 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f06dc078850 0x7f06dc07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:09.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.138+0000 7f0701200640 1 --2- 172.21.15.151:0/2550120907 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f06dc078850 0x7f06dc07ad10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f06ec034000 tx=0x7f06ec004ab0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:09.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.140+0000 7f06f8c00640 1 -- 172.21.15.151:0/2550120907 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f06e80629c0 con 0x7f06fc0ccf00 2024-09-16T15:40:09.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.394+0000 7f0703c00640 1 -- 172.21.15.151:0/2550120907 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f06fc153dc0 con 0x7f06fc0ccf00 2024-09-16T15:40:09.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.397+0000 7f06f8c00640 1 -- 172.21.15.151:0/2550120907 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7f06e80673f0 con 0x7f06fc0ccf00 2024-09-16T15:40:09.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.399+0000 7f0703c00640 1 -- 172.21.15.151:0/2550120907 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f06dc078850 msgr2=0x7f06dc07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:09.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.399+0000 7f0703c00640 1 --2- 172.21.15.151:0/2550120907 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f06dc078850 0x7f06dc07ad10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f06ec034000 tx=0x7f06ec004ab0 comp rx=0 tx=0).stop 2024-09-16T15:40:09.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.399+0000 7f0703c00640 1 -- 172.21.15.151:0/2550120907 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f06fc0ccf00 msgr2=0x7f06fc0c6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:09.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.399+0000 7f0703c00640 1 --2- 172.21.15.151:0/2550120907 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f06fc0ccf00 0x7f06fc0c6240 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f06e800d980 tx=0x7f06e800de40 comp rx=0 tx=0).stop 2024-09-16T15:40:09.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.399+0000 7f0703c00640 1 -- 172.21.15.151:0/2550120907 shutdown_connections 2024-09-16T15:40:09.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.399+0000 7f0703c00640 1 --2- 172.21.15.151:0/2550120907 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f06dc078850 0x7f06dc07ad10 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:09.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.399+0000 7f0703c00640 1 --2- 172.21.15.151:0/2550120907 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f06fc15f5a0 0x7f06fc1f5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:09.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.399+0000 7f0703c00640 1 --2- 172.21.15.151:0/2550120907 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f06fc0ccf00 0x7f06fc0c6240 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:09.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.399+0000 7f0703c00640 1 --2- 172.21.15.151:0/2550120907 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06fc0ad270 0x7f06fc0c4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:09.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.399+0000 7f0703c00640 1 -- 172.21.15.151:0/2550120907 >> 172.21.15.151:0/2550120907 conn(0x7f06fc0b1950 msgr2=0x7f06fc0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:09.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.399+0000 7f0703c00640 1 -- 172.21.15.151:0/2550120907 shutdown_connections 2024-09-16T15:40:09.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.400+0000 7f0703c00640 1 -- 172.21.15.151:0/2550120907 wait complete. 2024-09-16T15:40:09.418 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-16T15:40:09.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.576+0000 7fa1ab400640 1 -- 172.21.15.151:0/2614655580 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa1a40ccf00 msgr2=0x7fa1a40b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:09.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.576+0000 7fa1ab400640 1 --2- 172.21.15.151:0/2614655580 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa1a40ccf00 0x7fa1a40b2440 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa190005fd0 tx=0x7fa19002a540 comp rx=0 tx=0).stop 2024-09-16T15:40:09.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.576+0000 7fa1ab400640 1 -- 172.21.15.151:0/2614655580 shutdown_connections 2024-09-16T15:40:09.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.576+0000 7fa1ab400640 1 --2- 172.21.15.151:0/2614655580 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa1a4073fb0 0x7fa1a4074440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:09.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.576+0000 7fa1ab400640 1 --2- 172.21.15.151:0/2614655580 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa1a40ccf00 0x7fa1a40b2440 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:09.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.576+0000 7fa1ab400640 1 -- 172.21.15.151:0/2614655580 >> 172.21.15.151:0/2614655580 conn(0x7fa1a40b1950 msgr2=0x7fa1a40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:09.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.576+0000 7fa1ab400640 1 -- 172.21.15.151:0/2614655580 shutdown_connections 2024-09-16T15:40:09.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.576+0000 7fa1ab400640 1 -- 172.21.15.151:0/2614655580 wait complete. 2024-09-16T15:40:09.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.577+0000 7fa1ab400640 1 Processor -- start 2024-09-16T15:40:09.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.578+0000 7fa1ab400640 1 -- start start 2024-09-16T15:40:09.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.578+0000 7fa1ab400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa1a4073fb0 0x7fa1a40d0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:09.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.578+0000 7fa1ab400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa1a40ccf00 0x7fa1a4151f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:09.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.578+0000 7fa1ab400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1a415f5a0 0x7fa1a4158260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:09.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.578+0000 7fa1ab400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a406f2e0 con 0x7fa1a4073fb0 2024-09-16T15:40:09.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.578+0000 7fa1ab400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a406f160 con 0x7fa1a415f5a0 2024-09-16T15:40:09.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.578+0000 7fa1ab400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fa1a406f460 con 0x7fa1a40ccf00 2024-09-16T15:40:09.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.579+0000 7fa1a8a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa1a4073fb0 0x7fa1a40d0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:09.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.579+0000 7fa1a8a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa1a4073fb0 0x7fa1a40d0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:38640/0 (socket says 172.21.15.151:38640) 2024-09-16T15:40:09.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.579+0000 7fa1a8a00640 1 -- 172.21.15.151:0/1429588940 learned_addr learned my addr 172.21.15.151:0/1429588940 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:40:09.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.579+0000 7fa1a9400640 1 --2- 172.21.15.151:0/1429588940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1a415f5a0 0x7fa1a4158260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:09.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.579+0000 7fa1a3e00640 1 --2- 172.21.15.151:0/1429588940 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa1a40ccf00 0x7fa1a4151f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:09.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.579+0000 7fa1a8a00640 1 -- 172.21.15.151:0/1429588940 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa1a40ccf00 msgr2=0x7fa1a4151f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:09.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.579+0000 7fa1a8a00640 1 --2- 172.21.15.151:0/1429588940 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa1a40ccf00 0x7fa1a4151f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:09.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.579+0000 7fa1a8a00640 1 -- 172.21.15.151:0/1429588940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1a415f5a0 msgr2=0x7fa1a4158260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:09.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.579+0000 7fa1a8a00640 1 --2- 172.21.15.151:0/1429588940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1a415f5a0 0x7fa1a4158260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:09.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.579+0000 7fa1a8a00640 1 -- 172.21.15.151:0/1429588940 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a41f4bc0 con 0x7fa1a4073fb0 2024-09-16T15:40:09.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.579+0000 7fa1a3e00640 1 --2- 172.21.15.151:0/1429588940 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa1a40ccf00 0x7fa1a4151f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:40:09.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.579+0000 7fa1a9400640 1 --2- 172.21.15.151:0/1429588940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1a415f5a0 0x7fa1a4158260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:40:09.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.579+0000 7fa1a8a00640 1 --2- 172.21.15.151:0/1429588940 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa1a4073fb0 0x7fa1a40d0480 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa19400c730 tx=0x7fa19400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:09.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.580+0000 7fa1a0c00640 1 -- 172.21.15.151:0/1429588940 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa194003b70 con 0x7fa1a4073fb0 2024-09-16T15:40:09.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.580+0000 7fa1a0c00640 1 -- 172.21.15.151:0/1429588940 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa194003d30 con 0x7fa1a4073fb0 2024-09-16T15:40:09.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.580+0000 7fa1a0c00640 1 -- 172.21.15.151:0/1429588940 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19400aab0 con 0x7fa1a4073fb0 2024-09-16T15:40:09.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.580+0000 7fa1ab400640 1 -- 172.21.15.151:0/1429588940 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1a41f4ea0 con 0x7fa1a4073fb0 2024-09-16T15:40:09.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.580+0000 7fa1ab400640 1 -- 172.21.15.151:0/1429588940 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1a41f5460 con 0x7fa1a4073fb0 2024-09-16T15:40:09.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.581+0000 7fa1ab400640 1 -- 172.21.15.151:0/1429588940 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1a406f980 con 0x7fa1a4073fb0 2024-09-16T15:40:09.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.582+0000 7fa1a0c00640 1 -- 172.21.15.151:0/1429588940 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fa19400ac50 con 0x7fa1a4073fb0 2024-09-16T15:40:09.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.583+0000 7fa1a0c00640 1 --2- 172.21.15.151:0/1429588940 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa184078850 0x7fa18407ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:09.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.583+0000 7fa1a0c00640 1 -- 172.21.15.151:0/1429588940 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7fa19409a3f0 con 0x7fa1a4073fb0 2024-09-16T15:40:09.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.583+0000 7fa1a3e00640 1 --2- 172.21.15.151:0/1429588940 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa184078850 0x7fa18407ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:09.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.584+0000 7fa1a3e00640 1 --2- 172.21.15.151:0/1429588940 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa184078850 0x7fa18407ad10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa190033000 tx=0x7fa190004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:09.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.587+0000 7fa1a0c00640 1 -- 172.21.15.151:0/1429588940 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa194062da0 con 0x7fa1a4073fb0 2024-09-16T15:40:09.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.775+0000 7fa1ab400640 1 -- 172.21.15.151:0/1429588940 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fa1a4159180 con 0x7fa1a4073fb0 2024-09-16T15:40:09.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.775+0000 7fa1a0c00640 1 -- 172.21.15.151:0/1429588940 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v31) ==== 84+0+65 (secure 0 0 0) 0x7fa1940626a0 con 0x7fa1a4073fb0 2024-09-16T15:40:09.778 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:40:09.778 INFO:teuthology.orchestra.run.smithi151.stdout:[ 2024-09-16T15:40:09.778 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:40:09.778 INFO:teuthology.orchestra.run.smithi151.stdout: "poolnum": 1, 2024-09-16T15:40:09.778 INFO:teuthology.orchestra.run.smithi151.stdout: "poolname": ".mgr" 2024-09-16T15:40:09.778 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:40:09.778 INFO:teuthology.orchestra.run.smithi151.stdout:] 2024-09-16T15:40:09.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.778+0000 7fa1ab400640 1 -- 172.21.15.151:0/1429588940 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa184078850 msgr2=0x7fa18407ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:09.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.778+0000 7fa1ab400640 1 --2- 172.21.15.151:0/1429588940 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa184078850 0x7fa18407ad10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa190033000 tx=0x7fa190004730 comp rx=0 tx=0).stop 2024-09-16T15:40:09.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.778+0000 7fa1ab400640 1 -- 172.21.15.151:0/1429588940 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa1a4073fb0 msgr2=0x7fa1a40d0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:09.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.778+0000 7fa1ab400640 1 --2- 172.21.15.151:0/1429588940 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa1a4073fb0 0x7fa1a40d0480 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa19400c730 tx=0x7fa19400cbf0 comp rx=0 tx=0).stop 2024-09-16T15:40:09.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.778+0000 7fa1ab400640 1 -- 172.21.15.151:0/1429588940 shutdown_connections 2024-09-16T15:40:09.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.778+0000 7fa1ab400640 1 --2- 172.21.15.151:0/1429588940 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa184078850 0x7fa18407ad10 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:09.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.778+0000 7fa1ab400640 1 --2- 172.21.15.151:0/1429588940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1a415f5a0 0x7fa1a4158260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:09.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.778+0000 7fa1ab400640 1 --2- 172.21.15.151:0/1429588940 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa1a40ccf00 0x7fa1a4151f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:09.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.778+0000 7fa1ab400640 1 --2- 172.21.15.151:0/1429588940 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa1a4073fb0 0x7fa1a40d0480 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:09.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.778+0000 7fa1ab400640 1 -- 172.21.15.151:0/1429588940 >> 172.21.15.151:0/1429588940 conn(0x7fa1a40b1950 msgr2=0x7fa1a40b10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:09.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.778+0000 7fa1ab400640 1 -- 172.21.15.151:0/1429588940 shutdown_connections 2024-09-16T15:40:09.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.778+0000 7fa1ab400640 1 -- 172.21.15.151:0/1429588940 wait complete. 2024-09-16T15:40:09.796 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-16T15:40:09.796 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-16T15:40:09.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.956+0000 7fb903200640 1 -- 172.21.15.151:0/3263828944 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb8e8005230 msgr2=0x7fb8e800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:09.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.956+0000 7fb903200640 1 --2- 172.21.15.151:0/3263828944 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb8e8005230 0x7fb8e800b9c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc009ed0 tx=0x7fb8e8005830 comp rx=0 tx=0).stop 2024-09-16T15:40:09.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.956+0000 7fb903200640 1 -- 172.21.15.151:0/3263828944 shutdown_connections 2024-09-16T15:40:09.959 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.956+0000 7fb903200640 1 --2- 172.21.15.151:0/3263828944 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb8e8005230 0x7fb8e800b9c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:09.959 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.956+0000 7fb903200640 1 -- 172.21.15.151:0/3263828944 >> 172.21.15.151:0/3263828944 conn(0x7fb9040cfa70 msgr2=0x7fb9040cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:09.959 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.956+0000 7fb903200640 1 -- 172.21.15.151:0/3263828944 shutdown_connections 2024-09-16T15:40:09.959 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.956+0000 7fb903200640 1 -- 172.21.15.151:0/3263828944 wait complete. 2024-09-16T15:40:09.959 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.957+0000 7fb903200640 1 Processor -- start 2024-09-16T15:40:09.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.958+0000 7fb903200640 1 -- start start 2024-09-16T15:40:09.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.958+0000 7fb903200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8e8005230 0x7fb904156170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:09.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.959+0000 7fb903200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb90415c390 0x7fb90415ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:09.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.959+0000 7fb903200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb9040c5870 0x7fb9040c5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:09.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.959+0000 7fb903200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb904070280 con 0x7fb90415c390 2024-09-16T15:40:09.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.959+0000 7fb903200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb904070100 con 0x7fb8e8005230 2024-09-16T15:40:09.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.959+0000 7fb903200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fb904073010 con 0x7fb9040c5870 2024-09-16T15:40:09.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.959+0000 7fb901400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb90415c390 0x7fb90415ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:09.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.959+0000 7fb901400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb90415c390 0x7fb90415ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:38646/0 (socket says 172.21.15.151:38646) 2024-09-16T15:40:09.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.959+0000 7fb901400640 1 -- 172.21.15.151:0/1569617077 learned_addr learned my addr 172.21.15.151:0/1569617077 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:40:09.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.959+0000 7fb901e00640 1 --2- 172.21.15.151:0/1569617077 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8e8005230 0x7fb904156170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:09.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.959+0000 7fb902800640 1 --2- 172.21.15.151:0/1569617077 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb9040c5870 0x7fb9040c5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:09.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.959+0000 7fb901400640 1 -- 172.21.15.151:0/1569617077 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb9040c5870 msgr2=0x7fb9040c5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:09.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.959+0000 7fb901400640 1 --2- 172.21.15.151:0/1569617077 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb9040c5870 0x7fb9040c5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:09.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.959+0000 7fb901400640 1 -- 172.21.15.151:0/1569617077 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8e8005230 msgr2=0x7fb904156170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:09.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.959+0000 7fb901400640 1 --2- 172.21.15.151:0/1569617077 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8e8005230 0x7fb904156170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:09.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.959+0000 7fb901400640 1 -- 172.21.15.151:0/1569617077 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9041f23e0 con 0x7fb90415c390 2024-09-16T15:40:09.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.959+0000 7fb901400640 1 --2- 172.21.15.151:0/1569617077 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb90415c390 0x7fb90415ca10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb8f0006450 tx=0x7fb8f00039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:09.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.960+0000 7fb8fa000640 1 -- 172.21.15.151:0/1569617077 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f0015070 con 0x7fb90415c390 2024-09-16T15:40:09.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.960+0000 7fb8fa000640 1 -- 172.21.15.151:0/1569617077 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb8f0011840 con 0x7fb90415c390 2024-09-16T15:40:09.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.960+0000 7fb903200640 1 -- 172.21.15.151:0/1569617077 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9041f26b0 con 0x7fb90415c390 2024-09-16T15:40:09.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.960+0000 7fb8fa000640 1 -- 172.21.15.151:0/1569617077 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f0011c90 con 0x7fb90415c390 2024-09-16T15:40:09.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.960+0000 7fb903200640 1 -- 172.21.15.151:0/1569617077 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9041f2c70 con 0x7fb90415c390 2024-09-16T15:40:09.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.961+0000 7fb903200640 1 -- 172.21.15.151:0/1569617077 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb904073500 con 0x7fb90415c390 2024-09-16T15:40:09.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.962+0000 7fb8fa000640 1 -- 172.21.15.151:0/1569617077 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fb8f0018070 con 0x7fb90415c390 2024-09-16T15:40:09.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.963+0000 7fb8fa000640 1 --2- 172.21.15.151:0/1569617077 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb8e4078850 0x7fb8e407ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:09.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.963+0000 7fb8fa000640 1 -- 172.21.15.151:0/1569617077 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7fb8f009b2d0 con 0x7fb90415c390 2024-09-16T15:40:09.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.964+0000 7fb901e00640 1 --2- 172.21.15.151:0/1569617077 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb8e4078850 0x7fb8e407ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:09.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.965+0000 7fb901e00640 1 --2- 172.21.15.151:0/1569617077 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb8e4078850 0x7fb8e407ad10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4000c00 tx=0x7fb8f4006b10 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:09.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:09.967+0000 7fb8fa000640 1 -- 172.21.15.151:0/1569617077 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb8f0063ca0 con 0x7fb90415c390 2024-09-16T15:40:10.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:10.153+0000 7fb903200640 1 -- 172.21.15.151:0/1569617077 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fb9040b4c80 con 0x7fb90415c390 2024-09-16T15:40:10.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:10.402+0000 7fb8fa000640 1 -- 172.21.15.151:0/1569617077 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v32) ==== 138+0+0 (secure 0 0 0) 0x7fb8f00635a0 con 0x7fb90415c390 2024-09-16T15:40:10.404 INFO:teuthology.orchestra.run.smithi151.stderr:pool 'cephfs_metadata' created 2024-09-16T15:40:10.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:10.405+0000 7fb903200640 1 -- 172.21.15.151:0/1569617077 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb8e4078850 msgr2=0x7fb8e407ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:10.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:10.405+0000 7fb903200640 1 --2- 172.21.15.151:0/1569617077 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb8e4078850 0x7fb8e407ad10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4000c00 tx=0x7fb8f4006b10 comp rx=0 tx=0).stop 2024-09-16T15:40:10.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:10.405+0000 7fb903200640 1 -- 172.21.15.151:0/1569617077 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb90415c390 msgr2=0x7fb90415ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:10.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:10.405+0000 7fb903200640 1 --2- 172.21.15.151:0/1569617077 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb90415c390 0x7fb90415ca10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb8f0006450 tx=0x7fb8f00039c0 comp rx=0 tx=0).stop 2024-09-16T15:40:10.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:10.405+0000 7fb903200640 1 -- 172.21.15.151:0/1569617077 shutdown_connections 2024-09-16T15:40:10.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:10.405+0000 7fb903200640 1 --2- 172.21.15.151:0/1569617077 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb8e4078850 0x7fb8e407ad10 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:10.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:10.405+0000 7fb903200640 1 --2- 172.21.15.151:0/1569617077 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb9040c5870 0x7fb9040c5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:10.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:10.405+0000 7fb903200640 1 --2- 172.21.15.151:0/1569617077 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb90415c390 0x7fb90415ca10 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:10.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:10.405+0000 7fb903200640 1 --2- 172.21.15.151:0/1569617077 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8e8005230 0x7fb904156170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:10.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:10.405+0000 7fb903200640 1 -- 172.21.15.151:0/1569617077 >> 172.21.15.151:0/1569617077 conn(0x7fb9040cfa70 msgr2=0x7fb9040b3af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:10.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:10.405+0000 7fb903200640 1 -- 172.21.15.151:0/1569617077 shutdown_connections 2024-09-16T15:40:10.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:10.405+0000 7fb903200640 1 -- 172.21.15.151:0/1569617077 wait complete. 2024-09-16T15:40:10.426 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-16T15:40:10.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:10.591+0000 7f914be00640 1 -- 172.21.15.151:0/1053395508 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f914c0c5730 msgr2=0x7f914c15fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:10.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:10.591+0000 7f914be00640 1 --2- 172.21.15.151:0/1053395508 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f914c0c5730 0x7f914c15fa50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9134009830 tx=0x7f913402a510 comp rx=0 tx=0).stop 2024-09-16T15:40:10.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:10.592+0000 7f914be00640 1 -- 172.21.15.151:0/1053395508 shutdown_connections 2024-09-16T15:40:10.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:10.592+0000 7f914be00640 1 --2- 172.21.15.151:0/1053395508 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f914c0c5730 0x7f914c15fa50 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:10.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:10.592+0000 7f914be00640 1 --2- 172.21.15.151:0/1053395508 >> v2:172.21.15.167:3300/0 conn(0x7f914c15f670 0x7f914c0c6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:10.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:10.592+0000 7f914be00640 1 -- 172.21.15.151:0/1053395508 >> 172.21.15.151:0/1053395508 conn(0x7f914c0cc490 msgr2=0x7f914c0cc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:10.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:10.592+0000 7f914be00640 1 -- 172.21.15.151:0/1053395508 shutdown_connections 2024-09-16T15:40:10.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:10.592+0000 7f914be00640 1 -- 172.21.15.151:0/1053395508 wait complete. 2024-09-16T15:40:10.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:10.593+0000 7f914be00640 1 Processor -- start 2024-09-16T15:40:10.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:10.593+0000 7f914be00640 1 -- start start 2024-09-16T15:40:10.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:10.594+0000 7f914be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f914c0c5730 0x7f914c14b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:10.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:10.594+0000 7f914be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f914c15c0f0 0x7f914c0c9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:10.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:10.594+0000 7f914be00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f914c15f670 0x7f914c0c95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:10.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:10.594+0000 7f914be00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f914c0ca370 con 0x7f914c0c5730 2024-09-16T15:40:10.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:10.594+0000 7f914be00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f914c0700d0 con 0x7f914c15c0f0 2024-09-16T15:40:10.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:10.594+0000 7f914be00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f914c0ca4f0 con 0x7f914c15f670 2024-09-16T15:40:10.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:10.595+0000 7f914aa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f914c0c5730 0x7f914c14b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:10.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:10.595+0000 7f914aa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f914c0c5730 0x7f914c14b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:38662/0 (socket says 172.21.15.151:38662) 2024-09-16T15:40:10.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:10.595+0000 7f914aa00640 1 -- 172.21.15.151:0/855281563 learned_addr learned my addr 172.21.15.151:0/855281563 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:40:10.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:10.595+0000 7f914b400640 1 --2- 172.21.15.151:0/855281563 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f914c15f670 0x7f914c0c95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:10.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:10.595+0000 7f9143e00640 1 --2- 172.21.15.151:0/855281563 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f914c15c0f0 0x7f914c0c9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:10.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:10.595+0000 7f914aa00640 1 -- 172.21.15.151:0/855281563 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f914c15f670 msgr2=0x7f914c0c95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:10.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:10.595+0000 7f914aa00640 1 --2- 172.21.15.151:0/855281563 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f914c15f670 0x7f914c0c95a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:10.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:10.595+0000 7f914aa00640 1 -- 172.21.15.151:0/855281563 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f914c15c0f0 msgr2=0x7f914c0c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:10.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:10.595+0000 7f914aa00640 1 --2- 172.21.15.151:0/855281563 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f914c15c0f0 0x7f914c0c9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:10.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:10.595+0000 7f914aa00640 1 -- 172.21.15.151:0/855281563 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f914c14b980 con 0x7f914c0c5730 2024-09-16T15:40:10.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:10.595+0000 7f914aa00640 1 --2- 172.21.15.151:0/855281563 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f914c0c5730 0x7f914c14b440 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f913c00e9f0 tx=0x7f913c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:10.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:10.595+0000 7f9142a00640 1 -- 172.21.15.151:0/855281563 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f913c00cd80 con 0x7f914c0c5730 2024-09-16T15:40:10.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:10.596+0000 7f9142a00640 1 -- 172.21.15.151:0/855281563 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f913c004020 con 0x7f914c0c5730 2024-09-16T15:40:10.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:10.596+0000 7f9142a00640 1 -- 172.21.15.151:0/855281563 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f913c004470 con 0x7f914c0c5730 2024-09-16T15:40:10.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:10.596+0000 7f914be00640 1 -- 172.21.15.151:0/855281563 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f914c0c7820 con 0x7f914c0c5730 2024-09-16T15:40:10.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:10.596+0000 7f914be00640 1 -- 172.21.15.151:0/855281563 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f914c0cdb30 con 0x7f914c0c5730 2024-09-16T15:40:10.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:10.597+0000 7f914be00640 1 -- 172.21.15.151:0/855281563 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f914c15ce90 con 0x7f914c0c5730 2024-09-16T15:40:10.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:10.598+0000 7f9142a00640 1 -- 172.21.15.151:0/855281563 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f913c0035f0 con 0x7f914c0c5730 2024-09-16T15:40:10.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:10.599+0000 7f9142a00640 1 --2- 172.21.15.151:0/855281563 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9128078850 0x7f912807ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:10.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:10.599+0000 7f9142a00640 1 -- 172.21.15.151:0/855281563 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5227+0+0 (secure 0 0 0) 0x7f913c09b270 con 0x7f914c0c5730 2024-09-16T15:40:10.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:10.600+0000 7f9143e00640 1 --2- 172.21.15.151:0/855281563 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9128078850 0x7f912807ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:10.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:10.600+0000 7f9143e00640 1 --2- 172.21.15.151:0/855281563 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9128078850 0x7f912807ad10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9134034000 tx=0x7f9134004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:10.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:10.603+0000 7f9142a00640 1 -- 172.21.15.151:0/855281563 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f913c063b30 con 0x7f914c0c5730 2024-09-16T15:40:10.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:10.788+0000 7f914be00640 1 -- 172.21.15.151:0/855281563 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f914c072bc0 con 0x7f914c0c5730 2024-09-16T15:40:11.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:11.405+0000 7f9142a00640 1 -- 172.21.15.151:0/855281563 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v33) ==== 170+0+0 (secure 0 0 0) 0x7f913c063430 con 0x7f914c0c5730 2024-09-16T15:40:11.407 INFO:teuthology.orchestra.run.smithi151.stderr:pool 'cephfs_data' created 2024-09-16T15:40:11.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:11.408+0000 7f914be00640 1 -- 172.21.15.151:0/855281563 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9128078850 msgr2=0x7f912807ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:11.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:11.408+0000 7f914be00640 1 --2- 172.21.15.151:0/855281563 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9128078850 0x7f912807ad10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9134034000 tx=0x7f9134004730 comp rx=0 tx=0).stop 2024-09-16T15:40:11.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:11.408+0000 7f914be00640 1 -- 172.21.15.151:0/855281563 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f914c0c5730 msgr2=0x7f914c14b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:11.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:11.408+0000 7f914be00640 1 --2- 172.21.15.151:0/855281563 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f914c0c5730 0x7f914c14b440 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f913c00e9f0 tx=0x7f913c00eeb0 comp rx=0 tx=0).stop 2024-09-16T15:40:11.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:11.408+0000 7f914be00640 1 -- 172.21.15.151:0/855281563 shutdown_connections 2024-09-16T15:40:11.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:11.408+0000 7f914be00640 1 --2- 172.21.15.151:0/855281563 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9128078850 0x7f912807ad10 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:11.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:11.408+0000 7f914be00640 1 --2- 172.21.15.151:0/855281563 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f914c15c0f0 0x7f914c0c9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:11.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:11.408+0000 7f914be00640 1 --2- 172.21.15.151:0/855281563 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f914c15f670 0x7f914c0c95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:11.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:11.408+0000 7f914be00640 1 --2- 172.21.15.151:0/855281563 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f914c0c5730 0x7f914c14b440 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:11.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:11.408+0000 7f914be00640 1 -- 172.21.15.151:0/855281563 >> 172.21.15.151:0/855281563 conn(0x7f914c0cc490 msgr2=0x7f914c0739a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:11.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:11.408+0000 7f914be00640 1 -- 172.21.15.151:0/855281563 shutdown_connections 2024-09-16T15:40:11.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:11.408+0000 7f914be00640 1 -- 172.21.15.151:0/855281563 wait complete. 2024-09-16T15:40:11.439 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-16T15:40:11.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:11.600+0000 7f97f6800640 1 -- 172.21.15.151:0/3272847514 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f97d4004800 msgr2=0x7f97d4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:11.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:11.600+0000 7f97f6800640 1 --2- 172.21.15.151:0/3272847514 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f97d4004800 0x7f97d4004c80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f97dc00c290 tx=0x7f97d4005830 comp rx=0 tx=0).stop 2024-09-16T15:40:11.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:11.600+0000 7f97efe00640 1 -- 172.21.15.151:0/3272847514 reap_dead start 2024-09-16T15:40:11.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:11.600+0000 7f97f6800640 1 -- 172.21.15.151:0/3272847514 shutdown_connections 2024-09-16T15:40:11.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:11.600+0000 7f97f6800640 1 -- 172.21.15.151:0/3272847514 >> 172.21.15.151:0/3272847514 conn(0x7f97f00b1840 msgr2=0x7f97f00b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:11.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:11.600+0000 7f97f6800640 1 -- 172.21.15.151:0/3272847514 shutdown_connections 2024-09-16T15:40:11.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:11.600+0000 7f97f6800640 1 -- 172.21.15.151:0/3272847514 wait complete. 2024-09-16T15:40:11.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:11.602+0000 7f97f6800640 1 Processor -- start 2024-09-16T15:40:11.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:11.602+0000 7f97f6800640 1 -- start start 2024-09-16T15:40:11.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:11.603+0000 7f97f6800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97d4004800 0x7f97f01f27f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:11.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:11.603+0000 7f97f6800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97f0142b10 0x7f97f00ab890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:11.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:11.603+0000 7f97f6800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f97f00aa8f0 0x7f97f013ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:11.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:11.603+0000 7f97f6800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f97f00cdf40 con 0x7f97f00aa8f0 2024-09-16T15:40:11.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:11.603+0000 7f97f6800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f97f00cddc0 con 0x7f97d4004800 2024-09-16T15:40:11.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:11.603+0000 7f97f6800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f97f00ce0c0 con 0x7f97f0142b10 2024-09-16T15:40:11.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:11.603+0000 7f97efe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f97f00aa8f0 0x7f97f013ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:11.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:11.603+0000 7f97eea00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97f0142b10 0x7f97f00ab890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:11.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:11.603+0000 7f97efe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f97f00aa8f0 0x7f97f013ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:38674/0 (socket says 172.21.15.151:38674) 2024-09-16T15:40:11.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:11.603+0000 7f97ef400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97d4004800 0x7f97f01f27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:11.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:11.603+0000 7f97efe00640 1 -- 172.21.15.151:0/95278611 learned_addr learned my addr 172.21.15.151:0/95278611 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:40:11.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:11.604+0000 7f97efe00640 1 -- 172.21.15.151:0/95278611 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97f0142b10 msgr2=0x7f97f00ab890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:11.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:11.604+0000 7f97efe00640 1 --2- 172.21.15.151:0/95278611 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97f0142b10 0x7f97f00ab890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:11.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:11.604+0000 7f97efe00640 1 -- 172.21.15.151:0/95278611 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97d4004800 msgr2=0x7f97f01f27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:11.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:11.604+0000 7f97efe00640 1 --2- 172.21.15.151:0/95278611 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97d4004800 0x7f97f01f27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:11.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:11.604+0000 7f97efe00640 1 -- 172.21.15.151:0/95278611 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97f01f2f60 con 0x7f97f00aa8f0 2024-09-16T15:40:11.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:11.604+0000 7f97efe00640 1 --2- 172.21.15.151:0/95278611 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f97f00aa8f0 0x7f97f013ed40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f97f00cc550 tx=0x7f97e8011b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:11.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:11.604+0000 7f97e7400640 1 -- 172.21.15.151:0/95278611 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e801f070 con 0x7f97f00aa8f0 2024-09-16T15:40:11.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:11.604+0000 7f97e7400640 1 -- 172.21.15.151:0/95278611 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f97e800fb70 con 0x7f97f00aa8f0 2024-09-16T15:40:11.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:11.604+0000 7f97f6800640 1 -- 172.21.15.151:0/95278611 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97f01f3240 con 0x7f97f00aa8f0 2024-09-16T15:40:11.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:11.604+0000 7f97e7400640 1 -- 172.21.15.151:0/95278611 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e8012450 con 0x7f97f00aa8f0 2024-09-16T15:40:11.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:11.604+0000 7f97f6800640 1 -- 172.21.15.151:0/95278611 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97f00abdd0 con 0x7f97f00aa8f0 2024-09-16T15:40:11.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:11.606+0000 7f97f6800640 1 -- 172.21.15.151:0/95278611 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97f01f3760 con 0x7f97f00aa8f0 2024-09-16T15:40:11.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:11.607+0000 7f97e7400640 1 -- 172.21.15.151:0/95278611 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f97e8022070 con 0x7f97f00aa8f0 2024-09-16T15:40:11.610 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:11.608+0000 7f97e7400640 1 --2- 172.21.15.151:0/95278611 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f97d0078850 0x7f97d007ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:11.610 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:11.608+0000 7f97e7400640 1 -- 172.21.15.151:0/95278611 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5609+0+0 (secure 0 0 0) 0x7f97e809b250 con 0x7f97f00aa8f0 2024-09-16T15:40:11.610 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:11.608+0000 7f97ef400640 1 --2- 172.21.15.151:0/95278611 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f97d0078850 0x7f97d007ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:11.611 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:11.609+0000 7f97ef400640 1 --2- 172.21.15.151:0/95278611 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f97d0078850 0x7f97d007ad10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f97f00cbdf0 tx=0x7f97e0009500 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:11.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:11.613+0000 7f97e7400640 1 -- 172.21.15.151:0/95278611 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f97f01f3760 con 0x7f97f00aa8f0 2024-09-16T15:40:11.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:11.834+0000 7f97f6800640 1 -- 172.21.15.151:0/95278611 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f97f013e680 con 0x7f97f00aa8f0 2024-09-16T15:40:12.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:12.426+0000 7f97e7400640 1 -- 172.21.15.151:0/95278611 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) ==== 172+0+0 (secure 0 0 0) 0x7f97e8063240 con 0x7f97f00aa8f0 2024-09-16T15:40:12.431 INFO:teuthology.orchestra.run.smithi151.stderr:new fs with metadata pool 4 and data pool 5 2024-09-16T15:40:12.439 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:12.436+0000 7f97f6800640 1 -- 172.21.15.151:0/95278611 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f97d0078850 msgr2=0x7f97d007ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:12.439 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:12.436+0000 7f97f6800640 1 --2- 172.21.15.151:0/95278611 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f97d0078850 0x7f97d007ad10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f97f00cbdf0 tx=0x7f97e0009500 comp rx=0 tx=0).stop 2024-09-16T15:40:12.439 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:12.436+0000 7f97f6800640 1 -- 172.21.15.151:0/95278611 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f97f00aa8f0 msgr2=0x7f97f013ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:12.439 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:12.436+0000 7f97f6800640 1 --2- 172.21.15.151:0/95278611 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f97f00aa8f0 0x7f97f013ed40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f97f00cc550 tx=0x7f97e8011b10 comp rx=0 tx=0).stop 2024-09-16T15:40:12.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:12.438+0000 7f97f6800640 1 -- 172.21.15.151:0/95278611 shutdown_connections 2024-09-16T15:40:12.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:12.438+0000 7f97f6800640 1 --2- 172.21.15.151:0/95278611 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f97d0078850 0x7f97d007ad10 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:12.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:12.438+0000 7f97f6800640 1 --2- 172.21.15.151:0/95278611 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f97f00aa8f0 0x7f97f013ed40 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:12.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:12.438+0000 7f97f6800640 1 --2- 172.21.15.151:0/95278611 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97f0142b10 0x7f97f00ab890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:12.441 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:12.438+0000 7f97f6800640 1 --2- 172.21.15.151:0/95278611 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97d4004800 0x7f97f01f27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:12.441 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:12.438+0000 7f97f6800640 1 -- 172.21.15.151:0/95278611 >> 172.21.15.151:0/95278611 conn(0x7f97f00b1840 msgr2=0x7f97f00b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:12.441 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:12.439+0000 7f97f6800640 1 -- 172.21.15.151:0/95278611 shutdown_connections 2024-09-16T15:40:12.441 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:12.439+0000 7f97f6800640 1 -- 172.21.15.151:0/95278611 wait complete. 2024-09-16T15:40:12.468 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:40:12.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:12.625+0000 7fc1b7200640 1 -- 172.21.15.151:0/2673769160 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1b015f5a0 msgr2=0x7fc1b00b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:12.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:12.625+0000 7fc1b7200640 1 --2- 172.21.15.151:0/2673769160 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1b015f5a0 0x7fc1b00b2440 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0009830 tx=0x7fc1a002a530 comp rx=0 tx=0).stop 2024-09-16T15:40:12.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:12.625+0000 7fc1b7200640 1 -- 172.21.15.151:0/2673769160 shutdown_connections 2024-09-16T15:40:12.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:12.625+0000 7fc1b7200640 1 --2- 172.21.15.151:0/2673769160 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1b015f5a0 0x7fc1b00b2440 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:12.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:12.625+0000 7fc1b7200640 1 -- 172.21.15.151:0/2673769160 >> 172.21.15.151:0/2673769160 conn(0x7fc1b00b1950 msgr2=0x7fc1b00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:12.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:12.625+0000 7fc1b7200640 1 -- 172.21.15.151:0/2673769160 shutdown_connections 2024-09-16T15:40:12.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:12.625+0000 7fc1b7200640 1 -- 172.21.15.151:0/2673769160 wait complete. 2024-09-16T15:40:12.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:12.627+0000 7fc1b7200640 1 Processor -- start 2024-09-16T15:40:12.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:12.627+0000 7fc1b7200640 1 -- start start 2024-09-16T15:40:12.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:12.628+0000 7fc1b7200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1b0073fb0 0x7fc1b01559f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:12.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:12.628+0000 7fc1b7200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1b00ccf00 0x7fc1b01475e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:12.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:12.628+0000 7fc1b7200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1b015f5a0 0x7fc1b009c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:12.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:12.628+0000 7fc1b7200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b006f9f0 con 0x7fc1b0073fb0 2024-09-16T15:40:12.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:12.628+0000 7fc1b7200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b006f870 con 0x7fc1b015f5a0 2024-09-16T15:40:12.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:12.628+0000 7fc1b7200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fc1b0074420 con 0x7fc1b00ccf00 2024-09-16T15:40:12.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:12.628+0000 7fc1afe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1b0073fb0 0x7fc1b01559f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:12.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:12.628+0000 7fc1afe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1b0073fb0 0x7fc1b01559f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:38694/0 (socket says 172.21.15.151:38694) 2024-09-16T15:40:12.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:12.628+0000 7fc1afe00640 1 -- 172.21.15.151:0/130760559 learned_addr learned my addr 172.21.15.151:0/130760559 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:40:12.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:12.628+0000 7fc1a7400640 1 --2- 172.21.15.151:0/130760559 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1b00ccf00 0x7fc1b01475e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:12.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:12.628+0000 7fc1b5200640 1 --2- 172.21.15.151:0/130760559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1b015f5a0 0x7fc1b009c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:12.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:12.629+0000 7fc1afe00640 1 -- 172.21.15.151:0/130760559 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1b00ccf00 msgr2=0x7fc1b01475e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:12.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:12.629+0000 7fc1afe00640 1 --2- 172.21.15.151:0/130760559 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1b00ccf00 0x7fc1b01475e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:12.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:12.629+0000 7fc1afe00640 1 -- 172.21.15.151:0/130760559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1b015f5a0 msgr2=0x7fc1b009c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:12.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:12.629+0000 7fc1afe00640 1 --2- 172.21.15.151:0/130760559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1b015f5a0 0x7fc1b009c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:12.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:12.629+0000 7fc1afe00640 1 -- 172.21.15.151:0/130760559 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1b0147b20 con 0x7fc1b0073fb0 2024-09-16T15:40:12.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:12.629+0000 7fc1afe00640 1 --2- 172.21.15.151:0/130760559 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1b0073fb0 0x7fc1b01559f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0009950 tx=0x7fc1a0033c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:12.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:12.629+0000 7fc1acc00640 1 -- 172.21.15.151:0/130760559 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a0031b60 con 0x7fc1b0073fb0 2024-09-16T15:40:12.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:12.629+0000 7fc1acc00640 1 -- 172.21.15.151:0/130760559 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc1a0031df0 con 0x7fc1b0073fb0 2024-09-16T15:40:12.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:12.629+0000 7fc1b7200640 1 -- 172.21.15.151:0/130760559 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1b00cdd30 con 0x7fc1b0073fb0 2024-09-16T15:40:12.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:12.629+0000 7fc1acc00640 1 -- 172.21.15.151:0/130760559 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a0005280 con 0x7fc1b0073fb0 2024-09-16T15:40:12.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:12.629+0000 7fc1b7200640 1 -- 172.21.15.151:0/130760559 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1b00c9170 con 0x7fc1b0073fb0 2024-09-16T15:40:12.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:12.631+0000 7fc1b7200640 1 -- 172.21.15.151:0/130760559 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1b00b2440 con 0x7fc1b0073fb0 2024-09-16T15:40:12.634 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:12.632+0000 7fc1acc00640 1 -- 172.21.15.151:0/130760559 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fc1a0005420 con 0x7fc1b0073fb0 2024-09-16T15:40:12.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:12.633+0000 7fc1acc00640 1 --2- 172.21.15.151:0/130760559 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc18c078850 0x7fc18c07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:12.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:12.633+0000 7fc1acc00640 1 -- 172.21.15.151:0/130760559 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fc1a00bba50 con 0x7fc1b0073fb0 2024-09-16T15:40:12.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:12.633+0000 7fc1a7400640 1 --2- 172.21.15.151:0/130760559 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc18c078850 0x7fc18c07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:12.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:12.634+0000 7fc1a7400640 1 --2- 172.21.15.151:0/130760559 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc18c078850 0x7fc18c07ad10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc1b00b4660 tx=0x7fc198009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:12.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:12.637+0000 7fc1acc00640 1 -- 172.21.15.151:0/130760559 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc1a003f030 con 0x7fc1b0073fb0 2024-09-16T15:40:12.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:12.822+0000 7fc1b7200640 1 -- 172.21.15.151:0/130760559 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc1b00b0280 con 0x7fc1b0073fb0 2024-09-16T15:40:12.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:12.823+0000 7fc1acc00640 1 -- 172.21.15.151:0/130760559 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16742 (secure 0 0 0) 0x7fc1a0031770 con 0x7fc1b0073fb0 2024-09-16T15:40:12.826 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:40:12.826 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":34,"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","created":"2024-09-16T15:38:56.348535+0000","modified":"2024-09-16T15:40:11.836717+0000","last_up_change":"2024-09-16T15:39:10.681384+0000","last_in_change":"2024-09-16T15:39:00.544477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:39:11.602252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:40:10.163597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-16T15:40:10.798968+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5524076-a903-4d0a-af6a-caf474c4765d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6809","nonce":3111342741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6811","nonce":3111342741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6815","nonce":3111342741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6813","nonce":3111342741}]},"public_addr":"172.21.15.151:6809/3111342741","cluster_addr":"172.21.15.151:6811/3111342741","heartbeat_back_addr":"172.21.15.151:6815/3111342741","heartbeat_front_addr":"172.21.15.151:6813/3111342741","state":["exists","up"]},{"osd":1,"uuid":"dd25925f-0ae1-494e-8529-4dc205759db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6801","nonce":3368304087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6803","nonce":3368304087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6807","nonce":3368304087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6805","nonce":3368304087}]},"public_addr":"172.21.15.151:6801/3368304087","cluster_addr":"172.21.15.151:6803/3368304087","heartbeat_back_addr":"172.21.15.151:6807/3368304087","heartbeat_front_addr":"172.21.15.151:6805/3368304087","state":["exists","up"]},{"osd":2,"uuid":"c2f18115-013d-4da6-a0fe-cb978f4494b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6823","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6825","nonce":3079373271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6827","nonce":3079373271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6831","nonce":3079373271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6829","nonce":3079373271}]},"public_addr":"172.21.15.151:6825/3079373271","cluster_addr":"172.21.15.151:6827/3079373271","heartbeat_back_addr":"172.21.15.151:6831/3079373271","heartbeat_front_addr":"172.21.15.151:6829/3079373271","state":["exists","up"]},{"osd":3,"uuid":"36edb718-2fcb-4d88-8ddc-c321eed4279d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6817","nonce":1142509041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6819","nonce":1142509041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6824","nonce":1142509041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6821","nonce":1142509041}]},"public_addr":"172.21.15.151:6817/1142509041","cluster_addr":"172.21.15.151:6819/1142509041","heartbeat_back_addr":"172.21.15.151:6824/1142509041","heartbeat_front_addr":"172.21.15.151:6821/1142509041","state":["exists","up"]},{"osd":4,"uuid":"64d33708-cd0a-4236-abe3-1b646e82ab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6811","nonce":2735495271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6813","nonce":2735495271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6817","nonce":2735495271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6815","nonce":2735495271}]},"public_addr":"172.21.15.167:6811/2735495271","cluster_addr":"172.21.15.167:6813/2735495271","heartbeat_back_addr":"172.21.15.167:6817/2735495271","heartbeat_front_addr":"172.21.15.167:6815/2735495271","state":["exists","up"]},{"osd":5,"uuid":"f14b44a9-5910-458c-b518-09a8d0de94de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6827","nonce":1494916572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6829","nonce":1494916572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6833","nonce":1494916572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6831","nonce":1494916572}]},"public_addr":"172.21.15.167:6827/1494916572","cluster_addr":"172.21.15.167:6829/1494916572","heartbeat_back_addr":"172.21.15.167:6833/1494916572","heartbeat_front_addr":"172.21.15.167:6831/1494916572","state":["exists","up"]},{"osd":6,"uuid":"39335545-2654-4710-84b0-a9aa47cfcbba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6819","nonce":2538273273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6821","nonce":2538273273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6825","nonce":2538273273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6823","nonce":2538273273}]},"public_addr":"172.21.15.167:6819/2538273273","cluster_addr":"172.21.15.167:6821/2538273273","heartbeat_back_addr":"172.21.15.167:6825/2538273273","heartbeat_front_addr":"172.21.15.167:6823/2538273273","state":["exists","up"]},{"osd":7,"uuid":"4f8bfc81-53dd-4339-ac22-1a4007d9f963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6803","nonce":265072227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6805","nonce":265072227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6809","nonce":265072227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6807","nonce":265072227}]},"public_addr":"172.21.15.167:6803/265072227","cluster_addr":"172.21.15.167:6805/265072227","heartbeat_back_addr":"172.21.15.167:6809/265072227","heartbeat_front_addr":"172.21.15.167:6807/265072227","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.150359+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.084358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.722368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.332809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.320709+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.395153+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.935250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.801443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:40:12.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:12.826+0000 7fc1b7200640 1 -- 172.21.15.151:0/130760559 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc18c078850 msgr2=0x7fc18c07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:12.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:12.826+0000 7fc1b7200640 1 --2- 172.21.15.151:0/130760559 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc18c078850 0x7fc18c07ad10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc1b00b4660 tx=0x7fc198009040 comp rx=0 tx=0).stop 2024-09-16T15:40:12.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:12.826+0000 7fc1b7200640 1 -- 172.21.15.151:0/130760559 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1b0073fb0 msgr2=0x7fc1b01559f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:12.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:12.826+0000 7fc1b7200640 1 --2- 172.21.15.151:0/130760559 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1b0073fb0 0x7fc1b01559f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0009950 tx=0x7fc1a0033c10 comp rx=0 tx=0).stop 2024-09-16T15:40:12.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:12.826+0000 7fc1b7200640 1 -- 172.21.15.151:0/130760559 shutdown_connections 2024-09-16T15:40:12.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:12.826+0000 7fc1b7200640 1 --2- 172.21.15.151:0/130760559 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc18c078850 0x7fc18c07ad10 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:12.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:12.826+0000 7fc1b7200640 1 --2- 172.21.15.151:0/130760559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1b015f5a0 0x7fc1b009c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:12.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:12.826+0000 7fc1b7200640 1 --2- 172.21.15.151:0/130760559 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1b00ccf00 0x7fc1b01475e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:12.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:12.826+0000 7fc1b7200640 1 --2- 172.21.15.151:0/130760559 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1b0073fb0 0x7fc1b01559f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:12.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:12.826+0000 7fc1b7200640 1 -- 172.21.15.151:0/130760559 >> 172.21.15.151:0/130760559 conn(0x7fc1b00b1950 msgr2=0x7fc1b00af5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:12.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:12.826+0000 7fc1b7200640 1 -- 172.21.15.151:0/130760559 shutdown_connections 2024-09-16T15:40:12.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:12.826+0000 7fc1b7200640 1 -- 172.21.15.151:0/130760559 wait complete. 2024-09-16T15:40:12.844 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:40:13.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.000+0000 7f86c9600640 1 -- 172.21.15.151:0/3787441483 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86c40d4980 msgr2=0x7f86c40d1c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:13.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.000+0000 7f86c9600640 1 --2- 172.21.15.151:0/3787441483 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86c40d4980 0x7f86c40d1c60 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f86b0009830 tx=0x7f86b002a530 comp rx=0 tx=0).stop 2024-09-16T15:40:13.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.000+0000 7f86c9600640 1 -- 172.21.15.151:0/3787441483 shutdown_connections 2024-09-16T15:40:13.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.000+0000 7f86c9600640 1 --2- 172.21.15.151:0/3787441483 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86c40d4980 0x7f86c40d1c60 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:13.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.000+0000 7f86c9600640 1 -- 172.21.15.151:0/3787441483 >> 172.21.15.151:0/3787441483 conn(0x7f86c40d3df0 msgr2=0x7f86c40d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:13.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.000+0000 7f86c9600640 1 -- 172.21.15.151:0/3787441483 shutdown_connections 2024-09-16T15:40:13.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.000+0000 7f86c9600640 1 -- 172.21.15.151:0/3787441483 wait complete. 2024-09-16T15:40:13.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.001+0000 7f86c9600640 1 Processor -- start 2024-09-16T15:40:13.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.002+0000 7f86c9600640 1 -- start start 2024-09-16T15:40:13.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.002+0000 7f86c9600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86c40d4980 0x7f86c415c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:13.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.002+0000 7f86c9600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f86c41612d0 0x7f86c40b8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:13.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.002+0000 7f86c9600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86c41616b0 0x7f86c40b9520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:13.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.002+0000 7f86c9600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f86c41690d0 con 0x7f86c40d4980 2024-09-16T15:40:13.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.002+0000 7f86c9600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f86c4168f50 con 0x7f86c41616b0 2024-09-16T15:40:13.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.002+0000 7f86c9600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f86c4169250 con 0x7f86c41612d0 2024-09-16T15:40:13.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.003+0000 7f86c3e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86c40d4980 0x7f86c415c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:13.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.003+0000 7f86c3e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86c40d4980 0x7f86c415c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37224/0 (socket says 172.21.15.151:37224) 2024-09-16T15:40:13.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.003+0000 7f86c3e00640 1 -- 172.21.15.151:0/3700049104 learned_addr learned my addr 172.21.15.151:0/3700049104 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:40:13.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.003+0000 7f86c3400640 1 --2- 172.21.15.151:0/3700049104 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f86c41612d0 0x7f86c40b8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:13.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.003+0000 7f86c8c00640 1 --2- 172.21.15.151:0/3700049104 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86c41616b0 0x7f86c40b9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:13.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.003+0000 7f86c3e00640 1 -- 172.21.15.151:0/3700049104 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f86c41612d0 msgr2=0x7f86c40b8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:13.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.003+0000 7f86c3e00640 1 --2- 172.21.15.151:0/3700049104 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f86c41612d0 0x7f86c40b8fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:13.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.003+0000 7f86c3e00640 1 -- 172.21.15.151:0/3700049104 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86c41616b0 msgr2=0x7f86c40b9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:13.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.003+0000 7f86c3e00640 1 --2- 172.21.15.151:0/3700049104 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86c41616b0 0x7f86c40b9520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:13.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.003+0000 7f86c3e00640 1 -- 172.21.15.151:0/3700049104 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86c415e940 con 0x7f86c40d4980 2024-09-16T15:40:13.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.003+0000 7f86c8c00640 1 --2- 172.21.15.151:0/3700049104 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86c41616b0 0x7f86c40b9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:40:13.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.003+0000 7f86c3e00640 1 --2- 172.21.15.151:0/3700049104 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86c40d4980 0x7f86c415c8c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f86b400c730 tx=0x7f86b400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:13.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.004+0000 7f86bbe00640 1 -- 172.21.15.151:0/3700049104 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b4017070 con 0x7f86c40d4980 2024-09-16T15:40:13.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.004+0000 7f86bbe00640 1 -- 172.21.15.151:0/3700049104 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f86b4007e50 con 0x7f86c40d4980 2024-09-16T15:40:13.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.004+0000 7f86c9600640 1 -- 172.21.15.151:0/3700049104 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86c40b7dc0 con 0x7f86c40d4980 2024-09-16T15:40:13.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.004+0000 7f86bbe00640 1 -- 172.21.15.151:0/3700049104 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b400a640 con 0x7f86c40d4980 2024-09-16T15:40:13.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.004+0000 7f86c9600640 1 -- 172.21.15.151:0/3700049104 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86c40b8380 con 0x7f86c40d4980 2024-09-16T15:40:13.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.005+0000 7f86c9600640 1 -- 172.21.15.151:0/3700049104 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86c40b4e00 con 0x7f86c40d4980 2024-09-16T15:40:13.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.006+0000 7f86bbe00640 1 -- 172.21.15.151:0/3700049104 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f86b401a070 con 0x7f86c40d4980 2024-09-16T15:40:13.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.007+0000 7f86bbe00640 1 --2- 172.21.15.151:0/3700049104 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f86a4078850 0x7f86a407ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:13.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.007+0000 7f86bbe00640 1 -- 172.21.15.151:0/3700049104 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f86b4099d20 con 0x7f86c40d4980 2024-09-16T15:40:13.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.008+0000 7f86c3400640 1 --2- 172.21.15.151:0/3700049104 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f86a4078850 0x7f86a407ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:13.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.008+0000 7f86c3400640 1 --2- 172.21.15.151:0/3700049104 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f86a4078850 0x7f86a407ad10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f86b0034000 tx=0x7f86b0004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:13.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.011+0000 7f86bbe00640 1 -- 172.21.15.151:0/3700049104 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f86c415e940 con 0x7f86c40d4980 2024-09-16T15:40:13.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.196+0000 7f86c9600640 1 -- 172.21.15.151:0/3700049104 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f86c40b3dc0 con 0x7f86c40d4980 2024-09-16T15:40:13.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.197+0000 7f86bbe00640 1 -- 172.21.15.151:0/3700049104 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16742 (secure 0 0 0) 0x7f86c40b5510 con 0x7f86c40d4980 2024-09-16T15:40:13.199 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:40:13.200 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":34,"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","created":"2024-09-16T15:38:56.348535+0000","modified":"2024-09-16T15:40:11.836717+0000","last_up_change":"2024-09-16T15:39:10.681384+0000","last_in_change":"2024-09-16T15:39:00.544477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:39:11.602252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:40:10.163597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-16T15:40:10.798968+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5524076-a903-4d0a-af6a-caf474c4765d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6809","nonce":3111342741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6811","nonce":3111342741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6815","nonce":3111342741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6813","nonce":3111342741}]},"public_addr":"172.21.15.151:6809/3111342741","cluster_addr":"172.21.15.151:6811/3111342741","heartbeat_back_addr":"172.21.15.151:6815/3111342741","heartbeat_front_addr":"172.21.15.151:6813/3111342741","state":["exists","up"]},{"osd":1,"uuid":"dd25925f-0ae1-494e-8529-4dc205759db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6801","nonce":3368304087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6803","nonce":3368304087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6807","nonce":3368304087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6805","nonce":3368304087}]},"public_addr":"172.21.15.151:6801/3368304087","cluster_addr":"172.21.15.151:6803/3368304087","heartbeat_back_addr":"172.21.15.151:6807/3368304087","heartbeat_front_addr":"172.21.15.151:6805/3368304087","state":["exists","up"]},{"osd":2,"uuid":"c2f18115-013d-4da6-a0fe-cb978f4494b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6823","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6825","nonce":3079373271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6827","nonce":3079373271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6831","nonce":3079373271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6829","nonce":3079373271}]},"public_addr":"172.21.15.151:6825/3079373271","cluster_addr":"172.21.15.151:6827/3079373271","heartbeat_back_addr":"172.21.15.151:6831/3079373271","heartbeat_front_addr":"172.21.15.151:6829/3079373271","state":["exists","up"]},{"osd":3,"uuid":"36edb718-2fcb-4d88-8ddc-c321eed4279d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6817","nonce":1142509041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6819","nonce":1142509041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6824","nonce":1142509041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6821","nonce":1142509041}]},"public_addr":"172.21.15.151:6817/1142509041","cluster_addr":"172.21.15.151:6819/1142509041","heartbeat_back_addr":"172.21.15.151:6824/1142509041","heartbeat_front_addr":"172.21.15.151:6821/1142509041","state":["exists","up"]},{"osd":4,"uuid":"64d33708-cd0a-4236-abe3-1b646e82ab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6811","nonce":2735495271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6813","nonce":2735495271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6817","nonce":2735495271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6815","nonce":2735495271}]},"public_addr":"172.21.15.167:6811/2735495271","cluster_addr":"172.21.15.167:6813/2735495271","heartbeat_back_addr":"172.21.15.167:6817/2735495271","heartbeat_front_addr":"172.21.15.167:6815/2735495271","state":["exists","up"]},{"osd":5,"uuid":"f14b44a9-5910-458c-b518-09a8d0de94de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6827","nonce":1494916572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6829","nonce":1494916572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6833","nonce":1494916572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6831","nonce":1494916572}]},"public_addr":"172.21.15.167:6827/1494916572","cluster_addr":"172.21.15.167:6829/1494916572","heartbeat_back_addr":"172.21.15.167:6833/1494916572","heartbeat_front_addr":"172.21.15.167:6831/1494916572","state":["exists","up"]},{"osd":6,"uuid":"39335545-2654-4710-84b0-a9aa47cfcbba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6819","nonce":2538273273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6821","nonce":2538273273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6825","nonce":2538273273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6823","nonce":2538273273}]},"public_addr":"172.21.15.167:6819/2538273273","cluster_addr":"172.21.15.167:6821/2538273273","heartbeat_back_addr":"172.21.15.167:6825/2538273273","heartbeat_front_addr":"172.21.15.167:6823/2538273273","state":["exists","up"]},{"osd":7,"uuid":"4f8bfc81-53dd-4339-ac22-1a4007d9f963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6803","nonce":265072227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6805","nonce":265072227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6809","nonce":265072227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6807","nonce":265072227}]},"public_addr":"172.21.15.167:6803/265072227","cluster_addr":"172.21.15.167:6805/265072227","heartbeat_back_addr":"172.21.15.167:6809/265072227","heartbeat_front_addr":"172.21.15.167:6807/265072227","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.150359+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.084358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.722368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.332809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.320709+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.395153+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.935250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.801443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:40:13.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.199+0000 7f86c9600640 1 -- 172.21.15.151:0/3700049104 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f86a4078850 msgr2=0x7f86a407ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:13.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.199+0000 7f86c9600640 1 --2- 172.21.15.151:0/3700049104 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f86a4078850 0x7f86a407ad10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f86b0034000 tx=0x7f86b0004730 comp rx=0 tx=0).stop 2024-09-16T15:40:13.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.199+0000 7f86c9600640 1 -- 172.21.15.151:0/3700049104 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86c40d4980 msgr2=0x7f86c415c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:13.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.199+0000 7f86c9600640 1 --2- 172.21.15.151:0/3700049104 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86c40d4980 0x7f86c415c8c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f86b400c730 tx=0x7f86b400cbf0 comp rx=0 tx=0).stop 2024-09-16T15:40:13.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.200+0000 7f86c9600640 1 -- 172.21.15.151:0/3700049104 shutdown_connections 2024-09-16T15:40:13.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.200+0000 7f86c9600640 1 --2- 172.21.15.151:0/3700049104 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f86a4078850 0x7f86a407ad10 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:13.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.200+0000 7f86c9600640 1 --2- 172.21.15.151:0/3700049104 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86c41616b0 0x7f86c40b9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:13.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.200+0000 7f86c9600640 1 --2- 172.21.15.151:0/3700049104 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f86c41612d0 0x7f86c40b8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:13.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.200+0000 7f86c9600640 1 --2- 172.21.15.151:0/3700049104 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86c40d4980 0x7f86c415c8c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:13.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.200+0000 7f86c9600640 1 -- 172.21.15.151:0/3700049104 >> 172.21.15.151:0/3700049104 conn(0x7f86c40d3df0 msgr2=0x7f86c406e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:13.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.200+0000 7f86c9600640 1 -- 172.21.15.151:0/3700049104 shutdown_connections 2024-09-16T15:40:13.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.200+0000 7f86c9600640 1 -- 172.21.15.151:0/3700049104 wait complete. 2024-09-16T15:40:13.219 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-16T15:40:13.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.378+0000 7fe5dac00640 1 -- 172.21.15.151:0/1643128521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5d4161460 msgr2=0x7fe5d40d22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:13.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.378+0000 7fe5dac00640 1 --2- 172.21.15.151:0/1643128521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5d4161460 0x7fe5d40d22f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe5c8009830 tx=0x7fe5c802a530 comp rx=0 tx=0).stop 2024-09-16T15:40:13.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.378+0000 7fe5dac00640 1 -- 172.21.15.151:0/1643128521 shutdown_connections 2024-09-16T15:40:13.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.378+0000 7fe5dac00640 1 --2- 172.21.15.151:0/1643128521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5d4161460 0x7fe5d40d22f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:13.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.378+0000 7fe5dac00640 1 -- 172.21.15.151:0/1643128521 >> 172.21.15.151:0/1643128521 conn(0x7fe5d409d0b0 msgr2=0x7fe5d40d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:13.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.378+0000 7fe5dac00640 1 -- 172.21.15.151:0/1643128521 shutdown_connections 2024-09-16T15:40:13.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.378+0000 7fe5dac00640 1 -- 172.21.15.151:0/1643128521 wait complete. 2024-09-16T15:40:13.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.380+0000 7fe5dac00640 1 Processor -- start 2024-09-16T15:40:13.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.380+0000 7fe5dac00640 1 -- start start 2024-09-16T15:40:13.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.381+0000 7fe5dac00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe5d40b43c0 0x7fe5d415f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:13.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.381+0000 7fe5dac00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe5d40b5480 0x7fe5d40b3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:13.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.381+0000 7fe5dac00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5d4161460 0x7fe5d40cb860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:13.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.381+0000 7fe5dac00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d40d29b0 con 0x7fe5d40b5480 2024-09-16T15:40:13.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.381+0000 7fe5dac00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d40d2830 con 0x7fe5d4161460 2024-09-16T15:40:13.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.381+0000 7fe5dac00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fe5d40d2b30 con 0x7fe5d40b43c0 2024-09-16T15:40:13.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.381+0000 7fe5d8e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe5d40b5480 0x7fe5d40b3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:13.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.381+0000 7fe5d8e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe5d40b5480 0x7fe5d40b3930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37228/0 (socket says 172.21.15.151:37228) 2024-09-16T15:40:13.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.381+0000 7fe5d8e00640 1 -- 172.21.15.151:0/4261512704 learned_addr learned my addr 172.21.15.151:0/4261512704 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:40:13.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.382+0000 7fe5da200640 1 --2- 172.21.15.151:0/4261512704 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5d4161460 0x7fe5d40cb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:13.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.382+0000 7fe5d9800640 1 --2- 172.21.15.151:0/4261512704 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe5d40b43c0 0x7fe5d415f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:13.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.382+0000 7fe5d8e00640 1 -- 172.21.15.151:0/4261512704 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe5d40b43c0 msgr2=0x7fe5d415f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:13.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.382+0000 7fe5d8e00640 1 --2- 172.21.15.151:0/4261512704 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe5d40b43c0 0x7fe5d415f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:13.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.382+0000 7fe5d8e00640 1 -- 172.21.15.151:0/4261512704 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5d4161460 msgr2=0x7fe5d40cb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:13.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.382+0000 7fe5d8e00640 1 --2- 172.21.15.151:0/4261512704 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5d4161460 0x7fe5d40cb860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:13.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.382+0000 7fe5d8e00640 1 -- 172.21.15.151:0/4261512704 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5d41f49d0 con 0x7fe5d40b5480 2024-09-16T15:40:13.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.382+0000 7fe5d9800640 1 --2- 172.21.15.151:0/4261512704 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe5d40b43c0 0x7fe5d415f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:40:13.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.382+0000 7fe5da200640 1 --2- 172.21.15.151:0/4261512704 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5d4161460 0x7fe5d40cb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:40:13.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.382+0000 7fe5d8e00640 1 --2- 172.21.15.151:0/4261512704 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe5d40b5480 0x7fe5d40b3930 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe5c000c400 tx=0x7fe5c000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:13.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.382+0000 7fe5d1600640 1 -- 172.21.15.151:0/4261512704 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c0011790 con 0x7fe5d40b5480 2024-09-16T15:40:13.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.382+0000 7fe5d1600640 1 -- 172.21.15.151:0/4261512704 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe5c0017070 con 0x7fe5d40b5480 2024-09-16T15:40:13.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.382+0000 7fe5dac00640 1 -- 172.21.15.151:0/4261512704 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5d41f4cb0 con 0x7fe5d40b5480 2024-09-16T15:40:13.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.382+0000 7fe5d1600640 1 -- 172.21.15.151:0/4261512704 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c0011d00 con 0x7fe5d40b5480 2024-09-16T15:40:13.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.382+0000 7fe5dac00640 1 -- 172.21.15.151:0/4261512704 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5d41f5270 con 0x7fe5d40b5480 2024-09-16T15:40:13.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.384+0000 7fe5dac00640 1 -- 172.21.15.151:0/4261512704 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5d40cf870 con 0x7fe5d40b5480 2024-09-16T15:40:13.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.385+0000 7fe5d1600640 1 -- 172.21.15.151:0/4261512704 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fe5c001c460 con 0x7fe5d40b5480 2024-09-16T15:40:13.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.385+0000 7fe5d1600640 1 --2- 172.21.15.151:0/4261512704 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe5b4078880 0x7fe5b407ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:13.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.386+0000 7fe5d1600640 1 -- 172.21.15.151:0/4261512704 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fe5c009a660 con 0x7fe5d40b5480 2024-09-16T15:40:13.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.386+0000 7fe5d9800640 1 --2- 172.21.15.151:0/4261512704 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe5b4078880 0x7fe5b407ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:13.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.387+0000 7fe5d9800640 1 --2- 172.21.15.151:0/4261512704 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe5b4078880 0x7fe5b407ad40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe5c8034000 tx=0x7fe5c8004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:13.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.390+0000 7fe5d1600640 1 -- 172.21.15.151:0/4261512704 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe5c0062d60 con 0x7fe5d40b5480 2024-09-16T15:40:13.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:13.614+0000 7fe5dac00640 1 -- 172.21.15.151:0/4261512704 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fe5d409f500 con 0x7fe5d40b5480 2024-09-16T15:40:14.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:14.449+0000 7fe5d1600640 1 -- 172.21.15.151:0/4261512704 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7fe5c0062660 con 0x7fe5d40b5480 2024-09-16T15:40:14.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:14.451+0000 7fe5dac00640 1 -- 172.21.15.151:0/4261512704 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe5b4078880 msgr2=0x7fe5b407ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:14.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:14.451+0000 7fe5dac00640 1 --2- 172.21.15.151:0/4261512704 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe5b4078880 0x7fe5b407ad40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe5c8034000 tx=0x7fe5c8004730 comp rx=0 tx=0).stop 2024-09-16T15:40:14.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:14.451+0000 7fe5dac00640 1 -- 172.21.15.151:0/4261512704 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe5d40b5480 msgr2=0x7fe5d40b3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:14.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:14.451+0000 7fe5dac00640 1 --2- 172.21.15.151:0/4261512704 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe5d40b5480 0x7fe5d40b3930 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe5c000c400 tx=0x7fe5c000c8c0 comp rx=0 tx=0).stop 2024-09-16T15:40:14.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:14.452+0000 7fe5dac00640 1 -- 172.21.15.151:0/4261512704 shutdown_connections 2024-09-16T15:40:14.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:14.452+0000 7fe5dac00640 1 --2- 172.21.15.151:0/4261512704 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe5b4078880 0x7fe5b407ad40 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:14.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:14.452+0000 7fe5dac00640 1 --2- 172.21.15.151:0/4261512704 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5d4161460 0x7fe5d40cb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:14.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:14.452+0000 7fe5dac00640 1 --2- 172.21.15.151:0/4261512704 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe5d40b5480 0x7fe5d40b3930 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:14.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:14.452+0000 7fe5dac00640 1 --2- 172.21.15.151:0/4261512704 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe5d40b43c0 0x7fe5d415f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:14.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:14.452+0000 7fe5dac00640 1 -- 172.21.15.151:0/4261512704 >> 172.21.15.151:0/4261512704 conn(0x7fe5d409d0b0 msgr2=0x7fe5d40ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:14.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:14.452+0000 7fe5dac00640 1 -- 172.21.15.151:0/4261512704 shutdown_connections 2024-09-16T15:40:14.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:14.452+0000 7fe5dac00640 1 -- 172.21.15.151:0/4261512704 wait complete. 2024-09-16T15:40:14.471 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-16T15:40:14.471 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-16T15:40:14.634 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:14.631+0000 7fde1be00640 1 -- 172.21.15.151:0/2377216621 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde1c0cb4f0 msgr2=0x7fde1c0cb9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:14.634 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:14.631+0000 7fde1be00640 1 --2- 172.21.15.151:0/2377216621 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde1c0cb4f0 0x7fde1c0cb9a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fde1c072e80 tx=0x7fde1402a560 comp rx=0 tx=0).stop 2024-09-16T15:40:14.634 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:14.631+0000 7fde1be00640 1 -- 172.21.15.151:0/2377216621 shutdown_connections 2024-09-16T15:40:14.634 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:14.631+0000 7fde1be00640 1 --2- 172.21.15.151:0/2377216621 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde1c0cb4f0 0x7fde1c0cb9a0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:14.634 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:14.631+0000 7fde1be00640 1 --2- 172.21.15.151:0/2377216621 >> v2:172.21.15.167:3300/0 conn(0x7fde1c072560 0x7fde1c073ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:14.634 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:14.631+0000 7fde1be00640 1 --2- 172.21.15.151:0/2377216621 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde1c15bec0 0x7fde1c06fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:14.634 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:14.631+0000 7fde1be00640 1 -- 172.21.15.151:0/2377216621 >> 172.21.15.151:0/2377216621 conn(0x7fde1c0cf9f0 msgr2=0x7fde1c0cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:14.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:14.631+0000 7fde1be00640 1 -- 172.21.15.151:0/2377216621 shutdown_connections 2024-09-16T15:40:14.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:14.632+0000 7fde1be00640 1 -- 172.21.15.151:0/2377216621 wait complete. 2024-09-16T15:40:14.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:14.633+0000 7fde1be00640 1 Processor -- start 2024-09-16T15:40:14.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:14.633+0000 7fde1be00640 1 -- start start 2024-09-16T15:40:14.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:14.634+0000 7fde1be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde1c072560 0x7fde1c14d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:14.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:14.634+0000 7fde1be00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde1c0cb4f0 0x7fde1c14d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:14.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:14.634+0000 7fde1be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde1c15bec0 0x7fde1c0c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:14.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:14.634+0000 7fde1be00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fde1c0ae850 con 0x7fde1c15bec0 2024-09-16T15:40:14.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:14.634+0000 7fde1be00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fde1c0ae6d0 con 0x7fde1c072560 2024-09-16T15:40:14.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:14.634+0000 7fde1be00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fde1c0ae9d0 con 0x7fde1c0cb4f0 2024-09-16T15:40:14.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:14.634+0000 7fde1b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde1c15bec0 0x7fde1c0c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:14.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:14.634+0000 7fde1b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde1c15bec0 0x7fde1c0c94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37254/0 (socket says 172.21.15.151:37254) 2024-09-16T15:40:14.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:14.634+0000 7fde1b400640 1 -- 172.21.15.151:0/2753159884 learned_addr learned my addr 172.21.15.151:0/2753159884 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:40:14.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:14.635+0000 7fde1aa00640 1 --2- 172.21.15.151:0/2753159884 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde1c072560 0x7fde1c14d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:14.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:14.635+0000 7fde1a000640 1 --2- 172.21.15.151:0/2753159884 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde1c0cb4f0 0x7fde1c14d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:14.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:14.635+0000 7fde1b400640 1 -- 172.21.15.151:0/2753159884 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde1c0cb4f0 msgr2=0x7fde1c14d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:14.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:14.635+0000 7fde1b400640 1 --2- 172.21.15.151:0/2753159884 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde1c0cb4f0 0x7fde1c14d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:14.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:14.635+0000 7fde1b400640 1 -- 172.21.15.151:0/2753159884 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde1c072560 msgr2=0x7fde1c14d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:14.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:14.635+0000 7fde1b400640 1 --2- 172.21.15.151:0/2753159884 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde1c072560 0x7fde1c14d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:14.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:14.635+0000 7fde1b400640 1 -- 172.21.15.151:0/2753159884 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde1c14d6c0 con 0x7fde1c15bec0 2024-09-16T15:40:14.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:14.635+0000 7fde1aa00640 1 --2- 172.21.15.151:0/2753159884 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde1c072560 0x7fde1c14d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:40:14.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:14.635+0000 7fde1b400640 1 --2- 172.21.15.151:0/2753159884 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde1c15bec0 0x7fde1c0c94a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fde14009230 tx=0x7fde14003960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:14.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:14.635+0000 7fde12a00640 1 -- 172.21.15.151:0/2753159884 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1403f070 con 0x7fde1c15bec0 2024-09-16T15:40:14.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:14.635+0000 7fde12a00640 1 -- 172.21.15.151:0/2753159884 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fde140047a0 con 0x7fde1c15bec0 2024-09-16T15:40:14.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:14.635+0000 7fde12a00640 1 -- 172.21.15.151:0/2753159884 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde14035030 con 0x7fde1c15bec0 2024-09-16T15:40:14.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:14.635+0000 7fde1be00640 1 -- 172.21.15.151:0/2753159884 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde1c1f4710 con 0x7fde1c15bec0 2024-09-16T15:40:14.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:14.636+0000 7fde1be00640 1 -- 172.21.15.151:0/2753159884 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde1c1f4cd0 con 0x7fde1c15bec0 2024-09-16T15:40:14.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:14.637+0000 7fde1be00640 1 -- 172.21.15.151:0/2753159884 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde1c0aee80 con 0x7fde1c15bec0 2024-09-16T15:40:14.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:14.638+0000 7fde12a00640 1 -- 172.21.15.151:0/2753159884 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fde14042070 con 0x7fde1c15bec0 2024-09-16T15:40:14.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:14.639+0000 7fde12a00640 1 --2- 172.21.15.151:0/2753159884 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fddfc078850 0x7fddfc07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:14.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:14.639+0000 7fde12a00640 1 -- 172.21.15.151:0/2753159884 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fde140bb6e0 con 0x7fde1c15bec0 2024-09-16T15:40:14.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:14.639+0000 7fde1aa00640 1 --2- 172.21.15.151:0/2753159884 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fddfc078850 0x7fddfc07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:14.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:14.640+0000 7fde1aa00640 1 --2- 172.21.15.151:0/2753159884 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fddfc078850 0x7fddfc07ad10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fde0c005fd0 tx=0x7fde0c009340 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:14.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:14.643+0000 7fde12a00640 1 -- 172.21.15.151:0/2753159884 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fde14083dc0 con 0x7fde1c15bec0 2024-09-16T15:40:14.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:14.865+0000 7fde1be00640 1 -- 172.21.15.151:0/2753159884 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fde1c0d00e0 con 0x7fde1c15bec0 2024-09-16T15:40:15.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:15.450+0000 7fde12a00640 1 -- 172.21.15.151:0/2753159884 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) ==== 122+0+0 (secure 0 0 0) 0x7fde140836c0 con 0x7fde1c15bec0 2024-09-16T15:40:15.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:15.453+0000 7fde1be00640 1 -- 172.21.15.151:0/2753159884 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fddfc078850 msgr2=0x7fddfc07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:15.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:15.453+0000 7fde1be00640 1 --2- 172.21.15.151:0/2753159884 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fddfc078850 0x7fddfc07ad10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fde0c005fd0 tx=0x7fde0c009340 comp rx=0 tx=0).stop 2024-09-16T15:40:15.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:15.453+0000 7fde1be00640 1 -- 172.21.15.151:0/2753159884 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde1c15bec0 msgr2=0x7fde1c0c94a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:15.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:15.453+0000 7fde1be00640 1 --2- 172.21.15.151:0/2753159884 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde1c15bec0 0x7fde1c0c94a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fde14009230 tx=0x7fde14003960 comp rx=0 tx=0).stop 2024-09-16T15:40:15.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:15.453+0000 7fde1be00640 1 -- 172.21.15.151:0/2753159884 shutdown_connections 2024-09-16T15:40:15.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:15.453+0000 7fde1be00640 1 --2- 172.21.15.151:0/2753159884 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fddfc078850 0x7fddfc07ad10 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:15.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:15.453+0000 7fde1be00640 1 --2- 172.21.15.151:0/2753159884 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde1c15bec0 0x7fde1c0c94a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:15.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:15.453+0000 7fde1be00640 1 --2- 172.21.15.151:0/2753159884 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde1c0cb4f0 0x7fde1c14d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:15.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:15.453+0000 7fde1be00640 1 --2- 172.21.15.151:0/2753159884 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde1c072560 0x7fde1c14d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:15.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:15.453+0000 7fde1be00640 1 -- 172.21.15.151:0/2753159884 >> 172.21.15.151:0/2753159884 conn(0x7fde1c0cf9f0 msgr2=0x7fde1c0cab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:15.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:15.453+0000 7fde1be00640 1 -- 172.21.15.151:0/2753159884 shutdown_connections 2024-09-16T15:40:15.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:15.454+0000 7fde1be00640 1 -- 172.21.15.151:0/2753159884 wait complete. 2024-09-16T15:40:15.472 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-16T15:40:15.472 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:40:15.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:15.628+0000 7fdbcb600640 1 -- 172.21.15.151:0/1898209052 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdbc4072df0 msgr2=0x7fdbc40732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:15.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:15.628+0000 7fdbcb600640 1 --2- 172.21.15.151:0/1898209052 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdbc4072df0 0x7fdbc40732a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc005fd0 tx=0x7fdbbc02a820 comp rx=0 tx=0).stop 2024-09-16T15:40:15.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:15.628+0000 7fdbcb600640 1 -- 172.21.15.151:0/1898209052 shutdown_connections 2024-09-16T15:40:15.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:15.628+0000 7fdbcb600640 1 --2- 172.21.15.151:0/1898209052 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdbc4072df0 0x7fdbc40732a0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:15.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:15.628+0000 7fdbcb600640 1 --2- 172.21.15.151:0/1898209052 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdbc4161550 0x7fdbc406ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:15.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:15.628+0000 7fdbcb600640 1 -- 172.21.15.151:0/1898209052 >> 172.21.15.151:0/1898209052 conn(0x7fdbc40b1950 msgr2=0x7fdbc40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:15.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:15.629+0000 7fdbcb600640 1 -- 172.21.15.151:0/1898209052 shutdown_connections 2024-09-16T15:40:15.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:15.629+0000 7fdbcb600640 1 -- 172.21.15.151:0/1898209052 wait complete. 2024-09-16T15:40:15.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:15.630+0000 7fdbcb600640 1 Processor -- start 2024-09-16T15:40:15.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:15.630+0000 7fdbcb600640 1 -- start start 2024-09-16T15:40:15.634 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:15.631+0000 7fdbcb600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbc4072df0 0x7fdbc41f48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:15.634 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:15.631+0000 7fdbcb600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdbc40af3d0 0x7fdbc414bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:15.634 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:15.631+0000 7fdbcb600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdbc4161550 0x7fdbc41f8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:15.634 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:15.631+0000 7fdbcb600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc40737e0 con 0x7fdbc4161550 2024-09-16T15:40:15.634 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:15.631+0000 7fdbcb600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc406fdb0 con 0x7fdbc4072df0 2024-09-16T15:40:15.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:15.631+0000 7fdbcb600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fdbc4073960 con 0x7fdbc40af3d0 2024-09-16T15:40:15.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:15.632+0000 7fdbc9600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdbc4161550 0x7fdbc41f8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:15.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:15.632+0000 7fdbc9600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdbc4161550 0x7fdbc41f8480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37284/0 (socket says 172.21.15.151:37284) 2024-09-16T15:40:15.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:15.632+0000 7fdbc9600640 1 -- 172.21.15.151:0/2590018498 learned_addr learned my addr 172.21.15.151:0/2590018498 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:40:15.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:15.632+0000 7fdbc8c00640 1 --2- 172.21.15.151:0/2590018498 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbc4072df0 0x7fdbc41f48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:15.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:15.632+0000 7fdbc3e00640 1 --2- 172.21.15.151:0/2590018498 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdbc40af3d0 0x7fdbc414bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:15.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:15.632+0000 7fdbc9600640 1 -- 172.21.15.151:0/2590018498 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdbc40af3d0 msgr2=0x7fdbc414bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:15.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:15.632+0000 7fdbc9600640 1 --2- 172.21.15.151:0/2590018498 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdbc40af3d0 0x7fdbc414bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:15.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:15.632+0000 7fdbc9600640 1 -- 172.21.15.151:0/2590018498 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbc4072df0 msgr2=0x7fdbc41f48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:15.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:15.632+0000 7fdbc9600640 1 --2- 172.21.15.151:0/2590018498 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbc4072df0 0x7fdbc41f48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:15.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:15.632+0000 7fdbc9600640 1 -- 172.21.15.151:0/2590018498 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbc41f8d80 con 0x7fdbc4161550 2024-09-16T15:40:15.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:15.633+0000 7fdbc9600640 1 --2- 172.21.15.151:0/2590018498 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdbc4161550 0x7fdbc41f8480 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc006010 tx=0x7fdbbc032c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:15.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:15.633+0000 7fdbc0c00640 1 -- 172.21.15.151:0/2590018498 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbbc037040 con 0x7fdbc4161550 2024-09-16T15:40:15.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:15.633+0000 7fdbc0c00640 1 -- 172.21.15.151:0/2590018498 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdbbc042070 con 0x7fdbc4161550 2024-09-16T15:40:15.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:15.633+0000 7fdbc0c00640 1 -- 172.21.15.151:0/2590018498 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbbc0042a0 con 0x7fdbc4161550 2024-09-16T15:40:15.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:15.633+0000 7fdbcb600640 1 -- 172.21.15.151:0/2590018498 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbc41f9060 con 0x7fdbc4161550 2024-09-16T15:40:15.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:15.633+0000 7fdbcb600640 1 -- 172.21.15.151:0/2590018498 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbc414b600 con 0x7fdbc4161550 2024-09-16T15:40:15.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:15.634+0000 7fdbcb600640 1 -- 172.21.15.151:0/2590018498 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbc4073e70 con 0x7fdbc4161550 2024-09-16T15:40:15.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:15.635+0000 7fdbc0c00640 1 -- 172.21.15.151:0/2590018498 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fdbbc004440 con 0x7fdbc4161550 2024-09-16T15:40:15.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:15.636+0000 7fdbc0c00640 1 --2- 172.21.15.151:0/2590018498 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdba4078850 0x7fdba407ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:15.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:15.636+0000 7fdbc0c00640 1 -- 172.21.15.151:0/2590018498 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fdbbc0bb440 con 0x7fdbc4161550 2024-09-16T15:40:15.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:15.636+0000 7fdbc8c00640 1 --2- 172.21.15.151:0/2590018498 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdba4078850 0x7fdba407ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:15.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:15.637+0000 7fdbc8c00640 1 --2- 172.21.15.151:0/2590018498 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdba4078850 0x7fdba407ad10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdbc40cd5d0 tx=0x7fdbb8005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:15.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:15.640+0000 7fdbc0c00640 1 -- 172.21.15.151:0/2590018498 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fdbbc083450 con 0x7fdbc4161550 2024-09-16T15:40:15.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:15.858+0000 7fdbcb600640 1 -- 172.21.15.151:0/2590018498 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdbc40b3f40 con 0x7fdbc4161550 2024-09-16T15:40:15.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:15.859+0000 7fdbc0c00640 1 -- 172.21.15.151:0/2590018498 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5232 (secure 0 0 0) 0x7fdbbc087e80 con 0x7fdbc4161550 2024-09-16T15:40:15.861 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:40:15.861 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":17,"btime":"2024-09-16T15:40:15:450685+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4661,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/873312529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":873312529},{"type":"v1","addr":"172.21.15.151:6834","nonce":873312529}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/2683082898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2683082898},{"type":"v1","addr":"172.21.15.167:6835","nonce":2683082898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4843,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2443370538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":2443370538},{"type":"v1","addr":"172.21.15.151:6835","nonce":2443370538}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:40:11.836645+0000","modified":"2024-09-16T15:40:14.867199+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4855},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4855":{"gid":4855,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6837/4150137107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":4150137107},{"type":"v1","addr":"172.21.15.167:6837","nonce":4150137107}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4855,"qdb_cluster":[4855]},"id":2}]} 2024-09-16T15:40:15.862 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 17 2024-09-16T15:40:15.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:15.862+0000 7fdbcb600640 1 -- 172.21.15.151:0/2590018498 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdba4078850 msgr2=0x7fdba407ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:15.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:15.862+0000 7fdbcb600640 1 --2- 172.21.15.151:0/2590018498 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdba4078850 0x7fdba407ad10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdbc40cd5d0 tx=0x7fdbb8005f90 comp rx=0 tx=0).stop 2024-09-16T15:40:15.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:15.862+0000 7fdbcb600640 1 -- 172.21.15.151:0/2590018498 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdbc4161550 msgr2=0x7fdbc41f8480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:15.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:15.862+0000 7fdbcb600640 1 --2- 172.21.15.151:0/2590018498 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdbc4161550 0x7fdbc41f8480 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc006010 tx=0x7fdbbc032c90 comp rx=0 tx=0).stop 2024-09-16T15:40:15.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:15.862+0000 7fdbcb600640 1 -- 172.21.15.151:0/2590018498 shutdown_connections 2024-09-16T15:40:15.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:15.862+0000 7fdbcb600640 1 --2- 172.21.15.151:0/2590018498 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdba4078850 0x7fdba407ad10 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:15.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:15.862+0000 7fdbcb600640 1 --2- 172.21.15.151:0/2590018498 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdbc4161550 0x7fdbc41f8480 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:15.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:15.862+0000 7fdbcb600640 1 --2- 172.21.15.151:0/2590018498 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdbc40af3d0 0x7fdbc414bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:15.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:15.862+0000 7fdbcb600640 1 --2- 172.21.15.151:0/2590018498 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbc4072df0 0x7fdbc41f48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:15.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:15.862+0000 7fdbcb600640 1 -- 172.21.15.151:0/2590018498 >> 172.21.15.151:0/2590018498 conn(0x7fdbc40b1950 msgr2=0x7fdbc40ae870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:15.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:15.862+0000 7fdbcb600640 1 -- 172.21.15.151:0/2590018498 shutdown_connections 2024-09-16T15:40:15.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:15.862+0000 7fdbcb600640 1 -- 172.21.15.151:0/2590018498 wait complete. 2024-09-16T15:40:15.882 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:40:16.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.037+0000 7fbbf8e00640 1 -- 172.21.15.151:0/2938155306 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbbf40b2440 msgr2=0x7fbbf406f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:16.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.037+0000 7fbbf8e00640 1 --2- 172.21.15.151:0/2938155306 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbbf40b2440 0x7fbbf406f160 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fbbec005fd0 tx=0x7fbbec02a8b0 comp rx=0 tx=0).stop 2024-09-16T15:40:16.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.037+0000 7fbbf8e00640 1 -- 172.21.15.151:0/2938155306 shutdown_connections 2024-09-16T15:40:16.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.037+0000 7fbbf8e00640 1 --2- 172.21.15.151:0/2938155306 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbbf40b2440 0x7fbbf406f160 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:16.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.037+0000 7fbbf8e00640 1 -- 172.21.15.151:0/2938155306 >> 172.21.15.151:0/2938155306 conn(0x7fbbf40b1950 msgr2=0x7fbbf40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:16.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.037+0000 7fbbf8e00640 1 -- 172.21.15.151:0/2938155306 shutdown_connections 2024-09-16T15:40:16.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.037+0000 7fbbf8e00640 1 -- 172.21.15.151:0/2938155306 wait complete. 2024-09-16T15:40:16.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.039+0000 7fbbf8e00640 1 Processor -- start 2024-09-16T15:40:16.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.039+0000 7fbbf8e00640 1 -- start start 2024-09-16T15:40:16.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.040+0000 7fbbf8e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbbf40b2440 0x7fbbf41537a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:16.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.040+0000 7fbbf8e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbbf40ccf00 0x7fbbf4159a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:16.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.040+0000 7fbbf8e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbf415f5a0 0x7fbbf41f2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:16.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.040+0000 7fbbf8e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf4074840 con 0x7fbbf40b2440 2024-09-16T15:40:16.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.040+0000 7fbbf8e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf40746c0 con 0x7fbbf415f5a0 2024-09-16T15:40:16.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.040+0000 7fbbf8e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fbbf406f6a0 con 0x7fbbf40ccf00 2024-09-16T15:40:16.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.040+0000 7fbbf1e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbbf40b2440 0x7fbbf41537a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:16.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.040+0000 7fbbf1e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbbf40b2440 0x7fbbf41537a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37306/0 (socket says 172.21.15.151:37306) 2024-09-16T15:40:16.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.040+0000 7fbbf1e00640 1 -- 172.21.15.151:0/248129960 learned_addr learned my addr 172.21.15.151:0/248129960 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:40:16.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.041+0000 7fbbf2800640 1 --2- 172.21.15.151:0/248129960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbf415f5a0 0x7fbbf41f2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:16.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.041+0000 7fbbf1400640 1 --2- 172.21.15.151:0/248129960 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbbf40ccf00 0x7fbbf4159a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:16.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.041+0000 7fbbf1e00640 1 -- 172.21.15.151:0/248129960 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbbf40ccf00 msgr2=0x7fbbf4159a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:16.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.041+0000 7fbbf1e00640 1 --2- 172.21.15.151:0/248129960 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbbf40ccf00 0x7fbbf4159a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:16.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.041+0000 7fbbf1e00640 1 -- 172.21.15.151:0/248129960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbf415f5a0 msgr2=0x7fbbf41f2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:16.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.041+0000 7fbbf1e00640 1 --2- 172.21.15.151:0/248129960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbf415f5a0 0x7fbbf41f2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:16.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.041+0000 7fbbf1e00640 1 -- 172.21.15.151:0/248129960 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbf41f3060 con 0x7fbbf40b2440 2024-09-16T15:40:16.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.041+0000 7fbbf2800640 1 --2- 172.21.15.151:0/248129960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbf415f5a0 0x7fbbf41f2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:40:16.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.041+0000 7fbbf1e00640 1 --2- 172.21.15.151:0/248129960 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbbf40b2440 0x7fbbf41537a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fbbe400c400 tx=0x7fbbe400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:16.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.041+0000 7fbbea000640 1 -- 172.21.15.151:0/248129960 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe4017070 con 0x7fbbf40b2440 2024-09-16T15:40:16.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.041+0000 7fbbea000640 1 -- 172.21.15.151:0/248129960 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbbe40053c0 con 0x7fbbf40b2440 2024-09-16T15:40:16.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.041+0000 7fbbea000640 1 -- 172.21.15.151:0/248129960 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe4003850 con 0x7fbbf40b2440 2024-09-16T15:40:16.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.041+0000 7fbbf8e00640 1 -- 172.21.15.151:0/248129960 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbf41f3340 con 0x7fbbf40b2440 2024-09-16T15:40:16.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.042+0000 7fbbf8e00640 1 -- 172.21.15.151:0/248129960 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbf4153ce0 con 0x7fbbf40b2440 2024-09-16T15:40:16.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.043+0000 7fbbf8e00640 1 -- 172.21.15.151:0/248129960 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbf41f3580 con 0x7fbbf40b2440 2024-09-16T15:40:16.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.044+0000 7fbbea000640 1 -- 172.21.15.151:0/248129960 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fbbe401a070 con 0x7fbbf40b2440 2024-09-16T15:40:16.047 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.045+0000 7fbbea000640 1 --2- 172.21.15.151:0/248129960 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbbd4078880 0x7fbbd407ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:16.047 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.045+0000 7fbbea000640 1 -- 172.21.15.151:0/248129960 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fbbe4099ce0 con 0x7fbbf40b2440 2024-09-16T15:40:16.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.046+0000 7fbbf1400640 1 --2- 172.21.15.151:0/248129960 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbbd4078880 0x7fbbd407ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:16.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.046+0000 7fbbf1400640 1 --2- 172.21.15.151:0/248129960 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbbd4078880 0x7fbbd407ad40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0000f60 tx=0x7fbbe0009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:16.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.050+0000 7fbbea000640 1 -- 172.21.15.151:0/248129960 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fbbf41f3580 con 0x7fbbf40b2440 2024-09-16T15:40:16.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.238+0000 7fbbf8e00640 1 -- 172.21.15.151:0/248129960 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbbf406f160 con 0x7fbbf40b2440 2024-09-16T15:40:16.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.240+0000 7fbbea000640 1 -- 172.21.15.151:0/248129960 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16729 (secure 0 0 0) 0x7fbbe4062340 con 0x7fbbf40b2440 2024-09-16T15:40:16.242 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:40:16.242 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":35,"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","created":"2024-09-16T15:38:56.348535+0000","modified":"2024-09-16T15:40:13.427542+0000","last_up_change":"2024-09-16T15:39:10.681384+0000","last_in_change":"2024-09-16T15:39:00.544477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:39:11.602252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:40:10.163597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-16T15:40:10.798968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5524076-a903-4d0a-af6a-caf474c4765d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6809","nonce":3111342741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6811","nonce":3111342741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6815","nonce":3111342741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6813","nonce":3111342741}]},"public_addr":"172.21.15.151:6809/3111342741","cluster_addr":"172.21.15.151:6811/3111342741","heartbeat_back_addr":"172.21.15.151:6815/3111342741","heartbeat_front_addr":"172.21.15.151:6813/3111342741","state":["exists","up"]},{"osd":1,"uuid":"dd25925f-0ae1-494e-8529-4dc205759db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6801","nonce":3368304087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6803","nonce":3368304087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6807","nonce":3368304087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6805","nonce":3368304087}]},"public_addr":"172.21.15.151:6801/3368304087","cluster_addr":"172.21.15.151:6803/3368304087","heartbeat_back_addr":"172.21.15.151:6807/3368304087","heartbeat_front_addr":"172.21.15.151:6805/3368304087","state":["exists","up"]},{"osd":2,"uuid":"c2f18115-013d-4da6-a0fe-cb978f4494b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6823","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6825","nonce":3079373271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6827","nonce":3079373271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6831","nonce":3079373271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6829","nonce":3079373271}]},"public_addr":"172.21.15.151:6825/3079373271","cluster_addr":"172.21.15.151:6827/3079373271","heartbeat_back_addr":"172.21.15.151:6831/3079373271","heartbeat_front_addr":"172.21.15.151:6829/3079373271","state":["exists","up"]},{"osd":3,"uuid":"36edb718-2fcb-4d88-8ddc-c321eed4279d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6817","nonce":1142509041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6819","nonce":1142509041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6824","nonce":1142509041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6821","nonce":1142509041}]},"public_addr":"172.21.15.151:6817/1142509041","cluster_addr":"172.21.15.151:6819/1142509041","heartbeat_back_addr":"172.21.15.151:6824/1142509041","heartbeat_front_addr":"172.21.15.151:6821/1142509041","state":["exists","up"]},{"osd":4,"uuid":"64d33708-cd0a-4236-abe3-1b646e82ab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6811","nonce":2735495271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6813","nonce":2735495271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6817","nonce":2735495271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6815","nonce":2735495271}]},"public_addr":"172.21.15.167:6811/2735495271","cluster_addr":"172.21.15.167:6813/2735495271","heartbeat_back_addr":"172.21.15.167:6817/2735495271","heartbeat_front_addr":"172.21.15.167:6815/2735495271","state":["exists","up"]},{"osd":5,"uuid":"f14b44a9-5910-458c-b518-09a8d0de94de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6827","nonce":1494916572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6829","nonce":1494916572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6833","nonce":1494916572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6831","nonce":1494916572}]},"public_addr":"172.21.15.167:6827/1494916572","cluster_addr":"172.21.15.167:6829/1494916572","heartbeat_back_addr":"172.21.15.167:6833/1494916572","heartbeat_front_addr":"172.21.15.167:6831/1494916572","state":["exists","up"]},{"osd":6,"uuid":"39335545-2654-4710-84b0-a9aa47cfcbba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6819","nonce":2538273273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6821","nonce":2538273273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6825","nonce":2538273273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6823","nonce":2538273273}]},"public_addr":"172.21.15.167:6819/2538273273","cluster_addr":"172.21.15.167:6821/2538273273","heartbeat_back_addr":"172.21.15.167:6825/2538273273","heartbeat_front_addr":"172.21.15.167:6823/2538273273","state":["exists","up"]},{"osd":7,"uuid":"4f8bfc81-53dd-4339-ac22-1a4007d9f963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6803","nonce":265072227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6805","nonce":265072227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6809","nonce":265072227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6807","nonce":265072227}]},"public_addr":"172.21.15.167:6803/265072227","cluster_addr":"172.21.15.167:6805/265072227","heartbeat_back_addr":"172.21.15.167:6809/265072227","heartbeat_front_addr":"172.21.15.167:6807/265072227","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.150359+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.084358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.722368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.332809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.320709+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.395153+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.935250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.801443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:40:16.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.242+0000 7fbbf8e00640 1 -- 172.21.15.151:0/248129960 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbbd4078880 msgr2=0x7fbbd407ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:16.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.242+0000 7fbbf8e00640 1 --2- 172.21.15.151:0/248129960 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbbd4078880 0x7fbbd407ad40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0000f60 tx=0x7fbbe0009040 comp rx=0 tx=0).stop 2024-09-16T15:40:16.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.242+0000 7fbbf8e00640 1 -- 172.21.15.151:0/248129960 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbbf40b2440 msgr2=0x7fbbf41537a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:16.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.242+0000 7fbbf8e00640 1 --2- 172.21.15.151:0/248129960 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbbf40b2440 0x7fbbf41537a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fbbe400c400 tx=0x7fbbe400c8c0 comp rx=0 tx=0).stop 2024-09-16T15:40:16.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.242+0000 7fbbf8e00640 1 -- 172.21.15.151:0/248129960 shutdown_connections 2024-09-16T15:40:16.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.242+0000 7fbbf8e00640 1 --2- 172.21.15.151:0/248129960 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbbd4078880 0x7fbbd407ad40 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:16.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.242+0000 7fbbf8e00640 1 --2- 172.21.15.151:0/248129960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbf415f5a0 0x7fbbf41f2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:16.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.242+0000 7fbbf8e00640 1 --2- 172.21.15.151:0/248129960 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbbf40ccf00 0x7fbbf4159a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:16.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.242+0000 7fbbf8e00640 1 --2- 172.21.15.151:0/248129960 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbbf40b2440 0x7fbbf41537a0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:16.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.242+0000 7fbbf8e00640 1 -- 172.21.15.151:0/248129960 >> 172.21.15.151:0/248129960 conn(0x7fbbf40b1950 msgr2=0x7fbbf40ad710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:16.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.242+0000 7fbbf8e00640 1 -- 172.21.15.151:0/248129960 shutdown_connections 2024-09-16T15:40:16.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.242+0000 7fbbf8e00640 1 -- 172.21.15.151:0/248129960 wait complete. 2024-09-16T15:40:16.261 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T15:40:16.261 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:40:16.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.418+0000 7fedd6c00640 1 -- 172.21.15.151:0/2160565424 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fedb8004100 msgr2=0x7fedb8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:16.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.418+0000 7fedd6c00640 1 --2- 172.21.15.151:0/2160565424 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fedb8004100 0x7fedb8004800 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fedbc001800 tx=0x7fedb8005830 comp rx=0 tx=0).stop 2024-09-16T15:40:16.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.418+0000 7fedd6200640 1 -- 172.21.15.151:0/2160565424 reap_dead start 2024-09-16T15:40:16.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.418+0000 7fedd6c00640 1 -- 172.21.15.151:0/2160565424 shutdown_connections 2024-09-16T15:40:16.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.418+0000 7fedd6c00640 1 -- 172.21.15.151:0/2160565424 >> 172.21.15.151:0/2160565424 conn(0x7fedd00cfa20 msgr2=0x7fedd00cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:16.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.419+0000 7fedd6c00640 1 -- 172.21.15.151:0/2160565424 shutdown_connections 2024-09-16T15:40:16.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.419+0000 7fedd6c00640 1 -- 172.21.15.151:0/2160565424 wait complete. 2024-09-16T15:40:16.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.420+0000 7fedd6c00640 1 Processor -- start 2024-09-16T15:40:16.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.420+0000 7fedd6c00640 1 -- start start 2024-09-16T15:40:16.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.421+0000 7fedd6c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fedb8004100 0x7fedd0159370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:16.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.421+0000 7fedd6c00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fedd00cecb0 0x7fedd00cf130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:16.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.421+0000 7fedd6c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fedd00cd9b0 0x7fedd00cde60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:16.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.421+0000 7fedd6c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fedd0074660 con 0x7fedb8004100 2024-09-16T15:40:16.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.421+0000 7fedd6c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fedd00b28f0 con 0x7fedd00cd9b0 2024-09-16T15:40:16.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.421+0000 7fedd6c00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fedd01f7d30 con 0x7fedd00cecb0 2024-09-16T15:40:16.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.422+0000 7fedd5800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fedb8004100 0x7fedd0159370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:16.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.422+0000 7fedd5800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fedb8004100 0x7fedd0159370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37314/0 (socket says 172.21.15.151:37314) 2024-09-16T15:40:16.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.422+0000 7fedd5800640 1 -- 172.21.15.151:0/3750567459 learned_addr learned my addr 172.21.15.151:0/3750567459 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:40:16.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.422+0000 7fedd6200640 1 --2- 172.21.15.151:0/3750567459 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fedd00cd9b0 0x7fedd00cde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:16.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.422+0000 7fedd4e00640 1 --2- 172.21.15.151:0/3750567459 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fedd00cecb0 0x7fedd00cf130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:16.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.422+0000 7fedd5800640 1 -- 172.21.15.151:0/3750567459 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fedd00cecb0 msgr2=0x7fedd00cf130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:16.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.422+0000 7fedd5800640 1 --2- 172.21.15.151:0/3750567459 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fedd00cecb0 0x7fedd00cf130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:16.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.422+0000 7fedd5800640 1 -- 172.21.15.151:0/3750567459 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fedd00cd9b0 msgr2=0x7fedd00cde60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:16.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.422+0000 7fedd5800640 1 --2- 172.21.15.151:0/3750567459 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fedd00cd9b0 0x7fedd00cde60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:16.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.422+0000 7fedd5800640 1 -- 172.21.15.151:0/3750567459 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedd01f7fd0 con 0x7fedb8004100 2024-09-16T15:40:16.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.422+0000 7fedd6200640 1 --2- 172.21.15.151:0/3750567459 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fedd00cd9b0 0x7fedd00cde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:40:16.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.422+0000 7fedd4e00640 1 --2- 172.21.15.151:0/3750567459 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fedd00cecb0 0x7fedd00cf130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:40:16.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.422+0000 7fedd5800640 1 --2- 172.21.15.151:0/3750567459 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fedb8004100 0x7fedd0159370 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fedc4011b10 tx=0x7fedc4009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:16.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.423+0000 7fedcd600640 1 -- 172.21.15.151:0/3750567459 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc401e070 con 0x7fedb8004100 2024-09-16T15:40:16.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.423+0000 7fedcd600640 1 -- 172.21.15.151:0/3750567459 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fedc4004050 con 0x7fedb8004100 2024-09-16T15:40:16.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.423+0000 7fedd6c00640 1 -- 172.21.15.151:0/3750567459 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedd01f82b0 con 0x7fedb8004100 2024-09-16T15:40:16.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.423+0000 7fedcd600640 1 -- 172.21.15.151:0/3750567459 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc400fea0 con 0x7fedb8004100 2024-09-16T15:40:16.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.423+0000 7fedd6c00640 1 -- 172.21.15.151:0/3750567459 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedd01f8870 con 0x7fedb8004100 2024-09-16T15:40:16.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.424+0000 7fedd6c00640 1 -- 172.21.15.151:0/3750567459 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedd0073d40 con 0x7fedb8004100 2024-09-16T15:40:16.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.425+0000 7fedcd600640 1 -- 172.21.15.151:0/3750567459 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fedc4021070 con 0x7fedb8004100 2024-09-16T15:40:16.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.426+0000 7fedcd600640 1 --2- 172.21.15.151:0/3750567459 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fedb0078880 0x7fedb007ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:16.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.426+0000 7fedcd600640 1 -- 172.21.15.151:0/3750567459 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fedc409b300 con 0x7fedb8004100 2024-09-16T15:40:16.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.426+0000 7fedd4e00640 1 --2- 172.21.15.151:0/3750567459 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fedb0078880 0x7fedb007ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:16.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.427+0000 7fedd4e00640 1 --2- 172.21.15.151:0/3750567459 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fedb0078880 0x7fedb007ad40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fedbc032000 tx=0x7fedbc001360 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:16.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.430+0000 7fedcd600640 1 -- 172.21.15.151:0/3750567459 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fedc4063350 con 0x7fedb8004100 2024-09-16T15:40:16.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.615+0000 7fedd6c00640 1 -- 172.21.15.151:0/3750567459 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fedd00c6210 con 0x7fedb0078880 2024-09-16T15:40:16.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.620+0000 7fedcd600640 1 -- 172.21.15.151:0/3750567459 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+241325 (secure 0 0 0) 0x7fedd00c6210 con 0x7fedb0078880 2024-09-16T15:40:16.624 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:40:16.625 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-09-16T15:40:16.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.623+0000 7fedd6c00640 1 -- 172.21.15.151:0/3750567459 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fedb0078880 msgr2=0x7fedb007ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:16.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.623+0000 7fedd6c00640 1 --2- 172.21.15.151:0/3750567459 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fedb0078880 0x7fedb007ad40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fedbc032000 tx=0x7fedbc001360 comp rx=0 tx=0).stop 2024-09-16T15:40:16.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.623+0000 7fedd6c00640 1 -- 172.21.15.151:0/3750567459 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fedb8004100 msgr2=0x7fedd0159370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:16.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.623+0000 7fedd6c00640 1 --2- 172.21.15.151:0/3750567459 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fedb8004100 0x7fedd0159370 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fedc4011b10 tx=0x7fedc4009e90 comp rx=0 tx=0).stop 2024-09-16T15:40:16.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.623+0000 7fedd6c00640 1 -- 172.21.15.151:0/3750567459 shutdown_connections 2024-09-16T15:40:16.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.623+0000 7fedd6c00640 1 --2- 172.21.15.151:0/3750567459 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fedb0078880 0x7fedb007ad40 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:16.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.623+0000 7fedd6c00640 1 --2- 172.21.15.151:0/3750567459 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fedd00cd9b0 0x7fedd00cde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:16.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.623+0000 7fedd6c00640 1 --2- 172.21.15.151:0/3750567459 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fedd00cecb0 0x7fedd00cf130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:16.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.623+0000 7fedd6c00640 1 --2- 172.21.15.151:0/3750567459 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fedb8004100 0x7fedd0159370 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:16.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.623+0000 7fedd6c00640 1 -- 172.21.15.151:0/3750567459 >> 172.21.15.151:0/3750567459 conn(0x7fedd00cfa20 msgr2=0x7fedd00cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:16.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.624+0000 7fedd6c00640 1 -- 172.21.15.151:0/3750567459 shutdown_connections 2024-09-16T15:40:16.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.624+0000 7fedd6c00640 1 -- 172.21.15.151:0/3750567459 wait complete. 2024-09-16T15:40:16.642 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":61,"stamp":"2024-09-16T15:40:15.588969+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590390,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":117,"num_write_kb":1629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":78,"acting":78,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":28,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220160,"kb_used_data":3904,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754560,"statfs":{"total":773094113280,"available":772868669440,"internally_reserved":0,"allocated":3997696,"data_stored":2234504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12738,"internal_metadata":221236798},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.183547"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.435856+0000","last_change":"2024-09-16T15:40:12.444964+0000","last_active":"2024-09-16T15:40:13.435856+0000","last_peered":"2024-09-16T15:40:13.435856+0000","last_clean":"2024-09-16T15:40:13.435856+0000","last_became_active":"2024-09-16T15:40:12.442884+0000","last_became_peered":"2024-09-16T15:40:12.442884+0000","last_unstale":"2024-09-16T15:40:13.435856+0000","last_undegraded":"2024-09-16T15:40:13.435856+0000","last_fullsized":"2024-09-16T15:40:13.435856+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:06:23.829480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.436268+0000","last_change":"2024-09-16T15:40:12.446472+0000","last_active":"2024-09-16T15:40:13.436268+0000","last_peered":"2024-09-16T15:40:13.436268+0000","last_clean":"2024-09-16T15:40:13.436268+0000","last_became_active":"2024-09-16T15:40:12.446204+0000","last_became_peered":"2024-09-16T15:40:12.446204+0000","last_unstale":"2024-09-16T15:40:13.436268+0000","last_undegraded":"2024-09-16T15:40:13.436268+0000","last_fullsized":"2024-09-16T15:40:13.436268+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:04:11.653249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.436116+0000","last_change":"2024-09-16T15:40:12.446725+0000","last_active":"2024-09-16T15:40:13.436116+0000","last_peered":"2024-09-16T15:40:13.436116+0000","last_clean":"2024-09-16T15:40:13.436116+0000","last_became_active":"2024-09-16T15:40:12.441589+0000","last_became_peered":"2024-09-16T15:40:12.441589+0000","last_unstale":"2024-09-16T15:40:13.436116+0000","last_undegraded":"2024-09-16T15:40:13.436116+0000","last_fullsized":"2024-09-16T15:40:13.436116+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:50:22.432002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.411274+0000","last_change":"2024-09-16T15:40:10.411274+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.411274+0000","last_undegraded":"2024-09-16T15:40:10.411274+0000","last_fullsized":"2024-09-16T15:40:10.411274+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.408093+0000","last_change":"2024-09-16T15:40:10.408093+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.408093+0000","last_undegraded":"2024-09-16T15:40:10.408093+0000","last_fullsized":"2024-09-16T15:40:10.408093+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"34'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.436310+0000","last_change":"2024-09-16T15:40:11.429304+0000","last_active":"2024-09-16T15:40:13.436310+0000","last_peered":"2024-09-16T15:40:13.436310+0000","last_clean":"2024-09-16T15:40:13.436310+0000","last_became_active":"2024-09-16T15:40:11.429029+0000","last_became_peered":"2024-09-16T15:40:11.429029+0000","last_unstale":"2024-09-16T15:40:13.436310+0000","last_undegraded":"2024-09-16T15:40:13.436310+0000","last_fullsized":"2024-09-16T15:40:13.436310+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:31:10.242510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.408118+0000","last_change":"2024-09-16T15:40:10.408118+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.408118+0000","last_undegraded":"2024-09-16T15:40:10.408118+0000","last_fullsized":"2024-09-16T15:40:10.408118+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.567666+0000","last_change":"2024-09-16T15:40:10.567666+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.567666+0000","last_undegraded":"2024-09-16T15:40:10.567666+0000","last_fullsized":"2024-09-16T15:40:10.567666+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.567420+0000","last_change":"2024-09-16T15:40:10.567420+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.567420+0000","last_undegraded":"2024-09-16T15:40:10.567420+0000","last_fullsized":"2024-09-16T15:40:10.567420+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.435951+0000","last_change":"2024-09-16T15:40:11.431791+0000","last_active":"2024-09-16T15:40:13.435951+0000","last_peered":"2024-09-16T15:40:13.435951+0000","last_clean":"2024-09-16T15:40:13.435951+0000","last_became_active":"2024-09-16T15:40:11.431517+0000","last_became_peered":"2024-09-16T15:40:11.431517+0000","last_unstale":"2024-09-16T15:40:13.435951+0000","last_undegraded":"2024-09-16T15:40:13.435951+0000","last_fullsized":"2024-09-16T15:40:13.435951+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:17:20.232340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.566960+0000","last_change":"2024-09-16T15:40:10.566960+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.566960+0000","last_undegraded":"2024-09-16T15:40:10.566960+0000","last_fullsized":"2024-09-16T15:40:10.566960+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.408736+0000","last_change":"2024-09-16T15:40:10.408736+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.408736+0000","last_undegraded":"2024-09-16T15:40:10.408736+0000","last_fullsized":"2024-09-16T15:40:10.408736+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.408900+0000","last_change":"2024-09-16T15:40:10.408900+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.408900+0000","last_undegraded":"2024-09-16T15:40:10.408900+0000","last_fullsized":"2024-09-16T15:40:10.408900+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.412253+0000","last_change":"2024-09-16T15:40:10.412253+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.412253+0000","last_undegraded":"2024-09-16T15:40:10.412253+0000","last_fullsized":"2024-09-16T15:40:10.412253+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.412131+0000","last_change":"2024-09-16T15:40:10.412131+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.412131+0000","last_undegraded":"2024-09-16T15:40:10.412131+0000","last_fullsized":"2024-09-16T15:40:10.412131+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.567001+0000","last_change":"2024-09-16T15:40:10.567001+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.567001+0000","last_undegraded":"2024-09-16T15:40:10.567001+0000","last_fullsized":"2024-09-16T15:40:10.567001+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.408382+0000","last_change":"2024-09-16T15:40:10.408382+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.408382+0000","last_undegraded":"2024-09-16T15:40:10.408382+0000","last_fullsized":"2024-09-16T15:40:10.408382+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.408111+0000","last_change":"2024-09-16T15:40:10.408111+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.408111+0000","last_undegraded":"2024-09-16T15:40:10.408111+0000","last_fullsized":"2024-09-16T15:40:10.408111+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"34'8","reported_seq":23,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.435447+0000","last_change":"2024-09-16T15:40:11.425677+0000","last_active":"2024-09-16T15:40:13.435447+0000","last_peered":"2024-09-16T15:40:13.435447+0000","last_clean":"2024-09-16T15:40:13.435447+0000","last_became_active":"2024-09-16T15:40:11.425390+0000","last_became_peered":"2024-09-16T15:40:11.425390+0000","last_unstale":"2024-09-16T15:40:13.435447+0000","last_undegraded":"2024-09-16T15:40:13.435447+0000","last_fullsized":"2024-09-16T15:40:13.435447+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:02:34.885995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"34'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.436125+0000","last_change":"2024-09-16T15:40:11.433816+0000","last_active":"2024-09-16T15:40:13.436125+0000","last_peered":"2024-09-16T15:40:13.436125+0000","last_clean":"2024-09-16T15:40:13.436125+0000","last_became_active":"2024-09-16T15:40:11.433028+0000","last_became_peered":"2024-09-16T15:40:11.433028+0000","last_unstale":"2024-09-16T15:40:13.436125+0000","last_undegraded":"2024-09-16T15:40:13.436125+0000","last_fullsized":"2024-09-16T15:40:13.436125+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:05:55.235854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.567075+0000","last_change":"2024-09-16T15:40:10.567075+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.567075+0000","last_undegraded":"2024-09-16T15:40:10.567075+0000","last_fullsized":"2024-09-16T15:40:10.567075+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.412002+0000","last_change":"2024-09-16T15:40:10.412002+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.412002+0000","last_undegraded":"2024-09-16T15:40:10.412002+0000","last_fullsized":"2024-09-16T15:40:10.412002+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.436182+0000","last_change":"2024-09-16T15:40:12.438579+0000","last_active":"2024-09-16T15:40:13.436182+0000","last_peered":"2024-09-16T15:40:13.436182+0000","last_clean":"2024-09-16T15:40:13.436182+0000","last_became_active":"2024-09-16T15:40:12.437311+0000","last_became_peered":"2024-09-16T15:40:12.437311+0000","last_unstale":"2024-09-16T15:40:13.436182+0000","last_undegraded":"2024-09-16T15:40:13.436182+0000","last_fullsized":"2024-09-16T15:40:13.436182+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:04:48.883149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.406699+0000","last_change":"2024-09-16T15:40:10.406699+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.406699+0000","last_undegraded":"2024-09-16T15:40:10.406699+0000","last_fullsized":"2024-09-16T15:40:10.406699+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.408267+0000","last_change":"2024-09-16T15:40:10.408267+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.408267+0000","last_undegraded":"2024-09-16T15:40:10.408267+0000","last_fullsized":"2024-09-16T15:40:10.408267+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.567019+0000","last_change":"2024-09-16T15:40:10.567019+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.567019+0000","last_undegraded":"2024-09-16T15:40:10.567019+0000","last_fullsized":"2024-09-16T15:40:10.567019+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.410409+0000","last_change":"2024-09-16T15:40:10.410409+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.410409+0000","last_undegraded":"2024-09-16T15:40:10.410409+0000","last_fullsized":"2024-09-16T15:40:10.410409+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.411274+0000","last_change":"2024-09-16T15:40:10.411274+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.411274+0000","last_undegraded":"2024-09-16T15:40:10.411274+0000","last_fullsized":"2024-09-16T15:40:10.411274+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.436365+0000","last_change":"2024-09-16T15:40:12.437260+0000","last_active":"2024-09-16T15:40:13.436365+0000","last_peered":"2024-09-16T15:40:13.436365+0000","last_clean":"2024-09-16T15:40:13.436365+0000","last_became_active":"2024-09-16T15:40:12.437001+0000","last_became_peered":"2024-09-16T15:40:12.437001+0000","last_unstale":"2024-09-16T15:40:13.436365+0000","last_undegraded":"2024-09-16T15:40:13.436365+0000","last_fullsized":"2024-09-16T15:40:13.436365+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:24:21.290065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.412057+0000","last_change":"2024-09-16T15:40:10.412057+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.412057+0000","last_undegraded":"2024-09-16T15:40:10.412057+0000","last_fullsized":"2024-09-16T15:40:10.412057+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.408276+0000","last_change":"2024-09-16T15:40:10.408276+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.408276+0000","last_undegraded":"2024-09-16T15:40:10.408276+0000","last_fullsized":"2024-09-16T15:40:10.408276+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.436527+0000","last_change":"2024-09-16T15:40:12.451657+0000","last_active":"2024-09-16T15:40:13.436527+0000","last_peered":"2024-09-16T15:40:13.436527+0000","last_clean":"2024-09-16T15:40:13.436527+0000","last_became_active":"2024-09-16T15:40:12.450795+0000","last_became_peered":"2024-09-16T15:40:12.450795+0000","last_unstale":"2024-09-16T15:40:13.436527+0000","last_undegraded":"2024-09-16T15:40:13.436527+0000","last_fullsized":"2024-09-16T15:40:13.436527+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:46:24.970506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.411635+0000","last_change":"2024-09-16T15:40:10.411635+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.411635+0000","last_undegraded":"2024-09-16T15:40:10.411635+0000","last_fullsized":"2024-09-16T15:40:10.411635+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.411513+0000","last_change":"2024-09-16T15:40:10.411513+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.411513+0000","last_undegraded":"2024-09-16T15:40:10.411513+0000","last_fullsized":"2024-09-16T15:40:10.411513+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.411507+0000","last_change":"2024-09-16T15:40:10.411507+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.411507+0000","last_undegraded":"2024-09-16T15:40:10.411507+0000","last_fullsized":"2024-09-16T15:40:10.411507+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.566998+0000","last_change":"2024-09-16T15:40:10.566998+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.566998+0000","last_undegraded":"2024-09-16T15:40:10.566998+0000","last_fullsized":"2024-09-16T15:40:10.566998+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.408225+0000","last_change":"2024-09-16T15:40:10.408225+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.408225+0000","last_undegraded":"2024-09-16T15:40:10.408225+0000","last_fullsized":"2024-09-16T15:40:10.408225+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.411572+0000","last_change":"2024-09-16T15:40:10.411572+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.411572+0000","last_undegraded":"2024-09-16T15:40:10.411572+0000","last_fullsized":"2024-09-16T15:40:10.411572+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":129,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-16T15:40:10.410693+0000","last_change":"2024-09-16T15:39:12.698813+0000","last_active":"2024-09-16T15:40:10.410693+0000","last_peered":"2024-09-16T15:40:10.410693+0000","last_clean":"2024-09-16T15:40:10.410693+0000","last_became_active":"2024-09-16T15:39:12.697695+0000","last_became_peered":"2024-09-16T15:39:12.697695+0000","last_unstale":"2024-09-16T15:40:10.410693+0000","last_undegraded":"2024-09-16T15:40:10.410693+0000","last_fullsized":"2024-09-16T15:40:10.410693+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_clean_scrub_stamp":"2024-09-16T15:39:11.684967+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:02.061081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":12,"acting":12,"num_store_stats":1},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":22,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11,"ondisk_log_size":11,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509457,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":352256,"data_stored":132208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509456,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27952,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343888,"statfs":{"total":96636764160,"available":96608141312,"internally_reserved":0,"allocated":946176,"data_stored":722576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509455,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":339968,"data_stored":126391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509455,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":339968,"data_stored":126391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509455,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":352256,"data_stored":132208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509456,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27948,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343892,"statfs":{"total":96636764160,"available":96608145408,"internally_reserved":0,"allocated":933888,"data_stored":716759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509454,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":339968,"data_stored":126391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542162,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":393216,"data_stored":151580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":27654604},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:40:16.646 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:40:16.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.838+0000 7ff6a6c00640 1 -- 172.21.15.151:0/2940758486 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6a00ccf00 msgr2=0x7ff6a00b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:16.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.838+0000 7ff6a6c00640 1 --2- 172.21.15.151:0/2940758486 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6a00ccf00 0x7ff6a00b2440 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff68c009830 tx=0x7ff68c02a480 comp rx=0 tx=0).stop 2024-09-16T15:40:16.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.838+0000 7ff6a6c00640 1 -- 172.21.15.151:0/2940758486 shutdown_connections 2024-09-16T15:40:16.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.838+0000 7ff6a6c00640 1 --2- 172.21.15.151:0/2940758486 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6a00ccf00 0x7ff6a00b2440 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:16.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.838+0000 7ff6a6c00640 1 -- 172.21.15.151:0/2940758486 >> 172.21.15.151:0/2940758486 conn(0x7ff6a00b1950 msgr2=0x7ff6a00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:16.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.838+0000 7ff6a6c00640 1 -- 172.21.15.151:0/2940758486 shutdown_connections 2024-09-16T15:40:16.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.838+0000 7ff6a6c00640 1 -- 172.21.15.151:0/2940758486 wait complete. 2024-09-16T15:40:16.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.840+0000 7ff6a6c00640 1 Processor -- start 2024-09-16T15:40:16.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.840+0000 7ff6a6c00640 1 -- start start 2024-09-16T15:40:16.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.841+0000 7ff6a6c00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff6a0073fb0 0x7ff6a0146210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:16.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.841+0000 7ff6a6c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6a00ccf00 0x7ff6a0148910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:16.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.841+0000 7ff6a6c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6a015f5a0 0x7ff6a00cf700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:16.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.841+0000 7ff6a6c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a006f9f0 con 0x7ff6a015f5a0 2024-09-16T15:40:16.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.841+0000 7ff6a6c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a006f870 con 0x7ff6a00ccf00 2024-09-16T15:40:16.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.841+0000 7ff6a6c00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7ff6a0074420 con 0x7ff6a0073fb0 2024-09-16T15:40:16.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.841+0000 7ff6a4c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6a015f5a0 0x7ff6a00cf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:16.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.841+0000 7ff6a4c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6a015f5a0 0x7ff6a00cf700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37328/0 (socket says 172.21.15.151:37328) 2024-09-16T15:40:16.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.841+0000 7ff6a4c00640 1 -- 172.21.15.151:0/1336107856 learned_addr learned my addr 172.21.15.151:0/1336107856 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:40:16.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.841+0000 7ff69f400640 1 --2- 172.21.15.151:0/1336107856 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6a00ccf00 0x7ff6a0148910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:16.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.841+0000 7ff69fe00640 1 --2- 172.21.15.151:0/1336107856 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff6a0073fb0 0x7ff6a0146210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:16.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.841+0000 7ff6a4c00640 1 -- 172.21.15.151:0/1336107856 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff6a0073fb0 msgr2=0x7ff6a0146210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:16.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.842+0000 7ff6a4c00640 1 --2- 172.21.15.151:0/1336107856 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff6a0073fb0 0x7ff6a0146210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:16.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.842+0000 7ff6a4c00640 1 -- 172.21.15.151:0/1336107856 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6a00ccf00 msgr2=0x7ff6a0148910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:16.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.842+0000 7ff6a4c00640 1 --2- 172.21.15.151:0/1336107856 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6a00ccf00 0x7ff6a0148910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:16.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.842+0000 7ff6a4c00640 1 -- 172.21.15.151:0/1336107856 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6a01550f0 con 0x7ff6a015f5a0 2024-09-16T15:40:16.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.842+0000 7ff69fe00640 1 --2- 172.21.15.151:0/1336107856 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff6a0073fb0 0x7ff6a0146210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:40:16.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.842+0000 7ff69f400640 1 --2- 172.21.15.151:0/1336107856 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6a00ccf00 0x7ff6a0148910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:40:16.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.842+0000 7ff6a4c00640 1 --2- 172.21.15.151:0/1336107856 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6a015f5a0 0x7ff6a00cf700 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff69800cbc0 tx=0x7ff69800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:16.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.842+0000 7ff697e00640 1 -- 172.21.15.151:0/1336107856 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff698016020 con 0x7ff6a015f5a0 2024-09-16T15:40:16.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.842+0000 7ff697e00640 1 -- 172.21.15.151:0/1336107856 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff69800add0 con 0x7ff6a015f5a0 2024-09-16T15:40:16.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.842+0000 7ff697e00640 1 -- 172.21.15.151:0/1336107856 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff698012680 con 0x7ff6a015f5a0 2024-09-16T15:40:16.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.842+0000 7ff6a6c00640 1 -- 172.21.15.151:0/1336107856 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6a0156130 con 0x7ff6a015f5a0 2024-09-16T15:40:16.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.842+0000 7ff6a6c00640 1 -- 172.21.15.151:0/1336107856 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6a0149220 con 0x7ff6a015f5a0 2024-09-16T15:40:16.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.843+0000 7ff6a6c00640 1 -- 172.21.15.151:0/1336107856 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6a0156300 con 0x7ff6a015f5a0 2024-09-16T15:40:16.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.844+0000 7ff697e00640 1 -- 172.21.15.151:0/1336107856 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7ff698019020 con 0x7ff6a015f5a0 2024-09-16T15:40:16.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.845+0000 7ff697e00640 1 --2- 172.21.15.151:0/1336107856 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff680078850 0x7ff68007ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:16.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.845+0000 7ff697e00640 1 -- 172.21.15.151:0/1336107856 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7ff69809ad50 con 0x7ff6a015f5a0 2024-09-16T15:40:16.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.846+0000 7ff69fe00640 1 --2- 172.21.15.151:0/1336107856 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff680078850 0x7ff68007ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:16.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.847+0000 7ff69fe00640 1 --2- 172.21.15.151:0/1336107856 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff680078850 0x7ff68007ad10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff690006fd0 tx=0x7ff690008040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:16.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:16.849+0000 7ff697e00640 1 -- 172.21.15.151:0/1336107856 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff698020030 con 0x7ff6a015f5a0 2024-09-16T15:40:16.970 DEBUG:teuthology.orchestra.run.smithi151:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:40:16.974 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:40:17.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.035+0000 7ff6a6c00640 1 -- 172.21.15.151:0/1336107856 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff6a00b2440 con 0x7ff680078850 2024-09-16T15:40:17.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.040+0000 7ff697e00640 1 -- 172.21.15.151:0/1336107856 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+241325 (secure 0 0 0) 0x7ff6a00b2440 con 0x7ff680078850 2024-09-16T15:40:17.045 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:40:17.046 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-09-16T15:40:17.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.044+0000 7ff6a6c00640 1 -- 172.21.15.151:0/1336107856 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff680078850 msgr2=0x7ff68007ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:17.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.044+0000 7ff6a6c00640 1 --2- 172.21.15.151:0/1336107856 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff680078850 0x7ff68007ad10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff690006fd0 tx=0x7ff690008040 comp rx=0 tx=0).stop 2024-09-16T15:40:17.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.044+0000 7ff6a6c00640 1 -- 172.21.15.151:0/1336107856 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6a015f5a0 msgr2=0x7ff6a00cf700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:17.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.044+0000 7ff6a6c00640 1 --2- 172.21.15.151:0/1336107856 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6a015f5a0 0x7ff6a00cf700 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff69800cbc0 tx=0x7ff69800a430 comp rx=0 tx=0).stop 2024-09-16T15:40:17.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.044+0000 7ff6a6c00640 1 -- 172.21.15.151:0/1336107856 shutdown_connections 2024-09-16T15:40:17.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.044+0000 7ff6a6c00640 1 --2- 172.21.15.151:0/1336107856 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff680078850 0x7ff68007ad10 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:17.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.044+0000 7ff6a6c00640 1 --2- 172.21.15.151:0/1336107856 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6a00ccf00 0x7ff6a0148910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:17.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.044+0000 7ff6a6c00640 1 --2- 172.21.15.151:0/1336107856 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6a015f5a0 0x7ff6a00cf700 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:17.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.044+0000 7ff6a6c00640 1 --2- 172.21.15.151:0/1336107856 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff6a0073fb0 0x7ff6a0146210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:17.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.044+0000 7ff6a6c00640 1 -- 172.21.15.151:0/1336107856 >> 172.21.15.151:0/1336107856 conn(0x7ff6a00b1950 msgr2=0x7ff6a015e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:17.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.044+0000 7ff6a6c00640 1 -- 172.21.15.151:0/1336107856 shutdown_connections 2024-09-16T15:40:17.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.044+0000 7ff6a6c00640 1 -- 172.21.15.151:0/1336107856 wait complete. 2024-09-16T15:40:17.062 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":61,"stamp":"2024-09-16T15:40:15.588969+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590390,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":117,"num_write_kb":1629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":78,"acting":78,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":28,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220160,"kb_used_data":3904,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754560,"statfs":{"total":773094113280,"available":772868669440,"internally_reserved":0,"allocated":3997696,"data_stored":2234504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12738,"internal_metadata":221236798},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.183547"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.435856+0000","last_change":"2024-09-16T15:40:12.444964+0000","last_active":"2024-09-16T15:40:13.435856+0000","last_peered":"2024-09-16T15:40:13.435856+0000","last_clean":"2024-09-16T15:40:13.435856+0000","last_became_active":"2024-09-16T15:40:12.442884+0000","last_became_peered":"2024-09-16T15:40:12.442884+0000","last_unstale":"2024-09-16T15:40:13.435856+0000","last_undegraded":"2024-09-16T15:40:13.435856+0000","last_fullsized":"2024-09-16T15:40:13.435856+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:06:23.829480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.436268+0000","last_change":"2024-09-16T15:40:12.446472+0000","last_active":"2024-09-16T15:40:13.436268+0000","last_peered":"2024-09-16T15:40:13.436268+0000","last_clean":"2024-09-16T15:40:13.436268+0000","last_became_active":"2024-09-16T15:40:12.446204+0000","last_became_peered":"2024-09-16T15:40:12.446204+0000","last_unstale":"2024-09-16T15:40:13.436268+0000","last_undegraded":"2024-09-16T15:40:13.436268+0000","last_fullsized":"2024-09-16T15:40:13.436268+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:04:11.653249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.436116+0000","last_change":"2024-09-16T15:40:12.446725+0000","last_active":"2024-09-16T15:40:13.436116+0000","last_peered":"2024-09-16T15:40:13.436116+0000","last_clean":"2024-09-16T15:40:13.436116+0000","last_became_active":"2024-09-16T15:40:12.441589+0000","last_became_peered":"2024-09-16T15:40:12.441589+0000","last_unstale":"2024-09-16T15:40:13.436116+0000","last_undegraded":"2024-09-16T15:40:13.436116+0000","last_fullsized":"2024-09-16T15:40:13.436116+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:50:22.432002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.411274+0000","last_change":"2024-09-16T15:40:10.411274+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.411274+0000","last_undegraded":"2024-09-16T15:40:10.411274+0000","last_fullsized":"2024-09-16T15:40:10.411274+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.408093+0000","last_change":"2024-09-16T15:40:10.408093+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.408093+0000","last_undegraded":"2024-09-16T15:40:10.408093+0000","last_fullsized":"2024-09-16T15:40:10.408093+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"34'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.436310+0000","last_change":"2024-09-16T15:40:11.429304+0000","last_active":"2024-09-16T15:40:13.436310+0000","last_peered":"2024-09-16T15:40:13.436310+0000","last_clean":"2024-09-16T15:40:13.436310+0000","last_became_active":"2024-09-16T15:40:11.429029+0000","last_became_peered":"2024-09-16T15:40:11.429029+0000","last_unstale":"2024-09-16T15:40:13.436310+0000","last_undegraded":"2024-09-16T15:40:13.436310+0000","last_fullsized":"2024-09-16T15:40:13.436310+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:31:10.242510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.408118+0000","last_change":"2024-09-16T15:40:10.408118+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.408118+0000","last_undegraded":"2024-09-16T15:40:10.408118+0000","last_fullsized":"2024-09-16T15:40:10.408118+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.567666+0000","last_change":"2024-09-16T15:40:10.567666+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.567666+0000","last_undegraded":"2024-09-16T15:40:10.567666+0000","last_fullsized":"2024-09-16T15:40:10.567666+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.567420+0000","last_change":"2024-09-16T15:40:10.567420+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.567420+0000","last_undegraded":"2024-09-16T15:40:10.567420+0000","last_fullsized":"2024-09-16T15:40:10.567420+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.435951+0000","last_change":"2024-09-16T15:40:11.431791+0000","last_active":"2024-09-16T15:40:13.435951+0000","last_peered":"2024-09-16T15:40:13.435951+0000","last_clean":"2024-09-16T15:40:13.435951+0000","last_became_active":"2024-09-16T15:40:11.431517+0000","last_became_peered":"2024-09-16T15:40:11.431517+0000","last_unstale":"2024-09-16T15:40:13.435951+0000","last_undegraded":"2024-09-16T15:40:13.435951+0000","last_fullsized":"2024-09-16T15:40:13.435951+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:17:20.232340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.566960+0000","last_change":"2024-09-16T15:40:10.566960+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.566960+0000","last_undegraded":"2024-09-16T15:40:10.566960+0000","last_fullsized":"2024-09-16T15:40:10.566960+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.408736+0000","last_change":"2024-09-16T15:40:10.408736+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.408736+0000","last_undegraded":"2024-09-16T15:40:10.408736+0000","last_fullsized":"2024-09-16T15:40:10.408736+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.408900+0000","last_change":"2024-09-16T15:40:10.408900+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.408900+0000","last_undegraded":"2024-09-16T15:40:10.408900+0000","last_fullsized":"2024-09-16T15:40:10.408900+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.412253+0000","last_change":"2024-09-16T15:40:10.412253+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.412253+0000","last_undegraded":"2024-09-16T15:40:10.412253+0000","last_fullsized":"2024-09-16T15:40:10.412253+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.412131+0000","last_change":"2024-09-16T15:40:10.412131+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.412131+0000","last_undegraded":"2024-09-16T15:40:10.412131+0000","last_fullsized":"2024-09-16T15:40:10.412131+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.567001+0000","last_change":"2024-09-16T15:40:10.567001+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.567001+0000","last_undegraded":"2024-09-16T15:40:10.567001+0000","last_fullsized":"2024-09-16T15:40:10.567001+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.408382+0000","last_change":"2024-09-16T15:40:10.408382+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.408382+0000","last_undegraded":"2024-09-16T15:40:10.408382+0000","last_fullsized":"2024-09-16T15:40:10.408382+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.408111+0000","last_change":"2024-09-16T15:40:10.408111+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.408111+0000","last_undegraded":"2024-09-16T15:40:10.408111+0000","last_fullsized":"2024-09-16T15:40:10.408111+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"34'8","reported_seq":23,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.435447+0000","last_change":"2024-09-16T15:40:11.425677+0000","last_active":"2024-09-16T15:40:13.435447+0000","last_peered":"2024-09-16T15:40:13.435447+0000","last_clean":"2024-09-16T15:40:13.435447+0000","last_became_active":"2024-09-16T15:40:11.425390+0000","last_became_peered":"2024-09-16T15:40:11.425390+0000","last_unstale":"2024-09-16T15:40:13.435447+0000","last_undegraded":"2024-09-16T15:40:13.435447+0000","last_fullsized":"2024-09-16T15:40:13.435447+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:02:34.885995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"34'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.436125+0000","last_change":"2024-09-16T15:40:11.433816+0000","last_active":"2024-09-16T15:40:13.436125+0000","last_peered":"2024-09-16T15:40:13.436125+0000","last_clean":"2024-09-16T15:40:13.436125+0000","last_became_active":"2024-09-16T15:40:11.433028+0000","last_became_peered":"2024-09-16T15:40:11.433028+0000","last_unstale":"2024-09-16T15:40:13.436125+0000","last_undegraded":"2024-09-16T15:40:13.436125+0000","last_fullsized":"2024-09-16T15:40:13.436125+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:05:55.235854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.567075+0000","last_change":"2024-09-16T15:40:10.567075+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.567075+0000","last_undegraded":"2024-09-16T15:40:10.567075+0000","last_fullsized":"2024-09-16T15:40:10.567075+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.412002+0000","last_change":"2024-09-16T15:40:10.412002+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.412002+0000","last_undegraded":"2024-09-16T15:40:10.412002+0000","last_fullsized":"2024-09-16T15:40:10.412002+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.436182+0000","last_change":"2024-09-16T15:40:12.438579+0000","last_active":"2024-09-16T15:40:13.436182+0000","last_peered":"2024-09-16T15:40:13.436182+0000","last_clean":"2024-09-16T15:40:13.436182+0000","last_became_active":"2024-09-16T15:40:12.437311+0000","last_became_peered":"2024-09-16T15:40:12.437311+0000","last_unstale":"2024-09-16T15:40:13.436182+0000","last_undegraded":"2024-09-16T15:40:13.436182+0000","last_fullsized":"2024-09-16T15:40:13.436182+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:04:48.883149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.406699+0000","last_change":"2024-09-16T15:40:10.406699+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.406699+0000","last_undegraded":"2024-09-16T15:40:10.406699+0000","last_fullsized":"2024-09-16T15:40:10.406699+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.408267+0000","last_change":"2024-09-16T15:40:10.408267+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.408267+0000","last_undegraded":"2024-09-16T15:40:10.408267+0000","last_fullsized":"2024-09-16T15:40:10.408267+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.567019+0000","last_change":"2024-09-16T15:40:10.567019+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.567019+0000","last_undegraded":"2024-09-16T15:40:10.567019+0000","last_fullsized":"2024-09-16T15:40:10.567019+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.410409+0000","last_change":"2024-09-16T15:40:10.410409+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.410409+0000","last_undegraded":"2024-09-16T15:40:10.410409+0000","last_fullsized":"2024-09-16T15:40:10.410409+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.411274+0000","last_change":"2024-09-16T15:40:10.411274+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.411274+0000","last_undegraded":"2024-09-16T15:40:10.411274+0000","last_fullsized":"2024-09-16T15:40:10.411274+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.436365+0000","last_change":"2024-09-16T15:40:12.437260+0000","last_active":"2024-09-16T15:40:13.436365+0000","last_peered":"2024-09-16T15:40:13.436365+0000","last_clean":"2024-09-16T15:40:13.436365+0000","last_became_active":"2024-09-16T15:40:12.437001+0000","last_became_peered":"2024-09-16T15:40:12.437001+0000","last_unstale":"2024-09-16T15:40:13.436365+0000","last_undegraded":"2024-09-16T15:40:13.436365+0000","last_fullsized":"2024-09-16T15:40:13.436365+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:24:21.290065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.412057+0000","last_change":"2024-09-16T15:40:10.412057+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.412057+0000","last_undegraded":"2024-09-16T15:40:10.412057+0000","last_fullsized":"2024-09-16T15:40:10.412057+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.408276+0000","last_change":"2024-09-16T15:40:10.408276+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.408276+0000","last_undegraded":"2024-09-16T15:40:10.408276+0000","last_fullsized":"2024-09-16T15:40:10.408276+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.436527+0000","last_change":"2024-09-16T15:40:12.451657+0000","last_active":"2024-09-16T15:40:13.436527+0000","last_peered":"2024-09-16T15:40:13.436527+0000","last_clean":"2024-09-16T15:40:13.436527+0000","last_became_active":"2024-09-16T15:40:12.450795+0000","last_became_peered":"2024-09-16T15:40:12.450795+0000","last_unstale":"2024-09-16T15:40:13.436527+0000","last_undegraded":"2024-09-16T15:40:13.436527+0000","last_fullsized":"2024-09-16T15:40:13.436527+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:46:24.970506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.411635+0000","last_change":"2024-09-16T15:40:10.411635+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.411635+0000","last_undegraded":"2024-09-16T15:40:10.411635+0000","last_fullsized":"2024-09-16T15:40:10.411635+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.411513+0000","last_change":"2024-09-16T15:40:10.411513+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.411513+0000","last_undegraded":"2024-09-16T15:40:10.411513+0000","last_fullsized":"2024-09-16T15:40:10.411513+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.411507+0000","last_change":"2024-09-16T15:40:10.411507+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.411507+0000","last_undegraded":"2024-09-16T15:40:10.411507+0000","last_fullsized":"2024-09-16T15:40:10.411507+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.566998+0000","last_change":"2024-09-16T15:40:10.566998+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.566998+0000","last_undegraded":"2024-09-16T15:40:10.566998+0000","last_fullsized":"2024-09-16T15:40:10.566998+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.408225+0000","last_change":"2024-09-16T15:40:10.408225+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.408225+0000","last_undegraded":"2024-09-16T15:40:10.408225+0000","last_fullsized":"2024-09-16T15:40:10.408225+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.411572+0000","last_change":"2024-09-16T15:40:10.411572+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.411572+0000","last_undegraded":"2024-09-16T15:40:10.411572+0000","last_fullsized":"2024-09-16T15:40:10.411572+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":129,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-16T15:40:10.410693+0000","last_change":"2024-09-16T15:39:12.698813+0000","last_active":"2024-09-16T15:40:10.410693+0000","last_peered":"2024-09-16T15:40:10.410693+0000","last_clean":"2024-09-16T15:40:10.410693+0000","last_became_active":"2024-09-16T15:39:12.697695+0000","last_became_peered":"2024-09-16T15:39:12.697695+0000","last_unstale":"2024-09-16T15:40:10.410693+0000","last_undegraded":"2024-09-16T15:40:10.410693+0000","last_fullsized":"2024-09-16T15:40:10.410693+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_clean_scrub_stamp":"2024-09-16T15:39:11.684967+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:02.061081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":12,"acting":12,"num_store_stats":1},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":22,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11,"ondisk_log_size":11,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509457,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":352256,"data_stored":132208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509456,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27952,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343888,"statfs":{"total":96636764160,"available":96608141312,"internally_reserved":0,"allocated":946176,"data_stored":722576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509455,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":339968,"data_stored":126391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509455,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":339968,"data_stored":126391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509455,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":352256,"data_stored":132208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509456,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27948,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343892,"statfs":{"total":96636764160,"available":96608145408,"internally_reserved":0,"allocated":933888,"data_stored":716759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509454,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":339968,"data_stored":126391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542162,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":393216,"data_stored":151580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":27654604},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:40:17.066 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:40:17.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.259+0000 7f66d4e00640 1 -- 172.21.15.151:0/165167541 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66d0070920 msgr2=0x7f66d01438b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:17.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.259+0000 7f66d4e00640 1 --2- 172.21.15.151:0/165167541 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66d0070920 0x7f66d01438b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f66c0005fd0 tx=0x7f66c002a5c0 comp rx=0 tx=0).stop 2024-09-16T15:40:17.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.259+0000 7f66d4e00640 1 -- 172.21.15.151:0/165167541 shutdown_connections 2024-09-16T15:40:17.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.259+0000 7f66d4e00640 1 --2- 172.21.15.151:0/165167541 >> v2:172.21.15.167:3300/0 conn(0x7f66d00b3f00 0x7f66d00b43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:17.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.259+0000 7f66d4e00640 1 --2- 172.21.15.151:0/165167541 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66d0070920 0x7f66d01438b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:17.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.259+0000 7f66d4e00640 1 -- 172.21.15.151:0/165167541 >> 172.21.15.151:0/165167541 conn(0x7f66d00b1950 msgr2=0x7f66d00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:17.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.259+0000 7f66d4e00640 1 -- 172.21.15.151:0/165167541 shutdown_connections 2024-09-16T15:40:17.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.259+0000 7f66d4e00640 1 -- 172.21.15.151:0/165167541 wait complete. 2024-09-16T15:40:17.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.261+0000 7f66d4e00640 1 Processor -- start 2024-09-16T15:40:17.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.261+0000 7f66d4e00640 1 -- start start 2024-09-16T15:40:17.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.262+0000 7f66d4e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66d006f850 0x7f66d0153110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:17.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.262+0000 7f66d4e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66d0070920 0x7f66d013f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:17.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.262+0000 7f66d4e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f66d00b3f00 0x7f66d014d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:17.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.262+0000 7f66d4e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f66d00b4dc0 con 0x7f66d00b3f00 2024-09-16T15:40:17.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.262+0000 7f66d4e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f66d00b4c40 con 0x7f66d0070920 2024-09-16T15:40:17.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.262+0000 7f66d4e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f66d00b4f40 con 0x7f66d006f850 2024-09-16T15:40:17.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.262+0000 7f66ce800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f66d00b3f00 0x7f66d014d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:17.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.262+0000 7f66ce800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f66d00b3f00 0x7f66d014d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37330/0 (socket says 172.21.15.151:37330) 2024-09-16T15:40:17.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.262+0000 7f66ce800640 1 -- 172.21.15.151:0/500403347 learned_addr learned my addr 172.21.15.151:0/500403347 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:40:17.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.262+0000 7f66cde00640 1 --2- 172.21.15.151:0/500403347 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66d006f850 0x7f66d0153110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:17.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.262+0000 7f66cd400640 1 --2- 172.21.15.151:0/500403347 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66d0070920 0x7f66d013f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:17.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.262+0000 7f66ce800640 1 -- 172.21.15.151:0/500403347 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66d006f850 msgr2=0x7f66d0153110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:17.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.262+0000 7f66ce800640 1 --2- 172.21.15.151:0/500403347 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66d006f850 0x7f66d0153110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:17.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.262+0000 7f66ce800640 1 -- 172.21.15.151:0/500403347 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66d0070920 msgr2=0x7f66d013f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:17.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.262+0000 7f66ce800640 1 --2- 172.21.15.151:0/500403347 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66d0070920 0x7f66d013f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:17.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.262+0000 7f66ce800640 1 -- 172.21.15.151:0/500403347 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66d0153650 con 0x7f66d00b3f00 2024-09-16T15:40:17.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.263+0000 7f66ce800640 1 --2- 172.21.15.151:0/500403347 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f66d00b3f00 0x7f66d014d5c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f66c800af20 tx=0x7f66c800c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:17.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.263+0000 7f66c6000640 1 -- 172.21.15.151:0/500403347 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c8003970 con 0x7f66d00b3f00 2024-09-16T15:40:17.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.263+0000 7f66d4e00640 1 -- 172.21.15.151:0/500403347 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66d009d5d0 con 0x7f66d00b3f00 2024-09-16T15:40:17.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.263+0000 7f66c6000640 1 -- 172.21.15.151:0/500403347 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f66c8003c00 con 0x7f66d00b3f00 2024-09-16T15:40:17.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.263+0000 7f66c6000640 1 -- 172.21.15.151:0/500403347 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c8004020 con 0x7f66d00b3f00 2024-09-16T15:40:17.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.263+0000 7f66d4e00640 1 -- 172.21.15.151:0/500403347 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66d013fb70 con 0x7f66d00b3f00 2024-09-16T15:40:17.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.264+0000 7f66d4e00640 1 -- 172.21.15.151:0/500403347 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66d01438b0 con 0x7f66d00b3f00 2024-09-16T15:40:17.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.265+0000 7f66c6000640 1 -- 172.21.15.151:0/500403347 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f66c801d020 con 0x7f66d00b3f00 2024-09-16T15:40:17.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.266+0000 7f66c6000640 1 --2- 172.21.15.151:0/500403347 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f66b0078850 0x7f66b007ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:17.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.266+0000 7f66c6000640 1 -- 172.21.15.151:0/500403347 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f66c809add0 con 0x7f66d00b3f00 2024-09-16T15:40:17.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.266+0000 7f66cde00640 1 --2- 172.21.15.151:0/500403347 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f66b0078850 0x7f66b007ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:17.269 INFO:tasks.ceph.mds.b.smithi167.stderr:2024-09-16T15:40:17.268+0000 7f0598400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-16T15:40:17.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.267+0000 7f66cde00640 1 --2- 172.21.15.151:0/500403347 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f66b0078850 0x7f66b007ad10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f66c0033000 tx=0x7f66c0004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:17.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.270+0000 7f66c6000640 1 -- 172.21.15.151:0/500403347 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f66d0153650 con 0x7f66d00b3f00 2024-09-16T15:40:17.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.453+0000 7f66d4e00640 1 -- 172.21.15.151:0/500403347 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f66d00cc3b0 con 0x7f66b0078850 2024-09-16T15:40:17.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.458+0000 7f66c6000640 1 -- 172.21.15.151:0/500403347 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+241325 (secure 0 0 0) 0x7f66d00cc3b0 con 0x7f66b0078850 2024-09-16T15:40:17.463 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:40:17.464 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-09-16T15:40:17.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.463+0000 7f66d4e00640 1 -- 172.21.15.151:0/500403347 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f66b0078850 msgr2=0x7f66b007ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:17.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.463+0000 7f66d4e00640 1 --2- 172.21.15.151:0/500403347 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f66b0078850 0x7f66b007ad10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f66c0033000 tx=0x7f66c0004730 comp rx=0 tx=0).stop 2024-09-16T15:40:17.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.463+0000 7f66d4e00640 1 -- 172.21.15.151:0/500403347 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f66d00b3f00 msgr2=0x7f66d014d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:17.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.463+0000 7f66d4e00640 1 --2- 172.21.15.151:0/500403347 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f66d00b3f00 0x7f66d014d5c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f66c800af20 tx=0x7f66c800c7a0 comp rx=0 tx=0).stop 2024-09-16T15:40:17.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.463+0000 7f66d4e00640 1 -- 172.21.15.151:0/500403347 shutdown_connections 2024-09-16T15:40:17.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.463+0000 7f66d4e00640 1 --2- 172.21.15.151:0/500403347 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f66b0078850 0x7f66b007ad10 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:17.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.463+0000 7f66d4e00640 1 --2- 172.21.15.151:0/500403347 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66d0070920 0x7f66d013f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:17.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.463+0000 7f66d4e00640 1 --2- 172.21.15.151:0/500403347 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f66d00b3f00 0x7f66d014d5c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:17.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.463+0000 7f66d4e00640 1 --2- 172.21.15.151:0/500403347 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66d006f850 0x7f66d0153110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:17.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.463+0000 7f66d4e00640 1 -- 172.21.15.151:0/500403347 >> 172.21.15.151:0/500403347 conn(0x7f66d00b1950 msgr2=0x7f66d00cec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:17.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.463+0000 7f66d4e00640 1 -- 172.21.15.151:0/500403347 shutdown_connections 2024-09-16T15:40:17.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.463+0000 7f66d4e00640 1 -- 172.21.15.151:0/500403347 wait complete. 2024-09-16T15:40:17.482 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":61,"stamp":"2024-09-16T15:40:15.588969+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590390,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":117,"num_write_kb":1629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":78,"acting":78,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":28,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220160,"kb_used_data":3904,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754560,"statfs":{"total":773094113280,"available":772868669440,"internally_reserved":0,"allocated":3997696,"data_stored":2234504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12738,"internal_metadata":221236798},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.183547"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.435856+0000","last_change":"2024-09-16T15:40:12.444964+0000","last_active":"2024-09-16T15:40:13.435856+0000","last_peered":"2024-09-16T15:40:13.435856+0000","last_clean":"2024-09-16T15:40:13.435856+0000","last_became_active":"2024-09-16T15:40:12.442884+0000","last_became_peered":"2024-09-16T15:40:12.442884+0000","last_unstale":"2024-09-16T15:40:13.435856+0000","last_undegraded":"2024-09-16T15:40:13.435856+0000","last_fullsized":"2024-09-16T15:40:13.435856+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:06:23.829480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.436268+0000","last_change":"2024-09-16T15:40:12.446472+0000","last_active":"2024-09-16T15:40:13.436268+0000","last_peered":"2024-09-16T15:40:13.436268+0000","last_clean":"2024-09-16T15:40:13.436268+0000","last_became_active":"2024-09-16T15:40:12.446204+0000","last_became_peered":"2024-09-16T15:40:12.446204+0000","last_unstale":"2024-09-16T15:40:13.436268+0000","last_undegraded":"2024-09-16T15:40:13.436268+0000","last_fullsized":"2024-09-16T15:40:13.436268+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:04:11.653249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.436116+0000","last_change":"2024-09-16T15:40:12.446725+0000","last_active":"2024-09-16T15:40:13.436116+0000","last_peered":"2024-09-16T15:40:13.436116+0000","last_clean":"2024-09-16T15:40:13.436116+0000","last_became_active":"2024-09-16T15:40:12.441589+0000","last_became_peered":"2024-09-16T15:40:12.441589+0000","last_unstale":"2024-09-16T15:40:13.436116+0000","last_undegraded":"2024-09-16T15:40:13.436116+0000","last_fullsized":"2024-09-16T15:40:13.436116+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:50:22.432002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.411274+0000","last_change":"2024-09-16T15:40:10.411274+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.411274+0000","last_undegraded":"2024-09-16T15:40:10.411274+0000","last_fullsized":"2024-09-16T15:40:10.411274+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.408093+0000","last_change":"2024-09-16T15:40:10.408093+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.408093+0000","last_undegraded":"2024-09-16T15:40:10.408093+0000","last_fullsized":"2024-09-16T15:40:10.408093+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"34'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.436310+0000","last_change":"2024-09-16T15:40:11.429304+0000","last_active":"2024-09-16T15:40:13.436310+0000","last_peered":"2024-09-16T15:40:13.436310+0000","last_clean":"2024-09-16T15:40:13.436310+0000","last_became_active":"2024-09-16T15:40:11.429029+0000","last_became_peered":"2024-09-16T15:40:11.429029+0000","last_unstale":"2024-09-16T15:40:13.436310+0000","last_undegraded":"2024-09-16T15:40:13.436310+0000","last_fullsized":"2024-09-16T15:40:13.436310+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:31:10.242510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.408118+0000","last_change":"2024-09-16T15:40:10.408118+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.408118+0000","last_undegraded":"2024-09-16T15:40:10.408118+0000","last_fullsized":"2024-09-16T15:40:10.408118+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.567666+0000","last_change":"2024-09-16T15:40:10.567666+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.567666+0000","last_undegraded":"2024-09-16T15:40:10.567666+0000","last_fullsized":"2024-09-16T15:40:10.567666+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.567420+0000","last_change":"2024-09-16T15:40:10.567420+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.567420+0000","last_undegraded":"2024-09-16T15:40:10.567420+0000","last_fullsized":"2024-09-16T15:40:10.567420+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.435951+0000","last_change":"2024-09-16T15:40:11.431791+0000","last_active":"2024-09-16T15:40:13.435951+0000","last_peered":"2024-09-16T15:40:13.435951+0000","last_clean":"2024-09-16T15:40:13.435951+0000","last_became_active":"2024-09-16T15:40:11.431517+0000","last_became_peered":"2024-09-16T15:40:11.431517+0000","last_unstale":"2024-09-16T15:40:13.435951+0000","last_undegraded":"2024-09-16T15:40:13.435951+0000","last_fullsized":"2024-09-16T15:40:13.435951+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:17:20.232340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.566960+0000","last_change":"2024-09-16T15:40:10.566960+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.566960+0000","last_undegraded":"2024-09-16T15:40:10.566960+0000","last_fullsized":"2024-09-16T15:40:10.566960+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.408736+0000","last_change":"2024-09-16T15:40:10.408736+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.408736+0000","last_undegraded":"2024-09-16T15:40:10.408736+0000","last_fullsized":"2024-09-16T15:40:10.408736+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.408900+0000","last_change":"2024-09-16T15:40:10.408900+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.408900+0000","last_undegraded":"2024-09-16T15:40:10.408900+0000","last_fullsized":"2024-09-16T15:40:10.408900+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.412253+0000","last_change":"2024-09-16T15:40:10.412253+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.412253+0000","last_undegraded":"2024-09-16T15:40:10.412253+0000","last_fullsized":"2024-09-16T15:40:10.412253+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.412131+0000","last_change":"2024-09-16T15:40:10.412131+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.412131+0000","last_undegraded":"2024-09-16T15:40:10.412131+0000","last_fullsized":"2024-09-16T15:40:10.412131+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.567001+0000","last_change":"2024-09-16T15:40:10.567001+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.567001+0000","last_undegraded":"2024-09-16T15:40:10.567001+0000","last_fullsized":"2024-09-16T15:40:10.567001+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.408382+0000","last_change":"2024-09-16T15:40:10.408382+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.408382+0000","last_undegraded":"2024-09-16T15:40:10.408382+0000","last_fullsized":"2024-09-16T15:40:10.408382+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.408111+0000","last_change":"2024-09-16T15:40:10.408111+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.408111+0000","last_undegraded":"2024-09-16T15:40:10.408111+0000","last_fullsized":"2024-09-16T15:40:10.408111+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"34'8","reported_seq":23,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.435447+0000","last_change":"2024-09-16T15:40:11.425677+0000","last_active":"2024-09-16T15:40:13.435447+0000","last_peered":"2024-09-16T15:40:13.435447+0000","last_clean":"2024-09-16T15:40:13.435447+0000","last_became_active":"2024-09-16T15:40:11.425390+0000","last_became_peered":"2024-09-16T15:40:11.425390+0000","last_unstale":"2024-09-16T15:40:13.435447+0000","last_undegraded":"2024-09-16T15:40:13.435447+0000","last_fullsized":"2024-09-16T15:40:13.435447+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:02:34.885995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"34'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.436125+0000","last_change":"2024-09-16T15:40:11.433816+0000","last_active":"2024-09-16T15:40:13.436125+0000","last_peered":"2024-09-16T15:40:13.436125+0000","last_clean":"2024-09-16T15:40:13.436125+0000","last_became_active":"2024-09-16T15:40:11.433028+0000","last_became_peered":"2024-09-16T15:40:11.433028+0000","last_unstale":"2024-09-16T15:40:13.436125+0000","last_undegraded":"2024-09-16T15:40:13.436125+0000","last_fullsized":"2024-09-16T15:40:13.436125+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:05:55.235854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.567075+0000","last_change":"2024-09-16T15:40:10.567075+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.567075+0000","last_undegraded":"2024-09-16T15:40:10.567075+0000","last_fullsized":"2024-09-16T15:40:10.567075+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.412002+0000","last_change":"2024-09-16T15:40:10.412002+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.412002+0000","last_undegraded":"2024-09-16T15:40:10.412002+0000","last_fullsized":"2024-09-16T15:40:10.412002+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.436182+0000","last_change":"2024-09-16T15:40:12.438579+0000","last_active":"2024-09-16T15:40:13.436182+0000","last_peered":"2024-09-16T15:40:13.436182+0000","last_clean":"2024-09-16T15:40:13.436182+0000","last_became_active":"2024-09-16T15:40:12.437311+0000","last_became_peered":"2024-09-16T15:40:12.437311+0000","last_unstale":"2024-09-16T15:40:13.436182+0000","last_undegraded":"2024-09-16T15:40:13.436182+0000","last_fullsized":"2024-09-16T15:40:13.436182+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:04:48.883149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.406699+0000","last_change":"2024-09-16T15:40:10.406699+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.406699+0000","last_undegraded":"2024-09-16T15:40:10.406699+0000","last_fullsized":"2024-09-16T15:40:10.406699+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.408267+0000","last_change":"2024-09-16T15:40:10.408267+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.408267+0000","last_undegraded":"2024-09-16T15:40:10.408267+0000","last_fullsized":"2024-09-16T15:40:10.408267+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.567019+0000","last_change":"2024-09-16T15:40:10.567019+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.567019+0000","last_undegraded":"2024-09-16T15:40:10.567019+0000","last_fullsized":"2024-09-16T15:40:10.567019+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.410409+0000","last_change":"2024-09-16T15:40:10.410409+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.410409+0000","last_undegraded":"2024-09-16T15:40:10.410409+0000","last_fullsized":"2024-09-16T15:40:10.410409+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.411274+0000","last_change":"2024-09-16T15:40:10.411274+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.411274+0000","last_undegraded":"2024-09-16T15:40:10.411274+0000","last_fullsized":"2024-09-16T15:40:10.411274+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.436365+0000","last_change":"2024-09-16T15:40:12.437260+0000","last_active":"2024-09-16T15:40:13.436365+0000","last_peered":"2024-09-16T15:40:13.436365+0000","last_clean":"2024-09-16T15:40:13.436365+0000","last_became_active":"2024-09-16T15:40:12.437001+0000","last_became_peered":"2024-09-16T15:40:12.437001+0000","last_unstale":"2024-09-16T15:40:13.436365+0000","last_undegraded":"2024-09-16T15:40:13.436365+0000","last_fullsized":"2024-09-16T15:40:13.436365+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:24:21.290065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.412057+0000","last_change":"2024-09-16T15:40:10.412057+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.412057+0000","last_undegraded":"2024-09-16T15:40:10.412057+0000","last_fullsized":"2024-09-16T15:40:10.412057+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.408276+0000","last_change":"2024-09-16T15:40:10.408276+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.408276+0000","last_undegraded":"2024-09-16T15:40:10.408276+0000","last_fullsized":"2024-09-16T15:40:10.408276+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.436527+0000","last_change":"2024-09-16T15:40:12.451657+0000","last_active":"2024-09-16T15:40:13.436527+0000","last_peered":"2024-09-16T15:40:13.436527+0000","last_clean":"2024-09-16T15:40:13.436527+0000","last_became_active":"2024-09-16T15:40:12.450795+0000","last_became_peered":"2024-09-16T15:40:12.450795+0000","last_unstale":"2024-09-16T15:40:13.436527+0000","last_undegraded":"2024-09-16T15:40:13.436527+0000","last_fullsized":"2024-09-16T15:40:13.436527+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:46:24.970506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.411635+0000","last_change":"2024-09-16T15:40:10.411635+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.411635+0000","last_undegraded":"2024-09-16T15:40:10.411635+0000","last_fullsized":"2024-09-16T15:40:10.411635+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.411513+0000","last_change":"2024-09-16T15:40:10.411513+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.411513+0000","last_undegraded":"2024-09-16T15:40:10.411513+0000","last_fullsized":"2024-09-16T15:40:10.411513+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.411507+0000","last_change":"2024-09-16T15:40:10.411507+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.411507+0000","last_undegraded":"2024-09-16T15:40:10.411507+0000","last_fullsized":"2024-09-16T15:40:10.411507+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.566998+0000","last_change":"2024-09-16T15:40:10.566998+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.566998+0000","last_undegraded":"2024-09-16T15:40:10.566998+0000","last_fullsized":"2024-09-16T15:40:10.566998+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.408225+0000","last_change":"2024-09-16T15:40:10.408225+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.408225+0000","last_undegraded":"2024-09-16T15:40:10.408225+0000","last_fullsized":"2024-09-16T15:40:10.408225+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:40:11.403725+0000","last_change":"2024-09-16T15:40:11.403725+0000","last_active":"2024-09-16T15:40:11.403725+0000","last_peered":"2024-09-16T15:40:11.403725+0000","last_clean":"2024-09-16T15:40:11.403725+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:11.403725+0000","last_undegraded":"2024-09-16T15:40:11.403725+0000","last_fullsized":"2024-09-16T15:40:11.403725+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-16T15:40:10.411572+0000","last_change":"2024-09-16T15:40:10.411572+0000","last_active":"2024-09-16T15:40:10.399062+0000","last_peered":"2024-09-16T15:40:10.399062+0000","last_clean":"2024-09-16T15:40:10.399062+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:40:10.411572+0000","last_undegraded":"2024-09-16T15:40:10.411572+0000","last_fullsized":"2024-09-16T15:40:10.411572+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":129,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-16T15:40:10.410693+0000","last_change":"2024-09-16T15:39:12.698813+0000","last_active":"2024-09-16T15:40:10.410693+0000","last_peered":"2024-09-16T15:40:10.410693+0000","last_clean":"2024-09-16T15:40:10.410693+0000","last_became_active":"2024-09-16T15:39:12.697695+0000","last_became_peered":"2024-09-16T15:39:12.697695+0000","last_unstale":"2024-09-16T15:40:10.410693+0000","last_undegraded":"2024-09-16T15:40:10.410693+0000","last_fullsized":"2024-09-16T15:40:10.410693+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_clean_scrub_stamp":"2024-09-16T15:39:11.684967+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:02.061081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":12,"acting":12,"num_store_stats":1},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":22,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11,"ondisk_log_size":11,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509457,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":352256,"data_stored":132208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509456,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27952,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343888,"statfs":{"total":96636764160,"available":96608141312,"internally_reserved":0,"allocated":946176,"data_stored":722576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509455,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":339968,"data_stored":126391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509455,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":339968,"data_stored":126391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509455,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":352256,"data_stored":132208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509456,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27948,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343892,"statfs":{"total":96636764160,"available":96608145408,"internally_reserved":0,"allocated":933888,"data_stored":716759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509454,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":339968,"data_stored":126391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542162,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":393216,"data_stored":151580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":27654604},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:40:17.485 INFO:tasks.ceph.ceph_manager.ceph:PG 5.3f is not active+clean 2024-09-16T15:40:17.486 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:40:11.403725+0000', 'last_change': '2024-09-16T15:40:11.403725+0000', 'last_active': '2024-09-16T15:40:11.403725+0000', 'last_peered': '2024-09-16T15:40:11.403725+0000', 'last_clean': '2024-09-16T15:40:11.403725+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:11.403725+0000', 'last_undegraded': '2024-09-16T15:40:11.403725+0000', 'last_fullsized': '2024-09-16T15:40:11.403725+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:40:17.486 INFO:tasks.ceph.ceph_manager.ceph:PG 5.3e is not active+clean 2024-09-16T15:40:17.486 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:40:11.403725+0000', 'last_change': '2024-09-16T15:40:11.403725+0000', 'last_active': '2024-09-16T15:40:11.403725+0000', 'last_peered': '2024-09-16T15:40:11.403725+0000', 'last_clean': '2024-09-16T15:40:11.403725+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:11.403725+0000', 'last_undegraded': '2024-09-16T15:40:11.403725+0000', 'last_fullsized': '2024-09-16T15:40:11.403725+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:40:17.486 INFO:tasks.ceph.ceph_manager.ceph:PG 5.3d is not active+clean 2024-09-16T15:40:17.486 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:40:11.403725+0000', 'last_change': '2024-09-16T15:40:11.403725+0000', 'last_active': '2024-09-16T15:40:11.403725+0000', 'last_peered': '2024-09-16T15:40:11.403725+0000', 'last_clean': '2024-09-16T15:40:11.403725+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:11.403725+0000', 'last_undegraded': '2024-09-16T15:40:11.403725+0000', 'last_fullsized': '2024-09-16T15:40:11.403725+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:40:17.486 INFO:tasks.ceph.ceph_manager.ceph:PG 5.3c is not active+clean 2024-09-16T15:40:17.486 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:40:11.403725+0000', 'last_change': '2024-09-16T15:40:11.403725+0000', 'last_active': '2024-09-16T15:40:11.403725+0000', 'last_peered': '2024-09-16T15:40:11.403725+0000', 'last_clean': '2024-09-16T15:40:11.403725+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:11.403725+0000', 'last_undegraded': '2024-09-16T15:40:11.403725+0000', 'last_fullsized': '2024-09-16T15:40:11.403725+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:40:17.486 INFO:tasks.ceph.ceph_manager.ceph:PG 5.3b is not active+clean 2024-09-16T15:40:17.486 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:40:11.403725+0000', 'last_change': '2024-09-16T15:40:11.403725+0000', 'last_active': '2024-09-16T15:40:11.403725+0000', 'last_peered': '2024-09-16T15:40:11.403725+0000', 'last_clean': '2024-09-16T15:40:11.403725+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:11.403725+0000', 'last_undegraded': '2024-09-16T15:40:11.403725+0000', 'last_fullsized': '2024-09-16T15:40:11.403725+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:40:17.486 INFO:tasks.ceph.ceph_manager.ceph:PG 5.39 is not active+clean 2024-09-16T15:40:17.486 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:40:11.403725+0000', 'last_change': '2024-09-16T15:40:11.403725+0000', 'last_active': '2024-09-16T15:40:11.403725+0000', 'last_peered': '2024-09-16T15:40:11.403725+0000', 'last_clean': '2024-09-16T15:40:11.403725+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:11.403725+0000', 'last_undegraded': '2024-09-16T15:40:11.403725+0000', 'last_fullsized': '2024-09-16T15:40:11.403725+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:40:17.487 INFO:tasks.ceph.ceph_manager.ceph:PG 5.38 is not active+clean 2024-09-16T15:40:17.487 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:40:11.403725+0000', 'last_change': '2024-09-16T15:40:11.403725+0000', 'last_active': '2024-09-16T15:40:11.403725+0000', 'last_peered': '2024-09-16T15:40:11.403725+0000', 'last_clean': '2024-09-16T15:40:11.403725+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:11.403725+0000', 'last_undegraded': '2024-09-16T15:40:11.403725+0000', 'last_fullsized': '2024-09-16T15:40:11.403725+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:40:17.487 INFO:tasks.ceph.ceph_manager.ceph:PG 5.37 is not active+clean 2024-09-16T15:40:17.487 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:40:11.403725+0000', 'last_change': '2024-09-16T15:40:11.403725+0000', 'last_active': '2024-09-16T15:40:11.403725+0000', 'last_peered': '2024-09-16T15:40:11.403725+0000', 'last_clean': '2024-09-16T15:40:11.403725+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:11.403725+0000', 'last_undegraded': '2024-09-16T15:40:11.403725+0000', 'last_fullsized': '2024-09-16T15:40:11.403725+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:40:17.487 INFO:tasks.ceph.ceph_manager.ceph:PG 5.36 is not active+clean 2024-09-16T15:40:17.487 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:40:11.403725+0000', 'last_change': '2024-09-16T15:40:11.403725+0000', 'last_active': '2024-09-16T15:40:11.403725+0000', 'last_peered': '2024-09-16T15:40:11.403725+0000', 'last_clean': '2024-09-16T15:40:11.403725+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:11.403725+0000', 'last_undegraded': '2024-09-16T15:40:11.403725+0000', 'last_fullsized': '2024-09-16T15:40:11.403725+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:40:17.487 INFO:tasks.ceph.ceph_manager.ceph:PG 5.35 is not active+clean 2024-09-16T15:40:17.487 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:40:11.403725+0000', 'last_change': '2024-09-16T15:40:11.403725+0000', 'last_active': '2024-09-16T15:40:11.403725+0000', 'last_peered': '2024-09-16T15:40:11.403725+0000', 'last_clean': '2024-09-16T15:40:11.403725+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:11.403725+0000', 'last_undegraded': '2024-09-16T15:40:11.403725+0000', 'last_fullsized': '2024-09-16T15:40:11.403725+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:40:17.487 INFO:tasks.ceph.ceph_manager.ceph:PG 5.34 is not active+clean 2024-09-16T15:40:17.487 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:40:11.403725+0000', 'last_change': '2024-09-16T15:40:11.403725+0000', 'last_active': '2024-09-16T15:40:11.403725+0000', 'last_peered': '2024-09-16T15:40:11.403725+0000', 'last_clean': '2024-09-16T15:40:11.403725+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:11.403725+0000', 'last_undegraded': '2024-09-16T15:40:11.403725+0000', 'last_fullsized': '2024-09-16T15:40:11.403725+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:40:17.487 INFO:tasks.ceph.ceph_manager.ceph:PG 5.33 is not active+clean 2024-09-16T15:40:17.487 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:40:11.403725+0000', 'last_change': '2024-09-16T15:40:11.403725+0000', 'last_active': '2024-09-16T15:40:11.403725+0000', 'last_peered': '2024-09-16T15:40:11.403725+0000', 'last_clean': '2024-09-16T15:40:11.403725+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:11.403725+0000', 'last_undegraded': '2024-09-16T15:40:11.403725+0000', 'last_fullsized': '2024-09-16T15:40:11.403725+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:40:17.488 INFO:tasks.ceph.ceph_manager.ceph:PG 5.32 is not active+clean 2024-09-16T15:40:17.488 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:40:11.403725+0000', 'last_change': '2024-09-16T15:40:11.403725+0000', 'last_active': '2024-09-16T15:40:11.403725+0000', 'last_peered': '2024-09-16T15:40:11.403725+0000', 'last_clean': '2024-09-16T15:40:11.403725+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:11.403725+0000', 'last_undegraded': '2024-09-16T15:40:11.403725+0000', 'last_fullsized': '2024-09-16T15:40:11.403725+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:40:17.488 INFO:tasks.ceph.ceph_manager.ceph:PG 5.31 is not active+clean 2024-09-16T15:40:17.488 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:40:11.403725+0000', 'last_change': '2024-09-16T15:40:11.403725+0000', 'last_active': '2024-09-16T15:40:11.403725+0000', 'last_peered': '2024-09-16T15:40:11.403725+0000', 'last_clean': '2024-09-16T15:40:11.403725+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:11.403725+0000', 'last_undegraded': '2024-09-16T15:40:11.403725+0000', 'last_fullsized': '2024-09-16T15:40:11.403725+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:40:17.488 INFO:tasks.ceph.ceph_manager.ceph:PG 5.30 is not active+clean 2024-09-16T15:40:17.488 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:40:11.403725+0000', 'last_change': '2024-09-16T15:40:11.403725+0000', 'last_active': '2024-09-16T15:40:11.403725+0000', 'last_peered': '2024-09-16T15:40:11.403725+0000', 'last_clean': '2024-09-16T15:40:11.403725+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:11.403725+0000', 'last_undegraded': '2024-09-16T15:40:11.403725+0000', 'last_fullsized': '2024-09-16T15:40:11.403725+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:40:17.488 INFO:tasks.ceph.ceph_manager.ceph:PG 5.2f is not active+clean 2024-09-16T15:40:17.488 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:40:11.403725+0000', 'last_change': '2024-09-16T15:40:11.403725+0000', 'last_active': '2024-09-16T15:40:11.403725+0000', 'last_peered': '2024-09-16T15:40:11.403725+0000', 'last_clean': '2024-09-16T15:40:11.403725+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:11.403725+0000', 'last_undegraded': '2024-09-16T15:40:11.403725+0000', 'last_fullsized': '2024-09-16T15:40:11.403725+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:40:17.488 INFO:tasks.ceph.ceph_manager.ceph:PG 5.2c is not active+clean 2024-09-16T15:40:17.488 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:40:11.403725+0000', 'last_change': '2024-09-16T15:40:11.403725+0000', 'last_active': '2024-09-16T15:40:11.403725+0000', 'last_peered': '2024-09-16T15:40:11.403725+0000', 'last_clean': '2024-09-16T15:40:11.403725+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:11.403725+0000', 'last_undegraded': '2024-09-16T15:40:11.403725+0000', 'last_fullsized': '2024-09-16T15:40:11.403725+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:40:17.488 INFO:tasks.ceph.ceph_manager.ceph:PG 5.2b is not active+clean 2024-09-16T15:40:17.488 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:40:11.403725+0000', 'last_change': '2024-09-16T15:40:11.403725+0000', 'last_active': '2024-09-16T15:40:11.403725+0000', 'last_peered': '2024-09-16T15:40:11.403725+0000', 'last_clean': '2024-09-16T15:40:11.403725+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:11.403725+0000', 'last_undegraded': '2024-09-16T15:40:11.403725+0000', 'last_fullsized': '2024-09-16T15:40:11.403725+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:40:17.488 INFO:tasks.ceph.ceph_manager.ceph:PG 5.2a is not active+clean 2024-09-16T15:40:17.489 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:40:11.403725+0000', 'last_change': '2024-09-16T15:40:11.403725+0000', 'last_active': '2024-09-16T15:40:11.403725+0000', 'last_peered': '2024-09-16T15:40:11.403725+0000', 'last_clean': '2024-09-16T15:40:11.403725+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:11.403725+0000', 'last_undegraded': '2024-09-16T15:40:11.403725+0000', 'last_fullsized': '2024-09-16T15:40:11.403725+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:40:17.489 INFO:tasks.ceph.ceph_manager.ceph:PG 5.29 is not active+clean 2024-09-16T15:40:17.489 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:40:11.403725+0000', 'last_change': '2024-09-16T15:40:11.403725+0000', 'last_active': '2024-09-16T15:40:11.403725+0000', 'last_peered': '2024-09-16T15:40:11.403725+0000', 'last_clean': '2024-09-16T15:40:11.403725+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:11.403725+0000', 'last_undegraded': '2024-09-16T15:40:11.403725+0000', 'last_fullsized': '2024-09-16T15:40:11.403725+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:40:17.489 INFO:tasks.ceph.ceph_manager.ceph:PG 5.28 is not active+clean 2024-09-16T15:40:17.489 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:40:11.403725+0000', 'last_change': '2024-09-16T15:40:11.403725+0000', 'last_active': '2024-09-16T15:40:11.403725+0000', 'last_peered': '2024-09-16T15:40:11.403725+0000', 'last_clean': '2024-09-16T15:40:11.403725+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:11.403725+0000', 'last_undegraded': '2024-09-16T15:40:11.403725+0000', 'last_fullsized': '2024-09-16T15:40:11.403725+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:40:17.489 INFO:tasks.ceph.ceph_manager.ceph:PG 5.27 is not active+clean 2024-09-16T15:40:17.489 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:40:11.403725+0000', 'last_change': '2024-09-16T15:40:11.403725+0000', 'last_active': '2024-09-16T15:40:11.403725+0000', 'last_peered': '2024-09-16T15:40:11.403725+0000', 'last_clean': '2024-09-16T15:40:11.403725+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:11.403725+0000', 'last_undegraded': '2024-09-16T15:40:11.403725+0000', 'last_fullsized': '2024-09-16T15:40:11.403725+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:40:17.489 INFO:tasks.ceph.ceph_manager.ceph:PG 5.26 is not active+clean 2024-09-16T15:40:17.489 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:40:11.403725+0000', 'last_change': '2024-09-16T15:40:11.403725+0000', 'last_active': '2024-09-16T15:40:11.403725+0000', 'last_peered': '2024-09-16T15:40:11.403725+0000', 'last_clean': '2024-09-16T15:40:11.403725+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:11.403725+0000', 'last_undegraded': '2024-09-16T15:40:11.403725+0000', 'last_fullsized': '2024-09-16T15:40:11.403725+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:40:17.489 INFO:tasks.ceph.ceph_manager.ceph:PG 5.25 is not active+clean 2024-09-16T15:40:17.489 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:40:11.403725+0000', 'last_change': '2024-09-16T15:40:11.403725+0000', 'last_active': '2024-09-16T15:40:11.403725+0000', 'last_peered': '2024-09-16T15:40:11.403725+0000', 'last_clean': '2024-09-16T15:40:11.403725+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:11.403725+0000', 'last_undegraded': '2024-09-16T15:40:11.403725+0000', 'last_fullsized': '2024-09-16T15:40:11.403725+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:40:17.489 INFO:tasks.ceph.ceph_manager.ceph:PG 5.24 is not active+clean 2024-09-16T15:40:17.490 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:40:11.403725+0000', 'last_change': '2024-09-16T15:40:11.403725+0000', 'last_active': '2024-09-16T15:40:11.403725+0000', 'last_peered': '2024-09-16T15:40:11.403725+0000', 'last_clean': '2024-09-16T15:40:11.403725+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:11.403725+0000', 'last_undegraded': '2024-09-16T15:40:11.403725+0000', 'last_fullsized': '2024-09-16T15:40:11.403725+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:40:17.490 INFO:tasks.ceph.ceph_manager.ceph:PG 5.23 is not active+clean 2024-09-16T15:40:17.490 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:40:11.403725+0000', 'last_change': '2024-09-16T15:40:11.403725+0000', 'last_active': '2024-09-16T15:40:11.403725+0000', 'last_peered': '2024-09-16T15:40:11.403725+0000', 'last_clean': '2024-09-16T15:40:11.403725+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:11.403725+0000', 'last_undegraded': '2024-09-16T15:40:11.403725+0000', 'last_fullsized': '2024-09-16T15:40:11.403725+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:40:17.490 INFO:tasks.ceph.ceph_manager.ceph:PG 5.22 is not active+clean 2024-09-16T15:40:17.490 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:40:11.403725+0000', 'last_change': '2024-09-16T15:40:11.403725+0000', 'last_active': '2024-09-16T15:40:11.403725+0000', 'last_peered': '2024-09-16T15:40:11.403725+0000', 'last_clean': '2024-09-16T15:40:11.403725+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:11.403725+0000', 'last_undegraded': '2024-09-16T15:40:11.403725+0000', 'last_fullsized': '2024-09-16T15:40:11.403725+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:40:17.490 INFO:tasks.ceph.ceph_manager.ceph:PG 5.21 is not active+clean 2024-09-16T15:40:17.490 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:40:11.403725+0000', 'last_change': '2024-09-16T15:40:11.403725+0000', 'last_active': '2024-09-16T15:40:11.403725+0000', 'last_peered': '2024-09-16T15:40:11.403725+0000', 'last_clean': '2024-09-16T15:40:11.403725+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:11.403725+0000', 'last_undegraded': '2024-09-16T15:40:11.403725+0000', 'last_fullsized': '2024-09-16T15:40:11.403725+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:40:17.490 INFO:tasks.ceph.ceph_manager.ceph:PG 5.20 is not active+clean 2024-09-16T15:40:17.490 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:40:11.403725+0000', 'last_change': '2024-09-16T15:40:11.403725+0000', 'last_active': '2024-09-16T15:40:11.403725+0000', 'last_peered': '2024-09-16T15:40:11.403725+0000', 'last_clean': '2024-09-16T15:40:11.403725+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:11.403725+0000', 'last_undegraded': '2024-09-16T15:40:11.403725+0000', 'last_fullsized': '2024-09-16T15:40:11.403725+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:40:17.490 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1e is not active+clean 2024-09-16T15:40:17.490 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:40:11.403725+0000', 'last_change': '2024-09-16T15:40:11.403725+0000', 'last_active': '2024-09-16T15:40:11.403725+0000', 'last_peered': '2024-09-16T15:40:11.403725+0000', 'last_clean': '2024-09-16T15:40:11.403725+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:11.403725+0000', 'last_undegraded': '2024-09-16T15:40:11.403725+0000', 'last_fullsized': '2024-09-16T15:40:11.403725+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:40:17.490 INFO:tasks.ceph.ceph_manager.ceph:PG 4.1f is not active+clean 2024-09-16T15:40:17.491 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:40:10.411274+0000', 'last_change': '2024-09-16T15:40:10.411274+0000', 'last_active': '2024-09-16T15:40:10.399062+0000', 'last_peered': '2024-09-16T15:40:10.399062+0000', 'last_clean': '2024-09-16T15:40:10.399062+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:10.411274+0000', 'last_undegraded': '2024-09-16T15:40:10.411274+0000', 'last_fullsized': '2024-09-16T15:40:10.411274+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T15:40:17.491 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1f is not active+clean 2024-09-16T15:40:17.491 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:40:11.403725+0000', 'last_change': '2024-09-16T15:40:11.403725+0000', 'last_active': '2024-09-16T15:40:11.403725+0000', 'last_peered': '2024-09-16T15:40:11.403725+0000', 'last_clean': '2024-09-16T15:40:11.403725+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:11.403725+0000', 'last_undegraded': '2024-09-16T15:40:11.403725+0000', 'last_fullsized': '2024-09-16T15:40:11.403725+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:40:17.491 INFO:tasks.ceph.ceph_manager.ceph:PG 4.1e is not active+clean 2024-09-16T15:40:17.491 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:40:10.408093+0000', 'last_change': '2024-09-16T15:40:10.408093+0000', 'last_active': '2024-09-16T15:40:10.399062+0000', 'last_peered': '2024-09-16T15:40:10.399062+0000', 'last_clean': '2024-09-16T15:40:10.399062+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:10.408093+0000', 'last_undegraded': '2024-09-16T15:40:10.408093+0000', 'last_fullsized': '2024-09-16T15:40:10.408093+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-16T15:40:17.491 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1c is not active+clean 2024-09-16T15:40:17.491 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:40:11.403725+0000', 'last_change': '2024-09-16T15:40:11.403725+0000', 'last_active': '2024-09-16T15:40:11.403725+0000', 'last_peered': '2024-09-16T15:40:11.403725+0000', 'last_clean': '2024-09-16T15:40:11.403725+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:11.403725+0000', 'last_undegraded': '2024-09-16T15:40:11.403725+0000', 'last_fullsized': '2024-09-16T15:40:11.403725+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:40:17.491 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1d is not active+clean 2024-09-16T15:40:17.491 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:40:11.403725+0000', 'last_change': '2024-09-16T15:40:11.403725+0000', 'last_active': '2024-09-16T15:40:11.403725+0000', 'last_peered': '2024-09-16T15:40:11.403725+0000', 'last_clean': '2024-09-16T15:40:11.403725+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:11.403725+0000', 'last_undegraded': '2024-09-16T15:40:11.403725+0000', 'last_fullsized': '2024-09-16T15:40:11.403725+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:40:17.491 INFO:tasks.ceph.ceph_manager.ceph:PG 4.1c is not active+clean 2024-09-16T15:40:17.491 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:40:10.408118+0000', 'last_change': '2024-09-16T15:40:10.408118+0000', 'last_active': '2024-09-16T15:40:10.399062+0000', 'last_peered': '2024-09-16T15:40:10.399062+0000', 'last_clean': '2024-09-16T15:40:10.399062+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:10.408118+0000', 'last_undegraded': '2024-09-16T15:40:10.408118+0000', 'last_fullsized': '2024-09-16T15:40:10.408118+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-16T15:40:17.491 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1a is not active+clean 2024-09-16T15:40:17.492 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:40:11.403725+0000', 'last_change': '2024-09-16T15:40:11.403725+0000', 'last_active': '2024-09-16T15:40:11.403725+0000', 'last_peered': '2024-09-16T15:40:11.403725+0000', 'last_clean': '2024-09-16T15:40:11.403725+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:11.403725+0000', 'last_undegraded': '2024-09-16T15:40:11.403725+0000', 'last_fullsized': '2024-09-16T15:40:11.403725+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:40:17.492 INFO:tasks.ceph.ceph_manager.ceph:PG 4.1b is not active+clean 2024-09-16T15:40:17.492 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:40:10.567666+0000', 'last_change': '2024-09-16T15:40:10.567666+0000', 'last_active': '2024-09-16T15:40:10.399062+0000', 'last_peered': '2024-09-16T15:40:10.399062+0000', 'last_clean': '2024-09-16T15:40:10.399062+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:10.567666+0000', 'last_undegraded': '2024-09-16T15:40:10.567666+0000', 'last_fullsized': '2024-09-16T15:40:10.567666+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T15:40:17.492 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1b is not active+clean 2024-09-16T15:40:17.492 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:40:11.403725+0000', 'last_change': '2024-09-16T15:40:11.403725+0000', 'last_active': '2024-09-16T15:40:11.403725+0000', 'last_peered': '2024-09-16T15:40:11.403725+0000', 'last_clean': '2024-09-16T15:40:11.403725+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:11.403725+0000', 'last_undegraded': '2024-09-16T15:40:11.403725+0000', 'last_fullsized': '2024-09-16T15:40:11.403725+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:40:17.492 INFO:tasks.ceph.ceph_manager.ceph:PG 4.1a is not active+clean 2024-09-16T15:40:17.492 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:40:10.567420+0000', 'last_change': '2024-09-16T15:40:10.567420+0000', 'last_active': '2024-09-16T15:40:10.399062+0000', 'last_peered': '2024-09-16T15:40:10.399062+0000', 'last_clean': '2024-09-16T15:40:10.399062+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:10.567420+0000', 'last_undegraded': '2024-09-16T15:40:10.567420+0000', 'last_fullsized': '2024-09-16T15:40:10.567420+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T15:40:17.492 INFO:tasks.ceph.ceph_manager.ceph:PG 5.18 is not active+clean 2024-09-16T15:40:17.492 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:40:11.403725+0000', 'last_change': '2024-09-16T15:40:11.403725+0000', 'last_active': '2024-09-16T15:40:11.403725+0000', 'last_peered': '2024-09-16T15:40:11.403725+0000', 'last_clean': '2024-09-16T15:40:11.403725+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:11.403725+0000', 'last_undegraded': '2024-09-16T15:40:11.403725+0000', 'last_fullsized': '2024-09-16T15:40:11.403725+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:40:17.492 INFO:tasks.ceph.ceph_manager.ceph:PG 5.19 is not active+clean 2024-09-16T15:40:17.492 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:40:11.403725+0000', 'last_change': '2024-09-16T15:40:11.403725+0000', 'last_active': '2024-09-16T15:40:11.403725+0000', 'last_peered': '2024-09-16T15:40:11.403725+0000', 'last_clean': '2024-09-16T15:40:11.403725+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:11.403725+0000', 'last_undegraded': '2024-09-16T15:40:11.403725+0000', 'last_fullsized': '2024-09-16T15:40:11.403725+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:40:17.492 INFO:tasks.ceph.ceph_manager.ceph:PG 4.18 is not active+clean 2024-09-16T15:40:17.493 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:40:10.566960+0000', 'last_change': '2024-09-16T15:40:10.566960+0000', 'last_active': '2024-09-16T15:40:10.399062+0000', 'last_peered': '2024-09-16T15:40:10.399062+0000', 'last_clean': '2024-09-16T15:40:10.399062+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:10.566960+0000', 'last_undegraded': '2024-09-16T15:40:10.566960+0000', 'last_fullsized': '2024-09-16T15:40:10.566960+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T15:40:17.493 INFO:tasks.ceph.ceph_manager.ceph:PG 5.16 is not active+clean 2024-09-16T15:40:17.493 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:40:11.403725+0000', 'last_change': '2024-09-16T15:40:11.403725+0000', 'last_active': '2024-09-16T15:40:11.403725+0000', 'last_peered': '2024-09-16T15:40:11.403725+0000', 'last_clean': '2024-09-16T15:40:11.403725+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:11.403725+0000', 'last_undegraded': '2024-09-16T15:40:11.403725+0000', 'last_fullsized': '2024-09-16T15:40:11.403725+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:40:17.493 INFO:tasks.ceph.ceph_manager.ceph:PG 4.17 is not active+clean 2024-09-16T15:40:17.493 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:40:10.408736+0000', 'last_change': '2024-09-16T15:40:10.408736+0000', 'last_active': '2024-09-16T15:40:10.399062+0000', 'last_peered': '2024-09-16T15:40:10.399062+0000', 'last_clean': '2024-09-16T15:40:10.399062+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:10.408736+0000', 'last_undegraded': '2024-09-16T15:40:10.408736+0000', 'last_fullsized': '2024-09-16T15:40:10.408736+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-16T15:40:17.493 INFO:tasks.ceph.ceph_manager.ceph:PG 5.17 is not active+clean 2024-09-16T15:40:17.493 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:40:11.403725+0000', 'last_change': '2024-09-16T15:40:11.403725+0000', 'last_active': '2024-09-16T15:40:11.403725+0000', 'last_peered': '2024-09-16T15:40:11.403725+0000', 'last_clean': '2024-09-16T15:40:11.403725+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:11.403725+0000', 'last_undegraded': '2024-09-16T15:40:11.403725+0000', 'last_fullsized': '2024-09-16T15:40:11.403725+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:40:17.493 INFO:tasks.ceph.ceph_manager.ceph:PG 4.16 is not active+clean 2024-09-16T15:40:17.493 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:40:10.408900+0000', 'last_change': '2024-09-16T15:40:10.408900+0000', 'last_active': '2024-09-16T15:40:10.399062+0000', 'last_peered': '2024-09-16T15:40:10.399062+0000', 'last_clean': '2024-09-16T15:40:10.399062+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:10.408900+0000', 'last_undegraded': '2024-09-16T15:40:10.408900+0000', 'last_fullsized': '2024-09-16T15:40:10.408900+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-16T15:40:17.493 INFO:tasks.ceph.ceph_manager.ceph:PG 5.14 is not active+clean 2024-09-16T15:40:17.493 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:40:11.403725+0000', 'last_change': '2024-09-16T15:40:11.403725+0000', 'last_active': '2024-09-16T15:40:11.403725+0000', 'last_peered': '2024-09-16T15:40:11.403725+0000', 'last_clean': '2024-09-16T15:40:11.403725+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:11.403725+0000', 'last_undegraded': '2024-09-16T15:40:11.403725+0000', 'last_fullsized': '2024-09-16T15:40:11.403725+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:40:17.493 INFO:tasks.ceph.ceph_manager.ceph:PG 4.15 is not active+clean 2024-09-16T15:40:17.494 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:40:10.412253+0000', 'last_change': '2024-09-16T15:40:10.412253+0000', 'last_active': '2024-09-16T15:40:10.399062+0000', 'last_peered': '2024-09-16T15:40:10.399062+0000', 'last_clean': '2024-09-16T15:40:10.399062+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:10.412253+0000', 'last_undegraded': '2024-09-16T15:40:10.412253+0000', 'last_fullsized': '2024-09-16T15:40:10.412253+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T15:40:17.494 INFO:tasks.ceph.ceph_manager.ceph:PG 5.15 is not active+clean 2024-09-16T15:40:17.494 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:40:11.403725+0000', 'last_change': '2024-09-16T15:40:11.403725+0000', 'last_active': '2024-09-16T15:40:11.403725+0000', 'last_peered': '2024-09-16T15:40:11.403725+0000', 'last_clean': '2024-09-16T15:40:11.403725+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:11.403725+0000', 'last_undegraded': '2024-09-16T15:40:11.403725+0000', 'last_fullsized': '2024-09-16T15:40:11.403725+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:40:17.494 INFO:tasks.ceph.ceph_manager.ceph:PG 4.14 is not active+clean 2024-09-16T15:40:17.494 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:40:10.412131+0000', 'last_change': '2024-09-16T15:40:10.412131+0000', 'last_active': '2024-09-16T15:40:10.399062+0000', 'last_peered': '2024-09-16T15:40:10.399062+0000', 'last_clean': '2024-09-16T15:40:10.399062+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:10.412131+0000', 'last_undegraded': '2024-09-16T15:40:10.412131+0000', 'last_fullsized': '2024-09-16T15:40:10.412131+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-16T15:40:17.494 INFO:tasks.ceph.ceph_manager.ceph:PG 5.12 is not active+clean 2024-09-16T15:40:17.494 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:40:11.403725+0000', 'last_change': '2024-09-16T15:40:11.403725+0000', 'last_active': '2024-09-16T15:40:11.403725+0000', 'last_peered': '2024-09-16T15:40:11.403725+0000', 'last_clean': '2024-09-16T15:40:11.403725+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:11.403725+0000', 'last_undegraded': '2024-09-16T15:40:11.403725+0000', 'last_fullsized': '2024-09-16T15:40:11.403725+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:40:17.494 INFO:tasks.ceph.ceph_manager.ceph:PG 4.13 is not active+clean 2024-09-16T15:40:17.494 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:40:10.567001+0000', 'last_change': '2024-09-16T15:40:10.567001+0000', 'last_active': '2024-09-16T15:40:10.399062+0000', 'last_peered': '2024-09-16T15:40:10.399062+0000', 'last_clean': '2024-09-16T15:40:10.399062+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:10.567001+0000', 'last_undegraded': '2024-09-16T15:40:10.567001+0000', 'last_fullsized': '2024-09-16T15:40:10.567001+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T15:40:17.494 INFO:tasks.ceph.ceph_manager.ceph:PG 5.13 is not active+clean 2024-09-16T15:40:17.494 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:40:11.403725+0000', 'last_change': '2024-09-16T15:40:11.403725+0000', 'last_active': '2024-09-16T15:40:11.403725+0000', 'last_peered': '2024-09-16T15:40:11.403725+0000', 'last_clean': '2024-09-16T15:40:11.403725+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:11.403725+0000', 'last_undegraded': '2024-09-16T15:40:11.403725+0000', 'last_fullsized': '2024-09-16T15:40:11.403725+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:40:17.494 INFO:tasks.ceph.ceph_manager.ceph:PG 4.12 is not active+clean 2024-09-16T15:40:17.495 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:40:10.408382+0000', 'last_change': '2024-09-16T15:40:10.408382+0000', 'last_active': '2024-09-16T15:40:10.399062+0000', 'last_peered': '2024-09-16T15:40:10.399062+0000', 'last_clean': '2024-09-16T15:40:10.399062+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:10.408382+0000', 'last_undegraded': '2024-09-16T15:40:10.408382+0000', 'last_fullsized': '2024-09-16T15:40:10.408382+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-16T15:40:17.495 INFO:tasks.ceph.ceph_manager.ceph:PG 5.10 is not active+clean 2024-09-16T15:40:17.495 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:40:11.403725+0000', 'last_change': '2024-09-16T15:40:11.403725+0000', 'last_active': '2024-09-16T15:40:11.403725+0000', 'last_peered': '2024-09-16T15:40:11.403725+0000', 'last_clean': '2024-09-16T15:40:11.403725+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:11.403725+0000', 'last_undegraded': '2024-09-16T15:40:11.403725+0000', 'last_fullsized': '2024-09-16T15:40:11.403725+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:40:17.495 INFO:tasks.ceph.ceph_manager.ceph:PG 4.11 is not active+clean 2024-09-16T15:40:17.495 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:40:10.408111+0000', 'last_change': '2024-09-16T15:40:10.408111+0000', 'last_active': '2024-09-16T15:40:10.399062+0000', 'last_peered': '2024-09-16T15:40:10.399062+0000', 'last_clean': '2024-09-16T15:40:10.399062+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:10.408111+0000', 'last_undegraded': '2024-09-16T15:40:10.408111+0000', 'last_fullsized': '2024-09-16T15:40:10.408111+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-16T15:40:17.495 INFO:tasks.ceph.ceph_manager.ceph:PG 5.11 is not active+clean 2024-09-16T15:40:17.495 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:40:11.403725+0000', 'last_change': '2024-09-16T15:40:11.403725+0000', 'last_active': '2024-09-16T15:40:11.403725+0000', 'last_peered': '2024-09-16T15:40:11.403725+0000', 'last_clean': '2024-09-16T15:40:11.403725+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:11.403725+0000', 'last_undegraded': '2024-09-16T15:40:11.403725+0000', 'last_fullsized': '2024-09-16T15:40:11.403725+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:40:17.495 INFO:tasks.ceph.ceph_manager.ceph:PG 5.e is not active+clean 2024-09-16T15:40:17.495 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:40:11.403725+0000', 'last_change': '2024-09-16T15:40:11.403725+0000', 'last_active': '2024-09-16T15:40:11.403725+0000', 'last_peered': '2024-09-16T15:40:11.403725+0000', 'last_clean': '2024-09-16T15:40:11.403725+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:11.403725+0000', 'last_undegraded': '2024-09-16T15:40:11.403725+0000', 'last_fullsized': '2024-09-16T15:40:11.403725+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:40:17.495 INFO:tasks.ceph.ceph_manager.ceph:PG 5.f is not active+clean 2024-09-16T15:40:17.495 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:40:11.403725+0000', 'last_change': '2024-09-16T15:40:11.403725+0000', 'last_active': '2024-09-16T15:40:11.403725+0000', 'last_peered': '2024-09-16T15:40:11.403725+0000', 'last_clean': '2024-09-16T15:40:11.403725+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:11.403725+0000', 'last_undegraded': '2024-09-16T15:40:11.403725+0000', 'last_fullsized': '2024-09-16T15:40:11.403725+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:40:17.495 INFO:tasks.ceph.ceph_manager.ceph:PG 4.e is not active+clean 2024-09-16T15:40:17.495 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:40:10.567075+0000', 'last_change': '2024-09-16T15:40:10.567075+0000', 'last_active': '2024-09-16T15:40:10.399062+0000', 'last_peered': '2024-09-16T15:40:10.399062+0000', 'last_clean': '2024-09-16T15:40:10.399062+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:10.567075+0000', 'last_undegraded': '2024-09-16T15:40:10.567075+0000', 'last_fullsized': '2024-09-16T15:40:10.567075+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T15:40:17.496 INFO:tasks.ceph.ceph_manager.ceph:PG 5.c is not active+clean 2024-09-16T15:40:17.496 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:40:11.403725+0000', 'last_change': '2024-09-16T15:40:11.403725+0000', 'last_active': '2024-09-16T15:40:11.403725+0000', 'last_peered': '2024-09-16T15:40:11.403725+0000', 'last_clean': '2024-09-16T15:40:11.403725+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:11.403725+0000', 'last_undegraded': '2024-09-16T15:40:11.403725+0000', 'last_fullsized': '2024-09-16T15:40:11.403725+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:40:17.496 INFO:tasks.ceph.ceph_manager.ceph:PG 4.d is not active+clean 2024-09-16T15:40:17.496 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:40:10.412002+0000', 'last_change': '2024-09-16T15:40:10.412002+0000', 'last_active': '2024-09-16T15:40:10.399062+0000', 'last_peered': '2024-09-16T15:40:10.399062+0000', 'last_clean': '2024-09-16T15:40:10.399062+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:10.412002+0000', 'last_undegraded': '2024-09-16T15:40:10.412002+0000', 'last_fullsized': '2024-09-16T15:40:10.412002+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T15:40:17.496 INFO:tasks.ceph.ceph_manager.ceph:PG 4.c is not active+clean 2024-09-16T15:40:17.496 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:40:10.406699+0000', 'last_change': '2024-09-16T15:40:10.406699+0000', 'last_active': '2024-09-16T15:40:10.399062+0000', 'last_peered': '2024-09-16T15:40:10.399062+0000', 'last_clean': '2024-09-16T15:40:10.399062+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:10.406699+0000', 'last_undegraded': '2024-09-16T15:40:10.406699+0000', 'last_fullsized': '2024-09-16T15:40:10.406699+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T15:40:17.496 INFO:tasks.ceph.ceph_manager.ceph:PG 5.a is not active+clean 2024-09-16T15:40:17.496 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:40:11.403725+0000', 'last_change': '2024-09-16T15:40:11.403725+0000', 'last_active': '2024-09-16T15:40:11.403725+0000', 'last_peered': '2024-09-16T15:40:11.403725+0000', 'last_clean': '2024-09-16T15:40:11.403725+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:11.403725+0000', 'last_undegraded': '2024-09-16T15:40:11.403725+0000', 'last_fullsized': '2024-09-16T15:40:11.403725+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:40:17.496 INFO:tasks.ceph.ceph_manager.ceph:PG 4.b is not active+clean 2024-09-16T15:40:17.496 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:40:10.408267+0000', 'last_change': '2024-09-16T15:40:10.408267+0000', 'last_active': '2024-09-16T15:40:10.399062+0000', 'last_peered': '2024-09-16T15:40:10.399062+0000', 'last_clean': '2024-09-16T15:40:10.399062+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:10.408267+0000', 'last_undegraded': '2024-09-16T15:40:10.408267+0000', 'last_fullsized': '2024-09-16T15:40:10.408267+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-16T15:40:17.496 INFO:tasks.ceph.ceph_manager.ceph:PG 5.b is not active+clean 2024-09-16T15:40:17.496 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:40:11.403725+0000', 'last_change': '2024-09-16T15:40:11.403725+0000', 'last_active': '2024-09-16T15:40:11.403725+0000', 'last_peered': '2024-09-16T15:40:11.403725+0000', 'last_clean': '2024-09-16T15:40:11.403725+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:11.403725+0000', 'last_undegraded': '2024-09-16T15:40:11.403725+0000', 'last_fullsized': '2024-09-16T15:40:11.403725+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:40:17.497 INFO:tasks.ceph.ceph_manager.ceph:PG 4.a is not active+clean 2024-09-16T15:40:17.497 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:40:10.567019+0000', 'last_change': '2024-09-16T15:40:10.567019+0000', 'last_active': '2024-09-16T15:40:10.399062+0000', 'last_peered': '2024-09-16T15:40:10.399062+0000', 'last_clean': '2024-09-16T15:40:10.399062+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:10.567019+0000', 'last_undegraded': '2024-09-16T15:40:10.567019+0000', 'last_fullsized': '2024-09-16T15:40:10.567019+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T15:40:17.497 INFO:tasks.ceph.ceph_manager.ceph:PG 5.8 is not active+clean 2024-09-16T15:40:17.497 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:40:11.403725+0000', 'last_change': '2024-09-16T15:40:11.403725+0000', 'last_active': '2024-09-16T15:40:11.403725+0000', 'last_peered': '2024-09-16T15:40:11.403725+0000', 'last_clean': '2024-09-16T15:40:11.403725+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:11.403725+0000', 'last_undegraded': '2024-09-16T15:40:11.403725+0000', 'last_fullsized': '2024-09-16T15:40:11.403725+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:40:17.497 INFO:tasks.ceph.ceph_manager.ceph:PG 4.9 is not active+clean 2024-09-16T15:40:17.497 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:40:10.410409+0000', 'last_change': '2024-09-16T15:40:10.410409+0000', 'last_active': '2024-09-16T15:40:10.399062+0000', 'last_peered': '2024-09-16T15:40:10.399062+0000', 'last_clean': '2024-09-16T15:40:10.399062+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:10.410409+0000', 'last_undegraded': '2024-09-16T15:40:10.410409+0000', 'last_fullsized': '2024-09-16T15:40:10.410409+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-16T15:40:17.497 INFO:tasks.ceph.ceph_manager.ceph:PG 5.9 is not active+clean 2024-09-16T15:40:17.497 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:40:11.403725+0000', 'last_change': '2024-09-16T15:40:11.403725+0000', 'last_active': '2024-09-16T15:40:11.403725+0000', 'last_peered': '2024-09-16T15:40:11.403725+0000', 'last_clean': '2024-09-16T15:40:11.403725+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:11.403725+0000', 'last_undegraded': '2024-09-16T15:40:11.403725+0000', 'last_fullsized': '2024-09-16T15:40:11.403725+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:40:17.497 INFO:tasks.ceph.ceph_manager.ceph:PG 4.8 is not active+clean 2024-09-16T15:40:17.497 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:40:10.411274+0000', 'last_change': '2024-09-16T15:40:10.411274+0000', 'last_active': '2024-09-16T15:40:10.399062+0000', 'last_peered': '2024-09-16T15:40:10.399062+0000', 'last_clean': '2024-09-16T15:40:10.399062+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:10.411274+0000', 'last_undegraded': '2024-09-16T15:40:10.411274+0000', 'last_fullsized': '2024-09-16T15:40:10.411274+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T15:40:17.497 INFO:tasks.ceph.ceph_manager.ceph:PG 4.7 is not active+clean 2024-09-16T15:40:17.497 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:40:10.412057+0000', 'last_change': '2024-09-16T15:40:10.412057+0000', 'last_active': '2024-09-16T15:40:10.399062+0000', 'last_peered': '2024-09-16T15:40:10.399062+0000', 'last_clean': '2024-09-16T15:40:10.399062+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:10.412057+0000', 'last_undegraded': '2024-09-16T15:40:10.412057+0000', 'last_fullsized': '2024-09-16T15:40:10.412057+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-16T15:40:17.498 INFO:tasks.ceph.ceph_manager.ceph:PG 5.7 is not active+clean 2024-09-16T15:40:17.498 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:40:11.403725+0000', 'last_change': '2024-09-16T15:40:11.403725+0000', 'last_active': '2024-09-16T15:40:11.403725+0000', 'last_peered': '2024-09-16T15:40:11.403725+0000', 'last_clean': '2024-09-16T15:40:11.403725+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:11.403725+0000', 'last_undegraded': '2024-09-16T15:40:11.403725+0000', 'last_fullsized': '2024-09-16T15:40:11.403725+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:40:17.498 INFO:tasks.ceph.ceph_manager.ceph:PG 4.6 is not active+clean 2024-09-16T15:40:17.498 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:40:10.408276+0000', 'last_change': '2024-09-16T15:40:10.408276+0000', 'last_active': '2024-09-16T15:40:10.399062+0000', 'last_peered': '2024-09-16T15:40:10.399062+0000', 'last_clean': '2024-09-16T15:40:10.399062+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:10.408276+0000', 'last_undegraded': '2024-09-16T15:40:10.408276+0000', 'last_fullsized': '2024-09-16T15:40:10.408276+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-16T15:40:17.498 INFO:tasks.ceph.ceph_manager.ceph:PG 4.4 is not active+clean 2024-09-16T15:40:17.498 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:40:10.411635+0000', 'last_change': '2024-09-16T15:40:10.411635+0000', 'last_active': '2024-09-16T15:40:10.399062+0000', 'last_peered': '2024-09-16T15:40:10.399062+0000', 'last_clean': '2024-09-16T15:40:10.399062+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:10.411635+0000', 'last_undegraded': '2024-09-16T15:40:10.411635+0000', 'last_fullsized': '2024-09-16T15:40:10.411635+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-16T15:40:17.498 INFO:tasks.ceph.ceph_manager.ceph:PG 5.2 is not active+clean 2024-09-16T15:40:17.498 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:40:11.403725+0000', 'last_change': '2024-09-16T15:40:11.403725+0000', 'last_active': '2024-09-16T15:40:11.403725+0000', 'last_peered': '2024-09-16T15:40:11.403725+0000', 'last_clean': '2024-09-16T15:40:11.403725+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:11.403725+0000', 'last_undegraded': '2024-09-16T15:40:11.403725+0000', 'last_fullsized': '2024-09-16T15:40:11.403725+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:40:17.498 INFO:tasks.ceph.ceph_manager.ceph:PG 4.3 is not active+clean 2024-09-16T15:40:17.498 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:40:10.411513+0000', 'last_change': '2024-09-16T15:40:10.411513+0000', 'last_active': '2024-09-16T15:40:10.399062+0000', 'last_peered': '2024-09-16T15:40:10.399062+0000', 'last_clean': '2024-09-16T15:40:10.399062+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:10.411513+0000', 'last_undegraded': '2024-09-16T15:40:10.411513+0000', 'last_fullsized': '2024-09-16T15:40:10.411513+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-16T15:40:17.498 INFO:tasks.ceph.ceph_manager.ceph:PG 5.3 is not active+clean 2024-09-16T15:40:17.498 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:40:11.403725+0000', 'last_change': '2024-09-16T15:40:11.403725+0000', 'last_active': '2024-09-16T15:40:11.403725+0000', 'last_peered': '2024-09-16T15:40:11.403725+0000', 'last_clean': '2024-09-16T15:40:11.403725+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:11.403725+0000', 'last_undegraded': '2024-09-16T15:40:11.403725+0000', 'last_fullsized': '2024-09-16T15:40:11.403725+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:40:17.499 INFO:tasks.ceph.ceph_manager.ceph:PG 4.2 is not active+clean 2024-09-16T15:40:17.499 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:40:10.411507+0000', 'last_change': '2024-09-16T15:40:10.411507+0000', 'last_active': '2024-09-16T15:40:10.399062+0000', 'last_peered': '2024-09-16T15:40:10.399062+0000', 'last_clean': '2024-09-16T15:40:10.399062+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:10.411507+0000', 'last_undegraded': '2024-09-16T15:40:10.411507+0000', 'last_fullsized': '2024-09-16T15:40:10.411507+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-16T15:40:17.499 INFO:tasks.ceph.ceph_manager.ceph:PG 5.0 is not active+clean 2024-09-16T15:40:17.499 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:40:11.403725+0000', 'last_change': '2024-09-16T15:40:11.403725+0000', 'last_active': '2024-09-16T15:40:11.403725+0000', 'last_peered': '2024-09-16T15:40:11.403725+0000', 'last_clean': '2024-09-16T15:40:11.403725+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:11.403725+0000', 'last_undegraded': '2024-09-16T15:40:11.403725+0000', 'last_fullsized': '2024-09-16T15:40:11.403725+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:40:17.499 INFO:tasks.ceph.ceph_manager.ceph:PG 4.1 is not active+clean 2024-09-16T15:40:17.499 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:40:10.566998+0000', 'last_change': '2024-09-16T15:40:10.566998+0000', 'last_active': '2024-09-16T15:40:10.399062+0000', 'last_peered': '2024-09-16T15:40:10.399062+0000', 'last_clean': '2024-09-16T15:40:10.399062+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:10.566998+0000', 'last_undegraded': '2024-09-16T15:40:10.566998+0000', 'last_fullsized': '2024-09-16T15:40:10.566998+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T15:40:17.499 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1 is not active+clean 2024-09-16T15:40:17.499 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:40:11.403725+0000', 'last_change': '2024-09-16T15:40:11.403725+0000', 'last_active': '2024-09-16T15:40:11.403725+0000', 'last_peered': '2024-09-16T15:40:11.403725+0000', 'last_clean': '2024-09-16T15:40:11.403725+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:11.403725+0000', 'last_undegraded': '2024-09-16T15:40:11.403725+0000', 'last_fullsized': '2024-09-16T15:40:11.403725+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:40:17.499 INFO:tasks.ceph.ceph_manager.ceph:PG 4.0 is not active+clean 2024-09-16T15:40:17.499 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:40:10.408225+0000', 'last_change': '2024-09-16T15:40:10.408225+0000', 'last_active': '2024-09-16T15:40:10.399062+0000', 'last_peered': '2024-09-16T15:40:10.399062+0000', 'last_clean': '2024-09-16T15:40:10.399062+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:10.408225+0000', 'last_undegraded': '2024-09-16T15:40:10.408225+0000', 'last_fullsized': '2024-09-16T15:40:10.408225+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-16T15:40:17.499 INFO:tasks.ceph.ceph_manager.ceph:PG 5.4 is not active+clean 2024-09-16T15:40:17.499 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:40:11.403725+0000', 'last_change': '2024-09-16T15:40:11.403725+0000', 'last_active': '2024-09-16T15:40:11.403725+0000', 'last_peered': '2024-09-16T15:40:11.403725+0000', 'last_clean': '2024-09-16T15:40:11.403725+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:11.403725+0000', 'last_undegraded': '2024-09-16T15:40:11.403725+0000', 'last_fullsized': '2024-09-16T15:40:11.403725+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:11.403725+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:40:17.500 INFO:tasks.ceph.ceph_manager.ceph:PG 4.5 is not active+clean 2024-09-16T15:40:17.500 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:40:10.411572+0000', 'last_change': '2024-09-16T15:40:10.411572+0000', 'last_active': '2024-09-16T15:40:10.399062+0000', 'last_peered': '2024-09-16T15:40:10.399062+0000', 'last_clean': '2024-09-16T15:40:10.399062+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:40:10.411572+0000', 'last_undegraded': '2024-09-16T15:40:10.411572+0000', 'last_fullsized': '2024-09-16T15:40:10.411572+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'last_clean_scrub_stamp': '2024-09-16T15:40:10.399062+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T15:40:17.500 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T15:40:17.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.676+0000 7f7bfb200640 1 -- 172.21.15.151:0/1821873564 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7bfc0ccd70 msgr2=0x7f7bfc166b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:17.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.676+0000 7f7bfb200640 1 --2- 172.21.15.151:0/1821873564 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7bfc0ccd70 0x7f7bfc166b30 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7bf4005fd0 tx=0x7f7bf402a940 comp rx=0 tx=0).stop 2024-09-16T15:40:17.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.676+0000 7f7bfb200640 1 -- 172.21.15.151:0/1821873564 shutdown_connections 2024-09-16T15:40:17.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.676+0000 7f7bfb200640 1 --2- 172.21.15.151:0/1821873564 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7bfc0ccd70 0x7f7bfc166b30 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:17.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.676+0000 7f7bfb200640 1 -- 172.21.15.151:0/1821873564 >> 172.21.15.151:0/1821873564 conn(0x7f7bfc0b18e0 msgr2=0x7f7bfc0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:17.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.677+0000 7f7bfb200640 1 -- 172.21.15.151:0/1821873564 shutdown_connections 2024-09-16T15:40:17.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.677+0000 7f7bfb200640 1 -- 172.21.15.151:0/1821873564 wait complete. 2024-09-16T15:40:17.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.678+0000 7f7bfb200640 1 Processor -- start 2024-09-16T15:40:17.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.678+0000 7f7bfb200640 1 -- start start 2024-09-16T15:40:17.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.679+0000 7f7bfb200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7bfc06e9b0 0x7f7bfc14cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:17.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.679+0000 7f7bfb200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7bfc0ccd70 0x7f7bfc150920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:17.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.679+0000 7f7bfb200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7bfc1563a0 0x7f7bfc159f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:17.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.679+0000 7f7bfb200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7bfc1671f0 con 0x7f7bfc06e9b0 2024-09-16T15:40:17.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.679+0000 7f7bfb200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7bfc167070 con 0x7f7bfc1563a0 2024-09-16T15:40:17.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.679+0000 7f7bfb200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f7bfc167370 con 0x7f7bfc0ccd70 2024-09-16T15:40:17.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.679+0000 7f7bf9e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7bfc06e9b0 0x7f7bfc14cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:17.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.679+0000 7f7bf9e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7bfc06e9b0 0x7f7bfc14cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37344/0 (socket says 172.21.15.151:37344) 2024-09-16T15:40:17.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.679+0000 7f7bf9e00640 1 -- 172.21.15.151:0/1699092404 learned_addr learned my addr 172.21.15.151:0/1699092404 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:40:17.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.679+0000 7f7bfa800640 1 --2- 172.21.15.151:0/1699092404 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7bfc1563a0 0x7f7bfc159f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:17.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.679+0000 7f7bf3e00640 1 --2- 172.21.15.151:0/1699092404 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7bfc0ccd70 0x7f7bfc150920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:17.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.680+0000 7f7bf9e00640 1 -- 172.21.15.151:0/1699092404 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7bfc0ccd70 msgr2=0x7f7bfc150920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:17.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.680+0000 7f7bf9e00640 1 --2- 172.21.15.151:0/1699092404 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7bfc0ccd70 0x7f7bfc150920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:17.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.680+0000 7f7bf9e00640 1 -- 172.21.15.151:0/1699092404 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7bfc1563a0 msgr2=0x7f7bfc159f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:17.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.680+0000 7f7bf9e00640 1 --2- 172.21.15.151:0/1699092404 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7bfc1563a0 0x7f7bfc159f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:17.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.680+0000 7f7bf9e00640 1 -- 172.21.15.151:0/1699092404 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bfc1513e0 con 0x7f7bfc06e9b0 2024-09-16T15:40:17.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.680+0000 7f7bf9e00640 1 --2- 172.21.15.151:0/1699092404 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7bfc06e9b0 0x7f7bfc14cd10 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7bec001220 tx=0x7f7bec00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:17.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.680+0000 7f7bf2000640 1 -- 172.21.15.151:0/1699092404 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bec011790 con 0x7f7bfc06e9b0 2024-09-16T15:40:17.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.680+0000 7f7bf2000640 1 -- 172.21.15.151:0/1699092404 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7bec017070 con 0x7f7bfc06e9b0 2024-09-16T15:40:17.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.680+0000 7f7bfb200640 1 -- 172.21.15.151:0/1699092404 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bfc1516c0 con 0x7f7bfc06e9b0 2024-09-16T15:40:17.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.680+0000 7f7bf2000640 1 -- 172.21.15.151:0/1699092404 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bec011cc0 con 0x7f7bfc06e9b0 2024-09-16T15:40:17.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.680+0000 7f7bfb200640 1 -- 172.21.15.151:0/1699092404 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bfc151c80 con 0x7f7bfc06e9b0 2024-09-16T15:40:17.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.681+0000 7f7bfb200640 1 -- 172.21.15.151:0/1699092404 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bfc0cc660 con 0x7f7bfc06e9b0 2024-09-16T15:40:17.685 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.682+0000 7f7bf2000640 1 -- 172.21.15.151:0/1699092404 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f7bec01c460 con 0x7f7bfc06e9b0 2024-09-16T15:40:17.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.684+0000 7f7bf2000640 1 --2- 172.21.15.151:0/1699092404 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7bd8078850 0x7f7bd807ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:17.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.684+0000 7f7bf2000640 1 -- 172.21.15.151:0/1699092404 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f7bec09a6e0 con 0x7f7bfc06e9b0 2024-09-16T15:40:17.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.684+0000 7f7bf3e00640 1 --2- 172.21.15.151:0/1699092404 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7bd8078850 0x7f7bd807ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:17.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.685+0000 7f7bf3e00640 1 --2- 172.21.15.151:0/1699092404 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7bd8078850 0x7f7bd807ad10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7be4000c00 tx=0x7f7be4009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:17.690 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.688+0000 7f7bf2000640 1 -- 172.21.15.151:0/1699092404 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7bec062cb0 con 0x7f7bfc06e9b0 2024-09-16T15:40:17.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.947+0000 7f7bfb200640 1 -- 172.21.15.151:0/1699092404 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f7bfc070ae0 con 0x7f7bfc06e9b0 2024-09-16T15:40:17.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.948+0000 7f7bf2000640 1 -- 172.21.15.151:0/1699092404 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1762 (secure 0 0 0) 0x7f7bec0625b0 con 0x7f7bfc06e9b0 2024-09-16T15:40:17.950 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:40:17.951 INFO:teuthology.orchestra.run.smithi151.stdout:{"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":81,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":35,"num_osds":8,"num_up_osds":8,"osd_up_since":1726501150,"num_in_osds":8,"osd_in_since":1726501140,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":58},{"state_name":"creating+peering","count":28},{"state_name":"active+clean","count":11}],"num_pgs":97,"num_pools":3,"num_objects":8,"data_bytes":590390,"bytes_used":225443840,"bytes_avail":772868669440,"bytes_total":773094113280,"unknown_pgs_ratio":0.59793812036514282,"inactive_pgs_ratio":0.28865978121757507,"write_bytes_sec":197,"read_op_per_sec":0,"write_op_per_sec":3},"fsmap":{"epoch":17,"btime":"2024-09-16T15:40:15:450685+0000","id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"b","status":"up:active","gid":4855}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-09-16T15:40:11.588148+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-16T15:40:17.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.950+0000 7f7bfb200640 1 -- 172.21.15.151:0/1699092404 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7bd8078850 msgr2=0x7f7bd807ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:17.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.950+0000 7f7bfb200640 1 --2- 172.21.15.151:0/1699092404 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7bd8078850 0x7f7bd807ad10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7be4000c00 tx=0x7f7be4009040 comp rx=0 tx=0).stop 2024-09-16T15:40:17.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.950+0000 7f7bfb200640 1 -- 172.21.15.151:0/1699092404 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7bfc06e9b0 msgr2=0x7f7bfc14cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:17.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.950+0000 7f7bfb200640 1 --2- 172.21.15.151:0/1699092404 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7bfc06e9b0 0x7f7bfc14cd10 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7bec001220 tx=0x7f7bec00c890 comp rx=0 tx=0).stop 2024-09-16T15:40:17.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.950+0000 7f7bfb200640 1 -- 172.21.15.151:0/1699092404 shutdown_connections 2024-09-16T15:40:17.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.950+0000 7f7bfb200640 1 --2- 172.21.15.151:0/1699092404 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7bd8078850 0x7f7bd807ad10 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:17.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.950+0000 7f7bfb200640 1 --2- 172.21.15.151:0/1699092404 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7bfc1563a0 0x7f7bfc159f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:17.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.950+0000 7f7bfb200640 1 --2- 172.21.15.151:0/1699092404 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7bfc0ccd70 0x7f7bfc150920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:17.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.950+0000 7f7bfb200640 1 --2- 172.21.15.151:0/1699092404 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7bfc06e9b0 0x7f7bfc14cd10 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:17.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.950+0000 7f7bfb200640 1 -- 172.21.15.151:0/1699092404 >> 172.21.15.151:0/1699092404 conn(0x7f7bfc0b18e0 msgr2=0x7f7bfc0b0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:17.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.951+0000 7f7bfb200640 1 -- 172.21.15.151:0/1699092404 shutdown_connections 2024-09-16T15:40:17.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:17.951+0000 7f7bfb200640 1 -- 172.21.15.151:0/1699092404 wait complete. 2024-09-16T15:40:17.968 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-16T15:40:17.968 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:40:18.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:18.124+0000 7fa307200640 1 -- 172.21.15.151:0/3274749844 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3000cf300 msgr2=0x7fa3000729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:18.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:18.124+0000 7fa307200640 1 --2- 172.21.15.151:0/3274749844 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3000cf300 0x7fa3000729a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec009860 tx=0x7fa2ec02a5f0 comp rx=0 tx=0).stop 2024-09-16T15:40:18.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:18.124+0000 7fa307200640 1 -- 172.21.15.151:0/3274749844 shutdown_connections 2024-09-16T15:40:18.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:18.124+0000 7fa307200640 1 --2- 172.21.15.151:0/3274749844 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3000cf300 0x7fa3000729a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:18.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:18.124+0000 7fa307200640 1 -- 172.21.15.151:0/3274749844 >> 172.21.15.151:0/3274749844 conn(0x7fa3000b1950 msgr2=0x7fa3000b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:18.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:18.125+0000 7fa307200640 1 -- 172.21.15.151:0/3274749844 shutdown_connections 2024-09-16T15:40:18.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:18.125+0000 7fa307200640 1 -- 172.21.15.151:0/3274749844 wait complete. 2024-09-16T15:40:18.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:18.126+0000 7fa307200640 1 Processor -- start 2024-09-16T15:40:18.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:18.126+0000 7fa307200640 1 -- start start 2024-09-16T15:40:18.131 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:18.127+0000 7fa307200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3000ccfd0 0x7fa30009d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:18.131 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:18.127+0000 7fa307200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa3000cf300 0x7fa3001666f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:18.131 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:18.127+0000 7fa307200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa3001445d0 0x7fa3001f8a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:18.131 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:18.127+0000 7fa307200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa3000735f0 con 0x7fa3001445d0 2024-09-16T15:40:18.131 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:18.127+0000 7fa307200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa3000b4f60 con 0x7fa3000ccfd0 2024-09-16T15:40:18.131 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:18.127+0000 7fa307200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fa300073770 con 0x7fa3000cf300 2024-09-16T15:40:18.131 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:18.127+0000 7fa305200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa3001445d0 0x7fa3001f8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:18.131 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:18.127+0000 7fa305200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa3001445d0 0x7fa3001f8a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37350/0 (socket says 172.21.15.151:37350) 2024-09-16T15:40:18.131 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:18.127+0000 7fa305200640 1 -- 172.21.15.151:0/1443315716 learned_addr learned my addr 172.21.15.151:0/1443315716 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:40:18.131 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:18.127+0000 7fa2ff400640 1 --2- 172.21.15.151:0/1443315716 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa3000cf300 0x7fa3001666f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:18.131 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:18.128+0000 7fa2ffe00640 1 --2- 172.21.15.151:0/1443315716 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3000ccfd0 0x7fa30009d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:18.131 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:18.128+0000 7fa305200640 1 -- 172.21.15.151:0/1443315716 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa3000cf300 msgr2=0x7fa3001666f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:18.131 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:18.128+0000 7fa305200640 1 --2- 172.21.15.151:0/1443315716 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa3000cf300 0x7fa3001666f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:18.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:18.128+0000 7fa305200640 1 -- 172.21.15.151:0/1443315716 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3000ccfd0 msgr2=0x7fa30009d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:18.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:18.128+0000 7fa305200640 1 --2- 172.21.15.151:0/1443315716 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3000ccfd0 0x7fa30009d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:18.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:18.128+0000 7fa305200640 1 -- 172.21.15.151:0/1443315716 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa300166c30 con 0x7fa3001445d0 2024-09-16T15:40:18.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:18.128+0000 7fa305200640 1 --2- 172.21.15.151:0/1443315716 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa3001445d0 0x7fa3001f8a40 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa2f800cbc0 tx=0x7fa2f800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:18.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:18.128+0000 7fa2f7e00640 1 -- 172.21.15.151:0/1443315716 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f8016020 con 0x7fa3001445d0 2024-09-16T15:40:18.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:18.128+0000 7fa2f7e00640 1 -- 172.21.15.151:0/1443315716 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa2f800ae30 con 0x7fa3001445d0 2024-09-16T15:40:18.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:18.128+0000 7fa2f7e00640 1 -- 172.21.15.151:0/1443315716 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f8012660 con 0x7fa3001445d0 2024-09-16T15:40:18.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:18.128+0000 7fa307200640 1 -- 172.21.15.151:0/1443315716 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3001f8f80 con 0x7fa3001445d0 2024-09-16T15:40:18.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:18.129+0000 7fa307200640 1 -- 172.21.15.151:0/1443315716 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa30013ab20 con 0x7fa3001445d0 2024-09-16T15:40:18.133 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:18.130+0000 7fa307200640 1 -- 172.21.15.151:0/1443315716 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa30013ad20 con 0x7fa3001445d0 2024-09-16T15:40:18.133 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:18.131+0000 7fa2f7e00640 1 -- 172.21.15.151:0/1443315716 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fa2f8019020 con 0x7fa3001445d0 2024-09-16T15:40:18.134 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:18.132+0000 7fa2f7e00640 1 --2- 172.21.15.151:0/1443315716 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa2e0078880 0x7fa2e007ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:18.134 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:18.132+0000 7fa2f7e00640 1 -- 172.21.15.151:0/1443315716 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fa2f809aee0 con 0x7fa3001445d0 2024-09-16T15:40:18.134 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:18.132+0000 7fa2ffe00640 1 --2- 172.21.15.151:0/1443315716 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa2e0078880 0x7fa2e007ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:18.134 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:18.133+0000 7fa2ffe00640 1 --2- 172.21.15.151:0/1443315716 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa2e0078880 0x7fa2e007ad40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa2f0001120 tx=0x7fa2f0009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:18.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:18.136+0000 7fa2f7e00640 1 -- 172.21.15.151:0/1443315716 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa30013ad20 con 0x7fa3001445d0 2024-09-16T15:40:18.320 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:18.318+0000 7fa307200640 1 -- 172.21.15.151:0/1443315716 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa300073c20 con 0x7fa2e0078880 2024-09-16T15:40:18.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:18.323+0000 7fa2f7e00640 1 -- 172.21.15.151:0/1443315716 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251944 (secure 0 0 0) 0x7fa300073c20 con 0x7fa2e0078880 2024-09-16T15:40:18.328 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:40:18.329 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-09-16T15:40:18.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:18.327+0000 7fa307200640 1 -- 172.21.15.151:0/1443315716 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa2e0078880 msgr2=0x7fa2e007ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:18.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:18.327+0000 7fa307200640 1 --2- 172.21.15.151:0/1443315716 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa2e0078880 0x7fa2e007ad40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa2f0001120 tx=0x7fa2f0009040 comp rx=0 tx=0).stop 2024-09-16T15:40:18.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:18.327+0000 7fa307200640 1 -- 172.21.15.151:0/1443315716 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa3001445d0 msgr2=0x7fa3001f8a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:18.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:18.327+0000 7fa307200640 1 --2- 172.21.15.151:0/1443315716 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa3001445d0 0x7fa3001f8a40 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa2f800cbc0 tx=0x7fa2f800a430 comp rx=0 tx=0).stop 2024-09-16T15:40:18.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:18.327+0000 7fa307200640 1 -- 172.21.15.151:0/1443315716 shutdown_connections 2024-09-16T15:40:18.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:18.327+0000 7fa307200640 1 --2- 172.21.15.151:0/1443315716 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa2e0078880 0x7fa2e007ad40 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:18.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:18.327+0000 7fa307200640 1 --2- 172.21.15.151:0/1443315716 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa3001445d0 0x7fa3001f8a40 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:18.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:18.327+0000 7fa307200640 1 --2- 172.21.15.151:0/1443315716 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa3000cf300 0x7fa3001666f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:18.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:18.327+0000 7fa307200640 1 --2- 172.21.15.151:0/1443315716 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3000ccfd0 0x7fa30009d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:18.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:18.327+0000 7fa307200640 1 -- 172.21.15.151:0/1443315716 >> 172.21.15.151:0/1443315716 conn(0x7fa3000b1950 msgr2=0x7fa30006e920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:18.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:18.327+0000 7fa307200640 1 -- 172.21.15.151:0/1443315716 shutdown_connections 2024-09-16T15:40:18.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:18.327+0000 7fa307200640 1 -- 172.21.15.151:0/1443315716 wait complete. 2024-09-16T15:40:18.345 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":62,"stamp":"2024-09-16T15:40:17.589283+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220524,"kb_used_data":4264,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754196,"statfs":{"total":773094113280,"available":772868296704,"internally_reserved":0,"allocated":4366336,"data_stored":2398020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12738,"internal_metadata":221236798},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.181163"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.077510+0000","last_change":"2024-09-16T15:40:12.444116+0000","last_active":"2024-09-16T15:40:14.077510+0000","last_peered":"2024-09-16T15:40:14.077510+0000","last_clean":"2024-09-16T15:40:14.077510+0000","last_became_active":"2024-09-16T15:40:12.440025+0000","last_became_peered":"2024-09-16T15:40:12.440025+0000","last_unstale":"2024-09-16T15:40:14.077510+0000","last_undegraded":"2024-09-16T15:40:14.077510+0000","last_fullsized":"2024-09-16T15:40:14.077510+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:37:22.915098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.449833+0000","last_change":"2024-09-16T15:40:12.444480+0000","last_active":"2024-09-16T15:40:13.449833+0000","last_peered":"2024-09-16T15:40:13.449833+0000","last_clean":"2024-09-16T15:40:13.449833+0000","last_became_active":"2024-09-16T15:40:12.444220+0000","last_became_peered":"2024-09-16T15:40:12.444220+0000","last_unstale":"2024-09-16T15:40:13.449833+0000","last_undegraded":"2024-09-16T15:40:13.449833+0000","last_fullsized":"2024-09-16T15:40:13.449833+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:12:52.714116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.076722+0000","last_change":"2024-09-16T15:40:12.440385+0000","last_active":"2024-09-16T15:40:14.076722+0000","last_peered":"2024-09-16T15:40:14.076722+0000","last_clean":"2024-09-16T15:40:14.076722+0000","last_became_active":"2024-09-16T15:40:12.440138+0000","last_became_peered":"2024-09-16T15:40:12.440138+0000","last_unstale":"2024-09-16T15:40:14.076722+0000","last_undegraded":"2024-09-16T15:40:14.076722+0000","last_fullsized":"2024-09-16T15:40:14.076722+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:43:01.714863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.077877+0000","last_change":"2024-09-16T15:40:12.450256+0000","last_active":"2024-09-16T15:40:14.077877+0000","last_peered":"2024-09-16T15:40:14.077877+0000","last_clean":"2024-09-16T15:40:14.077877+0000","last_became_active":"2024-09-16T15:40:12.450019+0000","last_became_peered":"2024-09-16T15:40:12.450019+0000","last_unstale":"2024-09-16T15:40:14.077877+0000","last_undegraded":"2024-09-16T15:40:14.077877+0000","last_fullsized":"2024-09-16T15:40:14.077877+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:44:19.883823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.449738+0000","last_change":"2024-09-16T15:40:12.448384+0000","last_active":"2024-09-16T15:40:13.449738+0000","last_peered":"2024-09-16T15:40:13.449738+0000","last_clean":"2024-09-16T15:40:13.449738+0000","last_became_active":"2024-09-16T15:40:12.448088+0000","last_became_peered":"2024-09-16T15:40:12.448088+0000","last_unstale":"2024-09-16T15:40:13.449738+0000","last_undegraded":"2024-09-16T15:40:13.449738+0000","last_fullsized":"2024-09-16T15:40:13.449738+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:03:28.331224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.435856+0000","last_change":"2024-09-16T15:40:12.444964+0000","last_active":"2024-09-16T15:40:13.435856+0000","last_peered":"2024-09-16T15:40:13.435856+0000","last_clean":"2024-09-16T15:40:13.435856+0000","last_became_active":"2024-09-16T15:40:12.442884+0000","last_became_peered":"2024-09-16T15:40:12.442884+0000","last_unstale":"2024-09-16T15:40:13.435856+0000","last_undegraded":"2024-09-16T15:40:13.435856+0000","last_fullsized":"2024-09-16T15:40:13.435856+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:06:23.829480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.076833+0000","last_change":"2024-09-16T15:40:12.437515+0000","last_active":"2024-09-16T15:40:14.076833+0000","last_peered":"2024-09-16T15:40:14.076833+0000","last_clean":"2024-09-16T15:40:14.076833+0000","last_became_active":"2024-09-16T15:40:12.436738+0000","last_became_peered":"2024-09-16T15:40:12.436738+0000","last_unstale":"2024-09-16T15:40:14.076833+0000","last_undegraded":"2024-09-16T15:40:14.076833+0000","last_fullsized":"2024-09-16T15:40:14.076833+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:15:43.023428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.077093+0000","last_change":"2024-09-16T15:40:12.444472+0000","last_active":"2024-09-16T15:40:14.077093+0000","last_peered":"2024-09-16T15:40:14.077093+0000","last_clean":"2024-09-16T15:40:14.077093+0000","last_became_active":"2024-09-16T15:40:12.440102+0000","last_became_peered":"2024-09-16T15:40:12.440102+0000","last_unstale":"2024-09-16T15:40:14.077093+0000","last_undegraded":"2024-09-16T15:40:14.077093+0000","last_fullsized":"2024-09-16T15:40:14.077093+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:12:52.714116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.076787+0000","last_change":"2024-09-16T15:40:12.453854+0000","last_active":"2024-09-16T15:40:14.076787+0000","last_peered":"2024-09-16T15:40:14.076787+0000","last_clean":"2024-09-16T15:40:14.076787+0000","last_became_active":"2024-09-16T15:40:12.453625+0000","last_became_peered":"2024-09-16T15:40:12.453625+0000","last_unstale":"2024-09-16T15:40:14.076787+0000","last_undegraded":"2024-09-16T15:40:14.076787+0000","last_fullsized":"2024-09-16T15:40:14.076787+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:05:41.468279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.078173+0000","last_change":"2024-09-16T15:40:12.440112+0000","last_active":"2024-09-16T15:40:14.078173+0000","last_peered":"2024-09-16T15:40:14.078173+0000","last_clean":"2024-09-16T15:40:14.078173+0000","last_became_active":"2024-09-16T15:40:12.439159+0000","last_became_peered":"2024-09-16T15:40:12.439159+0000","last_unstale":"2024-09-16T15:40:14.078173+0000","last_undegraded":"2024-09-16T15:40:14.078173+0000","last_fullsized":"2024-09-16T15:40:14.078173+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:13:20.028628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.434696+0000","last_change":"2024-09-16T15:40:12.451264+0000","last_active":"2024-09-16T15:40:13.434696+0000","last_peered":"2024-09-16T15:40:13.434696+0000","last_clean":"2024-09-16T15:40:13.434696+0000","last_became_active":"2024-09-16T15:40:12.451000+0000","last_became_peered":"2024-09-16T15:40:12.451000+0000","last_unstale":"2024-09-16T15:40:13.434696+0000","last_undegraded":"2024-09-16T15:40:13.434696+0000","last_fullsized":"2024-09-16T15:40:13.434696+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:54:02.892922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.076784+0000","last_change":"2024-09-16T15:40:12.446787+0000","last_active":"2024-09-16T15:40:14.076784+0000","last_peered":"2024-09-16T15:40:14.076784+0000","last_clean":"2024-09-16T15:40:14.076784+0000","last_became_active":"2024-09-16T15:40:12.446456+0000","last_became_peered":"2024-09-16T15:40:12.446456+0000","last_unstale":"2024-09-16T15:40:14.076784+0000","last_undegraded":"2024-09-16T15:40:14.076784+0000","last_fullsized":"2024-09-16T15:40:14.076784+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:16:27.816839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.437025+0000","last_change":"2024-09-16T15:40:12.453651+0000","last_active":"2024-09-16T15:40:13.437025+0000","last_peered":"2024-09-16T15:40:13.437025+0000","last_clean":"2024-09-16T15:40:13.437025+0000","last_became_active":"2024-09-16T15:40:12.453385+0000","last_became_peered":"2024-09-16T15:40:12.453385+0000","last_unstale":"2024-09-16T15:40:13.437025+0000","last_undegraded":"2024-09-16T15:40:13.437025+0000","last_fullsized":"2024-09-16T15:40:13.437025+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:21:20.804588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.077490+0000","last_change":"2024-09-16T15:40:12.442875+0000","last_active":"2024-09-16T15:40:14.077490+0000","last_peered":"2024-09-16T15:40:14.077490+0000","last_clean":"2024-09-16T15:40:14.077490+0000","last_became_active":"2024-09-16T15:40:12.442552+0000","last_became_peered":"2024-09-16T15:40:12.442552+0000","last_unstale":"2024-09-16T15:40:14.077490+0000","last_undegraded":"2024-09-16T15:40:14.077490+0000","last_fullsized":"2024-09-16T15:40:14.077490+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:29:40.745908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.014602+0000","last_change":"2024-09-16T15:40:12.429955+0000","last_active":"2024-09-16T15:40:14.014602+0000","last_peered":"2024-09-16T15:40:14.014602+0000","last_clean":"2024-09-16T15:40:14.014602+0000","last_became_active":"2024-09-16T15:40:12.428839+0000","last_became_peered":"2024-09-16T15:40:12.428839+0000","last_unstale":"2024-09-16T15:40:14.014602+0000","last_undegraded":"2024-09-16T15:40:14.014602+0000","last_fullsized":"2024-09-16T15:40:14.014602+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:03:14.135051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.014596+0000","last_change":"2024-09-16T15:40:12.439315+0000","last_active":"2024-09-16T15:40:14.014596+0000","last_peered":"2024-09-16T15:40:14.014596+0000","last_clean":"2024-09-16T15:40:14.014596+0000","last_became_active":"2024-09-16T15:40:12.438629+0000","last_became_peered":"2024-09-16T15:40:12.438629+0000","last_unstale":"2024-09-16T15:40:14.014596+0000","last_undegraded":"2024-09-16T15:40:14.014596+0000","last_fullsized":"2024-09-16T15:40:14.014596+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:33:58.217901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.434759+0000","last_change":"2024-09-16T15:40:12.445302+0000","last_active":"2024-09-16T15:40:13.434759+0000","last_peered":"2024-09-16T15:40:13.434759+0000","last_clean":"2024-09-16T15:40:13.434759+0000","last_became_active":"2024-09-16T15:40:12.445014+0000","last_became_peered":"2024-09-16T15:40:12.445014+0000","last_unstale":"2024-09-16T15:40:13.434759+0000","last_undegraded":"2024-09-16T15:40:13.434759+0000","last_fullsized":"2024-09-16T15:40:13.434759+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:41:30.413571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.436268+0000","last_change":"2024-09-16T15:40:12.446472+0000","last_active":"2024-09-16T15:40:13.436268+0000","last_peered":"2024-09-16T15:40:13.436268+0000","last_clean":"2024-09-16T15:40:13.436268+0000","last_became_active":"2024-09-16T15:40:12.446204+0000","last_became_peered":"2024-09-16T15:40:12.446204+0000","last_unstale":"2024-09-16T15:40:13.436268+0000","last_undegraded":"2024-09-16T15:40:13.436268+0000","last_fullsized":"2024-09-16T15:40:13.436268+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:04:11.653249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.436116+0000","last_change":"2024-09-16T15:40:12.446725+0000","last_active":"2024-09-16T15:40:13.436116+0000","last_peered":"2024-09-16T15:40:13.436116+0000","last_clean":"2024-09-16T15:40:13.436116+0000","last_became_active":"2024-09-16T15:40:12.441589+0000","last_became_peered":"2024-09-16T15:40:12.441589+0000","last_unstale":"2024-09-16T15:40:13.436116+0000","last_undegraded":"2024-09-16T15:40:13.436116+0000","last_fullsized":"2024-09-16T15:40:13.436116+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:50:22.432002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.437263+0000","last_change":"2024-09-16T15:40:12.441110+0000","last_active":"2024-09-16T15:40:13.437263+0000","last_peered":"2024-09-16T15:40:13.437263+0000","last_clean":"2024-09-16T15:40:13.437263+0000","last_became_active":"2024-09-16T15:40:12.439699+0000","last_became_peered":"2024-09-16T15:40:12.439699+0000","last_unstale":"2024-09-16T15:40:13.437263+0000","last_undegraded":"2024-09-16T15:40:13.437263+0000","last_fullsized":"2024-09-16T15:40:13.437263+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:33:13.502942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.014799+0000","last_change":"2024-09-16T15:40:12.429020+0000","last_active":"2024-09-16T15:40:14.014799+0000","last_peered":"2024-09-16T15:40:14.014799+0000","last_clean":"2024-09-16T15:40:14.014799+0000","last_became_active":"2024-09-16T15:40:12.428774+0000","last_became_peered":"2024-09-16T15:40:12.428774+0000","last_unstale":"2024-09-16T15:40:14.014799+0000","last_undegraded":"2024-09-16T15:40:14.014799+0000","last_fullsized":"2024-09-16T15:40:14.014799+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:37:22.915098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.077059+0000","last_change":"2024-09-16T15:40:12.440419+0000","last_active":"2024-09-16T15:40:14.077059+0000","last_peered":"2024-09-16T15:40:14.077059+0000","last_clean":"2024-09-16T15:40:14.077059+0000","last_became_active":"2024-09-16T15:40:12.439833+0000","last_became_peered":"2024-09-16T15:40:12.439833+0000","last_unstale":"2024-09-16T15:40:14.077059+0000","last_undegraded":"2024-09-16T15:40:14.077059+0000","last_fullsized":"2024-09-16T15:40:14.077059+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:29:40.745908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.435353+0000","last_change":"2024-09-16T15:40:12.440517+0000","last_active":"2024-09-16T15:40:13.435353+0000","last_peered":"2024-09-16T15:40:13.435353+0000","last_clean":"2024-09-16T15:40:13.435353+0000","last_became_active":"2024-09-16T15:40:12.439873+0000","last_became_peered":"2024-09-16T15:40:12.439873+0000","last_unstale":"2024-09-16T15:40:13.435353+0000","last_undegraded":"2024-09-16T15:40:13.435353+0000","last_fullsized":"2024-09-16T15:40:13.435353+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:49:56.803512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.076598+0000","last_change":"2024-09-16T15:40:12.451869+0000","last_active":"2024-09-16T15:40:14.076598+0000","last_peered":"2024-09-16T15:40:14.076598+0000","last_clean":"2024-09-16T15:40:14.076598+0000","last_became_active":"2024-09-16T15:40:12.451593+0000","last_became_peered":"2024-09-16T15:40:12.451593+0000","last_unstale":"2024-09-16T15:40:14.076598+0000","last_undegraded":"2024-09-16T15:40:14.076598+0000","last_fullsized":"2024-09-16T15:40:14.076598+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:55:42.068271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.078341+0000","last_change":"2024-09-16T15:40:12.441706+0000","last_active":"2024-09-16T15:40:14.078341+0000","last_peered":"2024-09-16T15:40:14.078341+0000","last_clean":"2024-09-16T15:40:14.078341+0000","last_became_active":"2024-09-16T15:40:12.440405+0000","last_became_peered":"2024-09-16T15:40:12.440405+0000","last_unstale":"2024-09-16T15:40:14.078341+0000","last_undegraded":"2024-09-16T15:40:14.078341+0000","last_fullsized":"2024-09-16T15:40:14.078341+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:28:21.595104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.078227+0000","last_change":"2024-09-16T15:40:12.439748+0000","last_active":"2024-09-16T15:40:14.078227+0000","last_peered":"2024-09-16T15:40:14.078227+0000","last_clean":"2024-09-16T15:40:14.078227+0000","last_became_active":"2024-09-16T15:40:12.439238+0000","last_became_peered":"2024-09-16T15:40:12.439238+0000","last_unstale":"2024-09-16T15:40:14.078227+0000","last_undegraded":"2024-09-16T15:40:14.078227+0000","last_fullsized":"2024-09-16T15:40:14.078227+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:16:49.213767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.449976+0000","last_change":"2024-09-16T15:40:12.435380+0000","last_active":"2024-09-16T15:40:13.449976+0000","last_peered":"2024-09-16T15:40:13.449976+0000","last_clean":"2024-09-16T15:40:13.449976+0000","last_became_active":"2024-09-16T15:40:12.434744+0000","last_became_peered":"2024-09-16T15:40:12.434744+0000","last_unstale":"2024-09-16T15:40:13.449976+0000","last_undegraded":"2024-09-16T15:40:13.449976+0000","last_fullsized":"2024-09-16T15:40:13.449976+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:37:22.915098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.077539+0000","last_change":"2024-09-16T15:40:12.441070+0000","last_active":"2024-09-16T15:40:14.077539+0000","last_peered":"2024-09-16T15:40:14.077539+0000","last_clean":"2024-09-16T15:40:14.077539+0000","last_became_active":"2024-09-16T15:40:12.440809+0000","last_became_peered":"2024-09-16T15:40:12.440809+0000","last_unstale":"2024-09-16T15:40:14.077539+0000","last_undegraded":"2024-09-16T15:40:14.077539+0000","last_fullsized":"2024-09-16T15:40:14.077539+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:57:43.248746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.014592+0000","last_change":"2024-09-16T15:40:12.453157+0000","last_active":"2024-09-16T15:40:14.014592+0000","last_peered":"2024-09-16T15:40:14.014592+0000","last_clean":"2024-09-16T15:40:14.014592+0000","last_became_active":"2024-09-16T15:40:12.452815+0000","last_became_peered":"2024-09-16T15:40:12.452815+0000","last_unstale":"2024-09-16T15:40:14.014592+0000","last_undegraded":"2024-09-16T15:40:14.014592+0000","last_fullsized":"2024-09-16T15:40:14.014592+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:10:35.804720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.014581+0000","last_change":"2024-09-16T15:40:12.433443+0000","last_active":"2024-09-16T15:40:14.014581+0000","last_peered":"2024-09-16T15:40:14.014581+0000","last_clean":"2024-09-16T15:40:14.014581+0000","last_became_active":"2024-09-16T15:40:12.433169+0000","last_became_peered":"2024-09-16T15:40:12.433169+0000","last_unstale":"2024-09-16T15:40:14.014581+0000","last_undegraded":"2024-09-16T15:40:14.014581+0000","last_fullsized":"2024-09-16T15:40:14.014581+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:03:28.331224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.077033+0000","last_change":"2024-09-16T15:40:12.437804+0000","last_active":"2024-09-16T15:40:14.077033+0000","last_peered":"2024-09-16T15:40:14.077033+0000","last_clean":"2024-09-16T15:40:14.077033+0000","last_became_active":"2024-09-16T15:40:12.437269+0000","last_became_peered":"2024-09-16T15:40:12.437269+0000","last_unstale":"2024-09-16T15:40:14.077033+0000","last_undegraded":"2024-09-16T15:40:14.077033+0000","last_fullsized":"2024-09-16T15:40:14.077033+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:10:35.804720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.434631+0000","last_change":"2024-09-16T15:40:12.445107+0000","last_active":"2024-09-16T15:40:13.434631+0000","last_peered":"2024-09-16T15:40:13.434631+0000","last_clean":"2024-09-16T15:40:13.434631+0000","last_became_active":"2024-09-16T15:40:12.444866+0000","last_became_peered":"2024-09-16T15:40:12.444866+0000","last_unstale":"2024-09-16T15:40:13.434631+0000","last_undegraded":"2024-09-16T15:40:13.434631+0000","last_fullsized":"2024-09-16T15:40:13.434631+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:13:58.747871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.437465+0000","last_change":"2024-09-16T15:40:12.436931+0000","last_active":"2024-09-16T15:40:13.437465+0000","last_peered":"2024-09-16T15:40:13.437465+0000","last_clean":"2024-09-16T15:40:13.437465+0000","last_became_active":"2024-09-16T15:40:12.436211+0000","last_became_peered":"2024-09-16T15:40:12.436211+0000","last_unstale":"2024-09-16T15:40:13.437465+0000","last_undegraded":"2024-09-16T15:40:13.437465+0000","last_fullsized":"2024-09-16T15:40:13.437465+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:21:12.132805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"34'4","reported_seq":19,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.434826+0000","last_change":"2024-09-16T15:40:11.426141+0000","last_active":"2024-09-16T15:40:13.434826+0000","last_peered":"2024-09-16T15:40:13.434826+0000","last_clean":"2024-09-16T15:40:13.434826+0000","last_became_active":"2024-09-16T15:40:11.425862+0000","last_became_peered":"2024-09-16T15:40:11.425862+0000","last_unstale":"2024-09-16T15:40:13.434826+0000","last_undegraded":"2024-09-16T15:40:13.434826+0000","last_fullsized":"2024-09-16T15:40:13.434826+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:37:21.910435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.077683+0000","last_change":"2024-09-16T15:40:12.438772+0000","last_active":"2024-09-16T15:40:14.077683+0000","last_peered":"2024-09-16T15:40:14.077683+0000","last_clean":"2024-09-16T15:40:14.077683+0000","last_became_active":"2024-09-16T15:40:12.436489+0000","last_became_peered":"2024-09-16T15:40:12.436489+0000","last_unstale":"2024-09-16T15:40:14.077683+0000","last_undegraded":"2024-09-16T15:40:14.077683+0000","last_fullsized":"2024-09-16T15:40:14.077683+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:15:46.661424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"34'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.078242+0000","last_change":"2024-09-16T15:40:11.429198+0000","last_active":"2024-09-16T15:40:14.078242+0000","last_peered":"2024-09-16T15:40:14.078242+0000","last_clean":"2024-09-16T15:40:14.078242+0000","last_became_active":"2024-09-16T15:40:11.428894+0000","last_became_peered":"2024-09-16T15:40:11.428894+0000","last_unstale":"2024-09-16T15:40:14.078242+0000","last_undegraded":"2024-09-16T15:40:14.078242+0000","last_fullsized":"2024-09-16T15:40:14.078242+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:46:10.721679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.077561+0000","last_change":"2024-09-16T15:40:12.452116+0000","last_active":"2024-09-16T15:40:14.077561+0000","last_peered":"2024-09-16T15:40:14.077561+0000","last_clean":"2024-09-16T15:40:14.077561+0000","last_became_active":"2024-09-16T15:40:12.451065+0000","last_became_peered":"2024-09-16T15:40:12.451065+0000","last_unstale":"2024-09-16T15:40:14.077561+0000","last_undegraded":"2024-09-16T15:40:14.077561+0000","last_fullsized":"2024-09-16T15:40:14.077561+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:00:15.773418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"34'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.436310+0000","last_change":"2024-09-16T15:40:11.429304+0000","last_active":"2024-09-16T15:40:13.436310+0000","last_peered":"2024-09-16T15:40:13.436310+0000","last_clean":"2024-09-16T15:40:13.436310+0000","last_became_active":"2024-09-16T15:40:11.429029+0000","last_became_peered":"2024-09-16T15:40:11.429029+0000","last_unstale":"2024-09-16T15:40:13.436310+0000","last_undegraded":"2024-09-16T15:40:13.436310+0000","last_fullsized":"2024-09-16T15:40:13.436310+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:31:10.242510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.078117+0000","last_change":"2024-09-16T15:40:12.448098+0000","last_active":"2024-09-16T15:40:14.078117+0000","last_peered":"2024-09-16T15:40:14.078117+0000","last_clean":"2024-09-16T15:40:14.078117+0000","last_became_active":"2024-09-16T15:40:12.447795+0000","last_became_peered":"2024-09-16T15:40:12.447795+0000","last_unstale":"2024-09-16T15:40:14.078117+0000","last_undegraded":"2024-09-16T15:40:14.078117+0000","last_fullsized":"2024-09-16T15:40:14.078117+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:08:40.384527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"34'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.077947+0000","last_change":"2024-09-16T15:40:11.432946+0000","last_active":"2024-09-16T15:40:14.077947+0000","last_peered":"2024-09-16T15:40:14.077947+0000","last_clean":"2024-09-16T15:40:14.077947+0000","last_became_active":"2024-09-16T15:40:11.431415+0000","last_became_peered":"2024-09-16T15:40:11.431415+0000","last_unstale":"2024-09-16T15:40:14.077947+0000","last_undegraded":"2024-09-16T15:40:14.077947+0000","last_fullsized":"2024-09-16T15:40:14.077947+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:59:15.340271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.076564+0000","last_change":"2024-09-16T15:40:12.452431+0000","last_active":"2024-09-16T15:40:14.076564+0000","last_peered":"2024-09-16T15:40:14.076564+0000","last_clean":"2024-09-16T15:40:14.076564+0000","last_became_active":"2024-09-16T15:40:12.452156+0000","last_became_peered":"2024-09-16T15:40:12.452156+0000","last_unstale":"2024-09-16T15:40:14.076564+0000","last_undegraded":"2024-09-16T15:40:14.076564+0000","last_fullsized":"2024-09-16T15:40:14.076564+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:09:37.707603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"34'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.077221+0000","last_change":"2024-09-16T15:40:11.437040+0000","last_active":"2024-09-16T15:40:14.077221+0000","last_peered":"2024-09-16T15:40:14.077221+0000","last_clean":"2024-09-16T15:40:14.077221+0000","last_became_active":"2024-09-16T15:40:11.436779+0000","last_became_peered":"2024-09-16T15:40:11.436779+0000","last_unstale":"2024-09-16T15:40:14.077221+0000","last_undegraded":"2024-09-16T15:40:14.077221+0000","last_fullsized":"2024-09-16T15:40:14.077221+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:43:13.979677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.077292+0000","last_change":"2024-09-16T15:40:12.439366+0000","last_active":"2024-09-16T15:40:14.077292+0000","last_peered":"2024-09-16T15:40:14.077292+0000","last_clean":"2024-09-16T15:40:14.077292+0000","last_became_active":"2024-09-16T15:40:12.437351+0000","last_became_peered":"2024-09-16T15:40:12.437351+0000","last_unstale":"2024-09-16T15:40:14.077292+0000","last_undegraded":"2024-09-16T15:40:14.077292+0000","last_fullsized":"2024-09-16T15:40:14.077292+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:54:46.876716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"34'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.077173+0000","last_change":"2024-09-16T15:40:11.436026+0000","last_active":"2024-09-16T15:40:14.077173+0000","last_peered":"2024-09-16T15:40:14.077173+0000","last_clean":"2024-09-16T15:40:14.077173+0000","last_became_active":"2024-09-16T15:40:11.435807+0000","last_became_peered":"2024-09-16T15:40:11.435807+0000","last_unstale":"2024-09-16T15:40:14.077173+0000","last_undegraded":"2024-09-16T15:40:14.077173+0000","last_fullsized":"2024-09-16T15:40:14.077173+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:27:41.510684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.076590+0000","last_change":"2024-09-16T15:40:12.435295+0000","last_active":"2024-09-16T15:40:14.076590+0000","last_peered":"2024-09-16T15:40:14.076590+0000","last_clean":"2024-09-16T15:40:14.076590+0000","last_became_active":"2024-09-16T15:40:12.434978+0000","last_became_peered":"2024-09-16T15:40:12.434978+0000","last_unstale":"2024-09-16T15:40:14.076590+0000","last_undegraded":"2024-09-16T15:40:14.076590+0000","last_fullsized":"2024-09-16T15:40:14.076590+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:43:14.984339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.435951+0000","last_change":"2024-09-16T15:40:11.431791+0000","last_active":"2024-09-16T15:40:13.435951+0000","last_peered":"2024-09-16T15:40:13.435951+0000","last_clean":"2024-09-16T15:40:13.435951+0000","last_became_active":"2024-09-16T15:40:11.431517+0000","last_became_peered":"2024-09-16T15:40:11.431517+0000","last_unstale":"2024-09-16T15:40:13.435951+0000","last_undegraded":"2024-09-16T15:40:13.435951+0000","last_fullsized":"2024-09-16T15:40:13.435951+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:17:20.232340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.077597+0000","last_change":"2024-09-16T15:40:12.447547+0000","last_active":"2024-09-16T15:40:14.077597+0000","last_peered":"2024-09-16T15:40:14.077597+0000","last_clean":"2024-09-16T15:40:14.077597+0000","last_became_active":"2024-09-16T15:40:12.447244+0000","last_became_peered":"2024-09-16T15:40:12.447244+0000","last_unstale":"2024-09-16T15:40:14.077597+0000","last_undegraded":"2024-09-16T15:40:14.077597+0000","last_fullsized":"2024-09-16T15:40:14.077597+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:09:28.653840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.077113+0000","last_change":"2024-09-16T15:40:11.434384+0000","last_active":"2024-09-16T15:40:14.077113+0000","last_peered":"2024-09-16T15:40:14.077113+0000","last_clean":"2024-09-16T15:40:14.077113+0000","last_became_active":"2024-09-16T15:40:11.434028+0000","last_became_peered":"2024-09-16T15:40:11.434028+0000","last_unstale":"2024-09-16T15:40:14.077113+0000","last_undegraded":"2024-09-16T15:40:14.077113+0000","last_fullsized":"2024-09-16T15:40:14.077113+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:59:15.340271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.434969+0000","last_change":"2024-09-16T15:40:12.442342+0000","last_active":"2024-09-16T15:40:13.434969+0000","last_peered":"2024-09-16T15:40:13.434969+0000","last_clean":"2024-09-16T15:40:13.434969+0000","last_became_active":"2024-09-16T15:40:12.442051+0000","last_became_peered":"2024-09-16T15:40:12.442051+0000","last_unstale":"2024-09-16T15:40:13.434969+0000","last_undegraded":"2024-09-16T15:40:13.434969+0000","last_fullsized":"2024-09-16T15:40:13.434969+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:47:50.659996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.449970+0000","last_change":"2024-09-16T15:40:11.430380+0000","last_active":"2024-09-16T15:40:13.449970+0000","last_peered":"2024-09-16T15:40:13.449970+0000","last_clean":"2024-09-16T15:40:13.449970+0000","last_became_active":"2024-09-16T15:40:11.428309+0000","last_became_peered":"2024-09-16T15:40:11.428309+0000","last_unstale":"2024-09-16T15:40:13.449970+0000","last_undegraded":"2024-09-16T15:40:13.449970+0000","last_fullsized":"2024-09-16T15:40:13.449970+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:37:43.766286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.078318+0000","last_change":"2024-09-16T15:40:12.439953+0000","last_active":"2024-09-16T15:40:14.078318+0000","last_peered":"2024-09-16T15:40:14.078318+0000","last_clean":"2024-09-16T15:40:14.078318+0000","last_became_active":"2024-09-16T15:40:12.438714+0000","last_became_peered":"2024-09-16T15:40:12.438714+0000","last_unstale":"2024-09-16T15:40:14.078318+0000","last_undegraded":"2024-09-16T15:40:14.078318+0000","last_fullsized":"2024-09-16T15:40:14.078318+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:24:50.389299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.449724+0000","last_change":"2024-09-16T15:40:11.426767+0000","last_active":"2024-09-16T15:40:13.449724+0000","last_peered":"2024-09-16T15:40:13.449724+0000","last_clean":"2024-09-16T15:40:13.449724+0000","last_became_active":"2024-09-16T15:40:11.426512+0000","last_became_peered":"2024-09-16T15:40:11.426512+0000","last_unstale":"2024-09-16T15:40:13.449724+0000","last_undegraded":"2024-09-16T15:40:13.449724+0000","last_fullsized":"2024-09-16T15:40:13.449724+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:48:35.010191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.077913+0000","last_change":"2024-09-16T15:40:12.447596+0000","last_active":"2024-09-16T15:40:14.077913+0000","last_peered":"2024-09-16T15:40:14.077913+0000","last_clean":"2024-09-16T15:40:14.077913+0000","last_became_active":"2024-09-16T15:40:12.447303+0000","last_became_peered":"2024-09-16T15:40:12.447303+0000","last_unstale":"2024-09-16T15:40:14.077913+0000","last_undegraded":"2024-09-16T15:40:14.077913+0000","last_fullsized":"2024-09-16T15:40:14.077913+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:36:34.759651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"34'3","reported_seq":18,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.434442+0000","last_change":"2024-09-16T15:40:11.433864+0000","last_active":"2024-09-16T15:40:13.434442+0000","last_peered":"2024-09-16T15:40:13.434442+0000","last_clean":"2024-09-16T15:40:13.434442+0000","last_became_active":"2024-09-16T15:40:11.426559+0000","last_became_peered":"2024-09-16T15:40:11.426559+0000","last_unstale":"2024-09-16T15:40:13.434442+0000","last_undegraded":"2024-09-16T15:40:13.434442+0000","last_fullsized":"2024-09-16T15:40:13.434442+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:12:51.709454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.434553+0000","last_change":"2024-09-16T15:40:12.456376+0000","last_active":"2024-09-16T15:40:13.434553+0000","last_peered":"2024-09-16T15:40:13.434553+0000","last_clean":"2024-09-16T15:40:13.434553+0000","last_became_active":"2024-09-16T15:40:12.455834+0000","last_became_peered":"2024-09-16T15:40:12.455834+0000","last_unstale":"2024-09-16T15:40:13.434553+0000","last_undegraded":"2024-09-16T15:40:13.434553+0000","last_fullsized":"2024-09-16T15:40:13.434553+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:58:20.955348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"34'6","reported_seq":21,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.437346+0000","last_change":"2024-09-16T15:40:11.432845+0000","last_active":"2024-09-16T15:40:13.437346+0000","last_peered":"2024-09-16T15:40:13.437346+0000","last_clean":"2024-09-16T15:40:13.437346+0000","last_became_active":"2024-09-16T15:40:11.432442+0000","last_became_peered":"2024-09-16T15:40:11.432442+0000","last_unstale":"2024-09-16T15:40:13.437346+0000","last_undegraded":"2024-09-16T15:40:13.437346+0000","last_fullsized":"2024-09-16T15:40:13.437346+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:16:26.812176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.436372+0000","last_change":"2024-09-16T15:40:12.436479+0000","last_active":"2024-09-16T15:40:13.436372+0000","last_peered":"2024-09-16T15:40:13.436372+0000","last_clean":"2024-09-16T15:40:13.436372+0000","last_became_active":"2024-09-16T15:40:12.436247+0000","last_became_peered":"2024-09-16T15:40:12.436247+0000","last_unstale":"2024-09-16T15:40:13.436372+0000","last_undegraded":"2024-09-16T15:40:13.436372+0000","last_fullsized":"2024-09-16T15:40:13.436372+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:13:44.038668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"34'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.077325+0000","last_change":"2024-09-16T15:40:11.439929+0000","last_active":"2024-09-16T15:40:14.077325+0000","last_peered":"2024-09-16T15:40:14.077325+0000","last_clean":"2024-09-16T15:40:14.077325+0000","last_became_active":"2024-09-16T15:40:11.439583+0000","last_became_peered":"2024-09-16T15:40:11.439583+0000","last_unstale":"2024-09-16T15:40:14.077325+0000","last_undegraded":"2024-09-16T15:40:14.077325+0000","last_fullsized":"2024-09-16T15:40:14.077325+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:43:00.710200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.077800+0000","last_change":"2024-09-16T15:40:12.444800+0000","last_active":"2024-09-16T15:40:14.077800+0000","last_peered":"2024-09-16T15:40:14.077800+0000","last_clean":"2024-09-16T15:40:14.077800+0000","last_became_active":"2024-09-16T15:40:12.444523+0000","last_became_peered":"2024-09-16T15:40:12.444523+0000","last_unstale":"2024-09-16T15:40:14.077800+0000","last_undegraded":"2024-09-16T15:40:14.077800+0000","last_fullsized":"2024-09-16T15:40:14.077800+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:03:15.062570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.077333+0000","last_change":"2024-09-16T15:40:11.426982+0000","last_active":"2024-09-16T15:40:14.077333+0000","last_peered":"2024-09-16T15:40:14.077333+0000","last_clean":"2024-09-16T15:40:14.077333+0000","last_became_active":"2024-09-16T15:40:11.426667+0000","last_became_peered":"2024-09-16T15:40:11.426667+0000","last_unstale":"2024-09-16T15:40:14.077333+0000","last_undegraded":"2024-09-16T15:40:14.077333+0000","last_fullsized":"2024-09-16T15:40:14.077333+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:27:41.510684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.077055+0000","last_change":"2024-09-16T15:40:12.436737+0000","last_active":"2024-09-16T15:40:14.077055+0000","last_peered":"2024-09-16T15:40:14.077055+0000","last_clean":"2024-09-16T15:40:14.077055+0000","last_became_active":"2024-09-16T15:40:12.435893+0000","last_became_peered":"2024-09-16T15:40:12.435893+0000","last_unstale":"2024-09-16T15:40:14.077055+0000","last_undegraded":"2024-09-16T15:40:14.077055+0000","last_fullsized":"2024-09-16T15:40:14.077055+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:03:28.331224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.077161+0000","last_change":"2024-09-16T15:40:11.426297+0000","last_active":"2024-09-16T15:40:14.077161+0000","last_peered":"2024-09-16T15:40:14.077161+0000","last_clean":"2024-09-16T15:40:14.077161+0000","last_became_active":"2024-09-16T15:40:11.425935+0000","last_became_peered":"2024-09-16T15:40:11.425935+0000","last_unstale":"2024-09-16T15:40:14.077161+0000","last_undegraded":"2024-09-16T15:40:14.077161+0000","last_fullsized":"2024-09-16T15:40:14.077161+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:05:40.463616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.434380+0000","last_change":"2024-09-16T15:40:12.444134+0000","last_active":"2024-09-16T15:40:13.434380+0000","last_peered":"2024-09-16T15:40:13.434380+0000","last_clean":"2024-09-16T15:40:13.434380+0000","last_became_active":"2024-09-16T15:40:12.443929+0000","last_became_peered":"2024-09-16T15:40:12.443929+0000","last_unstale":"2024-09-16T15:40:13.434380+0000","last_undegraded":"2024-09-16T15:40:13.434380+0000","last_fullsized":"2024-09-16T15:40:13.434380+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:51:48.095296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"34'8","reported_seq":23,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.435447+0000","last_change":"2024-09-16T15:40:11.425677+0000","last_active":"2024-09-16T15:40:13.435447+0000","last_peered":"2024-09-16T15:40:13.435447+0000","last_clean":"2024-09-16T15:40:13.435447+0000","last_became_active":"2024-09-16T15:40:11.425390+0000","last_became_peered":"2024-09-16T15:40:11.425390+0000","last_unstale":"2024-09-16T15:40:13.435447+0000","last_undegraded":"2024-09-16T15:40:13.435447+0000","last_fullsized":"2024-09-16T15:40:13.435447+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:02:34.885995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.434958+0000","last_change":"2024-09-16T15:40:12.455503+0000","last_active":"2024-09-16T15:40:13.434958+0000","last_peered":"2024-09-16T15:40:13.434958+0000","last_clean":"2024-09-16T15:40:13.434958+0000","last_became_active":"2024-09-16T15:40:12.455167+0000","last_became_peered":"2024-09-16T15:40:12.455167+0000","last_unstale":"2024-09-16T15:40:13.434958+0000","last_undegraded":"2024-09-16T15:40:13.434958+0000","last_fullsized":"2024-09-16T15:40:13.434958+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:50:57.607623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"34'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.436125+0000","last_change":"2024-09-16T15:40:11.433816+0000","last_active":"2024-09-16T15:40:13.436125+0000","last_peered":"2024-09-16T15:40:13.436125+0000","last_clean":"2024-09-16T15:40:13.436125+0000","last_became_active":"2024-09-16T15:40:11.433028+0000","last_became_peered":"2024-09-16T15:40:11.433028+0000","last_unstale":"2024-09-16T15:40:13.436125+0000","last_undegraded":"2024-09-16T15:40:13.436125+0000","last_fullsized":"2024-09-16T15:40:13.436125+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:05:55.235854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.076987+0000","last_change":"2024-09-16T15:40:12.435921+0000","last_active":"2024-09-16T15:40:14.076987+0000","last_peered":"2024-09-16T15:40:14.076987+0000","last_clean":"2024-09-16T15:40:14.076987+0000","last_became_active":"2024-09-16T15:40:12.435533+0000","last_became_peered":"2024-09-16T15:40:12.435533+0000","last_unstale":"2024-09-16T15:40:14.076987+0000","last_undegraded":"2024-09-16T15:40:14.076987+0000","last_fullsized":"2024-09-16T15:40:14.076987+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:27:42.515347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"34'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.077304+0000","last_change":"2024-09-16T15:40:11.439905+0000","last_active":"2024-09-16T15:40:14.077304+0000","last_peered":"2024-09-16T15:40:14.077304+0000","last_clean":"2024-09-16T15:40:14.077304+0000","last_became_active":"2024-09-16T15:40:11.439567+0000","last_became_peered":"2024-09-16T15:40:11.439567+0000","last_unstale":"2024-09-16T15:40:14.077304+0000","last_undegraded":"2024-09-16T15:40:14.077304+0000","last_fullsized":"2024-09-16T15:40:14.077304+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:40:52.912231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.449794+0000","last_change":"2024-09-16T15:40:12.436487+0000","last_active":"2024-09-16T15:40:13.449794+0000","last_peered":"2024-09-16T15:40:13.449794+0000","last_clean":"2024-09-16T15:40:13.449794+0000","last_became_active":"2024-09-16T15:40:12.436193+0000","last_became_peered":"2024-09-16T15:40:12.436193+0000","last_unstale":"2024-09-16T15:40:13.449794+0000","last_undegraded":"2024-09-16T15:40:13.449794+0000","last_fullsized":"2024-09-16T15:40:13.449794+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:33:58.217901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"34'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.434609+0000","last_change":"2024-09-16T15:40:11.432937+0000","last_active":"2024-09-16T15:40:13.434609+0000","last_peered":"2024-09-16T15:40:13.434609+0000","last_clean":"2024-09-16T15:40:13.434609+0000","last_became_active":"2024-09-16T15:40:11.428199+0000","last_became_peered":"2024-09-16T15:40:11.428199+0000","last_unstale":"2024-09-16T15:40:13.434609+0000","last_undegraded":"2024-09-16T15:40:13.434609+0000","last_fullsized":"2024-09-16T15:40:13.434609+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:33:57.213238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.436182+0000","last_change":"2024-09-16T15:40:12.438579+0000","last_active":"2024-09-16T15:40:13.436182+0000","last_peered":"2024-09-16T15:40:13.436182+0000","last_clean":"2024-09-16T15:40:13.436182+0000","last_became_active":"2024-09-16T15:40:12.437311+0000","last_became_peered":"2024-09-16T15:40:12.437311+0000","last_unstale":"2024-09-16T15:40:13.436182+0000","last_undegraded":"2024-09-16T15:40:13.436182+0000","last_fullsized":"2024-09-16T15:40:13.436182+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:04:48.883149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.014769+0000","last_change":"2024-09-16T15:40:11.433487+0000","last_active":"2024-09-16T15:40:14.014769+0000","last_peered":"2024-09-16T15:40:14.014769+0000","last_clean":"2024-09-16T15:40:14.014769+0000","last_became_active":"2024-09-16T15:40:11.431531+0000","last_became_peered":"2024-09-16T15:40:11.431531+0000","last_unstale":"2024-09-16T15:40:14.014769+0000","last_undegraded":"2024-09-16T15:40:14.014769+0000","last_fullsized":"2024-09-16T15:40:14.014769+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:26:31.371355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.435633+0000","last_change":"2024-09-16T15:40:12.437874+0000","last_active":"2024-09-16T15:40:13.435633+0000","last_peered":"2024-09-16T15:40:13.435633+0000","last_clean":"2024-09-16T15:40:13.435633+0000","last_became_active":"2024-09-16T15:40:12.436684+0000","last_became_peered":"2024-09-16T15:40:12.436684+0000","last_unstale":"2024-09-16T15:40:13.435633+0000","last_undegraded":"2024-09-16T15:40:13.435633+0000","last_fullsized":"2024-09-16T15:40:13.435633+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:38:27.284097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"34'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.449810+0000","last_change":"2024-09-16T15:40:11.421728+0000","last_active":"2024-09-16T15:40:13.449810+0000","last_peered":"2024-09-16T15:40:13.449810+0000","last_clean":"2024-09-16T15:40:13.449810+0000","last_became_active":"2024-09-16T15:40:11.421458+0000","last_became_peered":"2024-09-16T15:40:11.421458+0000","last_unstale":"2024-09-16T15:40:13.449810+0000","last_undegraded":"2024-09-16T15:40:13.449810+0000","last_fullsized":"2024-09-16T15:40:13.449810+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:19:01.909987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.449835+0000","last_change":"2024-09-16T15:40:12.438536+0000","last_active":"2024-09-16T15:40:13.449835+0000","last_peered":"2024-09-16T15:40:13.449835+0000","last_clean":"2024-09-16T15:40:13.449835+0000","last_became_active":"2024-09-16T15:40:12.438227+0000","last_became_peered":"2024-09-16T15:40:12.438227+0000","last_unstale":"2024-09-16T15:40:13.449835+0000","last_undegraded":"2024-09-16T15:40:13.449835+0000","last_fullsized":"2024-09-16T15:40:13.449835+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:26:09.350575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.077176+0000","last_change":"2024-09-16T15:40:11.440951+0000","last_active":"2024-09-16T15:40:14.077176+0000","last_peered":"2024-09-16T15:40:14.077176+0000","last_clean":"2024-09-16T15:40:14.077176+0000","last_became_active":"2024-09-16T15:40:11.440590+0000","last_became_peered":"2024-09-16T15:40:11.440590+0000","last_unstale":"2024-09-16T15:40:14.077176+0000","last_undegraded":"2024-09-16T15:40:14.077176+0000","last_fullsized":"2024-09-16T15:40:14.077176+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:16:26.812176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.435374+0000","last_change":"2024-09-16T15:40:12.452533+0000","last_active":"2024-09-16T15:40:13.435374+0000","last_peered":"2024-09-16T15:40:13.435374+0000","last_clean":"2024-09-16T15:40:13.435374+0000","last_became_active":"2024-09-16T15:40:12.451706+0000","last_became_peered":"2024-09-16T15:40:12.451706+0000","last_unstale":"2024-09-16T15:40:13.435374+0000","last_undegraded":"2024-09-16T15:40:13.435374+0000","last_fullsized":"2024-09-16T15:40:13.435374+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:18:02.318732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.077074+0000","last_change":"2024-09-16T15:40:11.429461+0000","last_active":"2024-09-16T15:40:14.077074+0000","last_peered":"2024-09-16T15:40:14.077074+0000","last_clean":"2024-09-16T15:40:14.077074+0000","last_became_active":"2024-09-16T15:40:11.429262+0000","last_became_peered":"2024-09-16T15:40:11.429262+0000","last_unstale":"2024-09-16T15:40:14.077074+0000","last_undegraded":"2024-09-16T15:40:14.077074+0000","last_fullsized":"2024-09-16T15:40:14.077074+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:23:23.370290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.076578+0000","last_change":"2024-09-16T15:40:12.435345+0000","last_active":"2024-09-16T15:40:14.076578+0000","last_peered":"2024-09-16T15:40:14.076578+0000","last_clean":"2024-09-16T15:40:14.076578+0000","last_became_active":"2024-09-16T15:40:12.435060+0000","last_became_peered":"2024-09-16T15:40:12.435060+0000","last_unstale":"2024-09-16T15:40:14.076578+0000","last_undegraded":"2024-09-16T15:40:14.076578+0000","last_fullsized":"2024-09-16T15:40:14.076578+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:40:53.916894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.434156+0000","last_change":"2024-09-16T15:40:11.435516+0000","last_active":"2024-09-16T15:40:13.434156+0000","last_peered":"2024-09-16T15:40:13.434156+0000","last_clean":"2024-09-16T15:40:13.434156+0000","last_became_active":"2024-09-16T15:40:11.433754+0000","last_became_peered":"2024-09-16T15:40:11.433754+0000","last_unstale":"2024-09-16T15:40:13.434156+0000","last_undegraded":"2024-09-16T15:40:13.434156+0000","last_fullsized":"2024-09-16T15:40:13.434156+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:03:27.326561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.436365+0000","last_change":"2024-09-16T15:40:12.437260+0000","last_active":"2024-09-16T15:40:13.436365+0000","last_peered":"2024-09-16T15:40:13.436365+0000","last_clean":"2024-09-16T15:40:13.436365+0000","last_became_active":"2024-09-16T15:40:12.437001+0000","last_became_peered":"2024-09-16T15:40:12.437001+0000","last_unstale":"2024-09-16T15:40:13.436365+0000","last_undegraded":"2024-09-16T15:40:13.436365+0000","last_fullsized":"2024-09-16T15:40:13.436365+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:24:21.290065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"34'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.437092+0000","last_change":"2024-09-16T15:40:11.429377+0000","last_active":"2024-09-16T15:40:13.437092+0000","last_peered":"2024-09-16T15:40:13.437092+0000","last_clean":"2024-09-16T15:40:13.437092+0000","last_became_active":"2024-09-16T15:40:11.429116+0000","last_became_peered":"2024-09-16T15:40:11.429116+0000","last_unstale":"2024-09-16T15:40:13.437092+0000","last_undegraded":"2024-09-16T15:40:13.437092+0000","last_fullsized":"2024-09-16T15:40:13.437092+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:05:40.463616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.434894+0000","last_change":"2024-09-16T15:40:12.439353+0000","last_active":"2024-09-16T15:40:13.434894+0000","last_peered":"2024-09-16T15:40:13.434894+0000","last_clean":"2024-09-16T15:40:13.434894+0000","last_became_active":"2024-09-16T15:40:12.438075+0000","last_became_peered":"2024-09-16T15:40:12.438075+0000","last_unstale":"2024-09-16T15:40:13.434894+0000","last_undegraded":"2024-09-16T15:40:13.434894+0000","last_fullsized":"2024-09-16T15:40:13.434894+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:54:34.251717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"35'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.449901+0000","last_change":"2024-09-16T15:40:11.423158+0000","last_active":"2024-09-16T15:40:13.449901+0000","last_peered":"2024-09-16T15:40:13.449901+0000","last_clean":"2024-09-16T15:40:13.449901+0000","last_became_active":"2024-09-16T15:40:11.422892+0000","last_became_peered":"2024-09-16T15:40:11.422892+0000","last_unstale":"2024-09-16T15:40:13.449901+0000","last_undegraded":"2024-09-16T15:40:13.449901+0000","last_fullsized":"2024-09-16T15:40:13.449901+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:10:39.453392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.436527+0000","last_change":"2024-09-16T15:40:12.451657+0000","last_active":"2024-09-16T15:40:13.436527+0000","last_peered":"2024-09-16T15:40:13.436527+0000","last_clean":"2024-09-16T15:40:13.436527+0000","last_became_active":"2024-09-16T15:40:12.450795+0000","last_became_peered":"2024-09-16T15:40:12.450795+0000","last_unstale":"2024-09-16T15:40:13.436527+0000","last_undegraded":"2024-09-16T15:40:13.436527+0000","last_fullsized":"2024-09-16T15:40:13.436527+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:46:24.970506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.437395+0000","last_change":"2024-09-16T15:40:11.434071+0000","last_active":"2024-09-16T15:40:13.437395+0000","last_peered":"2024-09-16T15:40:13.437395+0000","last_clean":"2024-09-16T15:40:13.437395+0000","last_became_active":"2024-09-16T15:40:11.433566+0000","last_became_peered":"2024-09-16T15:40:11.433566+0000","last_unstale":"2024-09-16T15:40:13.437395+0000","last_undegraded":"2024-09-16T15:40:13.437395+0000","last_fullsized":"2024-09-16T15:40:13.437395+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:27:41.510684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.014650+0000","last_change":"2024-09-16T15:40:12.443182+0000","last_active":"2024-09-16T15:40:14.014650+0000","last_peered":"2024-09-16T15:40:14.014650+0000","last_clean":"2024-09-16T15:40:14.014650+0000","last_became_active":"2024-09-16T15:40:12.442931+0000","last_became_peered":"2024-09-16T15:40:12.442931+0000","last_unstale":"2024-09-16T15:40:14.014650+0000","last_undegraded":"2024-09-16T15:40:14.014650+0000","last_fullsized":"2024-09-16T15:40:14.014650+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:54:46.876716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.437007+0000","last_change":"2024-09-16T15:40:11.430371+0000","last_active":"2024-09-16T15:40:13.437007+0000","last_peered":"2024-09-16T15:40:13.437007+0000","last_clean":"2024-09-16T15:40:13.437007+0000","last_became_active":"2024-09-16T15:40:11.430082+0000","last_became_peered":"2024-09-16T15:40:11.430082+0000","last_unstale":"2024-09-16T15:40:13.437007+0000","last_undegraded":"2024-09-16T15:40:13.437007+0000","last_fullsized":"2024-09-16T15:40:13.437007+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:43:13.979677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.077802+0000","last_change":"2024-09-16T15:40:12.444868+0000","last_active":"2024-09-16T15:40:14.077802+0000","last_peered":"2024-09-16T15:40:14.077802+0000","last_clean":"2024-09-16T15:40:14.077802+0000","last_became_active":"2024-09-16T15:40:12.444573+0000","last_became_peered":"2024-09-16T15:40:12.444573+0000","last_unstale":"2024-09-16T15:40:14.077802+0000","last_undegraded":"2024-09-16T15:40:14.077802+0000","last_fullsized":"2024-09-16T15:40:14.077802+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:13:54.801126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.436446+0000","last_change":"2024-09-16T15:40:11.437894+0000","last_active":"2024-09-16T15:40:13.436446+0000","last_peered":"2024-09-16T15:40:13.436446+0000","last_clean":"2024-09-16T15:40:13.436446+0000","last_became_active":"2024-09-16T15:40:11.437657+0000","last_became_peered":"2024-09-16T15:40:11.437657+0000","last_unstale":"2024-09-16T15:40:13.436446+0000","last_undegraded":"2024-09-16T15:40:13.436446+0000","last_fullsized":"2024-09-16T15:40:13.436446+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:43:00.710200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.077598+0000","last_change":"2024-09-16T15:40:12.441083+0000","last_active":"2024-09-16T15:40:14.077598+0000","last_peered":"2024-09-16T15:40:14.077598+0000","last_clean":"2024-09-16T15:40:14.077598+0000","last_became_active":"2024-09-16T15:40:12.440787+0000","last_became_peered":"2024-09-16T15:40:12.440787+0000","last_unstale":"2024-09-16T15:40:14.077598+0000","last_undegraded":"2024-09-16T15:40:14.077598+0000","last_fullsized":"2024-09-16T15:40:14.077598+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:12:52.714116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.076955+0000","last_change":"2024-09-16T15:40:11.436770+0000","last_active":"2024-09-16T15:40:14.076955+0000","last_peered":"2024-09-16T15:40:14.076955+0000","last_clean":"2024-09-16T15:40:14.076955+0000","last_became_active":"2024-09-16T15:40:11.435878+0000","last_became_peered":"2024-09-16T15:40:11.435878+0000","last_unstale":"2024-09-16T15:40:14.076955+0000","last_undegraded":"2024-09-16T15:40:14.076955+0000","last_fullsized":"2024-09-16T15:40:14.076955+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:05:40.463616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.434410+0000","last_change":"2024-09-16T15:40:12.437726+0000","last_active":"2024-09-16T15:40:13.434410+0000","last_peered":"2024-09-16T15:40:13.434410+0000","last_clean":"2024-09-16T15:40:13.434410+0000","last_became_active":"2024-09-16T15:40:12.436801+0000","last_became_peered":"2024-09-16T15:40:12.436801+0000","last_unstale":"2024-09-16T15:40:13.434410+0000","last_undegraded":"2024-09-16T15:40:13.434410+0000","last_fullsized":"2024-09-16T15:40:13.434410+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:10:10.169816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.077439+0000","last_change":"2024-09-16T15:40:11.428945+0000","last_active":"2024-09-16T15:40:14.077439+0000","last_peered":"2024-09-16T15:40:14.077439+0000","last_clean":"2024-09-16T15:40:14.077439+0000","last_became_active":"2024-09-16T15:40:11.428296+0000","last_became_peered":"2024-09-16T15:40:11.428296+0000","last_unstale":"2024-09-16T15:40:14.077439+0000","last_undegraded":"2024-09-16T15:40:14.077439+0000","last_fullsized":"2024-09-16T15:40:14.077439+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:32:21.531717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.014863+0000","last_change":"2024-09-16T15:40:12.437406+0000","last_active":"2024-09-16T15:40:14.014863+0000","last_peered":"2024-09-16T15:40:14.014863+0000","last_clean":"2024-09-16T15:40:14.014863+0000","last_became_active":"2024-09-16T15:40:12.437117+0000","last_became_peered":"2024-09-16T15:40:12.437117+0000","last_unstale":"2024-09-16T15:40:14.014863+0000","last_undegraded":"2024-09-16T15:40:14.014863+0000","last_fullsized":"2024-09-16T15:40:14.014863+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:57:43.248746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.434720+0000","last_change":"2024-09-16T15:40:11.428439+0000","last_active":"2024-09-16T15:40:13.434720+0000","last_peered":"2024-09-16T15:40:13.434720+0000","last_clean":"2024-09-16T15:40:13.434720+0000","last_became_active":"2024-09-16T15:40:11.427829+0000","last_became_peered":"2024-09-16T15:40:11.427829+0000","last_unstale":"2024-09-16T15:40:13.434720+0000","last_undegraded":"2024-09-16T15:40:13.434720+0000","last_fullsized":"2024-09-16T15:40:13.434720+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:54:45.872053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":135,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.076426+0000","last_change":"2024-09-16T15:39:12.698813+0000","last_active":"2024-09-16T15:40:14.076426+0000","last_peered":"2024-09-16T15:40:14.076426+0000","last_clean":"2024-09-16T15:40:14.076426+0000","last_became_active":"2024-09-16T15:39:12.697695+0000","last_became_peered":"2024-09-16T15:39:12.697695+0000","last_unstale":"2024-09-16T15:40:14.076426+0000","last_undegraded":"2024-09-16T15:40:14.076426+0000","last_fullsized":"2024-09-16T15:40:14.076426+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_clean_scrub_stamp":"2024-09-16T15:39:11.684967+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:02.061081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509458,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":401408,"data_stored":153170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509457,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27996,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343844,"statfs":{"total":96636764160,"available":96608096256,"internally_reserved":0,"allocated":987136,"data_stored":742508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509456,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":405504,"data_stored":153192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509456,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":405504,"data_stored":152334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509456,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":393216,"data_stored":151592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509457,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28000,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":987136,"data_stored":741996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509455,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":393216,"data_stored":151648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542162,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":393216,"data_stored":151580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":27654604},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:40:21.349 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:40:21.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.517+0000 7f3bd6200640 1 -- 172.21.15.151:0/2423724957 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3bd00b4020 msgr2=0x7f3bd00b1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:21.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.517+0000 7f3bd6200640 1 --2- 172.21.15.151:0/2423724957 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3bd00b4020 0x7f3bd00b1770 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3bd00d4ea0 tx=0x7f3bc802a9f0 comp rx=0 tx=0).stop 2024-09-16T15:40:21.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.517+0000 7f3bd6200640 1 -- 172.21.15.151:0/2423724957 shutdown_connections 2024-09-16T15:40:21.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.517+0000 7f3bd6200640 1 --2- 172.21.15.151:0/2423724957 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3bd00b4020 0x7f3bd00b1770 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:21.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.517+0000 7f3bd6200640 1 --2- 172.21.15.151:0/2423724957 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3bd0161440 0x7f3bd00b5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:21.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.517+0000 7f3bd6200640 1 -- 172.21.15.151:0/2423724957 >> 172.21.15.151:0/2423724957 conn(0x7f3bd00d3df0 msgr2=0x7f3bd00d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:21.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.517+0000 7f3bd6200640 1 -- 172.21.15.151:0/2423724957 shutdown_connections 2024-09-16T15:40:21.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.517+0000 7f3bd6200640 1 -- 172.21.15.151:0/2423724957 wait complete. 2024-09-16T15:40:21.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.518+0000 7f3bd6200640 1 Processor -- start 2024-09-16T15:40:21.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.519+0000 7f3bd6200640 1 -- start start 2024-09-16T15:40:21.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.520+0000 7f3bd6200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bd00b34b0 0x7f3bd00c8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:21.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.520+0000 7f3bd6200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3bd00b4020 0x7f3bd01f7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:21.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.520+0000 7f3bd6200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3bd0161440 0x7f3bd00ca010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:21.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.520+0000 7f3bd6200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3bd00b2040 con 0x7f3bd00b4020 2024-09-16T15:40:21.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.520+0000 7f3bd6200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3bd00b1ec0 con 0x7f3bd00b34b0 2024-09-16T15:40:21.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.520+0000 7f3bd6200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f3bd00b21c0 con 0x7f3bd0161440 2024-09-16T15:40:21.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.520+0000 7f3bcfe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3bd00b4020 0x7f3bd01f7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:21.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.520+0000 7f3bcfe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3bd00b4020 0x7f3bd01f7430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37358/0 (socket says 172.21.15.151:37358) 2024-09-16T15:40:21.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.520+0000 7f3bcfe00640 1 -- 172.21.15.151:0/1529111913 learned_addr learned my addr 172.21.15.151:0/1529111913 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:40:21.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.520+0000 7f3bd5800640 1 --2- 172.21.15.151:0/1529111913 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3bd0161440 0x7f3bd00ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:21.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.520+0000 7f3bd4e00640 1 --2- 172.21.15.151:0/1529111913 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bd00b34b0 0x7f3bd00c8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:21.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.520+0000 7f3bcfe00640 1 -- 172.21.15.151:0/1529111913 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3bd0161440 msgr2=0x7f3bd00ca010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:21.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.520+0000 7f3bcfe00640 1 --2- 172.21.15.151:0/1529111913 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3bd0161440 0x7f3bd00ca010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:21.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.520+0000 7f3bcfe00640 1 -- 172.21.15.151:0/1529111913 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bd00b34b0 msgr2=0x7f3bd00c8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:21.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.520+0000 7f3bcfe00640 1 --2- 172.21.15.151:0/1529111913 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bd00b34b0 0x7f3bd00c8700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:21.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.520+0000 7f3bcfe00640 1 -- 172.21.15.151:0/1529111913 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bd00ca840 con 0x7f3bd00b4020 2024-09-16T15:40:21.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.521+0000 7f3bcfe00640 1 --2- 172.21.15.151:0/1529111913 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3bd00b4020 0x7f3bd01f7430 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc00c4d0 tx=0x7f3bbc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:21.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.521+0000 7f3bccc00640 1 -- 172.21.15.151:0/1529111913 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc0052d0 con 0x7f3bd00b4020 2024-09-16T15:40:21.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.521+0000 7f3bccc00640 1 -- 172.21.15.151:0/1529111913 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3bbc005470 con 0x7f3bd00b4020 2024-09-16T15:40:21.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.521+0000 7f3bccc00640 1 -- 172.21.15.151:0/1529111913 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc012870 con 0x7f3bd00b4020 2024-09-16T15:40:21.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.521+0000 7f3bd6200640 1 -- 172.21.15.151:0/1529111913 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bd01f47b0 con 0x7f3bd00b4020 2024-09-16T15:40:21.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.521+0000 7f3bd6200640 1 -- 172.21.15.151:0/1529111913 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bd00ca550 con 0x7f3bd00b4020 2024-09-16T15:40:21.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.522+0000 7f3bd6200640 1 -- 172.21.15.151:0/1529111913 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bd00b1770 con 0x7f3bd00b4020 2024-09-16T15:40:21.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.523+0000 7f3bccc00640 1 -- 172.21.15.151:0/1529111913 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f3bbc0035f0 con 0x7f3bd00b4020 2024-09-16T15:40:21.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.524+0000 7f3bccc00640 1 --2- 172.21.15.151:0/1529111913 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3bb0078880 0x7f3bb007ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:21.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.524+0000 7f3bccc00640 1 -- 172.21.15.151:0/1529111913 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f3bbc09a470 con 0x7f3bd00b4020 2024-09-16T15:40:21.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.524+0000 7f3bd4e00640 1 --2- 172.21.15.151:0/1529111913 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3bb0078880 0x7f3bb007ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:21.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.525+0000 7f3bd4e00640 1 --2- 172.21.15.151:0/1529111913 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3bb0078880 0x7f3bb007ad40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3bc400b810 tx=0x7f3bc4008040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:21.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.528+0000 7f3bccc00640 1 -- 172.21.15.151:0/1529111913 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f3bbc062ad0 con 0x7f3bd00b4020 2024-09-16T15:40:21.711 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.709+0000 7f3bd6200640 1 -- 172.21.15.151:0/1529111913 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3bd00760b0 con 0x7f3bb0078880 2024-09-16T15:40:21.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.715+0000 7f3bccc00640 1 -- 172.21.15.151:0/1529111913 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251832 (secure 0 0 0) 0x7f3bd00760b0 con 0x7f3bb0078880 2024-09-16T15:40:21.719 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:40:21.720 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-09-16T15:40:21.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.718+0000 7f3bd6200640 1 -- 172.21.15.151:0/1529111913 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3bb0078880 msgr2=0x7f3bb007ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:21.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.718+0000 7f3bd6200640 1 --2- 172.21.15.151:0/1529111913 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3bb0078880 0x7f3bb007ad40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3bc400b810 tx=0x7f3bc4008040 comp rx=0 tx=0).stop 2024-09-16T15:40:21.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.718+0000 7f3bd6200640 1 -- 172.21.15.151:0/1529111913 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3bd00b4020 msgr2=0x7f3bd01f7430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:21.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.718+0000 7f3bd6200640 1 --2- 172.21.15.151:0/1529111913 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3bd00b4020 0x7f3bd01f7430 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc00c4d0 tx=0x7f3bbc00c990 comp rx=0 tx=0).stop 2024-09-16T15:40:21.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.718+0000 7f3bd6200640 1 -- 172.21.15.151:0/1529111913 shutdown_connections 2024-09-16T15:40:21.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.718+0000 7f3bd6200640 1 --2- 172.21.15.151:0/1529111913 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3bb0078880 0x7f3bb007ad40 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:21.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.718+0000 7f3bd6200640 1 --2- 172.21.15.151:0/1529111913 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3bd0161440 0x7f3bd00ca010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:21.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.718+0000 7f3bd6200640 1 --2- 172.21.15.151:0/1529111913 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3bd00b4020 0x7f3bd01f7430 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:21.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.718+0000 7f3bd6200640 1 --2- 172.21.15.151:0/1529111913 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bd00b34b0 0x7f3bd00c8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:21.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.718+0000 7f3bd6200640 1 -- 172.21.15.151:0/1529111913 >> 172.21.15.151:0/1529111913 conn(0x7f3bd00d3df0 msgr2=0x7f3bd00b3910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:21.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.718+0000 7f3bd6200640 1 -- 172.21.15.151:0/1529111913 shutdown_connections 2024-09-16T15:40:21.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.718+0000 7f3bd6200640 1 -- 172.21.15.151:0/1529111913 wait complete. 2024-09-16T15:40:21.736 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2024-09-16T15:40:21.590074+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220524,"kb_used_data":4264,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754196,"statfs":{"total":773094113280,"available":772868296704,"internally_reserved":0,"allocated":4366336,"data_stored":2398020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12738,"internal_metadata":221236798},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.171138"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.077510+0000","last_change":"2024-09-16T15:40:12.444116+0000","last_active":"2024-09-16T15:40:14.077510+0000","last_peered":"2024-09-16T15:40:14.077510+0000","last_clean":"2024-09-16T15:40:14.077510+0000","last_became_active":"2024-09-16T15:40:12.440025+0000","last_became_peered":"2024-09-16T15:40:12.440025+0000","last_unstale":"2024-09-16T15:40:14.077510+0000","last_undegraded":"2024-09-16T15:40:14.077510+0000","last_fullsized":"2024-09-16T15:40:14.077510+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:37:22.915098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.449833+0000","last_change":"2024-09-16T15:40:12.444480+0000","last_active":"2024-09-16T15:40:13.449833+0000","last_peered":"2024-09-16T15:40:13.449833+0000","last_clean":"2024-09-16T15:40:13.449833+0000","last_became_active":"2024-09-16T15:40:12.444220+0000","last_became_peered":"2024-09-16T15:40:12.444220+0000","last_unstale":"2024-09-16T15:40:13.449833+0000","last_undegraded":"2024-09-16T15:40:13.449833+0000","last_fullsized":"2024-09-16T15:40:13.449833+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:12:52.714116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.076722+0000","last_change":"2024-09-16T15:40:12.440385+0000","last_active":"2024-09-16T15:40:14.076722+0000","last_peered":"2024-09-16T15:40:14.076722+0000","last_clean":"2024-09-16T15:40:14.076722+0000","last_became_active":"2024-09-16T15:40:12.440138+0000","last_became_peered":"2024-09-16T15:40:12.440138+0000","last_unstale":"2024-09-16T15:40:14.076722+0000","last_undegraded":"2024-09-16T15:40:14.076722+0000","last_fullsized":"2024-09-16T15:40:14.076722+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:43:01.714863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.077877+0000","last_change":"2024-09-16T15:40:12.450256+0000","last_active":"2024-09-16T15:40:14.077877+0000","last_peered":"2024-09-16T15:40:14.077877+0000","last_clean":"2024-09-16T15:40:14.077877+0000","last_became_active":"2024-09-16T15:40:12.450019+0000","last_became_peered":"2024-09-16T15:40:12.450019+0000","last_unstale":"2024-09-16T15:40:14.077877+0000","last_undegraded":"2024-09-16T15:40:14.077877+0000","last_fullsized":"2024-09-16T15:40:14.077877+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:44:19.883823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.449738+0000","last_change":"2024-09-16T15:40:12.448384+0000","last_active":"2024-09-16T15:40:13.449738+0000","last_peered":"2024-09-16T15:40:13.449738+0000","last_clean":"2024-09-16T15:40:13.449738+0000","last_became_active":"2024-09-16T15:40:12.448088+0000","last_became_peered":"2024-09-16T15:40:12.448088+0000","last_unstale":"2024-09-16T15:40:13.449738+0000","last_undegraded":"2024-09-16T15:40:13.449738+0000","last_fullsized":"2024-09-16T15:40:13.449738+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:03:28.331224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.435856+0000","last_change":"2024-09-16T15:40:12.444964+0000","last_active":"2024-09-16T15:40:13.435856+0000","last_peered":"2024-09-16T15:40:13.435856+0000","last_clean":"2024-09-16T15:40:13.435856+0000","last_became_active":"2024-09-16T15:40:12.442884+0000","last_became_peered":"2024-09-16T15:40:12.442884+0000","last_unstale":"2024-09-16T15:40:13.435856+0000","last_undegraded":"2024-09-16T15:40:13.435856+0000","last_fullsized":"2024-09-16T15:40:13.435856+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:06:23.829480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.076833+0000","last_change":"2024-09-16T15:40:12.437515+0000","last_active":"2024-09-16T15:40:14.076833+0000","last_peered":"2024-09-16T15:40:14.076833+0000","last_clean":"2024-09-16T15:40:14.076833+0000","last_became_active":"2024-09-16T15:40:12.436738+0000","last_became_peered":"2024-09-16T15:40:12.436738+0000","last_unstale":"2024-09-16T15:40:14.076833+0000","last_undegraded":"2024-09-16T15:40:14.076833+0000","last_fullsized":"2024-09-16T15:40:14.076833+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:15:43.023428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.077093+0000","last_change":"2024-09-16T15:40:12.444472+0000","last_active":"2024-09-16T15:40:14.077093+0000","last_peered":"2024-09-16T15:40:14.077093+0000","last_clean":"2024-09-16T15:40:14.077093+0000","last_became_active":"2024-09-16T15:40:12.440102+0000","last_became_peered":"2024-09-16T15:40:12.440102+0000","last_unstale":"2024-09-16T15:40:14.077093+0000","last_undegraded":"2024-09-16T15:40:14.077093+0000","last_fullsized":"2024-09-16T15:40:14.077093+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:12:52.714116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.076787+0000","last_change":"2024-09-16T15:40:12.453854+0000","last_active":"2024-09-16T15:40:14.076787+0000","last_peered":"2024-09-16T15:40:14.076787+0000","last_clean":"2024-09-16T15:40:14.076787+0000","last_became_active":"2024-09-16T15:40:12.453625+0000","last_became_peered":"2024-09-16T15:40:12.453625+0000","last_unstale":"2024-09-16T15:40:14.076787+0000","last_undegraded":"2024-09-16T15:40:14.076787+0000","last_fullsized":"2024-09-16T15:40:14.076787+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:05:41.468279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.078173+0000","last_change":"2024-09-16T15:40:12.440112+0000","last_active":"2024-09-16T15:40:14.078173+0000","last_peered":"2024-09-16T15:40:14.078173+0000","last_clean":"2024-09-16T15:40:14.078173+0000","last_became_active":"2024-09-16T15:40:12.439159+0000","last_became_peered":"2024-09-16T15:40:12.439159+0000","last_unstale":"2024-09-16T15:40:14.078173+0000","last_undegraded":"2024-09-16T15:40:14.078173+0000","last_fullsized":"2024-09-16T15:40:14.078173+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:13:20.028628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.434696+0000","last_change":"2024-09-16T15:40:12.451264+0000","last_active":"2024-09-16T15:40:13.434696+0000","last_peered":"2024-09-16T15:40:13.434696+0000","last_clean":"2024-09-16T15:40:13.434696+0000","last_became_active":"2024-09-16T15:40:12.451000+0000","last_became_peered":"2024-09-16T15:40:12.451000+0000","last_unstale":"2024-09-16T15:40:13.434696+0000","last_undegraded":"2024-09-16T15:40:13.434696+0000","last_fullsized":"2024-09-16T15:40:13.434696+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:54:02.892922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.076784+0000","last_change":"2024-09-16T15:40:12.446787+0000","last_active":"2024-09-16T15:40:14.076784+0000","last_peered":"2024-09-16T15:40:14.076784+0000","last_clean":"2024-09-16T15:40:14.076784+0000","last_became_active":"2024-09-16T15:40:12.446456+0000","last_became_peered":"2024-09-16T15:40:12.446456+0000","last_unstale":"2024-09-16T15:40:14.076784+0000","last_undegraded":"2024-09-16T15:40:14.076784+0000","last_fullsized":"2024-09-16T15:40:14.076784+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:16:27.816839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.437025+0000","last_change":"2024-09-16T15:40:12.453651+0000","last_active":"2024-09-16T15:40:13.437025+0000","last_peered":"2024-09-16T15:40:13.437025+0000","last_clean":"2024-09-16T15:40:13.437025+0000","last_became_active":"2024-09-16T15:40:12.453385+0000","last_became_peered":"2024-09-16T15:40:12.453385+0000","last_unstale":"2024-09-16T15:40:13.437025+0000","last_undegraded":"2024-09-16T15:40:13.437025+0000","last_fullsized":"2024-09-16T15:40:13.437025+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:21:20.804588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.077490+0000","last_change":"2024-09-16T15:40:12.442875+0000","last_active":"2024-09-16T15:40:14.077490+0000","last_peered":"2024-09-16T15:40:14.077490+0000","last_clean":"2024-09-16T15:40:14.077490+0000","last_became_active":"2024-09-16T15:40:12.442552+0000","last_became_peered":"2024-09-16T15:40:12.442552+0000","last_unstale":"2024-09-16T15:40:14.077490+0000","last_undegraded":"2024-09-16T15:40:14.077490+0000","last_fullsized":"2024-09-16T15:40:14.077490+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:29:40.745908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.014602+0000","last_change":"2024-09-16T15:40:12.429955+0000","last_active":"2024-09-16T15:40:14.014602+0000","last_peered":"2024-09-16T15:40:14.014602+0000","last_clean":"2024-09-16T15:40:14.014602+0000","last_became_active":"2024-09-16T15:40:12.428839+0000","last_became_peered":"2024-09-16T15:40:12.428839+0000","last_unstale":"2024-09-16T15:40:14.014602+0000","last_undegraded":"2024-09-16T15:40:14.014602+0000","last_fullsized":"2024-09-16T15:40:14.014602+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:03:14.135051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.014596+0000","last_change":"2024-09-16T15:40:12.439315+0000","last_active":"2024-09-16T15:40:14.014596+0000","last_peered":"2024-09-16T15:40:14.014596+0000","last_clean":"2024-09-16T15:40:14.014596+0000","last_became_active":"2024-09-16T15:40:12.438629+0000","last_became_peered":"2024-09-16T15:40:12.438629+0000","last_unstale":"2024-09-16T15:40:14.014596+0000","last_undegraded":"2024-09-16T15:40:14.014596+0000","last_fullsized":"2024-09-16T15:40:14.014596+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:33:58.217901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.434759+0000","last_change":"2024-09-16T15:40:12.445302+0000","last_active":"2024-09-16T15:40:13.434759+0000","last_peered":"2024-09-16T15:40:13.434759+0000","last_clean":"2024-09-16T15:40:13.434759+0000","last_became_active":"2024-09-16T15:40:12.445014+0000","last_became_peered":"2024-09-16T15:40:12.445014+0000","last_unstale":"2024-09-16T15:40:13.434759+0000","last_undegraded":"2024-09-16T15:40:13.434759+0000","last_fullsized":"2024-09-16T15:40:13.434759+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:41:30.413571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.436268+0000","last_change":"2024-09-16T15:40:12.446472+0000","last_active":"2024-09-16T15:40:13.436268+0000","last_peered":"2024-09-16T15:40:13.436268+0000","last_clean":"2024-09-16T15:40:13.436268+0000","last_became_active":"2024-09-16T15:40:12.446204+0000","last_became_peered":"2024-09-16T15:40:12.446204+0000","last_unstale":"2024-09-16T15:40:13.436268+0000","last_undegraded":"2024-09-16T15:40:13.436268+0000","last_fullsized":"2024-09-16T15:40:13.436268+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:04:11.653249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.436116+0000","last_change":"2024-09-16T15:40:12.446725+0000","last_active":"2024-09-16T15:40:13.436116+0000","last_peered":"2024-09-16T15:40:13.436116+0000","last_clean":"2024-09-16T15:40:13.436116+0000","last_became_active":"2024-09-16T15:40:12.441589+0000","last_became_peered":"2024-09-16T15:40:12.441589+0000","last_unstale":"2024-09-16T15:40:13.436116+0000","last_undegraded":"2024-09-16T15:40:13.436116+0000","last_fullsized":"2024-09-16T15:40:13.436116+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:50:22.432002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.437263+0000","last_change":"2024-09-16T15:40:12.441110+0000","last_active":"2024-09-16T15:40:13.437263+0000","last_peered":"2024-09-16T15:40:13.437263+0000","last_clean":"2024-09-16T15:40:13.437263+0000","last_became_active":"2024-09-16T15:40:12.439699+0000","last_became_peered":"2024-09-16T15:40:12.439699+0000","last_unstale":"2024-09-16T15:40:13.437263+0000","last_undegraded":"2024-09-16T15:40:13.437263+0000","last_fullsized":"2024-09-16T15:40:13.437263+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:33:13.502942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.014799+0000","last_change":"2024-09-16T15:40:12.429020+0000","last_active":"2024-09-16T15:40:14.014799+0000","last_peered":"2024-09-16T15:40:14.014799+0000","last_clean":"2024-09-16T15:40:14.014799+0000","last_became_active":"2024-09-16T15:40:12.428774+0000","last_became_peered":"2024-09-16T15:40:12.428774+0000","last_unstale":"2024-09-16T15:40:14.014799+0000","last_undegraded":"2024-09-16T15:40:14.014799+0000","last_fullsized":"2024-09-16T15:40:14.014799+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:37:22.915098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.077059+0000","last_change":"2024-09-16T15:40:12.440419+0000","last_active":"2024-09-16T15:40:14.077059+0000","last_peered":"2024-09-16T15:40:14.077059+0000","last_clean":"2024-09-16T15:40:14.077059+0000","last_became_active":"2024-09-16T15:40:12.439833+0000","last_became_peered":"2024-09-16T15:40:12.439833+0000","last_unstale":"2024-09-16T15:40:14.077059+0000","last_undegraded":"2024-09-16T15:40:14.077059+0000","last_fullsized":"2024-09-16T15:40:14.077059+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:29:40.745908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.435353+0000","last_change":"2024-09-16T15:40:12.440517+0000","last_active":"2024-09-16T15:40:13.435353+0000","last_peered":"2024-09-16T15:40:13.435353+0000","last_clean":"2024-09-16T15:40:13.435353+0000","last_became_active":"2024-09-16T15:40:12.439873+0000","last_became_peered":"2024-09-16T15:40:12.439873+0000","last_unstale":"2024-09-16T15:40:13.435353+0000","last_undegraded":"2024-09-16T15:40:13.435353+0000","last_fullsized":"2024-09-16T15:40:13.435353+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:49:56.803512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.076598+0000","last_change":"2024-09-16T15:40:12.451869+0000","last_active":"2024-09-16T15:40:14.076598+0000","last_peered":"2024-09-16T15:40:14.076598+0000","last_clean":"2024-09-16T15:40:14.076598+0000","last_became_active":"2024-09-16T15:40:12.451593+0000","last_became_peered":"2024-09-16T15:40:12.451593+0000","last_unstale":"2024-09-16T15:40:14.076598+0000","last_undegraded":"2024-09-16T15:40:14.076598+0000","last_fullsized":"2024-09-16T15:40:14.076598+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:55:42.068271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.078341+0000","last_change":"2024-09-16T15:40:12.441706+0000","last_active":"2024-09-16T15:40:14.078341+0000","last_peered":"2024-09-16T15:40:14.078341+0000","last_clean":"2024-09-16T15:40:14.078341+0000","last_became_active":"2024-09-16T15:40:12.440405+0000","last_became_peered":"2024-09-16T15:40:12.440405+0000","last_unstale":"2024-09-16T15:40:14.078341+0000","last_undegraded":"2024-09-16T15:40:14.078341+0000","last_fullsized":"2024-09-16T15:40:14.078341+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:28:21.595104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.078227+0000","last_change":"2024-09-16T15:40:12.439748+0000","last_active":"2024-09-16T15:40:14.078227+0000","last_peered":"2024-09-16T15:40:14.078227+0000","last_clean":"2024-09-16T15:40:14.078227+0000","last_became_active":"2024-09-16T15:40:12.439238+0000","last_became_peered":"2024-09-16T15:40:12.439238+0000","last_unstale":"2024-09-16T15:40:14.078227+0000","last_undegraded":"2024-09-16T15:40:14.078227+0000","last_fullsized":"2024-09-16T15:40:14.078227+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:16:49.213767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.449976+0000","last_change":"2024-09-16T15:40:12.435380+0000","last_active":"2024-09-16T15:40:13.449976+0000","last_peered":"2024-09-16T15:40:13.449976+0000","last_clean":"2024-09-16T15:40:13.449976+0000","last_became_active":"2024-09-16T15:40:12.434744+0000","last_became_peered":"2024-09-16T15:40:12.434744+0000","last_unstale":"2024-09-16T15:40:13.449976+0000","last_undegraded":"2024-09-16T15:40:13.449976+0000","last_fullsized":"2024-09-16T15:40:13.449976+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:37:22.915098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.077539+0000","last_change":"2024-09-16T15:40:12.441070+0000","last_active":"2024-09-16T15:40:14.077539+0000","last_peered":"2024-09-16T15:40:14.077539+0000","last_clean":"2024-09-16T15:40:14.077539+0000","last_became_active":"2024-09-16T15:40:12.440809+0000","last_became_peered":"2024-09-16T15:40:12.440809+0000","last_unstale":"2024-09-16T15:40:14.077539+0000","last_undegraded":"2024-09-16T15:40:14.077539+0000","last_fullsized":"2024-09-16T15:40:14.077539+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:57:43.248746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.014592+0000","last_change":"2024-09-16T15:40:12.453157+0000","last_active":"2024-09-16T15:40:14.014592+0000","last_peered":"2024-09-16T15:40:14.014592+0000","last_clean":"2024-09-16T15:40:14.014592+0000","last_became_active":"2024-09-16T15:40:12.452815+0000","last_became_peered":"2024-09-16T15:40:12.452815+0000","last_unstale":"2024-09-16T15:40:14.014592+0000","last_undegraded":"2024-09-16T15:40:14.014592+0000","last_fullsized":"2024-09-16T15:40:14.014592+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:10:35.804720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.014581+0000","last_change":"2024-09-16T15:40:12.433443+0000","last_active":"2024-09-16T15:40:14.014581+0000","last_peered":"2024-09-16T15:40:14.014581+0000","last_clean":"2024-09-16T15:40:14.014581+0000","last_became_active":"2024-09-16T15:40:12.433169+0000","last_became_peered":"2024-09-16T15:40:12.433169+0000","last_unstale":"2024-09-16T15:40:14.014581+0000","last_undegraded":"2024-09-16T15:40:14.014581+0000","last_fullsized":"2024-09-16T15:40:14.014581+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:03:28.331224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.077033+0000","last_change":"2024-09-16T15:40:12.437804+0000","last_active":"2024-09-16T15:40:14.077033+0000","last_peered":"2024-09-16T15:40:14.077033+0000","last_clean":"2024-09-16T15:40:14.077033+0000","last_became_active":"2024-09-16T15:40:12.437269+0000","last_became_peered":"2024-09-16T15:40:12.437269+0000","last_unstale":"2024-09-16T15:40:14.077033+0000","last_undegraded":"2024-09-16T15:40:14.077033+0000","last_fullsized":"2024-09-16T15:40:14.077033+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:10:35.804720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.434631+0000","last_change":"2024-09-16T15:40:12.445107+0000","last_active":"2024-09-16T15:40:13.434631+0000","last_peered":"2024-09-16T15:40:13.434631+0000","last_clean":"2024-09-16T15:40:13.434631+0000","last_became_active":"2024-09-16T15:40:12.444866+0000","last_became_peered":"2024-09-16T15:40:12.444866+0000","last_unstale":"2024-09-16T15:40:13.434631+0000","last_undegraded":"2024-09-16T15:40:13.434631+0000","last_fullsized":"2024-09-16T15:40:13.434631+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:13:58.747871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.437465+0000","last_change":"2024-09-16T15:40:12.436931+0000","last_active":"2024-09-16T15:40:13.437465+0000","last_peered":"2024-09-16T15:40:13.437465+0000","last_clean":"2024-09-16T15:40:13.437465+0000","last_became_active":"2024-09-16T15:40:12.436211+0000","last_became_peered":"2024-09-16T15:40:12.436211+0000","last_unstale":"2024-09-16T15:40:13.437465+0000","last_undegraded":"2024-09-16T15:40:13.437465+0000","last_fullsized":"2024-09-16T15:40:13.437465+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:21:12.132805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"34'4","reported_seq":19,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.434826+0000","last_change":"2024-09-16T15:40:11.426141+0000","last_active":"2024-09-16T15:40:13.434826+0000","last_peered":"2024-09-16T15:40:13.434826+0000","last_clean":"2024-09-16T15:40:13.434826+0000","last_became_active":"2024-09-16T15:40:11.425862+0000","last_became_peered":"2024-09-16T15:40:11.425862+0000","last_unstale":"2024-09-16T15:40:13.434826+0000","last_undegraded":"2024-09-16T15:40:13.434826+0000","last_fullsized":"2024-09-16T15:40:13.434826+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:37:21.910435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.077683+0000","last_change":"2024-09-16T15:40:12.438772+0000","last_active":"2024-09-16T15:40:14.077683+0000","last_peered":"2024-09-16T15:40:14.077683+0000","last_clean":"2024-09-16T15:40:14.077683+0000","last_became_active":"2024-09-16T15:40:12.436489+0000","last_became_peered":"2024-09-16T15:40:12.436489+0000","last_unstale":"2024-09-16T15:40:14.077683+0000","last_undegraded":"2024-09-16T15:40:14.077683+0000","last_fullsized":"2024-09-16T15:40:14.077683+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:15:46.661424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"34'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.078242+0000","last_change":"2024-09-16T15:40:11.429198+0000","last_active":"2024-09-16T15:40:14.078242+0000","last_peered":"2024-09-16T15:40:14.078242+0000","last_clean":"2024-09-16T15:40:14.078242+0000","last_became_active":"2024-09-16T15:40:11.428894+0000","last_became_peered":"2024-09-16T15:40:11.428894+0000","last_unstale":"2024-09-16T15:40:14.078242+0000","last_undegraded":"2024-09-16T15:40:14.078242+0000","last_fullsized":"2024-09-16T15:40:14.078242+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:46:10.721679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.077561+0000","last_change":"2024-09-16T15:40:12.452116+0000","last_active":"2024-09-16T15:40:14.077561+0000","last_peered":"2024-09-16T15:40:14.077561+0000","last_clean":"2024-09-16T15:40:14.077561+0000","last_became_active":"2024-09-16T15:40:12.451065+0000","last_became_peered":"2024-09-16T15:40:12.451065+0000","last_unstale":"2024-09-16T15:40:14.077561+0000","last_undegraded":"2024-09-16T15:40:14.077561+0000","last_fullsized":"2024-09-16T15:40:14.077561+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:00:15.773418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"34'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.436310+0000","last_change":"2024-09-16T15:40:11.429304+0000","last_active":"2024-09-16T15:40:13.436310+0000","last_peered":"2024-09-16T15:40:13.436310+0000","last_clean":"2024-09-16T15:40:13.436310+0000","last_became_active":"2024-09-16T15:40:11.429029+0000","last_became_peered":"2024-09-16T15:40:11.429029+0000","last_unstale":"2024-09-16T15:40:13.436310+0000","last_undegraded":"2024-09-16T15:40:13.436310+0000","last_fullsized":"2024-09-16T15:40:13.436310+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:31:10.242510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.078117+0000","last_change":"2024-09-16T15:40:12.448098+0000","last_active":"2024-09-16T15:40:14.078117+0000","last_peered":"2024-09-16T15:40:14.078117+0000","last_clean":"2024-09-16T15:40:14.078117+0000","last_became_active":"2024-09-16T15:40:12.447795+0000","last_became_peered":"2024-09-16T15:40:12.447795+0000","last_unstale":"2024-09-16T15:40:14.078117+0000","last_undegraded":"2024-09-16T15:40:14.078117+0000","last_fullsized":"2024-09-16T15:40:14.078117+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:08:40.384527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"34'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.077947+0000","last_change":"2024-09-16T15:40:11.432946+0000","last_active":"2024-09-16T15:40:14.077947+0000","last_peered":"2024-09-16T15:40:14.077947+0000","last_clean":"2024-09-16T15:40:14.077947+0000","last_became_active":"2024-09-16T15:40:11.431415+0000","last_became_peered":"2024-09-16T15:40:11.431415+0000","last_unstale":"2024-09-16T15:40:14.077947+0000","last_undegraded":"2024-09-16T15:40:14.077947+0000","last_fullsized":"2024-09-16T15:40:14.077947+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:59:15.340271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.076564+0000","last_change":"2024-09-16T15:40:12.452431+0000","last_active":"2024-09-16T15:40:14.076564+0000","last_peered":"2024-09-16T15:40:14.076564+0000","last_clean":"2024-09-16T15:40:14.076564+0000","last_became_active":"2024-09-16T15:40:12.452156+0000","last_became_peered":"2024-09-16T15:40:12.452156+0000","last_unstale":"2024-09-16T15:40:14.076564+0000","last_undegraded":"2024-09-16T15:40:14.076564+0000","last_fullsized":"2024-09-16T15:40:14.076564+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:09:37.707603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"34'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.077221+0000","last_change":"2024-09-16T15:40:11.437040+0000","last_active":"2024-09-16T15:40:14.077221+0000","last_peered":"2024-09-16T15:40:14.077221+0000","last_clean":"2024-09-16T15:40:14.077221+0000","last_became_active":"2024-09-16T15:40:11.436779+0000","last_became_peered":"2024-09-16T15:40:11.436779+0000","last_unstale":"2024-09-16T15:40:14.077221+0000","last_undegraded":"2024-09-16T15:40:14.077221+0000","last_fullsized":"2024-09-16T15:40:14.077221+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:43:13.979677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.077292+0000","last_change":"2024-09-16T15:40:12.439366+0000","last_active":"2024-09-16T15:40:14.077292+0000","last_peered":"2024-09-16T15:40:14.077292+0000","last_clean":"2024-09-16T15:40:14.077292+0000","last_became_active":"2024-09-16T15:40:12.437351+0000","last_became_peered":"2024-09-16T15:40:12.437351+0000","last_unstale":"2024-09-16T15:40:14.077292+0000","last_undegraded":"2024-09-16T15:40:14.077292+0000","last_fullsized":"2024-09-16T15:40:14.077292+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:54:46.876716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"34'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.077173+0000","last_change":"2024-09-16T15:40:11.436026+0000","last_active":"2024-09-16T15:40:14.077173+0000","last_peered":"2024-09-16T15:40:14.077173+0000","last_clean":"2024-09-16T15:40:14.077173+0000","last_became_active":"2024-09-16T15:40:11.435807+0000","last_became_peered":"2024-09-16T15:40:11.435807+0000","last_unstale":"2024-09-16T15:40:14.077173+0000","last_undegraded":"2024-09-16T15:40:14.077173+0000","last_fullsized":"2024-09-16T15:40:14.077173+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:27:41.510684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.076590+0000","last_change":"2024-09-16T15:40:12.435295+0000","last_active":"2024-09-16T15:40:14.076590+0000","last_peered":"2024-09-16T15:40:14.076590+0000","last_clean":"2024-09-16T15:40:14.076590+0000","last_became_active":"2024-09-16T15:40:12.434978+0000","last_became_peered":"2024-09-16T15:40:12.434978+0000","last_unstale":"2024-09-16T15:40:14.076590+0000","last_undegraded":"2024-09-16T15:40:14.076590+0000","last_fullsized":"2024-09-16T15:40:14.076590+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:43:14.984339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.435951+0000","last_change":"2024-09-16T15:40:11.431791+0000","last_active":"2024-09-16T15:40:13.435951+0000","last_peered":"2024-09-16T15:40:13.435951+0000","last_clean":"2024-09-16T15:40:13.435951+0000","last_became_active":"2024-09-16T15:40:11.431517+0000","last_became_peered":"2024-09-16T15:40:11.431517+0000","last_unstale":"2024-09-16T15:40:13.435951+0000","last_undegraded":"2024-09-16T15:40:13.435951+0000","last_fullsized":"2024-09-16T15:40:13.435951+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:17:20.232340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.077597+0000","last_change":"2024-09-16T15:40:12.447547+0000","last_active":"2024-09-16T15:40:14.077597+0000","last_peered":"2024-09-16T15:40:14.077597+0000","last_clean":"2024-09-16T15:40:14.077597+0000","last_became_active":"2024-09-16T15:40:12.447244+0000","last_became_peered":"2024-09-16T15:40:12.447244+0000","last_unstale":"2024-09-16T15:40:14.077597+0000","last_undegraded":"2024-09-16T15:40:14.077597+0000","last_fullsized":"2024-09-16T15:40:14.077597+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:09:28.653840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.077113+0000","last_change":"2024-09-16T15:40:11.434384+0000","last_active":"2024-09-16T15:40:14.077113+0000","last_peered":"2024-09-16T15:40:14.077113+0000","last_clean":"2024-09-16T15:40:14.077113+0000","last_became_active":"2024-09-16T15:40:11.434028+0000","last_became_peered":"2024-09-16T15:40:11.434028+0000","last_unstale":"2024-09-16T15:40:14.077113+0000","last_undegraded":"2024-09-16T15:40:14.077113+0000","last_fullsized":"2024-09-16T15:40:14.077113+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:59:15.340271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.434969+0000","last_change":"2024-09-16T15:40:12.442342+0000","last_active":"2024-09-16T15:40:13.434969+0000","last_peered":"2024-09-16T15:40:13.434969+0000","last_clean":"2024-09-16T15:40:13.434969+0000","last_became_active":"2024-09-16T15:40:12.442051+0000","last_became_peered":"2024-09-16T15:40:12.442051+0000","last_unstale":"2024-09-16T15:40:13.434969+0000","last_undegraded":"2024-09-16T15:40:13.434969+0000","last_fullsized":"2024-09-16T15:40:13.434969+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:47:50.659996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.449970+0000","last_change":"2024-09-16T15:40:11.430380+0000","last_active":"2024-09-16T15:40:13.449970+0000","last_peered":"2024-09-16T15:40:13.449970+0000","last_clean":"2024-09-16T15:40:13.449970+0000","last_became_active":"2024-09-16T15:40:11.428309+0000","last_became_peered":"2024-09-16T15:40:11.428309+0000","last_unstale":"2024-09-16T15:40:13.449970+0000","last_undegraded":"2024-09-16T15:40:13.449970+0000","last_fullsized":"2024-09-16T15:40:13.449970+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:37:43.766286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.078318+0000","last_change":"2024-09-16T15:40:12.439953+0000","last_active":"2024-09-16T15:40:14.078318+0000","last_peered":"2024-09-16T15:40:14.078318+0000","last_clean":"2024-09-16T15:40:14.078318+0000","last_became_active":"2024-09-16T15:40:12.438714+0000","last_became_peered":"2024-09-16T15:40:12.438714+0000","last_unstale":"2024-09-16T15:40:14.078318+0000","last_undegraded":"2024-09-16T15:40:14.078318+0000","last_fullsized":"2024-09-16T15:40:14.078318+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:24:50.389299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.449724+0000","last_change":"2024-09-16T15:40:11.426767+0000","last_active":"2024-09-16T15:40:13.449724+0000","last_peered":"2024-09-16T15:40:13.449724+0000","last_clean":"2024-09-16T15:40:13.449724+0000","last_became_active":"2024-09-16T15:40:11.426512+0000","last_became_peered":"2024-09-16T15:40:11.426512+0000","last_unstale":"2024-09-16T15:40:13.449724+0000","last_undegraded":"2024-09-16T15:40:13.449724+0000","last_fullsized":"2024-09-16T15:40:13.449724+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:48:35.010191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.077913+0000","last_change":"2024-09-16T15:40:12.447596+0000","last_active":"2024-09-16T15:40:14.077913+0000","last_peered":"2024-09-16T15:40:14.077913+0000","last_clean":"2024-09-16T15:40:14.077913+0000","last_became_active":"2024-09-16T15:40:12.447303+0000","last_became_peered":"2024-09-16T15:40:12.447303+0000","last_unstale":"2024-09-16T15:40:14.077913+0000","last_undegraded":"2024-09-16T15:40:14.077913+0000","last_fullsized":"2024-09-16T15:40:14.077913+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:36:34.759651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"34'3","reported_seq":18,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.434442+0000","last_change":"2024-09-16T15:40:11.433864+0000","last_active":"2024-09-16T15:40:13.434442+0000","last_peered":"2024-09-16T15:40:13.434442+0000","last_clean":"2024-09-16T15:40:13.434442+0000","last_became_active":"2024-09-16T15:40:11.426559+0000","last_became_peered":"2024-09-16T15:40:11.426559+0000","last_unstale":"2024-09-16T15:40:13.434442+0000","last_undegraded":"2024-09-16T15:40:13.434442+0000","last_fullsized":"2024-09-16T15:40:13.434442+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:12:51.709454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.434553+0000","last_change":"2024-09-16T15:40:12.456376+0000","last_active":"2024-09-16T15:40:13.434553+0000","last_peered":"2024-09-16T15:40:13.434553+0000","last_clean":"2024-09-16T15:40:13.434553+0000","last_became_active":"2024-09-16T15:40:12.455834+0000","last_became_peered":"2024-09-16T15:40:12.455834+0000","last_unstale":"2024-09-16T15:40:13.434553+0000","last_undegraded":"2024-09-16T15:40:13.434553+0000","last_fullsized":"2024-09-16T15:40:13.434553+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:58:20.955348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"34'6","reported_seq":21,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.437346+0000","last_change":"2024-09-16T15:40:11.432845+0000","last_active":"2024-09-16T15:40:13.437346+0000","last_peered":"2024-09-16T15:40:13.437346+0000","last_clean":"2024-09-16T15:40:13.437346+0000","last_became_active":"2024-09-16T15:40:11.432442+0000","last_became_peered":"2024-09-16T15:40:11.432442+0000","last_unstale":"2024-09-16T15:40:13.437346+0000","last_undegraded":"2024-09-16T15:40:13.437346+0000","last_fullsized":"2024-09-16T15:40:13.437346+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:16:26.812176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.436372+0000","last_change":"2024-09-16T15:40:12.436479+0000","last_active":"2024-09-16T15:40:13.436372+0000","last_peered":"2024-09-16T15:40:13.436372+0000","last_clean":"2024-09-16T15:40:13.436372+0000","last_became_active":"2024-09-16T15:40:12.436247+0000","last_became_peered":"2024-09-16T15:40:12.436247+0000","last_unstale":"2024-09-16T15:40:13.436372+0000","last_undegraded":"2024-09-16T15:40:13.436372+0000","last_fullsized":"2024-09-16T15:40:13.436372+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:13:44.038668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"34'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.077325+0000","last_change":"2024-09-16T15:40:11.439929+0000","last_active":"2024-09-16T15:40:14.077325+0000","last_peered":"2024-09-16T15:40:14.077325+0000","last_clean":"2024-09-16T15:40:14.077325+0000","last_became_active":"2024-09-16T15:40:11.439583+0000","last_became_peered":"2024-09-16T15:40:11.439583+0000","last_unstale":"2024-09-16T15:40:14.077325+0000","last_undegraded":"2024-09-16T15:40:14.077325+0000","last_fullsized":"2024-09-16T15:40:14.077325+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:43:00.710200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.077800+0000","last_change":"2024-09-16T15:40:12.444800+0000","last_active":"2024-09-16T15:40:14.077800+0000","last_peered":"2024-09-16T15:40:14.077800+0000","last_clean":"2024-09-16T15:40:14.077800+0000","last_became_active":"2024-09-16T15:40:12.444523+0000","last_became_peered":"2024-09-16T15:40:12.444523+0000","last_unstale":"2024-09-16T15:40:14.077800+0000","last_undegraded":"2024-09-16T15:40:14.077800+0000","last_fullsized":"2024-09-16T15:40:14.077800+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:03:15.062570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.077333+0000","last_change":"2024-09-16T15:40:11.426982+0000","last_active":"2024-09-16T15:40:14.077333+0000","last_peered":"2024-09-16T15:40:14.077333+0000","last_clean":"2024-09-16T15:40:14.077333+0000","last_became_active":"2024-09-16T15:40:11.426667+0000","last_became_peered":"2024-09-16T15:40:11.426667+0000","last_unstale":"2024-09-16T15:40:14.077333+0000","last_undegraded":"2024-09-16T15:40:14.077333+0000","last_fullsized":"2024-09-16T15:40:14.077333+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:27:41.510684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.077055+0000","last_change":"2024-09-16T15:40:12.436737+0000","last_active":"2024-09-16T15:40:14.077055+0000","last_peered":"2024-09-16T15:40:14.077055+0000","last_clean":"2024-09-16T15:40:14.077055+0000","last_became_active":"2024-09-16T15:40:12.435893+0000","last_became_peered":"2024-09-16T15:40:12.435893+0000","last_unstale":"2024-09-16T15:40:14.077055+0000","last_undegraded":"2024-09-16T15:40:14.077055+0000","last_fullsized":"2024-09-16T15:40:14.077055+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:03:28.331224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.077161+0000","last_change":"2024-09-16T15:40:11.426297+0000","last_active":"2024-09-16T15:40:14.077161+0000","last_peered":"2024-09-16T15:40:14.077161+0000","last_clean":"2024-09-16T15:40:14.077161+0000","last_became_active":"2024-09-16T15:40:11.425935+0000","last_became_peered":"2024-09-16T15:40:11.425935+0000","last_unstale":"2024-09-16T15:40:14.077161+0000","last_undegraded":"2024-09-16T15:40:14.077161+0000","last_fullsized":"2024-09-16T15:40:14.077161+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:05:40.463616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.434380+0000","last_change":"2024-09-16T15:40:12.444134+0000","last_active":"2024-09-16T15:40:13.434380+0000","last_peered":"2024-09-16T15:40:13.434380+0000","last_clean":"2024-09-16T15:40:13.434380+0000","last_became_active":"2024-09-16T15:40:12.443929+0000","last_became_peered":"2024-09-16T15:40:12.443929+0000","last_unstale":"2024-09-16T15:40:13.434380+0000","last_undegraded":"2024-09-16T15:40:13.434380+0000","last_fullsized":"2024-09-16T15:40:13.434380+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:51:48.095296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"34'8","reported_seq":23,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.435447+0000","last_change":"2024-09-16T15:40:11.425677+0000","last_active":"2024-09-16T15:40:13.435447+0000","last_peered":"2024-09-16T15:40:13.435447+0000","last_clean":"2024-09-16T15:40:13.435447+0000","last_became_active":"2024-09-16T15:40:11.425390+0000","last_became_peered":"2024-09-16T15:40:11.425390+0000","last_unstale":"2024-09-16T15:40:13.435447+0000","last_undegraded":"2024-09-16T15:40:13.435447+0000","last_fullsized":"2024-09-16T15:40:13.435447+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:02:34.885995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.434958+0000","last_change":"2024-09-16T15:40:12.455503+0000","last_active":"2024-09-16T15:40:13.434958+0000","last_peered":"2024-09-16T15:40:13.434958+0000","last_clean":"2024-09-16T15:40:13.434958+0000","last_became_active":"2024-09-16T15:40:12.455167+0000","last_became_peered":"2024-09-16T15:40:12.455167+0000","last_unstale":"2024-09-16T15:40:13.434958+0000","last_undegraded":"2024-09-16T15:40:13.434958+0000","last_fullsized":"2024-09-16T15:40:13.434958+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:50:57.607623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"34'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.436125+0000","last_change":"2024-09-16T15:40:11.433816+0000","last_active":"2024-09-16T15:40:13.436125+0000","last_peered":"2024-09-16T15:40:13.436125+0000","last_clean":"2024-09-16T15:40:13.436125+0000","last_became_active":"2024-09-16T15:40:11.433028+0000","last_became_peered":"2024-09-16T15:40:11.433028+0000","last_unstale":"2024-09-16T15:40:13.436125+0000","last_undegraded":"2024-09-16T15:40:13.436125+0000","last_fullsized":"2024-09-16T15:40:13.436125+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:05:55.235854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.076987+0000","last_change":"2024-09-16T15:40:12.435921+0000","last_active":"2024-09-16T15:40:14.076987+0000","last_peered":"2024-09-16T15:40:14.076987+0000","last_clean":"2024-09-16T15:40:14.076987+0000","last_became_active":"2024-09-16T15:40:12.435533+0000","last_became_peered":"2024-09-16T15:40:12.435533+0000","last_unstale":"2024-09-16T15:40:14.076987+0000","last_undegraded":"2024-09-16T15:40:14.076987+0000","last_fullsized":"2024-09-16T15:40:14.076987+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:27:42.515347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"34'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.077304+0000","last_change":"2024-09-16T15:40:11.439905+0000","last_active":"2024-09-16T15:40:14.077304+0000","last_peered":"2024-09-16T15:40:14.077304+0000","last_clean":"2024-09-16T15:40:14.077304+0000","last_became_active":"2024-09-16T15:40:11.439567+0000","last_became_peered":"2024-09-16T15:40:11.439567+0000","last_unstale":"2024-09-16T15:40:14.077304+0000","last_undegraded":"2024-09-16T15:40:14.077304+0000","last_fullsized":"2024-09-16T15:40:14.077304+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:40:52.912231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.449794+0000","last_change":"2024-09-16T15:40:12.436487+0000","last_active":"2024-09-16T15:40:13.449794+0000","last_peered":"2024-09-16T15:40:13.449794+0000","last_clean":"2024-09-16T15:40:13.449794+0000","last_became_active":"2024-09-16T15:40:12.436193+0000","last_became_peered":"2024-09-16T15:40:12.436193+0000","last_unstale":"2024-09-16T15:40:13.449794+0000","last_undegraded":"2024-09-16T15:40:13.449794+0000","last_fullsized":"2024-09-16T15:40:13.449794+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:33:58.217901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"34'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.434609+0000","last_change":"2024-09-16T15:40:11.432937+0000","last_active":"2024-09-16T15:40:13.434609+0000","last_peered":"2024-09-16T15:40:13.434609+0000","last_clean":"2024-09-16T15:40:13.434609+0000","last_became_active":"2024-09-16T15:40:11.428199+0000","last_became_peered":"2024-09-16T15:40:11.428199+0000","last_unstale":"2024-09-16T15:40:13.434609+0000","last_undegraded":"2024-09-16T15:40:13.434609+0000","last_fullsized":"2024-09-16T15:40:13.434609+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:33:57.213238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.436182+0000","last_change":"2024-09-16T15:40:12.438579+0000","last_active":"2024-09-16T15:40:13.436182+0000","last_peered":"2024-09-16T15:40:13.436182+0000","last_clean":"2024-09-16T15:40:13.436182+0000","last_became_active":"2024-09-16T15:40:12.437311+0000","last_became_peered":"2024-09-16T15:40:12.437311+0000","last_unstale":"2024-09-16T15:40:13.436182+0000","last_undegraded":"2024-09-16T15:40:13.436182+0000","last_fullsized":"2024-09-16T15:40:13.436182+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:04:48.883149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.014769+0000","last_change":"2024-09-16T15:40:11.433487+0000","last_active":"2024-09-16T15:40:14.014769+0000","last_peered":"2024-09-16T15:40:14.014769+0000","last_clean":"2024-09-16T15:40:14.014769+0000","last_became_active":"2024-09-16T15:40:11.431531+0000","last_became_peered":"2024-09-16T15:40:11.431531+0000","last_unstale":"2024-09-16T15:40:14.014769+0000","last_undegraded":"2024-09-16T15:40:14.014769+0000","last_fullsized":"2024-09-16T15:40:14.014769+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:26:31.371355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.435633+0000","last_change":"2024-09-16T15:40:12.437874+0000","last_active":"2024-09-16T15:40:13.435633+0000","last_peered":"2024-09-16T15:40:13.435633+0000","last_clean":"2024-09-16T15:40:13.435633+0000","last_became_active":"2024-09-16T15:40:12.436684+0000","last_became_peered":"2024-09-16T15:40:12.436684+0000","last_unstale":"2024-09-16T15:40:13.435633+0000","last_undegraded":"2024-09-16T15:40:13.435633+0000","last_fullsized":"2024-09-16T15:40:13.435633+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:38:27.284097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"34'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.449810+0000","last_change":"2024-09-16T15:40:11.421728+0000","last_active":"2024-09-16T15:40:13.449810+0000","last_peered":"2024-09-16T15:40:13.449810+0000","last_clean":"2024-09-16T15:40:13.449810+0000","last_became_active":"2024-09-16T15:40:11.421458+0000","last_became_peered":"2024-09-16T15:40:11.421458+0000","last_unstale":"2024-09-16T15:40:13.449810+0000","last_undegraded":"2024-09-16T15:40:13.449810+0000","last_fullsized":"2024-09-16T15:40:13.449810+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:19:01.909987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.449835+0000","last_change":"2024-09-16T15:40:12.438536+0000","last_active":"2024-09-16T15:40:13.449835+0000","last_peered":"2024-09-16T15:40:13.449835+0000","last_clean":"2024-09-16T15:40:13.449835+0000","last_became_active":"2024-09-16T15:40:12.438227+0000","last_became_peered":"2024-09-16T15:40:12.438227+0000","last_unstale":"2024-09-16T15:40:13.449835+0000","last_undegraded":"2024-09-16T15:40:13.449835+0000","last_fullsized":"2024-09-16T15:40:13.449835+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:26:09.350575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.077176+0000","last_change":"2024-09-16T15:40:11.440951+0000","last_active":"2024-09-16T15:40:14.077176+0000","last_peered":"2024-09-16T15:40:14.077176+0000","last_clean":"2024-09-16T15:40:14.077176+0000","last_became_active":"2024-09-16T15:40:11.440590+0000","last_became_peered":"2024-09-16T15:40:11.440590+0000","last_unstale":"2024-09-16T15:40:14.077176+0000","last_undegraded":"2024-09-16T15:40:14.077176+0000","last_fullsized":"2024-09-16T15:40:14.077176+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:16:26.812176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.435374+0000","last_change":"2024-09-16T15:40:12.452533+0000","last_active":"2024-09-16T15:40:13.435374+0000","last_peered":"2024-09-16T15:40:13.435374+0000","last_clean":"2024-09-16T15:40:13.435374+0000","last_became_active":"2024-09-16T15:40:12.451706+0000","last_became_peered":"2024-09-16T15:40:12.451706+0000","last_unstale":"2024-09-16T15:40:13.435374+0000","last_undegraded":"2024-09-16T15:40:13.435374+0000","last_fullsized":"2024-09-16T15:40:13.435374+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:18:02.318732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.077074+0000","last_change":"2024-09-16T15:40:11.429461+0000","last_active":"2024-09-16T15:40:14.077074+0000","last_peered":"2024-09-16T15:40:14.077074+0000","last_clean":"2024-09-16T15:40:14.077074+0000","last_became_active":"2024-09-16T15:40:11.429262+0000","last_became_peered":"2024-09-16T15:40:11.429262+0000","last_unstale":"2024-09-16T15:40:14.077074+0000","last_undegraded":"2024-09-16T15:40:14.077074+0000","last_fullsized":"2024-09-16T15:40:14.077074+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:23:23.370290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.076578+0000","last_change":"2024-09-16T15:40:12.435345+0000","last_active":"2024-09-16T15:40:14.076578+0000","last_peered":"2024-09-16T15:40:14.076578+0000","last_clean":"2024-09-16T15:40:14.076578+0000","last_became_active":"2024-09-16T15:40:12.435060+0000","last_became_peered":"2024-09-16T15:40:12.435060+0000","last_unstale":"2024-09-16T15:40:14.076578+0000","last_undegraded":"2024-09-16T15:40:14.076578+0000","last_fullsized":"2024-09-16T15:40:14.076578+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:40:53.916894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.434156+0000","last_change":"2024-09-16T15:40:11.435516+0000","last_active":"2024-09-16T15:40:13.434156+0000","last_peered":"2024-09-16T15:40:13.434156+0000","last_clean":"2024-09-16T15:40:13.434156+0000","last_became_active":"2024-09-16T15:40:11.433754+0000","last_became_peered":"2024-09-16T15:40:11.433754+0000","last_unstale":"2024-09-16T15:40:13.434156+0000","last_undegraded":"2024-09-16T15:40:13.434156+0000","last_fullsized":"2024-09-16T15:40:13.434156+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:03:27.326561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.436365+0000","last_change":"2024-09-16T15:40:12.437260+0000","last_active":"2024-09-16T15:40:13.436365+0000","last_peered":"2024-09-16T15:40:13.436365+0000","last_clean":"2024-09-16T15:40:13.436365+0000","last_became_active":"2024-09-16T15:40:12.437001+0000","last_became_peered":"2024-09-16T15:40:12.437001+0000","last_unstale":"2024-09-16T15:40:13.436365+0000","last_undegraded":"2024-09-16T15:40:13.436365+0000","last_fullsized":"2024-09-16T15:40:13.436365+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:24:21.290065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"34'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.437092+0000","last_change":"2024-09-16T15:40:11.429377+0000","last_active":"2024-09-16T15:40:13.437092+0000","last_peered":"2024-09-16T15:40:13.437092+0000","last_clean":"2024-09-16T15:40:13.437092+0000","last_became_active":"2024-09-16T15:40:11.429116+0000","last_became_peered":"2024-09-16T15:40:11.429116+0000","last_unstale":"2024-09-16T15:40:13.437092+0000","last_undegraded":"2024-09-16T15:40:13.437092+0000","last_fullsized":"2024-09-16T15:40:13.437092+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:05:40.463616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.434894+0000","last_change":"2024-09-16T15:40:12.439353+0000","last_active":"2024-09-16T15:40:13.434894+0000","last_peered":"2024-09-16T15:40:13.434894+0000","last_clean":"2024-09-16T15:40:13.434894+0000","last_became_active":"2024-09-16T15:40:12.438075+0000","last_became_peered":"2024-09-16T15:40:12.438075+0000","last_unstale":"2024-09-16T15:40:13.434894+0000","last_undegraded":"2024-09-16T15:40:13.434894+0000","last_fullsized":"2024-09-16T15:40:13.434894+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:54:34.251717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"35'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.449901+0000","last_change":"2024-09-16T15:40:11.423158+0000","last_active":"2024-09-16T15:40:13.449901+0000","last_peered":"2024-09-16T15:40:13.449901+0000","last_clean":"2024-09-16T15:40:13.449901+0000","last_became_active":"2024-09-16T15:40:11.422892+0000","last_became_peered":"2024-09-16T15:40:11.422892+0000","last_unstale":"2024-09-16T15:40:13.449901+0000","last_undegraded":"2024-09-16T15:40:13.449901+0000","last_fullsized":"2024-09-16T15:40:13.449901+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:10:39.453392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.436527+0000","last_change":"2024-09-16T15:40:12.451657+0000","last_active":"2024-09-16T15:40:13.436527+0000","last_peered":"2024-09-16T15:40:13.436527+0000","last_clean":"2024-09-16T15:40:13.436527+0000","last_became_active":"2024-09-16T15:40:12.450795+0000","last_became_peered":"2024-09-16T15:40:12.450795+0000","last_unstale":"2024-09-16T15:40:13.436527+0000","last_undegraded":"2024-09-16T15:40:13.436527+0000","last_fullsized":"2024-09-16T15:40:13.436527+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:46:24.970506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.437395+0000","last_change":"2024-09-16T15:40:11.434071+0000","last_active":"2024-09-16T15:40:13.437395+0000","last_peered":"2024-09-16T15:40:13.437395+0000","last_clean":"2024-09-16T15:40:13.437395+0000","last_became_active":"2024-09-16T15:40:11.433566+0000","last_became_peered":"2024-09-16T15:40:11.433566+0000","last_unstale":"2024-09-16T15:40:13.437395+0000","last_undegraded":"2024-09-16T15:40:13.437395+0000","last_fullsized":"2024-09-16T15:40:13.437395+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:27:41.510684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.014650+0000","last_change":"2024-09-16T15:40:12.443182+0000","last_active":"2024-09-16T15:40:14.014650+0000","last_peered":"2024-09-16T15:40:14.014650+0000","last_clean":"2024-09-16T15:40:14.014650+0000","last_became_active":"2024-09-16T15:40:12.442931+0000","last_became_peered":"2024-09-16T15:40:12.442931+0000","last_unstale":"2024-09-16T15:40:14.014650+0000","last_undegraded":"2024-09-16T15:40:14.014650+0000","last_fullsized":"2024-09-16T15:40:14.014650+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:54:46.876716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.437007+0000","last_change":"2024-09-16T15:40:11.430371+0000","last_active":"2024-09-16T15:40:13.437007+0000","last_peered":"2024-09-16T15:40:13.437007+0000","last_clean":"2024-09-16T15:40:13.437007+0000","last_became_active":"2024-09-16T15:40:11.430082+0000","last_became_peered":"2024-09-16T15:40:11.430082+0000","last_unstale":"2024-09-16T15:40:13.437007+0000","last_undegraded":"2024-09-16T15:40:13.437007+0000","last_fullsized":"2024-09-16T15:40:13.437007+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:43:13.979677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.077802+0000","last_change":"2024-09-16T15:40:12.444868+0000","last_active":"2024-09-16T15:40:14.077802+0000","last_peered":"2024-09-16T15:40:14.077802+0000","last_clean":"2024-09-16T15:40:14.077802+0000","last_became_active":"2024-09-16T15:40:12.444573+0000","last_became_peered":"2024-09-16T15:40:12.444573+0000","last_unstale":"2024-09-16T15:40:14.077802+0000","last_undegraded":"2024-09-16T15:40:14.077802+0000","last_fullsized":"2024-09-16T15:40:14.077802+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:13:54.801126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.436446+0000","last_change":"2024-09-16T15:40:11.437894+0000","last_active":"2024-09-16T15:40:13.436446+0000","last_peered":"2024-09-16T15:40:13.436446+0000","last_clean":"2024-09-16T15:40:13.436446+0000","last_became_active":"2024-09-16T15:40:11.437657+0000","last_became_peered":"2024-09-16T15:40:11.437657+0000","last_unstale":"2024-09-16T15:40:13.436446+0000","last_undegraded":"2024-09-16T15:40:13.436446+0000","last_fullsized":"2024-09-16T15:40:13.436446+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:43:00.710200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.077598+0000","last_change":"2024-09-16T15:40:12.441083+0000","last_active":"2024-09-16T15:40:14.077598+0000","last_peered":"2024-09-16T15:40:14.077598+0000","last_clean":"2024-09-16T15:40:14.077598+0000","last_became_active":"2024-09-16T15:40:12.440787+0000","last_became_peered":"2024-09-16T15:40:12.440787+0000","last_unstale":"2024-09-16T15:40:14.077598+0000","last_undegraded":"2024-09-16T15:40:14.077598+0000","last_fullsized":"2024-09-16T15:40:14.077598+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:12:52.714116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.076955+0000","last_change":"2024-09-16T15:40:11.436770+0000","last_active":"2024-09-16T15:40:14.076955+0000","last_peered":"2024-09-16T15:40:14.076955+0000","last_clean":"2024-09-16T15:40:14.076955+0000","last_became_active":"2024-09-16T15:40:11.435878+0000","last_became_peered":"2024-09-16T15:40:11.435878+0000","last_unstale":"2024-09-16T15:40:14.076955+0000","last_undegraded":"2024-09-16T15:40:14.076955+0000","last_fullsized":"2024-09-16T15:40:14.076955+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:05:40.463616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.434410+0000","last_change":"2024-09-16T15:40:12.437726+0000","last_active":"2024-09-16T15:40:13.434410+0000","last_peered":"2024-09-16T15:40:13.434410+0000","last_clean":"2024-09-16T15:40:13.434410+0000","last_became_active":"2024-09-16T15:40:12.436801+0000","last_became_peered":"2024-09-16T15:40:12.436801+0000","last_unstale":"2024-09-16T15:40:13.434410+0000","last_undegraded":"2024-09-16T15:40:13.434410+0000","last_fullsized":"2024-09-16T15:40:13.434410+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:10:10.169816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.077439+0000","last_change":"2024-09-16T15:40:11.428945+0000","last_active":"2024-09-16T15:40:14.077439+0000","last_peered":"2024-09-16T15:40:14.077439+0000","last_clean":"2024-09-16T15:40:14.077439+0000","last_became_active":"2024-09-16T15:40:11.428296+0000","last_became_peered":"2024-09-16T15:40:11.428296+0000","last_unstale":"2024-09-16T15:40:14.077439+0000","last_undegraded":"2024-09-16T15:40:14.077439+0000","last_fullsized":"2024-09-16T15:40:14.077439+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:32:21.531717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.014863+0000","last_change":"2024-09-16T15:40:12.437406+0000","last_active":"2024-09-16T15:40:14.014863+0000","last_peered":"2024-09-16T15:40:14.014863+0000","last_clean":"2024-09-16T15:40:14.014863+0000","last_became_active":"2024-09-16T15:40:12.437117+0000","last_became_peered":"2024-09-16T15:40:12.437117+0000","last_unstale":"2024-09-16T15:40:14.014863+0000","last_undegraded":"2024-09-16T15:40:14.014863+0000","last_fullsized":"2024-09-16T15:40:14.014863+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:11.403725+0000","last_clean_scrub_stamp":"2024-09-16T15:40:11.403725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:57:43.248746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:13.434720+0000","last_change":"2024-09-16T15:40:11.428439+0000","last_active":"2024-09-16T15:40:13.434720+0000","last_peered":"2024-09-16T15:40:13.434720+0000","last_clean":"2024-09-16T15:40:13.434720+0000","last_became_active":"2024-09-16T15:40:11.427829+0000","last_became_peered":"2024-09-16T15:40:11.427829+0000","last_unstale":"2024-09-16T15:40:13.434720+0000","last_undegraded":"2024-09-16T15:40:13.434720+0000","last_fullsized":"2024-09-16T15:40:13.434720+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:40:10.399062+0000","last_clean_scrub_stamp":"2024-09-16T15:40:10.399062+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:54:45.872053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":135,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T15:40:14.076426+0000","last_change":"2024-09-16T15:39:12.698813+0000","last_active":"2024-09-16T15:40:14.076426+0000","last_peered":"2024-09-16T15:40:14.076426+0000","last_clean":"2024-09-16T15:40:14.076426+0000","last_became_active":"2024-09-16T15:39:12.697695+0000","last_became_peered":"2024-09-16T15:39:12.697695+0000","last_unstale":"2024-09-16T15:40:14.076426+0000","last_undegraded":"2024-09-16T15:40:14.076426+0000","last_fullsized":"2024-09-16T15:40:14.076426+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_clean_scrub_stamp":"2024-09-16T15:39:11.684967+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:02.061081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509459,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":401408,"data_stored":153170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509458,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27996,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343844,"statfs":{"total":96636764160,"available":96608096256,"internally_reserved":0,"allocated":987136,"data_stored":742508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509457,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":405504,"data_stored":153192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509457,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":405504,"data_stored":152334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509457,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":393216,"data_stored":151592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509458,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28000,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":987136,"data_stored":741996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509456,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":393216,"data_stored":151648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542163,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":393216,"data_stored":151580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":27654604},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:40:21.740 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T15:40:21.740 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-16T15:40:21.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.940+0000 7f1ddf800640 1 -- 172.21.15.151:0/1426973683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1dd806efa0 msgr2=0x7f1dd80cc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:21.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.940+0000 7f1ddf800640 1 --2- 172.21.15.151:0/1426973683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1dd806efa0 0x7f1dd80cc5c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1dd80d0cd0 tx=0x7f1dc402a4d0 comp rx=0 tx=0).stop 2024-09-16T15:40:21.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.940+0000 7f1ddf800640 1 -- 172.21.15.151:0/1426973683 shutdown_connections 2024-09-16T15:40:21.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.940+0000 7f1ddf800640 1 --2- 172.21.15.151:0/1426973683 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1dd80ccb00 0x7f1dd815f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:21.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.941+0000 7f1ddf800640 1 --2- 172.21.15.151:0/1426973683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1dd806efa0 0x7f1dd80cc5c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:21.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.941+0000 7f1ddf800640 1 --2- 172.21.15.151:0/1426973683 >> v2:172.21.15.167:3300/0 conn(0x7f1dd8145750 0x7f1dd8073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:21.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.941+0000 7f1ddf800640 1 -- 172.21.15.151:0/1426973683 >> 172.21.15.151:0/1426973683 conn(0x7f1dd80b1840 msgr2=0x7f1dd80b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:21.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.941+0000 7f1ddf800640 1 -- 172.21.15.151:0/1426973683 shutdown_connections 2024-09-16T15:40:21.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.941+0000 7f1ddf800640 1 -- 172.21.15.151:0/1426973683 wait complete. 2024-09-16T15:40:21.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.942+0000 7f1ddf800640 1 Processor -- start 2024-09-16T15:40:21.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.942+0000 7f1ddf800640 1 -- start start 2024-09-16T15:40:21.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.943+0000 7f1ddf800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1dd806efa0 0x7f1dd815e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:21.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.943+0000 7f1ddf800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1dd80ccb00 0x7f1dd815e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:21.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.943+0000 7f1ddf800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1dd8145750 0x7f1dd80aa310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:21.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.943+0000 7f1ddf800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd81602c0 con 0x7f1dd806efa0 2024-09-16T15:40:21.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.943+0000 7f1ddf800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd8160140 con 0x7f1dd8145750 2024-09-16T15:40:21.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.943+0000 7f1ddf800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f1dd8160440 con 0x7f1dd80ccb00 2024-09-16T15:40:21.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.943+0000 7f1ddce00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1dd806efa0 0x7f1dd815e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:21.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.943+0000 7f1ddce00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1dd806efa0 0x7f1dd815e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37382/0 (socket says 172.21.15.151:37382) 2024-09-16T15:40:21.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.943+0000 7f1ddce00640 1 -- 172.21.15.151:0/2138544895 learned_addr learned my addr 172.21.15.151:0/2138544895 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:40:21.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.944+0000 7f1ddce00640 1 -- 172.21.15.151:0/2138544895 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1dd80ccb00 msgr2=0x7f1dd815e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:21.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.944+0000 7f1ddce00640 1 --2- 172.21.15.151:0/2138544895 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1dd80ccb00 0x7f1dd815e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:21.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.944+0000 7f1ddce00640 1 -- 172.21.15.151:0/2138544895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1dd8145750 msgr2=0x7f1dd80aa310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:21.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.944+0000 7f1ddce00640 1 --2- 172.21.15.151:0/2138544895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1dd8145750 0x7f1dd80aa310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:21.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.944+0000 7f1ddce00640 1 -- 172.21.15.151:0/2138544895 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dd81f8e20 con 0x7f1dd806efa0 2024-09-16T15:40:21.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.944+0000 7f1ddce00640 1 --2- 172.21.15.151:0/2138544895 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1dd806efa0 0x7f1dd815e4a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1dc800eb30 tx=0x7f1dc800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:21.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.944+0000 7f1dd4c00640 1 -- 172.21.15.151:0/2138544895 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc8010430 con 0x7f1dd806efa0 2024-09-16T15:40:21.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.944+0000 7f1dd4c00640 1 -- 172.21.15.151:0/2138544895 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1dc80105d0 con 0x7f1dd806efa0 2024-09-16T15:40:21.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.944+0000 7f1ddf800640 1 -- 172.21.15.151:0/2138544895 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dd80ad650 con 0x7f1dd806efa0 2024-09-16T15:40:21.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.945+0000 7f1dd4c00640 1 -- 172.21.15.151:0/2138544895 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc8010a20 con 0x7f1dd806efa0 2024-09-16T15:40:21.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.945+0000 7f1ddf800640 1 -- 172.21.15.151:0/2138544895 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dd80a9800 con 0x7f1dd806efa0 2024-09-16T15:40:21.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.946+0000 7f1ddf800640 1 -- 172.21.15.151:0/2138544895 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dd815f9f0 con 0x7f1dd806efa0 2024-09-16T15:40:21.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.947+0000 7f1dd4c00640 1 -- 172.21.15.151:0/2138544895 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f1dc8010bc0 con 0x7f1dd806efa0 2024-09-16T15:40:21.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.948+0000 7f1dd4c00640 1 --2- 172.21.15.151:0/2138544895 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1db8078850 0x7f1db807ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:21.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.948+0000 7f1dd4c00640 1 -- 172.21.15.151:0/2138544895 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f1dc809b760 con 0x7f1dd806efa0 2024-09-16T15:40:21.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.948+0000 7f1dd7e00640 1 --2- 172.21.15.151:0/2138544895 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1db8078850 0x7f1db807ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:21.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.949+0000 7f1dd7e00640 1 --2- 172.21.15.151:0/2138544895 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1db8078850 0x7f1db807ad10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1dc4033000 tx=0x7f1dc40046c0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:21.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:21.952+0000 7f1dd4c00640 1 -- 172.21.15.151:0/2138544895 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1dc8063dc0 con 0x7f1dd806efa0 2024-09-16T15:40:22.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:22.206+0000 7f1ddf800640 1 -- 172.21.15.151:0/2138544895 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f1dd81f8c10 con 0x7f1dd806efa0 2024-09-16T15:40:22.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:22.209+0000 7f1dd4c00640 1 -- 172.21.15.151:0/2138544895 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v9) ==== 190+0+147 (secure 0 0 0) 0x7f1dc80636c0 con 0x7f1dd806efa0 2024-09-16T15:40:22.211 INFO:teuthology.orchestra.run.smithi151.stdout:[client.0] 2024-09-16T15:40:22.211 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQDsUOhmf5PBKBAAKE4Y4zbA4h07IBNjPkyptw== 2024-09-16T15:40:22.211 INFO:teuthology.orchestra.run.smithi151.stdout: caps mds = "allow" 2024-09-16T15:40:22.212 INFO:teuthology.orchestra.run.smithi151.stdout: caps mon = "allow r" 2024-09-16T15:40:22.212 INFO:teuthology.orchestra.run.smithi151.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-16T15:40:22.212 INFO:teuthology.orchestra.run.smithi151.stderr:updated caps for client.0 2024-09-16T15:40:22.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:22.212+0000 7f1ddf800640 1 -- 172.21.15.151:0/2138544895 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1db8078850 msgr2=0x7f1db807ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:22.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:22.212+0000 7f1ddf800640 1 --2- 172.21.15.151:0/2138544895 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1db8078850 0x7f1db807ad10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1dc4033000 tx=0x7f1dc40046c0 comp rx=0 tx=0).stop 2024-09-16T15:40:22.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:22.212+0000 7f1ddf800640 1 -- 172.21.15.151:0/2138544895 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1dd806efa0 msgr2=0x7f1dd815e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:22.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:22.212+0000 7f1ddf800640 1 --2- 172.21.15.151:0/2138544895 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1dd806efa0 0x7f1dd815e4a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1dc800eb30 tx=0x7f1dc800c2d0 comp rx=0 tx=0).stop 2024-09-16T15:40:22.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:22.212+0000 7f1ddf800640 1 -- 172.21.15.151:0/2138544895 shutdown_connections 2024-09-16T15:40:22.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:22.212+0000 7f1ddf800640 1 --2- 172.21.15.151:0/2138544895 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1db8078850 0x7f1db807ad10 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:22.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:22.212+0000 7f1ddf800640 1 --2- 172.21.15.151:0/2138544895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1dd8145750 0x7f1dd80aa310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:22.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:22.212+0000 7f1ddf800640 1 --2- 172.21.15.151:0/2138544895 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1dd80ccb00 0x7f1dd815e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:22.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:22.212+0000 7f1ddf800640 1 --2- 172.21.15.151:0/2138544895 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1dd806efa0 0x7f1dd815e4a0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:22.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:22.212+0000 7f1ddf800640 1 -- 172.21.15.151:0/2138544895 >> 172.21.15.151:0/2138544895 conn(0x7f1dd80b1840 msgr2=0x7f1dd80cdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:22.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:22.212+0000 7f1ddf800640 1 -- 172.21.15.151:0/2138544895 shutdown_connections 2024-09-16T15:40:22.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:22.212+0000 7f1ddf800640 1 -- 172.21.15.151:0/2138544895 wait complete. 2024-09-16T15:40:22.231 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:40:22.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:22.387+0000 7f2ce6400640 1 -- 172.21.15.151:0/1853980316 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2ce00b2440 msgr2=0x7f2ce006f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:22.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:22.387+0000 7f2ce6400640 1 --2- 172.21.15.151:0/1853980316 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2ce00b2440 0x7f2ce006f160 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8009be0 tx=0x7f2cd802a820 comp rx=0 tx=0).stop 2024-09-16T15:40:22.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:22.387+0000 7f2ce6400640 1 -- 172.21.15.151:0/1853980316 shutdown_connections 2024-09-16T15:40:22.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:22.387+0000 7f2ce6400640 1 --2- 172.21.15.151:0/1853980316 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2ce00b2440 0x7f2ce006f160 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:22.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:22.387+0000 7f2ce6400640 1 -- 172.21.15.151:0/1853980316 >> 172.21.15.151:0/1853980316 conn(0x7f2ce00b1950 msgr2=0x7f2ce00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:22.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:22.387+0000 7f2ce6400640 1 -- 172.21.15.151:0/1853980316 shutdown_connections 2024-09-16T15:40:22.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:22.387+0000 7f2ce6400640 1 -- 172.21.15.151:0/1853980316 wait complete. 2024-09-16T15:40:22.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:22.388+0000 7f2ce6400640 1 Processor -- start 2024-09-16T15:40:22.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:22.389+0000 7f2ce6400640 1 -- start start 2024-09-16T15:40:22.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:22.389+0000 7f2ce6400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ce00b2440 0x7f2ce014bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:22.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:22.389+0000 7f2ce6400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2ce00ccf00 0x7f2ce01577e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:22.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:22.389+0000 7f2ce6400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2ce015f5a0 0x7f2ce01f80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:22.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:22.389+0000 7f2ce6400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce0074840 con 0x7f2ce00ccf00 2024-09-16T15:40:22.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:22.389+0000 7f2ce6400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce00746c0 con 0x7f2ce00b2440 2024-09-16T15:40:22.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:22.389+0000 7f2ce6400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f2ce006f6a0 con 0x7f2ce015f5a0 2024-09-16T15:40:22.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:22.390+0000 7f2cdea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2ce00ccf00 0x7f2ce01577e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:22.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:22.390+0000 7f2cdea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2ce00ccf00 0x7f2ce01577e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37386/0 (socket says 172.21.15.151:37386) 2024-09-16T15:40:22.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:22.390+0000 7f2cdea00640 1 -- 172.21.15.151:0/1509189173 learned_addr learned my addr 172.21.15.151:0/1509189173 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:40:22.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:22.390+0000 7f2cdfe00640 1 --2- 172.21.15.151:0/1509189173 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2ce015f5a0 0x7f2ce01f80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:22.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:22.390+0000 7f2cdf400640 1 --2- 172.21.15.151:0/1509189173 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ce00b2440 0x7f2ce014bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:22.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:22.390+0000 7f2cdea00640 1 -- 172.21.15.151:0/1509189173 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2ce015f5a0 msgr2=0x7f2ce01f80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:22.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:22.390+0000 7f2cdea00640 1 --2- 172.21.15.151:0/1509189173 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2ce015f5a0 0x7f2ce01f80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:22.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:22.390+0000 7f2cdea00640 1 -- 172.21.15.151:0/1509189173 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ce00b2440 msgr2=0x7f2ce014bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:22.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:22.390+0000 7f2cdea00640 1 --2- 172.21.15.151:0/1509189173 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ce00b2440 0x7f2ce014bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:22.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:22.390+0000 7f2cdea00640 1 -- 172.21.15.151:0/1509189173 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ce014c280 con 0x7f2ce00ccf00 2024-09-16T15:40:22.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:22.390+0000 7f2cdfe00640 1 --2- 172.21.15.151:0/1509189173 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2ce015f5a0 0x7f2ce01f80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:40:22.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:22.390+0000 7f2cdf400640 1 --2- 172.21.15.151:0/1509189173 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ce00b2440 0x7f2ce014bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:40:22.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:22.391+0000 7f2cdea00640 1 --2- 172.21.15.151:0/1509189173 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2ce00ccf00 0x7f2ce01577e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc00c490 tx=0x7f2ccc00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:22.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:22.391+0000 7f2cd7400640 1 -- 172.21.15.151:0/1509189173 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc0052d0 con 0x7f2ce00ccf00 2024-09-16T15:40:22.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:22.391+0000 7f2cd7400640 1 -- 172.21.15.151:0/1509189173 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ccc005490 con 0x7f2ce00ccf00 2024-09-16T15:40:22.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:22.391+0000 7f2ce6400640 1 -- 172.21.15.151:0/1509189173 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ce0157d20 con 0x7f2ce00ccf00 2024-09-16T15:40:22.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:22.391+0000 7f2cd7400640 1 -- 172.21.15.151:0/1509189173 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc012890 con 0x7f2ce00ccf00 2024-09-16T15:40:22.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:22.391+0000 7f2ce6400640 1 -- 172.21.15.151:0/1509189173 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ce00ced40 con 0x7f2ce00ccf00 2024-09-16T15:40:22.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:22.392+0000 7f2ce6400640 1 -- 172.21.15.151:0/1509189173 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ce006f160 con 0x7f2ce00ccf00 2024-09-16T15:40:22.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:22.393+0000 7f2cd7400640 1 -- 172.21.15.151:0/1509189173 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f2ccc012a30 con 0x7f2ce00ccf00 2024-09-16T15:40:22.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:22.394+0000 7f2cd7400640 1 --2- 172.21.15.151:0/1509189173 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2cc0078850 0x7f2cc007ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:22.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:22.394+0000 7f2cd7400640 1 -- 172.21.15.151:0/1509189173 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f2ccc09a4e0 con 0x7f2ce00ccf00 2024-09-16T15:40:22.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:22.395+0000 7f2cdf400640 1 --2- 172.21.15.151:0/1509189173 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2cc0078850 0x7f2cc007ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:22.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:22.395+0000 7f2cdf400640 1 --2- 172.21.15.151:0/1509189173 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2cc0078850 0x7f2cc007ad10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0000c00 tx=0x7f2cd0009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:22.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:22.398+0000 7f2cd7400640 1 -- 172.21.15.151:0/1509189173 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2ce014c280 con 0x7f2ce00ccf00 2024-09-16T15:40:22.624 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:22.622+0000 7f2ce6400640 1 -- 172.21.15.151:0/1509189173 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ce00acff0 con 0x7f2ce00ccf00 2024-09-16T15:40:22.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:22.622+0000 7f2cd7400640 1 -- 172.21.15.151:0/1509189173 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5232 (secure 0 0 0) 0x7f2ccc062b60 con 0x7f2ce00ccf00 2024-09-16T15:40:22.625 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:40:22.625 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":17,"btime":"2024-09-16T15:40:15:450685+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4661,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/873312529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":873312529},{"type":"v1","addr":"172.21.15.151:6834","nonce":873312529}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/2683082898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2683082898},{"type":"v1","addr":"172.21.15.167:6835","nonce":2683082898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4843,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2443370538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":2443370538},{"type":"v1","addr":"172.21.15.151:6835","nonce":2443370538}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:40:11.836645+0000","modified":"2024-09-16T15:40:14.867199+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4855},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4855":{"gid":4855,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6837/4150137107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":4150137107},{"type":"v1","addr":"172.21.15.167:6837","nonce":4150137107}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4855,"qdb_cluster":[4855]},"id":2}]} 2024-09-16T15:40:22.625 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 17 2024-09-16T15:40:22.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:22.625+0000 7f2ce6400640 1 -- 172.21.15.151:0/1509189173 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2cc0078850 msgr2=0x7f2cc007ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:22.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:22.625+0000 7f2ce6400640 1 --2- 172.21.15.151:0/1509189173 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2cc0078850 0x7f2cc007ad10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0000c00 tx=0x7f2cd0009040 comp rx=0 tx=0).stop 2024-09-16T15:40:22.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:22.625+0000 7f2ce6400640 1 -- 172.21.15.151:0/1509189173 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2ce00ccf00 msgr2=0x7f2ce01577e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:22.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:22.625+0000 7f2ce6400640 1 --2- 172.21.15.151:0/1509189173 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2ce00ccf00 0x7f2ce01577e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc00c490 tx=0x7f2ccc00c950 comp rx=0 tx=0).stop 2024-09-16T15:40:22.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:22.625+0000 7f2ce6400640 1 -- 172.21.15.151:0/1509189173 shutdown_connections 2024-09-16T15:40:22.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:22.625+0000 7f2ce6400640 1 --2- 172.21.15.151:0/1509189173 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2cc0078850 0x7f2cc007ad10 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:22.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:22.625+0000 7f2ce6400640 1 --2- 172.21.15.151:0/1509189173 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2ce015f5a0 0x7f2ce01f80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:22.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:22.625+0000 7f2ce6400640 1 --2- 172.21.15.151:0/1509189173 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2ce00ccf00 0x7f2ce01577e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:22.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:22.625+0000 7f2ce6400640 1 --2- 172.21.15.151:0/1509189173 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ce00b2440 0x7f2ce014bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:22.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:22.625+0000 7f2ce6400640 1 -- 172.21.15.151:0/1509189173 >> 172.21.15.151:0/1509189173 conn(0x7f2ce00b1950 msgr2=0x7f2ce015d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:22.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:22.625+0000 7f2ce6400640 1 -- 172.21.15.151:0/1509189173 shutdown_connections 2024-09-16T15:40:22.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:22.625+0000 7f2ce6400640 1 -- 172.21.15.151:0/1509189173 wait complete. 2024-09-16T15:40:23.046 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-16T15:40:11.836645+0000', 'modified': '2024-09-16T15:40:14.867199+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4855}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4855': {'gid': 4855, 'name': 'b', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.167:6837/4150137107', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6836', 'nonce': 4150137107}, {'type': 'v1', 'addr': '172.21.15.167:6837', 'nonce': 4150137107}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4855, 'qdb_cluster': [4855]} 2024-09-16T15:40:23.046 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-16T15:40:23.046 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-16T15:40:23.207 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.205+0000 7fa997400640 1 -- 172.21.15.151:0/2073091129 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa99015f5a0 msgr2=0x7fa9900b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:23.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.205+0000 7fa997400640 1 --2- 172.21.15.151:0/2073091129 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa99015f5a0 0x7fa9900b2440 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa9900cfbb0 tx=0x7fa98002a980 comp rx=0 tx=0).stop 2024-09-16T15:40:23.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.205+0000 7fa997400640 1 -- 172.21.15.151:0/2073091129 shutdown_connections 2024-09-16T15:40:23.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.205+0000 7fa997400640 1 --2- 172.21.15.151:0/2073091129 >> v2:172.21.15.167:3300/0 conn(0x7fa9900ccf00 0x7fa9900acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.205+0000 7fa997400640 1 --2- 172.21.15.151:0/2073091129 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa99015f5a0 0x7fa9900b2440 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.205+0000 7fa997400640 1 -- 172.21.15.151:0/2073091129 >> 172.21.15.151:0/2073091129 conn(0x7fa9900b1950 msgr2=0x7fa9900b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:23.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.205+0000 7fa997400640 1 -- 172.21.15.151:0/2073091129 shutdown_connections 2024-09-16T15:40:23.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.205+0000 7fa997400640 1 -- 172.21.15.151:0/2073091129 wait complete. 2024-09-16T15:40:23.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.207+0000 7fa997400640 1 Processor -- start 2024-09-16T15:40:23.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.207+0000 7fa997400640 1 -- start start 2024-09-16T15:40:23.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.208+0000 7fa997400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9900ad270 0x7fa9900b31f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:23.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.208+0000 7fa997400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9900ccf00 0x7fa990150090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:23.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.208+0000 7fa997400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa99015f5a0 0x7fa9901f8320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:23.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.208+0000 7fa997400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa99006f160 con 0x7fa99015f5a0 2024-09-16T15:40:23.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.208+0000 7fa997400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa99006f2e0 con 0x7fa9900ccf00 2024-09-16T15:40:23.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.208+0000 7fa997400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fa99006f460 con 0x7fa9900ad270 2024-09-16T15:40:23.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.208+0000 7fa995400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa99015f5a0 0x7fa9901f8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:23.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.208+0000 7fa994a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9900ad270 0x7fa9900b31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:23.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.208+0000 7fa995400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa99015f5a0 0x7fa9901f8320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49352/0 (socket says 172.21.15.151:49352) 2024-09-16T15:40:23.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.208+0000 7fa995400640 1 -- 172.21.15.151:0/2530751369 learned_addr learned my addr 172.21.15.151:0/2530751369 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:40:23.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.208+0000 7fa98fe00640 1 --2- 172.21.15.151:0/2530751369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9900ccf00 0x7fa990150090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:23.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.208+0000 7fa995400640 1 -- 172.21.15.151:0/2530751369 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9900ad270 msgr2=0x7fa9900b31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:23.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.208+0000 7fa995400640 1 --2- 172.21.15.151:0/2530751369 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9900ad270 0x7fa9900b31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.208+0000 7fa995400640 1 -- 172.21.15.151:0/2530751369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9900ccf00 msgr2=0x7fa990150090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:23.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.208+0000 7fa995400640 1 --2- 172.21.15.151:0/2530751369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9900ccf00 0x7fa990150090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.208+0000 7fa995400640 1 -- 172.21.15.151:0/2530751369 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9900b3730 con 0x7fa99015f5a0 2024-09-16T15:40:23.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.209+0000 7fa995400640 1 --2- 172.21.15.151:0/2530751369 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa99015f5a0 0x7fa9901f8320 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa98800c440 tx=0x7fa98800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:23.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.209+0000 7fa98cc00640 1 -- 172.21.15.151:0/2530751369 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa988011790 con 0x7fa99015f5a0 2024-09-16T15:40:23.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.209+0000 7fa98cc00640 1 -- 172.21.15.151:0/2530751369 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa988017070 con 0x7fa99015f5a0 2024-09-16T15:40:23.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.209+0000 7fa997400640 1 -- 172.21.15.151:0/2530751369 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9900b39b0 con 0x7fa99015f5a0 2024-09-16T15:40:23.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.209+0000 7fa98cc00640 1 -- 172.21.15.151:0/2530751369 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa988011cc0 con 0x7fa99015f5a0 2024-09-16T15:40:23.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.209+0000 7fa997400640 1 -- 172.21.15.151:0/2530751369 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9900b3f70 con 0x7fa99015f5a0 2024-09-16T15:40:23.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.212+0000 7fa98cc00640 1 -- 172.21.15.151:0/2530751369 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fa98801c460 con 0x7fa99015f5a0 2024-09-16T15:40:23.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.213+0000 7fa98cc00640 1 --2- 172.21.15.151:0/2530751369 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa970078850 0x7fa97007ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:23.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.213+0000 7fa98cc00640 1 -- 172.21.15.151:0/2530751369 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fa98809a530 con 0x7fa99015f5a0 2024-09-16T15:40:23.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.213+0000 7fa99a400740 1 Processor -- start 2024-09-16T15:40:23.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.213+0000 7fa99a400740 1 -- start start 2024-09-16T15:40:23.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.213+0000 7fa994a00640 1 --2- 172.21.15.151:0/2530751369 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa970078850 0x7fa97007ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:23.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.214+0000 7fa994a00640 1 --2- 172.21.15.151:0/2530751369 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa970078850 0x7fa97007ad10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa980035000 tx=0x7fa980004ab0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:23.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.214+0000 7fa99a400740 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d1cc6758b0 0x55d1cc675c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:23.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.214+0000 7fa99a400740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1cc630a20 0x55d1cc674390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:23.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.214+0000 7fa99a400740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1cc630fa0 0x55d1cc6748d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:23.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.214+0000 7fa99a400740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d1cc5b3430 con 0x55d1cc630fa0 2024-09-16T15:40:23.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.214+0000 7fa99a400740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d1cc5a35d0 con 0x55d1cc630a20 2024-09-16T15:40:23.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.214+0000 7fa99a400740 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x55d1cc592a00 con 0x55d1cc6758b0 2024-09-16T15:40:23.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.214+0000 7fa995400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1cc630fa0 0x55d1cc6748d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:23.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.214+0000 7fa995400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1cc630fa0 0x55d1cc6748d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49368/0 (socket says 172.21.15.151:49368) 2024-09-16T15:40:23.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.214+0000 7fa995400640 1 -- 172.21.15.151:0/1422777788 learned_addr learned my addr 172.21.15.151:0/1422777788 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:40:23.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.215+0000 7fa994a00640 1 --2- 172.21.15.151:0/1422777788 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d1cc6758b0 0x55d1cc675c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:23.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.215+0000 7fa98fe00640 1 --2- 172.21.15.151:0/1422777788 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1cc630a20 0x55d1cc674390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:23.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.215+0000 7fa995400640 1 -- 172.21.15.151:0/1422777788 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d1cc6758b0 msgr2=0x55d1cc675c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:40:23.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.215+0000 7fa995400640 1 --2- 172.21.15.151:0/1422777788 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d1cc6758b0 0x55d1cc675c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.215+0000 7fa995400640 1 -- 172.21.15.151:0/1422777788 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1cc630a20 msgr2=0x55d1cc674390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:40:23.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.215+0000 7fa995400640 1 --2- 172.21.15.151:0/1422777788 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1cc630a20 0x55d1cc674390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.215+0000 7fa995400640 1 -- 172.21.15.151:0/1422777788 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1cc55b520 con 0x55d1cc630fa0 2024-09-16T15:40:23.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.215+0000 7fa98fe00640 1 --2- 172.21.15.151:0/1422777788 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1cc630a20 0x55d1cc674390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:40:23.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.215+0000 7fa995400640 1 --2- 172.21.15.151:0/1422777788 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1cc630fa0 0x55d1cc6748d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa988098990 tx=0x7fa988061f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=751c8917746867aa server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:23.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.215+0000 7fa985200640 1 -- 172.21.15.151:0/1422777788 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa988062490 con 0x55d1cc630fa0 2024-09-16T15:40:23.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.215+0000 7fa985200640 1 -- 172.21.15.151:0/1422777788 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa988066f70 con 0x55d1cc630fa0 2024-09-16T15:40:23.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.215+0000 7fa985200640 1 -- 172.21.15.151:0/1422777788 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa98809a300 con 0x55d1cc630fa0 2024-09-16T15:40:23.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.216+0000 7fa99a400740 1 -- 172.21.15.151:0/1422777788 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1cc630fa0 msgr2=0x55d1cc6748d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:23.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.216+0000 7fa99a400740 1 --2- 172.21.15.151:0/1422777788 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1cc630fa0 0x55d1cc6748d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa988098990 tx=0x7fa988061f10 comp rx=0 tx=0).stop 2024-09-16T15:40:23.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.216+0000 7fa99a400740 1 -- 172.21.15.151:0/1422777788 shutdown_connections 2024-09-16T15:40:23.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.216+0000 7fa99a400740 1 --2- 172.21.15.151:0/1422777788 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1cc630a20 0x55d1cc674390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.216+0000 7fa99a400740 1 --2- 172.21.15.151:0/1422777788 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1cc630fa0 0x55d1cc6748d0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.216+0000 7fa99a400740 1 --2- 172.21.15.151:0/1422777788 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d1cc6758b0 0x55d1cc675c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.216+0000 7fa99a400740 1 -- 172.21.15.151:0/1422777788 >> 172.21.15.151:0/1422777788 conn(0x55d1cc62bff0 msgr2=0x55d1cc62fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:23.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.216+0000 7fa99a400740 1 -- 172.21.15.151:0/1422777788 shutdown_connections 2024-09-16T15:40:23.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.216+0000 7fa99a400740 1 -- 172.21.15.151:0/1422777788 wait complete. 2024-09-16T15:40:23.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.217+0000 7fa99a400740 1 -- start start 2024-09-16T15:40:23.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.217+0000 7fa99a400740 1 Processor -- start 2024-09-16T15:40:23.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.218+0000 7fa99a400740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1cc630fa0 0x55d1cc66fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:23.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.218+0000 7fa99a400740 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d1cc6758b0 0x55d1cc658de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:23.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.218+0000 7fa99a400740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1cc659320 0x55d1cc681e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:23.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.218+0000 7fa99a400740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d1cc539330 con 0x55d1cc659320 2024-09-16T15:40:23.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.218+0000 7fa99a400740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d1cc596810 con 0x55d1cc630fa0 2024-09-16T15:40:23.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.218+0000 7fa99a400740 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x55d1cc303620 con 0x55d1cc6758b0 2024-09-16T15:40:23.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.218+0000 7fa99a400740 10 client.0 _ll_register_callbacks cb 0x55d1cc53e420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T15:40:23.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.218+0000 7fa995400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1cc659320 0x55d1cc681e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:23.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.218+0000 7fa995400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1cc659320 0x55d1cc681e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49378/0 (socket says 172.21.15.151:49378) 2024-09-16T15:40:23.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.218+0000 7fa995400640 1 -- 172.21.15.151:0/1389122809 learned_addr learned my addr 172.21.15.151:0/1389122809 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:40:23.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.218+0000 7fa994a00640 1 --2- 172.21.15.151:0/1389122809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1cc630fa0 0x55d1cc66fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:23.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.218+0000 7fa98fe00640 1 --2- 172.21.15.151:0/1389122809 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d1cc6758b0 0x55d1cc658de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:23.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.218+0000 7fa995400640 1 -- 172.21.15.151:0/1389122809 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d1cc6758b0 msgr2=0x55d1cc658de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:40:23.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.218+0000 7fa995400640 1 --2- 172.21.15.151:0/1389122809 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d1cc6758b0 0x55d1cc658de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.218+0000 7fa995400640 1 -- 172.21.15.151:0/1389122809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1cc630fa0 msgr2=0x55d1cc66fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:40:23.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.218+0000 7fa995400640 1 --2- 172.21.15.151:0/1389122809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1cc630fa0 0x55d1cc66fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.219+0000 7fa995400640 1 -- 172.21.15.151:0/1389122809 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1cc2f4850 con 0x55d1cc659320 2024-09-16T15:40:23.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.219+0000 7fa995400640 1 --2- 172.21.15.151:0/1389122809 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1cc659320 0x55d1cc681e60 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa988062190 tx=0x7fa988062c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=334bc3ce9919fd1b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:23.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.219+0000 7fa979600640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-16T15:40:23.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.219+0000 7fa979600640 1 -- 172.21.15.151:0/1389122809 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa988080020 con 0x55d1cc659320 2024-09-16T15:40:23.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.219+0000 7fa979600640 1 -- 172.21.15.151:0/1389122809 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fa98809a250 con 0x55d1cc659320 2024-09-16T15:40:23.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.219+0000 7fa99a400740 1 -- 172.21.15.151:0/1389122809 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa988066f70 con 0x55d1cc659320 2024-09-16T15:40:23.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.219+0000 7fa979600640 1 -- 172.21.15.151:0/1389122809 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fa988062490 con 0x55d1cc659320 2024-09-16T15:40:23.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.219+0000 7fa99a400740 10 client.5070 fetch_fsmap learned FSMap version 17 2024-09-16T15:40:23.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.219+0000 7fa99a400740 1 -- 172.21.15.151:0/1389122809 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55d1cc36b6b0 con 0x55d1cc659320 2024-09-16T15:40:23.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.219+0000 7fa979600640 1 -- 172.21.15.151:0/1389122809 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa988066f70 con 0x55d1cc659320 2024-09-16T15:40:23.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.220+0000 7fa979600640 1 -- 172.21.15.151:0/1389122809 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 17) ==== 3502+0+0 (secure 0 0 0) 0x7fa9880691a0 con 0x55d1cc659320 2024-09-16T15:40:23.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.220+0000 7fa99a400740 10 client.5070 fetch_fsmap finished waiting for FSMap version 17 2024-09-16T15:40:23.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.220+0000 7fa99a400740 10 client.5070 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-16T15:40:23.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.220+0000 7fa99a400740 20 client.5070 populate_metadata read hostname 'smithi151' 2024-09-16T15:40:23.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.220+0000 7fa99a400740 1 --2- 172.21.15.151:0/1389122809 >> [v2:172.21.15.167:6836/4150137107,v1:172.21.15.167:6837/4150137107] conn(0x55d1cc66eba0 0x55d1cc66ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:23.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.220+0000 7fa99a400740 4 client.5070 mds_command: new command op to 4855 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:40:23.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.220+0000 7fa99a400740 1 -- 172.21.15.151:0/1389122809 --> [v2:172.21.15.167:6836/4150137107,v1:172.21.15.167:6837/4150137107] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1cc5a5520 con 0x55d1cc66eba0 2024-09-16T15:40:23.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.220+0000 7fa994a00640 1 --2- 172.21.15.151:0/1389122809 >> [v2:172.21.15.167:6836/4150137107,v1:172.21.15.167:6837/4150137107] conn(0x55d1cc66eba0 0x55d1cc66ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:23.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.221+0000 7fa994a00640 1 --2- 172.21.15.151:0/1389122809 >> [v2:172.21.15.167:6836/4150137107,v1:172.21.15.167:6837/4150137107] conn(0x55d1cc66eba0 0x55d1cc66ef80 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ae24a25d15ba0af server_cookie=259437c335619136 in_seq=0 out_seq=0 2024-09-16T15:40:23.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.221+0000 7fa979600640 10 client.5070 ms_handle_connect on v2:172.21.15.167:6836/4150137107 2024-09-16T15:40:23.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.222+0000 7fa979600640 1 -- 172.21.15.151:0/1389122809 <== mds.0 v2:172.21.15.167:6836/4150137107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d1cc5a5520 con 0x55d1cc66eba0 2024-09-16T15:40:23.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.222+0000 7fa979600640 10 client.5070 handle_command_reply: tid=0 2024-09-16T15:40:23.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.222+0000 7fa99a400740 1 client.5070 shutdown 2024-09-16T15:40:23.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.222+0000 7fa99a400740 1 -- 172.21.15.151:0/1389122809 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1cc659320 msgr2=0x55d1cc681e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:23.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.222+0000 7fa99a400740 1 --2- 172.21.15.151:0/1389122809 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1cc659320 0x55d1cc681e60 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa988062190 tx=0x7fa988062c50 comp rx=0 tx=0).stop 2024-09-16T15:40:23.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.222+0000 7fa99a400740 1 -- 172.21.15.151:0/1389122809 shutdown_connections 2024-09-16T15:40:23.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.222+0000 7fa99a400740 1 --2- 172.21.15.151:0/1389122809 >> [v2:172.21.15.167:6836/4150137107,v1:172.21.15.167:6837/4150137107] conn(0x55d1cc66eba0 0x55d1cc66ef80 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.222+0000 7fa99a400740 1 --2- 172.21.15.151:0/1389122809 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1cc659320 0x55d1cc681e60 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.222+0000 7fa99a400740 1 --2- 172.21.15.151:0/1389122809 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d1cc6758b0 0x55d1cc658de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.223+0000 7fa99a400740 1 --2- 172.21.15.151:0/1389122809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1cc630fa0 0x55d1cc66fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.223+0000 7fa99a400740 1 -- 172.21.15.151:0/1389122809 >> 172.21.15.151:0/1389122809 conn(0x55d1cc630a20 msgr2=0x55d1cc66d630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:23.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.223+0000 7fa979600640 0 client.5070 ms_handle_reset on v2:172.21.15.167:6836/4150137107 2024-09-16T15:40:23.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.223+0000 7fa99a400740 1 -- 172.21.15.151:0/1389122809 shutdown_connections 2024-09-16T15:40:23.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.223+0000 7fa99a400740 1 -- 172.21.15.151:0/1389122809 wait complete. 2024-09-16T15:40:23.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.223+0000 7fa99a400740 20 client.5070 trim_cache size 0 max 16384 2024-09-16T15:40:23.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.242+0000 7fa99a400740 1 Processor -- start 2024-09-16T15:40:23.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.242+0000 7fa99a400740 1 -- start start 2024-09-16T15:40:23.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.243+0000 7fa99a400740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1cc630fa0 0x55d1cc659f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:23.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.243+0000 7fa99a400740 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d1cc659320 0x55d1cc677ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:23.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.243+0000 7fa99a400740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1cc6758b0 0x55d1cc699de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:23.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.243+0000 7fa99a400740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa99014cba0 con 0x55d1cc6758b0 2024-09-16T15:40:23.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.243+0000 7fa99a400740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa9901f5710 con 0x55d1cc630fa0 2024-09-16T15:40:23.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.243+0000 7fa99a400740 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fa99014c5a0 con 0x55d1cc659320 2024-09-16T15:40:23.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.243+0000 7fa995400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1cc6758b0 0x55d1cc699de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:23.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.243+0000 7fa995400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1cc6758b0 0x55d1cc699de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49382/0 (socket says 172.21.15.151:49382) 2024-09-16T15:40:23.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.243+0000 7fa995400640 1 -- 172.21.15.151:0/277632431 learned_addr learned my addr 172.21.15.151:0/277632431 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:40:23.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.243+0000 7fa995400640 1 -- 172.21.15.151:0/277632431 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d1cc659320 msgr2=0x55d1cc677ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:40:23.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.243+0000 7fa995400640 1 --2- 172.21.15.151:0/277632431 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d1cc659320 0x55d1cc677ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.243+0000 7fa994a00640 1 --2- 172.21.15.151:0/277632431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1cc630fa0 0x55d1cc659f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:23.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.243+0000 7fa995400640 1 -- 172.21.15.151:0/277632431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1cc630fa0 msgr2=0x55d1cc659f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:40:23.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.243+0000 7fa995400640 1 --2- 172.21.15.151:0/277632431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1cc630fa0 0x55d1cc659f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.243+0000 7fa995400640 1 -- 172.21.15.151:0/277632431 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1cc677c20 con 0x55d1cc6758b0 2024-09-16T15:40:23.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.244+0000 7fa995400640 1 --2- 172.21.15.151:0/277632431 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1cc6758b0 0x55d1cc699de0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa988062420 tx=0x7fa9880036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec07ff2451e3324c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:23.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.244+0000 7fa97be00640 1 -- 172.21.15.151:0/277632431 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa988080070 con 0x55d1cc6758b0 2024-09-16T15:40:23.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.244+0000 7fa97be00640 1 -- 172.21.15.151:0/277632431 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa988066f70 con 0x55d1cc6758b0 2024-09-16T15:40:23.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.244+0000 7fa97be00640 1 -- 172.21.15.151:0/277632431 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa98809a300 con 0x55d1cc6758b0 2024-09-16T15:40:23.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.244+0000 7fa99a400740 1 -- 172.21.15.151:0/277632431 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1cc6758b0 msgr2=0x55d1cc699de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:23.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.244+0000 7fa99a400740 1 --2- 172.21.15.151:0/277632431 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1cc6758b0 0x55d1cc699de0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa988062420 tx=0x7fa9880036f0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.244+0000 7fa99a400740 1 -- 172.21.15.151:0/277632431 shutdown_connections 2024-09-16T15:40:23.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.245+0000 7fa99a400740 1 --2- 172.21.15.151:0/277632431 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1cc6758b0 0x55d1cc699de0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.245+0000 7fa99a400740 1 --2- 172.21.15.151:0/277632431 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d1cc659320 0x55d1cc677ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.245+0000 7fa99a400740 1 --2- 172.21.15.151:0/277632431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1cc630fa0 0x55d1cc659f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.245+0000 7fa99a400740 1 -- 172.21.15.151:0/277632431 >> 172.21.15.151:0/277632431 conn(0x55d1cc630a20 msgr2=0x55d1cc625780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:23.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.245+0000 7fa99a400740 1 -- 172.21.15.151:0/277632431 shutdown_connections 2024-09-16T15:40:23.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.245+0000 7fa99a400740 1 -- 172.21.15.151:0/277632431 wait complete. 2024-09-16T15:40:23.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.245+0000 7fa99a400740 1 -- start start 2024-09-16T15:40:23.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.245+0000 7fa99a400740 1 Processor -- start 2024-09-16T15:40:23.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.246+0000 7fa99a400740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1cc630fa0 0x55d1cc6f9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:23.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.246+0000 7fa99a400740 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d1cc659320 0x55d1cc66d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:23.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.246+0000 7fa99a400740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1cc6758b0 0x55d1cc6f5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:23.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.246+0000 7fa99a400740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d1cc539330 con 0x55d1cc630fa0 2024-09-16T15:40:23.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.246+0000 7fa99a400740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d1cc596810 con 0x55d1cc6758b0 2024-09-16T15:40:23.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.246+0000 7fa99a400740 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x55d1cc672930 con 0x55d1cc659320 2024-09-16T15:40:23.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.246+0000 7fa99a400740 10 client.0 _ll_register_callbacks cb 0x7fa988061de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T15:40:23.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.247+0000 7fa994a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1cc630fa0 0x55d1cc6f9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:23.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.247+0000 7fa994a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1cc630fa0 0x55d1cc6f9880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49386/0 (socket says 172.21.15.151:49386) 2024-09-16T15:40:23.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.247+0000 7fa994a00640 1 -- 172.21.15.151:0/3553389972 learned_addr learned my addr 172.21.15.151:0/3553389972 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:40:23.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.247+0000 7fa994a00640 1 -- 172.21.15.151:0/3553389972 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d1cc659320 msgr2=0x55d1cc66d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:40:23.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.247+0000 7fa994a00640 1 --2- 172.21.15.151:0/3553389972 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d1cc659320 0x55d1cc66d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.247+0000 7fa994a00640 1 -- 172.21.15.151:0/3553389972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1cc6758b0 msgr2=0x55d1cc6f5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:40:23.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.247+0000 7fa994a00640 1 --2- 172.21.15.151:0/3553389972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1cc6758b0 0x55d1cc6f5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.247+0000 7fa994a00640 1 -- 172.21.15.151:0/3553389972 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1cc62b500 con 0x55d1cc630fa0 2024-09-16T15:40:23.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.247+0000 7fa994a00640 1 --2- 172.21.15.151:0/3553389972 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1cc630fa0 0x55d1cc6f9880 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa980009430 tx=0x7fa9800095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41bdcbef20398a1e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:23.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.247+0000 7fa979600640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-16T15:40:23.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.248+0000 7fa979600640 1 -- 172.21.15.151:0/3553389972 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa980034040 con 0x55d1cc630fa0 2024-09-16T15:40:23.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.248+0000 7fa979600640 1 -- 172.21.15.151:0/3553389972 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fa980040030 con 0x55d1cc630fa0 2024-09-16T15:40:23.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.248+0000 7fa99a400740 1 -- 172.21.15.151:0/3553389972 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa988066f70 con 0x55d1cc630fa0 2024-09-16T15:40:23.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.248+0000 7fa979600640 1 -- 172.21.15.151:0/3553389972 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fa9800498d0 con 0x55d1cc630fa0 2024-09-16T15:40:23.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.248+0000 7fa99a400740 10 client.5076 fetch_fsmap learned FSMap version 17 2024-09-16T15:40:23.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.248+0000 7fa99a400740 1 -- 172.21.15.151:0/3553389972 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55d1cc674770 con 0x55d1cc630fa0 2024-09-16T15:40:23.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.248+0000 7fa979600640 1 -- 172.21.15.151:0/3553389972 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa988066f70 con 0x55d1cc630fa0 2024-09-16T15:40:23.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.248+0000 7fa979600640 1 -- 172.21.15.151:0/3553389972 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 17) ==== 3502+0+0 (secure 0 0 0) 0x7fa980049a70 con 0x55d1cc630fa0 2024-09-16T15:40:23.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.248+0000 7fa99a400740 10 client.5076 fetch_fsmap finished waiting for FSMap version 17 2024-09-16T15:40:23.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.248+0000 7fa99a400740 10 client.5076 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-16T15:40:23.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.248+0000 7fa99a400740 20 client.5076 populate_metadata read hostname 'smithi151' 2024-09-16T15:40:23.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.248+0000 7fa99a400740 1 --2- 172.21.15.151:0/3553389972 >> [v2:172.21.15.167:6836/4150137107,v1:172.21.15.167:6837/4150137107] conn(0x55d1cc680ed0 0x55d1cc645e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:23.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.248+0000 7fa99a400740 4 client.5076 mds_command: new command op to 4855 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T15:40:23.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.248+0000 7fa99a400740 1 -- 172.21.15.151:0/3553389972 --> [v2:172.21.15.167:6836/4150137107,v1:172.21.15.167:6837/4150137107] -- command(tid 0: {"prefix": "status"}) -- 0x55d1cc5a0fd0 con 0x55d1cc680ed0 2024-09-16T15:40:23.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.249+0000 7fa98fe00640 1 --2- 172.21.15.151:0/3553389972 >> [v2:172.21.15.167:6836/4150137107,v1:172.21.15.167:6837/4150137107] conn(0x55d1cc680ed0 0x55d1cc645e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:23.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.249+0000 7fa98fe00640 1 --2- 172.21.15.151:0/3553389972 >> [v2:172.21.15.167:6836/4150137107,v1:172.21.15.167:6837/4150137107] conn(0x55d1cc680ed0 0x55d1cc645e00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f189710726d463 server_cookie=699c0b9a610d1ebc in_seq=0 out_seq=0 2024-09-16T15:40:23.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.249+0000 7fa979600640 10 client.5076 ms_handle_connect on v2:172.21.15.167:6836/4150137107 2024-09-16T15:40:23.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.250+0000 7fa979600640 1 -- 172.21.15.151:0/3553389972 <== mds.0 v2:172.21.15.167:6836/4150137107 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55d1cc5a0fd0 con 0x55d1cc680ed0 2024-09-16T15:40:23.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.250+0000 7fa979600640 10 client.5076 handle_command_reply: tid=0 2024-09-16T15:40:23.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.250+0000 7fa99a400740 1 client.5076 shutdown 2024-09-16T15:40:23.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.250+0000 7fa99a400740 1 -- 172.21.15.151:0/3553389972 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1cc630fa0 msgr2=0x55d1cc6f9880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:23.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.250+0000 7fa99a400740 1 --2- 172.21.15.151:0/3553389972 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1cc630fa0 0x55d1cc6f9880 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa980009430 tx=0x7fa9800095c0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.250+0000 7fa99a400740 1 -- 172.21.15.151:0/3553389972 shutdown_connections 2024-09-16T15:40:23.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.250+0000 7fa99a400740 1 --2- 172.21.15.151:0/3553389972 >> [v2:172.21.15.167:6836/4150137107,v1:172.21.15.167:6837/4150137107] conn(0x55d1cc680ed0 0x55d1cc645e00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.250+0000 7fa99a400740 1 --2- 172.21.15.151:0/3553389972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1cc6758b0 0x55d1cc6f5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.250+0000 7fa99a400740 1 --2- 172.21.15.151:0/3553389972 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d1cc659320 0x55d1cc66d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.250+0000 7fa99a400740 1 --2- 172.21.15.151:0/3553389972 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1cc630fa0 0x55d1cc6f9880 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.250+0000 7fa979600640 0 client.5076 ms_handle_reset on v2:172.21.15.167:6836/4150137107 2024-09-16T15:40:23.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.250+0000 7fa99a400740 1 -- 172.21.15.151:0/3553389972 >> 172.21.15.151:0/3553389972 conn(0x55d1cc630a20 msgr2=0x55d1cc62a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:23.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.250+0000 7fa99a400740 1 -- 172.21.15.151:0/3553389972 shutdown_connections 2024-09-16T15:40:23.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.250+0000 7fa99a400740 1 -- 172.21.15.151:0/3553389972 wait complete. 2024-09-16T15:40:23.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.250+0000 7fa99a400740 20 client.5076 trim_cache size 0 max 16384 2024-09-16T15:40:23.256 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-16T15:40:23.256 INFO:teuthology.orchestra.run.smithi151.stdout: "cluster_fsid": "1757be18-e41a-4446-bb4f-ee4ac81c39df", 2024-09-16T15:40:23.256 INFO:teuthology.orchestra.run.smithi151.stdout: "whoami": 0, 2024-09-16T15:40:23.256 INFO:teuthology.orchestra.run.smithi151.stdout: "id": 4855, 2024-09-16T15:40:23.256 INFO:teuthology.orchestra.run.smithi151.stdout: "want_state": "up:active", 2024-09-16T15:40:23.256 INFO:teuthology.orchestra.run.smithi151.stdout: "state": "up:active", 2024-09-16T15:40:23.256 INFO:teuthology.orchestra.run.smithi151.stdout: "fs_name": "cephfs", 2024-09-16T15:40:23.256 INFO:teuthology.orchestra.run.smithi151.stdout: "rank_uptime": 10.819092161, 2024-09-16T15:40:23.256 INFO:teuthology.orchestra.run.smithi151.stdout: "mdsmap_epoch": 17, 2024-09-16T15:40:23.256 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_epoch": 35, 2024-09-16T15:40:23.256 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_epoch_barrier": 35, 2024-09-16T15:40:23.256 INFO:teuthology.orchestra.run.smithi151.stdout: "uptime": 17.757098121999999 2024-09-16T15:40:23.256 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-16T15:40:23.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.251+0000 7fa997400640 1 -- 172.21.15.151:0/2530751369 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa970078850 msgr2=0x7fa97007ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:23.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.251+0000 7fa997400640 1 --2- 172.21.15.151:0/2530751369 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa970078850 0x7fa97007ad10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa980035000 tx=0x7fa980004ab0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.251+0000 7fa997400640 1 -- 172.21.15.151:0/2530751369 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa99015f5a0 msgr2=0x7fa9901f8320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:23.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.251+0000 7fa997400640 1 --2- 172.21.15.151:0/2530751369 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa99015f5a0 0x7fa9901f8320 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa98800c440 tx=0x7fa98800c840 comp rx=0 tx=0).stop 2024-09-16T15:40:23.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.251+0000 7fa997400640 1 -- 172.21.15.151:0/2530751369 shutdown_connections 2024-09-16T15:40:23.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.251+0000 7fa997400640 1 --2- 172.21.15.151:0/2530751369 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa970078850 0x7fa97007ad10 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.251+0000 7fa997400640 1 --2- 172.21.15.151:0/2530751369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9900ccf00 0x7fa990150090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.251+0000 7fa997400640 1 --2- 172.21.15.151:0/2530751369 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa99015f5a0 0x7fa9901f8320 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.251+0000 7fa997400640 1 --2- 172.21.15.151:0/2530751369 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9900ad270 0x7fa9900b31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.251+0000 7fa997400640 1 -- 172.21.15.151:0/2530751369 >> 172.21.15.151:0/2530751369 conn(0x7fa9900b1950 msgr2=0x7fa99009bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:23.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.251+0000 7fa997400640 1 -- 172.21.15.151:0/2530751369 shutdown_connections 2024-09-16T15:40:23.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.251+0000 7fa997400640 1 -- 172.21.15.151:0/2530751369 wait complete. 2024-09-16T15:40:23.268 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-16T15:40:23.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.424+0000 7eff30600640 1 -- 172.21.15.151:0/351244830 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff2815f5a0 msgr2=0x7eff280b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:23.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.424+0000 7eff30600640 1 --2- 172.21.15.151:0/351244830 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff2815f5a0 0x7eff280b2440 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7eff280cfbb0 tx=0x7eff1c02a4a0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.424+0000 7eff30600640 1 -- 172.21.15.151:0/351244830 shutdown_connections 2024-09-16T15:40:23.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.424+0000 7eff30600640 1 --2- 172.21.15.151:0/351244830 >> v2:172.21.15.167:3300/0 conn(0x7eff280ccf00 0x7eff280acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.424+0000 7eff30600640 1 --2- 172.21.15.151:0/351244830 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff2815f5a0 0x7eff280b2440 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.424+0000 7eff30600640 1 -- 172.21.15.151:0/351244830 >> 172.21.15.151:0/351244830 conn(0x7eff280b1950 msgr2=0x7eff280b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:23.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.424+0000 7eff30600640 1 -- 172.21.15.151:0/351244830 shutdown_connections 2024-09-16T15:40:23.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.424+0000 7eff30600640 1 -- 172.21.15.151:0/351244830 wait complete. 2024-09-16T15:40:23.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.426+0000 7eff30600640 1 Processor -- start 2024-09-16T15:40:23.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.426+0000 7eff30600640 1 -- start start 2024-09-16T15:40:23.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.427+0000 7eff30600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7eff280ad270 0x7eff2814c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:23.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.427+0000 7eff30600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff280ccf00 0x7eff28152220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:23.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.427+0000 7eff30600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff2815f5a0 0x7eff28152760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:23.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.427+0000 7eff30600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7eff2806f160 con 0x7eff2815f5a0 2024-09-16T15:40:23.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.427+0000 7eff30600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7eff2806f2e0 con 0x7eff280ccf00 2024-09-16T15:40:23.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.427+0000 7eff30600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7eff2806f460 con 0x7eff280ad270 2024-09-16T15:40:23.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.427+0000 7eff2e600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff2815f5a0 0x7eff28152760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:23.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.427+0000 7eff2e600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff2815f5a0 0x7eff28152760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49400/0 (socket says 172.21.15.151:49400) 2024-09-16T15:40:23.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.427+0000 7eff2e600640 1 -- 172.21.15.151:0/302884914 learned_addr learned my addr 172.21.15.151:0/302884914 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:40:23.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.427+0000 7eff2dc00640 1 --2- 172.21.15.151:0/302884914 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7eff280ad270 0x7eff2814c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:23.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.427+0000 7eff2d200640 1 --2- 172.21.15.151:0/302884914 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff280ccf00 0x7eff28152220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:23.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.428+0000 7eff2e600640 1 -- 172.21.15.151:0/302884914 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7eff280ad270 msgr2=0x7eff2814c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:23.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.428+0000 7eff2e600640 1 --2- 172.21.15.151:0/302884914 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7eff280ad270 0x7eff2814c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.428+0000 7eff2e600640 1 -- 172.21.15.151:0/302884914 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff280ccf00 msgr2=0x7eff28152220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:23.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.428+0000 7eff2e600640 1 --2- 172.21.15.151:0/302884914 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff280ccf00 0x7eff28152220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.428+0000 7eff2e600640 1 -- 172.21.15.151:0/302884914 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff281f2c40 con 0x7eff2815f5a0 2024-09-16T15:40:23.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.428+0000 7eff2e600640 1 --2- 172.21.15.151:0/302884914 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff2815f5a0 0x7eff28152760 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7eff2000c770 tx=0x7eff2000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:23.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.428+0000 7eff25600640 1 -- 172.21.15.151:0/302884914 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff20003da0 con 0x7eff2815f5a0 2024-09-16T15:40:23.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.428+0000 7eff25600640 1 -- 172.21.15.151:0/302884914 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7eff20005070 con 0x7eff2815f5a0 2024-09-16T15:40:23.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.428+0000 7eff25600640 1 -- 172.21.15.151:0/302884914 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff20005500 con 0x7eff2815f5a0 2024-09-16T15:40:23.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.428+0000 7eff30600640 1 -- 172.21.15.151:0/302884914 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff28150990 con 0x7eff2815f5a0 2024-09-16T15:40:23.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.428+0000 7eff30600640 1 -- 172.21.15.151:0/302884914 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff281f2ee0 con 0x7eff2815f5a0 2024-09-16T15:40:23.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.429+0000 7eff30600640 1 -- 172.21.15.151:0/302884914 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff2806f9f0 con 0x7eff2815f5a0 2024-09-16T15:40:23.433 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.431+0000 7eff25600640 1 -- 172.21.15.151:0/302884914 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7eff200056a0 con 0x7eff2815f5a0 2024-09-16T15:40:23.434 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.432+0000 7eff25600640 1 --2- 172.21.15.151:0/302884914 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7eff08078850 0x7eff0807ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:23.434 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.432+0000 7eff25600640 1 -- 172.21.15.151:0/302884914 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7eff2009a670 con 0x7eff2815f5a0 2024-09-16T15:40:23.434 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.432+0000 7eff2dc00640 1 --2- 172.21.15.151:0/302884914 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7eff08078850 0x7eff0807ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:23.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.433+0000 7eff2dc00640 1 --2- 172.21.15.151:0/302884914 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7eff08078850 0x7eff0807ad10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7eff1c034000 tx=0x7eff1c004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:23.441 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.437+0000 7eff25600640 1 -- 172.21.15.151:0/302884914 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7eff20062690 con 0x7eff2815f5a0 2024-09-16T15:40:23.660 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.658+0000 7eff30600640 1 -- 172.21.15.151:0/302884914 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7eff2815e060 con 0x7eff2815f5a0 2024-09-16T15:40:23.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.659+0000 7eff25600640 1 -- 172.21.15.151:0/302884914 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7eff200670c0 con 0x7eff2815f5a0 2024-09-16T15:40:23.661 INFO:teuthology.orchestra.run.smithi151.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-16T15:40:23.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.661+0000 7eff30600640 1 -- 172.21.15.151:0/302884914 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7eff08078850 msgr2=0x7eff0807ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:23.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.661+0000 7eff30600640 1 --2- 172.21.15.151:0/302884914 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7eff08078850 0x7eff0807ad10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7eff1c034000 tx=0x7eff1c004730 comp rx=0 tx=0).stop 2024-09-16T15:40:23.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.661+0000 7eff30600640 1 -- 172.21.15.151:0/302884914 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff2815f5a0 msgr2=0x7eff28152760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:23.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.661+0000 7eff30600640 1 --2- 172.21.15.151:0/302884914 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff2815f5a0 0x7eff28152760 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7eff2000c770 tx=0x7eff2000cc30 comp rx=0 tx=0).stop 2024-09-16T15:40:23.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.661+0000 7eff30600640 1 -- 172.21.15.151:0/302884914 shutdown_connections 2024-09-16T15:40:23.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.661+0000 7eff30600640 1 --2- 172.21.15.151:0/302884914 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7eff08078850 0x7eff0807ad10 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.661+0000 7eff30600640 1 --2- 172.21.15.151:0/302884914 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff280ccf00 0x7eff28152220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.661+0000 7eff30600640 1 --2- 172.21.15.151:0/302884914 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff2815f5a0 0x7eff28152760 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.661+0000 7eff30600640 1 --2- 172.21.15.151:0/302884914 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7eff280ad270 0x7eff2814c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:23.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.661+0000 7eff30600640 1 -- 172.21.15.151:0/302884914 >> 172.21.15.151:0/302884914 conn(0x7eff280b1950 msgr2=0x7eff2815e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:23.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.661+0000 7eff30600640 1 -- 172.21.15.151:0/302884914 shutdown_connections 2024-09-16T15:40:23.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23.661+0000 7eff30600640 1 -- 172.21.15.151:0/302884914 wait complete. 2024-09-16T15:40:23.679 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-16T15:40:23.679 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T15:40:23.679 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi151.front.sepia.ceph.com 2024-09-16T15:40:23.679 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-16T15:40:23.679 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-16T15:40:23.679 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-16T15:40:23.679 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-16T15:40:23.679 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T15:40:23.679 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T15:40:23.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:40:23.679 DEBUG:teuthology.orchestra.run.smithi151:> ip addr 2024-09-16T15:40:23.714 INFO:teuthology.orchestra.run.smithi151.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T15:40:23.715 INFO:teuthology.orchestra.run.smithi151.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T15:40:23.715 INFO:teuthology.orchestra.run.smithi151.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T15:40:23.715 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:40:23.715 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 ::1/128 scope host 2024-09-16T15:40:23.715 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:40:23.715 INFO:teuthology.orchestra.run.smithi151.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:40:23.715 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:da:6e:0c brd ff:ff:ff:ff:ff:ff 2024-09-16T15:40:23.715 INFO:teuthology.orchestra.run.smithi151.stdout: altname enp7s0f0 2024-09-16T15:40:23.715 INFO:teuthology.orchestra.run.smithi151.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:40:23.715 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:da:6e:0d brd ff:ff:ff:ff:ff:ff 2024-09-16T15:40:23.715 INFO:teuthology.orchestra.run.smithi151.stdout: altname enp7s0f1 2024-09-16T15:40:23.715 INFO:teuthology.orchestra.run.smithi151.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:40:23.716 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:88:6f:62 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:40:23.716 INFO:teuthology.orchestra.run.smithi151.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T15:40:23.716 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:88:6f:63 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:40:23.716 INFO:teuthology.orchestra.run.smithi151.stdout: inet 172.21.15.151/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-16T15:40:23.716 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft 42666sec preferred_lft 42666sec 2024-09-16T15:40:23.716 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 fe80::ec4:7aff:fe88:6f63/64 scope link noprefixroute 2024-09-16T15:40:23.716 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:40:23.716 INFO:teuthology.orchestra.run.smithi151.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-16T15:40:23.716 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 62:3f:3b:48:ad:f8 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:40:23.716 INFO:teuthology.orchestra.run.smithi151.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-16T15:40:23.716 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:40:23.716 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 fe80::40fa:7fff:fedf:2a00/64 scope link 2024-09-16T15:40:23.716 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:40:23.717 INFO:teuthology.orchestra.run.smithi151.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T15:40:23.717 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 62:3f:3b:48:ad:f8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T15:40:23.717 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 fe80::603f:3bff:fe48:adf8/64 scope link 2024-09-16T15:40:23.717 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:40:23.717 INFO:teuthology.orchestra.run.smithi151.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T15:40:23.717 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether b6:ac:fe:34:a7:66 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-16T15:40:23.717 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 fe80::b4ac:feff:fe34:a766/64 scope link 2024-09-16T15:40:23.717 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:40:23.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:40:23.718 DEBUG:teuthology.orchestra.run.smithi151:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T15:40:23.811 INFO:teuthology.orchestra.run.smithi151.stdout:1 2024-09-16T15:40:23.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:40:23.814 DEBUG:teuthology.orchestra.run.smithi151:> route 2024-09-16T15:40:23.850 INFO:teuthology.orchestra.run.smithi151.stdout:Kernel IP routing table 2024-09-16T15:40:23.851 INFO:teuthology.orchestra.run.smithi151.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T15:40:23.851 INFO:teuthology.orchestra.run.smithi151.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-16T15:40:23.851 INFO:teuthology.orchestra.run.smithi151.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-16T15:40:23.851 INFO:teuthology.orchestra.run.smithi151.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T15:40:23.851 DEBUG:teuthology.orchestra.run.smithi151:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T15:40:23.851 DEBUG:teuthology.orchestra.run.smithi151:> set -e 2024-09-16T15:40:23.851 DEBUG:teuthology.orchestra.run.smithi151:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-16T15:40:23.851 DEBUG:teuthology.orchestra.run.smithi151:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-16T15:40:23.851 DEBUG:teuthology.orchestra.run.smithi151:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-16T15:40:23.851 DEBUG:teuthology.orchestra.run.smithi151:> ') 2024-09-16T15:40:23.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:40:24.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:40:24.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:40:24.047 DEBUG:teuthology.orchestra.run.smithi151:> ip netns list 2024-09-16T15:40:24.082 INFO:teuthology.orchestra.run.smithi151.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-16T15:40:24.082 INFO:teuthology.orchestra.run.smithi151.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-16T15:40:24.083 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T15:40:24.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:40:24.083 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip link set brx.0 up 2024-09-16T15:40:24.180 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:40:24.180 DEBUG:teuthology.orchestra.run.smithi151:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-16T15:40:24.216 INFO:teuthology.orchestra.run.smithi151.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-16T15:40:24.216 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:40:24.216 DEBUG:teuthology.orchestra.run.smithi151:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-16T15:40:24.290 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-16T15:40:24.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:40:24.291 DEBUG:teuthology.orchestra.run.smithi151:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-16T15:40:24.419 INFO:teuthology.orchestra.run.smithi151.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-16T15:40:24.420 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-16T15:40:24.420 INFO:teuthology.orchestra.run.smithi151.stdout:invalid new device string format 2024-09-16T15:40:24.436 INFO:teuthology.orchestra.run.smithi151.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-16T15:40:24.436 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-16T15:40:24.436 INFO:teuthology.orchestra.run.smithi151.stdout:invalid new device string format 2024-09-16T15:40:24.436 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: resolved to: "172.21.15.151:3300,172.21.15.167:3300,172.21.15.167:3301" 2024-09-16T15:40:24.436 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-16T15:40:24.461 INFO:teuthology.orchestra.run.smithi151.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-16T15:40:24.461 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-16T15:40:24.461 INFO:teuthology.orchestra.run.smithi151.stdout:invalid new device string format 2024-09-16T15:40:24.461 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: resolved to: "172.21.15.151:3300,172.21.15.167:3300,172.21.15.167:3301" 2024-09-16T15:40:24.462 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-16T15:40:24.462 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: resolved to: "172.21.15.151:6789,172.21.15.167:6789,172.21.15.167:6790" 2024-09-16T15:40:24.462 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.151:6789,172.21.15.167:6789,172.21.15.167:6790:/ 2024-09-16T15:40:24.462 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=1757be18-e41a-4446-bb4f-ee4ac81c39df" will pass to kernel 2024-09-16T15:40:24.463 INFO:teuthology.orchestra.run.smithi151.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-16T15:40:24.463 INFO:teuthology.orchestra.run.smithi151.stdout: You just mounted a file system that supports labels which does not 2024-09-16T15:40:24.463 INFO:teuthology.orchestra.run.smithi151.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-16T15:40:24.463 INFO:teuthology.orchestra.run.smithi151.stdout: applications will generate AVC messages and not be allowed access to 2024-09-16T15:40:24.463 INFO:teuthology.orchestra.run.smithi151.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-16T15:40:24.466 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-16T15:40:24.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:40:24.466 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-16T15:40:24.524 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T15:40:24.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:24 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:40:24.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:40:24.625 INFO:teuthology.orchestra.run.smithi151.stdout:client5088 2024-09-16T15:40:24.626 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:40:24.626 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:40:24.662 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-16T15:40:24.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:24.861+0000 7f12b9200640 1 -- 172.21.15.151:0/1767327204 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12b406f220 msgr2=0x7f12b406f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:24.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:24.861+0000 7f12b9200640 1 --2- 172.21.15.151:0/1767327204 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12b406f220 0x7f12b406f6d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f12ac005fd0 tx=0x7f12ac02aa50 comp rx=0 tx=0).stop 2024-09-16T15:40:24.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:24.862+0000 7f12b9200640 1 -- 172.21.15.151:0/1767327204 shutdown_connections 2024-09-16T15:40:24.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:24.862+0000 7f12b9200640 1 --2- 172.21.15.151:0/1767327204 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12b406f220 0x7f12b406f6d0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:24.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:24.862+0000 7f12b9200640 1 --2- 172.21.15.151:0/1767327204 >> v2:172.21.15.167:3300/0 conn(0x7f12b415cff0 0x7f12b406e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:24.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:24.862+0000 7f12b9200640 1 --2- 172.21.15.151:0/1767327204 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f12b4159aa0 0x7f12b40741a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:24.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:24.862+0000 7f12b9200640 1 -- 172.21.15.151:0/1767327204 >> 172.21.15.151:0/1767327204 conn(0x7f12b40b18e0 msgr2=0x7f12b40b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:24.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:24.862+0000 7f12b9200640 1 -- 172.21.15.151:0/1767327204 shutdown_connections 2024-09-16T15:40:24.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:24.862+0000 7f12b9200640 1 -- 172.21.15.151:0/1767327204 wait complete. 2024-09-16T15:40:24.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:24.863+0000 7f12b9200640 1 Processor -- start 2024-09-16T15:40:24.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:24.864+0000 7f12b9200640 1 -- start start 2024-09-16T15:40:24.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:24.864+0000 7f12b9200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12b406f220 0x7f12b40ae4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:24.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:24.864+0000 7f12b9200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f12b4159aa0 0x7f12b4153c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:24.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:24.864+0000 7f12b9200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12b415cff0 0x7f12b40ad5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:24.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:24.864+0000 7f12b9200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f12b40cc310 con 0x7f12b415cff0 2024-09-16T15:40:24.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:24.864+0000 7f12b9200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f12b40cc190 con 0x7f12b406f220 2024-09-16T15:40:24.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:24.864+0000 7f12b9200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f12b40cc490 con 0x7f12b4159aa0 2024-09-16T15:40:24.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:24.865+0000 7f12b2800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12b415cff0 0x7f12b40ad5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:24.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:24.865+0000 7f12b2800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12b415cff0 0x7f12b40ad5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49422/0 (socket says 172.21.15.151:49422) 2024-09-16T15:40:24.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:24.865+0000 7f12b2800640 1 -- 172.21.15.151:0/1389125348 learned_addr learned my addr 172.21.15.151:0/1389125348 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:40:24.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:24.865+0000 7f12b1400640 1 --2- 172.21.15.151:0/1389125348 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f12b4159aa0 0x7f12b4153c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:24.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:24.865+0000 7f12b1e00640 1 --2- 172.21.15.151:0/1389125348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12b406f220 0x7f12b40ae4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:24.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:24.865+0000 7f12b2800640 1 -- 172.21.15.151:0/1389125348 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f12b4159aa0 msgr2=0x7f12b4153c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:24.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:24.865+0000 7f12b2800640 1 --2- 172.21.15.151:0/1389125348 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f12b4159aa0 0x7f12b4153c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:24.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:24.865+0000 7f12b2800640 1 -- 172.21.15.151:0/1389125348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12b406f220 msgr2=0x7f12b40ae4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:24.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:24.865+0000 7f12b2800640 1 --2- 172.21.15.151:0/1389125348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12b406f220 0x7f12b40ae4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:24.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:24.865+0000 7f12b2800640 1 -- 172.21.15.151:0/1389125348 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12b40aea00 con 0x7f12b415cff0 2024-09-16T15:40:24.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:24.865+0000 7f12b2800640 1 --2- 172.21.15.151:0/1389125348 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12b415cff0 0x7f12b40ad5d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f12ac006010 tx=0x7f12ac003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:24.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:24.866+0000 7f12aa000640 1 -- 172.21.15.151:0/1389125348 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ac047070 con 0x7f12b415cff0 2024-09-16T15:40:24.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:24.866+0000 7f12aa000640 1 -- 172.21.15.151:0/1389125348 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f12ac004250 con 0x7f12b415cff0 2024-09-16T15:40:24.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:24.866+0000 7f12aa000640 1 -- 172.21.15.151:0/1389125348 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ac033050 con 0x7f12b415cff0 2024-09-16T15:40:24.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:24.866+0000 7f12b9200640 1 -- 172.21.15.151:0/1389125348 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12b40aeca0 con 0x7f12b415cff0 2024-09-16T15:40:24.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:24.866+0000 7f12b9200640 1 -- 172.21.15.151:0/1389125348 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12b40af260 con 0x7f12b415cff0 2024-09-16T15:40:24.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:24.867+0000 7f12b9200640 1 -- 172.21.15.151:0/1389125348 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12b40741a0 con 0x7f12b415cff0 2024-09-16T15:40:24.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:24.868+0000 7f12aa000640 1 -- 172.21.15.151:0/1389125348 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f12ac04a070 con 0x7f12b415cff0 2024-09-16T15:40:24.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:24.869+0000 7f12aa000640 1 --2- 172.21.15.151:0/1389125348 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1294078880 0x7f129407ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:24.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:24.869+0000 7f12aa000640 1 -- 172.21.15.151:0/1389125348 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f12ac0bbbd0 con 0x7f12b415cff0 2024-09-16T15:40:24.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:24.869+0000 7f12b1e00640 1 --2- 172.21.15.151:0/1389125348 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1294078880 0x7f129407ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:24.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:24.870+0000 7f12b1e00640 1 --2- 172.21.15.151:0/1389125348 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1294078880 0x7f129407ad40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f12a4005e00 tx=0x7f12a4005c50 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:24.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:24.873+0000 7f12aa000640 1 -- 172.21.15.151:0/1389125348 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f12ac0841a0 con 0x7f12b415cff0 2024-09-16T15:40:25.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:25.094+0000 7f12b9200640 1 -- 172.21.15.151:0/1389125348 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f12b40b20d0 con 0x7f12b415cff0 2024-09-16T15:40:25.488 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:25.486+0000 7f12aa000640 1 -- 172.21.15.151:0/1389125348 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v18) ==== 136+0+0 (secure 0 0 0) 0x7f12ac083aa0 con 0x7f12b415cff0 2024-09-16T15:40:25.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:25.489+0000 7f12b9200640 1 -- 172.21.15.151:0/1389125348 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1294078880 msgr2=0x7f129407ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:25.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:25.489+0000 7f12b9200640 1 --2- 172.21.15.151:0/1389125348 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1294078880 0x7f129407ad40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f12a4005e00 tx=0x7f12a4005c50 comp rx=0 tx=0).stop 2024-09-16T15:40:25.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:25.489+0000 7f12b9200640 1 -- 172.21.15.151:0/1389125348 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12b415cff0 msgr2=0x7f12b40ad5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:25.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:25.489+0000 7f12b9200640 1 --2- 172.21.15.151:0/1389125348 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12b415cff0 0x7f12b40ad5d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f12ac006010 tx=0x7f12ac003f10 comp rx=0 tx=0).stop 2024-09-16T15:40:25.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:25.489+0000 7f12b9200640 1 -- 172.21.15.151:0/1389125348 shutdown_connections 2024-09-16T15:40:25.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:25.489+0000 7f12b9200640 1 --2- 172.21.15.151:0/1389125348 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1294078880 0x7f129407ad40 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:25.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:25.489+0000 7f12b9200640 1 --2- 172.21.15.151:0/1389125348 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12b415cff0 0x7f12b40ad5d0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:25.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:25.489+0000 7f12b9200640 1 --2- 172.21.15.151:0/1389125348 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f12b4159aa0 0x7f12b4153c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:25.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:25.489+0000 7f12b9200640 1 --2- 172.21.15.151:0/1389125348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12b406f220 0x7f12b40ae4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:25.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:25.489+0000 7f12b9200640 1 -- 172.21.15.151:0/1389125348 >> 172.21.15.151:0/1389125348 conn(0x7f12b40b18e0 msgr2=0x7f12b4155f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:25.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:25.489+0000 7f12b9200640 1 -- 172.21.15.151:0/1389125348 shutdown_connections 2024-09-16T15:40:25.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:25.489+0000 7f12b9200640 1 -- 172.21.15.151:0/1389125348 wait complete. 2024-09-16T15:40:25.509 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-16T15:40:25.668 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:25.666+0000 7f8d7b200640 1 -- 172.21.15.151:0/2135693707 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8d7c0725d0 msgr2=0x7f8d7c073d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:25.668 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:25.666+0000 7f8d7b200640 1 --2- 172.21.15.151:0/2135693707 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8d7c0725d0 0x7f8d7c073d40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8d68005fd0 tx=0x7f8d6802a540 comp rx=0 tx=0).stop 2024-09-16T15:40:25.668 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:25.666+0000 7f8d7b200640 1 -- 172.21.15.151:0/2135693707 shutdown_connections 2024-09-16T15:40:25.668 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:25.666+0000 7f8d7b200640 1 --2- 172.21.15.151:0/2135693707 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8d7c0725d0 0x7f8d7c073d40 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:25.668 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:25.666+0000 7f8d7b200640 1 --2- 172.21.15.151:0/2135693707 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8d7c15bc30 0x7f8d7c0b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:25.668 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:25.666+0000 7f8d7b200640 1 -- 172.21.15.151:0/2135693707 >> 172.21.15.151:0/2135693707 conn(0x7f8d7c0cfa20 msgr2=0x7f8d7c0cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:25.668 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:25.666+0000 7f8d7b200640 1 -- 172.21.15.151:0/2135693707 shutdown_connections 2024-09-16T15:40:25.668 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:25.666+0000 7f8d7b200640 1 -- 172.21.15.151:0/2135693707 wait complete. 2024-09-16T15:40:25.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:25.668+0000 7f8d7b200640 1 Processor -- start 2024-09-16T15:40:25.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:25.668+0000 7f8d7b200640 1 -- start start 2024-09-16T15:40:25.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:25.669+0000 7f8d7b200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8d7c0d0540 0x7f8d7c0c8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:25.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:25.669+0000 7f8d7b200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8d7c15bc30 0x7f8d7c1f4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:25.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:25.669+0000 7f8d7b200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d7c0c81d0 0x7f8d7c0ad880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:25.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:25.669+0000 7f8d7b200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7c074610 con 0x7f8d7c15bc30 2024-09-16T15:40:25.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:25.669+0000 7f8d7b200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7c074490 con 0x7f8d7c0c81d0 2024-09-16T15:40:25.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:25.669+0000 7f8d7b200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f8d7c074790 con 0x7f8d7c0d0540 2024-09-16T15:40:25.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:25.669+0000 7f8d79400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8d7c15bc30 0x7f8d7c1f4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:25.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:25.669+0000 7f8d79400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8d7c15bc30 0x7f8d7c1f4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49426/0 (socket says 172.21.15.151:49426) 2024-09-16T15:40:25.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:25.669+0000 7f8d79400640 1 -- 172.21.15.151:0/592665438 learned_addr learned my addr 172.21.15.151:0/592665438 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:40:25.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:25.669+0000 7f8d7a800640 1 --2- 172.21.15.151:0/592665438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d7c0c81d0 0x7f8d7c0ad880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:25.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:25.670+0000 7f8d79e00640 1 --2- 172.21.15.151:0/592665438 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8d7c0d0540 0x7f8d7c0c8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:25.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:25.670+0000 7f8d79400640 1 -- 172.21.15.151:0/592665438 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8d7c0d0540 msgr2=0x7f8d7c0c8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:25.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:25.670+0000 7f8d79400640 1 --2- 172.21.15.151:0/592665438 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8d7c0d0540 0x7f8d7c0c8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:25.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:25.670+0000 7f8d79400640 1 -- 172.21.15.151:0/592665438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d7c0c81d0 msgr2=0x7f8d7c0ad880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:25.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:25.670+0000 7f8d79400640 1 --2- 172.21.15.151:0/592665438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d7c0c81d0 0x7f8d7c0ad880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:25.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:25.670+0000 7f8d79400640 1 -- 172.21.15.151:0/592665438 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d7c0aecd0 con 0x7f8d7c15bc30 2024-09-16T15:40:25.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:25.670+0000 7f8d79e00640 1 --2- 172.21.15.151:0/592665438 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8d7c0d0540 0x7f8d7c0c8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:40:25.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:25.670+0000 7f8d79400640 1 --2- 172.21.15.151:0/592665438 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8d7c15bc30 0x7f8d7c1f4e30 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8d68009850 tx=0x7f8d680039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:25.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:25.670+0000 7f8d72000640 1 -- 172.21.15.151:0/592665438 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d6803f070 con 0x7f8d7c15bc30 2024-09-16T15:40:25.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:25.670+0000 7f8d72000640 1 -- 172.21.15.151:0/592665438 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8d68004890 con 0x7f8d7c15bc30 2024-09-16T15:40:25.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:25.670+0000 7f8d7b200640 1 -- 172.21.15.151:0/592665438 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d7c0aefb0 con 0x7f8d7c15bc30 2024-09-16T15:40:25.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:25.671+0000 7f8d72000640 1 -- 172.21.15.151:0/592665438 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d68035030 con 0x7f8d7c15bc30 2024-09-16T15:40:25.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:25.671+0000 7f8d7b200640 1 -- 172.21.15.151:0/592665438 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d7c0c8680 con 0x7f8d7c15bc30 2024-09-16T15:40:25.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:25.672+0000 7f8d7b200640 1 -- 172.21.15.151:0/592665438 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d7c06fc60 con 0x7f8d7c15bc30 2024-09-16T15:40:25.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:25.673+0000 7f8d72000640 1 -- 172.21.15.151:0/592665438 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f8d68042070 con 0x7f8d7c15bc30 2024-09-16T15:40:25.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:25.674+0000 7f8d72000640 1 --2- 172.21.15.151:0/592665438 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8d5c078850 0x7f8d5c07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:25.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:25.674+0000 7f8d72000640 1 -- 172.21.15.151:0/592665438 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f8d680bb6d0 con 0x7f8d7c15bc30 2024-09-16T15:40:25.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:25.675+0000 7f8d79e00640 1 --2- 172.21.15.151:0/592665438 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8d5c078850 0x7f8d5c07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:25.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:25.675+0000 7f8d79e00640 1 --2- 172.21.15.151:0/592665438 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8d5c078850 0x7f8d5c07ad10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c005fd0 tx=0x7f8d6c005e20 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:25.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:25.678+0000 7f8d72000640 1 -- 172.21.15.151:0/592665438 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8d68083780 con 0x7f8d7c15bc30 2024-09-16T15:40:25.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:25.888+0000 7f8d7b200640 1 -- 172.21.15.151:0/592665438 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f8d7c0c9250 con 0x7f8d5c078850 2024-09-16T15:40:25.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:25.890+0000 7f8d72000640 1 -- 172.21.15.151:0/592665438 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f8d7c0c9250 con 0x7f8d5c078850 2024-09-16T15:40:25.892 INFO:teuthology.orchestra.run.smithi151.stdout:[ 2024-09-16T15:40:25.892 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:40:25.892 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "cephfs" 2024-09-16T15:40:25.892 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:40:25.892 INFO:teuthology.orchestra.run.smithi151.stdout:] 2024-09-16T15:40:25.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:25.893+0000 7f8d7b200640 1 -- 172.21.15.151:0/592665438 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8d5c078850 msgr2=0x7f8d5c07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:25.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:25.893+0000 7f8d7b200640 1 --2- 172.21.15.151:0/592665438 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8d5c078850 0x7f8d5c07ad10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c005fd0 tx=0x7f8d6c005e20 comp rx=0 tx=0).stop 2024-09-16T15:40:25.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:25.893+0000 7f8d7b200640 1 -- 172.21.15.151:0/592665438 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8d7c15bc30 msgr2=0x7f8d7c1f4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:25.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:25.893+0000 7f8d7b200640 1 --2- 172.21.15.151:0/592665438 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8d7c15bc30 0x7f8d7c1f4e30 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8d68009850 tx=0x7f8d680039e0 comp rx=0 tx=0).stop 2024-09-16T15:40:25.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:25.893+0000 7f8d7b200640 1 -- 172.21.15.151:0/592665438 shutdown_connections 2024-09-16T15:40:25.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:25.893+0000 7f8d7b200640 1 --2- 172.21.15.151:0/592665438 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8d5c078850 0x7f8d5c07ad10 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:25.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:25.893+0000 7f8d7b200640 1 --2- 172.21.15.151:0/592665438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d7c0c81d0 0x7f8d7c0ad880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:25.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:25.893+0000 7f8d7b200640 1 --2- 172.21.15.151:0/592665438 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8d7c15bc30 0x7f8d7c1f4e30 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:25.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:25.893+0000 7f8d7b200640 1 --2- 172.21.15.151:0/592665438 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8d7c0d0540 0x7f8d7c0c8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:25.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:25.893+0000 7f8d7b200640 1 -- 172.21.15.151:0/592665438 >> 172.21.15.151:0/592665438 conn(0x7f8d7c0cfa20 msgr2=0x7f8d7c15ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:25.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:25.893+0000 7f8d7b200640 1 -- 172.21.15.151:0/592665438 shutdown_connections 2024-09-16T15:40:25.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:25.893+0000 7f8d7b200640 1 -- 172.21.15.151:0/592665438 wait complete. 2024-09-16T15:40:25.910 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-16T15:40:26.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.070+0000 7f6479400640 1 -- 172.21.15.151:0/2374020573 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6458004800 msgr2=0x7f6458004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:26.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.070+0000 7f6479400640 1 --2- 172.21.15.151:0/2374020573 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6458004800 0x7f6458004c80 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f6460006f80 tx=0x7f6458005830 comp rx=0 tx=0).stop 2024-09-16T15:40:26.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.071+0000 7f6479400640 1 -- 172.21.15.151:0/2374020573 shutdown_connections 2024-09-16T15:40:26.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.071+0000 7f6479400640 1 --2- 172.21.15.151:0/2374020573 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6458004800 0x7f6458004c80 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:26.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.071+0000 7f6479400640 1 -- 172.21.15.151:0/2374020573 >> 172.21.15.151:0/2374020573 conn(0x7f647409d0b0 msgr2=0x7f64740d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:26.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.071+0000 7f6479400640 1 -- 172.21.15.151:0/2374020573 shutdown_connections 2024-09-16T15:40:26.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.071+0000 7f6479400640 1 -- 172.21.15.151:0/2374020573 wait complete. 2024-09-16T15:40:26.073 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.072+0000 7f6479400640 1 Processor -- start 2024-09-16T15:40:26.073 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.072+0000 7f6479400640 1 -- start start 2024-09-16T15:40:26.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.073+0000 7f6479400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6458004800 0x7f64740ca730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:26.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.073+0000 7f6479400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f64741f7bb0 0x7f647415df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:26.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.073+0000 7f6479400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f64741f8030 0x7f647415e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:26.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.073+0000 7f6479400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f64740b50c0 con 0x7f64741f7bb0 2024-09-16T15:40:26.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.073+0000 7f6479400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f64740b4f40 con 0x7f6458004800 2024-09-16T15:40:26.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.073+0000 7f6479400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f64740d22f0 con 0x7f64741f8030 2024-09-16T15:40:26.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.074+0000 7f6473400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f64741f7bb0 0x7f647415df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:26.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.074+0000 7f6473400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f64741f7bb0 0x7f647415df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49440/0 (socket says 172.21.15.151:49440) 2024-09-16T15:40:26.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.074+0000 7f6473400640 1 -- 172.21.15.151:0/2965305319 learned_addr learned my addr 172.21.15.151:0/2965305319 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:40:26.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.074+0000 7f6473400640 1 -- 172.21.15.151:0/2965305319 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f64741f8030 msgr2=0x7f647415e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:26.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.074+0000 7f6473400640 1 --2- 172.21.15.151:0/2965305319 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f64741f8030 0x7f647415e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:26.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.074+0000 7f6473400640 1 -- 172.21.15.151:0/2965305319 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6458004800 msgr2=0x7f64740ca730 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T15:40:26.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.074+0000 7f6473400640 1 --2- 172.21.15.151:0/2965305319 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6458004800 0x7f64740ca730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:26.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.074+0000 7f6473400640 1 -- 172.21.15.151:0/2965305319 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64740a19e0 con 0x7f64741f7bb0 2024-09-16T15:40:26.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.074+0000 7f6473400640 1 --2- 172.21.15.151:0/2965305319 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f64741f7bb0 0x7f647415df50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f646002b060 tx=0x7f6460006bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:26.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.074+0000 7f646be00640 1 -- 172.21.15.151:0/2965305319 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64600037f0 con 0x7f64741f7bb0 2024-09-16T15:40:26.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.074+0000 7f646be00640 1 -- 172.21.15.151:0/2965305319 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6460003a80 con 0x7f64741f7bb0 2024-09-16T15:40:26.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.074+0000 7f646be00640 1 -- 172.21.15.151:0/2965305319 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6460042440 con 0x7f64741f7bb0 2024-09-16T15:40:26.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.075+0000 7f6479400640 1 -- 172.21.15.151:0/2965305319 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64740b9860 con 0x7f64741f7bb0 2024-09-16T15:40:26.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.075+0000 7f6479400640 1 -- 172.21.15.151:0/2965305319 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64740a1700 con 0x7f64741f7bb0 2024-09-16T15:40:26.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.076+0000 7f6479400640 1 -- 172.21.15.151:0/2965305319 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64740ce820 con 0x7f64741f7bb0 2024-09-16T15:40:26.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.077+0000 7f646be00640 1 -- 172.21.15.151:0/2965305319 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f6460035020 con 0x7f64741f7bb0 2024-09-16T15:40:26.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.078+0000 7f646be00640 1 --2- 172.21.15.151:0/2965305319 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6454078850 0x7f645407ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:26.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.078+0000 7f646be00640 1 -- 172.21.15.151:0/2965305319 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f64600baf60 con 0x7f64741f7bb0 2024-09-16T15:40:26.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.078+0000 7f6473e00640 1 --2- 172.21.15.151:0/2965305319 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6454078850 0x7f645407ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:26.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.079+0000 7f6473e00640 1 --2- 172.21.15.151:0/2965305319 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6454078850 0x7f645407ad10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6464000c00 tx=0x7f6464006b10 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:26.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.082+0000 7f646be00640 1 -- 172.21.15.151:0/2965305319 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f64600835c0 con 0x7f64741f7bb0 2024-09-16T15:40:26.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.267+0000 7f6479400640 1 -- 172.21.15.151:0/2965305319 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f64740a11f0 con 0x7f64741f7bb0 2024-09-16T15:40:26.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.267+0000 7f646be00640 1 -- 172.21.15.151:0/2965305319 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) ==== 122+0+0 (secure 0 0 0) 0x7f6460082ec0 con 0x7f64741f7bb0 2024-09-16T15:40:26.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.270+0000 7f6479400640 1 -- 172.21.15.151:0/2965305319 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6454078850 msgr2=0x7f645407ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:26.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.270+0000 7f6479400640 1 --2- 172.21.15.151:0/2965305319 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6454078850 0x7f645407ad10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6464000c00 tx=0x7f6464006b10 comp rx=0 tx=0).stop 2024-09-16T15:40:26.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.270+0000 7f6479400640 1 -- 172.21.15.151:0/2965305319 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f64741f7bb0 msgr2=0x7f647415df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:26.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.270+0000 7f6479400640 1 --2- 172.21.15.151:0/2965305319 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f64741f7bb0 0x7f647415df50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f646002b060 tx=0x7f6460006bb0 comp rx=0 tx=0).stop 2024-09-16T15:40:26.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.270+0000 7f6479400640 1 -- 172.21.15.151:0/2965305319 shutdown_connections 2024-09-16T15:40:26.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.270+0000 7f6479400640 1 --2- 172.21.15.151:0/2965305319 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6454078850 0x7f645407ad10 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:26.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.270+0000 7f6479400640 1 --2- 172.21.15.151:0/2965305319 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f64741f8030 0x7f647415e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:26.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.270+0000 7f6479400640 1 --2- 172.21.15.151:0/2965305319 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f64741f7bb0 0x7f647415df50 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:26.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.270+0000 7f6479400640 1 --2- 172.21.15.151:0/2965305319 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6458004800 0x7f64740ca730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:26.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.270+0000 7f6479400640 1 -- 172.21.15.151:0/2965305319 >> 172.21.15.151:0/2965305319 conn(0x7f647409d0b0 msgr2=0x7f64740d07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:26.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.270+0000 7f6479400640 1 -- 172.21.15.151:0/2965305319 shutdown_connections 2024-09-16T15:40:26.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.270+0000 7f6479400640 1 -- 172.21.15.151:0/2965305319 wait complete. 2024-09-16T15:40:26.287 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mgr mgr/volumes/max_concurrent_clones 2024-09-16T15:40:26.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.445+0000 7fb2c2000640 1 -- 172.21.15.151:0/3037314104 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb2bc1517a0 msgr2=0x7fb2bc0cdad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:26.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.446+0000 7fb2c2000640 1 --2- 172.21.15.151:0/3037314104 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb2bc1517a0 0x7fb2bc0cdad0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac009830 tx=0x7fb2ac02a510 comp rx=0 tx=0).stop 2024-09-16T15:40:26.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.446+0000 7fb2c2000640 1 -- 172.21.15.151:0/3037314104 shutdown_connections 2024-09-16T15:40:26.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.446+0000 7fb2c2000640 1 --2- 172.21.15.151:0/3037314104 >> v2:172.21.15.167:3300/0 conn(0x7fb2bc0cd490 0x7fb2bc0732c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:26.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.446+0000 7fb2c2000640 1 --2- 172.21.15.151:0/3037314104 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb2bc1517a0 0x7fb2bc0cdad0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:26.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.446+0000 7fb2c2000640 1 -- 172.21.15.151:0/3037314104 >> 172.21.15.151:0/3037314104 conn(0x7fb2bc0b1840 msgr2=0x7fb2bc0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:26.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.446+0000 7fb2c2000640 1 -- 172.21.15.151:0/3037314104 shutdown_connections 2024-09-16T15:40:26.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.446+0000 7fb2c2000640 1 -- 172.21.15.151:0/3037314104 wait complete. 2024-09-16T15:40:26.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.447+0000 7fb2c2000640 1 Processor -- start 2024-09-16T15:40:26.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.448+0000 7fb2c2000640 1 -- start start 2024-09-16T15:40:26.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.448+0000 7fb2c2000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2bc073800 0x7fb2bc154310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:26.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.448+0000 7fb2c2000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb2bc0cd490 0x7fb2bc157ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:26.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.449+0000 7fb2c2000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb2bc1517a0 0x7fb2bc1f56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:26.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.449+0000 7fb2c2000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb2bc0cc650 con 0x7fb2bc1517a0 2024-09-16T15:40:26.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.449+0000 7fb2c2000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb2bc073fc0 con 0x7fb2bc073800 2024-09-16T15:40:26.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.449+0000 7fb2c2000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fb2bc0cc7d0 con 0x7fb2bc0cd490 2024-09-16T15:40:26.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.449+0000 7fb2bbe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb2bc1517a0 0x7fb2bc1f56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:26.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.449+0000 7fb2bb400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2bc073800 0x7fb2bc154310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:26.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.449+0000 7fb2bb400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2bc073800 0x7fb2bc154310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.151:44632/0 (socket says 172.21.15.151:44632) 2024-09-16T15:40:26.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.449+0000 7fb2baa00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb2bc0cd490 0x7fb2bc157ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:26.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.449+0000 7fb2bb400640 1 -- 172.21.15.151:0/3461158180 learned_addr learned my addr 172.21.15.151:0/3461158180 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:40:26.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.449+0000 7fb2bbe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb2bc1517a0 0x7fb2bc1f56f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49454/0 (socket says 172.21.15.151:49454) 2024-09-16T15:40:26.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.449+0000 7fb2bbe00640 1 -- 172.21.15.151:0/3461158180 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb2bc0cd490 msgr2=0x7fb2bc157ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:26.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.449+0000 7fb2bbe00640 1 --2- 172.21.15.151:0/3461158180 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb2bc0cd490 0x7fb2bc157ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:26.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.449+0000 7fb2bbe00640 1 -- 172.21.15.151:0/3461158180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2bc073800 msgr2=0x7fb2bc154310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:26.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.449+0000 7fb2bbe00640 1 --2- 172.21.15.151:0/3461158180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2bc073800 0x7fb2bc154310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:26.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.449+0000 7fb2bbe00640 1 -- 172.21.15.151:0/3461158180 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2bc1549d0 con 0x7fb2bc1517a0 2024-09-16T15:40:26.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.449+0000 7fb2bb400640 1 --2- 172.21.15.151:0/3461158180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2bc073800 0x7fb2bc154310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:40:26.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.450+0000 7fb2bbe00640 1 --2- 172.21.15.151:0/3461158180 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb2bc1517a0 0x7fb2bc1f56f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb2b400c730 tx=0x7fb2b400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:26.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.450+0000 7fb2b3400640 1 -- 172.21.15.151:0/3461158180 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b4017070 con 0x7fb2bc1517a0 2024-09-16T15:40:26.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.450+0000 7fb2b3400640 1 -- 172.21.15.151:0/3461158180 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb2b4003ca0 con 0x7fb2bc1517a0 2024-09-16T15:40:26.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.450+0000 7fb2c2000640 1 -- 172.21.15.151:0/3461158180 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2bc154cb0 con 0x7fb2bc1517a0 2024-09-16T15:40:26.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.450+0000 7fb2b3400640 1 -- 172.21.15.151:0/3461158180 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b40052d0 con 0x7fb2bc1517a0 2024-09-16T15:40:26.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.450+0000 7fb2c2000640 1 -- 172.21.15.151:0/3461158180 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2bc158400 con 0x7fb2bc1517a0 2024-09-16T15:40:26.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.451+0000 7fb2c2000640 1 -- 172.21.15.151:0/3461158180 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2bc0ccc80 con 0x7fb2bc1517a0 2024-09-16T15:40:26.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.452+0000 7fb2b3400640 1 -- 172.21.15.151:0/3461158180 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fb2b401a070 con 0x7fb2bc1517a0 2024-09-16T15:40:26.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.454+0000 7fb2b3400640 1 --2- 172.21.15.151:0/3461158180 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb29c078850 0x7fb29c07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:26.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.454+0000 7fb2b3400640 1 -- 172.21.15.151:0/3461158180 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fb2b409a100 con 0x7fb2bc1517a0 2024-09-16T15:40:26.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.454+0000 7fb2bb400640 1 --2- 172.21.15.151:0/3461158180 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb29c078850 0x7fb29c07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:26.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.455+0000 7fb2bb400640 1 --2- 172.21.15.151:0/3461158180 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb29c078850 0x7fb29c07ad10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac009830 tx=0x7fb2ac004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:26.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.458+0000 7fb2b3400640 1 -- 172.21.15.151:0/3461158180 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb2b4062080 con 0x7fb2bc1517a0 2024-09-16T15:40:26.648 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.646+0000 7fb2c2000640 1 -- 172.21.15.151:0/3461158180 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"} v 0) -- 0x7fb2bc0b2430 con 0x7fb2bc1517a0 2024-09-16T15:40:26.648 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.647+0000 7fb2b3400640 1 -- 172.21.15.151:0/3461158180 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"}]=0 v28) ==== 116+0+2 (secure 0 0 0) 0x7fb2b4066ab0 con 0x7fb2bc1517a0 2024-09-16T15:40:26.648 INFO:teuthology.orchestra.run.smithi151.stdout:4 2024-09-16T15:40:26.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.649+0000 7fb2c2000640 1 -- 172.21.15.151:0/3461158180 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb29c078850 msgr2=0x7fb29c07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:26.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.649+0000 7fb2c2000640 1 --2- 172.21.15.151:0/3461158180 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb29c078850 0x7fb29c07ad10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac009830 tx=0x7fb2ac004730 comp rx=0 tx=0).stop 2024-09-16T15:40:26.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.649+0000 7fb2c2000640 1 -- 172.21.15.151:0/3461158180 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb2bc1517a0 msgr2=0x7fb2bc1f56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:26.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.649+0000 7fb2c2000640 1 --2- 172.21.15.151:0/3461158180 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb2bc1517a0 0x7fb2bc1f56f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb2b400c730 tx=0x7fb2b400cbf0 comp rx=0 tx=0).stop 2024-09-16T15:40:26.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.650+0000 7fb2c2000640 1 -- 172.21.15.151:0/3461158180 shutdown_connections 2024-09-16T15:40:26.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.650+0000 7fb2c2000640 1 --2- 172.21.15.151:0/3461158180 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb29c078850 0x7fb29c07ad10 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:26.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.650+0000 7fb2c2000640 1 --2- 172.21.15.151:0/3461158180 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb2bc1517a0 0x7fb2bc1f56f0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:26.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.650+0000 7fb2c2000640 1 --2- 172.21.15.151:0/3461158180 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb2bc0cd490 0x7fb2bc157ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:26.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.650+0000 7fb2c2000640 1 --2- 172.21.15.151:0/3461158180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2bc073800 0x7fb2bc154310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:26.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.650+0000 7fb2c2000640 1 -- 172.21.15.151:0/3461158180 >> 172.21.15.151:0/3461158180 conn(0x7fb2bc0b1840 msgr2=0x7fb2bc0cdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:26.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.650+0000 7fb2c2000640 1 -- 172.21.15.151:0/3461158180 shutdown_connections 2024-09-16T15:40:26.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.650+0000 7fb2c2000640 1 -- 172.21.15.151:0/3461158180 wait complete. 2024-09-16T15:40:26.666 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-16T15:40:26.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.826+0000 7fef48c00640 1 -- 172.21.15.151:0/1106121094 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fef4406efa0 msgr2=0x7fef440cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:26.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.826+0000 7fef48c00640 1 --2- 172.21.15.151:0/1106121094 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fef4406efa0 0x7fef440cdbb0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fef440af780 tx=0x7fef3c02a9a0 comp rx=0 tx=0).stop 2024-09-16T15:40:26.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.826+0000 7fef48c00640 1 -- 172.21.15.151:0/1106121094 shutdown_connections 2024-09-16T15:40:26.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.826+0000 7fef48c00640 1 --2- 172.21.15.151:0/1106121094 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fef4406efa0 0x7fef440cdbb0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:26.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.826+0000 7fef48c00640 1 --2- 172.21.15.151:0/1106121094 >> v2:172.21.15.167:3300/0 conn(0x7fef440cccd0 0x7fef44073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:26.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.826+0000 7fef48c00640 1 -- 172.21.15.151:0/1106121094 >> 172.21.15.151:0/1106121094 conn(0x7fef440b1840 msgr2=0x7fef440b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:26.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.826+0000 7fef48c00640 1 -- 172.21.15.151:0/1106121094 shutdown_connections 2024-09-16T15:40:26.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.826+0000 7fef48c00640 1 -- 172.21.15.151:0/1106121094 wait complete. 2024-09-16T15:40:26.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.828+0000 7fef48c00640 1 Processor -- start 2024-09-16T15:40:26.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.828+0000 7fef48c00640 1 -- start start 2024-09-16T15:40:26.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.829+0000 7fef48c00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fef4406efa0 0x7fef4415e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:26.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.829+0000 7fef48c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fef440cccd0 0x7fef4415e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:26.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.829+0000 7fef48c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef44145750 0x7fef440c7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:26.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.829+0000 7fef48c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fef440ce480 con 0x7fef440cccd0 2024-09-16T15:40:26.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.829+0000 7fef48c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fef440ce300 con 0x7fef44145750 2024-09-16T15:40:26.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.829+0000 7fef48c00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fef440ce600 con 0x7fef4406efa0 2024-09-16T15:40:26.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.829+0000 7fef41400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fef440cccd0 0x7fef4415e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:26.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.829+0000 7fef41400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fef440cccd0 0x7fef4415e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49460/0 (socket says 172.21.15.151:49460) 2024-09-16T15:40:26.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.829+0000 7fef41400640 1 -- 172.21.15.151:0/659135950 learned_addr learned my addr 172.21.15.151:0/659135950 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:40:26.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.829+0000 7fef41e00640 1 --2- 172.21.15.151:0/659135950 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fef4406efa0 0x7fef4415e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:26.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.829+0000 7fef42800640 1 --2- 172.21.15.151:0/659135950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef44145750 0x7fef440c7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:26.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.829+0000 7fef41400640 1 -- 172.21.15.151:0/659135950 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fef4406efa0 msgr2=0x7fef4415e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:26.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.829+0000 7fef41400640 1 --2- 172.21.15.151:0/659135950 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fef4406efa0 0x7fef4415e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:26.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.829+0000 7fef41400640 1 -- 172.21.15.151:0/659135950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef44145750 msgr2=0x7fef440c7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:26.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.829+0000 7fef41400640 1 --2- 172.21.15.151:0/659135950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef44145750 0x7fef440c7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:26.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.829+0000 7fef41400640 1 -- 172.21.15.151:0/659135950 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef44155220 con 0x7fef440cccd0 2024-09-16T15:40:26.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.830+0000 7fef41400640 1 --2- 172.21.15.151:0/659135950 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fef440cccd0 0x7fef4415e590 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fef3000c950 tx=0x7fef3000ce10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:26.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.830+0000 7fef3a000640 1 -- 172.21.15.151:0/659135950 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef30007c70 con 0x7fef440cccd0 2024-09-16T15:40:26.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.830+0000 7fef3a000640 1 -- 172.21.15.151:0/659135950 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fef3000f450 con 0x7fef440cccd0 2024-09-16T15:40:26.833 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.830+0000 7fef48c00640 1 -- 172.21.15.151:0/659135950 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef44155500 con 0x7fef440cccd0 2024-09-16T15:40:26.833 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.830+0000 7fef3a000640 1 -- 172.21.15.151:0/659135950 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef3000f8a0 con 0x7fef440cccd0 2024-09-16T15:40:26.833 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.830+0000 7fef48c00640 1 -- 172.21.15.151:0/659135950 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef4413fcf0 con 0x7fef440cccd0 2024-09-16T15:40:26.833 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.831+0000 7fef48c00640 1 -- 172.21.15.151:0/659135950 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef441401e0 con 0x7fef440cccd0 2024-09-16T15:40:26.833 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.832+0000 7fef3a000640 1 -- 172.21.15.151:0/659135950 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fef3000fa40 con 0x7fef440cccd0 2024-09-16T15:40:26.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.833+0000 7fef3a000640 1 --2- 172.21.15.151:0/659135950 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fef24078850 0x7fef2407ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:26.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.834+0000 7fef3a000640 1 -- 172.21.15.151:0/659135950 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fef3009b210 con 0x7fef440cccd0 2024-09-16T15:40:26.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.834+0000 7fef41e00640 1 --2- 172.21.15.151:0/659135950 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fef24078850 0x7fef2407ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:26.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.835+0000 7fef41e00640 1 --2- 172.21.15.151:0/659135950 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fef24078850 0x7fef2407ad10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fef34006fd0 tx=0x7fef34008040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:26.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:26.837+0000 7fef3a000640 1 -- 172.21.15.151:0/659135950 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fef30014030 con 0x7fef440cccd0 2024-09-16T15:40:27.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.022+0000 7fef48c00640 1 -- 172.21.15.151:0/659135950 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7fef4415f770 con 0x7fef440cccd0 2024-09-16T15:40:27.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.026+0000 7fef3a000640 1 -- 172.21.15.151:0/659135950 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v29)=0 v29) ==== 131+0+0 (secure 0 0 0) 0x7fef4415f770 con 0x7fef440cccd0 2024-09-16T15:40:27.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.028+0000 7fef48c00640 1 -- 172.21.15.151:0/659135950 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fef24078850 msgr2=0x7fef2407ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:27.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.028+0000 7fef48c00640 1 --2- 172.21.15.151:0/659135950 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fef24078850 0x7fef2407ad10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fef34006fd0 tx=0x7fef34008040 comp rx=0 tx=0).stop 2024-09-16T15:40:27.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.028+0000 7fef48c00640 1 -- 172.21.15.151:0/659135950 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fef440cccd0 msgr2=0x7fef4415e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:27.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.028+0000 7fef48c00640 1 --2- 172.21.15.151:0/659135950 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fef440cccd0 0x7fef4415e590 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fef3000c950 tx=0x7fef3000ce10 comp rx=0 tx=0).stop 2024-09-16T15:40:27.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.028+0000 7fef48c00640 1 -- 172.21.15.151:0/659135950 shutdown_connections 2024-09-16T15:40:27.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.028+0000 7fef48c00640 1 --2- 172.21.15.151:0/659135950 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fef24078850 0x7fef2407ad10 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:27.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.028+0000 7fef48c00640 1 --2- 172.21.15.151:0/659135950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef44145750 0x7fef440c7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:27.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.028+0000 7fef48c00640 1 --2- 172.21.15.151:0/659135950 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fef440cccd0 0x7fef4415e590 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:27.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.028+0000 7fef48c00640 1 --2- 172.21.15.151:0/659135950 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fef4406efa0 0x7fef4415e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:27.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.028+0000 7fef48c00640 1 -- 172.21.15.151:0/659135950 >> 172.21.15.151:0/659135950 conn(0x7fef440b1840 msgr2=0x7fef440b38e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:27.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.029+0000 7fef48c00640 1 -- 172.21.15.151:0/659135950 shutdown_connections 2024-09-16T15:40:27.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.029+0000 7fef48c00640 1 -- 172.21.15.151:0/659135950 wait complete. 2024-09-16T15:40:27.045 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs group1 2024-09-16T15:40:27.205 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.204+0000 7fadbfe00640 1 -- 172.21.15.151:0/251987320 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fadb806efa0 msgr2=0x7fadb80cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:27.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.204+0000 7fadbfe00640 1 --2- 172.21.15.151:0/251987320 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fadb806efa0 0x7fadb80cdbb0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fadb0005fd0 tx=0x7fadb002a540 comp rx=0 tx=0).stop 2024-09-16T15:40:27.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.204+0000 7fadbfe00640 1 -- 172.21.15.151:0/251987320 shutdown_connections 2024-09-16T15:40:27.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.204+0000 7fadbfe00640 1 --2- 172.21.15.151:0/251987320 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fadb806efa0 0x7fadb80cdbb0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:27.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.204+0000 7fadbfe00640 1 --2- 172.21.15.151:0/251987320 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fadb80cccd0 0x7fadb8073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:27.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.204+0000 7fadbfe00640 1 -- 172.21.15.151:0/251987320 >> 172.21.15.151:0/251987320 conn(0x7fadb80b1840 msgr2=0x7fadb80b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:27.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.204+0000 7fadbfe00640 1 -- 172.21.15.151:0/251987320 shutdown_connections 2024-09-16T15:40:27.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.204+0000 7fadbfe00640 1 -- 172.21.15.151:0/251987320 wait complete. 2024-09-16T15:40:27.207 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.206+0000 7fadbfe00640 1 Processor -- start 2024-09-16T15:40:27.207 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.206+0000 7fadbfe00640 1 -- start start 2024-09-16T15:40:27.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.207+0000 7fadbfe00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fadb806efa0 0x7fadb813fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:27.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.207+0000 7fadbfe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fadb80cccd0 0x7fadb8141aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:27.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.207+0000 7fadbfe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadb8145750 0x7fadb81f7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:27.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.207+0000 7fadbfe00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fadb80ce480 con 0x7fadb80cccd0 2024-09-16T15:40:27.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.207+0000 7fadbfe00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fadb80ce300 con 0x7fadb8145750 2024-09-16T15:40:27.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.207+0000 7fadbfe00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fadb80ce600 con 0x7fadb806efa0 2024-09-16T15:40:27.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.207+0000 7fadbca00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fadb80cccd0 0x7fadb8141aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:27.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.207+0000 7fadbca00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fadb80cccd0 0x7fadb8141aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49478/0 (socket says 172.21.15.151:49478) 2024-09-16T15:40:27.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.207+0000 7fadbca00640 1 -- 172.21.15.151:0/319410056 learned_addr learned my addr 172.21.15.151:0/319410056 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:40:27.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.207+0000 7fadbd400640 1 --2- 172.21.15.151:0/319410056 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fadb806efa0 0x7fadb813fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:27.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.207+0000 7fadbde00640 1 --2- 172.21.15.151:0/319410056 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadb8145750 0x7fadb81f7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:27.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.207+0000 7fadbca00640 1 -- 172.21.15.151:0/319410056 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fadb806efa0 msgr2=0x7fadb813fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:27.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.207+0000 7fadbca00640 1 --2- 172.21.15.151:0/319410056 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fadb806efa0 0x7fadb813fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:27.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.207+0000 7fadbca00640 1 -- 172.21.15.151:0/319410056 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadb8145750 msgr2=0x7fadb81f7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:27.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.207+0000 7fadbca00640 1 --2- 172.21.15.151:0/319410056 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadb8145750 0x7fadb81f7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:27.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.207+0000 7fadbca00640 1 -- 172.21.15.151:0/319410056 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadb813f300 con 0x7fadb80cccd0 2024-09-16T15:40:27.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.208+0000 7fadbca00640 1 --2- 172.21.15.151:0/319410056 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fadb80cccd0 0x7fadb8141aa0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fada400c920 tx=0x7fada400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:27.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.208+0000 7fadb5600640 1 -- 172.21.15.151:0/319410056 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada4007c70 con 0x7fadb80cccd0 2024-09-16T15:40:27.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.208+0000 7fadb5600640 1 -- 172.21.15.151:0/319410056 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fada400f450 con 0x7fadb80cccd0 2024-09-16T15:40:27.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.208+0000 7fadb5600640 1 -- 172.21.15.151:0/319410056 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada400f8a0 con 0x7fadb80cccd0 2024-09-16T15:40:27.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.208+0000 7fadbfe00640 1 -- 172.21.15.151:0/319410056 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadb8140090 con 0x7fadb80cccd0 2024-09-16T15:40:27.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.208+0000 7fadbfe00640 1 -- 172.21.15.151:0/319410056 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadb81426b0 con 0x7fadb80cccd0 2024-09-16T15:40:27.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.209+0000 7fadbfe00640 1 -- 172.21.15.151:0/319410056 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadb80cdbb0 con 0x7fadb80cccd0 2024-09-16T15:40:27.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.210+0000 7fadb5600640 1 -- 172.21.15.151:0/319410056 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7fada400fa40 con 0x7fadb80cccd0 2024-09-16T15:40:27.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.211+0000 7fadb5600640 1 --2- 172.21.15.151:0/319410056 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fad98078850 0x7fad9807ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:27.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.211+0000 7fadb5600640 1 -- 172.21.15.151:0/319410056 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fada409b230 con 0x7fadb80cccd0 2024-09-16T15:40:27.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.211+0000 7fadbd400640 1 --2- 172.21.15.151:0/319410056 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fad98078850 0x7fad9807ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:27.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.212+0000 7fadbd400640 1 --2- 172.21.15.151:0/319410056 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fad98078850 0x7fad9807ad10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fadac001ce0 tx=0x7fadac009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:27.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.215+0000 7fadb5600640 1 -- 172.21.15.151:0/319410056 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fada4014030 con 0x7fadb80cccd0 2024-09-16T15:40:27.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.423+0000 7fadbfe00640 1 -- 172.21.15.151:0/319410056 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "group1", "target": ["mon-mgr", ""]}) -- 0x7fadb815f5b0 con 0x7fad98078850 2024-09-16T15:40:27.432 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:40:27.430+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:40:27.432 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:40:27.430+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:40:27.432 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:40:27.430+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:40:27.433 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:40:27.430+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:40:27.433 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:40:27.430+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:40:27.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.451+0000 7fadb5600640 1 -- 172.21.15.151:0/319410056 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fadb815f5b0 con 0x7fad98078850 2024-09-16T15:40:27.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.454+0000 7fadbfe00640 1 -- 172.21.15.151:0/319410056 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fad98078850 msgr2=0x7fad9807ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:27.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.454+0000 7fadbfe00640 1 --2- 172.21.15.151:0/319410056 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fad98078850 0x7fad9807ad10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fadac001ce0 tx=0x7fadac009040 comp rx=0 tx=0).stop 2024-09-16T15:40:27.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.454+0000 7fadbfe00640 1 -- 172.21.15.151:0/319410056 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fadb80cccd0 msgr2=0x7fadb8141aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:27.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.454+0000 7fadbfe00640 1 --2- 172.21.15.151:0/319410056 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fadb80cccd0 0x7fadb8141aa0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fada400c920 tx=0x7fada400cde0 comp rx=0 tx=0).stop 2024-09-16T15:40:27.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.454+0000 7fadbfe00640 1 -- 172.21.15.151:0/319410056 shutdown_connections 2024-09-16T15:40:27.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.454+0000 7fadbfe00640 1 --2- 172.21.15.151:0/319410056 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fad98078850 0x7fad9807ad10 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:27.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.454+0000 7fadbfe00640 1 --2- 172.21.15.151:0/319410056 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadb8145750 0x7fadb81f7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:27.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.454+0000 7fadbfe00640 1 --2- 172.21.15.151:0/319410056 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fadb80cccd0 0x7fadb8141aa0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:27.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.454+0000 7fadbfe00640 1 --2- 172.21.15.151:0/319410056 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fadb806efa0 0x7fadb813fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:27.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.454+0000 7fadbfe00640 1 -- 172.21.15.151:0/319410056 >> 172.21.15.151:0/319410056 conn(0x7fadb80b1840 msgr2=0x7fadb80aa4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:27.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.454+0000 7fadbfe00640 1 -- 172.21.15.151:0/319410056 shutdown_connections 2024-09-16T15:40:27.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.454+0000 7fadbfe00640 1 -- 172.21.15.151:0/319410056 wait complete. 2024-09-16T15:40:27.471 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 group1 --mode=777 2024-09-16T15:40:27.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.635+0000 7f64e5a00640 1 -- 172.21.15.151:0/3906674289 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f64e015aed0 msgr2=0x7f64e00ca0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:27.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.635+0000 7f64e5a00640 1 --2- 172.21.15.151:0/3906674289 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f64e015aed0 0x7f64e00ca0c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f64c8005fd0 tx=0x7f64c802a5d0 comp rx=0 tx=0).stop 2024-09-16T15:40:27.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.635+0000 7f64e5a00640 1 -- 172.21.15.151:0/3906674289 shutdown_connections 2024-09-16T15:40:27.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.635+0000 7f64e5a00640 1 --2- 172.21.15.151:0/3906674289 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f64e015aed0 0x7f64e00ca0c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:27.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.635+0000 7f64e5a00640 1 --2- 172.21.15.151:0/3906674289 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f64e00aec10 0x7f64e00ce530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:27.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.635+0000 7f64e5a00640 1 -- 172.21.15.151:0/3906674289 >> 172.21.15.151:0/3906674289 conn(0x7f64e00cfa70 msgr2=0x7f64e00cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:27.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.635+0000 7f64e5a00640 1 -- 172.21.15.151:0/3906674289 shutdown_connections 2024-09-16T15:40:27.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.635+0000 7f64e5a00640 1 -- 172.21.15.151:0/3906674289 wait complete. 2024-09-16T15:40:27.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.637+0000 7f64e5a00640 1 Processor -- start 2024-09-16T15:40:27.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.637+0000 7f64e5a00640 1 -- start start 2024-09-16T15:40:27.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.638+0000 7f64e5a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64e00aec10 0x7f64e00b50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:27.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.638+0000 7f64e5a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f64e00cf170 0x7f64e0156d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:27.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.638+0000 7f64e5a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f64e015aed0 0x7f64e014f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:27.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.638+0000 7f64e5a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f64e00ca600 con 0x7f64e015aed0 2024-09-16T15:40:27.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.638+0000 7f64e5a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f64e0070900 con 0x7f64e00aec10 2024-09-16T15:40:27.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.638+0000 7f64e5a00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f64e00ca780 con 0x7f64e00cf170 2024-09-16T15:40:27.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.638+0000 7f64e5000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f64e015aed0 0x7f64e014f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:27.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.638+0000 7f64e5000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f64e015aed0 0x7f64e014f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49506/0 (socket says 172.21.15.151:49506) 2024-09-16T15:40:27.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.638+0000 7f64e5000640 1 -- 172.21.15.151:0/1779138808 learned_addr learned my addr 172.21.15.151:0/1779138808 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:40:27.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.638+0000 7f64d7e00640 1 --2- 172.21.15.151:0/1779138808 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f64e00cf170 0x7f64e0156d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:27.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.638+0000 7f64dfe00640 1 --2- 172.21.15.151:0/1779138808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64e00aec10 0x7f64e00b50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:27.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.639+0000 7f64e5000640 1 -- 172.21.15.151:0/1779138808 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f64e00cf170 msgr2=0x7f64e0156d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:27.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.639+0000 7f64e5000640 1 --2- 172.21.15.151:0/1779138808 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f64e00cf170 0x7f64e0156d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:27.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.639+0000 7f64e5000640 1 -- 172.21.15.151:0/1779138808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64e00aec10 msgr2=0x7f64e00b50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:27.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.639+0000 7f64e5000640 1 --2- 172.21.15.151:0/1779138808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64e00aec10 0x7f64e00b50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:27.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.639+0000 7f64e5000640 1 -- 172.21.15.151:0/1779138808 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64e00b46d0 con 0x7f64e015aed0 2024-09-16T15:40:27.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.639+0000 7f64e5000640 1 --2- 172.21.15.151:0/1779138808 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f64e015aed0 0x7f64e014f090 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f64d800c7a0 tx=0x7f64d800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:27.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.639+0000 7f64dcc00640 1 -- 172.21.15.151:0/1779138808 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d8017070 con 0x7f64e015aed0 2024-09-16T15:40:27.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.639+0000 7f64dcc00640 1 -- 172.21.15.151:0/1779138808 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f64d8005070 con 0x7f64e015aed0 2024-09-16T15:40:27.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.639+0000 7f64e5a00640 1 -- 172.21.15.151:0/1779138808 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64e0157980 con 0x7f64e015aed0 2024-09-16T15:40:27.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.639+0000 7f64dcc00640 1 -- 172.21.15.151:0/1779138808 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d80054c0 con 0x7f64e015aed0 2024-09-16T15:40:27.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.639+0000 7f64e5a00640 1 -- 172.21.15.151:0/1779138808 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64e00b2940 con 0x7f64e015aed0 2024-09-16T15:40:27.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.640+0000 7f64e5a00640 1 -- 172.21.15.151:0/1779138808 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64e00ce530 con 0x7f64e015aed0 2024-09-16T15:40:27.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.641+0000 7f64dcc00640 1 -- 172.21.15.151:0/1779138808 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f64d801a070 con 0x7f64e015aed0 2024-09-16T15:40:27.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.643+0000 7f64dcc00640 1 --2- 172.21.15.151:0/1779138808 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f64bc078850 0x7f64bc07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:27.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.643+0000 7f64dcc00640 1 -- 172.21.15.151:0/1779138808 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f64d8099fe0 con 0x7f64e015aed0 2024-09-16T15:40:27.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.643+0000 7f64dfe00640 1 --2- 172.21.15.151:0/1779138808 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f64bc078850 0x7f64bc07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:27.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.644+0000 7f64dfe00640 1 --2- 172.21.15.151:0/1779138808 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f64bc078850 0x7f64bc07ad10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f64d0009b30 tx=0x7f64d0005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:27.648 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.647+0000 7f64dcc00640 1 -- 172.21.15.151:0/1779138808 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f64d8061f80 con 0x7f64e015aed0 2024-09-16T15:40:27.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.857+0000 7f64e5a00640 1 -- 172.21.15.151:0/1779138808 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "group_name": "group1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7f64e00b59f0 con 0x7f64bc078850 2024-09-16T15:40:27.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.892+0000 7f64dcc00640 1 -- 172.21.15.151:0/1779138808 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f64e00b59f0 con 0x7f64bc078850 2024-09-16T15:40:27.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.894+0000 7f64e5a00640 1 -- 172.21.15.151:0/1779138808 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f64bc078850 msgr2=0x7f64bc07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:27.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.894+0000 7f64e5a00640 1 --2- 172.21.15.151:0/1779138808 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f64bc078850 0x7f64bc07ad10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f64d0009b30 tx=0x7f64d0005f90 comp rx=0 tx=0).stop 2024-09-16T15:40:27.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.894+0000 7f64e5a00640 1 -- 172.21.15.151:0/1779138808 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f64e015aed0 msgr2=0x7f64e014f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:27.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.894+0000 7f64e5a00640 1 --2- 172.21.15.151:0/1779138808 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f64e015aed0 0x7f64e014f090 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f64d800c7a0 tx=0x7f64d800cc60 comp rx=0 tx=0).stop 2024-09-16T15:40:27.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.894+0000 7f64e5a00640 1 -- 172.21.15.151:0/1779138808 shutdown_connections 2024-09-16T15:40:27.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.894+0000 7f64e5a00640 1 --2- 172.21.15.151:0/1779138808 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f64bc078850 0x7f64bc07ad10 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:27.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.894+0000 7f64e5a00640 1 --2- 172.21.15.151:0/1779138808 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f64e015aed0 0x7f64e014f090 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:27.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.894+0000 7f64e5a00640 1 --2- 172.21.15.151:0/1779138808 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f64e00cf170 0x7f64e0156d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:27.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.894+0000 7f64e5a00640 1 --2- 172.21.15.151:0/1779138808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64e00aec10 0x7f64e00b50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:27.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.894+0000 7f64e5a00640 1 -- 172.21.15.151:0/1779138808 >> 172.21.15.151:0/1779138808 conn(0x7f64e00cfa70 msgr2=0x7f64e0158a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:27.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.894+0000 7f64e5a00640 1 -- 172.21.15.151:0/1779138808 shutdown_connections 2024-09-16T15:40:27.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:27.894+0000 7f64e5a00640 1 -- 172.21.15.151:0/1779138808 wait complete. 2024-09-16T15:40:27.911 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 group1 2024-09-16T15:40:28.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:28.072+0000 7f4a4da00640 1 -- 172.21.15.151:0/3172812194 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a4806f220 msgr2=0x7f4a4806f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:28.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:28.072+0000 7f4a4da00640 1 --2- 172.21.15.151:0/3172812194 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a4806f220 0x7f4a4806f6b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4a480b2680 tx=0x7f4a4002aa30 comp rx=0 tx=0).stop 2024-09-16T15:40:28.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:28.073+0000 7f4a4da00640 1 -- 172.21.15.151:0/3172812194 shutdown_connections 2024-09-16T15:40:28.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:28.073+0000 7f4a4da00640 1 --2- 172.21.15.151:0/3172812194 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a4806f220 0x7f4a4806f6b0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:28.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:28.073+0000 7f4a4da00640 1 --2- 172.21.15.151:0/3172812194 >> v2:172.21.15.167:3300/0 conn(0x7f4a48159aa0 0x7f4a480741a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:28.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:28.073+0000 7f4a4da00640 1 -- 172.21.15.151:0/3172812194 >> 172.21.15.151:0/3172812194 conn(0x7f4a480b18e0 msgr2=0x7f4a480b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:28.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:28.073+0000 7f4a4da00640 1 -- 172.21.15.151:0/3172812194 shutdown_connections 2024-09-16T15:40:28.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:28.073+0000 7f4a4da00640 1 -- 172.21.15.151:0/3172812194 wait complete. 2024-09-16T15:40:28.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:28.074+0000 7f4a4da00640 1 Processor -- start 2024-09-16T15:40:28.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:28.074+0000 7f4a4da00640 1 -- start start 2024-09-16T15:40:28.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:28.075+0000 7f4a4da00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a4806f220 0x7f4a481f5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:28.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:28.075+0000 7f4a4da00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a48159aa0 0x7f4a48157aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:28.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:28.075+0000 7f4a4da00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a4815cff0 0x7f4a48154410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:28.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:28.075+0000 7f4a4da00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4a480cc520 con 0x7f4a4815cff0 2024-09-16T15:40:28.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:28.075+0000 7f4a4da00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4a480cc3a0 con 0x7f4a4806f220 2024-09-16T15:40:28.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:28.075+0000 7f4a4da00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f4a480cc6a0 con 0x7f4a48159aa0 2024-09-16T15:40:28.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:28.075+0000 7f4a47400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a4815cff0 0x7f4a48154410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:28.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:28.076+0000 7f4a47400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a4815cff0 0x7f4a48154410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49532/0 (socket says 172.21.15.151:49532) 2024-09-16T15:40:28.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:28.076+0000 7f4a47400640 1 -- 172.21.15.151:0/2651407532 learned_addr learned my addr 172.21.15.151:0/2651407532 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:40:28.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:28.076+0000 7f4a46a00640 1 --2- 172.21.15.151:0/2651407532 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a4806f220 0x7f4a481f5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:28.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:28.076+0000 7f4a46000640 1 --2- 172.21.15.151:0/2651407532 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a48159aa0 0x7f4a48157aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:28.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:28.076+0000 7f4a47400640 1 -- 172.21.15.151:0/2651407532 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a48159aa0 msgr2=0x7f4a48157aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:28.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:28.076+0000 7f4a47400640 1 --2- 172.21.15.151:0/2651407532 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a48159aa0 0x7f4a48157aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:28.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:28.076+0000 7f4a47400640 1 -- 172.21.15.151:0/2651407532 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a4806f220 msgr2=0x7f4a481f5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:28.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:28.076+0000 7f4a47400640 1 --2- 172.21.15.151:0/2651407532 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a4806f220 0x7f4a481f5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:28.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:28.076+0000 7f4a47400640 1 -- 172.21.15.151:0/2651407532 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a481583f0 con 0x7f4a4815cff0 2024-09-16T15:40:28.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:28.076+0000 7f4a47400640 1 --2- 172.21.15.151:0/2651407532 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a4815cff0 0x7f4a48154410 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4a4002b020 tx=0x7f4a40003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:28.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:28.076+0000 7f4a3ea00640 1 -- 172.21.15.151:0/2651407532 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a40046070 con 0x7f4a4815cff0 2024-09-16T15:40:28.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:28.077+0000 7f4a3ea00640 1 -- 172.21.15.151:0/2651407532 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4a40004250 con 0x7f4a4815cff0 2024-09-16T15:40:28.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:28.077+0000 7f4a3ea00640 1 -- 172.21.15.151:0/2651407532 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a40033050 con 0x7f4a4815cff0 2024-09-16T15:40:28.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:28.077+0000 7f4a4da00640 1 -- 172.21.15.151:0/2651407532 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a481586d0 con 0x7f4a4815cff0 2024-09-16T15:40:28.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:28.077+0000 7f4a4da00640 1 -- 172.21.15.151:0/2651407532 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a481f4f90 con 0x7f4a4815cff0 2024-09-16T15:40:28.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:28.078+0000 7f4a4da00640 1 -- 172.21.15.151:0/2651407532 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a48069360 con 0x7f4a4815cff0 2024-09-16T15:40:28.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:28.079+0000 7f4a3ea00640 1 -- 172.21.15.151:0/2651407532 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 5) ==== 101287+0+0 (secure 0 0 0) 0x7f4a40049070 con 0x7f4a4815cff0 2024-09-16T15:40:28.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:28.080+0000 7f4a3ea00640 1 --2- 172.21.15.151:0/2651407532 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4a28078850 0x7f4a2807ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:40:28.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:28.080+0000 7f4a3ea00640 1 -- 172.21.15.151:0/2651407532 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f4a400bb8a0 con 0x7f4a4815cff0 2024-09-16T15:40:28.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:28.080+0000 7f4a46a00640 1 --2- 172.21.15.151:0/2651407532 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4a28078850 0x7f4a2807ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:40:28.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:28.081+0000 7f4a46a00640 1 --2- 172.21.15.151:0/2651407532 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4a28078850 0x7f4a2807ad10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4a38005fd0 tx=0x7f4a3800b390 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:40:28.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:28.084+0000 7f4a3ea00640 1 -- 172.21.15.151:0/2651407532 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4a40083820 con 0x7f4a4815cff0 2024-09-16T15:40:28.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:28.297+0000 7f4a4da00640 1 -- 172.21.15.151:0/2651407532 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "group_name": "group1", "target": ["mon-mgr", ""]}) -- 0x7f4a48070980 con 0x7f4a28078850 2024-09-16T15:40:28.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:28.304+0000 7f4a3ea00640 1 -- 172.21.15.151:0/2651407532 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+57 (secure 0 0 0) 0x7f4a48070980 con 0x7f4a28078850 2024-09-16T15:40:28.306 INFO:teuthology.orchestra.run.smithi151.stdout:/volumes/group1/sv1/09fc3174-3017-43a8-942f-69c9fd383ea0 2024-09-16T15:40:28.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:28.307+0000 7f4a4da00640 1 -- 172.21.15.151:0/2651407532 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4a28078850 msgr2=0x7f4a2807ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:28.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:28.307+0000 7f4a4da00640 1 --2- 172.21.15.151:0/2651407532 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4a28078850 0x7f4a2807ad10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4a38005fd0 tx=0x7f4a3800b390 comp rx=0 tx=0).stop 2024-09-16T15:40:28.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:28.307+0000 7f4a4da00640 1 -- 172.21.15.151:0/2651407532 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a4815cff0 msgr2=0x7f4a48154410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:40:28.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:28.307+0000 7f4a4da00640 1 --2- 172.21.15.151:0/2651407532 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a4815cff0 0x7f4a48154410 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4a4002b020 tx=0x7f4a40003f10 comp rx=0 tx=0).stop 2024-09-16T15:40:28.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:28.307+0000 7f4a4da00640 1 -- 172.21.15.151:0/2651407532 shutdown_connections 2024-09-16T15:40:28.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:28.307+0000 7f4a4da00640 1 --2- 172.21.15.151:0/2651407532 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4a28078850 0x7f4a2807ad10 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:28.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:28.307+0000 7f4a4da00640 1 --2- 172.21.15.151:0/2651407532 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a4815cff0 0x7f4a48154410 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:28.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:28.307+0000 7f4a4da00640 1 --2- 172.21.15.151:0/2651407532 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a48159aa0 0x7f4a48157aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:28.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:28.307+0000 7f4a4da00640 1 --2- 172.21.15.151:0/2651407532 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a4806f220 0x7f4a481f5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:40:28.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:28.307+0000 7f4a4da00640 1 -- 172.21.15.151:0/2651407532 >> 172.21.15.151:0/2651407532 conn(0x7f4a480b18e0 msgr2=0x7f4a48159f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:40:28.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:28.307+0000 7f4a4da00640 1 -- 172.21.15.151:0/2651407532 shutdown_connections 2024-09-16T15:40:28.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:40:28.307+0000 7f4a4da00640 1 -- 172.21.15.151:0/2651407532 wait complete. 2024-09-16T15:40:28.323 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 10 files each 1024MB size under directory volumes/group1/sv1/09fc3174-3017-43a8-942f-69c9fd383ea0 2024-09-16T15:40:28.324 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:40:28.324 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:40:28.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:40:28.359 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/09fc3174-3017-43a8-942f-69c9fd383ea0/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T15:40:36.293 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records in 2024-09-16T15:40:36.293 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records out 2024-09-16T15:40:36.293 INFO:teuthology.orchestra.run.smithi151.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.84718 s, 137 MB/s 2024-09-16T15:40:36.294 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:40:36.294 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:40:36.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:40:36.330 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/09fc3174-3017-43a8-942f-69c9fd383ea0/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T15:40:44.339 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records in 2024-09-16T15:40:44.339 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records out 2024-09-16T15:40:44.339 INFO:teuthology.orchestra.run.smithi151.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.93164 s, 135 MB/s 2024-09-16T15:40:44.340 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:40:44.340 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:40:44.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:40:44.376 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/09fc3174-3017-43a8-942f-69c9fd383ea0/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T15:40:47.068 DEBUG:teuthology.orchestra.run.smithi151:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:40:47.072 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:40:52.241 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records in 2024-09-16T15:40:52.242 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records out 2024-09-16T15:40:52.242 INFO:teuthology.orchestra.run.smithi151.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.7867 s, 138 MB/s 2024-09-16T15:40:52.243 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:40:52.243 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:40:52.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:40:52.278 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/09fc3174-3017-43a8-942f-69c9fd383ea0/subvolume_file.3 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T15:41:00.676 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records in 2024-09-16T15:41:00.676 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records out 2024-09-16T15:41:00.676 INFO:teuthology.orchestra.run.smithi151.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.32077 s, 129 MB/s 2024-09-16T15:41:00.678 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:41:00.678 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:41:00.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:41:00.714 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/09fc3174-3017-43a8-942f-69c9fd383ea0/subvolume_file.4 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T15:41:09.071 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records in 2024-09-16T15:41:09.071 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records out 2024-09-16T15:41:09.071 INFO:teuthology.orchestra.run.smithi151.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.28091 s, 130 MB/s 2024-09-16T15:41:09.072 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:41:09.072 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:41:09.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:41:09.108 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/09fc3174-3017-43a8-942f-69c9fd383ea0/subvolume_file.5 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T15:41:17.171 DEBUG:teuthology.orchestra.run.smithi151:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:41:17.175 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:41:17.183 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records in 2024-09-16T15:41:17.184 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records out 2024-09-16T15:41:17.184 INFO:teuthology.orchestra.run.smithi151.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.99949 s, 134 MB/s 2024-09-16T15:41:17.185 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:41:17.185 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:41:17.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:41:17.261 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/09fc3174-3017-43a8-942f-69c9fd383ea0/subvolume_file.6 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T15:41:25.454 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records in 2024-09-16T15:41:25.455 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records out 2024-09-16T15:41:25.455 INFO:teuthology.orchestra.run.smithi151.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.11734 s, 132 MB/s 2024-09-16T15:41:25.456 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:41:25.456 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:41:25.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:41:25.493 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/09fc3174-3017-43a8-942f-69c9fd383ea0/subvolume_file.7 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T15:41:33.686 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records in 2024-09-16T15:41:33.687 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records out 2024-09-16T15:41:33.687 INFO:teuthology.orchestra.run.smithi151.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.11752 s, 132 MB/s 2024-09-16T15:41:33.688 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:41:33.688 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:41:33.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:41:33.723 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/09fc3174-3017-43a8-942f-69c9fd383ea0/subvolume_file.8 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T15:41:41.862 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records in 2024-09-16T15:41:41.862 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records out 2024-09-16T15:41:41.862 INFO:teuthology.orchestra.run.smithi151.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.05852 s, 133 MB/s 2024-09-16T15:41:41.864 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:41:41.864 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:41:41.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:41:41.901 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/group1/sv1/09fc3174-3017-43a8-942f-69c9fd383ea0/subvolume_file.9 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T15:41:47.268 DEBUG:teuthology.orchestra.run.smithi151:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:41:47.274 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:41:50.198 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records in 2024-09-16T15:41:50.198 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records out 2024-09-16T15:41:50.198 INFO:teuthology.orchestra.run.smithi151.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.22104 s, 131 MB/s 2024-09-16T15:41:50.199 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 group1 2024-09-16T15:41:50.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.363+0000 7f24a9e00640 1 -- 172.21.15.151:0/2739360100 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24a406efa0 msgr2=0x7f24a4073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:50.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.363+0000 7f24a9e00640 1 --2- 172.21.15.151:0/2739360100 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24a406efa0 0x7f24a4073440 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f249c009830 tx=0x7f249c02a4b0 comp rx=0 tx=0).stop 2024-09-16T15:41:50.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.363+0000 7f24a9e00640 1 -- 172.21.15.151:0/2739360100 shutdown_connections 2024-09-16T15:41:50.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.363+0000 7f24a9e00640 1 --2- 172.21.15.151:0/2739360100 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24a406efa0 0x7f24a4073440 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:50.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.363+0000 7f24a9e00640 1 -- 172.21.15.151:0/2739360100 >> 172.21.15.151:0/2739360100 conn(0x7f24a40b1840 msgr2=0x7f24a40b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:50.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.363+0000 7f24a9e00640 1 -- 172.21.15.151:0/2739360100 shutdown_connections 2024-09-16T15:41:50.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.364+0000 7f24a9e00640 1 -- 172.21.15.151:0/2739360100 wait complete. 2024-09-16T15:41:50.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.365+0000 7f24a9e00640 1 Processor -- start 2024-09-16T15:41:50.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.365+0000 7f24a9e00640 1 -- start start 2024-09-16T15:41:50.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.366+0000 7f24a9e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24a406efa0 0x7f24a413bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:50.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.366+0000 7f24a9e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f24a40cccd0 0x7f24a415d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:50.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.366+0000 7f24a9e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24a4145750 0x7f24a415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:50.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.366+0000 7f24a9e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f24a40ce6a0 con 0x7f24a40cccd0 2024-09-16T15:41:50.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.366+0000 7f24a9e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f24a40ce520 con 0x7f24a4145750 2024-09-16T15:41:50.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.366+0000 7f24a9e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f24a40ce820 con 0x7f24a406efa0 2024-09-16T15:41:50.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.366+0000 7f24a2000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f24a40cccd0 0x7f24a415d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:50.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.367+0000 7f24a2000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f24a40cccd0 0x7f24a415d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41512/0 (socket says 172.21.15.151:41512) 2024-09-16T15:41:50.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.367+0000 7f24a2000640 1 -- 172.21.15.151:0/3510359695 learned_addr learned my addr 172.21.15.151:0/3510359695 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:41:50.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.367+0000 7f24a2a00640 1 --2- 172.21.15.151:0/3510359695 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24a406efa0 0x7f24a413bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:50.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.367+0000 7f24a3400640 1 --2- 172.21.15.151:0/3510359695 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24a4145750 0x7f24a415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:50.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.367+0000 7f24a2000640 1 -- 172.21.15.151:0/3510359695 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24a406efa0 msgr2=0x7f24a413bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:50.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.367+0000 7f24a2000640 1 --2- 172.21.15.151:0/3510359695 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24a406efa0 0x7f24a413bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:50.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.367+0000 7f24a2000640 1 -- 172.21.15.151:0/3510359695 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24a4145750 msgr2=0x7f24a415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:50.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.367+0000 7f24a2000640 1 --2- 172.21.15.151:0/3510359695 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24a4145750 0x7f24a415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:50.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.367+0000 7f24a2000640 1 -- 172.21.15.151:0/3510359695 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24a413c210 con 0x7f24a40cccd0 2024-09-16T15:41:50.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.367+0000 7f24a2000640 1 --2- 172.21.15.151:0/3510359695 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f24a40cccd0 0x7f24a415d090 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f249000c700 tx=0x7f249000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:50.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.367+0000 7f249b400640 1 -- 172.21.15.151:0/3510359695 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2490016020 con 0x7f24a40cccd0 2024-09-16T15:41:50.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.367+0000 7f249b400640 1 -- 172.21.15.151:0/3510359695 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f249000a930 con 0x7f24a40cccd0 2024-09-16T15:41:50.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.367+0000 7f24a9e00640 1 -- 172.21.15.151:0/3510359695 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24a4141250 con 0x7f24a40cccd0 2024-09-16T15:41:50.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.368+0000 7f249b400640 1 -- 172.21.15.151:0/3510359695 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249000adc0 con 0x7f24a40cccd0 2024-09-16T15:41:50.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.368+0000 7f24a9e00640 1 -- 172.21.15.151:0/3510359695 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24a415ba20 con 0x7f24a40cccd0 2024-09-16T15:41:50.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.369+0000 7f24a9e00640 1 -- 172.21.15.151:0/3510359695 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24a413fc60 con 0x7f24a40cccd0 2024-09-16T15:41:50.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.370+0000 7f249b400640 1 -- 172.21.15.151:0/3510359695 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 7) ==== 101287+0+0 (secure 0 0 0) 0x7f2490019020 con 0x7f24a40cccd0 2024-09-16T15:41:50.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.371+0000 7f249b400640 1 --2- 172.21.15.151:0/3510359695 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2484078850 0x7f248407ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:50.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.371+0000 7f249b400640 1 -- 172.21.15.151:0/3510359695 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5825+0+0 (secure 0 0 0) 0x7f249009a770 con 0x7f24a40cccd0 2024-09-16T15:41:50.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.371+0000 7f24a2a00640 1 --2- 172.21.15.151:0/3510359695 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2484078850 0x7f248407ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:50.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.372+0000 7f24a2a00640 1 --2- 172.21.15.151:0/3510359695 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2484078850 0x7f248407ad10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f24940014a0 tx=0x7f2494009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:50.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.375+0000 7f249b400640 1 -- 172.21.15.151:0/3510359695 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2490020030 con 0x7f24a40cccd0 2024-09-16T15:41:50.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.592+0000 7f24a9e00640 1 -- 172.21.15.151:0/3510359695 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "group_name": "group1", "target": ["mon-mgr", ""]}) -- 0x7f24a40b1f10 con 0x7f2484078850 2024-09-16T15:41:50.598 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:41:50.597+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:41:50.598 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:41:50.597+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:41:50.598 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:41:50.597+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:41:50.598 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:41:50.597+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:41:50.599 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:41:50.597+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:41:50.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.618+0000 7f249b400640 1 -- 172.21.15.151:0/3510359695 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f24a40b1f10 con 0x7f2484078850 2024-09-16T15:41:50.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.620+0000 7f24a9e00640 1 -- 172.21.15.151:0/3510359695 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2484078850 msgr2=0x7f248407ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:50.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.620+0000 7f24a9e00640 1 --2- 172.21.15.151:0/3510359695 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2484078850 0x7f248407ad10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f24940014a0 tx=0x7f2494009040 comp rx=0 tx=0).stop 2024-09-16T15:41:50.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.620+0000 7f24a9e00640 1 -- 172.21.15.151:0/3510359695 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f24a40cccd0 msgr2=0x7f24a415d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:50.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.620+0000 7f24a9e00640 1 --2- 172.21.15.151:0/3510359695 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f24a40cccd0 0x7f24a415d090 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f249000c700 tx=0x7f249000cbc0 comp rx=0 tx=0).stop 2024-09-16T15:41:50.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.620+0000 7f24a9e00640 1 -- 172.21.15.151:0/3510359695 shutdown_connections 2024-09-16T15:41:50.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.620+0000 7f24a9e00640 1 --2- 172.21.15.151:0/3510359695 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2484078850 0x7f248407ad10 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:50.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.620+0000 7f24a9e00640 1 --2- 172.21.15.151:0/3510359695 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24a4145750 0x7f24a415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:50.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.620+0000 7f24a9e00640 1 --2- 172.21.15.151:0/3510359695 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f24a40cccd0 0x7f24a415d090 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:50.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.620+0000 7f24a9e00640 1 --2- 172.21.15.151:0/3510359695 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24a406efa0 0x7f24a413bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:50.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.620+0000 7f24a9e00640 1 -- 172.21.15.151:0/3510359695 >> 172.21.15.151:0/3510359695 conn(0x7f24a40b1840 msgr2=0x7f24a40b0d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:50.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.621+0000 7f24a9e00640 1 -- 172.21.15.151:0/3510359695 shutdown_connections 2024-09-16T15:41:50.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.621+0000 7f24a9e00640 1 -- 172.21.15.151:0/3510359695 wait complete. 2024-09-16T15:41:50.637 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 group1 2024-09-16T15:41:50.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.800+0000 7f5871a00640 1 -- 172.21.15.151:0/1327388637 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5850004800 msgr2=0x7f5850004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:50.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.800+0000 7f5871a00640 1 --2- 172.21.15.151:0/1327388637 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5850004800 0x7f5850004c80 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5858001830 tx=0x7f5850005830 comp rx=0 tx=0).stop 2024-09-16T15:41:50.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.800+0000 7f5871a00640 1 -- 172.21.15.151:0/1327388637 shutdown_connections 2024-09-16T15:41:50.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.800+0000 7f5871a00640 1 --2- 172.21.15.151:0/1327388637 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5850004800 0x7f5850004c80 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:50.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.800+0000 7f5871a00640 1 -- 172.21.15.151:0/1327388637 >> 172.21.15.151:0/1327388637 conn(0x7f586c0b18e0 msgr2=0x7f586c0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:50.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.800+0000 7f5871a00640 1 -- 172.21.15.151:0/1327388637 shutdown_connections 2024-09-16T15:41:50.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.801+0000 7f5871a00640 1 -- 172.21.15.151:0/1327388637 wait complete. 2024-09-16T15:41:50.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.802+0000 7f5871a00640 1 Processor -- start 2024-09-16T15:41:50.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.802+0000 7f5871a00640 1 -- start start 2024-09-16T15:41:50.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.803+0000 7f5871a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5850004800 0x7f586c0cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:50.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.803+0000 7f5871a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f586c0ca540 0x7f586c15e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:50.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.803+0000 7f5871a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f586c0c92b0 0x7f586c0c9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:50.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.803+0000 7f5871a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f586c0cd550 con 0x7f5850004800 2024-09-16T15:41:50.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.803+0000 7f5871a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f586c0cd3d0 con 0x7f586c0ca540 2024-09-16T15:41:50.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.803+0000 7f5871a00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f586c15fd80 con 0x7f586c0c92b0 2024-09-16T15:41:50.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.803+0000 7f586aa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5850004800 0x7f586c0cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:50.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.803+0000 7f586aa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5850004800 0x7f586c0cf0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41518/0 (socket says 172.21.15.151:41518) 2024-09-16T15:41:50.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.803+0000 7f586aa00640 1 -- 172.21.15.151:0/964593284 learned_addr learned my addr 172.21.15.151:0/964593284 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:41:50.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.804+0000 7f586a000640 1 --2- 172.21.15.151:0/964593284 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f586c0ca540 0x7f586c15e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:50.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.804+0000 7f586b400640 1 --2- 172.21.15.151:0/964593284 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f586c0c92b0 0x7f586c0c9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:50.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.804+0000 7f586aa00640 1 -- 172.21.15.151:0/964593284 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f586c0c92b0 msgr2=0x7f586c0c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:50.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.804+0000 7f586aa00640 1 --2- 172.21.15.151:0/964593284 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f586c0c92b0 0x7f586c0c9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:50.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.804+0000 7f586aa00640 1 -- 172.21.15.151:0/964593284 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f586c0ca540 msgr2=0x7f586c15e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:50.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.804+0000 7f586aa00640 1 --2- 172.21.15.151:0/964593284 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f586c0ca540 0x7f586c15e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:50.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.804+0000 7f586aa00640 1 -- 172.21.15.151:0/964593284 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f586c0cf5f0 con 0x7f5850004800 2024-09-16T15:41:50.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.804+0000 7f586b400640 1 --2- 172.21.15.151:0/964593284 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f586c0c92b0 0x7f586c0c9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:41:50.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.804+0000 7f586aa00640 1 --2- 172.21.15.151:0/964593284 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5850004800 0x7f586c0cf0b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f585c011a50 tx=0x7f585c011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:50.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.804+0000 7f5862a00640 1 -- 172.21.15.151:0/964593284 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585c009720 con 0x7f5850004800 2024-09-16T15:41:50.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.804+0000 7f5862a00640 1 -- 172.21.15.151:0/964593284 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f585c00fd10 con 0x7f5850004800 2024-09-16T15:41:50.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.804+0000 7f5862a00640 1 -- 172.21.15.151:0/964593284 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585c005280 con 0x7f5850004800 2024-09-16T15:41:50.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.804+0000 7f5871a00640 1 -- 172.21.15.151:0/964593284 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f586c1f55f0 con 0x7f5850004800 2024-09-16T15:41:50.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.805+0000 7f5871a00640 1 -- 172.21.15.151:0/964593284 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f586c1f5030 con 0x7f5850004800 2024-09-16T15:41:50.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.806+0000 7f5871a00640 1 -- 172.21.15.151:0/964593284 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f586c069360 con 0x7f5850004800 2024-09-16T15:41:50.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.807+0000 7f5862a00640 1 -- 172.21.15.151:0/964593284 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 7) ==== 101287+0+0 (secure 0 0 0) 0x7f585c005420 con 0x7f5850004800 2024-09-16T15:41:50.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.808+0000 7f5862a00640 1 --2- 172.21.15.151:0/964593284 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f584c078850 0x7f584c07ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:50.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.808+0000 7f5862a00640 1 -- 172.21.15.151:0/964593284 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5825+0+0 (secure 0 0 0) 0x7f585c09b2d0 con 0x7f5850004800 2024-09-16T15:41:50.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.808+0000 7f586a000640 1 --2- 172.21.15.151:0/964593284 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f584c078850 0x7f584c07ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:50.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.809+0000 7f586a000640 1 --2- 172.21.15.151:0/964593284 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f584c078850 0x7f584c07ad10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5858032000 tx=0x7f5858001360 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:50.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:50.812+0000 7f5862a00640 1 -- 172.21.15.151:0/964593284 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f585c0631d0 con 0x7f5850004800 2024-09-16T15:41:51.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:51.025+0000 7f5871a00640 1 -- 172.21.15.151:0/964593284 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "group_name": "group1", "target": ["mon-mgr", ""]}) -- 0x7f586c0cd7b0 con 0x7f584c078850 2024-09-16T15:41:51.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:51.031+0000 7f5862a00640 1 -- 172.21.15.151:0/964593284 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+57 (secure 0 0 0) 0x7f586c0cd7b0 con 0x7f584c078850 2024-09-16T15:41:51.032 INFO:teuthology.orchestra.run.smithi151.stdout:/volumes/group1/sv1/09fc3174-3017-43a8-942f-69c9fd383ea0 2024-09-16T15:41:51.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:51.033+0000 7f5871a00640 1 -- 172.21.15.151:0/964593284 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f584c078850 msgr2=0x7f584c07ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:51.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:51.034+0000 7f5871a00640 1 --2- 172.21.15.151:0/964593284 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f584c078850 0x7f584c07ad10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5858032000 tx=0x7f5858001360 comp rx=0 tx=0).stop 2024-09-16T15:41:51.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:51.034+0000 7f5871a00640 1 -- 172.21.15.151:0/964593284 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5850004800 msgr2=0x7f586c0cf0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:51.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:51.034+0000 7f5871a00640 1 --2- 172.21.15.151:0/964593284 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5850004800 0x7f586c0cf0b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f585c011a50 tx=0x7f585c011f10 comp rx=0 tx=0).stop 2024-09-16T15:41:51.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:51.034+0000 7f5871a00640 1 -- 172.21.15.151:0/964593284 shutdown_connections 2024-09-16T15:41:51.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:51.034+0000 7f5871a00640 1 --2- 172.21.15.151:0/964593284 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f584c078850 0x7f584c07ad10 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:51.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:51.034+0000 7f5871a00640 1 --2- 172.21.15.151:0/964593284 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f586c0ca540 0x7f586c15e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:51.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:51.034+0000 7f5871a00640 1 --2- 172.21.15.151:0/964593284 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f586c0c92b0 0x7f586c0c9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:51.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:51.034+0000 7f5871a00640 1 --2- 172.21.15.151:0/964593284 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5850004800 0x7f586c0cf0b0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:51.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:51.034+0000 7f5871a00640 1 -- 172.21.15.151:0/964593284 >> 172.21.15.151:0/964593284 conn(0x7f586c0b18e0 msgr2=0x7f586c0b2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:51.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:51.034+0000 7f5871a00640 1 -- 172.21.15.151:0/964593284 shutdown_connections 2024-09-16T15:41:51.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:51.034+0000 7f5871a00640 1 -- 172.21.15.151:0/964593284 wait complete. 2024-09-16T15:41:51.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:41:51.050 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/group1/sv1/09fc3174-3017-43a8-942f-69c9fd383ea0 2024-09-16T15:41:51.050 DEBUG:teuthology.orchestra.run.smithi151:> ) 2024-09-16T15:41:51.086 INFO:teuthology.orchestra.run.smithi151.stdout:# file: volumes/group1/sv1/09fc3174-3017-43a8-942f-69c9fd383ea0 2024-09-16T15:41:51.086 INFO:teuthology.orchestra.run.smithi151.stdout:ceph.dir.rbytes="10405347328" 2024-09-16T15:41:51.086 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:41:51.087 INFO:tasks.cephfs.test_volumes:r_size = 10405347328 exp_size = 10737418240 2024-09-16T15:41:53.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:41:53.088 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/group1/sv1/09fc3174-3017-43a8-942f-69c9fd383ea0 2024-09-16T15:41:53.088 DEBUG:teuthology.orchestra.run.smithi151:> ) 2024-09-16T15:41:53.129 INFO:teuthology.orchestra.run.smithi151.stdout:# file: volumes/group1/sv1/09fc3174-3017-43a8-942f-69c9fd383ea0 2024-09-16T15:41:53.129 INFO:teuthology.orchestra.run.smithi151.stdout:ceph.dir.rbytes="10405347328" 2024-09-16T15:41:53.130 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:41:53.131 INFO:tasks.cephfs.test_volumes:r_size = 10405347328 exp_size = 10737418240 2024-09-16T15:41:55.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:41:55.132 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/group1/sv1/09fc3174-3017-43a8-942f-69c9fd383ea0 2024-09-16T15:41:55.132 DEBUG:teuthology.orchestra.run.smithi151:> ) 2024-09-16T15:41:55.172 INFO:teuthology.orchestra.run.smithi151.stdout:# file: volumes/group1/sv1/09fc3174-3017-43a8-942f-69c9fd383ea0 2024-09-16T15:41:55.172 INFO:teuthology.orchestra.run.smithi151.stdout:ceph.dir.rbytes="10405347328" 2024-09-16T15:41:55.172 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:41:55.173 INFO:tasks.cephfs.test_volumes:r_size = 10405347328 exp_size = 10737418240 2024-09-16T15:41:57.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:41:57.174 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/group1/sv1/09fc3174-3017-43a8-942f-69c9fd383ea0 2024-09-16T15:41:57.174 DEBUG:teuthology.orchestra.run.smithi151:> ) 2024-09-16T15:41:57.215 INFO:teuthology.orchestra.run.smithi151.stdout:# file: volumes/group1/sv1/09fc3174-3017-43a8-942f-69c9fd383ea0 2024-09-16T15:41:57.215 INFO:teuthology.orchestra.run.smithi151.stdout:ceph.dir.rbytes="10737418240" 2024-09-16T15:41:57.215 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:41:57.216 INFO:tasks.cephfs.test_volumes:r_size = 10737418240 exp_size = 10737418240 2024-09-16T15:41:57.216 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 ss1clone1 --group-name group1 2024-09-16T15:41:57.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.428+0000 7fdd26600640 1 -- 172.21.15.151:0/3013563725 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdd20152e30 msgr2=0x7fdd200d00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:57.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.428+0000 7fdd26600640 1 --2- 172.21.15.151:0/3013563725 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdd20152e30 0x7fdd200d00c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fdd2009e080 tx=0x7fdd1002a8b0 comp rx=0 tx=0).stop 2024-09-16T15:41:57.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.428+0000 7fdd26600640 1 -- 172.21.15.151:0/3013563725 shutdown_connections 2024-09-16T15:41:57.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.428+0000 7fdd26600640 1 --2- 172.21.15.151:0/3013563725 >> v2:172.21.15.167:3300/0 conn(0x7fdd200cd210 0x7fdd200d0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:57.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.428+0000 7fdd26600640 1 --2- 172.21.15.151:0/3013563725 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdd20152e30 0x7fdd200d00c0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:57.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.428+0000 7fdd26600640 1 -- 172.21.15.151:0/3013563725 >> 172.21.15.151:0/3013563725 conn(0x7fdd2006cf90 msgr2=0x7fdd2006d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:57.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.429+0000 7fdd26600640 1 -- 172.21.15.151:0/3013563725 shutdown_connections 2024-09-16T15:41:57.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.429+0000 7fdd26600640 1 -- 172.21.15.151:0/3013563725 wait complete. 2024-09-16T15:41:57.434 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.430+0000 7fdd26600640 1 Processor -- start 2024-09-16T15:41:57.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.430+0000 7fdd26600640 1 -- start start 2024-09-16T15:41:57.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.431+0000 7fdd26600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdd200cd210 0x7fdd20145090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:57.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.431+0000 7fdd26600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdd200cecc0 0x7fdd20155530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:57.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.431+0000 7fdd26600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd20152e30 0x7fdd20155a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:57.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.431+0000 7fdd26600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdd2006f7d0 con 0x7fdd200cd210 2024-09-16T15:41:57.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.431+0000 7fdd26600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdd2006f650 con 0x7fdd20152e30 2024-09-16T15:41:57.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.431+0000 7fdd26600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fdd2006f950 con 0x7fdd200cecc0 2024-09-16T15:41:57.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.431+0000 7fdd1f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdd200cd210 0x7fdd20145090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:57.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.431+0000 7fdd1f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdd200cd210 0x7fdd20145090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:46050/0 (socket says 172.21.15.151:46050) 2024-09-16T15:41:57.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.431+0000 7fdd1f400640 1 -- 172.21.15.151:0/1777764890 learned_addr learned my addr 172.21.15.151:0/1777764890 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:41:57.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.432+0000 7fdd1f400640 1 -- 172.21.15.151:0/1777764890 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdd200cecc0 msgr2=0x7fdd20155530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:57.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.432+0000 7fdd1f400640 1 --2- 172.21.15.151:0/1777764890 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdd200cecc0 0x7fdd20155530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:57.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.432+0000 7fdd1f400640 1 -- 172.21.15.151:0/1777764890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd20152e30 msgr2=0x7fdd20155a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:57.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.432+0000 7fdd1f400640 1 --2- 172.21.15.151:0/1777764890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd20152e30 0x7fdd20155a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:57.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.432+0000 7fdd1f400640 1 -- 172.21.15.151:0/1777764890 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd201f2100 con 0x7fdd200cd210 2024-09-16T15:41:57.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.432+0000 7fdd1f400640 1 --2- 172.21.15.151:0/1777764890 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdd200cd210 0x7fdd20145090 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fdd10009ca0 tx=0x7fdd10003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:57.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.432+0000 7fdd17400640 1 -- 172.21.15.151:0/1777764890 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd10031da0 con 0x7fdd200cd210 2024-09-16T15:41:57.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.432+0000 7fdd17400640 1 -- 172.21.15.151:0/1777764890 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdd10003e10 con 0x7fdd200cd210 2024-09-16T15:41:57.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.432+0000 7fdd26600640 1 -- 172.21.15.151:0/1777764890 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd201455d0 con 0x7fdd200cd210 2024-09-16T15:41:57.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.432+0000 7fdd17400640 1 -- 172.21.15.151:0/1777764890 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd10033310 con 0x7fdd200cd210 2024-09-16T15:41:57.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.433+0000 7fdd26600640 1 -- 172.21.15.151:0/1777764890 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd201f60c0 con 0x7fdd200cd210 2024-09-16T15:41:57.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.434+0000 7fdd26600640 1 -- 172.21.15.151:0/1777764890 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd2006fe00 con 0x7fdd200cd210 2024-09-16T15:41:57.437 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.435+0000 7fdd17400640 1 -- 172.21.15.151:0/1777764890 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 8) ==== 101338+0+0 (secure 0 0 0) 0x7fdd100334b0 con 0x7fdd200cd210 2024-09-16T15:41:57.437 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.436+0000 7fdd17400640 1 --2- 172.21.15.151:0/1777764890 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdd000788e0 0x7fdd0007ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:57.437 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.436+0000 7fdd17400640 1 -- 172.21.15.151:0/1777764890 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5825+0+0 (secure 0 0 0) 0x7fdd100bc160 con 0x7fdd200cd210 2024-09-16T15:41:57.437 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.436+0000 7fdd1ea00640 1 --2- 172.21.15.151:0/1777764890 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdd000788e0 0x7fdd0007ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:57.438 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.437+0000 7fdd1ea00640 1 --2- 172.21.15.151:0/1777764890 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdd000788e0 0x7fdd0007ada0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdd201526a0 tx=0x7fdd0c005c70 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:57.441 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.440+0000 7fdd17400640 1 -- 172.21.15.151:0/1777764890 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fdd1003f030 con 0x7fdd200cd210 2024-09-16T15:41:57.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.657+0000 7fdd26600640 1 -- 172.21.15.151:0/1777764890 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "ss1clone1", "group_name": "group1", "target": ["mon-mgr", ""]}) -- 0x7fdd2014f440 con 0x7fdd000788e0 2024-09-16T15:41:57.738 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.737+0000 7fdd17400640 1 -- 172.21.15.151:0/1777764890 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fdd2014f440 con 0x7fdd000788e0 2024-09-16T15:41:57.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.739+0000 7fdd26600640 1 -- 172.21.15.151:0/1777764890 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdd000788e0 msgr2=0x7fdd0007ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:57.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.739+0000 7fdd26600640 1 --2- 172.21.15.151:0/1777764890 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdd000788e0 0x7fdd0007ada0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdd201526a0 tx=0x7fdd0c005c70 comp rx=0 tx=0).stop 2024-09-16T15:41:57.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.739+0000 7fdd26600640 1 -- 172.21.15.151:0/1777764890 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdd200cd210 msgr2=0x7fdd20145090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:57.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.739+0000 7fdd26600640 1 --2- 172.21.15.151:0/1777764890 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdd200cd210 0x7fdd20145090 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fdd10009ca0 tx=0x7fdd10003c20 comp rx=0 tx=0).stop 2024-09-16T15:41:57.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.739+0000 7fdd26600640 1 -- 172.21.15.151:0/1777764890 shutdown_connections 2024-09-16T15:41:57.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.739+0000 7fdd26600640 1 --2- 172.21.15.151:0/1777764890 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdd000788e0 0x7fdd0007ada0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:57.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.739+0000 7fdd26600640 1 --2- 172.21.15.151:0/1777764890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd20152e30 0x7fdd20155a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:57.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.739+0000 7fdd26600640 1 --2- 172.21.15.151:0/1777764890 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdd200cecc0 0x7fdd20155530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:57.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.739+0000 7fdd26600640 1 --2- 172.21.15.151:0/1777764890 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdd200cd210 0x7fdd20145090 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:57.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.739+0000 7fdd26600640 1 -- 172.21.15.151:0/1777764890 >> 172.21.15.151:0/1777764890 conn(0x7fdd2006cf90 msgr2=0x7fdd200b2c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:57.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.740+0000 7fdd26600640 1 -- 172.21.15.151:0/1777764890 shutdown_connections 2024-09-16T15:41:57.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.740+0000 7fdd26600640 1 -- 172.21.15.151:0/1777764890 wait complete. 2024-09-16T15:41:57.742 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:41:57.741+0000 7f48f6200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:41:57.742 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:41:57.741+0000 7f48f6200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:41:57.742 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:41:57.741+0000 7f48f6200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:41:57.743 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:41:57.741+0000 7f48f6200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:41:57.743 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:41:57.741+0000 7f48f6200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:41:57.758 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-16T15:41:57.761 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:41:57.759+0000 7f48f7600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:41:57.761 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:41:57.759+0000 7f48f7600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:41:57.761 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:41:57.759+0000 7f48f7600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:41:57.761 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:41:57.759+0000 7f48f7600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:41:57.761 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:41:57.759+0000 7f48f7600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:41:57.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.914+0000 7f1d5a200640 1 -- 172.21.15.151:0/2744860185 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d5415cff0 msgr2=0x7f1d5406e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:57.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.914+0000 7f1d5a200640 1 --2- 172.21.15.151:0/2744860185 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d5415cff0 0x7f1d5406e6f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1d540b2680 tx=0x7f1d4002a530 comp rx=0 tx=0).stop 2024-09-16T15:41:57.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.914+0000 7f1d5a200640 1 -- 172.21.15.151:0/2744860185 shutdown_connections 2024-09-16T15:41:57.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.914+0000 7f1d5a200640 1 --2- 172.21.15.151:0/2744860185 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d5415cff0 0x7f1d5406e6f0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:57.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.914+0000 7f1d5a200640 1 --2- 172.21.15.151:0/2744860185 >> v2:172.21.15.167:3300/0 conn(0x7f1d54159aa0 0x7f1d540741a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:57.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.914+0000 7f1d5a200640 1 -- 172.21.15.151:0/2744860185 >> 172.21.15.151:0/2744860185 conn(0x7f1d540b18e0 msgr2=0x7f1d540b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:57.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.914+0000 7f1d5a200640 1 -- 172.21.15.151:0/2744860185 shutdown_connections 2024-09-16T15:41:57.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.914+0000 7f1d5a200640 1 -- 172.21.15.151:0/2744860185 wait complete. 2024-09-16T15:41:57.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.916+0000 7f1d5a200640 1 Processor -- start 2024-09-16T15:41:57.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.916+0000 7f1d5a200640 1 -- start start 2024-09-16T15:41:57.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.917+0000 7f1d5a200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1d5406f220 0x7f1d540cff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:57.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.917+0000 7f1d5a200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d54159aa0 0x7f1d5414c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:57.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.917+0000 7f1d5a200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d5415cff0 0x7f1d540ae370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:57.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.917+0000 7f1d5a200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1d540ccc30 con 0x7f1d5415cff0 2024-09-16T15:41:57.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.917+0000 7f1d5a200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1d540ccab0 con 0x7f1d54159aa0 2024-09-16T15:41:57.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.917+0000 7f1d5a200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f1d540ccdb0 con 0x7f1d5406f220 2024-09-16T15:41:57.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.917+0000 7f1d53e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d5415cff0 0x7f1d540ae370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:57.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.917+0000 7f1d53e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d5415cff0 0x7f1d540ae370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:46080/0 (socket says 172.21.15.151:46080) 2024-09-16T15:41:57.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.917+0000 7f1d53e00640 1 -- 172.21.15.151:0/334116176 learned_addr learned my addr 172.21.15.151:0/334116176 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:41:57.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.917+0000 7f1d53400640 1 --2- 172.21.15.151:0/334116176 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1d5406f220 0x7f1d540cff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:57.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.917+0000 7f1d52a00640 1 --2- 172.21.15.151:0/334116176 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d54159aa0 0x7f1d5414c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:57.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.918+0000 7f1d53e00640 1 -- 172.21.15.151:0/334116176 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1d5406f220 msgr2=0x7f1d540cff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:57.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.918+0000 7f1d53e00640 1 --2- 172.21.15.151:0/334116176 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1d5406f220 0x7f1d540cff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:57.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.918+0000 7f1d53e00640 1 -- 172.21.15.151:0/334116176 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d54159aa0 msgr2=0x7f1d5414c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:57.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.918+0000 7f1d53e00640 1 --2- 172.21.15.151:0/334116176 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d54159aa0 0x7f1d5414c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:57.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.918+0000 7f1d53e00640 1 -- 172.21.15.151:0/334116176 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d540cdf40 con 0x7f1d5415cff0 2024-09-16T15:41:57.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.918+0000 7f1d53e00640 1 --2- 172.21.15.151:0/334116176 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d5415cff0 0x7f1d540ae370 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c00c770 tx=0x7f1d4c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:57.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.918+0000 7f1d4b400640 1 -- 172.21.15.151:0/334116176 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d4c017070 con 0x7f1d5415cff0 2024-09-16T15:41:57.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.918+0000 7f1d4b400640 1 -- 172.21.15.151:0/334116176 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1d4c005070 con 0x7f1d5415cff0 2024-09-16T15:41:57.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.918+0000 7f1d4b400640 1 -- 172.21.15.151:0/334116176 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d4c0054c0 con 0x7f1d5415cff0 2024-09-16T15:41:57.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.918+0000 7f1d5a200640 1 -- 172.21.15.151:0/334116176 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d540ce220 con 0x7f1d5415cff0 2024-09-16T15:41:57.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.918+0000 7f1d5a200640 1 -- 172.21.15.151:0/334116176 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d540ce7e0 con 0x7f1d5415cff0 2024-09-16T15:41:57.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.919+0000 7f1d5a200640 1 -- 172.21.15.151:0/334116176 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d5406f9c0 con 0x7f1d5415cff0 2024-09-16T15:41:57.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.920+0000 7f1d4b400640 1 -- 172.21.15.151:0/334116176 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 8) ==== 101338+0+0 (secure 0 0 0) 0x7f1d4c01a070 con 0x7f1d5415cff0 2024-09-16T15:41:57.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.921+0000 7f1d4b400640 1 --2- 172.21.15.151:0/334116176 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1d340788e0 0x7f1d3407ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:57.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.921+0000 7f1d4b400640 1 -- 172.21.15.151:0/334116176 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5825+0+0 (secure 0 0 0) 0x7f1d4c09a110 con 0x7f1d5415cff0 2024-09-16T15:41:57.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.922+0000 7f1d53400640 1 --2- 172.21.15.151:0/334116176 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1d340788e0 0x7f1d3407ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:57.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.922+0000 7f1d53400640 1 --2- 172.21.15.151:0/334116176 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1d340788e0 0x7f1d3407ada0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1d5414cec0 tx=0x7f1d4400b390 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:57.926 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:57.925+0000 7f1d4b400640 1 -- 172.21.15.151:0/334116176 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1d4c061f40 con 0x7f1d5415cff0 2024-09-16T15:41:58.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:58.180+0000 7f1d5a200640 1 -- 172.21.15.151:0/334116176 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f1d5415d9d0 con 0x7f1d5415cff0 2024-09-16T15:41:58.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:58.180+0000 7f1d4b400640 1 -- 172.21.15.151:0/334116176 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1912 (secure 0 0 0) 0x7f1d4c066970 con 0x7f1d5415cff0 2024-09-16T15:41:58.181 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:41:58.181 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-16T15:41:58.182 INFO:teuthology.orchestra.run.smithi151.stdout: "fsid": "1757be18-e41a-4446-bb4f-ee4ac81c39df", 2024-09-16T15:41:58.182 INFO:teuthology.orchestra.run.smithi151.stdout: "health": { 2024-09-16T15:41:58.182 INFO:teuthology.orchestra.run.smithi151.stdout: "status": "HEALTH_OK", 2024-09-16T15:41:58.182 INFO:teuthology.orchestra.run.smithi151.stdout: "checks": {}, 2024-09-16T15:41:58.182 INFO:teuthology.orchestra.run.smithi151.stdout: "mutes": [] 2024-09-16T15:41:58.182 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:41:58.182 INFO:teuthology.orchestra.run.smithi151.stdout: "election_epoch": 4, 2024-09-16T15:41:58.182 INFO:teuthology.orchestra.run.smithi151.stdout: "quorum": [ 2024-09-16T15:41:58.182 INFO:teuthology.orchestra.run.smithi151.stdout: 0, 2024-09-16T15:41:58.182 INFO:teuthology.orchestra.run.smithi151.stdout: 1, 2024-09-16T15:41:58.182 INFO:teuthology.orchestra.run.smithi151.stdout: 2 2024-09-16T15:41:58.182 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:41:58.182 INFO:teuthology.orchestra.run.smithi151.stdout: "quorum_names": [ 2024-09-16T15:41:58.182 INFO:teuthology.orchestra.run.smithi151.stdout: "a", 2024-09-16T15:41:58.182 INFO:teuthology.orchestra.run.smithi151.stdout: "b", 2024-09-16T15:41:58.183 INFO:teuthology.orchestra.run.smithi151.stdout: "c" 2024-09-16T15:41:58.183 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:41:58.183 INFO:teuthology.orchestra.run.smithi151.stdout: "quorum_age": 181, 2024-09-16T15:41:58.183 INFO:teuthology.orchestra.run.smithi151.stdout: "monmap": { 2024-09-16T15:41:58.183 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 1, 2024-09-16T15:41:58.183 INFO:teuthology.orchestra.run.smithi151.stdout: "min_mon_release_name": "squid", 2024-09-16T15:41:58.183 INFO:teuthology.orchestra.run.smithi151.stdout: "num_mons": 3 2024-09-16T15:41:58.183 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:41:58.183 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap": { 2024-09-16T15:41:58.183 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 37, 2024-09-16T15:41:58.183 INFO:teuthology.orchestra.run.smithi151.stdout: "num_osds": 8, 2024-09-16T15:41:58.183 INFO:teuthology.orchestra.run.smithi151.stdout: "num_up_osds": 8, 2024-09-16T15:41:58.183 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_up_since": 1726501150, 2024-09-16T15:41:58.183 INFO:teuthology.orchestra.run.smithi151.stdout: "num_in_osds": 8, 2024-09-16T15:41:58.183 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_in_since": 1726501140, 2024-09-16T15:41:58.184 INFO:teuthology.orchestra.run.smithi151.stdout: "num_remapped_pgs": 0 2024-09-16T15:41:58.184 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:41:58.184 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap": { 2024-09-16T15:41:58.184 INFO:teuthology.orchestra.run.smithi151.stdout: "pgs_by_state": [ 2024-09-16T15:41:58.184 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:41:58.184 INFO:teuthology.orchestra.run.smithi151.stdout: "state_name": "active+clean", 2024-09-16T15:41:58.184 INFO:teuthology.orchestra.run.smithi151.stdout: "count": 97 2024-09-16T15:41:58.184 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:41:58.184 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:41:58.184 INFO:teuthology.orchestra.run.smithi151.stdout: "num_pgs": 97, 2024-09-16T15:41:58.184 INFO:teuthology.orchestra.run.smithi151.stdout: "num_pools": 3, 2024-09-16T15:41:58.184 INFO:teuthology.orchestra.run.smithi151.stdout: "num_objects": 2585, 2024-09-16T15:41:58.184 INFO:teuthology.orchestra.run.smithi151.stdout: "data_bytes": 10738576615, 2024-09-16T15:41:58.184 INFO:teuthology.orchestra.run.smithi151.stdout: "bytes_used": 21735948288, 2024-09-16T15:41:58.185 INFO:teuthology.orchestra.run.smithi151.stdout: "bytes_avail": 751358164992, 2024-09-16T15:41:58.185 INFO:teuthology.orchestra.run.smithi151.stdout: "bytes_total": 773094113280, 2024-09-16T15:41:58.185 INFO:teuthology.orchestra.run.smithi151.stdout: "read_bytes_sec": 85, 2024-09-16T15:41:58.185 INFO:teuthology.orchestra.run.smithi151.stdout: "write_bytes_sec": 114968846, 2024-09-16T15:41:58.185 INFO:teuthology.orchestra.run.smithi151.stdout: "read_op_per_sec": 0, 2024-09-16T15:41:58.185 INFO:teuthology.orchestra.run.smithi151.stdout: "write_op_per_sec": 29 2024-09-16T15:41:58.185 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:41:58.185 INFO:teuthology.orchestra.run.smithi151.stdout: "fsmap": { 2024-09-16T15:41:58.185 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 18, 2024-09-16T15:41:58.185 INFO:teuthology.orchestra.run.smithi151.stdout: "btime": "2024-09-16T15:40:25:485623+0000", 2024-09-16T15:41:58.185 INFO:teuthology.orchestra.run.smithi151.stdout: "id": 2, 2024-09-16T15:41:58.185 INFO:teuthology.orchestra.run.smithi151.stdout: "up": 1, 2024-09-16T15:41:58.185 INFO:teuthology.orchestra.run.smithi151.stdout: "in": 1, 2024-09-16T15:41:58.185 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 1, 2024-09-16T15:41:58.185 INFO:teuthology.orchestra.run.smithi151.stdout: "by_rank": [ 2024-09-16T15:41:58.185 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:41:58.186 INFO:teuthology.orchestra.run.smithi151.stdout: "filesystem_id": 2, 2024-09-16T15:41:58.186 INFO:teuthology.orchestra.run.smithi151.stdout: "rank": 0, 2024-09-16T15:41:58.186 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "b", 2024-09-16T15:41:58.186 INFO:teuthology.orchestra.run.smithi151.stdout: "status": "up:active", 2024-09-16T15:41:58.186 INFO:teuthology.orchestra.run.smithi151.stdout: "gid": 4855 2024-09-16T15:41:58.186 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:41:58.186 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:41:58.186 INFO:teuthology.orchestra.run.smithi151.stdout: "up:standby": 3 2024-09-16T15:41:58.186 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:41:58.186 INFO:teuthology.orchestra.run.smithi151.stdout: "mgrmap": { 2024-09-16T15:41:58.186 INFO:teuthology.orchestra.run.smithi151.stdout: "available": true, 2024-09-16T15:41:58.186 INFO:teuthology.orchestra.run.smithi151.stdout: "num_standbys": 1, 2024-09-16T15:41:58.186 INFO:teuthology.orchestra.run.smithi151.stdout: "modules": [ 2024-09-16T15:41:58.186 INFO:teuthology.orchestra.run.smithi151.stdout: "iostat", 2024-09-16T15:41:58.186 INFO:teuthology.orchestra.run.smithi151.stdout: "nfs", 2024-09-16T15:41:58.187 INFO:teuthology.orchestra.run.smithi151.stdout: "restful" 2024-09-16T15:41:58.187 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:41:58.187 INFO:teuthology.orchestra.run.smithi151.stdout: "services": {} 2024-09-16T15:41:58.187 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:41:58.187 INFO:teuthology.orchestra.run.smithi151.stdout: "servicemap": { 2024-09-16T15:41:58.187 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 5, 2024-09-16T15:41:58.187 INFO:teuthology.orchestra.run.smithi151.stdout: "modified": "2024-09-16T15:41:17.604239+0000", 2024-09-16T15:41:58.187 INFO:teuthology.orchestra.run.smithi151.stdout: "services": {} 2024-09-16T15:41:58.187 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:41:58.187 INFO:teuthology.orchestra.run.smithi151.stdout: "progress_events": {} 2024-09-16T15:41:58.187 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-16T15:41:58.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:58.182+0000 7f1d5a200640 1 -- 172.21.15.151:0/334116176 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1d340788e0 msgr2=0x7f1d3407ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:58.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:58.182+0000 7f1d5a200640 1 --2- 172.21.15.151:0/334116176 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1d340788e0 0x7f1d3407ada0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1d5414cec0 tx=0x7f1d4400b390 comp rx=0 tx=0).stop 2024-09-16T15:41:58.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:58.182+0000 7f1d5a200640 1 -- 172.21.15.151:0/334116176 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d5415cff0 msgr2=0x7f1d540ae370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:58.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:58.182+0000 7f1d5a200640 1 --2- 172.21.15.151:0/334116176 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d5415cff0 0x7f1d540ae370 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c00c770 tx=0x7f1d4c00cc30 comp rx=0 tx=0).stop 2024-09-16T15:41:58.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:58.183+0000 7f1d5a200640 1 -- 172.21.15.151:0/334116176 shutdown_connections 2024-09-16T15:41:58.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:58.183+0000 7f1d5a200640 1 --2- 172.21.15.151:0/334116176 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1d340788e0 0x7f1d3407ada0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:58.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:58.183+0000 7f1d5a200640 1 --2- 172.21.15.151:0/334116176 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d54159aa0 0x7f1d5414c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:58.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:58.183+0000 7f1d5a200640 1 --2- 172.21.15.151:0/334116176 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d5415cff0 0x7f1d540ae370 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:58.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:58.183+0000 7f1d5a200640 1 --2- 172.21.15.151:0/334116176 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1d5406f220 0x7f1d540cff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:58.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:58.183+0000 7f1d5a200640 1 -- 172.21.15.151:0/334116176 >> 172.21.15.151:0/334116176 conn(0x7f1d540b18e0 msgr2=0x7f1d5415c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:58.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:58.183+0000 7f1d5a200640 1 -- 172.21.15.151:0/334116176 shutdown_connections 2024-09-16T15:41:58.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:58.183+0000 7f1d5a200640 1 -- 172.21.15.151:0/334116176 wait complete. 2024-09-16T15:41:58.747 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:41:58.745+0000 7f48e3600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:41:58.747 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:41:58.745+0000 7f48e3600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:41:58.747 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:41:58.745+0000 7f48e3600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:41:58.747 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:41:58.745+0000 7f48e3600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:41:58.747 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:41:58.745+0000 7f48e3600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:41:59.201 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-16T15:41:59.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:59.451+0000 7f7d35800640 1 -- 172.21.15.151:0/1594879799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d14004800 msgr2=0x7f7d14004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:59.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:59.451+0000 7f7d35800640 1 --2- 172.21.15.151:0/1594879799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d14004800 0x7f7d14004c80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c0016c0 tx=0x7f7d14005830 comp rx=0 tx=0).stop 2024-09-16T15:41:59.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:59.451+0000 7f7d35800640 1 -- 172.21.15.151:0/1594879799 shutdown_connections 2024-09-16T15:41:59.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:59.451+0000 7f7d35800640 1 --2- 172.21.15.151:0/1594879799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d14004800 0x7f7d14004c80 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:59.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:59.451+0000 7f7d35800640 1 -- 172.21.15.151:0/1594879799 >> 172.21.15.151:0/1594879799 conn(0x7f7d3006cd50 msgr2=0x7f7d3006d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:59.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:59.451+0000 7f7d35800640 1 -- 172.21.15.151:0/1594879799 shutdown_connections 2024-09-16T15:41:59.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:59.451+0000 7f7d35800640 1 -- 172.21.15.151:0/1594879799 wait complete. 2024-09-16T15:41:59.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:59.453+0000 7f7d35800640 1 Processor -- start 2024-09-16T15:41:59.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:59.453+0000 7f7d35800640 1 -- start start 2024-09-16T15:41:59.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:59.455+0000 7f7d35800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d14004800 0x7f7d301efd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:59.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:59.455+0000 7f7d35800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d301596d0 0x7f7d301571a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:59.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:59.455+0000 7f7d35800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7d301576e0 0x7f7d30157b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:59.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:59.455+0000 7f7d35800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3015fca0 con 0x7f7d14004800 2024-09-16T15:41:59.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:59.455+0000 7f7d35800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3015fb20 con 0x7f7d301596d0 2024-09-16T15:41:59.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:59.455+0000 7f7d35800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f7d3015fe20 con 0x7f7d301576e0 2024-09-16T15:41:59.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:59.455+0000 7f7d2fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d14004800 0x7f7d301efd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:59.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:59.455+0000 7f7d2fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d14004800 0x7f7d301efd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:46102/0 (socket says 172.21.15.151:46102) 2024-09-16T15:41:59.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:59.455+0000 7f7d2fe00640 1 -- 172.21.15.151:0/1353214294 learned_addr learned my addr 172.21.15.151:0/1353214294 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:41:59.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:59.455+0000 7f7d2f400640 1 --2- 172.21.15.151:0/1353214294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d301596d0 0x7f7d301571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:59.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:59.455+0000 7f7d34e00640 1 --2- 172.21.15.151:0/1353214294 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7d301576e0 0x7f7d30157b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:59.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:59.455+0000 7f7d2fe00640 1 -- 172.21.15.151:0/1353214294 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7d301576e0 msgr2=0x7f7d30157b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:59.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:59.455+0000 7f7d2fe00640 1 --2- 172.21.15.151:0/1353214294 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7d301576e0 0x7f7d30157b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:59.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:59.455+0000 7f7d2fe00640 1 -- 172.21.15.151:0/1353214294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d301596d0 msgr2=0x7f7d301571a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:59.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:59.456+0000 7f7d2fe00640 1 --2- 172.21.15.151:0/1353214294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d301596d0 0x7f7d301571a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:59.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:59.456+0000 7f7d2fe00640 1 -- 172.21.15.151:0/1353214294 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d301f37a0 con 0x7f7d14004800 2024-09-16T15:41:59.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:59.456+0000 7f7d2fe00640 1 --2- 172.21.15.151:0/1353214294 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d14004800 0x7f7d301efd80 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7d20006e50 tx=0x7f7d200049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:59.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:59.456+0000 7f7d27e00640 1 -- 172.21.15.151:0/1353214294 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d20015020 con 0x7f7d14004800 2024-09-16T15:41:59.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:59.456+0000 7f7d27e00640 1 -- 172.21.15.151:0/1353214294 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7d200109a0 con 0x7f7d14004800 2024-09-16T15:41:59.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:59.456+0000 7f7d27e00640 1 -- 172.21.15.151:0/1353214294 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d20010df0 con 0x7f7d14004800 2024-09-16T15:41:59.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:59.456+0000 7f7d35800640 1 -- 172.21.15.151:0/1353214294 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d301f3d70 con 0x7f7d14004800 2024-09-16T15:41:59.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:59.456+0000 7f7d35800640 1 -- 172.21.15.151:0/1353214294 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d301f4330 con 0x7f7d14004800 2024-09-16T15:41:59.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:59.457+0000 7f7d35800640 1 -- 172.21.15.151:0/1353214294 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d300d0140 con 0x7f7d14004800 2024-09-16T15:41:59.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:59.458+0000 7f7d27e00640 1 -- 172.21.15.151:0/1353214294 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 8) ==== 101338+0+0 (secure 0 0 0) 0x7f7d20018020 con 0x7f7d14004800 2024-09-16T15:41:59.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:59.459+0000 7f7d27e00640 1 --2- 172.21.15.151:0/1353214294 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7d10078910 0x7f7d1007add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:41:59.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:59.459+0000 7f7d27e00640 1 -- 172.21.15.151:0/1353214294 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5825+0+0 (secure 0 0 0) 0x7f7d2009b810 con 0x7f7d14004800 2024-09-16T15:41:59.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:59.464+0000 7f7d27e00640 1 -- 172.21.15.151:0/1353214294 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7d300d0140 con 0x7f7d14004800 2024-09-16T15:41:59.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:59.465+0000 7f7d2f400640 1 --2- 172.21.15.151:0/1353214294 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7d10078910 0x7f7d1007add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:41:59.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:59.469+0000 7f7d2f400640 1 --2- 172.21.15.151:0/1353214294 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7d10078910 0x7f7d1007add0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c032000 tx=0x7f7d1c001190 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:41:59.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:59.877+0000 7f7d35800640 1 -- 172.21.15.151:0/1353214294 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f7d301f4500 con 0x7f7d14004800 2024-09-16T15:41:59.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:59.882+0000 7f7d27e00640 1 -- 172.21.15.151:0/1353214294 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mgrmap(e 9) ==== 101491+0+0 (secure 0 0 0) 0x7f7d2005f770 con 0x7f7d14004800 2024-09-16T15:41:59.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:59.885+0000 7f7d27e00640 1 -- 172.21.15.151:0/1353214294 <== mon.0 v2:172.21.15.151:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2137 (secure 0 0 0) 0x7f7d2001f020 con 0x7f7d14004800 2024-09-16T15:41:59.886 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:41:59.886 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-16T15:41:59.886 INFO:teuthology.orchestra.run.smithi151.stdout: "fsid": "1757be18-e41a-4446-bb4f-ee4ac81c39df", 2024-09-16T15:41:59.886 INFO:teuthology.orchestra.run.smithi151.stdout: "health": { 2024-09-16T15:41:59.886 INFO:teuthology.orchestra.run.smithi151.stdout: "status": "HEALTH_OK", 2024-09-16T15:41:59.886 INFO:teuthology.orchestra.run.smithi151.stdout: "checks": {}, 2024-09-16T15:41:59.886 INFO:teuthology.orchestra.run.smithi151.stdout: "mutes": [] 2024-09-16T15:41:59.886 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:41:59.887 INFO:teuthology.orchestra.run.smithi151.stdout: "election_epoch": 4, 2024-09-16T15:41:59.887 INFO:teuthology.orchestra.run.smithi151.stdout: "quorum": [ 2024-09-16T15:41:59.887 INFO:teuthology.orchestra.run.smithi151.stdout: 0, 2024-09-16T15:41:59.887 INFO:teuthology.orchestra.run.smithi151.stdout: 1, 2024-09-16T15:41:59.887 INFO:teuthology.orchestra.run.smithi151.stdout: 2 2024-09-16T15:41:59.887 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:41:59.887 INFO:teuthology.orchestra.run.smithi151.stdout: "quorum_names": [ 2024-09-16T15:41:59.887 INFO:teuthology.orchestra.run.smithi151.stdout: "a", 2024-09-16T15:41:59.887 INFO:teuthology.orchestra.run.smithi151.stdout: "b", 2024-09-16T15:41:59.887 INFO:teuthology.orchestra.run.smithi151.stdout: "c" 2024-09-16T15:41:59.887 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:41:59.887 INFO:teuthology.orchestra.run.smithi151.stdout: "quorum_age": 183, 2024-09-16T15:41:59.887 INFO:teuthology.orchestra.run.smithi151.stdout: "monmap": { 2024-09-16T15:41:59.887 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 1, 2024-09-16T15:41:59.887 INFO:teuthology.orchestra.run.smithi151.stdout: "min_mon_release_name": "squid", 2024-09-16T15:41:59.888 INFO:teuthology.orchestra.run.smithi151.stdout: "num_mons": 3 2024-09-16T15:41:59.888 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:41:59.888 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap": { 2024-09-16T15:41:59.888 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 37, 2024-09-16T15:41:59.888 INFO:teuthology.orchestra.run.smithi151.stdout: "num_osds": 8, 2024-09-16T15:41:59.888 INFO:teuthology.orchestra.run.smithi151.stdout: "num_up_osds": 8, 2024-09-16T15:41:59.888 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_up_since": 1726501150, 2024-09-16T15:41:59.888 INFO:teuthology.orchestra.run.smithi151.stdout: "num_in_osds": 8, 2024-09-16T15:41:59.888 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_in_since": 1726501140, 2024-09-16T15:41:59.888 INFO:teuthology.orchestra.run.smithi151.stdout: "num_remapped_pgs": 0 2024-09-16T15:41:59.888 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:41:59.888 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap": { 2024-09-16T15:41:59.888 INFO:teuthology.orchestra.run.smithi151.stdout: "pgs_by_state": [ 2024-09-16T15:41:59.888 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:41:59.889 INFO:teuthology.orchestra.run.smithi151.stdout: "state_name": "active+clean", 2024-09-16T15:41:59.889 INFO:teuthology.orchestra.run.smithi151.stdout: "count": 97 2024-09-16T15:41:59.889 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:41:59.889 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:41:59.889 INFO:teuthology.orchestra.run.smithi151.stdout: "num_pgs": 97, 2024-09-16T15:41:59.889 INFO:teuthology.orchestra.run.smithi151.stdout: "num_pools": 3, 2024-09-16T15:41:59.889 INFO:teuthology.orchestra.run.smithi151.stdout: "num_objects": 2585, 2024-09-16T15:41:59.889 INFO:teuthology.orchestra.run.smithi151.stdout: "data_bytes": 10738576615, 2024-09-16T15:41:59.889 INFO:teuthology.orchestra.run.smithi151.stdout: "bytes_used": 21735948288, 2024-09-16T15:41:59.889 INFO:teuthology.orchestra.run.smithi151.stdout: "bytes_avail": 751358164992, 2024-09-16T15:41:59.889 INFO:teuthology.orchestra.run.smithi151.stdout: "bytes_total": 773094113280, 2024-09-16T15:41:59.889 INFO:teuthology.orchestra.run.smithi151.stdout: "read_bytes_sec": 85, 2024-09-16T15:41:59.889 INFO:teuthology.orchestra.run.smithi151.stdout: "write_bytes_sec": 89446175, 2024-09-16T15:41:59.889 INFO:teuthology.orchestra.run.smithi151.stdout: "read_op_per_sec": 0, 2024-09-16T15:41:59.889 INFO:teuthology.orchestra.run.smithi151.stdout: "write_op_per_sec": 22 2024-09-16T15:41:59.890 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:41:59.890 INFO:teuthology.orchestra.run.smithi151.stdout: "fsmap": { 2024-09-16T15:41:59.890 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 18, 2024-09-16T15:41:59.890 INFO:teuthology.orchestra.run.smithi151.stdout: "btime": "2024-09-16T15:40:25:485623+0000", 2024-09-16T15:41:59.890 INFO:teuthology.orchestra.run.smithi151.stdout: "id": 2, 2024-09-16T15:41:59.890 INFO:teuthology.orchestra.run.smithi151.stdout: "up": 1, 2024-09-16T15:41:59.890 INFO:teuthology.orchestra.run.smithi151.stdout: "in": 1, 2024-09-16T15:41:59.890 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 1, 2024-09-16T15:41:59.890 INFO:teuthology.orchestra.run.smithi151.stdout: "by_rank": [ 2024-09-16T15:41:59.890 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:41:59.890 INFO:teuthology.orchestra.run.smithi151.stdout: "filesystem_id": 2, 2024-09-16T15:41:59.890 INFO:teuthology.orchestra.run.smithi151.stdout: "rank": 0, 2024-09-16T15:41:59.890 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "b", 2024-09-16T15:41:59.890 INFO:teuthology.orchestra.run.smithi151.stdout: "status": "up:active", 2024-09-16T15:41:59.890 INFO:teuthology.orchestra.run.smithi151.stdout: "gid": 4855 2024-09-16T15:41:59.891 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:41:59.891 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:41:59.891 INFO:teuthology.orchestra.run.smithi151.stdout: "up:standby": 3 2024-09-16T15:41:59.891 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:41:59.891 INFO:teuthology.orchestra.run.smithi151.stdout: "mgrmap": { 2024-09-16T15:41:59.891 INFO:teuthology.orchestra.run.smithi151.stdout: "available": true, 2024-09-16T15:41:59.891 INFO:teuthology.orchestra.run.smithi151.stdout: "num_standbys": 1, 2024-09-16T15:41:59.891 INFO:teuthology.orchestra.run.smithi151.stdout: "modules": [ 2024-09-16T15:41:59.891 INFO:teuthology.orchestra.run.smithi151.stdout: "iostat", 2024-09-16T15:41:59.891 INFO:teuthology.orchestra.run.smithi151.stdout: "nfs", 2024-09-16T15:41:59.891 INFO:teuthology.orchestra.run.smithi151.stdout: "restful" 2024-09-16T15:41:59.891 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:41:59.891 INFO:teuthology.orchestra.run.smithi151.stdout: "services": {} 2024-09-16T15:41:59.891 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:41:59.891 INFO:teuthology.orchestra.run.smithi151.stdout: "servicemap": { 2024-09-16T15:41:59.892 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 5, 2024-09-16T15:41:59.892 INFO:teuthology.orchestra.run.smithi151.stdout: "modified": "2024-09-16T15:41:17.604239+0000", 2024-09-16T15:41:59.892 INFO:teuthology.orchestra.run.smithi151.stdout: "services": {} 2024-09-16T15:41:59.892 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:41:59.892 INFO:teuthology.orchestra.run.smithi151.stdout: "progress_events": { 2024-09-16T15:41:59.892 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr-vol-ongoing-clones": { 2024-09-16T15:41:59.892 INFO:teuthology.orchestra.run.smithi151.stdout: "message": "1 ongoing clones - average progress is 0.0% (0s)\n [............................] ", 2024-09-16T15:41:59.892 INFO:teuthology.orchestra.run.smithi151.stdout: "progress": 0, 2024-09-16T15:41:59.892 INFO:teuthology.orchestra.run.smithi151.stdout: "add_to_ceph_s": true 2024-09-16T15:41:59.892 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:41:59.892 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:41:59.892 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-16T15:41:59.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:59.887+0000 7f7d35800640 1 -- 172.21.15.151:0/1353214294 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7d10078910 msgr2=0x7f7d1007add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:59.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:59.887+0000 7f7d35800640 1 --2- 172.21.15.151:0/1353214294 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7d10078910 0x7f7d1007add0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c032000 tx=0x7f7d1c001190 comp rx=0 tx=0).stop 2024-09-16T15:41:59.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:59.887+0000 7f7d35800640 1 -- 172.21.15.151:0/1353214294 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d14004800 msgr2=0x7f7d301efd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:41:59.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:59.887+0000 7f7d35800640 1 --2- 172.21.15.151:0/1353214294 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d14004800 0x7f7d301efd80 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7d20006e50 tx=0x7f7d200049d0 comp rx=0 tx=0).stop 2024-09-16T15:41:59.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:59.888+0000 7f7d35800640 1 -- 172.21.15.151:0/1353214294 shutdown_connections 2024-09-16T15:41:59.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:59.888+0000 7f7d35800640 1 --2- 172.21.15.151:0/1353214294 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7d10078910 0x7f7d1007add0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:59.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:59.888+0000 7f7d35800640 1 --2- 172.21.15.151:0/1353214294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d301596d0 0x7f7d301571a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:59.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:59.888+0000 7f7d35800640 1 --2- 172.21.15.151:0/1353214294 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7d301576e0 0x7f7d30157b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:59.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:59.888+0000 7f7d35800640 1 --2- 172.21.15.151:0/1353214294 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d14004800 0x7f7d301efd80 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:41:59.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:59.888+0000 7f7d35800640 1 -- 172.21.15.151:0/1353214294 >> 172.21.15.151:0/1353214294 conn(0x7f7d3006cd50 msgr2=0x7f7d3006d130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:41:59.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:59.888+0000 7f7d35800640 1 -- 172.21.15.151:0/1353214294 shutdown_connections 2024-09-16T15:41:59.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:41:59.888+0000 7f7d35800640 1 -- 172.21.15.151:0/1353214294 wait complete. 2024-09-16T15:41:59.907 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs ss1clone1 2024-09-16T15:42:00.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.187+0000 7f43c0800640 1 -- 172.21.15.151:0/2499459750 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f43b8161bb0 msgr2=0x7f43b806f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:00.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.187+0000 7f43c0800640 1 --2- 172.21.15.151:0/2499459750 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f43b8161bb0 0x7f43b806f8a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f43b4005fd0 tx=0x7f43b402a840 comp rx=0 tx=0).stop 2024-09-16T15:42:00.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.187+0000 7f43c0800640 1 -- 172.21.15.151:0/2499459750 shutdown_connections 2024-09-16T15:42:00.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.187+0000 7f43c0800640 1 --2- 172.21.15.151:0/2499459750 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f43b8161bb0 0x7f43b806f8a0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:00.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.187+0000 7f43c0800640 1 -- 172.21.15.151:0/2499459750 >> 172.21.15.151:0/2499459750 conn(0x7f43b80b1840 msgr2=0x7f43b80b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:00.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.187+0000 7f43c0800640 1 -- 172.21.15.151:0/2499459750 shutdown_connections 2024-09-16T15:42:00.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.187+0000 7f43c0800640 1 -- 172.21.15.151:0/2499459750 wait complete. 2024-09-16T15:42:00.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.191+0000 7f43c0800640 1 Processor -- start 2024-09-16T15:42:00.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.191+0000 7f43c0800640 1 -- start start 2024-09-16T15:42:00.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.192+0000 7f43c0800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43b80af170 0x7f43b814e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:00.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.192+0000 7f43c0800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f43b8161490 0x7f43b80a9e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:00.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.192+0000 7f43c0800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43b8161bb0 0x7f43b8151140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:00.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.192+0000 7f43c0800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f43b80ce360 con 0x7f43b8161490 2024-09-16T15:42:00.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.192+0000 7f43c0800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f43b80ce1e0 con 0x7f43b80af170 2024-09-16T15:42:00.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.192+0000 7f43c0800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f43b806fde0 con 0x7f43b8161bb0 2024-09-16T15:42:00.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.192+0000 7f43bd400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f43b8161490 0x7f43b80a9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:00.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.192+0000 7f43bd400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f43b8161490 0x7f43b80a9e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:46112/0 (socket says 172.21.15.151:46112) 2024-09-16T15:42:00.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.192+0000 7f43bd400640 1 -- 172.21.15.151:0/812151195 learned_addr learned my addr 172.21.15.151:0/812151195 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:42:00.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.193+0000 7f43be800640 1 --2- 172.21.15.151:0/812151195 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43b8161bb0 0x7f43b8151140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:00.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.193+0000 7f43bd400640 1 -- 172.21.15.151:0/812151195 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43b8161bb0 msgr2=0x7f43b8151140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:00.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.193+0000 7f43bd400640 1 --2- 172.21.15.151:0/812151195 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43b8161bb0 0x7f43b8151140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:00.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.193+0000 7f43bd400640 1 -- 172.21.15.151:0/812151195 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43b80af170 msgr2=0x7f43b814e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:00.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.193+0000 7f43bd400640 1 --2- 172.21.15.151:0/812151195 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43b80af170 0x7f43b814e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:00.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.193+0000 7f43bd400640 1 -- 172.21.15.151:0/812151195 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43b814ee00 con 0x7f43b8161490 2024-09-16T15:42:00.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.194+0000 7f43bd400640 1 --2- 172.21.15.151:0/812151195 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f43b8161490 0x7f43b80a9e00 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f43a400c700 tx=0x7f43a400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:00.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.194+0000 7f43b2000640 1 -- 172.21.15.151:0/812151195 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a400a840 con 0x7f43b8161490 2024-09-16T15:42:00.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.194+0000 7f43b2000640 1 -- 172.21.15.151:0/812151195 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f43a4017070 con 0x7f43b8161490 2024-09-16T15:42:00.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.194+0000 7f43b2000640 1 -- 172.21.15.151:0/812151195 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a400add0 con 0x7f43b8161490 2024-09-16T15:42:00.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.194+0000 7f43c0800640 1 -- 172.21.15.151:0/812151195 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43b8151bc0 con 0x7f43b8161490 2024-09-16T15:42:00.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.194+0000 7f43c0800640 1 -- 172.21.15.151:0/812151195 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43b80a9600 con 0x7f43b8161490 2024-09-16T15:42:00.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.197+0000 7f43c0800640 1 -- 172.21.15.151:0/812151195 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43b81520e0 con 0x7f43b8161490 2024-09-16T15:42:00.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.199+0000 7f43b2000640 1 -- 172.21.15.151:0/812151195 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 9) ==== 101491+0+0 (secure 0 0 0) 0x7f43a401c460 con 0x7f43b8161490 2024-09-16T15:42:00.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.201+0000 7f43b2000640 1 --2- 172.21.15.151:0/812151195 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4398078a90 0x7f439807af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:00.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.202+0000 7f43b2000640 1 -- 172.21.15.151:0/812151195 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5825+0+0 (secure 0 0 0) 0x7f43a409ada0 con 0x7f43b8161490 2024-09-16T15:42:00.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.204+0000 7f43bde00640 1 --2- 172.21.15.151:0/812151195 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4398078a90 0x7f439807af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:00.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.210+0000 7f43bde00640 1 --2- 172.21.15.151:0/812151195 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4398078a90 0x7f439807af50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f43ac003cc0 tx=0x7f43ac0079b0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:00.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.214+0000 7f43b2000640 1 -- 172.21.15.151:0/812151195 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f43b81520e0 con 0x7f43b8161490 2024-09-16T15:42:00.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.579+0000 7f43c0800640 1 -- 172.21.15.151:0/812151195 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f43b806f8a0 con 0x7f4398078a90 2024-09-16T15:42:00.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.713+0000 7f43b2000640 1 -- 172.21.15.151:0/812151195 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f43b806f8a0 con 0x7f4398078a90 2024-09-16T15:42:00.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.715+0000 7f43c0800640 1 -- 172.21.15.151:0/812151195 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4398078a90 msgr2=0x7f439807af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:00.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.715+0000 7f43c0800640 1 --2- 172.21.15.151:0/812151195 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4398078a90 0x7f439807af50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f43ac003cc0 tx=0x7f43ac0079b0 comp rx=0 tx=0).stop 2024-09-16T15:42:00.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.716+0000 7f43c0800640 1 -- 172.21.15.151:0/812151195 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f43b8161490 msgr2=0x7f43b80a9e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:00.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.716+0000 7f43c0800640 1 --2- 172.21.15.151:0/812151195 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f43b8161490 0x7f43b80a9e00 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f43a400c700 tx=0x7f43a400cbc0 comp rx=0 tx=0).stop 2024-09-16T15:42:00.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.716+0000 7f43c0800640 1 -- 172.21.15.151:0/812151195 shutdown_connections 2024-09-16T15:42:00.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.716+0000 7f43c0800640 1 --2- 172.21.15.151:0/812151195 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4398078a90 0x7f439807af50 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:00.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.716+0000 7f43c0800640 1 --2- 172.21.15.151:0/812151195 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43b8161bb0 0x7f43b8151140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:00.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.716+0000 7f43c0800640 1 --2- 172.21.15.151:0/812151195 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f43b8161490 0x7f43b80a9e00 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:00.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.716+0000 7f43c0800640 1 --2- 172.21.15.151:0/812151195 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43b80af170 0x7f43b814e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:00.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.716+0000 7f43c0800640 1 -- 172.21.15.151:0/812151195 >> 172.21.15.151:0/812151195 conn(0x7f43b80b1840 msgr2=0x7f43b81603b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:00.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.716+0000 7f43c0800640 1 -- 172.21.15.151:0/812151195 shutdown_connections 2024-09-16T15:42:00.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.716+0000 7f43c0800640 1 -- 172.21.15.151:0/812151195 wait complete. 2024-09-16T15:42:00.733 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-16T15:42:00.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.891+0000 7f3d35e00640 1 -- 172.21.15.151:0/1026210679 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3d300d0540 msgr2=0x7f3d14003c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:00.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.891+0000 7f3d35e00640 1 --2- 172.21.15.151:0/1026210679 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3d300d0540 0x7f3d14003c10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3d20001830 tx=0x7f3d14005830 comp rx=0 tx=0).stop 2024-09-16T15:42:00.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.891+0000 7f3d35400640 1 -- 172.21.15.151:0/1026210679 reap_dead start 2024-09-16T15:42:00.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.891+0000 7f3d35e00640 1 -- 172.21.15.151:0/1026210679 shutdown_connections 2024-09-16T15:42:00.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.891+0000 7f3d35e00640 1 -- 172.21.15.151:0/1026210679 >> 172.21.15.151:0/1026210679 conn(0x7f3d300cfa20 msgr2=0x7f3d300cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:00.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.891+0000 7f3d35e00640 1 -- 172.21.15.151:0/1026210679 shutdown_connections 2024-09-16T15:42:00.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.891+0000 7f3d35e00640 1 -- 172.21.15.151:0/1026210679 wait complete. 2024-09-16T15:42:00.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.893+0000 7f3d35e00640 1 Processor -- start 2024-09-16T15:42:00.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.893+0000 7f3d35e00640 1 -- start start 2024-09-16T15:42:00.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.894+0000 7f3d35e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3d300d0540 0x7f3d300af050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:00.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.894+0000 7f3d35e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3d301f7d10 0x7f3d30150e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:00.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.894+0000 7f3d35e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d301e23f0 0x7f3d300ce010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:00.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.894+0000 7f3d35e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3d30074660 con 0x7f3d300d0540 2024-09-16T15:42:00.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.894+0000 7f3d35e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3d300b28f0 con 0x7f3d301e23f0 2024-09-16T15:42:00.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.894+0000 7f3d35e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f3d300af590 con 0x7f3d301f7d10 2024-09-16T15:42:00.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.894+0000 7f3d34a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3d300d0540 0x7f3d300af050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:00.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.894+0000 7f3d34a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3d300d0540 0x7f3d300af050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:46136/0 (socket says 172.21.15.151:46136) 2024-09-16T15:42:00.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.894+0000 7f3d34a00640 1 -- 172.21.15.151:0/932266197 learned_addr learned my addr 172.21.15.151:0/932266197 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:42:00.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.894+0000 7f3d2fe00640 1 --2- 172.21.15.151:0/932266197 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3d301f7d10 0x7f3d30150e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:00.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.894+0000 7f3d35400640 1 --2- 172.21.15.151:0/932266197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d301e23f0 0x7f3d300ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:00.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.894+0000 7f3d34a00640 1 -- 172.21.15.151:0/932266197 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3d301f7d10 msgr2=0x7f3d30150e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:00.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.894+0000 7f3d34a00640 1 --2- 172.21.15.151:0/932266197 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3d301f7d10 0x7f3d30150e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:00.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.894+0000 7f3d34a00640 1 -- 172.21.15.151:0/932266197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d301e23f0 msgr2=0x7f3d300ce010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:00.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.894+0000 7f3d34a00640 1 --2- 172.21.15.151:0/932266197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d301e23f0 0x7f3d300ce010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:00.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.894+0000 7f3d34a00640 1 -- 172.21.15.151:0/932266197 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d301f81a0 con 0x7f3d300d0540 2024-09-16T15:42:00.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.895+0000 7f3d34a00640 1 --2- 172.21.15.151:0/932266197 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3d300d0540 0x7f3d300af050 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3d2002aa50 tx=0x7f3d20006150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:00.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.895+0000 7f3d2cc00640 1 -- 172.21.15.151:0/932266197 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d20043070 con 0x7f3d300d0540 2024-09-16T15:42:00.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.895+0000 7f3d2cc00640 1 -- 172.21.15.151:0/932266197 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3d20003af0 con 0x7f3d300d0540 2024-09-16T15:42:00.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.895+0000 7f3d35e00640 1 -- 172.21.15.151:0/932266197 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d301f8480 con 0x7f3d300d0540 2024-09-16T15:42:00.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.895+0000 7f3d2cc00640 1 -- 172.21.15.151:0/932266197 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d20033480 con 0x7f3d300d0540 2024-09-16T15:42:00.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.895+0000 7f3d35e00640 1 -- 172.21.15.151:0/932266197 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d301f8a40 con 0x7f3d300d0540 2024-09-16T15:42:00.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.896+0000 7f3d35e00640 1 -- 172.21.15.151:0/932266197 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d30073d40 con 0x7f3d300d0540 2024-09-16T15:42:00.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.897+0000 7f3d2cc00640 1 -- 172.21.15.151:0/932266197 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 9) ==== 101491+0+0 (secure 0 0 0) 0x7f3d20046070 con 0x7f3d300d0540 2024-09-16T15:42:00.899 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.898+0000 7f3d2cc00640 1 --2- 172.21.15.151:0/932266197 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3d10078a90 0x7f3d1007af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:00.899 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.898+0000 7f3d2cc00640 1 -- 172.21.15.151:0/932266197 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5825+0+0 (secure 0 0 0) 0x7f3d200bab70 con 0x7f3d300d0540 2024-09-16T15:42:00.899 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.898+0000 7f3d2fe00640 1 --2- 172.21.15.151:0/932266197 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3d10078a90 0x7f3d1007af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:00.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.899+0000 7f3d2fe00640 1 --2- 172.21.15.151:0/932266197 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3d10078a90 0x7f3d1007af50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3d3015b3e0 tx=0x7f3d1c001c10 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:00.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:00.902+0000 7f3d2cc00640 1 -- 172.21.15.151:0/932266197 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f3d200826d0 con 0x7f3d300d0540 2024-09-16T15:42:01.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.107+0000 7f3d35e00640 1 -- 172.21.15.151:0/932266197 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f3d3006ad20 con 0x7f3d10078a90 2024-09-16T15:42:01.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.109+0000 7f3d2cc00640 1 -- 172.21.15.151:0/932266197 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f3d3006ad20 con 0x7f3d10078a90 2024-09-16T15:42:01.110 INFO:teuthology.orchestra.run.smithi151.stdout:[ 2024-09-16T15:42:01.110 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:42:01.110 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "cephfs" 2024-09-16T15:42:01.110 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:42:01.110 INFO:teuthology.orchestra.run.smithi151.stdout:] 2024-09-16T15:42:01.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.111+0000 7f3d35e00640 1 -- 172.21.15.151:0/932266197 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3d10078a90 msgr2=0x7f3d1007af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:01.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.111+0000 7f3d35e00640 1 --2- 172.21.15.151:0/932266197 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3d10078a90 0x7f3d1007af50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3d3015b3e0 tx=0x7f3d1c001c10 comp rx=0 tx=0).stop 2024-09-16T15:42:01.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.111+0000 7f3d35e00640 1 -- 172.21.15.151:0/932266197 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3d300d0540 msgr2=0x7f3d300af050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:01.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.111+0000 7f3d35e00640 1 --2- 172.21.15.151:0/932266197 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3d300d0540 0x7f3d300af050 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3d2002aa50 tx=0x7f3d20006150 comp rx=0 tx=0).stop 2024-09-16T15:42:01.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.111+0000 7f3d35e00640 1 -- 172.21.15.151:0/932266197 shutdown_connections 2024-09-16T15:42:01.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.111+0000 7f3d35e00640 1 --2- 172.21.15.151:0/932266197 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3d10078a90 0x7f3d1007af50 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:01.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.111+0000 7f3d35e00640 1 --2- 172.21.15.151:0/932266197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d301e23f0 0x7f3d300ce010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:01.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.111+0000 7f3d35e00640 1 --2- 172.21.15.151:0/932266197 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3d301f7d10 0x7f3d30150e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:01.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.111+0000 7f3d35e00640 1 --2- 172.21.15.151:0/932266197 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3d300d0540 0x7f3d300af050 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:01.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.111+0000 7f3d35e00640 1 -- 172.21.15.151:0/932266197 >> 172.21.15.151:0/932266197 conn(0x7f3d300cfa20 msgr2=0x7f3d300c91e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:01.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.111+0000 7f3d35e00640 1 -- 172.21.15.151:0/932266197 shutdown_connections 2024-09-16T15:42:01.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.111+0000 7f3d35e00640 1 -- 172.21.15.151:0/932266197 wait complete. 2024-09-16T15:42:01.128 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-09-16T15:42:01.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.286+0000 7faedde00640 1 -- 172.21.15.151:0/1880581524 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faebc004800 msgr2=0x7faebc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:01.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.286+0000 7faedde00640 1 --2- 172.21.15.151:0/1880581524 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faebc004800 0x7faebc004c80 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7faec40016c0 tx=0x7faebc005830 comp rx=0 tx=0).stop 2024-09-16T15:42:01.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.287+0000 7faedde00640 1 -- 172.21.15.151:0/1880581524 shutdown_connections 2024-09-16T15:42:01.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.287+0000 7faedde00640 1 --2- 172.21.15.151:0/1880581524 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faebc004800 0x7faebc004c80 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:01.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.287+0000 7faedde00640 1 -- 172.21.15.151:0/1880581524 >> 172.21.15.151:0/1880581524 conn(0x7faed80b1840 msgr2=0x7faed80b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:01.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.287+0000 7faedde00640 1 -- 172.21.15.151:0/1880581524 shutdown_connections 2024-09-16T15:42:01.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.287+0000 7faedde00640 1 -- 172.21.15.151:0/1880581524 wait complete. 2024-09-16T15:42:01.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.288+0000 7faedde00640 1 Processor -- start 2024-09-16T15:42:01.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.288+0000 7faedde00640 1 -- start start 2024-09-16T15:42:01.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.289+0000 7faedde00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faebc004800 0x7faed815e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:01.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.289+0000 7faedde00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faed81f5a30 0x7faed80cb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:01.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.289+0000 7faedde00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faed81419f0 0x7faed81e58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:01.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.289+0000 7faedde00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7faed80cc5c0 con 0x7faed81f5a30 2024-09-16T15:42:01.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.289+0000 7faedde00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7faed80736a0 con 0x7faebc004800 2024-09-16T15:42:01.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.289+0000 7faedde00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7faed80cc740 con 0x7faed81419f0 2024-09-16T15:42:01.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.289+0000 7faed6000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faed81f5a30 0x7faed80cb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:01.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.290+0000 7faed6000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faed81f5a30 0x7faed80cb7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:46154/0 (socket says 172.21.15.151:46154) 2024-09-16T15:42:01.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.290+0000 7faed6000640 1 -- 172.21.15.151:0/2242806302 learned_addr learned my addr 172.21.15.151:0/2242806302 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:42:01.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.290+0000 7faed7400640 1 --2- 172.21.15.151:0/2242806302 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faed81419f0 0x7faed81e58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:01.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.290+0000 7faed6a00640 1 --2- 172.21.15.151:0/2242806302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faebc004800 0x7faed815e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:01.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.290+0000 7faed6000640 1 -- 172.21.15.151:0/2242806302 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faed81419f0 msgr2=0x7faed81e58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:01.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.290+0000 7faed6000640 1 --2- 172.21.15.151:0/2242806302 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faed81419f0 0x7faed81e58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:01.293 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.290+0000 7faed6000640 1 -- 172.21.15.151:0/2242806302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faebc004800 msgr2=0x7faed815e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:01.293 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.290+0000 7faed6000640 1 --2- 172.21.15.151:0/2242806302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faebc004800 0x7faed815e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:01.293 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.290+0000 7faed6000640 1 -- 172.21.15.151:0/2242806302 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faed80afae0 con 0x7faed81f5a30 2024-09-16T15:42:01.293 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.290+0000 7faed6000640 1 --2- 172.21.15.151:0/2242806302 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faed81f5a30 0x7faed80cb7f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7faec40017e0 tx=0x7faec40063a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:01.293 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.290+0000 7faecf400640 1 -- 172.21.15.151:0/2242806302 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec4042070 con 0x7faed81f5a30 2024-09-16T15:42:01.293 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.291+0000 7faecf400640 1 -- 172.21.15.151:0/2242806302 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faec4039070 con 0x7faed81f5a30 2024-09-16T15:42:01.293 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.291+0000 7faecf400640 1 -- 172.21.15.151:0/2242806302 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec4003d70 con 0x7faed81f5a30 2024-09-16T15:42:01.293 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.291+0000 7faedde00640 1 -- 172.21.15.151:0/2242806302 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faed80afdc0 con 0x7faed81f5a30 2024-09-16T15:42:01.293 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.291+0000 7faedde00640 1 -- 172.21.15.151:0/2242806302 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faed80b0380 con 0x7faed81f5a30 2024-09-16T15:42:01.294 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.292+0000 7faedde00640 1 -- 172.21.15.151:0/2242806302 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faed815f9f0 con 0x7faed81f5a30 2024-09-16T15:42:01.294 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.293+0000 7faecf400640 1 -- 172.21.15.151:0/2242806302 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 9) ==== 101491+0+0 (secure 0 0 0) 0x7faec403b070 con 0x7faed81f5a30 2024-09-16T15:42:01.294 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.293+0000 7faecf400640 1 --2- 172.21.15.151:0/2242806302 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7faeb8078ac0 0x7faeb807af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:01.294 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.294+0000 7faecf400640 1 -- 172.21.15.151:0/2242806302 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5825+0+0 (secure 0 0 0) 0x7faec40ba230 con 0x7faed81f5a30 2024-09-16T15:42:01.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.294+0000 7faed6a00640 1 --2- 172.21.15.151:0/2242806302 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7faeb8078ac0 0x7faeb807af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:01.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.295+0000 7faed6a00640 1 --2- 172.21.15.151:0/2242806302 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7faeb8078ac0 0x7faeb807af80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7faed8052890 tx=0x7faec8005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:01.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.298+0000 7faecf400640 1 -- 172.21.15.151:0/2242806302 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7faec4081d90 con 0x7faed81f5a30 2024-09-16T15:42:01.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.511+0000 7faedde00640 1 -- 172.21.15.151:0/2242806302 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7faed80cbd30 con 0x7faeb8078ac0 2024-09-16T15:42:01.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.515+0000 7faecf400640 1 -- 172.21.15.151:0/2242806302 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+44 (secure 0 0 0) 0x7faec8012b20 con 0x7faeb8078ac0 2024-09-16T15:42:01.517 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:42:01.517 INFO:teuthology.orchestra.run.smithi151.stdout:[ 2024-09-16T15:42:01.517 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:42:01.517 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ss1clone1" 2024-09-16T15:42:01.517 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:42:01.517 INFO:teuthology.orchestra.run.smithi151.stdout:] 2024-09-16T15:42:01.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.517+0000 7faedde00640 1 -- 172.21.15.151:0/2242806302 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7faeb8078ac0 msgr2=0x7faeb807af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:01.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.517+0000 7faedde00640 1 --2- 172.21.15.151:0/2242806302 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7faeb8078ac0 0x7faeb807af80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7faed8052890 tx=0x7faec8005f90 comp rx=0 tx=0).stop 2024-09-16T15:42:01.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.517+0000 7faedde00640 1 -- 172.21.15.151:0/2242806302 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faed81f5a30 msgr2=0x7faed80cb7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:01.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.517+0000 7faedde00640 1 --2- 172.21.15.151:0/2242806302 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faed81f5a30 0x7faed80cb7f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7faec40017e0 tx=0x7faec40063a0 comp rx=0 tx=0).stop 2024-09-16T15:42:01.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.518+0000 7faedde00640 1 -- 172.21.15.151:0/2242806302 shutdown_connections 2024-09-16T15:42:01.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.518+0000 7faedde00640 1 --2- 172.21.15.151:0/2242806302 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7faeb8078ac0 0x7faeb807af80 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:01.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.518+0000 7faedde00640 1 --2- 172.21.15.151:0/2242806302 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faed81419f0 0x7faed81e58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:01.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.518+0000 7faedde00640 1 --2- 172.21.15.151:0/2242806302 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faed81f5a30 0x7faed80cb7f0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:01.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.518+0000 7faedde00640 1 --2- 172.21.15.151:0/2242806302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faebc004800 0x7faed815e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:01.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.518+0000 7faedde00640 1 -- 172.21.15.151:0/2242806302 >> 172.21.15.151:0/2242806302 conn(0x7faed80b1840 msgr2=0x7faed8141170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:01.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.518+0000 7faedde00640 1 -- 172.21.15.151:0/2242806302 shutdown_connections 2024-09-16T15:42:01.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.518+0000 7faedde00640 1 -- 172.21.15.151:0/2242806302 wait complete. 2024-09-16T15:42:01.534 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs ss1clone1 2024-09-16T15:42:01.697 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.696+0000 7f301e200640 1 -- 172.21.15.151:0/1851251557 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f301806e6f0 msgr2=0x7f301806eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:01.697 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.696+0000 7f301e200640 1 --2- 172.21.15.151:0/1851251557 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f301806e6f0 0x7f301806eb60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3010005fd0 tx=0x7f301002a5d0 comp rx=0 tx=0).stop 2024-09-16T15:42:01.697 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.696+0000 7f301e200640 1 -- 172.21.15.151:0/1851251557 shutdown_connections 2024-09-16T15:42:01.697 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.696+0000 7f301e200640 1 --2- 172.21.15.151:0/1851251557 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f301806e6f0 0x7f301806eb60 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:01.697 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.696+0000 7f301e200640 1 --2- 172.21.15.151:0/1851251557 >> v2:172.21.15.167:3300/0 conn(0x7f30180ccca0 0x7f30180cd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:01.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.696+0000 7f301e200640 1 -- 172.21.15.151:0/1851251557 >> 172.21.15.151:0/1851251557 conn(0x7f30180b18e0 msgr2=0x7f30180b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:01.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.696+0000 7f301e200640 1 -- 172.21.15.151:0/1851251557 shutdown_connections 2024-09-16T15:42:01.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.696+0000 7f301e200640 1 -- 172.21.15.151:0/1851251557 wait complete. 2024-09-16T15:42:01.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.697+0000 7f301e200640 1 Processor -- start 2024-09-16T15:42:01.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.698+0000 7f301e200640 1 -- start start 2024-09-16T15:42:01.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.698+0000 7f301e200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f301806e6f0 0x7f30180ca280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:01.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.699+0000 7f301e200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30180ccca0 0x7f3018159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:01.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.699+0000 7f301e200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3018158900 0x7f30181501e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:01.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.699+0000 7f301e200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f30180ced60 con 0x7f30180ccca0 2024-09-16T15:42:01.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.699+0000 7f301e200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f30180cebe0 con 0x7f301806e6f0 2024-09-16T15:42:01.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.699+0000 7f301e200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f30180ceee0 con 0x7f3018158900 2024-09-16T15:42:01.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.699+0000 7f3016a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30180ccca0 0x7f3018159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:01.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.699+0000 7f3016a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30180ccca0 0x7f3018159ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:46170/0 (socket says 172.21.15.151:46170) 2024-09-16T15:42:01.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.699+0000 7f3016a00640 1 -- 172.21.15.151:0/1315478508 learned_addr learned my addr 172.21.15.151:0/1315478508 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:42:01.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.699+0000 7f3017e00640 1 --2- 172.21.15.151:0/1315478508 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3018158900 0x7f30181501e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:01.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.699+0000 7f3017400640 1 --2- 172.21.15.151:0/1315478508 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f301806e6f0 0x7f30180ca280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:01.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.699+0000 7f3016a00640 1 -- 172.21.15.151:0/1315478508 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3018158900 msgr2=0x7f30181501e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:01.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.699+0000 7f3016a00640 1 --2- 172.21.15.151:0/1315478508 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3018158900 0x7f30181501e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:01.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.699+0000 7f3016a00640 1 -- 172.21.15.151:0/1315478508 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f301806e6f0 msgr2=0x7f30180ca280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:01.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.699+0000 7f3016a00640 1 --2- 172.21.15.151:0/1315478508 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f301806e6f0 0x7f30180ca280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:01.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.699+0000 7f3016a00640 1 -- 172.21.15.151:0/1315478508 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3018156060 con 0x7f30180ccca0 2024-09-16T15:42:01.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.700+0000 7f3016a00640 1 --2- 172.21.15.151:0/1315478508 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30180ccca0 0x7f3018159ce0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f300400ea60 tx=0x7f300400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:01.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.700+0000 7f300f400640 1 -- 172.21.15.151:0/1315478508 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f300400cde0 con 0x7f30180ccca0 2024-09-16T15:42:01.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.700+0000 7f300f400640 1 -- 172.21.15.151:0/1315478508 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3004004020 con 0x7f30180ccca0 2024-09-16T15:42:01.703 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.700+0000 7f301e200640 1 -- 172.21.15.151:0/1315478508 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3018156340 con 0x7f30180ccca0 2024-09-16T15:42:01.703 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.700+0000 7f300f400640 1 -- 172.21.15.151:0/1315478508 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3004004450 con 0x7f30180ccca0 2024-09-16T15:42:01.703 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.700+0000 7f301e200640 1 -- 172.21.15.151:0/1315478508 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3018156900 con 0x7f30180ccca0 2024-09-16T15:42:01.703 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.701+0000 7f301e200640 1 -- 172.21.15.151:0/1315478508 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3018069360 con 0x7f30180ccca0 2024-09-16T15:42:01.703 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.702+0000 7f300f400640 1 -- 172.21.15.151:0/1315478508 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 9) ==== 101491+0+0 (secure 0 0 0) 0x7f30040035f0 con 0x7f30180ccca0 2024-09-16T15:42:01.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.704+0000 7f300f400640 1 --2- 172.21.15.151:0/1315478508 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2ff8078a90 0x7f2ff807af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:01.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.704+0000 7f300f400640 1 -- 172.21.15.151:0/1315478508 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5825+0+0 (secure 0 0 0) 0x7f300409b900 con 0x7f30180ccca0 2024-09-16T15:42:01.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.704+0000 7f3017400640 1 --2- 172.21.15.151:0/1315478508 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2ff8078a90 0x7f2ff807af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:01.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.705+0000 7f3017400640 1 --2- 172.21.15.151:0/1315478508 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2ff8078a90 0x7f2ff807af50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3008001e20 tx=0x7f3008009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:01.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.708+0000 7f300f400640 1 -- 172.21.15.151:0/1315478508 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f3004063310 con 0x7f30180ccca0 2024-09-16T15:42:01.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:01.921+0000 7f301e200640 1 -- 172.21.15.151:0/1315478508 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f301815b4b0 con 0x7f2ff8078a90 2024-09-16T15:42:05.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:05.525+0000 7f300f400640 1 -- 172.21.15.151:0/1315478508 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f301815b4b0 con 0x7f2ff8078a90 2024-09-16T15:42:05.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:05.529+0000 7f301e200640 1 -- 172.21.15.151:0/1315478508 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2ff8078a90 msgr2=0x7f2ff807af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:05.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:05.529+0000 7f301e200640 1 --2- 172.21.15.151:0/1315478508 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2ff8078a90 0x7f2ff807af50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3008001e20 tx=0x7f3008009040 comp rx=0 tx=0).stop 2024-09-16T15:42:05.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:05.529+0000 7f301e200640 1 -- 172.21.15.151:0/1315478508 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30180ccca0 msgr2=0x7f3018159ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:05.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:05.529+0000 7f301e200640 1 --2- 172.21.15.151:0/1315478508 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30180ccca0 0x7f3018159ce0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f300400ea60 tx=0x7f300400ef20 comp rx=0 tx=0).stop 2024-09-16T15:42:05.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:05.529+0000 7f301e200640 1 -- 172.21.15.151:0/1315478508 shutdown_connections 2024-09-16T15:42:05.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:05.529+0000 7f301e200640 1 --2- 172.21.15.151:0/1315478508 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2ff8078a90 0x7f2ff807af50 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:05.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:05.529+0000 7f301e200640 1 --2- 172.21.15.151:0/1315478508 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3018158900 0x7f30181501e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:05.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:05.529+0000 7f301e200640 1 --2- 172.21.15.151:0/1315478508 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30180ccca0 0x7f3018159ce0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:05.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:05.529+0000 7f301e200640 1 --2- 172.21.15.151:0/1315478508 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f301806e6f0 0x7f30180ca280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:05.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:05.529+0000 7f301e200640 1 -- 172.21.15.151:0/1315478508 >> 172.21.15.151:0/1315478508 conn(0x7f30180b18e0 msgr2=0x7f3018159060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:05.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:05.529+0000 7f301e200640 1 -- 172.21.15.151:0/1315478508 shutdown_connections 2024-09-16T15:42:05.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:05.529+0000 7f301e200640 1 -- 172.21.15.151:0/1315478508 wait complete. 2024-09-16T15:42:05.532 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:42:05.530+0000 7f48f3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:42:05.532 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:42:05.530+0000 7f48f3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:42:05.532 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:42:05.530+0000 7f48f3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:42:05.532 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:42:05.530+0000 7f48f3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:42:05.532 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:42:05.530+0000 7f48f3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:42:05.549 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:42:05.548+0000 7f48f1c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:42:05.549 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:42:05.548+0000 7f48f1c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:42:05.549 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:42:05.548+0000 7f48f1c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:42:05.549 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:42:05.548+0000 7f48f1c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:42:05.549 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:42:05.548+0000 7f48f1c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:42:05.550 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-16T15:42:05.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:05 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:42:05.639 INFO:teuthology.orchestra.run.smithi151.stdout:2 2024-09-16T15:42:05.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:42:05.648 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-16T15:42:05.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:05.809+0000 7fba68a00640 1 -- 172.21.15.151:0/1242935792 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba64154c90 msgr2=0x7fba640cc7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:05.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:05.809+0000 7fba68a00640 1 --2- 172.21.15.151:0/1242935792 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba64154c90 0x7fba640cc7b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fba6415be60 tx=0x7fba5402a4f0 comp rx=0 tx=0).stop 2024-09-16T15:42:05.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:05.809+0000 7fba68a00640 1 -- 172.21.15.151:0/1242935792 shutdown_connections 2024-09-16T15:42:05.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:05.809+0000 7fba68a00640 1 --2- 172.21.15.151:0/1242935792 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fba640ad880 0x7fba640adcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:05.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:05.809+0000 7fba68a00640 1 --2- 172.21.15.151:0/1242935792 >> v2:172.21.15.167:3300/0 conn(0x7fba6406e580 0x7fba6406e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:05.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:05.809+0000 7fba68a00640 1 --2- 172.21.15.151:0/1242935792 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba64154c90 0x7fba640cc7b0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:05.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:05.809+0000 7fba68a00640 1 -- 172.21.15.151:0/1242935792 >> 172.21.15.151:0/1242935792 conn(0x7fba640b1510 msgr2=0x7fba640b18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:05.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:05.809+0000 7fba68a00640 1 -- 172.21.15.151:0/1242935792 shutdown_connections 2024-09-16T15:42:05.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:05.809+0000 7fba68a00640 1 -- 172.21.15.151:0/1242935792 wait complete. 2024-09-16T15:42:05.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:05.811+0000 7fba68a00640 1 Processor -- start 2024-09-16T15:42:05.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:05.811+0000 7fba68a00640 1 -- start start 2024-09-16T15:42:05.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:05.812+0000 7fba68a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba6406e580 0x7fba6414a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:05.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:05.812+0000 7fba68a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fba640ad880 0x7fba64158950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:05.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:05.812+0000 7fba68a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba64154c90 0x7fba64143e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:05.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:05.812+0000 7fba68a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fba640ca130 con 0x7fba6406e580 2024-09-16T15:42:05.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:05.812+0000 7fba68a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fba640c9fb0 con 0x7fba64154c90 2024-09-16T15:42:05.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:05.812+0000 7fba68a00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fba640ca2b0 con 0x7fba640ad880 2024-09-16T15:42:05.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:05.812+0000 7fba63400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba6406e580 0x7fba6414a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:05.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:05.812+0000 7fba63400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba6406e580 0x7fba6414a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41730/0 (socket says 172.21.15.151:41730) 2024-09-16T15:42:05.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:05.812+0000 7fba63400640 1 -- 172.21.15.151:0/580336551 learned_addr learned my addr 172.21.15.151:0/580336551 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:42:05.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:05.812+0000 7fba63e00640 1 --2- 172.21.15.151:0/580336551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba64154c90 0x7fba64143e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:05.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:05.812+0000 7fba63400640 1 -- 172.21.15.151:0/580336551 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fba640ad880 msgr2=0x7fba64158950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:05.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:05.812+0000 7fba62a00640 1 --2- 172.21.15.151:0/580336551 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fba640ad880 0x7fba64158950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:05.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:05.813+0000 7fba63400640 1 --2- 172.21.15.151:0/580336551 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fba640ad880 0x7fba64158950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:05.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:05.813+0000 7fba63400640 1 -- 172.21.15.151:0/580336551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba64154c90 msgr2=0x7fba64143e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:05.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:05.813+0000 7fba63400640 1 --2- 172.21.15.151:0/580336551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba64154c90 0x7fba64143e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:05.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:05.813+0000 7fba63400640 1 -- 172.21.15.151:0/580336551 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba641468c0 con 0x7fba6406e580 2024-09-16T15:42:05.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:05.813+0000 7fba63e00640 1 --2- 172.21.15.151:0/580336551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba64154c90 0x7fba64143e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:42:05.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:05.813+0000 7fba62a00640 1 --2- 172.21.15.151:0/580336551 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fba640ad880 0x7fba64158950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:42:05.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:05.813+0000 7fba63400640 1 --2- 172.21.15.151:0/580336551 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba6406e580 0x7fba6414a1e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fba54034040 tx=0x7fba540039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:05.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:05.813+0000 7fba5b400640 1 -- 172.21.15.151:0/580336551 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba5403e030 con 0x7fba6406e580 2024-09-16T15:42:05.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:05.813+0000 7fba5b400640 1 -- 172.21.15.151:0/580336551 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fba540047a0 con 0x7fba6406e580 2024-09-16T15:42:05.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:05.813+0000 7fba5b400640 1 -- 172.21.15.151:0/580336551 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba54035040 con 0x7fba6406e580 2024-09-16T15:42:05.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:05.813+0000 7fba68a00640 1 -- 172.21.15.151:0/580336551 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba64146ba0 con 0x7fba6406e580 2024-09-16T15:42:05.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:05.813+0000 7fba68a00640 1 -- 172.21.15.151:0/580336551 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba64147680 con 0x7fba6406e580 2024-09-16T15:42:05.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:05.814+0000 7fba68a00640 1 -- 172.21.15.151:0/580336551 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba640ca840 con 0x7fba6406e580 2024-09-16T15:42:05.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:05.815+0000 7fba5b400640 1 -- 172.21.15.151:0/580336551 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 9) ==== 101491+0+0 (secure 0 0 0) 0x7fba54041030 con 0x7fba6406e580 2024-09-16T15:42:05.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:05.816+0000 7fba5b400640 1 --2- 172.21.15.151:0/580336551 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fba44078ac0 0x7fba4407af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:05.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:05.816+0000 7fba5b400640 1 -- 172.21.15.151:0/580336551 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5825+0+0 (secure 0 0 0) 0x7fba540bb5d0 con 0x7fba6406e580 2024-09-16T15:42:05.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:05.817+0000 7fba62a00640 1 --2- 172.21.15.151:0/580336551 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fba44078ac0 0x7fba4407af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:05.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:05.817+0000 7fba62a00640 1 --2- 172.21.15.151:0/580336551 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fba44078ac0 0x7fba4407af80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fba640b0ba0 tx=0x7fba50005d30 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:05.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:05.820+0000 7fba5b400640 1 -- 172.21.15.151:0/580336551 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fba54083780 con 0x7fba6406e580 2024-09-16T15:42:05.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:05.897+0000 7fba5b400640 1 -- 172.21.15.151:0/580336551 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7fba5407f2f0 con 0x7fba6406e580 2024-09-16T15:42:06.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:06.009+0000 7fba68a00640 1 -- 172.21.15.151:0/580336551 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7fba64069d20 con 0x7fba6406e580 2024-09-16T15:42:06.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:06.009+0000 7fba5b400640 1 -- 172.21.15.151:0/580336551 <== mon.0 v2:172.21.15.151:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v29)=0 v29) ==== 131+0+0 (secure 0 0 0) 0x7fba540657d0 con 0x7fba6406e580 2024-09-16T15:42:06.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:06.012+0000 7fba68a00640 1 -- 172.21.15.151:0/580336551 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fba44078ac0 msgr2=0x7fba4407af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:06.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:06.012+0000 7fba68a00640 1 --2- 172.21.15.151:0/580336551 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fba44078ac0 0x7fba4407af80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fba640b0ba0 tx=0x7fba50005d30 comp rx=0 tx=0).stop 2024-09-16T15:42:06.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:06.012+0000 7fba68a00640 1 -- 172.21.15.151:0/580336551 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba6406e580 msgr2=0x7fba6414a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:06.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:06.012+0000 7fba68a00640 1 --2- 172.21.15.151:0/580336551 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba6406e580 0x7fba6414a1e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fba54034040 tx=0x7fba540039e0 comp rx=0 tx=0).stop 2024-09-16T15:42:06.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:06.012+0000 7fba68a00640 1 -- 172.21.15.151:0/580336551 shutdown_connections 2024-09-16T15:42:06.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:06.012+0000 7fba68a00640 1 --2- 172.21.15.151:0/580336551 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fba44078ac0 0x7fba4407af80 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:06.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:06.012+0000 7fba68a00640 1 --2- 172.21.15.151:0/580336551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba64154c90 0x7fba64143e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:06.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:06.012+0000 7fba68a00640 1 --2- 172.21.15.151:0/580336551 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fba640ad880 0x7fba64158950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:06.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:06.012+0000 7fba68a00640 1 --2- 172.21.15.151:0/580336551 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba6406e580 0x7fba6414a1e0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:06.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:06.012+0000 7fba68a00640 1 -- 172.21.15.151:0/580336551 >> 172.21.15.151:0/580336551 conn(0x7fba640b1510 msgr2=0x7fba640cf870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:06.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:06.012+0000 7fba68a00640 1 -- 172.21.15.151:0/580336551 shutdown_connections 2024-09-16T15:42:06.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:06.012+0000 7fba68a00640 1 -- 172.21.15.151:0/580336551 wait complete. 2024-09-16T15:42:06.029 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-16T15:42:06.030 DEBUG:teuthology.orchestra.run.smithi151:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:42:06.110 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:42:06.202 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-16T15:42:06.202 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:42:06.202 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:42:06.202 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:42:06.238 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:42:06.238 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:42:06.314 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-16T15:42:06.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:06.513+0000 7f339fe00640 1 -- 172.21.15.151:0/3604401259 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3384003a50 msgr2=0x7f3384003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:06.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:06.513+0000 7f339fe00640 1 --2- 172.21.15.151:0/3604401259 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3384003a50 0x7f3384003e70 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f33900016c0 tx=0x7f3384005830 comp rx=0 tx=0).stop 2024-09-16T15:42:06.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:06.513+0000 7f339fe00640 1 -- 172.21.15.151:0/3604401259 shutdown_connections 2024-09-16T15:42:06.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:06.513+0000 7f339fe00640 1 --2- 172.21.15.151:0/3604401259 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3384003a50 0x7f3384003e70 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:06.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:06.513+0000 7f339fe00640 1 -- 172.21.15.151:0/3604401259 >> 172.21.15.151:0/3604401259 conn(0x7f33a00cfa20 msgr2=0x7f33a00cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:06.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:06.513+0000 7f339fe00640 1 -- 172.21.15.151:0/3604401259 shutdown_connections 2024-09-16T15:42:06.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:06.513+0000 7f339fe00640 1 -- 172.21.15.151:0/3604401259 wait complete. 2024-09-16T15:42:06.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:06.514+0000 7f339fe00640 1 Processor -- start 2024-09-16T15:42:06.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:06.515+0000 7f339fe00640 1 -- start start 2024-09-16T15:42:06.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:06.515+0000 7f339fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3384003a50 0x7f33a0156990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:06.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:06.515+0000 7f339fe00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33a015d790 0x7f33a01f2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:06.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:06.515+0000 7f339fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f33a0153c60 0x7f33a0154670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:06.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:06.515+0000 7f339fe00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f33a00b2c80 con 0x7f33a0153c60 2024-09-16T15:42:06.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:06.516+0000 7f339fe00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f33a00b2b00 con 0x7f3384003a50 2024-09-16T15:42:06.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:06.516+0000 7f339fe00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f33a0073d40 con 0x7f33a015d790 2024-09-16T15:42:06.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:06.516+0000 7f339f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f33a0153c60 0x7f33a0154670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:06.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:06.516+0000 7f339f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f33a0153c60 0x7f33a0154670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41752/0 (socket says 172.21.15.151:41752) 2024-09-16T15:42:06.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:06.516+0000 7f339f400640 1 -- 172.21.15.151:0/3349246727 learned_addr learned my addr 172.21.15.151:0/3349246727 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:42:06.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:06.516+0000 7f339e000640 1 --2- 172.21.15.151:0/3349246727 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33a015d790 0x7f33a01f2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:06.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:06.516+0000 7f339ea00640 1 --2- 172.21.15.151:0/3349246727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3384003a50 0x7f33a0156990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:06.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:06.516+0000 7f339f400640 1 -- 172.21.15.151:0/3349246727 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33a015d790 msgr2=0x7f33a01f2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:06.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:06.516+0000 7f339f400640 1 --2- 172.21.15.151:0/3349246727 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33a015d790 0x7f33a01f2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:06.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:06.516+0000 7f339f400640 1 -- 172.21.15.151:0/3349246727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3384003a50 msgr2=0x7f33a0156990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:06.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:06.516+0000 7f339f400640 1 --2- 172.21.15.151:0/3349246727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3384003a50 0x7f33a0156990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:06.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:06.516+0000 7f339f400640 1 -- 172.21.15.151:0/3349246727 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33a0159e60 con 0x7f33a0153c60 2024-09-16T15:42:06.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:06.517+0000 7f339f400640 1 --2- 172.21.15.151:0/3349246727 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f33a0153c60 0x7f33a0154670 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3398002100 tx=0x7f339800eaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:06.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:06.517+0000 7f3396a00640 1 -- 172.21.15.151:0/3349246727 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3398015070 con 0x7f33a0153c60 2024-09-16T15:42:06.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:06.517+0000 7f3396a00640 1 -- 172.21.15.151:0/3349246727 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f339800ca00 con 0x7f33a0153c60 2024-09-16T15:42:06.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:06.517+0000 7f3396a00640 1 -- 172.21.15.151:0/3349246727 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339800ce50 con 0x7f33a0153c60 2024-09-16T15:42:06.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:06.517+0000 7f339fe00640 1 -- 172.21.15.151:0/3349246727 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33a015a140 con 0x7f33a0153c60 2024-09-16T15:42:06.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:06.517+0000 7f339fe00640 1 -- 172.21.15.151:0/3349246727 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33a015a700 con 0x7f33a0153c60 2024-09-16T15:42:06.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:06.518+0000 7f339fe00640 1 -- 172.21.15.151:0/3349246727 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33a00742a0 con 0x7f33a0153c60 2024-09-16T15:42:06.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:06.519+0000 7f3396a00640 1 -- 172.21.15.151:0/3349246727 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7f3398018070 con 0x7f33a0153c60 2024-09-16T15:42:06.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:06.520+0000 7f3396a00640 1 --2- 172.21.15.151:0/3349246727 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3380078be0 0x7f338007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:06.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:06.520+0000 7f3396a00640 1 -- 172.21.15.151:0/3349246727 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5825+0+0 (secure 0 0 0) 0x7f339809b3e0 con 0x7f33a0153c60 2024-09-16T15:42:06.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:06.521+0000 7f339ea00640 1 --2- 172.21.15.151:0/3349246727 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3380078be0 0x7f338007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:06.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:06.521+0000 7f339ea00640 1 --2- 172.21.15.151:0/3349246727 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3380078be0 0x7f338007b0a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3390032000 tx=0x7f339002c040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:06.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:06.524+0000 7f3396a00640 1 -- 172.21.15.151:0/3349246727 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f3398062db0 con 0x7f33a0153c60 2024-09-16T15:42:06.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:06.719+0000 7f339fe00640 1 -- 172.21.15.151:0/3349246727 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f33a0157d20 con 0x7f33a0153c60 2024-09-16T15:42:06.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:06.719+0000 7f3396a00640 1 -- 172.21.15.151:0/3349246727 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v37) ==== 80+0+0 (secure 0 0 0) 0x7f33980677e0 con 0x7f33a0153c60 2024-09-16T15:42:06.721 INFO:teuthology.orchestra.run.smithi151.stderr:listed 0 entries 2024-09-16T15:42:06.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:06.722+0000 7f339fe00640 1 -- 172.21.15.151:0/3349246727 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3380078be0 msgr2=0x7f338007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:06.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:06.722+0000 7f339fe00640 1 --2- 172.21.15.151:0/3349246727 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3380078be0 0x7f338007b0a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3390032000 tx=0x7f339002c040 comp rx=0 tx=0).stop 2024-09-16T15:42:06.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:06.722+0000 7f339fe00640 1 -- 172.21.15.151:0/3349246727 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f33a0153c60 msgr2=0x7f33a0154670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:06.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:06.722+0000 7f339fe00640 1 --2- 172.21.15.151:0/3349246727 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f33a0153c60 0x7f33a0154670 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3398002100 tx=0x7f339800eaa0 comp rx=0 tx=0).stop 2024-09-16T15:42:06.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:06.722+0000 7f339fe00640 1 -- 172.21.15.151:0/3349246727 shutdown_connections 2024-09-16T15:42:06.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:06.722+0000 7f339fe00640 1 --2- 172.21.15.151:0/3349246727 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3380078be0 0x7f338007b0a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:06.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:06.722+0000 7f339fe00640 1 --2- 172.21.15.151:0/3349246727 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f33a0153c60 0x7f33a0154670 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:06.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:06.722+0000 7f339fe00640 1 --2- 172.21.15.151:0/3349246727 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33a015d790 0x7f33a01f2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:06.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:06.722+0000 7f339fe00640 1 --2- 172.21.15.151:0/3349246727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3384003a50 0x7f33a0156990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:06.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:06.722+0000 7f339fe00640 1 -- 172.21.15.151:0/3349246727 >> 172.21.15.151:0/3349246727 conn(0x7f33a00cfa20 msgr2=0x7f33a00cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:06.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:06.722+0000 7f339fe00640 1 -- 172.21.15.151:0/3349246727 shutdown_connections 2024-09-16T15:42:06.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:06.722+0000 7f339fe00640 1 -- 172.21.15.151:0/3349246727 wait complete. 2024-09-16T15:42:06.739 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-16T15:42:06.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:42:06.739 DEBUG:teuthology.orchestra.run.smithi151:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-16T15:42:08.529 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi151.front.sepia.ceph.com 2024-09-16T15:42:08.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:42:08.530 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-16T15:42:08.593 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:42:08.593 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-16T15:42:08.593 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:42:08.648 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:42:08.648 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:42:08.648 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:42:08.648 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:42:08.695 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T15:42:08.696 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:42:08.696 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-16T15:42:08.696 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:42:08.732 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:42:08.732 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:42:08.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:08.893+0000 7fe189000640 1 -- 172.21.15.151:0/1517574383 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe1840ccf00 msgr2=0x7fe1840b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:08.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:08.893+0000 7fe189000640 1 --2- 172.21.15.151:0/1517574383 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe1840ccf00 0x7fe1840b2440 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe170009830 tx=0x7fe17002a480 comp rx=0 tx=0).stop 2024-09-16T15:42:08.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:08.893+0000 7fe189000640 1 -- 172.21.15.151:0/1517574383 shutdown_connections 2024-09-16T15:42:08.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:08.893+0000 7fe189000640 1 --2- 172.21.15.151:0/1517574383 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe1840ccf00 0x7fe1840b2440 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:08.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:08.893+0000 7fe189000640 1 -- 172.21.15.151:0/1517574383 >> 172.21.15.151:0/1517574383 conn(0x7fe1840b1950 msgr2=0x7fe1840b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:08.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:08.894+0000 7fe189000640 1 -- 172.21.15.151:0/1517574383 shutdown_connections 2024-09-16T15:42:08.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:08.894+0000 7fe189000640 1 -- 172.21.15.151:0/1517574383 wait complete. 2024-09-16T15:42:08.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:08.895+0000 7fe189000640 1 Processor -- start 2024-09-16T15:42:08.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:08.895+0000 7fe189000640 1 -- start start 2024-09-16T15:42:08.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:08.896+0000 7fe189000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe184073fb0 0x7fe18415a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:08.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:08.896+0000 7fe189000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1840ccf00 0x7fe1840cf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:08.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:08.896+0000 7fe189000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe18415f5a0 0x7fe1841572d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:08.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:08.896+0000 7fe189000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe18406f9f0 con 0x7fe18415f5a0 2024-09-16T15:42:08.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:08.896+0000 7fe189000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe18406f870 con 0x7fe1840ccf00 2024-09-16T15:42:08.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:08.896+0000 7fe189000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fe184074420 con 0x7fe184073fb0 2024-09-16T15:42:08.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:08.897+0000 7fe182800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe18415f5a0 0x7fe1841572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:08.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:08.897+0000 7fe181400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1840ccf00 0x7fe1840cf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:08.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:08.897+0000 7fe181e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe184073fb0 0x7fe18415a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:08.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:08.897+0000 7fe182800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe18415f5a0 0x7fe1841572d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41772/0 (socket says 172.21.15.151:41772) 2024-09-16T15:42:08.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:08.897+0000 7fe181400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1840ccf00 0x7fe1840cf5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.151:43184/0 (socket says 172.21.15.151:43184) 2024-09-16T15:42:08.899 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:08.897+0000 7fe182800640 1 -- 172.21.15.151:0/617538827 learned_addr learned my addr 172.21.15.151:0/617538827 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:42:08.899 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:08.897+0000 7fe182800640 1 -- 172.21.15.151:0/617538827 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe184073fb0 msgr2=0x7fe18415a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:08.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:08.897+0000 7fe182800640 1 --2- 172.21.15.151:0/617538827 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe184073fb0 0x7fe18415a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:08.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:08.897+0000 7fe182800640 1 -- 172.21.15.151:0/617538827 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1840ccf00 msgr2=0x7fe1840cf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:08.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:08.897+0000 7fe182800640 1 --2- 172.21.15.151:0/617538827 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1840ccf00 0x7fe1840cf5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:08.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:08.897+0000 7fe182800640 1 -- 172.21.15.151:0/617538827 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1841f6370 con 0x7fe18415f5a0 2024-09-16T15:42:08.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:08.897+0000 7fe182800640 1 --2- 172.21.15.151:0/617538827 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe18415f5a0 0x7fe1841572d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe17c00c770 tx=0x7fe17c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:08.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:08.898+0000 7fe17a000640 1 -- 172.21.15.151:0/617538827 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe17c017070 con 0x7fe18415f5a0 2024-09-16T15:42:08.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:08.898+0000 7fe17a000640 1 -- 172.21.15.151:0/617538827 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe17c005070 con 0x7fe18415f5a0 2024-09-16T15:42:08.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:08.898+0000 7fe189000640 1 -- 172.21.15.151:0/617538827 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1841f6650 con 0x7fe18415f5a0 2024-09-16T15:42:08.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:08.898+0000 7fe17a000640 1 -- 172.21.15.151:0/617538827 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe17c0054c0 con 0x7fe18415f5a0 2024-09-16T15:42:08.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:08.898+0000 7fe189000640 1 -- 172.21.15.151:0/617538827 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1841f28a0 con 0x7fe18415f5a0 2024-09-16T15:42:08.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:08.899+0000 7fe189000640 1 -- 172.21.15.151:0/617538827 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1841f2ad0 con 0x7fe18415f5a0 2024-09-16T15:42:08.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:08.900+0000 7fe17a000640 1 -- 172.21.15.151:0/617538827 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7fe17c01a070 con 0x7fe18415f5a0 2024-09-16T15:42:08.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:08.901+0000 7fe17a000640 1 --2- 172.21.15.151:0/617538827 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe164078bb0 0x7fe16407b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:08.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:08.901+0000 7fe17a000640 1 -- 172.21.15.151:0/617538827 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5825+0+0 (secure 0 0 0) 0x7fe17c09a5a0 con 0x7fe18415f5a0 2024-09-16T15:42:08.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:08.901+0000 7fe181e00640 1 --2- 172.21.15.151:0/617538827 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe164078bb0 0x7fe16407b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:08.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:08.902+0000 7fe181e00640 1 --2- 172.21.15.151:0/617538827 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe164078bb0 0x7fe16407b070 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe174006fd0 tx=0x7fe174009060 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:08.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:08.906+0000 7fe17a000640 1 -- 172.21.15.151:0/617538827 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe1841f2ad0 con 0x7fe18415f5a0 2024-09-16T15:42:09.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.128+0000 7fe189000640 1 -- 172.21.15.151:0/617538827 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe1840b2440 con 0x7fe18415f5a0 2024-09-16T15:42:09.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.128+0000 7fe17a000640 1 -- 172.21.15.151:0/617538827 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5232 (secure 0 0 0) 0x7fe17c062070 con 0x7fe18415f5a0 2024-09-16T15:42:09.129 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:42:09.130 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":18,"btime":"2024-09-16T15:40:25:485623+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4661,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/873312529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":873312529},{"type":"v1","addr":"172.21.15.151:6834","nonce":873312529}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/2683082898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2683082898},{"type":"v1","addr":"172.21.15.167:6835","nonce":2683082898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4843,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2443370538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":2443370538},{"type":"v1","addr":"172.21.15.151:6835","nonce":2443370538}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:40:11.836645+0000","modified":"2024-09-16T15:40:14.867199+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4855},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4855":{"gid":4855,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6837/4150137107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":4150137107},{"type":"v1","addr":"172.21.15.167:6837","nonce":4150137107}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4855,"qdb_cluster":[4855]},"id":2}]} 2024-09-16T15:42:09.130 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 18 2024-09-16T15:42:09.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.130+0000 7fe189000640 1 -- 172.21.15.151:0/617538827 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe164078bb0 msgr2=0x7fe16407b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:09.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.130+0000 7fe189000640 1 --2- 172.21.15.151:0/617538827 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe164078bb0 0x7fe16407b070 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe174006fd0 tx=0x7fe174009060 comp rx=0 tx=0).stop 2024-09-16T15:42:09.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.130+0000 7fe189000640 1 -- 172.21.15.151:0/617538827 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe18415f5a0 msgr2=0x7fe1841572d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:09.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.130+0000 7fe189000640 1 --2- 172.21.15.151:0/617538827 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe18415f5a0 0x7fe1841572d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe17c00c770 tx=0x7fe17c00cc30 comp rx=0 tx=0).stop 2024-09-16T15:42:09.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.131+0000 7fe189000640 1 -- 172.21.15.151:0/617538827 shutdown_connections 2024-09-16T15:42:09.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.131+0000 7fe189000640 1 --2- 172.21.15.151:0/617538827 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe164078bb0 0x7fe16407b070 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:09.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.131+0000 7fe189000640 1 --2- 172.21.15.151:0/617538827 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1840ccf00 0x7fe1840cf5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:09.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.131+0000 7fe189000640 1 --2- 172.21.15.151:0/617538827 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe18415f5a0 0x7fe1841572d0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:09.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.131+0000 7fe189000640 1 --2- 172.21.15.151:0/617538827 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe184073fb0 0x7fe18415a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:09.133 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.131+0000 7fe189000640 1 -- 172.21.15.151:0/617538827 >> 172.21.15.151:0/617538827 conn(0x7fe1840b1950 msgr2=0x7fe18409caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:09.133 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.131+0000 7fe189000640 1 -- 172.21.15.151:0/617538827 shutdown_connections 2024-09-16T15:42:09.133 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.131+0000 7fe189000640 1 -- 172.21.15.151:0/617538827 wait complete. 2024-09-16T15:42:09.148 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:42:09.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.305+0000 7f318dc00640 1 -- 172.21.15.151:0/3828649467 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31880d0540 msgr2=0x7f318806f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:09.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.305+0000 7f318dc00640 1 --2- 172.21.15.151:0/3828649467 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31880d0540 0x7f318806f320 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3180005fd0 tx=0x7f318002aa90 comp rx=0 tx=0).stop 2024-09-16T15:42:09.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.305+0000 7f318dc00640 1 -- 172.21.15.151:0/3828649467 shutdown_connections 2024-09-16T15:42:09.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.305+0000 7f318dc00640 1 --2- 172.21.15.151:0/3828649467 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31880d0540 0x7f318806f320 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:09.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.305+0000 7f318dc00640 1 --2- 172.21.15.151:0/3828649467 >> v2:172.21.15.167:3300/0 conn(0x7f31880725d0 0x7f31880b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:09.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.305+0000 7f318dc00640 1 -- 172.21.15.151:0/3828649467 >> 172.21.15.151:0/3828649467 conn(0x7f31880cfa20 msgr2=0x7f31880cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:09.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.305+0000 7f318dc00640 1 -- 172.21.15.151:0/3828649467 shutdown_connections 2024-09-16T15:42:09.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.305+0000 7f318dc00640 1 -- 172.21.15.151:0/3828649467 wait complete. 2024-09-16T15:42:09.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.307+0000 7f318dc00640 1 Processor -- start 2024-09-16T15:42:09.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.307+0000 7f318dc00640 1 -- start start 2024-09-16T15:42:09.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.308+0000 7f318dc00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31880d0540 0x7f31880ac580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:09.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.308+0000 7f318dc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f318815bc30 0x7f3188151f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:09.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.308+0000 7f318dc00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31880acac0 0x7f3188152450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:09.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.308+0000 7f318dc00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3188074660 con 0x7f318815bc30 2024-09-16T15:42:09.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.308+0000 7f318dc00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f318806f860 con 0x7f31880d0540 2024-09-16T15:42:09.311 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.308+0000 7f318dc00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f31880c8cb0 con 0x7f31880acac0 2024-09-16T15:42:09.311 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.308+0000 7f3187400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f318815bc30 0x7f3188151f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:09.311 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.308+0000 7f3187400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f318815bc30 0x7f3188151f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41786/0 (socket says 172.21.15.151:41786) 2024-09-16T15:42:09.311 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.308+0000 7f3187400640 1 -- 172.21.15.151:0/133049387 learned_addr learned my addr 172.21.15.151:0/133049387 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:42:09.311 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.308+0000 7f318d200640 1 --2- 172.21.15.151:0/133049387 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31880acac0 0x7f3188152450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:09.311 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.308+0000 7f3187e00640 1 --2- 172.21.15.151:0/133049387 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31880d0540 0x7f31880ac580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:09.311 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.308+0000 7f3187400640 1 -- 172.21.15.151:0/133049387 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31880acac0 msgr2=0x7f3188152450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:09.311 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.308+0000 7f3187400640 1 --2- 172.21.15.151:0/133049387 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31880acac0 0x7f3188152450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:09.311 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.308+0000 7f3187400640 1 -- 172.21.15.151:0/133049387 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31880d0540 msgr2=0x7f31880ac580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:09.311 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.308+0000 7f3187400640 1 --2- 172.21.15.151:0/133049387 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31880d0540 0x7f31880ac580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:09.311 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.308+0000 7f3187400640 1 -- 172.21.15.151:0/133049387 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31880a9b00 con 0x7f318815bc30 2024-09-16T15:42:09.311 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.309+0000 7f3187400640 1 --2- 172.21.15.151:0/133049387 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f318815bc30 0x7f3188151f10 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f317400e9c0 tx=0x7f317400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:09.311 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.309+0000 7f317fe00640 1 -- 172.21.15.151:0/133049387 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f317400cd80 con 0x7f318815bc30 2024-09-16T15:42:09.312 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.309+0000 7f317fe00640 1 -- 172.21.15.151:0/133049387 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3174005020 con 0x7f318815bc30 2024-09-16T15:42:09.312 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.309+0000 7f318dc00640 1 -- 172.21.15.151:0/133049387 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31880a9de0 con 0x7f318815bc30 2024-09-16T15:42:09.312 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.309+0000 7f317fe00640 1 -- 172.21.15.151:0/133049387 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3174005470 con 0x7f318815bc30 2024-09-16T15:42:09.312 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.309+0000 7f318dc00640 1 -- 172.21.15.151:0/133049387 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31880c7f40 con 0x7f318815bc30 2024-09-16T15:42:09.312 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.310+0000 7f318dc00640 1 -- 172.21.15.151:0/133049387 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f318806fd60 con 0x7f318815bc30 2024-09-16T15:42:09.312 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.311+0000 7f317fe00640 1 -- 172.21.15.151:0/133049387 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7f31740047a0 con 0x7f318815bc30 2024-09-16T15:42:09.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.313+0000 7f317fe00640 1 --2- 172.21.15.151:0/133049387 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3168078bb0 0x7f316807b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:09.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.313+0000 7f317fe00640 1 -- 172.21.15.151:0/133049387 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5825+0+0 (secure 0 0 0) 0x7f317409bae0 con 0x7f318815bc30 2024-09-16T15:42:09.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.313+0000 7f3187e00640 1 --2- 172.21.15.151:0/133049387 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3168078bb0 0x7f316807b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:09.315 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.314+0000 7f3187e00640 1 --2- 172.21.15.151:0/133049387 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3168078bb0 0x7f316807b070 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3178001720 tx=0x7f3178009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:09.318 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.317+0000 7f317fe00640 1 -- 172.21.15.151:0/133049387 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f31740633a0 con 0x7f318815bc30 2024-09-16T15:42:09.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.540+0000 7f318dc00640 1 -- 172.21.15.151:0/133049387 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f31880cacb0 con 0x7f318815bc30 2024-09-16T15:42:09.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.540+0000 7f317fe00640 1 -- 172.21.15.151:0/133049387 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5232 (secure 0 0 0) 0x7f3174067dd0 con 0x7f318815bc30 2024-09-16T15:42:09.541 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:42:09.542 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":18,"btime":"2024-09-16T15:40:25:485623+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4661,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/873312529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":873312529},{"type":"v1","addr":"172.21.15.151:6834","nonce":873312529}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/2683082898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2683082898},{"type":"v1","addr":"172.21.15.167:6835","nonce":2683082898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4843,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2443370538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":2443370538},{"type":"v1","addr":"172.21.15.151:6835","nonce":2443370538}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:40:11.836645+0000","modified":"2024-09-16T15:40:14.867199+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4855},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4855":{"gid":4855,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6837/4150137107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":4150137107},{"type":"v1","addr":"172.21.15.167:6837","nonce":4150137107}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4855,"qdb_cluster":[4855]},"id":2}]} 2024-09-16T15:42:09.542 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 18 2024-09-16T15:42:09.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.543+0000 7f318dc00640 1 -- 172.21.15.151:0/133049387 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3168078bb0 msgr2=0x7f316807b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:09.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.543+0000 7f318dc00640 1 --2- 172.21.15.151:0/133049387 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3168078bb0 0x7f316807b070 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3178001720 tx=0x7f3178009040 comp rx=0 tx=0).stop 2024-09-16T15:42:09.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.543+0000 7f318dc00640 1 -- 172.21.15.151:0/133049387 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f318815bc30 msgr2=0x7f3188151f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:09.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.543+0000 7f318dc00640 1 --2- 172.21.15.151:0/133049387 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f318815bc30 0x7f3188151f10 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f317400e9c0 tx=0x7f317400ee80 comp rx=0 tx=0).stop 2024-09-16T15:42:09.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.543+0000 7f318dc00640 1 -- 172.21.15.151:0/133049387 shutdown_connections 2024-09-16T15:42:09.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.543+0000 7f318dc00640 1 --2- 172.21.15.151:0/133049387 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3168078bb0 0x7f316807b070 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:09.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.543+0000 7f318dc00640 1 --2- 172.21.15.151:0/133049387 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31880acac0 0x7f3188152450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:09.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.543+0000 7f318dc00640 1 --2- 172.21.15.151:0/133049387 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f318815bc30 0x7f3188151f10 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:09.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.543+0000 7f318dc00640 1 --2- 172.21.15.151:0/133049387 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31880d0540 0x7f31880ac580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:09.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.543+0000 7f318dc00640 1 -- 172.21.15.151:0/133049387 >> 172.21.15.151:0/133049387 conn(0x7f31880cfa20 msgr2=0x7f31880cd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:09.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.543+0000 7f318dc00640 1 -- 172.21.15.151:0/133049387 shutdown_connections 2024-09-16T15:42:09.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.543+0000 7f318dc00640 1 -- 172.21.15.151:0/133049387 wait complete. 2024-09-16T15:42:09.560 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:42:09.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.717+0000 7fbca0a00640 1 -- 172.21.15.151:0/2658588703 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbc9c15bc30 msgr2=0x7fbc9c0b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:09.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.717+0000 7fbca0a00640 1 --2- 172.21.15.151:0/2658588703 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbc9c15bc30 0x7fbc9c0b28f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c009830 tx=0x7fbc8c02a510 comp rx=0 tx=0).stop 2024-09-16T15:42:09.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.717+0000 7fbca0a00640 1 -- 172.21.15.151:0/2658588703 shutdown_connections 2024-09-16T15:42:09.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.717+0000 7fbca0a00640 1 --2- 172.21.15.151:0/2658588703 >> v2:172.21.15.167:3300/0 conn(0x7fbc9c0d0540 0x7fbc9c06f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:09.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.717+0000 7fbca0a00640 1 --2- 172.21.15.151:0/2658588703 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbc9c15bc30 0x7fbc9c0b28f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:09.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.717+0000 7fbca0a00640 1 -- 172.21.15.151:0/2658588703 >> 172.21.15.151:0/2658588703 conn(0x7fbc9c0cfa20 msgr2=0x7fbc9c0cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:09.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.717+0000 7fbca0a00640 1 -- 172.21.15.151:0/2658588703 shutdown_connections 2024-09-16T15:42:09.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.717+0000 7fbca0a00640 1 -- 172.21.15.151:0/2658588703 wait complete. 2024-09-16T15:42:09.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.719+0000 7fbca0a00640 1 Processor -- start 2024-09-16T15:42:09.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.719+0000 7fbca0a00640 1 -- start start 2024-09-16T15:42:09.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.720+0000 7fbca0a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc9c0d0540 0x7fbc9c157db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:09.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.720+0000 7fbca0a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbc9c15bc30 0x7fbc9c0c86c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:09.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.720+0000 7fbca0a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc9c0c6af0 0x7fbc9c0c8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:09.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.720+0000 7fbca0a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9c074660 con 0x7fbc9c0d0540 2024-09-16T15:42:09.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.720+0000 7fbca0a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9c15c200 con 0x7fbc9c0c6af0 2024-09-16T15:42:09.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.720+0000 7fbca0a00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fbc9c1f7a20 con 0x7fbc9c15bc30 2024-09-16T15:42:09.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.720+0000 7fbc9a800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc9c0d0540 0x7fbc9c157db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:09.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.720+0000 7fbc9a800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc9c0d0540 0x7fbc9c157db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41798/0 (socket says 172.21.15.151:41798) 2024-09-16T15:42:09.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.720+0000 7fbc9a800640 1 -- 172.21.15.151:0/4157347937 learned_addr learned my addr 172.21.15.151:0/4157347937 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:42:09.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.720+0000 7fbc9b200640 1 --2- 172.21.15.151:0/4157347937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc9c0c6af0 0x7fbc9c0c8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:09.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.720+0000 7fbc99e00640 1 --2- 172.21.15.151:0/4157347937 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbc9c15bc30 0x7fbc9c0c86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:09.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.720+0000 7fbc9a800640 1 -- 172.21.15.151:0/4157347937 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbc9c15bc30 msgr2=0x7fbc9c0c86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:09.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.720+0000 7fbc9a800640 1 --2- 172.21.15.151:0/4157347937 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbc9c15bc30 0x7fbc9c0c86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:09.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.720+0000 7fbc9a800640 1 -- 172.21.15.151:0/4157347937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc9c0c6af0 msgr2=0x7fbc9c0c8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:09.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.720+0000 7fbc9a800640 1 --2- 172.21.15.151:0/4157347937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc9c0c6af0 0x7fbc9c0c8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:09.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.720+0000 7fbc9a800640 1 -- 172.21.15.151:0/4157347937 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc9c1f7d00 con 0x7fbc9c0d0540 2024-09-16T15:42:09.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.721+0000 7fbc9a800640 1 --2- 172.21.15.151:0/4157347937 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc9c0d0540 0x7fbc9c157db0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c009950 tx=0x7fbc8c033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:09.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.721+0000 7fbc92a00640 1 -- 172.21.15.151:0/4157347937 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc8c005c90 con 0x7fbc9c0d0540 2024-09-16T15:42:09.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.721+0000 7fbc92a00640 1 -- 172.21.15.151:0/4157347937 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbc8c031e50 con 0x7fbc9c0d0540 2024-09-16T15:42:09.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.721+0000 7fbca0a00640 1 -- 172.21.15.151:0/4157347937 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc9c1f7fe0 con 0x7fbc9c0d0540 2024-09-16T15:42:09.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.721+0000 7fbc92a00640 1 -- 172.21.15.151:0/4157347937 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc8c005260 con 0x7fbc9c0d0540 2024-09-16T15:42:09.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.721+0000 7fbca0a00640 1 -- 172.21.15.151:0/4157347937 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc9c1f85a0 con 0x7fbc9c0d0540 2024-09-16T15:42:09.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.722+0000 7fbca0a00640 1 -- 172.21.15.151:0/4157347937 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc9c06fc40 con 0x7fbc9c0d0540 2024-09-16T15:42:09.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.723+0000 7fbc92a00640 1 -- 172.21.15.151:0/4157347937 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7fbc8c005400 con 0x7fbc9c0d0540 2024-09-16T15:42:09.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.724+0000 7fbc92a00640 1 --2- 172.21.15.151:0/4157347937 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbc7c078bb0 0x7fbc7c07b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:09.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.724+0000 7fbc92a00640 1 -- 172.21.15.151:0/4157347937 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5825+0+0 (secure 0 0 0) 0x7fbc8c0bbf00 con 0x7fbc9c0d0540 2024-09-16T15:42:09.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.725+0000 7fbc99e00640 1 --2- 172.21.15.151:0/4157347937 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbc7c078bb0 0x7fbc7c07b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:09.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.725+0000 7fbc99e00640 1 --2- 172.21.15.151:0/4157347937 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbc7c078bb0 0x7fbc7c07b070 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fbc88007fd0 tx=0x7fbc88001430 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:09.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.728+0000 7fbc92a00640 1 -- 172.21.15.151:0/4157347937 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fbc8c03f030 con 0x7fbc9c0d0540 2024-09-16T15:42:09.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.913+0000 7fbca0a00640 1 -- 172.21.15.151:0/4157347937 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbc9c15f600 con 0x7fbc9c0d0540 2024-09-16T15:42:09.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.914+0000 7fbc92a00640 1 -- 172.21.15.151:0/4157347937 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) ==== 74+0+16978 (secure 0 0 0) 0x7fbc8c083840 con 0x7fbc9c0d0540 2024-09-16T15:42:09.916 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:42:09.916 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":37,"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","created":"2024-09-16T15:38:56.348535+0000","modified":"2024-09-16T15:41:03.700706+0000","last_up_change":"2024-09-16T15:39:10.681384+0000","last_in_change":"2024-09-16T15:39:00.544477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:39:11.602252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:40:10.163597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-16T15:40:10.798968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5524076-a903-4d0a-af6a-caf474c4765d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6809","nonce":3111342741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6811","nonce":3111342741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6815","nonce":3111342741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6813","nonce":3111342741}]},"public_addr":"172.21.15.151:6809/3111342741","cluster_addr":"172.21.15.151:6811/3111342741","heartbeat_back_addr":"172.21.15.151:6815/3111342741","heartbeat_front_addr":"172.21.15.151:6813/3111342741","state":["exists","up"]},{"osd":1,"uuid":"dd25925f-0ae1-494e-8529-4dc205759db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6801","nonce":3368304087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6803","nonce":3368304087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6807","nonce":3368304087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6805","nonce":3368304087}]},"public_addr":"172.21.15.151:6801/3368304087","cluster_addr":"172.21.15.151:6803/3368304087","heartbeat_back_addr":"172.21.15.151:6807/3368304087","heartbeat_front_addr":"172.21.15.151:6805/3368304087","state":["exists","up"]},{"osd":2,"uuid":"c2f18115-013d-4da6-a0fe-cb978f4494b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6823","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6825","nonce":3079373271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6827","nonce":3079373271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6831","nonce":3079373271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6829","nonce":3079373271}]},"public_addr":"172.21.15.151:6825/3079373271","cluster_addr":"172.21.15.151:6827/3079373271","heartbeat_back_addr":"172.21.15.151:6831/3079373271","heartbeat_front_addr":"172.21.15.151:6829/3079373271","state":["exists","up"]},{"osd":3,"uuid":"36edb718-2fcb-4d88-8ddc-c321eed4279d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6817","nonce":1142509041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6819","nonce":1142509041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6824","nonce":1142509041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6821","nonce":1142509041}]},"public_addr":"172.21.15.151:6817/1142509041","cluster_addr":"172.21.15.151:6819/1142509041","heartbeat_back_addr":"172.21.15.151:6824/1142509041","heartbeat_front_addr":"172.21.15.151:6821/1142509041","state":["exists","up"]},{"osd":4,"uuid":"64d33708-cd0a-4236-abe3-1b646e82ab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6811","nonce":2735495271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6813","nonce":2735495271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6817","nonce":2735495271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6815","nonce":2735495271}]},"public_addr":"172.21.15.167:6811/2735495271","cluster_addr":"172.21.15.167:6813/2735495271","heartbeat_back_addr":"172.21.15.167:6817/2735495271","heartbeat_front_addr":"172.21.15.167:6815/2735495271","state":["exists","up"]},{"osd":5,"uuid":"f14b44a9-5910-458c-b518-09a8d0de94de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6827","nonce":1494916572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6829","nonce":1494916572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6833","nonce":1494916572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6831","nonce":1494916572}]},"public_addr":"172.21.15.167:6827/1494916572","cluster_addr":"172.21.15.167:6829/1494916572","heartbeat_back_addr":"172.21.15.167:6833/1494916572","heartbeat_front_addr":"172.21.15.167:6831/1494916572","state":["exists","up"]},{"osd":6,"uuid":"39335545-2654-4710-84b0-a9aa47cfcbba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6819","nonce":2538273273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6821","nonce":2538273273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6825","nonce":2538273273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6823","nonce":2538273273}]},"public_addr":"172.21.15.167:6819/2538273273","cluster_addr":"172.21.15.167:6821/2538273273","heartbeat_back_addr":"172.21.15.167:6825/2538273273","heartbeat_front_addr":"172.21.15.167:6823/2538273273","state":["exists","up"]},{"osd":7,"uuid":"4f8bfc81-53dd-4339-ac22-1a4007d9f963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6803","nonce":265072227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6805","nonce":265072227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6809","nonce":265072227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6807","nonce":265072227}]},"public_addr":"172.21.15.167:6803/265072227","cluster_addr":"172.21.15.167:6805/265072227","heartbeat_back_addr":"172.21.15.167:6809/265072227","heartbeat_front_addr":"172.21.15.167:6807/265072227","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.150359+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.084358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.722368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.332809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.320709+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.395153+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.935250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.801443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"5.e","mappings":[{"from":4,"to":5}]},{"pgid":"5.17","mappings":[{"from":4,"to":3}]},{"pgid":"5.1a","mappings":[{"from":4,"to":5}]},{"pgid":"5.27","mappings":[{"from":4,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:42:09.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.917+0000 7fbca0a00640 1 -- 172.21.15.151:0/4157347937 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbc7c078bb0 msgr2=0x7fbc7c07b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:09.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.917+0000 7fbca0a00640 1 --2- 172.21.15.151:0/4157347937 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbc7c078bb0 0x7fbc7c07b070 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fbc88007fd0 tx=0x7fbc88001430 comp rx=0 tx=0).stop 2024-09-16T15:42:09.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.917+0000 7fbca0a00640 1 -- 172.21.15.151:0/4157347937 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc9c0d0540 msgr2=0x7fbc9c157db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:09.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.917+0000 7fbca0a00640 1 --2- 172.21.15.151:0/4157347937 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc9c0d0540 0x7fbc9c157db0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c009950 tx=0x7fbc8c033c20 comp rx=0 tx=0).stop 2024-09-16T15:42:09.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.917+0000 7fbca0a00640 1 -- 172.21.15.151:0/4157347937 shutdown_connections 2024-09-16T15:42:09.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.917+0000 7fbca0a00640 1 --2- 172.21.15.151:0/4157347937 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbc7c078bb0 0x7fbc7c07b070 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:09.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.917+0000 7fbca0a00640 1 --2- 172.21.15.151:0/4157347937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc9c0c6af0 0x7fbc9c0c8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:09.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.917+0000 7fbca0a00640 1 --2- 172.21.15.151:0/4157347937 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbc9c15bc30 0x7fbc9c0c86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:09.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.917+0000 7fbca0a00640 1 --2- 172.21.15.151:0/4157347937 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc9c0d0540 0x7fbc9c157db0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:09.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.917+0000 7fbca0a00640 1 -- 172.21.15.151:0/4157347937 >> 172.21.15.151:0/4157347937 conn(0x7fbc9c0cfa20 msgr2=0x7fbc9c06e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:09.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.917+0000 7fbca0a00640 1 -- 172.21.15.151:0/4157347937 shutdown_connections 2024-09-16T15:42:09.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:09.917+0000 7fbca0a00640 1 -- 172.21.15.151:0/4157347937 wait complete. 2024-09-16T15:42:09.935 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-16T15:42:09.935 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:42:10.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.090+0000 7f5fab200640 1 -- 172.21.15.151:0/1754258541 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5fac0d0540 msgr2=0x7f5fac074280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:10.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.090+0000 7f5fab200640 1 --2- 172.21.15.151:0/1754258541 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5fac0d0540 0x7f5fac074280 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5fac0cd570 tx=0x7f5fa402aa10 comp rx=0 tx=0).stop 2024-09-16T15:42:10.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.091+0000 7f5fab200640 1 -- 172.21.15.151:0/1754258541 shutdown_connections 2024-09-16T15:42:10.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.091+0000 7f5fab200640 1 --2- 172.21.15.151:0/1754258541 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5fac0d0540 0x7f5fac074280 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:10.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.091+0000 7f5fab200640 1 --2- 172.21.15.151:0/1754258541 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5fac0725d0 0x7f5fac073d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:10.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.091+0000 7f5fab200640 1 --2- 172.21.15.151:0/1754258541 >> v2:172.21.15.167:3300/0 conn(0x7f5fac15bc30 0x7f5fac0b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:10.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.091+0000 7f5fab200640 1 -- 172.21.15.151:0/1754258541 >> 172.21.15.151:0/1754258541 conn(0x7f5fac0cfa20 msgr2=0x7f5fac0cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:10.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.091+0000 7f5fab200640 1 -- 172.21.15.151:0/1754258541 shutdown_connections 2024-09-16T15:42:10.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.091+0000 7f5fab200640 1 -- 172.21.15.151:0/1754258541 wait complete. 2024-09-16T15:42:10.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.092+0000 7f5fab200640 1 Processor -- start 2024-09-16T15:42:10.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.092+0000 7f5fab200640 1 -- start start 2024-09-16T15:42:10.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.093+0000 7f5fab200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5fac0d0540 0x7f5fac1f8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:10.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.093+0000 7f5fab200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5fac15bc30 0x7f5fac0c7690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:10.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.093+0000 7f5fab200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5fac0c7bd0 0x7f5fac155630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:10.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.093+0000 7f5fab200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac06f320 con 0x7f5fac0c7bd0 2024-09-16T15:42:10.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.093+0000 7f5fab200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac06f4a0 con 0x7f5fac0d0540 2024-09-16T15:42:10.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.093+0000 7f5fab200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f5fac06f620 con 0x7f5fac15bc30 2024-09-16T15:42:10.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.094+0000 7f5faa800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5fac0c7bd0 0x7f5fac155630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:10.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.094+0000 7f5faa800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5fac0c7bd0 0x7f5fac155630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41816/0 (socket says 172.21.15.151:41816) 2024-09-16T15:42:10.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.094+0000 7f5faa800640 1 -- 172.21.15.151:0/2760044789 learned_addr learned my addr 172.21.15.151:0/2760044789 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:42:10.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.094+0000 7f5fa1400640 1 --2- 172.21.15.151:0/2760044789 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5fac15bc30 0x7f5fac0c7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:10.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.094+0000 7f5fa9e00640 1 --2- 172.21.15.151:0/2760044789 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5fac0d0540 0x7f5fac1f8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:10.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.094+0000 7f5faa800640 1 -- 172.21.15.151:0/2760044789 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5fac15bc30 msgr2=0x7f5fac0c7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:10.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.094+0000 7f5faa800640 1 --2- 172.21.15.151:0/2760044789 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5fac15bc30 0x7f5fac0c7690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:10.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.094+0000 7f5faa800640 1 -- 172.21.15.151:0/2760044789 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5fac0d0540 msgr2=0x7f5fac1f8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:10.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.094+0000 7f5faa800640 1 --2- 172.21.15.151:0/2760044789 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5fac0d0540 0x7f5fac1f8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:10.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.094+0000 7f5faa800640 1 -- 172.21.15.151:0/2760044789 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fac0c8070 con 0x7f5fac0c7bd0 2024-09-16T15:42:10.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.094+0000 7f5fa9e00640 1 --2- 172.21.15.151:0/2760044789 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5fac0d0540 0x7f5fac1f8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:42:10.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.094+0000 7f5fa1400640 1 --2- 172.21.15.151:0/2760044789 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5fac15bc30 0x7f5fac0c7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:42:10.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.094+0000 7f5faa800640 1 --2- 172.21.15.151:0/2760044789 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5fac0c7bd0 0x7f5fac155630 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4009d60 tx=0x7f5fa4003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:10.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.094+0000 7f5fa2a00640 1 -- 172.21.15.151:0/2760044789 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa4047070 con 0x7f5fac0c7bd0 2024-09-16T15:42:10.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.095+0000 7f5fa2a00640 1 -- 172.21.15.151:0/2760044789 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5fa4004250 con 0x7f5fac0c7bd0 2024-09-16T15:42:10.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.095+0000 7f5fab200640 1 -- 172.21.15.151:0/2760044789 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fac1f7c00 con 0x7f5fac0c7bd0 2024-09-16T15:42:10.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.095+0000 7f5fa2a00640 1 -- 172.21.15.151:0/2760044789 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa4033050 con 0x7f5fac0c7bd0 2024-09-16T15:42:10.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.095+0000 7f5fab200640 1 -- 172.21.15.151:0/2760044789 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fac0c8350 con 0x7f5fac0c7bd0 2024-09-16T15:42:10.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.096+0000 7f5fab200640 1 -- 172.21.15.151:0/2760044789 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fac0b28f0 con 0x7f5fac0c7bd0 2024-09-16T15:42:10.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.097+0000 7f5fa2a00640 1 -- 172.21.15.151:0/2760044789 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7f5fa404a070 con 0x7f5fac0c7bd0 2024-09-16T15:42:10.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.098+0000 7f5fa2a00640 1 --2- 172.21.15.151:0/2760044789 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5f88078bb0 0x7f5f8807b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:10.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.098+0000 7f5fa2a00640 1 -- 172.21.15.151:0/2760044789 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5825+0+0 (secure 0 0 0) 0x7f5fa40bc320 con 0x7f5fac0c7bd0 2024-09-16T15:42:10.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.098+0000 7f5fa9e00640 1 --2- 172.21.15.151:0/2760044789 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5f88078bb0 0x7f5f8807b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:10.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.099+0000 7f5fa9e00640 1 --2- 172.21.15.151:0/2760044789 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5f88078bb0 0x7f5f8807b070 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c00ba60 tx=0x7f5f9c005cf0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:10.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.102+0000 7f5fa2a00640 1 -- 172.21.15.151:0/2760044789 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f5fa4084440 con 0x7f5fac0c7bd0 2024-09-16T15:42:10.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.321+0000 7f5fab200640 1 -- 172.21.15.151:0/2760044789 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5fac156970 con 0x7f5fac0c7bd0 2024-09-16T15:42:10.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.322+0000 7f5fa2a00640 1 -- 172.21.15.151:0/2760044789 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5232 (secure 0 0 0) 0x7f5fa4083d40 con 0x7f5fac0c7bd0 2024-09-16T15:42:10.323 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:42:10.323 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":18,"btime":"2024-09-16T15:40:25:485623+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4661,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/873312529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":873312529},{"type":"v1","addr":"172.21.15.151:6834","nonce":873312529}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/2683082898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2683082898},{"type":"v1","addr":"172.21.15.167:6835","nonce":2683082898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4843,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2443370538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":2443370538},{"type":"v1","addr":"172.21.15.151:6835","nonce":2443370538}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:40:11.836645+0000","modified":"2024-09-16T15:40:14.867199+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4855},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4855":{"gid":4855,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6837/4150137107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":4150137107},{"type":"v1","addr":"172.21.15.167:6837","nonce":4150137107}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4855,"qdb_cluster":[4855]},"id":2}]} 2024-09-16T15:42:10.323 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 18 2024-09-16T15:42:10.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.324+0000 7f5fab200640 1 -- 172.21.15.151:0/2760044789 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5f88078bb0 msgr2=0x7f5f8807b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:10.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.324+0000 7f5fab200640 1 --2- 172.21.15.151:0/2760044789 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5f88078bb0 0x7f5f8807b070 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c00ba60 tx=0x7f5f9c005cf0 comp rx=0 tx=0).stop 2024-09-16T15:42:10.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.324+0000 7f5fab200640 1 -- 172.21.15.151:0/2760044789 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5fac0c7bd0 msgr2=0x7f5fac155630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:10.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.324+0000 7f5fab200640 1 --2- 172.21.15.151:0/2760044789 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5fac0c7bd0 0x7f5fac155630 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4009d60 tx=0x7f5fa4003f10 comp rx=0 tx=0).stop 2024-09-16T15:42:10.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.324+0000 7f5fab200640 1 -- 172.21.15.151:0/2760044789 shutdown_connections 2024-09-16T15:42:10.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.324+0000 7f5fab200640 1 --2- 172.21.15.151:0/2760044789 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5f88078bb0 0x7f5f8807b070 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:10.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.324+0000 7f5fab200640 1 --2- 172.21.15.151:0/2760044789 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5fac0c7bd0 0x7f5fac155630 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:10.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.324+0000 7f5fab200640 1 --2- 172.21.15.151:0/2760044789 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5fac15bc30 0x7f5fac0c7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:10.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.324+0000 7f5fab200640 1 --2- 172.21.15.151:0/2760044789 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5fac0d0540 0x7f5fac1f8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:10.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.324+0000 7f5fab200640 1 -- 172.21.15.151:0/2760044789 >> 172.21.15.151:0/2760044789 conn(0x7f5fac0cfa20 msgr2=0x7f5fac0b4fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:10.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.324+0000 7f5fab200640 1 -- 172.21.15.151:0/2760044789 shutdown_connections 2024-09-16T15:42:10.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.324+0000 7f5fab200640 1 -- 172.21.15.151:0/2760044789 wait complete. 2024-09-16T15:42:10.342 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:42:10.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.498+0000 7fdd7a800640 1 -- 172.21.15.151:0/1481878990 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdd7415bd30 msgr2=0x7fdd740d2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:10.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.498+0000 7fdd7a800640 1 --2- 172.21.15.151:0/1481878990 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdd7415bd30 0x7fdd740d2760 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fdd740c52c0 tx=0x7fdd6802a560 comp rx=0 tx=0).stop 2024-09-16T15:42:10.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.498+0000 7fdd7a800640 1 -- 172.21.15.151:0/1481878990 shutdown_connections 2024-09-16T15:42:10.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.498+0000 7fdd7a800640 1 --2- 172.21.15.151:0/1481878990 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdd740cc750 0x7fdd740ccc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:10.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.498+0000 7fdd7a800640 1 --2- 172.21.15.151:0/1481878990 >> v2:172.21.15.167:3300/0 conn(0x7fdd740cff60 0x7fdd740d03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:10.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.498+0000 7fdd7a800640 1 --2- 172.21.15.151:0/1481878990 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdd7415bd30 0x7fdd740d2760 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:10.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.498+0000 7fdd7a800640 1 -- 172.21.15.151:0/1481878990 >> 172.21.15.151:0/1481878990 conn(0x7fdd740d1c40 msgr2=0x7fdd740d2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:10.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.498+0000 7fdd7a800640 1 -- 172.21.15.151:0/1481878990 shutdown_connections 2024-09-16T15:42:10.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.498+0000 7fdd7a800640 1 -- 172.21.15.151:0/1481878990 wait complete. 2024-09-16T15:42:10.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.499+0000 7fdd7a800640 1 Processor -- start 2024-09-16T15:42:10.501 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.500+0000 7fdd7a800640 1 -- start start 2024-09-16T15:42:10.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.500+0000 7fdd7a800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdd740cc750 0x7fdd7406f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:10.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.501+0000 7fdd7a800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd740cff60 0x7fdd74152200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:10.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.501+0000 7fdd7a800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdd7415bd30 0x7fdd740b2020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:10.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.501+0000 7fdd7a800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdd740cd140 con 0x7fdd740cc750 2024-09-16T15:42:10.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.501+0000 7fdd7a800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdd740cfa20 con 0x7fdd740cff60 2024-09-16T15:42:10.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.501+0000 7fdd7a800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fdd740cd2c0 con 0x7fdd7415bd30 2024-09-16T15:42:10.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.501+0000 7fdd79400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdd740cc750 0x7fdd7406f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:10.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.501+0000 7fdd79400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdd740cc750 0x7fdd7406f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41828/0 (socket says 172.21.15.151:41828) 2024-09-16T15:42:10.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.501+0000 7fdd79400640 1 -- 172.21.15.151:0/2966409101 learned_addr learned my addr 172.21.15.151:0/2966409101 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:42:10.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.501+0000 7fdd79e00640 1 --2- 172.21.15.151:0/2966409101 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdd7415bd30 0x7fdd740b2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:10.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.501+0000 7fdd78a00640 1 --2- 172.21.15.151:0/2966409101 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd740cff60 0x7fdd74152200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:10.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.501+0000 7fdd79400640 1 -- 172.21.15.151:0/2966409101 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdd7415bd30 msgr2=0x7fdd740b2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:10.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.501+0000 7fdd79400640 1 --2- 172.21.15.151:0/2966409101 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdd7415bd30 0x7fdd740b2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:10.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.501+0000 7fdd79400640 1 -- 172.21.15.151:0/2966409101 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd740cff60 msgr2=0x7fdd74152200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:10.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.501+0000 7fdd79400640 1 --2- 172.21.15.151:0/2966409101 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd740cff60 0x7fdd74152200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:10.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.501+0000 7fdd79400640 1 -- 172.21.15.151:0/2966409101 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd740b5b70 con 0x7fdd740cc750 2024-09-16T15:42:10.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.501+0000 7fdd78a00640 1 --2- 172.21.15.151:0/2966409101 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd740cff60 0x7fdd74152200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:42:10.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.502+0000 7fdd79400640 1 --2- 172.21.15.151:0/2966409101 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdd740cc750 0x7fdd7406f680 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fdd68000c00 tx=0x7fdd68003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:10.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.502+0000 7fdd71600640 1 -- 172.21.15.151:0/2966409101 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd6803e030 con 0x7fdd740cc750 2024-09-16T15:42:10.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.502+0000 7fdd71600640 1 -- 172.21.15.151:0/2966409101 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdd68003e00 con 0x7fdd740cc750 2024-09-16T15:42:10.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.502+0000 7fdd71600640 1 -- 172.21.15.151:0/2966409101 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd68035040 con 0x7fdd740cc750 2024-09-16T15:42:10.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.502+0000 7fdd7a800640 1 -- 172.21.15.151:0/2966409101 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd740b5e50 con 0x7fdd740cc750 2024-09-16T15:42:10.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.502+0000 7fdd7a800640 1 -- 172.21.15.151:0/2966409101 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd740b6410 con 0x7fdd740cc750 2024-09-16T15:42:10.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.503+0000 7fdd7a800640 1 -- 172.21.15.151:0/2966409101 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd740d03a0 con 0x7fdd740cc750 2024-09-16T15:42:10.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.504+0000 7fdd71600640 1 -- 172.21.15.151:0/2966409101 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7fdd68041030 con 0x7fdd740cc750 2024-09-16T15:42:10.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.505+0000 7fdd71600640 1 --2- 172.21.15.151:0/2966409101 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdd54078bb0 0x7fdd5407b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:10.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.505+0000 7fdd71600640 1 -- 172.21.15.151:0/2966409101 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5825+0+0 (secure 0 0 0) 0x7fdd680bb5d0 con 0x7fdd740cc750 2024-09-16T15:42:10.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.506+0000 7fdd78a00640 1 --2- 172.21.15.151:0/2966409101 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdd54078bb0 0x7fdd5407b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:10.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.506+0000 7fdd78a00640 1 --2- 172.21.15.151:0/2966409101 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdd54078bb0 0x7fdd5407b070 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdd600058c0 tx=0x7fdd60005850 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:10.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.509+0000 7fdd71600640 1 -- 172.21.15.151:0/2966409101 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fdd68083010 con 0x7fdd740cc750 2024-09-16T15:42:10.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.728+0000 7fdd7a800640 1 -- 172.21.15.151:0/2966409101 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdd740b0900 con 0x7fdd740cc750 2024-09-16T15:42:10.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.729+0000 7fdd71600640 1 -- 172.21.15.151:0/2966409101 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5232 (secure 0 0 0) 0x7fdd68087a40 con 0x7fdd740cc750 2024-09-16T15:42:10.730 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:42:10.730 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":18,"btime":"2024-09-16T15:40:25:485623+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4661,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/873312529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":873312529},{"type":"v1","addr":"172.21.15.151:6834","nonce":873312529}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/2683082898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2683082898},{"type":"v1","addr":"172.21.15.167:6835","nonce":2683082898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4843,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2443370538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":2443370538},{"type":"v1","addr":"172.21.15.151:6835","nonce":2443370538}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:40:11.836645+0000","modified":"2024-09-16T15:40:14.867199+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4855},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4855":{"gid":4855,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6837/4150137107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":4150137107},{"type":"v1","addr":"172.21.15.167:6837","nonce":4150137107}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4855,"qdb_cluster":[4855]},"id":2}]} 2024-09-16T15:42:10.730 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 18 2024-09-16T15:42:10.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.731+0000 7fdd7a800640 1 -- 172.21.15.151:0/2966409101 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdd54078bb0 msgr2=0x7fdd5407b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:10.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.731+0000 7fdd7a800640 1 --2- 172.21.15.151:0/2966409101 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdd54078bb0 0x7fdd5407b070 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdd600058c0 tx=0x7fdd60005850 comp rx=0 tx=0).stop 2024-09-16T15:42:10.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.731+0000 7fdd7a800640 1 -- 172.21.15.151:0/2966409101 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdd740cc750 msgr2=0x7fdd7406f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:10.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.731+0000 7fdd7a800640 1 --2- 172.21.15.151:0/2966409101 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdd740cc750 0x7fdd7406f680 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fdd68000c00 tx=0x7fdd68003910 comp rx=0 tx=0).stop 2024-09-16T15:42:10.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.731+0000 7fdd7a800640 1 -- 172.21.15.151:0/2966409101 shutdown_connections 2024-09-16T15:42:10.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.731+0000 7fdd7a800640 1 --2- 172.21.15.151:0/2966409101 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdd54078bb0 0x7fdd5407b070 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:10.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.731+0000 7fdd7a800640 1 --2- 172.21.15.151:0/2966409101 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd740cff60 0x7fdd74152200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:10.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.731+0000 7fdd7a800640 1 --2- 172.21.15.151:0/2966409101 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdd7415bd30 0x7fdd740b2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:10.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.731+0000 7fdd7a800640 1 --2- 172.21.15.151:0/2966409101 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdd740cc750 0x7fdd7406f680 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:10.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.731+0000 7fdd7a800640 1 -- 172.21.15.151:0/2966409101 >> 172.21.15.151:0/2966409101 conn(0x7fdd740d1c40 msgr2=0x7fdd74072180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:10.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.731+0000 7fdd7a800640 1 -- 172.21.15.151:0/2966409101 shutdown_connections 2024-09-16T15:42:10.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.731+0000 7fdd7a800640 1 -- 172.21.15.151:0/2966409101 wait complete. 2024-09-16T15:42:10.748 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:42:10.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.904+0000 7f2301200640 1 -- 172.21.15.151:0/3306133091 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f22fc161550 msgr2=0x7f22fc06ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:10.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.904+0000 7f2301200640 1 --2- 172.21.15.151:0/3306133091 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f22fc161550 0x7f22fc06ef50 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f22ec005fd0 tx=0x7f22ec02a820 comp rx=0 tx=0).stop 2024-09-16T15:42:10.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.904+0000 7f2301200640 1 -- 172.21.15.151:0/3306133091 shutdown_connections 2024-09-16T15:42:10.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.904+0000 7f2301200640 1 --2- 172.21.15.151:0/3306133091 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f22fc0af3d0 0x7f22fc06f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:10.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.904+0000 7f2301200640 1 --2- 172.21.15.151:0/3306133091 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f22fc161550 0x7f22fc06ef50 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:10.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.904+0000 7f2301200640 1 -- 172.21.15.151:0/3306133091 >> 172.21.15.151:0/3306133091 conn(0x7f22fc0b1950 msgr2=0x7f22fc0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:10.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.904+0000 7f2301200640 1 -- 172.21.15.151:0/3306133091 shutdown_connections 2024-09-16T15:42:10.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.905+0000 7f2301200640 1 -- 172.21.15.151:0/3306133091 wait complete. 2024-09-16T15:42:10.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.906+0000 7f2301200640 1 Processor -- start 2024-09-16T15:42:10.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.906+0000 7f2301200640 1 -- start start 2024-09-16T15:42:10.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.907+0000 7f2301200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f22fc06f9d0 0x7f22fc146480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:10.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.907+0000 7f2301200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22fc0af3d0 0x7f22fc09d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:10.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.907+0000 7f2301200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f22fc161550 0x7f22fc0ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:10.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.907+0000 7f2301200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f22fc1623a0 con 0x7f22fc06f9d0 2024-09-16T15:42:10.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.907+0000 7f2301200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f22fc074010 con 0x7f22fc0af3d0 2024-09-16T15:42:10.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.907+0000 7f2301200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f22fc162520 con 0x7f22fc161550 2024-09-16T15:42:10.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.908+0000 7f22f9e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f22fc06f9d0 0x7f22fc146480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:10.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.908+0000 7f22f9e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f22fc06f9d0 0x7f22fc146480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41848/0 (socket says 172.21.15.151:41848) 2024-09-16T15:42:10.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.908+0000 7f22f9e00640 1 -- 172.21.15.151:0/737642153 learned_addr learned my addr 172.21.15.151:0/737642153 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:42:10.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.908+0000 7f22f9400640 1 --2- 172.21.15.151:0/737642153 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22fc0af3d0 0x7f22fc09d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:10.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.908+0000 7f22fa800640 1 --2- 172.21.15.151:0/737642153 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f22fc161550 0x7f22fc0ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:10.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.908+0000 7f22f9e00640 1 -- 172.21.15.151:0/737642153 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f22fc161550 msgr2=0x7f22fc0ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:10.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.908+0000 7f22f9e00640 1 --2- 172.21.15.151:0/737642153 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f22fc161550 0x7f22fc0ccb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:10.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.908+0000 7f22f9e00640 1 -- 172.21.15.151:0/737642153 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22fc0af3d0 msgr2=0x7f22fc09d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:10.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.908+0000 7f22f9e00640 1 --2- 172.21.15.151:0/737642153 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22fc0af3d0 0x7f22fc09d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:10.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.908+0000 7f22f9e00640 1 -- 172.21.15.151:0/737642153 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22fc1f5960 con 0x7f22fc06f9d0 2024-09-16T15:42:10.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.908+0000 7f22f9400640 1 --2- 172.21.15.151:0/737642153 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22fc0af3d0 0x7f22fc09d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:42:10.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.908+0000 7f22f9e00640 1 --2- 172.21.15.151:0/737642153 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f22fc06f9d0 0x7f22fc146480 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f22ec006010 tx=0x7f22ec032c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:10.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.909+0000 7f22f2000640 1 -- 172.21.15.151:0/737642153 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ec037040 con 0x7f22fc06f9d0 2024-09-16T15:42:10.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.909+0000 7f22f2000640 1 -- 172.21.15.151:0/737642153 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f22ec042070 con 0x7f22fc06f9d0 2024-09-16T15:42:10.912 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.909+0000 7f22f2000640 1 -- 172.21.15.151:0/737642153 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ec0042a0 con 0x7f22fc06f9d0 2024-09-16T15:42:10.912 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.909+0000 7f2301200640 1 -- 172.21.15.151:0/737642153 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22fc0ced80 con 0x7f22fc06f9d0 2024-09-16T15:42:10.912 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.909+0000 7f2301200640 1 -- 172.21.15.151:0/737642153 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22fc0cf340 con 0x7f22fc06f9d0 2024-09-16T15:42:10.912 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.910+0000 7f2301200640 1 -- 172.21.15.151:0/737642153 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22fc0760b0 con 0x7f22fc06f9d0 2024-09-16T15:42:10.912 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.911+0000 7f22f2000640 1 -- 172.21.15.151:0/737642153 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7f22ec004440 con 0x7f22fc06f9d0 2024-09-16T15:42:10.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.912+0000 7f22f2000640 1 --2- 172.21.15.151:0/737642153 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f22dc078be0 0x7f22dc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:10.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.912+0000 7f22f2000640 1 -- 172.21.15.151:0/737642153 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5825+0+0 (secure 0 0 0) 0x7f22ec0bbdd0 con 0x7f22fc06f9d0 2024-09-16T15:42:10.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.913+0000 7f22f9400640 1 --2- 172.21.15.151:0/737642153 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f22dc078be0 0x7f22dc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:10.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.913+0000 7f22f9400640 1 --2- 172.21.15.151:0/737642153 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f22dc078be0 0x7f22dc07b0a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f22e80058c0 tx=0x7f22e8005850 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:10.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:10.916+0000 7f22f2000640 1 -- 172.21.15.151:0/737642153 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f22ec083e60 con 0x7f22fc06f9d0 2024-09-16T15:42:11.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.101+0000 7f2301200640 1 -- 172.21.15.151:0/737642153 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f22fc162ab0 con 0x7f22fc06f9d0 2024-09-16T15:42:11.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.102+0000 7f22f2000640 1 -- 172.21.15.151:0/737642153 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) ==== 74+0+16978 (secure 0 0 0) 0x7f22ec083760 con 0x7f22fc06f9d0 2024-09-16T15:42:11.103 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:42:11.103 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":37,"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","created":"2024-09-16T15:38:56.348535+0000","modified":"2024-09-16T15:41:03.700706+0000","last_up_change":"2024-09-16T15:39:10.681384+0000","last_in_change":"2024-09-16T15:39:00.544477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:39:11.602252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:40:10.163597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-16T15:40:10.798968+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5524076-a903-4d0a-af6a-caf474c4765d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6809","nonce":3111342741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6811","nonce":3111342741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6815","nonce":3111342741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6813","nonce":3111342741}]},"public_addr":"172.21.15.151:6809/3111342741","cluster_addr":"172.21.15.151:6811/3111342741","heartbeat_back_addr":"172.21.15.151:6815/3111342741","heartbeat_front_addr":"172.21.15.151:6813/3111342741","state":["exists","up"]},{"osd":1,"uuid":"dd25925f-0ae1-494e-8529-4dc205759db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6801","nonce":3368304087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6803","nonce":3368304087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6807","nonce":3368304087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6805","nonce":3368304087}]},"public_addr":"172.21.15.151:6801/3368304087","cluster_addr":"172.21.15.151:6803/3368304087","heartbeat_back_addr":"172.21.15.151:6807/3368304087","heartbeat_front_addr":"172.21.15.151:6805/3368304087","state":["exists","up"]},{"osd":2,"uuid":"c2f18115-013d-4da6-a0fe-cb978f4494b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6823","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6825","nonce":3079373271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6827","nonce":3079373271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6831","nonce":3079373271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6829","nonce":3079373271}]},"public_addr":"172.21.15.151:6825/3079373271","cluster_addr":"172.21.15.151:6827/3079373271","heartbeat_back_addr":"172.21.15.151:6831/3079373271","heartbeat_front_addr":"172.21.15.151:6829/3079373271","state":["exists","up"]},{"osd":3,"uuid":"36edb718-2fcb-4d88-8ddc-c321eed4279d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6817","nonce":1142509041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6819","nonce":1142509041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6824","nonce":1142509041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6821","nonce":1142509041}]},"public_addr":"172.21.15.151:6817/1142509041","cluster_addr":"172.21.15.151:6819/1142509041","heartbeat_back_addr":"172.21.15.151:6824/1142509041","heartbeat_front_addr":"172.21.15.151:6821/1142509041","state":["exists","up"]},{"osd":4,"uuid":"64d33708-cd0a-4236-abe3-1b646e82ab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6811","nonce":2735495271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6813","nonce":2735495271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6817","nonce":2735495271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6815","nonce":2735495271}]},"public_addr":"172.21.15.167:6811/2735495271","cluster_addr":"172.21.15.167:6813/2735495271","heartbeat_back_addr":"172.21.15.167:6817/2735495271","heartbeat_front_addr":"172.21.15.167:6815/2735495271","state":["exists","up"]},{"osd":5,"uuid":"f14b44a9-5910-458c-b518-09a8d0de94de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6827","nonce":1494916572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6829","nonce":1494916572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6833","nonce":1494916572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6831","nonce":1494916572}]},"public_addr":"172.21.15.167:6827/1494916572","cluster_addr":"172.21.15.167:6829/1494916572","heartbeat_back_addr":"172.21.15.167:6833/1494916572","heartbeat_front_addr":"172.21.15.167:6831/1494916572","state":["exists","up"]},{"osd":6,"uuid":"39335545-2654-4710-84b0-a9aa47cfcbba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6819","nonce":2538273273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6821","nonce":2538273273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6825","nonce":2538273273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6823","nonce":2538273273}]},"public_addr":"172.21.15.167:6819/2538273273","cluster_addr":"172.21.15.167:6821/2538273273","heartbeat_back_addr":"172.21.15.167:6825/2538273273","heartbeat_front_addr":"172.21.15.167:6823/2538273273","state":["exists","up"]},{"osd":7,"uuid":"4f8bfc81-53dd-4339-ac22-1a4007d9f963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6803","nonce":265072227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6805","nonce":265072227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6809","nonce":265072227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6807","nonce":265072227}]},"public_addr":"172.21.15.167:6803/265072227","cluster_addr":"172.21.15.167:6805/265072227","heartbeat_back_addr":"172.21.15.167:6809/265072227","heartbeat_front_addr":"172.21.15.167:6807/265072227","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.150359+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.084358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.722368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.332809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.320709+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.395153+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.935250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.801443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"5.e","mappings":[{"from":4,"to":5}]},{"pgid":"5.17","mappings":[{"from":4,"to":3}]},{"pgid":"5.1a","mappings":[{"from":4,"to":5}]},{"pgid":"5.27","mappings":[{"from":4,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:42:11.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.104+0000 7f2301200640 1 -- 172.21.15.151:0/737642153 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f22dc078be0 msgr2=0x7f22dc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:11.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.104+0000 7f2301200640 1 --2- 172.21.15.151:0/737642153 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f22dc078be0 0x7f22dc07b0a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f22e80058c0 tx=0x7f22e8005850 comp rx=0 tx=0).stop 2024-09-16T15:42:11.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.104+0000 7f2301200640 1 -- 172.21.15.151:0/737642153 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f22fc06f9d0 msgr2=0x7f22fc146480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:11.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.104+0000 7f2301200640 1 --2- 172.21.15.151:0/737642153 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f22fc06f9d0 0x7f22fc146480 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f22ec006010 tx=0x7f22ec032c90 comp rx=0 tx=0).stop 2024-09-16T15:42:11.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.104+0000 7f2301200640 1 -- 172.21.15.151:0/737642153 shutdown_connections 2024-09-16T15:42:11.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.104+0000 7f2301200640 1 --2- 172.21.15.151:0/737642153 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f22dc078be0 0x7f22dc07b0a0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:11.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.104+0000 7f2301200640 1 --2- 172.21.15.151:0/737642153 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f22fc0af3d0 0x7f22fc09d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:11.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.104+0000 7f2301200640 1 --2- 172.21.15.151:0/737642153 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f22fc161550 0x7f22fc0ccb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:11.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.104+0000 7f2301200640 1 --2- 172.21.15.151:0/737642153 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f22fc06f9d0 0x7f22fc146480 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:11.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.104+0000 7f2301200640 1 -- 172.21.15.151:0/737642153 >> 172.21.15.151:0/737642153 conn(0x7f22fc0b1950 msgr2=0x7f22fc15ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:11.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.104+0000 7f2301200640 1 -- 172.21.15.151:0/737642153 shutdown_connections 2024-09-16T15:42:11.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.104+0000 7f2301200640 1 -- 172.21.15.151:0/737642153 wait complete. 2024-09-16T15:42:11.122 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-16T15:42:11.281 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.280+0000 7f5073200640 1 -- 172.21.15.151:0/2226925550 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f50740725d0 msgr2=0x7f50740b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:11.281 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.280+0000 7f5073200640 1 --2- 172.21.15.151:0/2226925550 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f50740725d0 0x7f50740b28f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5060009830 tx=0x7f506002a510 comp rx=0 tx=0).stop 2024-09-16T15:42:11.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.280+0000 7f5073200640 1 -- 172.21.15.151:0/2226925550 shutdown_connections 2024-09-16T15:42:11.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.280+0000 7f5073200640 1 --2- 172.21.15.151:0/2226925550 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50740d0540 0x7f507406f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:11.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.280+0000 7f5073200640 1 --2- 172.21.15.151:0/2226925550 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f50740725d0 0x7f50740b28f0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:11.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.280+0000 7f5073200640 1 -- 172.21.15.151:0/2226925550 >> 172.21.15.151:0/2226925550 conn(0x7f50740cfa20 msgr2=0x7f50740cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:11.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.280+0000 7f5073200640 1 -- 172.21.15.151:0/2226925550 shutdown_connections 2024-09-16T15:42:11.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.280+0000 7f5073200640 1 -- 172.21.15.151:0/2226925550 wait complete. 2024-09-16T15:42:11.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.282+0000 7f5073200640 1 Processor -- start 2024-09-16T15:42:11.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.282+0000 7f5073200640 1 -- start start 2024-09-16T15:42:11.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.283+0000 7f5073200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f50740d0540 0x7f50740aadc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:11.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.283+0000 7f5073200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f507415bc30 0x7f50740c88d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:11.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.283+0000 7f5073200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50740c8e10 0x7f50741f4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:11.286 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.283+0000 7f5073200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5074074660 con 0x7f50740d0540 2024-09-16T15:42:11.286 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.283+0000 7f5073200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f507406f860 con 0x7f507415bc30 2024-09-16T15:42:11.286 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.283+0000 7f5073200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f50740a9470 con 0x7f50740c8e10 2024-09-16T15:42:11.286 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.283+0000 7f5071e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f50740d0540 0x7f50740aadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:11.286 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.283+0000 7f5071e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f50740d0540 0x7f50740aadc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41868/0 (socket says 172.21.15.151:41868) 2024-09-16T15:42:11.286 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.283+0000 7f5071e00640 1 -- 172.21.15.151:0/3055577872 learned_addr learned my addr 172.21.15.151:0/3055577872 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:42:11.286 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.283+0000 7f5071400640 1 --2- 172.21.15.151:0/3055577872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f507415bc30 0x7f50740c88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:11.286 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.283+0000 7f5072800640 1 --2- 172.21.15.151:0/3055577872 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50740c8e10 0x7f50741f4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:11.286 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.284+0000 7f5071e00640 1 -- 172.21.15.151:0/3055577872 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50740c8e10 msgr2=0x7f50741f4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:11.286 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.284+0000 7f5071e00640 1 --2- 172.21.15.151:0/3055577872 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50740c8e10 0x7f50741f4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:11.286 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.284+0000 7f5071e00640 1 -- 172.21.15.151:0/3055577872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f507415bc30 msgr2=0x7f50740c88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:11.287 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.284+0000 7f5071e00640 1 --2- 172.21.15.151:0/3055577872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f507415bc30 0x7f50740c88d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:11.287 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.284+0000 7f5071e00640 1 -- 172.21.15.151:0/3055577872 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50740a9750 con 0x7f50740d0540 2024-09-16T15:42:11.287 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.284+0000 7f5071e00640 1 --2- 172.21.15.151:0/3055577872 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f50740d0540 0x7f50740aadc0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f506400c430 tx=0x7f506400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:11.287 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.284+0000 7f506a000640 1 -- 172.21.15.151:0/3055577872 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5064011790 con 0x7f50740d0540 2024-09-16T15:42:11.287 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.284+0000 7f506a000640 1 -- 172.21.15.151:0/3055577872 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5064017070 con 0x7f50740d0540 2024-09-16T15:42:11.287 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.284+0000 7f506a000640 1 -- 172.21.15.151:0/3055577872 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5064011ce0 con 0x7f50740d0540 2024-09-16T15:42:11.287 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.284+0000 7f5073200640 1 -- 172.21.15.151:0/3055577872 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50740a9a30 con 0x7f50740d0540 2024-09-16T15:42:11.287 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.285+0000 7f5073200640 1 -- 172.21.15.151:0/3055577872 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50740a9ff0 con 0x7f50740d0540 2024-09-16T15:42:11.287 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.285+0000 7f5073200640 1 -- 172.21.15.151:0/3055577872 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f507406fdf0 con 0x7f50740d0540 2024-09-16T15:42:11.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.287+0000 7f506a000640 1 -- 172.21.15.151:0/3055577872 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7f506401c460 con 0x7f50740d0540 2024-09-16T15:42:11.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.288+0000 7f506a000640 1 --2- 172.21.15.151:0/3055577872 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5054078bb0 0x7f505407b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:11.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.288+0000 7f506a000640 1 -- 172.21.15.151:0/3055577872 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5825+0+0 (secure 0 0 0) 0x7f506409ad80 con 0x7f50740d0540 2024-09-16T15:42:11.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.288+0000 7f5071400640 1 --2- 172.21.15.151:0/3055577872 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5054078bb0 0x7f505407b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:11.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.289+0000 7f5071400640 1 --2- 172.21.15.151:0/3055577872 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5054078bb0 0x7f505407b070 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5060034000 tx=0x7f5060004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:11.293 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.292+0000 7f506a000640 1 -- 172.21.15.151:0/3055577872 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f50640627c0 con 0x7f50740d0540 2024-09-16T15:42:11.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.515+0000 7f5073200640 1 -- 172.21.15.151:0/3055577872 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f50740cd000 con 0x7f50740d0540 2024-09-16T15:42:11.517 INFO:tasks.ceph.mon.a.smithi151.stderr:2024-09-16T15:42:11.516+0000 7f62c3200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T15:42:11.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.518+0000 7f506a000640 1 -- 172.21.15.151:0/3055577872 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v19) ==== 161+0+0 (secure 0 0 0) 0x7f50640671f0 con 0x7f50740d0540 2024-09-16T15:42:11.519 INFO:teuthology.orchestra.run.smithi151.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-16T15:42:11.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.521+0000 7f5073200640 1 -- 172.21.15.151:0/3055577872 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5054078bb0 msgr2=0x7f505407b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:11.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.521+0000 7f5073200640 1 --2- 172.21.15.151:0/3055577872 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5054078bb0 0x7f505407b070 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5060034000 tx=0x7f5060004730 comp rx=0 tx=0).stop 2024-09-16T15:42:11.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.521+0000 7f5073200640 1 -- 172.21.15.151:0/3055577872 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f50740d0540 msgr2=0x7f50740aadc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:11.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.521+0000 7f5073200640 1 --2- 172.21.15.151:0/3055577872 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f50740d0540 0x7f50740aadc0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f506400c430 tx=0x7f506400c8f0 comp rx=0 tx=0).stop 2024-09-16T15:42:11.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.522+0000 7f5073200640 1 -- 172.21.15.151:0/3055577872 shutdown_connections 2024-09-16T15:42:11.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.522+0000 7f5073200640 1 --2- 172.21.15.151:0/3055577872 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5054078bb0 0x7f505407b070 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:11.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.522+0000 7f5073200640 1 --2- 172.21.15.151:0/3055577872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f507415bc30 0x7f50740c88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:11.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.522+0000 7f5073200640 1 --2- 172.21.15.151:0/3055577872 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50740c8e10 0x7f50741f4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:11.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.522+0000 7f5073200640 1 --2- 172.21.15.151:0/3055577872 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f50740d0540 0x7f50740aadc0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:11.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.522+0000 7f5073200640 1 -- 172.21.15.151:0/3055577872 >> 172.21.15.151:0/3055577872 conn(0x7f50740cfa20 msgr2=0x7f50740ca7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:11.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.522+0000 7f5073200640 1 -- 172.21.15.151:0/3055577872 shutdown_connections 2024-09-16T15:42:11.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.522+0000 7f5073200640 1 -- 172.21.15.151:0/3055577872 wait complete. 2024-09-16T15:42:11.540 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-16T15:42:11.593 INFO:tasks.ceph.mds.b.smithi167.stdout:starting mds.b at 2024-09-16T15:42:11.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.700+0000 7f481be00640 1 -- 172.21.15.151:0/3618501986 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4814161760 msgr2=0x7f48140ce4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:11.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.700+0000 7f481be00640 1 --2- 172.21.15.151:0/3618501986 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4814161760 0x7f48140ce4a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4808005fd0 tx=0x7f480802aaa0 comp rx=0 tx=0).stop 2024-09-16T15:42:11.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.700+0000 7f481be00640 1 -- 172.21.15.151:0/3618501986 shutdown_connections 2024-09-16T15:42:11.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.700+0000 7f481be00640 1 --2- 172.21.15.151:0/3618501986 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48140cec60 0x7f48140cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:11.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.700+0000 7f481be00640 1 --2- 172.21.15.151:0/3618501986 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4814161760 0x7f48140ce4a0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:11.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.700+0000 7f481be00640 1 -- 172.21.15.151:0/3618501986 >> 172.21.15.151:0/3618501986 conn(0x7f48140b1840 msgr2=0x7f48140b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:11.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.700+0000 7f481be00640 1 -- 172.21.15.151:0/3618501986 shutdown_connections 2024-09-16T15:42:11.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.700+0000 7f481be00640 1 -- 172.21.15.151:0/3618501986 wait complete. 2024-09-16T15:42:11.703 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.702+0000 7f481be00640 1 Processor -- start 2024-09-16T15:42:11.703 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.702+0000 7f481be00640 1 -- start start 2024-09-16T15:42:11.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.703+0000 7f481be00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48140af190 0x7f481415ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:11.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.703+0000 7f481be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48140cec60 0x7f4814158880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:11.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.703+0000 7f481be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4814161760 0x7f4814073910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:11.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.703+0000 7f481be00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4814162310 con 0x7f4814161760 2024-09-16T15:42:11.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.703+0000 7f481be00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4814162190 con 0x7f48140cec60 2024-09-16T15:42:11.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.703+0000 7f481be00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f4814162490 con 0x7f48140af190 2024-09-16T15:42:11.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.703+0000 7f4819e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4814161760 0x7f4814073910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:11.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.703+0000 7f4819400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48140af190 0x7f481415ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:11.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.703+0000 7f4819e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4814161760 0x7f4814073910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41890/0 (socket says 172.21.15.151:41890) 2024-09-16T15:42:11.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.703+0000 7f4819400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48140af190 0x7f481415ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.151:43272/0 (socket says 172.21.15.151:43272) 2024-09-16T15:42:11.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.703+0000 7f4819400640 1 -- 172.21.15.151:0/1467244735 learned_addr learned my addr 172.21.15.151:0/1467244735 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:42:11.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.703+0000 7f4818a00640 1 --2- 172.21.15.151:0/1467244735 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48140cec60 0x7f4814158880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:11.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.704+0000 7f4819400640 1 -- 172.21.15.151:0/1467244735 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48140cec60 msgr2=0x7f4814158880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:11.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.704+0000 7f4819400640 1 --2- 172.21.15.151:0/1467244735 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48140cec60 0x7f4814158880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:11.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.704+0000 7f4819400640 1 -- 172.21.15.151:0/1467244735 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4814161760 msgr2=0x7f4814073910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:11.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.704+0000 7f4819400640 1 --2- 172.21.15.151:0/1467244735 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4814161760 0x7f4814073910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:11.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.704+0000 7f4819400640 1 -- 172.21.15.151:0/1467244735 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48141472f0 con 0x7f48140af190 2024-09-16T15:42:11.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.704+0000 7f4819e00640 1 --2- 172.21.15.151:0/1467244735 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4814161760 0x7f4814073910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:42:11.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.704+0000 7f4819400640 1 --2- 172.21.15.151:0/1467244735 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48140af190 0x7f481415ce90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4808000f80 tx=0x7f4808003f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:11.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.704+0000 7f4811600640 1 -- 172.21.15.151:0/1467244735 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4808044070 con 0x7f48140af190 2024-09-16T15:42:11.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.704+0000 7f4811600640 1 -- 172.21.15.151:0/1467244735 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4808004250 con 0x7f48140af190 2024-09-16T15:42:11.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.704+0000 7f4811600640 1 -- 172.21.15.151:0/1467244735 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4808004660 con 0x7f48140af190 2024-09-16T15:42:11.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.704+0000 7f481be00640 1 -- 172.21.15.151:0/1467244735 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48141475d0 con 0x7f48140af190 2024-09-16T15:42:11.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.705+0000 7f481be00640 1 -- 172.21.15.151:0/1467244735 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4814147b90 con 0x7f48140af190 2024-09-16T15:42:11.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.706+0000 7f481be00640 1 -- 172.21.15.151:0/1467244735 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48141629d0 con 0x7f48140af190 2024-09-16T15:42:11.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.707+0000 7f4811600640 1 -- 172.21.15.151:0/1467244735 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7f480802c040 con 0x7f48140af190 2024-09-16T15:42:11.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.708+0000 7f4811600640 1 --2- 172.21.15.151:0/1467244735 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f47f4078bb0 0x7f47f407b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:11.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.708+0000 7f4811600640 1 -- 172.21.15.151:0/1467244735 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(38..38 src has 1..38) ==== 5911+0+0 (secure 0 0 0) 0x7f48080bbba0 con 0x7f48140af190 2024-09-16T15:42:11.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.708+0000 7f4818a00640 1 --2- 172.21.15.151:0/1467244735 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f47f4078bb0 0x7f47f407b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:11.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.709+0000 7f4818a00640 1 --2- 172.21.15.151:0/1467244735 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f47f4078bb0 0x7f47f407b070 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f48000017c0 tx=0x7f4800009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:11.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.713+0000 7f4811600640 1 -- 172.21.15.151:0/1467244735 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4808032030 con 0x7f48140af190 2024-09-16T15:42:11.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:11.937+0000 7f481be00640 1 -- 172.21.15.151:0/1467244735 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f481415d3d0 con 0x7f48140af190 2024-09-16T15:42:12.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:12.529+0000 7f4811600640 1 -- 172.21.15.151:0/1467244735 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v20) ==== 104+0+0 (secure 0 0 0) 0x7f48080835a0 con 0x7f48140af190 2024-09-16T15:42:12.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:12.531+0000 7f481be00640 1 -- 172.21.15.151:0/1467244735 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f47f4078bb0 msgr2=0x7f47f407b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:12.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:12.531+0000 7f481be00640 1 --2- 172.21.15.151:0/1467244735 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f47f4078bb0 0x7f47f407b070 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f48000017c0 tx=0x7f4800009040 comp rx=0 tx=0).stop 2024-09-16T15:42:12.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:12.531+0000 7f481be00640 1 -- 172.21.15.151:0/1467244735 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48140af190 msgr2=0x7f481415ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:12.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:12.531+0000 7f481be00640 1 --2- 172.21.15.151:0/1467244735 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48140af190 0x7f481415ce90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4808000f80 tx=0x7f4808003f10 comp rx=0 tx=0).stop 2024-09-16T15:42:12.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:12.532+0000 7f481be00640 1 -- 172.21.15.151:0/1467244735 shutdown_connections 2024-09-16T15:42:12.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:12.532+0000 7f481be00640 1 --2- 172.21.15.151:0/1467244735 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f47f4078bb0 0x7f47f407b070 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:12.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:12.532+0000 7f481be00640 1 --2- 172.21.15.151:0/1467244735 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48140cec60 0x7f4814158880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:12.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:12.532+0000 7f481be00640 1 --2- 172.21.15.151:0/1467244735 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4814161760 0x7f4814073910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:12.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:12.532+0000 7f481be00640 1 --2- 172.21.15.151:0/1467244735 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48140af190 0x7f481415ce90 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:12.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:12.532+0000 7f481be00640 1 -- 172.21.15.151:0/1467244735 >> 172.21.15.151:0/1467244735 conn(0x7f48140b1840 msgr2=0x7f48140add50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:12.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:12.532+0000 7f481be00640 1 -- 172.21.15.151:0/1467244735 shutdown_connections 2024-09-16T15:42:12.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:12.532+0000 7f481be00640 1 -- 172.21.15.151:0/1467244735 wait complete. 2024-09-16T15:42:12.550 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-16T15:42:12.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:12.711+0000 7fd352a00640 1 -- 172.21.15.151:0/247534951 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd34c074460 msgr2=0x7fd34c0cc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:12.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:12.711+0000 7fd352a00640 1 --2- 172.21.15.151:0/247534951 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd34c074460 0x7fd34c0cc5b0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd34c0af7f0 tx=0x7fd34402aac0 comp rx=0 tx=0).stop 2024-09-16T15:42:12.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:12.711+0000 7fd352a00640 1 -- 172.21.15.151:0/247534951 shutdown_connections 2024-09-16T15:42:12.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:12.711+0000 7fd352a00640 1 --2- 172.21.15.151:0/247534951 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd34c074460 0x7fd34c0cc5b0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:12.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:12.711+0000 7fd352a00640 1 --2- 172.21.15.151:0/247534951 >> v2:172.21.15.167:3300/0 conn(0x7fd34c06e910 0x7fd34c0cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:12.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:12.711+0000 7fd352a00640 1 -- 172.21.15.151:0/247534951 >> 172.21.15.151:0/247534951 conn(0x7fd34c0b18e0 msgr2=0x7fd34c0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:12.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:12.711+0000 7fd352a00640 1 -- 172.21.15.151:0/247534951 shutdown_connections 2024-09-16T15:42:12.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:12.712+0000 7fd352a00640 1 -- 172.21.15.151:0/247534951 wait complete. 2024-09-16T15:42:12.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:12.713+0000 7fd352a00640 1 Processor -- start 2024-09-16T15:42:12.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:12.713+0000 7fd352a00640 1 -- start start 2024-09-16T15:42:12.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:12.714+0000 7fd352a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd34c06e910 0x7fd34c140660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:12.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:12.714+0000 7fd352a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd34c074460 0x7fd34c1f63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:12.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:12.714+0000 7fd352a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd34c15bd60 0x7fd34c1f6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:12.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:12.714+0000 7fd352a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c0ccc70 con 0x7fd34c074460 2024-09-16T15:42:12.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:12.714+0000 7fd352a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c0ccaf0 con 0x7fd34c15bd60 2024-09-16T15:42:12.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:12.714+0000 7fd352a00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fd34c0ccdf0 con 0x7fd34c06e910 2024-09-16T15:42:12.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:12.714+0000 7fd34b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd34c074460 0x7fd34c1f63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:12.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:12.715+0000 7fd34b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd34c074460 0x7fd34c1f63c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41916/0 (socket says 172.21.15.151:41916) 2024-09-16T15:42:12.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:12.715+0000 7fd34b400640 1 -- 172.21.15.151:0/3541804924 learned_addr learned my addr 172.21.15.151:0/3541804924 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:42:12.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:12.715+0000 7fd350a00640 1 --2- 172.21.15.151:0/3541804924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd34c15bd60 0x7fd34c1f6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:12.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:12.715+0000 7fd34be00640 1 --2- 172.21.15.151:0/3541804924 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd34c06e910 0x7fd34c140660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:12.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:12.715+0000 7fd34b400640 1 -- 172.21.15.151:0/3541804924 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd34c06e910 msgr2=0x7fd34c140660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:12.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:12.715+0000 7fd34b400640 1 --2- 172.21.15.151:0/3541804924 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd34c06e910 0x7fd34c140660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:12.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:12.715+0000 7fd34b400640 1 -- 172.21.15.151:0/3541804924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd34c15bd60 msgr2=0x7fd34c1f6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:12.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:12.715+0000 7fd34b400640 1 --2- 172.21.15.151:0/3541804924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd34c15bd60 0x7fd34c1f6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:12.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:12.715+0000 7fd34b400640 1 -- 172.21.15.151:0/3541804924 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd34c154af0 con 0x7fd34c074460 2024-09-16T15:42:12.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:12.715+0000 7fd34be00640 1 --2- 172.21.15.151:0/3541804924 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd34c06e910 0x7fd34c140660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:42:12.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:12.715+0000 7fd34b400640 1 --2- 172.21.15.151:0/3541804924 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd34c074460 0x7fd34c1f63c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd33800ea20 tx=0x7fd33800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:12.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:12.715+0000 7fd343e00640 1 -- 172.21.15.151:0/3541804924 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd33800cd80 con 0x7fd34c074460 2024-09-16T15:42:12.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:12.716+0000 7fd343e00640 1 -- 172.21.15.151:0/3541804924 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd338005020 con 0x7fd34c074460 2024-09-16T15:42:12.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:12.716+0000 7fd352a00640 1 -- 172.21.15.151:0/3541804924 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd34c154dd0 con 0x7fd34c074460 2024-09-16T15:42:12.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:12.716+0000 7fd343e00640 1 -- 172.21.15.151:0/3541804924 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd338005450 con 0x7fd34c074460 2024-09-16T15:42:12.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:12.716+0000 7fd352a00640 1 -- 172.21.15.151:0/3541804924 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd34c1e5770 con 0x7fd34c074460 2024-09-16T15:42:12.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:12.717+0000 7fd352a00640 1 -- 172.21.15.151:0/3541804924 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd34c1512b0 con 0x7fd34c074460 2024-09-16T15:42:12.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:12.718+0000 7fd343e00640 1 -- 172.21.15.151:0/3541804924 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7fd3380047a0 con 0x7fd34c074460 2024-09-16T15:42:12.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:12.719+0000 7fd343e00640 1 --2- 172.21.15.151:0/3541804924 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd32c078bb0 0x7fd32c07b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:12.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:12.719+0000 7fd343e00640 1 -- 172.21.15.151:0/3541804924 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5911+0+0 (secure 0 0 0) 0x7fd33809bbc0 con 0x7fd34c074460 2024-09-16T15:42:12.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:12.719+0000 7fd34be00640 1 --2- 172.21.15.151:0/3541804924 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd32c078bb0 0x7fd32c07b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:12.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:12.720+0000 7fd34be00640 1 --2- 172.21.15.151:0/3541804924 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd32c078bb0 0x7fd32c07b070 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd33c007fd0 tx=0x7fd33c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:12.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:12.723+0000 7fd343e00640 1 -- 172.21.15.151:0/3541804924 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd338063b90 con 0x7fd34c074460 2024-09-16T15:42:12.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:12.908+0000 7fd352a00640 1 -- 172.21.15.151:0/3541804924 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd34c069360 con 0x7fd34c074460 2024-09-16T15:42:13.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:13.527+0000 7fd343e00640 1 -- 172.21.15.151:0/3541804924 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v39) ==== 181+0+0 (secure 0 0 0) 0x7fd338063490 con 0x7fd34c074460 2024-09-16T15:42:13.529 INFO:teuthology.orchestra.run.smithi151.stderr:pool 'cephfs_metadata' removed 2024-09-16T15:42:13.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:13.531+0000 7fd352a00640 1 -- 172.21.15.151:0/3541804924 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd32c078bb0 msgr2=0x7fd32c07b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:13.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:13.531+0000 7fd352a00640 1 --2- 172.21.15.151:0/3541804924 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd32c078bb0 0x7fd32c07b070 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd33c007fd0 tx=0x7fd33c009040 comp rx=0 tx=0).stop 2024-09-16T15:42:13.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:13.531+0000 7fd352a00640 1 -- 172.21.15.151:0/3541804924 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd34c074460 msgr2=0x7fd34c1f63c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:13.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:13.531+0000 7fd352a00640 1 --2- 172.21.15.151:0/3541804924 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd34c074460 0x7fd34c1f63c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd33800ea20 tx=0x7fd33800ee20 comp rx=0 tx=0).stop 2024-09-16T15:42:13.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:13.531+0000 7fd352a00640 1 -- 172.21.15.151:0/3541804924 shutdown_connections 2024-09-16T15:42:13.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:13.531+0000 7fd352a00640 1 --2- 172.21.15.151:0/3541804924 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd32c078bb0 0x7fd32c07b070 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:13.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:13.531+0000 7fd352a00640 1 --2- 172.21.15.151:0/3541804924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd34c15bd60 0x7fd34c1f6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:13.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:13.531+0000 7fd352a00640 1 --2- 172.21.15.151:0/3541804924 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd34c074460 0x7fd34c1f63c0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:13.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:13.531+0000 7fd352a00640 1 --2- 172.21.15.151:0/3541804924 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd34c06e910 0x7fd34c140660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:13.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:13.531+0000 7fd352a00640 1 -- 172.21.15.151:0/3541804924 >> 172.21.15.151:0/3541804924 conn(0x7fd34c0b18e0 msgr2=0x7fd34c0ce1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:13.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:13.531+0000 7fd352a00640 1 -- 172.21.15.151:0/3541804924 shutdown_connections 2024-09-16T15:42:13.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:13.532+0000 7fd352a00640 1 -- 172.21.15.151:0/3541804924 wait complete. 2024-09-16T15:42:13.551 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-16T15:42:13.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:13.720+0000 7fb5e7200640 1 -- 172.21.15.151:0/3184107318 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5cc003c60 msgr2=0x7fb5cc004060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:13.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:13.720+0000 7fb5e7200640 1 --2- 172.21.15.151:0/3184107318 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5cc003c60 0x7fb5cc004060 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8009b50 tx=0x7fb5cc005830 comp rx=0 tx=0).stop 2024-09-16T15:42:13.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:13.720+0000 7fb5e6800640 1 -- 172.21.15.151:0/3184107318 reap_dead start 2024-09-16T15:42:13.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:13.720+0000 7fb5e7200640 1 -- 172.21.15.151:0/3184107318 shutdown_connections 2024-09-16T15:42:13.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:13.720+0000 7fb5e7200640 1 -- 172.21.15.151:0/3184107318 >> 172.21.15.151:0/3184107318 conn(0x7fb5e80cc4f0 msgr2=0x7fb5e80cc900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:13.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:13.720+0000 7fb5e7200640 1 -- 172.21.15.151:0/3184107318 shutdown_connections 2024-09-16T15:42:13.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:13.720+0000 7fb5e7200640 1 -- 172.21.15.151:0/3184107318 wait complete. 2024-09-16T15:42:13.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:13.722+0000 7fb5e7200640 1 Processor -- start 2024-09-16T15:42:13.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:13.722+0000 7fb5e7200640 1 -- start start 2024-09-16T15:42:13.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:13.723+0000 7fb5e7200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5cc003c60 0x7fb5e80c5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:13.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:13.723+0000 7fb5e7200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5e81f4740 0x7fb5e80cbb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:13.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:13.723+0000 7fb5e7200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5e80c6b40 0x7fb5e80c4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:13.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:13.723+0000 7fb5e7200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e806fd60 con 0x7fb5cc003c60 2024-09-16T15:42:13.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:13.723+0000 7fb5e7200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e806fbe0 con 0x7fb5e81f4740 2024-09-16T15:42:13.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:13.723+0000 7fb5e7200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fb5e806fee0 con 0x7fb5e80c6b40 2024-09-16T15:42:13.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:13.723+0000 7fb5e5e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5cc003c60 0x7fb5e80c5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:13.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:13.723+0000 7fb5e5e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5cc003c60 0x7fb5e80c5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:56236/0 (socket says 172.21.15.151:56236) 2024-09-16T15:42:13.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:13.723+0000 7fb5e5e00640 1 -- 172.21.15.151:0/3407784445 learned_addr learned my addr 172.21.15.151:0/3407784445 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:42:13.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:13.723+0000 7fb5e5400640 1 --2- 172.21.15.151:0/3407784445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5e81f4740 0x7fb5e80cbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:13.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:13.723+0000 7fb5e6800640 1 --2- 172.21.15.151:0/3407784445 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5e80c6b40 0x7fb5e80c4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:13.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:13.724+0000 7fb5e5e00640 1 -- 172.21.15.151:0/3407784445 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5e80c6b40 msgr2=0x7fb5e80c4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:13.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:13.724+0000 7fb5e5e00640 1 --2- 172.21.15.151:0/3407784445 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5e80c6b40 0x7fb5e80c4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:13.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:13.724+0000 7fb5e5e00640 1 -- 172.21.15.151:0/3407784445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5e81f4740 msgr2=0x7fb5e80cbb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:13.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:13.724+0000 7fb5e5e00640 1 --2- 172.21.15.151:0/3407784445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5e81f4740 0x7fb5e80cbb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:13.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:13.724+0000 7fb5e5e00640 1 -- 172.21.15.151:0/3407784445 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5e81f3ca0 con 0x7fb5cc003c60 2024-09-16T15:42:13.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:13.724+0000 7fb5e5e00640 1 --2- 172.21.15.151:0/3407784445 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5cc003c60 0x7fb5e80c5a30 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8009c70 tx=0x7fb5d8037b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:13.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:13.724+0000 7fb5de000640 1 -- 172.21.15.151:0/3407784445 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d803c040 con 0x7fb5cc003c60 2024-09-16T15:42:13.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:13.724+0000 7fb5de000640 1 -- 172.21.15.151:0/3407784445 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb5d8037c80 con 0x7fb5cc003c60 2024-09-16T15:42:13.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:13.724+0000 7fb5e7200640 1 -- 172.21.15.151:0/3407784445 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5e80c62f0 con 0x7fb5cc003c60 2024-09-16T15:42:13.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:13.724+0000 7fb5de000640 1 -- 172.21.15.151:0/3407784445 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d8004280 con 0x7fb5cc003c60 2024-09-16T15:42:13.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:13.724+0000 7fb5e7200640 1 -- 172.21.15.151:0/3407784445 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5e81f4220 con 0x7fb5cc003c60 2024-09-16T15:42:13.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:13.725+0000 7fb5e7200640 1 -- 172.21.15.151:0/3407784445 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5e80cb680 con 0x7fb5cc003c60 2024-09-16T15:42:13.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:13.726+0000 7fb5de000640 1 -- 172.21.15.151:0/3407784445 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7fb5d8049020 con 0x7fb5cc003c60 2024-09-16T15:42:13.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:13.728+0000 7fb5de000640 1 --2- 172.21.15.151:0/3407784445 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb5c8078bb0 0x7fb5c807b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:13.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:13.728+0000 7fb5de000640 1 -- 172.21.15.151:0/3407784445 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5473+0+0 (secure 0 0 0) 0x7fb5d80bfa90 con 0x7fb5cc003c60 2024-09-16T15:42:13.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:13.728+0000 7fb5e5400640 1 --2- 172.21.15.151:0/3407784445 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb5c8078bb0 0x7fb5c807b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:13.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:13.729+0000 7fb5e5400640 1 --2- 172.21.15.151:0/3407784445 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb5c8078bb0 0x7fb5c807b070 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4000fb0 tx=0x7fb5d4001bb0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:13.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:13.732+0000 7fb5de000640 1 -- 172.21.15.151:0/3407784445 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb5d8087630 con 0x7fb5cc003c60 2024-09-16T15:42:13.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:13.918+0000 7fb5e7200640 1 -- 172.21.15.151:0/3407784445 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb5e80cafb0 con 0x7fb5cc003c60 2024-09-16T15:42:14.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:14.922+0000 7fb5de000640 1 -- 172.21.15.151:0/3407784445 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v40) ==== 169+0+0 (secure 0 0 0) 0x7fb5d808c060 con 0x7fb5cc003c60 2024-09-16T15:42:14.923 INFO:teuthology.orchestra.run.smithi151.stderr:pool 'cephfs_data' removed 2024-09-16T15:42:14.926 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:14.925+0000 7fb5e7200640 1 -- 172.21.15.151:0/3407784445 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb5c8078bb0 msgr2=0x7fb5c807b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:14.926 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:14.925+0000 7fb5e7200640 1 --2- 172.21.15.151:0/3407784445 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb5c8078bb0 0x7fb5c807b070 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4000fb0 tx=0x7fb5d4001bb0 comp rx=0 tx=0).stop 2024-09-16T15:42:14.926 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:14.925+0000 7fb5e7200640 1 -- 172.21.15.151:0/3407784445 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5cc003c60 msgr2=0x7fb5e80c5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:14.926 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:14.925+0000 7fb5e7200640 1 --2- 172.21.15.151:0/3407784445 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5cc003c60 0x7fb5e80c5a30 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8009c70 tx=0x7fb5d8037b10 comp rx=0 tx=0).stop 2024-09-16T15:42:14.926 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:14.925+0000 7fb5e7200640 1 -- 172.21.15.151:0/3407784445 shutdown_connections 2024-09-16T15:42:14.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:14.925+0000 7fb5e7200640 1 --2- 172.21.15.151:0/3407784445 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb5c8078bb0 0x7fb5c807b070 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:14.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:14.925+0000 7fb5e7200640 1 --2- 172.21.15.151:0/3407784445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5e81f4740 0x7fb5e80cbb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:14.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:14.925+0000 7fb5e7200640 1 --2- 172.21.15.151:0/3407784445 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5e80c6b40 0x7fb5e80c4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:14.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:14.925+0000 7fb5e7200640 1 --2- 172.21.15.151:0/3407784445 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5cc003c60 0x7fb5e80c5a30 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:14.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:14.925+0000 7fb5e7200640 1 -- 172.21.15.151:0/3407784445 >> 172.21.15.151:0/3407784445 conn(0x7fb5e80cc4f0 msgr2=0x7fb5e80b0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:14.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:14.925+0000 7fb5e7200640 1 -- 172.21.15.151:0/3407784445 shutdown_connections 2024-09-16T15:42:14.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:14.925+0000 7fb5e7200640 1 -- 172.21.15.151:0/3407784445 wait complete. 2024-09-16T15:42:14.953 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-16T15:42:15.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.151+0000 7fb4c2400640 1 -- 172.21.15.151:0/2177597183 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4bc0d0540 msgr2=0x7fb4bc06f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:15.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.151+0000 7fb4c2400640 1 --2- 172.21.15.151:0/2177597183 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4bc0d0540 0x7fb4bc06f320 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc0cd570 tx=0x7fb4b402aac0 comp rx=0 tx=0).stop 2024-09-16T15:42:15.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.152+0000 7fb4c2400640 1 -- 172.21.15.151:0/2177597183 shutdown_connections 2024-09-16T15:42:15.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.152+0000 7fb4c2400640 1 --2- 172.21.15.151:0/2177597183 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4bc0d0540 0x7fb4bc06f320 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:15.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.152+0000 7fb4c2400640 1 --2- 172.21.15.151:0/2177597183 >> v2:172.21.15.167:3300/0 conn(0x7fb4bc15bc30 0x7fb4bc0b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:15.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.152+0000 7fb4c2400640 1 -- 172.21.15.151:0/2177597183 >> 172.21.15.151:0/2177597183 conn(0x7fb4bc0cfa20 msgr2=0x7fb4bc0cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:15.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.152+0000 7fb4c2400640 1 -- 172.21.15.151:0/2177597183 shutdown_connections 2024-09-16T15:42:15.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.152+0000 7fb4c2400640 1 -- 172.21.15.151:0/2177597183 wait complete. 2024-09-16T15:42:15.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.153+0000 7fb4c2400640 1 Processor -- start 2024-09-16T15:42:15.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.153+0000 7fb4c2400640 1 -- start start 2024-09-16T15:42:15.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.154+0000 7fb4c2400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4bc0d0540 0x7fb4bc0cec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:15.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.154+0000 7fb4c2400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4bc15bc30 0x7fb4bc1f7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:15.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.154+0000 7fb4c2400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4bc0cd750 0x7fb4bc0a9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:15.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.154+0000 7fb4c2400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb4bc074660 con 0x7fb4bc15bc30 2024-09-16T15:42:15.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.154+0000 7fb4c2400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb4bc06f860 con 0x7fb4bc0d0540 2024-09-16T15:42:15.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.154+0000 7fb4c2400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fb4bc0cdbe0 con 0x7fb4bc0cd750 2024-09-16T15:42:15.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.155+0000 7fb4bbe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4bc15bc30 0x7fb4bc1f7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:15.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.155+0000 7fb4bbe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4bc15bc30 0x7fb4bc1f7940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:56246/0 (socket says 172.21.15.151:56246) 2024-09-16T15:42:15.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.155+0000 7fb4bbe00640 1 -- 172.21.15.151:0/3303001226 learned_addr learned my addr 172.21.15.151:0/3303001226 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:42:15.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.155+0000 7fb4c1000640 1 --2- 172.21.15.151:0/3303001226 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4bc0d0540 0x7fb4bc0cec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:15.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.155+0000 7fb4c1a00640 1 --2- 172.21.15.151:0/3303001226 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4bc0cd750 0x7fb4bc0a9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:15.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.155+0000 7fb4bbe00640 1 -- 172.21.15.151:0/3303001226 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4bc0cd750 msgr2=0x7fb4bc0a9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:15.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.155+0000 7fb4bbe00640 1 --2- 172.21.15.151:0/3303001226 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4bc0cd750 0x7fb4bc0a9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:15.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.155+0000 7fb4bbe00640 1 -- 172.21.15.151:0/3303001226 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4bc0d0540 msgr2=0x7fb4bc0cec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:15.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.155+0000 7fb4bbe00640 1 --2- 172.21.15.151:0/3303001226 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4bc0d0540 0x7fb4bc0cec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:15.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.155+0000 7fb4bbe00640 1 -- 172.21.15.151:0/3303001226 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4bc0cdec0 con 0x7fb4bc15bc30 2024-09-16T15:42:15.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.155+0000 7fb4bbe00640 1 --2- 172.21.15.151:0/3303001226 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4bc15bc30 0x7fb4bc1f7940 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb4a800c450 tx=0x7fb4a800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:15.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.156+0000 7fb4b8c00640 1 -- 172.21.15.151:0/3303001226 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a80052d0 con 0x7fb4bc15bc30 2024-09-16T15:42:15.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.156+0000 7fb4b8c00640 1 -- 172.21.15.151:0/3303001226 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb4a8005470 con 0x7fb4bc15bc30 2024-09-16T15:42:15.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.156+0000 7fb4b8c00640 1 -- 172.21.15.151:0/3303001226 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a8012870 con 0x7fb4bc15bc30 2024-09-16T15:42:15.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.156+0000 7fb4c2400640 1 -- 172.21.15.151:0/3303001226 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4bc0ce1a0 con 0x7fb4bc15bc30 2024-09-16T15:42:15.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.156+0000 7fb4c2400640 1 -- 172.21.15.151:0/3303001226 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4bc0ce760 con 0x7fb4bc15bc30 2024-09-16T15:42:15.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.157+0000 7fb4c2400640 1 -- 172.21.15.151:0/3303001226 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4bc06fdf0 con 0x7fb4bc15bc30 2024-09-16T15:42:15.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.158+0000 7fb4b8c00640 1 -- 172.21.15.151:0/3303001226 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7fb4a80035f0 con 0x7fb4bc15bc30 2024-09-16T15:42:15.160 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.159+0000 7fb4b8c00640 1 --2- 172.21.15.151:0/3303001226 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb49c078bb0 0x7fb49c07b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:15.160 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.159+0000 7fb4b8c00640 1 -- 172.21.15.151:0/3303001226 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4943+0+0 (secure 0 0 0) 0x7fb4a809a6e0 con 0x7fb4bc15bc30 2024-09-16T15:42:15.160 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.159+0000 7fb4c1000640 1 --2- 172.21.15.151:0/3303001226 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb49c078bb0 0x7fb49c07b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:15.160 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.160+0000 7fb4c1000640 1 --2- 172.21.15.151:0/3303001226 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb49c078bb0 0x7fb49c07b070 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb4b000bb50 tx=0x7fb4b000b2e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:15.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.163+0000 7fb4b8c00640 1 -- 172.21.15.151:0/3303001226 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb4a8062430 con 0x7fb4bc15bc30 2024-09-16T15:42:15.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.348+0000 7fb4c2400640 1 -- 172.21.15.151:0/3303001226 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7fb4bc0b29d0 con 0x7fb4bc15bc30 2024-09-16T15:42:15.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.352+0000 7fb4b8c00640 1 -- 172.21.15.151:0/3303001226 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v30) ==== 71+0+0 (secure 0 0 0) 0x7fb4a8066e60 con 0x7fb4bc15bc30 2024-09-16T15:42:15.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.355+0000 7fb4c2400640 1 -- 172.21.15.151:0/3303001226 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb49c078bb0 msgr2=0x7fb49c07b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:15.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.355+0000 7fb4c2400640 1 --2- 172.21.15.151:0/3303001226 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb49c078bb0 0x7fb49c07b070 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb4b000bb50 tx=0x7fb4b000b2e0 comp rx=0 tx=0).stop 2024-09-16T15:42:15.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.355+0000 7fb4c2400640 1 -- 172.21.15.151:0/3303001226 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4bc15bc30 msgr2=0x7fb4bc1f7940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:15.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.355+0000 7fb4c2400640 1 --2- 172.21.15.151:0/3303001226 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4bc15bc30 0x7fb4bc1f7940 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb4a800c450 tx=0x7fb4a800c910 comp rx=0 tx=0).stop 2024-09-16T15:42:15.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.355+0000 7fb4c2400640 1 -- 172.21.15.151:0/3303001226 shutdown_connections 2024-09-16T15:42:15.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.355+0000 7fb4c2400640 1 --2- 172.21.15.151:0/3303001226 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb49c078bb0 0x7fb49c07b070 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:15.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.355+0000 7fb4c2400640 1 --2- 172.21.15.151:0/3303001226 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4bc0cd750 0x7fb4bc0a9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:15.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.355+0000 7fb4c2400640 1 --2- 172.21.15.151:0/3303001226 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4bc15bc30 0x7fb4bc1f7940 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:15.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.355+0000 7fb4c2400640 1 --2- 172.21.15.151:0/3303001226 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4bc0d0540 0x7fb4bc0cec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:15.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.355+0000 7fb4c2400640 1 -- 172.21.15.151:0/3303001226 >> 172.21.15.151:0/3303001226 conn(0x7fb4bc0cfa20 msgr2=0x7fb4bc0b3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:15.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.355+0000 7fb4c2400640 1 -- 172.21.15.151:0/3303001226 shutdown_connections 2024-09-16T15:42:15.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.355+0000 7fb4c2400640 1 -- 172.21.15.151:0/3303001226 wait complete. 2024-09-16T15:42:15.373 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clone_to_diff_group_and_less_than_cloner_threads' 2024-09-16T15:42:15.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.619+0000 7f0acec00640 1 -- 172.21.15.151:0/81559552 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ac80d1fd0 msgr2=0x7f0ac80b4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:15.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.619+0000 7f0acec00640 1 --2- 172.21.15.151:0/81559552 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ac80d1fd0 0x7f0ac80b4860 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0abc005fd0 tx=0x7f0abc02aa00 comp rx=0 tx=0).stop 2024-09-16T15:42:15.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.620+0000 7f0acec00640 1 -- 172.21.15.151:0/81559552 shutdown_connections 2024-09-16T15:42:15.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.620+0000 7f0acec00640 1 --2- 172.21.15.151:0/81559552 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ac80d1fd0 0x7f0ac80b4860 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:15.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.620+0000 7f0acec00640 1 -- 172.21.15.151:0/81559552 >> 172.21.15.151:0/81559552 conn(0x7f0ac80d3df0 msgr2=0x7f0ac80d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:15.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.621+0000 7f0acec00640 1 -- 172.21.15.151:0/81559552 shutdown_connections 2024-09-16T15:42:15.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.621+0000 7f0acec00640 1 -- 172.21.15.151:0/81559552 wait complete. 2024-09-16T15:42:15.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.624+0000 7f0acec00640 1 Processor -- start 2024-09-16T15:42:15.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.625+0000 7f0acec00640 1 -- start start 2024-09-16T15:42:15.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.628+0000 7f0acec00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ac80b0a80 0x7f0ac81f95e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:15.630 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.628+0000 7f0acec00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ac80d1fd0 0x7f0ac81f9b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:15.630 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.628+0000 7f0acec00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ac80d23b0 0x7f0ac80ce890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:15.630 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.628+0000 7f0acec00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac80b1720 con 0x7f0ac80d23b0 2024-09-16T15:42:15.630 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.629+0000 7f0acec00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac80b15a0 con 0x7f0ac80d1fd0 2024-09-16T15:42:15.630 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.629+0000 7f0acec00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f0ac815fc80 con 0x7f0ac80b0a80 2024-09-16T15:42:15.630 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.629+0000 7f0ace200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ac80d23b0 0x7f0ac80ce890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:15.630 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.629+0000 7f0ace200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ac80d23b0 0x7f0ac80ce890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:56268/0 (socket says 172.21.15.151:56268) 2024-09-16T15:42:15.630 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.629+0000 7f0ace200640 1 -- 172.21.15.151:0/3025179600 learned_addr learned my addr 172.21.15.151:0/3025179600 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:42:15.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.629+0000 7f0acd800640 1 --2- 172.21.15.151:0/3025179600 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ac80b0a80 0x7f0ac81f95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:15.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.630+0000 7f0acce00640 1 --2- 172.21.15.151:0/3025179600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ac80d1fd0 0x7f0ac81f9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:15.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.630+0000 7f0ace200640 1 -- 172.21.15.151:0/3025179600 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ac80b0a80 msgr2=0x7f0ac81f95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:15.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.630+0000 7f0ace200640 1 --2- 172.21.15.151:0/3025179600 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ac80b0a80 0x7f0ac81f95e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:15.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.630+0000 7f0ace200640 1 -- 172.21.15.151:0/3025179600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ac80d1fd0 msgr2=0x7f0ac81f9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:15.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.630+0000 7f0ace200640 1 --2- 172.21.15.151:0/3025179600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ac80d1fd0 0x7f0ac81f9b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:15.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.630+0000 7f0ace200640 1 -- 172.21.15.151:0/3025179600 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ac81fd300 con 0x7f0ac80d23b0 2024-09-16T15:42:15.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.631+0000 7f0ace200640 1 --2- 172.21.15.151:0/3025179600 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ac80d23b0 0x7f0ac80ce890 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f0ac000c390 tx=0x7f0ac000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:15.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.632+0000 7f0ac5600640 1 -- 172.21.15.151:0/3025179600 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac0017070 con 0x7f0ac80d23b0 2024-09-16T15:42:15.634 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.633+0000 7f0ac5600640 1 -- 172.21.15.151:0/3025179600 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0ac00035f0 con 0x7f0ac80d23b0 2024-09-16T15:42:15.634 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.633+0000 7f0acec00640 1 -- 172.21.15.151:0/3025179600 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ac815be60 con 0x7f0ac80d23b0 2024-09-16T15:42:15.634 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.633+0000 7f0acec00640 1 -- 172.21.15.151:0/3025179600 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ac81f90c0 con 0x7f0ac80d23b0 2024-09-16T15:42:15.634 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.633+0000 7f0ac5600640 1 -- 172.21.15.151:0/3025179600 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac0003a40 con 0x7f0ac80d23b0 2024-09-16T15:42:15.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.639+0000 7f0ac5600640 1 -- 172.21.15.151:0/3025179600 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7f0ac0003be0 con 0x7f0ac80d23b0 2024-09-16T15:42:15.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.639+0000 7f0acec00640 1 -- 172.21.15.151:0/3025179600 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0aa4005290 con 0x7f0ac80d23b0 2024-09-16T15:42:15.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.642+0000 7f0ac5600640 1 --2- 172.21.15.151:0/3025179600 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0aa8078bb0 0x7f0aa807b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:15.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.642+0000 7f0ac5600640 1 -- 172.21.15.151:0/3025179600 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4943+0+0 (secure 0 0 0) 0x7f0ac0099d50 con 0x7f0ac80d23b0 2024-09-16T15:42:15.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.646+0000 7f0acd800640 1 --2- 172.21.15.151:0/3025179600 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0aa8078bb0 0x7f0aa807b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:15.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.649+0000 7f0acd800640 1 --2- 172.21.15.151:0/3025179600 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0aa8078bb0 0x7f0aa807b070 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0abc035000 tx=0x7f0abc03a040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:15.654 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.653+0000 7f0ac5600640 1 -- 172.21.15.151:0/3025179600 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0ac0061aa0 con 0x7f0ac80d23b0 2024-09-16T15:42:15.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:15.965+0000 7f0acec00640 1 -- 172.21.15.151:0/3025179600 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clone_to_diff_group_and_less_than_cloner_threads"]} v 0) -- 0x7f0aa4005580 con 0x7f0ac80d23b0 2024-09-16T15:42:16.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:16.925+0000 7f0ac5600640 1 -- 172.21.15.151:0/3025179600 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clone_to_diff_group_and_less_than_cloner_threads"]}]=0 v148) ==== 184+0+0 (secure 0 0 0) 0x7f0ac00664d0 con 0x7f0ac80d23b0 2024-09-16T15:42:16.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:16.928+0000 7f0acec00640 1 -- 172.21.15.151:0/3025179600 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0aa8078bb0 msgr2=0x7f0aa807b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:16.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:16.928+0000 7f0acec00640 1 --2- 172.21.15.151:0/3025179600 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0aa8078bb0 0x7f0aa807b070 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0abc035000 tx=0x7f0abc03a040 comp rx=0 tx=0).stop 2024-09-16T15:42:16.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:16.928+0000 7f0acec00640 1 -- 172.21.15.151:0/3025179600 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ac80d23b0 msgr2=0x7f0ac80ce890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:16.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:16.928+0000 7f0acec00640 1 --2- 172.21.15.151:0/3025179600 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ac80d23b0 0x7f0ac80ce890 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f0ac000c390 tx=0x7f0ac000c850 comp rx=0 tx=0).stop 2024-09-16T15:42:16.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:16.928+0000 7f0acec00640 1 -- 172.21.15.151:0/3025179600 shutdown_connections 2024-09-16T15:42:16.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:16.928+0000 7f0acec00640 1 --2- 172.21.15.151:0/3025179600 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0aa8078bb0 0x7f0aa807b070 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:16.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:16.928+0000 7f0acec00640 1 --2- 172.21.15.151:0/3025179600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ac80d1fd0 0x7f0ac81f9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:16.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:16.928+0000 7f0acec00640 1 --2- 172.21.15.151:0/3025179600 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ac80d23b0 0x7f0ac80ce890 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:16.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:16.928+0000 7f0acec00640 1 --2- 172.21.15.151:0/3025179600 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ac80b0a80 0x7f0ac81f95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:16.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:16.928+0000 7f0acec00640 1 -- 172.21.15.151:0/3025179600 >> 172.21.15.151:0/3025179600 conn(0x7f0ac80d3df0 msgr2=0x7f0ac8158ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:16.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:16.929+0000 7f0acec00640 1 -- 172.21.15.151:0/3025179600 shutdown_connections 2024-09-16T15:42:16.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:16.929+0000 7f0acec00640 1 -- 172.21.15.151:0/3025179600 wait complete. 2024-09-16T15:42:16.949 INFO:tasks.cephfs_test_runner:Initiate cloning where clone subvolume and source subvolume are located ... ok 2024-09-16T15:42:16.949 INFO:tasks.cephfs_test_runner:Starting test: test_clones_equal_to_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that one progress bar is printed in output of "ceph status" output 2024-09-16T15:42:16.949 INFO:tasks.cephfs_test_runner:test_clones_equal_to_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-09-16T15:42:16.950 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:42:17.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.111+0000 7ffa71a00640 1 -- 172.21.15.151:0/1486604330 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa6c06fb90 msgr2=0x7ffa6c06ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:17.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.111+0000 7ffa71a00640 1 --2- 172.21.15.151:0/1486604330 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa6c06fb90 0x7ffa6c06ff70 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c005fd0 tx=0x7ffa5c02a8b0 comp rx=0 tx=0).stop 2024-09-16T15:42:17.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.111+0000 7ffa71a00640 1 -- 172.21.15.151:0/1486604330 shutdown_connections 2024-09-16T15:42:17.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.111+0000 7ffa71a00640 1 --2- 172.21.15.151:0/1486604330 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa6c06fb90 0x7ffa6c06ff70 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:17.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.111+0000 7ffa71a00640 1 -- 172.21.15.151:0/1486604330 >> 172.21.15.151:0/1486604330 conn(0x7ffa6c0cfa20 msgr2=0x7ffa6c0cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:17.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.111+0000 7ffa71a00640 1 -- 172.21.15.151:0/1486604330 shutdown_connections 2024-09-16T15:42:17.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.111+0000 7ffa71a00640 1 -- 172.21.15.151:0/1486604330 wait complete. 2024-09-16T15:42:17.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.113+0000 7ffa71a00640 1 Processor -- start 2024-09-16T15:42:17.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.113+0000 7ffa71a00640 1 -- start start 2024-09-16T15:42:17.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.114+0000 7ffa71a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffa6c06fb90 0x7ffa6c0b40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:17.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.114+0000 7ffa71a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa6c0ad880 0x7ffa6c0b5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:17.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.114+0000 7ffa71a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa6c0cdd00 0x7ffa6c0c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:17.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.114+0000 7ffa71a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6c0ae060 con 0x7ffa6c0ad880 2024-09-16T15:42:17.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.114+0000 7ffa71a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6c0adee0 con 0x7ffa6c0cdd00 2024-09-16T15:42:17.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.114+0000 7ffa71a00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7ffa6c15ed80 con 0x7ffa6c06fb90 2024-09-16T15:42:17.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.114+0000 7ffa6b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa6c0ad880 0x7ffa6c0b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:17.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.114+0000 7ffa6b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa6c0ad880 0x7ffa6c0b5030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:56270/0 (socket says 172.21.15.151:56270) 2024-09-16T15:42:17.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.114+0000 7ffa6b400640 1 -- 172.21.15.151:0/504596071 learned_addr learned my addr 172.21.15.151:0/504596071 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:42:17.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.114+0000 7ffa71000640 1 --2- 172.21.15.151:0/504596071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa6c0cdd00 0x7ffa6c0c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:17.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.114+0000 7ffa6be00640 1 --2- 172.21.15.151:0/504596071 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffa6c06fb90 0x7ffa6c0b40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:17.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.115+0000 7ffa6b400640 1 -- 172.21.15.151:0/504596071 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffa6c06fb90 msgr2=0x7ffa6c0b40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:17.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.115+0000 7ffa6b400640 1 --2- 172.21.15.151:0/504596071 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffa6c06fb90 0x7ffa6c0b40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:17.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.115+0000 7ffa6b400640 1 -- 172.21.15.151:0/504596071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa6c0cdd00 msgr2=0x7ffa6c0c57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:17.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.115+0000 7ffa6b400640 1 --2- 172.21.15.151:0/504596071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa6c0cdd00 0x7ffa6c0c57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:17.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.115+0000 7ffa6b400640 1 -- 172.21.15.151:0/504596071 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa6c0c7da0 con 0x7ffa6c0ad880 2024-09-16T15:42:17.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.115+0000 7ffa6b400640 1 --2- 172.21.15.151:0/504596071 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa6c0ad880 0x7ffa6c0b5030 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ffa5800c500 tx=0x7ffa5800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:17.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.115+0000 7ffa63e00640 1 -- 172.21.15.151:0/504596071 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa580052d0 con 0x7ffa6c0ad880 2024-09-16T15:42:17.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.115+0000 7ffa63e00640 1 -- 172.21.15.151:0/504596071 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffa58005470 con 0x7ffa6c0ad880 2024-09-16T15:42:17.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.115+0000 7ffa63e00640 1 -- 172.21.15.151:0/504596071 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa58012870 con 0x7ffa6c0ad880 2024-09-16T15:42:17.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.115+0000 7ffa71a00640 1 -- 172.21.15.151:0/504596071 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa6c0b56a0 con 0x7ffa6c0ad880 2024-09-16T15:42:17.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.115+0000 7ffa71a00640 1 -- 172.21.15.151:0/504596071 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa6c0b35e0 con 0x7ffa6c0ad880 2024-09-16T15:42:17.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.116+0000 7ffa71a00640 1 -- 172.21.15.151:0/504596071 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa6c06a180 con 0x7ffa6c0ad880 2024-09-16T15:42:17.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.117+0000 7ffa63e00640 1 -- 172.21.15.151:0/504596071 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7ffa580035f0 con 0x7ffa6c0ad880 2024-09-16T15:42:17.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.118+0000 7ffa63e00640 1 --2- 172.21.15.151:0/504596071 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ffa4c078bb0 0x7ffa4c07b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:17.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.119+0000 7ffa63e00640 1 -- 172.21.15.151:0/504596071 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4943+0+0 (secure 0 0 0) 0x7ffa5809a8c0 con 0x7ffa6c0ad880 2024-09-16T15:42:17.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.119+0000 7ffa6be00640 1 --2- 172.21.15.151:0/504596071 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ffa4c078bb0 0x7ffa4c07b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:17.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.120+0000 7ffa6be00640 1 --2- 172.21.15.151:0/504596071 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ffa4c078bb0 0x7ffa4c07b070 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c034000 tx=0x7ffa5c004ab0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:17.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.123+0000 7ffa63e00640 1 -- 172.21.15.151:0/504596071 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ffa58062be0 con 0x7ffa6c0ad880 2024-09-16T15:42:17.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.306+0000 7ffa71a00640 1 -- 172.21.15.151:0/504596071 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffa6c15f980 con 0x7ffa6c0ad880 2024-09-16T15:42:17.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.307+0000 7ffa63e00640 1 -- 172.21.15.151:0/504596071 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+13094 (secure 0 0 0) 0x7ffa580624e0 con 0x7ffa6c0ad880 2024-09-16T15:42:17.309 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:42:17.309 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":40,"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","created":"2024-09-16T15:38:56.348535+0000","modified":"2024-09-16T15:42:14.912751+0000","last_up_change":"2024-09-16T15:39:10.681384+0000","last_in_change":"2024-09-16T15:39:00.544477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:39:11.602252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5524076-a903-4d0a-af6a-caf474c4765d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6809","nonce":3111342741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6811","nonce":3111342741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6815","nonce":3111342741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6813","nonce":3111342741}]},"public_addr":"172.21.15.151:6809/3111342741","cluster_addr":"172.21.15.151:6811/3111342741","heartbeat_back_addr":"172.21.15.151:6815/3111342741","heartbeat_front_addr":"172.21.15.151:6813/3111342741","state":["exists","up"]},{"osd":1,"uuid":"dd25925f-0ae1-494e-8529-4dc205759db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6801","nonce":3368304087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6803","nonce":3368304087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6807","nonce":3368304087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6805","nonce":3368304087}]},"public_addr":"172.21.15.151:6801/3368304087","cluster_addr":"172.21.15.151:6803/3368304087","heartbeat_back_addr":"172.21.15.151:6807/3368304087","heartbeat_front_addr":"172.21.15.151:6805/3368304087","state":["exists","up"]},{"osd":2,"uuid":"c2f18115-013d-4da6-a0fe-cb978f4494b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6823","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6825","nonce":3079373271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6827","nonce":3079373271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6831","nonce":3079373271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6829","nonce":3079373271}]},"public_addr":"172.21.15.151:6825/3079373271","cluster_addr":"172.21.15.151:6827/3079373271","heartbeat_back_addr":"172.21.15.151:6831/3079373271","heartbeat_front_addr":"172.21.15.151:6829/3079373271","state":["exists","up"]},{"osd":3,"uuid":"36edb718-2fcb-4d88-8ddc-c321eed4279d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6817","nonce":1142509041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6819","nonce":1142509041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6824","nonce":1142509041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6821","nonce":1142509041}]},"public_addr":"172.21.15.151:6817/1142509041","cluster_addr":"172.21.15.151:6819/1142509041","heartbeat_back_addr":"172.21.15.151:6824/1142509041","heartbeat_front_addr":"172.21.15.151:6821/1142509041","state":["exists","up"]},{"osd":4,"uuid":"64d33708-cd0a-4236-abe3-1b646e82ab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6811","nonce":2735495271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6813","nonce":2735495271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6817","nonce":2735495271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6815","nonce":2735495271}]},"public_addr":"172.21.15.167:6811/2735495271","cluster_addr":"172.21.15.167:6813/2735495271","heartbeat_back_addr":"172.21.15.167:6817/2735495271","heartbeat_front_addr":"172.21.15.167:6815/2735495271","state":["exists","up"]},{"osd":5,"uuid":"f14b44a9-5910-458c-b518-09a8d0de94de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6827","nonce":1494916572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6829","nonce":1494916572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6833","nonce":1494916572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6831","nonce":1494916572}]},"public_addr":"172.21.15.167:6827/1494916572","cluster_addr":"172.21.15.167:6829/1494916572","heartbeat_back_addr":"172.21.15.167:6833/1494916572","heartbeat_front_addr":"172.21.15.167:6831/1494916572","state":["exists","up"]},{"osd":6,"uuid":"39335545-2654-4710-84b0-a9aa47cfcbba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6819","nonce":2538273273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6821","nonce":2538273273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6825","nonce":2538273273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6823","nonce":2538273273}]},"public_addr":"172.21.15.167:6819/2538273273","cluster_addr":"172.21.15.167:6821/2538273273","heartbeat_back_addr":"172.21.15.167:6825/2538273273","heartbeat_front_addr":"172.21.15.167:6823/2538273273","state":["exists","up"]},{"osd":7,"uuid":"4f8bfc81-53dd-4339-ac22-1a4007d9f963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6803","nonce":265072227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6805","nonce":265072227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6809","nonce":265072227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6807","nonce":265072227}]},"public_addr":"172.21.15.167:6803/265072227","cluster_addr":"172.21.15.167:6805/265072227","heartbeat_back_addr":"172.21.15.167:6809/265072227","heartbeat_front_addr":"172.21.15.167:6807/265072227","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.150359+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.084358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.722368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.332809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.320709+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.395153+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.935250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.801443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.167:6837/4150137107":"2024-09-17T15:42:11.516984+0000","172.21.15.167:6836/4150137107":"2024-09-17T15:42:11.516984+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:42:17.311 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.310+0000 7ffa71a00640 1 -- 172.21.15.151:0/504596071 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ffa4c078bb0 msgr2=0x7ffa4c07b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:17.311 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.310+0000 7ffa71a00640 1 --2- 172.21.15.151:0/504596071 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ffa4c078bb0 0x7ffa4c07b070 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c034000 tx=0x7ffa5c004ab0 comp rx=0 tx=0).stop 2024-09-16T15:42:17.311 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.310+0000 7ffa71a00640 1 -- 172.21.15.151:0/504596071 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa6c0ad880 msgr2=0x7ffa6c0b5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:17.311 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.310+0000 7ffa71a00640 1 --2- 172.21.15.151:0/504596071 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa6c0ad880 0x7ffa6c0b5030 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ffa5800c500 tx=0x7ffa5800c9c0 comp rx=0 tx=0).stop 2024-09-16T15:42:17.311 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.310+0000 7ffa71a00640 1 -- 172.21.15.151:0/504596071 shutdown_connections 2024-09-16T15:42:17.311 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.310+0000 7ffa71a00640 1 --2- 172.21.15.151:0/504596071 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ffa4c078bb0 0x7ffa4c07b070 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:17.311 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.310+0000 7ffa71a00640 1 --2- 172.21.15.151:0/504596071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa6c0cdd00 0x7ffa6c0c57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:17.312 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.310+0000 7ffa71a00640 1 --2- 172.21.15.151:0/504596071 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa6c0ad880 0x7ffa6c0b5030 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:17.312 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.310+0000 7ffa71a00640 1 --2- 172.21.15.151:0/504596071 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffa6c06fb90 0x7ffa6c0b40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:17.312 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.310+0000 7ffa71a00640 1 -- 172.21.15.151:0/504596071 >> 172.21.15.151:0/504596071 conn(0x7ffa6c0cfa20 msgr2=0x7ffa6c073c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:17.312 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.310+0000 7ffa71a00640 1 -- 172.21.15.151:0/504596071 shutdown_connections 2024-09-16T15:42:17.312 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.310+0000 7ffa71a00640 1 -- 172.21.15.151:0/504596071 wait complete. 2024-09-16T15:42:17.329 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T15:39:11.602252+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-16T15:42:17.329 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-16T15:42:17.395 DEBUG:teuthology.orchestra.run.smithi151:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:42:17.399 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:42:17.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.493+0000 7f59e1800640 1 -- 172.21.15.151:0/270128448 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59dc0d0db0 msgr2=0x7f59dc0b12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:17.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.493+0000 7f59e1800640 1 --2- 172.21.15.151:0/270128448 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59dc0d0db0 0x7f59dc0b12d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f59c8009830 tx=0x7f59c802a5a0 comp rx=0 tx=0).stop 2024-09-16T15:42:17.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.493+0000 7f59e1800640 1 -- 172.21.15.151:0/270128448 shutdown_connections 2024-09-16T15:42:17.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.493+0000 7f59e1800640 1 --2- 172.21.15.151:0/270128448 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f59dc074930 0x7f59dc0cbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:17.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.493+0000 7f59e1800640 1 --2- 172.21.15.151:0/270128448 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59dc0d0db0 0x7f59dc0b12d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:17.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.493+0000 7f59e1800640 1 -- 172.21.15.151:0/270128448 >> 172.21.15.151:0/270128448 conn(0x7f59dc0d0290 msgr2=0x7f59dc0d06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:17.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.493+0000 7f59e1800640 1 -- 172.21.15.151:0/270128448 shutdown_connections 2024-09-16T15:42:17.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.493+0000 7f59e1800640 1 -- 172.21.15.151:0/270128448 wait complete. 2024-09-16T15:42:17.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.495+0000 7f59e1800640 1 Processor -- start 2024-09-16T15:42:17.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.495+0000 7f59e1800640 1 -- start start 2024-09-16T15:42:17.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.496+0000 7f59e1800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f59dc074930 0x7f59dc153800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:17.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.496+0000 7f59e1800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f59dc15bd20 0x7f59dc153d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:17.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.496+0000 7f59e1800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59dc0afc90 0x7f59dc154280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:17.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.496+0000 7f59e1800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f59dc0cc390 con 0x7f59dc074930 2024-09-16T15:42:17.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.496+0000 7f59e1800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f59dc0cc210 con 0x7f59dc0afc90 2024-09-16T15:42:17.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.496+0000 7f59e1800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f59dc0cc510 con 0x7f59dc15bd20 2024-09-16T15:42:17.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.496+0000 7f59dbe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f59dc074930 0x7f59dc153800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:17.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.496+0000 7f59dbe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f59dc074930 0x7f59dc153800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:56276/0 (socket says 172.21.15.151:56276) 2024-09-16T15:42:17.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.497+0000 7f59dbe00640 1 -- 172.21.15.151:0/3545505055 learned_addr learned my addr 172.21.15.151:0/3545505055 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:42:17.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.497+0000 7f59e0e00640 1 --2- 172.21.15.151:0/3545505055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59dc0afc90 0x7f59dc154280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:17.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.497+0000 7f59db400640 1 --2- 172.21.15.151:0/3545505055 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f59dc15bd20 0x7f59dc153d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:17.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.497+0000 7f59dbe00640 1 -- 172.21.15.151:0/3545505055 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f59dc15bd20 msgr2=0x7f59dc153d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:17.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.497+0000 7f59dbe00640 1 --2- 172.21.15.151:0/3545505055 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f59dc15bd20 0x7f59dc153d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:17.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.497+0000 7f59dbe00640 1 -- 172.21.15.151:0/3545505055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59dc0afc90 msgr2=0x7f59dc154280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:17.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.497+0000 7f59dbe00640 1 --2- 172.21.15.151:0/3545505055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59dc0afc90 0x7f59dc154280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:17.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.497+0000 7f59dbe00640 1 -- 172.21.15.151:0/3545505055 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59dc0c91b0 con 0x7f59dc074930 2024-09-16T15:42:17.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.497+0000 7f59db400640 1 --2- 172.21.15.151:0/3545505055 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f59dc15bd20 0x7f59dc153d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:42:17.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.497+0000 7f59dbe00640 1 --2- 172.21.15.151:0/3545505055 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f59dc074930 0x7f59dc153800 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f59cc00c730 tx=0x7f59cc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:17.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.497+0000 7f59d3e00640 1 -- 172.21.15.151:0/3545505055 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59cc003b70 con 0x7f59dc074930 2024-09-16T15:42:17.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.498+0000 7f59d3e00640 1 -- 172.21.15.151:0/3545505055 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f59cc003d30 con 0x7f59dc074930 2024-09-16T15:42:17.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.498+0000 7f59d3e00640 1 -- 172.21.15.151:0/3545505055 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59cc00aab0 con 0x7f59dc074930 2024-09-16T15:42:17.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.498+0000 7f59e1800640 1 -- 172.21.15.151:0/3545505055 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59dc0c9490 con 0x7f59dc074930 2024-09-16T15:42:17.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.498+0000 7f59e1800640 1 -- 172.21.15.151:0/3545505055 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59dc0c9a50 con 0x7f59dc074930 2024-09-16T15:42:17.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.499+0000 7f59e1800640 1 -- 172.21.15.151:0/3545505055 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59dc0cbac0 con 0x7f59dc074930 2024-09-16T15:42:17.501 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.500+0000 7f59d3e00640 1 -- 172.21.15.151:0/3545505055 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7f59cc00ac50 con 0x7f59dc074930 2024-09-16T15:42:17.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.501+0000 7f59d3e00640 1 --2- 172.21.15.151:0/3545505055 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f59bc078bb0 0x7f59bc07b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:17.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.501+0000 7f59d3e00640 1 -- 172.21.15.151:0/3545505055 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4943+0+0 (secure 0 0 0) 0x7f59cc09ab60 con 0x7f59dc074930 2024-09-16T15:42:17.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.501+0000 7f59db400640 1 --2- 172.21.15.151:0/3545505055 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f59bc078bb0 0x7f59bc07b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:17.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.502+0000 7f59db400640 1 --2- 172.21.15.151:0/3545505055 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f59bc078bb0 0x7f59bc07b070 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f59c8034000 tx=0x7f59c8004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:17.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.505+0000 7f59d3e00640 1 -- 172.21.15.151:0/3545505055 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f59cc062e20 con 0x7f59dc074930 2024-09-16T15:42:17.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.691+0000 7f59e1800640 1 -- 172.21.15.151:0/3545505055 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f59dc0af230 con 0x7f59dc074930 2024-09-16T15:42:17.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.691+0000 7f59d3e00640 1 -- 172.21.15.151:0/3545505055 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) ==== 93+0+10 (secure 0 0 0) 0x7f59cc062720 con 0x7f59dc074930 2024-09-16T15:42:17.692 INFO:teuthology.orchestra.run.smithi151.stdout:pg_num: 1 2024-09-16T15:42:17.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.694+0000 7f59e1800640 1 -- 172.21.15.151:0/3545505055 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f59bc078bb0 msgr2=0x7f59bc07b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:17.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.694+0000 7f59e1800640 1 --2- 172.21.15.151:0/3545505055 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f59bc078bb0 0x7f59bc07b070 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f59c8034000 tx=0x7f59c8004730 comp rx=0 tx=0).stop 2024-09-16T15:42:17.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.694+0000 7f59e1800640 1 -- 172.21.15.151:0/3545505055 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f59dc074930 msgr2=0x7f59dc153800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:17.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.694+0000 7f59e1800640 1 --2- 172.21.15.151:0/3545505055 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f59dc074930 0x7f59dc153800 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f59cc00c730 tx=0x7f59cc00cbf0 comp rx=0 tx=0).stop 2024-09-16T15:42:17.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.694+0000 7f59e1800640 1 -- 172.21.15.151:0/3545505055 shutdown_connections 2024-09-16T15:42:17.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.694+0000 7f59e1800640 1 --2- 172.21.15.151:0/3545505055 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f59bc078bb0 0x7f59bc07b070 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:17.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.694+0000 7f59e1800640 1 --2- 172.21.15.151:0/3545505055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59dc0afc90 0x7f59dc154280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:17.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.694+0000 7f59e1800640 1 --2- 172.21.15.151:0/3545505055 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f59dc15bd20 0x7f59dc153d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:17.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.694+0000 7f59e1800640 1 --2- 172.21.15.151:0/3545505055 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f59dc074930 0x7f59dc153800 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:17.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.694+0000 7f59e1800640 1 -- 172.21.15.151:0/3545505055 >> 172.21.15.151:0/3545505055 conn(0x7f59dc0d0290 msgr2=0x7f59dc0b0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:17.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.694+0000 7f59e1800640 1 -- 172.21.15.151:0/3545505055 shutdown_connections 2024-09-16T15:42:17.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.694+0000 7f59e1800640 1 -- 172.21.15.151:0/3545505055 wait complete. 2024-09-16T15:42:17.710 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_equal_to_cloner_threads' 2024-09-16T15:42:17.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.908+0000 7f2de1600640 1 -- 172.21.15.151:0/4156745079 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2ddc06d280 msgr2=0x7f2ddc162150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:17.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.908+0000 7f2de1600640 1 --2- 172.21.15.151:0/4156745079 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2ddc06d280 0x7f2ddc162150 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2dd4009860 tx=0x7f2dd402a410 comp rx=0 tx=0).stop 2024-09-16T15:42:17.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.908+0000 7f2de1600640 1 -- 172.21.15.151:0/4156745079 shutdown_connections 2024-09-16T15:42:17.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.908+0000 7f2de1600640 1 --2- 172.21.15.151:0/4156745079 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2ddc06d280 0x7f2ddc162150 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:17.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.908+0000 7f2de1600640 1 -- 172.21.15.151:0/4156745079 >> 172.21.15.151:0/4156745079 conn(0x7f2ddc0b2230 msgr2=0x7f2ddc0b2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:17.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.908+0000 7f2de1600640 1 -- 172.21.15.151:0/4156745079 shutdown_connections 2024-09-16T15:42:17.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.908+0000 7f2de1600640 1 -- 172.21.15.151:0/4156745079 wait complete. 2024-09-16T15:42:17.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.910+0000 7f2de1600640 1 Processor -- start 2024-09-16T15:42:17.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.910+0000 7f2de1600640 1 -- start start 2024-09-16T15:42:17.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.911+0000 7f2de1600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ddc0688b0 0x7f2ddc1604f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:17.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.911+0000 7f2de1600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2ddc06d280 0x7f2ddc15a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:17.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.911+0000 7f2de1600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2ddc0b3e20 0x7f2ddc0aeba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:17.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.911+0000 7f2de1600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2ddc162810 con 0x7f2ddc0b3e20 2024-09-16T15:42:17.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.911+0000 7f2de1600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2ddc162690 con 0x7f2ddc0688b0 2024-09-16T15:42:17.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.911+0000 7f2de1600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f2ddc162990 con 0x7f2ddc06d280 2024-09-16T15:42:17.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.911+0000 7f2ddb400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2ddc0b3e20 0x7f2ddc0aeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:17.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.911+0000 7f2ddb400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2ddc0b3e20 0x7f2ddc0aeba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:56278/0 (socket says 172.21.15.151:56278) 2024-09-16T15:42:17.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.911+0000 7f2ddb400640 1 -- 172.21.15.151:0/2934179964 learned_addr learned my addr 172.21.15.151:0/2934179964 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:42:17.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.911+0000 7f2ddaa00640 1 --2- 172.21.15.151:0/2934179964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ddc0688b0 0x7f2ddc1604f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:17.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.911+0000 7f2dda000640 1 --2- 172.21.15.151:0/2934179964 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2ddc06d280 0x7f2ddc15a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:17.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.912+0000 7f2ddb400640 1 -- 172.21.15.151:0/2934179964 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2ddc06d280 msgr2=0x7f2ddc15a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:17.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.912+0000 7f2ddb400640 1 --2- 172.21.15.151:0/2934179964 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2ddc06d280 0x7f2ddc15a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:17.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.912+0000 7f2ddb400640 1 -- 172.21.15.151:0/2934179964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ddc0688b0 msgr2=0x7f2ddc1604f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:17.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.912+0000 7f2ddb400640 1 --2- 172.21.15.151:0/2934179964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ddc0688b0 0x7f2ddc1604f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:17.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.912+0000 7f2ddb400640 1 -- 172.21.15.151:0/2934179964 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ddc1f5930 con 0x7f2ddc0b3e20 2024-09-16T15:42:17.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.912+0000 7f2ddb400640 1 --2- 172.21.15.151:0/2934179964 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2ddc0b3e20 0x7f2ddc0aeba0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2dd4009980 tx=0x7f2dd4003b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:17.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.912+0000 7f2dd2a00640 1 -- 172.21.15.151:0/2934179964 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd403f030 con 0x7f2ddc0b3e20 2024-09-16T15:42:17.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.912+0000 7f2de1600640 1 -- 172.21.15.151:0/2934179964 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ddc15fa10 con 0x7f2ddc0b3e20 2024-09-16T15:42:17.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.912+0000 7f2dd2a00640 1 -- 172.21.15.151:0/2934179964 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2dd4003d50 con 0x7f2ddc0b3e20 2024-09-16T15:42:17.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.912+0000 7f2dd2a00640 1 -- 172.21.15.151:0/2934179964 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd4037040 con 0x7f2ddc0b3e20 2024-09-16T15:42:17.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.912+0000 7f2de1600640 1 -- 172.21.15.151:0/2934179964 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ddc15ffd0 con 0x7f2ddc0b3e20 2024-09-16T15:42:17.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.913+0000 7f2de1600640 1 -- 172.21.15.151:0/2934179964 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ddc15f800 con 0x7f2ddc0b3e20 2024-09-16T15:42:17.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.915+0000 7f2dd2a00640 1 -- 172.21.15.151:0/2934179964 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7f2dd4042070 con 0x7f2ddc0b3e20 2024-09-16T15:42:17.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.915+0000 7f2dd2a00640 1 --2- 172.21.15.151:0/2934179964 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2db8078bb0 0x7f2db807b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:17.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.916+0000 7f2dd2a00640 1 -- 172.21.15.151:0/2934179964 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4943+0+0 (secure 0 0 0) 0x7f2dd40baac0 con 0x7f2ddc0b3e20 2024-09-16T15:42:17.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.916+0000 7f2ddaa00640 1 --2- 172.21.15.151:0/2934179964 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2db8078bb0 0x7f2db807b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:17.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.917+0000 7f2ddaa00640 1 --2- 172.21.15.151:0/2934179964 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2db8078bb0 0x7f2db807b070 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc0015a0 tx=0x7f2dcc001ff0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:17.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:17.919+0000 7f2dd2a00640 1 -- 172.21.15.151:0/2934179964 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2dd4082e90 con 0x7f2ddc0b3e20 2024-09-16T15:42:18.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:18.175+0000 7f2de1600640 1 -- 172.21.15.151:0/2934179964 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_equal_to_cloner_threads"]} v 0) -- 0x7f2ddc162150 con 0x7f2ddc0b3e20 2024-09-16T15:42:18.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:18.934+0000 7f2dd2a00640 1 -- 172.21.15.151:0/2934179964 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_equal_to_cloner_threads"]}]=0 v150) ==== 169+0+0 (secure 0 0 0) 0x7f2dd4082790 con 0x7f2ddc0b3e20 2024-09-16T15:42:18.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:18.937+0000 7f2de1600640 1 -- 172.21.15.151:0/2934179964 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2db8078bb0 msgr2=0x7f2db807b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:18.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:18.937+0000 7f2de1600640 1 --2- 172.21.15.151:0/2934179964 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2db8078bb0 0x7f2db807b070 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc0015a0 tx=0x7f2dcc001ff0 comp rx=0 tx=0).stop 2024-09-16T15:42:18.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:18.937+0000 7f2de1600640 1 -- 172.21.15.151:0/2934179964 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2ddc0b3e20 msgr2=0x7f2ddc0aeba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:18.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:18.937+0000 7f2de1600640 1 --2- 172.21.15.151:0/2934179964 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2ddc0b3e20 0x7f2ddc0aeba0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2dd4009980 tx=0x7f2dd4003b60 comp rx=0 tx=0).stop 2024-09-16T15:42:18.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:18.937+0000 7f2de1600640 1 -- 172.21.15.151:0/2934179964 shutdown_connections 2024-09-16T15:42:18.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:18.937+0000 7f2de1600640 1 --2- 172.21.15.151:0/2934179964 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2db8078bb0 0x7f2db807b070 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:18.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:18.937+0000 7f2de1600640 1 --2- 172.21.15.151:0/2934179964 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2ddc0b3e20 0x7f2ddc0aeba0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:18.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:18.937+0000 7f2de1600640 1 --2- 172.21.15.151:0/2934179964 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2ddc06d280 0x7f2ddc15a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:18.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:18.937+0000 7f2de1600640 1 --2- 172.21.15.151:0/2934179964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ddc0688b0 0x7f2ddc1604f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:18.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:18.937+0000 7f2de1600640 1 -- 172.21.15.151:0/2934179964 >> 172.21.15.151:0/2934179964 conn(0x7f2ddc0b2230 msgr2=0x7f2ddc06cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:18.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:18.938+0000 7f2de1600640 1 -- 172.21.15.151:0/2934179964 shutdown_connections 2024-09-16T15:42:18.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:18.938+0000 7f2de1600640 1 -- 172.21.15.151:0/2934179964 wait complete. 2024-09-16T15:42:18.956 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-16T15:42:19.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.113+0000 7ff0afa00640 1 -- 172.21.15.151:0/2913029170 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0a815f5a0 msgr2=0x7ff0a80b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:19.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.113+0000 7ff0afa00640 1 --2- 172.21.15.151:0/2913029170 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0a815f5a0 0x7ff0a80b2440 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff098005fd0 tx=0x7ff09802a8b0 comp rx=0 tx=0).stop 2024-09-16T15:42:19.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.113+0000 7ff0afa00640 1 -- 172.21.15.151:0/2913029170 shutdown_connections 2024-09-16T15:42:19.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.113+0000 7ff0afa00640 1 --2- 172.21.15.151:0/2913029170 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0a815f5a0 0x7ff0a80b2440 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:19.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.113+0000 7ff0afa00640 1 -- 172.21.15.151:0/2913029170 >> 172.21.15.151:0/2913029170 conn(0x7ff0a80b1950 msgr2=0x7ff0a80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:19.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.113+0000 7ff0afa00640 1 -- 172.21.15.151:0/2913029170 shutdown_connections 2024-09-16T15:42:19.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.113+0000 7ff0afa00640 1 -- 172.21.15.151:0/2913029170 wait complete. 2024-09-16T15:42:19.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.115+0000 7ff0afa00640 1 Processor -- start 2024-09-16T15:42:19.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.115+0000 7ff0afa00640 1 -- start start 2024-09-16T15:42:19.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.116+0000 7ff0afa00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff0a8073fb0 0x7ff0a80b3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:19.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.116+0000 7ff0afa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0a80ccf00 0x7ff0a815dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:19.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.116+0000 7ff0afa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0a815f5a0 0x7ff0a809bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:19.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.116+0000 7ff0afa00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a806f9f0 con 0x7ff0a80ccf00 2024-09-16T15:42:19.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.116+0000 7ff0afa00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a806f870 con 0x7ff0a815f5a0 2024-09-16T15:42:19.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.116+0000 7ff0afa00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7ff0a8074420 con 0x7ff0a8073fb0 2024-09-16T15:42:19.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.116+0000 7ff0a7e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0a80ccf00 0x7ff0a815dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:19.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.116+0000 7ff0a7e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0a80ccf00 0x7ff0a815dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:56298/0 (socket says 172.21.15.151:56298) 2024-09-16T15:42:19.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.116+0000 7ff0a7e00640 1 -- 172.21.15.151:0/1012957451 learned_addr learned my addr 172.21.15.151:0/1012957451 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:42:19.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.117+0000 7ff0a7e00640 1 -- 172.21.15.151:0/1012957451 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff0a8073fb0 msgr2=0x7ff0a80b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:19.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.117+0000 7ff0ad000640 1 --2- 172.21.15.151:0/1012957451 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff0a8073fb0 0x7ff0a80b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:19.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.117+0000 7ff0a7e00640 1 --2- 172.21.15.151:0/1012957451 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff0a8073fb0 0x7ff0a80b3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:19.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.117+0000 7ff0ada00640 1 --2- 172.21.15.151:0/1012957451 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0a815f5a0 0x7ff0a809bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:19.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.117+0000 7ff0a7e00640 1 -- 172.21.15.151:0/1012957451 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0a815f5a0 msgr2=0x7ff0a809bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:19.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.117+0000 7ff0a7e00640 1 --2- 172.21.15.151:0/1012957451 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0a815f5a0 0x7ff0a809bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:19.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.117+0000 7ff0a7e00640 1 -- 172.21.15.151:0/1012957451 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0a80b2ed0 con 0x7ff0a80ccf00 2024-09-16T15:42:19.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.117+0000 7ff0ad000640 1 --2- 172.21.15.151:0/1012957451 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff0a8073fb0 0x7ff0a80b3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:42:19.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.117+0000 7ff0a7e00640 1 --2- 172.21.15.151:0/1012957451 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0a80ccf00 0x7ff0a815dd80 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff09400c500 tx=0x7ff09400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:19.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.117+0000 7ff0a4c00640 1 -- 172.21.15.151:0/1012957451 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0940052d0 con 0x7ff0a80ccf00 2024-09-16T15:42:19.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.117+0000 7ff0a4c00640 1 -- 172.21.15.151:0/1012957451 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff094005470 con 0x7ff0a80ccf00 2024-09-16T15:42:19.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.117+0000 7ff0a4c00640 1 -- 172.21.15.151:0/1012957451 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff094012870 con 0x7ff0a80ccf00 2024-09-16T15:42:19.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.117+0000 7ff0afa00640 1 -- 172.21.15.151:0/1012957451 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0a81f5580 con 0x7ff0a80ccf00 2024-09-16T15:42:19.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.118+0000 7ff0afa00640 1 -- 172.21.15.151:0/1012957451 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0a80b3470 con 0x7ff0a80ccf00 2024-09-16T15:42:19.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.118+0000 7ff0afa00640 1 -- 172.21.15.151:0/1012957451 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0a80b2440 con 0x7ff0a80ccf00 2024-09-16T15:42:19.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.120+0000 7ff0a4c00640 1 -- 172.21.15.151:0/1012957451 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7ff0940035f0 con 0x7ff0a80ccf00 2024-09-16T15:42:19.122 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.121+0000 7ff0a4c00640 1 --2- 172.21.15.151:0/1012957451 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff088078bb0 0x7ff08807b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:19.122 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.121+0000 7ff0a4c00640 1 -- 172.21.15.151:0/1012957451 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4943+0+0 (secure 0 0 0) 0x7ff09409a8e0 con 0x7ff0a80ccf00 2024-09-16T15:42:19.122 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.121+0000 7ff0ad000640 1 --2- 172.21.15.151:0/1012957451 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff088078bb0 0x7ff08807b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:19.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.122+0000 7ff0ad000640 1 --2- 172.21.15.151:0/1012957451 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff088078bb0 0x7ff08807b070 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff098034000 tx=0x7ff098004ab0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:19.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.125+0000 7ff0a4c00640 1 -- 172.21.15.151:0/1012957451 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff094062c00 con 0x7ff0a80ccf00 2024-09-16T15:42:19.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.309+0000 7ff0afa00640 1 -- 172.21.15.151:0/1012957451 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7ff0a8153d30 con 0x7ff0a80ccf00 2024-09-16T15:42:19.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.309+0000 7ff0a4c00640 1 -- 172.21.15.151:0/1012957451 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v30) ==== 59+0+1998 (secure 0 0 0) 0x7ff094062500 con 0x7ff0a80ccf00 2024-09-16T15:42:19.311 INFO:teuthology.orchestra.run.smithi151.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-16T15:42:19.311 INFO:teuthology.orchestra.run.smithi151.stdout:mon advanced mon_op_complaint_time 120 2024-09-16T15:42:19.311 INFO:teuthology.orchestra.run.smithi151.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-16T15:42:19.311 INFO:teuthology.orchestra.run.smithi151.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-16T15:42:19.311 INFO:teuthology.orchestra.run.smithi151.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3795.338492 2024-09-16T15:42:19.311 INFO:teuthology.orchestra.run.smithi151.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 4030.355715 2024-09-16T15:42:19.311 INFO:teuthology.orchestra.run.smithi151.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3817.533640 2024-09-16T15:42:19.311 INFO:teuthology.orchestra.run.smithi151.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3950.656136 2024-09-16T15:42:19.311 INFO:teuthology.orchestra.run.smithi151.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4449.408052 2024-09-16T15:42:19.311 INFO:teuthology.orchestra.run.smithi151.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4439.932296 2024-09-16T15:42:19.311 INFO:teuthology.orchestra.run.smithi151.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4566.875579 2024-09-16T15:42:19.312 INFO:teuthology.orchestra.run.smithi151.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 6320.073632 2024-09-16T15:42:19.312 INFO:teuthology.orchestra.run.smithi151.stdout:mds advanced debug_mds 20/20 2024-09-16T15:42:19.312 INFO:teuthology.orchestra.run.smithi151.stdout:mds advanced debug_mds_balancer 20/20 2024-09-16T15:42:19.312 INFO:teuthology.orchestra.run.smithi151.stdout:mds advanced debug_ms 2/2 2024-09-16T15:42:19.312 INFO:teuthology.orchestra.run.smithi151.stdout:mds dev mds_debug_frag true 2024-09-16T15:42:19.312 INFO:teuthology.orchestra.run.smithi151.stdout:mds dev mds_debug_scatterstat true 2024-09-16T15:42:19.312 INFO:teuthology.orchestra.run.smithi151.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-16T15:42:19.312 INFO:teuthology.orchestra.run.smithi151.stdout:mds dev mds_verify_scatter true 2024-09-16T15:42:19.312 INFO:teuthology.orchestra.run.smithi151.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-16T15:42:19.312 INFO:teuthology.orchestra.run.smithi151.stdout:mds advanced rados_mon_op_timeout 900 2024-09-16T15:42:19.312 INFO:teuthology.orchestra.run.smithi151.stdout:mds advanced rados_osd_op_timeout 900 2024-09-16T15:42:19.312 INFO:teuthology.orchestra.run.smithi151.stdout:client advanced client_mount_timeout 600 2024-09-16T15:42:19.312 INFO:teuthology.orchestra.run.smithi151.stdout:client advanced debug_client 20/20 2024-09-16T15:42:19.312 INFO:teuthology.orchestra.run.smithi151.stdout:client advanced debug_ms 1/1 2024-09-16T15:42:19.313 INFO:teuthology.orchestra.run.smithi151.stdout:client advanced rados_mon_op_timeout 900 2024-09-16T15:42:19.313 INFO:teuthology.orchestra.run.smithi151.stdout:client advanced rados_osd_op_timeout 900 2024-09-16T15:42:19.313 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.312+0000 7ff0afa00640 1 -- 172.21.15.151:0/1012957451 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff088078bb0 msgr2=0x7ff08807b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:19.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.312+0000 7ff0afa00640 1 --2- 172.21.15.151:0/1012957451 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff088078bb0 0x7ff08807b070 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff098034000 tx=0x7ff098004ab0 comp rx=0 tx=0).stop 2024-09-16T15:42:19.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.312+0000 7ff0afa00640 1 -- 172.21.15.151:0/1012957451 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0a80ccf00 msgr2=0x7ff0a815dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:19.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.312+0000 7ff0afa00640 1 --2- 172.21.15.151:0/1012957451 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0a80ccf00 0x7ff0a815dd80 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff09400c500 tx=0x7ff09400c9c0 comp rx=0 tx=0).stop 2024-09-16T15:42:19.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.312+0000 7ff0afa00640 1 -- 172.21.15.151:0/1012957451 shutdown_connections 2024-09-16T15:42:19.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.312+0000 7ff0afa00640 1 --2- 172.21.15.151:0/1012957451 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff088078bb0 0x7ff08807b070 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:19.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.312+0000 7ff0afa00640 1 --2- 172.21.15.151:0/1012957451 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0a815f5a0 0x7ff0a809bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:19.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.312+0000 7ff0afa00640 1 --2- 172.21.15.151:0/1012957451 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0a80ccf00 0x7ff0a815dd80 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:19.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.312+0000 7ff0afa00640 1 --2- 172.21.15.151:0/1012957451 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff0a8073fb0 0x7ff0a80b3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:19.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.312+0000 7ff0afa00640 1 -- 172.21.15.151:0/1012957451 >> 172.21.15.151:0/1012957451 conn(0x7ff0a80b1950 msgr2=0x7ff0a8075890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:19.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.312+0000 7ff0afa00640 1 -- 172.21.15.151:0/1012957451 shutdown_connections 2024-09-16T15:42:19.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.312+0000 7ff0afa00640 1 -- 172.21.15.151:0/1012957451 wait complete. 2024-09-16T15:42:19.328 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-16T15:42:19.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.488+0000 7ffbb2a00640 1 -- 172.21.15.151:0/1249727082 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffbac145750 msgr2=0x7ffbac073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:19.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.488+0000 7ffbb2a00640 1 --2- 172.21.15.151:0/1249727082 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffbac145750 0x7ffbac073440 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c005fd0 tx=0x7ffb9c02a820 comp rx=0 tx=0).stop 2024-09-16T15:42:19.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.488+0000 7ffbb2a00640 1 -- 172.21.15.151:0/1249727082 shutdown_connections 2024-09-16T15:42:19.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.488+0000 7ffbb2a00640 1 --2- 172.21.15.151:0/1249727082 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffbac06efa0 0x7ffbac0cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:19.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.488+0000 7ffbb2a00640 1 --2- 172.21.15.151:0/1249727082 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffbac145750 0x7ffbac073440 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:19.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.488+0000 7ffbb2a00640 1 -- 172.21.15.151:0/1249727082 >> 172.21.15.151:0/1249727082 conn(0x7ffbac0b1840 msgr2=0x7ffbac0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:19.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.488+0000 7ffbb2a00640 1 -- 172.21.15.151:0/1249727082 shutdown_connections 2024-09-16T15:42:19.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.488+0000 7ffbb2a00640 1 -- 172.21.15.151:0/1249727082 wait complete. 2024-09-16T15:42:19.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.490+0000 7ffbb2a00640 1 Processor -- start 2024-09-16T15:42:19.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.490+0000 7ffbb2a00640 1 -- start start 2024-09-16T15:42:19.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.491+0000 7ffbb2a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffbac06efa0 0x7ffbac1f63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:19.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.491+0000 7ffbb2a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffbac0ccb00 0x7ffbac1411d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:19.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.491+0000 7ffbb2a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffbac145750 0x7ffbac13dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:19.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.491+0000 7ffbb2a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ffbac160100 con 0x7ffbac0ccb00 2024-09-16T15:42:19.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.491+0000 7ffbb2a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ffbac0cedd0 con 0x7ffbac145750 2024-09-16T15:42:19.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.491+0000 7ffbb2a00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7ffbac160280 con 0x7ffbac06efa0 2024-09-16T15:42:19.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.491+0000 7ffbab400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffbac0ccb00 0x7ffbac1411d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:19.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.491+0000 7ffbab400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffbac0ccb00 0x7ffbac1411d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:56312/0 (socket says 172.21.15.151:56312) 2024-09-16T15:42:19.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.491+0000 7ffbab400640 1 -- 172.21.15.151:0/1998828381 learned_addr learned my addr 172.21.15.151:0/1998828381 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:42:19.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.491+0000 7ffbabe00640 1 --2- 172.21.15.151:0/1998828381 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffbac06efa0 0x7ffbac1f63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:19.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.491+0000 7ffbb0a00640 1 --2- 172.21.15.151:0/1998828381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffbac145750 0x7ffbac13dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:19.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.491+0000 7ffbab400640 1 -- 172.21.15.151:0/1998828381 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffbac06efa0 msgr2=0x7ffbac1f63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:19.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.491+0000 7ffbab400640 1 --2- 172.21.15.151:0/1998828381 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffbac06efa0 0x7ffbac1f63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:19.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.491+0000 7ffbab400640 1 -- 172.21.15.151:0/1998828381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffbac145750 msgr2=0x7ffbac13dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:19.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.491+0000 7ffbab400640 1 --2- 172.21.15.151:0/1998828381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffbac145750 0x7ffbac13dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:19.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.491+0000 7ffbab400640 1 -- 172.21.15.151:0/1998828381 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbac1f6930 con 0x7ffbac0ccb00 2024-09-16T15:42:19.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.491+0000 7ffbb0a00640 1 --2- 172.21.15.151:0/1998828381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffbac145750 0x7ffbac13dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:42:19.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.492+0000 7ffbab400640 1 --2- 172.21.15.151:0/1998828381 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffbac0ccb00 0x7ffbac1411d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ffb9800d950 tx=0x7ffb9800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:19.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.492+0000 7ffba3e00640 1 -- 172.21.15.151:0/1998828381 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb98017070 con 0x7ffbac0ccb00 2024-09-16T15:42:19.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.492+0000 7ffba3e00640 1 -- 172.21.15.151:0/1998828381 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffb98009d20 con 0x7ffbac0ccb00 2024-09-16T15:42:19.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.492+0000 7ffba3e00640 1 -- 172.21.15.151:0/1998828381 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb980106c0 con 0x7ffbac0ccb00 2024-09-16T15:42:19.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.492+0000 7ffbb2a00640 1 -- 172.21.15.151:0/1998828381 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbac1f28b0 con 0x7ffbac0ccb00 2024-09-16T15:42:19.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.492+0000 7ffbb2a00640 1 -- 172.21.15.151:0/1998828381 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbac1f2e70 con 0x7ffbac0ccb00 2024-09-16T15:42:19.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.493+0000 7ffbb2a00640 1 -- 172.21.15.151:0/1998828381 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbac160730 con 0x7ffbac0ccb00 2024-09-16T15:42:19.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.494+0000 7ffba3e00640 1 -- 172.21.15.151:0/1998828381 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7ffb98010860 con 0x7ffbac0ccb00 2024-09-16T15:42:19.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.496+0000 7ffba3e00640 1 --2- 172.21.15.151:0/1998828381 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ffb8c078bb0 0x7ffb8c07b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:19.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.496+0000 7ffba3e00640 1 -- 172.21.15.151:0/1998828381 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4943+0+0 (secure 0 0 0) 0x7ffb9809b170 con 0x7ffbac0ccb00 2024-09-16T15:42:19.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.496+0000 7ffbabe00640 1 --2- 172.21.15.151:0/1998828381 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ffb8c078bb0 0x7ffb8c07b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:19.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.497+0000 7ffbabe00640 1 --2- 172.21.15.151:0/1998828381 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ffb8c078bb0 0x7ffb8c07b070 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c034000 tx=0x7ffb9c004ab0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:19.501 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.500+0000 7ffba3e00640 1 -- 172.21.15.151:0/1998828381 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ffbac1f6930 con 0x7ffbac0ccb00 2024-09-16T15:42:19.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.686+0000 7ffbb2a00640 1 -- 172.21.15.151:0/1998828381 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7ffbac0d08a0 con 0x7ffbac0ccb00 2024-09-16T15:42:19.690 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.689+0000 7ffba3e00640 1 -- 172.21.15.151:0/1998828381 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v31)=0 v31) ==== 122+0+0 (secure 0 0 0) 0x7ffb980634c0 con 0x7ffbac0ccb00 2024-09-16T15:42:19.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.692+0000 7ffbb2a00640 1 -- 172.21.15.151:0/1998828381 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ffb8c078bb0 msgr2=0x7ffb8c07b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:19.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.692+0000 7ffbb2a00640 1 --2- 172.21.15.151:0/1998828381 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ffb8c078bb0 0x7ffb8c07b070 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c034000 tx=0x7ffb9c004ab0 comp rx=0 tx=0).stop 2024-09-16T15:42:19.694 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.692+0000 7ffbb2a00640 1 -- 172.21.15.151:0/1998828381 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffbac0ccb00 msgr2=0x7ffbac1411d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:19.694 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.692+0000 7ffbb2a00640 1 --2- 172.21.15.151:0/1998828381 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffbac0ccb00 0x7ffbac1411d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ffb9800d950 tx=0x7ffb9800de10 comp rx=0 tx=0).stop 2024-09-16T15:42:19.694 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.692+0000 7ffbb2a00640 1 -- 172.21.15.151:0/1998828381 shutdown_connections 2024-09-16T15:42:19.694 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.692+0000 7ffbb2a00640 1 --2- 172.21.15.151:0/1998828381 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ffb8c078bb0 0x7ffb8c07b070 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:19.694 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.692+0000 7ffbb2a00640 1 --2- 172.21.15.151:0/1998828381 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffbac145750 0x7ffbac13dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:19.694 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.692+0000 7ffbb2a00640 1 --2- 172.21.15.151:0/1998828381 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffbac0ccb00 0x7ffbac1411d0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:19.694 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.692+0000 7ffbb2a00640 1 --2- 172.21.15.151:0/1998828381 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffbac06efa0 0x7ffbac1f63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:19.694 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.692+0000 7ffbb2a00640 1 -- 172.21.15.151:0/1998828381 >> 172.21.15.151:0/1998828381 conn(0x7ffbac0b1840 msgr2=0x7ffbac0cfbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:19.694 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.692+0000 7ffbb2a00640 1 -- 172.21.15.151:0/1998828381 shutdown_connections 2024-09-16T15:42:19.694 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:19.692+0000 7ffbb2a00640 1 -- 172.21.15.151:0/1998828381 wait complete. 2024-09-16T15:42:19.709 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-16T15:42:19.709 DEBUG:teuthology.orchestra.run.smithi151:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:42:19.804 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:42:19.898 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-16T15:42:19.899 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:42:19.899 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:42:19.934 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T15:42:19.935 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-16T15:42:19.935 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:42:19.973 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:42:19.973 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:42:19.973 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:42:20.010 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T15:42:20.010 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-16T15:42:20.010 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:42:20.050 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:42:20.050 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-16T15:42:20.050 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:42:20.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.207+0000 7f059f200640 1 -- 172.21.15.151:0/79716588 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05a00b2780 msgr2=0x7f05a0168c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:20.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.207+0000 7f059f200640 1 --2- 172.21.15.151:0/79716588 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05a00b2780 0x7f05a0168c30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f05a00d4ea0 tx=0x7f059802a890 comp rx=0 tx=0).stop 2024-09-16T15:42:20.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.207+0000 7f059f200640 1 -- 172.21.15.151:0/79716588 shutdown_connections 2024-09-16T15:42:20.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.207+0000 7f059f200640 1 --2- 172.21.15.151:0/79716588 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05a00b2780 0x7f05a0168c30 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:20.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.207+0000 7f059f200640 1 -- 172.21.15.151:0/79716588 >> 172.21.15.151:0/79716588 conn(0x7f05a00d3df0 msgr2=0x7f05a00d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:20.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.207+0000 7f059f200640 1 -- 172.21.15.151:0/79716588 shutdown_connections 2024-09-16T15:42:20.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.207+0000 7f059f200640 1 -- 172.21.15.151:0/79716588 wait complete. 2024-09-16T15:42:20.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.209+0000 7f059f200640 1 Processor -- start 2024-09-16T15:42:20.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.209+0000 7f059f200640 1 -- start start 2024-09-16T15:42:20.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.210+0000 7f059f200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05a00b2780 0x7f05a0156590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:20.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.210+0000 7f059f200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f05a00b5370 0x7f05a0157f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:20.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.210+0000 7f059f200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f05a0161440 0x7f05a00b36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:20.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.210+0000 7f059f200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f05a01692f0 con 0x7f05a00b5370 2024-09-16T15:42:20.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.210+0000 7f059f200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f05a0169170 con 0x7f05a00b2780 2024-09-16T15:42:20.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.210+0000 7f059f200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f05a0169470 con 0x7f05a0161440 2024-09-16T15:42:20.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.210+0000 7f059d400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f05a00b5370 0x7f05a0157f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:20.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.210+0000 7f059d400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f05a00b5370 0x7f05a0157f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:56320/0 (socket says 172.21.15.151:56320) 2024-09-16T15:42:20.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.210+0000 7f059d400640 1 -- 172.21.15.151:0/820343294 learned_addr learned my addr 172.21.15.151:0/820343294 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:42:20.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.210+0000 7f059de00640 1 --2- 172.21.15.151:0/820343294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05a00b2780 0x7f05a0156590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:20.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.210+0000 7f059e800640 1 --2- 172.21.15.151:0/820343294 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f05a0161440 0x7f05a00b36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:20.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.210+0000 7f059d400640 1 -- 172.21.15.151:0/820343294 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f05a0161440 msgr2=0x7f05a00b36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:20.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.210+0000 7f059d400640 1 --2- 172.21.15.151:0/820343294 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f05a0161440 0x7f05a00b36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:20.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.210+0000 7f059d400640 1 -- 172.21.15.151:0/820343294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05a00b2780 msgr2=0x7f05a0156590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:20.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.210+0000 7f059d400640 1 --2- 172.21.15.151:0/820343294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05a00b2780 0x7f05a0156590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:20.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.210+0000 7f059d400640 1 -- 172.21.15.151:0/820343294 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05a014ec90 con 0x7f05a00b5370 2024-09-16T15:42:20.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.211+0000 7f059d400640 1 --2- 172.21.15.151:0/820343294 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f05a00b5370 0x7f05a0157f20 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f058c00c4c0 tx=0x7f058c00c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:20.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.211+0000 7f0596000640 1 -- 172.21.15.151:0/820343294 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f058c016020 con 0x7f05a00b5370 2024-09-16T15:42:20.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.211+0000 7f0596000640 1 -- 172.21.15.151:0/820343294 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f058c012940 con 0x7f05a00b5370 2024-09-16T15:42:20.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.211+0000 7f059f200640 1 -- 172.21.15.151:0/820343294 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05a0158460 con 0x7f05a00b5370 2024-09-16T15:42:20.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.211+0000 7f0596000640 1 -- 172.21.15.151:0/820343294 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f058c012d90 con 0x7f05a00b5370 2024-09-16T15:42:20.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.211+0000 7f059f200640 1 -- 172.21.15.151:0/820343294 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05a01589e0 con 0x7f05a00b5370 2024-09-16T15:42:20.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.212+0000 7f059f200640 1 -- 172.21.15.151:0/820343294 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05a00b34b0 con 0x7f05a00b5370 2024-09-16T15:42:20.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.213+0000 7f0596000640 1 -- 172.21.15.151:0/820343294 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7f058c019020 con 0x7f05a00b5370 2024-09-16T15:42:20.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.215+0000 7f0596000640 1 --2- 172.21.15.151:0/820343294 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0580078bb0 0x7f058007b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:20.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.216+0000 7f0596000640 1 -- 172.21.15.151:0/820343294 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4943+0+0 (secure 0 0 0) 0x7f058c09a8c0 con 0x7f05a00b5370 2024-09-16T15:42:20.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.216+0000 7f059de00640 1 --2- 172.21.15.151:0/820343294 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0580078bb0 0x7f058007b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:20.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.216+0000 7f059de00640 1 --2- 172.21.15.151:0/820343294 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0580078bb0 0x7f058007b070 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0590001720 tx=0x7f0590009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:20.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.220+0000 7f0596000640 1 -- 172.21.15.151:0/820343294 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f058c020030 con 0x7f05a00b5370 2024-09-16T15:42:20.442 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.441+0000 7f059f200640 1 -- 172.21.15.151:0/820343294 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f05a0158bb0 con 0x7f05a00b5370 2024-09-16T15:42:20.443 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.442+0000 7f0596000640 1 -- 172.21.15.151:0/820343294 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+3920 (secure 0 0 0) 0x7f058c062600 con 0x7f05a00b5370 2024-09-16T15:42:20.443 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:42:20.443 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":20,"btime":"2024-09-16T15:42:12:519594+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4661,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/873312529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":873312529},{"type":"v1","addr":"172.21.15.151:6834","nonce":873312529}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4667,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/2683082898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":2683082898},{"type":"v1","addr":"172.21.15.167:6835","nonce":2683082898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4843,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2443370538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":2443370538},{"type":"v1","addr":"172.21.15.151:6835","nonce":2443370538}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5182,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2685402570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2685402570},{"type":"v1","addr":"172.21.15.167:6837","nonce":2685402570}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[]} 2024-09-16T15:42:20.444 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 20 2024-09-16T15:42:20.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.444+0000 7f059f200640 1 -- 172.21.15.151:0/820343294 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0580078bb0 msgr2=0x7f058007b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:20.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.444+0000 7f059f200640 1 --2- 172.21.15.151:0/820343294 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0580078bb0 0x7f058007b070 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0590001720 tx=0x7f0590009040 comp rx=0 tx=0).stop 2024-09-16T15:42:20.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.444+0000 7f059f200640 1 -- 172.21.15.151:0/820343294 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f05a00b5370 msgr2=0x7f05a0157f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:20.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.444+0000 7f059f200640 1 --2- 172.21.15.151:0/820343294 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f05a00b5370 0x7f05a0157f20 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f058c00c4c0 tx=0x7f058c00c980 comp rx=0 tx=0).stop 2024-09-16T15:42:20.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.444+0000 7f059f200640 1 -- 172.21.15.151:0/820343294 shutdown_connections 2024-09-16T15:42:20.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.444+0000 7f059f200640 1 --2- 172.21.15.151:0/820343294 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0580078bb0 0x7f058007b070 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:20.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.444+0000 7f059f200640 1 --2- 172.21.15.151:0/820343294 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f05a0161440 0x7f05a00b36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:20.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.444+0000 7f059f200640 1 --2- 172.21.15.151:0/820343294 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f05a00b5370 0x7f05a0157f20 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:20.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.444+0000 7f059f200640 1 --2- 172.21.15.151:0/820343294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f05a00b2780 0x7f05a0156590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:20.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.444+0000 7f059f200640 1 -- 172.21.15.151:0/820343294 >> 172.21.15.151:0/820343294 conn(0x7f05a00d3df0 msgr2=0x7f05a015fa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:20.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.445+0000 7f059f200640 1 -- 172.21.15.151:0/820343294 shutdown_connections 2024-09-16T15:42:20.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.445+0000 7f059f200640 1 -- 172.21.15.151:0/820343294 wait complete. 2024-09-16T15:42:20.463 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-16T15:42:20.463 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-16T15:42:20.463 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-16T15:42:20.463 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:42:20.463 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-16T15:42:20.463 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-16T15:42:20.463 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-16T15:42:20.464 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:42:20.464 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-16T15:42:20.464 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-16T15:42:20.464 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-16T15:42:20.464 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:42:20.464 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-16T15:42:20.464 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-16T15:42:20.465 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-16T15:42:20.465 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:42:20.478 INFO:tasks.ceph.mds.a:Stopped 2024-09-16T15:42:20.478 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-16T15:42:20.478 INFO:tasks.ceph.mds.c:Stopped 2024-09-16T15:42:20.479 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-16T15:42:20.481 INFO:tasks.ceph.mds.b:Stopped 2024-09-16T15:42:20.481 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-16T15:42:20.481 INFO:tasks.ceph.mds.d:Stopped 2024-09-16T15:42:20.482 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-16T15:42:20.522 INFO:tasks.ceph.mds.a:Started 2024-09-16T15:42:20.522 DEBUG:teuthology.parallel:result is None 2024-09-16T15:42:20.524 INFO:tasks.ceph.mds.c:Started 2024-09-16T15:42:20.524 DEBUG:teuthology.parallel:result is None 2024-09-16T15:42:20.526 INFO:tasks.ceph.mds.b:Started 2024-09-16T15:42:20.526 DEBUG:teuthology.parallel:result is None 2024-09-16T15:42:20.528 INFO:tasks.ceph.mds.d:Started 2024-09-16T15:42:20.528 DEBUG:teuthology.parallel:result is None 2024-09-16T15:42:20.528 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-16T15:42:20.651 INFO:tasks.ceph.mds.a.smithi151.stdout:starting mds.a at 2024-09-16T15:42:20.652 INFO:tasks.ceph.mds.c.smithi151.stdout:starting mds.c at 2024-09-16T15:42:20.655 INFO:tasks.ceph.mds.d.smithi167.stdout:starting mds.d at 2024-09-16T15:42:20.655 INFO:tasks.ceph.mds.b.smithi167.stdout:starting mds.b at 2024-09-16T15:42:20.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.694+0000 7fb55fe00640 1 -- 172.21.15.151:0/2880941559 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5600b1680 msgr2=0x7fb5600b1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:20.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.694+0000 7fb55fe00640 1 --2- 172.21.15.151:0/2880941559 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5600b1680 0x7fb5600b1b30 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb558009be0 tx=0x7fb55802a8e0 comp rx=0 tx=0).stop 2024-09-16T15:42:20.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.694+0000 7fb55fe00640 1 -- 172.21.15.151:0/2880941559 shutdown_connections 2024-09-16T15:42:20.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.694+0000 7fb55fe00640 1 --2- 172.21.15.151:0/2880941559 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5600b1680 0x7fb5600b1b30 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:20.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.694+0000 7fb55fe00640 1 --2- 172.21.15.151:0/2880941559 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5600d0670 0x7fb5600d2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:20.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.694+0000 7fb55fe00640 1 -- 172.21.15.151:0/2880941559 >> 172.21.15.151:0/2880941559 conn(0x7fb5600d1c40 msgr2=0x7fb5600d2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:20.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.694+0000 7fb55fe00640 1 -- 172.21.15.151:0/2880941559 shutdown_connections 2024-09-16T15:42:20.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.694+0000 7fb55fe00640 1 -- 172.21.15.151:0/2880941559 wait complete. 2024-09-16T15:42:20.697 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.696+0000 7fb55fe00640 1 Processor -- start 2024-09-16T15:42:20.697 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.696+0000 7fb55fe00640 1 -- start start 2024-09-16T15:42:20.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.697+0000 7fb55fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5600b1680 0x7fb560071ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:20.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.697+0000 7fb55fe00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5600d0670 0x7fb5600cb810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:20.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.697+0000 7fb55fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb56015bd30 0x7fb5601f4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:20.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.697+0000 7fb55fe00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb5600b21f0 con 0x7fb56015bd30 2024-09-16T15:42:20.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.697+0000 7fb55fe00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb5600b2070 con 0x7fb5600b1680 2024-09-16T15:42:20.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.697+0000 7fb55fe00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fb5600b2370 con 0x7fb5600d0670 2024-09-16T15:42:20.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.697+0000 7fb55f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb56015bd30 0x7fb5601f4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:20.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.697+0000 7fb55f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb56015bd30 0x7fb5601f4d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:56364/0 (socket says 172.21.15.151:56364) 2024-09-16T15:42:20.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.697+0000 7fb55f400640 1 -- 172.21.15.151:0/863386544 learned_addr learned my addr 172.21.15.151:0/863386544 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:42:20.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.697+0000 7fb55e000640 1 --2- 172.21.15.151:0/863386544 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5600d0670 0x7fb5600cb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:20.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.697+0000 7fb55ea00640 1 --2- 172.21.15.151:0/863386544 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5600b1680 0x7fb560071ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:20.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.698+0000 7fb55f400640 1 -- 172.21.15.151:0/863386544 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5600d0670 msgr2=0x7fb5600cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:20.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.698+0000 7fb55f400640 1 --2- 172.21.15.151:0/863386544 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5600d0670 0x7fb5600cb810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:20.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.698+0000 7fb55f400640 1 -- 172.21.15.151:0/863386544 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5600b1680 msgr2=0x7fb560071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:20.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.698+0000 7fb55f400640 1 --2- 172.21.15.151:0/863386544 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5600b1680 0x7fb560071ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:20.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.698+0000 7fb55f400640 1 -- 172.21.15.151:0/863386544 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5600ca240 con 0x7fb56015bd30 2024-09-16T15:42:20.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.698+0000 7fb55f400640 1 --2- 172.21.15.151:0/863386544 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb56015bd30 0x7fb5601f4d80 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb558006010 tx=0x7fb558032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:20.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.698+0000 7fb556a00640 1 -- 172.21.15.151:0/863386544 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb558031430 con 0x7fb56015bd30 2024-09-16T15:42:20.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.698+0000 7fb556a00640 1 -- 172.21.15.151:0/863386544 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb558004450 con 0x7fb56015bd30 2024-09-16T15:42:20.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.698+0000 7fb55fe00640 1 -- 172.21.15.151:0/863386544 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5600ca520 con 0x7fb56015bd30 2024-09-16T15:42:20.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.698+0000 7fb556a00640 1 -- 172.21.15.151:0/863386544 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb558045410 con 0x7fb56015bd30 2024-09-16T15:42:20.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.699+0000 7fb55fe00640 1 -- 172.21.15.151:0/863386544 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5600caae0 con 0x7fb56015bd30 2024-09-16T15:42:20.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.699+0000 7fb55fe00640 1 -- 172.21.15.151:0/863386544 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5600b2900 con 0x7fb56015bd30 2024-09-16T15:42:20.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.701+0000 7fb556a00640 1 -- 172.21.15.151:0/863386544 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7fb558043020 con 0x7fb56015bd30 2024-09-16T15:42:20.703 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.702+0000 7fb556a00640 1 --2- 172.21.15.151:0/863386544 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb540078bb0 0x7fb54007b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:20.703 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.702+0000 7fb556a00640 1 -- 172.21.15.151:0/863386544 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4943+0+0 (secure 0 0 0) 0x7fb55802e030 con 0x7fb56015bd30 2024-09-16T15:42:20.703 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.702+0000 7fb55ea00640 1 --2- 172.21.15.151:0/863386544 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb540078bb0 0x7fb54007b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:20.703 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.703+0000 7fb55ea00640 1 --2- 172.21.15.151:0/863386544 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb540078bb0 0x7fb54007b070 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb56006f8e0 tx=0x7fb550007e90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:20.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.706+0000 7fb556a00640 1 -- 172.21.15.151:0/863386544 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb55804a060 con 0x7fb56015bd30 2024-09-16T15:42:20.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.900+0000 7fb55fe00640 1 -- 172.21.15.151:0/863386544 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fb5600cde20 con 0x7fb56015bd30 2024-09-16T15:42:20.957 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.956+0000 7fb556a00640 1 -- 172.21.15.151:0/863386544 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v42) ==== 97+0+0 (secure 0 0 0) 0x7fb558083e90 con 0x7fb56015bd30 2024-09-16T15:42:20.957 INFO:teuthology.orchestra.run.smithi151.stderr: removed all blocklist entries 2024-09-16T15:42:20.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.958+0000 7fb55fe00640 1 -- 172.21.15.151:0/863386544 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb540078bb0 msgr2=0x7fb54007b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:20.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.958+0000 7fb55fe00640 1 --2- 172.21.15.151:0/863386544 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb540078bb0 0x7fb54007b070 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb56006f8e0 tx=0x7fb550007e90 comp rx=0 tx=0).stop 2024-09-16T15:42:20.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.958+0000 7fb55fe00640 1 -- 172.21.15.151:0/863386544 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb56015bd30 msgr2=0x7fb5601f4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:20.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.958+0000 7fb55fe00640 1 --2- 172.21.15.151:0/863386544 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb56015bd30 0x7fb5601f4d80 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb558006010 tx=0x7fb558032d10 comp rx=0 tx=0).stop 2024-09-16T15:42:20.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.958+0000 7fb55fe00640 1 -- 172.21.15.151:0/863386544 shutdown_connections 2024-09-16T15:42:20.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.958+0000 7fb55fe00640 1 --2- 172.21.15.151:0/863386544 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb540078bb0 0x7fb54007b070 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:20.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.958+0000 7fb55fe00640 1 --2- 172.21.15.151:0/863386544 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb56015bd30 0x7fb5601f4d80 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:20.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.958+0000 7fb55fe00640 1 --2- 172.21.15.151:0/863386544 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5600d0670 0x7fb5600cb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:20.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.958+0000 7fb55fe00640 1 --2- 172.21.15.151:0/863386544 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5600b1680 0x7fb560071ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:20.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.958+0000 7fb55fe00640 1 -- 172.21.15.151:0/863386544 >> 172.21.15.151:0/863386544 conn(0x7fb5600d1c40 msgr2=0x7fb56015a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:20.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.958+0000 7fb55fe00640 1 -- 172.21.15.151:0/863386544 shutdown_connections 2024-09-16T15:42:20.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:20.959+0000 7fb55fe00640 1 -- 172.21.15.151:0/863386544 wait complete. 2024-09-16T15:42:20.978 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-16T15:42:21.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.137+0000 7f07b9800640 1 -- 172.21.15.151:0/1731482049 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f07b4073e00 msgr2=0x7f07b4168e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:21.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.137+0000 7f07b9800640 1 --2- 172.21.15.151:0/1731482049 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f07b4073e00 0x7f07b4168e10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f07ac009860 tx=0x7f07ac02a510 comp rx=0 tx=0).stop 2024-09-16T15:42:21.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.137+0000 7f07b9800640 1 -- 172.21.15.151:0/1731482049 shutdown_connections 2024-09-16T15:42:21.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.137+0000 7f07b9800640 1 --2- 172.21.15.151:0/1731482049 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f07b4073e00 0x7f07b4168e10 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:21.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.137+0000 7f07b9800640 1 -- 172.21.15.151:0/1731482049 >> 172.21.15.151:0/1731482049 conn(0x7f07b40b1950 msgr2=0x7f07b40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:21.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.138+0000 7f07b9800640 1 -- 172.21.15.151:0/1731482049 shutdown_connections 2024-09-16T15:42:21.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.138+0000 7f07b9800640 1 -- 172.21.15.151:0/1731482049 wait complete. 2024-09-16T15:42:21.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.139+0000 7f07b9800640 1 Processor -- start 2024-09-16T15:42:21.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.139+0000 7f07b9800640 1 -- start start 2024-09-16T15:42:21.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.140+0000 7f07b9800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07b4073e00 0x7f07b414f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:21.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.140+0000 7f07b9800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f07b40af3d0 0x7f07b415fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:21.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.140+0000 7f07b9800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f07b4161550 0x7f07b415ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:21.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.140+0000 7f07b9800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f07b41694d0 con 0x7f07b40af3d0 2024-09-16T15:42:21.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.140+0000 7f07b9800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f07b4169350 con 0x7f07b4073e00 2024-09-16T15:42:21.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.140+0000 7f07b9800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f07b4169650 con 0x7f07b4161550 2024-09-16T15:42:21.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.140+0000 7f07b2000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f07b40af3d0 0x7f07b415fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:21.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.140+0000 7f07b2000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f07b40af3d0 0x7f07b415fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:56370/0 (socket says 172.21.15.151:56370) 2024-09-16T15:42:21.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.140+0000 7f07b2000640 1 -- 172.21.15.151:0/116126286 learned_addr learned my addr 172.21.15.151:0/116126286 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:42:21.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.141+0000 7f07b2a00640 1 --2- 172.21.15.151:0/116126286 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07b4073e00 0x7f07b414f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:21.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.141+0000 7f07b3400640 1 --2- 172.21.15.151:0/116126286 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f07b4161550 0x7f07b415ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:21.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.141+0000 7f07b2000640 1 -- 172.21.15.151:0/116126286 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f07b4161550 msgr2=0x7f07b415ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:21.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.141+0000 7f07b2000640 1 --2- 172.21.15.151:0/116126286 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f07b4161550 0x7f07b415ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:21.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.141+0000 7f07b2000640 1 -- 172.21.15.151:0/116126286 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07b4073e00 msgr2=0x7f07b414f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:21.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.141+0000 7f07b2000640 1 --2- 172.21.15.151:0/116126286 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07b4073e00 0x7f07b414f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:21.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.141+0000 7f07b2000640 1 -- 172.21.15.151:0/116126286 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07b40cfbc0 con 0x7f07b40af3d0 2024-09-16T15:42:21.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.141+0000 7f07b3400640 1 --2- 172.21.15.151:0/116126286 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f07b4161550 0x7f07b415ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:42:21.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.141+0000 7f07b2000640 1 --2- 172.21.15.151:0/116126286 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f07b40af3d0 0x7f07b415fa80 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f07a000c4c0 tx=0x7f07a000c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:21.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.141+0000 7f07aaa00640 1 -- 172.21.15.151:0/116126286 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a00052d0 con 0x7f07b40af3d0 2024-09-16T15:42:21.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.141+0000 7f07aaa00640 1 -- 172.21.15.151:0/116126286 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f07a0005490 con 0x7f07b40af3d0 2024-09-16T15:42:21.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.141+0000 7f07aaa00640 1 -- 172.21.15.151:0/116126286 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a0012890 con 0x7f07b40af3d0 2024-09-16T15:42:21.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.141+0000 7f07b9800640 1 -- 172.21.15.151:0/116126286 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07b40cfea0 con 0x7f07b40af3d0 2024-09-16T15:42:21.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.142+0000 7f07b9800640 1 -- 172.21.15.151:0/116126286 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07b4156740 con 0x7f07b40af3d0 2024-09-16T15:42:21.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.143+0000 7f07b9800640 1 -- 172.21.15.151:0/116126286 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07b40745c0 con 0x7f07b40af3d0 2024-09-16T15:42:21.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.144+0000 7f07aaa00640 1 -- 172.21.15.151:0/116126286 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7f07a0012a30 con 0x7f07b40af3d0 2024-09-16T15:42:21.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.144+0000 7f07aaa00640 1 --2- 172.21.15.151:0/116126286 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0794078bb0 0x7f079407b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:21.146 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.145+0000 7f07aaa00640 1 -- 172.21.15.151:0/116126286 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 4857+0+0 (secure 0 0 0) 0x7f07a009a800 con 0x7f07b40af3d0 2024-09-16T15:42:21.146 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.145+0000 7f07b2a00640 1 --2- 172.21.15.151:0/116126286 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0794078bb0 0x7f079407b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:21.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.146+0000 7f07b2a00640 1 --2- 172.21.15.151:0/116126286 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0794078bb0 0x7f079407b070 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f07a40017a0 tx=0x7f07a4001730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:21.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.148+0000 7f07aaa00640 1 -- 172.21.15.151:0/116126286 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f07b40745c0 con 0x7f07b40af3d0 2024-09-16T15:42:21.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.403+0000 7f07b9800640 1 -- 172.21.15.151:0/116126286 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f07b4156940 con 0x7f07b40af3d0 2024-09-16T15:42:21.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.403+0000 7f07aaa00640 1 -- 172.21.15.151:0/116126286 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v9) ==== 80+0+5304 (secure 0 0 0) 0x7f07b4156940 con 0x7f07b40af3d0 2024-09-16T15:42:21.404 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:42:21.405 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-16T15:42:21.405 INFO:teuthology.orchestra.run.smithi151.stdout: "auth_dump": [ 2024-09-16T15:42:21.405 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:42:21.405 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "mds.a", 2024-09-16T15:42:21.405 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDsUOhmJVK6AhAANfmTPS6kKn41LSD/MuyW9g==", 2024-09-16T15:42:21.405 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:42:21.405 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:42:21.405 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow *", 2024-09-16T15:42:21.405 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow *", 2024-09-16T15:42:21.405 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:42:21.405 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:42:21.405 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:42:21.405 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:42:21.406 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "mds.b", 2024-09-16T15:42:21.406 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDsUOhmX1R3FhAAUQgUsE/6wUyokXcZuBuIWg==", 2024-09-16T15:42:21.406 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:42:21.406 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:42:21.406 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow *", 2024-09-16T15:42:21.406 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow *", 2024-09-16T15:42:21.406 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:42:21.406 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:42:21.406 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:42:21.406 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:42:21.406 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "mds.c", 2024-09-16T15:42:21.406 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDsUOhmYGCKDBAAY2EFNmiFNf1x2so9wxp4YQ==", 2024-09-16T15:42:21.406 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:42:21.407 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:42:21.407 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow *", 2024-09-16T15:42:21.407 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow *", 2024-09-16T15:42:21.407 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:42:21.407 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:42:21.407 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:42:21.407 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:42:21.407 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "mds.d", 2024-09-16T15:42:21.407 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDsUOhmfCk4IBAAx12Gw2QudQ2nrPb8VlE0SA==", 2024-09-16T15:42:21.407 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:42:21.407 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:42:21.407 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow *", 2024-09-16T15:42:21.407 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow *", 2024-09-16T15:42:21.407 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:42:21.408 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:42:21.408 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:42:21.408 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:42:21.408 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.0", 2024-09-16T15:42:21.408 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDvUOhmKji5NxAAn/E3YQR7jeMnBTLBXc1G6A==", 2024-09-16T15:42:21.408 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:42:21.408 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:42:21.408 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:42:21.408 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:42:21.408 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:42:21.408 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:42:21.408 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:42:21.408 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.1", 2024-09-16T15:42:21.408 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDzUOhmmBrNBhAA0Uwh4Fam2bmKKvz5jAQBWw==", 2024-09-16T15:42:21.408 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:42:21.409 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:42:21.409 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:42:21.409 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:42:21.409 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:42:21.409 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:42:21.409 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:42:21.409 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.2", 2024-09-16T15:42:21.409 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQD2UOhmOV6GERAAjD+PQLTZrCFR1669UEEIAQ==", 2024-09-16T15:42:21.409 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:42:21.409 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:42:21.409 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:42:21.409 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:42:21.409 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:42:21.410 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:42:21.410 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:42:21.410 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.3", 2024-09-16T15:42:21.410 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQD5UOhmCRrdGRAAQC0bqWX6N1jniVGEQ2aAvg==", 2024-09-16T15:42:21.410 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:42:21.410 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:42:21.410 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:42:21.410 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:42:21.410 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:42:21.410 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:42:21.410 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:42:21.410 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.4", 2024-09-16T15:42:21.410 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQD/UOhmXI/nGRAA7t6XveKvrWgqy6rHuMQSZQ==", 2024-09-16T15:42:21.410 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:42:21.410 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:42:21.410 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:42:21.411 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:42:21.411 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:42:21.411 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:42:21.411 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:42:21.411 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.5", 2024-09-16T15:42:21.411 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQACUehmKM//IxAAsoFpiiTXNs//v3YXeUf/Iw==", 2024-09-16T15:42:21.411 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:42:21.411 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:42:21.411 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:42:21.411 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:42:21.411 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:42:21.411 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:42:21.411 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:42:21.411 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.6", 2024-09-16T15:42:21.411 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQAFUehmlo/FLRAAp5mvUSWCaBbm85QyTZY0eA==", 2024-09-16T15:42:21.412 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:42:21.412 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:42:21.412 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:42:21.412 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:42:21.412 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:42:21.412 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:42:21.412 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:42:21.412 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.7", 2024-09-16T15:42:21.412 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQAIUehmR32JNxAA27antOIKot1nScQbLZDf5g==", 2024-09-16T15:42:21.412 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:42:21.412 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:42:21.412 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:42:21.412 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:42:21.412 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:42:21.412 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:42:21.412 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:42:21.413 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "client.0", 2024-09-16T15:42:21.413 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDsUOhmf5PBKBAAKE4Y4zbA4h07IBNjPkyptw==", 2024-09-16T15:42:21.413 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:42:21.413 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:42:21.413 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow r", 2024-09-16T15:42:21.413 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-16T15:42:21.413 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:42:21.413 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:42:21.413 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:42:21.413 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "client.1", 2024-09-16T15:42:21.413 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDsUOhm15emNRAAwhK7QmA2c9IYGn04ERvY0g==", 2024-09-16T15:42:21.413 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:42:21.413 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:42:21.413 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow r", 2024-09-16T15:42:21.413 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow rw", 2024-09-16T15:42:21.413 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow rwx" 2024-09-16T15:42:21.414 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:42:21.414 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:42:21.414 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:42:21.414 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "client.2", 2024-09-16T15:42:21.414 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDsUOhm6zUyLxAAkzeiIQTuNZkMM+Oit9rC5A==", 2024-09-16T15:42:21.414 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:42:21.414 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:42:21.414 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow r", 2024-09-16T15:42:21.414 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow rw", 2024-09-16T15:42:21.414 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow rwx" 2024-09-16T15:42:21.414 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:42:21.414 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:42:21.414 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:42:21.414 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "client.3", 2024-09-16T15:42:21.414 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDtUOhmD82EABAAeDf82/hPxpPfAUk8oj5h/Q==", 2024-09-16T15:42:21.415 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:42:21.415 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:42:21.415 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow r", 2024-09-16T15:42:21.415 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow rw", 2024-09-16T15:42:21.415 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow rwx" 2024-09-16T15:42:21.415 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:42:21.415 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:42:21.415 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:42:21.415 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "client.admin", 2024-09-16T15:42:21.415 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDrUOhmQFmVDRAA8NKysbAe475xvg4CPAcFyw==", 2024-09-16T15:42:21.415 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:42:21.415 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow *", 2024-09-16T15:42:21.415 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow *", 2024-09-16T15:42:21.415 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow *", 2024-09-16T15:42:21.415 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:42:21.415 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:42:21.416 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:42:21.416 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:42:21.416 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "mgr.x", 2024-09-16T15:42:21.416 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDrUOhmloW3NxAA8UoLUfrw3wf1nzKbmtK/jw==", 2024-09-16T15:42:21.416 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:42:21.416 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow *", 2024-09-16T15:42:21.416 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile mgr", 2024-09-16T15:42:21.416 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:42:21.416 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:42:21.416 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:42:21.416 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:42:21.416 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "mgr.y", 2024-09-16T15:42:21.416 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDrUOhm1uZOLxAATgjA79vYf0psqQVo03sdDA==", 2024-09-16T15:42:21.417 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:42:21.417 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow *", 2024-09-16T15:42:21.417 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile mgr", 2024-09-16T15:42:21.417 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:42:21.417 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:42:21.417 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:42:21.417 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-09-16T15:42:21.417 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-16T15:42:21.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.405+0000 7f07b9800640 1 -- 172.21.15.151:0/116126286 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0794078bb0 msgr2=0x7f079407b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:21.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.405+0000 7f07b9800640 1 --2- 172.21.15.151:0/116126286 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0794078bb0 0x7f079407b070 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f07a40017a0 tx=0x7f07a4001730 comp rx=0 tx=0).stop 2024-09-16T15:42:21.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.405+0000 7f07b9800640 1 -- 172.21.15.151:0/116126286 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f07b40af3d0 msgr2=0x7f07b415fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:21.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.405+0000 7f07b9800640 1 --2- 172.21.15.151:0/116126286 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f07b40af3d0 0x7f07b415fa80 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f07a000c4c0 tx=0x7f07a000c980 comp rx=0 tx=0).stop 2024-09-16T15:42:21.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.405+0000 7f07b9800640 1 -- 172.21.15.151:0/116126286 shutdown_connections 2024-09-16T15:42:21.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.405+0000 7f07b9800640 1 --2- 172.21.15.151:0/116126286 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0794078bb0 0x7f079407b070 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:21.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.405+0000 7f07b9800640 1 --2- 172.21.15.151:0/116126286 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f07b4161550 0x7f07b415ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:21.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.405+0000 7f07b9800640 1 --2- 172.21.15.151:0/116126286 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f07b40af3d0 0x7f07b415fa80 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:21.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.405+0000 7f07b9800640 1 --2- 172.21.15.151:0/116126286 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07b4073e00 0x7f07b414f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:21.419 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.405+0000 7f07b9800640 1 -- 172.21.15.151:0/116126286 >> 172.21.15.151:0/116126286 conn(0x7f07b40b1950 msgr2=0x7f07b406f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:21.419 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.406+0000 7f07b9800640 1 -- 172.21.15.151:0/116126286 shutdown_connections 2024-09-16T15:42:21.419 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.406+0000 7f07b9800640 1 -- 172.21.15.151:0/116126286 wait complete. 2024-09-16T15:42:21.422 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-16T15:42:21.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.578+0000 7f7d5d800640 1 -- 172.21.15.151:0/4255394846 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d58073fb0 msgr2=0x7f7d58074420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:21.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.578+0000 7f7d5d800640 1 --2- 172.21.15.151:0/4255394846 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d58073fb0 0x7f7d58074420 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7d58075130 tx=0x7f7d5002a980 comp rx=0 tx=0).stop 2024-09-16T15:42:21.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.578+0000 7f7d5d800640 1 -- 172.21.15.151:0/4255394846 shutdown_connections 2024-09-16T15:42:21.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.578+0000 7f7d5d800640 1 --2- 172.21.15.151:0/4255394846 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d58073fb0 0x7f7d58074420 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:21.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.578+0000 7f7d5d800640 1 --2- 172.21.15.151:0/4255394846 >> v2:172.21.15.167:3300/0 conn(0x7f7d5815f5a0 0x7f7d580b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:21.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.578+0000 7f7d5d800640 1 -- 172.21.15.151:0/4255394846 >> 172.21.15.151:0/4255394846 conn(0x7f7d580b1950 msgr2=0x7f7d580b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:21.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.578+0000 7f7d5d800640 1 -- 172.21.15.151:0/4255394846 shutdown_connections 2024-09-16T15:42:21.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.578+0000 7f7d5d800640 1 -- 172.21.15.151:0/4255394846 wait complete. 2024-09-16T15:42:21.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.580+0000 7f7d5d800640 1 Processor -- start 2024-09-16T15:42:21.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.580+0000 7f7d5d800640 1 -- start start 2024-09-16T15:42:21.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.581+0000 7f7d5d800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d58073fb0 0x7f7d580cb580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:21.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.581+0000 7f7d5d800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7d580ccf00 0x7f7d580c95f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:21.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.581+0000 7f7d5d800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d5815f5a0 0x7f7d58154840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:21.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.581+0000 7f7d5d800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7d5806f2e0 con 0x7f7d5815f5a0 2024-09-16T15:42:21.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.581+0000 7f7d5d800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7d5806f160 con 0x7f7d58073fb0 2024-09-16T15:42:21.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.581+0000 7f7d5d800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f7d5806f460 con 0x7f7d580ccf00 2024-09-16T15:42:21.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.581+0000 7f7d57400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d5815f5a0 0x7f7d58154840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:21.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.581+0000 7f7d57400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d5815f5a0 0x7f7d58154840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:56390/0 (socket says 172.21.15.151:56390) 2024-09-16T15:42:21.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.581+0000 7f7d57400640 1 -- 172.21.15.151:0/1013682335 learned_addr learned my addr 172.21.15.151:0/1013682335 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:42:21.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.581+0000 7f7d56a00640 1 --2- 172.21.15.151:0/1013682335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d58073fb0 0x7f7d580cb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:21.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.581+0000 7f7d56000640 1 --2- 172.21.15.151:0/1013682335 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7d580ccf00 0x7f7d580c95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:21.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.582+0000 7f7d57400640 1 -- 172.21.15.151:0/1013682335 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7d580ccf00 msgr2=0x7f7d580c95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:21.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.582+0000 7f7d57400640 1 --2- 172.21.15.151:0/1013682335 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7d580ccf00 0x7f7d580c95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:21.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.582+0000 7f7d57400640 1 -- 172.21.15.151:0/1013682335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d58073fb0 msgr2=0x7f7d580cb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:21.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.582+0000 7f7d57400640 1 --2- 172.21.15.151:0/1013682335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d58073fb0 0x7f7d580cb580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:21.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.582+0000 7f7d57400640 1 -- 172.21.15.151:0/1013682335 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d58156cb0 con 0x7f7d5815f5a0 2024-09-16T15:42:21.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.582+0000 7f7d57400640 1 --2- 172.21.15.151:0/1013682335 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d5815f5a0 0x7f7d58154840 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7d50009d60 tx=0x7f7d50003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:21.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.582+0000 7f7d4ea00640 1 -- 172.21.15.151:0/1013682335 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d50034050 con 0x7f7d5815f5a0 2024-09-16T15:42:21.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.582+0000 7f7d4ea00640 1 -- 172.21.15.151:0/1013682335 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7d50004450 con 0x7f7d5815f5a0 2024-09-16T15:42:21.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.582+0000 7f7d4ea00640 1 -- 172.21.15.151:0/1013682335 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d50043440 con 0x7f7d5815f5a0 2024-09-16T15:42:21.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.582+0000 7f7d5d800640 1 -- 172.21.15.151:0/1013682335 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d581561c0 con 0x7f7d5815f5a0 2024-09-16T15:42:21.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.582+0000 7f7d5d800640 1 -- 172.21.15.151:0/1013682335 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d580c9b30 con 0x7f7d5815f5a0 2024-09-16T15:42:21.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.583+0000 7f7d5d800640 1 -- 172.21.15.151:0/1013682335 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d581566e0 con 0x7f7d5815f5a0 2024-09-16T15:42:21.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.584+0000 7f7d4ea00640 1 -- 172.21.15.151:0/1013682335 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7f7d50037020 con 0x7f7d5815f5a0 2024-09-16T15:42:21.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.585+0000 7f7d4ea00640 1 --2- 172.21.15.151:0/1013682335 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7d38078bb0 0x7f7d3807b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:21.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.585+0000 7f7d4ea00640 1 -- 172.21.15.151:0/1013682335 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 4857+0+0 (secure 0 0 0) 0x7f7d500bbef0 con 0x7f7d5815f5a0 2024-09-16T15:42:21.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.586+0000 7f7d56a00640 1 --2- 172.21.15.151:0/1013682335 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7d38078bb0 0x7f7d3807b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:21.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.587+0000 7f7d56a00640 1 --2- 172.21.15.151:0/1013682335 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7d38078bb0 0x7f7d3807b070 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7d48004630 tx=0x7f7d4800b2e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:21.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.590+0000 7f7d4ea00640 1 -- 172.21.15.151:0/1013682335 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7d581566e0 con 0x7f7d5815f5a0 2024-09-16T15:42:21.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.776+0000 7f7d5d800640 1 -- 172.21.15.151:0/1013682335 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f7d580b3810 con 0x7f7d5815f5a0 2024-09-16T15:42:21.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.777+0000 7f7d4ea00640 1 -- 172.21.15.151:0/1013682335 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v44) ==== 84+0+65 (secure 0 0 0) 0x7f7d50083d10 con 0x7f7d5815f5a0 2024-09-16T15:42:21.778 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:42:21.778 INFO:teuthology.orchestra.run.smithi151.stdout:[ 2024-09-16T15:42:21.778 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:42:21.778 INFO:teuthology.orchestra.run.smithi151.stdout: "poolnum": 1, 2024-09-16T15:42:21.778 INFO:teuthology.orchestra.run.smithi151.stdout: "poolname": ".mgr" 2024-09-16T15:42:21.778 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:42:21.778 INFO:teuthology.orchestra.run.smithi151.stdout:] 2024-09-16T15:42:21.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.779+0000 7f7d5d800640 1 -- 172.21.15.151:0/1013682335 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7d38078bb0 msgr2=0x7f7d3807b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:21.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.779+0000 7f7d5d800640 1 --2- 172.21.15.151:0/1013682335 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7d38078bb0 0x7f7d3807b070 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7d48004630 tx=0x7f7d4800b2e0 comp rx=0 tx=0).stop 2024-09-16T15:42:21.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.779+0000 7f7d5d800640 1 -- 172.21.15.151:0/1013682335 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d5815f5a0 msgr2=0x7f7d58154840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:21.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.779+0000 7f7d5d800640 1 --2- 172.21.15.151:0/1013682335 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d5815f5a0 0x7f7d58154840 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7d50009d60 tx=0x7f7d50003f10 comp rx=0 tx=0).stop 2024-09-16T15:42:21.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.779+0000 7f7d5d800640 1 -- 172.21.15.151:0/1013682335 shutdown_connections 2024-09-16T15:42:21.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.779+0000 7f7d5d800640 1 --2- 172.21.15.151:0/1013682335 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7d38078bb0 0x7f7d3807b070 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:21.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.779+0000 7f7d5d800640 1 --2- 172.21.15.151:0/1013682335 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d5815f5a0 0x7f7d58154840 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:21.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.779+0000 7f7d5d800640 1 --2- 172.21.15.151:0/1013682335 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7d580ccf00 0x7f7d580c95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:21.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.779+0000 7f7d5d800640 1 --2- 172.21.15.151:0/1013682335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d58073fb0 0x7f7d580cb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:21.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.779+0000 7f7d5d800640 1 -- 172.21.15.151:0/1013682335 >> 172.21.15.151:0/1013682335 conn(0x7f7d580b1950 msgr2=0x7f7d580ce0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:21.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.779+0000 7f7d5d800640 1 -- 172.21.15.151:0/1013682335 shutdown_connections 2024-09-16T15:42:21.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.779+0000 7f7d5d800640 1 -- 172.21.15.151:0/1013682335 wait complete. 2024-09-16T15:42:21.796 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-16T15:42:21.796 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-16T15:42:21.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.958+0000 7fe758c00640 1 -- 172.21.15.151:0/191962811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe7540b32f0 msgr2=0x7fe7540cc9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:21.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.958+0000 7fe758c00640 1 --2- 172.21.15.151:0/191962811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe7540b32f0 0x7fe7540cc9e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe74c005fd0 tx=0x7fe74c02aa00 comp rx=0 tx=0).stop 2024-09-16T15:42:21.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.958+0000 7fe758c00640 1 -- 172.21.15.151:0/191962811 shutdown_connections 2024-09-16T15:42:21.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.958+0000 7fe758c00640 1 --2- 172.21.15.151:0/191962811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe7540b32f0 0x7fe7540cc9e0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:21.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.958+0000 7fe758c00640 1 --2- 172.21.15.151:0/191962811 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe754158900 0x7fe75415bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:21.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.958+0000 7fe758c00640 1 -- 172.21.15.151:0/191962811 >> 172.21.15.151:0/191962811 conn(0x7fe7540b18e0 msgr2=0x7fe7540b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:21.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.958+0000 7fe758c00640 1 -- 172.21.15.151:0/191962811 shutdown_connections 2024-09-16T15:42:21.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.958+0000 7fe758c00640 1 -- 172.21.15.151:0/191962811 wait complete. 2024-09-16T15:42:21.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.960+0000 7fe758c00640 1 Processor -- start 2024-09-16T15:42:21.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.960+0000 7fe758c00640 1 -- start start 2024-09-16T15:42:21.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.961+0000 7fe758c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe75406e9d0 0x7fe754153560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:21.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.961+0000 7fe758c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe7540b32f0 0x7fe7540cb200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:21.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.961+0000 7fe758c00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe754158900 0x7fe7540cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:21.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.961+0000 7fe758c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe75406efb0 con 0x7fe7540b32f0 2024-09-16T15:42:21.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.961+0000 7fe758c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe75406ee30 con 0x7fe75406e9d0 2024-09-16T15:42:21.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.961+0000 7fe758c00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fe7540ccf20 con 0x7fe754158900 2024-09-16T15:42:21.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.961+0000 7fe751400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe7540b32f0 0x7fe7540cb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:21.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.961+0000 7fe751e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe75406e9d0 0x7fe754153560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:21.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.961+0000 7fe751e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe75406e9d0 0x7fe754153560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.151:59524/0 (socket says 172.21.15.151:59524) 2024-09-16T15:42:21.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.961+0000 7fe751e00640 1 -- 172.21.15.151:0/260892076 learned_addr learned my addr 172.21.15.151:0/260892076 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:42:21.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.961+0000 7fe752800640 1 --2- 172.21.15.151:0/260892076 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe754158900 0x7fe7540cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:21.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.962+0000 7fe751400640 1 -- 172.21.15.151:0/260892076 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe754158900 msgr2=0x7fe7540cdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:21.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.962+0000 7fe751400640 1 --2- 172.21.15.151:0/260892076 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe754158900 0x7fe7540cdad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:21.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.962+0000 7fe751400640 1 -- 172.21.15.151:0/260892076 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe75406e9d0 msgr2=0x7fe754153560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:21.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.962+0000 7fe751400640 1 --2- 172.21.15.151:0/260892076 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe75406e9d0 0x7fe754153560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:21.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.962+0000 7fe751400640 1 -- 172.21.15.151:0/260892076 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe75409c870 con 0x7fe7540b32f0 2024-09-16T15:42:21.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.962+0000 7fe751400640 1 --2- 172.21.15.151:0/260892076 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe7540b32f0 0x7fe7540cb200 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe74000c3c0 tx=0x7fe74000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:21.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.962+0000 7fe74a000640 1 -- 172.21.15.151:0/260892076 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe740017070 con 0x7fe7540b32f0 2024-09-16T15:42:21.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.962+0000 7fe74a000640 1 -- 172.21.15.151:0/260892076 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe7400053c0 con 0x7fe7540b32f0 2024-09-16T15:42:21.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.962+0000 7fe74a000640 1 -- 172.21.15.151:0/260892076 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe740003850 con 0x7fe7540b32f0 2024-09-16T15:42:21.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.962+0000 7fe758c00640 1 -- 172.21.15.151:0/260892076 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe75409cb50 con 0x7fe7540b32f0 2024-09-16T15:42:21.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.962+0000 7fe758c00640 1 -- 172.21.15.151:0/260892076 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe75409d110 con 0x7fe7540b32f0 2024-09-16T15:42:21.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.964+0000 7fe758c00640 1 -- 172.21.15.151:0/260892076 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe75409d310 con 0x7fe7540b32f0 2024-09-16T15:42:21.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.965+0000 7fe74a000640 1 -- 172.21.15.151:0/260892076 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7fe74001a070 con 0x7fe7540b32f0 2024-09-16T15:42:21.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.966+0000 7fe74a000640 1 --2- 172.21.15.151:0/260892076 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe734078bb0 0x7fe73407b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:21.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.966+0000 7fe74a000640 1 -- 172.21.15.151:0/260892076 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 4857+0+0 (secure 0 0 0) 0x7fe740099e40 con 0x7fe7540b32f0 2024-09-16T15:42:21.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.966+0000 7fe751e00640 1 --2- 172.21.15.151:0/260892076 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe734078bb0 0x7fe73407b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:21.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.967+0000 7fe751e00640 1 --2- 172.21.15.151:0/260892076 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe734078bb0 0x7fe73407b070 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe7541f8430 tx=0x7fe744005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:21.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:21.970+0000 7fe74a000640 1 -- 172.21.15.151:0/260892076 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe740061c60 con 0x7fe7540b32f0 2024-09-16T15:42:22.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:22.156+0000 7fe758c00640 1 -- 172.21.15.151:0/260892076 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fe754071150 con 0x7fe7540b32f0 2024-09-16T15:42:22.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:22.974+0000 7fe74a000640 1 -- 172.21.15.151:0/260892076 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v45) ==== 138+0+0 (secure 0 0 0) 0x7fe740066690 con 0x7fe7540b32f0 2024-09-16T15:42:22.976 INFO:teuthology.orchestra.run.smithi151.stderr:pool 'cephfs_metadata' created 2024-09-16T15:42:22.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:22.978+0000 7fe758c00640 1 -- 172.21.15.151:0/260892076 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe734078bb0 msgr2=0x7fe73407b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:22.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:22.978+0000 7fe758c00640 1 --2- 172.21.15.151:0/260892076 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe734078bb0 0x7fe73407b070 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe7541f8430 tx=0x7fe744005f90 comp rx=0 tx=0).stop 2024-09-16T15:42:22.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:22.978+0000 7fe758c00640 1 -- 172.21.15.151:0/260892076 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe7540b32f0 msgr2=0x7fe7540cb200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:22.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:22.978+0000 7fe758c00640 1 --2- 172.21.15.151:0/260892076 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe7540b32f0 0x7fe7540cb200 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe74000c3c0 tx=0x7fe74000c880 comp rx=0 tx=0).stop 2024-09-16T15:42:22.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:22.979+0000 7fe758c00640 1 -- 172.21.15.151:0/260892076 shutdown_connections 2024-09-16T15:42:22.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:22.979+0000 7fe758c00640 1 --2- 172.21.15.151:0/260892076 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe734078bb0 0x7fe73407b070 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:22.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:22.979+0000 7fe758c00640 1 --2- 172.21.15.151:0/260892076 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe754158900 0x7fe7540cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:22.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:22.979+0000 7fe758c00640 1 --2- 172.21.15.151:0/260892076 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe7540b32f0 0x7fe7540cb200 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:22.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:22.979+0000 7fe758c00640 1 --2- 172.21.15.151:0/260892076 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe75406e9d0 0x7fe754153560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:22.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:22.979+0000 7fe758c00640 1 -- 172.21.15.151:0/260892076 >> 172.21.15.151:0/260892076 conn(0x7fe7540b18e0 msgr2=0x7fe7540af8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:22.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:22.979+0000 7fe758c00640 1 -- 172.21.15.151:0/260892076 shutdown_connections 2024-09-16T15:42:22.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:22.979+0000 7fe758c00640 1 -- 172.21.15.151:0/260892076 wait complete. 2024-09-16T15:42:23.003 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-16T15:42:23.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:23.170+0000 7f1d7e200640 1 -- 172.21.15.151:0/1856265037 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d780cccd0 msgr2=0x7f1d780cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:23.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:23.170+0000 7f1d7e200640 1 --2- 172.21.15.151:0/1856265037 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d780cccd0 0x7f1d780cdbb0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f1d780d0cd0 tx=0x7f1d7002a4b0 comp rx=0 tx=0).stop 2024-09-16T15:42:23.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:23.170+0000 7f1d7e200640 1 -- 172.21.15.151:0/1856265037 shutdown_connections 2024-09-16T15:42:23.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:23.170+0000 7f1d7e200640 1 --2- 172.21.15.151:0/1856265037 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d780cccd0 0x7f1d780cdbb0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:23.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:23.170+0000 7f1d7e200640 1 --2- 172.21.15.151:0/1856265037 >> v2:172.21.15.167:3300/0 conn(0x7f1d78145750 0x7f1d78073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:23.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:23.171+0000 7f1d7e200640 1 -- 172.21.15.151:0/1856265037 >> 172.21.15.151:0/1856265037 conn(0x7f1d780b1840 msgr2=0x7f1d780b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:23.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:23.171+0000 7f1d7e200640 1 -- 172.21.15.151:0/1856265037 shutdown_connections 2024-09-16T15:42:23.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:23.171+0000 7f1d7e200640 1 -- 172.21.15.151:0/1856265037 wait complete. 2024-09-16T15:42:23.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:23.172+0000 7f1d7e200640 1 Processor -- start 2024-09-16T15:42:23.174 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:23.172+0000 7f1d7e200640 1 -- start start 2024-09-16T15:42:23.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:23.173+0000 7f1d7e200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1d7806efa0 0x7f1d781f7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:23.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:23.173+0000 7f1d7e200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d780cccd0 0x7f1d781423a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:23.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:23.173+0000 7f1d7e200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d78145750 0x7f1d781428e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:23.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:23.173+0000 7f1d7e200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1d780ce480 con 0x7f1d78145750 2024-09-16T15:42:23.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:23.173+0000 7f1d7e200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1d780ce300 con 0x7f1d780cccd0 2024-09-16T15:42:23.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:23.173+0000 7f1d7e200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f1d780ce600 con 0x7f1d7806efa0 2024-09-16T15:42:23.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:23.174+0000 7f1d77e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d78145750 0x7f1d781428e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:23.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:23.174+0000 7f1d77e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d78145750 0x7f1d781428e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49800/0 (socket says 172.21.15.151:49800) 2024-09-16T15:42:23.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:23.174+0000 7f1d77e00640 1 -- 172.21.15.151:0/3829577020 learned_addr learned my addr 172.21.15.151:0/3829577020 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:42:23.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:23.174+0000 7f1d77400640 1 --2- 172.21.15.151:0/3829577020 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1d7806efa0 0x7f1d781f7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:23.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:23.174+0000 7f1d76a00640 1 --2- 172.21.15.151:0/3829577020 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d780cccd0 0x7f1d781423a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:23.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:23.174+0000 7f1d77e00640 1 -- 172.21.15.151:0/3829577020 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1d7806efa0 msgr2=0x7f1d781f7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:23.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:23.174+0000 7f1d77e00640 1 --2- 172.21.15.151:0/3829577020 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1d7806efa0 0x7f1d781f7360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:23.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:23.174+0000 7f1d77e00640 1 -- 172.21.15.151:0/3829577020 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d780cccd0 msgr2=0x7f1d781423a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:23.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:23.174+0000 7f1d77e00640 1 --2- 172.21.15.151:0/3829577020 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d780cccd0 0x7f1d781423a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:23.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:23.174+0000 7f1d77e00640 1 -- 172.21.15.151:0/3829577020 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d7813dce0 con 0x7f1d78145750 2024-09-16T15:42:23.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:23.174+0000 7f1d77400640 1 --2- 172.21.15.151:0/3829577020 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1d7806efa0 0x7f1d781f7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:42:23.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:23.175+0000 7f1d77e00640 1 --2- 172.21.15.151:0/3829577020 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d78145750 0x7f1d781428e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f1d7002aaa0 tx=0x7f1d70032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:23.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:23.175+0000 7f1d6f400640 1 -- 172.21.15.151:0/3829577020 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d70040070 con 0x7f1d78145750 2024-09-16T15:42:23.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:23.175+0000 7f1d6f400640 1 -- 172.21.15.151:0/3829577020 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1d70005310 con 0x7f1d78145750 2024-09-16T15:42:23.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:23.175+0000 7f1d7e200640 1 -- 172.21.15.151:0/3829577020 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d7813dfc0 con 0x7f1d78145750 2024-09-16T15:42:23.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:23.175+0000 7f1d6f400640 1 -- 172.21.15.151:0/3829577020 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d70005c90 con 0x7f1d78145750 2024-09-16T15:42:23.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:23.175+0000 7f1d7e200640 1 -- 172.21.15.151:0/3829577020 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d7813e580 con 0x7f1d78145750 2024-09-16T15:42:23.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:23.176+0000 7f1d7e200640 1 -- 172.21.15.151:0/3829577020 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d780cdbb0 con 0x7f1d78145750 2024-09-16T15:42:23.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:23.177+0000 7f1d6f400640 1 -- 172.21.15.151:0/3829577020 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7f1d70039040 con 0x7f1d78145750 2024-09-16T15:42:23.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:23.178+0000 7f1d6f400640 1 --2- 172.21.15.151:0/3829577020 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1d58078bb0 0x7f1d5807b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:23.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:23.178+0000 7f1d6f400640 1 -- 172.21.15.151:0/3829577020 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5227+0+0 (secure 0 0 0) 0x7f1d700bc150 con 0x7f1d78145750 2024-09-16T15:42:23.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:23.179+0000 7f1d77400640 1 --2- 172.21.15.151:0/3829577020 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1d58078bb0 0x7f1d5807b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:23.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:23.179+0000 7f1d77400640 1 --2- 172.21.15.151:0/3829577020 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1d58078bb0 0x7f1d5807b070 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1d68005fd0 tx=0x7f1d68009500 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:23.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:23.182+0000 7f1d6f400640 1 -- 172.21.15.151:0/3829577020 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1d70084550 con 0x7f1d78145750 2024-09-16T15:42:23.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:23.370+0000 7f1d7e200640 1 -- 172.21.15.151:0/3829577020 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f1d780b2330 con 0x7f1d78145750 2024-09-16T15:42:23.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:23.980+0000 7f1d6f400640 1 -- 172.21.15.151:0/3829577020 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v46) ==== 170+0+0 (secure 0 0 0) 0x7f1d70083e50 con 0x7f1d78145750 2024-09-16T15:42:23.982 INFO:teuthology.orchestra.run.smithi151.stderr:pool 'cephfs_data' created 2024-09-16T15:42:23.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:23.983+0000 7f1d7e200640 1 -- 172.21.15.151:0/3829577020 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1d58078bb0 msgr2=0x7f1d5807b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:23.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:23.983+0000 7f1d7e200640 1 --2- 172.21.15.151:0/3829577020 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1d58078bb0 0x7f1d5807b070 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1d68005fd0 tx=0x7f1d68009500 comp rx=0 tx=0).stop 2024-09-16T15:42:23.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:23.983+0000 7f1d7e200640 1 -- 172.21.15.151:0/3829577020 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d78145750 msgr2=0x7f1d781428e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:23.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:23.983+0000 7f1d7e200640 1 --2- 172.21.15.151:0/3829577020 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d78145750 0x7f1d781428e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f1d7002aaa0 tx=0x7f1d70032c10 comp rx=0 tx=0).stop 2024-09-16T15:42:23.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:23.983+0000 7f1d7e200640 1 -- 172.21.15.151:0/3829577020 shutdown_connections 2024-09-16T15:42:23.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:23.983+0000 7f1d7e200640 1 --2- 172.21.15.151:0/3829577020 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1d58078bb0 0x7f1d5807b070 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:23.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:23.983+0000 7f1d7e200640 1 --2- 172.21.15.151:0/3829577020 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d780cccd0 0x7f1d781423a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:23.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:23.983+0000 7f1d7e200640 1 --2- 172.21.15.151:0/3829577020 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d78145750 0x7f1d781428e0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:23.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:23.983+0000 7f1d7e200640 1 --2- 172.21.15.151:0/3829577020 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1d7806efa0 0x7f1d781f7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:23.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:23.983+0000 7f1d7e200640 1 -- 172.21.15.151:0/3829577020 >> 172.21.15.151:0/3829577020 conn(0x7f1d780b1840 msgr2=0x7f1d780b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:23.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:23.984+0000 7f1d7e200640 1 -- 172.21.15.151:0/3829577020 shutdown_connections 2024-09-16T15:42:23.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:23.986+0000 7f1d7e200640 1 -- 172.21.15.151:0/3829577020 wait complete. 2024-09-16T15:42:24.019 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-16T15:42:24.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:24.179+0000 7f9cdfe00640 1 -- 172.21.15.151:0/3405156769 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9cd80cccd0 msgr2=0x7f9cd8073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:24.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:24.179+0000 7f9cdfe00640 1 --2- 172.21.15.151:0/3405156769 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9cd80cccd0 0x7f9cd8073440 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f9cc4005fd0 tx=0x7f9cc402a540 comp rx=0 tx=0).stop 2024-09-16T15:42:24.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:24.180+0000 7f9cdfe00640 1 -- 172.21.15.151:0/3405156769 shutdown_connections 2024-09-16T15:42:24.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:24.180+0000 7f9cdfe00640 1 --2- 172.21.15.151:0/3405156769 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9cd806efa0 0x7f9cd80cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:24.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:24.180+0000 7f9cdfe00640 1 --2- 172.21.15.151:0/3405156769 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9cd80cccd0 0x7f9cd8073440 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:24.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:24.180+0000 7f9cdfe00640 1 -- 172.21.15.151:0/3405156769 >> 172.21.15.151:0/3405156769 conn(0x7f9cd80b1840 msgr2=0x7f9cd80b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:24.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:24.180+0000 7f9cdfe00640 1 -- 172.21.15.151:0/3405156769 shutdown_connections 2024-09-16T15:42:24.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:24.180+0000 7f9cdfe00640 1 -- 172.21.15.151:0/3405156769 wait complete. 2024-09-16T15:42:24.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:24.181+0000 7f9cdfe00640 1 Processor -- start 2024-09-16T15:42:24.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:24.181+0000 7f9cdfe00640 1 -- start start 2024-09-16T15:42:24.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:24.182+0000 7f9cdfe00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9cd806efa0 0x7f9cd813bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:24.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:24.182+0000 7f9cdfe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cd80cccd0 0x7f9cd8152490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:24.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:24.182+0000 7f9cdfe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9cd8145750 0x7f9cd81529d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:24.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:24.182+0000 7f9cdfe00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd80ce480 con 0x7f9cd8145750 2024-09-16T15:42:24.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:24.182+0000 7f9cdfe00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd80ce300 con 0x7f9cd80cccd0 2024-09-16T15:42:24.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:24.182+0000 7f9cdfe00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f9cd80ce600 con 0x7f9cd806efa0 2024-09-16T15:42:24.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:24.182+0000 7f9cdde00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9cd8145750 0x7f9cd81529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:24.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:24.182+0000 7f9cdde00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9cd8145750 0x7f9cd81529d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49818/0 (socket says 172.21.15.151:49818) 2024-09-16T15:42:24.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:24.182+0000 7f9cdde00640 1 -- 172.21.15.151:0/3216027415 learned_addr learned my addr 172.21.15.151:0/3216027415 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:42:24.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:24.183+0000 7f9cdde00640 1 -- 172.21.15.151:0/3216027415 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9cd806efa0 msgr2=0x7f9cd813bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:24.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:24.183+0000 7f9cdca00640 1 --2- 172.21.15.151:0/3216027415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cd80cccd0 0x7f9cd8152490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:24.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:24.183+0000 7f9cdd400640 1 --2- 172.21.15.151:0/3216027415 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9cd806efa0 0x7f9cd813bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:24.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:24.183+0000 7f9cdde00640 1 --2- 172.21.15.151:0/3216027415 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9cd806efa0 0x7f9cd813bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:24.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:24.183+0000 7f9cdde00640 1 -- 172.21.15.151:0/3216027415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cd80cccd0 msgr2=0x7f9cd8152490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:24.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:24.183+0000 7f9cdde00640 1 --2- 172.21.15.151:0/3216027415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cd80cccd0 0x7f9cd8152490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:24.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:24.183+0000 7f9cdde00640 1 -- 172.21.15.151:0/3216027415 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cd813db00 con 0x7f9cd8145750 2024-09-16T15:42:24.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:24.183+0000 7f9cdd400640 1 --2- 172.21.15.151:0/3216027415 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9cd806efa0 0x7f9cd813bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:42:24.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:24.183+0000 7f9cdca00640 1 --2- 172.21.15.151:0/3216027415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cd80cccd0 0x7f9cd8152490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:42:24.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:24.183+0000 7f9cdde00640 1 --2- 172.21.15.151:0/3216027415 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9cd8145750 0x7f9cd81529d0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f9cd000d750 tx=0x7f9cd000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:24.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:24.183+0000 7f9cd5600640 1 -- 172.21.15.151:0/3216027415 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd0017070 con 0x7f9cd8145750 2024-09-16T15:42:24.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:24.183+0000 7f9cd5600640 1 -- 172.21.15.151:0/3216027415 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9cd0009cc0 con 0x7f9cd8145750 2024-09-16T15:42:24.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:24.183+0000 7f9cd5600640 1 -- 172.21.15.151:0/3216027415 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd000de60 con 0x7f9cd8145750 2024-09-16T15:42:24.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:24.184+0000 7f9cdfe00640 1 -- 172.21.15.151:0/3216027415 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cd813dde0 con 0x7f9cd8145750 2024-09-16T15:42:24.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:24.184+0000 7f9cdfe00640 1 -- 172.21.15.151:0/3216027415 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cd813e3a0 con 0x7f9cd8145750 2024-09-16T15:42:24.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:24.185+0000 7f9cdfe00640 1 -- 172.21.15.151:0/3216027415 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cd80cdbb0 con 0x7f9cd8145750 2024-09-16T15:42:24.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:24.186+0000 7f9cd5600640 1 -- 172.21.15.151:0/3216027415 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7f9cd0005280 con 0x7f9cd8145750 2024-09-16T15:42:24.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:24.187+0000 7f9cd5600640 1 --2- 172.21.15.151:0/3216027415 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9cb8078bb0 0x7f9cb807b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:24.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:24.187+0000 7f9cd5600640 1 -- 172.21.15.151:0/3216027415 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5609+0+0 (secure 0 0 0) 0x7f9cd009b0f0 con 0x7f9cd8145750 2024-09-16T15:42:24.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:24.187+0000 7f9cdd400640 1 --2- 172.21.15.151:0/3216027415 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9cb8078bb0 0x7f9cb807b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:24.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:24.188+0000 7f9cdd400640 1 --2- 172.21.15.151:0/3216027415 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9cb8078bb0 0x7f9cb807b070 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc001d10 tx=0x7f9ccc009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:24.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:24.192+0000 7f9cd5600640 1 -- 172.21.15.151:0/3216027415 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9cd00631f0 con 0x7f9cd8145750 2024-09-16T15:42:24.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:24.410+0000 7f9cdfe00640 1 -- 172.21.15.151:0/3216027415 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f9cd8143940 con 0x7f9cd8145750 2024-09-16T15:42:24.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:24.983+0000 7f9cd5600640 1 -- 172.21.15.151:0/3216027415 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v23) ==== 172+0+0 (secure 0 0 0) 0x7f9cd0062af0 con 0x7f9cd8145750 2024-09-16T15:42:24.984 INFO:teuthology.orchestra.run.smithi151.stderr:new fs with metadata pool 6 and data pool 7 2024-09-16T15:42:24.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:24.985+0000 7f9cdfe00640 1 -- 172.21.15.151:0/3216027415 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9cb8078bb0 msgr2=0x7f9cb807b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:24.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:24.985+0000 7f9cdfe00640 1 --2- 172.21.15.151:0/3216027415 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9cb8078bb0 0x7f9cb807b070 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc001d10 tx=0x7f9ccc009040 comp rx=0 tx=0).stop 2024-09-16T15:42:24.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:24.985+0000 7f9cdfe00640 1 -- 172.21.15.151:0/3216027415 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9cd8145750 msgr2=0x7f9cd81529d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:24.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:24.985+0000 7f9cdfe00640 1 --2- 172.21.15.151:0/3216027415 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9cd8145750 0x7f9cd81529d0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f9cd000d750 tx=0x7f9cd000dc10 comp rx=0 tx=0).stop 2024-09-16T15:42:24.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:24.986+0000 7f9cdfe00640 1 -- 172.21.15.151:0/3216027415 shutdown_connections 2024-09-16T15:42:24.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:24.986+0000 7f9cdfe00640 1 --2- 172.21.15.151:0/3216027415 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9cb8078bb0 0x7f9cb807b070 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:24.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:24.986+0000 7f9cdfe00640 1 --2- 172.21.15.151:0/3216027415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cd80cccd0 0x7f9cd8152490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:24.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:24.986+0000 7f9cdfe00640 1 --2- 172.21.15.151:0/3216027415 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9cd8145750 0x7f9cd81529d0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:24.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:24.986+0000 7f9cdfe00640 1 --2- 172.21.15.151:0/3216027415 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9cd806efa0 0x7f9cd813bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:24.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:24.986+0000 7f9cdfe00640 1 -- 172.21.15.151:0/3216027415 >> 172.21.15.151:0/3216027415 conn(0x7f9cd80b1840 msgr2=0x7f9cd80a94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:24.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:24.986+0000 7f9cdfe00640 1 -- 172.21.15.151:0/3216027415 shutdown_connections 2024-09-16T15:42:24.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:24.987+0000 7f9cdfe00640 1 -- 172.21.15.151:0/3216027415 wait complete. 2024-09-16T15:42:25.021 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:42:25.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.182+0000 7fdacac00640 1 -- 172.21.15.151:0/4233780760 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdac415f5a0 msgr2=0x7fdac40b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:25.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.182+0000 7fdacac00640 1 --2- 172.21.15.151:0/4233780760 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdac415f5a0 0x7fdac40b2440 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fdab4009830 tx=0x7fdab402a480 comp rx=0 tx=0).stop 2024-09-16T15:42:25.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.182+0000 7fdacac00640 1 -- 172.21.15.151:0/4233780760 shutdown_connections 2024-09-16T15:42:25.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.182+0000 7fdacac00640 1 --2- 172.21.15.151:0/4233780760 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdac40ccf00 0x7fdac40acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:25.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.182+0000 7fdacac00640 1 --2- 172.21.15.151:0/4233780760 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdac415f5a0 0x7fdac40b2440 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:25.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.182+0000 7fdacac00640 1 -- 172.21.15.151:0/4233780760 >> 172.21.15.151:0/4233780760 conn(0x7fdac40b1950 msgr2=0x7fdac40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:25.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.182+0000 7fdacac00640 1 -- 172.21.15.151:0/4233780760 shutdown_connections 2024-09-16T15:42:25.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.182+0000 7fdacac00640 1 -- 172.21.15.151:0/4233780760 wait complete. 2024-09-16T15:42:25.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.184+0000 7fdacac00640 1 Processor -- start 2024-09-16T15:42:25.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.184+0000 7fdacac00640 1 -- start start 2024-09-16T15:42:25.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.185+0000 7fdacac00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdac40ad270 0x7fdac40cf980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:25.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.185+0000 7fdacac00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdac40ccf00 0x7fdac4150320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:25.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.185+0000 7fdacac00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdac415f5a0 0x7fdac4150aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:25.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.185+0000 7fdacac00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdac406f2e0 con 0x7fdac40ccf00 2024-09-16T15:42:25.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.185+0000 7fdacac00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdac406f160 con 0x7fdac415f5a0 2024-09-16T15:42:25.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.185+0000 7fdacac00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fdac406f460 con 0x7fdac40ad270 2024-09-16T15:42:25.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.185+0000 7fdac3400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdac40ccf00 0x7fdac4150320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:25.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.185+0000 7fdac3400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdac40ccf00 0x7fdac4150320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49832/0 (socket says 172.21.15.151:49832) 2024-09-16T15:42:25.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.185+0000 7fdac3400640 1 -- 172.21.15.151:0/4238478580 learned_addr learned my addr 172.21.15.151:0/4238478580 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:42:25.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.185+0000 7fdac3e00640 1 --2- 172.21.15.151:0/4238478580 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdac40ad270 0x7fdac40cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:25.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.185+0000 7fdac8c00640 1 --2- 172.21.15.151:0/4238478580 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdac415f5a0 0x7fdac4150aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:25.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.185+0000 7fdac3400640 1 -- 172.21.15.151:0/4238478580 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdac40ad270 msgr2=0x7fdac40cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:25.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.185+0000 7fdac3400640 1 --2- 172.21.15.151:0/4238478580 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdac40ad270 0x7fdac40cf980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:25.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.185+0000 7fdac3400640 1 -- 172.21.15.151:0/4238478580 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdac415f5a0 msgr2=0x7fdac4150aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:25.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.185+0000 7fdac3400640 1 --2- 172.21.15.151:0/4238478580 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdac415f5a0 0x7fdac4150aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:25.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.185+0000 7fdac3400640 1 -- 172.21.15.151:0/4238478580 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdac40cffa0 con 0x7fdac40ccf00 2024-09-16T15:42:25.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.186+0000 7fdac3400640 1 --2- 172.21.15.151:0/4238478580 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdac40ccf00 0x7fdac4150320 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fdab000d950 tx=0x7fdab000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:25.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.186+0000 7fdabbe00640 1 -- 172.21.15.151:0/4238478580 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab0017070 con 0x7fdac40ccf00 2024-09-16T15:42:25.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.186+0000 7fdabbe00640 1 -- 172.21.15.151:0/4238478580 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdab0009d20 con 0x7fdac40ccf00 2024-09-16T15:42:25.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.186+0000 7fdabbe00640 1 -- 172.21.15.151:0/4238478580 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab00106c0 con 0x7fdac40ccf00 2024-09-16T15:42:25.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.186+0000 7fdacac00640 1 -- 172.21.15.151:0/4238478580 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdac40d0280 con 0x7fdac40ccf00 2024-09-16T15:42:25.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.186+0000 7fdacac00640 1 -- 172.21.15.151:0/4238478580 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdac40d0840 con 0x7fdac40ccf00 2024-09-16T15:42:25.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.187+0000 7fdacac00640 1 -- 172.21.15.151:0/4238478580 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdac406f9f0 con 0x7fdac40ccf00 2024-09-16T15:42:25.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.188+0000 7fdabbe00640 1 -- 172.21.15.151:0/4238478580 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7fdab0010860 con 0x7fdac40ccf00 2024-09-16T15:42:25.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.189+0000 7fdabbe00640 1 --2- 172.21.15.151:0/4238478580 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdaa4078bb0 0x7fdaa407b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:25.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.189+0000 7fdabbe00640 1 -- 172.21.15.151:0/4238478580 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7fdab009b460 con 0x7fdac40ccf00 2024-09-16T15:42:25.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.190+0000 7fdac3e00640 1 --2- 172.21.15.151:0/4238478580 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdaa4078bb0 0x7fdaa407b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:25.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.190+0000 7fdac3e00640 1 --2- 172.21.15.151:0/4238478580 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdaa4078bb0 0x7fdaa407b070 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fdab4034000 tx=0x7fdab4004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:25.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.193+0000 7fdabbe00640 1 -- 172.21.15.151:0/4238478580 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fdab0062e20 con 0x7fdac40ccf00 2024-09-16T15:42:25.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.380+0000 7fdacac00640 1 -- 172.21.15.151:0/4238478580 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdac40b1710 con 0x7fdac40ccf00 2024-09-16T15:42:25.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.381+0000 7fdabbe00640 1 -- 172.21.15.151:0/4238478580 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) ==== 74+0+16742 (secure 0 0 0) 0x7fdab0067850 con 0x7fdac40ccf00 2024-09-16T15:42:25.383 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:42:25.383 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":47,"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","created":"2024-09-16T15:38:56.348535+0000","modified":"2024-09-16T15:42:24.411607+0000","last_up_change":"2024-09-16T15:39:10.681384+0000","last_in_change":"2024-09-16T15:39:00.544477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:39:11.602252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:42:22.166447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-16T15:42:23.379899+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5524076-a903-4d0a-af6a-caf474c4765d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6809","nonce":3111342741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6811","nonce":3111342741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6815","nonce":3111342741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6813","nonce":3111342741}]},"public_addr":"172.21.15.151:6809/3111342741","cluster_addr":"172.21.15.151:6811/3111342741","heartbeat_back_addr":"172.21.15.151:6815/3111342741","heartbeat_front_addr":"172.21.15.151:6813/3111342741","state":["exists","up"]},{"osd":1,"uuid":"dd25925f-0ae1-494e-8529-4dc205759db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6801","nonce":3368304087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6803","nonce":3368304087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6807","nonce":3368304087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6805","nonce":3368304087}]},"public_addr":"172.21.15.151:6801/3368304087","cluster_addr":"172.21.15.151:6803/3368304087","heartbeat_back_addr":"172.21.15.151:6807/3368304087","heartbeat_front_addr":"172.21.15.151:6805/3368304087","state":["exists","up"]},{"osd":2,"uuid":"c2f18115-013d-4da6-a0fe-cb978f4494b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6823","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6825","nonce":3079373271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6827","nonce":3079373271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6831","nonce":3079373271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6829","nonce":3079373271}]},"public_addr":"172.21.15.151:6825/3079373271","cluster_addr":"172.21.15.151:6827/3079373271","heartbeat_back_addr":"172.21.15.151:6831/3079373271","heartbeat_front_addr":"172.21.15.151:6829/3079373271","state":["exists","up"]},{"osd":3,"uuid":"36edb718-2fcb-4d88-8ddc-c321eed4279d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6817","nonce":1142509041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6819","nonce":1142509041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6824","nonce":1142509041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6821","nonce":1142509041}]},"public_addr":"172.21.15.151:6817/1142509041","cluster_addr":"172.21.15.151:6819/1142509041","heartbeat_back_addr":"172.21.15.151:6824/1142509041","heartbeat_front_addr":"172.21.15.151:6821/1142509041","state":["exists","up"]},{"osd":4,"uuid":"64d33708-cd0a-4236-abe3-1b646e82ab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6811","nonce":2735495271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6813","nonce":2735495271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6817","nonce":2735495271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6815","nonce":2735495271}]},"public_addr":"172.21.15.167:6811/2735495271","cluster_addr":"172.21.15.167:6813/2735495271","heartbeat_back_addr":"172.21.15.167:6817/2735495271","heartbeat_front_addr":"172.21.15.167:6815/2735495271","state":["exists","up"]},{"osd":5,"uuid":"f14b44a9-5910-458c-b518-09a8d0de94de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6827","nonce":1494916572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6829","nonce":1494916572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6833","nonce":1494916572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6831","nonce":1494916572}]},"public_addr":"172.21.15.167:6827/1494916572","cluster_addr":"172.21.15.167:6829/1494916572","heartbeat_back_addr":"172.21.15.167:6833/1494916572","heartbeat_front_addr":"172.21.15.167:6831/1494916572","state":["exists","up"]},{"osd":6,"uuid":"39335545-2654-4710-84b0-a9aa47cfcbba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6819","nonce":2538273273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6821","nonce":2538273273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6825","nonce":2538273273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6823","nonce":2538273273}]},"public_addr":"172.21.15.167:6819/2538273273","cluster_addr":"172.21.15.167:6821/2538273273","heartbeat_back_addr":"172.21.15.167:6825/2538273273","heartbeat_front_addr":"172.21.15.167:6823/2538273273","state":["exists","up"]},{"osd":7,"uuid":"4f8bfc81-53dd-4339-ac22-1a4007d9f963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6803","nonce":265072227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6805","nonce":265072227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6809","nonce":265072227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6807","nonce":265072227}]},"public_addr":"172.21.15.167:6803/265072227","cluster_addr":"172.21.15.167:6805/265072227","heartbeat_back_addr":"172.21.15.167:6809/265072227","heartbeat_front_addr":"172.21.15.167:6807/265072227","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.150359+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.084358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.722368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.332809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.320709+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.395153+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.935250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.801443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:42:25.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.384+0000 7fdacac00640 1 -- 172.21.15.151:0/4238478580 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdaa4078bb0 msgr2=0x7fdaa407b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:25.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.384+0000 7fdacac00640 1 --2- 172.21.15.151:0/4238478580 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdaa4078bb0 0x7fdaa407b070 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fdab4034000 tx=0x7fdab4004730 comp rx=0 tx=0).stop 2024-09-16T15:42:25.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.384+0000 7fdacac00640 1 -- 172.21.15.151:0/4238478580 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdac40ccf00 msgr2=0x7fdac4150320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:25.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.384+0000 7fdacac00640 1 --2- 172.21.15.151:0/4238478580 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdac40ccf00 0x7fdac4150320 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fdab000d950 tx=0x7fdab000de10 comp rx=0 tx=0).stop 2024-09-16T15:42:25.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.384+0000 7fdacac00640 1 -- 172.21.15.151:0/4238478580 shutdown_connections 2024-09-16T15:42:25.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.384+0000 7fdacac00640 1 --2- 172.21.15.151:0/4238478580 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdaa4078bb0 0x7fdaa407b070 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:25.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.384+0000 7fdacac00640 1 --2- 172.21.15.151:0/4238478580 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdac415f5a0 0x7fdac4150aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:25.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.384+0000 7fdacac00640 1 --2- 172.21.15.151:0/4238478580 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdac40ccf00 0x7fdac4150320 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:25.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.384+0000 7fdacac00640 1 --2- 172.21.15.151:0/4238478580 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdac40ad270 0x7fdac40cf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:25.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.384+0000 7fdacac00640 1 -- 172.21.15.151:0/4238478580 >> 172.21.15.151:0/4238478580 conn(0x7fdac40b1950 msgr2=0x7fdac40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:25.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.384+0000 7fdacac00640 1 -- 172.21.15.151:0/4238478580 shutdown_connections 2024-09-16T15:42:25.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.384+0000 7fdacac00640 1 -- 172.21.15.151:0/4238478580 wait complete. 2024-09-16T15:42:25.402 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:42:25.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.557+0000 7f2df6000640 1 -- 172.21.15.151:0/2422496421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2df00ccd00 msgr2=0x7f2df006f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:25.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.557+0000 7f2df6000640 1 --2- 172.21.15.151:0/2422496421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2df00ccd00 0x7f2df006f850 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2dd8009830 tx=0x7f2dd802a510 comp rx=0 tx=0).stop 2024-09-16T15:42:25.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.558+0000 7f2df6000640 1 -- 172.21.15.151:0/2422496421 shutdown_connections 2024-09-16T15:42:25.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.558+0000 7f2df6000640 1 --2- 172.21.15.151:0/2422496421 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2df006fd90 0x7f2df00b3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:25.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.558+0000 7f2df6000640 1 --2- 172.21.15.151:0/2422496421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2df00ccd00 0x7f2df006f850 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:25.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.558+0000 7f2df6000640 1 -- 172.21.15.151:0/2422496421 >> 172.21.15.151:0/2422496421 conn(0x7f2df00b1950 msgr2=0x7f2df00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:25.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.558+0000 7f2df6000640 1 -- 172.21.15.151:0/2422496421 shutdown_connections 2024-09-16T15:42:25.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.558+0000 7f2df6000640 1 -- 172.21.15.151:0/2422496421 wait complete. 2024-09-16T15:42:25.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.559+0000 7f2df6000640 1 Processor -- start 2024-09-16T15:42:25.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.559+0000 7f2df6000640 1 -- start start 2024-09-16T15:42:25.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.560+0000 7f2df6000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2df006fd90 0x7f2df01f5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:25.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.560+0000 7f2df6000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2df00ccd00 0x7f2df00cb260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:25.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.560+0000 7f2df6000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2df01438b0 0x7f2df00cdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:25.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.560+0000 7f2df6000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2df0070b30 con 0x7f2df00ccd00 2024-09-16T15:42:25.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.560+0000 7f2df6000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2df00b4440 con 0x7f2df01438b0 2024-09-16T15:42:25.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.560+0000 7f2df6000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f2df01f6330 con 0x7f2df006fd90 2024-09-16T15:42:25.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.560+0000 7f2de6a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2df00ccd00 0x7f2df00cb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:25.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.561+0000 7f2de6a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2df00ccd00 0x7f2df00cb260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49844/0 (socket says 172.21.15.151:49844) 2024-09-16T15:42:25.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.561+0000 7f2de6a00640 1 -- 172.21.15.151:0/1418270818 learned_addr learned my addr 172.21.15.151:0/1418270818 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:42:25.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.561+0000 7f2defe00640 1 --2- 172.21.15.151:0/1418270818 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2df01438b0 0x7f2df00cdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:25.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.561+0000 7f2de6a00640 1 -- 172.21.15.151:0/1418270818 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2df006fd90 msgr2=0x7f2df01f5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:25.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.561+0000 7f2de6a00640 1 --2- 172.21.15.151:0/1418270818 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2df006fd90 0x7f2df01f5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:25.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.561+0000 7f2de6a00640 1 -- 172.21.15.151:0/1418270818 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2df01438b0 msgr2=0x7f2df00cdcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:25.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.561+0000 7f2de6a00640 1 --2- 172.21.15.151:0/1418270818 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2df01438b0 0x7f2df00cdcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:25.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.561+0000 7f2de6a00640 1 -- 172.21.15.151:0/1418270818 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2df01f5330 con 0x7f2df00ccd00 2024-09-16T15:42:25.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.561+0000 7f2defe00640 1 --2- 172.21.15.151:0/1418270818 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2df01438b0 0x7f2df00cdcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:42:25.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.561+0000 7f2de6a00640 1 --2- 172.21.15.151:0/1418270818 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2df00ccd00 0x7f2df00cb260 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2dd8009950 tx=0x7f2dd80040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:25.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.561+0000 7f2de7e00640 1 -- 172.21.15.151:0/1418270818 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd8040070 con 0x7f2df00ccd00 2024-09-16T15:42:25.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.562+0000 7f2de7e00640 1 -- 172.21.15.151:0/1418270818 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2dd8005310 con 0x7f2df00ccd00 2024-09-16T15:42:25.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.562+0000 7f2df6000640 1 -- 172.21.15.151:0/1418270818 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2df01f93d0 con 0x7f2df00ccd00 2024-09-16T15:42:25.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.562+0000 7f2de7e00640 1 -- 172.21.15.151:0/1418270818 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd80044c0 con 0x7f2df00ccd00 2024-09-16T15:42:25.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.562+0000 7f2df6000640 1 -- 172.21.15.151:0/1418270818 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2df01f55f0 con 0x7f2df00ccd00 2024-09-16T15:42:25.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.563+0000 7f2df6000640 1 -- 172.21.15.151:0/1418270818 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2df00b3f00 con 0x7f2df00ccd00 2024-09-16T15:42:25.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.564+0000 7f2de7e00640 1 -- 172.21.15.151:0/1418270818 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7f2dd8039040 con 0x7f2df00ccd00 2024-09-16T15:42:25.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.565+0000 7f2de7e00640 1 --2- 172.21.15.151:0/1418270818 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2dcc078bb0 0x7f2dcc07b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:25.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.565+0000 7f2de7e00640 1 -- 172.21.15.151:0/1418270818 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f2dd80bbf40 con 0x7f2df00ccd00 2024-09-16T15:42:25.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.565+0000 7f2def400640 1 --2- 172.21.15.151:0/1418270818 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2dcc078bb0 0x7f2dcc07b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:25.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.566+0000 7f2def400640 1 --2- 172.21.15.151:0/1418270818 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2dcc078bb0 0x7f2dcc07b070 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2de00046b0 tx=0x7f2de00012b0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:25.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.569+0000 7f2de7e00640 1 -- 172.21.15.151:0/1418270818 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2dd8031770 con 0x7f2df00ccd00 2024-09-16T15:42:25.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.754+0000 7f2df6000640 1 -- 172.21.15.151:0/1418270818 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2df00cd430 con 0x7f2df00ccd00 2024-09-16T15:42:25.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.756+0000 7f2de7e00640 1 -- 172.21.15.151:0/1418270818 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) ==== 74+0+16742 (secure 0 0 0) 0x7f2dd80840e0 con 0x7f2df00ccd00 2024-09-16T15:42:25.757 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:42:25.757 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":47,"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","created":"2024-09-16T15:38:56.348535+0000","modified":"2024-09-16T15:42:24.411607+0000","last_up_change":"2024-09-16T15:39:10.681384+0000","last_in_change":"2024-09-16T15:39:00.544477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:39:11.602252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:42:22.166447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-16T15:42:23.379899+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5524076-a903-4d0a-af6a-caf474c4765d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6809","nonce":3111342741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6811","nonce":3111342741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6815","nonce":3111342741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6813","nonce":3111342741}]},"public_addr":"172.21.15.151:6809/3111342741","cluster_addr":"172.21.15.151:6811/3111342741","heartbeat_back_addr":"172.21.15.151:6815/3111342741","heartbeat_front_addr":"172.21.15.151:6813/3111342741","state":["exists","up"]},{"osd":1,"uuid":"dd25925f-0ae1-494e-8529-4dc205759db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6801","nonce":3368304087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6803","nonce":3368304087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6807","nonce":3368304087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6805","nonce":3368304087}]},"public_addr":"172.21.15.151:6801/3368304087","cluster_addr":"172.21.15.151:6803/3368304087","heartbeat_back_addr":"172.21.15.151:6807/3368304087","heartbeat_front_addr":"172.21.15.151:6805/3368304087","state":["exists","up"]},{"osd":2,"uuid":"c2f18115-013d-4da6-a0fe-cb978f4494b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6823","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6825","nonce":3079373271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6827","nonce":3079373271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6831","nonce":3079373271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6829","nonce":3079373271}]},"public_addr":"172.21.15.151:6825/3079373271","cluster_addr":"172.21.15.151:6827/3079373271","heartbeat_back_addr":"172.21.15.151:6831/3079373271","heartbeat_front_addr":"172.21.15.151:6829/3079373271","state":["exists","up"]},{"osd":3,"uuid":"36edb718-2fcb-4d88-8ddc-c321eed4279d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6817","nonce":1142509041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6819","nonce":1142509041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6824","nonce":1142509041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6821","nonce":1142509041}]},"public_addr":"172.21.15.151:6817/1142509041","cluster_addr":"172.21.15.151:6819/1142509041","heartbeat_back_addr":"172.21.15.151:6824/1142509041","heartbeat_front_addr":"172.21.15.151:6821/1142509041","state":["exists","up"]},{"osd":4,"uuid":"64d33708-cd0a-4236-abe3-1b646e82ab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6811","nonce":2735495271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6813","nonce":2735495271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6817","nonce":2735495271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6815","nonce":2735495271}]},"public_addr":"172.21.15.167:6811/2735495271","cluster_addr":"172.21.15.167:6813/2735495271","heartbeat_back_addr":"172.21.15.167:6817/2735495271","heartbeat_front_addr":"172.21.15.167:6815/2735495271","state":["exists","up"]},{"osd":5,"uuid":"f14b44a9-5910-458c-b518-09a8d0de94de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6827","nonce":1494916572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6829","nonce":1494916572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6833","nonce":1494916572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6831","nonce":1494916572}]},"public_addr":"172.21.15.167:6827/1494916572","cluster_addr":"172.21.15.167:6829/1494916572","heartbeat_back_addr":"172.21.15.167:6833/1494916572","heartbeat_front_addr":"172.21.15.167:6831/1494916572","state":["exists","up"]},{"osd":6,"uuid":"39335545-2654-4710-84b0-a9aa47cfcbba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6819","nonce":2538273273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6821","nonce":2538273273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6825","nonce":2538273273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6823","nonce":2538273273}]},"public_addr":"172.21.15.167:6819/2538273273","cluster_addr":"172.21.15.167:6821/2538273273","heartbeat_back_addr":"172.21.15.167:6825/2538273273","heartbeat_front_addr":"172.21.15.167:6823/2538273273","state":["exists","up"]},{"osd":7,"uuid":"4f8bfc81-53dd-4339-ac22-1a4007d9f963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6803","nonce":265072227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6805","nonce":265072227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6809","nonce":265072227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6807","nonce":265072227}]},"public_addr":"172.21.15.167:6803/265072227","cluster_addr":"172.21.15.167:6805/265072227","heartbeat_back_addr":"172.21.15.167:6809/265072227","heartbeat_front_addr":"172.21.15.167:6807/265072227","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.150359+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.084358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.722368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.332809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.320709+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.395153+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.935250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.801443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:42:25.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.758+0000 7f2df6000640 1 -- 172.21.15.151:0/1418270818 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2dcc078bb0 msgr2=0x7f2dcc07b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:25.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.758+0000 7f2df6000640 1 --2- 172.21.15.151:0/1418270818 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2dcc078bb0 0x7f2dcc07b070 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2de00046b0 tx=0x7f2de00012b0 comp rx=0 tx=0).stop 2024-09-16T15:42:25.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.758+0000 7f2df6000640 1 -- 172.21.15.151:0/1418270818 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2df00ccd00 msgr2=0x7f2df00cb260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:25.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.758+0000 7f2df6000640 1 --2- 172.21.15.151:0/1418270818 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2df00ccd00 0x7f2df00cb260 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2dd8009950 tx=0x7f2dd80040c0 comp rx=0 tx=0).stop 2024-09-16T15:42:25.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.758+0000 7f2df6000640 1 -- 172.21.15.151:0/1418270818 shutdown_connections 2024-09-16T15:42:25.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.758+0000 7f2df6000640 1 --2- 172.21.15.151:0/1418270818 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2dcc078bb0 0x7f2dcc07b070 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:25.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.758+0000 7f2df6000640 1 --2- 172.21.15.151:0/1418270818 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2df01438b0 0x7f2df00cdcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:25.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.758+0000 7f2df6000640 1 --2- 172.21.15.151:0/1418270818 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2df00ccd00 0x7f2df00cb260 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:25.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.758+0000 7f2df6000640 1 --2- 172.21.15.151:0/1418270818 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2df006fd90 0x7f2df01f5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:25.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.758+0000 7f2df6000640 1 -- 172.21.15.151:0/1418270818 >> 172.21.15.151:0/1418270818 conn(0x7f2df00b1950 msgr2=0x7f2df015d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:25.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.758+0000 7f2df6000640 1 -- 172.21.15.151:0/1418270818 shutdown_connections 2024-09-16T15:42:25.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.758+0000 7f2df6000640 1 -- 172.21.15.151:0/1418270818 wait complete. 2024-09-16T15:42:25.775 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-16T15:42:25.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.936+0000 7fc8ad600640 1 -- 172.21.15.151:0/2670766624 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc88c004800 msgr2=0x7fc88c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:25.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.936+0000 7fc8ad600640 1 --2- 172.21.15.151:0/2670766624 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc88c004800 0x7fc88c004c80 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fc890001690 tx=0x7fc88c005830 comp rx=0 tx=0).stop 2024-09-16T15:42:25.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.936+0000 7fc8ad600640 1 -- 172.21.15.151:0/2670766624 shutdown_connections 2024-09-16T15:42:25.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.936+0000 7fc8ad600640 1 --2- 172.21.15.151:0/2670766624 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc88c004800 0x7fc88c004c80 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:25.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.936+0000 7fc8ad600640 1 -- 172.21.15.151:0/2670766624 >> 172.21.15.151:0/2670766624 conn(0x7fc8a80b1840 msgr2=0x7fc8a80b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:25.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.936+0000 7fc8ad600640 1 -- 172.21.15.151:0/2670766624 shutdown_connections 2024-09-16T15:42:25.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.936+0000 7fc8ad600640 1 -- 172.21.15.151:0/2670766624 wait complete. 2024-09-16T15:42:25.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.938+0000 7fc8ad600640 1 Processor -- start 2024-09-16T15:42:25.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.938+0000 7fc8ad600640 1 -- start start 2024-09-16T15:42:25.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.939+0000 7fc8ad600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc88c004800 0x7fc8a815e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:25.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.939+0000 7fc8ad600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc8a81f59a0 0x7fc8a81e91c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:25.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.939+0000 7fc8ad600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8a81e9700 0x7fc8a81f82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:25.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.939+0000 7fc8ad600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a80737d0 con 0x7fc8a81e9700 2024-09-16T15:42:25.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.939+0000 7fc8ad600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a8073650 con 0x7fc88c004800 2024-09-16T15:42:25.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.939+0000 7fc8ad600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fc8a80cc5c0 con 0x7fc8a81f59a0 2024-09-16T15:42:25.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.939+0000 7fc8a7400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8a81e9700 0x7fc8a81f82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:25.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.939+0000 7fc8a7400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8a81e9700 0x7fc8a81f82a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49874/0 (socket says 172.21.15.151:49874) 2024-09-16T15:42:25.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.939+0000 7fc8a7400640 1 -- 172.21.15.151:0/4131867033 learned_addr learned my addr 172.21.15.151:0/4131867033 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:42:25.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.939+0000 7fc89e000640 1 --2- 172.21.15.151:0/4131867033 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc8a81f59a0 0x7fc8a81e91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:25.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.939+0000 7fc8a6a00640 1 --2- 172.21.15.151:0/4131867033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc88c004800 0x7fc8a815e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:25.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.940+0000 7fc8a7400640 1 -- 172.21.15.151:0/4131867033 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc8a81f59a0 msgr2=0x7fc8a81e91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:25.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.940+0000 7fc8a7400640 1 --2- 172.21.15.151:0/4131867033 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc8a81f59a0 0x7fc8a81e91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:25.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.940+0000 7fc8a7400640 1 -- 172.21.15.151:0/4131867033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc88c004800 msgr2=0x7fc8a815e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:25.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.940+0000 7fc8a7400640 1 --2- 172.21.15.151:0/4131867033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc88c004800 0x7fc8a815e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:25.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.940+0000 7fc8a7400640 1 -- 172.21.15.151:0/4131867033 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8a81f8f90 con 0x7fc8a81e9700 2024-09-16T15:42:25.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.940+0000 7fc8a7400640 1 --2- 172.21.15.151:0/4131867033 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8a81e9700 0x7fc8a81f82a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fc8a80cc550 tx=0x7fc8a0006660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:25.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.940+0000 7fc89f400640 1 -- 172.21.15.151:0/4131867033 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a000fa80 con 0x7fc8a81e9700 2024-09-16T15:42:25.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.940+0000 7fc89f400640 1 -- 172.21.15.151:0/4131867033 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc8a0018070 con 0x7fc8a81e9700 2024-09-16T15:42:25.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.940+0000 7fc89f400640 1 -- 172.21.15.151:0/4131867033 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a0011430 con 0x7fc8a81e9700 2024-09-16T15:42:25.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.940+0000 7fc8ad600640 1 -- 172.21.15.151:0/4131867033 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8a81f4ec0 con 0x7fc8a81e9700 2024-09-16T15:42:25.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.940+0000 7fc8ad600640 1 -- 172.21.15.151:0/4131867033 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8a815e8d0 con 0x7fc8a81e9700 2024-09-16T15:42:25.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.941+0000 7fc8ad600640 1 -- 172.21.15.151:0/4131867033 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8a81f4cb0 con 0x7fc8a81e9700 2024-09-16T15:42:25.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.943+0000 7fc89f400640 1 -- 172.21.15.151:0/4131867033 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7fc8a00115d0 con 0x7fc8a81e9700 2024-09-16T15:42:25.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.944+0000 7fc89f400640 1 --2- 172.21.15.151:0/4131867033 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc884078be0 0x7fc88407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:25.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.944+0000 7fc89f400640 1 -- 172.21.15.151:0/4131867033 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7fc8a009b600 con 0x7fc8a81e9700 2024-09-16T15:42:25.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.944+0000 7fc8a6a00640 1 --2- 172.21.15.151:0/4131867033 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc884078be0 0x7fc88407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:25.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.945+0000 7fc8a6a00640 1 --2- 172.21.15.151:0/4131867033 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc884078be0 0x7fc88407b0a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8052890 tx=0x7fc898005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:25.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:25.948+0000 7fc89f400640 1 -- 172.21.15.151:0/4131867033 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc8a0063710 con 0x7fc8a81e9700 2024-09-16T15:42:26.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:26.176+0000 7fc8ad600640 1 -- 172.21.15.151:0/4131867033 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fc8a815f9f0 con 0x7fc8a81e9700 2024-09-16T15:42:27.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:27.008+0000 7fc89f400640 1 -- 172.21.15.151:0/4131867033 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v26) ==== 118+0+0 (secure 0 0 0) 0x7fc8a0063010 con 0x7fc8a81e9700 2024-09-16T15:42:27.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:27.011+0000 7fc8ad600640 1 -- 172.21.15.151:0/4131867033 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc884078be0 msgr2=0x7fc88407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:27.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:27.011+0000 7fc8ad600640 1 --2- 172.21.15.151:0/4131867033 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc884078be0 0x7fc88407b0a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8052890 tx=0x7fc898005f90 comp rx=0 tx=0).stop 2024-09-16T15:42:27.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:27.011+0000 7fc8ad600640 1 -- 172.21.15.151:0/4131867033 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8a81e9700 msgr2=0x7fc8a81f82a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:27.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:27.011+0000 7fc8ad600640 1 --2- 172.21.15.151:0/4131867033 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8a81e9700 0x7fc8a81f82a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fc8a80cc550 tx=0x7fc8a0006660 comp rx=0 tx=0).stop 2024-09-16T15:42:27.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:27.011+0000 7fc8ad600640 1 -- 172.21.15.151:0/4131867033 shutdown_connections 2024-09-16T15:42:27.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:27.011+0000 7fc8ad600640 1 --2- 172.21.15.151:0/4131867033 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc884078be0 0x7fc88407b0a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:27.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:27.011+0000 7fc8ad600640 1 --2- 172.21.15.151:0/4131867033 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8a81e9700 0x7fc8a81f82a0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:27.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:27.011+0000 7fc8ad600640 1 --2- 172.21.15.151:0/4131867033 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc8a81f59a0 0x7fc8a81e91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:27.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:27.011+0000 7fc8ad600640 1 --2- 172.21.15.151:0/4131867033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc88c004800 0x7fc8a815e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:27.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:27.011+0000 7fc8ad600640 1 -- 172.21.15.151:0/4131867033 >> 172.21.15.151:0/4131867033 conn(0x7fc8a80b1840 msgr2=0x7fc8a80ad870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:27.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:27.011+0000 7fc8ad600640 1 -- 172.21.15.151:0/4131867033 shutdown_connections 2024-09-16T15:42:27.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:27.012+0000 7fc8ad600640 1 -- 172.21.15.151:0/4131867033 wait complete. 2024-09-16T15:42:27.029 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-16T15:42:27.030 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-16T15:42:27.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:27.190+0000 7f415e800640 1 -- 172.21.15.151:0/1809312851 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4158145750 msgr2=0x7f4158073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:27.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:27.190+0000 7f415e800640 1 --2- 172.21.15.151:0/1809312851 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4158145750 0x7f4158073440 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4148009860 tx=0x7f414802a510 comp rx=0 tx=0).stop 2024-09-16T15:42:27.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:27.190+0000 7f415e800640 1 -- 172.21.15.151:0/1809312851 shutdown_connections 2024-09-16T15:42:27.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:27.190+0000 7f415e800640 1 --2- 172.21.15.151:0/1809312851 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4158145750 0x7f4158073440 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:27.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:27.190+0000 7f415e800640 1 -- 172.21.15.151:0/1809312851 >> 172.21.15.151:0/1809312851 conn(0x7f41580b1840 msgr2=0x7f41580b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:27.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:27.190+0000 7f415e800640 1 -- 172.21.15.151:0/1809312851 shutdown_connections 2024-09-16T15:42:27.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:27.190+0000 7f415e800640 1 -- 172.21.15.151:0/1809312851 wait complete. 2024-09-16T15:42:27.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:27.191+0000 7f415e800640 1 Processor -- start 2024-09-16T15:42:27.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:27.192+0000 7f415e800640 1 -- start start 2024-09-16T15:42:27.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:27.192+0000 7f415e800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f415806efa0 0x7f415813f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:27.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:27.192+0000 7f415e800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f41580cccd0 0x7f41580ac1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:27.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:27.193+0000 7f415e800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4158145750 0x7f41581e6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:27.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:27.193+0000 7f415e800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f41580ce6d0 con 0x7f41580cccd0 2024-09-16T15:42:27.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:27.193+0000 7f415e800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f41580ce550 con 0x7f4158145750 2024-09-16T15:42:27.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:27.193+0000 7f415e800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f41580ce850 con 0x7f415806efa0 2024-09-16T15:42:27.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:27.193+0000 7f4156a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f41580cccd0 0x7f41580ac1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:27.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:27.193+0000 7f4156a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f41580cccd0 0x7f41580ac1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49884/0 (socket says 172.21.15.151:49884) 2024-09-16T15:42:27.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:27.193+0000 7f4156a00640 1 -- 172.21.15.151:0/853387057 learned_addr learned my addr 172.21.15.151:0/853387057 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:42:27.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:27.193+0000 7f4157400640 1 --2- 172.21.15.151:0/853387057 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f415806efa0 0x7f415813f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:27.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:27.193+0000 7f4157e00640 1 --2- 172.21.15.151:0/853387057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4158145750 0x7f41581e6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:27.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:27.193+0000 7f4156a00640 1 -- 172.21.15.151:0/853387057 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f415806efa0 msgr2=0x7f415813f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:27.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:27.193+0000 7f4156a00640 1 --2- 172.21.15.151:0/853387057 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f415806efa0 0x7f415813f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:27.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:27.193+0000 7f4156a00640 1 -- 172.21.15.151:0/853387057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4158145750 msgr2=0x7f41581e6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:27.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:27.193+0000 7f4156a00640 1 --2- 172.21.15.151:0/853387057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4158145750 0x7f41581e6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:27.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:27.193+0000 7f4156a00640 1 -- 172.21.15.151:0/853387057 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41581f3540 con 0x7f41580cccd0 2024-09-16T15:42:27.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:27.193+0000 7f4156a00640 1 --2- 172.21.15.151:0/853387057 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f41580cccd0 0x7f41580ac1b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f414400c500 tx=0x7f414400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:27.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:27.194+0000 7f414f400640 1 -- 172.21.15.151:0/853387057 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4144016020 con 0x7f41580cccd0 2024-09-16T15:42:27.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:27.194+0000 7f414f400640 1 -- 172.21.15.151:0/853387057 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4144012940 con 0x7f41580cccd0 2024-09-16T15:42:27.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:27.194+0000 7f414f400640 1 -- 172.21.15.151:0/853387057 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4144012dd0 con 0x7f41580cccd0 2024-09-16T15:42:27.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:27.194+0000 7f415e800640 1 -- 172.21.15.151:0/853387057 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41581f3820 con 0x7f41580cccd0 2024-09-16T15:42:27.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:27.194+0000 7f415e800640 1 -- 172.21.15.151:0/853387057 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41581e4970 con 0x7f41580cccd0 2024-09-16T15:42:27.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:27.195+0000 7f415e800640 1 -- 172.21.15.151:0/853387057 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41580b1c20 con 0x7f41580cccd0 2024-09-16T15:42:27.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:27.196+0000 7f414f400640 1 -- 172.21.15.151:0/853387057 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7f4144019020 con 0x7f41580cccd0 2024-09-16T15:42:27.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:27.197+0000 7f414f400640 1 --2- 172.21.15.151:0/853387057 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4138078bb0 0x7f413807b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:27.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:27.197+0000 7f414f400640 1 -- 172.21.15.151:0/853387057 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f414409abf0 con 0x7f41580cccd0 2024-09-16T15:42:27.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:27.197+0000 7f4157400640 1 --2- 172.21.15.151:0/853387057 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4138078bb0 0x7f413807b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:27.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:27.198+0000 7f4157400640 1 --2- 172.21.15.151:0/853387057 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4138078bb0 0x7f413807b070 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4148034000 tx=0x7f4148004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:27.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:27.201+0000 7f414f400640 1 -- 172.21.15.151:0/853387057 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f41580b1c20 con 0x7f41580cccd0 2024-09-16T15:42:27.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:27.424+0000 7f415e800640 1 -- 172.21.15.151:0/853387057 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f41580ced00 con 0x7f41580cccd0 2024-09-16T15:42:28.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.015+0000 7f414f400640 1 -- 172.21.15.151:0/853387057 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v27) ==== 122+0+0 (secure 0 0 0) 0x7f4144020030 con 0x7f41580cccd0 2024-09-16T15:42:28.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.017+0000 7f415e800640 1 -- 172.21.15.151:0/853387057 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4138078bb0 msgr2=0x7f413807b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:28.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.017+0000 7f415e800640 1 --2- 172.21.15.151:0/853387057 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4138078bb0 0x7f413807b070 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4148034000 tx=0x7f4148004730 comp rx=0 tx=0).stop 2024-09-16T15:42:28.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.017+0000 7f415e800640 1 -- 172.21.15.151:0/853387057 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f41580cccd0 msgr2=0x7f41580ac1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:28.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.017+0000 7f415e800640 1 --2- 172.21.15.151:0/853387057 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f41580cccd0 0x7f41580ac1b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f414400c500 tx=0x7f414400c9c0 comp rx=0 tx=0).stop 2024-09-16T15:42:28.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.018+0000 7f415e800640 1 -- 172.21.15.151:0/853387057 shutdown_connections 2024-09-16T15:42:28.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.018+0000 7f415e800640 1 --2- 172.21.15.151:0/853387057 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4138078bb0 0x7f413807b070 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:28.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.018+0000 7f415e800640 1 --2- 172.21.15.151:0/853387057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4158145750 0x7f41581e6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:28.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.018+0000 7f415e800640 1 --2- 172.21.15.151:0/853387057 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f41580cccd0 0x7f41580ac1b0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:28.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.018+0000 7f415e800640 1 --2- 172.21.15.151:0/853387057 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f415806efa0 0x7f415813f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:28.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.018+0000 7f415e800640 1 -- 172.21.15.151:0/853387057 >> 172.21.15.151:0/853387057 conn(0x7f41580b1840 msgr2=0x7f41580a9460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:28.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.018+0000 7f415e800640 1 -- 172.21.15.151:0/853387057 shutdown_connections 2024-09-16T15:42:28.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.018+0000 7f415e800640 1 -- 172.21.15.151:0/853387057 wait complete. 2024-09-16T15:42:28.036 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-16T15:42:28.036 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:42:28.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.195+0000 7fa6c7e00640 1 -- 172.21.15.151:0/1355317941 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6c80cf440 msgr2=0x7fa6c80d4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:28.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.195+0000 7fa6c7e00640 1 --2- 172.21.15.151:0/1355317941 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6c80cf440 0x7fa6c80d4940 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0005fd0 tx=0x7fa6c002ab40 comp rx=0 tx=0).stop 2024-09-16T15:42:28.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.195+0000 7fa6c7e00640 1 -- 172.21.15.151:0/1355317941 shutdown_connections 2024-09-16T15:42:28.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.195+0000 7fa6c7e00640 1 --2- 172.21.15.151:0/1355317941 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6c80cf440 0x7fa6c80d4940 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:28.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.195+0000 7fa6c7e00640 1 -- 172.21.15.151:0/1355317941 >> 172.21.15.151:0/1355317941 conn(0x7fa6c80d3e20 msgr2=0x7fa6c80d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:28.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.195+0000 7fa6c7e00640 1 -- 172.21.15.151:0/1355317941 shutdown_connections 2024-09-16T15:42:28.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.195+0000 7fa6c7e00640 1 -- 172.21.15.151:0/1355317941 wait complete. 2024-09-16T15:42:28.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.196+0000 7fa6c7e00640 1 Processor -- start 2024-09-16T15:42:28.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.196+0000 7fa6c7e00640 1 -- start start 2024-09-16T15:42:28.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.197+0000 7fa6c7e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6c80cf440 0x7fa6c81509a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:28.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.197+0000 7fa6c7e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6c80d25b0 0x7fa6c81f4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:28.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.197+0000 7fa6c7e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6c8161520 0x7fa6c80b3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:28.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.197+0000 7fa6c7e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c80b8500 con 0x7fa6c8161520 2024-09-16T15:42:28.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.197+0000 7fa6c7e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c80b8380 con 0x7fa6c80d25b0 2024-09-16T15:42:28.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.197+0000 7fa6c7e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fa6c80b8680 con 0x7fa6c80cf440 2024-09-16T15:42:28.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.198+0000 7fa6c7400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6c8161520 0x7fa6c80b3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:28.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.198+0000 7fa6c7400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6c8161520 0x7fa6c80b3960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49914/0 (socket says 172.21.15.151:49914) 2024-09-16T15:42:28.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.198+0000 7fa6c7400640 1 -- 172.21.15.151:0/3667875076 learned_addr learned my addr 172.21.15.151:0/3667875076 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:42:28.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.198+0000 7fa6c6000640 1 --2- 172.21.15.151:0/3667875076 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6c80d25b0 0x7fa6c81f4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:28.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.198+0000 7fa6c6a00640 1 --2- 172.21.15.151:0/3667875076 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6c80cf440 0x7fa6c81509a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:28.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.198+0000 7fa6c7400640 1 -- 172.21.15.151:0/3667875076 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6c80cf440 msgr2=0x7fa6c81509a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:28.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.198+0000 7fa6c7400640 1 --2- 172.21.15.151:0/3667875076 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6c80cf440 0x7fa6c81509a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:28.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.198+0000 7fa6c7400640 1 -- 172.21.15.151:0/3667875076 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6c80d25b0 msgr2=0x7fa6c81f4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:28.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.198+0000 7fa6c7400640 1 --2- 172.21.15.151:0/3667875076 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6c80d25b0 0x7fa6c81f4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:28.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.198+0000 7fa6c7400640 1 -- 172.21.15.151:0/3667875076 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6c81f5480 con 0x7fa6c8161520 2024-09-16T15:42:28.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.198+0000 7fa6c7400640 1 --2- 172.21.15.151:0/3667875076 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6c8161520 0x7fa6c80b3960 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0009bf0 tx=0x7fa6c002c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:28.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.199+0000 7fa6bf400640 1 -- 172.21.15.151:0/3667875076 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c0044070 con 0x7fa6c8161520 2024-09-16T15:42:28.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.199+0000 7fa6bf400640 1 -- 172.21.15.151:0/3667875076 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa6c0004250 con 0x7fa6c8161520 2024-09-16T15:42:28.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.199+0000 7fa6c7e00640 1 -- 172.21.15.151:0/3667875076 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6c81f5760 con 0x7fa6c8161520 2024-09-16T15:42:28.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.199+0000 7fa6bf400640 1 -- 172.21.15.151:0/3667875076 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c0004680 con 0x7fa6c8161520 2024-09-16T15:42:28.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.199+0000 7fa6c7e00640 1 -- 172.21.15.151:0/3667875076 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6c809f570 con 0x7fa6c8161520 2024-09-16T15:42:28.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.200+0000 7fa6c7e00640 1 -- 172.21.15.151:0/3667875076 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6c80d4940 con 0x7fa6c8161520 2024-09-16T15:42:28.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.201+0000 7fa6bf400640 1 -- 172.21.15.151:0/3667875076 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7fa6c0036400 con 0x7fa6c8161520 2024-09-16T15:42:28.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.203+0000 7fa6bf400640 1 --2- 172.21.15.151:0/3667875076 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa6a8078be0 0x7fa6a807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:28.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.203+0000 7fa6bf400640 1 -- 172.21.15.151:0/3667875076 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7fa6c00bb8a0 con 0x7fa6c8161520 2024-09-16T15:42:28.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.203+0000 7fa6c6a00640 1 --2- 172.21.15.151:0/3667875076 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa6a8078be0 0x7fa6a807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:28.205 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.204+0000 7fa6c6a00640 1 --2- 172.21.15.151:0/3667875076 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa6a8078be0 0x7fa6a807b0a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa6c8074720 tx=0x7fa6b8009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:28.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.207+0000 7fa6bf400640 1 -- 172.21.15.151:0/3667875076 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa6c0032030 con 0x7fa6c8161520 2024-09-16T15:42:28.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.427+0000 7fa6c7e00640 1 -- 172.21.15.151:0/3667875076 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa6c80d0dd0 con 0x7fa6c8161520 2024-09-16T15:42:28.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.428+0000 7fa6bf400640 1 -- 172.21.15.151:0/3667875076 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5232 (secure 0 0 0) 0x7fa6c00bb090 con 0x7fa6c8161520 2024-09-16T15:42:28.429 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:42:28.429 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":27,"btime":"2024-09-16T15:42:28:009764+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5272,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1043845735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1043845735},{"type":"v1","addr":"172.21.15.167:6836","nonce":1043845735}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5275,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/816612106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":816612106},{"type":"v1","addr":"172.21.15.167:6837","nonce":816612106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5397,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/1986755100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1986755100},{"type":"v1","addr":"172.21.15.151:6834","nonce":1986755100}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:42:24.411561+0000","modified":"2024-09-16T15:42:27.425402+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5400":{"gid":5400,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.151:6835/1974411916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":1974411916},{"type":"v1","addr":"172.21.15.151:6835","nonce":1974411916}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5400,"qdb_cluster":[5400]},"id":3}]} 2024-09-16T15:42:28.429 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 27 2024-09-16T15:42:28.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.430+0000 7fa6c7e00640 1 -- 172.21.15.151:0/3667875076 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa6a8078be0 msgr2=0x7fa6a807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:28.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.430+0000 7fa6c7e00640 1 --2- 172.21.15.151:0/3667875076 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa6a8078be0 0x7fa6a807b0a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa6c8074720 tx=0x7fa6b8009040 comp rx=0 tx=0).stop 2024-09-16T15:42:28.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.430+0000 7fa6c7e00640 1 -- 172.21.15.151:0/3667875076 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6c8161520 msgr2=0x7fa6c80b3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:28.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.430+0000 7fa6c7e00640 1 --2- 172.21.15.151:0/3667875076 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6c8161520 0x7fa6c80b3960 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0009bf0 tx=0x7fa6c002c8b0 comp rx=0 tx=0).stop 2024-09-16T15:42:28.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.430+0000 7fa6c7e00640 1 -- 172.21.15.151:0/3667875076 shutdown_connections 2024-09-16T15:42:28.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.430+0000 7fa6c7e00640 1 --2- 172.21.15.151:0/3667875076 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa6a8078be0 0x7fa6a807b0a0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:28.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.430+0000 7fa6c7e00640 1 --2- 172.21.15.151:0/3667875076 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa6c80d25b0 0x7fa6c81f4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:28.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.430+0000 7fa6c7e00640 1 --2- 172.21.15.151:0/3667875076 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6c8161520 0x7fa6c80b3960 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:28.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.430+0000 7fa6c7e00640 1 --2- 172.21.15.151:0/3667875076 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa6c80cf440 0x7fa6c81509a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:28.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.430+0000 7fa6c7e00640 1 -- 172.21.15.151:0/3667875076 >> 172.21.15.151:0/3667875076 conn(0x7fa6c80d3e20 msgr2=0x7fa6c80cd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:28.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.430+0000 7fa6c7e00640 1 -- 172.21.15.151:0/3667875076 shutdown_connections 2024-09-16T15:42:28.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.430+0000 7fa6c7e00640 1 -- 172.21.15.151:0/3667875076 wait complete. 2024-09-16T15:42:28.448 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:42:28.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.604+0000 7f0477000640 1 -- 172.21.15.151:0/4000461117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f047015f5a0 msgr2=0x7f04700b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:28.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.604+0000 7f0477000640 1 --2- 172.21.15.151:0/4000461117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f047015f5a0 0x7f04700b2440 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0460009830 tx=0x7f046002a530 comp rx=0 tx=0).stop 2024-09-16T15:42:28.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.604+0000 7f0477000640 1 -- 172.21.15.151:0/4000461117 shutdown_connections 2024-09-16T15:42:28.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.604+0000 7f0477000640 1 --2- 172.21.15.151:0/4000461117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f047015f5a0 0x7f04700b2440 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:28.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.604+0000 7f0477000640 1 -- 172.21.15.151:0/4000461117 >> 172.21.15.151:0/4000461117 conn(0x7f04700b1950 msgr2=0x7f04700b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:28.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.605+0000 7f0477000640 1 -- 172.21.15.151:0/4000461117 shutdown_connections 2024-09-16T15:42:28.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.605+0000 7f0477000640 1 -- 172.21.15.151:0/4000461117 wait complete. 2024-09-16T15:42:28.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.606+0000 7f0477000640 1 Processor -- start 2024-09-16T15:42:28.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.606+0000 7f0477000640 1 -- start start 2024-09-16T15:42:28.610 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.607+0000 7f0477000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0470073fb0 0x7f04701f8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:28.610 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.607+0000 7f0477000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f04700ccf00 0x7f0470156300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:28.610 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.607+0000 7f0477000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f047015f5a0 0x7f047014f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:28.610 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.607+0000 7f0477000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f047006f9f0 con 0x7f0470073fb0 2024-09-16T15:42:28.610 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.607+0000 7f0477000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f047006f870 con 0x7f047015f5a0 2024-09-16T15:42:28.610 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.607+0000 7f0477000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f0470074420 con 0x7f04700ccf00 2024-09-16T15:42:28.610 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.607+0000 7f046fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0470073fb0 0x7f04701f8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:28.610 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.608+0000 7f046fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0470073fb0 0x7f04701f8af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49920/0 (socket says 172.21.15.151:49920) 2024-09-16T15:42:28.610 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.608+0000 7f046fe00640 1 -- 172.21.15.151:0/1992497551 learned_addr learned my addr 172.21.15.151:0/1992497551 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:42:28.610 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.608+0000 7f0475000640 1 --2- 172.21.15.151:0/1992497551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f047015f5a0 0x7f047014f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:28.610 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.608+0000 7f046f400640 1 --2- 172.21.15.151:0/1992497551 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f04700ccf00 0x7f0470156300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:28.611 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.608+0000 7f046fe00640 1 -- 172.21.15.151:0/1992497551 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f04700ccf00 msgr2=0x7f0470156300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:28.611 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.608+0000 7f046fe00640 1 --2- 172.21.15.151:0/1992497551 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f04700ccf00 0x7f0470156300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:28.611 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.608+0000 7f046fe00640 1 -- 172.21.15.151:0/1992497551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f047015f5a0 msgr2=0x7f047014f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:28.611 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.608+0000 7f046fe00640 1 --2- 172.21.15.151:0/1992497551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f047015f5a0 0x7f047014f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:28.611 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.608+0000 7f046fe00640 1 -- 172.21.15.151:0/1992497551 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04701f9030 con 0x7f0470073fb0 2024-09-16T15:42:28.611 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.608+0000 7f046fe00640 1 --2- 172.21.15.151:0/1992497551 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0470073fb0 0x7f04701f8af0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0460000c00 tx=0x7f0460033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:28.611 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.608+0000 7f0467e00640 1 -- 172.21.15.151:0/1992497551 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0460031d60 con 0x7f0470073fb0 2024-09-16T15:42:28.611 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.609+0000 7f0467e00640 1 -- 172.21.15.151:0/1992497551 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0460033e10 con 0x7f0470073fb0 2024-09-16T15:42:28.611 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.609+0000 7f0467e00640 1 -- 172.21.15.151:0/1992497551 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04600332b0 con 0x7f0470073fb0 2024-09-16T15:42:28.611 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.609+0000 7f0477000640 1 -- 172.21.15.151:0/1992497551 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04700cc2a0 con 0x7f0470073fb0 2024-09-16T15:42:28.611 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.609+0000 7f0477000640 1 -- 172.21.15.151:0/1992497551 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04700c9500 con 0x7f0470073fb0 2024-09-16T15:42:28.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.610+0000 7f0477000640 1 -- 172.21.15.151:0/1992497551 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04700c9030 con 0x7f0470073fb0 2024-09-16T15:42:28.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.611+0000 7f0467e00640 1 -- 172.21.15.151:0/1992497551 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7f0460033450 con 0x7f0470073fb0 2024-09-16T15:42:28.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.612+0000 7f0467e00640 1 --2- 172.21.15.151:0/1992497551 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0450078bb0 0x7f045007b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:28.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.612+0000 7f0467e00640 1 -- 172.21.15.151:0/1992497551 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f04600bc190 con 0x7f0470073fb0 2024-09-16T15:42:28.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.612+0000 7f046f400640 1 --2- 172.21.15.151:0/1992497551 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0450078bb0 0x7f045007b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:28.614 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.613+0000 7f046f400640 1 --2- 172.21.15.151:0/1992497551 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0450078bb0 0x7f045007b070 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f047015e890 tx=0x7f045c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:28.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.616+0000 7f0467e00640 1 -- 172.21.15.151:0/1992497551 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f046003f030 con 0x7f0470073fb0 2024-09-16T15:42:28.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.800+0000 7f0477000640 1 -- 172.21.15.151:0/1992497551 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f047009b2a0 con 0x7f0470073fb0 2024-09-16T15:42:28.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.801+0000 7f0467e00640 1 -- 172.21.15.151:0/1992497551 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) ==== 74+0+16729 (secure 0 0 0) 0x7f0460083c50 con 0x7f0470073fb0 2024-09-16T15:42:28.802 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:42:28.803 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":48,"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","created":"2024-09-16T15:38:56.348535+0000","modified":"2024-09-16T15:42:25.983537+0000","last_up_change":"2024-09-16T15:39:10.681384+0000","last_in_change":"2024-09-16T15:39:00.544477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:39:11.602252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:42:22.166447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-16T15:42:23.379899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5524076-a903-4d0a-af6a-caf474c4765d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6809","nonce":3111342741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6811","nonce":3111342741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6815","nonce":3111342741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6813","nonce":3111342741}]},"public_addr":"172.21.15.151:6809/3111342741","cluster_addr":"172.21.15.151:6811/3111342741","heartbeat_back_addr":"172.21.15.151:6815/3111342741","heartbeat_front_addr":"172.21.15.151:6813/3111342741","state":["exists","up"]},{"osd":1,"uuid":"dd25925f-0ae1-494e-8529-4dc205759db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6801","nonce":3368304087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6803","nonce":3368304087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6807","nonce":3368304087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6805","nonce":3368304087}]},"public_addr":"172.21.15.151:6801/3368304087","cluster_addr":"172.21.15.151:6803/3368304087","heartbeat_back_addr":"172.21.15.151:6807/3368304087","heartbeat_front_addr":"172.21.15.151:6805/3368304087","state":["exists","up"]},{"osd":2,"uuid":"c2f18115-013d-4da6-a0fe-cb978f4494b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6823","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6825","nonce":3079373271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6827","nonce":3079373271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6831","nonce":3079373271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6829","nonce":3079373271}]},"public_addr":"172.21.15.151:6825/3079373271","cluster_addr":"172.21.15.151:6827/3079373271","heartbeat_back_addr":"172.21.15.151:6831/3079373271","heartbeat_front_addr":"172.21.15.151:6829/3079373271","state":["exists","up"]},{"osd":3,"uuid":"36edb718-2fcb-4d88-8ddc-c321eed4279d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6817","nonce":1142509041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6819","nonce":1142509041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6824","nonce":1142509041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6821","nonce":1142509041}]},"public_addr":"172.21.15.151:6817/1142509041","cluster_addr":"172.21.15.151:6819/1142509041","heartbeat_back_addr":"172.21.15.151:6824/1142509041","heartbeat_front_addr":"172.21.15.151:6821/1142509041","state":["exists","up"]},{"osd":4,"uuid":"64d33708-cd0a-4236-abe3-1b646e82ab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6811","nonce":2735495271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6813","nonce":2735495271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6817","nonce":2735495271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6815","nonce":2735495271}]},"public_addr":"172.21.15.167:6811/2735495271","cluster_addr":"172.21.15.167:6813/2735495271","heartbeat_back_addr":"172.21.15.167:6817/2735495271","heartbeat_front_addr":"172.21.15.167:6815/2735495271","state":["exists","up"]},{"osd":5,"uuid":"f14b44a9-5910-458c-b518-09a8d0de94de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6827","nonce":1494916572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6829","nonce":1494916572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6833","nonce":1494916572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6831","nonce":1494916572}]},"public_addr":"172.21.15.167:6827/1494916572","cluster_addr":"172.21.15.167:6829/1494916572","heartbeat_back_addr":"172.21.15.167:6833/1494916572","heartbeat_front_addr":"172.21.15.167:6831/1494916572","state":["exists","up"]},{"osd":6,"uuid":"39335545-2654-4710-84b0-a9aa47cfcbba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6819","nonce":2538273273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6821","nonce":2538273273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6825","nonce":2538273273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6823","nonce":2538273273}]},"public_addr":"172.21.15.167:6819/2538273273","cluster_addr":"172.21.15.167:6821/2538273273","heartbeat_back_addr":"172.21.15.167:6825/2538273273","heartbeat_front_addr":"172.21.15.167:6823/2538273273","state":["exists","up"]},{"osd":7,"uuid":"4f8bfc81-53dd-4339-ac22-1a4007d9f963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6803","nonce":265072227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6805","nonce":265072227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6809","nonce":265072227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6807","nonce":265072227}]},"public_addr":"172.21.15.167:6803/265072227","cluster_addr":"172.21.15.167:6805/265072227","heartbeat_back_addr":"172.21.15.167:6809/265072227","heartbeat_front_addr":"172.21.15.167:6807/265072227","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.150359+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.084358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.722368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.332809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.320709+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.395153+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.935250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.801443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:42:28.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.803+0000 7f0477000640 1 -- 172.21.15.151:0/1992497551 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0450078bb0 msgr2=0x7f045007b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:28.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.803+0000 7f0477000640 1 --2- 172.21.15.151:0/1992497551 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0450078bb0 0x7f045007b070 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f047015e890 tx=0x7f045c009040 comp rx=0 tx=0).stop 2024-09-16T15:42:28.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.803+0000 7f0477000640 1 -- 172.21.15.151:0/1992497551 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0470073fb0 msgr2=0x7f04701f8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:28.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.803+0000 7f0477000640 1 --2- 172.21.15.151:0/1992497551 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0470073fb0 0x7f04701f8af0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0460000c00 tx=0x7f0460033c20 comp rx=0 tx=0).stop 2024-09-16T15:42:28.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.803+0000 7f0477000640 1 -- 172.21.15.151:0/1992497551 shutdown_connections 2024-09-16T15:42:28.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.803+0000 7f0477000640 1 --2- 172.21.15.151:0/1992497551 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0450078bb0 0x7f045007b070 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:28.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.803+0000 7f0477000640 1 --2- 172.21.15.151:0/1992497551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f047015f5a0 0x7f047014f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:28.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.803+0000 7f0477000640 1 --2- 172.21.15.151:0/1992497551 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f04700ccf00 0x7f0470156300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:28.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.803+0000 7f0477000640 1 --2- 172.21.15.151:0/1992497551 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0470073fb0 0x7f04701f8af0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:28.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.803+0000 7f0477000640 1 -- 172.21.15.151:0/1992497551 >> 172.21.15.151:0/1992497551 conn(0x7f04700b1950 msgr2=0x7f04700b3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:28.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.803+0000 7f0477000640 1 -- 172.21.15.151:0/1992497551 shutdown_connections 2024-09-16T15:42:28.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.803+0000 7f0477000640 1 -- 172.21.15.151:0/1992497551 wait complete. 2024-09-16T15:42:28.822 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T15:42:28.822 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:42:28.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.978+0000 7fdaf7400640 1 -- 172.21.15.151:0/1033140642 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdaf015f5a0 msgr2=0x7fdaf00b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:28.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.978+0000 7fdaf7400640 1 --2- 172.21.15.151:0/1033140642 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdaf015f5a0 0x7fdaf00b2440 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fdaf00cfbb0 tx=0x7fdae402a4d0 comp rx=0 tx=0).stop 2024-09-16T15:42:28.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.978+0000 7fdaf7400640 1 -- 172.21.15.151:0/1033140642 shutdown_connections 2024-09-16T15:42:28.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.978+0000 7fdaf7400640 1 --2- 172.21.15.151:0/1033140642 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdaf00ad270 0x7fdaf0073fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:28.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.978+0000 7fdaf7400640 1 --2- 172.21.15.151:0/1033140642 >> v2:172.21.15.167:3300/0 conn(0x7fdaf00ccf00 0x7fdaf00acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:28.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.978+0000 7fdaf7400640 1 --2- 172.21.15.151:0/1033140642 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdaf015f5a0 0x7fdaf00b2440 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:28.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.978+0000 7fdaf7400640 1 -- 172.21.15.151:0/1033140642 >> 172.21.15.151:0/1033140642 conn(0x7fdaf00b1950 msgr2=0x7fdaf00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:28.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.978+0000 7fdaf7400640 1 -- 172.21.15.151:0/1033140642 shutdown_connections 2024-09-16T15:42:28.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.978+0000 7fdaf7400640 1 -- 172.21.15.151:0/1033140642 wait complete. 2024-09-16T15:42:28.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.980+0000 7fdaf7400640 1 Processor -- start 2024-09-16T15:42:28.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.980+0000 7fdaf7400640 1 -- start start 2024-09-16T15:42:28.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.981+0000 7fdaf7400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdaf00ad270 0x7fdaf00cff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:28.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.981+0000 7fdaf7400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdaf00ccf00 0x7fdaf0156780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:28.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.981+0000 7fdaf7400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdaf015f5a0 0x7fdaf0156cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:28.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.981+0000 7fdaf7400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf0074880 con 0x7fdaf00ccf00 2024-09-16T15:42:28.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.981+0000 7fdaf7400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf0074700 con 0x7fdaf00ad270 2024-09-16T15:42:28.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.981+0000 7fdaf7400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fdaf006f160 con 0x7fdaf015f5a0 2024-09-16T15:42:28.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.981+0000 7fdaefe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdaf00ccf00 0x7fdaf0156780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:28.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.981+0000 7fdaefe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdaf00ccf00 0x7fdaf0156780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49944/0 (socket says 172.21.15.151:49944) 2024-09-16T15:42:28.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.981+0000 7fdaefe00640 1 -- 172.21.15.151:0/2442894829 learned_addr learned my addr 172.21.15.151:0/2442894829 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:42:28.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.981+0000 7fdaf4a00640 1 --2- 172.21.15.151:0/2442894829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdaf00ad270 0x7fdaf00cff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:28.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.981+0000 7fdaf5400640 1 --2- 172.21.15.151:0/2442894829 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdaf015f5a0 0x7fdaf0156cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:28.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.981+0000 7fdaefe00640 1 -- 172.21.15.151:0/2442894829 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdaf015f5a0 msgr2=0x7fdaf0156cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:28.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.981+0000 7fdaefe00640 1 --2- 172.21.15.151:0/2442894829 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdaf015f5a0 0x7fdaf0156cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:28.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.981+0000 7fdaefe00640 1 -- 172.21.15.151:0/2442894829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdaf00ad270 msgr2=0x7fdaf00cff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:28.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.981+0000 7fdaefe00640 1 --2- 172.21.15.151:0/2442894829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdaf00ad270 0x7fdaf00cff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:28.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.981+0000 7fdaefe00640 1 -- 172.21.15.151:0/2442894829 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaf01f5b80 con 0x7fdaf00ccf00 2024-09-16T15:42:28.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.982+0000 7fdaefe00640 1 --2- 172.21.15.151:0/2442894829 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdaf00ccf00 0x7fdaf0156780 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fdadc00d950 tx=0x7fdadc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:28.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.982+0000 7fdaecc00640 1 -- 172.21.15.151:0/2442894829 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdadc017070 con 0x7fdaf00ccf00 2024-09-16T15:42:28.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.982+0000 7fdaecc00640 1 -- 172.21.15.151:0/2442894829 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdadc009d20 con 0x7fdaf00ccf00 2024-09-16T15:42:28.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.982+0000 7fdaecc00640 1 -- 172.21.15.151:0/2442894829 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdadc004280 con 0x7fdaf00ccf00 2024-09-16T15:42:28.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.982+0000 7fdaf7400640 1 -- 172.21.15.151:0/2442894829 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaf0157350 con 0x7fdaf00ccf00 2024-09-16T15:42:28.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.982+0000 7fdaf7400640 1 -- 172.21.15.151:0/2442894829 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaf01f5380 con 0x7fdaf00ccf00 2024-09-16T15:42:28.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.983+0000 7fdaf7400640 1 -- 172.21.15.151:0/2442894829 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaf00b31a0 con 0x7fdaf00ccf00 2024-09-16T15:42:28.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.984+0000 7fdaecc00640 1 -- 172.21.15.151:0/2442894829 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7fdadc0035f0 con 0x7fdaf00ccf00 2024-09-16T15:42:28.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.985+0000 7fdaecc00640 1 --2- 172.21.15.151:0/2442894829 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdad0078bb0 0x7fdad007b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:28.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.985+0000 7fdaecc00640 1 -- 172.21.15.151:0/2442894829 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7fdadc09b1f0 con 0x7fdaf00ccf00 2024-09-16T15:42:28.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.986+0000 7fdaf4a00640 1 --2- 172.21.15.151:0/2442894829 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdad0078bb0 0x7fdad007b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:28.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.986+0000 7fdaf4a00640 1 --2- 172.21.15.151:0/2442894829 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdad0078bb0 0x7fdad007b070 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdae4033000 tx=0x7fdae4004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:28.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:28.989+0000 7fdaecc00640 1 -- 172.21.15.151:0/2442894829 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fdaf00b31a0 con 0x7fdaf00ccf00 2024-09-16T15:42:29.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.170+0000 7fdaf7400640 1 -- 172.21.15.151:0/2442894829 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdaf009b3d0 con 0x7fdad0078bb0 2024-09-16T15:42:29.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.175+0000 7fdaecc00640 1 -- 172.21.15.151:0/2442894829 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251449 (secure 0 0 0) 0x7fdaf009b3d0 con 0x7fdad0078bb0 2024-09-16T15:42:29.180 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:42:29.181 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-09-16T15:42:29.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.180+0000 7fdaf7400640 1 -- 172.21.15.151:0/2442894829 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdad0078bb0 msgr2=0x7fdad007b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:29.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.180+0000 7fdaf7400640 1 --2- 172.21.15.151:0/2442894829 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdad0078bb0 0x7fdad007b070 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdae4033000 tx=0x7fdae4004730 comp rx=0 tx=0).stop 2024-09-16T15:42:29.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.180+0000 7fdaf7400640 1 -- 172.21.15.151:0/2442894829 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdaf00ccf00 msgr2=0x7fdaf0156780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:29.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.180+0000 7fdaf7400640 1 --2- 172.21.15.151:0/2442894829 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdaf00ccf00 0x7fdaf0156780 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fdadc00d950 tx=0x7fdadc00de10 comp rx=0 tx=0).stop 2024-09-16T15:42:29.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.180+0000 7fdaf7400640 1 -- 172.21.15.151:0/2442894829 shutdown_connections 2024-09-16T15:42:29.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.180+0000 7fdaf7400640 1 --2- 172.21.15.151:0/2442894829 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdad0078bb0 0x7fdad007b070 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:29.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.180+0000 7fdaf7400640 1 --2- 172.21.15.151:0/2442894829 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdaf015f5a0 0x7fdaf0156cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:29.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.180+0000 7fdaf7400640 1 --2- 172.21.15.151:0/2442894829 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdaf00ccf00 0x7fdaf0156780 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:29.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.180+0000 7fdaf7400640 1 --2- 172.21.15.151:0/2442894829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdaf00ad270 0x7fdaf00cff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:29.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.180+0000 7fdaf7400640 1 -- 172.21.15.151:0/2442894829 >> 172.21.15.151:0/2442894829 conn(0x7fdaf00b1950 msgr2=0x7fdaf00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:29.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.180+0000 7fdaf7400640 1 -- 172.21.15.151:0/2442894829 shutdown_connections 2024-09-16T15:42:29.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.180+0000 7fdaf7400640 1 -- 172.21.15.151:0/2442894829 wait complete. 2024-09-16T15:42:29.196 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":140,"stamp":"2024-09-16T15:42:27.624647+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592744,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":142,"num_write_kb":1635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":108,"ondisk_log_size":108,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":109,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259116,"kb_used_data":5436,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754715604,"statfs":{"total":773094113280,"available":772828778496,"internally_reserved":0,"allocated":5566464,"data_stored":2933211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12738,"internal_metadata":259575358},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2376,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.648484"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.026564+0000","last_change":"2024-09-16T15:42:25.021065+0000","last_active":"2024-09-16T15:42:25.026564+0000","last_peered":"2024-09-16T15:42:25.026564+0000","last_clean":"2024-09-16T15:42:25.026564+0000","last_became_active":"2024-09-16T15:42:25.020615+0000","last_became_peered":"2024-09-16T15:42:25.020615+0000","last_unstale":"2024-09-16T15:42:25.026564+0000","last_undegraded":"2024-09-16T15:42:25.026564+0000","last_fullsized":"2024-09-16T15:42:25.026564+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:42:34.040967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.012261+0000","last_change":"2024-09-16T15:42:25.006484+0000","last_active":"2024-09-16T15:42:25.012261+0000","last_peered":"2024-09-16T15:42:25.012261+0000","last_clean":"2024-09-16T15:42:25.012261+0000","last_became_active":"2024-09-16T15:42:25.006265+0000","last_became_peered":"2024-09-16T15:42:25.006265+0000","last_unstale":"2024-09-16T15:42:25.012261+0000","last_undegraded":"2024-09-16T15:42:25.012261+0000","last_fullsized":"2024-09-16T15:42:25.012261+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:01:36.409724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.023590+0000","last_change":"2024-09-16T15:42:25.015717+0000","last_active":"2024-09-16T15:42:25.023590+0000","last_peered":"2024-09-16T15:42:25.023590+0000","last_clean":"2024-09-16T15:42:25.023590+0000","last_became_active":"2024-09-16T15:42:25.006423+0000","last_became_peered":"2024-09-16T15:42:25.006423+0000","last_unstale":"2024-09-16T15:42:25.023590+0000","last_undegraded":"2024-09-16T15:42:25.023590+0000","last_fullsized":"2024-09-16T15:42:25.023590+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:06:46.712953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.018165+0000","last_change":"2024-09-16T15:42:25.005437+0000","last_active":"2024-09-16T15:42:25.018165+0000","last_peered":"2024-09-16T15:42:25.018165+0000","last_clean":"2024-09-16T15:42:25.018165+0000","last_became_active":"2024-09-16T15:42:25.005104+0000","last_became_peered":"2024-09-16T15:42:25.005104+0000","last_unstale":"2024-09-16T15:42:25.018165+0000","last_undegraded":"2024-09-16T15:42:25.018165+0000","last_fullsized":"2024-09-16T15:42:25.018165+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:40:43.407767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.026346+0000","last_change":"2024-09-16T15:42:25.015162+0000","last_active":"2024-09-16T15:42:25.026346+0000","last_peered":"2024-09-16T15:42:25.026346+0000","last_clean":"2024-09-16T15:42:25.026346+0000","last_became_active":"2024-09-16T15:42:25.014915+0000","last_became_peered":"2024-09-16T15:42:25.014915+0000","last_unstale":"2024-09-16T15:42:25.026346+0000","last_undegraded":"2024-09-16T15:42:25.026346+0000","last_fullsized":"2024-09-16T15:42:25.026346+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:05.773773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.022662+0000","last_change":"2024-09-16T15:42:25.018752+0000","last_active":"2024-09-16T15:42:25.022662+0000","last_peered":"2024-09-16T15:42:25.022662+0000","last_clean":"2024-09-16T15:42:25.022662+0000","last_became_active":"2024-09-16T15:42:25.016661+0000","last_became_peered":"2024-09-16T15:42:25.016661+0000","last_unstale":"2024-09-16T15:42:25.022662+0000","last_undegraded":"2024-09-16T15:42:25.022662+0000","last_fullsized":"2024-09-16T15:42:25.022662+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:10:05.060630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.007667+0000","last_change":"2024-09-16T15:42:25.005098+0000","last_active":"2024-09-16T15:42:25.007667+0000","last_peered":"2024-09-16T15:42:25.007667+0000","last_clean":"2024-09-16T15:42:25.007667+0000","last_became_active":"2024-09-16T15:42:25.004817+0000","last_became_peered":"2024-09-16T15:42:25.004817+0000","last_unstale":"2024-09-16T15:42:25.007667+0000","last_undegraded":"2024-09-16T15:42:25.007667+0000","last_fullsized":"2024-09-16T15:42:25.007667+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:12:07.715267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.016184+0000","last_change":"2024-09-16T15:42:25.003653+0000","last_active":"2024-09-16T15:42:25.016184+0000","last_peered":"2024-09-16T15:42:25.016184+0000","last_clean":"2024-09-16T15:42:25.016184+0000","last_became_active":"2024-09-16T15:42:25.002998+0000","last_became_peered":"2024-09-16T15:42:25.002998+0000","last_unstale":"2024-09-16T15:42:25.016184+0000","last_undegraded":"2024-09-16T15:42:25.016184+0000","last_fullsized":"2024-09-16T15:42:25.016184+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:23:19.274003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.022272+0000","last_change":"2024-09-16T15:42:25.013856+0000","last_active":"2024-09-16T15:42:25.022272+0000","last_peered":"2024-09-16T15:42:25.022272+0000","last_clean":"2024-09-16T15:42:25.022272+0000","last_became_active":"2024-09-16T15:42:25.013394+0000","last_became_peered":"2024-09-16T15:42:25.013394+0000","last_unstale":"2024-09-16T15:42:25.022272+0000","last_undegraded":"2024-09-16T15:42:25.022272+0000","last_fullsized":"2024-09-16T15:42:25.022272+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:01:02.223927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.017434+0000","last_change":"2024-09-16T15:42:25.012665+0000","last_active":"2024-09-16T15:42:25.017434+0000","last_peered":"2024-09-16T15:42:25.017434+0000","last_clean":"2024-09-16T15:42:25.017434+0000","last_became_active":"2024-09-16T15:42:25.012429+0000","last_became_peered":"2024-09-16T15:42:25.012429+0000","last_unstale":"2024-09-16T15:42:25.017434+0000","last_undegraded":"2024-09-16T15:42:25.017434+0000","last_fullsized":"2024-09-16T15:42:25.017434+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:06:26.478901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.017884+0000","last_change":"2024-09-16T15:42:25.011076+0000","last_active":"2024-09-16T15:42:25.017884+0000","last_peered":"2024-09-16T15:42:25.017884+0000","last_clean":"2024-09-16T15:42:25.017884+0000","last_became_active":"2024-09-16T15:42:25.010704+0000","last_became_peered":"2024-09-16T15:42:25.010704+0000","last_unstale":"2024-09-16T15:42:25.017884+0000","last_undegraded":"2024-09-16T15:42:25.017884+0000","last_fullsized":"2024-09-16T15:42:25.017884+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:32:58.789647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.012380+0000","last_change":"2024-09-16T15:42:25.007233+0000","last_active":"2024-09-16T15:42:25.012380+0000","last_peered":"2024-09-16T15:42:25.012380+0000","last_clean":"2024-09-16T15:42:25.012380+0000","last_became_active":"2024-09-16T15:42:25.006852+0000","last_became_peered":"2024-09-16T15:42:25.006852+0000","last_unstale":"2024-09-16T15:42:25.012380+0000","last_undegraded":"2024-09-16T15:42:25.012380+0000","last_fullsized":"2024-09-16T15:42:25.012380+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:24:38.283829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.026402+0000","last_change":"2024-09-16T15:42:25.023442+0000","last_active":"2024-09-16T15:42:25.026402+0000","last_peered":"2024-09-16T15:42:25.026402+0000","last_clean":"2024-09-16T15:42:25.026402+0000","last_became_active":"2024-09-16T15:42:25.022319+0000","last_became_peered":"2024-09-16T15:42:25.022319+0000","last_unstale":"2024-09-16T15:42:25.026402+0000","last_undegraded":"2024-09-16T15:42:25.026402+0000","last_fullsized":"2024-09-16T15:42:25.026402+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:47:44.920547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.016169+0000","last_change":"2024-09-16T15:42:25.001358+0000","last_active":"2024-09-16T15:42:25.016169+0000","last_peered":"2024-09-16T15:42:25.016169+0000","last_clean":"2024-09-16T15:42:25.016169+0000","last_became_active":"2024-09-16T15:42:25.001076+0000","last_became_peered":"2024-09-16T15:42:25.001076+0000","last_unstale":"2024-09-16T15:42:25.016169+0000","last_undegraded":"2024-09-16T15:42:25.016169+0000","last_fullsized":"2024-09-16T15:42:25.016169+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:47:44.920547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.018055+0000","last_change":"2024-09-16T15:42:25.012888+0000","last_active":"2024-09-16T15:42:25.018055+0000","last_peered":"2024-09-16T15:42:25.018055+0000","last_clean":"2024-09-16T15:42:25.018055+0000","last_became_active":"2024-09-16T15:42:25.009003+0000","last_became_peered":"2024-09-16T15:42:25.009003+0000","last_unstale":"2024-09-16T15:42:25.018055+0000","last_undegraded":"2024-09-16T15:42:25.018055+0000","last_fullsized":"2024-09-16T15:42:25.018055+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:25:19.893903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.026416+0000","last_change":"2024-09-16T15:42:25.017302+0000","last_active":"2024-09-16T15:42:25.026416+0000","last_peered":"2024-09-16T15:42:25.026416+0000","last_clean":"2024-09-16T15:42:25.026416+0000","last_became_active":"2024-09-16T15:42:25.017045+0000","last_became_peered":"2024-09-16T15:42:25.017045+0000","last_unstale":"2024-09-16T15:42:25.026416+0000","last_undegraded":"2024-09-16T15:42:25.026416+0000","last_fullsized":"2024-09-16T15:42:25.026416+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:32:58.789647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.027927+0000","last_change":"2024-09-16T15:42:25.021592+0000","last_active":"2024-09-16T15:42:25.027927+0000","last_peered":"2024-09-16T15:42:25.027927+0000","last_clean":"2024-09-16T15:42:25.027927+0000","last_became_active":"2024-09-16T15:42:25.021274+0000","last_became_peered":"2024-09-16T15:42:25.021274+0000","last_unstale":"2024-09-16T15:42:25.027927+0000","last_undegraded":"2024-09-16T15:42:25.027927+0000","last_fullsized":"2024-09-16T15:42:25.027927+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:15:26.922530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.004788+0000","last_change":"2024-09-16T15:42:24.995956+0000","last_active":"2024-09-16T15:42:25.004788+0000","last_peered":"2024-09-16T15:42:25.004788+0000","last_clean":"2024-09-16T15:42:25.004788+0000","last_became_active":"2024-09-16T15:42:24.995598+0000","last_became_peered":"2024-09-16T15:42:24.995598+0000","last_unstale":"2024-09-16T15:42:25.004788+0000","last_undegraded":"2024-09-16T15:42:25.004788+0000","last_fullsized":"2024-09-16T15:42:25.004788+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:13:48.480802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.029228+0000","last_change":"2024-09-16T15:42:25.024290+0000","last_active":"2024-09-16T15:42:25.029228+0000","last_peered":"2024-09-16T15:42:25.029228+0000","last_clean":"2024-09-16T15:42:25.029228+0000","last_became_active":"2024-09-16T15:42:25.024029+0000","last_became_peered":"2024-09-16T15:42:25.024029+0000","last_unstale":"2024-09-16T15:42:25.029228+0000","last_undegraded":"2024-09-16T15:42:25.029228+0000","last_fullsized":"2024-09-16T15:42:25.029228+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:44:41.840907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.011900+0000","last_change":"2024-09-16T15:42:25.009320+0000","last_active":"2024-09-16T15:42:25.011900+0000","last_peered":"2024-09-16T15:42:25.011900+0000","last_clean":"2024-09-16T15:42:25.011900+0000","last_became_active":"2024-09-16T15:42:25.008705+0000","last_became_peered":"2024-09-16T15:42:25.008705+0000","last_unstale":"2024-09-16T15:42:25.011900+0000","last_undegraded":"2024-09-16T15:42:25.011900+0000","last_fullsized":"2024-09-16T15:42:25.011900+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:25:11.105280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.022645+0000","last_change":"2024-09-16T15:42:25.014765+0000","last_active":"2024-09-16T15:42:25.022645+0000","last_peered":"2024-09-16T15:42:25.022645+0000","last_clean":"2024-09-16T15:42:25.022645+0000","last_became_active":"2024-09-16T15:42:25.009165+0000","last_became_peered":"2024-09-16T15:42:25.009165+0000","last_unstale":"2024-09-16T15:42:25.022645+0000","last_undegraded":"2024-09-16T15:42:25.022645+0000","last_fullsized":"2024-09-16T15:42:25.022645+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:44:41.840907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-16T15:42:23.991455+0000","last_change":"2024-09-16T15:42:23.991455+0000","last_active":"2024-09-16T15:42:23.975506+0000","last_peered":"2024-09-16T15:42:23.975506+0000","last_clean":"2024-09-16T15:42:23.975506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:42:23.991455+0000","last_undegraded":"2024-09-16T15:42:23.991455+0000","last_fullsized":"2024-09-16T15:42:23.991455+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.028105+0000","last_change":"2024-09-16T15:42:25.024263+0000","last_active":"2024-09-16T15:42:25.028105+0000","last_peered":"2024-09-16T15:42:25.028105+0000","last_clean":"2024-09-16T15:42:25.028105+0000","last_became_active":"2024-09-16T15:42:25.024006+0000","last_became_peered":"2024-09-16T15:42:25.024006+0000","last_unstale":"2024-09-16T15:42:25.028105+0000","last_undegraded":"2024-09-16T15:42:25.028105+0000","last_fullsized":"2024-09-16T15:42:25.028105+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:07:07.004615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.025458+0000","last_change":"2024-09-16T15:42:25.018894+0000","last_active":"2024-09-16T15:42:25.025458+0000","last_peered":"2024-09-16T15:42:25.025458+0000","last_clean":"2024-09-16T15:42:25.025458+0000","last_became_active":"2024-09-16T15:42:25.018023+0000","last_became_peered":"2024-09-16T15:42:25.018023+0000","last_unstale":"2024-09-16T15:42:25.025458+0000","last_undegraded":"2024-09-16T15:42:25.025458+0000","last_fullsized":"2024-09-16T15:42:25.025458+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:22:47.419249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.026541+0000","last_change":"2024-09-16T15:42:25.019057+0000","last_active":"2024-09-16T15:42:25.026541+0000","last_peered":"2024-09-16T15:42:25.026541+0000","last_clean":"2024-09-16T15:42:25.026541+0000","last_became_active":"2024-09-16T15:42:25.018803+0000","last_became_peered":"2024-09-16T15:42:25.018803+0000","last_unstale":"2024-09-16T15:42:25.026541+0000","last_undegraded":"2024-09-16T15:42:25.026541+0000","last_fullsized":"2024-09-16T15:42:25.026541+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:25:19.893903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.026062+0000","last_change":"2024-09-16T15:42:25.022339+0000","last_active":"2024-09-16T15:42:25.026062+0000","last_peered":"2024-09-16T15:42:25.026062+0000","last_clean":"2024-09-16T15:42:25.026062+0000","last_became_active":"2024-09-16T15:42:25.021986+0000","last_became_peered":"2024-09-16T15:42:25.021986+0000","last_unstale":"2024-09-16T15:42:25.026062+0000","last_undegraded":"2024-09-16T15:42:25.026062+0000","last_fullsized":"2024-09-16T15:42:25.026062+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:29:45.056795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.020215+0000","last_change":"2024-09-16T15:42:25.016721+0000","last_active":"2024-09-16T15:42:25.020215+0000","last_peered":"2024-09-16T15:42:25.020215+0000","last_clean":"2024-09-16T15:42:25.020215+0000","last_became_active":"2024-09-16T15:42:25.016383+0000","last_became_peered":"2024-09-16T15:42:25.016383+0000","last_unstale":"2024-09-16T15:42:25.020215+0000","last_undegraded":"2024-09-16T15:42:25.020215+0000","last_fullsized":"2024-09-16T15:42:25.020215+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:55:33.513714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.014293+0000","last_change":"2024-09-16T15:42:25.011808+0000","last_active":"2024-09-16T15:42:25.014293+0000","last_peered":"2024-09-16T15:42:25.014293+0000","last_clean":"2024-09-16T15:42:25.014293+0000","last_became_active":"2024-09-16T15:42:25.011410+0000","last_became_peered":"2024-09-16T15:42:25.011410+0000","last_unstale":"2024-09-16T15:42:25.014293+0000","last_undegraded":"2024-09-16T15:42:25.014293+0000","last_fullsized":"2024-09-16T15:42:25.014293+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:55:58.347779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.026371+0000","last_change":"2024-09-16T15:42:25.020917+0000","last_active":"2024-09-16T15:42:25.026371+0000","last_peered":"2024-09-16T15:42:25.026371+0000","last_clean":"2024-09-16T15:42:25.026371+0000","last_became_active":"2024-09-16T15:42:25.016487+0000","last_became_peered":"2024-09-16T15:42:25.016487+0000","last_unstale":"2024-09-16T15:42:25.026371+0000","last_undegraded":"2024-09-16T15:42:25.026371+0000","last_fullsized":"2024-09-16T15:42:25.026371+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:16:20.188903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.015755+0000","last_change":"2024-09-16T15:42:25.008914+0000","last_active":"2024-09-16T15:42:25.015755+0000","last_peered":"2024-09-16T15:42:25.015755+0000","last_clean":"2024-09-16T15:42:25.015755+0000","last_became_active":"2024-09-16T15:42:25.008631+0000","last_became_peered":"2024-09-16T15:42:25.008631+0000","last_unstale":"2024-09-16T15:42:25.015755+0000","last_undegraded":"2024-09-16T15:42:25.015755+0000","last_fullsized":"2024-09-16T15:42:25.015755+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:55:06.255740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.027325+0000","last_change":"2024-09-16T15:42:25.024127+0000","last_active":"2024-09-16T15:42:25.027325+0000","last_peered":"2024-09-16T15:42:25.027325+0000","last_clean":"2024-09-16T15:42:25.027325+0000","last_became_active":"2024-09-16T15:42:25.023603+0000","last_became_peered":"2024-09-16T15:42:25.023603+0000","last_unstale":"2024-09-16T15:42:25.027325+0000","last_undegraded":"2024-09-16T15:42:25.027325+0000","last_fullsized":"2024-09-16T15:42:25.027325+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:23:19.274003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.017833+0000","last_change":"2024-09-16T15:42:25.006974+0000","last_active":"2024-09-16T15:42:25.017833+0000","last_peered":"2024-09-16T15:42:25.017833+0000","last_clean":"2024-09-16T15:42:25.017833+0000","last_became_active":"2024-09-16T15:42:25.006382+0000","last_became_peered":"2024-09-16T15:42:25.006382+0000","last_unstale":"2024-09-16T15:42:25.017833+0000","last_undegraded":"2024-09-16T15:42:25.017833+0000","last_fullsized":"2024-09-16T15:42:25.017833+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:05.773773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-16T15:42:23.991805+0000","last_change":"2024-09-16T15:42:23.991805+0000","last_active":"2024-09-16T15:42:23.975506+0000","last_peered":"2024-09-16T15:42:23.975506+0000","last_clean":"2024-09-16T15:42:23.975506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:42:23.991805+0000","last_undegraded":"2024-09-16T15:42:23.991805+0000","last_fullsized":"2024-09-16T15:42:23.991805+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"47'4","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.027253+0000","last_change":"2024-09-16T15:42:24.001081+0000","last_active":"2024-09-16T15:42:25.027253+0000","last_peered":"2024-09-16T15:42:25.027253+0000","last_clean":"2024-09-16T15:42:25.027253+0000","last_became_active":"2024-09-16T15:42:24.000071+0000","last_became_peered":"2024-09-16T15:42:24.000071+0000","last_unstale":"2024-09-16T15:42:25.027253+0000","last_undegraded":"2024-09-16T15:42:25.027253+0000","last_fullsized":"2024-09-16T15:42:25.027253+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:24:51.105941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.026516+0000","last_change":"2024-09-16T15:42:25.016739+0000","last_active":"2024-09-16T15:42:25.026516+0000","last_peered":"2024-09-16T15:42:25.026516+0000","last_clean":"2024-09-16T15:42:25.026516+0000","last_became_active":"2024-09-16T15:42:25.016451+0000","last_became_peered":"2024-09-16T15:42:25.016451+0000","last_unstale":"2024-09-16T15:42:25.026516+0000","last_undegraded":"2024-09-16T15:42:25.026516+0000","last_fullsized":"2024-09-16T15:42:25.026516+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:05:52.459658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"47'1","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.016363+0000","last_change":"2024-09-16T15:42:24.000482+0000","last_active":"2024-09-16T15:42:25.016363+0000","last_peered":"2024-09-16T15:42:25.016363+0000","last_clean":"2024-09-16T15:42:25.016363+0000","last_became_active":"2024-09-16T15:42:24.000216+0000","last_became_peered":"2024-09-16T15:42:24.000216+0000","last_unstale":"2024-09-16T15:42:25.016363+0000","last_undegraded":"2024-09-16T15:42:25.016363+0000","last_fullsized":"2024-09-16T15:42:25.016363+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:41:02.855031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.012164+0000","last_change":"2024-09-16T15:42:25.005828+0000","last_active":"2024-09-16T15:42:25.012164+0000","last_peered":"2024-09-16T15:42:25.012164+0000","last_clean":"2024-09-16T15:42:25.012164+0000","last_became_active":"2024-09-16T15:42:25.005398+0000","last_became_peered":"2024-09-16T15:42:25.005398+0000","last_unstale":"2024-09-16T15:42:25.012164+0000","last_undegraded":"2024-09-16T15:42:25.012164+0000","last_fullsized":"2024-09-16T15:42:25.012164+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:43:52.404756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"47'2","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.025469+0000","last_change":"2024-09-16T15:42:24.002867+0000","last_active":"2024-09-16T15:42:25.025469+0000","last_peered":"2024-09-16T15:42:25.025469+0000","last_clean":"2024-09-16T15:42:25.025469+0000","last_became_active":"2024-09-16T15:42:24.002523+0000","last_became_peered":"2024-09-16T15:42:24.002523+0000","last_unstale":"2024-09-16T15:42:25.025469+0000","last_undegraded":"2024-09-16T15:42:25.025469+0000","last_fullsized":"2024-09-16T15:42:25.025469+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:57:21.332836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.016986+0000","last_change":"2024-09-16T15:42:25.006017+0000","last_active":"2024-09-16T15:42:25.016986+0000","last_peered":"2024-09-16T15:42:25.016986+0000","last_clean":"2024-09-16T15:42:25.016986+0000","last_became_active":"2024-09-16T15:42:25.005752+0000","last_became_peered":"2024-09-16T15:42:25.005752+0000","last_unstale":"2024-09-16T15:42:25.016986+0000","last_undegraded":"2024-09-16T15:42:25.016986+0000","last_fullsized":"2024-09-16T15:42:25.016986+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:49:11.296322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"47'2","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.013054+0000","last_change":"2024-09-16T15:42:24.002580+0000","last_active":"2024-09-16T15:42:25.013054+0000","last_peered":"2024-09-16T15:42:25.013054+0000","last_clean":"2024-09-16T15:42:25.013054+0000","last_became_active":"2024-09-16T15:42:24.000516+0000","last_became_peered":"2024-09-16T15:42:24.000516+0000","last_unstale":"2024-09-16T15:42:25.013054+0000","last_undegraded":"2024-09-16T15:42:25.013054+0000","last_fullsized":"2024-09-16T15:42:25.013054+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:34:18.490825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.015597+0000","last_change":"2024-09-16T15:42:25.014465+0000","last_active":"2024-09-16T15:42:25.015597+0000","last_peered":"2024-09-16T15:42:25.015597+0000","last_clean":"2024-09-16T15:42:25.015597+0000","last_became_active":"2024-09-16T15:42:25.013555+0000","last_became_peered":"2024-09-16T15:42:25.013555+0000","last_unstale":"2024-09-16T15:42:25.015597+0000","last_undegraded":"2024-09-16T15:42:25.015597+0000","last_fullsized":"2024-09-16T15:42:25.015597+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:34:09.462316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"47'1","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.019842+0000","last_change":"2024-09-16T15:42:23.997791+0000","last_active":"2024-09-16T15:42:25.019842+0000","last_peered":"2024-09-16T15:42:25.019842+0000","last_clean":"2024-09-16T15:42:25.019842+0000","last_became_active":"2024-09-16T15:42:23.997514+0000","last_became_peered":"2024-09-16T15:42:23.997514+0000","last_unstale":"2024-09-16T15:42:25.019842+0000","last_undegraded":"2024-09-16T15:42:25.019842+0000","last_fullsized":"2024-09-16T15:42:25.019842+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:59:17.932649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.000501+0000","last_change":"2024-09-16T15:42:24.996753+0000","last_active":"2024-09-16T15:42:25.000501+0000","last_peered":"2024-09-16T15:42:25.000501+0000","last_clean":"2024-09-16T15:42:25.000501+0000","last_became_active":"2024-09-16T15:42:24.996402+0000","last_became_peered":"2024-09-16T15:42:24.996402+0000","last_unstale":"2024-09-16T15:42:25.000501+0000","last_undegraded":"2024-09-16T15:42:25.000501+0000","last_fullsized":"2024-09-16T15:42:25.000501+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:26:15.252054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"47'1","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.026472+0000","last_change":"2024-09-16T15:42:23.999519+0000","last_active":"2024-09-16T15:42:25.026472+0000","last_peered":"2024-09-16T15:42:25.026472+0000","last_clean":"2024-09-16T15:42:25.026472+0000","last_became_active":"2024-09-16T15:42:23.999239+0000","last_became_peered":"2024-09-16T15:42:23.999239+0000","last_unstale":"2024-09-16T15:42:25.026472+0000","last_undegraded":"2024-09-16T15:42:25.026472+0000","last_fullsized":"2024-09-16T15:42:25.026472+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:16:13.768386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.002801+0000","last_change":"2024-09-16T15:42:24.999204+0000","last_active":"2024-09-16T15:42:25.002801+0000","last_peered":"2024-09-16T15:42:25.002801+0000","last_clean":"2024-09-16T15:42:25.002801+0000","last_became_active":"2024-09-16T15:42:24.998947+0000","last_became_peered":"2024-09-16T15:42:24.998947+0000","last_unstale":"2024-09-16T15:42:25.002801+0000","last_undegraded":"2024-09-16T15:42:25.002801+0000","last_fullsized":"2024-09-16T15:42:25.002801+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:31:38.819252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.005366+0000","last_change":"2024-09-16T15:42:24.003842+0000","last_active":"2024-09-16T15:42:25.005366+0000","last_peered":"2024-09-16T15:42:25.005366+0000","last_clean":"2024-09-16T15:42:25.005366+0000","last_became_active":"2024-09-16T15:42:24.003517+0000","last_became_peered":"2024-09-16T15:42:24.003517+0000","last_unstale":"2024-09-16T15:42:25.005366+0000","last_undegraded":"2024-09-16T15:42:25.005366+0000","last_fullsized":"2024-09-16T15:42:25.005366+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:40:42.405299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.013648+0000","last_change":"2024-09-16T15:42:25.010354+0000","last_active":"2024-09-16T15:42:25.013648+0000","last_peered":"2024-09-16T15:42:25.013648+0000","last_clean":"2024-09-16T15:42:25.013648+0000","last_became_active":"2024-09-16T15:42:25.010088+0000","last_became_peered":"2024-09-16T15:42:25.010088+0000","last_unstale":"2024-09-16T15:42:25.013648+0000","last_undegraded":"2024-09-16T15:42:25.013648+0000","last_fullsized":"2024-09-16T15:42:25.013648+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:27:39.904292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:24.990744+0000","last_change":"2024-09-16T15:42:24.006237+0000","last_active":"2024-09-16T15:42:24.990744+0000","last_peered":"2024-09-16T15:42:24.990744+0000","last_clean":"2024-09-16T15:42:24.990744+0000","last_became_active":"2024-09-16T15:42:24.005943+0000","last_became_peered":"2024-09-16T15:42:24.005943+0000","last_unstale":"2024-09-16T15:42:24.990744+0000","last_undegraded":"2024-09-16T15:42:24.990744+0000","last_fullsized":"2024-09-16T15:42:24.990744+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:49:26.725206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.009972+0000","last_change":"2024-09-16T15:42:25.005518+0000","last_active":"2024-09-16T15:42:25.009972+0000","last_peered":"2024-09-16T15:42:25.009972+0000","last_clean":"2024-09-16T15:42:25.009972+0000","last_became_active":"2024-09-16T15:42:25.005266+0000","last_became_peered":"2024-09-16T15:42:25.005266+0000","last_unstale":"2024-09-16T15:42:25.009972+0000","last_undegraded":"2024-09-16T15:42:25.009972+0000","last_fullsized":"2024-09-16T15:42:25.009972+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:23:36.266102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:24.989354+0000","last_change":"2024-09-16T15:42:23.997205+0000","last_active":"2024-09-16T15:42:24.989354+0000","last_peered":"2024-09-16T15:42:24.989354+0000","last_clean":"2024-09-16T15:42:24.989354+0000","last_became_active":"2024-09-16T15:42:23.996928+0000","last_became_peered":"2024-09-16T15:42:23.996928+0000","last_unstale":"2024-09-16T15:42:24.989354+0000","last_undegraded":"2024-09-16T15:42:24.989354+0000","last_fullsized":"2024-09-16T15:42:24.989354+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:17:58.230738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.026385+0000","last_change":"2024-09-16T15:42:25.023166+0000","last_active":"2024-09-16T15:42:25.026385+0000","last_peered":"2024-09-16T15:42:25.026385+0000","last_clean":"2024-09-16T15:42:25.026385+0000","last_became_active":"2024-09-16T15:42:25.022883+0000","last_became_peered":"2024-09-16T15:42:25.022883+0000","last_unstale":"2024-09-16T15:42:25.026385+0000","last_undegraded":"2024-09-16T15:42:25.026385+0000","last_fullsized":"2024-09-16T15:42:25.026385+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:43:30.009281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:24.989940+0000","last_change":"2024-09-16T15:42:24.007502+0000","last_active":"2024-09-16T15:42:24.989940+0000","last_peered":"2024-09-16T15:42:24.989940+0000","last_clean":"2024-09-16T15:42:24.989940+0000","last_became_active":"2024-09-16T15:42:24.007051+0000","last_became_peered":"2024-09-16T15:42:24.007051+0000","last_unstale":"2024-09-16T15:42:24.989940+0000","last_undegraded":"2024-09-16T15:42:24.989940+0000","last_fullsized":"2024-09-16T15:42:24.989940+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:42:33.038498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.015757+0000","last_change":"2024-09-16T15:42:24.998718+0000","last_active":"2024-09-16T15:42:25.015757+0000","last_peered":"2024-09-16T15:42:25.015757+0000","last_clean":"2024-09-16T15:42:25.015757+0000","last_became_active":"2024-09-16T15:42:24.998449+0000","last_became_peered":"2024-09-16T15:42:24.998449+0000","last_unstale":"2024-09-16T15:42:25.015757+0000","last_undegraded":"2024-09-16T15:42:25.015757+0000","last_fullsized":"2024-09-16T15:42:25.015757+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:10:05.060630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"47'3","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.026863+0000","last_change":"2024-09-16T15:42:24.000199+0000","last_active":"2024-09-16T15:42:25.026863+0000","last_peered":"2024-09-16T15:42:25.026863+0000","last_clean":"2024-09-16T15:42:25.026863+0000","last_became_active":"2024-09-16T15:42:23.999884+0000","last_became_peered":"2024-09-16T15:42:23.999884+0000","last_unstale":"2024-09-16T15:42:25.026863+0000","last_undegraded":"2024-09-16T15:42:25.026863+0000","last_fullsized":"2024-09-16T15:42:25.026863+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:19:10.773082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.023454+0000","last_change":"2024-09-16T15:42:25.018590+0000","last_active":"2024-09-16T15:42:25.023454+0000","last_peered":"2024-09-16T15:42:25.023454+0000","last_clean":"2024-09-16T15:42:25.023454+0000","last_became_active":"2024-09-16T15:42:25.018027+0000","last_became_peered":"2024-09-16T15:42:25.018027+0000","last_unstale":"2024-09-16T15:42:25.023454+0000","last_undegraded":"2024-09-16T15:42:25.023454+0000","last_fullsized":"2024-09-16T15:42:25.023454+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:30:22.114563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"47'6","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.031072+0000","last_change":"2024-09-16T15:42:24.005558+0000","last_active":"2024-09-16T15:42:25.031072+0000","last_peered":"2024-09-16T15:42:25.031072+0000","last_clean":"2024-09-16T15:42:25.031072+0000","last_became_active":"2024-09-16T15:42:24.004712+0000","last_became_peered":"2024-09-16T15:42:24.004712+0000","last_unstale":"2024-09-16T15:42:25.031072+0000","last_undegraded":"2024-09-16T15:42:25.031072+0000","last_fullsized":"2024-09-16T15:42:25.031072+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:44:40.838439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.008322+0000","last_change":"2024-09-16T15:42:24.997997+0000","last_active":"2024-09-16T15:42:25.008322+0000","last_peered":"2024-09-16T15:42:25.008322+0000","last_clean":"2024-09-16T15:42:25.008322+0000","last_became_active":"2024-09-16T15:42:24.997734+0000","last_became_peered":"2024-09-16T15:42:24.997734+0000","last_unstale":"2024-09-16T15:42:25.008322+0000","last_undegraded":"2024-09-16T15:42:25.008322+0000","last_fullsized":"2024-09-16T15:42:25.008322+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:52:34.420492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T15:42:24.005906+0000","last_change":"2024-09-16T15:42:24.001353+0000","last_active":"2024-09-16T15:42:24.005906+0000","last_peered":"2024-09-16T15:42:24.005906+0000","last_clean":"2024-09-16T15:42:24.005906+0000","last_became_active":"2024-09-16T15:42:24.000702+0000","last_became_peered":"2024-09-16T15:42:24.000702+0000","last_unstale":"2024-09-16T15:42:24.005906+0000","last_undegraded":"2024-09-16T15:42:24.005906+0000","last_fullsized":"2024-09-16T15:42:24.005906+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:11:04.849660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-16T15:42:23.991166+0000","last_change":"2024-09-16T15:42:23.991166+0000","last_active":"2024-09-16T15:42:23.975506+0000","last_peered":"2024-09-16T15:42:23.975506+0000","last_clean":"2024-09-16T15:42:23.975506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:42:23.991166+0000","last_undegraded":"2024-09-16T15:42:23.991166+0000","last_fullsized":"2024-09-16T15:42:23.991166+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.004484+0000","last_change":"2024-09-16T15:42:24.004317+0000","last_active":"2024-09-16T15:42:25.004484+0000","last_peered":"2024-09-16T15:42:25.004484+0000","last_clean":"2024-09-16T15:42:25.004484+0000","last_became_active":"2024-09-16T15:42:24.003249+0000","last_became_peered":"2024-09-16T15:42:24.003249+0000","last_unstale":"2024-09-16T15:42:25.004484+0000","last_undegraded":"2024-09-16T15:42:25.004484+0000","last_fullsized":"2024-09-16T15:42:25.004484+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:47:43.918079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-16T15:42:23.990754+0000","last_change":"2024-09-16T15:42:23.990754+0000","last_active":"2024-09-16T15:42:23.975506+0000","last_peered":"2024-09-16T15:42:23.975506+0000","last_clean":"2024-09-16T15:42:23.975506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:42:23.990754+0000","last_undegraded":"2024-09-16T15:42:23.990754+0000","last_fullsized":"2024-09-16T15:42:23.990754+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:24.999568+0000","last_change":"2024-09-16T15:42:23.999141+0000","last_active":"2024-09-16T15:42:24.999568+0000","last_peered":"2024-09-16T15:42:24.999568+0000","last_clean":"2024-09-16T15:42:24.999568+0000","last_became_active":"2024-09-16T15:42:23.998768+0000","last_became_peered":"2024-09-16T15:42:23.998768+0000","last_unstale":"2024-09-16T15:42:24.999568+0000","last_undegraded":"2024-09-16T15:42:24.999568+0000","last_fullsized":"2024-09-16T15:42:24.999568+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:35:18.477892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.027942+0000","last_change":"2024-09-16T15:42:25.022864+0000","last_active":"2024-09-16T15:42:25.027942+0000","last_peered":"2024-09-16T15:42:25.027942+0000","last_clean":"2024-09-16T15:42:25.027942+0000","last_became_active":"2024-09-16T15:42:25.022447+0000","last_became_peered":"2024-09-16T15:42:25.022447+0000","last_unstale":"2024-09-16T15:42:25.027942+0000","last_undegraded":"2024-09-16T15:42:25.027942+0000","last_fullsized":"2024-09-16T15:42:25.027942+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:10:55.771726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"47'8","reported_seq":21,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.026632+0000","last_change":"2024-09-16T15:42:24.004256+0000","last_active":"2024-09-16T15:42:25.026632+0000","last_peered":"2024-09-16T15:42:25.026632+0000","last_clean":"2024-09-16T15:42:25.026632+0000","last_became_active":"2024-09-16T15:42:24.002811+0000","last_became_peered":"2024-09-16T15:42:24.002811+0000","last_unstale":"2024-09-16T15:42:25.026632+0000","last_undegraded":"2024-09-16T15:42:25.026632+0000","last_fullsized":"2024-09-16T15:42:25.026632+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:29:44.054327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-16T15:42:23.990875+0000","last_change":"2024-09-16T15:42:23.990875+0000","last_active":"2024-09-16T15:42:23.975506+0000","last_peered":"2024-09-16T15:42:23.975506+0000","last_clean":"2024-09-16T15:42:23.975506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:42:23.990875+0000","last_undegraded":"2024-09-16T15:42:23.990875+0000","last_fullsized":"2024-09-16T15:42:23.990875+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"47'1","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.027090+0000","last_change":"2024-09-16T15:42:24.003907+0000","last_active":"2024-09-16T15:42:25.027090+0000","last_peered":"2024-09-16T15:42:25.027090+0000","last_clean":"2024-09-16T15:42:25.027090+0000","last_became_active":"2024-09-16T15:42:24.003647+0000","last_became_peered":"2024-09-16T15:42:24.003647+0000","last_unstale":"2024-09-16T15:42:25.027090+0000","last_undegraded":"2024-09-16T15:42:25.027090+0000","last_fullsized":"2024-09-16T15:42:25.027090+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:18:29.135782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-16T15:42:23.991889+0000","last_change":"2024-09-16T15:42:23.991889+0000","last_active":"2024-09-16T15:42:23.975506+0000","last_peered":"2024-09-16T15:42:23.975506+0000","last_clean":"2024-09-16T15:42:23.975506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:42:23.991889+0000","last_undegraded":"2024-09-16T15:42:23.991889+0000","last_fullsized":"2024-09-16T15:42:23.991889+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"47'2","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.024714+0000","last_change":"2024-09-16T15:42:23.999546+0000","last_active":"2024-09-16T15:42:25.024714+0000","last_peered":"2024-09-16T15:42:25.024714+0000","last_clean":"2024-09-16T15:42:25.024714+0000","last_became_active":"2024-09-16T15:42:23.999283+0000","last_became_peered":"2024-09-16T15:42:23.999283+0000","last_unstale":"2024-09-16T15:42:25.024714+0000","last_undegraded":"2024-09-16T15:42:25.024714+0000","last_fullsized":"2024-09-16T15:42:25.024714+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:48:31.728751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.022059+0000","last_change":"2024-09-16T15:42:25.017755+0000","last_active":"2024-09-16T15:42:25.022059+0000","last_peered":"2024-09-16T15:42:25.022059+0000","last_clean":"2024-09-16T15:42:25.022059+0000","last_became_active":"2024-09-16T15:42:25.017488+0000","last_became_peered":"2024-09-16T15:42:25.017488+0000","last_unstale":"2024-09-16T15:42:25.022059+0000","last_undegraded":"2024-09-16T15:42:25.022059+0000","last_fullsized":"2024-09-16T15:42:25.022059+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:11:08.521397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"47'1","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.025904+0000","last_change":"2024-09-16T15:42:24.006459+0000","last_active":"2024-09-16T15:42:25.025904+0000","last_peered":"2024-09-16T15:42:25.025904+0000","last_clean":"2024-09-16T15:42:25.025904+0000","last_became_active":"2024-09-16T15:42:24.006025+0000","last_became_peered":"2024-09-16T15:42:24.006025+0000","last_unstale":"2024-09-16T15:42:25.025904+0000","last_undegraded":"2024-09-16T15:42:25.025904+0000","last_fullsized":"2024-09-16T15:42:25.025904+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:48:42.868847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.022267+0000","last_change":"2024-09-16T15:42:25.011131+0000","last_active":"2024-09-16T15:42:25.022267+0000","last_peered":"2024-09-16T15:42:25.022267+0000","last_clean":"2024-09-16T15:42:25.022267+0000","last_became_active":"2024-09-16T15:42:25.008480+0000","last_became_peered":"2024-09-16T15:42:25.008480+0000","last_unstale":"2024-09-16T15:42:25.022267+0000","last_undegraded":"2024-09-16T15:42:25.022267+0000","last_fullsized":"2024-09-16T15:42:25.022267+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:49:05.814118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:24.999299+0000","last_change":"2024-09-16T15:42:24.005143+0000","last_active":"2024-09-16T15:42:24.999299+0000","last_peered":"2024-09-16T15:42:24.999299+0000","last_clean":"2024-09-16T15:42:24.999299+0000","last_became_active":"2024-09-16T15:42:24.004889+0000","last_became_peered":"2024-09-16T15:42:24.004889+0000","last_unstale":"2024-09-16T15:42:24.999299+0000","last_undegraded":"2024-09-16T15:42:24.999299+0000","last_fullsized":"2024-09-16T15:42:24.999299+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:27:58.850861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.014588+0000","last_change":"2024-09-16T15:42:25.010574+0000","last_active":"2024-09-16T15:42:25.014588+0000","last_peered":"2024-09-16T15:42:25.014588+0000","last_clean":"2024-09-16T15:42:25.014588+0000","last_became_active":"2024-09-16T15:42:25.009246+0000","last_became_peered":"2024-09-16T15:42:25.009246+0000","last_unstale":"2024-09-16T15:42:25.014588+0000","last_undegraded":"2024-09-16T15:42:25.014588+0000","last_fullsized":"2024-09-16T15:42:25.014588+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:56:12.198490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"47'2","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.026956+0000","last_change":"2024-09-16T15:42:24.007687+0000","last_active":"2024-09-16T15:42:25.026956+0000","last_peered":"2024-09-16T15:42:25.026956+0000","last_clean":"2024-09-16T15:42:25.026956+0000","last_became_active":"2024-09-16T15:42:24.006760+0000","last_became_peered":"2024-09-16T15:42:24.006760+0000","last_unstale":"2024-09-16T15:42:25.026956+0000","last_undegraded":"2024-09-16T15:42:25.026956+0000","last_fullsized":"2024-09-16T15:42:25.026956+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:16:19.186435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.022504+0000","last_change":"2024-09-16T15:42:25.012518+0000","last_active":"2024-09-16T15:42:25.022504+0000","last_peered":"2024-09-16T15:42:25.022504+0000","last_clean":"2024-09-16T15:42:25.022504+0000","last_became_active":"2024-09-16T15:42:25.012013+0000","last_became_peered":"2024-09-16T15:42:25.012013+0000","last_unstale":"2024-09-16T15:42:25.022504+0000","last_undegraded":"2024-09-16T15:42:25.022504+0000","last_fullsized":"2024-09-16T15:42:25.022504+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:18:30.138250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.005690+0000","last_change":"2024-09-16T15:42:24.007715+0000","last_active":"2024-09-16T15:42:25.005690+0000","last_peered":"2024-09-16T15:42:25.005690+0000","last_clean":"2024-09-16T15:42:25.005690+0000","last_became_active":"2024-09-16T15:42:24.007435+0000","last_became_peered":"2024-09-16T15:42:24.007435+0000","last_unstale":"2024-09-16T15:42:25.005690+0000","last_undegraded":"2024-09-16T15:42:25.005690+0000","last_fullsized":"2024-09-16T15:42:25.005690+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:23:18.271535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.017207+0000","last_change":"2024-09-16T15:42:25.007606+0000","last_active":"2024-09-16T15:42:25.017207+0000","last_peered":"2024-09-16T15:42:25.017207+0000","last_clean":"2024-09-16T15:42:25.017207+0000","last_became_active":"2024-09-16T15:42:25.006710+0000","last_became_peered":"2024-09-16T15:42:25.006710+0000","last_unstale":"2024-09-16T15:42:25.017207+0000","last_undegraded":"2024-09-16T15:42:25.017207+0000","last_fullsized":"2024-09-16T15:42:25.017207+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:05:52.459658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:24.989139+0000","last_change":"2024-09-16T15:42:24.007575+0000","last_active":"2024-09-16T15:42:24.989139+0000","last_peered":"2024-09-16T15:42:24.989139+0000","last_clean":"2024-09-16T15:42:24.989139+0000","last_became_active":"2024-09-16T15:42:24.007186+0000","last_became_peered":"2024-09-16T15:42:24.007186+0000","last_unstale":"2024-09-16T15:42:24.989139+0000","last_undegraded":"2024-09-16T15:42:24.989139+0000","last_fullsized":"2024-09-16T15:42:24.989139+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:25:18.891435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.018082+0000","last_change":"2024-09-16T15:42:25.013201+0000","last_active":"2024-09-16T15:42:25.018082+0000","last_peered":"2024-09-16T15:42:25.018082+0000","last_clean":"2024-09-16T15:42:25.018082+0000","last_became_active":"2024-09-16T15:42:25.001679+0000","last_became_peered":"2024-09-16T15:42:25.001679+0000","last_unstale":"2024-09-16T15:42:25.018082+0000","last_undegraded":"2024-09-16T15:42:25.018082+0000","last_fullsized":"2024-09-16T15:42:25.018082+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:16:20.188903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:24.993392+0000","last_change":"2024-09-16T15:42:23.995573+0000","last_active":"2024-09-16T15:42:24.993392+0000","last_peered":"2024-09-16T15:42:24.993392+0000","last_clean":"2024-09-16T15:42:24.993392+0000","last_became_active":"2024-09-16T15:42:23.995328+0000","last_became_peered":"2024-09-16T15:42:23.995328+0000","last_unstale":"2024-09-16T15:42:24.993392+0000","last_undegraded":"2024-09-16T15:42:24.993392+0000","last_fullsized":"2024-09-16T15:42:24.993392+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:10:17.404307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.013767+0000","last_change":"2024-09-16T15:42:25.010788+0000","last_active":"2024-09-16T15:42:25.013767+0000","last_peered":"2024-09-16T15:42:25.013767+0000","last_clean":"2024-09-16T15:42:25.013767+0000","last_became_active":"2024-09-16T15:42:25.010513+0000","last_became_peered":"2024-09-16T15:42:25.010513+0000","last_unstale":"2024-09-16T15:42:25.013767+0000","last_undegraded":"2024-09-16T15:42:25.013767+0000","last_fullsized":"2024-09-16T15:42:25.013767+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:31:12.499632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:24.991964+0000","last_change":"2024-09-16T15:42:24.002775+0000","last_active":"2024-09-16T15:42:24.991964+0000","last_peered":"2024-09-16T15:42:24.991964+0000","last_clean":"2024-09-16T15:42:24.991964+0000","last_became_active":"2024-09-16T15:42:24.002505+0000","last_became_peered":"2024-09-16T15:42:24.002505+0000","last_unstale":"2024-09-16T15:42:24.991964+0000","last_undegraded":"2024-09-16T15:42:24.991964+0000","last_fullsized":"2024-09-16T15:42:24.991964+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:01:01.221459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.014146+0000","last_change":"2024-09-16T15:42:25.010363+0000","last_active":"2024-09-16T15:42:25.014146+0000","last_peered":"2024-09-16T15:42:25.014146+0000","last_clean":"2024-09-16T15:42:25.014146+0000","last_became_active":"2024-09-16T15:42:25.010070+0000","last_became_peered":"2024-09-16T15:42:25.010070+0000","last_unstale":"2024-09-16T15:42:25.014146+0000","last_undegraded":"2024-09-16T15:42:25.014146+0000","last_fullsized":"2024-09-16T15:42:25.014146+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:55:24.766427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.010856+0000","last_change":"2024-09-16T15:42:24.007633+0000","last_active":"2024-09-16T15:42:25.010856+0000","last_peered":"2024-09-16T15:42:25.010856+0000","last_clean":"2024-09-16T15:42:25.010856+0000","last_became_active":"2024-09-16T15:42:24.007382+0000","last_became_peered":"2024-09-16T15:42:24.007382+0000","last_unstale":"2024-09-16T15:42:25.010856+0000","last_undegraded":"2024-09-16T15:42:25.010856+0000","last_fullsized":"2024-09-16T15:42:25.010856+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:49:10.293854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.023482+0000","last_change":"2024-09-16T15:42:25.019597+0000","last_active":"2024-09-16T15:42:25.023482+0000","last_peered":"2024-09-16T15:42:25.023482+0000","last_clean":"2024-09-16T15:42:25.023482+0000","last_became_active":"2024-09-16T15:42:25.018344+0000","last_became_peered":"2024-09-16T15:42:25.018344+0000","last_unstale":"2024-09-16T15:42:25.023482+0000","last_undegraded":"2024-09-16T15:42:25.023482+0000","last_fullsized":"2024-09-16T15:42:25.023482+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:33:44.733751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T15:42:24.008224+0000","last_change":"2024-09-16T15:42:24.007325+0000","last_active":"2024-09-16T15:42:24.008224+0000","last_peered":"2024-09-16T15:42:24.008224+0000","last_clean":"2024-09-16T15:42:24.008224+0000","last_became_active":"2024-09-16T15:42:24.006971+0000","last_became_peered":"2024-09-16T15:42:24.006971+0000","last_unstale":"2024-09-16T15:42:24.008224+0000","last_undegraded":"2024-09-16T15:42:24.008224+0000","last_fullsized":"2024-09-16T15:42:24.008224+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:11:36.047586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.014611+0000","last_change":"2024-09-16T15:42:25.011747+0000","last_active":"2024-09-16T15:42:25.014611+0000","last_peered":"2024-09-16T15:42:25.014611+0000","last_clean":"2024-09-16T15:42:25.014611+0000","last_became_active":"2024-09-16T15:42:25.009391+0000","last_became_peered":"2024-09-16T15:42:25.009391+0000","last_unstale":"2024-09-16T15:42:25.014611+0000","last_undegraded":"2024-09-16T15:42:25.014611+0000","last_fullsized":"2024-09-16T15:42:25.014611+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:48:20.712100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.004476+0000","last_change":"2024-09-16T15:42:24.005893+0000","last_active":"2024-09-16T15:42:25.004476+0000","last_peered":"2024-09-16T15:42:25.004476+0000","last_clean":"2024-09-16T15:42:25.004476+0000","last_became_active":"2024-09-16T15:42:24.005708+0000","last_became_peered":"2024-09-16T15:42:24.005708+0000","last_unstale":"2024-09-16T15:42:25.004476+0000","last_undegraded":"2024-09-16T15:42:25.004476+0000","last_fullsized":"2024-09-16T15:42:25.004476+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:10:04.058162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.018028+0000","last_change":"2024-09-16T15:42:25.014435+0000","last_active":"2024-09-16T15:42:25.018028+0000","last_peered":"2024-09-16T15:42:25.018028+0000","last_clean":"2024-09-16T15:42:25.018028+0000","last_became_active":"2024-09-16T15:42:25.014126+0000","last_became_peered":"2024-09-16T15:42:25.014126+0000","last_unstale":"2024-09-16T15:42:25.018028+0000","last_undegraded":"2024-09-16T15:42:25.018028+0000","last_fullsized":"2024-09-16T15:42:25.018028+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:42:34.040967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:24.995096+0000","last_change":"2024-09-16T15:42:24.003312+0000","last_active":"2024-09-16T15:42:24.995096+0000","last_peered":"2024-09-16T15:42:24.995096+0000","last_clean":"2024-09-16T15:42:24.995096+0000","last_became_active":"2024-09-16T15:42:24.002853+0000","last_became_peered":"2024-09-16T15:42:24.002853+0000","last_unstale":"2024-09-16T15:42:24.995096+0000","last_undegraded":"2024-09-16T15:42:24.995096+0000","last_fullsized":"2024-09-16T15:42:24.995096+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:21:53.045720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.009682+0000","last_change":"2024-09-16T15:42:25.000979+0000","last_active":"2024-09-16T15:42:25.009682+0000","last_peered":"2024-09-16T15:42:25.009682+0000","last_clean":"2024-09-16T15:42:25.009682+0000","last_became_active":"2024-09-16T15:42:25.000590+0000","last_became_peered":"2024-09-16T15:42:25.000590+0000","last_unstale":"2024-09-16T15:42:25.009682+0000","last_undegraded":"2024-09-16T15:42:25.009682+0000","last_fullsized":"2024-09-16T15:42:25.009682+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:08:12.361923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:24.999545+0000","last_change":"2024-09-16T15:42:23.999897+0000","last_active":"2024-09-16T15:42:24.999545+0000","last_peered":"2024-09-16T15:42:24.999545+0000","last_clean":"2024-09-16T15:42:24.999545+0000","last_became_active":"2024-09-16T15:42:23.999535+0000","last_became_peered":"2024-09-16T15:42:23.999535+0000","last_unstale":"2024-09-16T15:42:24.999545+0000","last_undegraded":"2024-09-16T15:42:24.999545+0000","last_fullsized":"2024-09-16T15:42:24.999545+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:46:06.187678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.018713+0000","last_change":"2024-09-16T15:42:25.013782+0000","last_active":"2024-09-16T15:42:25.018713+0000","last_peered":"2024-09-16T15:42:25.018713+0000","last_clean":"2024-09-16T15:42:25.018713+0000","last_became_active":"2024-09-16T15:42:25.013470+0000","last_became_peered":"2024-09-16T15:42:25.013470+0000","last_unstale":"2024-09-16T15:42:25.018713+0000","last_undegraded":"2024-09-16T15:42:25.018713+0000","last_fullsized":"2024-09-16T15:42:25.018713+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:56:50.552574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T15:42:24.008031+0000","last_change":"2024-09-16T15:42:24.003909+0000","last_active":"2024-09-16T15:42:24.008031+0000","last_peered":"2024-09-16T15:42:24.008031+0000","last_clean":"2024-09-16T15:42:24.008031+0000","last_became_active":"2024-09-16T15:42:24.003617+0000","last_became_peered":"2024-09-16T15:42:24.003617+0000","last_unstale":"2024-09-16T15:42:24.008031+0000","last_undegraded":"2024-09-16T15:42:24.008031+0000","last_fullsized":"2024-09-16T15:42:24.008031+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:54:56.021717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.023981+0000","last_change":"2024-09-16T15:42:25.018915+0000","last_active":"2024-09-16T15:42:25.023981+0000","last_peered":"2024-09-16T15:42:25.023981+0000","last_clean":"2024-09-16T15:42:25.023981+0000","last_became_active":"2024-09-16T15:42:25.018630+0000","last_became_peered":"2024-09-16T15:42:25.018630+0000","last_unstale":"2024-09-16T15:42:25.023981+0000","last_undegraded":"2024-09-16T15:42:25.023981+0000","last_fullsized":"2024-09-16T15:42:25.023981+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:49:27.727674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"47'2","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.022468+0000","last_change":"2024-09-16T15:42:24.002759+0000","last_active":"2024-09-16T15:42:25.022468+0000","last_peered":"2024-09-16T15:42:25.022468+0000","last_clean":"2024-09-16T15:42:25.022468+0000","last_became_active":"2024-09-16T15:42:24.002401+0000","last_became_peered":"2024-09-16T15:42:24.002401+0000","last_unstale":"2024-09-16T15:42:25.022468+0000","last_undegraded":"2024-09-16T15:42:25.022468+0000","last_fullsized":"2024-09-16T15:42:25.022468+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:35:47.858965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":158,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:24.988829+0000","last_change":"2024-09-16T15:39:12.698813+0000","last_active":"2024-09-16T15:42:24.988829+0000","last_peered":"2024-09-16T15:42:24.988829+0000","last_clean":"2024-09-16T15:42:24.988829+0000","last_became_active":"2024-09-16T15:39:12.697695+0000","last_became_peered":"2024-09-16T15:39:12.697695+0000","last_unstale":"2024-09-16T15:42:24.988829+0000","last_undegraded":"2024-09-16T15:42:24.988829+0000","last_fullsized":"2024-09-16T15:42:24.988829+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_clean_scrub_stamp":"2024-09-16T15:39:11.684967+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:02.061081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2376,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":36,"ondisk_log_size":36,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509484,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31728,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340112,"statfs":{"total":96636764160,"available":96604274688,"internally_reserved":0,"allocated":552960,"data_stored":221564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509483,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32304,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339536,"statfs":{"total":96636764160,"available":96603684864,"internally_reserved":0,"allocated":1142784,"data_stored":811338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509482,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35900,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335940,"statfs":{"total":96636764160,"available":96600002560,"internally_reserved":0,"allocated":565248,"data_stored":222652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509482,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31740,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340100,"statfs":{"total":96636764160,"available":96604262400,"internally_reserved":0,"allocated":557056,"data_stored":222560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31914441},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509482,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31724,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340116,"statfs":{"total":96636764160,"available":96604278784,"internally_reserved":0,"allocated":548864,"data_stored":221038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509483,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32288,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339552,"statfs":{"total":96636764160,"available":96603701248,"internally_reserved":0,"allocated":1126400,"data_stored":804541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509481,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31728,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340112,"statfs":{"total":96636764160,"available":96604274688,"internally_reserved":0,"allocated":552960,"data_stored":221620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542188,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31704,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340136,"statfs":{"total":96636764160,"available":96604299264,"internally_reserved":0,"allocated":520192,"data_stored":207898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":31914444},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:42:29.201 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:42:29.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.396+0000 7f7322600640 1 -- 172.21.15.151:0/2398589193 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f731c0725d0 msgr2=0x7f731c073d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:29.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.396+0000 7f7322600640 1 --2- 172.21.15.151:0/2398589193 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f731c0725d0 0x7f731c073d40 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7304005fd0 tx=0x7f730402a5d0 comp rx=0 tx=0).stop 2024-09-16T15:42:29.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.396+0000 7f7322600640 1 -- 172.21.15.151:0/2398589193 shutdown_connections 2024-09-16T15:42:29.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.396+0000 7f7322600640 1 --2- 172.21.15.151:0/2398589193 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f731c0725d0 0x7f731c073d40 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:29.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.396+0000 7f7322600640 1 --2- 172.21.15.151:0/2398589193 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f731c15bc30 0x7f731c0b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:29.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.396+0000 7f7322600640 1 -- 172.21.15.151:0/2398589193 >> 172.21.15.151:0/2398589193 conn(0x7f731c0cfa20 msgr2=0x7f731c0cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:29.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.396+0000 7f7322600640 1 -- 172.21.15.151:0/2398589193 shutdown_connections 2024-09-16T15:42:29.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.396+0000 7f7322600640 1 -- 172.21.15.151:0/2398589193 wait complete. 2024-09-16T15:42:29.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.398+0000 7f7322600640 1 Processor -- start 2024-09-16T15:42:29.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.398+0000 7f7322600640 1 -- start start 2024-09-16T15:42:29.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.399+0000 7f7322600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f731c0d0540 0x7f731c1f51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:29.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.399+0000 7f7322600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f731c15bc30 0x7f731c1f1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:29.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.399+0000 7f7322600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f731c0cd760 0x7f731c0cdc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:29.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.399+0000 7f7322600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f731c074610 con 0x7f731c0cd760 2024-09-16T15:42:29.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.399+0000 7f7322600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f731c074490 con 0x7f731c0d0540 2024-09-16T15:42:29.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.399+0000 7f7322600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f731c074790 con 0x7f731c15bc30 2024-09-16T15:42:29.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.399+0000 7f7321c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f731c0cd760 0x7f731c0cdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:29.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.399+0000 7f7321c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f731c0cd760 0x7f731c0cdc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49958/0 (socket says 172.21.15.151:49958) 2024-09-16T15:42:29.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.399+0000 7f7321c00640 1 -- 172.21.15.151:0/2095428449 learned_addr learned my addr 172.21.15.151:0/2095428449 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:42:29.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.399+0000 7f7321200640 1 --2- 172.21.15.151:0/2095428449 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f731c0d0540 0x7f731c1f51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:29.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.399+0000 7f7313e00640 1 --2- 172.21.15.151:0/2095428449 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f731c15bc30 0x7f731c1f1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:29.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.400+0000 7f7321c00640 1 -- 172.21.15.151:0/2095428449 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f731c15bc30 msgr2=0x7f731c1f1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:29.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.400+0000 7f7321c00640 1 --2- 172.21.15.151:0/2095428449 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f731c15bc30 0x7f731c1f1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:29.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.400+0000 7f7321c00640 1 -- 172.21.15.151:0/2095428449 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f731c0d0540 msgr2=0x7f731c1f51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:29.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.400+0000 7f7321c00640 1 --2- 172.21.15.151:0/2095428449 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f731c0d0540 0x7f731c1f51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:29.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.400+0000 7f7321c00640 1 -- 172.21.15.151:0/2095428449 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f731c1f56f0 con 0x7f731c0cd760 2024-09-16T15:42:29.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.400+0000 7f7321c00640 1 --2- 172.21.15.151:0/2095428449 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f731c0cd760 0x7f731c0cdc10 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f731400c7a0 tx=0x7f731400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:29.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.400+0000 7f7319600640 1 -- 172.21.15.151:0/2095428449 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7314003b70 con 0x7f731c0cd760 2024-09-16T15:42:29.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.400+0000 7f7319600640 1 -- 172.21.15.151:0/2095428449 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7314003d30 con 0x7f731c0cd760 2024-09-16T15:42:29.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.400+0000 7f7319600640 1 -- 172.21.15.151:0/2095428449 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731400aab0 con 0x7f731c0cd760 2024-09-16T15:42:29.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.400+0000 7f7322600640 1 -- 172.21.15.151:0/2095428449 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f731c0c7ee0 con 0x7f731c0cd760 2024-09-16T15:42:29.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.400+0000 7f7322600640 1 -- 172.21.15.151:0/2095428449 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f731c0c84a0 con 0x7f731c0cd760 2024-09-16T15:42:29.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.402+0000 7f7322600640 1 -- 172.21.15.151:0/2095428449 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f731c06fc60 con 0x7f731c0cd760 2024-09-16T15:42:29.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.403+0000 7f7319600640 1 -- 172.21.15.151:0/2095428449 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7f731400ac50 con 0x7f731c0cd760 2024-09-16T15:42:29.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.404+0000 7f7319600640 1 --2- 172.21.15.151:0/2095428449 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f72f8078bb0 0x7f72f807b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:29.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.404+0000 7f7319600640 1 -- 172.21.15.151:0/2095428449 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f731409adb0 con 0x7f731c0cd760 2024-09-16T15:42:29.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.404+0000 7f7321200640 1 --2- 172.21.15.151:0/2095428449 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f72f8078bb0 0x7f72f807b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:29.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.405+0000 7f7321200640 1 --2- 172.21.15.151:0/2095428449 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f72f8078bb0 0x7f72f807b070 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f731c0cd080 tx=0x7f730c00b2e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:29.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.408+0000 7f7319600640 1 -- 172.21.15.151:0/2095428449 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f731409f070 con 0x7f731c0cd760 2024-09-16T15:42:29.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.588+0000 7f7322600640 1 -- 172.21.15.151:0/2095428449 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f731c0cafd0 con 0x7f72f8078bb0 2024-09-16T15:42:29.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.592+0000 7f7319600640 1 -- 172.21.15.151:0/2095428449 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251449 (secure 0 0 0) 0x7f731c0cafd0 con 0x7f72f8078bb0 2024-09-16T15:42:29.596 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:42:29.598 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-09-16T15:42:29.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.596+0000 7f7322600640 1 -- 172.21.15.151:0/2095428449 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f72f8078bb0 msgr2=0x7f72f807b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:29.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.596+0000 7f7322600640 1 --2- 172.21.15.151:0/2095428449 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f72f8078bb0 0x7f72f807b070 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f731c0cd080 tx=0x7f730c00b2e0 comp rx=0 tx=0).stop 2024-09-16T15:42:29.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.596+0000 7f7322600640 1 -- 172.21.15.151:0/2095428449 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f731c0cd760 msgr2=0x7f731c0cdc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:29.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.596+0000 7f7322600640 1 --2- 172.21.15.151:0/2095428449 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f731c0cd760 0x7f731c0cdc10 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f731400c7a0 tx=0x7f731400cc60 comp rx=0 tx=0).stop 2024-09-16T15:42:29.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.597+0000 7f7322600640 1 -- 172.21.15.151:0/2095428449 shutdown_connections 2024-09-16T15:42:29.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.597+0000 7f7322600640 1 --2- 172.21.15.151:0/2095428449 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f72f8078bb0 0x7f72f807b070 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:29.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.597+0000 7f7322600640 1 --2- 172.21.15.151:0/2095428449 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f731c0cd760 0x7f731c0cdc10 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:29.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.597+0000 7f7322600640 1 --2- 172.21.15.151:0/2095428449 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f731c15bc30 0x7f731c1f1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:29.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.597+0000 7f7322600640 1 --2- 172.21.15.151:0/2095428449 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f731c0d0540 0x7f731c1f51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:29.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.597+0000 7f7322600640 1 -- 172.21.15.151:0/2095428449 >> 172.21.15.151:0/2095428449 conn(0x7f731c0cfa20 msgr2=0x7f731c0cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:29.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.597+0000 7f7322600640 1 -- 172.21.15.151:0/2095428449 shutdown_connections 2024-09-16T15:42:29.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.597+0000 7f7322600640 1 -- 172.21.15.151:0/2095428449 wait complete. 2024-09-16T15:42:29.614 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":140,"stamp":"2024-09-16T15:42:27.624647+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592744,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":142,"num_write_kb":1635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":108,"ondisk_log_size":108,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":109,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259116,"kb_used_data":5436,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754715604,"statfs":{"total":773094113280,"available":772828778496,"internally_reserved":0,"allocated":5566464,"data_stored":2933211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12738,"internal_metadata":259575358},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2376,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.648484"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.026564+0000","last_change":"2024-09-16T15:42:25.021065+0000","last_active":"2024-09-16T15:42:25.026564+0000","last_peered":"2024-09-16T15:42:25.026564+0000","last_clean":"2024-09-16T15:42:25.026564+0000","last_became_active":"2024-09-16T15:42:25.020615+0000","last_became_peered":"2024-09-16T15:42:25.020615+0000","last_unstale":"2024-09-16T15:42:25.026564+0000","last_undegraded":"2024-09-16T15:42:25.026564+0000","last_fullsized":"2024-09-16T15:42:25.026564+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:42:34.040967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.012261+0000","last_change":"2024-09-16T15:42:25.006484+0000","last_active":"2024-09-16T15:42:25.012261+0000","last_peered":"2024-09-16T15:42:25.012261+0000","last_clean":"2024-09-16T15:42:25.012261+0000","last_became_active":"2024-09-16T15:42:25.006265+0000","last_became_peered":"2024-09-16T15:42:25.006265+0000","last_unstale":"2024-09-16T15:42:25.012261+0000","last_undegraded":"2024-09-16T15:42:25.012261+0000","last_fullsized":"2024-09-16T15:42:25.012261+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:01:36.409724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.023590+0000","last_change":"2024-09-16T15:42:25.015717+0000","last_active":"2024-09-16T15:42:25.023590+0000","last_peered":"2024-09-16T15:42:25.023590+0000","last_clean":"2024-09-16T15:42:25.023590+0000","last_became_active":"2024-09-16T15:42:25.006423+0000","last_became_peered":"2024-09-16T15:42:25.006423+0000","last_unstale":"2024-09-16T15:42:25.023590+0000","last_undegraded":"2024-09-16T15:42:25.023590+0000","last_fullsized":"2024-09-16T15:42:25.023590+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:06:46.712953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.018165+0000","last_change":"2024-09-16T15:42:25.005437+0000","last_active":"2024-09-16T15:42:25.018165+0000","last_peered":"2024-09-16T15:42:25.018165+0000","last_clean":"2024-09-16T15:42:25.018165+0000","last_became_active":"2024-09-16T15:42:25.005104+0000","last_became_peered":"2024-09-16T15:42:25.005104+0000","last_unstale":"2024-09-16T15:42:25.018165+0000","last_undegraded":"2024-09-16T15:42:25.018165+0000","last_fullsized":"2024-09-16T15:42:25.018165+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:40:43.407767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.026346+0000","last_change":"2024-09-16T15:42:25.015162+0000","last_active":"2024-09-16T15:42:25.026346+0000","last_peered":"2024-09-16T15:42:25.026346+0000","last_clean":"2024-09-16T15:42:25.026346+0000","last_became_active":"2024-09-16T15:42:25.014915+0000","last_became_peered":"2024-09-16T15:42:25.014915+0000","last_unstale":"2024-09-16T15:42:25.026346+0000","last_undegraded":"2024-09-16T15:42:25.026346+0000","last_fullsized":"2024-09-16T15:42:25.026346+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:05.773773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.022662+0000","last_change":"2024-09-16T15:42:25.018752+0000","last_active":"2024-09-16T15:42:25.022662+0000","last_peered":"2024-09-16T15:42:25.022662+0000","last_clean":"2024-09-16T15:42:25.022662+0000","last_became_active":"2024-09-16T15:42:25.016661+0000","last_became_peered":"2024-09-16T15:42:25.016661+0000","last_unstale":"2024-09-16T15:42:25.022662+0000","last_undegraded":"2024-09-16T15:42:25.022662+0000","last_fullsized":"2024-09-16T15:42:25.022662+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:10:05.060630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.007667+0000","last_change":"2024-09-16T15:42:25.005098+0000","last_active":"2024-09-16T15:42:25.007667+0000","last_peered":"2024-09-16T15:42:25.007667+0000","last_clean":"2024-09-16T15:42:25.007667+0000","last_became_active":"2024-09-16T15:42:25.004817+0000","last_became_peered":"2024-09-16T15:42:25.004817+0000","last_unstale":"2024-09-16T15:42:25.007667+0000","last_undegraded":"2024-09-16T15:42:25.007667+0000","last_fullsized":"2024-09-16T15:42:25.007667+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:12:07.715267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.016184+0000","last_change":"2024-09-16T15:42:25.003653+0000","last_active":"2024-09-16T15:42:25.016184+0000","last_peered":"2024-09-16T15:42:25.016184+0000","last_clean":"2024-09-16T15:42:25.016184+0000","last_became_active":"2024-09-16T15:42:25.002998+0000","last_became_peered":"2024-09-16T15:42:25.002998+0000","last_unstale":"2024-09-16T15:42:25.016184+0000","last_undegraded":"2024-09-16T15:42:25.016184+0000","last_fullsized":"2024-09-16T15:42:25.016184+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:23:19.274003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.022272+0000","last_change":"2024-09-16T15:42:25.013856+0000","last_active":"2024-09-16T15:42:25.022272+0000","last_peered":"2024-09-16T15:42:25.022272+0000","last_clean":"2024-09-16T15:42:25.022272+0000","last_became_active":"2024-09-16T15:42:25.013394+0000","last_became_peered":"2024-09-16T15:42:25.013394+0000","last_unstale":"2024-09-16T15:42:25.022272+0000","last_undegraded":"2024-09-16T15:42:25.022272+0000","last_fullsized":"2024-09-16T15:42:25.022272+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:01:02.223927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.017434+0000","last_change":"2024-09-16T15:42:25.012665+0000","last_active":"2024-09-16T15:42:25.017434+0000","last_peered":"2024-09-16T15:42:25.017434+0000","last_clean":"2024-09-16T15:42:25.017434+0000","last_became_active":"2024-09-16T15:42:25.012429+0000","last_became_peered":"2024-09-16T15:42:25.012429+0000","last_unstale":"2024-09-16T15:42:25.017434+0000","last_undegraded":"2024-09-16T15:42:25.017434+0000","last_fullsized":"2024-09-16T15:42:25.017434+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:06:26.478901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.017884+0000","last_change":"2024-09-16T15:42:25.011076+0000","last_active":"2024-09-16T15:42:25.017884+0000","last_peered":"2024-09-16T15:42:25.017884+0000","last_clean":"2024-09-16T15:42:25.017884+0000","last_became_active":"2024-09-16T15:42:25.010704+0000","last_became_peered":"2024-09-16T15:42:25.010704+0000","last_unstale":"2024-09-16T15:42:25.017884+0000","last_undegraded":"2024-09-16T15:42:25.017884+0000","last_fullsized":"2024-09-16T15:42:25.017884+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:32:58.789647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.012380+0000","last_change":"2024-09-16T15:42:25.007233+0000","last_active":"2024-09-16T15:42:25.012380+0000","last_peered":"2024-09-16T15:42:25.012380+0000","last_clean":"2024-09-16T15:42:25.012380+0000","last_became_active":"2024-09-16T15:42:25.006852+0000","last_became_peered":"2024-09-16T15:42:25.006852+0000","last_unstale":"2024-09-16T15:42:25.012380+0000","last_undegraded":"2024-09-16T15:42:25.012380+0000","last_fullsized":"2024-09-16T15:42:25.012380+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:24:38.283829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.026402+0000","last_change":"2024-09-16T15:42:25.023442+0000","last_active":"2024-09-16T15:42:25.026402+0000","last_peered":"2024-09-16T15:42:25.026402+0000","last_clean":"2024-09-16T15:42:25.026402+0000","last_became_active":"2024-09-16T15:42:25.022319+0000","last_became_peered":"2024-09-16T15:42:25.022319+0000","last_unstale":"2024-09-16T15:42:25.026402+0000","last_undegraded":"2024-09-16T15:42:25.026402+0000","last_fullsized":"2024-09-16T15:42:25.026402+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:47:44.920547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.016169+0000","last_change":"2024-09-16T15:42:25.001358+0000","last_active":"2024-09-16T15:42:25.016169+0000","last_peered":"2024-09-16T15:42:25.016169+0000","last_clean":"2024-09-16T15:42:25.016169+0000","last_became_active":"2024-09-16T15:42:25.001076+0000","last_became_peered":"2024-09-16T15:42:25.001076+0000","last_unstale":"2024-09-16T15:42:25.016169+0000","last_undegraded":"2024-09-16T15:42:25.016169+0000","last_fullsized":"2024-09-16T15:42:25.016169+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:47:44.920547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.018055+0000","last_change":"2024-09-16T15:42:25.012888+0000","last_active":"2024-09-16T15:42:25.018055+0000","last_peered":"2024-09-16T15:42:25.018055+0000","last_clean":"2024-09-16T15:42:25.018055+0000","last_became_active":"2024-09-16T15:42:25.009003+0000","last_became_peered":"2024-09-16T15:42:25.009003+0000","last_unstale":"2024-09-16T15:42:25.018055+0000","last_undegraded":"2024-09-16T15:42:25.018055+0000","last_fullsized":"2024-09-16T15:42:25.018055+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:25:19.893903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.026416+0000","last_change":"2024-09-16T15:42:25.017302+0000","last_active":"2024-09-16T15:42:25.026416+0000","last_peered":"2024-09-16T15:42:25.026416+0000","last_clean":"2024-09-16T15:42:25.026416+0000","last_became_active":"2024-09-16T15:42:25.017045+0000","last_became_peered":"2024-09-16T15:42:25.017045+0000","last_unstale":"2024-09-16T15:42:25.026416+0000","last_undegraded":"2024-09-16T15:42:25.026416+0000","last_fullsized":"2024-09-16T15:42:25.026416+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:32:58.789647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.027927+0000","last_change":"2024-09-16T15:42:25.021592+0000","last_active":"2024-09-16T15:42:25.027927+0000","last_peered":"2024-09-16T15:42:25.027927+0000","last_clean":"2024-09-16T15:42:25.027927+0000","last_became_active":"2024-09-16T15:42:25.021274+0000","last_became_peered":"2024-09-16T15:42:25.021274+0000","last_unstale":"2024-09-16T15:42:25.027927+0000","last_undegraded":"2024-09-16T15:42:25.027927+0000","last_fullsized":"2024-09-16T15:42:25.027927+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:15:26.922530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.004788+0000","last_change":"2024-09-16T15:42:24.995956+0000","last_active":"2024-09-16T15:42:25.004788+0000","last_peered":"2024-09-16T15:42:25.004788+0000","last_clean":"2024-09-16T15:42:25.004788+0000","last_became_active":"2024-09-16T15:42:24.995598+0000","last_became_peered":"2024-09-16T15:42:24.995598+0000","last_unstale":"2024-09-16T15:42:25.004788+0000","last_undegraded":"2024-09-16T15:42:25.004788+0000","last_fullsized":"2024-09-16T15:42:25.004788+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:13:48.480802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.029228+0000","last_change":"2024-09-16T15:42:25.024290+0000","last_active":"2024-09-16T15:42:25.029228+0000","last_peered":"2024-09-16T15:42:25.029228+0000","last_clean":"2024-09-16T15:42:25.029228+0000","last_became_active":"2024-09-16T15:42:25.024029+0000","last_became_peered":"2024-09-16T15:42:25.024029+0000","last_unstale":"2024-09-16T15:42:25.029228+0000","last_undegraded":"2024-09-16T15:42:25.029228+0000","last_fullsized":"2024-09-16T15:42:25.029228+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:44:41.840907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.011900+0000","last_change":"2024-09-16T15:42:25.009320+0000","last_active":"2024-09-16T15:42:25.011900+0000","last_peered":"2024-09-16T15:42:25.011900+0000","last_clean":"2024-09-16T15:42:25.011900+0000","last_became_active":"2024-09-16T15:42:25.008705+0000","last_became_peered":"2024-09-16T15:42:25.008705+0000","last_unstale":"2024-09-16T15:42:25.011900+0000","last_undegraded":"2024-09-16T15:42:25.011900+0000","last_fullsized":"2024-09-16T15:42:25.011900+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:25:11.105280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.022645+0000","last_change":"2024-09-16T15:42:25.014765+0000","last_active":"2024-09-16T15:42:25.022645+0000","last_peered":"2024-09-16T15:42:25.022645+0000","last_clean":"2024-09-16T15:42:25.022645+0000","last_became_active":"2024-09-16T15:42:25.009165+0000","last_became_peered":"2024-09-16T15:42:25.009165+0000","last_unstale":"2024-09-16T15:42:25.022645+0000","last_undegraded":"2024-09-16T15:42:25.022645+0000","last_fullsized":"2024-09-16T15:42:25.022645+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:44:41.840907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-16T15:42:23.991455+0000","last_change":"2024-09-16T15:42:23.991455+0000","last_active":"2024-09-16T15:42:23.975506+0000","last_peered":"2024-09-16T15:42:23.975506+0000","last_clean":"2024-09-16T15:42:23.975506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:42:23.991455+0000","last_undegraded":"2024-09-16T15:42:23.991455+0000","last_fullsized":"2024-09-16T15:42:23.991455+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.028105+0000","last_change":"2024-09-16T15:42:25.024263+0000","last_active":"2024-09-16T15:42:25.028105+0000","last_peered":"2024-09-16T15:42:25.028105+0000","last_clean":"2024-09-16T15:42:25.028105+0000","last_became_active":"2024-09-16T15:42:25.024006+0000","last_became_peered":"2024-09-16T15:42:25.024006+0000","last_unstale":"2024-09-16T15:42:25.028105+0000","last_undegraded":"2024-09-16T15:42:25.028105+0000","last_fullsized":"2024-09-16T15:42:25.028105+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:07:07.004615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.025458+0000","last_change":"2024-09-16T15:42:25.018894+0000","last_active":"2024-09-16T15:42:25.025458+0000","last_peered":"2024-09-16T15:42:25.025458+0000","last_clean":"2024-09-16T15:42:25.025458+0000","last_became_active":"2024-09-16T15:42:25.018023+0000","last_became_peered":"2024-09-16T15:42:25.018023+0000","last_unstale":"2024-09-16T15:42:25.025458+0000","last_undegraded":"2024-09-16T15:42:25.025458+0000","last_fullsized":"2024-09-16T15:42:25.025458+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:22:47.419249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.026541+0000","last_change":"2024-09-16T15:42:25.019057+0000","last_active":"2024-09-16T15:42:25.026541+0000","last_peered":"2024-09-16T15:42:25.026541+0000","last_clean":"2024-09-16T15:42:25.026541+0000","last_became_active":"2024-09-16T15:42:25.018803+0000","last_became_peered":"2024-09-16T15:42:25.018803+0000","last_unstale":"2024-09-16T15:42:25.026541+0000","last_undegraded":"2024-09-16T15:42:25.026541+0000","last_fullsized":"2024-09-16T15:42:25.026541+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:25:19.893903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.026062+0000","last_change":"2024-09-16T15:42:25.022339+0000","last_active":"2024-09-16T15:42:25.026062+0000","last_peered":"2024-09-16T15:42:25.026062+0000","last_clean":"2024-09-16T15:42:25.026062+0000","last_became_active":"2024-09-16T15:42:25.021986+0000","last_became_peered":"2024-09-16T15:42:25.021986+0000","last_unstale":"2024-09-16T15:42:25.026062+0000","last_undegraded":"2024-09-16T15:42:25.026062+0000","last_fullsized":"2024-09-16T15:42:25.026062+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:29:45.056795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.020215+0000","last_change":"2024-09-16T15:42:25.016721+0000","last_active":"2024-09-16T15:42:25.020215+0000","last_peered":"2024-09-16T15:42:25.020215+0000","last_clean":"2024-09-16T15:42:25.020215+0000","last_became_active":"2024-09-16T15:42:25.016383+0000","last_became_peered":"2024-09-16T15:42:25.016383+0000","last_unstale":"2024-09-16T15:42:25.020215+0000","last_undegraded":"2024-09-16T15:42:25.020215+0000","last_fullsized":"2024-09-16T15:42:25.020215+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:55:33.513714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.014293+0000","last_change":"2024-09-16T15:42:25.011808+0000","last_active":"2024-09-16T15:42:25.014293+0000","last_peered":"2024-09-16T15:42:25.014293+0000","last_clean":"2024-09-16T15:42:25.014293+0000","last_became_active":"2024-09-16T15:42:25.011410+0000","last_became_peered":"2024-09-16T15:42:25.011410+0000","last_unstale":"2024-09-16T15:42:25.014293+0000","last_undegraded":"2024-09-16T15:42:25.014293+0000","last_fullsized":"2024-09-16T15:42:25.014293+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:55:58.347779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.026371+0000","last_change":"2024-09-16T15:42:25.020917+0000","last_active":"2024-09-16T15:42:25.026371+0000","last_peered":"2024-09-16T15:42:25.026371+0000","last_clean":"2024-09-16T15:42:25.026371+0000","last_became_active":"2024-09-16T15:42:25.016487+0000","last_became_peered":"2024-09-16T15:42:25.016487+0000","last_unstale":"2024-09-16T15:42:25.026371+0000","last_undegraded":"2024-09-16T15:42:25.026371+0000","last_fullsized":"2024-09-16T15:42:25.026371+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:16:20.188903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.015755+0000","last_change":"2024-09-16T15:42:25.008914+0000","last_active":"2024-09-16T15:42:25.015755+0000","last_peered":"2024-09-16T15:42:25.015755+0000","last_clean":"2024-09-16T15:42:25.015755+0000","last_became_active":"2024-09-16T15:42:25.008631+0000","last_became_peered":"2024-09-16T15:42:25.008631+0000","last_unstale":"2024-09-16T15:42:25.015755+0000","last_undegraded":"2024-09-16T15:42:25.015755+0000","last_fullsized":"2024-09-16T15:42:25.015755+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:55:06.255740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.027325+0000","last_change":"2024-09-16T15:42:25.024127+0000","last_active":"2024-09-16T15:42:25.027325+0000","last_peered":"2024-09-16T15:42:25.027325+0000","last_clean":"2024-09-16T15:42:25.027325+0000","last_became_active":"2024-09-16T15:42:25.023603+0000","last_became_peered":"2024-09-16T15:42:25.023603+0000","last_unstale":"2024-09-16T15:42:25.027325+0000","last_undegraded":"2024-09-16T15:42:25.027325+0000","last_fullsized":"2024-09-16T15:42:25.027325+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:23:19.274003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.017833+0000","last_change":"2024-09-16T15:42:25.006974+0000","last_active":"2024-09-16T15:42:25.017833+0000","last_peered":"2024-09-16T15:42:25.017833+0000","last_clean":"2024-09-16T15:42:25.017833+0000","last_became_active":"2024-09-16T15:42:25.006382+0000","last_became_peered":"2024-09-16T15:42:25.006382+0000","last_unstale":"2024-09-16T15:42:25.017833+0000","last_undegraded":"2024-09-16T15:42:25.017833+0000","last_fullsized":"2024-09-16T15:42:25.017833+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:05.773773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-16T15:42:23.991805+0000","last_change":"2024-09-16T15:42:23.991805+0000","last_active":"2024-09-16T15:42:23.975506+0000","last_peered":"2024-09-16T15:42:23.975506+0000","last_clean":"2024-09-16T15:42:23.975506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:42:23.991805+0000","last_undegraded":"2024-09-16T15:42:23.991805+0000","last_fullsized":"2024-09-16T15:42:23.991805+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"47'4","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.027253+0000","last_change":"2024-09-16T15:42:24.001081+0000","last_active":"2024-09-16T15:42:25.027253+0000","last_peered":"2024-09-16T15:42:25.027253+0000","last_clean":"2024-09-16T15:42:25.027253+0000","last_became_active":"2024-09-16T15:42:24.000071+0000","last_became_peered":"2024-09-16T15:42:24.000071+0000","last_unstale":"2024-09-16T15:42:25.027253+0000","last_undegraded":"2024-09-16T15:42:25.027253+0000","last_fullsized":"2024-09-16T15:42:25.027253+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:24:51.105941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.026516+0000","last_change":"2024-09-16T15:42:25.016739+0000","last_active":"2024-09-16T15:42:25.026516+0000","last_peered":"2024-09-16T15:42:25.026516+0000","last_clean":"2024-09-16T15:42:25.026516+0000","last_became_active":"2024-09-16T15:42:25.016451+0000","last_became_peered":"2024-09-16T15:42:25.016451+0000","last_unstale":"2024-09-16T15:42:25.026516+0000","last_undegraded":"2024-09-16T15:42:25.026516+0000","last_fullsized":"2024-09-16T15:42:25.026516+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:05:52.459658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"47'1","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.016363+0000","last_change":"2024-09-16T15:42:24.000482+0000","last_active":"2024-09-16T15:42:25.016363+0000","last_peered":"2024-09-16T15:42:25.016363+0000","last_clean":"2024-09-16T15:42:25.016363+0000","last_became_active":"2024-09-16T15:42:24.000216+0000","last_became_peered":"2024-09-16T15:42:24.000216+0000","last_unstale":"2024-09-16T15:42:25.016363+0000","last_undegraded":"2024-09-16T15:42:25.016363+0000","last_fullsized":"2024-09-16T15:42:25.016363+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:41:02.855031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.012164+0000","last_change":"2024-09-16T15:42:25.005828+0000","last_active":"2024-09-16T15:42:25.012164+0000","last_peered":"2024-09-16T15:42:25.012164+0000","last_clean":"2024-09-16T15:42:25.012164+0000","last_became_active":"2024-09-16T15:42:25.005398+0000","last_became_peered":"2024-09-16T15:42:25.005398+0000","last_unstale":"2024-09-16T15:42:25.012164+0000","last_undegraded":"2024-09-16T15:42:25.012164+0000","last_fullsized":"2024-09-16T15:42:25.012164+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:43:52.404756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"47'2","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.025469+0000","last_change":"2024-09-16T15:42:24.002867+0000","last_active":"2024-09-16T15:42:25.025469+0000","last_peered":"2024-09-16T15:42:25.025469+0000","last_clean":"2024-09-16T15:42:25.025469+0000","last_became_active":"2024-09-16T15:42:24.002523+0000","last_became_peered":"2024-09-16T15:42:24.002523+0000","last_unstale":"2024-09-16T15:42:25.025469+0000","last_undegraded":"2024-09-16T15:42:25.025469+0000","last_fullsized":"2024-09-16T15:42:25.025469+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:57:21.332836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.016986+0000","last_change":"2024-09-16T15:42:25.006017+0000","last_active":"2024-09-16T15:42:25.016986+0000","last_peered":"2024-09-16T15:42:25.016986+0000","last_clean":"2024-09-16T15:42:25.016986+0000","last_became_active":"2024-09-16T15:42:25.005752+0000","last_became_peered":"2024-09-16T15:42:25.005752+0000","last_unstale":"2024-09-16T15:42:25.016986+0000","last_undegraded":"2024-09-16T15:42:25.016986+0000","last_fullsized":"2024-09-16T15:42:25.016986+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:49:11.296322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"47'2","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.013054+0000","last_change":"2024-09-16T15:42:24.002580+0000","last_active":"2024-09-16T15:42:25.013054+0000","last_peered":"2024-09-16T15:42:25.013054+0000","last_clean":"2024-09-16T15:42:25.013054+0000","last_became_active":"2024-09-16T15:42:24.000516+0000","last_became_peered":"2024-09-16T15:42:24.000516+0000","last_unstale":"2024-09-16T15:42:25.013054+0000","last_undegraded":"2024-09-16T15:42:25.013054+0000","last_fullsized":"2024-09-16T15:42:25.013054+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:34:18.490825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.015597+0000","last_change":"2024-09-16T15:42:25.014465+0000","last_active":"2024-09-16T15:42:25.015597+0000","last_peered":"2024-09-16T15:42:25.015597+0000","last_clean":"2024-09-16T15:42:25.015597+0000","last_became_active":"2024-09-16T15:42:25.013555+0000","last_became_peered":"2024-09-16T15:42:25.013555+0000","last_unstale":"2024-09-16T15:42:25.015597+0000","last_undegraded":"2024-09-16T15:42:25.015597+0000","last_fullsized":"2024-09-16T15:42:25.015597+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:34:09.462316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"47'1","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.019842+0000","last_change":"2024-09-16T15:42:23.997791+0000","last_active":"2024-09-16T15:42:25.019842+0000","last_peered":"2024-09-16T15:42:25.019842+0000","last_clean":"2024-09-16T15:42:25.019842+0000","last_became_active":"2024-09-16T15:42:23.997514+0000","last_became_peered":"2024-09-16T15:42:23.997514+0000","last_unstale":"2024-09-16T15:42:25.019842+0000","last_undegraded":"2024-09-16T15:42:25.019842+0000","last_fullsized":"2024-09-16T15:42:25.019842+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:59:17.932649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.000501+0000","last_change":"2024-09-16T15:42:24.996753+0000","last_active":"2024-09-16T15:42:25.000501+0000","last_peered":"2024-09-16T15:42:25.000501+0000","last_clean":"2024-09-16T15:42:25.000501+0000","last_became_active":"2024-09-16T15:42:24.996402+0000","last_became_peered":"2024-09-16T15:42:24.996402+0000","last_unstale":"2024-09-16T15:42:25.000501+0000","last_undegraded":"2024-09-16T15:42:25.000501+0000","last_fullsized":"2024-09-16T15:42:25.000501+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:26:15.252054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"47'1","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.026472+0000","last_change":"2024-09-16T15:42:23.999519+0000","last_active":"2024-09-16T15:42:25.026472+0000","last_peered":"2024-09-16T15:42:25.026472+0000","last_clean":"2024-09-16T15:42:25.026472+0000","last_became_active":"2024-09-16T15:42:23.999239+0000","last_became_peered":"2024-09-16T15:42:23.999239+0000","last_unstale":"2024-09-16T15:42:25.026472+0000","last_undegraded":"2024-09-16T15:42:25.026472+0000","last_fullsized":"2024-09-16T15:42:25.026472+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:16:13.768386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.002801+0000","last_change":"2024-09-16T15:42:24.999204+0000","last_active":"2024-09-16T15:42:25.002801+0000","last_peered":"2024-09-16T15:42:25.002801+0000","last_clean":"2024-09-16T15:42:25.002801+0000","last_became_active":"2024-09-16T15:42:24.998947+0000","last_became_peered":"2024-09-16T15:42:24.998947+0000","last_unstale":"2024-09-16T15:42:25.002801+0000","last_undegraded":"2024-09-16T15:42:25.002801+0000","last_fullsized":"2024-09-16T15:42:25.002801+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:31:38.819252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.005366+0000","last_change":"2024-09-16T15:42:24.003842+0000","last_active":"2024-09-16T15:42:25.005366+0000","last_peered":"2024-09-16T15:42:25.005366+0000","last_clean":"2024-09-16T15:42:25.005366+0000","last_became_active":"2024-09-16T15:42:24.003517+0000","last_became_peered":"2024-09-16T15:42:24.003517+0000","last_unstale":"2024-09-16T15:42:25.005366+0000","last_undegraded":"2024-09-16T15:42:25.005366+0000","last_fullsized":"2024-09-16T15:42:25.005366+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:40:42.405299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.013648+0000","last_change":"2024-09-16T15:42:25.010354+0000","last_active":"2024-09-16T15:42:25.013648+0000","last_peered":"2024-09-16T15:42:25.013648+0000","last_clean":"2024-09-16T15:42:25.013648+0000","last_became_active":"2024-09-16T15:42:25.010088+0000","last_became_peered":"2024-09-16T15:42:25.010088+0000","last_unstale":"2024-09-16T15:42:25.013648+0000","last_undegraded":"2024-09-16T15:42:25.013648+0000","last_fullsized":"2024-09-16T15:42:25.013648+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:27:39.904292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:24.990744+0000","last_change":"2024-09-16T15:42:24.006237+0000","last_active":"2024-09-16T15:42:24.990744+0000","last_peered":"2024-09-16T15:42:24.990744+0000","last_clean":"2024-09-16T15:42:24.990744+0000","last_became_active":"2024-09-16T15:42:24.005943+0000","last_became_peered":"2024-09-16T15:42:24.005943+0000","last_unstale":"2024-09-16T15:42:24.990744+0000","last_undegraded":"2024-09-16T15:42:24.990744+0000","last_fullsized":"2024-09-16T15:42:24.990744+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:49:26.725206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.009972+0000","last_change":"2024-09-16T15:42:25.005518+0000","last_active":"2024-09-16T15:42:25.009972+0000","last_peered":"2024-09-16T15:42:25.009972+0000","last_clean":"2024-09-16T15:42:25.009972+0000","last_became_active":"2024-09-16T15:42:25.005266+0000","last_became_peered":"2024-09-16T15:42:25.005266+0000","last_unstale":"2024-09-16T15:42:25.009972+0000","last_undegraded":"2024-09-16T15:42:25.009972+0000","last_fullsized":"2024-09-16T15:42:25.009972+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:23:36.266102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:24.989354+0000","last_change":"2024-09-16T15:42:23.997205+0000","last_active":"2024-09-16T15:42:24.989354+0000","last_peered":"2024-09-16T15:42:24.989354+0000","last_clean":"2024-09-16T15:42:24.989354+0000","last_became_active":"2024-09-16T15:42:23.996928+0000","last_became_peered":"2024-09-16T15:42:23.996928+0000","last_unstale":"2024-09-16T15:42:24.989354+0000","last_undegraded":"2024-09-16T15:42:24.989354+0000","last_fullsized":"2024-09-16T15:42:24.989354+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:17:58.230738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.026385+0000","last_change":"2024-09-16T15:42:25.023166+0000","last_active":"2024-09-16T15:42:25.026385+0000","last_peered":"2024-09-16T15:42:25.026385+0000","last_clean":"2024-09-16T15:42:25.026385+0000","last_became_active":"2024-09-16T15:42:25.022883+0000","last_became_peered":"2024-09-16T15:42:25.022883+0000","last_unstale":"2024-09-16T15:42:25.026385+0000","last_undegraded":"2024-09-16T15:42:25.026385+0000","last_fullsized":"2024-09-16T15:42:25.026385+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:43:30.009281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:24.989940+0000","last_change":"2024-09-16T15:42:24.007502+0000","last_active":"2024-09-16T15:42:24.989940+0000","last_peered":"2024-09-16T15:42:24.989940+0000","last_clean":"2024-09-16T15:42:24.989940+0000","last_became_active":"2024-09-16T15:42:24.007051+0000","last_became_peered":"2024-09-16T15:42:24.007051+0000","last_unstale":"2024-09-16T15:42:24.989940+0000","last_undegraded":"2024-09-16T15:42:24.989940+0000","last_fullsized":"2024-09-16T15:42:24.989940+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:42:33.038498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.015757+0000","last_change":"2024-09-16T15:42:24.998718+0000","last_active":"2024-09-16T15:42:25.015757+0000","last_peered":"2024-09-16T15:42:25.015757+0000","last_clean":"2024-09-16T15:42:25.015757+0000","last_became_active":"2024-09-16T15:42:24.998449+0000","last_became_peered":"2024-09-16T15:42:24.998449+0000","last_unstale":"2024-09-16T15:42:25.015757+0000","last_undegraded":"2024-09-16T15:42:25.015757+0000","last_fullsized":"2024-09-16T15:42:25.015757+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:10:05.060630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"47'3","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.026863+0000","last_change":"2024-09-16T15:42:24.000199+0000","last_active":"2024-09-16T15:42:25.026863+0000","last_peered":"2024-09-16T15:42:25.026863+0000","last_clean":"2024-09-16T15:42:25.026863+0000","last_became_active":"2024-09-16T15:42:23.999884+0000","last_became_peered":"2024-09-16T15:42:23.999884+0000","last_unstale":"2024-09-16T15:42:25.026863+0000","last_undegraded":"2024-09-16T15:42:25.026863+0000","last_fullsized":"2024-09-16T15:42:25.026863+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:19:10.773082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.023454+0000","last_change":"2024-09-16T15:42:25.018590+0000","last_active":"2024-09-16T15:42:25.023454+0000","last_peered":"2024-09-16T15:42:25.023454+0000","last_clean":"2024-09-16T15:42:25.023454+0000","last_became_active":"2024-09-16T15:42:25.018027+0000","last_became_peered":"2024-09-16T15:42:25.018027+0000","last_unstale":"2024-09-16T15:42:25.023454+0000","last_undegraded":"2024-09-16T15:42:25.023454+0000","last_fullsized":"2024-09-16T15:42:25.023454+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:30:22.114563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"47'6","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.031072+0000","last_change":"2024-09-16T15:42:24.005558+0000","last_active":"2024-09-16T15:42:25.031072+0000","last_peered":"2024-09-16T15:42:25.031072+0000","last_clean":"2024-09-16T15:42:25.031072+0000","last_became_active":"2024-09-16T15:42:24.004712+0000","last_became_peered":"2024-09-16T15:42:24.004712+0000","last_unstale":"2024-09-16T15:42:25.031072+0000","last_undegraded":"2024-09-16T15:42:25.031072+0000","last_fullsized":"2024-09-16T15:42:25.031072+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:44:40.838439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.008322+0000","last_change":"2024-09-16T15:42:24.997997+0000","last_active":"2024-09-16T15:42:25.008322+0000","last_peered":"2024-09-16T15:42:25.008322+0000","last_clean":"2024-09-16T15:42:25.008322+0000","last_became_active":"2024-09-16T15:42:24.997734+0000","last_became_peered":"2024-09-16T15:42:24.997734+0000","last_unstale":"2024-09-16T15:42:25.008322+0000","last_undegraded":"2024-09-16T15:42:25.008322+0000","last_fullsized":"2024-09-16T15:42:25.008322+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:52:34.420492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T15:42:24.005906+0000","last_change":"2024-09-16T15:42:24.001353+0000","last_active":"2024-09-16T15:42:24.005906+0000","last_peered":"2024-09-16T15:42:24.005906+0000","last_clean":"2024-09-16T15:42:24.005906+0000","last_became_active":"2024-09-16T15:42:24.000702+0000","last_became_peered":"2024-09-16T15:42:24.000702+0000","last_unstale":"2024-09-16T15:42:24.005906+0000","last_undegraded":"2024-09-16T15:42:24.005906+0000","last_fullsized":"2024-09-16T15:42:24.005906+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:11:04.849660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-16T15:42:23.991166+0000","last_change":"2024-09-16T15:42:23.991166+0000","last_active":"2024-09-16T15:42:23.975506+0000","last_peered":"2024-09-16T15:42:23.975506+0000","last_clean":"2024-09-16T15:42:23.975506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:42:23.991166+0000","last_undegraded":"2024-09-16T15:42:23.991166+0000","last_fullsized":"2024-09-16T15:42:23.991166+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.004484+0000","last_change":"2024-09-16T15:42:24.004317+0000","last_active":"2024-09-16T15:42:25.004484+0000","last_peered":"2024-09-16T15:42:25.004484+0000","last_clean":"2024-09-16T15:42:25.004484+0000","last_became_active":"2024-09-16T15:42:24.003249+0000","last_became_peered":"2024-09-16T15:42:24.003249+0000","last_unstale":"2024-09-16T15:42:25.004484+0000","last_undegraded":"2024-09-16T15:42:25.004484+0000","last_fullsized":"2024-09-16T15:42:25.004484+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:47:43.918079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-16T15:42:23.990754+0000","last_change":"2024-09-16T15:42:23.990754+0000","last_active":"2024-09-16T15:42:23.975506+0000","last_peered":"2024-09-16T15:42:23.975506+0000","last_clean":"2024-09-16T15:42:23.975506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:42:23.990754+0000","last_undegraded":"2024-09-16T15:42:23.990754+0000","last_fullsized":"2024-09-16T15:42:23.990754+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:24.999568+0000","last_change":"2024-09-16T15:42:23.999141+0000","last_active":"2024-09-16T15:42:24.999568+0000","last_peered":"2024-09-16T15:42:24.999568+0000","last_clean":"2024-09-16T15:42:24.999568+0000","last_became_active":"2024-09-16T15:42:23.998768+0000","last_became_peered":"2024-09-16T15:42:23.998768+0000","last_unstale":"2024-09-16T15:42:24.999568+0000","last_undegraded":"2024-09-16T15:42:24.999568+0000","last_fullsized":"2024-09-16T15:42:24.999568+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:35:18.477892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.027942+0000","last_change":"2024-09-16T15:42:25.022864+0000","last_active":"2024-09-16T15:42:25.027942+0000","last_peered":"2024-09-16T15:42:25.027942+0000","last_clean":"2024-09-16T15:42:25.027942+0000","last_became_active":"2024-09-16T15:42:25.022447+0000","last_became_peered":"2024-09-16T15:42:25.022447+0000","last_unstale":"2024-09-16T15:42:25.027942+0000","last_undegraded":"2024-09-16T15:42:25.027942+0000","last_fullsized":"2024-09-16T15:42:25.027942+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:10:55.771726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"47'8","reported_seq":21,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.026632+0000","last_change":"2024-09-16T15:42:24.004256+0000","last_active":"2024-09-16T15:42:25.026632+0000","last_peered":"2024-09-16T15:42:25.026632+0000","last_clean":"2024-09-16T15:42:25.026632+0000","last_became_active":"2024-09-16T15:42:24.002811+0000","last_became_peered":"2024-09-16T15:42:24.002811+0000","last_unstale":"2024-09-16T15:42:25.026632+0000","last_undegraded":"2024-09-16T15:42:25.026632+0000","last_fullsized":"2024-09-16T15:42:25.026632+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:29:44.054327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-16T15:42:23.990875+0000","last_change":"2024-09-16T15:42:23.990875+0000","last_active":"2024-09-16T15:42:23.975506+0000","last_peered":"2024-09-16T15:42:23.975506+0000","last_clean":"2024-09-16T15:42:23.975506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:42:23.990875+0000","last_undegraded":"2024-09-16T15:42:23.990875+0000","last_fullsized":"2024-09-16T15:42:23.990875+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"47'1","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.027090+0000","last_change":"2024-09-16T15:42:24.003907+0000","last_active":"2024-09-16T15:42:25.027090+0000","last_peered":"2024-09-16T15:42:25.027090+0000","last_clean":"2024-09-16T15:42:25.027090+0000","last_became_active":"2024-09-16T15:42:24.003647+0000","last_became_peered":"2024-09-16T15:42:24.003647+0000","last_unstale":"2024-09-16T15:42:25.027090+0000","last_undegraded":"2024-09-16T15:42:25.027090+0000","last_fullsized":"2024-09-16T15:42:25.027090+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:18:29.135782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-16T15:42:23.991889+0000","last_change":"2024-09-16T15:42:23.991889+0000","last_active":"2024-09-16T15:42:23.975506+0000","last_peered":"2024-09-16T15:42:23.975506+0000","last_clean":"2024-09-16T15:42:23.975506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:42:23.991889+0000","last_undegraded":"2024-09-16T15:42:23.991889+0000","last_fullsized":"2024-09-16T15:42:23.991889+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"47'2","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.024714+0000","last_change":"2024-09-16T15:42:23.999546+0000","last_active":"2024-09-16T15:42:25.024714+0000","last_peered":"2024-09-16T15:42:25.024714+0000","last_clean":"2024-09-16T15:42:25.024714+0000","last_became_active":"2024-09-16T15:42:23.999283+0000","last_became_peered":"2024-09-16T15:42:23.999283+0000","last_unstale":"2024-09-16T15:42:25.024714+0000","last_undegraded":"2024-09-16T15:42:25.024714+0000","last_fullsized":"2024-09-16T15:42:25.024714+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:48:31.728751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.022059+0000","last_change":"2024-09-16T15:42:25.017755+0000","last_active":"2024-09-16T15:42:25.022059+0000","last_peered":"2024-09-16T15:42:25.022059+0000","last_clean":"2024-09-16T15:42:25.022059+0000","last_became_active":"2024-09-16T15:42:25.017488+0000","last_became_peered":"2024-09-16T15:42:25.017488+0000","last_unstale":"2024-09-16T15:42:25.022059+0000","last_undegraded":"2024-09-16T15:42:25.022059+0000","last_fullsized":"2024-09-16T15:42:25.022059+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:11:08.521397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"47'1","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.025904+0000","last_change":"2024-09-16T15:42:24.006459+0000","last_active":"2024-09-16T15:42:25.025904+0000","last_peered":"2024-09-16T15:42:25.025904+0000","last_clean":"2024-09-16T15:42:25.025904+0000","last_became_active":"2024-09-16T15:42:24.006025+0000","last_became_peered":"2024-09-16T15:42:24.006025+0000","last_unstale":"2024-09-16T15:42:25.025904+0000","last_undegraded":"2024-09-16T15:42:25.025904+0000","last_fullsized":"2024-09-16T15:42:25.025904+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:48:42.868847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.022267+0000","last_change":"2024-09-16T15:42:25.011131+0000","last_active":"2024-09-16T15:42:25.022267+0000","last_peered":"2024-09-16T15:42:25.022267+0000","last_clean":"2024-09-16T15:42:25.022267+0000","last_became_active":"2024-09-16T15:42:25.008480+0000","last_became_peered":"2024-09-16T15:42:25.008480+0000","last_unstale":"2024-09-16T15:42:25.022267+0000","last_undegraded":"2024-09-16T15:42:25.022267+0000","last_fullsized":"2024-09-16T15:42:25.022267+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:49:05.814118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:24.999299+0000","last_change":"2024-09-16T15:42:24.005143+0000","last_active":"2024-09-16T15:42:24.999299+0000","last_peered":"2024-09-16T15:42:24.999299+0000","last_clean":"2024-09-16T15:42:24.999299+0000","last_became_active":"2024-09-16T15:42:24.004889+0000","last_became_peered":"2024-09-16T15:42:24.004889+0000","last_unstale":"2024-09-16T15:42:24.999299+0000","last_undegraded":"2024-09-16T15:42:24.999299+0000","last_fullsized":"2024-09-16T15:42:24.999299+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:27:58.850861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.014588+0000","last_change":"2024-09-16T15:42:25.010574+0000","last_active":"2024-09-16T15:42:25.014588+0000","last_peered":"2024-09-16T15:42:25.014588+0000","last_clean":"2024-09-16T15:42:25.014588+0000","last_became_active":"2024-09-16T15:42:25.009246+0000","last_became_peered":"2024-09-16T15:42:25.009246+0000","last_unstale":"2024-09-16T15:42:25.014588+0000","last_undegraded":"2024-09-16T15:42:25.014588+0000","last_fullsized":"2024-09-16T15:42:25.014588+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:56:12.198490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"47'2","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.026956+0000","last_change":"2024-09-16T15:42:24.007687+0000","last_active":"2024-09-16T15:42:25.026956+0000","last_peered":"2024-09-16T15:42:25.026956+0000","last_clean":"2024-09-16T15:42:25.026956+0000","last_became_active":"2024-09-16T15:42:24.006760+0000","last_became_peered":"2024-09-16T15:42:24.006760+0000","last_unstale":"2024-09-16T15:42:25.026956+0000","last_undegraded":"2024-09-16T15:42:25.026956+0000","last_fullsized":"2024-09-16T15:42:25.026956+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:16:19.186435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.022504+0000","last_change":"2024-09-16T15:42:25.012518+0000","last_active":"2024-09-16T15:42:25.022504+0000","last_peered":"2024-09-16T15:42:25.022504+0000","last_clean":"2024-09-16T15:42:25.022504+0000","last_became_active":"2024-09-16T15:42:25.012013+0000","last_became_peered":"2024-09-16T15:42:25.012013+0000","last_unstale":"2024-09-16T15:42:25.022504+0000","last_undegraded":"2024-09-16T15:42:25.022504+0000","last_fullsized":"2024-09-16T15:42:25.022504+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:18:30.138250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.005690+0000","last_change":"2024-09-16T15:42:24.007715+0000","last_active":"2024-09-16T15:42:25.005690+0000","last_peered":"2024-09-16T15:42:25.005690+0000","last_clean":"2024-09-16T15:42:25.005690+0000","last_became_active":"2024-09-16T15:42:24.007435+0000","last_became_peered":"2024-09-16T15:42:24.007435+0000","last_unstale":"2024-09-16T15:42:25.005690+0000","last_undegraded":"2024-09-16T15:42:25.005690+0000","last_fullsized":"2024-09-16T15:42:25.005690+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:23:18.271535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.017207+0000","last_change":"2024-09-16T15:42:25.007606+0000","last_active":"2024-09-16T15:42:25.017207+0000","last_peered":"2024-09-16T15:42:25.017207+0000","last_clean":"2024-09-16T15:42:25.017207+0000","last_became_active":"2024-09-16T15:42:25.006710+0000","last_became_peered":"2024-09-16T15:42:25.006710+0000","last_unstale":"2024-09-16T15:42:25.017207+0000","last_undegraded":"2024-09-16T15:42:25.017207+0000","last_fullsized":"2024-09-16T15:42:25.017207+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:05:52.459658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:24.989139+0000","last_change":"2024-09-16T15:42:24.007575+0000","last_active":"2024-09-16T15:42:24.989139+0000","last_peered":"2024-09-16T15:42:24.989139+0000","last_clean":"2024-09-16T15:42:24.989139+0000","last_became_active":"2024-09-16T15:42:24.007186+0000","last_became_peered":"2024-09-16T15:42:24.007186+0000","last_unstale":"2024-09-16T15:42:24.989139+0000","last_undegraded":"2024-09-16T15:42:24.989139+0000","last_fullsized":"2024-09-16T15:42:24.989139+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:25:18.891435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.018082+0000","last_change":"2024-09-16T15:42:25.013201+0000","last_active":"2024-09-16T15:42:25.018082+0000","last_peered":"2024-09-16T15:42:25.018082+0000","last_clean":"2024-09-16T15:42:25.018082+0000","last_became_active":"2024-09-16T15:42:25.001679+0000","last_became_peered":"2024-09-16T15:42:25.001679+0000","last_unstale":"2024-09-16T15:42:25.018082+0000","last_undegraded":"2024-09-16T15:42:25.018082+0000","last_fullsized":"2024-09-16T15:42:25.018082+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:16:20.188903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:24.993392+0000","last_change":"2024-09-16T15:42:23.995573+0000","last_active":"2024-09-16T15:42:24.993392+0000","last_peered":"2024-09-16T15:42:24.993392+0000","last_clean":"2024-09-16T15:42:24.993392+0000","last_became_active":"2024-09-16T15:42:23.995328+0000","last_became_peered":"2024-09-16T15:42:23.995328+0000","last_unstale":"2024-09-16T15:42:24.993392+0000","last_undegraded":"2024-09-16T15:42:24.993392+0000","last_fullsized":"2024-09-16T15:42:24.993392+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:10:17.404307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.013767+0000","last_change":"2024-09-16T15:42:25.010788+0000","last_active":"2024-09-16T15:42:25.013767+0000","last_peered":"2024-09-16T15:42:25.013767+0000","last_clean":"2024-09-16T15:42:25.013767+0000","last_became_active":"2024-09-16T15:42:25.010513+0000","last_became_peered":"2024-09-16T15:42:25.010513+0000","last_unstale":"2024-09-16T15:42:25.013767+0000","last_undegraded":"2024-09-16T15:42:25.013767+0000","last_fullsized":"2024-09-16T15:42:25.013767+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:31:12.499632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:24.991964+0000","last_change":"2024-09-16T15:42:24.002775+0000","last_active":"2024-09-16T15:42:24.991964+0000","last_peered":"2024-09-16T15:42:24.991964+0000","last_clean":"2024-09-16T15:42:24.991964+0000","last_became_active":"2024-09-16T15:42:24.002505+0000","last_became_peered":"2024-09-16T15:42:24.002505+0000","last_unstale":"2024-09-16T15:42:24.991964+0000","last_undegraded":"2024-09-16T15:42:24.991964+0000","last_fullsized":"2024-09-16T15:42:24.991964+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:01:01.221459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.014146+0000","last_change":"2024-09-16T15:42:25.010363+0000","last_active":"2024-09-16T15:42:25.014146+0000","last_peered":"2024-09-16T15:42:25.014146+0000","last_clean":"2024-09-16T15:42:25.014146+0000","last_became_active":"2024-09-16T15:42:25.010070+0000","last_became_peered":"2024-09-16T15:42:25.010070+0000","last_unstale":"2024-09-16T15:42:25.014146+0000","last_undegraded":"2024-09-16T15:42:25.014146+0000","last_fullsized":"2024-09-16T15:42:25.014146+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:55:24.766427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.010856+0000","last_change":"2024-09-16T15:42:24.007633+0000","last_active":"2024-09-16T15:42:25.010856+0000","last_peered":"2024-09-16T15:42:25.010856+0000","last_clean":"2024-09-16T15:42:25.010856+0000","last_became_active":"2024-09-16T15:42:24.007382+0000","last_became_peered":"2024-09-16T15:42:24.007382+0000","last_unstale":"2024-09-16T15:42:25.010856+0000","last_undegraded":"2024-09-16T15:42:25.010856+0000","last_fullsized":"2024-09-16T15:42:25.010856+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:49:10.293854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.023482+0000","last_change":"2024-09-16T15:42:25.019597+0000","last_active":"2024-09-16T15:42:25.023482+0000","last_peered":"2024-09-16T15:42:25.023482+0000","last_clean":"2024-09-16T15:42:25.023482+0000","last_became_active":"2024-09-16T15:42:25.018344+0000","last_became_peered":"2024-09-16T15:42:25.018344+0000","last_unstale":"2024-09-16T15:42:25.023482+0000","last_undegraded":"2024-09-16T15:42:25.023482+0000","last_fullsized":"2024-09-16T15:42:25.023482+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:33:44.733751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T15:42:24.008224+0000","last_change":"2024-09-16T15:42:24.007325+0000","last_active":"2024-09-16T15:42:24.008224+0000","last_peered":"2024-09-16T15:42:24.008224+0000","last_clean":"2024-09-16T15:42:24.008224+0000","last_became_active":"2024-09-16T15:42:24.006971+0000","last_became_peered":"2024-09-16T15:42:24.006971+0000","last_unstale":"2024-09-16T15:42:24.008224+0000","last_undegraded":"2024-09-16T15:42:24.008224+0000","last_fullsized":"2024-09-16T15:42:24.008224+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:11:36.047586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.014611+0000","last_change":"2024-09-16T15:42:25.011747+0000","last_active":"2024-09-16T15:42:25.014611+0000","last_peered":"2024-09-16T15:42:25.014611+0000","last_clean":"2024-09-16T15:42:25.014611+0000","last_became_active":"2024-09-16T15:42:25.009391+0000","last_became_peered":"2024-09-16T15:42:25.009391+0000","last_unstale":"2024-09-16T15:42:25.014611+0000","last_undegraded":"2024-09-16T15:42:25.014611+0000","last_fullsized":"2024-09-16T15:42:25.014611+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:48:20.712100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.004476+0000","last_change":"2024-09-16T15:42:24.005893+0000","last_active":"2024-09-16T15:42:25.004476+0000","last_peered":"2024-09-16T15:42:25.004476+0000","last_clean":"2024-09-16T15:42:25.004476+0000","last_became_active":"2024-09-16T15:42:24.005708+0000","last_became_peered":"2024-09-16T15:42:24.005708+0000","last_unstale":"2024-09-16T15:42:25.004476+0000","last_undegraded":"2024-09-16T15:42:25.004476+0000","last_fullsized":"2024-09-16T15:42:25.004476+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:10:04.058162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.018028+0000","last_change":"2024-09-16T15:42:25.014435+0000","last_active":"2024-09-16T15:42:25.018028+0000","last_peered":"2024-09-16T15:42:25.018028+0000","last_clean":"2024-09-16T15:42:25.018028+0000","last_became_active":"2024-09-16T15:42:25.014126+0000","last_became_peered":"2024-09-16T15:42:25.014126+0000","last_unstale":"2024-09-16T15:42:25.018028+0000","last_undegraded":"2024-09-16T15:42:25.018028+0000","last_fullsized":"2024-09-16T15:42:25.018028+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:42:34.040967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:24.995096+0000","last_change":"2024-09-16T15:42:24.003312+0000","last_active":"2024-09-16T15:42:24.995096+0000","last_peered":"2024-09-16T15:42:24.995096+0000","last_clean":"2024-09-16T15:42:24.995096+0000","last_became_active":"2024-09-16T15:42:24.002853+0000","last_became_peered":"2024-09-16T15:42:24.002853+0000","last_unstale":"2024-09-16T15:42:24.995096+0000","last_undegraded":"2024-09-16T15:42:24.995096+0000","last_fullsized":"2024-09-16T15:42:24.995096+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:21:53.045720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.009682+0000","last_change":"2024-09-16T15:42:25.000979+0000","last_active":"2024-09-16T15:42:25.009682+0000","last_peered":"2024-09-16T15:42:25.009682+0000","last_clean":"2024-09-16T15:42:25.009682+0000","last_became_active":"2024-09-16T15:42:25.000590+0000","last_became_peered":"2024-09-16T15:42:25.000590+0000","last_unstale":"2024-09-16T15:42:25.009682+0000","last_undegraded":"2024-09-16T15:42:25.009682+0000","last_fullsized":"2024-09-16T15:42:25.009682+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:08:12.361923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:24.999545+0000","last_change":"2024-09-16T15:42:23.999897+0000","last_active":"2024-09-16T15:42:24.999545+0000","last_peered":"2024-09-16T15:42:24.999545+0000","last_clean":"2024-09-16T15:42:24.999545+0000","last_became_active":"2024-09-16T15:42:23.999535+0000","last_became_peered":"2024-09-16T15:42:23.999535+0000","last_unstale":"2024-09-16T15:42:24.999545+0000","last_undegraded":"2024-09-16T15:42:24.999545+0000","last_fullsized":"2024-09-16T15:42:24.999545+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:46:06.187678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.018713+0000","last_change":"2024-09-16T15:42:25.013782+0000","last_active":"2024-09-16T15:42:25.018713+0000","last_peered":"2024-09-16T15:42:25.018713+0000","last_clean":"2024-09-16T15:42:25.018713+0000","last_became_active":"2024-09-16T15:42:25.013470+0000","last_became_peered":"2024-09-16T15:42:25.013470+0000","last_unstale":"2024-09-16T15:42:25.018713+0000","last_undegraded":"2024-09-16T15:42:25.018713+0000","last_fullsized":"2024-09-16T15:42:25.018713+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:56:50.552574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T15:42:24.008031+0000","last_change":"2024-09-16T15:42:24.003909+0000","last_active":"2024-09-16T15:42:24.008031+0000","last_peered":"2024-09-16T15:42:24.008031+0000","last_clean":"2024-09-16T15:42:24.008031+0000","last_became_active":"2024-09-16T15:42:24.003617+0000","last_became_peered":"2024-09-16T15:42:24.003617+0000","last_unstale":"2024-09-16T15:42:24.008031+0000","last_undegraded":"2024-09-16T15:42:24.008031+0000","last_fullsized":"2024-09-16T15:42:24.008031+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:54:56.021717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.023981+0000","last_change":"2024-09-16T15:42:25.018915+0000","last_active":"2024-09-16T15:42:25.023981+0000","last_peered":"2024-09-16T15:42:25.023981+0000","last_clean":"2024-09-16T15:42:25.023981+0000","last_became_active":"2024-09-16T15:42:25.018630+0000","last_became_peered":"2024-09-16T15:42:25.018630+0000","last_unstale":"2024-09-16T15:42:25.023981+0000","last_undegraded":"2024-09-16T15:42:25.023981+0000","last_fullsized":"2024-09-16T15:42:25.023981+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:49:27.727674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"47'2","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.022468+0000","last_change":"2024-09-16T15:42:24.002759+0000","last_active":"2024-09-16T15:42:25.022468+0000","last_peered":"2024-09-16T15:42:25.022468+0000","last_clean":"2024-09-16T15:42:25.022468+0000","last_became_active":"2024-09-16T15:42:24.002401+0000","last_became_peered":"2024-09-16T15:42:24.002401+0000","last_unstale":"2024-09-16T15:42:25.022468+0000","last_undegraded":"2024-09-16T15:42:25.022468+0000","last_fullsized":"2024-09-16T15:42:25.022468+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:35:47.858965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":158,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:24.988829+0000","last_change":"2024-09-16T15:39:12.698813+0000","last_active":"2024-09-16T15:42:24.988829+0000","last_peered":"2024-09-16T15:42:24.988829+0000","last_clean":"2024-09-16T15:42:24.988829+0000","last_became_active":"2024-09-16T15:39:12.697695+0000","last_became_peered":"2024-09-16T15:39:12.697695+0000","last_unstale":"2024-09-16T15:42:24.988829+0000","last_undegraded":"2024-09-16T15:42:24.988829+0000","last_fullsized":"2024-09-16T15:42:24.988829+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_clean_scrub_stamp":"2024-09-16T15:39:11.684967+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:02.061081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2376,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":36,"ondisk_log_size":36,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509484,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31728,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340112,"statfs":{"total":96636764160,"available":96604274688,"internally_reserved":0,"allocated":552960,"data_stored":221564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509483,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32304,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339536,"statfs":{"total":96636764160,"available":96603684864,"internally_reserved":0,"allocated":1142784,"data_stored":811338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509482,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35900,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335940,"statfs":{"total":96636764160,"available":96600002560,"internally_reserved":0,"allocated":565248,"data_stored":222652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509482,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31740,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340100,"statfs":{"total":96636764160,"available":96604262400,"internally_reserved":0,"allocated":557056,"data_stored":222560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31914441},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509482,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31724,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340116,"statfs":{"total":96636764160,"available":96604278784,"internally_reserved":0,"allocated":548864,"data_stored":221038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509483,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32288,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339552,"statfs":{"total":96636764160,"available":96603701248,"internally_reserved":0,"allocated":1126400,"data_stored":804541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509481,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31728,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340112,"statfs":{"total":96636764160,"available":96604274688,"internally_reserved":0,"allocated":552960,"data_stored":221620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542188,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31704,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340136,"statfs":{"total":96636764160,"available":96604299264,"internally_reserved":0,"allocated":520192,"data_stored":207898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":31914444},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:42:29.618 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:42:29.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.809+0000 7f1178a00640 1 -- 172.21.15.151:0/2094111733 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f117415f5a0 msgr2=0x7f11740b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:29.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.809+0000 7f1178a00640 1 --2- 172.21.15.151:0/2094111733 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f117415f5a0 0x7f11740b2440 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1164005fd0 tx=0x7f116402a540 comp rx=0 tx=0).stop 2024-09-16T15:42:29.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.809+0000 7f1178a00640 1 -- 172.21.15.151:0/2094111733 shutdown_connections 2024-09-16T15:42:29.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.809+0000 7f1178a00640 1 --2- 172.21.15.151:0/2094111733 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11740ccf00 0x7f11740acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:29.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.809+0000 7f1178a00640 1 --2- 172.21.15.151:0/2094111733 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f117415f5a0 0x7f11740b2440 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:29.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.809+0000 7f1178a00640 1 -- 172.21.15.151:0/2094111733 >> 172.21.15.151:0/2094111733 conn(0x7f11740b1950 msgr2=0x7f11740b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:29.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.809+0000 7f1178a00640 1 -- 172.21.15.151:0/2094111733 shutdown_connections 2024-09-16T15:42:29.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.809+0000 7f1178a00640 1 -- 172.21.15.151:0/2094111733 wait complete. 2024-09-16T15:42:29.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.811+0000 7f1178a00640 1 Processor -- start 2024-09-16T15:42:29.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.811+0000 7f1178a00640 1 -- start start 2024-09-16T15:42:29.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.812+0000 7f1178a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f11740ad270 0x7f117414d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:29.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.812+0000 7f1178a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11740ccf00 0x7f11741520e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:29.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.812+0000 7f1178a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f117415f5a0 0x7f1174152620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:29.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.812+0000 7f1178a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f117406f2e0 con 0x7f11740ad270 2024-09-16T15:42:29.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.812+0000 7f1178a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f117406f160 con 0x7f11740ccf00 2024-09-16T15:42:29.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.812+0000 7f1178a00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f117406f460 con 0x7f117415f5a0 2024-09-16T15:42:29.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.812+0000 7f1171e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f11740ad270 0x7f117414d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:29.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.812+0000 7f1171e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f11740ad270 0x7f117414d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49960/0 (socket says 172.21.15.151:49960) 2024-09-16T15:42:29.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.812+0000 7f1171e00640 1 -- 172.21.15.151:0/937649648 learned_addr learned my addr 172.21.15.151:0/937649648 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:42:29.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.812+0000 7f1171400640 1 --2- 172.21.15.151:0/937649648 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11740ccf00 0x7f11741520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:29.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.812+0000 7f1172800640 1 --2- 172.21.15.151:0/937649648 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f117415f5a0 0x7f1174152620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:29.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.813+0000 7f1171e00640 1 -- 172.21.15.151:0/937649648 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f117415f5a0 msgr2=0x7f1174152620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:29.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.813+0000 7f1171e00640 1 --2- 172.21.15.151:0/937649648 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f117415f5a0 0x7f1174152620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:29.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.813+0000 7f1171e00640 1 -- 172.21.15.151:0/937649648 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11740ccf00 msgr2=0x7f11741520e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:29.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.813+0000 7f1171e00640 1 --2- 172.21.15.151:0/937649648 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11740ccf00 0x7f11741520e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:29.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.813+0000 7f1171e00640 1 -- 172.21.15.151:0/937649648 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11741503d0 con 0x7f11740ad270 2024-09-16T15:42:29.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.813+0000 7f1171e00640 1 --2- 172.21.15.151:0/937649648 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f11740ad270 0x7f117414d630 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1164034040 tx=0x7f11640038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:29.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.813+0000 7f116a000640 1 -- 172.21.15.151:0/937649648 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116403e030 con 0x7f11740ad270 2024-09-16T15:42:29.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.813+0000 7f116a000640 1 -- 172.21.15.151:0/937649648 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1164003d90 con 0x7f11740ad270 2024-09-16T15:42:29.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.813+0000 7f116a000640 1 -- 172.21.15.151:0/937649648 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1164035040 con 0x7f11740ad270 2024-09-16T15:42:29.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.813+0000 7f1178a00640 1 -- 172.21.15.151:0/937649648 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11741506b0 con 0x7f11740ad270 2024-09-16T15:42:29.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.814+0000 7f1178a00640 1 -- 172.21.15.151:0/937649648 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f117415ea90 con 0x7f11740ad270 2024-09-16T15:42:29.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.815+0000 7f1178a00640 1 -- 172.21.15.151:0/937649648 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f117406f9f0 con 0x7f11740ad270 2024-09-16T15:42:29.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.816+0000 7f116a000640 1 -- 172.21.15.151:0/937649648 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7f1164041030 con 0x7f11740ad270 2024-09-16T15:42:29.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.817+0000 7f116a000640 1 --2- 172.21.15.151:0/937649648 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1154078bb0 0x7f115407b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:29.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.817+0000 7f116a000640 1 -- 172.21.15.151:0/937649648 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f11640bb8b0 con 0x7f11740ad270 2024-09-16T15:42:29.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.817+0000 7f1171400640 1 --2- 172.21.15.151:0/937649648 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1154078bb0 0x7f115407b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:29.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.818+0000 7f1171400640 1 --2- 172.21.15.151:0/937649648 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1154078bb0 0x7f115407b070 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1174150160 tx=0x7f1160005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:29.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:29.821+0000 7f116a000640 1 -- 172.21.15.151:0/937649648 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1164083370 con 0x7f11740ad270 2024-09-16T15:42:30.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.001+0000 7f1178a00640 1 -- 172.21.15.151:0/937649648 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1174151e00 con 0x7f1154078bb0 2024-09-16T15:42:30.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.006+0000 7f116a000640 1 -- 172.21.15.151:0/937649648 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251449 (secure 0 0 0) 0x7f116001b3f0 con 0x7f1154078bb0 2024-09-16T15:42:30.009 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:42:30.010 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-09-16T15:42:30.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.010+0000 7f1178a00640 1 -- 172.21.15.151:0/937649648 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1154078bb0 msgr2=0x7f115407b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:30.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.010+0000 7f1178a00640 1 --2- 172.21.15.151:0/937649648 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1154078bb0 0x7f115407b070 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1174150160 tx=0x7f1160005f90 comp rx=0 tx=0).stop 2024-09-16T15:42:30.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.010+0000 7f1178a00640 1 -- 172.21.15.151:0/937649648 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f11740ad270 msgr2=0x7f117414d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:30.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.010+0000 7f1178a00640 1 --2- 172.21.15.151:0/937649648 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f11740ad270 0x7f117414d630 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1164034040 tx=0x7f11640038a0 comp rx=0 tx=0).stop 2024-09-16T15:42:30.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.010+0000 7f1178a00640 1 -- 172.21.15.151:0/937649648 shutdown_connections 2024-09-16T15:42:30.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.010+0000 7f1178a00640 1 --2- 172.21.15.151:0/937649648 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1154078bb0 0x7f115407b070 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:30.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.010+0000 7f1178a00640 1 --2- 172.21.15.151:0/937649648 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f11740ccf00 0x7f11741520e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:30.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.010+0000 7f1178a00640 1 --2- 172.21.15.151:0/937649648 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f117415f5a0 0x7f1174152620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:30.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.010+0000 7f1178a00640 1 --2- 172.21.15.151:0/937649648 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f11740ad270 0x7f117414d630 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:30.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.010+0000 7f1178a00640 1 -- 172.21.15.151:0/937649648 >> 172.21.15.151:0/937649648 conn(0x7f11740b1950 msgr2=0x7f117409c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:30.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.010+0000 7f1178a00640 1 -- 172.21.15.151:0/937649648 shutdown_connections 2024-09-16T15:42:30.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.010+0000 7f1178a00640 1 -- 172.21.15.151:0/937649648 wait complete. 2024-09-16T15:42:30.027 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":141,"stamp":"2024-09-16T15:42:29.625136+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592744,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":142,"num_write_kb":1635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":108,"ondisk_log_size":108,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":109,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259116,"kb_used_data":5436,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754715604,"statfs":{"total":773094113280,"available":772828778496,"internally_reserved":0,"allocated":5566464,"data_stored":2933211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12738,"internal_metadata":259575358},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2376,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001529"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.026564+0000","last_change":"2024-09-16T15:42:25.021065+0000","last_active":"2024-09-16T15:42:25.026564+0000","last_peered":"2024-09-16T15:42:25.026564+0000","last_clean":"2024-09-16T15:42:25.026564+0000","last_became_active":"2024-09-16T15:42:25.020615+0000","last_became_peered":"2024-09-16T15:42:25.020615+0000","last_unstale":"2024-09-16T15:42:25.026564+0000","last_undegraded":"2024-09-16T15:42:25.026564+0000","last_fullsized":"2024-09-16T15:42:25.026564+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:42:34.040967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.012261+0000","last_change":"2024-09-16T15:42:25.006484+0000","last_active":"2024-09-16T15:42:25.012261+0000","last_peered":"2024-09-16T15:42:25.012261+0000","last_clean":"2024-09-16T15:42:25.012261+0000","last_became_active":"2024-09-16T15:42:25.006265+0000","last_became_peered":"2024-09-16T15:42:25.006265+0000","last_unstale":"2024-09-16T15:42:25.012261+0000","last_undegraded":"2024-09-16T15:42:25.012261+0000","last_fullsized":"2024-09-16T15:42:25.012261+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:01:36.409724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.023590+0000","last_change":"2024-09-16T15:42:25.015717+0000","last_active":"2024-09-16T15:42:25.023590+0000","last_peered":"2024-09-16T15:42:25.023590+0000","last_clean":"2024-09-16T15:42:25.023590+0000","last_became_active":"2024-09-16T15:42:25.006423+0000","last_became_peered":"2024-09-16T15:42:25.006423+0000","last_unstale":"2024-09-16T15:42:25.023590+0000","last_undegraded":"2024-09-16T15:42:25.023590+0000","last_fullsized":"2024-09-16T15:42:25.023590+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:06:46.712953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.018165+0000","last_change":"2024-09-16T15:42:25.005437+0000","last_active":"2024-09-16T15:42:25.018165+0000","last_peered":"2024-09-16T15:42:25.018165+0000","last_clean":"2024-09-16T15:42:25.018165+0000","last_became_active":"2024-09-16T15:42:25.005104+0000","last_became_peered":"2024-09-16T15:42:25.005104+0000","last_unstale":"2024-09-16T15:42:25.018165+0000","last_undegraded":"2024-09-16T15:42:25.018165+0000","last_fullsized":"2024-09-16T15:42:25.018165+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:40:43.407767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.026346+0000","last_change":"2024-09-16T15:42:25.015162+0000","last_active":"2024-09-16T15:42:25.026346+0000","last_peered":"2024-09-16T15:42:25.026346+0000","last_clean":"2024-09-16T15:42:25.026346+0000","last_became_active":"2024-09-16T15:42:25.014915+0000","last_became_peered":"2024-09-16T15:42:25.014915+0000","last_unstale":"2024-09-16T15:42:25.026346+0000","last_undegraded":"2024-09-16T15:42:25.026346+0000","last_fullsized":"2024-09-16T15:42:25.026346+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:05.773773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.022662+0000","last_change":"2024-09-16T15:42:25.018752+0000","last_active":"2024-09-16T15:42:25.022662+0000","last_peered":"2024-09-16T15:42:25.022662+0000","last_clean":"2024-09-16T15:42:25.022662+0000","last_became_active":"2024-09-16T15:42:25.016661+0000","last_became_peered":"2024-09-16T15:42:25.016661+0000","last_unstale":"2024-09-16T15:42:25.022662+0000","last_undegraded":"2024-09-16T15:42:25.022662+0000","last_fullsized":"2024-09-16T15:42:25.022662+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:10:05.060630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.007667+0000","last_change":"2024-09-16T15:42:25.005098+0000","last_active":"2024-09-16T15:42:25.007667+0000","last_peered":"2024-09-16T15:42:25.007667+0000","last_clean":"2024-09-16T15:42:25.007667+0000","last_became_active":"2024-09-16T15:42:25.004817+0000","last_became_peered":"2024-09-16T15:42:25.004817+0000","last_unstale":"2024-09-16T15:42:25.007667+0000","last_undegraded":"2024-09-16T15:42:25.007667+0000","last_fullsized":"2024-09-16T15:42:25.007667+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:12:07.715267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.016184+0000","last_change":"2024-09-16T15:42:25.003653+0000","last_active":"2024-09-16T15:42:25.016184+0000","last_peered":"2024-09-16T15:42:25.016184+0000","last_clean":"2024-09-16T15:42:25.016184+0000","last_became_active":"2024-09-16T15:42:25.002998+0000","last_became_peered":"2024-09-16T15:42:25.002998+0000","last_unstale":"2024-09-16T15:42:25.016184+0000","last_undegraded":"2024-09-16T15:42:25.016184+0000","last_fullsized":"2024-09-16T15:42:25.016184+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:23:19.274003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.022272+0000","last_change":"2024-09-16T15:42:25.013856+0000","last_active":"2024-09-16T15:42:25.022272+0000","last_peered":"2024-09-16T15:42:25.022272+0000","last_clean":"2024-09-16T15:42:25.022272+0000","last_became_active":"2024-09-16T15:42:25.013394+0000","last_became_peered":"2024-09-16T15:42:25.013394+0000","last_unstale":"2024-09-16T15:42:25.022272+0000","last_undegraded":"2024-09-16T15:42:25.022272+0000","last_fullsized":"2024-09-16T15:42:25.022272+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:01:02.223927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.017434+0000","last_change":"2024-09-16T15:42:25.012665+0000","last_active":"2024-09-16T15:42:25.017434+0000","last_peered":"2024-09-16T15:42:25.017434+0000","last_clean":"2024-09-16T15:42:25.017434+0000","last_became_active":"2024-09-16T15:42:25.012429+0000","last_became_peered":"2024-09-16T15:42:25.012429+0000","last_unstale":"2024-09-16T15:42:25.017434+0000","last_undegraded":"2024-09-16T15:42:25.017434+0000","last_fullsized":"2024-09-16T15:42:25.017434+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:06:26.478901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.017884+0000","last_change":"2024-09-16T15:42:25.011076+0000","last_active":"2024-09-16T15:42:25.017884+0000","last_peered":"2024-09-16T15:42:25.017884+0000","last_clean":"2024-09-16T15:42:25.017884+0000","last_became_active":"2024-09-16T15:42:25.010704+0000","last_became_peered":"2024-09-16T15:42:25.010704+0000","last_unstale":"2024-09-16T15:42:25.017884+0000","last_undegraded":"2024-09-16T15:42:25.017884+0000","last_fullsized":"2024-09-16T15:42:25.017884+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:32:58.789647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.012380+0000","last_change":"2024-09-16T15:42:25.007233+0000","last_active":"2024-09-16T15:42:25.012380+0000","last_peered":"2024-09-16T15:42:25.012380+0000","last_clean":"2024-09-16T15:42:25.012380+0000","last_became_active":"2024-09-16T15:42:25.006852+0000","last_became_peered":"2024-09-16T15:42:25.006852+0000","last_unstale":"2024-09-16T15:42:25.012380+0000","last_undegraded":"2024-09-16T15:42:25.012380+0000","last_fullsized":"2024-09-16T15:42:25.012380+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:24:38.283829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.026402+0000","last_change":"2024-09-16T15:42:25.023442+0000","last_active":"2024-09-16T15:42:25.026402+0000","last_peered":"2024-09-16T15:42:25.026402+0000","last_clean":"2024-09-16T15:42:25.026402+0000","last_became_active":"2024-09-16T15:42:25.022319+0000","last_became_peered":"2024-09-16T15:42:25.022319+0000","last_unstale":"2024-09-16T15:42:25.026402+0000","last_undegraded":"2024-09-16T15:42:25.026402+0000","last_fullsized":"2024-09-16T15:42:25.026402+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:47:44.920547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.016169+0000","last_change":"2024-09-16T15:42:25.001358+0000","last_active":"2024-09-16T15:42:25.016169+0000","last_peered":"2024-09-16T15:42:25.016169+0000","last_clean":"2024-09-16T15:42:25.016169+0000","last_became_active":"2024-09-16T15:42:25.001076+0000","last_became_peered":"2024-09-16T15:42:25.001076+0000","last_unstale":"2024-09-16T15:42:25.016169+0000","last_undegraded":"2024-09-16T15:42:25.016169+0000","last_fullsized":"2024-09-16T15:42:25.016169+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:47:44.920547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.018055+0000","last_change":"2024-09-16T15:42:25.012888+0000","last_active":"2024-09-16T15:42:25.018055+0000","last_peered":"2024-09-16T15:42:25.018055+0000","last_clean":"2024-09-16T15:42:25.018055+0000","last_became_active":"2024-09-16T15:42:25.009003+0000","last_became_peered":"2024-09-16T15:42:25.009003+0000","last_unstale":"2024-09-16T15:42:25.018055+0000","last_undegraded":"2024-09-16T15:42:25.018055+0000","last_fullsized":"2024-09-16T15:42:25.018055+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:25:19.893903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.026416+0000","last_change":"2024-09-16T15:42:25.017302+0000","last_active":"2024-09-16T15:42:25.026416+0000","last_peered":"2024-09-16T15:42:25.026416+0000","last_clean":"2024-09-16T15:42:25.026416+0000","last_became_active":"2024-09-16T15:42:25.017045+0000","last_became_peered":"2024-09-16T15:42:25.017045+0000","last_unstale":"2024-09-16T15:42:25.026416+0000","last_undegraded":"2024-09-16T15:42:25.026416+0000","last_fullsized":"2024-09-16T15:42:25.026416+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:32:58.789647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.027927+0000","last_change":"2024-09-16T15:42:25.021592+0000","last_active":"2024-09-16T15:42:25.027927+0000","last_peered":"2024-09-16T15:42:25.027927+0000","last_clean":"2024-09-16T15:42:25.027927+0000","last_became_active":"2024-09-16T15:42:25.021274+0000","last_became_peered":"2024-09-16T15:42:25.021274+0000","last_unstale":"2024-09-16T15:42:25.027927+0000","last_undegraded":"2024-09-16T15:42:25.027927+0000","last_fullsized":"2024-09-16T15:42:25.027927+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:15:26.922530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.004788+0000","last_change":"2024-09-16T15:42:24.995956+0000","last_active":"2024-09-16T15:42:25.004788+0000","last_peered":"2024-09-16T15:42:25.004788+0000","last_clean":"2024-09-16T15:42:25.004788+0000","last_became_active":"2024-09-16T15:42:24.995598+0000","last_became_peered":"2024-09-16T15:42:24.995598+0000","last_unstale":"2024-09-16T15:42:25.004788+0000","last_undegraded":"2024-09-16T15:42:25.004788+0000","last_fullsized":"2024-09-16T15:42:25.004788+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:13:48.480802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.029228+0000","last_change":"2024-09-16T15:42:25.024290+0000","last_active":"2024-09-16T15:42:25.029228+0000","last_peered":"2024-09-16T15:42:25.029228+0000","last_clean":"2024-09-16T15:42:25.029228+0000","last_became_active":"2024-09-16T15:42:25.024029+0000","last_became_peered":"2024-09-16T15:42:25.024029+0000","last_unstale":"2024-09-16T15:42:25.029228+0000","last_undegraded":"2024-09-16T15:42:25.029228+0000","last_fullsized":"2024-09-16T15:42:25.029228+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:44:41.840907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.011900+0000","last_change":"2024-09-16T15:42:25.009320+0000","last_active":"2024-09-16T15:42:25.011900+0000","last_peered":"2024-09-16T15:42:25.011900+0000","last_clean":"2024-09-16T15:42:25.011900+0000","last_became_active":"2024-09-16T15:42:25.008705+0000","last_became_peered":"2024-09-16T15:42:25.008705+0000","last_unstale":"2024-09-16T15:42:25.011900+0000","last_undegraded":"2024-09-16T15:42:25.011900+0000","last_fullsized":"2024-09-16T15:42:25.011900+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:25:11.105280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.022645+0000","last_change":"2024-09-16T15:42:25.014765+0000","last_active":"2024-09-16T15:42:25.022645+0000","last_peered":"2024-09-16T15:42:25.022645+0000","last_clean":"2024-09-16T15:42:25.022645+0000","last_became_active":"2024-09-16T15:42:25.009165+0000","last_became_peered":"2024-09-16T15:42:25.009165+0000","last_unstale":"2024-09-16T15:42:25.022645+0000","last_undegraded":"2024-09-16T15:42:25.022645+0000","last_fullsized":"2024-09-16T15:42:25.022645+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:44:41.840907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-16T15:42:23.991455+0000","last_change":"2024-09-16T15:42:23.991455+0000","last_active":"2024-09-16T15:42:23.975506+0000","last_peered":"2024-09-16T15:42:23.975506+0000","last_clean":"2024-09-16T15:42:23.975506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:42:23.991455+0000","last_undegraded":"2024-09-16T15:42:23.991455+0000","last_fullsized":"2024-09-16T15:42:23.991455+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.028105+0000","last_change":"2024-09-16T15:42:25.024263+0000","last_active":"2024-09-16T15:42:25.028105+0000","last_peered":"2024-09-16T15:42:25.028105+0000","last_clean":"2024-09-16T15:42:25.028105+0000","last_became_active":"2024-09-16T15:42:25.024006+0000","last_became_peered":"2024-09-16T15:42:25.024006+0000","last_unstale":"2024-09-16T15:42:25.028105+0000","last_undegraded":"2024-09-16T15:42:25.028105+0000","last_fullsized":"2024-09-16T15:42:25.028105+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:07:07.004615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.025458+0000","last_change":"2024-09-16T15:42:25.018894+0000","last_active":"2024-09-16T15:42:25.025458+0000","last_peered":"2024-09-16T15:42:25.025458+0000","last_clean":"2024-09-16T15:42:25.025458+0000","last_became_active":"2024-09-16T15:42:25.018023+0000","last_became_peered":"2024-09-16T15:42:25.018023+0000","last_unstale":"2024-09-16T15:42:25.025458+0000","last_undegraded":"2024-09-16T15:42:25.025458+0000","last_fullsized":"2024-09-16T15:42:25.025458+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:22:47.419249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.026541+0000","last_change":"2024-09-16T15:42:25.019057+0000","last_active":"2024-09-16T15:42:25.026541+0000","last_peered":"2024-09-16T15:42:25.026541+0000","last_clean":"2024-09-16T15:42:25.026541+0000","last_became_active":"2024-09-16T15:42:25.018803+0000","last_became_peered":"2024-09-16T15:42:25.018803+0000","last_unstale":"2024-09-16T15:42:25.026541+0000","last_undegraded":"2024-09-16T15:42:25.026541+0000","last_fullsized":"2024-09-16T15:42:25.026541+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:25:19.893903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.026062+0000","last_change":"2024-09-16T15:42:25.022339+0000","last_active":"2024-09-16T15:42:25.026062+0000","last_peered":"2024-09-16T15:42:25.026062+0000","last_clean":"2024-09-16T15:42:25.026062+0000","last_became_active":"2024-09-16T15:42:25.021986+0000","last_became_peered":"2024-09-16T15:42:25.021986+0000","last_unstale":"2024-09-16T15:42:25.026062+0000","last_undegraded":"2024-09-16T15:42:25.026062+0000","last_fullsized":"2024-09-16T15:42:25.026062+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:29:45.056795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.020215+0000","last_change":"2024-09-16T15:42:25.016721+0000","last_active":"2024-09-16T15:42:25.020215+0000","last_peered":"2024-09-16T15:42:25.020215+0000","last_clean":"2024-09-16T15:42:25.020215+0000","last_became_active":"2024-09-16T15:42:25.016383+0000","last_became_peered":"2024-09-16T15:42:25.016383+0000","last_unstale":"2024-09-16T15:42:25.020215+0000","last_undegraded":"2024-09-16T15:42:25.020215+0000","last_fullsized":"2024-09-16T15:42:25.020215+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:55:33.513714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.014293+0000","last_change":"2024-09-16T15:42:25.011808+0000","last_active":"2024-09-16T15:42:25.014293+0000","last_peered":"2024-09-16T15:42:25.014293+0000","last_clean":"2024-09-16T15:42:25.014293+0000","last_became_active":"2024-09-16T15:42:25.011410+0000","last_became_peered":"2024-09-16T15:42:25.011410+0000","last_unstale":"2024-09-16T15:42:25.014293+0000","last_undegraded":"2024-09-16T15:42:25.014293+0000","last_fullsized":"2024-09-16T15:42:25.014293+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:55:58.347779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.026371+0000","last_change":"2024-09-16T15:42:25.020917+0000","last_active":"2024-09-16T15:42:25.026371+0000","last_peered":"2024-09-16T15:42:25.026371+0000","last_clean":"2024-09-16T15:42:25.026371+0000","last_became_active":"2024-09-16T15:42:25.016487+0000","last_became_peered":"2024-09-16T15:42:25.016487+0000","last_unstale":"2024-09-16T15:42:25.026371+0000","last_undegraded":"2024-09-16T15:42:25.026371+0000","last_fullsized":"2024-09-16T15:42:25.026371+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:16:20.188903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.015755+0000","last_change":"2024-09-16T15:42:25.008914+0000","last_active":"2024-09-16T15:42:25.015755+0000","last_peered":"2024-09-16T15:42:25.015755+0000","last_clean":"2024-09-16T15:42:25.015755+0000","last_became_active":"2024-09-16T15:42:25.008631+0000","last_became_peered":"2024-09-16T15:42:25.008631+0000","last_unstale":"2024-09-16T15:42:25.015755+0000","last_undegraded":"2024-09-16T15:42:25.015755+0000","last_fullsized":"2024-09-16T15:42:25.015755+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:55:06.255740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.027325+0000","last_change":"2024-09-16T15:42:25.024127+0000","last_active":"2024-09-16T15:42:25.027325+0000","last_peered":"2024-09-16T15:42:25.027325+0000","last_clean":"2024-09-16T15:42:25.027325+0000","last_became_active":"2024-09-16T15:42:25.023603+0000","last_became_peered":"2024-09-16T15:42:25.023603+0000","last_unstale":"2024-09-16T15:42:25.027325+0000","last_undegraded":"2024-09-16T15:42:25.027325+0000","last_fullsized":"2024-09-16T15:42:25.027325+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:23:19.274003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.017833+0000","last_change":"2024-09-16T15:42:25.006974+0000","last_active":"2024-09-16T15:42:25.017833+0000","last_peered":"2024-09-16T15:42:25.017833+0000","last_clean":"2024-09-16T15:42:25.017833+0000","last_became_active":"2024-09-16T15:42:25.006382+0000","last_became_peered":"2024-09-16T15:42:25.006382+0000","last_unstale":"2024-09-16T15:42:25.017833+0000","last_undegraded":"2024-09-16T15:42:25.017833+0000","last_fullsized":"2024-09-16T15:42:25.017833+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:05.773773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-16T15:42:23.991805+0000","last_change":"2024-09-16T15:42:23.991805+0000","last_active":"2024-09-16T15:42:23.975506+0000","last_peered":"2024-09-16T15:42:23.975506+0000","last_clean":"2024-09-16T15:42:23.975506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:42:23.991805+0000","last_undegraded":"2024-09-16T15:42:23.991805+0000","last_fullsized":"2024-09-16T15:42:23.991805+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"47'4","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.027253+0000","last_change":"2024-09-16T15:42:24.001081+0000","last_active":"2024-09-16T15:42:25.027253+0000","last_peered":"2024-09-16T15:42:25.027253+0000","last_clean":"2024-09-16T15:42:25.027253+0000","last_became_active":"2024-09-16T15:42:24.000071+0000","last_became_peered":"2024-09-16T15:42:24.000071+0000","last_unstale":"2024-09-16T15:42:25.027253+0000","last_undegraded":"2024-09-16T15:42:25.027253+0000","last_fullsized":"2024-09-16T15:42:25.027253+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:24:51.105941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.026516+0000","last_change":"2024-09-16T15:42:25.016739+0000","last_active":"2024-09-16T15:42:25.026516+0000","last_peered":"2024-09-16T15:42:25.026516+0000","last_clean":"2024-09-16T15:42:25.026516+0000","last_became_active":"2024-09-16T15:42:25.016451+0000","last_became_peered":"2024-09-16T15:42:25.016451+0000","last_unstale":"2024-09-16T15:42:25.026516+0000","last_undegraded":"2024-09-16T15:42:25.026516+0000","last_fullsized":"2024-09-16T15:42:25.026516+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:05:52.459658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"47'1","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.016363+0000","last_change":"2024-09-16T15:42:24.000482+0000","last_active":"2024-09-16T15:42:25.016363+0000","last_peered":"2024-09-16T15:42:25.016363+0000","last_clean":"2024-09-16T15:42:25.016363+0000","last_became_active":"2024-09-16T15:42:24.000216+0000","last_became_peered":"2024-09-16T15:42:24.000216+0000","last_unstale":"2024-09-16T15:42:25.016363+0000","last_undegraded":"2024-09-16T15:42:25.016363+0000","last_fullsized":"2024-09-16T15:42:25.016363+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:41:02.855031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.012164+0000","last_change":"2024-09-16T15:42:25.005828+0000","last_active":"2024-09-16T15:42:25.012164+0000","last_peered":"2024-09-16T15:42:25.012164+0000","last_clean":"2024-09-16T15:42:25.012164+0000","last_became_active":"2024-09-16T15:42:25.005398+0000","last_became_peered":"2024-09-16T15:42:25.005398+0000","last_unstale":"2024-09-16T15:42:25.012164+0000","last_undegraded":"2024-09-16T15:42:25.012164+0000","last_fullsized":"2024-09-16T15:42:25.012164+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:43:52.404756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"47'2","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.025469+0000","last_change":"2024-09-16T15:42:24.002867+0000","last_active":"2024-09-16T15:42:25.025469+0000","last_peered":"2024-09-16T15:42:25.025469+0000","last_clean":"2024-09-16T15:42:25.025469+0000","last_became_active":"2024-09-16T15:42:24.002523+0000","last_became_peered":"2024-09-16T15:42:24.002523+0000","last_unstale":"2024-09-16T15:42:25.025469+0000","last_undegraded":"2024-09-16T15:42:25.025469+0000","last_fullsized":"2024-09-16T15:42:25.025469+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:57:21.332836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.016986+0000","last_change":"2024-09-16T15:42:25.006017+0000","last_active":"2024-09-16T15:42:25.016986+0000","last_peered":"2024-09-16T15:42:25.016986+0000","last_clean":"2024-09-16T15:42:25.016986+0000","last_became_active":"2024-09-16T15:42:25.005752+0000","last_became_peered":"2024-09-16T15:42:25.005752+0000","last_unstale":"2024-09-16T15:42:25.016986+0000","last_undegraded":"2024-09-16T15:42:25.016986+0000","last_fullsized":"2024-09-16T15:42:25.016986+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:49:11.296322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"47'2","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.013054+0000","last_change":"2024-09-16T15:42:24.002580+0000","last_active":"2024-09-16T15:42:25.013054+0000","last_peered":"2024-09-16T15:42:25.013054+0000","last_clean":"2024-09-16T15:42:25.013054+0000","last_became_active":"2024-09-16T15:42:24.000516+0000","last_became_peered":"2024-09-16T15:42:24.000516+0000","last_unstale":"2024-09-16T15:42:25.013054+0000","last_undegraded":"2024-09-16T15:42:25.013054+0000","last_fullsized":"2024-09-16T15:42:25.013054+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:34:18.490825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.015597+0000","last_change":"2024-09-16T15:42:25.014465+0000","last_active":"2024-09-16T15:42:25.015597+0000","last_peered":"2024-09-16T15:42:25.015597+0000","last_clean":"2024-09-16T15:42:25.015597+0000","last_became_active":"2024-09-16T15:42:25.013555+0000","last_became_peered":"2024-09-16T15:42:25.013555+0000","last_unstale":"2024-09-16T15:42:25.015597+0000","last_undegraded":"2024-09-16T15:42:25.015597+0000","last_fullsized":"2024-09-16T15:42:25.015597+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:34:09.462316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"47'1","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.019842+0000","last_change":"2024-09-16T15:42:23.997791+0000","last_active":"2024-09-16T15:42:25.019842+0000","last_peered":"2024-09-16T15:42:25.019842+0000","last_clean":"2024-09-16T15:42:25.019842+0000","last_became_active":"2024-09-16T15:42:23.997514+0000","last_became_peered":"2024-09-16T15:42:23.997514+0000","last_unstale":"2024-09-16T15:42:25.019842+0000","last_undegraded":"2024-09-16T15:42:25.019842+0000","last_fullsized":"2024-09-16T15:42:25.019842+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:59:17.932649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.000501+0000","last_change":"2024-09-16T15:42:24.996753+0000","last_active":"2024-09-16T15:42:25.000501+0000","last_peered":"2024-09-16T15:42:25.000501+0000","last_clean":"2024-09-16T15:42:25.000501+0000","last_became_active":"2024-09-16T15:42:24.996402+0000","last_became_peered":"2024-09-16T15:42:24.996402+0000","last_unstale":"2024-09-16T15:42:25.000501+0000","last_undegraded":"2024-09-16T15:42:25.000501+0000","last_fullsized":"2024-09-16T15:42:25.000501+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:26:15.252054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"47'1","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.026472+0000","last_change":"2024-09-16T15:42:23.999519+0000","last_active":"2024-09-16T15:42:25.026472+0000","last_peered":"2024-09-16T15:42:25.026472+0000","last_clean":"2024-09-16T15:42:25.026472+0000","last_became_active":"2024-09-16T15:42:23.999239+0000","last_became_peered":"2024-09-16T15:42:23.999239+0000","last_unstale":"2024-09-16T15:42:25.026472+0000","last_undegraded":"2024-09-16T15:42:25.026472+0000","last_fullsized":"2024-09-16T15:42:25.026472+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:16:13.768386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.002801+0000","last_change":"2024-09-16T15:42:24.999204+0000","last_active":"2024-09-16T15:42:25.002801+0000","last_peered":"2024-09-16T15:42:25.002801+0000","last_clean":"2024-09-16T15:42:25.002801+0000","last_became_active":"2024-09-16T15:42:24.998947+0000","last_became_peered":"2024-09-16T15:42:24.998947+0000","last_unstale":"2024-09-16T15:42:25.002801+0000","last_undegraded":"2024-09-16T15:42:25.002801+0000","last_fullsized":"2024-09-16T15:42:25.002801+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:31:38.819252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.005366+0000","last_change":"2024-09-16T15:42:24.003842+0000","last_active":"2024-09-16T15:42:25.005366+0000","last_peered":"2024-09-16T15:42:25.005366+0000","last_clean":"2024-09-16T15:42:25.005366+0000","last_became_active":"2024-09-16T15:42:24.003517+0000","last_became_peered":"2024-09-16T15:42:24.003517+0000","last_unstale":"2024-09-16T15:42:25.005366+0000","last_undegraded":"2024-09-16T15:42:25.005366+0000","last_fullsized":"2024-09-16T15:42:25.005366+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:40:42.405299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.013648+0000","last_change":"2024-09-16T15:42:25.010354+0000","last_active":"2024-09-16T15:42:25.013648+0000","last_peered":"2024-09-16T15:42:25.013648+0000","last_clean":"2024-09-16T15:42:25.013648+0000","last_became_active":"2024-09-16T15:42:25.010088+0000","last_became_peered":"2024-09-16T15:42:25.010088+0000","last_unstale":"2024-09-16T15:42:25.013648+0000","last_undegraded":"2024-09-16T15:42:25.013648+0000","last_fullsized":"2024-09-16T15:42:25.013648+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:27:39.904292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:24.990744+0000","last_change":"2024-09-16T15:42:24.006237+0000","last_active":"2024-09-16T15:42:24.990744+0000","last_peered":"2024-09-16T15:42:24.990744+0000","last_clean":"2024-09-16T15:42:24.990744+0000","last_became_active":"2024-09-16T15:42:24.005943+0000","last_became_peered":"2024-09-16T15:42:24.005943+0000","last_unstale":"2024-09-16T15:42:24.990744+0000","last_undegraded":"2024-09-16T15:42:24.990744+0000","last_fullsized":"2024-09-16T15:42:24.990744+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:49:26.725206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.009972+0000","last_change":"2024-09-16T15:42:25.005518+0000","last_active":"2024-09-16T15:42:25.009972+0000","last_peered":"2024-09-16T15:42:25.009972+0000","last_clean":"2024-09-16T15:42:25.009972+0000","last_became_active":"2024-09-16T15:42:25.005266+0000","last_became_peered":"2024-09-16T15:42:25.005266+0000","last_unstale":"2024-09-16T15:42:25.009972+0000","last_undegraded":"2024-09-16T15:42:25.009972+0000","last_fullsized":"2024-09-16T15:42:25.009972+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:23:36.266102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:24.989354+0000","last_change":"2024-09-16T15:42:23.997205+0000","last_active":"2024-09-16T15:42:24.989354+0000","last_peered":"2024-09-16T15:42:24.989354+0000","last_clean":"2024-09-16T15:42:24.989354+0000","last_became_active":"2024-09-16T15:42:23.996928+0000","last_became_peered":"2024-09-16T15:42:23.996928+0000","last_unstale":"2024-09-16T15:42:24.989354+0000","last_undegraded":"2024-09-16T15:42:24.989354+0000","last_fullsized":"2024-09-16T15:42:24.989354+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:17:58.230738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.026385+0000","last_change":"2024-09-16T15:42:25.023166+0000","last_active":"2024-09-16T15:42:25.026385+0000","last_peered":"2024-09-16T15:42:25.026385+0000","last_clean":"2024-09-16T15:42:25.026385+0000","last_became_active":"2024-09-16T15:42:25.022883+0000","last_became_peered":"2024-09-16T15:42:25.022883+0000","last_unstale":"2024-09-16T15:42:25.026385+0000","last_undegraded":"2024-09-16T15:42:25.026385+0000","last_fullsized":"2024-09-16T15:42:25.026385+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:43:30.009281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:24.989940+0000","last_change":"2024-09-16T15:42:24.007502+0000","last_active":"2024-09-16T15:42:24.989940+0000","last_peered":"2024-09-16T15:42:24.989940+0000","last_clean":"2024-09-16T15:42:24.989940+0000","last_became_active":"2024-09-16T15:42:24.007051+0000","last_became_peered":"2024-09-16T15:42:24.007051+0000","last_unstale":"2024-09-16T15:42:24.989940+0000","last_undegraded":"2024-09-16T15:42:24.989940+0000","last_fullsized":"2024-09-16T15:42:24.989940+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:42:33.038498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.015757+0000","last_change":"2024-09-16T15:42:24.998718+0000","last_active":"2024-09-16T15:42:25.015757+0000","last_peered":"2024-09-16T15:42:25.015757+0000","last_clean":"2024-09-16T15:42:25.015757+0000","last_became_active":"2024-09-16T15:42:24.998449+0000","last_became_peered":"2024-09-16T15:42:24.998449+0000","last_unstale":"2024-09-16T15:42:25.015757+0000","last_undegraded":"2024-09-16T15:42:25.015757+0000","last_fullsized":"2024-09-16T15:42:25.015757+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:10:05.060630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"47'3","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.026863+0000","last_change":"2024-09-16T15:42:24.000199+0000","last_active":"2024-09-16T15:42:25.026863+0000","last_peered":"2024-09-16T15:42:25.026863+0000","last_clean":"2024-09-16T15:42:25.026863+0000","last_became_active":"2024-09-16T15:42:23.999884+0000","last_became_peered":"2024-09-16T15:42:23.999884+0000","last_unstale":"2024-09-16T15:42:25.026863+0000","last_undegraded":"2024-09-16T15:42:25.026863+0000","last_fullsized":"2024-09-16T15:42:25.026863+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:19:10.773082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.023454+0000","last_change":"2024-09-16T15:42:25.018590+0000","last_active":"2024-09-16T15:42:25.023454+0000","last_peered":"2024-09-16T15:42:25.023454+0000","last_clean":"2024-09-16T15:42:25.023454+0000","last_became_active":"2024-09-16T15:42:25.018027+0000","last_became_peered":"2024-09-16T15:42:25.018027+0000","last_unstale":"2024-09-16T15:42:25.023454+0000","last_undegraded":"2024-09-16T15:42:25.023454+0000","last_fullsized":"2024-09-16T15:42:25.023454+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:30:22.114563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"47'6","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.031072+0000","last_change":"2024-09-16T15:42:24.005558+0000","last_active":"2024-09-16T15:42:25.031072+0000","last_peered":"2024-09-16T15:42:25.031072+0000","last_clean":"2024-09-16T15:42:25.031072+0000","last_became_active":"2024-09-16T15:42:24.004712+0000","last_became_peered":"2024-09-16T15:42:24.004712+0000","last_unstale":"2024-09-16T15:42:25.031072+0000","last_undegraded":"2024-09-16T15:42:25.031072+0000","last_fullsized":"2024-09-16T15:42:25.031072+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:44:40.838439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.008322+0000","last_change":"2024-09-16T15:42:24.997997+0000","last_active":"2024-09-16T15:42:25.008322+0000","last_peered":"2024-09-16T15:42:25.008322+0000","last_clean":"2024-09-16T15:42:25.008322+0000","last_became_active":"2024-09-16T15:42:24.997734+0000","last_became_peered":"2024-09-16T15:42:24.997734+0000","last_unstale":"2024-09-16T15:42:25.008322+0000","last_undegraded":"2024-09-16T15:42:25.008322+0000","last_fullsized":"2024-09-16T15:42:25.008322+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:52:34.420492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T15:42:24.005906+0000","last_change":"2024-09-16T15:42:24.001353+0000","last_active":"2024-09-16T15:42:24.005906+0000","last_peered":"2024-09-16T15:42:24.005906+0000","last_clean":"2024-09-16T15:42:24.005906+0000","last_became_active":"2024-09-16T15:42:24.000702+0000","last_became_peered":"2024-09-16T15:42:24.000702+0000","last_unstale":"2024-09-16T15:42:24.005906+0000","last_undegraded":"2024-09-16T15:42:24.005906+0000","last_fullsized":"2024-09-16T15:42:24.005906+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:11:04.849660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-16T15:42:23.991166+0000","last_change":"2024-09-16T15:42:23.991166+0000","last_active":"2024-09-16T15:42:23.975506+0000","last_peered":"2024-09-16T15:42:23.975506+0000","last_clean":"2024-09-16T15:42:23.975506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:42:23.991166+0000","last_undegraded":"2024-09-16T15:42:23.991166+0000","last_fullsized":"2024-09-16T15:42:23.991166+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.004484+0000","last_change":"2024-09-16T15:42:24.004317+0000","last_active":"2024-09-16T15:42:25.004484+0000","last_peered":"2024-09-16T15:42:25.004484+0000","last_clean":"2024-09-16T15:42:25.004484+0000","last_became_active":"2024-09-16T15:42:24.003249+0000","last_became_peered":"2024-09-16T15:42:24.003249+0000","last_unstale":"2024-09-16T15:42:25.004484+0000","last_undegraded":"2024-09-16T15:42:25.004484+0000","last_fullsized":"2024-09-16T15:42:25.004484+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:47:43.918079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-16T15:42:23.990754+0000","last_change":"2024-09-16T15:42:23.990754+0000","last_active":"2024-09-16T15:42:23.975506+0000","last_peered":"2024-09-16T15:42:23.975506+0000","last_clean":"2024-09-16T15:42:23.975506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:42:23.990754+0000","last_undegraded":"2024-09-16T15:42:23.990754+0000","last_fullsized":"2024-09-16T15:42:23.990754+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:24.999568+0000","last_change":"2024-09-16T15:42:23.999141+0000","last_active":"2024-09-16T15:42:24.999568+0000","last_peered":"2024-09-16T15:42:24.999568+0000","last_clean":"2024-09-16T15:42:24.999568+0000","last_became_active":"2024-09-16T15:42:23.998768+0000","last_became_peered":"2024-09-16T15:42:23.998768+0000","last_unstale":"2024-09-16T15:42:24.999568+0000","last_undegraded":"2024-09-16T15:42:24.999568+0000","last_fullsized":"2024-09-16T15:42:24.999568+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:35:18.477892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.027942+0000","last_change":"2024-09-16T15:42:25.022864+0000","last_active":"2024-09-16T15:42:25.027942+0000","last_peered":"2024-09-16T15:42:25.027942+0000","last_clean":"2024-09-16T15:42:25.027942+0000","last_became_active":"2024-09-16T15:42:25.022447+0000","last_became_peered":"2024-09-16T15:42:25.022447+0000","last_unstale":"2024-09-16T15:42:25.027942+0000","last_undegraded":"2024-09-16T15:42:25.027942+0000","last_fullsized":"2024-09-16T15:42:25.027942+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:10:55.771726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"47'8","reported_seq":21,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.026632+0000","last_change":"2024-09-16T15:42:24.004256+0000","last_active":"2024-09-16T15:42:25.026632+0000","last_peered":"2024-09-16T15:42:25.026632+0000","last_clean":"2024-09-16T15:42:25.026632+0000","last_became_active":"2024-09-16T15:42:24.002811+0000","last_became_peered":"2024-09-16T15:42:24.002811+0000","last_unstale":"2024-09-16T15:42:25.026632+0000","last_undegraded":"2024-09-16T15:42:25.026632+0000","last_fullsized":"2024-09-16T15:42:25.026632+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:29:44.054327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-16T15:42:23.990875+0000","last_change":"2024-09-16T15:42:23.990875+0000","last_active":"2024-09-16T15:42:23.975506+0000","last_peered":"2024-09-16T15:42:23.975506+0000","last_clean":"2024-09-16T15:42:23.975506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:42:23.990875+0000","last_undegraded":"2024-09-16T15:42:23.990875+0000","last_fullsized":"2024-09-16T15:42:23.990875+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"47'1","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.027090+0000","last_change":"2024-09-16T15:42:24.003907+0000","last_active":"2024-09-16T15:42:25.027090+0000","last_peered":"2024-09-16T15:42:25.027090+0000","last_clean":"2024-09-16T15:42:25.027090+0000","last_became_active":"2024-09-16T15:42:24.003647+0000","last_became_peered":"2024-09-16T15:42:24.003647+0000","last_unstale":"2024-09-16T15:42:25.027090+0000","last_undegraded":"2024-09-16T15:42:25.027090+0000","last_fullsized":"2024-09-16T15:42:25.027090+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:18:29.135782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-16T15:42:23.991889+0000","last_change":"2024-09-16T15:42:23.991889+0000","last_active":"2024-09-16T15:42:23.975506+0000","last_peered":"2024-09-16T15:42:23.975506+0000","last_clean":"2024-09-16T15:42:23.975506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:42:23.991889+0000","last_undegraded":"2024-09-16T15:42:23.991889+0000","last_fullsized":"2024-09-16T15:42:23.991889+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"47'2","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.024714+0000","last_change":"2024-09-16T15:42:23.999546+0000","last_active":"2024-09-16T15:42:25.024714+0000","last_peered":"2024-09-16T15:42:25.024714+0000","last_clean":"2024-09-16T15:42:25.024714+0000","last_became_active":"2024-09-16T15:42:23.999283+0000","last_became_peered":"2024-09-16T15:42:23.999283+0000","last_unstale":"2024-09-16T15:42:25.024714+0000","last_undegraded":"2024-09-16T15:42:25.024714+0000","last_fullsized":"2024-09-16T15:42:25.024714+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:48:31.728751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.022059+0000","last_change":"2024-09-16T15:42:25.017755+0000","last_active":"2024-09-16T15:42:25.022059+0000","last_peered":"2024-09-16T15:42:25.022059+0000","last_clean":"2024-09-16T15:42:25.022059+0000","last_became_active":"2024-09-16T15:42:25.017488+0000","last_became_peered":"2024-09-16T15:42:25.017488+0000","last_unstale":"2024-09-16T15:42:25.022059+0000","last_undegraded":"2024-09-16T15:42:25.022059+0000","last_fullsized":"2024-09-16T15:42:25.022059+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:11:08.521397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"47'1","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.025904+0000","last_change":"2024-09-16T15:42:24.006459+0000","last_active":"2024-09-16T15:42:25.025904+0000","last_peered":"2024-09-16T15:42:25.025904+0000","last_clean":"2024-09-16T15:42:25.025904+0000","last_became_active":"2024-09-16T15:42:24.006025+0000","last_became_peered":"2024-09-16T15:42:24.006025+0000","last_unstale":"2024-09-16T15:42:25.025904+0000","last_undegraded":"2024-09-16T15:42:25.025904+0000","last_fullsized":"2024-09-16T15:42:25.025904+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:48:42.868847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.022267+0000","last_change":"2024-09-16T15:42:25.011131+0000","last_active":"2024-09-16T15:42:25.022267+0000","last_peered":"2024-09-16T15:42:25.022267+0000","last_clean":"2024-09-16T15:42:25.022267+0000","last_became_active":"2024-09-16T15:42:25.008480+0000","last_became_peered":"2024-09-16T15:42:25.008480+0000","last_unstale":"2024-09-16T15:42:25.022267+0000","last_undegraded":"2024-09-16T15:42:25.022267+0000","last_fullsized":"2024-09-16T15:42:25.022267+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:49:05.814118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:24.999299+0000","last_change":"2024-09-16T15:42:24.005143+0000","last_active":"2024-09-16T15:42:24.999299+0000","last_peered":"2024-09-16T15:42:24.999299+0000","last_clean":"2024-09-16T15:42:24.999299+0000","last_became_active":"2024-09-16T15:42:24.004889+0000","last_became_peered":"2024-09-16T15:42:24.004889+0000","last_unstale":"2024-09-16T15:42:24.999299+0000","last_undegraded":"2024-09-16T15:42:24.999299+0000","last_fullsized":"2024-09-16T15:42:24.999299+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:27:58.850861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.014588+0000","last_change":"2024-09-16T15:42:25.010574+0000","last_active":"2024-09-16T15:42:25.014588+0000","last_peered":"2024-09-16T15:42:25.014588+0000","last_clean":"2024-09-16T15:42:25.014588+0000","last_became_active":"2024-09-16T15:42:25.009246+0000","last_became_peered":"2024-09-16T15:42:25.009246+0000","last_unstale":"2024-09-16T15:42:25.014588+0000","last_undegraded":"2024-09-16T15:42:25.014588+0000","last_fullsized":"2024-09-16T15:42:25.014588+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:56:12.198490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"47'2","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.026956+0000","last_change":"2024-09-16T15:42:24.007687+0000","last_active":"2024-09-16T15:42:25.026956+0000","last_peered":"2024-09-16T15:42:25.026956+0000","last_clean":"2024-09-16T15:42:25.026956+0000","last_became_active":"2024-09-16T15:42:24.006760+0000","last_became_peered":"2024-09-16T15:42:24.006760+0000","last_unstale":"2024-09-16T15:42:25.026956+0000","last_undegraded":"2024-09-16T15:42:25.026956+0000","last_fullsized":"2024-09-16T15:42:25.026956+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:16:19.186435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.022504+0000","last_change":"2024-09-16T15:42:25.012518+0000","last_active":"2024-09-16T15:42:25.022504+0000","last_peered":"2024-09-16T15:42:25.022504+0000","last_clean":"2024-09-16T15:42:25.022504+0000","last_became_active":"2024-09-16T15:42:25.012013+0000","last_became_peered":"2024-09-16T15:42:25.012013+0000","last_unstale":"2024-09-16T15:42:25.022504+0000","last_undegraded":"2024-09-16T15:42:25.022504+0000","last_fullsized":"2024-09-16T15:42:25.022504+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:18:30.138250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.005690+0000","last_change":"2024-09-16T15:42:24.007715+0000","last_active":"2024-09-16T15:42:25.005690+0000","last_peered":"2024-09-16T15:42:25.005690+0000","last_clean":"2024-09-16T15:42:25.005690+0000","last_became_active":"2024-09-16T15:42:24.007435+0000","last_became_peered":"2024-09-16T15:42:24.007435+0000","last_unstale":"2024-09-16T15:42:25.005690+0000","last_undegraded":"2024-09-16T15:42:25.005690+0000","last_fullsized":"2024-09-16T15:42:25.005690+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:23:18.271535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.017207+0000","last_change":"2024-09-16T15:42:25.007606+0000","last_active":"2024-09-16T15:42:25.017207+0000","last_peered":"2024-09-16T15:42:25.017207+0000","last_clean":"2024-09-16T15:42:25.017207+0000","last_became_active":"2024-09-16T15:42:25.006710+0000","last_became_peered":"2024-09-16T15:42:25.006710+0000","last_unstale":"2024-09-16T15:42:25.017207+0000","last_undegraded":"2024-09-16T15:42:25.017207+0000","last_fullsized":"2024-09-16T15:42:25.017207+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:05:52.459658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:24.989139+0000","last_change":"2024-09-16T15:42:24.007575+0000","last_active":"2024-09-16T15:42:24.989139+0000","last_peered":"2024-09-16T15:42:24.989139+0000","last_clean":"2024-09-16T15:42:24.989139+0000","last_became_active":"2024-09-16T15:42:24.007186+0000","last_became_peered":"2024-09-16T15:42:24.007186+0000","last_unstale":"2024-09-16T15:42:24.989139+0000","last_undegraded":"2024-09-16T15:42:24.989139+0000","last_fullsized":"2024-09-16T15:42:24.989139+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:25:18.891435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.018082+0000","last_change":"2024-09-16T15:42:25.013201+0000","last_active":"2024-09-16T15:42:25.018082+0000","last_peered":"2024-09-16T15:42:25.018082+0000","last_clean":"2024-09-16T15:42:25.018082+0000","last_became_active":"2024-09-16T15:42:25.001679+0000","last_became_peered":"2024-09-16T15:42:25.001679+0000","last_unstale":"2024-09-16T15:42:25.018082+0000","last_undegraded":"2024-09-16T15:42:25.018082+0000","last_fullsized":"2024-09-16T15:42:25.018082+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:16:20.188903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:24.993392+0000","last_change":"2024-09-16T15:42:23.995573+0000","last_active":"2024-09-16T15:42:24.993392+0000","last_peered":"2024-09-16T15:42:24.993392+0000","last_clean":"2024-09-16T15:42:24.993392+0000","last_became_active":"2024-09-16T15:42:23.995328+0000","last_became_peered":"2024-09-16T15:42:23.995328+0000","last_unstale":"2024-09-16T15:42:24.993392+0000","last_undegraded":"2024-09-16T15:42:24.993392+0000","last_fullsized":"2024-09-16T15:42:24.993392+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:10:17.404307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.013767+0000","last_change":"2024-09-16T15:42:25.010788+0000","last_active":"2024-09-16T15:42:25.013767+0000","last_peered":"2024-09-16T15:42:25.013767+0000","last_clean":"2024-09-16T15:42:25.013767+0000","last_became_active":"2024-09-16T15:42:25.010513+0000","last_became_peered":"2024-09-16T15:42:25.010513+0000","last_unstale":"2024-09-16T15:42:25.013767+0000","last_undegraded":"2024-09-16T15:42:25.013767+0000","last_fullsized":"2024-09-16T15:42:25.013767+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:31:12.499632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:24.991964+0000","last_change":"2024-09-16T15:42:24.002775+0000","last_active":"2024-09-16T15:42:24.991964+0000","last_peered":"2024-09-16T15:42:24.991964+0000","last_clean":"2024-09-16T15:42:24.991964+0000","last_became_active":"2024-09-16T15:42:24.002505+0000","last_became_peered":"2024-09-16T15:42:24.002505+0000","last_unstale":"2024-09-16T15:42:24.991964+0000","last_undegraded":"2024-09-16T15:42:24.991964+0000","last_fullsized":"2024-09-16T15:42:24.991964+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:01:01.221459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.014146+0000","last_change":"2024-09-16T15:42:25.010363+0000","last_active":"2024-09-16T15:42:25.014146+0000","last_peered":"2024-09-16T15:42:25.014146+0000","last_clean":"2024-09-16T15:42:25.014146+0000","last_became_active":"2024-09-16T15:42:25.010070+0000","last_became_peered":"2024-09-16T15:42:25.010070+0000","last_unstale":"2024-09-16T15:42:25.014146+0000","last_undegraded":"2024-09-16T15:42:25.014146+0000","last_fullsized":"2024-09-16T15:42:25.014146+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:55:24.766427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.010856+0000","last_change":"2024-09-16T15:42:24.007633+0000","last_active":"2024-09-16T15:42:25.010856+0000","last_peered":"2024-09-16T15:42:25.010856+0000","last_clean":"2024-09-16T15:42:25.010856+0000","last_became_active":"2024-09-16T15:42:24.007382+0000","last_became_peered":"2024-09-16T15:42:24.007382+0000","last_unstale":"2024-09-16T15:42:25.010856+0000","last_undegraded":"2024-09-16T15:42:25.010856+0000","last_fullsized":"2024-09-16T15:42:25.010856+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:49:10.293854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.023482+0000","last_change":"2024-09-16T15:42:25.019597+0000","last_active":"2024-09-16T15:42:25.023482+0000","last_peered":"2024-09-16T15:42:25.023482+0000","last_clean":"2024-09-16T15:42:25.023482+0000","last_became_active":"2024-09-16T15:42:25.018344+0000","last_became_peered":"2024-09-16T15:42:25.018344+0000","last_unstale":"2024-09-16T15:42:25.023482+0000","last_undegraded":"2024-09-16T15:42:25.023482+0000","last_fullsized":"2024-09-16T15:42:25.023482+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:33:44.733751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T15:42:24.008224+0000","last_change":"2024-09-16T15:42:24.007325+0000","last_active":"2024-09-16T15:42:24.008224+0000","last_peered":"2024-09-16T15:42:24.008224+0000","last_clean":"2024-09-16T15:42:24.008224+0000","last_became_active":"2024-09-16T15:42:24.006971+0000","last_became_peered":"2024-09-16T15:42:24.006971+0000","last_unstale":"2024-09-16T15:42:24.008224+0000","last_undegraded":"2024-09-16T15:42:24.008224+0000","last_fullsized":"2024-09-16T15:42:24.008224+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:11:36.047586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.014611+0000","last_change":"2024-09-16T15:42:25.011747+0000","last_active":"2024-09-16T15:42:25.014611+0000","last_peered":"2024-09-16T15:42:25.014611+0000","last_clean":"2024-09-16T15:42:25.014611+0000","last_became_active":"2024-09-16T15:42:25.009391+0000","last_became_peered":"2024-09-16T15:42:25.009391+0000","last_unstale":"2024-09-16T15:42:25.014611+0000","last_undegraded":"2024-09-16T15:42:25.014611+0000","last_fullsized":"2024-09-16T15:42:25.014611+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:48:20.712100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.004476+0000","last_change":"2024-09-16T15:42:24.005893+0000","last_active":"2024-09-16T15:42:25.004476+0000","last_peered":"2024-09-16T15:42:25.004476+0000","last_clean":"2024-09-16T15:42:25.004476+0000","last_became_active":"2024-09-16T15:42:24.005708+0000","last_became_peered":"2024-09-16T15:42:24.005708+0000","last_unstale":"2024-09-16T15:42:25.004476+0000","last_undegraded":"2024-09-16T15:42:25.004476+0000","last_fullsized":"2024-09-16T15:42:25.004476+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:10:04.058162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.018028+0000","last_change":"2024-09-16T15:42:25.014435+0000","last_active":"2024-09-16T15:42:25.018028+0000","last_peered":"2024-09-16T15:42:25.018028+0000","last_clean":"2024-09-16T15:42:25.018028+0000","last_became_active":"2024-09-16T15:42:25.014126+0000","last_became_peered":"2024-09-16T15:42:25.014126+0000","last_unstale":"2024-09-16T15:42:25.018028+0000","last_undegraded":"2024-09-16T15:42:25.018028+0000","last_fullsized":"2024-09-16T15:42:25.018028+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:42:34.040967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:24.995096+0000","last_change":"2024-09-16T15:42:24.003312+0000","last_active":"2024-09-16T15:42:24.995096+0000","last_peered":"2024-09-16T15:42:24.995096+0000","last_clean":"2024-09-16T15:42:24.995096+0000","last_became_active":"2024-09-16T15:42:24.002853+0000","last_became_peered":"2024-09-16T15:42:24.002853+0000","last_unstale":"2024-09-16T15:42:24.995096+0000","last_undegraded":"2024-09-16T15:42:24.995096+0000","last_fullsized":"2024-09-16T15:42:24.995096+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:21:53.045720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.009682+0000","last_change":"2024-09-16T15:42:25.000979+0000","last_active":"2024-09-16T15:42:25.009682+0000","last_peered":"2024-09-16T15:42:25.009682+0000","last_clean":"2024-09-16T15:42:25.009682+0000","last_became_active":"2024-09-16T15:42:25.000590+0000","last_became_peered":"2024-09-16T15:42:25.000590+0000","last_unstale":"2024-09-16T15:42:25.009682+0000","last_undegraded":"2024-09-16T15:42:25.009682+0000","last_fullsized":"2024-09-16T15:42:25.009682+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:08:12.361923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:24.999545+0000","last_change":"2024-09-16T15:42:23.999897+0000","last_active":"2024-09-16T15:42:24.999545+0000","last_peered":"2024-09-16T15:42:24.999545+0000","last_clean":"2024-09-16T15:42:24.999545+0000","last_became_active":"2024-09-16T15:42:23.999535+0000","last_became_peered":"2024-09-16T15:42:23.999535+0000","last_unstale":"2024-09-16T15:42:24.999545+0000","last_undegraded":"2024-09-16T15:42:24.999545+0000","last_fullsized":"2024-09-16T15:42:24.999545+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:46:06.187678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.018713+0000","last_change":"2024-09-16T15:42:25.013782+0000","last_active":"2024-09-16T15:42:25.018713+0000","last_peered":"2024-09-16T15:42:25.018713+0000","last_clean":"2024-09-16T15:42:25.018713+0000","last_became_active":"2024-09-16T15:42:25.013470+0000","last_became_peered":"2024-09-16T15:42:25.013470+0000","last_unstale":"2024-09-16T15:42:25.018713+0000","last_undegraded":"2024-09-16T15:42:25.018713+0000","last_fullsized":"2024-09-16T15:42:25.018713+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:56:50.552574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T15:42:24.008031+0000","last_change":"2024-09-16T15:42:24.003909+0000","last_active":"2024-09-16T15:42:24.008031+0000","last_peered":"2024-09-16T15:42:24.008031+0000","last_clean":"2024-09-16T15:42:24.008031+0000","last_became_active":"2024-09-16T15:42:24.003617+0000","last_became_peered":"2024-09-16T15:42:24.003617+0000","last_unstale":"2024-09-16T15:42:24.008031+0000","last_undegraded":"2024-09-16T15:42:24.008031+0000","last_fullsized":"2024-09-16T15:42:24.008031+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:54:56.021717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.023981+0000","last_change":"2024-09-16T15:42:25.018915+0000","last_active":"2024-09-16T15:42:25.023981+0000","last_peered":"2024-09-16T15:42:25.023981+0000","last_clean":"2024-09-16T15:42:25.023981+0000","last_became_active":"2024-09-16T15:42:25.018630+0000","last_became_peered":"2024-09-16T15:42:25.018630+0000","last_unstale":"2024-09-16T15:42:25.023981+0000","last_undegraded":"2024-09-16T15:42:25.023981+0000","last_fullsized":"2024-09-16T15:42:25.023981+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:49:27.727674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"47'2","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.022468+0000","last_change":"2024-09-16T15:42:24.002759+0000","last_active":"2024-09-16T15:42:25.022468+0000","last_peered":"2024-09-16T15:42:25.022468+0000","last_clean":"2024-09-16T15:42:25.022468+0000","last_became_active":"2024-09-16T15:42:24.002401+0000","last_became_peered":"2024-09-16T15:42:24.002401+0000","last_unstale":"2024-09-16T15:42:25.022468+0000","last_undegraded":"2024-09-16T15:42:25.022468+0000","last_fullsized":"2024-09-16T15:42:25.022468+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:35:47.858965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":158,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:24.988829+0000","last_change":"2024-09-16T15:39:12.698813+0000","last_active":"2024-09-16T15:42:24.988829+0000","last_peered":"2024-09-16T15:42:24.988829+0000","last_clean":"2024-09-16T15:42:24.988829+0000","last_became_active":"2024-09-16T15:39:12.697695+0000","last_became_peered":"2024-09-16T15:39:12.697695+0000","last_unstale":"2024-09-16T15:42:24.988829+0000","last_undegraded":"2024-09-16T15:42:24.988829+0000","last_fullsized":"2024-09-16T15:42:24.988829+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_clean_scrub_stamp":"2024-09-16T15:39:11.684967+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:02.061081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2376,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":36,"ondisk_log_size":36,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509484,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31728,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340112,"statfs":{"total":96636764160,"available":96604274688,"internally_reserved":0,"allocated":552960,"data_stored":221564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509483,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32304,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339536,"statfs":{"total":96636764160,"available":96603684864,"internally_reserved":0,"allocated":1142784,"data_stored":811338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509482,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35900,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335940,"statfs":{"total":96636764160,"available":96600002560,"internally_reserved":0,"allocated":565248,"data_stored":222652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509482,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31740,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340100,"statfs":{"total":96636764160,"available":96604262400,"internally_reserved":0,"allocated":557056,"data_stored":222560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31914441},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509482,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31724,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340116,"statfs":{"total":96636764160,"available":96604278784,"internally_reserved":0,"allocated":548864,"data_stored":221038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509483,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32288,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339552,"statfs":{"total":96636764160,"available":96603701248,"internally_reserved":0,"allocated":1126400,"data_stored":804541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509481,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31728,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340112,"statfs":{"total":96636764160,"available":96604274688,"internally_reserved":0,"allocated":552960,"data_stored":221620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542188,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31704,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340136,"statfs":{"total":96636764160,"available":96604299264,"internally_reserved":0,"allocated":520192,"data_stored":207898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":31914444},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:42:30.084 INFO:tasks.ceph.ceph_manager.ceph:PG 7.2a is not active+clean 2024-09-16T15:42:30.084 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:42:23.991455+0000', 'last_change': '2024-09-16T15:42:23.991455+0000', 'last_active': '2024-09-16T15:42:23.975506+0000', 'last_peered': '2024-09-16T15:42:23.975506+0000', 'last_clean': '2024-09-16T15:42:23.975506+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:42:23.991455+0000', 'last_undegraded': '2024-09-16T15:42:23.991455+0000', 'last_fullsized': '2024-09-16T15:42:23.991455+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:42:23.975506+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:42:23.975506+0000', 'last_clean_scrub_stamp': '2024-09-16T15:42:23.975506+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-16T15:42:30.084 INFO:tasks.ceph.ceph_manager.ceph:PG 7.1e is not active+clean 2024-09-16T15:42:30.084 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:42:23.991805+0000', 'last_change': '2024-09-16T15:42:23.991805+0000', 'last_active': '2024-09-16T15:42:23.975506+0000', 'last_peered': '2024-09-16T15:42:23.975506+0000', 'last_clean': '2024-09-16T15:42:23.975506+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:42:23.991805+0000', 'last_undegraded': '2024-09-16T15:42:23.991805+0000', 'last_fullsized': '2024-09-16T15:42:23.991805+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:42:23.975506+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:42:23.975506+0000', 'last_clean_scrub_stamp': '2024-09-16T15:42:23.975506+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-16T15:42:30.085 INFO:tasks.ceph.ceph_manager.ceph:PG 7.13 is not active+clean 2024-09-16T15:42:30.085 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:42:23.991166+0000', 'last_change': '2024-09-16T15:42:23.991166+0000', 'last_active': '2024-09-16T15:42:23.975506+0000', 'last_peered': '2024-09-16T15:42:23.975506+0000', 'last_clean': '2024-09-16T15:42:23.975506+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:42:23.991166+0000', 'last_undegraded': '2024-09-16T15:42:23.991166+0000', 'last_fullsized': '2024-09-16T15:42:23.991166+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:42:23.975506+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:42:23.975506+0000', 'last_clean_scrub_stamp': '2024-09-16T15:42:23.975506+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-16T15:42:30.085 INFO:tasks.ceph.ceph_manager.ceph:PG 7.10 is not active+clean 2024-09-16T15:42:30.085 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:42:23.990754+0000', 'last_change': '2024-09-16T15:42:23.990754+0000', 'last_active': '2024-09-16T15:42:23.975506+0000', 'last_peered': '2024-09-16T15:42:23.975506+0000', 'last_clean': '2024-09-16T15:42:23.975506+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:42:23.990754+0000', 'last_undegraded': '2024-09-16T15:42:23.990754+0000', 'last_fullsized': '2024-09-16T15:42:23.990754+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:42:23.975506+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:42:23.975506+0000', 'last_clean_scrub_stamp': '2024-09-16T15:42:23.975506+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-16T15:42:30.085 INFO:tasks.ceph.ceph_manager.ceph:PG 7.e is not active+clean 2024-09-16T15:42:30.085 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:42:23.990875+0000', 'last_change': '2024-09-16T15:42:23.990875+0000', 'last_active': '2024-09-16T15:42:23.975506+0000', 'last_peered': '2024-09-16T15:42:23.975506+0000', 'last_clean': '2024-09-16T15:42:23.975506+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:42:23.990875+0000', 'last_undegraded': '2024-09-16T15:42:23.990875+0000', 'last_fullsized': '2024-09-16T15:42:23.990875+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:42:23.975506+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:42:23.975506+0000', 'last_clean_scrub_stamp': '2024-09-16T15:42:23.975506+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-16T15:42:30.085 INFO:tasks.ceph.ceph_manager.ceph:PG 7.f is not active+clean 2024-09-16T15:42:30.085 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:42:23.991889+0000', 'last_change': '2024-09-16T15:42:23.991889+0000', 'last_active': '2024-09-16T15:42:23.975506+0000', 'last_peered': '2024-09-16T15:42:23.975506+0000', 'last_clean': '2024-09-16T15:42:23.975506+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:42:23.991889+0000', 'last_undegraded': '2024-09-16T15:42:23.991889+0000', 'last_fullsized': '2024-09-16T15:42:23.991889+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:42:23.975506+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:42:23.975506+0000', 'last_clean_scrub_stamp': '2024-09-16T15:42:23.975506+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-16T15:42:30.086 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T15:42:30.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.242+0000 7fadcce00640 1 -- 172.21.15.151:0/1636027045 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fadac003a30 msgr2=0x7fadac003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:30.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.242+0000 7fadcce00640 1 --2- 172.21.15.151:0/1636027045 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fadac003a30 0x7fadac003e50 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fadb80016c0 tx=0x7fadac005830 comp rx=0 tx=0).stop 2024-09-16T15:42:30.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.242+0000 7fadc6800640 1 -- 172.21.15.151:0/1636027045 reap_dead start 2024-09-16T15:42:30.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.242+0000 7fadcce00640 1 -- 172.21.15.151:0/1636027045 shutdown_connections 2024-09-16T15:42:30.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.242+0000 7fadcce00640 1 -- 172.21.15.151:0/1636027045 >> 172.21.15.151:0/1636027045 conn(0x7fadc80b18e0 msgr2=0x7fadc80b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:30.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.242+0000 7fadcce00640 1 -- 172.21.15.151:0/1636027045 shutdown_connections 2024-09-16T15:42:30.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.242+0000 7fadcce00640 1 -- 172.21.15.151:0/1636027045 wait complete. 2024-09-16T15:42:30.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.244+0000 7fadcce00640 1 Processor -- start 2024-09-16T15:42:30.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.244+0000 7fadcce00640 1 -- start start 2024-09-16T15:42:30.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.245+0000 7fadcce00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fadac003a30 0x7fadc8151540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:30.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.245+0000 7fadcce00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadc80b2ce0 0x7fadc80b46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:30.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.245+0000 7fadcce00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fadc80ca010 0x7fadc815e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:30.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.245+0000 7fadcce00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fadc8165220 con 0x7fadc80ca010 2024-09-16T15:42:30.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.245+0000 7fadcce00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fadc81650a0 con 0x7fadc80b2ce0 2024-09-16T15:42:30.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.245+0000 7fadcce00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fadc81653a0 con 0x7fadac003a30 2024-09-16T15:42:30.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.245+0000 7fadc6800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fadc80ca010 0x7fadc815e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:30.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.245+0000 7fadc6800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fadc80ca010 0x7fadc815e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49986/0 (socket says 172.21.15.151:49986) 2024-09-16T15:42:30.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.245+0000 7fadc6800640 1 -- 172.21.15.151:0/4178579018 learned_addr learned my addr 172.21.15.151:0/4178579018 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:42:30.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.245+0000 7fadc5e00640 1 --2- 172.21.15.151:0/4178579018 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fadac003a30 0x7fadc8151540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:30.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.245+0000 7fadbd400640 1 --2- 172.21.15.151:0/4178579018 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadc80b2ce0 0x7fadc80b46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:30.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.245+0000 7fadc6800640 1 -- 172.21.15.151:0/4178579018 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fadac003a30 msgr2=0x7fadc8151540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:30.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.245+0000 7fadc6800640 1 --2- 172.21.15.151:0/4178579018 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fadac003a30 0x7fadc8151540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:30.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.245+0000 7fadc6800640 1 -- 172.21.15.151:0/4178579018 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadc80b2ce0 msgr2=0x7fadc80b46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:30.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.245+0000 7fadc6800640 1 --2- 172.21.15.151:0/4178579018 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadc80b2ce0 0x7fadc80b46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:30.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.245+0000 7fadc6800640 1 -- 172.21.15.151:0/4178579018 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadc8151a80 con 0x7fadc80ca010 2024-09-16T15:42:30.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.245+0000 7fadbd400640 1 --2- 172.21.15.151:0/4178579018 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadc80b2ce0 0x7fadc80b46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:42:30.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.246+0000 7fadc6800640 1 --2- 172.21.15.151:0/4178579018 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fadc80ca010 0x7fadc815e420 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fadc80cfba0 tx=0x7fadc0010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:30.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.246+0000 7fadbea00640 1 -- 172.21.15.151:0/4178579018 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc000ee50 con 0x7fadc80ca010 2024-09-16T15:42:30.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.246+0000 7fadbea00640 1 -- 172.21.15.151:0/4178579018 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fadc0015070 con 0x7fadc80ca010 2024-09-16T15:42:30.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.246+0000 7fadcce00640 1 -- 172.21.15.151:0/4178579018 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadc80b3210 con 0x7fadc80ca010 2024-09-16T15:42:30.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.246+0000 7fadbea00640 1 -- 172.21.15.151:0/4178579018 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc0003d00 con 0x7fadc80ca010 2024-09-16T15:42:30.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.246+0000 7fadcce00640 1 -- 172.21.15.151:0/4178579018 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadc80b37d0 con 0x7fadc80ca010 2024-09-16T15:42:30.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.247+0000 7fadcce00640 1 -- 172.21.15.151:0/4178579018 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadc8069360 con 0x7fadc80ca010 2024-09-16T15:42:30.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.249+0000 7fadbea00640 1 -- 172.21.15.151:0/4178579018 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7fadc0004030 con 0x7fadc80ca010 2024-09-16T15:42:30.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.249+0000 7fadbea00640 1 --2- 172.21.15.151:0/4178579018 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fada4078b20 0x7fada407afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:30.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.250+0000 7fadbea00640 1 -- 172.21.15.151:0/4178579018 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7fadc009b610 con 0x7fadc80ca010 2024-09-16T15:42:30.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.250+0000 7fadc5e00640 1 --2- 172.21.15.151:0/4178579018 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fada4078b20 0x7fada407afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:30.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.251+0000 7fadc5e00640 1 --2- 172.21.15.151:0/4178579018 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fada4078b20 0x7fada407afe0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fadb8032000 tx=0x7fadb8001190 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:30.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.254+0000 7fadbea00640 1 -- 172.21.15.151:0/4178579018 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fadc0063740 con 0x7fadc80ca010 2024-09-16T15:42:30.509 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.507+0000 7fadcce00640 1 -- 172.21.15.151:0/4178579018 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fadc815fcf0 con 0x7fadc80ca010 2024-09-16T15:42:30.509 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.508+0000 7fadbea00640 1 -- 172.21.15.151:0/4178579018 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1141 (secure 0 0 0) 0x7fadc0063040 con 0x7fadc80ca010 2024-09-16T15:42:30.509 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:42:30.509 INFO:teuthology.orchestra.run.smithi151.stdout:{"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":214,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":48,"num_osds":8,"num_up_osds":8,"osd_up_since":1726501150,"num_in_osds":8,"osd_in_since":1726501140,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":91},{"state_name":"creating+peering","count":6}],"num_pgs":97,"num_pools":3,"num_objects":21,"data_bytes":592744,"bytes_used":265334784,"bytes_avail":772828778496,"bytes_total":773094113280,"inactive_pgs_ratio":0.061855670064687729,"write_bytes_sec":1194,"read_op_per_sec":0,"write_op_per_sec":6},"fsmap":{"epoch":27,"btime":"2024-09-16T15:42:28:009764+0000","id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"c","status":"up:active","gid":5400}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-16T15:41:17.604239+0000","services":{}},"progress_events":{}} 2024-09-16T15:42:30.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.510+0000 7fadcce00640 1 -- 172.21.15.151:0/4178579018 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fada4078b20 msgr2=0x7fada407afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:30.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.510+0000 7fadcce00640 1 --2- 172.21.15.151:0/4178579018 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fada4078b20 0x7fada407afe0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fadb8032000 tx=0x7fadb8001190 comp rx=0 tx=0).stop 2024-09-16T15:42:30.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.510+0000 7fadcce00640 1 -- 172.21.15.151:0/4178579018 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fadc80ca010 msgr2=0x7fadc815e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:30.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.510+0000 7fadcce00640 1 --2- 172.21.15.151:0/4178579018 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fadc80ca010 0x7fadc815e420 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fadc80cfba0 tx=0x7fadc0010f40 comp rx=0 tx=0).stop 2024-09-16T15:42:30.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.510+0000 7fadcce00640 1 -- 172.21.15.151:0/4178579018 shutdown_connections 2024-09-16T15:42:30.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.510+0000 7fadcce00640 1 --2- 172.21.15.151:0/4178579018 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fada4078b20 0x7fada407afe0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:30.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.510+0000 7fadcce00640 1 --2- 172.21.15.151:0/4178579018 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadc80b2ce0 0x7fadc80b46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:30.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.510+0000 7fadcce00640 1 --2- 172.21.15.151:0/4178579018 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fadc80ca010 0x7fadc815e420 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:30.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.510+0000 7fadcce00640 1 --2- 172.21.15.151:0/4178579018 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fadac003a30 0x7fadc8151540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:30.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.510+0000 7fadcce00640 1 -- 172.21.15.151:0/4178579018 >> 172.21.15.151:0/4178579018 conn(0x7fadc80b18e0 msgr2=0x7fadc80cc110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:30.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.510+0000 7fadcce00640 1 -- 172.21.15.151:0/4178579018 shutdown_connections 2024-09-16T15:42:30.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.510+0000 7fadcce00640 1 -- 172.21.15.151:0/4178579018 wait complete. 2024-09-16T15:42:30.528 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-16T15:42:30.528 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:42:30.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.684+0000 7f2c10e00640 1 -- 172.21.15.151:0/3801494841 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c0c15f5a0 msgr2=0x7f2c0c0b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:30.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.684+0000 7f2c10e00640 1 --2- 172.21.15.151:0/3801494841 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c0c15f5a0 0x7f2c0c0b2440 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc005fd0 tx=0x7f2bfc02a5f0 comp rx=0 tx=0).stop 2024-09-16T15:42:30.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.684+0000 7f2c10e00640 1 -- 172.21.15.151:0/3801494841 shutdown_connections 2024-09-16T15:42:30.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.684+0000 7f2c10e00640 1 --2- 172.21.15.151:0/3801494841 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c0c15f5a0 0x7f2c0c0b2440 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:30.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.684+0000 7f2c10e00640 1 -- 172.21.15.151:0/3801494841 >> 172.21.15.151:0/3801494841 conn(0x7f2c0c0b1950 msgr2=0x7f2c0c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:30.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.684+0000 7f2c10e00640 1 -- 172.21.15.151:0/3801494841 shutdown_connections 2024-09-16T15:42:30.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.684+0000 7f2c10e00640 1 -- 172.21.15.151:0/3801494841 wait complete. 2024-09-16T15:42:30.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.686+0000 7f2c10e00640 1 Processor -- start 2024-09-16T15:42:30.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.686+0000 7f2c10e00640 1 -- start start 2024-09-16T15:42:30.689 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.687+0000 7f2c10e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c0c073fb0 0x7f2c0c143240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:30.689 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.687+0000 7f2c10e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c0c0ccf00 0x7f2c0c1f8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:30.689 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.687+0000 7f2c10e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c0c15f5a0 0x7f2c0c1f8550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:30.689 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.687+0000 7f2c10e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2c0c06f9f0 con 0x7f2c0c0ccf00 2024-09-16T15:42:30.690 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.687+0000 7f2c10e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2c0c06f870 con 0x7f2c0c15f5a0 2024-09-16T15:42:30.690 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.687+0000 7f2c10e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f2c0c074420 con 0x7f2c0c073fb0 2024-09-16T15:42:30.690 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.687+0000 7f2c01400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c0c0ccf00 0x7f2c0c1f8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:30.690 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.687+0000 7f2c01400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c0c0ccf00 0x7f2c0c1f8010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49998/0 (socket says 172.21.15.151:49998) 2024-09-16T15:42:30.690 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.687+0000 7f2c01400640 1 -- 172.21.15.151:0/600393613 learned_addr learned my addr 172.21.15.151:0/600393613 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:42:30.690 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.687+0000 7f2c09e00640 1 --2- 172.21.15.151:0/600393613 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c0c073fb0 0x7f2c0c143240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:30.690 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.687+0000 7f2c0a800640 1 --2- 172.21.15.151:0/600393613 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c0c15f5a0 0x7f2c0c1f8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:30.690 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.688+0000 7f2c01400640 1 -- 172.21.15.151:0/600393613 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c0c073fb0 msgr2=0x7f2c0c143240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:30.690 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.688+0000 7f2c01400640 1 --2- 172.21.15.151:0/600393613 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c0c073fb0 0x7f2c0c143240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:30.690 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.688+0000 7f2c01400640 1 -- 172.21.15.151:0/600393613 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c0c15f5a0 msgr2=0x7f2c0c1f8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:30.690 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.688+0000 7f2c01400640 1 --2- 172.21.15.151:0/600393613 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c0c15f5a0 0x7f2c0c1f8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:30.690 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.688+0000 7f2c01400640 1 -- 172.21.15.151:0/600393613 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c0c146120 con 0x7f2c0c0ccf00 2024-09-16T15:42:30.690 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.688+0000 7f2c0a800640 1 --2- 172.21.15.151:0/600393613 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c0c15f5a0 0x7f2c0c1f8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:42:30.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.688+0000 7f2c01400640 1 --2- 172.21.15.151:0/600393613 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c0c0ccf00 0x7f2c0c1f8010 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2bf400c450 tx=0x7f2bf400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:30.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.688+0000 7f2c02a00640 1 -- 172.21.15.151:0/600393613 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf40052d0 con 0x7f2c0c0ccf00 2024-09-16T15:42:30.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.688+0000 7f2c02a00640 1 -- 172.21.15.151:0/600393613 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2bf4005470 con 0x7f2c0c0ccf00 2024-09-16T15:42:30.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.688+0000 7f2c02a00640 1 -- 172.21.15.151:0/600393613 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf4012870 con 0x7f2c0c0ccf00 2024-09-16T15:42:30.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.688+0000 7f2c10e00640 1 -- 172.21.15.151:0/600393613 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c0c0b2db0 con 0x7f2c0c0ccf00 2024-09-16T15:42:30.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.688+0000 7f2c10e00640 1 -- 172.21.15.151:0/600393613 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c0c0b35f0 con 0x7f2c0c0ccf00 2024-09-16T15:42:30.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.689+0000 7f2c10e00640 1 -- 172.21.15.151:0/600393613 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c0c0b37f0 con 0x7f2c0c0ccf00 2024-09-16T15:42:30.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.690+0000 7f2c02a00640 1 -- 172.21.15.151:0/600393613 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7f2bf40035f0 con 0x7f2c0c0ccf00 2024-09-16T15:42:30.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.691+0000 7f2c02a00640 1 --2- 172.21.15.151:0/600393613 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2be8078bb0 0x7f2be807b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:30.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.691+0000 7f2c02a00640 1 -- 172.21.15.151:0/600393613 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f2bf409ab90 con 0x7f2c0c0ccf00 2024-09-16T15:42:30.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.692+0000 7f2c09e00640 1 --2- 172.21.15.151:0/600393613 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2be8078bb0 0x7f2be807b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:30.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.693+0000 7f2c09e00640 1 --2- 172.21.15.151:0/600393613 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2be8078bb0 0x7f2be807b070 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc033000 tx=0x7f2bfc004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:30.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.695+0000 7f2c02a00640 1 -- 172.21.15.151:0/600393613 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2c0c0b37f0 con 0x7f2c0c0ccf00 2024-09-16T15:42:30.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.878+0000 7f2c10e00640 1 -- 172.21.15.151:0/600393613 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2c0c143780 con 0x7f2be8078bb0 2024-09-16T15:42:30.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.883+0000 7f2c02a00640 1 -- 172.21.15.151:0/600393613 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251449 (secure 0 0 0) 0x7f2c0c143780 con 0x7f2be8078bb0 2024-09-16T15:42:30.887 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:42:30.888 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-09-16T15:42:30.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.887+0000 7f2c10e00640 1 -- 172.21.15.151:0/600393613 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2be8078bb0 msgr2=0x7f2be807b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:30.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.887+0000 7f2c10e00640 1 --2- 172.21.15.151:0/600393613 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2be8078bb0 0x7f2be807b070 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc033000 tx=0x7f2bfc004730 comp rx=0 tx=0).stop 2024-09-16T15:42:30.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.887+0000 7f2c10e00640 1 -- 172.21.15.151:0/600393613 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c0c0ccf00 msgr2=0x7f2c0c1f8010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:30.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.887+0000 7f2c10e00640 1 --2- 172.21.15.151:0/600393613 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c0c0ccf00 0x7f2c0c1f8010 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2bf400c450 tx=0x7f2bf400c910 comp rx=0 tx=0).stop 2024-09-16T15:42:30.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.887+0000 7f2c10e00640 1 -- 172.21.15.151:0/600393613 shutdown_connections 2024-09-16T15:42:30.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.887+0000 7f2c10e00640 1 --2- 172.21.15.151:0/600393613 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2be8078bb0 0x7f2be807b070 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:30.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.887+0000 7f2c10e00640 1 --2- 172.21.15.151:0/600393613 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c0c15f5a0 0x7f2c0c1f8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:30.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.887+0000 7f2c10e00640 1 --2- 172.21.15.151:0/600393613 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c0c0ccf00 0x7f2c0c1f8010 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:30.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.887+0000 7f2c10e00640 1 --2- 172.21.15.151:0/600393613 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c0c073fb0 0x7f2c0c143240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:30.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.887+0000 7f2c10e00640 1 -- 172.21.15.151:0/600393613 >> 172.21.15.151:0/600393613 conn(0x7f2c0c0b1950 msgr2=0x7f2c0c09c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:30.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.887+0000 7f2c10e00640 1 -- 172.21.15.151:0/600393613 shutdown_connections 2024-09-16T15:42:30.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:30.887+0000 7f2c10e00640 1 -- 172.21.15.151:0/600393613 wait complete. 2024-09-16T15:42:30.905 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":141,"stamp":"2024-09-16T15:42:29.625136+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592744,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":142,"num_write_kb":1635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":108,"ondisk_log_size":108,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":109,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259116,"kb_used_data":5436,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754715604,"statfs":{"total":773094113280,"available":772828778496,"internally_reserved":0,"allocated":5566464,"data_stored":2933211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12738,"internal_metadata":259575358},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2376,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001529"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.026564+0000","last_change":"2024-09-16T15:42:25.021065+0000","last_active":"2024-09-16T15:42:25.026564+0000","last_peered":"2024-09-16T15:42:25.026564+0000","last_clean":"2024-09-16T15:42:25.026564+0000","last_became_active":"2024-09-16T15:42:25.020615+0000","last_became_peered":"2024-09-16T15:42:25.020615+0000","last_unstale":"2024-09-16T15:42:25.026564+0000","last_undegraded":"2024-09-16T15:42:25.026564+0000","last_fullsized":"2024-09-16T15:42:25.026564+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:42:34.040967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.012261+0000","last_change":"2024-09-16T15:42:25.006484+0000","last_active":"2024-09-16T15:42:25.012261+0000","last_peered":"2024-09-16T15:42:25.012261+0000","last_clean":"2024-09-16T15:42:25.012261+0000","last_became_active":"2024-09-16T15:42:25.006265+0000","last_became_peered":"2024-09-16T15:42:25.006265+0000","last_unstale":"2024-09-16T15:42:25.012261+0000","last_undegraded":"2024-09-16T15:42:25.012261+0000","last_fullsized":"2024-09-16T15:42:25.012261+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:01:36.409724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.023590+0000","last_change":"2024-09-16T15:42:25.015717+0000","last_active":"2024-09-16T15:42:25.023590+0000","last_peered":"2024-09-16T15:42:25.023590+0000","last_clean":"2024-09-16T15:42:25.023590+0000","last_became_active":"2024-09-16T15:42:25.006423+0000","last_became_peered":"2024-09-16T15:42:25.006423+0000","last_unstale":"2024-09-16T15:42:25.023590+0000","last_undegraded":"2024-09-16T15:42:25.023590+0000","last_fullsized":"2024-09-16T15:42:25.023590+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:06:46.712953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.018165+0000","last_change":"2024-09-16T15:42:25.005437+0000","last_active":"2024-09-16T15:42:25.018165+0000","last_peered":"2024-09-16T15:42:25.018165+0000","last_clean":"2024-09-16T15:42:25.018165+0000","last_became_active":"2024-09-16T15:42:25.005104+0000","last_became_peered":"2024-09-16T15:42:25.005104+0000","last_unstale":"2024-09-16T15:42:25.018165+0000","last_undegraded":"2024-09-16T15:42:25.018165+0000","last_fullsized":"2024-09-16T15:42:25.018165+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:40:43.407767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.026346+0000","last_change":"2024-09-16T15:42:25.015162+0000","last_active":"2024-09-16T15:42:25.026346+0000","last_peered":"2024-09-16T15:42:25.026346+0000","last_clean":"2024-09-16T15:42:25.026346+0000","last_became_active":"2024-09-16T15:42:25.014915+0000","last_became_peered":"2024-09-16T15:42:25.014915+0000","last_unstale":"2024-09-16T15:42:25.026346+0000","last_undegraded":"2024-09-16T15:42:25.026346+0000","last_fullsized":"2024-09-16T15:42:25.026346+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:05.773773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.022662+0000","last_change":"2024-09-16T15:42:25.018752+0000","last_active":"2024-09-16T15:42:25.022662+0000","last_peered":"2024-09-16T15:42:25.022662+0000","last_clean":"2024-09-16T15:42:25.022662+0000","last_became_active":"2024-09-16T15:42:25.016661+0000","last_became_peered":"2024-09-16T15:42:25.016661+0000","last_unstale":"2024-09-16T15:42:25.022662+0000","last_undegraded":"2024-09-16T15:42:25.022662+0000","last_fullsized":"2024-09-16T15:42:25.022662+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:10:05.060630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.007667+0000","last_change":"2024-09-16T15:42:25.005098+0000","last_active":"2024-09-16T15:42:25.007667+0000","last_peered":"2024-09-16T15:42:25.007667+0000","last_clean":"2024-09-16T15:42:25.007667+0000","last_became_active":"2024-09-16T15:42:25.004817+0000","last_became_peered":"2024-09-16T15:42:25.004817+0000","last_unstale":"2024-09-16T15:42:25.007667+0000","last_undegraded":"2024-09-16T15:42:25.007667+0000","last_fullsized":"2024-09-16T15:42:25.007667+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:12:07.715267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.016184+0000","last_change":"2024-09-16T15:42:25.003653+0000","last_active":"2024-09-16T15:42:25.016184+0000","last_peered":"2024-09-16T15:42:25.016184+0000","last_clean":"2024-09-16T15:42:25.016184+0000","last_became_active":"2024-09-16T15:42:25.002998+0000","last_became_peered":"2024-09-16T15:42:25.002998+0000","last_unstale":"2024-09-16T15:42:25.016184+0000","last_undegraded":"2024-09-16T15:42:25.016184+0000","last_fullsized":"2024-09-16T15:42:25.016184+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:23:19.274003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.022272+0000","last_change":"2024-09-16T15:42:25.013856+0000","last_active":"2024-09-16T15:42:25.022272+0000","last_peered":"2024-09-16T15:42:25.022272+0000","last_clean":"2024-09-16T15:42:25.022272+0000","last_became_active":"2024-09-16T15:42:25.013394+0000","last_became_peered":"2024-09-16T15:42:25.013394+0000","last_unstale":"2024-09-16T15:42:25.022272+0000","last_undegraded":"2024-09-16T15:42:25.022272+0000","last_fullsized":"2024-09-16T15:42:25.022272+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:01:02.223927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.017434+0000","last_change":"2024-09-16T15:42:25.012665+0000","last_active":"2024-09-16T15:42:25.017434+0000","last_peered":"2024-09-16T15:42:25.017434+0000","last_clean":"2024-09-16T15:42:25.017434+0000","last_became_active":"2024-09-16T15:42:25.012429+0000","last_became_peered":"2024-09-16T15:42:25.012429+0000","last_unstale":"2024-09-16T15:42:25.017434+0000","last_undegraded":"2024-09-16T15:42:25.017434+0000","last_fullsized":"2024-09-16T15:42:25.017434+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:06:26.478901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.017884+0000","last_change":"2024-09-16T15:42:25.011076+0000","last_active":"2024-09-16T15:42:25.017884+0000","last_peered":"2024-09-16T15:42:25.017884+0000","last_clean":"2024-09-16T15:42:25.017884+0000","last_became_active":"2024-09-16T15:42:25.010704+0000","last_became_peered":"2024-09-16T15:42:25.010704+0000","last_unstale":"2024-09-16T15:42:25.017884+0000","last_undegraded":"2024-09-16T15:42:25.017884+0000","last_fullsized":"2024-09-16T15:42:25.017884+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:32:58.789647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.012380+0000","last_change":"2024-09-16T15:42:25.007233+0000","last_active":"2024-09-16T15:42:25.012380+0000","last_peered":"2024-09-16T15:42:25.012380+0000","last_clean":"2024-09-16T15:42:25.012380+0000","last_became_active":"2024-09-16T15:42:25.006852+0000","last_became_peered":"2024-09-16T15:42:25.006852+0000","last_unstale":"2024-09-16T15:42:25.012380+0000","last_undegraded":"2024-09-16T15:42:25.012380+0000","last_fullsized":"2024-09-16T15:42:25.012380+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:24:38.283829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.026402+0000","last_change":"2024-09-16T15:42:25.023442+0000","last_active":"2024-09-16T15:42:25.026402+0000","last_peered":"2024-09-16T15:42:25.026402+0000","last_clean":"2024-09-16T15:42:25.026402+0000","last_became_active":"2024-09-16T15:42:25.022319+0000","last_became_peered":"2024-09-16T15:42:25.022319+0000","last_unstale":"2024-09-16T15:42:25.026402+0000","last_undegraded":"2024-09-16T15:42:25.026402+0000","last_fullsized":"2024-09-16T15:42:25.026402+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:47:44.920547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.016169+0000","last_change":"2024-09-16T15:42:25.001358+0000","last_active":"2024-09-16T15:42:25.016169+0000","last_peered":"2024-09-16T15:42:25.016169+0000","last_clean":"2024-09-16T15:42:25.016169+0000","last_became_active":"2024-09-16T15:42:25.001076+0000","last_became_peered":"2024-09-16T15:42:25.001076+0000","last_unstale":"2024-09-16T15:42:25.016169+0000","last_undegraded":"2024-09-16T15:42:25.016169+0000","last_fullsized":"2024-09-16T15:42:25.016169+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:47:44.920547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.018055+0000","last_change":"2024-09-16T15:42:25.012888+0000","last_active":"2024-09-16T15:42:25.018055+0000","last_peered":"2024-09-16T15:42:25.018055+0000","last_clean":"2024-09-16T15:42:25.018055+0000","last_became_active":"2024-09-16T15:42:25.009003+0000","last_became_peered":"2024-09-16T15:42:25.009003+0000","last_unstale":"2024-09-16T15:42:25.018055+0000","last_undegraded":"2024-09-16T15:42:25.018055+0000","last_fullsized":"2024-09-16T15:42:25.018055+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:25:19.893903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.026416+0000","last_change":"2024-09-16T15:42:25.017302+0000","last_active":"2024-09-16T15:42:25.026416+0000","last_peered":"2024-09-16T15:42:25.026416+0000","last_clean":"2024-09-16T15:42:25.026416+0000","last_became_active":"2024-09-16T15:42:25.017045+0000","last_became_peered":"2024-09-16T15:42:25.017045+0000","last_unstale":"2024-09-16T15:42:25.026416+0000","last_undegraded":"2024-09-16T15:42:25.026416+0000","last_fullsized":"2024-09-16T15:42:25.026416+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:32:58.789647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.027927+0000","last_change":"2024-09-16T15:42:25.021592+0000","last_active":"2024-09-16T15:42:25.027927+0000","last_peered":"2024-09-16T15:42:25.027927+0000","last_clean":"2024-09-16T15:42:25.027927+0000","last_became_active":"2024-09-16T15:42:25.021274+0000","last_became_peered":"2024-09-16T15:42:25.021274+0000","last_unstale":"2024-09-16T15:42:25.027927+0000","last_undegraded":"2024-09-16T15:42:25.027927+0000","last_fullsized":"2024-09-16T15:42:25.027927+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:15:26.922530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.004788+0000","last_change":"2024-09-16T15:42:24.995956+0000","last_active":"2024-09-16T15:42:25.004788+0000","last_peered":"2024-09-16T15:42:25.004788+0000","last_clean":"2024-09-16T15:42:25.004788+0000","last_became_active":"2024-09-16T15:42:24.995598+0000","last_became_peered":"2024-09-16T15:42:24.995598+0000","last_unstale":"2024-09-16T15:42:25.004788+0000","last_undegraded":"2024-09-16T15:42:25.004788+0000","last_fullsized":"2024-09-16T15:42:25.004788+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:13:48.480802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.029228+0000","last_change":"2024-09-16T15:42:25.024290+0000","last_active":"2024-09-16T15:42:25.029228+0000","last_peered":"2024-09-16T15:42:25.029228+0000","last_clean":"2024-09-16T15:42:25.029228+0000","last_became_active":"2024-09-16T15:42:25.024029+0000","last_became_peered":"2024-09-16T15:42:25.024029+0000","last_unstale":"2024-09-16T15:42:25.029228+0000","last_undegraded":"2024-09-16T15:42:25.029228+0000","last_fullsized":"2024-09-16T15:42:25.029228+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:44:41.840907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.011900+0000","last_change":"2024-09-16T15:42:25.009320+0000","last_active":"2024-09-16T15:42:25.011900+0000","last_peered":"2024-09-16T15:42:25.011900+0000","last_clean":"2024-09-16T15:42:25.011900+0000","last_became_active":"2024-09-16T15:42:25.008705+0000","last_became_peered":"2024-09-16T15:42:25.008705+0000","last_unstale":"2024-09-16T15:42:25.011900+0000","last_undegraded":"2024-09-16T15:42:25.011900+0000","last_fullsized":"2024-09-16T15:42:25.011900+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:25:11.105280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.022645+0000","last_change":"2024-09-16T15:42:25.014765+0000","last_active":"2024-09-16T15:42:25.022645+0000","last_peered":"2024-09-16T15:42:25.022645+0000","last_clean":"2024-09-16T15:42:25.022645+0000","last_became_active":"2024-09-16T15:42:25.009165+0000","last_became_peered":"2024-09-16T15:42:25.009165+0000","last_unstale":"2024-09-16T15:42:25.022645+0000","last_undegraded":"2024-09-16T15:42:25.022645+0000","last_fullsized":"2024-09-16T15:42:25.022645+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:44:41.840907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-16T15:42:23.991455+0000","last_change":"2024-09-16T15:42:23.991455+0000","last_active":"2024-09-16T15:42:23.975506+0000","last_peered":"2024-09-16T15:42:23.975506+0000","last_clean":"2024-09-16T15:42:23.975506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:42:23.991455+0000","last_undegraded":"2024-09-16T15:42:23.991455+0000","last_fullsized":"2024-09-16T15:42:23.991455+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.028105+0000","last_change":"2024-09-16T15:42:25.024263+0000","last_active":"2024-09-16T15:42:25.028105+0000","last_peered":"2024-09-16T15:42:25.028105+0000","last_clean":"2024-09-16T15:42:25.028105+0000","last_became_active":"2024-09-16T15:42:25.024006+0000","last_became_peered":"2024-09-16T15:42:25.024006+0000","last_unstale":"2024-09-16T15:42:25.028105+0000","last_undegraded":"2024-09-16T15:42:25.028105+0000","last_fullsized":"2024-09-16T15:42:25.028105+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:07:07.004615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.025458+0000","last_change":"2024-09-16T15:42:25.018894+0000","last_active":"2024-09-16T15:42:25.025458+0000","last_peered":"2024-09-16T15:42:25.025458+0000","last_clean":"2024-09-16T15:42:25.025458+0000","last_became_active":"2024-09-16T15:42:25.018023+0000","last_became_peered":"2024-09-16T15:42:25.018023+0000","last_unstale":"2024-09-16T15:42:25.025458+0000","last_undegraded":"2024-09-16T15:42:25.025458+0000","last_fullsized":"2024-09-16T15:42:25.025458+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:22:47.419249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.026541+0000","last_change":"2024-09-16T15:42:25.019057+0000","last_active":"2024-09-16T15:42:25.026541+0000","last_peered":"2024-09-16T15:42:25.026541+0000","last_clean":"2024-09-16T15:42:25.026541+0000","last_became_active":"2024-09-16T15:42:25.018803+0000","last_became_peered":"2024-09-16T15:42:25.018803+0000","last_unstale":"2024-09-16T15:42:25.026541+0000","last_undegraded":"2024-09-16T15:42:25.026541+0000","last_fullsized":"2024-09-16T15:42:25.026541+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:25:19.893903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.026062+0000","last_change":"2024-09-16T15:42:25.022339+0000","last_active":"2024-09-16T15:42:25.026062+0000","last_peered":"2024-09-16T15:42:25.026062+0000","last_clean":"2024-09-16T15:42:25.026062+0000","last_became_active":"2024-09-16T15:42:25.021986+0000","last_became_peered":"2024-09-16T15:42:25.021986+0000","last_unstale":"2024-09-16T15:42:25.026062+0000","last_undegraded":"2024-09-16T15:42:25.026062+0000","last_fullsized":"2024-09-16T15:42:25.026062+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:29:45.056795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.020215+0000","last_change":"2024-09-16T15:42:25.016721+0000","last_active":"2024-09-16T15:42:25.020215+0000","last_peered":"2024-09-16T15:42:25.020215+0000","last_clean":"2024-09-16T15:42:25.020215+0000","last_became_active":"2024-09-16T15:42:25.016383+0000","last_became_peered":"2024-09-16T15:42:25.016383+0000","last_unstale":"2024-09-16T15:42:25.020215+0000","last_undegraded":"2024-09-16T15:42:25.020215+0000","last_fullsized":"2024-09-16T15:42:25.020215+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:55:33.513714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.014293+0000","last_change":"2024-09-16T15:42:25.011808+0000","last_active":"2024-09-16T15:42:25.014293+0000","last_peered":"2024-09-16T15:42:25.014293+0000","last_clean":"2024-09-16T15:42:25.014293+0000","last_became_active":"2024-09-16T15:42:25.011410+0000","last_became_peered":"2024-09-16T15:42:25.011410+0000","last_unstale":"2024-09-16T15:42:25.014293+0000","last_undegraded":"2024-09-16T15:42:25.014293+0000","last_fullsized":"2024-09-16T15:42:25.014293+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:55:58.347779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.026371+0000","last_change":"2024-09-16T15:42:25.020917+0000","last_active":"2024-09-16T15:42:25.026371+0000","last_peered":"2024-09-16T15:42:25.026371+0000","last_clean":"2024-09-16T15:42:25.026371+0000","last_became_active":"2024-09-16T15:42:25.016487+0000","last_became_peered":"2024-09-16T15:42:25.016487+0000","last_unstale":"2024-09-16T15:42:25.026371+0000","last_undegraded":"2024-09-16T15:42:25.026371+0000","last_fullsized":"2024-09-16T15:42:25.026371+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:16:20.188903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.015755+0000","last_change":"2024-09-16T15:42:25.008914+0000","last_active":"2024-09-16T15:42:25.015755+0000","last_peered":"2024-09-16T15:42:25.015755+0000","last_clean":"2024-09-16T15:42:25.015755+0000","last_became_active":"2024-09-16T15:42:25.008631+0000","last_became_peered":"2024-09-16T15:42:25.008631+0000","last_unstale":"2024-09-16T15:42:25.015755+0000","last_undegraded":"2024-09-16T15:42:25.015755+0000","last_fullsized":"2024-09-16T15:42:25.015755+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:55:06.255740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.027325+0000","last_change":"2024-09-16T15:42:25.024127+0000","last_active":"2024-09-16T15:42:25.027325+0000","last_peered":"2024-09-16T15:42:25.027325+0000","last_clean":"2024-09-16T15:42:25.027325+0000","last_became_active":"2024-09-16T15:42:25.023603+0000","last_became_peered":"2024-09-16T15:42:25.023603+0000","last_unstale":"2024-09-16T15:42:25.027325+0000","last_undegraded":"2024-09-16T15:42:25.027325+0000","last_fullsized":"2024-09-16T15:42:25.027325+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:23:19.274003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.017833+0000","last_change":"2024-09-16T15:42:25.006974+0000","last_active":"2024-09-16T15:42:25.017833+0000","last_peered":"2024-09-16T15:42:25.017833+0000","last_clean":"2024-09-16T15:42:25.017833+0000","last_became_active":"2024-09-16T15:42:25.006382+0000","last_became_peered":"2024-09-16T15:42:25.006382+0000","last_unstale":"2024-09-16T15:42:25.017833+0000","last_undegraded":"2024-09-16T15:42:25.017833+0000","last_fullsized":"2024-09-16T15:42:25.017833+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:05.773773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-16T15:42:23.991805+0000","last_change":"2024-09-16T15:42:23.991805+0000","last_active":"2024-09-16T15:42:23.975506+0000","last_peered":"2024-09-16T15:42:23.975506+0000","last_clean":"2024-09-16T15:42:23.975506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:42:23.991805+0000","last_undegraded":"2024-09-16T15:42:23.991805+0000","last_fullsized":"2024-09-16T15:42:23.991805+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"47'4","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.027253+0000","last_change":"2024-09-16T15:42:24.001081+0000","last_active":"2024-09-16T15:42:25.027253+0000","last_peered":"2024-09-16T15:42:25.027253+0000","last_clean":"2024-09-16T15:42:25.027253+0000","last_became_active":"2024-09-16T15:42:24.000071+0000","last_became_peered":"2024-09-16T15:42:24.000071+0000","last_unstale":"2024-09-16T15:42:25.027253+0000","last_undegraded":"2024-09-16T15:42:25.027253+0000","last_fullsized":"2024-09-16T15:42:25.027253+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:24:51.105941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.026516+0000","last_change":"2024-09-16T15:42:25.016739+0000","last_active":"2024-09-16T15:42:25.026516+0000","last_peered":"2024-09-16T15:42:25.026516+0000","last_clean":"2024-09-16T15:42:25.026516+0000","last_became_active":"2024-09-16T15:42:25.016451+0000","last_became_peered":"2024-09-16T15:42:25.016451+0000","last_unstale":"2024-09-16T15:42:25.026516+0000","last_undegraded":"2024-09-16T15:42:25.026516+0000","last_fullsized":"2024-09-16T15:42:25.026516+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:05:52.459658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"47'1","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.016363+0000","last_change":"2024-09-16T15:42:24.000482+0000","last_active":"2024-09-16T15:42:25.016363+0000","last_peered":"2024-09-16T15:42:25.016363+0000","last_clean":"2024-09-16T15:42:25.016363+0000","last_became_active":"2024-09-16T15:42:24.000216+0000","last_became_peered":"2024-09-16T15:42:24.000216+0000","last_unstale":"2024-09-16T15:42:25.016363+0000","last_undegraded":"2024-09-16T15:42:25.016363+0000","last_fullsized":"2024-09-16T15:42:25.016363+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:41:02.855031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.012164+0000","last_change":"2024-09-16T15:42:25.005828+0000","last_active":"2024-09-16T15:42:25.012164+0000","last_peered":"2024-09-16T15:42:25.012164+0000","last_clean":"2024-09-16T15:42:25.012164+0000","last_became_active":"2024-09-16T15:42:25.005398+0000","last_became_peered":"2024-09-16T15:42:25.005398+0000","last_unstale":"2024-09-16T15:42:25.012164+0000","last_undegraded":"2024-09-16T15:42:25.012164+0000","last_fullsized":"2024-09-16T15:42:25.012164+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:43:52.404756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"47'2","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.025469+0000","last_change":"2024-09-16T15:42:24.002867+0000","last_active":"2024-09-16T15:42:25.025469+0000","last_peered":"2024-09-16T15:42:25.025469+0000","last_clean":"2024-09-16T15:42:25.025469+0000","last_became_active":"2024-09-16T15:42:24.002523+0000","last_became_peered":"2024-09-16T15:42:24.002523+0000","last_unstale":"2024-09-16T15:42:25.025469+0000","last_undegraded":"2024-09-16T15:42:25.025469+0000","last_fullsized":"2024-09-16T15:42:25.025469+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:57:21.332836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.016986+0000","last_change":"2024-09-16T15:42:25.006017+0000","last_active":"2024-09-16T15:42:25.016986+0000","last_peered":"2024-09-16T15:42:25.016986+0000","last_clean":"2024-09-16T15:42:25.016986+0000","last_became_active":"2024-09-16T15:42:25.005752+0000","last_became_peered":"2024-09-16T15:42:25.005752+0000","last_unstale":"2024-09-16T15:42:25.016986+0000","last_undegraded":"2024-09-16T15:42:25.016986+0000","last_fullsized":"2024-09-16T15:42:25.016986+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:49:11.296322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"47'2","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.013054+0000","last_change":"2024-09-16T15:42:24.002580+0000","last_active":"2024-09-16T15:42:25.013054+0000","last_peered":"2024-09-16T15:42:25.013054+0000","last_clean":"2024-09-16T15:42:25.013054+0000","last_became_active":"2024-09-16T15:42:24.000516+0000","last_became_peered":"2024-09-16T15:42:24.000516+0000","last_unstale":"2024-09-16T15:42:25.013054+0000","last_undegraded":"2024-09-16T15:42:25.013054+0000","last_fullsized":"2024-09-16T15:42:25.013054+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:34:18.490825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.015597+0000","last_change":"2024-09-16T15:42:25.014465+0000","last_active":"2024-09-16T15:42:25.015597+0000","last_peered":"2024-09-16T15:42:25.015597+0000","last_clean":"2024-09-16T15:42:25.015597+0000","last_became_active":"2024-09-16T15:42:25.013555+0000","last_became_peered":"2024-09-16T15:42:25.013555+0000","last_unstale":"2024-09-16T15:42:25.015597+0000","last_undegraded":"2024-09-16T15:42:25.015597+0000","last_fullsized":"2024-09-16T15:42:25.015597+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:34:09.462316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"47'1","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.019842+0000","last_change":"2024-09-16T15:42:23.997791+0000","last_active":"2024-09-16T15:42:25.019842+0000","last_peered":"2024-09-16T15:42:25.019842+0000","last_clean":"2024-09-16T15:42:25.019842+0000","last_became_active":"2024-09-16T15:42:23.997514+0000","last_became_peered":"2024-09-16T15:42:23.997514+0000","last_unstale":"2024-09-16T15:42:25.019842+0000","last_undegraded":"2024-09-16T15:42:25.019842+0000","last_fullsized":"2024-09-16T15:42:25.019842+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:59:17.932649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.000501+0000","last_change":"2024-09-16T15:42:24.996753+0000","last_active":"2024-09-16T15:42:25.000501+0000","last_peered":"2024-09-16T15:42:25.000501+0000","last_clean":"2024-09-16T15:42:25.000501+0000","last_became_active":"2024-09-16T15:42:24.996402+0000","last_became_peered":"2024-09-16T15:42:24.996402+0000","last_unstale":"2024-09-16T15:42:25.000501+0000","last_undegraded":"2024-09-16T15:42:25.000501+0000","last_fullsized":"2024-09-16T15:42:25.000501+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:26:15.252054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"47'1","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.026472+0000","last_change":"2024-09-16T15:42:23.999519+0000","last_active":"2024-09-16T15:42:25.026472+0000","last_peered":"2024-09-16T15:42:25.026472+0000","last_clean":"2024-09-16T15:42:25.026472+0000","last_became_active":"2024-09-16T15:42:23.999239+0000","last_became_peered":"2024-09-16T15:42:23.999239+0000","last_unstale":"2024-09-16T15:42:25.026472+0000","last_undegraded":"2024-09-16T15:42:25.026472+0000","last_fullsized":"2024-09-16T15:42:25.026472+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:16:13.768386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.002801+0000","last_change":"2024-09-16T15:42:24.999204+0000","last_active":"2024-09-16T15:42:25.002801+0000","last_peered":"2024-09-16T15:42:25.002801+0000","last_clean":"2024-09-16T15:42:25.002801+0000","last_became_active":"2024-09-16T15:42:24.998947+0000","last_became_peered":"2024-09-16T15:42:24.998947+0000","last_unstale":"2024-09-16T15:42:25.002801+0000","last_undegraded":"2024-09-16T15:42:25.002801+0000","last_fullsized":"2024-09-16T15:42:25.002801+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:31:38.819252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.005366+0000","last_change":"2024-09-16T15:42:24.003842+0000","last_active":"2024-09-16T15:42:25.005366+0000","last_peered":"2024-09-16T15:42:25.005366+0000","last_clean":"2024-09-16T15:42:25.005366+0000","last_became_active":"2024-09-16T15:42:24.003517+0000","last_became_peered":"2024-09-16T15:42:24.003517+0000","last_unstale":"2024-09-16T15:42:25.005366+0000","last_undegraded":"2024-09-16T15:42:25.005366+0000","last_fullsized":"2024-09-16T15:42:25.005366+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:40:42.405299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.013648+0000","last_change":"2024-09-16T15:42:25.010354+0000","last_active":"2024-09-16T15:42:25.013648+0000","last_peered":"2024-09-16T15:42:25.013648+0000","last_clean":"2024-09-16T15:42:25.013648+0000","last_became_active":"2024-09-16T15:42:25.010088+0000","last_became_peered":"2024-09-16T15:42:25.010088+0000","last_unstale":"2024-09-16T15:42:25.013648+0000","last_undegraded":"2024-09-16T15:42:25.013648+0000","last_fullsized":"2024-09-16T15:42:25.013648+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:27:39.904292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:24.990744+0000","last_change":"2024-09-16T15:42:24.006237+0000","last_active":"2024-09-16T15:42:24.990744+0000","last_peered":"2024-09-16T15:42:24.990744+0000","last_clean":"2024-09-16T15:42:24.990744+0000","last_became_active":"2024-09-16T15:42:24.005943+0000","last_became_peered":"2024-09-16T15:42:24.005943+0000","last_unstale":"2024-09-16T15:42:24.990744+0000","last_undegraded":"2024-09-16T15:42:24.990744+0000","last_fullsized":"2024-09-16T15:42:24.990744+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:49:26.725206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.009972+0000","last_change":"2024-09-16T15:42:25.005518+0000","last_active":"2024-09-16T15:42:25.009972+0000","last_peered":"2024-09-16T15:42:25.009972+0000","last_clean":"2024-09-16T15:42:25.009972+0000","last_became_active":"2024-09-16T15:42:25.005266+0000","last_became_peered":"2024-09-16T15:42:25.005266+0000","last_unstale":"2024-09-16T15:42:25.009972+0000","last_undegraded":"2024-09-16T15:42:25.009972+0000","last_fullsized":"2024-09-16T15:42:25.009972+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:23:36.266102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:24.989354+0000","last_change":"2024-09-16T15:42:23.997205+0000","last_active":"2024-09-16T15:42:24.989354+0000","last_peered":"2024-09-16T15:42:24.989354+0000","last_clean":"2024-09-16T15:42:24.989354+0000","last_became_active":"2024-09-16T15:42:23.996928+0000","last_became_peered":"2024-09-16T15:42:23.996928+0000","last_unstale":"2024-09-16T15:42:24.989354+0000","last_undegraded":"2024-09-16T15:42:24.989354+0000","last_fullsized":"2024-09-16T15:42:24.989354+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:17:58.230738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.026385+0000","last_change":"2024-09-16T15:42:25.023166+0000","last_active":"2024-09-16T15:42:25.026385+0000","last_peered":"2024-09-16T15:42:25.026385+0000","last_clean":"2024-09-16T15:42:25.026385+0000","last_became_active":"2024-09-16T15:42:25.022883+0000","last_became_peered":"2024-09-16T15:42:25.022883+0000","last_unstale":"2024-09-16T15:42:25.026385+0000","last_undegraded":"2024-09-16T15:42:25.026385+0000","last_fullsized":"2024-09-16T15:42:25.026385+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:43:30.009281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:24.989940+0000","last_change":"2024-09-16T15:42:24.007502+0000","last_active":"2024-09-16T15:42:24.989940+0000","last_peered":"2024-09-16T15:42:24.989940+0000","last_clean":"2024-09-16T15:42:24.989940+0000","last_became_active":"2024-09-16T15:42:24.007051+0000","last_became_peered":"2024-09-16T15:42:24.007051+0000","last_unstale":"2024-09-16T15:42:24.989940+0000","last_undegraded":"2024-09-16T15:42:24.989940+0000","last_fullsized":"2024-09-16T15:42:24.989940+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:42:33.038498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.015757+0000","last_change":"2024-09-16T15:42:24.998718+0000","last_active":"2024-09-16T15:42:25.015757+0000","last_peered":"2024-09-16T15:42:25.015757+0000","last_clean":"2024-09-16T15:42:25.015757+0000","last_became_active":"2024-09-16T15:42:24.998449+0000","last_became_peered":"2024-09-16T15:42:24.998449+0000","last_unstale":"2024-09-16T15:42:25.015757+0000","last_undegraded":"2024-09-16T15:42:25.015757+0000","last_fullsized":"2024-09-16T15:42:25.015757+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:10:05.060630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"47'3","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.026863+0000","last_change":"2024-09-16T15:42:24.000199+0000","last_active":"2024-09-16T15:42:25.026863+0000","last_peered":"2024-09-16T15:42:25.026863+0000","last_clean":"2024-09-16T15:42:25.026863+0000","last_became_active":"2024-09-16T15:42:23.999884+0000","last_became_peered":"2024-09-16T15:42:23.999884+0000","last_unstale":"2024-09-16T15:42:25.026863+0000","last_undegraded":"2024-09-16T15:42:25.026863+0000","last_fullsized":"2024-09-16T15:42:25.026863+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:19:10.773082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.023454+0000","last_change":"2024-09-16T15:42:25.018590+0000","last_active":"2024-09-16T15:42:25.023454+0000","last_peered":"2024-09-16T15:42:25.023454+0000","last_clean":"2024-09-16T15:42:25.023454+0000","last_became_active":"2024-09-16T15:42:25.018027+0000","last_became_peered":"2024-09-16T15:42:25.018027+0000","last_unstale":"2024-09-16T15:42:25.023454+0000","last_undegraded":"2024-09-16T15:42:25.023454+0000","last_fullsized":"2024-09-16T15:42:25.023454+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:30:22.114563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"47'6","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.031072+0000","last_change":"2024-09-16T15:42:24.005558+0000","last_active":"2024-09-16T15:42:25.031072+0000","last_peered":"2024-09-16T15:42:25.031072+0000","last_clean":"2024-09-16T15:42:25.031072+0000","last_became_active":"2024-09-16T15:42:24.004712+0000","last_became_peered":"2024-09-16T15:42:24.004712+0000","last_unstale":"2024-09-16T15:42:25.031072+0000","last_undegraded":"2024-09-16T15:42:25.031072+0000","last_fullsized":"2024-09-16T15:42:25.031072+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:44:40.838439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.008322+0000","last_change":"2024-09-16T15:42:24.997997+0000","last_active":"2024-09-16T15:42:25.008322+0000","last_peered":"2024-09-16T15:42:25.008322+0000","last_clean":"2024-09-16T15:42:25.008322+0000","last_became_active":"2024-09-16T15:42:24.997734+0000","last_became_peered":"2024-09-16T15:42:24.997734+0000","last_unstale":"2024-09-16T15:42:25.008322+0000","last_undegraded":"2024-09-16T15:42:25.008322+0000","last_fullsized":"2024-09-16T15:42:25.008322+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:52:34.420492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T15:42:24.005906+0000","last_change":"2024-09-16T15:42:24.001353+0000","last_active":"2024-09-16T15:42:24.005906+0000","last_peered":"2024-09-16T15:42:24.005906+0000","last_clean":"2024-09-16T15:42:24.005906+0000","last_became_active":"2024-09-16T15:42:24.000702+0000","last_became_peered":"2024-09-16T15:42:24.000702+0000","last_unstale":"2024-09-16T15:42:24.005906+0000","last_undegraded":"2024-09-16T15:42:24.005906+0000","last_fullsized":"2024-09-16T15:42:24.005906+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:11:04.849660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-16T15:42:23.991166+0000","last_change":"2024-09-16T15:42:23.991166+0000","last_active":"2024-09-16T15:42:23.975506+0000","last_peered":"2024-09-16T15:42:23.975506+0000","last_clean":"2024-09-16T15:42:23.975506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:42:23.991166+0000","last_undegraded":"2024-09-16T15:42:23.991166+0000","last_fullsized":"2024-09-16T15:42:23.991166+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.004484+0000","last_change":"2024-09-16T15:42:24.004317+0000","last_active":"2024-09-16T15:42:25.004484+0000","last_peered":"2024-09-16T15:42:25.004484+0000","last_clean":"2024-09-16T15:42:25.004484+0000","last_became_active":"2024-09-16T15:42:24.003249+0000","last_became_peered":"2024-09-16T15:42:24.003249+0000","last_unstale":"2024-09-16T15:42:25.004484+0000","last_undegraded":"2024-09-16T15:42:25.004484+0000","last_fullsized":"2024-09-16T15:42:25.004484+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:47:43.918079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-16T15:42:23.990754+0000","last_change":"2024-09-16T15:42:23.990754+0000","last_active":"2024-09-16T15:42:23.975506+0000","last_peered":"2024-09-16T15:42:23.975506+0000","last_clean":"2024-09-16T15:42:23.975506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:42:23.990754+0000","last_undegraded":"2024-09-16T15:42:23.990754+0000","last_fullsized":"2024-09-16T15:42:23.990754+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:24.999568+0000","last_change":"2024-09-16T15:42:23.999141+0000","last_active":"2024-09-16T15:42:24.999568+0000","last_peered":"2024-09-16T15:42:24.999568+0000","last_clean":"2024-09-16T15:42:24.999568+0000","last_became_active":"2024-09-16T15:42:23.998768+0000","last_became_peered":"2024-09-16T15:42:23.998768+0000","last_unstale":"2024-09-16T15:42:24.999568+0000","last_undegraded":"2024-09-16T15:42:24.999568+0000","last_fullsized":"2024-09-16T15:42:24.999568+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:35:18.477892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.027942+0000","last_change":"2024-09-16T15:42:25.022864+0000","last_active":"2024-09-16T15:42:25.027942+0000","last_peered":"2024-09-16T15:42:25.027942+0000","last_clean":"2024-09-16T15:42:25.027942+0000","last_became_active":"2024-09-16T15:42:25.022447+0000","last_became_peered":"2024-09-16T15:42:25.022447+0000","last_unstale":"2024-09-16T15:42:25.027942+0000","last_undegraded":"2024-09-16T15:42:25.027942+0000","last_fullsized":"2024-09-16T15:42:25.027942+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:10:55.771726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"47'8","reported_seq":21,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.026632+0000","last_change":"2024-09-16T15:42:24.004256+0000","last_active":"2024-09-16T15:42:25.026632+0000","last_peered":"2024-09-16T15:42:25.026632+0000","last_clean":"2024-09-16T15:42:25.026632+0000","last_became_active":"2024-09-16T15:42:24.002811+0000","last_became_peered":"2024-09-16T15:42:24.002811+0000","last_unstale":"2024-09-16T15:42:25.026632+0000","last_undegraded":"2024-09-16T15:42:25.026632+0000","last_fullsized":"2024-09-16T15:42:25.026632+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:29:44.054327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-16T15:42:23.990875+0000","last_change":"2024-09-16T15:42:23.990875+0000","last_active":"2024-09-16T15:42:23.975506+0000","last_peered":"2024-09-16T15:42:23.975506+0000","last_clean":"2024-09-16T15:42:23.975506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:42:23.990875+0000","last_undegraded":"2024-09-16T15:42:23.990875+0000","last_fullsized":"2024-09-16T15:42:23.990875+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"47'1","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.027090+0000","last_change":"2024-09-16T15:42:24.003907+0000","last_active":"2024-09-16T15:42:25.027090+0000","last_peered":"2024-09-16T15:42:25.027090+0000","last_clean":"2024-09-16T15:42:25.027090+0000","last_became_active":"2024-09-16T15:42:24.003647+0000","last_became_peered":"2024-09-16T15:42:24.003647+0000","last_unstale":"2024-09-16T15:42:25.027090+0000","last_undegraded":"2024-09-16T15:42:25.027090+0000","last_fullsized":"2024-09-16T15:42:25.027090+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:18:29.135782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-09-16T15:42:23.991889+0000","last_change":"2024-09-16T15:42:23.991889+0000","last_active":"2024-09-16T15:42:23.975506+0000","last_peered":"2024-09-16T15:42:23.975506+0000","last_clean":"2024-09-16T15:42:23.975506+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:42:23.991889+0000","last_undegraded":"2024-09-16T15:42:23.991889+0000","last_fullsized":"2024-09-16T15:42:23.991889+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"47'2","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.024714+0000","last_change":"2024-09-16T15:42:23.999546+0000","last_active":"2024-09-16T15:42:25.024714+0000","last_peered":"2024-09-16T15:42:25.024714+0000","last_clean":"2024-09-16T15:42:25.024714+0000","last_became_active":"2024-09-16T15:42:23.999283+0000","last_became_peered":"2024-09-16T15:42:23.999283+0000","last_unstale":"2024-09-16T15:42:25.024714+0000","last_undegraded":"2024-09-16T15:42:25.024714+0000","last_fullsized":"2024-09-16T15:42:25.024714+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:48:31.728751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.022059+0000","last_change":"2024-09-16T15:42:25.017755+0000","last_active":"2024-09-16T15:42:25.022059+0000","last_peered":"2024-09-16T15:42:25.022059+0000","last_clean":"2024-09-16T15:42:25.022059+0000","last_became_active":"2024-09-16T15:42:25.017488+0000","last_became_peered":"2024-09-16T15:42:25.017488+0000","last_unstale":"2024-09-16T15:42:25.022059+0000","last_undegraded":"2024-09-16T15:42:25.022059+0000","last_fullsized":"2024-09-16T15:42:25.022059+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:11:08.521397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"47'1","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.025904+0000","last_change":"2024-09-16T15:42:24.006459+0000","last_active":"2024-09-16T15:42:25.025904+0000","last_peered":"2024-09-16T15:42:25.025904+0000","last_clean":"2024-09-16T15:42:25.025904+0000","last_became_active":"2024-09-16T15:42:24.006025+0000","last_became_peered":"2024-09-16T15:42:24.006025+0000","last_unstale":"2024-09-16T15:42:25.025904+0000","last_undegraded":"2024-09-16T15:42:25.025904+0000","last_fullsized":"2024-09-16T15:42:25.025904+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:48:42.868847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.022267+0000","last_change":"2024-09-16T15:42:25.011131+0000","last_active":"2024-09-16T15:42:25.022267+0000","last_peered":"2024-09-16T15:42:25.022267+0000","last_clean":"2024-09-16T15:42:25.022267+0000","last_became_active":"2024-09-16T15:42:25.008480+0000","last_became_peered":"2024-09-16T15:42:25.008480+0000","last_unstale":"2024-09-16T15:42:25.022267+0000","last_undegraded":"2024-09-16T15:42:25.022267+0000","last_fullsized":"2024-09-16T15:42:25.022267+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:49:05.814118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:24.999299+0000","last_change":"2024-09-16T15:42:24.005143+0000","last_active":"2024-09-16T15:42:24.999299+0000","last_peered":"2024-09-16T15:42:24.999299+0000","last_clean":"2024-09-16T15:42:24.999299+0000","last_became_active":"2024-09-16T15:42:24.004889+0000","last_became_peered":"2024-09-16T15:42:24.004889+0000","last_unstale":"2024-09-16T15:42:24.999299+0000","last_undegraded":"2024-09-16T15:42:24.999299+0000","last_fullsized":"2024-09-16T15:42:24.999299+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:27:58.850861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.014588+0000","last_change":"2024-09-16T15:42:25.010574+0000","last_active":"2024-09-16T15:42:25.014588+0000","last_peered":"2024-09-16T15:42:25.014588+0000","last_clean":"2024-09-16T15:42:25.014588+0000","last_became_active":"2024-09-16T15:42:25.009246+0000","last_became_peered":"2024-09-16T15:42:25.009246+0000","last_unstale":"2024-09-16T15:42:25.014588+0000","last_undegraded":"2024-09-16T15:42:25.014588+0000","last_fullsized":"2024-09-16T15:42:25.014588+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:56:12.198490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"47'2","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.026956+0000","last_change":"2024-09-16T15:42:24.007687+0000","last_active":"2024-09-16T15:42:25.026956+0000","last_peered":"2024-09-16T15:42:25.026956+0000","last_clean":"2024-09-16T15:42:25.026956+0000","last_became_active":"2024-09-16T15:42:24.006760+0000","last_became_peered":"2024-09-16T15:42:24.006760+0000","last_unstale":"2024-09-16T15:42:25.026956+0000","last_undegraded":"2024-09-16T15:42:25.026956+0000","last_fullsized":"2024-09-16T15:42:25.026956+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:16:19.186435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.022504+0000","last_change":"2024-09-16T15:42:25.012518+0000","last_active":"2024-09-16T15:42:25.022504+0000","last_peered":"2024-09-16T15:42:25.022504+0000","last_clean":"2024-09-16T15:42:25.022504+0000","last_became_active":"2024-09-16T15:42:25.012013+0000","last_became_peered":"2024-09-16T15:42:25.012013+0000","last_unstale":"2024-09-16T15:42:25.022504+0000","last_undegraded":"2024-09-16T15:42:25.022504+0000","last_fullsized":"2024-09-16T15:42:25.022504+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:18:30.138250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.005690+0000","last_change":"2024-09-16T15:42:24.007715+0000","last_active":"2024-09-16T15:42:25.005690+0000","last_peered":"2024-09-16T15:42:25.005690+0000","last_clean":"2024-09-16T15:42:25.005690+0000","last_became_active":"2024-09-16T15:42:24.007435+0000","last_became_peered":"2024-09-16T15:42:24.007435+0000","last_unstale":"2024-09-16T15:42:25.005690+0000","last_undegraded":"2024-09-16T15:42:25.005690+0000","last_fullsized":"2024-09-16T15:42:25.005690+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:23:18.271535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.017207+0000","last_change":"2024-09-16T15:42:25.007606+0000","last_active":"2024-09-16T15:42:25.017207+0000","last_peered":"2024-09-16T15:42:25.017207+0000","last_clean":"2024-09-16T15:42:25.017207+0000","last_became_active":"2024-09-16T15:42:25.006710+0000","last_became_peered":"2024-09-16T15:42:25.006710+0000","last_unstale":"2024-09-16T15:42:25.017207+0000","last_undegraded":"2024-09-16T15:42:25.017207+0000","last_fullsized":"2024-09-16T15:42:25.017207+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:05:52.459658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:24.989139+0000","last_change":"2024-09-16T15:42:24.007575+0000","last_active":"2024-09-16T15:42:24.989139+0000","last_peered":"2024-09-16T15:42:24.989139+0000","last_clean":"2024-09-16T15:42:24.989139+0000","last_became_active":"2024-09-16T15:42:24.007186+0000","last_became_peered":"2024-09-16T15:42:24.007186+0000","last_unstale":"2024-09-16T15:42:24.989139+0000","last_undegraded":"2024-09-16T15:42:24.989139+0000","last_fullsized":"2024-09-16T15:42:24.989139+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:25:18.891435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.018082+0000","last_change":"2024-09-16T15:42:25.013201+0000","last_active":"2024-09-16T15:42:25.018082+0000","last_peered":"2024-09-16T15:42:25.018082+0000","last_clean":"2024-09-16T15:42:25.018082+0000","last_became_active":"2024-09-16T15:42:25.001679+0000","last_became_peered":"2024-09-16T15:42:25.001679+0000","last_unstale":"2024-09-16T15:42:25.018082+0000","last_undegraded":"2024-09-16T15:42:25.018082+0000","last_fullsized":"2024-09-16T15:42:25.018082+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:16:20.188903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:24.993392+0000","last_change":"2024-09-16T15:42:23.995573+0000","last_active":"2024-09-16T15:42:24.993392+0000","last_peered":"2024-09-16T15:42:24.993392+0000","last_clean":"2024-09-16T15:42:24.993392+0000","last_became_active":"2024-09-16T15:42:23.995328+0000","last_became_peered":"2024-09-16T15:42:23.995328+0000","last_unstale":"2024-09-16T15:42:24.993392+0000","last_undegraded":"2024-09-16T15:42:24.993392+0000","last_fullsized":"2024-09-16T15:42:24.993392+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:10:17.404307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.013767+0000","last_change":"2024-09-16T15:42:25.010788+0000","last_active":"2024-09-16T15:42:25.013767+0000","last_peered":"2024-09-16T15:42:25.013767+0000","last_clean":"2024-09-16T15:42:25.013767+0000","last_became_active":"2024-09-16T15:42:25.010513+0000","last_became_peered":"2024-09-16T15:42:25.010513+0000","last_unstale":"2024-09-16T15:42:25.013767+0000","last_undegraded":"2024-09-16T15:42:25.013767+0000","last_fullsized":"2024-09-16T15:42:25.013767+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:31:12.499632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:24.991964+0000","last_change":"2024-09-16T15:42:24.002775+0000","last_active":"2024-09-16T15:42:24.991964+0000","last_peered":"2024-09-16T15:42:24.991964+0000","last_clean":"2024-09-16T15:42:24.991964+0000","last_became_active":"2024-09-16T15:42:24.002505+0000","last_became_peered":"2024-09-16T15:42:24.002505+0000","last_unstale":"2024-09-16T15:42:24.991964+0000","last_undegraded":"2024-09-16T15:42:24.991964+0000","last_fullsized":"2024-09-16T15:42:24.991964+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:01:01.221459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.014146+0000","last_change":"2024-09-16T15:42:25.010363+0000","last_active":"2024-09-16T15:42:25.014146+0000","last_peered":"2024-09-16T15:42:25.014146+0000","last_clean":"2024-09-16T15:42:25.014146+0000","last_became_active":"2024-09-16T15:42:25.010070+0000","last_became_peered":"2024-09-16T15:42:25.010070+0000","last_unstale":"2024-09-16T15:42:25.014146+0000","last_undegraded":"2024-09-16T15:42:25.014146+0000","last_fullsized":"2024-09-16T15:42:25.014146+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:55:24.766427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.010856+0000","last_change":"2024-09-16T15:42:24.007633+0000","last_active":"2024-09-16T15:42:25.010856+0000","last_peered":"2024-09-16T15:42:25.010856+0000","last_clean":"2024-09-16T15:42:25.010856+0000","last_became_active":"2024-09-16T15:42:24.007382+0000","last_became_peered":"2024-09-16T15:42:24.007382+0000","last_unstale":"2024-09-16T15:42:25.010856+0000","last_undegraded":"2024-09-16T15:42:25.010856+0000","last_fullsized":"2024-09-16T15:42:25.010856+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:49:10.293854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.023482+0000","last_change":"2024-09-16T15:42:25.019597+0000","last_active":"2024-09-16T15:42:25.023482+0000","last_peered":"2024-09-16T15:42:25.023482+0000","last_clean":"2024-09-16T15:42:25.023482+0000","last_became_active":"2024-09-16T15:42:25.018344+0000","last_became_peered":"2024-09-16T15:42:25.018344+0000","last_unstale":"2024-09-16T15:42:25.023482+0000","last_undegraded":"2024-09-16T15:42:25.023482+0000","last_fullsized":"2024-09-16T15:42:25.023482+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:33:44.733751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T15:42:24.008224+0000","last_change":"2024-09-16T15:42:24.007325+0000","last_active":"2024-09-16T15:42:24.008224+0000","last_peered":"2024-09-16T15:42:24.008224+0000","last_clean":"2024-09-16T15:42:24.008224+0000","last_became_active":"2024-09-16T15:42:24.006971+0000","last_became_peered":"2024-09-16T15:42:24.006971+0000","last_unstale":"2024-09-16T15:42:24.008224+0000","last_undegraded":"2024-09-16T15:42:24.008224+0000","last_fullsized":"2024-09-16T15:42:24.008224+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:11:36.047586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.014611+0000","last_change":"2024-09-16T15:42:25.011747+0000","last_active":"2024-09-16T15:42:25.014611+0000","last_peered":"2024-09-16T15:42:25.014611+0000","last_clean":"2024-09-16T15:42:25.014611+0000","last_became_active":"2024-09-16T15:42:25.009391+0000","last_became_peered":"2024-09-16T15:42:25.009391+0000","last_unstale":"2024-09-16T15:42:25.014611+0000","last_undegraded":"2024-09-16T15:42:25.014611+0000","last_fullsized":"2024-09-16T15:42:25.014611+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:48:20.712100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.004476+0000","last_change":"2024-09-16T15:42:24.005893+0000","last_active":"2024-09-16T15:42:25.004476+0000","last_peered":"2024-09-16T15:42:25.004476+0000","last_clean":"2024-09-16T15:42:25.004476+0000","last_became_active":"2024-09-16T15:42:24.005708+0000","last_became_peered":"2024-09-16T15:42:24.005708+0000","last_unstale":"2024-09-16T15:42:25.004476+0000","last_undegraded":"2024-09-16T15:42:25.004476+0000","last_fullsized":"2024-09-16T15:42:25.004476+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:10:04.058162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.018028+0000","last_change":"2024-09-16T15:42:25.014435+0000","last_active":"2024-09-16T15:42:25.018028+0000","last_peered":"2024-09-16T15:42:25.018028+0000","last_clean":"2024-09-16T15:42:25.018028+0000","last_became_active":"2024-09-16T15:42:25.014126+0000","last_became_peered":"2024-09-16T15:42:25.014126+0000","last_unstale":"2024-09-16T15:42:25.018028+0000","last_undegraded":"2024-09-16T15:42:25.018028+0000","last_fullsized":"2024-09-16T15:42:25.018028+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:42:34.040967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:24.995096+0000","last_change":"2024-09-16T15:42:24.003312+0000","last_active":"2024-09-16T15:42:24.995096+0000","last_peered":"2024-09-16T15:42:24.995096+0000","last_clean":"2024-09-16T15:42:24.995096+0000","last_became_active":"2024-09-16T15:42:24.002853+0000","last_became_peered":"2024-09-16T15:42:24.002853+0000","last_unstale":"2024-09-16T15:42:24.995096+0000","last_undegraded":"2024-09-16T15:42:24.995096+0000","last_fullsized":"2024-09-16T15:42:24.995096+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:21:53.045720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.009682+0000","last_change":"2024-09-16T15:42:25.000979+0000","last_active":"2024-09-16T15:42:25.009682+0000","last_peered":"2024-09-16T15:42:25.009682+0000","last_clean":"2024-09-16T15:42:25.009682+0000","last_became_active":"2024-09-16T15:42:25.000590+0000","last_became_peered":"2024-09-16T15:42:25.000590+0000","last_unstale":"2024-09-16T15:42:25.009682+0000","last_undegraded":"2024-09-16T15:42:25.009682+0000","last_fullsized":"2024-09-16T15:42:25.009682+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:08:12.361923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:24.999545+0000","last_change":"2024-09-16T15:42:23.999897+0000","last_active":"2024-09-16T15:42:24.999545+0000","last_peered":"2024-09-16T15:42:24.999545+0000","last_clean":"2024-09-16T15:42:24.999545+0000","last_became_active":"2024-09-16T15:42:23.999535+0000","last_became_peered":"2024-09-16T15:42:23.999535+0000","last_unstale":"2024-09-16T15:42:24.999545+0000","last_undegraded":"2024-09-16T15:42:24.999545+0000","last_fullsized":"2024-09-16T15:42:24.999545+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:46:06.187678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.018713+0000","last_change":"2024-09-16T15:42:25.013782+0000","last_active":"2024-09-16T15:42:25.018713+0000","last_peered":"2024-09-16T15:42:25.018713+0000","last_clean":"2024-09-16T15:42:25.018713+0000","last_became_active":"2024-09-16T15:42:25.013470+0000","last_became_peered":"2024-09-16T15:42:25.013470+0000","last_unstale":"2024-09-16T15:42:25.018713+0000","last_undegraded":"2024-09-16T15:42:25.018713+0000","last_fullsized":"2024-09-16T15:42:25.018713+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:56:50.552574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T15:42:24.008031+0000","last_change":"2024-09-16T15:42:24.003909+0000","last_active":"2024-09-16T15:42:24.008031+0000","last_peered":"2024-09-16T15:42:24.008031+0000","last_clean":"2024-09-16T15:42:24.008031+0000","last_became_active":"2024-09-16T15:42:24.003617+0000","last_became_peered":"2024-09-16T15:42:24.003617+0000","last_unstale":"2024-09-16T15:42:24.008031+0000","last_undegraded":"2024-09-16T15:42:24.008031+0000","last_fullsized":"2024-09-16T15:42:24.008031+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:54:56.021717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.023981+0000","last_change":"2024-09-16T15:42:25.018915+0000","last_active":"2024-09-16T15:42:25.023981+0000","last_peered":"2024-09-16T15:42:25.023981+0000","last_clean":"2024-09-16T15:42:25.023981+0000","last_became_active":"2024-09-16T15:42:25.018630+0000","last_became_peered":"2024-09-16T15:42:25.018630+0000","last_unstale":"2024-09-16T15:42:25.023981+0000","last_undegraded":"2024-09-16T15:42:25.023981+0000","last_fullsized":"2024-09-16T15:42:25.023981+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:49:27.727674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"47'2","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.022468+0000","last_change":"2024-09-16T15:42:24.002759+0000","last_active":"2024-09-16T15:42:25.022468+0000","last_peered":"2024-09-16T15:42:25.022468+0000","last_clean":"2024-09-16T15:42:25.022468+0000","last_became_active":"2024-09-16T15:42:24.002401+0000","last_became_peered":"2024-09-16T15:42:24.002401+0000","last_unstale":"2024-09-16T15:42:25.022468+0000","last_undegraded":"2024-09-16T15:42:25.022468+0000","last_fullsized":"2024-09-16T15:42:25.022468+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:35:47.858965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":158,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T15:42:24.988829+0000","last_change":"2024-09-16T15:39:12.698813+0000","last_active":"2024-09-16T15:42:24.988829+0000","last_peered":"2024-09-16T15:42:24.988829+0000","last_clean":"2024-09-16T15:42:24.988829+0000","last_became_active":"2024-09-16T15:39:12.697695+0000","last_became_peered":"2024-09-16T15:39:12.697695+0000","last_unstale":"2024-09-16T15:42:24.988829+0000","last_undegraded":"2024-09-16T15:42:24.988829+0000","last_fullsized":"2024-09-16T15:42:24.988829+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_clean_scrub_stamp":"2024-09-16T15:39:11.684967+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:02.061081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2376,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":36,"ondisk_log_size":36,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509484,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31728,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340112,"statfs":{"total":96636764160,"available":96604274688,"internally_reserved":0,"allocated":552960,"data_stored":221564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509483,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32304,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339536,"statfs":{"total":96636764160,"available":96603684864,"internally_reserved":0,"allocated":1142784,"data_stored":811338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509482,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35900,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335940,"statfs":{"total":96636764160,"available":96600002560,"internally_reserved":0,"allocated":565248,"data_stored":222652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509482,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31740,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340100,"statfs":{"total":96636764160,"available":96604262400,"internally_reserved":0,"allocated":557056,"data_stored":222560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31914441},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509482,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31724,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340116,"statfs":{"total":96636764160,"available":96604278784,"internally_reserved":0,"allocated":548864,"data_stored":221038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509483,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32288,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339552,"statfs":{"total":96636764160,"available":96603701248,"internally_reserved":0,"allocated":1126400,"data_stored":804541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509481,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31728,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340112,"statfs":{"total":96636764160,"available":96604274688,"internally_reserved":0,"allocated":552960,"data_stored":221620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542188,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31704,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340136,"statfs":{"total":96636764160,"available":96604299264,"internally_reserved":0,"allocated":520192,"data_stored":207898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":31914444},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:42:30.951 INFO:tasks.ceph.mds.c.smithi151.stderr:2024-09-16T15:42:30.950+0000 7ff4f2600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-16T15:42:33.910 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:42:34.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.078+0000 7ff543200640 1 -- 172.21.15.151:0/1379980198 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff528004d40 msgr2=0x7ff5280051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:34.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.078+0000 7ff543200640 1 --2- 172.21.15.151:0/1379980198 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff528004d40 0x7ff5280051c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7ff53c001830 tx=0x7ff528005830 comp rx=0 tx=0).stop 2024-09-16T15:42:34.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.078+0000 7ff542800640 1 -- 172.21.15.151:0/1379980198 reap_dead start 2024-09-16T15:42:34.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.078+0000 7ff543200640 1 -- 172.21.15.151:0/1379980198 shutdown_connections 2024-09-16T15:42:34.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.078+0000 7ff543200640 1 -- 172.21.15.151:0/1379980198 >> 172.21.15.151:0/1379980198 conn(0x7ff5440cfa20 msgr2=0x7ff5440cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:34.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.078+0000 7ff543200640 1 -- 172.21.15.151:0/1379980198 shutdown_connections 2024-09-16T15:42:34.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.078+0000 7ff543200640 1 -- 172.21.15.151:0/1379980198 wait complete. 2024-09-16T15:42:34.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.080+0000 7ff543200640 1 Processor -- start 2024-09-16T15:42:34.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.080+0000 7ff543200640 1 -- start start 2024-09-16T15:42:34.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.081+0000 7ff543200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff528004d40 0x7ff54415a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:34.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.081+0000 7ff543200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff54415a3f0 0x7ff5440c9190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:34.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.081+0000 7ff543200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5440c96d0 0x7ff5441f24d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:34.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.081+0000 7ff543200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff5440cb1c0 con 0x7ff54415a3f0 2024-09-16T15:42:34.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.081+0000 7ff543200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff5440cb040 con 0x7ff5440c96d0 2024-09-16T15:42:34.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.081+0000 7ff543200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7ff544073350 con 0x7ff528004d40 2024-09-16T15:42:34.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.081+0000 7ff541400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff54415a3f0 0x7ff5440c9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:34.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.081+0000 7ff541400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff54415a3f0 0x7ff5440c9190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36998/0 (socket says 172.21.15.151:36998) 2024-09-16T15:42:34.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.081+0000 7ff541400640 1 -- 172.21.15.151:0/3910346978 learned_addr learned my addr 172.21.15.151:0/3910346978 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:42:34.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.081+0000 7ff541e00640 1 --2- 172.21.15.151:0/3910346978 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff528004d40 0x7ff54415a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:34.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.081+0000 7ff542800640 1 --2- 172.21.15.151:0/3910346978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5440c96d0 0x7ff5441f24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:34.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.082+0000 7ff541400640 1 -- 172.21.15.151:0/3910346978 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff528004d40 msgr2=0x7ff54415a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:34.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.082+0000 7ff541400640 1 --2- 172.21.15.151:0/3910346978 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff528004d40 0x7ff54415a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:34.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.082+0000 7ff541400640 1 -- 172.21.15.151:0/3910346978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5440c96d0 msgr2=0x7ff5441f24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:34.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.082+0000 7ff541400640 1 --2- 172.21.15.151:0/3910346978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5440c96d0 0x7ff5441f24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:34.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.082+0000 7ff541400640 1 -- 172.21.15.151:0/3910346978 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff54415b170 con 0x7ff54415a3f0 2024-09-16T15:42:34.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.082+0000 7ff541400640 1 --2- 172.21.15.151:0/3910346978 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff54415a3f0 0x7ff5440c9190 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ff530006630 tx=0x7ff5300039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:34.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.082+0000 7ff53a000640 1 -- 172.21.15.151:0/3910346978 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff530015070 con 0x7ff54415a3f0 2024-09-16T15:42:34.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.082+0000 7ff53a000640 1 -- 172.21.15.151:0/3910346978 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff530011920 con 0x7ff54415a3f0 2024-09-16T15:42:34.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.082+0000 7ff53a000640 1 -- 172.21.15.151:0/3910346978 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff530011d70 con 0x7ff54415a3f0 2024-09-16T15:42:34.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.082+0000 7ff543200640 1 -- 172.21.15.151:0/3910346978 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff544159930 con 0x7ff54415a3f0 2024-09-16T15:42:34.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.082+0000 7ff543200640 1 -- 172.21.15.151:0/3910346978 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff544159ef0 con 0x7ff54415a3f0 2024-09-16T15:42:34.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.083+0000 7ff543200640 1 -- 172.21.15.151:0/3910346978 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5440c9ae0 con 0x7ff54415a3f0 2024-09-16T15:42:34.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.085+0000 7ff53a000640 1 -- 172.21.15.151:0/3910346978 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7ff530018070 con 0x7ff54415a3f0 2024-09-16T15:42:34.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.085+0000 7ff53a000640 1 --2- 172.21.15.151:0/3910346978 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff524078bb0 0x7ff52407b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:34.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.086+0000 7ff53a000640 1 -- 172.21.15.151:0/3910346978 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7ff53009be40 con 0x7ff54415a3f0 2024-09-16T15:42:34.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.086+0000 7ff541e00640 1 --2- 172.21.15.151:0/3910346978 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff524078bb0 0x7ff52407b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:34.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.087+0000 7ff541e00640 1 --2- 172.21.15.151:0/3910346978 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff524078bb0 0x7ff52407b070 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff5441576e0 tx=0x7ff534009500 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:34.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.090+0000 7ff53a000640 1 -- 172.21.15.151:0/3910346978 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff5440c9ae0 con 0x7ff54415a3f0 2024-09-16T15:42:34.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.273+0000 7ff543200640 1 -- 172.21.15.151:0/3910346978 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff54415dc30 con 0x7ff524078bb0 2024-09-16T15:42:34.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.278+0000 7ff53a000640 1 -- 172.21.15.151:0/3910346978 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251936 (secure 0 0 0) 0x7ff54415dc30 con 0x7ff524078bb0 2024-09-16T15:42:34.282 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:42:34.283 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-09-16T15:42:34.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.281+0000 7ff543200640 1 -- 172.21.15.151:0/3910346978 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff524078bb0 msgr2=0x7ff52407b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:34.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.281+0000 7ff543200640 1 --2- 172.21.15.151:0/3910346978 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff524078bb0 0x7ff52407b070 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff5441576e0 tx=0x7ff534009500 comp rx=0 tx=0).stop 2024-09-16T15:42:34.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.282+0000 7ff543200640 1 -- 172.21.15.151:0/3910346978 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff54415a3f0 msgr2=0x7ff5440c9190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:34.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.282+0000 7ff543200640 1 --2- 172.21.15.151:0/3910346978 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff54415a3f0 0x7ff5440c9190 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ff530006630 tx=0x7ff5300039c0 comp rx=0 tx=0).stop 2024-09-16T15:42:34.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.282+0000 7ff543200640 1 -- 172.21.15.151:0/3910346978 shutdown_connections 2024-09-16T15:42:34.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.282+0000 7ff543200640 1 --2- 172.21.15.151:0/3910346978 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff524078bb0 0x7ff52407b070 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:34.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.282+0000 7ff543200640 1 --2- 172.21.15.151:0/3910346978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5440c96d0 0x7ff5441f24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:34.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.282+0000 7ff543200640 1 --2- 172.21.15.151:0/3910346978 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff54415a3f0 0x7ff5440c9190 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:34.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.282+0000 7ff543200640 1 --2- 172.21.15.151:0/3910346978 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff528004d40 0x7ff54415a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:34.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.282+0000 7ff543200640 1 -- 172.21.15.151:0/3910346978 >> 172.21.15.151:0/3910346978 conn(0x7ff5440cfa20 msgr2=0x7ff5440b2a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:34.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.282+0000 7ff543200640 1 -- 172.21.15.151:0/3910346978 shutdown_connections 2024-09-16T15:42:34.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.282+0000 7ff543200640 1 -- 172.21.15.151:0/3910346978 wait complete. 2024-09-16T15:42:34.299 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":143,"stamp":"2024-09-16T15:42:33.626154+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259268,"kb_used_data":5576,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754715452,"statfs":{"total":773094113280,"available":772828622848,"internally_reserved":0,"allocated":5709824,"data_stored":3003540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12738,"internal_metadata":259575358},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.640203"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:26.032504+0000","last_change":"2024-09-16T15:42:25.021065+0000","last_active":"2024-09-16T15:42:26.032504+0000","last_peered":"2024-09-16T15:42:26.032504+0000","last_clean":"2024-09-16T15:42:26.032504+0000","last_became_active":"2024-09-16T15:42:25.020615+0000","last_became_peered":"2024-09-16T15:42:25.020615+0000","last_unstale":"2024-09-16T15:42:26.032504+0000","last_undegraded":"2024-09-16T15:42:26.032504+0000","last_fullsized":"2024-09-16T15:42:26.032504+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:06:06.403757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:26.840988+0000","last_change":"2024-09-16T15:42:25.006484+0000","last_active":"2024-09-16T15:42:26.840988+0000","last_peered":"2024-09-16T15:42:26.840988+0000","last_clean":"2024-09-16T15:42:26.840988+0000","last_became_active":"2024-09-16T15:42:25.006265+0000","last_became_peered":"2024-09-16T15:42:25.006265+0000","last_unstale":"2024-09-16T15:42:26.840988+0000","last_undegraded":"2024-09-16T15:42:26.840988+0000","last_fullsized":"2024-09-16T15:42:26.840988+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:53:44.198668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:26.841183+0000","last_change":"2024-09-16T15:42:25.015717+0000","last_active":"2024-09-16T15:42:26.841183+0000","last_peered":"2024-09-16T15:42:26.841183+0000","last_clean":"2024-09-16T15:42:26.841183+0000","last_became_active":"2024-09-16T15:42:25.006423+0000","last_became_peered":"2024-09-16T15:42:25.006423+0000","last_unstale":"2024-09-16T15:42:26.841183+0000","last_undegraded":"2024-09-16T15:42:26.841183+0000","last_fullsized":"2024-09-16T15:42:26.841183+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:18:32.212580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:26.841629+0000","last_change":"2024-09-16T15:42:25.005437+0000","last_active":"2024-09-16T15:42:26.841629+0000","last_peered":"2024-09-16T15:42:26.841629+0000","last_clean":"2024-09-16T15:42:26.841629+0000","last_became_active":"2024-09-16T15:42:25.005104+0000","last_became_peered":"2024-09-16T15:42:25.005104+0000","last_unstale":"2024-09-16T15:42:26.841629+0000","last_undegraded":"2024-09-16T15:42:26.841629+0000","last_fullsized":"2024-09-16T15:42:26.841629+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:47:42.974655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:26.032238+0000","last_change":"2024-09-16T15:42:25.015162+0000","last_active":"2024-09-16T15:42:26.032238+0000","last_peered":"2024-09-16T15:42:26.032238+0000","last_clean":"2024-09-16T15:42:26.032238+0000","last_became_active":"2024-09-16T15:42:25.014915+0000","last_became_peered":"2024-09-16T15:42:25.014915+0000","last_unstale":"2024-09-16T15:42:26.032238+0000","last_undegraded":"2024-09-16T15:42:26.032238+0000","last_fullsized":"2024-09-16T15:42:26.032238+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:25:33.890418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:26.840819+0000","last_change":"2024-09-16T15:42:25.018752+0000","last_active":"2024-09-16T15:42:26.840819+0000","last_peered":"2024-09-16T15:42:26.840819+0000","last_clean":"2024-09-16T15:42:26.840819+0000","last_became_active":"2024-09-16T15:42:25.016661+0000","last_became_peered":"2024-09-16T15:42:25.016661+0000","last_unstale":"2024-09-16T15:42:26.840819+0000","last_undegraded":"2024-09-16T15:42:26.840819+0000","last_fullsized":"2024-09-16T15:42:26.840819+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:40:43.407767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.990550+0000","last_change":"2024-09-16T15:42:25.005098+0000","last_active":"2024-09-16T15:42:25.990550+0000","last_peered":"2024-09-16T15:42:25.990550+0000","last_clean":"2024-09-16T15:42:25.990550+0000","last_became_active":"2024-09-16T15:42:25.004817+0000","last_became_peered":"2024-09-16T15:42:25.004817+0000","last_unstale":"2024-09-16T15:42:25.990550+0000","last_undegraded":"2024-09-16T15:42:25.990550+0000","last_fullsized":"2024-09-16T15:42:25.990550+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:31:01.170400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:26.840448+0000","last_change":"2024-09-16T15:42:25.003653+0000","last_active":"2024-09-16T15:42:26.840448+0000","last_peered":"2024-09-16T15:42:26.840448+0000","last_clean":"2024-09-16T15:42:26.840448+0000","last_became_active":"2024-09-16T15:42:25.002998+0000","last_became_peered":"2024-09-16T15:42:25.002998+0000","last_unstale":"2024-09-16T15:42:26.840448+0000","last_undegraded":"2024-09-16T15:42:26.840448+0000","last_fullsized":"2024-09-16T15:42:26.840448+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:53:56.337177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:26.841353+0000","last_change":"2024-09-16T15:42:25.013856+0000","last_active":"2024-09-16T15:42:26.841353+0000","last_peered":"2024-09-16T15:42:26.841353+0000","last_clean":"2024-09-16T15:42:26.841353+0000","last_became_active":"2024-09-16T15:42:25.013394+0000","last_became_peered":"2024-09-16T15:42:25.013394+0000","last_unstale":"2024-09-16T15:42:26.841353+0000","last_undegraded":"2024-09-16T15:42:26.841353+0000","last_fullsized":"2024-09-16T15:42:26.841353+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:32:58.789647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.991278+0000","last_change":"2024-09-16T15:42:25.012665+0000","last_active":"2024-09-16T15:42:25.991278+0000","last_peered":"2024-09-16T15:42:25.991278+0000","last_clean":"2024-09-16T15:42:25.991278+0000","last_became_active":"2024-09-16T15:42:25.012429+0000","last_became_peered":"2024-09-16T15:42:25.012429+0000","last_unstale":"2024-09-16T15:42:25.991278+0000","last_undegraded":"2024-09-16T15:42:25.991278+0000","last_fullsized":"2024-09-16T15:42:25.991278+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:28:57.800376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:26.841446+0000","last_change":"2024-09-16T15:42:25.011076+0000","last_active":"2024-09-16T15:42:26.841446+0000","last_peered":"2024-09-16T15:42:26.841446+0000","last_clean":"2024-09-16T15:42:26.841446+0000","last_became_active":"2024-09-16T15:42:25.010704+0000","last_became_peered":"2024-09-16T15:42:25.010704+0000","last_unstale":"2024-09-16T15:42:26.841446+0000","last_undegraded":"2024-09-16T15:42:26.841446+0000","last_fullsized":"2024-09-16T15:42:26.841446+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:08:00.910495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:26.840433+0000","last_change":"2024-09-16T15:42:25.007233+0000","last_active":"2024-09-16T15:42:26.840433+0000","last_peered":"2024-09-16T15:42:26.840433+0000","last_clean":"2024-09-16T15:42:26.840433+0000","last_became_active":"2024-09-16T15:42:25.006852+0000","last_became_peered":"2024-09-16T15:42:25.006852+0000","last_unstale":"2024-09-16T15:42:26.840433+0000","last_undegraded":"2024-09-16T15:42:26.840433+0000","last_fullsized":"2024-09-16T15:42:26.840433+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:35:29.545840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:26.841041+0000","last_change":"2024-09-16T15:42:25.023442+0000","last_active":"2024-09-16T15:42:26.841041+0000","last_peered":"2024-09-16T15:42:26.841041+0000","last_clean":"2024-09-16T15:42:26.841041+0000","last_became_active":"2024-09-16T15:42:25.022319+0000","last_became_peered":"2024-09-16T15:42:25.022319+0000","last_unstale":"2024-09-16T15:42:26.841041+0000","last_undegraded":"2024-09-16T15:42:26.841041+0000","last_fullsized":"2024-09-16T15:42:26.841041+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:05:52.459658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:26.841296+0000","last_change":"2024-09-16T15:42:25.001358+0000","last_active":"2024-09-16T15:42:26.841296+0000","last_peered":"2024-09-16T15:42:26.841296+0000","last_clean":"2024-09-16T15:42:26.841296+0000","last_became_active":"2024-09-16T15:42:25.001076+0000","last_became_peered":"2024-09-16T15:42:25.001076+0000","last_unstale":"2024-09-16T15:42:26.841296+0000","last_undegraded":"2024-09-16T15:42:26.841296+0000","last_fullsized":"2024-09-16T15:42:26.841296+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:02:59.519286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:26.840405+0000","last_change":"2024-09-16T15:42:25.012888+0000","last_active":"2024-09-16T15:42:26.840405+0000","last_peered":"2024-09-16T15:42:26.840405+0000","last_clean":"2024-09-16T15:42:26.840405+0000","last_became_active":"2024-09-16T15:42:25.009003+0000","last_became_peered":"2024-09-16T15:42:25.009003+0000","last_unstale":"2024-09-16T15:42:26.840405+0000","last_undegraded":"2024-09-16T15:42:26.840405+0000","last_fullsized":"2024-09-16T15:42:26.840405+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:48:23.305236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:26.032058+0000","last_change":"2024-09-16T15:42:25.017302+0000","last_active":"2024-09-16T15:42:26.032058+0000","last_peered":"2024-09-16T15:42:26.032058+0000","last_clean":"2024-09-16T15:42:26.032058+0000","last_became_active":"2024-09-16T15:42:25.017045+0000","last_became_peered":"2024-09-16T15:42:25.017045+0000","last_unstale":"2024-09-16T15:42:26.032058+0000","last_undegraded":"2024-09-16T15:42:26.032058+0000","last_fullsized":"2024-09-16T15:42:26.032058+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:59:40.200685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:26.840899+0000","last_change":"2024-09-16T15:42:25.021592+0000","last_active":"2024-09-16T15:42:26.840899+0000","last_peered":"2024-09-16T15:42:26.840899+0000","last_clean":"2024-09-16T15:42:26.840899+0000","last_became_active":"2024-09-16T15:42:25.021274+0000","last_became_peered":"2024-09-16T15:42:25.021274+0000","last_unstale":"2024-09-16T15:42:26.840899+0000","last_undegraded":"2024-09-16T15:42:26.840899+0000","last_fullsized":"2024-09-16T15:42:26.840899+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:36:59.764293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:26.840595+0000","last_change":"2024-09-16T15:42:24.995956+0000","last_active":"2024-09-16T15:42:26.840595+0000","last_peered":"2024-09-16T15:42:26.840595+0000","last_clean":"2024-09-16T15:42:26.840595+0000","last_became_active":"2024-09-16T15:42:24.995598+0000","last_became_peered":"2024-09-16T15:42:24.995598+0000","last_unstale":"2024-09-16T15:42:26.840595+0000","last_undegraded":"2024-09-16T15:42:26.840595+0000","last_fullsized":"2024-09-16T15:42:26.840595+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:39:14.978498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:26.032328+0000","last_change":"2024-09-16T15:42:25.024290+0000","last_active":"2024-09-16T15:42:26.032328+0000","last_peered":"2024-09-16T15:42:26.032328+0000","last_clean":"2024-09-16T15:42:26.032328+0000","last_became_active":"2024-09-16T15:42:25.024029+0000","last_became_peered":"2024-09-16T15:42:25.024029+0000","last_unstale":"2024-09-16T15:42:26.032328+0000","last_undegraded":"2024-09-16T15:42:26.032328+0000","last_fullsized":"2024-09-16T15:42:26.032328+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:53:56.337177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.990029+0000","last_change":"2024-09-16T15:42:25.009320+0000","last_active":"2024-09-16T15:42:25.990029+0000","last_peered":"2024-09-16T15:42:25.990029+0000","last_clean":"2024-09-16T15:42:25.990029+0000","last_became_active":"2024-09-16T15:42:25.008705+0000","last_became_peered":"2024-09-16T15:42:25.008705+0000","last_unstale":"2024-09-16T15:42:25.990029+0000","last_undegraded":"2024-09-16T15:42:25.990029+0000","last_fullsized":"2024-09-16T15:42:25.990029+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:47:42.974655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:26.841709+0000","last_change":"2024-09-16T15:42:25.014765+0000","last_active":"2024-09-16T15:42:26.841709+0000","last_peered":"2024-09-16T15:42:26.841709+0000","last_clean":"2024-09-16T15:42:26.841709+0000","last_became_active":"2024-09-16T15:42:25.009165+0000","last_became_peered":"2024-09-16T15:42:25.009165+0000","last_unstale":"2024-09-16T15:42:26.841709+0000","last_undegraded":"2024-09-16T15:42:26.841709+0000","last_fullsized":"2024-09-16T15:42:26.841709+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:18:32.636255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.990646+0000","last_change":"2024-09-16T15:42:25.018740+0000","last_active":"2024-09-16T15:42:25.990646+0000","last_peered":"2024-09-16T15:42:25.990646+0000","last_clean":"2024-09-16T15:42:25.990646+0000","last_became_active":"2024-09-16T15:42:25.016648+0000","last_became_peered":"2024-09-16T15:42:25.016648+0000","last_unstale":"2024-09-16T15:42:25.990646+0000","last_undegraded":"2024-09-16T15:42:25.990646+0000","last_fullsized":"2024-09-16T15:42:25.990646+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:26:41.139964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.990805+0000","last_change":"2024-09-16T15:42:25.024263+0000","last_active":"2024-09-16T15:42:25.990805+0000","last_peered":"2024-09-16T15:42:25.990805+0000","last_clean":"2024-09-16T15:42:25.990805+0000","last_became_active":"2024-09-16T15:42:25.024006+0000","last_became_peered":"2024-09-16T15:42:25.024006+0000","last_unstale":"2024-09-16T15:42:25.990805+0000","last_undegraded":"2024-09-16T15:42:25.990805+0000","last_fullsized":"2024-09-16T15:42:25.990805+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:28:30.999846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:26.840338+0000","last_change":"2024-09-16T15:42:25.018894+0000","last_active":"2024-09-16T15:42:26.840338+0000","last_peered":"2024-09-16T15:42:26.840338+0000","last_clean":"2024-09-16T15:42:26.840338+0000","last_became_active":"2024-09-16T15:42:25.018023+0000","last_became_peered":"2024-09-16T15:42:25.018023+0000","last_unstale":"2024-09-16T15:42:26.840338+0000","last_undegraded":"2024-09-16T15:42:26.840338+0000","last_fullsized":"2024-09-16T15:42:26.840338+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:20:03.403522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:26.032512+0000","last_change":"2024-09-16T15:42:25.019057+0000","last_active":"2024-09-16T15:42:26.032512+0000","last_peered":"2024-09-16T15:42:26.032512+0000","last_clean":"2024-09-16T15:42:26.032512+0000","last_became_active":"2024-09-16T15:42:25.018803+0000","last_became_peered":"2024-09-16T15:42:25.018803+0000","last_unstale":"2024-09-16T15:42:26.032512+0000","last_undegraded":"2024-09-16T15:42:26.032512+0000","last_fullsized":"2024-09-16T15:42:26.032512+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:36:59.764293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:26.841405+0000","last_change":"2024-09-16T15:42:25.022339+0000","last_active":"2024-09-16T15:42:26.841405+0000","last_peered":"2024-09-16T15:42:26.841405+0000","last_clean":"2024-09-16T15:42:26.841405+0000","last_became_active":"2024-09-16T15:42:25.021986+0000","last_became_peered":"2024-09-16T15:42:25.021986+0000","last_unstale":"2024-09-16T15:42:26.841405+0000","last_undegraded":"2024-09-16T15:42:26.841405+0000","last_fullsized":"2024-09-16T15:42:26.841405+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:36:02.510367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.990932+0000","last_change":"2024-09-16T15:42:25.016721+0000","last_active":"2024-09-16T15:42:25.990932+0000","last_peered":"2024-09-16T15:42:25.990932+0000","last_clean":"2024-09-16T15:42:25.990932+0000","last_became_active":"2024-09-16T15:42:25.016383+0000","last_became_peered":"2024-09-16T15:42:25.016383+0000","last_unstale":"2024-09-16T15:42:25.990932+0000","last_undegraded":"2024-09-16T15:42:25.990932+0000","last_fullsized":"2024-09-16T15:42:25.990932+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:02:55.978122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.990904+0000","last_change":"2024-09-16T15:42:25.011808+0000","last_active":"2024-09-16T15:42:25.990904+0000","last_peered":"2024-09-16T15:42:25.990904+0000","last_clean":"2024-09-16T15:42:25.990904+0000","last_became_active":"2024-09-16T15:42:25.011410+0000","last_became_peered":"2024-09-16T15:42:25.011410+0000","last_unstale":"2024-09-16T15:42:25.990904+0000","last_undegraded":"2024-09-16T15:42:25.990904+0000","last_fullsized":"2024-09-16T15:42:25.990904+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:11:26.269706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:26.032238+0000","last_change":"2024-09-16T15:42:25.020917+0000","last_active":"2024-09-16T15:42:26.032238+0000","last_peered":"2024-09-16T15:42:26.032238+0000","last_clean":"2024-09-16T15:42:26.032238+0000","last_became_active":"2024-09-16T15:42:25.016487+0000","last_became_peered":"2024-09-16T15:42:25.016487+0000","last_unstale":"2024-09-16T15:42:26.032238+0000","last_undegraded":"2024-09-16T15:42:26.032238+0000","last_fullsized":"2024-09-16T15:42:26.032238+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:48:23.305236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:26.840294+0000","last_change":"2024-09-16T15:42:25.008914+0000","last_active":"2024-09-16T15:42:26.840294+0000","last_peered":"2024-09-16T15:42:26.840294+0000","last_clean":"2024-09-16T15:42:26.840294+0000","last_became_active":"2024-09-16T15:42:25.008631+0000","last_became_peered":"2024-09-16T15:42:25.008631+0000","last_unstale":"2024-09-16T15:42:26.840294+0000","last_undegraded":"2024-09-16T15:42:26.840294+0000","last_fullsized":"2024-09-16T15:42:26.840294+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:25:33.890418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:26.840923+0000","last_change":"2024-09-16T15:42:25.024127+0000","last_active":"2024-09-16T15:42:26.840923+0000","last_peered":"2024-09-16T15:42:26.840923+0000","last_clean":"2024-09-16T15:42:26.840923+0000","last_became_active":"2024-09-16T15:42:25.023603+0000","last_became_peered":"2024-09-16T15:42:25.023603+0000","last_unstale":"2024-09-16T15:42:26.840923+0000","last_undegraded":"2024-09-16T15:42:26.840923+0000","last_fullsized":"2024-09-16T15:42:26.840923+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:49:11.296322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:26.840635+0000","last_change":"2024-09-16T15:42:25.006974+0000","last_active":"2024-09-16T15:42:26.840635+0000","last_peered":"2024-09-16T15:42:26.840635+0000","last_clean":"2024-09-16T15:42:26.840635+0000","last_became_active":"2024-09-16T15:42:25.006382+0000","last_became_peered":"2024-09-16T15:42:25.006382+0000","last_unstale":"2024-09-16T15:42:26.840635+0000","last_undegraded":"2024-09-16T15:42:26.840635+0000","last_fullsized":"2024-09-16T15:42:26.840635+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:36:59.764293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.990926+0000","last_change":"2024-09-16T15:42:25.013006+0000","last_active":"2024-09-16T15:42:25.990926+0000","last_peered":"2024-09-16T15:42:25.990926+0000","last_clean":"2024-09-16T15:42:25.990926+0000","last_became_active":"2024-09-16T15:42:25.012719+0000","last_became_peered":"2024-09-16T15:42:25.012719+0000","last_unstale":"2024-09-16T15:42:25.990926+0000","last_undegraded":"2024-09-16T15:42:25.990926+0000","last_fullsized":"2024-09-16T15:42:25.990926+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:09:36.460897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"47'4","reported_seq":19,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:26.841514+0000","last_change":"2024-09-16T15:42:24.001081+0000","last_active":"2024-09-16T15:42:26.841514+0000","last_peered":"2024-09-16T15:42:26.841514+0000","last_clean":"2024-09-16T15:42:26.841514+0000","last_became_active":"2024-09-16T15:42:24.000071+0000","last_became_peered":"2024-09-16T15:42:24.000071+0000","last_unstale":"2024-09-16T15:42:26.841514+0000","last_undegraded":"2024-09-16T15:42:26.841514+0000","last_fullsized":"2024-09-16T15:42:26.841514+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:24:51.105941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:26.032620+0000","last_change":"2024-09-16T15:42:25.016739+0000","last_active":"2024-09-16T15:42:26.032620+0000","last_peered":"2024-09-16T15:42:26.032620+0000","last_clean":"2024-09-16T15:42:26.032620+0000","last_became_active":"2024-09-16T15:42:25.016451+0000","last_became_peered":"2024-09-16T15:42:25.016451+0000","last_unstale":"2024-09-16T15:42:26.032620+0000","last_undegraded":"2024-09-16T15:42:26.032620+0000","last_fullsized":"2024-09-16T15:42:26.032620+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:53:44.198668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"47'1","reported_seq":16,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.991279+0000","last_change":"2024-09-16T15:42:24.000482+0000","last_active":"2024-09-16T15:42:25.991279+0000","last_peered":"2024-09-16T15:42:25.991279+0000","last_clean":"2024-09-16T15:42:25.991279+0000","last_became_active":"2024-09-16T15:42:24.000216+0000","last_became_peered":"2024-09-16T15:42:24.000216+0000","last_unstale":"2024-09-16T15:42:25.991279+0000","last_undegraded":"2024-09-16T15:42:25.991279+0000","last_fullsized":"2024-09-16T15:42:25.991279+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:41:02.855031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:26.840625+0000","last_change":"2024-09-16T15:42:25.005828+0000","last_active":"2024-09-16T15:42:26.840625+0000","last_peered":"2024-09-16T15:42:26.840625+0000","last_clean":"2024-09-16T15:42:26.840625+0000","last_became_active":"2024-09-16T15:42:25.005398+0000","last_became_peered":"2024-09-16T15:42:25.005398+0000","last_unstale":"2024-09-16T15:42:26.840625+0000","last_undegraded":"2024-09-16T15:42:26.840625+0000","last_fullsized":"2024-09-16T15:42:26.840625+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:48:23.305236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"47'2","reported_seq":17,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:26.840539+0000","last_change":"2024-09-16T15:42:24.002867+0000","last_active":"2024-09-16T15:42:26.840539+0000","last_peered":"2024-09-16T15:42:26.840539+0000","last_clean":"2024-09-16T15:42:26.840539+0000","last_became_active":"2024-09-16T15:42:24.002523+0000","last_became_peered":"2024-09-16T15:42:24.002523+0000","last_unstale":"2024-09-16T15:42:26.840539+0000","last_undegraded":"2024-09-16T15:42:26.840539+0000","last_fullsized":"2024-09-16T15:42:26.840539+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:57:21.332836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:26.841657+0000","last_change":"2024-09-16T15:42:25.006017+0000","last_active":"2024-09-16T15:42:26.841657+0000","last_peered":"2024-09-16T15:42:26.841657+0000","last_clean":"2024-09-16T15:42:26.841657+0000","last_became_active":"2024-09-16T15:42:25.005752+0000","last_became_peered":"2024-09-16T15:42:25.005752+0000","last_unstale":"2024-09-16T15:42:26.841657+0000","last_undegraded":"2024-09-16T15:42:26.841657+0000","last_fullsized":"2024-09-16T15:42:26.841657+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:41:03.857499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"47'2","reported_seq":17,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.991047+0000","last_change":"2024-09-16T15:42:24.002580+0000","last_active":"2024-09-16T15:42:25.991047+0000","last_peered":"2024-09-16T15:42:25.991047+0000","last_clean":"2024-09-16T15:42:25.991047+0000","last_became_active":"2024-09-16T15:42:24.000516+0000","last_became_peered":"2024-09-16T15:42:24.000516+0000","last_unstale":"2024-09-16T15:42:25.991047+0000","last_undegraded":"2024-09-16T15:42:25.991047+0000","last_fullsized":"2024-09-16T15:42:25.991047+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:34:18.490825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.990648+0000","last_change":"2024-09-16T15:42:25.014465+0000","last_active":"2024-09-16T15:42:25.990648+0000","last_peered":"2024-09-16T15:42:25.990648+0000","last_clean":"2024-09-16T15:42:25.990648+0000","last_became_active":"2024-09-16T15:42:25.013555+0000","last_became_peered":"2024-09-16T15:42:25.013555+0000","last_unstale":"2024-09-16T15:42:25.990648+0000","last_undegraded":"2024-09-16T15:42:25.990648+0000","last_fullsized":"2024-09-16T15:42:25.990648+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:01:26.813610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"47'1","reported_seq":16,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:26.841201+0000","last_change":"2024-09-16T15:42:23.997791+0000","last_active":"2024-09-16T15:42:26.841201+0000","last_peered":"2024-09-16T15:42:26.841201+0000","last_clean":"2024-09-16T15:42:26.841201+0000","last_became_active":"2024-09-16T15:42:23.997514+0000","last_became_peered":"2024-09-16T15:42:23.997514+0000","last_unstale":"2024-09-16T15:42:26.841201+0000","last_undegraded":"2024-09-16T15:42:26.841201+0000","last_fullsized":"2024-09-16T15:42:26.841201+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:59:17.932649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.989951+0000","last_change":"2024-09-16T15:42:24.996753+0000","last_active":"2024-09-16T15:42:25.989951+0000","last_peered":"2024-09-16T15:42:25.989951+0000","last_clean":"2024-09-16T15:42:25.989951+0000","last_became_active":"2024-09-16T15:42:24.996402+0000","last_became_peered":"2024-09-16T15:42:24.996402+0000","last_unstale":"2024-09-16T15:42:25.989951+0000","last_undegraded":"2024-09-16T15:42:25.989951+0000","last_fullsized":"2024-09-16T15:42:25.989951+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:29:54.963053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"47'1","reported_seq":16,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.990624+0000","last_change":"2024-09-16T15:42:23.999519+0000","last_active":"2024-09-16T15:42:25.990624+0000","last_peered":"2024-09-16T15:42:25.990624+0000","last_clean":"2024-09-16T15:42:25.990624+0000","last_became_active":"2024-09-16T15:42:23.999239+0000","last_became_peered":"2024-09-16T15:42:23.999239+0000","last_unstale":"2024-09-16T15:42:25.990624+0000","last_undegraded":"2024-09-16T15:42:25.990624+0000","last_fullsized":"2024-09-16T15:42:25.990624+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:16:13.768386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.990009+0000","last_change":"2024-09-16T15:42:24.999204+0000","last_active":"2024-09-16T15:42:25.990009+0000","last_peered":"2024-09-16T15:42:25.990009+0000","last_clean":"2024-09-16T15:42:25.990009+0000","last_became_active":"2024-09-16T15:42:24.998947+0000","last_became_peered":"2024-09-16T15:42:24.998947+0000","last_unstale":"2024-09-16T15:42:25.990009+0000","last_undegraded":"2024-09-16T15:42:25.990009+0000","last_fullsized":"2024-09-16T15:42:25.990009+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:08:00.910495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:26.032103+0000","last_change":"2024-09-16T15:42:24.003842+0000","last_active":"2024-09-16T15:42:26.032103+0000","last_peered":"2024-09-16T15:42:26.032103+0000","last_clean":"2024-09-16T15:42:26.032103+0000","last_became_active":"2024-09-16T15:42:24.003517+0000","last_became_peered":"2024-09-16T15:42:24.003517+0000","last_unstale":"2024-09-16T15:42:26.032103+0000","last_undegraded":"2024-09-16T15:42:26.032103+0000","last_fullsized":"2024-09-16T15:42:26.032103+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:40:42.405299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.990602+0000","last_change":"2024-09-16T15:42:25.010354+0000","last_active":"2024-09-16T15:42:25.990602+0000","last_peered":"2024-09-16T15:42:25.990602+0000","last_clean":"2024-09-16T15:42:25.990602+0000","last_became_active":"2024-09-16T15:42:25.010088+0000","last_became_peered":"2024-09-16T15:42:25.010088+0000","last_unstale":"2024-09-16T15:42:25.990602+0000","last_undegraded":"2024-09-16T15:42:25.990602+0000","last_fullsized":"2024-09-16T15:42:25.990602+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:36:52.080550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.990194+0000","last_change":"2024-09-16T15:42:24.006237+0000","last_active":"2024-09-16T15:42:25.990194+0000","last_peered":"2024-09-16T15:42:25.990194+0000","last_clean":"2024-09-16T15:42:25.990194+0000","last_became_active":"2024-09-16T15:42:24.005943+0000","last_became_peered":"2024-09-16T15:42:24.005943+0000","last_unstale":"2024-09-16T15:42:25.990194+0000","last_undegraded":"2024-09-16T15:42:25.990194+0000","last_fullsized":"2024-09-16T15:42:25.990194+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:49:26.725206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:26.840866+0000","last_change":"2024-09-16T15:42:25.005518+0000","last_active":"2024-09-16T15:42:26.840866+0000","last_peered":"2024-09-16T15:42:26.840866+0000","last_clean":"2024-09-16T15:42:26.840866+0000","last_became_active":"2024-09-16T15:42:25.005266+0000","last_became_peered":"2024-09-16T15:42:25.005266+0000","last_unstale":"2024-09-16T15:42:26.840866+0000","last_undegraded":"2024-09-16T15:42:26.840866+0000","last_fullsized":"2024-09-16T15:42:26.840866+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:29:58.862084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:26.840774+0000","last_change":"2024-09-16T15:42:23.997205+0000","last_active":"2024-09-16T15:42:26.840774+0000","last_peered":"2024-09-16T15:42:26.840774+0000","last_clean":"2024-09-16T15:42:26.840774+0000","last_became_active":"2024-09-16T15:42:23.996928+0000","last_became_peered":"2024-09-16T15:42:23.996928+0000","last_unstale":"2024-09-16T15:42:26.840774+0000","last_undegraded":"2024-09-16T15:42:26.840774+0000","last_fullsized":"2024-09-16T15:42:26.840774+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:17:58.230738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:26.840607+0000","last_change":"2024-09-16T15:42:25.023166+0000","last_active":"2024-09-16T15:42:26.840607+0000","last_peered":"2024-09-16T15:42:26.840607+0000","last_clean":"2024-09-16T15:42:26.840607+0000","last_became_active":"2024-09-16T15:42:25.022883+0000","last_became_peered":"2024-09-16T15:42:25.022883+0000","last_unstale":"2024-09-16T15:42:26.840607+0000","last_undegraded":"2024-09-16T15:42:26.840607+0000","last_fullsized":"2024-09-16T15:42:26.840607+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:41:39.991712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.991251+0000","last_change":"2024-09-16T15:42:24.007502+0000","last_active":"2024-09-16T15:42:25.991251+0000","last_peered":"2024-09-16T15:42:25.991251+0000","last_clean":"2024-09-16T15:42:25.991251+0000","last_became_active":"2024-09-16T15:42:24.007051+0000","last_became_peered":"2024-09-16T15:42:24.007051+0000","last_unstale":"2024-09-16T15:42:25.991251+0000","last_undegraded":"2024-09-16T15:42:25.991251+0000","last_fullsized":"2024-09-16T15:42:25.991251+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:42:33.038498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:26.841945+0000","last_change":"2024-09-16T15:42:24.998718+0000","last_active":"2024-09-16T15:42:26.841945+0000","last_peered":"2024-09-16T15:42:26.841945+0000","last_clean":"2024-09-16T15:42:26.841945+0000","last_became_active":"2024-09-16T15:42:24.998449+0000","last_became_peered":"2024-09-16T15:42:24.998449+0000","last_unstale":"2024-09-16T15:42:26.841945+0000","last_undegraded":"2024-09-16T15:42:26.841945+0000","last_fullsized":"2024-09-16T15:42:26.841945+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:28:57.800376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"47'3","reported_seq":18,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:26.840647+0000","last_change":"2024-09-16T15:42:24.000199+0000","last_active":"2024-09-16T15:42:26.840647+0000","last_peered":"2024-09-16T15:42:26.840647+0000","last_clean":"2024-09-16T15:42:26.840647+0000","last_became_active":"2024-09-16T15:42:23.999884+0000","last_became_peered":"2024-09-16T15:42:23.999884+0000","last_unstale":"2024-09-16T15:42:26.840647+0000","last_undegraded":"2024-09-16T15:42:26.840647+0000","last_fullsized":"2024-09-16T15:42:26.840647+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:19:10.773082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.991046+0000","last_change":"2024-09-16T15:42:25.018590+0000","last_active":"2024-09-16T15:42:25.991046+0000","last_peered":"2024-09-16T15:42:25.991046+0000","last_clean":"2024-09-16T15:42:25.991046+0000","last_became_active":"2024-09-16T15:42:25.018027+0000","last_became_peered":"2024-09-16T15:42:25.018027+0000","last_unstale":"2024-09-16T15:42:25.991046+0000","last_undegraded":"2024-09-16T15:42:25.991046+0000","last_fullsized":"2024-09-16T15:42:25.991046+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:59:43.116689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"47'6","reported_seq":21,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.990677+0000","last_change":"2024-09-16T15:42:24.005558+0000","last_active":"2024-09-16T15:42:25.990677+0000","last_peered":"2024-09-16T15:42:25.990677+0000","last_clean":"2024-09-16T15:42:25.990677+0000","last_became_active":"2024-09-16T15:42:24.004712+0000","last_became_peered":"2024-09-16T15:42:24.004712+0000","last_unstale":"2024-09-16T15:42:25.990677+0000","last_undegraded":"2024-09-16T15:42:25.990677+0000","last_fullsized":"2024-09-16T15:42:25.990677+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:44:40.838439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:26.840634+0000","last_change":"2024-09-16T15:42:24.997997+0000","last_active":"2024-09-16T15:42:26.840634+0000","last_peered":"2024-09-16T15:42:26.840634+0000","last_clean":"2024-09-16T15:42:26.840634+0000","last_became_active":"2024-09-16T15:42:24.997734+0000","last_became_peered":"2024-09-16T15:42:24.997734+0000","last_unstale":"2024-09-16T15:42:26.840634+0000","last_undegraded":"2024-09-16T15:42:26.840634+0000","last_fullsized":"2024-09-16T15:42:26.840634+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:46:31.473959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"47'2","reported_seq":17,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.990625+0000","last_change":"2024-09-16T15:42:24.001353+0000","last_active":"2024-09-16T15:42:25.990625+0000","last_peered":"2024-09-16T15:42:25.990625+0000","last_clean":"2024-09-16T15:42:25.990625+0000","last_became_active":"2024-09-16T15:42:24.000702+0000","last_became_peered":"2024-09-16T15:42:24.000702+0000","last_unstale":"2024-09-16T15:42:25.990625+0000","last_undegraded":"2024-09-16T15:42:25.990625+0000","last_fullsized":"2024-09-16T15:42:25.990625+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:56:20.839846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.990611+0000","last_change":"2024-09-16T15:42:25.015629+0000","last_active":"2024-09-16T15:42:25.990611+0000","last_peered":"2024-09-16T15:42:25.990611+0000","last_clean":"2024-09-16T15:42:25.990611+0000","last_became_active":"2024-09-16T15:42:25.015379+0000","last_became_peered":"2024-09-16T15:42:25.015379+0000","last_unstale":"2024-09-16T15:42:25.990611+0000","last_undegraded":"2024-09-16T15:42:25.990611+0000","last_fullsized":"2024-09-16T15:42:25.990611+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:40:18.380704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:26.032039+0000","last_change":"2024-09-16T15:42:24.004317+0000","last_active":"2024-09-16T15:42:26.032039+0000","last_peered":"2024-09-16T15:42:26.032039+0000","last_clean":"2024-09-16T15:42:26.032039+0000","last_became_active":"2024-09-16T15:42:24.003249+0000","last_became_peered":"2024-09-16T15:42:24.003249+0000","last_unstale":"2024-09-16T15:42:26.032039+0000","last_undegraded":"2024-09-16T15:42:26.032039+0000","last_fullsized":"2024-09-16T15:42:26.032039+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:47:43.918079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.989585+0000","last_change":"2024-09-16T15:42:25.011604+0000","last_active":"2024-09-16T15:42:25.989585+0000","last_peered":"2024-09-16T15:42:25.989585+0000","last_clean":"2024-09-16T15:42:25.989585+0000","last_became_active":"2024-09-16T15:42:25.011293+0000","last_became_peered":"2024-09-16T15:42:25.011293+0000","last_unstale":"2024-09-16T15:42:25.989585+0000","last_undegraded":"2024-09-16T15:42:25.989585+0000","last_fullsized":"2024-09-16T15:42:25.989585+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:16:35.693581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:26.840439+0000","last_change":"2024-09-16T15:42:23.999141+0000","last_active":"2024-09-16T15:42:26.840439+0000","last_peered":"2024-09-16T15:42:26.840439+0000","last_clean":"2024-09-16T15:42:26.840439+0000","last_became_active":"2024-09-16T15:42:23.998768+0000","last_became_peered":"2024-09-16T15:42:23.998768+0000","last_unstale":"2024-09-16T15:42:26.840439+0000","last_undegraded":"2024-09-16T15:42:26.840439+0000","last_fullsized":"2024-09-16T15:42:26.840439+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:35:18.477892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:26.840753+0000","last_change":"2024-09-16T15:42:25.022864+0000","last_active":"2024-09-16T15:42:26.840753+0000","last_peered":"2024-09-16T15:42:26.840753+0000","last_clean":"2024-09-16T15:42:26.840753+0000","last_became_active":"2024-09-16T15:42:25.022447+0000","last_became_peered":"2024-09-16T15:42:25.022447+0000","last_unstale":"2024-09-16T15:42:26.840753+0000","last_undegraded":"2024-09-16T15:42:26.840753+0000","last_fullsized":"2024-09-16T15:42:26.840753+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:53:56.337177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"47'8","reported_seq":23,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.990304+0000","last_change":"2024-09-16T15:42:24.004256+0000","last_active":"2024-09-16T15:42:25.990304+0000","last_peered":"2024-09-16T15:42:25.990304+0000","last_clean":"2024-09-16T15:42:25.990304+0000","last_became_active":"2024-09-16T15:42:24.002811+0000","last_became_peered":"2024-09-16T15:42:24.002811+0000","last_unstale":"2024-09-16T15:42:25.990304+0000","last_undegraded":"2024-09-16T15:42:25.990304+0000","last_fullsized":"2024-09-16T15:42:25.990304+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:29:44.054327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.990997+0000","last_change":"2024-09-16T15:42:25.017096+0000","last_active":"2024-09-16T15:42:25.990997+0000","last_peered":"2024-09-16T15:42:25.990997+0000","last_clean":"2024-09-16T15:42:25.990997+0000","last_became_active":"2024-09-16T15:42:25.016221+0000","last_became_peered":"2024-09-16T15:42:25.016221+0000","last_unstale":"2024-09-16T15:42:25.990997+0000","last_undegraded":"2024-09-16T15:42:25.990997+0000","last_fullsized":"2024-09-16T15:42:25.990997+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:33:58.512780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"47'1","reported_seq":16,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:26.841880+0000","last_change":"2024-09-16T15:42:24.003907+0000","last_active":"2024-09-16T15:42:26.841880+0000","last_peered":"2024-09-16T15:42:26.841880+0000","last_clean":"2024-09-16T15:42:26.841880+0000","last_became_active":"2024-09-16T15:42:24.003647+0000","last_became_peered":"2024-09-16T15:42:24.003647+0000","last_unstale":"2024-09-16T15:42:26.841880+0000","last_undegraded":"2024-09-16T15:42:26.841880+0000","last_fullsized":"2024-09-16T15:42:26.841880+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:18:29.135782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.990985+0000","last_change":"2024-09-16T15:42:25.011725+0000","last_active":"2024-09-16T15:42:25.990985+0000","last_peered":"2024-09-16T15:42:25.990985+0000","last_clean":"2024-09-16T15:42:25.990985+0000","last_became_active":"2024-09-16T15:42:25.010960+0000","last_became_peered":"2024-09-16T15:42:25.010960+0000","last_unstale":"2024-09-16T15:42:25.990985+0000","last_undegraded":"2024-09-16T15:42:25.990985+0000","last_fullsized":"2024-09-16T15:42:25.990985+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:41:07.081665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"47'2","reported_seq":17,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:26.840902+0000","last_change":"2024-09-16T15:42:23.999546+0000","last_active":"2024-09-16T15:42:26.840902+0000","last_peered":"2024-09-16T15:42:26.840902+0000","last_clean":"2024-09-16T15:42:26.840902+0000","last_became_active":"2024-09-16T15:42:23.999283+0000","last_became_peered":"2024-09-16T15:42:23.999283+0000","last_unstale":"2024-09-16T15:42:26.840902+0000","last_undegraded":"2024-09-16T15:42:26.840902+0000","last_fullsized":"2024-09-16T15:42:26.840902+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:48:31.728751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:26.840918+0000","last_change":"2024-09-16T15:42:25.017755+0000","last_active":"2024-09-16T15:42:26.840918+0000","last_peered":"2024-09-16T15:42:26.840918+0000","last_clean":"2024-09-16T15:42:26.840918+0000","last_became_active":"2024-09-16T15:42:25.017488+0000","last_became_peered":"2024-09-16T15:42:25.017488+0000","last_unstale":"2024-09-16T15:42:26.840918+0000","last_undegraded":"2024-09-16T15:42:26.840918+0000","last_fullsized":"2024-09-16T15:42:26.840918+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:06:06.403757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"47'1","reported_seq":16,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:26.841105+0000","last_change":"2024-09-16T15:42:24.006459+0000","last_active":"2024-09-16T15:42:26.841105+0000","last_peered":"2024-09-16T15:42:26.841105+0000","last_clean":"2024-09-16T15:42:26.841105+0000","last_became_active":"2024-09-16T15:42:24.006025+0000","last_became_peered":"2024-09-16T15:42:24.006025+0000","last_unstale":"2024-09-16T15:42:26.841105+0000","last_undegraded":"2024-09-16T15:42:26.841105+0000","last_fullsized":"2024-09-16T15:42:26.841105+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:48:42.868847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:26.841014+0000","last_change":"2024-09-16T15:42:25.011131+0000","last_active":"2024-09-16T15:42:26.841014+0000","last_peered":"2024-09-16T15:42:26.841014+0000","last_clean":"2024-09-16T15:42:26.841014+0000","last_became_active":"2024-09-16T15:42:25.008480+0000","last_became_peered":"2024-09-16T15:42:25.008480+0000","last_unstale":"2024-09-16T15:42:26.841014+0000","last_undegraded":"2024-09-16T15:42:26.841014+0000","last_fullsized":"2024-09-16T15:42:26.841014+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:47:51.465430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:26.840636+0000","last_change":"2024-09-16T15:42:24.005143+0000","last_active":"2024-09-16T15:42:26.840636+0000","last_peered":"2024-09-16T15:42:26.840636+0000","last_clean":"2024-09-16T15:42:26.840636+0000","last_became_active":"2024-09-16T15:42:24.004889+0000","last_became_peered":"2024-09-16T15:42:24.004889+0000","last_unstale":"2024-09-16T15:42:26.840636+0000","last_undegraded":"2024-09-16T15:42:26.840636+0000","last_fullsized":"2024-09-16T15:42:26.840636+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:27:58.850861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.990504+0000","last_change":"2024-09-16T15:42:25.010574+0000","last_active":"2024-09-16T15:42:25.990504+0000","last_peered":"2024-09-16T15:42:25.990504+0000","last_clean":"2024-09-16T15:42:25.990504+0000","last_became_active":"2024-09-16T15:42:25.009246+0000","last_became_peered":"2024-09-16T15:42:25.009246+0000","last_unstale":"2024-09-16T15:42:25.990504+0000","last_undegraded":"2024-09-16T15:42:25.990504+0000","last_fullsized":"2024-09-16T15:42:25.990504+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:18:32.636255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"47'2","reported_seq":17,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.989991+0000","last_change":"2024-09-16T15:42:24.007687+0000","last_active":"2024-09-16T15:42:25.989991+0000","last_peered":"2024-09-16T15:42:25.989991+0000","last_clean":"2024-09-16T15:42:25.989991+0000","last_became_active":"2024-09-16T15:42:24.006760+0000","last_became_peered":"2024-09-16T15:42:24.006760+0000","last_unstale":"2024-09-16T15:42:25.989991+0000","last_undegraded":"2024-09-16T15:42:25.989991+0000","last_fullsized":"2024-09-16T15:42:25.989991+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:16:19.186435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:26.841029+0000","last_change":"2024-09-16T15:42:25.012518+0000","last_active":"2024-09-16T15:42:26.841029+0000","last_peered":"2024-09-16T15:42:26.841029+0000","last_clean":"2024-09-16T15:42:26.841029+0000","last_became_active":"2024-09-16T15:42:25.012013+0000","last_became_peered":"2024-09-16T15:42:25.012013+0000","last_unstale":"2024-09-16T15:42:26.841029+0000","last_undegraded":"2024-09-16T15:42:26.841029+0000","last_fullsized":"2024-09-16T15:42:26.841029+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:05.773773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:26.032071+0000","last_change":"2024-09-16T15:42:24.007715+0000","last_active":"2024-09-16T15:42:26.032071+0000","last_peered":"2024-09-16T15:42:26.032071+0000","last_clean":"2024-09-16T15:42:26.032071+0000","last_became_active":"2024-09-16T15:42:24.007435+0000","last_became_peered":"2024-09-16T15:42:24.007435+0000","last_unstale":"2024-09-16T15:42:26.032071+0000","last_undegraded":"2024-09-16T15:42:26.032071+0000","last_fullsized":"2024-09-16T15:42:26.032071+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:23:18.271535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:26.840663+0000","last_change":"2024-09-16T15:42:25.007606+0000","last_active":"2024-09-16T15:42:26.840663+0000","last_peered":"2024-09-16T15:42:26.840663+0000","last_clean":"2024-09-16T15:42:26.840663+0000","last_became_active":"2024-09-16T15:42:25.006710+0000","last_became_peered":"2024-09-16T15:42:25.006710+0000","last_unstale":"2024-09-16T15:42:26.840663+0000","last_undegraded":"2024-09-16T15:42:26.840663+0000","last_fullsized":"2024-09-16T15:42:26.840663+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:55:24.766427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.990421+0000","last_change":"2024-09-16T15:42:24.007575+0000","last_active":"2024-09-16T15:42:25.990421+0000","last_peered":"2024-09-16T15:42:25.990421+0000","last_clean":"2024-09-16T15:42:25.990421+0000","last_became_active":"2024-09-16T15:42:24.007186+0000","last_became_peered":"2024-09-16T15:42:24.007186+0000","last_unstale":"2024-09-16T15:42:25.990421+0000","last_undegraded":"2024-09-16T15:42:25.990421+0000","last_fullsized":"2024-09-16T15:42:25.990421+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:25:18.891435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:26.840996+0000","last_change":"2024-09-16T15:42:25.013201+0000","last_active":"2024-09-16T15:42:26.840996+0000","last_peered":"2024-09-16T15:42:26.840996+0000","last_clean":"2024-09-16T15:42:26.840996+0000","last_became_active":"2024-09-16T15:42:25.001679+0000","last_became_peered":"2024-09-16T15:42:25.001679+0000","last_unstale":"2024-09-16T15:42:26.840996+0000","last_undegraded":"2024-09-16T15:42:26.840996+0000","last_fullsized":"2024-09-16T15:42:26.840996+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:53:44.198668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.990513+0000","last_change":"2024-09-16T15:42:23.995573+0000","last_active":"2024-09-16T15:42:25.990513+0000","last_peered":"2024-09-16T15:42:25.990513+0000","last_clean":"2024-09-16T15:42:25.990513+0000","last_became_active":"2024-09-16T15:42:23.995328+0000","last_became_peered":"2024-09-16T15:42:23.995328+0000","last_unstale":"2024-09-16T15:42:25.990513+0000","last_undegraded":"2024-09-16T15:42:25.990513+0000","last_fullsized":"2024-09-16T15:42:25.990513+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:10:17.404307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.991115+0000","last_change":"2024-09-16T15:42:25.010788+0000","last_active":"2024-09-16T15:42:25.991115+0000","last_peered":"2024-09-16T15:42:25.991115+0000","last_clean":"2024-09-16T15:42:25.991115+0000","last_became_active":"2024-09-16T15:42:25.010513+0000","last_became_peered":"2024-09-16T15:42:25.010513+0000","last_unstale":"2024-09-16T15:42:25.991115+0000","last_undegraded":"2024-09-16T15:42:25.991115+0000","last_fullsized":"2024-09-16T15:42:25.991115+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:42:07.318108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"48'1","reported_seq":16,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:26.841951+0000","last_change":"2024-09-16T15:42:24.002775+0000","last_active":"2024-09-16T15:42:26.841951+0000","last_peered":"2024-09-16T15:42:26.841951+0000","last_clean":"2024-09-16T15:42:26.841951+0000","last_became_active":"2024-09-16T15:42:24.002505+0000","last_became_peered":"2024-09-16T15:42:24.002505+0000","last_unstale":"2024-09-16T15:42:26.841951+0000","last_undegraded":"2024-09-16T15:42:26.841951+0000","last_fullsized":"2024-09-16T15:42:26.841951+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:01:01.221459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.990305+0000","last_change":"2024-09-16T15:42:25.010363+0000","last_active":"2024-09-16T15:42:25.990305+0000","last_peered":"2024-09-16T15:42:25.990305+0000","last_clean":"2024-09-16T15:42:25.990305+0000","last_became_active":"2024-09-16T15:42:25.010070+0000","last_became_peered":"2024-09-16T15:42:25.010070+0000","last_unstale":"2024-09-16T15:42:25.990305+0000","last_undegraded":"2024-09-16T15:42:25.990305+0000","last_fullsized":"2024-09-16T15:42:25.990305+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:41:13.922979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:26.032291+0000","last_change":"2024-09-16T15:42:24.007633+0000","last_active":"2024-09-16T15:42:26.032291+0000","last_peered":"2024-09-16T15:42:26.032291+0000","last_clean":"2024-09-16T15:42:26.032291+0000","last_became_active":"2024-09-16T15:42:24.007382+0000","last_became_peered":"2024-09-16T15:42:24.007382+0000","last_unstale":"2024-09-16T15:42:26.032291+0000","last_undegraded":"2024-09-16T15:42:26.032291+0000","last_fullsized":"2024-09-16T15:42:26.032291+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:49:10.293854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.991126+0000","last_change":"2024-09-16T15:42:25.019597+0000","last_active":"2024-09-16T15:42:25.991126+0000","last_peered":"2024-09-16T15:42:25.991126+0000","last_clean":"2024-09-16T15:42:25.991126+0000","last_became_active":"2024-09-16T15:42:25.018344+0000","last_became_peered":"2024-09-16T15:42:25.018344+0000","last_unstale":"2024-09-16T15:42:25.991126+0000","last_undegraded":"2024-09-16T15:42:25.991126+0000","last_fullsized":"2024-09-16T15:42:25.991126+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:44:59.045454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.990811+0000","last_change":"2024-09-16T15:42:24.007325+0000","last_active":"2024-09-16T15:42:25.990811+0000","last_peered":"2024-09-16T15:42:25.990811+0000","last_clean":"2024-09-16T15:42:25.990811+0000","last_became_active":"2024-09-16T15:42:24.006971+0000","last_became_peered":"2024-09-16T15:42:24.006971+0000","last_unstale":"2024-09-16T15:42:25.990811+0000","last_undegraded":"2024-09-16T15:42:25.990811+0000","last_fullsized":"2024-09-16T15:42:25.990811+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:27:56.298771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.990080+0000","last_change":"2024-09-16T15:42:25.011747+0000","last_active":"2024-09-16T15:42:25.990080+0000","last_peered":"2024-09-16T15:42:25.990080+0000","last_clean":"2024-09-16T15:42:25.990080+0000","last_became_active":"2024-09-16T15:42:25.009391+0000","last_became_peered":"2024-09-16T15:42:25.009391+0000","last_unstale":"2024-09-16T15:42:25.990080+0000","last_undegraded":"2024-09-16T15:42:25.990080+0000","last_fullsized":"2024-09-16T15:42:25.990080+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:44:14.620192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:26.032401+0000","last_change":"2024-09-16T15:42:24.005893+0000","last_active":"2024-09-16T15:42:26.032401+0000","last_peered":"2024-09-16T15:42:26.032401+0000","last_clean":"2024-09-16T15:42:26.032401+0000","last_became_active":"2024-09-16T15:42:24.005708+0000","last_became_peered":"2024-09-16T15:42:24.005708+0000","last_unstale":"2024-09-16T15:42:26.032401+0000","last_undegraded":"2024-09-16T15:42:26.032401+0000","last_fullsized":"2024-09-16T15:42:26.032401+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:10:04.058162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:26.841397+0000","last_change":"2024-09-16T15:42:25.014435+0000","last_active":"2024-09-16T15:42:26.841397+0000","last_peered":"2024-09-16T15:42:26.841397+0000","last_clean":"2024-09-16T15:42:26.841397+0000","last_became_active":"2024-09-16T15:42:25.014126+0000","last_became_peered":"2024-09-16T15:42:25.014126+0000","last_unstale":"2024-09-16T15:42:26.841397+0000","last_undegraded":"2024-09-16T15:42:26.841397+0000","last_fullsized":"2024-09-16T15:42:26.841397+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:17:26.096354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.990735+0000","last_change":"2024-09-16T15:42:24.003312+0000","last_active":"2024-09-16T15:42:25.990735+0000","last_peered":"2024-09-16T15:42:25.990735+0000","last_clean":"2024-09-16T15:42:25.990735+0000","last_became_active":"2024-09-16T15:42:24.002853+0000","last_became_peered":"2024-09-16T15:42:24.002853+0000","last_unstale":"2024-09-16T15:42:25.990735+0000","last_undegraded":"2024-09-16T15:42:25.990735+0000","last_fullsized":"2024-09-16T15:42:25.990735+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:21:53.045720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:26.841076+0000","last_change":"2024-09-16T15:42:25.000979+0000","last_active":"2024-09-16T15:42:26.841076+0000","last_peered":"2024-09-16T15:42:26.841076+0000","last_clean":"2024-09-16T15:42:26.841076+0000","last_became_active":"2024-09-16T15:42:25.000590+0000","last_became_peered":"2024-09-16T15:42:25.000590+0000","last_unstale":"2024-09-16T15:42:26.841076+0000","last_undegraded":"2024-09-16T15:42:26.841076+0000","last_fullsized":"2024-09-16T15:42:26.841076+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:53:10.179404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:26.840567+0000","last_change":"2024-09-16T15:42:23.999897+0000","last_active":"2024-09-16T15:42:26.840567+0000","last_peered":"2024-09-16T15:42:26.840567+0000","last_clean":"2024-09-16T15:42:26.840567+0000","last_became_active":"2024-09-16T15:42:23.999535+0000","last_became_peered":"2024-09-16T15:42:23.999535+0000","last_unstale":"2024-09-16T15:42:26.840567+0000","last_undegraded":"2024-09-16T15:42:26.840567+0000","last_fullsized":"2024-09-16T15:42:26.840567+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:46:06.187678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:26.840440+0000","last_change":"2024-09-16T15:42:25.013782+0000","last_active":"2024-09-16T15:42:26.840440+0000","last_peered":"2024-09-16T15:42:26.840440+0000","last_clean":"2024-09-16T15:42:26.840440+0000","last_became_active":"2024-09-16T15:42:25.013470+0000","last_became_peered":"2024-09-16T15:42:25.013470+0000","last_unstale":"2024-09-16T15:42:26.840440+0000","last_undegraded":"2024-09-16T15:42:26.840440+0000","last_fullsized":"2024-09-16T15:42:26.840440+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:49:27.727674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":15,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.989613+0000","last_change":"2024-09-16T15:42:24.003909+0000","last_active":"2024-09-16T15:42:25.989613+0000","last_peered":"2024-09-16T15:42:25.989613+0000","last_clean":"2024-09-16T15:42:25.989613+0000","last_became_active":"2024-09-16T15:42:24.003617+0000","last_became_peered":"2024-09-16T15:42:24.003617+0000","last_unstale":"2024-09-16T15:42:25.989613+0000","last_undegraded":"2024-09-16T15:42:25.989613+0000","last_fullsized":"2024-09-16T15:42:25.989613+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:49:43.331112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:26.841648+0000","last_change":"2024-09-16T15:42:25.018915+0000","last_active":"2024-09-16T15:42:26.841648+0000","last_peered":"2024-09-16T15:42:26.841648+0000","last_clean":"2024-09-16T15:42:26.841648+0000","last_became_active":"2024-09-16T15:42:25.018630+0000","last_became_peered":"2024-09-16T15:42:25.018630+0000","last_unstale":"2024-09-16T15:42:26.841648+0000","last_undegraded":"2024-09-16T15:42:26.841648+0000","last_fullsized":"2024-09-16T15:42:26.841648+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:23.975506+0000","last_clean_scrub_stamp":"2024-09-16T15:42:23.975506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:41:13.922979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"47'2","reported_seq":17,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:25.991098+0000","last_change":"2024-09-16T15:42:24.002759+0000","last_active":"2024-09-16T15:42:25.991098+0000","last_peered":"2024-09-16T15:42:25.991098+0000","last_clean":"2024-09-16T15:42:25.991098+0000","last_became_active":"2024-09-16T15:42:24.002401+0000","last_became_peered":"2024-09-16T15:42:24.002401+0000","last_unstale":"2024-09-16T15:42:25.991098+0000","last_undegraded":"2024-09-16T15:42:25.991098+0000","last_fullsized":"2024-09-16T15:42:25.991098+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:42:22.973038+0000","last_clean_scrub_stamp":"2024-09-16T15:42:22.973038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:35:47.858965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":160,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T15:42:26.840251+0000","last_change":"2024-09-16T15:39:12.698813+0000","last_active":"2024-09-16T15:42:26.840251+0000","last_peered":"2024-09-16T15:42:26.840251+0000","last_clean":"2024-09-16T15:42:26.840251+0000","last_became_active":"2024-09-16T15:39:12.697695+0000","last_became_peered":"2024-09-16T15:39:12.697695+0000","last_unstale":"2024-09-16T15:42:26.840251+0000","last_undegraded":"2024-09-16T15:42:26.840251+0000","last_fullsized":"2024-09-16T15:42:26.840251+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_clean_scrub_stamp":"2024-09-16T15:39:11.684967+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:02.061081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509485,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31744,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340096,"statfs":{"total":96636764160,"available":96604258304,"internally_reserved":0,"allocated":565248,"data_stored":227864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509484,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32320,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339520,"statfs":{"total":96636764160,"available":96603668480,"internally_reserved":0,"allocated":1155072,"data_stored":817638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509483,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35916,"kb_used_data":564,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335924,"statfs":{"total":96636764160,"available":96599986176,"internally_reserved":0,"allocated":577536,"data_stored":228952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509483,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31744,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340096,"statfs":{"total":96636764160,"available":96604258304,"internally_reserved":0,"allocated":569344,"data_stored":228860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31914441},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509483,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31740,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340100,"statfs":{"total":96636764160,"available":96604262400,"internally_reserved":0,"allocated":561152,"data_stored":227338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509484,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32316,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339524,"statfs":{"total":96636764160,"available":96603672576,"internally_reserved":0,"allocated":1155072,"data_stored":817686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509482,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31744,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340096,"statfs":{"total":96636764160,"available":96604258304,"internally_reserved":0,"allocated":565248,"data_stored":227920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542189,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31744,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340096,"statfs":{"total":96636764160,"available":96604258304,"internally_reserved":0,"allocated":561152,"data_stored":227282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":31914444},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:42:34.302 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T15:42:34.303 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-16T15:42:34.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.503+0000 7f8283e00640 1 -- 172.21.15.151:0/1471686208 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8268003a00 msgr2=0x7f8268003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:34.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.503+0000 7f8283e00640 1 --2- 172.21.15.151:0/1471686208 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8268003a00 0x7f8268003de0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f8274001740 tx=0x7f8268005830 comp rx=0 tx=0).stop 2024-09-16T15:42:34.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.503+0000 7f8283e00640 1 -- 172.21.15.151:0/1471686208 shutdown_connections 2024-09-16T15:42:34.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.503+0000 7f8283e00640 1 --2- 172.21.15.151:0/1471686208 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8268003a00 0x7f8268003de0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:34.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.503+0000 7f8283e00640 1 -- 172.21.15.151:0/1471686208 >> 172.21.15.151:0/1471686208 conn(0x7f82840d3dd0 msgr2=0x7f82840d41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:34.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.503+0000 7f8283e00640 1 -- 172.21.15.151:0/1471686208 shutdown_connections 2024-09-16T15:42:34.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.503+0000 7f8283e00640 1 -- 172.21.15.151:0/1471686208 wait complete. 2024-09-16T15:42:34.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.505+0000 7f8283e00640 1 Processor -- start 2024-09-16T15:42:34.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.505+0000 7f8283e00640 1 -- start start 2024-09-16T15:42:34.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.506+0000 7f8283e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8268003a00 0x7f8284154f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:34.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.506+0000 7f8283e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82840c85e0 0x7f82840b2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:34.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.506+0000 7f8283e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f82840b3090 0x7f82840c9f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:34.509 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.506+0000 7f8283e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f82840cfab0 con 0x7f8268003a00 2024-09-16T15:42:34.509 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.506+0000 7f8283e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f82840d4b00 con 0x7f82840c85e0 2024-09-16T15:42:34.509 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.506+0000 7f8283e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f82840cfc30 con 0x7f82840b3090 2024-09-16T15:42:34.509 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.506+0000 7f8282a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8268003a00 0x7f8284154f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:34.509 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.506+0000 7f8282a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8268003a00 0x7f8284154f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37022/0 (socket says 172.21.15.151:37022) 2024-09-16T15:42:34.509 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.506+0000 7f8282a00640 1 -- 172.21.15.151:0/4112452094 learned_addr learned my addr 172.21.15.151:0/4112452094 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:42:34.509 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.506+0000 7f8282000640 1 --2- 172.21.15.151:0/4112452094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82840c85e0 0x7f82840b2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:34.509 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.506+0000 7f8283400640 1 --2- 172.21.15.151:0/4112452094 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f82840b3090 0x7f82840c9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:34.509 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.506+0000 7f8282a00640 1 -- 172.21.15.151:0/4112452094 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f82840b3090 msgr2=0x7f82840c9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:34.509 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.506+0000 7f8282a00640 1 --2- 172.21.15.151:0/4112452094 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f82840b3090 0x7f82840c9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:34.509 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.506+0000 7f8282a00640 1 -- 172.21.15.151:0/4112452094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82840c85e0 msgr2=0x7f82840b2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:34.509 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.506+0000 7f8282a00640 1 --2- 172.21.15.151:0/4112452094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82840c85e0 0x7f82840b2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:34.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.506+0000 7f8282a00640 1 -- 172.21.15.151:0/4112452094 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82840c7820 con 0x7f8268003a00 2024-09-16T15:42:34.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.507+0000 7f8282a00640 1 --2- 172.21.15.151:0/4112452094 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8268003a00 0x7f8284154f30 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f827402a9d0 tx=0x7f8274006140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:34.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.507+0000 7f827aa00640 1 -- 172.21.15.151:0/4112452094 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8274040030 con 0x7f8268003a00 2024-09-16T15:42:34.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.507+0000 7f827aa00640 1 -- 172.21.15.151:0/4112452094 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8274004640 con 0x7f8268003a00 2024-09-16T15:42:34.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.507+0000 7f827aa00640 1 -- 172.21.15.151:0/4112452094 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8274037040 con 0x7f8268003a00 2024-09-16T15:42:34.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.507+0000 7f8283e00640 1 -- 172.21.15.151:0/4112452094 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82840c7b00 con 0x7f8268003a00 2024-09-16T15:42:34.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.507+0000 7f8283e00640 1 -- 172.21.15.151:0/4112452094 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8284155470 con 0x7f8268003a00 2024-09-16T15:42:34.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.508+0000 7f8283e00640 1 -- 172.21.15.151:0/4112452094 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82840c75b0 con 0x7f8268003a00 2024-09-16T15:42:34.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.509+0000 7f827aa00640 1 -- 172.21.15.151:0/4112452094 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7f8274043070 con 0x7f8268003a00 2024-09-16T15:42:34.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.510+0000 7f827aa00640 1 --2- 172.21.15.151:0/4112452094 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8264078be0 0x7f826407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:34.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.510+0000 7f827aa00640 1 -- 172.21.15.151:0/4112452094 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f82740ba7f0 con 0x7f8268003a00 2024-09-16T15:42:34.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.511+0000 7f8282000640 1 --2- 172.21.15.151:0/4112452094 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8264078be0 0x7f826407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:34.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.511+0000 7f8282000640 1 --2- 172.21.15.151:0/4112452094 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8264078be0 0x7f826407b0a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f82840b3a10 tx=0x7f8270005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:34.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.514+0000 7f827aa00640 1 -- 172.21.15.151:0/4112452094 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f827404a020 con 0x7f8268003a00 2024-09-16T15:42:34.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.765+0000 7f8283e00640 1 -- 172.21.15.151:0/4112452094 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f82840b5350 con 0x7f8268003a00 2024-09-16T15:42:34.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.768+0000 7f827aa00640 1 -- 172.21.15.151:0/4112452094 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v10) ==== 190+0+147 (secure 0 0 0) 0x7f82840b5350 con 0x7f8268003a00 2024-09-16T15:42:34.769 INFO:teuthology.orchestra.run.smithi151.stdout:[client.0] 2024-09-16T15:42:34.769 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQDsUOhmf5PBKBAAKE4Y4zbA4h07IBNjPkyptw== 2024-09-16T15:42:34.769 INFO:teuthology.orchestra.run.smithi151.stdout: caps mds = "allow" 2024-09-16T15:42:34.769 INFO:teuthology.orchestra.run.smithi151.stdout: caps mon = "allow r" 2024-09-16T15:42:34.770 INFO:teuthology.orchestra.run.smithi151.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-16T15:42:34.770 INFO:teuthology.orchestra.run.smithi151.stderr:updated caps for client.0 2024-09-16T15:42:34.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.770+0000 7f8283e00640 1 -- 172.21.15.151:0/4112452094 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8264078be0 msgr2=0x7f826407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:34.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.770+0000 7f8283e00640 1 --2- 172.21.15.151:0/4112452094 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8264078be0 0x7f826407b0a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f82840b3a10 tx=0x7f8270005f90 comp rx=0 tx=0).stop 2024-09-16T15:42:34.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.770+0000 7f8283e00640 1 -- 172.21.15.151:0/4112452094 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8268003a00 msgr2=0x7f8284154f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:34.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.770+0000 7f8283e00640 1 --2- 172.21.15.151:0/4112452094 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8268003a00 0x7f8284154f30 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f827402a9d0 tx=0x7f8274006140 comp rx=0 tx=0).stop 2024-09-16T15:42:34.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.770+0000 7f8283e00640 1 -- 172.21.15.151:0/4112452094 shutdown_connections 2024-09-16T15:42:34.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.770+0000 7f8283e00640 1 --2- 172.21.15.151:0/4112452094 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8264078be0 0x7f826407b0a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:34.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.770+0000 7f8283e00640 1 --2- 172.21.15.151:0/4112452094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82840c85e0 0x7f82840b2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:34.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.770+0000 7f8283e00640 1 --2- 172.21.15.151:0/4112452094 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f82840b3090 0x7f82840c9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:34.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.770+0000 7f8283e00640 1 --2- 172.21.15.151:0/4112452094 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8268003a00 0x7f8284154f30 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:34.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.771+0000 7f8283e00640 1 -- 172.21.15.151:0/4112452094 >> 172.21.15.151:0/4112452094 conn(0x7f82840d3dd0 msgr2=0x7f82840b8ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:34.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.771+0000 7f8283e00640 1 -- 172.21.15.151:0/4112452094 shutdown_connections 2024-09-16T15:42:34.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.771+0000 7f8283e00640 1 -- 172.21.15.151:0/4112452094 wait complete. 2024-09-16T15:42:34.789 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:42:34.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.946+0000 7f96b9000640 1 -- 172.21.15.151:0/4124526751 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f96b40725d0 msgr2=0x7f96b40b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:34.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.946+0000 7f96b9000640 1 --2- 172.21.15.151:0/4124526751 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f96b40725d0 0x7f96b40b28f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f96b4074a80 tx=0x7f96a002a5f0 comp rx=0 tx=0).stop 2024-09-16T15:42:34.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.946+0000 7f96b9000640 1 -- 172.21.15.151:0/4124526751 shutdown_connections 2024-09-16T15:42:34.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.946+0000 7f96b9000640 1 --2- 172.21.15.151:0/4124526751 >> v2:172.21.15.167:3300/0 conn(0x7f96b40d0540 0x7f96b406f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:34.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.946+0000 7f96b9000640 1 --2- 172.21.15.151:0/4124526751 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f96b40725d0 0x7f96b40b28f0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:34.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.946+0000 7f96b9000640 1 -- 172.21.15.151:0/4124526751 >> 172.21.15.151:0/4124526751 conn(0x7f96b40cfa20 msgr2=0x7f96b40cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:34.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.946+0000 7f96b9000640 1 -- 172.21.15.151:0/4124526751 shutdown_connections 2024-09-16T15:42:34.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.946+0000 7f96b9000640 1 -- 172.21.15.151:0/4124526751 wait complete. 2024-09-16T15:42:34.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.948+0000 7f96b9000640 1 Processor -- start 2024-09-16T15:42:34.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.948+0000 7f96b9000640 1 -- start start 2024-09-16T15:42:34.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.949+0000 7f96b9000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f96b40d0540 0x7f96b4156160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:34.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.949+0000 7f96b9000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96b415bc30 0x7f96b40cef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:34.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.949+0000 7f96b9000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f96b40c94e0 0x7f96b4153f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:34.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.949+0000 7f96b9000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f96b4074660 con 0x7f96b40d0540 2024-09-16T15:42:34.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.949+0000 7f96b9000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f96b406f860 con 0x7f96b415bc30 2024-09-16T15:42:34.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.949+0000 7f96b9000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f96b41566a0 con 0x7f96b40c94e0 2024-09-16T15:42:34.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.949+0000 7f96b3400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f96b40d0540 0x7f96b4156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:34.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.949+0000 7f96b3400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f96b40d0540 0x7f96b4156160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37032/0 (socket says 172.21.15.151:37032) 2024-09-16T15:42:34.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.949+0000 7f96b3400640 1 -- 172.21.15.151:0/1937932955 learned_addr learned my addr 172.21.15.151:0/1937932955 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:42:34.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.949+0000 7f96b2a00640 1 --2- 172.21.15.151:0/1937932955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96b415bc30 0x7f96b40cef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:34.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.950+0000 7f96b3e00640 1 --2- 172.21.15.151:0/1937932955 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f96b40c94e0 0x7f96b4153f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:34.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.950+0000 7f96b3400640 1 -- 172.21.15.151:0/1937932955 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f96b40c94e0 msgr2=0x7f96b4153f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:34.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.950+0000 7f96b3400640 1 --2- 172.21.15.151:0/1937932955 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f96b40c94e0 0x7f96b4153f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:34.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.950+0000 7f96b3400640 1 -- 172.21.15.151:0/1937932955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96b415bc30 msgr2=0x7f96b40cef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:34.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.950+0000 7f96b3400640 1 --2- 172.21.15.151:0/1937932955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96b415bc30 0x7f96b40cef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:34.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.950+0000 7f96b3400640 1 -- 172.21.15.151:0/1937932955 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96b4156960 con 0x7f96b40d0540 2024-09-16T15:42:34.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.950+0000 7f96b2a00640 1 --2- 172.21.15.151:0/1937932955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96b415bc30 0x7f96b40cef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:42:34.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.950+0000 7f96b3400640 1 --2- 172.21.15.151:0/1937932955 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f96b40d0540 0x7f96b4156160 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f96a400c3c0 tx=0x7f96a400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:34.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.950+0000 7f96ab400640 1 -- 172.21.15.151:0/1937932955 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a4017070 con 0x7f96b40d0540 2024-09-16T15:42:34.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.950+0000 7f96ab400640 1 -- 172.21.15.151:0/1937932955 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f96a40035f0 con 0x7f96b40d0540 2024-09-16T15:42:34.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.950+0000 7f96ab400640 1 -- 172.21.15.151:0/1937932955 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a4003a80 con 0x7f96b40d0540 2024-09-16T15:42:34.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.950+0000 7f96b9000640 1 -- 172.21.15.151:0/1937932955 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96b40c8720 con 0x7f96b40d0540 2024-09-16T15:42:34.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.950+0000 7f96b9000640 1 -- 172.21.15.151:0/1937932955 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96b40c8ce0 con 0x7f96b40d0540 2024-09-16T15:42:34.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.951+0000 7f96b9000640 1 -- 172.21.15.151:0/1937932955 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96b406fdf0 con 0x7f96b40d0540 2024-09-16T15:42:34.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.953+0000 7f96ab400640 1 -- 172.21.15.151:0/1937932955 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7f96a401a070 con 0x7f96b40d0540 2024-09-16T15:42:34.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.953+0000 7f96ab400640 1 --2- 172.21.15.151:0/1937932955 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9694078bb0 0x7f969407b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:34.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.954+0000 7f96ab400640 1 -- 172.21.15.151:0/1937932955 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f96a409a270 con 0x7f96b40d0540 2024-09-16T15:42:34.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.954+0000 7f96b2a00640 1 --2- 172.21.15.151:0/1937932955 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9694078bb0 0x7f969407b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:34.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.955+0000 7f96b2a00640 1 --2- 172.21.15.151:0/1937932955 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9694078bb0 0x7f969407b070 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f96a0033000 tx=0x7f96a0004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:34.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:34.957+0000 7f96ab400640 1 -- 172.21.15.151:0/1937932955 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f96a4062380 con 0x7f96b40d0540 2024-09-16T15:42:35.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.179+0000 7f96b9000640 1 -- 172.21.15.151:0/1937932955 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f96b40b2990 con 0x7f96b40d0540 2024-09-16T15:42:35.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.179+0000 7f96ab400640 1 -- 172.21.15.151:0/1937932955 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5232 (secure 0 0 0) 0x7f96a4061c80 con 0x7f96b40d0540 2024-09-16T15:42:35.180 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:42:35.181 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":27,"btime":"2024-09-16T15:42:28:009764+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5272,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1043845735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1043845735},{"type":"v1","addr":"172.21.15.167:6836","nonce":1043845735}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5275,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/816612106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":816612106},{"type":"v1","addr":"172.21.15.167:6837","nonce":816612106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5397,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/1986755100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1986755100},{"type":"v1","addr":"172.21.15.151:6834","nonce":1986755100}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:42:24.411561+0000","modified":"2024-09-16T15:42:27.425402+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5400":{"gid":5400,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.151:6835/1974411916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":1974411916},{"type":"v1","addr":"172.21.15.151:6835","nonce":1974411916}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5400,"qdb_cluster":[5400]},"id":3}]} 2024-09-16T15:42:35.181 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 27 2024-09-16T15:42:35.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.182+0000 7f96b9000640 1 -- 172.21.15.151:0/1937932955 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9694078bb0 msgr2=0x7f969407b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:35.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.182+0000 7f96b9000640 1 --2- 172.21.15.151:0/1937932955 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9694078bb0 0x7f969407b070 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f96a0033000 tx=0x7f96a0004730 comp rx=0 tx=0).stop 2024-09-16T15:42:35.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.182+0000 7f96b9000640 1 -- 172.21.15.151:0/1937932955 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f96b40d0540 msgr2=0x7f96b4156160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:35.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.182+0000 7f96b9000640 1 --2- 172.21.15.151:0/1937932955 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f96b40d0540 0x7f96b4156160 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f96a400c3c0 tx=0x7f96a400c880 comp rx=0 tx=0).stop 2024-09-16T15:42:35.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.182+0000 7f96b9000640 1 -- 172.21.15.151:0/1937932955 shutdown_connections 2024-09-16T15:42:35.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.182+0000 7f96b9000640 1 --2- 172.21.15.151:0/1937932955 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9694078bb0 0x7f969407b070 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:35.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.182+0000 7f96b9000640 1 --2- 172.21.15.151:0/1937932955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96b415bc30 0x7f96b40cef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:35.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.182+0000 7f96b9000640 1 --2- 172.21.15.151:0/1937932955 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f96b40c94e0 0x7f96b4153f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:35.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.182+0000 7f96b9000640 1 --2- 172.21.15.151:0/1937932955 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f96b40d0540 0x7f96b4156160 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:35.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.182+0000 7f96b9000640 1 -- 172.21.15.151:0/1937932955 >> 172.21.15.151:0/1937932955 conn(0x7f96b40cfa20 msgr2=0x7f96b4152f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:35.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.182+0000 7f96b9000640 1 -- 172.21.15.151:0/1937932955 shutdown_connections 2024-09-16T15:42:35.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.182+0000 7f96b9000640 1 -- 172.21.15.151:0/1937932955 wait complete. 2024-09-16T15:42:35.601 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-16T15:42:24.411561+0000', 'modified': '2024-09-16T15:42:27.425402+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5400}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5400': {'gid': 5400, 'name': 'c', 'rank': 0, 'incarnation': 23, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.151:6835/1974411916', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6833', 'nonce': 1974411916}, {'type': 'v1', 'addr': '172.21.15.151:6835', 'nonce': 1974411916}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5400, 'qdb_cluster': [5400]} 2024-09-16T15:42:35.601 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-16T15:42:35.602 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-16T15:42:35.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.762+0000 7f7ce6a00640 1 -- 172.21.15.151:0/4033817943 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ce0073fb0 msgr2=0x7f7ce0074420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:35.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.762+0000 7f7ce6a00640 1 --2- 172.21.15.151:0/4033817943 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ce0073fb0 0x7f7ce0074420 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0075130 tx=0x7f7cd802a980 comp rx=0 tx=0).stop 2024-09-16T15:42:35.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.762+0000 7f7ce6a00640 1 -- 172.21.15.151:0/4033817943 shutdown_connections 2024-09-16T15:42:35.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.762+0000 7f7ce6a00640 1 --2- 172.21.15.151:0/4033817943 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ce0073fb0 0x7f7ce0074420 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:35.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.762+0000 7f7ce6a00640 1 --2- 172.21.15.151:0/4033817943 >> v2:172.21.15.167:3300/0 conn(0x7f7ce015f5a0 0x7f7ce00b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:35.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.762+0000 7f7ce6a00640 1 -- 172.21.15.151:0/4033817943 >> 172.21.15.151:0/4033817943 conn(0x7f7ce00b1950 msgr2=0x7f7ce00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:35.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.762+0000 7f7ce6a00640 1 -- 172.21.15.151:0/4033817943 shutdown_connections 2024-09-16T15:42:35.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.762+0000 7f7ce6a00640 1 -- 172.21.15.151:0/4033817943 wait complete. 2024-09-16T15:42:35.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.764+0000 7f7ce6a00640 1 Processor -- start 2024-09-16T15:42:35.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.764+0000 7f7ce6a00640 1 -- start start 2024-09-16T15:42:35.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.765+0000 7f7ce6a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ce0073fb0 0x7f7ce00cf170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:35.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.765+0000 7f7ce6a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7ce00ccf00 0x7f7ce0143f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:35.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.765+0000 7f7ce6a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ce015f5a0 0x7f7ce01444c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:35.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.765+0000 7f7ce6a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce006f2e0 con 0x7f7ce0073fb0 2024-09-16T15:42:35.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.765+0000 7f7ce6a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce006f160 con 0x7f7ce015f5a0 2024-09-16T15:42:35.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.765+0000 7f7ce6a00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f7ce006f460 con 0x7f7ce00ccf00 2024-09-16T15:42:35.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.765+0000 7f7cdfe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ce0073fb0 0x7f7ce00cf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:35.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.765+0000 7f7cdfe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ce0073fb0 0x7f7ce00cf170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37042/0 (socket says 172.21.15.151:37042) 2024-09-16T15:42:35.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.765+0000 7f7cdfe00640 1 -- 172.21.15.151:0/349054885 learned_addr learned my addr 172.21.15.151:0/349054885 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:42:35.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.765+0000 7f7cd7e00640 1 --2- 172.21.15.151:0/349054885 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7ce00ccf00 0x7f7ce0143f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:35.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.765+0000 7f7ce4a00640 1 --2- 172.21.15.151:0/349054885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ce015f5a0 0x7f7ce01444c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:35.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.766+0000 7f7cdfe00640 1 -- 172.21.15.151:0/349054885 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7ce00ccf00 msgr2=0x7f7ce0143f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:35.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.766+0000 7f7cdfe00640 1 --2- 172.21.15.151:0/349054885 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7ce00ccf00 0x7f7ce0143f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:35.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.766+0000 7f7cdfe00640 1 -- 172.21.15.151:0/349054885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ce015f5a0 msgr2=0x7f7ce01444c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:35.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.766+0000 7f7cdfe00640 1 --2- 172.21.15.151:0/349054885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ce015f5a0 0x7f7ce01444c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:35.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.766+0000 7f7cdfe00640 1 -- 172.21.15.151:0/349054885 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ce0147ba0 con 0x7f7ce0073fb0 2024-09-16T15:42:35.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.766+0000 7f7cdfe00640 1 --2- 172.21.15.151:0/349054885 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ce0073fb0 0x7f7ce00cf170 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f7cd000d700 tx=0x7f7cd000dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:35.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.766+0000 7f7cdcc00640 1 -- 172.21.15.151:0/349054885 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd0017070 con 0x7f7ce0073fb0 2024-09-16T15:42:35.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.766+0000 7f7cdcc00640 1 -- 172.21.15.151:0/349054885 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7cd0005310 con 0x7f7ce0073fb0 2024-09-16T15:42:35.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.766+0000 7f7ce6a00640 1 -- 172.21.15.151:0/349054885 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ce0146de0 con 0x7f7ce0073fb0 2024-09-16T15:42:35.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.766+0000 7f7cdcc00640 1 -- 172.21.15.151:0/349054885 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd00097c0 con 0x7f7ce0073fb0 2024-09-16T15:42:35.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.766+0000 7f7ce6a00640 1 -- 172.21.15.151:0/349054885 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ce01473a0 con 0x7f7ce0073fb0 2024-09-16T15:42:35.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.768+0000 7f7cdcc00640 1 -- 172.21.15.151:0/349054885 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7f7cd0009960 con 0x7f7ce0073fb0 2024-09-16T15:42:35.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.769+0000 7f7cdcc00640 1 --2- 172.21.15.151:0/349054885 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7cbc078bb0 0x7f7cbc07b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:35.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.770+0000 7f7cdcc00640 1 -- 172.21.15.151:0/349054885 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f7cd009b140 con 0x7f7ce0073fb0 2024-09-16T15:42:35.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.770+0000 7f7ce9819740 1 Processor -- start 2024-09-16T15:42:35.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.770+0000 7f7cd7e00640 1 --2- 172.21.15.151:0/349054885 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7cbc078bb0 0x7f7cbc07b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:35.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.770+0000 7f7ce9819740 1 -- start start 2024-09-16T15:42:35.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.771+0000 7f7cd7e00640 1 --2- 172.21.15.151:0/349054885 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7cbc078bb0 0x7f7cbc07b070 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0070000 tx=0x7f7cc8009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:35.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.771+0000 7f7ce9819740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555e68cf88b0 0x555e68cf8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:35.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.771+0000 7f7ce9819740 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x555e68cadf20 0x555e68cd9610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:35.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.771+0000 7f7ce9819740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e68cad000 0x555e68cd9b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:35.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.771+0000 7f7ce9819740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555e68c36430 con 0x555e68cad000 2024-09-16T15:42:35.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.771+0000 7f7ce9819740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x555e68c265d0 con 0x555e68cf88b0 2024-09-16T15:42:35.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.771+0000 7f7ce9819740 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x555e68c15a00 con 0x555e68cadf20 2024-09-16T15:42:35.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.771+0000 7f7ce4a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e68cad000 0x555e68cd9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:35.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.771+0000 7f7ce4a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e68cad000 0x555e68cd9b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37050/0 (socket says 172.21.15.151:37050) 2024-09-16T15:42:35.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.771+0000 7f7ce4a00640 1 -- 172.21.15.151:0/3837880487 learned_addr learned my addr 172.21.15.151:0/3837880487 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:42:35.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.771+0000 7f7cd7e00640 1 --2- 172.21.15.151:0/3837880487 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x555e68cadf20 0x555e68cd9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:35.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.772+0000 7f7cdfe00640 1 --2- 172.21.15.151:0/3837880487 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555e68cf88b0 0x555e68cf8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:35.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.772+0000 7f7ce4a00640 1 -- 172.21.15.151:0/3837880487 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x555e68cadf20 msgr2=0x555e68cd9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:42:35.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.772+0000 7f7ce4a00640 1 --2- 172.21.15.151:0/3837880487 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x555e68cadf20 0x555e68cd9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:35.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.772+0000 7f7ce4a00640 1 -- 172.21.15.151:0/3837880487 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555e68cf88b0 msgr2=0x555e68cf8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:42:35.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.772+0000 7f7ce4a00640 1 --2- 172.21.15.151:0/3837880487 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555e68cf88b0 0x555e68cf8c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:35.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.772+0000 7f7ce4a00640 1 -- 172.21.15.151:0/3837880487 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e68bde520 con 0x555e68cad000 2024-09-16T15:42:35.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.772+0000 7f7ce4a00640 1 --2- 172.21.15.151:0/3837880487 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e68cad000 0x555e68cd9b50 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f7cd802af70 tx=0x7f7cd8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=908db1ffcb0dd3d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:35.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.772+0000 7f7ccfe00640 1 -- 172.21.15.151:0/3837880487 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd8034070 con 0x555e68cad000 2024-09-16T15:42:35.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.772+0000 7f7ccfe00640 1 -- 172.21.15.151:0/3837880487 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7cd8003bb0 con 0x555e68cad000 2024-09-16T15:42:35.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.772+0000 7f7ccfe00640 1 -- 172.21.15.151:0/3837880487 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd8030680 con 0x555e68cad000 2024-09-16T15:42:35.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.773+0000 7f7ce9819740 1 -- 172.21.15.151:0/3837880487 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e68cad000 msgr2=0x555e68cd9b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:35.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.773+0000 7f7ce9819740 1 --2- 172.21.15.151:0/3837880487 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e68cad000 0x555e68cd9b50 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f7cd802af70 tx=0x7f7cd8004a40 comp rx=0 tx=0).stop 2024-09-16T15:42:35.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.773+0000 7f7ce9819740 1 -- 172.21.15.151:0/3837880487 shutdown_connections 2024-09-16T15:42:35.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.773+0000 7f7ce9819740 1 --2- 172.21.15.151:0/3837880487 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e68cad000 0x555e68cd9b50 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:35.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.773+0000 7f7ce9819740 1 --2- 172.21.15.151:0/3837880487 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x555e68cadf20 0x555e68cd9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:35.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.773+0000 7f7ce9819740 1 --2- 172.21.15.151:0/3837880487 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555e68cf88b0 0x555e68cf8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:35.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.773+0000 7f7ce9819740 1 -- 172.21.15.151:0/3837880487 >> 172.21.15.151:0/3837880487 conn(0x555e68caeff0 msgr2=0x555e68cb2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:35.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.773+0000 7f7ce9819740 1 -- 172.21.15.151:0/3837880487 shutdown_connections 2024-09-16T15:42:35.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.773+0000 7f7ce9819740 1 -- 172.21.15.151:0/3837880487 wait complete. 2024-09-16T15:42:35.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.774+0000 7f7ce9819740 1 -- start start 2024-09-16T15:42:35.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.774+0000 7f7ce9819740 1 Processor -- start 2024-09-16T15:42:35.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.775+0000 7f7ce9819740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555e68cadf20 0x555e68cd2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:35.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.775+0000 7f7ce9819740 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x555e68cf88b0 0x555e68cd3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:35.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.775+0000 7f7ce9819740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e68cf5de0 0x555e68cd39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:35.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.775+0000 7f7ce9819740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555e68bbc330 con 0x555e68cf5de0 2024-09-16T15:42:35.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.775+0000 7f7ce9819740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x555e68c19810 con 0x555e68cadf20 2024-09-16T15:42:35.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.775+0000 7f7ce9819740 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x555e68986620 con 0x555e68cf88b0 2024-09-16T15:42:35.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.775+0000 7f7ce9819740 10 client.0 _ll_register_callbacks cb 0x555e68bc1420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T15:42:35.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.775+0000 7f7ce4a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e68cf5de0 0x555e68cd39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:35.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.775+0000 7f7ce4a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e68cf5de0 0x555e68cd39b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37056/0 (socket says 172.21.15.151:37056) 2024-09-16T15:42:35.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.775+0000 7f7ce4a00640 1 -- 172.21.15.151:0/4087186964 learned_addr learned my addr 172.21.15.151:0/4087186964 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:42:35.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.775+0000 7f7cdfe00640 1 --2- 172.21.15.151:0/4087186964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555e68cadf20 0x555e68cd2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:35.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.775+0000 7f7cd7e00640 1 --2- 172.21.15.151:0/4087186964 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x555e68cf88b0 0x555e68cd3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:35.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.776+0000 7f7ce4a00640 1 -- 172.21.15.151:0/4087186964 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x555e68cf88b0 msgr2=0x555e68cd3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:42:35.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.776+0000 7f7ce4a00640 1 --2- 172.21.15.151:0/4087186964 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x555e68cf88b0 0x555e68cd3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:35.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.776+0000 7f7ce4a00640 1 -- 172.21.15.151:0/4087186964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555e68cadf20 msgr2=0x555e68cd2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:42:35.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.776+0000 7f7ce4a00640 1 --2- 172.21.15.151:0/4087186964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555e68cadf20 0x555e68cd2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:35.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.776+0000 7f7ce4a00640 1 -- 172.21.15.151:0/4087186964 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e68977850 con 0x555e68cf5de0 2024-09-16T15:42:35.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.776+0000 7f7ce4a00640 1 --2- 172.21.15.151:0/4087186964 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e68cf5de0 0x555e68cd39b0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f7cd8030bc0 tx=0x7f7cd8030bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b41c1b57f220b560 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:35.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.776+0000 7f7cccc00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-16T15:42:35.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.776+0000 7f7cccc00640 1 -- 172.21.15.151:0/4087186964 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd8034030 con 0x555e68cf5de0 2024-09-16T15:42:35.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.776+0000 7f7ce9819740 1 -- 172.21.15.151:0/4087186964 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7cd8003bb0 con 0x555e68cf5de0 2024-09-16T15:42:35.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.776+0000 7f7cccc00640 1 -- 172.21.15.151:0/4087186964 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f7cd804daa0 con 0x555e68cf5de0 2024-09-16T15:42:35.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.777+0000 7f7cccc00640 1 -- 172.21.15.151:0/4087186964 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7cd8054050 con 0x555e68cf5de0 2024-09-16T15:42:35.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.777+0000 7f7ce9819740 10 client.5556 fetch_fsmap learned FSMap version 27 2024-09-16T15:42:35.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.777+0000 7f7ce9819740 1 -- 172.21.15.151:0/4087186964 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) -- 0x555e689ee6b0 con 0x555e68cf5de0 2024-09-16T15:42:35.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.777+0000 7f7cccc00640 1 -- 172.21.15.151:0/4087186964 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7cd8003bb0 con 0x555e68cf5de0 2024-09-16T15:42:35.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.777+0000 7f7cccc00640 1 -- 172.21.15.151:0/4087186964 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 27) ==== 3502+0+0 (secure 0 0 0) 0x7f7cd8057020 con 0x555e68cf5de0 2024-09-16T15:42:35.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.777+0000 7f7ce9819740 10 client.5556 fetch_fsmap finished waiting for FSMap version 27 2024-09-16T15:42:35.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.777+0000 7f7ce9819740 10 client.5556 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-16T15:42:35.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.777+0000 7f7ce9819740 20 client.5556 populate_metadata read hostname 'smithi151' 2024-09-16T15:42:35.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.777+0000 7f7ce9819740 1 --2- 172.21.15.151:0/4087186964 >> [v2:172.21.15.151:6833/1974411916,v1:172.21.15.151:6835/1974411916] conn(0x555e68d0f470 0x555e68d0f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:35.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.777+0000 7f7ce9819740 4 client.5556 mds_command: new command op to 5400 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:42:35.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.777+0000 7f7ce9819740 1 -- 172.21.15.151:0/4087186964 --> [v2:172.21.15.151:6833/1974411916,v1:172.21.15.151:6835/1974411916] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e68c28520 con 0x555e68d0f470 2024-09-16T15:42:35.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.777+0000 7f7cdfe00640 1 --2- 172.21.15.151:0/4087186964 >> [v2:172.21.15.151:6833/1974411916,v1:172.21.15.151:6835/1974411916] conn(0x555e68d0f470 0x555e68d0f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:35.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.778+0000 7f7cdfe00640 1 --2- 172.21.15.151:0/4087186964 >> [v2:172.21.15.151:6833/1974411916,v1:172.21.15.151:6835/1974411916] conn(0x555e68d0f470 0x555e68d0f850 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96b52c7a4624f7b server_cookie=a58d0fd109e13c6b in_seq=0 out_seq=0 2024-09-16T15:42:35.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.778+0000 7f7cccc00640 10 client.5556 ms_handle_connect on v2:172.21.15.151:6833/1974411916 2024-09-16T15:42:35.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.779+0000 7f7cccc00640 1 -- 172.21.15.151:0/4087186964 <== mds.0 v2:172.21.15.151:6833/1974411916 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555e68c28520 con 0x555e68d0f470 2024-09-16T15:42:35.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.779+0000 7f7cccc00640 10 client.5556 handle_command_reply: tid=0 2024-09-16T15:42:35.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.779+0000 7f7ce9819740 1 client.5556 shutdown 2024-09-16T15:42:35.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.779+0000 7f7ce9819740 1 -- 172.21.15.151:0/4087186964 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e68cf5de0 msgr2=0x555e68cd39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:35.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.779+0000 7f7ce9819740 1 --2- 172.21.15.151:0/4087186964 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e68cf5de0 0x555e68cd39b0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f7cd8030bc0 tx=0x7f7cd8030bf0 comp rx=0 tx=0).stop 2024-09-16T15:42:35.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.779+0000 7f7ce9819740 1 -- 172.21.15.151:0/4087186964 shutdown_connections 2024-09-16T15:42:35.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.779+0000 7f7ce9819740 1 --2- 172.21.15.151:0/4087186964 >> [v2:172.21.15.151:6833/1974411916,v1:172.21.15.151:6835/1974411916] conn(0x555e68d0f470 0x555e68d0f850 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:35.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.779+0000 7f7ce9819740 1 --2- 172.21.15.151:0/4087186964 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e68cf5de0 0x555e68cd39b0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:35.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.779+0000 7f7ce9819740 1 --2- 172.21.15.151:0/4087186964 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x555e68cf88b0 0x555e68cd3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:35.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.779+0000 7f7cccc00640 0 client.5556 ms_handle_reset on v2:172.21.15.151:6833/1974411916 2024-09-16T15:42:35.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.779+0000 7f7ce9819740 1 --2- 172.21.15.151:0/4087186964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555e68cadf20 0x555e68cd2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:35.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.779+0000 7f7ce9819740 1 -- 172.21.15.151:0/4087186964 >> 172.21.15.151:0/4087186964 conn(0x555e68cad000 msgr2=0x555e68cd93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:35.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.779+0000 7f7ce9819740 1 -- 172.21.15.151:0/4087186964 shutdown_connections 2024-09-16T15:42:35.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.779+0000 7f7ce9819740 1 -- 172.21.15.151:0/4087186964 wait complete. 2024-09-16T15:42:35.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.779+0000 7f7ce9819740 20 client.5556 trim_cache size 0 max 16384 2024-09-16T15:42:35.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.798+0000 7f7ce9819740 1 Processor -- start 2024-09-16T15:42:35.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.798+0000 7f7ce9819740 1 -- start start 2024-09-16T15:42:35.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.799+0000 7f7ce9819740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e68cadf20 0x555e68cad3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:35.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.799+0000 7f7ce9819740 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x555e68cf5de0 0x555e68cad920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:35.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.799+0000 7f7ce9819740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555e68cf88b0 0x555e68cb17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:35.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.799+0000 7f7ce9819740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce00b45b0 con 0x555e68cadf20 2024-09-16T15:42:35.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.799+0000 7f7ce9819740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce0146620 con 0x555e68cf88b0 2024-09-16T15:42:35.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.799+0000 7f7ce9819740 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f7ce00b3f60 con 0x555e68cf5de0 2024-09-16T15:42:35.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.799+0000 7f7cdfe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e68cadf20 0x555e68cad3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:35.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.799+0000 7f7cdfe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e68cadf20 0x555e68cad3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37068/0 (socket says 172.21.15.151:37068) 2024-09-16T15:42:35.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.799+0000 7f7cdfe00640 1 -- 172.21.15.151:0/3827113045 learned_addr learned my addr 172.21.15.151:0/3827113045 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:42:35.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.799+0000 7f7cd7e00640 1 --2- 172.21.15.151:0/3827113045 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x555e68cf5de0 0x555e68cad920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:35.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.799+0000 7f7ce4a00640 1 --2- 172.21.15.151:0/3827113045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555e68cf88b0 0x555e68cb17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:35.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.800+0000 7f7cdfe00640 1 -- 172.21.15.151:0/3827113045 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x555e68cf5de0 msgr2=0x555e68cad920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:42:35.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.800+0000 7f7cdfe00640 1 --2- 172.21.15.151:0/3827113045 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x555e68cf5de0 0x555e68cad920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:35.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.800+0000 7f7cdfe00640 1 -- 172.21.15.151:0/3827113045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555e68cf88b0 msgr2=0x555e68cb17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:42:35.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.800+0000 7f7cdfe00640 1 --2- 172.21.15.151:0/3827113045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555e68cf88b0 0x555e68cb17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:35.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.800+0000 7f7cdfe00640 1 -- 172.21.15.151:0/3827113045 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e68cf8160 con 0x555e68cadf20 2024-09-16T15:42:35.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.800+0000 7f7ce4a00640 1 --2- 172.21.15.151:0/3827113045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555e68cf88b0 0x555e68cb17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:42:35.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.800+0000 7f7cdfe00640 1 --2- 172.21.15.151:0/3827113045 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e68cadf20 0x555e68cad3e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f7cd005fcc0 tx=0x7f7cd0066040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21b220cbe446432f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:35.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.800+0000 7f7ccf400640 1 -- 172.21.15.151:0/3827113045 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd0020040 con 0x555e68cadf20 2024-09-16T15:42:35.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.800+0000 7f7ccf400640 1 -- 172.21.15.151:0/3827113045 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7cd0062a50 con 0x555e68cadf20 2024-09-16T15:42:35.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.800+0000 7f7ccf400640 1 -- 172.21.15.151:0/3827113045 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd0009e50 con 0x555e68cadf20 2024-09-16T15:42:35.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.801+0000 7f7ce9819740 1 -- 172.21.15.151:0/3827113045 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e68cadf20 msgr2=0x555e68cad3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:35.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.801+0000 7f7ce9819740 1 --2- 172.21.15.151:0/3827113045 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e68cadf20 0x555e68cad3e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f7cd005fcc0 tx=0x7f7cd0066040 comp rx=0 tx=0).stop 2024-09-16T15:42:35.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.801+0000 7f7ce9819740 1 -- 172.21.15.151:0/3827113045 shutdown_connections 2024-09-16T15:42:35.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.801+0000 7f7ce9819740 1 --2- 172.21.15.151:0/3827113045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555e68cf88b0 0x555e68cb17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:35.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.801+0000 7f7ce9819740 1 --2- 172.21.15.151:0/3827113045 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x555e68cf5de0 0x555e68cad920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:35.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.801+0000 7f7ce9819740 1 --2- 172.21.15.151:0/3827113045 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e68cadf20 0x555e68cad3e0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:35.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.801+0000 7f7ce9819740 1 -- 172.21.15.151:0/3827113045 >> 172.21.15.151:0/3827113045 conn(0x555e68cad000 msgr2=0x555e68cf4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:35.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.801+0000 7f7ce9819740 1 -- 172.21.15.151:0/3827113045 shutdown_connections 2024-09-16T15:42:35.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.801+0000 7f7ce9819740 1 -- 172.21.15.151:0/3827113045 wait complete. 2024-09-16T15:42:35.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.801+0000 7f7ce9819740 1 -- start start 2024-09-16T15:42:35.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.802+0000 7f7ce9819740 1 Processor -- start 2024-09-16T15:42:35.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.802+0000 7f7ce9819740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555e68cf5de0 0x555e68d80b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:35.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.802+0000 7f7ce9819740 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x555e68cf88b0 0x555e68d1f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:35.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.802+0000 7f7ce9819740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e68cadaf0 0x555e68d1fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:35.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.802+0000 7f7ce9819740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555e68cd3130 con 0x555e68cadaf0 2024-09-16T15:42:35.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.802+0000 7f7ce9819740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x555e68d0c8b0 con 0x555e68cf5de0 2024-09-16T15:42:35.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.802+0000 7f7ce9819740 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x555e68cf6f00 con 0x555e68cf88b0 2024-09-16T15:42:35.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.803+0000 7f7ce9819740 10 client.0 _ll_register_callbacks cb 0x7f7cd8030a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T15:42:35.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.803+0000 7f7ce4a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e68cadaf0 0x555e68d1fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:35.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.803+0000 7f7ce4a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e68cadaf0 0x555e68d1fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37082/0 (socket says 172.21.15.151:37082) 2024-09-16T15:42:35.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.803+0000 7f7ce4a00640 1 -- 172.21.15.151:0/2431770026 learned_addr learned my addr 172.21.15.151:0/2431770026 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:42:35.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.803+0000 7f7cd7e00640 1 --2- 172.21.15.151:0/2431770026 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x555e68cf88b0 0x555e68d1f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:35.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.803+0000 7f7cdfe00640 1 --2- 172.21.15.151:0/2431770026 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555e68cf5de0 0x555e68d80b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:35.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.803+0000 7f7ce4a00640 1 -- 172.21.15.151:0/2431770026 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x555e68cf88b0 msgr2=0x555e68d1f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:42:35.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.803+0000 7f7ce4a00640 1 --2- 172.21.15.151:0/2431770026 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x555e68cf88b0 0x555e68d1f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:35.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.803+0000 7f7ce4a00640 1 -- 172.21.15.151:0/2431770026 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555e68cf5de0 msgr2=0x555e68d80b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:42:35.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.803+0000 7f7ce4a00640 1 --2- 172.21.15.151:0/2431770026 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555e68cf5de0 0x555e68d80b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:35.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.803+0000 7f7ce4a00640 1 -- 172.21.15.151:0/2431770026 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e68cd3e00 con 0x555e68cadaf0 2024-09-16T15:42:35.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.803+0000 7f7cdfe00640 1 --2- 172.21.15.151:0/2431770026 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555e68cf5de0 0x555e68d80b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:42:35.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.803+0000 7f7ce4a00640 1 --2- 172.21.15.151:0/2431770026 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e68cadaf0 0x555e68d1fd00 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7cd8004af0 tx=0x7f7cd8044e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80d983028cec75ef server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:35.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.803+0000 7f7cccc00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-16T15:42:35.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.804+0000 7f7cccc00640 1 -- 172.21.15.151:0/2431770026 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd8034040 con 0x555e68cadaf0 2024-09-16T15:42:35.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.804+0000 7f7cccc00640 1 -- 172.21.15.151:0/2431770026 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f7cd8059aa0 con 0x555e68cadaf0 2024-09-16T15:42:35.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.804+0000 7f7ce9819740 1 -- 172.21.15.151:0/2431770026 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7cd0062a50 con 0x555e68cadaf0 2024-09-16T15:42:35.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.804+0000 7f7cccc00640 1 -- 172.21.15.151:0/2431770026 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7cd8030c60 con 0x555e68cadaf0 2024-09-16T15:42:35.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.804+0000 7f7ce9819740 10 client.5562 fetch_fsmap learned FSMap version 27 2024-09-16T15:42:35.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.804+0000 7f7ce9819740 1 -- 172.21.15.151:0/2431770026 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) -- 0x555e68cb05c0 con 0x555e68cadaf0 2024-09-16T15:42:35.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.804+0000 7f7cccc00640 1 -- 172.21.15.151:0/2431770026 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7cd0062a50 con 0x555e68cadaf0 2024-09-16T15:42:35.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.804+0000 7f7cccc00640 1 -- 172.21.15.151:0/2431770026 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 27) ==== 3502+0+0 (secure 0 0 0) 0x7f7cd8057040 con 0x555e68cadaf0 2024-09-16T15:42:35.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.804+0000 7f7ce9819740 10 client.5562 fetch_fsmap finished waiting for FSMap version 27 2024-09-16T15:42:35.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.804+0000 7f7ce9819740 10 client.5562 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-16T15:42:35.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.804+0000 7f7ce9819740 20 client.5562 populate_metadata read hostname 'smithi151' 2024-09-16T15:42:35.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.804+0000 7f7ce9819740 1 --2- 172.21.15.151:0/2431770026 >> [v2:172.21.15.151:6833/1974411916,v1:172.21.15.151:6835/1974411916] conn(0x555e68d81400 0x555e68d817e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:35.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.804+0000 7f7ce9819740 4 client.5562 mds_command: new command op to 5400 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T15:42:35.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.804+0000 7f7ce9819740 1 -- 172.21.15.151:0/2431770026 --> [v2:172.21.15.151:6833/1974411916,v1:172.21.15.151:6835/1974411916] -- command(tid 0: {"prefix": "status"}) -- 0x555e68c23fd0 con 0x555e68d81400 2024-09-16T15:42:35.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.805+0000 7f7cdfe00640 1 --2- 172.21.15.151:0/2431770026 >> [v2:172.21.15.151:6833/1974411916,v1:172.21.15.151:6835/1974411916] conn(0x555e68d81400 0x555e68d817e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:35.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.805+0000 7f7cdfe00640 1 --2- 172.21.15.151:0/2431770026 >> [v2:172.21.15.151:6833/1974411916,v1:172.21.15.151:6835/1974411916] conn(0x555e68d81400 0x555e68d817e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39aca61c04e9168d server_cookie=c98fe2a1e493c812 in_seq=0 out_seq=0 2024-09-16T15:42:35.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.805+0000 7f7cccc00640 10 client.5562 ms_handle_connect on v2:172.21.15.151:6833/1974411916 2024-09-16T15:42:35.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.805+0000 7f7cccc00640 1 -- 172.21.15.151:0/2431770026 <== mds.0 v2:172.21.15.151:6833/1974411916 1 ==== command_reply(tid 0: 0 ) ==== 8+0+319 (crc 0 0 0) 0x555e68c23fd0 con 0x555e68d81400 2024-09-16T15:42:35.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.805+0000 7f7cccc00640 10 client.5562 handle_command_reply: tid=0 2024-09-16T15:42:35.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.805+0000 7f7ce9819740 1 client.5562 shutdown 2024-09-16T15:42:35.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.806+0000 7f7ce9819740 1 -- 172.21.15.151:0/2431770026 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e68cadaf0 msgr2=0x555e68d1fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:35.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.806+0000 7f7ce9819740 1 --2- 172.21.15.151:0/2431770026 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e68cadaf0 0x555e68d1fd00 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7cd8004af0 tx=0x7f7cd8044e60 comp rx=0 tx=0).stop 2024-09-16T15:42:35.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.806+0000 7f7ce9819740 1 -- 172.21.15.151:0/2431770026 shutdown_connections 2024-09-16T15:42:35.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.806+0000 7f7ce9819740 1 --2- 172.21.15.151:0/2431770026 >> [v2:172.21.15.151:6833/1974411916,v1:172.21.15.151:6835/1974411916] conn(0x555e68d81400 0x555e68d817e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:35.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.806+0000 7f7ce9819740 1 --2- 172.21.15.151:0/2431770026 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555e68cadaf0 0x555e68d1fd00 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:35.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.806+0000 7f7ce9819740 1 --2- 172.21.15.151:0/2431770026 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x555e68cf88b0 0x555e68d1f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:35.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.806+0000 7f7cccc00640 0 client.5562 ms_handle_reset on v2:172.21.15.151:6833/1974411916 2024-09-16T15:42:35.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.806+0000 7f7ce9819740 1 --2- 172.21.15.151:0/2431770026 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555e68cf5de0 0x555e68d80b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:35.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.806+0000 7f7ce9819740 1 -- 172.21.15.151:0/2431770026 >> 172.21.15.151:0/2431770026 conn(0x555e68cad000 msgr2=0x555e68cad3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:35.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.806+0000 7f7ce9819740 1 -- 172.21.15.151:0/2431770026 shutdown_connections 2024-09-16T15:42:35.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.806+0000 7f7ce9819740 1 -- 172.21.15.151:0/2431770026 wait complete. 2024-09-16T15:42:35.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.806+0000 7f7ce9819740 20 client.5562 trim_cache size 0 max 16384 2024-09-16T15:42:35.813 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-16T15:42:35.813 INFO:teuthology.orchestra.run.smithi151.stdout: "cluster_fsid": "1757be18-e41a-4446-bb4f-ee4ac81c39df", 2024-09-16T15:42:35.813 INFO:teuthology.orchestra.run.smithi151.stdout: "whoami": 0, 2024-09-16T15:42:35.813 INFO:teuthology.orchestra.run.smithi151.stdout: "id": 5400, 2024-09-16T15:42:35.813 INFO:teuthology.orchestra.run.smithi151.stdout: "want_state": "up:active", 2024-09-16T15:42:35.814 INFO:teuthology.orchestra.run.smithi151.stdout: "state": "up:active", 2024-09-16T15:42:35.814 INFO:teuthology.orchestra.run.smithi151.stdout: "fs_name": "cephfs", 2024-09-16T15:42:35.814 INFO:teuthology.orchestra.run.smithi151.stdout: "rank_uptime": 10.822063682, 2024-09-16T15:42:35.814 INFO:teuthology.orchestra.run.smithi151.stdout: "mdsmap_epoch": 27, 2024-09-16T15:42:35.814 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_epoch": 48, 2024-09-16T15:42:35.814 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_epoch_barrier": 48, 2024-09-16T15:42:35.814 INFO:teuthology.orchestra.run.smithi151.stdout: "uptime": 15.149311441 2024-09-16T15:42:35.814 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-16T15:42:35.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.807+0000 7f7ce6a00640 1 -- 172.21.15.151:0/349054885 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7cbc078bb0 msgr2=0x7f7cbc07b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:35.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.807+0000 7f7ce6a00640 1 --2- 172.21.15.151:0/349054885 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7cbc078bb0 0x7f7cbc07b070 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0070000 tx=0x7f7cc8009040 comp rx=0 tx=0).stop 2024-09-16T15:42:35.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.807+0000 7f7ce6a00640 1 -- 172.21.15.151:0/349054885 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ce0073fb0 msgr2=0x7f7ce00cf170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:35.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.807+0000 7f7ce6a00640 1 --2- 172.21.15.151:0/349054885 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ce0073fb0 0x7f7ce00cf170 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f7cd000d700 tx=0x7f7cd000dbc0 comp rx=0 tx=0).stop 2024-09-16T15:42:35.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.807+0000 7f7ce6a00640 1 -- 172.21.15.151:0/349054885 shutdown_connections 2024-09-16T15:42:35.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.807+0000 7f7ce6a00640 1 --2- 172.21.15.151:0/349054885 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7cbc078bb0 0x7f7cbc07b070 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:35.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.807+0000 7f7ce6a00640 1 --2- 172.21.15.151:0/349054885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ce015f5a0 0x7f7ce01444c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:35.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.807+0000 7f7ce6a00640 1 --2- 172.21.15.151:0/349054885 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7ce00ccf00 0x7f7ce0143f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:35.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.807+0000 7f7ce6a00640 1 --2- 172.21.15.151:0/349054885 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ce0073fb0 0x7f7ce00cf170 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:35.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.807+0000 7f7ce6a00640 1 -- 172.21.15.151:0/349054885 >> 172.21.15.151:0/349054885 conn(0x7f7ce00b1950 msgr2=0x7f7ce00cdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:35.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.807+0000 7f7ce6a00640 1 -- 172.21.15.151:0/349054885 shutdown_connections 2024-09-16T15:42:35.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.807+0000 7f7ce6a00640 1 -- 172.21.15.151:0/349054885 wait complete. 2024-09-16T15:42:35.824 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-16T15:42:35.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.981+0000 7f0be0200640 1 -- 172.21.15.151:0/3911339587 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0bd80ccd00 msgr2=0x7f0bd806f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:35.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.981+0000 7f0be0200640 1 --2- 172.21.15.151:0/3911339587 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0bd80ccd00 0x7f0bd806f850 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0bd806f3d0 tx=0x7f0bc402aef0 comp rx=0 tx=0).stop 2024-09-16T15:42:35.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.982+0000 7f0be0200640 1 -- 172.21.15.151:0/3911339587 shutdown_connections 2024-09-16T15:42:35.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.982+0000 7f0be0200640 1 --2- 172.21.15.151:0/3911339587 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0bd80ccd00 0x7f0bd806f850 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:35.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.982+0000 7f0be0200640 1 -- 172.21.15.151:0/3911339587 >> 172.21.15.151:0/3911339587 conn(0x7f0bd80b1950 msgr2=0x7f0bd80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:35.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.982+0000 7f0be0200640 1 -- 172.21.15.151:0/3911339587 shutdown_connections 2024-09-16T15:42:35.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.982+0000 7f0be0200640 1 -- 172.21.15.151:0/3911339587 wait complete. 2024-09-16T15:42:35.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.983+0000 7f0be0200640 1 Processor -- start 2024-09-16T15:42:35.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.983+0000 7f0be0200640 1 -- start start 2024-09-16T15:42:35.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.984+0000 7f0be0200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0bd806fd90 0x7f0bd815a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:35.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.984+0000 7f0be0200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0bd80ccd00 0x7f0bd80745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:35.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.984+0000 7f0be0200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0bd81438b0 0x7f0bd813a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:35.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.984+0000 7f0be0200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd8070aa0 con 0x7f0bd81438b0 2024-09-16T15:42:35.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.984+0000 7f0be0200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd8070920 con 0x7f0bd80ccd00 2024-09-16T15:42:35.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.984+0000 7f0be0200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f0bd80b4610 con 0x7f0bd806fd90 2024-09-16T15:42:35.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.985+0000 7f0bde200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0bd81438b0 0x7f0bd813a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:35.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.985+0000 7f0bde200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0bd81438b0 0x7f0bd813a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37086/0 (socket says 172.21.15.151:37086) 2024-09-16T15:42:35.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.985+0000 7f0bde200640 1 -- 172.21.15.151:0/1154626053 learned_addr learned my addr 172.21.15.151:0/1154626053 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:42:35.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.985+0000 7f0bdce00640 1 --2- 172.21.15.151:0/1154626053 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0bd80ccd00 0x7f0bd80745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:35.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.985+0000 7f0bdd800640 1 --2- 172.21.15.151:0/1154626053 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0bd806fd90 0x7f0bd815a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:35.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.985+0000 7f0bde200640 1 -- 172.21.15.151:0/1154626053 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0bd806fd90 msgr2=0x7f0bd815a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:35.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.985+0000 7f0bde200640 1 --2- 172.21.15.151:0/1154626053 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0bd806fd90 0x7f0bd815a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:35.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.985+0000 7f0bde200640 1 -- 172.21.15.151:0/1154626053 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0bd80ccd00 msgr2=0x7f0bd80745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:35.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.985+0000 7f0bde200640 1 --2- 172.21.15.151:0/1154626053 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0bd80ccd00 0x7f0bd80745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:35.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.985+0000 7f0bde200640 1 -- 172.21.15.151:0/1154626053 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bd80b1290 con 0x7f0bd81438b0 2024-09-16T15:42:35.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.985+0000 7f0bdce00640 1 --2- 172.21.15.151:0/1154626053 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0bd80ccd00 0x7f0bd80745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:42:35.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.985+0000 7f0bde200640 1 --2- 172.21.15.151:0/1154626053 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0bd81438b0 0x7f0bd813a210 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0bd000c430 tx=0x7f0bd000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:35.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.986+0000 7f0bd5600640 1 -- 172.21.15.151:0/1154626053 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd0017070 con 0x7f0bd81438b0 2024-09-16T15:42:35.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.986+0000 7f0bd5600640 1 -- 172.21.15.151:0/1154626053 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0bd00053c0 con 0x7f0bd81438b0 2024-09-16T15:42:35.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.986+0000 7f0bd5600640 1 -- 172.21.15.151:0/1154626053 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd0003850 con 0x7f0bd81438b0 2024-09-16T15:42:35.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.986+0000 7f0be0200640 1 -- 172.21.15.151:0/1154626053 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bd8141a10 con 0x7f0bd81438b0 2024-09-16T15:42:35.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.986+0000 7f0be0200640 1 -- 172.21.15.151:0/1154626053 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bd80729f0 con 0x7f0bd81438b0 2024-09-16T15:42:35.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.987+0000 7f0be0200640 1 -- 172.21.15.151:0/1154626053 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bd806f850 con 0x7f0bd81438b0 2024-09-16T15:42:35.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.988+0000 7f0bd5600640 1 -- 172.21.15.151:0/1154626053 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7f0bd001a070 con 0x7f0bd81438b0 2024-09-16T15:42:35.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.990+0000 7f0bd5600640 1 --2- 172.21.15.151:0/1154626053 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0bb8078bb0 0x7f0bb807b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:35.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.990+0000 7f0bd5600640 1 -- 172.21.15.151:0/1154626053 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f0bd009a370 con 0x7f0bd81438b0 2024-09-16T15:42:35.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.990+0000 7f0bdd800640 1 --2- 172.21.15.151:0/1154626053 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0bb8078bb0 0x7f0bb807b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:35.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.991+0000 7f0bdd800640 1 --2- 172.21.15.151:0/1154626053 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0bb8078bb0 0x7f0bb807b070 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc001ce0 tx=0x7f0bcc009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:35.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:35.994+0000 7f0bd5600640 1 -- 172.21.15.151:0/1154626053 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0bd80cdae0 con 0x7f0bd81438b0 2024-09-16T15:42:36.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:36.216+0000 7f0be0200640 1 -- 172.21.15.151:0/1154626053 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0bd80cd430 con 0x7f0bd81438b0 2024-09-16T15:42:36.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:36.216+0000 7f0bd5600640 1 -- 172.21.15.151:0/1154626053 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) ==== 53+0+73 (secure 0 0 0) 0x7f0bd0062480 con 0x7f0bd81438b0 2024-09-16T15:42:36.218 INFO:teuthology.orchestra.run.smithi151.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-16T15:42:36.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:36.219+0000 7f0be0200640 1 -- 172.21.15.151:0/1154626053 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0bb8078bb0 msgr2=0x7f0bb807b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:36.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:36.219+0000 7f0be0200640 1 --2- 172.21.15.151:0/1154626053 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0bb8078bb0 0x7f0bb807b070 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc001ce0 tx=0x7f0bcc009040 comp rx=0 tx=0).stop 2024-09-16T15:42:36.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:36.219+0000 7f0be0200640 1 -- 172.21.15.151:0/1154626053 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0bd81438b0 msgr2=0x7f0bd813a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:36.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:36.219+0000 7f0be0200640 1 --2- 172.21.15.151:0/1154626053 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0bd81438b0 0x7f0bd813a210 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0bd000c430 tx=0x7f0bd000c8f0 comp rx=0 tx=0).stop 2024-09-16T15:42:36.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:36.219+0000 7f0be0200640 1 -- 172.21.15.151:0/1154626053 shutdown_connections 2024-09-16T15:42:36.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:36.219+0000 7f0be0200640 1 --2- 172.21.15.151:0/1154626053 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0bb8078bb0 0x7f0bb807b070 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:36.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:36.219+0000 7f0be0200640 1 --2- 172.21.15.151:0/1154626053 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0bd80ccd00 0x7f0bd80745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:36.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:36.219+0000 7f0be0200640 1 --2- 172.21.15.151:0/1154626053 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0bd81438b0 0x7f0bd813a210 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:36.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:36.219+0000 7f0be0200640 1 --2- 172.21.15.151:0/1154626053 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0bd806fd90 0x7f0bd815a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:36.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:36.219+0000 7f0be0200640 1 -- 172.21.15.151:0/1154626053 >> 172.21.15.151:0/1154626053 conn(0x7f0bd80b1950 msgr2=0x7f0bd8142b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:36.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:36.219+0000 7f0be0200640 1 -- 172.21.15.151:0/1154626053 shutdown_connections 2024-09-16T15:42:36.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:36.219+0000 7f0be0200640 1 -- 172.21.15.151:0/1154626053 wait complete. 2024-09-16T15:42:36.236 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-16T15:42:36.236 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T15:42:36.236 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi151.front.sepia.ceph.com 2024-09-16T15:42:36.236 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-16T15:42:36.236 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-16T15:42:36.236 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-16T15:42:36.236 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-16T15:42:36.236 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T15:42:36.236 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T15:42:36.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:42:36.237 DEBUG:teuthology.orchestra.run.smithi151:> ip addr 2024-09-16T15:42:36.272 INFO:teuthology.orchestra.run.smithi151.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T15:42:36.272 INFO:teuthology.orchestra.run.smithi151.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T15:42:36.273 INFO:teuthology.orchestra.run.smithi151.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T15:42:36.273 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:42:36.273 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 ::1/128 scope host 2024-09-16T15:42:36.273 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:42:36.273 INFO:teuthology.orchestra.run.smithi151.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:42:36.273 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:da:6e:0c brd ff:ff:ff:ff:ff:ff 2024-09-16T15:42:36.273 INFO:teuthology.orchestra.run.smithi151.stdout: altname enp7s0f0 2024-09-16T15:42:36.273 INFO:teuthology.orchestra.run.smithi151.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:42:36.273 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:da:6e:0d brd ff:ff:ff:ff:ff:ff 2024-09-16T15:42:36.273 INFO:teuthology.orchestra.run.smithi151.stdout: altname enp7s0f1 2024-09-16T15:42:36.273 INFO:teuthology.orchestra.run.smithi151.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:42:36.273 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:88:6f:62 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:42:36.273 INFO:teuthology.orchestra.run.smithi151.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T15:42:36.273 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:88:6f:63 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:42:36.273 INFO:teuthology.orchestra.run.smithi151.stdout: inet 172.21.15.151/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-16T15:42:36.274 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft 42533sec preferred_lft 42533sec 2024-09-16T15:42:36.274 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 fe80::ec4:7aff:fe88:6f63/64 scope link noprefixroute 2024-09-16T15:42:36.274 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:42:36.274 INFO:teuthology.orchestra.run.smithi151.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-16T15:42:36.274 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 62:3f:3b:48:ad:f8 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:42:36.274 INFO:teuthology.orchestra.run.smithi151.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-16T15:42:36.274 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:42:36.274 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 fe80::40fa:7fff:fedf:2a00/64 scope link 2024-09-16T15:42:36.274 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:42:36.274 INFO:teuthology.orchestra.run.smithi151.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T15:42:36.274 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 62:3f:3b:48:ad:f8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T15:42:36.274 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 fe80::603f:3bff:fe48:adf8/64 scope link 2024-09-16T15:42:36.274 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:42:36.274 INFO:teuthology.orchestra.run.smithi151.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T15:42:36.274 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether b6:ac:fe:34:a7:66 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-16T15:42:36.275 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 fe80::b4ac:feff:fe34:a766/64 scope link 2024-09-16T15:42:36.275 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:42:36.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:42:36.275 DEBUG:teuthology.orchestra.run.smithi151:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T15:42:36.366 INFO:teuthology.orchestra.run.smithi151.stdout:1 2024-09-16T15:42:36.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:42:36.369 DEBUG:teuthology.orchestra.run.smithi151:> route 2024-09-16T15:42:36.405 INFO:teuthology.orchestra.run.smithi151.stdout:Kernel IP routing table 2024-09-16T15:42:36.405 INFO:teuthology.orchestra.run.smithi151.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T15:42:36.405 INFO:teuthology.orchestra.run.smithi151.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-16T15:42:36.405 INFO:teuthology.orchestra.run.smithi151.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-16T15:42:36.405 INFO:teuthology.orchestra.run.smithi151.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T15:42:36.405 DEBUG:teuthology.orchestra.run.smithi151:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T15:42:36.406 DEBUG:teuthology.orchestra.run.smithi151:> set -e 2024-09-16T15:42:36.406 DEBUG:teuthology.orchestra.run.smithi151:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-16T15:42:36.406 DEBUG:teuthology.orchestra.run.smithi151:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-16T15:42:36.406 DEBUG:teuthology.orchestra.run.smithi151:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-16T15:42:36.406 DEBUG:teuthology.orchestra.run.smithi151:> ') 2024-09-16T15:42:36.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:42:36.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:42:36.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:42:36.602 DEBUG:teuthology.orchestra.run.smithi151:> ip netns list 2024-09-16T15:42:36.637 INFO:teuthology.orchestra.run.smithi151.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-16T15:42:36.637 INFO:teuthology.orchestra.run.smithi151.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-16T15:42:36.638 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T15:42:36.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:42:36.638 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip link set brx.0 up 2024-09-16T15:42:36.734 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:42:36.734 DEBUG:teuthology.orchestra.run.smithi151:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-16T15:42:36.769 INFO:teuthology.orchestra.run.smithi151.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-16T15:42:36.769 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:42:36.769 DEBUG:teuthology.orchestra.run.smithi151:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-16T15:42:36.845 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-16T15:42:36.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:42:36.845 DEBUG:teuthology.orchestra.run.smithi151:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-16T15:42:36.974 INFO:teuthology.orchestra.run.smithi151.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-16T15:42:36.974 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-16T15:42:36.974 INFO:teuthology.orchestra.run.smithi151.stdout:invalid new device string format 2024-09-16T15:42:36.990 INFO:teuthology.orchestra.run.smithi151.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-16T15:42:36.990 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-16T15:42:36.990 INFO:teuthology.orchestra.run.smithi151.stdout:invalid new device string format 2024-09-16T15:42:36.990 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: resolved to: "172.21.15.151:3300,172.21.15.167:3300,172.21.15.167:3301" 2024-09-16T15:42:36.990 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-16T15:42:37.013 INFO:teuthology.orchestra.run.smithi151.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-16T15:42:37.013 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-16T15:42:37.013 INFO:teuthology.orchestra.run.smithi151.stdout:invalid new device string format 2024-09-16T15:42:37.013 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: resolved to: "172.21.15.151:3300,172.21.15.167:3300,172.21.15.167:3301" 2024-09-16T15:42:37.013 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-16T15:42:37.014 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: resolved to: "172.21.15.151:6789,172.21.15.167:6789,172.21.15.167:6790" 2024-09-16T15:42:37.014 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.151:6789,172.21.15.167:6789,172.21.15.167:6790:/ 2024-09-16T15:42:37.014 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=1757be18-e41a-4446-bb4f-ee4ac81c39df" will pass to kernel 2024-09-16T15:42:37.016 INFO:teuthology.orchestra.run.smithi151.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-16T15:42:37.016 INFO:teuthology.orchestra.run.smithi151.stdout: You just mounted a file system that supports labels which does not 2024-09-16T15:42:37.016 INFO:teuthology.orchestra.run.smithi151.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-16T15:42:37.016 INFO:teuthology.orchestra.run.smithi151.stdout: applications will generate AVC messages and not be allowed access to 2024-09-16T15:42:37.016 INFO:teuthology.orchestra.run.smithi151.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-16T15:42:37.019 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-16T15:42:37.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:42:37.019 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-16T15:42:37.075 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T15:42:37.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:37 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:42:37.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:42:37.170 INFO:teuthology.orchestra.run.smithi151.stdout:client5410 2024-09-16T15:42:37.170 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:42:37.170 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:42:37.206 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-16T15:42:37.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:37.405+0000 7fbb30c00640 1 -- 172.21.15.151:0/508019503 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb10004020 msgr2=0x7fbb10004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:37.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:37.405+0000 7fbb30c00640 1 --2- 172.21.15.151:0/508019503 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb10004020 0x7fbb10004800 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fbb18009b20 tx=0x7fbb10005830 comp rx=0 tx=0).stop 2024-09-16T15:42:37.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:37.405+0000 7fbb2a800640 1 -- 172.21.15.151:0/508019503 reap_dead start 2024-09-16T15:42:37.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:37.405+0000 7fbb30c00640 1 -- 172.21.15.151:0/508019503 shutdown_connections 2024-09-16T15:42:37.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:37.405+0000 7fbb30c00640 1 -- 172.21.15.151:0/508019503 >> 172.21.15.151:0/508019503 conn(0x7fbb2c0b18e0 msgr2=0x7fbb2c0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:37.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:37.405+0000 7fbb30c00640 1 -- 172.21.15.151:0/508019503 shutdown_connections 2024-09-16T15:42:37.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:37.406+0000 7fbb30c00640 1 -- 172.21.15.151:0/508019503 wait complete. 2024-09-16T15:42:37.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:37.407+0000 7fbb30c00640 1 Processor -- start 2024-09-16T15:42:37.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:37.407+0000 7fbb30c00640 1 -- start start 2024-09-16T15:42:37.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:37.408+0000 7fbb30c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb10004020 0x7fbb2c1e34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:37.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:37.408+0000 7fbb30c00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbb2c09cc50 0x7fbb2c148260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:37.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:37.408+0000 7fbb30c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb2c1487a0 0x7fbb2c1599e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:37.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:37.408+0000 7fbb30c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbb2c0cd260 con 0x7fbb2c1487a0 2024-09-16T15:42:37.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:37.408+0000 7fbb30c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbb2c0cd0e0 con 0x7fbb10004020 2024-09-16T15:42:37.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:37.408+0000 7fbb30c00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fbb2c0cd3e0 con 0x7fbb2c09cc50 2024-09-16T15:42:37.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:37.408+0000 7fbb2a800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb2c1487a0 0x7fbb2c1599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:37.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:37.408+0000 7fbb2a800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb2c1487a0 0x7fbb2c1599e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37096/0 (socket says 172.21.15.151:37096) 2024-09-16T15:42:37.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:37.408+0000 7fbb2a800640 1 -- 172.21.15.151:0/4201419464 learned_addr learned my addr 172.21.15.151:0/4201419464 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:42:37.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:37.409+0000 7fbb2a800640 1 -- 172.21.15.151:0/4201419464 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbb2c09cc50 msgr2=0x7fbb2c148260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:37.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:37.409+0000 7fbb29400640 1 --2- 172.21.15.151:0/4201419464 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbb2c09cc50 0x7fbb2c148260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:37.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:37.409+0000 7fbb29e00640 1 --2- 172.21.15.151:0/4201419464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb10004020 0x7fbb2c1e34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:37.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:37.409+0000 7fbb2a800640 1 --2- 172.21.15.151:0/4201419464 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbb2c09cc50 0x7fbb2c148260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:37.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:37.409+0000 7fbb2a800640 1 -- 172.21.15.151:0/4201419464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb10004020 msgr2=0x7fbb2c1e34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:37.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:37.409+0000 7fbb2a800640 1 --2- 172.21.15.151:0/4201419464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb10004020 0x7fbb2c1e34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:37.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:37.409+0000 7fbb2a800640 1 -- 172.21.15.151:0/4201419464 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb2c1f5b10 con 0x7fbb2c1487a0 2024-09-16T15:42:37.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:37.409+0000 7fbb29400640 1 --2- 172.21.15.151:0/4201419464 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbb2c09cc50 0x7fbb2c148260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:42:37.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:37.409+0000 7fbb29e00640 1 --2- 172.21.15.151:0/4201419464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb10004020 0x7fbb2c1e34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:42:37.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:37.409+0000 7fbb2a800640 1 --2- 172.21.15.151:0/4201419464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb2c1487a0 0x7fbb2c1599e0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c070850 tx=0x7fbb2400fe60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:37.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:37.409+0000 7fbb22000640 1 -- 172.21.15.151:0/4201419464 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb2400dce0 con 0x7fbb2c1487a0 2024-09-16T15:42:37.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:37.409+0000 7fbb22000640 1 -- 172.21.15.151:0/4201419464 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbb24011440 con 0x7fbb2c1487a0 2024-09-16T15:42:37.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:37.409+0000 7fbb22000640 1 -- 172.21.15.151:0/4201419464 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb24011890 con 0x7fbb2c1487a0 2024-09-16T15:42:37.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:37.409+0000 7fbb30c00640 1 -- 172.21.15.151:0/4201419464 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb2c1f5df0 con 0x7fbb2c1487a0 2024-09-16T15:42:37.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:37.410+0000 7fbb30c00640 1 -- 172.21.15.151:0/4201419464 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb2c1f63b0 con 0x7fbb2c1487a0 2024-09-16T15:42:37.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:37.410+0000 7fbb30c00640 1 -- 172.21.15.151:0/4201419464 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb2c069360 con 0x7fbb2c1487a0 2024-09-16T15:42:37.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:37.412+0000 7fbb22000640 1 -- 172.21.15.151:0/4201419464 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7fbb24019020 con 0x7fbb2c1487a0 2024-09-16T15:42:37.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:37.412+0000 7fbb22000640 1 --2- 172.21.15.151:0/4201419464 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbb0c078bb0 0x7fbb0c07b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:37.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:37.413+0000 7fbb22000640 1 -- 172.21.15.151:0/4201419464 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7fbb2409b940 con 0x7fbb2c1487a0 2024-09-16T15:42:37.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:37.413+0000 7fbb29e00640 1 --2- 172.21.15.151:0/4201419464 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbb0c078bb0 0x7fbb0c07b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:37.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:37.414+0000 7fbb29e00640 1 --2- 172.21.15.151:0/4201419464 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbb0c078bb0 0x7fbb0c07b070 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c001eb0 tx=0x7fbb1c000ed0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:37.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:37.417+0000 7fbb22000640 1 -- 172.21.15.151:0/4201419464 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fbb24063950 con 0x7fbb2c1487a0 2024-09-16T15:42:37.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:37.640+0000 7fbb30c00640 1 -- 172.21.15.151:0/4201419464 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fbb2c06eb80 con 0x7fbb2c1487a0 2024-09-16T15:42:38.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.055+0000 7fbb22000640 1 -- 172.21.15.151:0/4201419464 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v28) ==== 136+0+0 (secure 0 0 0) 0x7fbb24063250 con 0x7fbb2c1487a0 2024-09-16T15:42:38.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.058+0000 7fbb30c00640 1 -- 172.21.15.151:0/4201419464 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbb0c078bb0 msgr2=0x7fbb0c07b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:38.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.058+0000 7fbb30c00640 1 --2- 172.21.15.151:0/4201419464 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbb0c078bb0 0x7fbb0c07b070 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c001eb0 tx=0x7fbb1c000ed0 comp rx=0 tx=0).stop 2024-09-16T15:42:38.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.058+0000 7fbb30c00640 1 -- 172.21.15.151:0/4201419464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb2c1487a0 msgr2=0x7fbb2c1599e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:38.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.058+0000 7fbb30c00640 1 --2- 172.21.15.151:0/4201419464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb2c1487a0 0x7fbb2c1599e0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c070850 tx=0x7fbb2400fe60 comp rx=0 tx=0).stop 2024-09-16T15:42:38.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.058+0000 7fbb30c00640 1 -- 172.21.15.151:0/4201419464 shutdown_connections 2024-09-16T15:42:38.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.058+0000 7fbb30c00640 1 --2- 172.21.15.151:0/4201419464 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbb0c078bb0 0x7fbb0c07b070 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:38.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.058+0000 7fbb30c00640 1 --2- 172.21.15.151:0/4201419464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb2c1487a0 0x7fbb2c1599e0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:38.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.058+0000 7fbb30c00640 1 --2- 172.21.15.151:0/4201419464 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbb2c09cc50 0x7fbb2c148260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:38.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.058+0000 7fbb30c00640 1 --2- 172.21.15.151:0/4201419464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb10004020 0x7fbb2c1e34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:38.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.058+0000 7fbb30c00640 1 -- 172.21.15.151:0/4201419464 >> 172.21.15.151:0/4201419464 conn(0x7fbb2c0b18e0 msgr2=0x7fbb2c0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:38.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.058+0000 7fbb30c00640 1 -- 172.21.15.151:0/4201419464 shutdown_connections 2024-09-16T15:42:38.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.058+0000 7fbb30c00640 1 -- 172.21.15.151:0/4201419464 wait complete. 2024-09-16T15:42:38.075 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-16T15:42:38.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.232+0000 7f7db4600640 1 -- 172.21.15.151:0/2960228390 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7dac161550 msgr2=0x7f7dac06ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:38.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.232+0000 7f7db4600640 1 --2- 172.21.15.151:0/2960228390 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7dac161550 0x7f7dac06ef50 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f7da0005fd0 tx=0x7f7da002a540 comp rx=0 tx=0).stop 2024-09-16T15:42:38.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.232+0000 7f7db4600640 1 -- 172.21.15.151:0/2960228390 shutdown_connections 2024-09-16T15:42:38.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.232+0000 7f7db4600640 1 --2- 172.21.15.151:0/2960228390 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7dac072df0 0x7f7dac0732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:38.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.232+0000 7f7db4600640 1 --2- 172.21.15.151:0/2960228390 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7dac161550 0x7f7dac06ef50 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:38.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.232+0000 7f7db4600640 1 -- 172.21.15.151:0/2960228390 >> 172.21.15.151:0/2960228390 conn(0x7f7dac0b1950 msgr2=0x7f7dac0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:38.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.232+0000 7f7db4600640 1 -- 172.21.15.151:0/2960228390 shutdown_connections 2024-09-16T15:42:38.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.232+0000 7f7db4600640 1 -- 172.21.15.151:0/2960228390 wait complete. 2024-09-16T15:42:38.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.234+0000 7f7db4600640 1 Processor -- start 2024-09-16T15:42:38.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.234+0000 7f7db4600640 1 -- start start 2024-09-16T15:42:38.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.235+0000 7f7db4600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7dac072df0 0x7f7dac1f8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:38.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.235+0000 7f7db4600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7dac0af3d0 0x7f7dac0afb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:38.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.235+0000 7f7db4600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7dac161550 0x7f7dac0b0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:38.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.235+0000 7f7db4600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7dac0737e0 con 0x7f7dac161550 2024-09-16T15:42:38.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.235+0000 7f7db4600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7dac06fdb0 con 0x7f7dac072df0 2024-09-16T15:42:38.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.235+0000 7f7db4600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f7dac073960 con 0x7f7dac0af3d0 2024-09-16T15:42:38.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.235+0000 7f7db2600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7dac161550 0x7f7dac0b0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:38.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.235+0000 7f7db2600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7dac161550 0x7f7dac0b0050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37122/0 (socket says 172.21.15.151:37122) 2024-09-16T15:42:38.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.235+0000 7f7db2600640 1 -- 172.21.15.151:0/2426696273 learned_addr learned my addr 172.21.15.151:0/2426696273 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:42:38.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.236+0000 7f7db1c00640 1 --2- 172.21.15.151:0/2426696273 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7dac072df0 0x7f7dac1f8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:38.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.236+0000 7f7db1200640 1 --2- 172.21.15.151:0/2426696273 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7dac0af3d0 0x7f7dac0afb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:38.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.236+0000 7f7db2600640 1 -- 172.21.15.151:0/2426696273 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7dac0af3d0 msgr2=0x7f7dac0afb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:38.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.236+0000 7f7db2600640 1 --2- 172.21.15.151:0/2426696273 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7dac0af3d0 0x7f7dac0afb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:38.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.236+0000 7f7db2600640 1 -- 172.21.15.151:0/2426696273 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7dac072df0 msgr2=0x7f7dac1f8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:38.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.236+0000 7f7db2600640 1 --2- 172.21.15.151:0/2426696273 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7dac072df0 0x7f7dac1f8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:38.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.236+0000 7f7db2600640 1 -- 172.21.15.151:0/2426696273 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dac1f4e70 con 0x7f7dac161550 2024-09-16T15:42:38.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.236+0000 7f7db2600640 1 --2- 172.21.15.151:0/2426696273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7dac161550 0x7f7dac0b0050 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f7da400eaa0 tx=0x7f7da400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:38.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.236+0000 7f7da9600640 1 -- 172.21.15.151:0/2426696273 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da400cdb0 con 0x7f7dac161550 2024-09-16T15:42:38.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.236+0000 7f7da9600640 1 -- 172.21.15.151:0/2426696273 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7da4004c10 con 0x7f7dac161550 2024-09-16T15:42:38.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.236+0000 7f7db4600640 1 -- 172.21.15.151:0/2426696273 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dac1f5bf0 con 0x7f7dac161550 2024-09-16T15:42:38.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.236+0000 7f7da9600640 1 -- 172.21.15.151:0/2426696273 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da4005020 con 0x7f7dac161550 2024-09-16T15:42:38.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.237+0000 7f7db4600640 1 -- 172.21.15.151:0/2426696273 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dac1f53f0 con 0x7f7dac161550 2024-09-16T15:42:38.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.238+0000 7f7db4600640 1 -- 172.21.15.151:0/2426696273 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dac073e50 con 0x7f7dac161550 2024-09-16T15:42:38.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.239+0000 7f7da9600640 1 -- 172.21.15.151:0/2426696273 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7f7da40051c0 con 0x7f7dac161550 2024-09-16T15:42:38.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.240+0000 7f7da9600640 1 --2- 172.21.15.151:0/2426696273 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7d8c078bb0 0x7f7d8c07b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:38.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.240+0000 7f7da9600640 1 -- 172.21.15.151:0/2426696273 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f7da409b930 con 0x7f7dac161550 2024-09-16T15:42:38.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.240+0000 7f7db1c00640 1 --2- 172.21.15.151:0/2426696273 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7d8c078bb0 0x7f7d8c07b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:38.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.241+0000 7f7db1c00640 1 --2- 172.21.15.151:0/2426696273 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7d8c078bb0 0x7f7d8c07b070 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7da0033000 tx=0x7f7da0004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:38.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.244+0000 7f7da9600640 1 -- 172.21.15.151:0/2426696273 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7da4016030 con 0x7f7dac161550 2024-09-16T15:42:38.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.452+0000 7f7db4600640 1 -- 172.21.15.151:0/2426696273 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f7dac0adb40 con 0x7f7d8c078bb0 2024-09-16T15:42:38.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.454+0000 7f7da9600640 1 -- 172.21.15.151:0/2426696273 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f7dac0adb40 con 0x7f7d8c078bb0 2024-09-16T15:42:38.455 INFO:teuthology.orchestra.run.smithi151.stdout:[ 2024-09-16T15:42:38.455 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:42:38.455 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "cephfs" 2024-09-16T15:42:38.455 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:42:38.456 INFO:teuthology.orchestra.run.smithi151.stdout:] 2024-09-16T15:42:38.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.456+0000 7f7db4600640 1 -- 172.21.15.151:0/2426696273 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7d8c078bb0 msgr2=0x7f7d8c07b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:38.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.456+0000 7f7db4600640 1 --2- 172.21.15.151:0/2426696273 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7d8c078bb0 0x7f7d8c07b070 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7da0033000 tx=0x7f7da0004730 comp rx=0 tx=0).stop 2024-09-16T15:42:38.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.456+0000 7f7db4600640 1 -- 172.21.15.151:0/2426696273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7dac161550 msgr2=0x7f7dac0b0050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:38.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.456+0000 7f7db4600640 1 --2- 172.21.15.151:0/2426696273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7dac161550 0x7f7dac0b0050 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f7da400eaa0 tx=0x7f7da400ef60 comp rx=0 tx=0).stop 2024-09-16T15:42:38.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.456+0000 7f7db4600640 1 -- 172.21.15.151:0/2426696273 shutdown_connections 2024-09-16T15:42:38.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.456+0000 7f7db4600640 1 --2- 172.21.15.151:0/2426696273 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7d8c078bb0 0x7f7d8c07b070 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:38.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.456+0000 7f7db4600640 1 --2- 172.21.15.151:0/2426696273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7dac161550 0x7f7dac0b0050 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:38.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.456+0000 7f7db4600640 1 --2- 172.21.15.151:0/2426696273 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7dac0af3d0 0x7f7dac0afb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:38.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.456+0000 7f7db4600640 1 --2- 172.21.15.151:0/2426696273 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7dac072df0 0x7f7dac1f8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:38.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.456+0000 7f7db4600640 1 -- 172.21.15.151:0/2426696273 >> 172.21.15.151:0/2426696273 conn(0x7f7dac0b1950 msgr2=0x7f7dac0ae060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:38.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.456+0000 7f7db4600640 1 -- 172.21.15.151:0/2426696273 shutdown_connections 2024-09-16T15:42:38.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.456+0000 7f7db4600640 1 -- 172.21.15.151:0/2426696273 wait complete. 2024-09-16T15:42:38.473 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-16T15:42:38.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.633+0000 7fa77fe00640 1 -- 172.21.15.151:0/1585494915 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa7780cccd0 msgr2=0x7fa778073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:38.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.633+0000 7fa77fe00640 1 --2- 172.21.15.151:0/1585494915 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa7780cccd0 0x7fa778073440 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa764005fd0 tx=0x7fa76402a4a0 comp rx=0 tx=0).stop 2024-09-16T15:42:38.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.633+0000 7fa77fe00640 1 -- 172.21.15.151:0/1585494915 shutdown_connections 2024-09-16T15:42:38.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.633+0000 7fa77fe00640 1 --2- 172.21.15.151:0/1585494915 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa77806efa0 0x7fa7780cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:38.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.633+0000 7fa77fe00640 1 --2- 172.21.15.151:0/1585494915 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa7780cccd0 0x7fa778073440 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:38.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.633+0000 7fa77fe00640 1 -- 172.21.15.151:0/1585494915 >> 172.21.15.151:0/1585494915 conn(0x7fa7780b1840 msgr2=0x7fa7780b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:38.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.634+0000 7fa77fe00640 1 -- 172.21.15.151:0/1585494915 shutdown_connections 2024-09-16T15:42:38.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.634+0000 7fa77fe00640 1 -- 172.21.15.151:0/1585494915 wait complete. 2024-09-16T15:42:38.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.635+0000 7fa77fe00640 1 Processor -- start 2024-09-16T15:42:38.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.635+0000 7fa77fe00640 1 -- start start 2024-09-16T15:42:38.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.636+0000 7fa77fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa77806efa0 0x7fa778140e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:38.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.636+0000 7fa77fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa7780cccd0 0x7fa7780c4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:38.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.636+0000 7fa77fe00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa778145750 0x7fa77813fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:38.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.636+0000 7fa77fe00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa7780ce480 con 0x7fa7780cccd0 2024-09-16T15:42:38.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.636+0000 7fa77fe00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa7780ce300 con 0x7fa77806efa0 2024-09-16T15:42:38.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.636+0000 7fa77fe00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fa7780ce600 con 0x7fa778145750 2024-09-16T15:42:38.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.636+0000 7fa77ca00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa7780cccd0 0x7fa7780c4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:38.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.637+0000 7fa77ca00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa7780cccd0 0x7fa7780c4ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37150/0 (socket says 172.21.15.151:37150) 2024-09-16T15:42:38.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.637+0000 7fa77ca00640 1 -- 172.21.15.151:0/1236845732 learned_addr learned my addr 172.21.15.151:0/1236845732 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:42:38.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.637+0000 7fa77d400640 1 --2- 172.21.15.151:0/1236845732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa77806efa0 0x7fa778140e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:38.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.637+0000 7fa77de00640 1 --2- 172.21.15.151:0/1236845732 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa778145750 0x7fa77813fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:38.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.637+0000 7fa77ca00640 1 -- 172.21.15.151:0/1236845732 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa778145750 msgr2=0x7fa77813fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:38.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.637+0000 7fa77ca00640 1 --2- 172.21.15.151:0/1236845732 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa778145750 0x7fa77813fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:38.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.637+0000 7fa77ca00640 1 -- 172.21.15.151:0/1236845732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa77806efa0 msgr2=0x7fa778140e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:38.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.637+0000 7fa77ca00640 1 --2- 172.21.15.151:0/1236845732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa77806efa0 0x7fa778140e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:38.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.637+0000 7fa77ca00640 1 -- 172.21.15.151:0/1236845732 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7780c5500 con 0x7fa7780cccd0 2024-09-16T15:42:38.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.637+0000 7fa77d400640 1 --2- 172.21.15.151:0/1236845732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa77806efa0 0x7fa778140e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:42:38.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.637+0000 7fa77ca00640 1 --2- 172.21.15.151:0/1236845732 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa7780cccd0 0x7fa7780c4ab0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fa764009850 tx=0x7fa7640039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:38.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.638+0000 7fa775600640 1 -- 172.21.15.151:0/1236845732 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa76403f070 con 0x7fa7780cccd0 2024-09-16T15:42:38.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.638+0000 7fa775600640 1 -- 172.21.15.151:0/1236845732 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa764004890 con 0x7fa7780cccd0 2024-09-16T15:42:38.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.638+0000 7fa775600640 1 -- 172.21.15.151:0/1236845732 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa764035030 con 0x7fa7780cccd0 2024-09-16T15:42:38.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.638+0000 7fa77fe00640 1 -- 172.21.15.151:0/1236845732 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7780c57e0 con 0x7fa7780cccd0 2024-09-16T15:42:38.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.638+0000 7fa77fe00640 1 -- 172.21.15.151:0/1236845732 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7780c6080 con 0x7fa7780cccd0 2024-09-16T15:42:38.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.639+0000 7fa77fe00640 1 -- 172.21.15.151:0/1236845732 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7780c5bd0 con 0x7fa7780cccd0 2024-09-16T15:42:38.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.640+0000 7fa775600640 1 -- 172.21.15.151:0/1236845732 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7fa764042070 con 0x7fa7780cccd0 2024-09-16T15:42:38.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.641+0000 7fa775600640 1 --2- 172.21.15.151:0/1236845732 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa758078be0 0x7fa75807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:38.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.641+0000 7fa775600640 1 -- 172.21.15.151:0/1236845732 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7fa7640bbdd0 con 0x7fa7780cccd0 2024-09-16T15:42:38.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.641+0000 7fa77d400640 1 --2- 172.21.15.151:0/1236845732 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa758078be0 0x7fa75807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:38.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.642+0000 7fa77d400640 1 --2- 172.21.15.151:0/1236845732 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa758078be0 0x7fa75807b0a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa76c00b820 tx=0x7fa76c00d040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:38.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.645+0000 7fa775600640 1 -- 172.21.15.151:0/1236845732 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa764083920 con 0x7fa7780cccd0 2024-09-16T15:42:38.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.829+0000 7fa77fe00640 1 -- 172.21.15.151:0/1236845732 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fa7780cdbb0 con 0x7fa7780cccd0 2024-09-16T15:42:38.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.830+0000 7fa775600640 1 -- 172.21.15.151:0/1236845732 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v31)=0 v31) ==== 122+0+0 (secure 0 0 0) 0x7fa764088350 con 0x7fa7780cccd0 2024-09-16T15:42:38.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.832+0000 7fa77fe00640 1 -- 172.21.15.151:0/1236845732 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa758078be0 msgr2=0x7fa75807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:38.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.832+0000 7fa77fe00640 1 --2- 172.21.15.151:0/1236845732 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa758078be0 0x7fa75807b0a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa76c00b820 tx=0x7fa76c00d040 comp rx=0 tx=0).stop 2024-09-16T15:42:38.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.832+0000 7fa77fe00640 1 -- 172.21.15.151:0/1236845732 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa7780cccd0 msgr2=0x7fa7780c4ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:38.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.832+0000 7fa77fe00640 1 --2- 172.21.15.151:0/1236845732 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa7780cccd0 0x7fa7780c4ab0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fa764009850 tx=0x7fa7640039e0 comp rx=0 tx=0).stop 2024-09-16T15:42:38.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.832+0000 7fa77fe00640 1 -- 172.21.15.151:0/1236845732 shutdown_connections 2024-09-16T15:42:38.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.832+0000 7fa77fe00640 1 --2- 172.21.15.151:0/1236845732 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa758078be0 0x7fa75807b0a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:38.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.832+0000 7fa77fe00640 1 --2- 172.21.15.151:0/1236845732 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa778145750 0x7fa77813fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:38.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.832+0000 7fa77fe00640 1 --2- 172.21.15.151:0/1236845732 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa7780cccd0 0x7fa7780c4ab0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:38.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.832+0000 7fa77fe00640 1 --2- 172.21.15.151:0/1236845732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa77806efa0 0x7fa778140e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:38.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.832+0000 7fa77fe00640 1 -- 172.21.15.151:0/1236845732 >> 172.21.15.151:0/1236845732 conn(0x7fa7780b1840 msgr2=0x7fa7780b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:38.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.832+0000 7fa77fe00640 1 -- 172.21.15.151:0/1236845732 shutdown_connections 2024-09-16T15:42:38.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:38.832+0000 7fa77fe00640 1 -- 172.21.15.151:0/1236845732 wait complete. 2024-09-16T15:42:38.851 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mgr mgr/volumes/max_concurrent_clones 2024-09-16T15:42:39.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.007+0000 7fe94f600640 1 -- 172.21.15.151:0/1294873345 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe948145750 msgr2=0x7fe948073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:39.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.007+0000 7fe94f600640 1 --2- 172.21.15.151:0/1294873345 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe948145750 0x7fe948073440 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe938005fd0 tx=0x7fe93802a540 comp rx=0 tx=0).stop 2024-09-16T15:42:39.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.007+0000 7fe94f600640 1 -- 172.21.15.151:0/1294873345 shutdown_connections 2024-09-16T15:42:39.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.007+0000 7fe94f600640 1 --2- 172.21.15.151:0/1294873345 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe948145750 0x7fe948073440 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:39.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.007+0000 7fe94f600640 1 -- 172.21.15.151:0/1294873345 >> 172.21.15.151:0/1294873345 conn(0x7fe9480b1840 msgr2=0x7fe9480b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:39.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.007+0000 7fe94f600640 1 -- 172.21.15.151:0/1294873345 shutdown_connections 2024-09-16T15:42:39.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.007+0000 7fe94f600640 1 -- 172.21.15.151:0/1294873345 wait complete. 2024-09-16T15:42:39.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.009+0000 7fe94f600640 1 Processor -- start 2024-09-16T15:42:39.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.009+0000 7fe94f600640 1 -- start start 2024-09-16T15:42:39.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.010+0000 7fe94f600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe94806efa0 0x7fe948135340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:39.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.010+0000 7fe94f600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe9480cccd0 0x7fe9480ca860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:39.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.010+0000 7fe94f600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe948145750 0x7fe9481ef970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:39.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.010+0000 7fe94f600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe9480ce6a0 con 0x7fe94806efa0 2024-09-16T15:42:39.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.010+0000 7fe94f600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe9480ce520 con 0x7fe948145750 2024-09-16T15:42:39.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.010+0000 7fe94f600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fe9480ce820 con 0x7fe9480cccd0 2024-09-16T15:42:39.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.010+0000 7fe94cc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe94806efa0 0x7fe948135340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:39.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.010+0000 7fe94cc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe94806efa0 0x7fe948135340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37162/0 (socket says 172.21.15.151:37162) 2024-09-16T15:42:39.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.010+0000 7fe94cc00640 1 -- 172.21.15.151:0/1158083758 learned_addr learned my addr 172.21.15.151:0/1158083758 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:42:39.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.010+0000 7fe94d600640 1 --2- 172.21.15.151:0/1158083758 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe948145750 0x7fe9481ef970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:39.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.010+0000 7fe947e00640 1 --2- 172.21.15.151:0/1158083758 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe9480cccd0 0x7fe9480ca860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:39.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.011+0000 7fe94cc00640 1 -- 172.21.15.151:0/1158083758 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe9480cccd0 msgr2=0x7fe9480ca860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:39.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.011+0000 7fe94cc00640 1 --2- 172.21.15.151:0/1158083758 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe9480cccd0 0x7fe9480ca860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:39.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.011+0000 7fe94cc00640 1 -- 172.21.15.151:0/1158083758 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe948145750 msgr2=0x7fe9481ef970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:39.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.011+0000 7fe94cc00640 1 --2- 172.21.15.151:0/1158083758 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe948145750 0x7fe9481ef970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:39.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.011+0000 7fe94cc00640 1 -- 172.21.15.151:0/1158083758 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe948138330 con 0x7fe94806efa0 2024-09-16T15:42:39.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.011+0000 7fe947e00640 1 --2- 172.21.15.151:0/1158083758 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe9480cccd0 0x7fe9480ca860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:42:39.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.011+0000 7fe94cc00640 1 --2- 172.21.15.151:0/1158083758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe94806efa0 0x7fe948135340 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe938034040 tx=0x7fe9380039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:39.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.011+0000 7fe944c00640 1 -- 172.21.15.151:0/1158083758 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93803e030 con 0x7fe94806efa0 2024-09-16T15:42:39.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.011+0000 7fe944c00640 1 -- 172.21.15.151:0/1158083758 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe9380047a0 con 0x7fe94806efa0 2024-09-16T15:42:39.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.011+0000 7fe944c00640 1 -- 172.21.15.151:0/1158083758 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe938035040 con 0x7fe94806efa0 2024-09-16T15:42:39.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.011+0000 7fe94f600640 1 -- 172.21.15.151:0/1158083758 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9480c5070 con 0x7fe94806efa0 2024-09-16T15:42:39.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.012+0000 7fe94f600640 1 -- 172.21.15.151:0/1158083758 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9480c5630 con 0x7fe94806efa0 2024-09-16T15:42:39.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.013+0000 7fe94f600640 1 -- 172.21.15.151:0/1158083758 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9480cecf0 con 0x7fe94806efa0 2024-09-16T15:42:39.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.014+0000 7fe944c00640 1 -- 172.21.15.151:0/1158083758 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7fe938041030 con 0x7fe94806efa0 2024-09-16T15:42:39.017 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.015+0000 7fe944c00640 1 --2- 172.21.15.151:0/1158083758 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe928078bb0 0x7fe92807b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:39.017 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.015+0000 7fe944c00640 1 -- 172.21.15.151:0/1158083758 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7fe9380bb5e0 con 0x7fe94806efa0 2024-09-16T15:42:39.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.016+0000 7fe947e00640 1 --2- 172.21.15.151:0/1158083758 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe928078bb0 0x7fe92807b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:39.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.016+0000 7fe947e00640 1 --2- 172.21.15.151:0/1158083758 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe928078bb0 0x7fe92807b070 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe934007fd0 tx=0x7fe934009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:39.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.020+0000 7fe944c00640 1 -- 172.21.15.151:0/1158083758 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe9380830e0 con 0x7fe94806efa0 2024-09-16T15:42:39.207 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.205+0000 7fe94f600640 1 -- 172.21.15.151:0/1158083758 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"} v 0) -- 0x7fe948144e50 con 0x7fe94806efa0 2024-09-16T15:42:39.207 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.206+0000 7fe944c00640 1 -- 172.21.15.151:0/1158083758 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"}]=0 v31) ==== 116+0+2 (secure 0 0 0) 0x7fe938087b10 con 0x7fe94806efa0 2024-09-16T15:42:39.208 INFO:teuthology.orchestra.run.smithi151.stdout:4 2024-09-16T15:42:39.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.208+0000 7fe94f600640 1 -- 172.21.15.151:0/1158083758 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe928078bb0 msgr2=0x7fe92807b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:39.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.208+0000 7fe94f600640 1 --2- 172.21.15.151:0/1158083758 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe928078bb0 0x7fe92807b070 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe934007fd0 tx=0x7fe934009040 comp rx=0 tx=0).stop 2024-09-16T15:42:39.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.208+0000 7fe94f600640 1 -- 172.21.15.151:0/1158083758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe94806efa0 msgr2=0x7fe948135340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:39.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.208+0000 7fe94f600640 1 --2- 172.21.15.151:0/1158083758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe94806efa0 0x7fe948135340 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe938034040 tx=0x7fe9380039e0 comp rx=0 tx=0).stop 2024-09-16T15:42:39.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.208+0000 7fe94f600640 1 -- 172.21.15.151:0/1158083758 shutdown_connections 2024-09-16T15:42:39.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.208+0000 7fe94f600640 1 --2- 172.21.15.151:0/1158083758 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe928078bb0 0x7fe92807b070 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:39.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.208+0000 7fe94f600640 1 --2- 172.21.15.151:0/1158083758 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe948145750 0x7fe9481ef970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:39.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.208+0000 7fe94f600640 1 --2- 172.21.15.151:0/1158083758 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe9480cccd0 0x7fe9480ca860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:39.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.208+0000 7fe94f600640 1 --2- 172.21.15.151:0/1158083758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe94806efa0 0x7fe948135340 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:39.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.208+0000 7fe94f600640 1 -- 172.21.15.151:0/1158083758 >> 172.21.15.151:0/1158083758 conn(0x7fe9480b1840 msgr2=0x7fe9480b0660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:39.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.208+0000 7fe94f600640 1 -- 172.21.15.151:0/1158083758 shutdown_connections 2024-09-16T15:42:39.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.208+0000 7fe94f600640 1 -- 172.21.15.151:0/1158083758 wait complete. 2024-09-16T15:42:39.226 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-16T15:42:39.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.384+0000 7f42f6800640 1 -- 172.21.15.151:0/3822952927 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f42f01618c0 msgr2=0x7f42f00d4bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:39.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.384+0000 7f42f6800640 1 --2- 172.21.15.151:0/3822952927 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f42f01618c0 0x7f42f00d4bc0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f42f00ce670 tx=0x7f42dc02a510 comp rx=0 tx=0).stop 2024-09-16T15:42:39.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.385+0000 7f42f6800640 1 -- 172.21.15.151:0/3822952927 shutdown_connections 2024-09-16T15:42:39.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.385+0000 7f42f6800640 1 --2- 172.21.15.151:0/3822952927 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42f00cf630 0x7f42f00cfae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:39.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.385+0000 7f42f6800640 1 --2- 172.21.15.151:0/3822952927 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f42f01618c0 0x7f42f00d4bc0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:39.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.385+0000 7f42f6800640 1 -- 172.21.15.151:0/3822952927 >> 172.21.15.151:0/3822952927 conn(0x7f42f00d40a0 msgr2=0x7f42f00d44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:39.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.385+0000 7f42f6800640 1 -- 172.21.15.151:0/3822952927 shutdown_connections 2024-09-16T15:42:39.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.385+0000 7f42f6800640 1 -- 172.21.15.151:0/3822952927 wait complete. 2024-09-16T15:42:39.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.386+0000 7f42f6800640 1 Processor -- start 2024-09-16T15:42:39.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.386+0000 7f42f6800640 1 -- start start 2024-09-16T15:42:39.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.387+0000 7f42f6800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42f00cf630 0x7f42f014d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:39.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.387+0000 7f42f6800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42f01614e0 0x7f42f00b1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:39.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.387+0000 7f42f6800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f42f01618c0 0x7f42f00b2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:39.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.387+0000 7f42f6800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f42f00d2560 con 0x7f42f01618c0 2024-09-16T15:42:39.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.387+0000 7f42f6800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f42f00d23e0 con 0x7f42f01614e0 2024-09-16T15:42:39.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.387+0000 7f42f6800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f42f00d26e0 con 0x7f42f00cf630 2024-09-16T15:42:39.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.388+0000 7f42efe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f42f01618c0 0x7f42f00b2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:39.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.388+0000 7f42efe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f42f01618c0 0x7f42f00b2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37188/0 (socket says 172.21.15.151:37188) 2024-09-16T15:42:39.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.388+0000 7f42efe00640 1 -- 172.21.15.151:0/257395355 learned_addr learned my addr 172.21.15.151:0/257395355 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:42:39.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.388+0000 7f42eea00640 1 --2- 172.21.15.151:0/257395355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42f01614e0 0x7f42f00b1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:39.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.388+0000 7f42ef400640 1 --2- 172.21.15.151:0/257395355 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42f00cf630 0x7f42f014d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:39.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.388+0000 7f42efe00640 1 -- 172.21.15.151:0/257395355 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42f00cf630 msgr2=0x7f42f014d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:39.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.388+0000 7f42efe00640 1 --2- 172.21.15.151:0/257395355 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42f00cf630 0x7f42f014d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:39.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.388+0000 7f42efe00640 1 -- 172.21.15.151:0/257395355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42f01614e0 msgr2=0x7f42f00b1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:39.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.388+0000 7f42efe00640 1 --2- 172.21.15.151:0/257395355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42f01614e0 0x7f42f00b1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:39.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.388+0000 7f42efe00640 1 -- 172.21.15.151:0/257395355 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42f00b8050 con 0x7f42f01618c0 2024-09-16T15:42:39.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.388+0000 7f42efe00640 1 --2- 172.21.15.151:0/257395355 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f42f01618c0 0x7f42f00b2220 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f42e800eaa0 tx=0x7f42e800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:39.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.389+0000 7f42e7400640 1 -- 172.21.15.151:0/257395355 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e800cdb0 con 0x7f42f01618c0 2024-09-16T15:42:39.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.389+0000 7f42e7400640 1 -- 172.21.15.151:0/257395355 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f42e8004c10 con 0x7f42f01618c0 2024-09-16T15:42:39.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.389+0000 7f42e7400640 1 -- 172.21.15.151:0/257395355 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e8005020 con 0x7f42f01618c0 2024-09-16T15:42:39.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.389+0000 7f42f6800640 1 -- 172.21.15.151:0/257395355 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42f00b82f0 con 0x7f42f01618c0 2024-09-16T15:42:39.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.389+0000 7f42f6800640 1 -- 172.21.15.151:0/257395355 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42f00b88b0 con 0x7f42f01618c0 2024-09-16T15:42:39.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.390+0000 7f42f6800640 1 -- 172.21.15.151:0/257395355 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42f00cfae0 con 0x7f42f01618c0 2024-09-16T15:42:39.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.391+0000 7f42e7400640 1 -- 172.21.15.151:0/257395355 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7f42e80051c0 con 0x7f42f01618c0 2024-09-16T15:42:39.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.392+0000 7f42e7400640 1 --2- 172.21.15.151:0/257395355 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f42d0078bb0 0x7f42d007b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:39.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.392+0000 7f42e7400640 1 -- 172.21.15.151:0/257395355 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f42e809b8e0 con 0x7f42f01618c0 2024-09-16T15:42:39.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.393+0000 7f42ef400640 1 --2- 172.21.15.151:0/257395355 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f42d0078bb0 0x7f42d007b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:39.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.393+0000 7f42ef400640 1 --2- 172.21.15.151:0/257395355 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f42d0078bb0 0x7f42d007b070 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f42f00b9f60 tx=0x7f42e0007e80 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:39.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.397+0000 7f42e7400640 1 -- 172.21.15.151:0/257395355 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f42e8016030 con 0x7f42f01618c0 2024-09-16T15:42:39.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.581+0000 7f42f6800640 1 -- 172.21.15.151:0/257395355 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f42f00ba020 con 0x7f42f01618c0 2024-09-16T15:42:39.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.585+0000 7f42e7400640 1 -- 172.21.15.151:0/257395355 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v32)=0 v32) ==== 131+0+0 (secure 0 0 0) 0x7f42e8063330 con 0x7f42f01618c0 2024-09-16T15:42:39.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.588+0000 7f42f6800640 1 -- 172.21.15.151:0/257395355 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f42d0078bb0 msgr2=0x7f42d007b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:39.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.588+0000 7f42f6800640 1 --2- 172.21.15.151:0/257395355 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f42d0078bb0 0x7f42d007b070 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f42f00b9f60 tx=0x7f42e0007e80 comp rx=0 tx=0).stop 2024-09-16T15:42:39.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.588+0000 7f42f6800640 1 -- 172.21.15.151:0/257395355 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f42f01618c0 msgr2=0x7f42f00b2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:39.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.588+0000 7f42f6800640 1 --2- 172.21.15.151:0/257395355 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f42f01618c0 0x7f42f00b2220 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f42e800eaa0 tx=0x7f42e800ef60 comp rx=0 tx=0).stop 2024-09-16T15:42:39.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.588+0000 7f42f6800640 1 -- 172.21.15.151:0/257395355 shutdown_connections 2024-09-16T15:42:39.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.588+0000 7f42f6800640 1 --2- 172.21.15.151:0/257395355 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f42d0078bb0 0x7f42d007b070 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:39.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.588+0000 7f42f6800640 1 --2- 172.21.15.151:0/257395355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42f01614e0 0x7f42f00b1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:39.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.588+0000 7f42f6800640 1 --2- 172.21.15.151:0/257395355 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f42f01618c0 0x7f42f00b2220 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:39.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.588+0000 7f42f6800640 1 --2- 172.21.15.151:0/257395355 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42f00cf630 0x7f42f014d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:39.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.588+0000 7f42f6800640 1 -- 172.21.15.151:0/257395355 >> 172.21.15.151:0/257395355 conn(0x7f42f00d40a0 msgr2=0x7f42f006e760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:39.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.588+0000 7f42f6800640 1 -- 172.21.15.151:0/257395355 shutdown_connections 2024-09-16T15:42:39.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.588+0000 7f42f6800640 1 -- 172.21.15.151:0/257395355 wait complete. 2024-09-16T15:42:39.602 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:42:39.601+0000 7f48f1c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:42:39.602 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:42:39.601+0000 7f48f1c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:42:39.602 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:42:39.601+0000 7f48f1c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:42:39.602 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:42:39.601+0000 7f48f1c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:42:39.602 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:42:39.601+0000 7f48f1c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:42:39.606 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-09-16T15:42:39.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.765+0000 7f570e200640 1 -- 172.21.15.151:0/838675881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5708073de0 msgr2=0x7f5708074240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:39.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.765+0000 7f570e200640 1 --2- 172.21.15.151:0/838675881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5708073de0 0x7f5708074240 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f56f4005fd0 tx=0x7f56f402a540 comp rx=0 tx=0).stop 2024-09-16T15:42:39.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.766+0000 7f570e200640 1 -- 172.21.15.151:0/838675881 shutdown_connections 2024-09-16T15:42:39.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.766+0000 7f570e200640 1 --2- 172.21.15.151:0/838675881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5708073de0 0x7f5708074240 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:39.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.766+0000 7f570e200640 1 --2- 172.21.15.151:0/838675881 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f570815f9a0 0x7f570815fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:39.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.766+0000 7f570e200640 1 -- 172.21.15.151:0/838675881 >> 172.21.15.151:0/838675881 conn(0x7f57080b18e0 msgr2=0x7f57080b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:39.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.766+0000 7f570e200640 1 -- 172.21.15.151:0/838675881 shutdown_connections 2024-09-16T15:42:39.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.766+0000 7f570e200640 1 -- 172.21.15.151:0/838675881 wait complete. 2024-09-16T15:42:39.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.767+0000 7f570e200640 1 Processor -- start 2024-09-16T15:42:39.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.767+0000 7f570e200640 1 -- start start 2024-09-16T15:42:39.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.768+0000 7f570e200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5708073de0 0x7f570814cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:39.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.768+0000 7f570e200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f570815c3f0 0x7f5708158ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:39.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.768+0000 7f570e200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f570815f9a0 0x7f5708159410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:39.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.768+0000 7f570e200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f57080cd3d0 con 0x7f5708073de0 2024-09-16T15:42:39.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.768+0000 7f570e200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f57080ccbd0 con 0x7f570815c3f0 2024-09-16T15:42:39.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.768+0000 7f570e200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f57080cd550 con 0x7f570815f9a0 2024-09-16T15:42:39.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.769+0000 7f5707400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5708073de0 0x7f570814cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:39.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.769+0000 7f5707400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5708073de0 0x7f570814cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37192/0 (socket says 172.21.15.151:37192) 2024-09-16T15:42:39.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.769+0000 7f5707400640 1 -- 172.21.15.151:0/1651410002 learned_addr learned my addr 172.21.15.151:0/1651410002 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:42:39.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.769+0000 7f5706a00640 1 --2- 172.21.15.151:0/1651410002 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f570815c3f0 0x7f5708158ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:39.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.769+0000 7f5707e00640 1 --2- 172.21.15.151:0/1651410002 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f570815f9a0 0x7f5708159410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:39.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.769+0000 7f5707400640 1 -- 172.21.15.151:0/1651410002 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f570815f9a0 msgr2=0x7f5708159410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:39.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.769+0000 7f5707400640 1 --2- 172.21.15.151:0/1651410002 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f570815f9a0 0x7f5708159410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:39.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.769+0000 7f5707400640 1 -- 172.21.15.151:0/1651410002 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f570815c3f0 msgr2=0x7f5708158ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:39.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.769+0000 7f5707400640 1 --2- 172.21.15.151:0/1651410002 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f570815c3f0 0x7f5708158ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:39.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.769+0000 7f5707400640 1 -- 172.21.15.151:0/1651410002 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5708153a70 con 0x7f5708073de0 2024-09-16T15:42:39.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.769+0000 7f5706a00640 1 --2- 172.21.15.151:0/1651410002 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f570815c3f0 0x7f5708158ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:42:39.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.769+0000 7f5707e00640 1 --2- 172.21.15.151:0/1651410002 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f570815f9a0 0x7f5708159410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:42:39.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.769+0000 7f5707400640 1 --2- 172.21.15.151:0/1651410002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5708073de0 0x7f570814cfb0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f56f800e790 tx=0x7f56f800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:39.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.770+0000 7f56ff400640 1 -- 172.21.15.151:0/1651410002 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f800eea0 con 0x7f5708073de0 2024-09-16T15:42:39.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.770+0000 7f56ff400640 1 -- 172.21.15.151:0/1651410002 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f56f800cdb0 con 0x7f5708073de0 2024-09-16T15:42:39.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.770+0000 7f56ff400640 1 -- 172.21.15.151:0/1651410002 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f8010690 con 0x7f5708073de0 2024-09-16T15:42:39.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.770+0000 7f570e200640 1 -- 172.21.15.151:0/1651410002 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5708153d50 con 0x7f5708073de0 2024-09-16T15:42:39.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.770+0000 7f570e200640 1 -- 172.21.15.151:0/1651410002 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5708154310 con 0x7f5708073de0 2024-09-16T15:42:39.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.771+0000 7f570e200640 1 -- 172.21.15.151:0/1651410002 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5708069360 con 0x7f5708073de0 2024-09-16T15:42:39.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.772+0000 7f56ff400640 1 -- 172.21.15.151:0/1651410002 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 10) ==== 101593+0+0 (secure 0 0 0) 0x7f56f8010830 con 0x7f5708073de0 2024-09-16T15:42:39.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.773+0000 7f56ff400640 1 --2- 172.21.15.151:0/1651410002 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f56e8078bb0 0x7f56e807b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:39.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.773+0000 7f56ff400640 1 -- 172.21.15.151:0/1651410002 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f56f809b980 con 0x7f5708073de0 2024-09-16T15:42:39.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.774+0000 7f5706a00640 1 --2- 172.21.15.151:0/1651410002 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f56e8078bb0 0x7f56e807b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:39.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.774+0000 7f5706a00640 1 --2- 172.21.15.151:0/1651410002 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f56e8078bb0 0x7f56e807b070 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f56f4033000 tx=0x7f56f4004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:39.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.777+0000 7f56ff400640 1 -- 172.21.15.151:0/1651410002 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f56f8063b20 con 0x7f5708073de0 2024-09-16T15:42:39.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:39.986+0000 7f570e200640 1 -- 172.21.15.151:0/1651410002 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7f5708154550 con 0x7f56e8078bb0 2024-09-16T15:42:40.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:40.594+0000 7f56ff400640 1 -- 172.21.15.151:0/1651410002 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mgrmap(e 11) ==== 101644+0+0 (secure 0 0 0) 0x7f56f805f690 con 0x7f5708073de0 2024-09-16T15:42:40.763 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:42:40.761+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:42:40.763 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:42:40.762+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:42:40.763 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:42:40.762+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:42:40.763 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:42:40.762+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:42:40.763 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:42:40.762+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:42:40.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:40.801+0000 7f56ff400640 1 -- 172.21.15.151:0/1651410002 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f56f40042c0 con 0x7f56e8078bb0 2024-09-16T15:42:40.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:40.803+0000 7f570e200640 1 -- 172.21.15.151:0/1651410002 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f56e8078bb0 msgr2=0x7f56e807b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:40.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:40.803+0000 7f570e200640 1 --2- 172.21.15.151:0/1651410002 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f56e8078bb0 0x7f56e807b070 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f56f4033000 tx=0x7f56f4004730 comp rx=0 tx=0).stop 2024-09-16T15:42:40.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:40.803+0000 7f570e200640 1 -- 172.21.15.151:0/1651410002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5708073de0 msgr2=0x7f570814cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:40.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:40.803+0000 7f570e200640 1 --2- 172.21.15.151:0/1651410002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5708073de0 0x7f570814cfb0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f56f800e790 tx=0x7f56f800ec50 comp rx=0 tx=0).stop 2024-09-16T15:42:40.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:40.803+0000 7f570e200640 1 -- 172.21.15.151:0/1651410002 shutdown_connections 2024-09-16T15:42:40.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:40.803+0000 7f570e200640 1 --2- 172.21.15.151:0/1651410002 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f56e8078bb0 0x7f56e807b070 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:40.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:40.803+0000 7f570e200640 1 --2- 172.21.15.151:0/1651410002 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f570815c3f0 0x7f5708158ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:40.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:40.803+0000 7f570e200640 1 --2- 172.21.15.151:0/1651410002 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f570815f9a0 0x7f5708159410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:40.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:40.803+0000 7f570e200640 1 --2- 172.21.15.151:0/1651410002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5708073de0 0x7f570814cfb0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:40.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:40.804+0000 7f570e200640 1 -- 172.21.15.151:0/1651410002 >> 172.21.15.151:0/1651410002 conn(0x7f57080b18e0 msgr2=0x7f57080aed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:40.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:40.804+0000 7f570e200640 1 -- 172.21.15.151:0/1651410002 shutdown_connections 2024-09-16T15:42:40.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:40.804+0000 7f570e200640 1 -- 172.21.15.151:0/1651410002 wait complete. 2024-09-16T15:42:40.821 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-16T15:42:40.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:40.980+0000 7efca8c00640 1 -- 172.21.15.151:0/1094003261 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efca40cccd0 msgr2=0x7efca4073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:40.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:40.980+0000 7efca8c00640 1 --2- 172.21.15.151:0/1094003261 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efca40cccd0 0x7efca4073440 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7efc8c005fd0 tx=0x7efc8c02a540 comp rx=0 tx=0).stop 2024-09-16T15:42:40.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:40.980+0000 7efca8c00640 1 -- 172.21.15.151:0/1094003261 shutdown_connections 2024-09-16T15:42:40.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:40.980+0000 7efca8c00640 1 --2- 172.21.15.151:0/1094003261 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efca40cccd0 0x7efca4073440 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:40.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:40.980+0000 7efca8c00640 1 -- 172.21.15.151:0/1094003261 >> 172.21.15.151:0/1094003261 conn(0x7efca40b1840 msgr2=0x7efca40b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:40.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:40.980+0000 7efca8c00640 1 -- 172.21.15.151:0/1094003261 shutdown_connections 2024-09-16T15:42:40.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:40.980+0000 7efca8c00640 1 -- 172.21.15.151:0/1094003261 wait complete. 2024-09-16T15:42:40.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:40.981+0000 7efca8c00640 1 Processor -- start 2024-09-16T15:42:40.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:40.982+0000 7efca8c00640 1 -- start start 2024-09-16T15:42:40.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:40.982+0000 7efca8c00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efca406efa0 0x7efca40b4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:40.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:40.982+0000 7efca8c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efca40cccd0 0x7efca41f84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:40.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:40.982+0000 7efca8c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efca4145750 0x7efca41f8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:40.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:40.983+0000 7efca8c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efca40ce6a0 con 0x7efca40cccd0 2024-09-16T15:42:40.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:40.983+0000 7efca8c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efca40ce520 con 0x7efca4145750 2024-09-16T15:42:40.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:40.983+0000 7efca8c00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7efca40ce820 con 0x7efca406efa0 2024-09-16T15:42:40.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:40.983+0000 7efc9be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efca40cccd0 0x7efca41f84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:40.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:40.983+0000 7efc9be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efca40cccd0 0x7efca41f84f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37202/0 (socket says 172.21.15.151:37202) 2024-09-16T15:42:40.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:40.983+0000 7efc9be00640 1 -- 172.21.15.151:0/426626594 learned_addr learned my addr 172.21.15.151:0/426626594 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:42:40.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:40.983+0000 7efca1e00640 1 --2- 172.21.15.151:0/426626594 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efca406efa0 0x7efca40b4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:40.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:40.983+0000 7efca2800640 1 --2- 172.21.15.151:0/426626594 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efca4145750 0x7efca41f8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:40.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:40.983+0000 7efc9be00640 1 -- 172.21.15.151:0/426626594 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efca406efa0 msgr2=0x7efca40b4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:40.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:40.983+0000 7efc9be00640 1 --2- 172.21.15.151:0/426626594 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efca406efa0 0x7efca40b4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:40.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:40.983+0000 7efc9be00640 1 -- 172.21.15.151:0/426626594 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efca4145750 msgr2=0x7efca41f8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:40.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:40.983+0000 7efc9be00640 1 --2- 172.21.15.151:0/426626594 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efca4145750 0x7efca41f8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:40.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:40.983+0000 7efc9be00640 1 -- 172.21.15.151:0/426626594 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efca40b3070 con 0x7efca40cccd0 2024-09-16T15:42:40.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:40.984+0000 7efc9be00640 1 --2- 172.21.15.151:0/426626594 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efca40cccd0 0x7efca41f84f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7efc8c006010 tx=0x7efc8c0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:40.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:40.984+0000 7efc9a000640 1 -- 172.21.15.151:0/426626594 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc8c03f070 con 0x7efca40cccd0 2024-09-16T15:42:40.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:40.984+0000 7efc9a000640 1 -- 172.21.15.151:0/426626594 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc8c004890 con 0x7efca40cccd0 2024-09-16T15:42:40.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:40.984+0000 7efc9a000640 1 -- 172.21.15.151:0/426626594 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc8c035030 con 0x7efca40cccd0 2024-09-16T15:42:40.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:40.984+0000 7efca8c00640 1 -- 172.21.15.151:0/426626594 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efca40b3350 con 0x7efca40cccd0 2024-09-16T15:42:40.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:40.984+0000 7efca8c00640 1 -- 172.21.15.151:0/426626594 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efca40b3bf0 con 0x7efca40cccd0 2024-09-16T15:42:40.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:40.985+0000 7efca8c00640 1 -- 172.21.15.151:0/426626594 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efca40b3740 con 0x7efca40cccd0 2024-09-16T15:42:40.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:40.986+0000 7efc9a000640 1 -- 172.21.15.151:0/426626594 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 11) ==== 101644+0+0 (secure 0 0 0) 0x7efc8c042070 con 0x7efca40cccd0 2024-09-16T15:42:40.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:40.987+0000 7efc9a000640 1 --2- 172.21.15.151:0/426626594 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efc80078c40 0x7efc8007b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:42:40.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:40.987+0000 7efc9a000640 1 -- 172.21.15.151:0/426626594 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7efc8c0bbf80 con 0x7efca40cccd0 2024-09-16T15:42:40.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:40.988+0000 7efca1e00640 1 --2- 172.21.15.151:0/426626594 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efc80078c40 0x7efc8007b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:42:40.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:40.988+0000 7efca1e00640 1 --2- 172.21.15.151:0/426626594 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efc80078c40 0x7efc8007b100 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7efc94001720 tx=0x7efc94009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:42:40.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:40.992+0000 7efc9a000640 1 -- 172.21.15.151:0/426626594 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7efc8c083900 con 0x7efca40cccd0 2024-09-16T15:42:41.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:41.210+0000 7efca8c00640 1 -- 172.21.15.151:0/426626594 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7efca40cecd0 con 0x7efc80078c40 2024-09-16T15:42:41.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:41.216+0000 7efc9a000640 1 -- 172.21.15.151:0/426626594 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7efca40cecd0 con 0x7efc80078c40 2024-09-16T15:42:41.218 INFO:teuthology.orchestra.run.smithi151.stdout:/volumes/_nogroup/sv1/e5d6f34b-74d3-4dc6-ae17-74c26374cc07 2024-09-16T15:42:41.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:41.218+0000 7efca8c00640 1 -- 172.21.15.151:0/426626594 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efc80078c40 msgr2=0x7efc8007b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:41.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:41.218+0000 7efca8c00640 1 --2- 172.21.15.151:0/426626594 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efc80078c40 0x7efc8007b100 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7efc94001720 tx=0x7efc94009040 comp rx=0 tx=0).stop 2024-09-16T15:42:41.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:41.218+0000 7efca8c00640 1 -- 172.21.15.151:0/426626594 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efca40cccd0 msgr2=0x7efca41f84f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:42:41.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:41.218+0000 7efca8c00640 1 --2- 172.21.15.151:0/426626594 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efca40cccd0 0x7efca41f84f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7efc8c006010 tx=0x7efc8c0039e0 comp rx=0 tx=0).stop 2024-09-16T15:42:41.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:41.218+0000 7efca8c00640 1 -- 172.21.15.151:0/426626594 shutdown_connections 2024-09-16T15:42:41.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:41.218+0000 7efca8c00640 1 --2- 172.21.15.151:0/426626594 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efc80078c40 0x7efc8007b100 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:41.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:41.218+0000 7efca8c00640 1 --2- 172.21.15.151:0/426626594 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efca4145750 0x7efca41f8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:41.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:41.218+0000 7efca8c00640 1 --2- 172.21.15.151:0/426626594 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efca40cccd0 0x7efca41f84f0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:41.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:41.218+0000 7efca8c00640 1 --2- 172.21.15.151:0/426626594 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efca406efa0 0x7efca40b4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:42:41.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:41.218+0000 7efca8c00640 1 -- 172.21.15.151:0/426626594 >> 172.21.15.151:0/426626594 conn(0x7efca40b1840 msgr2=0x7efca40b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:42:41.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:41.218+0000 7efca8c00640 1 -- 172.21.15.151:0/426626594 shutdown_connections 2024-09-16T15:42:41.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:42:41.218+0000 7efca8c00640 1 -- 172.21.15.151:0/426626594 wait complete. 2024-09-16T15:42:41.235 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 10 files each 1024MB size under directory volumes/_nogroup/sv1/e5d6f34b-74d3-4dc6-ae17-74c26374cc07 2024-09-16T15:42:41.235 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:42:41.236 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:42:41.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:42:41.271 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/e5d6f34b-74d3-4dc6-ae17-74c26374cc07/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T15:42:47.496 DEBUG:teuthology.orchestra.run.smithi151:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:42:47.502 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:42:49.203 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records in 2024-09-16T15:42:49.204 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records out 2024-09-16T15:42:49.204 INFO:teuthology.orchestra.run.smithi151.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.85042 s, 137 MB/s 2024-09-16T15:42:49.205 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:42:49.205 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:42:49.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:42:49.241 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/e5d6f34b-74d3-4dc6-ae17-74c26374cc07/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T15:42:56.957 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records in 2024-09-16T15:42:56.957 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records out 2024-09-16T15:42:56.958 INFO:teuthology.orchestra.run.smithi151.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.64005 s, 141 MB/s 2024-09-16T15:42:56.959 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:42:56.959 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:42:56.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:42:56.994 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/e5d6f34b-74d3-4dc6-ae17-74c26374cc07/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T15:43:05.077 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records in 2024-09-16T15:43:05.077 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records out 2024-09-16T15:43:05.077 INFO:teuthology.orchestra.run.smithi151.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.00546 s, 134 MB/s 2024-09-16T15:43:05.078 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:43:05.078 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:43:05.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:43:05.114 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/e5d6f34b-74d3-4dc6-ae17-74c26374cc07/subvolume_file.3 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T15:43:13.142 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records in 2024-09-16T15:43:13.143 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records out 2024-09-16T15:43:13.143 INFO:teuthology.orchestra.run.smithi151.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.95174 s, 135 MB/s 2024-09-16T15:43:13.144 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:43:13.144 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:43:13.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:43:13.180 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/e5d6f34b-74d3-4dc6-ae17-74c26374cc07/subvolume_file.4 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T15:43:17.636 DEBUG:teuthology.orchestra.run.smithi151:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:43:17.641 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:43:21.276 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records in 2024-09-16T15:43:21.276 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records out 2024-09-16T15:43:21.276 INFO:teuthology.orchestra.run.smithi151.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.01939 s, 134 MB/s 2024-09-16T15:43:21.277 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:43:21.278 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:43:21.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:43:21.313 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/e5d6f34b-74d3-4dc6-ae17-74c26374cc07/subvolume_file.5 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T15:43:29.273 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records in 2024-09-16T15:43:29.273 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records out 2024-09-16T15:43:29.273 INFO:teuthology.orchestra.run.smithi151.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.88425 s, 136 MB/s 2024-09-16T15:43:29.274 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:43:29.274 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:43:29.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:43:29.310 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/e5d6f34b-74d3-4dc6-ae17-74c26374cc07/subvolume_file.6 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T15:43:37.428 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records in 2024-09-16T15:43:37.429 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records out 2024-09-16T15:43:37.429 INFO:teuthology.orchestra.run.smithi151.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.04282 s, 134 MB/s 2024-09-16T15:43:37.430 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:43:37.430 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:43:37.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:43:37.466 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/e5d6f34b-74d3-4dc6-ae17-74c26374cc07/subvolume_file.7 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T15:43:45.440 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records in 2024-09-16T15:43:45.440 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records out 2024-09-16T15:43:45.440 INFO:teuthology.orchestra.run.smithi151.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.89749 s, 136 MB/s 2024-09-16T15:43:45.441 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:43:45.441 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:43:45.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:43:45.477 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/e5d6f34b-74d3-4dc6-ae17-74c26374cc07/subvolume_file.8 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T15:43:47.737 DEBUG:teuthology.orchestra.run.smithi151:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:43:47.741 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:43:53.565 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records in 2024-09-16T15:43:53.565 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records out 2024-09-16T15:43:53.565 INFO:teuthology.orchestra.run.smithi151.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.01233 s, 134 MB/s 2024-09-16T15:43:53.567 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:43:53.567 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:43:53.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:43:53.602 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/e5d6f34b-74d3-4dc6-ae17-74c26374cc07/subvolume_file.9 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T15:44:01.504 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records in 2024-09-16T15:44:01.504 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records out 2024-09-16T15:44:01.504 INFO:teuthology.orchestra.run.smithi151.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.82609 s, 137 MB/s 2024-09-16T15:44:01.506 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-09-16T15:44:01.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:01.696+0000 7efc88c00640 1 -- 172.21.15.151:0/1709054990 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc840b32f0 msgr2=0x7efc840b4ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:01.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:01.696+0000 7efc88c00640 1 --2- 172.21.15.151:0/1709054990 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc840b32f0 0x7efc840b4ac0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7efc70009830 tx=0x7efc7002a480 comp rx=0 tx=0).stop 2024-09-16T15:44:01.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:01.697+0000 7efc88c00640 1 -- 172.21.15.151:0/1709054990 shutdown_connections 2024-09-16T15:44:01.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:01.697+0000 7efc88c00640 1 --2- 172.21.15.151:0/1709054990 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efc840cc5e0 0x7efc840cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:01.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:01.697+0000 7efc88c00640 1 --2- 172.21.15.151:0/1709054990 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc840b32f0 0x7efc840b4ac0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:01.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:01.697+0000 7efc88c00640 1 -- 172.21.15.151:0/1709054990 >> 172.21.15.151:0/1709054990 conn(0x7efc840b18e0 msgr2=0x7efc840b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:01.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:01.697+0000 7efc88c00640 1 -- 172.21.15.151:0/1709054990 shutdown_connections 2024-09-16T15:44:01.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:01.697+0000 7efc88c00640 1 -- 172.21.15.151:0/1709054990 wait complete. 2024-09-16T15:44:01.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:01.699+0000 7efc88c00640 1 Processor -- start 2024-09-16T15:44:01.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:01.699+0000 7efc88c00640 1 -- start start 2024-09-16T15:44:01.703 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:01.700+0000 7efc88c00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efc840b32f0 0x7efc841375f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:01.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:01.700+0000 7efc88c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc840cc5e0 0x7efc841fd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:01.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:01.701+0000 7efc88c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc84144730 0x7efc84134e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:01.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:01.701+0000 7efc88c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efc840ce4e0 con 0x7efc840cc5e0 2024-09-16T15:44:01.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:01.701+0000 7efc88c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efc840ce360 con 0x7efc84144730 2024-09-16T15:44:01.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:01.701+0000 7efc88c00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7efc840ce660 con 0x7efc840b32f0 2024-09-16T15:44:01.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:01.701+0000 7efc81400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc840cc5e0 0x7efc841fd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:01.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:01.701+0000 7efc81400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc840cc5e0 0x7efc841fd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:54482/0 (socket says 172.21.15.151:54482) 2024-09-16T15:44:01.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:01.701+0000 7efc81400640 1 -- 172.21.15.151:0/2364730233 learned_addr learned my addr 172.21.15.151:0/2364730233 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:01.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:01.701+0000 7efc81e00640 1 --2- 172.21.15.151:0/2364730233 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efc840b32f0 0x7efc841375f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:01.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:01.701+0000 7efc82800640 1 --2- 172.21.15.151:0/2364730233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc84144730 0x7efc84134e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:01.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:01.701+0000 7efc81e00640 1 -- 172.21.15.151:0/2364730233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc84144730 msgr2=0x7efc84134e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:01.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:01.701+0000 7efc81e00640 1 --2- 172.21.15.151:0/2364730233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc84144730 0x7efc84134e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:01.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:01.701+0000 7efc81e00640 1 -- 172.21.15.151:0/2364730233 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc840cc5e0 msgr2=0x7efc841fd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:01.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:01.701+0000 7efc81e00640 1 --2- 172.21.15.151:0/2364730233 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc840cc5e0 0x7efc841fd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:01.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:01.701+0000 7efc81e00640 1 -- 172.21.15.151:0/2364730233 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc841fc950 con 0x7efc840b32f0 2024-09-16T15:44:01.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:01.701+0000 7efc81400640 1 --2- 172.21.15.151:0/2364730233 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc840cc5e0 0x7efc841fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:44:01.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:01.702+0000 7efc81e00640 1 --2- 172.21.15.151:0/2364730233 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efc840b32f0 0x7efc841375f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7efc7400c3f0 tx=0x7efc7400c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:01.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:01.702+0000 7efc7a000640 1 -- 172.21.15.151:0/2364730233 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc74011790 con 0x7efc840b32f0 2024-09-16T15:44:01.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:01.702+0000 7efc7a000640 1 -- 172.21.15.151:0/2364730233 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc74017070 con 0x7efc840b32f0 2024-09-16T15:44:01.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:01.702+0000 7efc7a000640 1 -- 172.21.15.151:0/2364730233 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc74011ce0 con 0x7efc840b32f0 2024-09-16T15:44:01.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:01.702+0000 7efc88c00640 1 -- 172.21.15.151:0/2364730233 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc84137c40 con 0x7efc840b32f0 2024-09-16T15:44:01.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:01.702+0000 7efc88c00640 1 -- 172.21.15.151:0/2364730233 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7efc841fcbf0 con 0x7efc840b32f0 2024-09-16T15:44:01.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:01.703+0000 7efc88c00640 1 -- 172.21.15.151:0/2364730233 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc840b4ac0 con 0x7efc840b32f0 2024-09-16T15:44:01.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:01.705+0000 7efc7a000640 1 -- 172.21.15.151:0/2364730233 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 12) ==== 101695+0+0 (secure 0 0 0) 0x7efc7401c460 con 0x7efc840b32f0 2024-09-16T15:44:01.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:01.706+0000 7efc7a000640 1 --2- 172.21.15.151:0/2364730233 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efc64078cd0 0x7efc6407b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:01.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:01.706+0000 7efc7a000640 1 -- 172.21.15.151:0/2364730233 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(50..50 src has 1..50) ==== 5854+0+0 (secure 0 0 0) 0x7efc7409afc0 con 0x7efc840b32f0 2024-09-16T15:44:01.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:01.706+0000 7efc81400640 1 --2- 172.21.15.151:0/2364730233 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efc64078cd0 0x7efc6407b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:01.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:01.707+0000 7efc81400640 1 --2- 172.21.15.151:0/2364730233 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efc64078cd0 0x7efc6407b190 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7efc70009830 tx=0x7efc70004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:01.711 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:01.709+0000 7efc7a000640 1 -- 172.21.15.151:0/2364730233 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7efc74062a40 con 0x7efc840b32f0 2024-09-16T15:44:01.926 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:01.924+0000 7efc88c00640 1 -- 172.21.15.151:0/2364730233 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7efc84074a00 con 0x7efc64078cd0 2024-09-16T15:44:01.932 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:44:01.931+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:01.932 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:44:01.931+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:01.932 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:44:01.931+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:01.932 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:44:01.931+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:01.932 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:44:01.931+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:01.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:01.958+0000 7efc7a000640 1 -- 172.21.15.151:0/2364730233 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7efc84074a00 con 0x7efc64078cd0 2024-09-16T15:44:01.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:01.962+0000 7efc88c00640 1 -- 172.21.15.151:0/2364730233 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efc64078cd0 msgr2=0x7efc6407b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:01.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:01.962+0000 7efc88c00640 1 --2- 172.21.15.151:0/2364730233 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efc64078cd0 0x7efc6407b190 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7efc70009830 tx=0x7efc70004730 comp rx=0 tx=0).stop 2024-09-16T15:44:01.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:01.962+0000 7efc88c00640 1 -- 172.21.15.151:0/2364730233 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efc840b32f0 msgr2=0x7efc841375f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:01.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:01.962+0000 7efc88c00640 1 --2- 172.21.15.151:0/2364730233 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efc840b32f0 0x7efc841375f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7efc7400c3f0 tx=0x7efc7400c8b0 comp rx=0 tx=0).stop 2024-09-16T15:44:01.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:01.962+0000 7efc88c00640 1 -- 172.21.15.151:0/2364730233 shutdown_connections 2024-09-16T15:44:01.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:01.962+0000 7efc88c00640 1 --2- 172.21.15.151:0/2364730233 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efc64078cd0 0x7efc6407b190 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:01.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:01.962+0000 7efc88c00640 1 --2- 172.21.15.151:0/2364730233 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc84144730 0x7efc84134e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:01.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:01.962+0000 7efc88c00640 1 --2- 172.21.15.151:0/2364730233 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc840cc5e0 0x7efc841fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:01.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:01.962+0000 7efc88c00640 1 --2- 172.21.15.151:0/2364730233 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efc840b32f0 0x7efc841375f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:01.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:01.962+0000 7efc88c00640 1 -- 172.21.15.151:0/2364730233 >> 172.21.15.151:0/2364730233 conn(0x7efc840b18e0 msgr2=0x7efc8413e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:01.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:01.962+0000 7efc88c00640 1 -- 172.21.15.151:0/2364730233 shutdown_connections 2024-09-16T15:44:01.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:01.962+0000 7efc88c00640 1 -- 172.21.15.151:0/2364730233 wait complete. 2024-09-16T15:44:01.980 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-16T15:44:02.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:02.138+0000 7fbbaf800640 1 -- 172.21.15.151:0/1322196304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbba8145750 msgr2=0x7fbba8073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:02.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:02.138+0000 7fbbaf800640 1 --2- 172.21.15.151:0/1322196304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbba8145750 0x7fbba8073440 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fbb98009830 tx=0x7fbb9802a480 comp rx=0 tx=0).stop 2024-09-16T15:44:02.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:02.139+0000 7fbbaf800640 1 -- 172.21.15.151:0/1322196304 shutdown_connections 2024-09-16T15:44:02.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:02.139+0000 7fbbaf800640 1 --2- 172.21.15.151:0/1322196304 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbba806efa0 0x7fbba80cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:02.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:02.139+0000 7fbbaf800640 1 --2- 172.21.15.151:0/1322196304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbba8145750 0x7fbba8073440 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:02.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:02.139+0000 7fbbaf800640 1 -- 172.21.15.151:0/1322196304 >> 172.21.15.151:0/1322196304 conn(0x7fbba80b1840 msgr2=0x7fbba80b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:02.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:02.139+0000 7fbbaf800640 1 -- 172.21.15.151:0/1322196304 shutdown_connections 2024-09-16T15:44:02.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:02.139+0000 7fbbaf800640 1 -- 172.21.15.151:0/1322196304 wait complete. 2024-09-16T15:44:02.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:02.140+0000 7fbbaf800640 1 Processor -- start 2024-09-16T15:44:02.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:02.140+0000 7fbbaf800640 1 -- start start 2024-09-16T15:44:02.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:02.141+0000 7fbbaf800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbba806efa0 0x7fbba815ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:02.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:02.141+0000 7fbbaf800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbba80ccb00 0x7fbba81f7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:02.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:02.141+0000 7fbbaf800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbba8145750 0x7fbba81f83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:02.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:02.141+0000 7fbbaf800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbba80b26b0 con 0x7fbba806efa0 2024-09-16T15:44:02.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:02.141+0000 7fbbaf800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbba80cedd0 con 0x7fbba80ccb00 2024-09-16T15:44:02.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:02.141+0000 7fbbaf800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fbba8160100 con 0x7fbba8145750 2024-09-16T15:44:02.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:02.142+0000 7fbbace00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbba806efa0 0x7fbba815ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:02.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:02.142+0000 7fbbace00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbba806efa0 0x7fbba815ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:54486/0 (socket says 172.21.15.151:54486) 2024-09-16T15:44:02.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:02.142+0000 7fbbace00640 1 -- 172.21.15.151:0/2146978658 learned_addr learned my addr 172.21.15.151:0/2146978658 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:02.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:02.142+0000 7fbba7e00640 1 --2- 172.21.15.151:0/2146978658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbba80ccb00 0x7fbba81f7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:02.146 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:02.142+0000 7fbbad800640 1 --2- 172.21.15.151:0/2146978658 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbba8145750 0x7fbba81f83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:02.146 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:02.142+0000 7fbbace00640 1 -- 172.21.15.151:0/2146978658 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbba8145750 msgr2=0x7fbba81f83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:02.146 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:02.142+0000 7fbbace00640 1 --2- 172.21.15.151:0/2146978658 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbba8145750 0x7fbba81f83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:02.146 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:02.142+0000 7fbbace00640 1 -- 172.21.15.151:0/2146978658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbba80ccb00 msgr2=0x7fbba81f7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:02.146 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:02.142+0000 7fbbace00640 1 --2- 172.21.15.151:0/2146978658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbba80ccb00 0x7fbba81f7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:02.146 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:02.142+0000 7fbbace00640 1 -- 172.21.15.151:0/2146978658 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbba80cb3c0 con 0x7fbba806efa0 2024-09-16T15:44:02.146 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:02.142+0000 7fbbad800640 1 --2- 172.21.15.151:0/2146978658 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbba8145750 0x7fbba81f83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:44:02.146 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:02.142+0000 7fbbace00640 1 --2- 172.21.15.151:0/2146978658 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbba806efa0 0x7fbba815ee10 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fbb98009950 tx=0x7fbb98033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:02.146 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:02.143+0000 7fbba4c00640 1 -- 172.21.15.151:0/2146978658 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb98005c90 con 0x7fbba806efa0 2024-09-16T15:44:02.146 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:02.143+0000 7fbba4c00640 1 -- 172.21.15.151:0/2146978658 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbb98031e50 con 0x7fbba806efa0 2024-09-16T15:44:02.146 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:02.143+0000 7fbbaf800640 1 -- 172.21.15.151:0/2146978658 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbba8139040 con 0x7fbba806efa0 2024-09-16T15:44:02.146 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:02.143+0000 7fbba4c00640 1 -- 172.21.15.151:0/2146978658 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb98005260 con 0x7fbba806efa0 2024-09-16T15:44:02.146 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:02.143+0000 7fbbaf800640 1 -- 172.21.15.151:0/2146978658 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbba80cb910 con 0x7fbba806efa0 2024-09-16T15:44:02.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:02.144+0000 7fbbaf800640 1 -- 172.21.15.151:0/2146978658 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbba81605b0 con 0x7fbba806efa0 2024-09-16T15:44:02.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:02.145+0000 7fbba4c00640 1 -- 172.21.15.151:0/2146978658 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 12) ==== 101695+0+0 (secure 0 0 0) 0x7fbb98005400 con 0x7fbba806efa0 2024-09-16T15:44:02.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:02.146+0000 7fbba4c00640 1 --2- 172.21.15.151:0/2146978658 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbb88078cd0 0x7fbb8807b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:02.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:02.146+0000 7fbba4c00640 1 -- 172.21.15.151:0/2146978658 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5854+0+0 (secure 0 0 0) 0x7fbb980bc170 con 0x7fbba806efa0 2024-09-16T15:44:02.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:02.146+0000 7fbba7e00640 1 --2- 172.21.15.151:0/2146978658 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbb88078cd0 0x7fbb8807b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:02.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:02.147+0000 7fbba7e00640 1 --2- 172.21.15.151:0/2146978658 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbb88078cd0 0x7fbb8807b190 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fbba80c8290 tx=0x7fbb94005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:02.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:02.151+0000 7fbba4c00640 1 -- 172.21.15.151:0/2146978658 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fbb9803f030 con 0x7fbba806efa0 2024-09-16T15:44:02.312 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:02.309+0000 7fbba4c00640 1 -- 172.21.15.151:0/2146978658 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mgrmap(e 13) ==== 101644+0+0 (secure 0 0 0) 0x7fbb9807fbf0 con 0x7fbba806efa0 2024-09-16T15:44:02.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:02.368+0000 7fbbaf800640 1 -- 172.21.15.151:0/2146978658 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fbba8073390 con 0x7fbb88078cd0 2024-09-16T15:44:02.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:02.374+0000 7fbba4c00640 1 -- 172.21.15.151:0/2146978658 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7fbba8073390 con 0x7fbb88078cd0 2024-09-16T15:44:02.376 INFO:teuthology.orchestra.run.smithi151.stdout:/volumes/_nogroup/sv1/e5d6f34b-74d3-4dc6-ae17-74c26374cc07 2024-09-16T15:44:02.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:02.376+0000 7fbbaf800640 1 -- 172.21.15.151:0/2146978658 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbb88078cd0 msgr2=0x7fbb8807b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:02.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:02.376+0000 7fbbaf800640 1 --2- 172.21.15.151:0/2146978658 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbb88078cd0 0x7fbb8807b190 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fbba80c8290 tx=0x7fbb94005f90 comp rx=0 tx=0).stop 2024-09-16T15:44:02.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:02.376+0000 7fbbaf800640 1 -- 172.21.15.151:0/2146978658 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbba806efa0 msgr2=0x7fbba815ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:02.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:02.376+0000 7fbbaf800640 1 --2- 172.21.15.151:0/2146978658 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbba806efa0 0x7fbba815ee10 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fbb98009950 tx=0x7fbb98033c20 comp rx=0 tx=0).stop 2024-09-16T15:44:02.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:02.376+0000 7fbbaf800640 1 -- 172.21.15.151:0/2146978658 shutdown_connections 2024-09-16T15:44:02.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:02.376+0000 7fbbaf800640 1 --2- 172.21.15.151:0/2146978658 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbb88078cd0 0x7fbb8807b190 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:02.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:02.376+0000 7fbbaf800640 1 --2- 172.21.15.151:0/2146978658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbba80ccb00 0x7fbba81f7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:02.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:02.376+0000 7fbbaf800640 1 --2- 172.21.15.151:0/2146978658 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbba8145750 0x7fbba81f83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:02.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:02.376+0000 7fbbaf800640 1 --2- 172.21.15.151:0/2146978658 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbba806efa0 0x7fbba815ee10 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:02.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:02.376+0000 7fbbaf800640 1 -- 172.21.15.151:0/2146978658 >> 172.21.15.151:0/2146978658 conn(0x7fbba80b1840 msgr2=0x7fbba80b2c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:02.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:02.376+0000 7fbbaf800640 1 -- 172.21.15.151:0/2146978658 shutdown_connections 2024-09-16T15:44:02.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:02.376+0000 7fbbaf800640 1 -- 172.21.15.151:0/2146978658 wait complete. 2024-09-16T15:44:02.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:02.394 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/e5d6f34b-74d3-4dc6-ae17-74c26374cc07 2024-09-16T15:44:02.395 DEBUG:teuthology.orchestra.run.smithi151:> ) 2024-09-16T15:44:02.431 INFO:teuthology.orchestra.run.smithi151.stdout:# file: volumes/_nogroup/sv1/e5d6f34b-74d3-4dc6-ae17-74c26374cc07 2024-09-16T15:44:02.431 INFO:teuthology.orchestra.run.smithi151.stdout:ceph.dir.rbytes="10405347328" 2024-09-16T15:44:02.431 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:44:02.431 INFO:tasks.cephfs.test_volumes:r_size = 10405347328 exp_size = 10737418240 2024-09-16T15:44:04.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:04.432 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/e5d6f34b-74d3-4dc6-ae17-74c26374cc07 2024-09-16T15:44:04.432 DEBUG:teuthology.orchestra.run.smithi151:> ) 2024-09-16T15:44:04.472 INFO:teuthology.orchestra.run.smithi151.stdout:# file: volumes/_nogroup/sv1/e5d6f34b-74d3-4dc6-ae17-74c26374cc07 2024-09-16T15:44:04.472 INFO:teuthology.orchestra.run.smithi151.stdout:ceph.dir.rbytes="10405347328" 2024-09-16T15:44:04.472 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:44:04.473 INFO:tasks.cephfs.test_volumes:r_size = 10405347328 exp_size = 10737418240 2024-09-16T15:44:06.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:06.474 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/e5d6f34b-74d3-4dc6-ae17-74c26374cc07 2024-09-16T15:44:06.474 DEBUG:teuthology.orchestra.run.smithi151:> ) 2024-09-16T15:44:06.514 INFO:teuthology.orchestra.run.smithi151.stdout:# file: volumes/_nogroup/sv1/e5d6f34b-74d3-4dc6-ae17-74c26374cc07 2024-09-16T15:44:06.514 INFO:teuthology.orchestra.run.smithi151.stdout:ceph.dir.rbytes="10737418240" 2024-09-16T15:44:06.515 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:44:06.515 INFO:tasks.cephfs.test_volumes:r_size = 10737418240 exp_size = 10737418240 2024-09-16T15:44:06.516 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone27010 2024-09-16T15:44:06.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:06.721+0000 7fa323a00640 1 -- 172.21.15.151:0/3536881079 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa31c0cec60 msgr2=0x7fa31c0cf070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:06.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:06.721+0000 7fa323a00640 1 --2- 172.21.15.151:0/3536881079 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa31c0cec60 0x7fa31c0cf070 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fa314005fd0 tx=0x7fa31402a940 comp rx=0 tx=0).stop 2024-09-16T15:44:06.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:06.722+0000 7fa323a00640 1 -- 172.21.15.151:0/3536881079 shutdown_connections 2024-09-16T15:44:06.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:06.722+0000 7fa323a00640 1 --2- 172.21.15.151:0/3536881079 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa31c0cec60 0x7fa31c0cf070 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:06.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:06.722+0000 7fa323a00640 1 -- 172.21.15.151:0/3536881079 >> 172.21.15.151:0/3536881079 conn(0x7fa31c0b1840 msgr2=0x7fa31c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:06.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:06.722+0000 7fa323a00640 1 -- 172.21.15.151:0/3536881079 shutdown_connections 2024-09-16T15:44:06.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:06.722+0000 7fa323a00640 1 -- 172.21.15.151:0/3536881079 wait complete. 2024-09-16T15:44:06.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:06.723+0000 7fa323a00640 1 Processor -- start 2024-09-16T15:44:06.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:06.723+0000 7fa323a00640 1 -- start start 2024-09-16T15:44:06.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:06.724+0000 7fa323a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa31c0af190 0x7fa31c1593f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:06.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:06.724+0000 7fa323a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa31c0cec60 0x7fa31c14e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:06.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:06.724+0000 7fa323a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa31c161760 0x7fa31c15fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:06.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:06.724+0000 7fa323a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa31c1623a0 con 0x7fa31c161760 2024-09-16T15:44:06.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:06.724+0000 7fa323a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa31c070530 con 0x7fa31c0cec60 2024-09-16T15:44:06.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:06.724+0000 7fa323a00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fa31c162520 con 0x7fa31c0af190 2024-09-16T15:44:06.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:06.725+0000 7fa321a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa31c161760 0x7fa31c15fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:06.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:06.725+0000 7fa321a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa31c161760 0x7fa31c15fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:54962/0 (socket says 172.21.15.151:54962) 2024-09-16T15:44:06.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:06.725+0000 7fa321a00640 1 -- 172.21.15.151:0/541304064 learned_addr learned my addr 172.21.15.151:0/541304064 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:06.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:06.725+0000 7fa321000640 1 --2- 172.21.15.151:0/541304064 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa31c0af190 0x7fa31c1593f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:06.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:06.725+0000 7fa31be00640 1 --2- 172.21.15.151:0/541304064 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa31c0cec60 0x7fa31c14e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:06.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:06.725+0000 7fa321a00640 1 -- 172.21.15.151:0/541304064 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa31c0af190 msgr2=0x7fa31c1593f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:06.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:06.725+0000 7fa321a00640 1 --2- 172.21.15.151:0/541304064 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa31c0af190 0x7fa31c1593f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:06.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:06.725+0000 7fa321a00640 1 -- 172.21.15.151:0/541304064 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa31c0cec60 msgr2=0x7fa31c14e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:06.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:06.725+0000 7fa321a00640 1 --2- 172.21.15.151:0/541304064 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa31c0cec60 0x7fa31c14e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:06.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:06.725+0000 7fa321a00640 1 -- 172.21.15.151:0/541304064 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa31c1f87a0 con 0x7fa31c161760 2024-09-16T15:44:06.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:06.725+0000 7fa321a00640 1 --2- 172.21.15.151:0/541304064 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa31c161760 0x7fa31c15fda0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa314006010 tx=0x7fa314031e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:06.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:06.726+0000 7fa318c00640 1 -- 172.21.15.151:0/541304064 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa314004240 con 0x7fa31c161760 2024-09-16T15:44:06.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:06.726+0000 7fa318c00640 1 -- 172.21.15.151:0/541304064 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa3140043e0 con 0x7fa31c161760 2024-09-16T15:44:06.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:06.726+0000 7fa318c00640 1 -- 172.21.15.151:0/541304064 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa314004830 con 0x7fa31c161760 2024-09-16T15:44:06.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:06.726+0000 7fa323a00640 1 -- 172.21.15.151:0/541304064 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa31c1f8a40 con 0x7fa31c161760 2024-09-16T15:44:06.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:06.726+0000 7fa323a00640 1 -- 172.21.15.151:0/541304064 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa31c1f9000 con 0x7fa31c161760 2024-09-16T15:44:06.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:06.727+0000 7fa323a00640 1 -- 172.21.15.151:0/541304064 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa31c1629d0 con 0x7fa31c161760 2024-09-16T15:44:06.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:06.728+0000 7fa318c00640 1 -- 172.21.15.151:0/541304064 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 14) ==== 101644+0+0 (secure 0 0 0) 0x7fa314039440 con 0x7fa31c161760 2024-09-16T15:44:06.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:06.729+0000 7fa318c00640 1 --2- 172.21.15.151:0/541304064 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa2fc078c40 0x7fa2fc07b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:06.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:06.729+0000 7fa318c00640 1 -- 172.21.15.151:0/541304064 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5854+0+0 (secure 0 0 0) 0x7fa3140bccd0 con 0x7fa31c161760 2024-09-16T15:44:06.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:06.729+0000 7fa321000640 1 --2- 172.21.15.151:0/541304064 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa2fc078c40 0x7fa2fc07b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:06.732 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:06.730+0000 7fa321000640 1 --2- 172.21.15.151:0/541304064 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa2fc078c40 0x7fa2fc07b100 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa30c000f60 tx=0x7fa30c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:06.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:06.733+0000 7fa318c00640 1 -- 172.21.15.151:0/541304064 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa314084b80 con 0x7fa31c161760 2024-09-16T15:44:06.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:06.951+0000 7fa323a00640 1 -- 172.21.15.151:0/541304064 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone27010", "target": ["mon-mgr", ""]}) -- 0x7fa31c0b4a90 con 0x7fa2fc078c40 2024-09-16T15:44:07.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:07.029+0000 7fa318c00640 1 -- 172.21.15.151:0/541304064 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa31c0b4a90 con 0x7fa2fc078c40 2024-09-16T15:44:07.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:07.031+0000 7fa323a00640 1 -- 172.21.15.151:0/541304064 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa2fc078c40 msgr2=0x7fa2fc07b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:07.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:07.031+0000 7fa323a00640 1 --2- 172.21.15.151:0/541304064 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa2fc078c40 0x7fa2fc07b100 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa30c000f60 tx=0x7fa30c009040 comp rx=0 tx=0).stop 2024-09-16T15:44:07.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:07.031+0000 7fa323a00640 1 -- 172.21.15.151:0/541304064 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa31c161760 msgr2=0x7fa31c15fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:07.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:07.031+0000 7fa323a00640 1 --2- 172.21.15.151:0/541304064 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa31c161760 0x7fa31c15fda0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa314006010 tx=0x7fa314031e20 comp rx=0 tx=0).stop 2024-09-16T15:44:07.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:07.031+0000 7fa323a00640 1 -- 172.21.15.151:0/541304064 shutdown_connections 2024-09-16T15:44:07.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:07.031+0000 7fa323a00640 1 --2- 172.21.15.151:0/541304064 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa2fc078c40 0x7fa2fc07b100 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:07.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:07.031+0000 7fa323a00640 1 --2- 172.21.15.151:0/541304064 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa31c0cec60 0x7fa31c14e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:07.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:07.031+0000 7fa323a00640 1 --2- 172.21.15.151:0/541304064 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa31c161760 0x7fa31c15fda0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:07.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:07.031+0000 7fa323a00640 1 --2- 172.21.15.151:0/541304064 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa31c0af190 0x7fa31c1593f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:07.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:07.031+0000 7fa323a00640 1 -- 172.21.15.151:0/541304064 >> 172.21.15.151:0/541304064 conn(0x7fa31c0b1840 msgr2=0x7fa31c072810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:07.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:07.032+0000 7fa323a00640 1 -- 172.21.15.151:0/541304064 shutdown_connections 2024-09-16T15:44:07.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:07.032+0000 7fa323a00640 1 -- 172.21.15.151:0/541304064 wait complete. 2024-09-16T15:44:07.035 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:44:07.033+0000 7f48f6200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:07.035 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:44:07.033+0000 7f48f6200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:07.035 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:44:07.033+0000 7f48f6200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:07.035 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:44:07.033+0000 7f48f6200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:07.035 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:44:07.033+0000 7f48f6200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:07.051 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone48851 2024-09-16T15:44:07.052 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:44:07.051+0000 7f48f7600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:07.052 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:44:07.051+0000 7f48f7600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:07.052 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:44:07.051+0000 7f48f7600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:07.052 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:44:07.051+0000 7f48f7600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:07.052 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:44:07.051+0000 7f48f7600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:07.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:07.216+0000 7f3f2d600640 1 -- 172.21.15.151:0/2383606481 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3f280cccd0 msgr2=0x7f3f280cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:07.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:07.216+0000 7f3f2d600640 1 --2- 172.21.15.151:0/2383606481 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3f280cccd0 0x7f3f280cdbb0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3f20005fd0 tx=0x7f3f2002a820 comp rx=0 tx=0).stop 2024-09-16T15:44:07.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:07.216+0000 7f3f2d600640 1 -- 172.21.15.151:0/2383606481 shutdown_connections 2024-09-16T15:44:07.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:07.216+0000 7f3f2d600640 1 --2- 172.21.15.151:0/2383606481 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3f280cccd0 0x7f3f280cdbb0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:07.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:07.216+0000 7f3f2d600640 1 --2- 172.21.15.151:0/2383606481 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3f28145750 0x7f3f28073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:07.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:07.216+0000 7f3f2d600640 1 -- 172.21.15.151:0/2383606481 >> 172.21.15.151:0/2383606481 conn(0x7f3f280b1840 msgr2=0x7f3f280b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:07.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:07.216+0000 7f3f2d600640 1 -- 172.21.15.151:0/2383606481 shutdown_connections 2024-09-16T15:44:07.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:07.216+0000 7f3f2d600640 1 -- 172.21.15.151:0/2383606481 wait complete. 2024-09-16T15:44:07.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:07.217+0000 7f3f2d600640 1 Processor -- start 2024-09-16T15:44:07.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:07.218+0000 7f3f2d600640 1 -- start start 2024-09-16T15:44:07.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:07.219+0000 7f3f2d600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3f2806efa0 0x7f3f280adf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:07.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:07.219+0000 7f3f2d600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3f280cccd0 0x7f3f281581a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:07.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:07.219+0000 7f3f2d600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3f28145750 0x7f3f280ab500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:07.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:07.219+0000 7f3f2d600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3f280ce480 con 0x7f3f280cccd0 2024-09-16T15:44:07.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:07.219+0000 7f3f2d600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3f280ce300 con 0x7f3f2806efa0 2024-09-16T15:44:07.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:07.219+0000 7f3f2d600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f3f280ce600 con 0x7f3f28145750 2024-09-16T15:44:07.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:07.219+0000 7f3f26a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3f2806efa0 0x7f3f280adf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:07.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:07.219+0000 7f3f26000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3f280cccd0 0x7f3f281581a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:07.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:07.219+0000 7f3f26a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3f2806efa0 0x7f3f280adf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.151:34474/0 (socket says 172.21.15.151:34474) 2024-09-16T15:44:07.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:07.219+0000 7f3f26a00640 1 -- 172.21.15.151:0/1708019116 learned_addr learned my addr 172.21.15.151:0/1708019116 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:07.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:07.219+0000 7f3f27400640 1 --2- 172.21.15.151:0/1708019116 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3f28145750 0x7f3f280ab500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:07.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:07.219+0000 7f3f26a00640 1 -- 172.21.15.151:0/1708019116 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3f28145750 msgr2=0x7f3f280ab500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:07.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:07.219+0000 7f3f26a00640 1 --2- 172.21.15.151:0/1708019116 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3f28145750 0x7f3f280ab500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:07.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:07.219+0000 7f3f26a00640 1 -- 172.21.15.151:0/1708019116 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3f280cccd0 msgr2=0x7f3f281581a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:07.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:07.219+0000 7f3f26a00640 1 --2- 172.21.15.151:0/1708019116 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3f280cccd0 0x7f3f281581a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:07.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:07.219+0000 7f3f26a00640 1 -- 172.21.15.151:0/1708019116 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f280b2de0 con 0x7f3f2806efa0 2024-09-16T15:44:07.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:07.220+0000 7f3f26a00640 1 --2- 172.21.15.151:0/1708019116 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3f2806efa0 0x7f3f280adf20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3f1800e9c0 tx=0x7f3f1800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:07.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:07.220+0000 7f3f1ea00640 1 -- 172.21.15.151:0/1708019116 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f1800cd80 con 0x7f3f2806efa0 2024-09-16T15:44:07.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:07.220+0000 7f3f1ea00640 1 -- 172.21.15.151:0/1708019116 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3f18004020 con 0x7f3f2806efa0 2024-09-16T15:44:07.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:07.220+0000 7f3f1ea00640 1 -- 172.21.15.151:0/1708019116 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f18004450 con 0x7f3f2806efa0 2024-09-16T15:44:07.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:07.220+0000 7f3f2d600640 1 -- 172.21.15.151:0/1708019116 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f280ae500 con 0x7f3f2806efa0 2024-09-16T15:44:07.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:07.220+0000 7f3f2d600640 1 -- 172.21.15.151:0/1708019116 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f280aeac0 con 0x7f3f2806efa0 2024-09-16T15:44:07.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:07.221+0000 7f3f2d600640 1 -- 172.21.15.151:0/1708019116 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f280cdbb0 con 0x7f3f2806efa0 2024-09-16T15:44:07.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:07.222+0000 7f3f1ea00640 1 -- 172.21.15.151:0/1708019116 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 14) ==== 101644+0+0 (secure 0 0 0) 0x7f3f180035f0 con 0x7f3f2806efa0 2024-09-16T15:44:07.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:07.223+0000 7f3f1ea00640 1 --2- 172.21.15.151:0/1708019116 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3f08078c40 0x7f3f0807b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:07.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:07.223+0000 7f3f1ea00640 1 -- 172.21.15.151:0/1708019116 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5854+0+0 (secure 0 0 0) 0x7f3f1809b980 con 0x7f3f2806efa0 2024-09-16T15:44:07.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:07.223+0000 7f3f26000640 1 --2- 172.21.15.151:0/1708019116 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3f08078c40 0x7f3f0807b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:07.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:07.224+0000 7f3f26000640 1 --2- 172.21.15.151:0/1708019116 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3f08078c40 0x7f3f0807b100 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3f280ad130 tx=0x7f3f14009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:07.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:07.228+0000 7f3f1ea00640 1 -- 172.21.15.151:0/1708019116 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f3f180632b0 con 0x7f3f2806efa0 2024-09-16T15:44:07.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:07.450+0000 7f3f2d600640 1 -- 172.21.15.151:0/1708019116 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone48851", "target": ["mon-mgr", ""]}) -- 0x7f3f281435a0 con 0x7f3f08078c40 2024-09-16T15:44:08.036 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:44:08.034+0000 7f48e4a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:08.036 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:44:08.034+0000 7f48e4a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:08.036 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:44:08.034+0000 7f48e4a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:08.036 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:44:08.034+0000 7f48e4a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:08.036 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:44:08.034+0000 7f48e4a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:08.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:08.126+0000 7f3f1ea00640 1 -- 172.21.15.151:0/1708019116 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3f281435a0 con 0x7f3f08078c40 2024-09-16T15:44:08.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:08.129+0000 7f3f2d600640 1 -- 172.21.15.151:0/1708019116 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3f08078c40 msgr2=0x7f3f0807b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:08.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:08.129+0000 7f3f2d600640 1 --2- 172.21.15.151:0/1708019116 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3f08078c40 0x7f3f0807b100 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3f280ad130 tx=0x7f3f14009040 comp rx=0 tx=0).stop 2024-09-16T15:44:08.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:08.129+0000 7f3f2d600640 1 -- 172.21.15.151:0/1708019116 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3f2806efa0 msgr2=0x7f3f280adf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:08.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:08.129+0000 7f3f2d600640 1 --2- 172.21.15.151:0/1708019116 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3f2806efa0 0x7f3f280adf20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3f1800e9c0 tx=0x7f3f1800ee80 comp rx=0 tx=0).stop 2024-09-16T15:44:08.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:08.129+0000 7f3f2d600640 1 -- 172.21.15.151:0/1708019116 shutdown_connections 2024-09-16T15:44:08.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:08.129+0000 7f3f2d600640 1 --2- 172.21.15.151:0/1708019116 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3f08078c40 0x7f3f0807b100 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:08.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:08.129+0000 7f3f2d600640 1 --2- 172.21.15.151:0/1708019116 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3f28145750 0x7f3f280ab500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:08.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:08.129+0000 7f3f2d600640 1 --2- 172.21.15.151:0/1708019116 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3f280cccd0 0x7f3f281581a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:08.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:08.129+0000 7f3f2d600640 1 --2- 172.21.15.151:0/1708019116 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3f2806efa0 0x7f3f280adf20 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:08.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:08.129+0000 7f3f2d600640 1 -- 172.21.15.151:0/1708019116 >> 172.21.15.151:0/1708019116 conn(0x7f3f280b1840 msgr2=0x7f3f28143da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:08.133 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:08.130+0000 7f3f2d600640 1 -- 172.21.15.151:0/1708019116 shutdown_connections 2024-09-16T15:44:08.133 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:08.130+0000 7f3f2d600640 1 -- 172.21.15.151:0/1708019116 wait complete. 2024-09-16T15:44:08.157 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone92812 2024-09-16T15:44:08.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:08.394+0000 7f910d000640 1 -- 172.21.15.151:0/3060680038 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f91081437c0 msgr2=0x7f91080cdad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:08.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:08.394+0000 7f910d000640 1 --2- 172.21.15.151:0/3060680038 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f91081437c0 0x7f91080cdad0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f90f8005fd0 tx=0x7f90f802bbc0 comp rx=0 tx=0).stop 2024-09-16T15:44:08.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:08.394+0000 7f910d000640 1 -- 172.21.15.151:0/3060680038 shutdown_connections 2024-09-16T15:44:08.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:08.394+0000 7f910d000640 1 --2- 172.21.15.151:0/3060680038 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f91081437c0 0x7f91080cdad0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:08.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:08.394+0000 7f910d000640 1 -- 172.21.15.151:0/3060680038 >> 172.21.15.151:0/3060680038 conn(0x7f91080b1840 msgr2=0x7f91080b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:08.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:08.394+0000 7f910d000640 1 -- 172.21.15.151:0/3060680038 shutdown_connections 2024-09-16T15:44:08.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:08.394+0000 7f910d000640 1 -- 172.21.15.151:0/3060680038 wait complete. 2024-09-16T15:44:08.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:08.396+0000 7f910d000640 1 Processor -- start 2024-09-16T15:44:08.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:08.396+0000 7f910d000640 1 -- start start 2024-09-16T15:44:08.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:08.398+0000 7f910d000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91080b45d0 0x7f91080cc230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:08.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:08.398+0000 7f910d000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f91080b4bd0 0x7f91081f7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:08.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:08.398+0000 7f910d000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f91081437c0 0x7f91081f8420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:08.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:08.398+0000 7f910d000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f910806f0f0 con 0x7f91081437c0 2024-09-16T15:44:08.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:08.398+0000 7f910d000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f910806ef70 con 0x7f91080b45d0 2024-09-16T15:44:08.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:08.398+0000 7f910d000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f910806f270 con 0x7f91080b4bd0 2024-09-16T15:44:08.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:08.398+0000 7f9106800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f91081437c0 0x7f91081f8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:08.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:08.398+0000 7f9106800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f91081437c0 0x7f91081f8420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:54990/0 (socket says 172.21.15.151:54990) 2024-09-16T15:44:08.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:08.398+0000 7f9106800640 1 -- 172.21.15.151:0/3971838450 learned_addr learned my addr 172.21.15.151:0/3971838450 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:08.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:08.398+0000 7f9105e00640 1 --2- 172.21.15.151:0/3971838450 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91080b45d0 0x7f91080cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:08.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:08.399+0000 7f9106800640 1 -- 172.21.15.151:0/3971838450 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f91080b4bd0 msgr2=0x7f91081f7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:08.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:08.399+0000 7f9106800640 1 --2- 172.21.15.151:0/3971838450 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f91080b4bd0 0x7f91081f7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:08.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:08.399+0000 7f9106800640 1 -- 172.21.15.151:0/3971838450 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91080b45d0 msgr2=0x7f91080cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:08.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:08.399+0000 7f9106800640 1 --2- 172.21.15.151:0/3971838450 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91080b45d0 0x7f91080cc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:08.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:08.399+0000 7f9106800640 1 -- 172.21.15.151:0/3971838450 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91080aaaf0 con 0x7f91081437c0 2024-09-16T15:44:08.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:08.399+0000 7f9106800640 1 --2- 172.21.15.151:0/3971838450 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f91081437c0 0x7f91081f8420 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f910000c3c0 tx=0x7f910000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:08.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:08.399+0000 7f90fe000640 1 -- 172.21.15.151:0/3971838450 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9100017070 con 0x7f91081437c0 2024-09-16T15:44:08.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:08.400+0000 7f90fe000640 1 -- 172.21.15.151:0/3971838450 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f91000035f0 con 0x7f91081437c0 2024-09-16T15:44:08.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:08.400+0000 7f910d000640 1 -- 172.21.15.151:0/3971838450 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91080aadd0 con 0x7f91081437c0 2024-09-16T15:44:08.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:08.400+0000 7f910d000640 1 -- 172.21.15.151:0/3971838450 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91081f5e30 con 0x7f91081437c0 2024-09-16T15:44:08.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:08.400+0000 7f90fe000640 1 -- 172.21.15.151:0/3971838450 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9100003a40 con 0x7f91081437c0 2024-09-16T15:44:08.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:08.401+0000 7f910d000640 1 -- 172.21.15.151:0/3971838450 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90e0005290 con 0x7f91081437c0 2024-09-16T15:44:08.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:08.403+0000 7f90fe000640 1 -- 172.21.15.151:0/3971838450 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 15) ==== 101746+0+0 (secure 0 0 0) 0x7f910001a070 con 0x7f91081437c0 2024-09-16T15:44:08.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:08.404+0000 7f90fe000640 1 --2- 172.21.15.151:0/3971838450 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f90e8078d90 0x7f90e807b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:08.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:08.404+0000 7f90fe000640 1 -- 172.21.15.151:0/3971838450 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5854+0+0 (secure 0 0 0) 0x7f910009a650 con 0x7f91081437c0 2024-09-16T15:44:08.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:08.408+0000 7f90fe000640 1 -- 172.21.15.151:0/3971838450 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9100062350 con 0x7f91081437c0 2024-09-16T15:44:08.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:08.415+0000 7f9105e00640 1 --2- 172.21.15.151:0/3971838450 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f90e8078d90 0x7f90e807b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:08.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:08.420+0000 7f9105e00640 1 --2- 172.21.15.151:0/3971838450 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f90e8078d90 0x7f90e807b250 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f90f8035000 tx=0x7f90f8004ab0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:08.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:08.753+0000 7f910d000640 1 -- 172.21.15.151:0/3971838450 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone92812", "target": ["mon-mgr", ""]}) -- 0x7f90e0000c80 con 0x7f90e8078d90 2024-09-16T15:44:09.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:09.166+0000 7f90fe000640 1 -- 172.21.15.151:0/3971838450 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f90e0000c80 con 0x7f90e8078d90 2024-09-16T15:44:09.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:09.169+0000 7f910d000640 1 -- 172.21.15.151:0/3971838450 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f90e8078d90 msgr2=0x7f90e807b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:09.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:09.169+0000 7f910d000640 1 --2- 172.21.15.151:0/3971838450 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f90e8078d90 0x7f90e807b250 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f90f8035000 tx=0x7f90f8004ab0 comp rx=0 tx=0).stop 2024-09-16T15:44:09.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:09.169+0000 7f910d000640 1 -- 172.21.15.151:0/3971838450 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f91081437c0 msgr2=0x7f91081f8420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:09.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:09.169+0000 7f910d000640 1 --2- 172.21.15.151:0/3971838450 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f91081437c0 0x7f91081f8420 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f910000c3c0 tx=0x7f910000c880 comp rx=0 tx=0).stop 2024-09-16T15:44:09.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:09.169+0000 7f910d000640 1 -- 172.21.15.151:0/3971838450 shutdown_connections 2024-09-16T15:44:09.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:09.169+0000 7f910d000640 1 --2- 172.21.15.151:0/3971838450 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f90e8078d90 0x7f90e807b250 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:09.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:09.169+0000 7f910d000640 1 --2- 172.21.15.151:0/3971838450 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f91081437c0 0x7f91081f8420 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:09.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:09.169+0000 7f910d000640 1 --2- 172.21.15.151:0/3971838450 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f91080b4bd0 0x7f91081f7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:09.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:09.169+0000 7f910d000640 1 --2- 172.21.15.151:0/3971838450 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91080b45d0 0x7f91080cc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:09.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:09.169+0000 7f910d000640 1 -- 172.21.15.151:0/3971838450 >> 172.21.15.151:0/3971838450 conn(0x7f91080b1840 msgr2=0x7f9108072780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:09.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:09.170+0000 7f910d000640 1 -- 172.21.15.151:0/3971838450 shutdown_connections 2024-09-16T15:44:09.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:09.170+0000 7f910d000640 1 -- 172.21.15.151:0/3971838450 wait complete. 2024-09-16T15:44:09.192 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone88453 2024-09-16T15:44:09.229 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:44:09.225+0000 7f48f6c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:09.229 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:44:09.225+0000 7f48f6c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:09.229 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:44:09.225+0000 7f48f6c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:09.229 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:44:09.225+0000 7f48f6c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:09.229 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:44:09.225+0000 7f48f6c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:09.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:09.459+0000 7fafd8600640 1 -- 172.21.15.151:0/4158035839 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fafd00cccd0 msgr2=0x7fafd00cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:09.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:09.459+0000 7fafd8600640 1 --2- 172.21.15.151:0/4158035839 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fafd00cccd0 0x7fafd00cdbb0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fafd00d0cd0 tx=0x7fafc802a830 comp rx=0 tx=0).stop 2024-09-16T15:44:09.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:09.460+0000 7fafd8600640 1 -- 172.21.15.151:0/4158035839 shutdown_connections 2024-09-16T15:44:09.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:09.460+0000 7fafd8600640 1 --2- 172.21.15.151:0/4158035839 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fafd00cccd0 0x7fafd00cdbb0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:09.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:09.460+0000 7fafd8600640 1 --2- 172.21.15.151:0/4158035839 >> v2:172.21.15.167:3300/0 conn(0x7fafd0145750 0x7fafd0073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:09.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:09.460+0000 7fafd8600640 1 -- 172.21.15.151:0/4158035839 >> 172.21.15.151:0/4158035839 conn(0x7fafd00b1840 msgr2=0x7fafd00b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:09.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:09.460+0000 7fafd8600640 1 -- 172.21.15.151:0/4158035839 shutdown_connections 2024-09-16T15:44:09.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:09.460+0000 7fafd8600640 1 -- 172.21.15.151:0/4158035839 wait complete. 2024-09-16T15:44:09.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:09.462+0000 7fafd8600640 1 Processor -- start 2024-09-16T15:44:09.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:09.462+0000 7fafd8600640 1 -- start start 2024-09-16T15:44:09.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:09.463+0000 7fafd8600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fafd006efa0 0x7fafd00ad160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:09.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:09.463+0000 7fafd8600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fafd00cccd0 0x7fafd01fc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:09.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:09.463+0000 7fafd8600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafd0145750 0x7fafd0140ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:09.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:09.463+0000 7fafd8600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fafd00ce480 con 0x7fafd006efa0 2024-09-16T15:44:09.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:09.463+0000 7fafd8600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fafd00ce300 con 0x7fafd0145750 2024-09-16T15:44:09.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:09.463+0000 7fafd8600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fafd00ce600 con 0x7fafd00cccd0 2024-09-16T15:44:09.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:09.464+0000 7fafd5c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fafd006efa0 0x7fafd00ad160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:09.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:09.464+0000 7fafd5c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fafd006efa0 0x7fafd00ad160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:55008/0 (socket says 172.21.15.151:55008) 2024-09-16T15:44:09.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:09.464+0000 7fafd5c00640 1 -- 172.21.15.151:0/1104733803 learned_addr learned my addr 172.21.15.151:0/1104733803 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:09.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:09.464+0000 7fafd5c00640 1 -- 172.21.15.151:0/1104733803 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fafd00cccd0 msgr2=0x7fafd01fc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:09.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:09.464+0000 7fafd5c00640 1 --2- 172.21.15.151:0/1104733803 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fafd00cccd0 0x7fafd01fc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:09.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:09.464+0000 7fafd5c00640 1 -- 172.21.15.151:0/1104733803 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafd0145750 msgr2=0x7fafd0140ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:09.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:09.464+0000 7fafd5c00640 1 --2- 172.21.15.151:0/1104733803 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafd0145750 0x7fafd0140ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:09.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:09.464+0000 7fafd5c00640 1 -- 172.21.15.151:0/1104733803 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafd01fccd0 con 0x7fafd006efa0 2024-09-16T15:44:09.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:09.464+0000 7fafd5c00640 1 --2- 172.21.15.151:0/1104733803 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fafd006efa0 0x7fafd00ad160 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fafc400b410 tx=0x7fafc400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:09.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:09.465+0000 7fafcd600640 1 -- 172.21.15.151:0/1104733803 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc4015020 con 0x7fafd006efa0 2024-09-16T15:44:09.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:09.465+0000 7fafcd600640 1 -- 172.21.15.151:0/1104733803 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fafc4011920 con 0x7fafd006efa0 2024-09-16T15:44:09.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:09.465+0000 7fafd8600640 1 -- 172.21.15.151:0/1104733803 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafd01fcfb0 con 0x7fafd006efa0 2024-09-16T15:44:09.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:09.465+0000 7fafcd600640 1 -- 172.21.15.151:0/1104733803 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc4011d70 con 0x7fafd006efa0 2024-09-16T15:44:09.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:09.465+0000 7fafd8600640 1 -- 172.21.15.151:0/1104733803 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafd015cdd0 con 0x7fafd006efa0 2024-09-16T15:44:09.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:09.466+0000 7fafd8600640 1 -- 172.21.15.151:0/1104733803 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafd00cdbb0 con 0x7fafd006efa0 2024-09-16T15:44:09.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:09.468+0000 7fafcd600640 1 -- 172.21.15.151:0/1104733803 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 15) ==== 101746+0+0 (secure 0 0 0) 0x7fafc4018020 con 0x7fafd006efa0 2024-09-16T15:44:09.471 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:09.469+0000 7fafcd600640 1 --2- 172.21.15.151:0/1104733803 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fafb0078d90 0x7fafb007b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:09.471 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:09.469+0000 7fafcd600640 1 -- 172.21.15.151:0/1104733803 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5854+0+0 (secure 0 0 0) 0x7fafc409b350 con 0x7fafd006efa0 2024-09-16T15:44:09.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:09.471+0000 7fafd5200640 1 --2- 172.21.15.151:0/1104733803 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fafb0078d90 0x7fafb007b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:09.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:09.474+0000 7fafcd600640 1 -- 172.21.15.151:0/1104733803 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fafc401f020 con 0x7fafd006efa0 2024-09-16T15:44:09.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:09.475+0000 7fafd5200640 1 --2- 172.21.15.151:0/1104733803 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fafb0078d90 0x7fafb007b250 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fafd0141ba0 tx=0x7fafbc0079f0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:09.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:09.834+0000 7fafd8600640 1 -- 172.21.15.151:0/1104733803 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone88453", "target": ["mon-mgr", ""]}) -- 0x7fafd00af0f0 con 0x7fafb0078d90 2024-09-16T15:44:10.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:10.387+0000 7fafcd600640 1 -- 172.21.15.151:0/1104733803 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mgrmap(e 16) ==== 101848+0+0 (secure 0 0 0) 0x7fafc405ebd0 con 0x7fafd006efa0 2024-09-16T15:44:11.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:11.778+0000 7fafcd600640 1 -- 172.21.15.151:0/1104733803 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fafd00af0f0 con 0x7fafb0078d90 2024-09-16T15:44:11.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:11.780+0000 7fafd8600640 1 -- 172.21.15.151:0/1104733803 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fafb0078d90 msgr2=0x7fafb007b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:11.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:11.780+0000 7fafd8600640 1 --2- 172.21.15.151:0/1104733803 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fafb0078d90 0x7fafb007b250 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fafd0141ba0 tx=0x7fafbc0079f0 comp rx=0 tx=0).stop 2024-09-16T15:44:11.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:11.780+0000 7fafd8600640 1 -- 172.21.15.151:0/1104733803 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fafd006efa0 msgr2=0x7fafd00ad160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:11.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:11.780+0000 7fafd8600640 1 --2- 172.21.15.151:0/1104733803 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fafd006efa0 0x7fafd00ad160 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fafc400b410 tx=0x7fafc400b8d0 comp rx=0 tx=0).stop 2024-09-16T15:44:11.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:11.780+0000 7fafd8600640 1 -- 172.21.15.151:0/1104733803 shutdown_connections 2024-09-16T15:44:11.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:11.780+0000 7fafd8600640 1 --2- 172.21.15.151:0/1104733803 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fafb0078d90 0x7fafb007b250 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:11.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:11.780+0000 7fafd8600640 1 --2- 172.21.15.151:0/1104733803 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fafd0145750 0x7fafd0140ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:11.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:11.780+0000 7fafd8600640 1 --2- 172.21.15.151:0/1104733803 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fafd00cccd0 0x7fafd01fc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:11.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:11.780+0000 7fafd8600640 1 --2- 172.21.15.151:0/1104733803 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fafd006efa0 0x7fafd00ad160 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:11.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:11.780+0000 7fafd8600640 1 -- 172.21.15.151:0/1104733803 >> 172.21.15.151:0/1104733803 conn(0x7fafd00b1840 msgr2=0x7fafd006f410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:11.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:11.780+0000 7fafd8600640 1 -- 172.21.15.151:0/1104733803 shutdown_connections 2024-09-16T15:44:11.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:11.780+0000 7fafd8600640 1 -- 172.21.15.151:0/1104733803 wait complete. 2024-09-16T15:44:11.807 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-16T15:44:11.819 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:44:11.810+0000 7f48f7600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:11.819 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:44:11.810+0000 7f48f7600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:11.819 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:44:11.810+0000 7f48f7600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:11.819 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:44:11.810+0000 7f48f7600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:11.819 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:44:11.810+0000 7f48f7600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:12.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.069+0000 7fc5b5a00640 1 -- 172.21.15.151:0/3606216342 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc594003a00 msgr2=0x7fc594003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:12.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.069+0000 7fc5b5a00640 1 --2- 172.21.15.151:0/3606216342 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc594003a00 0x7fc594003e00 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0006fd0 tx=0x7fc594005830 comp rx=0 tx=0).stop 2024-09-16T15:44:12.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.069+0000 7fc5b5a00640 1 -- 172.21.15.151:0/3606216342 shutdown_connections 2024-09-16T15:44:12.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.069+0000 7fc5b5a00640 1 --2- 172.21.15.151:0/3606216342 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc594003a00 0x7fc594003e00 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:12.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.069+0000 7fc5b5a00640 1 -- 172.21.15.151:0/3606216342 >> 172.21.15.151:0/3606216342 conn(0x7fc5b00b18e0 msgr2=0x7fc5b00b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:12.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.069+0000 7fc5b5a00640 1 -- 172.21.15.151:0/3606216342 shutdown_connections 2024-09-16T15:44:12.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.069+0000 7fc5b5a00640 1 -- 172.21.15.151:0/3606216342 wait complete. 2024-09-16T15:44:12.073 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.071+0000 7fc5b5a00640 1 Processor -- start 2024-09-16T15:44:12.073 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.071+0000 7fc5b5a00640 1 -- start start 2024-09-16T15:44:12.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.072+0000 7fc5b5a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc594003a00 0x7fc5b0159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:12.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.072+0000 7fc5b5a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5b009b230 0x7fc5b015a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:12.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.072+0000 7fc5b5a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5b0155480 0x7fc5b00ad510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:12.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.072+0000 7fc5b5a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b0160080 con 0x7fc594003a00 2024-09-16T15:44:12.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.072+0000 7fc5b5a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b015ff00 con 0x7fc5b009b230 2024-09-16T15:44:12.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.072+0000 7fc5b5a00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fc5b0160200 con 0x7fc5b0155480 2024-09-16T15:44:12.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.073+0000 7fc5aea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc594003a00 0x7fc5b0159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:12.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.073+0000 7fc5aea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc594003a00 0x7fc5b0159ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:55026/0 (socket says 172.21.15.151:55026) 2024-09-16T15:44:12.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.073+0000 7fc5aea00640 1 -- 172.21.15.151:0/1434141777 learned_addr learned my addr 172.21.15.151:0/1434141777 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:12.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.073+0000 7fc5aea00640 1 -- 172.21.15.151:0/1434141777 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5b0155480 msgr2=0x7fc5b00ad510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:12.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.073+0000 7fc5aea00640 1 --2- 172.21.15.151:0/1434141777 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5b0155480 0x7fc5b00ad510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:12.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.073+0000 7fc5aea00640 1 -- 172.21.15.151:0/1434141777 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5b009b230 msgr2=0x7fc5b015a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:12.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.073+0000 7fc5aea00640 1 --2- 172.21.15.151:0/1434141777 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5b009b230 0x7fc5b015a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:12.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.073+0000 7fc5aea00640 1 -- 172.21.15.151:0/1434141777 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5b0155c10 con 0x7fc594003a00 2024-09-16T15:44:12.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.074+0000 7fc5aea00640 1 --2- 172.21.15.151:0/1434141777 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc594003a00 0x7fc5b0159ee0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fc5a002b490 tx=0x7fc5a0001b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:12.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.074+0000 7fc5a6a00640 1 -- 172.21.15.151:0/1434141777 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a0039070 con 0x7fc594003a00 2024-09-16T15:44:12.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.074+0000 7fc5a6a00640 1 -- 172.21.15.151:0/1434141777 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc5a002bbf0 con 0x7fc594003a00 2024-09-16T15:44:12.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.074+0000 7fc5a6a00640 1 -- 172.21.15.151:0/1434141777 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a0006070 con 0x7fc594003a00 2024-09-16T15:44:12.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.074+0000 7fc5b5a00640 1 -- 172.21.15.151:0/1434141777 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5b015d480 con 0x7fc594003a00 2024-09-16T15:44:12.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.074+0000 7fc5b5a00640 1 -- 172.21.15.151:0/1434141777 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5b015da40 con 0x7fc594003a00 2024-09-16T15:44:12.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.076+0000 7fc5b5a00640 1 -- 172.21.15.151:0/1434141777 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5b0069360 con 0x7fc594003a00 2024-09-16T15:44:12.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.079+0000 7fc5a6a00640 1 -- 172.21.15.151:0/1434141777 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 16) ==== 101848+0+0 (secure 0 0 0) 0x7fc5a0006210 con 0x7fc594003a00 2024-09-16T15:44:12.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.081+0000 7fc5a6a00640 1 --2- 172.21.15.151:0/1434141777 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc590078eb0 0x7fc59007b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:12.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.081+0000 7fc5a6a00640 1 -- 172.21.15.151:0/1434141777 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5854+0+0 (secure 0 0 0) 0x7fc5a00bb8f0 con 0x7fc594003a00 2024-09-16T15:44:12.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.087+0000 7fc5a6a00640 1 -- 172.21.15.151:0/1434141777 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc5b0155c10 con 0x7fc594003a00 2024-09-16T15:44:12.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.107+0000 7fc5ae000640 1 --2- 172.21.15.151:0/1434141777 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc590078eb0 0x7fc59007b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:12.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.110+0000 7fc5ae000640 1 --2- 172.21.15.151:0/1434141777 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc590078eb0 0x7fc59007b370 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc59c005b10 tx=0x7fc59c005aa0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:12.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.462+0000 7fc5b5a00640 1 -- 172.21.15.151:0/1434141777 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fc5b00709f0 con 0x7fc594003a00 2024-09-16T15:44:12.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.463+0000 7fc5a6a00640 1 -- 172.21.15.151:0/1434141777 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2167 (secure 0 0 0) 0x7fc5a00834d0 con 0x7fc594003a00 2024-09-16T15:44:12.465 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:44:12.465 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-16T15:44:12.465 INFO:teuthology.orchestra.run.smithi151.stdout: "fsid": "1757be18-e41a-4446-bb4f-ee4ac81c39df", 2024-09-16T15:44:12.465 INFO:teuthology.orchestra.run.smithi151.stdout: "health": { 2024-09-16T15:44:12.465 INFO:teuthology.orchestra.run.smithi151.stdout: "status": "HEALTH_OK", 2024-09-16T15:44:12.465 INFO:teuthology.orchestra.run.smithi151.stdout: "checks": {}, 2024-09-16T15:44:12.465 INFO:teuthology.orchestra.run.smithi151.stdout: "mutes": [] 2024-09-16T15:44:12.465 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:44:12.465 INFO:teuthology.orchestra.run.smithi151.stdout: "election_epoch": 4, 2024-09-16T15:44:12.465 INFO:teuthology.orchestra.run.smithi151.stdout: "quorum": [ 2024-09-16T15:44:12.465 INFO:teuthology.orchestra.run.smithi151.stdout: 0, 2024-09-16T15:44:12.466 INFO:teuthology.orchestra.run.smithi151.stdout: 1, 2024-09-16T15:44:12.466 INFO:teuthology.orchestra.run.smithi151.stdout: 2 2024-09-16T15:44:12.466 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:44:12.466 INFO:teuthology.orchestra.run.smithi151.stdout: "quorum_names": [ 2024-09-16T15:44:12.466 INFO:teuthology.orchestra.run.smithi151.stdout: "a", 2024-09-16T15:44:12.466 INFO:teuthology.orchestra.run.smithi151.stdout: "b", 2024-09-16T15:44:12.466 INFO:teuthology.orchestra.run.smithi151.stdout: "c" 2024-09-16T15:44:12.466 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:44:12.466 INFO:teuthology.orchestra.run.smithi151.stdout: "quorum_age": 316, 2024-09-16T15:44:12.466 INFO:teuthology.orchestra.run.smithi151.stdout: "monmap": { 2024-09-16T15:44:12.466 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 1, 2024-09-16T15:44:12.466 INFO:teuthology.orchestra.run.smithi151.stdout: "min_mon_release_name": "squid", 2024-09-16T15:44:12.466 INFO:teuthology.orchestra.run.smithi151.stdout: "num_mons": 3 2024-09-16T15:44:12.466 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:44:12.466 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap": { 2024-09-16T15:44:12.467 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 50, 2024-09-16T15:44:12.467 INFO:teuthology.orchestra.run.smithi151.stdout: "num_osds": 8, 2024-09-16T15:44:12.467 INFO:teuthology.orchestra.run.smithi151.stdout: "num_up_osds": 8, 2024-09-16T15:44:12.467 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_up_since": 1726501150, 2024-09-16T15:44:12.467 INFO:teuthology.orchestra.run.smithi151.stdout: "num_in_osds": 8, 2024-09-16T15:44:12.467 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_in_since": 1726501140, 2024-09-16T15:44:12.467 INFO:teuthology.orchestra.run.smithi151.stdout: "num_remapped_pgs": 0 2024-09-16T15:44:12.467 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:44:12.467 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap": { 2024-09-16T15:44:12.467 INFO:teuthology.orchestra.run.smithi151.stdout: "pgs_by_state": [ 2024-09-16T15:44:12.467 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:44:12.467 INFO:teuthology.orchestra.run.smithi151.stdout: "state_name": "active+clean", 2024-09-16T15:44:12.467 INFO:teuthology.orchestra.run.smithi151.stdout: "count": 97 2024-09-16T15:44:12.467 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:44:12.467 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:44:12.468 INFO:teuthology.orchestra.run.smithi151.stdout: "num_pgs": 97, 2024-09-16T15:44:12.468 INFO:teuthology.orchestra.run.smithi151.stdout: "num_pools": 3, 2024-09-16T15:44:12.468 INFO:teuthology.orchestra.run.smithi151.stdout: "num_objects": 2966, 2024-09-16T15:44:12.468 INFO:teuthology.orchestra.run.smithi151.stdout: "data_bytes": 12307497161, 2024-09-16T15:44:12.468 INFO:teuthology.orchestra.run.smithi151.stdout: "bytes_used": 24630050816, 2024-09-16T15:44:12.468 INFO:teuthology.orchestra.run.smithi151.stdout: "bytes_avail": 748464062464, 2024-09-16T15:44:12.468 INFO:teuthology.orchestra.run.smithi151.stdout: "bytes_total": 773094113280, 2024-09-16T15:44:12.468 INFO:teuthology.orchestra.run.smithi151.stdout: "read_bytes_sec": 141840283, 2024-09-16T15:44:12.468 INFO:teuthology.orchestra.run.smithi151.stdout: "write_bytes_sec": 220124185, 2024-09-16T15:44:12.468 INFO:teuthology.orchestra.run.smithi151.stdout: "read_op_per_sec": 34, 2024-09-16T15:44:12.468 INFO:teuthology.orchestra.run.smithi151.stdout: "write_op_per_sec": 60 2024-09-16T15:44:12.468 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:44:12.469 INFO:teuthology.orchestra.run.smithi151.stdout: "fsmap": { 2024-09-16T15:44:12.469 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 28, 2024-09-16T15:44:12.469 INFO:teuthology.orchestra.run.smithi151.stdout: "btime": "2024-09-16T15:42:38:048451+0000", 2024-09-16T15:44:12.469 INFO:teuthology.orchestra.run.smithi151.stdout: "id": 3, 2024-09-16T15:44:12.469 INFO:teuthology.orchestra.run.smithi151.stdout: "up": 1, 2024-09-16T15:44:12.469 INFO:teuthology.orchestra.run.smithi151.stdout: "in": 1, 2024-09-16T15:44:12.469 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 1, 2024-09-16T15:44:12.469 INFO:teuthology.orchestra.run.smithi151.stdout: "by_rank": [ 2024-09-16T15:44:12.469 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:44:12.469 INFO:teuthology.orchestra.run.smithi151.stdout: "filesystem_id": 3, 2024-09-16T15:44:12.469 INFO:teuthology.orchestra.run.smithi151.stdout: "rank": 0, 2024-09-16T15:44:12.469 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "c", 2024-09-16T15:44:12.469 INFO:teuthology.orchestra.run.smithi151.stdout: "status": "up:active", 2024-09-16T15:44:12.470 INFO:teuthology.orchestra.run.smithi151.stdout: "gid": 5400 2024-09-16T15:44:12.470 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:44:12.470 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:44:12.470 INFO:teuthology.orchestra.run.smithi151.stdout: "up:standby": 3 2024-09-16T15:44:12.470 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:44:12.470 INFO:teuthology.orchestra.run.smithi151.stdout: "mgrmap": { 2024-09-16T15:44:12.470 INFO:teuthology.orchestra.run.smithi151.stdout: "available": true, 2024-09-16T15:44:12.470 INFO:teuthology.orchestra.run.smithi151.stdout: "num_standbys": 1, 2024-09-16T15:44:12.470 INFO:teuthology.orchestra.run.smithi151.stdout: "modules": [ 2024-09-16T15:44:12.470 INFO:teuthology.orchestra.run.smithi151.stdout: "iostat", 2024-09-16T15:44:12.470 INFO:teuthology.orchestra.run.smithi151.stdout: "nfs", 2024-09-16T15:44:12.470 INFO:teuthology.orchestra.run.smithi151.stdout: "restful" 2024-09-16T15:44:12.470 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:44:12.470 INFO:teuthology.orchestra.run.smithi151.stdout: "services": {} 2024-09-16T15:44:12.471 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:44:12.471 INFO:teuthology.orchestra.run.smithi151.stdout: "servicemap": { 2024-09-16T15:44:12.471 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 7, 2024-09-16T15:44:12.471 INFO:teuthology.orchestra.run.smithi151.stdout: "modified": "2024-09-16T15:43:31.641478+0000", 2024-09-16T15:44:12.471 INFO:teuthology.orchestra.run.smithi151.stdout: "services": {} 2024-09-16T15:44:12.471 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:44:12.471 INFO:teuthology.orchestra.run.smithi151.stdout: "progress_events": { 2024-09-16T15:44:12.471 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr-vol-ongoing-clones": { 2024-09-16T15:44:12.471 INFO:teuthology.orchestra.run.smithi151.stdout: "message": "1 ongoing clones - average progress is 1.328% (0s)\n [............................] ", 2024-09-16T15:44:12.471 INFO:teuthology.orchestra.run.smithi151.stdout: "progress": 0.013279999606311321, 2024-09-16T15:44:12.471 INFO:teuthology.orchestra.run.smithi151.stdout: "add_to_ceph_s": true 2024-09-16T15:44:12.471 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:44:12.471 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:44:12.471 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-16T15:44:12.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.466+0000 7fc5b5a00640 1 -- 172.21.15.151:0/1434141777 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc590078eb0 msgr2=0x7fc59007b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:12.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.466+0000 7fc5b5a00640 1 --2- 172.21.15.151:0/1434141777 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc590078eb0 0x7fc59007b370 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc59c005b10 tx=0x7fc59c005aa0 comp rx=0 tx=0).stop 2024-09-16T15:44:12.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.466+0000 7fc5b5a00640 1 -- 172.21.15.151:0/1434141777 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc594003a00 msgr2=0x7fc5b0159ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:12.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.466+0000 7fc5b5a00640 1 --2- 172.21.15.151:0/1434141777 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc594003a00 0x7fc5b0159ee0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fc5a002b490 tx=0x7fc5a0001b20 comp rx=0 tx=0).stop 2024-09-16T15:44:12.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.466+0000 7fc5b5a00640 1 -- 172.21.15.151:0/1434141777 shutdown_connections 2024-09-16T15:44:12.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.466+0000 7fc5b5a00640 1 --2- 172.21.15.151:0/1434141777 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc590078eb0 0x7fc59007b370 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:12.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.466+0000 7fc5b5a00640 1 --2- 172.21.15.151:0/1434141777 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5b009b230 0x7fc5b015a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:12.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.466+0000 7fc5b5a00640 1 --2- 172.21.15.151:0/1434141777 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5b0155480 0x7fc5b00ad510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:12.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.466+0000 7fc5b5a00640 1 --2- 172.21.15.151:0/1434141777 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc594003a00 0x7fc5b0159ee0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:12.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.466+0000 7fc5b5a00640 1 -- 172.21.15.151:0/1434141777 >> 172.21.15.151:0/1434141777 conn(0x7fc5b00b18e0 msgr2=0x7fc5b00acd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:12.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.466+0000 7fc5b5a00640 1 -- 172.21.15.151:0/1434141777 shutdown_connections 2024-09-16T15:44:12.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.466+0000 7fc5b5a00640 1 -- 172.21.15.151:0/1434141777 wait complete. 2024-09-16T15:44:12.503 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone27010 2024-09-16T15:44:12.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.752+0000 7f2c1f000640 1 -- 172.21.15.151:0/2531700927 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2bfc0078f0 msgr2=0x7f2bfc009d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:12.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.752+0000 7f2c1f000640 1 --2- 172.21.15.151:0/2531700927 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2bfc0078f0 0x7f2bfc009d60 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f2c040099a0 tx=0x7f2bfc003810 comp rx=0 tx=0).stop 2024-09-16T15:44:12.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.752+0000 7f2c1d000640 1 -- 172.21.15.151:0/2531700927 reap_dead start 2024-09-16T15:44:12.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.752+0000 7f2c1f000640 1 -- 172.21.15.151:0/2531700927 shutdown_connections 2024-09-16T15:44:12.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.752+0000 7f2c1f000640 1 -- 172.21.15.151:0/2531700927 >> 172.21.15.151:0/2531700927 conn(0x7f2c180b1840 msgr2=0x7f2c180b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:12.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.753+0000 7f2c1f000640 1 -- 172.21.15.151:0/2531700927 shutdown_connections 2024-09-16T15:44:12.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.753+0000 7f2c1f000640 1 -- 172.21.15.151:0/2531700927 wait complete. 2024-09-16T15:44:12.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.755+0000 7f2c1f000640 1 Processor -- start 2024-09-16T15:44:12.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.756+0000 7f2c1f000640 1 -- start start 2024-09-16T15:44:12.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.757+0000 7f2c1f000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2bfc0078f0 0x7f2c181e6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:12.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.757+0000 7f2c1f000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c181f5a70 0x7f2c181e3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:12.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.757+0000 7f2c1f000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c180ae5a0 0x7f2c180aea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:12.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.757+0000 7f2c1f000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2c180b4040 con 0x7f2c181f5a70 2024-09-16T15:44:12.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.757+0000 7f2c1f000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2c180b3ec0 con 0x7f2bfc0078f0 2024-09-16T15:44:12.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.757+0000 7f2c1f000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f2c180b41c0 con 0x7f2c180ae5a0 2024-09-16T15:44:12.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.757+0000 7f2c17400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c181f5a70 0x7f2c181e3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:12.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.757+0000 7f2c17400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c181f5a70 0x7f2c181e3c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:55052/0 (socket says 172.21.15.151:55052) 2024-09-16T15:44:12.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.757+0000 7f2c17400640 1 -- 172.21.15.151:0/2367057919 learned_addr learned my addr 172.21.15.151:0/2367057919 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:12.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.758+0000 7f2c1d000640 1 --2- 172.21.15.151:0/2367057919 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c180ae5a0 0x7f2c180aea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:12.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.758+0000 7f2c17400640 1 -- 172.21.15.151:0/2367057919 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c180ae5a0 msgr2=0x7f2c180aea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:12.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.758+0000 7f2c17400640 1 --2- 172.21.15.151:0/2367057919 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c180ae5a0 0x7f2c180aea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:12.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.758+0000 7f2c17400640 1 -- 172.21.15.151:0/2367057919 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2bfc0078f0 msgr2=0x7f2c181e6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:12.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.758+0000 7f2c17400640 1 --2- 172.21.15.151:0/2367057919 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2bfc0078f0 0x7f2c181e6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:12.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.758+0000 7f2c17400640 1 -- 172.21.15.151:0/2367057919 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c181f5ef0 con 0x7f2c181f5a70 2024-09-16T15:44:12.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.758+0000 7f2c1d000640 1 --2- 172.21.15.151:0/2367057919 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c180ae5a0 0x7f2c180aea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:44:12.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.758+0000 7f2c17400640 1 --2- 172.21.15.151:0/2367057919 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c181f5a70 0x7f2c181e3c00 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f2c04035040 tx=0x7f2c04032df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:12.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.759+0000 7f2c0fe00640 1 -- 172.21.15.151:0/2367057919 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c04037050 con 0x7f2c181f5a70 2024-09-16T15:44:12.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.759+0000 7f2c0fe00640 1 -- 172.21.15.151:0/2367057919 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2c04038040 con 0x7f2c181f5a70 2024-09-16T15:44:12.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.759+0000 7f2c1f000640 1 -- 172.21.15.151:0/2367057919 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c181e99e0 con 0x7f2c181f5a70 2024-09-16T15:44:12.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.759+0000 7f2c0fe00640 1 -- 172.21.15.151:0/2367057919 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c0403c870 con 0x7f2c181f5a70 2024-09-16T15:44:12.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.759+0000 7f2c1f000640 1 -- 172.21.15.151:0/2367057919 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c181e4d70 con 0x7f2c181f5a70 2024-09-16T15:44:12.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.760+0000 7f2c1f000640 1 -- 172.21.15.151:0/2367057919 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c1806e860 con 0x7f2c181f5a70 2024-09-16T15:44:12.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.764+0000 7f2c0fe00640 1 -- 172.21.15.151:0/2367057919 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 16) ==== 101848+0+0 (secure 0 0 0) 0x7f2c04045020 con 0x7f2c181f5a70 2024-09-16T15:44:12.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.765+0000 7f2c0fe00640 1 --2- 172.21.15.151:0/2367057919 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2bf8078e80 0x7f2bf807b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:12.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.766+0000 7f2c0fe00640 1 -- 172.21.15.151:0/2367057919 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5854+0+0 (secure 0 0 0) 0x7f2c040bb620 con 0x7f2c181f5a70 2024-09-16T15:44:12.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.772+0000 7f2c0fe00640 1 -- 172.21.15.151:0/2367057919 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2c0404c020 con 0x7f2c181f5a70 2024-09-16T15:44:12.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.826+0000 7f2c17e00640 1 --2- 172.21.15.151:0/2367057919 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2bf8078e80 0x7f2bf807b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:12.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:12.827+0000 7f2c17e00640 1 --2- 172.21.15.151:0/2367057919 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2bf8078e80 0x7f2bf807b340 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2c08006590 tx=0x7f2c08006ae0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:13.122 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:13.119+0000 7f2c1f000640 1 -- 172.21.15.151:0/2367057919 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone27010", "target": ["mon-mgr", ""]}) -- 0x7f2c180cd570 con 0x7f2bf8078e80 2024-09-16T15:44:13.278 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:44:13.275+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:13.278 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:44:13.275+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:13.278 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:44:13.275+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:13.278 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:44:13.275+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:13.278 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:44:13.275+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:14.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:14.506+0000 7f2c0fe00640 1 -- 172.21.15.151:0/2367057919 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mgrmap(e 17) ==== 101950+0+0 (secure 0 0 0) 0x7f2c0407edb0 con 0x7f2c181f5a70 2024-09-16T15:44:15.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:15.689+0000 7f2c0fe00640 1 -- 172.21.15.151:0/2367057919 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2c180cd570 con 0x7f2bf8078e80 2024-09-16T15:44:15.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:15.696+0000 7f2c1f000640 1 -- 172.21.15.151:0/2367057919 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2bf8078e80 msgr2=0x7f2bf807b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:15.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:15.696+0000 7f2c1f000640 1 --2- 172.21.15.151:0/2367057919 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2bf8078e80 0x7f2bf807b340 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2c08006590 tx=0x7f2c08006ae0 comp rx=0 tx=0).stop 2024-09-16T15:44:15.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:15.696+0000 7f2c1f000640 1 -- 172.21.15.151:0/2367057919 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c181f5a70 msgr2=0x7f2c181e3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:15.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:15.696+0000 7f2c1f000640 1 --2- 172.21.15.151:0/2367057919 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c181f5a70 0x7f2c181e3c00 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f2c04035040 tx=0x7f2c04032df0 comp rx=0 tx=0).stop 2024-09-16T15:44:15.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:15.696+0000 7f2c1f000640 1 -- 172.21.15.151:0/2367057919 shutdown_connections 2024-09-16T15:44:15.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:15.696+0000 7f2c1f000640 1 --2- 172.21.15.151:0/2367057919 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2bf8078e80 0x7f2bf807b340 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:15.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:15.696+0000 7f2c1f000640 1 --2- 172.21.15.151:0/2367057919 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c180ae5a0 0x7f2c180aea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:15.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:15.696+0000 7f2c1f000640 1 --2- 172.21.15.151:0/2367057919 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c181f5a70 0x7f2c181e3c00 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:15.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:15.696+0000 7f2c1f000640 1 --2- 172.21.15.151:0/2367057919 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2bfc0078f0 0x7f2c181e6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:15.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:15.696+0000 7f2c1f000640 1 -- 172.21.15.151:0/2367057919 >> 172.21.15.151:0/2367057919 conn(0x7f2c180b1840 msgr2=0x7f2c1815d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:15.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:15.696+0000 7f2c1f000640 1 -- 172.21.15.151:0/2367057919 shutdown_connections 2024-09-16T15:44:15.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:15.697+0000 7f2c1f000640 1 -- 172.21.15.151:0/2367057919 wait complete. 2024-09-16T15:44:15.739 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone48851 2024-09-16T15:44:16.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:16.023+0000 7f5664e00640 1 -- 172.21.15.151:0/2144897919 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f566015c330 msgr2=0x7f56600cfee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:16.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:16.023+0000 7f5664e00640 1 --2- 172.21.15.151:0/2144897919 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f566015c330 0x7f56600cfee0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f56600747c0 tx=0x7f564802a460 comp rx=0 tx=0).stop 2024-09-16T15:44:16.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:16.023+0000 7f5664e00640 1 -- 172.21.15.151:0/2144897919 shutdown_connections 2024-09-16T15:44:16.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:16.023+0000 7f5664e00640 1 --2- 172.21.15.151:0/2144897919 >> v2:172.21.15.167:3300/0 conn(0x7f56600cc870 0x7f56600d0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:16.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:16.023+0000 7f5664e00640 1 --2- 172.21.15.151:0/2144897919 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f566015c330 0x7f56600cfee0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:16.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:16.023+0000 7f5664e00640 1 --2- 172.21.15.151:0/2144897919 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f566015bf50 0x7f56600cf9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:16.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:16.023+0000 7f5664e00640 1 -- 172.21.15.151:0/2144897919 >> 172.21.15.151:0/2144897919 conn(0x7f56600d1dd0 msgr2=0x7f56600d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:16.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:16.024+0000 7f5664e00640 1 -- 172.21.15.151:0/2144897919 shutdown_connections 2024-09-16T15:44:16.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:16.024+0000 7f5664e00640 1 -- 172.21.15.151:0/2144897919 wait complete. 2024-09-16T15:44:16.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:16.026+0000 7f5664e00640 1 Processor -- start 2024-09-16T15:44:16.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:16.026+0000 7f5664e00640 1 -- start start 2024-09-16T15:44:16.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:16.027+0000 7f5664e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56600cc870 0x7f566015c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:16.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:16.027+0000 7f5664e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f566015bf50 0x7f5660158660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:16.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:16.027+0000 7f5664e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f566015c330 0x7f56600b74c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:16.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:16.027+0000 7f5664e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f56600cee40 con 0x7f56600cc870 2024-09-16T15:44:16.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:16.027+0000 7f5664e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f56600cecc0 con 0x7f566015c330 2024-09-16T15:44:16.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:16.027+0000 7f5664e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f56600cefc0 con 0x7f566015bf50 2024-09-16T15:44:16.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:16.027+0000 7f565de00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56600cc870 0x7f566015c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:16.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:16.027+0000 7f565de00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56600cc870 0x7f566015c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:55850/0 (socket says 172.21.15.151:55850) 2024-09-16T15:44:16.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:16.027+0000 7f565de00640 1 -- 172.21.15.151:0/1300082435 learned_addr learned my addr 172.21.15.151:0/1300082435 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:16.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:16.027+0000 7f565d400640 1 --2- 172.21.15.151:0/1300082435 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f566015bf50 0x7f5660158660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:16.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:16.028+0000 7f565de00640 1 -- 172.21.15.151:0/1300082435 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f566015bf50 msgr2=0x7f5660158660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:16.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:16.028+0000 7f565de00640 1 --2- 172.21.15.151:0/1300082435 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f566015bf50 0x7f5660158660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:16.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:16.028+0000 7f565de00640 1 -- 172.21.15.151:0/1300082435 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f566015c330 msgr2=0x7f56600b74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:16.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:16.028+0000 7f565de00640 1 --2- 172.21.15.151:0/1300082435 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f566015c330 0x7f56600b74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:16.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:16.028+0000 7f565de00640 1 -- 172.21.15.151:0/1300082435 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56601f00f0 con 0x7f56600cc870 2024-09-16T15:44:16.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:16.028+0000 7f565de00640 1 --2- 172.21.15.151:0/1300082435 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56600cc870 0x7f566015c710 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f565000e9f0 tx=0x7f565000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:16.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:16.028+0000 7f5656000640 1 -- 172.21.15.151:0/1300082435 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565000cd80 con 0x7f56600cc870 2024-09-16T15:44:16.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:16.028+0000 7f5656000640 1 -- 172.21.15.151:0/1300082435 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5650005020 con 0x7f56600cc870 2024-09-16T15:44:16.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:16.028+0000 7f5656000640 1 -- 172.21.15.151:0/1300082435 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5650005450 con 0x7f56600cc870 2024-09-16T15:44:16.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:16.028+0000 7f5664e00640 1 -- 172.21.15.151:0/1300082435 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56601f03b0 con 0x7f56600cc870 2024-09-16T15:44:16.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:16.028+0000 7f5664e00640 1 -- 172.21.15.151:0/1300082435 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56601f0970 con 0x7f56600cc870 2024-09-16T15:44:16.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:16.030+0000 7f5664e00640 1 -- 172.21.15.151:0/1300082435 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56600cf470 con 0x7f56600cc870 2024-09-16T15:44:16.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:16.032+0000 7f5656000640 1 -- 172.21.15.151:0/1300082435 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 17) ==== 101950+0+0 (secure 0 0 0) 0x7f56500047a0 con 0x7f56600cc870 2024-09-16T15:44:16.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:16.033+0000 7f5656000640 1 --2- 172.21.15.151:0/1300082435 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f563c078fa0 0x7f563c07b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:16.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:16.033+0000 7f5656000640 1 -- 172.21.15.151:0/1300082435 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5854+0+0 (secure 0 0 0) 0x7f565009c6c0 con 0x7f56600cc870 2024-09-16T15:44:16.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:16.037+0000 7f5656000640 1 -- 172.21.15.151:0/1300082435 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f5650063dd0 con 0x7f56600cc870 2024-09-16T15:44:16.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:16.084+0000 7f565d400640 1 --2- 172.21.15.151:0/1300082435 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f563c078fa0 0x7f563c07b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:16.125 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:16.123+0000 7f565d400640 1 --2- 172.21.15.151:0/1300082435 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f563c078fa0 0x7f563c07b460 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5648032000 tx=0x7f5648004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:16.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:16.428+0000 7f5664e00640 1 -- 172.21.15.151:0/1300082435 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone48851", "target": ["mon-mgr", ""]}) -- 0x7f56600d2350 con 0x7f563c078fa0 2024-09-16T15:44:17.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:17.521+0000 7f5656000640 1 -- 172.21.15.151:0/1300082435 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f56600d2350 con 0x7f563c078fa0 2024-09-16T15:44:17.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:17.525+0000 7f5664e00640 1 -- 172.21.15.151:0/1300082435 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f563c078fa0 msgr2=0x7f563c07b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:17.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:17.525+0000 7f5664e00640 1 --2- 172.21.15.151:0/1300082435 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f563c078fa0 0x7f563c07b460 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5648032000 tx=0x7f5648004730 comp rx=0 tx=0).stop 2024-09-16T15:44:17.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:17.525+0000 7f5664e00640 1 -- 172.21.15.151:0/1300082435 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56600cc870 msgr2=0x7f566015c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:17.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:17.525+0000 7f5664e00640 1 --2- 172.21.15.151:0/1300082435 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56600cc870 0x7f566015c710 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f565000e9f0 tx=0x7f565000edf0 comp rx=0 tx=0).stop 2024-09-16T15:44:17.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:17.525+0000 7f5664e00640 1 -- 172.21.15.151:0/1300082435 shutdown_connections 2024-09-16T15:44:17.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:17.525+0000 7f5664e00640 1 --2- 172.21.15.151:0/1300082435 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f563c078fa0 0x7f563c07b460 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:17.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:17.525+0000 7f5664e00640 1 --2- 172.21.15.151:0/1300082435 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f566015c330 0x7f56600b74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:17.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:17.525+0000 7f5664e00640 1 --2- 172.21.15.151:0/1300082435 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f566015bf50 0x7f5660158660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:17.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:17.525+0000 7f5664e00640 1 --2- 172.21.15.151:0/1300082435 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56600cc870 0x7f566015c710 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:17.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:17.525+0000 7f5664e00640 1 -- 172.21.15.151:0/1300082435 >> 172.21.15.151:0/1300082435 conn(0x7f56600d1dd0 msgr2=0x7f56600b3780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:17.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:17.525+0000 7f5664e00640 1 -- 172.21.15.151:0/1300082435 shutdown_connections 2024-09-16T15:44:17.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:17.526+0000 7f5664e00640 1 -- 172.21.15.151:0/1300082435 wait complete. 2024-09-16T15:44:17.559 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone92812 2024-09-16T15:44:17.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:17.789+0000 7f5a12a00640 1 -- 172.21.15.151:0/907973404 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a0c06efa0 msgr2=0x7f5a0c0cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:17.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:17.789+0000 7f5a12a00640 1 --2- 172.21.15.151:0/907973404 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a0c06efa0 0x7f5a0c0cdbb0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f5a04005fd0 tx=0x7f5a0402a970 comp rx=0 tx=0).stop 2024-09-16T15:44:17.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:17.789+0000 7f5a12a00640 1 -- 172.21.15.151:0/907973404 shutdown_connections 2024-09-16T15:44:17.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:17.789+0000 7f5a12a00640 1 --2- 172.21.15.151:0/907973404 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a0c06efa0 0x7f5a0c0cdbb0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:17.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:17.789+0000 7f5a12a00640 1 --2- 172.21.15.151:0/907973404 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a0c0cccd0 0x7f5a0c073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:17.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:17.789+0000 7f5a12a00640 1 -- 172.21.15.151:0/907973404 >> 172.21.15.151:0/907973404 conn(0x7f5a0c0b1840 msgr2=0x7f5a0c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:17.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:17.789+0000 7f5a12a00640 1 -- 172.21.15.151:0/907973404 shutdown_connections 2024-09-16T15:44:17.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:17.789+0000 7f5a12a00640 1 -- 172.21.15.151:0/907973404 wait complete. 2024-09-16T15:44:17.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:17.790+0000 7f5a12a00640 1 Processor -- start 2024-09-16T15:44:17.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:17.791+0000 7f5a12a00640 1 -- start start 2024-09-16T15:44:17.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:17.791+0000 7f5a12a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a0c06efa0 0x7f5a0c13b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:17.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:17.791+0000 7f5a12a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a0c0cccd0 0x7f5a0c0b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:17.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:17.792+0000 7f5a12a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a0c145750 0x7f5a0c0ab0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:17.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:17.792+0000 7f5a12a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5a0c0ce480 con 0x7f5a0c145750 2024-09-16T15:44:17.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:17.792+0000 7f5a12a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5a0c0ce300 con 0x7f5a0c0cccd0 2024-09-16T15:44:17.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:17.792+0000 7f5a12a00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f5a0c0ce600 con 0x7f5a0c06efa0 2024-09-16T15:44:17.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:17.792+0000 7f5a10a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a0c145750 0x7f5a0c0ab0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:17.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:17.792+0000 7f5a10a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a0c145750 0x7f5a0c0ab0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:55868/0 (socket says 172.21.15.151:55868) 2024-09-16T15:44:17.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:17.792+0000 7f5a10a00640 1 -- 172.21.15.151:0/444797781 learned_addr learned my addr 172.21.15.151:0/444797781 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:17.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:17.792+0000 7f5a03400640 1 --2- 172.21.15.151:0/444797781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a0c0cccd0 0x7f5a0c0b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:17.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:17.792+0000 7f5a0be00640 1 --2- 172.21.15.151:0/444797781 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a0c06efa0 0x7f5a0c13b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:17.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:17.792+0000 7f5a10a00640 1 -- 172.21.15.151:0/444797781 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a0c06efa0 msgr2=0x7f5a0c13b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:17.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:17.792+0000 7f5a10a00640 1 --2- 172.21.15.151:0/444797781 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a0c06efa0 0x7f5a0c13b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:17.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:17.792+0000 7f5a10a00640 1 -- 172.21.15.151:0/444797781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a0c0cccd0 msgr2=0x7f5a0c0b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:17.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:17.792+0000 7f5a10a00640 1 --2- 172.21.15.151:0/444797781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a0c0cccd0 0x7f5a0c0b3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:17.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:17.792+0000 7f5a10a00640 1 -- 172.21.15.151:0/444797781 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a0c0b2de0 con 0x7f5a0c145750 2024-09-16T15:44:17.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:17.792+0000 7f5a0be00640 1 --2- 172.21.15.151:0/444797781 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a0c06efa0 0x7f5a0c13b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:44:17.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:17.793+0000 7f5a10a00640 1 --2- 172.21.15.151:0/444797781 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a0c145750 0x7f5a0c0ab0c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f5a04006010 tx=0x7f5a04003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:17.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:17.793+0000 7f5a08c00640 1 -- 172.21.15.151:0/444797781 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a04047070 con 0x7f5a0c145750 2024-09-16T15:44:17.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:17.793+0000 7f5a08c00640 1 -- 172.21.15.151:0/444797781 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a040045c0 con 0x7f5a0c145750 2024-09-16T15:44:17.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:17.793+0000 7f5a08c00640 1 -- 172.21.15.151:0/444797781 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a04034070 con 0x7f5a0c145750 2024-09-16T15:44:17.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:17.794+0000 7f5a12a00640 1 -- 172.21.15.151:0/444797781 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a0c0b3080 con 0x7f5a0c145750 2024-09-16T15:44:17.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:17.794+0000 7f5a12a00640 1 -- 172.21.15.151:0/444797781 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a0c0b3640 con 0x7f5a0c145750 2024-09-16T15:44:17.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:17.796+0000 7f5a08c00640 1 -- 172.21.15.151:0/444797781 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 17) ==== 101950+0+0 (secure 0 0 0) 0x7f5a0404a070 con 0x7f5a0c145750 2024-09-16T15:44:17.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:17.797+0000 7f5a08c00640 1 --2- 172.21.15.151:0/444797781 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f59e8079070 0x7f59e807b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:17.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:17.797+0000 7f5a08c00640 1 -- 172.21.15.151:0/444797781 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5854+0+0 (secure 0 0 0) 0x7f5a040bcd30 con 0x7f5a0c145750 2024-09-16T15:44:17.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:17.800+0000 7f5a12a00640 1 -- 172.21.15.151:0/444797781 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a0c0cdbb0 con 0x7f5a0c145750 2024-09-16T15:44:17.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:17.806+0000 7f5a08c00640 1 -- 172.21.15.151:0/444797781 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f5a0403f070 con 0x7f5a0c145750 2024-09-16T15:44:17.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:17.813+0000 7f5a0be00640 1 --2- 172.21.15.151:0/444797781 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f59e8079070 0x7f59e807b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:17.830 DEBUG:teuthology.orchestra.run.smithi151:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:44:17.836 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:44:17.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:17.832+0000 7f5a0be00640 1 --2- 172.21.15.151:0/444797781 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f59e8079070 0x7f59e807b530 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f59fc00a810 tx=0x7f59fc008040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:18.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:18.197+0000 7f5a12a00640 1 -- 172.21.15.151:0/444797781 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone92812", "target": ["mon-mgr", ""]}) -- 0x7f5a0c0aa4c0 con 0x7f59e8079070 2024-09-16T15:44:19.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:19.169+0000 7f5a08c00640 1 -- 172.21.15.151:0/444797781 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5a0c0aa4c0 con 0x7f59e8079070 2024-09-16T15:44:19.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:19.173+0000 7f5a12a00640 1 -- 172.21.15.151:0/444797781 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f59e8079070 msgr2=0x7f59e807b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:19.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:19.173+0000 7f5a12a00640 1 --2- 172.21.15.151:0/444797781 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f59e8079070 0x7f59e807b530 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f59fc00a810 tx=0x7f59fc008040 comp rx=0 tx=0).stop 2024-09-16T15:44:19.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:19.173+0000 7f5a12a00640 1 -- 172.21.15.151:0/444797781 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a0c145750 msgr2=0x7f5a0c0ab0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:19.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:19.173+0000 7f5a12a00640 1 --2- 172.21.15.151:0/444797781 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a0c145750 0x7f5a0c0ab0c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f5a04006010 tx=0x7f5a04003f10 comp rx=0 tx=0).stop 2024-09-16T15:44:19.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:19.174+0000 7f5a12a00640 1 -- 172.21.15.151:0/444797781 shutdown_connections 2024-09-16T15:44:19.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:19.174+0000 7f5a12a00640 1 --2- 172.21.15.151:0/444797781 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f59e8079070 0x7f59e807b530 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:19.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:19.174+0000 7f5a12a00640 1 --2- 172.21.15.151:0/444797781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a0c0cccd0 0x7f5a0c0b3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:19.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:19.174+0000 7f5a12a00640 1 --2- 172.21.15.151:0/444797781 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a0c145750 0x7f5a0c0ab0c0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:19.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:19.174+0000 7f5a12a00640 1 --2- 172.21.15.151:0/444797781 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a0c06efa0 0x7f5a0c13b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:19.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:19.174+0000 7f5a12a00640 1 -- 172.21.15.151:0/444797781 >> 172.21.15.151:0/444797781 conn(0x7f5a0c0b1840 msgr2=0x7f5a0c15f230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:19.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:19.174+0000 7f5a12a00640 1 -- 172.21.15.151:0/444797781 shutdown_connections 2024-09-16T15:44:19.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:19.174+0000 7f5a12a00640 1 -- 172.21.15.151:0/444797781 wait complete. 2024-09-16T15:44:19.197 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone88453 2024-09-16T15:44:19.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:19.465+0000 7f1503400640 1 -- 172.21.15.151:0/3932731260 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14e00053c0 msgr2=0x7f14e000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:19.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:19.465+0000 7f1503400640 1 --2- 172.21.15.151:0/3932731260 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14e00053c0 0x7f14e000b9c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f14f4009f90 tx=0x7f14e0005830 comp rx=0 tx=0).stop 2024-09-16T15:44:19.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:19.466+0000 7f1503400640 1 -- 172.21.15.151:0/3932731260 shutdown_connections 2024-09-16T15:44:19.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:19.466+0000 7f1503400640 1 --2- 172.21.15.151:0/3932731260 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14e00053c0 0x7f14e000b9c0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:19.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:19.466+0000 7f1503400640 1 -- 172.21.15.151:0/3932731260 >> 172.21.15.151:0/3932731260 conn(0x7f14fc0b1840 msgr2=0x7f14fc0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:19.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:19.466+0000 7f1503400640 1 -- 172.21.15.151:0/3932731260 shutdown_connections 2024-09-16T15:44:19.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:19.466+0000 7f1503400640 1 -- 172.21.15.151:0/3932731260 wait complete. 2024-09-16T15:44:19.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:19.468+0000 7f1503400640 1 Processor -- start 2024-09-16T15:44:19.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:19.468+0000 7f1503400640 1 -- start start 2024-09-16T15:44:19.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:19.469+0000 7f1503400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14e00053c0 0x7f14fc13def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:19.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:19.470+0000 7f1503400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14fc1fc200 0x7f14fc1fc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:19.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:19.470+0000 7f1503400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14fc1fcbc0 0x7f14fc13f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:19.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:19.470+0000 7f1503400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc0737d0 con 0x7f14fc1fc200 2024-09-16T15:44:19.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:19.470+0000 7f1503400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc073650 con 0x7f14e00053c0 2024-09-16T15:44:19.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:19.470+0000 7f1503400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f14fc0cc5c0 con 0x7f14fc1fcbc0 2024-09-16T15:44:19.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:19.470+0000 7f1501400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14fc1fcbc0 0x7f14fc13f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:19.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:19.470+0000 7f1501400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14fc1fcbc0 0x7f14fc13f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.151:60254/0 (socket says 172.21.15.151:60254) 2024-09-16T15:44:19.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:19.470+0000 7f1501400640 1 -- 172.21.15.151:0/1040883521 learned_addr learned my addr 172.21.15.151:0/1040883521 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:19.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:19.470+0000 7f14f3e00640 1 --2- 172.21.15.151:0/1040883521 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14fc1fc200 0x7f14fc1fc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:19.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:19.471+0000 7f14f3e00640 1 -- 172.21.15.151:0/1040883521 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14fc1fcbc0 msgr2=0x7f14fc13f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:19.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:19.471+0000 7f14f3e00640 1 --2- 172.21.15.151:0/1040883521 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14fc1fcbc0 0x7f14fc13f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:19.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:19.471+0000 7f14f3e00640 1 -- 172.21.15.151:0/1040883521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14e00053c0 msgr2=0x7f14fc13def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:19.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:19.471+0000 7f14f3e00640 1 --2- 172.21.15.151:0/1040883521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14e00053c0 0x7f14fc13def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:19.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:19.471+0000 7f14f3e00640 1 -- 172.21.15.151:0/1040883521 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14fc1fd070 con 0x7f14fc1fc200 2024-09-16T15:44:19.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:19.471+0000 7f1501400640 1 --2- 172.21.15.151:0/1040883521 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14fc1fcbc0 0x7f14fc13f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:44:19.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:19.472+0000 7f14f3e00640 1 --2- 172.21.15.151:0/1040883521 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14fc1fc200 0x7f14fc1fc680 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f14e400c410 tx=0x7f14e400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:19.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:19.472+0000 7f14f9600640 1 -- 172.21.15.151:0/1040883521 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e4017070 con 0x7f14fc1fc200 2024-09-16T15:44:19.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:19.472+0000 7f14f9600640 1 -- 172.21.15.151:0/1040883521 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f14e40035f0 con 0x7f14fc1fc200 2024-09-16T15:44:19.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:19.472+0000 7f14f9600640 1 -- 172.21.15.151:0/1040883521 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e4003a40 con 0x7f14fc1fc200 2024-09-16T15:44:19.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:19.473+0000 7f1503400640 1 -- 172.21.15.151:0/1040883521 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14fc1eeea0 con 0x7f14fc1fc200 2024-09-16T15:44:19.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:19.473+0000 7f1503400640 1 -- 172.21.15.151:0/1040883521 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14fc13ebb0 con 0x7f14fc1fc200 2024-09-16T15:44:19.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:19.476+0000 7f1503400640 1 -- 172.21.15.151:0/1040883521 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14fc06a370 con 0x7f14fc1fc200 2024-09-16T15:44:19.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:19.480+0000 7f14f9600640 1 -- 172.21.15.151:0/1040883521 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 17) ==== 101950+0+0 (secure 0 0 0) 0x7f14e4003be0 con 0x7f14fc1fc200 2024-09-16T15:44:19.484 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:19.482+0000 7f14f9600640 1 --2- 172.21.15.151:0/1040883521 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f14d8078fd0 0x7f14d807b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:19.484 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:19.482+0000 7f14f9600640 1 -- 172.21.15.151:0/1040883521 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5854+0+0 (secure 0 0 0) 0x7f14e409b070 con 0x7f14fc1fc200 2024-09-16T15:44:19.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:19.483+0000 7f1500a00640 1 --2- 172.21.15.151:0/1040883521 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f14d8078fd0 0x7f14d807b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:19.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:19.484+0000 7f1500a00640 1 --2- 172.21.15.151:0/1040883521 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f14d8078fd0 0x7f14d807b490 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f14ec001eb0 tx=0x7f14ec001660 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:19.488 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:19.486+0000 7f14f9600640 1 -- 172.21.15.151:0/1040883521 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f14fc1fd070 con 0x7f14fc1fc200 2024-09-16T15:44:19.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:19.836+0000 7f1503400640 1 -- 172.21.15.151:0/1040883521 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone88453", "target": ["mon-mgr", ""]}) -- 0x7f14fc0aed70 con 0x7f14d8078fd0 2024-09-16T15:44:20.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.006+0000 7f14f9600640 1 -- 172.21.15.151:0/1040883521 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f14fc0aed70 con 0x7f14d8078fd0 2024-09-16T15:44:20.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.009+0000 7f1503400640 1 -- 172.21.15.151:0/1040883521 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f14d8078fd0 msgr2=0x7f14d807b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:20.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.009+0000 7f1503400640 1 --2- 172.21.15.151:0/1040883521 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f14d8078fd0 0x7f14d807b490 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f14ec001eb0 tx=0x7f14ec001660 comp rx=0 tx=0).stop 2024-09-16T15:44:20.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.009+0000 7f1503400640 1 -- 172.21.15.151:0/1040883521 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14fc1fc200 msgr2=0x7f14fc1fc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:20.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.009+0000 7f1503400640 1 --2- 172.21.15.151:0/1040883521 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14fc1fc200 0x7f14fc1fc680 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f14e400c410 tx=0x7f14e400c8d0 comp rx=0 tx=0).stop 2024-09-16T15:44:20.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.009+0000 7f1503400640 1 -- 172.21.15.151:0/1040883521 shutdown_connections 2024-09-16T15:44:20.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.009+0000 7f1503400640 1 --2- 172.21.15.151:0/1040883521 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f14d8078fd0 0x7f14d807b490 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:20.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.009+0000 7f1503400640 1 --2- 172.21.15.151:0/1040883521 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14fc1fcbc0 0x7f14fc13f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:20.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.009+0000 7f1503400640 1 --2- 172.21.15.151:0/1040883521 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14fc1fc200 0x7f14fc1fc680 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:20.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.009+0000 7f1503400640 1 --2- 172.21.15.151:0/1040883521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14e00053c0 0x7f14fc13def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:20.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.009+0000 7f1503400640 1 -- 172.21.15.151:0/1040883521 >> 172.21.15.151:0/1040883521 conn(0x7f14fc0b1840 msgr2=0x7f14fc15ed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:20.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.009+0000 7f1503400640 1 -- 172.21.15.151:0/1040883521 shutdown_connections 2024-09-16T15:44:20.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.009+0000 7f1503400640 1 -- 172.21.15.151:0/1040883521 wait complete. 2024-09-16T15:44:20.027 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-16T15:44:20.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.188+0000 7f06ac800640 1 -- 172.21.15.151:0/1497670467 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f06a40ccf00 msgr2=0x7f06a40b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:20.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.188+0000 7f06ac800640 1 --2- 172.21.15.151:0/1497670467 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f06a40ccf00 0x7f06a40b2440 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f06a406dac0 tx=0x7f069402a560 comp rx=0 tx=0).stop 2024-09-16T15:44:20.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.188+0000 7f06ac800640 1 -- 172.21.15.151:0/1497670467 shutdown_connections 2024-09-16T15:44:20.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.188+0000 7f06ac800640 1 --2- 172.21.15.151:0/1497670467 >> v2:172.21.15.167:3300/0 conn(0x7f06a4073fb0 0x7f06a4074420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:20.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.188+0000 7f06ac800640 1 --2- 172.21.15.151:0/1497670467 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f06a40ccf00 0x7f06a40b2440 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:20.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.188+0000 7f06ac800640 1 -- 172.21.15.151:0/1497670467 >> 172.21.15.151:0/1497670467 conn(0x7f06a40b1950 msgr2=0x7f06a40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:20.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.188+0000 7f06ac800640 1 -- 172.21.15.151:0/1497670467 shutdown_connections 2024-09-16T15:44:20.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.188+0000 7f06ac800640 1 -- 172.21.15.151:0/1497670467 wait complete. 2024-09-16T15:44:20.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.189+0000 7f06ac800640 1 Processor -- start 2024-09-16T15:44:20.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.190+0000 7f06ac800640 1 -- start start 2024-09-16T15:44:20.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.190+0000 7f06ac800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06a4073fb0 0x7f06a4140c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:20.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.190+0000 7f06ac800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f06a40ccf00 0x7f06a413d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:20.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.190+0000 7f06ac800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f06a415f5a0 0x7f06a40ceb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:20.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.190+0000 7f06ac800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f06a406f2e0 con 0x7f06a40ccf00 2024-09-16T15:44:20.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.190+0000 7f06ac800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f06a406f160 con 0x7f06a4073fb0 2024-09-16T15:44:20.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.190+0000 7f06ac800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f06a406f460 con 0x7f06a415f5a0 2024-09-16T15:44:20.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.191+0000 7f06a9e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06a4073fb0 0x7f06a4140c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:20.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.191+0000 7f06a9400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f06a40ccf00 0x7f06a413d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:20.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.191+0000 7f06a9e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06a4073fb0 0x7f06a4140c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.151:38632/0 (socket says 172.21.15.151:38632) 2024-09-16T15:44:20.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.191+0000 7f06a9e00640 1 -- 172.21.15.151:0/4207077201 learned_addr learned my addr 172.21.15.151:0/4207077201 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:20.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.191+0000 7f06aa800640 1 --2- 172.21.15.151:0/4207077201 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f06a415f5a0 0x7f06a40ceb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:20.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.191+0000 7f06a9e00640 1 -- 172.21.15.151:0/4207077201 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f06a415f5a0 msgr2=0x7f06a40ceb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:20.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.191+0000 7f06a9e00640 1 --2- 172.21.15.151:0/4207077201 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f06a415f5a0 0x7f06a40ceb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:20.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.191+0000 7f06a9e00640 1 -- 172.21.15.151:0/4207077201 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f06a40ccf00 msgr2=0x7f06a413d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:20.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.191+0000 7f06a9e00640 1 --2- 172.21.15.151:0/4207077201 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f06a40ccf00 0x7f06a413d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:20.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.191+0000 7f06a9e00640 1 -- 172.21.15.151:0/4207077201 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06a4150d70 con 0x7f06a4073fb0 2024-09-16T15:44:20.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.191+0000 7f06a9e00640 1 --2- 172.21.15.151:0/4207077201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06a4073fb0 0x7f06a4140c20 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f069800c3a0 tx=0x7f069800c860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:20.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.192+0000 7f069e000640 1 -- 172.21.15.151:0/4207077201 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0698017070 con 0x7f06a4073fb0 2024-09-16T15:44:20.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.192+0000 7f069e000640 1 -- 172.21.15.151:0/4207077201 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f06980053c0 con 0x7f06a4073fb0 2024-09-16T15:44:20.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.192+0000 7f069e000640 1 -- 172.21.15.151:0/4207077201 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0698003850 con 0x7f06a4073fb0 2024-09-16T15:44:20.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.192+0000 7f06ac800640 1 -- 172.21.15.151:0/4207077201 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06a4151050 con 0x7f06a4073fb0 2024-09-16T15:44:20.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.192+0000 7f06ac800640 1 -- 172.21.15.151:0/4207077201 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06a4152cf0 con 0x7f06a4073fb0 2024-09-16T15:44:20.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.193+0000 7f06ac800640 1 -- 172.21.15.151:0/4207077201 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06a406f9f0 con 0x7f06a4073fb0 2024-09-16T15:44:20.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.194+0000 7f069e000640 1 -- 172.21.15.151:0/4207077201 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 17) ==== 101950+0+0 (secure 0 0 0) 0x7f069801a070 con 0x7f06a4073fb0 2024-09-16T15:44:20.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.195+0000 7f069e000640 1 --2- 172.21.15.151:0/4207077201 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0688078fa0 0x7f068807b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:20.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.195+0000 7f069e000640 1 -- 172.21.15.151:0/4207077201 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5854+0+0 (secure 0 0 0) 0x7f069809af50 con 0x7f06a4073fb0 2024-09-16T15:44:20.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.195+0000 7f06a9400640 1 --2- 172.21.15.151:0/4207077201 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0688078fa0 0x7f068807b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:20.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.196+0000 7f06a9400640 1 --2- 172.21.15.151:0/4207077201 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0688078fa0 0x7f068807b460 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0694033000 tx=0x7f0694004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:20.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.199+0000 7f069e000640 1 -- 172.21.15.151:0/4207077201 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0698061e90 con 0x7f06a4073fb0 2024-09-16T15:44:20.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.406+0000 7f06ac800640 1 -- 172.21.15.151:0/4207077201 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f06a40b0080 con 0x7f0688078fa0 2024-09-16T15:44:20.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.409+0000 7f069e000640 1 -- 172.21.15.151:0/4207077201 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f06a40b0080 con 0x7f0688078fa0 2024-09-16T15:44:20.410 INFO:teuthology.orchestra.run.smithi151.stdout:[ 2024-09-16T15:44:20.411 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:44:20.411 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "cephfs" 2024-09-16T15:44:20.411 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:44:20.411 INFO:teuthology.orchestra.run.smithi151.stdout:] 2024-09-16T15:44:20.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.411+0000 7f06ac800640 1 -- 172.21.15.151:0/4207077201 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0688078fa0 msgr2=0x7f068807b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:20.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.411+0000 7f06ac800640 1 --2- 172.21.15.151:0/4207077201 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0688078fa0 0x7f068807b460 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0694033000 tx=0x7f0694004730 comp rx=0 tx=0).stop 2024-09-16T15:44:20.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.411+0000 7f06ac800640 1 -- 172.21.15.151:0/4207077201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06a4073fb0 msgr2=0x7f06a4140c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:20.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.411+0000 7f06ac800640 1 --2- 172.21.15.151:0/4207077201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06a4073fb0 0x7f06a4140c20 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f069800c3a0 tx=0x7f069800c860 comp rx=0 tx=0).stop 2024-09-16T15:44:20.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.411+0000 7f06ac800640 1 -- 172.21.15.151:0/4207077201 shutdown_connections 2024-09-16T15:44:20.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.411+0000 7f06ac800640 1 --2- 172.21.15.151:0/4207077201 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0688078fa0 0x7f068807b460 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:20.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.411+0000 7f06ac800640 1 --2- 172.21.15.151:0/4207077201 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f06a415f5a0 0x7f06a40ceb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:20.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.411+0000 7f06ac800640 1 --2- 172.21.15.151:0/4207077201 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f06a40ccf00 0x7f06a413d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:20.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.411+0000 7f06ac800640 1 --2- 172.21.15.151:0/4207077201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06a4073fb0 0x7f06a4140c20 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:20.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.411+0000 7f06ac800640 1 -- 172.21.15.151:0/4207077201 >> 172.21.15.151:0/4207077201 conn(0x7f06a40b1950 msgr2=0x7f06a409cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:20.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.411+0000 7f06ac800640 1 -- 172.21.15.151:0/4207077201 shutdown_connections 2024-09-16T15:44:20.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.411+0000 7f06ac800640 1 -- 172.21.15.151:0/4207077201 wait complete. 2024-09-16T15:44:20.430 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-09-16T15:44:20.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.589+0000 7fe19ec00640 1 -- 172.21.15.151:0/3765461845 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe19815aad0 msgr2=0x7fe19815efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:20.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.589+0000 7fe19ec00640 1 --2- 172.21.15.151:0/3765461845 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe19815aad0 0x7fe19815efa0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe18c009830 tx=0x7fe18c02a530 comp rx=0 tx=0).stop 2024-09-16T15:44:20.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.589+0000 7fe19ec00640 1 -- 172.21.15.151:0/3765461845 shutdown_connections 2024-09-16T15:44:20.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.589+0000 7fe19ec00640 1 --2- 172.21.15.151:0/3765461845 >> v2:172.21.15.167:3300/0 conn(0x7fe19806f6b0 0x7fe1980cefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:20.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.589+0000 7fe19ec00640 1 --2- 172.21.15.151:0/3765461845 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe19815aad0 0x7fe19815efa0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:20.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.589+0000 7fe19ec00640 1 -- 172.21.15.151:0/3765461845 >> 172.21.15.151:0/3765461845 conn(0x7fe1980b2670 msgr2=0x7fe1980b2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:20.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.589+0000 7fe19ec00640 1 -- 172.21.15.151:0/3765461845 shutdown_connections 2024-09-16T15:44:20.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.589+0000 7fe19ec00640 1 -- 172.21.15.151:0/3765461845 wait complete. 2024-09-16T15:44:20.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.591+0000 7fe19ec00640 1 Processor -- start 2024-09-16T15:44:20.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.591+0000 7fe19ec00640 1 -- start start 2024-09-16T15:44:20.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.592+0000 7fe19ec00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe19806f6b0 0x7fe198151d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:20.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.592+0000 7fe19ec00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1980cf500 0x7fe198142d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:20.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.592+0000 7fe19ec00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe19815aad0 0x7fe1980ca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:20.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.592+0000 7fe19ec00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe1980cc830 con 0x7fe19815aad0 2024-09-16T15:44:20.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.592+0000 7fe19ec00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe198073d80 con 0x7fe19806f6b0 2024-09-16T15:44:20.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.592+0000 7fe19ec00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fe198154d90 con 0x7fe1980cf500 2024-09-16T15:44:20.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.592+0000 7fe19e200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe19815aad0 0x7fe1980ca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:20.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.592+0000 7fe19e200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe19815aad0 0x7fe1980ca0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:55922/0 (socket says 172.21.15.151:55922) 2024-09-16T15:44:20.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.592+0000 7fe19e200640 1 -- 172.21.15.151:0/2727839895 learned_addr learned my addr 172.21.15.151:0/2727839895 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:20.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.592+0000 7fe19ce00640 1 --2- 172.21.15.151:0/2727839895 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1980cf500 0x7fe198142d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:20.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.592+0000 7fe19d800640 1 --2- 172.21.15.151:0/2727839895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe19806f6b0 0x7fe198151d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:20.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.592+0000 7fe19e200640 1 -- 172.21.15.151:0/2727839895 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1980cf500 msgr2=0x7fe198142d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:20.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.592+0000 7fe19e200640 1 --2- 172.21.15.151:0/2727839895 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1980cf500 0x7fe198142d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:20.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.592+0000 7fe19e200640 1 -- 172.21.15.151:0/2727839895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe19806f6b0 msgr2=0x7fe198151d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:20.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.592+0000 7fe19e200640 1 --2- 172.21.15.151:0/2727839895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe19806f6b0 0x7fe198151d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:20.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.592+0000 7fe19e200640 1 -- 172.21.15.151:0/2727839895 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1980c92f0 con 0x7fe19815aad0 2024-09-16T15:44:20.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.593+0000 7fe19e200640 1 --2- 172.21.15.151:0/2727839895 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe19815aad0 0x7fe1980ca0b0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fe19000cb10 tx=0x7fe19000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:20.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.593+0000 7fe195600640 1 -- 172.21.15.151:0/2727839895 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe190016020 con 0x7fe19815aad0 2024-09-16T15:44:20.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.593+0000 7fe195600640 1 -- 172.21.15.151:0/2727839895 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe19000ad80 con 0x7fe19815aad0 2024-09-16T15:44:20.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.593+0000 7fe195600640 1 -- 172.21.15.151:0/2727839895 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe190012680 con 0x7fe19815aad0 2024-09-16T15:44:20.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.593+0000 7fe19ec00640 1 -- 172.21.15.151:0/2727839895 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1980c95d0 con 0x7fe19815aad0 2024-09-16T15:44:20.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.593+0000 7fe19ec00640 1 -- 172.21.15.151:0/2727839895 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe198151020 con 0x7fe19815aad0 2024-09-16T15:44:20.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.594+0000 7fe19ec00640 1 -- 172.21.15.151:0/2727839895 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1980741e0 con 0x7fe19815aad0 2024-09-16T15:44:20.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.595+0000 7fe195600640 1 -- 172.21.15.151:0/2727839895 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 17) ==== 101950+0+0 (secure 0 0 0) 0x7fe190019020 con 0x7fe19815aad0 2024-09-16T15:44:20.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.596+0000 7fe195600640 1 --2- 172.21.15.151:0/2727839895 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe178078fa0 0x7fe17807b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:20.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.596+0000 7fe195600640 1 -- 172.21.15.151:0/2727839895 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5854+0+0 (secure 0 0 0) 0x7fe19009c0e0 con 0x7fe19815aad0 2024-09-16T15:44:20.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.597+0000 7fe19d800640 1 --2- 172.21.15.151:0/2727839895 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe178078fa0 0x7fe17807b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:20.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.597+0000 7fe19d800640 1 --2- 172.21.15.151:0/2727839895 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe178078fa0 0x7fe17807b460 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe18c034000 tx=0x7fe18c004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:20.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.600+0000 7fe195600640 1 -- 172.21.15.151:0/2727839895 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe190020020 con 0x7fe19815aad0 2024-09-16T15:44:20.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.812+0000 7fe19ec00640 1 -- 172.21.15.151:0/2727839895 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe1980c97a0 con 0x7fe178078fa0 2024-09-16T15:44:20.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.819+0000 7fe195600640 1 -- 172.21.15.151:0/2727839895 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+234 (secure 0 0 0) 0x7fe1980c97a0 con 0x7fe178078fa0 2024-09-16T15:44:20.821 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:44:20.821 INFO:teuthology.orchestra.run.smithi151.stdout:[ 2024-09-16T15:44:20.821 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:44:20.822 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sv1" 2024-09-16T15:44:20.822 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:44:20.822 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:44:20.822 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "subvol_clone48851" 2024-09-16T15:44:20.822 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:44:20.822 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:44:20.822 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "subvol_clone92812" 2024-09-16T15:44:20.822 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:44:20.822 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:44:20.822 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "subvol_clone88453" 2024-09-16T15:44:20.822 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:44:20.822 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:44:20.823 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "subvol_clone27010" 2024-09-16T15:44:20.823 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:44:20.823 INFO:teuthology.orchestra.run.smithi151.stdout:] 2024-09-16T15:44:20.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.821+0000 7fe19ec00640 1 -- 172.21.15.151:0/2727839895 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe178078fa0 msgr2=0x7fe17807b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:20.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.821+0000 7fe19ec00640 1 --2- 172.21.15.151:0/2727839895 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe178078fa0 0x7fe17807b460 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe18c034000 tx=0x7fe18c004730 comp rx=0 tx=0).stop 2024-09-16T15:44:20.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.821+0000 7fe19ec00640 1 -- 172.21.15.151:0/2727839895 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe19815aad0 msgr2=0x7fe1980ca0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:20.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.821+0000 7fe19ec00640 1 --2- 172.21.15.151:0/2727839895 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe19815aad0 0x7fe1980ca0b0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fe19000cb10 tx=0x7fe19000a060 comp rx=0 tx=0).stop 2024-09-16T15:44:20.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.821+0000 7fe19ec00640 1 -- 172.21.15.151:0/2727839895 shutdown_connections 2024-09-16T15:44:20.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.821+0000 7fe19ec00640 1 --2- 172.21.15.151:0/2727839895 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe178078fa0 0x7fe17807b460 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:20.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.821+0000 7fe19ec00640 1 --2- 172.21.15.151:0/2727839895 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe19815aad0 0x7fe1980ca0b0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:20.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.821+0000 7fe19ec00640 1 --2- 172.21.15.151:0/2727839895 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1980cf500 0x7fe198142d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:20.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.821+0000 7fe19ec00640 1 --2- 172.21.15.151:0/2727839895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe19806f6b0 0x7fe198151d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:20.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.822+0000 7fe19ec00640 1 -- 172.21.15.151:0/2727839895 >> 172.21.15.151:0/2727839895 conn(0x7fe1980b2670 msgr2=0x7fe1980b6180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:20.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.822+0000 7fe19ec00640 1 -- 172.21.15.151:0/2727839895 shutdown_connections 2024-09-16T15:44:20.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:20.822+0000 7fe19ec00640 1 -- 172.21.15.151:0/2727839895 wait complete. 2024-09-16T15:44:20.840 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-09-16T15:44:21.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:21.001+0000 7f0773400640 1 -- 172.21.15.151:0/2202417472 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f076c15f9a0 msgr2=0x7f076c15fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:21.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:21.001+0000 7f0773400640 1 --2- 172.21.15.151:0/2202417472 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f076c15f9a0 0x7f076c15fd80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f075c005fd0 tx=0x7f075c02a540 comp rx=0 tx=0).stop 2024-09-16T15:44:21.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:21.001+0000 7f0773400640 1 -- 172.21.15.151:0/2202417472 shutdown_connections 2024-09-16T15:44:21.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:21.001+0000 7f0773400640 1 --2- 172.21.15.151:0/2202417472 >> v2:172.21.15.167:3300/0 conn(0x7f076c06e6f0 0x7f076c06eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:21.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:21.001+0000 7f0773400640 1 --2- 172.21.15.151:0/2202417472 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f076c15f9a0 0x7f076c15fd80 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:21.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:21.001+0000 7f0773400640 1 -- 172.21.15.151:0/2202417472 >> 172.21.15.151:0/2202417472 conn(0x7f076c0b18e0 msgr2=0x7f076c0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:21.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:21.001+0000 7f0773400640 1 -- 172.21.15.151:0/2202417472 shutdown_connections 2024-09-16T15:44:21.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:21.001+0000 7f0773400640 1 -- 172.21.15.151:0/2202417472 wait complete. 2024-09-16T15:44:21.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:21.002+0000 7f0773400640 1 Processor -- start 2024-09-16T15:44:21.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:21.003+0000 7f0773400640 1 -- start start 2024-09-16T15:44:21.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:21.003+0000 7f0773400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f076c06e6f0 0x7f076c09cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:21.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:21.003+0000 7f0773400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f076c073de0 0x7f076c14fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:21.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:21.003+0000 7f0773400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f076c15f9a0 0x7f076c0ced80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:21.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:21.003+0000 7f0773400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f076c06f0b0 con 0x7f076c073de0 2024-09-16T15:44:21.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:21.003+0000 7f0773400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f076c0cd5e0 con 0x7f076c15f9a0 2024-09-16T15:44:21.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:21.003+0000 7f0773400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f076c06f230 con 0x7f076c06e6f0 2024-09-16T15:44:21.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:21.004+0000 7f0763e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f076c073de0 0x7f076c14fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:21.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:21.004+0000 7f0763e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f076c073de0 0x7f076c14fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:55930/0 (socket says 172.21.15.151:55930) 2024-09-16T15:44:21.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:21.004+0000 7f0763e00640 1 -- 172.21.15.151:0/4209188340 learned_addr learned my addr 172.21.15.151:0/4209188340 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:21.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:21.004+0000 7f0770a00640 1 --2- 172.21.15.151:0/4209188340 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f076c06e6f0 0x7f076c09cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:21.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:21.004+0000 7f0771400640 1 --2- 172.21.15.151:0/4209188340 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f076c15f9a0 0x7f076c0ced80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:21.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:21.004+0000 7f0763e00640 1 -- 172.21.15.151:0/4209188340 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f076c06e6f0 msgr2=0x7f076c09cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:21.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:21.004+0000 7f0763e00640 1 --2- 172.21.15.151:0/4209188340 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f076c06e6f0 0x7f076c09cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:21.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:21.004+0000 7f0763e00640 1 -- 172.21.15.151:0/4209188340 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f076c15f9a0 msgr2=0x7f076c0ced80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:21.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:21.004+0000 7f0763e00640 1 --2- 172.21.15.151:0/4209188340 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f076c15f9a0 0x7f076c0ced80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:21.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:21.004+0000 7f0763e00640 1 -- 172.21.15.151:0/4209188340 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f076c0cdce0 con 0x7f076c073de0 2024-09-16T15:44:21.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:21.004+0000 7f0763e00640 1 --2- 172.21.15.151:0/4209188340 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f076c073de0 0x7f076c14fb00 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f075400c500 tx=0x7f075400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:21.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:21.005+0000 7f0769600640 1 -- 172.21.15.151:0/4209188340 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07540052d0 con 0x7f076c073de0 2024-09-16T15:44:21.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:21.005+0000 7f0773400640 1 -- 172.21.15.151:0/4209188340 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f076c0cdfc0 con 0x7f076c073de0 2024-09-16T15:44:21.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:21.005+0000 7f0769600640 1 -- 172.21.15.151:0/4209188340 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0754005490 con 0x7f076c073de0 2024-09-16T15:44:21.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:21.005+0000 7f0769600640 1 -- 172.21.15.151:0/4209188340 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0754012890 con 0x7f076c073de0 2024-09-16T15:44:21.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:21.005+0000 7f0773400640 1 -- 172.21.15.151:0/4209188340 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f076c0ce580 con 0x7f076c073de0 2024-09-16T15:44:21.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:21.006+0000 7f0773400640 1 -- 172.21.15.151:0/4209188340 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f076c069360 con 0x7f076c073de0 2024-09-16T15:44:21.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:21.007+0000 7f0769600640 1 -- 172.21.15.151:0/4209188340 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 17) ==== 101950+0+0 (secure 0 0 0) 0x7f0754012a30 con 0x7f076c073de0 2024-09-16T15:44:21.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:21.008+0000 7f0769600640 1 --2- 172.21.15.151:0/4209188340 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0748078fd0 0x7f074807b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:21.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:21.008+0000 7f0769600640 1 -- 172.21.15.151:0/4209188340 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5854+0+0 (secure 0 0 0) 0x7f075409b710 con 0x7f076c073de0 2024-09-16T15:44:21.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:21.008+0000 7f0770a00640 1 --2- 172.21.15.151:0/4209188340 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0748078fd0 0x7f074807b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:21.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:21.009+0000 7f0770a00640 1 --2- 172.21.15.151:0/4209188340 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0748078fd0 0x7f074807b490 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f075c033000 tx=0x7f075c004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:21.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:21.012+0000 7f0769600640 1 -- 172.21.15.151:0/4209188340 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0754062660 con 0x7f076c073de0 2024-09-16T15:44:21.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:21.231+0000 7f0773400640 1 -- 172.21.15.151:0/4209188340 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f076c06eb70 con 0x7f0748078fd0 2024-09-16T15:44:25.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:25.677+0000 7f0769600640 1 -- 172.21.15.151:0/4209188340 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f076c06eb70 con 0x7f0748078fd0 2024-09-16T15:44:25.680 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:44:25.680 INFO:teuthology.orchestra.run.smithi151.stdout:[ 2024-09-16T15:44:25.680 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:44:25.680 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ss1" 2024-09-16T15:44:25.680 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:44:25.680 INFO:teuthology.orchestra.run.smithi151.stdout:] 2024-09-16T15:44:25.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:25.680+0000 7f0773400640 1 -- 172.21.15.151:0/4209188340 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0748078fd0 msgr2=0x7f074807b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:25.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:25.680+0000 7f0773400640 1 --2- 172.21.15.151:0/4209188340 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0748078fd0 0x7f074807b490 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f075c033000 tx=0x7f075c004730 comp rx=0 tx=0).stop 2024-09-16T15:44:25.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:25.680+0000 7f0773400640 1 -- 172.21.15.151:0/4209188340 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f076c073de0 msgr2=0x7f076c14fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:25.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:25.680+0000 7f0773400640 1 --2- 172.21.15.151:0/4209188340 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f076c073de0 0x7f076c14fb00 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f075400c500 tx=0x7f075400c9c0 comp rx=0 tx=0).stop 2024-09-16T15:44:25.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:25.680+0000 7f0773400640 1 -- 172.21.15.151:0/4209188340 shutdown_connections 2024-09-16T15:44:25.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:25.680+0000 7f0773400640 1 --2- 172.21.15.151:0/4209188340 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0748078fd0 0x7f074807b490 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:25.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:25.680+0000 7f0773400640 1 --2- 172.21.15.151:0/4209188340 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f076c15f9a0 0x7f076c0ced80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:25.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:25.680+0000 7f0773400640 1 --2- 172.21.15.151:0/4209188340 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f076c073de0 0x7f076c14fb00 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:25.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:25.680+0000 7f0773400640 1 --2- 172.21.15.151:0/4209188340 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f076c06e6f0 0x7f076c09cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:25.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:25.680+0000 7f0773400640 1 -- 172.21.15.151:0/4209188340 >> 172.21.15.151:0/4209188340 conn(0x7f076c0b18e0 msgr2=0x7f076c06f690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:25.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:25.680+0000 7f0773400640 1 -- 172.21.15.151:0/4209188340 shutdown_connections 2024-09-16T15:44:25.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:25.680+0000 7f0773400640 1 -- 172.21.15.151:0/4209188340 wait complete. 2024-09-16T15:44:25.702 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-09-16T15:44:25.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:25.866+0000 7fcfba200640 1 -- 172.21.15.151:0/1102574132 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfb40c5730 msgr2=0x7fcfb40c6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:25.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:25.866+0000 7fcfba200640 1 --2- 172.21.15.151:0/1102574132 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfb40c5730 0x7fcfb40c6430 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0009830 tx=0x7fcfa002a510 comp rx=0 tx=0).stop 2024-09-16T15:44:25.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:25.866+0000 7fcfba200640 1 -- 172.21.15.151:0/1102574132 shutdown_connections 2024-09-16T15:44:25.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:25.866+0000 7fcfba200640 1 --2- 172.21.15.151:0/1102574132 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcfb406e660 0x7fcfb406eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:25.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:25.866+0000 7fcfba200640 1 --2- 172.21.15.151:0/1102574132 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfb40c5730 0x7fcfb40c6430 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:25.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:25.866+0000 7fcfba200640 1 -- 172.21.15.151:0/1102574132 >> 172.21.15.151:0/1102574132 conn(0x7fcfb40cc490 msgr2=0x7fcfb40cc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:25.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:25.866+0000 7fcfba200640 1 -- 172.21.15.151:0/1102574132 shutdown_connections 2024-09-16T15:44:25.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:25.866+0000 7fcfba200640 1 -- 172.21.15.151:0/1102574132 wait complete. 2024-09-16T15:44:25.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:25.868+0000 7fcfba200640 1 Processor -- start 2024-09-16T15:44:25.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:25.868+0000 7fcfba200640 1 -- start start 2024-09-16T15:44:25.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:25.869+0000 7fcfba200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfb406e660 0x7fcfb41f44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:25.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:25.869+0000 7fcfba200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcfb40c5730 0x7fcfb40b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:25.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:25.869+0000 7fcfba200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcfb415f670 0x7fcfb40ce030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:25.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:25.869+0000 7fcfba200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb40ca4f0 con 0x7fcfb406e660 2024-09-16T15:44:25.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:25.869+0000 7fcfba200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb40ca370 con 0x7fcfb40c5730 2024-09-16T15:44:25.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:25.869+0000 7fcfba200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fcfb406f000 con 0x7fcfb415f670 2024-09-16T15:44:25.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:25.869+0000 7fcfb8e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfb406e660 0x7fcfb41f44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:25.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:25.869+0000 7fcfb8e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfb406e660 0x7fcfb41f44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:32772/0 (socket says 172.21.15.151:32772) 2024-09-16T15:44:25.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:25.869+0000 7fcfb8e00640 1 -- 172.21.15.151:0/1546482782 learned_addr learned my addr 172.21.15.151:0/1546482782 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:25.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:25.869+0000 7fcfb3e00640 1 --2- 172.21.15.151:0/1546482782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcfb40c5730 0x7fcfb40b0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:25.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:25.869+0000 7fcfb9800640 1 --2- 172.21.15.151:0/1546482782 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcfb415f670 0x7fcfb40ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:25.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:25.869+0000 7fcfb8e00640 1 -- 172.21.15.151:0/1546482782 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcfb415f670 msgr2=0x7fcfb40ce030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:25.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:25.870+0000 7fcfb8e00640 1 --2- 172.21.15.151:0/1546482782 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcfb415f670 0x7fcfb40ce030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:25.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:25.870+0000 7fcfb8e00640 1 -- 172.21.15.151:0/1546482782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcfb40c5730 msgr2=0x7fcfb40b0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:25.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:25.870+0000 7fcfb8e00640 1 --2- 172.21.15.151:0/1546482782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcfb40c5730 0x7fcfb40b0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:25.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:25.870+0000 7fcfb8e00640 1 -- 172.21.15.151:0/1546482782 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfb40b06c0 con 0x7fcfb406e660 2024-09-16T15:44:25.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:25.870+0000 7fcfb3e00640 1 --2- 172.21.15.151:0/1546482782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcfb40c5730 0x7fcfb40b0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:44:25.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:25.870+0000 7fcfb9800640 1 --2- 172.21.15.151:0/1546482782 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcfb415f670 0x7fcfb40ce030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:44:25.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:25.870+0000 7fcfb8e00640 1 --2- 172.21.15.151:0/1546482782 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfb406e660 0x7fcfb41f44e0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fcfa400c3e0 tx=0x7fcfa400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:25.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:25.870+0000 7fcfb0c00640 1 -- 172.21.15.151:0/1546482782 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa4011790 con 0x7fcfb406e660 2024-09-16T15:44:25.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:25.870+0000 7fcfb0c00640 1 -- 172.21.15.151:0/1546482782 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcfa4017070 con 0x7fcfb406e660 2024-09-16T15:44:25.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:25.870+0000 7fcfba200640 1 -- 172.21.15.151:0/1546482782 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfb41f4d80 con 0x7fcfb406e660 2024-09-16T15:44:25.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:25.870+0000 7fcfb0c00640 1 -- 172.21.15.151:0/1546482782 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa4011ce0 con 0x7fcfb406e660 2024-09-16T15:44:25.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:25.870+0000 7fcfba200640 1 -- 172.21.15.151:0/1546482782 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfb41f4a20 con 0x7fcfb406e660 2024-09-16T15:44:25.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:25.871+0000 7fcfba200640 1 -- 172.21.15.151:0/1546482782 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfb406eac0 con 0x7fcfb406e660 2024-09-16T15:44:25.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:25.872+0000 7fcfb0c00640 1 -- 172.21.15.151:0/1546482782 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 17) ==== 101950+0+0 (secure 0 0 0) 0x7fcfa401c460 con 0x7fcfb406e660 2024-09-16T15:44:25.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:25.873+0000 7fcfb0c00640 1 --2- 172.21.15.151:0/1546482782 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcf94078fd0 0x7fcf9407b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:25.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:25.874+0000 7fcfb0c00640 1 -- 172.21.15.151:0/1546482782 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5854+0+0 (secure 0 0 0) 0x7fcfa409b9f0 con 0x7fcfb406e660 2024-09-16T15:44:25.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:25.874+0000 7fcfb3e00640 1 --2- 172.21.15.151:0/1546482782 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcf94078fd0 0x7fcf9407b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:25.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:25.875+0000 7fcfb3e00640 1 --2- 172.21.15.151:0/1546482782 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcf94078fd0 0x7fcf9407b490 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0034000 tx=0x7fcfa0004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:25.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:25.878+0000 7fcfb0c00640 1 -- 172.21.15.151:0/1546482782 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fcfa4063040 con 0x7fcfb406e660 2024-09-16T15:44:26.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.094+0000 7fcfba200640 1 -- 172.21.15.151:0/1546482782 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcfb40aedc0 con 0x7fcf94078fd0 2024-09-16T15:44:26.125 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.123+0000 7fcfb0c00640 1 -- 172.21.15.151:0/1546482782 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcfb40aedc0 con 0x7fcf94078fd0 2024-09-16T15:44:26.126 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:44:26.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.126+0000 7fcfba200640 1 -- 172.21.15.151:0/1546482782 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcf94078fd0 msgr2=0x7fcf9407b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:26.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.126+0000 7fcfba200640 1 --2- 172.21.15.151:0/1546482782 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcf94078fd0 0x7fcf9407b490 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0034000 tx=0x7fcfa0004730 comp rx=0 tx=0).stop 2024-09-16T15:44:26.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.126+0000 7fcfba200640 1 -- 172.21.15.151:0/1546482782 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfb406e660 msgr2=0x7fcfb41f44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:26.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.126+0000 7fcfba200640 1 --2- 172.21.15.151:0/1546482782 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfb406e660 0x7fcfb41f44e0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fcfa400c3e0 tx=0x7fcfa400c8a0 comp rx=0 tx=0).stop 2024-09-16T15:44:26.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.126+0000 7fcfba200640 1 -- 172.21.15.151:0/1546482782 shutdown_connections 2024-09-16T15:44:26.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.126+0000 7fcfba200640 1 --2- 172.21.15.151:0/1546482782 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcf94078fd0 0x7fcf9407b490 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:26.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.126+0000 7fcfba200640 1 --2- 172.21.15.151:0/1546482782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcfb40c5730 0x7fcfb40b0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:26.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.126+0000 7fcfba200640 1 --2- 172.21.15.151:0/1546482782 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcfb415f670 0x7fcfb40ce030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:26.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.126+0000 7fcfba200640 1 --2- 172.21.15.151:0/1546482782 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfb406e660 0x7fcfb41f44e0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:26.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.126+0000 7fcfba200640 1 -- 172.21.15.151:0/1546482782 >> 172.21.15.151:0/1546482782 conn(0x7fcfb40cc490 msgr2=0x7fcfb40cd550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:26.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.126+0000 7fcfba200640 1 -- 172.21.15.151:0/1546482782 shutdown_connections 2024-09-16T15:44:26.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.126+0000 7fcfba200640 1 -- 172.21.15.151:0/1546482782 wait complete. 2024-09-16T15:44:26.145 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-09-16T15:44:26.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.304+0000 7f25a5e00640 1 -- 172.21.15.151:0/2403730034 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f25a006efa0 msgr2=0x7f25a00cc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:26.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.304+0000 7f25a5e00640 1 --2- 172.21.15.151:0/2403730034 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f25a006efa0 0x7f25a00cc5c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f25a00d0cd0 tx=0x7f258c02a4d0 comp rx=0 tx=0).stop 2024-09-16T15:44:26.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.304+0000 7f25a5e00640 1 -- 172.21.15.151:0/2403730034 shutdown_connections 2024-09-16T15:44:26.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.304+0000 7f25a5e00640 1 --2- 172.21.15.151:0/2403730034 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f25a006efa0 0x7f25a00cc5c0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:26.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.304+0000 7f25a5e00640 1 --2- 172.21.15.151:0/2403730034 >> v2:172.21.15.167:3300/0 conn(0x7f25a0145750 0x7f25a0073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:26.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.304+0000 7f25a5e00640 1 -- 172.21.15.151:0/2403730034 >> 172.21.15.151:0/2403730034 conn(0x7f25a00b1840 msgr2=0x7f25a00b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:26.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.304+0000 7f25a5e00640 1 -- 172.21.15.151:0/2403730034 shutdown_connections 2024-09-16T15:44:26.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.304+0000 7f25a5e00640 1 -- 172.21.15.151:0/2403730034 wait complete. 2024-09-16T15:44:26.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.306+0000 7f25a5e00640 1 Processor -- start 2024-09-16T15:44:26.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.306+0000 7f25a5e00640 1 -- start start 2024-09-16T15:44:26.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.307+0000 7f25a5e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f25a006efa0 0x7f25a01418c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:26.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.307+0000 7f25a5e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25a00ccb00 0x7f25a006f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:26.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.307+0000 7f25a5e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25a0145750 0x7f25a006f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:26.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.307+0000 7f25a5e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f25a00ced40 con 0x7f25a006efa0 2024-09-16T15:44:26.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.307+0000 7f25a5e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f25a00cebc0 con 0x7f25a00ccb00 2024-09-16T15:44:26.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.307+0000 7f25a5e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f25a00ceec0 con 0x7f25a0145750 2024-09-16T15:44:26.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.307+0000 7f259ea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f25a006efa0 0x7f25a01418c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:26.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.307+0000 7f259ea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f25a006efa0 0x7f25a01418c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:32788/0 (socket says 172.21.15.151:32788) 2024-09-16T15:44:26.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.307+0000 7f259ea00640 1 -- 172.21.15.151:0/1071142586 learned_addr learned my addr 172.21.15.151:0/1071142586 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:26.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.307+0000 7f259e000640 1 --2- 172.21.15.151:0/1071142586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25a00ccb00 0x7f25a006f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:26.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.307+0000 7f259f400640 1 --2- 172.21.15.151:0/1071142586 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25a0145750 0x7f25a006f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:26.311 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.308+0000 7f259ea00640 1 -- 172.21.15.151:0/1071142586 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25a0145750 msgr2=0x7f25a006f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:26.311 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.308+0000 7f259ea00640 1 --2- 172.21.15.151:0/1071142586 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25a0145750 0x7f25a006f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:26.311 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.308+0000 7f259ea00640 1 -- 172.21.15.151:0/1071142586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25a00ccb00 msgr2=0x7f25a006f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:26.311 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.308+0000 7f259ea00640 1 --2- 172.21.15.151:0/1071142586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25a00ccb00 0x7f25a006f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:26.311 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.308+0000 7f259ea00640 1 -- 172.21.15.151:0/1071142586 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25a01e6350 con 0x7f25a006efa0 2024-09-16T15:44:26.311 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.308+0000 7f259ea00640 1 --2- 172.21.15.151:0/1071142586 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f25a006efa0 0x7f25a01418c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f259000c970 tx=0x7f259000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:26.312 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.308+0000 7f2597400640 1 -- 172.21.15.151:0/1071142586 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2590007bb0 con 0x7f25a006efa0 2024-09-16T15:44:26.312 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.308+0000 7f2597400640 1 -- 172.21.15.151:0/1071142586 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2590007e40 con 0x7f25a006efa0 2024-09-16T15:44:26.312 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.308+0000 7f25a5e00640 1 -- 172.21.15.151:0/1071142586 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25a01e6630 con 0x7f25a006efa0 2024-09-16T15:44:26.312 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.308+0000 7f2597400640 1 -- 172.21.15.151:0/1071142586 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f259000f690 con 0x7f25a006efa0 2024-09-16T15:44:26.312 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.308+0000 7f25a5e00640 1 -- 172.21.15.151:0/1071142586 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25a01e6bf0 con 0x7f25a006efa0 2024-09-16T15:44:26.312 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.309+0000 7f25a5e00640 1 -- 172.21.15.151:0/1071142586 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25a00cc5c0 con 0x7f25a006efa0 2024-09-16T15:44:26.312 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.310+0000 7f2597400640 1 -- 172.21.15.151:0/1071142586 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 17) ==== 101950+0+0 (secure 0 0 0) 0x7f259000f830 con 0x7f25a006efa0 2024-09-16T15:44:26.313 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.311+0000 7f2597400640 1 --2- 172.21.15.151:0/1071142586 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2580078fa0 0x7f258007b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:26.313 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.312+0000 7f2597400640 1 -- 172.21.15.151:0/1071142586 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5854+0+0 (secure 0 0 0) 0x7f259009c2f0 con 0x7f25a006efa0 2024-09-16T15:44:26.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.312+0000 7f259e000640 1 --2- 172.21.15.151:0/1071142586 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2580078fa0 0x7f258007b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:26.315 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.313+0000 7f259e000640 1 --2- 172.21.15.151:0/1071142586 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2580078fa0 0x7f258007b460 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f258c033000 tx=0x7f258c004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:26.318 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.316+0000 7f2597400640 1 -- 172.21.15.151:0/1071142586 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2590014030 con 0x7f25a006efa0 2024-09-16T15:44:26.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.527+0000 7f25a5e00640 1 -- 172.21.15.151:0/1071142586 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f25a00cba00 con 0x7f2580078fa0 2024-09-16T15:44:26.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.549+0000 7f2597400640 1 -- 172.21.15.151:0/1071142586 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f25a00cba00 con 0x7f2580078fa0 2024-09-16T15:44:26.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.551+0000 7f25a5e00640 1 -- 172.21.15.151:0/1071142586 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2580078fa0 msgr2=0x7f258007b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:26.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.551+0000 7f25a5e00640 1 --2- 172.21.15.151:0/1071142586 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2580078fa0 0x7f258007b460 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f258c033000 tx=0x7f258c004730 comp rx=0 tx=0).stop 2024-09-16T15:44:26.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.551+0000 7f25a5e00640 1 -- 172.21.15.151:0/1071142586 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f25a006efa0 msgr2=0x7f25a01418c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:26.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.551+0000 7f25a5e00640 1 --2- 172.21.15.151:0/1071142586 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f25a006efa0 0x7f25a01418c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f259000c970 tx=0x7f259000ce30 comp rx=0 tx=0).stop 2024-09-16T15:44:26.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.551+0000 7f25a5e00640 1 -- 172.21.15.151:0/1071142586 shutdown_connections 2024-09-16T15:44:26.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.551+0000 7f25a5e00640 1 --2- 172.21.15.151:0/1071142586 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2580078fa0 0x7f258007b460 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:26.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.551+0000 7f25a5e00640 1 --2- 172.21.15.151:0/1071142586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25a00ccb00 0x7f25a006f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:26.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.551+0000 7f25a5e00640 1 --2- 172.21.15.151:0/1071142586 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25a0145750 0x7f25a006f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:26.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.551+0000 7f25a5e00640 1 --2- 172.21.15.151:0/1071142586 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f25a006efa0 0x7f25a01418c0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:26.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.551+0000 7f25a5e00640 1 -- 172.21.15.151:0/1071142586 >> 172.21.15.151:0/1071142586 conn(0x7f25a00b1840 msgr2=0x7f25a00af750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:26.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.551+0000 7f25a5e00640 1 -- 172.21.15.151:0/1071142586 shutdown_connections 2024-09-16T15:44:26.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.551+0000 7f25a5e00640 1 -- 172.21.15.151:0/1071142586 wait complete. 2024-09-16T15:44:26.556 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:44:26.555+0000 7f48f1c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:26.556 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:44:26.555+0000 7f48f1c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:26.556 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:44:26.555+0000 7f48f1c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:26.556 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:44:26.555+0000 7f48f1c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:26.556 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:44:26.555+0000 7f48f1c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:26.572 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone48851 2024-09-16T15:44:26.576 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:44:26.575+0000 7f48f2600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:26.576 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:44:26.575+0000 7f48f2600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:26.577 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:44:26.575+0000 7f48f2600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:26.577 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:44:26.575+0000 7f48f2600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:26.577 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:44:26.575+0000 7f48f2600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:26.740 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.738+0000 7f02b7200640 1 -- 172.21.15.151:0/3097116498 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f029c003fd0 msgr2=0x7f029c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:26.740 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.738+0000 7f02b7200640 1 --2- 172.21.15.151:0/3097116498 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f029c003fd0 0x7f029c004800 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f02a000c2c0 tx=0x7f029c005830 comp rx=0 tx=0).stop 2024-09-16T15:44:26.740 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.738+0000 7f02b6800640 1 -- 172.21.15.151:0/3097116498 reap_dead start 2024-09-16T15:44:26.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.738+0000 7f02b7200640 1 -- 172.21.15.151:0/3097116498 shutdown_connections 2024-09-16T15:44:26.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.738+0000 7f02b7200640 1 -- 172.21.15.151:0/3097116498 >> 172.21.15.151:0/3097116498 conn(0x7f02b80d3f00 msgr2=0x7f02b80d4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:26.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.738+0000 7f02b7200640 1 -- 172.21.15.151:0/3097116498 shutdown_connections 2024-09-16T15:44:26.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.738+0000 7f02b7200640 1 -- 172.21.15.151:0/3097116498 wait complete. 2024-09-16T15:44:26.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.740+0000 7f02b7200640 1 Processor -- start 2024-09-16T15:44:26.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.740+0000 7f02b7200640 1 -- start start 2024-09-16T15:44:26.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.742+0000 7f02b7200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f029c003fd0 0x7f02b80b6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:26.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.742+0000 7f02b7200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f02b80b3170 0x7f02b80cd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:26.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.742+0000 7f02b7200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f02b80cc020 0x7f02b80cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:26.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.742+0000 7f02b7200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f02b815f310 con 0x7f029c003fd0 2024-09-16T15:44:26.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.742+0000 7f02b7200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f02b815f190 con 0x7f02b80cc020 2024-09-16T15:44:26.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.742+0000 7f02b7200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f02b80cfe90 con 0x7f02b80b3170 2024-09-16T15:44:26.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.742+0000 7f02b5e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f029c003fd0 0x7f02b80b6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:26.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.742+0000 7f02b5e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f029c003fd0 0x7f02b80b6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:32806/0 (socket says 172.21.15.151:32806) 2024-09-16T15:44:26.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.742+0000 7f02b5e00640 1 -- 172.21.15.151:0/2223041657 learned_addr learned my addr 172.21.15.151:0/2223041657 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:26.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.742+0000 7f02afe00640 1 --2- 172.21.15.151:0/2223041657 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f02b80b3170 0x7f02b80cd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:26.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.742+0000 7f02b6800640 1 --2- 172.21.15.151:0/2223041657 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f02b80cc020 0x7f02b80cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:26.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.742+0000 7f02b5e00640 1 -- 172.21.15.151:0/2223041657 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f02b80b3170 msgr2=0x7f02b80cd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:26.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.743+0000 7f02b5e00640 1 --2- 172.21.15.151:0/2223041657 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f02b80b3170 0x7f02b80cd3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:26.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.743+0000 7f02b5e00640 1 -- 172.21.15.151:0/2223041657 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f02b80cc020 msgr2=0x7f02b80cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:26.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.743+0000 7f02b5e00640 1 --2- 172.21.15.151:0/2223041657 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f02b80cc020 0x7f02b80cc4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:26.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.743+0000 7f02b5e00640 1 -- 172.21.15.151:0/2223041657 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02b80b35f0 con 0x7f029c003fd0 2024-09-16T15:44:26.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.743+0000 7f02b6800640 1 --2- 172.21.15.151:0/2223041657 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f02b80cc020 0x7f02b80cc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:44:26.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.743+0000 7f02b5e00640 1 --2- 172.21.15.151:0/2223041657 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f029c003fd0 0x7f02b80b6050 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f02a8011ae0 tx=0x7f02a8009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:26.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.743+0000 7f02ae000640 1 -- 172.21.15.151:0/2223041657 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a800fcb0 con 0x7f029c003fd0 2024-09-16T15:44:26.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.743+0000 7f02ae000640 1 -- 172.21.15.151:0/2223041657 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f02a800fe50 con 0x7f029c003fd0 2024-09-16T15:44:26.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.743+0000 7f02ae000640 1 -- 172.21.15.151:0/2223041657 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a8004280 con 0x7f029c003fd0 2024-09-16T15:44:26.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.743+0000 7f02b7200640 1 -- 172.21.15.151:0/2223041657 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02b80b9800 con 0x7f029c003fd0 2024-09-16T15:44:26.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.743+0000 7f02b7200640 1 -- 172.21.15.151:0/2223041657 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02b80b5850 con 0x7f029c003fd0 2024-09-16T15:44:26.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.744+0000 7f02b7200640 1 -- 172.21.15.151:0/2223041657 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02b8069a50 con 0x7f029c003fd0 2024-09-16T15:44:26.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.746+0000 7f02ae000640 1 -- 172.21.15.151:0/2223041657 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 17) ==== 101950+0+0 (secure 0 0 0) 0x7f02a80035f0 con 0x7f029c003fd0 2024-09-16T15:44:26.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.747+0000 7f02ae000640 1 --2- 172.21.15.151:0/2223041657 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0294078fa0 0x7f029407b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:26.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.747+0000 7f02ae000640 1 -- 172.21.15.151:0/2223041657 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5854+0+0 (secure 0 0 0) 0x7f02a809c700 con 0x7f029c003fd0 2024-09-16T15:44:26.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.747+0000 7f02afe00640 1 --2- 172.21.15.151:0/2223041657 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0294078fa0 0x7f029407b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:26.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.748+0000 7f02afe00640 1 --2- 172.21.15.151:0/2223041657 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0294078fa0 0x7f029407b460 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f02a0033000 tx=0x7f02a002d000 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:26.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.751+0000 7f02ae000640 1 -- 172.21.15.151:0/2223041657 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f02b8069a50 con 0x7f029c003fd0 2024-09-16T15:44:26.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.969+0000 7f02b7200640 1 -- 172.21.15.151:0/2223041657 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone48851", "target": ["mon-mgr", ""]}) -- 0x7f02b80d4a20 con 0x7f0294078fa0 2024-09-16T15:44:26.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.989+0000 7f02ae000640 1 -- 172.21.15.151:0/2223041657 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f02b80d4a20 con 0x7f0294078fa0 2024-09-16T15:44:26.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.992+0000 7f02b7200640 1 -- 172.21.15.151:0/2223041657 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0294078fa0 msgr2=0x7f029407b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:26.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.992+0000 7f02b7200640 1 --2- 172.21.15.151:0/2223041657 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0294078fa0 0x7f029407b460 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f02a0033000 tx=0x7f02a002d000 comp rx=0 tx=0).stop 2024-09-16T15:44:26.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.992+0000 7f02b7200640 1 -- 172.21.15.151:0/2223041657 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f029c003fd0 msgr2=0x7f02b80b6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:26.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.992+0000 7f02b7200640 1 --2- 172.21.15.151:0/2223041657 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f029c003fd0 0x7f02b80b6050 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f02a8011ae0 tx=0x7f02a8009f10 comp rx=0 tx=0).stop 2024-09-16T15:44:26.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.992+0000 7f02b7200640 1 -- 172.21.15.151:0/2223041657 shutdown_connections 2024-09-16T15:44:26.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.992+0000 7f02b7200640 1 --2- 172.21.15.151:0/2223041657 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0294078fa0 0x7f029407b460 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:26.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.992+0000 7f02b7200640 1 --2- 172.21.15.151:0/2223041657 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f02b80cc020 0x7f02b80cc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:26.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.992+0000 7f02b7200640 1 --2- 172.21.15.151:0/2223041657 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f02b80b3170 0x7f02b80cd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:26.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.992+0000 7f02b7200640 1 --2- 172.21.15.151:0/2223041657 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f029c003fd0 0x7f02b80b6050 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:26.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.992+0000 7f02b7200640 1 -- 172.21.15.151:0/2223041657 >> 172.21.15.151:0/2223041657 conn(0x7f02b80d3f00 msgr2=0x7f02b815a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:26.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.992+0000 7f02b7200640 1 -- 172.21.15.151:0/2223041657 shutdown_connections 2024-09-16T15:44:26.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:26.992+0000 7f02b7200640 1 -- 172.21.15.151:0/2223041657 wait complete. 2024-09-16T15:44:27.006 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:44:27.004+0000 7f48f3a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:27.006 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:44:27.004+0000 7f48f3a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:27.006 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:44:27.004+0000 7f48f3a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:27.006 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:44:27.005+0000 7f48f3a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:27.006 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:44:27.005+0000 7f48f3a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:44:27.012 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone92812 2024-09-16T15:44:27.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.177+0000 7f63ff200640 1 -- 172.21.15.151:0/2422715106 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f640006cc00 msgr2=0x7f6400073710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:27.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.177+0000 7f63ff200640 1 --2- 172.21.15.151:0/2422715106 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f640006cc00 0x7f6400073710 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f63e8009830 tx=0x7f63e802a510 comp rx=0 tx=0).stop 2024-09-16T15:44:27.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.178+0000 7f63ff200640 1 -- 172.21.15.151:0/2422715106 shutdown_connections 2024-09-16T15:44:27.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.178+0000 7f63ff200640 1 --2- 172.21.15.151:0/2422715106 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f64000cc510 0x7f64000cc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:27.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.178+0000 7f63ff200640 1 --2- 172.21.15.151:0/2422715106 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f640006cc00 0x7f6400073710 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:27.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.178+0000 7f63ff200640 1 -- 172.21.15.151:0/2422715106 >> 172.21.15.151:0/2422715106 conn(0x7f64000d0b80 msgr2=0x7f64000d0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:27.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.178+0000 7f63ff200640 1 -- 172.21.15.151:0/2422715106 shutdown_connections 2024-09-16T15:44:27.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.178+0000 7f63ff200640 1 -- 172.21.15.151:0/2422715106 wait complete. 2024-09-16T15:44:27.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.179+0000 7f63ff200640 1 Processor -- start 2024-09-16T15:44:27.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.179+0000 7f63ff200640 1 -- start start 2024-09-16T15:44:27.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.180+0000 7f63ff200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f640006cc00 0x7f64001f1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:27.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.180+0000 7f63ff200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6400071cf0 0x7f64001f2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:27.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.180+0000 7f63ff200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f64000cc510 0x7f64001f5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:27.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.180+0000 7f63ff200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6400070af0 con 0x7f64000cc510 2024-09-16T15:44:27.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.180+0000 7f63ff200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6400070970 con 0x7f640006cc00 2024-09-16T15:44:27.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.180+0000 7f63ff200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f6400070c70 con 0x7f6400071cf0 2024-09-16T15:44:27.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.180+0000 7f63fe800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f64000cc510 0x7f64001f5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:27.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.181+0000 7f63fe800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f64000cc510 0x7f64001f5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:32820/0 (socket says 172.21.15.151:32820) 2024-09-16T15:44:27.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.181+0000 7f63fe800640 1 -- 172.21.15.151:0/429305847 learned_addr learned my addr 172.21.15.151:0/429305847 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:27.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.181+0000 7f63f7e00640 1 --2- 172.21.15.151:0/429305847 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6400071cf0 0x7f64001f2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:27.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.181+0000 7f63fde00640 1 --2- 172.21.15.151:0/429305847 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f640006cc00 0x7f64001f1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:27.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.181+0000 7f63fe800640 1 -- 172.21.15.151:0/429305847 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6400071cf0 msgr2=0x7f64001f2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:27.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.181+0000 7f63fe800640 1 --2- 172.21.15.151:0/429305847 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6400071cf0 0x7f64001f2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:27.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.181+0000 7f63fe800640 1 -- 172.21.15.151:0/429305847 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f640006cc00 msgr2=0x7f64001f1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:27.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.181+0000 7f63fe800640 1 --2- 172.21.15.151:0/429305847 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f640006cc00 0x7f64001f1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:27.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.181+0000 7f63fe800640 1 -- 172.21.15.151:0/429305847 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64001f62f0 con 0x7f64000cc510 2024-09-16T15:44:27.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.181+0000 7f63f7e00640 1 --2- 172.21.15.151:0/429305847 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6400071cf0 0x7f64001f2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:44:27.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.181+0000 7f63fe800640 1 --2- 172.21.15.151:0/429305847 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f64000cc510 0x7f64001f5980 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f63f800ea80 tx=0x7f63f800ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:27.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.182+0000 7f63f6000640 1 -- 172.21.15.151:0/429305847 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f800ce40 con 0x7f64000cc510 2024-09-16T15:44:27.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.182+0000 7f63f6000640 1 -- 172.21.15.151:0/429305847 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f63f8004b20 con 0x7f64000cc510 2024-09-16T15:44:27.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.182+0000 7f63f6000640 1 -- 172.21.15.151:0/429305847 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f8005020 con 0x7f64000cc510 2024-09-16T15:44:27.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.182+0000 7f63ff200640 1 -- 172.21.15.151:0/429305847 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64001f65d0 con 0x7f64000cc510 2024-09-16T15:44:27.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.182+0000 7f63ff200640 1 -- 172.21.15.151:0/429305847 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64001f3270 con 0x7f64000cc510 2024-09-16T15:44:27.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.183+0000 7f63ff200640 1 -- 172.21.15.151:0/429305847 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6400071200 con 0x7f64000cc510 2024-09-16T15:44:27.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.184+0000 7f63f6000640 1 -- 172.21.15.151:0/429305847 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 17) ==== 101950+0+0 (secure 0 0 0) 0x7f63f80051c0 con 0x7f64000cc510 2024-09-16T15:44:27.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.185+0000 7f63f6000640 1 --2- 172.21.15.151:0/429305847 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f63dc078fa0 0x7f63dc07b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:27.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.185+0000 7f63f6000640 1 -- 172.21.15.151:0/429305847 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5854+0+0 (secure 0 0 0) 0x7f63f809c8c0 con 0x7f64000cc510 2024-09-16T15:44:27.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.185+0000 7f63fde00640 1 --2- 172.21.15.151:0/429305847 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f63dc078fa0 0x7f63dc07b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:27.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.187+0000 7f63fde00640 1 --2- 172.21.15.151:0/429305847 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f63dc078fa0 0x7f63dc07b460 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f63f00016b0 tx=0x7f63f0009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:27.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.189+0000 7f63f6000640 1 -- 172.21.15.151:0/429305847 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f63f8063ee0 con 0x7f64000cc510 2024-09-16T15:44:27.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.407+0000 7f63ff200640 1 -- 172.21.15.151:0/429305847 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone92812", "target": ["mon-mgr", ""]}) -- 0x7f64000b5e10 con 0x7f63dc078fa0 2024-09-16T15:44:27.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.429+0000 7f63f6000640 1 -- 172.21.15.151:0/429305847 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f64000b5e10 con 0x7f63dc078fa0 2024-09-16T15:44:27.434 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.432+0000 7f63ff200640 1 -- 172.21.15.151:0/429305847 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f63dc078fa0 msgr2=0x7f63dc07b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:27.434 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.432+0000 7f63ff200640 1 --2- 172.21.15.151:0/429305847 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f63dc078fa0 0x7f63dc07b460 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f63f00016b0 tx=0x7f63f0009040 comp rx=0 tx=0).stop 2024-09-16T15:44:27.434 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.432+0000 7f63ff200640 1 -- 172.21.15.151:0/429305847 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f64000cc510 msgr2=0x7f64001f5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:27.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.432+0000 7f63ff200640 1 --2- 172.21.15.151:0/429305847 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f64000cc510 0x7f64001f5980 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f63f800ea80 tx=0x7f63f800ef40 comp rx=0 tx=0).stop 2024-09-16T15:44:27.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.432+0000 7f63ff200640 1 -- 172.21.15.151:0/429305847 shutdown_connections 2024-09-16T15:44:27.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.432+0000 7f63ff200640 1 --2- 172.21.15.151:0/429305847 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f63dc078fa0 0x7f63dc07b460 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:27.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.432+0000 7f63ff200640 1 --2- 172.21.15.151:0/429305847 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f64000cc510 0x7f64001f5980 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:27.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.432+0000 7f63ff200640 1 --2- 172.21.15.151:0/429305847 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6400071cf0 0x7f64001f2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:27.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.432+0000 7f63ff200640 1 --2- 172.21.15.151:0/429305847 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f640006cc00 0x7f64001f1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:27.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.432+0000 7f63ff200640 1 -- 172.21.15.151:0/429305847 >> 172.21.15.151:0/429305847 conn(0x7f64000d0b80 msgr2=0x7f64000b4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:27.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.432+0000 7f63ff200640 1 -- 172.21.15.151:0/429305847 shutdown_connections 2024-09-16T15:44:27.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.432+0000 7f63ff200640 1 -- 172.21.15.151:0/429305847 wait complete. 2024-09-16T15:44:27.453 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone88453 2024-09-16T15:44:27.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.618+0000 7f824f200640 1 -- 172.21.15.151:0/1911627145 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f825006cc00 msgr2=0x7f8250073710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:27.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.618+0000 7f824f200640 1 --2- 172.21.15.151:0/1911627145 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f825006cc00 0x7f8250073710 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f823c005fd0 tx=0x7f823c02a5d0 comp rx=0 tx=0).stop 2024-09-16T15:44:27.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.618+0000 7f824f200640 1 -- 172.21.15.151:0/1911627145 shutdown_connections 2024-09-16T15:44:27.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.618+0000 7f824f200640 1 --2- 172.21.15.151:0/1911627145 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f825006cc00 0x7f8250073710 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:27.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.618+0000 7f824f200640 1 --2- 172.21.15.151:0/1911627145 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8250071cf0 0x7f8250160210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:27.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.618+0000 7f824f200640 1 -- 172.21.15.151:0/1911627145 >> 172.21.15.151:0/1911627145 conn(0x7f82500d0b80 msgr2=0x7f82500d0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:27.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.618+0000 7f824f200640 1 -- 172.21.15.151:0/1911627145 shutdown_connections 2024-09-16T15:44:27.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.618+0000 7f824f200640 1 -- 172.21.15.151:0/1911627145 wait complete. 2024-09-16T15:44:27.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.619+0000 7f824f200640 1 Processor -- start 2024-09-16T15:44:27.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.620+0000 7f824f200640 1 -- start start 2024-09-16T15:44:27.624 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.620+0000 7f824f200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f825006cc00 0x7f82500ca2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:27.624 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.620+0000 7f824f200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8250071cf0 0x7f82501f8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:27.624 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.620+0000 7f824f200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f82500cc510 0x7f82501f7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:27.624 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.621+0000 7f824f200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8250071200 con 0x7f825006cc00 2024-09-16T15:44:27.624 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.621+0000 7f824f200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8250071080 con 0x7f8250071cf0 2024-09-16T15:44:27.624 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.621+0000 7f824f200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f8250071380 con 0x7f82500cc510 2024-09-16T15:44:27.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.621+0000 7f824de00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f825006cc00 0x7f82500ca2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:27.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.621+0000 7f824de00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f825006cc00 0x7f82500ca2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:32834/0 (socket says 172.21.15.151:32834) 2024-09-16T15:44:27.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.621+0000 7f824de00640 1 -- 172.21.15.151:0/651301425 learned_addr learned my addr 172.21.15.151:0/651301425 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:27.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.621+0000 7f824d400640 1 --2- 172.21.15.151:0/651301425 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8250071cf0 0x7f82501f8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:27.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.621+0000 7f824e800640 1 --2- 172.21.15.151:0/651301425 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f82500cc510 0x7f82501f7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:27.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.621+0000 7f824de00640 1 -- 172.21.15.151:0/651301425 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f82500cc510 msgr2=0x7f82501f7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:27.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.621+0000 7f824de00640 1 --2- 172.21.15.151:0/651301425 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f82500cc510 0x7f82501f7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:27.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.621+0000 7f824de00640 1 -- 172.21.15.151:0/651301425 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8250071cf0 msgr2=0x7f82501f8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:27.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.621+0000 7f824de00640 1 --2- 172.21.15.151:0/651301425 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8250071cf0 0x7f82501f8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:27.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.621+0000 7f824de00640 1 -- 172.21.15.151:0/651301425 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82500c9210 con 0x7f825006cc00 2024-09-16T15:44:27.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.621+0000 7f824d400640 1 --2- 172.21.15.151:0/651301425 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8250071cf0 0x7f82501f8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:44:27.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.621+0000 7f824e800640 1 --2- 172.21.15.151:0/651301425 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f82500cc510 0x7f82501f7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:44:27.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.621+0000 7f824de00640 1 --2- 172.21.15.151:0/651301425 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f825006cc00 0x7f82500ca2b0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f824000d980 tx=0x7f824000de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:27.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.623+0000 7f8246000640 1 -- 172.21.15.151:0/651301425 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8240017070 con 0x7f825006cc00 2024-09-16T15:44:27.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.623+0000 7f8246000640 1 -- 172.21.15.151:0/651301425 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8240009d20 con 0x7f825006cc00 2024-09-16T15:44:27.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.623+0000 7f824f200640 1 -- 172.21.15.151:0/651301425 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82500c94f0 con 0x7f825006cc00 2024-09-16T15:44:27.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.623+0000 7f8246000640 1 -- 172.21.15.151:0/651301425 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82400106c0 con 0x7f825006cc00 2024-09-16T15:44:27.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.623+0000 7f824f200640 1 -- 172.21.15.151:0/651301425 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82500c9ab0 con 0x7f825006cc00 2024-09-16T15:44:27.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.623+0000 7f824f200640 1 -- 172.21.15.151:0/651301425 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82500b02e0 con 0x7f825006cc00 2024-09-16T15:44:27.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.625+0000 7f8246000640 1 -- 172.21.15.151:0/651301425 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 17) ==== 101950+0+0 (secure 0 0 0) 0x7f8240010860 con 0x7f825006cc00 2024-09-16T15:44:27.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.627+0000 7f8246000640 1 --2- 172.21.15.151:0/651301425 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8230078fa0 0x7f823007b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:27.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.627+0000 7f8246000640 1 -- 172.21.15.151:0/651301425 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5854+0+0 (secure 0 0 0) 0x7f824009bfe0 con 0x7f825006cc00 2024-09-16T15:44:27.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.627+0000 7f824d400640 1 --2- 172.21.15.151:0/651301425 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8230078fa0 0x7f823007b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:27.630 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.628+0000 7f824d400640 1 --2- 172.21.15.151:0/651301425 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8230078fa0 0x7f823007b460 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f823c033000 tx=0x7f823c004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:27.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.631+0000 7f8246000640 1 -- 172.21.15.151:0/651301425 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f82500b02e0 con 0x7f825006cc00 2024-09-16T15:44:27.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.847+0000 7f824f200640 1 -- 172.21.15.151:0/651301425 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone88453", "target": ["mon-mgr", ""]}) -- 0x7f82500c9ef0 con 0x7f8230078fa0 2024-09-16T15:44:27.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.872+0000 7f8246000640 1 -- 172.21.15.151:0/651301425 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f82500c9ef0 con 0x7f8230078fa0 2024-09-16T15:44:27.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.875+0000 7f824f200640 1 -- 172.21.15.151:0/651301425 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8230078fa0 msgr2=0x7f823007b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:27.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.875+0000 7f824f200640 1 --2- 172.21.15.151:0/651301425 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8230078fa0 0x7f823007b460 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f823c033000 tx=0x7f823c004730 comp rx=0 tx=0).stop 2024-09-16T15:44:27.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.875+0000 7f824f200640 1 -- 172.21.15.151:0/651301425 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f825006cc00 msgr2=0x7f82500ca2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:27.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.875+0000 7f824f200640 1 --2- 172.21.15.151:0/651301425 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f825006cc00 0x7f82500ca2b0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f824000d980 tx=0x7f824000de40 comp rx=0 tx=0).stop 2024-09-16T15:44:27.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.875+0000 7f824f200640 1 -- 172.21.15.151:0/651301425 shutdown_connections 2024-09-16T15:44:27.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.875+0000 7f824f200640 1 --2- 172.21.15.151:0/651301425 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8230078fa0 0x7f823007b460 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:27.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.875+0000 7f824f200640 1 --2- 172.21.15.151:0/651301425 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8250071cf0 0x7f82501f8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:27.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.875+0000 7f824f200640 1 --2- 172.21.15.151:0/651301425 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f82500cc510 0x7f82501f7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:27.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.875+0000 7f824f200640 1 --2- 172.21.15.151:0/651301425 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f825006cc00 0x7f82500ca2b0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:27.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.875+0000 7f824f200640 1 -- 172.21.15.151:0/651301425 >> 172.21.15.151:0/651301425 conn(0x7f82500d0b80 msgr2=0x7f82500b4af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:27.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.875+0000 7f824f200640 1 -- 172.21.15.151:0/651301425 shutdown_connections 2024-09-16T15:44:27.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:27.875+0000 7f824f200640 1 -- 172.21.15.151:0/651301425 wait complete. 2024-09-16T15:44:27.896 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone27010 2024-09-16T15:44:28.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.065+0000 7fd3a5c00640 1 -- 172.21.15.151:0/1036188878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3a015f200 msgr2=0x7fd3a00d4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:28.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.065+0000 7fd3a5c00640 1 --2- 172.21.15.151:0/1036188878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3a015f200 0x7fd3a00d4a20 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd388009830 tx=0x7fd38802a540 comp rx=0 tx=0).stop 2024-09-16T15:44:28.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.065+0000 7fd3a5c00640 1 -- 172.21.15.151:0/1036188878 shutdown_connections 2024-09-16T15:44:28.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.065+0000 7fd3a5c00640 1 --2- 172.21.15.151:0/1036188878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3a015f200 0x7fd3a00d4a20 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:28.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.065+0000 7fd3a5c00640 1 -- 172.21.15.151:0/1036188878 >> 172.21.15.151:0/1036188878 conn(0x7fd3a00d3f00 msgr2=0x7fd3a00d4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:28.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.065+0000 7fd3a5c00640 1 -- 172.21.15.151:0/1036188878 shutdown_connections 2024-09-16T15:44:28.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.065+0000 7fd3a5c00640 1 -- 172.21.15.151:0/1036188878 wait complete. 2024-09-16T15:44:28.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.067+0000 7fd3a5c00640 1 Processor -- start 2024-09-16T15:44:28.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.069+0000 7fd3a5c00640 1 -- start start 2024-09-16T15:44:28.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.069+0000 7fd3a5c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3a00d08d0 0x7fd3a01f8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:28.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.069+0000 7fd3a5c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd3a015e710 0x7fd3a015dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:28.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.069+0000 7fd3a5c00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd3a015f200 0x7fd3a0151fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:28.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.069+0000 7fd3a5c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a00d21e0 con 0x7fd3a015e710 2024-09-16T15:44:28.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.069+0000 7fd3a5c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a00d11a0 con 0x7fd3a00d08d0 2024-09-16T15:44:28.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.070+0000 7fd3a5c00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fd3a00d2360 con 0x7fd3a015f200 2024-09-16T15:44:28.073 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.070+0000 7fd397e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd3a015e710 0x7fd3a015dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:28.073 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.070+0000 7fd39fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3a00d08d0 0x7fd3a01f8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:28.073 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.070+0000 7fd397e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd3a015e710 0x7fd3a015dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:32850/0 (socket says 172.21.15.151:32850) 2024-09-16T15:44:28.073 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.070+0000 7fd397e00640 1 -- 172.21.15.151:0/3087277186 learned_addr learned my addr 172.21.15.151:0/3087277186 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:28.073 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.070+0000 7fd3a5200640 1 --2- 172.21.15.151:0/3087277186 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd3a015f200 0x7fd3a0151fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:28.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.070+0000 7fd397e00640 1 -- 172.21.15.151:0/3087277186 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd3a015f200 msgr2=0x7fd3a0151fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:28.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.070+0000 7fd397e00640 1 --2- 172.21.15.151:0/3087277186 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd3a015f200 0x7fd3a0151fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:28.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.070+0000 7fd397e00640 1 -- 172.21.15.151:0/3087277186 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3a00d08d0 msgr2=0x7fd3a01f8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:28.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.070+0000 7fd397e00640 1 --2- 172.21.15.151:0/3087277186 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3a00d08d0 0x7fd3a01f8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:28.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.070+0000 7fd397e00640 1 -- 172.21.15.151:0/3087277186 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3a00b2090 con 0x7fd3a015e710 2024-09-16T15:44:28.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.070+0000 7fd39fe00640 1 --2- 172.21.15.151:0/3087277186 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3a00d08d0 0x7fd3a01f8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:44:28.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.072+0000 7fd397e00640 1 --2- 172.21.15.151:0/3087277186 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd3a015e710 0x7fd3a015dae0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fd388009950 tx=0x7fd388003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:28.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.072+0000 7fd39cc00640 1 -- 172.21.15.151:0/3087277186 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd388005c90 con 0x7fd3a015e710 2024-09-16T15:44:28.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.072+0000 7fd3a5c00640 1 -- 172.21.15.151:0/3087277186 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3a01f8180 con 0x7fd3a015e710 2024-09-16T15:44:28.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.072+0000 7fd39cc00640 1 -- 172.21.15.151:0/3087277186 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd388005020 con 0x7fd3a015e710 2024-09-16T15:44:28.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.072+0000 7fd39cc00640 1 -- 172.21.15.151:0/3087277186 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd388005450 con 0x7fd3a015e710 2024-09-16T15:44:28.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.072+0000 7fd3a5c00640 1 -- 172.21.15.151:0/3087277186 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3a00cc230 con 0x7fd3a015e710 2024-09-16T15:44:28.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.072+0000 7fd3a5c00640 1 -- 172.21.15.151:0/3087277186 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3a01f7fa0 con 0x7fd3a015e710 2024-09-16T15:44:28.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.074+0000 7fd39cc00640 1 -- 172.21.15.151:0/3087277186 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 17) ==== 101950+0+0 (secure 0 0 0) 0x7fd388039040 con 0x7fd3a015e710 2024-09-16T15:44:28.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.076+0000 7fd39cc00640 1 --2- 172.21.15.151:0/3087277186 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd37c078fa0 0x7fd37c07b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:28.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.076+0000 7fd39cc00640 1 -- 172.21.15.151:0/3087277186 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5854+0+0 (secure 0 0 0) 0x7fd3880bcc00 con 0x7fd3a015e710 2024-09-16T15:44:28.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.076+0000 7fd39fe00640 1 --2- 172.21.15.151:0/3087277186 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd37c078fa0 0x7fd37c07b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:28.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.076+0000 7fd39fe00640 1 --2- 172.21.15.151:0/3087277186 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd37c078fa0 0x7fd37c07b460 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd390000f60 tx=0x7fd390009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:28.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.080+0000 7fd39cc00640 1 -- 172.21.15.151:0/3087277186 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd3a01f7fa0 con 0x7fd3a015e710 2024-09-16T15:44:28.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.300+0000 7fd3a5c00640 1 -- 172.21.15.151:0/3087277186 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone27010", "target": ["mon-mgr", ""]}) -- 0x7fd3a00cc5e0 con 0x7fd37c078fa0 2024-09-16T15:44:28.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.324+0000 7fd39cc00640 1 -- 172.21.15.151:0/3087277186 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd39000fc60 con 0x7fd37c078fa0 2024-09-16T15:44:28.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.328+0000 7fd3a5c00640 1 -- 172.21.15.151:0/3087277186 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd37c078fa0 msgr2=0x7fd37c07b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:28.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.328+0000 7fd3a5c00640 1 --2- 172.21.15.151:0/3087277186 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd37c078fa0 0x7fd37c07b460 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd390000f60 tx=0x7fd390009040 comp rx=0 tx=0).stop 2024-09-16T15:44:28.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.328+0000 7fd3a5c00640 1 -- 172.21.15.151:0/3087277186 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd3a015e710 msgr2=0x7fd3a015dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:28.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.328+0000 7fd3a5c00640 1 --2- 172.21.15.151:0/3087277186 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd3a015e710 0x7fd3a015dae0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fd388009950 tx=0x7fd388003c10 comp rx=0 tx=0).stop 2024-09-16T15:44:28.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.328+0000 7fd3a5c00640 1 -- 172.21.15.151:0/3087277186 shutdown_connections 2024-09-16T15:44:28.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.328+0000 7fd3a5c00640 1 --2- 172.21.15.151:0/3087277186 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd37c078fa0 0x7fd37c07b460 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:28.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.328+0000 7fd3a5c00640 1 --2- 172.21.15.151:0/3087277186 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd3a015f200 0x7fd3a0151fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:28.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.328+0000 7fd3a5c00640 1 --2- 172.21.15.151:0/3087277186 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd3a015e710 0x7fd3a015dae0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:28.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.328+0000 7fd3a5c00640 1 --2- 172.21.15.151:0/3087277186 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3a00d08d0 0x7fd3a01f8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:28.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.328+0000 7fd3a5c00640 1 -- 172.21.15.151:0/3087277186 >> 172.21.15.151:0/3087277186 conn(0x7fd3a00d3f00 msgr2=0x7fd3a00d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:28.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.328+0000 7fd3a5c00640 1 -- 172.21.15.151:0/3087277186 shutdown_connections 2024-09-16T15:44:28.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.328+0000 7fd3a5c00640 1 -- 172.21.15.151:0/3087277186 wait complete. 2024-09-16T15:44:28.345 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-16T15:44:28.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:44:28.602 INFO:teuthology.orchestra.run.smithi151.stdout:2 2024-09-16T15:44:28.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:44:28.611 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-16T15:44:28.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.770+0000 7fec94e00640 1 -- 172.21.15.151:0/3239103261 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec90145750 msgr2=0x7fec90073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:28.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.770+0000 7fec94e00640 1 --2- 172.21.15.151:0/3239103261 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec90145750 0x7fec90073440 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fec900cfb70 tx=0x7fec8002aa30 comp rx=0 tx=0).stop 2024-09-16T15:44:28.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.770+0000 7fec94e00640 1 -- 172.21.15.151:0/3239103261 shutdown_connections 2024-09-16T15:44:28.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.770+0000 7fec94e00640 1 --2- 172.21.15.151:0/3239103261 >> v2:172.21.15.167:3300/0 conn(0x7fec900cccd0 0x7fec900cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:28.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.770+0000 7fec94e00640 1 --2- 172.21.15.151:0/3239103261 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec90145750 0x7fec90073440 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:28.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.770+0000 7fec94e00640 1 -- 172.21.15.151:0/3239103261 >> 172.21.15.151:0/3239103261 conn(0x7fec900b1840 msgr2=0x7fec900b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:28.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.770+0000 7fec94e00640 1 -- 172.21.15.151:0/3239103261 shutdown_connections 2024-09-16T15:44:28.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.770+0000 7fec94e00640 1 -- 172.21.15.151:0/3239103261 wait complete. 2024-09-16T15:44:28.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.772+0000 7fec94e00640 1 Processor -- start 2024-09-16T15:44:28.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.772+0000 7fec94e00640 1 -- start start 2024-09-16T15:44:28.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.773+0000 7fec94e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec9006efa0 0x7fec9015e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:28.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.773+0000 7fec94e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fec900cccd0 0x7fec901f8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:28.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.773+0000 7fec94e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec90145750 0x7fec901f8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:28.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.773+0000 7fec94e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fec900ce480 con 0x7fec90145750 2024-09-16T15:44:28.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.773+0000 7fec94e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fec900ce300 con 0x7fec9006efa0 2024-09-16T15:44:28.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.773+0000 7fec94e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fec900ce600 con 0x7fec900cccd0 2024-09-16T15:44:28.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.773+0000 7fec8e800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec90145750 0x7fec901f8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:28.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.774+0000 7fec8e800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec90145750 0x7fec901f8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:32866/0 (socket says 172.21.15.151:32866) 2024-09-16T15:44:28.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.774+0000 7fec8e800640 1 -- 172.21.15.151:0/3424881193 learned_addr learned my addr 172.21.15.151:0/3424881193 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:28.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.774+0000 7fec8de00640 1 --2- 172.21.15.151:0/3424881193 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec9006efa0 0x7fec9015e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:28.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.774+0000 7fec8d400640 1 --2- 172.21.15.151:0/3424881193 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fec900cccd0 0x7fec901f8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:28.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.774+0000 7fec8e800640 1 -- 172.21.15.151:0/3424881193 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fec900cccd0 msgr2=0x7fec901f8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:28.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.774+0000 7fec8e800640 1 --2- 172.21.15.151:0/3424881193 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fec900cccd0 0x7fec901f8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:28.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.774+0000 7fec8e800640 1 -- 172.21.15.151:0/3424881193 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec9006efa0 msgr2=0x7fec9015e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:28.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.774+0000 7fec8e800640 1 --2- 172.21.15.151:0/3424881193 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec9006efa0 0x7fec9015e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:28.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.774+0000 7fec8e800640 1 -- 172.21.15.151:0/3424881193 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec9013c320 con 0x7fec90145750 2024-09-16T15:44:28.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.774+0000 7fec8de00640 1 --2- 172.21.15.151:0/3424881193 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec9006efa0 0x7fec9015e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:44:28.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.774+0000 7fec8e800640 1 --2- 172.21.15.151:0/3424881193 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec90145750 0x7fec901f8680 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fec8800e970 tx=0x7fec8800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:28.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.774+0000 7fec86000640 1 -- 172.21.15.151:0/3424881193 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec8800ccc0 con 0x7fec90145750 2024-09-16T15:44:28.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.775+0000 7fec86000640 1 -- 172.21.15.151:0/3424881193 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fec8800ce60 con 0x7fec90145750 2024-09-16T15:44:28.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.775+0000 7fec86000640 1 -- 172.21.15.151:0/3424881193 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec88005280 con 0x7fec90145750 2024-09-16T15:44:28.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.775+0000 7fec94e00640 1 -- 172.21.15.151:0/3424881193 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec90155040 con 0x7fec90145750 2024-09-16T15:44:28.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.775+0000 7fec94e00640 1 -- 172.21.15.151:0/3424881193 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec900b2ce0 con 0x7fec90145750 2024-09-16T15:44:28.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.776+0000 7fec94e00640 1 -- 172.21.15.151:0/3424881193 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec900cdbb0 con 0x7fec90145750 2024-09-16T15:44:28.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.777+0000 7fec86000640 1 -- 172.21.15.151:0/3424881193 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 102103+0+0 (secure 0 0 0) 0x7fec88005420 con 0x7fec90145750 2024-09-16T15:44:28.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.778+0000 7fec86000640 1 --2- 172.21.15.151:0/3424881193 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fec70079150 0x7fec7007b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:28.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.778+0000 7fec86000640 1 -- 172.21.15.151:0/3424881193 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5854+0+0 (secure 0 0 0) 0x7fec8809c730 con 0x7fec90145750 2024-09-16T15:44:28.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.778+0000 7fec8de00640 1 --2- 172.21.15.151:0/3424881193 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fec70079150 0x7fec7007b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:28.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.779+0000 7fec8de00640 1 --2- 172.21.15.151:0/3424881193 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fec70079150 0x7fec7007b610 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fec80035000 tx=0x7fec80004ab0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:28.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.782+0000 7fec86000640 1 -- 172.21.15.151:0/3424881193 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fec9013c320 con 0x7fec90145750 2024-09-16T15:44:28.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.969+0000 7fec94e00640 1 -- 172.21.15.151:0/3424881193 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7fec90144db0 con 0x7fec90145750 2024-09-16T15:44:28.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.969+0000 7fec86000640 1 -- 172.21.15.151:0/3424881193 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v32)=0 v32) ==== 131+0+0 (secure 0 0 0) 0x7fec88063d60 con 0x7fec90145750 2024-09-16T15:44:28.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.972+0000 7fec94e00640 1 -- 172.21.15.151:0/3424881193 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fec70079150 msgr2=0x7fec7007b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:28.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.972+0000 7fec94e00640 1 --2- 172.21.15.151:0/3424881193 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fec70079150 0x7fec7007b610 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fec80035000 tx=0x7fec80004ab0 comp rx=0 tx=0).stop 2024-09-16T15:44:28.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.972+0000 7fec94e00640 1 -- 172.21.15.151:0/3424881193 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec90145750 msgr2=0x7fec901f8680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:28.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.972+0000 7fec94e00640 1 --2- 172.21.15.151:0/3424881193 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec90145750 0x7fec901f8680 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fec8800e970 tx=0x7fec8800ed70 comp rx=0 tx=0).stop 2024-09-16T15:44:28.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.972+0000 7fec94e00640 1 -- 172.21.15.151:0/3424881193 shutdown_connections 2024-09-16T15:44:28.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.972+0000 7fec94e00640 1 --2- 172.21.15.151:0/3424881193 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fec70079150 0x7fec7007b610 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:28.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.972+0000 7fec94e00640 1 --2- 172.21.15.151:0/3424881193 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec90145750 0x7fec901f8680 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:28.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.972+0000 7fec94e00640 1 --2- 172.21.15.151:0/3424881193 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fec900cccd0 0x7fec901f8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:28.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.972+0000 7fec94e00640 1 --2- 172.21.15.151:0/3424881193 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec9006efa0 0x7fec9015e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:28.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.972+0000 7fec94e00640 1 -- 172.21.15.151:0/3424881193 >> 172.21.15.151:0/3424881193 conn(0x7fec900b1840 msgr2=0x7fec900b0610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:28.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.972+0000 7fec94e00640 1 -- 172.21.15.151:0/3424881193 shutdown_connections 2024-09-16T15:44:28.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:28.972+0000 7fec94e00640 1 -- 172.21.15.151:0/3424881193 wait complete. 2024-09-16T15:44:28.989 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-16T15:44:28.989 DEBUG:teuthology.orchestra.run.smithi151:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:44:29.077 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:44:29.204 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-16T15:44:29.205 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:44:29.205 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:44:29.205 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:44:29.260 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:44:29.260 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:44:29.310 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-16T15:44:29.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:29.468+0000 7fec97600640 1 -- 172.21.15.151:0/1684000146 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec90073fb0 msgr2=0x7fec90074460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:29.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:29.468+0000 7fec97600640 1 --2- 172.21.15.151:0/1684000146 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec90073fb0 0x7fec90074460 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fec88005fd0 tx=0x7fec8802a820 comp rx=0 tx=0).stop 2024-09-16T15:44:29.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:29.468+0000 7fec97600640 1 -- 172.21.15.151:0/1684000146 shutdown_connections 2024-09-16T15:44:29.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:29.468+0000 7fec97600640 1 --2- 172.21.15.151:0/1684000146 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec90073fb0 0x7fec90074460 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:29.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:29.468+0000 7fec97600640 1 --2- 172.21.15.151:0/1684000146 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fec9015f5a0 0x7fec900b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:29.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:29.468+0000 7fec97600640 1 -- 172.21.15.151:0/1684000146 >> 172.21.15.151:0/1684000146 conn(0x7fec900b1950 msgr2=0x7fec900b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:29.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:29.469+0000 7fec97600640 1 -- 172.21.15.151:0/1684000146 shutdown_connections 2024-09-16T15:44:29.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:29.469+0000 7fec97600640 1 -- 172.21.15.151:0/1684000146 wait complete. 2024-09-16T15:44:29.471 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:29.470+0000 7fec97600640 1 Processor -- start 2024-09-16T15:44:29.471 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:29.470+0000 7fec97600640 1 -- start start 2024-09-16T15:44:29.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:29.471+0000 7fec97600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fec90073fb0 0x7fec9009bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:29.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:29.471+0000 7fec97600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec900ccf00 0x7fec90141f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:29.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:29.471+0000 7fec97600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec9015f5a0 0x7fec9014c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:29.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:29.471+0000 7fec97600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fec9006f2e0 con 0x7fec9015f5a0 2024-09-16T15:44:29.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:29.471+0000 7fec97600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fec9006f160 con 0x7fec900ccf00 2024-09-16T15:44:29.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:29.471+0000 7fec97600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fec9006f460 con 0x7fec90073fb0 2024-09-16T15:44:29.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:29.471+0000 7fec95600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec9015f5a0 0x7fec9014c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:29.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:29.472+0000 7fec94c00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fec90073fb0 0x7fec9009bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:29.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:29.472+0000 7fec95600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec9015f5a0 0x7fec9014c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:32870/0 (socket says 172.21.15.151:32870) 2024-09-16T15:44:29.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:29.472+0000 7fec95600640 1 -- 172.21.15.151:0/2398280508 learned_addr learned my addr 172.21.15.151:0/2398280508 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:29.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:29.472+0000 7fec87e00640 1 --2- 172.21.15.151:0/2398280508 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec900ccf00 0x7fec90141f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:29.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:29.472+0000 7fec95600640 1 -- 172.21.15.151:0/2398280508 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fec90073fb0 msgr2=0x7fec9009bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:29.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:29.472+0000 7fec95600640 1 --2- 172.21.15.151:0/2398280508 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fec90073fb0 0x7fec9009bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:29.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:29.472+0000 7fec95600640 1 -- 172.21.15.151:0/2398280508 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec900ccf00 msgr2=0x7fec90141f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:29.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:29.472+0000 7fec95600640 1 --2- 172.21.15.151:0/2398280508 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec900ccf00 0x7fec90141f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:29.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:29.472+0000 7fec95600640 1 -- 172.21.15.151:0/2398280508 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec9009c420 con 0x7fec9015f5a0 2024-09-16T15:44:29.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:29.472+0000 7fec95600640 1 --2- 172.21.15.151:0/2398280508 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec9015f5a0 0x7fec9014c1f0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fec88009cd0 tx=0x7fec88032c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:29.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:29.472+0000 7fec8d600640 1 -- 172.21.15.151:0/2398280508 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec88037040 con 0x7fec9015f5a0 2024-09-16T15:44:29.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:29.472+0000 7fec8d600640 1 -- 172.21.15.151:0/2398280508 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fec88042070 con 0x7fec9015f5a0 2024-09-16T15:44:29.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:29.472+0000 7fec8d600640 1 -- 172.21.15.151:0/2398280508 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec88003e60 con 0x7fec9015f5a0 2024-09-16T15:44:29.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:29.472+0000 7fec97600640 1 -- 172.21.15.151:0/2398280508 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec901f8310 con 0x7fec9015f5a0 2024-09-16T15:44:29.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:29.473+0000 7fec97600640 1 -- 172.21.15.151:0/2398280508 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec901f88d0 con 0x7fec9015f5a0 2024-09-16T15:44:29.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:29.474+0000 7fec97600640 1 -- 172.21.15.151:0/2398280508 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec900ad200 con 0x7fec9015f5a0 2024-09-16T15:44:29.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:29.475+0000 7fec8d600640 1 -- 172.21.15.151:0/2398280508 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 102103+0+0 (secure 0 0 0) 0x7fec88004280 con 0x7fec9015f5a0 2024-09-16T15:44:29.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:29.476+0000 7fec8d600640 1 --2- 172.21.15.151:0/2398280508 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fec6c079150 0x7fec6c07b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:29.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:29.476+0000 7fec8d600640 1 -- 172.21.15.151:0/2398280508 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5854+0+0 (secure 0 0 0) 0x7fec880bc710 con 0x7fec9015f5a0 2024-09-16T15:44:29.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:29.476+0000 7fec94c00640 1 --2- 172.21.15.151:0/2398280508 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fec6c079150 0x7fec6c07b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:29.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:29.477+0000 7fec94c00640 1 --2- 172.21.15.151:0/2398280508 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fec6c079150 0x7fec6c07b610 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fec800095f0 tx=0x7fec80005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:29.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:29.480+0000 7fec8d600640 1 -- 172.21.15.151:0/2398280508 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fec88083700 con 0x7fec9015f5a0 2024-09-16T15:44:29.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:29.679+0000 7fec97600640 1 -- 172.21.15.151:0/2398280508 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fec900b0d10 con 0x7fec9015f5a0 2024-09-16T15:44:29.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:29.679+0000 7fec8d600640 1 -- 172.21.15.151:0/2398280508 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v50) ==== 80+0+0 (secure 0 0 0) 0x7fec88088130 con 0x7fec9015f5a0 2024-09-16T15:44:29.680 INFO:teuthology.orchestra.run.smithi151.stderr:listed 0 entries 2024-09-16T15:44:29.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:29.681+0000 7fec97600640 1 -- 172.21.15.151:0/2398280508 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fec6c079150 msgr2=0x7fec6c07b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:29.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:29.681+0000 7fec97600640 1 --2- 172.21.15.151:0/2398280508 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fec6c079150 0x7fec6c07b610 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fec800095f0 tx=0x7fec80005f90 comp rx=0 tx=0).stop 2024-09-16T15:44:29.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:29.681+0000 7fec97600640 1 -- 172.21.15.151:0/2398280508 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec9015f5a0 msgr2=0x7fec9014c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:29.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:29.681+0000 7fec97600640 1 --2- 172.21.15.151:0/2398280508 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec9015f5a0 0x7fec9014c1f0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fec88009cd0 tx=0x7fec88032c60 comp rx=0 tx=0).stop 2024-09-16T15:44:29.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:29.681+0000 7fec97600640 1 -- 172.21.15.151:0/2398280508 shutdown_connections 2024-09-16T15:44:29.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:29.681+0000 7fec97600640 1 --2- 172.21.15.151:0/2398280508 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fec6c079150 0x7fec6c07b610 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:29.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:29.681+0000 7fec97600640 1 --2- 172.21.15.151:0/2398280508 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec900ccf00 0x7fec90141f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:29.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:29.681+0000 7fec97600640 1 --2- 172.21.15.151:0/2398280508 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec9015f5a0 0x7fec9014c1f0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:29.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:29.681+0000 7fec97600640 1 --2- 172.21.15.151:0/2398280508 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fec90073fb0 0x7fec9009bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:29.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:29.681+0000 7fec97600640 1 -- 172.21.15.151:0/2398280508 >> 172.21.15.151:0/2398280508 conn(0x7fec900b1950 msgr2=0x7fec9014eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:29.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:29.681+0000 7fec97600640 1 -- 172.21.15.151:0/2398280508 shutdown_connections 2024-09-16T15:44:29.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:29.682+0000 7fec97600640 1 -- 172.21.15.151:0/2398280508 wait complete. 2024-09-16T15:44:29.699 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-16T15:44:29.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:29.700 DEBUG:teuthology.orchestra.run.smithi151:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-16T15:44:29.791 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi151.front.sepia.ceph.com 2024-09-16T15:44:29.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:29.792 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-16T15:44:29.839 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:44:29.839 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-16T15:44:29.839 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:44:29.874 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:44:29.874 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:44:29.874 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:44:29.874 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:44:29.914 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T15:44:29.914 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:44:29.914 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-16T15:44:29.914 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:44:29.949 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:44:29.950 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:44:30.133 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.131+0000 7f30fda00640 1 -- 172.21.15.151:0/873199198 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f30f815f5a0 msgr2=0x7f30f80b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:30.133 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.131+0000 7f30fda00640 1 --2- 172.21.15.151:0/873199198 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f30f815f5a0 0x7f30f80b2440 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f30f0005fd0 tx=0x7f30f002a770 comp rx=0 tx=0).stop 2024-09-16T15:44:30.134 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.132+0000 7f30fda00640 1 -- 172.21.15.151:0/873199198 shutdown_connections 2024-09-16T15:44:30.134 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.132+0000 7f30fda00640 1 --2- 172.21.15.151:0/873199198 >> v2:172.21.15.167:3300/0 conn(0x7f30f8073fb0 0x7f30f8074460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:30.134 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.132+0000 7f30fda00640 1 --2- 172.21.15.151:0/873199198 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f30f815f5a0 0x7f30f80b2440 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:30.134 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.132+0000 7f30fda00640 1 -- 172.21.15.151:0/873199198 >> 172.21.15.151:0/873199198 conn(0x7f30f80b1950 msgr2=0x7f30f80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:30.134 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.132+0000 7f30fda00640 1 -- 172.21.15.151:0/873199198 shutdown_connections 2024-09-16T15:44:30.134 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.133+0000 7f30fda00640 1 -- 172.21.15.151:0/873199198 wait complete. 2024-09-16T15:44:30.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.135+0000 7f30fda00640 1 Processor -- start 2024-09-16T15:44:30.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.136+0000 7f30fda00640 1 -- start start 2024-09-16T15:44:30.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.137+0000 7f30fda00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30f8073fb0 0x7f30f80cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:30.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.138+0000 7f30fda00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f30f80ccf00 0x7f30f80d0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:30.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.138+0000 7f30fda00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30f815f5a0 0x7f30f81455c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:30.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.138+0000 7f30fda00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f30f806f2e0 con 0x7f30f8073fb0 2024-09-16T15:44:30.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.138+0000 7f30fda00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f30f806f160 con 0x7f30f815f5a0 2024-09-16T15:44:30.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.138+0000 7f30fda00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f30f806f460 con 0x7f30f80ccf00 2024-09-16T15:44:30.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.138+0000 7f30f6a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30f8073fb0 0x7f30f80cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:30.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.138+0000 7f30f6a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30f8073fb0 0x7f30f80cdae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:32876/0 (socket says 172.21.15.151:32876) 2024-09-16T15:44:30.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.138+0000 7f30f6a00640 1 -- 172.21.15.151:0/1791052849 learned_addr learned my addr 172.21.15.151:0/1791052849 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:30.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.139+0000 7f30f6a00640 1 -- 172.21.15.151:0/1791052849 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f30f80ccf00 msgr2=0x7f30f80d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:30.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.139+0000 7f30f6000640 1 --2- 172.21.15.151:0/1791052849 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f30f80ccf00 0x7f30f80d0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:30.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.139+0000 7f30f7400640 1 --2- 172.21.15.151:0/1791052849 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30f815f5a0 0x7f30f81455c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:30.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.139+0000 7f30f6a00640 1 --2- 172.21.15.151:0/1791052849 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f30f80ccf00 0x7f30f80d0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:30.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.139+0000 7f30f6a00640 1 -- 172.21.15.151:0/1791052849 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30f815f5a0 msgr2=0x7f30f81455c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:30.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.139+0000 7f30f6a00640 1 --2- 172.21.15.151:0/1791052849 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30f815f5a0 0x7f30f81455c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:30.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.139+0000 7f30f6a00640 1 -- 172.21.15.151:0/1791052849 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30f80d0560 con 0x7f30f8073fb0 2024-09-16T15:44:30.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.139+0000 7f30f6000640 1 --2- 172.21.15.151:0/1791052849 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f30f80ccf00 0x7f30f80d0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:44:30.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.139+0000 7f30f6a00640 1 --2- 172.21.15.151:0/1791052849 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30f8073fb0 0x7f30f80cdae0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f30f0006010 tx=0x7f30f0032f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:30.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.140+0000 7f30eea00640 1 -- 172.21.15.151:0/1791052849 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f0041020 con 0x7f30f8073fb0 2024-09-16T15:44:30.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.140+0000 7f30eea00640 1 -- 172.21.15.151:0/1791052849 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f30f0004150 con 0x7f30f8073fb0 2024-09-16T15:44:30.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.140+0000 7f30eea00640 1 -- 172.21.15.151:0/1791052849 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f0036040 con 0x7f30f8073fb0 2024-09-16T15:44:30.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.140+0000 7f30fda00640 1 -- 172.21.15.151:0/1791052849 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30f80ce170 con 0x7f30f8073fb0 2024-09-16T15:44:30.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.140+0000 7f30fda00640 1 -- 172.21.15.151:0/1791052849 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30f80ce730 con 0x7f30f8073fb0 2024-09-16T15:44:30.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.143+0000 7f30fda00640 1 -- 172.21.15.151:0/1791052849 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30f806f990 con 0x7f30f8073fb0 2024-09-16T15:44:30.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.143+0000 7f30eea00640 1 -- 172.21.15.151:0/1791052849 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 102103+0+0 (secure 0 0 0) 0x7f30f0044020 con 0x7f30f8073fb0 2024-09-16T15:44:30.146 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.145+0000 7f30eea00640 1 --2- 172.21.15.151:0/1791052849 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f30d8079150 0x7f30d807b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:30.146 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.145+0000 7f30eea00640 1 -- 172.21.15.151:0/1791052849 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5854+0+0 (secure 0 0 0) 0x7f30f00b9f40 con 0x7f30f8073fb0 2024-09-16T15:44:30.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.146+0000 7f30f6000640 1 --2- 172.21.15.151:0/1791052849 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f30d8079150 0x7f30d807b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:30.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.148+0000 7f30f6000640 1 --2- 172.21.15.151:0/1791052849 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f30d8079150 0x7f30d807b610 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f30f81fc230 tx=0x7f30e4009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:30.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.154+0000 7f30eea00640 1 -- 172.21.15.151:0/1791052849 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f30f004b070 con 0x7f30f8073fb0 2024-09-16T15:44:30.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.581+0000 7f30fda00640 1 -- 172.21.15.151:0/1791052849 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f30f815eb60 con 0x7f30f8073fb0 2024-09-16T15:44:30.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.585+0000 7f30eea00640 1 -- 172.21.15.151:0/1791052849 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+5232 (secure 0 0 0) 0x7f30f0084060 con 0x7f30f8073fb0 2024-09-16T15:44:30.586 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:44:30.586 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":28,"btime":"2024-09-16T15:42:38:048451+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5272,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1043845735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1043845735},{"type":"v1","addr":"172.21.15.167:6836","nonce":1043845735}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5275,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/816612106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":816612106},{"type":"v1","addr":"172.21.15.167:6837","nonce":816612106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5397,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/1986755100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1986755100},{"type":"v1","addr":"172.21.15.151:6834","nonce":1986755100}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:42:24.411561+0000","modified":"2024-09-16T15:42:27.425402+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5400":{"gid":5400,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.151:6835/1974411916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":1974411916},{"type":"v1","addr":"172.21.15.151:6835","nonce":1974411916}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5400,"qdb_cluster":[5400]},"id":3}]} 2024-09-16T15:44:30.586 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 28 2024-09-16T15:44:30.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.588+0000 7f30fda00640 1 -- 172.21.15.151:0/1791052849 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f30d8079150 msgr2=0x7f30d807b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:30.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.588+0000 7f30fda00640 1 --2- 172.21.15.151:0/1791052849 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f30d8079150 0x7f30d807b610 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f30f81fc230 tx=0x7f30e4009040 comp rx=0 tx=0).stop 2024-09-16T15:44:30.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.588+0000 7f30fda00640 1 -- 172.21.15.151:0/1791052849 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30f8073fb0 msgr2=0x7f30f80cdae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:30.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.588+0000 7f30fda00640 1 --2- 172.21.15.151:0/1791052849 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30f8073fb0 0x7f30f80cdae0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f30f0006010 tx=0x7f30f0032f10 comp rx=0 tx=0).stop 2024-09-16T15:44:30.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.589+0000 7f30fda00640 1 -- 172.21.15.151:0/1791052849 shutdown_connections 2024-09-16T15:44:30.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.589+0000 7f30fda00640 1 --2- 172.21.15.151:0/1791052849 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f30d8079150 0x7f30d807b610 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:30.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.589+0000 7f30fda00640 1 --2- 172.21.15.151:0/1791052849 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30f815f5a0 0x7f30f81455c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:30.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.589+0000 7f30fda00640 1 --2- 172.21.15.151:0/1791052849 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f30f80ccf00 0x7f30f80d0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:30.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.589+0000 7f30fda00640 1 --2- 172.21.15.151:0/1791052849 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30f8073fb0 0x7f30f80cdae0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:30.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.589+0000 7f30fda00640 1 -- 172.21.15.151:0/1791052849 >> 172.21.15.151:0/1791052849 conn(0x7f30f80b1950 msgr2=0x7f30f80b47f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:30.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.589+0000 7f30fda00640 1 -- 172.21.15.151:0/1791052849 shutdown_connections 2024-09-16T15:44:30.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.589+0000 7f30fda00640 1 -- 172.21.15.151:0/1791052849 wait complete. 2024-09-16T15:44:30.615 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:44:30.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.868+0000 7f21f0c00640 1 -- 172.21.15.151:0/4008328539 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f21ec0cff50 msgr2=0x7f21ec0b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:30.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.868+0000 7f21f0c00640 1 --2- 172.21.15.151:0/4008328539 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f21ec0cff50 0x7f21ec0b3410 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f21e4005fd0 tx=0x7f21e402a8e0 comp rx=0 tx=0).stop 2024-09-16T15:44:30.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.869+0000 7f21f0c00640 1 -- 172.21.15.151:0/4008328539 shutdown_connections 2024-09-16T15:44:30.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.869+0000 7f21f0c00640 1 --2- 172.21.15.151:0/4008328539 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f21ec0cff50 0x7f21ec0b3410 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:30.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.869+0000 7f21f0c00640 1 --2- 172.21.15.151:0/4008328539 >> v2:172.21.15.151:3300/0 conn(0x7f21ec161430 0x7f21ec0b5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:30.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.869+0000 7f21f0c00640 1 -- 172.21.15.151:0/4008328539 >> 172.21.15.151:0/4008328539 conn(0x7f21ec0d3dc0 msgr2=0x7f21ec0d41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:30.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.869+0000 7f21f0c00640 1 -- 172.21.15.151:0/4008328539 shutdown_connections 2024-09-16T15:44:30.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.869+0000 7f21f0c00640 1 -- 172.21.15.151:0/4008328539 wait complete. 2024-09-16T15:44:30.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.871+0000 7f21f0c00640 1 Processor -- start 2024-09-16T15:44:30.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.872+0000 7f21f0c00640 1 -- start start 2024-09-16T15:44:30.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.873+0000 7f21f0c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f21ec0b4aa0 0x7f21ec0cd660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:30.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.873+0000 7f21f0c00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f21ec0cff50 0x7f21ec0a0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:30.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.873+0000 7f21f0c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f21ec161430 0x7f21ec0a13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:30.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.873+0000 7f21f0c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f21ec0b3ce0 con 0x7f21ec0b4aa0 2024-09-16T15:44:30.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.873+0000 7f21f0c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f21ec0b3b60 con 0x7f21ec161430 2024-09-16T15:44:30.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.873+0000 7f21f0c00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f21ec0b3e60 con 0x7f21ec0cff50 2024-09-16T15:44:30.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.873+0000 7f21ebe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f21ec161430 0x7f21ec0a13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:30.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.873+0000 7f21ebe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f21ec161430 0x7f21ec0a13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.151:36090/0 (socket says 172.21.15.151:36090) 2024-09-16T15:44:30.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.873+0000 7f21ebe00640 1 -- 172.21.15.151:0/2946085224 learned_addr learned my addr 172.21.15.151:0/2946085224 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:30.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.874+0000 7f21ebe00640 1 -- 172.21.15.151:0/2946085224 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f21ec0cff50 msgr2=0x7f21ec0a0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:30.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.874+0000 7f21ebe00640 1 --2- 172.21.15.151:0/2946085224 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f21ec0cff50 0x7f21ec0a0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:30.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.874+0000 7f21ebe00640 1 -- 172.21.15.151:0/2946085224 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f21ec0b4aa0 msgr2=0x7f21ec0cd660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:30.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.874+0000 7f21eb400640 1 --2- 172.21.15.151:0/2946085224 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f21ec0b4aa0 0x7f21ec0cd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:30.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.874+0000 7f21ebe00640 1 --2- 172.21.15.151:0/2946085224 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f21ec0b4aa0 0x7f21ec0cd660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:30.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.874+0000 7f21ebe00640 1 -- 172.21.15.151:0/2946085224 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21ec0cdba0 con 0x7f21ec161430 2024-09-16T15:44:30.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.874+0000 7f21eb400640 1 --2- 172.21.15.151:0/2946085224 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f21ec0b4aa0 0x7f21ec0cd660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:44:30.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.875+0000 7f21ebe00640 1 --2- 172.21.15.151:0/2946085224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f21ec161430 0x7f21ec0a13c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f21e4033040 tx=0x7f21e4003f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:30.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.876+0000 7f21e3400640 1 -- 172.21.15.151:0/2946085224 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e4004040 con 0x7f21ec161430 2024-09-16T15:44:30.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.876+0000 7f21e3400640 1 -- 172.21.15.151:0/2946085224 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f21e40041e0 con 0x7f21ec161430 2024-09-16T15:44:30.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.876+0000 7f21e3400640 1 -- 172.21.15.151:0/2946085224 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e40045f0 con 0x7f21ec161430 2024-09-16T15:44:30.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.876+0000 7f21f0c00640 1 -- 172.21.15.151:0/2946085224 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21ec0c81f0 con 0x7f21ec161430 2024-09-16T15:44:30.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.876+0000 7f21f0c00640 1 -- 172.21.15.151:0/2946085224 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21ec0c87b0 con 0x7f21ec161430 2024-09-16T15:44:30.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.878+0000 7f21f0c00640 1 -- 172.21.15.151:0/2946085224 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21ec0d3a60 con 0x7f21ec161430 2024-09-16T15:44:30.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.882+0000 7f21e3400640 1 -- 172.21.15.151:0/2946085224 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 102103+0+0 (secure 0 0 0) 0x7f21e4004b40 con 0x7f21ec161430 2024-09-16T15:44:30.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.883+0000 7f21e3400640 1 --2- 172.21.15.151:0/2946085224 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f21cc0791a0 0x7f21cc07b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:30.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.883+0000 7f21e3400640 1 -- 172.21.15.151:0/2946085224 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5854+0+0 (secure 0 0 0) 0x7f21e40bab60 con 0x7f21ec161430 2024-09-16T15:44:30.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.884+0000 7f21eb400640 1 --2- 172.21.15.151:0/2946085224 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f21cc0791a0 0x7f21cc07b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:30.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.885+0000 7f21eb400640 1 --2- 172.21.15.151:0/2946085224 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f21cc0791a0 0x7f21cc07b660 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f21dc009ab0 tx=0x7f21dc005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:30.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:30.889+0000 7f21e3400640 1 -- 172.21.15.151:0/2946085224 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f21ec0d3a60 con 0x7f21ec161430 2024-09-16T15:44:31.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.296+0000 7f21f0c00640 1 -- 172.21.15.151:0/2946085224 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f21ec0b3410 con 0x7f21ec161430 2024-09-16T15:44:31.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.297+0000 7f21e3400640 1 -- 172.21.15.151:0/2946085224 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+5232 (secure 0 0 0) 0x7f21ec0ce7a0 con 0x7f21ec161430 2024-09-16T15:44:31.301 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:44:31.302 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":28,"btime":"2024-09-16T15:42:38:048451+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5272,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1043845735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1043845735},{"type":"v1","addr":"172.21.15.167:6836","nonce":1043845735}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5275,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/816612106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":816612106},{"type":"v1","addr":"172.21.15.167:6837","nonce":816612106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5397,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/1986755100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1986755100},{"type":"v1","addr":"172.21.15.151:6834","nonce":1986755100}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:42:24.411561+0000","modified":"2024-09-16T15:42:27.425402+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5400":{"gid":5400,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.151:6835/1974411916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":1974411916},{"type":"v1","addr":"172.21.15.151:6835","nonce":1974411916}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5400,"qdb_cluster":[5400]},"id":3}]} 2024-09-16T15:44:31.302 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 28 2024-09-16T15:44:31.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.301+0000 7f21f0c00640 1 -- 172.21.15.151:0/2946085224 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f21cc0791a0 msgr2=0x7f21cc07b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:31.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.301+0000 7f21f0c00640 1 --2- 172.21.15.151:0/2946085224 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f21cc0791a0 0x7f21cc07b660 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f21dc009ab0 tx=0x7f21dc005f90 comp rx=0 tx=0).stop 2024-09-16T15:44:31.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.301+0000 7f21f0c00640 1 -- 172.21.15.151:0/2946085224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f21ec161430 msgr2=0x7f21ec0a13c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:31.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.301+0000 7f21f0c00640 1 --2- 172.21.15.151:0/2946085224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f21ec161430 0x7f21ec0a13c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f21e4033040 tx=0x7f21e4003f10 comp rx=0 tx=0).stop 2024-09-16T15:44:31.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.301+0000 7f21f0c00640 1 -- 172.21.15.151:0/2946085224 shutdown_connections 2024-09-16T15:44:31.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.301+0000 7f21f0c00640 1 --2- 172.21.15.151:0/2946085224 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f21cc0791a0 0x7f21cc07b660 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:31.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.301+0000 7f21f0c00640 1 --2- 172.21.15.151:0/2946085224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f21ec161430 0x7f21ec0a13c0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:31.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.301+0000 7f21f0c00640 1 --2- 172.21.15.151:0/2946085224 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f21ec0cff50 0x7f21ec0a0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:31.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.301+0000 7f21f0c00640 1 --2- 172.21.15.151:0/2946085224 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f21ec0b4aa0 0x7f21ec0cd660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:31.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.301+0000 7f21f0c00640 1 -- 172.21.15.151:0/2946085224 >> 172.21.15.151:0/2946085224 conn(0x7f21ec0d3dc0 msgr2=0x7f21ec0b1890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:31.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.301+0000 7f21f0c00640 1 -- 172.21.15.151:0/2946085224 shutdown_connections 2024-09-16T15:44:31.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.301+0000 7f21f0c00640 1 -- 172.21.15.151:0/2946085224 wait complete. 2024-09-16T15:44:31.326 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:44:31.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.577+0000 7f5131600640 1 -- 172.21.15.151:0/2674518077 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f512c0aecc0 msgr2=0x7f512c15dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:31.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.577+0000 7f5131600640 1 --2- 172.21.15.151:0/2674518077 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f512c0aecc0 0x7f512c15dc30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f511c009900 tx=0x7f511c02a600 comp rx=0 tx=0).stop 2024-09-16T15:44:31.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.577+0000 7f5131600640 1 -- 172.21.15.151:0/2674518077 shutdown_connections 2024-09-16T15:44:31.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.577+0000 7f5131600640 1 --2- 172.21.15.151:0/2674518077 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f512c0aecc0 0x7f512c15dc30 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:31.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.577+0000 7f5131600640 1 --2- 172.21.15.151:0/2674518077 >> v2:172.21.15.151:3300/0 conn(0x7f512c0caa50 0x7f512c0cae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:31.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.577+0000 7f5131600640 1 -- 172.21.15.151:0/2674518077 >> 172.21.15.151:0/2674518077 conn(0x7f512c0cfa20 msgr2=0x7f512c0cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:31.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.577+0000 7f5131600640 1 -- 172.21.15.151:0/2674518077 shutdown_connections 2024-09-16T15:44:31.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.577+0000 7f5131600640 1 -- 172.21.15.151:0/2674518077 wait complete. 2024-09-16T15:44:31.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.580+0000 7f5131600640 1 Processor -- start 2024-09-16T15:44:31.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.580+0000 7f5131600640 1 -- start start 2024-09-16T15:44:31.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.581+0000 7f5131600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f512c0aecc0 0x7f512c1f4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:31.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.581+0000 7f5131600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f512c0caa50 0x7f512c1f4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:31.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.581+0000 7f5131600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f512c0cb370 0x7f512c0c5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:31.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.581+0000 7f5131600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f512c0cdd00 con 0x7f512c0caa50 2024-09-16T15:44:31.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.581+0000 7f5131600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f512c15e680 con 0x7f512c0aecc0 2024-09-16T15:44:31.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.581+0000 7f5131600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f512c0cde80 con 0x7f512c0cb370 2024-09-16T15:44:31.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.581+0000 7f512be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f512c0aecc0 0x7f512c1f4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:31.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.581+0000 7f512b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f512c0caa50 0x7f512c1f4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:31.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.581+0000 7f512b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f512c0caa50 0x7f512c1f4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:32916/0 (socket says 172.21.15.151:32916) 2024-09-16T15:44:31.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.581+0000 7f512b400640 1 -- 172.21.15.151:0/4292365471 learned_addr learned my addr 172.21.15.151:0/4292365471 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:31.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.581+0000 7f512be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f512c0aecc0 0x7f512c1f4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.151:36098/0 (socket says 172.21.15.151:36098) 2024-09-16T15:44:31.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.582+0000 7f5130c00640 1 --2- 172.21.15.151:0/4292365471 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f512c0cb370 0x7f512c0c5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:31.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.582+0000 7f512be00640 1 -- 172.21.15.151:0/4292365471 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f512c0cb370 msgr2=0x7f512c0c5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:31.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.582+0000 7f512be00640 1 --2- 172.21.15.151:0/4292365471 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f512c0cb370 0x7f512c0c5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:31.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.582+0000 7f512be00640 1 -- 172.21.15.151:0/4292365471 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f512c0caa50 msgr2=0x7f512c1f4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:31.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.582+0000 7f512be00640 1 --2- 172.21.15.151:0/4292365471 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f512c0caa50 0x7f512c1f4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:31.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.582+0000 7f512be00640 1 -- 172.21.15.151:0/4292365471 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f512c0b3e30 con 0x7f512c0aecc0 2024-09-16T15:44:31.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.582+0000 7f512b400640 1 --2- 172.21.15.151:0/4292365471 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f512c0caa50 0x7f512c1f4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:44:31.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.583+0000 7f512be00640 1 --2- 172.21.15.151:0/4292365471 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f512c0aecc0 0x7f512c1f4150 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f511400e9f0 tx=0x7f511400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:31.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.583+0000 7f5123e00640 1 -- 172.21.15.151:0/4292365471 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511400cd80 con 0x7f512c0aecc0 2024-09-16T15:44:31.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.583+0000 7f5123e00640 1 -- 172.21.15.151:0/4292365471 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5114004020 con 0x7f512c0aecc0 2024-09-16T15:44:31.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.583+0000 7f5123e00640 1 -- 172.21.15.151:0/4292365471 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5114004470 con 0x7f512c0aecc0 2024-09-16T15:44:31.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.583+0000 7f5131600640 1 -- 172.21.15.151:0/4292365471 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f512c1f1080 con 0x7f512c0aecc0 2024-09-16T15:44:31.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.584+0000 7f5131600640 1 -- 172.21.15.151:0/4292365471 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f512c1f1640 con 0x7f512c0aecc0 2024-09-16T15:44:31.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.585+0000 7f5131600640 1 -- 172.21.15.151:0/4292365471 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f512c0ce370 con 0x7f512c0aecc0 2024-09-16T15:44:31.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.587+0000 7f5123e00640 1 -- 172.21.15.151:0/4292365471 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 102103+0+0 (secure 0 0 0) 0x7f51140035f0 con 0x7f512c0aecc0 2024-09-16T15:44:31.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.588+0000 7f5123e00640 1 --2- 172.21.15.151:0/4292365471 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5108079150 0x7f510807b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:31.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.588+0000 7f5123e00640 1 -- 172.21.15.151:0/4292365471 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5854+0+0 (secure 0 0 0) 0x7f5114099b00 con 0x7f512c0aecc0 2024-09-16T15:44:31.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.589+0000 7f512b400640 1 --2- 172.21.15.151:0/4292365471 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5108079150 0x7f510807b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:31.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.592+0000 7f512b400640 1 --2- 172.21.15.151:0/4292365471 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5108079150 0x7f510807b610 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f511c009900 tx=0x7f511c004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:31.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.598+0000 7f5123e00640 1 -- 172.21.15.151:0/4292365471 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f5114063ba0 con 0x7f512c0aecc0 2024-09-16T15:44:31.957 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.956+0000 7f5131600640 1 -- 172.21.15.151:0/4292365471 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f512c1f5080 con 0x7f512c0aecc0 2024-09-16T15:44:31.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.959+0000 7f5123e00640 1 -- 172.21.15.151:0/4292365471 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) ==== 74+0+17072 (secure 0 0 0) 0x7f51140634a0 con 0x7f512c0aecc0 2024-09-16T15:44:31.960 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:44:31.961 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":51,"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","created":"2024-09-16T15:38:56.348535+0000","modified":"2024-09-16T15:44:31.589945+0000","last_up_change":"2024-09-16T15:39:10.681384+0000","last_in_change":"2024-09-16T15:39:00.544477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:39:11.602252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:42:22.166447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-16T15:42:23.379899+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":51,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5524076-a903-4d0a-af6a-caf474c4765d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6809","nonce":3111342741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6811","nonce":3111342741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6815","nonce":3111342741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6813","nonce":3111342741}]},"public_addr":"172.21.15.151:6809/3111342741","cluster_addr":"172.21.15.151:6811/3111342741","heartbeat_back_addr":"172.21.15.151:6815/3111342741","heartbeat_front_addr":"172.21.15.151:6813/3111342741","state":["exists","up"]},{"osd":1,"uuid":"dd25925f-0ae1-494e-8529-4dc205759db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6801","nonce":3368304087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6803","nonce":3368304087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6807","nonce":3368304087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6805","nonce":3368304087}]},"public_addr":"172.21.15.151:6801/3368304087","cluster_addr":"172.21.15.151:6803/3368304087","heartbeat_back_addr":"172.21.15.151:6807/3368304087","heartbeat_front_addr":"172.21.15.151:6805/3368304087","state":["exists","up"]},{"osd":2,"uuid":"c2f18115-013d-4da6-a0fe-cb978f4494b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6823","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6825","nonce":3079373271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6827","nonce":3079373271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6831","nonce":3079373271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6829","nonce":3079373271}]},"public_addr":"172.21.15.151:6825/3079373271","cluster_addr":"172.21.15.151:6827/3079373271","heartbeat_back_addr":"172.21.15.151:6831/3079373271","heartbeat_front_addr":"172.21.15.151:6829/3079373271","state":["exists","up"]},{"osd":3,"uuid":"36edb718-2fcb-4d88-8ddc-c321eed4279d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6817","nonce":1142509041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6819","nonce":1142509041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6824","nonce":1142509041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6821","nonce":1142509041}]},"public_addr":"172.21.15.151:6817/1142509041","cluster_addr":"172.21.15.151:6819/1142509041","heartbeat_back_addr":"172.21.15.151:6824/1142509041","heartbeat_front_addr":"172.21.15.151:6821/1142509041","state":["exists","up"]},{"osd":4,"uuid":"64d33708-cd0a-4236-abe3-1b646e82ab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6811","nonce":2735495271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6813","nonce":2735495271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6817","nonce":2735495271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6815","nonce":2735495271}]},"public_addr":"172.21.15.167:6811/2735495271","cluster_addr":"172.21.15.167:6813/2735495271","heartbeat_back_addr":"172.21.15.167:6817/2735495271","heartbeat_front_addr":"172.21.15.167:6815/2735495271","state":["exists","up"]},{"osd":5,"uuid":"f14b44a9-5910-458c-b518-09a8d0de94de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6827","nonce":1494916572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6829","nonce":1494916572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6833","nonce":1494916572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6831","nonce":1494916572}]},"public_addr":"172.21.15.167:6827/1494916572","cluster_addr":"172.21.15.167:6829/1494916572","heartbeat_back_addr":"172.21.15.167:6833/1494916572","heartbeat_front_addr":"172.21.15.167:6831/1494916572","state":["exists","up"]},{"osd":6,"uuid":"39335545-2654-4710-84b0-a9aa47cfcbba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6819","nonce":2538273273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6821","nonce":2538273273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6825","nonce":2538273273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6823","nonce":2538273273}]},"public_addr":"172.21.15.167:6819/2538273273","cluster_addr":"172.21.15.167:6821/2538273273","heartbeat_back_addr":"172.21.15.167:6825/2538273273","heartbeat_front_addr":"172.21.15.167:6823/2538273273","state":["exists","up"]},{"osd":7,"uuid":"4f8bfc81-53dd-4339-ac22-1a4007d9f963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6803","nonce":265072227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6805","nonce":265072227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6809","nonce":265072227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6807","nonce":265072227}]},"public_addr":"172.21.15.167:6803/265072227","cluster_addr":"172.21.15.167:6805/265072227","heartbeat_back_addr":"172.21.15.167:6809/265072227","heartbeat_front_addr":"172.21.15.167:6807/265072227","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.150359+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.084358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.722368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.332809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.320709+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.395153+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.935250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.801443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.3","mappings":[{"from":5,"to":7}]},{"pgid":"7.9","mappings":[{"from":5,"to":7}]},{"pgid":"7.12","mappings":[{"from":5,"to":7}]},{"pgid":"7.1d","mappings":[{"from":5,"to":0}]},{"pgid":"7.38","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:44:31.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.963+0000 7f5131600640 1 -- 172.21.15.151:0/4292365471 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5108079150 msgr2=0x7f510807b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:31.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.963+0000 7f5131600640 1 --2- 172.21.15.151:0/4292365471 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5108079150 0x7f510807b610 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f511c009900 tx=0x7f511c004730 comp rx=0 tx=0).stop 2024-09-16T15:44:31.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.963+0000 7f5131600640 1 -- 172.21.15.151:0/4292365471 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f512c0aecc0 msgr2=0x7f512c1f4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:31.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.963+0000 7f5131600640 1 --2- 172.21.15.151:0/4292365471 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f512c0aecc0 0x7f512c1f4150 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f511400e9f0 tx=0x7f511400eeb0 comp rx=0 tx=0).stop 2024-09-16T15:44:31.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.963+0000 7f5131600640 1 -- 172.21.15.151:0/4292365471 shutdown_connections 2024-09-16T15:44:31.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.963+0000 7f5131600640 1 --2- 172.21.15.151:0/4292365471 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5108079150 0x7f510807b610 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:31.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.963+0000 7f5131600640 1 --2- 172.21.15.151:0/4292365471 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f512c0cb370 0x7f512c0c5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:31.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.963+0000 7f5131600640 1 --2- 172.21.15.151:0/4292365471 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f512c0caa50 0x7f512c1f4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:31.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.963+0000 7f5131600640 1 --2- 172.21.15.151:0/4292365471 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f512c0aecc0 0x7f512c1f4150 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:31.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.963+0000 7f5131600640 1 -- 172.21.15.151:0/4292365471 >> 172.21.15.151:0/4292365471 conn(0x7f512c0cfa20 msgr2=0x7f512c0cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:31.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.964+0000 7f5131600640 1 -- 172.21.15.151:0/4292365471 shutdown_connections 2024-09-16T15:44:31.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:31.964+0000 7f5131600640 1 -- 172.21.15.151:0/4292365471 wait complete. 2024-09-16T15:44:31.991 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-16T15:44:31.992 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:44:32.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.151+0000 7ff0a1e00640 1 -- 172.21.15.151:0/270652663 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff09c0725d0 msgr2=0x7ff09c0b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:32.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.151+0000 7ff0a1e00640 1 --2- 172.21.15.151:0/270652663 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff09c0725d0 0x7ff09c0b28f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7ff09c074a80 tx=0x7ff08802a530 comp rx=0 tx=0).stop 2024-09-16T15:44:32.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.151+0000 7ff0a1e00640 1 -- 172.21.15.151:0/270652663 shutdown_connections 2024-09-16T15:44:32.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.151+0000 7ff0a1e00640 1 --2- 172.21.15.151:0/270652663 >> v2:172.21.15.167:3300/0 conn(0x7ff09c0d0540 0x7ff09c06f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:32.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.151+0000 7ff0a1e00640 1 --2- 172.21.15.151:0/270652663 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff09c0725d0 0x7ff09c0b28f0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:32.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.151+0000 7ff0a1e00640 1 -- 172.21.15.151:0/270652663 >> 172.21.15.151:0/270652663 conn(0x7ff09c0cfa20 msgr2=0x7ff09c0cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:32.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.151+0000 7ff0a1e00640 1 -- 172.21.15.151:0/270652663 shutdown_connections 2024-09-16T15:44:32.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.151+0000 7ff0a1e00640 1 -- 172.21.15.151:0/270652663 wait complete. 2024-09-16T15:44:32.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.153+0000 7ff0a1e00640 1 Processor -- start 2024-09-16T15:44:32.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.153+0000 7ff0a1e00640 1 -- start start 2024-09-16T15:44:32.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.154+0000 7ff0a1e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff09c0d0540 0x7ff09c14c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:32.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.154+0000 7ff0a1e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff09c15bc30 0x7ff09c0c7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:32.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.154+0000 7ff0a1e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff09c1f7360 0x7ff09c0acb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:32.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.154+0000 7ff0a1e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff09c074660 con 0x7ff09c1f7360 2024-09-16T15:44:32.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.154+0000 7ff0a1e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff09c06f860 con 0x7ff09c0d0540 2024-09-16T15:44:32.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.154+0000 7ff0a1e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7ff09c14cbb0 con 0x7ff09c15bc30 2024-09-16T15:44:32.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.154+0000 7ff0a0a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff09c0d0540 0x7ff09c14c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:32.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.154+0000 7ff0a1400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff09c1f7360 0x7ff09c0acb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:32.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.154+0000 7ff0a0a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff09c0d0540 0x7ff09c14c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.151:36120/0 (socket says 172.21.15.151:36120) 2024-09-16T15:44:32.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.154+0000 7ff0a0a00640 1 -- 172.21.15.151:0/1779428260 learned_addr learned my addr 172.21.15.151:0/1779428260 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:32.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.154+0000 7ff09be00640 1 --2- 172.21.15.151:0/1779428260 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff09c15bc30 0x7ff09c0c7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:32.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.154+0000 7ff0a0a00640 1 -- 172.21.15.151:0/1779428260 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff09c15bc30 msgr2=0x7ff09c0c7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:32.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.154+0000 7ff0a0a00640 1 --2- 172.21.15.151:0/1779428260 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff09c15bc30 0x7ff09c0c7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:32.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.154+0000 7ff0a0a00640 1 -- 172.21.15.151:0/1779428260 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff09c1f7360 msgr2=0x7ff09c0acb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:32.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.154+0000 7ff0a0a00640 1 --2- 172.21.15.151:0/1779428260 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff09c1f7360 0x7ff09c0acb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:32.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.154+0000 7ff0a0a00640 1 -- 172.21.15.151:0/1779428260 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff09c0c82e0 con 0x7ff09c0d0540 2024-09-16T15:44:32.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.154+0000 7ff0a1400640 1 --2- 172.21.15.151:0/1779428260 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff09c1f7360 0x7ff09c0acb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:44:32.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.155+0000 7ff0a0a00640 1 --2- 172.21.15.151:0/1779428260 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff09c0d0540 0x7ff09c14c670 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff09000c430 tx=0x7ff09000c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:32.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.155+0000 7ff098c00640 1 -- 172.21.15.151:0/1779428260 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff090011790 con 0x7ff09c0d0540 2024-09-16T15:44:32.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.155+0000 7ff098c00640 1 -- 172.21.15.151:0/1779428260 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff090017070 con 0x7ff09c0d0540 2024-09-16T15:44:32.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.155+0000 7ff0a1e00640 1 -- 172.21.15.151:0/1779428260 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff09c0c85b0 con 0x7ff09c0d0540 2024-09-16T15:44:32.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.155+0000 7ff098c00640 1 -- 172.21.15.151:0/1779428260 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff090011ce0 con 0x7ff09c0d0540 2024-09-16T15:44:32.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.155+0000 7ff0a1e00640 1 -- 172.21.15.151:0/1779428260 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff09c0c8b70 con 0x7ff09c0d0540 2024-09-16T15:44:32.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.156+0000 7ff0a1e00640 1 -- 172.21.15.151:0/1779428260 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff09c06fdf0 con 0x7ff09c0d0540 2024-09-16T15:44:32.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.158+0000 7ff098c00640 1 -- 172.21.15.151:0/1779428260 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 18) ==== 102103+0+0 (secure 0 0 0) 0x7ff09001c460 con 0x7ff09c0d0540 2024-09-16T15:44:32.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.158+0000 7ff098c00640 1 --2- 172.21.15.151:0/1779428260 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff07c079150 0x7ff07c07b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:32.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.159+0000 7ff098c00640 1 -- 172.21.15.151:0/1779428260 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5910+0+0 (secure 0 0 0) 0x7ff090098f00 con 0x7ff09c0d0540 2024-09-16T15:44:32.160 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.159+0000 7ff09be00640 1 --2- 172.21.15.151:0/1779428260 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff07c079150 0x7ff07c07b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:32.160 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.160+0000 7ff09be00640 1 --2- 172.21.15.151:0/1779428260 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff07c079150 0x7ff07c07b610 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff088034000 tx=0x7ff088004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:32.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.163+0000 7ff098c00640 1 -- 172.21.15.151:0/1779428260 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff0900628f0 con 0x7ff09c0d0540 2024-09-16T15:44:32.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.383+0000 7ff0a1e00640 1 -- 172.21.15.151:0/1779428260 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff09c1534f0 con 0x7ff09c0d0540 2024-09-16T15:44:32.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.384+0000 7ff098c00640 1 -- 172.21.15.151:0/1779428260 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+5232 (secure 0 0 0) 0x7ff090067320 con 0x7ff09c0d0540 2024-09-16T15:44:32.385 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:44:32.385 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":28,"btime":"2024-09-16T15:42:38:048451+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5272,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1043845735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1043845735},{"type":"v1","addr":"172.21.15.167:6836","nonce":1043845735}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5275,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/816612106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":816612106},{"type":"v1","addr":"172.21.15.167:6837","nonce":816612106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5397,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/1986755100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1986755100},{"type":"v1","addr":"172.21.15.151:6834","nonce":1986755100}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:42:24.411561+0000","modified":"2024-09-16T15:42:27.425402+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5400":{"gid":5400,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.151:6835/1974411916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":1974411916},{"type":"v1","addr":"172.21.15.151:6835","nonce":1974411916}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5400,"qdb_cluster":[5400]},"id":3}]} 2024-09-16T15:44:32.385 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 28 2024-09-16T15:44:32.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.386+0000 7ff0a1e00640 1 -- 172.21.15.151:0/1779428260 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff07c079150 msgr2=0x7ff07c07b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:32.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.386+0000 7ff0a1e00640 1 --2- 172.21.15.151:0/1779428260 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff07c079150 0x7ff07c07b610 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff088034000 tx=0x7ff088004730 comp rx=0 tx=0).stop 2024-09-16T15:44:32.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.386+0000 7ff0a1e00640 1 -- 172.21.15.151:0/1779428260 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff09c0d0540 msgr2=0x7ff09c14c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:32.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.386+0000 7ff0a1e00640 1 --2- 172.21.15.151:0/1779428260 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff09c0d0540 0x7ff09c14c670 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff09000c430 tx=0x7ff09000c8f0 comp rx=0 tx=0).stop 2024-09-16T15:44:32.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.386+0000 7ff0a1e00640 1 -- 172.21.15.151:0/1779428260 shutdown_connections 2024-09-16T15:44:32.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.386+0000 7ff0a1e00640 1 --2- 172.21.15.151:0/1779428260 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff07c079150 0x7ff07c07b610 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:32.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.386+0000 7ff0a1e00640 1 --2- 172.21.15.151:0/1779428260 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff09c1f7360 0x7ff09c0acb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:32.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.386+0000 7ff0a1e00640 1 --2- 172.21.15.151:0/1779428260 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff09c15bc30 0x7ff09c0c7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:32.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.386+0000 7ff0a1e00640 1 --2- 172.21.15.151:0/1779428260 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff09c0d0540 0x7ff09c14c670 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:32.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.386+0000 7ff0a1e00640 1 -- 172.21.15.151:0/1779428260 >> 172.21.15.151:0/1779428260 conn(0x7ff09c0cfa20 msgr2=0x7ff09c15a230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:32.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.386+0000 7ff0a1e00640 1 -- 172.21.15.151:0/1779428260 shutdown_connections 2024-09-16T15:44:32.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.386+0000 7ff0a1e00640 1 -- 172.21.15.151:0/1779428260 wait complete. 2024-09-16T15:44:32.403 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:44:32.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.560+0000 7fe8b4400640 1 -- 172.21.15.151:0/1176606262 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe8ac0ccfd0 msgr2=0x7fe8ac072ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:32.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.560+0000 7fe8b4400640 1 --2- 172.21.15.151:0/1176606262 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe8ac0ccfd0 0x7fe8ac072ee0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4009860 tx=0x7fe8a402a5d0 comp rx=0 tx=0).stop 2024-09-16T15:44:32.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.560+0000 7fe8b4400640 1 -- 172.21.15.151:0/1176606262 shutdown_connections 2024-09-16T15:44:32.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.560+0000 7fe8b4400640 1 --2- 172.21.15.151:0/1176606262 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe8ac0ccfd0 0x7fe8ac072ee0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:32.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.560+0000 7fe8b4400640 1 --2- 172.21.15.151:0/1176606262 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8ac0cf300 0x7fe8ac0729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:32.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.560+0000 7fe8b4400640 1 -- 172.21.15.151:0/1176606262 >> 172.21.15.151:0/1176606262 conn(0x7fe8ac0b1950 msgr2=0x7fe8ac0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:32.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.560+0000 7fe8b4400640 1 -- 172.21.15.151:0/1176606262 shutdown_connections 2024-09-16T15:44:32.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.560+0000 7fe8b4400640 1 -- 172.21.15.151:0/1176606262 wait complete. 2024-09-16T15:44:32.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.562+0000 7fe8b4400640 1 Processor -- start 2024-09-16T15:44:32.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.562+0000 7fe8b4400640 1 -- start start 2024-09-16T15:44:32.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.563+0000 7fe8b4400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe8ac0ccfd0 0x7fe8ac06f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:32.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.563+0000 7fe8b4400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8ac0cf300 0x7fe8ac1f85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:32.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.563+0000 7fe8b4400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8ac1445d0 0x7fe8ac0cb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:32.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.563+0000 7fe8b4400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe8ac073420 con 0x7fe8ac0ccfd0 2024-09-16T15:44:32.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.563+0000 7fe8b4400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe8ac0b4eb0 con 0x7fe8ac0cf300 2024-09-16T15:44:32.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.563+0000 7fe8b4400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fe8ac0735a0 con 0x7fe8ac1445d0 2024-09-16T15:44:32.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.563+0000 7fe8b1a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe8ac0ccfd0 0x7fe8ac06f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:32.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.563+0000 7fe8b1a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe8ac0ccfd0 0x7fe8ac06f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:32964/0 (socket says 172.21.15.151:32964) 2024-09-16T15:44:32.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.563+0000 7fe8b1a00640 1 -- 172.21.15.151:0/3655100499 learned_addr learned my addr 172.21.15.151:0/3655100499 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:32.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.564+0000 7fe8b2400640 1 --2- 172.21.15.151:0/3655100499 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8ac1445d0 0x7fe8ac0cb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:32.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.564+0000 7fe8b1000640 1 --2- 172.21.15.151:0/3655100499 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8ac0cf300 0x7fe8ac1f85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:32.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.564+0000 7fe8b1a00640 1 -- 172.21.15.151:0/3655100499 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8ac1445d0 msgr2=0x7fe8ac0cb8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:32.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.564+0000 7fe8b1a00640 1 --2- 172.21.15.151:0/3655100499 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8ac1445d0 0x7fe8ac0cb8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:32.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.564+0000 7fe8b1a00640 1 -- 172.21.15.151:0/3655100499 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8ac0cf300 msgr2=0x7fe8ac1f85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:32.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.564+0000 7fe8b1a00640 1 --2- 172.21.15.151:0/3655100499 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8ac0cf300 0x7fe8ac1f85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:32.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.564+0000 7fe8b1a00640 1 -- 172.21.15.151:0/3655100499 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8ac06e890 con 0x7fe8ac0ccfd0 2024-09-16T15:44:32.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.564+0000 7fe8b2400640 1 --2- 172.21.15.151:0/3655100499 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8ac1445d0 0x7fe8ac0cb8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:44:32.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.564+0000 7fe8b1000640 1 --2- 172.21.15.151:0/3655100499 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8ac0cf300 0x7fe8ac1f85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:44:32.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.564+0000 7fe8b1a00640 1 --2- 172.21.15.151:0/3655100499 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe8ac0ccfd0 0x7fe8ac06f930 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fe8a000cbc0 tx=0x7fe8a000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:32.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.564+0000 7fe8a9600640 1 -- 172.21.15.151:0/3655100499 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a0016020 con 0x7fe8ac0ccfd0 2024-09-16T15:44:32.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.564+0000 7fe8a9600640 1 -- 172.21.15.151:0/3655100499 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe8a000ad90 con 0x7fe8ac0ccfd0 2024-09-16T15:44:32.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.564+0000 7fe8a9600640 1 -- 172.21.15.151:0/3655100499 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a0012680 con 0x7fe8ac0ccfd0 2024-09-16T15:44:32.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.564+0000 7fe8b4400640 1 -- 172.21.15.151:0/3655100499 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8ac06eb70 con 0x7fe8ac0ccfd0 2024-09-16T15:44:32.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.564+0000 7fe8b4400640 1 -- 172.21.15.151:0/3655100499 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8ac06f130 con 0x7fe8ac0ccfd0 2024-09-16T15:44:32.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.566+0000 7fe8b4400640 1 -- 172.21.15.151:0/3655100499 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8ac073a30 con 0x7fe8ac0ccfd0 2024-09-16T15:44:32.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.567+0000 7fe8a9600640 1 -- 172.21.15.151:0/3655100499 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 102103+0+0 (secure 0 0 0) 0x7fe8a0019020 con 0x7fe8ac0ccfd0 2024-09-16T15:44:32.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.567+0000 7fe8a9600640 1 --2- 172.21.15.151:0/3655100499 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe88c079150 0x7fe88c07b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:32.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.568+0000 7fe8a9600640 1 -- 172.21.15.151:0/3655100499 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5910+0+0 (secure 0 0 0) 0x7fe8a0099900 con 0x7fe8ac0ccfd0 2024-09-16T15:44:32.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.568+0000 7fe8b1000640 1 --2- 172.21.15.151:0/3655100499 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe88c079150 0x7fe88c07b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:32.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.568+0000 7fe8b1000640 1 --2- 172.21.15.151:0/3655100499 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe88c079150 0x7fe88c07b610 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac0b2670 tx=0x7fe898005d90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:32.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.572+0000 7fe8a9600640 1 -- 172.21.15.151:0/3655100499 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe8a0020020 con 0x7fe8ac0ccfd0 2024-09-16T15:44:32.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.795+0000 7fe8b4400640 1 -- 172.21.15.151:0/3655100499 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe8ac070a20 con 0x7fe8ac0ccfd0 2024-09-16T15:44:32.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.796+0000 7fe8a9600640 1 -- 172.21.15.151:0/3655100499 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+5232 (secure 0 0 0) 0x7fe8a0063940 con 0x7fe8ac0ccfd0 2024-09-16T15:44:32.797 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:44:32.797 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":28,"btime":"2024-09-16T15:42:38:048451+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5272,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1043845735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1043845735},{"type":"v1","addr":"172.21.15.167:6836","nonce":1043845735}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5275,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/816612106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":816612106},{"type":"v1","addr":"172.21.15.167:6837","nonce":816612106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5397,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/1986755100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1986755100},{"type":"v1","addr":"172.21.15.151:6834","nonce":1986755100}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:42:24.411561+0000","modified":"2024-09-16T15:42:27.425402+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5400},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5400":{"gid":5400,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.151:6835/1974411916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":1974411916},{"type":"v1","addr":"172.21.15.151:6835","nonce":1974411916}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5400,"qdb_cluster":[5400]},"id":3}]} 2024-09-16T15:44:32.797 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 28 2024-09-16T15:44:32.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.798+0000 7fe8b4400640 1 -- 172.21.15.151:0/3655100499 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe88c079150 msgr2=0x7fe88c07b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:32.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.798+0000 7fe8b4400640 1 --2- 172.21.15.151:0/3655100499 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe88c079150 0x7fe88c07b610 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac0b2670 tx=0x7fe898005d90 comp rx=0 tx=0).stop 2024-09-16T15:44:32.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.798+0000 7fe8b4400640 1 -- 172.21.15.151:0/3655100499 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe8ac0ccfd0 msgr2=0x7fe8ac06f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:32.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.798+0000 7fe8b4400640 1 --2- 172.21.15.151:0/3655100499 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe8ac0ccfd0 0x7fe8ac06f930 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fe8a000cbc0 tx=0x7fe8a000a060 comp rx=0 tx=0).stop 2024-09-16T15:44:32.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.798+0000 7fe8b4400640 1 -- 172.21.15.151:0/3655100499 shutdown_connections 2024-09-16T15:44:32.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.798+0000 7fe8b4400640 1 --2- 172.21.15.151:0/3655100499 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe88c079150 0x7fe88c07b610 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:32.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.798+0000 7fe8b4400640 1 --2- 172.21.15.151:0/3655100499 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8ac0cf300 0x7fe8ac1f85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:32.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.798+0000 7fe8b4400640 1 --2- 172.21.15.151:0/3655100499 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8ac1445d0 0x7fe8ac0cb8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:32.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.798+0000 7fe8b4400640 1 --2- 172.21.15.151:0/3655100499 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe8ac0ccfd0 0x7fe8ac06f930 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:32.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.798+0000 7fe8b4400640 1 -- 172.21.15.151:0/3655100499 >> 172.21.15.151:0/3655100499 conn(0x7fe8ac0b1950 msgr2=0x7fe8ac0ae020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:32.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.798+0000 7fe8b4400640 1 -- 172.21.15.151:0/3655100499 shutdown_connections 2024-09-16T15:44:32.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.798+0000 7fe8b4400640 1 -- 172.21.15.151:0/3655100499 wait complete. 2024-09-16T15:44:32.816 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:44:32.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.973+0000 7f752da00640 1 -- 172.21.15.151:0/2745523550 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7528073fb0 msgr2=0x7f7528074460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:32.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.973+0000 7f752da00640 1 --2- 172.21.15.151:0/2745523550 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7528073fb0 0x7f7528074460 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f7520005fd0 tx=0x7f752002aa10 comp rx=0 tx=0).stop 2024-09-16T15:44:32.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.973+0000 7f752da00640 1 -- 172.21.15.151:0/2745523550 shutdown_connections 2024-09-16T15:44:32.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.973+0000 7f752da00640 1 --2- 172.21.15.151:0/2745523550 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7528073fb0 0x7f7528074460 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:32.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.973+0000 7f752da00640 1 --2- 172.21.15.151:0/2745523550 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f752815f5a0 0x7f75280b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:32.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.973+0000 7f752da00640 1 -- 172.21.15.151:0/2745523550 >> 172.21.15.151:0/2745523550 conn(0x7f75280b1950 msgr2=0x7f75280b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:32.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.973+0000 7f752da00640 1 -- 172.21.15.151:0/2745523550 shutdown_connections 2024-09-16T15:44:32.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.973+0000 7f752da00640 1 -- 172.21.15.151:0/2745523550 wait complete. 2024-09-16T15:44:32.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.975+0000 7f752da00640 1 Processor -- start 2024-09-16T15:44:32.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.975+0000 7f752da00640 1 -- start start 2024-09-16T15:44:32.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.976+0000 7f752da00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7528073fb0 0x7f75281470b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:32.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.976+0000 7f752da00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75280ccf00 0x7f752815c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:32.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.976+0000 7f752da00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f752815f5a0 0x7f7528159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:32.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.976+0000 7f752da00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f752806f2e0 con 0x7f752815f5a0 2024-09-16T15:44:32.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.976+0000 7f752da00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f752806f160 con 0x7f75280ccf00 2024-09-16T15:44:32.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.976+0000 7f752da00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f752806f460 con 0x7f7528073fb0 2024-09-16T15:44:32.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.976+0000 7f7527400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f752815f5a0 0x7f7528159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:32.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.976+0000 7f7527400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f752815f5a0 0x7f7528159ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33858/0 (socket says 172.21.15.151:33858) 2024-09-16T15:44:32.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.976+0000 7f7527400640 1 -- 172.21.15.151:0/2880262162 learned_addr learned my addr 172.21.15.151:0/2880262162 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:32.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.976+0000 7f7526a00640 1 --2- 172.21.15.151:0/2880262162 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7528073fb0 0x7f75281470b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:32.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.976+0000 7f7526000640 1 --2- 172.21.15.151:0/2880262162 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75280ccf00 0x7f752815c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:32.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.976+0000 7f7527400640 1 -- 172.21.15.151:0/2880262162 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7528073fb0 msgr2=0x7f75281470b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:32.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.976+0000 7f7527400640 1 --2- 172.21.15.151:0/2880262162 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7528073fb0 0x7f75281470b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:32.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.977+0000 7f7527400640 1 -- 172.21.15.151:0/2880262162 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75280ccf00 msgr2=0x7f752815c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:32.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.977+0000 7f7527400640 1 --2- 172.21.15.151:0/2880262162 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75280ccf00 0x7f752815c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:32.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.977+0000 7f7527400640 1 -- 172.21.15.151:0/2880262162 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f752815a6c0 con 0x7f752815f5a0 2024-09-16T15:44:32.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.977+0000 7f7527400640 1 --2- 172.21.15.151:0/2880262162 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f752815f5a0 0x7f7528159ce0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f7520009d90 tx=0x7f7520003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:32.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.977+0000 7f751ea00640 1 -- 172.21.15.151:0/2880262162 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7520047070 con 0x7f752815f5a0 2024-09-16T15:44:32.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.977+0000 7f751ea00640 1 -- 172.21.15.151:0/2880262162 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7520004250 con 0x7f752815f5a0 2024-09-16T15:44:32.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.977+0000 7f752da00640 1 -- 172.21.15.151:0/2880262162 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f752815a9a0 con 0x7f752815f5a0 2024-09-16T15:44:32.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.977+0000 7f751ea00640 1 -- 172.21.15.151:0/2880262162 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7520033050 con 0x7f752815f5a0 2024-09-16T15:44:32.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.977+0000 7f752da00640 1 -- 172.21.15.151:0/2880262162 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f752815af60 con 0x7f752815f5a0 2024-09-16T15:44:32.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.978+0000 7f752da00640 1 -- 172.21.15.151:0/2880262162 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f752806f9f0 con 0x7f752815f5a0 2024-09-16T15:44:32.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.979+0000 7f751ea00640 1 -- 172.21.15.151:0/2880262162 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 102103+0+0 (secure 0 0 0) 0x7f752004a070 con 0x7f752815f5a0 2024-09-16T15:44:32.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.980+0000 7f751ea00640 1 --2- 172.21.15.151:0/2880262162 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7508079150 0x7f750807b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:32.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.980+0000 7f751ea00640 1 -- 172.21.15.151:0/2880262162 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5910+0+0 (secure 0 0 0) 0x7f75200ba270 con 0x7f752815f5a0 2024-09-16T15:44:32.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.981+0000 7f7526a00640 1 --2- 172.21.15.151:0/2880262162 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7508079150 0x7f750807b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:32.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.982+0000 7f7526a00640 1 --2- 172.21.15.151:0/2880262162 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7508079150 0x7f750807b610 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f751800bb50 tx=0x7f751800b2e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:32.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:32.985+0000 7f751ea00640 1 -- 172.21.15.151:0/2880262162 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7520083ce0 con 0x7f752815f5a0 2024-09-16T15:44:33.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.176+0000 7f752da00640 1 -- 172.21.15.151:0/2880262162 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f75280b13d0 con 0x7f752815f5a0 2024-09-16T15:44:33.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.178+0000 7f751ea00640 1 -- 172.21.15.151:0/2880262162 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) ==== 74+0+17072 (secure 0 0 0) 0x7f7520088710 con 0x7f752815f5a0 2024-09-16T15:44:33.179 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:44:33.179 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":51,"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","created":"2024-09-16T15:38:56.348535+0000","modified":"2024-09-16T15:44:31.589945+0000","last_up_change":"2024-09-16T15:39:10.681384+0000","last_in_change":"2024-09-16T15:39:00.544477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:39:11.602252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:42:22.166447+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-16T15:42:23.379899+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":51,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5524076-a903-4d0a-af6a-caf474c4765d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6809","nonce":3111342741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6811","nonce":3111342741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6815","nonce":3111342741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6813","nonce":3111342741}]},"public_addr":"172.21.15.151:6809/3111342741","cluster_addr":"172.21.15.151:6811/3111342741","heartbeat_back_addr":"172.21.15.151:6815/3111342741","heartbeat_front_addr":"172.21.15.151:6813/3111342741","state":["exists","up"]},{"osd":1,"uuid":"dd25925f-0ae1-494e-8529-4dc205759db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6801","nonce":3368304087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6803","nonce":3368304087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6807","nonce":3368304087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6805","nonce":3368304087}]},"public_addr":"172.21.15.151:6801/3368304087","cluster_addr":"172.21.15.151:6803/3368304087","heartbeat_back_addr":"172.21.15.151:6807/3368304087","heartbeat_front_addr":"172.21.15.151:6805/3368304087","state":["exists","up"]},{"osd":2,"uuid":"c2f18115-013d-4da6-a0fe-cb978f4494b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6823","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6825","nonce":3079373271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6827","nonce":3079373271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6831","nonce":3079373271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6829","nonce":3079373271}]},"public_addr":"172.21.15.151:6825/3079373271","cluster_addr":"172.21.15.151:6827/3079373271","heartbeat_back_addr":"172.21.15.151:6831/3079373271","heartbeat_front_addr":"172.21.15.151:6829/3079373271","state":["exists","up"]},{"osd":3,"uuid":"36edb718-2fcb-4d88-8ddc-c321eed4279d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6817","nonce":1142509041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6819","nonce":1142509041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6824","nonce":1142509041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6821","nonce":1142509041}]},"public_addr":"172.21.15.151:6817/1142509041","cluster_addr":"172.21.15.151:6819/1142509041","heartbeat_back_addr":"172.21.15.151:6824/1142509041","heartbeat_front_addr":"172.21.15.151:6821/1142509041","state":["exists","up"]},{"osd":4,"uuid":"64d33708-cd0a-4236-abe3-1b646e82ab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6811","nonce":2735495271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6813","nonce":2735495271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6817","nonce":2735495271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6815","nonce":2735495271}]},"public_addr":"172.21.15.167:6811/2735495271","cluster_addr":"172.21.15.167:6813/2735495271","heartbeat_back_addr":"172.21.15.167:6817/2735495271","heartbeat_front_addr":"172.21.15.167:6815/2735495271","state":["exists","up"]},{"osd":5,"uuid":"f14b44a9-5910-458c-b518-09a8d0de94de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6827","nonce":1494916572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6829","nonce":1494916572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6833","nonce":1494916572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6831","nonce":1494916572}]},"public_addr":"172.21.15.167:6827/1494916572","cluster_addr":"172.21.15.167:6829/1494916572","heartbeat_back_addr":"172.21.15.167:6833/1494916572","heartbeat_front_addr":"172.21.15.167:6831/1494916572","state":["exists","up"]},{"osd":6,"uuid":"39335545-2654-4710-84b0-a9aa47cfcbba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6819","nonce":2538273273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6821","nonce":2538273273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6825","nonce":2538273273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6823","nonce":2538273273}]},"public_addr":"172.21.15.167:6819/2538273273","cluster_addr":"172.21.15.167:6821/2538273273","heartbeat_back_addr":"172.21.15.167:6825/2538273273","heartbeat_front_addr":"172.21.15.167:6823/2538273273","state":["exists","up"]},{"osd":7,"uuid":"4f8bfc81-53dd-4339-ac22-1a4007d9f963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6803","nonce":265072227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6805","nonce":265072227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6809","nonce":265072227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6807","nonce":265072227}]},"public_addr":"172.21.15.167:6803/265072227","cluster_addr":"172.21.15.167:6805/265072227","heartbeat_back_addr":"172.21.15.167:6809/265072227","heartbeat_front_addr":"172.21.15.167:6807/265072227","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.150359+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.084358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.722368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.332809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.320709+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.395153+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.935250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.801443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.3","mappings":[{"from":5,"to":7}]},{"pgid":"7.9","mappings":[{"from":5,"to":7}]},{"pgid":"7.12","mappings":[{"from":5,"to":7}]},{"pgid":"7.1d","mappings":[{"from":5,"to":0}]},{"pgid":"7.38","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:44:33.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.180+0000 7f752da00640 1 -- 172.21.15.151:0/2880262162 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7508079150 msgr2=0x7f750807b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:33.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.180+0000 7f752da00640 1 --2- 172.21.15.151:0/2880262162 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7508079150 0x7f750807b610 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f751800bb50 tx=0x7f751800b2e0 comp rx=0 tx=0).stop 2024-09-16T15:44:33.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.180+0000 7f752da00640 1 -- 172.21.15.151:0/2880262162 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f752815f5a0 msgr2=0x7f7528159ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:33.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.180+0000 7f752da00640 1 --2- 172.21.15.151:0/2880262162 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f752815f5a0 0x7f7528159ce0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f7520009d90 tx=0x7f7520003f10 comp rx=0 tx=0).stop 2024-09-16T15:44:33.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.180+0000 7f752da00640 1 -- 172.21.15.151:0/2880262162 shutdown_connections 2024-09-16T15:44:33.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.180+0000 7f752da00640 1 --2- 172.21.15.151:0/2880262162 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7508079150 0x7f750807b610 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:33.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.180+0000 7f752da00640 1 --2- 172.21.15.151:0/2880262162 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75280ccf00 0x7f752815c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:33.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.180+0000 7f752da00640 1 --2- 172.21.15.151:0/2880262162 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f752815f5a0 0x7f7528159ce0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:33.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.180+0000 7f752da00640 1 --2- 172.21.15.151:0/2880262162 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7528073fb0 0x7f75281470b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:33.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.180+0000 7f752da00640 1 -- 172.21.15.151:0/2880262162 >> 172.21.15.151:0/2880262162 conn(0x7f75280b1950 msgr2=0x7f75280acd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:33.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.180+0000 7f752da00640 1 -- 172.21.15.151:0/2880262162 shutdown_connections 2024-09-16T15:44:33.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.180+0000 7f752da00640 1 -- 172.21.15.151:0/2880262162 wait complete. 2024-09-16T15:44:33.197 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-16T15:44:33.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.353+0000 7f7cad000640 1 -- 172.21.15.151:0/3069719367 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7ca80ccf00 msgr2=0x7f7ca80b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:33.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.353+0000 7f7cad000640 1 --2- 172.21.15.151:0/3069719367 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7ca80ccf00 0x7f7ca80b2440 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7c94009830 tx=0x7f7c9402a480 comp rx=0 tx=0).stop 2024-09-16T15:44:33.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.354+0000 7f7cad000640 1 -- 172.21.15.151:0/3069719367 shutdown_connections 2024-09-16T15:44:33.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.354+0000 7f7cad000640 1 --2- 172.21.15.151:0/3069719367 >> v2:172.21.15.167:3300/0 conn(0x7f7ca8073fb0 0x7f7ca8074440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:33.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.354+0000 7f7cad000640 1 --2- 172.21.15.151:0/3069719367 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7ca80ccf00 0x7f7ca80b2440 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:33.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.354+0000 7f7cad000640 1 -- 172.21.15.151:0/3069719367 >> 172.21.15.151:0/3069719367 conn(0x7f7ca80b1950 msgr2=0x7f7ca80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:33.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.354+0000 7f7cad000640 1 -- 172.21.15.151:0/3069719367 shutdown_connections 2024-09-16T15:44:33.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.354+0000 7f7cad000640 1 -- 172.21.15.151:0/3069719367 wait complete. 2024-09-16T15:44:33.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.355+0000 7f7cad000640 1 Processor -- start 2024-09-16T15:44:33.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.355+0000 7f7cad000640 1 -- start start 2024-09-16T15:44:33.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.356+0000 7f7cad000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7ca8073fb0 0x7f7ca80cc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:33.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.356+0000 7f7cad000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ca80ccf00 0x7f7ca814d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:33.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.356+0000 7f7cad000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ca815f5a0 0x7f7ca815a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:33.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.356+0000 7f7cad000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca806f2e0 con 0x7f7ca815f5a0 2024-09-16T15:44:33.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.356+0000 7f7cad000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca806f160 con 0x7f7ca80ccf00 2024-09-16T15:44:33.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.356+0000 7f7cad000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f7ca806f460 con 0x7f7ca8073fb0 2024-09-16T15:44:33.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.357+0000 7f7ca6800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ca815f5a0 0x7f7ca815a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:33.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.357+0000 7f7ca6800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ca815f5a0 0x7f7ca815a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33874/0 (socket says 172.21.15.151:33874) 2024-09-16T15:44:33.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.357+0000 7f7ca6800640 1 -- 172.21.15.151:0/2803595462 learned_addr learned my addr 172.21.15.151:0/2803595462 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:33.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.357+0000 7f7ca5e00640 1 --2- 172.21.15.151:0/2803595462 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7ca8073fb0 0x7f7ca80cc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:33.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.357+0000 7f7ca5400640 1 --2- 172.21.15.151:0/2803595462 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ca80ccf00 0x7f7ca814d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:33.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.357+0000 7f7ca6800640 1 -- 172.21.15.151:0/2803595462 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7ca8073fb0 msgr2=0x7f7ca80cc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:33.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.357+0000 7f7ca6800640 1 --2- 172.21.15.151:0/2803595462 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7ca8073fb0 0x7f7ca80cc8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:33.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.357+0000 7f7ca6800640 1 -- 172.21.15.151:0/2803595462 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ca80ccf00 msgr2=0x7f7ca814d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:33.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.357+0000 7f7ca6800640 1 --2- 172.21.15.151:0/2803595462 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ca80ccf00 0x7f7ca814d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:33.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.357+0000 7f7ca6800640 1 -- 172.21.15.151:0/2803595462 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ca80cf1e0 con 0x7f7ca815f5a0 2024-09-16T15:44:33.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.357+0000 7f7ca5e00640 1 --2- 172.21.15.151:0/2803595462 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7ca8073fb0 0x7f7ca80cc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:44:33.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.357+0000 7f7ca6800640 1 --2- 172.21.15.151:0/2803595462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ca815f5a0 0x7f7ca815a020 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f7ca0009fd0 tx=0x7f7ca000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:33.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.358+0000 7f7c9e000640 1 -- 172.21.15.151:0/2803595462 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca0004b20 con 0x7f7ca815f5a0 2024-09-16T15:44:33.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.358+0000 7f7c9e000640 1 -- 172.21.15.151:0/2803595462 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ca0004dd0 con 0x7f7ca815f5a0 2024-09-16T15:44:33.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.358+0000 7f7c9e000640 1 -- 172.21.15.151:0/2803595462 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca0003bd0 con 0x7f7ca815f5a0 2024-09-16T15:44:33.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.358+0000 7f7cad000640 1 -- 172.21.15.151:0/2803595462 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ca80cf4c0 con 0x7f7ca815f5a0 2024-09-16T15:44:33.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.358+0000 7f7cad000640 1 -- 172.21.15.151:0/2803595462 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ca80b3d80 con 0x7f7ca815f5a0 2024-09-16T15:44:33.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.359+0000 7f7cad000640 1 -- 172.21.15.151:0/2803595462 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ca806f9f0 con 0x7f7ca815f5a0 2024-09-16T15:44:33.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.360+0000 7f7c9e000640 1 -- 172.21.15.151:0/2803595462 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 102103+0+0 (secure 0 0 0) 0x7f7ca001d020 con 0x7f7ca815f5a0 2024-09-16T15:44:33.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.361+0000 7f7c9e000640 1 --2- 172.21.15.151:0/2803595462 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7c88079150 0x7f7c8807b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:33.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.361+0000 7f7c9e000640 1 -- 172.21.15.151:0/2803595462 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5910+0+0 (secure 0 0 0) 0x7f7ca00999c0 con 0x7f7ca815f5a0 2024-09-16T15:44:33.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.361+0000 7f7ca5e00640 1 --2- 172.21.15.151:0/2803595462 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7c88079150 0x7f7c8807b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:33.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.362+0000 7f7ca5e00640 1 --2- 172.21.15.151:0/2803595462 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7c88079150 0x7f7c8807b610 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7ca80b3b30 tx=0x7f7c98009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:33.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.365+0000 7f7c9e000640 1 -- 172.21.15.151:0/2803595462 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7ca0063a80 con 0x7f7ca815f5a0 2024-09-16T15:44:33.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.586+0000 7f7cad000640 1 -- 172.21.15.151:0/2803595462 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f7ca81551e0 con 0x7f7ca815f5a0 2024-09-16T15:44:33.587 INFO:tasks.ceph.mon.a.smithi151.stderr:2024-09-16T15:44:33.586+0000 7f62c3200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T15:44:33.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.589+0000 7f7c9e000640 1 -- 172.21.15.151:0/2803595462 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v29) ==== 161+0+0 (secure 0 0 0) 0x7f7ca0063380 con 0x7f7ca815f5a0 2024-09-16T15:44:33.590 INFO:teuthology.orchestra.run.smithi151.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-16T15:44:33.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.592+0000 7f7cad000640 1 -- 172.21.15.151:0/2803595462 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7c88079150 msgr2=0x7f7c8807b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:33.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.592+0000 7f7cad000640 1 --2- 172.21.15.151:0/2803595462 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7c88079150 0x7f7c8807b610 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7ca80b3b30 tx=0x7f7c98009040 comp rx=0 tx=0).stop 2024-09-16T15:44:33.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.592+0000 7f7cad000640 1 -- 172.21.15.151:0/2803595462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ca815f5a0 msgr2=0x7f7ca815a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:33.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.592+0000 7f7cad000640 1 --2- 172.21.15.151:0/2803595462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ca815f5a0 0x7f7ca815a020 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f7ca0009fd0 tx=0x7f7ca000c800 comp rx=0 tx=0).stop 2024-09-16T15:44:33.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.593+0000 7f7cad000640 1 -- 172.21.15.151:0/2803595462 shutdown_connections 2024-09-16T15:44:33.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.593+0000 7f7cad000640 1 --2- 172.21.15.151:0/2803595462 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7c88079150 0x7f7c8807b610 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:33.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.593+0000 7f7cad000640 1 --2- 172.21.15.151:0/2803595462 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ca80ccf00 0x7f7ca814d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:33.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.593+0000 7f7cad000640 1 --2- 172.21.15.151:0/2803595462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ca815f5a0 0x7f7ca815a020 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:33.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.593+0000 7f7cad000640 1 --2- 172.21.15.151:0/2803595462 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7ca8073fb0 0x7f7ca80cc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:33.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.593+0000 7f7cad000640 1 -- 172.21.15.151:0/2803595462 >> 172.21.15.151:0/2803595462 conn(0x7f7ca80b1950 msgr2=0x7f7ca815e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:33.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.593+0000 7f7cad000640 1 -- 172.21.15.151:0/2803595462 shutdown_connections 2024-09-16T15:44:33.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.593+0000 7f7cad000640 1 -- 172.21.15.151:0/2803595462 wait complete. 2024-09-16T15:44:33.612 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-16T15:44:33.661 INFO:tasks.ceph.mds.c.smithi151.stdout:starting mds.c at 2024-09-16T15:44:33.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.773+0000 7f88f4c00640 1 -- 172.21.15.151:0/2276880506 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f88f00cccd0 msgr2=0x7f88f0073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:33.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.773+0000 7f88f4c00640 1 --2- 172.21.15.151:0/2276880506 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f88f00cccd0 0x7f88f0073440 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f88d8005fd0 tx=0x7f88d802a5f0 comp rx=0 tx=0).stop 2024-09-16T15:44:33.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.773+0000 7f88f4c00640 1 -- 172.21.15.151:0/2276880506 shutdown_connections 2024-09-16T15:44:33.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.773+0000 7f88f4c00640 1 --2- 172.21.15.151:0/2276880506 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f88f00cccd0 0x7f88f0073440 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:33.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.773+0000 7f88f4c00640 1 -- 172.21.15.151:0/2276880506 >> 172.21.15.151:0/2276880506 conn(0x7f88f00b1840 msgr2=0x7f88f00b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:33.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.773+0000 7f88f4c00640 1 -- 172.21.15.151:0/2276880506 shutdown_connections 2024-09-16T15:44:33.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.773+0000 7f88f4c00640 1 -- 172.21.15.151:0/2276880506 wait complete. 2024-09-16T15:44:33.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.775+0000 7f88f4c00640 1 Processor -- start 2024-09-16T15:44:33.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.775+0000 7f88f4c00640 1 -- start start 2024-09-16T15:44:33.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.776+0000 7f88f4c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f88f006efa0 0x7f88f01f8880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:33.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.776+0000 7f88f4c00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f88f00cccd0 0x7f88f0137240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:33.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.776+0000 7f88f4c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88f0145750 0x7f88f01431a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:33.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.776+0000 7f88f4c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f88f00ce6a0 con 0x7f88f006efa0 2024-09-16T15:44:33.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.776+0000 7f88f4c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f88f00ce520 con 0x7f88f0145750 2024-09-16T15:44:33.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.776+0000 7f88f4c00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f88f00ce820 con 0x7f88f00cccd0 2024-09-16T15:44:33.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.776+0000 7f88ede00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f88f006efa0 0x7f88f01f8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:33.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.776+0000 7f88ede00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f88f006efa0 0x7f88f01f8880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33906/0 (socket says 172.21.15.151:33906) 2024-09-16T15:44:33.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.776+0000 7f88ede00640 1 -- 172.21.15.151:0/3223244699 learned_addr learned my addr 172.21.15.151:0/3223244699 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:33.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.776+0000 7f88e5400640 1 --2- 172.21.15.151:0/3223244699 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f88f00cccd0 0x7f88f0137240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:33.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.776+0000 7f88ee800640 1 --2- 172.21.15.151:0/3223244699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88f0145750 0x7f88f01431a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:33.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.777+0000 7f88ede00640 1 -- 172.21.15.151:0/3223244699 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f88f00cccd0 msgr2=0x7f88f0137240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:33.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.777+0000 7f88ede00640 1 --2- 172.21.15.151:0/3223244699 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f88f00cccd0 0x7f88f0137240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:33.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.777+0000 7f88ede00640 1 -- 172.21.15.151:0/3223244699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88f0145750 msgr2=0x7f88f01431a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:33.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.777+0000 7f88ede00640 1 --2- 172.21.15.151:0/3223244699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88f0145750 0x7f88f01431a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:33.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.777+0000 7f88ede00640 1 -- 172.21.15.151:0/3223244699 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88f00ca210 con 0x7f88f006efa0 2024-09-16T15:44:33.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.777+0000 7f88ede00640 1 --2- 172.21.15.151:0/3223244699 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f88f006efa0 0x7f88f01f8880 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f88e000c3d0 tx=0x7f88e000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:33.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.777+0000 7f88e6a00640 1 -- 172.21.15.151:0/3223244699 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e0017070 con 0x7f88f006efa0 2024-09-16T15:44:33.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.777+0000 7f88e6a00640 1 -- 172.21.15.151:0/3223244699 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f88e00053c0 con 0x7f88f006efa0 2024-09-16T15:44:33.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.777+0000 7f88e6a00640 1 -- 172.21.15.151:0/3223244699 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e0003850 con 0x7f88f006efa0 2024-09-16T15:44:33.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.777+0000 7f88f4c00640 1 -- 172.21.15.151:0/3223244699 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88f0137970 con 0x7f88f006efa0 2024-09-16T15:44:33.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.777+0000 7f88f4c00640 1 -- 172.21.15.151:0/3223244699 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88f0137f30 con 0x7f88f006efa0 2024-09-16T15:44:33.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.778+0000 7f88f4c00640 1 -- 172.21.15.151:0/3223244699 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88f00cecd0 con 0x7f88f006efa0 2024-09-16T15:44:33.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.780+0000 7f88e6a00640 1 -- 172.21.15.151:0/3223244699 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 102103+0+0 (secure 0 0 0) 0x7f88e001a070 con 0x7f88f006efa0 2024-09-16T15:44:33.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.781+0000 7f88e6a00640 1 --2- 172.21.15.151:0/3223244699 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f88cc079150 0x7f88cc07b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:33.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.781+0000 7f88e6a00640 1 -- 172.21.15.151:0/3223244699 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5968+0+0 (secure 0 0 0) 0x7f88e00984a0 con 0x7f88f006efa0 2024-09-16T15:44:33.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.781+0000 7f88e5400640 1 --2- 172.21.15.151:0/3223244699 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f88cc079150 0x7f88cc07b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:33.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.782+0000 7f88e5400640 1 --2- 172.21.15.151:0/3223244699 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f88cc079150 0x7f88cc07b610 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f88d8033000 tx=0x7f88d8004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:33.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:33.785+0000 7f88e6a00640 1 -- 172.21.15.151:0/3223244699 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f88f00ca210 con 0x7f88f006efa0 2024-09-16T15:44:34.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:34.008+0000 7f88f4c00640 1 -- 172.21.15.151:0/3223244699 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f88f00afac0 con 0x7f88f006efa0 2024-09-16T15:44:34.624 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:34.623+0000 7f88e6a00640 1 -- 172.21.15.151:0/3223244699 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v30) ==== 104+0+0 (secure 0 0 0) 0x7f88e0061f30 con 0x7f88f006efa0 2024-09-16T15:44:34.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:34.626+0000 7f88f4c00640 1 -- 172.21.15.151:0/3223244699 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f88cc079150 msgr2=0x7f88cc07b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:34.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:34.626+0000 7f88f4c00640 1 --2- 172.21.15.151:0/3223244699 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f88cc079150 0x7f88cc07b610 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f88d8033000 tx=0x7f88d8004730 comp rx=0 tx=0).stop 2024-09-16T15:44:34.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:34.626+0000 7f88f4c00640 1 -- 172.21.15.151:0/3223244699 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f88f006efa0 msgr2=0x7f88f01f8880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:34.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:34.626+0000 7f88f4c00640 1 --2- 172.21.15.151:0/3223244699 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f88f006efa0 0x7f88f01f8880 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f88e000c3d0 tx=0x7f88e000c890 comp rx=0 tx=0).stop 2024-09-16T15:44:34.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:34.626+0000 7f88f4c00640 1 -- 172.21.15.151:0/3223244699 shutdown_connections 2024-09-16T15:44:34.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:34.626+0000 7f88f4c00640 1 --2- 172.21.15.151:0/3223244699 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f88cc079150 0x7f88cc07b610 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:34.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:34.626+0000 7f88f4c00640 1 --2- 172.21.15.151:0/3223244699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88f0145750 0x7f88f01431a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:34.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:34.626+0000 7f88f4c00640 1 --2- 172.21.15.151:0/3223244699 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f88f00cccd0 0x7f88f0137240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:34.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:34.626+0000 7f88f4c00640 1 --2- 172.21.15.151:0/3223244699 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f88f006efa0 0x7f88f01f8880 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:34.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:34.626+0000 7f88f4c00640 1 -- 172.21.15.151:0/3223244699 >> 172.21.15.151:0/3223244699 conn(0x7f88f00b1840 msgr2=0x7f88f00afe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:34.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:34.626+0000 7f88f4c00640 1 -- 172.21.15.151:0/3223244699 shutdown_connections 2024-09-16T15:44:34.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:34.626+0000 7f88f4c00640 1 -- 172.21.15.151:0/3223244699 wait complete. 2024-09-16T15:44:34.645 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-16T15:44:34.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:34.805+0000 7f4cad600640 1 -- 172.21.15.151:0/1808808415 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4c8c004800 msgr2=0x7f4c8c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:34.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:34.805+0000 7f4cad600640 1 --2- 172.21.15.151:0/1808808415 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4c8c004800 0x7f4c8c004c80 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f4c900017d0 tx=0x7f4c8c005830 comp rx=0 tx=0).stop 2024-09-16T15:44:34.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:34.805+0000 7f4ca7400640 1 -- 172.21.15.151:0/1808808415 reap_dead start 2024-09-16T15:44:34.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:34.805+0000 7f4cad600640 1 -- 172.21.15.151:0/1808808415 shutdown_connections 2024-09-16T15:44:34.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:34.805+0000 7f4cad600640 1 -- 172.21.15.151:0/1808808415 >> 172.21.15.151:0/1808808415 conn(0x7f4ca80b18e0 msgr2=0x7f4ca80b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:34.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:34.806+0000 7f4cad600640 1 -- 172.21.15.151:0/1808808415 shutdown_connections 2024-09-16T15:44:34.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:34.806+0000 7f4cad600640 1 -- 172.21.15.151:0/1808808415 wait complete. 2024-09-16T15:44:34.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:34.807+0000 7f4cad600640 1 Processor -- start 2024-09-16T15:44:34.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:34.807+0000 7f4cad600640 1 -- start start 2024-09-16T15:44:34.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:34.808+0000 7f4cad600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c8c004800 0x7f4ca81f50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:34.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:34.808+0000 7f4cad600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ca814f980 0x7f4ca815a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:34.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:34.808+0000 7f4cad600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4ca815c740 0x7f4ca815cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:34.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:34.808+0000 7f4cad600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca80b4b10 con 0x7f4ca814f980 2024-09-16T15:44:34.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:34.808+0000 7f4cad600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca80adde0 con 0x7f4c8c004800 2024-09-16T15:44:34.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:34.808+0000 7f4cad600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f4ca815acd0 con 0x7f4ca815c740 2024-09-16T15:44:34.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:34.809+0000 7f4c9e000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ca814f980 0x7f4ca815a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:34.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:34.809+0000 7f4c9e000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ca814f980 0x7f4ca815a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33920/0 (socket says 172.21.15.151:33920) 2024-09-16T15:44:34.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:34.809+0000 7f4c9e000640 1 -- 172.21.15.151:0/433577037 learned_addr learned my addr 172.21.15.151:0/433577037 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:34.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:34.809+0000 7f4ca7400640 1 --2- 172.21.15.151:0/433577037 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4ca815c740 0x7f4ca815cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:34.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:34.809+0000 7f4ca6a00640 1 --2- 172.21.15.151:0/433577037 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c8c004800 0x7f4ca81f50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:34.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:34.809+0000 7f4c9e000640 1 -- 172.21.15.151:0/433577037 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4ca815c740 msgr2=0x7f4ca815cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:34.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:34.809+0000 7f4c9e000640 1 --2- 172.21.15.151:0/433577037 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4ca815c740 0x7f4ca815cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:34.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:34.809+0000 7f4c9e000640 1 -- 172.21.15.151:0/433577037 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c8c004800 msgr2=0x7f4ca81f50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:34.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:34.809+0000 7f4c9e000640 1 --2- 172.21.15.151:0/433577037 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c8c004800 0x7f4ca81f50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:34.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:34.809+0000 7f4c9e000640 1 -- 172.21.15.151:0/433577037 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ca815afb0 con 0x7f4ca814f980 2024-09-16T15:44:34.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:34.809+0000 7f4ca6a00640 1 --2- 172.21.15.151:0/433577037 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c8c004800 0x7f4ca81f50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:44:34.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:34.809+0000 7f4c9e000640 1 --2- 172.21.15.151:0/433577037 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ca814f980 0x7f4ca815a480 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f4c900018f0 tx=0x7f4c900069d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:34.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:34.810+0000 7f4c9f400640 1 -- 172.21.15.151:0/433577037 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c90043070 con 0x7f4ca814f980 2024-09-16T15:44:34.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:34.810+0000 7f4c9f400640 1 -- 172.21.15.151:0/433577037 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4c900038e0 con 0x7f4ca814f980 2024-09-16T15:44:34.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:34.810+0000 7f4cad600640 1 -- 172.21.15.151:0/433577037 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ca815b290 con 0x7f4ca814f980 2024-09-16T15:44:34.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:34.810+0000 7f4c9f400640 1 -- 172.21.15.151:0/433577037 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c90003d10 con 0x7f4ca814f980 2024-09-16T15:44:34.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:34.810+0000 7f4cad600640 1 -- 172.21.15.151:0/433577037 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ca815b850 con 0x7f4ca814f980 2024-09-16T15:44:34.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:34.811+0000 7f4cad600640 1 -- 172.21.15.151:0/433577037 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ca8069360 con 0x7f4ca814f980 2024-09-16T15:44:34.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:34.812+0000 7f4c9f400640 1 -- 172.21.15.151:0/433577037 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 102103+0+0 (secure 0 0 0) 0x7f4c90046070 con 0x7f4ca814f980 2024-09-16T15:44:34.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:34.813+0000 7f4c9f400640 1 --2- 172.21.15.151:0/433577037 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4c84079180 0x7f4c8407b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:34.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:34.813+0000 7f4c9f400640 1 -- 172.21.15.151:0/433577037 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5968+0+0 (secure 0 0 0) 0x7f4c900b8f10 con 0x7f4ca814f980 2024-09-16T15:44:34.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:34.814+0000 7f4ca6a00640 1 --2- 172.21.15.151:0/433577037 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4c84079180 0x7f4c8407b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:34.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:34.814+0000 7f4ca6a00640 1 --2- 172.21.15.151:0/433577037 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4c84079180 0x7f4c8407b640 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4c98001240 tx=0x7f4c98001130 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:34.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:34.817+0000 7f4c9f400640 1 -- 172.21.15.151:0/433577037 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4c900829a0 con 0x7f4ca814f980 2024-09-16T15:44:35.001 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:35.000+0000 7f4cad600640 1 -- 172.21.15.151:0/433577037 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4ca80ad6d0 con 0x7f4ca814f980 2024-09-16T15:44:35.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:35.630+0000 7f4c9f400640 1 -- 172.21.15.151:0/433577037 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v53) ==== 181+0+0 (secure 0 0 0) 0x7f4c900873d0 con 0x7f4ca814f980 2024-09-16T15:44:35.631 INFO:teuthology.orchestra.run.smithi151.stderr:pool 'cephfs_metadata' removed 2024-09-16T15:44:35.634 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:35.633+0000 7f4cad600640 1 -- 172.21.15.151:0/433577037 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4c84079180 msgr2=0x7f4c8407b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:35.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:35.633+0000 7f4cad600640 1 --2- 172.21.15.151:0/433577037 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4c84079180 0x7f4c8407b640 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4c98001240 tx=0x7f4c98001130 comp rx=0 tx=0).stop 2024-09-16T15:44:35.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:35.633+0000 7f4cad600640 1 -- 172.21.15.151:0/433577037 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ca814f980 msgr2=0x7f4ca815a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:35.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:35.633+0000 7f4cad600640 1 --2- 172.21.15.151:0/433577037 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ca814f980 0x7f4ca815a480 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f4c900018f0 tx=0x7f4c900069d0 comp rx=0 tx=0).stop 2024-09-16T15:44:35.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:35.634+0000 7f4cad600640 1 -- 172.21.15.151:0/433577037 shutdown_connections 2024-09-16T15:44:35.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:35.634+0000 7f4cad600640 1 --2- 172.21.15.151:0/433577037 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4c84079180 0x7f4c8407b640 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:35.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:35.634+0000 7f4cad600640 1 --2- 172.21.15.151:0/433577037 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4ca815c740 0x7f4ca815cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:35.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:35.634+0000 7f4cad600640 1 --2- 172.21.15.151:0/433577037 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ca814f980 0x7f4ca815a480 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:35.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:35.634+0000 7f4cad600640 1 --2- 172.21.15.151:0/433577037 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c8c004800 0x7f4ca81f50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:35.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:35.634+0000 7f4cad600640 1 -- 172.21.15.151:0/433577037 >> 172.21.15.151:0/433577037 conn(0x7f4ca80b18e0 msgr2=0x7f4ca80b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:35.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:35.634+0000 7f4cad600640 1 -- 172.21.15.151:0/433577037 shutdown_connections 2024-09-16T15:44:35.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:35.634+0000 7f4cad600640 1 -- 172.21.15.151:0/433577037 wait complete. 2024-09-16T15:44:35.653 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-16T15:44:35.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:35.816+0000 7efcaea00640 1 -- 172.21.15.151:0/107609590 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efca8159aa0 msgr2=0x7efca80741a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:35.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:35.816+0000 7efcaea00640 1 --2- 172.21.15.151:0/107609590 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efca8159aa0 0x7efca80741a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7efc98009860 tx=0x7efc9802a540 comp rx=0 tx=0).stop 2024-09-16T15:44:35.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:35.816+0000 7efcaea00640 1 -- 172.21.15.151:0/107609590 shutdown_connections 2024-09-16T15:44:35.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:35.816+0000 7efcaea00640 1 --2- 172.21.15.151:0/107609590 >> v2:172.21.15.167:3300/0 conn(0x7efca815cff0 0x7efca806e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:35.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:35.816+0000 7efcaea00640 1 --2- 172.21.15.151:0/107609590 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efca8159aa0 0x7efca80741a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:35.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:35.816+0000 7efcaea00640 1 -- 172.21.15.151:0/107609590 >> 172.21.15.151:0/107609590 conn(0x7efca80b18e0 msgr2=0x7efca80b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:35.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:35.816+0000 7efcaea00640 1 -- 172.21.15.151:0/107609590 shutdown_connections 2024-09-16T15:44:35.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:35.816+0000 7efcaea00640 1 -- 172.21.15.151:0/107609590 wait complete. 2024-09-16T15:44:35.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:35.818+0000 7efcaea00640 1 Processor -- start 2024-09-16T15:44:35.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:35.818+0000 7efcaea00640 1 -- start start 2024-09-16T15:44:35.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:35.819+0000 7efcaea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efca806f220 0x7efca80c8f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:35.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:35.819+0000 7efcaea00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efca8159aa0 0x7efca81f1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:35.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:35.819+0000 7efcaea00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efca815cff0 0x7efca80cade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:35.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:35.819+0000 7efcaea00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efca806fa60 con 0x7efca806f220 2024-09-16T15:44:35.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:35.819+0000 7efcaea00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efca806f8e0 con 0x7efca8159aa0 2024-09-16T15:44:35.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:35.819+0000 7efcaea00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7efca80cc8a0 con 0x7efca815cff0 2024-09-16T15:44:35.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:35.819+0000 7efca7e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efca806f220 0x7efca80c8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:35.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:35.819+0000 7efca7e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efca806f220 0x7efca80c8f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33936/0 (socket says 172.21.15.151:33936) 2024-09-16T15:44:35.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:35.819+0000 7efca7e00640 1 -- 172.21.15.151:0/3032278662 learned_addr learned my addr 172.21.15.151:0/3032278662 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:35.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:35.820+0000 7efca7400640 1 --2- 172.21.15.151:0/3032278662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efca8159aa0 0x7efca81f1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:35.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:35.820+0000 7efcaca00640 1 --2- 172.21.15.151:0/3032278662 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efca815cff0 0x7efca80cade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:35.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:35.820+0000 7efca7e00640 1 -- 172.21.15.151:0/3032278662 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efca815cff0 msgr2=0x7efca80cade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:35.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:35.820+0000 7efca7e00640 1 --2- 172.21.15.151:0/3032278662 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efca815cff0 0x7efca80cade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:35.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:35.820+0000 7efca7e00640 1 -- 172.21.15.151:0/3032278662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efca8159aa0 msgr2=0x7efca81f1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:35.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:35.820+0000 7efca7e00640 1 --2- 172.21.15.151:0/3032278662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efca8159aa0 0x7efca81f1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:35.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:35.820+0000 7efca7e00640 1 -- 172.21.15.151:0/3032278662 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efca81f84d0 con 0x7efca806f220 2024-09-16T15:44:35.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:35.820+0000 7efca7400640 1 --2- 172.21.15.151:0/3032278662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efca8159aa0 0x7efca81f1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:44:35.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:35.820+0000 7efca7e00640 1 --2- 172.21.15.151:0/3032278662 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efca806f220 0x7efca80c8f60 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7efc98009980 tx=0x7efc98032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:35.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:35.820+0000 7efc9fe00640 1 -- 172.21.15.151:0/3032278662 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc98040070 con 0x7efca806f220 2024-09-16T15:44:35.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:35.820+0000 7efc9fe00640 1 -- 172.21.15.151:0/3032278662 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc980038e0 con 0x7efca806f220 2024-09-16T15:44:35.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:35.820+0000 7efc9fe00640 1 -- 172.21.15.151:0/3032278662 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc98005c90 con 0x7efca806f220 2024-09-16T15:44:35.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:35.821+0000 7efcaea00640 1 -- 172.21.15.151:0/3032278662 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efca81f87b0 con 0x7efca806f220 2024-09-16T15:44:35.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:35.821+0000 7efcaea00640 1 -- 172.21.15.151:0/3032278662 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efca81f8d70 con 0x7efca806f220 2024-09-16T15:44:35.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:35.822+0000 7efcaea00640 1 -- 172.21.15.151:0/3032278662 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efca8069360 con 0x7efca806f220 2024-09-16T15:44:35.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:35.823+0000 7efc9fe00640 1 -- 172.21.15.151:0/3032278662 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 102103+0+0 (secure 0 0 0) 0x7efc98039040 con 0x7efca806f220 2024-09-16T15:44:35.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:35.824+0000 7efc9fe00640 1 --2- 172.21.15.151:0/3032278662 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efc88079150 0x7efc8807b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:35.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:35.824+0000 7efc9fe00640 1 -- 172.21.15.151:0/3032278662 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5530+0+0 (secure 0 0 0) 0x7efc980ba7f0 con 0x7efca806f220 2024-09-16T15:44:35.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:35.824+0000 7efca7400640 1 --2- 172.21.15.151:0/3032278662 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efc88079150 0x7efc8807b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:35.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:35.825+0000 7efca7400640 1 --2- 172.21.15.151:0/3032278662 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efc88079150 0x7efc8807b610 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7efca80ae6c0 tx=0x7efc94005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:35.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:35.828+0000 7efc9fe00640 1 -- 172.21.15.151:0/3032278662 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7efc98084830 con 0x7efca806f220 2024-09-16T15:44:36.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:36.012+0000 7efcaea00640 1 -- 172.21.15.151:0/3032278662 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7efca8070980 con 0x7efca806f220 2024-09-16T15:44:36.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:36.637+0000 7efc9fe00640 1 -- 172.21.15.151:0/3032278662 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v54) ==== 169+0+0 (secure 0 0 0) 0x7efc98084130 con 0x7efca806f220 2024-09-16T15:44:36.638 INFO:teuthology.orchestra.run.smithi151.stderr:pool 'cephfs_data' removed 2024-09-16T15:44:36.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:36.640+0000 7efcaea00640 1 -- 172.21.15.151:0/3032278662 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efc88079150 msgr2=0x7efc8807b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:36.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:36.640+0000 7efcaea00640 1 --2- 172.21.15.151:0/3032278662 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efc88079150 0x7efc8807b610 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7efca80ae6c0 tx=0x7efc94005f90 comp rx=0 tx=0).stop 2024-09-16T15:44:36.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:36.640+0000 7efcaea00640 1 -- 172.21.15.151:0/3032278662 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efca806f220 msgr2=0x7efca80c8f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:36.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:36.640+0000 7efcaea00640 1 --2- 172.21.15.151:0/3032278662 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efca806f220 0x7efca80c8f60 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7efc98009980 tx=0x7efc98032c10 comp rx=0 tx=0).stop 2024-09-16T15:44:36.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:36.641+0000 7efcaea00640 1 -- 172.21.15.151:0/3032278662 shutdown_connections 2024-09-16T15:44:36.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:36.641+0000 7efcaea00640 1 --2- 172.21.15.151:0/3032278662 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efc88079150 0x7efc8807b610 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:36.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:36.641+0000 7efcaea00640 1 --2- 172.21.15.151:0/3032278662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efca8159aa0 0x7efca81f1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:36.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:36.641+0000 7efcaea00640 1 --2- 172.21.15.151:0/3032278662 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efca815cff0 0x7efca80cade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:36.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:36.641+0000 7efcaea00640 1 --2- 172.21.15.151:0/3032278662 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efca806f220 0x7efca80c8f60 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:36.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:36.641+0000 7efcaea00640 1 -- 172.21.15.151:0/3032278662 >> 172.21.15.151:0/3032278662 conn(0x7efca80b18e0 msgr2=0x7efca80ccce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:36.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:36.641+0000 7efcaea00640 1 -- 172.21.15.151:0/3032278662 shutdown_connections 2024-09-16T15:44:36.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:36.642+0000 7efcaea00640 1 -- 172.21.15.151:0/3032278662 wait complete. 2024-09-16T15:44:36.662 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-16T15:44:36.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:36.824+0000 7fb28be00640 1 -- 172.21.15.151:0/636198760 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb2700052e0 msgr2=0x7fb27000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:36.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:36.824+0000 7fb28be00640 1 --2- 172.21.15.151:0/636198760 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb2700052e0 0x7fb27000b9c0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fb284009ed0 tx=0x7fb270005830 comp rx=0 tx=0).stop 2024-09-16T15:44:36.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:36.824+0000 7fb28be00640 1 -- 172.21.15.151:0/636198760 shutdown_connections 2024-09-16T15:44:36.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:36.824+0000 7fb28be00640 1 --2- 172.21.15.151:0/636198760 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb2700052e0 0x7fb27000b9c0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:36.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:36.824+0000 7fb28be00640 1 -- 172.21.15.151:0/636198760 >> 172.21.15.151:0/636198760 conn(0x7fb28c0d3df0 msgr2=0x7fb28c0d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:36.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:36.824+0000 7fb28be00640 1 -- 172.21.15.151:0/636198760 shutdown_connections 2024-09-16T15:44:36.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:36.824+0000 7fb28be00640 1 -- 172.21.15.151:0/636198760 wait complete. 2024-09-16T15:44:36.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:36.825+0000 7fb28be00640 1 Processor -- start 2024-09-16T15:44:36.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:36.826+0000 7fb28be00640 1 -- start start 2024-09-16T15:44:36.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:36.826+0000 7fb28be00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb2700052e0 0x7fb28c155e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:36.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:36.826+0000 7fb28be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb28c0b6c20 0x7fb28c0ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:36.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:36.826+0000 7fb28be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb28c0ccff0 0x7fb28c0ca2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:36.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:36.826+0000 7fb28be00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb28c0b1b00 con 0x7fb28c0b6c20 2024-09-16T15:44:36.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:36.826+0000 7fb28be00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb28c0b1980 con 0x7fb28c0ccff0 2024-09-16T15:44:36.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:36.826+0000 7fb28be00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fb28c0b1c80 con 0x7fb2700052e0 2024-09-16T15:44:36.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:36.827+0000 7fb28a000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb28c0b6c20 0x7fb28c0ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:36.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:36.827+0000 7fb28a000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb28c0b6c20 0x7fb28c0ccab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33956/0 (socket says 172.21.15.151:33956) 2024-09-16T15:44:36.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:36.827+0000 7fb28a000640 1 -- 172.21.15.151:0/3116870881 learned_addr learned my addr 172.21.15.151:0/3116870881 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:36.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:36.827+0000 7fb28b400640 1 --2- 172.21.15.151:0/3116870881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb28c0ccff0 0x7fb28c0ca2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:36.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:36.827+0000 7fb28aa00640 1 --2- 172.21.15.151:0/3116870881 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb2700052e0 0x7fb28c155e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:36.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:36.827+0000 7fb28a000640 1 -- 172.21.15.151:0/3116870881 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb2700052e0 msgr2=0x7fb28c155e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:36.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:36.827+0000 7fb28a000640 1 --2- 172.21.15.151:0/3116870881 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb2700052e0 0x7fb28c155e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:36.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:36.827+0000 7fb28a000640 1 -- 172.21.15.151:0/3116870881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb28c0ccff0 msgr2=0x7fb28c0ca2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:36.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:36.827+0000 7fb28a000640 1 --2- 172.21.15.151:0/3116870881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb28c0ccff0 0x7fb28c0ca2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:36.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:36.827+0000 7fb28a000640 1 -- 172.21.15.151:0/3116870881 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb28c0cd750 con 0x7fb28c0b6c20 2024-09-16T15:44:36.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:36.827+0000 7fb28a000640 1 --2- 172.21.15.151:0/3116870881 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb28c0b6c20 0x7fb28c0ccab0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fb278003f50 tx=0x7fb2780016f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:36.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:36.827+0000 7fb282a00640 1 -- 172.21.15.151:0/3116870881 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb278015070 con 0x7fb28c0b6c20 2024-09-16T15:44:36.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:36.828+0000 7fb282a00640 1 -- 172.21.15.151:0/3116870881 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb278011860 con 0x7fb28c0b6c20 2024-09-16T15:44:36.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:36.828+0000 7fb28be00640 1 -- 172.21.15.151:0/3116870881 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb28c0cd470 con 0x7fb28c0b6c20 2024-09-16T15:44:36.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:36.828+0000 7fb282a00640 1 -- 172.21.15.151:0/3116870881 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb278011cb0 con 0x7fb28c0b6c20 2024-09-16T15:44:36.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:36.828+0000 7fb28be00640 1 -- 172.21.15.151:0/3116870881 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb28c0cb240 con 0x7fb28c0b6c20 2024-09-16T15:44:36.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:36.829+0000 7fb28be00640 1 -- 172.21.15.151:0/3116870881 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb28c0b21e0 con 0x7fb28c0b6c20 2024-09-16T15:44:36.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:36.830+0000 7fb282a00640 1 -- 172.21.15.151:0/3116870881 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 102103+0+0 (secure 0 0 0) 0x7fb278018070 con 0x7fb28c0b6c20 2024-09-16T15:44:36.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:36.831+0000 7fb282a00640 1 --2- 172.21.15.151:0/3116870881 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb26c079150 0x7fb26c07b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:36.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:36.831+0000 7fb282a00640 1 -- 172.21.15.151:0/3116870881 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4971+0+0 (secure 0 0 0) 0x7fb278099e00 con 0x7fb28c0b6c20 2024-09-16T15:44:36.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:36.831+0000 7fb28aa00640 1 --2- 172.21.15.151:0/3116870881 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb26c079150 0x7fb26c07b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:36.833 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:36.832+0000 7fb28aa00640 1 --2- 172.21.15.151:0/3116870881 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb26c079150 0x7fb26c07b610 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb27c006bd0 tx=0x7fb27c0067a0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:36.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:36.835+0000 7fb282a00640 1 -- 172.21.15.151:0/3116870881 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb278063870 con 0x7fb28c0b6c20 2024-09-16T15:44:37.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.039+0000 7fb28be00640 1 -- 172.21.15.151:0/3116870881 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7fb28c158a90 con 0x7fb28c0b6c20 2024-09-16T15:44:37.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.043+0000 7fb282a00640 1 -- 172.21.15.151:0/3116870881 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v33) ==== 71+0+0 (secure 0 0 0) 0x7fb2780682a0 con 0x7fb28c0b6c20 2024-09-16T15:44:37.047 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.045+0000 7fb28be00640 1 -- 172.21.15.151:0/3116870881 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb26c079150 msgr2=0x7fb26c07b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:37.047 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.045+0000 7fb28be00640 1 --2- 172.21.15.151:0/3116870881 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb26c079150 0x7fb26c07b610 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb27c006bd0 tx=0x7fb27c0067a0 comp rx=0 tx=0).stop 2024-09-16T15:44:37.047 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.045+0000 7fb28be00640 1 -- 172.21.15.151:0/3116870881 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb28c0b6c20 msgr2=0x7fb28c0ccab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:37.047 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.045+0000 7fb28be00640 1 --2- 172.21.15.151:0/3116870881 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb28c0b6c20 0x7fb28c0ccab0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fb278003f50 tx=0x7fb2780016f0 comp rx=0 tx=0).stop 2024-09-16T15:44:37.047 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.046+0000 7fb28be00640 1 -- 172.21.15.151:0/3116870881 shutdown_connections 2024-09-16T15:44:37.047 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.046+0000 7fb28be00640 1 --2- 172.21.15.151:0/3116870881 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb26c079150 0x7fb26c07b610 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:37.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.046+0000 7fb28be00640 1 --2- 172.21.15.151:0/3116870881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb28c0ccff0 0x7fb28c0ca2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:37.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.046+0000 7fb28be00640 1 --2- 172.21.15.151:0/3116870881 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb28c0b6c20 0x7fb28c0ccab0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:37.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.046+0000 7fb28be00640 1 --2- 172.21.15.151:0/3116870881 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb2700052e0 0x7fb28c155e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:37.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.046+0000 7fb28be00640 1 -- 172.21.15.151:0/3116870881 >> 172.21.15.151:0/3116870881 conn(0x7fb28c0d3df0 msgr2=0x7fb28c06d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:37.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.046+0000 7fb28be00640 1 -- 172.21.15.151:0/3116870881 shutdown_connections 2024-09-16T15:44:37.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.046+0000 7fb28be00640 1 -- 172.21.15.151:0/3116870881 wait complete. 2024-09-16T15:44:37.063 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_equal_to_cloner_threads' 2024-09-16T15:44:37.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.221+0000 7fa5c7e00640 1 -- 172.21.15.151:0/1878565526 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5ac004d40 msgr2=0x7fa5ac0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:37.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.221+0000 7fa5c7e00640 1 --2- 172.21.15.151:0/1878565526 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5ac004d40 0x7fa5ac0051c0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0009ed0 tx=0x7fa5ac005830 comp rx=0 tx=0).stop 2024-09-16T15:44:37.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.221+0000 7fa5c7400640 1 -- 172.21.15.151:0/1878565526 reap_dead start 2024-09-16T15:44:37.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.221+0000 7fa5c7e00640 1 -- 172.21.15.151:0/1878565526 shutdown_connections 2024-09-16T15:44:37.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.221+0000 7fa5c7e00640 1 -- 172.21.15.151:0/1878565526 >> 172.21.15.151:0/1878565526 conn(0x7fa5c80cfa20 msgr2=0x7fa5c80cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:37.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.221+0000 7fa5c7e00640 1 -- 172.21.15.151:0/1878565526 shutdown_connections 2024-09-16T15:44:37.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.221+0000 7fa5c7e00640 1 -- 172.21.15.151:0/1878565526 wait complete. 2024-09-16T15:44:37.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.223+0000 7fa5c7e00640 1 Processor -- start 2024-09-16T15:44:37.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.223+0000 7fa5c7e00640 1 -- start start 2024-09-16T15:44:37.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.223+0000 7fa5c7e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa5ac004d40 0x7fa5c8148560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:37.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.223+0000 7fa5c7e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa5c80c7b40 0x7fa5c80c7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:37.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.224+0000 7fa5c7e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5c81f78c0 0x7fa5c81f7d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:37.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.224+0000 7fa5c7e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c8074660 con 0x7fa5c81f78c0 2024-09-16T15:44:37.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.224+0000 7fa5c7e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c806f320 con 0x7fa5c80c7b40 2024-09-16T15:44:37.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.224+0000 7fa5c7e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fa5c80c8590 con 0x7fa5ac004d40 2024-09-16T15:44:37.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.224+0000 7fa5c7400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5c81f78c0 0x7fa5c81f7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:37.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.224+0000 7fa5c7400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5c81f78c0 0x7fa5c81f7d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33972/0 (socket says 172.21.15.151:33972) 2024-09-16T15:44:37.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.224+0000 7fa5c7400640 1 -- 172.21.15.151:0/2729228604 learned_addr learned my addr 172.21.15.151:0/2729228604 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:37.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.224+0000 7fa5c6000640 1 --2- 172.21.15.151:0/2729228604 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa5c80c7b40 0x7fa5c80c7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:37.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.224+0000 7fa5c6a00640 1 --2- 172.21.15.151:0/2729228604 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa5ac004d40 0x7fa5c8148560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:37.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.224+0000 7fa5c7400640 1 -- 172.21.15.151:0/2729228604 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa5ac004d40 msgr2=0x7fa5c8148560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:37.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.224+0000 7fa5c7400640 1 --2- 172.21.15.151:0/2729228604 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa5ac004d40 0x7fa5c8148560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:37.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.224+0000 7fa5c7400640 1 -- 172.21.15.151:0/2729228604 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa5c80c7b40 msgr2=0x7fa5c80c7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:37.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.224+0000 7fa5c7400640 1 --2- 172.21.15.151:0/2729228604 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa5c80c7b40 0x7fa5c80c7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:37.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.224+0000 7fa5c7400640 1 -- 172.21.15.151:0/2729228604 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5c80c8870 con 0x7fa5c81f78c0 2024-09-16T15:44:37.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.225+0000 7fa5c7400640 1 --2- 172.21.15.151:0/2729228604 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5c81f78c0 0x7fa5c81f7d40 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0009ed0 tx=0x7fa5c0033bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:37.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.225+0000 7fa5bea00640 1 -- 172.21.15.151:0/2729228604 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c0043070 con 0x7fa5c81f78c0 2024-09-16T15:44:37.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.225+0000 7fa5bea00640 1 -- 172.21.15.151:0/2729228604 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa5c0033ce0 con 0x7fa5c81f78c0 2024-09-16T15:44:37.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.225+0000 7fa5c7e00640 1 -- 172.21.15.151:0/2729228604 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5c80c8b50 con 0x7fa5c81f78c0 2024-09-16T15:44:37.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.225+0000 7fa5bea00640 1 -- 172.21.15.151:0/2729228604 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c0038040 con 0x7fa5c81f78c0 2024-09-16T15:44:37.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.225+0000 7fa5c7e00640 1 -- 172.21.15.151:0/2729228604 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5c80c9110 con 0x7fa5c81f78c0 2024-09-16T15:44:37.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.226+0000 7fa5c7e00640 1 -- 172.21.15.151:0/2729228604 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5c80b28f0 con 0x7fa5c81f78c0 2024-09-16T15:44:37.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.227+0000 7fa5bea00640 1 -- 172.21.15.151:0/2729228604 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 102103+0+0 (secure 0 0 0) 0x7fa5c0003bd0 con 0x7fa5c81f78c0 2024-09-16T15:44:37.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.228+0000 7fa5bea00640 1 --2- 172.21.15.151:0/2729228604 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa5a8079180 0x7fa5a807b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:37.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.228+0000 7fa5bea00640 1 -- 172.21.15.151:0/2729228604 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4971+0+0 (secure 0 0 0) 0x7fa5c00b8e20 con 0x7fa5c81f78c0 2024-09-16T15:44:37.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.229+0000 7fa5c6a00640 1 --2- 172.21.15.151:0/2729228604 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa5a8079180 0x7fa5a807b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:37.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.229+0000 7fa5c6a00640 1 --2- 172.21.15.151:0/2729228604 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa5a8079180 0x7fa5a807b640 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8001cc0 tx=0x7fa5b8006720 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:37.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.232+0000 7fa5bea00640 1 -- 172.21.15.151:0/2729228604 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa5c0082800 con 0x7fa5c81f78c0 2024-09-16T15:44:37.488 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.487+0000 7fa5c7e00640 1 -- 172.21.15.151:0/2729228604 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_equal_to_cloner_threads"]} v 0) -- 0x7fa5c80c92e0 con 0x7fa5c81f78c0 2024-09-16T15:44:37.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.638+0000 7fa5bea00640 1 -- 172.21.15.151:0/2729228604 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_equal_to_cloner_threads"]}]=0 v240) ==== 166+0+0 (secure 0 0 0) 0x7fa5c0087230 con 0x7fa5c81f78c0 2024-09-16T15:44:37.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.641+0000 7fa5c7e00640 1 -- 172.21.15.151:0/2729228604 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa5a8079180 msgr2=0x7fa5a807b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:37.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.641+0000 7fa5c7e00640 1 --2- 172.21.15.151:0/2729228604 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa5a8079180 0x7fa5a807b640 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8001cc0 tx=0x7fa5b8006720 comp rx=0 tx=0).stop 2024-09-16T15:44:37.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.641+0000 7fa5c7e00640 1 -- 172.21.15.151:0/2729228604 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5c81f78c0 msgr2=0x7fa5c81f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:37.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.641+0000 7fa5c7e00640 1 --2- 172.21.15.151:0/2729228604 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5c81f78c0 0x7fa5c81f7d40 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0009ed0 tx=0x7fa5c0033bf0 comp rx=0 tx=0).stop 2024-09-16T15:44:37.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.641+0000 7fa5c7e00640 1 -- 172.21.15.151:0/2729228604 shutdown_connections 2024-09-16T15:44:37.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.641+0000 7fa5c7e00640 1 --2- 172.21.15.151:0/2729228604 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa5a8079180 0x7fa5a807b640 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:37.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.641+0000 7fa5c7e00640 1 --2- 172.21.15.151:0/2729228604 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa5c80c7b40 0x7fa5c80c7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:37.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.641+0000 7fa5c7e00640 1 --2- 172.21.15.151:0/2729228604 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5c81f78c0 0x7fa5c81f7d40 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:37.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.641+0000 7fa5c7e00640 1 --2- 172.21.15.151:0/2729228604 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa5ac004d40 0x7fa5c8148560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:37.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.641+0000 7fa5c7e00640 1 -- 172.21.15.151:0/2729228604 >> 172.21.15.151:0/2729228604 conn(0x7fa5c80cfa20 msgr2=0x7fa5c80cd370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:37.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.641+0000 7fa5c7e00640 1 -- 172.21.15.151:0/2729228604 shutdown_connections 2024-09-16T15:44:37.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.641+0000 7fa5c7e00640 1 -- 172.21.15.151:0/2729228604 wait complete. 2024-09-16T15:44:37.659 INFO:tasks.cephfs_test_runner:Test that one progress bar is printed in output of "ceph status" output ... ok 2024-09-16T15:44:37.659 INFO:tasks.cephfs_test_runner:Starting test: test_clones_less_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that one progress bar is printed in output of "ceph status" output 2024-09-16T15:44:37.659 INFO:tasks.cephfs_test_runner:test_clones_less_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-09-16T15:44:37.660 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:44:37.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.815+0000 7f9504a00640 1 -- 172.21.15.151:0/1906283316 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f95000af3d0 msgr2=0x7f950006f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:37.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.815+0000 7f9504a00640 1 --2- 172.21.15.151:0/1906283316 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f95000af3d0 0x7f950006f490 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f950006d9e0 tx=0x7f94ec02a4d0 comp rx=0 tx=0).stop 2024-09-16T15:44:37.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.816+0000 7f9504a00640 1 -- 172.21.15.151:0/1906283316 shutdown_connections 2024-09-16T15:44:37.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.816+0000 7f9504a00640 1 --2- 172.21.15.151:0/1906283316 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f950006f9d0 0x7f9500161c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:37.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.816+0000 7f9504a00640 1 --2- 172.21.15.151:0/1906283316 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f95000af3d0 0x7f950006f490 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:37.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.816+0000 7f9504a00640 1 --2- 172.21.15.151:0/1906283316 >> v2:172.21.15.167:3300/0 conn(0x7f9500161550 0x7f950006ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:37.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.816+0000 7f9504a00640 1 -- 172.21.15.151:0/1906283316 >> 172.21.15.151:0/1906283316 conn(0x7f95000b1950 msgr2=0x7f95000b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:37.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.816+0000 7f9504a00640 1 -- 172.21.15.151:0/1906283316 shutdown_connections 2024-09-16T15:44:37.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.816+0000 7f9504a00640 1 -- 172.21.15.151:0/1906283316 wait complete. 2024-09-16T15:44:37.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.817+0000 7f9504a00640 1 Processor -- start 2024-09-16T15:44:37.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.817+0000 7f9504a00640 1 -- start start 2024-09-16T15:44:37.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.818+0000 7f9504a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f950006f9d0 0x7f95000ae040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:37.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.818+0000 7f9504a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f95000af3d0 0x7f950015a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:37.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.818+0000 7f9504a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9500161550 0x7f95001f7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:37.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.818+0000 7f9504a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9500162560 con 0x7f950006f9d0 2024-09-16T15:44:37.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.818+0000 7f9504a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f95001623e0 con 0x7f9500161550 2024-09-16T15:44:37.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.818+0000 7f9504a00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f95001626e0 con 0x7f95000af3d0 2024-09-16T15:44:37.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.819+0000 7f94fde00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f950006f9d0 0x7f95000ae040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:37.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.819+0000 7f94fde00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f950006f9d0 0x7f95000ae040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33982/0 (socket says 172.21.15.151:33982) 2024-09-16T15:44:37.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.819+0000 7f94fde00640 1 -- 172.21.15.151:0/1867318210 learned_addr learned my addr 172.21.15.151:0/1867318210 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:37.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.819+0000 7f94fe800640 1 --2- 172.21.15.151:0/1867318210 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9500161550 0x7f95001f7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:37.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.819+0000 7f94fd400640 1 --2- 172.21.15.151:0/1867318210 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f95000af3d0 0x7f950015a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:37.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.819+0000 7f94fde00640 1 -- 172.21.15.151:0/1867318210 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f95000af3d0 msgr2=0x7f950015a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:37.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.819+0000 7f94fde00640 1 --2- 172.21.15.151:0/1867318210 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f95000af3d0 0x7f950015a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:37.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.819+0000 7f94fde00640 1 -- 172.21.15.151:0/1867318210 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9500161550 msgr2=0x7f95001f7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:37.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.819+0000 7f94fde00640 1 --2- 172.21.15.151:0/1867318210 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9500161550 0x7f95001f7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:37.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.819+0000 7f94fde00640 1 -- 172.21.15.151:0/1867318210 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95001f82d0 con 0x7f950006f9d0 2024-09-16T15:44:37.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.819+0000 7f94fde00640 1 --2- 172.21.15.151:0/1867318210 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f950006f9d0 0x7f95000ae040 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f94f000d740 tx=0x7f94f000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:37.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.820+0000 7f94f6000640 1 -- 172.21.15.151:0/1867318210 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f0017070 con 0x7f950006f9d0 2024-09-16T15:44:37.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.820+0000 7f94f6000640 1 -- 172.21.15.151:0/1867318210 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f94f0009d20 con 0x7f950006f9d0 2024-09-16T15:44:37.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.820+0000 7f94f6000640 1 -- 172.21.15.151:0/1867318210 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94f00106c0 con 0x7f950006f9d0 2024-09-16T15:44:37.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.820+0000 7f9504a00640 1 -- 172.21.15.151:0/1867318210 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95001f85b0 con 0x7f950006f9d0 2024-09-16T15:44:37.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.820+0000 7f9504a00640 1 -- 172.21.15.151:0/1867318210 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95001f8b70 con 0x7f950006f9d0 2024-09-16T15:44:37.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.821+0000 7f9504a00640 1 -- 172.21.15.151:0/1867318210 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9500161c90 con 0x7f950006f9d0 2024-09-16T15:44:37.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.822+0000 7f94f6000640 1 -- 172.21.15.151:0/1867318210 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 102103+0+0 (secure 0 0 0) 0x7f94f001a070 con 0x7f950006f9d0 2024-09-16T15:44:37.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.823+0000 7f94f6000640 1 --2- 172.21.15.151:0/1867318210 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f94e0079150 0x7f94e007b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:37.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.823+0000 7f94f6000640 1 -- 172.21.15.151:0/1867318210 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4971+0+0 (secure 0 0 0) 0x7f94f00992d0 con 0x7f950006f9d0 2024-09-16T15:44:37.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.824+0000 7f94fd400640 1 --2- 172.21.15.151:0/1867318210 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f94e0079150 0x7f94e007b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:37.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.824+0000 7f94fd400640 1 --2- 172.21.15.151:0/1867318210 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f94e0079150 0x7f94e007b610 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f94ec033000 tx=0x7f94ec004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:37.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:37.827+0000 7f94f6000640 1 -- 172.21.15.151:0/1867318210 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f94f0062d40 con 0x7f950006f9d0 2024-09-16T15:44:38.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.014+0000 7f9504a00640 1 -- 172.21.15.151:0/1867318210 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f95000b3cf0 con 0x7f950006f9d0 2024-09-16T15:44:38.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.015+0000 7f94f6000640 1 -- 172.21.15.151:0/1867318210 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) ==== 74+0+13137 (secure 0 0 0) 0x7f94f0067770 con 0x7f950006f9d0 2024-09-16T15:44:38.016 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:44:38.016 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":54,"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","created":"2024-09-16T15:38:56.348535+0000","modified":"2024-09-16T15:44:36.631444+0000","last_up_change":"2024-09-16T15:39:10.681384+0000","last_in_change":"2024-09-16T15:39:00.544477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:39:11.602252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5524076-a903-4d0a-af6a-caf474c4765d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6809","nonce":3111342741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6811","nonce":3111342741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6815","nonce":3111342741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6813","nonce":3111342741}]},"public_addr":"172.21.15.151:6809/3111342741","cluster_addr":"172.21.15.151:6811/3111342741","heartbeat_back_addr":"172.21.15.151:6815/3111342741","heartbeat_front_addr":"172.21.15.151:6813/3111342741","state":["exists","up"]},{"osd":1,"uuid":"dd25925f-0ae1-494e-8529-4dc205759db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6801","nonce":3368304087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6803","nonce":3368304087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6807","nonce":3368304087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6805","nonce":3368304087}]},"public_addr":"172.21.15.151:6801/3368304087","cluster_addr":"172.21.15.151:6803/3368304087","heartbeat_back_addr":"172.21.15.151:6807/3368304087","heartbeat_front_addr":"172.21.15.151:6805/3368304087","state":["exists","up"]},{"osd":2,"uuid":"c2f18115-013d-4da6-a0fe-cb978f4494b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6823","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6825","nonce":3079373271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6827","nonce":3079373271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6831","nonce":3079373271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6829","nonce":3079373271}]},"public_addr":"172.21.15.151:6825/3079373271","cluster_addr":"172.21.15.151:6827/3079373271","heartbeat_back_addr":"172.21.15.151:6831/3079373271","heartbeat_front_addr":"172.21.15.151:6829/3079373271","state":["exists","up"]},{"osd":3,"uuid":"36edb718-2fcb-4d88-8ddc-c321eed4279d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6817","nonce":1142509041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6819","nonce":1142509041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6824","nonce":1142509041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6821","nonce":1142509041}]},"public_addr":"172.21.15.151:6817/1142509041","cluster_addr":"172.21.15.151:6819/1142509041","heartbeat_back_addr":"172.21.15.151:6824/1142509041","heartbeat_front_addr":"172.21.15.151:6821/1142509041","state":["exists","up"]},{"osd":4,"uuid":"64d33708-cd0a-4236-abe3-1b646e82ab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6811","nonce":2735495271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6813","nonce":2735495271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6817","nonce":2735495271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6815","nonce":2735495271}]},"public_addr":"172.21.15.167:6811/2735495271","cluster_addr":"172.21.15.167:6813/2735495271","heartbeat_back_addr":"172.21.15.167:6817/2735495271","heartbeat_front_addr":"172.21.15.167:6815/2735495271","state":["exists","up"]},{"osd":5,"uuid":"f14b44a9-5910-458c-b518-09a8d0de94de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6827","nonce":1494916572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6829","nonce":1494916572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6833","nonce":1494916572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6831","nonce":1494916572}]},"public_addr":"172.21.15.167:6827/1494916572","cluster_addr":"172.21.15.167:6829/1494916572","heartbeat_back_addr":"172.21.15.167:6833/1494916572","heartbeat_front_addr":"172.21.15.167:6831/1494916572","state":["exists","up"]},{"osd":6,"uuid":"39335545-2654-4710-84b0-a9aa47cfcbba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6819","nonce":2538273273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6821","nonce":2538273273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6825","nonce":2538273273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6823","nonce":2538273273}]},"public_addr":"172.21.15.167:6819/2538273273","cluster_addr":"172.21.15.167:6821/2538273273","heartbeat_back_addr":"172.21.15.167:6825/2538273273","heartbeat_front_addr":"172.21.15.167:6823/2538273273","state":["exists","up"]},{"osd":7,"uuid":"4f8bfc81-53dd-4339-ac22-1a4007d9f963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6803","nonce":265072227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6805","nonce":265072227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6809","nonce":265072227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6807","nonce":265072227}]},"public_addr":"172.21.15.167:6803/265072227","cluster_addr":"172.21.15.167:6805/265072227","heartbeat_back_addr":"172.21.15.167:6809/265072227","heartbeat_front_addr":"172.21.15.167:6807/265072227","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.150359+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.084358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.722368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.332809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.320709+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.395153+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.935250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.801443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.151:6835/1974411916":"2024-09-17T15:44:33.587267+0000","172.21.15.151:6833/1974411916":"2024-09-17T15:44:33.587267+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:44:38.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.017+0000 7f9504a00640 1 -- 172.21.15.151:0/1867318210 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f94e0079150 msgr2=0x7f94e007b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:38.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.017+0000 7f9504a00640 1 --2- 172.21.15.151:0/1867318210 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f94e0079150 0x7f94e007b610 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f94ec033000 tx=0x7f94ec004730 comp rx=0 tx=0).stop 2024-09-16T15:44:38.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.017+0000 7f9504a00640 1 -- 172.21.15.151:0/1867318210 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f950006f9d0 msgr2=0x7f95000ae040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:38.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.017+0000 7f9504a00640 1 --2- 172.21.15.151:0/1867318210 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f950006f9d0 0x7f95000ae040 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f94f000d740 tx=0x7f94f000dc00 comp rx=0 tx=0).stop 2024-09-16T15:44:38.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.017+0000 7f9504a00640 1 -- 172.21.15.151:0/1867318210 shutdown_connections 2024-09-16T15:44:38.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.017+0000 7f9504a00640 1 --2- 172.21.15.151:0/1867318210 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f94e0079150 0x7f94e007b610 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:38.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.017+0000 7f9504a00640 1 --2- 172.21.15.151:0/1867318210 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9500161550 0x7f95001f7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:38.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.017+0000 7f9504a00640 1 --2- 172.21.15.151:0/1867318210 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f95000af3d0 0x7f950015a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:38.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.017+0000 7f9504a00640 1 --2- 172.21.15.151:0/1867318210 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f950006f9d0 0x7f95000ae040 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:38.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.017+0000 7f9504a00640 1 -- 172.21.15.151:0/1867318210 >> 172.21.15.151:0/1867318210 conn(0x7f95000b1950 msgr2=0x7f9500073a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:38.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.017+0000 7f9504a00640 1 -- 172.21.15.151:0/1867318210 shutdown_connections 2024-09-16T15:44:38.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.017+0000 7f9504a00640 1 -- 172.21.15.151:0/1867318210 wait complete. 2024-09-16T15:44:38.035 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T15:39:11.602252+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-16T15:44:38.035 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-16T15:44:38.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.196+0000 7f5bebc00640 1 -- 172.21.15.151:0/357056161 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5be40b3370 msgr2=0x7f5be40cebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:38.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.196+0000 7f5bebc00640 1 --2- 172.21.15.151:0/357056161 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5be40b3370 0x7f5be40cebe0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f5bd4009830 tx=0x7f5bd402a510 comp rx=0 tx=0).stop 2024-09-16T15:44:38.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.196+0000 7f5bebc00640 1 -- 172.21.15.151:0/357056161 shutdown_connections 2024-09-16T15:44:38.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.196+0000 7f5bebc00640 1 --2- 172.21.15.151:0/357056161 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5be415f090 0x7f5be40cf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:38.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.196+0000 7f5bebc00640 1 --2- 172.21.15.151:0/357056161 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5be40b3370 0x7f5be40cebe0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:38.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.196+0000 7f5bebc00640 1 -- 172.21.15.151:0/357056161 >> 172.21.15.151:0/357056161 conn(0x7f5be40b1840 msgr2=0x7f5be40b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:38.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.196+0000 7f5bebc00640 1 -- 172.21.15.151:0/357056161 shutdown_connections 2024-09-16T15:44:38.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.196+0000 7f5bebc00640 1 -- 172.21.15.151:0/357056161 wait complete. 2024-09-16T15:44:38.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.198+0000 7f5bebc00640 1 Processor -- start 2024-09-16T15:44:38.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.198+0000 7f5bebc00640 1 -- start start 2024-09-16T15:44:38.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.199+0000 7f5bebc00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5be40b3370 0x7f5be41f8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:38.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.199+0000 7f5bebc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5be415c080 0x7f5be413b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:38.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.199+0000 7f5bebc00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5be415f090 0x7f5be4159df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:38.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.199+0000 7f5bebc00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5be4164e50 con 0x7f5be415c080 2024-09-16T15:44:38.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.199+0000 7f5bebc00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5be4164cd0 con 0x7f5be415f090 2024-09-16T15:44:38.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.199+0000 7f5bebc00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f5be4164fd0 con 0x7f5be40b3370 2024-09-16T15:44:38.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.199+0000 7f5bdbe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5be415c080 0x7f5be413b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:38.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.199+0000 7f5bdbe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5be415c080 0x7f5be413b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:34008/0 (socket says 172.21.15.151:34008) 2024-09-16T15:44:38.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.199+0000 7f5bdbe00640 1 -- 172.21.15.151:0/611795343 learned_addr learned my addr 172.21.15.151:0/611795343 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:38.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.199+0000 7f5be9c00640 1 --2- 172.21.15.151:0/611795343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5be415f090 0x7f5be4159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:38.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.199+0000 7f5be9200640 1 --2- 172.21.15.151:0/611795343 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5be40b3370 0x7f5be41f8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:38.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.200+0000 7f5bdbe00640 1 -- 172.21.15.151:0/611795343 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5be40b3370 msgr2=0x7f5be41f8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:38.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.200+0000 7f5bdbe00640 1 --2- 172.21.15.151:0/611795343 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5be40b3370 0x7f5be41f8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:38.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.200+0000 7f5bdbe00640 1 -- 172.21.15.151:0/611795343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5be415f090 msgr2=0x7f5be4159df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:38.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.200+0000 7f5bdbe00640 1 --2- 172.21.15.151:0/611795343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5be415f090 0x7f5be4159df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:38.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.200+0000 7f5bdbe00640 1 -- 172.21.15.151:0/611795343 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5be41f9190 con 0x7f5be415c080 2024-09-16T15:44:38.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.200+0000 7f5bdbe00640 1 --2- 172.21.15.151:0/611795343 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5be415c080 0x7f5be413b1f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc00c730 tx=0x7f5bcc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:38.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.200+0000 7f5be1600640 1 -- 172.21.15.151:0/611795343 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc006890 con 0x7f5be415c080 2024-09-16T15:44:38.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.200+0000 7f5be1600640 1 -- 172.21.15.151:0/611795343 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5bcc017070 con 0x7f5be415c080 2024-09-16T15:44:38.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.200+0000 7f5bebc00640 1 -- 172.21.15.151:0/611795343 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5be4152a10 con 0x7f5be415c080 2024-09-16T15:44:38.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.200+0000 7f5be1600640 1 -- 172.21.15.151:0/611795343 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc006de0 con 0x7f5be415c080 2024-09-16T15:44:38.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.200+0000 7f5bebc00640 1 -- 172.21.15.151:0/611795343 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5be4152fd0 con 0x7f5be415c080 2024-09-16T15:44:38.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.201+0000 7f5bebc00640 1 -- 172.21.15.151:0/611795343 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5be415e980 con 0x7f5be415c080 2024-09-16T15:44:38.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.202+0000 7f5be1600640 1 -- 172.21.15.151:0/611795343 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 102103+0+0 (secure 0 0 0) 0x7f5bcc007260 con 0x7f5be415c080 2024-09-16T15:44:38.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.203+0000 7f5be1600640 1 --2- 172.21.15.151:0/611795343 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5bc0079180 0x7f5bc007b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:38.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.203+0000 7f5be1600640 1 -- 172.21.15.151:0/611795343 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4971+0+0 (secure 0 0 0) 0x7f5bcc0991a0 con 0x7f5be415c080 2024-09-16T15:44:38.205 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.204+0000 7f5be9200640 1 --2- 172.21.15.151:0/611795343 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5bc0079180 0x7f5bc007b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:38.205 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.204+0000 7f5be9200640 1 --2- 172.21.15.151:0/611795343 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5bc0079180 0x7f5bc007b640 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f5bd4034000 tx=0x7f5bd4004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:38.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.207+0000 7f5be1600640 1 -- 172.21.15.151:0/611795343 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f5be41f9190 con 0x7f5be415c080 2024-09-16T15:44:38.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.393+0000 7f5bebc00640 1 -- 172.21.15.151:0/611795343 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5be413c680 con 0x7f5be415c080 2024-09-16T15:44:38.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.394+0000 7f5be1600640 1 -- 172.21.15.151:0/611795343 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v54) ==== 93+0+10 (secure 0 0 0) 0x7f5be415b320 con 0x7f5be415c080 2024-09-16T15:44:38.395 INFO:teuthology.orchestra.run.smithi151.stdout:pg_num: 1 2024-09-16T15:44:38.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.397+0000 7f5bebc00640 1 -- 172.21.15.151:0/611795343 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5bc0079180 msgr2=0x7f5bc007b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:38.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.397+0000 7f5bebc00640 1 --2- 172.21.15.151:0/611795343 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5bc0079180 0x7f5bc007b640 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f5bd4034000 tx=0x7f5bd4004730 comp rx=0 tx=0).stop 2024-09-16T15:44:38.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.397+0000 7f5bebc00640 1 -- 172.21.15.151:0/611795343 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5be415c080 msgr2=0x7f5be413b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:38.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.397+0000 7f5bebc00640 1 --2- 172.21.15.151:0/611795343 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5be415c080 0x7f5be413b1f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc00c730 tx=0x7f5bcc00cbf0 comp rx=0 tx=0).stop 2024-09-16T15:44:38.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.397+0000 7f5bebc00640 1 -- 172.21.15.151:0/611795343 shutdown_connections 2024-09-16T15:44:38.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.397+0000 7f5bebc00640 1 --2- 172.21.15.151:0/611795343 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5bc0079180 0x7f5bc007b640 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:38.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.397+0000 7f5bebc00640 1 --2- 172.21.15.151:0/611795343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5be415f090 0x7f5be4159df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:38.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.397+0000 7f5bebc00640 1 --2- 172.21.15.151:0/611795343 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5be415c080 0x7f5be413b1f0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:38.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.397+0000 7f5bebc00640 1 --2- 172.21.15.151:0/611795343 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5be40b3370 0x7f5be41f8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:38.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.397+0000 7f5bebc00640 1 -- 172.21.15.151:0/611795343 >> 172.21.15.151:0/611795343 conn(0x7f5be40b1840 msgr2=0x7f5be413ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:38.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.397+0000 7f5bebc00640 1 -- 172.21.15.151:0/611795343 shutdown_connections 2024-09-16T15:44:38.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.397+0000 7f5bebc00640 1 -- 172.21.15.151:0/611795343 wait complete. 2024-09-16T15:44:38.414 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_less_than_cloner_threads' 2024-09-16T15:44:38.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.571+0000 7ff112600640 1 -- 172.21.15.151:0/573881619 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff10c0ccf00 msgr2=0x7ff10c0acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:38.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.571+0000 7ff112600640 1 --2- 172.21.15.151:0/573881619 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff10c0ccf00 0x7ff10c0acd30 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7ff10c075130 tx=0x7ff0f802a4a0 comp rx=0 tx=0).stop 2024-09-16T15:44:38.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.572+0000 7ff112600640 1 -- 172.21.15.151:0/573881619 shutdown_connections 2024-09-16T15:44:38.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.572+0000 7ff112600640 1 --2- 172.21.15.151:0/573881619 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff10c0ccf00 0x7ff10c0acd30 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:38.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.572+0000 7ff112600640 1 --2- 172.21.15.151:0/573881619 >> v2:172.21.15.167:3300/0 conn(0x7ff10c15f5a0 0x7ff10c0b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:38.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.572+0000 7ff112600640 1 -- 172.21.15.151:0/573881619 >> 172.21.15.151:0/573881619 conn(0x7ff10c0b1950 msgr2=0x7ff10c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:38.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.572+0000 7ff112600640 1 -- 172.21.15.151:0/573881619 shutdown_connections 2024-09-16T15:44:38.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.572+0000 7ff112600640 1 -- 172.21.15.151:0/573881619 wait complete. 2024-09-16T15:44:38.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.573+0000 7ff112600640 1 Processor -- start 2024-09-16T15:44:38.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.573+0000 7ff112600640 1 -- start start 2024-09-16T15:44:38.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.574+0000 7ff112600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff10c0ad270 0x7ff10c14b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:38.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.574+0000 7ff112600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff10c0ccf00 0x7ff10c15ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:38.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.574+0000 7ff112600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff10c15f5a0 0x7ff10c15bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:38.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.574+0000 7ff112600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff10c06f160 con 0x7ff10c0ad270 2024-09-16T15:44:38.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.574+0000 7ff112600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff10c06f2e0 con 0x7ff10c15f5a0 2024-09-16T15:44:38.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.574+0000 7ff112600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7ff10c06f460 con 0x7ff10c0ccf00 2024-09-16T15:44:38.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.575+0000 7ff10b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff10c0ad270 0x7ff10c14b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:38.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.575+0000 7ff10b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff10c0ad270 0x7ff10c14b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:34032/0 (socket says 172.21.15.151:34032) 2024-09-16T15:44:38.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.575+0000 7ff10b400640 1 -- 172.21.15.151:0/1483306839 learned_addr learned my addr 172.21.15.151:0/1483306839 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:38.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.575+0000 7ff10b400640 1 -- 172.21.15.151:0/1483306839 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff10c0ccf00 msgr2=0x7ff10c15ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:38.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.575+0000 7ff10b400640 1 --2- 172.21.15.151:0/1483306839 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff10c0ccf00 0x7ff10c15ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:38.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.575+0000 7ff10b400640 1 -- 172.21.15.151:0/1483306839 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff10c15f5a0 msgr2=0x7ff10c15bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:38.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.575+0000 7ff10b400640 1 --2- 172.21.15.151:0/1483306839 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff10c15f5a0 0x7ff10c15bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:38.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.575+0000 7ff10b400640 1 -- 172.21.15.151:0/1483306839 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff10c14baf0 con 0x7ff10c0ad270 2024-09-16T15:44:38.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.575+0000 7ff10b400640 1 --2- 172.21.15.151:0/1483306839 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff10c0ad270 0x7ff10c14b5b0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc00bfa0 tx=0x7ff0fc00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:38.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.575+0000 7ff103400640 1 -- 172.21.15.151:0/1483306839 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc018070 con 0x7ff10c0ad270 2024-09-16T15:44:38.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.576+0000 7ff103400640 1 -- 172.21.15.151:0/1483306839 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff0fc009cf0 con 0x7ff10c0ad270 2024-09-16T15:44:38.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.576+0000 7ff103400640 1 -- 172.21.15.151:0/1483306839 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc004280 con 0x7ff10c0ad270 2024-09-16T15:44:38.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.576+0000 7ff112600640 1 -- 172.21.15.151:0/1483306839 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff10c15a200 con 0x7ff10c0ad270 2024-09-16T15:44:38.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.576+0000 7ff112600640 1 -- 172.21.15.151:0/1483306839 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff10c1554d0 con 0x7ff10c0ad270 2024-09-16T15:44:38.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.577+0000 7ff112600640 1 -- 172.21.15.151:0/1483306839 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff10c155900 con 0x7ff10c0ad270 2024-09-16T15:44:38.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.578+0000 7ff103400640 1 -- 172.21.15.151:0/1483306839 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 102103+0+0 (secure 0 0 0) 0x7ff0fc005020 con 0x7ff10c0ad270 2024-09-16T15:44:38.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.579+0000 7ff103400640 1 --2- 172.21.15.151:0/1483306839 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff0ec079150 0x7ff0ec07b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:38.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.579+0000 7ff103400640 1 -- 172.21.15.151:0/1483306839 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4971+0+0 (secure 0 0 0) 0x7ff0fc09d0b0 con 0x7ff10c0ad270 2024-09-16T15:44:38.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.580+0000 7ff10aa00640 1 --2- 172.21.15.151:0/1483306839 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff0ec079150 0x7ff0ec07b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:38.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.580+0000 7ff10aa00640 1 --2- 172.21.15.151:0/1483306839 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff0ec079150 0x7ff0ec07b610 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8034000 tx=0x7ff0f80046c0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:38.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.583+0000 7ff103400640 1 -- 172.21.15.151:0/1483306839 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff0fc0670e0 con 0x7ff10c0ad270 2024-09-16T15:44:38.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:38.838+0000 7ff112600640 1 -- 172.21.15.151:0/1483306839 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_less_than_cloner_threads"]} v 0) -- 0x7ff10c06f9f0 con 0x7ff10c0ad270 2024-09-16T15:44:39.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:39.646+0000 7ff103400640 1 -- 172.21.15.151:0/1483306839 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_less_than_cloner_threads"]}]=0 v242) ==== 170+0+0 (secure 0 0 0) 0x7ff0fc0669e0 con 0x7ff10c0ad270 2024-09-16T15:44:39.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:39.649+0000 7ff112600640 1 -- 172.21.15.151:0/1483306839 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff0ec079150 msgr2=0x7ff0ec07b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:39.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:39.649+0000 7ff112600640 1 --2- 172.21.15.151:0/1483306839 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff0ec079150 0x7ff0ec07b610 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8034000 tx=0x7ff0f80046c0 comp rx=0 tx=0).stop 2024-09-16T15:44:39.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:39.649+0000 7ff112600640 1 -- 172.21.15.151:0/1483306839 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff10c0ad270 msgr2=0x7ff10c14b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:39.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:39.649+0000 7ff112600640 1 --2- 172.21.15.151:0/1483306839 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff10c0ad270 0x7ff10c14b5b0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc00bfa0 tx=0x7ff0fc00dbe0 comp rx=0 tx=0).stop 2024-09-16T15:44:39.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:39.649+0000 7ff112600640 1 -- 172.21.15.151:0/1483306839 shutdown_connections 2024-09-16T15:44:39.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:39.649+0000 7ff112600640 1 --2- 172.21.15.151:0/1483306839 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff0ec079150 0x7ff0ec07b610 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:39.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:39.649+0000 7ff112600640 1 --2- 172.21.15.151:0/1483306839 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff10c15f5a0 0x7ff10c15bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:39.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:39.649+0000 7ff112600640 1 --2- 172.21.15.151:0/1483306839 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff10c0ccf00 0x7ff10c15ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:39.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:39.649+0000 7ff112600640 1 --2- 172.21.15.151:0/1483306839 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff10c0ad270 0x7ff10c14b5b0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:39.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:39.649+0000 7ff112600640 1 -- 172.21.15.151:0/1483306839 >> 172.21.15.151:0/1483306839 conn(0x7ff10c0b1950 msgr2=0x7ff10c0af5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:39.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:39.649+0000 7ff112600640 1 -- 172.21.15.151:0/1483306839 shutdown_connections 2024-09-16T15:44:39.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:39.649+0000 7ff112600640 1 -- 172.21.15.151:0/1483306839 wait complete. 2024-09-16T15:44:39.670 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-16T15:44:39.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:39.829+0000 7fbe75200640 1 -- 172.21.15.151:0/2300818450 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbe70073fb0 msgr2=0x7fbe70074420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:39.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:39.829+0000 7fbe75200640 1 --2- 172.21.15.151:0/2300818450 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbe70073fb0 0x7fbe70074420 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fbe700cfbb0 tx=0x7fbe6802aa30 comp rx=0 tx=0).stop 2024-09-16T15:44:39.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:39.829+0000 7fbe75200640 1 -- 172.21.15.151:0/2300818450 shutdown_connections 2024-09-16T15:44:39.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:39.829+0000 7fbe75200640 1 --2- 172.21.15.151:0/2300818450 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbe70073fb0 0x7fbe70074420 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:39.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:39.829+0000 7fbe75200640 1 --2- 172.21.15.151:0/2300818450 >> v2:172.21.15.167:3300/0 conn(0x7fbe700ccf00 0x7fbe700b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:39.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:39.829+0000 7fbe75200640 1 -- 172.21.15.151:0/2300818450 >> 172.21.15.151:0/2300818450 conn(0x7fbe700b1950 msgr2=0x7fbe700b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:39.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:39.829+0000 7fbe75200640 1 -- 172.21.15.151:0/2300818450 shutdown_connections 2024-09-16T15:44:39.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:39.829+0000 7fbe75200640 1 -- 172.21.15.151:0/2300818450 wait complete. 2024-09-16T15:44:39.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:39.831+0000 7fbe75200640 1 Processor -- start 2024-09-16T15:44:39.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:39.831+0000 7fbe75200640 1 -- start start 2024-09-16T15:44:39.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:39.832+0000 7fbe75200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbe70073fb0 0x7fbe7014e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:39.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:39.832+0000 7fbe75200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbe700ccf00 0x7fbe7014dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:39.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:39.832+0000 7fbe75200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe7015f5a0 0x7fbe700ceb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:39.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:39.832+0000 7fbe75200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbe7006f2e0 con 0x7fbe70073fb0 2024-09-16T15:44:39.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:39.832+0000 7fbe75200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbe7006f160 con 0x7fbe7015f5a0 2024-09-16T15:44:39.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:39.832+0000 7fbe75200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fbe7006f460 con 0x7fbe700ccf00 2024-09-16T15:44:39.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:39.832+0000 7fbe6de00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbe70073fb0 0x7fbe7014e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:39.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:39.832+0000 7fbe6de00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbe70073fb0 0x7fbe7014e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:34052/0 (socket says 172.21.15.151:34052) 2024-09-16T15:44:39.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:39.832+0000 7fbe6de00640 1 -- 172.21.15.151:0/1912220628 learned_addr learned my addr 172.21.15.151:0/1912220628 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:39.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:39.833+0000 7fbe6de00640 1 -- 172.21.15.151:0/1912220628 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbe700ccf00 msgr2=0x7fbe7014dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:39.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:39.833+0000 7fbe6d400640 1 --2- 172.21.15.151:0/1912220628 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbe700ccf00 0x7fbe7014dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:39.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:39.833+0000 7fbe6e800640 1 --2- 172.21.15.151:0/1912220628 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe7015f5a0 0x7fbe700ceb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:39.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:39.833+0000 7fbe6de00640 1 --2- 172.21.15.151:0/1912220628 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbe700ccf00 0x7fbe7014dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:39.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:39.833+0000 7fbe6de00640 1 -- 172.21.15.151:0/1912220628 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe7015f5a0 msgr2=0x7fbe700ceb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:39.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:39.833+0000 7fbe6de00640 1 --2- 172.21.15.151:0/1912220628 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe7015f5a0 0x7fbe700ceb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:39.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:39.833+0000 7fbe6de00640 1 -- 172.21.15.151:0/1912220628 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe701f5fd0 con 0x7fbe70073fb0 2024-09-16T15:44:39.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:39.833+0000 7fbe6d400640 1 --2- 172.21.15.151:0/1912220628 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbe700ccf00 0x7fbe7014dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:44:39.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:39.833+0000 7fbe6e800640 1 --2- 172.21.15.151:0/1912220628 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe7015f5a0 0x7fbe700ceb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:44:39.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:39.833+0000 7fbe6de00640 1 --2- 172.21.15.151:0/1912220628 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbe70073fb0 0x7fbe7014e080 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fbe6000c3d0 tx=0x7fbe6000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:39.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:39.833+0000 7fbe66000640 1 -- 172.21.15.151:0/1912220628 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe60017070 con 0x7fbe70073fb0 2024-09-16T15:44:39.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:39.833+0000 7fbe66000640 1 -- 172.21.15.151:0/1912220628 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbe600053c0 con 0x7fbe70073fb0 2024-09-16T15:44:39.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:39.833+0000 7fbe66000640 1 -- 172.21.15.151:0/1912220628 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe60003850 con 0x7fbe70073fb0 2024-09-16T15:44:39.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:39.834+0000 7fbe75200640 1 -- 172.21.15.151:0/1912220628 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe7014ed70 con 0x7fbe70073fb0 2024-09-16T15:44:39.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:39.834+0000 7fbe75200640 1 -- 172.21.15.151:0/1912220628 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe700d0780 con 0x7fbe70073fb0 2024-09-16T15:44:39.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:39.835+0000 7fbe75200640 1 -- 172.21.15.151:0/1912220628 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe7006f9f0 con 0x7fbe70073fb0 2024-09-16T15:44:39.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:39.836+0000 7fbe66000640 1 -- 172.21.15.151:0/1912220628 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 102103+0+0 (secure 0 0 0) 0x7fbe6001a070 con 0x7fbe70073fb0 2024-09-16T15:44:39.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:39.837+0000 7fbe66000640 1 --2- 172.21.15.151:0/1912220628 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbe50079150 0x7fbe5007b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:39.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:39.838+0000 7fbe66000640 1 -- 172.21.15.151:0/1912220628 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4971+0+0 (secure 0 0 0) 0x7fbe60098520 con 0x7fbe70073fb0 2024-09-16T15:44:39.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:39.838+0000 7fbe6d400640 1 --2- 172.21.15.151:0/1912220628 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbe50079150 0x7fbe5007b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:39.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:39.838+0000 7fbe6d400640 1 --2- 172.21.15.151:0/1912220628 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbe50079150 0x7fbe5007b610 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fbe7014f070 tx=0x7fbe5c005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:39.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:39.842+0000 7fbe66000640 1 -- 172.21.15.151:0/1912220628 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fbe600625e0 con 0x7fbe70073fb0 2024-09-16T15:44:40.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.028+0000 7fbe75200640 1 -- 172.21.15.151:0/1912220628 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fbe701f61a0 con 0x7fbe70073fb0 2024-09-16T15:44:40.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.029+0000 7fbe66000640 1 -- 172.21.15.151:0/1912220628 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v33) ==== 59+0+1998 (secure 0 0 0) 0x7fbe60061ee0 con 0x7fbe70073fb0 2024-09-16T15:44:40.030 INFO:teuthology.orchestra.run.smithi151.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-16T15:44:40.030 INFO:teuthology.orchestra.run.smithi151.stdout:mon advanced mon_op_complaint_time 120 2024-09-16T15:44:40.031 INFO:teuthology.orchestra.run.smithi151.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-16T15:44:40.031 INFO:teuthology.orchestra.run.smithi151.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-16T15:44:40.031 INFO:teuthology.orchestra.run.smithi151.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3795.338492 2024-09-16T15:44:40.031 INFO:teuthology.orchestra.run.smithi151.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 4030.355715 2024-09-16T15:44:40.031 INFO:teuthology.orchestra.run.smithi151.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3817.533640 2024-09-16T15:44:40.031 INFO:teuthology.orchestra.run.smithi151.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3950.656136 2024-09-16T15:44:40.031 INFO:teuthology.orchestra.run.smithi151.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4449.408052 2024-09-16T15:44:40.031 INFO:teuthology.orchestra.run.smithi151.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4439.932296 2024-09-16T15:44:40.031 INFO:teuthology.orchestra.run.smithi151.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4566.875579 2024-09-16T15:44:40.031 INFO:teuthology.orchestra.run.smithi151.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 6320.073632 2024-09-16T15:44:40.031 INFO:teuthology.orchestra.run.smithi151.stdout:mds advanced debug_mds 20/20 2024-09-16T15:44:40.031 INFO:teuthology.orchestra.run.smithi151.stdout:mds advanced debug_mds_balancer 20/20 2024-09-16T15:44:40.031 INFO:teuthology.orchestra.run.smithi151.stdout:mds advanced debug_ms 2/2 2024-09-16T15:44:40.032 INFO:teuthology.orchestra.run.smithi151.stdout:mds dev mds_debug_frag true 2024-09-16T15:44:40.032 INFO:teuthology.orchestra.run.smithi151.stdout:mds dev mds_debug_scatterstat true 2024-09-16T15:44:40.032 INFO:teuthology.orchestra.run.smithi151.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-16T15:44:40.032 INFO:teuthology.orchestra.run.smithi151.stdout:mds dev mds_verify_scatter true 2024-09-16T15:44:40.032 INFO:teuthology.orchestra.run.smithi151.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-16T15:44:40.032 INFO:teuthology.orchestra.run.smithi151.stdout:mds advanced rados_mon_op_timeout 900 2024-09-16T15:44:40.032 INFO:teuthology.orchestra.run.smithi151.stdout:mds advanced rados_osd_op_timeout 900 2024-09-16T15:44:40.032 INFO:teuthology.orchestra.run.smithi151.stdout:client advanced client_mount_timeout 600 2024-09-16T15:44:40.032 INFO:teuthology.orchestra.run.smithi151.stdout:client advanced debug_client 20/20 2024-09-16T15:44:40.032 INFO:teuthology.orchestra.run.smithi151.stdout:client advanced debug_ms 1/1 2024-09-16T15:44:40.032 INFO:teuthology.orchestra.run.smithi151.stdout:client advanced rados_mon_op_timeout 900 2024-09-16T15:44:40.032 INFO:teuthology.orchestra.run.smithi151.stdout:client advanced rados_osd_op_timeout 900 2024-09-16T15:44:40.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.031+0000 7fbe75200640 1 -- 172.21.15.151:0/1912220628 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbe50079150 msgr2=0x7fbe5007b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:40.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.031+0000 7fbe75200640 1 --2- 172.21.15.151:0/1912220628 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbe50079150 0x7fbe5007b610 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fbe7014f070 tx=0x7fbe5c005f90 comp rx=0 tx=0).stop 2024-09-16T15:44:40.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.031+0000 7fbe75200640 1 -- 172.21.15.151:0/1912220628 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbe70073fb0 msgr2=0x7fbe7014e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:40.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.031+0000 7fbe75200640 1 --2- 172.21.15.151:0/1912220628 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbe70073fb0 0x7fbe7014e080 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fbe6000c3d0 tx=0x7fbe6000c890 comp rx=0 tx=0).stop 2024-09-16T15:44:40.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.032+0000 7fbe75200640 1 -- 172.21.15.151:0/1912220628 shutdown_connections 2024-09-16T15:44:40.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.032+0000 7fbe75200640 1 --2- 172.21.15.151:0/1912220628 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbe50079150 0x7fbe5007b610 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:40.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.032+0000 7fbe75200640 1 --2- 172.21.15.151:0/1912220628 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe7015f5a0 0x7fbe700ceb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:40.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.032+0000 7fbe75200640 1 --2- 172.21.15.151:0/1912220628 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbe700ccf00 0x7fbe7014dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:40.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.032+0000 7fbe75200640 1 --2- 172.21.15.151:0/1912220628 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbe70073fb0 0x7fbe7014e080 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:40.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.032+0000 7fbe75200640 1 -- 172.21.15.151:0/1912220628 >> 172.21.15.151:0/1912220628 conn(0x7fbe700b1950 msgr2=0x7fbe7015e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:40.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.032+0000 7fbe75200640 1 -- 172.21.15.151:0/1912220628 shutdown_connections 2024-09-16T15:44:40.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.032+0000 7fbe75200640 1 -- 172.21.15.151:0/1912220628 wait complete. 2024-09-16T15:44:40.049 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-16T15:44:40.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.209+0000 7f475ac00640 1 -- 172.21.15.151:0/4157611879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f475415d970 msgr2=0x7f475415de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:40.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.209+0000 7f475ac00640 1 --2- 172.21.15.151:0/4157611879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f475415d970 0x7f475415de00 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f474c005fd0 tx=0x7f474c02a8b0 comp rx=0 tx=0).stop 2024-09-16T15:44:40.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.209+0000 7f475ac00640 1 -- 172.21.15.151:0/4157611879 shutdown_connections 2024-09-16T15:44:40.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.209+0000 7f475ac00640 1 --2- 172.21.15.151:0/4157611879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f475415d970 0x7f475415de00 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:40.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.209+0000 7f475ac00640 1 --2- 172.21.15.151:0/4157611879 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f47540b1880 0x7f47540688b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:40.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.209+0000 7f475ac00640 1 -- 172.21.15.151:0/4157611879 >> 172.21.15.151:0/4157611879 conn(0x7f47540b5680 msgr2=0x7f47540b5a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:40.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.209+0000 7f475ac00640 1 -- 172.21.15.151:0/4157611879 shutdown_connections 2024-09-16T15:44:40.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.209+0000 7f475ac00640 1 -- 172.21.15.151:0/4157611879 wait complete. 2024-09-16T15:44:40.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.210+0000 7f475ac00640 1 Processor -- start 2024-09-16T15:44:40.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.211+0000 7f475ac00640 1 -- start start 2024-09-16T15:44:40.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.211+0000 7f475ac00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47540b1880 0x7f47540cfc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:40.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.212+0000 7f475ac00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f475415d970 0x7f47541f47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:40.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.212+0000 7f475ac00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f475415f790 0x7f47540cec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:40.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.212+0000 7f475ac00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f475415e4c0 con 0x7f475415f790 2024-09-16T15:44:40.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.212+0000 7f475ac00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f475415e340 con 0x7f47540b1880 2024-09-16T15:44:40.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.212+0000 7f475ac00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f475415e640 con 0x7f475415d970 2024-09-16T15:44:40.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.212+0000 7f475a200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f475415f790 0x7f47540cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:40.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.212+0000 7f475a200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f475415f790 0x7f47540cec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:34064/0 (socket says 172.21.15.151:34064) 2024-09-16T15:44:40.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.212+0000 7f475a200640 1 -- 172.21.15.151:0/2896633162 learned_addr learned my addr 172.21.15.151:0/2896633162 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:40.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.212+0000 7f4758e00640 1 --2- 172.21.15.151:0/2896633162 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f475415d970 0x7f47541f47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:40.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.212+0000 7f4759800640 1 --2- 172.21.15.151:0/2896633162 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47540b1880 0x7f47540cfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:40.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.212+0000 7f475a200640 1 -- 172.21.15.151:0/2896633162 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f475415d970 msgr2=0x7f47541f47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:40.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.212+0000 7f475a200640 1 --2- 172.21.15.151:0/2896633162 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f475415d970 0x7f47541f47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:40.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.212+0000 7f475a200640 1 -- 172.21.15.151:0/2896633162 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47540b1880 msgr2=0x7f47540cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:40.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.212+0000 7f475a200640 1 --2- 172.21.15.151:0/2896633162 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47540b1880 0x7f47540cfc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:40.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.212+0000 7f475a200640 1 -- 172.21.15.151:0/2896633162 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47541f5190 con 0x7f475415f790 2024-09-16T15:44:40.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.213+0000 7f475a200640 1 --2- 172.21.15.151:0/2896633162 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f475415f790 0x7f47540cec50 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f474c009cd0 tx=0x7f474c0316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:40.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.213+0000 7f4751600640 1 -- 172.21.15.151:0/2896633162 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474c040070 con 0x7f475415f790 2024-09-16T15:44:40.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.213+0000 7f4751600640 1 -- 172.21.15.151:0/2896633162 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f474c004040 con 0x7f475415f790 2024-09-16T15:44:40.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.213+0000 7f475ac00640 1 -- 172.21.15.151:0/2896633162 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47541f5470 con 0x7f475415f790 2024-09-16T15:44:40.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.213+0000 7f4751600640 1 -- 172.21.15.151:0/2896633162 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474c004470 con 0x7f475415f790 2024-09-16T15:44:40.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.213+0000 7f475ac00640 1 -- 172.21.15.151:0/2896633162 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47540ccba0 con 0x7f475415f790 2024-09-16T15:44:40.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.214+0000 7f475ac00640 1 -- 172.21.15.151:0/2896633162 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47540ccda0 con 0x7f475415f790 2024-09-16T15:44:40.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.215+0000 7f4751600640 1 -- 172.21.15.151:0/2896633162 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 102103+0+0 (secure 0 0 0) 0x7f474c039040 con 0x7f475415f790 2024-09-16T15:44:40.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.216+0000 7f4751600640 1 --2- 172.21.15.151:0/2896633162 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4734079150 0x7f473407b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:40.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.216+0000 7f4751600640 1 -- 172.21.15.151:0/2896633162 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4971+0+0 (secure 0 0 0) 0x7f474c0ba540 con 0x7f475415f790 2024-09-16T15:44:40.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.217+0000 7f4759800640 1 --2- 172.21.15.151:0/2896633162 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4734079150 0x7f473407b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:40.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.217+0000 7f4759800640 1 --2- 172.21.15.151:0/2896633162 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4734079150 0x7f473407b610 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4748009bd0 tx=0x7f4748005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:40.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.220+0000 7f4751600640 1 -- 172.21.15.151:0/2896633162 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f474c083fb0 con 0x7f475415f790 2024-09-16T15:44:40.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.404+0000 7f475ac00640 1 -- 172.21.15.151:0/2896633162 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f475415eaa0 con 0x7f475415f790 2024-09-16T15:44:40.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.408+0000 7f4751600640 1 -- 172.21.15.151:0/2896633162 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v34)=0 v34) ==== 122+0+0 (secure 0 0 0) 0x7f474c0889e0 con 0x7f475415f790 2024-09-16T15:44:40.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.411+0000 7f475ac00640 1 -- 172.21.15.151:0/2896633162 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4734079150 msgr2=0x7f473407b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:40.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.411+0000 7f475ac00640 1 --2- 172.21.15.151:0/2896633162 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4734079150 0x7f473407b610 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4748009bd0 tx=0x7f4748005f90 comp rx=0 tx=0).stop 2024-09-16T15:44:40.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.411+0000 7f475ac00640 1 -- 172.21.15.151:0/2896633162 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f475415f790 msgr2=0x7f47540cec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:40.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.411+0000 7f475ac00640 1 --2- 172.21.15.151:0/2896633162 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f475415f790 0x7f47540cec50 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f474c009cd0 tx=0x7f474c0316f0 comp rx=0 tx=0).stop 2024-09-16T15:44:40.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.411+0000 7f475ac00640 1 -- 172.21.15.151:0/2896633162 shutdown_connections 2024-09-16T15:44:40.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.411+0000 7f475ac00640 1 --2- 172.21.15.151:0/2896633162 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4734079150 0x7f473407b610 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:40.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.411+0000 7f475ac00640 1 --2- 172.21.15.151:0/2896633162 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f475415f790 0x7f47540cec50 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:40.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.411+0000 7f475ac00640 1 --2- 172.21.15.151:0/2896633162 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f475415d970 0x7f47541f47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:40.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.411+0000 7f475ac00640 1 --2- 172.21.15.151:0/2896633162 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47540b1880 0x7f47540cfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:40.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.411+0000 7f475ac00640 1 -- 172.21.15.151:0/2896633162 >> 172.21.15.151:0/2896633162 conn(0x7f47540b5680 msgr2=0x7f47540d0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:40.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.411+0000 7f475ac00640 1 -- 172.21.15.151:0/2896633162 shutdown_connections 2024-09-16T15:44:40.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.411+0000 7f475ac00640 1 -- 172.21.15.151:0/2896633162 wait complete. 2024-09-16T15:44:40.428 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-16T15:44:40.428 DEBUG:teuthology.orchestra.run.smithi151:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:44:40.507 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:44:40.613 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-16T15:44:40.613 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:44:40.614 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:44:40.649 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T15:44:40.650 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-16T15:44:40.650 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:44:40.687 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:44:40.687 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:44:40.687 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:44:40.725 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T15:44:40.725 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-16T15:44:40.726 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:44:40.764 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:44:40.764 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-16T15:44:40.764 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:44:40.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.925+0000 7f3191000640 1 -- 172.21.15.151:0/114930093 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f318c0ccf00 msgr2=0x7f318c0b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:40.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.925+0000 7f3191000640 1 --2- 172.21.15.151:0/114930093 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f318c0ccf00 0x7f318c0b2440 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3178005fd0 tx=0x7f317802a540 comp rx=0 tx=0).stop 2024-09-16T15:44:40.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.925+0000 7f3191000640 1 -- 172.21.15.151:0/114930093 shutdown_connections 2024-09-16T15:44:40.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.925+0000 7f3191000640 1 --2- 172.21.15.151:0/114930093 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f318c073fb0 0x7f318c074440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:40.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.925+0000 7f3191000640 1 --2- 172.21.15.151:0/114930093 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f318c0ccf00 0x7f318c0b2440 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:40.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.925+0000 7f3191000640 1 -- 172.21.15.151:0/114930093 >> 172.21.15.151:0/114930093 conn(0x7f318c0b1950 msgr2=0x7f318c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:40.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.925+0000 7f3191000640 1 -- 172.21.15.151:0/114930093 shutdown_connections 2024-09-16T15:44:40.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.925+0000 7f3191000640 1 -- 172.21.15.151:0/114930093 wait complete. 2024-09-16T15:44:40.928 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.927+0000 7f3191000640 1 Processor -- start 2024-09-16T15:44:40.928 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.927+0000 7f3191000640 1 -- start start 2024-09-16T15:44:40.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.928+0000 7f3191000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f318c073fb0 0x7f318c1f6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:40.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.928+0000 7f3191000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f318c0ccf00 0x7f318c1462e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:40.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.928+0000 7f3191000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f318c15f5a0 0x7f318c147b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:40.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.928+0000 7f3191000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f318c06f2e0 con 0x7f318c0ccf00 2024-09-16T15:44:40.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.928+0000 7f3191000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f318c06f160 con 0x7f318c073fb0 2024-09-16T15:44:40.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.928+0000 7f3191000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f318c06f460 con 0x7f318c15f5a0 2024-09-16T15:44:40.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.928+0000 7f3189400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f318c0ccf00 0x7f318c1462e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:40.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.928+0000 7f3189400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f318c0ccf00 0x7f318c1462e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:34076/0 (socket says 172.21.15.151:34076) 2024-09-16T15:44:40.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.928+0000 7f3189400640 1 -- 172.21.15.151:0/848507111 learned_addr learned my addr 172.21.15.151:0/848507111 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:40.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.928+0000 7f3189e00640 1 --2- 172.21.15.151:0/848507111 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f318c073fb0 0x7f318c1f6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:40.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.928+0000 7f318a800640 1 --2- 172.21.15.151:0/848507111 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f318c15f5a0 0x7f318c147b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:40.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.929+0000 7f3189400640 1 -- 172.21.15.151:0/848507111 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f318c15f5a0 msgr2=0x7f318c147b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:40.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.929+0000 7f3189400640 1 --2- 172.21.15.151:0/848507111 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f318c15f5a0 0x7f318c147b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:40.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.929+0000 7f3189400640 1 -- 172.21.15.151:0/848507111 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f318c073fb0 msgr2=0x7f318c1f6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:40.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.929+0000 7f3189400640 1 --2- 172.21.15.151:0/848507111 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f318c073fb0 0x7f318c1f6ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:40.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.929+0000 7f3189400640 1 -- 172.21.15.151:0/848507111 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f318c146a80 con 0x7f318c0ccf00 2024-09-16T15:44:40.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.929+0000 7f3189400640 1 --2- 172.21.15.151:0/848507111 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f318c0ccf00 0x7f318c1462e0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f3178006010 tx=0x7f31780038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:40.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.929+0000 7f3182000640 1 -- 172.21.15.151:0/848507111 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f317803f070 con 0x7f318c0ccf00 2024-09-16T15:44:40.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.929+0000 7f3182000640 1 -- 172.21.15.151:0/848507111 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f31780047a0 con 0x7f318c0ccf00 2024-09-16T15:44:40.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.929+0000 7f3191000640 1 -- 172.21.15.151:0/848507111 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f318c146d60 con 0x7f318c0ccf00 2024-09-16T15:44:40.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.929+0000 7f3182000640 1 -- 172.21.15.151:0/848507111 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3178035030 con 0x7f318c0ccf00 2024-09-16T15:44:40.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.930+0000 7f3191000640 1 -- 172.21.15.151:0/848507111 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f318c147320 con 0x7f318c0ccf00 2024-09-16T15:44:40.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.931+0000 7f3191000640 1 -- 172.21.15.151:0/848507111 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f318c06f980 con 0x7f318c0ccf00 2024-09-16T15:44:40.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.932+0000 7f3182000640 1 -- 172.21.15.151:0/848507111 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 102103+0+0 (secure 0 0 0) 0x7f3178042070 con 0x7f318c0ccf00 2024-09-16T15:44:40.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.932+0000 7f3182000640 1 --2- 172.21.15.151:0/848507111 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f316c079150 0x7f316c07b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:40.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.933+0000 7f3182000640 1 -- 172.21.15.151:0/848507111 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4971+0+0 (secure 0 0 0) 0x7f31780b9fd0 con 0x7f318c0ccf00 2024-09-16T15:44:40.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.933+0000 7f3189e00640 1 --2- 172.21.15.151:0/848507111 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f316c079150 0x7f316c07b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:40.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.934+0000 7f3189e00640 1 --2- 172.21.15.151:0/848507111 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f316c079150 0x7f316c07b610 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f317c001720 tx=0x7f317c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:40.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:40.937+0000 7f3182000640 1 -- 172.21.15.151:0/848507111 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f3178083a40 con 0x7f318c0ccf00 2024-09-16T15:44:41.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.161+0000 7f3191000640 1 -- 172.21.15.151:0/848507111 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f318c09bbc0 con 0x7f318c0ccf00 2024-09-16T15:44:41.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.161+0000 7f3182000640 1 -- 172.21.15.151:0/848507111 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+3920 (secure 0 0 0) 0x7f3178088470 con 0x7f318c0ccf00 2024-09-16T15:44:41.162 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:44:41.163 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":30,"btime":"2024-09-16T15:44:34:618073+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5272,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1043845735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1043845735},{"type":"v1","addr":"172.21.15.167:6836","nonce":1043845735}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5275,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/816612106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":816612106},{"type":"v1","addr":"172.21.15.167:6837","nonce":816612106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5397,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/1986755100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1986755100},{"type":"v1","addr":"172.21.15.151:6834","nonce":1986755100}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5871,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/3675478090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":3675478090},{"type":"v1","addr":"172.21.15.151:6835","nonce":3675478090}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[]} 2024-09-16T15:44:41.163 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 30 2024-09-16T15:44:41.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.163+0000 7f3191000640 1 -- 172.21.15.151:0/848507111 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f316c079150 msgr2=0x7f316c07b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:41.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.163+0000 7f3191000640 1 --2- 172.21.15.151:0/848507111 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f316c079150 0x7f316c07b610 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f317c001720 tx=0x7f317c009040 comp rx=0 tx=0).stop 2024-09-16T15:44:41.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.163+0000 7f3191000640 1 -- 172.21.15.151:0/848507111 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f318c0ccf00 msgr2=0x7f318c1462e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:41.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.163+0000 7f3191000640 1 --2- 172.21.15.151:0/848507111 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f318c0ccf00 0x7f318c1462e0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f3178006010 tx=0x7f31780038a0 comp rx=0 tx=0).stop 2024-09-16T15:44:41.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.163+0000 7f3191000640 1 -- 172.21.15.151:0/848507111 shutdown_connections 2024-09-16T15:44:41.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.163+0000 7f3191000640 1 --2- 172.21.15.151:0/848507111 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f316c079150 0x7f316c07b610 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:41.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.163+0000 7f3191000640 1 --2- 172.21.15.151:0/848507111 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f318c15f5a0 0x7f318c147b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:41.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.163+0000 7f3191000640 1 --2- 172.21.15.151:0/848507111 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f318c0ccf00 0x7f318c1462e0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:41.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.163+0000 7f3191000640 1 --2- 172.21.15.151:0/848507111 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f318c073fb0 0x7f318c1f6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:41.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.164+0000 7f3191000640 1 -- 172.21.15.151:0/848507111 >> 172.21.15.151:0/848507111 conn(0x7f318c0b1950 msgr2=0x7f318c09c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:41.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.164+0000 7f3191000640 1 -- 172.21.15.151:0/848507111 shutdown_connections 2024-09-16T15:44:41.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.164+0000 7f3191000640 1 -- 172.21.15.151:0/848507111 wait complete. 2024-09-16T15:44:41.180 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-16T15:44:41.181 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-16T15:44:41.181 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-16T15:44:41.181 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:44:41.181 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-16T15:44:41.181 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-16T15:44:41.181 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-16T15:44:41.181 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:44:41.182 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-16T15:44:41.182 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-16T15:44:41.182 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-16T15:44:41.182 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:44:41.182 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-16T15:44:41.182 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-16T15:44:41.182 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-16T15:44:41.182 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:44:41.198 INFO:tasks.ceph.mds.b:Stopped 2024-09-16T15:44:41.198 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-16T15:44:41.199 INFO:tasks.ceph.mds.d:Stopped 2024-09-16T15:44:41.199 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-16T15:44:41.200 INFO:tasks.ceph.mds.a:Stopped 2024-09-16T15:44:41.200 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-16T15:44:41.201 INFO:tasks.ceph.mds.c:Stopped 2024-09-16T15:44:41.201 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-16T15:44:41.243 INFO:tasks.ceph.mds.b:Started 2024-09-16T15:44:41.243 DEBUG:teuthology.parallel:result is None 2024-09-16T15:44:41.244 INFO:tasks.ceph.mds.a:Started 2024-09-16T15:44:41.244 DEBUG:teuthology.parallel:result is None 2024-09-16T15:44:41.245 INFO:tasks.ceph.mds.d:Started 2024-09-16T15:44:41.245 DEBUG:teuthology.parallel:result is None 2024-09-16T15:44:41.246 INFO:tasks.ceph.mds.c:Started 2024-09-16T15:44:41.247 DEBUG:teuthology.parallel:result is None 2024-09-16T15:44:41.247 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-16T15:44:41.375 INFO:tasks.ceph.mds.a.smithi151.stdout:starting mds.a at 2024-09-16T15:44:41.375 INFO:tasks.ceph.mds.c.smithi151.stdout:starting mds.c at 2024-09-16T15:44:41.375 INFO:tasks.ceph.mds.b.smithi167.stdout:starting mds.b at 2024-09-16T15:44:41.375 INFO:tasks.ceph.mds.d.smithi167.stdout:starting mds.d at 2024-09-16T15:44:41.376 INFO:tasks.ceph.mds.d.smithi167.stderr:2024-09-16T15:44:41.376+0000 7f6a44000640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-09-16T15:44:41.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.412+0000 7f491a800640 1 -- 172.21.15.151:0/3399805024 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f49141438b0 msgr2=0x7f49140cc5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:41.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.412+0000 7f491a800640 1 --2- 172.21.15.151:0/3399805024 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f49141438b0 0x7f49140cc5f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f4904005fd0 tx=0x7f490402a530 comp rx=0 tx=0).stop 2024-09-16T15:44:41.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.412+0000 7f491a800640 1 -- 172.21.15.151:0/3399805024 shutdown_connections 2024-09-16T15:44:41.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.412+0000 7f491a800640 1 --2- 172.21.15.151:0/3399805024 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f491406ff60 0x7f49140b3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:41.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.412+0000 7f491a800640 1 --2- 172.21.15.151:0/3399805024 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f49141438b0 0x7f49140cc5f0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:41.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.412+0000 7f491a800640 1 -- 172.21.15.151:0/3399805024 >> 172.21.15.151:0/3399805024 conn(0x7f49140b1950 msgr2=0x7f49140b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:41.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.412+0000 7f491a800640 1 -- 172.21.15.151:0/3399805024 shutdown_connections 2024-09-16T15:44:41.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.413+0000 7f491a800640 1 -- 172.21.15.151:0/3399805024 wait complete. 2024-09-16T15:44:41.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.414+0000 7f491a800640 1 Processor -- start 2024-09-16T15:44:41.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.414+0000 7f491a800640 1 -- start start 2024-09-16T15:44:41.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.415+0000 7f491a800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f491406ff60 0x7f491413d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:41.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.415+0000 7f491a800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f49140ccb30 0x7f49140ced10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:41.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.415+0000 7f491a800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f49141438b0 0x7f49140cf250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:41.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.415+0000 7f491a800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4914070b30 con 0x7f49140ccb30 2024-09-16T15:44:41.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.415+0000 7f491a800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f491406efa0 con 0x7f491406ff60 2024-09-16T15:44:41.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.415+0000 7f491a800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f49140b4440 con 0x7f49141438b0 2024-09-16T15:44:41.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.415+0000 7f4912a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f49140ccb30 0x7f49140ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:41.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.415+0000 7f4913400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f491406ff60 0x7f491413d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:41.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.416+0000 7f4912a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f49140ccb30 0x7f49140ced10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:34112/0 (socket says 172.21.15.151:34112) 2024-09-16T15:44:41.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.416+0000 7f4912a00640 1 -- 172.21.15.151:0/532906940 learned_addr learned my addr 172.21.15.151:0/532906940 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:41.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.416+0000 7f4913e00640 1 --2- 172.21.15.151:0/532906940 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f49141438b0 0x7f49140cf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:41.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.416+0000 7f4912a00640 1 -- 172.21.15.151:0/532906940 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f49141438b0 msgr2=0x7f49140cf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:41.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.416+0000 7f4912a00640 1 --2- 172.21.15.151:0/532906940 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f49141438b0 0x7f49140cf250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:41.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.416+0000 7f4912a00640 1 -- 172.21.15.151:0/532906940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f491406ff60 msgr2=0x7f491413d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:41.419 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.416+0000 7f4912a00640 1 --2- 172.21.15.151:0/532906940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f491406ff60 0x7f491413d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:41.419 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.416+0000 7f4912a00640 1 -- 172.21.15.151:0/532906940 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4914074b20 con 0x7f49140ccb30 2024-09-16T15:44:41.419 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.416+0000 7f4913400640 1 --2- 172.21.15.151:0/532906940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f491406ff60 0x7f491413d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:44:41.419 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.416+0000 7f4912a00640 1 --2- 172.21.15.151:0/532906940 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f49140ccb30 0x7f49140ced10 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f490000c460 tx=0x7f490000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:41.419 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.416+0000 7f490b400640 1 -- 172.21.15.151:0/532906940 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4900016020 con 0x7f49140ccb30 2024-09-16T15:44:41.419 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.417+0000 7f490b400640 1 -- 172.21.15.151:0/532906940 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4900012960 con 0x7f49140ccb30 2024-09-16T15:44:41.419 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.417+0000 7f490b400640 1 -- 172.21.15.151:0/532906940 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4900012db0 con 0x7f49140ccb30 2024-09-16T15:44:41.419 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.417+0000 7f491a800640 1 -- 172.21.15.151:0/532906940 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49141f5e50 con 0x7f49140ccb30 2024-09-16T15:44:41.419 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.417+0000 7f491a800640 1 -- 172.21.15.151:0/532906940 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49141f60f0 con 0x7f49140ccb30 2024-09-16T15:44:41.419 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.418+0000 7f491a800640 1 -- 172.21.15.151:0/532906940 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49140b3f00 con 0x7f49140ccb30 2024-09-16T15:44:41.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.419+0000 7f490b400640 1 -- 172.21.15.151:0/532906940 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 102103+0+0 (secure 0 0 0) 0x7f4900019020 con 0x7f49140ccb30 2024-09-16T15:44:41.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.420+0000 7f490b400640 1 --2- 172.21.15.151:0/532906940 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f48f4079180 0x7f48f407b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:41.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.421+0000 7f490b400640 1 -- 172.21.15.151:0/532906940 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4971+0+0 (secure 0 0 0) 0x7f4900098b50 con 0x7f49140ccb30 2024-09-16T15:44:41.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.421+0000 7f4913400640 1 --2- 172.21.15.151:0/532906940 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f48f4079180 0x7f48f407b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:41.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.421+0000 7f4913400640 1 --2- 172.21.15.151:0/532906940 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f48f4079180 0x7f48f407b640 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4904033000 tx=0x7f4904004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:41.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.425+0000 7f490b400640 1 -- 172.21.15.151:0/532906940 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4900020070 con 0x7f49140ccb30 2024-09-16T15:44:41.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.622+0000 7f491a800640 1 -- 172.21.15.151:0/532906940 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f49140cd430 con 0x7f49140ccb30 2024-09-16T15:44:41.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.661+0000 7f490b400640 1 -- 172.21.15.151:0/532906940 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v56) ==== 97+0+0 (secure 0 0 0) 0x7f49000625c0 con 0x7f49140ccb30 2024-09-16T15:44:41.662 INFO:teuthology.orchestra.run.smithi151.stderr: removed all blocklist entries 2024-09-16T15:44:41.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.664+0000 7f491a800640 1 -- 172.21.15.151:0/532906940 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f48f4079180 msgr2=0x7f48f407b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:41.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.664+0000 7f491a800640 1 --2- 172.21.15.151:0/532906940 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f48f4079180 0x7f48f407b640 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4904033000 tx=0x7f4904004730 comp rx=0 tx=0).stop 2024-09-16T15:44:41.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.664+0000 7f491a800640 1 -- 172.21.15.151:0/532906940 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f49140ccb30 msgr2=0x7f49140ced10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:41.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.664+0000 7f491a800640 1 --2- 172.21.15.151:0/532906940 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f49140ccb30 0x7f49140ced10 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f490000c460 tx=0x7f490000c920 comp rx=0 tx=0).stop 2024-09-16T15:44:41.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.664+0000 7f491a800640 1 -- 172.21.15.151:0/532906940 shutdown_connections 2024-09-16T15:44:41.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.664+0000 7f491a800640 1 --2- 172.21.15.151:0/532906940 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f48f4079180 0x7f48f407b640 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:41.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.664+0000 7f491a800640 1 --2- 172.21.15.151:0/532906940 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f49141438b0 0x7f49140cf250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:41.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.664+0000 7f491a800640 1 --2- 172.21.15.151:0/532906940 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f49140ccb30 0x7f49140ced10 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:41.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.664+0000 7f491a800640 1 --2- 172.21.15.151:0/532906940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f491406ff60 0x7f491413d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:41.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.664+0000 7f491a800640 1 -- 172.21.15.151:0/532906940 >> 172.21.15.151:0/532906940 conn(0x7f49140b1950 msgr2=0x7f49140ad380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:41.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.664+0000 7f491a800640 1 -- 172.21.15.151:0/532906940 shutdown_connections 2024-09-16T15:44:41.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.664+0000 7f491a800640 1 -- 172.21.15.151:0/532906940 wait complete. 2024-09-16T15:44:41.682 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-16T15:44:41.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.840+0000 7f6e4f800640 1 -- 172.21.15.151:0/1818846486 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6e48161550 msgr2=0x7f6e4806ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:41.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.840+0000 7f6e4f800640 1 --2- 172.21.15.151:0/1818846486 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6e48161550 0x7f6e4806ef50 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f6e480cd5d0 tx=0x7f6e3c02a4a0 comp rx=0 tx=0).stop 2024-09-16T15:44:41.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.840+0000 7f6e4f800640 1 -- 172.21.15.151:0/1818846486 shutdown_connections 2024-09-16T15:44:41.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.840+0000 7f6e4f800640 1 --2- 172.21.15.151:0/1818846486 >> v2:172.21.15.167:3300/0 conn(0x7f6e48072df0 0x7f6e480732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:41.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.840+0000 7f6e4f800640 1 --2- 172.21.15.151:0/1818846486 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6e48161550 0x7f6e4806ef50 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:41.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.840+0000 7f6e4f800640 1 -- 172.21.15.151:0/1818846486 >> 172.21.15.151:0/1818846486 conn(0x7f6e480b1950 msgr2=0x7f6e480b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:41.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.840+0000 7f6e4f800640 1 -- 172.21.15.151:0/1818846486 shutdown_connections 2024-09-16T15:44:41.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.840+0000 7f6e4f800640 1 -- 172.21.15.151:0/1818846486 wait complete. 2024-09-16T15:44:41.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.842+0000 7f6e4f800640 1 Processor -- start 2024-09-16T15:44:41.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.842+0000 7f6e4f800640 1 -- start start 2024-09-16T15:44:41.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.843+0000 7f6e4f800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6e48072df0 0x7f6e4814c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:41.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.843+0000 7f6e4f800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e480af3d0 0x7f6e480ca9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:41.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.843+0000 7f6e4f800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e48161550 0x7f6e480d0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:41.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.843+0000 7f6e4f800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6e480737e0 con 0x7f6e48072df0 2024-09-16T15:44:41.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.843+0000 7f6e4f800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6e4806fdb0 con 0x7f6e48161550 2024-09-16T15:44:41.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.843+0000 7f6e4f800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f6e48073960 con 0x7f6e480af3d0 2024-09-16T15:44:41.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.843+0000 7f6e4ce00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6e48072df0 0x7f6e4814c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:41.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.843+0000 7f6e4ce00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6e48072df0 0x7f6e4814c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:34134/0 (socket says 172.21.15.151:34134) 2024-09-16T15:44:41.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.843+0000 7f6e4ce00640 1 -- 172.21.15.151:0/3266925584 learned_addr learned my addr 172.21.15.151:0/3266925584 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:41.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.843+0000 7f6e4d800640 1 --2- 172.21.15.151:0/3266925584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e48161550 0x7f6e480d0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:41.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.843+0000 7f6e47e00640 1 --2- 172.21.15.151:0/3266925584 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e480af3d0 0x7f6e480ca9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:41.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.843+0000 7f6e4ce00640 1 -- 172.21.15.151:0/3266925584 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e480af3d0 msgr2=0x7f6e480ca9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:41.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.843+0000 7f6e4ce00640 1 --2- 172.21.15.151:0/3266925584 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e480af3d0 0x7f6e480ca9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:41.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.843+0000 7f6e4ce00640 1 -- 172.21.15.151:0/3266925584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e48161550 msgr2=0x7f6e480d0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:41.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.843+0000 7f6e4ce00640 1 --2- 172.21.15.151:0/3266925584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e48161550 0x7f6e480d0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:41.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.843+0000 7f6e4ce00640 1 -- 172.21.15.151:0/3266925584 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e480c9290 con 0x7f6e48072df0 2024-09-16T15:44:41.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.843+0000 7f6e47e00640 1 --2- 172.21.15.151:0/3266925584 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e480af3d0 0x7f6e480ca9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:44:41.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.844+0000 7f6e4ce00640 1 --2- 172.21.15.151:0/3266925584 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6e48072df0 0x7f6e4814c730 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c02aa90 tx=0x7f6e3c033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:41.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.844+0000 7f6e44c00640 1 -- 172.21.15.151:0/3266925584 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c031d60 con 0x7f6e48072df0 2024-09-16T15:44:41.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.844+0000 7f6e44c00640 1 -- 172.21.15.151:0/3266925584 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6e3c033e10 con 0x7f6e48072df0 2024-09-16T15:44:41.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.844+0000 7f6e4f800640 1 -- 172.21.15.151:0/3266925584 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e4809d030 con 0x7f6e48072df0 2024-09-16T15:44:41.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.844+0000 7f6e44c00640 1 -- 172.21.15.151:0/3266925584 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c005260 con 0x7f6e48072df0 2024-09-16T15:44:41.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.844+0000 7f6e4f800640 1 -- 172.21.15.151:0/3266925584 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e4809d5f0 con 0x7f6e48072df0 2024-09-16T15:44:41.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.845+0000 7f6e4f800640 1 -- 172.21.15.151:0/3266925584 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e48073de0 con 0x7f6e48072df0 2024-09-16T15:44:41.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.846+0000 7f6e44c00640 1 -- 172.21.15.151:0/3266925584 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 102103+0+0 (secure 0 0 0) 0x7f6e3c005400 con 0x7f6e48072df0 2024-09-16T15:44:41.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.847+0000 7f6e44c00640 1 --2- 172.21.15.151:0/3266925584 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6e28079180 0x7f6e2807b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:41.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.847+0000 7f6e44c00640 1 -- 172.21.15.151:0/3266925584 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 4885+0+0 (secure 0 0 0) 0x7f6e3c0ba190 con 0x7f6e48072df0 2024-09-16T15:44:41.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.848+0000 7f6e47e00640 1 --2- 172.21.15.151:0/3266925584 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6e28079180 0x7f6e2807b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:41.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.848+0000 7f6e47e00640 1 --2- 172.21.15.151:0/3266925584 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6e28079180 0x7f6e2807b640 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6e481566b0 tx=0x7f6e34009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:41.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:41.851+0000 7f6e44c00640 1 -- 172.21.15.151:0/3266925584 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6e3c03f030 con 0x7f6e48072df0 2024-09-16T15:44:42.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.101+0000 7f6e4f800640 1 -- 172.21.15.151:0/3266925584 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f6e4815e670 con 0x7f6e48072df0 2024-09-16T15:44:42.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.102+0000 7f6e44c00640 1 -- 172.21.15.151:0/3266925584 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v10) ==== 80+0+5304 (secure 0 0 0) 0x7f6e3c031770 con 0x7f6e48072df0 2024-09-16T15:44:42.103 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:44:42.103 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-16T15:44:42.103 INFO:teuthology.orchestra.run.smithi151.stdout: "auth_dump": [ 2024-09-16T15:44:42.103 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:44:42.103 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "mds.a", 2024-09-16T15:44:42.103 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDsUOhmJVK6AhAANfmTPS6kKn41LSD/MuyW9g==", 2024-09-16T15:44:42.104 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:44:42.104 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:44:42.104 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow *", 2024-09-16T15:44:42.104 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow *", 2024-09-16T15:44:42.104 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:44:42.104 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:44:42.104 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:44:42.104 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:44:42.104 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "mds.b", 2024-09-16T15:44:42.104 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDsUOhmX1R3FhAAUQgUsE/6wUyokXcZuBuIWg==", 2024-09-16T15:44:42.104 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:44:42.104 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:44:42.104 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow *", 2024-09-16T15:44:42.104 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow *", 2024-09-16T15:44:42.104 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:44:42.105 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:44:42.105 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:44:42.105 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:44:42.105 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "mds.c", 2024-09-16T15:44:42.105 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDsUOhmYGCKDBAAY2EFNmiFNf1x2so9wxp4YQ==", 2024-09-16T15:44:42.105 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:44:42.105 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:44:42.105 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow *", 2024-09-16T15:44:42.105 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow *", 2024-09-16T15:44:42.105 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:44:42.105 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:44:42.105 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:44:42.105 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:44:42.105 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "mds.d", 2024-09-16T15:44:42.106 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDsUOhmfCk4IBAAx12Gw2QudQ2nrPb8VlE0SA==", 2024-09-16T15:44:42.106 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:44:42.106 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:44:42.106 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow *", 2024-09-16T15:44:42.106 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow *", 2024-09-16T15:44:42.106 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:44:42.106 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:44:42.106 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:44:42.106 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:44:42.106 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.0", 2024-09-16T15:44:42.108 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDvUOhmKji5NxAAn/E3YQR7jeMnBTLBXc1G6A==", 2024-09-16T15:44:42.108 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:44:42.108 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:44:42.108 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:44:42.108 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:44:42.108 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:44:42.108 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:44:42.108 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:44:42.108 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.1", 2024-09-16T15:44:42.108 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDzUOhmmBrNBhAA0Uwh4Fam2bmKKvz5jAQBWw==", 2024-09-16T15:44:42.109 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:44:42.109 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:44:42.109 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:44:42.109 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:44:42.109 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:44:42.109 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:44:42.109 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:44:42.109 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.2", 2024-09-16T15:44:42.109 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQD2UOhmOV6GERAAjD+PQLTZrCFR1669UEEIAQ==", 2024-09-16T15:44:42.109 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:44:42.109 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:44:42.109 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:44:42.109 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:44:42.109 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:44:42.110 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:44:42.110 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:44:42.110 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.3", 2024-09-16T15:44:42.110 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQD5UOhmCRrdGRAAQC0bqWX6N1jniVGEQ2aAvg==", 2024-09-16T15:44:42.110 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:44:42.110 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:44:42.110 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:44:42.110 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:44:42.110 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:44:42.110 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:44:42.110 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:44:42.110 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.4", 2024-09-16T15:44:42.110 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQD/UOhmXI/nGRAA7t6XveKvrWgqy6rHuMQSZQ==", 2024-09-16T15:44:42.110 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:44:42.110 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:44:42.110 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:44:42.111 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:44:42.111 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:44:42.111 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:44:42.111 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:44:42.111 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.5", 2024-09-16T15:44:42.111 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQACUehmKM//IxAAsoFpiiTXNs//v3YXeUf/Iw==", 2024-09-16T15:44:42.111 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:44:42.111 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:44:42.111 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:44:42.111 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:44:42.111 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:44:42.111 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:44:42.111 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:44:42.111 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.6", 2024-09-16T15:44:42.112 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQAFUehmlo/FLRAAp5mvUSWCaBbm85QyTZY0eA==", 2024-09-16T15:44:42.112 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:44:42.112 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:44:42.112 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:44:42.112 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:44:42.112 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:44:42.112 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:44:42.112 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:44:42.112 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.7", 2024-09-16T15:44:42.112 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQAIUehmR32JNxAA27antOIKot1nScQbLZDf5g==", 2024-09-16T15:44:42.112 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:44:42.112 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:44:42.112 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:44:42.112 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:44:42.113 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:44:42.113 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:44:42.113 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:44:42.113 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "client.0", 2024-09-16T15:44:42.113 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDsUOhmf5PBKBAAKE4Y4zbA4h07IBNjPkyptw==", 2024-09-16T15:44:42.113 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:44:42.113 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:44:42.113 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow r", 2024-09-16T15:44:42.113 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-16T15:44:42.113 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:44:42.113 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:44:42.113 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:44:42.113 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "client.1", 2024-09-16T15:44:42.113 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDsUOhm15emNRAAwhK7QmA2c9IYGn04ERvY0g==", 2024-09-16T15:44:42.113 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:44:42.114 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:44:42.114 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow r", 2024-09-16T15:44:42.114 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow rw", 2024-09-16T15:44:42.114 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow rwx" 2024-09-16T15:44:42.114 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:44:42.114 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:44:42.114 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:44:42.114 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "client.2", 2024-09-16T15:44:42.114 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDsUOhm6zUyLxAAkzeiIQTuNZkMM+Oit9rC5A==", 2024-09-16T15:44:42.114 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:44:42.114 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:44:42.114 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow r", 2024-09-16T15:44:42.114 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow rw", 2024-09-16T15:44:42.114 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow rwx" 2024-09-16T15:44:42.114 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:44:42.115 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:44:42.115 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:44:42.115 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "client.3", 2024-09-16T15:44:42.115 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDtUOhmD82EABAAeDf82/hPxpPfAUk8oj5h/Q==", 2024-09-16T15:44:42.115 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:44:42.115 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:44:42.115 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow r", 2024-09-16T15:44:42.115 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow rw", 2024-09-16T15:44:42.115 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow rwx" 2024-09-16T15:44:42.115 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:44:42.115 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:44:42.115 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:44:42.115 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "client.admin", 2024-09-16T15:44:42.115 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDrUOhmQFmVDRAA8NKysbAe475xvg4CPAcFyw==", 2024-09-16T15:44:42.116 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:44:42.116 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow *", 2024-09-16T15:44:42.116 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow *", 2024-09-16T15:44:42.116 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow *", 2024-09-16T15:44:42.116 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:44:42.116 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:44:42.116 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:44:42.116 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:44:42.116 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "mgr.x", 2024-09-16T15:44:42.116 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDrUOhmloW3NxAA8UoLUfrw3wf1nzKbmtK/jw==", 2024-09-16T15:44:42.116 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:44:42.116 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow *", 2024-09-16T15:44:42.116 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile mgr", 2024-09-16T15:44:42.116 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:44:42.116 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:44:42.117 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:44:42.117 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:44:42.117 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "mgr.y", 2024-09-16T15:44:42.117 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDrUOhm1uZOLxAATgjA79vYf0psqQVo03sdDA==", 2024-09-16T15:44:42.117 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:44:42.117 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow *", 2024-09-16T15:44:42.117 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile mgr", 2024-09-16T15:44:42.117 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:44:42.117 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:44:42.117 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:44:42.117 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-09-16T15:44:42.117 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-16T15:44:42.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.104+0000 7f6e4f800640 1 -- 172.21.15.151:0/3266925584 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6e28079180 msgr2=0x7f6e2807b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:42.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.104+0000 7f6e4f800640 1 --2- 172.21.15.151:0/3266925584 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6e28079180 0x7f6e2807b640 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6e481566b0 tx=0x7f6e34009040 comp rx=0 tx=0).stop 2024-09-16T15:44:42.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.104+0000 7f6e4f800640 1 -- 172.21.15.151:0/3266925584 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6e48072df0 msgr2=0x7f6e4814c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:42.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.104+0000 7f6e4f800640 1 --2- 172.21.15.151:0/3266925584 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6e48072df0 0x7f6e4814c730 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c02aa90 tx=0x7f6e3c033c20 comp rx=0 tx=0).stop 2024-09-16T15:44:42.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.104+0000 7f6e4f800640 1 -- 172.21.15.151:0/3266925584 shutdown_connections 2024-09-16T15:44:42.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.104+0000 7f6e4f800640 1 --2- 172.21.15.151:0/3266925584 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6e28079180 0x7f6e2807b640 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:42.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.104+0000 7f6e4f800640 1 --2- 172.21.15.151:0/3266925584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e48161550 0x7f6e480d0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:42.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.104+0000 7f6e4f800640 1 --2- 172.21.15.151:0/3266925584 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e480af3d0 0x7f6e480ca9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:42.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.104+0000 7f6e4f800640 1 --2- 172.21.15.151:0/3266925584 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6e48072df0 0x7f6e4814c730 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:42.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.104+0000 7f6e4f800640 1 -- 172.21.15.151:0/3266925584 >> 172.21.15.151:0/3266925584 conn(0x7f6e480b1950 msgr2=0x7f6e4806ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:42.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.105+0000 7f6e4f800640 1 -- 172.21.15.151:0/3266925584 shutdown_connections 2024-09-16T15:44:42.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.105+0000 7f6e4f800640 1 -- 172.21.15.151:0/3266925584 wait complete. 2024-09-16T15:44:42.122 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-16T15:44:42.280 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.278+0000 7f67ebe00640 1 -- 172.21.15.151:0/2889369868 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67ec0ad880 msgr2=0x7f67ec0adcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:42.280 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.278+0000 7f67ebe00640 1 --2- 172.21.15.151:0/2889369868 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67ec0ad880 0x7f67ec0adcd0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f67d4009830 tx=0x7f67d402a5c0 comp rx=0 tx=0).stop 2024-09-16T15:44:42.280 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.279+0000 7f67ebe00640 1 -- 172.21.15.151:0/2889369868 shutdown_connections 2024-09-16T15:44:42.280 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.279+0000 7f67ebe00640 1 --2- 172.21.15.151:0/2889369868 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67ec0ad880 0x7f67ec0adcd0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:42.280 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.279+0000 7f67ebe00640 1 -- 172.21.15.151:0/2889369868 >> 172.21.15.151:0/2889369868 conn(0x7f67ec0cfa20 msgr2=0x7f67ec0cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:42.281 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.279+0000 7f67ebe00640 1 -- 172.21.15.151:0/2889369868 shutdown_connections 2024-09-16T15:44:42.281 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.279+0000 7f67ebe00640 1 -- 172.21.15.151:0/2889369868 wait complete. 2024-09-16T15:44:42.281 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.280+0000 7f67ebe00640 1 Processor -- start 2024-09-16T15:44:42.281 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.280+0000 7f67ebe00640 1 -- start start 2024-09-16T15:44:42.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.281+0000 7f67ebe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f67ec06fb90 0x7f67ec0c36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:42.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.281+0000 7f67ebe00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f67ec0ad880 0x7f67ec0c5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:42.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.281+0000 7f67ebe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67ec0cd5f0 0x7f67ec1f4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:42.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.281+0000 7f67ebe00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f67ec0cdec0 con 0x7f67ec06fb90 2024-09-16T15:44:42.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.281+0000 7f67ebe00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f67ec0cdd40 con 0x7f67ec0cd5f0 2024-09-16T15:44:42.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.281+0000 7f67ebe00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f67ec15ed80 con 0x7f67ec0ad880 2024-09-16T15:44:42.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.281+0000 7f67eaa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f67ec06fb90 0x7f67ec0c36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:42.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.282+0000 7f67eaa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f67ec06fb90 0x7f67ec0c36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:34144/0 (socket says 172.21.15.151:34144) 2024-09-16T15:44:42.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.282+0000 7f67eaa00640 1 -- 172.21.15.151:0/60858277 learned_addr learned my addr 172.21.15.151:0/60858277 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:42.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.282+0000 7f67eb400640 1 --2- 172.21.15.151:0/60858277 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67ec0cd5f0 0x7f67ec1f4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:42.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.282+0000 7f67e2000640 1 --2- 172.21.15.151:0/60858277 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f67ec0ad880 0x7f67ec0c5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:42.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.282+0000 7f67eaa00640 1 -- 172.21.15.151:0/60858277 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f67ec0ad880 msgr2=0x7f67ec0c5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:42.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.282+0000 7f67eaa00640 1 --2- 172.21.15.151:0/60858277 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f67ec0ad880 0x7f67ec0c5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:42.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.282+0000 7f67eaa00640 1 -- 172.21.15.151:0/60858277 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67ec0cd5f0 msgr2=0x7f67ec1f4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:42.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.282+0000 7f67eaa00640 1 --2- 172.21.15.151:0/60858277 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67ec0cd5f0 0x7f67ec1f4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:42.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.282+0000 7f67eaa00640 1 -- 172.21.15.151:0/60858277 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67ec0c5e50 con 0x7f67ec06fb90 2024-09-16T15:44:42.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.282+0000 7f67eb400640 1 --2- 172.21.15.151:0/60858277 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67ec0cd5f0 0x7f67ec1f4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:44:42.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.282+0000 7f67eaa00640 1 --2- 172.21.15.151:0/60858277 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f67ec06fb90 0x7f67ec0c36d0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f67dc00c500 tx=0x7f67dc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:42.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.282+0000 7f67e3e00640 1 -- 172.21.15.151:0/60858277 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67dc0052d0 con 0x7f67ec06fb90 2024-09-16T15:44:42.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.282+0000 7f67e3e00640 1 -- 172.21.15.151:0/60858277 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f67dc005470 con 0x7f67ec06fb90 2024-09-16T15:44:42.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.282+0000 7f67e3e00640 1 -- 172.21.15.151:0/60858277 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67dc012870 con 0x7f67ec06fb90 2024-09-16T15:44:42.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.283+0000 7f67ebe00640 1 -- 172.21.15.151:0/60858277 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67ec0c6130 con 0x7f67ec06fb90 2024-09-16T15:44:42.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.283+0000 7f67ebe00640 1 -- 172.21.15.151:0/60858277 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67ec159d80 con 0x7f67ec06fb90 2024-09-16T15:44:42.286 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.284+0000 7f67ebe00640 1 -- 172.21.15.151:0/60858277 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67ec15e0f0 con 0x7f67ec06fb90 2024-09-16T15:44:42.286 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.285+0000 7f67e3e00640 1 -- 172.21.15.151:0/60858277 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 102103+0+0 (secure 0 0 0) 0x7f67dc0035f0 con 0x7f67ec06fb90 2024-09-16T15:44:42.287 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.286+0000 7f67e3e00640 1 --2- 172.21.15.151:0/60858277 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f67c8079150 0x7f67c807b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:42.287 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.286+0000 7f67e3e00640 1 -- 172.21.15.151:0/60858277 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 4885+0+0 (secure 0 0 0) 0x7f67dc098e30 con 0x7f67ec06fb90 2024-09-16T15:44:42.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.287+0000 7f67e2000640 1 --2- 172.21.15.151:0/60858277 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f67c8079150 0x7f67c807b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:42.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.287+0000 7f67e2000640 1 --2- 172.21.15.151:0/60858277 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f67c8079150 0x7f67c807b610 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f67d4034000 tx=0x7f67d4004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:42.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.291+0000 7f67e3e00640 1 -- 172.21.15.151:0/60858277 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f67ec15e0f0 con 0x7f67ec06fb90 2024-09-16T15:44:42.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.476+0000 7f67ebe00640 1 -- 172.21.15.151:0/60858277 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f67ec0b29b0 con 0x7f67ec06fb90 2024-09-16T15:44:42.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.476+0000 7f67e3e00640 1 -- 172.21.15.151:0/60858277 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v58) ==== 84+0+65 (secure 0 0 0) 0x7f67dc062e70 con 0x7f67ec06fb90 2024-09-16T15:44:42.477 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:44:42.477 INFO:teuthology.orchestra.run.smithi151.stdout:[ 2024-09-16T15:44:42.477 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:44:42.478 INFO:teuthology.orchestra.run.smithi151.stdout: "poolnum": 1, 2024-09-16T15:44:42.478 INFO:teuthology.orchestra.run.smithi151.stdout: "poolname": ".mgr" 2024-09-16T15:44:42.478 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:44:42.478 INFO:teuthology.orchestra.run.smithi151.stdout:] 2024-09-16T15:44:42.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.478+0000 7f67ebe00640 1 -- 172.21.15.151:0/60858277 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f67c8079150 msgr2=0x7f67c807b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:42.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.478+0000 7f67ebe00640 1 --2- 172.21.15.151:0/60858277 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f67c8079150 0x7f67c807b610 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f67d4034000 tx=0x7f67d4004730 comp rx=0 tx=0).stop 2024-09-16T15:44:42.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.478+0000 7f67ebe00640 1 -- 172.21.15.151:0/60858277 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f67ec06fb90 msgr2=0x7f67ec0c36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:42.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.478+0000 7f67ebe00640 1 --2- 172.21.15.151:0/60858277 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f67ec06fb90 0x7f67ec0c36d0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f67dc00c500 tx=0x7f67dc00c9c0 comp rx=0 tx=0).stop 2024-09-16T15:44:42.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.478+0000 7f67ebe00640 1 -- 172.21.15.151:0/60858277 shutdown_connections 2024-09-16T15:44:42.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.478+0000 7f67ebe00640 1 --2- 172.21.15.151:0/60858277 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f67c8079150 0x7f67c807b610 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:42.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.478+0000 7f67ebe00640 1 --2- 172.21.15.151:0/60858277 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67ec0cd5f0 0x7f67ec1f4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:42.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.478+0000 7f67ebe00640 1 --2- 172.21.15.151:0/60858277 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f67ec0ad880 0x7f67ec0c5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:42.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.478+0000 7f67ebe00640 1 --2- 172.21.15.151:0/60858277 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f67ec06fb90 0x7f67ec0c36d0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:42.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.478+0000 7f67ebe00640 1 -- 172.21.15.151:0/60858277 >> 172.21.15.151:0/60858277 conn(0x7f67ec0cfa20 msgr2=0x7f67ec160c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:42.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.478+0000 7f67ebe00640 1 -- 172.21.15.151:0/60858277 shutdown_connections 2024-09-16T15:44:42.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.479+0000 7f67ebe00640 1 -- 172.21.15.151:0/60858277 wait complete. 2024-09-16T15:44:42.495 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-16T15:44:42.496 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-16T15:44:42.660 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.659+0000 7fe80a400640 1 -- 172.21.15.151:0/1165533395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe80415dfb0 msgr2=0x7fe8040d4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:42.660 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.659+0000 7fe80a400640 1 --2- 172.21.15.151:0/1165533395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe80415dfb0 0x7fe8040d4790 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8009830 tx=0x7fe7f802a5c0 comp rx=0 tx=0).stop 2024-09-16T15:44:42.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.659+0000 7fe80a400640 1 -- 172.21.15.151:0/1165533395 shutdown_connections 2024-09-16T15:44:42.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.659+0000 7fe80a400640 1 --2- 172.21.15.151:0/1165533395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe80415dfb0 0x7fe8040d4790 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:42.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.659+0000 7fe80a400640 1 -- 172.21.15.151:0/1165533395 >> 172.21.15.151:0/1165533395 conn(0x7fe80409d180 msgr2=0x7fe8040d3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:42.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.659+0000 7fe80a400640 1 -- 172.21.15.151:0/1165533395 shutdown_connections 2024-09-16T15:44:42.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.659+0000 7fe80a400640 1 -- 172.21.15.151:0/1165533395 wait complete. 2024-09-16T15:44:42.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.662+0000 7fe80a400640 1 Processor -- start 2024-09-16T15:44:42.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.662+0000 7fe80a400640 1 -- start start 2024-09-16T15:44:42.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.663+0000 7fe80a400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe8040cf470 0x7fe80414f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:42.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.663+0000 7fe80a400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8040d0500 0x7fe8040b3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:42.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.663+0000 7fe80a400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe80415dfb0 0x7fe8040b3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:42.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.663+0000 7fe80a400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe8040d2e30 con 0x7fe8040cf470 2024-09-16T15:44:42.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.663+0000 7fe80a400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe8040b7630 con 0x7fe80415dfb0 2024-09-16T15:44:42.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.663+0000 7fe80a400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fe8040d0990 con 0x7fe8040d0500 2024-09-16T15:44:42.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.663+0000 7fe809000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe8040cf470 0x7fe80414f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:42.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.663+0000 7fe809000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe8040cf470 0x7fe80414f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:34156/0 (socket says 172.21.15.151:34156) 2024-09-16T15:44:42.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.663+0000 7fe809000640 1 -- 172.21.15.151:0/1985163950 learned_addr learned my addr 172.21.15.151:0/1985163950 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:42.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.663+0000 7fe803e00640 1 --2- 172.21.15.151:0/1985163950 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8040d0500 0x7fe8040b3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:42.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.664+0000 7fe809a00640 1 --2- 172.21.15.151:0/1985163950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe80415dfb0 0x7fe8040b3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:42.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.664+0000 7fe809000640 1 -- 172.21.15.151:0/1985163950 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8040d0500 msgr2=0x7fe8040b3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:42.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.664+0000 7fe809000640 1 --2- 172.21.15.151:0/1985163950 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8040d0500 0x7fe8040b3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:42.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.664+0000 7fe809000640 1 -- 172.21.15.151:0/1985163950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe80415dfb0 msgr2=0x7fe8040b3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:42.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.664+0000 7fe809000640 1 --2- 172.21.15.151:0/1985163950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe80415dfb0 0x7fe8040b3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:42.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.664+0000 7fe809000640 1 -- 172.21.15.151:0/1985163950 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe80414fb00 con 0x7fe8040cf470 2024-09-16T15:44:42.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.664+0000 7fe809000640 1 --2- 172.21.15.151:0/1985163950 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe8040cf470 0x7fe80414f5c0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8000c00 tx=0x7fe7f8033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:42.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.664+0000 7fe800c00640 1 -- 172.21.15.151:0/1985163950 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f8031d60 con 0x7fe8040cf470 2024-09-16T15:44:42.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.665+0000 7fe800c00640 1 -- 172.21.15.151:0/1985163950 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe7f8033e10 con 0x7fe8040cf470 2024-09-16T15:44:42.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.665+0000 7fe800c00640 1 -- 172.21.15.151:0/1985163950 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f8033310 con 0x7fe8040cf470 2024-09-16T15:44:42.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.665+0000 7fe80a400640 1 -- 172.21.15.151:0/1985163950 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8040ca850 con 0x7fe8040cf470 2024-09-16T15:44:42.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.665+0000 7fe80a400640 1 -- 172.21.15.151:0/1985163950 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8040c9fb0 con 0x7fe8040cf470 2024-09-16T15:44:42.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.666+0000 7fe80a400640 1 -- 172.21.15.151:0/1985163950 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe80408df30 con 0x7fe8040cf470 2024-09-16T15:44:42.668 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.667+0000 7fe800c00640 1 -- 172.21.15.151:0/1985163950 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 102103+0+0 (secure 0 0 0) 0x7fe7f80334b0 con 0x7fe8040cf470 2024-09-16T15:44:42.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.668+0000 7fe800c00640 1 --2- 172.21.15.151:0/1985163950 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe7e4079180 0x7fe7e407b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:42.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.668+0000 7fe800c00640 1 -- 172.21.15.151:0/1985163950 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 4885+0+0 (secure 0 0 0) 0x7fe7f80ba3c0 con 0x7fe8040cf470 2024-09-16T15:44:42.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.668+0000 7fe803e00640 1 --2- 172.21.15.151:0/1985163950 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe7e4079180 0x7fe7e407b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:42.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.669+0000 7fe803e00640 1 --2- 172.21.15.151:0/1985163950 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe7e4079180 0x7fe7e407b640 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe80415cfb0 tx=0x7fe7f0007cc0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:42.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.674+0000 7fe800c00640 1 -- 172.21.15.151:0/1985163950 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe7f803f030 con 0x7fe8040cf470 2024-09-16T15:44:42.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:42.856+0000 7fe80a400640 1 -- 172.21.15.151:0/1985163950 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fe8040d4710 con 0x7fe8040cf470 2024-09-16T15:44:43.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:43.678+0000 7fe800c00640 1 -- 172.21.15.151:0/1985163950 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v59) ==== 138+0+0 (secure 0 0 0) 0x7fe7f8083e30 con 0x7fe8040cf470 2024-09-16T15:44:43.680 INFO:teuthology.orchestra.run.smithi151.stderr:pool 'cephfs_metadata' created 2024-09-16T15:44:43.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:43.682+0000 7fe80a400640 1 -- 172.21.15.151:0/1985163950 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe7e4079180 msgr2=0x7fe7e407b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:43.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:43.682+0000 7fe80a400640 1 --2- 172.21.15.151:0/1985163950 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe7e4079180 0x7fe7e407b640 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe80415cfb0 tx=0x7fe7f0007cc0 comp rx=0 tx=0).stop 2024-09-16T15:44:43.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:43.682+0000 7fe80a400640 1 -- 172.21.15.151:0/1985163950 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe8040cf470 msgr2=0x7fe80414f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:43.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:43.682+0000 7fe80a400640 1 --2- 172.21.15.151:0/1985163950 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe8040cf470 0x7fe80414f5c0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8000c00 tx=0x7fe7f8033c20 comp rx=0 tx=0).stop 2024-09-16T15:44:43.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:43.682+0000 7fe80a400640 1 -- 172.21.15.151:0/1985163950 shutdown_connections 2024-09-16T15:44:43.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:43.682+0000 7fe80a400640 1 --2- 172.21.15.151:0/1985163950 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe7e4079180 0x7fe7e407b640 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:43.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:43.682+0000 7fe80a400640 1 --2- 172.21.15.151:0/1985163950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe80415dfb0 0x7fe8040b3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:43.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:43.682+0000 7fe80a400640 1 --2- 172.21.15.151:0/1985163950 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8040d0500 0x7fe8040b3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:43.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:43.682+0000 7fe80a400640 1 --2- 172.21.15.151:0/1985163950 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe8040cf470 0x7fe80414f5c0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:43.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:43.682+0000 7fe80a400640 1 -- 172.21.15.151:0/1985163950 >> 172.21.15.151:0/1985163950 conn(0x7fe80409d180 msgr2=0x7fe8040b7840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:43.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:43.682+0000 7fe80a400640 1 -- 172.21.15.151:0/1985163950 shutdown_connections 2024-09-16T15:44:43.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:43.682+0000 7fe80a400640 1 -- 172.21.15.151:0/1985163950 wait complete. 2024-09-16T15:44:43.706 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-16T15:44:43.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:43.874+0000 7fcf17400640 1 -- 172.21.15.151:0/3144216775 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcf100cccd0 msgr2=0x7fcf10073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:43.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:43.874+0000 7fcf17400640 1 --2- 172.21.15.151:0/3144216775 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcf100cccd0 0x7fcf10073440 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fcefc009860 tx=0x7fcefc02a510 comp rx=0 tx=0).stop 2024-09-16T15:44:43.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:43.874+0000 7fcf17400640 1 -- 172.21.15.151:0/3144216775 shutdown_connections 2024-09-16T15:44:43.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:43.874+0000 7fcf17400640 1 --2- 172.21.15.151:0/3144216775 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcf100cccd0 0x7fcf10073440 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:43.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:43.874+0000 7fcf17400640 1 -- 172.21.15.151:0/3144216775 >> 172.21.15.151:0/3144216775 conn(0x7fcf100b1840 msgr2=0x7fcf100b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:43.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:43.874+0000 7fcf17400640 1 -- 172.21.15.151:0/3144216775 shutdown_connections 2024-09-16T15:44:43.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:43.874+0000 7fcf17400640 1 -- 172.21.15.151:0/3144216775 wait complete. 2024-09-16T15:44:43.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:43.876+0000 7fcf17400640 1 Processor -- start 2024-09-16T15:44:43.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:43.876+0000 7fcf17400640 1 -- start start 2024-09-16T15:44:43.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:43.877+0000 7fcf17400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf1006efa0 0x7fcf100cbd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:43.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:43.877+0000 7fcf17400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf100cccd0 0x7fcf10134bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:43.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:43.877+0000 7fcf17400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcf10145750 0x7fcf100b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:43.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:43.877+0000 7fcf17400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcf100ce6d0 con 0x7fcf1006efa0 2024-09-16T15:44:43.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:43.877+0000 7fcf17400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcf100ce550 con 0x7fcf100cccd0 2024-09-16T15:44:43.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:43.877+0000 7fcf17400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fcf100ce850 con 0x7fcf10145750 2024-09-16T15:44:43.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:43.877+0000 7fcf14a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf1006efa0 0x7fcf100cbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:43.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:43.877+0000 7fcf14a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf1006efa0 0x7fcf100cbd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44190/0 (socket says 172.21.15.151:44190) 2024-09-16T15:44:43.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:43.877+0000 7fcf14a00640 1 -- 172.21.15.151:0/1747391661 learned_addr learned my addr 172.21.15.151:0/1747391661 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:43.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:43.877+0000 7fcf15400640 1 --2- 172.21.15.151:0/1747391661 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcf10145750 0x7fcf100b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:43.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:43.877+0000 7fcf0fe00640 1 --2- 172.21.15.151:0/1747391661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf100cccd0 0x7fcf10134bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:43.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:43.878+0000 7fcf14a00640 1 -- 172.21.15.151:0/1747391661 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcf10145750 msgr2=0x7fcf100b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:43.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:43.878+0000 7fcf14a00640 1 --2- 172.21.15.151:0/1747391661 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcf10145750 0x7fcf100b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:43.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:43.878+0000 7fcf14a00640 1 -- 172.21.15.151:0/1747391661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf100cccd0 msgr2=0x7fcf10134bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:43.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:43.878+0000 7fcf14a00640 1 --2- 172.21.15.151:0/1747391661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf100cccd0 0x7fcf10134bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:43.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:43.878+0000 7fcf14a00640 1 -- 172.21.15.151:0/1747391661 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf10135130 con 0x7fcf1006efa0 2024-09-16T15:44:43.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:43.878+0000 7fcf14a00640 1 --2- 172.21.15.151:0/1747391661 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf1006efa0 0x7fcf100cbd00 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fcf0400c430 tx=0x7fcf0400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:43.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:43.878+0000 7fcf0cc00640 1 -- 172.21.15.151:0/1747391661 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf04017070 con 0x7fcf1006efa0 2024-09-16T15:44:43.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:43.878+0000 7fcf0cc00640 1 -- 172.21.15.151:0/1747391661 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcf040053e0 con 0x7fcf1006efa0 2024-09-16T15:44:43.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:43.878+0000 7fcf0cc00640 1 -- 172.21.15.151:0/1747391661 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf04003850 con 0x7fcf1006efa0 2024-09-16T15:44:43.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:43.878+0000 7fcf17400640 1 -- 172.21.15.151:0/1747391661 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf100b2e90 con 0x7fcf1006efa0 2024-09-16T15:44:43.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:43.878+0000 7fcf17400640 1 -- 172.21.15.151:0/1747391661 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf100b3450 con 0x7fcf1006efa0 2024-09-16T15:44:43.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:43.879+0000 7fcf17400640 1 -- 172.21.15.151:0/1747391661 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf100ced00 con 0x7fcf1006efa0 2024-09-16T15:44:43.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:43.881+0000 7fcf0cc00640 1 -- 172.21.15.151:0/1747391661 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 102103+0+0 (secure 0 0 0) 0x7fcf0401a070 con 0x7fcf1006efa0 2024-09-16T15:44:43.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:43.882+0000 7fcf0cc00640 1 --2- 172.21.15.151:0/1747391661 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcef0079150 0x7fcef007b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:43.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:43.882+0000 7fcf0cc00640 1 -- 172.21.15.151:0/1747391661 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5255+0+0 (secure 0 0 0) 0x7fcf04098950 con 0x7fcf1006efa0 2024-09-16T15:44:43.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:43.882+0000 7fcf0fe00640 1 --2- 172.21.15.151:0/1747391661 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcef0079150 0x7fcef007b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:43.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:43.883+0000 7fcf0fe00640 1 --2- 172.21.15.151:0/1747391661 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcef0079150 0x7fcef007b610 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fcefc034000 tx=0x7fcefc004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:43.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:43.885+0000 7fcf0cc00640 1 -- 172.21.15.151:0/1747391661 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fcf04021020 con 0x7fcf1006efa0 2024-09-16T15:44:44.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:44.071+0000 7fcf17400640 1 -- 172.21.15.151:0/1747391661 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fcf10144bb0 con 0x7fcf1006efa0 2024-09-16T15:44:44.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:44.681+0000 7fcf0cc00640 1 -- 172.21.15.151:0/1747391661 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v60) ==== 170+0+0 (secure 0 0 0) 0x7fcf04062420 con 0x7fcf1006efa0 2024-09-16T15:44:44.682 INFO:teuthology.orchestra.run.smithi151.stderr:pool 'cephfs_data' created 2024-09-16T15:44:44.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:44.684+0000 7fcf17400640 1 -- 172.21.15.151:0/1747391661 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcef0079150 msgr2=0x7fcef007b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:44.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:44.684+0000 7fcf17400640 1 --2- 172.21.15.151:0/1747391661 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcef0079150 0x7fcef007b610 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fcefc034000 tx=0x7fcefc004730 comp rx=0 tx=0).stop 2024-09-16T15:44:44.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:44.684+0000 7fcf17400640 1 -- 172.21.15.151:0/1747391661 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf1006efa0 msgr2=0x7fcf100cbd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:44.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:44.684+0000 7fcf17400640 1 --2- 172.21.15.151:0/1747391661 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf1006efa0 0x7fcf100cbd00 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fcf0400c430 tx=0x7fcf0400c8f0 comp rx=0 tx=0).stop 2024-09-16T15:44:44.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:44.685+0000 7fcf17400640 1 -- 172.21.15.151:0/1747391661 shutdown_connections 2024-09-16T15:44:44.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:44.685+0000 7fcf17400640 1 --2- 172.21.15.151:0/1747391661 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcef0079150 0x7fcef007b610 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:44.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:44.685+0000 7fcf17400640 1 --2- 172.21.15.151:0/1747391661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcf100cccd0 0x7fcf10134bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:44.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:44.685+0000 7fcf17400640 1 --2- 172.21.15.151:0/1747391661 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcf10145750 0x7fcf100b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:44.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:44.685+0000 7fcf17400640 1 --2- 172.21.15.151:0/1747391661 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf1006efa0 0x7fcf100cbd00 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:44.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:44.685+0000 7fcf17400640 1 -- 172.21.15.151:0/1747391661 >> 172.21.15.151:0/1747391661 conn(0x7fcf100b1840 msgr2=0x7fcf100a95e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:44.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:44.685+0000 7fcf17400640 1 -- 172.21.15.151:0/1747391661 shutdown_connections 2024-09-16T15:44:44.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:44.685+0000 7fcf17400640 1 -- 172.21.15.151:0/1747391661 wait complete. 2024-09-16T15:44:44.719 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-16T15:44:44.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:44.879+0000 7f8e4fe00640 1 -- 172.21.15.151:0/2286136402 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e5015f790 msgr2=0x7f8e5015fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:44.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:44.879+0000 7f8e4fe00640 1 --2- 172.21.15.151:0/2286136402 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e5015f790 0x7f8e5015fbe0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f8e500d1350 tx=0x7f8e3c02a560 comp rx=0 tx=0).stop 2024-09-16T15:44:44.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:44.879+0000 7f8e4fe00640 1 -- 172.21.15.151:0/2286136402 shutdown_connections 2024-09-16T15:44:44.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:44.879+0000 7f8e4fe00640 1 --2- 172.21.15.151:0/2286136402 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8e5015c1a0 0x7f8e5015c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:44.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:44.879+0000 7f8e4fe00640 1 --2- 172.21.15.151:0/2286136402 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e5015f790 0x7f8e5015fbe0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:44.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:44.879+0000 7f8e4fe00640 1 --2- 172.21.15.151:0/2286136402 >> v2:172.21.15.167:3300/0 conn(0x7f8e500b1880 0x7f8e500688b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:44.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:44.879+0000 7f8e4fe00640 1 -- 172.21.15.151:0/2286136402 >> 172.21.15.151:0/2286136402 conn(0x7f8e500b5680 msgr2=0x7f8e500b5a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:44.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:44.879+0000 7f8e4fe00640 1 -- 172.21.15.151:0/2286136402 shutdown_connections 2024-09-16T15:44:44.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:44.879+0000 7f8e4fe00640 1 -- 172.21.15.151:0/2286136402 wait complete. 2024-09-16T15:44:44.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:44.881+0000 7f8e4fe00640 1 Processor -- start 2024-09-16T15:44:44.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:44.881+0000 7f8e4fe00640 1 -- start start 2024-09-16T15:44:44.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:44.882+0000 7f8e4fe00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8e500b1880 0x7f8e501f4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:44.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:44.882+0000 7f8e4fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e5015c1a0 0x7f8e501584e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:44.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:44.882+0000 7f8e4fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e5015f790 0x7f8e501543a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:44.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:44.882+0000 7f8e4fe00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5015cb50 con 0x7f8e5015f790 2024-09-16T15:44:44.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:44.882+0000 7f8e4fe00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8e500b1e70 con 0x7f8e5015c1a0 2024-09-16T15:44:44.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:44.882+0000 7f8e4fe00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f8e5015ccd0 con 0x7f8e500b1880 2024-09-16T15:44:44.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:44.882+0000 7f8e4f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e5015f790 0x7f8e501543a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:44.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:44.882+0000 7f8e4f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e5015f790 0x7f8e501543a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44208/0 (socket says 172.21.15.151:44208) 2024-09-16T15:44:44.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:44.882+0000 7f8e4f400640 1 -- 172.21.15.151:0/126595850 learned_addr learned my addr 172.21.15.151:0/126595850 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:44.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:44.882+0000 7f8e4ea00640 1 --2- 172.21.15.151:0/126595850 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8e500b1880 0x7f8e501f4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:44.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:44.882+0000 7f8e4e000640 1 --2- 172.21.15.151:0/126595850 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e5015c1a0 0x7f8e501584e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:44.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:44.882+0000 7f8e4f400640 1 -- 172.21.15.151:0/126595850 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8e500b1880 msgr2=0x7f8e501f4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:44.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:44.882+0000 7f8e4f400640 1 --2- 172.21.15.151:0/126595850 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8e500b1880 0x7f8e501f4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:44.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:44.882+0000 7f8e4f400640 1 -- 172.21.15.151:0/126595850 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e5015c1a0 msgr2=0x7f8e501584e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:44.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:44.882+0000 7f8e4f400640 1 --2- 172.21.15.151:0/126595850 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e5015c1a0 0x7f8e501584e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:44.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:44.883+0000 7f8e4f400640 1 -- 172.21.15.151:0/126595850 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e50153020 con 0x7f8e5015f790 2024-09-16T15:44:44.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:44.883+0000 7f8e4f400640 1 --2- 172.21.15.151:0/126595850 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e5015f790 0x7f8e501543a0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f8e4800e970 tx=0x7f8e4800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:44.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:44.883+0000 7f8e46a00640 1 -- 172.21.15.151:0/126595850 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e4800ccc0 con 0x7f8e5015f790 2024-09-16T15:44:44.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:44.883+0000 7f8e46a00640 1 -- 172.21.15.151:0/126595850 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8e4800ce60 con 0x7f8e5015f790 2024-09-16T15:44:44.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:44.883+0000 7f8e46a00640 1 -- 172.21.15.151:0/126595850 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e48005280 con 0x7f8e5015f790 2024-09-16T15:44:44.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:44.883+0000 7f8e4fe00640 1 -- 172.21.15.151:0/126595850 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e50153300 con 0x7f8e5015f790 2024-09-16T15:44:44.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:44.883+0000 7f8e4fe00640 1 -- 172.21.15.151:0/126595850 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e501538c0 con 0x7f8e5015f790 2024-09-16T15:44:44.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:44.884+0000 7f8e4fe00640 1 -- 172.21.15.151:0/126595850 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e5015c610 con 0x7f8e5015f790 2024-09-16T15:44:44.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:44.886+0000 7f8e46a00640 1 -- 172.21.15.151:0/126595850 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 102103+0+0 (secure 0 0 0) 0x7f8e48005420 con 0x7f8e5015f790 2024-09-16T15:44:44.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:44.887+0000 7f8e46a00640 1 --2- 172.21.15.151:0/126595850 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8e30079150 0x7f8e3007b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:44.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:44.887+0000 7f8e46a00640 1 -- 172.21.15.151:0/126595850 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5637+0+0 (secure 0 0 0) 0x7f8e48099b30 con 0x7f8e5015f790 2024-09-16T15:44:44.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:44.887+0000 7f8e4ea00640 1 --2- 172.21.15.151:0/126595850 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8e30079150 0x7f8e3007b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:44.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:44.888+0000 7f8e4ea00640 1 --2- 172.21.15.151:0/126595850 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8e30079150 0x7f8e3007b610 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f8e40005fd0 tx=0x7f8e40009500 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:44.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:44.891+0000 7f8e46a00640 1 -- 172.21.15.151:0/126595850 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8e48063bf0 con 0x7f8e5015f790 2024-09-16T15:44:45.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:45.113+0000 7f8e4fe00640 1 -- 172.21.15.151:0/126595850 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f8e500750a0 con 0x7f8e5015f790 2024-09-16T15:44:45.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:45.703+0000 7f8e46a00640 1 -- 172.21.15.151:0/126595850 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v33) ==== 172+0+0 (secure 0 0 0) 0x7f8e480634f0 con 0x7f8e5015f790 2024-09-16T15:44:45.707 INFO:teuthology.orchestra.run.smithi151.stderr:new fs with metadata pool 8 and data pool 9 2024-09-16T15:44:45.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:45.713+0000 7f8e4fe00640 1 -- 172.21.15.151:0/126595850 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8e30079150 msgr2=0x7f8e3007b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:45.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:45.713+0000 7f8e4fe00640 1 --2- 172.21.15.151:0/126595850 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8e30079150 0x7f8e3007b610 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f8e40005fd0 tx=0x7f8e40009500 comp rx=0 tx=0).stop 2024-09-16T15:44:45.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:45.713+0000 7f8e4fe00640 1 -- 172.21.15.151:0/126595850 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e5015f790 msgr2=0x7f8e501543a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:45.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:45.713+0000 7f8e4fe00640 1 --2- 172.21.15.151:0/126595850 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e5015f790 0x7f8e501543a0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f8e4800e970 tx=0x7f8e4800ed70 comp rx=0 tx=0).stop 2024-09-16T15:44:45.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:45.713+0000 7f8e4fe00640 1 -- 172.21.15.151:0/126595850 shutdown_connections 2024-09-16T15:44:45.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:45.713+0000 7f8e4fe00640 1 --2- 172.21.15.151:0/126595850 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8e30079150 0x7f8e3007b610 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:45.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:45.713+0000 7f8e4fe00640 1 --2- 172.21.15.151:0/126595850 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e5015c1a0 0x7f8e501584e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:45.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:45.713+0000 7f8e4fe00640 1 --2- 172.21.15.151:0/126595850 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e5015f790 0x7f8e501543a0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:45.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:45.713+0000 7f8e4fe00640 1 --2- 172.21.15.151:0/126595850 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8e500b1880 0x7f8e501f4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:45.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:45.713+0000 7f8e4fe00640 1 -- 172.21.15.151:0/126595850 >> 172.21.15.151:0/126595850 conn(0x7f8e500b5680 msgr2=0x7f8e50074990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:45.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:45.715+0000 7f8e4fe00640 1 -- 172.21.15.151:0/126595850 shutdown_connections 2024-09-16T15:44:45.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:45.715+0000 7f8e4fe00640 1 -- 172.21.15.151:0/126595850 wait complete. 2024-09-16T15:44:45.745 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:44:45.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:45.902+0000 7fa878a00640 1 -- 172.21.15.151:0/3658643717 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa858003a50 msgr2=0x7fa858003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:45.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:45.902+0000 7fa878a00640 1 --2- 172.21.15.151:0/3658643717 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa858003a50 0x7fa858003e70 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fa864001eb0 tx=0x7fa858005830 comp rx=0 tx=0).stop 2024-09-16T15:44:45.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:45.902+0000 7fa878a00640 1 -- 172.21.15.151:0/3658643717 shutdown_connections 2024-09-16T15:44:45.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:45.902+0000 7fa878a00640 1 --2- 172.21.15.151:0/3658643717 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa858003a50 0x7fa858003e70 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:45.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:45.902+0000 7fa878a00640 1 -- 172.21.15.151:0/3658643717 >> 172.21.15.151:0/3658643717 conn(0x7fa8740b1950 msgr2=0x7fa8740b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:45.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:45.902+0000 7fa878a00640 1 -- 172.21.15.151:0/3658643717 shutdown_connections 2024-09-16T15:44:45.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:45.902+0000 7fa878a00640 1 -- 172.21.15.151:0/3658643717 wait complete. 2024-09-16T15:44:45.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:45.904+0000 7fa878a00640 1 Processor -- start 2024-09-16T15:44:45.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:45.904+0000 7fa878a00640 1 -- start start 2024-09-16T15:44:45.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:45.905+0000 7fa878a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa858003a50 0x7fa87414ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:45.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:45.905+0000 7fa878a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8741f3100 0x7fa8740cb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:45.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:45.905+0000 7fa878a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8740cb910 0x7fa874150d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:45.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:45.905+0000 7fa878a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa874074420 con 0x7fa858003a50 2024-09-16T15:44:45.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:45.905+0000 7fa878a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa8740b2700 con 0x7fa8740cb910 2024-09-16T15:44:45.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:45.905+0000 7fa878a00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fa8740745a0 con 0x7fa8741f3100 2024-09-16T15:44:45.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:45.905+0000 7fa871e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa858003a50 0x7fa87414ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:45.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:45.905+0000 7fa871e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa858003a50 0x7fa87414ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44224/0 (socket says 172.21.15.151:44224) 2024-09-16T15:44:45.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:45.905+0000 7fa871e00640 1 -- 172.21.15.151:0/529438764 learned_addr learned my addr 172.21.15.151:0/529438764 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:45.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:45.905+0000 7fa871400640 1 --2- 172.21.15.151:0/529438764 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8741f3100 0x7fa8740cb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:45.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:45.905+0000 7fa872800640 1 --2- 172.21.15.151:0/529438764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8740cb910 0x7fa874150d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:45.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:45.905+0000 7fa871e00640 1 -- 172.21.15.151:0/529438764 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8741f3100 msgr2=0x7fa8740cb3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:45.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:45.905+0000 7fa871e00640 1 --2- 172.21.15.151:0/529438764 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8741f3100 0x7fa8740cb3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:45.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:45.905+0000 7fa871e00640 1 -- 172.21.15.151:0/529438764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8740cb910 msgr2=0x7fa874150d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:45.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:45.905+0000 7fa871e00640 1 --2- 172.21.15.151:0/529438764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8740cb910 0x7fa874150d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:45.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:45.905+0000 7fa871e00640 1 -- 172.21.15.151:0/529438764 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8740c5ec0 con 0x7fa858003a50 2024-09-16T15:44:45.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:45.906+0000 7fa871e00640 1 --2- 172.21.15.151:0/529438764 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa858003a50 0x7fa87414ea40 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fa864001eb0 tx=0x7fa864006d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:45.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:45.906+0000 7fa86a000640 1 -- 172.21.15.151:0/529438764 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa864031040 con 0x7fa858003a50 2024-09-16T15:44:45.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:45.906+0000 7fa86a000640 1 -- 172.21.15.151:0/529438764 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa8640065a0 con 0x7fa858003a50 2024-09-16T15:44:45.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:45.906+0000 7fa86a000640 1 -- 172.21.15.151:0/529438764 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa864003a40 con 0x7fa858003a50 2024-09-16T15:44:45.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:45.906+0000 7fa878a00640 1 -- 172.21.15.151:0/529438764 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8740c61a0 con 0x7fa858003a50 2024-09-16T15:44:45.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:45.906+0000 7fa878a00640 1 -- 172.21.15.151:0/529438764 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa87414ef80 con 0x7fa858003a50 2024-09-16T15:44:45.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:45.907+0000 7fa878a00640 1 -- 172.21.15.151:0/529438764 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8740ad4f0 con 0x7fa858003a50 2024-09-16T15:44:45.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:45.908+0000 7fa86a000640 1 -- 172.21.15.151:0/529438764 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 102103+0+0 (secure 0 0 0) 0x7fa864034070 con 0x7fa858003a50 2024-09-16T15:44:45.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:45.909+0000 7fa86a000640 1 --2- 172.21.15.151:0/529438764 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa854079180 0x7fa85407b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:45.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:45.909+0000 7fa86a000640 1 -- 172.21.15.151:0/529438764 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7fa8640b9010 con 0x7fa858003a50 2024-09-16T15:44:45.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:45.910+0000 7fa871400640 1 --2- 172.21.15.151:0/529438764 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa854079180 0x7fa85407b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:45.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:45.910+0000 7fa871400640 1 --2- 172.21.15.151:0/529438764 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa854079180 0x7fa85407b640 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa860009b30 tx=0x7fa860005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:45.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:45.913+0000 7fa86a000640 1 -- 172.21.15.151:0/529438764 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa8640830d0 con 0x7fa858003a50 2024-09-16T15:44:46.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.098+0000 7fa878a00640 1 -- 172.21.15.151:0/529438764 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa8740cf8d0 con 0x7fa858003a50 2024-09-16T15:44:46.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.099+0000 7fa86a000640 1 -- 172.21.15.151:0/529438764 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) ==== 74+0+16841 (secure 0 0 0) 0x7fa8640829d0 con 0x7fa858003a50 2024-09-16T15:44:46.101 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:44:46.101 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":61,"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","created":"2024-09-16T15:38:56.348535+0000","modified":"2024-09-16T15:44:45.114672+0000","last_up_change":"2024-09-16T15:39:10.681384+0000","last_in_change":"2024-09-16T15:39:00.544477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:39:11.602252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:44:42.866298+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-16T15:44:44.081060+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5524076-a903-4d0a-af6a-caf474c4765d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6809","nonce":3111342741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6811","nonce":3111342741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6815","nonce":3111342741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6813","nonce":3111342741}]},"public_addr":"172.21.15.151:6809/3111342741","cluster_addr":"172.21.15.151:6811/3111342741","heartbeat_back_addr":"172.21.15.151:6815/3111342741","heartbeat_front_addr":"172.21.15.151:6813/3111342741","state":["exists","up"]},{"osd":1,"uuid":"dd25925f-0ae1-494e-8529-4dc205759db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6801","nonce":3368304087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6803","nonce":3368304087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6807","nonce":3368304087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6805","nonce":3368304087}]},"public_addr":"172.21.15.151:6801/3368304087","cluster_addr":"172.21.15.151:6803/3368304087","heartbeat_back_addr":"172.21.15.151:6807/3368304087","heartbeat_front_addr":"172.21.15.151:6805/3368304087","state":["exists","up"]},{"osd":2,"uuid":"c2f18115-013d-4da6-a0fe-cb978f4494b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6823","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6825","nonce":3079373271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6827","nonce":3079373271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6831","nonce":3079373271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6829","nonce":3079373271}]},"public_addr":"172.21.15.151:6825/3079373271","cluster_addr":"172.21.15.151:6827/3079373271","heartbeat_back_addr":"172.21.15.151:6831/3079373271","heartbeat_front_addr":"172.21.15.151:6829/3079373271","state":["exists","up"]},{"osd":3,"uuid":"36edb718-2fcb-4d88-8ddc-c321eed4279d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6817","nonce":1142509041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6819","nonce":1142509041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6824","nonce":1142509041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6821","nonce":1142509041}]},"public_addr":"172.21.15.151:6817/1142509041","cluster_addr":"172.21.15.151:6819/1142509041","heartbeat_back_addr":"172.21.15.151:6824/1142509041","heartbeat_front_addr":"172.21.15.151:6821/1142509041","state":["exists","up"]},{"osd":4,"uuid":"64d33708-cd0a-4236-abe3-1b646e82ab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6811","nonce":2735495271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6813","nonce":2735495271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6817","nonce":2735495271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6815","nonce":2735495271}]},"public_addr":"172.21.15.167:6811/2735495271","cluster_addr":"172.21.15.167:6813/2735495271","heartbeat_back_addr":"172.21.15.167:6817/2735495271","heartbeat_front_addr":"172.21.15.167:6815/2735495271","state":["exists","up"]},{"osd":5,"uuid":"f14b44a9-5910-458c-b518-09a8d0de94de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6827","nonce":1494916572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6829","nonce":1494916572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6833","nonce":1494916572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6831","nonce":1494916572}]},"public_addr":"172.21.15.167:6827/1494916572","cluster_addr":"172.21.15.167:6829/1494916572","heartbeat_back_addr":"172.21.15.167:6833/1494916572","heartbeat_front_addr":"172.21.15.167:6831/1494916572","state":["exists","up"]},{"osd":6,"uuid":"39335545-2654-4710-84b0-a9aa47cfcbba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6819","nonce":2538273273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6821","nonce":2538273273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6825","nonce":2538273273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6823","nonce":2538273273}]},"public_addr":"172.21.15.167:6819/2538273273","cluster_addr":"172.21.15.167:6821/2538273273","heartbeat_back_addr":"172.21.15.167:6825/2538273273","heartbeat_front_addr":"172.21.15.167:6823/2538273273","state":["exists","up"]},{"osd":7,"uuid":"4f8bfc81-53dd-4339-ac22-1a4007d9f963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6803","nonce":265072227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6805","nonce":265072227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6809","nonce":265072227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6807","nonce":265072227}]},"public_addr":"172.21.15.167:6803/265072227","cluster_addr":"172.21.15.167:6805/265072227","heartbeat_back_addr":"172.21.15.167:6809/265072227","heartbeat_front_addr":"172.21.15.167:6807/265072227","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.150359+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.084358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.722368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.332809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.320709+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.395153+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.935250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.801443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:44:46.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.102+0000 7fa878a00640 1 -- 172.21.15.151:0/529438764 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa854079180 msgr2=0x7fa85407b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:46.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.102+0000 7fa878a00640 1 --2- 172.21.15.151:0/529438764 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa854079180 0x7fa85407b640 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa860009b30 tx=0x7fa860005f90 comp rx=0 tx=0).stop 2024-09-16T15:44:46.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.102+0000 7fa878a00640 1 -- 172.21.15.151:0/529438764 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa858003a50 msgr2=0x7fa87414ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:46.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.102+0000 7fa878a00640 1 --2- 172.21.15.151:0/529438764 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa858003a50 0x7fa87414ea40 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fa864001eb0 tx=0x7fa864006d90 comp rx=0 tx=0).stop 2024-09-16T15:44:46.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.102+0000 7fa878a00640 1 -- 172.21.15.151:0/529438764 shutdown_connections 2024-09-16T15:44:46.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.102+0000 7fa878a00640 1 --2- 172.21.15.151:0/529438764 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa854079180 0x7fa85407b640 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:46.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.102+0000 7fa878a00640 1 --2- 172.21.15.151:0/529438764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8740cb910 0x7fa874150d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:46.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.102+0000 7fa878a00640 1 --2- 172.21.15.151:0/529438764 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8741f3100 0x7fa8740cb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:46.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.102+0000 7fa878a00640 1 --2- 172.21.15.151:0/529438764 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa858003a50 0x7fa87414ea40 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:46.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.102+0000 7fa878a00640 1 -- 172.21.15.151:0/529438764 >> 172.21.15.151:0/529438764 conn(0x7fa8740b1950 msgr2=0x7fa87406f080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:46.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.102+0000 7fa878a00640 1 -- 172.21.15.151:0/529438764 shutdown_connections 2024-09-16T15:44:46.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.102+0000 7fa878a00640 1 -- 172.21.15.151:0/529438764 wait complete. 2024-09-16T15:44:46.120 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:44:46.280 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.278+0000 7faf22e00640 1 -- 172.21.15.151:0/70469363 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf1c0cf300 msgr2=0x7faf1c0729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:46.280 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.278+0000 7faf22e00640 1 --2- 172.21.15.151:0/70469363 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf1c0cf300 0x7faf1c0729a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7faf08009890 tx=0x7faf0802a650 comp rx=0 tx=0).stop 2024-09-16T15:44:46.280 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.278+0000 7faf22e00640 1 -- 172.21.15.151:0/70469363 shutdown_connections 2024-09-16T15:44:46.280 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.278+0000 7faf22e00640 1 --2- 172.21.15.151:0/70469363 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf1c0cf300 0x7faf1c0729a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:46.280 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.278+0000 7faf22e00640 1 -- 172.21.15.151:0/70469363 >> 172.21.15.151:0/70469363 conn(0x7faf1c0b1950 msgr2=0x7faf1c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:46.280 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.279+0000 7faf22e00640 1 -- 172.21.15.151:0/70469363 shutdown_connections 2024-09-16T15:44:46.280 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.279+0000 7faf22e00640 1 -- 172.21.15.151:0/70469363 wait complete. 2024-09-16T15:44:46.281 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.280+0000 7faf22e00640 1 Processor -- start 2024-09-16T15:44:46.281 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.280+0000 7faf22e00640 1 -- start start 2024-09-16T15:44:46.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.281+0000 7faf22e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf1c0ccfd0 0x7faf1c06f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:46.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.281+0000 7faf22e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf1c0cf300 0x7faf1c1f86c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:46.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.281+0000 7faf22e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf1c1445d0 0x7faf1c0b0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:46.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.281+0000 7faf22e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7faf1c0735f0 con 0x7faf1c0cf300 2024-09-16T15:44:46.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.281+0000 7faf22e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7faf1c0b4f30 con 0x7faf1c1445d0 2024-09-16T15:44:46.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.281+0000 7faf22e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7faf1c073770 con 0x7faf1c0ccfd0 2024-09-16T15:44:46.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.282+0000 7faf1b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf1c0cf300 0x7faf1c1f86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:46.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.282+0000 7faf1b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf1c0cf300 0x7faf1c1f86c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44228/0 (socket says 172.21.15.151:44228) 2024-09-16T15:44:46.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.282+0000 7faf1b400640 1 -- 172.21.15.151:0/193637949 learned_addr learned my addr 172.21.15.151:0/193637949 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:46.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.282+0000 7faf1be00640 1 --2- 172.21.15.151:0/193637949 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf1c0ccfd0 0x7faf1c06f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:46.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.282+0000 7faf20e00640 1 --2- 172.21.15.151:0/193637949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf1c1445d0 0x7faf1c0b0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:46.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.282+0000 7faf1b400640 1 -- 172.21.15.151:0/193637949 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf1c0ccfd0 msgr2=0x7faf1c06f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:46.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.282+0000 7faf1b400640 1 --2- 172.21.15.151:0/193637949 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf1c0ccfd0 0x7faf1c06f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:46.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.282+0000 7faf1b400640 1 -- 172.21.15.151:0/193637949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf1c1445d0 msgr2=0x7faf1c0b0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:46.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.282+0000 7faf1b400640 1 --2- 172.21.15.151:0/193637949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf1c1445d0 0x7faf1c0b0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:46.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.282+0000 7faf1b400640 1 -- 172.21.15.151:0/193637949 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf1c1f4c30 con 0x7faf1c0cf300 2024-09-16T15:44:46.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.282+0000 7faf1b400640 1 --2- 172.21.15.151:0/193637949 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf1c0cf300 0x7faf1c1f86c0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7faf080099b0 tx=0x7faf08032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:46.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.283+0000 7faf13e00640 1 -- 172.21.15.151:0/193637949 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf08040070 con 0x7faf1c0cf300 2024-09-16T15:44:46.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.283+0000 7faf13e00640 1 -- 172.21.15.151:0/193637949 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faf08005350 con 0x7faf1c0cf300 2024-09-16T15:44:46.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.283+0000 7faf13e00640 1 -- 172.21.15.151:0/193637949 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf08005c90 con 0x7faf1c0cf300 2024-09-16T15:44:46.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.283+0000 7faf22e00640 1 -- 172.21.15.151:0/193637949 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf1c1f4f10 con 0x7faf1c0cf300 2024-09-16T15:44:46.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.283+0000 7faf22e00640 1 -- 172.21.15.151:0/193637949 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf1c1f54d0 con 0x7faf1c0cf300 2024-09-16T15:44:46.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.284+0000 7faf22e00640 1 -- 172.21.15.151:0/193637949 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf1c073c20 con 0x7faf1c0cf300 2024-09-16T15:44:46.286 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.285+0000 7faf13e00640 1 -- 172.21.15.151:0/193637949 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 102103+0+0 (secure 0 0 0) 0x7faf08039040 con 0x7faf1c0cf300 2024-09-16T15:44:46.287 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.286+0000 7faf13e00640 1 --2- 172.21.15.151:0/193637949 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7faefc079150 0x7faefc07b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:46.287 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.286+0000 7faf13e00640 1 -- 172.21.15.151:0/193637949 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7faf080ba470 con 0x7faf1c0cf300 2024-09-16T15:44:46.287 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.286+0000 7faf1be00640 1 --2- 172.21.15.151:0/193637949 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7faefc079150 0x7faefc07b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:46.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.287+0000 7faf1be00640 1 --2- 172.21.15.151:0/193637949 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7faefc079150 0x7faefc07b610 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7faf1c15f980 tx=0x7faf0c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:46.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.290+0000 7faf13e00640 1 -- 172.21.15.151:0/193637949 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7faf08084530 con 0x7faf1c0cf300 2024-09-16T15:44:46.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.478+0000 7faf22e00640 1 -- 172.21.15.151:0/193637949 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faf1c0cc2f0 con 0x7faf1c0cf300 2024-09-16T15:44:46.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.479+0000 7faf13e00640 1 -- 172.21.15.151:0/193637949 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) ==== 74+0+16841 (secure 0 0 0) 0x7faf08083e30 con 0x7faf1c0cf300 2024-09-16T15:44:46.480 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:44:46.480 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":61,"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","created":"2024-09-16T15:38:56.348535+0000","modified":"2024-09-16T15:44:45.114672+0000","last_up_change":"2024-09-16T15:39:10.681384+0000","last_in_change":"2024-09-16T15:39:00.544477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:39:11.602252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:44:42.866298+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-16T15:44:44.081060+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5524076-a903-4d0a-af6a-caf474c4765d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6809","nonce":3111342741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6811","nonce":3111342741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6815","nonce":3111342741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6813","nonce":3111342741}]},"public_addr":"172.21.15.151:6809/3111342741","cluster_addr":"172.21.15.151:6811/3111342741","heartbeat_back_addr":"172.21.15.151:6815/3111342741","heartbeat_front_addr":"172.21.15.151:6813/3111342741","state":["exists","up"]},{"osd":1,"uuid":"dd25925f-0ae1-494e-8529-4dc205759db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6801","nonce":3368304087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6803","nonce":3368304087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6807","nonce":3368304087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6805","nonce":3368304087}]},"public_addr":"172.21.15.151:6801/3368304087","cluster_addr":"172.21.15.151:6803/3368304087","heartbeat_back_addr":"172.21.15.151:6807/3368304087","heartbeat_front_addr":"172.21.15.151:6805/3368304087","state":["exists","up"]},{"osd":2,"uuid":"c2f18115-013d-4da6-a0fe-cb978f4494b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6823","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6825","nonce":3079373271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6827","nonce":3079373271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6831","nonce":3079373271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6829","nonce":3079373271}]},"public_addr":"172.21.15.151:6825/3079373271","cluster_addr":"172.21.15.151:6827/3079373271","heartbeat_back_addr":"172.21.15.151:6831/3079373271","heartbeat_front_addr":"172.21.15.151:6829/3079373271","state":["exists","up"]},{"osd":3,"uuid":"36edb718-2fcb-4d88-8ddc-c321eed4279d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6817","nonce":1142509041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6819","nonce":1142509041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6824","nonce":1142509041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6821","nonce":1142509041}]},"public_addr":"172.21.15.151:6817/1142509041","cluster_addr":"172.21.15.151:6819/1142509041","heartbeat_back_addr":"172.21.15.151:6824/1142509041","heartbeat_front_addr":"172.21.15.151:6821/1142509041","state":["exists","up"]},{"osd":4,"uuid":"64d33708-cd0a-4236-abe3-1b646e82ab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6811","nonce":2735495271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6813","nonce":2735495271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6817","nonce":2735495271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6815","nonce":2735495271}]},"public_addr":"172.21.15.167:6811/2735495271","cluster_addr":"172.21.15.167:6813/2735495271","heartbeat_back_addr":"172.21.15.167:6817/2735495271","heartbeat_front_addr":"172.21.15.167:6815/2735495271","state":["exists","up"]},{"osd":5,"uuid":"f14b44a9-5910-458c-b518-09a8d0de94de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6827","nonce":1494916572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6829","nonce":1494916572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6833","nonce":1494916572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6831","nonce":1494916572}]},"public_addr":"172.21.15.167:6827/1494916572","cluster_addr":"172.21.15.167:6829/1494916572","heartbeat_back_addr":"172.21.15.167:6833/1494916572","heartbeat_front_addr":"172.21.15.167:6831/1494916572","state":["exists","up"]},{"osd":6,"uuid":"39335545-2654-4710-84b0-a9aa47cfcbba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6819","nonce":2538273273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6821","nonce":2538273273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6825","nonce":2538273273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6823","nonce":2538273273}]},"public_addr":"172.21.15.167:6819/2538273273","cluster_addr":"172.21.15.167:6821/2538273273","heartbeat_back_addr":"172.21.15.167:6825/2538273273","heartbeat_front_addr":"172.21.15.167:6823/2538273273","state":["exists","up"]},{"osd":7,"uuid":"4f8bfc81-53dd-4339-ac22-1a4007d9f963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6803","nonce":265072227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6805","nonce":265072227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6809","nonce":265072227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6807","nonce":265072227}]},"public_addr":"172.21.15.167:6803/265072227","cluster_addr":"172.21.15.167:6805/265072227","heartbeat_back_addr":"172.21.15.167:6809/265072227","heartbeat_front_addr":"172.21.15.167:6807/265072227","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.150359+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.084358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.722368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.332809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.320709+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.395153+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.935250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.801443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:44:46.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.481+0000 7faf22e00640 1 -- 172.21.15.151:0/193637949 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7faefc079150 msgr2=0x7faefc07b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:46.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.481+0000 7faf22e00640 1 --2- 172.21.15.151:0/193637949 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7faefc079150 0x7faefc07b610 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7faf1c15f980 tx=0x7faf0c009040 comp rx=0 tx=0).stop 2024-09-16T15:44:46.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.481+0000 7faf22e00640 1 -- 172.21.15.151:0/193637949 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf1c0cf300 msgr2=0x7faf1c1f86c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:46.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.481+0000 7faf22e00640 1 --2- 172.21.15.151:0/193637949 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf1c0cf300 0x7faf1c1f86c0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7faf080099b0 tx=0x7faf08032c10 comp rx=0 tx=0).stop 2024-09-16T15:44:46.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.481+0000 7faf22e00640 1 -- 172.21.15.151:0/193637949 shutdown_connections 2024-09-16T15:44:46.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.481+0000 7faf22e00640 1 --2- 172.21.15.151:0/193637949 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7faefc079150 0x7faefc07b610 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:46.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.481+0000 7faf22e00640 1 --2- 172.21.15.151:0/193637949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf1c1445d0 0x7faf1c0b0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:46.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.481+0000 7faf22e00640 1 --2- 172.21.15.151:0/193637949 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf1c0cf300 0x7faf1c1f86c0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:46.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.481+0000 7faf22e00640 1 --2- 172.21.15.151:0/193637949 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf1c0ccfd0 0x7faf1c06f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:46.484 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.481+0000 7faf22e00640 1 -- 172.21.15.151:0/193637949 >> 172.21.15.151:0/193637949 conn(0x7faf1c0b1950 msgr2=0x7faf1c0aed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:46.484 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.482+0000 7faf22e00640 1 -- 172.21.15.151:0/193637949 shutdown_connections 2024-09-16T15:44:46.484 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.482+0000 7faf22e00640 1 -- 172.21.15.151:0/193637949 wait complete. 2024-09-16T15:44:46.498 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-16T15:44:46.660 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.659+0000 7faa58200640 1 -- 172.21.15.151:0/774698853 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faa500cccd0 msgr2=0x7faa500cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:46.660 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.659+0000 7faa58200640 1 --2- 172.21.15.151:0/774698853 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faa500cccd0 0x7faa500cdbb0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7faa500d0cd0 tx=0x7faa4802a560 comp rx=0 tx=0).stop 2024-09-16T15:44:46.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.659+0000 7faa58200640 1 -- 172.21.15.151:0/774698853 shutdown_connections 2024-09-16T15:44:46.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.659+0000 7faa58200640 1 --2- 172.21.15.151:0/774698853 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faa500cccd0 0x7faa500cdbb0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:46.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.659+0000 7faa58200640 1 --2- 172.21.15.151:0/774698853 >> v2:172.21.15.167:3300/0 conn(0x7faa50145750 0x7faa50073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:46.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.659+0000 7faa58200640 1 -- 172.21.15.151:0/774698853 >> 172.21.15.151:0/774698853 conn(0x7faa500b1840 msgr2=0x7faa500b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:46.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.659+0000 7faa58200640 1 -- 172.21.15.151:0/774698853 shutdown_connections 2024-09-16T15:44:46.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.659+0000 7faa58200640 1 -- 172.21.15.151:0/774698853 wait complete. 2024-09-16T15:44:46.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.661+0000 7faa58200640 1 Processor -- start 2024-09-16T15:44:46.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.661+0000 7faa58200640 1 -- start start 2024-09-16T15:44:46.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.662+0000 7faa58200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa5006efa0 0x7faa501428f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:46.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.662+0000 7faa58200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faa500cccd0 0x7faa501f8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:46.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.662+0000 7faa58200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faa50145750 0x7faa501f86c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:46.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.662+0000 7faa58200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7faa500ce480 con 0x7faa50145750 2024-09-16T15:44:46.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.662+0000 7faa58200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7faa500ce300 con 0x7faa5006efa0 2024-09-16T15:44:46.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.662+0000 7faa58200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7faa500ce600 con 0x7faa500cccd0 2024-09-16T15:44:46.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.662+0000 7faa56200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faa50145750 0x7faa501f86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:46.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.662+0000 7faa56200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faa50145750 0x7faa501f86c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44234/0 (socket says 172.21.15.151:44234) 2024-09-16T15:44:46.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.662+0000 7faa56200640 1 -- 172.21.15.151:0/3440952777 learned_addr learned my addr 172.21.15.151:0/3440952777 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:46.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.662+0000 7faa54e00640 1 --2- 172.21.15.151:0/3440952777 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faa500cccd0 0x7faa501f8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:46.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.662+0000 7faa55800640 1 --2- 172.21.15.151:0/3440952777 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa5006efa0 0x7faa501428f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:46.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.663+0000 7faa56200640 1 -- 172.21.15.151:0/3440952777 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faa500cccd0 msgr2=0x7faa501f8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:46.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.663+0000 7faa56200640 1 --2- 172.21.15.151:0/3440952777 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faa500cccd0 0x7faa501f8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:46.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.663+0000 7faa56200640 1 -- 172.21.15.151:0/3440952777 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa5006efa0 msgr2=0x7faa501428f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:46.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.663+0000 7faa56200640 1 --2- 172.21.15.151:0/3440952777 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa5006efa0 0x7faa501428f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:46.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.663+0000 7faa56200640 1 -- 172.21.15.151:0/3440952777 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa500b4230 con 0x7faa50145750 2024-09-16T15:44:46.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.663+0000 7faa56200640 1 --2- 172.21.15.151:0/3440952777 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faa50145750 0x7faa501f86c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7faa480099c0 tx=0x7faa480039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:46.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.663+0000 7faa4d600640 1 -- 172.21.15.151:0/3440952777 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4803f070 con 0x7faa50145750 2024-09-16T15:44:46.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.663+0000 7faa4d600640 1 -- 172.21.15.151:0/3440952777 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faa480048b0 con 0x7faa50145750 2024-09-16T15:44:46.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.663+0000 7faa4d600640 1 -- 172.21.15.151:0/3440952777 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa48035030 con 0x7faa50145750 2024-09-16T15:44:46.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.663+0000 7faa58200640 1 -- 172.21.15.151:0/3440952777 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa500b4510 con 0x7faa50145750 2024-09-16T15:44:46.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.663+0000 7faa58200640 1 -- 172.21.15.151:0/3440952777 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa50140740 con 0x7faa50145750 2024-09-16T15:44:46.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.665+0000 7faa58200640 1 -- 172.21.15.151:0/3440952777 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa500cdbb0 con 0x7faa50145750 2024-09-16T15:44:46.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.666+0000 7faa4d600640 1 -- 172.21.15.151:0/3440952777 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 102103+0+0 (secure 0 0 0) 0x7faa48042070 con 0x7faa50145750 2024-09-16T15:44:46.668 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.667+0000 7faa4d600640 1 --2- 172.21.15.151:0/3440952777 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7faa30079180 0x7faa3007b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:46.668 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.667+0000 7faa4d600640 1 -- 172.21.15.151:0/3440952777 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5737+0+0 (secure 0 0 0) 0x7faa480b9fc0 con 0x7faa50145750 2024-09-16T15:44:46.668 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.667+0000 7faa55800640 1 --2- 172.21.15.151:0/3440952777 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7faa30079180 0x7faa3007b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:46.668 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.668+0000 7faa55800640 1 --2- 172.21.15.151:0/3440952777 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7faa30079180 0x7faa3007b640 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7faa4400ae50 tx=0x7faa4400a540 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:46.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.671+0000 7faa4d600640 1 -- 172.21.15.151:0/3440952777 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7faa48083a90 con 0x7faa50145750 2024-09-16T15:44:46.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:46.901+0000 7faa58200640 1 -- 172.21.15.151:0/3440952777 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7faa500cbc00 con 0x7faa50145750 2024-09-16T15:44:47.674 INFO:tasks.ceph.mds.c.smithi151.stderr:2024-09-16T15:44:47.673+0000 7f80f4a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-16T15:44:47.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:47.723+0000 7faa4d600640 1 -- 172.21.15.151:0/3440952777 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v35) ==== 118+0+0 (secure 0 0 0) 0x7faa480884c0 con 0x7faa50145750 2024-09-16T15:44:47.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:47.727+0000 7faa58200640 1 -- 172.21.15.151:0/3440952777 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7faa30079180 msgr2=0x7faa3007b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:47.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:47.727+0000 7faa58200640 1 --2- 172.21.15.151:0/3440952777 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7faa30079180 0x7faa3007b640 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7faa4400ae50 tx=0x7faa4400a540 comp rx=0 tx=0).stop 2024-09-16T15:44:47.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:47.727+0000 7faa58200640 1 -- 172.21.15.151:0/3440952777 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faa50145750 msgr2=0x7faa501f86c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:47.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:47.727+0000 7faa58200640 1 --2- 172.21.15.151:0/3440952777 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faa50145750 0x7faa501f86c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7faa480099c0 tx=0x7faa480039e0 comp rx=0 tx=0).stop 2024-09-16T15:44:47.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:47.727+0000 7faa58200640 1 -- 172.21.15.151:0/3440952777 shutdown_connections 2024-09-16T15:44:47.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:47.727+0000 7faa58200640 1 --2- 172.21.15.151:0/3440952777 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7faa30079180 0x7faa3007b640 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:47.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:47.727+0000 7faa58200640 1 --2- 172.21.15.151:0/3440952777 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faa50145750 0x7faa501f86c0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:47.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:47.727+0000 7faa58200640 1 --2- 172.21.15.151:0/3440952777 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faa500cccd0 0x7faa501f8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:47.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:47.727+0000 7faa58200640 1 --2- 172.21.15.151:0/3440952777 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa5006efa0 0x7faa501428f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:47.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:47.727+0000 7faa58200640 1 -- 172.21.15.151:0/3440952777 >> 172.21.15.151:0/3440952777 conn(0x7faa500b1840 msgr2=0x7faa500a9b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:47.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:47.727+0000 7faa58200640 1 -- 172.21.15.151:0/3440952777 shutdown_connections 2024-09-16T15:44:47.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:47.727+0000 7faa58200640 1 -- 172.21.15.151:0/3440952777 wait complete. 2024-09-16T15:44:47.746 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-16T15:44:47.747 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-16T15:44:47.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:47.907+0000 7f8ee0a00640 1 -- 172.21.15.151:0/3126163739 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8edc161760 msgr2=0x7f8edc0ce4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:47.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:47.907+0000 7f8ee0a00640 1 --2- 172.21.15.151:0/3126163739 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8edc161760 0x7f8edc0ce4a0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f8edc0b2860 tx=0x7f8ecc02a540 comp rx=0 tx=0).stop 2024-09-16T15:44:47.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:47.907+0000 7f8ee0a00640 1 -- 172.21.15.151:0/3126163739 shutdown_connections 2024-09-16T15:44:47.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:47.907+0000 7f8ee0a00640 1 --2- 172.21.15.151:0/3126163739 >> v2:172.21.15.167:3300/0 conn(0x7f8edc0af190 0x7f8edc0b2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:47.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:47.907+0000 7f8ee0a00640 1 --2- 172.21.15.151:0/3126163739 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8edc161760 0x7f8edc0ce4a0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:47.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:47.907+0000 7f8ee0a00640 1 -- 172.21.15.151:0/3126163739 >> 172.21.15.151:0/3126163739 conn(0x7f8edc0b1840 msgr2=0x7f8edc0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:47.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:47.907+0000 7f8ee0a00640 1 -- 172.21.15.151:0/3126163739 shutdown_connections 2024-09-16T15:44:47.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:47.907+0000 7f8ee0a00640 1 -- 172.21.15.151:0/3126163739 wait complete. 2024-09-16T15:44:47.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:47.909+0000 7f8ee0a00640 1 Processor -- start 2024-09-16T15:44:47.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:47.909+0000 7f8ee0a00640 1 -- start start 2024-09-16T15:44:47.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:47.910+0000 7f8ee0a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8edc0af190 0x7f8edc156c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:47.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:47.910+0000 7f8ee0a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8edc0cec60 0x7f8edc1452b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:47.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:47.910+0000 7f8ee0a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8edc161760 0x7f8edc1457f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:47.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:47.910+0000 7f8ee0a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8edc162310 con 0x7f8edc161760 2024-09-16T15:44:47.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:47.910+0000 7f8ee0a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8edc162190 con 0x7f8edc0cec60 2024-09-16T15:44:47.912 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:47.910+0000 7f8ee0a00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f8edc162490 con 0x7f8edc0af190 2024-09-16T15:44:47.912 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:47.910+0000 7f8eda800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8edc161760 0x7f8edc1457f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:47.912 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:47.910+0000 7f8eda800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8edc161760 0x7f8edc1457f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44250/0 (socket says 172.21.15.151:44250) 2024-09-16T15:44:47.912 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:47.910+0000 7f8eda800640 1 -- 172.21.15.151:0/3149040594 learned_addr learned my addr 172.21.15.151:0/3149040594 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:47.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:47.911+0000 7f8ed9e00640 1 --2- 172.21.15.151:0/3149040594 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8edc0af190 0x7f8edc156c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:47.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:47.911+0000 7f8ed9400640 1 --2- 172.21.15.151:0/3149040594 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8edc0cec60 0x7f8edc1452b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:47.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:47.911+0000 7f8eda800640 1 -- 172.21.15.151:0/3149040594 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8edc0af190 msgr2=0x7f8edc156c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:47.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:47.911+0000 7f8eda800640 1 --2- 172.21.15.151:0/3149040594 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8edc0af190 0x7f8edc156c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:47.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:47.911+0000 7f8eda800640 1 -- 172.21.15.151:0/3149040594 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8edc0cec60 msgr2=0x7f8edc1452b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:47.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:47.911+0000 7f8eda800640 1 --2- 172.21.15.151:0/3149040594 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8edc0cec60 0x7f8edc1452b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:47.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:47.911+0000 7f8eda800640 1 -- 172.21.15.151:0/3149040594 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8edc1f90d0 con 0x7f8edc161760 2024-09-16T15:44:47.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:47.911+0000 7f8ed9e00640 1 --2- 172.21.15.151:0/3149040594 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8edc0af190 0x7f8edc156c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:44:47.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:47.911+0000 7f8ed9400640 1 --2- 172.21.15.151:0/3149040594 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8edc0cec60 0x7f8edc1452b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:44:47.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:47.911+0000 7f8eda800640 1 --2- 172.21.15.151:0/3149040594 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8edc161760 0x7f8edc1457f0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f8ed400cb50 tx=0x7f8ed400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:47.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:47.911+0000 7f8ed2000640 1 -- 172.21.15.151:0/3149040594 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed4016020 con 0x7f8edc161760 2024-09-16T15:44:47.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:47.911+0000 7f8ed2000640 1 -- 172.21.15.151:0/3149040594 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ed400ae30 con 0x7f8edc161760 2024-09-16T15:44:47.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:47.911+0000 7f8ee0a00640 1 -- 172.21.15.151:0/3149040594 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8edc1f4fe0 con 0x7f8edc161760 2024-09-16T15:44:47.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:47.911+0000 7f8ed2000640 1 -- 172.21.15.151:0/3149040594 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed4012660 con 0x7f8edc161760 2024-09-16T15:44:47.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:47.911+0000 7f8ee0a00640 1 -- 172.21.15.151:0/3149040594 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8edc1f4cb0 con 0x7f8edc161760 2024-09-16T15:44:47.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:47.912+0000 7f8ee0a00640 1 -- 172.21.15.151:0/3149040594 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8edc0ce4a0 con 0x7f8edc161760 2024-09-16T15:44:47.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:47.914+0000 7f8ed2000640 1 -- 172.21.15.151:0/3149040594 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 102103+0+0 (secure 0 0 0) 0x7f8ed4019020 con 0x7f8edc161760 2024-09-16T15:44:47.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:47.914+0000 7f8ed2000640 1 --2- 172.21.15.151:0/3149040594 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8ebc079150 0x7f8ebc07b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:47.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:47.915+0000 7f8ed2000640 1 -- 172.21.15.151:0/3149040594 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7f8ed4099580 con 0x7f8edc161760 2024-09-16T15:44:47.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:47.915+0000 7f8ed9e00640 1 --2- 172.21.15.151:0/3149040594 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8ebc079150 0x7f8ebc07b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:47.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:47.916+0000 7f8ed9e00640 1 --2- 172.21.15.151:0/3149040594 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8ebc079150 0x7f8ebc07b610 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc034000 tx=0x7f8ecc004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:47.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:47.919+0000 7f8ed2000640 1 -- 172.21.15.151:0/3149040594 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8ed4020030 con 0x7f8edc161760 2024-09-16T15:44:47.981 DEBUG:teuthology.orchestra.run.smithi151:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:44:48.024 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:44:48.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:48.148+0000 7f8ee0a00640 1 -- 172.21.15.151:0/3149040594 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f8edc0afe10 con 0x7f8edc161760 2024-09-16T15:44:48.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:48.724+0000 7f8ed2000640 1 -- 172.21.15.151:0/3149040594 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v36) ==== 122+0+0 (secure 0 0 0) 0x7f8ed4063640 con 0x7f8edc161760 2024-09-16T15:44:48.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:48.727+0000 7f8ee0a00640 1 -- 172.21.15.151:0/3149040594 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8ebc079150 msgr2=0x7f8ebc07b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:48.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:48.727+0000 7f8ee0a00640 1 --2- 172.21.15.151:0/3149040594 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8ebc079150 0x7f8ebc07b610 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc034000 tx=0x7f8ecc004730 comp rx=0 tx=0).stop 2024-09-16T15:44:48.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:48.727+0000 7f8ee0a00640 1 -- 172.21.15.151:0/3149040594 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8edc161760 msgr2=0x7f8edc1457f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:48.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:48.727+0000 7f8ee0a00640 1 --2- 172.21.15.151:0/3149040594 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8edc161760 0x7f8edc1457f0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f8ed400cb50 tx=0x7f8ed400a430 comp rx=0 tx=0).stop 2024-09-16T15:44:48.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:48.727+0000 7f8ee0a00640 1 -- 172.21.15.151:0/3149040594 shutdown_connections 2024-09-16T15:44:48.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:48.727+0000 7f8ee0a00640 1 --2- 172.21.15.151:0/3149040594 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8ebc079150 0x7f8ebc07b610 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:48.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:48.727+0000 7f8ee0a00640 1 --2- 172.21.15.151:0/3149040594 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8edc0cec60 0x7f8edc1452b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:48.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:48.727+0000 7f8ee0a00640 1 --2- 172.21.15.151:0/3149040594 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8edc161760 0x7f8edc1457f0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:48.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:48.727+0000 7f8ee0a00640 1 --2- 172.21.15.151:0/3149040594 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8edc0af190 0x7f8edc156c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:48.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:48.727+0000 7f8ee0a00640 1 -- 172.21.15.151:0/3149040594 >> 172.21.15.151:0/3149040594 conn(0x7f8edc0b1840 msgr2=0x7f8edc0ad870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:48.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:48.727+0000 7f8ee0a00640 1 -- 172.21.15.151:0/3149040594 shutdown_connections 2024-09-16T15:44:48.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:48.727+0000 7f8ee0a00640 1 -- 172.21.15.151:0/3149040594 wait complete. 2024-09-16T15:44:48.746 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-16T15:44:48.746 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:44:48.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:48.944+0000 7f1fe4a00640 1 -- 172.21.15.151:0/2497754317 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1fc4005280 msgr2=0x7f1fc400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:48.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:48.944+0000 7f1fe4a00640 1 --2- 172.21.15.151:0/2497754317 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1fc4005280 0x7f1fc400b9c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8001830 tx=0x7f1fc4005830 comp rx=0 tx=0).stop 2024-09-16T15:44:48.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:48.944+0000 7f1fe4a00640 1 -- 172.21.15.151:0/2497754317 shutdown_connections 2024-09-16T15:44:48.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:48.944+0000 7f1fe4a00640 1 --2- 172.21.15.151:0/2497754317 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1fc4005280 0x7f1fc400b9c0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:48.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:48.944+0000 7f1fe4a00640 1 -- 172.21.15.151:0/2497754317 >> 172.21.15.151:0/2497754317 conn(0x7f1fe00b1950 msgr2=0x7f1fe00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:48.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:48.944+0000 7f1fe4a00640 1 -- 172.21.15.151:0/2497754317 shutdown_connections 2024-09-16T15:44:48.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:48.944+0000 7f1fe4a00640 1 -- 172.21.15.151:0/2497754317 wait complete. 2024-09-16T15:44:48.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:48.946+0000 7f1fe4a00640 1 Processor -- start 2024-09-16T15:44:48.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:48.946+0000 7f1fe4a00640 1 -- start start 2024-09-16T15:44:48.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:48.947+0000 7f1fe4a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1fc4005280 0x7f1fe01f4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:48.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:48.947+0000 7f1fe4a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1fe009c220 0x7f1fe015a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:48.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:48.947+0000 7f1fe4a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1fe009c6a0 0x7f1fe014f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:48.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:48.947+0000 7f1fe4a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe0074420 con 0x7f1fe009c6a0 2024-09-16T15:44:48.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:48.947+0000 7f1fe4a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe00b26a0 con 0x7f1fc4005280 2024-09-16T15:44:48.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:48.947+0000 7f1fe4a00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f1fe00745a0 con 0x7f1fe009c220 2024-09-16T15:44:48.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:48.947+0000 7f1fde800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1fe009c6a0 0x7f1fe014f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:48.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:48.947+0000 7f1fde800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1fe009c6a0 0x7f1fe014f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44266/0 (socket says 172.21.15.151:44266) 2024-09-16T15:44:48.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:48.947+0000 7f1fde800640 1 -- 172.21.15.151:0/4106050179 learned_addr learned my addr 172.21.15.151:0/4106050179 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:48.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:48.947+0000 7f1fdde00640 1 --2- 172.21.15.151:0/4106050179 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1fc4005280 0x7f1fe01f4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:48.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:48.947+0000 7f1fdd400640 1 --2- 172.21.15.151:0/4106050179 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1fe009c220 0x7f1fe015a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:48.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:48.947+0000 7f1fde800640 1 -- 172.21.15.151:0/4106050179 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1fe009c220 msgr2=0x7f1fe015a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:48.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:48.947+0000 7f1fde800640 1 --2- 172.21.15.151:0/4106050179 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1fe009c220 0x7f1fe015a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:48.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:48.947+0000 7f1fde800640 1 -- 172.21.15.151:0/4106050179 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1fc4005280 msgr2=0x7f1fe01f4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:48.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:48.947+0000 7f1fde800640 1 --2- 172.21.15.151:0/4106050179 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1fc4005280 0x7f1fe01f4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:48.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:48.947+0000 7f1fde800640 1 -- 172.21.15.151:0/4106050179 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fe015b190 con 0x7f1fe009c6a0 2024-09-16T15:44:48.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:48.947+0000 7f1fdde00640 1 --2- 172.21.15.151:0/4106050179 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1fc4005280 0x7f1fe01f4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:44:48.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:48.948+0000 7f1fdd400640 1 --2- 172.21.15.151:0/4106050179 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1fe009c220 0x7f1fe015a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:44:48.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:48.948+0000 7f1fde800640 1 --2- 172.21.15.151:0/4106050179 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1fe009c6a0 0x7f1fe014f250 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8001950 tx=0x7f1fd8033aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:48.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:48.948+0000 7f1fd6000640 1 -- 172.21.15.151:0/4106050179 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd80068c0 con 0x7f1fe009c6a0 2024-09-16T15:44:48.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:48.948+0000 7f1fd6000640 1 -- 172.21.15.151:0/4106050179 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1fd80035f0 con 0x7f1fe009c6a0 2024-09-16T15:44:48.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:48.948+0000 7f1fe4a00640 1 -- 172.21.15.151:0/4106050179 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fe015b470 con 0x7f1fe009c6a0 2024-09-16T15:44:48.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:48.948+0000 7f1fd6000640 1 -- 172.21.15.151:0/4106050179 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd8003a40 con 0x7f1fe009c6a0 2024-09-16T15:44:48.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:48.948+0000 7f1fe4a00640 1 -- 172.21.15.151:0/4106050179 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fe01f5130 con 0x7f1fe009c6a0 2024-09-16T15:44:48.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:48.949+0000 7f1fe4a00640 1 -- 172.21.15.151:0/4106050179 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fe015b6b0 con 0x7f1fe009c6a0 2024-09-16T15:44:48.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:48.950+0000 7f1fd6000640 1 -- 172.21.15.151:0/4106050179 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 102103+0+0 (secure 0 0 0) 0x7f1fd803d3f0 con 0x7f1fe009c6a0 2024-09-16T15:44:48.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:48.951+0000 7f1fd6000640 1 --2- 172.21.15.151:0/4106050179 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1fc0079150 0x7f1fc007b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:48.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:48.951+0000 7f1fd6000640 1 -- 172.21.15.151:0/4106050179 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7f1fd80b93e0 con 0x7f1fe009c6a0 2024-09-16T15:44:48.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:48.952+0000 7f1fdde00640 1 --2- 172.21.15.151:0/4106050179 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1fc0079150 0x7f1fc007b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:48.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:48.953+0000 7f1fdde00640 1 --2- 172.21.15.151:0/4106050179 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1fc0079150 0x7f1fc007b610 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1fd0005cc0 tx=0x7f1fd0009500 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:48.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:48.955+0000 7f1fd6000640 1 -- 172.21.15.151:0/4106050179 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1fe015b6b0 con 0x7f1fe009c6a0 2024-09-16T15:44:49.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.177+0000 7f1fe4a00640 1 -- 172.21.15.151:0/4106050179 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1fe00acd30 con 0x7f1fe009c6a0 2024-09-16T15:44:49.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.178+0000 7f1fd6000640 1 -- 172.21.15.151:0/4106050179 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5232 (secure 0 0 0) 0x7f1fd80834a0 con 0x7f1fe009c6a0 2024-09-16T15:44:49.179 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:44:49.179 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":36,"btime":"2024-09-16T15:44:48:723811+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5357,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1715048770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1715048770},{"type":"v1","addr":"172.21.15.167:6837","nonce":1715048770}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":5722,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/784205427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":784205427},{"type":"v1","addr":"172.21.15.167:6836","nonce":784205427}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":5958,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/3837674321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3837674321},{"type":"v1","addr":"172.21.15.151:6834","nonce":3837674321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:44:45.114621+0000","modified":"2024-09-16T15:44:48.149095+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5961},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5961":{"gid":5961,"name":"c","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.151:6835/1724192711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":1724192711},{"type":"v1","addr":"172.21.15.151:6835","nonce":1724192711}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5961,"qdb_cluster":[5961]},"id":4}]} 2024-09-16T15:44:49.179 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 36 2024-09-16T15:44:49.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.180+0000 7f1fe4a00640 1 -- 172.21.15.151:0/4106050179 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1fc0079150 msgr2=0x7f1fc007b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:49.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.180+0000 7f1fe4a00640 1 --2- 172.21.15.151:0/4106050179 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1fc0079150 0x7f1fc007b610 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1fd0005cc0 tx=0x7f1fd0009500 comp rx=0 tx=0).stop 2024-09-16T15:44:49.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.180+0000 7f1fe4a00640 1 -- 172.21.15.151:0/4106050179 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1fe009c6a0 msgr2=0x7f1fe014f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:49.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.180+0000 7f1fe4a00640 1 --2- 172.21.15.151:0/4106050179 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1fe009c6a0 0x7f1fe014f250 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8001950 tx=0x7f1fd8033aa0 comp rx=0 tx=0).stop 2024-09-16T15:44:49.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.180+0000 7f1fe4a00640 1 -- 172.21.15.151:0/4106050179 shutdown_connections 2024-09-16T15:44:49.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.180+0000 7f1fe4a00640 1 --2- 172.21.15.151:0/4106050179 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1fc0079150 0x7f1fc007b610 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:49.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.180+0000 7f1fe4a00640 1 --2- 172.21.15.151:0/4106050179 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1fe009c6a0 0x7f1fe014f250 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:49.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.180+0000 7f1fe4a00640 1 --2- 172.21.15.151:0/4106050179 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1fe009c220 0x7f1fe015a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:49.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.180+0000 7f1fe4a00640 1 --2- 172.21.15.151:0/4106050179 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1fc4005280 0x7f1fe01f4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:49.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.180+0000 7f1fe4a00640 1 -- 172.21.15.151:0/4106050179 >> 172.21.15.151:0/4106050179 conn(0x7f1fe00b1950 msgr2=0x7f1fe00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:49.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.180+0000 7f1fe4a00640 1 -- 172.21.15.151:0/4106050179 shutdown_connections 2024-09-16T15:44:49.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.180+0000 7f1fe4a00640 1 -- 172.21.15.151:0/4106050179 wait complete. 2024-09-16T15:44:49.198 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:44:49.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.353+0000 7f6a60000640 1 -- 172.21.15.151:0/4228958632 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6a5815f5a0 msgr2=0x7f6a580b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:49.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.353+0000 7f6a60000640 1 --2- 172.21.15.151:0/4228958632 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6a5815f5a0 0x7f6a580b2440 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c009860 tx=0x7f6a4c02a510 comp rx=0 tx=0).stop 2024-09-16T15:44:49.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.353+0000 7f6a60000640 1 -- 172.21.15.151:0/4228958632 shutdown_connections 2024-09-16T15:44:49.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.353+0000 7f6a60000640 1 --2- 172.21.15.151:0/4228958632 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6a5815f5a0 0x7f6a580b2440 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:49.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.353+0000 7f6a60000640 1 -- 172.21.15.151:0/4228958632 >> 172.21.15.151:0/4228958632 conn(0x7f6a580b1950 msgr2=0x7f6a580b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:49.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.353+0000 7f6a60000640 1 -- 172.21.15.151:0/4228958632 shutdown_connections 2024-09-16T15:44:49.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.353+0000 7f6a60000640 1 -- 172.21.15.151:0/4228958632 wait complete. 2024-09-16T15:44:49.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.355+0000 7f6a60000640 1 Processor -- start 2024-09-16T15:44:49.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.355+0000 7f6a60000640 1 -- start start 2024-09-16T15:44:49.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.356+0000 7f6a60000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a58073fb0 0x7f6a580cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:49.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.356+0000 7f6a60000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6a580ccf00 0x7f6a580b0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:49.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.356+0000 7f6a60000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a5815f5a0 0x7f6a580b1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:49.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.356+0000 7f6a60000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6a5806ef30 con 0x7f6a5815f5a0 2024-09-16T15:44:49.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.356+0000 7f6a60000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6a5806fa80 con 0x7f6a58073fb0 2024-09-16T15:44:49.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.356+0000 7f6a60000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f6a58074460 con 0x7f6a580ccf00 2024-09-16T15:44:49.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.356+0000 7f6a5e000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a5815f5a0 0x7f6a580b1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:49.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.356+0000 7f6a5e000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a5815f5a0 0x7f6a580b1330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44278/0 (socket says 172.21.15.151:44278) 2024-09-16T15:44:49.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.356+0000 7f6a5e000640 1 -- 172.21.15.151:0/891229322 learned_addr learned my addr 172.21.15.151:0/891229322 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:49.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.357+0000 7f6a5d600640 1 --2- 172.21.15.151:0/891229322 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a58073fb0 0x7f6a580cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:49.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.357+0000 7f6a5cc00640 1 --2- 172.21.15.151:0/891229322 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6a580ccf00 0x7f6a580b0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:49.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.357+0000 7f6a5e000640 1 -- 172.21.15.151:0/891229322 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6a580ccf00 msgr2=0x7f6a580b0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:49.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.357+0000 7f6a5e000640 1 --2- 172.21.15.151:0/891229322 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6a580ccf00 0x7f6a580b0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:49.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.357+0000 7f6a5e000640 1 -- 172.21.15.151:0/891229322 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a58073fb0 msgr2=0x7f6a580cb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:49.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.357+0000 7f6a5e000640 1 --2- 172.21.15.151:0/891229322 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a58073fb0 0x7f6a580cb9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:49.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.357+0000 7f6a5e000640 1 -- 172.21.15.151:0/891229322 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a581f45c0 con 0x7f6a5815f5a0 2024-09-16T15:44:49.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.357+0000 7f6a5e000640 1 --2- 172.21.15.151:0/891229322 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a5815f5a0 0x7f6a580b1330 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f6a5000c7a0 tx=0x7f6a5000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:49.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.357+0000 7f6a55600640 1 -- 172.21.15.151:0/891229322 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a50003b70 con 0x7f6a5815f5a0 2024-09-16T15:44:49.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.357+0000 7f6a55600640 1 -- 172.21.15.151:0/891229322 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6a50003d10 con 0x7f6a5815f5a0 2024-09-16T15:44:49.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.357+0000 7f6a55600640 1 -- 172.21.15.151:0/891229322 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a5000aa50 con 0x7f6a5815f5a0 2024-09-16T15:44:49.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.357+0000 7f6a60000640 1 -- 172.21.15.151:0/891229322 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a581f88d0 con 0x7f6a5815f5a0 2024-09-16T15:44:49.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.358+0000 7f6a60000640 1 -- 172.21.15.151:0/891229322 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a581f4b10 con 0x7f6a5815f5a0 2024-09-16T15:44:49.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.358+0000 7f6a60000640 1 -- 172.21.15.151:0/891229322 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a580b2440 con 0x7f6a5815f5a0 2024-09-16T15:44:49.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.360+0000 7f6a55600640 1 -- 172.21.15.151:0/891229322 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 102103+0+0 (secure 0 0 0) 0x7f6a5000abf0 con 0x7f6a5815f5a0 2024-09-16T15:44:49.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.361+0000 7f6a55600640 1 --2- 172.21.15.151:0/891229322 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6a38079150 0x7f6a3807b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:49.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.361+0000 7f6a55600640 1 -- 172.21.15.151:0/891229322 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7f6a50099020 con 0x7f6a5815f5a0 2024-09-16T15:44:49.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.361+0000 7f6a5d600640 1 --2- 172.21.15.151:0/891229322 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6a38079150 0x7f6a3807b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:49.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.362+0000 7f6a5d600640 1 --2- 172.21.15.151:0/891229322 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6a38079150 0x7f6a3807b610 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c034000 tx=0x7f6a4c004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:49.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.365+0000 7f6a55600640 1 -- 172.21.15.151:0/891229322 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6a50063140 con 0x7f6a5815f5a0 2024-09-16T15:44:49.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.550+0000 7f6a60000640 1 -- 172.21.15.151:0/891229322 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6a58069ac0 con 0x7f6a5815f5a0 2024-09-16T15:44:49.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.551+0000 7f6a55600640 1 -- 172.21.15.151:0/891229322 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) ==== 74+0+16828 (secure 0 0 0) 0x7f6a50062a40 con 0x7f6a5815f5a0 2024-09-16T15:44:49.552 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:44:49.552 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":62,"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","created":"2024-09-16T15:38:56.348535+0000","modified":"2024-09-16T15:44:46.703302+0000","last_up_change":"2024-09-16T15:39:10.681384+0000","last_in_change":"2024-09-16T15:39:00.544477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:39:11.602252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:44:42.866298+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-16T15:44:44.081060+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5524076-a903-4d0a-af6a-caf474c4765d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6809","nonce":3111342741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6811","nonce":3111342741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6815","nonce":3111342741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6813","nonce":3111342741}]},"public_addr":"172.21.15.151:6809/3111342741","cluster_addr":"172.21.15.151:6811/3111342741","heartbeat_back_addr":"172.21.15.151:6815/3111342741","heartbeat_front_addr":"172.21.15.151:6813/3111342741","state":["exists","up"]},{"osd":1,"uuid":"dd25925f-0ae1-494e-8529-4dc205759db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6801","nonce":3368304087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6803","nonce":3368304087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6807","nonce":3368304087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6805","nonce":3368304087}]},"public_addr":"172.21.15.151:6801/3368304087","cluster_addr":"172.21.15.151:6803/3368304087","heartbeat_back_addr":"172.21.15.151:6807/3368304087","heartbeat_front_addr":"172.21.15.151:6805/3368304087","state":["exists","up"]},{"osd":2,"uuid":"c2f18115-013d-4da6-a0fe-cb978f4494b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6823","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6825","nonce":3079373271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6827","nonce":3079373271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6831","nonce":3079373271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6829","nonce":3079373271}]},"public_addr":"172.21.15.151:6825/3079373271","cluster_addr":"172.21.15.151:6827/3079373271","heartbeat_back_addr":"172.21.15.151:6831/3079373271","heartbeat_front_addr":"172.21.15.151:6829/3079373271","state":["exists","up"]},{"osd":3,"uuid":"36edb718-2fcb-4d88-8ddc-c321eed4279d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6817","nonce":1142509041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6819","nonce":1142509041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6824","nonce":1142509041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6821","nonce":1142509041}]},"public_addr":"172.21.15.151:6817/1142509041","cluster_addr":"172.21.15.151:6819/1142509041","heartbeat_back_addr":"172.21.15.151:6824/1142509041","heartbeat_front_addr":"172.21.15.151:6821/1142509041","state":["exists","up"]},{"osd":4,"uuid":"64d33708-cd0a-4236-abe3-1b646e82ab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6811","nonce":2735495271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6813","nonce":2735495271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6817","nonce":2735495271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6815","nonce":2735495271}]},"public_addr":"172.21.15.167:6811/2735495271","cluster_addr":"172.21.15.167:6813/2735495271","heartbeat_back_addr":"172.21.15.167:6817/2735495271","heartbeat_front_addr":"172.21.15.167:6815/2735495271","state":["exists","up"]},{"osd":5,"uuid":"f14b44a9-5910-458c-b518-09a8d0de94de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6827","nonce":1494916572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6829","nonce":1494916572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6833","nonce":1494916572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6831","nonce":1494916572}]},"public_addr":"172.21.15.167:6827/1494916572","cluster_addr":"172.21.15.167:6829/1494916572","heartbeat_back_addr":"172.21.15.167:6833/1494916572","heartbeat_front_addr":"172.21.15.167:6831/1494916572","state":["exists","up"]},{"osd":6,"uuid":"39335545-2654-4710-84b0-a9aa47cfcbba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6819","nonce":2538273273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6821","nonce":2538273273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6825","nonce":2538273273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6823","nonce":2538273273}]},"public_addr":"172.21.15.167:6819/2538273273","cluster_addr":"172.21.15.167:6821/2538273273","heartbeat_back_addr":"172.21.15.167:6825/2538273273","heartbeat_front_addr":"172.21.15.167:6823/2538273273","state":["exists","up"]},{"osd":7,"uuid":"4f8bfc81-53dd-4339-ac22-1a4007d9f963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6803","nonce":265072227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6805","nonce":265072227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6809","nonce":265072227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6807","nonce":265072227}]},"public_addr":"172.21.15.167:6803/265072227","cluster_addr":"172.21.15.167:6805/265072227","heartbeat_back_addr":"172.21.15.167:6809/265072227","heartbeat_front_addr":"172.21.15.167:6807/265072227","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.150359+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.084358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.722368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.332809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.320709+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.395153+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.935250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.801443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:44:49.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.553+0000 7f6a60000640 1 -- 172.21.15.151:0/891229322 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6a38079150 msgr2=0x7f6a3807b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:49.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.553+0000 7f6a60000640 1 --2- 172.21.15.151:0/891229322 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6a38079150 0x7f6a3807b610 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c034000 tx=0x7f6a4c004730 comp rx=0 tx=0).stop 2024-09-16T15:44:49.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.553+0000 7f6a60000640 1 -- 172.21.15.151:0/891229322 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a5815f5a0 msgr2=0x7f6a580b1330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:49.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.553+0000 7f6a60000640 1 --2- 172.21.15.151:0/891229322 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a5815f5a0 0x7f6a580b1330 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f6a5000c7a0 tx=0x7f6a5000cc60 comp rx=0 tx=0).stop 2024-09-16T15:44:49.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.553+0000 7f6a60000640 1 -- 172.21.15.151:0/891229322 shutdown_connections 2024-09-16T15:44:49.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.553+0000 7f6a60000640 1 --2- 172.21.15.151:0/891229322 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6a38079150 0x7f6a3807b610 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:49.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.553+0000 7f6a60000640 1 --2- 172.21.15.151:0/891229322 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a5815f5a0 0x7f6a580b1330 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:49.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.553+0000 7f6a60000640 1 --2- 172.21.15.151:0/891229322 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6a580ccf00 0x7f6a580b0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:49.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.553+0000 7f6a60000640 1 --2- 172.21.15.151:0/891229322 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a58073fb0 0x7f6a580cb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:49.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.553+0000 7f6a60000640 1 -- 172.21.15.151:0/891229322 >> 172.21.15.151:0/891229322 conn(0x7f6a580b1950 msgr2=0x7f6a5809d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:49.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.553+0000 7f6a60000640 1 -- 172.21.15.151:0/891229322 shutdown_connections 2024-09-16T15:44:49.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.553+0000 7f6a60000640 1 -- 172.21.15.151:0/891229322 wait complete. 2024-09-16T15:44:49.570 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T15:44:49.570 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:44:49.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.729+0000 7fa916000640 1 -- 172.21.15.151:0/1800012293 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa910073fb0 msgr2=0x7fa910074440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:49.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.729+0000 7fa916000640 1 --2- 172.21.15.151:0/1800012293 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa910073fb0 0x7fa910074440 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fa908005fd0 tx=0x7fa90802a820 comp rx=0 tx=0).stop 2024-09-16T15:44:49.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.729+0000 7fa916000640 1 -- 172.21.15.151:0/1800012293 shutdown_connections 2024-09-16T15:44:49.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.729+0000 7fa916000640 1 --2- 172.21.15.151:0/1800012293 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa910073fb0 0x7fa910074440 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:49.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.729+0000 7fa916000640 1 --2- 172.21.15.151:0/1800012293 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9100ccf00 0x7fa9100b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:49.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.729+0000 7fa916000640 1 -- 172.21.15.151:0/1800012293 >> 172.21.15.151:0/1800012293 conn(0x7fa9100b1950 msgr2=0x7fa9100b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:49.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.729+0000 7fa916000640 1 -- 172.21.15.151:0/1800012293 shutdown_connections 2024-09-16T15:44:49.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.729+0000 7fa916000640 1 -- 172.21.15.151:0/1800012293 wait complete. 2024-09-16T15:44:49.732 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.731+0000 7fa916000640 1 Processor -- start 2024-09-16T15:44:49.732 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.731+0000 7fa916000640 1 -- start start 2024-09-16T15:44:49.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.732+0000 7fa916000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa910073fb0 0x7fa9100cc650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:49.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.732+0000 7fa916000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9100ccf00 0x7fa9100ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:49.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.732+0000 7fa916000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa91015f5a0 0x7fa91009c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:49.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.732+0000 7fa916000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa91006f2e0 con 0x7fa91015f5a0 2024-09-16T15:44:49.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.732+0000 7fa916000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa91006f160 con 0x7fa910073fb0 2024-09-16T15:44:49.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.732+0000 7fa916000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fa91006f460 con 0x7fa9100ccf00 2024-09-16T15:44:49.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.732+0000 7fa90fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa91015f5a0 0x7fa91009c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:49.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.732+0000 7fa90fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa91015f5a0 0x7fa91009c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44284/0 (socket says 172.21.15.151:44284) 2024-09-16T15:44:49.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.732+0000 7fa90fe00640 1 -- 172.21.15.151:0/2830487178 learned_addr learned my addr 172.21.15.151:0/2830487178 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:49.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.732+0000 7fa90ea00640 1 --2- 172.21.15.151:0/2830487178 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9100ccf00 0x7fa9100ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:49.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.732+0000 7fa90f400640 1 --2- 172.21.15.151:0/2830487178 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa910073fb0 0x7fa9100cc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:49.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.733+0000 7fa90fe00640 1 -- 172.21.15.151:0/2830487178 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9100ccf00 msgr2=0x7fa9100ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:49.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.733+0000 7fa90fe00640 1 --2- 172.21.15.151:0/2830487178 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9100ccf00 0x7fa9100ce4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:49.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.733+0000 7fa90fe00640 1 -- 172.21.15.151:0/2830487178 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa910073fb0 msgr2=0x7fa9100cc650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:49.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.733+0000 7fa90fe00640 1 --2- 172.21.15.151:0/2830487178 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa910073fb0 0x7fa9100cc650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:49.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.733+0000 7fa90fe00640 1 -- 172.21.15.151:0/2830487178 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9100ce2b0 con 0x7fa91015f5a0 2024-09-16T15:44:49.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.733+0000 7fa90fe00640 1 --2- 172.21.15.151:0/2830487178 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa91015f5a0 0x7fa91009c410 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fa908006010 tx=0x7fa908032c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:49.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.733+0000 7fa907400640 1 -- 172.21.15.151:0/2830487178 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa908037040 con 0x7fa91015f5a0 2024-09-16T15:44:49.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.733+0000 7fa907400640 1 -- 172.21.15.151:0/2830487178 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa908042070 con 0x7fa91015f5a0 2024-09-16T15:44:49.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.733+0000 7fa916000640 1 -- 172.21.15.151:0/2830487178 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9101f4830 con 0x7fa91015f5a0 2024-09-16T15:44:49.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.733+0000 7fa907400640 1 -- 172.21.15.151:0/2830487178 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa908004280 con 0x7fa91015f5a0 2024-09-16T15:44:49.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.734+0000 7fa916000640 1 -- 172.21.15.151:0/2830487178 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9101f4df0 con 0x7fa91015f5a0 2024-09-16T15:44:49.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.735+0000 7fa916000640 1 -- 172.21.15.151:0/2830487178 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa91006f990 con 0x7fa91015f5a0 2024-09-16T15:44:49.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.736+0000 7fa907400640 1 -- 172.21.15.151:0/2830487178 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 102103+0+0 (secure 0 0 0) 0x7fa908004420 con 0x7fa91015f5a0 2024-09-16T15:44:49.738 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.737+0000 7fa907400640 1 --2- 172.21.15.151:0/2830487178 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa8f0079150 0x7fa8f007b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:49.738 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.737+0000 7fa907400640 1 -- 172.21.15.151:0/2830487178 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7fa9080b9dd0 con 0x7fa91015f5a0 2024-09-16T15:44:49.738 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.738+0000 7fa90f400640 1 --2- 172.21.15.151:0/2830487178 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa8f0079150 0x7fa8f007b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:49.739 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.738+0000 7fa90f400640 1 --2- 172.21.15.151:0/2830487178 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa8f0079150 0x7fa8f007b610 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa900001720 tx=0x7fa900007e90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:49.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.741+0000 7fa907400640 1 -- 172.21.15.151:0/2830487178 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa908083ef0 con 0x7fa91015f5a0 2024-09-16T15:44:49.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.926+0000 7fa916000640 1 -- 172.21.15.151:0/2830487178 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa9100ad0b0 con 0x7fa8f0079150 2024-09-16T15:44:49.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.931+0000 7fa907400640 1 -- 172.21.15.151:0/2830487178 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251283 (secure 0 0 0) 0x7fa9100ad0b0 con 0x7fa8f0079150 2024-09-16T15:44:49.935 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:44:49.936 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-09-16T15:44:49.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.935+0000 7fa916000640 1 -- 172.21.15.151:0/2830487178 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa8f0079150 msgr2=0x7fa8f007b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:49.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.935+0000 7fa916000640 1 --2- 172.21.15.151:0/2830487178 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa8f0079150 0x7fa8f007b610 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa900001720 tx=0x7fa900007e90 comp rx=0 tx=0).stop 2024-09-16T15:44:49.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.935+0000 7fa916000640 1 -- 172.21.15.151:0/2830487178 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa91015f5a0 msgr2=0x7fa91009c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:49.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.935+0000 7fa916000640 1 --2- 172.21.15.151:0/2830487178 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa91015f5a0 0x7fa91009c410 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fa908006010 tx=0x7fa908032c90 comp rx=0 tx=0).stop 2024-09-16T15:44:49.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.935+0000 7fa916000640 1 -- 172.21.15.151:0/2830487178 shutdown_connections 2024-09-16T15:44:49.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.935+0000 7fa916000640 1 --2- 172.21.15.151:0/2830487178 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa8f0079150 0x7fa8f007b610 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:49.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.935+0000 7fa916000640 1 --2- 172.21.15.151:0/2830487178 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa91015f5a0 0x7fa91009c410 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:49.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.935+0000 7fa916000640 1 --2- 172.21.15.151:0/2830487178 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9100ccf00 0x7fa9100ce4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:49.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.935+0000 7fa916000640 1 --2- 172.21.15.151:0/2830487178 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa910073fb0 0x7fa9100cc650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:49.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.935+0000 7fa916000640 1 -- 172.21.15.151:0/2830487178 >> 172.21.15.151:0/2830487178 conn(0x7fa9100b1950 msgr2=0x7fa91015cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:49.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.935+0000 7fa916000640 1 -- 172.21.15.151:0/2830487178 shutdown_connections 2024-09-16T15:44:49.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:49.935+0000 7fa916000640 1 -- 172.21.15.151:0/2830487178 wait complete. 2024-09-16T15:44:49.953 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":225,"stamp":"2024-09-16T15:44:49.670408+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592756,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":141,"num_write_kb":1641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":106,"ondisk_log_size":106,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":32,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":356200,"kb_used_data":6760,"kb_used_omap":12,"kb_used_meta":349235,"kb_avail":754618520,"statfs":{"total":773094113280,"available":772729364480,"internally_reserved":0,"allocated":6922240,"data_stored":3563759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12738,"internal_metadata":357617214},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2388,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.988477"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.732875+0000","last_change":"2024-09-16T15:44:45.728739+0000","last_active":"2024-09-16T15:44:45.732875+0000","last_peered":"2024-09-16T15:44:45.732875+0000","last_clean":"2024-09-16T15:44:45.732875+0000","last_became_active":"2024-09-16T15:44:45.727368+0000","last_became_peered":"2024-09-16T15:44:45.727368+0000","last_unstale":"2024-09-16T15:44:45.732875+0000","last_undegraded":"2024-09-16T15:44:45.732875+0000","last_fullsized":"2024-09-16T15:44:45.732875+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:34:55.972702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T15:44:44.692975+0000","last_change":"2024-09-16T15:44:44.692975+0000","last_active":"2024-09-16T15:44:44.679289+0000","last_peered":"2024-09-16T15:44:44.679289+0000","last_clean":"2024-09-16T15:44:44.679289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:44:44.692975+0000","last_undegraded":"2024-09-16T15:44:44.692975+0000","last_fullsized":"2024-09-16T15:44:44.692975+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.732389+0000","last_change":"2024-09-16T15:44:45.725780+0000","last_active":"2024-09-16T15:44:45.732389+0000","last_peered":"2024-09-16T15:44:45.732389+0000","last_clean":"2024-09-16T15:44:45.732389+0000","last_became_active":"2024-09-16T15:44:45.721333+0000","last_became_peered":"2024-09-16T15:44:45.721333+0000","last_unstale":"2024-09-16T15:44:45.732389+0000","last_undegraded":"2024-09-16T15:44:45.732389+0000","last_fullsized":"2024-09-16T15:44:45.732389+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:13:04.403633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.737448+0000","last_change":"2024-09-16T15:44:45.729388+0000","last_active":"2024-09-16T15:44:45.737448+0000","last_peered":"2024-09-16T15:44:45.737448+0000","last_clean":"2024-09-16T15:44:45.737448+0000","last_became_active":"2024-09-16T15:44:45.729119+0000","last_became_peered":"2024-09-16T15:44:45.729119+0000","last_unstale":"2024-09-16T15:44:45.737448+0000","last_undegraded":"2024-09-16T15:44:45.737448+0000","last_fullsized":"2024-09-16T15:44:45.737448+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:08:12.766873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.738580+0000","last_change":"2024-09-16T15:44:45.733586+0000","last_active":"2024-09-16T15:44:45.738580+0000","last_peered":"2024-09-16T15:44:45.738580+0000","last_clean":"2024-09-16T15:44:45.738580+0000","last_became_active":"2024-09-16T15:44:45.733144+0000","last_became_peered":"2024-09-16T15:44:45.733144+0000","last_unstale":"2024-09-16T15:44:45.738580+0000","last_undegraded":"2024-09-16T15:44:45.738580+0000","last_fullsized":"2024-09-16T15:44:45.738580+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:04:50.517949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.730726+0000","last_change":"2024-09-16T15:44:45.718049+0000","last_active":"2024-09-16T15:44:45.730726+0000","last_peered":"2024-09-16T15:44:45.730726+0000","last_clean":"2024-09-16T15:44:45.730726+0000","last_became_active":"2024-09-16T15:44:45.716815+0000","last_became_peered":"2024-09-16T15:44:45.716815+0000","last_unstale":"2024-09-16T15:44:45.730726+0000","last_undegraded":"2024-09-16T15:44:45.730726+0000","last_fullsized":"2024-09-16T15:44:45.730726+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:30:51.703628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.724140+0000","last_change":"2024-09-16T15:44:45.718523+0000","last_active":"2024-09-16T15:44:45.724140+0000","last_peered":"2024-09-16T15:44:45.724140+0000","last_clean":"2024-09-16T15:44:45.724140+0000","last_became_active":"2024-09-16T15:44:45.718208+0000","last_became_peered":"2024-09-16T15:44:45.718208+0000","last_unstale":"2024-09-16T15:44:45.724140+0000","last_undegraded":"2024-09-16T15:44:45.724140+0000","last_fullsized":"2024-09-16T15:44:45.724140+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:58:34.310843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.724066+0000","last_change":"2024-09-16T15:44:45.720647+0000","last_active":"2024-09-16T15:44:45.724066+0000","last_peered":"2024-09-16T15:44:45.724066+0000","last_clean":"2024-09-16T15:44:45.724066+0000","last_became_active":"2024-09-16T15:44:45.720351+0000","last_became_peered":"2024-09-16T15:44:45.720351+0000","last_unstale":"2024-09-16T15:44:45.724066+0000","last_undegraded":"2024-09-16T15:44:45.724066+0000","last_fullsized":"2024-09-16T15:44:45.724066+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:44:06.889108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.734233+0000","last_change":"2024-09-16T15:44:45.732987+0000","last_active":"2024-09-16T15:44:45.734233+0000","last_peered":"2024-09-16T15:44:45.734233+0000","last_clean":"2024-09-16T15:44:45.734233+0000","last_became_active":"2024-09-16T15:44:45.732680+0000","last_became_peered":"2024-09-16T15:44:45.732680+0000","last_unstale":"2024-09-16T15:44:45.734233+0000","last_undegraded":"2024-09-16T15:44:45.734233+0000","last_fullsized":"2024-09-16T15:44:45.734233+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:21:56.720871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.730690+0000","last_change":"2024-09-16T15:44:45.719120+0000","last_active":"2024-09-16T15:44:45.730690+0000","last_peered":"2024-09-16T15:44:45.730690+0000","last_clean":"2024-09-16T15:44:45.730690+0000","last_became_active":"2024-09-16T15:44:45.718746+0000","last_became_peered":"2024-09-16T15:44:45.718746+0000","last_unstale":"2024-09-16T15:44:45.730690+0000","last_undegraded":"2024-09-16T15:44:45.730690+0000","last_fullsized":"2024-09-16T15:44:45.730690+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:13:46.973489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.726293+0000","last_change":"2024-09-16T15:44:45.718730+0000","last_active":"2024-09-16T15:44:45.726293+0000","last_peered":"2024-09-16T15:44:45.726293+0000","last_clean":"2024-09-16T15:44:45.726293+0000","last_became_active":"2024-09-16T15:44:45.718442+0000","last_became_peered":"2024-09-16T15:44:45.718442+0000","last_unstale":"2024-09-16T15:44:45.726293+0000","last_undegraded":"2024-09-16T15:44:45.726293+0000","last_fullsized":"2024-09-16T15:44:45.726293+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:13:04.403633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.720994+0000","last_change":"2024-09-16T15:44:45.717536+0000","last_active":"2024-09-16T15:44:45.720994+0000","last_peered":"2024-09-16T15:44:45.720994+0000","last_clean":"2024-09-16T15:44:45.720994+0000","last_became_active":"2024-09-16T15:44:45.717226+0000","last_became_peered":"2024-09-16T15:44:45.717226+0000","last_unstale":"2024-09-16T15:44:45.720994+0000","last_undegraded":"2024-09-16T15:44:45.720994+0000","last_fullsized":"2024-09-16T15:44:45.720994+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:46:33.994315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.719703+0000","last_change":"2024-09-16T15:44:45.711525+0000","last_active":"2024-09-16T15:44:45.719703+0000","last_peered":"2024-09-16T15:44:45.719703+0000","last_clean":"2024-09-16T15:44:45.719703+0000","last_became_active":"2024-09-16T15:44:45.711195+0000","last_became_peered":"2024-09-16T15:44:45.711195+0000","last_unstale":"2024-09-16T15:44:45.719703+0000","last_undegraded":"2024-09-16T15:44:45.719703+0000","last_fullsized":"2024-09-16T15:44:45.719703+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:42:47.930299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.732183+0000","last_change":"2024-09-16T15:44:45.726781+0000","last_active":"2024-09-16T15:44:45.732183+0000","last_peered":"2024-09-16T15:44:45.732183+0000","last_clean":"2024-09-16T15:44:45.732183+0000","last_became_active":"2024-09-16T15:44:45.726045+0000","last_became_peered":"2024-09-16T15:44:45.726045+0000","last_unstale":"2024-09-16T15:44:45.732183+0000","last_undegraded":"2024-09-16T15:44:45.732183+0000","last_fullsized":"2024-09-16T15:44:45.732183+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:36:30.549487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.719163+0000","last_change":"2024-09-16T15:44:45.716384+0000","last_active":"2024-09-16T15:44:45.719163+0000","last_peered":"2024-09-16T15:44:45.719163+0000","last_clean":"2024-09-16T15:44:45.719163+0000","last_became_active":"2024-09-16T15:44:45.716101+0000","last_became_peered":"2024-09-16T15:44:45.716101+0000","last_unstale":"2024-09-16T15:44:45.719163+0000","last_undegraded":"2024-09-16T15:44:45.719163+0000","last_fullsized":"2024-09-16T15:44:45.719163+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:21:51.586704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.730872+0000","last_change":"2024-09-16T15:44:45.721590+0000","last_active":"2024-09-16T15:44:45.730872+0000","last_peered":"2024-09-16T15:44:45.730872+0000","last_clean":"2024-09-16T15:44:45.730872+0000","last_became_active":"2024-09-16T15:44:45.721343+0000","last_became_peered":"2024-09-16T15:44:45.721343+0000","last_unstale":"2024-09-16T15:44:45.730872+0000","last_undegraded":"2024-09-16T15:44:45.730872+0000","last_fullsized":"2024-09-16T15:44:45.730872+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:44:28.021891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.730936+0000","last_change":"2024-09-16T15:44:45.722006+0000","last_active":"2024-09-16T15:44:45.730936+0000","last_peered":"2024-09-16T15:44:45.730936+0000","last_clean":"2024-09-16T15:44:45.730936+0000","last_became_active":"2024-09-16T15:44:45.721714+0000","last_became_peered":"2024-09-16T15:44:45.721714+0000","last_unstale":"2024-09-16T15:44:45.730936+0000","last_undegraded":"2024-09-16T15:44:45.730936+0000","last_fullsized":"2024-09-16T15:44:45.730936+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:53:59.005172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.736449+0000","last_change":"2024-09-16T15:44:45.732870+0000","last_active":"2024-09-16T15:44:45.736449+0000","last_peered":"2024-09-16T15:44:45.736449+0000","last_clean":"2024-09-16T15:44:45.736449+0000","last_became_active":"2024-09-16T15:44:45.732609+0000","last_became_peered":"2024-09-16T15:44:45.732609+0000","last_unstale":"2024-09-16T15:44:45.736449+0000","last_undegraded":"2024-09-16T15:44:45.736449+0000","last_fullsized":"2024-09-16T15:44:45.736449+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:44:16.275512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T15:44:44.691707+0000","last_change":"2024-09-16T15:44:44.691707+0000","last_active":"2024-09-16T15:44:44.679289+0000","last_peered":"2024-09-16T15:44:44.679289+0000","last_clean":"2024-09-16T15:44:44.679289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:44:44.691707+0000","last_undegraded":"2024-09-16T15:44:44.691707+0000","last_fullsized":"2024-09-16T15:44:44.691707+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.734176+0000","last_change":"2024-09-16T15:44:45.731373+0000","last_active":"2024-09-16T15:44:45.734176+0000","last_peered":"2024-09-16T15:44:45.734176+0000","last_clean":"2024-09-16T15:44:45.734176+0000","last_became_active":"2024-09-16T15:44:45.731068+0000","last_became_peered":"2024-09-16T15:44:45.731068+0000","last_unstale":"2024-09-16T15:44:45.734176+0000","last_undegraded":"2024-09-16T15:44:45.734176+0000","last_fullsized":"2024-09-16T15:44:45.734176+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:51:57.452114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.724617+0000","last_change":"2024-09-16T15:44:45.719386+0000","last_active":"2024-09-16T15:44:45.724617+0000","last_peered":"2024-09-16T15:44:45.724617+0000","last_clean":"2024-09-16T15:44:45.724617+0000","last_became_active":"2024-09-16T15:44:45.719154+0000","last_became_peered":"2024-09-16T15:44:45.719154+0000","last_unstale":"2024-09-16T15:44:45.724617+0000","last_undegraded":"2024-09-16T15:44:45.724617+0000","last_fullsized":"2024-09-16T15:44:45.724617+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:11:14.213089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.723225+0000","last_change":"2024-09-16T15:44:45.718300+0000","last_active":"2024-09-16T15:44:45.723225+0000","last_peered":"2024-09-16T15:44:45.723225+0000","last_clean":"2024-09-16T15:44:45.723225+0000","last_became_active":"2024-09-16T15:44:45.717725+0000","last_became_peered":"2024-09-16T15:44:45.717725+0000","last_unstale":"2024-09-16T15:44:45.723225+0000","last_undegraded":"2024-09-16T15:44:45.723225+0000","last_fullsized":"2024-09-16T15:44:45.723225+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:03:17.351052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.721190+0000","last_change":"2024-09-16T15:44:45.716075+0000","last_active":"2024-09-16T15:44:45.721190+0000","last_peered":"2024-09-16T15:44:45.721190+0000","last_clean":"2024-09-16T15:44:45.721190+0000","last_became_active":"2024-09-16T15:44:45.715816+0000","last_became_peered":"2024-09-16T15:44:45.715816+0000","last_unstale":"2024-09-16T15:44:45.721190+0000","last_undegraded":"2024-09-16T15:44:45.721190+0000","last_fullsized":"2024-09-16T15:44:45.721190+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:38:03.977000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.725164+0000","last_change":"2024-09-16T15:44:45.721703+0000","last_active":"2024-09-16T15:44:45.725164+0000","last_peered":"2024-09-16T15:44:45.725164+0000","last_clean":"2024-09-16T15:44:45.725164+0000","last_became_active":"2024-09-16T15:44:45.721406+0000","last_became_peered":"2024-09-16T15:44:45.721406+0000","last_unstale":"2024-09-16T15:44:45.725164+0000","last_undegraded":"2024-09-16T15:44:45.725164+0000","last_fullsized":"2024-09-16T15:44:45.725164+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:42:26.902672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T15:44:44.691255+0000","last_change":"2024-09-16T15:44:44.691255+0000","last_active":"2024-09-16T15:44:44.679289+0000","last_peered":"2024-09-16T15:44:44.679289+0000","last_clean":"2024-09-16T15:44:44.679289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:44:44.691255+0000","last_undegraded":"2024-09-16T15:44:44.691255+0000","last_fullsized":"2024-09-16T15:44:44.691255+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.725345+0000","last_change":"2024-09-16T15:44:45.722538+0000","last_active":"2024-09-16T15:44:45.725345+0000","last_peered":"2024-09-16T15:44:45.725345+0000","last_clean":"2024-09-16T15:44:45.725345+0000","last_became_active":"2024-09-16T15:44:45.722286+0000","last_became_peered":"2024-09-16T15:44:45.722286+0000","last_unstale":"2024-09-16T15:44:45.725345+0000","last_undegraded":"2024-09-16T15:44:45.725345+0000","last_fullsized":"2024-09-16T15:44:45.725345+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:45:58.209870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T15:44:44.691255+0000","last_change":"2024-09-16T15:44:44.691255+0000","last_active":"2024-09-16T15:44:44.679289+0000","last_peered":"2024-09-16T15:44:44.679289+0000","last_clean":"2024-09-16T15:44:44.679289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:44:44.691255+0000","last_undegraded":"2024-09-16T15:44:44.691255+0000","last_fullsized":"2024-09-16T15:44:44.691255+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.722818+0000","last_change":"2024-09-16T15:44:45.707684+0000","last_active":"2024-09-16T15:44:45.722818+0000","last_peered":"2024-09-16T15:44:45.722818+0000","last_clean":"2024-09-16T15:44:45.722818+0000","last_became_active":"2024-09-16T15:44:45.707370+0000","last_became_peered":"2024-09-16T15:44:45.707370+0000","last_unstale":"2024-09-16T15:44:45.722818+0000","last_undegraded":"2024-09-16T15:44:45.722818+0000","last_fullsized":"2024-09-16T15:44:45.722818+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:32:11.331276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.724083+0000","last_change":"2024-09-16T15:44:45.712553+0000","last_active":"2024-09-16T15:44:45.724083+0000","last_peered":"2024-09-16T15:44:45.724083+0000","last_clean":"2024-09-16T15:44:45.724083+0000","last_became_active":"2024-09-16T15:44:45.711955+0000","last_became_peered":"2024-09-16T15:44:45.711955+0000","last_unstale":"2024-09-16T15:44:45.724083+0000","last_undegraded":"2024-09-16T15:44:45.724083+0000","last_fullsized":"2024-09-16T15:44:45.724083+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:33:23.442599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.714841+0000","last_change":"2024-09-16T15:44:45.711869+0000","last_active":"2024-09-16T15:44:45.714841+0000","last_peered":"2024-09-16T15:44:45.714841+0000","last_clean":"2024-09-16T15:44:45.714841+0000","last_became_active":"2024-09-16T15:44:45.711534+0000","last_became_peered":"2024-09-16T15:44:45.711534+0000","last_unstale":"2024-09-16T15:44:45.714841+0000","last_undegraded":"2024-09-16T15:44:45.714841+0000","last_fullsized":"2024-09-16T15:44:45.714841+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:31:16.141860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T15:44:44.691565+0000","last_change":"2024-09-16T15:44:44.691565+0000","last_active":"2024-09-16T15:44:44.679289+0000","last_peered":"2024-09-16T15:44:44.679289+0000","last_clean":"2024-09-16T15:44:44.679289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:44:44.691565+0000","last_undegraded":"2024-09-16T15:44:44.691565+0000","last_fullsized":"2024-09-16T15:44:44.691565+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.729551+0000","last_change":"2024-09-16T15:44:45.726051+0000","last_active":"2024-09-16T15:44:45.729551+0000","last_peered":"2024-09-16T15:44:45.729551+0000","last_clean":"2024-09-16T15:44:45.729551+0000","last_became_active":"2024-09-16T15:44:45.720897+0000","last_became_peered":"2024-09-16T15:44:45.720897+0000","last_unstale":"2024-09-16T15:44:45.729551+0000","last_undegraded":"2024-09-16T15:44:45.729551+0000","last_fullsized":"2024-09-16T15:44:45.729551+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:36:30.549487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.732821+0000","last_change":"2024-09-16T15:44:45.724124+0000","last_active":"2024-09-16T15:44:45.732821+0000","last_peered":"2024-09-16T15:44:45.732821+0000","last_clean":"2024-09-16T15:44:45.732821+0000","last_became_active":"2024-09-16T15:44:45.721712+0000","last_became_peered":"2024-09-16T15:44:45.721712+0000","last_unstale":"2024-09-16T15:44:45.732821+0000","last_undegraded":"2024-09-16T15:44:45.732821+0000","last_fullsized":"2024-09-16T15:44:45.732821+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:15:22.807776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"61'4","reported_seq":17,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.732008+0000","last_change":"2024-09-16T15:44:44.700510+0000","last_active":"2024-09-16T15:44:45.732008+0000","last_peered":"2024-09-16T15:44:45.732008+0000","last_clean":"2024-09-16T15:44:45.732008+0000","last_became_active":"2024-09-16T15:44:44.697806+0000","last_became_peered":"2024-09-16T15:44:44.697806+0000","last_unstale":"2024-09-16T15:44:45.732008+0000","last_undegraded":"2024-09-16T15:44:45.732008+0000","last_fullsized":"2024-09-16T15:44:45.732008+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:50:52.878214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.729071+0000","last_change":"2024-09-16T15:44:45.715988+0000","last_active":"2024-09-16T15:44:45.729071+0000","last_peered":"2024-09-16T15:44:45.729071+0000","last_clean":"2024-09-16T15:44:45.729071+0000","last_became_active":"2024-09-16T15:44:45.715735+0000","last_became_peered":"2024-09-16T15:44:45.715735+0000","last_unstale":"2024-09-16T15:44:45.729071+0000","last_undegraded":"2024-09-16T15:44:45.729071+0000","last_fullsized":"2024-09-16T15:44:45.729071+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:39:12.784333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T15:44:44.710442+0000","last_change":"2024-09-16T15:44:44.708252+0000","last_active":"2024-09-16T15:44:44.710442+0000","last_peered":"2024-09-16T15:44:44.710442+0000","last_clean":"2024-09-16T15:44:44.710442+0000","last_became_active":"2024-09-16T15:44:44.707995+0000","last_became_peered":"2024-09-16T15:44:44.707995+0000","last_unstale":"2024-09-16T15:44:44.710442+0000","last_undegraded":"2024-09-16T15:44:44.710442+0000","last_fullsized":"2024-09-16T15:44:44.710442+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:26:28.068533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.734118+0000","last_change":"2024-09-16T15:44:45.729683+0000","last_active":"2024-09-16T15:44:45.734118+0000","last_peered":"2024-09-16T15:44:45.734118+0000","last_clean":"2024-09-16T15:44:45.734118+0000","last_became_active":"2024-09-16T15:44:45.729258+0000","last_became_peered":"2024-09-16T15:44:45.729258+0000","last_unstale":"2024-09-16T15:44:45.734118+0000","last_undegraded":"2024-09-16T15:44:45.734118+0000","last_fullsized":"2024-09-16T15:44:45.734118+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:34:41.864039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"61'2","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.739158+0000","last_change":"2024-09-16T15:44:44.705295+0000","last_active":"2024-09-16T15:44:45.739158+0000","last_peered":"2024-09-16T15:44:45.739158+0000","last_clean":"2024-09-16T15:44:45.739158+0000","last_became_active":"2024-09-16T15:44:44.704913+0000","last_became_peered":"2024-09-16T15:44:44.704913+0000","last_unstale":"2024-09-16T15:44:45.739158+0000","last_undegraded":"2024-09-16T15:44:45.739158+0000","last_fullsized":"2024-09-16T15:44:45.739158+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:33:20.871493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.732416+0000","last_change":"2024-09-16T15:44:45.724296+0000","last_active":"2024-09-16T15:44:45.732416+0000","last_peered":"2024-09-16T15:44:45.732416+0000","last_clean":"2024-09-16T15:44:45.732416+0000","last_became_active":"2024-09-16T15:44:45.722552+0000","last_became_peered":"2024-09-16T15:44:45.722552+0000","last_unstale":"2024-09-16T15:44:45.732416+0000","last_undegraded":"2024-09-16T15:44:45.732416+0000","last_fullsized":"2024-09-16T15:44:45.732416+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:35:05.861042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"61'2","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.737888+0000","last_change":"2024-09-16T15:44:44.701949+0000","last_active":"2024-09-16T15:44:45.737888+0000","last_peered":"2024-09-16T15:44:45.737888+0000","last_clean":"2024-09-16T15:44:45.737888+0000","last_became_active":"2024-09-16T15:44:44.701522+0000","last_became_peered":"2024-09-16T15:44:44.701522+0000","last_unstale":"2024-09-16T15:44:45.737888+0000","last_undegraded":"2024-09-16T15:44:45.737888+0000","last_fullsized":"2024-09-16T15:44:45.737888+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:18:22.181140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.723888+0000","last_change":"2024-09-16T15:44:45.712376+0000","last_active":"2024-09-16T15:44:45.723888+0000","last_peered":"2024-09-16T15:44:45.723888+0000","last_clean":"2024-09-16T15:44:45.723888+0000","last_became_active":"2024-09-16T15:44:45.712045+0000","last_became_peered":"2024-09-16T15:44:45.712045+0000","last_unstale":"2024-09-16T15:44:45.723888+0000","last_undegraded":"2024-09-16T15:44:45.723888+0000","last_fullsized":"2024-09-16T15:44:45.723888+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:20:02.525231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"61'1","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.741440+0000","last_change":"2024-09-16T15:44:44.705164+0000","last_active":"2024-09-16T15:44:45.741440+0000","last_peered":"2024-09-16T15:44:45.741440+0000","last_clean":"2024-09-16T15:44:45.741440+0000","last_became_active":"2024-09-16T15:44:44.704525+0000","last_became_peered":"2024-09-16T15:44:44.704525+0000","last_unstale":"2024-09-16T15:44:45.741440+0000","last_undegraded":"2024-09-16T15:44:45.741440+0000","last_fullsized":"2024-09-16T15:44:45.741440+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:37:15.032653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.724052+0000","last_change":"2024-09-16T15:44:45.719896+0000","last_active":"2024-09-16T15:44:45.724052+0000","last_peered":"2024-09-16T15:44:45.724052+0000","last_clean":"2024-09-16T15:44:45.724052+0000","last_became_active":"2024-09-16T15:44:45.718945+0000","last_became_peered":"2024-09-16T15:44:45.718945+0000","last_unstale":"2024-09-16T15:44:45.724052+0000","last_undegraded":"2024-09-16T15:44:45.724052+0000","last_fullsized":"2024-09-16T15:44:45.724052+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:31:16.283923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"61'1","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.736043+0000","last_change":"2024-09-16T15:44:44.710842+0000","last_active":"2024-09-16T15:44:45.736043+0000","last_peered":"2024-09-16T15:44:45.736043+0000","last_clean":"2024-09-16T15:44:45.736043+0000","last_became_active":"2024-09-16T15:44:44.710529+0000","last_became_peered":"2024-09-16T15:44:44.710529+0000","last_unstale":"2024-09-16T15:44:45.736043+0000","last_undegraded":"2024-09-16T15:44:45.736043+0000","last_fullsized":"2024-09-16T15:44:45.736043+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:36:04.434843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.716864+0000","last_change":"2024-09-16T15:44:45.708591+0000","last_active":"2024-09-16T15:44:45.716864+0000","last_peered":"2024-09-16T15:44:45.716864+0000","last_clean":"2024-09-16T15:44:45.716864+0000","last_became_active":"2024-09-16T15:44:45.708350+0000","last_became_peered":"2024-09-16T15:44:45.708350+0000","last_unstale":"2024-09-16T15:44:45.716864+0000","last_undegraded":"2024-09-16T15:44:45.716864+0000","last_fullsized":"2024-09-16T15:44:45.716864+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:28:57.953253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.704237+0000","last_change":"2024-09-16T15:44:44.697711+0000","last_active":"2024-09-16T15:44:45.704237+0000","last_peered":"2024-09-16T15:44:45.704237+0000","last_clean":"2024-09-16T15:44:45.704237+0000","last_became_active":"2024-09-16T15:44:44.697357+0000","last_became_peered":"2024-09-16T15:44:44.697357+0000","last_unstale":"2024-09-16T15:44:45.704237+0000","last_undegraded":"2024-09-16T15:44:45.704237+0000","last_fullsized":"2024-09-16T15:44:45.704237+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:20:52.337348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.730799+0000","last_change":"2024-09-16T15:44:45.719359+0000","last_active":"2024-09-16T15:44:45.730799+0000","last_peered":"2024-09-16T15:44:45.730799+0000","last_clean":"2024-09-16T15:44:45.730799+0000","last_became_active":"2024-09-16T15:44:45.719093+0000","last_became_peered":"2024-09-16T15:44:45.719093+0000","last_unstale":"2024-09-16T15:44:45.730799+0000","last_undegraded":"2024-09-16T15:44:45.730799+0000","last_fullsized":"2024-09-16T15:44:45.730799+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:02:03.820471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.703161+0000","last_change":"2024-09-16T15:44:44.694298+0000","last_active":"2024-09-16T15:44:45.703161+0000","last_peered":"2024-09-16T15:44:45.703161+0000","last_clean":"2024-09-16T15:44:45.703161+0000","last_became_active":"2024-09-16T15:44:44.694052+0000","last_became_peered":"2024-09-16T15:44:44.694052+0000","last_unstale":"2024-09-16T15:44:45.703161+0000","last_undegraded":"2024-09-16T15:44:45.703161+0000","last_fullsized":"2024-09-16T15:44:45.703161+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:25:26.438122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.715275+0000","last_change":"2024-09-16T15:44:45.713168+0000","last_active":"2024-09-16T15:44:45.715275+0000","last_peered":"2024-09-16T15:44:45.715275+0000","last_clean":"2024-09-16T15:44:45.715275+0000","last_became_active":"2024-09-16T15:44:45.711840+0000","last_became_peered":"2024-09-16T15:44:45.711840+0000","last_unstale":"2024-09-16T15:44:45.715275+0000","last_undegraded":"2024-09-16T15:44:45.715275+0000","last_fullsized":"2024-09-16T15:44:45.715275+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:21:48.643444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.709187+0000","last_change":"2024-09-16T15:44:44.703702+0000","last_active":"2024-09-16T15:44:45.709187+0000","last_peered":"2024-09-16T15:44:45.709187+0000","last_clean":"2024-09-16T15:44:45.709187+0000","last_became_active":"2024-09-16T15:44:44.703424+0000","last_became_peered":"2024-09-16T15:44:44.703424+0000","last_unstale":"2024-09-16T15:44:45.709187+0000","last_undegraded":"2024-09-16T15:44:45.709187+0000","last_fullsized":"2024-09-16T15:44:45.709187+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:24:00.275655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.734732+0000","last_change":"2024-09-16T15:44:45.726605+0000","last_active":"2024-09-16T15:44:45.734732+0000","last_peered":"2024-09-16T15:44:45.734732+0000","last_clean":"2024-09-16T15:44:45.734732+0000","last_became_active":"2024-09-16T15:44:45.726368+0000","last_became_peered":"2024-09-16T15:44:45.726368+0000","last_unstale":"2024-09-16T15:44:45.734732+0000","last_undegraded":"2024-09-16T15:44:45.734732+0000","last_fullsized":"2024-09-16T15:44:45.734732+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:31:16.283923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.704573+0000","last_change":"2024-09-16T15:44:44.695923+0000","last_active":"2024-09-16T15:44:45.704573+0000","last_peered":"2024-09-16T15:44:45.704573+0000","last_clean":"2024-09-16T15:44:45.704573+0000","last_became_active":"2024-09-16T15:44:44.695618+0000","last_became_peered":"2024-09-16T15:44:44.695618+0000","last_unstale":"2024-09-16T15:44:45.704573+0000","last_undegraded":"2024-09-16T15:44:45.704573+0000","last_fullsized":"2024-09-16T15:44:45.704573+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:41:35.721501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.728654+0000","last_change":"2024-09-16T15:44:45.723911+0000","last_active":"2024-09-16T15:44:45.728654+0000","last_peered":"2024-09-16T15:44:45.728654+0000","last_clean":"2024-09-16T15:44:45.728654+0000","last_became_active":"2024-09-16T15:44:45.723557+0000","last_became_peered":"2024-09-16T15:44:45.723557+0000","last_unstale":"2024-09-16T15:44:45.728654+0000","last_undegraded":"2024-09-16T15:44:45.728654+0000","last_fullsized":"2024-09-16T15:44:45.728654+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:18:23.183829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"61'3","reported_seq":16,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.737703+0000","last_change":"2024-09-16T15:44:44.700702+0000","last_active":"2024-09-16T15:44:45.737703+0000","last_peered":"2024-09-16T15:44:45.737703+0000","last_clean":"2024-09-16T15:44:45.737703+0000","last_became_active":"2024-09-16T15:44:44.700438+0000","last_became_peered":"2024-09-16T15:44:44.700438+0000","last_unstale":"2024-09-16T15:44:45.737703+0000","last_undegraded":"2024-09-16T15:44:45.737703+0000","last_fullsized":"2024-09-16T15:44:45.737703+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:46:15.633475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.713366+0000","last_change":"2024-09-16T15:44:45.709431+0000","last_active":"2024-09-16T15:44:45.713366+0000","last_peered":"2024-09-16T15:44:45.713366+0000","last_clean":"2024-09-16T15:44:45.713366+0000","last_became_active":"2024-09-16T15:44:45.709179+0000","last_became_peered":"2024-09-16T15:44:45.709179+0000","last_unstale":"2024-09-16T15:44:45.713366+0000","last_undegraded":"2024-09-16T15:44:45.713366+0000","last_fullsized":"2024-09-16T15:44:45.713366+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:34:54.933570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"61'6","reported_seq":19,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.743392+0000","last_change":"2024-09-16T15:44:44.691387+0000","last_active":"2024-09-16T15:44:45.743392+0000","last_peered":"2024-09-16T15:44:45.743392+0000","last_clean":"2024-09-16T15:44:45.743392+0000","last_became_active":"2024-09-16T15:44:44.691144+0000","last_became_peered":"2024-09-16T15:44:44.691144+0000","last_unstale":"2024-09-16T15:44:45.743392+0000","last_undegraded":"2024-09-16T15:44:45.743392+0000","last_fullsized":"2024-09-16T15:44:45.743392+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:09:54.112439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.723072+0000","last_change":"2024-09-16T15:44:45.717715+0000","last_active":"2024-09-16T15:44:45.723072+0000","last_peered":"2024-09-16T15:44:45.723072+0000","last_clean":"2024-09-16T15:44:45.723072+0000","last_became_active":"2024-09-16T15:44:45.717366+0000","last_became_peered":"2024-09-16T15:44:45.717366+0000","last_unstale":"2024-09-16T15:44:45.723072+0000","last_undegraded":"2024-09-16T15:44:45.723072+0000","last_fullsized":"2024-09-16T15:44:45.723072+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:12:07.274566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"61'2","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.730407+0000","last_change":"2024-09-16T15:44:44.702026+0000","last_active":"2024-09-16T15:44:45.730407+0000","last_peered":"2024-09-16T15:44:45.730407+0000","last_clean":"2024-09-16T15:44:45.730407+0000","last_became_active":"2024-09-16T15:44:44.701701+0000","last_became_peered":"2024-09-16T15:44:44.701701+0000","last_unstale":"2024-09-16T15:44:45.730407+0000","last_undegraded":"2024-09-16T15:44:45.730407+0000","last_fullsized":"2024-09-16T15:44:45.730407+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:30:25.597599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.737862+0000","last_change":"2024-09-16T15:44:45.734880+0000","last_active":"2024-09-16T15:44:45.737862+0000","last_peered":"2024-09-16T15:44:45.737862+0000","last_clean":"2024-09-16T15:44:45.737862+0000","last_became_active":"2024-09-16T15:44:45.734616+0000","last_became_peered":"2024-09-16T15:44:45.734616+0000","last_unstale":"2024-09-16T15:44:45.737862+0000","last_undegraded":"2024-09-16T15:44:45.737862+0000","last_fullsized":"2024-09-16T15:44:45.737862+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:19:00.003404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.697639+0000","last_change":"2024-09-16T15:44:44.701584+0000","last_active":"2024-09-16T15:44:45.697639+0000","last_peered":"2024-09-16T15:44:45.697639+0000","last_clean":"2024-09-16T15:44:45.697639+0000","last_became_active":"2024-09-16T15:44:44.701327+0000","last_became_peered":"2024-09-16T15:44:44.701327+0000","last_unstale":"2024-09-16T15:44:45.697639+0000","last_undegraded":"2024-09-16T15:44:45.697639+0000","last_fullsized":"2024-09-16T15:44:45.697639+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:44:05.886418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.720386+0000","last_change":"2024-09-16T15:44:45.717514+0000","last_active":"2024-09-16T15:44:45.720386+0000","last_peered":"2024-09-16T15:44:45.720386+0000","last_clean":"2024-09-16T15:44:45.720386+0000","last_became_active":"2024-09-16T15:44:45.717270+0000","last_became_peered":"2024-09-16T15:44:45.717270+0000","last_unstale":"2024-09-16T15:44:45.720386+0000","last_undegraded":"2024-09-16T15:44:45.720386+0000","last_fullsized":"2024-09-16T15:44:45.720386+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:12:47.737828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.698166+0000","last_change":"2024-09-16T15:44:44.702562+0000","last_active":"2024-09-16T15:44:45.698166+0000","last_peered":"2024-09-16T15:44:45.698166+0000","last_clean":"2024-09-16T15:44:45.698166+0000","last_became_active":"2024-09-16T15:44:44.702282+0000","last_became_peered":"2024-09-16T15:44:44.702282+0000","last_unstale":"2024-09-16T15:44:45.698166+0000","last_undegraded":"2024-09-16T15:44:45.698166+0000","last_fullsized":"2024-09-16T15:44:45.698166+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:34:40.861349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.723514+0000","last_change":"2024-09-16T15:44:45.717730+0000","last_active":"2024-09-16T15:44:45.723514+0000","last_peered":"2024-09-16T15:44:45.723514+0000","last_clean":"2024-09-16T15:44:45.723514+0000","last_became_active":"2024-09-16T15:44:45.717456+0000","last_became_peered":"2024-09-16T15:44:45.717456+0000","last_unstale":"2024-09-16T15:44:45.723514+0000","last_undegraded":"2024-09-16T15:44:45.723514+0000","last_fullsized":"2024-09-16T15:44:45.723514+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:09:14.114379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"61'8","reported_seq":21,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.740384+0000","last_change":"2024-09-16T15:44:44.700378+0000","last_active":"2024-09-16T15:44:45.740384+0000","last_peered":"2024-09-16T15:44:45.740384+0000","last_clean":"2024-09-16T15:44:45.740384+0000","last_became_active":"2024-09-16T15:44:44.700114+0000","last_became_peered":"2024-09-16T15:44:44.700114+0000","last_unstale":"2024-09-16T15:44:45.740384+0000","last_undegraded":"2024-09-16T15:44:45.740384+0000","last_fullsized":"2024-09-16T15:44:45.740384+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:46:16.051799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.720485+0000","last_change":"2024-09-16T15:44:45.716464+0000","last_active":"2024-09-16T15:44:45.720485+0000","last_peered":"2024-09-16T15:44:45.720485+0000","last_clean":"2024-09-16T15:44:45.720485+0000","last_became_active":"2024-09-16T15:44:45.716140+0000","last_became_peered":"2024-09-16T15:44:45.716140+0000","last_unstale":"2024-09-16T15:44:45.720485+0000","last_undegraded":"2024-09-16T15:44:45.720485+0000","last_fullsized":"2024-09-16T15:44:45.720485+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:31:46.215684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T15:44:44.711883+0000","last_change":"2024-09-16T15:44:44.711284+0000","last_active":"2024-09-16T15:44:44.711883+0000","last_peered":"2024-09-16T15:44:44.711883+0000","last_clean":"2024-09-16T15:44:44.711883+0000","last_became_active":"2024-09-16T15:44:44.709695+0000","last_became_peered":"2024-09-16T15:44:44.709695+0000","last_unstale":"2024-09-16T15:44:44.711883+0000","last_undegraded":"2024-09-16T15:44:44.711883+0000","last_fullsized":"2024-09-16T15:44:44.711883+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:29:18.863386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.718328+0000","last_change":"2024-09-16T15:44:45.714790+0000","last_active":"2024-09-16T15:44:45.718328+0000","last_peered":"2024-09-16T15:44:45.718328+0000","last_clean":"2024-09-16T15:44:45.718328+0000","last_became_active":"2024-09-16T15:44:45.713900+0000","last_became_peered":"2024-09-16T15:44:45.713900+0000","last_unstale":"2024-09-16T15:44:45.718328+0000","last_undegraded":"2024-09-16T15:44:45.718328+0000","last_fullsized":"2024-09-16T15:44:45.718328+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:50:53.880904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"61'2","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.730326+0000","last_change":"2024-09-16T15:44:44.698462+0000","last_active":"2024-09-16T15:44:45.730326+0000","last_peered":"2024-09-16T15:44:45.730326+0000","last_clean":"2024-09-16T15:44:45.730326+0000","last_became_active":"2024-09-16T15:44:44.698179+0000","last_became_peered":"2024-09-16T15:44:44.698179+0000","last_unstale":"2024-09-16T15:44:45.730326+0000","last_undegraded":"2024-09-16T15:44:45.730326+0000","last_fullsized":"2024-09-16T15:44:45.730326+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:13:21.284936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T15:44:44.693602+0000","last_change":"2024-09-16T15:44:44.693602+0000","last_active":"2024-09-16T15:44:44.679289+0000","last_peered":"2024-09-16T15:44:44.679289+0000","last_clean":"2024-09-16T15:44:44.679289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:44:44.693602+0000","last_undegraded":"2024-09-16T15:44:44.693602+0000","last_fullsized":"2024-09-16T15:44:44.693602+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"61'1","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.741605+0000","last_change":"2024-09-16T15:44:44.706373+0000","last_active":"2024-09-16T15:44:45.741605+0000","last_peered":"2024-09-16T15:44:45.741605+0000","last_clean":"2024-09-16T15:44:45.741605+0000","last_became_active":"2024-09-16T15:44:44.705120+0000","last_became_peered":"2024-09-16T15:44:44.705120+0000","last_unstale":"2024-09-16T15:44:45.741605+0000","last_undegraded":"2024-09-16T15:44:45.741605+0000","last_fullsized":"2024-09-16T15:44:45.741605+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:07:54.665094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.735750+0000","last_change":"2024-09-16T15:44:45.726953+0000","last_active":"2024-09-16T15:44:45.735750+0000","last_peered":"2024-09-16T15:44:45.735750+0000","last_clean":"2024-09-16T15:44:45.735750+0000","last_became_active":"2024-09-16T15:44:45.723508+0000","last_became_peered":"2024-09-16T15:44:45.723508+0000","last_unstale":"2024-09-16T15:44:45.735750+0000","last_undegraded":"2024-09-16T15:44:45.735750+0000","last_fullsized":"2024-09-16T15:44:45.735750+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:39:26.814713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.705171+0000","last_change":"2024-09-16T15:44:44.708721+0000","last_active":"2024-09-16T15:44:45.705171+0000","last_peered":"2024-09-16T15:44:45.705171+0000","last_clean":"2024-09-16T15:44:45.705171+0000","last_became_active":"2024-09-16T15:44:44.708471+0000","last_became_peered":"2024-09-16T15:44:44.708471+0000","last_unstale":"2024-09-16T15:44:45.705171+0000","last_undegraded":"2024-09-16T15:44:45.705171+0000","last_fullsized":"2024-09-16T15:44:45.705171+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:39:49.307383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.720161+0000","last_change":"2024-09-16T15:44:45.716983+0000","last_active":"2024-09-16T15:44:45.720161+0000","last_peered":"2024-09-16T15:44:45.720161+0000","last_clean":"2024-09-16T15:44:45.720161+0000","last_became_active":"2024-09-16T15:44:45.715530+0000","last_became_peered":"2024-09-16T15:44:45.715530+0000","last_unstale":"2024-09-16T15:44:45.720161+0000","last_undegraded":"2024-09-16T15:44:45.720161+0000","last_fullsized":"2024-09-16T15:44:45.720161+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:03:43.262795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"61'2","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.733237+0000","last_change":"2024-09-16T15:44:44.696118+0000","last_active":"2024-09-16T15:44:45.733237+0000","last_peered":"2024-09-16T15:44:45.733237+0000","last_clean":"2024-09-16T15:44:45.733237+0000","last_became_active":"2024-09-16T15:44:44.695842+0000","last_became_peered":"2024-09-16T15:44:44.695842+0000","last_unstale":"2024-09-16T15:44:45.733237+0000","last_undegraded":"2024-09-16T15:44:45.733237+0000","last_fullsized":"2024-09-16T15:44:45.733237+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:31:45.212994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.724057+0000","last_change":"2024-09-16T15:44:45.712249+0000","last_active":"2024-09-16T15:44:45.724057+0000","last_peered":"2024-09-16T15:44:45.724057+0000","last_clean":"2024-09-16T15:44:45.724057+0000","last_became_active":"2024-09-16T15:44:45.711524+0000","last_became_peered":"2024-09-16T15:44:45.711524+0000","last_unstale":"2024-09-16T15:44:45.724057+0000","last_undegraded":"2024-09-16T15:44:45.724057+0000","last_fullsized":"2024-09-16T15:44:45.724057+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:45:42.401451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.703897+0000","last_change":"2024-09-16T15:44:44.695883+0000","last_active":"2024-09-16T15:44:45.703897+0000","last_peered":"2024-09-16T15:44:45.703897+0000","last_clean":"2024-09-16T15:44:45.703897+0000","last_became_active":"2024-09-16T15:44:44.695562+0000","last_became_peered":"2024-09-16T15:44:44.695562+0000","last_unstale":"2024-09-16T15:44:45.703897+0000","last_undegraded":"2024-09-16T15:44:45.703897+0000","last_fullsized":"2024-09-16T15:44:45.703897+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:22:24.705292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.722865+0000","last_change":"2024-09-16T15:44:45.708395+0000","last_active":"2024-09-16T15:44:45.722865+0000","last_peered":"2024-09-16T15:44:45.722865+0000","last_clean":"2024-09-16T15:44:45.722865+0000","last_became_active":"2024-09-16T15:44:45.708090+0000","last_became_peered":"2024-09-16T15:44:45.708090+0000","last_unstale":"2024-09-16T15:44:45.722865+0000","last_undegraded":"2024-09-16T15:44:45.722865+0000","last_fullsized":"2024-09-16T15:44:45.722865+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:13:37.240255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.712718+0000","last_change":"2024-09-16T15:44:44.705798+0000","last_active":"2024-09-16T15:44:45.712718+0000","last_peered":"2024-09-16T15:44:45.712718+0000","last_clean":"2024-09-16T15:44:45.712718+0000","last_became_active":"2024-09-16T15:44:44.705468+0000","last_became_peered":"2024-09-16T15:44:44.705468+0000","last_unstale":"2024-09-16T15:44:45.712718+0000","last_undegraded":"2024-09-16T15:44:45.712718+0000","last_fullsized":"2024-09-16T15:44:45.712718+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:21:59.839074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.736422+0000","last_change":"2024-09-16T15:44:45.726903+0000","last_active":"2024-09-16T15:44:45.736422+0000","last_peered":"2024-09-16T15:44:45.736422+0000","last_clean":"2024-09-16T15:44:45.736422+0000","last_became_active":"2024-09-16T15:44:45.726666+0000","last_became_peered":"2024-09-16T15:44:45.726666+0000","last_unstale":"2024-09-16T15:44:45.736422+0000","last_undegraded":"2024-09-16T15:44:45.736422+0000","last_fullsized":"2024-09-16T15:44:45.736422+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:51:57.452114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T15:44:44.710737+0000","last_change":"2024-09-16T15:44:44.708470+0000","last_active":"2024-09-16T15:44:44.710737+0000","last_peered":"2024-09-16T15:44:44.710737+0000","last_clean":"2024-09-16T15:44:44.710737+0000","last_became_active":"2024-09-16T15:44:44.708182+0000","last_became_peered":"2024-09-16T15:44:44.708182+0000","last_unstale":"2024-09-16T15:44:44.710737+0000","last_undegraded":"2024-09-16T15:44:44.710737+0000","last_fullsized":"2024-09-16T15:44:44.710737+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:37:09.093235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.718282+0000","last_change":"2024-09-16T15:44:45.707416+0000","last_active":"2024-09-16T15:44:45.718282+0000","last_peered":"2024-09-16T15:44:45.718282+0000","last_clean":"2024-09-16T15:44:45.718282+0000","last_became_active":"2024-09-16T15:44:45.707148+0000","last_became_peered":"2024-09-16T15:44:45.707148+0000","last_unstale":"2024-09-16T15:44:45.718282+0000","last_undegraded":"2024-09-16T15:44:45.718282+0000","last_fullsized":"2024-09-16T15:44:45.718282+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:34:31.566558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.713344+0000","last_change":"2024-09-16T15:44:44.710583+0000","last_active":"2024-09-16T15:44:45.713344+0000","last_peered":"2024-09-16T15:44:45.713344+0000","last_clean":"2024-09-16T15:44:45.713344+0000","last_became_active":"2024-09-16T15:44:44.710293+0000","last_became_peered":"2024-09-16T15:44:44.710293+0000","last_unstale":"2024-09-16T15:44:45.713344+0000","last_undegraded":"2024-09-16T15:44:45.713344+0000","last_fullsized":"2024-09-16T15:44:45.713344+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:03:46.514703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T15:44:44.692278+0000","last_change":"2024-09-16T15:44:44.692278+0000","last_active":"2024-09-16T15:44:44.679289+0000","last_peered":"2024-09-16T15:44:44.679289+0000","last_clean":"2024-09-16T15:44:44.679289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:44:44.692278+0000","last_undegraded":"2024-09-16T15:44:44.692278+0000","last_fullsized":"2024-09-16T15:44:44.692278+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.704857+0000","last_change":"2024-09-16T15:44:44.703904+0000","last_active":"2024-09-16T15:44:45.704857+0000","last_peered":"2024-09-16T15:44:45.704857+0000","last_clean":"2024-09-16T15:44:45.704857+0000","last_became_active":"2024-09-16T15:44:44.703644+0000","last_became_peered":"2024-09-16T15:44:44.703644+0000","last_unstale":"2024-09-16T15:44:45.704857+0000","last_undegraded":"2024-09-16T15:44:45.704857+0000","last_fullsized":"2024-09-16T15:44:45.704857+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:31:17.501469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.724319+0000","last_change":"2024-09-16T15:44:45.714221+0000","last_active":"2024-09-16T15:44:45.724319+0000","last_peered":"2024-09-16T15:44:45.724319+0000","last_clean":"2024-09-16T15:44:45.724319+0000","last_became_active":"2024-09-16T15:44:45.713871+0000","last_became_peered":"2024-09-16T15:44:45.713871+0000","last_unstale":"2024-09-16T15:44:45.724319+0000","last_undegraded":"2024-09-16T15:44:45.724319+0000","last_fullsized":"2024-09-16T15:44:45.724319+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:09:31.284857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.717881+0000","last_change":"2024-09-16T15:44:44.698858+0000","last_active":"2024-09-16T15:44:45.717881+0000","last_peered":"2024-09-16T15:44:45.717881+0000","last_clean":"2024-09-16T15:44:45.717881+0000","last_became_active":"2024-09-16T15:44:44.698546+0000","last_became_peered":"2024-09-16T15:44:44.698546+0000","last_unstale":"2024-09-16T15:44:45.717881+0000","last_undegraded":"2024-09-16T15:44:45.717881+0000","last_fullsized":"2024-09-16T15:44:45.717881+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:03:16.348362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.730750+0000","last_change":"2024-09-16T15:44:45.719192+0000","last_active":"2024-09-16T15:44:45.730750+0000","last_peered":"2024-09-16T15:44:45.730750+0000","last_clean":"2024-09-16T15:44:45.730750+0000","last_became_active":"2024-09-16T15:44:45.716454+0000","last_became_peered":"2024-09-16T15:44:45.716454+0000","last_unstale":"2024-09-16T15:44:45.730750+0000","last_undegraded":"2024-09-16T15:44:45.730750+0000","last_fullsized":"2024-09-16T15:44:45.730750+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:05:16.681904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T15:44:44.710050+0000","last_change":"2024-09-16T15:44:44.707043+0000","last_active":"2024-09-16T15:44:44.710050+0000","last_peered":"2024-09-16T15:44:44.710050+0000","last_clean":"2024-09-16T15:44:44.710050+0000","last_became_active":"2024-09-16T15:44:44.706747+0000","last_became_peered":"2024-09-16T15:44:44.706747+0000","last_unstale":"2024-09-16T15:44:44.710050+0000","last_undegraded":"2024-09-16T15:44:44.710050+0000","last_fullsized":"2024-09-16T15:44:44.710050+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:15:01.043845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.730635+0000","last_change":"2024-09-16T15:44:45.726366+0000","last_active":"2024-09-16T15:44:45.730635+0000","last_peered":"2024-09-16T15:44:45.730635+0000","last_clean":"2024-09-16T15:44:45.730635+0000","last_became_active":"2024-09-16T15:44:45.726122+0000","last_became_peered":"2024-09-16T15:44:45.726122+0000","last_unstale":"2024-09-16T15:44:45.730635+0000","last_undegraded":"2024-09-16T15:44:45.730635+0000","last_fullsized":"2024-09-16T15:44:45.730635+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:13:21.061051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.705257+0000","last_change":"2024-09-16T15:44:44.704189+0000","last_active":"2024-09-16T15:44:45.705257+0000","last_peered":"2024-09-16T15:44:45.705257+0000","last_clean":"2024-09-16T15:44:45.705257+0000","last_became_active":"2024-09-16T15:44:44.703916+0000","last_became_peered":"2024-09-16T15:44:44.703916+0000","last_unstale":"2024-09-16T15:44:45.705257+0000","last_undegraded":"2024-09-16T15:44:45.705257+0000","last_fullsized":"2024-09-16T15:44:45.705257+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:50:09.467231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.724010+0000","last_change":"2024-09-16T15:44:45.710420+0000","last_active":"2024-09-16T15:44:45.724010+0000","last_peered":"2024-09-16T15:44:45.724010+0000","last_clean":"2024-09-16T15:44:45.724010+0000","last_became_active":"2024-09-16T15:44:45.710144+0000","last_became_peered":"2024-09-16T15:44:45.710144+0000","last_unstale":"2024-09-16T15:44:45.724010+0000","last_undegraded":"2024-09-16T15:44:45.724010+0000","last_fullsized":"2024-09-16T15:44:45.724010+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:23:06.419124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.707293+0000","last_change":"2024-09-16T15:44:44.708397+0000","last_active":"2024-09-16T15:44:45.707293+0000","last_peered":"2024-09-16T15:44:45.707293+0000","last_clean":"2024-09-16T15:44:45.707293+0000","last_became_active":"2024-09-16T15:44:44.708101+0000","last_became_peered":"2024-09-16T15:44:44.708101+0000","last_unstale":"2024-09-16T15:44:45.707293+0000","last_undegraded":"2024-09-16T15:44:45.707293+0000","last_fullsized":"2024-09-16T15:44:45.707293+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:09:26.705708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T15:44:44.691091+0000","last_change":"2024-09-16T15:44:44.691091+0000","last_active":"2024-09-16T15:44:44.679289+0000","last_peered":"2024-09-16T15:44:44.679289+0000","last_clean":"2024-09-16T15:44:44.679289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:44:44.691091+0000","last_undegraded":"2024-09-16T15:44:44.691091+0000","last_fullsized":"2024-09-16T15:44:44.691091+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.703426+0000","last_change":"2024-09-16T15:44:44.698425+0000","last_active":"2024-09-16T15:44:45.703426+0000","last_peered":"2024-09-16T15:44:45.703426+0000","last_clean":"2024-09-16T15:44:45.703426+0000","last_became_active":"2024-09-16T15:44:44.698141+0000","last_became_peered":"2024-09-16T15:44:44.698141+0000","last_unstale":"2024-09-16T15:44:45.703426+0000","last_undegraded":"2024-09-16T15:44:45.703426+0000","last_fullsized":"2024-09-16T15:44:45.703426+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:55:27.169366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.724043+0000","last_change":"2024-09-16T15:44:45.720393+0000","last_active":"2024-09-16T15:44:45.724043+0000","last_peered":"2024-09-16T15:44:45.724043+0000","last_clean":"2024-09-16T15:44:45.724043+0000","last_became_active":"2024-09-16T15:44:45.720028+0000","last_became_peered":"2024-09-16T15:44:45.720028+0000","last_unstale":"2024-09-16T15:44:45.724043+0000","last_undegraded":"2024-09-16T15:44:45.724043+0000","last_fullsized":"2024-09-16T15:44:45.724043+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:46:17.054489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.703722+0000","last_change":"2024-09-16T15:44:44.690580+0000","last_active":"2024-09-16T15:44:45.703722+0000","last_peered":"2024-09-16T15:44:45.703722+0000","last_clean":"2024-09-16T15:44:45.703722+0000","last_became_active":"2024-09-16T15:44:44.690302+0000","last_became_peered":"2024-09-16T15:44:44.690302+0000","last_unstale":"2024-09-16T15:44:45.703722+0000","last_undegraded":"2024-09-16T15:44:45.703722+0000","last_fullsized":"2024-09-16T15:44:45.703722+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:45:53.368994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":185,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.709646+0000","last_change":"2024-09-16T15:39:12.698813+0000","last_active":"2024-09-16T15:44:45.709646+0000","last_peered":"2024-09-16T15:44:45.709646+0000","last_clean":"2024-09-16T15:44:45.709646+0000","last_became_active":"2024-09-16T15:39:12.697695+0000","last_became_peered":"2024-09-16T15:39:12.697695+0000","last_unstale":"2024-09-16T15:44:45.709646+0000","last_undegraded":"2024-09-16T15:44:45.709646+0000","last_fullsized":"2024-09-16T15:44:45.709646+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_clean_scrub_stamp":"2024-09-16T15:39:11.684967+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:02.061081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2388,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":34,"ondisk_log_size":34,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509512,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40208,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331632,"statfs":{"total":96636764160,"available":96595591168,"internally_reserved":0,"allocated":712704,"data_stored":296806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":40434120},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509511,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46356,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325484,"statfs":{"total":96636764160,"available":96589295616,"internally_reserved":0,"allocated":1306624,"data_stored":887174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509510,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45772,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326068,"statfs":{"total":96636764160,"available":96589893632,"internally_reserved":0,"allocated":712704,"data_stored":296806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509510,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40204,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331636,"statfs":{"total":96636764160,"available":96595595264,"internally_reserved":0,"allocated":712704,"data_stored":296806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":40434121},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509510,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62456,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94309384,"statfs":{"total":96636764160,"available":96572809216,"internally_reserved":0,"allocated":745472,"data_stored":305381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":63175111},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509511,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40788,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331052,"statfs":{"total":96636764160,"available":96594997248,"internally_reserved":0,"allocated":1306624,"data_stored":887174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509509,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40208,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331632,"statfs":{"total":96636764160,"available":96595591168,"internally_reserved":0,"allocated":712704,"data_stored":296806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542216,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40208,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331632,"statfs":{"total":96636764160,"available":96595591168,"internally_reserved":0,"allocated":712704,"data_stored":296806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":40434124},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:44:49.958 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:44:50.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.150+0000 7f8f82800640 1 -- 172.21.15.151:0/2986853651 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f7c15bc30 msgr2=0x7f8f7c0b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:50.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.150+0000 7f8f82800640 1 --2- 172.21.15.151:0/2986853651 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f7c15bc30 0x7f8f7c0b28f0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c073b00 tx=0x7f8f7002aa30 comp rx=0 tx=0).stop 2024-09-16T15:44:50.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.150+0000 7f8f82800640 1 -- 172.21.15.151:0/2986853651 shutdown_connections 2024-09-16T15:44:50.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.150+0000 7f8f82800640 1 --2- 172.21.15.151:0/2986853651 >> v2:172.21.15.167:3300/0 conn(0x7f8f7c0725d0 0x7f8f7c073d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:50.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.150+0000 7f8f82800640 1 --2- 172.21.15.151:0/2986853651 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f7c15bc30 0x7f8f7c0b28f0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:50.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.150+0000 7f8f82800640 1 -- 172.21.15.151:0/2986853651 >> 172.21.15.151:0/2986853651 conn(0x7f8f7c0cfa20 msgr2=0x7f8f7c0cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:50.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.150+0000 7f8f82800640 1 -- 172.21.15.151:0/2986853651 shutdown_connections 2024-09-16T15:44:50.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.150+0000 7f8f82800640 1 -- 172.21.15.151:0/2986853651 wait complete. 2024-09-16T15:44:50.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.151+0000 7f8f82800640 1 Processor -- start 2024-09-16T15:44:50.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.152+0000 7f8f82800640 1 -- start start 2024-09-16T15:44:50.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.152+0000 7f8f82800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8f7c0d0540 0x7f8f7c1f4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:50.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.152+0000 7f8f82800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f7c15bc30 0x7f8f7c150e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:50.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.153+0000 7f8f82800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f7c0ce640 0x7f8f7c157b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:50.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.153+0000 7f8f82800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7c074610 con 0x7f8f7c15bc30 2024-09-16T15:44:50.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.153+0000 7f8f82800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7c074490 con 0x7f8f7c0ce640 2024-09-16T15:44:50.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.153+0000 7f8f82800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f8f7c074790 con 0x7f8f7c0d0540 2024-09-16T15:44:50.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.153+0000 7f8f80a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f7c15bc30 0x7f8f7c150e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:50.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.153+0000 7f8f80a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f7c15bc30 0x7f8f7c150e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44292/0 (socket says 172.21.15.151:44292) 2024-09-16T15:44:50.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.153+0000 7f8f80a00640 1 -- 172.21.15.151:0/4250270638 learned_addr learned my addr 172.21.15.151:0/4250270638 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:50.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.153+0000 7f8f81e00640 1 --2- 172.21.15.151:0/4250270638 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f7c0ce640 0x7f8f7c157b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:50.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.153+0000 7f8f81400640 1 --2- 172.21.15.151:0/4250270638 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8f7c0d0540 0x7f8f7c1f4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:50.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.153+0000 7f8f80a00640 1 -- 172.21.15.151:0/4250270638 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8f7c0d0540 msgr2=0x7f8f7c1f4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:50.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.153+0000 7f8f80a00640 1 --2- 172.21.15.151:0/4250270638 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8f7c0d0540 0x7f8f7c1f4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:50.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.153+0000 7f8f80a00640 1 -- 172.21.15.151:0/4250270638 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f7c0ce640 msgr2=0x7f8f7c157b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:50.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.153+0000 7f8f80a00640 1 --2- 172.21.15.151:0/4250270638 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f7c0ce640 0x7f8f7c157b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:50.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.153+0000 7f8f80a00640 1 -- 172.21.15.151:0/4250270638 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f7c0cd800 con 0x7f8f7c15bc30 2024-09-16T15:44:50.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.154+0000 7f8f80a00640 1 --2- 172.21.15.151:0/4250270638 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f7c15bc30 0x7f8f7c150e40 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f8f6800d740 tx=0x7f8f6800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:50.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.154+0000 7f8f79600640 1 -- 172.21.15.151:0/4250270638 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f68017070 con 0x7f8f7c15bc30 2024-09-16T15:44:50.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.154+0000 7f8f79600640 1 -- 172.21.15.151:0/4250270638 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8f6800de50 con 0x7f8f7c15bc30 2024-09-16T15:44:50.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.154+0000 7f8f82800640 1 -- 172.21.15.151:0/4250270638 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f7c0cdae0 con 0x7f8f7c15bc30 2024-09-16T15:44:50.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.154+0000 7f8f79600640 1 -- 172.21.15.151:0/4250270638 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f68005020 con 0x7f8f7c15bc30 2024-09-16T15:44:50.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.154+0000 7f8f82800640 1 -- 172.21.15.151:0/4250270638 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f7c0ce0a0 con 0x7f8f7c15bc30 2024-09-16T15:44:50.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.155+0000 7f8f82800640 1 -- 172.21.15.151:0/4250270638 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f7c0b4a10 con 0x7f8f7c15bc30 2024-09-16T15:44:50.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.156+0000 7f8f79600640 1 -- 172.21.15.151:0/4250270638 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 102103+0+0 (secure 0 0 0) 0x7f8f680051c0 con 0x7f8f7c15bc30 2024-09-16T15:44:50.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.157+0000 7f8f79600640 1 --2- 172.21.15.151:0/4250270638 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8f5c079150 0x7f8f5c07b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:50.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.157+0000 7f8f79600640 1 -- 172.21.15.151:0/4250270638 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7f8f680992b0 con 0x7f8f7c15bc30 2024-09-16T15:44:50.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.157+0000 7f8f81400640 1 --2- 172.21.15.151:0/4250270638 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8f5c079150 0x7f8f5c07b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:50.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.158+0000 7f8f81400640 1 --2- 172.21.15.151:0/4250270638 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8f5c079150 0x7f8f5c07b610 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8f70035000 tx=0x7f8f7003b040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:50.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.161+0000 7f8f79600640 1 -- 172.21.15.151:0/4250270638 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8f7c0b4a10 con 0x7f8f7c15bc30 2024-09-16T15:44:50.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.344+0000 7f8f82800640 1 -- 172.21.15.151:0/4250270638 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8f7c06fd20 con 0x7f8f5c079150 2024-09-16T15:44:50.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.350+0000 7f8f79600640 1 -- 172.21.15.151:0/4250270638 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251283 (secure 0 0 0) 0x7f8f7c06fd20 con 0x7f8f5c079150 2024-09-16T15:44:50.354 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:44:50.355 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-09-16T15:44:50.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.354+0000 7f8f82800640 1 -- 172.21.15.151:0/4250270638 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8f5c079150 msgr2=0x7f8f5c07b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:50.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.354+0000 7f8f82800640 1 --2- 172.21.15.151:0/4250270638 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8f5c079150 0x7f8f5c07b610 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8f70035000 tx=0x7f8f7003b040 comp rx=0 tx=0).stop 2024-09-16T15:44:50.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.354+0000 7f8f82800640 1 -- 172.21.15.151:0/4250270638 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f7c15bc30 msgr2=0x7f8f7c150e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:50.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.354+0000 7f8f82800640 1 --2- 172.21.15.151:0/4250270638 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f7c15bc30 0x7f8f7c150e40 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f8f6800d740 tx=0x7f8f6800dc00 comp rx=0 tx=0).stop 2024-09-16T15:44:50.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.354+0000 7f8f82800640 1 -- 172.21.15.151:0/4250270638 shutdown_connections 2024-09-16T15:44:50.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.354+0000 7f8f82800640 1 --2- 172.21.15.151:0/4250270638 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8f5c079150 0x7f8f5c07b610 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:50.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.354+0000 7f8f82800640 1 --2- 172.21.15.151:0/4250270638 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f7c0ce640 0x7f8f7c157b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:50.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.354+0000 7f8f82800640 1 --2- 172.21.15.151:0/4250270638 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f7c15bc30 0x7f8f7c150e40 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:50.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.354+0000 7f8f82800640 1 --2- 172.21.15.151:0/4250270638 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8f7c0d0540 0x7f8f7c1f4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:50.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.354+0000 7f8f82800640 1 -- 172.21.15.151:0/4250270638 >> 172.21.15.151:0/4250270638 conn(0x7f8f7c0cfa20 msgr2=0x7f8f7c0c9ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:50.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.354+0000 7f8f82800640 1 -- 172.21.15.151:0/4250270638 shutdown_connections 2024-09-16T15:44:50.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.354+0000 7f8f82800640 1 -- 172.21.15.151:0/4250270638 wait complete. 2024-09-16T15:44:50.372 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":225,"stamp":"2024-09-16T15:44:49.670408+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592756,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":141,"num_write_kb":1641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":106,"ondisk_log_size":106,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":32,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":356200,"kb_used_data":6760,"kb_used_omap":12,"kb_used_meta":349235,"kb_avail":754618520,"statfs":{"total":773094113280,"available":772729364480,"internally_reserved":0,"allocated":6922240,"data_stored":3563759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12738,"internal_metadata":357617214},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2388,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.988477"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.732875+0000","last_change":"2024-09-16T15:44:45.728739+0000","last_active":"2024-09-16T15:44:45.732875+0000","last_peered":"2024-09-16T15:44:45.732875+0000","last_clean":"2024-09-16T15:44:45.732875+0000","last_became_active":"2024-09-16T15:44:45.727368+0000","last_became_peered":"2024-09-16T15:44:45.727368+0000","last_unstale":"2024-09-16T15:44:45.732875+0000","last_undegraded":"2024-09-16T15:44:45.732875+0000","last_fullsized":"2024-09-16T15:44:45.732875+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:34:55.972702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T15:44:44.692975+0000","last_change":"2024-09-16T15:44:44.692975+0000","last_active":"2024-09-16T15:44:44.679289+0000","last_peered":"2024-09-16T15:44:44.679289+0000","last_clean":"2024-09-16T15:44:44.679289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:44:44.692975+0000","last_undegraded":"2024-09-16T15:44:44.692975+0000","last_fullsized":"2024-09-16T15:44:44.692975+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.732389+0000","last_change":"2024-09-16T15:44:45.725780+0000","last_active":"2024-09-16T15:44:45.732389+0000","last_peered":"2024-09-16T15:44:45.732389+0000","last_clean":"2024-09-16T15:44:45.732389+0000","last_became_active":"2024-09-16T15:44:45.721333+0000","last_became_peered":"2024-09-16T15:44:45.721333+0000","last_unstale":"2024-09-16T15:44:45.732389+0000","last_undegraded":"2024-09-16T15:44:45.732389+0000","last_fullsized":"2024-09-16T15:44:45.732389+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:13:04.403633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.737448+0000","last_change":"2024-09-16T15:44:45.729388+0000","last_active":"2024-09-16T15:44:45.737448+0000","last_peered":"2024-09-16T15:44:45.737448+0000","last_clean":"2024-09-16T15:44:45.737448+0000","last_became_active":"2024-09-16T15:44:45.729119+0000","last_became_peered":"2024-09-16T15:44:45.729119+0000","last_unstale":"2024-09-16T15:44:45.737448+0000","last_undegraded":"2024-09-16T15:44:45.737448+0000","last_fullsized":"2024-09-16T15:44:45.737448+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:08:12.766873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.738580+0000","last_change":"2024-09-16T15:44:45.733586+0000","last_active":"2024-09-16T15:44:45.738580+0000","last_peered":"2024-09-16T15:44:45.738580+0000","last_clean":"2024-09-16T15:44:45.738580+0000","last_became_active":"2024-09-16T15:44:45.733144+0000","last_became_peered":"2024-09-16T15:44:45.733144+0000","last_unstale":"2024-09-16T15:44:45.738580+0000","last_undegraded":"2024-09-16T15:44:45.738580+0000","last_fullsized":"2024-09-16T15:44:45.738580+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:04:50.517949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.730726+0000","last_change":"2024-09-16T15:44:45.718049+0000","last_active":"2024-09-16T15:44:45.730726+0000","last_peered":"2024-09-16T15:44:45.730726+0000","last_clean":"2024-09-16T15:44:45.730726+0000","last_became_active":"2024-09-16T15:44:45.716815+0000","last_became_peered":"2024-09-16T15:44:45.716815+0000","last_unstale":"2024-09-16T15:44:45.730726+0000","last_undegraded":"2024-09-16T15:44:45.730726+0000","last_fullsized":"2024-09-16T15:44:45.730726+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:30:51.703628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.724140+0000","last_change":"2024-09-16T15:44:45.718523+0000","last_active":"2024-09-16T15:44:45.724140+0000","last_peered":"2024-09-16T15:44:45.724140+0000","last_clean":"2024-09-16T15:44:45.724140+0000","last_became_active":"2024-09-16T15:44:45.718208+0000","last_became_peered":"2024-09-16T15:44:45.718208+0000","last_unstale":"2024-09-16T15:44:45.724140+0000","last_undegraded":"2024-09-16T15:44:45.724140+0000","last_fullsized":"2024-09-16T15:44:45.724140+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:58:34.310843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.724066+0000","last_change":"2024-09-16T15:44:45.720647+0000","last_active":"2024-09-16T15:44:45.724066+0000","last_peered":"2024-09-16T15:44:45.724066+0000","last_clean":"2024-09-16T15:44:45.724066+0000","last_became_active":"2024-09-16T15:44:45.720351+0000","last_became_peered":"2024-09-16T15:44:45.720351+0000","last_unstale":"2024-09-16T15:44:45.724066+0000","last_undegraded":"2024-09-16T15:44:45.724066+0000","last_fullsized":"2024-09-16T15:44:45.724066+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:44:06.889108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.734233+0000","last_change":"2024-09-16T15:44:45.732987+0000","last_active":"2024-09-16T15:44:45.734233+0000","last_peered":"2024-09-16T15:44:45.734233+0000","last_clean":"2024-09-16T15:44:45.734233+0000","last_became_active":"2024-09-16T15:44:45.732680+0000","last_became_peered":"2024-09-16T15:44:45.732680+0000","last_unstale":"2024-09-16T15:44:45.734233+0000","last_undegraded":"2024-09-16T15:44:45.734233+0000","last_fullsized":"2024-09-16T15:44:45.734233+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:21:56.720871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.730690+0000","last_change":"2024-09-16T15:44:45.719120+0000","last_active":"2024-09-16T15:44:45.730690+0000","last_peered":"2024-09-16T15:44:45.730690+0000","last_clean":"2024-09-16T15:44:45.730690+0000","last_became_active":"2024-09-16T15:44:45.718746+0000","last_became_peered":"2024-09-16T15:44:45.718746+0000","last_unstale":"2024-09-16T15:44:45.730690+0000","last_undegraded":"2024-09-16T15:44:45.730690+0000","last_fullsized":"2024-09-16T15:44:45.730690+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:13:46.973489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.726293+0000","last_change":"2024-09-16T15:44:45.718730+0000","last_active":"2024-09-16T15:44:45.726293+0000","last_peered":"2024-09-16T15:44:45.726293+0000","last_clean":"2024-09-16T15:44:45.726293+0000","last_became_active":"2024-09-16T15:44:45.718442+0000","last_became_peered":"2024-09-16T15:44:45.718442+0000","last_unstale":"2024-09-16T15:44:45.726293+0000","last_undegraded":"2024-09-16T15:44:45.726293+0000","last_fullsized":"2024-09-16T15:44:45.726293+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:13:04.403633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.720994+0000","last_change":"2024-09-16T15:44:45.717536+0000","last_active":"2024-09-16T15:44:45.720994+0000","last_peered":"2024-09-16T15:44:45.720994+0000","last_clean":"2024-09-16T15:44:45.720994+0000","last_became_active":"2024-09-16T15:44:45.717226+0000","last_became_peered":"2024-09-16T15:44:45.717226+0000","last_unstale":"2024-09-16T15:44:45.720994+0000","last_undegraded":"2024-09-16T15:44:45.720994+0000","last_fullsized":"2024-09-16T15:44:45.720994+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:46:33.994315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.719703+0000","last_change":"2024-09-16T15:44:45.711525+0000","last_active":"2024-09-16T15:44:45.719703+0000","last_peered":"2024-09-16T15:44:45.719703+0000","last_clean":"2024-09-16T15:44:45.719703+0000","last_became_active":"2024-09-16T15:44:45.711195+0000","last_became_peered":"2024-09-16T15:44:45.711195+0000","last_unstale":"2024-09-16T15:44:45.719703+0000","last_undegraded":"2024-09-16T15:44:45.719703+0000","last_fullsized":"2024-09-16T15:44:45.719703+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:42:47.930299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.732183+0000","last_change":"2024-09-16T15:44:45.726781+0000","last_active":"2024-09-16T15:44:45.732183+0000","last_peered":"2024-09-16T15:44:45.732183+0000","last_clean":"2024-09-16T15:44:45.732183+0000","last_became_active":"2024-09-16T15:44:45.726045+0000","last_became_peered":"2024-09-16T15:44:45.726045+0000","last_unstale":"2024-09-16T15:44:45.732183+0000","last_undegraded":"2024-09-16T15:44:45.732183+0000","last_fullsized":"2024-09-16T15:44:45.732183+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:36:30.549487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.719163+0000","last_change":"2024-09-16T15:44:45.716384+0000","last_active":"2024-09-16T15:44:45.719163+0000","last_peered":"2024-09-16T15:44:45.719163+0000","last_clean":"2024-09-16T15:44:45.719163+0000","last_became_active":"2024-09-16T15:44:45.716101+0000","last_became_peered":"2024-09-16T15:44:45.716101+0000","last_unstale":"2024-09-16T15:44:45.719163+0000","last_undegraded":"2024-09-16T15:44:45.719163+0000","last_fullsized":"2024-09-16T15:44:45.719163+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:21:51.586704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.730872+0000","last_change":"2024-09-16T15:44:45.721590+0000","last_active":"2024-09-16T15:44:45.730872+0000","last_peered":"2024-09-16T15:44:45.730872+0000","last_clean":"2024-09-16T15:44:45.730872+0000","last_became_active":"2024-09-16T15:44:45.721343+0000","last_became_peered":"2024-09-16T15:44:45.721343+0000","last_unstale":"2024-09-16T15:44:45.730872+0000","last_undegraded":"2024-09-16T15:44:45.730872+0000","last_fullsized":"2024-09-16T15:44:45.730872+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:44:28.021891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.730936+0000","last_change":"2024-09-16T15:44:45.722006+0000","last_active":"2024-09-16T15:44:45.730936+0000","last_peered":"2024-09-16T15:44:45.730936+0000","last_clean":"2024-09-16T15:44:45.730936+0000","last_became_active":"2024-09-16T15:44:45.721714+0000","last_became_peered":"2024-09-16T15:44:45.721714+0000","last_unstale":"2024-09-16T15:44:45.730936+0000","last_undegraded":"2024-09-16T15:44:45.730936+0000","last_fullsized":"2024-09-16T15:44:45.730936+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:53:59.005172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.736449+0000","last_change":"2024-09-16T15:44:45.732870+0000","last_active":"2024-09-16T15:44:45.736449+0000","last_peered":"2024-09-16T15:44:45.736449+0000","last_clean":"2024-09-16T15:44:45.736449+0000","last_became_active":"2024-09-16T15:44:45.732609+0000","last_became_peered":"2024-09-16T15:44:45.732609+0000","last_unstale":"2024-09-16T15:44:45.736449+0000","last_undegraded":"2024-09-16T15:44:45.736449+0000","last_fullsized":"2024-09-16T15:44:45.736449+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:44:16.275512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T15:44:44.691707+0000","last_change":"2024-09-16T15:44:44.691707+0000","last_active":"2024-09-16T15:44:44.679289+0000","last_peered":"2024-09-16T15:44:44.679289+0000","last_clean":"2024-09-16T15:44:44.679289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:44:44.691707+0000","last_undegraded":"2024-09-16T15:44:44.691707+0000","last_fullsized":"2024-09-16T15:44:44.691707+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.734176+0000","last_change":"2024-09-16T15:44:45.731373+0000","last_active":"2024-09-16T15:44:45.734176+0000","last_peered":"2024-09-16T15:44:45.734176+0000","last_clean":"2024-09-16T15:44:45.734176+0000","last_became_active":"2024-09-16T15:44:45.731068+0000","last_became_peered":"2024-09-16T15:44:45.731068+0000","last_unstale":"2024-09-16T15:44:45.734176+0000","last_undegraded":"2024-09-16T15:44:45.734176+0000","last_fullsized":"2024-09-16T15:44:45.734176+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:51:57.452114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.724617+0000","last_change":"2024-09-16T15:44:45.719386+0000","last_active":"2024-09-16T15:44:45.724617+0000","last_peered":"2024-09-16T15:44:45.724617+0000","last_clean":"2024-09-16T15:44:45.724617+0000","last_became_active":"2024-09-16T15:44:45.719154+0000","last_became_peered":"2024-09-16T15:44:45.719154+0000","last_unstale":"2024-09-16T15:44:45.724617+0000","last_undegraded":"2024-09-16T15:44:45.724617+0000","last_fullsized":"2024-09-16T15:44:45.724617+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:11:14.213089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.723225+0000","last_change":"2024-09-16T15:44:45.718300+0000","last_active":"2024-09-16T15:44:45.723225+0000","last_peered":"2024-09-16T15:44:45.723225+0000","last_clean":"2024-09-16T15:44:45.723225+0000","last_became_active":"2024-09-16T15:44:45.717725+0000","last_became_peered":"2024-09-16T15:44:45.717725+0000","last_unstale":"2024-09-16T15:44:45.723225+0000","last_undegraded":"2024-09-16T15:44:45.723225+0000","last_fullsized":"2024-09-16T15:44:45.723225+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:03:17.351052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.721190+0000","last_change":"2024-09-16T15:44:45.716075+0000","last_active":"2024-09-16T15:44:45.721190+0000","last_peered":"2024-09-16T15:44:45.721190+0000","last_clean":"2024-09-16T15:44:45.721190+0000","last_became_active":"2024-09-16T15:44:45.715816+0000","last_became_peered":"2024-09-16T15:44:45.715816+0000","last_unstale":"2024-09-16T15:44:45.721190+0000","last_undegraded":"2024-09-16T15:44:45.721190+0000","last_fullsized":"2024-09-16T15:44:45.721190+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:38:03.977000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.725164+0000","last_change":"2024-09-16T15:44:45.721703+0000","last_active":"2024-09-16T15:44:45.725164+0000","last_peered":"2024-09-16T15:44:45.725164+0000","last_clean":"2024-09-16T15:44:45.725164+0000","last_became_active":"2024-09-16T15:44:45.721406+0000","last_became_peered":"2024-09-16T15:44:45.721406+0000","last_unstale":"2024-09-16T15:44:45.725164+0000","last_undegraded":"2024-09-16T15:44:45.725164+0000","last_fullsized":"2024-09-16T15:44:45.725164+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:42:26.902672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T15:44:44.691255+0000","last_change":"2024-09-16T15:44:44.691255+0000","last_active":"2024-09-16T15:44:44.679289+0000","last_peered":"2024-09-16T15:44:44.679289+0000","last_clean":"2024-09-16T15:44:44.679289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:44:44.691255+0000","last_undegraded":"2024-09-16T15:44:44.691255+0000","last_fullsized":"2024-09-16T15:44:44.691255+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.725345+0000","last_change":"2024-09-16T15:44:45.722538+0000","last_active":"2024-09-16T15:44:45.725345+0000","last_peered":"2024-09-16T15:44:45.725345+0000","last_clean":"2024-09-16T15:44:45.725345+0000","last_became_active":"2024-09-16T15:44:45.722286+0000","last_became_peered":"2024-09-16T15:44:45.722286+0000","last_unstale":"2024-09-16T15:44:45.725345+0000","last_undegraded":"2024-09-16T15:44:45.725345+0000","last_fullsized":"2024-09-16T15:44:45.725345+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:45:58.209870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T15:44:44.691255+0000","last_change":"2024-09-16T15:44:44.691255+0000","last_active":"2024-09-16T15:44:44.679289+0000","last_peered":"2024-09-16T15:44:44.679289+0000","last_clean":"2024-09-16T15:44:44.679289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:44:44.691255+0000","last_undegraded":"2024-09-16T15:44:44.691255+0000","last_fullsized":"2024-09-16T15:44:44.691255+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.722818+0000","last_change":"2024-09-16T15:44:45.707684+0000","last_active":"2024-09-16T15:44:45.722818+0000","last_peered":"2024-09-16T15:44:45.722818+0000","last_clean":"2024-09-16T15:44:45.722818+0000","last_became_active":"2024-09-16T15:44:45.707370+0000","last_became_peered":"2024-09-16T15:44:45.707370+0000","last_unstale":"2024-09-16T15:44:45.722818+0000","last_undegraded":"2024-09-16T15:44:45.722818+0000","last_fullsized":"2024-09-16T15:44:45.722818+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:32:11.331276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.724083+0000","last_change":"2024-09-16T15:44:45.712553+0000","last_active":"2024-09-16T15:44:45.724083+0000","last_peered":"2024-09-16T15:44:45.724083+0000","last_clean":"2024-09-16T15:44:45.724083+0000","last_became_active":"2024-09-16T15:44:45.711955+0000","last_became_peered":"2024-09-16T15:44:45.711955+0000","last_unstale":"2024-09-16T15:44:45.724083+0000","last_undegraded":"2024-09-16T15:44:45.724083+0000","last_fullsized":"2024-09-16T15:44:45.724083+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:33:23.442599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.714841+0000","last_change":"2024-09-16T15:44:45.711869+0000","last_active":"2024-09-16T15:44:45.714841+0000","last_peered":"2024-09-16T15:44:45.714841+0000","last_clean":"2024-09-16T15:44:45.714841+0000","last_became_active":"2024-09-16T15:44:45.711534+0000","last_became_peered":"2024-09-16T15:44:45.711534+0000","last_unstale":"2024-09-16T15:44:45.714841+0000","last_undegraded":"2024-09-16T15:44:45.714841+0000","last_fullsized":"2024-09-16T15:44:45.714841+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:31:16.141860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T15:44:44.691565+0000","last_change":"2024-09-16T15:44:44.691565+0000","last_active":"2024-09-16T15:44:44.679289+0000","last_peered":"2024-09-16T15:44:44.679289+0000","last_clean":"2024-09-16T15:44:44.679289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:44:44.691565+0000","last_undegraded":"2024-09-16T15:44:44.691565+0000","last_fullsized":"2024-09-16T15:44:44.691565+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.729551+0000","last_change":"2024-09-16T15:44:45.726051+0000","last_active":"2024-09-16T15:44:45.729551+0000","last_peered":"2024-09-16T15:44:45.729551+0000","last_clean":"2024-09-16T15:44:45.729551+0000","last_became_active":"2024-09-16T15:44:45.720897+0000","last_became_peered":"2024-09-16T15:44:45.720897+0000","last_unstale":"2024-09-16T15:44:45.729551+0000","last_undegraded":"2024-09-16T15:44:45.729551+0000","last_fullsized":"2024-09-16T15:44:45.729551+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:36:30.549487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.732821+0000","last_change":"2024-09-16T15:44:45.724124+0000","last_active":"2024-09-16T15:44:45.732821+0000","last_peered":"2024-09-16T15:44:45.732821+0000","last_clean":"2024-09-16T15:44:45.732821+0000","last_became_active":"2024-09-16T15:44:45.721712+0000","last_became_peered":"2024-09-16T15:44:45.721712+0000","last_unstale":"2024-09-16T15:44:45.732821+0000","last_undegraded":"2024-09-16T15:44:45.732821+0000","last_fullsized":"2024-09-16T15:44:45.732821+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:15:22.807776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"61'4","reported_seq":17,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.732008+0000","last_change":"2024-09-16T15:44:44.700510+0000","last_active":"2024-09-16T15:44:45.732008+0000","last_peered":"2024-09-16T15:44:45.732008+0000","last_clean":"2024-09-16T15:44:45.732008+0000","last_became_active":"2024-09-16T15:44:44.697806+0000","last_became_peered":"2024-09-16T15:44:44.697806+0000","last_unstale":"2024-09-16T15:44:45.732008+0000","last_undegraded":"2024-09-16T15:44:45.732008+0000","last_fullsized":"2024-09-16T15:44:45.732008+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:50:52.878214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.729071+0000","last_change":"2024-09-16T15:44:45.715988+0000","last_active":"2024-09-16T15:44:45.729071+0000","last_peered":"2024-09-16T15:44:45.729071+0000","last_clean":"2024-09-16T15:44:45.729071+0000","last_became_active":"2024-09-16T15:44:45.715735+0000","last_became_peered":"2024-09-16T15:44:45.715735+0000","last_unstale":"2024-09-16T15:44:45.729071+0000","last_undegraded":"2024-09-16T15:44:45.729071+0000","last_fullsized":"2024-09-16T15:44:45.729071+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:39:12.784333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T15:44:44.710442+0000","last_change":"2024-09-16T15:44:44.708252+0000","last_active":"2024-09-16T15:44:44.710442+0000","last_peered":"2024-09-16T15:44:44.710442+0000","last_clean":"2024-09-16T15:44:44.710442+0000","last_became_active":"2024-09-16T15:44:44.707995+0000","last_became_peered":"2024-09-16T15:44:44.707995+0000","last_unstale":"2024-09-16T15:44:44.710442+0000","last_undegraded":"2024-09-16T15:44:44.710442+0000","last_fullsized":"2024-09-16T15:44:44.710442+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:26:28.068533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.734118+0000","last_change":"2024-09-16T15:44:45.729683+0000","last_active":"2024-09-16T15:44:45.734118+0000","last_peered":"2024-09-16T15:44:45.734118+0000","last_clean":"2024-09-16T15:44:45.734118+0000","last_became_active":"2024-09-16T15:44:45.729258+0000","last_became_peered":"2024-09-16T15:44:45.729258+0000","last_unstale":"2024-09-16T15:44:45.734118+0000","last_undegraded":"2024-09-16T15:44:45.734118+0000","last_fullsized":"2024-09-16T15:44:45.734118+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:34:41.864039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"61'2","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.739158+0000","last_change":"2024-09-16T15:44:44.705295+0000","last_active":"2024-09-16T15:44:45.739158+0000","last_peered":"2024-09-16T15:44:45.739158+0000","last_clean":"2024-09-16T15:44:45.739158+0000","last_became_active":"2024-09-16T15:44:44.704913+0000","last_became_peered":"2024-09-16T15:44:44.704913+0000","last_unstale":"2024-09-16T15:44:45.739158+0000","last_undegraded":"2024-09-16T15:44:45.739158+0000","last_fullsized":"2024-09-16T15:44:45.739158+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:33:20.871493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.732416+0000","last_change":"2024-09-16T15:44:45.724296+0000","last_active":"2024-09-16T15:44:45.732416+0000","last_peered":"2024-09-16T15:44:45.732416+0000","last_clean":"2024-09-16T15:44:45.732416+0000","last_became_active":"2024-09-16T15:44:45.722552+0000","last_became_peered":"2024-09-16T15:44:45.722552+0000","last_unstale":"2024-09-16T15:44:45.732416+0000","last_undegraded":"2024-09-16T15:44:45.732416+0000","last_fullsized":"2024-09-16T15:44:45.732416+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:35:05.861042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"61'2","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.737888+0000","last_change":"2024-09-16T15:44:44.701949+0000","last_active":"2024-09-16T15:44:45.737888+0000","last_peered":"2024-09-16T15:44:45.737888+0000","last_clean":"2024-09-16T15:44:45.737888+0000","last_became_active":"2024-09-16T15:44:44.701522+0000","last_became_peered":"2024-09-16T15:44:44.701522+0000","last_unstale":"2024-09-16T15:44:45.737888+0000","last_undegraded":"2024-09-16T15:44:45.737888+0000","last_fullsized":"2024-09-16T15:44:45.737888+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:18:22.181140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.723888+0000","last_change":"2024-09-16T15:44:45.712376+0000","last_active":"2024-09-16T15:44:45.723888+0000","last_peered":"2024-09-16T15:44:45.723888+0000","last_clean":"2024-09-16T15:44:45.723888+0000","last_became_active":"2024-09-16T15:44:45.712045+0000","last_became_peered":"2024-09-16T15:44:45.712045+0000","last_unstale":"2024-09-16T15:44:45.723888+0000","last_undegraded":"2024-09-16T15:44:45.723888+0000","last_fullsized":"2024-09-16T15:44:45.723888+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:20:02.525231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"61'1","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.741440+0000","last_change":"2024-09-16T15:44:44.705164+0000","last_active":"2024-09-16T15:44:45.741440+0000","last_peered":"2024-09-16T15:44:45.741440+0000","last_clean":"2024-09-16T15:44:45.741440+0000","last_became_active":"2024-09-16T15:44:44.704525+0000","last_became_peered":"2024-09-16T15:44:44.704525+0000","last_unstale":"2024-09-16T15:44:45.741440+0000","last_undegraded":"2024-09-16T15:44:45.741440+0000","last_fullsized":"2024-09-16T15:44:45.741440+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:37:15.032653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.724052+0000","last_change":"2024-09-16T15:44:45.719896+0000","last_active":"2024-09-16T15:44:45.724052+0000","last_peered":"2024-09-16T15:44:45.724052+0000","last_clean":"2024-09-16T15:44:45.724052+0000","last_became_active":"2024-09-16T15:44:45.718945+0000","last_became_peered":"2024-09-16T15:44:45.718945+0000","last_unstale":"2024-09-16T15:44:45.724052+0000","last_undegraded":"2024-09-16T15:44:45.724052+0000","last_fullsized":"2024-09-16T15:44:45.724052+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:31:16.283923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"61'1","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.736043+0000","last_change":"2024-09-16T15:44:44.710842+0000","last_active":"2024-09-16T15:44:45.736043+0000","last_peered":"2024-09-16T15:44:45.736043+0000","last_clean":"2024-09-16T15:44:45.736043+0000","last_became_active":"2024-09-16T15:44:44.710529+0000","last_became_peered":"2024-09-16T15:44:44.710529+0000","last_unstale":"2024-09-16T15:44:45.736043+0000","last_undegraded":"2024-09-16T15:44:45.736043+0000","last_fullsized":"2024-09-16T15:44:45.736043+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:36:04.434843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.716864+0000","last_change":"2024-09-16T15:44:45.708591+0000","last_active":"2024-09-16T15:44:45.716864+0000","last_peered":"2024-09-16T15:44:45.716864+0000","last_clean":"2024-09-16T15:44:45.716864+0000","last_became_active":"2024-09-16T15:44:45.708350+0000","last_became_peered":"2024-09-16T15:44:45.708350+0000","last_unstale":"2024-09-16T15:44:45.716864+0000","last_undegraded":"2024-09-16T15:44:45.716864+0000","last_fullsized":"2024-09-16T15:44:45.716864+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:28:57.953253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.704237+0000","last_change":"2024-09-16T15:44:44.697711+0000","last_active":"2024-09-16T15:44:45.704237+0000","last_peered":"2024-09-16T15:44:45.704237+0000","last_clean":"2024-09-16T15:44:45.704237+0000","last_became_active":"2024-09-16T15:44:44.697357+0000","last_became_peered":"2024-09-16T15:44:44.697357+0000","last_unstale":"2024-09-16T15:44:45.704237+0000","last_undegraded":"2024-09-16T15:44:45.704237+0000","last_fullsized":"2024-09-16T15:44:45.704237+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:20:52.337348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.730799+0000","last_change":"2024-09-16T15:44:45.719359+0000","last_active":"2024-09-16T15:44:45.730799+0000","last_peered":"2024-09-16T15:44:45.730799+0000","last_clean":"2024-09-16T15:44:45.730799+0000","last_became_active":"2024-09-16T15:44:45.719093+0000","last_became_peered":"2024-09-16T15:44:45.719093+0000","last_unstale":"2024-09-16T15:44:45.730799+0000","last_undegraded":"2024-09-16T15:44:45.730799+0000","last_fullsized":"2024-09-16T15:44:45.730799+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:02:03.820471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.703161+0000","last_change":"2024-09-16T15:44:44.694298+0000","last_active":"2024-09-16T15:44:45.703161+0000","last_peered":"2024-09-16T15:44:45.703161+0000","last_clean":"2024-09-16T15:44:45.703161+0000","last_became_active":"2024-09-16T15:44:44.694052+0000","last_became_peered":"2024-09-16T15:44:44.694052+0000","last_unstale":"2024-09-16T15:44:45.703161+0000","last_undegraded":"2024-09-16T15:44:45.703161+0000","last_fullsized":"2024-09-16T15:44:45.703161+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:25:26.438122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.715275+0000","last_change":"2024-09-16T15:44:45.713168+0000","last_active":"2024-09-16T15:44:45.715275+0000","last_peered":"2024-09-16T15:44:45.715275+0000","last_clean":"2024-09-16T15:44:45.715275+0000","last_became_active":"2024-09-16T15:44:45.711840+0000","last_became_peered":"2024-09-16T15:44:45.711840+0000","last_unstale":"2024-09-16T15:44:45.715275+0000","last_undegraded":"2024-09-16T15:44:45.715275+0000","last_fullsized":"2024-09-16T15:44:45.715275+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:21:48.643444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.709187+0000","last_change":"2024-09-16T15:44:44.703702+0000","last_active":"2024-09-16T15:44:45.709187+0000","last_peered":"2024-09-16T15:44:45.709187+0000","last_clean":"2024-09-16T15:44:45.709187+0000","last_became_active":"2024-09-16T15:44:44.703424+0000","last_became_peered":"2024-09-16T15:44:44.703424+0000","last_unstale":"2024-09-16T15:44:45.709187+0000","last_undegraded":"2024-09-16T15:44:45.709187+0000","last_fullsized":"2024-09-16T15:44:45.709187+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:24:00.275655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.734732+0000","last_change":"2024-09-16T15:44:45.726605+0000","last_active":"2024-09-16T15:44:45.734732+0000","last_peered":"2024-09-16T15:44:45.734732+0000","last_clean":"2024-09-16T15:44:45.734732+0000","last_became_active":"2024-09-16T15:44:45.726368+0000","last_became_peered":"2024-09-16T15:44:45.726368+0000","last_unstale":"2024-09-16T15:44:45.734732+0000","last_undegraded":"2024-09-16T15:44:45.734732+0000","last_fullsized":"2024-09-16T15:44:45.734732+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:31:16.283923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.704573+0000","last_change":"2024-09-16T15:44:44.695923+0000","last_active":"2024-09-16T15:44:45.704573+0000","last_peered":"2024-09-16T15:44:45.704573+0000","last_clean":"2024-09-16T15:44:45.704573+0000","last_became_active":"2024-09-16T15:44:44.695618+0000","last_became_peered":"2024-09-16T15:44:44.695618+0000","last_unstale":"2024-09-16T15:44:45.704573+0000","last_undegraded":"2024-09-16T15:44:45.704573+0000","last_fullsized":"2024-09-16T15:44:45.704573+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:41:35.721501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.728654+0000","last_change":"2024-09-16T15:44:45.723911+0000","last_active":"2024-09-16T15:44:45.728654+0000","last_peered":"2024-09-16T15:44:45.728654+0000","last_clean":"2024-09-16T15:44:45.728654+0000","last_became_active":"2024-09-16T15:44:45.723557+0000","last_became_peered":"2024-09-16T15:44:45.723557+0000","last_unstale":"2024-09-16T15:44:45.728654+0000","last_undegraded":"2024-09-16T15:44:45.728654+0000","last_fullsized":"2024-09-16T15:44:45.728654+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:18:23.183829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"61'3","reported_seq":16,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.737703+0000","last_change":"2024-09-16T15:44:44.700702+0000","last_active":"2024-09-16T15:44:45.737703+0000","last_peered":"2024-09-16T15:44:45.737703+0000","last_clean":"2024-09-16T15:44:45.737703+0000","last_became_active":"2024-09-16T15:44:44.700438+0000","last_became_peered":"2024-09-16T15:44:44.700438+0000","last_unstale":"2024-09-16T15:44:45.737703+0000","last_undegraded":"2024-09-16T15:44:45.737703+0000","last_fullsized":"2024-09-16T15:44:45.737703+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:46:15.633475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.713366+0000","last_change":"2024-09-16T15:44:45.709431+0000","last_active":"2024-09-16T15:44:45.713366+0000","last_peered":"2024-09-16T15:44:45.713366+0000","last_clean":"2024-09-16T15:44:45.713366+0000","last_became_active":"2024-09-16T15:44:45.709179+0000","last_became_peered":"2024-09-16T15:44:45.709179+0000","last_unstale":"2024-09-16T15:44:45.713366+0000","last_undegraded":"2024-09-16T15:44:45.713366+0000","last_fullsized":"2024-09-16T15:44:45.713366+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:34:54.933570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"61'6","reported_seq":19,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.743392+0000","last_change":"2024-09-16T15:44:44.691387+0000","last_active":"2024-09-16T15:44:45.743392+0000","last_peered":"2024-09-16T15:44:45.743392+0000","last_clean":"2024-09-16T15:44:45.743392+0000","last_became_active":"2024-09-16T15:44:44.691144+0000","last_became_peered":"2024-09-16T15:44:44.691144+0000","last_unstale":"2024-09-16T15:44:45.743392+0000","last_undegraded":"2024-09-16T15:44:45.743392+0000","last_fullsized":"2024-09-16T15:44:45.743392+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:09:54.112439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.723072+0000","last_change":"2024-09-16T15:44:45.717715+0000","last_active":"2024-09-16T15:44:45.723072+0000","last_peered":"2024-09-16T15:44:45.723072+0000","last_clean":"2024-09-16T15:44:45.723072+0000","last_became_active":"2024-09-16T15:44:45.717366+0000","last_became_peered":"2024-09-16T15:44:45.717366+0000","last_unstale":"2024-09-16T15:44:45.723072+0000","last_undegraded":"2024-09-16T15:44:45.723072+0000","last_fullsized":"2024-09-16T15:44:45.723072+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:12:07.274566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"61'2","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.730407+0000","last_change":"2024-09-16T15:44:44.702026+0000","last_active":"2024-09-16T15:44:45.730407+0000","last_peered":"2024-09-16T15:44:45.730407+0000","last_clean":"2024-09-16T15:44:45.730407+0000","last_became_active":"2024-09-16T15:44:44.701701+0000","last_became_peered":"2024-09-16T15:44:44.701701+0000","last_unstale":"2024-09-16T15:44:45.730407+0000","last_undegraded":"2024-09-16T15:44:45.730407+0000","last_fullsized":"2024-09-16T15:44:45.730407+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:30:25.597599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.737862+0000","last_change":"2024-09-16T15:44:45.734880+0000","last_active":"2024-09-16T15:44:45.737862+0000","last_peered":"2024-09-16T15:44:45.737862+0000","last_clean":"2024-09-16T15:44:45.737862+0000","last_became_active":"2024-09-16T15:44:45.734616+0000","last_became_peered":"2024-09-16T15:44:45.734616+0000","last_unstale":"2024-09-16T15:44:45.737862+0000","last_undegraded":"2024-09-16T15:44:45.737862+0000","last_fullsized":"2024-09-16T15:44:45.737862+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:19:00.003404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.697639+0000","last_change":"2024-09-16T15:44:44.701584+0000","last_active":"2024-09-16T15:44:45.697639+0000","last_peered":"2024-09-16T15:44:45.697639+0000","last_clean":"2024-09-16T15:44:45.697639+0000","last_became_active":"2024-09-16T15:44:44.701327+0000","last_became_peered":"2024-09-16T15:44:44.701327+0000","last_unstale":"2024-09-16T15:44:45.697639+0000","last_undegraded":"2024-09-16T15:44:45.697639+0000","last_fullsized":"2024-09-16T15:44:45.697639+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:44:05.886418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.720386+0000","last_change":"2024-09-16T15:44:45.717514+0000","last_active":"2024-09-16T15:44:45.720386+0000","last_peered":"2024-09-16T15:44:45.720386+0000","last_clean":"2024-09-16T15:44:45.720386+0000","last_became_active":"2024-09-16T15:44:45.717270+0000","last_became_peered":"2024-09-16T15:44:45.717270+0000","last_unstale":"2024-09-16T15:44:45.720386+0000","last_undegraded":"2024-09-16T15:44:45.720386+0000","last_fullsized":"2024-09-16T15:44:45.720386+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:12:47.737828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.698166+0000","last_change":"2024-09-16T15:44:44.702562+0000","last_active":"2024-09-16T15:44:45.698166+0000","last_peered":"2024-09-16T15:44:45.698166+0000","last_clean":"2024-09-16T15:44:45.698166+0000","last_became_active":"2024-09-16T15:44:44.702282+0000","last_became_peered":"2024-09-16T15:44:44.702282+0000","last_unstale":"2024-09-16T15:44:45.698166+0000","last_undegraded":"2024-09-16T15:44:45.698166+0000","last_fullsized":"2024-09-16T15:44:45.698166+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:34:40.861349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.723514+0000","last_change":"2024-09-16T15:44:45.717730+0000","last_active":"2024-09-16T15:44:45.723514+0000","last_peered":"2024-09-16T15:44:45.723514+0000","last_clean":"2024-09-16T15:44:45.723514+0000","last_became_active":"2024-09-16T15:44:45.717456+0000","last_became_peered":"2024-09-16T15:44:45.717456+0000","last_unstale":"2024-09-16T15:44:45.723514+0000","last_undegraded":"2024-09-16T15:44:45.723514+0000","last_fullsized":"2024-09-16T15:44:45.723514+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:09:14.114379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"61'8","reported_seq":21,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.740384+0000","last_change":"2024-09-16T15:44:44.700378+0000","last_active":"2024-09-16T15:44:45.740384+0000","last_peered":"2024-09-16T15:44:45.740384+0000","last_clean":"2024-09-16T15:44:45.740384+0000","last_became_active":"2024-09-16T15:44:44.700114+0000","last_became_peered":"2024-09-16T15:44:44.700114+0000","last_unstale":"2024-09-16T15:44:45.740384+0000","last_undegraded":"2024-09-16T15:44:45.740384+0000","last_fullsized":"2024-09-16T15:44:45.740384+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:46:16.051799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.720485+0000","last_change":"2024-09-16T15:44:45.716464+0000","last_active":"2024-09-16T15:44:45.720485+0000","last_peered":"2024-09-16T15:44:45.720485+0000","last_clean":"2024-09-16T15:44:45.720485+0000","last_became_active":"2024-09-16T15:44:45.716140+0000","last_became_peered":"2024-09-16T15:44:45.716140+0000","last_unstale":"2024-09-16T15:44:45.720485+0000","last_undegraded":"2024-09-16T15:44:45.720485+0000","last_fullsized":"2024-09-16T15:44:45.720485+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:31:46.215684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T15:44:44.711883+0000","last_change":"2024-09-16T15:44:44.711284+0000","last_active":"2024-09-16T15:44:44.711883+0000","last_peered":"2024-09-16T15:44:44.711883+0000","last_clean":"2024-09-16T15:44:44.711883+0000","last_became_active":"2024-09-16T15:44:44.709695+0000","last_became_peered":"2024-09-16T15:44:44.709695+0000","last_unstale":"2024-09-16T15:44:44.711883+0000","last_undegraded":"2024-09-16T15:44:44.711883+0000","last_fullsized":"2024-09-16T15:44:44.711883+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:29:18.863386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.718328+0000","last_change":"2024-09-16T15:44:45.714790+0000","last_active":"2024-09-16T15:44:45.718328+0000","last_peered":"2024-09-16T15:44:45.718328+0000","last_clean":"2024-09-16T15:44:45.718328+0000","last_became_active":"2024-09-16T15:44:45.713900+0000","last_became_peered":"2024-09-16T15:44:45.713900+0000","last_unstale":"2024-09-16T15:44:45.718328+0000","last_undegraded":"2024-09-16T15:44:45.718328+0000","last_fullsized":"2024-09-16T15:44:45.718328+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:50:53.880904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"61'2","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.730326+0000","last_change":"2024-09-16T15:44:44.698462+0000","last_active":"2024-09-16T15:44:45.730326+0000","last_peered":"2024-09-16T15:44:45.730326+0000","last_clean":"2024-09-16T15:44:45.730326+0000","last_became_active":"2024-09-16T15:44:44.698179+0000","last_became_peered":"2024-09-16T15:44:44.698179+0000","last_unstale":"2024-09-16T15:44:45.730326+0000","last_undegraded":"2024-09-16T15:44:45.730326+0000","last_fullsized":"2024-09-16T15:44:45.730326+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:13:21.284936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T15:44:44.693602+0000","last_change":"2024-09-16T15:44:44.693602+0000","last_active":"2024-09-16T15:44:44.679289+0000","last_peered":"2024-09-16T15:44:44.679289+0000","last_clean":"2024-09-16T15:44:44.679289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:44:44.693602+0000","last_undegraded":"2024-09-16T15:44:44.693602+0000","last_fullsized":"2024-09-16T15:44:44.693602+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"61'1","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.741605+0000","last_change":"2024-09-16T15:44:44.706373+0000","last_active":"2024-09-16T15:44:45.741605+0000","last_peered":"2024-09-16T15:44:45.741605+0000","last_clean":"2024-09-16T15:44:45.741605+0000","last_became_active":"2024-09-16T15:44:44.705120+0000","last_became_peered":"2024-09-16T15:44:44.705120+0000","last_unstale":"2024-09-16T15:44:45.741605+0000","last_undegraded":"2024-09-16T15:44:45.741605+0000","last_fullsized":"2024-09-16T15:44:45.741605+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:07:54.665094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.735750+0000","last_change":"2024-09-16T15:44:45.726953+0000","last_active":"2024-09-16T15:44:45.735750+0000","last_peered":"2024-09-16T15:44:45.735750+0000","last_clean":"2024-09-16T15:44:45.735750+0000","last_became_active":"2024-09-16T15:44:45.723508+0000","last_became_peered":"2024-09-16T15:44:45.723508+0000","last_unstale":"2024-09-16T15:44:45.735750+0000","last_undegraded":"2024-09-16T15:44:45.735750+0000","last_fullsized":"2024-09-16T15:44:45.735750+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:39:26.814713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.705171+0000","last_change":"2024-09-16T15:44:44.708721+0000","last_active":"2024-09-16T15:44:45.705171+0000","last_peered":"2024-09-16T15:44:45.705171+0000","last_clean":"2024-09-16T15:44:45.705171+0000","last_became_active":"2024-09-16T15:44:44.708471+0000","last_became_peered":"2024-09-16T15:44:44.708471+0000","last_unstale":"2024-09-16T15:44:45.705171+0000","last_undegraded":"2024-09-16T15:44:45.705171+0000","last_fullsized":"2024-09-16T15:44:45.705171+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:39:49.307383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.720161+0000","last_change":"2024-09-16T15:44:45.716983+0000","last_active":"2024-09-16T15:44:45.720161+0000","last_peered":"2024-09-16T15:44:45.720161+0000","last_clean":"2024-09-16T15:44:45.720161+0000","last_became_active":"2024-09-16T15:44:45.715530+0000","last_became_peered":"2024-09-16T15:44:45.715530+0000","last_unstale":"2024-09-16T15:44:45.720161+0000","last_undegraded":"2024-09-16T15:44:45.720161+0000","last_fullsized":"2024-09-16T15:44:45.720161+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:03:43.262795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"61'2","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.733237+0000","last_change":"2024-09-16T15:44:44.696118+0000","last_active":"2024-09-16T15:44:45.733237+0000","last_peered":"2024-09-16T15:44:45.733237+0000","last_clean":"2024-09-16T15:44:45.733237+0000","last_became_active":"2024-09-16T15:44:44.695842+0000","last_became_peered":"2024-09-16T15:44:44.695842+0000","last_unstale":"2024-09-16T15:44:45.733237+0000","last_undegraded":"2024-09-16T15:44:45.733237+0000","last_fullsized":"2024-09-16T15:44:45.733237+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:31:45.212994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.724057+0000","last_change":"2024-09-16T15:44:45.712249+0000","last_active":"2024-09-16T15:44:45.724057+0000","last_peered":"2024-09-16T15:44:45.724057+0000","last_clean":"2024-09-16T15:44:45.724057+0000","last_became_active":"2024-09-16T15:44:45.711524+0000","last_became_peered":"2024-09-16T15:44:45.711524+0000","last_unstale":"2024-09-16T15:44:45.724057+0000","last_undegraded":"2024-09-16T15:44:45.724057+0000","last_fullsized":"2024-09-16T15:44:45.724057+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:45:42.401451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.703897+0000","last_change":"2024-09-16T15:44:44.695883+0000","last_active":"2024-09-16T15:44:45.703897+0000","last_peered":"2024-09-16T15:44:45.703897+0000","last_clean":"2024-09-16T15:44:45.703897+0000","last_became_active":"2024-09-16T15:44:44.695562+0000","last_became_peered":"2024-09-16T15:44:44.695562+0000","last_unstale":"2024-09-16T15:44:45.703897+0000","last_undegraded":"2024-09-16T15:44:45.703897+0000","last_fullsized":"2024-09-16T15:44:45.703897+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:22:24.705292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.722865+0000","last_change":"2024-09-16T15:44:45.708395+0000","last_active":"2024-09-16T15:44:45.722865+0000","last_peered":"2024-09-16T15:44:45.722865+0000","last_clean":"2024-09-16T15:44:45.722865+0000","last_became_active":"2024-09-16T15:44:45.708090+0000","last_became_peered":"2024-09-16T15:44:45.708090+0000","last_unstale":"2024-09-16T15:44:45.722865+0000","last_undegraded":"2024-09-16T15:44:45.722865+0000","last_fullsized":"2024-09-16T15:44:45.722865+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:13:37.240255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.712718+0000","last_change":"2024-09-16T15:44:44.705798+0000","last_active":"2024-09-16T15:44:45.712718+0000","last_peered":"2024-09-16T15:44:45.712718+0000","last_clean":"2024-09-16T15:44:45.712718+0000","last_became_active":"2024-09-16T15:44:44.705468+0000","last_became_peered":"2024-09-16T15:44:44.705468+0000","last_unstale":"2024-09-16T15:44:45.712718+0000","last_undegraded":"2024-09-16T15:44:45.712718+0000","last_fullsized":"2024-09-16T15:44:45.712718+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:21:59.839074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.736422+0000","last_change":"2024-09-16T15:44:45.726903+0000","last_active":"2024-09-16T15:44:45.736422+0000","last_peered":"2024-09-16T15:44:45.736422+0000","last_clean":"2024-09-16T15:44:45.736422+0000","last_became_active":"2024-09-16T15:44:45.726666+0000","last_became_peered":"2024-09-16T15:44:45.726666+0000","last_unstale":"2024-09-16T15:44:45.736422+0000","last_undegraded":"2024-09-16T15:44:45.736422+0000","last_fullsized":"2024-09-16T15:44:45.736422+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:51:57.452114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T15:44:44.710737+0000","last_change":"2024-09-16T15:44:44.708470+0000","last_active":"2024-09-16T15:44:44.710737+0000","last_peered":"2024-09-16T15:44:44.710737+0000","last_clean":"2024-09-16T15:44:44.710737+0000","last_became_active":"2024-09-16T15:44:44.708182+0000","last_became_peered":"2024-09-16T15:44:44.708182+0000","last_unstale":"2024-09-16T15:44:44.710737+0000","last_undegraded":"2024-09-16T15:44:44.710737+0000","last_fullsized":"2024-09-16T15:44:44.710737+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:37:09.093235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.718282+0000","last_change":"2024-09-16T15:44:45.707416+0000","last_active":"2024-09-16T15:44:45.718282+0000","last_peered":"2024-09-16T15:44:45.718282+0000","last_clean":"2024-09-16T15:44:45.718282+0000","last_became_active":"2024-09-16T15:44:45.707148+0000","last_became_peered":"2024-09-16T15:44:45.707148+0000","last_unstale":"2024-09-16T15:44:45.718282+0000","last_undegraded":"2024-09-16T15:44:45.718282+0000","last_fullsized":"2024-09-16T15:44:45.718282+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:34:31.566558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.713344+0000","last_change":"2024-09-16T15:44:44.710583+0000","last_active":"2024-09-16T15:44:45.713344+0000","last_peered":"2024-09-16T15:44:45.713344+0000","last_clean":"2024-09-16T15:44:45.713344+0000","last_became_active":"2024-09-16T15:44:44.710293+0000","last_became_peered":"2024-09-16T15:44:44.710293+0000","last_unstale":"2024-09-16T15:44:45.713344+0000","last_undegraded":"2024-09-16T15:44:45.713344+0000","last_fullsized":"2024-09-16T15:44:45.713344+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:03:46.514703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T15:44:44.692278+0000","last_change":"2024-09-16T15:44:44.692278+0000","last_active":"2024-09-16T15:44:44.679289+0000","last_peered":"2024-09-16T15:44:44.679289+0000","last_clean":"2024-09-16T15:44:44.679289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:44:44.692278+0000","last_undegraded":"2024-09-16T15:44:44.692278+0000","last_fullsized":"2024-09-16T15:44:44.692278+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.704857+0000","last_change":"2024-09-16T15:44:44.703904+0000","last_active":"2024-09-16T15:44:45.704857+0000","last_peered":"2024-09-16T15:44:45.704857+0000","last_clean":"2024-09-16T15:44:45.704857+0000","last_became_active":"2024-09-16T15:44:44.703644+0000","last_became_peered":"2024-09-16T15:44:44.703644+0000","last_unstale":"2024-09-16T15:44:45.704857+0000","last_undegraded":"2024-09-16T15:44:45.704857+0000","last_fullsized":"2024-09-16T15:44:45.704857+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:31:17.501469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.724319+0000","last_change":"2024-09-16T15:44:45.714221+0000","last_active":"2024-09-16T15:44:45.724319+0000","last_peered":"2024-09-16T15:44:45.724319+0000","last_clean":"2024-09-16T15:44:45.724319+0000","last_became_active":"2024-09-16T15:44:45.713871+0000","last_became_peered":"2024-09-16T15:44:45.713871+0000","last_unstale":"2024-09-16T15:44:45.724319+0000","last_undegraded":"2024-09-16T15:44:45.724319+0000","last_fullsized":"2024-09-16T15:44:45.724319+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:09:31.284857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.717881+0000","last_change":"2024-09-16T15:44:44.698858+0000","last_active":"2024-09-16T15:44:45.717881+0000","last_peered":"2024-09-16T15:44:45.717881+0000","last_clean":"2024-09-16T15:44:45.717881+0000","last_became_active":"2024-09-16T15:44:44.698546+0000","last_became_peered":"2024-09-16T15:44:44.698546+0000","last_unstale":"2024-09-16T15:44:45.717881+0000","last_undegraded":"2024-09-16T15:44:45.717881+0000","last_fullsized":"2024-09-16T15:44:45.717881+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:03:16.348362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.730750+0000","last_change":"2024-09-16T15:44:45.719192+0000","last_active":"2024-09-16T15:44:45.730750+0000","last_peered":"2024-09-16T15:44:45.730750+0000","last_clean":"2024-09-16T15:44:45.730750+0000","last_became_active":"2024-09-16T15:44:45.716454+0000","last_became_peered":"2024-09-16T15:44:45.716454+0000","last_unstale":"2024-09-16T15:44:45.730750+0000","last_undegraded":"2024-09-16T15:44:45.730750+0000","last_fullsized":"2024-09-16T15:44:45.730750+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:05:16.681904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T15:44:44.710050+0000","last_change":"2024-09-16T15:44:44.707043+0000","last_active":"2024-09-16T15:44:44.710050+0000","last_peered":"2024-09-16T15:44:44.710050+0000","last_clean":"2024-09-16T15:44:44.710050+0000","last_became_active":"2024-09-16T15:44:44.706747+0000","last_became_peered":"2024-09-16T15:44:44.706747+0000","last_unstale":"2024-09-16T15:44:44.710050+0000","last_undegraded":"2024-09-16T15:44:44.710050+0000","last_fullsized":"2024-09-16T15:44:44.710050+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:15:01.043845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.730635+0000","last_change":"2024-09-16T15:44:45.726366+0000","last_active":"2024-09-16T15:44:45.730635+0000","last_peered":"2024-09-16T15:44:45.730635+0000","last_clean":"2024-09-16T15:44:45.730635+0000","last_became_active":"2024-09-16T15:44:45.726122+0000","last_became_peered":"2024-09-16T15:44:45.726122+0000","last_unstale":"2024-09-16T15:44:45.730635+0000","last_undegraded":"2024-09-16T15:44:45.730635+0000","last_fullsized":"2024-09-16T15:44:45.730635+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:13:21.061051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.705257+0000","last_change":"2024-09-16T15:44:44.704189+0000","last_active":"2024-09-16T15:44:45.705257+0000","last_peered":"2024-09-16T15:44:45.705257+0000","last_clean":"2024-09-16T15:44:45.705257+0000","last_became_active":"2024-09-16T15:44:44.703916+0000","last_became_peered":"2024-09-16T15:44:44.703916+0000","last_unstale":"2024-09-16T15:44:45.705257+0000","last_undegraded":"2024-09-16T15:44:45.705257+0000","last_fullsized":"2024-09-16T15:44:45.705257+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:50:09.467231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.724010+0000","last_change":"2024-09-16T15:44:45.710420+0000","last_active":"2024-09-16T15:44:45.724010+0000","last_peered":"2024-09-16T15:44:45.724010+0000","last_clean":"2024-09-16T15:44:45.724010+0000","last_became_active":"2024-09-16T15:44:45.710144+0000","last_became_peered":"2024-09-16T15:44:45.710144+0000","last_unstale":"2024-09-16T15:44:45.724010+0000","last_undegraded":"2024-09-16T15:44:45.724010+0000","last_fullsized":"2024-09-16T15:44:45.724010+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:23:06.419124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.707293+0000","last_change":"2024-09-16T15:44:44.708397+0000","last_active":"2024-09-16T15:44:45.707293+0000","last_peered":"2024-09-16T15:44:45.707293+0000","last_clean":"2024-09-16T15:44:45.707293+0000","last_became_active":"2024-09-16T15:44:44.708101+0000","last_became_peered":"2024-09-16T15:44:44.708101+0000","last_unstale":"2024-09-16T15:44:45.707293+0000","last_undegraded":"2024-09-16T15:44:45.707293+0000","last_fullsized":"2024-09-16T15:44:45.707293+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:09:26.705708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T15:44:44.691091+0000","last_change":"2024-09-16T15:44:44.691091+0000","last_active":"2024-09-16T15:44:44.679289+0000","last_peered":"2024-09-16T15:44:44.679289+0000","last_clean":"2024-09-16T15:44:44.679289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:44:44.691091+0000","last_undegraded":"2024-09-16T15:44:44.691091+0000","last_fullsized":"2024-09-16T15:44:44.691091+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.703426+0000","last_change":"2024-09-16T15:44:44.698425+0000","last_active":"2024-09-16T15:44:45.703426+0000","last_peered":"2024-09-16T15:44:45.703426+0000","last_clean":"2024-09-16T15:44:45.703426+0000","last_became_active":"2024-09-16T15:44:44.698141+0000","last_became_peered":"2024-09-16T15:44:44.698141+0000","last_unstale":"2024-09-16T15:44:45.703426+0000","last_undegraded":"2024-09-16T15:44:45.703426+0000","last_fullsized":"2024-09-16T15:44:45.703426+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:55:27.169366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.724043+0000","last_change":"2024-09-16T15:44:45.720393+0000","last_active":"2024-09-16T15:44:45.724043+0000","last_peered":"2024-09-16T15:44:45.724043+0000","last_clean":"2024-09-16T15:44:45.724043+0000","last_became_active":"2024-09-16T15:44:45.720028+0000","last_became_peered":"2024-09-16T15:44:45.720028+0000","last_unstale":"2024-09-16T15:44:45.724043+0000","last_undegraded":"2024-09-16T15:44:45.724043+0000","last_fullsized":"2024-09-16T15:44:45.724043+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:46:17.054489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.703722+0000","last_change":"2024-09-16T15:44:44.690580+0000","last_active":"2024-09-16T15:44:45.703722+0000","last_peered":"2024-09-16T15:44:45.703722+0000","last_clean":"2024-09-16T15:44:45.703722+0000","last_became_active":"2024-09-16T15:44:44.690302+0000","last_became_peered":"2024-09-16T15:44:44.690302+0000","last_unstale":"2024-09-16T15:44:45.703722+0000","last_undegraded":"2024-09-16T15:44:45.703722+0000","last_fullsized":"2024-09-16T15:44:45.703722+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:45:53.368994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":185,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.709646+0000","last_change":"2024-09-16T15:39:12.698813+0000","last_active":"2024-09-16T15:44:45.709646+0000","last_peered":"2024-09-16T15:44:45.709646+0000","last_clean":"2024-09-16T15:44:45.709646+0000","last_became_active":"2024-09-16T15:39:12.697695+0000","last_became_peered":"2024-09-16T15:39:12.697695+0000","last_unstale":"2024-09-16T15:44:45.709646+0000","last_undegraded":"2024-09-16T15:44:45.709646+0000","last_fullsized":"2024-09-16T15:44:45.709646+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_clean_scrub_stamp":"2024-09-16T15:39:11.684967+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:02.061081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2388,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":34,"ondisk_log_size":34,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509512,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40208,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331632,"statfs":{"total":96636764160,"available":96595591168,"internally_reserved":0,"allocated":712704,"data_stored":296806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":40434120},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509511,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46356,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325484,"statfs":{"total":96636764160,"available":96589295616,"internally_reserved":0,"allocated":1306624,"data_stored":887174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509510,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45772,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326068,"statfs":{"total":96636764160,"available":96589893632,"internally_reserved":0,"allocated":712704,"data_stored":296806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509510,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40204,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331636,"statfs":{"total":96636764160,"available":96595595264,"internally_reserved":0,"allocated":712704,"data_stored":296806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":40434121},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509510,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62456,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94309384,"statfs":{"total":96636764160,"available":96572809216,"internally_reserved":0,"allocated":745472,"data_stored":305381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":63175111},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509511,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40788,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331052,"statfs":{"total":96636764160,"available":96594997248,"internally_reserved":0,"allocated":1306624,"data_stored":887174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509509,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40208,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331632,"statfs":{"total":96636764160,"available":96595591168,"internally_reserved":0,"allocated":712704,"data_stored":296806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542216,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40208,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331632,"statfs":{"total":96636764160,"available":96595591168,"internally_reserved":0,"allocated":712704,"data_stored":296806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":40434124},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:44:50.375 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:44:50.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.570+0000 7f1dbf200640 1 -- 172.21.15.151:0/4155822096 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1da0004800 msgr2=0x7f1da0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:50.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.570+0000 7f1dbf200640 1 --2- 172.21.15.151:0/4155822096 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1da0004800 0x7f1da0004c80 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f1da800c4a0 tx=0x7f1da0005830 comp rx=0 tx=0).stop 2024-09-16T15:44:50.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.570+0000 7f1dbf200640 1 -- 172.21.15.151:0/4155822096 shutdown_connections 2024-09-16T15:44:50.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.570+0000 7f1dbf200640 1 --2- 172.21.15.151:0/4155822096 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1da0004800 0x7f1da0004c80 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:50.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.570+0000 7f1dbf200640 1 -- 172.21.15.151:0/4155822096 >> 172.21.15.151:0/4155822096 conn(0x7f1db80d3df0 msgr2=0x7f1db80d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:50.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.570+0000 7f1dbf200640 1 -- 172.21.15.151:0/4155822096 shutdown_connections 2024-09-16T15:44:50.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.570+0000 7f1dbf200640 1 -- 172.21.15.151:0/4155822096 wait complete. 2024-09-16T15:44:50.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.571+0000 7f1dbf200640 1 Processor -- start 2024-09-16T15:44:50.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.572+0000 7f1dbf200640 1 -- start start 2024-09-16T15:44:50.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.572+0000 7f1dbf200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1da0004800 0x7f1db81f8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:50.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.572+0000 7f1dbf200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1db80c98f0 0x7f1db80ccb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:50.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.573+0000 7f1dbf200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1db81f5b50 0x7f1db80cfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:50.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.573+0000 7f1dbf200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1db80d1ff0 con 0x7f1db80c98f0 2024-09-16T15:44:50.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.573+0000 7f1dbf200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1db80d1e70 con 0x7f1da0004800 2024-09-16T15:44:50.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.573+0000 7f1dbf200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f1db80d2170 con 0x7f1db81f5b50 2024-09-16T15:44:50.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.573+0000 7f1dbd400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1db80c98f0 0x7f1db80ccb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:50.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.573+0000 7f1dbd400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1db80c98f0 0x7f1db80ccb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44300/0 (socket says 172.21.15.151:44300) 2024-09-16T15:44:50.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.573+0000 7f1dbd400640 1 -- 172.21.15.151:0/3792573038 learned_addr learned my addr 172.21.15.151:0/3792573038 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:50.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.573+0000 7f1dbe800640 1 --2- 172.21.15.151:0/3792573038 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1db81f5b50 0x7f1db80cfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:50.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.573+0000 7f1dbde00640 1 --2- 172.21.15.151:0/3792573038 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1da0004800 0x7f1db81f8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:50.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.573+0000 7f1dbd400640 1 -- 172.21.15.151:0/3792573038 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1db81f5b50 msgr2=0x7f1db80cfcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:50.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.573+0000 7f1dbd400640 1 --2- 172.21.15.151:0/3792573038 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1db81f5b50 0x7f1db80cfcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:50.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.573+0000 7f1dbd400640 1 -- 172.21.15.151:0/3792573038 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1da0004800 msgr2=0x7f1db81f8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:50.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.573+0000 7f1dbd400640 1 --2- 172.21.15.151:0/3792573038 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1da0004800 0x7f1db81f8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:50.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.573+0000 7f1dbd400640 1 -- 172.21.15.151:0/3792573038 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1db81f86c0 con 0x7f1db80c98f0 2024-09-16T15:44:50.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.574+0000 7f1dbd400640 1 --2- 172.21.15.151:0/3792573038 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1db80c98f0 0x7f1db80ccb90 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f1da802b350 tx=0x7f1da80013c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:50.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.574+0000 7f1db2000640 1 -- 172.21.15.151:0/3792573038 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da8042070 con 0x7f1db80c98f0 2024-09-16T15:44:50.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.574+0000 7f1db2000640 1 -- 172.21.15.151:0/3792573038 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1da80066e0 con 0x7f1db80c98f0 2024-09-16T15:44:50.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.574+0000 7f1db2000640 1 -- 172.21.15.151:0/3792573038 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da8039070 con 0x7f1db80c98f0 2024-09-16T15:44:50.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.574+0000 7f1dbf200640 1 -- 172.21.15.151:0/3792573038 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1db81f89a0 con 0x7f1db80c98f0 2024-09-16T15:44:50.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.574+0000 7f1dbf200640 1 -- 172.21.15.151:0/3792573038 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1db80cb060 con 0x7f1db80c98f0 2024-09-16T15:44:50.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.575+0000 7f1dbf200640 1 -- 172.21.15.151:0/3792573038 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1db81f8ec0 con 0x7f1db80c98f0 2024-09-16T15:44:50.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.576+0000 7f1db2000640 1 -- 172.21.15.151:0/3792573038 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 102103+0+0 (secure 0 0 0) 0x7f1da8045070 con 0x7f1db80c98f0 2024-09-16T15:44:50.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.577+0000 7f1db2000640 1 --2- 172.21.15.151:0/3792573038 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1d9c079180 0x7f1d9c07b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:50.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.577+0000 7f1db2000640 1 -- 172.21.15.151:0/3792573038 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7f1da80b8a00 con 0x7f1db80c98f0 2024-09-16T15:44:50.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.577+0000 7f1dbde00640 1 --2- 172.21.15.151:0/3792573038 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1d9c079180 0x7f1d9c07b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:50.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.578+0000 7f1dbde00640 1 --2- 172.21.15.151:0/3792573038 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1d9c079180 0x7f1d9c07b640 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1db8052150 tx=0x7f1dac005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:50.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.581+0000 7f1db2000640 1 -- 172.21.15.151:0/3792573038 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1db81f8ec0 con 0x7f1db80c98f0 2024-09-16T15:44:50.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.763+0000 7f1dbf200640 1 -- 172.21.15.151:0/3792573038 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1db80caea0 con 0x7f1d9c079180 2024-09-16T15:44:50.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.768+0000 7f1db2000640 1 -- 172.21.15.151:0/3792573038 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251283 (secure 0 0 0) 0x7f1dac05e5b0 con 0x7f1d9c079180 2024-09-16T15:44:50.772 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:44:50.773 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-09-16T15:44:50.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.772+0000 7f1dbf200640 1 -- 172.21.15.151:0/3792573038 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1d9c079180 msgr2=0x7f1d9c07b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:50.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.772+0000 7f1dbf200640 1 --2- 172.21.15.151:0/3792573038 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1d9c079180 0x7f1d9c07b640 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1db8052150 tx=0x7f1dac005f90 comp rx=0 tx=0).stop 2024-09-16T15:44:50.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.772+0000 7f1dbf200640 1 -- 172.21.15.151:0/3792573038 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1db80c98f0 msgr2=0x7f1db80ccb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:50.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.772+0000 7f1dbf200640 1 --2- 172.21.15.151:0/3792573038 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1db80c98f0 0x7f1db80ccb90 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f1da802b350 tx=0x7f1da80013c0 comp rx=0 tx=0).stop 2024-09-16T15:44:50.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.772+0000 7f1dbf200640 1 -- 172.21.15.151:0/3792573038 shutdown_connections 2024-09-16T15:44:50.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.772+0000 7f1dbf200640 1 --2- 172.21.15.151:0/3792573038 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1d9c079180 0x7f1d9c07b640 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:50.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.772+0000 7f1dbf200640 1 --2- 172.21.15.151:0/3792573038 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1db81f5b50 0x7f1db80cfcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:50.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.772+0000 7f1dbf200640 1 --2- 172.21.15.151:0/3792573038 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1db80c98f0 0x7f1db80ccb90 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:50.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.772+0000 7f1dbf200640 1 --2- 172.21.15.151:0/3792573038 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1da0004800 0x7f1db81f8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:50.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.772+0000 7f1dbf200640 1 -- 172.21.15.151:0/3792573038 >> 172.21.15.151:0/3792573038 conn(0x7f1db80d3df0 msgr2=0x7f1db80d0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:50.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.772+0000 7f1dbf200640 1 -- 172.21.15.151:0/3792573038 shutdown_connections 2024-09-16T15:44:50.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.772+0000 7f1dbf200640 1 -- 172.21.15.151:0/3792573038 wait complete. 2024-09-16T15:44:50.789 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":225,"stamp":"2024-09-16T15:44:49.670408+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592756,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":141,"num_write_kb":1641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":106,"ondisk_log_size":106,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":32,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":356200,"kb_used_data":6760,"kb_used_omap":12,"kb_used_meta":349235,"kb_avail":754618520,"statfs":{"total":773094113280,"available":772729364480,"internally_reserved":0,"allocated":6922240,"data_stored":3563759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12738,"internal_metadata":357617214},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2388,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.988477"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.732875+0000","last_change":"2024-09-16T15:44:45.728739+0000","last_active":"2024-09-16T15:44:45.732875+0000","last_peered":"2024-09-16T15:44:45.732875+0000","last_clean":"2024-09-16T15:44:45.732875+0000","last_became_active":"2024-09-16T15:44:45.727368+0000","last_became_peered":"2024-09-16T15:44:45.727368+0000","last_unstale":"2024-09-16T15:44:45.732875+0000","last_undegraded":"2024-09-16T15:44:45.732875+0000","last_fullsized":"2024-09-16T15:44:45.732875+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:34:55.972702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T15:44:44.692975+0000","last_change":"2024-09-16T15:44:44.692975+0000","last_active":"2024-09-16T15:44:44.679289+0000","last_peered":"2024-09-16T15:44:44.679289+0000","last_clean":"2024-09-16T15:44:44.679289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:44:44.692975+0000","last_undegraded":"2024-09-16T15:44:44.692975+0000","last_fullsized":"2024-09-16T15:44:44.692975+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.732389+0000","last_change":"2024-09-16T15:44:45.725780+0000","last_active":"2024-09-16T15:44:45.732389+0000","last_peered":"2024-09-16T15:44:45.732389+0000","last_clean":"2024-09-16T15:44:45.732389+0000","last_became_active":"2024-09-16T15:44:45.721333+0000","last_became_peered":"2024-09-16T15:44:45.721333+0000","last_unstale":"2024-09-16T15:44:45.732389+0000","last_undegraded":"2024-09-16T15:44:45.732389+0000","last_fullsized":"2024-09-16T15:44:45.732389+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:13:04.403633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.737448+0000","last_change":"2024-09-16T15:44:45.729388+0000","last_active":"2024-09-16T15:44:45.737448+0000","last_peered":"2024-09-16T15:44:45.737448+0000","last_clean":"2024-09-16T15:44:45.737448+0000","last_became_active":"2024-09-16T15:44:45.729119+0000","last_became_peered":"2024-09-16T15:44:45.729119+0000","last_unstale":"2024-09-16T15:44:45.737448+0000","last_undegraded":"2024-09-16T15:44:45.737448+0000","last_fullsized":"2024-09-16T15:44:45.737448+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:08:12.766873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.738580+0000","last_change":"2024-09-16T15:44:45.733586+0000","last_active":"2024-09-16T15:44:45.738580+0000","last_peered":"2024-09-16T15:44:45.738580+0000","last_clean":"2024-09-16T15:44:45.738580+0000","last_became_active":"2024-09-16T15:44:45.733144+0000","last_became_peered":"2024-09-16T15:44:45.733144+0000","last_unstale":"2024-09-16T15:44:45.738580+0000","last_undegraded":"2024-09-16T15:44:45.738580+0000","last_fullsized":"2024-09-16T15:44:45.738580+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:04:50.517949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.730726+0000","last_change":"2024-09-16T15:44:45.718049+0000","last_active":"2024-09-16T15:44:45.730726+0000","last_peered":"2024-09-16T15:44:45.730726+0000","last_clean":"2024-09-16T15:44:45.730726+0000","last_became_active":"2024-09-16T15:44:45.716815+0000","last_became_peered":"2024-09-16T15:44:45.716815+0000","last_unstale":"2024-09-16T15:44:45.730726+0000","last_undegraded":"2024-09-16T15:44:45.730726+0000","last_fullsized":"2024-09-16T15:44:45.730726+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:30:51.703628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.724140+0000","last_change":"2024-09-16T15:44:45.718523+0000","last_active":"2024-09-16T15:44:45.724140+0000","last_peered":"2024-09-16T15:44:45.724140+0000","last_clean":"2024-09-16T15:44:45.724140+0000","last_became_active":"2024-09-16T15:44:45.718208+0000","last_became_peered":"2024-09-16T15:44:45.718208+0000","last_unstale":"2024-09-16T15:44:45.724140+0000","last_undegraded":"2024-09-16T15:44:45.724140+0000","last_fullsized":"2024-09-16T15:44:45.724140+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:58:34.310843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.724066+0000","last_change":"2024-09-16T15:44:45.720647+0000","last_active":"2024-09-16T15:44:45.724066+0000","last_peered":"2024-09-16T15:44:45.724066+0000","last_clean":"2024-09-16T15:44:45.724066+0000","last_became_active":"2024-09-16T15:44:45.720351+0000","last_became_peered":"2024-09-16T15:44:45.720351+0000","last_unstale":"2024-09-16T15:44:45.724066+0000","last_undegraded":"2024-09-16T15:44:45.724066+0000","last_fullsized":"2024-09-16T15:44:45.724066+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:44:06.889108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.734233+0000","last_change":"2024-09-16T15:44:45.732987+0000","last_active":"2024-09-16T15:44:45.734233+0000","last_peered":"2024-09-16T15:44:45.734233+0000","last_clean":"2024-09-16T15:44:45.734233+0000","last_became_active":"2024-09-16T15:44:45.732680+0000","last_became_peered":"2024-09-16T15:44:45.732680+0000","last_unstale":"2024-09-16T15:44:45.734233+0000","last_undegraded":"2024-09-16T15:44:45.734233+0000","last_fullsized":"2024-09-16T15:44:45.734233+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:21:56.720871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.730690+0000","last_change":"2024-09-16T15:44:45.719120+0000","last_active":"2024-09-16T15:44:45.730690+0000","last_peered":"2024-09-16T15:44:45.730690+0000","last_clean":"2024-09-16T15:44:45.730690+0000","last_became_active":"2024-09-16T15:44:45.718746+0000","last_became_peered":"2024-09-16T15:44:45.718746+0000","last_unstale":"2024-09-16T15:44:45.730690+0000","last_undegraded":"2024-09-16T15:44:45.730690+0000","last_fullsized":"2024-09-16T15:44:45.730690+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:13:46.973489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.726293+0000","last_change":"2024-09-16T15:44:45.718730+0000","last_active":"2024-09-16T15:44:45.726293+0000","last_peered":"2024-09-16T15:44:45.726293+0000","last_clean":"2024-09-16T15:44:45.726293+0000","last_became_active":"2024-09-16T15:44:45.718442+0000","last_became_peered":"2024-09-16T15:44:45.718442+0000","last_unstale":"2024-09-16T15:44:45.726293+0000","last_undegraded":"2024-09-16T15:44:45.726293+0000","last_fullsized":"2024-09-16T15:44:45.726293+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:13:04.403633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.720994+0000","last_change":"2024-09-16T15:44:45.717536+0000","last_active":"2024-09-16T15:44:45.720994+0000","last_peered":"2024-09-16T15:44:45.720994+0000","last_clean":"2024-09-16T15:44:45.720994+0000","last_became_active":"2024-09-16T15:44:45.717226+0000","last_became_peered":"2024-09-16T15:44:45.717226+0000","last_unstale":"2024-09-16T15:44:45.720994+0000","last_undegraded":"2024-09-16T15:44:45.720994+0000","last_fullsized":"2024-09-16T15:44:45.720994+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:46:33.994315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.719703+0000","last_change":"2024-09-16T15:44:45.711525+0000","last_active":"2024-09-16T15:44:45.719703+0000","last_peered":"2024-09-16T15:44:45.719703+0000","last_clean":"2024-09-16T15:44:45.719703+0000","last_became_active":"2024-09-16T15:44:45.711195+0000","last_became_peered":"2024-09-16T15:44:45.711195+0000","last_unstale":"2024-09-16T15:44:45.719703+0000","last_undegraded":"2024-09-16T15:44:45.719703+0000","last_fullsized":"2024-09-16T15:44:45.719703+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:42:47.930299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.732183+0000","last_change":"2024-09-16T15:44:45.726781+0000","last_active":"2024-09-16T15:44:45.732183+0000","last_peered":"2024-09-16T15:44:45.732183+0000","last_clean":"2024-09-16T15:44:45.732183+0000","last_became_active":"2024-09-16T15:44:45.726045+0000","last_became_peered":"2024-09-16T15:44:45.726045+0000","last_unstale":"2024-09-16T15:44:45.732183+0000","last_undegraded":"2024-09-16T15:44:45.732183+0000","last_fullsized":"2024-09-16T15:44:45.732183+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:36:30.549487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.719163+0000","last_change":"2024-09-16T15:44:45.716384+0000","last_active":"2024-09-16T15:44:45.719163+0000","last_peered":"2024-09-16T15:44:45.719163+0000","last_clean":"2024-09-16T15:44:45.719163+0000","last_became_active":"2024-09-16T15:44:45.716101+0000","last_became_peered":"2024-09-16T15:44:45.716101+0000","last_unstale":"2024-09-16T15:44:45.719163+0000","last_undegraded":"2024-09-16T15:44:45.719163+0000","last_fullsized":"2024-09-16T15:44:45.719163+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:21:51.586704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.730872+0000","last_change":"2024-09-16T15:44:45.721590+0000","last_active":"2024-09-16T15:44:45.730872+0000","last_peered":"2024-09-16T15:44:45.730872+0000","last_clean":"2024-09-16T15:44:45.730872+0000","last_became_active":"2024-09-16T15:44:45.721343+0000","last_became_peered":"2024-09-16T15:44:45.721343+0000","last_unstale":"2024-09-16T15:44:45.730872+0000","last_undegraded":"2024-09-16T15:44:45.730872+0000","last_fullsized":"2024-09-16T15:44:45.730872+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:44:28.021891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.730936+0000","last_change":"2024-09-16T15:44:45.722006+0000","last_active":"2024-09-16T15:44:45.730936+0000","last_peered":"2024-09-16T15:44:45.730936+0000","last_clean":"2024-09-16T15:44:45.730936+0000","last_became_active":"2024-09-16T15:44:45.721714+0000","last_became_peered":"2024-09-16T15:44:45.721714+0000","last_unstale":"2024-09-16T15:44:45.730936+0000","last_undegraded":"2024-09-16T15:44:45.730936+0000","last_fullsized":"2024-09-16T15:44:45.730936+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:53:59.005172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.736449+0000","last_change":"2024-09-16T15:44:45.732870+0000","last_active":"2024-09-16T15:44:45.736449+0000","last_peered":"2024-09-16T15:44:45.736449+0000","last_clean":"2024-09-16T15:44:45.736449+0000","last_became_active":"2024-09-16T15:44:45.732609+0000","last_became_peered":"2024-09-16T15:44:45.732609+0000","last_unstale":"2024-09-16T15:44:45.736449+0000","last_undegraded":"2024-09-16T15:44:45.736449+0000","last_fullsized":"2024-09-16T15:44:45.736449+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:44:16.275512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T15:44:44.691707+0000","last_change":"2024-09-16T15:44:44.691707+0000","last_active":"2024-09-16T15:44:44.679289+0000","last_peered":"2024-09-16T15:44:44.679289+0000","last_clean":"2024-09-16T15:44:44.679289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:44:44.691707+0000","last_undegraded":"2024-09-16T15:44:44.691707+0000","last_fullsized":"2024-09-16T15:44:44.691707+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.734176+0000","last_change":"2024-09-16T15:44:45.731373+0000","last_active":"2024-09-16T15:44:45.734176+0000","last_peered":"2024-09-16T15:44:45.734176+0000","last_clean":"2024-09-16T15:44:45.734176+0000","last_became_active":"2024-09-16T15:44:45.731068+0000","last_became_peered":"2024-09-16T15:44:45.731068+0000","last_unstale":"2024-09-16T15:44:45.734176+0000","last_undegraded":"2024-09-16T15:44:45.734176+0000","last_fullsized":"2024-09-16T15:44:45.734176+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:51:57.452114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.724617+0000","last_change":"2024-09-16T15:44:45.719386+0000","last_active":"2024-09-16T15:44:45.724617+0000","last_peered":"2024-09-16T15:44:45.724617+0000","last_clean":"2024-09-16T15:44:45.724617+0000","last_became_active":"2024-09-16T15:44:45.719154+0000","last_became_peered":"2024-09-16T15:44:45.719154+0000","last_unstale":"2024-09-16T15:44:45.724617+0000","last_undegraded":"2024-09-16T15:44:45.724617+0000","last_fullsized":"2024-09-16T15:44:45.724617+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:11:14.213089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.723225+0000","last_change":"2024-09-16T15:44:45.718300+0000","last_active":"2024-09-16T15:44:45.723225+0000","last_peered":"2024-09-16T15:44:45.723225+0000","last_clean":"2024-09-16T15:44:45.723225+0000","last_became_active":"2024-09-16T15:44:45.717725+0000","last_became_peered":"2024-09-16T15:44:45.717725+0000","last_unstale":"2024-09-16T15:44:45.723225+0000","last_undegraded":"2024-09-16T15:44:45.723225+0000","last_fullsized":"2024-09-16T15:44:45.723225+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:03:17.351052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.721190+0000","last_change":"2024-09-16T15:44:45.716075+0000","last_active":"2024-09-16T15:44:45.721190+0000","last_peered":"2024-09-16T15:44:45.721190+0000","last_clean":"2024-09-16T15:44:45.721190+0000","last_became_active":"2024-09-16T15:44:45.715816+0000","last_became_peered":"2024-09-16T15:44:45.715816+0000","last_unstale":"2024-09-16T15:44:45.721190+0000","last_undegraded":"2024-09-16T15:44:45.721190+0000","last_fullsized":"2024-09-16T15:44:45.721190+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:38:03.977000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.725164+0000","last_change":"2024-09-16T15:44:45.721703+0000","last_active":"2024-09-16T15:44:45.725164+0000","last_peered":"2024-09-16T15:44:45.725164+0000","last_clean":"2024-09-16T15:44:45.725164+0000","last_became_active":"2024-09-16T15:44:45.721406+0000","last_became_peered":"2024-09-16T15:44:45.721406+0000","last_unstale":"2024-09-16T15:44:45.725164+0000","last_undegraded":"2024-09-16T15:44:45.725164+0000","last_fullsized":"2024-09-16T15:44:45.725164+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:42:26.902672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T15:44:44.691255+0000","last_change":"2024-09-16T15:44:44.691255+0000","last_active":"2024-09-16T15:44:44.679289+0000","last_peered":"2024-09-16T15:44:44.679289+0000","last_clean":"2024-09-16T15:44:44.679289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:44:44.691255+0000","last_undegraded":"2024-09-16T15:44:44.691255+0000","last_fullsized":"2024-09-16T15:44:44.691255+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.725345+0000","last_change":"2024-09-16T15:44:45.722538+0000","last_active":"2024-09-16T15:44:45.725345+0000","last_peered":"2024-09-16T15:44:45.725345+0000","last_clean":"2024-09-16T15:44:45.725345+0000","last_became_active":"2024-09-16T15:44:45.722286+0000","last_became_peered":"2024-09-16T15:44:45.722286+0000","last_unstale":"2024-09-16T15:44:45.725345+0000","last_undegraded":"2024-09-16T15:44:45.725345+0000","last_fullsized":"2024-09-16T15:44:45.725345+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:45:58.209870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T15:44:44.691255+0000","last_change":"2024-09-16T15:44:44.691255+0000","last_active":"2024-09-16T15:44:44.679289+0000","last_peered":"2024-09-16T15:44:44.679289+0000","last_clean":"2024-09-16T15:44:44.679289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:44:44.691255+0000","last_undegraded":"2024-09-16T15:44:44.691255+0000","last_fullsized":"2024-09-16T15:44:44.691255+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.722818+0000","last_change":"2024-09-16T15:44:45.707684+0000","last_active":"2024-09-16T15:44:45.722818+0000","last_peered":"2024-09-16T15:44:45.722818+0000","last_clean":"2024-09-16T15:44:45.722818+0000","last_became_active":"2024-09-16T15:44:45.707370+0000","last_became_peered":"2024-09-16T15:44:45.707370+0000","last_unstale":"2024-09-16T15:44:45.722818+0000","last_undegraded":"2024-09-16T15:44:45.722818+0000","last_fullsized":"2024-09-16T15:44:45.722818+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:32:11.331276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.724083+0000","last_change":"2024-09-16T15:44:45.712553+0000","last_active":"2024-09-16T15:44:45.724083+0000","last_peered":"2024-09-16T15:44:45.724083+0000","last_clean":"2024-09-16T15:44:45.724083+0000","last_became_active":"2024-09-16T15:44:45.711955+0000","last_became_peered":"2024-09-16T15:44:45.711955+0000","last_unstale":"2024-09-16T15:44:45.724083+0000","last_undegraded":"2024-09-16T15:44:45.724083+0000","last_fullsized":"2024-09-16T15:44:45.724083+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:33:23.442599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.714841+0000","last_change":"2024-09-16T15:44:45.711869+0000","last_active":"2024-09-16T15:44:45.714841+0000","last_peered":"2024-09-16T15:44:45.714841+0000","last_clean":"2024-09-16T15:44:45.714841+0000","last_became_active":"2024-09-16T15:44:45.711534+0000","last_became_peered":"2024-09-16T15:44:45.711534+0000","last_unstale":"2024-09-16T15:44:45.714841+0000","last_undegraded":"2024-09-16T15:44:45.714841+0000","last_fullsized":"2024-09-16T15:44:45.714841+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:31:16.141860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T15:44:44.691565+0000","last_change":"2024-09-16T15:44:44.691565+0000","last_active":"2024-09-16T15:44:44.679289+0000","last_peered":"2024-09-16T15:44:44.679289+0000","last_clean":"2024-09-16T15:44:44.679289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:44:44.691565+0000","last_undegraded":"2024-09-16T15:44:44.691565+0000","last_fullsized":"2024-09-16T15:44:44.691565+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.729551+0000","last_change":"2024-09-16T15:44:45.726051+0000","last_active":"2024-09-16T15:44:45.729551+0000","last_peered":"2024-09-16T15:44:45.729551+0000","last_clean":"2024-09-16T15:44:45.729551+0000","last_became_active":"2024-09-16T15:44:45.720897+0000","last_became_peered":"2024-09-16T15:44:45.720897+0000","last_unstale":"2024-09-16T15:44:45.729551+0000","last_undegraded":"2024-09-16T15:44:45.729551+0000","last_fullsized":"2024-09-16T15:44:45.729551+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:36:30.549487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.732821+0000","last_change":"2024-09-16T15:44:45.724124+0000","last_active":"2024-09-16T15:44:45.732821+0000","last_peered":"2024-09-16T15:44:45.732821+0000","last_clean":"2024-09-16T15:44:45.732821+0000","last_became_active":"2024-09-16T15:44:45.721712+0000","last_became_peered":"2024-09-16T15:44:45.721712+0000","last_unstale":"2024-09-16T15:44:45.732821+0000","last_undegraded":"2024-09-16T15:44:45.732821+0000","last_fullsized":"2024-09-16T15:44:45.732821+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:15:22.807776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"61'4","reported_seq":17,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.732008+0000","last_change":"2024-09-16T15:44:44.700510+0000","last_active":"2024-09-16T15:44:45.732008+0000","last_peered":"2024-09-16T15:44:45.732008+0000","last_clean":"2024-09-16T15:44:45.732008+0000","last_became_active":"2024-09-16T15:44:44.697806+0000","last_became_peered":"2024-09-16T15:44:44.697806+0000","last_unstale":"2024-09-16T15:44:45.732008+0000","last_undegraded":"2024-09-16T15:44:45.732008+0000","last_fullsized":"2024-09-16T15:44:45.732008+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:50:52.878214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.729071+0000","last_change":"2024-09-16T15:44:45.715988+0000","last_active":"2024-09-16T15:44:45.729071+0000","last_peered":"2024-09-16T15:44:45.729071+0000","last_clean":"2024-09-16T15:44:45.729071+0000","last_became_active":"2024-09-16T15:44:45.715735+0000","last_became_peered":"2024-09-16T15:44:45.715735+0000","last_unstale":"2024-09-16T15:44:45.729071+0000","last_undegraded":"2024-09-16T15:44:45.729071+0000","last_fullsized":"2024-09-16T15:44:45.729071+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:39:12.784333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T15:44:44.710442+0000","last_change":"2024-09-16T15:44:44.708252+0000","last_active":"2024-09-16T15:44:44.710442+0000","last_peered":"2024-09-16T15:44:44.710442+0000","last_clean":"2024-09-16T15:44:44.710442+0000","last_became_active":"2024-09-16T15:44:44.707995+0000","last_became_peered":"2024-09-16T15:44:44.707995+0000","last_unstale":"2024-09-16T15:44:44.710442+0000","last_undegraded":"2024-09-16T15:44:44.710442+0000","last_fullsized":"2024-09-16T15:44:44.710442+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:26:28.068533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.734118+0000","last_change":"2024-09-16T15:44:45.729683+0000","last_active":"2024-09-16T15:44:45.734118+0000","last_peered":"2024-09-16T15:44:45.734118+0000","last_clean":"2024-09-16T15:44:45.734118+0000","last_became_active":"2024-09-16T15:44:45.729258+0000","last_became_peered":"2024-09-16T15:44:45.729258+0000","last_unstale":"2024-09-16T15:44:45.734118+0000","last_undegraded":"2024-09-16T15:44:45.734118+0000","last_fullsized":"2024-09-16T15:44:45.734118+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:34:41.864039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"61'2","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.739158+0000","last_change":"2024-09-16T15:44:44.705295+0000","last_active":"2024-09-16T15:44:45.739158+0000","last_peered":"2024-09-16T15:44:45.739158+0000","last_clean":"2024-09-16T15:44:45.739158+0000","last_became_active":"2024-09-16T15:44:44.704913+0000","last_became_peered":"2024-09-16T15:44:44.704913+0000","last_unstale":"2024-09-16T15:44:45.739158+0000","last_undegraded":"2024-09-16T15:44:45.739158+0000","last_fullsized":"2024-09-16T15:44:45.739158+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:33:20.871493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.732416+0000","last_change":"2024-09-16T15:44:45.724296+0000","last_active":"2024-09-16T15:44:45.732416+0000","last_peered":"2024-09-16T15:44:45.732416+0000","last_clean":"2024-09-16T15:44:45.732416+0000","last_became_active":"2024-09-16T15:44:45.722552+0000","last_became_peered":"2024-09-16T15:44:45.722552+0000","last_unstale":"2024-09-16T15:44:45.732416+0000","last_undegraded":"2024-09-16T15:44:45.732416+0000","last_fullsized":"2024-09-16T15:44:45.732416+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:35:05.861042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"61'2","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.737888+0000","last_change":"2024-09-16T15:44:44.701949+0000","last_active":"2024-09-16T15:44:45.737888+0000","last_peered":"2024-09-16T15:44:45.737888+0000","last_clean":"2024-09-16T15:44:45.737888+0000","last_became_active":"2024-09-16T15:44:44.701522+0000","last_became_peered":"2024-09-16T15:44:44.701522+0000","last_unstale":"2024-09-16T15:44:45.737888+0000","last_undegraded":"2024-09-16T15:44:45.737888+0000","last_fullsized":"2024-09-16T15:44:45.737888+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:18:22.181140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.723888+0000","last_change":"2024-09-16T15:44:45.712376+0000","last_active":"2024-09-16T15:44:45.723888+0000","last_peered":"2024-09-16T15:44:45.723888+0000","last_clean":"2024-09-16T15:44:45.723888+0000","last_became_active":"2024-09-16T15:44:45.712045+0000","last_became_peered":"2024-09-16T15:44:45.712045+0000","last_unstale":"2024-09-16T15:44:45.723888+0000","last_undegraded":"2024-09-16T15:44:45.723888+0000","last_fullsized":"2024-09-16T15:44:45.723888+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:20:02.525231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"61'1","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.741440+0000","last_change":"2024-09-16T15:44:44.705164+0000","last_active":"2024-09-16T15:44:45.741440+0000","last_peered":"2024-09-16T15:44:45.741440+0000","last_clean":"2024-09-16T15:44:45.741440+0000","last_became_active":"2024-09-16T15:44:44.704525+0000","last_became_peered":"2024-09-16T15:44:44.704525+0000","last_unstale":"2024-09-16T15:44:45.741440+0000","last_undegraded":"2024-09-16T15:44:45.741440+0000","last_fullsized":"2024-09-16T15:44:45.741440+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:37:15.032653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.724052+0000","last_change":"2024-09-16T15:44:45.719896+0000","last_active":"2024-09-16T15:44:45.724052+0000","last_peered":"2024-09-16T15:44:45.724052+0000","last_clean":"2024-09-16T15:44:45.724052+0000","last_became_active":"2024-09-16T15:44:45.718945+0000","last_became_peered":"2024-09-16T15:44:45.718945+0000","last_unstale":"2024-09-16T15:44:45.724052+0000","last_undegraded":"2024-09-16T15:44:45.724052+0000","last_fullsized":"2024-09-16T15:44:45.724052+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:31:16.283923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"61'1","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.736043+0000","last_change":"2024-09-16T15:44:44.710842+0000","last_active":"2024-09-16T15:44:45.736043+0000","last_peered":"2024-09-16T15:44:45.736043+0000","last_clean":"2024-09-16T15:44:45.736043+0000","last_became_active":"2024-09-16T15:44:44.710529+0000","last_became_peered":"2024-09-16T15:44:44.710529+0000","last_unstale":"2024-09-16T15:44:45.736043+0000","last_undegraded":"2024-09-16T15:44:45.736043+0000","last_fullsized":"2024-09-16T15:44:45.736043+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:36:04.434843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.716864+0000","last_change":"2024-09-16T15:44:45.708591+0000","last_active":"2024-09-16T15:44:45.716864+0000","last_peered":"2024-09-16T15:44:45.716864+0000","last_clean":"2024-09-16T15:44:45.716864+0000","last_became_active":"2024-09-16T15:44:45.708350+0000","last_became_peered":"2024-09-16T15:44:45.708350+0000","last_unstale":"2024-09-16T15:44:45.716864+0000","last_undegraded":"2024-09-16T15:44:45.716864+0000","last_fullsized":"2024-09-16T15:44:45.716864+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:28:57.953253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.704237+0000","last_change":"2024-09-16T15:44:44.697711+0000","last_active":"2024-09-16T15:44:45.704237+0000","last_peered":"2024-09-16T15:44:45.704237+0000","last_clean":"2024-09-16T15:44:45.704237+0000","last_became_active":"2024-09-16T15:44:44.697357+0000","last_became_peered":"2024-09-16T15:44:44.697357+0000","last_unstale":"2024-09-16T15:44:45.704237+0000","last_undegraded":"2024-09-16T15:44:45.704237+0000","last_fullsized":"2024-09-16T15:44:45.704237+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:20:52.337348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.730799+0000","last_change":"2024-09-16T15:44:45.719359+0000","last_active":"2024-09-16T15:44:45.730799+0000","last_peered":"2024-09-16T15:44:45.730799+0000","last_clean":"2024-09-16T15:44:45.730799+0000","last_became_active":"2024-09-16T15:44:45.719093+0000","last_became_peered":"2024-09-16T15:44:45.719093+0000","last_unstale":"2024-09-16T15:44:45.730799+0000","last_undegraded":"2024-09-16T15:44:45.730799+0000","last_fullsized":"2024-09-16T15:44:45.730799+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:02:03.820471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.703161+0000","last_change":"2024-09-16T15:44:44.694298+0000","last_active":"2024-09-16T15:44:45.703161+0000","last_peered":"2024-09-16T15:44:45.703161+0000","last_clean":"2024-09-16T15:44:45.703161+0000","last_became_active":"2024-09-16T15:44:44.694052+0000","last_became_peered":"2024-09-16T15:44:44.694052+0000","last_unstale":"2024-09-16T15:44:45.703161+0000","last_undegraded":"2024-09-16T15:44:45.703161+0000","last_fullsized":"2024-09-16T15:44:45.703161+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:25:26.438122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.715275+0000","last_change":"2024-09-16T15:44:45.713168+0000","last_active":"2024-09-16T15:44:45.715275+0000","last_peered":"2024-09-16T15:44:45.715275+0000","last_clean":"2024-09-16T15:44:45.715275+0000","last_became_active":"2024-09-16T15:44:45.711840+0000","last_became_peered":"2024-09-16T15:44:45.711840+0000","last_unstale":"2024-09-16T15:44:45.715275+0000","last_undegraded":"2024-09-16T15:44:45.715275+0000","last_fullsized":"2024-09-16T15:44:45.715275+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:21:48.643444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.709187+0000","last_change":"2024-09-16T15:44:44.703702+0000","last_active":"2024-09-16T15:44:45.709187+0000","last_peered":"2024-09-16T15:44:45.709187+0000","last_clean":"2024-09-16T15:44:45.709187+0000","last_became_active":"2024-09-16T15:44:44.703424+0000","last_became_peered":"2024-09-16T15:44:44.703424+0000","last_unstale":"2024-09-16T15:44:45.709187+0000","last_undegraded":"2024-09-16T15:44:45.709187+0000","last_fullsized":"2024-09-16T15:44:45.709187+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:24:00.275655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.734732+0000","last_change":"2024-09-16T15:44:45.726605+0000","last_active":"2024-09-16T15:44:45.734732+0000","last_peered":"2024-09-16T15:44:45.734732+0000","last_clean":"2024-09-16T15:44:45.734732+0000","last_became_active":"2024-09-16T15:44:45.726368+0000","last_became_peered":"2024-09-16T15:44:45.726368+0000","last_unstale":"2024-09-16T15:44:45.734732+0000","last_undegraded":"2024-09-16T15:44:45.734732+0000","last_fullsized":"2024-09-16T15:44:45.734732+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:31:16.283923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.704573+0000","last_change":"2024-09-16T15:44:44.695923+0000","last_active":"2024-09-16T15:44:45.704573+0000","last_peered":"2024-09-16T15:44:45.704573+0000","last_clean":"2024-09-16T15:44:45.704573+0000","last_became_active":"2024-09-16T15:44:44.695618+0000","last_became_peered":"2024-09-16T15:44:44.695618+0000","last_unstale":"2024-09-16T15:44:45.704573+0000","last_undegraded":"2024-09-16T15:44:45.704573+0000","last_fullsized":"2024-09-16T15:44:45.704573+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:41:35.721501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.728654+0000","last_change":"2024-09-16T15:44:45.723911+0000","last_active":"2024-09-16T15:44:45.728654+0000","last_peered":"2024-09-16T15:44:45.728654+0000","last_clean":"2024-09-16T15:44:45.728654+0000","last_became_active":"2024-09-16T15:44:45.723557+0000","last_became_peered":"2024-09-16T15:44:45.723557+0000","last_unstale":"2024-09-16T15:44:45.728654+0000","last_undegraded":"2024-09-16T15:44:45.728654+0000","last_fullsized":"2024-09-16T15:44:45.728654+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:18:23.183829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"61'3","reported_seq":16,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.737703+0000","last_change":"2024-09-16T15:44:44.700702+0000","last_active":"2024-09-16T15:44:45.737703+0000","last_peered":"2024-09-16T15:44:45.737703+0000","last_clean":"2024-09-16T15:44:45.737703+0000","last_became_active":"2024-09-16T15:44:44.700438+0000","last_became_peered":"2024-09-16T15:44:44.700438+0000","last_unstale":"2024-09-16T15:44:45.737703+0000","last_undegraded":"2024-09-16T15:44:45.737703+0000","last_fullsized":"2024-09-16T15:44:45.737703+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:46:15.633475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.713366+0000","last_change":"2024-09-16T15:44:45.709431+0000","last_active":"2024-09-16T15:44:45.713366+0000","last_peered":"2024-09-16T15:44:45.713366+0000","last_clean":"2024-09-16T15:44:45.713366+0000","last_became_active":"2024-09-16T15:44:45.709179+0000","last_became_peered":"2024-09-16T15:44:45.709179+0000","last_unstale":"2024-09-16T15:44:45.713366+0000","last_undegraded":"2024-09-16T15:44:45.713366+0000","last_fullsized":"2024-09-16T15:44:45.713366+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:34:54.933570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"61'6","reported_seq":19,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.743392+0000","last_change":"2024-09-16T15:44:44.691387+0000","last_active":"2024-09-16T15:44:45.743392+0000","last_peered":"2024-09-16T15:44:45.743392+0000","last_clean":"2024-09-16T15:44:45.743392+0000","last_became_active":"2024-09-16T15:44:44.691144+0000","last_became_peered":"2024-09-16T15:44:44.691144+0000","last_unstale":"2024-09-16T15:44:45.743392+0000","last_undegraded":"2024-09-16T15:44:45.743392+0000","last_fullsized":"2024-09-16T15:44:45.743392+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:09:54.112439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.723072+0000","last_change":"2024-09-16T15:44:45.717715+0000","last_active":"2024-09-16T15:44:45.723072+0000","last_peered":"2024-09-16T15:44:45.723072+0000","last_clean":"2024-09-16T15:44:45.723072+0000","last_became_active":"2024-09-16T15:44:45.717366+0000","last_became_peered":"2024-09-16T15:44:45.717366+0000","last_unstale":"2024-09-16T15:44:45.723072+0000","last_undegraded":"2024-09-16T15:44:45.723072+0000","last_fullsized":"2024-09-16T15:44:45.723072+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:12:07.274566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"61'2","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.730407+0000","last_change":"2024-09-16T15:44:44.702026+0000","last_active":"2024-09-16T15:44:45.730407+0000","last_peered":"2024-09-16T15:44:45.730407+0000","last_clean":"2024-09-16T15:44:45.730407+0000","last_became_active":"2024-09-16T15:44:44.701701+0000","last_became_peered":"2024-09-16T15:44:44.701701+0000","last_unstale":"2024-09-16T15:44:45.730407+0000","last_undegraded":"2024-09-16T15:44:45.730407+0000","last_fullsized":"2024-09-16T15:44:45.730407+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:30:25.597599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.737862+0000","last_change":"2024-09-16T15:44:45.734880+0000","last_active":"2024-09-16T15:44:45.737862+0000","last_peered":"2024-09-16T15:44:45.737862+0000","last_clean":"2024-09-16T15:44:45.737862+0000","last_became_active":"2024-09-16T15:44:45.734616+0000","last_became_peered":"2024-09-16T15:44:45.734616+0000","last_unstale":"2024-09-16T15:44:45.737862+0000","last_undegraded":"2024-09-16T15:44:45.737862+0000","last_fullsized":"2024-09-16T15:44:45.737862+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:19:00.003404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.697639+0000","last_change":"2024-09-16T15:44:44.701584+0000","last_active":"2024-09-16T15:44:45.697639+0000","last_peered":"2024-09-16T15:44:45.697639+0000","last_clean":"2024-09-16T15:44:45.697639+0000","last_became_active":"2024-09-16T15:44:44.701327+0000","last_became_peered":"2024-09-16T15:44:44.701327+0000","last_unstale":"2024-09-16T15:44:45.697639+0000","last_undegraded":"2024-09-16T15:44:45.697639+0000","last_fullsized":"2024-09-16T15:44:45.697639+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:44:05.886418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.720386+0000","last_change":"2024-09-16T15:44:45.717514+0000","last_active":"2024-09-16T15:44:45.720386+0000","last_peered":"2024-09-16T15:44:45.720386+0000","last_clean":"2024-09-16T15:44:45.720386+0000","last_became_active":"2024-09-16T15:44:45.717270+0000","last_became_peered":"2024-09-16T15:44:45.717270+0000","last_unstale":"2024-09-16T15:44:45.720386+0000","last_undegraded":"2024-09-16T15:44:45.720386+0000","last_fullsized":"2024-09-16T15:44:45.720386+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:12:47.737828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.698166+0000","last_change":"2024-09-16T15:44:44.702562+0000","last_active":"2024-09-16T15:44:45.698166+0000","last_peered":"2024-09-16T15:44:45.698166+0000","last_clean":"2024-09-16T15:44:45.698166+0000","last_became_active":"2024-09-16T15:44:44.702282+0000","last_became_peered":"2024-09-16T15:44:44.702282+0000","last_unstale":"2024-09-16T15:44:45.698166+0000","last_undegraded":"2024-09-16T15:44:45.698166+0000","last_fullsized":"2024-09-16T15:44:45.698166+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:34:40.861349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.723514+0000","last_change":"2024-09-16T15:44:45.717730+0000","last_active":"2024-09-16T15:44:45.723514+0000","last_peered":"2024-09-16T15:44:45.723514+0000","last_clean":"2024-09-16T15:44:45.723514+0000","last_became_active":"2024-09-16T15:44:45.717456+0000","last_became_peered":"2024-09-16T15:44:45.717456+0000","last_unstale":"2024-09-16T15:44:45.723514+0000","last_undegraded":"2024-09-16T15:44:45.723514+0000","last_fullsized":"2024-09-16T15:44:45.723514+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:09:14.114379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"61'8","reported_seq":21,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.740384+0000","last_change":"2024-09-16T15:44:44.700378+0000","last_active":"2024-09-16T15:44:45.740384+0000","last_peered":"2024-09-16T15:44:45.740384+0000","last_clean":"2024-09-16T15:44:45.740384+0000","last_became_active":"2024-09-16T15:44:44.700114+0000","last_became_peered":"2024-09-16T15:44:44.700114+0000","last_unstale":"2024-09-16T15:44:45.740384+0000","last_undegraded":"2024-09-16T15:44:45.740384+0000","last_fullsized":"2024-09-16T15:44:45.740384+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:46:16.051799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.720485+0000","last_change":"2024-09-16T15:44:45.716464+0000","last_active":"2024-09-16T15:44:45.720485+0000","last_peered":"2024-09-16T15:44:45.720485+0000","last_clean":"2024-09-16T15:44:45.720485+0000","last_became_active":"2024-09-16T15:44:45.716140+0000","last_became_peered":"2024-09-16T15:44:45.716140+0000","last_unstale":"2024-09-16T15:44:45.720485+0000","last_undegraded":"2024-09-16T15:44:45.720485+0000","last_fullsized":"2024-09-16T15:44:45.720485+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:31:46.215684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T15:44:44.711883+0000","last_change":"2024-09-16T15:44:44.711284+0000","last_active":"2024-09-16T15:44:44.711883+0000","last_peered":"2024-09-16T15:44:44.711883+0000","last_clean":"2024-09-16T15:44:44.711883+0000","last_became_active":"2024-09-16T15:44:44.709695+0000","last_became_peered":"2024-09-16T15:44:44.709695+0000","last_unstale":"2024-09-16T15:44:44.711883+0000","last_undegraded":"2024-09-16T15:44:44.711883+0000","last_fullsized":"2024-09-16T15:44:44.711883+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:29:18.863386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.718328+0000","last_change":"2024-09-16T15:44:45.714790+0000","last_active":"2024-09-16T15:44:45.718328+0000","last_peered":"2024-09-16T15:44:45.718328+0000","last_clean":"2024-09-16T15:44:45.718328+0000","last_became_active":"2024-09-16T15:44:45.713900+0000","last_became_peered":"2024-09-16T15:44:45.713900+0000","last_unstale":"2024-09-16T15:44:45.718328+0000","last_undegraded":"2024-09-16T15:44:45.718328+0000","last_fullsized":"2024-09-16T15:44:45.718328+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:50:53.880904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"61'2","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.730326+0000","last_change":"2024-09-16T15:44:44.698462+0000","last_active":"2024-09-16T15:44:45.730326+0000","last_peered":"2024-09-16T15:44:45.730326+0000","last_clean":"2024-09-16T15:44:45.730326+0000","last_became_active":"2024-09-16T15:44:44.698179+0000","last_became_peered":"2024-09-16T15:44:44.698179+0000","last_unstale":"2024-09-16T15:44:45.730326+0000","last_undegraded":"2024-09-16T15:44:45.730326+0000","last_fullsized":"2024-09-16T15:44:45.730326+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:13:21.284936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T15:44:44.693602+0000","last_change":"2024-09-16T15:44:44.693602+0000","last_active":"2024-09-16T15:44:44.679289+0000","last_peered":"2024-09-16T15:44:44.679289+0000","last_clean":"2024-09-16T15:44:44.679289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:44:44.693602+0000","last_undegraded":"2024-09-16T15:44:44.693602+0000","last_fullsized":"2024-09-16T15:44:44.693602+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"61'1","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.741605+0000","last_change":"2024-09-16T15:44:44.706373+0000","last_active":"2024-09-16T15:44:45.741605+0000","last_peered":"2024-09-16T15:44:45.741605+0000","last_clean":"2024-09-16T15:44:45.741605+0000","last_became_active":"2024-09-16T15:44:44.705120+0000","last_became_peered":"2024-09-16T15:44:44.705120+0000","last_unstale":"2024-09-16T15:44:45.741605+0000","last_undegraded":"2024-09-16T15:44:45.741605+0000","last_fullsized":"2024-09-16T15:44:45.741605+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:07:54.665094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.735750+0000","last_change":"2024-09-16T15:44:45.726953+0000","last_active":"2024-09-16T15:44:45.735750+0000","last_peered":"2024-09-16T15:44:45.735750+0000","last_clean":"2024-09-16T15:44:45.735750+0000","last_became_active":"2024-09-16T15:44:45.723508+0000","last_became_peered":"2024-09-16T15:44:45.723508+0000","last_unstale":"2024-09-16T15:44:45.735750+0000","last_undegraded":"2024-09-16T15:44:45.735750+0000","last_fullsized":"2024-09-16T15:44:45.735750+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:39:26.814713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.705171+0000","last_change":"2024-09-16T15:44:44.708721+0000","last_active":"2024-09-16T15:44:45.705171+0000","last_peered":"2024-09-16T15:44:45.705171+0000","last_clean":"2024-09-16T15:44:45.705171+0000","last_became_active":"2024-09-16T15:44:44.708471+0000","last_became_peered":"2024-09-16T15:44:44.708471+0000","last_unstale":"2024-09-16T15:44:45.705171+0000","last_undegraded":"2024-09-16T15:44:45.705171+0000","last_fullsized":"2024-09-16T15:44:45.705171+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:39:49.307383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.720161+0000","last_change":"2024-09-16T15:44:45.716983+0000","last_active":"2024-09-16T15:44:45.720161+0000","last_peered":"2024-09-16T15:44:45.720161+0000","last_clean":"2024-09-16T15:44:45.720161+0000","last_became_active":"2024-09-16T15:44:45.715530+0000","last_became_peered":"2024-09-16T15:44:45.715530+0000","last_unstale":"2024-09-16T15:44:45.720161+0000","last_undegraded":"2024-09-16T15:44:45.720161+0000","last_fullsized":"2024-09-16T15:44:45.720161+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:03:43.262795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"61'2","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.733237+0000","last_change":"2024-09-16T15:44:44.696118+0000","last_active":"2024-09-16T15:44:45.733237+0000","last_peered":"2024-09-16T15:44:45.733237+0000","last_clean":"2024-09-16T15:44:45.733237+0000","last_became_active":"2024-09-16T15:44:44.695842+0000","last_became_peered":"2024-09-16T15:44:44.695842+0000","last_unstale":"2024-09-16T15:44:45.733237+0000","last_undegraded":"2024-09-16T15:44:45.733237+0000","last_fullsized":"2024-09-16T15:44:45.733237+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:31:45.212994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.724057+0000","last_change":"2024-09-16T15:44:45.712249+0000","last_active":"2024-09-16T15:44:45.724057+0000","last_peered":"2024-09-16T15:44:45.724057+0000","last_clean":"2024-09-16T15:44:45.724057+0000","last_became_active":"2024-09-16T15:44:45.711524+0000","last_became_peered":"2024-09-16T15:44:45.711524+0000","last_unstale":"2024-09-16T15:44:45.724057+0000","last_undegraded":"2024-09-16T15:44:45.724057+0000","last_fullsized":"2024-09-16T15:44:45.724057+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:45:42.401451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.703897+0000","last_change":"2024-09-16T15:44:44.695883+0000","last_active":"2024-09-16T15:44:45.703897+0000","last_peered":"2024-09-16T15:44:45.703897+0000","last_clean":"2024-09-16T15:44:45.703897+0000","last_became_active":"2024-09-16T15:44:44.695562+0000","last_became_peered":"2024-09-16T15:44:44.695562+0000","last_unstale":"2024-09-16T15:44:45.703897+0000","last_undegraded":"2024-09-16T15:44:45.703897+0000","last_fullsized":"2024-09-16T15:44:45.703897+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:22:24.705292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.722865+0000","last_change":"2024-09-16T15:44:45.708395+0000","last_active":"2024-09-16T15:44:45.722865+0000","last_peered":"2024-09-16T15:44:45.722865+0000","last_clean":"2024-09-16T15:44:45.722865+0000","last_became_active":"2024-09-16T15:44:45.708090+0000","last_became_peered":"2024-09-16T15:44:45.708090+0000","last_unstale":"2024-09-16T15:44:45.722865+0000","last_undegraded":"2024-09-16T15:44:45.722865+0000","last_fullsized":"2024-09-16T15:44:45.722865+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:13:37.240255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.712718+0000","last_change":"2024-09-16T15:44:44.705798+0000","last_active":"2024-09-16T15:44:45.712718+0000","last_peered":"2024-09-16T15:44:45.712718+0000","last_clean":"2024-09-16T15:44:45.712718+0000","last_became_active":"2024-09-16T15:44:44.705468+0000","last_became_peered":"2024-09-16T15:44:44.705468+0000","last_unstale":"2024-09-16T15:44:45.712718+0000","last_undegraded":"2024-09-16T15:44:45.712718+0000","last_fullsized":"2024-09-16T15:44:45.712718+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:21:59.839074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.736422+0000","last_change":"2024-09-16T15:44:45.726903+0000","last_active":"2024-09-16T15:44:45.736422+0000","last_peered":"2024-09-16T15:44:45.736422+0000","last_clean":"2024-09-16T15:44:45.736422+0000","last_became_active":"2024-09-16T15:44:45.726666+0000","last_became_peered":"2024-09-16T15:44:45.726666+0000","last_unstale":"2024-09-16T15:44:45.736422+0000","last_undegraded":"2024-09-16T15:44:45.736422+0000","last_fullsized":"2024-09-16T15:44:45.736422+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:51:57.452114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T15:44:44.710737+0000","last_change":"2024-09-16T15:44:44.708470+0000","last_active":"2024-09-16T15:44:44.710737+0000","last_peered":"2024-09-16T15:44:44.710737+0000","last_clean":"2024-09-16T15:44:44.710737+0000","last_became_active":"2024-09-16T15:44:44.708182+0000","last_became_peered":"2024-09-16T15:44:44.708182+0000","last_unstale":"2024-09-16T15:44:44.710737+0000","last_undegraded":"2024-09-16T15:44:44.710737+0000","last_fullsized":"2024-09-16T15:44:44.710737+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:37:09.093235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.718282+0000","last_change":"2024-09-16T15:44:45.707416+0000","last_active":"2024-09-16T15:44:45.718282+0000","last_peered":"2024-09-16T15:44:45.718282+0000","last_clean":"2024-09-16T15:44:45.718282+0000","last_became_active":"2024-09-16T15:44:45.707148+0000","last_became_peered":"2024-09-16T15:44:45.707148+0000","last_unstale":"2024-09-16T15:44:45.718282+0000","last_undegraded":"2024-09-16T15:44:45.718282+0000","last_fullsized":"2024-09-16T15:44:45.718282+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:34:31.566558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.713344+0000","last_change":"2024-09-16T15:44:44.710583+0000","last_active":"2024-09-16T15:44:45.713344+0000","last_peered":"2024-09-16T15:44:45.713344+0000","last_clean":"2024-09-16T15:44:45.713344+0000","last_became_active":"2024-09-16T15:44:44.710293+0000","last_became_peered":"2024-09-16T15:44:44.710293+0000","last_unstale":"2024-09-16T15:44:45.713344+0000","last_undegraded":"2024-09-16T15:44:45.713344+0000","last_fullsized":"2024-09-16T15:44:45.713344+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:03:46.514703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T15:44:44.692278+0000","last_change":"2024-09-16T15:44:44.692278+0000","last_active":"2024-09-16T15:44:44.679289+0000","last_peered":"2024-09-16T15:44:44.679289+0000","last_clean":"2024-09-16T15:44:44.679289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:44:44.692278+0000","last_undegraded":"2024-09-16T15:44:44.692278+0000","last_fullsized":"2024-09-16T15:44:44.692278+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.704857+0000","last_change":"2024-09-16T15:44:44.703904+0000","last_active":"2024-09-16T15:44:45.704857+0000","last_peered":"2024-09-16T15:44:45.704857+0000","last_clean":"2024-09-16T15:44:45.704857+0000","last_became_active":"2024-09-16T15:44:44.703644+0000","last_became_peered":"2024-09-16T15:44:44.703644+0000","last_unstale":"2024-09-16T15:44:45.704857+0000","last_undegraded":"2024-09-16T15:44:45.704857+0000","last_fullsized":"2024-09-16T15:44:45.704857+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:31:17.501469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.724319+0000","last_change":"2024-09-16T15:44:45.714221+0000","last_active":"2024-09-16T15:44:45.724319+0000","last_peered":"2024-09-16T15:44:45.724319+0000","last_clean":"2024-09-16T15:44:45.724319+0000","last_became_active":"2024-09-16T15:44:45.713871+0000","last_became_peered":"2024-09-16T15:44:45.713871+0000","last_unstale":"2024-09-16T15:44:45.724319+0000","last_undegraded":"2024-09-16T15:44:45.724319+0000","last_fullsized":"2024-09-16T15:44:45.724319+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:09:31.284857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.717881+0000","last_change":"2024-09-16T15:44:44.698858+0000","last_active":"2024-09-16T15:44:45.717881+0000","last_peered":"2024-09-16T15:44:45.717881+0000","last_clean":"2024-09-16T15:44:45.717881+0000","last_became_active":"2024-09-16T15:44:44.698546+0000","last_became_peered":"2024-09-16T15:44:44.698546+0000","last_unstale":"2024-09-16T15:44:45.717881+0000","last_undegraded":"2024-09-16T15:44:45.717881+0000","last_fullsized":"2024-09-16T15:44:45.717881+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:03:16.348362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.730750+0000","last_change":"2024-09-16T15:44:45.719192+0000","last_active":"2024-09-16T15:44:45.730750+0000","last_peered":"2024-09-16T15:44:45.730750+0000","last_clean":"2024-09-16T15:44:45.730750+0000","last_became_active":"2024-09-16T15:44:45.716454+0000","last_became_peered":"2024-09-16T15:44:45.716454+0000","last_unstale":"2024-09-16T15:44:45.730750+0000","last_undegraded":"2024-09-16T15:44:45.730750+0000","last_fullsized":"2024-09-16T15:44:45.730750+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:05:16.681904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T15:44:44.710050+0000","last_change":"2024-09-16T15:44:44.707043+0000","last_active":"2024-09-16T15:44:44.710050+0000","last_peered":"2024-09-16T15:44:44.710050+0000","last_clean":"2024-09-16T15:44:44.710050+0000","last_became_active":"2024-09-16T15:44:44.706747+0000","last_became_peered":"2024-09-16T15:44:44.706747+0000","last_unstale":"2024-09-16T15:44:44.710050+0000","last_undegraded":"2024-09-16T15:44:44.710050+0000","last_fullsized":"2024-09-16T15:44:44.710050+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:15:01.043845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.730635+0000","last_change":"2024-09-16T15:44:45.726366+0000","last_active":"2024-09-16T15:44:45.730635+0000","last_peered":"2024-09-16T15:44:45.730635+0000","last_clean":"2024-09-16T15:44:45.730635+0000","last_became_active":"2024-09-16T15:44:45.726122+0000","last_became_peered":"2024-09-16T15:44:45.726122+0000","last_unstale":"2024-09-16T15:44:45.730635+0000","last_undegraded":"2024-09-16T15:44:45.730635+0000","last_fullsized":"2024-09-16T15:44:45.730635+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:13:21.061051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.705257+0000","last_change":"2024-09-16T15:44:44.704189+0000","last_active":"2024-09-16T15:44:45.705257+0000","last_peered":"2024-09-16T15:44:45.705257+0000","last_clean":"2024-09-16T15:44:45.705257+0000","last_became_active":"2024-09-16T15:44:44.703916+0000","last_became_peered":"2024-09-16T15:44:44.703916+0000","last_unstale":"2024-09-16T15:44:45.705257+0000","last_undegraded":"2024-09-16T15:44:45.705257+0000","last_fullsized":"2024-09-16T15:44:45.705257+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:50:09.467231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.724010+0000","last_change":"2024-09-16T15:44:45.710420+0000","last_active":"2024-09-16T15:44:45.724010+0000","last_peered":"2024-09-16T15:44:45.724010+0000","last_clean":"2024-09-16T15:44:45.724010+0000","last_became_active":"2024-09-16T15:44:45.710144+0000","last_became_peered":"2024-09-16T15:44:45.710144+0000","last_unstale":"2024-09-16T15:44:45.724010+0000","last_undegraded":"2024-09-16T15:44:45.724010+0000","last_fullsized":"2024-09-16T15:44:45.724010+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:23:06.419124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.707293+0000","last_change":"2024-09-16T15:44:44.708397+0000","last_active":"2024-09-16T15:44:45.707293+0000","last_peered":"2024-09-16T15:44:45.707293+0000","last_clean":"2024-09-16T15:44:45.707293+0000","last_became_active":"2024-09-16T15:44:44.708101+0000","last_became_peered":"2024-09-16T15:44:44.708101+0000","last_unstale":"2024-09-16T15:44:45.707293+0000","last_undegraded":"2024-09-16T15:44:45.707293+0000","last_fullsized":"2024-09-16T15:44:45.707293+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:09:26.705708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T15:44:44.691091+0000","last_change":"2024-09-16T15:44:44.691091+0000","last_active":"2024-09-16T15:44:44.679289+0000","last_peered":"2024-09-16T15:44:44.679289+0000","last_clean":"2024-09-16T15:44:44.679289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:44:44.691091+0000","last_undegraded":"2024-09-16T15:44:44.691091+0000","last_fullsized":"2024-09-16T15:44:44.691091+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.703426+0000","last_change":"2024-09-16T15:44:44.698425+0000","last_active":"2024-09-16T15:44:45.703426+0000","last_peered":"2024-09-16T15:44:45.703426+0000","last_clean":"2024-09-16T15:44:45.703426+0000","last_became_active":"2024-09-16T15:44:44.698141+0000","last_became_peered":"2024-09-16T15:44:44.698141+0000","last_unstale":"2024-09-16T15:44:45.703426+0000","last_undegraded":"2024-09-16T15:44:45.703426+0000","last_fullsized":"2024-09-16T15:44:45.703426+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:55:27.169366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.724043+0000","last_change":"2024-09-16T15:44:45.720393+0000","last_active":"2024-09-16T15:44:45.724043+0000","last_peered":"2024-09-16T15:44:45.724043+0000","last_clean":"2024-09-16T15:44:45.724043+0000","last_became_active":"2024-09-16T15:44:45.720028+0000","last_became_peered":"2024-09-16T15:44:45.720028+0000","last_unstale":"2024-09-16T15:44:45.724043+0000","last_undegraded":"2024-09-16T15:44:45.724043+0000","last_fullsized":"2024-09-16T15:44:45.724043+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:46:17.054489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.703722+0000","last_change":"2024-09-16T15:44:44.690580+0000","last_active":"2024-09-16T15:44:45.703722+0000","last_peered":"2024-09-16T15:44:45.703722+0000","last_clean":"2024-09-16T15:44:45.703722+0000","last_became_active":"2024-09-16T15:44:44.690302+0000","last_became_peered":"2024-09-16T15:44:44.690302+0000","last_unstale":"2024-09-16T15:44:45.703722+0000","last_undegraded":"2024-09-16T15:44:45.703722+0000","last_fullsized":"2024-09-16T15:44:45.703722+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:45:53.368994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":185,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.709646+0000","last_change":"2024-09-16T15:39:12.698813+0000","last_active":"2024-09-16T15:44:45.709646+0000","last_peered":"2024-09-16T15:44:45.709646+0000","last_clean":"2024-09-16T15:44:45.709646+0000","last_became_active":"2024-09-16T15:39:12.697695+0000","last_became_peered":"2024-09-16T15:39:12.697695+0000","last_unstale":"2024-09-16T15:44:45.709646+0000","last_undegraded":"2024-09-16T15:44:45.709646+0000","last_fullsized":"2024-09-16T15:44:45.709646+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_clean_scrub_stamp":"2024-09-16T15:39:11.684967+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:02.061081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2388,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":34,"ondisk_log_size":34,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509512,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40208,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331632,"statfs":{"total":96636764160,"available":96595591168,"internally_reserved":0,"allocated":712704,"data_stored":296806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":40434120},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509511,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46356,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325484,"statfs":{"total":96636764160,"available":96589295616,"internally_reserved":0,"allocated":1306624,"data_stored":887174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509510,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45772,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326068,"statfs":{"total":96636764160,"available":96589893632,"internally_reserved":0,"allocated":712704,"data_stored":296806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509510,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40204,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331636,"statfs":{"total":96636764160,"available":96595595264,"internally_reserved":0,"allocated":712704,"data_stored":296806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":40434121},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509510,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62456,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94309384,"statfs":{"total":96636764160,"available":96572809216,"internally_reserved":0,"allocated":745472,"data_stored":305381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":63175111},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509511,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40788,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331052,"statfs":{"total":96636764160,"available":96594997248,"internally_reserved":0,"allocated":1306624,"data_stored":887174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509509,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40208,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331632,"statfs":{"total":96636764160,"available":96595591168,"internally_reserved":0,"allocated":712704,"data_stored":296806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542216,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40208,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331632,"statfs":{"total":96636764160,"available":96595591168,"internally_reserved":0,"allocated":712704,"data_stored":296806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":40434124},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:44:50.792 INFO:tasks.ceph.ceph_manager.ceph:PG 9.3e is not active+clean 2024-09-16T15:44:50.792 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:44:44.692975+0000', 'last_change': '2024-09-16T15:44:44.692975+0000', 'last_active': '2024-09-16T15:44:44.679289+0000', 'last_peered': '2024-09-16T15:44:44.679289+0000', 'last_clean': '2024-09-16T15:44:44.679289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:44:44.692975+0000', 'last_undegraded': '2024-09-16T15:44:44.692975+0000', 'last_fullsized': '2024-09-16T15:44:44.692975+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:44:44.679289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:44:44.679289+0000', 'last_clean_scrub_stamp': '2024-09-16T15:44:44.679289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-16T15:44:50.793 INFO:tasks.ceph.ceph_manager.ceph:PG 9.2d is not active+clean 2024-09-16T15:44:50.793 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:44:44.691707+0000', 'last_change': '2024-09-16T15:44:44.691707+0000', 'last_active': '2024-09-16T15:44:44.679289+0000', 'last_peered': '2024-09-16T15:44:44.679289+0000', 'last_clean': '2024-09-16T15:44:44.679289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:44:44.691707+0000', 'last_undegraded': '2024-09-16T15:44:44.691707+0000', 'last_fullsized': '2024-09-16T15:44:44.691707+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:44:44.679289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:44:44.679289+0000', 'last_clean_scrub_stamp': '2024-09-16T15:44:44.679289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-16T15:44:50.793 INFO:tasks.ceph.ceph_manager.ceph:PG 9.27 is not active+clean 2024-09-16T15:44:50.793 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:44:44.691255+0000', 'last_change': '2024-09-16T15:44:44.691255+0000', 'last_active': '2024-09-16T15:44:44.679289+0000', 'last_peered': '2024-09-16T15:44:44.679289+0000', 'last_clean': '2024-09-16T15:44:44.679289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:44:44.691255+0000', 'last_undegraded': '2024-09-16T15:44:44.691255+0000', 'last_fullsized': '2024-09-16T15:44:44.691255+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:44:44.679289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:44:44.679289+0000', 'last_clean_scrub_stamp': '2024-09-16T15:44:44.679289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-16T15:44:50.793 INFO:tasks.ceph.ceph_manager.ceph:PG 9.25 is not active+clean 2024-09-16T15:44:50.793 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:44:44.691255+0000', 'last_change': '2024-09-16T15:44:44.691255+0000', 'last_active': '2024-09-16T15:44:44.679289+0000', 'last_peered': '2024-09-16T15:44:44.679289+0000', 'last_clean': '2024-09-16T15:44:44.679289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:44:44.691255+0000', 'last_undegraded': '2024-09-16T15:44:44.691255+0000', 'last_fullsized': '2024-09-16T15:44:44.691255+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:44:44.679289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:44:44.679289+0000', 'last_clean_scrub_stamp': '2024-09-16T15:44:44.679289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-16T15:44:50.793 INFO:tasks.ceph.ceph_manager.ceph:PG 9.21 is not active+clean 2024-09-16T15:44:50.793 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:44:44.691565+0000', 'last_change': '2024-09-16T15:44:44.691565+0000', 'last_active': '2024-09-16T15:44:44.679289+0000', 'last_peered': '2024-09-16T15:44:44.679289+0000', 'last_clean': '2024-09-16T15:44:44.679289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:44:44.691565+0000', 'last_undegraded': '2024-09-16T15:44:44.691565+0000', 'last_fullsized': '2024-09-16T15:44:44.691565+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:44:44.679289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:44:44.679289+0000', 'last_clean_scrub_stamp': '2024-09-16T15:44:44.679289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-16T15:44:50.793 INFO:tasks.ceph.ceph_manager.ceph:PG 9.c is not active+clean 2024-09-16T15:44:50.793 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:44:44.693602+0000', 'last_change': '2024-09-16T15:44:44.693602+0000', 'last_active': '2024-09-16T15:44:44.679289+0000', 'last_peered': '2024-09-16T15:44:44.679289+0000', 'last_clean': '2024-09-16T15:44:44.679289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:44:44.693602+0000', 'last_undegraded': '2024-09-16T15:44:44.693602+0000', 'last_fullsized': '2024-09-16T15:44:44.693602+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:44:44.679289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:44:44.679289+0000', 'last_clean_scrub_stamp': '2024-09-16T15:44:44.679289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-16T15:44:50.794 INFO:tasks.ceph.ceph_manager.ceph:PG 9.4 is not active+clean 2024-09-16T15:44:50.794 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:44:44.692278+0000', 'last_change': '2024-09-16T15:44:44.692278+0000', 'last_active': '2024-09-16T15:44:44.679289+0000', 'last_peered': '2024-09-16T15:44:44.679289+0000', 'last_clean': '2024-09-16T15:44:44.679289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:44:44.692278+0000', 'last_undegraded': '2024-09-16T15:44:44.692278+0000', 'last_fullsized': '2024-09-16T15:44:44.692278+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:44:44.679289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:44:44.679289+0000', 'last_clean_scrub_stamp': '2024-09-16T15:44:44.679289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-16T15:44:50.794 INFO:tasks.ceph.ceph_manager.ceph:PG 9.1 is not active+clean 2024-09-16T15:44:50.794 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:44:44.691091+0000', 'last_change': '2024-09-16T15:44:44.691091+0000', 'last_active': '2024-09-16T15:44:44.679289+0000', 'last_peered': '2024-09-16T15:44:44.679289+0000', 'last_clean': '2024-09-16T15:44:44.679289+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:44:44.691091+0000', 'last_undegraded': '2024-09-16T15:44:44.691091+0000', 'last_fullsized': '2024-09-16T15:44:44.691091+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:44:44.679289+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:44:44.679289+0000', 'last_clean_scrub_stamp': '2024-09-16T15:44:44.679289+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-16T15:44:50.794 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T15:44:50.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.984+0000 7fbeb0a00640 1 -- 172.21.15.151:0/235671727 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbeac0d4940 msgr2=0x7fbeac160ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:50.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.984+0000 7fbeb0a00640 1 --2- 172.21.15.151:0/235671727 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbeac0d4940 0x7fbeac160ff0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fbe98009900 tx=0x7fbe9802a600 comp rx=0 tx=0).stop 2024-09-16T15:44:50.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.984+0000 7fbeb0a00640 1 -- 172.21.15.151:0/235671727 shutdown_connections 2024-09-16T15:44:50.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.984+0000 7fbeb0a00640 1 --2- 172.21.15.151:0/235671727 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbeac0d4940 0x7fbeac160ff0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:50.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.985+0000 7fbeb0a00640 1 -- 172.21.15.151:0/235671727 >> 172.21.15.151:0/235671727 conn(0x7fbeac0d3e20 msgr2=0x7fbeac0d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:50.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.985+0000 7fbeb0a00640 1 -- 172.21.15.151:0/235671727 shutdown_connections 2024-09-16T15:44:50.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.985+0000 7fbeb0a00640 1 -- 172.21.15.151:0/235671727 wait complete. 2024-09-16T15:44:50.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.986+0000 7fbeb0a00640 1 Processor -- start 2024-09-16T15:44:50.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.986+0000 7fbeb0a00640 1 -- start start 2024-09-16T15:44:50.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.987+0000 7fbeb0a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbeac0d07d0 0x7fbeac0b4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:50.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.987+0000 7fbeb0a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbeac0d4940 0x7fbeac15ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:50.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.987+0000 7fbeb0a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbeac161530 0x7fbeac1f7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:50.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.987+0000 7fbeb0a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbeac0b7b30 con 0x7fbeac0d07d0 2024-09-16T15:44:50.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.987+0000 7fbeb0a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbeac0b79b0 con 0x7fbeac161530 2024-09-16T15:44:50.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.987+0000 7fbeb0a00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fbeac0b7cb0 con 0x7fbeac0d4940 2024-09-16T15:44:50.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.987+0000 7fbeaa800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbeac0d07d0 0x7fbeac0b4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:50.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.987+0000 7fbeaa800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbeac0d07d0 0x7fbeac0b4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44318/0 (socket says 172.21.15.151:44318) 2024-09-16T15:44:50.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.987+0000 7fbeaa800640 1 -- 172.21.15.151:0/1360689463 learned_addr learned my addr 172.21.15.151:0/1360689463 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:50.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.988+0000 7fbea9e00640 1 --2- 172.21.15.151:0/1360689463 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbeac0d4940 0x7fbeac15ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:50.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.988+0000 7fbeab200640 1 --2- 172.21.15.151:0/1360689463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbeac161530 0x7fbeac1f7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:50.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.988+0000 7fbeaa800640 1 -- 172.21.15.151:0/1360689463 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbeac0d4940 msgr2=0x7fbeac15ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:50.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.988+0000 7fbeaa800640 1 --2- 172.21.15.151:0/1360689463 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbeac0d4940 0x7fbeac15ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:50.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.988+0000 7fbeaa800640 1 -- 172.21.15.151:0/1360689463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbeac161530 msgr2=0x7fbeac1f7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:50.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.988+0000 7fbeaa800640 1 --2- 172.21.15.151:0/1360689463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbeac161530 0x7fbeac1f7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:50.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.988+0000 7fbeaa800640 1 -- 172.21.15.151:0/1360689463 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeac15d710 con 0x7fbeac0d07d0 2024-09-16T15:44:50.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.988+0000 7fbeaa800640 1 --2- 172.21.15.151:0/1360689463 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbeac0d07d0 0x7fbeac0b4360 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c00c480 tx=0x7fbe9c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:50.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.988+0000 7fbea2a00640 1 -- 172.21.15.151:0/1360689463 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe9c0052d0 con 0x7fbeac0d07d0 2024-09-16T15:44:50.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.988+0000 7fbea2a00640 1 -- 172.21.15.151:0/1360689463 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbe9c005470 con 0x7fbeac0d07d0 2024-09-16T15:44:50.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.988+0000 7fbea2a00640 1 -- 172.21.15.151:0/1360689463 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe9c012870 con 0x7fbeac0d07d0 2024-09-16T15:44:50.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.988+0000 7fbeb0a00640 1 -- 172.21.15.151:0/1360689463 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbeac15d9f0 con 0x7fbeac0d07d0 2024-09-16T15:44:50.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.989+0000 7fbeb0a00640 1 -- 172.21.15.151:0/1360689463 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbeac15dfb0 con 0x7fbeac0d07d0 2024-09-16T15:44:50.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.990+0000 7fbeb0a00640 1 -- 172.21.15.151:0/1360689463 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbeac0b93c0 con 0x7fbeac0d07d0 2024-09-16T15:44:50.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.991+0000 7fbea2a00640 1 -- 172.21.15.151:0/1360689463 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 102103+0+0 (secure 0 0 0) 0x7fbe9c0035f0 con 0x7fbeac0d07d0 2024-09-16T15:44:50.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.992+0000 7fbea2a00640 1 --2- 172.21.15.151:0/1360689463 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbe8c079150 0x7fbe8c07b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:50.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.992+0000 7fbea2a00640 1 -- 172.21.15.151:0/1360689463 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7fbe9c098dd0 con 0x7fbeac0d07d0 2024-09-16T15:44:50.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.992+0000 7fbea9e00640 1 --2- 172.21.15.151:0/1360689463 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbe8c079150 0x7fbe8c07b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:50.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.993+0000 7fbea9e00640 1 --2- 172.21.15.151:0/1360689463 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbe8c079150 0x7fbe8c07b610 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fbe98033000 tx=0x7fbe98004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:50.996 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:50.995+0000 7fbea2a00640 1 -- 172.21.15.151:0/1360689463 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fbe9c062e90 con 0x7fbeac0d07d0 2024-09-16T15:44:51.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:51.247+0000 7fbeb0a00640 1 -- 172.21.15.151:0/1360689463 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fbeac0760b0 con 0x7fbeac0d07d0 2024-09-16T15:44:51.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:51.248+0000 7fbea2a00640 1 -- 172.21.15.151:0/1360689463 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1141 (secure 0 0 0) 0x7fbe9c062790 con 0x7fbeac0d07d0 2024-09-16T15:44:51.249 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:44:51.249 INFO:teuthology.orchestra.run.smithi151.stdout:{"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":354,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":62,"num_osds":8,"num_up_osds":8,"osd_up_since":1726501150,"num_in_osds":8,"osd_in_since":1726501140,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":89},{"state_name":"creating+peering","count":8}],"num_pgs":97,"num_pools":3,"num_objects":21,"data_bytes":592756,"bytes_used":364748800,"bytes_avail":772729364480,"bytes_total":773094113280,"inactive_pgs_ratio":0.082474224269390106,"write_bytes_sec":2222,"read_op_per_sec":0,"write_op_per_sec":6},"fsmap":{"epoch":36,"btime":"2024-09-16T15:44:48:723811+0000","id":4,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":4,"rank":0,"name":"c","status":"up:active","gid":5961}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-16T15:43:31.641478+0000","services":{}},"progress_events":{}} 2024-09-16T15:44:51.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:51.250+0000 7fbeb0a00640 1 -- 172.21.15.151:0/1360689463 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbe8c079150 msgr2=0x7fbe8c07b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:51.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:51.250+0000 7fbeb0a00640 1 --2- 172.21.15.151:0/1360689463 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbe8c079150 0x7fbe8c07b610 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fbe98033000 tx=0x7fbe98004730 comp rx=0 tx=0).stop 2024-09-16T15:44:51.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:51.250+0000 7fbeb0a00640 1 -- 172.21.15.151:0/1360689463 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbeac0d07d0 msgr2=0x7fbeac0b4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:51.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:51.250+0000 7fbeb0a00640 1 --2- 172.21.15.151:0/1360689463 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbeac0d07d0 0x7fbeac0b4360 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c00c480 tx=0x7fbe9c00c940 comp rx=0 tx=0).stop 2024-09-16T15:44:51.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:51.250+0000 7fbeb0a00640 1 -- 172.21.15.151:0/1360689463 shutdown_connections 2024-09-16T15:44:51.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:51.250+0000 7fbeb0a00640 1 --2- 172.21.15.151:0/1360689463 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbe8c079150 0x7fbe8c07b610 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:51.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:51.250+0000 7fbeb0a00640 1 --2- 172.21.15.151:0/1360689463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbeac161530 0x7fbeac1f7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:51.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:51.250+0000 7fbeb0a00640 1 --2- 172.21.15.151:0/1360689463 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbeac0d4940 0x7fbeac15ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:51.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:51.250+0000 7fbeb0a00640 1 --2- 172.21.15.151:0/1360689463 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbeac0d07d0 0x7fbeac0b4360 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:51.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:51.250+0000 7fbeb0a00640 1 -- 172.21.15.151:0/1360689463 >> 172.21.15.151:0/1360689463 conn(0x7fbeac0d3e20 msgr2=0x7fbeac06dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:51.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:51.250+0000 7fbeb0a00640 1 -- 172.21.15.151:0/1360689463 shutdown_connections 2024-09-16T15:44:51.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:51.251+0000 7fbeb0a00640 1 -- 172.21.15.151:0/1360689463 wait complete. 2024-09-16T15:44:51.267 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-16T15:44:51.267 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:44:51.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:51.424+0000 7f51b3200640 1 -- 172.21.15.151:0/3914768563 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f51b40d0540 msgr2=0x7f51b4074280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:51.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:51.424+0000 7f51b3200640 1 --2- 172.21.15.151:0/3914768563 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f51b40d0540 0x7f51b4074280 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f51b40cf8f0 tx=0x7f51ac02aa30 comp rx=0 tx=0).stop 2024-09-16T15:44:51.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:51.424+0000 7f51b3200640 1 -- 172.21.15.151:0/3914768563 shutdown_connections 2024-09-16T15:44:51.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:51.424+0000 7f51b3200640 1 --2- 172.21.15.151:0/3914768563 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f51b40d0540 0x7f51b4074280 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:51.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:51.424+0000 7f51b3200640 1 --2- 172.21.15.151:0/3914768563 >> v2:172.21.15.167:3300/0 conn(0x7f51b40725d0 0x7f51b4073d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:51.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:51.424+0000 7f51b3200640 1 --2- 172.21.15.151:0/3914768563 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51b415bc30 0x7f51b40b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:51.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:51.424+0000 7f51b3200640 1 -- 172.21.15.151:0/3914768563 >> 172.21.15.151:0/3914768563 conn(0x7f51b40cfa20 msgr2=0x7f51b40cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:51.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:51.424+0000 7f51b3200640 1 -- 172.21.15.151:0/3914768563 shutdown_connections 2024-09-16T15:44:51.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:51.424+0000 7f51b3200640 1 -- 172.21.15.151:0/3914768563 wait complete. 2024-09-16T15:44:51.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:51.426+0000 7f51b3200640 1 Processor -- start 2024-09-16T15:44:51.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:51.426+0000 7f51b3200640 1 -- start start 2024-09-16T15:44:51.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:51.427+0000 7f51b3200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51b40d0540 0x7f51b40cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:51.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:51.427+0000 7f51b3200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51b415bc30 0x7f51b415e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:51.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:51.427+0000 7f51b3200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f51b415ef20 0x7f51b40ca030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:51.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:51.427+0000 7f51b3200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f51b406f320 con 0x7f51b415ef20 2024-09-16T15:44:51.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:51.427+0000 7f51b3200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f51b406f4a0 con 0x7f51b415bc30 2024-09-16T15:44:51.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:51.427+0000 7f51b3200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f51b406f620 con 0x7f51b40d0540 2024-09-16T15:44:51.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:51.427+0000 7f51b2800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f51b415ef20 0x7f51b40ca030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:51.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:51.427+0000 7f51b2800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f51b415ef20 0x7f51b40ca030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44330/0 (socket says 172.21.15.151:44330) 2024-09-16T15:44:51.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:51.427+0000 7f51b2800640 1 -- 172.21.15.151:0/2786012933 learned_addr learned my addr 172.21.15.151:0/2786012933 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:51.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:51.427+0000 7f51a9400640 1 --2- 172.21.15.151:0/2786012933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51b415bc30 0x7f51b415e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:51.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:51.427+0000 7f51b1e00640 1 --2- 172.21.15.151:0/2786012933 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51b40d0540 0x7f51b40cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:51.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:51.428+0000 7f51b2800640 1 -- 172.21.15.151:0/2786012933 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51b40d0540 msgr2=0x7f51b40cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:51.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:51.428+0000 7f51b2800640 1 --2- 172.21.15.151:0/2786012933 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51b40d0540 0x7f51b40cd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:51.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:51.428+0000 7f51b2800640 1 -- 172.21.15.151:0/2786012933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51b415bc30 msgr2=0x7f51b415e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:51.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:51.428+0000 7f51b2800640 1 --2- 172.21.15.151:0/2786012933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51b415bc30 0x7f51b415e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:51.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:51.428+0000 7f51b2800640 1 -- 172.21.15.151:0/2786012933 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51b40c6c20 con 0x7f51b415ef20 2024-09-16T15:44:51.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:51.428+0000 7f51b2800640 1 --2- 172.21.15.151:0/2786012933 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f51b415ef20 0x7f51b40ca030 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f51ac02b020 tx=0x7f51ac003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:51.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:51.428+0000 7f51aaa00640 1 -- 172.21.15.151:0/2786012933 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51ac047070 con 0x7f51b415ef20 2024-09-16T15:44:51.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:51.428+0000 7f51aaa00640 1 -- 172.21.15.151:0/2786012933 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f51ac004250 con 0x7f51b415ef20 2024-09-16T15:44:51.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:51.428+0000 7f51b3200640 1 -- 172.21.15.151:0/2786012933 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51b40c6f00 con 0x7f51b415ef20 2024-09-16T15:44:51.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:51.428+0000 7f51aaa00640 1 -- 172.21.15.151:0/2786012933 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51ac033050 con 0x7f51b415ef20 2024-09-16T15:44:51.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:51.428+0000 7f51b3200640 1 -- 172.21.15.151:0/2786012933 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51b40cdb30 con 0x7f51b415ef20 2024-09-16T15:44:51.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:51.429+0000 7f51b3200640 1 -- 172.21.15.151:0/2786012933 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51b40b28f0 con 0x7f51b415ef20 2024-09-16T15:44:51.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:51.430+0000 7f51aaa00640 1 -- 172.21.15.151:0/2786012933 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 102103+0+0 (secure 0 0 0) 0x7f51ac04a070 con 0x7f51b415ef20 2024-09-16T15:44:51.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:51.431+0000 7f51aaa00640 1 --2- 172.21.15.151:0/2786012933 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5190079150 0x7f519007b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:51.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:51.432+0000 7f51aaa00640 1 -- 172.21.15.151:0/2786012933 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7f51ac0ba400 con 0x7f51b415ef20 2024-09-16T15:44:51.433 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:51.432+0000 7f51b1e00640 1 --2- 172.21.15.151:0/2786012933 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5190079150 0x7f519007b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:51.433 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:51.432+0000 7f51b1e00640 1 --2- 172.21.15.151:0/2786012933 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5190079150 0x7f519007b610 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f51a4004630 tx=0x7f51a400b2e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:51.437 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:51.436+0000 7f51aaa00640 1 -- 172.21.15.151:0/2786012933 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f51ac083e70 con 0x7f51b415ef20 2024-09-16T15:44:51.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:51.616+0000 7f51b3200640 1 -- 172.21.15.151:0/2786012933 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f51b415f430 con 0x7f5190079150 2024-09-16T15:44:51.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:51.621+0000 7f51aaa00640 1 -- 172.21.15.151:0/2786012933 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251283 (secure 0 0 0) 0x7f51a400b800 con 0x7f5190079150 2024-09-16T15:44:51.625 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:44:51.626 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-09-16T15:44:51.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:51.625+0000 7f51b3200640 1 -- 172.21.15.151:0/2786012933 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5190079150 msgr2=0x7f519007b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:51.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:51.625+0000 7f51b3200640 1 --2- 172.21.15.151:0/2786012933 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5190079150 0x7f519007b610 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f51a4004630 tx=0x7f51a400b2e0 comp rx=0 tx=0).stop 2024-09-16T15:44:51.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:51.625+0000 7f51b3200640 1 -- 172.21.15.151:0/2786012933 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f51b415ef20 msgr2=0x7f51b40ca030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:51.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:51.625+0000 7f51b3200640 1 --2- 172.21.15.151:0/2786012933 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f51b415ef20 0x7f51b40ca030 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f51ac02b020 tx=0x7f51ac003f10 comp rx=0 tx=0).stop 2024-09-16T15:44:51.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:51.625+0000 7f51b3200640 1 -- 172.21.15.151:0/2786012933 shutdown_connections 2024-09-16T15:44:51.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:51.625+0000 7f51b3200640 1 --2- 172.21.15.151:0/2786012933 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5190079150 0x7f519007b610 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:51.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:51.625+0000 7f51b3200640 1 --2- 172.21.15.151:0/2786012933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51b415bc30 0x7f51b415e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:51.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:51.625+0000 7f51b3200640 1 --2- 172.21.15.151:0/2786012933 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f51b415ef20 0x7f51b40ca030 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:51.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:51.625+0000 7f51b3200640 1 --2- 172.21.15.151:0/2786012933 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51b40d0540 0x7f51b40cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:51.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:51.625+0000 7f51b3200640 1 -- 172.21.15.151:0/2786012933 >> 172.21.15.151:0/2786012933 conn(0x7f51b40cfa20 msgr2=0x7f51b40cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:51.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:51.625+0000 7f51b3200640 1 -- 172.21.15.151:0/2786012933 shutdown_connections 2024-09-16T15:44:51.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:51.625+0000 7f51b3200640 1 -- 172.21.15.151:0/2786012933 wait complete. 2024-09-16T15:44:51.642 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":225,"stamp":"2024-09-16T15:44:49.670408+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592756,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":141,"num_write_kb":1641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":106,"ondisk_log_size":106,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":32,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":356200,"kb_used_data":6760,"kb_used_omap":12,"kb_used_meta":349235,"kb_avail":754618520,"statfs":{"total":773094113280,"available":772729364480,"internally_reserved":0,"allocated":6922240,"data_stored":3563759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12738,"internal_metadata":357617214},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2388,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.988477"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.732875+0000","last_change":"2024-09-16T15:44:45.728739+0000","last_active":"2024-09-16T15:44:45.732875+0000","last_peered":"2024-09-16T15:44:45.732875+0000","last_clean":"2024-09-16T15:44:45.732875+0000","last_became_active":"2024-09-16T15:44:45.727368+0000","last_became_peered":"2024-09-16T15:44:45.727368+0000","last_unstale":"2024-09-16T15:44:45.732875+0000","last_undegraded":"2024-09-16T15:44:45.732875+0000","last_fullsized":"2024-09-16T15:44:45.732875+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:34:55.972702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T15:44:44.692975+0000","last_change":"2024-09-16T15:44:44.692975+0000","last_active":"2024-09-16T15:44:44.679289+0000","last_peered":"2024-09-16T15:44:44.679289+0000","last_clean":"2024-09-16T15:44:44.679289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:44:44.692975+0000","last_undegraded":"2024-09-16T15:44:44.692975+0000","last_fullsized":"2024-09-16T15:44:44.692975+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.732389+0000","last_change":"2024-09-16T15:44:45.725780+0000","last_active":"2024-09-16T15:44:45.732389+0000","last_peered":"2024-09-16T15:44:45.732389+0000","last_clean":"2024-09-16T15:44:45.732389+0000","last_became_active":"2024-09-16T15:44:45.721333+0000","last_became_peered":"2024-09-16T15:44:45.721333+0000","last_unstale":"2024-09-16T15:44:45.732389+0000","last_undegraded":"2024-09-16T15:44:45.732389+0000","last_fullsized":"2024-09-16T15:44:45.732389+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:13:04.403633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.737448+0000","last_change":"2024-09-16T15:44:45.729388+0000","last_active":"2024-09-16T15:44:45.737448+0000","last_peered":"2024-09-16T15:44:45.737448+0000","last_clean":"2024-09-16T15:44:45.737448+0000","last_became_active":"2024-09-16T15:44:45.729119+0000","last_became_peered":"2024-09-16T15:44:45.729119+0000","last_unstale":"2024-09-16T15:44:45.737448+0000","last_undegraded":"2024-09-16T15:44:45.737448+0000","last_fullsized":"2024-09-16T15:44:45.737448+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:08:12.766873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.738580+0000","last_change":"2024-09-16T15:44:45.733586+0000","last_active":"2024-09-16T15:44:45.738580+0000","last_peered":"2024-09-16T15:44:45.738580+0000","last_clean":"2024-09-16T15:44:45.738580+0000","last_became_active":"2024-09-16T15:44:45.733144+0000","last_became_peered":"2024-09-16T15:44:45.733144+0000","last_unstale":"2024-09-16T15:44:45.738580+0000","last_undegraded":"2024-09-16T15:44:45.738580+0000","last_fullsized":"2024-09-16T15:44:45.738580+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:04:50.517949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.730726+0000","last_change":"2024-09-16T15:44:45.718049+0000","last_active":"2024-09-16T15:44:45.730726+0000","last_peered":"2024-09-16T15:44:45.730726+0000","last_clean":"2024-09-16T15:44:45.730726+0000","last_became_active":"2024-09-16T15:44:45.716815+0000","last_became_peered":"2024-09-16T15:44:45.716815+0000","last_unstale":"2024-09-16T15:44:45.730726+0000","last_undegraded":"2024-09-16T15:44:45.730726+0000","last_fullsized":"2024-09-16T15:44:45.730726+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:30:51.703628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.724140+0000","last_change":"2024-09-16T15:44:45.718523+0000","last_active":"2024-09-16T15:44:45.724140+0000","last_peered":"2024-09-16T15:44:45.724140+0000","last_clean":"2024-09-16T15:44:45.724140+0000","last_became_active":"2024-09-16T15:44:45.718208+0000","last_became_peered":"2024-09-16T15:44:45.718208+0000","last_unstale":"2024-09-16T15:44:45.724140+0000","last_undegraded":"2024-09-16T15:44:45.724140+0000","last_fullsized":"2024-09-16T15:44:45.724140+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:58:34.310843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.724066+0000","last_change":"2024-09-16T15:44:45.720647+0000","last_active":"2024-09-16T15:44:45.724066+0000","last_peered":"2024-09-16T15:44:45.724066+0000","last_clean":"2024-09-16T15:44:45.724066+0000","last_became_active":"2024-09-16T15:44:45.720351+0000","last_became_peered":"2024-09-16T15:44:45.720351+0000","last_unstale":"2024-09-16T15:44:45.724066+0000","last_undegraded":"2024-09-16T15:44:45.724066+0000","last_fullsized":"2024-09-16T15:44:45.724066+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:44:06.889108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.734233+0000","last_change":"2024-09-16T15:44:45.732987+0000","last_active":"2024-09-16T15:44:45.734233+0000","last_peered":"2024-09-16T15:44:45.734233+0000","last_clean":"2024-09-16T15:44:45.734233+0000","last_became_active":"2024-09-16T15:44:45.732680+0000","last_became_peered":"2024-09-16T15:44:45.732680+0000","last_unstale":"2024-09-16T15:44:45.734233+0000","last_undegraded":"2024-09-16T15:44:45.734233+0000","last_fullsized":"2024-09-16T15:44:45.734233+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:21:56.720871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.730690+0000","last_change":"2024-09-16T15:44:45.719120+0000","last_active":"2024-09-16T15:44:45.730690+0000","last_peered":"2024-09-16T15:44:45.730690+0000","last_clean":"2024-09-16T15:44:45.730690+0000","last_became_active":"2024-09-16T15:44:45.718746+0000","last_became_peered":"2024-09-16T15:44:45.718746+0000","last_unstale":"2024-09-16T15:44:45.730690+0000","last_undegraded":"2024-09-16T15:44:45.730690+0000","last_fullsized":"2024-09-16T15:44:45.730690+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:13:46.973489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.726293+0000","last_change":"2024-09-16T15:44:45.718730+0000","last_active":"2024-09-16T15:44:45.726293+0000","last_peered":"2024-09-16T15:44:45.726293+0000","last_clean":"2024-09-16T15:44:45.726293+0000","last_became_active":"2024-09-16T15:44:45.718442+0000","last_became_peered":"2024-09-16T15:44:45.718442+0000","last_unstale":"2024-09-16T15:44:45.726293+0000","last_undegraded":"2024-09-16T15:44:45.726293+0000","last_fullsized":"2024-09-16T15:44:45.726293+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:13:04.403633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.720994+0000","last_change":"2024-09-16T15:44:45.717536+0000","last_active":"2024-09-16T15:44:45.720994+0000","last_peered":"2024-09-16T15:44:45.720994+0000","last_clean":"2024-09-16T15:44:45.720994+0000","last_became_active":"2024-09-16T15:44:45.717226+0000","last_became_peered":"2024-09-16T15:44:45.717226+0000","last_unstale":"2024-09-16T15:44:45.720994+0000","last_undegraded":"2024-09-16T15:44:45.720994+0000","last_fullsized":"2024-09-16T15:44:45.720994+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:46:33.994315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.719703+0000","last_change":"2024-09-16T15:44:45.711525+0000","last_active":"2024-09-16T15:44:45.719703+0000","last_peered":"2024-09-16T15:44:45.719703+0000","last_clean":"2024-09-16T15:44:45.719703+0000","last_became_active":"2024-09-16T15:44:45.711195+0000","last_became_peered":"2024-09-16T15:44:45.711195+0000","last_unstale":"2024-09-16T15:44:45.719703+0000","last_undegraded":"2024-09-16T15:44:45.719703+0000","last_fullsized":"2024-09-16T15:44:45.719703+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:42:47.930299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.732183+0000","last_change":"2024-09-16T15:44:45.726781+0000","last_active":"2024-09-16T15:44:45.732183+0000","last_peered":"2024-09-16T15:44:45.732183+0000","last_clean":"2024-09-16T15:44:45.732183+0000","last_became_active":"2024-09-16T15:44:45.726045+0000","last_became_peered":"2024-09-16T15:44:45.726045+0000","last_unstale":"2024-09-16T15:44:45.732183+0000","last_undegraded":"2024-09-16T15:44:45.732183+0000","last_fullsized":"2024-09-16T15:44:45.732183+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:36:30.549487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.719163+0000","last_change":"2024-09-16T15:44:45.716384+0000","last_active":"2024-09-16T15:44:45.719163+0000","last_peered":"2024-09-16T15:44:45.719163+0000","last_clean":"2024-09-16T15:44:45.719163+0000","last_became_active":"2024-09-16T15:44:45.716101+0000","last_became_peered":"2024-09-16T15:44:45.716101+0000","last_unstale":"2024-09-16T15:44:45.719163+0000","last_undegraded":"2024-09-16T15:44:45.719163+0000","last_fullsized":"2024-09-16T15:44:45.719163+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:21:51.586704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.730872+0000","last_change":"2024-09-16T15:44:45.721590+0000","last_active":"2024-09-16T15:44:45.730872+0000","last_peered":"2024-09-16T15:44:45.730872+0000","last_clean":"2024-09-16T15:44:45.730872+0000","last_became_active":"2024-09-16T15:44:45.721343+0000","last_became_peered":"2024-09-16T15:44:45.721343+0000","last_unstale":"2024-09-16T15:44:45.730872+0000","last_undegraded":"2024-09-16T15:44:45.730872+0000","last_fullsized":"2024-09-16T15:44:45.730872+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:44:28.021891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.730936+0000","last_change":"2024-09-16T15:44:45.722006+0000","last_active":"2024-09-16T15:44:45.730936+0000","last_peered":"2024-09-16T15:44:45.730936+0000","last_clean":"2024-09-16T15:44:45.730936+0000","last_became_active":"2024-09-16T15:44:45.721714+0000","last_became_peered":"2024-09-16T15:44:45.721714+0000","last_unstale":"2024-09-16T15:44:45.730936+0000","last_undegraded":"2024-09-16T15:44:45.730936+0000","last_fullsized":"2024-09-16T15:44:45.730936+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:53:59.005172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.736449+0000","last_change":"2024-09-16T15:44:45.732870+0000","last_active":"2024-09-16T15:44:45.736449+0000","last_peered":"2024-09-16T15:44:45.736449+0000","last_clean":"2024-09-16T15:44:45.736449+0000","last_became_active":"2024-09-16T15:44:45.732609+0000","last_became_peered":"2024-09-16T15:44:45.732609+0000","last_unstale":"2024-09-16T15:44:45.736449+0000","last_undegraded":"2024-09-16T15:44:45.736449+0000","last_fullsized":"2024-09-16T15:44:45.736449+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:44:16.275512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T15:44:44.691707+0000","last_change":"2024-09-16T15:44:44.691707+0000","last_active":"2024-09-16T15:44:44.679289+0000","last_peered":"2024-09-16T15:44:44.679289+0000","last_clean":"2024-09-16T15:44:44.679289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:44:44.691707+0000","last_undegraded":"2024-09-16T15:44:44.691707+0000","last_fullsized":"2024-09-16T15:44:44.691707+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.734176+0000","last_change":"2024-09-16T15:44:45.731373+0000","last_active":"2024-09-16T15:44:45.734176+0000","last_peered":"2024-09-16T15:44:45.734176+0000","last_clean":"2024-09-16T15:44:45.734176+0000","last_became_active":"2024-09-16T15:44:45.731068+0000","last_became_peered":"2024-09-16T15:44:45.731068+0000","last_unstale":"2024-09-16T15:44:45.734176+0000","last_undegraded":"2024-09-16T15:44:45.734176+0000","last_fullsized":"2024-09-16T15:44:45.734176+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:51:57.452114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.724617+0000","last_change":"2024-09-16T15:44:45.719386+0000","last_active":"2024-09-16T15:44:45.724617+0000","last_peered":"2024-09-16T15:44:45.724617+0000","last_clean":"2024-09-16T15:44:45.724617+0000","last_became_active":"2024-09-16T15:44:45.719154+0000","last_became_peered":"2024-09-16T15:44:45.719154+0000","last_unstale":"2024-09-16T15:44:45.724617+0000","last_undegraded":"2024-09-16T15:44:45.724617+0000","last_fullsized":"2024-09-16T15:44:45.724617+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:11:14.213089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.723225+0000","last_change":"2024-09-16T15:44:45.718300+0000","last_active":"2024-09-16T15:44:45.723225+0000","last_peered":"2024-09-16T15:44:45.723225+0000","last_clean":"2024-09-16T15:44:45.723225+0000","last_became_active":"2024-09-16T15:44:45.717725+0000","last_became_peered":"2024-09-16T15:44:45.717725+0000","last_unstale":"2024-09-16T15:44:45.723225+0000","last_undegraded":"2024-09-16T15:44:45.723225+0000","last_fullsized":"2024-09-16T15:44:45.723225+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:03:17.351052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.721190+0000","last_change":"2024-09-16T15:44:45.716075+0000","last_active":"2024-09-16T15:44:45.721190+0000","last_peered":"2024-09-16T15:44:45.721190+0000","last_clean":"2024-09-16T15:44:45.721190+0000","last_became_active":"2024-09-16T15:44:45.715816+0000","last_became_peered":"2024-09-16T15:44:45.715816+0000","last_unstale":"2024-09-16T15:44:45.721190+0000","last_undegraded":"2024-09-16T15:44:45.721190+0000","last_fullsized":"2024-09-16T15:44:45.721190+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:38:03.977000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.725164+0000","last_change":"2024-09-16T15:44:45.721703+0000","last_active":"2024-09-16T15:44:45.725164+0000","last_peered":"2024-09-16T15:44:45.725164+0000","last_clean":"2024-09-16T15:44:45.725164+0000","last_became_active":"2024-09-16T15:44:45.721406+0000","last_became_peered":"2024-09-16T15:44:45.721406+0000","last_unstale":"2024-09-16T15:44:45.725164+0000","last_undegraded":"2024-09-16T15:44:45.725164+0000","last_fullsized":"2024-09-16T15:44:45.725164+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:42:26.902672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T15:44:44.691255+0000","last_change":"2024-09-16T15:44:44.691255+0000","last_active":"2024-09-16T15:44:44.679289+0000","last_peered":"2024-09-16T15:44:44.679289+0000","last_clean":"2024-09-16T15:44:44.679289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:44:44.691255+0000","last_undegraded":"2024-09-16T15:44:44.691255+0000","last_fullsized":"2024-09-16T15:44:44.691255+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.725345+0000","last_change":"2024-09-16T15:44:45.722538+0000","last_active":"2024-09-16T15:44:45.725345+0000","last_peered":"2024-09-16T15:44:45.725345+0000","last_clean":"2024-09-16T15:44:45.725345+0000","last_became_active":"2024-09-16T15:44:45.722286+0000","last_became_peered":"2024-09-16T15:44:45.722286+0000","last_unstale":"2024-09-16T15:44:45.725345+0000","last_undegraded":"2024-09-16T15:44:45.725345+0000","last_fullsized":"2024-09-16T15:44:45.725345+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:45:58.209870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T15:44:44.691255+0000","last_change":"2024-09-16T15:44:44.691255+0000","last_active":"2024-09-16T15:44:44.679289+0000","last_peered":"2024-09-16T15:44:44.679289+0000","last_clean":"2024-09-16T15:44:44.679289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:44:44.691255+0000","last_undegraded":"2024-09-16T15:44:44.691255+0000","last_fullsized":"2024-09-16T15:44:44.691255+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.722818+0000","last_change":"2024-09-16T15:44:45.707684+0000","last_active":"2024-09-16T15:44:45.722818+0000","last_peered":"2024-09-16T15:44:45.722818+0000","last_clean":"2024-09-16T15:44:45.722818+0000","last_became_active":"2024-09-16T15:44:45.707370+0000","last_became_peered":"2024-09-16T15:44:45.707370+0000","last_unstale":"2024-09-16T15:44:45.722818+0000","last_undegraded":"2024-09-16T15:44:45.722818+0000","last_fullsized":"2024-09-16T15:44:45.722818+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:32:11.331276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.724083+0000","last_change":"2024-09-16T15:44:45.712553+0000","last_active":"2024-09-16T15:44:45.724083+0000","last_peered":"2024-09-16T15:44:45.724083+0000","last_clean":"2024-09-16T15:44:45.724083+0000","last_became_active":"2024-09-16T15:44:45.711955+0000","last_became_peered":"2024-09-16T15:44:45.711955+0000","last_unstale":"2024-09-16T15:44:45.724083+0000","last_undegraded":"2024-09-16T15:44:45.724083+0000","last_fullsized":"2024-09-16T15:44:45.724083+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:33:23.442599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.714841+0000","last_change":"2024-09-16T15:44:45.711869+0000","last_active":"2024-09-16T15:44:45.714841+0000","last_peered":"2024-09-16T15:44:45.714841+0000","last_clean":"2024-09-16T15:44:45.714841+0000","last_became_active":"2024-09-16T15:44:45.711534+0000","last_became_peered":"2024-09-16T15:44:45.711534+0000","last_unstale":"2024-09-16T15:44:45.714841+0000","last_undegraded":"2024-09-16T15:44:45.714841+0000","last_fullsized":"2024-09-16T15:44:45.714841+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:31:16.141860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T15:44:44.691565+0000","last_change":"2024-09-16T15:44:44.691565+0000","last_active":"2024-09-16T15:44:44.679289+0000","last_peered":"2024-09-16T15:44:44.679289+0000","last_clean":"2024-09-16T15:44:44.679289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:44:44.691565+0000","last_undegraded":"2024-09-16T15:44:44.691565+0000","last_fullsized":"2024-09-16T15:44:44.691565+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.729551+0000","last_change":"2024-09-16T15:44:45.726051+0000","last_active":"2024-09-16T15:44:45.729551+0000","last_peered":"2024-09-16T15:44:45.729551+0000","last_clean":"2024-09-16T15:44:45.729551+0000","last_became_active":"2024-09-16T15:44:45.720897+0000","last_became_peered":"2024-09-16T15:44:45.720897+0000","last_unstale":"2024-09-16T15:44:45.729551+0000","last_undegraded":"2024-09-16T15:44:45.729551+0000","last_fullsized":"2024-09-16T15:44:45.729551+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:36:30.549487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.732821+0000","last_change":"2024-09-16T15:44:45.724124+0000","last_active":"2024-09-16T15:44:45.732821+0000","last_peered":"2024-09-16T15:44:45.732821+0000","last_clean":"2024-09-16T15:44:45.732821+0000","last_became_active":"2024-09-16T15:44:45.721712+0000","last_became_peered":"2024-09-16T15:44:45.721712+0000","last_unstale":"2024-09-16T15:44:45.732821+0000","last_undegraded":"2024-09-16T15:44:45.732821+0000","last_fullsized":"2024-09-16T15:44:45.732821+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:15:22.807776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"61'4","reported_seq":17,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.732008+0000","last_change":"2024-09-16T15:44:44.700510+0000","last_active":"2024-09-16T15:44:45.732008+0000","last_peered":"2024-09-16T15:44:45.732008+0000","last_clean":"2024-09-16T15:44:45.732008+0000","last_became_active":"2024-09-16T15:44:44.697806+0000","last_became_peered":"2024-09-16T15:44:44.697806+0000","last_unstale":"2024-09-16T15:44:45.732008+0000","last_undegraded":"2024-09-16T15:44:45.732008+0000","last_fullsized":"2024-09-16T15:44:45.732008+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:50:52.878214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.729071+0000","last_change":"2024-09-16T15:44:45.715988+0000","last_active":"2024-09-16T15:44:45.729071+0000","last_peered":"2024-09-16T15:44:45.729071+0000","last_clean":"2024-09-16T15:44:45.729071+0000","last_became_active":"2024-09-16T15:44:45.715735+0000","last_became_peered":"2024-09-16T15:44:45.715735+0000","last_unstale":"2024-09-16T15:44:45.729071+0000","last_undegraded":"2024-09-16T15:44:45.729071+0000","last_fullsized":"2024-09-16T15:44:45.729071+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:39:12.784333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T15:44:44.710442+0000","last_change":"2024-09-16T15:44:44.708252+0000","last_active":"2024-09-16T15:44:44.710442+0000","last_peered":"2024-09-16T15:44:44.710442+0000","last_clean":"2024-09-16T15:44:44.710442+0000","last_became_active":"2024-09-16T15:44:44.707995+0000","last_became_peered":"2024-09-16T15:44:44.707995+0000","last_unstale":"2024-09-16T15:44:44.710442+0000","last_undegraded":"2024-09-16T15:44:44.710442+0000","last_fullsized":"2024-09-16T15:44:44.710442+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:26:28.068533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.734118+0000","last_change":"2024-09-16T15:44:45.729683+0000","last_active":"2024-09-16T15:44:45.734118+0000","last_peered":"2024-09-16T15:44:45.734118+0000","last_clean":"2024-09-16T15:44:45.734118+0000","last_became_active":"2024-09-16T15:44:45.729258+0000","last_became_peered":"2024-09-16T15:44:45.729258+0000","last_unstale":"2024-09-16T15:44:45.734118+0000","last_undegraded":"2024-09-16T15:44:45.734118+0000","last_fullsized":"2024-09-16T15:44:45.734118+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:34:41.864039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"61'2","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.739158+0000","last_change":"2024-09-16T15:44:44.705295+0000","last_active":"2024-09-16T15:44:45.739158+0000","last_peered":"2024-09-16T15:44:45.739158+0000","last_clean":"2024-09-16T15:44:45.739158+0000","last_became_active":"2024-09-16T15:44:44.704913+0000","last_became_peered":"2024-09-16T15:44:44.704913+0000","last_unstale":"2024-09-16T15:44:45.739158+0000","last_undegraded":"2024-09-16T15:44:45.739158+0000","last_fullsized":"2024-09-16T15:44:45.739158+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:33:20.871493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.732416+0000","last_change":"2024-09-16T15:44:45.724296+0000","last_active":"2024-09-16T15:44:45.732416+0000","last_peered":"2024-09-16T15:44:45.732416+0000","last_clean":"2024-09-16T15:44:45.732416+0000","last_became_active":"2024-09-16T15:44:45.722552+0000","last_became_peered":"2024-09-16T15:44:45.722552+0000","last_unstale":"2024-09-16T15:44:45.732416+0000","last_undegraded":"2024-09-16T15:44:45.732416+0000","last_fullsized":"2024-09-16T15:44:45.732416+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:35:05.861042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"61'2","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.737888+0000","last_change":"2024-09-16T15:44:44.701949+0000","last_active":"2024-09-16T15:44:45.737888+0000","last_peered":"2024-09-16T15:44:45.737888+0000","last_clean":"2024-09-16T15:44:45.737888+0000","last_became_active":"2024-09-16T15:44:44.701522+0000","last_became_peered":"2024-09-16T15:44:44.701522+0000","last_unstale":"2024-09-16T15:44:45.737888+0000","last_undegraded":"2024-09-16T15:44:45.737888+0000","last_fullsized":"2024-09-16T15:44:45.737888+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:18:22.181140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.723888+0000","last_change":"2024-09-16T15:44:45.712376+0000","last_active":"2024-09-16T15:44:45.723888+0000","last_peered":"2024-09-16T15:44:45.723888+0000","last_clean":"2024-09-16T15:44:45.723888+0000","last_became_active":"2024-09-16T15:44:45.712045+0000","last_became_peered":"2024-09-16T15:44:45.712045+0000","last_unstale":"2024-09-16T15:44:45.723888+0000","last_undegraded":"2024-09-16T15:44:45.723888+0000","last_fullsized":"2024-09-16T15:44:45.723888+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:20:02.525231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"61'1","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.741440+0000","last_change":"2024-09-16T15:44:44.705164+0000","last_active":"2024-09-16T15:44:45.741440+0000","last_peered":"2024-09-16T15:44:45.741440+0000","last_clean":"2024-09-16T15:44:45.741440+0000","last_became_active":"2024-09-16T15:44:44.704525+0000","last_became_peered":"2024-09-16T15:44:44.704525+0000","last_unstale":"2024-09-16T15:44:45.741440+0000","last_undegraded":"2024-09-16T15:44:45.741440+0000","last_fullsized":"2024-09-16T15:44:45.741440+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:37:15.032653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.724052+0000","last_change":"2024-09-16T15:44:45.719896+0000","last_active":"2024-09-16T15:44:45.724052+0000","last_peered":"2024-09-16T15:44:45.724052+0000","last_clean":"2024-09-16T15:44:45.724052+0000","last_became_active":"2024-09-16T15:44:45.718945+0000","last_became_peered":"2024-09-16T15:44:45.718945+0000","last_unstale":"2024-09-16T15:44:45.724052+0000","last_undegraded":"2024-09-16T15:44:45.724052+0000","last_fullsized":"2024-09-16T15:44:45.724052+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:31:16.283923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"61'1","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.736043+0000","last_change":"2024-09-16T15:44:44.710842+0000","last_active":"2024-09-16T15:44:45.736043+0000","last_peered":"2024-09-16T15:44:45.736043+0000","last_clean":"2024-09-16T15:44:45.736043+0000","last_became_active":"2024-09-16T15:44:44.710529+0000","last_became_peered":"2024-09-16T15:44:44.710529+0000","last_unstale":"2024-09-16T15:44:45.736043+0000","last_undegraded":"2024-09-16T15:44:45.736043+0000","last_fullsized":"2024-09-16T15:44:45.736043+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:36:04.434843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.716864+0000","last_change":"2024-09-16T15:44:45.708591+0000","last_active":"2024-09-16T15:44:45.716864+0000","last_peered":"2024-09-16T15:44:45.716864+0000","last_clean":"2024-09-16T15:44:45.716864+0000","last_became_active":"2024-09-16T15:44:45.708350+0000","last_became_peered":"2024-09-16T15:44:45.708350+0000","last_unstale":"2024-09-16T15:44:45.716864+0000","last_undegraded":"2024-09-16T15:44:45.716864+0000","last_fullsized":"2024-09-16T15:44:45.716864+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:28:57.953253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.704237+0000","last_change":"2024-09-16T15:44:44.697711+0000","last_active":"2024-09-16T15:44:45.704237+0000","last_peered":"2024-09-16T15:44:45.704237+0000","last_clean":"2024-09-16T15:44:45.704237+0000","last_became_active":"2024-09-16T15:44:44.697357+0000","last_became_peered":"2024-09-16T15:44:44.697357+0000","last_unstale":"2024-09-16T15:44:45.704237+0000","last_undegraded":"2024-09-16T15:44:45.704237+0000","last_fullsized":"2024-09-16T15:44:45.704237+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:20:52.337348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.730799+0000","last_change":"2024-09-16T15:44:45.719359+0000","last_active":"2024-09-16T15:44:45.730799+0000","last_peered":"2024-09-16T15:44:45.730799+0000","last_clean":"2024-09-16T15:44:45.730799+0000","last_became_active":"2024-09-16T15:44:45.719093+0000","last_became_peered":"2024-09-16T15:44:45.719093+0000","last_unstale":"2024-09-16T15:44:45.730799+0000","last_undegraded":"2024-09-16T15:44:45.730799+0000","last_fullsized":"2024-09-16T15:44:45.730799+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:02:03.820471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.703161+0000","last_change":"2024-09-16T15:44:44.694298+0000","last_active":"2024-09-16T15:44:45.703161+0000","last_peered":"2024-09-16T15:44:45.703161+0000","last_clean":"2024-09-16T15:44:45.703161+0000","last_became_active":"2024-09-16T15:44:44.694052+0000","last_became_peered":"2024-09-16T15:44:44.694052+0000","last_unstale":"2024-09-16T15:44:45.703161+0000","last_undegraded":"2024-09-16T15:44:45.703161+0000","last_fullsized":"2024-09-16T15:44:45.703161+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:25:26.438122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.715275+0000","last_change":"2024-09-16T15:44:45.713168+0000","last_active":"2024-09-16T15:44:45.715275+0000","last_peered":"2024-09-16T15:44:45.715275+0000","last_clean":"2024-09-16T15:44:45.715275+0000","last_became_active":"2024-09-16T15:44:45.711840+0000","last_became_peered":"2024-09-16T15:44:45.711840+0000","last_unstale":"2024-09-16T15:44:45.715275+0000","last_undegraded":"2024-09-16T15:44:45.715275+0000","last_fullsized":"2024-09-16T15:44:45.715275+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:21:48.643444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.709187+0000","last_change":"2024-09-16T15:44:44.703702+0000","last_active":"2024-09-16T15:44:45.709187+0000","last_peered":"2024-09-16T15:44:45.709187+0000","last_clean":"2024-09-16T15:44:45.709187+0000","last_became_active":"2024-09-16T15:44:44.703424+0000","last_became_peered":"2024-09-16T15:44:44.703424+0000","last_unstale":"2024-09-16T15:44:45.709187+0000","last_undegraded":"2024-09-16T15:44:45.709187+0000","last_fullsized":"2024-09-16T15:44:45.709187+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:24:00.275655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.734732+0000","last_change":"2024-09-16T15:44:45.726605+0000","last_active":"2024-09-16T15:44:45.734732+0000","last_peered":"2024-09-16T15:44:45.734732+0000","last_clean":"2024-09-16T15:44:45.734732+0000","last_became_active":"2024-09-16T15:44:45.726368+0000","last_became_peered":"2024-09-16T15:44:45.726368+0000","last_unstale":"2024-09-16T15:44:45.734732+0000","last_undegraded":"2024-09-16T15:44:45.734732+0000","last_fullsized":"2024-09-16T15:44:45.734732+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:31:16.283923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.704573+0000","last_change":"2024-09-16T15:44:44.695923+0000","last_active":"2024-09-16T15:44:45.704573+0000","last_peered":"2024-09-16T15:44:45.704573+0000","last_clean":"2024-09-16T15:44:45.704573+0000","last_became_active":"2024-09-16T15:44:44.695618+0000","last_became_peered":"2024-09-16T15:44:44.695618+0000","last_unstale":"2024-09-16T15:44:45.704573+0000","last_undegraded":"2024-09-16T15:44:45.704573+0000","last_fullsized":"2024-09-16T15:44:45.704573+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:41:35.721501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.728654+0000","last_change":"2024-09-16T15:44:45.723911+0000","last_active":"2024-09-16T15:44:45.728654+0000","last_peered":"2024-09-16T15:44:45.728654+0000","last_clean":"2024-09-16T15:44:45.728654+0000","last_became_active":"2024-09-16T15:44:45.723557+0000","last_became_peered":"2024-09-16T15:44:45.723557+0000","last_unstale":"2024-09-16T15:44:45.728654+0000","last_undegraded":"2024-09-16T15:44:45.728654+0000","last_fullsized":"2024-09-16T15:44:45.728654+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:18:23.183829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"61'3","reported_seq":16,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.737703+0000","last_change":"2024-09-16T15:44:44.700702+0000","last_active":"2024-09-16T15:44:45.737703+0000","last_peered":"2024-09-16T15:44:45.737703+0000","last_clean":"2024-09-16T15:44:45.737703+0000","last_became_active":"2024-09-16T15:44:44.700438+0000","last_became_peered":"2024-09-16T15:44:44.700438+0000","last_unstale":"2024-09-16T15:44:45.737703+0000","last_undegraded":"2024-09-16T15:44:45.737703+0000","last_fullsized":"2024-09-16T15:44:45.737703+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:46:15.633475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.713366+0000","last_change":"2024-09-16T15:44:45.709431+0000","last_active":"2024-09-16T15:44:45.713366+0000","last_peered":"2024-09-16T15:44:45.713366+0000","last_clean":"2024-09-16T15:44:45.713366+0000","last_became_active":"2024-09-16T15:44:45.709179+0000","last_became_peered":"2024-09-16T15:44:45.709179+0000","last_unstale":"2024-09-16T15:44:45.713366+0000","last_undegraded":"2024-09-16T15:44:45.713366+0000","last_fullsized":"2024-09-16T15:44:45.713366+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:34:54.933570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"61'6","reported_seq":19,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.743392+0000","last_change":"2024-09-16T15:44:44.691387+0000","last_active":"2024-09-16T15:44:45.743392+0000","last_peered":"2024-09-16T15:44:45.743392+0000","last_clean":"2024-09-16T15:44:45.743392+0000","last_became_active":"2024-09-16T15:44:44.691144+0000","last_became_peered":"2024-09-16T15:44:44.691144+0000","last_unstale":"2024-09-16T15:44:45.743392+0000","last_undegraded":"2024-09-16T15:44:45.743392+0000","last_fullsized":"2024-09-16T15:44:45.743392+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:09:54.112439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.723072+0000","last_change":"2024-09-16T15:44:45.717715+0000","last_active":"2024-09-16T15:44:45.723072+0000","last_peered":"2024-09-16T15:44:45.723072+0000","last_clean":"2024-09-16T15:44:45.723072+0000","last_became_active":"2024-09-16T15:44:45.717366+0000","last_became_peered":"2024-09-16T15:44:45.717366+0000","last_unstale":"2024-09-16T15:44:45.723072+0000","last_undegraded":"2024-09-16T15:44:45.723072+0000","last_fullsized":"2024-09-16T15:44:45.723072+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:12:07.274566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"61'2","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.730407+0000","last_change":"2024-09-16T15:44:44.702026+0000","last_active":"2024-09-16T15:44:45.730407+0000","last_peered":"2024-09-16T15:44:45.730407+0000","last_clean":"2024-09-16T15:44:45.730407+0000","last_became_active":"2024-09-16T15:44:44.701701+0000","last_became_peered":"2024-09-16T15:44:44.701701+0000","last_unstale":"2024-09-16T15:44:45.730407+0000","last_undegraded":"2024-09-16T15:44:45.730407+0000","last_fullsized":"2024-09-16T15:44:45.730407+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:30:25.597599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.737862+0000","last_change":"2024-09-16T15:44:45.734880+0000","last_active":"2024-09-16T15:44:45.737862+0000","last_peered":"2024-09-16T15:44:45.737862+0000","last_clean":"2024-09-16T15:44:45.737862+0000","last_became_active":"2024-09-16T15:44:45.734616+0000","last_became_peered":"2024-09-16T15:44:45.734616+0000","last_unstale":"2024-09-16T15:44:45.737862+0000","last_undegraded":"2024-09-16T15:44:45.737862+0000","last_fullsized":"2024-09-16T15:44:45.737862+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:19:00.003404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.697639+0000","last_change":"2024-09-16T15:44:44.701584+0000","last_active":"2024-09-16T15:44:45.697639+0000","last_peered":"2024-09-16T15:44:45.697639+0000","last_clean":"2024-09-16T15:44:45.697639+0000","last_became_active":"2024-09-16T15:44:44.701327+0000","last_became_peered":"2024-09-16T15:44:44.701327+0000","last_unstale":"2024-09-16T15:44:45.697639+0000","last_undegraded":"2024-09-16T15:44:45.697639+0000","last_fullsized":"2024-09-16T15:44:45.697639+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:44:05.886418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.720386+0000","last_change":"2024-09-16T15:44:45.717514+0000","last_active":"2024-09-16T15:44:45.720386+0000","last_peered":"2024-09-16T15:44:45.720386+0000","last_clean":"2024-09-16T15:44:45.720386+0000","last_became_active":"2024-09-16T15:44:45.717270+0000","last_became_peered":"2024-09-16T15:44:45.717270+0000","last_unstale":"2024-09-16T15:44:45.720386+0000","last_undegraded":"2024-09-16T15:44:45.720386+0000","last_fullsized":"2024-09-16T15:44:45.720386+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:12:47.737828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.698166+0000","last_change":"2024-09-16T15:44:44.702562+0000","last_active":"2024-09-16T15:44:45.698166+0000","last_peered":"2024-09-16T15:44:45.698166+0000","last_clean":"2024-09-16T15:44:45.698166+0000","last_became_active":"2024-09-16T15:44:44.702282+0000","last_became_peered":"2024-09-16T15:44:44.702282+0000","last_unstale":"2024-09-16T15:44:45.698166+0000","last_undegraded":"2024-09-16T15:44:45.698166+0000","last_fullsized":"2024-09-16T15:44:45.698166+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:34:40.861349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.723514+0000","last_change":"2024-09-16T15:44:45.717730+0000","last_active":"2024-09-16T15:44:45.723514+0000","last_peered":"2024-09-16T15:44:45.723514+0000","last_clean":"2024-09-16T15:44:45.723514+0000","last_became_active":"2024-09-16T15:44:45.717456+0000","last_became_peered":"2024-09-16T15:44:45.717456+0000","last_unstale":"2024-09-16T15:44:45.723514+0000","last_undegraded":"2024-09-16T15:44:45.723514+0000","last_fullsized":"2024-09-16T15:44:45.723514+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:09:14.114379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"61'8","reported_seq":21,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.740384+0000","last_change":"2024-09-16T15:44:44.700378+0000","last_active":"2024-09-16T15:44:45.740384+0000","last_peered":"2024-09-16T15:44:45.740384+0000","last_clean":"2024-09-16T15:44:45.740384+0000","last_became_active":"2024-09-16T15:44:44.700114+0000","last_became_peered":"2024-09-16T15:44:44.700114+0000","last_unstale":"2024-09-16T15:44:45.740384+0000","last_undegraded":"2024-09-16T15:44:45.740384+0000","last_fullsized":"2024-09-16T15:44:45.740384+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:46:16.051799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.720485+0000","last_change":"2024-09-16T15:44:45.716464+0000","last_active":"2024-09-16T15:44:45.720485+0000","last_peered":"2024-09-16T15:44:45.720485+0000","last_clean":"2024-09-16T15:44:45.720485+0000","last_became_active":"2024-09-16T15:44:45.716140+0000","last_became_peered":"2024-09-16T15:44:45.716140+0000","last_unstale":"2024-09-16T15:44:45.720485+0000","last_undegraded":"2024-09-16T15:44:45.720485+0000","last_fullsized":"2024-09-16T15:44:45.720485+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:31:46.215684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T15:44:44.711883+0000","last_change":"2024-09-16T15:44:44.711284+0000","last_active":"2024-09-16T15:44:44.711883+0000","last_peered":"2024-09-16T15:44:44.711883+0000","last_clean":"2024-09-16T15:44:44.711883+0000","last_became_active":"2024-09-16T15:44:44.709695+0000","last_became_peered":"2024-09-16T15:44:44.709695+0000","last_unstale":"2024-09-16T15:44:44.711883+0000","last_undegraded":"2024-09-16T15:44:44.711883+0000","last_fullsized":"2024-09-16T15:44:44.711883+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:29:18.863386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.718328+0000","last_change":"2024-09-16T15:44:45.714790+0000","last_active":"2024-09-16T15:44:45.718328+0000","last_peered":"2024-09-16T15:44:45.718328+0000","last_clean":"2024-09-16T15:44:45.718328+0000","last_became_active":"2024-09-16T15:44:45.713900+0000","last_became_peered":"2024-09-16T15:44:45.713900+0000","last_unstale":"2024-09-16T15:44:45.718328+0000","last_undegraded":"2024-09-16T15:44:45.718328+0000","last_fullsized":"2024-09-16T15:44:45.718328+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:50:53.880904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"61'2","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.730326+0000","last_change":"2024-09-16T15:44:44.698462+0000","last_active":"2024-09-16T15:44:45.730326+0000","last_peered":"2024-09-16T15:44:45.730326+0000","last_clean":"2024-09-16T15:44:45.730326+0000","last_became_active":"2024-09-16T15:44:44.698179+0000","last_became_peered":"2024-09-16T15:44:44.698179+0000","last_unstale":"2024-09-16T15:44:45.730326+0000","last_undegraded":"2024-09-16T15:44:45.730326+0000","last_fullsized":"2024-09-16T15:44:45.730326+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:13:21.284936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T15:44:44.693602+0000","last_change":"2024-09-16T15:44:44.693602+0000","last_active":"2024-09-16T15:44:44.679289+0000","last_peered":"2024-09-16T15:44:44.679289+0000","last_clean":"2024-09-16T15:44:44.679289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:44:44.693602+0000","last_undegraded":"2024-09-16T15:44:44.693602+0000","last_fullsized":"2024-09-16T15:44:44.693602+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"61'1","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.741605+0000","last_change":"2024-09-16T15:44:44.706373+0000","last_active":"2024-09-16T15:44:45.741605+0000","last_peered":"2024-09-16T15:44:45.741605+0000","last_clean":"2024-09-16T15:44:45.741605+0000","last_became_active":"2024-09-16T15:44:44.705120+0000","last_became_peered":"2024-09-16T15:44:44.705120+0000","last_unstale":"2024-09-16T15:44:45.741605+0000","last_undegraded":"2024-09-16T15:44:45.741605+0000","last_fullsized":"2024-09-16T15:44:45.741605+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:07:54.665094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.735750+0000","last_change":"2024-09-16T15:44:45.726953+0000","last_active":"2024-09-16T15:44:45.735750+0000","last_peered":"2024-09-16T15:44:45.735750+0000","last_clean":"2024-09-16T15:44:45.735750+0000","last_became_active":"2024-09-16T15:44:45.723508+0000","last_became_peered":"2024-09-16T15:44:45.723508+0000","last_unstale":"2024-09-16T15:44:45.735750+0000","last_undegraded":"2024-09-16T15:44:45.735750+0000","last_fullsized":"2024-09-16T15:44:45.735750+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:39:26.814713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.705171+0000","last_change":"2024-09-16T15:44:44.708721+0000","last_active":"2024-09-16T15:44:45.705171+0000","last_peered":"2024-09-16T15:44:45.705171+0000","last_clean":"2024-09-16T15:44:45.705171+0000","last_became_active":"2024-09-16T15:44:44.708471+0000","last_became_peered":"2024-09-16T15:44:44.708471+0000","last_unstale":"2024-09-16T15:44:45.705171+0000","last_undegraded":"2024-09-16T15:44:45.705171+0000","last_fullsized":"2024-09-16T15:44:45.705171+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:39:49.307383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.720161+0000","last_change":"2024-09-16T15:44:45.716983+0000","last_active":"2024-09-16T15:44:45.720161+0000","last_peered":"2024-09-16T15:44:45.720161+0000","last_clean":"2024-09-16T15:44:45.720161+0000","last_became_active":"2024-09-16T15:44:45.715530+0000","last_became_peered":"2024-09-16T15:44:45.715530+0000","last_unstale":"2024-09-16T15:44:45.720161+0000","last_undegraded":"2024-09-16T15:44:45.720161+0000","last_fullsized":"2024-09-16T15:44:45.720161+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:03:43.262795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"61'2","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.733237+0000","last_change":"2024-09-16T15:44:44.696118+0000","last_active":"2024-09-16T15:44:45.733237+0000","last_peered":"2024-09-16T15:44:45.733237+0000","last_clean":"2024-09-16T15:44:45.733237+0000","last_became_active":"2024-09-16T15:44:44.695842+0000","last_became_peered":"2024-09-16T15:44:44.695842+0000","last_unstale":"2024-09-16T15:44:45.733237+0000","last_undegraded":"2024-09-16T15:44:45.733237+0000","last_fullsized":"2024-09-16T15:44:45.733237+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:31:45.212994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.724057+0000","last_change":"2024-09-16T15:44:45.712249+0000","last_active":"2024-09-16T15:44:45.724057+0000","last_peered":"2024-09-16T15:44:45.724057+0000","last_clean":"2024-09-16T15:44:45.724057+0000","last_became_active":"2024-09-16T15:44:45.711524+0000","last_became_peered":"2024-09-16T15:44:45.711524+0000","last_unstale":"2024-09-16T15:44:45.724057+0000","last_undegraded":"2024-09-16T15:44:45.724057+0000","last_fullsized":"2024-09-16T15:44:45.724057+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:45:42.401451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.703897+0000","last_change":"2024-09-16T15:44:44.695883+0000","last_active":"2024-09-16T15:44:45.703897+0000","last_peered":"2024-09-16T15:44:45.703897+0000","last_clean":"2024-09-16T15:44:45.703897+0000","last_became_active":"2024-09-16T15:44:44.695562+0000","last_became_peered":"2024-09-16T15:44:44.695562+0000","last_unstale":"2024-09-16T15:44:45.703897+0000","last_undegraded":"2024-09-16T15:44:45.703897+0000","last_fullsized":"2024-09-16T15:44:45.703897+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:22:24.705292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.722865+0000","last_change":"2024-09-16T15:44:45.708395+0000","last_active":"2024-09-16T15:44:45.722865+0000","last_peered":"2024-09-16T15:44:45.722865+0000","last_clean":"2024-09-16T15:44:45.722865+0000","last_became_active":"2024-09-16T15:44:45.708090+0000","last_became_peered":"2024-09-16T15:44:45.708090+0000","last_unstale":"2024-09-16T15:44:45.722865+0000","last_undegraded":"2024-09-16T15:44:45.722865+0000","last_fullsized":"2024-09-16T15:44:45.722865+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:13:37.240255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.712718+0000","last_change":"2024-09-16T15:44:44.705798+0000","last_active":"2024-09-16T15:44:45.712718+0000","last_peered":"2024-09-16T15:44:45.712718+0000","last_clean":"2024-09-16T15:44:45.712718+0000","last_became_active":"2024-09-16T15:44:44.705468+0000","last_became_peered":"2024-09-16T15:44:44.705468+0000","last_unstale":"2024-09-16T15:44:45.712718+0000","last_undegraded":"2024-09-16T15:44:45.712718+0000","last_fullsized":"2024-09-16T15:44:45.712718+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:21:59.839074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.736422+0000","last_change":"2024-09-16T15:44:45.726903+0000","last_active":"2024-09-16T15:44:45.736422+0000","last_peered":"2024-09-16T15:44:45.736422+0000","last_clean":"2024-09-16T15:44:45.736422+0000","last_became_active":"2024-09-16T15:44:45.726666+0000","last_became_peered":"2024-09-16T15:44:45.726666+0000","last_unstale":"2024-09-16T15:44:45.736422+0000","last_undegraded":"2024-09-16T15:44:45.736422+0000","last_fullsized":"2024-09-16T15:44:45.736422+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:51:57.452114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T15:44:44.710737+0000","last_change":"2024-09-16T15:44:44.708470+0000","last_active":"2024-09-16T15:44:44.710737+0000","last_peered":"2024-09-16T15:44:44.710737+0000","last_clean":"2024-09-16T15:44:44.710737+0000","last_became_active":"2024-09-16T15:44:44.708182+0000","last_became_peered":"2024-09-16T15:44:44.708182+0000","last_unstale":"2024-09-16T15:44:44.710737+0000","last_undegraded":"2024-09-16T15:44:44.710737+0000","last_fullsized":"2024-09-16T15:44:44.710737+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:37:09.093235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.718282+0000","last_change":"2024-09-16T15:44:45.707416+0000","last_active":"2024-09-16T15:44:45.718282+0000","last_peered":"2024-09-16T15:44:45.718282+0000","last_clean":"2024-09-16T15:44:45.718282+0000","last_became_active":"2024-09-16T15:44:45.707148+0000","last_became_peered":"2024-09-16T15:44:45.707148+0000","last_unstale":"2024-09-16T15:44:45.718282+0000","last_undegraded":"2024-09-16T15:44:45.718282+0000","last_fullsized":"2024-09-16T15:44:45.718282+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:34:31.566558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.713344+0000","last_change":"2024-09-16T15:44:44.710583+0000","last_active":"2024-09-16T15:44:45.713344+0000","last_peered":"2024-09-16T15:44:45.713344+0000","last_clean":"2024-09-16T15:44:45.713344+0000","last_became_active":"2024-09-16T15:44:44.710293+0000","last_became_peered":"2024-09-16T15:44:44.710293+0000","last_unstale":"2024-09-16T15:44:45.713344+0000","last_undegraded":"2024-09-16T15:44:45.713344+0000","last_fullsized":"2024-09-16T15:44:45.713344+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:03:46.514703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T15:44:44.692278+0000","last_change":"2024-09-16T15:44:44.692278+0000","last_active":"2024-09-16T15:44:44.679289+0000","last_peered":"2024-09-16T15:44:44.679289+0000","last_clean":"2024-09-16T15:44:44.679289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:44:44.692278+0000","last_undegraded":"2024-09-16T15:44:44.692278+0000","last_fullsized":"2024-09-16T15:44:44.692278+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.704857+0000","last_change":"2024-09-16T15:44:44.703904+0000","last_active":"2024-09-16T15:44:45.704857+0000","last_peered":"2024-09-16T15:44:45.704857+0000","last_clean":"2024-09-16T15:44:45.704857+0000","last_became_active":"2024-09-16T15:44:44.703644+0000","last_became_peered":"2024-09-16T15:44:44.703644+0000","last_unstale":"2024-09-16T15:44:45.704857+0000","last_undegraded":"2024-09-16T15:44:45.704857+0000","last_fullsized":"2024-09-16T15:44:45.704857+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:31:17.501469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.724319+0000","last_change":"2024-09-16T15:44:45.714221+0000","last_active":"2024-09-16T15:44:45.724319+0000","last_peered":"2024-09-16T15:44:45.724319+0000","last_clean":"2024-09-16T15:44:45.724319+0000","last_became_active":"2024-09-16T15:44:45.713871+0000","last_became_peered":"2024-09-16T15:44:45.713871+0000","last_unstale":"2024-09-16T15:44:45.724319+0000","last_undegraded":"2024-09-16T15:44:45.724319+0000","last_fullsized":"2024-09-16T15:44:45.724319+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:09:31.284857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.717881+0000","last_change":"2024-09-16T15:44:44.698858+0000","last_active":"2024-09-16T15:44:45.717881+0000","last_peered":"2024-09-16T15:44:45.717881+0000","last_clean":"2024-09-16T15:44:45.717881+0000","last_became_active":"2024-09-16T15:44:44.698546+0000","last_became_peered":"2024-09-16T15:44:44.698546+0000","last_unstale":"2024-09-16T15:44:45.717881+0000","last_undegraded":"2024-09-16T15:44:45.717881+0000","last_fullsized":"2024-09-16T15:44:45.717881+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:03:16.348362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.730750+0000","last_change":"2024-09-16T15:44:45.719192+0000","last_active":"2024-09-16T15:44:45.730750+0000","last_peered":"2024-09-16T15:44:45.730750+0000","last_clean":"2024-09-16T15:44:45.730750+0000","last_became_active":"2024-09-16T15:44:45.716454+0000","last_became_peered":"2024-09-16T15:44:45.716454+0000","last_unstale":"2024-09-16T15:44:45.730750+0000","last_undegraded":"2024-09-16T15:44:45.730750+0000","last_fullsized":"2024-09-16T15:44:45.730750+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:05:16.681904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-16T15:44:44.710050+0000","last_change":"2024-09-16T15:44:44.707043+0000","last_active":"2024-09-16T15:44:44.710050+0000","last_peered":"2024-09-16T15:44:44.710050+0000","last_clean":"2024-09-16T15:44:44.710050+0000","last_became_active":"2024-09-16T15:44:44.706747+0000","last_became_peered":"2024-09-16T15:44:44.706747+0000","last_unstale":"2024-09-16T15:44:44.710050+0000","last_undegraded":"2024-09-16T15:44:44.710050+0000","last_fullsized":"2024-09-16T15:44:44.710050+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:15:01.043845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.730635+0000","last_change":"2024-09-16T15:44:45.726366+0000","last_active":"2024-09-16T15:44:45.730635+0000","last_peered":"2024-09-16T15:44:45.730635+0000","last_clean":"2024-09-16T15:44:45.730635+0000","last_became_active":"2024-09-16T15:44:45.726122+0000","last_became_peered":"2024-09-16T15:44:45.726122+0000","last_unstale":"2024-09-16T15:44:45.730635+0000","last_undegraded":"2024-09-16T15:44:45.730635+0000","last_fullsized":"2024-09-16T15:44:45.730635+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:13:21.061051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.705257+0000","last_change":"2024-09-16T15:44:44.704189+0000","last_active":"2024-09-16T15:44:45.705257+0000","last_peered":"2024-09-16T15:44:45.705257+0000","last_clean":"2024-09-16T15:44:45.705257+0000","last_became_active":"2024-09-16T15:44:44.703916+0000","last_became_peered":"2024-09-16T15:44:44.703916+0000","last_unstale":"2024-09-16T15:44:45.705257+0000","last_undegraded":"2024-09-16T15:44:45.705257+0000","last_fullsized":"2024-09-16T15:44:45.705257+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:50:09.467231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.724010+0000","last_change":"2024-09-16T15:44:45.710420+0000","last_active":"2024-09-16T15:44:45.724010+0000","last_peered":"2024-09-16T15:44:45.724010+0000","last_clean":"2024-09-16T15:44:45.724010+0000","last_became_active":"2024-09-16T15:44:45.710144+0000","last_became_peered":"2024-09-16T15:44:45.710144+0000","last_unstale":"2024-09-16T15:44:45.724010+0000","last_undegraded":"2024-09-16T15:44:45.724010+0000","last_fullsized":"2024-09-16T15:44:45.724010+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:23:06.419124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.707293+0000","last_change":"2024-09-16T15:44:44.708397+0000","last_active":"2024-09-16T15:44:45.707293+0000","last_peered":"2024-09-16T15:44:45.707293+0000","last_clean":"2024-09-16T15:44:45.707293+0000","last_became_active":"2024-09-16T15:44:44.708101+0000","last_became_peered":"2024-09-16T15:44:44.708101+0000","last_unstale":"2024-09-16T15:44:45.707293+0000","last_undegraded":"2024-09-16T15:44:45.707293+0000","last_fullsized":"2024-09-16T15:44:45.707293+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:09:26.705708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-09-16T15:44:44.691091+0000","last_change":"2024-09-16T15:44:44.691091+0000","last_active":"2024-09-16T15:44:44.679289+0000","last_peered":"2024-09-16T15:44:44.679289+0000","last_clean":"2024-09-16T15:44:44.679289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:44:44.691091+0000","last_undegraded":"2024-09-16T15:44:44.691091+0000","last_fullsized":"2024-09-16T15:44:44.691091+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.703426+0000","last_change":"2024-09-16T15:44:44.698425+0000","last_active":"2024-09-16T15:44:45.703426+0000","last_peered":"2024-09-16T15:44:45.703426+0000","last_clean":"2024-09-16T15:44:45.703426+0000","last_became_active":"2024-09-16T15:44:44.698141+0000","last_became_peered":"2024-09-16T15:44:44.698141+0000","last_unstale":"2024-09-16T15:44:45.703426+0000","last_undegraded":"2024-09-16T15:44:45.703426+0000","last_fullsized":"2024-09-16T15:44:45.703426+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:55:27.169366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.724043+0000","last_change":"2024-09-16T15:44:45.720393+0000","last_active":"2024-09-16T15:44:45.724043+0000","last_peered":"2024-09-16T15:44:45.724043+0000","last_clean":"2024-09-16T15:44:45.724043+0000","last_became_active":"2024-09-16T15:44:45.720028+0000","last_became_peered":"2024-09-16T15:44:45.720028+0000","last_unstale":"2024-09-16T15:44:45.724043+0000","last_undegraded":"2024-09-16T15:44:45.724043+0000","last_fullsized":"2024-09-16T15:44:45.724043+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:46:17.054489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.703722+0000","last_change":"2024-09-16T15:44:44.690580+0000","last_active":"2024-09-16T15:44:45.703722+0000","last_peered":"2024-09-16T15:44:45.703722+0000","last_clean":"2024-09-16T15:44:45.703722+0000","last_became_active":"2024-09-16T15:44:44.690302+0000","last_became_peered":"2024-09-16T15:44:44.690302+0000","last_unstale":"2024-09-16T15:44:45.703722+0000","last_undegraded":"2024-09-16T15:44:45.703722+0000","last_fullsized":"2024-09-16T15:44:45.703722+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:45:53.368994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":185,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-16T15:44:45.709646+0000","last_change":"2024-09-16T15:39:12.698813+0000","last_active":"2024-09-16T15:44:45.709646+0000","last_peered":"2024-09-16T15:44:45.709646+0000","last_clean":"2024-09-16T15:44:45.709646+0000","last_became_active":"2024-09-16T15:39:12.697695+0000","last_became_peered":"2024-09-16T15:39:12.697695+0000","last_unstale":"2024-09-16T15:44:45.709646+0000","last_undegraded":"2024-09-16T15:44:45.709646+0000","last_fullsized":"2024-09-16T15:44:45.709646+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_clean_scrub_stamp":"2024-09-16T15:39:11.684967+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:02.061081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2388,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":34,"ondisk_log_size":34,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509512,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40208,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331632,"statfs":{"total":96636764160,"available":96595591168,"internally_reserved":0,"allocated":712704,"data_stored":296806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":40434120},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509511,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46356,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325484,"statfs":{"total":96636764160,"available":96589295616,"internally_reserved":0,"allocated":1306624,"data_stored":887174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509510,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45772,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326068,"statfs":{"total":96636764160,"available":96589893632,"internally_reserved":0,"allocated":712704,"data_stored":296806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509510,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40204,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331636,"statfs":{"total":96636764160,"available":96595595264,"internally_reserved":0,"allocated":712704,"data_stored":296806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":40434121},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509510,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62456,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94309384,"statfs":{"total":96636764160,"available":96572809216,"internally_reserved":0,"allocated":745472,"data_stored":305381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":63175111},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509511,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40788,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331052,"statfs":{"total":96636764160,"available":96594997248,"internally_reserved":0,"allocated":1306624,"data_stored":887174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509509,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40208,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331632,"statfs":{"total":96636764160,"available":96595591168,"internally_reserved":0,"allocated":712704,"data_stored":296806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542216,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40208,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331632,"statfs":{"total":96636764160,"available":96595591168,"internally_reserved":0,"allocated":712704,"data_stored":296806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":40434124},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:44:54.647 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:44:54.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:54.816+0000 7f83cc200640 1 -- 172.21.15.151:0/2458906983 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f83ac003a00 msgr2=0x7f83ac003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:54.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:54.816+0000 7f83cc200640 1 --2- 172.21.15.151:0/2458906983 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f83ac003a00 0x7f83ac003e20 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f83b8009b50 tx=0x7f83ac005830 comp rx=0 tx=0).stop 2024-09-16T15:44:54.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:54.816+0000 7f83ca200640 1 -- 172.21.15.151:0/2458906983 reap_dead start 2024-09-16T15:44:54.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:54.816+0000 7f83cc200640 1 -- 172.21.15.151:0/2458906983 shutdown_connections 2024-09-16T15:44:54.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:54.816+0000 7f83cc200640 1 -- 172.21.15.151:0/2458906983 >> 172.21.15.151:0/2458906983 conn(0x7f83c40b1950 msgr2=0x7f83c40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:54.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:54.816+0000 7f83cc200640 1 -- 172.21.15.151:0/2458906983 shutdown_connections 2024-09-16T15:44:54.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:54.817+0000 7f83cc200640 1 -- 172.21.15.151:0/2458906983 wait complete. 2024-09-16T15:44:54.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:54.818+0000 7f83cc200640 1 Processor -- start 2024-09-16T15:44:54.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:54.818+0000 7f83cc200640 1 -- start start 2024-09-16T15:44:54.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:54.819+0000 7f83cc200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f83ac003a00 0x7f83c4155a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:54.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:54.819+0000 7f83cc200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f83c4159bb0 0x7f83c40b4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:54.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:54.819+0000 7f83cc200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f83c40cc4a0 0x7f83c415c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:54.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:54.819+0000 7f83cc200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f83c40b25c0 con 0x7f83c40cc4a0 2024-09-16T15:44:54.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:54.819+0000 7f83cc200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f83c40b2440 con 0x7f83ac003a00 2024-09-16T15:44:54.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:54.819+0000 7f83cc200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f83c40b2740 con 0x7f83c4159bb0 2024-09-16T15:44:54.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:54.819+0000 7f83ca200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f83c40cc4a0 0x7f83c415c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:54.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:54.819+0000 7f83ca200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f83c40cc4a0 0x7f83c415c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:39466/0 (socket says 172.21.15.151:39466) 2024-09-16T15:44:54.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:54.819+0000 7f83ca200640 1 -- 172.21.15.151:0/3395770527 learned_addr learned my addr 172.21.15.151:0/3395770527 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:54.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:54.820+0000 7f83c8e00640 1 --2- 172.21.15.151:0/3395770527 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f83c4159bb0 0x7f83c40b4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:54.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:54.820+0000 7f83c9800640 1 --2- 172.21.15.151:0/3395770527 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f83ac003a00 0x7f83c4155a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:54.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:54.820+0000 7f83ca200640 1 -- 172.21.15.151:0/3395770527 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f83c4159bb0 msgr2=0x7f83c40b4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:54.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:54.820+0000 7f83ca200640 1 --2- 172.21.15.151:0/3395770527 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f83c4159bb0 0x7f83c40b4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:54.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:54.820+0000 7f83ca200640 1 -- 172.21.15.151:0/3395770527 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f83ac003a00 msgr2=0x7f83c4155a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:54.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:54.820+0000 7f83ca200640 1 --2- 172.21.15.151:0/3395770527 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f83ac003a00 0x7f83c4155a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:54.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:54.820+0000 7f83ca200640 1 -- 172.21.15.151:0/3395770527 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83c40b30f0 con 0x7f83c40cc4a0 2024-09-16T15:44:54.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:54.820+0000 7f83ca200640 1 --2- 172.21.15.151:0/3395770527 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f83c40cc4a0 0x7f83c415c270 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f83c406a510 tx=0x7f83bc010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:54.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:54.820+0000 7f83c1600640 1 -- 172.21.15.151:0/3395770527 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83bc003970 con 0x7f83c40cc4a0 2024-09-16T15:44:54.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:54.820+0000 7f83c1600640 1 -- 172.21.15.151:0/3395770527 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f83bc015070 con 0x7f83c40cc4a0 2024-09-16T15:44:54.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:54.820+0000 7f83c1600640 1 -- 172.21.15.151:0/3395770527 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83bc004030 con 0x7f83c40cc4a0 2024-09-16T15:44:54.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:54.820+0000 7f83cc200640 1 -- 172.21.15.151:0/3395770527 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83c40b33d0 con 0x7f83c40cc4a0 2024-09-16T15:44:54.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:54.821+0000 7f83cc200640 1 -- 172.21.15.151:0/3395770527 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83c40b3990 con 0x7f83c40cc4a0 2024-09-16T15:44:54.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:54.822+0000 7f83cc200640 1 -- 172.21.15.151:0/3395770527 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83c4074460 con 0x7f83c40cc4a0 2024-09-16T15:44:54.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:54.823+0000 7f83c1600640 1 -- 172.21.15.151:0/3395770527 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 102103+0+0 (secure 0 0 0) 0x7f83bc0041d0 con 0x7f83c40cc4a0 2024-09-16T15:44:54.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:54.824+0000 7f83c1600640 1 --2- 172.21.15.151:0/3395770527 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f83a4079150 0x7f83a407b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:54.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:54.824+0000 7f83c1600640 1 -- 172.21.15.151:0/3395770527 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7f83bc099870 con 0x7f83c40cc4a0 2024-09-16T15:44:54.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:54.824+0000 7f83c9800640 1 --2- 172.21.15.151:0/3395770527 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f83a4079150 0x7f83a407b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:54.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:54.825+0000 7f83c9800640 1 --2- 172.21.15.151:0/3395770527 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f83a4079150 0x7f83a407b610 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f83b8035000 tx=0x7f83b802d040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:54.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:54.828+0000 7f83c1600640 1 -- 172.21.15.151:0/3395770527 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f83bc0632e0 con 0x7f83c40cc4a0 2024-09-16T15:44:55.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.009+0000 7f83cc200640 1 -- 172.21.15.151:0/3395770527 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f83c40ae5b0 con 0x7f83a4079150 2024-09-16T15:44:55.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.013+0000 7f83c1600640 1 -- 172.21.15.151:0/3395770527 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251924 (secure 0 0 0) 0x7f83c40ae5b0 con 0x7f83a4079150 2024-09-16T15:44:55.018 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:44:55.019 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-09-16T15:44:55.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.017+0000 7f83cc200640 1 -- 172.21.15.151:0/3395770527 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f83a4079150 msgr2=0x7f83a407b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:55.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.017+0000 7f83cc200640 1 --2- 172.21.15.151:0/3395770527 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f83a4079150 0x7f83a407b610 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f83b8035000 tx=0x7f83b802d040 comp rx=0 tx=0).stop 2024-09-16T15:44:55.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.017+0000 7f83cc200640 1 -- 172.21.15.151:0/3395770527 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f83c40cc4a0 msgr2=0x7f83c415c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:55.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.017+0000 7f83cc200640 1 --2- 172.21.15.151:0/3395770527 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f83c40cc4a0 0x7f83c415c270 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f83c406a510 tx=0x7f83bc010f40 comp rx=0 tx=0).stop 2024-09-16T15:44:55.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.018+0000 7f83cc200640 1 -- 172.21.15.151:0/3395770527 shutdown_connections 2024-09-16T15:44:55.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.018+0000 7f83cc200640 1 --2- 172.21.15.151:0/3395770527 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f83a4079150 0x7f83a407b610 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:55.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.018+0000 7f83cc200640 1 --2- 172.21.15.151:0/3395770527 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f83c40cc4a0 0x7f83c415c270 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:55.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.018+0000 7f83cc200640 1 --2- 172.21.15.151:0/3395770527 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f83c4159bb0 0x7f83c40b4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:55.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.018+0000 7f83cc200640 1 --2- 172.21.15.151:0/3395770527 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f83ac003a00 0x7f83c4155a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:55.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.018+0000 7f83cc200640 1 -- 172.21.15.151:0/3395770527 >> 172.21.15.151:0/3395770527 conn(0x7f83c40b1950 msgr2=0x7f83c409ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:55.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.018+0000 7f83cc200640 1 -- 172.21.15.151:0/3395770527 shutdown_connections 2024-09-16T15:44:55.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.018+0000 7f83cc200640 1 -- 172.21.15.151:0/3395770527 wait complete. 2024-09-16T15:44:55.036 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":227,"stamp":"2024-09-16T15:44:53.671594+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":356408,"kb_used_data":6984,"kb_used_omap":12,"kb_used_meta":349235,"kb_avail":754618312,"statfs":{"total":773094113280,"available":772729151488,"internally_reserved":0,"allocated":7151616,"data_stored":3665052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12738,"internal_metadata":357617214},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002115"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:49.050117+0000","last_change":"2024-09-16T15:44:45.728739+0000","last_active":"2024-09-16T15:44:49.050117+0000","last_peered":"2024-09-16T15:44:49.050117+0000","last_clean":"2024-09-16T15:44:49.050117+0000","last_became_active":"2024-09-16T15:44:45.727368+0000","last_became_peered":"2024-09-16T15:44:45.727368+0000","last_unstale":"2024-09-16T15:44:49.050117+0000","last_undegraded":"2024-09-16T15:44:49.050117+0000","last_fullsized":"2024-09-16T15:44:49.050117+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:31:46.215684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:47.365349+0000","last_change":"2024-09-16T15:44:45.734534+0000","last_active":"2024-09-16T15:44:47.365349+0000","last_peered":"2024-09-16T15:44:47.365349+0000","last_clean":"2024-09-16T15:44:47.365349+0000","last_became_active":"2024-09-16T15:44:45.733861+0000","last_became_peered":"2024-09-16T15:44:45.733861+0000","last_unstale":"2024-09-16T15:44:47.365349+0000","last_undegraded":"2024-09-16T15:44:47.365349+0000","last_fullsized":"2024-09-16T15:44:47.365349+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:33:31.611138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:46.719052+0000","last_change":"2024-09-16T15:44:45.725780+0000","last_active":"2024-09-16T15:44:46.719052+0000","last_peered":"2024-09-16T15:44:46.719052+0000","last_clean":"2024-09-16T15:44:46.719052+0000","last_became_active":"2024-09-16T15:44:45.721333+0000","last_became_peered":"2024-09-16T15:44:45.721333+0000","last_unstale":"2024-09-16T15:44:46.719052+0000","last_undegraded":"2024-09-16T15:44:46.719052+0000","last_fullsized":"2024-09-16T15:44:46.719052+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:41:36.724191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:49.049338+0000","last_change":"2024-09-16T15:44:45.729388+0000","last_active":"2024-09-16T15:44:49.049338+0000","last_peered":"2024-09-16T15:44:49.049338+0000","last_clean":"2024-09-16T15:44:49.049338+0000","last_became_active":"2024-09-16T15:44:45.729119+0000","last_became_peered":"2024-09-16T15:44:45.729119+0000","last_unstale":"2024-09-16T15:44:49.049338+0000","last_undegraded":"2024-09-16T15:44:49.049338+0000","last_fullsized":"2024-09-16T15:44:49.049338+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:23:52.271059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:49.049896+0000","last_change":"2024-09-16T15:44:45.733586+0000","last_active":"2024-09-16T15:44:49.049896+0000","last_peered":"2024-09-16T15:44:49.049896+0000","last_clean":"2024-09-16T15:44:49.049896+0000","last_became_active":"2024-09-16T15:44:45.733144+0000","last_became_peered":"2024-09-16T15:44:45.733144+0000","last_unstale":"2024-09-16T15:44:49.049896+0000","last_undegraded":"2024-09-16T15:44:49.049896+0000","last_fullsized":"2024-09-16T15:44:49.049896+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:28:57.953253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:49.049634+0000","last_change":"2024-09-16T15:44:45.718049+0000","last_active":"2024-09-16T15:44:49.049634+0000","last_peered":"2024-09-16T15:44:49.049634+0000","last_clean":"2024-09-16T15:44:49.049634+0000","last_became_active":"2024-09-16T15:44:45.716815+0000","last_became_peered":"2024-09-16T15:44:45.716815+0000","last_unstale":"2024-09-16T15:44:49.049634+0000","last_undegraded":"2024-09-16T15:44:49.049634+0000","last_fullsized":"2024-09-16T15:44:49.049634+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:56:11.276174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:49.049027+0000","last_change":"2024-09-16T15:44:45.718523+0000","last_active":"2024-09-16T15:44:49.049027+0000","last_peered":"2024-09-16T15:44:49.049027+0000","last_clean":"2024-09-16T15:44:49.049027+0000","last_became_active":"2024-09-16T15:44:45.718208+0000","last_became_peered":"2024-09-16T15:44:45.718208+0000","last_unstale":"2024-09-16T15:44:49.049027+0000","last_undegraded":"2024-09-16T15:44:49.049027+0000","last_fullsized":"2024-09-16T15:44:49.049027+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:56:51.530203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:46.721803+0000","last_change":"2024-09-16T15:44:45.720647+0000","last_active":"2024-09-16T15:44:46.721803+0000","last_peered":"2024-09-16T15:44:46.721803+0000","last_clean":"2024-09-16T15:44:46.721803+0000","last_became_active":"2024-09-16T15:44:45.720351+0000","last_became_peered":"2024-09-16T15:44:45.720351+0000","last_unstale":"2024-09-16T15:44:46.721803+0000","last_undegraded":"2024-09-16T15:44:46.721803+0000","last_fullsized":"2024-09-16T15:44:46.721803+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:39:26.814713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:47.445689+0000","last_change":"2024-09-16T15:44:45.732987+0000","last_active":"2024-09-16T15:44:47.445689+0000","last_peered":"2024-09-16T15:44:47.445689+0000","last_clean":"2024-09-16T15:44:47.445689+0000","last_became_active":"2024-09-16T15:44:45.732680+0000","last_became_peered":"2024-09-16T15:44:45.732680+0000","last_unstale":"2024-09-16T15:44:47.445689+0000","last_undegraded":"2024-09-16T15:44:47.445689+0000","last_fullsized":"2024-09-16T15:44:47.445689+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:22:47.221395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:49.050189+0000","last_change":"2024-09-16T15:44:45.719120+0000","last_active":"2024-09-16T15:44:49.050189+0000","last_peered":"2024-09-16T15:44:49.050189+0000","last_clean":"2024-09-16T15:44:49.050189+0000","last_became_active":"2024-09-16T15:44:45.718746+0000","last_became_peered":"2024-09-16T15:44:45.718746+0000","last_unstale":"2024-09-16T15:44:49.050189+0000","last_undegraded":"2024-09-16T15:44:49.050189+0000","last_fullsized":"2024-09-16T15:44:49.050189+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:03:17.351052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:46.722019+0000","last_change":"2024-09-16T15:44:45.718730+0000","last_active":"2024-09-16T15:44:46.722019+0000","last_peered":"2024-09-16T15:44:46.722019+0000","last_clean":"2024-09-16T15:44:46.722019+0000","last_became_active":"2024-09-16T15:44:45.718442+0000","last_became_peered":"2024-09-16T15:44:45.718442+0000","last_unstale":"2024-09-16T15:44:46.722019+0000","last_undegraded":"2024-09-16T15:44:46.722019+0000","last_fullsized":"2024-09-16T15:44:46.722019+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:41:36.724191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:47.445452+0000","last_change":"2024-09-16T15:44:45.717536+0000","last_active":"2024-09-16T15:44:47.445452+0000","last_peered":"2024-09-16T15:44:47.445452+0000","last_clean":"2024-09-16T15:44:47.445452+0000","last_became_active":"2024-09-16T15:44:45.717226+0000","last_became_peered":"2024-09-16T15:44:45.717226+0000","last_unstale":"2024-09-16T15:44:47.445452+0000","last_undegraded":"2024-09-16T15:44:47.445452+0000","last_fullsized":"2024-09-16T15:44:47.445452+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:33:21.874183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:47.254674+0000","last_change":"2024-09-16T15:44:45.711525+0000","last_active":"2024-09-16T15:44:47.254674+0000","last_peered":"2024-09-16T15:44:47.254674+0000","last_clean":"2024-09-16T15:44:47.254674+0000","last_became_active":"2024-09-16T15:44:45.711195+0000","last_became_peered":"2024-09-16T15:44:45.711195+0000","last_unstale":"2024-09-16T15:44:47.254674+0000","last_undegraded":"2024-09-16T15:44:47.254674+0000","last_fullsized":"2024-09-16T15:44:47.254674+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:01:36.270361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:46.719196+0000","last_change":"2024-09-16T15:44:45.726781+0000","last_active":"2024-09-16T15:44:46.719196+0000","last_peered":"2024-09-16T15:44:46.719196+0000","last_clean":"2024-09-16T15:44:46.719196+0000","last_became_active":"2024-09-16T15:44:45.726045+0000","last_became_peered":"2024-09-16T15:44:45.726045+0000","last_unstale":"2024-09-16T15:44:46.719196+0000","last_undegraded":"2024-09-16T15:44:46.719196+0000","last_fullsized":"2024-09-16T15:44:46.719196+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:39:50.310073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:47.254631+0000","last_change":"2024-09-16T15:44:45.716384+0000","last_active":"2024-09-16T15:44:47.254631+0000","last_peered":"2024-09-16T15:44:47.254631+0000","last_clean":"2024-09-16T15:44:47.254631+0000","last_became_active":"2024-09-16T15:44:45.716101+0000","last_became_peered":"2024-09-16T15:44:45.716101+0000","last_unstale":"2024-09-16T15:44:47.254631+0000","last_undegraded":"2024-09-16T15:44:47.254631+0000","last_fullsized":"2024-09-16T15:44:47.254631+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:31:16.283923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:49.049662+0000","last_change":"2024-09-16T15:44:45.721590+0000","last_active":"2024-09-16T15:44:49.049662+0000","last_peered":"2024-09-16T15:44:49.049662+0000","last_clean":"2024-09-16T15:44:49.049662+0000","last_became_active":"2024-09-16T15:44:45.721343+0000","last_became_peered":"2024-09-16T15:44:45.721343+0000","last_unstale":"2024-09-16T15:44:49.049662+0000","last_undegraded":"2024-09-16T15:44:49.049662+0000","last_fullsized":"2024-09-16T15:44:49.049662+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:09:55.115129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:49.050068+0000","last_change":"2024-09-16T15:44:45.722006+0000","last_active":"2024-09-16T15:44:49.050068+0000","last_peered":"2024-09-16T15:44:49.050068+0000","last_clean":"2024-09-16T15:44:49.050068+0000","last_became_active":"2024-09-16T15:44:45.721714+0000","last_became_peered":"2024-09-16T15:44:45.721714+0000","last_unstale":"2024-09-16T15:44:49.050068+0000","last_undegraded":"2024-09-16T15:44:49.050068+0000","last_fullsized":"2024-09-16T15:44:49.050068+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:37:16.035343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:46.719436+0000","last_change":"2024-09-16T15:44:45.732870+0000","last_active":"2024-09-16T15:44:46.719436+0000","last_peered":"2024-09-16T15:44:46.719436+0000","last_clean":"2024-09-16T15:44:46.719436+0000","last_became_active":"2024-09-16T15:44:45.732609+0000","last_became_peered":"2024-09-16T15:44:45.732609+0000","last_unstale":"2024-09-16T15:44:46.719436+0000","last_undegraded":"2024-09-16T15:44:46.719436+0000","last_fullsized":"2024-09-16T15:44:46.719436+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:32:11.331276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:47.365290+0000","last_change":"2024-09-16T15:44:45.731257+0000","last_active":"2024-09-16T15:44:47.365290+0000","last_peered":"2024-09-16T15:44:47.365290+0000","last_clean":"2024-09-16T15:44:47.365290+0000","last_became_active":"2024-09-16T15:44:45.730914+0000","last_became_peered":"2024-09-16T15:44:45.730914+0000","last_unstale":"2024-09-16T15:44:47.365290+0000","last_undegraded":"2024-09-16T15:44:47.365290+0000","last_fullsized":"2024-09-16T15:44:47.365290+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:31:15.931632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:46.721946+0000","last_change":"2024-09-16T15:44:45.731373+0000","last_active":"2024-09-16T15:44:46.721946+0000","last_peered":"2024-09-16T15:44:46.721946+0000","last_clean":"2024-09-16T15:44:46.721946+0000","last_became_active":"2024-09-16T15:44:45.731068+0000","last_became_peered":"2024-09-16T15:44:45.731068+0000","last_unstale":"2024-09-16T15:44:46.721946+0000","last_undegraded":"2024-09-16T15:44:46.721946+0000","last_fullsized":"2024-09-16T15:44:46.721946+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:30:26.600289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:47.254646+0000","last_change":"2024-09-16T15:44:45.719386+0000","last_active":"2024-09-16T15:44:47.254646+0000","last_peered":"2024-09-16T15:44:47.254646+0000","last_clean":"2024-09-16T15:44:47.254646+0000","last_became_active":"2024-09-16T15:44:45.719154+0000","last_became_peered":"2024-09-16T15:44:45.719154+0000","last_unstale":"2024-09-16T15:44:47.254646+0000","last_undegraded":"2024-09-16T15:44:47.254646+0000","last_fullsized":"2024-09-16T15:44:47.254646+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:13:04.403633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:46.721438+0000","last_change":"2024-09-16T15:44:45.718300+0000","last_active":"2024-09-16T15:44:46.721438+0000","last_peered":"2024-09-16T15:44:46.721438+0000","last_clean":"2024-09-16T15:44:46.721438+0000","last_became_active":"2024-09-16T15:44:45.717725+0000","last_became_peered":"2024-09-16T15:44:45.717725+0000","last_unstale":"2024-09-16T15:44:46.721438+0000","last_undegraded":"2024-09-16T15:44:46.721438+0000","last_fullsized":"2024-09-16T15:44:46.721438+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:55:28.172055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:49.049110+0000","last_change":"2024-09-16T15:44:45.716075+0000","last_active":"2024-09-16T15:44:49.049110+0000","last_peered":"2024-09-16T15:44:49.049110+0000","last_clean":"2024-09-16T15:44:49.049110+0000","last_became_active":"2024-09-16T15:44:45.715816+0000","last_became_peered":"2024-09-16T15:44:45.715816+0000","last_unstale":"2024-09-16T15:44:49.049110+0000","last_undegraded":"2024-09-16T15:44:49.049110+0000","last_fullsized":"2024-09-16T15:44:49.049110+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:43:26.143337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:47.445459+0000","last_change":"2024-09-16T15:44:45.721703+0000","last_active":"2024-09-16T15:44:47.445459+0000","last_peered":"2024-09-16T15:44:47.445459+0000","last_clean":"2024-09-16T15:44:47.445459+0000","last_became_active":"2024-09-16T15:44:45.721406+0000","last_became_peered":"2024-09-16T15:44:45.721406+0000","last_unstale":"2024-09-16T15:44:47.445459+0000","last_undegraded":"2024-09-16T15:44:47.445459+0000","last_fullsized":"2024-09-16T15:44:47.445459+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:58:28.518264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:47.365382+0000","last_change":"2024-09-16T15:44:45.734857+0000","last_active":"2024-09-16T15:44:47.365382+0000","last_peered":"2024-09-16T15:44:47.365382+0000","last_clean":"2024-09-16T15:44:47.365382+0000","last_became_active":"2024-09-16T15:44:45.734432+0000","last_became_peered":"2024-09-16T15:44:45.734432+0000","last_unstale":"2024-09-16T15:44:47.365382+0000","last_undegraded":"2024-09-16T15:44:47.365382+0000","last_fullsized":"2024-09-16T15:44:47.365382+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:08:12.174812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:47.254799+0000","last_change":"2024-09-16T15:44:45.722538+0000","last_active":"2024-09-16T15:44:47.254799+0000","last_peered":"2024-09-16T15:44:47.254799+0000","last_clean":"2024-09-16T15:44:47.254799+0000","last_became_active":"2024-09-16T15:44:45.722286+0000","last_became_peered":"2024-09-16T15:44:45.722286+0000","last_unstale":"2024-09-16T15:44:47.254799+0000","last_undegraded":"2024-09-16T15:44:47.254799+0000","last_fullsized":"2024-09-16T15:44:47.254799+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:51:57.452114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:47.365286+0000","last_change":"2024-09-16T15:44:45.720405+0000","last_active":"2024-09-16T15:44:47.365286+0000","last_peered":"2024-09-16T15:44:47.365286+0000","last_clean":"2024-09-16T15:44:47.365286+0000","last_became_active":"2024-09-16T15:44:45.720120+0000","last_became_peered":"2024-09-16T15:44:45.720120+0000","last_unstale":"2024-09-16T15:44:47.365286+0000","last_undegraded":"2024-09-16T15:44:47.365286+0000","last_fullsized":"2024-09-16T15:44:47.365286+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:39:29.557339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:46.721662+0000","last_change":"2024-09-16T15:44:45.707684+0000","last_active":"2024-09-16T15:44:46.721662+0000","last_peered":"2024-09-16T15:44:46.721662+0000","last_clean":"2024-09-16T15:44:46.721662+0000","last_became_active":"2024-09-16T15:44:45.707370+0000","last_became_peered":"2024-09-16T15:44:45.707370+0000","last_unstale":"2024-09-16T15:44:46.721662+0000","last_undegraded":"2024-09-16T15:44:46.721662+0000","last_fullsized":"2024-09-16T15:44:46.721662+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:58:34.310843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:46.721109+0000","last_change":"2024-09-16T15:44:45.712553+0000","last_active":"2024-09-16T15:44:46.721109+0000","last_peered":"2024-09-16T15:44:46.721109+0000","last_clean":"2024-09-16T15:44:46.721109+0000","last_became_active":"2024-09-16T15:44:45.711955+0000","last_became_peered":"2024-09-16T15:44:45.711955+0000","last_unstale":"2024-09-16T15:44:46.721109+0000","last_undegraded":"2024-09-16T15:44:46.721109+0000","last_fullsized":"2024-09-16T15:44:46.721109+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:31:16.141860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:49.049053+0000","last_change":"2024-09-16T15:44:45.711869+0000","last_active":"2024-09-16T15:44:49.049053+0000","last_peered":"2024-09-16T15:44:49.049053+0000","last_clean":"2024-09-16T15:44:49.049053+0000","last_became_active":"2024-09-16T15:44:45.711534+0000","last_became_peered":"2024-09-16T15:44:45.711534+0000","last_unstale":"2024-09-16T15:44:49.049053+0000","last_undegraded":"2024-09-16T15:44:49.049053+0000","last_fullsized":"2024-09-16T15:44:49.049053+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:18:39.537789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:47.364894+0000","last_change":"2024-09-16T15:44:45.723913+0000","last_active":"2024-09-16T15:44:47.364894+0000","last_peered":"2024-09-16T15:44:47.364894+0000","last_clean":"2024-09-16T15:44:47.364894+0000","last_became_active":"2024-09-16T15:44:45.722767+0000","last_became_peered":"2024-09-16T15:44:45.722767+0000","last_unstale":"2024-09-16T15:44:47.364894+0000","last_undegraded":"2024-09-16T15:44:47.364894+0000","last_fullsized":"2024-09-16T15:44:47.364894+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:29:54.399264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:46.721558+0000","last_change":"2024-09-16T15:44:45.726051+0000","last_active":"2024-09-16T15:44:46.721558+0000","last_peered":"2024-09-16T15:44:46.721558+0000","last_clean":"2024-09-16T15:44:46.721558+0000","last_became_active":"2024-09-16T15:44:45.720897+0000","last_became_peered":"2024-09-16T15:44:45.720897+0000","last_unstale":"2024-09-16T15:44:46.721558+0000","last_undegraded":"2024-09-16T15:44:46.721558+0000","last_fullsized":"2024-09-16T15:44:46.721558+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:22:25.707982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:49.050264+0000","last_change":"2024-09-16T15:44:45.724124+0000","last_active":"2024-09-16T15:44:49.050264+0000","last_peered":"2024-09-16T15:44:49.050264+0000","last_clean":"2024-09-16T15:44:49.050264+0000","last_became_active":"2024-09-16T15:44:45.721712+0000","last_became_peered":"2024-09-16T15:44:45.721712+0000","last_unstale":"2024-09-16T15:44:49.050264+0000","last_undegraded":"2024-09-16T15:44:49.050264+0000","last_fullsized":"2024-09-16T15:44:49.050264+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:13:04.403633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"61'4","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:46.722332+0000","last_change":"2024-09-16T15:44:44.700510+0000","last_active":"2024-09-16T15:44:46.722332+0000","last_peered":"2024-09-16T15:44:46.722332+0000","last_clean":"2024-09-16T15:44:46.722332+0000","last_became_active":"2024-09-16T15:44:44.697806+0000","last_became_peered":"2024-09-16T15:44:44.697806+0000","last_unstale":"2024-09-16T15:44:46.722332+0000","last_undegraded":"2024-09-16T15:44:46.722332+0000","last_fullsized":"2024-09-16T15:44:46.722332+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:50:52.878214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:49.050320+0000","last_change":"2024-09-16T15:44:45.715988+0000","last_active":"2024-09-16T15:44:49.050320+0000","last_peered":"2024-09-16T15:44:49.050320+0000","last_clean":"2024-09-16T15:44:49.050320+0000","last_became_active":"2024-09-16T15:44:45.715735+0000","last_became_peered":"2024-09-16T15:44:45.715735+0000","last_unstale":"2024-09-16T15:44:49.050320+0000","last_undegraded":"2024-09-16T15:44:49.050320+0000","last_fullsized":"2024-09-16T15:44:49.050320+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:31:16.283923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"61'1","reported_seq":16,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:47.365340+0000","last_change":"2024-09-16T15:44:44.708252+0000","last_active":"2024-09-16T15:44:47.365340+0000","last_peered":"2024-09-16T15:44:47.365340+0000","last_clean":"2024-09-16T15:44:47.365340+0000","last_became_active":"2024-09-16T15:44:44.707995+0000","last_became_peered":"2024-09-16T15:44:44.707995+0000","last_unstale":"2024-09-16T15:44:47.365340+0000","last_undegraded":"2024-09-16T15:44:47.365340+0000","last_fullsized":"2024-09-16T15:44:47.365340+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:12:34.877518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:46.721891+0000","last_change":"2024-09-16T15:44:45.729683+0000","last_active":"2024-09-16T15:44:46.721891+0000","last_peered":"2024-09-16T15:44:46.721891+0000","last_clean":"2024-09-16T15:44:46.721891+0000","last_became_active":"2024-09-16T15:44:45.729258+0000","last_became_peered":"2024-09-16T15:44:45.729258+0000","last_unstale":"2024-09-16T15:44:46.721891+0000","last_undegraded":"2024-09-16T15:44:46.721891+0000","last_fullsized":"2024-09-16T15:44:46.721891+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:08:12.766873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"61'2","reported_seq":17,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:49.050305+0000","last_change":"2024-09-16T15:44:44.705295+0000","last_active":"2024-09-16T15:44:49.050305+0000","last_peered":"2024-09-16T15:44:49.050305+0000","last_clean":"2024-09-16T15:44:49.050305+0000","last_became_active":"2024-09-16T15:44:44.704913+0000","last_became_peered":"2024-09-16T15:44:44.704913+0000","last_unstale":"2024-09-16T15:44:49.050305+0000","last_undegraded":"2024-09-16T15:44:49.050305+0000","last_fullsized":"2024-09-16T15:44:49.050305+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:33:20.871493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:49.049784+0000","last_change":"2024-09-16T15:44:45.724296+0000","last_active":"2024-09-16T15:44:49.049784+0000","last_peered":"2024-09-16T15:44:49.049784+0000","last_clean":"2024-09-16T15:44:49.049784+0000","last_became_active":"2024-09-16T15:44:45.722552+0000","last_became_peered":"2024-09-16T15:44:45.722552+0000","last_unstale":"2024-09-16T15:44:49.049784+0000","last_undegraded":"2024-09-16T15:44:49.049784+0000","last_fullsized":"2024-09-16T15:44:49.049784+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:07:55.667783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"61'2","reported_seq":17,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:49.049341+0000","last_change":"2024-09-16T15:44:44.701949+0000","last_active":"2024-09-16T15:44:49.049341+0000","last_peered":"2024-09-16T15:44:49.049341+0000","last_clean":"2024-09-16T15:44:49.049341+0000","last_became_active":"2024-09-16T15:44:44.701522+0000","last_became_peered":"2024-09-16T15:44:44.701522+0000","last_unstale":"2024-09-16T15:44:49.049341+0000","last_undegraded":"2024-09-16T15:44:49.049341+0000","last_fullsized":"2024-09-16T15:44:49.049341+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:18:22.181140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:46.721887+0000","last_change":"2024-09-16T15:44:45.712376+0000","last_active":"2024-09-16T15:44:46.721887+0000","last_peered":"2024-09-16T15:44:46.721887+0000","last_clean":"2024-09-16T15:44:46.721887+0000","last_became_active":"2024-09-16T15:44:45.712045+0000","last_became_peered":"2024-09-16T15:44:45.712045+0000","last_unstale":"2024-09-16T15:44:46.721887+0000","last_undegraded":"2024-09-16T15:44:46.721887+0000","last_fullsized":"2024-09-16T15:44:46.721887+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:40:24.536876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"61'1","reported_seq":16,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:46.719131+0000","last_change":"2024-09-16T15:44:44.705164+0000","last_active":"2024-09-16T15:44:46.719131+0000","last_peered":"2024-09-16T15:44:46.719131+0000","last_clean":"2024-09-16T15:44:46.719131+0000","last_became_active":"2024-09-16T15:44:44.704525+0000","last_became_peered":"2024-09-16T15:44:44.704525+0000","last_unstale":"2024-09-16T15:44:46.719131+0000","last_undegraded":"2024-09-16T15:44:46.719131+0000","last_fullsized":"2024-09-16T15:44:46.719131+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:37:15.032653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:46.722174+0000","last_change":"2024-09-16T15:44:45.719896+0000","last_active":"2024-09-16T15:44:46.722174+0000","last_peered":"2024-09-16T15:44:46.722174+0000","last_clean":"2024-09-16T15:44:46.722174+0000","last_became_active":"2024-09-16T15:44:45.718945+0000","last_became_peered":"2024-09-16T15:44:45.718945+0000","last_unstale":"2024-09-16T15:44:46.722174+0000","last_undegraded":"2024-09-16T15:44:46.722174+0000","last_fullsized":"2024-09-16T15:44:46.722174+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:13:22.287626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"61'1","reported_seq":16,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:49.049889+0000","last_change":"2024-09-16T15:44:44.710842+0000","last_active":"2024-09-16T15:44:49.049889+0000","last_peered":"2024-09-16T15:44:49.049889+0000","last_clean":"2024-09-16T15:44:49.049889+0000","last_became_active":"2024-09-16T15:44:44.710529+0000","last_became_peered":"2024-09-16T15:44:44.710529+0000","last_unstale":"2024-09-16T15:44:49.049889+0000","last_undegraded":"2024-09-16T15:44:49.049889+0000","last_fullsized":"2024-09-16T15:44:49.049889+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:36:04.434843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:46.718936+0000","last_change":"2024-09-16T15:44:45.708591+0000","last_active":"2024-09-16T15:44:46.718936+0000","last_peered":"2024-09-16T15:44:46.718936+0000","last_clean":"2024-09-16T15:44:46.718936+0000","last_became_active":"2024-09-16T15:44:45.708350+0000","last_became_peered":"2024-09-16T15:44:45.708350+0000","last_unstale":"2024-09-16T15:44:46.718936+0000","last_undegraded":"2024-09-16T15:44:46.718936+0000","last_fullsized":"2024-09-16T15:44:46.718936+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:30:26.600289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:47.445498+0000","last_change":"2024-09-16T15:44:44.697711+0000","last_active":"2024-09-16T15:44:47.445498+0000","last_peered":"2024-09-16T15:44:47.445498+0000","last_clean":"2024-09-16T15:44:47.445498+0000","last_became_active":"2024-09-16T15:44:44.697357+0000","last_became_peered":"2024-09-16T15:44:44.697357+0000","last_unstale":"2024-09-16T15:44:47.445498+0000","last_undegraded":"2024-09-16T15:44:47.445498+0000","last_fullsized":"2024-09-16T15:44:47.445498+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:20:52.337348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:49.049689+0000","last_change":"2024-09-16T15:44:45.719359+0000","last_active":"2024-09-16T15:44:49.049689+0000","last_peered":"2024-09-16T15:44:49.049689+0000","last_clean":"2024-09-16T15:44:49.049689+0000","last_became_active":"2024-09-16T15:44:45.719093+0000","last_became_peered":"2024-09-16T15:44:45.719093+0000","last_unstale":"2024-09-16T15:44:49.049689+0000","last_undegraded":"2024-09-16T15:44:49.049689+0000","last_fullsized":"2024-09-16T15:44:49.049689+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:50:10.469921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:47.254543+0000","last_change":"2024-09-16T15:44:44.694298+0000","last_active":"2024-09-16T15:44:47.254543+0000","last_peered":"2024-09-16T15:44:47.254543+0000","last_clean":"2024-09-16T15:44:47.254543+0000","last_became_active":"2024-09-16T15:44:44.694052+0000","last_became_peered":"2024-09-16T15:44:44.694052+0000","last_unstale":"2024-09-16T15:44:47.254543+0000","last_undegraded":"2024-09-16T15:44:47.254543+0000","last_fullsized":"2024-09-16T15:44:47.254543+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:25:26.438122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:47.254871+0000","last_change":"2024-09-16T15:44:45.713168+0000","last_active":"2024-09-16T15:44:47.254871+0000","last_peered":"2024-09-16T15:44:47.254871+0000","last_clean":"2024-09-16T15:44:47.254871+0000","last_became_active":"2024-09-16T15:44:45.711840+0000","last_became_peered":"2024-09-16T15:44:45.711840+0000","last_unstale":"2024-09-16T15:44:47.254871+0000","last_undegraded":"2024-09-16T15:44:47.254871+0000","last_fullsized":"2024-09-16T15:44:47.254871+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:44:16.275512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:47.445711+0000","last_change":"2024-09-16T15:44:44.703702+0000","last_active":"2024-09-16T15:44:47.445711+0000","last_peered":"2024-09-16T15:44:47.445711+0000","last_clean":"2024-09-16T15:44:47.445711+0000","last_became_active":"2024-09-16T15:44:44.703424+0000","last_became_peered":"2024-09-16T15:44:44.703424+0000","last_unstale":"2024-09-16T15:44:47.445711+0000","last_undegraded":"2024-09-16T15:44:47.445711+0000","last_fullsized":"2024-09-16T15:44:47.445711+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:24:00.275655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:46.719660+0000","last_change":"2024-09-16T15:44:45.726605+0000","last_active":"2024-09-16T15:44:46.719660+0000","last_peered":"2024-09-16T15:44:46.719660+0000","last_clean":"2024-09-16T15:44:46.719660+0000","last_became_active":"2024-09-16T15:44:45.726368+0000","last_became_peered":"2024-09-16T15:44:45.726368+0000","last_unstale":"2024-09-16T15:44:46.719660+0000","last_undegraded":"2024-09-16T15:44:46.719660+0000","last_fullsized":"2024-09-16T15:44:46.719660+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:45:42.401451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:46.722278+0000","last_change":"2024-09-16T15:44:44.695923+0000","last_active":"2024-09-16T15:44:46.722278+0000","last_peered":"2024-09-16T15:44:46.722278+0000","last_clean":"2024-09-16T15:44:46.722278+0000","last_became_active":"2024-09-16T15:44:44.695618+0000","last_became_peered":"2024-09-16T15:44:44.695618+0000","last_unstale":"2024-09-16T15:44:46.722278+0000","last_undegraded":"2024-09-16T15:44:46.722278+0000","last_fullsized":"2024-09-16T15:44:46.722278+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:41:35.721501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:46.722135+0000","last_change":"2024-09-16T15:44:45.723911+0000","last_active":"2024-09-16T15:44:46.722135+0000","last_peered":"2024-09-16T15:44:46.722135+0000","last_clean":"2024-09-16T15:44:46.722135+0000","last_became_active":"2024-09-16T15:44:45.723557+0000","last_became_peered":"2024-09-16T15:44:45.723557+0000","last_unstale":"2024-09-16T15:44:46.722135+0000","last_undegraded":"2024-09-16T15:44:46.722135+0000","last_fullsized":"2024-09-16T15:44:46.722135+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:18:39.537789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"61'3","reported_seq":18,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:49.049477+0000","last_change":"2024-09-16T15:44:44.700702+0000","last_active":"2024-09-16T15:44:49.049477+0000","last_peered":"2024-09-16T15:44:49.049477+0000","last_clean":"2024-09-16T15:44:49.049477+0000","last_became_active":"2024-09-16T15:44:44.700438+0000","last_became_peered":"2024-09-16T15:44:44.700438+0000","last_unstale":"2024-09-16T15:44:49.049477+0000","last_undegraded":"2024-09-16T15:44:49.049477+0000","last_fullsized":"2024-09-16T15:44:49.049477+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:46:15.633475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:49.049338+0000","last_change":"2024-09-16T15:44:45.709431+0000","last_active":"2024-09-16T15:44:49.049338+0000","last_peered":"2024-09-16T15:44:49.049338+0000","last_clean":"2024-09-16T15:44:49.049338+0000","last_became_active":"2024-09-16T15:44:45.709179+0000","last_became_peered":"2024-09-16T15:44:45.709179+0000","last_unstale":"2024-09-16T15:44:49.049338+0000","last_undegraded":"2024-09-16T15:44:49.049338+0000","last_fullsized":"2024-09-16T15:44:49.049338+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:10:43.664970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"61'6","reported_seq":21,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:47.254938+0000","last_change":"2024-09-16T15:44:44.691387+0000","last_active":"2024-09-16T15:44:47.254938+0000","last_peered":"2024-09-16T15:44:47.254938+0000","last_clean":"2024-09-16T15:44:47.254938+0000","last_became_active":"2024-09-16T15:44:44.691144+0000","last_became_peered":"2024-09-16T15:44:44.691144+0000","last_unstale":"2024-09-16T15:44:47.254938+0000","last_undegraded":"2024-09-16T15:44:47.254938+0000","last_fullsized":"2024-09-16T15:44:47.254938+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:09:54.112439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:47.445477+0000","last_change":"2024-09-16T15:44:45.717715+0000","last_active":"2024-09-16T15:44:47.445477+0000","last_peered":"2024-09-16T15:44:47.445477+0000","last_clean":"2024-09-16T15:44:47.445477+0000","last_became_active":"2024-09-16T15:44:45.717366+0000","last_became_peered":"2024-09-16T15:44:45.717366+0000","last_unstale":"2024-09-16T15:44:47.445477+0000","last_undegraded":"2024-09-16T15:44:47.445477+0000","last_fullsized":"2024-09-16T15:44:47.445477+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:37:22.139667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"61'2","reported_seq":17,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:46.721470+0000","last_change":"2024-09-16T15:44:44.702026+0000","last_active":"2024-09-16T15:44:46.721470+0000","last_peered":"2024-09-16T15:44:46.721470+0000","last_clean":"2024-09-16T15:44:46.721470+0000","last_became_active":"2024-09-16T15:44:44.701701+0000","last_became_peered":"2024-09-16T15:44:44.701701+0000","last_unstale":"2024-09-16T15:44:46.721470+0000","last_undegraded":"2024-09-16T15:44:46.721470+0000","last_fullsized":"2024-09-16T15:44:46.721470+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:30:25.597599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:46.719153+0000","last_change":"2024-09-16T15:44:45.734880+0000","last_active":"2024-09-16T15:44:46.719153+0000","last_peered":"2024-09-16T15:44:46.719153+0000","last_clean":"2024-09-16T15:44:46.719153+0000","last_became_active":"2024-09-16T15:44:45.734616+0000","last_became_peered":"2024-09-16T15:44:45.734616+0000","last_unstale":"2024-09-16T15:44:46.719153+0000","last_undegraded":"2024-09-16T15:44:46.719153+0000","last_fullsized":"2024-09-16T15:44:46.719153+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:13:22.287626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:49.049006+0000","last_change":"2024-09-16T15:44:44.701584+0000","last_active":"2024-09-16T15:44:49.049006+0000","last_peered":"2024-09-16T15:44:49.049006+0000","last_clean":"2024-09-16T15:44:49.049006+0000","last_became_active":"2024-09-16T15:44:44.701327+0000","last_became_peered":"2024-09-16T15:44:44.701327+0000","last_unstale":"2024-09-16T15:44:49.049006+0000","last_undegraded":"2024-09-16T15:44:49.049006+0000","last_fullsized":"2024-09-16T15:44:49.049006+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:44:05.886418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:47.445700+0000","last_change":"2024-09-16T15:44:45.717514+0000","last_active":"2024-09-16T15:44:47.445700+0000","last_peered":"2024-09-16T15:44:47.445700+0000","last_clean":"2024-09-16T15:44:47.445700+0000","last_became_active":"2024-09-16T15:44:45.717270+0000","last_became_peered":"2024-09-16T15:44:45.717270+0000","last_unstale":"2024-09-16T15:44:47.445700+0000","last_undegraded":"2024-09-16T15:44:47.445700+0000","last_fullsized":"2024-09-16T15:44:47.445700+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:18:42.857035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:49.049272+0000","last_change":"2024-09-16T15:44:44.702562+0000","last_active":"2024-09-16T15:44:49.049272+0000","last_peered":"2024-09-16T15:44:49.049272+0000","last_clean":"2024-09-16T15:44:49.049272+0000","last_became_active":"2024-09-16T15:44:44.702282+0000","last_became_peered":"2024-09-16T15:44:44.702282+0000","last_unstale":"2024-09-16T15:44:49.049272+0000","last_undegraded":"2024-09-16T15:44:49.049272+0000","last_fullsized":"2024-09-16T15:44:49.049272+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:34:40.861349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:49.049282+0000","last_change":"2024-09-16T15:44:45.717730+0000","last_active":"2024-09-16T15:44:49.049282+0000","last_peered":"2024-09-16T15:44:49.049282+0000","last_clean":"2024-09-16T15:44:49.049282+0000","last_became_active":"2024-09-16T15:44:45.717456+0000","last_became_peered":"2024-09-16T15:44:45.717456+0000","last_unstale":"2024-09-16T15:44:49.049282+0000","last_undegraded":"2024-09-16T15:44:49.049282+0000","last_fullsized":"2024-09-16T15:44:49.049282+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:08:10.410425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"61'8","reported_seq":23,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:49.049078+0000","last_change":"2024-09-16T15:44:44.700378+0000","last_active":"2024-09-16T15:44:49.049078+0000","last_peered":"2024-09-16T15:44:49.049078+0000","last_clean":"2024-09-16T15:44:49.049078+0000","last_became_active":"2024-09-16T15:44:44.700114+0000","last_became_peered":"2024-09-16T15:44:44.700114+0000","last_unstale":"2024-09-16T15:44:49.049078+0000","last_undegraded":"2024-09-16T15:44:49.049078+0000","last_fullsized":"2024-09-16T15:44:49.049078+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:46:16.051799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:46.719441+0000","last_change":"2024-09-16T15:44:45.716464+0000","last_active":"2024-09-16T15:44:46.719441+0000","last_peered":"2024-09-16T15:44:46.719441+0000","last_clean":"2024-09-16T15:44:46.719441+0000","last_became_active":"2024-09-16T15:44:45.716140+0000","last_became_peered":"2024-09-16T15:44:45.716140+0000","last_unstale":"2024-09-16T15:44:46.719441+0000","last_undegraded":"2024-09-16T15:44:46.719441+0000","last_fullsized":"2024-09-16T15:44:46.719441+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:16:57.217488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"61'1","reported_seq":16,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:47.365416+0000","last_change":"2024-09-16T15:44:44.711284+0000","last_active":"2024-09-16T15:44:47.365416+0000","last_peered":"2024-09-16T15:44:47.365416+0000","last_clean":"2024-09-16T15:44:47.365416+0000","last_became_active":"2024-09-16T15:44:44.709695+0000","last_became_peered":"2024-09-16T15:44:44.709695+0000","last_unstale":"2024-09-16T15:44:47.365416+0000","last_undegraded":"2024-09-16T15:44:47.365416+0000","last_fullsized":"2024-09-16T15:44:47.365416+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:41:35.149924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:46.719710+0000","last_change":"2024-09-16T15:44:45.714790+0000","last_active":"2024-09-16T15:44:46.719710+0000","last_peered":"2024-09-16T15:44:46.719710+0000","last_clean":"2024-09-16T15:44:46.719710+0000","last_became_active":"2024-09-16T15:44:45.713900+0000","last_became_peered":"2024-09-16T15:44:45.713900+0000","last_unstale":"2024-09-16T15:44:46.719710+0000","last_undegraded":"2024-09-16T15:44:46.719710+0000","last_fullsized":"2024-09-16T15:44:46.719710+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:33:23.442599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"61'2","reported_seq":17,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:46.722290+0000","last_change":"2024-09-16T15:44:44.698462+0000","last_active":"2024-09-16T15:44:46.722290+0000","last_peered":"2024-09-16T15:44:46.722290+0000","last_clean":"2024-09-16T15:44:46.722290+0000","last_became_active":"2024-09-16T15:44:44.698179+0000","last_became_peered":"2024-09-16T15:44:44.698179+0000","last_unstale":"2024-09-16T15:44:46.722290+0000","last_undegraded":"2024-09-16T15:44:46.722290+0000","last_fullsized":"2024-09-16T15:44:46.722290+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:13:21.284936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:47.364959+0000","last_change":"2024-09-16T15:44:45.728206+0000","last_active":"2024-09-16T15:44:47.364959+0000","last_peered":"2024-09-16T15:44:47.364959+0000","last_clean":"2024-09-16T15:44:47.364959+0000","last_became_active":"2024-09-16T15:44:45.727895+0000","last_became_peered":"2024-09-16T15:44:45.727895+0000","last_unstale":"2024-09-16T15:44:47.364959+0000","last_undegraded":"2024-09-16T15:44:47.364959+0000","last_fullsized":"2024-09-16T15:44:47.364959+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:22:53.878378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"61'1","reported_seq":16,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:46.719252+0000","last_change":"2024-09-16T15:44:44.706373+0000","last_active":"2024-09-16T15:44:46.719252+0000","last_peered":"2024-09-16T15:44:46.719252+0000","last_clean":"2024-09-16T15:44:46.719252+0000","last_became_active":"2024-09-16T15:44:44.705120+0000","last_became_peered":"2024-09-16T15:44:44.705120+0000","last_unstale":"2024-09-16T15:44:46.719252+0000","last_undegraded":"2024-09-16T15:44:46.719252+0000","last_fullsized":"2024-09-16T15:44:46.719252+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:07:54.665094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:49.049470+0000","last_change":"2024-09-16T15:44:45.726953+0000","last_active":"2024-09-16T15:44:49.049470+0000","last_peered":"2024-09-16T15:44:49.049470+0000","last_clean":"2024-09-16T15:44:49.049470+0000","last_became_active":"2024-09-16T15:44:45.723508+0000","last_became_peered":"2024-09-16T15:44:45.723508+0000","last_unstale":"2024-09-16T15:44:49.049470+0000","last_undegraded":"2024-09-16T15:44:49.049470+0000","last_fullsized":"2024-09-16T15:44:49.049470+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:13:49.455830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:46.722318+0000","last_change":"2024-09-16T15:44:44.708721+0000","last_active":"2024-09-16T15:44:46.722318+0000","last_peered":"2024-09-16T15:44:46.722318+0000","last_clean":"2024-09-16T15:44:46.722318+0000","last_became_active":"2024-09-16T15:44:44.708471+0000","last_became_peered":"2024-09-16T15:44:44.708471+0000","last_unstale":"2024-09-16T15:44:46.722318+0000","last_undegraded":"2024-09-16T15:44:46.722318+0000","last_fullsized":"2024-09-16T15:44:46.722318+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:39:49.307383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:47.445573+0000","last_change":"2024-09-16T15:44:45.716983+0000","last_active":"2024-09-16T15:44:47.445573+0000","last_peered":"2024-09-16T15:44:47.445573+0000","last_clean":"2024-09-16T15:44:47.445573+0000","last_became_active":"2024-09-16T15:44:45.715530+0000","last_became_peered":"2024-09-16T15:44:45.715530+0000","last_unstale":"2024-09-16T15:44:47.445573+0000","last_undegraded":"2024-09-16T15:44:47.445573+0000","last_fullsized":"2024-09-16T15:44:47.445573+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:37:38.312760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"61'2","reported_seq":17,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:46.722340+0000","last_change":"2024-09-16T15:44:44.696118+0000","last_active":"2024-09-16T15:44:46.722340+0000","last_peered":"2024-09-16T15:44:46.722340+0000","last_clean":"2024-09-16T15:44:46.722340+0000","last_became_active":"2024-09-16T15:44:44.695842+0000","last_became_peered":"2024-09-16T15:44:44.695842+0000","last_unstale":"2024-09-16T15:44:46.722340+0000","last_undegraded":"2024-09-16T15:44:46.722340+0000","last_fullsized":"2024-09-16T15:44:46.722340+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:31:45.212994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:46.721321+0000","last_change":"2024-09-16T15:44:45.712249+0000","last_active":"2024-09-16T15:44:46.721321+0000","last_peered":"2024-09-16T15:44:46.721321+0000","last_clean":"2024-09-16T15:44:46.721321+0000","last_became_active":"2024-09-16T15:44:45.711524+0000","last_became_peered":"2024-09-16T15:44:45.711524+0000","last_unstale":"2024-09-16T15:44:46.721321+0000","last_undegraded":"2024-09-16T15:44:46.721321+0000","last_fullsized":"2024-09-16T15:44:46.721321+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:09:14.114379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:46.721648+0000","last_change":"2024-09-16T15:44:44.695883+0000","last_active":"2024-09-16T15:44:46.721648+0000","last_peered":"2024-09-16T15:44:46.721648+0000","last_clean":"2024-09-16T15:44:46.721648+0000","last_became_active":"2024-09-16T15:44:44.695562+0000","last_became_peered":"2024-09-16T15:44:44.695562+0000","last_unstale":"2024-09-16T15:44:46.721648+0000","last_undegraded":"2024-09-16T15:44:46.721648+0000","last_fullsized":"2024-09-16T15:44:46.721648+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:22:24.705292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:46.720834+0000","last_change":"2024-09-16T15:44:45.708395+0000","last_active":"2024-09-16T15:44:46.720834+0000","last_peered":"2024-09-16T15:44:46.720834+0000","last_clean":"2024-09-16T15:44:46.720834+0000","last_became_active":"2024-09-16T15:44:45.708090+0000","last_became_peered":"2024-09-16T15:44:45.708090+0000","last_unstale":"2024-09-16T15:44:46.720834+0000","last_undegraded":"2024-09-16T15:44:46.720834+0000","last_fullsized":"2024-09-16T15:44:46.720834+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:34:54.933570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:47.445757+0000","last_change":"2024-09-16T15:44:44.705798+0000","last_active":"2024-09-16T15:44:47.445757+0000","last_peered":"2024-09-16T15:44:47.445757+0000","last_clean":"2024-09-16T15:44:47.445757+0000","last_became_active":"2024-09-16T15:44:44.705468+0000","last_became_peered":"2024-09-16T15:44:44.705468+0000","last_unstale":"2024-09-16T15:44:47.445757+0000","last_undegraded":"2024-09-16T15:44:47.445757+0000","last_fullsized":"2024-09-16T15:44:47.445757+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:21:59.839074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:46.719628+0000","last_change":"2024-09-16T15:44:45.726903+0000","last_active":"2024-09-16T15:44:46.719628+0000","last_peered":"2024-09-16T15:44:46.719628+0000","last_clean":"2024-09-16T15:44:46.719628+0000","last_became_active":"2024-09-16T15:44:45.726666+0000","last_became_peered":"2024-09-16T15:44:45.726666+0000","last_unstale":"2024-09-16T15:44:46.719628+0000","last_undegraded":"2024-09-16T15:44:46.719628+0000","last_fullsized":"2024-09-16T15:44:46.719628+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:23:06.419124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"61'2","reported_seq":17,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:47.365542+0000","last_change":"2024-09-16T15:44:44.708470+0000","last_active":"2024-09-16T15:44:47.365542+0000","last_peered":"2024-09-16T15:44:47.365542+0000","last_clean":"2024-09-16T15:44:47.365542+0000","last_became_active":"2024-09-16T15:44:44.708182+0000","last_became_peered":"2024-09-16T15:44:44.708182+0000","last_unstale":"2024-09-16T15:44:47.365542+0000","last_undegraded":"2024-09-16T15:44:47.365542+0000","last_fullsized":"2024-09-16T15:44:47.365542+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:49:00.269048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:46.719769+0000","last_change":"2024-09-16T15:44:45.707416+0000","last_active":"2024-09-16T15:44:46.719769+0000","last_peered":"2024-09-16T15:44:46.719769+0000","last_clean":"2024-09-16T15:44:46.719769+0000","last_became_active":"2024-09-16T15:44:45.707148+0000","last_became_peered":"2024-09-16T15:44:45.707148+0000","last_unstale":"2024-09-16T15:44:46.719769+0000","last_undegraded":"2024-09-16T15:44:46.719769+0000","last_fullsized":"2024-09-16T15:44:46.719769+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:20:02.525231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"61'1","reported_seq":16,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:49.050397+0000","last_change":"2024-09-16T15:44:44.710583+0000","last_active":"2024-09-16T15:44:49.050397+0000","last_peered":"2024-09-16T15:44:49.050397+0000","last_clean":"2024-09-16T15:44:49.050397+0000","last_became_active":"2024-09-16T15:44:44.710293+0000","last_became_peered":"2024-09-16T15:44:44.710293+0000","last_unstale":"2024-09-16T15:44:49.050397+0000","last_undegraded":"2024-09-16T15:44:49.050397+0000","last_fullsized":"2024-09-16T15:44:49.050397+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:03:46.514703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:47.365101+0000","last_change":"2024-09-16T15:44:45.725330+0000","last_active":"2024-09-16T15:44:47.365101+0000","last_peered":"2024-09-16T15:44:47.365101+0000","last_clean":"2024-09-16T15:44:47.365101+0000","last_became_active":"2024-09-16T15:44:45.724571+0000","last_became_peered":"2024-09-16T15:44:45.724571+0000","last_unstale":"2024-09-16T15:44:47.365101+0000","last_undegraded":"2024-09-16T15:44:47.365101+0000","last_fullsized":"2024-09-16T15:44:47.365101+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:16:16.752459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:47.445523+0000","last_change":"2024-09-16T15:44:44.703904+0000","last_active":"2024-09-16T15:44:47.445523+0000","last_peered":"2024-09-16T15:44:47.445523+0000","last_clean":"2024-09-16T15:44:47.445523+0000","last_became_active":"2024-09-16T15:44:44.703644+0000","last_became_peered":"2024-09-16T15:44:44.703644+0000","last_unstale":"2024-09-16T15:44:47.445523+0000","last_undegraded":"2024-09-16T15:44:47.445523+0000","last_fullsized":"2024-09-16T15:44:47.445523+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:31:17.501469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:46.722292+0000","last_change":"2024-09-16T15:44:45.714221+0000","last_active":"2024-09-16T15:44:46.722292+0000","last_peered":"2024-09-16T15:44:46.722292+0000","last_clean":"2024-09-16T15:44:46.722292+0000","last_became_active":"2024-09-16T15:44:45.713871+0000","last_became_peered":"2024-09-16T15:44:45.713871+0000","last_unstale":"2024-09-16T15:44:46.722292+0000","last_undegraded":"2024-09-16T15:44:46.722292+0000","last_fullsized":"2024-09-16T15:44:46.722292+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:06:38.035205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:46.719375+0000","last_change":"2024-09-16T15:44:44.698858+0000","last_active":"2024-09-16T15:44:46.719375+0000","last_peered":"2024-09-16T15:44:46.719375+0000","last_clean":"2024-09-16T15:44:46.719375+0000","last_became_active":"2024-09-16T15:44:44.698546+0000","last_became_peered":"2024-09-16T15:44:44.698546+0000","last_unstale":"2024-09-16T15:44:46.719375+0000","last_undegraded":"2024-09-16T15:44:46.719375+0000","last_fullsized":"2024-09-16T15:44:46.719375+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:03:16.348362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:49.049758+0000","last_change":"2024-09-16T15:44:45.719192+0000","last_active":"2024-09-16T15:44:49.049758+0000","last_peered":"2024-09-16T15:44:49.049758+0000","last_clean":"2024-09-16T15:44:49.049758+0000","last_became_active":"2024-09-16T15:44:45.716454+0000","last_became_peered":"2024-09-16T15:44:45.716454+0000","last_unstale":"2024-09-16T15:44:49.049758+0000","last_undegraded":"2024-09-16T15:44:49.049758+0000","last_fullsized":"2024-09-16T15:44:49.049758+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:11:14.213089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:47.364959+0000","last_change":"2024-09-16T15:44:44.707043+0000","last_active":"2024-09-16T15:44:47.364959+0000","last_peered":"2024-09-16T15:44:47.364959+0000","last_clean":"2024-09-16T15:44:47.364959+0000","last_became_active":"2024-09-16T15:44:44.706747+0000","last_became_peered":"2024-09-16T15:44:44.706747+0000","last_unstale":"2024-09-16T15:44:47.364959+0000","last_undegraded":"2024-09-16T15:44:47.364959+0000","last_fullsized":"2024-09-16T15:44:47.364959+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:46:03.107567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:47.254899+0000","last_change":"2024-09-16T15:44:45.726366+0000","last_active":"2024-09-16T15:44:47.254899+0000","last_peered":"2024-09-16T15:44:47.254899+0000","last_clean":"2024-09-16T15:44:47.254899+0000","last_became_active":"2024-09-16T15:44:45.726122+0000","last_became_peered":"2024-09-16T15:44:45.726122+0000","last_unstale":"2024-09-16T15:44:47.254899+0000","last_undegraded":"2024-09-16T15:44:47.254899+0000","last_fullsized":"2024-09-16T15:44:47.254899+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:55:28.172055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:46.719040+0000","last_change":"2024-09-16T15:44:44.704189+0000","last_active":"2024-09-16T15:44:46.719040+0000","last_peered":"2024-09-16T15:44:46.719040+0000","last_clean":"2024-09-16T15:44:46.719040+0000","last_became_active":"2024-09-16T15:44:44.703916+0000","last_became_peered":"2024-09-16T15:44:44.703916+0000","last_unstale":"2024-09-16T15:44:46.719040+0000","last_undegraded":"2024-09-16T15:44:46.719040+0000","last_fullsized":"2024-09-16T15:44:46.719040+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:50:09.467231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:46.720786+0000","last_change":"2024-09-16T15:44:45.710420+0000","last_active":"2024-09-16T15:44:46.720786+0000","last_peered":"2024-09-16T15:44:46.720786+0000","last_clean":"2024-09-16T15:44:46.720786+0000","last_became_active":"2024-09-16T15:44:45.710144+0000","last_became_peered":"2024-09-16T15:44:45.710144+0000","last_unstale":"2024-09-16T15:44:46.720786+0000","last_undegraded":"2024-09-16T15:44:46.720786+0000","last_fullsized":"2024-09-16T15:44:46.720786+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:46:16.636165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:49.049920+0000","last_change":"2024-09-16T15:44:44.708397+0000","last_active":"2024-09-16T15:44:49.049920+0000","last_peered":"2024-09-16T15:44:49.049920+0000","last_clean":"2024-09-16T15:44:49.049920+0000","last_became_active":"2024-09-16T15:44:44.708101+0000","last_became_peered":"2024-09-16T15:44:44.708101+0000","last_unstale":"2024-09-16T15:44:49.049920+0000","last_undegraded":"2024-09-16T15:44:49.049920+0000","last_fullsized":"2024-09-16T15:44:49.049920+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:09:26.705708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:47.364975+0000","last_change":"2024-09-16T15:44:45.717063+0000","last_active":"2024-09-16T15:44:47.364975+0000","last_peered":"2024-09-16T15:44:47.364975+0000","last_clean":"2024-09-16T15:44:47.364975+0000","last_became_active":"2024-09-16T15:44:45.716754+0000","last_became_peered":"2024-09-16T15:44:45.716754+0000","last_unstale":"2024-09-16T15:44:47.364975+0000","last_undegraded":"2024-09-16T15:44:47.364975+0000","last_fullsized":"2024-09-16T15:44:47.364975+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:05:39.558163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:46.720989+0000","last_change":"2024-09-16T15:44:44.698425+0000","last_active":"2024-09-16T15:44:46.720989+0000","last_peered":"2024-09-16T15:44:46.720989+0000","last_clean":"2024-09-16T15:44:46.720989+0000","last_became_active":"2024-09-16T15:44:44.698141+0000","last_became_peered":"2024-09-16T15:44:44.698141+0000","last_unstale":"2024-09-16T15:44:46.720989+0000","last_undegraded":"2024-09-16T15:44:46.720989+0000","last_fullsized":"2024-09-16T15:44:46.720989+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:55:27.169366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:46.721145+0000","last_change":"2024-09-16T15:44:45.720393+0000","last_active":"2024-09-16T15:44:46.721145+0000","last_peered":"2024-09-16T15:44:46.721145+0000","last_clean":"2024-09-16T15:44:46.721145+0000","last_became_active":"2024-09-16T15:44:45.720028+0000","last_became_peered":"2024-09-16T15:44:45.720028+0000","last_unstale":"2024-09-16T15:44:46.721145+0000","last_undegraded":"2024-09-16T15:44:46.721145+0000","last_fullsized":"2024-09-16T15:44:46.721145+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:44.679289+0000","last_clean_scrub_stamp":"2024-09-16T15:44:44.679289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:38:03.977000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:47.254558+0000","last_change":"2024-09-16T15:44:44.690580+0000","last_active":"2024-09-16T15:44:47.254558+0000","last_peered":"2024-09-16T15:44:47.254558+0000","last_clean":"2024-09-16T15:44:47.254558+0000","last_became_active":"2024-09-16T15:44:44.690302+0000","last_became_peered":"2024-09-16T15:44:44.690302+0000","last_unstale":"2024-09-16T15:44:47.254558+0000","last_undegraded":"2024-09-16T15:44:47.254558+0000","last_fullsized":"2024-09-16T15:44:47.254558+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:44:43.676599+0000","last_clean_scrub_stamp":"2024-09-16T15:44:43.676599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:45:53.368994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":187,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T15:44:47.445797+0000","last_change":"2024-09-16T15:39:12.698813+0000","last_active":"2024-09-16T15:44:47.445797+0000","last_peered":"2024-09-16T15:44:47.445797+0000","last_clean":"2024-09-16T15:44:47.445797+0000","last_became_active":"2024-09-16T15:39:12.697695+0000","last_became_peered":"2024-09-16T15:39:12.697695+0000","last_unstale":"2024-09-16T15:44:47.445797+0000","last_undegraded":"2024-09-16T15:44:47.445797+0000","last_fullsized":"2024-09-16T15:44:47.445797+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_clean_scrub_stamp":"2024-09-16T15:39:11.684967+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:02.061081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509513,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40232,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331608,"statfs":{"total":96636764160,"available":96595566592,"internally_reserved":0,"allocated":741376,"data_stored":309959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":40434120},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509512,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46380,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325460,"statfs":{"total":96636764160,"available":96589271040,"internally_reserved":0,"allocated":1335296,"data_stored":900339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509511,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45812,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326028,"statfs":{"total":96636764160,"available":96589852672,"internally_reserved":0,"allocated":745472,"data_stored":310553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509511,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40232,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331608,"statfs":{"total":96636764160,"available":96595566592,"internally_reserved":0,"allocated":741376,"data_stored":310027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":40434121},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509511,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62456,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94309384,"statfs":{"total":96636764160,"available":96572809216,"internally_reserved":0,"allocated":757760,"data_stored":311709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":63175111},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509512,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40816,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331024,"statfs":{"total":96636764160,"available":96594968576,"internally_reserved":0,"allocated":1335296,"data_stored":900887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509510,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40240,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331600,"statfs":{"total":96636764160,"available":96595558400,"internally_reserved":0,"allocated":749568,"data_stored":311549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542217,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40240,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331600,"statfs":{"total":96636764160,"available":96595558400,"internally_reserved":0,"allocated":745472,"data_stored":310029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":40434124},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:44:55.040 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T15:44:55.041 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-16T15:44:55.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.240+0000 7f31ab000640 1 -- 172.21.15.151:0/480184127 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31a40c5730 msgr2=0x7f31a415fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:55.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.240+0000 7f31ab000640 1 --2- 172.21.15.151:0/480184127 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31a40c5730 0x7f31a415fa50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3190009830 tx=0x7f319002a520 comp rx=0 tx=0).stop 2024-09-16T15:44:55.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.240+0000 7f31ab000640 1 -- 172.21.15.151:0/480184127 shutdown_connections 2024-09-16T15:44:55.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.240+0000 7f31ab000640 1 --2- 172.21.15.151:0/480184127 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31a40c5730 0x7f31a415fa50 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:55.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.240+0000 7f31ab000640 1 --2- 172.21.15.151:0/480184127 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31a415f670 0x7f31a40c6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:55.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.240+0000 7f31ab000640 1 -- 172.21.15.151:0/480184127 >> 172.21.15.151:0/480184127 conn(0x7f31a40cc490 msgr2=0x7f31a40cc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:55.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.240+0000 7f31ab000640 1 -- 172.21.15.151:0/480184127 shutdown_connections 2024-09-16T15:44:55.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.240+0000 7f31ab000640 1 -- 172.21.15.151:0/480184127 wait complete. 2024-09-16T15:44:55.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.242+0000 7f31ab000640 1 Processor -- start 2024-09-16T15:44:55.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.242+0000 7f31ab000640 1 -- start start 2024-09-16T15:44:55.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.243+0000 7f31ab000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31a40c5730 0x7f31a40afb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:55.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.243+0000 7f31ab000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31a415c0f0 0x7f31a40b0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:55.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.243+0000 7f31ab000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f31a415f670 0x7f31a41f4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:55.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.243+0000 7f31ab000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f31a40ca370 con 0x7f31a415f670 2024-09-16T15:44:55.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.243+0000 7f31ab000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f31a40700d0 con 0x7f31a415c0f0 2024-09-16T15:44:55.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.243+0000 7f31ab000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f31a40ca4f0 con 0x7f31a40c5730 2024-09-16T15:44:55.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.243+0000 7f31aa600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f31a415f670 0x7f31a41f4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:55.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.243+0000 7f31aa600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f31a415f670 0x7f31a41f4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:39474/0 (socket says 172.21.15.151:39474) 2024-09-16T15:44:55.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.243+0000 7f31aa600640 1 -- 172.21.15.151:0/2612948897 learned_addr learned my addr 172.21.15.151:0/2612948897 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:55.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.243+0000 7f31a9c00640 1 --2- 172.21.15.151:0/2612948897 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31a40c5730 0x7f31a40afb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:55.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.243+0000 7f31a9200640 1 --2- 172.21.15.151:0/2612948897 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31a415c0f0 0x7f31a40b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:55.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.244+0000 7f31aa600640 1 -- 172.21.15.151:0/2612948897 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31a40c5730 msgr2=0x7f31a40afb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:55.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.244+0000 7f31aa600640 1 --2- 172.21.15.151:0/2612948897 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31a40c5730 0x7f31a40afb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:55.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.244+0000 7f31aa600640 1 -- 172.21.15.151:0/2612948897 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31a415c0f0 msgr2=0x7f31a40b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:55.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.244+0000 7f31aa600640 1 --2- 172.21.15.151:0/2612948897 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31a415c0f0 0x7f31a40b0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:55.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.244+0000 7f31aa600640 1 -- 172.21.15.151:0/2612948897 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31a40cd550 con 0x7f31a415f670 2024-09-16T15:44:55.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.244+0000 7f31aa600640 1 --2- 172.21.15.151:0/2612948897 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f31a415f670 0x7f31a41f4950 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f319c00cbc0 tx=0x7f319c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:55.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.244+0000 7f31a1600640 1 -- 172.21.15.151:0/2612948897 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f319c016020 con 0x7f31a415f670 2024-09-16T15:44:55.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.244+0000 7f31a1600640 1 -- 172.21.15.151:0/2612948897 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f319c00ad90 con 0x7f31a415f670 2024-09-16T15:44:55.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.244+0000 7f31a1600640 1 -- 172.21.15.151:0/2612948897 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f319c012680 con 0x7f31a415f670 2024-09-16T15:44:55.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.244+0000 7f31ab000640 1 -- 172.21.15.151:0/2612948897 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31a40b0740 con 0x7f31a415f670 2024-09-16T15:44:55.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.245+0000 7f31ab000640 1 -- 172.21.15.151:0/2612948897 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31a40b0d00 con 0x7f31a415f670 2024-09-16T15:44:55.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.246+0000 7f31ab000640 1 -- 172.21.15.151:0/2612948897 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31a415ce30 con 0x7f31a415f670 2024-09-16T15:44:55.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.247+0000 7f31a1600640 1 -- 172.21.15.151:0/2612948897 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 102103+0+0 (secure 0 0 0) 0x7f319c019020 con 0x7f31a415f670 2024-09-16T15:44:55.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.248+0000 7f31a1600640 1 --2- 172.21.15.151:0/2612948897 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3184079150 0x7f318407b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:55.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.248+0000 7f31a1600640 1 -- 172.21.15.151:0/2612948897 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7f319c0996f0 con 0x7f31a415f670 2024-09-16T15:44:55.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.248+0000 7f31a9c00640 1 --2- 172.21.15.151:0/2612948897 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3184079150 0x7f318407b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:55.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.249+0000 7f31a9c00640 1 --2- 172.21.15.151:0/2612948897 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3184079150 0x7f318407b610 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3198009b30 tx=0x7f3198005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:55.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.252+0000 7f31a1600640 1 -- 172.21.15.151:0/2612948897 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f319c020020 con 0x7f31a415f670 2024-09-16T15:44:55.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.507+0000 7f31ab000640 1 -- 172.21.15.151:0/2612948897 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f31a415da50 con 0x7f31a415f670 2024-09-16T15:44:55.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.509+0000 7f31a1600640 1 -- 172.21.15.151:0/2612948897 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) ==== 190+0+147 (secure 0 0 0) 0x7f319c063160 con 0x7f31a415f670 2024-09-16T15:44:55.511 INFO:teuthology.orchestra.run.smithi151.stdout:[client.0] 2024-09-16T15:44:55.511 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQDsUOhmf5PBKBAAKE4Y4zbA4h07IBNjPkyptw== 2024-09-16T15:44:55.511 INFO:teuthology.orchestra.run.smithi151.stdout: caps mds = "allow" 2024-09-16T15:44:55.511 INFO:teuthology.orchestra.run.smithi151.stdout: caps mon = "allow r" 2024-09-16T15:44:55.511 INFO:teuthology.orchestra.run.smithi151.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-16T15:44:55.511 INFO:teuthology.orchestra.run.smithi151.stderr:updated caps for client.0 2024-09-16T15:44:55.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.512+0000 7f31ab000640 1 -- 172.21.15.151:0/2612948897 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3184079150 msgr2=0x7f318407b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:55.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.512+0000 7f31ab000640 1 --2- 172.21.15.151:0/2612948897 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3184079150 0x7f318407b610 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3198009b30 tx=0x7f3198005f90 comp rx=0 tx=0).stop 2024-09-16T15:44:55.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.512+0000 7f31ab000640 1 -- 172.21.15.151:0/2612948897 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f31a415f670 msgr2=0x7f31a41f4950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:55.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.512+0000 7f31ab000640 1 --2- 172.21.15.151:0/2612948897 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f31a415f670 0x7f31a41f4950 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f319c00cbc0 tx=0x7f319c00a060 comp rx=0 tx=0).stop 2024-09-16T15:44:55.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.512+0000 7f31ab000640 1 -- 172.21.15.151:0/2612948897 shutdown_connections 2024-09-16T15:44:55.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.512+0000 7f31ab000640 1 --2- 172.21.15.151:0/2612948897 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3184079150 0x7f318407b610 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:55.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.512+0000 7f31ab000640 1 --2- 172.21.15.151:0/2612948897 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f31a415c0f0 0x7f31a40b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:55.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.512+0000 7f31ab000640 1 --2- 172.21.15.151:0/2612948897 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f31a415f670 0x7f31a41f4950 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:55.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.512+0000 7f31ab000640 1 --2- 172.21.15.151:0/2612948897 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f31a40c5730 0x7f31a40afb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:55.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.512+0000 7f31ab000640 1 -- 172.21.15.151:0/2612948897 >> 172.21.15.151:0/2612948897 conn(0x7f31a40cc490 msgr2=0x7f31a41520a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:55.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.512+0000 7f31ab000640 1 -- 172.21.15.151:0/2612948897 shutdown_connections 2024-09-16T15:44:55.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.512+0000 7f31ab000640 1 -- 172.21.15.151:0/2612948897 wait complete. 2024-09-16T15:44:55.530 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:44:55.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.686+0000 7f71ed800640 1 -- 172.21.15.151:0/1985984563 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71e80ccf00 msgr2=0x7f71e80b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:55.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.686+0000 7f71ed800640 1 --2- 172.21.15.151:0/1985984563 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71e80ccf00 0x7f71e80b2440 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f71d4009830 tx=0x7f71d402a480 comp rx=0 tx=0).stop 2024-09-16T15:44:55.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.686+0000 7f71ed800640 1 -- 172.21.15.151:0/1985984563 shutdown_connections 2024-09-16T15:44:55.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.686+0000 7f71ed800640 1 --2- 172.21.15.151:0/1985984563 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71e80ccf00 0x7f71e80b2440 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:55.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.686+0000 7f71ed800640 1 -- 172.21.15.151:0/1985984563 >> 172.21.15.151:0/1985984563 conn(0x7f71e80b1950 msgr2=0x7f71e80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:55.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.686+0000 7f71ed800640 1 -- 172.21.15.151:0/1985984563 shutdown_connections 2024-09-16T15:44:55.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.686+0000 7f71ed800640 1 -- 172.21.15.151:0/1985984563 wait complete. 2024-09-16T15:44:55.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.687+0000 7f71ed800640 1 Processor -- start 2024-09-16T15:44:55.689 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.688+0000 7f71ed800640 1 -- start start 2024-09-16T15:44:55.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.688+0000 7f71ed800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71e8073fb0 0x7f71e81f8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:55.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.688+0000 7f71ed800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f71e80ccf00 0x7f71e8158630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:55.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.688+0000 7f71ed800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71e815f5a0 0x7f71e8159610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:55.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.688+0000 7f71ed800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f71e806f9f0 con 0x7f71e815f5a0 2024-09-16T15:44:55.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.688+0000 7f71ed800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f71e806f870 con 0x7f71e8073fb0 2024-09-16T15:44:55.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.689+0000 7f71ed800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f71e8074420 con 0x7f71e80ccf00 2024-09-16T15:44:55.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.689+0000 7f71e7400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71e815f5a0 0x7f71e8159610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:55.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.689+0000 7f71e7400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71e815f5a0 0x7f71e8159610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:39496/0 (socket says 172.21.15.151:39496) 2024-09-16T15:44:55.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.689+0000 7f71e7400640 1 -- 172.21.15.151:0/2718494040 learned_addr learned my addr 172.21.15.151:0/2718494040 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:55.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.689+0000 7f71e6000640 1 --2- 172.21.15.151:0/2718494040 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f71e80ccf00 0x7f71e8158630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:55.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.689+0000 7f71e6a00640 1 --2- 172.21.15.151:0/2718494040 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71e8073fb0 0x7f71e81f8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:55.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.689+0000 7f71e7400640 1 -- 172.21.15.151:0/2718494040 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f71e80ccf00 msgr2=0x7f71e8158630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:55.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.689+0000 7f71e7400640 1 --2- 172.21.15.151:0/2718494040 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f71e80ccf00 0x7f71e8158630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:55.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.689+0000 7f71e7400640 1 -- 172.21.15.151:0/2718494040 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71e8073fb0 msgr2=0x7f71e81f8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:55.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.689+0000 7f71e7400640 1 --2- 172.21.15.151:0/2718494040 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71e8073fb0 0x7f71e81f8d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:55.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.689+0000 7f71e7400640 1 -- 172.21.15.151:0/2718494040 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71e81f92a0 con 0x7f71e815f5a0 2024-09-16T15:44:55.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.689+0000 7f71e6a00640 1 --2- 172.21.15.151:0/2718494040 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71e8073fb0 0x7f71e81f8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:44:55.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.689+0000 7f71e7400640 1 --2- 172.21.15.151:0/2718494040 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71e815f5a0 0x7f71e8159610 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f71e000c770 tx=0x7f71e000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:55.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.690+0000 7f71dea00640 1 -- 172.21.15.151:0/2718494040 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e0003da0 con 0x7f71e815f5a0 2024-09-16T15:44:55.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.690+0000 7f71dea00640 1 -- 172.21.15.151:0/2718494040 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f71e0005070 con 0x7f71e815f5a0 2024-09-16T15:44:55.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.690+0000 7f71dea00640 1 -- 172.21.15.151:0/2718494040 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e00054c0 con 0x7f71e815f5a0 2024-09-16T15:44:55.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.690+0000 7f71ed800640 1 -- 172.21.15.151:0/2718494040 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71e81f4e70 con 0x7f71e815f5a0 2024-09-16T15:44:55.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.690+0000 7f71ed800640 1 -- 172.21.15.151:0/2718494040 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71e81f5430 con 0x7f71e815f5a0 2024-09-16T15:44:55.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.691+0000 7f71ed800640 1 -- 172.21.15.151:0/2718494040 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71e80b2440 con 0x7f71e815f5a0 2024-09-16T15:44:55.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.692+0000 7f71dea00640 1 -- 172.21.15.151:0/2718494040 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 102103+0+0 (secure 0 0 0) 0x7f71e0005660 con 0x7f71e815f5a0 2024-09-16T15:44:55.694 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.693+0000 7f71dea00640 1 --2- 172.21.15.151:0/2718494040 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f71c8079150 0x7f71c807b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:55.694 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.693+0000 7f71dea00640 1 -- 172.21.15.151:0/2718494040 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7f71e0099020 con 0x7f71e815f5a0 2024-09-16T15:44:55.694 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.693+0000 7f71e6a00640 1 --2- 172.21.15.151:0/2718494040 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f71c8079150 0x7f71c807b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:55.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.694+0000 7f71e6a00640 1 --2- 172.21.15.151:0/2718494040 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f71c8079150 0x7f71c807b610 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f71d8006fd0 tx=0x7f71d8008040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:55.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.697+0000 7f71dea00640 1 -- 172.21.15.151:0/2718494040 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f71e81f92a0 con 0x7f71e815f5a0 2024-09-16T15:44:55.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.919+0000 7f71ed800640 1 -- 172.21.15.151:0/2718494040 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f71e8158b70 con 0x7f71e815f5a0 2024-09-16T15:44:55.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.919+0000 7f71dea00640 1 -- 172.21.15.151:0/2718494040 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5232 (secure 0 0 0) 0x7f71e00630e0 con 0x7f71e815f5a0 2024-09-16T15:44:55.920 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:44:55.921 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":36,"btime":"2024-09-16T15:44:48:723811+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5357,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1715048770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1715048770},{"type":"v1","addr":"172.21.15.167:6837","nonce":1715048770}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":5722,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/784205427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":784205427},{"type":"v1","addr":"172.21.15.167:6836","nonce":784205427}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":5958,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/3837674321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3837674321},{"type":"v1","addr":"172.21.15.151:6834","nonce":3837674321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:44:45.114621+0000","modified":"2024-09-16T15:44:48.149095+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5961},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5961":{"gid":5961,"name":"c","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.151:6835/1724192711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":1724192711},{"type":"v1","addr":"172.21.15.151:6835","nonce":1724192711}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5961,"qdb_cluster":[5961]},"id":4}]} 2024-09-16T15:44:55.921 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 36 2024-09-16T15:44:55.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.922+0000 7f71ed800640 1 -- 172.21.15.151:0/2718494040 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f71c8079150 msgr2=0x7f71c807b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:55.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.922+0000 7f71ed800640 1 --2- 172.21.15.151:0/2718494040 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f71c8079150 0x7f71c807b610 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f71d8006fd0 tx=0x7f71d8008040 comp rx=0 tx=0).stop 2024-09-16T15:44:55.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.922+0000 7f71ed800640 1 -- 172.21.15.151:0/2718494040 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71e815f5a0 msgr2=0x7f71e8159610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:55.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.922+0000 7f71ed800640 1 --2- 172.21.15.151:0/2718494040 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71e815f5a0 0x7f71e8159610 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f71e000c770 tx=0x7f71e000cc30 comp rx=0 tx=0).stop 2024-09-16T15:44:55.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.922+0000 7f71ed800640 1 -- 172.21.15.151:0/2718494040 shutdown_connections 2024-09-16T15:44:55.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.922+0000 7f71ed800640 1 --2- 172.21.15.151:0/2718494040 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f71c8079150 0x7f71c807b610 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:55.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.922+0000 7f71ed800640 1 --2- 172.21.15.151:0/2718494040 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71e815f5a0 0x7f71e8159610 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:55.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.922+0000 7f71ed800640 1 --2- 172.21.15.151:0/2718494040 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f71e80ccf00 0x7f71e8158630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:55.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.922+0000 7f71ed800640 1 --2- 172.21.15.151:0/2718494040 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71e8073fb0 0x7f71e81f8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:55.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.922+0000 7f71ed800640 1 -- 172.21.15.151:0/2718494040 >> 172.21.15.151:0/2718494040 conn(0x7f71e80b1950 msgr2=0x7f71e80b4280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:55.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.922+0000 7f71ed800640 1 -- 172.21.15.151:0/2718494040 shutdown_connections 2024-09-16T15:44:55.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:55.922+0000 7f71ed800640 1 -- 172.21.15.151:0/2718494040 wait complete. 2024-09-16T15:44:56.341 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 36, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-16T15:44:45.114621+0000', 'modified': '2024-09-16T15:44:48.149095+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5961}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5961': {'gid': 5961, 'name': 'c', 'rank': 0, 'incarnation': 33, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.151:6835/1724192711', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6833', 'nonce': 1724192711}, {'type': 'v1', 'addr': '172.21.15.151:6835', 'nonce': 1724192711}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5961, 'qdb_cluster': [5961]} 2024-09-16T15:44:56.342 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-16T15:44:56.342 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-16T15:44:56.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.503+0000 7f10ca200640 1 -- 172.21.15.151:0/2567009293 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10a8004800 msgr2=0x7f10a8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:56.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.503+0000 7f10ca200640 1 --2- 172.21.15.151:0/2567009293 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10a8004800 0x7f10a8004c80 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f10b0001830 tx=0x7f10a8005830 comp rx=0 tx=0).stop 2024-09-16T15:44:56.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.504+0000 7f10ca200640 1 -- 172.21.15.151:0/2567009293 shutdown_connections 2024-09-16T15:44:56.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.504+0000 7f10ca200640 1 --2- 172.21.15.151:0/2567009293 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10a8004800 0x7f10a8004c80 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:56.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.504+0000 7f10ca200640 1 -- 172.21.15.151:0/2567009293 >> 172.21.15.151:0/2567009293 conn(0x7f10c40b1950 msgr2=0x7f10c40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:56.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.504+0000 7f10ca200640 1 -- 172.21.15.151:0/2567009293 shutdown_connections 2024-09-16T15:44:56.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.504+0000 7f10ca200640 1 -- 172.21.15.151:0/2567009293 wait complete. 2024-09-16T15:44:56.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.505+0000 7f10ca200640 1 Processor -- start 2024-09-16T15:44:56.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.505+0000 7f10ca200640 1 -- start start 2024-09-16T15:44:56.509 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.506+0000 7f10ca200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10a8004800 0x7f10c40cdd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:56.509 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.506+0000 7f10ca200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10c415bca0 0x7f10c4151680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:56.509 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.506+0000 7f10ca200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f10c40ce610 0x7f10c40cea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:56.509 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.506+0000 7f10ca200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f10c4074420 con 0x7f10c415bca0 2024-09-16T15:44:56.509 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.506+0000 7f10ca200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f10c40b2700 con 0x7f10a8004800 2024-09-16T15:44:56.509 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.506+0000 7f10ca200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f10c40745a0 con 0x7f10c40ce610 2024-09-16T15:44:56.509 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.507+0000 7f10c2a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10c415bca0 0x7f10c4151680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:56.509 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.507+0000 7f10c2a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10c415bca0 0x7f10c4151680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:39520/0 (socket says 172.21.15.151:39520) 2024-09-16T15:44:56.509 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.507+0000 7f10c2a00640 1 -- 172.21.15.151:0/2869972189 learned_addr learned my addr 172.21.15.151:0/2869972189 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:56.509 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.507+0000 7f10c3400640 1 --2- 172.21.15.151:0/2869972189 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10a8004800 0x7f10c40cdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:56.509 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.507+0000 7f10c3e00640 1 --2- 172.21.15.151:0/2869972189 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f10c40ce610 0x7f10c40cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:56.509 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.507+0000 7f10c2a00640 1 -- 172.21.15.151:0/2869972189 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f10c40ce610 msgr2=0x7f10c40cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:56.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.507+0000 7f10c2a00640 1 --2- 172.21.15.151:0/2869972189 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f10c40ce610 0x7f10c40cea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:56.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.507+0000 7f10c2a00640 1 -- 172.21.15.151:0/2869972189 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10a8004800 msgr2=0x7f10c40cdd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:56.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.507+0000 7f10c2a00640 1 --2- 172.21.15.151:0/2869972189 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10a8004800 0x7f10c40cdd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:56.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.507+0000 7f10c2a00640 1 -- 172.21.15.151:0/2869972189 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10c415c1d0 con 0x7f10c415bca0 2024-09-16T15:44:56.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.507+0000 7f10c3e00640 1 --2- 172.21.15.151:0/2869972189 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f10c40ce610 0x7f10c40cea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:44:56.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.507+0000 7f10c2a00640 1 --2- 172.21.15.151:0/2869972189 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10c415bca0 0x7f10c4151680 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f10b002aa40 tx=0x7f10b0006220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:56.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.508+0000 7f10bb400640 1 -- 172.21.15.151:0/2869972189 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b0043070 con 0x7f10c415bca0 2024-09-16T15:44:56.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.508+0000 7f10bb400640 1 -- 172.21.15.151:0/2869972189 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f10b0003af0 con 0x7f10c415bca0 2024-09-16T15:44:56.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.508+0000 7f10bb400640 1 -- 172.21.15.151:0/2869972189 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b0033480 con 0x7f10c415bca0 2024-09-16T15:44:56.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.508+0000 7f10ca200640 1 -- 172.21.15.151:0/2869972189 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10c415c4b0 con 0x7f10c415bca0 2024-09-16T15:44:56.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.508+0000 7f10ca200640 1 -- 172.21.15.151:0/2869972189 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10c415ca70 con 0x7f10c415bca0 2024-09-16T15:44:56.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.510+0000 7f10bb400640 1 -- 172.21.15.151:0/2869972189 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 102103+0+0 (secure 0 0 0) 0x7f10b0046070 con 0x7f10c415bca0 2024-09-16T15:44:56.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.511+0000 7f10bb400640 1 --2- 172.21.15.151:0/2869972189 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f10a4079150 0x7f10a407b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:56.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.511+0000 7f10bb400640 1 -- 172.21.15.151:0/2869972189 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7f10b00b8f00 con 0x7f10c415bca0 2024-09-16T15:44:56.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.511+0000 7f10cd25e740 1 Processor -- start 2024-09-16T15:44:56.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.511+0000 7f10c3400640 1 --2- 172.21.15.151:0/2869972189 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f10a4079150 0x7f10a407b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:56.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.512+0000 7f10cd25e740 1 -- start start 2024-09-16T15:44:56.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.512+0000 7f10c3400640 1 --2- 172.21.15.151:0/2869972189 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f10a4079150 0x7f10a407b610 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f10b4005fd0 tx=0x7f10b4009500 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:56.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.512+0000 7f10cd25e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55613857b8b0 0x55613857bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:56.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.512+0000 7f10cd25e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556138536a20 0x55613857a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:56.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.512+0000 7f10cd25e740 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556138536fa0 0x55613857a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:56.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.512+0000 7f10cd25e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5561384b9430 con 0x55613857b8b0 2024-09-16T15:44:56.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.513+0000 7f10cd25e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5561384a95d0 con 0x556138536a20 2024-09-16T15:44:56.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.513+0000 7f10cd25e740 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x556138498a00 con 0x556138536fa0 2024-09-16T15:44:56.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.513+0000 7f10c3400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55613857b8b0 0x55613857bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:56.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.513+0000 7f10c3400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55613857b8b0 0x55613857bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:39528/0 (socket says 172.21.15.151:39528) 2024-09-16T15:44:56.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.513+0000 7f10c3400640 1 -- 172.21.15.151:0/2546590956 learned_addr learned my addr 172.21.15.151:0/2546590956 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:56.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.513+0000 7f10c2a00640 1 --2- 172.21.15.151:0/2546590956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556138536a20 0x55613857a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:56.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.513+0000 7f10c3e00640 1 --2- 172.21.15.151:0/2546590956 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556138536fa0 0x55613857a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:56.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.513+0000 7f10c3400640 1 -- 172.21.15.151:0/2546590956 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556138536fa0 msgr2=0x55613857a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:44:56.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.513+0000 7f10c3400640 1 --2- 172.21.15.151:0/2546590956 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556138536fa0 0x55613857a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:56.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.513+0000 7f10c3400640 1 -- 172.21.15.151:0/2546590956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556138536a20 msgr2=0x55613857a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:44:56.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.513+0000 7f10c3400640 1 --2- 172.21.15.151:0/2546590956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556138536a20 0x55613857a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:56.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.513+0000 7f10c3400640 1 -- 172.21.15.151:0/2546590956 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556138461520 con 0x55613857b8b0 2024-09-16T15:44:56.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.513+0000 7f10c3400640 1 --2- 172.21.15.151:0/2546590956 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55613857b8b0 0x55613857bc90 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f10b40097f0 tx=0x7f10b4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef8f65a9207ca5b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:56.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.514+0000 7f10af400640 1 -- 172.21.15.151:0/2546590956 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b40119c0 con 0x55613857b8b0 2024-09-16T15:44:56.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.514+0000 7f10af400640 1 -- 172.21.15.151:0/2546590956 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f10b4024030 con 0x55613857b8b0 2024-09-16T15:44:56.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.514+0000 7f10af400640 1 -- 172.21.15.151:0/2546590956 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b4011d80 con 0x55613857b8b0 2024-09-16T15:44:56.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.514+0000 7f10cd25e740 1 -- 172.21.15.151:0/2546590956 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55613857b8b0 msgr2=0x55613857bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:56.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.514+0000 7f10cd25e740 1 --2- 172.21.15.151:0/2546590956 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55613857b8b0 0x55613857bc90 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f10b40097f0 tx=0x7f10b4016000 comp rx=0 tx=0).stop 2024-09-16T15:44:56.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.514+0000 7f10cd25e740 1 -- 172.21.15.151:0/2546590956 shutdown_connections 2024-09-16T15:44:56.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.515+0000 7f10cd25e740 1 --2- 172.21.15.151:0/2546590956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556138536a20 0x55613857a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:56.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.515+0000 7f10cd25e740 1 --2- 172.21.15.151:0/2546590956 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556138536fa0 0x55613857a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:56.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.515+0000 7f10cd25e740 1 --2- 172.21.15.151:0/2546590956 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55613857b8b0 0x55613857bc90 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:56.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.515+0000 7f10cd25e740 1 -- 172.21.15.151:0/2546590956 >> 172.21.15.151:0/2546590956 conn(0x556138531ff0 msgr2=0x556138535a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:56.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.515+0000 7f10cd25e740 1 -- 172.21.15.151:0/2546590956 shutdown_connections 2024-09-16T15:44:56.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.515+0000 7f10cd25e740 1 -- 172.21.15.151:0/2546590956 wait complete. 2024-09-16T15:44:56.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.515+0000 7f10cd25e740 1 -- start start 2024-09-16T15:44:56.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.515+0000 7f10cd25e740 1 Processor -- start 2024-09-16T15:44:56.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.516+0000 7f10cd25e740 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556138536fa0 0x556138559ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:56.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.516+0000 7f10cd25e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55613857b8b0 0x55613857ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:56.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.516+0000 7f10cd25e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55613855a600 0x55613855f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:56.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.516+0000 7f10cd25e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55613843f330 con 0x55613857b8b0 2024-09-16T15:44:56.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.516+0000 7f10cd25e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55613849c810 con 0x55613855a600 2024-09-16T15:44:56.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.516+0000 7f10cd25e740 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x556138209620 con 0x556138536fa0 2024-09-16T15:44:56.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.517+0000 7f10cd25e740 10 client.0 _ll_register_callbacks cb 0x556138444420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T15:44:56.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.517+0000 7f10c2a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55613857b8b0 0x55613857ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:56.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.517+0000 7f10c2a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55613857b8b0 0x55613857ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:39538/0 (socket says 172.21.15.151:39538) 2024-09-16T15:44:56.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.517+0000 7f10c2a00640 1 -- 172.21.15.151:0/1064462855 learned_addr learned my addr 172.21.15.151:0/1064462855 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:56.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.517+0000 7f10c3e00640 1 --2- 172.21.15.151:0/1064462855 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55613855a600 0x55613855f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:56.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.517+0000 7f10c3400640 1 --2- 172.21.15.151:0/1064462855 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556138536fa0 0x556138559ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:56.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.517+0000 7f10c2a00640 1 -- 172.21.15.151:0/1064462855 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556138536fa0 msgr2=0x556138559ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:44:56.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.517+0000 7f10c2a00640 1 --2- 172.21.15.151:0/1064462855 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556138536fa0 0x556138559ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:56.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.517+0000 7f10c2a00640 1 -- 172.21.15.151:0/1064462855 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55613855a600 msgr2=0x55613855f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:44:56.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.517+0000 7f10c2a00640 1 --2- 172.21.15.151:0/1064462855 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55613855a600 0x55613855f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:56.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.517+0000 7f10c2a00640 1 -- 172.21.15.151:0/1064462855 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561381fa850 con 0x55613857b8b0 2024-09-16T15:44:56.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.517+0000 7f10c2a00640 1 --2- 172.21.15.151:0/1064462855 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55613857b8b0 0x55613857ac60 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f10b007fbe0 tx=0x7f10b008c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30eb3d9584840331 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:56.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.517+0000 7f10a3e00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-16T15:44:56.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.518+0000 7f10a3e00640 1 -- 172.21.15.151:0/1064462855 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b0082740 con 0x55613857b8b0 2024-09-16T15:44:56.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.518+0000 7f10a3e00640 1 -- 172.21.15.151:0/1064462855 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7f10b007e4c0 con 0x55613857b8b0 2024-09-16T15:44:56.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.518+0000 7f10cd25e740 1 -- 172.21.15.151:0/1064462855 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10b4024030 con 0x55613857b8b0 2024-09-16T15:44:56.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.518+0000 7f10a3e00640 1 -- 172.21.15.151:0/1064462855 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f10b00820e0 con 0x55613857b8b0 2024-09-16T15:44:56.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.518+0000 7f10cd25e740 10 client.6114 fetch_fsmap learned FSMap version 36 2024-09-16T15:44:56.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.518+0000 7f10cd25e740 1 -- 172.21.15.151:0/1064462855 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x5561382716b0 con 0x55613857b8b0 2024-09-16T15:44:56.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.518+0000 7f10a3e00640 1 -- 172.21.15.151:0/1064462855 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f10b4024030 con 0x55613857b8b0 2024-09-16T15:44:56.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.518+0000 7f10a3e00640 1 -- 172.21.15.151:0/1064462855 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 36) ==== 3502+0+0 (secure 0 0 0) 0x7f10b009cbe0 con 0x55613857b8b0 2024-09-16T15:44:56.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.518+0000 7f10cd25e740 10 client.6114 fetch_fsmap finished waiting for FSMap version 36 2024-09-16T15:44:56.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.518+0000 7f10cd25e740 10 client.6114 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-16T15:44:56.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.518+0000 7f10cd25e740 20 client.6114 populate_metadata read hostname 'smithi151' 2024-09-16T15:44:56.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.518+0000 7f10cd25e740 1 --2- 172.21.15.151:0/1064462855 >> [v2:172.21.15.151:6833/1724192711,v1:172.21.15.151:6835/1724192711] conn(0x55613858a690 0x55613858aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:56.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.518+0000 7f10cd25e740 4 client.6114 mds_command: new command op to 5961 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:44:56.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.518+0000 7f10cd25e740 1 -- 172.21.15.151:0/1064462855 --> [v2:172.21.15.151:6833/1724192711,v1:172.21.15.151:6835/1724192711] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561384ab520 con 0x55613858a690 2024-09-16T15:44:56.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.519+0000 7f10c3400640 1 --2- 172.21.15.151:0/1064462855 >> [v2:172.21.15.151:6833/1724192711,v1:172.21.15.151:6835/1724192711] conn(0x55613858a690 0x55613858aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:56.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.519+0000 7f10c3400640 1 --2- 172.21.15.151:0/1064462855 >> [v2:172.21.15.151:6833/1724192711,v1:172.21.15.151:6835/1724192711] conn(0x55613858a690 0x55613858aa70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3193873352d5258a server_cookie=72dcae36a0f1fd51 in_seq=0 out_seq=0 2024-09-16T15:44:56.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.519+0000 7f10a3e00640 10 client.6114 ms_handle_connect on v2:172.21.15.151:6833/1724192711 2024-09-16T15:44:56.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.520+0000 7f10a3e00640 1 -- 172.21.15.151:0/1064462855 <== mds.0 v2:172.21.15.151:6833/1724192711 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5561384ab520 con 0x55613858a690 2024-09-16T15:44:56.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.520+0000 7f10a3e00640 10 client.6114 handle_command_reply: tid=0 2024-09-16T15:44:56.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.520+0000 7f10cd25e740 1 client.6114 shutdown 2024-09-16T15:44:56.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.520+0000 7f10cd25e740 1 -- 172.21.15.151:0/1064462855 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55613857b8b0 msgr2=0x55613857ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:56.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.520+0000 7f10cd25e740 1 --2- 172.21.15.151:0/1064462855 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55613857b8b0 0x55613857ac60 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f10b007fbe0 tx=0x7f10b008c040 comp rx=0 tx=0).stop 2024-09-16T15:44:56.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.520+0000 7f10cd25e740 1 -- 172.21.15.151:0/1064462855 shutdown_connections 2024-09-16T15:44:56.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.520+0000 7f10cd25e740 1 --2- 172.21.15.151:0/1064462855 >> [v2:172.21.15.151:6833/1724192711,v1:172.21.15.151:6835/1724192711] conn(0x55613858a690 0x55613858aa70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:56.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.520+0000 7f10cd25e740 1 --2- 172.21.15.151:0/1064462855 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55613855a600 0x55613855f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:56.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.520+0000 7f10a3e00640 0 client.6114 ms_handle_reset on v2:172.21.15.151:6833/1724192711 2024-09-16T15:44:56.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.520+0000 7f10cd25e740 1 --2- 172.21.15.151:0/1064462855 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55613857b8b0 0x55613857ac60 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:56.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.520+0000 7f10cd25e740 1 --2- 172.21.15.151:0/1064462855 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556138536fa0 0x556138559ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:56.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.520+0000 7f10cd25e740 1 -- 172.21.15.151:0/1064462855 >> 172.21.15.151:0/1064462855 conn(0x556138536a20 msgr2=0x55613855c180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:56.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.520+0000 7f10cd25e740 1 -- 172.21.15.151:0/1064462855 shutdown_connections 2024-09-16T15:44:56.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.521+0000 7f10cd25e740 1 -- 172.21.15.151:0/1064462855 wait complete. 2024-09-16T15:44:56.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.521+0000 7f10cd25e740 20 client.6114 trim_cache size 0 max 16384 2024-09-16T15:44:56.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.539+0000 7f10cd25e740 1 Processor -- start 2024-09-16T15:44:56.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.539+0000 7f10cd25e740 1 -- start start 2024-09-16T15:44:56.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.540+0000 7f10cd25e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556138536fa0 0x556138574670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:56.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.540+0000 7f10cd25e740 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55613857b8b0 0x556138535270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:56.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.540+0000 7f10cd25e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55613858e600 0x5561385357b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:56.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.540+0000 7f10cd25e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f10c4200c20 con 0x55613858e600 2024-09-16T15:44:56.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.540+0000 7f10cd25e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f10c40b4590 con 0x556138536fa0 2024-09-16T15:44:56.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.540+0000 7f10cd25e740 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f10c42005d0 con 0x55613857b8b0 2024-09-16T15:44:56.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.541+0000 7f10c3e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55613858e600 0x5561385357b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:56.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.541+0000 7f10c3e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55613858e600 0x5561385357b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:39548/0 (socket says 172.21.15.151:39548) 2024-09-16T15:44:56.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.541+0000 7f10c3e00640 1 -- 172.21.15.151:0/488944835 learned_addr learned my addr 172.21.15.151:0/488944835 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:56.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.541+0000 7f10c3400640 1 --2- 172.21.15.151:0/488944835 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556138536fa0 0x556138574670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:56.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.541+0000 7f10c2a00640 1 --2- 172.21.15.151:0/488944835 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55613857b8b0 0x556138535270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:56.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.541+0000 7f10c3e00640 1 -- 172.21.15.151:0/488944835 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55613857b8b0 msgr2=0x556138535270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:44:56.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.541+0000 7f10c3e00640 1 --2- 172.21.15.151:0/488944835 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55613857b8b0 0x556138535270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:56.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.541+0000 7f10c3e00640 1 -- 172.21.15.151:0/488944835 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556138536fa0 msgr2=0x556138574670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:44:56.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.541+0000 7f10c3e00640 1 --2- 172.21.15.151:0/488944835 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556138536fa0 0x556138574670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:56.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.541+0000 7f10c3e00640 1 -- 172.21.15.151:0/488944835 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55613857d320 con 0x55613858e600 2024-09-16T15:44:56.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.541+0000 7f10c3e00640 1 --2- 172.21.15.151:0/488944835 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55613858e600 0x5561385357b0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f10bc001860 tx=0x7f10bc003cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a03ea349dfac814e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:56.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.541+0000 7f10aea00640 1 -- 172.21.15.151:0/488944835 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10bc004110 con 0x55613858e600 2024-09-16T15:44:56.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.542+0000 7f10aea00640 1 -- 172.21.15.151:0/488944835 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f10bc0042b0 con 0x55613858e600 2024-09-16T15:44:56.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.542+0000 7f10aea00640 1 -- 172.21.15.151:0/488944835 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10bc0046c0 con 0x55613858e600 2024-09-16T15:44:56.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.542+0000 7f10cd25e740 1 -- 172.21.15.151:0/488944835 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55613858e600 msgr2=0x5561385357b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:56.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.542+0000 7f10cd25e740 1 --2- 172.21.15.151:0/488944835 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55613858e600 0x5561385357b0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f10bc001860 tx=0x7f10bc003cc0 comp rx=0 tx=0).stop 2024-09-16T15:44:56.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.542+0000 7f10cd25e740 1 -- 172.21.15.151:0/488944835 shutdown_connections 2024-09-16T15:44:56.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.542+0000 7f10cd25e740 1 --2- 172.21.15.151:0/488944835 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55613858e600 0x5561385357b0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:56.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.542+0000 7f10cd25e740 1 --2- 172.21.15.151:0/488944835 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55613857b8b0 0x556138535270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:56.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.542+0000 7f10cd25e740 1 --2- 172.21.15.151:0/488944835 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556138536fa0 0x556138574670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:56.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.542+0000 7f10cd25e740 1 -- 172.21.15.151:0/488944835 >> 172.21.15.151:0/488944835 conn(0x556138536a20 msgr2=0x55613855b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:56.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.542+0000 7f10cd25e740 1 -- 172.21.15.151:0/488944835 shutdown_connections 2024-09-16T15:44:56.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.542+0000 7f10cd25e740 1 -- 172.21.15.151:0/488944835 wait complete. 2024-09-16T15:44:56.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.543+0000 7f10cd25e740 1 -- start start 2024-09-16T15:44:56.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.543+0000 7f10cd25e740 1 Processor -- start 2024-09-16T15:44:56.549 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.544+0000 7f10cd25e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556138536fa0 0x556138577c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:56.549 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.544+0000 7f10cd25e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55613857b8b0 0x5561385faed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:56.549 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.544+0000 7f10cd25e740 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55613858e600 0x55613859e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:56.549 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.544+0000 7f10cd25e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x556138575d40 con 0x55613857b8b0 2024-09-16T15:44:56.549 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.544+0000 7f10cd25e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55613843f330 con 0x556138536fa0 2024-09-16T15:44:56.549 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.544+0000 7f10cd25e740 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x556138575190 con 0x55613858e600 2024-09-16T15:44:56.549 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.544+0000 7f10cd25e740 10 client.0 _ll_register_callbacks cb 0x7f10b0082e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T15:44:56.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.544+0000 7f10c2a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55613857b8b0 0x5561385faed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:56.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.544+0000 7f10c2a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55613857b8b0 0x5561385faed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:39556/0 (socket says 172.21.15.151:39556) 2024-09-16T15:44:56.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.544+0000 7f10c2a00640 1 -- 172.21.15.151:0/1985057516 learned_addr learned my addr 172.21.15.151:0/1985057516 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:56.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.545+0000 7f10c2a00640 1 -- 172.21.15.151:0/1985057516 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55613858e600 msgr2=0x55613859e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:44:56.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.545+0000 7f10c3400640 1 --2- 172.21.15.151:0/1985057516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556138536fa0 0x556138577c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:56.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.545+0000 7f10c2a00640 1 --2- 172.21.15.151:0/1985057516 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55613858e600 0x55613859e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:56.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.545+0000 7f10c2a00640 1 -- 172.21.15.151:0/1985057516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556138536fa0 msgr2=0x556138577c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:44:56.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.545+0000 7f10c2a00640 1 --2- 172.21.15.151:0/1985057516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556138536fa0 0x556138577c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:56.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.545+0000 7f10c2a00640 1 -- 172.21.15.151:0/1985057516 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55613852fc00 con 0x55613857b8b0 2024-09-16T15:44:56.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.545+0000 7f10c3400640 1 --2- 172.21.15.151:0/1985057516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556138536fa0 0x556138577c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:44:56.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.545+0000 7f10c2a00640 1 --2- 172.21.15.151:0/1985057516 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55613857b8b0 0x5561385faed0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f10b0082670 tx=0x7f10b007e4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63f63de8e6dd8d7c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:56.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.545+0000 7f10a3e00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-16T15:44:56.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.545+0000 7f10a3e00640 1 -- 172.21.15.151:0/1985057516 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b00822c0 con 0x55613857b8b0 2024-09-16T15:44:56.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.545+0000 7f10a3e00640 1 -- 172.21.15.151:0/1985057516 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7f10b0081640 con 0x55613857b8b0 2024-09-16T15:44:56.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.545+0000 7f10cd25e740 1 -- 172.21.15.151:0/1985057516 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10bc0042b0 con 0x55613857b8b0 2024-09-16T15:44:56.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.545+0000 7f10a3e00640 1 -- 172.21.15.151:0/1985057516 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f10b0082740 con 0x55613857b8b0 2024-09-16T15:44:56.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.545+0000 7f10cd25e740 10 client.6120 fetch_fsmap learned FSMap version 36 2024-09-16T15:44:56.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.545+0000 7f10cd25e740 1 -- 172.21.15.151:0/1985057516 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x55613852fc00 con 0x55613857b8b0 2024-09-16T15:44:56.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.545+0000 7f10a3e00640 1 -- 172.21.15.151:0/1985057516 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f10bc0042b0 con 0x55613857b8b0 2024-09-16T15:44:56.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.546+0000 7f10a3e00640 1 -- 172.21.15.151:0/1985057516 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 36) ==== 3502+0+0 (secure 0 0 0) 0x7f10b00a3070 con 0x55613857b8b0 2024-09-16T15:44:56.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.546+0000 7f10cd25e740 10 client.6120 fetch_fsmap finished waiting for FSMap version 36 2024-09-16T15:44:56.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.546+0000 7f10cd25e740 10 client.6120 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-16T15:44:56.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.546+0000 7f10cd25e740 20 client.6120 populate_metadata read hostname 'smithi151' 2024-09-16T15:44:56.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.546+0000 7f10cd25e740 1 --2- 172.21.15.151:0/1985057516 >> [v2:172.21.15.151:6833/1724192711,v1:172.21.15.151:6835/1724192711] conn(0x55613857e2f0 0x556138588c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:56.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.546+0000 7f10cd25e740 4 client.6120 mds_command: new command op to 5961 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T15:44:56.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.546+0000 7f10cd25e740 1 -- 172.21.15.151:0/1985057516 --> [v2:172.21.15.151:6833/1724192711,v1:172.21.15.151:6835/1724192711] -- command(tid 0: {"prefix": "status"}) -- 0x5561384a6fd0 con 0x55613857e2f0 2024-09-16T15:44:56.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.546+0000 7f10c3400640 1 --2- 172.21.15.151:0/1985057516 >> [v2:172.21.15.151:6833/1724192711,v1:172.21.15.151:6835/1724192711] conn(0x55613857e2f0 0x556138588c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:56.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.546+0000 7f10c3400640 1 --2- 172.21.15.151:0/1985057516 >> [v2:172.21.15.151:6833/1724192711,v1:172.21.15.151:6835/1724192711] conn(0x55613857e2f0 0x556138588c50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=507a433bd3c07c7e server_cookie=a3a74f93ec183031 in_seq=0 out_seq=0 2024-09-16T15:44:56.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.546+0000 7f10a3e00640 10 client.6120 ms_handle_connect on v2:172.21.15.151:6833/1724192711 2024-09-16T15:44:56.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.547+0000 7f10a3e00640 1 -- 172.21.15.151:0/1985057516 <== mds.0 v2:172.21.15.151:6833/1724192711 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x5561384a6fd0 con 0x55613857e2f0 2024-09-16T15:44:56.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.547+0000 7f10a3e00640 10 client.6120 handle_command_reply: tid=0 2024-09-16T15:44:56.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.547+0000 7f10cd25e740 1 client.6120 shutdown 2024-09-16T15:44:56.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.547+0000 7f10cd25e740 1 -- 172.21.15.151:0/1985057516 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55613857b8b0 msgr2=0x5561385faed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:56.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.547+0000 7f10cd25e740 1 --2- 172.21.15.151:0/1985057516 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55613857b8b0 0x5561385faed0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f10b0082670 tx=0x7f10b007e4f0 comp rx=0 tx=0).stop 2024-09-16T15:44:56.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.547+0000 7f10cd25e740 1 -- 172.21.15.151:0/1985057516 shutdown_connections 2024-09-16T15:44:56.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.547+0000 7f10cd25e740 1 --2- 172.21.15.151:0/1985057516 >> [v2:172.21.15.151:6833/1724192711,v1:172.21.15.151:6835/1724192711] conn(0x55613857e2f0 0x556138588c50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:56.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.547+0000 7f10cd25e740 1 --2- 172.21.15.151:0/1985057516 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55613858e600 0x55613859e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:56.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.547+0000 7f10cd25e740 1 --2- 172.21.15.151:0/1985057516 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55613857b8b0 0x5561385faed0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:56.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.547+0000 7f10a3e00640 0 client.6120 ms_handle_reset on v2:172.21.15.151:6833/1724192711 2024-09-16T15:44:56.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.547+0000 7f10cd25e740 1 --2- 172.21.15.151:0/1985057516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556138536fa0 0x556138577c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:56.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.547+0000 7f10cd25e740 1 -- 172.21.15.151:0/1985057516 >> 172.21.15.151:0/1985057516 conn(0x556138536a20 msgr2=0x55613855b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:56.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.547+0000 7f10cd25e740 1 -- 172.21.15.151:0/1985057516 shutdown_connections 2024-09-16T15:44:56.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.547+0000 7f10cd25e740 1 -- 172.21.15.151:0/1985057516 wait complete. 2024-09-16T15:44:56.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.547+0000 7f10cd25e740 20 client.6120 trim_cache size 0 max 16384 2024-09-16T15:44:56.553 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-16T15:44:56.553 INFO:teuthology.orchestra.run.smithi151.stdout: "cluster_fsid": "1757be18-e41a-4446-bb4f-ee4ac81c39df", 2024-09-16T15:44:56.553 INFO:teuthology.orchestra.run.smithi151.stdout: "whoami": 0, 2024-09-16T15:44:56.553 INFO:teuthology.orchestra.run.smithi151.stdout: "id": 5961, 2024-09-16T15:44:56.553 INFO:teuthology.orchestra.run.smithi151.stdout: "want_state": "up:active", 2024-09-16T15:44:56.554 INFO:teuthology.orchestra.run.smithi151.stdout: "state": "up:active", 2024-09-16T15:44:56.554 INFO:teuthology.orchestra.run.smithi151.stdout: "fs_name": "cephfs", 2024-09-16T15:44:56.554 INFO:teuthology.orchestra.run.smithi151.stdout: "rank_uptime": 10.843281749000001, 2024-09-16T15:44:56.554 INFO:teuthology.orchestra.run.smithi151.stdout: "mdsmap_epoch": 36, 2024-09-16T15:44:56.554 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_epoch": 62, 2024-09-16T15:44:56.554 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_epoch_barrier": 62, 2024-09-16T15:44:56.554 INFO:teuthology.orchestra.run.smithi151.stdout: "uptime": 15.169326673 2024-09-16T15:44:56.554 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-16T15:44:56.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.548+0000 7f10ca200640 1 -- 172.21.15.151:0/2869972189 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f10a4079150 msgr2=0x7f10a407b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:56.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.548+0000 7f10ca200640 1 --2- 172.21.15.151:0/2869972189 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f10a4079150 0x7f10a407b610 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f10b4005fd0 tx=0x7f10b4009500 comp rx=0 tx=0).stop 2024-09-16T15:44:56.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.548+0000 7f10ca200640 1 -- 172.21.15.151:0/2869972189 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10c415bca0 msgr2=0x7f10c4151680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:56.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.548+0000 7f10ca200640 1 --2- 172.21.15.151:0/2869972189 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10c415bca0 0x7f10c4151680 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f10b002aa40 tx=0x7f10b0006220 comp rx=0 tx=0).stop 2024-09-16T15:44:56.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.548+0000 7f10ca200640 1 -- 172.21.15.151:0/2869972189 shutdown_connections 2024-09-16T15:44:56.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.548+0000 7f10ca200640 1 --2- 172.21.15.151:0/2869972189 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f10a4079150 0x7f10a407b610 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:56.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.548+0000 7f10ca200640 1 --2- 172.21.15.151:0/2869972189 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f10c40ce610 0x7f10c40cea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:56.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.548+0000 7f10ca200640 1 --2- 172.21.15.151:0/2869972189 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10c415bca0 0x7f10c4151680 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:56.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.548+0000 7f10ca200640 1 --2- 172.21.15.151:0/2869972189 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10a8004800 0x7f10c40cdd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:56.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.548+0000 7f10ca200640 1 -- 172.21.15.151:0/2869972189 >> 172.21.15.151:0/2869972189 conn(0x7f10c40b1950 msgr2=0x7f10c409c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:56.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.548+0000 7f10ca200640 1 -- 172.21.15.151:0/2869972189 shutdown_connections 2024-09-16T15:44:56.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.548+0000 7f10ca200640 1 -- 172.21.15.151:0/2869972189 wait complete. 2024-09-16T15:44:56.564 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-16T15:44:56.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.722+0000 7fa2c6c00640 1 -- 172.21.15.151:0/4080691791 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa2c00af3d0 msgr2=0x7fa2c006f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:56.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.722+0000 7fa2c6c00640 1 --2- 172.21.15.151:0/4080691791 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa2c00af3d0 0x7fa2c006f490 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac005fd0 tx=0x7fa2ac02a540 comp rx=0 tx=0).stop 2024-09-16T15:44:56.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.722+0000 7fa2c6c00640 1 -- 172.21.15.151:0/4080691791 shutdown_connections 2024-09-16T15:44:56.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.722+0000 7fa2c6c00640 1 --2- 172.21.15.151:0/4080691791 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa2c00af3d0 0x7fa2c006f490 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:56.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.722+0000 7fa2c6c00640 1 --2- 172.21.15.151:0/4080691791 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa2c0161550 0x7fa2c006ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:56.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.722+0000 7fa2c6c00640 1 -- 172.21.15.151:0/4080691791 >> 172.21.15.151:0/4080691791 conn(0x7fa2c00b1950 msgr2=0x7fa2c00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:56.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.722+0000 7fa2c6c00640 1 -- 172.21.15.151:0/4080691791 shutdown_connections 2024-09-16T15:44:56.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.722+0000 7fa2c6c00640 1 -- 172.21.15.151:0/4080691791 wait complete. 2024-09-16T15:44:56.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.724+0000 7fa2c6c00640 1 Processor -- start 2024-09-16T15:44:56.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.724+0000 7fa2c6c00640 1 -- start start 2024-09-16T15:44:56.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.725+0000 7fa2c6c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa2c006f9d0 0x7fa2c0147d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:56.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.725+0000 7fa2c6c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2c00af3d0 0x7fa2c0157fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:56.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.725+0000 7fa2c6c00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa2c0161550 0x7fa2c00d0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:56.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.725+0000 7fa2c6c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c01623a0 con 0x7fa2c006f9d0 2024-09-16T15:44:56.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.725+0000 7fa2c6c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c0074010 con 0x7fa2c00af3d0 2024-09-16T15:44:56.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.725+0000 7fa2c6c00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fa2c0162520 con 0x7fa2c0161550 2024-09-16T15:44:56.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.725+0000 7fa2bfe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa2c006f9d0 0x7fa2c0147d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:56.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.725+0000 7fa2bfe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa2c006f9d0 0x7fa2c0147d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:39566/0 (socket says 172.21.15.151:39566) 2024-09-16T15:44:56.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.725+0000 7fa2bfe00640 1 -- 172.21.15.151:0/2636991672 learned_addr learned my addr 172.21.15.151:0/2636991672 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:56.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.725+0000 7fa2c4c00640 1 --2- 172.21.15.151:0/2636991672 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa2c0161550 0x7fa2c00d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:56.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.725+0000 7fa2bf400640 1 --2- 172.21.15.151:0/2636991672 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2c00af3d0 0x7fa2c0157fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:56.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.726+0000 7fa2bfe00640 1 -- 172.21.15.151:0/2636991672 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa2c0161550 msgr2=0x7fa2c00d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:56.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.726+0000 7fa2bfe00640 1 --2- 172.21.15.151:0/2636991672 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa2c0161550 0x7fa2c00d0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:56.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.726+0000 7fa2bfe00640 1 -- 172.21.15.151:0/2636991672 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2c00af3d0 msgr2=0x7fa2c0157fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:56.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.726+0000 7fa2bfe00640 1 --2- 172.21.15.151:0/2636991672 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2c00af3d0 0x7fa2c0157fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:56.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.726+0000 7fa2bfe00640 1 -- 172.21.15.151:0/2636991672 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2c00b0310 con 0x7fa2c006f9d0 2024-09-16T15:44:56.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.726+0000 7fa2bfe00640 1 --2- 172.21.15.151:0/2636991672 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa2c006f9d0 0x7fa2c0147d50 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fa2b000eb30 tx=0x7fa2b000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:56.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.726+0000 7fa2b7e00640 1 -- 172.21.15.151:0/2636991672 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b0010430 con 0x7fa2c006f9d0 2024-09-16T15:44:56.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.726+0000 7fa2b7e00640 1 -- 172.21.15.151:0/2636991672 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa2b00105d0 con 0x7fa2c006f9d0 2024-09-16T15:44:56.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.726+0000 7fa2c6c00640 1 -- 172.21.15.151:0/2636991672 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2c00b05f0 con 0x7fa2c006f9d0 2024-09-16T15:44:56.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.726+0000 7fa2b7e00640 1 -- 172.21.15.151:0/2636991672 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b0010a20 con 0x7fa2c006f9d0 2024-09-16T15:44:56.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.727+0000 7fa2c6c00640 1 -- 172.21.15.151:0/2636991672 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2c00ad140 con 0x7fa2c006f9d0 2024-09-16T15:44:56.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.728+0000 7fa2c6c00640 1 -- 172.21.15.151:0/2636991672 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2c00760b0 con 0x7fa2c006f9d0 2024-09-16T15:44:56.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.729+0000 7fa2b7e00640 1 -- 172.21.15.151:0/2636991672 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 102103+0+0 (secure 0 0 0) 0x7fa2b0010bc0 con 0x7fa2c006f9d0 2024-09-16T15:44:56.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.730+0000 7fa2b7e00640 1 --2- 172.21.15.151:0/2636991672 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa2a0079150 0x7fa2a007b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:56.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.730+0000 7fa2b7e00640 1 -- 172.21.15.151:0/2636991672 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7fa2b0099f40 con 0x7fa2c006f9d0 2024-09-16T15:44:56.732 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.731+0000 7fa2bf400640 1 --2- 172.21.15.151:0/2636991672 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa2a0079150 0x7fa2a007b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:56.732 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.731+0000 7fa2bf400640 1 --2- 172.21.15.151:0/2636991672 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa2a0079150 0x7fa2a007b610 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac033000 tx=0x7fa2ac004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:56.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.735+0000 7fa2b7e00640 1 -- 172.21.15.151:0/2636991672 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa2b00639b0 con 0x7fa2c006f9d0 2024-09-16T15:44:56.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.959+0000 7fa2c6c00640 1 -- 172.21.15.151:0/2636991672 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa2c0162ab0 con 0x7fa2c006f9d0 2024-09-16T15:44:56.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.960+0000 7fa2b7e00640 1 -- 172.21.15.151:0/2636991672 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v36) ==== 53+0+73 (secure 0 0 0) 0x7fa2b00683e0 con 0x7fa2c006f9d0 2024-09-16T15:44:56.961 INFO:teuthology.orchestra.run.smithi151.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-16T15:44:56.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.962+0000 7fa2c6c00640 1 -- 172.21.15.151:0/2636991672 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa2a0079150 msgr2=0x7fa2a007b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:56.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.962+0000 7fa2c6c00640 1 --2- 172.21.15.151:0/2636991672 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa2a0079150 0x7fa2a007b610 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac033000 tx=0x7fa2ac004730 comp rx=0 tx=0).stop 2024-09-16T15:44:56.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.962+0000 7fa2c6c00640 1 -- 172.21.15.151:0/2636991672 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa2c006f9d0 msgr2=0x7fa2c0147d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:56.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.962+0000 7fa2c6c00640 1 --2- 172.21.15.151:0/2636991672 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa2c006f9d0 0x7fa2c0147d50 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fa2b000eb30 tx=0x7fa2b000c2d0 comp rx=0 tx=0).stop 2024-09-16T15:44:56.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.962+0000 7fa2c6c00640 1 -- 172.21.15.151:0/2636991672 shutdown_connections 2024-09-16T15:44:56.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.962+0000 7fa2c6c00640 1 --2- 172.21.15.151:0/2636991672 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa2a0079150 0x7fa2a007b610 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:56.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.962+0000 7fa2c6c00640 1 --2- 172.21.15.151:0/2636991672 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2c00af3d0 0x7fa2c0157fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:56.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.962+0000 7fa2c6c00640 1 --2- 172.21.15.151:0/2636991672 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa2c0161550 0x7fa2c00d0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:56.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.962+0000 7fa2c6c00640 1 --2- 172.21.15.151:0/2636991672 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa2c006f9d0 0x7fa2c0147d50 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:56.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.962+0000 7fa2c6c00640 1 -- 172.21.15.151:0/2636991672 >> 172.21.15.151:0/2636991672 conn(0x7fa2c00b1950 msgr2=0x7fa2c006ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:56.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.962+0000 7fa2c6c00640 1 -- 172.21.15.151:0/2636991672 shutdown_connections 2024-09-16T15:44:56.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:56.962+0000 7fa2c6c00640 1 -- 172.21.15.151:0/2636991672 wait complete. 2024-09-16T15:44:56.979 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-16T15:44:56.979 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T15:44:56.979 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi151.front.sepia.ceph.com 2024-09-16T15:44:56.979 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-16T15:44:56.979 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-16T15:44:56.979 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-16T15:44:56.979 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-16T15:44:56.979 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T15:44:56.979 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T15:44:56.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:56.980 DEBUG:teuthology.orchestra.run.smithi151:> ip addr 2024-09-16T15:44:57.015 INFO:teuthology.orchestra.run.smithi151.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T15:44:57.015 INFO:teuthology.orchestra.run.smithi151.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T15:44:57.015 INFO:teuthology.orchestra.run.smithi151.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T15:44:57.015 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:44:57.016 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 ::1/128 scope host 2024-09-16T15:44:57.016 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:44:57.016 INFO:teuthology.orchestra.run.smithi151.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:44:57.016 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:da:6e:0c brd ff:ff:ff:ff:ff:ff 2024-09-16T15:44:57.016 INFO:teuthology.orchestra.run.smithi151.stdout: altname enp7s0f0 2024-09-16T15:44:57.016 INFO:teuthology.orchestra.run.smithi151.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:44:57.016 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:da:6e:0d brd ff:ff:ff:ff:ff:ff 2024-09-16T15:44:57.016 INFO:teuthology.orchestra.run.smithi151.stdout: altname enp7s0f1 2024-09-16T15:44:57.016 INFO:teuthology.orchestra.run.smithi151.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:44:57.016 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:88:6f:62 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:44:57.016 INFO:teuthology.orchestra.run.smithi151.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T15:44:57.016 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:88:6f:63 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:44:57.016 INFO:teuthology.orchestra.run.smithi151.stdout: inet 172.21.15.151/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-16T15:44:57.016 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft 42392sec preferred_lft 42392sec 2024-09-16T15:44:57.017 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 fe80::ec4:7aff:fe88:6f63/64 scope link noprefixroute 2024-09-16T15:44:57.017 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:44:57.017 INFO:teuthology.orchestra.run.smithi151.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-16T15:44:57.017 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 62:3f:3b:48:ad:f8 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:44:57.017 INFO:teuthology.orchestra.run.smithi151.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-16T15:44:57.017 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:44:57.017 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 fe80::40fa:7fff:fedf:2a00/64 scope link 2024-09-16T15:44:57.017 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:44:57.017 INFO:teuthology.orchestra.run.smithi151.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T15:44:57.017 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 62:3f:3b:48:ad:f8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T15:44:57.017 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 fe80::603f:3bff:fe48:adf8/64 scope link 2024-09-16T15:44:57.017 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:44:57.017 INFO:teuthology.orchestra.run.smithi151.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T15:44:57.017 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether b6:ac:fe:34:a7:66 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-16T15:44:57.018 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 fe80::b4ac:feff:fe34:a766/64 scope link 2024-09-16T15:44:57.018 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:44:57.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:57.018 DEBUG:teuthology.orchestra.run.smithi151:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T15:44:57.110 INFO:teuthology.orchestra.run.smithi151.stdout:1 2024-09-16T15:44:57.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:57.113 DEBUG:teuthology.orchestra.run.smithi151:> route 2024-09-16T15:44:57.149 INFO:teuthology.orchestra.run.smithi151.stdout:Kernel IP routing table 2024-09-16T15:44:57.149 INFO:teuthology.orchestra.run.smithi151.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T15:44:57.149 INFO:teuthology.orchestra.run.smithi151.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-16T15:44:57.149 INFO:teuthology.orchestra.run.smithi151.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-16T15:44:57.149 INFO:teuthology.orchestra.run.smithi151.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T15:44:57.149 DEBUG:teuthology.orchestra.run.smithi151:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T15:44:57.149 DEBUG:teuthology.orchestra.run.smithi151:> set -e 2024-09-16T15:44:57.149 DEBUG:teuthology.orchestra.run.smithi151:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-16T15:44:57.149 DEBUG:teuthology.orchestra.run.smithi151:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-16T15:44:57.149 DEBUG:teuthology.orchestra.run.smithi151:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-16T15:44:57.150 DEBUG:teuthology.orchestra.run.smithi151:> ') 2024-09-16T15:44:57.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:44:57.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:44:57.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:57.346 DEBUG:teuthology.orchestra.run.smithi151:> ip netns list 2024-09-16T15:44:57.381 INFO:teuthology.orchestra.run.smithi151.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-16T15:44:57.381 INFO:teuthology.orchestra.run.smithi151.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-16T15:44:57.382 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T15:44:57.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:57.382 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip link set brx.0 up 2024-09-16T15:44:57.478 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:44:57.478 DEBUG:teuthology.orchestra.run.smithi151:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-16T15:44:57.513 INFO:teuthology.orchestra.run.smithi151.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-16T15:44:57.513 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:44:57.513 DEBUG:teuthology.orchestra.run.smithi151:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-16T15:44:57.588 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-16T15:44:57.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:57.589 DEBUG:teuthology.orchestra.run.smithi151:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-16T15:44:57.717 INFO:teuthology.orchestra.run.smithi151.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-16T15:44:57.717 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-16T15:44:57.717 INFO:teuthology.orchestra.run.smithi151.stdout:invalid new device string format 2024-09-16T15:44:57.733 INFO:teuthology.orchestra.run.smithi151.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-16T15:44:57.733 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-16T15:44:57.733 INFO:teuthology.orchestra.run.smithi151.stdout:invalid new device string format 2024-09-16T15:44:57.733 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: resolved to: "172.21.15.151:3300,172.21.15.167:3300,172.21.15.167:3301" 2024-09-16T15:44:57.733 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-16T15:44:57.755 INFO:teuthology.orchestra.run.smithi151.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-16T15:44:57.755 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-16T15:44:57.755 INFO:teuthology.orchestra.run.smithi151.stdout:invalid new device string format 2024-09-16T15:44:57.755 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: resolved to: "172.21.15.151:3300,172.21.15.167:3300,172.21.15.167:3301" 2024-09-16T15:44:57.755 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-16T15:44:57.755 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: resolved to: "172.21.15.151:6789,172.21.15.167:6789,172.21.15.167:6790" 2024-09-16T15:44:57.756 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.151:6789,172.21.15.167:6789,172.21.15.167:6790:/ 2024-09-16T15:44:57.756 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=1757be18-e41a-4446-bb4f-ee4ac81c39df" will pass to kernel 2024-09-16T15:44:57.758 INFO:teuthology.orchestra.run.smithi151.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-16T15:44:57.758 INFO:teuthology.orchestra.run.smithi151.stdout: You just mounted a file system that supports labels which does not 2024-09-16T15:44:57.758 INFO:teuthology.orchestra.run.smithi151.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-16T15:44:57.758 INFO:teuthology.orchestra.run.smithi151.stdout: applications will generate AVC messages and not be allowed access to 2024-09-16T15:44:57.758 INFO:teuthology.orchestra.run.smithi151.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-16T15:44:57.761 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-16T15:44:57.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:44:57.761 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-16T15:44:57.817 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T15:44:57.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:57 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:44:57.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:44:57.913 INFO:teuthology.orchestra.run.smithi151.stdout:client5453 2024-09-16T15:44:57.913 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:44:57.913 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:44:57.948 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-16T15:44:58.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.152+0000 7fa113e00640 1 -- 172.21.15.151:0/3955990192 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa1140d48a0 msgr2=0x7fa1140d2920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:58.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.152+0000 7fa113e00640 1 --2- 172.21.15.151:0/3955990192 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa1140d48a0 0x7fa1140d2920 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fa1140b31d0 tx=0x7fa0fc02a5f0 comp rx=0 tx=0).stop 2024-09-16T15:44:58.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.152+0000 7fa113e00640 1 -- 172.21.15.151:0/3955990192 shutdown_connections 2024-09-16T15:44:58.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.152+0000 7fa113e00640 1 --2- 172.21.15.151:0/3955990192 >> v2:172.21.15.167:3300/0 conn(0x7fa1140b4820 0x7fa1140d0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:58.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.152+0000 7fa113e00640 1 --2- 172.21.15.151:0/3955990192 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa1140d48a0 0x7fa1140d2920 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:58.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.152+0000 7fa113e00640 1 -- 172.21.15.151:0/3955990192 >> 172.21.15.151:0/3955990192 conn(0x7fa1140d3d80 msgr2=0x7fa1140d4190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:58.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.152+0000 7fa113e00640 1 -- 172.21.15.151:0/3955990192 shutdown_connections 2024-09-16T15:44:58.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.152+0000 7fa113e00640 1 -- 172.21.15.151:0/3955990192 wait complete. 2024-09-16T15:44:58.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.153+0000 7fa113e00640 1 Processor -- start 2024-09-16T15:44:58.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.154+0000 7fa113e00640 1 -- start start 2024-09-16T15:44:58.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.154+0000 7fa113e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa1140b4820 0x7fa1140cd310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:58.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.155+0000 7fa113e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa1140d0660 0x7fa1141f5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:58.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.155+0000 7fa113e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1140d48a0 0x7fa1140b83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:58.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.155+0000 7fa113e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa1140b3f30 con 0x7fa1140d0660 2024-09-16T15:44:58.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.155+0000 7fa113e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa1140b3db0 con 0x7fa1140d48a0 2024-09-16T15:44:58.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.155+0000 7fa113e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fa1140ce850 con 0x7fa1140b4820 2024-09-16T15:44:58.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.155+0000 7fa10be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa1140d0660 0x7fa1141f5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:58.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.155+0000 7fa10be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa1140d0660 0x7fa1141f5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:39580/0 (socket says 172.21.15.151:39580) 2024-09-16T15:44:58.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.155+0000 7fa10be00640 1 -- 172.21.15.151:0/1249210098 learned_addr learned my addr 172.21.15.151:0/1249210098 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:58.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.155+0000 7fa113400640 1 --2- 172.21.15.151:0/1249210098 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1140d48a0 0x7fa1140b83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:58.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.155+0000 7fa112a00640 1 --2- 172.21.15.151:0/1249210098 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa1140b4820 0x7fa1140cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:58.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.155+0000 7fa10be00640 1 -- 172.21.15.151:0/1249210098 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa1140b4820 msgr2=0x7fa1140cd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:58.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.155+0000 7fa10be00640 1 --2- 172.21.15.151:0/1249210098 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa1140b4820 0x7fa1140cd310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:58.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.155+0000 7fa10be00640 1 -- 172.21.15.151:0/1249210098 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1140d48a0 msgr2=0x7fa1140b83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:58.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.155+0000 7fa10be00640 1 --2- 172.21.15.151:0/1249210098 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1140d48a0 0x7fa1140b83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:58.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.155+0000 7fa10be00640 1 -- 172.21.15.151:0/1249210098 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1140b94c0 con 0x7fa1140d0660 2024-09-16T15:44:58.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.156+0000 7fa10be00640 1 --2- 172.21.15.151:0/1249210098 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa1140d0660 0x7fa1141f5630 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc034040 tx=0x7fa0fc003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:58.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.156+0000 7fa10aa00640 1 -- 172.21.15.151:0/1249210098 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc03e030 con 0x7fa1140d0660 2024-09-16T15:44:58.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.156+0000 7fa10aa00640 1 -- 172.21.15.151:0/1249210098 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa0fc003e00 con 0x7fa1140d0660 2024-09-16T15:44:58.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.156+0000 7fa10aa00640 1 -- 172.21.15.151:0/1249210098 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc035040 con 0x7fa1140d0660 2024-09-16T15:44:58.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.156+0000 7fa113e00640 1 -- 172.21.15.151:0/1249210098 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1140cc830 con 0x7fa1140d0660 2024-09-16T15:44:58.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.156+0000 7fa113e00640 1 -- 172.21.15.151:0/1249210098 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1140ccdf0 con 0x7fa1140d0660 2024-09-16T15:44:58.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.157+0000 7fa113e00640 1 -- 172.21.15.151:0/1249210098 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1140c7660 con 0x7fa1140d0660 2024-09-16T15:44:58.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.158+0000 7fa10aa00640 1 -- 172.21.15.151:0/1249210098 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 102103+0+0 (secure 0 0 0) 0x7fa0fc041030 con 0x7fa1140d0660 2024-09-16T15:44:58.160 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.159+0000 7fa10aa00640 1 --2- 172.21.15.151:0/1249210098 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa0f0079180 0x7fa0f007b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:58.160 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.159+0000 7fa10aa00640 1 -- 172.21.15.151:0/1249210098 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7fa0fc0b9870 con 0x7fa1140d0660 2024-09-16T15:44:58.161 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.160+0000 7fa112a00640 1 --2- 172.21.15.151:0/1249210098 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa0f0079180 0x7fa0f007b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:58.161 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.161+0000 7fa112a00640 1 --2- 172.21.15.151:0/1249210098 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa0f0079180 0x7fa0f007b640 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa104001680 tx=0x7fa104009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:58.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.163+0000 7fa10aa00640 1 -- 172.21.15.151:0/1249210098 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa1140c7660 con 0x7fa1140d0660 2024-09-16T15:44:58.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.384+0000 7fa113e00640 1 -- 172.21.15.151:0/1249210098 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fa1140cc650 con 0x7fa1140d0660 2024-09-16T15:44:58.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.760+0000 7fa10aa00640 1 -- 172.21.15.151:0/1249210098 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v37) ==== 136+0+0 (secure 0 0 0) 0x7fa1140cc650 con 0x7fa1140d0660 2024-09-16T15:44:58.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.762+0000 7fa113e00640 1 -- 172.21.15.151:0/1249210098 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa0f0079180 msgr2=0x7fa0f007b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:58.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.762+0000 7fa113e00640 1 --2- 172.21.15.151:0/1249210098 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa0f0079180 0x7fa0f007b640 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa104001680 tx=0x7fa104009040 comp rx=0 tx=0).stop 2024-09-16T15:44:58.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.762+0000 7fa113e00640 1 -- 172.21.15.151:0/1249210098 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa1140d0660 msgr2=0x7fa1141f5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:58.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.762+0000 7fa113e00640 1 --2- 172.21.15.151:0/1249210098 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa1140d0660 0x7fa1141f5630 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc034040 tx=0x7fa0fc003910 comp rx=0 tx=0).stop 2024-09-16T15:44:58.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.762+0000 7fa113e00640 1 -- 172.21.15.151:0/1249210098 shutdown_connections 2024-09-16T15:44:58.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.762+0000 7fa113e00640 1 --2- 172.21.15.151:0/1249210098 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa0f0079180 0x7fa0f007b640 secure :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa104001680 tx=0x7fa104009040 comp rx=0 tx=0).stop 2024-09-16T15:44:58.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.762+0000 7fa113e00640 1 --2- 172.21.15.151:0/1249210098 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1140d48a0 0x7fa1140b83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:58.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.762+0000 7fa113e00640 1 --2- 172.21.15.151:0/1249210098 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa1140d0660 0x7fa1141f5630 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:58.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.762+0000 7fa113e00640 1 --2- 172.21.15.151:0/1249210098 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa1140b4820 0x7fa1140cd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:58.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.762+0000 7fa113e00640 1 -- 172.21.15.151:0/1249210098 >> 172.21.15.151:0/1249210098 conn(0x7fa1140d3d80 msgr2=0x7fa1141612a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:58.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.763+0000 7fa113e00640 1 -- 172.21.15.151:0/1249210098 shutdown_connections 2024-09-16T15:44:58.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.763+0000 7fa113e00640 1 -- 172.21.15.151:0/1249210098 wait complete. 2024-09-16T15:44:58.780 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-16T15:44:58.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.938+0000 7fd169c00640 1 -- 172.21.15.151:0/2774807334 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd164073fb0 msgr2=0x7fd164074420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:58.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.938+0000 7fd169c00640 1 --2- 172.21.15.151:0/2774807334 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd164073fb0 0x7fd164074420 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fd1640cfbb0 tx=0x7fd15c02a820 comp rx=0 tx=0).stop 2024-09-16T15:44:58.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.938+0000 7fd169c00640 1 -- 172.21.15.151:0/2774807334 shutdown_connections 2024-09-16T15:44:58.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.938+0000 7fd169c00640 1 --2- 172.21.15.151:0/2774807334 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd164073fb0 0x7fd164074420 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:58.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.938+0000 7fd169c00640 1 --2- 172.21.15.151:0/2774807334 >> v2:172.21.15.167:3300/0 conn(0x7fd1640ccf00 0x7fd1640b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:58.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.938+0000 7fd169c00640 1 -- 172.21.15.151:0/2774807334 >> 172.21.15.151:0/2774807334 conn(0x7fd1640b1950 msgr2=0x7fd1640b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:58.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.938+0000 7fd169c00640 1 -- 172.21.15.151:0/2774807334 shutdown_connections 2024-09-16T15:44:58.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.938+0000 7fd169c00640 1 -- 172.21.15.151:0/2774807334 wait complete. 2024-09-16T15:44:58.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.940+0000 7fd169c00640 1 Processor -- start 2024-09-16T15:44:58.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.940+0000 7fd169c00640 1 -- start start 2024-09-16T15:44:58.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.941+0000 7fd169c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd164073fb0 0x7fd164153c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:58.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.941+0000 7fd169c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1640ccf00 0x7fd16415d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:58.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.941+0000 7fd169c00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd16415f5a0 0x7fd16415dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:58.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.941+0000 7fd169c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd16406f2e0 con 0x7fd164073fb0 2024-09-16T15:44:58.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.941+0000 7fd169c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd16406f160 con 0x7fd1640ccf00 2024-09-16T15:44:58.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.941+0000 7fd169c00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fd16406f460 con 0x7fd16415f5a0 2024-09-16T15:44:58.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.941+0000 7fd162a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd164073fb0 0x7fd164153c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:58.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.941+0000 7fd162a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd164073fb0 0x7fd164153c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:39596/0 (socket says 172.21.15.151:39596) 2024-09-16T15:44:58.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.941+0000 7fd162a00640 1 -- 172.21.15.151:0/2796305783 learned_addr learned my addr 172.21.15.151:0/2796305783 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:58.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.942+0000 7fd162000640 1 --2- 172.21.15.151:0/2796305783 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1640ccf00 0x7fd16415d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:58.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.942+0000 7fd163400640 1 --2- 172.21.15.151:0/2796305783 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd16415f5a0 0x7fd16415dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:58.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.942+0000 7fd162a00640 1 -- 172.21.15.151:0/2796305783 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd16415f5a0 msgr2=0x7fd16415dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:58.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.942+0000 7fd162a00640 1 --2- 172.21.15.151:0/2796305783 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd16415f5a0 0x7fd16415dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:58.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.942+0000 7fd162a00640 1 -- 172.21.15.151:0/2796305783 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1640ccf00 msgr2=0x7fd16415d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:58.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.942+0000 7fd162a00640 1 --2- 172.21.15.151:0/2796305783 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1640ccf00 0x7fd16415d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:58.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.942+0000 7fd162a00640 1 -- 172.21.15.151:0/2796305783 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1641541c0 con 0x7fd164073fb0 2024-09-16T15:44:58.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.942+0000 7fd163400640 1 --2- 172.21.15.151:0/2796305783 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd16415f5a0 0x7fd16415dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:44:58.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.942+0000 7fd162a00640 1 --2- 172.21.15.151:0/2796305783 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd164073fb0 0x7fd164153c80 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fd15400c3c0 tx=0x7fd15400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:58.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.942+0000 7fd15aa00640 1 -- 172.21.15.151:0/2796305783 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd154017070 con 0x7fd164073fb0 2024-09-16T15:44:58.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.942+0000 7fd15aa00640 1 -- 172.21.15.151:0/2796305783 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd1540035f0 con 0x7fd164073fb0 2024-09-16T15:44:58.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.942+0000 7fd169c00640 1 -- 172.21.15.151:0/2796305783 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1641f4730 con 0x7fd164073fb0 2024-09-16T15:44:58.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.942+0000 7fd15aa00640 1 -- 172.21.15.151:0/2796305783 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd154003a80 con 0x7fd164073fb0 2024-09-16T15:44:58.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.942+0000 7fd169c00640 1 -- 172.21.15.151:0/2796305783 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1641f4fd0 con 0x7fd164073fb0 2024-09-16T15:44:58.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.944+0000 7fd169c00640 1 -- 172.21.15.151:0/2796305783 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd16415e490 con 0x7fd164073fb0 2024-09-16T15:44:58.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.945+0000 7fd15aa00640 1 -- 172.21.15.151:0/2796305783 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 102103+0+0 (secure 0 0 0) 0x7fd15401a070 con 0x7fd164073fb0 2024-09-16T15:44:58.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.945+0000 7fd15aa00640 1 --2- 172.21.15.151:0/2796305783 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd144079180 0x7fd14407b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:58.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.946+0000 7fd15aa00640 1 -- 172.21.15.151:0/2796305783 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7fd1540983e0 con 0x7fd164073fb0 2024-09-16T15:44:58.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.946+0000 7fd162000640 1 --2- 172.21.15.151:0/2796305783 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd144079180 0x7fd14407b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:58.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.947+0000 7fd162000640 1 --2- 172.21.15.151:0/2796305783 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd144079180 0x7fd14407b640 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd150005fd0 tx=0x7fd150005850 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:58.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:58.950+0000 7fd15aa00640 1 -- 172.21.15.151:0/2796305783 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd154062500 con 0x7fd164073fb0 2024-09-16T15:44:59.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.157+0000 7fd169c00640 1 -- 172.21.15.151:0/2796305783 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fd1640b1480 con 0x7fd144079180 2024-09-16T15:44:59.160 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.159+0000 7fd15aa00640 1 -- 172.21.15.151:0/2796305783 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fd1640b1480 con 0x7fd144079180 2024-09-16T15:44:59.160 INFO:teuthology.orchestra.run.smithi151.stdout:[ 2024-09-16T15:44:59.160 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:44:59.160 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "cephfs" 2024-09-16T15:44:59.160 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:44:59.160 INFO:teuthology.orchestra.run.smithi151.stdout:] 2024-09-16T15:44:59.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.161+0000 7fd169c00640 1 -- 172.21.15.151:0/2796305783 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd144079180 msgr2=0x7fd14407b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:59.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.161+0000 7fd169c00640 1 --2- 172.21.15.151:0/2796305783 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd144079180 0x7fd14407b640 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd150005fd0 tx=0x7fd150005850 comp rx=0 tx=0).stop 2024-09-16T15:44:59.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.161+0000 7fd169c00640 1 -- 172.21.15.151:0/2796305783 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd164073fb0 msgr2=0x7fd164153c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:59.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.161+0000 7fd169c00640 1 --2- 172.21.15.151:0/2796305783 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd164073fb0 0x7fd164153c80 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fd15400c3c0 tx=0x7fd15400c880 comp rx=0 tx=0).stop 2024-09-16T15:44:59.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.161+0000 7fd169c00640 1 -- 172.21.15.151:0/2796305783 shutdown_connections 2024-09-16T15:44:59.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.161+0000 7fd169c00640 1 --2- 172.21.15.151:0/2796305783 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd144079180 0x7fd14407b640 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:59.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.161+0000 7fd169c00640 1 --2- 172.21.15.151:0/2796305783 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1640ccf00 0x7fd16415d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:59.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.161+0000 7fd169c00640 1 --2- 172.21.15.151:0/2796305783 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd16415f5a0 0x7fd16415dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:59.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.161+0000 7fd169c00640 1 --2- 172.21.15.151:0/2796305783 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd164073fb0 0x7fd164153c80 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:59.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.161+0000 7fd169c00640 1 -- 172.21.15.151:0/2796305783 >> 172.21.15.151:0/2796305783 conn(0x7fd1640b1950 msgr2=0x7fd1640acd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:59.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.161+0000 7fd169c00640 1 -- 172.21.15.151:0/2796305783 shutdown_connections 2024-09-16T15:44:59.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.161+0000 7fd169c00640 1 -- 172.21.15.151:0/2796305783 wait complete. 2024-09-16T15:44:59.179 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-16T15:44:59.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.340+0000 7fa9a0600640 1 -- 172.21.15.151:0/4076447488 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9980cccd0 msgr2=0x7fa9980cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:59.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.340+0000 7fa9a0600640 1 --2- 172.21.15.151:0/4076447488 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9980cccd0 0x7fa9980cdbb0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fa990005fd0 tx=0x7fa99002a820 comp rx=0 tx=0).stop 2024-09-16T15:44:59.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.340+0000 7fa9a0600640 1 -- 172.21.15.151:0/4076447488 shutdown_connections 2024-09-16T15:44:59.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.340+0000 7fa9a0600640 1 --2- 172.21.15.151:0/4076447488 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9980cccd0 0x7fa9980cdbb0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:59.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.340+0000 7fa9a0600640 1 --2- 172.21.15.151:0/4076447488 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa998145750 0x7fa998073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:59.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.340+0000 7fa9a0600640 1 -- 172.21.15.151:0/4076447488 >> 172.21.15.151:0/4076447488 conn(0x7fa9980b1840 msgr2=0x7fa9980b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:59.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.340+0000 7fa9a0600640 1 -- 172.21.15.151:0/4076447488 shutdown_connections 2024-09-16T15:44:59.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.340+0000 7fa9a0600640 1 -- 172.21.15.151:0/4076447488 wait complete. 2024-09-16T15:44:59.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.342+0000 7fa9a0600640 1 Processor -- start 2024-09-16T15:44:59.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.342+0000 7fa9a0600640 1 -- start start 2024-09-16T15:44:59.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.343+0000 7fa9a0600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa99806efa0 0x7fa9981f4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:59.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.343+0000 7fa9a0600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9980cccd0 0x7fa99813ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:59.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.343+0000 7fa9a0600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa998145750 0x7fa99813f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:59.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.343+0000 7fa9a0600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa9980ce480 con 0x7fa99806efa0 2024-09-16T15:44:59.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.343+0000 7fa9a0600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa9980ce300 con 0x7fa998145750 2024-09-16T15:44:59.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.343+0000 7fa9a0600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fa9980ce600 con 0x7fa9980cccd0 2024-09-16T15:44:59.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.343+0000 7fa99dc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa99806efa0 0x7fa9981f4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:59.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.343+0000 7fa99dc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa99806efa0 0x7fa9981f4da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:39606/0 (socket says 172.21.15.151:39606) 2024-09-16T15:44:59.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.343+0000 7fa99dc00640 1 -- 172.21.15.151:0/2678543497 learned_addr learned my addr 172.21.15.151:0/2678543497 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:59.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.344+0000 7fa99d200640 1 --2- 172.21.15.151:0/2678543497 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9980cccd0 0x7fa99813ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:59.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.344+0000 7fa99e600640 1 --2- 172.21.15.151:0/2678543497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa998145750 0x7fa99813f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:59.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.344+0000 7fa99dc00640 1 -- 172.21.15.151:0/2678543497 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9980cccd0 msgr2=0x7fa99813ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:59.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.344+0000 7fa99dc00640 1 --2- 172.21.15.151:0/2678543497 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9980cccd0 0x7fa99813ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:59.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.344+0000 7fa99dc00640 1 -- 172.21.15.151:0/2678543497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa998145750 msgr2=0x7fa99813f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:59.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.344+0000 7fa99dc00640 1 --2- 172.21.15.151:0/2678543497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa998145750 0x7fa99813f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:59.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.344+0000 7fa99dc00640 1 -- 172.21.15.151:0/2678543497 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9981f56f0 con 0x7fa99806efa0 2024-09-16T15:44:59.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.344+0000 7fa99dc00640 1 --2- 172.21.15.151:0/2678543497 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa99806efa0 0x7fa9981f4da0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fa98c00d740 tx=0x7fa98c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:59.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.344+0000 7fa995600640 1 -- 172.21.15.151:0/2678543497 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa98c017070 con 0x7fa99806efa0 2024-09-16T15:44:59.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.344+0000 7fa995600640 1 -- 172.21.15.151:0/2678543497 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa98c009d20 con 0x7fa99806efa0 2024-09-16T15:44:59.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.344+0000 7fa995600640 1 -- 172.21.15.151:0/2678543497 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa98c0106c0 con 0x7fa99806efa0 2024-09-16T15:44:59.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.344+0000 7fa9a0600640 1 -- 172.21.15.151:0/2678543497 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9981f59d0 con 0x7fa99806efa0 2024-09-16T15:44:59.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.345+0000 7fa9a0600640 1 -- 172.21.15.151:0/2678543497 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9980b2d10 con 0x7fa99806efa0 2024-09-16T15:44:59.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.346+0000 7fa9a0600640 1 -- 172.21.15.151:0/2678543497 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9980cdbb0 con 0x7fa99806efa0 2024-09-16T15:44:59.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.347+0000 7fa995600640 1 -- 172.21.15.151:0/2678543497 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 102103+0+0 (secure 0 0 0) 0x7fa98c01a070 con 0x7fa99806efa0 2024-09-16T15:44:59.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.347+0000 7fa995600640 1 --2- 172.21.15.151:0/2678543497 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa978079180 0x7fa97807b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:59.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.348+0000 7fa995600640 1 -- 172.21.15.151:0/2678543497 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7fa98c099330 con 0x7fa99806efa0 2024-09-16T15:44:59.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.348+0000 7fa99d200640 1 --2- 172.21.15.151:0/2678543497 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa978079180 0x7fa97807b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:59.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.349+0000 7fa99d200640 1 --2- 172.21.15.151:0/2678543497 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa978079180 0x7fa97807b640 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa9981445f0 tx=0x7fa984009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:59.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.352+0000 7fa995600640 1 -- 172.21.15.151:0/2678543497 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa98c0633f0 con 0x7fa99806efa0 2024-09-16T15:44:59.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.535+0000 7fa9a0600640 1 -- 172.21.15.151:0/2678543497 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fa99813c530 con 0x7fa99806efa0 2024-09-16T15:44:59.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.536+0000 7fa995600640 1 -- 172.21.15.151:0/2678543497 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v34)=0 v34) ==== 122+0+0 (secure 0 0 0) 0x7fa98c062cf0 con 0x7fa99806efa0 2024-09-16T15:44:59.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.539+0000 7fa9a0600640 1 -- 172.21.15.151:0/2678543497 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa978079180 msgr2=0x7fa97807b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:59.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.539+0000 7fa9a0600640 1 --2- 172.21.15.151:0/2678543497 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa978079180 0x7fa97807b640 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa9981445f0 tx=0x7fa984009040 comp rx=0 tx=0).stop 2024-09-16T15:44:59.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.539+0000 7fa9a0600640 1 -- 172.21.15.151:0/2678543497 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa99806efa0 msgr2=0x7fa9981f4da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:59.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.539+0000 7fa9a0600640 1 --2- 172.21.15.151:0/2678543497 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa99806efa0 0x7fa9981f4da0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fa98c00d740 tx=0x7fa98c00dc00 comp rx=0 tx=0).stop 2024-09-16T15:44:59.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.539+0000 7fa9a0600640 1 -- 172.21.15.151:0/2678543497 shutdown_connections 2024-09-16T15:44:59.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.539+0000 7fa9a0600640 1 --2- 172.21.15.151:0/2678543497 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa978079180 0x7fa97807b640 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:59.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.539+0000 7fa9a0600640 1 --2- 172.21.15.151:0/2678543497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa998145750 0x7fa99813f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:59.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.539+0000 7fa9a0600640 1 --2- 172.21.15.151:0/2678543497 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9980cccd0 0x7fa99813ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:59.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.539+0000 7fa9a0600640 1 --2- 172.21.15.151:0/2678543497 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa99806efa0 0x7fa9981f4da0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:59.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.539+0000 7fa9a0600640 1 -- 172.21.15.151:0/2678543497 >> 172.21.15.151:0/2678543497 conn(0x7fa9980b1840 msgr2=0x7fa9980a9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:59.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.539+0000 7fa9a0600640 1 -- 172.21.15.151:0/2678543497 shutdown_connections 2024-09-16T15:44:59.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.539+0000 7fa9a0600640 1 -- 172.21.15.151:0/2678543497 wait complete. 2024-09-16T15:44:59.556 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mgr mgr/volumes/max_concurrent_clones 2024-09-16T15:44:59.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.716+0000 7f276e200640 1 -- 172.21.15.151:0/2903362210 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f276815e6d0 msgr2=0x7f276815eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:59.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.716+0000 7f276e200640 1 --2- 172.21.15.151:0/2903362210 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f276815e6d0 0x7f276815eb60 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f2768076690 tx=0x7f276002a580 comp rx=0 tx=0).stop 2024-09-16T15:44:59.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.716+0000 7f276e200640 1 -- 172.21.15.151:0/2903362210 shutdown_connections 2024-09-16T15:44:59.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.716+0000 7f276e200640 1 --2- 172.21.15.151:0/2903362210 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f276815e6d0 0x7f276815eb60 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:59.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.716+0000 7f276e200640 1 --2- 172.21.15.151:0/2903362210 >> v2:172.21.15.167:3300/0 conn(0x7f27680aec50 0x7f27680af090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:59.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.716+0000 7f276e200640 1 --2- 172.21.15.151:0/2903362210 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27680737d0 0x7f2768073bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:59.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.716+0000 7f276e200640 1 -- 172.21.15.151:0/2903362210 >> 172.21.15.151:0/2903362210 conn(0x7f27680d0d40 msgr2=0x7f27680d1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:59.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.716+0000 7f276e200640 1 -- 172.21.15.151:0/2903362210 shutdown_connections 2024-09-16T15:44:59.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.716+0000 7f276e200640 1 -- 172.21.15.151:0/2903362210 wait complete. 2024-09-16T15:44:59.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.718+0000 7f276e200640 1 Processor -- start 2024-09-16T15:44:59.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.718+0000 7f276e200640 1 -- start start 2024-09-16T15:44:59.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.719+0000 7f276e200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f27680737d0 0x7f27680b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:59.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.719+0000 7f276e200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27680aec50 0x7f27681f3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:59.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.719+0000 7f276e200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f276815e6d0 0x7f276814c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:59.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.719+0000 7f276e200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f276815f0a0 con 0x7f27680737d0 2024-09-16T15:44:59.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.719+0000 7f276e200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2768072110 con 0x7f276815e6d0 2024-09-16T15:44:59.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.719+0000 7f276e200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f276815f220 con 0x7f27680aec50 2024-09-16T15:44:59.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.719+0000 7f2767400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f27680737d0 0x7f27680b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:59.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.719+0000 7f2767400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f27680737d0 0x7f27680b4d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:39626/0 (socket says 172.21.15.151:39626) 2024-09-16T15:44:59.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.719+0000 7f2767400640 1 -- 172.21.15.151:0/3965750465 learned_addr learned my addr 172.21.15.151:0/3965750465 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:44:59.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.719+0000 7f2767e00640 1 --2- 172.21.15.151:0/3965750465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f276815e6d0 0x7f276814c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:59.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.719+0000 7f2766a00640 1 --2- 172.21.15.151:0/3965750465 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27680aec50 0x7f27681f3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:59.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.720+0000 7f2767400640 1 -- 172.21.15.151:0/3965750465 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27680aec50 msgr2=0x7f27681f3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:59.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.720+0000 7f2767400640 1 --2- 172.21.15.151:0/3965750465 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27680aec50 0x7f27681f3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:59.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.720+0000 7f2767400640 1 -- 172.21.15.151:0/3965750465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f276815e6d0 msgr2=0x7f276814c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:59.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.720+0000 7f2767400640 1 --2- 172.21.15.151:0/3965750465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f276815e6d0 0x7f276814c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:59.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.720+0000 7f2767400640 1 -- 172.21.15.151:0/3965750465 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27681518f0 con 0x7f27680737d0 2024-09-16T15:44:59.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.720+0000 7f2767e00640 1 --2- 172.21.15.151:0/3965750465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f276815e6d0 0x7f276814c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:44:59.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.720+0000 7f2767400640 1 --2- 172.21.15.151:0/3965750465 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f27680737d0 0x7f27680b4d40 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f275800ead0 tx=0x7f275800ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:59.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.720+0000 7f275f400640 1 -- 172.21.15.151:0/3965750465 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f275800cd80 con 0x7f27680737d0 2024-09-16T15:44:59.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.720+0000 7f275f400640 1 -- 172.21.15.151:0/3965750465 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2758004b20 con 0x7f27680737d0 2024-09-16T15:44:59.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.720+0000 7f276e200640 1 -- 172.21.15.151:0/3965750465 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27681f39a0 con 0x7f27680737d0 2024-09-16T15:44:59.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.720+0000 7f275f400640 1 -- 172.21.15.151:0/3965750465 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2758005020 con 0x7f27680737d0 2024-09-16T15:44:59.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.720+0000 7f276e200640 1 -- 172.21.15.151:0/3965750465 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2768151e70 con 0x7f27680737d0 2024-09-16T15:44:59.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.722+0000 7f276e200640 1 -- 172.21.15.151:0/3965750465 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27680af090 con 0x7f27680737d0 2024-09-16T15:44:59.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.723+0000 7f275f400640 1 -- 172.21.15.151:0/3965750465 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 102103+0+0 (secure 0 0 0) 0x7f27580051c0 con 0x7f27680737d0 2024-09-16T15:44:59.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.724+0000 7f275f400640 1 --2- 172.21.15.151:0/3965750465 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2748079150 0x7f274807b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:44:59.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.724+0000 7f275f400640 1 -- 172.21.15.151:0/3965750465 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7f2758099b10 con 0x7f27680737d0 2024-09-16T15:44:59.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.724+0000 7f2766a00640 1 --2- 172.21.15.151:0/3965750465 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2748079150 0x7f274807b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:44:59.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.725+0000 7f2766a00640 1 --2- 172.21.15.151:0/3965750465 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2748079150 0x7f274807b610 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f276815efe0 tx=0x7f275400b5b0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:44:59.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.728+0000 7f275f400640 1 -- 172.21.15.151:0/3965750465 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2758016030 con 0x7f27680737d0 2024-09-16T15:44:59.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.916+0000 7f276e200640 1 -- 172.21.15.151:0/3965750465 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"} v 0) -- 0x7f2768074a90 con 0x7f27680737d0 2024-09-16T15:44:59.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.917+0000 7f275f400640 1 -- 172.21.15.151:0/3965750465 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"}]=0 v34) ==== 116+0+2 (secure 0 0 0) 0x7f2758063c30 con 0x7f27680737d0 2024-09-16T15:44:59.918 INFO:teuthology.orchestra.run.smithi151.stdout:4 2024-09-16T15:44:59.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.919+0000 7f276e200640 1 -- 172.21.15.151:0/3965750465 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2748079150 msgr2=0x7f274807b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:59.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.919+0000 7f276e200640 1 --2- 172.21.15.151:0/3965750465 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2748079150 0x7f274807b610 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f276815efe0 tx=0x7f275400b5b0 comp rx=0 tx=0).stop 2024-09-16T15:44:59.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.919+0000 7f276e200640 1 -- 172.21.15.151:0/3965750465 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f27680737d0 msgr2=0x7f27680b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:44:59.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.919+0000 7f276e200640 1 --2- 172.21.15.151:0/3965750465 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f27680737d0 0x7f27680b4d40 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f275800ead0 tx=0x7f275800ef90 comp rx=0 tx=0).stop 2024-09-16T15:44:59.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.919+0000 7f276e200640 1 -- 172.21.15.151:0/3965750465 shutdown_connections 2024-09-16T15:44:59.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.919+0000 7f276e200640 1 --2- 172.21.15.151:0/3965750465 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2748079150 0x7f274807b610 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:59.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.919+0000 7f276e200640 1 --2- 172.21.15.151:0/3965750465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f276815e6d0 0x7f276814c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:59.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.919+0000 7f276e200640 1 --2- 172.21.15.151:0/3965750465 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27680aec50 0x7f27681f3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:59.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.919+0000 7f276e200640 1 --2- 172.21.15.151:0/3965750465 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f27680737d0 0x7f27680b4d40 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:44:59.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.919+0000 7f276e200640 1 -- 172.21.15.151:0/3965750465 >> 172.21.15.151:0/3965750465 conn(0x7f27680d0d40 msgr2=0x7f2768071680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:44:59.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.919+0000 7f276e200640 1 -- 172.21.15.151:0/3965750465 shutdown_connections 2024-09-16T15:44:59.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:44:59.919+0000 7f276e200640 1 -- 172.21.15.151:0/3965750465 wait complete. 2024-09-16T15:44:59.937 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-16T15:45:00.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.097+0000 7f4355e00640 1 -- 172.21.15.151:0/403011485 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4350161760 msgr2=0x7f43500ce4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:00.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.097+0000 7f4355e00640 1 --2- 172.21.15.151:0/403011485 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4350161760 0x7f43500ce4a0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f43500cd800 tx=0x7f434002a5f0 comp rx=0 tx=0).stop 2024-09-16T15:45:00.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.098+0000 7f4355e00640 1 -- 172.21.15.151:0/403011485 shutdown_connections 2024-09-16T15:45:00.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.098+0000 7f4355e00640 1 --2- 172.21.15.151:0/403011485 >> v2:172.21.15.167:3300/0 conn(0x7f43500cec60 0x7f43500cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:00.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.098+0000 7f4355e00640 1 --2- 172.21.15.151:0/403011485 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4350161760 0x7f43500ce4a0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:00.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.098+0000 7f4355e00640 1 -- 172.21.15.151:0/403011485 >> 172.21.15.151:0/403011485 conn(0x7f43500b1840 msgr2=0x7f43500b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:00.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.098+0000 7f4355e00640 1 -- 172.21.15.151:0/403011485 shutdown_connections 2024-09-16T15:45:00.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.098+0000 7f4355e00640 1 -- 172.21.15.151:0/403011485 wait complete. 2024-09-16T15:45:00.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.099+0000 7f4355e00640 1 Processor -- start 2024-09-16T15:45:00.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.099+0000 7f4355e00640 1 -- start start 2024-09-16T15:45:00.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.100+0000 7f4355e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f43500af190 0x7f435015b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:00.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.100+0000 7f4355e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43500cec60 0x7f435015b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:00.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.100+0000 7f4355e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4350161760 0x7f43501f4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:00.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.100+0000 7f4355e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4350162310 con 0x7f43500af190 2024-09-16T15:45:00.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.100+0000 7f4355e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4350162190 con 0x7f43500cec60 2024-09-16T15:45:00.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.100+0000 7f4355e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f4350162490 con 0x7f4350161760 2024-09-16T15:45:00.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.101+0000 7f434ea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f43500af190 0x7f435015b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:00.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.101+0000 7f434ea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f43500af190 0x7f435015b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:39656/0 (socket says 172.21.15.151:39656) 2024-09-16T15:45:00.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.101+0000 7f434ea00640 1 -- 172.21.15.151:0/2515295845 learned_addr learned my addr 172.21.15.151:0/2515295845 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:45:00.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.101+0000 7f434e000640 1 --2- 172.21.15.151:0/2515295845 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43500cec60 0x7f435015b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:00.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.101+0000 7f434f400640 1 --2- 172.21.15.151:0/2515295845 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4350161760 0x7f43501f4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:00.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.101+0000 7f434ea00640 1 -- 172.21.15.151:0/2515295845 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4350161760 msgr2=0x7f43501f4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:00.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.101+0000 7f434ea00640 1 --2- 172.21.15.151:0/2515295845 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4350161760 0x7f43501f4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:00.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.101+0000 7f434ea00640 1 -- 172.21.15.151:0/2515295845 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43500cec60 msgr2=0x7f435015b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:00.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.101+0000 7f434ea00640 1 --2- 172.21.15.151:0/2515295845 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43500cec60 0x7f435015b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:00.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.101+0000 7f434ea00640 1 -- 172.21.15.151:0/2515295845 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43501477d0 con 0x7f43500af190 2024-09-16T15:45:00.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.101+0000 7f434ea00640 1 --2- 172.21.15.151:0/2515295845 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f43500af190 0x7f435015b0a0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f4340034040 tx=0x7f43400039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:00.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.102+0000 7f4347400640 1 -- 172.21.15.151:0/2515295845 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434003e030 con 0x7f43500af190 2024-09-16T15:45:00.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.102+0000 7f4347400640 1 -- 172.21.15.151:0/2515295845 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f43400047a0 con 0x7f43500af190 2024-09-16T15:45:00.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.102+0000 7f4347400640 1 -- 172.21.15.151:0/2515295845 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4340035040 con 0x7f43500af190 2024-09-16T15:45:00.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.102+0000 7f4355e00640 1 -- 172.21.15.151:0/2515295845 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4350147ab0 con 0x7f43500af190 2024-09-16T15:45:00.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.102+0000 7f4355e00640 1 -- 172.21.15.151:0/2515295845 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4350148070 con 0x7f43500af190 2024-09-16T15:45:00.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.103+0000 7f4355e00640 1 -- 172.21.15.151:0/2515295845 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4350162980 con 0x7f43500af190 2024-09-16T15:45:00.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.104+0000 7f4347400640 1 -- 172.21.15.151:0/2515295845 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 102103+0+0 (secure 0 0 0) 0x7f4340041030 con 0x7f43500af190 2024-09-16T15:45:00.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.105+0000 7f4347400640 1 --2- 172.21.15.151:0/2515295845 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f432c079150 0x7f432c07b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:00.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.105+0000 7f4347400640 1 -- 172.21.15.151:0/2515295845 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7f43400b9940 con 0x7f43500af190 2024-09-16T15:45:00.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.105+0000 7f434e000640 1 --2- 172.21.15.151:0/2515295845 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f432c079150 0x7f432c07b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:00.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.106+0000 7f434e000640 1 --2- 172.21.15.151:0/2515295845 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f432c079150 0x7f432c07b610 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4350160ec0 tx=0x7f4338009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:00.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.109+0000 7f4347400640 1 -- 172.21.15.151:0/2515295845 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4340083a00 con 0x7f43500af190 2024-09-16T15:45:00.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.294+0000 7f4355e00640 1 -- 172.21.15.151:0/2515295845 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f4350073630 con 0x7f43500af190 2024-09-16T15:45:00.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.298+0000 7f4347400640 1 -- 172.21.15.151:0/2515295845 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v35)=0 v35) ==== 131+0+0 (secure 0 0 0) 0x7f4340083300 con 0x7f43500af190 2024-09-16T15:45:00.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.300+0000 7f4355e00640 1 -- 172.21.15.151:0/2515295845 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f432c079150 msgr2=0x7f432c07b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:00.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.300+0000 7f4355e00640 1 --2- 172.21.15.151:0/2515295845 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f432c079150 0x7f432c07b610 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4350160ec0 tx=0x7f4338009040 comp rx=0 tx=0).stop 2024-09-16T15:45:00.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.300+0000 7f4355e00640 1 -- 172.21.15.151:0/2515295845 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f43500af190 msgr2=0x7f435015b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:00.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.300+0000 7f4355e00640 1 --2- 172.21.15.151:0/2515295845 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f43500af190 0x7f435015b0a0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f4340034040 tx=0x7f43400039e0 comp rx=0 tx=0).stop 2024-09-16T15:45:00.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.300+0000 7f4355e00640 1 -- 172.21.15.151:0/2515295845 shutdown_connections 2024-09-16T15:45:00.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.300+0000 7f4355e00640 1 --2- 172.21.15.151:0/2515295845 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f432c079150 0x7f432c07b610 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:00.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.300+0000 7f4355e00640 1 --2- 172.21.15.151:0/2515295845 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43500cec60 0x7f435015b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:00.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.300+0000 7f4355e00640 1 --2- 172.21.15.151:0/2515295845 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4350161760 0x7f43501f4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:00.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.300+0000 7f4355e00640 1 --2- 172.21.15.151:0/2515295845 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f43500af190 0x7f435015b0a0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:00.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.300+0000 7f4355e00640 1 -- 172.21.15.151:0/2515295845 >> 172.21.15.151:0/2515295845 conn(0x7f43500b1840 msgr2=0x7f4350073ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:00.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.300+0000 7f4355e00640 1 -- 172.21.15.151:0/2515295845 shutdown_connections 2024-09-16T15:45:00.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.301+0000 7f4355e00640 1 -- 172.21.15.151:0/2515295845 wait complete. 2024-09-16T15:45:00.317 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-09-16T15:45:00.317 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:45:00.315+0000 7f48f2600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:45:00.318 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:45:00.315+0000 7f48f2600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:45:00.318 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:45:00.315+0000 7f48f2600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:45:00.318 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:45:00.315+0000 7f48f2600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:45:00.318 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:45:00.315+0000 7f48f2600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:45:00.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.478+0000 7f8abd600640 1 -- 172.21.15.151:0/2423341912 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ab80d07b0 msgr2=0x7f8ab8072d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:00.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.478+0000 7f8abd600640 1 --2- 172.21.15.151:0/2423341912 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ab80d07b0 0x7f8ab8072d00 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8aa4005fd0 tx=0x7f8aa402a5d0 comp rx=0 tx=0).stop 2024-09-16T15:45:00.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.478+0000 7f8abd600640 1 -- 172.21.15.151:0/2423341912 shutdown_connections 2024-09-16T15:45:00.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.478+0000 7f8abd600640 1 --2- 172.21.15.151:0/2423341912 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8ab8073cf0 0x7f8ab806ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:00.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.478+0000 7f8abd600640 1 --2- 172.21.15.151:0/2423341912 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ab80d07b0 0x7f8ab8072d00 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:00.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.478+0000 7f8abd600640 1 -- 172.21.15.151:0/2423341912 >> 172.21.15.151:0/2423341912 conn(0x7f8ab80cfa70 msgr2=0x7f8ab80cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:00.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.478+0000 7f8abd600640 1 -- 172.21.15.151:0/2423341912 shutdown_connections 2024-09-16T15:45:00.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.478+0000 7f8abd600640 1 -- 172.21.15.151:0/2423341912 wait complete. 2024-09-16T15:45:00.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.480+0000 7f8abd600640 1 Processor -- start 2024-09-16T15:45:00.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.480+0000 7f8abd600640 1 -- start start 2024-09-16T15:45:00.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.481+0000 7f8abd600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8ab8073cf0 0x7f8ab8145050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:00.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.481+0000 7f8abd600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ab815abd0 0x7f8ab81f5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:00.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.481+0000 7f8abd600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ab81467b0 0x7f8ab8144570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:00.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.481+0000 7f8abd600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab80cd9d0 con 0x7f8ab81467b0 2024-09-16T15:45:00.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.481+0000 7f8abd600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab80cd850 con 0x7f8ab815abd0 2024-09-16T15:45:00.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.481+0000 7f8abd600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f8ab80cdb50 con 0x7f8ab8073cf0 2024-09-16T15:45:00.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.481+0000 7f8abcc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ab81467b0 0x7f8ab8144570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:00.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.481+0000 7f8abcc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ab81467b0 0x7f8ab8144570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:39670/0 (socket says 172.21.15.151:39670) 2024-09-16T15:45:00.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.481+0000 7f8abcc00640 1 -- 172.21.15.151:0/1490948973 learned_addr learned my addr 172.21.15.151:0/1490948973 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:45:00.484 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.482+0000 7f8ab7400640 1 --2- 172.21.15.151:0/1490948973 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ab815abd0 0x7f8ab81f5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:00.484 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.482+0000 7f8ab7e00640 1 --2- 172.21.15.151:0/1490948973 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8ab8073cf0 0x7f8ab8145050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:00.484 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.482+0000 7f8abcc00640 1 -- 172.21.15.151:0/1490948973 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8ab8073cf0 msgr2=0x7f8ab8145050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:00.484 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.482+0000 7f8abcc00640 1 --2- 172.21.15.151:0/1490948973 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8ab8073cf0 0x7f8ab8145050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:00.484 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.482+0000 7f8abcc00640 1 -- 172.21.15.151:0/1490948973 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ab815abd0 msgr2=0x7f8ab81f5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:00.484 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.482+0000 7f8abcc00640 1 --2- 172.21.15.151:0/1490948973 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ab815abd0 0x7f8ab81f5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:00.484 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.482+0000 7f8abcc00640 1 -- 172.21.15.151:0/1490948973 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ab8145710 con 0x7f8ab81467b0 2024-09-16T15:45:00.484 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.482+0000 7f8ab7e00640 1 --2- 172.21.15.151:0/1490948973 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8ab8073cf0 0x7f8ab8145050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:45:00.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.482+0000 7f8abcc00640 1 --2- 172.21.15.151:0/1490948973 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ab81467b0 0x7f8ab8144570 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f8ab000eb10 tx=0x7f8ab000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:00.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.482+0000 7f8aafe00640 1 -- 172.21.15.151:0/1490948973 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab0004b20 con 0x7f8ab81467b0 2024-09-16T15:45:00.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.482+0000 7f8aafe00640 1 -- 172.21.15.151:0/1490948973 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ab0004ce0 con 0x7f8ab81467b0 2024-09-16T15:45:00.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.482+0000 7f8aafe00640 1 -- 172.21.15.151:0/1490948973 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab0005280 con 0x7f8ab81467b0 2024-09-16T15:45:00.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.482+0000 7f8abd600640 1 -- 172.21.15.151:0/1490948973 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ab81459f0 con 0x7f8ab81467b0 2024-09-16T15:45:00.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.483+0000 7f8abd600640 1 -- 172.21.15.151:0/1490948973 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ab8145fb0 con 0x7f8ab81467b0 2024-09-16T15:45:00.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.484+0000 7f8abd600640 1 -- 172.21.15.151:0/1490948973 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ab80ce830 con 0x7f8ab81467b0 2024-09-16T15:45:00.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.485+0000 7f8aafe00640 1 -- 172.21.15.151:0/1490948973 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 102103+0+0 (secure 0 0 0) 0x7f8ab0005420 con 0x7f8ab81467b0 2024-09-16T15:45:00.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.486+0000 7f8aafe00640 1 --2- 172.21.15.151:0/1490948973 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8a98079150 0x7f8a9807b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:00.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.486+0000 7f8aafe00640 1 -- 172.21.15.151:0/1490948973 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7f8ab0099f80 con 0x7f8ab81467b0 2024-09-16T15:45:00.487 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.486+0000 7f8ab7e00640 1 --2- 172.21.15.151:0/1490948973 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8a98079150 0x7f8a9807b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:00.487 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.487+0000 7f8ab7e00640 1 --2- 172.21.15.151:0/1490948973 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8a98079150 0x7f8a9807b610 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8ab81f9470 tx=0x7f8aa8009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:00.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.490+0000 7f8aafe00640 1 -- 172.21.15.151:0/1490948973 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8ab0064040 con 0x7f8ab81467b0 2024-09-16T15:45:00.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.700+0000 7f8abd600640 1 -- 172.21.15.151:0/1490948973 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7f8ab80b59f0 con 0x7f8a98079150 2024-09-16T15:45:00.713 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:45:00.712+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:45:00.713 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:45:00.712+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:45:00.713 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:45:00.712+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:45:00.713 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:45:00.712+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:45:00.713 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:45:00.712+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:45:00.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.751+0000 7f8aafe00640 1 -- 172.21.15.151:0/1490948973 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8ab80b59f0 con 0x7f8a98079150 2024-09-16T15:45:00.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.754+0000 7f8abd600640 1 -- 172.21.15.151:0/1490948973 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8a98079150 msgr2=0x7f8a9807b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:00.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.754+0000 7f8abd600640 1 --2- 172.21.15.151:0/1490948973 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8a98079150 0x7f8a9807b610 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8ab81f9470 tx=0x7f8aa8009040 comp rx=0 tx=0).stop 2024-09-16T15:45:00.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.754+0000 7f8abd600640 1 -- 172.21.15.151:0/1490948973 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ab81467b0 msgr2=0x7f8ab8144570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:00.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.754+0000 7f8abd600640 1 --2- 172.21.15.151:0/1490948973 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ab81467b0 0x7f8ab8144570 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f8ab000eb10 tx=0x7f8ab000c2d0 comp rx=0 tx=0).stop 2024-09-16T15:45:00.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.754+0000 7f8abd600640 1 -- 172.21.15.151:0/1490948973 shutdown_connections 2024-09-16T15:45:00.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.754+0000 7f8abd600640 1 --2- 172.21.15.151:0/1490948973 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8a98079150 0x7f8a9807b610 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:00.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.754+0000 7f8abd600640 1 --2- 172.21.15.151:0/1490948973 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ab815abd0 0x7f8ab81f5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:00.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.754+0000 7f8abd600640 1 --2- 172.21.15.151:0/1490948973 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ab81467b0 0x7f8ab8144570 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:00.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.754+0000 7f8abd600640 1 --2- 172.21.15.151:0/1490948973 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8ab8073cf0 0x7f8ab8145050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:00.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.754+0000 7f8abd600640 1 -- 172.21.15.151:0/1490948973 >> 172.21.15.151:0/1490948973 conn(0x7f8ab80cfa70 msgr2=0x7f8ab80af9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:00.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.754+0000 7f8abd600640 1 -- 172.21.15.151:0/1490948973 shutdown_connections 2024-09-16T15:45:00.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.754+0000 7f8abd600640 1 -- 172.21.15.151:0/1490948973 wait complete. 2024-09-16T15:45:00.771 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-16T15:45:00.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.933+0000 7f1dd8e00640 1 -- 172.21.15.151:0/542647669 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1dd406ff60 msgr2=0x7f1dd40ca7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:00.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.933+0000 7f1dd8e00640 1 --2- 172.21.15.151:0/542647669 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1dd406ff60 0x7f1dd40ca7c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc005fd0 tx=0x7f1dcc02aa90 comp rx=0 tx=0).stop 2024-09-16T15:45:00.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.933+0000 7f1dd8e00640 1 -- 172.21.15.151:0/542647669 shutdown_connections 2024-09-16T15:45:00.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.933+0000 7f1dd8e00640 1 --2- 172.21.15.151:0/542647669 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1dd406ff60 0x7f1dd40ca7c0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:00.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.933+0000 7f1dd8e00640 1 --2- 172.21.15.151:0/542647669 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1dd40cdfa0 0x7f1dd40ae290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:00.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.933+0000 7f1dd8e00640 1 -- 172.21.15.151:0/542647669 >> 172.21.15.151:0/542647669 conn(0x7f1dd40cf9f0 msgr2=0x7f1dd40cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:00.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.933+0000 7f1dd8e00640 1 -- 172.21.15.151:0/542647669 shutdown_connections 2024-09-16T15:45:00.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.933+0000 7f1dd8e00640 1 -- 172.21.15.151:0/542647669 wait complete. 2024-09-16T15:45:00.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.935+0000 7f1dd8e00640 1 Processor -- start 2024-09-16T15:45:00.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.935+0000 7f1dd8e00640 1 -- start start 2024-09-16T15:45:00.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.936+0000 7f1dd8e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1dd406ff60 0x7f1dd40c8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:00.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.936+0000 7f1dd8e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1dd40cdfa0 0x7f1dd40c91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:00.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.936+0000 7f1dd8e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1dd415e0c0 0x7f1dd414afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:00.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.936+0000 7f1dd8e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd40cb1a0 con 0x7f1dd415e0c0 2024-09-16T15:45:00.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.936+0000 7f1dd8e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd40cb020 con 0x7f1dd40cdfa0 2024-09-16T15:45:00.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.936+0000 7f1dd8e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f1dd40cb320 con 0x7f1dd406ff60 2024-09-16T15:45:00.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.936+0000 7f1dd3e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1dd415e0c0 0x7f1dd414afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:00.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.936+0000 7f1dd3e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1dd415e0c0 0x7f1dd414afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:39696/0 (socket says 172.21.15.151:39696) 2024-09-16T15:45:00.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.936+0000 7f1dd3e00640 1 -- 172.21.15.151:0/942850359 learned_addr learned my addr 172.21.15.151:0/942850359 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:45:00.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.936+0000 7f1dcaa00640 1 --2- 172.21.15.151:0/942850359 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1dd40cdfa0 0x7f1dd40c91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:00.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.936+0000 7f1dd3400640 1 --2- 172.21.15.151:0/942850359 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1dd406ff60 0x7f1dd40c8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:00.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.937+0000 7f1dd3e00640 1 -- 172.21.15.151:0/942850359 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1dd406ff60 msgr2=0x7f1dd40c8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:00.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.937+0000 7f1dd3e00640 1 --2- 172.21.15.151:0/942850359 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1dd406ff60 0x7f1dd40c8c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:00.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.937+0000 7f1dd3e00640 1 -- 172.21.15.151:0/942850359 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1dd40cdfa0 msgr2=0x7f1dd40c91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:00.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.937+0000 7f1dd3e00640 1 --2- 172.21.15.151:0/942850359 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1dd40cdfa0 0x7f1dd40c91d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:00.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.937+0000 7f1dd3e00640 1 -- 172.21.15.151:0/942850359 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dd414bbd0 con 0x7f1dd415e0c0 2024-09-16T15:45:00.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.937+0000 7f1dcaa00640 1 --2- 172.21.15.151:0/942850359 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1dd40cdfa0 0x7f1dd40c91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:45:00.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.937+0000 7f1dd3e00640 1 --2- 172.21.15.151:0/942850359 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1dd415e0c0 0x7f1dd414afa0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc009d90 tx=0x7f1dcc003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:00.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.937+0000 7f1dcbe00640 1 -- 172.21.15.151:0/942850359 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dcc046070 con 0x7f1dd415e0c0 2024-09-16T15:45:00.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.937+0000 7f1dcbe00640 1 -- 172.21.15.151:0/942850359 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1dcc004250 con 0x7f1dd415e0c0 2024-09-16T15:45:00.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.937+0000 7f1dd8e00640 1 -- 172.21.15.151:0/942850359 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dd414beb0 con 0x7f1dd415e0c0 2024-09-16T15:45:00.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.937+0000 7f1dcbe00640 1 -- 172.21.15.151:0/942850359 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dcc004660 con 0x7f1dd415e0c0 2024-09-16T15:45:00.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.938+0000 7f1dd8e00640 1 -- 172.21.15.151:0/942850359 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dd414c470 con 0x7f1dd415e0c0 2024-09-16T15:45:00.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.939+0000 7f1dd8e00640 1 -- 172.21.15.151:0/942850359 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dd40cb8b0 con 0x7f1dd415e0c0 2024-09-16T15:45:00.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.940+0000 7f1dcbe00640 1 -- 172.21.15.151:0/942850359 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 18) ==== 102103+0+0 (secure 0 0 0) 0x7f1dcc049070 con 0x7f1dd415e0c0 2024-09-16T15:45:00.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.940+0000 7f1dcbe00640 1 --2- 172.21.15.151:0/942850359 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1db0079150 0x7f1db007b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:45:00.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.941+0000 7f1dcbe00640 1 -- 172.21.15.151:0/942850359 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5737+0+0 (secure 0 0 0) 0x7f1dcc0b9ff0 con 0x7f1dd415e0c0 2024-09-16T15:45:00.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.941+0000 7f1dd3400640 1 --2- 172.21.15.151:0/942850359 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1db0079150 0x7f1db007b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:45:00.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.942+0000 7f1dd3400640 1 --2- 172.21.15.151:0/942850359 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1db0079150 0x7f1db007b610 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f1dc4001720 tx=0x7f1dc4009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:45:00.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:00.945+0000 7f1dcbe00640 1 -- 172.21.15.151:0/942850359 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1dcc032030 con 0x7f1dd415e0c0 2024-09-16T15:45:01.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:01.157+0000 7f1dd8e00640 1 -- 172.21.15.151:0/942850359 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f1dd40cda10 con 0x7f1db0079150 2024-09-16T15:45:01.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:01.163+0000 7f1dcbe00640 1 -- 172.21.15.151:0/942850359 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f1dd40cda10 con 0x7f1db0079150 2024-09-16T15:45:01.165 INFO:teuthology.orchestra.run.smithi151.stdout:/volumes/_nogroup/sv1/bfa16850-416b-4673-9cfd-a73963609af7 2024-09-16T15:45:01.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:01.166+0000 7f1dd8e00640 1 -- 172.21.15.151:0/942850359 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1db0079150 msgr2=0x7f1db007b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:01.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:01.166+0000 7f1dd8e00640 1 --2- 172.21.15.151:0/942850359 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1db0079150 0x7f1db007b610 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f1dc4001720 tx=0x7f1dc4009040 comp rx=0 tx=0).stop 2024-09-16T15:45:01.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:01.166+0000 7f1dd8e00640 1 -- 172.21.15.151:0/942850359 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1dd415e0c0 msgr2=0x7f1dd414afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:45:01.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:01.166+0000 7f1dd8e00640 1 --2- 172.21.15.151:0/942850359 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1dd415e0c0 0x7f1dd414afa0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc009d90 tx=0x7f1dcc003ce0 comp rx=0 tx=0).stop 2024-09-16T15:45:01.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:01.166+0000 7f1dd8e00640 1 -- 172.21.15.151:0/942850359 shutdown_connections 2024-09-16T15:45:01.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:01.166+0000 7f1dd8e00640 1 --2- 172.21.15.151:0/942850359 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1db0079150 0x7f1db007b610 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:01.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:01.166+0000 7f1dd8e00640 1 --2- 172.21.15.151:0/942850359 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1dd40cdfa0 0x7f1dd40c91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:01.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:01.166+0000 7f1dd8e00640 1 --2- 172.21.15.151:0/942850359 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1dd415e0c0 0x7f1dd414afa0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:01.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:01.166+0000 7f1dd8e00640 1 --2- 172.21.15.151:0/942850359 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1dd406ff60 0x7f1dd40c8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:45:01.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:01.166+0000 7f1dd8e00640 1 -- 172.21.15.151:0/942850359 >> 172.21.15.151:0/942850359 conn(0x7f1dd40cf9f0 msgr2=0x7f1dd40af3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:45:01.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:01.166+0000 7f1dd8e00640 1 -- 172.21.15.151:0/942850359 shutdown_connections 2024-09-16T15:45:01.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:45:01.166+0000 7f1dd8e00640 1 -- 172.21.15.151:0/942850359 wait complete. 2024-09-16T15:45:01.183 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 10 files each 1024MB size under directory volumes/_nogroup/sv1/bfa16850-416b-4673-9cfd-a73963609af7 2024-09-16T15:45:01.183 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:45:01.183 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:45:01.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:45:01.219 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/bfa16850-416b-4673-9cfd-a73963609af7/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T15:45:09.159 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records in 2024-09-16T15:45:09.159 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records out 2024-09-16T15:45:09.159 INFO:teuthology.orchestra.run.smithi151.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.85737 s, 137 MB/s 2024-09-16T15:45:09.160 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:45:09.160 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:45:09.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:45:09.196 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/bfa16850-416b-4673-9cfd-a73963609af7/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T15:45:17.035 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records in 2024-09-16T15:45:17.036 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records out 2024-09-16T15:45:17.036 INFO:teuthology.orchestra.run.smithi151.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.76321 s, 138 MB/s 2024-09-16T15:45:17.037 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:45:17.037 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:45:17.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:45:17.073 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/bfa16850-416b-4673-9cfd-a73963609af7/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T15:45:18.142 DEBUG:teuthology.orchestra.run.smithi151:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:45:18.146 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:45:25.166 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records in 2024-09-16T15:45:25.166 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records out 2024-09-16T15:45:25.166 INFO:teuthology.orchestra.run.smithi151.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.01633 s, 134 MB/s 2024-09-16T15:45:25.167 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:45:25.168 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:45:25.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:45:25.203 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/bfa16850-416b-4673-9cfd-a73963609af7/subvolume_file.3 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T15:45:33.338 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records in 2024-09-16T15:45:33.339 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records out 2024-09-16T15:45:33.339 INFO:teuthology.orchestra.run.smithi151.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.05886 s, 133 MB/s 2024-09-16T15:45:33.340 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:45:33.340 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:45:33.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:45:33.375 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/bfa16850-416b-4673-9cfd-a73963609af7/subvolume_file.4 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T15:45:41.394 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records in 2024-09-16T15:45:41.395 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records out 2024-09-16T15:45:41.395 INFO:teuthology.orchestra.run.smithi151.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.94286 s, 135 MB/s 2024-09-16T15:45:41.396 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:45:41.396 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:45:41.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:45:41.431 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/bfa16850-416b-4673-9cfd-a73963609af7/subvolume_file.5 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T15:45:48.241 DEBUG:teuthology.orchestra.run.smithi151:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:45:48.247 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:45:49.425 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records in 2024-09-16T15:45:49.425 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records out 2024-09-16T15:45:49.425 INFO:teuthology.orchestra.run.smithi151.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.91866 s, 136 MB/s 2024-09-16T15:45:49.426 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:45:49.426 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:45:49.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:45:49.462 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/bfa16850-416b-4673-9cfd-a73963609af7/subvolume_file.6 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T15:45:57.435 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records in 2024-09-16T15:45:57.436 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records out 2024-09-16T15:45:57.436 INFO:teuthology.orchestra.run.smithi151.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.89794 s, 136 MB/s 2024-09-16T15:45:57.437 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:45:57.437 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:45:57.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:45:57.473 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/bfa16850-416b-4673-9cfd-a73963609af7/subvolume_file.7 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T15:46:05.919 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records in 2024-09-16T15:46:05.919 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records out 2024-09-16T15:46:05.919 INFO:teuthology.orchestra.run.smithi151.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.36947 s, 128 MB/s 2024-09-16T15:46:05.920 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:46:05.921 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:46:05.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:46:05.957 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/bfa16850-416b-4673-9cfd-a73963609af7/subvolume_file.8 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T15:46:14.114 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records in 2024-09-16T15:46:14.115 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records out 2024-09-16T15:46:14.115 INFO:teuthology.orchestra.run.smithi151.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.08144 s, 133 MB/s 2024-09-16T15:46:14.116 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:46:14.116 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:46:14.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:46:14.151 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/bfa16850-416b-4673-9cfd-a73963609af7/subvolume_file.9 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T15:46:18.375 DEBUG:teuthology.orchestra.run.smithi151:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:46:18.380 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:46:21.991 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records in 2024-09-16T15:46:21.991 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records out 2024-09-16T15:46:21.992 INFO:teuthology.orchestra.run.smithi151.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.76348 s, 138 MB/s 2024-09-16T15:46:21.993 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-09-16T15:46:22.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.157+0000 7fb882000640 1 -- 172.21.15.151:0/3908893445 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb87c06e590 msgr2=0x7fb87c06fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:22.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.157+0000 7fb882000640 1 --2- 172.21.15.151:0/3908893445 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb87c06e590 0x7fb87c06fe40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb868005fd0 tx=0x7fb86802a5d0 comp rx=0 tx=0).stop 2024-09-16T15:46:22.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.157+0000 7fb882000640 1 -- 172.21.15.151:0/3908893445 shutdown_connections 2024-09-16T15:46:22.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.157+0000 7fb882000640 1 --2- 172.21.15.151:0/3908893445 >> v2:172.21.15.167:3300/0 conn(0x7fb87c070380 0x7fb87c0cb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:22.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.157+0000 7fb882000640 1 --2- 172.21.15.151:0/3908893445 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb87c06e590 0x7fb87c06fe40 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:22.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.157+0000 7fb882000640 1 -- 172.21.15.151:0/3908893445 >> 172.21.15.151:0/3908893445 conn(0x7fb87c0cfa70 msgr2=0x7fb87c0cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:22.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.157+0000 7fb882000640 1 -- 172.21.15.151:0/3908893445 shutdown_connections 2024-09-16T15:46:22.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.157+0000 7fb882000640 1 -- 172.21.15.151:0/3908893445 wait complete. 2024-09-16T15:46:22.160 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.159+0000 7fb882000640 1 Processor -- start 2024-09-16T15:46:22.160 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.159+0000 7fb882000640 1 -- start start 2024-09-16T15:46:22.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.160+0000 7fb882000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb87c06e590 0x7fb87c151ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:22.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.160+0000 7fb882000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb87c070380 0x7fb87c1f2c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:22.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.160+0000 7fb882000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb87c15bda0 0x7fb87c1f5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:22.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.160+0000 7fb882000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb87c0cba90 con 0x7fb87c15bda0 2024-09-16T15:46:22.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.160+0000 7fb882000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb87c0cb910 con 0x7fb87c06e590 2024-09-16T15:46:22.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.160+0000 7fb882000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fb87c0cbc10 con 0x7fb87c070380 2024-09-16T15:46:22.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.160+0000 7fb881600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb87c15bda0 0x7fb87c1f5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:22.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.160+0000 7fb881600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb87c15bda0 0x7fb87c1f5840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:59208/0 (socket says 172.21.15.151:59208) 2024-09-16T15:46:22.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.160+0000 7fb881600640 1 -- 172.21.15.151:0/3438547739 learned_addr learned my addr 172.21.15.151:0/3438547739 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:46:22.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.160+0000 7fb87be00640 1 --2- 172.21.15.151:0/3438547739 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb87c070380 0x7fb87c1f2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:22.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.160+0000 7fb880c00640 1 --2- 172.21.15.151:0/3438547739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb87c06e590 0x7fb87c151ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:22.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.161+0000 7fb881600640 1 -- 172.21.15.151:0/3438547739 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb87c070380 msgr2=0x7fb87c1f2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:22.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.161+0000 7fb881600640 1 --2- 172.21.15.151:0/3438547739 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb87c070380 0x7fb87c1f2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:22.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.161+0000 7fb881600640 1 -- 172.21.15.151:0/3438547739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb87c06e590 msgr2=0x7fb87c151ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:22.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.161+0000 7fb881600640 1 --2- 172.21.15.151:0/3438547739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb87c06e590 0x7fb87c151ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:22.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.161+0000 7fb881600640 1 -- 172.21.15.151:0/3438547739 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb87c1f6290 con 0x7fb87c15bda0 2024-09-16T15:46:22.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.161+0000 7fb881600640 1 --2- 172.21.15.151:0/3438547739 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb87c15bda0 0x7fb87c1f5840 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fb87400e9e0 tx=0x7fb87400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:22.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.161+0000 7fb878c00640 1 -- 172.21.15.151:0/3438547739 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb87400cdb0 con 0x7fb87c15bda0 2024-09-16T15:46:22.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.161+0000 7fb878c00640 1 -- 172.21.15.151:0/3438547739 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb874004020 con 0x7fb87c15bda0 2024-09-16T15:46:22.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.161+0000 7fb882000640 1 -- 172.21.15.151:0/3438547739 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb87c1f6570 con 0x7fb87c15bda0 2024-09-16T15:46:22.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.161+0000 7fb878c00640 1 -- 172.21.15.151:0/3438547739 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb874004450 con 0x7fb87c15bda0 2024-09-16T15:46:22.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.161+0000 7fb882000640 1 -- 172.21.15.151:0/3438547739 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb87c1f2720 con 0x7fb87c15bda0 2024-09-16T15:46:22.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.163+0000 7fb882000640 1 -- 172.21.15.151:0/3438547739 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb87c1f2250 con 0x7fb87c15bda0 2024-09-16T15:46:22.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.164+0000 7fb878c00640 1 -- 172.21.15.151:0/3438547739 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 20) ==== 102103+0+0 (secure 0 0 0) 0x7fb8740035f0 con 0x7fb87c15bda0 2024-09-16T15:46:22.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.165+0000 7fb878c00640 1 --2- 172.21.15.151:0/3438547739 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb85c079150 0x7fb85c07b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:22.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.165+0000 7fb878c00640 1 -- 172.21.15.151:0/3438547739 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5795+0+0 (secure 0 0 0) 0x7fb874099a20 con 0x7fb87c15bda0 2024-09-16T15:46:22.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.165+0000 7fb880c00640 1 --2- 172.21.15.151:0/3438547739 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb85c079150 0x7fb85c07b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:22.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.166+0000 7fb880c00640 1 --2- 172.21.15.151:0/3438547739 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb85c079150 0x7fb85c07b610 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb86c0012e0 tx=0x7fb86c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:22.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.169+0000 7fb878c00640 1 -- 172.21.15.151:0/3438547739 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb8740634f0 con 0x7fb87c15bda0 2024-09-16T15:46:22.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.385+0000 7fb882000640 1 -- 172.21.15.151:0/3438547739 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7fb87c1f2920 con 0x7fb85c079150 2024-09-16T15:46:22.392 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:46:22.391+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:46:22.392 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:46:22.391+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:46:22.392 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:46:22.391+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:46:22.392 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:46:22.391+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:46:22.392 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:46:22.391+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:46:22.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.412+0000 7fb878c00640 1 -- 172.21.15.151:0/3438547739 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb87c1f2920 con 0x7fb85c079150 2024-09-16T15:46:22.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.414+0000 7fb882000640 1 -- 172.21.15.151:0/3438547739 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb85c079150 msgr2=0x7fb85c07b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:22.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.414+0000 7fb882000640 1 --2- 172.21.15.151:0/3438547739 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb85c079150 0x7fb85c07b610 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb86c0012e0 tx=0x7fb86c009040 comp rx=0 tx=0).stop 2024-09-16T15:46:22.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.414+0000 7fb882000640 1 -- 172.21.15.151:0/3438547739 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb87c15bda0 msgr2=0x7fb87c1f5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:22.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.414+0000 7fb882000640 1 --2- 172.21.15.151:0/3438547739 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb87c15bda0 0x7fb87c1f5840 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fb87400e9e0 tx=0x7fb87400eea0 comp rx=0 tx=0).stop 2024-09-16T15:46:22.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.415+0000 7fb882000640 1 -- 172.21.15.151:0/3438547739 shutdown_connections 2024-09-16T15:46:22.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.415+0000 7fb882000640 1 --2- 172.21.15.151:0/3438547739 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb85c079150 0x7fb85c07b610 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:22.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.415+0000 7fb882000640 1 --2- 172.21.15.151:0/3438547739 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb87c15bda0 0x7fb87c1f5840 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:22.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.415+0000 7fb882000640 1 --2- 172.21.15.151:0/3438547739 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb87c070380 0x7fb87c1f2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:22.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.415+0000 7fb882000640 1 --2- 172.21.15.151:0/3438547739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb87c06e590 0x7fb87c151ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:22.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.415+0000 7fb882000640 1 -- 172.21.15.151:0/3438547739 >> 172.21.15.151:0/3438547739 conn(0x7fb87c0cfa70 msgr2=0x7fb87c159f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:22.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.415+0000 7fb882000640 1 -- 172.21.15.151:0/3438547739 shutdown_connections 2024-09-16T15:46:22.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.415+0000 7fb882000640 1 -- 172.21.15.151:0/3438547739 wait complete. 2024-09-16T15:46:22.432 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-16T15:46:22.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.591+0000 7fc786a00640 1 -- 172.21.15.151:0/2533482206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc78006e880 msgr2=0x7fc78015ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:22.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.591+0000 7fc786a00640 1 --2- 172.21.15.151:0/2533482206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc78006e880 0x7fc78015ec50 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fc76c005fd0 tx=0x7fc76c02a5d0 comp rx=0 tx=0).stop 2024-09-16T15:46:22.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.591+0000 7fc786a00640 1 -- 172.21.15.151:0/2533482206 shutdown_connections 2024-09-16T15:46:22.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.591+0000 7fc786a00640 1 --2- 172.21.15.151:0/2533482206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc78006e880 0x7fc78015ec50 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:22.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.591+0000 7fc786a00640 1 --2- 172.21.15.151:0/2533482206 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc78015e870 0x7fc7800ae5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:22.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.591+0000 7fc786a00640 1 -- 172.21.15.151:0/2533482206 >> 172.21.15.151:0/2533482206 conn(0x7fc7800cf9f0 msgr2=0x7fc7800cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:22.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.591+0000 7fc786a00640 1 -- 172.21.15.151:0/2533482206 shutdown_connections 2024-09-16T15:46:22.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.591+0000 7fc786a00640 1 -- 172.21.15.151:0/2533482206 wait complete. 2024-09-16T15:46:22.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.592+0000 7fc786a00640 1 Processor -- start 2024-09-16T15:46:22.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.592+0000 7fc786a00640 1 -- start start 2024-09-16T15:46:22.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.593+0000 7fc786a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc78006e880 0x7fc7801f5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:22.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.593+0000 7fc786a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc78015e870 0x7fc78015d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:22.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.593+0000 7fc786a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc78015f190 0x7fc78015d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:22.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.593+0000 7fc786a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc780074230 con 0x7fc78015e870 2024-09-16T15:46:22.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.593+0000 7fc786a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc7800740b0 con 0x7fc78015f190 2024-09-16T15:46:22.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.593+0000 7fc786a00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fc7800743b0 con 0x7fc78006e880 2024-09-16T15:46:22.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.594+0000 7fc784c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc78015e870 0x7fc78015d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:22.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.594+0000 7fc784c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc78015e870 0x7fc78015d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:59226/0 (socket says 172.21.15.151:59226) 2024-09-16T15:46:22.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.594+0000 7fc784c00640 1 -- 172.21.15.151:0/3259004551 learned_addr learned my addr 172.21.15.151:0/3259004551 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:46:22.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.594+0000 7fc786000640 1 --2- 172.21.15.151:0/3259004551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc78015f190 0x7fc78015d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:22.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.594+0000 7fc785600640 1 --2- 172.21.15.151:0/3259004551 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc78006e880 0x7fc7801f5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:22.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.594+0000 7fc784c00640 1 -- 172.21.15.151:0/3259004551 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc78006e880 msgr2=0x7fc7801f5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:22.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.594+0000 7fc784c00640 1 --2- 172.21.15.151:0/3259004551 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc78006e880 0x7fc7801f5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:22.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.594+0000 7fc784c00640 1 -- 172.21.15.151:0/3259004551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc78015f190 msgr2=0x7fc78015d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:22.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.594+0000 7fc784c00640 1 --2- 172.21.15.151:0/3259004551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc78015f190 0x7fc78015d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:22.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.594+0000 7fc784c00640 1 -- 172.21.15.151:0/3259004551 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7800c4250 con 0x7fc78015e870 2024-09-16T15:46:22.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.594+0000 7fc784c00640 1 --2- 172.21.15.151:0/3259004551 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc78015e870 0x7fc78015d060 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fc76c006010 tx=0x7fc76c0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:22.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.595+0000 7fc77d600640 1 -- 172.21.15.151:0/3259004551 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76c03f070 con 0x7fc78015e870 2024-09-16T15:46:22.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.595+0000 7fc77d600640 1 -- 172.21.15.151:0/3259004551 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc76c005600 con 0x7fc78015e870 2024-09-16T15:46:22.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.595+0000 7fc77d600640 1 -- 172.21.15.151:0/3259004551 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76c035030 con 0x7fc78015e870 2024-09-16T15:46:22.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.595+0000 7fc786a00640 1 -- 172.21.15.151:0/3259004551 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7800c4530 con 0x7fc78015e870 2024-09-16T15:46:22.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.595+0000 7fc786a00640 1 -- 172.21.15.151:0/3259004551 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7800b3c60 con 0x7fc78015e870 2024-09-16T15:46:22.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.596+0000 7fc786a00640 1 -- 172.21.15.151:0/3259004551 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc78015ec50 con 0x7fc78015e870 2024-09-16T15:46:22.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.597+0000 7fc77d600640 1 -- 172.21.15.151:0/3259004551 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 20) ==== 102103+0+0 (secure 0 0 0) 0x7fc76c042070 con 0x7fc78015e870 2024-09-16T15:46:22.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.598+0000 7fc77d600640 1 --2- 172.21.15.151:0/3259004551 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc760079150 0x7fc76007b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:22.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.598+0000 7fc77d600640 1 -- 172.21.15.151:0/3259004551 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5795+0+0 (secure 0 0 0) 0x7fc76c0bc700 con 0x7fc78015e870 2024-09-16T15:46:22.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.598+0000 7fc785600640 1 --2- 172.21.15.151:0/3259004551 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc760079150 0x7fc76007b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:22.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.599+0000 7fc785600640 1 --2- 172.21.15.151:0/3259004551 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc760079150 0x7fc76007b610 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc7801590d0 tx=0x7fc774009680 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:22.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.602+0000 7fc77d600640 1 -- 172.21.15.151:0/3259004551 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc76c0837a0 con 0x7fc78015e870 2024-09-16T15:46:22.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.813+0000 7fc786a00640 1 -- 172.21.15.151:0/3259004551 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fc7800b3e30 con 0x7fc760079150 2024-09-16T15:46:22.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.820+0000 7fc77d600640 1 -- 172.21.15.151:0/3259004551 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7fc7800b3e30 con 0x7fc760079150 2024-09-16T15:46:22.821 INFO:teuthology.orchestra.run.smithi151.stdout:/volumes/_nogroup/sv1/bfa16850-416b-4673-9cfd-a73963609af7 2024-09-16T15:46:22.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.823+0000 7fc786a00640 1 -- 172.21.15.151:0/3259004551 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc760079150 msgr2=0x7fc76007b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:22.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.823+0000 7fc786a00640 1 --2- 172.21.15.151:0/3259004551 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc760079150 0x7fc76007b610 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc7801590d0 tx=0x7fc774009680 comp rx=0 tx=0).stop 2024-09-16T15:46:22.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.823+0000 7fc786a00640 1 -- 172.21.15.151:0/3259004551 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc78015e870 msgr2=0x7fc78015d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:22.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.823+0000 7fc786a00640 1 --2- 172.21.15.151:0/3259004551 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc78015e870 0x7fc78015d060 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fc76c006010 tx=0x7fc76c0038a0 comp rx=0 tx=0).stop 2024-09-16T15:46:22.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.823+0000 7fc786a00640 1 -- 172.21.15.151:0/3259004551 shutdown_connections 2024-09-16T15:46:22.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.823+0000 7fc786a00640 1 --2- 172.21.15.151:0/3259004551 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc760079150 0x7fc76007b610 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:22.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.823+0000 7fc786a00640 1 --2- 172.21.15.151:0/3259004551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc78015f190 0x7fc78015d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:22.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.823+0000 7fc786a00640 1 --2- 172.21.15.151:0/3259004551 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc78015e870 0x7fc78015d060 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:22.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.823+0000 7fc786a00640 1 --2- 172.21.15.151:0/3259004551 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc78006e880 0x7fc7801f5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:22.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.823+0000 7fc786a00640 1 -- 172.21.15.151:0/3259004551 >> 172.21.15.151:0/3259004551 conn(0x7fc7800cf9f0 msgr2=0x7fc7800cff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:22.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.823+0000 7fc786a00640 1 -- 172.21.15.151:0/3259004551 shutdown_connections 2024-09-16T15:46:22.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:22.823+0000 7fc786a00640 1 -- 172.21.15.151:0/3259004551 wait complete. 2024-09-16T15:46:22.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:46:22.839 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/bfa16850-416b-4673-9cfd-a73963609af7 2024-09-16T15:46:22.840 DEBUG:teuthology.orchestra.run.smithi151:> ) 2024-09-16T15:46:22.875 INFO:teuthology.orchestra.run.smithi151.stdout:# file: volumes/_nogroup/sv1/bfa16850-416b-4673-9cfd-a73963609af7 2024-09-16T15:46:22.875 INFO:teuthology.orchestra.run.smithi151.stdout:ceph.dir.rbytes="10405347328" 2024-09-16T15:46:22.876 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:46:22.876 INFO:tasks.cephfs.test_volumes:r_size = 10405347328 exp_size = 10737418240 2024-09-16T15:46:24.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:46:24.878 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/bfa16850-416b-4673-9cfd-a73963609af7 2024-09-16T15:46:24.878 DEBUG:teuthology.orchestra.run.smithi151:> ) 2024-09-16T15:46:24.919 INFO:teuthology.orchestra.run.smithi151.stdout:# file: volumes/_nogroup/sv1/bfa16850-416b-4673-9cfd-a73963609af7 2024-09-16T15:46:24.919 INFO:teuthology.orchestra.run.smithi151.stdout:ceph.dir.rbytes="10405347328" 2024-09-16T15:46:24.919 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:46:24.920 INFO:tasks.cephfs.test_volumes:r_size = 10405347328 exp_size = 10737418240 2024-09-16T15:46:26.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:46:26.922 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/bfa16850-416b-4673-9cfd-a73963609af7 2024-09-16T15:46:26.922 DEBUG:teuthology.orchestra.run.smithi151:> ) 2024-09-16T15:46:26.962 INFO:teuthology.orchestra.run.smithi151.stdout:# file: volumes/_nogroup/sv1/bfa16850-416b-4673-9cfd-a73963609af7 2024-09-16T15:46:26.962 INFO:teuthology.orchestra.run.smithi151.stdout:ceph.dir.rbytes="10737418240" 2024-09-16T15:46:26.963 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:46:26.964 INFO:tasks.cephfs.test_volumes:r_size = 10737418240 exp_size = 10737418240 2024-09-16T15:46:26.965 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 ss1clone1 2024-09-16T15:46:27.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.170+0000 7fd609a00640 1 -- 172.21.15.151:0/4004760633 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd604145750 msgr2=0x7fd604073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:27.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.170+0000 7fd609a00640 1 --2- 172.21.15.151:0/4004760633 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd604145750 0x7fd604073440 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4005fd0 tx=0x7fd5f402a540 comp rx=0 tx=0).stop 2024-09-16T15:46:27.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.170+0000 7fd609a00640 1 -- 172.21.15.151:0/4004760633 shutdown_connections 2024-09-16T15:46:27.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.170+0000 7fd609a00640 1 --2- 172.21.15.151:0/4004760633 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6040cccd0 0x7fd6040cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:27.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.170+0000 7fd609a00640 1 --2- 172.21.15.151:0/4004760633 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd604145750 0x7fd604073440 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:27.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.170+0000 7fd609a00640 1 -- 172.21.15.151:0/4004760633 >> 172.21.15.151:0/4004760633 conn(0x7fd6040b1840 msgr2=0x7fd6040b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:27.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.171+0000 7fd609a00640 1 -- 172.21.15.151:0/4004760633 shutdown_connections 2024-09-16T15:46:27.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.171+0000 7fd609a00640 1 -- 172.21.15.151:0/4004760633 wait complete. 2024-09-16T15:46:27.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.172+0000 7fd609a00640 1 Processor -- start 2024-09-16T15:46:27.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.172+0000 7fd609a00640 1 -- start start 2024-09-16T15:46:27.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.173+0000 7fd609a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd60406efa0 0x7fd60413bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:27.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.173+0000 7fd609a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6040cccd0 0x7fd6040ad1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:27.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.173+0000 7fd609a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd604145750 0x7fd6040ad6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:27.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.173+0000 7fd609a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd6040ce480 con 0x7fd604145750 2024-09-16T15:46:27.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.173+0000 7fd609a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd6040ce300 con 0x7fd6040cccd0 2024-09-16T15:46:27.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.173+0000 7fd609a00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fd6040ce600 con 0x7fd60406efa0 2024-09-16T15:46:27.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.173+0000 7fd603400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd604145750 0x7fd6040ad6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:27.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.173+0000 7fd603400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd604145750 0x7fd6040ad6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:58052/0 (socket says 172.21.15.151:58052) 2024-09-16T15:46:27.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.173+0000 7fd603400640 1 -- 172.21.15.151:0/311673438 learned_addr learned my addr 172.21.15.151:0/311673438 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:46:27.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.174+0000 7fd602000640 1 --2- 172.21.15.151:0/311673438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6040cccd0 0x7fd6040ad1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:27.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.174+0000 7fd602a00640 1 --2- 172.21.15.151:0/311673438 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd60406efa0 0x7fd60413bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:27.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.174+0000 7fd603400640 1 -- 172.21.15.151:0/311673438 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd60406efa0 msgr2=0x7fd60413bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:27.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.174+0000 7fd603400640 1 --2- 172.21.15.151:0/311673438 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd60406efa0 0x7fd60413bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:27.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.174+0000 7fd603400640 1 -- 172.21.15.151:0/311673438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6040cccd0 msgr2=0x7fd6040ad1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:27.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.174+0000 7fd603400640 1 --2- 172.21.15.151:0/311673438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6040cccd0 0x7fd6040ad1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:27.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.174+0000 7fd603400640 1 -- 172.21.15.151:0/311673438 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6041f6200 con 0x7fd604145750 2024-09-16T15:46:27.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.174+0000 7fd602a00640 1 --2- 172.21.15.151:0/311673438 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd60406efa0 0x7fd60413bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:46:27.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.174+0000 7fd603400640 1 --2- 172.21.15.151:0/311673438 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd604145750 0x7fd6040ad6e0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc00ead0 tx=0x7fd5fc00ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:27.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.174+0000 7fd5faa00640 1 -- 172.21.15.151:0/311673438 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc00cde0 con 0x7fd604145750 2024-09-16T15:46:27.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.174+0000 7fd5faa00640 1 -- 172.21.15.151:0/311673438 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd5fc004c10 con 0x7fd604145750 2024-09-16T15:46:27.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.174+0000 7fd609a00640 1 -- 172.21.15.151:0/311673438 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6041f3560 con 0x7fd604145750 2024-09-16T15:46:27.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.174+0000 7fd5faa00640 1 -- 172.21.15.151:0/311673438 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc005020 con 0x7fd604145750 2024-09-16T15:46:27.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.175+0000 7fd609a00640 1 -- 172.21.15.151:0/311673438 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd60413cd40 con 0x7fd604145750 2024-09-16T15:46:27.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.175+0000 7fd609a00640 1 -- 172.21.15.151:0/311673438 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6040cdbb0 con 0x7fd604145750 2024-09-16T15:46:27.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.177+0000 7fd5faa00640 1 -- 172.21.15.151:0/311673438 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 21) ==== 102154+0+0 (secure 0 0 0) 0x7fd5fc0051c0 con 0x7fd604145750 2024-09-16T15:46:27.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.177+0000 7fd5faa00640 1 --2- 172.21.15.151:0/311673438 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd5e40791e0 0x7fd5e407b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:27.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.178+0000 7fd5faa00640 1 -- 172.21.15.151:0/311673438 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5795+0+0 (secure 0 0 0) 0x7fd5fc099950 con 0x7fd604145750 2024-09-16T15:46:27.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.178+0000 7fd602a00640 1 --2- 172.21.15.151:0/311673438 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd5e40791e0 0x7fd5e407b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:27.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.179+0000 7fd602a00640 1 --2- 172.21.15.151:0/311673438 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd5e40791e0 0x7fd5e407b6a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4033000 tx=0x7fd5f4004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:27.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.182+0000 7fd5faa00640 1 -- 172.21.15.151:0/311673438 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd6041f6200 con 0x7fd604145750 2024-09-16T15:46:27.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.400+0000 7fd609a00640 1 -- 172.21.15.151:0/311673438 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7fd604143760 con 0x7fd5e40791e0 2024-09-16T15:46:27.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.472+0000 7fd5faa00640 1 -- 172.21.15.151:0/311673438 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd604143760 con 0x7fd5e40791e0 2024-09-16T15:46:27.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.474+0000 7fd609a00640 1 -- 172.21.15.151:0/311673438 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd5e40791e0 msgr2=0x7fd5e407b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:27.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.475+0000 7fd609a00640 1 --2- 172.21.15.151:0/311673438 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd5e40791e0 0x7fd5e407b6a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4033000 tx=0x7fd5f4004730 comp rx=0 tx=0).stop 2024-09-16T15:46:27.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.475+0000 7fd609a00640 1 -- 172.21.15.151:0/311673438 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd604145750 msgr2=0x7fd6040ad6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:27.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.475+0000 7fd609a00640 1 --2- 172.21.15.151:0/311673438 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd604145750 0x7fd6040ad6e0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc00ead0 tx=0x7fd5fc00ef90 comp rx=0 tx=0).stop 2024-09-16T15:46:27.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.475+0000 7fd609a00640 1 -- 172.21.15.151:0/311673438 shutdown_connections 2024-09-16T15:46:27.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.475+0000 7fd609a00640 1 --2- 172.21.15.151:0/311673438 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd5e40791e0 0x7fd5e407b6a0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:27.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.475+0000 7fd609a00640 1 --2- 172.21.15.151:0/311673438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6040cccd0 0x7fd6040ad1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:27.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.475+0000 7fd609a00640 1 --2- 172.21.15.151:0/311673438 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd604145750 0x7fd6040ad6e0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:27.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.475+0000 7fd609a00640 1 --2- 172.21.15.151:0/311673438 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd60406efa0 0x7fd60413bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:27.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.475+0000 7fd609a00640 1 -- 172.21.15.151:0/311673438 >> 172.21.15.151:0/311673438 conn(0x7fd6040b1840 msgr2=0x7fd6040cf2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:27.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.475+0000 7fd609a00640 1 -- 172.21.15.151:0/311673438 shutdown_connections 2024-09-16T15:46:27.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.475+0000 7fd609a00640 1 -- 172.21.15.151:0/311673438 wait complete. 2024-09-16T15:46:27.478 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:46:27.477+0000 7f48f6c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:46:27.478 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:46:27.477+0000 7f48f6c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:46:27.478 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:46:27.477+0000 7f48f6c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:46:27.478 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:46:27.477+0000 7f48f6c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:46:27.479 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:46:27.477+0000 7f48f6c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:46:27.493 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-16T15:46:27.495 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:46:27.494+0000 7f48f7600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:46:27.495 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:46:27.494+0000 7f48f7600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:46:27.495 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:46:27.494+0000 7f48f7600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:46:27.495 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:46:27.494+0000 7f48f7600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:46:27.495 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:46:27.494+0000 7f48f7600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:46:27.653 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.652+0000 7f2484e00640 1 -- 172.21.15.151:0/2159398747 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f248015cff0 msgr2=0x7f248006e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:27.653 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.652+0000 7f2484e00640 1 --2- 172.21.15.151:0/2159398747 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f248015cff0 0x7f248006e6f0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f246c005fd0 tx=0x7f246c02a5d0 comp rx=0 tx=0).stop 2024-09-16T15:46:27.654 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.652+0000 7f2484e00640 1 -- 172.21.15.151:0/2159398747 shutdown_connections 2024-09-16T15:46:27.654 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.652+0000 7f2484e00640 1 --2- 172.21.15.151:0/2159398747 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f248006f220 0x7f248006f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:27.654 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.652+0000 7f2484e00640 1 --2- 172.21.15.151:0/2159398747 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f248015cff0 0x7f248006e6f0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:27.654 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.652+0000 7f2484e00640 1 -- 172.21.15.151:0/2159398747 >> 172.21.15.151:0/2159398747 conn(0x7f24800b18e0 msgr2=0x7f24800b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:27.654 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.652+0000 7f2484e00640 1 -- 172.21.15.151:0/2159398747 shutdown_connections 2024-09-16T15:46:27.654 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.652+0000 7f2484e00640 1 -- 172.21.15.151:0/2159398747 wait complete. 2024-09-16T15:46:27.654 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.653+0000 7f2484e00640 1 Processor -- start 2024-09-16T15:46:27.654 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.654+0000 7f2484e00640 1 -- start start 2024-09-16T15:46:27.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.654+0000 7f2484e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f248006f220 0x7f248009c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:27.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.654+0000 7f2484e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2480159aa0 0x7f24800b4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:27.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.655+0000 7f2484e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f248015cff0 0x7f248009aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:27.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.655+0000 7f2484e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f24800cc520 con 0x7f248006f220 2024-09-16T15:46:27.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.655+0000 7f2484e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f24800cc3a0 con 0x7f2480159aa0 2024-09-16T15:46:27.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.655+0000 7f2484e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f24800cc6a0 con 0x7f248015cff0 2024-09-16T15:46:27.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.655+0000 7f247de00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f248006f220 0x7f248009c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:27.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.655+0000 7f247de00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f248006f220 0x7f248009c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:58058/0 (socket says 172.21.15.151:58058) 2024-09-16T15:46:27.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.655+0000 7f247de00640 1 -- 172.21.15.151:0/1423411212 learned_addr learned my addr 172.21.15.151:0/1423411212 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:46:27.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.655+0000 7f247e800640 1 --2- 172.21.15.151:0/1423411212 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f248015cff0 0x7f248009aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:27.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.655+0000 7f247d400640 1 --2- 172.21.15.151:0/1423411212 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2480159aa0 0x7f24800b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:27.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.655+0000 7f247de00640 1 -- 172.21.15.151:0/1423411212 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f248015cff0 msgr2=0x7f248009aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:27.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.655+0000 7f247de00640 1 --2- 172.21.15.151:0/1423411212 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f248015cff0 0x7f248009aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:27.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.655+0000 7f247de00640 1 -- 172.21.15.151:0/1423411212 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2480159aa0 msgr2=0x7f24800b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:27.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.655+0000 7f247de00640 1 --2- 172.21.15.151:0/1423411212 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2480159aa0 0x7f24800b4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:27.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.655+0000 7f247de00640 1 -- 172.21.15.151:0/1423411212 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f248009cce0 con 0x7f248006f220 2024-09-16T15:46:27.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.655+0000 7f247de00640 1 --2- 172.21.15.151:0/1423411212 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f248006f220 0x7f248009c770 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f247000c450 tx=0x7f247000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:27.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.656+0000 7f2476000640 1 -- 172.21.15.151:0/1423411212 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24700052d0 con 0x7f248006f220 2024-09-16T15:46:27.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.656+0000 7f2476000640 1 -- 172.21.15.151:0/1423411212 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2470005470 con 0x7f248006f220 2024-09-16T15:46:27.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.656+0000 7f2476000640 1 -- 172.21.15.151:0/1423411212 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2470012870 con 0x7f248006f220 2024-09-16T15:46:27.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.656+0000 7f2484e00640 1 -- 172.21.15.151:0/1423411212 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24801f8ad0 con 0x7f248006f220 2024-09-16T15:46:27.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.656+0000 7f2484e00640 1 -- 172.21.15.151:0/1423411212 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24801f9090 con 0x7f248006f220 2024-09-16T15:46:27.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.657+0000 7f2484e00640 1 -- 172.21.15.151:0/1423411212 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2480069360 con 0x7f248006f220 2024-09-16T15:46:27.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.658+0000 7f2476000640 1 -- 172.21.15.151:0/1423411212 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 21) ==== 102154+0+0 (secure 0 0 0) 0x7f24700035f0 con 0x7f248006f220 2024-09-16T15:46:27.660 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.659+0000 7f2476000640 1 --2- 172.21.15.151:0/1423411212 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f24600791e0 0x7f246007b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:27.660 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.659+0000 7f2476000640 1 -- 172.21.15.151:0/1423411212 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5795+0+0 (secure 0 0 0) 0x7f2470098a70 con 0x7f248006f220 2024-09-16T15:46:27.660 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.660+0000 7f247d400640 1 --2- 172.21.15.151:0/1423411212 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f24600791e0 0x7f246007b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:27.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.660+0000 7f247d400640 1 --2- 172.21.15.151:0/1423411212 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f24600791e0 0x7f246007b6a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f246c033000 tx=0x7f246c004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:27.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.663+0000 7f2476000640 1 -- 172.21.15.151:0/1423411212 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f248009cce0 con 0x7f248006f220 2024-09-16T15:46:27.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.918+0000 7f2484e00640 1 -- 172.21.15.151:0/1423411212 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f2480070980 con 0x7f248006f220 2024-09-16T15:46:27.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.918+0000 7f2476000640 1 -- 172.21.15.151:0/1423411212 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1883 (secure 0 0 0) 0x7f2470062690 con 0x7f248006f220 2024-09-16T15:46:27.920 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:46:27.920 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-16T15:46:27.920 INFO:teuthology.orchestra.run.smithi151.stdout: "fsid": "1757be18-e41a-4446-bb4f-ee4ac81c39df", 2024-09-16T15:46:27.920 INFO:teuthology.orchestra.run.smithi151.stdout: "health": { 2024-09-16T15:46:27.920 INFO:teuthology.orchestra.run.smithi151.stdout: "status": "HEALTH_OK", 2024-09-16T15:46:27.920 INFO:teuthology.orchestra.run.smithi151.stdout: "checks": {}, 2024-09-16T15:46:27.920 INFO:teuthology.orchestra.run.smithi151.stdout: "mutes": [] 2024-09-16T15:46:27.920 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:46:27.920 INFO:teuthology.orchestra.run.smithi151.stdout: "election_epoch": 4, 2024-09-16T15:46:27.920 INFO:teuthology.orchestra.run.smithi151.stdout: "quorum": [ 2024-09-16T15:46:27.920 INFO:teuthology.orchestra.run.smithi151.stdout: 0, 2024-09-16T15:46:27.921 INFO:teuthology.orchestra.run.smithi151.stdout: 1, 2024-09-16T15:46:27.921 INFO:teuthology.orchestra.run.smithi151.stdout: 2 2024-09-16T15:46:27.921 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:46:27.921 INFO:teuthology.orchestra.run.smithi151.stdout: "quorum_names": [ 2024-09-16T15:46:27.921 INFO:teuthology.orchestra.run.smithi151.stdout: "a", 2024-09-16T15:46:27.921 INFO:teuthology.orchestra.run.smithi151.stdout: "b", 2024-09-16T15:46:27.921 INFO:teuthology.orchestra.run.smithi151.stdout: "c" 2024-09-16T15:46:27.921 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:46:27.921 INFO:teuthology.orchestra.run.smithi151.stdout: "quorum_age": 451, 2024-09-16T15:46:27.921 INFO:teuthology.orchestra.run.smithi151.stdout: "monmap": { 2024-09-16T15:46:27.921 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 1, 2024-09-16T15:46:27.922 INFO:teuthology.orchestra.run.smithi151.stdout: "min_mon_release_name": "squid", 2024-09-16T15:46:27.922 INFO:teuthology.orchestra.run.smithi151.stdout: "num_mons": 3 2024-09-16T15:46:27.922 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:46:27.922 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap": { 2024-09-16T15:46:27.922 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 64, 2024-09-16T15:46:27.922 INFO:teuthology.orchestra.run.smithi151.stdout: "num_osds": 8, 2024-09-16T15:46:27.922 INFO:teuthology.orchestra.run.smithi151.stdout: "num_up_osds": 8, 2024-09-16T15:46:27.922 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_up_since": 1726501150, 2024-09-16T15:46:27.922 INFO:teuthology.orchestra.run.smithi151.stdout: "num_in_osds": 8, 2024-09-16T15:46:27.922 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_in_since": 1726501140, 2024-09-16T15:46:27.922 INFO:teuthology.orchestra.run.smithi151.stdout: "num_remapped_pgs": 0 2024-09-16T15:46:27.922 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:46:27.922 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap": { 2024-09-16T15:46:27.922 INFO:teuthology.orchestra.run.smithi151.stdout: "pgs_by_state": [ 2024-09-16T15:46:27.923 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:46:27.923 INFO:teuthology.orchestra.run.smithi151.stdout: "state_name": "active+clean", 2024-09-16T15:46:27.923 INFO:teuthology.orchestra.run.smithi151.stdout: "count": 97 2024-09-16T15:46:27.923 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:46:27.923 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:46:27.923 INFO:teuthology.orchestra.run.smithi151.stdout: "num_pgs": 97, 2024-09-16T15:46:27.923 INFO:teuthology.orchestra.run.smithi151.stdout: "num_pools": 3, 2024-09-16T15:46:27.923 INFO:teuthology.orchestra.run.smithi151.stdout: "num_objects": 2510, 2024-09-16T15:46:27.923 INFO:teuthology.orchestra.run.smithi151.stdout: "data_bytes": 10423980957, 2024-09-16T15:46:27.923 INFO:teuthology.orchestra.run.smithi151.stdout: "bytes_used": 20816838656, 2024-09-16T15:46:27.923 INFO:teuthology.orchestra.run.smithi151.stdout: "bytes_avail": 752277274624, 2024-09-16T15:46:27.923 INFO:teuthology.orchestra.run.smithi151.stdout: "bytes_total": 773094113280, 2024-09-16T15:46:27.923 INFO:teuthology.orchestra.run.smithi151.stdout: "write_bytes_sec": 152710416, 2024-09-16T15:46:27.923 INFO:teuthology.orchestra.run.smithi151.stdout: "read_op_per_sec": 0, 2024-09-16T15:46:27.924 INFO:teuthology.orchestra.run.smithi151.stdout: "write_op_per_sec": 38 2024-09-16T15:46:27.924 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:46:27.924 INFO:teuthology.orchestra.run.smithi151.stdout: "fsmap": { 2024-09-16T15:46:27.924 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 37, 2024-09-16T15:46:27.924 INFO:teuthology.orchestra.run.smithi151.stdout: "btime": "2024-09-16T15:44:58:754420+0000", 2024-09-16T15:46:27.924 INFO:teuthology.orchestra.run.smithi151.stdout: "id": 4, 2024-09-16T15:46:27.924 INFO:teuthology.orchestra.run.smithi151.stdout: "up": 1, 2024-09-16T15:46:27.924 INFO:teuthology.orchestra.run.smithi151.stdout: "in": 1, 2024-09-16T15:46:27.924 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 1, 2024-09-16T15:46:27.924 INFO:teuthology.orchestra.run.smithi151.stdout: "by_rank": [ 2024-09-16T15:46:27.924 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:46:27.924 INFO:teuthology.orchestra.run.smithi151.stdout: "filesystem_id": 4, 2024-09-16T15:46:27.924 INFO:teuthology.orchestra.run.smithi151.stdout: "rank": 0, 2024-09-16T15:46:27.924 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "c", 2024-09-16T15:46:27.924 INFO:teuthology.orchestra.run.smithi151.stdout: "status": "up:active", 2024-09-16T15:46:27.925 INFO:teuthology.orchestra.run.smithi151.stdout: "gid": 5961 2024-09-16T15:46:27.925 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:46:27.925 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:46:27.925 INFO:teuthology.orchestra.run.smithi151.stdout: "up:standby": 3 2024-09-16T15:46:27.925 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:46:27.925 INFO:teuthology.orchestra.run.smithi151.stdout: "mgrmap": { 2024-09-16T15:46:27.925 INFO:teuthology.orchestra.run.smithi151.stdout: "available": true, 2024-09-16T15:46:27.925 INFO:teuthology.orchestra.run.smithi151.stdout: "num_standbys": 1, 2024-09-16T15:46:27.925 INFO:teuthology.orchestra.run.smithi151.stdout: "modules": [ 2024-09-16T15:46:27.925 INFO:teuthology.orchestra.run.smithi151.stdout: "iostat", 2024-09-16T15:46:27.925 INFO:teuthology.orchestra.run.smithi151.stdout: "nfs", 2024-09-16T15:46:27.925 INFO:teuthology.orchestra.run.smithi151.stdout: "restful" 2024-09-16T15:46:27.925 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:46:27.925 INFO:teuthology.orchestra.run.smithi151.stdout: "services": {} 2024-09-16T15:46:27.926 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:46:27.926 INFO:teuthology.orchestra.run.smithi151.stdout: "servicemap": { 2024-09-16T15:46:27.926 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 10, 2024-09-16T15:46:27.926 INFO:teuthology.orchestra.run.smithi151.stdout: "modified": "2024-09-16T15:45:53.688171+0000", 2024-09-16T15:46:27.926 INFO:teuthology.orchestra.run.smithi151.stdout: "services": {} 2024-09-16T15:46:27.926 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:46:27.926 INFO:teuthology.orchestra.run.smithi151.stdout: "progress_events": {} 2024-09-16T15:46:27.926 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-16T15:46:27.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.921+0000 7f2484e00640 1 -- 172.21.15.151:0/1423411212 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f24600791e0 msgr2=0x7f246007b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:27.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.921+0000 7f2484e00640 1 --2- 172.21.15.151:0/1423411212 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f24600791e0 0x7f246007b6a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f246c033000 tx=0x7f246c004730 comp rx=0 tx=0).stop 2024-09-16T15:46:27.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.921+0000 7f2484e00640 1 -- 172.21.15.151:0/1423411212 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f248006f220 msgr2=0x7f248009c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:27.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.921+0000 7f2484e00640 1 --2- 172.21.15.151:0/1423411212 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f248006f220 0x7f248009c770 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f247000c450 tx=0x7f247000c910 comp rx=0 tx=0).stop 2024-09-16T15:46:27.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.921+0000 7f2484e00640 1 -- 172.21.15.151:0/1423411212 shutdown_connections 2024-09-16T15:46:27.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.921+0000 7f2484e00640 1 --2- 172.21.15.151:0/1423411212 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f24600791e0 0x7f246007b6a0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:27.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.921+0000 7f2484e00640 1 --2- 172.21.15.151:0/1423411212 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2480159aa0 0x7f24800b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:27.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.921+0000 7f2484e00640 1 --2- 172.21.15.151:0/1423411212 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f248015cff0 0x7f248009aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:27.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.921+0000 7f2484e00640 1 --2- 172.21.15.151:0/1423411212 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f248006f220 0x7f248009c770 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:27.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.921+0000 7f2484e00640 1 -- 172.21.15.151:0/1423411212 >> 172.21.15.151:0/1423411212 conn(0x7f24800b18e0 msgr2=0x7f248015d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:27.928 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.921+0000 7f2484e00640 1 -- 172.21.15.151:0/1423411212 shutdown_connections 2024-09-16T15:46:27.928 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:27.921+0000 7f2484e00640 1 -- 172.21.15.151:0/1423411212 wait complete. 2024-09-16T15:46:28.483 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:46:28.481+0000 7f48e4000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:46:28.483 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:46:28.481+0000 7f48e4000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:46:28.483 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:46:28.481+0000 7f48e4000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:46:28.483 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:46:28.481+0000 7f48e4000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:46:28.483 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:46:28.482+0000 7f48e4000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:46:28.939 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-16T15:46:29.134 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:29.133+0000 7fca09c00640 1 -- 172.21.15.151:0/4257108214 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca040d4900 msgr2=0x7fca040d0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:29.134 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:29.133+0000 7fca09c00640 1 --2- 172.21.15.151:0/4257108214 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca040d4900 0x7fca040d0a60 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec005fd0 tx=0x7fc9ec02a5c0 comp rx=0 tx=0).stop 2024-09-16T15:46:29.134 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:29.133+0000 7fca09c00640 1 -- 172.21.15.151:0/4257108214 shutdown_connections 2024-09-16T15:46:29.135 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:29.133+0000 7fca09c00640 1 --2- 172.21.15.151:0/4257108214 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca040d4900 0x7fca040d0a60 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:29.135 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:29.133+0000 7fca09c00640 1 --2- 172.21.15.151:0/4257108214 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca04161220 0x7fca040d0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:29.135 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:29.133+0000 7fca09c00640 1 -- 172.21.15.151:0/4257108214 >> 172.21.15.151:0/4257108214 conn(0x7fca040d3de0 msgr2=0x7fca040d41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:29.135 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:29.133+0000 7fca09c00640 1 -- 172.21.15.151:0/4257108214 shutdown_connections 2024-09-16T15:46:29.135 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:29.133+0000 7fca09c00640 1 -- 172.21.15.151:0/4257108214 wait complete. 2024-09-16T15:46:29.135 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:29.134+0000 7fca09c00640 1 Processor -- start 2024-09-16T15:46:29.135 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:29.135+0000 7fca09c00640 1 -- start start 2024-09-16T15:46:29.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:29.136+0000 7fca09c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca040b4050 0x7fca041575d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:29.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:29.136+0000 7fca09c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca040d4900 0x7fca040b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:29.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:29.136+0000 7fca09c00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca04161220 0x7fca040cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:29.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:29.136+0000 7fca09c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fca040cfc20 con 0x7fca040d4900 2024-09-16T15:46:29.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:29.136+0000 7fca09c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fca040cfaa0 con 0x7fca040b4050 2024-09-16T15:46:29.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:29.136+0000 7fca09c00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fca040b4bd0 con 0x7fca04161220 2024-09-16T15:46:29.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:29.136+0000 7fc9fbe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca040d4900 0x7fca040b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:29.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:29.136+0000 7fc9fbe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca040d4900 0x7fca040b8b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:58072/0 (socket says 172.21.15.151:58072) 2024-09-16T15:46:29.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:29.136+0000 7fc9fbe00640 1 -- 172.21.15.151:0/3465501489 learned_addr learned my addr 172.21.15.151:0/3465501489 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:46:29.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:29.136+0000 7fca03e00640 1 --2- 172.21.15.151:0/3465501489 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca040b4050 0x7fca041575d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:29.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:29.136+0000 7fca09200640 1 --2- 172.21.15.151:0/3465501489 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca04161220 0x7fca040cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:29.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:29.137+0000 7fc9fbe00640 1 -- 172.21.15.151:0/3465501489 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca04161220 msgr2=0x7fca040cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:29.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:29.137+0000 7fc9fbe00640 1 --2- 172.21.15.151:0/3465501489 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca04161220 0x7fca040cfda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:29.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:29.137+0000 7fc9fbe00640 1 -- 172.21.15.151:0/3465501489 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca040b4050 msgr2=0x7fca041575d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:29.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:29.137+0000 7fc9fbe00640 1 --2- 172.21.15.151:0/3465501489 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca040b4050 0x7fca041575d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:29.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:29.137+0000 7fc9fbe00640 1 -- 172.21.15.151:0/3465501489 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca040b7be0 con 0x7fca040d4900 2024-09-16T15:46:29.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:29.137+0000 7fc9fbe00640 1 --2- 172.21.15.151:0/3465501489 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca040d4900 0x7fca040b8b90 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec009850 tx=0x7fc9ec0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:29.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:29.137+0000 7fca00c00640 1 -- 172.21.15.151:0/3465501489 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9ec03f070 con 0x7fca040d4900 2024-09-16T15:46:29.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:29.137+0000 7fca00c00640 1 -- 172.21.15.151:0/3465501489 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc9ec004890 con 0x7fca040d4900 2024-09-16T15:46:29.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:29.137+0000 7fca00c00640 1 -- 172.21.15.151:0/3465501489 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9ec035030 con 0x7fca040d4900 2024-09-16T15:46:29.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:29.137+0000 7fca09c00640 1 -- 172.21.15.151:0/3465501489 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca040cb050 con 0x7fca040d4900 2024-09-16T15:46:29.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:29.137+0000 7fca09c00640 1 -- 172.21.15.151:0/3465501489 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca040cb610 con 0x7fca040d4900 2024-09-16T15:46:29.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:29.138+0000 7fca09c00640 1 -- 172.21.15.151:0/3465501489 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca040b5050 con 0x7fca040d4900 2024-09-16T15:46:29.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:29.139+0000 7fca00c00640 1 -- 172.21.15.151:0/3465501489 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 22) ==== 102256+0+0 (secure 0 0 0) 0x7fc9ec042070 con 0x7fca040d4900 2024-09-16T15:46:29.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:29.140+0000 7fca00c00640 1 --2- 172.21.15.151:0/3465501489 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc9e0079300 0x7fc9e007b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:29.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:29.140+0000 7fca00c00640 1 -- 172.21.15.151:0/3465501489 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5795+0+0 (secure 0 0 0) 0x7fc9ec0ba0f0 con 0x7fca040d4900 2024-09-16T15:46:29.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:29.141+0000 7fca03e00640 1 --2- 172.21.15.151:0/3465501489 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc9e0079300 0x7fc9e007b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:29.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:29.141+0000 7fca03e00640 1 --2- 172.21.15.151:0/3465501489 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc9e0079300 0x7fc9e007b7c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc9f4005fd0 tx=0x7fc9f400a250 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:29.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:29.144+0000 7fca00c00640 1 -- 172.21.15.151:0/3465501489 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc9ec083b60 con 0x7fca040d4900 2024-09-16T15:46:29.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:29.481+0000 7fca09c00640 1 -- 172.21.15.151:0/3465501489 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fca0408d630 con 0x7fca040d4900 2024-09-16T15:46:29.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:29.482+0000 7fca00c00640 1 -- 172.21.15.151:0/3465501489 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1913 (secure 0 0 0) 0x7fc9ec088590 con 0x7fca040d4900 2024-09-16T15:46:29.484 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:46:29.484 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-16T15:46:29.484 INFO:teuthology.orchestra.run.smithi151.stdout: "fsid": "1757be18-e41a-4446-bb4f-ee4ac81c39df", 2024-09-16T15:46:29.484 INFO:teuthology.orchestra.run.smithi151.stdout: "health": { 2024-09-16T15:46:29.484 INFO:teuthology.orchestra.run.smithi151.stdout: "status": "HEALTH_OK", 2024-09-16T15:46:29.484 INFO:teuthology.orchestra.run.smithi151.stdout: "checks": {}, 2024-09-16T15:46:29.484 INFO:teuthology.orchestra.run.smithi151.stdout: "mutes": [] 2024-09-16T15:46:29.484 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:46:29.484 INFO:teuthology.orchestra.run.smithi151.stdout: "election_epoch": 4, 2024-09-16T15:46:29.484 INFO:teuthology.orchestra.run.smithi151.stdout: "quorum": [ 2024-09-16T15:46:29.485 INFO:teuthology.orchestra.run.smithi151.stdout: 0, 2024-09-16T15:46:29.485 INFO:teuthology.orchestra.run.smithi151.stdout: 1, 2024-09-16T15:46:29.485 INFO:teuthology.orchestra.run.smithi151.stdout: 2 2024-09-16T15:46:29.485 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:46:29.485 INFO:teuthology.orchestra.run.smithi151.stdout: "quorum_names": [ 2024-09-16T15:46:29.485 INFO:teuthology.orchestra.run.smithi151.stdout: "a", 2024-09-16T15:46:29.485 INFO:teuthology.orchestra.run.smithi151.stdout: "b", 2024-09-16T15:46:29.485 INFO:teuthology.orchestra.run.smithi151.stdout: "c" 2024-09-16T15:46:29.485 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:46:29.485 INFO:teuthology.orchestra.run.smithi151.stdout: "quorum_age": 453, 2024-09-16T15:46:29.485 INFO:teuthology.orchestra.run.smithi151.stdout: "monmap": { 2024-09-16T15:46:29.485 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 1, 2024-09-16T15:46:29.485 INFO:teuthology.orchestra.run.smithi151.stdout: "min_mon_release_name": "squid", 2024-09-16T15:46:29.486 INFO:teuthology.orchestra.run.smithi151.stdout: "num_mons": 3 2024-09-16T15:46:29.486 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:46:29.486 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap": { 2024-09-16T15:46:29.486 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 64, 2024-09-16T15:46:29.486 INFO:teuthology.orchestra.run.smithi151.stdout: "num_osds": 8, 2024-09-16T15:46:29.486 INFO:teuthology.orchestra.run.smithi151.stdout: "num_up_osds": 8, 2024-09-16T15:46:29.486 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_up_since": 1726501150, 2024-09-16T15:46:29.486 INFO:teuthology.orchestra.run.smithi151.stdout: "num_in_osds": 8, 2024-09-16T15:46:29.486 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_in_since": 1726501140, 2024-09-16T15:46:29.486 INFO:teuthology.orchestra.run.smithi151.stdout: "num_remapped_pgs": 0 2024-09-16T15:46:29.486 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:46:29.487 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap": { 2024-09-16T15:46:29.487 INFO:teuthology.orchestra.run.smithi151.stdout: "pgs_by_state": [ 2024-09-16T15:46:29.487 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:46:29.487 INFO:teuthology.orchestra.run.smithi151.stdout: "state_name": "active+clean", 2024-09-16T15:46:29.487 INFO:teuthology.orchestra.run.smithi151.stdout: "count": 97 2024-09-16T15:46:29.487 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:46:29.487 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:46:29.487 INFO:teuthology.orchestra.run.smithi151.stdout: "num_pgs": 97, 2024-09-16T15:46:29.487 INFO:teuthology.orchestra.run.smithi151.stdout: "num_pools": 3, 2024-09-16T15:46:29.487 INFO:teuthology.orchestra.run.smithi151.stdout: "num_objects": 2585, 2024-09-16T15:46:29.487 INFO:teuthology.orchestra.run.smithi151.stdout: "data_bytes": 10738557764, 2024-09-16T15:46:29.487 INFO:teuthology.orchestra.run.smithi151.stdout: "bytes_used": 21873811456, 2024-09-16T15:46:29.487 INFO:teuthology.orchestra.run.smithi151.stdout: "bytes_avail": 751220301824, 2024-09-16T15:46:29.487 INFO:teuthology.orchestra.run.smithi151.stdout: "bytes_total": 773094113280, 2024-09-16T15:46:29.488 INFO:teuthology.orchestra.run.smithi151.stdout: "read_bytes_sec": 85, 2024-09-16T15:46:29.488 INFO:teuthology.orchestra.run.smithi151.stdout: "write_bytes_sec": 165639959, 2024-09-16T15:46:29.488 INFO:teuthology.orchestra.run.smithi151.stdout: "read_op_per_sec": 0, 2024-09-16T15:46:29.488 INFO:teuthology.orchestra.run.smithi151.stdout: "write_op_per_sec": 41 2024-09-16T15:46:29.488 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:46:29.488 INFO:teuthology.orchestra.run.smithi151.stdout: "fsmap": { 2024-09-16T15:46:29.488 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 37, 2024-09-16T15:46:29.488 INFO:teuthology.orchestra.run.smithi151.stdout: "btime": "2024-09-16T15:44:58:754420+0000", 2024-09-16T15:46:29.488 INFO:teuthology.orchestra.run.smithi151.stdout: "id": 4, 2024-09-16T15:46:29.488 INFO:teuthology.orchestra.run.smithi151.stdout: "up": 1, 2024-09-16T15:46:29.488 INFO:teuthology.orchestra.run.smithi151.stdout: "in": 1, 2024-09-16T15:46:29.488 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 1, 2024-09-16T15:46:29.488 INFO:teuthology.orchestra.run.smithi151.stdout: "by_rank": [ 2024-09-16T15:46:29.489 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:46:29.489 INFO:teuthology.orchestra.run.smithi151.stdout: "filesystem_id": 4, 2024-09-16T15:46:29.489 INFO:teuthology.orchestra.run.smithi151.stdout: "rank": 0, 2024-09-16T15:46:29.489 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "c", 2024-09-16T15:46:29.489 INFO:teuthology.orchestra.run.smithi151.stdout: "status": "up:active", 2024-09-16T15:46:29.489 INFO:teuthology.orchestra.run.smithi151.stdout: "gid": 5961 2024-09-16T15:46:29.489 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:46:29.489 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:46:29.489 INFO:teuthology.orchestra.run.smithi151.stdout: "up:standby": 3 2024-09-16T15:46:29.489 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:46:29.490 INFO:teuthology.orchestra.run.smithi151.stdout: "mgrmap": { 2024-09-16T15:46:29.490 INFO:teuthology.orchestra.run.smithi151.stdout: "available": true, 2024-09-16T15:46:29.490 INFO:teuthology.orchestra.run.smithi151.stdout: "num_standbys": 1, 2024-09-16T15:46:29.490 INFO:teuthology.orchestra.run.smithi151.stdout: "modules": [ 2024-09-16T15:46:29.490 INFO:teuthology.orchestra.run.smithi151.stdout: "iostat", 2024-09-16T15:46:29.490 INFO:teuthology.orchestra.run.smithi151.stdout: "nfs", 2024-09-16T15:46:29.490 INFO:teuthology.orchestra.run.smithi151.stdout: "restful" 2024-09-16T15:46:29.490 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:46:29.490 INFO:teuthology.orchestra.run.smithi151.stdout: "services": {} 2024-09-16T15:46:29.490 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:46:29.490 INFO:teuthology.orchestra.run.smithi151.stdout: "servicemap": { 2024-09-16T15:46:29.490 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 10, 2024-09-16T15:46:29.491 INFO:teuthology.orchestra.run.smithi151.stdout: "modified": "2024-09-16T15:45:53.688171+0000", 2024-09-16T15:46:29.491 INFO:teuthology.orchestra.run.smithi151.stdout: "services": {} 2024-09-16T15:46:29.491 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:46:29.491 INFO:teuthology.orchestra.run.smithi151.stdout: "progress_events": {} 2024-09-16T15:46:29.491 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-16T15:46:29.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:29.487+0000 7fca09c00640 1 -- 172.21.15.151:0/3465501489 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc9e0079300 msgr2=0x7fc9e007b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:29.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:29.487+0000 7fca09c00640 1 --2- 172.21.15.151:0/3465501489 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc9e0079300 0x7fc9e007b7c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc9f4005fd0 tx=0x7fc9f400a250 comp rx=0 tx=0).stop 2024-09-16T15:46:29.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:29.487+0000 7fca09c00640 1 -- 172.21.15.151:0/3465501489 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca040d4900 msgr2=0x7fca040b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:29.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:29.487+0000 7fca09c00640 1 --2- 172.21.15.151:0/3465501489 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca040d4900 0x7fca040b8b90 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec009850 tx=0x7fc9ec0039e0 comp rx=0 tx=0).stop 2024-09-16T15:46:29.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:29.487+0000 7fca09c00640 1 -- 172.21.15.151:0/3465501489 shutdown_connections 2024-09-16T15:46:29.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:29.487+0000 7fca09c00640 1 --2- 172.21.15.151:0/3465501489 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc9e0079300 0x7fc9e007b7c0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:29.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:29.487+0000 7fca09c00640 1 --2- 172.21.15.151:0/3465501489 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fca04161220 0x7fca040cfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:29.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:29.487+0000 7fca09c00640 1 --2- 172.21.15.151:0/3465501489 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca040d4900 0x7fca040b8b90 secure :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec009850 tx=0x7fc9ec0039e0 comp rx=0 tx=0).stop 2024-09-16T15:46:29.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:29.487+0000 7fca09c00640 1 --2- 172.21.15.151:0/3465501489 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca040b4050 0x7fca041575d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:29.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:29.487+0000 7fca09c00640 1 -- 172.21.15.151:0/3465501489 >> 172.21.15.151:0/3465501489 conn(0x7fca040d3de0 msgr2=0x7fca0406df50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:29.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:29.488+0000 7fca09c00640 1 -- 172.21.15.151:0/3465501489 shutdown_connections 2024-09-16T15:46:29.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:29.488+0000 7fca09c00640 1 -- 172.21.15.151:0/3465501489 wait complete. 2024-09-16T15:46:30.516 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-16T15:46:30.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:30.742+0000 7efc6aa00640 1 -- 172.21.15.151:0/2877418653 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc64073de0 msgr2=0x7efc64074240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:30.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:30.742+0000 7efc6aa00640 1 --2- 172.21.15.151:0/2877418653 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc64073de0 0x7efc64074240 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7efc54009be0 tx=0x7efc5402a850 comp rx=0 tx=0).stop 2024-09-16T15:46:30.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:30.742+0000 7efc6aa00640 1 -- 172.21.15.151:0/2877418653 shutdown_connections 2024-09-16T15:46:30.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:30.742+0000 7efc6aa00640 1 --2- 172.21.15.151:0/2877418653 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc64073de0 0x7efc64074240 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:30.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:30.742+0000 7efc6aa00640 1 -- 172.21.15.151:0/2877418653 >> 172.21.15.151:0/2877418653 conn(0x7efc640b18e0 msgr2=0x7efc640b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:30.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:30.742+0000 7efc6aa00640 1 -- 172.21.15.151:0/2877418653 shutdown_connections 2024-09-16T15:46:30.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:30.742+0000 7efc6aa00640 1 -- 172.21.15.151:0/2877418653 wait complete. 2024-09-16T15:46:30.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:30.745+0000 7efc6aa00640 1 Processor -- start 2024-09-16T15:46:30.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:30.745+0000 7efc6aa00640 1 -- start start 2024-09-16T15:46:30.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:30.746+0000 7efc6aa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc6406e6f0 0x7efc64142d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:30.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:30.746+0000 7efc6aa00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efc64073de0 0x7efc641515a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:30.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:30.746+0000 7efc6aa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc6415f9a0 0x7efc64151ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:30.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:30.746+0000 7efc6aa00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efc6406f2b0 con 0x7efc6415f9a0 2024-09-16T15:46:30.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:30.746+0000 7efc6aa00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efc640cd5e0 con 0x7efc6406e6f0 2024-09-16T15:46:30.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:30.746+0000 7efc6aa00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7efc6406f430 con 0x7efc64073de0 2024-09-16T15:46:30.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:30.746+0000 7efc68a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc6415f9a0 0x7efc64151ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:30.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:30.746+0000 7efc68a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc6415f9a0 0x7efc64151ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:58084/0 (socket says 172.21.15.151:58084) 2024-09-16T15:46:30.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:30.746+0000 7efc68a00640 1 -- 172.21.15.151:0/120224036 learned_addr learned my addr 172.21.15.151:0/120224036 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:46:30.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:30.747+0000 7efc68a00640 1 -- 172.21.15.151:0/120224036 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efc64073de0 msgr2=0x7efc641515a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:30.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:30.747+0000 7efc68a00640 1 --2- 172.21.15.151:0/120224036 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efc64073de0 0x7efc641515a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:30.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:30.747+0000 7efc68a00640 1 -- 172.21.15.151:0/120224036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc6406e6f0 msgr2=0x7efc64142d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:30.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:30.747+0000 7efc68a00640 1 --2- 172.21.15.151:0/120224036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc6406e6f0 0x7efc64142d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:30.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:30.747+0000 7efc68a00640 1 -- 172.21.15.151:0/120224036 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc64152020 con 0x7efc6415f9a0 2024-09-16T15:46:30.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:30.747+0000 7efc68a00640 1 --2- 172.21.15.151:0/120224036 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc6415f9a0 0x7efc64151ae0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7efc5c00b490 tx=0x7efc5c00b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:30.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:30.747+0000 7efc5be00640 1 -- 172.21.15.151:0/120224036 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc5c015020 con 0x7efc6415f9a0 2024-09-16T15:46:30.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:30.748+0000 7efc5be00640 1 -- 172.21.15.151:0/120224036 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc5c011940 con 0x7efc6415f9a0 2024-09-16T15:46:30.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:30.748+0000 7efc6aa00640 1 -- 172.21.15.151:0/120224036 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc640d0510 con 0x7efc6415f9a0 2024-09-16T15:46:30.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:30.748+0000 7efc5be00640 1 -- 172.21.15.151:0/120224036 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc5c011d90 con 0x7efc6415f9a0 2024-09-16T15:46:30.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:30.748+0000 7efc6aa00640 1 -- 172.21.15.151:0/120224036 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc641522e0 con 0x7efc6415f9a0 2024-09-16T15:46:30.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:30.749+0000 7efc6aa00640 1 -- 172.21.15.151:0/120224036 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc64069360 con 0x7efc6415f9a0 2024-09-16T15:46:30.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:30.752+0000 7efc5be00640 1 -- 172.21.15.151:0/120224036 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 23) ==== 102307+0+0 (secure 0 0 0) 0x7efc5c018020 con 0x7efc6415f9a0 2024-09-16T15:46:30.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:30.753+0000 7efc5be00640 1 --2- 172.21.15.151:0/120224036 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efc40079390 0x7efc4007b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:30.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:30.753+0000 7efc5be00640 1 -- 172.21.15.151:0/120224036 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5795+0+0 (secure 0 0 0) 0x7efc5c099130 con 0x7efc6415f9a0 2024-09-16T15:46:30.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:30.758+0000 7efc63e00640 1 --2- 172.21.15.151:0/120224036 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efc40079390 0x7efc4007b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:30.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:30.759+0000 7efc5be00640 1 -- 172.21.15.151:0/120224036 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7efc5c01f020 con 0x7efc6415f9a0 2024-09-16T15:46:30.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:30.759+0000 7efc63e00640 1 --2- 172.21.15.151:0/120224036 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efc40079390 0x7efc4007b850 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7efc4c001600 tx=0x7efc4c0079f0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:31.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:31.122+0000 7efc6aa00640 1 -- 172.21.15.151:0/120224036 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7efc64074240 con 0x7efc6415f9a0 2024-09-16T15:46:31.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:31.123+0000 7efc5be00640 1 -- 172.21.15.151:0/120224036 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2162 (secure 0 0 0) 0x7efc5c063250 con 0x7efc6415f9a0 2024-09-16T15:46:31.124 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:46:31.124 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-16T15:46:31.124 INFO:teuthology.orchestra.run.smithi151.stdout: "fsid": "1757be18-e41a-4446-bb4f-ee4ac81c39df", 2024-09-16T15:46:31.125 INFO:teuthology.orchestra.run.smithi151.stdout: "health": { 2024-09-16T15:46:31.125 INFO:teuthology.orchestra.run.smithi151.stdout: "status": "HEALTH_OK", 2024-09-16T15:46:31.125 INFO:teuthology.orchestra.run.smithi151.stdout: "checks": {}, 2024-09-16T15:46:31.125 INFO:teuthology.orchestra.run.smithi151.stdout: "mutes": [] 2024-09-16T15:46:31.125 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:46:31.125 INFO:teuthology.orchestra.run.smithi151.stdout: "election_epoch": 4, 2024-09-16T15:46:31.125 INFO:teuthology.orchestra.run.smithi151.stdout: "quorum": [ 2024-09-16T15:46:31.125 INFO:teuthology.orchestra.run.smithi151.stdout: 0, 2024-09-16T15:46:31.125 INFO:teuthology.orchestra.run.smithi151.stdout: 1, 2024-09-16T15:46:31.125 INFO:teuthology.orchestra.run.smithi151.stdout: 2 2024-09-16T15:46:31.125 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:46:31.125 INFO:teuthology.orchestra.run.smithi151.stdout: "quorum_names": [ 2024-09-16T15:46:31.125 INFO:teuthology.orchestra.run.smithi151.stdout: "a", 2024-09-16T15:46:31.125 INFO:teuthology.orchestra.run.smithi151.stdout: "b", 2024-09-16T15:46:31.125 INFO:teuthology.orchestra.run.smithi151.stdout: "c" 2024-09-16T15:46:31.126 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:46:31.126 INFO:teuthology.orchestra.run.smithi151.stdout: "quorum_age": 454, 2024-09-16T15:46:31.126 INFO:teuthology.orchestra.run.smithi151.stdout: "monmap": { 2024-09-16T15:46:31.126 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 1, 2024-09-16T15:46:31.126 INFO:teuthology.orchestra.run.smithi151.stdout: "min_mon_release_name": "squid", 2024-09-16T15:46:31.126 INFO:teuthology.orchestra.run.smithi151.stdout: "num_mons": 3 2024-09-16T15:46:31.126 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:46:31.126 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap": { 2024-09-16T15:46:31.126 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 64, 2024-09-16T15:46:31.126 INFO:teuthology.orchestra.run.smithi151.stdout: "num_osds": 8, 2024-09-16T15:46:31.126 INFO:teuthology.orchestra.run.smithi151.stdout: "num_up_osds": 8, 2024-09-16T15:46:31.126 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_up_since": 1726501150, 2024-09-16T15:46:31.126 INFO:teuthology.orchestra.run.smithi151.stdout: "num_in_osds": 8, 2024-09-16T15:46:31.126 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_in_since": 1726501140, 2024-09-16T15:46:31.126 INFO:teuthology.orchestra.run.smithi151.stdout: "num_remapped_pgs": 0 2024-09-16T15:46:31.127 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:46:31.127 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap": { 2024-09-16T15:46:31.127 INFO:teuthology.orchestra.run.smithi151.stdout: "pgs_by_state": [ 2024-09-16T15:46:31.127 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:46:31.127 INFO:teuthology.orchestra.run.smithi151.stdout: "state_name": "active+clean", 2024-09-16T15:46:31.127 INFO:teuthology.orchestra.run.smithi151.stdout: "count": 97 2024-09-16T15:46:31.127 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:46:31.127 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:46:31.127 INFO:teuthology.orchestra.run.smithi151.stdout: "num_pgs": 97, 2024-09-16T15:46:31.127 INFO:teuthology.orchestra.run.smithi151.stdout: "num_pools": 3, 2024-09-16T15:46:31.127 INFO:teuthology.orchestra.run.smithi151.stdout: "num_objects": 2585, 2024-09-16T15:46:31.127 INFO:teuthology.orchestra.run.smithi151.stdout: "data_bytes": 10738557764, 2024-09-16T15:46:31.127 INFO:teuthology.orchestra.run.smithi151.stdout: "bytes_used": 21873811456, 2024-09-16T15:46:31.128 INFO:teuthology.orchestra.run.smithi151.stdout: "bytes_avail": 751220301824, 2024-09-16T15:46:31.128 INFO:teuthology.orchestra.run.smithi151.stdout: "bytes_total": 773094113280, 2024-09-16T15:46:31.128 INFO:teuthology.orchestra.run.smithi151.stdout: "read_bytes_sec": 85, 2024-09-16T15:46:31.128 INFO:teuthology.orchestra.run.smithi151.stdout: "write_bytes_sec": 89459706, 2024-09-16T15:46:31.128 INFO:teuthology.orchestra.run.smithi151.stdout: "read_op_per_sec": 0, 2024-09-16T15:46:31.128 INFO:teuthology.orchestra.run.smithi151.stdout: "write_op_per_sec": 22 2024-09-16T15:46:31.128 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:46:31.128 INFO:teuthology.orchestra.run.smithi151.stdout: "fsmap": { 2024-09-16T15:46:31.128 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 37, 2024-09-16T15:46:31.128 INFO:teuthology.orchestra.run.smithi151.stdout: "btime": "2024-09-16T15:44:58:754420+0000", 2024-09-16T15:46:31.128 INFO:teuthology.orchestra.run.smithi151.stdout: "id": 4, 2024-09-16T15:46:31.128 INFO:teuthology.orchestra.run.smithi151.stdout: "up": 1, 2024-09-16T15:46:31.128 INFO:teuthology.orchestra.run.smithi151.stdout: "in": 1, 2024-09-16T15:46:31.128 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 1, 2024-09-16T15:46:31.128 INFO:teuthology.orchestra.run.smithi151.stdout: "by_rank": [ 2024-09-16T15:46:31.129 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:46:31.129 INFO:teuthology.orchestra.run.smithi151.stdout: "filesystem_id": 4, 2024-09-16T15:46:31.129 INFO:teuthology.orchestra.run.smithi151.stdout: "rank": 0, 2024-09-16T15:46:31.129 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "c", 2024-09-16T15:46:31.129 INFO:teuthology.orchestra.run.smithi151.stdout: "status": "up:active", 2024-09-16T15:46:31.129 INFO:teuthology.orchestra.run.smithi151.stdout: "gid": 5961 2024-09-16T15:46:31.129 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:46:31.129 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:46:31.129 INFO:teuthology.orchestra.run.smithi151.stdout: "up:standby": 3 2024-09-16T15:46:31.129 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:46:31.129 INFO:teuthology.orchestra.run.smithi151.stdout: "mgrmap": { 2024-09-16T15:46:31.129 INFO:teuthology.orchestra.run.smithi151.stdout: "available": true, 2024-09-16T15:46:31.129 INFO:teuthology.orchestra.run.smithi151.stdout: "num_standbys": 1, 2024-09-16T15:46:31.129 INFO:teuthology.orchestra.run.smithi151.stdout: "modules": [ 2024-09-16T15:46:31.129 INFO:teuthology.orchestra.run.smithi151.stdout: "iostat", 2024-09-16T15:46:31.129 INFO:teuthology.orchestra.run.smithi151.stdout: "nfs", 2024-09-16T15:46:31.130 INFO:teuthology.orchestra.run.smithi151.stdout: "restful" 2024-09-16T15:46:31.130 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:46:31.130 INFO:teuthology.orchestra.run.smithi151.stdout: "services": {} 2024-09-16T15:46:31.130 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:46:31.130 INFO:teuthology.orchestra.run.smithi151.stdout: "servicemap": { 2024-09-16T15:46:31.130 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 10, 2024-09-16T15:46:31.130 INFO:teuthology.orchestra.run.smithi151.stdout: "modified": "2024-09-16T15:45:53.688171+0000", 2024-09-16T15:46:31.130 INFO:teuthology.orchestra.run.smithi151.stdout: "services": {} 2024-09-16T15:46:31.130 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:46:31.130 INFO:teuthology.orchestra.run.smithi151.stdout: "progress_events": { 2024-09-16T15:46:31.130 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr-vol-ongoing-clones": { 2024-09-16T15:46:31.130 INFO:teuthology.orchestra.run.smithi151.stdout: "message": "1 ongoing clones - average progress is 3.125% (1s)\n [............................] (remaining: 33s)", 2024-09-16T15:46:31.130 INFO:teuthology.orchestra.run.smithi151.stdout: "progress": 0.03125, 2024-09-16T15:46:31.130 INFO:teuthology.orchestra.run.smithi151.stdout: "add_to_ceph_s": true 2024-09-16T15:46:31.130 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:46:31.131 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:46:31.131 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-16T15:46:31.131 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:31.126+0000 7efc6aa00640 1 -- 172.21.15.151:0/120224036 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efc40079390 msgr2=0x7efc4007b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:31.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:31.126+0000 7efc6aa00640 1 --2- 172.21.15.151:0/120224036 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efc40079390 0x7efc4007b850 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7efc4c001600 tx=0x7efc4c0079f0 comp rx=0 tx=0).stop 2024-09-16T15:46:31.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:31.127+0000 7efc6aa00640 1 -- 172.21.15.151:0/120224036 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc6415f9a0 msgr2=0x7efc64151ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:31.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:31.127+0000 7efc6aa00640 1 --2- 172.21.15.151:0/120224036 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc6415f9a0 0x7efc64151ae0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7efc5c00b490 tx=0x7efc5c00b950 comp rx=0 tx=0).stop 2024-09-16T15:46:31.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:31.127+0000 7efc6aa00640 1 -- 172.21.15.151:0/120224036 shutdown_connections 2024-09-16T15:46:31.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:31.127+0000 7efc6aa00640 1 --2- 172.21.15.151:0/120224036 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efc40079390 0x7efc4007b850 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:31.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:31.127+0000 7efc6aa00640 1 --2- 172.21.15.151:0/120224036 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc6415f9a0 0x7efc64151ae0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:31.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:31.127+0000 7efc6aa00640 1 --2- 172.21.15.151:0/120224036 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efc64073de0 0x7efc641515a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:31.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:31.127+0000 7efc6aa00640 1 --2- 172.21.15.151:0/120224036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc6406e6f0 0x7efc64142d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:31.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:31.127+0000 7efc6aa00640 1 -- 172.21.15.151:0/120224036 >> 172.21.15.151:0/120224036 conn(0x7efc640b18e0 msgr2=0x7efc6406eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:31.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:31.127+0000 7efc6aa00640 1 -- 172.21.15.151:0/120224036 shutdown_connections 2024-09-16T15:46:31.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:31.127+0000 7efc6aa00640 1 -- 172.21.15.151:0/120224036 wait complete. 2024-09-16T15:46:31.149 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs ss1clone1 2024-09-16T15:46:31.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:31.394+0000 7ff9a2c00640 1 -- 172.21.15.151:0/842774290 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff99c0af190 msgr2=0x7ff99c0b2b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:31.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:31.394+0000 7ff9a2c00640 1 --2- 172.21.15.151:0/842774290 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff99c0af190 0x7ff99c0b2b40 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7ff99c0cfba0 tx=0x7ff98802a5f0 comp rx=0 tx=0).stop 2024-09-16T15:46:31.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:31.394+0000 7ff9a2c00640 1 -- 172.21.15.151:0/842774290 shutdown_connections 2024-09-16T15:46:31.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:31.395+0000 7ff9a2c00640 1 --2- 172.21.15.151:0/842774290 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff99c0af190 0x7ff99c0b2b40 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:31.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:31.395+0000 7ff9a2c00640 1 --2- 172.21.15.151:0/842774290 >> v2:172.21.15.167:3300/0 conn(0x7ff99c161760 0x7ff99c0ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:31.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:31.395+0000 7ff9a2c00640 1 -- 172.21.15.151:0/842774290 >> 172.21.15.151:0/842774290 conn(0x7ff99c0b1840 msgr2=0x7ff99c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:31.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:31.395+0000 7ff9a2c00640 1 -- 172.21.15.151:0/842774290 shutdown_connections 2024-09-16T15:46:31.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:31.395+0000 7ff9a2c00640 1 -- 172.21.15.151:0/842774290 wait complete. 2024-09-16T15:46:31.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:31.399+0000 7ff9a2c00640 1 Processor -- start 2024-09-16T15:46:31.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:31.399+0000 7ff9a2c00640 1 -- start start 2024-09-16T15:46:31.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:31.400+0000 7ff9a2c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff99c0af190 0x7ff99c1446b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:31.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:31.400+0000 7ff9a2c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff99c0cec60 0x7ff99c147880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:31.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:31.400+0000 7ff9a2c00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff99c161760 0x7ff99c1587b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:31.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:31.400+0000 7ff9a2c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff99c162310 con 0x7ff99c0af190 2024-09-16T15:46:31.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:31.400+0000 7ff9a2c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff99c162190 con 0x7ff99c0cec60 2024-09-16T15:46:31.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:31.400+0000 7ff9a2c00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7ff99c162490 con 0x7ff99c161760 2024-09-16T15:46:31.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:31.401+0000 7ff99be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff99c0af190 0x7ff99c1446b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:31.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:31.401+0000 7ff99be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff99c0af190 0x7ff99c1446b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:58094/0 (socket says 172.21.15.151:58094) 2024-09-16T15:46:31.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:31.401+0000 7ff99be00640 1 -- 172.21.15.151:0/1382771410 learned_addr learned my addr 172.21.15.151:0/1382771410 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:46:31.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:31.401+0000 7ff99be00640 1 -- 172.21.15.151:0/1382771410 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff99c161760 msgr2=0x7ff99c1587b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:31.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:31.401+0000 7ff9a0c00640 1 --2- 172.21.15.151:0/1382771410 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff99c161760 0x7ff99c1587b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:31.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:31.401+0000 7ff99be00640 1 --2- 172.21.15.151:0/1382771410 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff99c161760 0x7ff99c1587b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:31.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:31.401+0000 7ff99be00640 1 -- 172.21.15.151:0/1382771410 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff99c0cec60 msgr2=0x7ff99c147880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:31.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:31.401+0000 7ff99be00640 1 --2- 172.21.15.151:0/1382771410 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff99c0cec60 0x7ff99c147880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:31.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:31.401+0000 7ff99be00640 1 -- 172.21.15.151:0/1382771410 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff99c147dc0 con 0x7ff99c0af190 2024-09-16T15:46:31.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:31.401+0000 7ff9a0c00640 1 --2- 172.21.15.151:0/1382771410 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff99c161760 0x7ff99c1587b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:46:31.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:31.402+0000 7ff99be00640 1 --2- 172.21.15.151:0/1382771410 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff99c0af190 0x7ff99c1446b0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7ff98c00e7f0 tx=0x7ff98c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:31.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:31.402+0000 7ff993e00640 1 -- 172.21.15.151:0/1382771410 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98c00ccc0 con 0x7ff99c0af190 2024-09-16T15:46:31.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:31.402+0000 7ff9a2c00640 1 -- 172.21.15.151:0/1382771410 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff99c145060 con 0x7ff99c0af190 2024-09-16T15:46:31.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:31.402+0000 7ff993e00640 1 -- 172.21.15.151:0/1382771410 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff98c00ce80 con 0x7ff99c0af190 2024-09-16T15:46:31.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:31.402+0000 7ff993e00640 1 -- 172.21.15.151:0/1382771410 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98c005280 con 0x7ff99c0af190 2024-09-16T15:46:31.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:31.402+0000 7ff9a2c00640 1 -- 172.21.15.151:0/1382771410 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff99c145620 con 0x7ff99c0af190 2024-09-16T15:46:31.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:31.404+0000 7ff9a2c00640 1 -- 172.21.15.151:0/1382771410 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff99c0ce4a0 con 0x7ff99c0af190 2024-09-16T15:46:31.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:31.405+0000 7ff993e00640 1 -- 172.21.15.151:0/1382771410 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 23) ==== 102307+0+0 (secure 0 0 0) 0x7ff98c005420 con 0x7ff99c0af190 2024-09-16T15:46:31.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:31.406+0000 7ff993e00640 1 --2- 172.21.15.151:0/1382771410 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff97c079390 0x7ff97c07b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:31.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:31.406+0000 7ff993e00640 1 -- 172.21.15.151:0/1382771410 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5795+0+0 (secure 0 0 0) 0x7ff98c099eb0 con 0x7ff99c0af190 2024-09-16T15:46:31.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:31.410+0000 7ff99b400640 1 --2- 172.21.15.151:0/1382771410 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff97c079390 0x7ff97c07b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:31.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:31.411+0000 7ff993e00640 1 -- 172.21.15.151:0/1382771410 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff98c09ee20 con 0x7ff99c0af190 2024-09-16T15:46:31.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:31.415+0000 7ff99b400640 1 --2- 172.21.15.151:0/1382771410 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff97c079390 0x7ff97c07b850 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff988033000 tx=0x7ff9880046c0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:31.732 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:31.731+0000 7ff9a2c00640 1 -- 172.21.15.151:0/1382771410 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7ff99c0cfe90 con 0x7ff97c079390 2024-09-16T15:46:31.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:31.978+0000 7ff993e00640 1 -- 172.21.15.151:0/1382771410 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff99c0cfe90 con 0x7ff97c079390 2024-09-16T15:46:31.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:31.981+0000 7ff9a2c00640 1 -- 172.21.15.151:0/1382771410 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff97c079390 msgr2=0x7ff97c07b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:31.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:31.981+0000 7ff9a2c00640 1 --2- 172.21.15.151:0/1382771410 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff97c079390 0x7ff97c07b850 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff988033000 tx=0x7ff9880046c0 comp rx=0 tx=0).stop 2024-09-16T15:46:31.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:31.981+0000 7ff9a2c00640 1 -- 172.21.15.151:0/1382771410 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff99c0af190 msgr2=0x7ff99c1446b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:31.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:31.981+0000 7ff9a2c00640 1 --2- 172.21.15.151:0/1382771410 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff99c0af190 0x7ff99c1446b0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7ff98c00e7f0 tx=0x7ff98c00ecb0 comp rx=0 tx=0).stop 2024-09-16T15:46:31.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:31.981+0000 7ff9a2c00640 1 -- 172.21.15.151:0/1382771410 shutdown_connections 2024-09-16T15:46:31.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:31.981+0000 7ff9a2c00640 1 --2- 172.21.15.151:0/1382771410 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff97c079390 0x7ff97c07b850 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:31.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:31.981+0000 7ff9a2c00640 1 --2- 172.21.15.151:0/1382771410 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff99c0cec60 0x7ff99c147880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:31.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:31.981+0000 7ff9a2c00640 1 --2- 172.21.15.151:0/1382771410 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff99c161760 0x7ff99c1587b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:31.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:31.981+0000 7ff9a2c00640 1 --2- 172.21.15.151:0/1382771410 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff99c0af190 0x7ff99c1446b0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:31.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:31.981+0000 7ff9a2c00640 1 -- 172.21.15.151:0/1382771410 >> 172.21.15.151:0/1382771410 conn(0x7ff99c0b1840 msgr2=0x7ff99c0adfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:31.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:31.981+0000 7ff9a2c00640 1 -- 172.21.15.151:0/1382771410 shutdown_connections 2024-09-16T15:46:31.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:31.981+0000 7ff9a2c00640 1 -- 172.21.15.151:0/1382771410 wait complete. 2024-09-16T15:46:31.998 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-16T15:46:32.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.157+0000 7f2957200640 1 -- 172.21.15.151:0/2132921807 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2938004d90 msgr2=0x7f2938005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:32.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.157+0000 7f2957200640 1 --2- 172.21.15.151:0/2132921807 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2938004d90 0x7f2938005210 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f294c001d30 tx=0x7f2938005830 comp rx=0 tx=0).stop 2024-09-16T15:46:32.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.157+0000 7f2956800640 1 -- 172.21.15.151:0/2132921807 reap_dead start 2024-09-16T15:46:32.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.158+0000 7f2957200640 1 -- 172.21.15.151:0/2132921807 shutdown_connections 2024-09-16T15:46:32.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.158+0000 7f2957200640 1 -- 172.21.15.151:0/2132921807 >> 172.21.15.151:0/2132921807 conn(0x7f29500d3df0 msgr2=0x7f29500d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:32.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.158+0000 7f2957200640 1 -- 172.21.15.151:0/2132921807 shutdown_connections 2024-09-16T15:46:32.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.158+0000 7f2957200640 1 -- 172.21.15.151:0/2132921807 wait complete. 2024-09-16T15:46:32.160 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.159+0000 7f2957200640 1 Processor -- start 2024-09-16T15:46:32.160 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.159+0000 7f2957200640 1 -- start start 2024-09-16T15:46:32.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.160+0000 7f2957200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2938004d90 0x7f29500ca240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:32.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.160+0000 7f2957200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f29500ca780 0x7f29500b7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:32.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.160+0000 7f2957200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29500b81b0 0x7f29501f7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:32.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.160+0000 7f2957200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f29500b3f50 con 0x7f2938004d90 2024-09-16T15:46:32.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.160+0000 7f2957200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f29500b3dd0 con 0x7f29500b81b0 2024-09-16T15:46:32.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.160+0000 7f2957200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f29500b1e80 con 0x7f29500ca780 2024-09-16T15:46:32.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.161+0000 7f2955e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2938004d90 0x7f29500ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:32.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.161+0000 7f2955e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2938004d90 0x7f29500ca240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:58110/0 (socket says 172.21.15.151:58110) 2024-09-16T15:46:32.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.161+0000 7f2955e00640 1 -- 172.21.15.151:0/3519713372 learned_addr learned my addr 172.21.15.151:0/3519713372 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:46:32.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.161+0000 7f2956800640 1 --2- 172.21.15.151:0/3519713372 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29500b81b0 0x7f29501f7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:32.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.161+0000 7f2955400640 1 --2- 172.21.15.151:0/3519713372 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f29500ca780 0x7f29500b7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:32.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.161+0000 7f2955e00640 1 -- 172.21.15.151:0/3519713372 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f29500ca780 msgr2=0x7f29500b7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:32.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.161+0000 7f2955e00640 1 --2- 172.21.15.151:0/3519713372 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f29500ca780 0x7f29500b7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:32.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.161+0000 7f2955e00640 1 -- 172.21.15.151:0/3519713372 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29500b81b0 msgr2=0x7f29501f7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:32.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.161+0000 7f2955e00640 1 --2- 172.21.15.151:0/3519713372 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29500b81b0 0x7f29501f7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:32.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.161+0000 7f2955e00640 1 -- 172.21.15.151:0/3519713372 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29500c94e0 con 0x7f2938004d90 2024-09-16T15:46:32.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.161+0000 7f2955e00640 1 --2- 172.21.15.151:0/3519713372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2938004d90 0x7f29500ca240 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f2944006680 tx=0x7f29440066b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:32.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.162+0000 7f294a000640 1 -- 172.21.15.151:0/3519713372 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2944015070 con 0x7f2938004d90 2024-09-16T15:46:32.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.162+0000 7f294a000640 1 -- 172.21.15.151:0/3519713372 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2944011920 con 0x7f2938004d90 2024-09-16T15:46:32.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.162+0000 7f2957200640 1 -- 172.21.15.151:0/3519713372 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29501f8920 con 0x7f2938004d90 2024-09-16T15:46:32.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.162+0000 7f294a000640 1 -- 172.21.15.151:0/3519713372 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2944011d70 con 0x7f2938004d90 2024-09-16T15:46:32.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.162+0000 7f2957200640 1 -- 172.21.15.151:0/3519713372 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29500c9760 con 0x7f2938004d90 2024-09-16T15:46:32.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.163+0000 7f2957200640 1 -- 172.21.15.151:0/3519713372 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29500b1770 con 0x7f2938004d90 2024-09-16T15:46:32.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.164+0000 7f294a000640 1 -- 172.21.15.151:0/3519713372 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 23) ==== 102307+0+0 (secure 0 0 0) 0x7f294400f430 con 0x7f2938004d90 2024-09-16T15:46:32.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.165+0000 7f294a000640 1 --2- 172.21.15.151:0/3519713372 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f29340793c0 0x7f293407b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:32.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.165+0000 7f294a000640 1 -- 172.21.15.151:0/3519713372 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5795+0+0 (secure 0 0 0) 0x7f2944099f30 con 0x7f2938004d90 2024-09-16T15:46:32.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.165+0000 7f2955400640 1 --2- 172.21.15.151:0/3519713372 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f29340793c0 0x7f293407b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:32.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.166+0000 7f2955400640 1 --2- 172.21.15.151:0/3519713372 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f29340793c0 0x7f293407b880 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f294000b6c0 tx=0x7f294000b260 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:32.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.169+0000 7f294a000640 1 -- 172.21.15.151:0/3519713372 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f29500c94e0 con 0x7f2938004d90 2024-09-16T15:46:32.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.376+0000 7f2957200640 1 -- 172.21.15.151:0/3519713372 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f29500d1680 con 0x7f29340793c0 2024-09-16T15:46:32.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.378+0000 7f294a000640 1 -- 172.21.15.151:0/3519713372 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f29500d1680 con 0x7f29340793c0 2024-09-16T15:46:32.379 INFO:teuthology.orchestra.run.smithi151.stdout:[ 2024-09-16T15:46:32.379 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:46:32.379 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "cephfs" 2024-09-16T15:46:32.379 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:46:32.379 INFO:teuthology.orchestra.run.smithi151.stdout:] 2024-09-16T15:46:32.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.379+0000 7f2957200640 1 -- 172.21.15.151:0/3519713372 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f29340793c0 msgr2=0x7f293407b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:32.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.379+0000 7f2957200640 1 --2- 172.21.15.151:0/3519713372 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f29340793c0 0x7f293407b880 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f294000b6c0 tx=0x7f294000b260 comp rx=0 tx=0).stop 2024-09-16T15:46:32.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.380+0000 7f2957200640 1 -- 172.21.15.151:0/3519713372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2938004d90 msgr2=0x7f29500ca240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:32.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.380+0000 7f2957200640 1 --2- 172.21.15.151:0/3519713372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2938004d90 0x7f29500ca240 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f2944006680 tx=0x7f29440066b0 comp rx=0 tx=0).stop 2024-09-16T15:46:32.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.380+0000 7f2957200640 1 -- 172.21.15.151:0/3519713372 shutdown_connections 2024-09-16T15:46:32.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.380+0000 7f2957200640 1 --2- 172.21.15.151:0/3519713372 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f29340793c0 0x7f293407b880 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:32.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.380+0000 7f2957200640 1 --2- 172.21.15.151:0/3519713372 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29500b81b0 0x7f29501f7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:32.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.380+0000 7f2957200640 1 --2- 172.21.15.151:0/3519713372 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f29500ca780 0x7f29500b7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:32.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.380+0000 7f2957200640 1 --2- 172.21.15.151:0/3519713372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2938004d90 0x7f29500ca240 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:32.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.380+0000 7f2957200640 1 -- 172.21.15.151:0/3519713372 >> 172.21.15.151:0/3519713372 conn(0x7f29500d3df0 msgr2=0x7f295006eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:32.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.380+0000 7f2957200640 1 -- 172.21.15.151:0/3519713372 shutdown_connections 2024-09-16T15:46:32.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.380+0000 7f2957200640 1 -- 172.21.15.151:0/3519713372 wait complete. 2024-09-16T15:46:32.396 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-09-16T15:46:32.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.556+0000 7f8125c00640 1 -- 172.21.15.151:0/2602012319 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8104003a50 msgr2=0x7f8104003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:32.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.556+0000 7f8125c00640 1 --2- 172.21.15.151:0/2602012319 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8104003a50 0x7f8104003e70 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f811000c290 tx=0x7f8104005830 comp rx=0 tx=0).stop 2024-09-16T15:46:32.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.556+0000 7f8125200640 1 -- 172.21.15.151:0/2602012319 reap_dead start 2024-09-16T15:46:32.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.556+0000 7f8125c00640 1 -- 172.21.15.151:0/2602012319 shutdown_connections 2024-09-16T15:46:32.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.556+0000 7f8125c00640 1 -- 172.21.15.151:0/2602012319 >> 172.21.15.151:0/2602012319 conn(0x7f81200b5680 msgr2=0x7f81200b5a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:32.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.556+0000 7f8125c00640 1 -- 172.21.15.151:0/2602012319 shutdown_connections 2024-09-16T15:46:32.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.556+0000 7f8125c00640 1 -- 172.21.15.151:0/2602012319 wait complete. 2024-09-16T15:46:32.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.557+0000 7f8125c00640 1 Processor -- start 2024-09-16T15:46:32.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.558+0000 7f8125c00640 1 -- start start 2024-09-16T15:46:32.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.558+0000 7f8125c00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8104003a50 0x7f8120157a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:32.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.558+0000 7f8125c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f812009ac00 0x7f81200ad100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:32.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.558+0000 7f8125c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f81201495b0 0x7f81200afca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:32.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.559+0000 7f8125c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8120163b00 con 0x7f81201495b0 2024-09-16T15:46:32.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.559+0000 7f8125c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f81200b4cb0 con 0x7f812009ac00 2024-09-16T15:46:32.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.559+0000 7f8125c00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f8120163c80 con 0x7f8104003a50 2024-09-16T15:46:32.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.559+0000 7f8125200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f81201495b0 0x7f81200afca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:32.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.559+0000 7f8125200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f81201495b0 0x7f81200afca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:58132/0 (socket says 172.21.15.151:58132) 2024-09-16T15:46:32.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.559+0000 7f8125200640 1 -- 172.21.15.151:0/2977753691 learned_addr learned my addr 172.21.15.151:0/2977753691 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:46:32.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.559+0000 7f811f400640 1 --2- 172.21.15.151:0/2977753691 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f812009ac00 0x7f81200ad100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:32.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.559+0000 7f811fe00640 1 --2- 172.21.15.151:0/2977753691 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8104003a50 0x7f8120157a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:32.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.559+0000 7f8125200640 1 -- 172.21.15.151:0/2977753691 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8104003a50 msgr2=0x7f8120157a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:32.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.559+0000 7f8125200640 1 --2- 172.21.15.151:0/2977753691 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8104003a50 0x7f8120157a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:32.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.559+0000 7f8125200640 1 -- 172.21.15.151:0/2977753691 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f812009ac00 msgr2=0x7f81200ad100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:32.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.559+0000 7f8125200640 1 --2- 172.21.15.151:0/2977753691 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f812009ac00 0x7f81200ad100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:32.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.559+0000 7f8125200640 1 -- 172.21.15.151:0/2977753691 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f812009b090 con 0x7f81201495b0 2024-09-16T15:46:32.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.559+0000 7f8125200640 1 --2- 172.21.15.151:0/2977753691 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f81201495b0 0x7f81200afca0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f812006e990 tx=0x7f811800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:32.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.560+0000 7f8117e00640 1 -- 172.21.15.151:0/2977753691 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810c009860 con 0x7f81201495b0 2024-09-16T15:46:32.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.560+0000 7f8117e00640 1 -- 172.21.15.151:0/2977753691 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8118003d80 con 0x7f81201495b0 2024-09-16T15:46:32.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.560+0000 7f8117e00640 1 -- 172.21.15.151:0/2977753691 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8118004d80 con 0x7f81201495b0 2024-09-16T15:46:32.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.560+0000 7f8125c00640 1 -- 172.21.15.151:0/2977753691 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f812009b370 con 0x7f81201495b0 2024-09-16T15:46:32.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.560+0000 7f8125c00640 1 -- 172.21.15.151:0/2977753691 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f812009cd80 con 0x7f81201495b0 2024-09-16T15:46:32.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.561+0000 7f8125c00640 1 -- 172.21.15.151:0/2977753691 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81200b4e30 con 0x7f81201495b0 2024-09-16T15:46:32.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.562+0000 7f8117e00640 1 -- 172.21.15.151:0/2977753691 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 23) ==== 102307+0+0 (secure 0 0 0) 0x7f8118004030 con 0x7f81201495b0 2024-09-16T15:46:32.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.563+0000 7f8117e00640 1 --2- 172.21.15.151:0/2977753691 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f81000793c0 0x7f810007b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:32.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.563+0000 7f8117e00640 1 -- 172.21.15.151:0/2977753691 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5795+0+0 (secure 0 0 0) 0x7f811809a0a0 con 0x7f81201495b0 2024-09-16T15:46:32.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.563+0000 7f811fe00640 1 --2- 172.21.15.151:0/2977753691 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f81000793c0 0x7f810007b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:32.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.564+0000 7f811fe00640 1 --2- 172.21.15.151:0/2977753691 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f81000793c0 0x7f810007b880 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8110033000 tx=0x7f8110038040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:32.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.567+0000 7f8117e00640 1 -- 172.21.15.151:0/2977753691 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f81180638c0 con 0x7f81201495b0 2024-09-16T15:46:32.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.778+0000 7f8125c00640 1 -- 172.21.15.151:0/2977753691 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8120159b10 con 0x7f81000793c0 2024-09-16T15:46:32.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.784+0000 7f8117e00640 1 -- 172.21.15.151:0/2977753691 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+79 (secure 0 0 0) 0x7f8120159b10 con 0x7f81000793c0 2024-09-16T15:46:32.785 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:46:32.785 INFO:teuthology.orchestra.run.smithi151.stdout:[ 2024-09-16T15:46:32.785 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:46:32.785 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sv1" 2024-09-16T15:46:32.785 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:46:32.786 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:46:32.786 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ss1clone1" 2024-09-16T15:46:32.786 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:46:32.786 INFO:teuthology.orchestra.run.smithi151.stdout:] 2024-09-16T15:46:32.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.786+0000 7f8125c00640 1 -- 172.21.15.151:0/2977753691 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f81000793c0 msgr2=0x7f810007b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:32.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.786+0000 7f8125c00640 1 --2- 172.21.15.151:0/2977753691 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f81000793c0 0x7f810007b880 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8110033000 tx=0x7f8110038040 comp rx=0 tx=0).stop 2024-09-16T15:46:32.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.786+0000 7f8125c00640 1 -- 172.21.15.151:0/2977753691 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f81201495b0 msgr2=0x7f81200afca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:32.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.786+0000 7f8125c00640 1 --2- 172.21.15.151:0/2977753691 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f81201495b0 0x7f81200afca0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f812006e990 tx=0x7f811800f790 comp rx=0 tx=0).stop 2024-09-16T15:46:32.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.787+0000 7f8125c00640 1 -- 172.21.15.151:0/2977753691 shutdown_connections 2024-09-16T15:46:32.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.787+0000 7f8125c00640 1 --2- 172.21.15.151:0/2977753691 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f81000793c0 0x7f810007b880 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:32.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.787+0000 7f8125c00640 1 --2- 172.21.15.151:0/2977753691 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f812009ac00 0x7f81200ad100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:32.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.787+0000 7f8125c00640 1 --2- 172.21.15.151:0/2977753691 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f81201495b0 0x7f81200afca0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:32.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.787+0000 7f8125c00640 1 --2- 172.21.15.151:0/2977753691 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8104003a50 0x7f8120157a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:32.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.787+0000 7f8125c00640 1 -- 172.21.15.151:0/2977753691 >> 172.21.15.151:0/2977753691 conn(0x7f81200b5680 msgr2=0x7f81201602c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:32.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.787+0000 7f8125c00640 1 -- 172.21.15.151:0/2977753691 shutdown_connections 2024-09-16T15:46:32.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.787+0000 7f8125c00640 1 -- 172.21.15.151:0/2977753691 wait complete. 2024-09-16T15:46:32.803 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-09-16T15:46:32.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.965+0000 7feea9000640 1 -- 172.21.15.151:0/558482303 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feea406e590 msgr2=0x7feea406fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:32.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.965+0000 7feea9000640 1 --2- 172.21.15.151:0/558482303 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feea406e590 0x7feea406fe40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fee90009830 tx=0x7fee9002a510 comp rx=0 tx=0).stop 2024-09-16T15:46:32.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.966+0000 7feea9000640 1 -- 172.21.15.151:0/558482303 shutdown_connections 2024-09-16T15:46:32.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.966+0000 7feea9000640 1 --2- 172.21.15.151:0/558482303 >> v2:172.21.15.167:3300/0 conn(0x7feea4070380 0x7feea40cb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:32.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.966+0000 7feea9000640 1 --2- 172.21.15.151:0/558482303 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feea406e590 0x7feea406fe40 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:32.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.966+0000 7feea9000640 1 -- 172.21.15.151:0/558482303 >> 172.21.15.151:0/558482303 conn(0x7feea40cfa70 msgr2=0x7feea40cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:32.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.966+0000 7feea9000640 1 -- 172.21.15.151:0/558482303 shutdown_connections 2024-09-16T15:46:32.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.966+0000 7feea9000640 1 -- 172.21.15.151:0/558482303 wait complete. 2024-09-16T15:46:32.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.967+0000 7feea9000640 1 Processor -- start 2024-09-16T15:46:32.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.967+0000 7feea9000640 1 -- start start 2024-09-16T15:46:32.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.968+0000 7feea9000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feea406e590 0x7feea40ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:32.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.968+0000 7feea9000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feea4070380 0x7feea40adaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:32.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.968+0000 7feea9000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feea415bda0 0x7feea4159af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:32.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.968+0000 7feea9000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7feea40cb910 con 0x7feea4070380 2024-09-16T15:46:32.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.968+0000 7feea9000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7feea40708f0 con 0x7feea415bda0 2024-09-16T15:46:32.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.968+0000 7feea9000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7feea40cba90 con 0x7feea406e590 2024-09-16T15:46:32.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.968+0000 7feea2a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feea4070380 0x7feea40adaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:32.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.968+0000 7feea2a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feea4070380 0x7feea40adaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47758/0 (socket says 172.21.15.151:47758) 2024-09-16T15:46:32.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.968+0000 7feea2a00640 1 -- 172.21.15.151:0/1547288944 learned_addr learned my addr 172.21.15.151:0/1547288944 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:46:32.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.969+0000 7feea3400640 1 --2- 172.21.15.151:0/1547288944 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feea406e590 0x7feea40ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:32.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.969+0000 7feea3e00640 1 --2- 172.21.15.151:0/1547288944 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feea415bda0 0x7feea4159af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:32.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.969+0000 7feea2a00640 1 -- 172.21.15.151:0/1547288944 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feea406e590 msgr2=0x7feea40ceb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:32.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.969+0000 7feea2a00640 1 --2- 172.21.15.151:0/1547288944 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feea406e590 0x7feea40ceb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:32.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.969+0000 7feea2a00640 1 -- 172.21.15.151:0/1547288944 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feea415bda0 msgr2=0x7feea4159af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:32.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.969+0000 7feea2a00640 1 --2- 172.21.15.151:0/1547288944 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feea415bda0 0x7feea4159af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:32.972 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.969+0000 7feea2a00640 1 -- 172.21.15.151:0/1547288944 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feea41f7c30 con 0x7feea4070380 2024-09-16T15:46:32.972 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.969+0000 7feea2a00640 1 --2- 172.21.15.151:0/1547288944 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feea4070380 0x7feea40adaf0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fee90009950 tx=0x7fee90032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:32.972 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.969+0000 7fee9b400640 1 -- 172.21.15.151:0/1547288944 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee90040070 con 0x7feea4070380 2024-09-16T15:46:32.972 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.969+0000 7fee9b400640 1 -- 172.21.15.151:0/1547288944 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fee90005310 con 0x7feea4070380 2024-09-16T15:46:32.972 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.969+0000 7feea9000640 1 -- 172.21.15.151:0/1547288944 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feea41f7f10 con 0x7feea4070380 2024-09-16T15:46:32.972 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.970+0000 7fee9b400640 1 -- 172.21.15.151:0/1547288944 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee90005c90 con 0x7feea4070380 2024-09-16T15:46:32.972 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.970+0000 7feea9000640 1 -- 172.21.15.151:0/1547288944 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feea41f84d0 con 0x7feea4070380 2024-09-16T15:46:32.972 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.971+0000 7feea9000640 1 -- 172.21.15.151:0/1547288944 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feea40cc020 con 0x7feea4070380 2024-09-16T15:46:32.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.972+0000 7fee9b400640 1 -- 172.21.15.151:0/1547288944 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 23) ==== 102307+0+0 (secure 0 0 0) 0x7fee90039040 con 0x7feea4070380 2024-09-16T15:46:32.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.973+0000 7fee9b400640 1 --2- 172.21.15.151:0/1547288944 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fee840793c0 0x7fee8407b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:32.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.973+0000 7fee9b400640 1 -- 172.21.15.151:0/1547288944 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5795+0+0 (secure 0 0 0) 0x7fee900ba610 con 0x7feea4070380 2024-09-16T15:46:32.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.973+0000 7feea3400640 1 --2- 172.21.15.151:0/1547288944 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fee840793c0 0x7fee8407b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:32.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.974+0000 7feea3400640 1 --2- 172.21.15.151:0/1547288944 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fee840793c0 0x7fee8407b880 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fee94007fd0 tx=0x7fee94001370 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:32.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:32.977+0000 7fee9b400640 1 -- 172.21.15.151:0/1547288944 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fee90083e30 con 0x7feea4070380 2024-09-16T15:46:33.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.192+0000 7feea9000640 1 -- 172.21.15.151:0/1547288944 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7feea40ccca0 con 0x7fee840793c0 2024-09-16T15:46:33.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.211+0000 7fee9b400640 1 -- 172.21.15.151:0/1547288944 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7feea40ccca0 con 0x7fee840793c0 2024-09-16T15:46:33.212 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:46:33.212 INFO:teuthology.orchestra.run.smithi151.stdout:[ 2024-09-16T15:46:33.212 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:46:33.213 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ss1" 2024-09-16T15:46:33.213 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:46:33.213 INFO:teuthology.orchestra.run.smithi151.stdout:] 2024-09-16T15:46:33.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.214+0000 7feea9000640 1 -- 172.21.15.151:0/1547288944 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fee840793c0 msgr2=0x7fee8407b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:33.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.214+0000 7feea9000640 1 --2- 172.21.15.151:0/1547288944 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fee840793c0 0x7fee8407b880 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fee94007fd0 tx=0x7fee94001370 comp rx=0 tx=0).stop 2024-09-16T15:46:33.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.214+0000 7feea9000640 1 -- 172.21.15.151:0/1547288944 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feea4070380 msgr2=0x7feea40adaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:33.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.214+0000 7feea9000640 1 --2- 172.21.15.151:0/1547288944 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feea4070380 0x7feea40adaf0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fee90009950 tx=0x7fee90032c10 comp rx=0 tx=0).stop 2024-09-16T15:46:33.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.214+0000 7feea9000640 1 -- 172.21.15.151:0/1547288944 shutdown_connections 2024-09-16T15:46:33.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.214+0000 7feea9000640 1 --2- 172.21.15.151:0/1547288944 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fee840793c0 0x7fee8407b880 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:33.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.214+0000 7feea9000640 1 --2- 172.21.15.151:0/1547288944 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feea415bda0 0x7feea4159af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:33.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.214+0000 7feea9000640 1 --2- 172.21.15.151:0/1547288944 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feea4070380 0x7feea40adaf0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:33.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.214+0000 7feea9000640 1 --2- 172.21.15.151:0/1547288944 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feea406e590 0x7feea40ceb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:33.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.214+0000 7feea9000640 1 -- 172.21.15.151:0/1547288944 >> 172.21.15.151:0/1547288944 conn(0x7feea40cfa70 msgr2=0x7feea40af4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:33.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.214+0000 7feea9000640 1 -- 172.21.15.151:0/1547288944 shutdown_connections 2024-09-16T15:46:33.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.214+0000 7feea9000640 1 -- 172.21.15.151:0/1547288944 wait complete. 2024-09-16T15:46:33.231 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-09-16T15:46:33.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.395+0000 7f5623e00640 1 -- 172.21.15.151:0/577898893 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f562415f670 msgr2=0x7f56240c6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:33.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.395+0000 7f5623e00640 1 --2- 172.21.15.151:0/577898893 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f562415f670 0x7f56240c6430 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f5624075410 tx=0x7f561402a5f0 comp rx=0 tx=0).stop 2024-09-16T15:46:33.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.395+0000 7f5623e00640 1 -- 172.21.15.151:0/577898893 shutdown_connections 2024-09-16T15:46:33.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.395+0000 7f5623e00640 1 --2- 172.21.15.151:0/577898893 >> v2:172.21.15.167:3300/0 conn(0x7f562406e660 0x7f562406eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:33.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.395+0000 7f5623e00640 1 --2- 172.21.15.151:0/577898893 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f562415f670 0x7f56240c6430 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:33.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.395+0000 7f5623e00640 1 -- 172.21.15.151:0/577898893 >> 172.21.15.151:0/577898893 conn(0x7f56240cc490 msgr2=0x7f56240cc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:33.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.395+0000 7f5623e00640 1 -- 172.21.15.151:0/577898893 shutdown_connections 2024-09-16T15:46:33.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.395+0000 7f5623e00640 1 -- 172.21.15.151:0/577898893 wait complete. 2024-09-16T15:46:33.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.397+0000 7f5623e00640 1 Processor -- start 2024-09-16T15:46:33.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.397+0000 7f5623e00640 1 -- start start 2024-09-16T15:46:33.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.398+0000 7f5623e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f562406e660 0x7f56241f58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:33.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.398+0000 7f5623e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56240c5730 0x7f5624155400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:33.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.398+0000 7f5623e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f562415f670 0x7f56240b1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:33.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.398+0000 7f5623e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f56240ca4f0 con 0x7f56240c5730 2024-09-16T15:46:33.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.398+0000 7f5623e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f56240ca370 con 0x7f562415f670 2024-09-16T15:46:33.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.398+0000 7f5623e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f562406f000 con 0x7f562406e660 2024-09-16T15:46:33.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.398+0000 7f5622000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56240c5730 0x7f5624155400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:33.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.398+0000 7f5622000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56240c5730 0x7f5624155400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47776/0 (socket says 172.21.15.151:47776) 2024-09-16T15:46:33.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.398+0000 7f5622000640 1 -- 172.21.15.151:0/585319249 learned_addr learned my addr 172.21.15.151:0/585319249 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:46:33.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.398+0000 7f5622a00640 1 --2- 172.21.15.151:0/585319249 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f562406e660 0x7f56241f58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:33.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.398+0000 7f5622000640 1 -- 172.21.15.151:0/585319249 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f562406e660 msgr2=0x7f56241f58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:33.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.398+0000 7f5623400640 1 --2- 172.21.15.151:0/585319249 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f562415f670 0x7f56240b1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:33.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.398+0000 7f5622000640 1 --2- 172.21.15.151:0/585319249 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f562406e660 0x7f56241f58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:33.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.398+0000 7f5622000640 1 -- 172.21.15.151:0/585319249 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f562415f670 msgr2=0x7f56240b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:33.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.398+0000 7f5622000640 1 --2- 172.21.15.151:0/585319249 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f562415f670 0x7f56240b1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:33.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.398+0000 7f5622000640 1 -- 172.21.15.151:0/585319249 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56240b05e0 con 0x7f56240c5730 2024-09-16T15:46:33.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.398+0000 7f5622a00640 1 --2- 172.21.15.151:0/585319249 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f562406e660 0x7f56241f58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:46:33.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.398+0000 7f5623400640 1 --2- 172.21.15.151:0/585319249 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f562415f670 0x7f56240b1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:46:33.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.399+0000 7f5622000640 1 --2- 172.21.15.151:0/585319249 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56240c5730 0x7f5624155400 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f561000c500 tx=0x7f561000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:33.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.399+0000 7f561aa00640 1 -- 172.21.15.151:0/585319249 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56100052d0 con 0x7f56240c5730 2024-09-16T15:46:33.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.399+0000 7f561aa00640 1 -- 172.21.15.151:0/585319249 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5610005470 con 0x7f56240c5730 2024-09-16T15:46:33.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.399+0000 7f561aa00640 1 -- 172.21.15.151:0/585319249 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5610012870 con 0x7f56240c5730 2024-09-16T15:46:33.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.399+0000 7f5623e00640 1 -- 172.21.15.151:0/585319249 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56240b08c0 con 0x7f56240c5730 2024-09-16T15:46:33.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.399+0000 7f5623e00640 1 -- 172.21.15.151:0/585319249 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56240b0e80 con 0x7f56240c5730 2024-09-16T15:46:33.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.400+0000 7f5623e00640 1 -- 172.21.15.151:0/585319249 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56241f4740 con 0x7f56240c5730 2024-09-16T15:46:33.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.401+0000 7f561aa00640 1 -- 172.21.15.151:0/585319249 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 23) ==== 102307+0+0 (secure 0 0 0) 0x7f56100035f0 con 0x7f56240c5730 2024-09-16T15:46:33.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.402+0000 7f561aa00640 1 --2- 172.21.15.151:0/585319249 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5604079390 0x7f560407b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:33.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.402+0000 7f561aa00640 1 -- 172.21.15.151:0/585319249 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5795+0+0 (secure 0 0 0) 0x7f56100990f0 con 0x7f56240c5730 2024-09-16T15:46:33.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.403+0000 7f5622a00640 1 --2- 172.21.15.151:0/585319249 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5604079390 0x7f560407b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:33.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.403+0000 7f5622a00640 1 --2- 172.21.15.151:0/585319249 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5604079390 0x7f560407b850 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5614033000 tx=0x7f5614004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:33.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.406+0000 7f561aa00640 1 -- 172.21.15.151:0/585319249 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f56241f4740 con 0x7f56240c5730 2024-09-16T15:46:33.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.624+0000 7f5623e00640 1 -- 172.21.15.151:0/585319249 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f562406eac0 con 0x7f5604079390 2024-09-16T15:46:33.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.654+0000 7f561aa00640 1 -- 172.21.15.151:0/585319249 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f562406eac0 con 0x7f5604079390 2024-09-16T15:46:33.655 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:46:33.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.657+0000 7f5623e00640 1 -- 172.21.15.151:0/585319249 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5604079390 msgr2=0x7f560407b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:33.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.657+0000 7f5623e00640 1 --2- 172.21.15.151:0/585319249 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5604079390 0x7f560407b850 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5614033000 tx=0x7f5614004730 comp rx=0 tx=0).stop 2024-09-16T15:46:33.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.657+0000 7f5623e00640 1 -- 172.21.15.151:0/585319249 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56240c5730 msgr2=0x7f5624155400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:33.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.657+0000 7f5623e00640 1 --2- 172.21.15.151:0/585319249 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56240c5730 0x7f5624155400 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f561000c500 tx=0x7f561000c9c0 comp rx=0 tx=0).stop 2024-09-16T15:46:33.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.657+0000 7f5623e00640 1 -- 172.21.15.151:0/585319249 shutdown_connections 2024-09-16T15:46:33.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.657+0000 7f5623e00640 1 --2- 172.21.15.151:0/585319249 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5604079390 0x7f560407b850 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:33.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.657+0000 7f5623e00640 1 --2- 172.21.15.151:0/585319249 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f562415f670 0x7f56240b1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:33.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.657+0000 7f5623e00640 1 --2- 172.21.15.151:0/585319249 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56240c5730 0x7f5624155400 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:33.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.657+0000 7f5623e00640 1 --2- 172.21.15.151:0/585319249 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f562406e660 0x7f56241f58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:33.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.657+0000 7f5623e00640 1 -- 172.21.15.151:0/585319249 >> 172.21.15.151:0/585319249 conn(0x7f56240cc490 msgr2=0x7f56240740d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:33.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.657+0000 7f5623e00640 1 -- 172.21.15.151:0/585319249 shutdown_connections 2024-09-16T15:46:33.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.657+0000 7f5623e00640 1 -- 172.21.15.151:0/585319249 wait complete. 2024-09-16T15:46:33.675 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-09-16T15:46:33.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.836+0000 7fa2b3e00640 1 -- 172.21.15.151:0/1270780837 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa2b40cd5c0 msgr2=0x7fa2b40ca250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:33.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.836+0000 7fa2b3e00640 1 --2- 172.21.15.151:0/1270780837 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa2b40cd5c0 0x7fa2b40ca250 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0009860 tx=0x7fa2a002a5d0 comp rx=0 tx=0).stop 2024-09-16T15:46:33.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.836+0000 7fa2b3e00640 1 -- 172.21.15.151:0/1270780837 shutdown_connections 2024-09-16T15:46:33.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.836+0000 7fa2b3e00640 1 --2- 172.21.15.151:0/1270780837 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa2b40cd5c0 0x7fa2b40ca250 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:33.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.836+0000 7fa2b3e00640 1 --2- 172.21.15.151:0/1270780837 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa2b40cf5a0 0x7fa2b406f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:33.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.836+0000 7fa2b3e00640 1 -- 172.21.15.151:0/1270780837 >> 172.21.15.151:0/1270780837 conn(0x7fa2b40cf9f0 msgr2=0x7fa2b40cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:33.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.837+0000 7fa2b3e00640 1 -- 172.21.15.151:0/1270780837 shutdown_connections 2024-09-16T15:46:33.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.837+0000 7fa2b3e00640 1 -- 172.21.15.151:0/1270780837 wait complete. 2024-09-16T15:46:33.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.838+0000 7fa2b3e00640 1 Processor -- start 2024-09-16T15:46:33.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.838+0000 7fa2b3e00640 1 -- start start 2024-09-16T15:46:33.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.839+0000 7fa2b3e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa2b40aeb30 0x7fa2b4157b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:33.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.839+0000 7fa2b3e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa2b40cd5c0 0x7fa2b40c5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:33.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.839+0000 7fa2b3e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2b40cf5a0 0x7fa2b40cc890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:33.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.839+0000 7fa2b3e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b40caba0 con 0x7fa2b40cd5c0 2024-09-16T15:46:33.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.839+0000 7fa2b3e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b40caa20 con 0x7fa2b40cf5a0 2024-09-16T15:46:33.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.839+0000 7fa2b3e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fa2b406e5a0 con 0x7fa2b40aeb30 2024-09-16T15:46:33.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.839+0000 7fa2b2000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa2b40cd5c0 0x7fa2b40c5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:33.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.840+0000 7fa2b2000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa2b40cd5c0 0x7fa2b40c5f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47798/0 (socket says 172.21.15.151:47798) 2024-09-16T15:46:33.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.840+0000 7fa2b2000640 1 -- 172.21.15.151:0/1729324813 learned_addr learned my addr 172.21.15.151:0/1729324813 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:46:33.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.840+0000 7fa2b2a00640 1 --2- 172.21.15.151:0/1729324813 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa2b40aeb30 0x7fa2b4157b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:33.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.840+0000 7fa2b3400640 1 --2- 172.21.15.151:0/1729324813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2b40cf5a0 0x7fa2b40cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:33.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.840+0000 7fa2b2000640 1 -- 172.21.15.151:0/1729324813 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa2b40aeb30 msgr2=0x7fa2b4157b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:33.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.840+0000 7fa2b2000640 1 --2- 172.21.15.151:0/1729324813 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa2b40aeb30 0x7fa2b4157b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:33.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.840+0000 7fa2b2000640 1 -- 172.21.15.151:0/1729324813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2b40cf5a0 msgr2=0x7fa2b40cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:33.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.840+0000 7fa2b2000640 1 --2- 172.21.15.151:0/1729324813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2b40cf5a0 0x7fa2b40cc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:33.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.840+0000 7fa2b2000640 1 -- 172.21.15.151:0/1729324813 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2b41f5070 con 0x7fa2b40cd5c0 2024-09-16T15:46:33.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.840+0000 7fa2b3400640 1 --2- 172.21.15.151:0/1729324813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2b40cf5a0 0x7fa2b40cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:46:33.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.840+0000 7fa2b2000640 1 --2- 172.21.15.151:0/1729324813 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa2b40cd5c0 0x7fa2b40c5f10 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0009980 tx=0x7fa2a0032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:33.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.840+0000 7fa2aaa00640 1 -- 172.21.15.151:0/1729324813 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0040070 con 0x7fa2b40cd5c0 2024-09-16T15:46:33.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.841+0000 7fa2aaa00640 1 -- 172.21.15.151:0/1729324813 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa2a0005310 con 0x7fa2b40cd5c0 2024-09-16T15:46:33.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.841+0000 7fa2aaa00640 1 -- 172.21.15.151:0/1729324813 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0005c90 con 0x7fa2b40cd5c0 2024-09-16T15:46:33.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.841+0000 7fa2b3e00640 1 -- 172.21.15.151:0/1729324813 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2b41f5350 con 0x7fa2b40cd5c0 2024-09-16T15:46:33.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.841+0000 7fa2b3e00640 1 -- 172.21.15.151:0/1729324813 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2b41f4d60 con 0x7fa2b40cd5c0 2024-09-16T15:46:33.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.842+0000 7fa2b3e00640 1 -- 172.21.15.151:0/1729324813 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2b406ead0 con 0x7fa2b40cd5c0 2024-09-16T15:46:33.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.843+0000 7fa2aaa00640 1 -- 172.21.15.151:0/1729324813 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 23) ==== 102307+0+0 (secure 0 0 0) 0x7fa2a0039040 con 0x7fa2b40cd5c0 2024-09-16T15:46:33.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.844+0000 7fa2aaa00640 1 --2- 172.21.15.151:0/1729324813 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa294079390 0x7fa29407b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:33.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.844+0000 7fa2aaa00640 1 -- 172.21.15.151:0/1729324813 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5795+0+0 (secure 0 0 0) 0x7fa2a00ba3d0 con 0x7fa2b40cd5c0 2024-09-16T15:46:33.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.844+0000 7fa2b2a00640 1 --2- 172.21.15.151:0/1729324813 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa294079390 0x7fa29407b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:33.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.845+0000 7fa2b2a00640 1 --2- 172.21.15.151:0/1729324813 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa294079390 0x7fa29407b850 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa2a400aba0 tx=0x7fa2a4010040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:33.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:33.848+0000 7fa2aaa00640 1 -- 172.21.15.151:0/1729324813 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa2a0084490 con 0x7fa2b40cd5c0 2024-09-16T15:46:34.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:34.055+0000 7fa2b3e00640 1 -- 172.21.15.151:0/1729324813 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fa2b40d01e0 con 0x7fa294079390 2024-09-16T15:46:34.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:34.077+0000 7fa2aaa00640 1 -- 172.21.15.151:0/1729324813 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa2b40d01e0 con 0x7fa294079390 2024-09-16T15:46:34.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:34.080+0000 7fa2b3e00640 1 -- 172.21.15.151:0/1729324813 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa294079390 msgr2=0x7fa29407b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:34.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:34.080+0000 7fa2b3e00640 1 --2- 172.21.15.151:0/1729324813 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa294079390 0x7fa29407b850 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa2a400aba0 tx=0x7fa2a4010040 comp rx=0 tx=0).stop 2024-09-16T15:46:34.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:34.080+0000 7fa2b3e00640 1 -- 172.21.15.151:0/1729324813 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa2b40cd5c0 msgr2=0x7fa2b40c5f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:34.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:34.080+0000 7fa2b3e00640 1 --2- 172.21.15.151:0/1729324813 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa2b40cd5c0 0x7fa2b40c5f10 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0009980 tx=0x7fa2a0032c10 comp rx=0 tx=0).stop 2024-09-16T15:46:34.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:34.080+0000 7fa2b3e00640 1 -- 172.21.15.151:0/1729324813 shutdown_connections 2024-09-16T15:46:34.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:34.080+0000 7fa2b3e00640 1 --2- 172.21.15.151:0/1729324813 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa294079390 0x7fa29407b850 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:34.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:34.080+0000 7fa2b3e00640 1 --2- 172.21.15.151:0/1729324813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2b40cf5a0 0x7fa2b40cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:34.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:34.080+0000 7fa2b3e00640 1 --2- 172.21.15.151:0/1729324813 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa2b40cd5c0 0x7fa2b40c5f10 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:34.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:34.080+0000 7fa2b3e00640 1 --2- 172.21.15.151:0/1729324813 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa2b40aeb30 0x7fa2b4157b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:34.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:34.080+0000 7fa2b3e00640 1 -- 172.21.15.151:0/1729324813 >> 172.21.15.151:0/1729324813 conn(0x7fa2b40cf9f0 msgr2=0x7fa2b40aff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:34.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:34.080+0000 7fa2b3e00640 1 -- 172.21.15.151:0/1729324813 shutdown_connections 2024-09-16T15:46:34.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:34.080+0000 7fa2b3e00640 1 -- 172.21.15.151:0/1729324813 wait complete. 2024-09-16T15:46:34.083 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:46:34.082+0000 7f48f1c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:46:34.083 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:46:34.082+0000 7f48f1c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:46:34.083 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:46:34.082+0000 7f48f1c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:46:34.083 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:46:34.082+0000 7f48f1c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:46:34.084 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:46:34.082+0000 7f48f1c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:46:34.099 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs ss1clone1 2024-09-16T15:46:34.100 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:46:34.098+0000 7f48f2600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:46:34.100 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:46:34.098+0000 7f48f2600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:46:34.100 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:46:34.098+0000 7f48f2600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:46:34.100 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:46:34.099+0000 7f48f2600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:46:34.100 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:46:34.099+0000 7f48f2600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:46:34.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:34.263+0000 7fe9b5200640 1 -- 172.21.15.151:0/3227467691 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9b00cf350 msgr2=0x7fe9b00b3490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:34.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:34.263+0000 7fe9b5200640 1 --2- 172.21.15.151:0/3227467691 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9b00cf350 0x7fe9b00b3490 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fe9a8005fd0 tx=0x7fe9a802a780 comp rx=0 tx=0).stop 2024-09-16T15:46:34.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:34.263+0000 7fe9b5200640 1 -- 172.21.15.151:0/3227467691 shutdown_connections 2024-09-16T15:46:34.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:34.263+0000 7fe9b5200640 1 --2- 172.21.15.151:0/3227467691 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9b00cf350 0x7fe9b00b3490 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:34.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:34.263+0000 7fe9b5200640 1 --2- 172.21.15.151:0/3227467691 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe9b015e140 0x7fe9b00d2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:34.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:34.263+0000 7fe9b5200640 1 -- 172.21.15.151:0/3227467691 >> 172.21.15.151:0/3227467691 conn(0x7fe9b00d3d30 msgr2=0x7fe9b00d4140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:34.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:34.263+0000 7fe9b5200640 1 -- 172.21.15.151:0/3227467691 shutdown_connections 2024-09-16T15:46:34.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:34.263+0000 7fe9b5200640 1 -- 172.21.15.151:0/3227467691 wait complete. 2024-09-16T15:46:34.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:34.265+0000 7fe9b5200640 1 Processor -- start 2024-09-16T15:46:34.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:34.265+0000 7fe9b5200640 1 -- start start 2024-09-16T15:46:34.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:34.266+0000 7fe9b5200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9b00cf350 0x7fe9b00c8e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:34.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:34.266+0000 7fe9b5200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9b00d0720 0x7fe9b01f5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:34.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:34.266+0000 7fe9b5200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe9b00c7860 0x7fe9b014f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:34.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:34.266+0000 7fe9b5200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b00b39d0 con 0x7fe9b00d0720 2024-09-16T15:46:34.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:34.266+0000 7fe9b5200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b00cf9f0 con 0x7fe9b00cf350 2024-09-16T15:46:34.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:34.266+0000 7fe9b5200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fe9b00b3b50 con 0x7fe9b00c7860 2024-09-16T15:46:34.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:34.266+0000 7fe9a7e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9b00d0720 0x7fe9b01f5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:34.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:34.266+0000 7fe9a7e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9b00d0720 0x7fe9b01f5010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47814/0 (socket says 172.21.15.151:47814) 2024-09-16T15:46:34.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:34.266+0000 7fe9a7e00640 1 -- 172.21.15.151:0/425277167 learned_addr learned my addr 172.21.15.151:0/425277167 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:46:34.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:34.266+0000 7fe9afe00640 1 --2- 172.21.15.151:0/425277167 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe9b00c7860 0x7fe9b014f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:34.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:34.266+0000 7fe9af400640 1 --2- 172.21.15.151:0/425277167 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9b00cf350 0x7fe9b00c8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:34.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:34.267+0000 7fe9a7e00640 1 -- 172.21.15.151:0/425277167 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe9b00c7860 msgr2=0x7fe9b014f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:34.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:34.267+0000 7fe9a7e00640 1 --2- 172.21.15.151:0/425277167 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe9b00c7860 0x7fe9b014f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:34.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:34.267+0000 7fe9a7e00640 1 -- 172.21.15.151:0/425277167 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9b00cf350 msgr2=0x7fe9b00c8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:34.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:34.267+0000 7fe9a7e00640 1 --2- 172.21.15.151:0/425277167 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9b00cf350 0x7fe9b00c8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:34.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:34.267+0000 7fe9a7e00640 1 -- 172.21.15.151:0/425277167 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9b01f5550 con 0x7fe9b00d0720 2024-09-16T15:46:34.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:34.267+0000 7fe9a7e00640 1 --2- 172.21.15.151:0/425277167 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9b00d0720 0x7fe9b01f5010 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fe99800d750 tx=0x7fe99800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:34.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:34.267+0000 7fe9a7400640 1 -- 172.21.15.151:0/425277167 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe998017070 con 0x7fe9b00d0720 2024-09-16T15:46:34.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:34.267+0000 7fe9a7400640 1 -- 172.21.15.151:0/425277167 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe998004310 con 0x7fe9b00d0720 2024-09-16T15:46:34.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:34.267+0000 7fe9a7400640 1 -- 172.21.15.151:0/425277167 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99800de60 con 0x7fe9b00d0720 2024-09-16T15:46:34.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:34.267+0000 7fe9b5200640 1 -- 172.21.15.151:0/425277167 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9b00c7d70 con 0x7fe9b00d0720 2024-09-16T15:46:34.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:34.268+0000 7fe9b5200640 1 -- 172.21.15.151:0/425277167 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9b00c8330 con 0x7fe9b00d0720 2024-09-16T15:46:34.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:34.269+0000 7fe9b5200640 1 -- 172.21.15.151:0/425277167 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9b00d4830 con 0x7fe9b00d0720 2024-09-16T15:46:34.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:34.270+0000 7fe9a7400640 1 -- 172.21.15.151:0/425277167 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 23) ==== 102307+0+0 (secure 0 0 0) 0x7fe998009c30 con 0x7fe9b00d0720 2024-09-16T15:46:34.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:34.271+0000 7fe9a7400640 1 --2- 172.21.15.151:0/425277167 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe98c079390 0x7fe98c07b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:34.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:34.271+0000 7fe9a7400640 1 -- 172.21.15.151:0/425277167 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5795+0+0 (secure 0 0 0) 0x7fe9980997e0 con 0x7fe9b00d0720 2024-09-16T15:46:34.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:34.272+0000 7fe9af400640 1 --2- 172.21.15.151:0/425277167 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe98c079390 0x7fe98c07b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:34.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:34.272+0000 7fe9af400640 1 --2- 172.21.15.151:0/425277167 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe98c079390 0x7fe98c07b850 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe9a00017c0 tx=0x7fe9a0007e90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:34.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:34.276+0000 7fe9a7400640 1 -- 172.21.15.151:0/425277167 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe9b00d4830 con 0x7fe9b00d0720 2024-09-16T15:46:34.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:34.510+0000 7fe9b5200640 1 -- 172.21.15.151:0/425277167 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7fe9b00b2a60 con 0x7fe98c079390 2024-09-16T15:46:35.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:35.988+0000 7fe9a7400640 1 -- 172.21.15.151:0/425277167 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe9b00b2a60 con 0x7fe98c079390 2024-09-16T15:46:35.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:35.990+0000 7fe9b5200640 1 -- 172.21.15.151:0/425277167 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe98c079390 msgr2=0x7fe98c07b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:35.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:35.990+0000 7fe9b5200640 1 --2- 172.21.15.151:0/425277167 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe98c079390 0x7fe98c07b850 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe9a00017c0 tx=0x7fe9a0007e90 comp rx=0 tx=0).stop 2024-09-16T15:46:35.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:35.990+0000 7fe9b5200640 1 -- 172.21.15.151:0/425277167 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9b00d0720 msgr2=0x7fe9b01f5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:35.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:35.990+0000 7fe9b5200640 1 --2- 172.21.15.151:0/425277167 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9b00d0720 0x7fe9b01f5010 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fe99800d750 tx=0x7fe99800dc10 comp rx=0 tx=0).stop 2024-09-16T15:46:35.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:35.990+0000 7fe9b5200640 1 -- 172.21.15.151:0/425277167 shutdown_connections 2024-09-16T15:46:35.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:35.990+0000 7fe9b5200640 1 --2- 172.21.15.151:0/425277167 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe98c079390 0x7fe98c07b850 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:35.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:35.990+0000 7fe9b5200640 1 --2- 172.21.15.151:0/425277167 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe9b00c7860 0x7fe9b014f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:35.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:35.990+0000 7fe9b5200640 1 --2- 172.21.15.151:0/425277167 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9b00d0720 0x7fe9b01f5010 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:35.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:35.990+0000 7fe9b5200640 1 --2- 172.21.15.151:0/425277167 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9b00cf350 0x7fe9b00c8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:35.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:35.991+0000 7fe9b5200640 1 -- 172.21.15.151:0/425277167 >> 172.21.15.151:0/425277167 conn(0x7fe9b00d3d30 msgr2=0x7fe9b0075890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:35.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:35.991+0000 7fe9b5200640 1 -- 172.21.15.151:0/425277167 shutdown_connections 2024-09-16T15:46:35.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:35.991+0000 7fe9b5200640 1 -- 172.21.15.151:0/425277167 wait complete. 2024-09-16T15:46:36.009 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-16T15:46:36.090 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:36 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:46:36.096 INFO:teuthology.orchestra.run.smithi151.stdout:2 2024-09-16T15:46:36.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:46:36.106 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-16T15:46:36.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:36.269+0000 7f6f67e00640 1 -- 172.21.15.151:0/3620037158 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6f680cc650 msgr2=0x7f6f68074220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:36.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:36.269+0000 7f6f67e00640 1 --2- 172.21.15.151:0/3620037158 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6f680cc650 0x7f6f68074220 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f6f680cccc0 tx=0x7f6f5402a4d0 comp rx=0 tx=0).stop 2024-09-16T15:46:36.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:36.269+0000 7f6f67e00640 1 -- 172.21.15.151:0/3620037158 shutdown_connections 2024-09-16T15:46:36.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:36.269+0000 7f6f67e00640 1 --2- 172.21.15.151:0/3620037158 >> v2:172.21.15.167:3300/0 conn(0x7f6f680d0db0 0x7f6f68074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:36.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:36.269+0000 7f6f67e00640 1 --2- 172.21.15.151:0/3620037158 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6f680cc650 0x7f6f68074220 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:36.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:36.269+0000 7f6f67e00640 1 --2- 172.21.15.151:0/3620037158 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f681612c0 0x7f6f680af990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:36.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:36.269+0000 7f6f67e00640 1 -- 172.21.15.151:0/3620037158 >> 172.21.15.151:0/3620037158 conn(0x7f6f680d0290 msgr2=0x7f6f680d06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:36.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:36.270+0000 7f6f67e00640 1 -- 172.21.15.151:0/3620037158 shutdown_connections 2024-09-16T15:46:36.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:36.270+0000 7f6f67e00640 1 -- 172.21.15.151:0/3620037158 wait complete. 2024-09-16T15:46:36.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:36.271+0000 7f6f67e00640 1 Processor -- start 2024-09-16T15:46:36.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:36.271+0000 7f6f67e00640 1 -- start start 2024-09-16T15:46:36.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:36.272+0000 7f6f67e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f680cc650 0x7f6f6809da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:36.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:36.272+0000 7f6f67e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6f680d0db0 0x7f6f681f5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:36.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:36.272+0000 7f6f67e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f681612c0 0x7f6f681570b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:36.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:36.272+0000 7f6f67e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6f680cb2f0 con 0x7f6f680d0db0 2024-09-16T15:46:36.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:36.272+0000 7f6f67e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6f68161930 con 0x7f6f681612c0 2024-09-16T15:46:36.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:36.272+0000 7f6f67e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f6f680cb470 con 0x7f6f680cc650 2024-09-16T15:46:36.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:36.272+0000 7f6f66000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6f680d0db0 0x7f6f681f5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:36.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:36.272+0000 7f6f66000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6f680d0db0 0x7f6f681f5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47832/0 (socket says 172.21.15.151:47832) 2024-09-16T15:46:36.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:36.272+0000 7f6f66000640 1 -- 172.21.15.151:0/1181347722 learned_addr learned my addr 172.21.15.151:0/1181347722 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:46:36.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:36.273+0000 7f6f66a00640 1 --2- 172.21.15.151:0/1181347722 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f680cc650 0x7f6f6809da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:36.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:36.273+0000 7f6f67400640 1 --2- 172.21.15.151:0/1181347722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f681612c0 0x7f6f681570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:36.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:36.273+0000 7f6f66000640 1 -- 172.21.15.151:0/1181347722 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f680cc650 msgr2=0x7f6f6809da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:36.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:36.273+0000 7f6f66000640 1 --2- 172.21.15.151:0/1181347722 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f680cc650 0x7f6f6809da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:36.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:36.273+0000 7f6f66000640 1 -- 172.21.15.151:0/1181347722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f681612c0 msgr2=0x7f6f681570b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:36.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:36.273+0000 7f6f66000640 1 --2- 172.21.15.151:0/1181347722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f681612c0 0x7f6f681570b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:36.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:36.273+0000 7f6f66000640 1 -- 172.21.15.151:0/1181347722 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f6809e070 con 0x7f6f680d0db0 2024-09-16T15:46:36.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:36.273+0000 7f6f66000640 1 --2- 172.21.15.151:0/1181347722 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6f680d0db0 0x7f6f681f5d50 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f6f54009850 tx=0x7f6f54003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:36.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:36.274+0000 7f6f5ea00640 1 -- 172.21.15.151:0/1181347722 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5403f070 con 0x7f6f680d0db0 2024-09-16T15:46:36.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:36.274+0000 7f6f5ea00640 1 -- 172.21.15.151:0/1181347722 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6f540047a0 con 0x7f6f680d0db0 2024-09-16T15:46:36.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:36.274+0000 7f6f5ea00640 1 -- 172.21.15.151:0/1181347722 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f54035030 con 0x7f6f680d0db0 2024-09-16T15:46:36.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:36.274+0000 7f6f67e00640 1 -- 172.21.15.151:0/1181347722 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f680c96d0 con 0x7f6f680d0db0 2024-09-16T15:46:36.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:36.274+0000 7f6f67e00640 1 -- 172.21.15.151:0/1181347722 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f681575f0 con 0x7f6f680d0db0 2024-09-16T15:46:36.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:36.275+0000 7f6f67e00640 1 -- 172.21.15.151:0/1181347722 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f680c94c0 con 0x7f6f680d0db0 2024-09-16T15:46:36.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:36.276+0000 7f6f5ea00640 1 -- 172.21.15.151:0/1181347722 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 24) ==== 102409+0+0 (secure 0 0 0) 0x7f6f54042070 con 0x7f6f680d0db0 2024-09-16T15:46:36.278 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:36.277+0000 7f6f5ea00640 1 --2- 172.21.15.151:0/1181347722 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6f480794b0 0x7f6f4807b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:36.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:36.278+0000 7f6f5ea00640 1 -- 172.21.15.151:0/1181347722 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5795+0+0 (secure 0 0 0) 0x7f6f540ba300 con 0x7f6f680d0db0 2024-09-16T15:46:36.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:36.278+0000 7f6f66a00640 1 --2- 172.21.15.151:0/1181347722 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6f480794b0 0x7f6f4807b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:36.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:36.278+0000 7f6f66a00640 1 --2- 172.21.15.151:0/1181347722 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6f480794b0 0x7f6f4807b970 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6f58004630 tx=0x7f6f5800b2e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:36.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:36.282+0000 7f6f5ea00640 1 -- 172.21.15.151:0/1181347722 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6f54083af0 con 0x7f6f680d0db0 2024-09-16T15:46:36.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:36.471+0000 7f6f67e00640 1 -- 172.21.15.151:0/1181347722 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f6f680af990 con 0x7f6f680d0db0 2024-09-16T15:46:36.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:36.472+0000 7f6f5ea00640 1 -- 172.21.15.151:0/1181347722 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v35)=0 v35) ==== 131+0+0 (secure 0 0 0) 0x7f6f54088520 con 0x7f6f680d0db0 2024-09-16T15:46:36.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:36.474+0000 7f6f67e00640 1 -- 172.21.15.151:0/1181347722 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6f480794b0 msgr2=0x7f6f4807b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:36.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:36.474+0000 7f6f67e00640 1 --2- 172.21.15.151:0/1181347722 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6f480794b0 0x7f6f4807b970 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6f58004630 tx=0x7f6f5800b2e0 comp rx=0 tx=0).stop 2024-09-16T15:46:36.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:36.474+0000 7f6f67e00640 1 -- 172.21.15.151:0/1181347722 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6f680d0db0 msgr2=0x7f6f681f5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:36.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:36.474+0000 7f6f67e00640 1 --2- 172.21.15.151:0/1181347722 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6f680d0db0 0x7f6f681f5d50 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f6f54009850 tx=0x7f6f54003910 comp rx=0 tx=0).stop 2024-09-16T15:46:36.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:36.474+0000 7f6f67e00640 1 -- 172.21.15.151:0/1181347722 shutdown_connections 2024-09-16T15:46:36.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:36.474+0000 7f6f67e00640 1 --2- 172.21.15.151:0/1181347722 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6f480794b0 0x7f6f4807b970 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:36.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:36.474+0000 7f6f67e00640 1 --2- 172.21.15.151:0/1181347722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f681612c0 0x7f6f681570b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:36.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:36.474+0000 7f6f67e00640 1 --2- 172.21.15.151:0/1181347722 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6f680d0db0 0x7f6f681f5d50 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:36.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:36.474+0000 7f6f67e00640 1 --2- 172.21.15.151:0/1181347722 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6f680cc650 0x7f6f6809da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:36.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:36.474+0000 7f6f67e00640 1 -- 172.21.15.151:0/1181347722 >> 172.21.15.151:0/1181347722 conn(0x7f6f680d0290 msgr2=0x7f6f680b0bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:36.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:36.474+0000 7f6f67e00640 1 -- 172.21.15.151:0/1181347722 shutdown_connections 2024-09-16T15:46:36.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:36.474+0000 7f6f67e00640 1 -- 172.21.15.151:0/1181347722 wait complete. 2024-09-16T15:46:36.491 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-16T15:46:36.491 DEBUG:teuthology.orchestra.run.smithi151:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:46:36.582 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:46:36.675 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-16T15:46:36.675 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:46:36.675 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:46:36.675 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:46:36.712 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:46:36.712 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:46:36.787 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-16T15:46:37.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:37.091+0000 7f6e7f200640 1 -- 172.21.15.151:0/888127878 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6e8015dee0 msgr2=0x7f6e8006ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:37.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:37.091+0000 7f6e7f200640 1 --2- 172.21.15.151:0/888127878 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6e8015dee0 0x7f6e8006ddd0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f6e78005fd0 tx=0x7f6e7802a940 comp rx=0 tx=0).stop 2024-09-16T15:46:37.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:37.091+0000 7f6e7f200640 1 -- 172.21.15.151:0/888127878 shutdown_connections 2024-09-16T15:46:37.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:37.091+0000 7f6e7f200640 1 --2- 172.21.15.151:0/888127878 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6e8015dee0 0x7f6e8006ddd0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:37.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:37.091+0000 7f6e7f200640 1 -- 172.21.15.151:0/888127878 >> 172.21.15.151:0/888127878 conn(0x7f6e800d1c40 msgr2=0x7f6e800d2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:37.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:37.091+0000 7f6e7f200640 1 -- 172.21.15.151:0/888127878 shutdown_connections 2024-09-16T15:46:37.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:37.091+0000 7f6e7f200640 1 -- 172.21.15.151:0/888127878 wait complete. 2024-09-16T15:46:37.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:37.093+0000 7f6e7f200640 1 Processor -- start 2024-09-16T15:46:37.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:37.094+0000 7f6e7f200640 1 -- start start 2024-09-16T15:46:37.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:37.095+0000 7f6e7f200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e800b54a0 0x7f6e801f4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:37.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:37.095+0000 7f6e7f200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e8015dee0 0x7f6e800cede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:37.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:37.095+0000 7f6e7f200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6e8015e2c0 0x7f6e8009f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:37.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:37.095+0000 7f6e7f200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6e800b74e0 con 0x7f6e8015e2c0 2024-09-16T15:46:37.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:37.095+0000 7f6e7f200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6e800b7360 con 0x7f6e800b54a0 2024-09-16T15:46:37.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:37.095+0000 7f6e7f200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f6e800b7660 con 0x7f6e8015dee0 2024-09-16T15:46:37.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:37.096+0000 7f6e7e800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6e8015e2c0 0x7f6e8009f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:37.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:37.096+0000 7f6e7d400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e8015dee0 0x7f6e800cede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:37.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:37.096+0000 7f6e7d400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e8015dee0 0x7f6e800cede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.151:44160/0 (socket says 172.21.15.151:44160) 2024-09-16T15:46:37.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:37.096+0000 7f6e7d400640 1 -- 172.21.15.151:0/1922793560 learned_addr learned my addr 172.21.15.151:0/1922793560 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:46:37.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:37.096+0000 7f6e7de00640 1 --2- 172.21.15.151:0/1922793560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e800b54a0 0x7f6e801f4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:37.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:37.097+0000 7f6e7d400640 1 -- 172.21.15.151:0/1922793560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e800b54a0 msgr2=0x7f6e801f4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:37.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:37.097+0000 7f6e7d400640 1 --2- 172.21.15.151:0/1922793560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e800b54a0 0x7f6e801f4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:37.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:37.097+0000 7f6e7d400640 1 -- 172.21.15.151:0/1922793560 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6e8015e2c0 msgr2=0x7f6e8009f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:37.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:37.097+0000 7f6e7d400640 1 --2- 172.21.15.151:0/1922793560 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6e8015e2c0 0x7f6e8009f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:37.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:37.097+0000 7f6e7d400640 1 -- 172.21.15.151:0/1922793560 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e801f5ce0 con 0x7f6e8015dee0 2024-09-16T15:46:37.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:37.097+0000 7f6e7d400640 1 --2- 172.21.15.151:0/1922793560 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e8015dee0 0x7f6e800cede0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6e7000c400 tx=0x7f6e7000c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:37.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:37.098+0000 7f6e76000640 1 -- 172.21.15.151:0/1922793560 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e70011850 con 0x7f6e8015dee0 2024-09-16T15:46:37.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:37.098+0000 7f6e76000640 1 -- 172.21.15.151:0/1922793560 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6e70017070 con 0x7f6e8015dee0 2024-09-16T15:46:37.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:37.098+0000 7f6e76000640 1 -- 172.21.15.151:0/1922793560 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e70011d60 con 0x7f6e8015dee0 2024-09-16T15:46:37.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:37.099+0000 7f6e7f200640 1 -- 172.21.15.151:0/1922793560 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e801f50d0 con 0x7f6e8015dee0 2024-09-16T15:46:37.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:37.099+0000 7f6e7f200640 1 -- 172.21.15.151:0/1922793560 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e801f5690 con 0x7f6e8015dee0 2024-09-16T15:46:37.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:37.103+0000 7f6e7f200640 1 -- 172.21.15.151:0/1922793560 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e8006ddd0 con 0x7f6e8015dee0 2024-09-16T15:46:37.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:37.108+0000 7f6e76000640 1 -- 172.21.15.151:0/1922793560 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 24) ==== 102409+0+0 (secure 0 0 0) 0x7f6e7001c460 con 0x7f6e8015dee0 2024-09-16T15:46:37.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:37.110+0000 7f6e76000640 1 --2- 172.21.15.151:0/1922793560 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6e5c079580 0x7f6e5c07ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:37.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:37.110+0000 7f6e76000640 1 -- 172.21.15.151:0/1922793560 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(64..64 src has 1..64) ==== 5795+0+0 (secure 0 0 0) 0x7f6e70098f90 con 0x7f6e8015dee0 2024-09-16T15:46:37.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:37.111+0000 7f6e7de00640 1 --2- 172.21.15.151:0/1922793560 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6e5c079580 0x7f6e5c07ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:37.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:37.113+0000 7f6e7de00640 1 --2- 172.21.15.151:0/1922793560 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6e5c079580 0x7f6e5c07ba40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6e78034000 tx=0x7f6e78004ab0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:37.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:37.118+0000 7f6e76000640 1 -- 172.21.15.151:0/1922793560 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6e70062a00 con 0x7f6e8015dee0 2024-09-16T15:46:37.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:37.489+0000 7f6e7f200640 1 -- 172.21.15.151:0/1922793560 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f6e800b45e0 con 0x7f6e8015dee0 2024-09-16T15:46:37.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:37.489+0000 7f6e76000640 1 -- 172.21.15.151:0/1922793560 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v65) ==== 80+0+0 (secure 0 0 0) 0x7f6e70067430 con 0x7f6e8015dee0 2024-09-16T15:46:37.491 INFO:teuthology.orchestra.run.smithi151.stderr:listed 0 entries 2024-09-16T15:46:37.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:37.494+0000 7f6e7f200640 1 -- 172.21.15.151:0/1922793560 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6e5c079580 msgr2=0x7f6e5c07ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:37.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:37.494+0000 7f6e7f200640 1 --2- 172.21.15.151:0/1922793560 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6e5c079580 0x7f6e5c07ba40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6e78034000 tx=0x7f6e78004ab0 comp rx=0 tx=0).stop 2024-09-16T15:46:37.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:37.494+0000 7f6e7f200640 1 -- 172.21.15.151:0/1922793560 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e8015dee0 msgr2=0x7f6e800cede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:37.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:37.494+0000 7f6e7f200640 1 --2- 172.21.15.151:0/1922793560 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e8015dee0 0x7f6e800cede0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6e7000c400 tx=0x7f6e7000c8c0 comp rx=0 tx=0).stop 2024-09-16T15:46:37.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:37.494+0000 7f6e7f200640 1 -- 172.21.15.151:0/1922793560 shutdown_connections 2024-09-16T15:46:37.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:37.494+0000 7f6e7f200640 1 --2- 172.21.15.151:0/1922793560 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6e5c079580 0x7f6e5c07ba40 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:37.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:37.494+0000 7f6e7f200640 1 --2- 172.21.15.151:0/1922793560 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6e8015e2c0 0x7f6e8009f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:37.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:37.494+0000 7f6e7f200640 1 --2- 172.21.15.151:0/1922793560 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e8015dee0 0x7f6e800cede0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:37.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:37.494+0000 7f6e7f200640 1 --2- 172.21.15.151:0/1922793560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e800b54a0 0x7f6e801f4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:37.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:37.494+0000 7f6e7f200640 1 -- 172.21.15.151:0/1922793560 >> 172.21.15.151:0/1922793560 conn(0x7f6e800d1c40 msgr2=0x7f6e8015e7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:37.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:37.494+0000 7f6e7f200640 1 -- 172.21.15.151:0/1922793560 shutdown_connections 2024-09-16T15:46:37.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:37.494+0000 7f6e7f200640 1 -- 172.21.15.151:0/1922793560 wait complete. 2024-09-16T15:46:37.520 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-16T15:46:37.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:46:37.521 DEBUG:teuthology.orchestra.run.smithi151:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-16T15:46:37.638 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi151.front.sepia.ceph.com 2024-09-16T15:46:37.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:46:37.638 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-16T15:46:37.708 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:46:37.708 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-16T15:46:37.708 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:46:37.764 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:46:37.765 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:46:37.765 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:46:37.765 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:46:37.819 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T15:46:37.819 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:46:37.819 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-16T15:46:37.819 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:46:37.878 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:46:37.878 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:46:38.122 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.121+0000 7fe223e00640 1 -- 172.21.15.151:0/3824173213 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2240cb740 msgr2=0x7fe2240736e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:38.122 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.121+0000 7fe223e00640 1 --2- 172.21.15.151:0/3824173213 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2240cb740 0x7fe2240736e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe2240b6ca0 tx=0x7fe21402a540 comp rx=0 tx=0).stop 2024-09-16T15:46:38.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.121+0000 7fe223e00640 1 -- 172.21.15.151:0/3824173213 shutdown_connections 2024-09-16T15:46:38.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.121+0000 7fe223e00640 1 --2- 172.21.15.151:0/3824173213 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2240cb740 0x7fe2240736e0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:38.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.121+0000 7fe223e00640 1 --2- 172.21.15.151:0/3824173213 >> v2:172.21.15.151:3300/0 conn(0x7fe224071130 0x7fe22415bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:38.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.121+0000 7fe223e00640 1 -- 172.21.15.151:0/3824173213 >> 172.21.15.151:0/3824173213 conn(0x7fe2240d0b30 msgr2=0x7fe2240d0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:38.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.121+0000 7fe223e00640 1 -- 172.21.15.151:0/3824173213 shutdown_connections 2024-09-16T15:46:38.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.121+0000 7fe223e00640 1 -- 172.21.15.151:0/3824173213 wait complete. 2024-09-16T15:46:38.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.123+0000 7fe223e00640 1 Processor -- start 2024-09-16T15:46:38.125 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.123+0000 7fe223e00640 1 -- start start 2024-09-16T15:46:38.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.125+0000 7fe223e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe224071130 0x7fe22414ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:38.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.125+0000 7fe223e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe224073c20 0x7fe224153bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:38.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.125+0000 7fe223e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe2240cb740 0x7fe224154110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:38.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.125+0000 7fe223e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe224154e30 con 0x7fe224073c20 2024-09-16T15:46:38.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.125+0000 7fe223e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe224154fb0 con 0x7fe224071130 2024-09-16T15:46:38.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.125+0000 7fe223e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fe224155130 con 0x7fe2240cb740 2024-09-16T15:46:38.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.125+0000 7fe222a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe224071130 0x7fe22414ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:38.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.125+0000 7fe222a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe224071130 0x7fe22414ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.151:51730/0 (socket says 172.21.15.151:51730) 2024-09-16T15:46:38.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.125+0000 7fe222a00640 1 -- 172.21.15.151:0/2539243097 learned_addr learned my addr 172.21.15.151:0/2539243097 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:46:38.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.125+0000 7fe21a000640 1 --2- 172.21.15.151:0/2539243097 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe224073c20 0x7fe224153bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:38.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.125+0000 7fe223400640 1 --2- 172.21.15.151:0/2539243097 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe2240cb740 0x7fe224154110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:38.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.126+0000 7fe21a000640 1 -- 172.21.15.151:0/2539243097 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe2240cb740 msgr2=0x7fe224154110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:38.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.126+0000 7fe21a000640 1 --2- 172.21.15.151:0/2539243097 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe2240cb740 0x7fe224154110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:38.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.126+0000 7fe21a000640 1 -- 172.21.15.151:0/2539243097 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe224071130 msgr2=0x7fe22414ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:38.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.126+0000 7fe21a000640 1 --2- 172.21.15.151:0/2539243097 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe224071130 0x7fe22414ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:38.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.126+0000 7fe21a000640 1 -- 172.21.15.151:0/2539243097 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2240c4dd0 con 0x7fe224073c20 2024-09-16T15:46:38.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.126+0000 7fe223400640 1 --2- 172.21.15.151:0/2539243097 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe2240cb740 0x7fe224154110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:46:38.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.126+0000 7fe222a00640 1 --2- 172.21.15.151:0/2539243097 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe224071130 0x7fe22414ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:46:38.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.127+0000 7fe21a000640 1 --2- 172.21.15.151:0/2539243097 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe224073c20 0x7fe224153bd0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fe214031000 tx=0x7fe214033c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:38.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.128+0000 7fe21be00640 1 -- 172.21.15.151:0/2539243097 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe214005c90 con 0x7fe224073c20 2024-09-16T15:46:38.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.128+0000 7fe21be00640 1 -- 172.21.15.151:0/2539243097 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe214005020 con 0x7fe224073c20 2024-09-16T15:46:38.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.128+0000 7fe21be00640 1 -- 172.21.15.151:0/2539243097 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe214005470 con 0x7fe224073c20 2024-09-16T15:46:38.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.128+0000 7fe223e00640 1 -- 172.21.15.151:0/2539243097 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe224150a60 con 0x7fe224073c20 2024-09-16T15:46:38.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.128+0000 7fe223e00640 1 -- 172.21.15.151:0/2539243097 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2241523e0 con 0x7fe224073c20 2024-09-16T15:46:38.130 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.129+0000 7fe223e00640 1 -- 172.21.15.151:0/2539243097 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe224150850 con 0x7fe224073c20 2024-09-16T15:46:38.131 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.130+0000 7fe21be00640 1 -- 172.21.15.151:0/2539243097 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 24) ==== 102409+0+0 (secure 0 0 0) 0x7fe2140047a0 con 0x7fe224073c20 2024-09-16T15:46:38.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.131+0000 7fe21be00640 1 --2- 172.21.15.151:0/2539243097 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe2000794b0 0x7fe20007b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:38.133 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.132+0000 7fe21be00640 1 -- 172.21.15.151:0/2539243097 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5851+0+0 (secure 0 0 0) 0x7fe2140ba510 con 0x7fe224073c20 2024-09-16T15:46:38.133 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.132+0000 7fe222a00640 1 --2- 172.21.15.151:0/2539243097 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe2000794b0 0x7fe20007b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:38.134 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.133+0000 7fe222a00640 1 --2- 172.21.15.151:0/2539243097 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe2000794b0 0x7fe20007b970 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe20c005fd0 tx=0x7fe20c00b480 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:38.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.138+0000 7fe21be00640 1 -- 172.21.15.151:0/2539243097 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe21403f030 con 0x7fe224073c20 2024-09-16T15:46:38.437 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.435+0000 7fe223e00640 1 -- 172.21.15.151:0/2539243097 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe22415bb90 con 0x7fe224073c20 2024-09-16T15:46:38.437 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.436+0000 7fe21be00640 1 -- 172.21.15.151:0/2539243097 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5232 (secure 0 0 0) 0x7fe214031770 con 0x7fe224073c20 2024-09-16T15:46:38.437 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:46:38.438 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":37,"btime":"2024-09-16T15:44:58:754420+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5357,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1715048770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1715048770},{"type":"v1","addr":"172.21.15.167:6837","nonce":1715048770}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":5722,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/784205427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":784205427},{"type":"v1","addr":"172.21.15.167:6836","nonce":784205427}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":5958,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/3837674321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3837674321},{"type":"v1","addr":"172.21.15.151:6834","nonce":3837674321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:44:45.114621+0000","modified":"2024-09-16T15:44:48.149095+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5961},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5961":{"gid":5961,"name":"c","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.151:6835/1724192711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":1724192711},{"type":"v1","addr":"172.21.15.151:6835","nonce":1724192711}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5961,"qdb_cluster":[5961]},"id":4}]} 2024-09-16T15:46:38.438 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 37 2024-09-16T15:46:38.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.438+0000 7fe223e00640 1 -- 172.21.15.151:0/2539243097 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe2000794b0 msgr2=0x7fe20007b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:38.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.438+0000 7fe223e00640 1 --2- 172.21.15.151:0/2539243097 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe2000794b0 0x7fe20007b970 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe20c005fd0 tx=0x7fe20c00b480 comp rx=0 tx=0).stop 2024-09-16T15:46:38.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.438+0000 7fe223e00640 1 -- 172.21.15.151:0/2539243097 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe224073c20 msgr2=0x7fe224153bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:38.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.438+0000 7fe223e00640 1 --2- 172.21.15.151:0/2539243097 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe224073c20 0x7fe224153bd0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fe214031000 tx=0x7fe214033c90 comp rx=0 tx=0).stop 2024-09-16T15:46:38.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.438+0000 7fe223e00640 1 -- 172.21.15.151:0/2539243097 shutdown_connections 2024-09-16T15:46:38.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.438+0000 7fe223e00640 1 --2- 172.21.15.151:0/2539243097 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe2000794b0 0x7fe20007b970 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:38.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.438+0000 7fe223e00640 1 --2- 172.21.15.151:0/2539243097 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe2240cb740 0x7fe224154110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:38.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.438+0000 7fe223e00640 1 --2- 172.21.15.151:0/2539243097 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe224073c20 0x7fe224153bd0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:38.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.438+0000 7fe223e00640 1 --2- 172.21.15.151:0/2539243097 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe224071130 0x7fe22414ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:38.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.438+0000 7fe223e00640 1 -- 172.21.15.151:0/2539243097 >> 172.21.15.151:0/2539243097 conn(0x7fe2240d0b30 msgr2=0x7fe2240ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:38.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.438+0000 7fe223e00640 1 -- 172.21.15.151:0/2539243097 shutdown_connections 2024-09-16T15:46:38.441 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.439+0000 7fe223e00640 1 -- 172.21.15.151:0/2539243097 wait complete. 2024-09-16T15:46:38.456 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:46:38.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.613+0000 7f4f3dc00640 1 -- 172.21.15.151:0/1698571527 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f380ccf00 msgr2=0x7f4f380acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:38.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.613+0000 7f4f3dc00640 1 --2- 172.21.15.151:0/1698571527 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f380ccf00 0x7f4f380acd30 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f4f38075130 tx=0x7f4f2002a4d0 comp rx=0 tx=0).stop 2024-09-16T15:46:38.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.613+0000 7f4f3dc00640 1 -- 172.21.15.151:0/1698571527 shutdown_connections 2024-09-16T15:46:38.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.613+0000 7f4f3dc00640 1 --2- 172.21.15.151:0/1698571527 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f380ccf00 0x7f4f380acd30 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:38.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.613+0000 7f4f3dc00640 1 --2- 172.21.15.151:0/1698571527 >> v2:172.21.15.167:3300/0 conn(0x7f4f3815f5a0 0x7f4f380b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:38.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.613+0000 7f4f3dc00640 1 -- 172.21.15.151:0/1698571527 >> 172.21.15.151:0/1698571527 conn(0x7f4f380b1950 msgr2=0x7f4f380b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:38.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.613+0000 7f4f3dc00640 1 -- 172.21.15.151:0/1698571527 shutdown_connections 2024-09-16T15:46:38.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.613+0000 7f4f3dc00640 1 -- 172.21.15.151:0/1698571527 wait complete. 2024-09-16T15:46:38.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.615+0000 7f4f3dc00640 1 Processor -- start 2024-09-16T15:46:38.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.615+0000 7f4f3dc00640 1 -- start start 2024-09-16T15:46:38.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.616+0000 7f4f3dc00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f380ad270 0x7f4f3814e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:38.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.616+0000 7f4f3dc00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4f380ccf00 0x7f4f3814c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:38.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.616+0000 7f4f3dc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f3815f5a0 0x7f4f381f2860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:38.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.616+0000 7f4f3dc00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3806f160 con 0x7f4f3815f5a0 2024-09-16T15:46:38.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.616+0000 7f4f3dc00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3806f2e0 con 0x7f4f380ad270 2024-09-16T15:46:38.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.616+0000 7f4f3dc00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f4f3806f460 con 0x7f4f380ccf00 2024-09-16T15:46:38.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.616+0000 7f4f36a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f380ad270 0x7f4f3814e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:38.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.616+0000 7f4f37400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f3815f5a0 0x7f4f381f2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:38.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.616+0000 7f4f36a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f380ad270 0x7f4f3814e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.151:51748/0 (socket says 172.21.15.151:51748) 2024-09-16T15:46:38.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.616+0000 7f4f36a00640 1 -- 172.21.15.151:0/1328188664 learned_addr learned my addr 172.21.15.151:0/1328188664 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:46:38.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.616+0000 7f4f2fe00640 1 --2- 172.21.15.151:0/1328188664 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4f380ccf00 0x7f4f3814c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:38.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.617+0000 7f4f36a00640 1 -- 172.21.15.151:0/1328188664 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4f380ccf00 msgr2=0x7f4f3814c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:38.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.617+0000 7f4f36a00640 1 --2- 172.21.15.151:0/1328188664 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4f380ccf00 0x7f4f3814c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:38.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.617+0000 7f4f36a00640 1 -- 172.21.15.151:0/1328188664 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f3815f5a0 msgr2=0x7f4f381f2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:38.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.617+0000 7f4f36a00640 1 --2- 172.21.15.151:0/1328188664 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f3815f5a0 0x7f4f381f2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:38.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.617+0000 7f4f36a00640 1 -- 172.21.15.151:0/1328188664 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f3814ed40 con 0x7f4f380ad270 2024-09-16T15:46:38.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.617+0000 7f4f37400640 1 --2- 172.21.15.151:0/1328188664 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f3815f5a0 0x7f4f381f2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:46:38.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.617+0000 7f4f36a00640 1 --2- 172.21.15.151:0/1328188664 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f380ad270 0x7f4f3814e640 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4f2800d740 tx=0x7f4f2800dc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:38.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.617+0000 7f4f2ea00640 1 -- 172.21.15.151:0/1328188664 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f28017070 con 0x7f4f380ad270 2024-09-16T15:46:38.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.617+0000 7f4f3dc00640 1 -- 172.21.15.151:0/1328188664 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f3814f020 con 0x7f4f380ad270 2024-09-16T15:46:38.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.617+0000 7f4f2ea00640 1 -- 172.21.15.151:0/1328188664 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4f28009d20 con 0x7f4f380ad270 2024-09-16T15:46:38.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.617+0000 7f4f2ea00640 1 -- 172.21.15.151:0/1328188664 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f280106c0 con 0x7f4f380ad270 2024-09-16T15:46:38.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.617+0000 7f4f3dc00640 1 -- 172.21.15.151:0/1328188664 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f3814f5e0 con 0x7f4f380ad270 2024-09-16T15:46:38.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.618+0000 7f4f3dc00640 1 -- 172.21.15.151:0/1328188664 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f3806f9f0 con 0x7f4f380ad270 2024-09-16T15:46:38.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.620+0000 7f4f2ea00640 1 -- 172.21.15.151:0/1328188664 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 24) ==== 102409+0+0 (secure 0 0 0) 0x7f4f2801a070 con 0x7f4f380ad270 2024-09-16T15:46:38.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.621+0000 7f4f2ea00640 1 --2- 172.21.15.151:0/1328188664 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4f140794b0 0x7f4f1407b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:38.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.621+0000 7f4f2ea00640 1 -- 172.21.15.151:0/1328188664 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5851+0+0 (secure 0 0 0) 0x7f4f280996d0 con 0x7f4f380ad270 2024-09-16T15:46:38.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.621+0000 7f4f2fe00640 1 --2- 172.21.15.151:0/1328188664 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4f140794b0 0x7f4f1407b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:38.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.621+0000 7f4f2fe00640 1 --2- 172.21.15.151:0/1328188664 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4f140794b0 0x7f4f1407b970 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4f20033000 tx=0x7f4f20004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:38.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.626+0000 7f4f2ea00640 1 -- 172.21.15.151:0/1328188664 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4f28062e40 con 0x7f4f380ad270 2024-09-16T15:46:38.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.844+0000 7f4f3dc00640 1 -- 172.21.15.151:0/1328188664 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4f380b2550 con 0x7f4f380ad270 2024-09-16T15:46:38.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.845+0000 7f4f2ea00640 1 -- 172.21.15.151:0/1328188664 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5232 (secure 0 0 0) 0x7f4f28067870 con 0x7f4f380ad270 2024-09-16T15:46:38.846 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:46:38.846 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":37,"btime":"2024-09-16T15:44:58:754420+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5357,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1715048770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1715048770},{"type":"v1","addr":"172.21.15.167:6837","nonce":1715048770}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":5722,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/784205427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":784205427},{"type":"v1","addr":"172.21.15.167:6836","nonce":784205427}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":5958,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/3837674321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3837674321},{"type":"v1","addr":"172.21.15.151:6834","nonce":3837674321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:44:45.114621+0000","modified":"2024-09-16T15:44:48.149095+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5961},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5961":{"gid":5961,"name":"c","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.151:6835/1724192711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":1724192711},{"type":"v1","addr":"172.21.15.151:6835","nonce":1724192711}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5961,"qdb_cluster":[5961]},"id":4}]} 2024-09-16T15:46:38.846 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 37 2024-09-16T15:46:38.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.847+0000 7f4f3dc00640 1 -- 172.21.15.151:0/1328188664 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4f140794b0 msgr2=0x7f4f1407b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:38.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.847+0000 7f4f3dc00640 1 --2- 172.21.15.151:0/1328188664 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4f140794b0 0x7f4f1407b970 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4f20033000 tx=0x7f4f20004730 comp rx=0 tx=0).stop 2024-09-16T15:46:38.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.847+0000 7f4f3dc00640 1 -- 172.21.15.151:0/1328188664 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f380ad270 msgr2=0x7f4f3814e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:38.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.847+0000 7f4f3dc00640 1 --2- 172.21.15.151:0/1328188664 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f380ad270 0x7f4f3814e640 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4f2800d740 tx=0x7f4f2800dc00 comp rx=0 tx=0).stop 2024-09-16T15:46:38.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.847+0000 7f4f3dc00640 1 -- 172.21.15.151:0/1328188664 shutdown_connections 2024-09-16T15:46:38.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.847+0000 7f4f3dc00640 1 --2- 172.21.15.151:0/1328188664 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4f140794b0 0x7f4f1407b970 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:38.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.847+0000 7f4f3dc00640 1 --2- 172.21.15.151:0/1328188664 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f3815f5a0 0x7f4f381f2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:38.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.847+0000 7f4f3dc00640 1 --2- 172.21.15.151:0/1328188664 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4f380ccf00 0x7f4f3814c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:38.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.847+0000 7f4f3dc00640 1 --2- 172.21.15.151:0/1328188664 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f380ad270 0x7f4f3814e640 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:38.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.847+0000 7f4f3dc00640 1 -- 172.21.15.151:0/1328188664 >> 172.21.15.151:0/1328188664 conn(0x7f4f380b1950 msgr2=0x7f4f3809c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:38.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.847+0000 7f4f3dc00640 1 -- 172.21.15.151:0/1328188664 shutdown_connections 2024-09-16T15:46:38.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:38.847+0000 7f4f3dc00640 1 -- 172.21.15.151:0/1328188664 wait complete. 2024-09-16T15:46:38.865 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:46:39.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.023+0000 7fc528e00640 1 -- 172.21.15.151:0/3377351199 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5240725d0 msgr2=0x7fc5240b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:39.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.023+0000 7fc528e00640 1 --2- 172.21.15.151:0/3377351199 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5240725d0 0x7fc5240b28f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc510009860 tx=0x7fc51002a5a0 comp rx=0 tx=0).stop 2024-09-16T15:46:39.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.023+0000 7fc528e00640 1 -- 172.21.15.151:0/3377351199 shutdown_connections 2024-09-16T15:46:39.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.023+0000 7fc528e00640 1 --2- 172.21.15.151:0/3377351199 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5240725d0 0x7fc5240b28f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:39.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.023+0000 7fc528e00640 1 -- 172.21.15.151:0/3377351199 >> 172.21.15.151:0/3377351199 conn(0x7fc5240cfa20 msgr2=0x7fc5240cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:39.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.024+0000 7fc528e00640 1 -- 172.21.15.151:0/3377351199 shutdown_connections 2024-09-16T15:46:39.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.024+0000 7fc528e00640 1 -- 172.21.15.151:0/3377351199 wait complete. 2024-09-16T15:46:39.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.025+0000 7fc528e00640 1 Processor -- start 2024-09-16T15:46:39.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.025+0000 7fc528e00640 1 -- start start 2024-09-16T15:46:39.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.026+0000 7fc528e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5240d0540 0x7fc5241489b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:39.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.026+0000 7fc528e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc52415bc30 0x7fc524148ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:39.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.026+0000 7fc528e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc524152210 0x7fc5241f82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:39.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.026+0000 7fc528e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc524074660 con 0x7fc524152210 2024-09-16T15:46:39.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.026+0000 7fc528e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc52406fa30 con 0x7fc52415bc30 2024-09-16T15:46:39.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.026+0000 7fc528e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fc5240a9470 con 0x7fc5240d0540 2024-09-16T15:46:39.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.026+0000 7fc523e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc524152210 0x7fc5241f82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:39.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.027+0000 7fc523e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc524152210 0x7fc5241f82f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47878/0 (socket says 172.21.15.151:47878) 2024-09-16T15:46:39.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.027+0000 7fc523e00640 1 -- 172.21.15.151:0/3333102830 learned_addr learned my addr 172.21.15.151:0/3333102830 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:46:39.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.027+0000 7fc523400640 1 --2- 172.21.15.151:0/3333102830 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5240d0540 0x7fc5241489b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:39.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.027+0000 7fc522a00640 1 --2- 172.21.15.151:0/3333102830 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc52415bc30 0x7fc524148ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:39.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.027+0000 7fc523e00640 1 -- 172.21.15.151:0/3333102830 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5240d0540 msgr2=0x7fc5241489b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:39.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.027+0000 7fc523e00640 1 --2- 172.21.15.151:0/3333102830 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5240d0540 0x7fc5241489b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:39.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.027+0000 7fc523e00640 1 -- 172.21.15.151:0/3333102830 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc52415bc30 msgr2=0x7fc524148ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:39.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.027+0000 7fc523e00640 1 --2- 172.21.15.151:0/3333102830 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc52415bc30 0x7fc524148ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:39.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.027+0000 7fc523e00640 1 -- 172.21.15.151:0/3333102830 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5240a96c0 con 0x7fc524152210 2024-09-16T15:46:39.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.027+0000 7fc523e00640 1 --2- 172.21.15.151:0/3333102830 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc524152210 0x7fc5241f82f0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fc51c00c400 tx=0x7fc51c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:39.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.027+0000 7fc51b400640 1 -- 172.21.15.151:0/3333102830 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc51c017070 con 0x7fc524152210 2024-09-16T15:46:39.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.027+0000 7fc51b400640 1 -- 172.21.15.151:0/3333102830 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc51c0053c0 con 0x7fc524152210 2024-09-16T15:46:39.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.027+0000 7fc51b400640 1 -- 172.21.15.151:0/3333102830 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc51c003850 con 0x7fc524152210 2024-09-16T15:46:39.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.027+0000 7fc528e00640 1 -- 172.21.15.151:0/3333102830 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5240a99a0 con 0x7fc524152210 2024-09-16T15:46:39.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.028+0000 7fc528e00640 1 -- 172.21.15.151:0/3333102830 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5241f8830 con 0x7fc524152210 2024-09-16T15:46:39.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.029+0000 7fc528e00640 1 -- 172.21.15.151:0/3333102830 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5240a9be0 con 0x7fc524152210 2024-09-16T15:46:39.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.030+0000 7fc51b400640 1 -- 172.21.15.151:0/3333102830 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 24) ==== 102409+0+0 (secure 0 0 0) 0x7fc51c01a070 con 0x7fc524152210 2024-09-16T15:46:39.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.031+0000 7fc51b400640 1 --2- 172.21.15.151:0/3333102830 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc5040794e0 0x7fc50407b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:39.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.031+0000 7fc51b400640 1 -- 172.21.15.151:0/3333102830 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5851+0+0 (secure 0 0 0) 0x7fc51c098960 con 0x7fc524152210 2024-09-16T15:46:39.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.031+0000 7fc523400640 1 --2- 172.21.15.151:0/3333102830 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc5040794e0 0x7fc50407b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:39.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.032+0000 7fc523400640 1 --2- 172.21.15.151:0/3333102830 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc5040794e0 0x7fc50407b9a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc514002090 tx=0x7fc514009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:39.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.035+0000 7fc51b400640 1 -- 172.21.15.151:0/3333102830 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc51c0627a0 con 0x7fc524152210 2024-09-16T15:46:39.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.221+0000 7fc528e00640 1 -- 172.21.15.151:0/3333102830 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc5240b28f0 con 0x7fc524152210 2024-09-16T15:46:39.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.223+0000 7fc51b400640 1 -- 172.21.15.151:0/3333102830 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) ==== 74+0+17032 (secure 0 0 0) 0x7fc51c0620a0 con 0x7fc524152210 2024-09-16T15:46:39.224 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:46:39.224 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":65,"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","created":"2024-09-16T15:38:56.348535+0000","modified":"2024-09-16T15:46:37.117088+0000","last_up_change":"2024-09-16T15:39:10.681384+0000","last_in_change":"2024-09-16T15:39:00.544477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:39:11.602252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:44:42.866298+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-16T15:44:44.081060+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":65,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5524076-a903-4d0a-af6a-caf474c4765d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6809","nonce":3111342741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6811","nonce":3111342741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6815","nonce":3111342741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6813","nonce":3111342741}]},"public_addr":"172.21.15.151:6809/3111342741","cluster_addr":"172.21.15.151:6811/3111342741","heartbeat_back_addr":"172.21.15.151:6815/3111342741","heartbeat_front_addr":"172.21.15.151:6813/3111342741","state":["exists","up"]},{"osd":1,"uuid":"dd25925f-0ae1-494e-8529-4dc205759db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6801","nonce":3368304087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6803","nonce":3368304087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6807","nonce":3368304087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6805","nonce":3368304087}]},"public_addr":"172.21.15.151:6801/3368304087","cluster_addr":"172.21.15.151:6803/3368304087","heartbeat_back_addr":"172.21.15.151:6807/3368304087","heartbeat_front_addr":"172.21.15.151:6805/3368304087","state":["exists","up"]},{"osd":2,"uuid":"c2f18115-013d-4da6-a0fe-cb978f4494b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6823","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6825","nonce":3079373271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6827","nonce":3079373271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6831","nonce":3079373271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6829","nonce":3079373271}]},"public_addr":"172.21.15.151:6825/3079373271","cluster_addr":"172.21.15.151:6827/3079373271","heartbeat_back_addr":"172.21.15.151:6831/3079373271","heartbeat_front_addr":"172.21.15.151:6829/3079373271","state":["exists","up"]},{"osd":3,"uuid":"36edb718-2fcb-4d88-8ddc-c321eed4279d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6817","nonce":1142509041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6819","nonce":1142509041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6824","nonce":1142509041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6821","nonce":1142509041}]},"public_addr":"172.21.15.151:6817/1142509041","cluster_addr":"172.21.15.151:6819/1142509041","heartbeat_back_addr":"172.21.15.151:6824/1142509041","heartbeat_front_addr":"172.21.15.151:6821/1142509041","state":["exists","up"]},{"osd":4,"uuid":"64d33708-cd0a-4236-abe3-1b646e82ab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6811","nonce":2735495271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6813","nonce":2735495271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6817","nonce":2735495271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6815","nonce":2735495271}]},"public_addr":"172.21.15.167:6811/2735495271","cluster_addr":"172.21.15.167:6813/2735495271","heartbeat_back_addr":"172.21.15.167:6817/2735495271","heartbeat_front_addr":"172.21.15.167:6815/2735495271","state":["exists","up"]},{"osd":5,"uuid":"f14b44a9-5910-458c-b518-09a8d0de94de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6827","nonce":1494916572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6829","nonce":1494916572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6833","nonce":1494916572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6831","nonce":1494916572}]},"public_addr":"172.21.15.167:6827/1494916572","cluster_addr":"172.21.15.167:6829/1494916572","heartbeat_back_addr":"172.21.15.167:6833/1494916572","heartbeat_front_addr":"172.21.15.167:6831/1494916572","state":["exists","up"]},{"osd":6,"uuid":"39335545-2654-4710-84b0-a9aa47cfcbba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6819","nonce":2538273273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6821","nonce":2538273273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6825","nonce":2538273273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6823","nonce":2538273273}]},"public_addr":"172.21.15.167:6819/2538273273","cluster_addr":"172.21.15.167:6821/2538273273","heartbeat_back_addr":"172.21.15.167:6825/2538273273","heartbeat_front_addr":"172.21.15.167:6823/2538273273","state":["exists","up"]},{"osd":7,"uuid":"4f8bfc81-53dd-4339-ac22-1a4007d9f963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6803","nonce":265072227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6805","nonce":265072227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6809","nonce":265072227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6807","nonce":265072227}]},"public_addr":"172.21.15.167:6803/265072227","cluster_addr":"172.21.15.167:6805/265072227","heartbeat_back_addr":"172.21.15.167:6809/265072227","heartbeat_front_addr":"172.21.15.167:6807/265072227","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.150359+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.084358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.722368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.332809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.320709+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.395153+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.935250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.801443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.a","mappings":[{"from":6,"to":4}]},{"pgid":"9.27","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:46:39.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.225+0000 7fc528e00640 1 -- 172.21.15.151:0/3333102830 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc5040794e0 msgr2=0x7fc50407b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:39.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.225+0000 7fc528e00640 1 --2- 172.21.15.151:0/3333102830 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc5040794e0 0x7fc50407b9a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc514002090 tx=0x7fc514009040 comp rx=0 tx=0).stop 2024-09-16T15:46:39.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.225+0000 7fc528e00640 1 -- 172.21.15.151:0/3333102830 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc524152210 msgr2=0x7fc5241f82f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:39.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.225+0000 7fc528e00640 1 --2- 172.21.15.151:0/3333102830 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc524152210 0x7fc5241f82f0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fc51c00c400 tx=0x7fc51c00c8c0 comp rx=0 tx=0).stop 2024-09-16T15:46:39.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.225+0000 7fc528e00640 1 -- 172.21.15.151:0/3333102830 shutdown_connections 2024-09-16T15:46:39.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.225+0000 7fc528e00640 1 --2- 172.21.15.151:0/3333102830 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc5040794e0 0x7fc50407b9a0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:39.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.225+0000 7fc528e00640 1 --2- 172.21.15.151:0/3333102830 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc52415bc30 0x7fc524148ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:39.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.225+0000 7fc528e00640 1 --2- 172.21.15.151:0/3333102830 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc524152210 0x7fc5241f82f0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:39.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.225+0000 7fc528e00640 1 --2- 172.21.15.151:0/3333102830 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5240d0540 0x7fc5241489b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:39.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.225+0000 7fc528e00640 1 -- 172.21.15.151:0/3333102830 >> 172.21.15.151:0/3333102830 conn(0x7fc5240cfa20 msgr2=0x7fc52415a230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:39.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.225+0000 7fc528e00640 1 -- 172.21.15.151:0/3333102830 shutdown_connections 2024-09-16T15:46:39.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.225+0000 7fc528e00640 1 -- 172.21.15.151:0/3333102830 wait complete. 2024-09-16T15:46:39.243 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-16T15:46:39.243 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:46:39.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.400+0000 7fc13fe00640 1 -- 172.21.15.151:0/3658487820 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1240052e0 msgr2=0x7fc12400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:39.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.400+0000 7fc13fe00640 1 --2- 172.21.15.151:0/3658487820 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1240052e0 0x7fc12400b9c0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fc1400b3290 tx=0x7fc124005830 comp rx=0 tx=0).stop 2024-09-16T15:46:39.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.401+0000 7fc13fe00640 1 -- 172.21.15.151:0/3658487820 shutdown_connections 2024-09-16T15:46:39.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.401+0000 7fc13fe00640 1 --2- 172.21.15.151:0/3658487820 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1240052e0 0x7fc12400b9c0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:39.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.401+0000 7fc13fe00640 1 -- 172.21.15.151:0/3658487820 >> 172.21.15.151:0/3658487820 conn(0x7fc1400d3df0 msgr2=0x7fc1400d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:39.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.401+0000 7fc13fe00640 1 -- 172.21.15.151:0/3658487820 shutdown_connections 2024-09-16T15:46:39.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.401+0000 7fc13fe00640 1 -- 172.21.15.151:0/3658487820 wait complete. 2024-09-16T15:46:39.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.402+0000 7fc13fe00640 1 Processor -- start 2024-09-16T15:46:39.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.402+0000 7fc13fe00640 1 -- start start 2024-09-16T15:46:39.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.403+0000 7fc13fe00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1240052e0 0x7fc140159bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:39.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.403+0000 7fc13fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1401f8270 0x7fc140157fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:39.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.403+0000 7fc13fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1400c9640 0x7fc1400b8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:39.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.403+0000 7fc13fe00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc1400d1ff0 con 0x7fc1401f8270 2024-09-16T15:46:39.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.403+0000 7fc13fe00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc1400d1e70 con 0x7fc1400c9640 2024-09-16T15:46:39.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.403+0000 7fc13fe00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fc1400d2170 con 0x7fc1240052e0 2024-09-16T15:46:39.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.404+0000 7fc13e000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1401f8270 0x7fc140157fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:39.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.404+0000 7fc13e000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1401f8270 0x7fc140157fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47892/0 (socket says 172.21.15.151:47892) 2024-09-16T15:46:39.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.404+0000 7fc13e000640 1 -- 172.21.15.151:0/3625956297 learned_addr learned my addr 172.21.15.151:0/3625956297 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:46:39.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.404+0000 7fc13ea00640 1 --2- 172.21.15.151:0/3625956297 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1240052e0 0x7fc140159bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:39.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.404+0000 7fc13f400640 1 --2- 172.21.15.151:0/3625956297 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1400c9640 0x7fc1400b8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:39.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.404+0000 7fc13e000640 1 -- 172.21.15.151:0/3625956297 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1240052e0 msgr2=0x7fc140159bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:39.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.404+0000 7fc13e000640 1 --2- 172.21.15.151:0/3625956297 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1240052e0 0x7fc140159bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:39.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.404+0000 7fc13e000640 1 -- 172.21.15.151:0/3625956297 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1400c9640 msgr2=0x7fc1400b8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:39.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.404+0000 7fc13e000640 1 --2- 172.21.15.151:0/3625956297 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1400c9640 0x7fc1400b8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:39.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.404+0000 7fc13e000640 1 -- 172.21.15.151:0/3625956297 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1401f8700 con 0x7fc1401f8270 2024-09-16T15:46:39.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.404+0000 7fc13e000640 1 --2- 172.21.15.151:0/3625956297 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1401f8270 0x7fc140157fa0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fc12c004e20 tx=0x7fc12c001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:39.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.404+0000 7fc136a00640 1 -- 172.21.15.151:0/3625956297 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc12c0107d0 con 0x7fc1401f8270 2024-09-16T15:46:39.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.404+0000 7fc136a00640 1 -- 172.21.15.151:0/3625956297 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc12c016070 con 0x7fc1401f8270 2024-09-16T15:46:39.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.404+0000 7fc136a00640 1 -- 172.21.15.151:0/3625956297 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc12c010cc0 con 0x7fc1401f8270 2024-09-16T15:46:39.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.404+0000 7fc13fe00640 1 -- 172.21.15.151:0/3625956297 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1401f89e0 con 0x7fc1401f8270 2024-09-16T15:46:39.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.405+0000 7fc13fe00640 1 -- 172.21.15.151:0/3625956297 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1401f8fa0 con 0x7fc1401f8270 2024-09-16T15:46:39.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.406+0000 7fc13fe00640 1 -- 172.21.15.151:0/3625956297 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1400b3ba0 con 0x7fc1401f8270 2024-09-16T15:46:39.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.407+0000 7fc136a00640 1 -- 172.21.15.151:0/3625956297 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 24) ==== 102409+0+0 (secure 0 0 0) 0x7fc12c01b460 con 0x7fc1401f8270 2024-09-16T15:46:39.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.408+0000 7fc136a00640 1 --2- 172.21.15.151:0/3625956297 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc1200794b0 0x7fc12007b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:39.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.408+0000 7fc136a00640 1 -- 172.21.15.151:0/3625956297 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5851+0+0 (secure 0 0 0) 0x7fc12c0999e0 con 0x7fc1401f8270 2024-09-16T15:46:39.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.408+0000 7fc13ea00640 1 --2- 172.21.15.151:0/3625956297 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc1200794b0 0x7fc12007b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:39.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.409+0000 7fc13ea00640 1 --2- 172.21.15.151:0/3625956297 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc1200794b0 0x7fc12007b970 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc130001a70 tx=0x7fc130006040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:39.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.412+0000 7fc136a00640 1 -- 172.21.15.151:0/3625956297 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc1400b5510 con 0x7fc1401f8270 2024-09-16T15:46:39.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.632+0000 7fc13fe00640 1 -- 172.21.15.151:0/3625956297 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc14015b2e0 con 0x7fc1401f8270 2024-09-16T15:46:39.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.632+0000 7fc136a00640 1 -- 172.21.15.151:0/3625956297 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5232 (secure 0 0 0) 0x7fc12c063450 con 0x7fc1401f8270 2024-09-16T15:46:39.633 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:46:39.633 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":37,"btime":"2024-09-16T15:44:58:754420+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5357,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1715048770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1715048770},{"type":"v1","addr":"172.21.15.167:6837","nonce":1715048770}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":5722,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/784205427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":784205427},{"type":"v1","addr":"172.21.15.167:6836","nonce":784205427}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":5958,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/3837674321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3837674321},{"type":"v1","addr":"172.21.15.151:6834","nonce":3837674321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:44:45.114621+0000","modified":"2024-09-16T15:44:48.149095+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5961},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5961":{"gid":5961,"name":"c","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.151:6835/1724192711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":1724192711},{"type":"v1","addr":"172.21.15.151:6835","nonce":1724192711}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5961,"qdb_cluster":[5961]},"id":4}]} 2024-09-16T15:46:39.634 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 37 2024-09-16T15:46:39.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.635+0000 7fc13fe00640 1 -- 172.21.15.151:0/3625956297 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc1200794b0 msgr2=0x7fc12007b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:39.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.635+0000 7fc13fe00640 1 --2- 172.21.15.151:0/3625956297 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc1200794b0 0x7fc12007b970 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc130001a70 tx=0x7fc130006040 comp rx=0 tx=0).stop 2024-09-16T15:46:39.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.635+0000 7fc13fe00640 1 -- 172.21.15.151:0/3625956297 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1401f8270 msgr2=0x7fc140157fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:39.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.635+0000 7fc13fe00640 1 --2- 172.21.15.151:0/3625956297 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1401f8270 0x7fc140157fa0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fc12c004e20 tx=0x7fc12c001630 comp rx=0 tx=0).stop 2024-09-16T15:46:39.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.635+0000 7fc13fe00640 1 -- 172.21.15.151:0/3625956297 shutdown_connections 2024-09-16T15:46:39.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.635+0000 7fc13fe00640 1 --2- 172.21.15.151:0/3625956297 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc1200794b0 0x7fc12007b970 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:39.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.635+0000 7fc13fe00640 1 --2- 172.21.15.151:0/3625956297 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1400c9640 0x7fc1400b8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:39.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.635+0000 7fc13fe00640 1 --2- 172.21.15.151:0/3625956297 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1401f8270 0x7fc140157fa0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:39.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.635+0000 7fc13fe00640 1 --2- 172.21.15.151:0/3625956297 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1240052e0 0x7fc140159bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:39.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.635+0000 7fc13fe00640 1 -- 172.21.15.151:0/3625956297 >> 172.21.15.151:0/3625956297 conn(0x7fc1400d3df0 msgr2=0x7fc1400b46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:39.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.635+0000 7fc13fe00640 1 -- 172.21.15.151:0/3625956297 shutdown_connections 2024-09-16T15:46:39.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.635+0000 7fc13fe00640 1 -- 172.21.15.151:0/3625956297 wait complete. 2024-09-16T15:46:39.653 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:46:39.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.810+0000 7fa571800640 1 -- 172.21.15.151:0/801715815 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa56c1616b0 msgr2=0x7fa56c0d21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:39.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.810+0000 7fa571800640 1 --2- 172.21.15.151:0/801715815 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa56c1616b0 0x7fa56c0d21a0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fa564005fd0 tx=0x7fa56402aa90 comp rx=0 tx=0).stop 2024-09-16T15:46:39.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.810+0000 7fa571800640 1 -- 172.21.15.151:0/801715815 shutdown_connections 2024-09-16T15:46:39.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.810+0000 7fa571800640 1 --2- 172.21.15.151:0/801715815 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa56c1616b0 0x7fa56c0d21a0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:39.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.810+0000 7fa571800640 1 --2- 172.21.15.151:0/801715815 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa56c0d4980 0x7fa56c0d1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:39.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.810+0000 7fa571800640 1 -- 172.21.15.151:0/801715815 >> 172.21.15.151:0/801715815 conn(0x7fa56c0d3df0 msgr2=0x7fa56c0d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:39.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.810+0000 7fa571800640 1 -- 172.21.15.151:0/801715815 shutdown_connections 2024-09-16T15:46:39.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.810+0000 7fa571800640 1 -- 172.21.15.151:0/801715815 wait complete. 2024-09-16T15:46:39.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.811+0000 7fa571800640 1 Processor -- start 2024-09-16T15:46:39.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.812+0000 7fa571800640 1 -- start start 2024-09-16T15:46:39.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.812+0000 7fa571800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa56c0d4980 0x7fa56c1f87b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:39.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.812+0000 7fa571800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa56c1612d0 0x7fa56c15d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:39.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.812+0000 7fa571800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa56c1616b0 0x7fa56c0cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:39.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.813+0000 7fa571800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa56c0b1c00 con 0x7fa56c1612d0 2024-09-16T15:46:39.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.813+0000 7fa571800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa56c0b1a80 con 0x7fa56c1616b0 2024-09-16T15:46:39.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.813+0000 7fa571800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fa56c0b1d80 con 0x7fa56c0d4980 2024-09-16T15:46:39.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.813+0000 7fa56b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa56c1612d0 0x7fa56c15d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:39.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.813+0000 7fa56b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa56c1612d0 0x7fa56c15d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47904/0 (socket says 172.21.15.151:47904) 2024-09-16T15:46:39.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.813+0000 7fa56b400640 1 -- 172.21.15.151:0/409563817 learned_addr learned my addr 172.21.15.151:0/409563817 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:46:39.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.813+0000 7fa56be00640 1 --2- 172.21.15.151:0/409563817 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa56c0d4980 0x7fa56c1f87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:39.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.813+0000 7fa570e00640 1 --2- 172.21.15.151:0/409563817 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa56c1616b0 0x7fa56c0cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:39.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.813+0000 7fa56b400640 1 -- 172.21.15.151:0/409563817 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa56c0d4980 msgr2=0x7fa56c1f87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:39.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.813+0000 7fa56b400640 1 --2- 172.21.15.151:0/409563817 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa56c0d4980 0x7fa56c1f87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:39.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.813+0000 7fa56b400640 1 -- 172.21.15.151:0/409563817 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa56c1616b0 msgr2=0x7fa56c0cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:39.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.813+0000 7fa56b400640 1 --2- 172.21.15.151:0/409563817 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa56c1616b0 0x7fa56c0cf7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:39.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.813+0000 7fa56b400640 1 -- 172.21.15.151:0/409563817 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa56c0ca1e0 con 0x7fa56c1612d0 2024-09-16T15:46:39.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.813+0000 7fa56be00640 1 --2- 172.21.15.151:0/409563817 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa56c0d4980 0x7fa56c1f87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:46:39.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.814+0000 7fa56b400640 1 --2- 172.21.15.151:0/409563817 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa56c1612d0 0x7fa56c15d2b0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fa55800e9f0 tx=0x7fa55800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:39.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.814+0000 7fa563e00640 1 -- 172.21.15.151:0/409563817 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55800ccc0 con 0x7fa56c1612d0 2024-09-16T15:46:39.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.814+0000 7fa563e00640 1 -- 172.21.15.151:0/409563817 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa558010430 con 0x7fa56c1612d0 2024-09-16T15:46:39.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.814+0000 7fa563e00640 1 -- 172.21.15.151:0/409563817 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa558010840 con 0x7fa56c1612d0 2024-09-16T15:46:39.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.814+0000 7fa571800640 1 -- 172.21.15.151:0/409563817 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa56c1f8cf0 con 0x7fa56c1612d0 2024-09-16T15:46:39.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.814+0000 7fa571800640 1 -- 172.21.15.151:0/409563817 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa56c1f4980 con 0x7fa56c1612d0 2024-09-16T15:46:39.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.815+0000 7fa571800640 1 -- 172.21.15.151:0/409563817 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa56c0b2230 con 0x7fa56c1612d0 2024-09-16T15:46:39.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.816+0000 7fa563e00640 1 -- 172.21.15.151:0/409563817 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 24) ==== 102409+0+0 (secure 0 0 0) 0x7fa5580109e0 con 0x7fa56c1612d0 2024-09-16T15:46:39.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.818+0000 7fa563e00640 1 --2- 172.21.15.151:0/409563817 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa54c0794b0 0x7fa54c07b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:39.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.818+0000 7fa563e00640 1 -- 172.21.15.151:0/409563817 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5851+0+0 (secure 0 0 0) 0x7fa558099e10 con 0x7fa56c1612d0 2024-09-16T15:46:39.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.818+0000 7fa56be00640 1 --2- 172.21.15.151:0/409563817 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa54c0794b0 0x7fa54c07b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:39.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.819+0000 7fa56be00640 1 --2- 172.21.15.151:0/409563817 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa54c0794b0 0x7fa54c07b970 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa56c0b4780 tx=0x7fa55c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:39.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:39.822+0000 7fa563e00640 1 -- 172.21.15.151:0/409563817 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa558063880 con 0x7fa56c1612d0 2024-09-16T15:46:40.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.039+0000 7fa571800640 1 -- 172.21.15.151:0/409563817 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa56c0b3c90 con 0x7fa56c1612d0 2024-09-16T15:46:40.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.040+0000 7fa563e00640 1 -- 172.21.15.151:0/409563817 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5232 (secure 0 0 0) 0x7fa5580682b0 con 0x7fa56c1612d0 2024-09-16T15:46:40.041 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:46:40.041 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":37,"btime":"2024-09-16T15:44:58:754420+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5357,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1715048770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1715048770},{"type":"v1","addr":"172.21.15.167:6837","nonce":1715048770}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":5722,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/784205427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":784205427},{"type":"v1","addr":"172.21.15.167:6836","nonce":784205427}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":5958,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/3837674321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3837674321},{"type":"v1","addr":"172.21.15.151:6834","nonce":3837674321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:44:45.114621+0000","modified":"2024-09-16T15:44:48.149095+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5961},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5961":{"gid":5961,"name":"c","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.151:6835/1724192711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":1724192711},{"type":"v1","addr":"172.21.15.151:6835","nonce":1724192711}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5961,"qdb_cluster":[5961]},"id":4}]} 2024-09-16T15:46:40.041 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 37 2024-09-16T15:46:40.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.042+0000 7fa571800640 1 -- 172.21.15.151:0/409563817 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa54c0794b0 msgr2=0x7fa54c07b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:40.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.042+0000 7fa571800640 1 --2- 172.21.15.151:0/409563817 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa54c0794b0 0x7fa54c07b970 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa56c0b4780 tx=0x7fa55c009040 comp rx=0 tx=0).stop 2024-09-16T15:46:40.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.042+0000 7fa571800640 1 -- 172.21.15.151:0/409563817 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa56c1612d0 msgr2=0x7fa56c15d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:40.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.042+0000 7fa571800640 1 --2- 172.21.15.151:0/409563817 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa56c1612d0 0x7fa56c15d2b0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fa55800e9f0 tx=0x7fa55800eeb0 comp rx=0 tx=0).stop 2024-09-16T15:46:40.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.042+0000 7fa571800640 1 -- 172.21.15.151:0/409563817 shutdown_connections 2024-09-16T15:46:40.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.042+0000 7fa571800640 1 --2- 172.21.15.151:0/409563817 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa54c0794b0 0x7fa54c07b970 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:40.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.042+0000 7fa571800640 1 --2- 172.21.15.151:0/409563817 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa56c1616b0 0x7fa56c0cf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:40.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.042+0000 7fa571800640 1 --2- 172.21.15.151:0/409563817 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa56c1612d0 0x7fa56c15d2b0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:40.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.042+0000 7fa571800640 1 --2- 172.21.15.151:0/409563817 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa56c0d4980 0x7fa56c1f87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:40.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.042+0000 7fa571800640 1 -- 172.21.15.151:0/409563817 >> 172.21.15.151:0/409563817 conn(0x7fa56c0d3df0 msgr2=0x7fa56c0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:40.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.043+0000 7fa571800640 1 -- 172.21.15.151:0/409563817 shutdown_connections 2024-09-16T15:46:40.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.043+0000 7fa571800640 1 -- 172.21.15.151:0/409563817 wait complete. 2024-09-16T15:46:40.061 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:46:40.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.218+0000 7f9d24600640 1 -- 172.21.15.151:0/3394776477 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d1c0ccf00 msgr2=0x7f9d1c0acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:40.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.218+0000 7f9d24600640 1 --2- 172.21.15.151:0/3394776477 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d1c0ccf00 0x7f9d1c0acd30 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c075130 tx=0x7f9d0802a4d0 comp rx=0 tx=0).stop 2024-09-16T15:46:40.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.218+0000 7f9d24600640 1 -- 172.21.15.151:0/3394776477 shutdown_connections 2024-09-16T15:46:40.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.218+0000 7f9d24600640 1 --2- 172.21.15.151:0/3394776477 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9d1c0ad270 0x7f9d1c073fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:40.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.218+0000 7f9d24600640 1 --2- 172.21.15.151:0/3394776477 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d1c0ccf00 0x7f9d1c0acd30 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:40.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.218+0000 7f9d24600640 1 --2- 172.21.15.151:0/3394776477 >> v2:172.21.15.167:3300/0 conn(0x7f9d1c15f5a0 0x7f9d1c0b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:40.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.218+0000 7f9d24600640 1 -- 172.21.15.151:0/3394776477 >> 172.21.15.151:0/3394776477 conn(0x7f9d1c0b1950 msgr2=0x7f9d1c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:40.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.218+0000 7f9d24600640 1 -- 172.21.15.151:0/3394776477 shutdown_connections 2024-09-16T15:46:40.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.219+0000 7f9d24600640 1 -- 172.21.15.151:0/3394776477 wait complete. 2024-09-16T15:46:40.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.220+0000 7f9d24600640 1 Processor -- start 2024-09-16T15:46:40.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.220+0000 7f9d24600640 1 -- start start 2024-09-16T15:46:40.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.221+0000 7f9d24600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d1c0ad270 0x7f9d1c154af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:40.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.221+0000 7f9d24600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d1c0ccf00 0x7f9d1c15b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:40.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.221+0000 7f9d24600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9d1c15f5a0 0x7f9d1c1f7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:40.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.221+0000 7f9d24600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1c074880 con 0x7f9d1c0ad270 2024-09-16T15:46:40.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.221+0000 7f9d24600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1c074700 con 0x7f9d1c0ccf00 2024-09-16T15:46:40.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.221+0000 7f9d24600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f9d1c06f160 con 0x7f9d1c15f5a0 2024-09-16T15:46:40.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.221+0000 7f9d21c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d1c0ad270 0x7f9d1c154af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:40.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.221+0000 7f9d21c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d1c0ad270 0x7f9d1c154af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47924/0 (socket says 172.21.15.151:47924) 2024-09-16T15:46:40.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.221+0000 7f9d21c00640 1 -- 172.21.15.151:0/2994596276 learned_addr learned my addr 172.21.15.151:0/2994596276 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:46:40.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.221+0000 7f9d22600640 1 --2- 172.21.15.151:0/2994596276 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9d1c15f5a0 0x7f9d1c1f7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:40.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.222+0000 7f9d21200640 1 --2- 172.21.15.151:0/2994596276 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d1c0ccf00 0x7f9d1c15b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:40.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.222+0000 7f9d21c00640 1 -- 172.21.15.151:0/2994596276 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9d1c15f5a0 msgr2=0x7f9d1c1f7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:40.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.222+0000 7f9d21c00640 1 --2- 172.21.15.151:0/2994596276 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9d1c15f5a0 0x7f9d1c1f7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:40.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.222+0000 7f9d21c00640 1 -- 172.21.15.151:0/2994596276 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d1c0ccf00 msgr2=0x7f9d1c15b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:40.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.222+0000 7f9d21c00640 1 --2- 172.21.15.151:0/2994596276 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d1c0ccf00 0x7f9d1c15b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:40.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.222+0000 7f9d21c00640 1 -- 172.21.15.151:0/2994596276 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d1c15a020 con 0x7f9d1c0ad270 2024-09-16T15:46:40.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.222+0000 7f9d21200640 1 --2- 172.21.15.151:0/2994596276 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d1c0ccf00 0x7f9d1c15b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:46:40.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.222+0000 7f9d21c00640 1 --2- 172.21.15.151:0/2994596276 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d1c0ad270 0x7f9d1c154af0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f9d1000d750 tx=0x7f9d1000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:40.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.222+0000 7f9d19600640 1 -- 172.21.15.151:0/2994596276 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d10017070 con 0x7f9d1c0ad270 2024-09-16T15:46:40.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.222+0000 7f9d19600640 1 -- 172.21.15.151:0/2994596276 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9d10004020 con 0x7f9d1c0ad270 2024-09-16T15:46:40.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.222+0000 7f9d19600640 1 -- 172.21.15.151:0/2994596276 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d10004450 con 0x7f9d1c0ad270 2024-09-16T15:46:40.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.222+0000 7f9d24600640 1 -- 172.21.15.151:0/2994596276 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d1c15a290 con 0x7f9d1c0ad270 2024-09-16T15:46:40.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.222+0000 7f9d24600640 1 -- 172.21.15.151:0/2994596276 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d1c15a850 con 0x7f9d1c0ad270 2024-09-16T15:46:40.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.224+0000 7f9d24600640 1 -- 172.21.15.151:0/2994596276 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d1c073fb0 con 0x7f9d1c0ad270 2024-09-16T15:46:40.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.224+0000 7f9d19600640 1 -- 172.21.15.151:0/2994596276 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 24) ==== 102409+0+0 (secure 0 0 0) 0x7f9d100099d0 con 0x7f9d1c0ad270 2024-09-16T15:46:40.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.225+0000 7f9d19600640 1 --2- 172.21.15.151:0/2994596276 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9cfc0794b0 0x7f9cfc07b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:40.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.226+0000 7f9d19600640 1 -- 172.21.15.151:0/2994596276 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5851+0+0 (secure 0 0 0) 0x7f9d10099970 con 0x7f9d1c0ad270 2024-09-16T15:46:40.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.226+0000 7f9d21200640 1 --2- 172.21.15.151:0/2994596276 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9cfc0794b0 0x7f9cfc07b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:40.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.227+0000 7f9d21200640 1 --2- 172.21.15.151:0/2994596276 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9cfc0794b0 0x7f9cfc07b970 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9d08033000 tx=0x7f9d08004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:40.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.229+0000 7f9d19600640 1 -- 172.21.15.151:0/2994596276 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9d10020020 con 0x7f9d1c0ad270 2024-09-16T15:46:40.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.414+0000 7f9d24600640 1 -- 172.21.15.151:0/2994596276 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9d1c06fbb0 con 0x7f9d1c0ad270 2024-09-16T15:46:40.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.415+0000 7f9d19600640 1 -- 172.21.15.151:0/2994596276 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) ==== 74+0+17032 (secure 0 0 0) 0x7f9d100636b0 con 0x7f9d1c0ad270 2024-09-16T15:46:40.417 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:46:40.417 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":65,"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","created":"2024-09-16T15:38:56.348535+0000","modified":"2024-09-16T15:46:37.117088+0000","last_up_change":"2024-09-16T15:39:10.681384+0000","last_in_change":"2024-09-16T15:39:00.544477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:39:11.602252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:44:42.866298+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-16T15:44:44.081060+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":65,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5524076-a903-4d0a-af6a-caf474c4765d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6809","nonce":3111342741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6811","nonce":3111342741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6815","nonce":3111342741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6813","nonce":3111342741}]},"public_addr":"172.21.15.151:6809/3111342741","cluster_addr":"172.21.15.151:6811/3111342741","heartbeat_back_addr":"172.21.15.151:6815/3111342741","heartbeat_front_addr":"172.21.15.151:6813/3111342741","state":["exists","up"]},{"osd":1,"uuid":"dd25925f-0ae1-494e-8529-4dc205759db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6801","nonce":3368304087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6803","nonce":3368304087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6807","nonce":3368304087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6805","nonce":3368304087}]},"public_addr":"172.21.15.151:6801/3368304087","cluster_addr":"172.21.15.151:6803/3368304087","heartbeat_back_addr":"172.21.15.151:6807/3368304087","heartbeat_front_addr":"172.21.15.151:6805/3368304087","state":["exists","up"]},{"osd":2,"uuid":"c2f18115-013d-4da6-a0fe-cb978f4494b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6823","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6825","nonce":3079373271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6827","nonce":3079373271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6831","nonce":3079373271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6829","nonce":3079373271}]},"public_addr":"172.21.15.151:6825/3079373271","cluster_addr":"172.21.15.151:6827/3079373271","heartbeat_back_addr":"172.21.15.151:6831/3079373271","heartbeat_front_addr":"172.21.15.151:6829/3079373271","state":["exists","up"]},{"osd":3,"uuid":"36edb718-2fcb-4d88-8ddc-c321eed4279d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6817","nonce":1142509041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6819","nonce":1142509041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6824","nonce":1142509041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6821","nonce":1142509041}]},"public_addr":"172.21.15.151:6817/1142509041","cluster_addr":"172.21.15.151:6819/1142509041","heartbeat_back_addr":"172.21.15.151:6824/1142509041","heartbeat_front_addr":"172.21.15.151:6821/1142509041","state":["exists","up"]},{"osd":4,"uuid":"64d33708-cd0a-4236-abe3-1b646e82ab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6811","nonce":2735495271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6813","nonce":2735495271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6817","nonce":2735495271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6815","nonce":2735495271}]},"public_addr":"172.21.15.167:6811/2735495271","cluster_addr":"172.21.15.167:6813/2735495271","heartbeat_back_addr":"172.21.15.167:6817/2735495271","heartbeat_front_addr":"172.21.15.167:6815/2735495271","state":["exists","up"]},{"osd":5,"uuid":"f14b44a9-5910-458c-b518-09a8d0de94de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6827","nonce":1494916572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6829","nonce":1494916572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6833","nonce":1494916572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6831","nonce":1494916572}]},"public_addr":"172.21.15.167:6827/1494916572","cluster_addr":"172.21.15.167:6829/1494916572","heartbeat_back_addr":"172.21.15.167:6833/1494916572","heartbeat_front_addr":"172.21.15.167:6831/1494916572","state":["exists","up"]},{"osd":6,"uuid":"39335545-2654-4710-84b0-a9aa47cfcbba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6819","nonce":2538273273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6821","nonce":2538273273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6825","nonce":2538273273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6823","nonce":2538273273}]},"public_addr":"172.21.15.167:6819/2538273273","cluster_addr":"172.21.15.167:6821/2538273273","heartbeat_back_addr":"172.21.15.167:6825/2538273273","heartbeat_front_addr":"172.21.15.167:6823/2538273273","state":["exists","up"]},{"osd":7,"uuid":"4f8bfc81-53dd-4339-ac22-1a4007d9f963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6803","nonce":265072227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6805","nonce":265072227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6809","nonce":265072227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6807","nonce":265072227}]},"public_addr":"172.21.15.167:6803/265072227","cluster_addr":"172.21.15.167:6805/265072227","heartbeat_back_addr":"172.21.15.167:6809/265072227","heartbeat_front_addr":"172.21.15.167:6807/265072227","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.150359+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.084358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.722368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.332809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.320709+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.395153+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.935250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.801443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.a","mappings":[{"from":6,"to":4}]},{"pgid":"9.27","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:46:40.419 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.418+0000 7f9d24600640 1 -- 172.21.15.151:0/2994596276 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9cfc0794b0 msgr2=0x7f9cfc07b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:40.419 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.418+0000 7f9d24600640 1 --2- 172.21.15.151:0/2994596276 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9cfc0794b0 0x7f9cfc07b970 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9d08033000 tx=0x7f9d08004730 comp rx=0 tx=0).stop 2024-09-16T15:46:40.419 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.418+0000 7f9d24600640 1 -- 172.21.15.151:0/2994596276 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d1c0ad270 msgr2=0x7f9d1c154af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:40.419 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.418+0000 7f9d24600640 1 --2- 172.21.15.151:0/2994596276 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d1c0ad270 0x7f9d1c154af0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f9d1000d750 tx=0x7f9d1000dc10 comp rx=0 tx=0).stop 2024-09-16T15:46:40.419 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.418+0000 7f9d24600640 1 -- 172.21.15.151:0/2994596276 shutdown_connections 2024-09-16T15:46:40.419 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.418+0000 7f9d24600640 1 --2- 172.21.15.151:0/2994596276 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9cfc0794b0 0x7f9cfc07b970 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:40.419 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.418+0000 7f9d24600640 1 --2- 172.21.15.151:0/2994596276 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d1c0ccf00 0x7f9d1c15b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:40.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.418+0000 7f9d24600640 1 --2- 172.21.15.151:0/2994596276 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9d1c15f5a0 0x7f9d1c1f7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:40.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.418+0000 7f9d24600640 1 --2- 172.21.15.151:0/2994596276 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d1c0ad270 0x7f9d1c154af0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:40.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.418+0000 7f9d24600640 1 -- 172.21.15.151:0/2994596276 >> 172.21.15.151:0/2994596276 conn(0x7f9d1c0b1950 msgr2=0x7f9d1c0af5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:40.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.418+0000 7f9d24600640 1 -- 172.21.15.151:0/2994596276 shutdown_connections 2024-09-16T15:46:40.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.418+0000 7f9d24600640 1 -- 172.21.15.151:0/2994596276 wait complete. 2024-09-16T15:46:40.435 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-16T15:46:40.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.594+0000 7f15c2200640 1 -- 172.21.15.151:0/3654481207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15bc073fb0 msgr2=0x7f15bc074460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:40.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.594+0000 7f15c2200640 1 --2- 172.21.15.151:0/3654481207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15bc073fb0 0x7f15bc074460 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f15b4005fd0 tx=0x7f15b402a540 comp rx=0 tx=0).stop 2024-09-16T15:46:40.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.594+0000 7f15c2200640 1 -- 172.21.15.151:0/3654481207 shutdown_connections 2024-09-16T15:46:40.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.594+0000 7f15c2200640 1 --2- 172.21.15.151:0/3654481207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15bc073fb0 0x7f15bc074460 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:40.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.594+0000 7f15c2200640 1 --2- 172.21.15.151:0/3654481207 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f15bc15f5a0 0x7f15bc0b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:40.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.594+0000 7f15c2200640 1 -- 172.21.15.151:0/3654481207 >> 172.21.15.151:0/3654481207 conn(0x7f15bc0b1950 msgr2=0x7f15bc0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:40.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.595+0000 7f15c2200640 1 -- 172.21.15.151:0/3654481207 shutdown_connections 2024-09-16T15:46:40.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.595+0000 7f15c2200640 1 -- 172.21.15.151:0/3654481207 wait complete. 2024-09-16T15:46:40.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.596+0000 7f15c2200640 1 Processor -- start 2024-09-16T15:46:40.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.596+0000 7f15c2200640 1 -- start start 2024-09-16T15:46:40.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.597+0000 7f15c2200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15bc073fb0 0x7f15bc14c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:40.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.597+0000 7f15c2200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f15bc0ccf00 0x7f15bc0cf880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:40.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.597+0000 7f15c2200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15bc15f5a0 0x7f15bc0cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:40.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.597+0000 7f15c2200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f15bc06f2e0 con 0x7f15bc073fb0 2024-09-16T15:46:40.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.597+0000 7f15c2200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f15bc06f160 con 0x7f15bc15f5a0 2024-09-16T15:46:40.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.597+0000 7f15c2200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f15bc06f460 con 0x7f15bc0ccf00 2024-09-16T15:46:40.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.597+0000 7f15bb400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15bc073fb0 0x7f15bc14c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:40.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.597+0000 7f15bb400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15bc073fb0 0x7f15bc14c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47926/0 (socket says 172.21.15.151:47926) 2024-09-16T15:46:40.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.597+0000 7f15bb400640 1 -- 172.21.15.151:0/3873341217 learned_addr learned my addr 172.21.15.151:0/3873341217 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:46:40.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.598+0000 7f15bbe00640 1 --2- 172.21.15.151:0/3873341217 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15bc15f5a0 0x7f15bc0cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:40.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.598+0000 7f15baa00640 1 --2- 172.21.15.151:0/3873341217 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f15bc0ccf00 0x7f15bc0cf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:40.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.598+0000 7f15bb400640 1 -- 172.21.15.151:0/3873341217 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f15bc0ccf00 msgr2=0x7f15bc0cf880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:40.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.598+0000 7f15bb400640 1 --2- 172.21.15.151:0/3873341217 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f15bc0ccf00 0x7f15bc0cf880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:40.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.598+0000 7f15bb400640 1 -- 172.21.15.151:0/3873341217 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15bc15f5a0 msgr2=0x7f15bc0cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:40.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.598+0000 7f15bb400640 1 --2- 172.21.15.151:0/3873341217 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15bc15f5a0 0x7f15bc0cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:40.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.598+0000 7f15bb400640 1 -- 172.21.15.151:0/3873341217 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15bc14cc70 con 0x7f15bc073fb0 2024-09-16T15:46:40.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.598+0000 7f15bb400640 1 --2- 172.21.15.151:0/3873341217 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15bc073fb0 0x7f15bc14c730 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f15ac00ece0 tx=0x7f15ac00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:40.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.598+0000 7f15b3400640 1 -- 172.21.15.151:0/3873341217 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ac003b70 con 0x7f15bc073fb0 2024-09-16T15:46:40.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.598+0000 7f15b3400640 1 -- 172.21.15.151:0/3873341217 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f15ac003d10 con 0x7f15bc073fb0 2024-09-16T15:46:40.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.598+0000 7f15c2200640 1 -- 172.21.15.151:0/3873341217 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15bc1f80f0 con 0x7f15bc073fb0 2024-09-16T15:46:40.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.598+0000 7f15b3400640 1 -- 172.21.15.151:0/3873341217 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ac004280 con 0x7f15bc073fb0 2024-09-16T15:46:40.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.599+0000 7f15c2200640 1 -- 172.21.15.151:0/3873341217 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15bc1f86b0 con 0x7f15bc073fb0 2024-09-16T15:46:40.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.600+0000 7f15c2200640 1 -- 172.21.15.151:0/3873341217 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15bc06f9f0 con 0x7f15bc073fb0 2024-09-16T15:46:40.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.601+0000 7f15b3400640 1 -- 172.21.15.151:0/3873341217 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 24) ==== 102409+0+0 (secure 0 0 0) 0x7f15ac004420 con 0x7f15bc073fb0 2024-09-16T15:46:40.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.602+0000 7f15b3400640 1 --2- 172.21.15.151:0/3873341217 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f159c0794b0 0x7f159c07b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:40.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.602+0000 7f15b3400640 1 -- 172.21.15.151:0/3873341217 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5851+0+0 (secure 0 0 0) 0x7f15ac09a060 con 0x7f15bc073fb0 2024-09-16T15:46:40.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.602+0000 7f15baa00640 1 --2- 172.21.15.151:0/3873341217 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f159c0794b0 0x7f159c07b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:40.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.603+0000 7f15baa00640 1 --2- 172.21.15.151:0/3873341217 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f159c0794b0 0x7f159c07b970 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f15bc0cf5f0 tx=0x7f15a8009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:40.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.606+0000 7f15b3400640 1 -- 172.21.15.151:0/3873341217 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f15bc14cc70 con 0x7f15bc073fb0 2024-09-16T15:46:40.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.827+0000 7f15c2200640 1 -- 172.21.15.151:0/3873341217 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f15bc0ae780 con 0x7f15bc073fb0 2024-09-16T15:46:40.829 INFO:tasks.ceph.mon.a.smithi151.stderr:2024-09-16T15:46:40.828+0000 7f62c3200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T15:46:40.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.829+0000 7f15b3400640 1 -- 172.21.15.151:0/3873341217 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v38) ==== 161+0+0 (secure 0 0 0) 0x7f15ac064100 con 0x7f15bc073fb0 2024-09-16T15:46:40.831 INFO:teuthology.orchestra.run.smithi151.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-16T15:46:40.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.833+0000 7f15c2200640 1 -- 172.21.15.151:0/3873341217 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f159c0794b0 msgr2=0x7f159c07b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:40.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.833+0000 7f15c2200640 1 --2- 172.21.15.151:0/3873341217 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f159c0794b0 0x7f159c07b970 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f15bc0cf5f0 tx=0x7f15a8009040 comp rx=0 tx=0).stop 2024-09-16T15:46:40.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.833+0000 7f15c2200640 1 -- 172.21.15.151:0/3873341217 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15bc073fb0 msgr2=0x7f15bc14c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:40.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.833+0000 7f15c2200640 1 --2- 172.21.15.151:0/3873341217 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15bc073fb0 0x7f15bc14c730 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f15ac00ece0 tx=0x7f15ac00c6a0 comp rx=0 tx=0).stop 2024-09-16T15:46:40.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.833+0000 7f15c2200640 1 -- 172.21.15.151:0/3873341217 shutdown_connections 2024-09-16T15:46:40.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.833+0000 7f15c2200640 1 --2- 172.21.15.151:0/3873341217 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f159c0794b0 0x7f159c07b970 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:40.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.833+0000 7f15c2200640 1 --2- 172.21.15.151:0/3873341217 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f15bc15f5a0 0x7f15bc0cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:40.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.833+0000 7f15c2200640 1 --2- 172.21.15.151:0/3873341217 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f15bc0ccf00 0x7f15bc0cf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:40.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.833+0000 7f15c2200640 1 --2- 172.21.15.151:0/3873341217 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15bc073fb0 0x7f15bc14c730 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:40.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.833+0000 7f15c2200640 1 -- 172.21.15.151:0/3873341217 >> 172.21.15.151:0/3873341217 conn(0x7f15bc0b1950 msgr2=0x7f15bc15e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:40.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.833+0000 7f15c2200640 1 -- 172.21.15.151:0/3873341217 shutdown_connections 2024-09-16T15:46:40.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:40.834+0000 7f15c2200640 1 -- 172.21.15.151:0/3873341217 wait complete. 2024-09-16T15:46:40.853 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-16T15:46:40.904 INFO:tasks.ceph.mds.c.smithi151.stdout:starting mds.c at 2024-09-16T15:46:41.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:41.016+0000 7f01d3400640 1 -- 172.21.15.151:0/3660801587 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01b00041c0 msgr2=0x7f01b0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:41.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:41.016+0000 7f01d3400640 1 --2- 172.21.15.151:0/3660801587 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01b00041c0 0x7f01b0004800 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f01cc0b23e0 tx=0x7f01b0005830 comp rx=0 tx=0).stop 2024-09-16T15:46:41.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:41.016+0000 7f01d1400640 1 -- 172.21.15.151:0/3660801587 reap_dead start 2024-09-16T15:46:41.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:41.016+0000 7f01d3400640 1 -- 172.21.15.151:0/3660801587 shutdown_connections 2024-09-16T15:46:41.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:41.016+0000 7f01d3400640 1 -- 172.21.15.151:0/3660801587 >> 172.21.15.151:0/3660801587 conn(0x7f01cc0b1840 msgr2=0x7f01cc0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:41.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:41.016+0000 7f01d3400640 1 -- 172.21.15.151:0/3660801587 shutdown_connections 2024-09-16T15:46:41.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:41.016+0000 7f01d3400640 1 -- 172.21.15.151:0/3660801587 wait complete. 2024-09-16T15:46:41.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:41.018+0000 7f01d3400640 1 Processor -- start 2024-09-16T15:46:41.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:41.018+0000 7f01d3400640 1 -- start start 2024-09-16T15:46:41.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:41.019+0000 7f01d3400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01b00041c0 0x7f01cc1f3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:41.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:41.019+0000 7f01d3400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01cc14f630 0x7f01cc06f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:41.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:41.019+0000 7f01d3400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01cc06fc20 0x7f01cc1f64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:41.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:41.019+0000 7f01d3400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f01cc0cde60 con 0x7f01b00041c0 2024-09-16T15:46:41.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:41.019+0000 7f01d3400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f01cc0cdce0 con 0x7f01cc14f630 2024-09-16T15:46:41.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:41.019+0000 7f01d3400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f01cc074030 con 0x7f01cc06fc20 2024-09-16T15:46:41.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:41.019+0000 7f01d0a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01b00041c0 0x7f01cc1f3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:41.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:41.019+0000 7f01d0a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01b00041c0 0x7f01cc1f3b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47946/0 (socket says 172.21.15.151:47946) 2024-09-16T15:46:41.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:41.019+0000 7f01d0a00640 1 -- 172.21.15.151:0/3482731308 learned_addr learned my addr 172.21.15.151:0/3482731308 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:46:41.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:41.019+0000 7f01cbe00640 1 --2- 172.21.15.151:0/3482731308 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01cc14f630 0x7f01cc06f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:41.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:41.019+0000 7f01d1400640 1 --2- 172.21.15.151:0/3482731308 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01cc06fc20 0x7f01cc1f64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:41.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:41.020+0000 7f01d0a00640 1 -- 172.21.15.151:0/3482731308 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01cc06fc20 msgr2=0x7f01cc1f64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:41.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:41.020+0000 7f01d0a00640 1 --2- 172.21.15.151:0/3482731308 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01cc06fc20 0x7f01cc1f64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:41.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:41.020+0000 7f01d0a00640 1 -- 172.21.15.151:0/3482731308 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01cc14f630 msgr2=0x7f01cc06f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:41.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:41.020+0000 7f01d0a00640 1 --2- 172.21.15.151:0/3482731308 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01cc14f630 0x7f01cc06f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:41.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:41.020+0000 7f01d0a00640 1 -- 172.21.15.151:0/3482731308 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01cc1f7400 con 0x7f01b00041c0 2024-09-16T15:46:41.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:41.020+0000 7f01d1400640 1 --2- 172.21.15.151:0/3482731308 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01cc06fc20 0x7f01cc1f64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:46:41.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:41.020+0000 7f01d0a00640 1 --2- 172.21.15.151:0/3482731308 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01b00041c0 0x7f01cc1f3b10 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f01bc003e40 tx=0x7f01bc001800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:41.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:41.020+0000 7f01c8c00640 1 -- 172.21.15.151:0/3482731308 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01bc011700 con 0x7f01b00041c0 2024-09-16T15:46:41.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:41.020+0000 7f01c8c00640 1 -- 172.21.15.151:0/3482731308 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f01bc016020 con 0x7f01b00041c0 2024-09-16T15:46:41.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:41.020+0000 7f01c8c00640 1 -- 172.21.15.151:0/3482731308 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01bc011b40 con 0x7f01b00041c0 2024-09-16T15:46:41.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:41.020+0000 7f01d3400640 1 -- 172.21.15.151:0/3482731308 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01cc1f7120 con 0x7f01b00041c0 2024-09-16T15:46:41.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:41.020+0000 7f01d3400640 1 -- 172.21.15.151:0/3482731308 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01cc1f3310 con 0x7f01b00041c0 2024-09-16T15:46:41.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:41.022+0000 7f01d3400640 1 -- 172.21.15.151:0/3482731308 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01cc1f2e50 con 0x7f01b00041c0 2024-09-16T15:46:41.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:41.023+0000 7f01c8c00640 1 -- 172.21.15.151:0/3482731308 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 24) ==== 102409+0+0 (secure 0 0 0) 0x7f01bc011ce0 con 0x7f01b00041c0 2024-09-16T15:46:41.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:41.023+0000 7f01c8c00640 1 --2- 172.21.15.151:0/3482731308 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f01ac0794b0 0x7f01ac07b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:41.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:41.024+0000 7f01c8c00640 1 -- 172.21.15.151:0/3482731308 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5909+0+0 (secure 0 0 0) 0x7f01bc099f10 con 0x7f01b00041c0 2024-09-16T15:46:41.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:41.024+0000 7f01cbe00640 1 --2- 172.21.15.151:0/3482731308 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f01ac0794b0 0x7f01ac07b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:41.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:41.025+0000 7f01cbe00640 1 --2- 172.21.15.151:0/3482731308 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f01ac0794b0 0x7f01ac07b970 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f01b8036000 tx=0x7f01b80046c0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:41.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:41.028+0000 7f01c8c00640 1 -- 172.21.15.151:0/3482731308 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f01cc1f2e50 con 0x7f01b00041c0 2024-09-16T15:46:41.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:41.251+0000 7f01d3400640 1 -- 172.21.15.151:0/3482731308 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f01cc06aaf0 con 0x7f01b00041c0 2024-09-16T15:46:41.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:41.831+0000 7f01c8c00640 1 -- 172.21.15.151:0/3482731308 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v39) ==== 104+0+0 (secure 0 0 0) 0x7f01cc06aaf0 con 0x7f01b00041c0 2024-09-16T15:46:41.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:41.834+0000 7f01d3400640 1 -- 172.21.15.151:0/3482731308 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f01ac0794b0 msgr2=0x7f01ac07b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:41.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:41.834+0000 7f01d3400640 1 --2- 172.21.15.151:0/3482731308 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f01ac0794b0 0x7f01ac07b970 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f01b8036000 tx=0x7f01b80046c0 comp rx=0 tx=0).stop 2024-09-16T15:46:41.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:41.834+0000 7f01d3400640 1 -- 172.21.15.151:0/3482731308 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01b00041c0 msgr2=0x7f01cc1f3b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:41.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:41.834+0000 7f01d3400640 1 --2- 172.21.15.151:0/3482731308 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01b00041c0 0x7f01cc1f3b10 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f01bc003e40 tx=0x7f01bc001800 comp rx=0 tx=0).stop 2024-09-16T15:46:41.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:41.834+0000 7f01d3400640 1 -- 172.21.15.151:0/3482731308 shutdown_connections 2024-09-16T15:46:41.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:41.834+0000 7f01d3400640 1 --2- 172.21.15.151:0/3482731308 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f01ac0794b0 0x7f01ac07b970 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:41.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:41.834+0000 7f01d3400640 1 --2- 172.21.15.151:0/3482731308 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01cc14f630 0x7f01cc06f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:41.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:41.835+0000 7f01d3400640 1 --2- 172.21.15.151:0/3482731308 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01cc06fc20 0x7f01cc1f64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:41.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:41.835+0000 7f01d3400640 1 --2- 172.21.15.151:0/3482731308 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01b00041c0 0x7f01cc1f3b10 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:41.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:41.835+0000 7f01d3400640 1 -- 172.21.15.151:0/3482731308 >> 172.21.15.151:0/3482731308 conn(0x7f01cc0b1840 msgr2=0x7f01cc0b4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:41.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:41.835+0000 7f01d3400640 1 -- 172.21.15.151:0/3482731308 shutdown_connections 2024-09-16T15:46:41.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:41.835+0000 7f01d3400640 1 -- 172.21.15.151:0/3482731308 wait complete. 2024-09-16T15:46:41.855 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-16T15:46:42.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:42.022+0000 7f132d800640 1 -- 172.21.15.151:0/2158152755 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f132806f220 msgr2=0x7f132806f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:42.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:42.022+0000 7f132d800640 1 --2- 172.21.15.151:0/2158152755 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f132806f220 0x7f132806f6d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1320009900 tx=0x7f132002a610 comp rx=0 tx=0).stop 2024-09-16T15:46:42.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:42.022+0000 7f132d800640 1 -- 172.21.15.151:0/2158152755 shutdown_connections 2024-09-16T15:46:42.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:42.022+0000 7f132d800640 1 --2- 172.21.15.151:0/2158152755 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f132806f220 0x7f132806f6d0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:42.024 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:42.022+0000 7f132d800640 1 --2- 172.21.15.151:0/2158152755 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1328159aa0 0x7f13280741a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:42.024 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:42.022+0000 7f132d800640 1 -- 172.21.15.151:0/2158152755 >> 172.21.15.151:0/2158152755 conn(0x7f13280b18e0 msgr2=0x7f13280b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:42.024 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:42.022+0000 7f132d800640 1 -- 172.21.15.151:0/2158152755 shutdown_connections 2024-09-16T15:46:42.024 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:42.022+0000 7f132d800640 1 -- 172.21.15.151:0/2158152755 wait complete. 2024-09-16T15:46:42.024 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:42.023+0000 7f132d800640 1 Processor -- start 2024-09-16T15:46:42.024 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:42.024+0000 7f132d800640 1 -- start start 2024-09-16T15:46:42.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:42.024+0000 7f132d800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f132806f220 0x7f1328151690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:42.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:42.024+0000 7f132d800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1328159aa0 0x7f13280cf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:42.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:42.024+0000 7f132d800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f132815cff0 0x7f13280cff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:42.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:42.024+0000 7f132d800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f13280cc520 con 0x7f1328159aa0 2024-09-16T15:46:42.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:42.024+0000 7f132d800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f13280cc3a0 con 0x7f132806f220 2024-09-16T15:46:42.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:42.024+0000 7f132d800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f13280cc6a0 con 0x7f132815cff0 2024-09-16T15:46:42.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:42.025+0000 7f1326000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1328159aa0 0x7f13280cf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:42.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:42.025+0000 7f1326000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1328159aa0 0x7f13280cf9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47954/0 (socket says 172.21.15.151:47954) 2024-09-16T15:46:42.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:42.025+0000 7f1326000640 1 -- 172.21.15.151:0/3743130643 learned_addr learned my addr 172.21.15.151:0/3743130643 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:46:42.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:42.025+0000 7f1326a00640 1 --2- 172.21.15.151:0/3743130643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f132806f220 0x7f1328151690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:42.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:42.025+0000 7f1327400640 1 --2- 172.21.15.151:0/3743130643 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f132815cff0 0x7f13280cff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:42.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:42.025+0000 7f1326000640 1 -- 172.21.15.151:0/3743130643 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f132815cff0 msgr2=0x7f13280cff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:42.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:42.025+0000 7f1326000640 1 --2- 172.21.15.151:0/3743130643 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f132815cff0 0x7f13280cff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:42.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:42.025+0000 7f1326000640 1 -- 172.21.15.151:0/3743130643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f132806f220 msgr2=0x7f1328151690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:42.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:42.025+0000 7f1326000640 1 --2- 172.21.15.151:0/3743130643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f132806f220 0x7f1328151690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:42.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:42.025+0000 7f1326000640 1 -- 172.21.15.151:0/3743130643 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13280cdad0 con 0x7f1328159aa0 2024-09-16T15:46:42.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:42.026+0000 7f1326000640 1 --2- 172.21.15.151:0/3743130643 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1328159aa0 0x7f13280cf9c0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f131400c370 tx=0x7f131400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:42.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:42.026+0000 7f131ea00640 1 -- 172.21.15.151:0/3743130643 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1314017070 con 0x7f1328159aa0 2024-09-16T15:46:42.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:42.026+0000 7f131ea00640 1 -- 172.21.15.151:0/3743130643 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f13140053c0 con 0x7f1328159aa0 2024-09-16T15:46:42.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:42.026+0000 7f132d800640 1 -- 172.21.15.151:0/3743130643 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13280cddb0 con 0x7f1328159aa0 2024-09-16T15:46:42.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:42.026+0000 7f131ea00640 1 -- 172.21.15.151:0/3743130643 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1314003850 con 0x7f1328159aa0 2024-09-16T15:46:42.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:42.026+0000 7f132d800640 1 -- 172.21.15.151:0/3743130643 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13280ce370 con 0x7f1328159aa0 2024-09-16T15:46:42.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:42.027+0000 7f132d800640 1 -- 172.21.15.151:0/3743130643 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13280ccba0 con 0x7f1328159aa0 2024-09-16T15:46:42.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:42.028+0000 7f131ea00640 1 -- 172.21.15.151:0/3743130643 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 24) ==== 102409+0+0 (secure 0 0 0) 0x7f131401a070 con 0x7f1328159aa0 2024-09-16T15:46:42.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:42.029+0000 7f131ea00640 1 --2- 172.21.15.151:0/3743130643 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f13080794b0 0x7f130807b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:42.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:42.029+0000 7f131ea00640 1 -- 172.21.15.151:0/3743130643 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5909+0+0 (secure 0 0 0) 0x7f1314098830 con 0x7f1328159aa0 2024-09-16T15:46:42.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:42.030+0000 7f1326a00640 1 --2- 172.21.15.151:0/3743130643 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f13080794b0 0x7f130807b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:42.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:42.031+0000 7f1326a00640 1 --2- 172.21.15.151:0/3743130643 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f13080794b0 0x7f130807b970 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f13180099a0 tx=0x7f1318009500 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:42.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:42.034+0000 7f131ea00640 1 -- 172.21.15.151:0/3743130643 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1314062000 con 0x7f1328159aa0 2024-09-16T15:46:42.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:42.219+0000 7f132d800640 1 -- 172.21.15.151:0/3743130643 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f13280b1cc0 con 0x7f1328159aa0 2024-09-16T15:46:43.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:43.174+0000 7f131ea00640 1 -- 172.21.15.151:0/3743130643 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v67) ==== 181+0+0 (secure 0 0 0) 0x7f1314066a30 con 0x7f1328159aa0 2024-09-16T15:46:43.175 INFO:teuthology.orchestra.run.smithi151.stderr:pool 'cephfs_metadata' removed 2024-09-16T15:46:43.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:43.178+0000 7f132d800640 1 -- 172.21.15.151:0/3743130643 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f13080794b0 msgr2=0x7f130807b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:43.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:43.178+0000 7f132d800640 1 --2- 172.21.15.151:0/3743130643 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f13080794b0 0x7f130807b970 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f13180099a0 tx=0x7f1318009500 comp rx=0 tx=0).stop 2024-09-16T15:46:43.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:43.178+0000 7f132d800640 1 -- 172.21.15.151:0/3743130643 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1328159aa0 msgr2=0x7f13280cf9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:43.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:43.178+0000 7f132d800640 1 --2- 172.21.15.151:0/3743130643 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1328159aa0 0x7f13280cf9c0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f131400c370 tx=0x7f131400c830 comp rx=0 tx=0).stop 2024-09-16T15:46:43.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:43.179+0000 7f132d800640 1 -- 172.21.15.151:0/3743130643 shutdown_connections 2024-09-16T15:46:43.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:43.179+0000 7f132d800640 1 --2- 172.21.15.151:0/3743130643 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f13080794b0 0x7f130807b970 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:43.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:43.179+0000 7f132d800640 1 --2- 172.21.15.151:0/3743130643 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f132815cff0 0x7f13280cff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:43.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:43.179+0000 7f132d800640 1 --2- 172.21.15.151:0/3743130643 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1328159aa0 0x7f13280cf9c0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:43.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:43.179+0000 7f132d800640 1 --2- 172.21.15.151:0/3743130643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f132806f220 0x7f1328151690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:43.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:43.179+0000 7f132d800640 1 -- 172.21.15.151:0/3743130643 >> 172.21.15.151:0/3743130643 conn(0x7f13280b18e0 msgr2=0x7f13280ccdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:43.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:43.179+0000 7f132d800640 1 -- 172.21.15.151:0/3743130643 shutdown_connections 2024-09-16T15:46:43.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:43.179+0000 7f132d800640 1 -- 172.21.15.151:0/3743130643 wait complete. 2024-09-16T15:46:43.200 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-16T15:46:43.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:43.368+0000 7fed24c00640 1 -- 172.21.15.151:0/919968855 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fed04004800 msgr2=0x7fed04004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:43.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:43.368+0000 7fed24c00640 1 --2- 172.21.15.151:0/919968855 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fed04004800 0x7fed04004c80 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fed0c009b50 tx=0x7fed04005830 comp rx=0 tx=0).stop 2024-09-16T15:46:43.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:43.368+0000 7fed1fe00640 1 -- 172.21.15.151:0/919968855 reap_dead start 2024-09-16T15:46:43.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:43.368+0000 7fed24c00640 1 -- 172.21.15.151:0/919968855 shutdown_connections 2024-09-16T15:46:43.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:43.368+0000 7fed24c00640 1 -- 172.21.15.151:0/919968855 >> 172.21.15.151:0/919968855 conn(0x7fed200cfa70 msgr2=0x7fed200cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:43.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:43.368+0000 7fed24c00640 1 -- 172.21.15.151:0/919968855 shutdown_connections 2024-09-16T15:46:43.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:43.368+0000 7fed24c00640 1 -- 172.21.15.151:0/919968855 wait complete. 2024-09-16T15:46:43.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:43.369+0000 7fed24c00640 1 Processor -- start 2024-09-16T15:46:43.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:43.370+0000 7fed24c00640 1 -- start start 2024-09-16T15:46:43.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:43.370+0000 7fed24c00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fed04004800 0x7fed2014b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:43.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:43.371+0000 7fed24c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fed200ccc70 0x7fed200c8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:43.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:43.371+0000 7fed24c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed200c8d60 0x7fed2014dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:43.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:43.371+0000 7fed24c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fed200701d0 con 0x7fed200ccc70 2024-09-16T15:46:43.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:43.371+0000 7fed24c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fed20070050 con 0x7fed200c8d60 2024-09-16T15:46:43.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:43.371+0000 7fed24c00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fed2014bd80 con 0x7fed04004800 2024-09-16T15:46:43.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:43.371+0000 7fed1ea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fed200ccc70 0x7fed200c8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:43.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:43.371+0000 7fed1f400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fed04004800 0x7fed2014b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:43.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:43.371+0000 7fed1ea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fed200ccc70 0x7fed200c8820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:55660/0 (socket says 172.21.15.151:55660) 2024-09-16T15:46:43.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:43.371+0000 7fed1ea00640 1 -- 172.21.15.151:0/1686386202 learned_addr learned my addr 172.21.15.151:0/1686386202 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:46:43.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:43.371+0000 7fed1fe00640 1 --2- 172.21.15.151:0/1686386202 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed200c8d60 0x7fed2014dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:43.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:43.371+0000 7fed1ea00640 1 -- 172.21.15.151:0/1686386202 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fed04004800 msgr2=0x7fed2014b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:43.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:43.371+0000 7fed1ea00640 1 --2- 172.21.15.151:0/1686386202 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fed04004800 0x7fed2014b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:43.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:43.371+0000 7fed1ea00640 1 -- 172.21.15.151:0/1686386202 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed200c8d60 msgr2=0x7fed2014dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:43.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:43.371+0000 7fed1ea00640 1 --2- 172.21.15.151:0/1686386202 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed200c8d60 0x7fed2014dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:43.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:43.371+0000 7fed1ea00640 1 -- 172.21.15.151:0/1686386202 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed200c5aa0 con 0x7fed200ccc70 2024-09-16T15:46:43.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:43.372+0000 7fed1ea00640 1 --2- 172.21.15.151:0/1686386202 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fed200ccc70 0x7fed200c8820 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fed0c009b50 tx=0x7fed0c033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:43.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:43.372+0000 7fed17400640 1 -- 172.21.15.151:0/1686386202 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed0c038040 con 0x7fed200ccc70 2024-09-16T15:46:43.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:43.372+0000 7fed17400640 1 -- 172.21.15.151:0/1686386202 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fed0c043070 con 0x7fed200ccc70 2024-09-16T15:46:43.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:43.372+0000 7fed24c00640 1 -- 172.21.15.151:0/1686386202 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed200c5d80 con 0x7fed200ccc70 2024-09-16T15:46:43.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:43.372+0000 7fed17400640 1 -- 172.21.15.151:0/1686386202 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed0c033b60 con 0x7fed200ccc70 2024-09-16T15:46:43.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:43.372+0000 7fed24c00640 1 -- 172.21.15.151:0/1686386202 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed200c5790 con 0x7fed200ccc70 2024-09-16T15:46:43.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:43.373+0000 7fed24c00640 1 -- 172.21.15.151:0/1686386202 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed2014bf00 con 0x7fed200ccc70 2024-09-16T15:46:43.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:43.374+0000 7fed17400640 1 -- 172.21.15.151:0/1686386202 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 24) ==== 102409+0+0 (secure 0 0 0) 0x7fed0c02b8a0 con 0x7fed200ccc70 2024-09-16T15:46:43.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:43.375+0000 7fed17400640 1 --2- 172.21.15.151:0/1686386202 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fed000794b0 0x7fed0007b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:43.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:43.375+0000 7fed17400640 1 -- 172.21.15.151:0/1686386202 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5471+0+0 (secure 0 0 0) 0x7fed0c0b99b0 con 0x7fed200ccc70 2024-09-16T15:46:43.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:43.375+0000 7fed1f400640 1 --2- 172.21.15.151:0/1686386202 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fed000794b0 0x7fed0007b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:43.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:43.376+0000 7fed1f400640 1 --2- 172.21.15.151:0/1686386202 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fed000794b0 0x7fed0007b970 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fed10001220 tx=0x7fed10006720 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:43.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:43.379+0000 7fed17400640 1 -- 172.21.15.151:0/1686386202 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fed0c04c020 con 0x7fed200ccc70 2024-09-16T15:46:43.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:43.572+0000 7fed24c00640 1 -- 172.21.15.151:0/1686386202 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fed200cb1c0 con 0x7fed200ccc70 2024-09-16T15:46:44.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.179+0000 7fed17400640 1 -- 172.21.15.151:0/1686386202 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v68) ==== 169+0+0 (secure 0 0 0) 0x7fed0c083420 con 0x7fed200ccc70 2024-09-16T15:46:44.181 INFO:teuthology.orchestra.run.smithi151.stderr:pool 'cephfs_data' removed 2024-09-16T15:46:44.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.181+0000 7fed24c00640 1 -- 172.21.15.151:0/1686386202 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fed000794b0 msgr2=0x7fed0007b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:44.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.181+0000 7fed24c00640 1 --2- 172.21.15.151:0/1686386202 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fed000794b0 0x7fed0007b970 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fed10001220 tx=0x7fed10006720 comp rx=0 tx=0).stop 2024-09-16T15:46:44.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.181+0000 7fed24c00640 1 -- 172.21.15.151:0/1686386202 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fed200ccc70 msgr2=0x7fed200c8820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:44.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.181+0000 7fed24c00640 1 --2- 172.21.15.151:0/1686386202 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fed200ccc70 0x7fed200c8820 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fed0c009b50 tx=0x7fed0c033670 comp rx=0 tx=0).stop 2024-09-16T15:46:44.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.181+0000 7fed24c00640 1 -- 172.21.15.151:0/1686386202 shutdown_connections 2024-09-16T15:46:44.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.181+0000 7fed24c00640 1 --2- 172.21.15.151:0/1686386202 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fed000794b0 0x7fed0007b970 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:44.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.181+0000 7fed24c00640 1 --2- 172.21.15.151:0/1686386202 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed200c8d60 0x7fed2014dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:44.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.181+0000 7fed24c00640 1 --2- 172.21.15.151:0/1686386202 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fed200ccc70 0x7fed200c8820 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:44.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.181+0000 7fed24c00640 1 --2- 172.21.15.151:0/1686386202 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fed04004800 0x7fed2014b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:44.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.181+0000 7fed24c00640 1 -- 172.21.15.151:0/1686386202 >> 172.21.15.151:0/1686386202 conn(0x7fed200cfa70 msgr2=0x7fed200737c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:44.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.182+0000 7fed24c00640 1 -- 172.21.15.151:0/1686386202 shutdown_connections 2024-09-16T15:46:44.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.182+0000 7fed24c00640 1 -- 172.21.15.151:0/1686386202 wait complete. 2024-09-16T15:46:44.205 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-16T15:46:44.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.364+0000 7f989c400640 1 -- 172.21.15.151:0/4029833880 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9894073fb0 msgr2=0x7f9894074420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:44.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.364+0000 7f989c400640 1 --2- 172.21.15.151:0/4029833880 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9894073fb0 0x7f9894074420 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f98940cfbb0 tx=0x7f988c02a820 comp rx=0 tx=0).stop 2024-09-16T15:46:44.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.364+0000 7f989c400640 1 -- 172.21.15.151:0/4029833880 shutdown_connections 2024-09-16T15:46:44.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.364+0000 7f989c400640 1 --2- 172.21.15.151:0/4029833880 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9894073fb0 0x7f9894074420 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:44.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.364+0000 7f989c400640 1 --2- 172.21.15.151:0/4029833880 >> v2:172.21.15.167:3300/0 conn(0x7f98940ccf00 0x7f98940b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:44.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.364+0000 7f989c400640 1 -- 172.21.15.151:0/4029833880 >> 172.21.15.151:0/4029833880 conn(0x7f98940b1950 msgr2=0x7f98940b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:44.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.364+0000 7f989c400640 1 -- 172.21.15.151:0/4029833880 shutdown_connections 2024-09-16T15:46:44.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.364+0000 7f989c400640 1 -- 172.21.15.151:0/4029833880 wait complete. 2024-09-16T15:46:44.368 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.366+0000 7f989c400640 1 Processor -- start 2024-09-16T15:46:44.368 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.366+0000 7f989c400640 1 -- start start 2024-09-16T15:46:44.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.367+0000 7f989c400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9894073fb0 0x7f9894154e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:44.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.367+0000 7f989c400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f98940ccf00 0x7f98940b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:44.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.367+0000 7f989c400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f989415f5a0 0x7f98940b3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:44.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.367+0000 7f989c400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f989406f2e0 con 0x7f98940ccf00 2024-09-16T15:46:44.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.367+0000 7f989c400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f989406f160 con 0x7f989415f5a0 2024-09-16T15:46:44.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.367+0000 7f989c400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f989406f460 con 0x7f9894073fb0 2024-09-16T15:46:44.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.367+0000 7f9899000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f98940ccf00 0x7f98940b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:44.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.367+0000 7f9899000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f98940ccf00 0x7f98940b2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:55674/0 (socket says 172.21.15.151:55674) 2024-09-16T15:46:44.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.367+0000 7f9899000640 1 -- 172.21.15.151:0/3675174229 learned_addr learned my addr 172.21.15.151:0/3675174229 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:46:44.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.367+0000 7f989a400640 1 --2- 172.21.15.151:0/3675174229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f989415f5a0 0x7f98940b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:44.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.367+0000 7f9899a00640 1 --2- 172.21.15.151:0/3675174229 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9894073fb0 0x7f9894154e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:44.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.368+0000 7f9899000640 1 -- 172.21.15.151:0/3675174229 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9894073fb0 msgr2=0x7f9894154e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:44.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.368+0000 7f9899000640 1 --2- 172.21.15.151:0/3675174229 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9894073fb0 0x7f9894154e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:44.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.368+0000 7f9899000640 1 -- 172.21.15.151:0/3675174229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f989415f5a0 msgr2=0x7f98940b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:44.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.368+0000 7f9899000640 1 --2- 172.21.15.151:0/3675174229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f989415f5a0 0x7f98940b3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:44.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.368+0000 7f9899000640 1 -- 172.21.15.151:0/3675174229 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98941f89a0 con 0x7f98940ccf00 2024-09-16T15:46:44.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.368+0000 7f9899a00640 1 --2- 172.21.15.151:0/3675174229 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9894073fb0 0x7f9894154e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:46:44.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.368+0000 7f9899000640 1 --2- 172.21.15.151:0/3675174229 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f98940ccf00 0x7f98940b2d50 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f988000d980 tx=0x7f988000de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:44.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.368+0000 7f9891600640 1 -- 172.21.15.151:0/3675174229 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9880017070 con 0x7f98940ccf00 2024-09-16T15:46:44.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.368+0000 7f9891600640 1 -- 172.21.15.151:0/3675174229 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9880009d20 con 0x7f98940ccf00 2024-09-16T15:46:44.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.368+0000 7f989c400640 1 -- 172.21.15.151:0/3675174229 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98941f49a0 con 0x7f98940ccf00 2024-09-16T15:46:44.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.368+0000 7f9891600640 1 -- 172.21.15.151:0/3675174229 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98800106c0 con 0x7f98940ccf00 2024-09-16T15:46:44.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.368+0000 7f989c400640 1 -- 172.21.15.151:0/3675174229 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98941f8620 con 0x7f98940ccf00 2024-09-16T15:46:44.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.369+0000 7f989c400640 1 -- 172.21.15.151:0/3675174229 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f989406f9f0 con 0x7f98940ccf00 2024-09-16T15:46:44.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.370+0000 7f9891600640 1 -- 172.21.15.151:0/3675174229 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 24) ==== 102409+0+0 (secure 0 0 0) 0x7f9880010860 con 0x7f98940ccf00 2024-09-16T15:46:44.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.371+0000 7f9891600640 1 --2- 172.21.15.151:0/3675174229 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f98740794b0 0x7f987407b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:44.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.371+0000 7f9891600640 1 -- 172.21.15.151:0/3675174229 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 4999+0+0 (secure 0 0 0) 0x7f9880099660 con 0x7f98940ccf00 2024-09-16T15:46:44.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.372+0000 7f9899a00640 1 --2- 172.21.15.151:0/3675174229 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f98740794b0 0x7f987407b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:44.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.373+0000 7f9899a00640 1 --2- 172.21.15.151:0/3675174229 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f98740794b0 0x7f987407b970 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f9888007a50 tx=0x7f9888009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:44.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.375+0000 7f9891600640 1 -- 172.21.15.151:0/3675174229 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f98800630d0 con 0x7f98940ccf00 2024-09-16T15:46:44.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.562+0000 7f989c400640 1 -- 172.21.15.151:0/3675174229 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f989415cf80 con 0x7f98940ccf00 2024-09-16T15:46:44.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.566+0000 7f9891600640 1 -- 172.21.15.151:0/3675174229 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v36) ==== 71+0+0 (secure 0 0 0) 0x7f9880067b00 con 0x7f98940ccf00 2024-09-16T15:46:44.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.568+0000 7f989c400640 1 -- 172.21.15.151:0/3675174229 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f98740794b0 msgr2=0x7f987407b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:44.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.568+0000 7f989c400640 1 --2- 172.21.15.151:0/3675174229 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f98740794b0 0x7f987407b970 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f9888007a50 tx=0x7f9888009040 comp rx=0 tx=0).stop 2024-09-16T15:46:44.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.568+0000 7f989c400640 1 -- 172.21.15.151:0/3675174229 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f98940ccf00 msgr2=0x7f98940b2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:44.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.568+0000 7f989c400640 1 --2- 172.21.15.151:0/3675174229 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f98940ccf00 0x7f98940b2d50 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f988000d980 tx=0x7f988000de40 comp rx=0 tx=0).stop 2024-09-16T15:46:44.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.568+0000 7f989c400640 1 -- 172.21.15.151:0/3675174229 shutdown_connections 2024-09-16T15:46:44.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.568+0000 7f989c400640 1 --2- 172.21.15.151:0/3675174229 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f98740794b0 0x7f987407b970 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:44.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.568+0000 7f989c400640 1 --2- 172.21.15.151:0/3675174229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f989415f5a0 0x7f98940b3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:44.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.568+0000 7f989c400640 1 --2- 172.21.15.151:0/3675174229 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f98940ccf00 0x7f98940b2d50 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:44.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.568+0000 7f989c400640 1 --2- 172.21.15.151:0/3675174229 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9894073fb0 0x7f9894154e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:44.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.568+0000 7f989c400640 1 -- 172.21.15.151:0/3675174229 >> 172.21.15.151:0/3675174229 conn(0x7f98940b1950 msgr2=0x7f98940ae4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:44.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.569+0000 7f989c400640 1 -- 172.21.15.151:0/3675174229 shutdown_connections 2024-09-16T15:46:44.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.569+0000 7f989c400640 1 -- 172.21.15.151:0/3675174229 wait complete. 2024-09-16T15:46:44.587 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_less_than_cloner_threads' 2024-09-16T15:46:44.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.744+0000 7fea8ce00640 1 -- 172.21.15.151:0/596134250 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea88161440 msgr2=0x7fea880b5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:44.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.744+0000 7fea8ce00640 1 --2- 172.21.15.151:0/596134250 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea88161440 0x7fea880b5370 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fea880d4ea0 tx=0x7fea7802a530 comp rx=0 tx=0).stop 2024-09-16T15:46:44.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.744+0000 7fea8ce00640 1 -- 172.21.15.151:0/596134250 shutdown_connections 2024-09-16T15:46:44.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.744+0000 7fea8ce00640 1 --2- 172.21.15.151:0/596134250 >> v2:172.21.15.167:3300/0 conn(0x7fea880b34b0 0x7fea880b3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:44.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.744+0000 7fea8ce00640 1 --2- 172.21.15.151:0/596134250 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea88161440 0x7fea880b5370 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:44.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.744+0000 7fea8ce00640 1 -- 172.21.15.151:0/596134250 >> 172.21.15.151:0/596134250 conn(0x7fea880d3df0 msgr2=0x7fea880d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:44.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.744+0000 7fea8ce00640 1 -- 172.21.15.151:0/596134250 shutdown_connections 2024-09-16T15:46:44.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.744+0000 7fea8ce00640 1 -- 172.21.15.151:0/596134250 wait complete. 2024-09-16T15:46:44.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.746+0000 7fea8ce00640 1 Processor -- start 2024-09-16T15:46:44.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.746+0000 7fea8ce00640 1 -- start start 2024-09-16T15:46:44.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.747+0000 7fea8ce00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea880b34b0 0x7fea881f45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:44.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.747+0000 7fea8ce00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea880b3e50 0x7fea88156180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:44.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.747+0000 7fea8ce00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea88161440 0x7fea8806df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:44.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.747+0000 7fea8ce00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fea881619c0 con 0x7fea880b34b0 2024-09-16T15:46:44.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.747+0000 7fea8ce00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fea880b2990 con 0x7fea880b3e50 2024-09-16T15:46:44.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.747+0000 7fea8ce00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fea880d2470 con 0x7fea88161440 2024-09-16T15:46:44.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.747+0000 7fea87400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea880b34b0 0x7fea881f45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:44.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.747+0000 7fea87400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea880b34b0 0x7fea881f45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:55688/0 (socket says 172.21.15.151:55688) 2024-09-16T15:46:44.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.747+0000 7fea87400640 1 -- 172.21.15.151:0/997012192 learned_addr learned my addr 172.21.15.151:0/997012192 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:46:44.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.748+0000 7fea86a00640 1 --2- 172.21.15.151:0/997012192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea880b3e50 0x7fea88156180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:44.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.748+0000 7fea87e00640 1 --2- 172.21.15.151:0/997012192 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea88161440 0x7fea8806df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:44.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.748+0000 7fea87400640 1 -- 172.21.15.151:0/997012192 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea88161440 msgr2=0x7fea8806df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:44.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.748+0000 7fea87400640 1 --2- 172.21.15.151:0/997012192 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea88161440 0x7fea8806df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:44.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.748+0000 7fea87400640 1 -- 172.21.15.151:0/997012192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea880b3e50 msgr2=0x7fea88156180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:44.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.748+0000 7fea87400640 1 --2- 172.21.15.151:0/997012192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea880b3e50 0x7fea88156180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:44.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.748+0000 7fea87400640 1 -- 172.21.15.151:0/997012192 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea881f4c20 con 0x7fea880b34b0 2024-09-16T15:46:44.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.748+0000 7fea86a00640 1 --2- 172.21.15.151:0/997012192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea880b3e50 0x7fea88156180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:46:44.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.748+0000 7fea87400640 1 --2- 172.21.15.151:0/997012192 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea880b34b0 0x7fea881f45f0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fea78034040 tx=0x7fea78004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:44.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.748+0000 7fea7f400640 1 -- 172.21.15.151:0/997012192 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7803e030 con 0x7fea880b34b0 2024-09-16T15:46:44.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.748+0000 7fea7f400640 1 -- 172.21.15.151:0/997012192 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fea780047a0 con 0x7fea880b34b0 2024-09-16T15:46:44.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.748+0000 7fea7f400640 1 -- 172.21.15.151:0/997012192 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea78035040 con 0x7fea880b34b0 2024-09-16T15:46:44.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.748+0000 7fea8ce00640 1 -- 172.21.15.151:0/997012192 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea881f4f00 con 0x7fea880b34b0 2024-09-16T15:46:44.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.748+0000 7fea8ce00640 1 -- 172.21.15.151:0/997012192 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea881f54c0 con 0x7fea880b34b0 2024-09-16T15:46:44.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.750+0000 7fea8ce00640 1 -- 172.21.15.151:0/997012192 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea880b3910 con 0x7fea880b34b0 2024-09-16T15:46:44.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.751+0000 7fea7f400640 1 -- 172.21.15.151:0/997012192 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 24) ==== 102409+0+0 (secure 0 0 0) 0x7fea78041030 con 0x7fea880b34b0 2024-09-16T15:46:44.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.752+0000 7fea7f400640 1 --2- 172.21.15.151:0/997012192 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fea680794e0 0x7fea6807b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:44.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.752+0000 7fea7f400640 1 -- 172.21.15.151:0/997012192 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 4999+0+0 (secure 0 0 0) 0x7fea780b9cc0 con 0x7fea880b34b0 2024-09-16T15:46:44.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.752+0000 7fea86a00640 1 --2- 172.21.15.151:0/997012192 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fea680794e0 0x7fea6807b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:44.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.753+0000 7fea86a00640 1 --2- 172.21.15.151:0/997012192 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fea680794e0 0x7fea6807b9a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fea8815f920 tx=0x7fea74005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:44.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:44.756+0000 7fea7f400640 1 -- 172.21.15.151:0/997012192 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fea78083de0 con 0x7fea880b34b0 2024-09-16T15:46:45.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.009+0000 7fea8ce00640 1 -- 172.21.15.151:0/997012192 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_less_than_cloner_threads"]} v 0) -- 0x7fea881566c0 con 0x7fea880b34b0 2024-09-16T15:46:45.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.180+0000 7fea7f400640 1 -- 172.21.15.151:0/997012192 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_less_than_cloner_threads"]}]=0 v326) ==== 167+0+0 (secure 0 0 0) 0x7fea780836e0 con 0x7fea880b34b0 2024-09-16T15:46:45.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.184+0000 7fea8ce00640 1 -- 172.21.15.151:0/997012192 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fea680794e0 msgr2=0x7fea6807b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:45.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.184+0000 7fea8ce00640 1 --2- 172.21.15.151:0/997012192 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fea680794e0 0x7fea6807b9a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fea8815f920 tx=0x7fea74005f90 comp rx=0 tx=0).stop 2024-09-16T15:46:45.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.184+0000 7fea8ce00640 1 -- 172.21.15.151:0/997012192 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea880b34b0 msgr2=0x7fea881f45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:45.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.184+0000 7fea8ce00640 1 --2- 172.21.15.151:0/997012192 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea880b34b0 0x7fea881f45f0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fea78034040 tx=0x7fea78004c80 comp rx=0 tx=0).stop 2024-09-16T15:46:45.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.184+0000 7fea8ce00640 1 -- 172.21.15.151:0/997012192 shutdown_connections 2024-09-16T15:46:45.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.184+0000 7fea8ce00640 1 --2- 172.21.15.151:0/997012192 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fea680794e0 0x7fea6807b9a0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:45.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.184+0000 7fea8ce00640 1 --2- 172.21.15.151:0/997012192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea880b3e50 0x7fea88156180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:45.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.184+0000 7fea8ce00640 1 --2- 172.21.15.151:0/997012192 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea88161440 0x7fea8806df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:45.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.184+0000 7fea8ce00640 1 --2- 172.21.15.151:0/997012192 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea880b34b0 0x7fea881f45f0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:45.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.184+0000 7fea8ce00640 1 -- 172.21.15.151:0/997012192 >> 172.21.15.151:0/997012192 conn(0x7fea880d3df0 msgr2=0x7fea880d08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:45.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.184+0000 7fea8ce00640 1 -- 172.21.15.151:0/997012192 shutdown_connections 2024-09-16T15:46:45.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.184+0000 7fea8ce00640 1 -- 172.21.15.151:0/997012192 wait complete. 2024-09-16T15:46:45.202 INFO:tasks.cephfs_test_runner:Test that one progress bar is printed in output of "ceph status" output ... ok 2024-09-16T15:46:45.202 INFO:tasks.cephfs_test_runner:Starting test: test_clones_more_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that 2 progress bars are printed in output of "ceph status" 2024-09-16T15:46:45.202 INFO:tasks.cephfs_test_runner:test_clones_more_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-09-16T15:46:45.203 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:46:45.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.360+0000 7fc124a00640 1 -- 172.21.15.151:0/1563991073 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1201616b0 msgr2=0x7fc1200d21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:45.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.360+0000 7fc124a00640 1 --2- 172.21.15.151:0/1563991073 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1201616b0 0x7fc1200d21a0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fc1200c7390 tx=0x7fc11802aac0 comp rx=0 tx=0).stop 2024-09-16T15:46:45.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.360+0000 7fc124a00640 1 -- 172.21.15.151:0/1563991073 shutdown_connections 2024-09-16T15:46:45.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.360+0000 7fc124a00640 1 --2- 172.21.15.151:0/1563991073 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1201616b0 0x7fc1200d21a0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:45.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.360+0000 7fc124a00640 1 --2- 172.21.15.151:0/1563991073 >> v2:172.21.15.167:3300/0 conn(0x7fc1200d4980 0x7fc1200d1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:45.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.360+0000 7fc124a00640 1 -- 172.21.15.151:0/1563991073 >> 172.21.15.151:0/1563991073 conn(0x7fc1200d3df0 msgr2=0x7fc1200d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:45.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.360+0000 7fc124a00640 1 -- 172.21.15.151:0/1563991073 shutdown_connections 2024-09-16T15:46:45.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.360+0000 7fc124a00640 1 -- 172.21.15.151:0/1563991073 wait complete. 2024-09-16T15:46:45.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.361+0000 7fc124a00640 1 Processor -- start 2024-09-16T15:46:45.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.362+0000 7fc124a00640 1 -- start start 2024-09-16T15:46:45.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.362+0000 7fc124a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1200d4980 0x7fc1201f7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:45.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.362+0000 7fc124a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1201612d0 0x7fc12006e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:45.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.362+0000 7fc124a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1201616b0 0x7fc12015dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:45.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.362+0000 7fc124a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc1200b1bb0 con 0x7fc1201616b0 2024-09-16T15:46:45.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.362+0000 7fc124a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc1200b1a30 con 0x7fc1200d4980 2024-09-16T15:46:45.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.362+0000 7fc124a00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fc1200b1d30 con 0x7fc1201612d0 2024-09-16T15:46:45.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.363+0000 7fc11fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1201616b0 0x7fc12015dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:45.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.363+0000 7fc11fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1201616b0 0x7fc12015dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:55702/0 (socket says 172.21.15.151:55702) 2024-09-16T15:46:45.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.363+0000 7fc11fe00640 1 -- 172.21.15.151:0/2808899432 learned_addr learned my addr 172.21.15.151:0/2808899432 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:46:45.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.363+0000 7fc11f400640 1 --2- 172.21.15.151:0/2808899432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1200d4980 0x7fc1201f7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:45.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.363+0000 7fc11ea00640 1 --2- 172.21.15.151:0/2808899432 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1201612d0 0x7fc12006e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:45.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.363+0000 7fc11fe00640 1 -- 172.21.15.151:0/2808899432 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1201612d0 msgr2=0x7fc12006e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:45.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.363+0000 7fc11fe00640 1 --2- 172.21.15.151:0/2808899432 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1201612d0 0x7fc12006e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:45.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.363+0000 7fc11fe00640 1 -- 172.21.15.151:0/2808899432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1200d4980 msgr2=0x7fc1201f7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:45.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.363+0000 7fc11fe00640 1 --2- 172.21.15.151:0/2808899432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1200d4980 0x7fc1201f7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:45.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.363+0000 7fc11fe00640 1 -- 172.21.15.151:0/2808899432 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc120156460 con 0x7fc1201616b0 2024-09-16T15:46:45.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.364+0000 7fc11fe00640 1 --2- 172.21.15.151:0/2808899432 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1201616b0 0x7fc12015dd50 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fc118009d60 tx=0x7fc118003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:45.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.364+0000 7fc117400640 1 -- 172.21.15.151:0/2808899432 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc118044070 con 0x7fc1201616b0 2024-09-16T15:46:45.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.364+0000 7fc117400640 1 -- 172.21.15.151:0/2808899432 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc118003e50 con 0x7fc1201616b0 2024-09-16T15:46:45.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.364+0000 7fc117400640 1 -- 172.21.15.151:0/2808899432 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1180044b0 con 0x7fc1201616b0 2024-09-16T15:46:45.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.364+0000 7fc124a00640 1 -- 172.21.15.151:0/2808899432 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc12015e290 con 0x7fc1201616b0 2024-09-16T15:46:45.368 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.364+0000 7fc124a00640 1 -- 172.21.15.151:0/2808899432 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1201f8560 con 0x7fc1201616b0 2024-09-16T15:46:45.368 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.365+0000 7fc124a00640 1 -- 172.21.15.151:0/2808899432 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1200b2250 con 0x7fc1201616b0 2024-09-16T15:46:45.368 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.366+0000 7fc117400640 1 -- 172.21.15.151:0/2808899432 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 24) ==== 102409+0+0 (secure 0 0 0) 0x7fc118004650 con 0x7fc1201616b0 2024-09-16T15:46:45.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.367+0000 7fc117400640 1 --2- 172.21.15.151:0/2808899432 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc1000794b0 0x7fc10007b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:45.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.367+0000 7fc117400640 1 -- 172.21.15.151:0/2808899432 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 4999+0+0 (secure 0 0 0) 0x7fc1180b9a50 con 0x7fc1201616b0 2024-09-16T15:46:45.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.367+0000 7fc11f400640 1 --2- 172.21.15.151:0/2808899432 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc1000794b0 0x7fc10007b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:45.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.368+0000 7fc11f400640 1 --2- 172.21.15.151:0/2808899432 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc1000794b0 0x7fc10007b970 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc1100017c0 tx=0x7fc110009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:45.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.371+0000 7fc117400640 1 -- 172.21.15.151:0/2808899432 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc118032030 con 0x7fc1201616b0 2024-09-16T15:46:45.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.558+0000 7fc124a00640 1 -- 172.21.15.151:0/2808899432 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc12015fac0 con 0x7fc1201616b0 2024-09-16T15:46:45.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.559+0000 7fc117400640 1 -- 172.21.15.151:0/2808899432 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) ==== 74+0+13181 (secure 0 0 0) 0x7fc118083240 con 0x7fc1201616b0 2024-09-16T15:46:45.561 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:46:45.561 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":68,"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","created":"2024-09-16T15:38:56.348535+0000","modified":"2024-09-16T15:46:44.174902+0000","last_up_change":"2024-09-16T15:39:10.681384+0000","last_in_change":"2024-09-16T15:39:00.544477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:39:11.602252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5524076-a903-4d0a-af6a-caf474c4765d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6809","nonce":3111342741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6811","nonce":3111342741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6815","nonce":3111342741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6813","nonce":3111342741}]},"public_addr":"172.21.15.151:6809/3111342741","cluster_addr":"172.21.15.151:6811/3111342741","heartbeat_back_addr":"172.21.15.151:6815/3111342741","heartbeat_front_addr":"172.21.15.151:6813/3111342741","state":["exists","up"]},{"osd":1,"uuid":"dd25925f-0ae1-494e-8529-4dc205759db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6801","nonce":3368304087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6803","nonce":3368304087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6807","nonce":3368304087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6805","nonce":3368304087}]},"public_addr":"172.21.15.151:6801/3368304087","cluster_addr":"172.21.15.151:6803/3368304087","heartbeat_back_addr":"172.21.15.151:6807/3368304087","heartbeat_front_addr":"172.21.15.151:6805/3368304087","state":["exists","up"]},{"osd":2,"uuid":"c2f18115-013d-4da6-a0fe-cb978f4494b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6823","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6825","nonce":3079373271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6827","nonce":3079373271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6831","nonce":3079373271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6829","nonce":3079373271}]},"public_addr":"172.21.15.151:6825/3079373271","cluster_addr":"172.21.15.151:6827/3079373271","heartbeat_back_addr":"172.21.15.151:6831/3079373271","heartbeat_front_addr":"172.21.15.151:6829/3079373271","state":["exists","up"]},{"osd":3,"uuid":"36edb718-2fcb-4d88-8ddc-c321eed4279d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6817","nonce":1142509041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6819","nonce":1142509041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6824","nonce":1142509041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6821","nonce":1142509041}]},"public_addr":"172.21.15.151:6817/1142509041","cluster_addr":"172.21.15.151:6819/1142509041","heartbeat_back_addr":"172.21.15.151:6824/1142509041","heartbeat_front_addr":"172.21.15.151:6821/1142509041","state":["exists","up"]},{"osd":4,"uuid":"64d33708-cd0a-4236-abe3-1b646e82ab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6811","nonce":2735495271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6813","nonce":2735495271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6817","nonce":2735495271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6815","nonce":2735495271}]},"public_addr":"172.21.15.167:6811/2735495271","cluster_addr":"172.21.15.167:6813/2735495271","heartbeat_back_addr":"172.21.15.167:6817/2735495271","heartbeat_front_addr":"172.21.15.167:6815/2735495271","state":["exists","up"]},{"osd":5,"uuid":"f14b44a9-5910-458c-b518-09a8d0de94de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6827","nonce":1494916572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6829","nonce":1494916572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6833","nonce":1494916572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6831","nonce":1494916572}]},"public_addr":"172.21.15.167:6827/1494916572","cluster_addr":"172.21.15.167:6829/1494916572","heartbeat_back_addr":"172.21.15.167:6833/1494916572","heartbeat_front_addr":"172.21.15.167:6831/1494916572","state":["exists","up"]},{"osd":6,"uuid":"39335545-2654-4710-84b0-a9aa47cfcbba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6819","nonce":2538273273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6821","nonce":2538273273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6825","nonce":2538273273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6823","nonce":2538273273}]},"public_addr":"172.21.15.167:6819/2538273273","cluster_addr":"172.21.15.167:6821/2538273273","heartbeat_back_addr":"172.21.15.167:6825/2538273273","heartbeat_front_addr":"172.21.15.167:6823/2538273273","state":["exists","up"]},{"osd":7,"uuid":"4f8bfc81-53dd-4339-ac22-1a4007d9f963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6803","nonce":265072227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6805","nonce":265072227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6809","nonce":265072227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6807","nonce":265072227}]},"public_addr":"172.21.15.167:6803/265072227","cluster_addr":"172.21.15.167:6805/265072227","heartbeat_back_addr":"172.21.15.167:6809/265072227","heartbeat_front_addr":"172.21.15.167:6807/265072227","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.150359+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.084358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.722368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.332809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.320709+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.395153+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.935250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.801443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.151:6835/1724192711":"2024-09-17T15:46:40.828525+0000","172.21.15.151:6833/1724192711":"2024-09-17T15:46:40.828525+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:46:45.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.561+0000 7fc124a00640 1 -- 172.21.15.151:0/2808899432 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc1000794b0 msgr2=0x7fc10007b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:45.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.561+0000 7fc124a00640 1 --2- 172.21.15.151:0/2808899432 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc1000794b0 0x7fc10007b970 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc1100017c0 tx=0x7fc110009040 comp rx=0 tx=0).stop 2024-09-16T15:46:45.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.561+0000 7fc124a00640 1 -- 172.21.15.151:0/2808899432 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1201616b0 msgr2=0x7fc12015dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:45.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.561+0000 7fc124a00640 1 --2- 172.21.15.151:0/2808899432 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1201616b0 0x7fc12015dd50 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fc118009d60 tx=0x7fc118003ce0 comp rx=0 tx=0).stop 2024-09-16T15:46:45.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.562+0000 7fc124a00640 1 -- 172.21.15.151:0/2808899432 shutdown_connections 2024-09-16T15:46:45.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.562+0000 7fc124a00640 1 --2- 172.21.15.151:0/2808899432 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc1000794b0 0x7fc10007b970 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:45.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.562+0000 7fc124a00640 1 --2- 172.21.15.151:0/2808899432 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1201616b0 0x7fc12015dd50 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:45.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.562+0000 7fc124a00640 1 --2- 172.21.15.151:0/2808899432 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1201612d0 0x7fc12006e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:45.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.562+0000 7fc124a00640 1 --2- 172.21.15.151:0/2808899432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1200d4980 0x7fc1201f7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:45.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.562+0000 7fc124a00640 1 -- 172.21.15.151:0/2808899432 >> 172.21.15.151:0/2808899432 conn(0x7fc1200d3df0 msgr2=0x7fc1200ce960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:45.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.562+0000 7fc124a00640 1 -- 172.21.15.151:0/2808899432 shutdown_connections 2024-09-16T15:46:45.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.562+0000 7fc124a00640 1 -- 172.21.15.151:0/2808899432 wait complete. 2024-09-16T15:46:45.581 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T15:39:11.602252+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-16T15:46:45.582 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-16T15:46:45.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.743+0000 7fd82a400640 1 -- 172.21.15.151:0/2591434518 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8240d0930 msgr2=0x7fd8240ce5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:45.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.743+0000 7fd82a400640 1 --2- 172.21.15.151:0/2591434518 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8240d0930 0x7fd8240ce5e0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fd82406d760 tx=0x7fd81c02aa10 comp rx=0 tx=0).stop 2024-09-16T15:46:45.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.743+0000 7fd82a400640 1 -- 172.21.15.151:0/2591434518 shutdown_connections 2024-09-16T15:46:45.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.743+0000 7fd82a400640 1 --2- 172.21.15.151:0/2591434518 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8240d0930 0x7fd8240ce5e0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:45.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.743+0000 7fd82a400640 1 --2- 172.21.15.151:0/2591434518 >> v2:172.21.15.167:3300/0 conn(0x7fd8240b34d0 0x7fd8240d03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:45.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.743+0000 7fd82a400640 1 --2- 172.21.15.151:0/2591434518 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd82415f530 0x7fd8240cfeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:45.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.743+0000 7fd82a400640 1 -- 172.21.15.151:0/2591434518 >> 172.21.15.151:0/2591434518 conn(0x7fd8240d1e30 msgr2=0x7fd8240d2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:45.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.743+0000 7fd82a400640 1 -- 172.21.15.151:0/2591434518 shutdown_connections 2024-09-16T15:46:45.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.743+0000 7fd82a400640 1 -- 172.21.15.151:0/2591434518 wait complete. 2024-09-16T15:46:45.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.744+0000 7fd82a400640 1 Processor -- start 2024-09-16T15:46:45.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.744+0000 7fd82a400640 1 -- start start 2024-09-16T15:46:45.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.745+0000 7fd82a400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8240b34d0 0x7fd8240cbb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:45.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.745+0000 7fd82a400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8240d0930 0x7fd824158ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:45.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.746+0000 7fd82a400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd82415f530 0x7fd824157eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:45.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.746+0000 7fd82a400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd8240ceeb0 con 0x7fd82415f530 2024-09-16T15:46:45.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.746+0000 7fd82a400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd8240ced30 con 0x7fd8240b34d0 2024-09-16T15:46:45.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.746+0000 7fd82a400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fd8240cf030 con 0x7fd8240d0930 2024-09-16T15:46:45.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.746+0000 7fd823e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd82415f530 0x7fd824157eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:45.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.746+0000 7fd823e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd82415f530 0x7fd824157eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:55722/0 (socket says 172.21.15.151:55722) 2024-09-16T15:46:45.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.746+0000 7fd823e00640 1 -- 172.21.15.151:0/1876547832 learned_addr learned my addr 172.21.15.151:0/1876547832 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:46:45.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.746+0000 7fd823400640 1 --2- 172.21.15.151:0/1876547832 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8240b34d0 0x7fd8240cbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:45.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.746+0000 7fd822a00640 1 --2- 172.21.15.151:0/1876547832 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8240d0930 0x7fd824158ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:45.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.746+0000 7fd823e00640 1 -- 172.21.15.151:0/1876547832 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8240d0930 msgr2=0x7fd824158ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:45.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.746+0000 7fd823e00640 1 --2- 172.21.15.151:0/1876547832 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8240d0930 0x7fd824158ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:45.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.746+0000 7fd823e00640 1 -- 172.21.15.151:0/1876547832 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8240b34d0 msgr2=0x7fd8240cbb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:45.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.746+0000 7fd823e00640 1 --2- 172.21.15.151:0/1876547832 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8240b34d0 0x7fd8240cbb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:45.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.746+0000 7fd823e00640 1 -- 172.21.15.151:0/1876547832 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8241f4c70 con 0x7fd82415f530 2024-09-16T15:46:45.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.747+0000 7fd823e00640 1 --2- 172.21.15.151:0/1876547832 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd82415f530 0x7fd824157eb0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fd81c009d60 tx=0x7fd81c003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:45.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.747+0000 7fd81b400640 1 -- 172.21.15.151:0/1876547832 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd81c047070 con 0x7fd82415f530 2024-09-16T15:46:45.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.747+0000 7fd81b400640 1 -- 172.21.15.151:0/1876547832 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd81c004250 con 0x7fd82415f530 2024-09-16T15:46:45.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.747+0000 7fd82a400640 1 -- 172.21.15.151:0/1876547832 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8240cc080 con 0x7fd82415f530 2024-09-16T15:46:45.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.747+0000 7fd81b400640 1 -- 172.21.15.151:0/1876547832 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd81c033050 con 0x7fd82415f530 2024-09-16T15:46:45.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.747+0000 7fd82a400640 1 -- 172.21.15.151:0/1876547832 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8241f8aa0 con 0x7fd82415f530 2024-09-16T15:46:45.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.748+0000 7fd82a400640 1 -- 172.21.15.151:0/1876547832 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8240cf5f0 con 0x7fd82415f530 2024-09-16T15:46:45.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.750+0000 7fd81b400640 1 -- 172.21.15.151:0/1876547832 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 24) ==== 102409+0+0 (secure 0 0 0) 0x7fd81c04a070 con 0x7fd82415f530 2024-09-16T15:46:45.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.751+0000 7fd81b400640 1 --2- 172.21.15.151:0/1876547832 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd8040794b0 0x7fd80407b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:45.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.751+0000 7fd81b400640 1 -- 172.21.15.151:0/1876547832 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 4999+0+0 (secure 0 0 0) 0x7fd81c0baae0 con 0x7fd82415f530 2024-09-16T15:46:45.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.752+0000 7fd823400640 1 --2- 172.21.15.151:0/1876547832 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd8040794b0 0x7fd80407b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:45.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.752+0000 7fd823400640 1 --2- 172.21.15.151:0/1876547832 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd8040794b0 0x7fd80407b970 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd814005fd0 tx=0x7fd814005850 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:45.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.754+0000 7fd81b400640 1 -- 172.21.15.151:0/1876547832 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd81c051080 con 0x7fd82415f530 2024-09-16T15:46:45.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.939+0000 7fd82a400640 1 -- 172.21.15.151:0/1876547832 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd8240715a0 con 0x7fd82415f530 2024-09-16T15:46:45.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.939+0000 7fd81b400640 1 -- 172.21.15.151:0/1876547832 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v68) ==== 93+0+10 (secure 0 0 0) 0x7fd81c084960 con 0x7fd82415f530 2024-09-16T15:46:45.942 INFO:teuthology.orchestra.run.smithi151.stdout:pg_num: 1 2024-09-16T15:46:45.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.942+0000 7fd82a400640 1 -- 172.21.15.151:0/1876547832 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd8040794b0 msgr2=0x7fd80407b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:45.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.942+0000 7fd82a400640 1 --2- 172.21.15.151:0/1876547832 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd8040794b0 0x7fd80407b970 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd814005fd0 tx=0x7fd814005850 comp rx=0 tx=0).stop 2024-09-16T15:46:45.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.942+0000 7fd82a400640 1 -- 172.21.15.151:0/1876547832 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd82415f530 msgr2=0x7fd824157eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:45.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.942+0000 7fd82a400640 1 --2- 172.21.15.151:0/1876547832 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd82415f530 0x7fd824157eb0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fd81c009d60 tx=0x7fd81c003f10 comp rx=0 tx=0).stop 2024-09-16T15:46:45.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.942+0000 7fd82a400640 1 -- 172.21.15.151:0/1876547832 shutdown_connections 2024-09-16T15:46:45.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.942+0000 7fd82a400640 1 --2- 172.21.15.151:0/1876547832 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd8040794b0 0x7fd80407b970 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:45.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.942+0000 7fd82a400640 1 --2- 172.21.15.151:0/1876547832 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd82415f530 0x7fd824157eb0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:45.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.942+0000 7fd82a400640 1 --2- 172.21.15.151:0/1876547832 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8240d0930 0x7fd824158ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:45.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.942+0000 7fd82a400640 1 --2- 172.21.15.151:0/1876547832 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8240b34d0 0x7fd8240cbb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:45.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.942+0000 7fd82a400640 1 -- 172.21.15.151:0/1876547832 >> 172.21.15.151:0/1876547832 conn(0x7fd8240d1e30 msgr2=0x7fd8240cdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:45.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.942+0000 7fd82a400640 1 -- 172.21.15.151:0/1876547832 shutdown_connections 2024-09-16T15:46:45.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:45.942+0000 7fd82a400640 1 -- 172.21.15.151:0/1876547832 wait complete. 2024-09-16T15:46:45.960 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_more_than_cloner_threads' 2024-09-16T15:46:46.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:46.116+0000 7f1c07000640 1 -- 172.21.15.151:0/2469229556 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1be4003c10 msgr2=0x7f1be4004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:46.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:46.116+0000 7f1c07000640 1 --2- 172.21.15.151:0/2469229556 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1be4003c10 0x7f1be4004010 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f1bf0009ed0 tx=0x7f1be4005830 comp rx=0 tx=0).stop 2024-09-16T15:46:46.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:46.116+0000 7f1c05000640 1 -- 172.21.15.151:0/2469229556 reap_dead start 2024-09-16T15:46:46.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:46.117+0000 7f1c07000640 1 -- 172.21.15.151:0/2469229556 shutdown_connections 2024-09-16T15:46:46.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:46.117+0000 7f1c07000640 1 -- 172.21.15.151:0/2469229556 >> 172.21.15.151:0/2469229556 conn(0x7f1c000b1950 msgr2=0x7f1c000b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:46.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:46.117+0000 7f1c07000640 1 -- 172.21.15.151:0/2469229556 shutdown_connections 2024-09-16T15:46:46.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:46.117+0000 7f1c07000640 1 -- 172.21.15.151:0/2469229556 wait complete. 2024-09-16T15:46:46.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:46.118+0000 7f1c07000640 1 Processor -- start 2024-09-16T15:46:46.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:46.118+0000 7f1c07000640 1 -- start start 2024-09-16T15:46:46.122 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:46.119+0000 7f1c07000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1be4003c10 0x7f1c000cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:46.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:46.119+0000 7f1c07000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c0015c0b0 0x7f1c00159a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:46.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:46.119+0000 7f1c07000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1c00159f60 0x7f1c001f7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:46.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:46.119+0000 7f1c07000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0006ef30 con 0x7f1be4003c10 2024-09-16T15:46:46.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:46.119+0000 7f1c07000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0006fa80 con 0x7f1c0015c0b0 2024-09-16T15:46:46.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:46.119+0000 7f1c07000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f1c000b2440 con 0x7f1c00159f60 2024-09-16T15:46:46.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:46.119+0000 7f1bffe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1be4003c10 0x7f1c000cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:46.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:46.119+0000 7f1bffe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1be4003c10 0x7f1c000cc4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:55748/0 (socket says 172.21.15.151:55748) 2024-09-16T15:46:46.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:46.119+0000 7f1bffe00640 1 -- 172.21.15.151:0/1505916280 learned_addr learned my addr 172.21.15.151:0/1505916280 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:46:46.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:46.120+0000 7f1c05000640 1 --2- 172.21.15.151:0/1505916280 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1c00159f60 0x7f1c001f7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:46.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:46.120+0000 7f1bff400640 1 --2- 172.21.15.151:0/1505916280 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c0015c0b0 0x7f1c00159a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:46.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:46.120+0000 7f1bffe00640 1 -- 172.21.15.151:0/1505916280 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1c00159f60 msgr2=0x7f1c001f7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:46.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:46.120+0000 7f1bffe00640 1 --2- 172.21.15.151:0/1505916280 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1c00159f60 0x7f1c001f7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:46.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:46.120+0000 7f1bffe00640 1 -- 172.21.15.151:0/1505916280 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c0015c0b0 msgr2=0x7f1c00159a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:46.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:46.120+0000 7f1bffe00640 1 --2- 172.21.15.151:0/1505916280 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c0015c0b0 0x7f1c00159a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:46.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:46.120+0000 7f1bffe00640 1 -- 172.21.15.151:0/1505916280 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c0015c590 con 0x7f1be4003c10 2024-09-16T15:46:46.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:46.120+0000 7f1bffe00640 1 --2- 172.21.15.151:0/1505916280 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1be4003c10 0x7f1c000cc4a0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f1bf0009ed0 tx=0x7f1bf0033c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:46.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:46.120+0000 7f1bf7e00640 1 -- 172.21.15.151:0/1505916280 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf0043070 con 0x7f1be4003c10 2024-09-16T15:46:46.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:46.120+0000 7f1bf7e00640 1 -- 172.21.15.151:0/1505916280 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1bf0033e30 con 0x7f1be4003c10 2024-09-16T15:46:46.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:46.120+0000 7f1bf7e00640 1 -- 172.21.15.151:0/1505916280 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf0038040 con 0x7f1be4003c10 2024-09-16T15:46:46.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:46.120+0000 7f1c07000640 1 -- 172.21.15.151:0/1505916280 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c0015c870 con 0x7f1be4003c10 2024-09-16T15:46:46.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:46.121+0000 7f1c07000640 1 -- 172.21.15.151:0/1505916280 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c000b2d50 con 0x7f1be4003c10 2024-09-16T15:46:46.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:46.122+0000 7f1c07000640 1 -- 172.21.15.151:0/1505916280 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c00074460 con 0x7f1be4003c10 2024-09-16T15:46:46.125 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:46.123+0000 7f1bf7e00640 1 -- 172.21.15.151:0/1505916280 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 24) ==== 102409+0+0 (secure 0 0 0) 0x7f1bf0046070 con 0x7f1be4003c10 2024-09-16T15:46:46.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:46.124+0000 7f1bf7e00640 1 --2- 172.21.15.151:0/1505916280 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1be00794b0 0x7f1be007b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:46.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:46.124+0000 7f1bf7e00640 1 -- 172.21.15.151:0/1505916280 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 4999+0+0 (secure 0 0 0) 0x7f1bf00b9b10 con 0x7f1be4003c10 2024-09-16T15:46:46.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:46.124+0000 7f1bff400640 1 --2- 172.21.15.151:0/1505916280 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1be00794b0 0x7f1be007b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:46.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:46.125+0000 7f1bff400640 1 --2- 172.21.15.151:0/1505916280 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1be00794b0 0x7f1be007b970 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f1bec000f60 tx=0x7f1bec000ed0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:46.130 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:46.128+0000 7f1bf7e00640 1 -- 172.21.15.151:0/1505916280 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1bf0083580 con 0x7f1be4003c10 2024-09-16T15:46:46.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:46.387+0000 7f1c07000640 1 -- 172.21.15.151:0/1505916280 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_more_than_cloner_threads"]} v 0) -- 0x7f1c0009dea0 con 0x7f1be4003c10 2024-09-16T15:46:47.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.188+0000 7f1bf7e00640 1 -- 172.21.15.151:0/1505916280 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_more_than_cloner_threads"]}]=0 v328) ==== 170+0+0 (secure 0 0 0) 0x7f1bf0087fb0 con 0x7f1be4003c10 2024-09-16T15:46:47.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.191+0000 7f1c07000640 1 -- 172.21.15.151:0/1505916280 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1be00794b0 msgr2=0x7f1be007b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:47.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.191+0000 7f1c07000640 1 --2- 172.21.15.151:0/1505916280 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1be00794b0 0x7f1be007b970 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f1bec000f60 tx=0x7f1bec000ed0 comp rx=0 tx=0).stop 2024-09-16T15:46:47.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.191+0000 7f1c07000640 1 -- 172.21.15.151:0/1505916280 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1be4003c10 msgr2=0x7f1c000cc4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:47.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.191+0000 7f1c07000640 1 --2- 172.21.15.151:0/1505916280 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1be4003c10 0x7f1c000cc4a0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f1bf0009ed0 tx=0x7f1bf0033c40 comp rx=0 tx=0).stop 2024-09-16T15:46:47.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.191+0000 7f1c07000640 1 -- 172.21.15.151:0/1505916280 shutdown_connections 2024-09-16T15:46:47.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.191+0000 7f1c07000640 1 --2- 172.21.15.151:0/1505916280 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1be00794b0 0x7f1be007b970 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:47.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.191+0000 7f1c07000640 1 --2- 172.21.15.151:0/1505916280 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c0015c0b0 0x7f1c00159a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:47.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.191+0000 7f1c07000640 1 --2- 172.21.15.151:0/1505916280 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1c00159f60 0x7f1c001f7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:47.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.191+0000 7f1c07000640 1 --2- 172.21.15.151:0/1505916280 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1be4003c10 0x7f1c000cc4a0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:47.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.191+0000 7f1c07000640 1 -- 172.21.15.151:0/1505916280 >> 172.21.15.151:0/1505916280 conn(0x7f1c000b1950 msgr2=0x7f1c000ae510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:47.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.192+0000 7f1c07000640 1 -- 172.21.15.151:0/1505916280 shutdown_connections 2024-09-16T15:46:47.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.192+0000 7f1c07000640 1 -- 172.21.15.151:0/1505916280 wait complete. 2024-09-16T15:46:47.211 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-16T15:46:47.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.369+0000 7fb3a7200640 1 -- 172.21.15.151:0/1939082263 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3840052e0 msgr2=0x7fb38400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:47.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.369+0000 7fb3a7200640 1 --2- 172.21.15.151:0/1939082263 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3840052e0 0x7fb38400b9c0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fb398001690 tx=0x7fb384005830 comp rx=0 tx=0).stop 2024-09-16T15:46:47.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.369+0000 7fb3a7200640 1 -- 172.21.15.151:0/1939082263 shutdown_connections 2024-09-16T15:46:47.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.369+0000 7fb3a7200640 1 --2- 172.21.15.151:0/1939082263 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3840052e0 0x7fb38400b9c0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:47.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.369+0000 7fb3a7200640 1 -- 172.21.15.151:0/1939082263 >> 172.21.15.151:0/1939082263 conn(0x7fb3a00b1950 msgr2=0x7fb3a00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:47.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.369+0000 7fb3a7200640 1 -- 172.21.15.151:0/1939082263 shutdown_connections 2024-09-16T15:46:47.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.369+0000 7fb3a7200640 1 -- 172.21.15.151:0/1939082263 wait complete. 2024-09-16T15:46:47.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.371+0000 7fb3a7200640 1 Processor -- start 2024-09-16T15:46:47.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.371+0000 7fb3a7200640 1 -- start start 2024-09-16T15:46:47.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.372+0000 7fb3a7200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3840052e0 0x7fb3a01445b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:47.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.372+0000 7fb3a7200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3a009aa00 0x7fb3a0155320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:47.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.372+0000 7fb3a7200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3a009ae80 0x7fb3a0155860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:47.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.372+0000 7fb3a7200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a0074420 con 0x7fb3a009aa00 2024-09-16T15:46:47.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.372+0000 7fb3a7200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a00b2700 con 0x7fb3840052e0 2024-09-16T15:46:47.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.372+0000 7fb3a7200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fb3a00745a0 con 0x7fb3a009ae80 2024-09-16T15:46:47.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.372+0000 7fb39f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3a009aa00 0x7fb3a0155320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:47.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.372+0000 7fb39f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3a009aa00 0x7fb3a0155320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:55766/0 (socket says 172.21.15.151:55766) 2024-09-16T15:46:47.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.372+0000 7fb39f400640 1 -- 172.21.15.151:0/1637905808 learned_addr learned my addr 172.21.15.151:0/1637905808 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:46:47.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.372+0000 7fb39fe00640 1 --2- 172.21.15.151:0/1637905808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3840052e0 0x7fb3a01445b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:47.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.372+0000 7fb3a5200640 1 --2- 172.21.15.151:0/1637905808 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3a009ae80 0x7fb3a0155860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:47.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.373+0000 7fb39f400640 1 -- 172.21.15.151:0/1637905808 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3a009ae80 msgr2=0x7fb3a0155860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:47.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.373+0000 7fb39f400640 1 --2- 172.21.15.151:0/1637905808 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3a009ae80 0x7fb3a0155860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:47.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.373+0000 7fb39f400640 1 -- 172.21.15.151:0/1637905808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3840052e0 msgr2=0x7fb3a01445b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:47.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.373+0000 7fb39f400640 1 --2- 172.21.15.151:0/1637905808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3840052e0 0x7fb3a01445b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:47.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.373+0000 7fb39f400640 1 -- 172.21.15.151:0/1637905808 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3a009b3e0 con 0x7fb3a009aa00 2024-09-16T15:46:47.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.373+0000 7fb39f400640 1 --2- 172.21.15.151:0/1637905808 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3a009aa00 0x7fb3a0155320 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fb38c011ab0 tx=0x7fb38c011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:47.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.373+0000 7fb397e00640 1 -- 172.21.15.151:0/1637905808 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb38c01e070 con 0x7fb3a009aa00 2024-09-16T15:46:47.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.373+0000 7fb397e00640 1 -- 172.21.15.151:0/1637905808 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb38c00fe70 con 0x7fb3a009aa00 2024-09-16T15:46:47.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.373+0000 7fb3a7200640 1 -- 172.21.15.151:0/1637905808 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3a01f82a0 con 0x7fb3a009aa00 2024-09-16T15:46:47.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.373+0000 7fb397e00640 1 -- 172.21.15.151:0/1637905808 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb38c0042b0 con 0x7fb3a009aa00 2024-09-16T15:46:47.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.373+0000 7fb3a7200640 1 -- 172.21.15.151:0/1637905808 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3a01f8860 con 0x7fb3a009aa00 2024-09-16T15:46:47.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.374+0000 7fb3a7200640 1 -- 172.21.15.151:0/1637905808 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3a00adeb0 con 0x7fb3a009aa00 2024-09-16T15:46:47.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.375+0000 7fb397e00640 1 -- 172.21.15.151:0/1637905808 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 24) ==== 102409+0+0 (secure 0 0 0) 0x7fb38c021070 con 0x7fb3a009aa00 2024-09-16T15:46:47.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.376+0000 7fb397e00640 1 --2- 172.21.15.151:0/1637905808 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb3800794b0 0x7fb38007b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:47.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.377+0000 7fb397e00640 1 -- 172.21.15.151:0/1637905808 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 4999+0+0 (secure 0 0 0) 0x7fb38c099d50 con 0x7fb3a009aa00 2024-09-16T15:46:47.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.377+0000 7fb39fe00640 1 --2- 172.21.15.151:0/1637905808 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb3800794b0 0x7fb38007b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:47.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.378+0000 7fb39fe00640 1 --2- 172.21.15.151:0/1637905808 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb3800794b0 0x7fb38007b970 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb390001be0 tx=0x7fb390001490 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:47.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.381+0000 7fb397e00640 1 -- 172.21.15.151:0/1637905808 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb38c0637c0 con 0x7fb3a009aa00 2024-09-16T15:46:47.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.565+0000 7fb3a7200640 1 -- 172.21.15.151:0/1637905808 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fb3a01ea380 con 0x7fb3a009aa00 2024-09-16T15:46:47.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.566+0000 7fb397e00640 1 -- 172.21.15.151:0/1637905808 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v36) ==== 59+0+1998 (secure 0 0 0) 0x7fb38c0681f0 con 0x7fb3a009aa00 2024-09-16T15:46:47.568 INFO:teuthology.orchestra.run.smithi151.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-16T15:46:47.568 INFO:teuthology.orchestra.run.smithi151.stdout:mon advanced mon_op_complaint_time 120 2024-09-16T15:46:47.568 INFO:teuthology.orchestra.run.smithi151.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-16T15:46:47.568 INFO:teuthology.orchestra.run.smithi151.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-16T15:46:47.568 INFO:teuthology.orchestra.run.smithi151.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3795.338492 2024-09-16T15:46:47.568 INFO:teuthology.orchestra.run.smithi151.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 4030.355715 2024-09-16T15:46:47.568 INFO:teuthology.orchestra.run.smithi151.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3817.533640 2024-09-16T15:46:47.568 INFO:teuthology.orchestra.run.smithi151.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3950.656136 2024-09-16T15:46:47.568 INFO:teuthology.orchestra.run.smithi151.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4449.408052 2024-09-16T15:46:47.568 INFO:teuthology.orchestra.run.smithi151.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4439.932296 2024-09-16T15:46:47.569 INFO:teuthology.orchestra.run.smithi151.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4566.875579 2024-09-16T15:46:47.569 INFO:teuthology.orchestra.run.smithi151.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 6320.073632 2024-09-16T15:46:47.569 INFO:teuthology.orchestra.run.smithi151.stdout:mds advanced debug_mds 20/20 2024-09-16T15:46:47.569 INFO:teuthology.orchestra.run.smithi151.stdout:mds advanced debug_mds_balancer 20/20 2024-09-16T15:46:47.569 INFO:teuthology.orchestra.run.smithi151.stdout:mds advanced debug_ms 2/2 2024-09-16T15:46:47.569 INFO:teuthology.orchestra.run.smithi151.stdout:mds dev mds_debug_frag true 2024-09-16T15:46:47.569 INFO:teuthology.orchestra.run.smithi151.stdout:mds dev mds_debug_scatterstat true 2024-09-16T15:46:47.569 INFO:teuthology.orchestra.run.smithi151.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-16T15:46:47.569 INFO:teuthology.orchestra.run.smithi151.stdout:mds dev mds_verify_scatter true 2024-09-16T15:46:47.569 INFO:teuthology.orchestra.run.smithi151.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-16T15:46:47.569 INFO:teuthology.orchestra.run.smithi151.stdout:mds advanced rados_mon_op_timeout 900 2024-09-16T15:46:47.569 INFO:teuthology.orchestra.run.smithi151.stdout:mds advanced rados_osd_op_timeout 900 2024-09-16T15:46:47.569 INFO:teuthology.orchestra.run.smithi151.stdout:client advanced client_mount_timeout 600 2024-09-16T15:46:47.569 INFO:teuthology.orchestra.run.smithi151.stdout:client advanced debug_client 20/20 2024-09-16T15:46:47.569 INFO:teuthology.orchestra.run.smithi151.stdout:client advanced debug_ms 1/1 2024-09-16T15:46:47.569 INFO:teuthology.orchestra.run.smithi151.stdout:client advanced rados_mon_op_timeout 900 2024-09-16T15:46:47.570 INFO:teuthology.orchestra.run.smithi151.stdout:client advanced rados_osd_op_timeout 900 2024-09-16T15:46:47.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.568+0000 7fb3a7200640 1 -- 172.21.15.151:0/1637905808 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb3800794b0 msgr2=0x7fb38007b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:47.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.568+0000 7fb3a7200640 1 --2- 172.21.15.151:0/1637905808 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb3800794b0 0x7fb38007b970 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb390001be0 tx=0x7fb390001490 comp rx=0 tx=0).stop 2024-09-16T15:46:47.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.568+0000 7fb3a7200640 1 -- 172.21.15.151:0/1637905808 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3a009aa00 msgr2=0x7fb3a0155320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:47.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.568+0000 7fb3a7200640 1 --2- 172.21.15.151:0/1637905808 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3a009aa00 0x7fb3a0155320 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fb38c011ab0 tx=0x7fb38c011f70 comp rx=0 tx=0).stop 2024-09-16T15:46:47.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.568+0000 7fb3a7200640 1 -- 172.21.15.151:0/1637905808 shutdown_connections 2024-09-16T15:46:47.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.568+0000 7fb3a7200640 1 --2- 172.21.15.151:0/1637905808 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb3800794b0 0x7fb38007b970 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:47.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.568+0000 7fb3a7200640 1 --2- 172.21.15.151:0/1637905808 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3a009ae80 0x7fb3a0155860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:47.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.568+0000 7fb3a7200640 1 --2- 172.21.15.151:0/1637905808 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3a009aa00 0x7fb3a0155320 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:47.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.568+0000 7fb3a7200640 1 --2- 172.21.15.151:0/1637905808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3840052e0 0x7fb3a01445b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:47.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.568+0000 7fb3a7200640 1 -- 172.21.15.151:0/1637905808 >> 172.21.15.151:0/1637905808 conn(0x7fb3a00b1950 msgr2=0x7fb3a00b2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:47.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.568+0000 7fb3a7200640 1 -- 172.21.15.151:0/1637905808 shutdown_connections 2024-09-16T15:46:47.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.568+0000 7fb3a7200640 1 -- 172.21.15.151:0/1637905808 wait complete. 2024-09-16T15:46:47.585 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-16T15:46:47.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.746+0000 7f0949800640 1 -- 172.21.15.151:0/1931286200 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0928003a50 msgr2=0x7f0928003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:47.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.746+0000 7f0949800640 1 --2- 172.21.15.151:0/1931286200 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0928003a50 0x7f0928003e70 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f093400bb20 tx=0x7f0928005830 comp rx=0 tx=0).stop 2024-09-16T15:46:47.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.746+0000 7f0948e00640 1 -- 172.21.15.151:0/1931286200 reap_dead start 2024-09-16T15:46:47.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.746+0000 7f0949800640 1 -- 172.21.15.151:0/1931286200 shutdown_connections 2024-09-16T15:46:47.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.746+0000 7f0949800640 1 -- 172.21.15.151:0/1931286200 >> 172.21.15.151:0/1931286200 conn(0x7f09440cf9f0 msgr2=0x7f09440cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:47.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.746+0000 7f0949800640 1 -- 172.21.15.151:0/1931286200 shutdown_connections 2024-09-16T15:46:47.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.746+0000 7f0949800640 1 -- 172.21.15.151:0/1931286200 wait complete. 2024-09-16T15:46:47.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.748+0000 7f0949800640 1 Processor -- start 2024-09-16T15:46:47.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.748+0000 7f0949800640 1 -- start start 2024-09-16T15:46:47.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.749+0000 7f0949800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0928003a50 0x7f09440c6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:47.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.749+0000 7f0949800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09441f4c50 0x7f094415e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:47.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.749+0000 7f0949800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0944158db0 0x7f0944159230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:47.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.749+0000 7f0949800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f09440ae4c0 con 0x7f0944158db0 2024-09-16T15:46:47.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.749+0000 7f0949800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0944070030 con 0x7f09441f4c50 2024-09-16T15:46:47.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.749+0000 7f0949800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f09440ae640 con 0x7f0928003a50 2024-09-16T15:46:47.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.749+0000 7f0948e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0944158db0 0x7f0944159230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:47.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.749+0000 7f0948e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0944158db0 0x7f0944159230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:55770/0 (socket says 172.21.15.151:55770) 2024-09-16T15:46:47.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.749+0000 7f0948e00640 1 -- 172.21.15.151:0/241772578 learned_addr learned my addr 172.21.15.151:0/241772578 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:46:47.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.749+0000 7f0943e00640 1 --2- 172.21.15.151:0/241772578 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0928003a50 0x7f09440c6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:47.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.749+0000 7f0943400640 1 --2- 172.21.15.151:0/241772578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09441f4c50 0x7f094415e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:47.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.749+0000 7f0948e00640 1 -- 172.21.15.151:0/241772578 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0928003a50 msgr2=0x7f09440c6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:47.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.749+0000 7f0948e00640 1 --2- 172.21.15.151:0/241772578 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0928003a50 0x7f09440c6ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:47.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.749+0000 7f0948e00640 1 -- 172.21.15.151:0/241772578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09441f4c50 msgr2=0x7f094415e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:47.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.749+0000 7f0948e00640 1 --2- 172.21.15.151:0/241772578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09441f4c50 0x7f094415e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:47.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.749+0000 7f0948e00640 1 -- 172.21.15.151:0/241772578 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0944159770 con 0x7f0944158db0 2024-09-16T15:46:47.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.750+0000 7f0948e00640 1 --2- 172.21.15.151:0/241772578 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0944158db0 0x7f0944159230 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f093c005fd0 tx=0x7f093c009a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:47.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.750+0000 7f093be00640 1 -- 172.21.15.151:0/241772578 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f093c004b20 con 0x7f0944158db0 2024-09-16T15:46:47.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.750+0000 7f093be00640 1 -- 172.21.15.151:0/241772578 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f093c004db0 con 0x7f0944158db0 2024-09-16T15:46:47.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.750+0000 7f093be00640 1 -- 172.21.15.151:0/241772578 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f093c00f9d0 con 0x7f0944158db0 2024-09-16T15:46:47.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.750+0000 7f0949800640 1 -- 172.21.15.151:0/241772578 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09440c9b50 con 0x7f0944158db0 2024-09-16T15:46:47.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.750+0000 7f0949800640 1 -- 172.21.15.151:0/241772578 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09440c9330 con 0x7f0944158db0 2024-09-16T15:46:47.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.751+0000 7f0949800640 1 -- 172.21.15.151:0/241772578 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0944073ef0 con 0x7f0944158db0 2024-09-16T15:46:47.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.752+0000 7f093be00640 1 -- 172.21.15.151:0/241772578 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 24) ==== 102409+0+0 (secure 0 0 0) 0x7f093c01b020 con 0x7f0944158db0 2024-09-16T15:46:47.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.753+0000 7f093be00640 1 --2- 172.21.15.151:0/241772578 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f09240794b0 0x7f092407b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:47.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.753+0000 7f093be00640 1 -- 172.21.15.151:0/241772578 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 4999+0+0 (secure 0 0 0) 0x7f093c099190 con 0x7f0944158db0 2024-09-16T15:46:47.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.753+0000 7f0943e00640 1 --2- 172.21.15.151:0/241772578 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f09240794b0 0x7f092407b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:47.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.754+0000 7f0943e00640 1 --2- 172.21.15.151:0/241772578 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f09240794b0 0x7f092407b970 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0934036000 tx=0x7f0934035040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:47.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.757+0000 7f093be00640 1 -- 172.21.15.151:0/241772578 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f093c062c00 con 0x7f0944158db0 2024-09-16T15:46:47.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.941+0000 7f0949800640 1 -- 172.21.15.151:0/241772578 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f0944000900 con 0x7f0944158db0 2024-09-16T15:46:47.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.945+0000 7f093be00640 1 -- 172.21.15.151:0/241772578 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v37)=0 v37) ==== 122+0+0 (secure 0 0 0) 0x7f093c067630 con 0x7f0944158db0 2024-09-16T15:46:47.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.948+0000 7f0949800640 1 -- 172.21.15.151:0/241772578 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f09240794b0 msgr2=0x7f092407b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:47.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.948+0000 7f0949800640 1 --2- 172.21.15.151:0/241772578 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f09240794b0 0x7f092407b970 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0934036000 tx=0x7f0934035040 comp rx=0 tx=0).stop 2024-09-16T15:46:47.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.948+0000 7f0949800640 1 -- 172.21.15.151:0/241772578 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0944158db0 msgr2=0x7f0944159230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:47.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.948+0000 7f0949800640 1 --2- 172.21.15.151:0/241772578 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0944158db0 0x7f0944159230 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f093c005fd0 tx=0x7f093c009a80 comp rx=0 tx=0).stop 2024-09-16T15:46:47.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.948+0000 7f0949800640 1 -- 172.21.15.151:0/241772578 shutdown_connections 2024-09-16T15:46:47.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.948+0000 7f0949800640 1 --2- 172.21.15.151:0/241772578 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f09240794b0 0x7f092407b970 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:47.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.948+0000 7f0949800640 1 --2- 172.21.15.151:0/241772578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09441f4c50 0x7f094415e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:47.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.948+0000 7f0949800640 1 --2- 172.21.15.151:0/241772578 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0944158db0 0x7f0944159230 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:47.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.948+0000 7f0949800640 1 --2- 172.21.15.151:0/241772578 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0928003a50 0x7f09440c6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:47.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.948+0000 7f0949800640 1 -- 172.21.15.151:0/241772578 >> 172.21.15.151:0/241772578 conn(0x7f09440cf9f0 msgr2=0x7f094406e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:47.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.948+0000 7f0949800640 1 -- 172.21.15.151:0/241772578 shutdown_connections 2024-09-16T15:46:47.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:47.948+0000 7f0949800640 1 -- 172.21.15.151:0/241772578 wait complete. 2024-09-16T15:46:47.966 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-16T15:46:47.966 DEBUG:teuthology.orchestra.run.smithi151:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:46:48.064 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:46:48.163 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-16T15:46:48.163 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:46:48.163 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:46:48.200 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T15:46:48.200 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-16T15:46:48.200 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:46:48.237 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:46:48.237 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:46:48.238 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:46:48.275 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T15:46:48.275 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-16T15:46:48.275 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:46:48.314 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:46:48.314 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-16T15:46:48.314 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:46:48.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.471+0000 7ff659c00640 1 -- 172.21.15.151:0/1512881157 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6540ad270 msgr2=0x7ff654073fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:48.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.471+0000 7ff659c00640 1 --2- 172.21.15.151:0/1512881157 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6540ad270 0x7ff654073fb0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7ff654075130 tx=0x7ff64c02a980 comp rx=0 tx=0).stop 2024-09-16T15:46:48.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.472+0000 7ff659c00640 1 -- 172.21.15.151:0/1512881157 shutdown_connections 2024-09-16T15:46:48.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.472+0000 7ff659c00640 1 --2- 172.21.15.151:0/1512881157 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6540ad270 0x7ff654073fb0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:48.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.472+0000 7ff659c00640 1 --2- 172.21.15.151:0/1512881157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff6540ccf00 0x7ff6540acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:48.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.472+0000 7ff659c00640 1 --2- 172.21.15.151:0/1512881157 >> v2:172.21.15.167:3300/0 conn(0x7ff65415f5a0 0x7ff6540b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:48.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.472+0000 7ff659c00640 1 -- 172.21.15.151:0/1512881157 >> 172.21.15.151:0/1512881157 conn(0x7ff6540b1950 msgr2=0x7ff6540b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:48.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.472+0000 7ff659c00640 1 -- 172.21.15.151:0/1512881157 shutdown_connections 2024-09-16T15:46:48.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.472+0000 7ff659c00640 1 -- 172.21.15.151:0/1512881157 wait complete. 2024-09-16T15:46:48.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.473+0000 7ff659c00640 1 Processor -- start 2024-09-16T15:46:48.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.473+0000 7ff659c00640 1 -- start start 2024-09-16T15:46:48.476 DEBUG:teuthology.orchestra.run.smithi151:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:46:48.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.474+0000 7ff659c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6540ad270 0x7ff6541f3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:48.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.474+0000 7ff659c00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff6540ccf00 0x7ff6541598b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:48.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.474+0000 7ff659c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff65415f5a0 0x7ff65414e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:48.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.474+0000 7ff659c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff65406f160 con 0x7ff65415f5a0 2024-09-16T15:46:48.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.474+0000 7ff659c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff654074780 con 0x7ff6540ad270 2024-09-16T15:46:48.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.474+0000 7ff659c00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7ff65406f2e0 con 0x7ff6540ccf00 2024-09-16T15:46:48.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.475+0000 7ff653400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff65415f5a0 0x7ff65414e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:48.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.475+0000 7ff652a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6540ad270 0x7ff6541f3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:48.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.475+0000 7ff653400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff65415f5a0 0x7ff65414e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:55774/0 (socket says 172.21.15.151:55774) 2024-09-16T15:46:48.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.475+0000 7ff652a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6540ad270 0x7ff6541f3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.151:41668/0 (socket says 172.21.15.151:41668) 2024-09-16T15:46:48.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.475+0000 7ff653400640 1 -- 172.21.15.151:0/3364181330 learned_addr learned my addr 172.21.15.151:0/3364181330 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:46:48.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.475+0000 7ff652000640 1 --2- 172.21.15.151:0/3364181330 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff6540ccf00 0x7ff6541598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:48.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.475+0000 7ff653400640 1 -- 172.21.15.151:0/3364181330 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff6540ccf00 msgr2=0x7ff6541598b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:48.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.475+0000 7ff653400640 1 --2- 172.21.15.151:0/3364181330 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff6540ccf00 0x7ff6541598b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:48.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.475+0000 7ff653400640 1 -- 172.21.15.151:0/3364181330 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6540ad270 msgr2=0x7ff6541f3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:48.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.475+0000 7ff653400640 1 --2- 172.21.15.151:0/3364181330 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6540ad270 0x7ff6541f3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:48.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.475+0000 7ff653400640 1 -- 172.21.15.151:0/3364181330 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6541f4200 con 0x7ff65415f5a0 2024-09-16T15:46:48.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.475+0000 7ff652a00640 1 --2- 172.21.15.151:0/3364181330 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6540ad270 0x7ff6541f3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:46:48.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.475+0000 7ff653400640 1 --2- 172.21.15.151:0/3364181330 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff65415f5a0 0x7ff65414e640 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7ff64c009d60 tx=0x7ff64c003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:48.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.476+0000 7ff64aa00640 1 -- 172.21.15.151:0/3364181330 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff64c034050 con 0x7ff65415f5a0 2024-09-16T15:46:48.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.476+0000 7ff64aa00640 1 -- 172.21.15.151:0/3364181330 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff64c004450 con 0x7ff65415f5a0 2024-09-16T15:46:48.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.476+0000 7ff64aa00640 1 -- 172.21.15.151:0/3364181330 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff64c043440 con 0x7ff65415f5a0 2024-09-16T15:46:48.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.476+0000 7ff659c00640 1 -- 172.21.15.151:0/3364181330 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff65415bfc0 con 0x7ff65415f5a0 2024-09-16T15:46:48.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.476+0000 7ff659c00640 1 -- 172.21.15.151:0/3364181330 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff65415c580 con 0x7ff65415f5a0 2024-09-16T15:46:48.481 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:46:48.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.477+0000 7ff659c00640 1 -- 172.21.15.151:0/3364181330 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6540cc980 con 0x7ff65415f5a0 2024-09-16T15:46:48.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.478+0000 7ff64aa00640 1 -- 172.21.15.151:0/3364181330 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 24) ==== 102409+0+0 (secure 0 0 0) 0x7ff64c037020 con 0x7ff65415f5a0 2024-09-16T15:46:48.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.479+0000 7ff64aa00640 1 --2- 172.21.15.151:0/3364181330 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff6300794b0 0x7ff63007b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:48.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.479+0000 7ff64aa00640 1 -- 172.21.15.151:0/3364181330 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 4999+0+0 (secure 0 0 0) 0x7ff64c0ba970 con 0x7ff65415f5a0 2024-09-16T15:46:48.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.479+0000 7ff652a00640 1 --2- 172.21.15.151:0/3364181330 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff6300794b0 0x7ff63007b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:48.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.480+0000 7ff652a00640 1 --2- 172.21.15.151:0/3364181330 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff6300794b0 0x7ff63007b970 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff63c005fd0 tx=0x7ff63c00b2e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:48.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.483+0000 7ff64aa00640 1 -- 172.21.15.151:0/3364181330 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff64c0840e0 con 0x7ff65415f5a0 2024-09-16T15:46:48.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.712+0000 7ff659c00640 1 -- 172.21.15.151:0/3364181330 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff65406f870 con 0x7ff65415f5a0 2024-09-16T15:46:48.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.712+0000 7ff64aa00640 1 -- 172.21.15.151:0/3364181330 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+3920 (secure 0 0 0) 0x7ff64c088b10 con 0x7ff65415f5a0 2024-09-16T15:46:48.714 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:46:48.715 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":39,"btime":"2024-09-16T15:46:41:830532+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5357,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1715048770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1715048770},{"type":"v1","addr":"172.21.15.167:6837","nonce":1715048770}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":5722,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/784205427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":784205427},{"type":"v1","addr":"172.21.15.167:6836","nonce":784205427}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":5958,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/3837674321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3837674321},{"type":"v1","addr":"172.21.15.151:6834","nonce":3837674321}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":6372,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2708513005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":2708513005},{"type":"v1","addr":"172.21.15.151:6835","nonce":2708513005}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[]} 2024-09-16T15:46:48.715 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 39 2024-09-16T15:46:48.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.714+0000 7ff659c00640 1 -- 172.21.15.151:0/3364181330 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff6300794b0 msgr2=0x7ff63007b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:48.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.714+0000 7ff659c00640 1 --2- 172.21.15.151:0/3364181330 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff6300794b0 0x7ff63007b970 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff63c005fd0 tx=0x7ff63c00b2e0 comp rx=0 tx=0).stop 2024-09-16T15:46:48.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.714+0000 7ff659c00640 1 -- 172.21.15.151:0/3364181330 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff65415f5a0 msgr2=0x7ff65414e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:48.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.714+0000 7ff659c00640 1 --2- 172.21.15.151:0/3364181330 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff65415f5a0 0x7ff65414e640 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7ff64c009d60 tx=0x7ff64c003f10 comp rx=0 tx=0).stop 2024-09-16T15:46:48.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.715+0000 7ff659c00640 1 -- 172.21.15.151:0/3364181330 shutdown_connections 2024-09-16T15:46:48.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.715+0000 7ff659c00640 1 --2- 172.21.15.151:0/3364181330 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff6300794b0 0x7ff63007b970 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:48.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.715+0000 7ff659c00640 1 --2- 172.21.15.151:0/3364181330 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff65415f5a0 0x7ff65414e640 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:48.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.715+0000 7ff659c00640 1 --2- 172.21.15.151:0/3364181330 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff6540ccf00 0x7ff6541598b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:48.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.715+0000 7ff659c00640 1 --2- 172.21.15.151:0/3364181330 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6540ad270 0x7ff6541f3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:48.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.715+0000 7ff659c00640 1 -- 172.21.15.151:0/3364181330 >> 172.21.15.151:0/3364181330 conn(0x7ff6540b1950 msgr2=0x7ff6540b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:48.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.715+0000 7ff659c00640 1 -- 172.21.15.151:0/3364181330 shutdown_connections 2024-09-16T15:46:48.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.715+0000 7ff659c00640 1 -- 172.21.15.151:0/3364181330 wait complete. 2024-09-16T15:46:48.734 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-16T15:46:48.734 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-16T15:46:48.734 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-16T15:46:48.734 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:46:48.734 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-16T15:46:48.734 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-16T15:46:48.735 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-16T15:46:48.735 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:46:48.735 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-16T15:46:48.735 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-16T15:46:48.735 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-16T15:46:48.735 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:46:48.735 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-16T15:46:48.736 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-16T15:46:48.736 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-16T15:46:48.736 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:46:48.749 INFO:tasks.ceph.mds.a:Stopped 2024-09-16T15:46:48.749 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-16T15:46:48.750 INFO:tasks.ceph.mds.c:Stopped 2024-09-16T15:46:48.750 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-16T15:46:48.756 INFO:tasks.ceph.mds.b:Stopped 2024-09-16T15:46:48.756 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-16T15:46:48.757 INFO:tasks.ceph.mds.d:Stopped 2024-09-16T15:46:48.757 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-16T15:46:48.794 INFO:tasks.ceph.mds.a:Started 2024-09-16T15:46:48.794 DEBUG:teuthology.parallel:result is None 2024-09-16T15:46:48.796 INFO:tasks.ceph.mds.c:Started 2024-09-16T15:46:48.797 DEBUG:teuthology.parallel:result is None 2024-09-16T15:46:48.800 INFO:tasks.ceph.mds.b:Started 2024-09-16T15:46:48.800 DEBUG:teuthology.parallel:result is None 2024-09-16T15:46:48.802 INFO:tasks.ceph.mds.d:Started 2024-09-16T15:46:48.802 DEBUG:teuthology.parallel:result is None 2024-09-16T15:46:48.803 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-16T15:46:48.925 INFO:tasks.ceph.mds.a.smithi151.stdout:starting mds.a at 2024-09-16T15:46:48.925 INFO:tasks.ceph.mds.c.smithi151.stdout:starting mds.c at 2024-09-16T15:46:48.929 INFO:tasks.ceph.mds.d.smithi167.stdout:starting mds.d at 2024-09-16T15:46:48.930 INFO:tasks.ceph.mds.b.smithi167.stdout:starting mds.b at 2024-09-16T15:46:48.972 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.969+0000 7f96ade00640 1 -- 172.21.15.151:0/4260959481 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96a80b2440 msgr2=0x7f96a806f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:48.972 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.969+0000 7f96ade00640 1 --2- 172.21.15.151:0/4260959481 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96a80b2440 0x7f96a806f160 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f96a0005fd0 tx=0x7f96a002a8b0 comp rx=0 tx=0).stop 2024-09-16T15:46:48.972 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.969+0000 7f96ade00640 1 -- 172.21.15.151:0/4260959481 shutdown_connections 2024-09-16T15:46:48.972 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.969+0000 7f96ade00640 1 --2- 172.21.15.151:0/4260959481 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96a80b2440 0x7f96a806f160 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:48.972 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.969+0000 7f96ade00640 1 -- 172.21.15.151:0/4260959481 >> 172.21.15.151:0/4260959481 conn(0x7f96a80b1950 msgr2=0x7f96a80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:48.972 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.969+0000 7f96ade00640 1 -- 172.21.15.151:0/4260959481 shutdown_connections 2024-09-16T15:46:48.972 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.969+0000 7f96ade00640 1 -- 172.21.15.151:0/4260959481 wait complete. 2024-09-16T15:46:48.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.971+0000 7f96ade00640 1 Processor -- start 2024-09-16T15:46:48.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.971+0000 7f96ade00640 1 -- start start 2024-09-16T15:46:48.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.972+0000 7f96ade00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f96a80b2440 0x7f96a809aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:48.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.972+0000 7f96ade00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96a80ccf00 0x7f96a8140d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:48.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.972+0000 7f96ade00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f96a815f5a0 0x7f96a815a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:48.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.972+0000 7f96ade00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f96a8074840 con 0x7f96a80b2440 2024-09-16T15:46:48.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.972+0000 7f96ade00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f96a80746c0 con 0x7f96a80ccf00 2024-09-16T15:46:48.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.972+0000 7f96ade00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f96a806f6a0 con 0x7f96a815f5a0 2024-09-16T15:46:48.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.972+0000 7f96a6a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f96a80b2440 0x7f96a809aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:48.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.972+0000 7f96a6a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f96a80b2440 0x7f96a809aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:55822/0 (socket says 172.21.15.151:55822) 2024-09-16T15:46:48.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.972+0000 7f96a6a00640 1 -- 172.21.15.151:0/2554062468 learned_addr learned my addr 172.21.15.151:0/2554062468 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:46:48.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.973+0000 7f96a7400640 1 --2- 172.21.15.151:0/2554062468 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f96a815f5a0 0x7f96a815a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:48.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.973+0000 7f969e000640 1 --2- 172.21.15.151:0/2554062468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96a80ccf00 0x7f96a8140d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:48.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.973+0000 7f96a6a00640 1 -- 172.21.15.151:0/2554062468 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f96a815f5a0 msgr2=0x7f96a815a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:48.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.973+0000 7f96a6a00640 1 --2- 172.21.15.151:0/2554062468 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f96a815f5a0 0x7f96a815a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:48.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.973+0000 7f96a6a00640 1 -- 172.21.15.151:0/2554062468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96a80ccf00 msgr2=0x7f96a8140d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:48.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.973+0000 7f96a6a00640 1 --2- 172.21.15.151:0/2554062468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96a80ccf00 0x7f96a8140d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:48.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.973+0000 7f96a6a00640 1 -- 172.21.15.151:0/2554062468 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96a81f8710 con 0x7f96a80b2440 2024-09-16T15:46:48.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.973+0000 7f96a7400640 1 --2- 172.21.15.151:0/2554062468 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f96a815f5a0 0x7f96a815a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:46:48.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.973+0000 7f969e000640 1 --2- 172.21.15.151:0/2554062468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96a80ccf00 0x7f96a8140d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:46:48.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.973+0000 7f96a6a00640 1 --2- 172.21.15.151:0/2554062468 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f96a80b2440 0x7f96a809aa00 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f969800c400 tx=0x7f969800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:48.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.973+0000 7f969fe00640 1 -- 172.21.15.151:0/2554062468 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9698011790 con 0x7f96a80b2440 2024-09-16T15:46:48.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.973+0000 7f969fe00640 1 -- 172.21.15.151:0/2554062468 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9698017070 con 0x7f96a80b2440 2024-09-16T15:46:48.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.973+0000 7f969fe00640 1 -- 172.21.15.151:0/2554062468 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9698011cc0 con 0x7f96a80b2440 2024-09-16T15:46:48.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.973+0000 7f96ade00640 1 -- 172.21.15.151:0/2554062468 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96a81f89f0 con 0x7f96a80b2440 2024-09-16T15:46:48.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.974+0000 7f96ade00640 1 -- 172.21.15.151:0/2554062468 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96a809c750 con 0x7f96a80b2440 2024-09-16T15:46:48.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.975+0000 7f96ade00640 1 -- 172.21.15.151:0/2554062468 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96a806f160 con 0x7f96a80b2440 2024-09-16T15:46:48.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.976+0000 7f969fe00640 1 -- 172.21.15.151:0/2554062468 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 24) ==== 102409+0+0 (secure 0 0 0) 0x7f969801c460 con 0x7f96a80b2440 2024-09-16T15:46:48.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.977+0000 7f969fe00640 1 --2- 172.21.15.151:0/2554062468 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f96840794b0 0x7f968407b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:48.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.977+0000 7f969fe00640 1 -- 172.21.15.151:0/2554062468 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 4999+0+0 (secure 0 0 0) 0x7f9698099040 con 0x7f96a80b2440 2024-09-16T15:46:48.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.977+0000 7f969e000640 1 --2- 172.21.15.151:0/2554062468 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f96840794b0 0x7f968407b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:48.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.978+0000 7f969e000640 1 --2- 172.21.15.151:0/2554062468 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f96840794b0 0x7f968407b970 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f96a81f7ac0 tx=0x7f9690009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:48.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:48.981+0000 7f969fe00640 1 -- 172.21.15.151:0/2554062468 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9698063160 con 0x7f96a80b2440 2024-09-16T15:46:49.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.175+0000 7f96ade00640 1 -- 172.21.15.151:0/2554062468 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f96a80ae970 con 0x7f96a80b2440 2024-09-16T15:46:49.207 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.205+0000 7f969fe00640 1 -- 172.21.15.151:0/2554062468 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v70) ==== 97+0+0 (secure 0 0 0) 0x7f9698062a60 con 0x7f96a80b2440 2024-09-16T15:46:49.207 INFO:teuthology.orchestra.run.smithi151.stderr: removed all blocklist entries 2024-09-16T15:46:49.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.209+0000 7f96ade00640 1 -- 172.21.15.151:0/2554062468 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f96840794b0 msgr2=0x7f968407b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:49.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.209+0000 7f96ade00640 1 --2- 172.21.15.151:0/2554062468 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f96840794b0 0x7f968407b970 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f96a81f7ac0 tx=0x7f9690009040 comp rx=0 tx=0).stop 2024-09-16T15:46:49.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.209+0000 7f96ade00640 1 -- 172.21.15.151:0/2554062468 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f96a80b2440 msgr2=0x7f96a809aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:49.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.209+0000 7f96ade00640 1 --2- 172.21.15.151:0/2554062468 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f96a80b2440 0x7f96a809aa00 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f969800c400 tx=0x7f969800c8c0 comp rx=0 tx=0).stop 2024-09-16T15:46:49.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.209+0000 7f96ade00640 1 -- 172.21.15.151:0/2554062468 shutdown_connections 2024-09-16T15:46:49.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.209+0000 7f96ade00640 1 --2- 172.21.15.151:0/2554062468 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f96840794b0 0x7f968407b970 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:49.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.209+0000 7f96ade00640 1 --2- 172.21.15.151:0/2554062468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96a80ccf00 0x7f96a8140d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:49.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.209+0000 7f96ade00640 1 --2- 172.21.15.151:0/2554062468 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f96a815f5a0 0x7f96a815a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:49.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.209+0000 7f96ade00640 1 --2- 172.21.15.151:0/2554062468 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f96a80b2440 0x7f96a809aa00 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:49.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.209+0000 7f96ade00640 1 -- 172.21.15.151:0/2554062468 >> 172.21.15.151:0/2554062468 conn(0x7f96a80b1950 msgr2=0x7f96a80af8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:49.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.209+0000 7f96ade00640 1 -- 172.21.15.151:0/2554062468 shutdown_connections 2024-09-16T15:46:49.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.209+0000 7f96ade00640 1 -- 172.21.15.151:0/2554062468 wait complete. 2024-09-16T15:46:49.228 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-16T15:46:49.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.386+0000 7f593d600640 1 -- 172.21.15.151:0/29505456 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f59380af3d0 msgr2=0x7f593806f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:49.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.386+0000 7f593d600640 1 --2- 172.21.15.151:0/29505456 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f59380af3d0 0x7f593806f660 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5924005fd0 tx=0x7f592402a540 comp rx=0 tx=0).stop 2024-09-16T15:46:49.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.386+0000 7f593d600640 1 -- 172.21.15.151:0/29505456 shutdown_connections 2024-09-16T15:46:49.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.386+0000 7f593d600640 1 --2- 172.21.15.151:0/29505456 >> v2:172.21.15.167:3300/0 conn(0x7f5938072df0 0x7f59380732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:49.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.386+0000 7f593d600640 1 --2- 172.21.15.151:0/29505456 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f59380af3d0 0x7f593806f660 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:49.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.386+0000 7f593d600640 1 -- 172.21.15.151:0/29505456 >> 172.21.15.151:0/29505456 conn(0x7f59380b1950 msgr2=0x7f59380b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:49.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.386+0000 7f593d600640 1 -- 172.21.15.151:0/29505456 shutdown_connections 2024-09-16T15:46:49.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.386+0000 7f593d600640 1 -- 172.21.15.151:0/29505456 wait complete. 2024-09-16T15:46:49.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.388+0000 7f593d600640 1 Processor -- start 2024-09-16T15:46:49.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.388+0000 7f593d600640 1 -- start start 2024-09-16T15:46:49.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.389+0000 7f593d600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5938072df0 0x7f59381f88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:49.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.389+0000 7f593d600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f59380af3d0 0x7f59381547d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:49.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.389+0000 7f593d600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5938161550 0x7f593809c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:49.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.389+0000 7f593d600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f59380737e0 con 0x7f5938072df0 2024-09-16T15:46:49.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.389+0000 7f593d600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f593806fdb0 con 0x7f5938161550 2024-09-16T15:46:49.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.389+0000 7f593d600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f5938073960 con 0x7f59380af3d0 2024-09-16T15:46:49.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.389+0000 7f5936a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5938072df0 0x7f59381f88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:49.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.389+0000 7f5936000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f59380af3d0 0x7f59381547d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:49.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.389+0000 7f5936000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f59380af3d0 0x7f59381547d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.151:60368/0 (socket says 172.21.15.151:60368) 2024-09-16T15:46:49.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.389+0000 7f5937400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5938161550 0x7f593809c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:49.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.389+0000 7f5936000640 1 -- 172.21.15.151:0/1767303870 learned_addr learned my addr 172.21.15.151:0/1767303870 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:46:49.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.389+0000 7f5937400640 1 -- 172.21.15.151:0/1767303870 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f59380af3d0 msgr2=0x7f59381547d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:49.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.389+0000 7f5937400640 1 --2- 172.21.15.151:0/1767303870 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f59380af3d0 0x7f59381547d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:49.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.389+0000 7f5937400640 1 -- 172.21.15.151:0/1767303870 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5938072df0 msgr2=0x7f59381f88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:49.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.389+0000 7f5937400640 1 --2- 172.21.15.151:0/1767303870 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5938072df0 0x7f59381f88c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:49.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.389+0000 7f5937400640 1 -- 172.21.15.151:0/1767303870 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59380cdb80 con 0x7f5938161550 2024-09-16T15:46:49.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.389+0000 7f5936a00640 1 --2- 172.21.15.151:0/1767303870 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5938072df0 0x7f59381f88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:46:49.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.390+0000 7f5937400640 1 --2- 172.21.15.151:0/1767303870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5938161550 0x7f593809c4c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f593000e970 tx=0x7f593000ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:49.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.390+0000 7f592ea00640 1 -- 172.21.15.151:0/1767303870 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f593000ccc0 con 0x7f5938161550 2024-09-16T15:46:49.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.390+0000 7f592ea00640 1 -- 172.21.15.151:0/1767303870 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5930004020 con 0x7f5938161550 2024-09-16T15:46:49.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.390+0000 7f593d600640 1 -- 172.21.15.151:0/1767303870 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59381f8e00 con 0x7f5938161550 2024-09-16T15:46:49.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.390+0000 7f592ea00640 1 -- 172.21.15.151:0/1767303870 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5930004470 con 0x7f5938161550 2024-09-16T15:46:49.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.390+0000 7f593d600640 1 -- 172.21.15.151:0/1767303870 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59380cddf0 con 0x7f5938161550 2024-09-16T15:46:49.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.391+0000 7f593d600640 1 -- 172.21.15.151:0/1767303870 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5938073ec0 con 0x7f5938161550 2024-09-16T15:46:49.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.392+0000 7f592ea00640 1 -- 172.21.15.151:0/1767303870 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 24) ==== 102409+0+0 (secure 0 0 0) 0x7f59300035f0 con 0x7f5938161550 2024-09-16T15:46:49.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.393+0000 7f592ea00640 1 --2- 172.21.15.151:0/1767303870 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f59180794b0 0x7f591807b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:49.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.393+0000 7f592ea00640 1 -- 172.21.15.151:0/1767303870 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 4913+0+0 (secure 0 0 0) 0x7f5930099c60 con 0x7f5938161550 2024-09-16T15:46:49.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.394+0000 7f5936a00640 1 --2- 172.21.15.151:0/1767303870 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f59180794b0 0x7f591807b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:49.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.394+0000 7f5936a00640 1 --2- 172.21.15.151:0/1767303870 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f59180794b0 0x7f591807b970 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f592800f1d0 tx=0x7f5928009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:49.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.398+0000 7f592ea00640 1 -- 172.21.15.151:0/1767303870 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f59381f8e00 con 0x7f5938161550 2024-09-16T15:46:49.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.655+0000 7f593d600640 1 -- 172.21.15.151:0/1767303870 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f5938157c90 con 0x7f5938161550 2024-09-16T15:46:49.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.656+0000 7f592ea00640 1 -- 172.21.15.151:0/1767303870 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) ==== 80+0+5304 (secure 0 0 0) 0x7f5930063610 con 0x7f5938161550 2024-09-16T15:46:49.659 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:46:49.659 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-16T15:46:49.659 INFO:teuthology.orchestra.run.smithi151.stdout: "auth_dump": [ 2024-09-16T15:46:49.659 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:46:49.659 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "mds.a", 2024-09-16T15:46:49.659 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDsUOhmJVK6AhAANfmTPS6kKn41LSD/MuyW9g==", 2024-09-16T15:46:49.659 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:46:49.659 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:46:49.659 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow *", 2024-09-16T15:46:49.659 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow *", 2024-09-16T15:46:49.659 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:46:49.660 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:46:49.660 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:46:49.660 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:46:49.660 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "mds.b", 2024-09-16T15:46:49.660 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDsUOhmX1R3FhAAUQgUsE/6wUyokXcZuBuIWg==", 2024-09-16T15:46:49.660 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:46:49.660 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:46:49.660 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow *", 2024-09-16T15:46:49.660 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow *", 2024-09-16T15:46:49.660 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:46:49.660 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:46:49.660 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:46:49.660 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:46:49.661 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "mds.c", 2024-09-16T15:46:49.661 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDsUOhmYGCKDBAAY2EFNmiFNf1x2so9wxp4YQ==", 2024-09-16T15:46:49.661 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:46:49.661 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:46:49.661 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow *", 2024-09-16T15:46:49.661 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow *", 2024-09-16T15:46:49.661 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:46:49.661 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:46:49.661 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:46:49.661 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:46:49.661 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "mds.d", 2024-09-16T15:46:49.661 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDsUOhmfCk4IBAAx12Gw2QudQ2nrPb8VlE0SA==", 2024-09-16T15:46:49.661 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:46:49.662 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:46:49.662 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow *", 2024-09-16T15:46:49.662 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow *", 2024-09-16T15:46:49.662 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:46:49.662 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:46:49.662 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:46:49.662 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:46:49.662 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.0", 2024-09-16T15:46:49.662 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDvUOhmKji5NxAAn/E3YQR7jeMnBTLBXc1G6A==", 2024-09-16T15:46:49.662 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:46:49.662 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:46:49.662 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:46:49.662 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:46:49.662 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:46:49.663 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:46:49.663 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:46:49.663 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.1", 2024-09-16T15:46:49.663 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDzUOhmmBrNBhAA0Uwh4Fam2bmKKvz5jAQBWw==", 2024-09-16T15:46:49.663 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:46:49.663 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:46:49.663 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:46:49.663 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:46:49.663 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:46:49.663 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:46:49.663 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:46:49.663 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.2", 2024-09-16T15:46:49.663 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQD2UOhmOV6GERAAjD+PQLTZrCFR1669UEEIAQ==", 2024-09-16T15:46:49.663 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:46:49.664 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:46:49.664 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:46:49.664 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:46:49.664 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:46:49.664 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:46:49.664 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:46:49.664 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.3", 2024-09-16T15:46:49.664 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQD5UOhmCRrdGRAAQC0bqWX6N1jniVGEQ2aAvg==", 2024-09-16T15:46:49.664 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:46:49.664 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:46:49.664 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:46:49.664 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:46:49.664 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:46:49.665 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:46:49.665 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:46:49.665 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.4", 2024-09-16T15:46:49.665 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQD/UOhmXI/nGRAA7t6XveKvrWgqy6rHuMQSZQ==", 2024-09-16T15:46:49.665 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:46:49.665 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:46:49.665 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:46:49.665 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:46:49.665 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:46:49.665 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:46:49.665 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:46:49.665 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.5", 2024-09-16T15:46:49.665 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQACUehmKM//IxAAsoFpiiTXNs//v3YXeUf/Iw==", 2024-09-16T15:46:49.665 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:46:49.666 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:46:49.666 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:46:49.666 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:46:49.666 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:46:49.666 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:46:49.666 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:46:49.666 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.6", 2024-09-16T15:46:49.666 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQAFUehmlo/FLRAAp5mvUSWCaBbm85QyTZY0eA==", 2024-09-16T15:46:49.666 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:46:49.666 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:46:49.666 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:46:49.666 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:46:49.666 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:46:49.666 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:46:49.667 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:46:49.667 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.7", 2024-09-16T15:46:49.667 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQAIUehmR32JNxAA27antOIKot1nScQbLZDf5g==", 2024-09-16T15:46:49.667 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:46:49.667 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:46:49.667 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:46:49.667 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:46:49.667 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:46:49.667 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:46:49.667 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:46:49.667 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "client.0", 2024-09-16T15:46:49.667 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDsUOhmf5PBKBAAKE4Y4zbA4h07IBNjPkyptw==", 2024-09-16T15:46:49.667 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:46:49.667 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:46:49.667 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow r", 2024-09-16T15:46:49.667 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-16T15:46:49.668 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:46:49.668 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:46:49.668 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:46:49.668 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "client.1", 2024-09-16T15:46:49.668 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDsUOhm15emNRAAwhK7QmA2c9IYGn04ERvY0g==", 2024-09-16T15:46:49.668 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:46:49.668 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:46:49.668 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow r", 2024-09-16T15:46:49.668 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow rw", 2024-09-16T15:46:49.668 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow rwx" 2024-09-16T15:46:49.668 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:46:49.668 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:46:49.668 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:46:49.668 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "client.2", 2024-09-16T15:46:49.669 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDsUOhm6zUyLxAAkzeiIQTuNZkMM+Oit9rC5A==", 2024-09-16T15:46:49.669 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:46:49.669 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:46:49.669 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow r", 2024-09-16T15:46:49.669 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow rw", 2024-09-16T15:46:49.669 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow rwx" 2024-09-16T15:46:49.669 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:46:49.669 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:46:49.669 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:46:49.669 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "client.3", 2024-09-16T15:46:49.669 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDtUOhmD82EABAAeDf82/hPxpPfAUk8oj5h/Q==", 2024-09-16T15:46:49.669 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:46:49.669 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:46:49.669 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow r", 2024-09-16T15:46:49.669 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow rw", 2024-09-16T15:46:49.670 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow rwx" 2024-09-16T15:46:49.670 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:46:49.670 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:46:49.670 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:46:49.670 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "client.admin", 2024-09-16T15:46:49.670 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDrUOhmQFmVDRAA8NKysbAe475xvg4CPAcFyw==", 2024-09-16T15:46:49.670 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:46:49.670 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow *", 2024-09-16T15:46:49.670 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow *", 2024-09-16T15:46:49.670 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow *", 2024-09-16T15:46:49.670 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:46:49.670 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:46:49.670 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:46:49.670 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:46:49.670 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "mgr.x", 2024-09-16T15:46:49.670 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDrUOhmloW3NxAA8UoLUfrw3wf1nzKbmtK/jw==", 2024-09-16T15:46:49.671 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:46:49.671 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow *", 2024-09-16T15:46:49.671 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile mgr", 2024-09-16T15:46:49.671 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:46:49.671 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:46:49.671 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:46:49.671 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:46:49.671 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "mgr.y", 2024-09-16T15:46:49.671 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDrUOhm1uZOLxAATgjA79vYf0psqQVo03sdDA==", 2024-09-16T15:46:49.671 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:46:49.671 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow *", 2024-09-16T15:46:49.671 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile mgr", 2024-09-16T15:46:49.671 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:46:49.671 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:46:49.671 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:46:49.671 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-09-16T15:46:49.672 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-16T15:46:49.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.659+0000 7f593d600640 1 -- 172.21.15.151:0/1767303870 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f59180794b0 msgr2=0x7f591807b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:49.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.659+0000 7f593d600640 1 --2- 172.21.15.151:0/1767303870 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f59180794b0 0x7f591807b970 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f592800f1d0 tx=0x7f5928009040 comp rx=0 tx=0).stop 2024-09-16T15:46:49.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.659+0000 7f593d600640 1 -- 172.21.15.151:0/1767303870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5938161550 msgr2=0x7f593809c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:49.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.659+0000 7f593d600640 1 --2- 172.21.15.151:0/1767303870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5938161550 0x7f593809c4c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f593000e970 tx=0x7f593000ee30 comp rx=0 tx=0).stop 2024-09-16T15:46:49.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.659+0000 7f593d600640 1 -- 172.21.15.151:0/1767303870 shutdown_connections 2024-09-16T15:46:49.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.659+0000 7f593d600640 1 --2- 172.21.15.151:0/1767303870 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f59180794b0 0x7f591807b970 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:49.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.659+0000 7f593d600640 1 --2- 172.21.15.151:0/1767303870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5938161550 0x7f593809c4c0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:49.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.659+0000 7f593d600640 1 --2- 172.21.15.151:0/1767303870 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f59380af3d0 0x7f59381547d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:49.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.659+0000 7f593d600640 1 --2- 172.21.15.151:0/1767303870 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5938072df0 0x7f59381f88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:49.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.659+0000 7f593d600640 1 -- 172.21.15.151:0/1767303870 >> 172.21.15.151:0/1767303870 conn(0x7f59380b1950 msgr2=0x7f59380740d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:49.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.659+0000 7f593d600640 1 -- 172.21.15.151:0/1767303870 shutdown_connections 2024-09-16T15:46:49.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.659+0000 7f593d600640 1 -- 172.21.15.151:0/1767303870 wait complete. 2024-09-16T15:46:49.678 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-16T15:46:49.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.835+0000 7fe84f200640 1 -- 172.21.15.151:0/3170823966 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe8500b4aa0 msgr2=0x7fe8500b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:49.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.835+0000 7fe84f200640 1 --2- 172.21.15.151:0/3170823966 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe8500b4aa0 0x7fe8500b5a80 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fe85006a1b0 tx=0x7fe83c02a420 comp rx=0 tx=0).stop 2024-09-16T15:46:49.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.835+0000 7fe84f200640 1 -- 172.21.15.151:0/3170823966 shutdown_connections 2024-09-16T15:46:49.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.835+0000 7fe84f200640 1 --2- 172.21.15.151:0/3170823966 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe8500cef40 0x7fe8500cf390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:49.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.835+0000 7fe84f200640 1 --2- 172.21.15.151:0/3170823966 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe8500b4aa0 0x7fe8500b5a80 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:49.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.835+0000 7fe84f200640 1 --2- 172.21.15.151:0/3170823966 >> v2:172.21.15.167:3300/0 conn(0x7fe850161430 0x7fe8500b5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:49.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.835+0000 7fe84f200640 1 -- 172.21.15.151:0/3170823966 >> 172.21.15.151:0/3170823966 conn(0x7fe8500d3dc0 msgr2=0x7fe8500d41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:49.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.835+0000 7fe84f200640 1 -- 172.21.15.151:0/3170823966 shutdown_connections 2024-09-16T15:46:49.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.835+0000 7fe84f200640 1 -- 172.21.15.151:0/3170823966 wait complete. 2024-09-16T15:46:49.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.836+0000 7fe84f200640 1 Processor -- start 2024-09-16T15:46:49.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.836+0000 7fe84f200640 1 -- start start 2024-09-16T15:46:49.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.837+0000 7fe84f200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8500b4aa0 0x7fe8501f5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:49.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.837+0000 7fe84f200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe8500cef40 0x7fe8501f7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:49.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.837+0000 7fe84f200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe850161430 0x7fe8500c9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:49.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.837+0000 7fe84f200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe8500cfc60 con 0x7fe8500cef40 2024-09-16T15:46:49.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.837+0000 7fe84f200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe8500cfae0 con 0x7fe8500b4aa0 2024-09-16T15:46:49.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.837+0000 7fe84f200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fe8500cfde0 con 0x7fe850161430 2024-09-16T15:46:49.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.838+0000 7fe84d400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe8500cef40 0x7fe8501f7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:49.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.838+0000 7fe84d400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe8500cef40 0x7fe8501f7280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:55854/0 (socket says 172.21.15.151:55854) 2024-09-16T15:46:49.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.838+0000 7fe84d400640 1 -- 172.21.15.151:0/4122161765 learned_addr learned my addr 172.21.15.151:0/4122161765 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:46:49.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.838+0000 7fe84de00640 1 --2- 172.21.15.151:0/4122161765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8500b4aa0 0x7fe8501f5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:49.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.838+0000 7fe84e800640 1 --2- 172.21.15.151:0/4122161765 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe850161430 0x7fe8500c9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:49.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.838+0000 7fe84d400640 1 -- 172.21.15.151:0/4122161765 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe850161430 msgr2=0x7fe8500c9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:49.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.838+0000 7fe84d400640 1 --2- 172.21.15.151:0/4122161765 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe850161430 0x7fe8500c9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:49.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.838+0000 7fe84d400640 1 -- 172.21.15.151:0/4122161765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8500b4aa0 msgr2=0x7fe8501f5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:49.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.838+0000 7fe84d400640 1 --2- 172.21.15.151:0/4122161765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8500b4aa0 0x7fe8501f5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:49.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.838+0000 7fe84d400640 1 -- 172.21.15.151:0/4122161765 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe850154270 con 0x7fe8500cef40 2024-09-16T15:46:49.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.838+0000 7fe84d400640 1 --2- 172.21.15.151:0/4122161765 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe8500cef40 0x7fe8501f7280 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fe83c009850 tx=0x7fe83c003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:49.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.839+0000 7fe846000640 1 -- 172.21.15.151:0/4122161765 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe83c03f070 con 0x7fe8500cef40 2024-09-16T15:46:49.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.839+0000 7fe846000640 1 -- 172.21.15.151:0/4122161765 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe83c0047a0 con 0x7fe8500cef40 2024-09-16T15:46:49.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.839+0000 7fe84f200640 1 -- 172.21.15.151:0/4122161765 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe850154550 con 0x7fe8500cef40 2024-09-16T15:46:49.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.839+0000 7fe846000640 1 -- 172.21.15.151:0/4122161765 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe83c035030 con 0x7fe8500cef40 2024-09-16T15:46:49.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.839+0000 7fe84f200640 1 -- 172.21.15.151:0/4122161765 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe850154b10 con 0x7fe8500cef40 2024-09-16T15:46:49.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.840+0000 7fe84f200640 1 -- 172.21.15.151:0/4122161765 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8500cf390 con 0x7fe8500cef40 2024-09-16T15:46:49.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.841+0000 7fe846000640 1 -- 172.21.15.151:0/4122161765 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 24) ==== 102409+0+0 (secure 0 0 0) 0x7fe83c042070 con 0x7fe8500cef40 2024-09-16T15:46:49.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.842+0000 7fe846000640 1 --2- 172.21.15.151:0/4122161765 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe8300794e0 0x7fe83007b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:49.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.842+0000 7fe846000640 1 -- 172.21.15.151:0/4122161765 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 4913+0+0 (secure 0 0 0) 0x7fe83c0ba3f0 con 0x7fe8500cef40 2024-09-16T15:46:49.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.843+0000 7fe84de00640 1 --2- 172.21.15.151:0/4122161765 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe8300794e0 0x7fe83007b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:49.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.843+0000 7fe84de00640 1 --2- 172.21.15.151:0/4122161765 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe8300794e0 0x7fe83007b9a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe84000bb50 tx=0x7fe84000b2e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:49.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:49.846+0000 7fe846000640 1 -- 172.21.15.151:0/4122161765 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe83c083be0 con 0x7fe8500cef40 2024-09-16T15:46:50.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:50.031+0000 7fe84f200640 1 -- 172.21.15.151:0/4122161765 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fe850160990 con 0x7fe8500cef40 2024-09-16T15:46:50.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:50.032+0000 7fe846000640 1 -- 172.21.15.151:0/4122161765 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v72) ==== 84+0+65 (secure 0 0 0) 0x7fe83c088610 con 0x7fe8500cef40 2024-09-16T15:46:50.034 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:46:50.035 INFO:teuthology.orchestra.run.smithi151.stdout:[ 2024-09-16T15:46:50.035 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:46:50.035 INFO:teuthology.orchestra.run.smithi151.stdout: "poolnum": 1, 2024-09-16T15:46:50.035 INFO:teuthology.orchestra.run.smithi151.stdout: "poolname": ".mgr" 2024-09-16T15:46:50.035 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:46:50.035 INFO:teuthology.orchestra.run.smithi151.stdout:] 2024-09-16T15:46:50.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:50.034+0000 7fe84f200640 1 -- 172.21.15.151:0/4122161765 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe8300794e0 msgr2=0x7fe83007b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:50.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:50.034+0000 7fe84f200640 1 --2- 172.21.15.151:0/4122161765 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe8300794e0 0x7fe83007b9a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe84000bb50 tx=0x7fe84000b2e0 comp rx=0 tx=0).stop 2024-09-16T15:46:50.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:50.034+0000 7fe84f200640 1 -- 172.21.15.151:0/4122161765 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe8500cef40 msgr2=0x7fe8501f7280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:50.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:50.034+0000 7fe84f200640 1 --2- 172.21.15.151:0/4122161765 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe8500cef40 0x7fe8501f7280 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fe83c009850 tx=0x7fe83c003910 comp rx=0 tx=0).stop 2024-09-16T15:46:50.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:50.035+0000 7fe84f200640 1 -- 172.21.15.151:0/4122161765 shutdown_connections 2024-09-16T15:46:50.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:50.035+0000 7fe84f200640 1 --2- 172.21.15.151:0/4122161765 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe8300794e0 0x7fe83007b9a0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:50.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:50.035+0000 7fe84f200640 1 --2- 172.21.15.151:0/4122161765 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe850161430 0x7fe8500c9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:50.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:50.035+0000 7fe84f200640 1 --2- 172.21.15.151:0/4122161765 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe8500cef40 0x7fe8501f7280 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:50.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:50.035+0000 7fe84f200640 1 --2- 172.21.15.151:0/4122161765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8500b4aa0 0x7fe8501f5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:50.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:50.035+0000 7fe84f200640 1 -- 172.21.15.151:0/4122161765 >> 172.21.15.151:0/4122161765 conn(0x7fe8500d3dc0 msgr2=0x7fe8500d1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:50.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:50.035+0000 7fe84f200640 1 -- 172.21.15.151:0/4122161765 shutdown_connections 2024-09-16T15:46:50.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:50.035+0000 7fe84f200640 1 -- 172.21.15.151:0/4122161765 wait complete. 2024-09-16T15:46:50.053 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-16T15:46:50.054 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-16T15:46:50.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:50.217+0000 7fb3b9e00640 1 -- 172.21.15.151:0/1190919077 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3b415e710 msgr2=0x7fb3b40d4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:50.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:50.217+0000 7fb3b9e00640 1 --2- 172.21.15.151:0/1190919077 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3b415e710 0x7fb3b40d4a20 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8005fd0 tx=0x7fb3a802a5d0 comp rx=0 tx=0).stop 2024-09-16T15:46:50.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:50.218+0000 7fb3b9e00640 1 -- 172.21.15.151:0/1190919077 shutdown_connections 2024-09-16T15:46:50.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:50.218+0000 7fb3b9e00640 1 --2- 172.21.15.151:0/1190919077 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3b415eaf0 0x7fb3b415ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:50.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:50.218+0000 7fb3b9e00640 1 --2- 172.21.15.151:0/1190919077 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3b415e710 0x7fb3b40d4a20 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:50.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:50.218+0000 7fb3b9e00640 1 -- 172.21.15.151:0/1190919077 >> 172.21.15.151:0/1190919077 conn(0x7fb3b40d3f00 msgr2=0x7fb3b40d4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:50.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:50.218+0000 7fb3b9e00640 1 -- 172.21.15.151:0/1190919077 shutdown_connections 2024-09-16T15:46:50.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:50.218+0000 7fb3b9e00640 1 -- 172.21.15.151:0/1190919077 wait complete. 2024-09-16T15:46:50.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:50.219+0000 7fb3b9e00640 1 Processor -- start 2024-09-16T15:46:50.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:50.220+0000 7fb3b9e00640 1 -- start start 2024-09-16T15:46:50.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:50.220+0000 7fb3b9e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3b40d08d0 0x7fb3b41f5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:50.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:50.220+0000 7fb3b9e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3b415e710 0x7fb3b414e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:50.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:50.221+0000 7fb3b9e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3b415eaf0 0x7fb3b40b5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:50.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:50.221+0000 7fb3b9e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b40d10f0 con 0x7fb3b415e710 2024-09-16T15:46:50.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:50.221+0000 7fb3b9e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b40d0f70 con 0x7fb3b40d08d0 2024-09-16T15:46:50.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:50.221+0000 7fb3b9e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fb3b40d1270 con 0x7fb3b415eaf0 2024-09-16T15:46:50.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:50.221+0000 7fb3b3e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3b415e710 0x7fb3b414e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:50.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:50.221+0000 7fb3b3e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3b415e710 0x7fb3b414e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:55870/0 (socket says 172.21.15.151:55870) 2024-09-16T15:46:50.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:50.221+0000 7fb3b8a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3b40d08d0 0x7fb3b41f5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:50.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:50.221+0000 7fb3b3e00640 1 -- 172.21.15.151:0/1278953936 learned_addr learned my addr 172.21.15.151:0/1278953936 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:46:50.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:50.221+0000 7fb3b9400640 1 --2- 172.21.15.151:0/1278953936 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3b415eaf0 0x7fb3b40b5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:50.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:50.221+0000 7fb3b3e00640 1 -- 172.21.15.151:0/1278953936 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3b415eaf0 msgr2=0x7fb3b40b5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:50.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:50.221+0000 7fb3b3e00640 1 --2- 172.21.15.151:0/1278953936 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3b415eaf0 0x7fb3b40b5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:50.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:50.221+0000 7fb3b3e00640 1 -- 172.21.15.151:0/1278953936 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3b40d08d0 msgr2=0x7fb3b41f5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:50.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:50.221+0000 7fb3b3e00640 1 --2- 172.21.15.151:0/1278953936 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3b40d08d0 0x7fb3b41f5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:50.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:50.221+0000 7fb3b3e00640 1 -- 172.21.15.151:0/1278953936 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3b40b62b0 con 0x7fb3b415e710 2024-09-16T15:46:50.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:50.221+0000 7fb3b8a00640 1 --2- 172.21.15.151:0/1278953936 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3b40d08d0 0x7fb3b41f5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:46:50.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:50.222+0000 7fb3b3e00640 1 --2- 172.21.15.151:0/1278953936 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3b415e710 0x7fb3b414e2b0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fb3a000ea60 tx=0x7fb3a000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:50.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:50.222+0000 7fb3b0c00640 1 -- 172.21.15.151:0/1278953936 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a000cde0 con 0x7fb3b415e710 2024-09-16T15:46:50.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:50.222+0000 7fb3b0c00640 1 -- 172.21.15.151:0/1278953936 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb3a0004020 con 0x7fb3b415e710 2024-09-16T15:46:50.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:50.222+0000 7fb3b9e00640 1 -- 172.21.15.151:0/1278953936 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3b40b6590 con 0x7fb3b415e710 2024-09-16T15:46:50.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:50.222+0000 7fb3b0c00640 1 -- 172.21.15.151:0/1278953936 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a0004450 con 0x7fb3b415e710 2024-09-16T15:46:50.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:50.222+0000 7fb3b9e00640 1 -- 172.21.15.151:0/1278953936 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3b414e7f0 con 0x7fb3b415e710 2024-09-16T15:46:50.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:50.223+0000 7fb3b9e00640 1 -- 172.21.15.151:0/1278953936 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3b40d4a20 con 0x7fb3b415e710 2024-09-16T15:46:50.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:50.224+0000 7fb3b0c00640 1 -- 172.21.15.151:0/1278953936 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 24) ==== 102409+0+0 (secure 0 0 0) 0x7fb3a00035f0 con 0x7fb3b415e710 2024-09-16T15:46:50.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:50.225+0000 7fb3b0c00640 1 --2- 172.21.15.151:0/1278953936 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb3940794b0 0x7fb39407b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:50.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:50.225+0000 7fb3b0c00640 1 -- 172.21.15.151:0/1278953936 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 4913+0+0 (secure 0 0 0) 0x7fb3a0099be0 con 0x7fb3b415e710 2024-09-16T15:46:50.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:50.226+0000 7fb3b8a00640 1 --2- 172.21.15.151:0/1278953936 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb3940794b0 0x7fb39407b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:50.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:50.226+0000 7fb3b8a00640 1 --2- 172.21.15.151:0/1278953936 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb3940794b0 0x7fb39407b970 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8033000 tx=0x7fb3a8004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:50.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:50.229+0000 7fb3b0c00640 1 -- 172.21.15.151:0/1278953936 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb3a0063650 con 0x7fb3b415e710 2024-09-16T15:46:50.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:50.414+0000 7fb3b9e00640 1 -- 172.21.15.151:0/1278953936 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fb3b40b9e80 con 0x7fb3b415e710 2024-09-16T15:46:51.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:51.231+0000 7fb3b0c00640 1 -- 172.21.15.151:0/1278953936 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v73) ==== 138+0+0 (secure 0 0 0) 0x7fb3a0068080 con 0x7fb3b415e710 2024-09-16T15:46:51.234 INFO:teuthology.orchestra.run.smithi151.stderr:pool 'cephfs_metadata' created 2024-09-16T15:46:51.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:51.235+0000 7fb3b9e00640 1 -- 172.21.15.151:0/1278953936 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb3940794b0 msgr2=0x7fb39407b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:51.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:51.235+0000 7fb3b9e00640 1 --2- 172.21.15.151:0/1278953936 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb3940794b0 0x7fb39407b970 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8033000 tx=0x7fb3a8004730 comp rx=0 tx=0).stop 2024-09-16T15:46:51.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:51.235+0000 7fb3b9e00640 1 -- 172.21.15.151:0/1278953936 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3b415e710 msgr2=0x7fb3b414e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:51.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:51.235+0000 7fb3b9e00640 1 --2- 172.21.15.151:0/1278953936 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3b415e710 0x7fb3b414e2b0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fb3a000ea60 tx=0x7fb3a000ef20 comp rx=0 tx=0).stop 2024-09-16T15:46:51.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:51.235+0000 7fb3b9e00640 1 -- 172.21.15.151:0/1278953936 shutdown_connections 2024-09-16T15:46:51.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:51.235+0000 7fb3b9e00640 1 --2- 172.21.15.151:0/1278953936 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb3940794b0 0x7fb39407b970 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:51.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:51.235+0000 7fb3b9e00640 1 --2- 172.21.15.151:0/1278953936 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3b415eaf0 0x7fb3b40b5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:51.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:51.235+0000 7fb3b9e00640 1 --2- 172.21.15.151:0/1278953936 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3b415e710 0x7fb3b414e2b0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:51.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:51.235+0000 7fb3b9e00640 1 --2- 172.21.15.151:0/1278953936 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3b40d08d0 0x7fb3b41f5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:51.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:51.235+0000 7fb3b9e00640 1 -- 172.21.15.151:0/1278953936 >> 172.21.15.151:0/1278953936 conn(0x7fb3b40d3f00 msgr2=0x7fb3b415b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:51.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:51.236+0000 7fb3b9e00640 1 -- 172.21.15.151:0/1278953936 shutdown_connections 2024-09-16T15:46:51.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:51.236+0000 7fb3b9e00640 1 -- 172.21.15.151:0/1278953936 wait complete. 2024-09-16T15:46:51.258 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-16T15:46:51.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:51.422+0000 7f9562800640 1 -- 172.21.15.151:0/3516999383 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f955c0ccf80 msgr2=0x7f955c06f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:51.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:51.422+0000 7f9562800640 1 --2- 172.21.15.151:0/3516999383 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f955c0ccf80 0x7f955c06f900 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f9548005fd0 tx=0x7f954802a660 comp rx=0 tx=0).stop 2024-09-16T15:46:51.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:51.422+0000 7f9562800640 1 -- 172.21.15.151:0/3516999383 shutdown_connections 2024-09-16T15:46:51.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:51.422+0000 7f9562800640 1 --2- 172.21.15.151:0/3516999383 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f955c0ccf80 0x7f955c06f900 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:51.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:51.422+0000 7f9562800640 1 --2- 172.21.15.151:0/3516999383 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f955c15a200 0x7f955c0ca080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:51.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:51.422+0000 7f9562800640 1 -- 172.21.15.151:0/3516999383 >> 172.21.15.151:0/3516999383 conn(0x7f955c0cc490 msgr2=0x7f955c0cc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:51.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:51.422+0000 7f9562800640 1 -- 172.21.15.151:0/3516999383 shutdown_connections 2024-09-16T15:46:51.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:51.422+0000 7f9562800640 1 -- 172.21.15.151:0/3516999383 wait complete. 2024-09-16T15:46:51.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:51.424+0000 7f9562800640 1 Processor -- start 2024-09-16T15:46:51.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:51.424+0000 7f9562800640 1 -- start start 2024-09-16T15:46:51.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:51.425+0000 7f9562800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f955c06fe40 0x7f955c1f5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:51.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:51.425+0000 7f9562800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f955c0ccf80 0x7f955c158b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:51.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:51.425+0000 7f9562800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f955c15a200 0x7f955c1f7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:51.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:51.425+0000 7f9562800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f955c0cdde0 con 0x7f955c06fe40 2024-09-16T15:46:51.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:51.425+0000 7f9562800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f955c0cdc60 con 0x7f955c15a200 2024-09-16T15:46:51.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:51.425+0000 7f9562800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f955c0cdf60 con 0x7f955c0ccf80 2024-09-16T15:46:51.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:51.425+0000 7f9561400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f955c06fe40 0x7f955c1f5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:51.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:51.425+0000 7f9561400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f955c06fe40 0x7f955c1f5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:55880/0 (socket says 172.21.15.151:55880) 2024-09-16T15:46:51.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:51.425+0000 7f9561400640 1 -- 172.21.15.151:0/1898771119 learned_addr learned my addr 172.21.15.151:0/1898771119 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:46:51.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:51.426+0000 7f9560a00640 1 --2- 172.21.15.151:0/1898771119 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f955c0ccf80 0x7f955c158b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:51.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:51.426+0000 7f9561e00640 1 --2- 172.21.15.151:0/1898771119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f955c15a200 0x7f955c1f7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:51.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:51.426+0000 7f9561400640 1 -- 172.21.15.151:0/1898771119 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f955c0ccf80 msgr2=0x7f955c158b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:51.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:51.426+0000 7f9561400640 1 --2- 172.21.15.151:0/1898771119 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f955c0ccf80 0x7f955c158b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:51.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:51.426+0000 7f9561400640 1 -- 172.21.15.151:0/1898771119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f955c15a200 msgr2=0x7f955c1f7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:51.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:51.426+0000 7f9561400640 1 --2- 172.21.15.151:0/1898771119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f955c15a200 0x7f955c1f7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:51.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:51.426+0000 7f9561400640 1 -- 172.21.15.151:0/1898771119 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f955c1f56c0 con 0x7f955c06fe40 2024-09-16T15:46:51.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:51.426+0000 7f9561e00640 1 --2- 172.21.15.151:0/1898771119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f955c15a200 0x7f955c1f7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:46:51.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:51.426+0000 7f9561400640 1 --2- 172.21.15.151:0/1898771119 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f955c06fe40 0x7f955c1f5180 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f955000ea20 tx=0x7f955000eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:51.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:51.426+0000 7f9559600640 1 -- 172.21.15.151:0/1898771119 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f955000cde0 con 0x7f955c06fe40 2024-09-16T15:46:51.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:51.426+0000 7f9559600640 1 -- 172.21.15.151:0/1898771119 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9550011560 con 0x7f955c06fe40 2024-09-16T15:46:51.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:51.426+0000 7f9562800640 1 -- 172.21.15.151:0/1898771119 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f955c1f8700 con 0x7f955c06fe40 2024-09-16T15:46:51.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:51.426+0000 7f9559600640 1 -- 172.21.15.151:0/1898771119 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9550011990 con 0x7f955c06fe40 2024-09-16T15:46:51.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:51.427+0000 7f9562800640 1 -- 172.21.15.151:0/1898771119 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f955c1f4980 con 0x7f955c06fe40 2024-09-16T15:46:51.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:51.428+0000 7f9562800640 1 -- 172.21.15.151:0/1898771119 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f955c06f900 con 0x7f955c06fe40 2024-09-16T15:46:51.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:51.429+0000 7f9559600640 1 -- 172.21.15.151:0/1898771119 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 24) ==== 102409+0+0 (secure 0 0 0) 0x7f955001a400 con 0x7f955c06fe40 2024-09-16T15:46:51.433 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:51.431+0000 7f9559600640 1 --2- 172.21.15.151:0/1898771119 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f953c0794b0 0x7f953c07b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:51.433 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:51.431+0000 7f9559600640 1 -- 172.21.15.151:0/1898771119 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5283+0+0 (secure 0 0 0) 0x7f955009a5d0 con 0x7f955c06fe40 2024-09-16T15:46:51.433 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:51.431+0000 7f9560a00640 1 --2- 172.21.15.151:0/1898771119 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f953c0794b0 0x7f953c07b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:51.434 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:51.432+0000 7f9560a00640 1 --2- 172.21.15.151:0/1898771119 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f953c0794b0 0x7f953c07b970 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9548033000 tx=0x7f9548004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:51.437 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:51.435+0000 7f9559600640 1 -- 172.21.15.151:0/1898771119 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9550028020 con 0x7f955c06fe40 2024-09-16T15:46:51.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:51.621+0000 7f9562800640 1 -- 172.21.15.151:0/1898771119 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f955c15e970 con 0x7f955c06fe40 2024-09-16T15:46:52.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:52.238+0000 7f9559600640 1 -- 172.21.15.151:0/1898771119 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v74) ==== 170+0+0 (secure 0 0 0) 0x7f9550016030 con 0x7f955c06fe40 2024-09-16T15:46:52.240 INFO:teuthology.orchestra.run.smithi151.stderr:pool 'cephfs_data' created 2024-09-16T15:46:52.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:52.241+0000 7f9562800640 1 -- 172.21.15.151:0/1898771119 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f953c0794b0 msgr2=0x7f953c07b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:52.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:52.241+0000 7f9562800640 1 --2- 172.21.15.151:0/1898771119 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f953c0794b0 0x7f953c07b970 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9548033000 tx=0x7f9548004730 comp rx=0 tx=0).stop 2024-09-16T15:46:52.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:52.241+0000 7f9562800640 1 -- 172.21.15.151:0/1898771119 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f955c06fe40 msgr2=0x7f955c1f5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:52.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:52.241+0000 7f9562800640 1 --2- 172.21.15.151:0/1898771119 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f955c06fe40 0x7f955c1f5180 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f955000ea20 tx=0x7f955000eee0 comp rx=0 tx=0).stop 2024-09-16T15:46:52.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:52.242+0000 7f9562800640 1 -- 172.21.15.151:0/1898771119 shutdown_connections 2024-09-16T15:46:52.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:52.242+0000 7f9562800640 1 --2- 172.21.15.151:0/1898771119 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f953c0794b0 0x7f953c07b970 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:52.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:52.242+0000 7f9562800640 1 --2- 172.21.15.151:0/1898771119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f955c15a200 0x7f955c1f7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:52.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:52.242+0000 7f9562800640 1 --2- 172.21.15.151:0/1898771119 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f955c0ccf80 0x7f955c158b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:52.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:52.242+0000 7f9562800640 1 --2- 172.21.15.151:0/1898771119 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f955c06fe40 0x7f955c1f5180 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:52.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:52.242+0000 7f9562800640 1 -- 172.21.15.151:0/1898771119 >> 172.21.15.151:0/1898771119 conn(0x7f955c0cc490 msgr2=0x7f955c0cc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:52.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:52.243+0000 7f9562800640 1 -- 172.21.15.151:0/1898771119 shutdown_connections 2024-09-16T15:46:52.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:52.243+0000 7f9562800640 1 -- 172.21.15.151:0/1898771119 wait complete. 2024-09-16T15:46:52.275 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-16T15:46:52.439 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:52.436+0000 7eff71200640 1 -- 172.21.15.151:0/3404198683 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff6c0b1880 msgr2=0x7eff6c0688b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:52.439 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:52.436+0000 7eff71200640 1 --2- 172.21.15.151:0/3404198683 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff6c0b1880 0x7eff6c0688b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7eff5c005fd0 tx=0x7eff5c02a5d0 comp rx=0 tx=0).stop 2024-09-16T15:46:52.439 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:52.436+0000 7eff71200640 1 -- 172.21.15.151:0/3404198683 shutdown_connections 2024-09-16T15:46:52.439 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:52.436+0000 7eff71200640 1 --2- 172.21.15.151:0/3404198683 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7eff6c15e980 0x7eff6c15fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:52.439 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:52.436+0000 7eff71200640 1 --2- 172.21.15.151:0/3404198683 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff6c0b1880 0x7eff6c0688b0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:52.439 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:52.436+0000 7eff71200640 1 -- 172.21.15.151:0/3404198683 >> 172.21.15.151:0/3404198683 conn(0x7eff6c0b5680 msgr2=0x7eff6c0b5a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:52.439 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:52.436+0000 7eff71200640 1 -- 172.21.15.151:0/3404198683 shutdown_connections 2024-09-16T15:46:52.439 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:52.436+0000 7eff71200640 1 -- 172.21.15.151:0/3404198683 wait complete. 2024-09-16T15:46:52.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:52.438+0000 7eff71200640 1 Processor -- start 2024-09-16T15:46:52.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:52.438+0000 7eff71200640 1 -- start start 2024-09-16T15:46:52.442 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:52.439+0000 7eff71200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7eff6c0b1880 0x7eff6c14f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:52.443 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:52.439+0000 7eff71200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff6c15e980 0x7eff6c0cd710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:52.443 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:52.439+0000 7eff71200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff6c15f790 0x7eff6c0cdc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:52.443 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:52.439+0000 7eff71200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7eff6c15c8b0 con 0x7eff6c15e980 2024-09-16T15:46:52.443 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:52.439+0000 7eff71200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7eff6c0b1e70 con 0x7eff6c15f790 2024-09-16T15:46:52.443 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:52.439+0000 7eff71200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7eff6c15ca30 con 0x7eff6c0b1880 2024-09-16T15:46:52.443 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:52.439+0000 7eff6aa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff6c15e980 0x7eff6c0cd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:52.443 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:52.439+0000 7eff6aa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff6c15e980 0x7eff6c0cd710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:55886/0 (socket says 172.21.15.151:55886) 2024-09-16T15:46:52.443 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:52.439+0000 7eff6aa00640 1 -- 172.21.15.151:0/226877935 learned_addr learned my addr 172.21.15.151:0/226877935 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:46:52.443 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:52.439+0000 7eff6b400640 1 --2- 172.21.15.151:0/226877935 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7eff6c0b1880 0x7eff6c14f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:52.443 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:52.439+0000 7eff6be00640 1 --2- 172.21.15.151:0/226877935 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff6c15f790 0x7eff6c0cdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:52.443 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:52.440+0000 7eff6aa00640 1 -- 172.21.15.151:0/226877935 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7eff6c0b1880 msgr2=0x7eff6c14f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:52.443 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:52.440+0000 7eff6aa00640 1 --2- 172.21.15.151:0/226877935 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7eff6c0b1880 0x7eff6c14f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:52.443 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:52.440+0000 7eff6aa00640 1 -- 172.21.15.151:0/226877935 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff6c15f790 msgr2=0x7eff6c0cdc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:52.443 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:52.440+0000 7eff6aa00640 1 --2- 172.21.15.151:0/226877935 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff6c15f790 0x7eff6c0cdc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:52.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:52.440+0000 7eff6aa00640 1 -- 172.21.15.151:0/226877935 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff6c0d0310 con 0x7eff6c15e980 2024-09-16T15:46:52.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:52.440+0000 7eff6aa00640 1 --2- 172.21.15.151:0/226877935 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff6c15e980 0x7eff6c0cd710 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7eff5800c400 tx=0x7eff5800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:52.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:52.440+0000 7eff63400640 1 -- 172.21.15.151:0/226877935 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff58017070 con 0x7eff6c15e980 2024-09-16T15:46:52.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:52.440+0000 7eff63400640 1 -- 172.21.15.151:0/226877935 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7eff580053c0 con 0x7eff6c15e980 2024-09-16T15:46:52.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:52.440+0000 7eff71200640 1 -- 172.21.15.151:0/226877935 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff6c0d05f0 con 0x7eff6c15e980 2024-09-16T15:46:52.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:52.440+0000 7eff63400640 1 -- 172.21.15.151:0/226877935 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff58003850 con 0x7eff6c15e980 2024-09-16T15:46:52.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:52.441+0000 7eff71200640 1 -- 172.21.15.151:0/226877935 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff6c0d0bb0 con 0x7eff6c15e980 2024-09-16T15:46:52.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:52.442+0000 7eff71200640 1 -- 172.21.15.151:0/226877935 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff6c15cfc0 con 0x7eff6c15e980 2024-09-16T15:46:52.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:52.443+0000 7eff63400640 1 -- 172.21.15.151:0/226877935 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 24) ==== 102409+0+0 (secure 0 0 0) 0x7eff5801a070 con 0x7eff6c15e980 2024-09-16T15:46:52.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:52.444+0000 7eff63400640 1 --2- 172.21.15.151:0/226877935 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7eff4c0794b0 0x7eff4c07b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:52.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:52.444+0000 7eff63400640 1 -- 172.21.15.151:0/226877935 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5665+0+0 (secure 0 0 0) 0x7eff580988b0 con 0x7eff6c15e980 2024-09-16T15:46:52.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:52.444+0000 7eff6b400640 1 --2- 172.21.15.151:0/226877935 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7eff4c0794b0 0x7eff4c07b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:52.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:52.445+0000 7eff6b400640 1 --2- 172.21.15.151:0/226877935 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7eff4c0794b0 0x7eff4c07b970 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7eff5c033000 tx=0x7eff5c004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:52.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:52.449+0000 7eff63400640 1 -- 172.21.15.151:0/226877935 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7eff580620a0 con 0x7eff6c15e980 2024-09-16T15:46:52.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:52.669+0000 7eff71200640 1 -- 172.21.15.151:0/226877935 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7eff6c0b5c70 con 0x7eff6c15e980 2024-09-16T15:46:53.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.240+0000 7eff63400640 1 -- 172.21.15.151:0/226877935 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v42) ==== 174+0+0 (secure 0 0 0) 0x7eff58066ad0 con 0x7eff6c15e980 2024-09-16T15:46:53.243 INFO:teuthology.orchestra.run.smithi151.stderr:new fs with metadata pool 10 and data pool 11 2024-09-16T15:46:53.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.244+0000 7eff71200640 1 -- 172.21.15.151:0/226877935 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7eff4c0794b0 msgr2=0x7eff4c07b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:53.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.244+0000 7eff71200640 1 --2- 172.21.15.151:0/226877935 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7eff4c0794b0 0x7eff4c07b970 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7eff5c033000 tx=0x7eff5c004730 comp rx=0 tx=0).stop 2024-09-16T15:46:53.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.244+0000 7eff71200640 1 -- 172.21.15.151:0/226877935 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff6c15e980 msgr2=0x7eff6c0cd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:53.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.244+0000 7eff71200640 1 --2- 172.21.15.151:0/226877935 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff6c15e980 0x7eff6c0cd710 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7eff5800c400 tx=0x7eff5800c8c0 comp rx=0 tx=0).stop 2024-09-16T15:46:53.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.244+0000 7eff71200640 1 -- 172.21.15.151:0/226877935 shutdown_connections 2024-09-16T15:46:53.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.244+0000 7eff71200640 1 --2- 172.21.15.151:0/226877935 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7eff4c0794b0 0x7eff4c07b970 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:53.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.244+0000 7eff71200640 1 --2- 172.21.15.151:0/226877935 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff6c15f790 0x7eff6c0cdc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:53.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.244+0000 7eff71200640 1 --2- 172.21.15.151:0/226877935 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff6c15e980 0x7eff6c0cd710 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:53.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.244+0000 7eff71200640 1 --2- 172.21.15.151:0/226877935 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7eff6c0b1880 0x7eff6c14f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:53.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.244+0000 7eff71200640 1 -- 172.21.15.151:0/226877935 >> 172.21.15.151:0/226877935 conn(0x7eff6c0b5680 msgr2=0x7eff6c074240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:53.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.246+0000 7eff71200640 1 -- 172.21.15.151:0/226877935 shutdown_connections 2024-09-16T15:46:53.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.246+0000 7eff71200640 1 -- 172.21.15.151:0/226877935 wait complete. 2024-09-16T15:46:53.290 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:46:53.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.447+0000 7f1b2de00640 1 -- 172.21.15.151:0/859522242 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1b280cd5f0 msgr2=0x7f1b280cda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:53.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.447+0000 7f1b2de00640 1 --2- 172.21.15.151:0/859522242 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1b280cd5f0 0x7f1b280cda10 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f1b20005fd0 tx=0x7f1b2002aa90 comp rx=0 tx=0).stop 2024-09-16T15:46:53.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.447+0000 7f1b2de00640 1 -- 172.21.15.151:0/859522242 shutdown_connections 2024-09-16T15:46:53.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.447+0000 7f1b2de00640 1 --2- 172.21.15.151:0/859522242 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1b280cd5f0 0x7f1b280cda10 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:53.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.447+0000 7f1b2de00640 1 --2- 172.21.15.151:0/859522242 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1b280ad880 0x7f1b280adcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:53.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.447+0000 7f1b2de00640 1 -- 172.21.15.151:0/859522242 >> 172.21.15.151:0/859522242 conn(0x7f1b280cfa20 msgr2=0x7f1b280cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:53.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.447+0000 7f1b2de00640 1 -- 172.21.15.151:0/859522242 shutdown_connections 2024-09-16T15:46:53.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.447+0000 7f1b2de00640 1 -- 172.21.15.151:0/859522242 wait complete. 2024-09-16T15:46:53.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.449+0000 7f1b2de00640 1 Processor -- start 2024-09-16T15:46:53.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.449+0000 7f1b2de00640 1 -- start start 2024-09-16T15:46:53.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.450+0000 7f1b2de00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1b2806fb90 0x7f1b281f1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:53.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.450+0000 7f1b2de00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1b280ad880 0x7f1b280c5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:53.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.450+0000 7f1b2de00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b280cd5f0 0x7f1b280b4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:53.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.450+0000 7f1b2de00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1b2815eda0 con 0x7f1b280ad880 2024-09-16T15:46:53.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.450+0000 7f1b2de00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1b2815ec20 con 0x7f1b280cd5f0 2024-09-16T15:46:53.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.450+0000 7f1b2de00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f1b2815ef20 con 0x7f1b2806fb90 2024-09-16T15:46:53.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.450+0000 7f1b27e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1b280ad880 0x7f1b280c5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:53.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.450+0000 7f1b27e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1b280ad880 0x7f1b280c5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:46028/0 (socket says 172.21.15.151:46028) 2024-09-16T15:46:53.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.450+0000 7f1b27e00640 1 -- 172.21.15.151:0/2290453206 learned_addr learned my addr 172.21.15.151:0/2290453206 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:46:53.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.450+0000 7f1b2ca00640 1 --2- 172.21.15.151:0/2290453206 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1b2806fb90 0x7f1b281f1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:53.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.450+0000 7f1b2d400640 1 --2- 172.21.15.151:0/2290453206 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b280cd5f0 0x7f1b280b4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:53.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.451+0000 7f1b27e00640 1 -- 172.21.15.151:0/2290453206 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1b2806fb90 msgr2=0x7f1b281f1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:53.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.451+0000 7f1b27e00640 1 --2- 172.21.15.151:0/2290453206 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1b2806fb90 0x7f1b281f1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:53.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.451+0000 7f1b27e00640 1 -- 172.21.15.151:0/2290453206 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b280cd5f0 msgr2=0x7f1b280b4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:53.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.451+0000 7f1b27e00640 1 --2- 172.21.15.151:0/2290453206 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b280cd5f0 0x7f1b280b4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:53.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.451+0000 7f1b27e00640 1 -- 172.21.15.151:0/2290453206 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b280c6260 con 0x7f1b280ad880 2024-09-16T15:46:53.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.451+0000 7f1b27e00640 1 --2- 172.21.15.151:0/2290453206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1b280ad880 0x7f1b280c5d20 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f1b1400d950 tx=0x7f1b1400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:53.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.451+0000 7f1b24c00640 1 -- 172.21.15.151:0/2290453206 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b14017070 con 0x7f1b280ad880 2024-09-16T15:46:53.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.451+0000 7f1b24c00640 1 -- 172.21.15.151:0/2290453206 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1b14009d20 con 0x7f1b280ad880 2024-09-16T15:46:53.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.451+0000 7f1b24c00640 1 -- 172.21.15.151:0/2290453206 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b14004280 con 0x7f1b280ad880 2024-09-16T15:46:53.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.451+0000 7f1b2de00640 1 -- 172.21.15.151:0/2290453206 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b280c3780 con 0x7f1b280ad880 2024-09-16T15:46:53.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.451+0000 7f1b2de00640 1 -- 172.21.15.151:0/2290453206 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b281f5c30 con 0x7f1b280ad880 2024-09-16T15:46:53.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.452+0000 7f1b2de00640 1 -- 172.21.15.151:0/2290453206 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b281f5a40 con 0x7f1b280ad880 2024-09-16T15:46:53.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.453+0000 7f1b24c00640 1 -- 172.21.15.151:0/2290453206 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 24) ==== 102409+0+0 (secure 0 0 0) 0x7f1b140035f0 con 0x7f1b280ad880 2024-09-16T15:46:53.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.454+0000 7f1b24c00640 1 --2- 172.21.15.151:0/2290453206 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1b080794b0 0x7f1b0807b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:53.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.454+0000 7f1b24c00640 1 -- 172.21.15.151:0/2290453206 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5765+0+0 (secure 0 0 0) 0x7f1b14099660 con 0x7f1b280ad880 2024-09-16T15:46:53.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.455+0000 7f1b2ca00640 1 --2- 172.21.15.151:0/2290453206 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1b080794b0 0x7f1b0807b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:53.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.455+0000 7f1b2ca00640 1 --2- 172.21.15.151:0/2290453206 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1b080794b0 0x7f1b0807b970 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f1b180017c0 tx=0x7f1b18009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:53.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.458+0000 7f1b24c00640 1 -- 172.21.15.151:0/2290453206 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1b14062e50 con 0x7f1b280ad880 2024-09-16T15:46:53.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.642+0000 7f1b2de00640 1 -- 172.21.15.151:0/2290453206 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1b280adcd0 con 0x7f1b280ad880 2024-09-16T15:46:53.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.643+0000 7f1b24c00640 1 -- 172.21.15.151:0/2290453206 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+16828 (secure 0 0 0) 0x7f1b14067880 con 0x7f1b280ad880 2024-09-16T15:46:53.646 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:46:53.646 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":75,"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","created":"2024-09-16T15:38:56.348535+0000","modified":"2024-09-16T15:46:52.671263+0000","last_up_change":"2024-09-16T15:39:10.681384+0000","last_in_change":"2024-09-16T15:39:00.544477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:39:11.602252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:46:50.425692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-16T15:46:51.632208+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5524076-a903-4d0a-af6a-caf474c4765d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6809","nonce":3111342741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6811","nonce":3111342741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6815","nonce":3111342741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6813","nonce":3111342741}]},"public_addr":"172.21.15.151:6809/3111342741","cluster_addr":"172.21.15.151:6811/3111342741","heartbeat_back_addr":"172.21.15.151:6815/3111342741","heartbeat_front_addr":"172.21.15.151:6813/3111342741","state":["exists","up"]},{"osd":1,"uuid":"dd25925f-0ae1-494e-8529-4dc205759db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6801","nonce":3368304087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6803","nonce":3368304087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6807","nonce":3368304087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6805","nonce":3368304087}]},"public_addr":"172.21.15.151:6801/3368304087","cluster_addr":"172.21.15.151:6803/3368304087","heartbeat_back_addr":"172.21.15.151:6807/3368304087","heartbeat_front_addr":"172.21.15.151:6805/3368304087","state":["exists","up"]},{"osd":2,"uuid":"c2f18115-013d-4da6-a0fe-cb978f4494b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6823","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6825","nonce":3079373271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6827","nonce":3079373271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6831","nonce":3079373271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6829","nonce":3079373271}]},"public_addr":"172.21.15.151:6825/3079373271","cluster_addr":"172.21.15.151:6827/3079373271","heartbeat_back_addr":"172.21.15.151:6831/3079373271","heartbeat_front_addr":"172.21.15.151:6829/3079373271","state":["exists","up"]},{"osd":3,"uuid":"36edb718-2fcb-4d88-8ddc-c321eed4279d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6817","nonce":1142509041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6819","nonce":1142509041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6824","nonce":1142509041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6821","nonce":1142509041}]},"public_addr":"172.21.15.151:6817/1142509041","cluster_addr":"172.21.15.151:6819/1142509041","heartbeat_back_addr":"172.21.15.151:6824/1142509041","heartbeat_front_addr":"172.21.15.151:6821/1142509041","state":["exists","up"]},{"osd":4,"uuid":"64d33708-cd0a-4236-abe3-1b646e82ab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6811","nonce":2735495271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6813","nonce":2735495271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6817","nonce":2735495271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6815","nonce":2735495271}]},"public_addr":"172.21.15.167:6811/2735495271","cluster_addr":"172.21.15.167:6813/2735495271","heartbeat_back_addr":"172.21.15.167:6817/2735495271","heartbeat_front_addr":"172.21.15.167:6815/2735495271","state":["exists","up"]},{"osd":5,"uuid":"f14b44a9-5910-458c-b518-09a8d0de94de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6827","nonce":1494916572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6829","nonce":1494916572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6833","nonce":1494916572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6831","nonce":1494916572}]},"public_addr":"172.21.15.167:6827/1494916572","cluster_addr":"172.21.15.167:6829/1494916572","heartbeat_back_addr":"172.21.15.167:6833/1494916572","heartbeat_front_addr":"172.21.15.167:6831/1494916572","state":["exists","up"]},{"osd":6,"uuid":"39335545-2654-4710-84b0-a9aa47cfcbba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6819","nonce":2538273273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6821","nonce":2538273273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6825","nonce":2538273273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6823","nonce":2538273273}]},"public_addr":"172.21.15.167:6819/2538273273","cluster_addr":"172.21.15.167:6821/2538273273","heartbeat_back_addr":"172.21.15.167:6825/2538273273","heartbeat_front_addr":"172.21.15.167:6823/2538273273","state":["exists","up"]},{"osd":7,"uuid":"4f8bfc81-53dd-4339-ac22-1a4007d9f963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6803","nonce":265072227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6805","nonce":265072227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6809","nonce":265072227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6807","nonce":265072227}]},"public_addr":"172.21.15.167:6803/265072227","cluster_addr":"172.21.15.167:6805/265072227","heartbeat_back_addr":"172.21.15.167:6809/265072227","heartbeat_front_addr":"172.21.15.167:6807/265072227","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.150359+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.084358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.722368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.332809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.320709+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.395153+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.935250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.801443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:46:53.648 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.646+0000 7f1b2de00640 1 -- 172.21.15.151:0/2290453206 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1b080794b0 msgr2=0x7f1b0807b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:53.648 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.646+0000 7f1b2de00640 1 --2- 172.21.15.151:0/2290453206 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1b080794b0 0x7f1b0807b970 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f1b180017c0 tx=0x7f1b18009040 comp rx=0 tx=0).stop 2024-09-16T15:46:53.648 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.646+0000 7f1b2de00640 1 -- 172.21.15.151:0/2290453206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1b280ad880 msgr2=0x7f1b280c5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:53.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.646+0000 7f1b2de00640 1 --2- 172.21.15.151:0/2290453206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1b280ad880 0x7f1b280c5d20 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f1b1400d950 tx=0x7f1b1400de10 comp rx=0 tx=0).stop 2024-09-16T15:46:53.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.646+0000 7f1b2de00640 1 -- 172.21.15.151:0/2290453206 shutdown_connections 2024-09-16T15:46:53.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.646+0000 7f1b2de00640 1 --2- 172.21.15.151:0/2290453206 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1b080794b0 0x7f1b0807b970 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:53.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.646+0000 7f1b2de00640 1 --2- 172.21.15.151:0/2290453206 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b280cd5f0 0x7f1b280b4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:53.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.646+0000 7f1b2de00640 1 --2- 172.21.15.151:0/2290453206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1b280ad880 0x7f1b280c5d20 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:53.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.646+0000 7f1b2de00640 1 --2- 172.21.15.151:0/2290453206 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1b2806fb90 0x7f1b281f1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:53.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.646+0000 7f1b2de00640 1 -- 172.21.15.151:0/2290453206 >> 172.21.15.151:0/2290453206 conn(0x7f1b280cfa20 msgr2=0x7f1b2806ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:53.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.646+0000 7f1b2de00640 1 -- 172.21.15.151:0/2290453206 shutdown_connections 2024-09-16T15:46:53.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.646+0000 7f1b2de00640 1 -- 172.21.15.151:0/2290453206 wait complete. 2024-09-16T15:46:53.664 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:46:53.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.820+0000 7f6955a00640 1 -- 172.21.15.151:0/1240869038 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69500af3d0 msgr2=0x7f695006f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:53.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.820+0000 7f6955a00640 1 --2- 172.21.15.151:0/1240869038 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69500af3d0 0x7f695006f490 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f695006d9e0 tx=0x7f693c02a4d0 comp rx=0 tx=0).stop 2024-09-16T15:46:53.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.821+0000 7f6955a00640 1 -- 172.21.15.151:0/1240869038 shutdown_connections 2024-09-16T15:46:53.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.821+0000 7f6955a00640 1 --2- 172.21.15.151:0/1240869038 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f695006f9d0 0x7f6950161c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:53.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.821+0000 7f6955a00640 1 --2- 172.21.15.151:0/1240869038 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69500af3d0 0x7f695006f490 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:53.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.821+0000 7f6955a00640 1 --2- 172.21.15.151:0/1240869038 >> v2:172.21.15.167:3300/0 conn(0x7f6950161550 0x7f695006ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:53.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.821+0000 7f6955a00640 1 -- 172.21.15.151:0/1240869038 >> 172.21.15.151:0/1240869038 conn(0x7f69500b1950 msgr2=0x7f69500b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:53.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.821+0000 7f6955a00640 1 -- 172.21.15.151:0/1240869038 shutdown_connections 2024-09-16T15:46:53.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.821+0000 7f6955a00640 1 -- 172.21.15.151:0/1240869038 wait complete. 2024-09-16T15:46:53.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.822+0000 7f6955a00640 1 Processor -- start 2024-09-16T15:46:53.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.822+0000 7f6955a00640 1 -- start start 2024-09-16T15:46:53.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.823+0000 7f6955a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f695006f9d0 0x7f6950151090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:53.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.823+0000 7f6955a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69500af3d0 0x7f69501f8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:53.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.823+0000 7f6955a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6950161550 0x7f695009c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:53.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.823+0000 7f6955a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6950162560 con 0x7f69500af3d0 2024-09-16T15:46:53.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.823+0000 7f6955a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f69501623e0 con 0x7f6950161550 2024-09-16T15:46:53.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.823+0000 7f6955a00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f69501626e0 con 0x7f695006f9d0 2024-09-16T15:46:53.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.824+0000 7f694e000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69500af3d0 0x7f69501f8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:53.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.824+0000 7f694e000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69500af3d0 0x7f69501f8a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:46042/0 (socket says 172.21.15.151:46042) 2024-09-16T15:46:53.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.824+0000 7f694e000640 1 -- 172.21.15.151:0/2132483396 learned_addr learned my addr 172.21.15.151:0/2132483396 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:46:53.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.824+0000 7f694ea00640 1 --2- 172.21.15.151:0/2132483396 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f695006f9d0 0x7f6950151090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:53.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.824+0000 7f694f400640 1 --2- 172.21.15.151:0/2132483396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6950161550 0x7f695009c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:53.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.824+0000 7f694e000640 1 -- 172.21.15.151:0/2132483396 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f695006f9d0 msgr2=0x7f6950151090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:53.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.824+0000 7f694e000640 1 --2- 172.21.15.151:0/2132483396 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f695006f9d0 0x7f6950151090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:53.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.824+0000 7f694e000640 1 -- 172.21.15.151:0/2132483396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6950161550 msgr2=0x7f695009c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:53.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.824+0000 7f694e000640 1 --2- 172.21.15.151:0/2132483396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6950161550 0x7f695009c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:53.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.824+0000 7f694e000640 1 -- 172.21.15.151:0/2132483396 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69501f48f0 con 0x7f69500af3d0 2024-09-16T15:46:53.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.825+0000 7f694e000640 1 --2- 172.21.15.151:0/2132483396 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69500af3d0 0x7f69501f8a90 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f693c000c00 tx=0x7f693c0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:53.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.825+0000 7f6946a00640 1 -- 172.21.15.151:0/2132483396 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f693c03f070 con 0x7f69500af3d0 2024-09-16T15:46:53.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.825+0000 7f6946a00640 1 -- 172.21.15.151:0/2132483396 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f693c004890 con 0x7f69500af3d0 2024-09-16T15:46:53.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.825+0000 7f6946a00640 1 -- 172.21.15.151:0/2132483396 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f693c035030 con 0x7f69500af3d0 2024-09-16T15:46:53.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.825+0000 7f6955a00640 1 -- 172.21.15.151:0/2132483396 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f695009cd80 con 0x7f69500af3d0 2024-09-16T15:46:53.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.825+0000 7f6955a00640 1 -- 172.21.15.151:0/2132483396 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69501f8290 con 0x7f69500af3d0 2024-09-16T15:46:53.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.826+0000 7f6955a00640 1 -- 172.21.15.151:0/2132483396 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69500b3030 con 0x7f69500af3d0 2024-09-16T15:46:53.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.827+0000 7f6946a00640 1 -- 172.21.15.151:0/2132483396 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 24) ==== 102409+0+0 (secure 0 0 0) 0x7f693c042070 con 0x7f69500af3d0 2024-09-16T15:46:53.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.828+0000 7f6946a00640 1 --2- 172.21.15.151:0/2132483396 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f69300794b0 0x7f693007b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:53.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.828+0000 7f6946a00640 1 -- 172.21.15.151:0/2132483396 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5765+0+0 (secure 0 0 0) 0x7f693c0ba3d0 con 0x7f69500af3d0 2024-09-16T15:46:53.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.828+0000 7f694ea00640 1 --2- 172.21.15.151:0/2132483396 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f69300794b0 0x7f693007b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:53.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.829+0000 7f694ea00640 1 --2- 172.21.15.151:0/2132483396 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f69300794b0 0x7f693007b970 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6940005fd0 tx=0x7f694000a250 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:53.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:53.832+0000 7f6946a00640 1 -- 172.21.15.151:0/2132483396 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f69500b3030 con 0x7f69500af3d0 2024-09-16T15:46:54.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:54.018+0000 7f6955a00640 1 -- 172.21.15.151:0/2132483396 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6950161c90 con 0x7f69500af3d0 2024-09-16T15:46:54.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:54.019+0000 7f6946a00640 1 -- 172.21.15.151:0/2132483396 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+16828 (secure 0 0 0) 0x7f693c083bc0 con 0x7f69500af3d0 2024-09-16T15:46:54.022 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:46:54.022 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":75,"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","created":"2024-09-16T15:38:56.348535+0000","modified":"2024-09-16T15:46:52.671263+0000","last_up_change":"2024-09-16T15:39:10.681384+0000","last_in_change":"2024-09-16T15:39:00.544477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:39:11.602252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:46:50.425692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-16T15:46:51.632208+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5524076-a903-4d0a-af6a-caf474c4765d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6809","nonce":3111342741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6811","nonce":3111342741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6815","nonce":3111342741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6813","nonce":3111342741}]},"public_addr":"172.21.15.151:6809/3111342741","cluster_addr":"172.21.15.151:6811/3111342741","heartbeat_back_addr":"172.21.15.151:6815/3111342741","heartbeat_front_addr":"172.21.15.151:6813/3111342741","state":["exists","up"]},{"osd":1,"uuid":"dd25925f-0ae1-494e-8529-4dc205759db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6801","nonce":3368304087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6803","nonce":3368304087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6807","nonce":3368304087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6805","nonce":3368304087}]},"public_addr":"172.21.15.151:6801/3368304087","cluster_addr":"172.21.15.151:6803/3368304087","heartbeat_back_addr":"172.21.15.151:6807/3368304087","heartbeat_front_addr":"172.21.15.151:6805/3368304087","state":["exists","up"]},{"osd":2,"uuid":"c2f18115-013d-4da6-a0fe-cb978f4494b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6823","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6825","nonce":3079373271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6827","nonce":3079373271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6831","nonce":3079373271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6829","nonce":3079373271}]},"public_addr":"172.21.15.151:6825/3079373271","cluster_addr":"172.21.15.151:6827/3079373271","heartbeat_back_addr":"172.21.15.151:6831/3079373271","heartbeat_front_addr":"172.21.15.151:6829/3079373271","state":["exists","up"]},{"osd":3,"uuid":"36edb718-2fcb-4d88-8ddc-c321eed4279d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6817","nonce":1142509041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6819","nonce":1142509041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6824","nonce":1142509041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6821","nonce":1142509041}]},"public_addr":"172.21.15.151:6817/1142509041","cluster_addr":"172.21.15.151:6819/1142509041","heartbeat_back_addr":"172.21.15.151:6824/1142509041","heartbeat_front_addr":"172.21.15.151:6821/1142509041","state":["exists","up"]},{"osd":4,"uuid":"64d33708-cd0a-4236-abe3-1b646e82ab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6811","nonce":2735495271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6813","nonce":2735495271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6817","nonce":2735495271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6815","nonce":2735495271}]},"public_addr":"172.21.15.167:6811/2735495271","cluster_addr":"172.21.15.167:6813/2735495271","heartbeat_back_addr":"172.21.15.167:6817/2735495271","heartbeat_front_addr":"172.21.15.167:6815/2735495271","state":["exists","up"]},{"osd":5,"uuid":"f14b44a9-5910-458c-b518-09a8d0de94de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6827","nonce":1494916572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6829","nonce":1494916572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6833","nonce":1494916572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6831","nonce":1494916572}]},"public_addr":"172.21.15.167:6827/1494916572","cluster_addr":"172.21.15.167:6829/1494916572","heartbeat_back_addr":"172.21.15.167:6833/1494916572","heartbeat_front_addr":"172.21.15.167:6831/1494916572","state":["exists","up"]},{"osd":6,"uuid":"39335545-2654-4710-84b0-a9aa47cfcbba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6819","nonce":2538273273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6821","nonce":2538273273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6825","nonce":2538273273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6823","nonce":2538273273}]},"public_addr":"172.21.15.167:6819/2538273273","cluster_addr":"172.21.15.167:6821/2538273273","heartbeat_back_addr":"172.21.15.167:6825/2538273273","heartbeat_front_addr":"172.21.15.167:6823/2538273273","state":["exists","up"]},{"osd":7,"uuid":"4f8bfc81-53dd-4339-ac22-1a4007d9f963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6803","nonce":265072227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6805","nonce":265072227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6809","nonce":265072227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6807","nonce":265072227}]},"public_addr":"172.21.15.167:6803/265072227","cluster_addr":"172.21.15.167:6805/265072227","heartbeat_back_addr":"172.21.15.167:6809/265072227","heartbeat_front_addr":"172.21.15.167:6807/265072227","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.150359+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.084358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.722368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.332809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.320709+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.395153+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.935250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.801443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:46:54.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:54.022+0000 7f6955a00640 1 -- 172.21.15.151:0/2132483396 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f69300794b0 msgr2=0x7f693007b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:54.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:54.022+0000 7f6955a00640 1 --2- 172.21.15.151:0/2132483396 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f69300794b0 0x7f693007b970 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6940005fd0 tx=0x7f694000a250 comp rx=0 tx=0).stop 2024-09-16T15:46:54.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:54.022+0000 7f6955a00640 1 -- 172.21.15.151:0/2132483396 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69500af3d0 msgr2=0x7f69501f8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:54.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:54.022+0000 7f6955a00640 1 --2- 172.21.15.151:0/2132483396 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69500af3d0 0x7f69501f8a90 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f693c000c00 tx=0x7f693c0039e0 comp rx=0 tx=0).stop 2024-09-16T15:46:54.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:54.022+0000 7f6955a00640 1 -- 172.21.15.151:0/2132483396 shutdown_connections 2024-09-16T15:46:54.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:54.022+0000 7f6955a00640 1 --2- 172.21.15.151:0/2132483396 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f69300794b0 0x7f693007b970 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:54.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:54.022+0000 7f6955a00640 1 --2- 172.21.15.151:0/2132483396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6950161550 0x7f695009c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:54.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:54.022+0000 7f6955a00640 1 --2- 172.21.15.151:0/2132483396 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69500af3d0 0x7f69501f8a90 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:54.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:54.022+0000 7f6955a00640 1 --2- 172.21.15.151:0/2132483396 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f695006f9d0 0x7f6950151090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:54.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:54.022+0000 7f6955a00640 1 -- 172.21.15.151:0/2132483396 >> 172.21.15.151:0/2132483396 conn(0x7f69500b1950 msgr2=0x7f695006ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:54.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:54.022+0000 7f6955a00640 1 -- 172.21.15.151:0/2132483396 shutdown_connections 2024-09-16T15:46:54.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:54.022+0000 7f6955a00640 1 -- 172.21.15.151:0/2132483396 wait complete. 2024-09-16T15:46:54.040 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-16T15:46:54.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:54.199+0000 7f8feca00640 1 -- 172.21.15.151:0/120137382 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fe815bec0 msgr2=0x7f8fe806fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:54.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:54.199+0000 7f8feca00640 1 --2- 172.21.15.151:0/120137382 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fe815bec0 0x7f8fe806fce0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f8fe80724f0 tx=0x7f8fd802a560 comp rx=0 tx=0).stop 2024-09-16T15:46:54.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:54.199+0000 7f8feca00640 1 -- 172.21.15.151:0/120137382 shutdown_connections 2024-09-16T15:46:54.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:54.199+0000 7f8feca00640 1 --2- 172.21.15.151:0/120137382 >> v2:172.21.15.167:3300/0 conn(0x7f8fe80cb4f0 0x7f8fe80cb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:54.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:54.199+0000 7f8feca00640 1 --2- 172.21.15.151:0/120137382 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8fe8072560 0x7f8fe8073ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:54.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:54.199+0000 7f8feca00640 1 --2- 172.21.15.151:0/120137382 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fe815bec0 0x7f8fe806fce0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:54.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:54.199+0000 7f8feca00640 1 -- 172.21.15.151:0/120137382 >> 172.21.15.151:0/120137382 conn(0x7f8fe80cf9f0 msgr2=0x7f8fe80cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:54.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:54.199+0000 7f8feca00640 1 -- 172.21.15.151:0/120137382 shutdown_connections 2024-09-16T15:46:54.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:54.199+0000 7f8feca00640 1 -- 172.21.15.151:0/120137382 wait complete. 2024-09-16T15:46:54.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:54.200+0000 7f8feca00640 1 Processor -- start 2024-09-16T15:46:54.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:54.201+0000 7f8feca00640 1 -- start start 2024-09-16T15:46:54.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:54.201+0000 7f8feca00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8fe8072560 0x7f8fe815ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:54.205 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:54.201+0000 7f8feca00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8fe80cb4f0 0x7f8fe815d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:54.205 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:54.202+0000 7f8feca00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fe815bec0 0x7f8fe814df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:54.205 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:54.202+0000 7f8feca00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe80ae850 con 0x7f8fe815bec0 2024-09-16T15:46:54.205 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:54.202+0000 7f8feca00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe80ae6d0 con 0x7f8fe8072560 2024-09-16T15:46:54.205 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:54.202+0000 7f8feca00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f8fe80ae9d0 con 0x7f8fe80cb4f0 2024-09-16T15:46:54.205 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:54.202+0000 7f8fe7e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fe815bec0 0x7f8fe814df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:54.205 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:54.202+0000 7f8fe7e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fe815bec0 0x7f8fe814df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:46060/0 (socket says 172.21.15.151:46060) 2024-09-16T15:46:54.205 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:54.202+0000 7f8fe7e00640 1 -- 172.21.15.151:0/3078052493 learned_addr learned my addr 172.21.15.151:0/3078052493 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:46:54.205 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:54.202+0000 7f8fe6a00640 1 --2- 172.21.15.151:0/3078052493 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8fe80cb4f0 0x7f8fe815d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:54.205 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:54.202+0000 7f8fe7400640 1 --2- 172.21.15.151:0/3078052493 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8fe8072560 0x7f8fe815ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:54.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:54.202+0000 7f8fe7e00640 1 -- 172.21.15.151:0/3078052493 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8fe80cb4f0 msgr2=0x7f8fe815d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:54.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:54.202+0000 7f8fe7e00640 1 --2- 172.21.15.151:0/3078052493 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8fe80cb4f0 0x7f8fe815d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:54.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:54.202+0000 7f8fe7e00640 1 -- 172.21.15.151:0/3078052493 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8fe8072560 msgr2=0x7f8fe815ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:54.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:54.202+0000 7f8fe7e00640 1 --2- 172.21.15.151:0/3078052493 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8fe8072560 0x7f8fe815ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:54.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:54.202+0000 7f8fe7e00640 1 -- 172.21.15.151:0/3078052493 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fe815db50 con 0x7f8fe815bec0 2024-09-16T15:46:54.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:54.203+0000 7f8fe7e00640 1 --2- 172.21.15.151:0/3078052493 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fe815bec0 0x7f8fe814df80 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f8fe000ead0 tx=0x7f8fe000ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:54.207 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:54.203+0000 7f8fdf400640 1 -- 172.21.15.151:0/3078052493 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe000cde0 con 0x7f8fe815bec0 2024-09-16T15:46:54.207 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:54.203+0000 7f8fdf400640 1 -- 172.21.15.151:0/3078052493 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8fe0004c10 con 0x7f8fe815bec0 2024-09-16T15:46:54.207 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:54.203+0000 7f8fdf400640 1 -- 172.21.15.151:0/3078052493 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe0005020 con 0x7f8fe815bec0 2024-09-16T15:46:54.207 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:54.203+0000 7f8feca00640 1 -- 172.21.15.151:0/3078052493 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fe815de30 con 0x7f8fe815bec0 2024-09-16T15:46:54.207 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:54.203+0000 7f8feca00640 1 -- 172.21.15.151:0/3078052493 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fe815e3f0 con 0x7f8fe815bec0 2024-09-16T15:46:54.207 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:54.204+0000 7f8feca00640 1 -- 172.21.15.151:0/3078052493 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fe80aee80 con 0x7f8fe815bec0 2024-09-16T15:46:54.207 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:54.205+0000 7f8fdf400640 1 -- 172.21.15.151:0/3078052493 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 24) ==== 102409+0+0 (secure 0 0 0) 0x7f8fe00051c0 con 0x7f8fe815bec0 2024-09-16T15:46:54.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:54.206+0000 7f8fdf400640 1 --2- 172.21.15.151:0/3078052493 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8fc80794b0 0x7f8fc807b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:54.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:54.206+0000 7f8fdf400640 1 -- 172.21.15.151:0/3078052493 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5765+0+0 (secure 0 0 0) 0x7f8fe0099c80 con 0x7f8fe815bec0 2024-09-16T15:46:54.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:54.206+0000 7f8fe7400640 1 --2- 172.21.15.151:0/3078052493 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8fc80794b0 0x7f8fc807b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:54.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:54.207+0000 7f8fe7400640 1 --2- 172.21.15.151:0/3078052493 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8fc80794b0 0x7f8fc807b970 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8033000 tx=0x7f8fd8004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:54.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:54.210+0000 7f8fdf400640 1 -- 172.21.15.151:0/3078052493 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8fe0016030 con 0x7f8fe815bec0 2024-09-16T15:46:54.438 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:54.436+0000 7f8feca00640 1 -- 172.21.15.151:0/3078052493 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f8fe80b1030 con 0x7f8fe815bec0 2024-09-16T15:46:55.213 INFO:tasks.ceph.mds.a.smithi151.stderr:2024-09-16T15:46:55.211+0000 7fb9a2000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-16T15:46:55.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:55.266+0000 7f8fdf400640 1 -- 172.21.15.151:0/3078052493 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v45) ==== 118+0+0 (secure 0 0 0) 0x7f8fe00636f0 con 0x7f8fe815bec0 2024-09-16T15:46:55.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:55.269+0000 7f8feca00640 1 -- 172.21.15.151:0/3078052493 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8fc80794b0 msgr2=0x7f8fc807b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:55.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:55.269+0000 7f8feca00640 1 --2- 172.21.15.151:0/3078052493 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8fc80794b0 0x7f8fc807b970 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8033000 tx=0x7f8fd8004730 comp rx=0 tx=0).stop 2024-09-16T15:46:55.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:55.269+0000 7f8feca00640 1 -- 172.21.15.151:0/3078052493 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fe815bec0 msgr2=0x7f8fe814df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:55.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:55.269+0000 7f8feca00640 1 --2- 172.21.15.151:0/3078052493 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fe815bec0 0x7f8fe814df80 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f8fe000ead0 tx=0x7f8fe000ef90 comp rx=0 tx=0).stop 2024-09-16T15:46:55.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:55.270+0000 7f8feca00640 1 -- 172.21.15.151:0/3078052493 shutdown_connections 2024-09-16T15:46:55.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:55.270+0000 7f8feca00640 1 --2- 172.21.15.151:0/3078052493 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8fc80794b0 0x7f8fc807b970 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:55.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:55.270+0000 7f8feca00640 1 --2- 172.21.15.151:0/3078052493 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fe815bec0 0x7f8fe814df80 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:55.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:55.270+0000 7f8feca00640 1 --2- 172.21.15.151:0/3078052493 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8fe80cb4f0 0x7f8fe815d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:55.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:55.270+0000 7f8feca00640 1 --2- 172.21.15.151:0/3078052493 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8fe8072560 0x7f8fe815ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:55.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:55.270+0000 7f8feca00640 1 -- 172.21.15.151:0/3078052493 >> 172.21.15.151:0/3078052493 conn(0x7f8fe80cf9f0 msgr2=0x7f8fe80b0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:55.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:55.270+0000 7f8feca00640 1 -- 172.21.15.151:0/3078052493 shutdown_connections 2024-09-16T15:46:55.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:55.270+0000 7f8feca00640 1 -- 172.21.15.151:0/3078052493 wait complete. 2024-09-16T15:46:55.291 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-16T15:46:55.291 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-16T15:46:55.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:55.455+0000 7f3352600640 1 -- 172.21.15.151:0/3797178949 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3330004800 msgr2=0x7f3330004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:55.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:55.455+0000 7f3352600640 1 --2- 172.21.15.151:0/3797178949 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3330004800 0x7f3330004c80 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f3338009b80 tx=0x7f3330005830 comp rx=0 tx=0).stop 2024-09-16T15:46:55.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:55.455+0000 7f334be00640 1 -- 172.21.15.151:0/3797178949 reap_dead start 2024-09-16T15:46:55.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:55.456+0000 7f3352600640 1 -- 172.21.15.151:0/3797178949 shutdown_connections 2024-09-16T15:46:55.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:55.456+0000 7f3352600640 1 -- 172.21.15.151:0/3797178949 >> 172.21.15.151:0/3797178949 conn(0x7f334c0b1840 msgr2=0x7f334c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:55.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:55.456+0000 7f3352600640 1 -- 172.21.15.151:0/3797178949 shutdown_connections 2024-09-16T15:46:55.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:55.456+0000 7f3352600640 1 -- 172.21.15.151:0/3797178949 wait complete. 2024-09-16T15:46:55.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:55.457+0000 7f3352600640 1 Processor -- start 2024-09-16T15:46:55.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:55.457+0000 7f3352600640 1 -- start start 2024-09-16T15:46:55.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:55.458+0000 7f3352600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3330004800 0x7f334c14f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:55.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:55.458+0000 7f3352600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f334c15f770 0x7f334c15fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:55.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:55.458+0000 7f3352600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f334c15dc20 0x7f334c15e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:55.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:55.458+0000 7f3352600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f334c0cf420 con 0x7f334c15f770 2024-09-16T15:46:55.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:55.458+0000 7f3352600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f334c0cf2a0 con 0x7f334c15dc20 2024-09-16T15:46:55.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:55.458+0000 7f3352600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f334c0cf5a0 con 0x7f3330004800 2024-09-16T15:46:55.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:55.459+0000 7f334aa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f334c15f770 0x7f334c15fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:55.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:55.459+0000 7f334aa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f334c15f770 0x7f334c15fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:46078/0 (socket says 172.21.15.151:46078) 2024-09-16T15:46:55.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:55.459+0000 7f334aa00640 1 -- 172.21.15.151:0/152821583 learned_addr learned my addr 172.21.15.151:0/152821583 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:46:55.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:55.459+0000 7f334be00640 1 --2- 172.21.15.151:0/152821583 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f334c15dc20 0x7f334c15e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:55.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:55.459+0000 7f334b400640 1 --2- 172.21.15.151:0/152821583 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3330004800 0x7f334c14f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:55.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:55.459+0000 7f334aa00640 1 -- 172.21.15.151:0/152821583 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3330004800 msgr2=0x7f334c14f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:55.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:55.459+0000 7f334aa00640 1 --2- 172.21.15.151:0/152821583 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3330004800 0x7f334c14f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:55.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:55.459+0000 7f334aa00640 1 -- 172.21.15.151:0/152821583 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f334c15dc20 msgr2=0x7f334c15e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:55.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:55.459+0000 7f334aa00640 1 --2- 172.21.15.151:0/152821583 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f334c15dc20 0x7f334c15e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:55.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:55.459+0000 7f334aa00640 1 -- 172.21.15.151:0/152821583 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f334c15e930 con 0x7f334c15f770 2024-09-16T15:46:55.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:55.459+0000 7f334aa00640 1 --2- 172.21.15.151:0/152821583 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f334c15f770 0x7f334c15fbf0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f333802b540 tx=0x7f3338033b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:55.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:55.460+0000 7f3343400640 1 -- 172.21.15.151:0/152821583 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3338038040 con 0x7f334c15f770 2024-09-16T15:46:55.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:55.460+0000 7f3343400640 1 -- 172.21.15.151:0/152821583 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f333802b850 con 0x7f334c15f770 2024-09-16T15:46:55.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:55.460+0000 7f3352600640 1 -- 172.21.15.151:0/152821583 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f334c152340 con 0x7f334c15f770 2024-09-16T15:46:55.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:55.460+0000 7f3343400640 1 -- 172.21.15.151:0/152821583 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3338004020 con 0x7f334c15f770 2024-09-16T15:46:55.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:55.460+0000 7f3352600640 1 -- 172.21.15.151:0/152821583 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f334c14fbc0 con 0x7f334c15f770 2024-09-16T15:46:55.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:55.461+0000 7f3352600640 1 -- 172.21.15.151:0/152821583 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f334c0b2b40 con 0x7f334c15f770 2024-09-16T15:46:55.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:55.462+0000 7f3343400640 1 -- 172.21.15.151:0/152821583 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 24) ==== 102409+0+0 (secure 0 0 0) 0x7f3338045020 con 0x7f334c15f770 2024-09-16T15:46:55.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:55.463+0000 7f3343400640 1 --2- 172.21.15.151:0/152821583 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f332c0794e0 0x7f332c07b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:55.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:55.463+0000 7f3343400640 1 -- 172.21.15.151:0/152821583 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5765+0+0 (secure 0 0 0) 0x7f33380b9e60 con 0x7f334c15f770 2024-09-16T15:46:55.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:55.463+0000 7f334b400640 1 --2- 172.21.15.151:0/152821583 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f332c0794e0 0x7f332c07b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:55.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:55.464+0000 7f334b400640 1 --2- 172.21.15.151:0/152821583 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f332c0794e0 0x7f332c07b9a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f333c006c70 tx=0x7f333c0067a0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:55.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:55.467+0000 7f3343400640 1 -- 172.21.15.151:0/152821583 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f333802ba20 con 0x7f334c15f770 2024-09-16T15:46:55.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:55.690+0000 7f3352600640 1 -- 172.21.15.151:0/152821583 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f334c0ae3f0 con 0x7f334c15f770 2024-09-16T15:46:56.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.273+0000 7f3343400640 1 -- 172.21.15.151:0/152821583 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v46) ==== 122+0+0 (secure 0 0 0) 0x7f3338083f20 con 0x7f334c15f770 2024-09-16T15:46:56.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.276+0000 7f3352600640 1 -- 172.21.15.151:0/152821583 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f332c0794e0 msgr2=0x7f332c07b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:56.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.276+0000 7f3352600640 1 --2- 172.21.15.151:0/152821583 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f332c0794e0 0x7f332c07b9a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f333c006c70 tx=0x7f333c0067a0 comp rx=0 tx=0).stop 2024-09-16T15:46:56.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.276+0000 7f3352600640 1 -- 172.21.15.151:0/152821583 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f334c15f770 msgr2=0x7f334c15fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:56.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.276+0000 7f3352600640 1 --2- 172.21.15.151:0/152821583 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f334c15f770 0x7f334c15fbf0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f333802b540 tx=0x7f3338033b30 comp rx=0 tx=0).stop 2024-09-16T15:46:56.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.276+0000 7f3352600640 1 -- 172.21.15.151:0/152821583 shutdown_connections 2024-09-16T15:46:56.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.276+0000 7f3352600640 1 --2- 172.21.15.151:0/152821583 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f332c0794e0 0x7f332c07b9a0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:56.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.276+0000 7f3352600640 1 --2- 172.21.15.151:0/152821583 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f334c15dc20 0x7f334c15e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:56.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.276+0000 7f3352600640 1 --2- 172.21.15.151:0/152821583 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f334c15f770 0x7f334c15fbf0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:56.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.276+0000 7f3352600640 1 --2- 172.21.15.151:0/152821583 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3330004800 0x7f334c14f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:56.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.276+0000 7f3352600640 1 -- 172.21.15.151:0/152821583 >> 172.21.15.151:0/152821583 conn(0x7f334c0b1840 msgr2=0x7f334c0ce340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:56.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.277+0000 7f3352600640 1 -- 172.21.15.151:0/152821583 shutdown_connections 2024-09-16T15:46:56.280 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.277+0000 7f3352600640 1 -- 172.21.15.151:0/152821583 wait complete. 2024-09-16T15:46:56.296 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-16T15:46:56.296 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:46:56.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.454+0000 7f655fe00640 1 -- 172.21.15.151:0/2771868664 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6544004800 msgr2=0x7f6544004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:56.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.454+0000 7f655fe00640 1 --2- 172.21.15.151:0/2771868664 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6544004800 0x7f6544004c80 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f6548006f60 tx=0x7f6544005830 comp rx=0 tx=0).stop 2024-09-16T15:46:56.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.455+0000 7f655fe00640 1 -- 172.21.15.151:0/2771868664 shutdown_connections 2024-09-16T15:46:56.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.455+0000 7f655fe00640 1 --2- 172.21.15.151:0/2771868664 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6544004800 0x7f6544004c80 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:56.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.455+0000 7f655fe00640 1 -- 172.21.15.151:0/2771868664 >> 172.21.15.151:0/2771868664 conn(0x7f65600cfa20 msgr2=0x7f65600cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:56.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.455+0000 7f655fe00640 1 -- 172.21.15.151:0/2771868664 shutdown_connections 2024-09-16T15:46:56.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.455+0000 7f655fe00640 1 -- 172.21.15.151:0/2771868664 wait complete. 2024-09-16T15:46:56.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.456+0000 7f655fe00640 1 Processor -- start 2024-09-16T15:46:56.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.456+0000 7f655fe00640 1 -- start start 2024-09-16T15:46:56.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.457+0000 7f655fe00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6544004800 0x7f65600c5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:56.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.457+0000 7f655fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f65601f7c70 0x7f65601f4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:56.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.457+0000 7f655fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f656015c2e0 0x7f656015c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:56.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.457+0000 7f655fe00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f65600b2c80 con 0x7f65601f7c70 2024-09-16T15:46:56.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.457+0000 7f655fe00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f65600b2b00 con 0x7f656015c2e0 2024-09-16T15:46:56.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.457+0000 7f655fe00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f6560073d40 con 0x7f6544004800 2024-09-16T15:46:56.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.458+0000 7f6556000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f65601f7c70 0x7f65601f4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:56.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.458+0000 7f6556000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f65601f7c70 0x7f65601f4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:46088/0 (socket says 172.21.15.151:46088) 2024-09-16T15:46:56.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.458+0000 7f6556000640 1 -- 172.21.15.151:0/785037972 learned_addr learned my addr 172.21.15.151:0/785037972 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:46:56.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.458+0000 7f655ea00640 1 --2- 172.21.15.151:0/785037972 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6544004800 0x7f65600c5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:56.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.458+0000 7f655f400640 1 --2- 172.21.15.151:0/785037972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f656015c2e0 0x7f656015c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:56.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.458+0000 7f6556000640 1 -- 172.21.15.151:0/785037972 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6544004800 msgr2=0x7f65600c5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:56.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.458+0000 7f6556000640 1 --2- 172.21.15.151:0/785037972 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6544004800 0x7f65600c5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:56.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.458+0000 7f6556000640 1 -- 172.21.15.151:0/785037972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f656015c2e0 msgr2=0x7f656015c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:56.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.458+0000 7f6556000640 1 --2- 172.21.15.151:0/785037972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f656015c2e0 0x7f656015c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:56.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.458+0000 7f6556000640 1 -- 172.21.15.151:0/785037972 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65600c5950 con 0x7f65601f7c70 2024-09-16T15:46:56.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.459+0000 7f6556000640 1 --2- 172.21.15.151:0/785037972 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f65601f7c70 0x7f65601f4680 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f654800c350 tx=0x7f6548001d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:56.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.459+0000 7f6557400640 1 -- 172.21.15.151:0/785037972 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6548032030 con 0x7f65601f7c70 2024-09-16T15:46:56.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.459+0000 7f6557400640 1 -- 172.21.15.151:0/785037972 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6548003b10 con 0x7f65601f7c70 2024-09-16T15:46:56.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.459+0000 7f6557400640 1 -- 172.21.15.151:0/785037972 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6548042430 con 0x7f65601f7c70 2024-09-16T15:46:56.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.459+0000 7f655fe00640 1 -- 172.21.15.151:0/785037972 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65601f4f20 con 0x7f65601f7c70 2024-09-16T15:46:56.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.459+0000 7f655fe00640 1 -- 172.21.15.151:0/785037972 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65601f54e0 con 0x7f65601f7c70 2024-09-16T15:46:56.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.460+0000 7f655fe00640 1 -- 172.21.15.151:0/785037972 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f656006a180 con 0x7f65601f7c70 2024-09-16T15:46:56.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.461+0000 7f6557400640 1 -- 172.21.15.151:0/785037972 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 24) ==== 102409+0+0 (secure 0 0 0) 0x7f6548035020 con 0x7f65601f7c70 2024-09-16T15:46:56.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.462+0000 7f6557400640 1 --2- 172.21.15.151:0/785037972 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f653c0794b0 0x7f653c07b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:56.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.462+0000 7f6557400640 1 -- 172.21.15.151:0/785037972 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5765+0+0 (secure 0 0 0) 0x7f65480b9440 con 0x7f65601f7c70 2024-09-16T15:46:56.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.463+0000 7f655ea00640 1 --2- 172.21.15.151:0/785037972 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f653c0794b0 0x7f653c07b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:56.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.464+0000 7f655ea00640 1 --2- 172.21.15.151:0/785037972 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f653c0794b0 0x7f653c07b970 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f65600c5d80 tx=0x7f6550004980 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:56.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.467+0000 7f6557400640 1 -- 172.21.15.151:0/785037972 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f656006a180 con 0x7f65601f7c70 2024-09-16T15:46:56.690 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.688+0000 7f655fe00640 1 -- 172.21.15.151:0/785037972 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f65600693c0 con 0x7f65601f7c70 2024-09-16T15:46:56.690 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.688+0000 7f6557400640 1 -- 172.21.15.151:0/785037972 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+5234 (secure 0 0 0) 0x7f65600693c0 con 0x7f65601f7c70 2024-09-16T15:46:56.690 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:46:56.691 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":46,"btime":"2024-09-16T15:46:56:268896+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5705,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3291846596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3291846596},{"type":"v1","addr":"172.21.15.167:6837","nonce":3291846596}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6175,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/2504229431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":2504229431},{"type":"v1","addr":"172.21.15.151:6834","nonce":2504229431}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6181,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/464376104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":464376104},{"type":"v1","addr":"172.21.15.167:6836","nonce":464376104}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:46:52.671228+0000","modified":"2024-09-16T15:46:55.692251+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6456},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6456":{"gid":6456,"name":"a","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.151:6835/3158835036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":3158835036},{"type":"v1","addr":"172.21.15.151:6835","nonce":3158835036}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6456,"qdb_cluster":[6456]},"id":5}]} 2024-09-16T15:46:56.691 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 46 2024-09-16T15:46:56.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.690+0000 7f655fe00640 1 -- 172.21.15.151:0/785037972 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f653c0794b0 msgr2=0x7f653c07b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:56.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.690+0000 7f655fe00640 1 --2- 172.21.15.151:0/785037972 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f653c0794b0 0x7f653c07b970 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f65600c5d80 tx=0x7f6550004980 comp rx=0 tx=0).stop 2024-09-16T15:46:56.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.690+0000 7f655fe00640 1 -- 172.21.15.151:0/785037972 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f65601f7c70 msgr2=0x7f65601f4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:56.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.690+0000 7f655fe00640 1 --2- 172.21.15.151:0/785037972 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f65601f7c70 0x7f65601f4680 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f654800c350 tx=0x7f6548001d40 comp rx=0 tx=0).stop 2024-09-16T15:46:56.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.691+0000 7f655fe00640 1 -- 172.21.15.151:0/785037972 shutdown_connections 2024-09-16T15:46:56.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.691+0000 7f655fe00640 1 --2- 172.21.15.151:0/785037972 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f653c0794b0 0x7f653c07b970 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:56.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.691+0000 7f655fe00640 1 --2- 172.21.15.151:0/785037972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f656015c2e0 0x7f656015c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:56.694 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.691+0000 7f655fe00640 1 --2- 172.21.15.151:0/785037972 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f65601f7c70 0x7f65601f4680 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:56.694 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.691+0000 7f655fe00640 1 --2- 172.21.15.151:0/785037972 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6544004800 0x7f65600c5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:56.694 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.691+0000 7f655fe00640 1 -- 172.21.15.151:0/785037972 >> 172.21.15.151:0/785037972 conn(0x7f65600cfa20 msgr2=0x7f6560158770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:56.694 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.691+0000 7f655fe00640 1 -- 172.21.15.151:0/785037972 shutdown_connections 2024-09-16T15:46:56.694 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.691+0000 7f655fe00640 1 -- 172.21.15.151:0/785037972 wait complete. 2024-09-16T15:46:56.709 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:46:56.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.867+0000 7ff80c000640 1 -- 172.21.15.151:0/3020187758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff7ec004800 msgr2=0x7ff7ec004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:56.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.867+0000 7ff80c000640 1 --2- 172.21.15.151:0/3020187758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff7ec004800 0x7ff7ec004c80 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7ff7f000c310 tx=0x7ff7ec005830 comp rx=0 tx=0).stop 2024-09-16T15:46:56.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.867+0000 7ff80a000640 1 -- 172.21.15.151:0/3020187758 reap_dead start 2024-09-16T15:46:56.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.867+0000 7ff80c000640 1 -- 172.21.15.151:0/3020187758 shutdown_connections 2024-09-16T15:46:56.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.867+0000 7ff80c000640 1 -- 172.21.15.151:0/3020187758 >> 172.21.15.151:0/3020187758 conn(0x7ff8040b2230 msgr2=0x7ff8040b2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:56.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.867+0000 7ff80c000640 1 -- 172.21.15.151:0/3020187758 shutdown_connections 2024-09-16T15:46:56.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.867+0000 7ff80c000640 1 -- 172.21.15.151:0/3020187758 wait complete. 2024-09-16T15:46:56.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.869+0000 7ff80c000640 1 Processor -- start 2024-09-16T15:46:56.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.869+0000 7ff80c000640 1 -- start start 2024-09-16T15:46:56.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.870+0000 7ff80c000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7ec004800 0x7ff80413a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:56.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.870+0000 7ff80c000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff8041f7f40 0x7ff80415d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:56.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.870+0000 7ff80c000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff80413ade0 0x7ff80415dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:56.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.870+0000 7ff80c000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff80413f490 con 0x7ff8041f7f40 2024-09-16T15:46:56.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.870+0000 7ff80c000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff804068c10 con 0x7ff7ec004800 2024-09-16T15:46:56.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.870+0000 7ff80c000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7ff8040cf670 con 0x7ff80413ade0 2024-09-16T15:46:56.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.870+0000 7ff808c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff8041f7f40 0x7ff80415d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:56.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.870+0000 7ff808c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff8041f7f40 0x7ff80415d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:46112/0 (socket says 172.21.15.151:46112) 2024-09-16T15:46:56.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.870+0000 7ff808c00640 1 -- 172.21.15.151:0/2994903311 learned_addr learned my addr 172.21.15.151:0/2994903311 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:46:56.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.870+0000 7ff809600640 1 --2- 172.21.15.151:0/2994903311 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7ec004800 0x7ff80413a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:56.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.870+0000 7ff80a000640 1 --2- 172.21.15.151:0/2994903311 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff80413ade0 0x7ff80415dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:56.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.871+0000 7ff808c00640 1 -- 172.21.15.151:0/2994903311 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff80413ade0 msgr2=0x7ff80415dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:56.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.871+0000 7ff808c00640 1 --2- 172.21.15.151:0/2994903311 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff80413ade0 0x7ff80415dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:56.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.871+0000 7ff808c00640 1 -- 172.21.15.151:0/2994903311 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7ec004800 msgr2=0x7ff80413a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:56.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.871+0000 7ff808c00640 1 --2- 172.21.15.151:0/2994903311 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7ec004800 0x7ff80413a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:56.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.871+0000 7ff808c00640 1 -- 172.21.15.151:0/2994903311 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8041f8490 con 0x7ff8041f7f40 2024-09-16T15:46:56.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.871+0000 7ff808c00640 1 --2- 172.21.15.151:0/2994903311 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff8041f7f40 0x7ff80415d6d0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7ff7f002b160 tx=0x7ff7f0006d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:56.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.871+0000 7ff801600640 1 -- 172.21.15.151:0/2994903311 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f0031040 con 0x7ff8041f7f40 2024-09-16T15:46:56.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.871+0000 7ff801600640 1 -- 172.21.15.151:0/2994903311 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff7f00035f0 con 0x7ff8041f7f40 2024-09-16T15:46:56.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.871+0000 7ff801600640 1 -- 172.21.15.151:0/2994903311 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f0003a40 con 0x7ff8041f7f40 2024-09-16T15:46:56.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.871+0000 7ff80c000640 1 -- 172.21.15.151:0/2994903311 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8041f8770 con 0x7ff8041f7f40 2024-09-16T15:46:56.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.872+0000 7ff80c000640 1 -- 172.21.15.151:0/2994903311 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8041f8d30 con 0x7ff8041f7f40 2024-09-16T15:46:56.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.872+0000 7ff80c000640 1 -- 172.21.15.151:0/2994903311 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8040cef60 con 0x7ff8041f7f40 2024-09-16T15:46:56.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.874+0000 7ff801600640 1 -- 172.21.15.151:0/2994903311 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 24) ==== 102409+0+0 (secure 0 0 0) 0x7ff7f0034070 con 0x7ff8041f7f40 2024-09-16T15:46:56.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.874+0000 7ff801600640 1 --2- 172.21.15.151:0/2994903311 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff7e40794b0 0x7ff7e407b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:56.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.875+0000 7ff801600640 1 -- 172.21.15.151:0/2994903311 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5765+0+0 (secure 0 0 0) 0x7ff7f00b9640 con 0x7ff8041f7f40 2024-09-16T15:46:56.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.875+0000 7ff809600640 1 --2- 172.21.15.151:0/2994903311 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff7e40794b0 0x7ff7e407b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:56.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.876+0000 7ff809600640 1 --2- 172.21.15.151:0/2994903311 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff7e40794b0 0x7ff7e407b970 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8000c00 tx=0x7ff7f8006b70 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:56.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:56.879+0000 7ff801600640 1 -- 172.21.15.151:0/2994903311 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff7f0082e30 con 0x7ff8041f7f40 2024-09-16T15:46:57.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.064+0000 7ff80c000640 1 -- 172.21.15.151:0/2994903311 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff80415fdc0 con 0x7ff8041f7f40 2024-09-16T15:46:57.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.065+0000 7ff801600640 1 -- 172.21.15.151:0/2994903311 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+16815 (secure 0 0 0) 0x7ff7f0087860 con 0x7ff8041f7f40 2024-09-16T15:46:57.068 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:46:57.068 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":76,"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","created":"2024-09-16T15:38:56.348535+0000","modified":"2024-09-16T15:46:54.252016+0000","last_up_change":"2024-09-16T15:39:10.681384+0000","last_in_change":"2024-09-16T15:39:00.544477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:39:11.602252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:46:50.425692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-16T15:46:51.632208+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5524076-a903-4d0a-af6a-caf474c4765d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6809","nonce":3111342741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6811","nonce":3111342741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6815","nonce":3111342741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6813","nonce":3111342741}]},"public_addr":"172.21.15.151:6809/3111342741","cluster_addr":"172.21.15.151:6811/3111342741","heartbeat_back_addr":"172.21.15.151:6815/3111342741","heartbeat_front_addr":"172.21.15.151:6813/3111342741","state":["exists","up"]},{"osd":1,"uuid":"dd25925f-0ae1-494e-8529-4dc205759db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6801","nonce":3368304087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6803","nonce":3368304087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6807","nonce":3368304087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6805","nonce":3368304087}]},"public_addr":"172.21.15.151:6801/3368304087","cluster_addr":"172.21.15.151:6803/3368304087","heartbeat_back_addr":"172.21.15.151:6807/3368304087","heartbeat_front_addr":"172.21.15.151:6805/3368304087","state":["exists","up"]},{"osd":2,"uuid":"c2f18115-013d-4da6-a0fe-cb978f4494b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6823","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6825","nonce":3079373271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6827","nonce":3079373271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6831","nonce":3079373271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6829","nonce":3079373271}]},"public_addr":"172.21.15.151:6825/3079373271","cluster_addr":"172.21.15.151:6827/3079373271","heartbeat_back_addr":"172.21.15.151:6831/3079373271","heartbeat_front_addr":"172.21.15.151:6829/3079373271","state":["exists","up"]},{"osd":3,"uuid":"36edb718-2fcb-4d88-8ddc-c321eed4279d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6817","nonce":1142509041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6819","nonce":1142509041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6824","nonce":1142509041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6821","nonce":1142509041}]},"public_addr":"172.21.15.151:6817/1142509041","cluster_addr":"172.21.15.151:6819/1142509041","heartbeat_back_addr":"172.21.15.151:6824/1142509041","heartbeat_front_addr":"172.21.15.151:6821/1142509041","state":["exists","up"]},{"osd":4,"uuid":"64d33708-cd0a-4236-abe3-1b646e82ab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6811","nonce":2735495271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6813","nonce":2735495271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6817","nonce":2735495271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6815","nonce":2735495271}]},"public_addr":"172.21.15.167:6811/2735495271","cluster_addr":"172.21.15.167:6813/2735495271","heartbeat_back_addr":"172.21.15.167:6817/2735495271","heartbeat_front_addr":"172.21.15.167:6815/2735495271","state":["exists","up"]},{"osd":5,"uuid":"f14b44a9-5910-458c-b518-09a8d0de94de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6827","nonce":1494916572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6829","nonce":1494916572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6833","nonce":1494916572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6831","nonce":1494916572}]},"public_addr":"172.21.15.167:6827/1494916572","cluster_addr":"172.21.15.167:6829/1494916572","heartbeat_back_addr":"172.21.15.167:6833/1494916572","heartbeat_front_addr":"172.21.15.167:6831/1494916572","state":["exists","up"]},{"osd":6,"uuid":"39335545-2654-4710-84b0-a9aa47cfcbba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6819","nonce":2538273273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6821","nonce":2538273273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6825","nonce":2538273273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6823","nonce":2538273273}]},"public_addr":"172.21.15.167:6819/2538273273","cluster_addr":"172.21.15.167:6821/2538273273","heartbeat_back_addr":"172.21.15.167:6825/2538273273","heartbeat_front_addr":"172.21.15.167:6823/2538273273","state":["exists","up"]},{"osd":7,"uuid":"4f8bfc81-53dd-4339-ac22-1a4007d9f963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6803","nonce":265072227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6805","nonce":265072227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6809","nonce":265072227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6807","nonce":265072227}]},"public_addr":"172.21.15.167:6803/265072227","cluster_addr":"172.21.15.167:6805/265072227","heartbeat_back_addr":"172.21.15.167:6809/265072227","heartbeat_front_addr":"172.21.15.167:6807/265072227","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.150359+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.084358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.722368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.332809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.320709+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.395153+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.935250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.801443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:46:57.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.068+0000 7ff80c000640 1 -- 172.21.15.151:0/2994903311 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff7e40794b0 msgr2=0x7ff7e407b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:57.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.068+0000 7ff80c000640 1 --2- 172.21.15.151:0/2994903311 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff7e40794b0 0x7ff7e407b970 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8000c00 tx=0x7ff7f8006b70 comp rx=0 tx=0).stop 2024-09-16T15:46:57.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.068+0000 7ff80c000640 1 -- 172.21.15.151:0/2994903311 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff8041f7f40 msgr2=0x7ff80415d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:57.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.068+0000 7ff80c000640 1 --2- 172.21.15.151:0/2994903311 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff8041f7f40 0x7ff80415d6d0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7ff7f002b160 tx=0x7ff7f0006d20 comp rx=0 tx=0).stop 2024-09-16T15:46:57.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.068+0000 7ff80c000640 1 -- 172.21.15.151:0/2994903311 shutdown_connections 2024-09-16T15:46:57.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.068+0000 7ff80c000640 1 --2- 172.21.15.151:0/2994903311 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff7e40794b0 0x7ff7e407b970 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:57.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.068+0000 7ff80c000640 1 --2- 172.21.15.151:0/2994903311 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff80413ade0 0x7ff80415dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:57.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.068+0000 7ff80c000640 1 --2- 172.21.15.151:0/2994903311 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff8041f7f40 0x7ff80415d6d0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:57.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.068+0000 7ff80c000640 1 --2- 172.21.15.151:0/2994903311 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7ec004800 0x7ff80413a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:57.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.068+0000 7ff80c000640 1 -- 172.21.15.151:0/2994903311 >> 172.21.15.151:0/2994903311 conn(0x7ff8040b2230 msgr2=0x7ff8040b2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:57.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.068+0000 7ff80c000640 1 -- 172.21.15.151:0/2994903311 shutdown_connections 2024-09-16T15:46:57.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.068+0000 7ff80c000640 1 -- 172.21.15.151:0/2994903311 wait complete. 2024-09-16T15:46:57.086 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T15:46:57.086 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:46:57.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.242+0000 7f6fbb200640 1 -- 172.21.15.151:0/2334298534 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6fbc161440 msgr2=0x7f6fbc0b5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:57.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.242+0000 7f6fbb200640 1 --2- 172.21.15.151:0/2334298534 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6fbc161440 0x7f6fbc0b5370 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc0d4ea0 tx=0x7f6fac02a540 comp rx=0 tx=0).stop 2024-09-16T15:46:57.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.243+0000 7f6fbb200640 1 -- 172.21.15.151:0/2334298534 shutdown_connections 2024-09-16T15:46:57.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.243+0000 7f6fbb200640 1 --2- 172.21.15.151:0/2334298534 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6fbc161440 0x7f6fbc0b5370 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:57.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.243+0000 7f6fbb200640 1 -- 172.21.15.151:0/2334298534 >> 172.21.15.151:0/2334298534 conn(0x7f6fbc0d3df0 msgr2=0x7f6fbc0d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:57.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.243+0000 7f6fbb200640 1 -- 172.21.15.151:0/2334298534 shutdown_connections 2024-09-16T15:46:57.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.243+0000 7f6fbb200640 1 -- 172.21.15.151:0/2334298534 wait complete. 2024-09-16T15:46:57.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.244+0000 7f6fbb200640 1 Processor -- start 2024-09-16T15:46:57.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.244+0000 7f6fbb200640 1 -- start start 2024-09-16T15:46:57.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.245+0000 7f6fbb200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6fbc0b34b0 0x7f6fbc1532f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:57.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.245+0000 7f6fbb200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6fbc0b4050 0x7f6fbc0c7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:57.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.245+0000 7f6fbb200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6fbc161440 0x7f6fbc0c7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:57.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.245+0000 7f6fbb200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6fbc0b2210 con 0x7f6fbc0b4050 2024-09-16T15:46:57.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.245+0000 7f6fbb200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6fbc0b2090 con 0x7f6fbc0b34b0 2024-09-16T15:46:57.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.245+0000 7f6fbb200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f6fbc0b2390 con 0x7f6fbc161440 2024-09-16T15:46:57.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.245+0000 7f6fb3e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6fbc0b4050 0x7f6fbc0c7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:57.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.245+0000 7f6fb3e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6fbc0b4050 0x7f6fbc0c7920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:46120/0 (socket says 172.21.15.151:46120) 2024-09-16T15:46:57.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.246+0000 7f6fb3e00640 1 -- 172.21.15.151:0/1225794803 learned_addr learned my addr 172.21.15.151:0/1225794803 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:46:57.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.246+0000 7f6fb9e00640 1 --2- 172.21.15.151:0/1225794803 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6fbc0b34b0 0x7f6fbc1532f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:57.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.246+0000 7f6fba800640 1 --2- 172.21.15.151:0/1225794803 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6fbc161440 0x7f6fbc0c7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:57.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.246+0000 7f6fb3e00640 1 -- 172.21.15.151:0/1225794803 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6fbc161440 msgr2=0x7f6fbc0c7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:57.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.246+0000 7f6fb3e00640 1 --2- 172.21.15.151:0/1225794803 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6fbc161440 0x7f6fbc0c7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:57.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.246+0000 7f6fb3e00640 1 -- 172.21.15.151:0/1225794803 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6fbc0b34b0 msgr2=0x7f6fbc1532f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:57.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.246+0000 7f6fb3e00640 1 --2- 172.21.15.151:0/1225794803 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6fbc0b34b0 0x7f6fbc1532f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:57.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.246+0000 7f6fb3e00640 1 -- 172.21.15.151:0/1225794803 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fbc153830 con 0x7f6fbc0b4050 2024-09-16T15:46:57.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.246+0000 7f6fb3e00640 1 --2- 172.21.15.151:0/1225794803 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6fbc0b4050 0x7f6fbc0c7920 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f6fa400c480 tx=0x7f6fa400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:57.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.246+0000 7f6fb2000640 1 -- 172.21.15.151:0/1225794803 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa40052d0 con 0x7f6fbc0b4050 2024-09-16T15:46:57.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.246+0000 7f6fb2000640 1 -- 172.21.15.151:0/1225794803 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6fa4005470 con 0x7f6fbc0b4050 2024-09-16T15:46:57.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.246+0000 7f6fb2000640 1 -- 172.21.15.151:0/1225794803 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa4012870 con 0x7f6fbc0b4050 2024-09-16T15:46:57.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.246+0000 7f6fbb200640 1 -- 172.21.15.151:0/1225794803 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fbc0c9b80 con 0x7f6fbc0b4050 2024-09-16T15:46:57.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.247+0000 7f6fbb200640 1 -- 172.21.15.151:0/1225794803 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fbc0b6e30 con 0x7f6fbc0b4050 2024-09-16T15:46:57.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.248+0000 7f6fbb200640 1 -- 172.21.15.151:0/1225794803 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fbc0b2830 con 0x7f6fbc0b4050 2024-09-16T15:46:57.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.249+0000 7f6fb2000640 1 -- 172.21.15.151:0/1225794803 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 24) ==== 102409+0+0 (secure 0 0 0) 0x7f6fa40035f0 con 0x7f6fbc0b4050 2024-09-16T15:46:57.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.250+0000 7f6fb2000640 1 --2- 172.21.15.151:0/1225794803 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6f980794e0 0x7f6f9807b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:57.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.250+0000 7f6fb2000640 1 -- 172.21.15.151:0/1225794803 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5765+0+0 (secure 0 0 0) 0x7f6fa4098db0 con 0x7f6fbc0b4050 2024-09-16T15:46:57.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.250+0000 7f6fb9e00640 1 --2- 172.21.15.151:0/1225794803 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6f980794e0 0x7f6f9807b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:57.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.251+0000 7f6fb9e00640 1 --2- 172.21.15.151:0/1225794803 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6f980794e0 0x7f6f9807b9a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6fac033000 tx=0x7f6fac004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:57.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.254+0000 7f6fb2000640 1 -- 172.21.15.151:0/1225794803 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6fbc153830 con 0x7f6fbc0b4050 2024-09-16T15:46:57.441 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.438+0000 7f6fbb200640 1 -- 172.21.15.151:0/1225794803 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6fbc0b6c80 con 0x7f6f980794e0 2024-09-16T15:46:57.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.444+0000 7f6fb2000640 1 -- 172.21.15.151:0/1225794803 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+238763 (secure 0 0 0) 0x7f6fbc0b6c80 con 0x7f6f980794e0 2024-09-16T15:46:57.449 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:46:57.450 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-09-16T15:46:57.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.448+0000 7f6fbb200640 1 -- 172.21.15.151:0/1225794803 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6f980794e0 msgr2=0x7f6f9807b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:57.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.448+0000 7f6fbb200640 1 --2- 172.21.15.151:0/1225794803 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6f980794e0 0x7f6f9807b9a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6fac033000 tx=0x7f6fac004730 comp rx=0 tx=0).stop 2024-09-16T15:46:57.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.448+0000 7f6fbb200640 1 -- 172.21.15.151:0/1225794803 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6fbc0b4050 msgr2=0x7f6fbc0c7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:57.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.448+0000 7f6fbb200640 1 --2- 172.21.15.151:0/1225794803 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6fbc0b4050 0x7f6fbc0c7920 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f6fa400c480 tx=0x7f6fa400c940 comp rx=0 tx=0).stop 2024-09-16T15:46:57.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.448+0000 7f6fbb200640 1 -- 172.21.15.151:0/1225794803 shutdown_connections 2024-09-16T15:46:57.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.448+0000 7f6fbb200640 1 --2- 172.21.15.151:0/1225794803 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6f980794e0 0x7f6f9807b9a0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:57.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.448+0000 7f6fbb200640 1 --2- 172.21.15.151:0/1225794803 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6fbc161440 0x7f6fbc0c7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:57.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.448+0000 7f6fbb200640 1 --2- 172.21.15.151:0/1225794803 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6fbc0b4050 0x7f6fbc0c7920 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:57.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.448+0000 7f6fbb200640 1 --2- 172.21.15.151:0/1225794803 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6fbc0b34b0 0x7f6fbc1532f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:57.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.448+0000 7f6fbb200640 1 -- 172.21.15.151:0/1225794803 >> 172.21.15.151:0/1225794803 conn(0x7f6fbc0d3df0 msgr2=0x7f6fbc15f030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:57.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.448+0000 7f6fbb200640 1 -- 172.21.15.151:0/1225794803 shutdown_connections 2024-09-16T15:46:57.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.448+0000 7f6fbb200640 1 -- 172.21.15.151:0/1225794803 wait complete. 2024-09-16T15:46:57.466 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":302,"stamp":"2024-09-16T15:46:55.706253+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":20,"acting":20,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":20,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":3,"kb":754974720,"kb_used":406076,"kb_used_data":7992,"kb_used_omap":12,"kb_used_meta":397875,"kb_avail":754568644,"statfs":{"total":773094113280,"available":772678291456,"internally_reserved":0,"allocated":8183808,"data_stored":4137743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12738,"internal_metadata":407424574},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.468314"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:52.233967+0000","last_change":"2024-09-16T15:46:52.233967+0000","last_active":"2024-09-16T15:46:52.233967+0000","last_peered":"2024-09-16T15:46:52.233967+0000","last_clean":"2024-09-16T15:46:52.233967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:52.233967+0000","last_undegraded":"2024-09-16T15:46:52.233967+0000","last_fullsized":"2024-09-16T15:46:52.233967+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:52.233967+0000","last_change":"2024-09-16T15:46:52.233967+0000","last_active":"2024-09-16T15:46:52.233967+0000","last_peered":"2024-09-16T15:46:52.233967+0000","last_clean":"2024-09-16T15:46:52.233967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:52.233967+0000","last_undegraded":"2024-09-16T15:46:52.233967+0000","last_fullsized":"2024-09-16T15:46:52.233967+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.264962+0000","last_change":"2024-09-16T15:46:53.288961+0000","last_active":"2024-09-16T15:46:54.264962+0000","last_peered":"2024-09-16T15:46:54.264962+0000","last_clean":"2024-09-16T15:46:54.264962+0000","last_became_active":"2024-09-16T15:46:53.288713+0000","last_became_peered":"2024-09-16T15:46:53.288713+0000","last_unstale":"2024-09-16T15:46:54.264962+0000","last_undegraded":"2024-09-16T15:46:54.264962+0000","last_fullsized":"2024-09-16T15:46:54.264962+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:18:24.768957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.264684+0000","last_change":"2024-09-16T15:46:53.281566+0000","last_active":"2024-09-16T15:46:54.264684+0000","last_peered":"2024-09-16T15:46:54.264684+0000","last_clean":"2024-09-16T15:46:54.264684+0000","last_became_active":"2024-09-16T15:46:53.281086+0000","last_became_peered":"2024-09-16T15:46:53.281086+0000","last_unstale":"2024-09-16T15:46:54.264684+0000","last_undegraded":"2024-09-16T15:46:54.264684+0000","last_fullsized":"2024-09-16T15:46:54.264684+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:00:00.388493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:52.233967+0000","last_change":"2024-09-16T15:46:52.233967+0000","last_active":"2024-09-16T15:46:52.233967+0000","last_peered":"2024-09-16T15:46:52.233967+0000","last_clean":"2024-09-16T15:46:52.233967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:52.233967+0000","last_undegraded":"2024-09-16T15:46:52.233967+0000","last_fullsized":"2024-09-16T15:46:52.233967+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:52.233967+0000","last_change":"2024-09-16T15:46:52.233967+0000","last_active":"2024-09-16T15:46:52.233967+0000","last_peered":"2024-09-16T15:46:52.233967+0000","last_clean":"2024-09-16T15:46:52.233967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:52.233967+0000","last_undegraded":"2024-09-16T15:46:52.233967+0000","last_fullsized":"2024-09-16T15:46:52.233967+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:52.233967+0000","last_change":"2024-09-16T15:46:52.233967+0000","last_active":"2024-09-16T15:46:52.233967+0000","last_peered":"2024-09-16T15:46:52.233967+0000","last_clean":"2024-09-16T15:46:52.233967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:52.233967+0000","last_undegraded":"2024-09-16T15:46:52.233967+0000","last_fullsized":"2024-09-16T15:46:52.233967+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:52.233967+0000","last_change":"2024-09-16T15:46:52.233967+0000","last_active":"2024-09-16T15:46:52.233967+0000","last_peered":"2024-09-16T15:46:52.233967+0000","last_clean":"2024-09-16T15:46:52.233967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:52.233967+0000","last_undegraded":"2024-09-16T15:46:52.233967+0000","last_fullsized":"2024-09-16T15:46:52.233967+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.265293+0000","last_change":"2024-09-16T15:46:53.267944+0000","last_active":"2024-09-16T15:46:54.265293+0000","last_peered":"2024-09-16T15:46:54.265293+0000","last_clean":"2024-09-16T15:46:54.265293+0000","last_became_active":"2024-09-16T15:46:53.267623+0000","last_became_peered":"2024-09-16T15:46:53.267623+0000","last_unstale":"2024-09-16T15:46:54.265293+0000","last_undegraded":"2024-09-16T15:46:54.265293+0000","last_fullsized":"2024-09-16T15:46:54.265293+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:43:34.363272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:52.233967+0000","last_change":"2024-09-16T15:46:52.233967+0000","last_active":"2024-09-16T15:46:52.233967+0000","last_peered":"2024-09-16T15:46:52.233967+0000","last_clean":"2024-09-16T15:46:52.233967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:52.233967+0000","last_undegraded":"2024-09-16T15:46:52.233967+0000","last_fullsized":"2024-09-16T15:46:52.233967+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:52.233967+0000","last_change":"2024-09-16T15:46:52.233967+0000","last_active":"2024-09-16T15:46:52.233967+0000","last_peered":"2024-09-16T15:46:52.233967+0000","last_clean":"2024-09-16T15:46:52.233967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:52.233967+0000","last_undegraded":"2024-09-16T15:46:52.233967+0000","last_fullsized":"2024-09-16T15:46:52.233967+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:52.233967+0000","last_change":"2024-09-16T15:46:52.233967+0000","last_active":"2024-09-16T15:46:52.233967+0000","last_peered":"2024-09-16T15:46:52.233967+0000","last_clean":"2024-09-16T15:46:52.233967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:52.233967+0000","last_undegraded":"2024-09-16T15:46:52.233967+0000","last_fullsized":"2024-09-16T15:46:52.233967+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.264754+0000","last_change":"2024-09-16T15:46:53.287367+0000","last_active":"2024-09-16T15:46:54.264754+0000","last_peered":"2024-09-16T15:46:54.264754+0000","last_clean":"2024-09-16T15:46:54.264754+0000","last_became_active":"2024-09-16T15:46:53.287061+0000","last_became_peered":"2024-09-16T15:46:53.287061+0000","last_unstale":"2024-09-16T15:46:54.264754+0000","last_undegraded":"2024-09-16T15:46:54.264754+0000","last_fullsized":"2024-09-16T15:46:54.264754+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:21:45.984928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:52.233967+0000","last_change":"2024-09-16T15:46:52.233967+0000","last_active":"2024-09-16T15:46:52.233967+0000","last_peered":"2024-09-16T15:46:52.233967+0000","last_clean":"2024-09-16T15:46:52.233967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:52.233967+0000","last_undegraded":"2024-09-16T15:46:52.233967+0000","last_fullsized":"2024-09-16T15:46:52.233967+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:52.233967+0000","last_change":"2024-09-16T15:46:52.233967+0000","last_active":"2024-09-16T15:46:52.233967+0000","last_peered":"2024-09-16T15:46:52.233967+0000","last_clean":"2024-09-16T15:46:52.233967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:52.233967+0000","last_undegraded":"2024-09-16T15:46:52.233967+0000","last_fullsized":"2024-09-16T15:46:52.233967+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:52.233967+0000","last_change":"2024-09-16T15:46:52.233967+0000","last_active":"2024-09-16T15:46:52.233967+0000","last_peered":"2024-09-16T15:46:52.233967+0000","last_clean":"2024-09-16T15:46:52.233967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:52.233967+0000","last_undegraded":"2024-09-16T15:46:52.233967+0000","last_fullsized":"2024-09-16T15:46:52.233967+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:52.233967+0000","last_change":"2024-09-16T15:46:52.233967+0000","last_active":"2024-09-16T15:46:52.233967+0000","last_peered":"2024-09-16T15:46:52.233967+0000","last_clean":"2024-09-16T15:46:52.233967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:52.233967+0000","last_undegraded":"2024-09-16T15:46:52.233967+0000","last_fullsized":"2024-09-16T15:46:52.233967+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:52.233967+0000","last_change":"2024-09-16T15:46:52.233967+0000","last_active":"2024-09-16T15:46:52.233967+0000","last_peered":"2024-09-16T15:46:52.233967+0000","last_clean":"2024-09-16T15:46:52.233967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:52.233967+0000","last_undegraded":"2024-09-16T15:46:52.233967+0000","last_fullsized":"2024-09-16T15:46:52.233967+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:52.233967+0000","last_change":"2024-09-16T15:46:52.233967+0000","last_active":"2024-09-16T15:46:52.233967+0000","last_peered":"2024-09-16T15:46:52.233967+0000","last_clean":"2024-09-16T15:46:52.233967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:52.233967+0000","last_undegraded":"2024-09-16T15:46:52.233967+0000","last_fullsized":"2024-09-16T15:46:52.233967+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:52.233967+0000","last_change":"2024-09-16T15:46:52.233967+0000","last_active":"2024-09-16T15:46:52.233967+0000","last_peered":"2024-09-16T15:46:52.233967+0000","last_clean":"2024-09-16T15:46:52.233967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:52.233967+0000","last_undegraded":"2024-09-16T15:46:52.233967+0000","last_fullsized":"2024-09-16T15:46:52.233967+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:52.233967+0000","last_change":"2024-09-16T15:46:52.233967+0000","last_active":"2024-09-16T15:46:52.233967+0000","last_peered":"2024-09-16T15:46:52.233967+0000","last_clean":"2024-09-16T15:46:52.233967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:52.233967+0000","last_undegraded":"2024-09-16T15:46:52.233967+0000","last_fullsized":"2024-09-16T15:46:52.233967+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:52.233967+0000","last_change":"2024-09-16T15:46:52.233967+0000","last_active":"2024-09-16T15:46:52.233967+0000","last_peered":"2024-09-16T15:46:52.233967+0000","last_clean":"2024-09-16T15:46:52.233967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:52.233967+0000","last_undegraded":"2024-09-16T15:46:52.233967+0000","last_fullsized":"2024-09-16T15:46:52.233967+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:52.233967+0000","last_change":"2024-09-16T15:46:52.233967+0000","last_active":"2024-09-16T15:46:52.233967+0000","last_peered":"2024-09-16T15:46:52.233967+0000","last_clean":"2024-09-16T15:46:52.233967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:52.233967+0000","last_undegraded":"2024-09-16T15:46:52.233967+0000","last_fullsized":"2024-09-16T15:46:52.233967+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:52.233967+0000","last_change":"2024-09-16T15:46:52.233967+0000","last_active":"2024-09-16T15:46:52.233967+0000","last_peered":"2024-09-16T15:46:52.233967+0000","last_clean":"2024-09-16T15:46:52.233967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:52.233967+0000","last_undegraded":"2024-09-16T15:46:52.233967+0000","last_fullsized":"2024-09-16T15:46:52.233967+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:52.233967+0000","last_change":"2024-09-16T15:46:52.233967+0000","last_active":"2024-09-16T15:46:52.233967+0000","last_peered":"2024-09-16T15:46:52.233967+0000","last_clean":"2024-09-16T15:46:52.233967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:52.233967+0000","last_undegraded":"2024-09-16T15:46:52.233967+0000","last_fullsized":"2024-09-16T15:46:52.233967+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:52.233967+0000","last_change":"2024-09-16T15:46:52.233967+0000","last_active":"2024-09-16T15:46:52.233967+0000","last_peered":"2024-09-16T15:46:52.233967+0000","last_clean":"2024-09-16T15:46:52.233967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:52.233967+0000","last_undegraded":"2024-09-16T15:46:52.233967+0000","last_fullsized":"2024-09-16T15:46:52.233967+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:52.233967+0000","last_change":"2024-09-16T15:46:52.233967+0000","last_active":"2024-09-16T15:46:52.233967+0000","last_peered":"2024-09-16T15:46:52.233967+0000","last_clean":"2024-09-16T15:46:52.233967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:52.233967+0000","last_undegraded":"2024-09-16T15:46:52.233967+0000","last_fullsized":"2024-09-16T15:46:52.233967+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:52.233967+0000","last_change":"2024-09-16T15:46:52.233967+0000","last_active":"2024-09-16T15:46:52.233967+0000","last_peered":"2024-09-16T15:46:52.233967+0000","last_clean":"2024-09-16T15:46:52.233967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:52.233967+0000","last_undegraded":"2024-09-16T15:46:52.233967+0000","last_fullsized":"2024-09-16T15:46:52.233967+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.264812+0000","last_change":"2024-09-16T15:46:53.268045+0000","last_active":"2024-09-16T15:46:54.264812+0000","last_peered":"2024-09-16T15:46:54.264812+0000","last_clean":"2024-09-16T15:46:54.264812+0000","last_became_active":"2024-09-16T15:46:53.267679+0000","last_became_peered":"2024-09-16T15:46:53.267679+0000","last_unstale":"2024-09-16T15:46:54.264812+0000","last_undegraded":"2024-09-16T15:46:54.264812+0000","last_fullsized":"2024-09-16T15:46:54.264812+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:07:25.144696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:52.233967+0000","last_change":"2024-09-16T15:46:52.233967+0000","last_active":"2024-09-16T15:46:52.233967+0000","last_peered":"2024-09-16T15:46:52.233967+0000","last_clean":"2024-09-16T15:46:52.233967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:52.233967+0000","last_undegraded":"2024-09-16T15:46:52.233967+0000","last_fullsized":"2024-09-16T15:46:52.233967+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:52.233967+0000","last_change":"2024-09-16T15:46:52.233967+0000","last_active":"2024-09-16T15:46:52.233967+0000","last_peered":"2024-09-16T15:46:52.233967+0000","last_clean":"2024-09-16T15:46:52.233967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:52.233967+0000","last_undegraded":"2024-09-16T15:46:52.233967+0000","last_fullsized":"2024-09-16T15:46:52.233967+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:52.233967+0000","last_change":"2024-09-16T15:46:52.233967+0000","last_active":"2024-09-16T15:46:52.233967+0000","last_peered":"2024-09-16T15:46:52.233967+0000","last_clean":"2024-09-16T15:46:52.233967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:52.233967+0000","last_undegraded":"2024-09-16T15:46:52.233967+0000","last_fullsized":"2024-09-16T15:46:52.233967+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:52.233967+0000","last_change":"2024-09-16T15:46:52.233967+0000","last_active":"2024-09-16T15:46:52.233967+0000","last_peered":"2024-09-16T15:46:52.233967+0000","last_clean":"2024-09-16T15:46:52.233967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:52.233967+0000","last_undegraded":"2024-09-16T15:46:52.233967+0000","last_fullsized":"2024-09-16T15:46:52.233967+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:51.231199+0000","last_change":"2024-09-16T15:46:51.231199+0000","last_active":"2024-09-16T15:46:51.231199+0000","last_peered":"2024-09-16T15:46:51.231199+0000","last_clean":"2024-09-16T15:46:51.231199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:51.231199+0000","last_undegraded":"2024-09-16T15:46:51.231199+0000","last_fullsized":"2024-09-16T15:46:51.231199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:51.231199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.265451+0000","last_change":"2024-09-16T15:46:53.274983+0000","last_active":"2024-09-16T15:46:54.265451+0000","last_peered":"2024-09-16T15:46:54.265451+0000","last_clean":"2024-09-16T15:46:54.265451+0000","last_became_active":"2024-09-16T15:46:53.274558+0000","last_became_peered":"2024-09-16T15:46:53.274558+0000","last_unstale":"2024-09-16T15:46:54.265451+0000","last_undegraded":"2024-09-16T15:46:54.265451+0000","last_fullsized":"2024-09-16T15:46:54.265451+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:37:10.502831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:51.231199+0000","last_change":"2024-09-16T15:46:51.231199+0000","last_active":"2024-09-16T15:46:51.231199+0000","last_peered":"2024-09-16T15:46:51.231199+0000","last_clean":"2024-09-16T15:46:51.231199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:51.231199+0000","last_undegraded":"2024-09-16T15:46:51.231199+0000","last_fullsized":"2024-09-16T15:46:51.231199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:51.231199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:52.233967+0000","last_change":"2024-09-16T15:46:52.233967+0000","last_active":"2024-09-16T15:46:52.233967+0000","last_peered":"2024-09-16T15:46:52.233967+0000","last_clean":"2024-09-16T15:46:52.233967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:52.233967+0000","last_undegraded":"2024-09-16T15:46:52.233967+0000","last_fullsized":"2024-09-16T15:46:52.233967+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:51.231199+0000","last_change":"2024-09-16T15:46:51.231199+0000","last_active":"2024-09-16T15:46:51.231199+0000","last_peered":"2024-09-16T15:46:51.231199+0000","last_clean":"2024-09-16T15:46:51.231199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:51.231199+0000","last_undegraded":"2024-09-16T15:46:51.231199+0000","last_fullsized":"2024-09-16T15:46:51.231199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:51.231199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:52.233967+0000","last_change":"2024-09-16T15:46:52.233967+0000","last_active":"2024-09-16T15:46:52.233967+0000","last_peered":"2024-09-16T15:46:52.233967+0000","last_clean":"2024-09-16T15:46:52.233967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:52.233967+0000","last_undegraded":"2024-09-16T15:46:52.233967+0000","last_fullsized":"2024-09-16T15:46:52.233967+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:51.231199+0000","last_change":"2024-09-16T15:46:51.231199+0000","last_active":"2024-09-16T15:46:51.231199+0000","last_peered":"2024-09-16T15:46:51.231199+0000","last_clean":"2024-09-16T15:46:51.231199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:51.231199+0000","last_undegraded":"2024-09-16T15:46:51.231199+0000","last_fullsized":"2024-09-16T15:46:51.231199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:51.231199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:52.233967+0000","last_change":"2024-09-16T15:46:52.233967+0000","last_active":"2024-09-16T15:46:52.233967+0000","last_peered":"2024-09-16T15:46:52.233967+0000","last_clean":"2024-09-16T15:46:52.233967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:52.233967+0000","last_undegraded":"2024-09-16T15:46:52.233967+0000","last_fullsized":"2024-09-16T15:46:52.233967+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:51.231199+0000","last_change":"2024-09-16T15:46:51.231199+0000","last_active":"2024-09-16T15:46:51.231199+0000","last_peered":"2024-09-16T15:46:51.231199+0000","last_clean":"2024-09-16T15:46:51.231199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:51.231199+0000","last_undegraded":"2024-09-16T15:46:51.231199+0000","last_fullsized":"2024-09-16T15:46:51.231199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:51.231199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:52.233967+0000","last_change":"2024-09-16T15:46:52.233967+0000","last_active":"2024-09-16T15:46:52.233967+0000","last_peered":"2024-09-16T15:46:52.233967+0000","last_clean":"2024-09-16T15:46:52.233967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:52.233967+0000","last_undegraded":"2024-09-16T15:46:52.233967+0000","last_fullsized":"2024-09-16T15:46:52.233967+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:51.231199+0000","last_change":"2024-09-16T15:46:51.231199+0000","last_active":"2024-09-16T15:46:51.231199+0000","last_peered":"2024-09-16T15:46:51.231199+0000","last_clean":"2024-09-16T15:46:51.231199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:51.231199+0000","last_undegraded":"2024-09-16T15:46:51.231199+0000","last_fullsized":"2024-09-16T15:46:51.231199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:51.231199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:52.233967+0000","last_change":"2024-09-16T15:46:52.233967+0000","last_active":"2024-09-16T15:46:52.233967+0000","last_peered":"2024-09-16T15:46:52.233967+0000","last_clean":"2024-09-16T15:46:52.233967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:52.233967+0000","last_undegraded":"2024-09-16T15:46:52.233967+0000","last_fullsized":"2024-09-16T15:46:52.233967+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:51.231199+0000","last_change":"2024-09-16T15:46:51.231199+0000","last_active":"2024-09-16T15:46:51.231199+0000","last_peered":"2024-09-16T15:46:51.231199+0000","last_clean":"2024-09-16T15:46:51.231199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:51.231199+0000","last_undegraded":"2024-09-16T15:46:51.231199+0000","last_fullsized":"2024-09-16T15:46:51.231199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:51.231199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:52.233967+0000","last_change":"2024-09-16T15:46:52.233967+0000","last_active":"2024-09-16T15:46:52.233967+0000","last_peered":"2024-09-16T15:46:52.233967+0000","last_clean":"2024-09-16T15:46:52.233967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:52.233967+0000","last_undegraded":"2024-09-16T15:46:52.233967+0000","last_fullsized":"2024-09-16T15:46:52.233967+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:51.231199+0000","last_change":"2024-09-16T15:46:51.231199+0000","last_active":"2024-09-16T15:46:51.231199+0000","last_peered":"2024-09-16T15:46:51.231199+0000","last_clean":"2024-09-16T15:46:51.231199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:51.231199+0000","last_undegraded":"2024-09-16T15:46:51.231199+0000","last_fullsized":"2024-09-16T15:46:51.231199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:51.231199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:52.233967+0000","last_change":"2024-09-16T15:46:52.233967+0000","last_active":"2024-09-16T15:46:52.233967+0000","last_peered":"2024-09-16T15:46:52.233967+0000","last_clean":"2024-09-16T15:46:52.233967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:52.233967+0000","last_undegraded":"2024-09-16T15:46:52.233967+0000","last_fullsized":"2024-09-16T15:46:52.233967+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:51.231199+0000","last_change":"2024-09-16T15:46:51.231199+0000","last_active":"2024-09-16T15:46:51.231199+0000","last_peered":"2024-09-16T15:46:51.231199+0000","last_clean":"2024-09-16T15:46:51.231199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:51.231199+0000","last_undegraded":"2024-09-16T15:46:51.231199+0000","last_fullsized":"2024-09-16T15:46:51.231199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:51.231199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:52.233967+0000","last_change":"2024-09-16T15:46:52.233967+0000","last_active":"2024-09-16T15:46:52.233967+0000","last_peered":"2024-09-16T15:46:52.233967+0000","last_clean":"2024-09-16T15:46:52.233967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:52.233967+0000","last_undegraded":"2024-09-16T15:46:52.233967+0000","last_fullsized":"2024-09-16T15:46:52.233967+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:51.231199+0000","last_change":"2024-09-16T15:46:51.231199+0000","last_active":"2024-09-16T15:46:51.231199+0000","last_peered":"2024-09-16T15:46:51.231199+0000","last_clean":"2024-09-16T15:46:51.231199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:51.231199+0000","last_undegraded":"2024-09-16T15:46:51.231199+0000","last_fullsized":"2024-09-16T15:46:51.231199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:51.231199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:52.233967+0000","last_change":"2024-09-16T15:46:52.233967+0000","last_active":"2024-09-16T15:46:52.233967+0000","last_peered":"2024-09-16T15:46:52.233967+0000","last_clean":"2024-09-16T15:46:52.233967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:52.233967+0000","last_undegraded":"2024-09-16T15:46:52.233967+0000","last_fullsized":"2024-09-16T15:46:52.233967+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:51.231199+0000","last_change":"2024-09-16T15:46:51.231199+0000","last_active":"2024-09-16T15:46:51.231199+0000","last_peered":"2024-09-16T15:46:51.231199+0000","last_clean":"2024-09-16T15:46:51.231199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:51.231199+0000","last_undegraded":"2024-09-16T15:46:51.231199+0000","last_fullsized":"2024-09-16T15:46:51.231199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:51.231199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:52.233967+0000","last_change":"2024-09-16T15:46:52.233967+0000","last_active":"2024-09-16T15:46:52.233967+0000","last_peered":"2024-09-16T15:46:52.233967+0000","last_clean":"2024-09-16T15:46:52.233967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:52.233967+0000","last_undegraded":"2024-09-16T15:46:52.233967+0000","last_fullsized":"2024-09-16T15:46:52.233967+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:51.231199+0000","last_change":"2024-09-16T15:46:51.231199+0000","last_active":"2024-09-16T15:46:51.231199+0000","last_peered":"2024-09-16T15:46:51.231199+0000","last_clean":"2024-09-16T15:46:51.231199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:51.231199+0000","last_undegraded":"2024-09-16T15:46:51.231199+0000","last_fullsized":"2024-09-16T15:46:51.231199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:51.231199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:52.233967+0000","last_change":"2024-09-16T15:46:52.233967+0000","last_active":"2024-09-16T15:46:52.233967+0000","last_peered":"2024-09-16T15:46:52.233967+0000","last_clean":"2024-09-16T15:46:52.233967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:52.233967+0000","last_undegraded":"2024-09-16T15:46:52.233967+0000","last_fullsized":"2024-09-16T15:46:52.233967+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:51.231199+0000","last_change":"2024-09-16T15:46:51.231199+0000","last_active":"2024-09-16T15:46:51.231199+0000","last_peered":"2024-09-16T15:46:51.231199+0000","last_clean":"2024-09-16T15:46:51.231199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:51.231199+0000","last_undegraded":"2024-09-16T15:46:51.231199+0000","last_fullsized":"2024-09-16T15:46:51.231199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:51.231199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:52.233967+0000","last_change":"2024-09-16T15:46:52.233967+0000","last_active":"2024-09-16T15:46:52.233967+0000","last_peered":"2024-09-16T15:46:52.233967+0000","last_clean":"2024-09-16T15:46:52.233967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:52.233967+0000","last_undegraded":"2024-09-16T15:46:52.233967+0000","last_fullsized":"2024-09-16T15:46:52.233967+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:51.231199+0000","last_change":"2024-09-16T15:46:51.231199+0000","last_active":"2024-09-16T15:46:51.231199+0000","last_peered":"2024-09-16T15:46:51.231199+0000","last_clean":"2024-09-16T15:46:51.231199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:51.231199+0000","last_undegraded":"2024-09-16T15:46:51.231199+0000","last_fullsized":"2024-09-16T15:46:51.231199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:51.231199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:52.233967+0000","last_change":"2024-09-16T15:46:52.233967+0000","last_active":"2024-09-16T15:46:52.233967+0000","last_peered":"2024-09-16T15:46:52.233967+0000","last_clean":"2024-09-16T15:46:52.233967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:52.233967+0000","last_undegraded":"2024-09-16T15:46:52.233967+0000","last_fullsized":"2024-09-16T15:46:52.233967+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:51.231199+0000","last_change":"2024-09-16T15:46:51.231199+0000","last_active":"2024-09-16T15:46:51.231199+0000","last_peered":"2024-09-16T15:46:51.231199+0000","last_clean":"2024-09-16T15:46:51.231199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:51.231199+0000","last_undegraded":"2024-09-16T15:46:51.231199+0000","last_fullsized":"2024-09-16T15:46:51.231199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:51.231199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.264245+0000","last_change":"2024-09-16T15:46:53.272955+0000","last_active":"2024-09-16T15:46:54.264245+0000","last_peered":"2024-09-16T15:46:54.264245+0000","last_clean":"2024-09-16T15:46:54.264245+0000","last_became_active":"2024-09-16T15:46:53.272155+0000","last_became_peered":"2024-09-16T15:46:53.272155+0000","last_unstale":"2024-09-16T15:46:54.264245+0000","last_undegraded":"2024-09-16T15:46:54.264245+0000","last_fullsized":"2024-09-16T15:46:54.264245+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:50:56.316541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:51.231199+0000","last_change":"2024-09-16T15:46:51.231199+0000","last_active":"2024-09-16T15:46:51.231199+0000","last_peered":"2024-09-16T15:46:51.231199+0000","last_clean":"2024-09-16T15:46:51.231199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:51.231199+0000","last_undegraded":"2024-09-16T15:46:51.231199+0000","last_fullsized":"2024-09-16T15:46:51.231199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:51.231199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:52.233967+0000","last_change":"2024-09-16T15:46:52.233967+0000","last_active":"2024-09-16T15:46:52.233967+0000","last_peered":"2024-09-16T15:46:52.233967+0000","last_clean":"2024-09-16T15:46:52.233967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:52.233967+0000","last_undegraded":"2024-09-16T15:46:52.233967+0000","last_fullsized":"2024-09-16T15:46:52.233967+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:51.231199+0000","last_change":"2024-09-16T15:46:51.231199+0000","last_active":"2024-09-16T15:46:51.231199+0000","last_peered":"2024-09-16T15:46:51.231199+0000","last_clean":"2024-09-16T15:46:51.231199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:51.231199+0000","last_undegraded":"2024-09-16T15:46:51.231199+0000","last_fullsized":"2024-09-16T15:46:51.231199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:51.231199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:52.233967+0000","last_change":"2024-09-16T15:46:52.233967+0000","last_active":"2024-09-16T15:46:52.233967+0000","last_peered":"2024-09-16T15:46:52.233967+0000","last_clean":"2024-09-16T15:46:52.233967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:52.233967+0000","last_undegraded":"2024-09-16T15:46:52.233967+0000","last_fullsized":"2024-09-16T15:46:52.233967+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:51.231199+0000","last_change":"2024-09-16T15:46:51.231199+0000","last_active":"2024-09-16T15:46:51.231199+0000","last_peered":"2024-09-16T15:46:51.231199+0000","last_clean":"2024-09-16T15:46:51.231199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:51.231199+0000","last_undegraded":"2024-09-16T15:46:51.231199+0000","last_fullsized":"2024-09-16T15:46:51.231199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:51.231199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.264936+0000","last_change":"2024-09-16T15:46:53.267133+0000","last_active":"2024-09-16T15:46:54.264936+0000","last_peered":"2024-09-16T15:46:54.264936+0000","last_clean":"2024-09-16T15:46:54.264936+0000","last_became_active":"2024-09-16T15:46:53.266542+0000","last_became_peered":"2024-09-16T15:46:53.266542+0000","last_unstale":"2024-09-16T15:46:54.264936+0000","last_undegraded":"2024-09-16T15:46:54.264936+0000","last_fullsized":"2024-09-16T15:46:54.264936+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:48:52.607177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:51.231199+0000","last_change":"2024-09-16T15:46:51.231199+0000","last_active":"2024-09-16T15:46:51.231199+0000","last_peered":"2024-09-16T15:46:51.231199+0000","last_clean":"2024-09-16T15:46:51.231199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:51.231199+0000","last_undegraded":"2024-09-16T15:46:51.231199+0000","last_fullsized":"2024-09-16T15:46:51.231199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:51.231199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.265091+0000","last_change":"2024-09-16T15:46:53.272941+0000","last_active":"2024-09-16T15:46:54.265091+0000","last_peered":"2024-09-16T15:46:54.265091+0000","last_clean":"2024-09-16T15:46:54.265091+0000","last_became_active":"2024-09-16T15:46:53.272641+0000","last_became_peered":"2024-09-16T15:46:53.272641+0000","last_unstale":"2024-09-16T15:46:54.265091+0000","last_undegraded":"2024-09-16T15:46:54.265091+0000","last_fullsized":"2024-09-16T15:46:54.265091+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:04:35.594627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:51.231199+0000","last_change":"2024-09-16T15:46:51.231199+0000","last_active":"2024-09-16T15:46:51.231199+0000","last_peered":"2024-09-16T15:46:51.231199+0000","last_clean":"2024-09-16T15:46:51.231199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:51.231199+0000","last_undegraded":"2024-09-16T15:46:51.231199+0000","last_fullsized":"2024-09-16T15:46:51.231199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:51.231199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:52.233967+0000","last_change":"2024-09-16T15:46:52.233967+0000","last_active":"2024-09-16T15:46:52.233967+0000","last_peered":"2024-09-16T15:46:52.233967+0000","last_clean":"2024-09-16T15:46:52.233967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:52.233967+0000","last_undegraded":"2024-09-16T15:46:52.233967+0000","last_fullsized":"2024-09-16T15:46:52.233967+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:51.231199+0000","last_change":"2024-09-16T15:46:51.231199+0000","last_active":"2024-09-16T15:46:51.231199+0000","last_peered":"2024-09-16T15:46:51.231199+0000","last_clean":"2024-09-16T15:46:51.231199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:51.231199+0000","last_undegraded":"2024-09-16T15:46:51.231199+0000","last_fullsized":"2024-09-16T15:46:51.231199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:51.231199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:52.233967+0000","last_change":"2024-09-16T15:46:52.233967+0000","last_active":"2024-09-16T15:46:52.233967+0000","last_peered":"2024-09-16T15:46:52.233967+0000","last_clean":"2024-09-16T15:46:52.233967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:52.233967+0000","last_undegraded":"2024-09-16T15:46:52.233967+0000","last_fullsized":"2024-09-16T15:46:52.233967+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:51.231199+0000","last_change":"2024-09-16T15:46:51.231199+0000","last_active":"2024-09-16T15:46:51.231199+0000","last_peered":"2024-09-16T15:46:51.231199+0000","last_clean":"2024-09-16T15:46:51.231199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:51.231199+0000","last_undegraded":"2024-09-16T15:46:51.231199+0000","last_fullsized":"2024-09-16T15:46:51.231199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:51.231199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:52.233967+0000","last_change":"2024-09-16T15:46:52.233967+0000","last_active":"2024-09-16T15:46:52.233967+0000","last_peered":"2024-09-16T15:46:52.233967+0000","last_clean":"2024-09-16T15:46:52.233967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:52.233967+0000","last_undegraded":"2024-09-16T15:46:52.233967+0000","last_fullsized":"2024-09-16T15:46:52.233967+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:51.231199+0000","last_change":"2024-09-16T15:46:51.231199+0000","last_active":"2024-09-16T15:46:51.231199+0000","last_peered":"2024-09-16T15:46:51.231199+0000","last_clean":"2024-09-16T15:46:51.231199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:51.231199+0000","last_undegraded":"2024-09-16T15:46:51.231199+0000","last_fullsized":"2024-09-16T15:46:51.231199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:51.231199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:52.233967+0000","last_change":"2024-09-16T15:46:52.233967+0000","last_active":"2024-09-16T15:46:52.233967+0000","last_peered":"2024-09-16T15:46:52.233967+0000","last_clean":"2024-09-16T15:46:52.233967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:52.233967+0000","last_undegraded":"2024-09-16T15:46:52.233967+0000","last_fullsized":"2024-09-16T15:46:52.233967+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:51.231199+0000","last_change":"2024-09-16T15:46:51.231199+0000","last_active":"2024-09-16T15:46:51.231199+0000","last_peered":"2024-09-16T15:46:51.231199+0000","last_clean":"2024-09-16T15:46:51.231199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:51.231199+0000","last_undegraded":"2024-09-16T15:46:51.231199+0000","last_fullsized":"2024-09-16T15:46:51.231199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:51.231199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:52.233967+0000","last_change":"2024-09-16T15:46:52.233967+0000","last_active":"2024-09-16T15:46:52.233967+0000","last_peered":"2024-09-16T15:46:52.233967+0000","last_clean":"2024-09-16T15:46:52.233967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:52.233967+0000","last_undegraded":"2024-09-16T15:46:52.233967+0000","last_fullsized":"2024-09-16T15:46:52.233967+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:51.231199+0000","last_change":"2024-09-16T15:46:51.231199+0000","last_active":"2024-09-16T15:46:51.231199+0000","last_peered":"2024-09-16T15:46:51.231199+0000","last_clean":"2024-09-16T15:46:51.231199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:51.231199+0000","last_undegraded":"2024-09-16T15:46:51.231199+0000","last_fullsized":"2024-09-16T15:46:51.231199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:51.231199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:52.233967+0000","last_change":"2024-09-16T15:46:52.233967+0000","last_active":"2024-09-16T15:46:52.233967+0000","last_peered":"2024-09-16T15:46:52.233967+0000","last_clean":"2024-09-16T15:46:52.233967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:52.233967+0000","last_undegraded":"2024-09-16T15:46:52.233967+0000","last_fullsized":"2024-09-16T15:46:52.233967+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:51.231199+0000","last_change":"2024-09-16T15:46:51.231199+0000","last_active":"2024-09-16T15:46:51.231199+0000","last_peered":"2024-09-16T15:46:51.231199+0000","last_clean":"2024-09-16T15:46:51.231199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:51.231199+0000","last_undegraded":"2024-09-16T15:46:51.231199+0000","last_fullsized":"2024-09-16T15:46:51.231199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:51.231199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:52.233967+0000","last_change":"2024-09-16T15:46:52.233967+0000","last_active":"2024-09-16T15:46:52.233967+0000","last_peered":"2024-09-16T15:46:52.233967+0000","last_clean":"2024-09-16T15:46:52.233967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:52.233967+0000","last_undegraded":"2024-09-16T15:46:52.233967+0000","last_fullsized":"2024-09-16T15:46:52.233967+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:51.231199+0000","last_change":"2024-09-16T15:46:51.231199+0000","last_active":"2024-09-16T15:46:51.231199+0000","last_peered":"2024-09-16T15:46:51.231199+0000","last_clean":"2024-09-16T15:46:51.231199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:51.231199+0000","last_undegraded":"2024-09-16T15:46:51.231199+0000","last_fullsized":"2024-09-16T15:46:51.231199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:51.231199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:52.233967+0000","last_change":"2024-09-16T15:46:52.233967+0000","last_active":"2024-09-16T15:46:52.233967+0000","last_peered":"2024-09-16T15:46:52.233967+0000","last_clean":"2024-09-16T15:46:52.233967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:52.233967+0000","last_undegraded":"2024-09-16T15:46:52.233967+0000","last_fullsized":"2024-09-16T15:46:52.233967+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:51.231199+0000","last_change":"2024-09-16T15:46:51.231199+0000","last_active":"2024-09-16T15:46:51.231199+0000","last_peered":"2024-09-16T15:46:51.231199+0000","last_clean":"2024-09-16T15:46:51.231199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:51.231199+0000","last_undegraded":"2024-09-16T15:46:51.231199+0000","last_fullsized":"2024-09-16T15:46:51.231199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:51.231199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:52.233967+0000","last_change":"2024-09-16T15:46:52.233967+0000","last_active":"2024-09-16T15:46:52.233967+0000","last_peered":"2024-09-16T15:46:52.233967+0000","last_clean":"2024-09-16T15:46:52.233967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:52.233967+0000","last_undegraded":"2024-09-16T15:46:52.233967+0000","last_fullsized":"2024-09-16T15:46:52.233967+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:51.231199+0000","last_change":"2024-09-16T15:46:51.231199+0000","last_active":"2024-09-16T15:46:51.231199+0000","last_peered":"2024-09-16T15:46:51.231199+0000","last_clean":"2024-09-16T15:46:51.231199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:51.231199+0000","last_undegraded":"2024-09-16T15:46:51.231199+0000","last_fullsized":"2024-09-16T15:46:51.231199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:51.231199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:52.233967+0000","last_change":"2024-09-16T15:46:52.233967+0000","last_active":"2024-09-16T15:46:52.233967+0000","last_peered":"2024-09-16T15:46:52.233967+0000","last_clean":"2024-09-16T15:46:52.233967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:52.233967+0000","last_undegraded":"2024-09-16T15:46:52.233967+0000","last_fullsized":"2024-09-16T15:46:52.233967+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:51.231199+0000","last_change":"2024-09-16T15:46:51.231199+0000","last_active":"2024-09-16T15:46:51.231199+0000","last_peered":"2024-09-16T15:46:51.231199+0000","last_clean":"2024-09-16T15:46:51.231199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:51.231199+0000","last_undegraded":"2024-09-16T15:46:51.231199+0000","last_fullsized":"2024-09-16T15:46:51.231199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:51.231199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:52.233967+0000","last_change":"2024-09-16T15:46:52.233967+0000","last_active":"2024-09-16T15:46:52.233967+0000","last_peered":"2024-09-16T15:46:52.233967+0000","last_clean":"2024-09-16T15:46:52.233967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:52.233967+0000","last_undegraded":"2024-09-16T15:46:52.233967+0000","last_fullsized":"2024-09-16T15:46:52.233967+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:51.231199+0000","last_change":"2024-09-16T15:46:51.231199+0000","last_active":"2024-09-16T15:46:51.231199+0000","last_peered":"2024-09-16T15:46:51.231199+0000","last_clean":"2024-09-16T15:46:51.231199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:51.231199+0000","last_undegraded":"2024-09-16T15:46:51.231199+0000","last_fullsized":"2024-09-16T15:46:51.231199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:51.231199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:52.233967+0000","last_change":"2024-09-16T15:46:52.233967+0000","last_active":"2024-09-16T15:46:52.233967+0000","last_peered":"2024-09-16T15:46:52.233967+0000","last_clean":"2024-09-16T15:46:52.233967+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:52.233967+0000","last_undegraded":"2024-09-16T15:46:52.233967+0000","last_fullsized":"2024-09-16T15:46:52.233967+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:46:51.231199+0000","last_change":"2024-09-16T15:46:51.231199+0000","last_active":"2024-09-16T15:46:51.231199+0000","last_peered":"2024-09-16T15:46:51.231199+0000","last_clean":"2024-09-16T15:46:51.231199+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:46:51.231199+0000","last_undegraded":"2024-09-16T15:46:51.231199+0000","last_fullsized":"2024-09-16T15:46:51.231199+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:51.231199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":205,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-16T15:46:49.227237+0000","last_change":"2024-09-16T15:39:12.698813+0000","last_active":"2024-09-16T15:46:49.227237+0000","last_peered":"2024-09-16T15:46:49.227237+0000","last_clean":"2024-09-16T15:46:49.227237+0000","last_became_active":"2024-09-16T15:39:12.697695+0000","last_became_peered":"2024-09-16T15:39:12.697695+0000","last_unstale":"2024-09-16T15:46:49.227237+0000","last_undegraded":"2024-09-16T15:46:49.227237+0000","last_fullsized":"2024-09-16T15:46:49.227237+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_clean_scrub_stamp":"2024-09-16T15:39:11.684967+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:02.061081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":1},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":1},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509537,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50084,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321756,"statfs":{"total":96636764160,"available":96585478144,"internally_reserved":0,"allocated":868352,"data_stored":367046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":50395592},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509536,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54832,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317008,"statfs":{"total":96636764160,"available":96580616192,"internally_reserved":0,"allocated":1462272,"data_stored":957414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509535,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":54256,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317584,"statfs":{"total":96636764160,"available":96581206016,"internally_reserved":0,"allocated":868352,"data_stored":367046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509535,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50084,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321756,"statfs":{"total":96636764160,"available":96585478144,"internally_reserved":0,"allocated":868352,"data_stored":367046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":50395593},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509535,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50092,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321748,"statfs":{"total":96636764160,"available":96585469952,"internally_reserved":0,"allocated":868352,"data_stored":367046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509536,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46504,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325336,"statfs":{"total":96636764160,"available":96589144064,"internally_reserved":0,"allocated":1462272,"data_stored":957414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509534,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50088,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321752,"statfs":{"total":96636764160,"available":96585474048,"internally_reserved":0,"allocated":868352,"data_stored":367046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542242,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50136,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321704,"statfs":{"total":96636764160,"available":96585424896,"internally_reserved":0,"allocated":917504,"data_stored":387685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":50395596},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:46:57.470 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:46:57.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.664+0000 7f4da9e00640 1 -- 172.21.15.151:0/3898327136 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d88003a00 msgr2=0x7f4d88003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:57.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.664+0000 7f4da9e00640 1 --2- 172.21.15.151:0/3898327136 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d88003a00 0x7f4d88003e20 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f4d94009ed0 tx=0x7f4d88005830 comp rx=0 tx=0).stop 2024-09-16T15:46:57.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.664+0000 7f4da3400640 1 -- 172.21.15.151:0/3898327136 reap_dead start 2024-09-16T15:46:57.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.665+0000 7f4da9e00640 1 -- 172.21.15.151:0/3898327136 shutdown_connections 2024-09-16T15:46:57.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.665+0000 7f4da9e00640 1 -- 172.21.15.151:0/3898327136 >> 172.21.15.151:0/3898327136 conn(0x7f4da40b1950 msgr2=0x7f4da40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:57.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.665+0000 7f4da9e00640 1 -- 172.21.15.151:0/3898327136 shutdown_connections 2024-09-16T15:46:57.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.665+0000 7f4da9e00640 1 -- 172.21.15.151:0/3898327136 wait complete. 2024-09-16T15:46:57.668 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.666+0000 7f4da9e00640 1 Processor -- start 2024-09-16T15:46:57.668 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.666+0000 7f4da9e00640 1 -- start start 2024-09-16T15:46:57.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.667+0000 7f4da9e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4d88003a00 0x7f4da414a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:57.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.667+0000 7f4da9e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4da40c8fd0 0x7f4da40c9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:57.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.667+0000 7f4da9e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4da40b0b30 0x7f4da40b0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:57.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.667+0000 7f4da9e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4da40b25c0 con 0x7f4da40b0b30 2024-09-16T15:46:57.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.667+0000 7f4da9e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4da40b2440 con 0x7f4da40c8fd0 2024-09-16T15:46:57.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.667+0000 7f4da9e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f4da40b2740 con 0x7f4d88003a00 2024-09-16T15:46:57.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.668+0000 7f4da3400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4da40b0b30 0x7f4da40b0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:57.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.668+0000 7f4da3400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4da40b0b30 0x7f4da40b0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:46138/0 (socket says 172.21.15.151:46138) 2024-09-16T15:46:57.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.668+0000 7f4da3400640 1 -- 172.21.15.151:0/3531632938 learned_addr learned my addr 172.21.15.151:0/3531632938 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:46:57.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.668+0000 7f4da2000640 1 --2- 172.21.15.151:0/3531632938 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4da40c8fd0 0x7f4da40c9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:57.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.668+0000 7f4da2a00640 1 --2- 172.21.15.151:0/3531632938 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4d88003a00 0x7f4da414a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:57.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.668+0000 7f4da3400640 1 -- 172.21.15.151:0/3531632938 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4d88003a00 msgr2=0x7f4da414a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:57.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.668+0000 7f4da3400640 1 --2- 172.21.15.151:0/3531632938 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4d88003a00 0x7f4da414a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:57.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.668+0000 7f4da3400640 1 -- 172.21.15.151:0/3531632938 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4da40c8fd0 msgr2=0x7f4da40c9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:57.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.668+0000 7f4da3400640 1 --2- 172.21.15.151:0/3531632938 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4da40c8fd0 0x7f4da40c9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:57.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.668+0000 7f4da3400640 1 -- 172.21.15.151:0/3531632938 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4da414ace0 con 0x7f4da40b0b30 2024-09-16T15:46:57.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.668+0000 7f4da2a00640 1 --2- 172.21.15.151:0/3531632938 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4d88003a00 0x7f4da414a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:46:57.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.668+0000 7f4da3400640 1 --2- 172.21.15.151:0/3531632938 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4da40b0b30 0x7f4da40b0f90 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f4da406a510 tx=0x7f4d9c00e980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:57.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.668+0000 7f4d9b400640 1 -- 172.21.15.151:0/3531632938 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d9c0107d0 con 0x7f4da40b0b30 2024-09-16T15:46:57.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.668+0000 7f4d9b400640 1 -- 172.21.15.151:0/3531632938 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4d9c016070 con 0x7f4da40b0b30 2024-09-16T15:46:57.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.668+0000 7f4da9e00640 1 -- 172.21.15.151:0/3531632938 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4da41e7de0 con 0x7f4da40b0b30 2024-09-16T15:46:57.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.669+0000 7f4d9b400640 1 -- 172.21.15.151:0/3531632938 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d9c010d40 con 0x7f4da40b0b30 2024-09-16T15:46:57.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.669+0000 7f4da9e00640 1 -- 172.21.15.151:0/3531632938 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4da41e83a0 con 0x7f4da40b0b30 2024-09-16T15:46:57.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.670+0000 7f4da9e00640 1 -- 172.21.15.151:0/3531632938 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4da40acd30 con 0x7f4da40b0b30 2024-09-16T15:46:57.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.671+0000 7f4d9b400640 1 -- 172.21.15.151:0/3531632938 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 24) ==== 102409+0+0 (secure 0 0 0) 0x7f4d9c01b460 con 0x7f4da40b0b30 2024-09-16T15:46:57.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.672+0000 7f4d9b400640 1 --2- 172.21.15.151:0/3531632938 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4d840794e0 0x7f4d8407b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:57.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.672+0000 7f4d9b400640 1 -- 172.21.15.151:0/3531632938 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5765+0+0 (secure 0 0 0) 0x7f4d9c0995e0 con 0x7f4da40b0b30 2024-09-16T15:46:57.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.673+0000 7f4da2a00640 1 --2- 172.21.15.151:0/3531632938 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4d840794e0 0x7f4d8407b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:57.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.673+0000 7f4da2a00640 1 --2- 172.21.15.151:0/3531632938 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4d840794e0 0x7f4d8407b9a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4d94035000 tx=0x7f4d94004a40 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:57.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.676+0000 7f4d9b400640 1 -- 172.21.15.151:0/3531632938 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4da414ace0 con 0x7f4da40b0b30 2024-09-16T15:46:57.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.859+0000 7f4da9e00640 1 -- 172.21.15.151:0/3531632938 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4da4074240 con 0x7f4d840794e0 2024-09-16T15:46:57.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.865+0000 7f4d9b400640 1 -- 172.21.15.151:0/3531632938 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252060 (secure 0 0 0) 0x7f4da4074240 con 0x7f4d840794e0 2024-09-16T15:46:57.870 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:46:57.871 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-09-16T15:46:57.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.869+0000 7f4da9e00640 1 -- 172.21.15.151:0/3531632938 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4d840794e0 msgr2=0x7f4d8407b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:57.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.869+0000 7f4da9e00640 1 --2- 172.21.15.151:0/3531632938 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4d840794e0 0x7f4d8407b9a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4d94035000 tx=0x7f4d94004a40 comp rx=0 tx=0).stop 2024-09-16T15:46:57.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.869+0000 7f4da9e00640 1 -- 172.21.15.151:0/3531632938 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4da40b0b30 msgr2=0x7f4da40b0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:57.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.869+0000 7f4da9e00640 1 --2- 172.21.15.151:0/3531632938 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4da40b0b30 0x7f4da40b0f90 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f4da406a510 tx=0x7f4d9c00e980 comp rx=0 tx=0).stop 2024-09-16T15:46:57.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.869+0000 7f4da9e00640 1 -- 172.21.15.151:0/3531632938 shutdown_connections 2024-09-16T15:46:57.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.869+0000 7f4da9e00640 1 --2- 172.21.15.151:0/3531632938 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4d840794e0 0x7f4d8407b9a0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:57.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.869+0000 7f4da9e00640 1 --2- 172.21.15.151:0/3531632938 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4da40c8fd0 0x7f4da40c9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:57.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.869+0000 7f4da9e00640 1 --2- 172.21.15.151:0/3531632938 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4da40b0b30 0x7f4da40b0f90 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:57.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.869+0000 7f4da9e00640 1 --2- 172.21.15.151:0/3531632938 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4d88003a00 0x7f4da414a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:57.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.869+0000 7f4da9e00640 1 -- 172.21.15.151:0/3531632938 >> 172.21.15.151:0/3531632938 conn(0x7f4da40b1950 msgr2=0x7f4da40b3a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:57.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.869+0000 7f4da9e00640 1 -- 172.21.15.151:0/3531632938 shutdown_connections 2024-09-16T15:46:57.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:57.869+0000 7f4da9e00640 1 -- 172.21.15.151:0/3531632938 wait complete. 2024-09-16T15:46:57.887 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":303,"stamp":"2024-09-16T15:46:57.706696+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":176,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":406460,"kb_used_data":8380,"kb_used_omap":12,"kb_used_meta":397875,"kb_avail":754568260,"statfs":{"total":773094113280,"available":772677898240,"internally_reserved":0,"allocated":8581120,"data_stored":4318632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12738,"internal_metadata":407424574},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001312"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.260164+0000","last_change":"2024-09-16T15:46:53.256645+0000","last_active":"2024-09-16T15:46:54.260164+0000","last_peered":"2024-09-16T15:46:54.260164+0000","last_clean":"2024-09-16T15:46:54.260164+0000","last_became_active":"2024-09-16T15:46:53.256398+0000","last_became_peered":"2024-09-16T15:46:53.256398+0000","last_unstale":"2024-09-16T15:46:54.260164+0000","last_undegraded":"2024-09-16T15:46:54.260164+0000","last_fullsized":"2024-09-16T15:46:54.260164+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:21:50.108943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.884470+0000","last_change":"2024-09-16T15:46:53.279028+0000","last_active":"2024-09-16T15:46:54.884470+0000","last_peered":"2024-09-16T15:46:54.884470+0000","last_clean":"2024-09-16T15:46:54.884470+0000","last_became_active":"2024-09-16T15:46:53.278763+0000","last_became_peered":"2024-09-16T15:46:53.278763+0000","last_unstale":"2024-09-16T15:46:54.884470+0000","last_undegraded":"2024-09-16T15:46:54.884470+0000","last_fullsized":"2024-09-16T15:46:54.884470+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:29:26.251286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.264962+0000","last_change":"2024-09-16T15:46:53.288961+0000","last_active":"2024-09-16T15:46:54.264962+0000","last_peered":"2024-09-16T15:46:54.264962+0000","last_clean":"2024-09-16T15:46:54.264962+0000","last_became_active":"2024-09-16T15:46:53.288713+0000","last_became_peered":"2024-09-16T15:46:53.288713+0000","last_unstale":"2024-09-16T15:46:54.264962+0000","last_undegraded":"2024-09-16T15:46:54.264962+0000","last_fullsized":"2024-09-16T15:46:54.264962+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:18:24.768957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.264684+0000","last_change":"2024-09-16T15:46:53.281566+0000","last_active":"2024-09-16T15:46:54.264684+0000","last_peered":"2024-09-16T15:46:54.264684+0000","last_clean":"2024-09-16T15:46:54.264684+0000","last_became_active":"2024-09-16T15:46:53.281086+0000","last_became_peered":"2024-09-16T15:46:53.281086+0000","last_unstale":"2024-09-16T15:46:54.264684+0000","last_undegraded":"2024-09-16T15:46:54.264684+0000","last_fullsized":"2024-09-16T15:46:54.264684+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:00:00.388493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.885440+0000","last_change":"2024-09-16T15:46:53.253408+0000","last_active":"2024-09-16T15:46:54.885440+0000","last_peered":"2024-09-16T15:46:54.885440+0000","last_clean":"2024-09-16T15:46:54.885440+0000","last_became_active":"2024-09-16T15:46:53.252992+0000","last_became_peered":"2024-09-16T15:46:53.252992+0000","last_unstale":"2024-09-16T15:46:54.885440+0000","last_undegraded":"2024-09-16T15:46:54.885440+0000","last_fullsized":"2024-09-16T15:46:54.885440+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:39:01.733297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.883808+0000","last_change":"2024-09-16T15:46:53.267940+0000","last_active":"2024-09-16T15:46:54.883808+0000","last_peered":"2024-09-16T15:46:54.883808+0000","last_clean":"2024-09-16T15:46:54.883808+0000","last_became_active":"2024-09-16T15:46:53.266965+0000","last_became_peered":"2024-09-16T15:46:53.266965+0000","last_unstale":"2024-09-16T15:46:54.883808+0000","last_undegraded":"2024-09-16T15:46:54.883808+0000","last_fullsized":"2024-09-16T15:46:54.883808+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:55:01.762139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.662937+0000","last_change":"2024-09-16T15:46:53.270314+0000","last_active":"2024-09-16T15:46:54.662937+0000","last_peered":"2024-09-16T15:46:54.662937+0000","last_clean":"2024-09-16T15:46:54.662937+0000","last_became_active":"2024-09-16T15:46:53.270028+0000","last_became_peered":"2024-09-16T15:46:53.270028+0000","last_unstale":"2024-09-16T15:46:54.662937+0000","last_undegraded":"2024-09-16T15:46:54.662937+0000","last_fullsized":"2024-09-16T15:46:54.662937+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:40:14.230003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.259445+0000","last_change":"2024-09-16T15:46:53.256166+0000","last_active":"2024-09-16T15:46:54.259445+0000","last_peered":"2024-09-16T15:46:54.259445+0000","last_clean":"2024-09-16T15:46:54.259445+0000","last_became_active":"2024-09-16T15:46:53.255915+0000","last_became_peered":"2024-09-16T15:46:53.255915+0000","last_unstale":"2024-09-16T15:46:54.259445+0000","last_undegraded":"2024-09-16T15:46:54.259445+0000","last_fullsized":"2024-09-16T15:46:54.259445+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:58:02.560350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.265293+0000","last_change":"2024-09-16T15:46:53.267944+0000","last_active":"2024-09-16T15:46:54.265293+0000","last_peered":"2024-09-16T15:46:54.265293+0000","last_clean":"2024-09-16T15:46:54.265293+0000","last_became_active":"2024-09-16T15:46:53.267623+0000","last_became_peered":"2024-09-16T15:46:53.267623+0000","last_unstale":"2024-09-16T15:46:54.265293+0000","last_undegraded":"2024-09-16T15:46:54.265293+0000","last_fullsized":"2024-09-16T15:46:54.265293+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:43:34.363272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.663325+0000","last_change":"2024-09-16T15:46:53.273301+0000","last_active":"2024-09-16T15:46:54.663325+0000","last_peered":"2024-09-16T15:46:54.663325+0000","last_clean":"2024-09-16T15:46:54.663325+0000","last_became_active":"2024-09-16T15:46:53.272943+0000","last_became_peered":"2024-09-16T15:46:53.272943+0000","last_unstale":"2024-09-16T15:46:54.663325+0000","last_undegraded":"2024-09-16T15:46:54.663325+0000","last_fullsized":"2024-09-16T15:46:54.663325+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:19:40.606353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.260065+0000","last_change":"2024-09-16T15:46:53.263681+0000","last_active":"2024-09-16T15:46:54.260065+0000","last_peered":"2024-09-16T15:46:54.260065+0000","last_clean":"2024-09-16T15:46:54.260065+0000","last_became_active":"2024-09-16T15:46:53.262544+0000","last_became_peered":"2024-09-16T15:46:53.262544+0000","last_unstale":"2024-09-16T15:46:54.260065+0000","last_undegraded":"2024-09-16T15:46:54.260065+0000","last_fullsized":"2024-09-16T15:46:54.260065+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:05:19.699888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.885142+0000","last_change":"2024-09-16T15:46:53.270879+0000","last_active":"2024-09-16T15:46:54.885142+0000","last_peered":"2024-09-16T15:46:54.885142+0000","last_clean":"2024-09-16T15:46:54.885142+0000","last_became_active":"2024-09-16T15:46:53.267069+0000","last_became_peered":"2024-09-16T15:46:53.267069+0000","last_unstale":"2024-09-16T15:46:54.885142+0000","last_undegraded":"2024-09-16T15:46:54.885142+0000","last_fullsized":"2024-09-16T15:46:54.885142+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:24:54.767690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.264754+0000","last_change":"2024-09-16T15:46:53.287367+0000","last_active":"2024-09-16T15:46:54.264754+0000","last_peered":"2024-09-16T15:46:54.264754+0000","last_clean":"2024-09-16T15:46:54.264754+0000","last_became_active":"2024-09-16T15:46:53.287061+0000","last_became_peered":"2024-09-16T15:46:53.287061+0000","last_unstale":"2024-09-16T15:46:54.264754+0000","last_undegraded":"2024-09-16T15:46:54.264754+0000","last_fullsized":"2024-09-16T15:46:54.264754+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:21:45.984928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.884994+0000","last_change":"2024-09-16T15:46:53.270658+0000","last_active":"2024-09-16T15:46:54.884994+0000","last_peered":"2024-09-16T15:46:54.884994+0000","last_clean":"2024-09-16T15:46:54.884994+0000","last_became_active":"2024-09-16T15:46:53.270321+0000","last_became_peered":"2024-09-16T15:46:53.270321+0000","last_unstale":"2024-09-16T15:46:54.884994+0000","last_undegraded":"2024-09-16T15:46:54.884994+0000","last_fullsized":"2024-09-16T15:46:54.884994+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:05:39.601833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.884621+0000","last_change":"2024-09-16T15:46:53.256225+0000","last_active":"2024-09-16T15:46:54.884621+0000","last_peered":"2024-09-16T15:46:54.884621+0000","last_clean":"2024-09-16T15:46:54.884621+0000","last_became_active":"2024-09-16T15:46:53.255947+0000","last_became_peered":"2024-09-16T15:46:53.255947+0000","last_unstale":"2024-09-16T15:46:54.884621+0000","last_undegraded":"2024-09-16T15:46:54.884621+0000","last_fullsized":"2024-09-16T15:46:54.884621+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:59:11.693108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.259430+0000","last_change":"2024-09-16T15:46:53.272154+0000","last_active":"2024-09-16T15:46:54.259430+0000","last_peered":"2024-09-16T15:46:54.259430+0000","last_clean":"2024-09-16T15:46:54.259430+0000","last_became_active":"2024-09-16T15:46:53.271930+0000","last_became_peered":"2024-09-16T15:46:53.271930+0000","last_unstale":"2024-09-16T15:46:54.259430+0000","last_undegraded":"2024-09-16T15:46:54.259430+0000","last_fullsized":"2024-09-16T15:46:54.259430+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:00:56.236313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.884494+0000","last_change":"2024-09-16T15:46:53.262615+0000","last_active":"2024-09-16T15:46:54.884494+0000","last_peered":"2024-09-16T15:46:54.884494+0000","last_clean":"2024-09-16T15:46:54.884494+0000","last_became_active":"2024-09-16T15:46:53.262366+0000","last_became_peered":"2024-09-16T15:46:53.262366+0000","last_unstale":"2024-09-16T15:46:54.884494+0000","last_undegraded":"2024-09-16T15:46:54.884494+0000","last_fullsized":"2024-09-16T15:46:54.884494+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:38:25.798380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.885032+0000","last_change":"2024-09-16T15:46:53.278187+0000","last_active":"2024-09-16T15:46:54.885032+0000","last_peered":"2024-09-16T15:46:54.885032+0000","last_clean":"2024-09-16T15:46:54.885032+0000","last_became_active":"2024-09-16T15:46:53.277208+0000","last_became_peered":"2024-09-16T15:46:53.277208+0000","last_unstale":"2024-09-16T15:46:54.885032+0000","last_undegraded":"2024-09-16T15:46:54.885032+0000","last_fullsized":"2024-09-16T15:46:54.885032+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:55:38.471450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.259895+0000","last_change":"2024-09-16T15:46:53.276855+0000","last_active":"2024-09-16T15:46:54.259895+0000","last_peered":"2024-09-16T15:46:54.259895+0000","last_clean":"2024-09-16T15:46:54.259895+0000","last_became_active":"2024-09-16T15:46:53.276583+0000","last_became_peered":"2024-09-16T15:46:53.276583+0000","last_unstale":"2024-09-16T15:46:54.259895+0000","last_undegraded":"2024-09-16T15:46:54.259895+0000","last_fullsized":"2024-09-16T15:46:54.259895+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:59:50.053303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.884239+0000","last_change":"2024-09-16T15:46:53.261889+0000","last_active":"2024-09-16T15:46:54.884239+0000","last_peered":"2024-09-16T15:46:54.884239+0000","last_clean":"2024-09-16T15:46:54.884239+0000","last_became_active":"2024-09-16T15:46:53.261573+0000","last_became_peered":"2024-09-16T15:46:53.261573+0000","last_unstale":"2024-09-16T15:46:54.884239+0000","last_undegraded":"2024-09-16T15:46:54.884239+0000","last_fullsized":"2024-09-16T15:46:54.884239+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:42:27.016749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.884721+0000","last_change":"2024-09-16T15:46:53.259263+0000","last_active":"2024-09-16T15:46:54.884721+0000","last_peered":"2024-09-16T15:46:54.884721+0000","last_clean":"2024-09-16T15:46:54.884721+0000","last_became_active":"2024-09-16T15:46:53.258965+0000","last_became_peered":"2024-09-16T15:46:53.258965+0000","last_unstale":"2024-09-16T15:46:54.884721+0000","last_undegraded":"2024-09-16T15:46:54.884721+0000","last_fullsized":"2024-09-16T15:46:54.884721+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:55:01.762139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.884584+0000","last_change":"2024-09-16T15:46:53.266806+0000","last_active":"2024-09-16T15:46:54.884584+0000","last_peered":"2024-09-16T15:46:54.884584+0000","last_clean":"2024-09-16T15:46:54.884584+0000","last_became_active":"2024-09-16T15:46:53.266328+0000","last_became_peered":"2024-09-16T15:46:53.266328+0000","last_unstale":"2024-09-16T15:46:54.884584+0000","last_undegraded":"2024-09-16T15:46:54.884584+0000","last_fullsized":"2024-09-16T15:46:54.884584+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:26:22.004583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.663119+0000","last_change":"2024-09-16T15:46:53.272023+0000","last_active":"2024-09-16T15:46:54.663119+0000","last_peered":"2024-09-16T15:46:54.663119+0000","last_clean":"2024-09-16T15:46:54.663119+0000","last_became_active":"2024-09-16T15:46:53.271740+0000","last_became_peered":"2024-09-16T15:46:53.271740+0000","last_unstale":"2024-09-16T15:46:54.663119+0000","last_undegraded":"2024-09-16T15:46:54.663119+0000","last_fullsized":"2024-09-16T15:46:54.663119+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:58:59.084881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.259640+0000","last_change":"2024-09-16T15:46:53.264276+0000","last_active":"2024-09-16T15:46:54.259640+0000","last_peered":"2024-09-16T15:46:54.259640+0000","last_clean":"2024-09-16T15:46:54.259640+0000","last_became_active":"2024-09-16T15:46:53.264018+0000","last_became_peered":"2024-09-16T15:46:53.264018+0000","last_unstale":"2024-09-16T15:46:54.259640+0000","last_undegraded":"2024-09-16T15:46:54.259640+0000","last_fullsized":"2024-09-16T15:46:54.259640+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:37.248928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.663270+0000","last_change":"2024-09-16T15:46:53.281930+0000","last_active":"2024-09-16T15:46:54.663270+0000","last_peered":"2024-09-16T15:46:54.663270+0000","last_clean":"2024-09-16T15:46:54.663270+0000","last_became_active":"2024-09-16T15:46:53.280700+0000","last_became_peered":"2024-09-16T15:46:53.280700+0000","last_unstale":"2024-09-16T15:46:54.663270+0000","last_undegraded":"2024-09-16T15:46:54.663270+0000","last_fullsized":"2024-09-16T15:46:54.663270+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:08:45.589883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.259818+0000","last_change":"2024-09-16T15:46:53.277441+0000","last_active":"2024-09-16T15:46:54.259818+0000","last_peered":"2024-09-16T15:46:54.259818+0000","last_clean":"2024-09-16T15:46:54.259818+0000","last_became_active":"2024-09-16T15:46:53.276973+0000","last_became_peered":"2024-09-16T15:46:53.276973+0000","last_unstale":"2024-09-16T15:46:54.259818+0000","last_undegraded":"2024-09-16T15:46:54.259818+0000","last_fullsized":"2024-09-16T15:46:54.259818+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:04:04.493463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.884349+0000","last_change":"2024-09-16T15:46:53.269295+0000","last_active":"2024-09-16T15:46:54.884349+0000","last_peered":"2024-09-16T15:46:54.884349+0000","last_clean":"2024-09-16T15:46:54.884349+0000","last_became_active":"2024-09-16T15:46:53.268868+0000","last_became_peered":"2024-09-16T15:46:53.268868+0000","last_unstale":"2024-09-16T15:46:54.884349+0000","last_undegraded":"2024-09-16T15:46:54.884349+0000","last_fullsized":"2024-09-16T15:46:54.884349+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:28:46.750829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.884316+0000","last_change":"2024-09-16T15:46:53.270463+0000","last_active":"2024-09-16T15:46:54.884316+0000","last_peered":"2024-09-16T15:46:54.884316+0000","last_clean":"2024-09-16T15:46:54.884316+0000","last_became_active":"2024-09-16T15:46:53.270200+0000","last_became_peered":"2024-09-16T15:46:53.270200+0000","last_unstale":"2024-09-16T15:46:54.884316+0000","last_undegraded":"2024-09-16T15:46:54.884316+0000","last_fullsized":"2024-09-16T15:46:54.884316+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:51:39.294710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.264812+0000","last_change":"2024-09-16T15:46:53.268045+0000","last_active":"2024-09-16T15:46:54.264812+0000","last_peered":"2024-09-16T15:46:54.264812+0000","last_clean":"2024-09-16T15:46:54.264812+0000","last_became_active":"2024-09-16T15:46:53.267679+0000","last_became_peered":"2024-09-16T15:46:53.267679+0000","last_unstale":"2024-09-16T15:46:54.264812+0000","last_undegraded":"2024-09-16T15:46:54.264812+0000","last_fullsized":"2024-09-16T15:46:54.264812+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:07:25.144696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.259530+0000","last_change":"2024-09-16T15:46:53.278071+0000","last_active":"2024-09-16T15:46:54.259530+0000","last_peered":"2024-09-16T15:46:54.259530+0000","last_clean":"2024-09-16T15:46:54.259530+0000","last_became_active":"2024-09-16T15:46:53.276119+0000","last_became_peered":"2024-09-16T15:46:53.276119+0000","last_unstale":"2024-09-16T15:46:54.259530+0000","last_undegraded":"2024-09-16T15:46:54.259530+0000","last_fullsized":"2024-09-16T15:46:54.259530+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:13:08.180437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.884026+0000","last_change":"2024-09-16T15:46:53.266336+0000","last_active":"2024-09-16T15:46:54.884026+0000","last_peered":"2024-09-16T15:46:54.884026+0000","last_clean":"2024-09-16T15:46:54.884026+0000","last_became_active":"2024-09-16T15:46:53.266068+0000","last_became_peered":"2024-09-16T15:46:53.266068+0000","last_unstale":"2024-09-16T15:46:54.884026+0000","last_undegraded":"2024-09-16T15:46:54.884026+0000","last_fullsized":"2024-09-16T15:46:54.884026+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:41:40.248733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.883866+0000","last_change":"2024-09-16T15:46:53.270104+0000","last_active":"2024-09-16T15:46:54.883866+0000","last_peered":"2024-09-16T15:46:54.883866+0000","last_clean":"2024-09-16T15:46:54.883866+0000","last_became_active":"2024-09-16T15:46:53.269626+0000","last_became_peered":"2024-09-16T15:46:53.269626+0000","last_unstale":"2024-09-16T15:46:54.883866+0000","last_undegraded":"2024-09-16T15:46:54.883866+0000","last_fullsized":"2024-09-16T15:46:54.883866+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:05:18.468226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.259787+0000","last_change":"2024-09-16T15:46:53.268491+0000","last_active":"2024-09-16T15:46:54.259787+0000","last_peered":"2024-09-16T15:46:54.259787+0000","last_clean":"2024-09-16T15:46:54.259787+0000","last_became_active":"2024-09-16T15:46:53.268160+0000","last_became_peered":"2024-09-16T15:46:53.268160+0000","last_unstale":"2024-09-16T15:46:54.259787+0000","last_undegraded":"2024-09-16T15:46:54.259787+0000","last_fullsized":"2024-09-16T15:46:54.259787+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:14:28.279612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1f","version":"75'4","reported_seq":19,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.884520+0000","last_change":"2024-09-16T15:46:52.262214+0000","last_active":"2024-09-16T15:46:54.884520+0000","last_peered":"2024-09-16T15:46:54.884520+0000","last_clean":"2024-09-16T15:46:54.884520+0000","last_became_active":"2024-09-16T15:46:52.261946+0000","last_became_peered":"2024-09-16T15:46:52.261946+0000","last_unstale":"2024-09-16T15:46:54.884520+0000","last_undegraded":"2024-09-16T15:46:54.884520+0000","last_fullsized":"2024-09-16T15:46:54.884520+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:51.231199+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:27:28.212178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.265451+0000","last_change":"2024-09-16T15:46:53.274983+0000","last_active":"2024-09-16T15:46:54.265451+0000","last_peered":"2024-09-16T15:46:54.265451+0000","last_clean":"2024-09-16T15:46:54.265451+0000","last_became_active":"2024-09-16T15:46:53.274558+0000","last_became_peered":"2024-09-16T15:46:53.274558+0000","last_unstale":"2024-09-16T15:46:54.265451+0000","last_undegraded":"2024-09-16T15:46:54.265451+0000","last_fullsized":"2024-09-16T15:46:54.265451+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:37:10.502831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1e","version":"75'1","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.885016+0000","last_change":"2024-09-16T15:46:52.255885+0000","last_active":"2024-09-16T15:46:54.885016+0000","last_peered":"2024-09-16T15:46:54.885016+0000","last_clean":"2024-09-16T15:46:54.885016+0000","last_became_active":"2024-09-16T15:46:52.255603+0000","last_became_peered":"2024-09-16T15:46:52.255603+0000","last_unstale":"2024-09-16T15:46:54.885016+0000","last_undegraded":"2024-09-16T15:46:54.885016+0000","last_fullsized":"2024-09-16T15:46:54.885016+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:51.231199+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:09:45.131603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.884445+0000","last_change":"2024-09-16T15:46:53.267146+0000","last_active":"2024-09-16T15:46:54.884445+0000","last_peered":"2024-09-16T15:46:54.884445+0000","last_clean":"2024-09-16T15:46:54.884445+0000","last_became_active":"2024-09-16T15:46:53.266886+0000","last_became_peered":"2024-09-16T15:46:53.266886+0000","last_unstale":"2024-09-16T15:46:54.884445+0000","last_undegraded":"2024-09-16T15:46:54.884445+0000","last_fullsized":"2024-09-16T15:46:54.884445+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:00:44.750363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"75'2","reported_seq":17,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.259986+0000","last_change":"2024-09-16T15:46:52.267282+0000","last_active":"2024-09-16T15:46:54.259986+0000","last_peered":"2024-09-16T15:46:54.259986+0000","last_clean":"2024-09-16T15:46:54.259986+0000","last_became_active":"2024-09-16T15:46:52.266684+0000","last_became_peered":"2024-09-16T15:46:52.266684+0000","last_unstale":"2024-09-16T15:46:54.259986+0000","last_undegraded":"2024-09-16T15:46:54.259986+0000","last_fullsized":"2024-09-16T15:46:54.259986+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:51.231199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:04:55.462525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.663083+0000","last_change":"2024-09-16T15:46:53.287593+0000","last_active":"2024-09-16T15:46:54.663083+0000","last_peered":"2024-09-16T15:46:54.663083+0000","last_clean":"2024-09-16T15:46:54.663083+0000","last_became_active":"2024-09-16T15:46:53.280166+0000","last_became_peered":"2024-09-16T15:46:53.280166+0000","last_unstale":"2024-09-16T15:46:54.663083+0000","last_undegraded":"2024-09-16T15:46:54.663083+0000","last_fullsized":"2024-09-16T15:46:54.663083+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:42:32.091554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1c","version":"75'2","reported_seq":17,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.884555+0000","last_change":"2024-09-16T15:46:52.254210+0000","last_active":"2024-09-16T15:46:54.884555+0000","last_peered":"2024-09-16T15:46:54.884555+0000","last_clean":"2024-09-16T15:46:54.884555+0000","last_became_active":"2024-09-16T15:46:52.253955+0000","last_became_peered":"2024-09-16T15:46:52.253955+0000","last_unstale":"2024-09-16T15:46:54.884555+0000","last_undegraded":"2024-09-16T15:46:54.884555+0000","last_fullsized":"2024-09-16T15:46:54.884555+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:51.231199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:05:17.465458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.259617+0000","last_change":"2024-09-16T15:46:53.256264+0000","last_active":"2024-09-16T15:46:54.259617+0000","last_peered":"2024-09-16T15:46:54.259617+0000","last_clean":"2024-09-16T15:46:54.259617+0000","last_became_active":"2024-09-16T15:46:53.256020+0000","last_became_peered":"2024-09-16T15:46:53.256020+0000","last_unstale":"2024-09-16T15:46:54.259617+0000","last_undegraded":"2024-09-16T15:46:54.259617+0000","last_fullsized":"2024-09-16T15:46:54.259617+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:39:01.733297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"75'1","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.884414+0000","last_change":"2024-09-16T15:46:52.264407+0000","last_active":"2024-09-16T15:46:54.884414+0000","last_peered":"2024-09-16T15:46:54.884414+0000","last_clean":"2024-09-16T15:46:54.884414+0000","last_became_active":"2024-09-16T15:46:52.263987+0000","last_became_peered":"2024-09-16T15:46:52.263987+0000","last_unstale":"2024-09-16T15:46:54.884414+0000","last_undegraded":"2024-09-16T15:46:54.884414+0000","last_fullsized":"2024-09-16T15:46:54.884414+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:51.231199+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:16:11.175810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.884239+0000","last_change":"2024-09-16T15:46:53.261256+0000","last_active":"2024-09-16T15:46:54.884239+0000","last_peered":"2024-09-16T15:46:54.884239+0000","last_clean":"2024-09-16T15:46:54.884239+0000","last_became_active":"2024-09-16T15:46:53.260984+0000","last_became_peered":"2024-09-16T15:46:53.260984+0000","last_unstale":"2024-09-16T15:46:54.884239+0000","last_undegraded":"2024-09-16T15:46:54.884239+0000","last_fullsized":"2024-09-16T15:46:54.884239+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:33:04.696187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1a","version":"75'1","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.662818+0000","last_change":"2024-09-16T15:46:52.253535+0000","last_active":"2024-09-16T15:46:54.662818+0000","last_peered":"2024-09-16T15:46:54.662818+0000","last_clean":"2024-09-16T15:46:54.662818+0000","last_became_active":"2024-09-16T15:46:52.253248+0000","last_became_peered":"2024-09-16T15:46:52.253248+0000","last_unstale":"2024-09-16T15:46:54.662818+0000","last_undegraded":"2024-09-16T15:46:54.662818+0000","last_fullsized":"2024-09-16T15:46:54.662818+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:51.231199+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:47:48.953361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.265170+0000","last_change":"2024-09-16T15:46:53.287392+0000","last_active":"2024-09-16T15:46:54.265170+0000","last_peered":"2024-09-16T15:46:54.265170+0000","last_clean":"2024-09-16T15:46:54.265170+0000","last_became_active":"2024-09-16T15:46:53.286966+0000","last_became_peered":"2024-09-16T15:46:53.286966+0000","last_unstale":"2024-09-16T15:46:54.265170+0000","last_undegraded":"2024-09-16T15:46:54.265170+0000","last_fullsized":"2024-09-16T15:46:54.265170+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:47:20.121266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.264174+0000","last_change":"2024-09-16T15:46:52.257570+0000","last_active":"2024-09-16T15:46:54.264174+0000","last_peered":"2024-09-16T15:46:54.264174+0000","last_clean":"2024-09-16T15:46:54.264174+0000","last_became_active":"2024-09-16T15:46:52.257298+0000","last_became_peered":"2024-09-16T15:46:52.257298+0000","last_unstale":"2024-09-16T15:46:54.264174+0000","last_undegraded":"2024-09-16T15:46:54.264174+0000","last_fullsized":"2024-09-16T15:46:54.264174+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:51.231199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:22:55.919466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.884800+0000","last_change":"2024-09-16T15:46:53.281195+0000","last_active":"2024-09-16T15:46:54.884800+0000","last_peered":"2024-09-16T15:46:54.884800+0000","last_clean":"2024-09-16T15:46:54.884800+0000","last_became_active":"2024-09-16T15:46:53.280872+0000","last_became_peered":"2024-09-16T15:46:53.280872+0000","last_unstale":"2024-09-16T15:46:54.884800+0000","last_undegraded":"2024-09-16T15:46:54.884800+0000","last_fullsized":"2024-09-16T15:46:54.884800+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:00:34.585954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.884164+0000","last_change":"2024-09-16T15:46:52.260963+0000","last_active":"2024-09-16T15:46:54.884164+0000","last_peered":"2024-09-16T15:46:54.884164+0000","last_clean":"2024-09-16T15:46:54.884164+0000","last_became_active":"2024-09-16T15:46:52.260675+0000","last_became_peered":"2024-09-16T15:46:52.260675+0000","last_unstale":"2024-09-16T15:46:54.884164+0000","last_undegraded":"2024-09-16T15:46:54.884164+0000","last_fullsized":"2024-09-16T15:46:54.884164+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:51.231199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:38:06.576042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.663342+0000","last_change":"2024-09-16T15:46:53.273560+0000","last_active":"2024-09-16T15:46:54.663342+0000","last_peered":"2024-09-16T15:46:54.663342+0000","last_clean":"2024-09-16T15:46:54.663342+0000","last_became_active":"2024-09-16T15:46:53.269867+0000","last_became_peered":"2024-09-16T15:46:53.269867+0000","last_unstale":"2024-09-16T15:46:54.663342+0000","last_undegraded":"2024-09-16T15:46:54.663342+0000","last_fullsized":"2024-09-16T15:46:54.663342+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:12:51.219648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.259715+0000","last_change":"2024-09-16T15:46:52.259498+0000","last_active":"2024-09-16T15:46:54.259715+0000","last_peered":"2024-09-16T15:46:54.259715+0000","last_clean":"2024-09-16T15:46:54.259715+0000","last_became_active":"2024-09-16T15:46:52.259218+0000","last_became_peered":"2024-09-16T15:46:52.259218+0000","last_unstale":"2024-09-16T15:46:54.259715+0000","last_undegraded":"2024-09-16T15:46:54.259715+0000","last_fullsized":"2024-09-16T15:46:54.259715+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:51.231199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:52:19.148902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.884530+0000","last_change":"2024-09-16T15:46:53.283456+0000","last_active":"2024-09-16T15:46:54.884530+0000","last_peered":"2024-09-16T15:46:54.884530+0000","last_clean":"2024-09-16T15:46:54.884530+0000","last_became_active":"2024-09-16T15:46:53.283224+0000","last_became_peered":"2024-09-16T15:46:53.283224+0000","last_unstale":"2024-09-16T15:46:54.884530+0000","last_undegraded":"2024-09-16T15:46:54.884530+0000","last_fullsized":"2024-09-16T15:46:54.884530+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:27:29.214946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.884571+0000","last_change":"2024-09-16T15:46:52.259506+0000","last_active":"2024-09-16T15:46:54.884571+0000","last_peered":"2024-09-16T15:46:54.884571+0000","last_clean":"2024-09-16T15:46:54.884571+0000","last_became_active":"2024-09-16T15:46:52.258320+0000","last_became_peered":"2024-09-16T15:46:52.258320+0000","last_unstale":"2024-09-16T15:46:54.884571+0000","last_undegraded":"2024-09-16T15:46:54.884571+0000","last_fullsized":"2024-09-16T15:46:54.884571+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:51.231199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:38:06.576042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.259719+0000","last_change":"2024-09-16T15:46:53.259903+0000","last_active":"2024-09-16T15:46:54.259719+0000","last_peered":"2024-09-16T15:46:54.259719+0000","last_clean":"2024-09-16T15:46:54.259719+0000","last_became_active":"2024-09-16T15:46:53.259641+0000","last_became_peered":"2024-09-16T15:46:53.259641+0000","last_unstale":"2024-09-16T15:46:54.259719+0000","last_undegraded":"2024-09-16T15:46:54.259719+0000","last_fullsized":"2024-09-16T15:46:54.259719+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:16:11.478219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"75'3","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.884715+0000","last_change":"2024-09-16T15:46:52.259147+0000","last_active":"2024-09-16T15:46:54.884715+0000","last_peered":"2024-09-16T15:46:54.884715+0000","last_clean":"2024-09-16T15:46:54.884715+0000","last_became_active":"2024-09-16T15:46:52.258628+0000","last_became_peered":"2024-09-16T15:46:52.258628+0000","last_unstale":"2024-09-16T15:46:54.884715+0000","last_undegraded":"2024-09-16T15:46:54.884715+0000","last_fullsized":"2024-09-16T15:46:54.884715+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:51.231199+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:33:33.300951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.884526+0000","last_change":"2024-09-16T15:46:53.269132+0000","last_active":"2024-09-16T15:46:54.884526+0000","last_peered":"2024-09-16T15:46:54.884526+0000","last_clean":"2024-09-16T15:46:54.884526+0000","last_became_active":"2024-09-16T15:46:53.268844+0000","last_became_peered":"2024-09-16T15:46:53.268844+0000","last_unstale":"2024-09-16T15:46:54.884526+0000","last_undegraded":"2024-09-16T15:46:54.884526+0000","last_fullsized":"2024-09-16T15:46:54.884526+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:21:58.823601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.14","version":"75'6","reported_seq":21,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.264475+0000","last_change":"2024-09-16T15:46:52.254862+0000","last_active":"2024-09-16T15:46:54.264475+0000","last_peered":"2024-09-16T15:46:54.264475+0000","last_clean":"2024-09-16T15:46:54.264475+0000","last_became_active":"2024-09-16T15:46:52.254571+0000","last_became_peered":"2024-09-16T15:46:52.254571+0000","last_unstale":"2024-09-16T15:46:54.264475+0000","last_undegraded":"2024-09-16T15:46:54.264475+0000","last_fullsized":"2024-09-16T15:46:54.264475+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:51.231199+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:51:35.113471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.884646+0000","last_change":"2024-09-16T15:46:53.257379+0000","last_active":"2024-09-16T15:46:54.884646+0000","last_peered":"2024-09-16T15:46:54.884646+0000","last_clean":"2024-09-16T15:46:54.884646+0000","last_became_active":"2024-09-16T15:46:53.257106+0000","last_became_peered":"2024-09-16T15:46:53.257106+0000","last_unstale":"2024-09-16T15:46:54.884646+0000","last_undegraded":"2024-09-16T15:46:54.884646+0000","last_fullsized":"2024-09-16T15:46:54.884646+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:33:48.960191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.13","version":"75'2","reported_seq":17,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.264919+0000","last_change":"2024-09-16T15:46:52.264968+0000","last_active":"2024-09-16T15:46:54.264919+0000","last_peered":"2024-09-16T15:46:54.264919+0000","last_clean":"2024-09-16T15:46:54.264919+0000","last_became_active":"2024-09-16T15:46:52.264657+0000","last_became_peered":"2024-09-16T15:46:52.264657+0000","last_unstale":"2024-09-16T15:46:54.264919+0000","last_undegraded":"2024-09-16T15:46:54.264919+0000","last_fullsized":"2024-09-16T15:46:54.264919+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:51.231199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:04:37.641360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.663022+0000","last_change":"2024-09-16T15:46:53.285012+0000","last_active":"2024-09-16T15:46:54.663022+0000","last_peered":"2024-09-16T15:46:54.663022+0000","last_clean":"2024-09-16T15:46:54.663022+0000","last_became_active":"2024-09-16T15:46:53.284699+0000","last_became_peered":"2024-09-16T15:46:53.284699+0000","last_unstale":"2024-09-16T15:46:54.663022+0000","last_undegraded":"2024-09-16T15:46:54.663022+0000","last_fullsized":"2024-09-16T15:46:54.663022+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:39:03.605336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.662826+0000","last_change":"2024-09-16T15:46:52.250054+0000","last_active":"2024-09-16T15:46:54.662826+0000","last_peered":"2024-09-16T15:46:54.662826+0000","last_clean":"2024-09-16T15:46:54.662826+0000","last_became_active":"2024-09-16T15:46:52.249770+0000","last_became_peered":"2024-09-16T15:46:52.249770+0000","last_unstale":"2024-09-16T15:46:54.662826+0000","last_undegraded":"2024-09-16T15:46:54.662826+0000","last_fullsized":"2024-09-16T15:46:54.662826+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:51.231199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:35:29.994510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.884180+0000","last_change":"2024-09-16T15:46:53.271789+0000","last_active":"2024-09-16T15:46:54.884180+0000","last_peered":"2024-09-16T15:46:54.884180+0000","last_clean":"2024-09-16T15:46:54.884180+0000","last_became_active":"2024-09-16T15:46:53.271503+0000","last_became_peered":"2024-09-16T15:46:53.271503+0000","last_unstale":"2024-09-16T15:46:54.884180+0000","last_undegraded":"2024-09-16T15:46:54.884180+0000","last_fullsized":"2024-09-16T15:46:54.884180+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:02:19.576713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.663066+0000","last_change":"2024-09-16T15:46:52.259989+0000","last_active":"2024-09-16T15:46:54.663066+0000","last_peered":"2024-09-16T15:46:54.663066+0000","last_clean":"2024-09-16T15:46:54.663066+0000","last_became_active":"2024-09-16T15:46:52.259740+0000","last_became_peered":"2024-09-16T15:46:52.259740+0000","last_unstale":"2024-09-16T15:46:54.663066+0000","last_undegraded":"2024-09-16T15:46:54.663066+0000","last_fullsized":"2024-09-16T15:46:54.663066+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:51.231199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:11:37.836767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.264245+0000","last_change":"2024-09-16T15:46:53.272955+0000","last_active":"2024-09-16T15:46:54.264245+0000","last_peered":"2024-09-16T15:46:54.264245+0000","last_clean":"2024-09-16T15:46:54.264245+0000","last_became_active":"2024-09-16T15:46:53.272155+0000","last_became_peered":"2024-09-16T15:46:53.272155+0000","last_unstale":"2024-09-16T15:46:54.264245+0000","last_undegraded":"2024-09-16T15:46:54.264245+0000","last_fullsized":"2024-09-16T15:46:54.264245+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:50:56.316541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.10","version":"75'8","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.662680+0000","last_change":"2024-09-16T15:46:52.256572+0000","last_active":"2024-09-16T15:46:54.662680+0000","last_peered":"2024-09-16T15:46:54.662680+0000","last_clean":"2024-09-16T15:46:54.662680+0000","last_became_active":"2024-09-16T15:46:52.255234+0000","last_became_peered":"2024-09-16T15:46:52.255234+0000","last_unstale":"2024-09-16T15:46:54.662680+0000","last_undegraded":"2024-09-16T15:46:54.662680+0000","last_fullsized":"2024-09-16T15:46:54.662680+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:51.231199+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:25:12.971034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.884507+0000","last_change":"2024-09-16T15:46:53.253256+0000","last_active":"2024-09-16T15:46:54.884507+0000","last_peered":"2024-09-16T15:46:54.884507+0000","last_clean":"2024-09-16T15:46:54.884507+0000","last_became_active":"2024-09-16T15:46:53.252459+0000","last_became_peered":"2024-09-16T15:46:53.252459+0000","last_unstale":"2024-09-16T15:46:54.884507+0000","last_undegraded":"2024-09-16T15:46:54.884507+0000","last_fullsized":"2024-09-16T15:46:54.884507+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:21:06.087592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.f","version":"75'1","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.259816+0000","last_change":"2024-09-16T15:46:52.265548+0000","last_active":"2024-09-16T15:46:54.259816+0000","last_peered":"2024-09-16T15:46:54.259816+0000","last_clean":"2024-09-16T15:46:54.259816+0000","last_became_active":"2024-09-16T15:46:52.265310+0000","last_became_peered":"2024-09-16T15:46:52.265310+0000","last_unstale":"2024-09-16T15:46:54.259816+0000","last_undegraded":"2024-09-16T15:46:54.259816+0000","last_fullsized":"2024-09-16T15:46:54.259816+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:51.231199+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:22:54.967969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.884716+0000","last_change":"2024-09-16T15:46:53.262451+0000","last_active":"2024-09-16T15:46:54.884716+0000","last_peered":"2024-09-16T15:46:54.884716+0000","last_clean":"2024-09-16T15:46:54.884716+0000","last_became_active":"2024-09-16T15:46:53.261650+0000","last_became_peered":"2024-09-16T15:46:53.261650+0000","last_unstale":"2024-09-16T15:46:54.884716+0000","last_undegraded":"2024-09-16T15:46:54.884716+0000","last_fullsized":"2024-09-16T15:46:54.884716+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:35:33.260676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"75'2","reported_seq":17,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.884624+0000","last_change":"2024-09-16T15:46:52.261512+0000","last_active":"2024-09-16T15:46:54.884624+0000","last_peered":"2024-09-16T15:46:54.884624+0000","last_clean":"2024-09-16T15:46:54.884624+0000","last_became_active":"2024-09-16T15:46:52.261247+0000","last_became_peered":"2024-09-16T15:46:52.261247+0000","last_unstale":"2024-09-16T15:46:54.884624+0000","last_undegraded":"2024-09-16T15:46:54.884624+0000","last_fullsized":"2024-09-16T15:46:54.884624+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:51.231199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:38:05.193309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.264936+0000","last_change":"2024-09-16T15:46:53.267133+0000","last_active":"2024-09-16T15:46:54.264936+0000","last_peered":"2024-09-16T15:46:54.264936+0000","last_clean":"2024-09-16T15:46:54.264936+0000","last_became_active":"2024-09-16T15:46:53.266542+0000","last_became_peered":"2024-09-16T15:46:53.266542+0000","last_unstale":"2024-09-16T15:46:54.264936+0000","last_undegraded":"2024-09-16T15:46:54.264936+0000","last_fullsized":"2024-09-16T15:46:54.264936+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:48:52.607177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"75'1","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.884132+0000","last_change":"2024-09-16T15:46:52.265326+0000","last_active":"2024-09-16T15:46:54.884132+0000","last_peered":"2024-09-16T15:46:54.884132+0000","last_clean":"2024-09-16T15:46:54.884132+0000","last_became_active":"2024-09-16T15:46:52.265050+0000","last_became_peered":"2024-09-16T15:46:52.265050+0000","last_unstale":"2024-09-16T15:46:54.884132+0000","last_undegraded":"2024-09-16T15:46:54.884132+0000","last_fullsized":"2024-09-16T15:46:54.884132+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:51.231199+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:16:11.175810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.265091+0000","last_change":"2024-09-16T15:46:53.272941+0000","last_active":"2024-09-16T15:46:54.265091+0000","last_peered":"2024-09-16T15:46:54.265091+0000","last_clean":"2024-09-16T15:46:54.265091+0000","last_became_active":"2024-09-16T15:46:53.272641+0000","last_became_peered":"2024-09-16T15:46:53.272641+0000","last_unstale":"2024-09-16T15:46:54.265091+0000","last_undegraded":"2024-09-16T15:46:54.265091+0000","last_fullsized":"2024-09-16T15:46:54.265091+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:04:35.594627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.884641+0000","last_change":"2024-09-16T15:46:52.268492+0000","last_active":"2024-09-16T15:46:54.884641+0000","last_peered":"2024-09-16T15:46:54.884641+0000","last_clean":"2024-09-16T15:46:54.884641+0000","last_became_active":"2024-09-16T15:46:52.268165+0000","last_became_peered":"2024-09-16T15:46:52.268165+0000","last_unstale":"2024-09-16T15:46:54.884641+0000","last_undegraded":"2024-09-16T15:46:54.884641+0000","last_fullsized":"2024-09-16T15:46:54.884641+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:51.231199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:41:39.245965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.265027+0000","last_change":"2024-09-16T15:46:53.264228+0000","last_active":"2024-09-16T15:46:54.265027+0000","last_peered":"2024-09-16T15:46:54.265027+0000","last_clean":"2024-09-16T15:46:54.265027+0000","last_became_active":"2024-09-16T15:46:53.263945+0000","last_became_peered":"2024-09-16T15:46:53.263945+0000","last_unstale":"2024-09-16T15:46:54.265027+0000","last_undegraded":"2024-09-16T15:46:54.265027+0000","last_fullsized":"2024-09-16T15:46:54.265027+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:31:28.192113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.884776+0000","last_change":"2024-09-16T15:46:52.263733+0000","last_active":"2024-09-16T15:46:54.884776+0000","last_peered":"2024-09-16T15:46:54.884776+0000","last_clean":"2024-09-16T15:46:54.884776+0000","last_became_active":"2024-09-16T15:46:52.263063+0000","last_became_peered":"2024-09-16T15:46:52.263063+0000","last_unstale":"2024-09-16T15:46:54.884776+0000","last_undegraded":"2024-09-16T15:46:54.884776+0000","last_fullsized":"2024-09-16T15:46:54.884776+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:51.231199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:28:01.953944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.265136+0000","last_change":"2024-09-16T15:46:53.268810+0000","last_active":"2024-09-16T15:46:54.265136+0000","last_peered":"2024-09-16T15:46:54.265136+0000","last_clean":"2024-09-16T15:46:54.265136+0000","last_became_active":"2024-09-16T15:46:53.267400+0000","last_became_peered":"2024-09-16T15:46:53.267400+0000","last_unstale":"2024-09-16T15:46:54.265136+0000","last_undegraded":"2024-09-16T15:46:54.265136+0000","last_fullsized":"2024-09-16T15:46:54.265136+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:25:45.675088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.884036+0000","last_change":"2024-09-16T15:46:52.259684+0000","last_active":"2024-09-16T15:46:54.884036+0000","last_peered":"2024-09-16T15:46:54.884036+0000","last_clean":"2024-09-16T15:46:54.884036+0000","last_became_active":"2024-09-16T15:46:52.259378+0000","last_became_peered":"2024-09-16T15:46:52.259378+0000","last_unstale":"2024-09-16T15:46:54.884036+0000","last_undegraded":"2024-09-16T15:46:54.884036+0000","last_fullsized":"2024-09-16T15:46:54.884036+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:51.231199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:24:27.876321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.885237+0000","last_change":"2024-09-16T15:46:53.260439+0000","last_active":"2024-09-16T15:46:54.885237+0000","last_peered":"2024-09-16T15:46:54.885237+0000","last_clean":"2024-09-16T15:46:54.885237+0000","last_became_active":"2024-09-16T15:46:53.260129+0000","last_became_peered":"2024-09-16T15:46:53.260129+0000","last_unstale":"2024-09-16T15:46:54.885237+0000","last_undegraded":"2024-09-16T15:46:54.885237+0000","last_fullsized":"2024-09-16T15:46:54.885237+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:56:33.813088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.883877+0000","last_change":"2024-09-16T15:46:52.256920+0000","last_active":"2024-09-16T15:46:54.883877+0000","last_peered":"2024-09-16T15:46:54.883877+0000","last_clean":"2024-09-16T15:46:54.883877+0000","last_became_active":"2024-09-16T15:46:52.256648+0000","last_became_peered":"2024-09-16T15:46:52.256648+0000","last_unstale":"2024-09-16T15:46:54.883877+0000","last_undegraded":"2024-09-16T15:46:54.883877+0000","last_fullsized":"2024-09-16T15:46:54.883877+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:51.231199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:48:41.114324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.884890+0000","last_change":"2024-09-16T15:46:53.267324+0000","last_active":"2024-09-16T15:46:54.884890+0000","last_peered":"2024-09-16T15:46:54.884890+0000","last_clean":"2024-09-16T15:46:54.884890+0000","last_became_active":"2024-09-16T15:46:53.267059+0000","last_became_peered":"2024-09-16T15:46:53.267059+0000","last_unstale":"2024-09-16T15:46:54.884890+0000","last_undegraded":"2024-09-16T15:46:54.884890+0000","last_fullsized":"2024-09-16T15:46:54.884890+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:00:56.236313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"75'2","reported_seq":17,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.260137+0000","last_change":"2024-09-16T15:46:52.265143+0000","last_active":"2024-09-16T15:46:54.260137+0000","last_peered":"2024-09-16T15:46:54.260137+0000","last_clean":"2024-09-16T15:46:54.260137+0000","last_became_active":"2024-09-16T15:46:52.264776+0000","last_became_peered":"2024-09-16T15:46:52.264776+0000","last_unstale":"2024-09-16T15:46:54.260137+0000","last_undegraded":"2024-09-16T15:46:54.260137+0000","last_fullsized":"2024-09-16T15:46:54.260137+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:51.231199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:41:33.466357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.663347+0000","last_change":"2024-09-16T15:46:53.281127+0000","last_active":"2024-09-16T15:46:54.663347+0000","last_peered":"2024-09-16T15:46:54.663347+0000","last_clean":"2024-09-16T15:46:54.663347+0000","last_became_active":"2024-09-16T15:46:53.280080+0000","last_became_peered":"2024-09-16T15:46:53.280080+0000","last_unstale":"2024-09-16T15:46:54.663347+0000","last_undegraded":"2024-09-16T15:46:54.663347+0000","last_fullsized":"2024-09-16T15:46:54.663347+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:10:17.965103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.6","version":"76'1","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:55.254676+0000","last_change":"2024-09-16T15:46:52.264764+0000","last_active":"2024-09-16T15:46:55.254676+0000","last_peered":"2024-09-16T15:46:55.254676+0000","last_clean":"2024-09-16T15:46:55.254676+0000","last_became_active":"2024-09-16T15:46:52.264283+0000","last_became_peered":"2024-09-16T15:46:52.264283+0000","last_unstale":"2024-09-16T15:46:55.254676+0000","last_undegraded":"2024-09-16T15:46:55.254676+0000","last_fullsized":"2024-09-16T15:46:55.254676+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:51.231199+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:29:44.764145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.884762+0000","last_change":"2024-09-16T15:46:53.270443+0000","last_active":"2024-09-16T15:46:54.884762+0000","last_peered":"2024-09-16T15:46:54.884762+0000","last_clean":"2024-09-16T15:46:54.884762+0000","last_became_active":"2024-09-16T15:46:53.269557+0000","last_became_peered":"2024-09-16T15:46:53.269557+0000","last_unstale":"2024-09-16T15:46:54.884762+0000","last_undegraded":"2024-09-16T15:46:54.884762+0000","last_fullsized":"2024-09-16T15:46:54.884762+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:55:44.157321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.884873+0000","last_change":"2024-09-16T15:46:52.261813+0000","last_active":"2024-09-16T15:46:54.884873+0000","last_peered":"2024-09-16T15:46:54.884873+0000","last_clean":"2024-09-16T15:46:54.884873+0000","last_became_active":"2024-09-16T15:46:52.261474+0000","last_became_peered":"2024-09-16T15:46:52.261474+0000","last_unstale":"2024-09-16T15:46:54.884873+0000","last_undegraded":"2024-09-16T15:46:54.884873+0000","last_fullsized":"2024-09-16T15:46:54.884873+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:51.231199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:51:39.260892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.663256+0000","last_change":"2024-09-16T15:46:53.288339+0000","last_active":"2024-09-16T15:46:54.663256+0000","last_peered":"2024-09-16T15:46:54.663256+0000","last_clean":"2024-09-16T15:46:54.663256+0000","last_became_active":"2024-09-16T15:46:53.280252+0000","last_became_peered":"2024-09-16T15:46:53.280252+0000","last_unstale":"2024-09-16T15:46:54.663256+0000","last_undegraded":"2024-09-16T15:46:54.663256+0000","last_fullsized":"2024-09-16T15:46:54.663256+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:21:19.572835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.885245+0000","last_change":"2024-09-16T15:46:52.260059+0000","last_active":"2024-09-16T15:46:54.885245+0000","last_peered":"2024-09-16T15:46:54.885245+0000","last_clean":"2024-09-16T15:46:54.885245+0000","last_became_active":"2024-09-16T15:46:52.259765+0000","last_became_peered":"2024-09-16T15:46:52.259765+0000","last_unstale":"2024-09-16T15:46:54.885245+0000","last_undegraded":"2024-09-16T15:46:54.885245+0000","last_fullsized":"2024-09-16T15:46:54.885245+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:51.231199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:38:05.193309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.885033+0000","last_change":"2024-09-16T15:46:53.259138+0000","last_active":"2024-09-16T15:46:54.885033+0000","last_peered":"2024-09-16T15:46:54.885033+0000","last_clean":"2024-09-16T15:46:54.885033+0000","last_became_active":"2024-09-16T15:46:53.258287+0000","last_became_peered":"2024-09-16T15:46:53.258287+0000","last_unstale":"2024-09-16T15:46:54.885033+0000","last_undegraded":"2024-09-16T15:46:54.885033+0000","last_fullsized":"2024-09-16T15:46:54.885033+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:18:37.720836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.884537+0000","last_change":"2024-09-16T15:46:52.262768+0000","last_active":"2024-09-16T15:46:54.884537+0000","last_peered":"2024-09-16T15:46:54.884537+0000","last_clean":"2024-09-16T15:46:54.884537+0000","last_became_active":"2024-09-16T15:46:52.262448+0000","last_became_peered":"2024-09-16T15:46:52.262448+0000","last_unstale":"2024-09-16T15:46:54.884537+0000","last_undegraded":"2024-09-16T15:46:54.884537+0000","last_fullsized":"2024-09-16T15:46:54.884537+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:51.231199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:54:55.276950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.259991+0000","last_change":"2024-09-16T15:46:53.270630+0000","last_active":"2024-09-16T15:46:54.259991+0000","last_peered":"2024-09-16T15:46:54.259991+0000","last_clean":"2024-09-16T15:46:54.259991+0000","last_became_active":"2024-09-16T15:46:53.270249+0000","last_became_peered":"2024-09-16T15:46:53.270249+0000","last_unstale":"2024-09-16T15:46:54.259991+0000","last_undegraded":"2024-09-16T15:46:54.259991+0000","last_fullsized":"2024-09-16T15:46:54.259991+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:14:47.056019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.885141+0000","last_change":"2024-09-16T15:46:52.262673+0000","last_active":"2024-09-16T15:46:54.885141+0000","last_peered":"2024-09-16T15:46:54.885141+0000","last_clean":"2024-09-16T15:46:54.885141+0000","last_became_active":"2024-09-16T15:46:52.262399+0000","last_became_peered":"2024-09-16T15:46:52.262399+0000","last_unstale":"2024-09-16T15:46:54.885141+0000","last_undegraded":"2024-09-16T15:46:54.885141+0000","last_fullsized":"2024-09-16T15:46:54.885141+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:51.231199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:00:08.401802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.265281+0000","last_change":"2024-09-16T15:46:53.264432+0000","last_active":"2024-09-16T15:46:54.265281+0000","last_peered":"2024-09-16T15:46:54.265281+0000","last_clean":"2024-09-16T15:46:54.265281+0000","last_became_active":"2024-09-16T15:46:53.264128+0000","last_became_peered":"2024-09-16T15:46:53.264128+0000","last_unstale":"2024-09-16T15:46:54.265281+0000","last_undegraded":"2024-09-16T15:46:54.265281+0000","last_fullsized":"2024-09-16T15:46:54.265281+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:07:03.130611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.884343+0000","last_change":"2024-09-16T15:46:52.258569+0000","last_active":"2024-09-16T15:46:54.884343+0000","last_peered":"2024-09-16T15:46:54.884343+0000","last_clean":"2024-09-16T15:46:54.884343+0000","last_became_active":"2024-09-16T15:46:52.258290+0000","last_became_peered":"2024-09-16T15:46:52.258290+0000","last_unstale":"2024-09-16T15:46:54.884343+0000","last_undegraded":"2024-09-16T15:46:54.884343+0000","last_fullsized":"2024-09-16T15:46:54.884343+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:51.231199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:54:47.895621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.883844+0000","last_change":"2024-09-16T15:46:53.281050+0000","last_active":"2024-09-16T15:46:54.883844+0000","last_peered":"2024-09-16T15:46:54.883844+0000","last_clean":"2024-09-16T15:46:54.883844+0000","last_became_active":"2024-09-16T15:46:53.280774+0000","last_became_peered":"2024-09-16T15:46:53.280774+0000","last_unstale":"2024-09-16T15:46:54.883844+0000","last_undegraded":"2024-09-16T15:46:54.883844+0000","last_fullsized":"2024-09-16T15:46:54.883844+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:16:58.103248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.883843+0000","last_change":"2024-09-16T15:46:52.254772+0000","last_active":"2024-09-16T15:46:54.883843+0000","last_peered":"2024-09-16T15:46:54.883843+0000","last_clean":"2024-09-16T15:46:54.883843+0000","last_became_active":"2024-09-16T15:46:52.254509+0000","last_became_peered":"2024-09-16T15:46:52.254509+0000","last_unstale":"2024-09-16T15:46:54.883843+0000","last_undegraded":"2024-09-16T15:46:54.883843+0000","last_fullsized":"2024-09-16T15:46:54.883843+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:51.231199+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:52:43.813433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.885513+0000","last_change":"2024-09-16T15:46:53.263458+0000","last_active":"2024-09-16T15:46:54.885513+0000","last_peered":"2024-09-16T15:46:54.885513+0000","last_clean":"2024-09-16T15:46:54.885513+0000","last_became_active":"2024-09-16T15:46:53.263177+0000","last_became_peered":"2024-09-16T15:46:53.263177+0000","last_unstale":"2024-09-16T15:46:54.885513+0000","last_undegraded":"2024-09-16T15:46:54.885513+0000","last_fullsized":"2024-09-16T15:46:54.885513+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:52.233967+0000","last_clean_scrub_stamp":"2024-09-16T15:46:52.233967+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:37.248928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"75'2","reported_seq":17,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.662969+0000","last_change":"2024-09-16T15:46:52.259014+0000","last_active":"2024-09-16T15:46:54.662969+0000","last_peered":"2024-09-16T15:46:54.662969+0000","last_clean":"2024-09-16T15:46:54.662969+0000","last_became_active":"2024-09-16T15:46:52.258730+0000","last_became_peered":"2024-09-16T15:46:52.258730+0000","last_unstale":"2024-09-16T15:46:54.662969+0000","last_undegraded":"2024-09-16T15:46:54.662969+0000","last_fullsized":"2024-09-16T15:46:54.662969+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:46:51.231199+0000","last_clean_scrub_stamp":"2024-09-16T15:46:51.231199+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:22:09.077141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":213,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-16T15:46:54.662669+0000","last_change":"2024-09-16T15:39:12.698813+0000","last_active":"2024-09-16T15:46:54.662669+0000","last_peered":"2024-09-16T15:46:54.662669+0000","last_clean":"2024-09-16T15:46:54.662669+0000","last_became_active":"2024-09-16T15:39:12.697695+0000","last_became_peered":"2024-09-16T15:39:12.697695+0000","last_unstale":"2024-09-16T15:46:54.662669+0000","last_undegraded":"2024-09-16T15:46:54.662669+0000","last_fullsized":"2024-09-16T15:46:54.662669+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_clean_scrub_stamp":"2024-09-16T15:39:11.684967+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:02.061081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509538,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50152,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321688,"statfs":{"total":96636764160,"available":96585408512,"internally_reserved":0,"allocated":933888,"data_stored":394075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":50395592},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509537,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54880,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316960,"statfs":{"total":96636764160,"available":96580567040,"internally_reserved":0,"allocated":1515520,"data_stored":982887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509536,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54304,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317536,"statfs":{"total":96636764160,"available":96581156864,"internally_reserved":0,"allocated":929792,"data_stored":393123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509536,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50144,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321696,"statfs":{"total":96636764160,"available":96585416704,"internally_reserved":0,"allocated":929792,"data_stored":392555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":50395593},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509536,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50140,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321700,"statfs":{"total":96636764160,"available":96585420800,"internally_reserved":0,"allocated":921600,"data_stored":393011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509537,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46560,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325280,"statfs":{"total":96636764160,"available":96589086720,"internally_reserved":0,"allocated":1511424,"data_stored":982797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509535,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50144,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321696,"statfs":{"total":96636764160,"available":96585416704,"internally_reserved":0,"allocated":921600,"data_stored":392499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542242,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50136,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321704,"statfs":{"total":96636764160,"available":96585424896,"internally_reserved":0,"allocated":917504,"data_stored":387685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":50395596},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:46:57.891 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T15:46:57.892 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-16T15:46:58.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.093+0000 7fac5fa00640 1 -- 172.21.15.151:0/2513726825 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac3c003c10 msgr2=0x7fac3c004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:58.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.093+0000 7fac5fa00640 1 --2- 172.21.15.151:0/2513726825 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac3c003c10 0x7fac3c004010 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fac48009b50 tx=0x7fac3c005830 comp rx=0 tx=0).stop 2024-09-16T15:46:58.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.093+0000 7fac5da00640 1 -- 172.21.15.151:0/2513726825 reap_dead start 2024-09-16T15:46:58.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.093+0000 7fac5fa00640 1 -- 172.21.15.151:0/2513726825 shutdown_connections 2024-09-16T15:46:58.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.093+0000 7fac5fa00640 1 -- 172.21.15.151:0/2513726825 >> 172.21.15.151:0/2513726825 conn(0x7fac580b1840 msgr2=0x7fac580b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:58.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.093+0000 7fac5fa00640 1 -- 172.21.15.151:0/2513726825 shutdown_connections 2024-09-16T15:46:58.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.093+0000 7fac5fa00640 1 -- 172.21.15.151:0/2513726825 wait complete. 2024-09-16T15:46:58.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.095+0000 7fac5fa00640 1 Processor -- start 2024-09-16T15:46:58.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.095+0000 7fac5fa00640 1 -- start start 2024-09-16T15:46:58.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.096+0000 7fac5fa00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac3c003c10 0x7fac5813db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:58.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.096+0000 7fac5fa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac5813f330 0x7fac581f69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:58.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.096+0000 7fac5fa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac5813bcd0 0x7fac580ad740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:58.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.096+0000 7fac5fa00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fac580737d0 con 0x7fac5813bcd0 2024-09-16T15:46:58.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.096+0000 7fac5fa00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fac58073650 con 0x7fac5813f330 2024-09-16T15:46:58.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.096+0000 7fac5fa00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fac580ce2c0 con 0x7fac3c003c10 2024-09-16T15:46:58.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.096+0000 7fac5da00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac5813bcd0 0x7fac580ad740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:58.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.096+0000 7fac5da00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac5813bcd0 0x7fac580ad740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:46152/0 (socket says 172.21.15.151:46152) 2024-09-16T15:46:58.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.096+0000 7fac5d000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac3c003c10 0x7fac5813db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:58.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.096+0000 7fac5da00640 1 -- 172.21.15.151:0/3107750382 learned_addr learned my addr 172.21.15.151:0/3107750382 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:46:58.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.096+0000 7fac4fe00640 1 --2- 172.21.15.151:0/3107750382 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac5813f330 0x7fac581f69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:58.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.096+0000 7fac5da00640 1 -- 172.21.15.151:0/3107750382 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac3c003c10 msgr2=0x7fac5813db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:58.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.096+0000 7fac5da00640 1 --2- 172.21.15.151:0/3107750382 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac3c003c10 0x7fac5813db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:58.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.096+0000 7fac5da00640 1 -- 172.21.15.151:0/3107750382 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac5813f330 msgr2=0x7fac581f69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:58.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.096+0000 7fac5da00640 1 --2- 172.21.15.151:0/3107750382 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac5813f330 0x7fac581f69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:58.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.096+0000 7fac5da00640 1 -- 172.21.15.151:0/3107750382 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac5813e290 con 0x7fac5813bcd0 2024-09-16T15:46:58.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.097+0000 7fac5da00640 1 --2- 172.21.15.151:0/3107750382 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac5813bcd0 0x7fac580ad740 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fac50002230 tx=0x7fac5000eb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:58.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.097+0000 7fac55600640 1 -- 172.21.15.151:0/3107750382 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac5000c970 con 0x7fac5813bcd0 2024-09-16T15:46:58.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.097+0000 7fac55600640 1 -- 172.21.15.151:0/3107750382 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fac50018070 con 0x7fac5813bcd0 2024-09-16T15:46:58.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.097+0000 7fac55600640 1 -- 172.21.15.151:0/3107750382 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac5000ce80 con 0x7fac5813bcd0 2024-09-16T15:46:58.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.097+0000 7fac5fa00640 1 -- 172.21.15.151:0/3107750382 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac5813e570 con 0x7fac5813bcd0 2024-09-16T15:46:58.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.097+0000 7fac5fa00640 1 -- 172.21.15.151:0/3107750382 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac5813eb30 con 0x7fac5813bcd0 2024-09-16T15:46:58.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.098+0000 7fac5fa00640 1 -- 172.21.15.151:0/3107750382 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac580cc5c0 con 0x7fac5813bcd0 2024-09-16T15:46:58.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.099+0000 7fac55600640 1 -- 172.21.15.151:0/3107750382 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 24) ==== 102409+0+0 (secure 0 0 0) 0x7fac5001d460 con 0x7fac5813bcd0 2024-09-16T15:46:58.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.101+0000 7fac55600640 1 --2- 172.21.15.151:0/3107750382 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fac340794b0 0x7fac3407b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:58.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.101+0000 7fac55600640 1 -- 172.21.15.151:0/3107750382 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5765+0+0 (secure 0 0 0) 0x7fac50099620 con 0x7fac5813bcd0 2024-09-16T15:46:58.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.101+0000 7fac5d000640 1 --2- 172.21.15.151:0/3107750382 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fac340794b0 0x7fac3407b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:58.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.102+0000 7fac5d000640 1 --2- 172.21.15.151:0/3107750382 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fac340794b0 0x7fac3407b970 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fac48035000 tx=0x7fac4802d040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:58.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.105+0000 7fac55600640 1 -- 172.21.15.151:0/3107750382 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fac50063090 con 0x7fac5813bcd0 2024-09-16T15:46:58.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.359+0000 7fac5fa00640 1 -- 172.21.15.151:0/3107750382 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fac58143990 con 0x7fac5813bcd0 2024-09-16T15:46:58.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.364+0000 7fac55600640 1 -- 172.21.15.151:0/3107750382 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) ==== 190+0+147 (secure 0 0 0) 0x7fac50067ac0 con 0x7fac5813bcd0 2024-09-16T15:46:58.366 INFO:teuthology.orchestra.run.smithi151.stdout:[client.0] 2024-09-16T15:46:58.366 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQDsUOhmf5PBKBAAKE4Y4zbA4h07IBNjPkyptw== 2024-09-16T15:46:58.366 INFO:teuthology.orchestra.run.smithi151.stdout: caps mds = "allow" 2024-09-16T15:46:58.366 INFO:teuthology.orchestra.run.smithi151.stdout: caps mon = "allow r" 2024-09-16T15:46:58.366 INFO:teuthology.orchestra.run.smithi151.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-16T15:46:58.366 INFO:teuthology.orchestra.run.smithi151.stderr:updated caps for client.0 2024-09-16T15:46:58.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.366+0000 7fac5fa00640 1 -- 172.21.15.151:0/3107750382 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fac340794b0 msgr2=0x7fac3407b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:58.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.366+0000 7fac5fa00640 1 --2- 172.21.15.151:0/3107750382 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fac340794b0 0x7fac3407b970 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fac48035000 tx=0x7fac4802d040 comp rx=0 tx=0).stop 2024-09-16T15:46:58.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.366+0000 7fac5fa00640 1 -- 172.21.15.151:0/3107750382 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac5813bcd0 msgr2=0x7fac580ad740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:58.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.366+0000 7fac5fa00640 1 --2- 172.21.15.151:0/3107750382 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac5813bcd0 0x7fac580ad740 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fac50002230 tx=0x7fac5000eb00 comp rx=0 tx=0).stop 2024-09-16T15:46:58.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.366+0000 7fac5fa00640 1 -- 172.21.15.151:0/3107750382 shutdown_connections 2024-09-16T15:46:58.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.366+0000 7fac5fa00640 1 --2- 172.21.15.151:0/3107750382 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fac340794b0 0x7fac3407b970 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:58.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.366+0000 7fac5fa00640 1 --2- 172.21.15.151:0/3107750382 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac5813f330 0x7fac581f69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:58.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.366+0000 7fac5fa00640 1 --2- 172.21.15.151:0/3107750382 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac5813bcd0 0x7fac580ad740 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:58.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.366+0000 7fac5fa00640 1 --2- 172.21.15.151:0/3107750382 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac3c003c10 0x7fac5813db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:58.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.366+0000 7fac5fa00640 1 -- 172.21.15.151:0/3107750382 >> 172.21.15.151:0/3107750382 conn(0x7fac580b1840 msgr2=0x7fac580b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:58.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.366+0000 7fac5fa00640 1 -- 172.21.15.151:0/3107750382 shutdown_connections 2024-09-16T15:46:58.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.367+0000 7fac5fa00640 1 -- 172.21.15.151:0/3107750382 wait complete. 2024-09-16T15:46:58.385 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:46:58.546 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.544+0000 7fd6d9c00640 1 -- 172.21.15.151:0/4030711807 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd6b8004800 msgr2=0x7fd6b8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:58.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.544+0000 7fd6d9c00640 1 --2- 172.21.15.151:0/4030711807 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd6b8004800 0x7fd6b8004c80 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4070620 tx=0x7fd6b8005830 comp rx=0 tx=0).stop 2024-09-16T15:46:58.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.544+0000 7fd6d3400640 1 -- 172.21.15.151:0/4030711807 reap_dead start 2024-09-16T15:46:58.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.544+0000 7fd6d9c00640 1 -- 172.21.15.151:0/4030711807 shutdown_connections 2024-09-16T15:46:58.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.544+0000 7fd6d9c00640 1 -- 172.21.15.151:0/4030711807 >> 172.21.15.151:0/4030711807 conn(0x7fd6d40b1950 msgr2=0x7fd6d40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:58.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.544+0000 7fd6d9c00640 1 -- 172.21.15.151:0/4030711807 shutdown_connections 2024-09-16T15:46:58.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.544+0000 7fd6d9c00640 1 -- 172.21.15.151:0/4030711807 wait complete. 2024-09-16T15:46:58.548 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.546+0000 7fd6d9c00640 1 Processor -- start 2024-09-16T15:46:58.548 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.546+0000 7fd6d9c00640 1 -- start start 2024-09-16T15:46:58.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.547+0000 7fd6d9c00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6b8004800 0x7fd6d40adee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:58.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.547+0000 7fd6d9c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6d415b2d0 0x7fd6d409ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:58.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.547+0000 7fd6d9c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd6d41534c0 0x7fd6d4153940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:58.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.547+0000 7fd6d9c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d406f9f0 con 0x7fd6d41534c0 2024-09-16T15:46:58.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.547+0000 7fd6d9c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d406f870 con 0x7fd6d415b2d0 2024-09-16T15:46:58.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.547+0000 7fd6d9c00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fd6d40b2440 con 0x7fd6b8004800 2024-09-16T15:46:58.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.547+0000 7fd6d3400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd6d41534c0 0x7fd6d4153940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:58.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.547+0000 7fd6d3400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd6d41534c0 0x7fd6d4153940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:46166/0 (socket says 172.21.15.151:46166) 2024-09-16T15:46:58.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.547+0000 7fd6d3400640 1 -- 172.21.15.151:0/626852715 learned_addr learned my addr 172.21.15.151:0/626852715 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:46:58.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.547+0000 7fd6d2000640 1 --2- 172.21.15.151:0/626852715 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6d415b2d0 0x7fd6d409ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:58.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.547+0000 7fd6d2a00640 1 --2- 172.21.15.151:0/626852715 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6b8004800 0x7fd6d40adee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:58.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.547+0000 7fd6d3400640 1 -- 172.21.15.151:0/626852715 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6b8004800 msgr2=0x7fd6d40adee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:58.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.547+0000 7fd6d3400640 1 --2- 172.21.15.151:0/626852715 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6b8004800 0x7fd6d40adee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:58.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.547+0000 7fd6d3400640 1 -- 172.21.15.151:0/626852715 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6d415b2d0 msgr2=0x7fd6d409ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:58.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.547+0000 7fd6d3400640 1 --2- 172.21.15.151:0/626852715 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6d415b2d0 0x7fd6d409ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:58.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.547+0000 7fd6d3400640 1 -- 172.21.15.151:0/626852715 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6d41f24b0 con 0x7fd6d41534c0 2024-09-16T15:46:58.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.548+0000 7fd6d3400640 1 --2- 172.21.15.151:0/626852715 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd6d41534c0 0x7fd6d4153940 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fd6d406e130 tx=0x7fd6cc0017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:58.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.548+0000 7fd6caa00640 1 -- 172.21.15.151:0/626852715 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc015070 con 0x7fd6d41534c0 2024-09-16T15:46:58.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.548+0000 7fd6caa00640 1 -- 172.21.15.151:0/626852715 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd6cc011650 con 0x7fd6d41534c0 2024-09-16T15:46:58.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.548+0000 7fd6caa00640 1 -- 172.21.15.151:0/626852715 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc011aa0 con 0x7fd6d41534c0 2024-09-16T15:46:58.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.548+0000 7fd6d9c00640 1 -- 172.21.15.151:0/626852715 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6d415b750 con 0x7fd6d41534c0 2024-09-16T15:46:58.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.548+0000 7fd6d9c00640 1 -- 172.21.15.151:0/626852715 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6d415bd10 con 0x7fd6d41534c0 2024-09-16T15:46:58.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.549+0000 7fd6d9c00640 1 -- 172.21.15.151:0/626852715 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6d40b2160 con 0x7fd6d41534c0 2024-09-16T15:46:58.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.550+0000 7fd6caa00640 1 -- 172.21.15.151:0/626852715 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 24) ==== 102409+0+0 (secure 0 0 0) 0x7fd6cc018070 con 0x7fd6d41534c0 2024-09-16T15:46:58.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.552+0000 7fd6caa00640 1 --2- 172.21.15.151:0/626852715 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd6b40794e0 0x7fd6b407b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:58.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.552+0000 7fd6caa00640 1 -- 172.21.15.151:0/626852715 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5765+0+0 (secure 0 0 0) 0x7fd6cc099c30 con 0x7fd6d41534c0 2024-09-16T15:46:58.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.552+0000 7fd6d2a00640 1 --2- 172.21.15.151:0/626852715 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd6b40794e0 0x7fd6b407b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:58.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.553+0000 7fd6d2a00640 1 --2- 172.21.15.151:0/626852715 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd6b40794e0 0x7fd6b407b9a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd6c4002230 tx=0x7fd6c4006d20 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:58.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.556+0000 7fd6caa00640 1 -- 172.21.15.151:0/626852715 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd6cc063cf0 con 0x7fd6d41534c0 2024-09-16T15:46:58.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.777+0000 7fd6d9c00640 1 -- 172.21.15.151:0/626852715 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd6d4075930 con 0x7fd6d41534c0 2024-09-16T15:46:58.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.778+0000 7fd6caa00640 1 -- 172.21.15.151:0/626852715 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+5234 (secure 0 0 0) 0x7fd6cc0635f0 con 0x7fd6d41534c0 2024-09-16T15:46:58.780 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:46:58.780 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":46,"btime":"2024-09-16T15:46:56:268896+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5705,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3291846596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3291846596},{"type":"v1","addr":"172.21.15.167:6837","nonce":3291846596}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6175,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/2504229431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":2504229431},{"type":"v1","addr":"172.21.15.151:6834","nonce":2504229431}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6181,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/464376104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":464376104},{"type":"v1","addr":"172.21.15.167:6836","nonce":464376104}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:46:52.671228+0000","modified":"2024-09-16T15:46:55.692251+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6456},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6456":{"gid":6456,"name":"a","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.151:6835/3158835036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":3158835036},{"type":"v1","addr":"172.21.15.151:6835","nonce":3158835036}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6456,"qdb_cluster":[6456]},"id":5}]} 2024-09-16T15:46:58.780 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 46 2024-09-16T15:46:58.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.780+0000 7fd6d9c00640 1 -- 172.21.15.151:0/626852715 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd6b40794e0 msgr2=0x7fd6b407b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:58.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.780+0000 7fd6d9c00640 1 --2- 172.21.15.151:0/626852715 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd6b40794e0 0x7fd6b407b9a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd6c4002230 tx=0x7fd6c4006d20 comp rx=0 tx=0).stop 2024-09-16T15:46:58.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.780+0000 7fd6d9c00640 1 -- 172.21.15.151:0/626852715 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd6d41534c0 msgr2=0x7fd6d4153940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:58.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.780+0000 7fd6d9c00640 1 --2- 172.21.15.151:0/626852715 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd6d41534c0 0x7fd6d4153940 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fd6d406e130 tx=0x7fd6cc0017a0 comp rx=0 tx=0).stop 2024-09-16T15:46:58.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.780+0000 7fd6d9c00640 1 -- 172.21.15.151:0/626852715 shutdown_connections 2024-09-16T15:46:58.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.780+0000 7fd6d9c00640 1 --2- 172.21.15.151:0/626852715 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd6b40794e0 0x7fd6b407b9a0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:58.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.780+0000 7fd6d9c00640 1 --2- 172.21.15.151:0/626852715 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd6d415b2d0 0x7fd6d409ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:58.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.780+0000 7fd6d9c00640 1 --2- 172.21.15.151:0/626852715 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd6d41534c0 0x7fd6d4153940 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:58.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.780+0000 7fd6d9c00640 1 --2- 172.21.15.151:0/626852715 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd6b8004800 0x7fd6d40adee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:58.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.780+0000 7fd6d9c00640 1 -- 172.21.15.151:0/626852715 >> 172.21.15.151:0/626852715 conn(0x7fd6d40b1950 msgr2=0x7fd6d40af920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:58.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.780+0000 7fd6d9c00640 1 -- 172.21.15.151:0/626852715 shutdown_connections 2024-09-16T15:46:58.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:58.780+0000 7fd6d9c00640 1 -- 172.21.15.151:0/626852715 wait complete. 2024-09-16T15:46:59.200 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 46, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-16T15:46:52.671228+0000', 'modified': '2024-09-16T15:46:55.692251+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6456}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6456': {'gid': 6456, 'name': 'a', 'rank': 0, 'incarnation': 42, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.151:6835/3158835036', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6833', 'nonce': 3158835036}, {'type': 'v1', 'addr': '172.21.15.151:6835', 'nonce': 3158835036}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6456, 'qdb_cluster': [6456]} 2024-09-16T15:46:59.200 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-16T15:46:59.200 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-16T15:46:59.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.359+0000 7f3bbbe00640 1 -- 172.21.15.151:0/2203815746 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3bb4073fb0 msgr2=0x7f3bb4074460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:59.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.359+0000 7f3bbbe00640 1 --2- 172.21.15.151:0/2203815746 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3bb4073fb0 0x7f3bb4074460 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f3bac005fd0 tx=0x7f3bac02a820 comp rx=0 tx=0).stop 2024-09-16T15:46:59.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.359+0000 7f3bbbe00640 1 -- 172.21.15.151:0/2203815746 shutdown_connections 2024-09-16T15:46:59.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.359+0000 7f3bbbe00640 1 --2- 172.21.15.151:0/2203815746 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3bb4073fb0 0x7f3bb4074460 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:59.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.359+0000 7f3bbbe00640 1 --2- 172.21.15.151:0/2203815746 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3bb415f5a0 0x7f3bb40b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:59.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.359+0000 7f3bbbe00640 1 -- 172.21.15.151:0/2203815746 >> 172.21.15.151:0/2203815746 conn(0x7f3bb40b1950 msgr2=0x7f3bb40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:59.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.359+0000 7f3bbbe00640 1 -- 172.21.15.151:0/2203815746 shutdown_connections 2024-09-16T15:46:59.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.359+0000 7f3bbbe00640 1 -- 172.21.15.151:0/2203815746 wait complete. 2024-09-16T15:46:59.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.361+0000 7f3bbbe00640 1 Processor -- start 2024-09-16T15:46:59.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.361+0000 7f3bbbe00640 1 -- start start 2024-09-16T15:46:59.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.362+0000 7f3bbbe00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3bb4073fb0 0x7f3bb41f6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:59.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.362+0000 7f3bbbe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bb40ccf00 0x7f3bb415c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:59.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.362+0000 7f3bbbe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3bb415f5a0 0x7f3bb41598b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:59.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.362+0000 7f3bbbe00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb406f2e0 con 0x7f3bb415f5a0 2024-09-16T15:46:59.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.362+0000 7f3bbbe00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb406f160 con 0x7f3bb40ccf00 2024-09-16T15:46:59.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.362+0000 7f3bbbe00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f3bb406f460 con 0x7f3bb4073fb0 2024-09-16T15:46:59.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.362+0000 7f3bb9e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3bb415f5a0 0x7f3bb41598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:59.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.362+0000 7f3bb9e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3bb415f5a0 0x7f3bb41598b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:46172/0 (socket says 172.21.15.151:46172) 2024-09-16T15:46:59.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.362+0000 7f3bb9e00640 1 -- 172.21.15.151:0/2751325085 learned_addr learned my addr 172.21.15.151:0/2751325085 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:46:59.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.362+0000 7f3bb8a00640 1 --2- 172.21.15.151:0/2751325085 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bb40ccf00 0x7f3bb415c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:59.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.362+0000 7f3bb9400640 1 --2- 172.21.15.151:0/2751325085 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3bb4073fb0 0x7f3bb41f6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:59.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.362+0000 7f3bb9e00640 1 -- 172.21.15.151:0/2751325085 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3bb4073fb0 msgr2=0x7f3bb41f6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:59.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.362+0000 7f3bb9e00640 1 --2- 172.21.15.151:0/2751325085 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3bb4073fb0 0x7f3bb41f6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:59.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.362+0000 7f3bb9e00640 1 -- 172.21.15.151:0/2751325085 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bb40ccf00 msgr2=0x7f3bb415c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:59.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.362+0000 7f3bb9e00640 1 --2- 172.21.15.151:0/2751325085 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bb40ccf00 0x7f3bb415c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:59.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.362+0000 7f3bb9e00640 1 -- 172.21.15.151:0/2751325085 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bb41500d0 con 0x7f3bb415f5a0 2024-09-16T15:46:59.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.362+0000 7f3bb9400640 1 --2- 172.21.15.151:0/2751325085 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3bb4073fb0 0x7f3bb41f6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:46:59.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.362+0000 7f3bb8a00640 1 --2- 172.21.15.151:0/2751325085 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bb40ccf00 0x7f3bb415c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:46:59.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.363+0000 7f3bb9e00640 1 --2- 172.21.15.151:0/2751325085 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3bb415f5a0 0x7f3bb41598b0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f3bac035040 tx=0x7f3bac032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:59.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.363+0000 7f3bb1600640 1 -- 172.21.15.151:0/2751325085 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bac037040 con 0x7f3bb415f5a0 2024-09-16T15:46:59.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.363+0000 7f3bb1600640 1 -- 172.21.15.151:0/2751325085 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3bac042070 con 0x7f3bb415f5a0 2024-09-16T15:46:59.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.363+0000 7f3bb1600640 1 -- 172.21.15.151:0/2751325085 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bac004260 con 0x7f3bb415f5a0 2024-09-16T15:46:59.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.363+0000 7f3bbbe00640 1 -- 172.21.15.151:0/2751325085 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bb414d450 con 0x7f3bb415f5a0 2024-09-16T15:46:59.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.363+0000 7f3bbbe00640 1 -- 172.21.15.151:0/2751325085 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bb414d6f0 con 0x7f3bb415f5a0 2024-09-16T15:46:59.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.365+0000 7f3bb1600640 1 -- 172.21.15.151:0/2751325085 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 24) ==== 102409+0+0 (secure 0 0 0) 0x7f3bac004400 con 0x7f3bb415f5a0 2024-09-16T15:46:59.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.366+0000 7f3bb1600640 1 --2- 172.21.15.151:0/2751325085 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3b940794b0 0x7f3b9407b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:59.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.366+0000 7f3bb1600640 1 -- 172.21.15.151:0/2751325085 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5765+0+0 (secure 0 0 0) 0x7f3bac0ba120 con 0x7f3bb415f5a0 2024-09-16T15:46:59.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.367+0000 7f3bbedd3740 1 Processor -- start 2024-09-16T15:46:59.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.367+0000 7f3bbedd3740 1 -- start start 2024-09-16T15:46:59.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.367+0000 7f3bb9400640 1 --2- 172.21.15.151:0/2751325085 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3b940794b0 0x7f3b9407b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:59.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.367+0000 7f3bb9400640 1 --2- 172.21.15.151:0/2751325085 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3b940794b0 0x7f3b9407b970 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3ba8009aa0 tx=0x7f3ba8005db0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:59.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.368+0000 7f3bbedd3740 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d8e5f358b0 0x55d8e5f35c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:59.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.368+0000 7f3bbedd3740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d8e5eeaf20 0x55d8e5f16610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:59.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.368+0000 7f3bbedd3740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8e5eea000 0x55d8e5f16b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:59.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.368+0000 7f3bbedd3740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d8e5e73430 con 0x55d8e5eeaf20 2024-09-16T15:46:59.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.368+0000 7f3bbedd3740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d8e5e635d0 con 0x55d8e5eea000 2024-09-16T15:46:59.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.368+0000 7f3bbedd3740 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x55d8e5e52a00 con 0x55d8e5f358b0 2024-09-16T15:46:59.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.368+0000 7f3bb9400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d8e5f358b0 0x55d8e5f35c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:59.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.368+0000 7f3bb8a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d8e5eeaf20 0x55d8e5f16610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:59.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.368+0000 7f3bb9400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d8e5f358b0 0x55d8e5f35c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.151:42762/0 (socket says 172.21.15.151:42762) 2024-09-16T15:46:59.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.368+0000 7f3bb9400640 1 -- 172.21.15.151:0/2451645929 learned_addr learned my addr 172.21.15.151:0/2451645929 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:46:59.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.368+0000 7f3bb9e00640 1 --2- 172.21.15.151:0/2451645929 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8e5eea000 0x55d8e5f16b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:59.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.368+0000 7f3bb8a00640 1 -- 172.21.15.151:0/2451645929 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d8e5f358b0 msgr2=0x55d8e5f35c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:46:59.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.368+0000 7f3bb8a00640 1 --2- 172.21.15.151:0/2451645929 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d8e5f358b0 0x55d8e5f35c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:59.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.368+0000 7f3bb8a00640 1 -- 172.21.15.151:0/2451645929 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8e5eea000 msgr2=0x55d8e5f16b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:46:59.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.368+0000 7f3bb8a00640 1 --2- 172.21.15.151:0/2451645929 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8e5eea000 0x55d8e5f16b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:59.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.368+0000 7f3bb8a00640 1 -- 172.21.15.151:0/2451645929 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8e5e1b520 con 0x55d8e5eeaf20 2024-09-16T15:46:59.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.369+0000 7f3bb8a00640 1 --2- 172.21.15.151:0/2451645929 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d8e5eeaf20 0x55d8e5f16610 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4070620 tx=0x7f3ba0001bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9903a7b119e2404 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:59.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.369+0000 7f3ba5c00640 1 -- 172.21.15.151:0/2451645929 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba0018070 con 0x55d8e5eeaf20 2024-09-16T15:46:59.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.369+0000 7f3ba5c00640 1 -- 172.21.15.151:0/2451645929 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ba000d730 con 0x55d8e5eeaf20 2024-09-16T15:46:59.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.369+0000 7f3ba5c00640 1 -- 172.21.15.151:0/2451645929 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba000db80 con 0x55d8e5eeaf20 2024-09-16T15:46:59.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.369+0000 7f3bbedd3740 1 -- 172.21.15.151:0/2451645929 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d8e5eeaf20 msgr2=0x55d8e5f16610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:59.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.369+0000 7f3bbedd3740 1 --2- 172.21.15.151:0/2451645929 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d8e5eeaf20 0x55d8e5f16610 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4070620 tx=0x7f3ba0001bd0 comp rx=0 tx=0).stop 2024-09-16T15:46:59.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.370+0000 7f3bbedd3740 1 -- 172.21.15.151:0/2451645929 shutdown_connections 2024-09-16T15:46:59.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.370+0000 7f3bbedd3740 1 --2- 172.21.15.151:0/2451645929 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8e5eea000 0x55d8e5f16b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:59.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.370+0000 7f3bbedd3740 1 --2- 172.21.15.151:0/2451645929 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d8e5eeaf20 0x55d8e5f16610 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:59.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.370+0000 7f3bbedd3740 1 --2- 172.21.15.151:0/2451645929 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d8e5f358b0 0x55d8e5f35c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:59.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.370+0000 7f3bbedd3740 1 -- 172.21.15.151:0/2451645929 >> 172.21.15.151:0/2451645929 conn(0x55d8e5eebff0 msgr2=0x55d8e5eefa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:59.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.370+0000 7f3bbedd3740 1 -- 172.21.15.151:0/2451645929 shutdown_connections 2024-09-16T15:46:59.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.370+0000 7f3bbedd3740 1 -- 172.21.15.151:0/2451645929 wait complete. 2024-09-16T15:46:59.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.370+0000 7f3bbedd3740 1 -- start start 2024-09-16T15:46:59.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.371+0000 7f3bbedd3740 1 Processor -- start 2024-09-16T15:46:59.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.372+0000 7f3bbedd3740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d8e5eeaf20 0x55d8e5f0fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:59.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.372+0000 7f3bbedd3740 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d8e5f358b0 0x55d8e5f2b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:59.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.372+0000 7f3bbedd3740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8e5f2d910 0x55d8e5f2bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:59.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.372+0000 7f3bbedd3740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d8e5df9330 con 0x55d8e5eeaf20 2024-09-16T15:46:59.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.372+0000 7f3bbedd3740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d8e5e56810 con 0x55d8e5f2d910 2024-09-16T15:46:59.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.372+0000 7f3bbedd3740 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x55d8e5bc3620 con 0x55d8e5f358b0 2024-09-16T15:46:59.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.372+0000 7f3bbedd3740 10 client.0 _ll_register_callbacks cb 0x55d8e5dfe420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T15:46:59.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.372+0000 7f3bb9400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d8e5eeaf20 0x55d8e5f0fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:59.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.372+0000 7f3bb9400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d8e5eeaf20 0x55d8e5f0fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:46186/0 (socket says 172.21.15.151:46186) 2024-09-16T15:46:59.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.372+0000 7f3bb9400640 1 -- 172.21.15.151:0/1098893254 learned_addr learned my addr 172.21.15.151:0/1098893254 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:46:59.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.372+0000 7f3bb8a00640 1 --2- 172.21.15.151:0/1098893254 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d8e5f358b0 0x55d8e5f2b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:59.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.372+0000 7f3bb9e00640 1 --2- 172.21.15.151:0/1098893254 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8e5f2d910 0x55d8e5f2bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:59.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.372+0000 7f3bb9400640 1 -- 172.21.15.151:0/1098893254 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d8e5f358b0 msgr2=0x55d8e5f2b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:46:59.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.372+0000 7f3bb9400640 1 --2- 172.21.15.151:0/1098893254 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d8e5f358b0 0x55d8e5f2b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:59.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.372+0000 7f3bb9400640 1 -- 172.21.15.151:0/1098893254 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8e5f2d910 msgr2=0x55d8e5f2bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:46:59.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.372+0000 7f3bb9400640 1 --2- 172.21.15.151:0/1098893254 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8e5f2d910 0x55d8e5f2bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:59.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.372+0000 7f3bb9400640 1 -- 172.21.15.151:0/1098893254 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8e5bb4850 con 0x55d8e5eeaf20 2024-09-16T15:46:59.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.373+0000 7f3bb9400640 1 --2- 172.21.15.151:0/1098893254 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d8e5eeaf20 0x55d8e5f0fd10 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f3ba8011980 tx=0x7f3ba80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b46fa042474920ff server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:59.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.373+0000 7f3b9a000640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-16T15:46:59.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.373+0000 7f3b9a000640 1 -- 172.21.15.151:0/1098893254 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba8010b50 con 0x55d8e5eeaf20 2024-09-16T15:46:59.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.373+0000 7f3bbedd3740 1 -- 172.21.15.151:0/1098893254 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ba000d730 con 0x55d8e5eeaf20 2024-09-16T15:46:59.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.373+0000 7f3b9a000640 1 -- 172.21.15.151:0/1098893254 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 5765+0+0 (secure 0 0 0) 0x7f3ba8010cf0 con 0x55d8e5eeaf20 2024-09-16T15:46:59.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.373+0000 7f3b9a000640 1 -- 172.21.15.151:0/1098893254 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f3ba802e050 con 0x55d8e5eeaf20 2024-09-16T15:46:59.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.373+0000 7f3bbedd3740 10 client.6567 fetch_fsmap learned FSMap version 46 2024-09-16T15:46:59.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.373+0000 7f3bbedd3740 1 -- 172.21.15.151:0/1098893254 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) -- 0x55d8e5c2b6b0 con 0x55d8e5eeaf20 2024-09-16T15:46:59.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.374+0000 7f3b9a000640 1 -- 172.21.15.151:0/1098893254 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ba000d730 con 0x55d8e5eeaf20 2024-09-16T15:46:59.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.374+0000 7f3b9a000640 1 -- 172.21.15.151:0/1098893254 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 46) ==== 3502+0+0 (secure 0 0 0) 0x7f3ba8031020 con 0x55d8e5eeaf20 2024-09-16T15:46:59.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.374+0000 7f3bbedd3740 10 client.6567 fetch_fsmap finished waiting for FSMap version 46 2024-09-16T15:46:59.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.374+0000 7f3bbedd3740 10 client.6567 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-16T15:46:59.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.374+0000 7f3bbedd3740 20 client.6567 populate_metadata read hostname 'smithi151' 2024-09-16T15:46:59.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.374+0000 7f3bbedd3740 1 --2- 172.21.15.151:0/1098893254 >> [v2:172.21.15.151:6833/3158835036,v1:172.21.15.151:6835/3158835036] conn(0x55d8e5f2cb80 0x55d8e5f330b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:59.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.374+0000 7f3bbedd3740 4 client.6567 mds_command: new command op to 6456 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:46:59.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.374+0000 7f3bbedd3740 1 -- 172.21.15.151:0/1098893254 --> [v2:172.21.15.151:6833/3158835036,v1:172.21.15.151:6835/3158835036] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8e5e65520 con 0x55d8e5f2cb80 2024-09-16T15:46:59.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.374+0000 7f3bb8a00640 1 --2- 172.21.15.151:0/1098893254 >> [v2:172.21.15.151:6833/3158835036,v1:172.21.15.151:6835/3158835036] conn(0x55d8e5f2cb80 0x55d8e5f330b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:59.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.375+0000 7f3b9a000640 10 client.6567 ms_handle_connect on v2:172.21.15.151:6833/3158835036 2024-09-16T15:46:59.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.375+0000 7f3bb8a00640 1 --2- 172.21.15.151:0/1098893254 >> [v2:172.21.15.151:6833/3158835036,v1:172.21.15.151:6835/3158835036] conn(0x55d8e5f2cb80 0x55d8e5f330b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a31fe21677eb9285 server_cookie=b9cd8d3f64b51ea2 in_seq=0 out_seq=0 2024-09-16T15:46:59.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.375+0000 7f3b9a000640 1 -- 172.21.15.151:0/1098893254 <== mds.0 v2:172.21.15.151:6833/3158835036 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d8e5e65520 con 0x55d8e5f2cb80 2024-09-16T15:46:59.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.375+0000 7f3b9a000640 10 client.6567 handle_command_reply: tid=0 2024-09-16T15:46:59.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.376+0000 7f3bbedd3740 1 client.6567 shutdown 2024-09-16T15:46:59.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.376+0000 7f3bbedd3740 1 -- 172.21.15.151:0/1098893254 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d8e5eeaf20 msgr2=0x55d8e5f0fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:59.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.376+0000 7f3bbedd3740 1 --2- 172.21.15.151:0/1098893254 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d8e5eeaf20 0x55d8e5f0fd10 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f3ba8011980 tx=0x7f3ba80046c0 comp rx=0 tx=0).stop 2024-09-16T15:46:59.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.376+0000 7f3bbedd3740 1 -- 172.21.15.151:0/1098893254 shutdown_connections 2024-09-16T15:46:59.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.376+0000 7f3bbedd3740 1 --2- 172.21.15.151:0/1098893254 >> [v2:172.21.15.151:6833/3158835036,v1:172.21.15.151:6835/3158835036] conn(0x55d8e5f2cb80 0x55d8e5f330b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:59.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.376+0000 7f3bbedd3740 1 --2- 172.21.15.151:0/1098893254 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8e5f2d910 0x55d8e5f2bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:59.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.376+0000 7f3bbedd3740 1 --2- 172.21.15.151:0/1098893254 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d8e5f358b0 0x55d8e5f2b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:59.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.376+0000 7f3bbedd3740 1 --2- 172.21.15.151:0/1098893254 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d8e5eeaf20 0x55d8e5f0fd10 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:59.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.376+0000 7f3b9a000640 0 client.6567 ms_handle_reset on v2:172.21.15.151:6833/3158835036 2024-09-16T15:46:59.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.376+0000 7f3bbedd3740 1 -- 172.21.15.151:0/1098893254 >> 172.21.15.151:0/1098893254 conn(0x55d8e5eea000 msgr2=0x55d8e5f15da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:59.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.376+0000 7f3bbedd3740 1 -- 172.21.15.151:0/1098893254 shutdown_connections 2024-09-16T15:46:59.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.376+0000 7f3bbedd3740 1 -- 172.21.15.151:0/1098893254 wait complete. 2024-09-16T15:46:59.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.376+0000 7f3bbedd3740 20 client.6567 trim_cache size 0 max 16384 2024-09-16T15:46:59.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.395+0000 7f3bbedd3740 1 Processor -- start 2024-09-16T15:46:59.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.395+0000 7f3bbedd3740 1 -- start start 2024-09-16T15:46:59.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.396+0000 7f3bbedd3740 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d8e5eeaf20 0x55d8e5f19490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:59.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.396+0000 7f3bbedd3740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d8e5f2d910 0x55d8e5f0faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:59.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.396+0000 7f3bbedd3740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8e5f358b0 0x55d8e5eea3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:59.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.396+0000 7f3bbedd3740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb41fccc0 con 0x55d8e5f2d910 2024-09-16T15:46:59.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.396+0000 7f3bbedd3740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb40b4220 con 0x55d8e5f358b0 2024-09-16T15:46:59.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.396+0000 7f3bbedd3740 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f3bb40d04f0 con 0x55d8e5eeaf20 2024-09-16T15:46:59.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.396+0000 7f3bb8a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d8e5f2d910 0x55d8e5f0faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:59.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.396+0000 7f3bb8a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d8e5f2d910 0x55d8e5f0faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:46188/0 (socket says 172.21.15.151:46188) 2024-09-16T15:46:59.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.396+0000 7f3bb8a00640 1 -- 172.21.15.151:0/3678610612 learned_addr learned my addr 172.21.15.151:0/3678610612 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:46:59.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.396+0000 7f3bb9400640 1 --2- 172.21.15.151:0/3678610612 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d8e5eeaf20 0x55d8e5f19490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:59.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.396+0000 7f3bb9e00640 1 --2- 172.21.15.151:0/3678610612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8e5f358b0 0x55d8e5eea3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:59.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.396+0000 7f3bb8a00640 1 -- 172.21.15.151:0/3678610612 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d8e5eeaf20 msgr2=0x55d8e5f19490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:46:59.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.396+0000 7f3bb8a00640 1 --2- 172.21.15.151:0/3678610612 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d8e5eeaf20 0x55d8e5f19490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:59.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.396+0000 7f3bb8a00640 1 -- 172.21.15.151:0/3678610612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8e5f358b0 msgr2=0x55d8e5eea3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:46:59.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.396+0000 7f3bb8a00640 1 --2- 172.21.15.151:0/3678610612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8e5f358b0 0x55d8e5eea3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:59.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.396+0000 7f3bb8a00640 1 -- 172.21.15.151:0/3678610612 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8e5f21d40 con 0x55d8e5f2d910 2024-09-16T15:46:59.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.396+0000 7f3bb9400640 1 --2- 172.21.15.151:0/3678610612 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d8e5eeaf20 0x55d8e5f19490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:46:59.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.396+0000 7f3bb9e00640 1 --2- 172.21.15.151:0/3678610612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8e5f358b0 0x55d8e5eea3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:46:59.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.397+0000 7f3bb8a00640 1 --2- 172.21.15.151:0/3678610612 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d8e5f2d910 0x55d8e5f0faf0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0000f60 tx=0x7f3ba0000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dc9eb788fdd6c99 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:59.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.397+0000 7f3ba5200640 1 -- 172.21.15.151:0/3678610612 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba0018040 con 0x55d8e5f2d910 2024-09-16T15:46:59.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.397+0000 7f3ba5200640 1 -- 172.21.15.151:0/3678610612 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ba0019040 con 0x55d8e5f2d910 2024-09-16T15:46:59.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.397+0000 7f3ba5200640 1 -- 172.21.15.151:0/3678610612 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba000d990 con 0x55d8e5f2d910 2024-09-16T15:46:59.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.397+0000 7f3bbedd3740 1 -- 172.21.15.151:0/3678610612 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d8e5f2d910 msgr2=0x55d8e5f0faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:59.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.397+0000 7f3bbedd3740 1 --2- 172.21.15.151:0/3678610612 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d8e5f2d910 0x55d8e5f0faf0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0000f60 tx=0x7f3ba0000ed0 comp rx=0 tx=0).stop 2024-09-16T15:46:59.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.398+0000 7f3bbedd3740 1 -- 172.21.15.151:0/3678610612 shutdown_connections 2024-09-16T15:46:59.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.398+0000 7f3bbedd3740 1 --2- 172.21.15.151:0/3678610612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8e5f358b0 0x55d8e5eea3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:59.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.398+0000 7f3bbedd3740 1 --2- 172.21.15.151:0/3678610612 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d8e5f2d910 0x55d8e5f0faf0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:59.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.398+0000 7f3bbedd3740 1 --2- 172.21.15.151:0/3678610612 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d8e5eeaf20 0x55d8e5f19490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:59.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.398+0000 7f3bbedd3740 1 -- 172.21.15.151:0/3678610612 >> 172.21.15.151:0/3678610612 conn(0x55d8e5eea000 msgr2=0x55d8e5f35c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:59.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.398+0000 7f3bbedd3740 1 -- 172.21.15.151:0/3678610612 shutdown_connections 2024-09-16T15:46:59.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.398+0000 7f3bbedd3740 1 -- 172.21.15.151:0/3678610612 wait complete. 2024-09-16T15:46:59.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.398+0000 7f3bbedd3740 1 -- start start 2024-09-16T15:46:59.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.398+0000 7f3bbedd3740 1 Processor -- start 2024-09-16T15:46:59.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.399+0000 7f3bbedd3740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d8e5eeaf20 0x55d8e5eea990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:59.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.399+0000 7f3bbedd3740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8e5f2d910 0x55d8e5eef130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:59.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.399+0000 7f3bbedd3740 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d8e5f358b0 0x55d8e5fbdad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:59.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.399+0000 7f3bbedd3740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d8e5f10300 con 0x55d8e5eeaf20 2024-09-16T15:46:59.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.399+0000 7f3bbedd3740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb414fc60 con 0x55d8e5f2d910 2024-09-16T15:46:59.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.399+0000 7f3bbedd3740 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x55d8e5f2ca00 con 0x55d8e5f358b0 2024-09-16T15:46:59.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.399+0000 7f3bbedd3740 10 client.0 _ll_register_callbacks cb 0x7f3ba80115d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T15:46:59.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.400+0000 7f3bb9400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d8e5eeaf20 0x55d8e5eea990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:59.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.400+0000 7f3bb9400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d8e5eeaf20 0x55d8e5eea990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:46200/0 (socket says 172.21.15.151:46200) 2024-09-16T15:46:59.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.400+0000 7f3bb9400640 1 -- 172.21.15.151:0/3099919281 learned_addr learned my addr 172.21.15.151:0/3099919281 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:46:59.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.400+0000 7f3bb9e00640 1 --2- 172.21.15.151:0/3099919281 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d8e5f358b0 0x55d8e5fbdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:59.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.400+0000 7f3bb8a00640 1 --2- 172.21.15.151:0/3099919281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8e5f2d910 0x55d8e5eef130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:59.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.400+0000 7f3bb9400640 1 -- 172.21.15.151:0/3099919281 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d8e5f358b0 msgr2=0x55d8e5fbdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:46:59.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.400+0000 7f3bb9400640 1 --2- 172.21.15.151:0/3099919281 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d8e5f358b0 0x55d8e5fbdad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:59.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.400+0000 7f3bb9400640 1 -- 172.21.15.151:0/3099919281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8e5f2d910 msgr2=0x55d8e5eef130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:46:59.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.400+0000 7f3bb9400640 1 --2- 172.21.15.151:0/3099919281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8e5f2d910 0x55d8e5eef130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:59.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.400+0000 7f3bb9400640 1 -- 172.21.15.151:0/3099919281 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8e5eef670 con 0x55d8e5eeaf20 2024-09-16T15:46:59.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.400+0000 7f3bb9e00640 1 --2- 172.21.15.151:0/3099919281 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d8e5f358b0 0x55d8e5fbdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:46:59.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.400+0000 7f3bb9400640 1 --2- 172.21.15.151:0/3099919281 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d8e5eeaf20 0x55d8e5eea990 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f3ba8005720 tx=0x7f3ba8005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f654b4be8d009d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:59.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.400+0000 7f3b9a000640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-16T15:46:59.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.401+0000 7f3b9a000640 1 -- 172.21.15.151:0/3099919281 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba8010c30 con 0x55d8e5eeaf20 2024-09-16T15:46:59.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.401+0000 7f3b9a000640 1 -- 172.21.15.151:0/3099919281 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 5765+0+0 (secure 0 0 0) 0x7f3ba8010dd0 con 0x55d8e5eeaf20 2024-09-16T15:46:59.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.401+0000 7f3bbedd3740 1 -- 172.21.15.151:0/3099919281 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ba0019040 con 0x55d8e5eeaf20 2024-09-16T15:46:59.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.401+0000 7f3b9a000640 1 -- 172.21.15.151:0/3099919281 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f3ba8004730 con 0x55d8e5eeaf20 2024-09-16T15:46:59.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.401+0000 7f3bbedd3740 10 client.6573 fetch_fsmap learned FSMap version 46 2024-09-16T15:46:59.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.401+0000 7f3bbedd3740 1 -- 172.21.15.151:0/3099919281 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) -- 0x55d8e5f21310 con 0x55d8e5eeaf20 2024-09-16T15:46:59.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.401+0000 7f3b9a000640 1 -- 172.21.15.151:0/3099919281 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ba0019040 con 0x55d8e5eeaf20 2024-09-16T15:46:59.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.401+0000 7f3b9a000640 1 -- 172.21.15.151:0/3099919281 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 46) ==== 3502+0+0 (secure 0 0 0) 0x7f3ba8045020 con 0x55d8e5eeaf20 2024-09-16T15:46:59.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.401+0000 7f3bbedd3740 10 client.6573 fetch_fsmap finished waiting for FSMap version 46 2024-09-16T15:46:59.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.401+0000 7f3bbedd3740 10 client.6573 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-16T15:46:59.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.401+0000 7f3bbedd3740 20 client.6573 populate_metadata read hostname 'smithi151' 2024-09-16T15:46:59.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.401+0000 7f3bbedd3740 1 --2- 172.21.15.151:0/3099919281 >> [v2:172.21.15.151:6833/3158835036,v1:172.21.15.151:6835/3158835036] conn(0x55d8e5f40290 0x55d8e5f118a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:59.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.401+0000 7f3bbedd3740 4 client.6573 mds_command: new command op to 6456 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T15:46:59.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.401+0000 7f3bbedd3740 1 -- 172.21.15.151:0/3099919281 --> [v2:172.21.15.151:6833/3158835036,v1:172.21.15.151:6835/3158835036] -- command(tid 0: {"prefix": "status"}) -- 0x55d8e5e60fd0 con 0x55d8e5f40290 2024-09-16T15:46:59.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.402+0000 7f3bb8a00640 1 --2- 172.21.15.151:0/3099919281 >> [v2:172.21.15.151:6833/3158835036,v1:172.21.15.151:6835/3158835036] conn(0x55d8e5f40290 0x55d8e5f118a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:59.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.402+0000 7f3b9a000640 10 client.6573 ms_handle_connect on v2:172.21.15.151:6833/3158835036 2024-09-16T15:46:59.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.402+0000 7f3bb8a00640 1 --2- 172.21.15.151:0/3099919281 >> [v2:172.21.15.151:6833/3158835036,v1:172.21.15.151:6835/3158835036] conn(0x55d8e5f40290 0x55d8e5f118a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11250d286a71da72 server_cookie=b1cb60ecf7660f12 in_seq=0 out_seq=0 2024-09-16T15:46:59.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.402+0000 7f3b9a000640 1 -- 172.21.15.151:0/3099919281 <== mds.0 v2:172.21.15.151:6833/3158835036 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55d8e5e60fd0 con 0x55d8e5f40290 2024-09-16T15:46:59.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.402+0000 7f3b9a000640 10 client.6573 handle_command_reply: tid=0 2024-09-16T15:46:59.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.402+0000 7f3bbedd3740 1 client.6573 shutdown 2024-09-16T15:46:59.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.403+0000 7f3bbedd3740 1 -- 172.21.15.151:0/3099919281 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d8e5eeaf20 msgr2=0x55d8e5eea990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:59.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.403+0000 7f3bbedd3740 1 --2- 172.21.15.151:0/3099919281 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d8e5eeaf20 0x55d8e5eea990 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f3ba8005720 tx=0x7f3ba8005750 comp rx=0 tx=0).stop 2024-09-16T15:46:59.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.403+0000 7f3bbedd3740 1 -- 172.21.15.151:0/3099919281 shutdown_connections 2024-09-16T15:46:59.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.403+0000 7f3bbedd3740 1 --2- 172.21.15.151:0/3099919281 >> [v2:172.21.15.151:6833/3158835036,v1:172.21.15.151:6835/3158835036] conn(0x55d8e5f40290 0x55d8e5f118a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:59.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.403+0000 7f3bbedd3740 1 --2- 172.21.15.151:0/3099919281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d8e5f2d910 0x55d8e5eef130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:59.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.403+0000 7f3bbedd3740 1 --2- 172.21.15.151:0/3099919281 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55d8e5f358b0 0x55d8e5fbdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:59.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.403+0000 7f3bbedd3740 1 --2- 172.21.15.151:0/3099919281 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d8e5eeaf20 0x55d8e5eea990 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:59.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.403+0000 7f3b9a000640 0 client.6573 ms_handle_reset on v2:172.21.15.151:6833/3158835036 2024-09-16T15:46:59.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.403+0000 7f3bbedd3740 1 -- 172.21.15.151:0/3099919281 >> 172.21.15.151:0/3099919281 conn(0x55d8e5eea000 msgr2=0x55d8e5f31e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:59.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.403+0000 7f3bbedd3740 1 -- 172.21.15.151:0/3099919281 shutdown_connections 2024-09-16T15:46:59.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.403+0000 7f3bbedd3740 1 -- 172.21.15.151:0/3099919281 wait complete. 2024-09-16T15:46:59.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.403+0000 7f3bbedd3740 20 client.6573 trim_cache size 0 max 16384 2024-09-16T15:46:59.410 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-16T15:46:59.410 INFO:teuthology.orchestra.run.smithi151.stdout: "cluster_fsid": "1757be18-e41a-4446-bb4f-ee4ac81c39df", 2024-09-16T15:46:59.410 INFO:teuthology.orchestra.run.smithi151.stdout: "whoami": 0, 2024-09-16T15:46:59.410 INFO:teuthology.orchestra.run.smithi151.stdout: "id": 6456, 2024-09-16T15:46:59.410 INFO:teuthology.orchestra.run.smithi151.stdout: "want_state": "up:active", 2024-09-16T15:46:59.410 INFO:teuthology.orchestra.run.smithi151.stdout: "state": "up:active", 2024-09-16T15:46:59.410 INFO:teuthology.orchestra.run.smithi151.stdout: "fs_name": "cephfs", 2024-09-16T15:46:59.410 INFO:teuthology.orchestra.run.smithi151.stdout: "rank_uptime": 6.1615997509999998, 2024-09-16T15:46:59.410 INFO:teuthology.orchestra.run.smithi151.stdout: "mdsmap_epoch": 46, 2024-09-16T15:46:59.410 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_epoch": 76, 2024-09-16T15:46:59.410 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_epoch_barrier": 76, 2024-09-16T15:46:59.410 INFO:teuthology.orchestra.run.smithi151.stdout: "uptime": 10.473826062000001 2024-09-16T15:46:59.411 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-16T15:46:59.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.404+0000 7f3bbbe00640 1 -- 172.21.15.151:0/2751325085 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3b940794b0 msgr2=0x7f3b9407b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:59.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.404+0000 7f3bbbe00640 1 --2- 172.21.15.151:0/2751325085 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3b940794b0 0x7f3b9407b970 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3ba8009aa0 tx=0x7f3ba8005db0 comp rx=0 tx=0).stop 2024-09-16T15:46:59.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.404+0000 7f3bbbe00640 1 -- 172.21.15.151:0/2751325085 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3bb415f5a0 msgr2=0x7f3bb41598b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:59.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.404+0000 7f3bbbe00640 1 --2- 172.21.15.151:0/2751325085 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3bb415f5a0 0x7f3bb41598b0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f3bac035040 tx=0x7f3bac032d10 comp rx=0 tx=0).stop 2024-09-16T15:46:59.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.404+0000 7f3bbbe00640 1 -- 172.21.15.151:0/2751325085 shutdown_connections 2024-09-16T15:46:59.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.404+0000 7f3bbbe00640 1 --2- 172.21.15.151:0/2751325085 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3b940794b0 0x7f3b9407b970 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:59.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.404+0000 7f3bbbe00640 1 --2- 172.21.15.151:0/2751325085 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bb40ccf00 0x7f3bb415c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:59.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.404+0000 7f3bbbe00640 1 --2- 172.21.15.151:0/2751325085 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3bb415f5a0 0x7f3bb41598b0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:59.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.404+0000 7f3bbbe00640 1 --2- 172.21.15.151:0/2751325085 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3bb4073fb0 0x7f3bb41f6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:59.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.404+0000 7f3bbbe00640 1 -- 172.21.15.151:0/2751325085 >> 172.21.15.151:0/2751325085 conn(0x7f3bb40b1950 msgr2=0x7f3bb409d150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:59.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.404+0000 7f3bbbe00640 1 -- 172.21.15.151:0/2751325085 shutdown_connections 2024-09-16T15:46:59.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.404+0000 7f3bbbe00640 1 -- 172.21.15.151:0/2751325085 wait complete. 2024-09-16T15:46:59.421 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-16T15:46:59.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.577+0000 7f9129400640 1 -- 172.21.15.151:0/2558608849 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f912415bc30 msgr2=0x7f91240b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:59.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.577+0000 7f9129400640 1 --2- 172.21.15.151:0/2558608849 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f912415bc30 0x7f91240b28f0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f9124073b00 tx=0x7f911402a560 comp rx=0 tx=0).stop 2024-09-16T15:46:59.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.578+0000 7f9129400640 1 -- 172.21.15.151:0/2558608849 shutdown_connections 2024-09-16T15:46:59.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.578+0000 7f9129400640 1 --2- 172.21.15.151:0/2558608849 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f91240d0540 0x7f9124074280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:59.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.578+0000 7f9129400640 1 --2- 172.21.15.151:0/2558608849 >> v2:172.21.15.167:3300/0 conn(0x7f91240725d0 0x7f9124073d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:59.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.578+0000 7f9129400640 1 --2- 172.21.15.151:0/2558608849 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f912415bc30 0x7f91240b28f0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:59.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.578+0000 7f9129400640 1 -- 172.21.15.151:0/2558608849 >> 172.21.15.151:0/2558608849 conn(0x7f91240cfa20 msgr2=0x7f91240cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:59.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.578+0000 7f9129400640 1 -- 172.21.15.151:0/2558608849 shutdown_connections 2024-09-16T15:46:59.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.578+0000 7f9129400640 1 -- 172.21.15.151:0/2558608849 wait complete. 2024-09-16T15:46:59.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.579+0000 7f9129400640 1 Processor -- start 2024-09-16T15:46:59.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.579+0000 7f9129400640 1 -- start start 2024-09-16T15:46:59.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.580+0000 7f9129400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f91240d0540 0x7f91241507d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:59.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.580+0000 7f9129400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f912415bc30 0x7f9124156e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:59.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.580+0000 7f9129400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9124150d10 0x7f9124157340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:59.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.580+0000 7f9129400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f912406f4a0 con 0x7f91240d0540 2024-09-16T15:46:59.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.580+0000 7f9129400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f912406f320 con 0x7f9124150d10 2024-09-16T15:46:59.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.580+0000 7f9129400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f912406f620 con 0x7f912415bc30 2024-09-16T15:46:59.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.581+0000 7f9123e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f91240d0540 0x7f91241507d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:59.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.581+0000 7f9123e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f91240d0540 0x7f91241507d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:46218/0 (socket says 172.21.15.151:46218) 2024-09-16T15:46:59.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.581+0000 7f9123e00640 1 -- 172.21.15.151:0/3120062149 learned_addr learned my addr 172.21.15.151:0/3120062149 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:46:59.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.581+0000 7f9123400640 1 --2- 172.21.15.151:0/3120062149 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f912415bc30 0x7f9124156e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:59.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.581+0000 7f9128a00640 1 --2- 172.21.15.151:0/3120062149 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9124150d10 0x7f9124157340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:59.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.581+0000 7f9123e00640 1 -- 172.21.15.151:0/3120062149 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f912415bc30 msgr2=0x7f9124156e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:59.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.581+0000 7f9123e00640 1 --2- 172.21.15.151:0/3120062149 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f912415bc30 0x7f9124156e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:59.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.581+0000 7f9123e00640 1 -- 172.21.15.151:0/3120062149 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9124150d10 msgr2=0x7f9124157340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:59.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.581+0000 7f9123e00640 1 --2- 172.21.15.151:0/3120062149 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9124150d10 0x7f9124157340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:59.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.581+0000 7f9123e00640 1 -- 172.21.15.151:0/3120062149 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f912414f190 con 0x7f91240d0540 2024-09-16T15:46:59.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.581+0000 7f9128a00640 1 --2- 172.21.15.151:0/3120062149 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9124150d10 0x7f9124157340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:46:59.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.581+0000 7f9123e00640 1 --2- 172.21.15.151:0/3120062149 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f91240d0540 0x7f91241507d0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f9114034040 tx=0x7f91140038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:59.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.582+0000 7f911be00640 1 -- 172.21.15.151:0/3120062149 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f911403e030 con 0x7f91240d0540 2024-09-16T15:46:59.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.582+0000 7f911be00640 1 -- 172.21.15.151:0/3120062149 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9114003ca0 con 0x7f91240d0540 2024-09-16T15:46:59.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.582+0000 7f911be00640 1 -- 172.21.15.151:0/3120062149 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9114035040 con 0x7f91240d0540 2024-09-16T15:46:59.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.582+0000 7f9129400640 1 -- 172.21.15.151:0/3120062149 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f912414f470 con 0x7f91240d0540 2024-09-16T15:46:59.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.582+0000 7f9129400640 1 -- 172.21.15.151:0/3120062149 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f912414fa30 con 0x7f91240d0540 2024-09-16T15:46:59.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.583+0000 7f9129400640 1 -- 172.21.15.151:0/3120062149 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91240b28f0 con 0x7f91240d0540 2024-09-16T15:46:59.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.584+0000 7f911be00640 1 -- 172.21.15.151:0/3120062149 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 24) ==== 102409+0+0 (secure 0 0 0) 0x7f9114041030 con 0x7f91240d0540 2024-09-16T15:46:59.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.585+0000 7f911be00640 1 --2- 172.21.15.151:0/3120062149 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f91040794b0 0x7f910407b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:46:59.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.585+0000 7f911be00640 1 -- 172.21.15.151:0/3120062149 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5765+0+0 (secure 0 0 0) 0x7f91140b98c0 con 0x7f91240d0540 2024-09-16T15:46:59.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.585+0000 7f9123400640 1 --2- 172.21.15.151:0/3120062149 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f91040794b0 0x7f910407b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:46:59.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.586+0000 7f9123400640 1 --2- 172.21.15.151:0/3120062149 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f91040794b0 0x7f910407b970 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f91240b53f0 tx=0x7f9110005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:46:59.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.589+0000 7f911be00640 1 -- 172.21.15.151:0/3120062149 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9114083980 con 0x7f91240d0540 2024-09-16T15:46:59.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.811+0000 7f9129400640 1 -- 172.21.15.151:0/3120062149 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f9124155830 con 0x7f91240d0540 2024-09-16T15:46:59.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.811+0000 7f911be00640 1 -- 172.21.15.151:0/3120062149 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v46) ==== 53+0+73 (secure 0 0 0) 0x7f9114083280 con 0x7f91240d0540 2024-09-16T15:46:59.814 INFO:teuthology.orchestra.run.smithi151.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-16T15:46:59.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.813+0000 7f9129400640 1 -- 172.21.15.151:0/3120062149 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f91040794b0 msgr2=0x7f910407b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:59.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.813+0000 7f9129400640 1 --2- 172.21.15.151:0/3120062149 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f91040794b0 0x7f910407b970 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f91240b53f0 tx=0x7f9110005f90 comp rx=0 tx=0).stop 2024-09-16T15:46:59.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.813+0000 7f9129400640 1 -- 172.21.15.151:0/3120062149 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f91240d0540 msgr2=0x7f91241507d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:46:59.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.813+0000 7f9129400640 1 --2- 172.21.15.151:0/3120062149 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f91240d0540 0x7f91241507d0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f9114034040 tx=0x7f91140038a0 comp rx=0 tx=0).stop 2024-09-16T15:46:59.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.814+0000 7f9129400640 1 -- 172.21.15.151:0/3120062149 shutdown_connections 2024-09-16T15:46:59.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.814+0000 7f9129400640 1 --2- 172.21.15.151:0/3120062149 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f91040794b0 0x7f910407b970 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:59.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.814+0000 7f9129400640 1 --2- 172.21.15.151:0/3120062149 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9124150d10 0x7f9124157340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:59.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.814+0000 7f9129400640 1 --2- 172.21.15.151:0/3120062149 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f912415bc30 0x7f9124156e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:59.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.814+0000 7f9129400640 1 --2- 172.21.15.151:0/3120062149 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f91240d0540 0x7f91241507d0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:46:59.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.814+0000 7f9129400640 1 -- 172.21.15.151:0/3120062149 >> 172.21.15.151:0/3120062149 conn(0x7f91240cfa20 msgr2=0x7f91240ca5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:46:59.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.814+0000 7f9129400640 1 -- 172.21.15.151:0/3120062149 shutdown_connections 2024-09-16T15:46:59.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:46:59.814+0000 7f9129400640 1 -- 172.21.15.151:0/3120062149 wait complete. 2024-09-16T15:46:59.832 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-16T15:46:59.833 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T15:46:59.833 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi151.front.sepia.ceph.com 2024-09-16T15:46:59.833 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-16T15:46:59.833 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-16T15:46:59.833 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-16T15:46:59.833 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-16T15:46:59.833 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T15:46:59.833 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T15:46:59.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:46:59.834 DEBUG:teuthology.orchestra.run.smithi151:> ip addr 2024-09-16T15:46:59.869 INFO:teuthology.orchestra.run.smithi151.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T15:46:59.870 INFO:teuthology.orchestra.run.smithi151.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T15:46:59.870 INFO:teuthology.orchestra.run.smithi151.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T15:46:59.870 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:46:59.870 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 ::1/128 scope host 2024-09-16T15:46:59.870 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:46:59.870 INFO:teuthology.orchestra.run.smithi151.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:46:59.870 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:da:6e:0c brd ff:ff:ff:ff:ff:ff 2024-09-16T15:46:59.870 INFO:teuthology.orchestra.run.smithi151.stdout: altname enp7s0f0 2024-09-16T15:46:59.870 INFO:teuthology.orchestra.run.smithi151.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:46:59.870 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:da:6e:0d brd ff:ff:ff:ff:ff:ff 2024-09-16T15:46:59.870 INFO:teuthology.orchestra.run.smithi151.stdout: altname enp7s0f1 2024-09-16T15:46:59.870 INFO:teuthology.orchestra.run.smithi151.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:46:59.870 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:88:6f:62 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:46:59.870 INFO:teuthology.orchestra.run.smithi151.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T15:46:59.870 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:88:6f:63 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:46:59.871 INFO:teuthology.orchestra.run.smithi151.stdout: inet 172.21.15.151/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-16T15:46:59.871 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft 42269sec preferred_lft 42269sec 2024-09-16T15:46:59.871 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 fe80::ec4:7aff:fe88:6f63/64 scope link noprefixroute 2024-09-16T15:46:59.871 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:46:59.871 INFO:teuthology.orchestra.run.smithi151.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-16T15:46:59.871 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 62:3f:3b:48:ad:f8 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:46:59.871 INFO:teuthology.orchestra.run.smithi151.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-16T15:46:59.871 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:46:59.871 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 fe80::40fa:7fff:fedf:2a00/64 scope link 2024-09-16T15:46:59.871 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:46:59.871 INFO:teuthology.orchestra.run.smithi151.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T15:46:59.871 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 62:3f:3b:48:ad:f8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T15:46:59.871 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 fe80::603f:3bff:fe48:adf8/64 scope link 2024-09-16T15:46:59.871 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:46:59.871 INFO:teuthology.orchestra.run.smithi151.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T15:46:59.872 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether b6:ac:fe:34:a7:66 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-16T15:46:59.872 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 fe80::b4ac:feff:fe34:a766/64 scope link 2024-09-16T15:46:59.872 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:46:59.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:46:59.872 DEBUG:teuthology.orchestra.run.smithi151:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T15:46:59.964 INFO:teuthology.orchestra.run.smithi151.stdout:1 2024-09-16T15:46:59.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:46:59.967 DEBUG:teuthology.orchestra.run.smithi151:> route 2024-09-16T15:47:00.003 INFO:teuthology.orchestra.run.smithi151.stdout:Kernel IP routing table 2024-09-16T15:47:00.004 INFO:teuthology.orchestra.run.smithi151.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T15:47:00.004 INFO:teuthology.orchestra.run.smithi151.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-16T15:47:00.004 INFO:teuthology.orchestra.run.smithi151.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-16T15:47:00.004 INFO:teuthology.orchestra.run.smithi151.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T15:47:00.004 DEBUG:teuthology.orchestra.run.smithi151:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T15:47:00.004 DEBUG:teuthology.orchestra.run.smithi151:> set -e 2024-09-16T15:47:00.004 DEBUG:teuthology.orchestra.run.smithi151:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-16T15:47:00.005 DEBUG:teuthology.orchestra.run.smithi151:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-16T15:47:00.005 DEBUG:teuthology.orchestra.run.smithi151:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-16T15:47:00.005 DEBUG:teuthology.orchestra.run.smithi151:> ') 2024-09-16T15:47:00.122 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:47:00.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:47:00.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:47:00.202 DEBUG:teuthology.orchestra.run.smithi151:> ip netns list 2024-09-16T15:47:00.238 INFO:teuthology.orchestra.run.smithi151.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-16T15:47:00.238 INFO:teuthology.orchestra.run.smithi151.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-16T15:47:00.238 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T15:47:00.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:47:00.239 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip link set brx.0 up 2024-09-16T15:47:00.336 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:47:00.336 DEBUG:teuthology.orchestra.run.smithi151:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-16T15:47:00.371 INFO:teuthology.orchestra.run.smithi151.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-16T15:47:00.372 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:47:00.372 DEBUG:teuthology.orchestra.run.smithi151:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-16T15:47:00.447 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-16T15:47:00.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:47:00.447 DEBUG:teuthology.orchestra.run.smithi151:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-16T15:47:00.575 INFO:teuthology.orchestra.run.smithi151.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-16T15:47:00.575 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-16T15:47:00.575 INFO:teuthology.orchestra.run.smithi151.stdout:invalid new device string format 2024-09-16T15:47:00.591 INFO:teuthology.orchestra.run.smithi151.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-16T15:47:00.591 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-16T15:47:00.591 INFO:teuthology.orchestra.run.smithi151.stdout:invalid new device string format 2024-09-16T15:47:00.591 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: resolved to: "172.21.15.151:3300,172.21.15.167:3300,172.21.15.167:3301" 2024-09-16T15:47:00.591 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-16T15:47:00.613 INFO:teuthology.orchestra.run.smithi151.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-16T15:47:00.613 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-16T15:47:00.614 INFO:teuthology.orchestra.run.smithi151.stdout:invalid new device string format 2024-09-16T15:47:00.614 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: resolved to: "172.21.15.151:3300,172.21.15.167:3300,172.21.15.167:3301" 2024-09-16T15:47:00.614 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-16T15:47:00.614 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: resolved to: "172.21.15.151:6789,172.21.15.167:6789,172.21.15.167:6790" 2024-09-16T15:47:00.614 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.151:6789,172.21.15.167:6789,172.21.15.167:6790:/ 2024-09-16T15:47:00.614 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=1757be18-e41a-4446-bb4f-ee4ac81c39df" will pass to kernel 2024-09-16T15:47:00.615 INFO:teuthology.orchestra.run.smithi151.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-16T15:47:00.615 INFO:teuthology.orchestra.run.smithi151.stdout: You just mounted a file system that supports labels which does not 2024-09-16T15:47:00.615 INFO:teuthology.orchestra.run.smithi151.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-16T15:47:00.616 INFO:teuthology.orchestra.run.smithi151.stdout: applications will generate AVC messages and not be allowed access to 2024-09-16T15:47:00.616 INFO:teuthology.orchestra.run.smithi151.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-16T15:47:00.619 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-16T15:47:00.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:47:00.619 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-16T15:47:00.675 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T15:47:00.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:00 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:47:00.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:47:00.769 INFO:teuthology.orchestra.run.smithi151.stdout:client5810 2024-09-16T15:47:00.770 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:47:00.770 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:47:00.805 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-16T15:47:01.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.007+0000 7f256e200640 1 -- 172.21.15.151:0/593518758 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f256806ee00 msgr2=0x7f256806f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:01.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.007+0000 7f256e200640 1 --2- 172.21.15.151:0/593518758 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f256806ee00 0x7f256806f300 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2554009830 tx=0x7f255402a510 comp rx=0 tx=0).stop 2024-09-16T15:47:01.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.007+0000 7f256e200640 1 -- 172.21.15.151:0/593518758 shutdown_connections 2024-09-16T15:47:01.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.007+0000 7f256e200640 1 --2- 172.21.15.151:0/593518758 >> v2:172.21.15.167:3300/0 conn(0x7f25680b4770 0x7f25680ad6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:01.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.007+0000 7f256e200640 1 --2- 172.21.15.151:0/593518758 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f256806ee00 0x7f256806f300 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:01.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.007+0000 7f256e200640 1 -- 172.21.15.151:0/593518758 >> 172.21.15.151:0/593518758 conn(0x7f25680b18e0 msgr2=0x7f25680b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:01.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.007+0000 7f256e200640 1 -- 172.21.15.151:0/593518758 shutdown_connections 2024-09-16T15:47:01.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.007+0000 7f256e200640 1 -- 172.21.15.151:0/593518758 wait complete. 2024-09-16T15:47:01.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.008+0000 7f256e200640 1 Processor -- start 2024-09-16T15:47:01.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.008+0000 7f256e200640 1 -- start start 2024-09-16T15:47:01.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.009+0000 7f256e200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f256806ee00 0x7f2568073c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:01.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.009+0000 7f256e200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25680b4770 0x7f256814cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:01.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.009+0000 7f256e200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f256815dfc0 0x7f2568072e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:01.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.009+0000 7f256e200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f25680ae0e0 con 0x7f256806ee00 2024-09-16T15:47:01.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.009+0000 7f256e200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f25680adf60 con 0x7f25680b4770 2024-09-16T15:47:01.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.009+0000 7f256e200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f25680ae260 con 0x7f256815dfc0 2024-09-16T15:47:01.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.010+0000 7f2567400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f256806ee00 0x7f2568073c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:01.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.010+0000 7f2567400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f256806ee00 0x7f2568073c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:46224/0 (socket says 172.21.15.151:46224) 2024-09-16T15:47:01.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.010+0000 7f2567400640 1 -- 172.21.15.151:0/3986106136 learned_addr learned my addr 172.21.15.151:0/3986106136 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:47:01.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.010+0000 7f2566a00640 1 --2- 172.21.15.151:0/3986106136 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25680b4770 0x7f256814cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:01.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.010+0000 7f2567e00640 1 --2- 172.21.15.151:0/3986106136 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f256815dfc0 0x7f2568072e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:01.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.010+0000 7f2567400640 1 -- 172.21.15.151:0/3986106136 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f256815dfc0 msgr2=0x7f2568072e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:01.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.010+0000 7f2567400640 1 --2- 172.21.15.151:0/3986106136 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f256815dfc0 0x7f2568072e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:01.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.010+0000 7f2567400640 1 -- 172.21.15.151:0/3986106136 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25680b4770 msgr2=0x7f256814cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:01.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.010+0000 7f2567400640 1 --2- 172.21.15.151:0/3986106136 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25680b4770 0x7f256814cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:01.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.010+0000 7f2567400640 1 -- 172.21.15.151:0/3986106136 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2568155560 con 0x7f256806ee00 2024-09-16T15:47:01.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.010+0000 7f2567400640 1 --2- 172.21.15.151:0/3986106136 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f256806ee00 0x7f2568073c80 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f255800c3a0 tx=0x7f255800c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:01.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.011+0000 7f255f400640 1 -- 172.21.15.151:0/3986106136 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2558011790 con 0x7f256806ee00 2024-09-16T15:47:01.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.011+0000 7f255f400640 1 -- 172.21.15.151:0/3986106136 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2558017070 con 0x7f256806ee00 2024-09-16T15:47:01.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.011+0000 7f256e200640 1 -- 172.21.15.151:0/3986106136 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2568155840 con 0x7f256806ee00 2024-09-16T15:47:01.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.011+0000 7f255f400640 1 -- 172.21.15.151:0/3986106136 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2558011cc0 con 0x7f256806ee00 2024-09-16T15:47:01.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.011+0000 7f256e200640 1 -- 172.21.15.151:0/3986106136 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2568155e00 con 0x7f256806ee00 2024-09-16T15:47:01.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.012+0000 7f256e200640 1 -- 172.21.15.151:0/3986106136 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2568069360 con 0x7f256806ee00 2024-09-16T15:47:01.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.013+0000 7f255f400640 1 -- 172.21.15.151:0/3986106136 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 24) ==== 102409+0+0 (secure 0 0 0) 0x7f255801c460 con 0x7f256806ee00 2024-09-16T15:47:01.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.014+0000 7f255f400640 1 --2- 172.21.15.151:0/3986106136 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f25480794b0 0x7f254807b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:01.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.014+0000 7f255f400640 1 -- 172.21.15.151:0/3986106136 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5765+0+0 (secure 0 0 0) 0x7f2558098ff0 con 0x7f256806ee00 2024-09-16T15:47:01.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.015+0000 7f2566a00640 1 --2- 172.21.15.151:0/3986106136 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f25480794b0 0x7f254807b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:01.017 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.015+0000 7f2566a00640 1 --2- 172.21.15.151:0/3986106136 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f25480794b0 0x7f254807b970 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f2554034000 tx=0x7f2554004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:01.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.018+0000 7f255f400640 1 -- 172.21.15.151:0/3986106136 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f25580630b0 con 0x7f256806ee00 2024-09-16T15:47:01.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.240+0000 7f256e200640 1 -- 172.21.15.151:0/3986106136 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f25680cd470 con 0x7f256806ee00 2024-09-16T15:47:01.293 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.291+0000 7f255f400640 1 -- 172.21.15.151:0/3986106136 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v47) ==== 136+0+0 (secure 0 0 0) 0x7f25580629b0 con 0x7f256806ee00 2024-09-16T15:47:01.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.294+0000 7f256e200640 1 -- 172.21.15.151:0/3986106136 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f25480794b0 msgr2=0x7f254807b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:01.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.294+0000 7f256e200640 1 --2- 172.21.15.151:0/3986106136 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f25480794b0 0x7f254807b970 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f2554034000 tx=0x7f2554004730 comp rx=0 tx=0).stop 2024-09-16T15:47:01.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.294+0000 7f256e200640 1 -- 172.21.15.151:0/3986106136 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f256806ee00 msgr2=0x7f2568073c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:01.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.294+0000 7f256e200640 1 --2- 172.21.15.151:0/3986106136 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f256806ee00 0x7f2568073c80 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f255800c3a0 tx=0x7f255800c7a0 comp rx=0 tx=0).stop 2024-09-16T15:47:01.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.294+0000 7f256e200640 1 -- 172.21.15.151:0/3986106136 shutdown_connections 2024-09-16T15:47:01.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.294+0000 7f256e200640 1 --2- 172.21.15.151:0/3986106136 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f25480794b0 0x7f254807b970 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:01.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.294+0000 7f256e200640 1 --2- 172.21.15.151:0/3986106136 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25680b4770 0x7f256814cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:01.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.294+0000 7f256e200640 1 --2- 172.21.15.151:0/3986106136 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f256815dfc0 0x7f2568072e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:01.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.294+0000 7f256e200640 1 --2- 172.21.15.151:0/3986106136 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f256806ee00 0x7f2568073c80 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:01.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.294+0000 7f256e200640 1 -- 172.21.15.151:0/3986106136 >> 172.21.15.151:0/3986106136 conn(0x7f25680b18e0 msgr2=0x7f25681587b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:01.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.294+0000 7f256e200640 1 -- 172.21.15.151:0/3986106136 shutdown_connections 2024-09-16T15:47:01.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.294+0000 7f256e200640 1 -- 172.21.15.151:0/3986106136 wait complete. 2024-09-16T15:47:01.313 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-16T15:47:01.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.470+0000 7fed3f600640 1 -- 172.21.15.151:0/3391618870 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fed381445d0 msgr2=0x7fed380cebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:01.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.470+0000 7fed3f600640 1 --2- 172.21.15.151:0/3391618870 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fed381445d0 0x7fed380cebf0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fed28009be0 tx=0x7fed2802a8d0 comp rx=0 tx=0).stop 2024-09-16T15:47:01.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.470+0000 7fed3f600640 1 -- 172.21.15.151:0/3391618870 shutdown_connections 2024-09-16T15:47:01.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.470+0000 7fed3f600640 1 --2- 172.21.15.151:0/3391618870 >> v2:172.21.15.167:3300/0 conn(0x7fed380ccfd0 0x7fed380cf130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:01.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.470+0000 7fed3f600640 1 --2- 172.21.15.151:0/3391618870 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fed381445d0 0x7fed380cebf0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:01.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.470+0000 7fed3f600640 1 -- 172.21.15.151:0/3391618870 >> 172.21.15.151:0/3391618870 conn(0x7fed380b1950 msgr2=0x7fed380b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:01.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.470+0000 7fed3f600640 1 -- 172.21.15.151:0/3391618870 shutdown_connections 2024-09-16T15:47:01.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.470+0000 7fed3f600640 1 -- 172.21.15.151:0/3391618870 wait complete. 2024-09-16T15:47:01.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.472+0000 7fed3f600640 1 Processor -- start 2024-09-16T15:47:01.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.472+0000 7fed3f600640 1 -- start start 2024-09-16T15:47:01.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.473+0000 7fed3f600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed380b4ca0 0x7fed381f8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:01.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.473+0000 7fed3f600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fed380ccfd0 0x7fed3815e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:01.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.473+0000 7fed3f600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fed381445d0 0x7fed3809bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:01.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.473+0000 7fed3f600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fed3815f080 con 0x7fed381445d0 2024-09-16T15:47:01.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.473+0000 7fed3f600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fed38073bc0 con 0x7fed380b4ca0 2024-09-16T15:47:01.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.473+0000 7fed3f600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fed3815f200 con 0x7fed380ccfd0 2024-09-16T15:47:01.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.473+0000 7fed3d600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fed381445d0 0x7fed3809bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:01.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.473+0000 7fed3d600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fed381445d0 0x7fed3809bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:46236/0 (socket says 172.21.15.151:46236) 2024-09-16T15:47:01.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.473+0000 7fed3d600640 1 -- 172.21.15.151:0/1540703496 learned_addr learned my addr 172.21.15.151:0/1540703496 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:47:01.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.473+0000 7fed3cc00640 1 --2- 172.21.15.151:0/1540703496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed380b4ca0 0x7fed381f8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:01.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.473+0000 7fed37e00640 1 --2- 172.21.15.151:0/1540703496 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fed380ccfd0 0x7fed3815e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:01.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.474+0000 7fed3d600640 1 -- 172.21.15.151:0/1540703496 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fed380ccfd0 msgr2=0x7fed3815e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:01.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.474+0000 7fed3d600640 1 --2- 172.21.15.151:0/1540703496 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fed380ccfd0 0x7fed3815e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:01.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.474+0000 7fed3d600640 1 -- 172.21.15.151:0/1540703496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed380b4ca0 msgr2=0x7fed381f8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:01.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.474+0000 7fed3d600640 1 --2- 172.21.15.151:0/1540703496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed380b4ca0 0x7fed381f8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:01.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.474+0000 7fed3d600640 1 -- 172.21.15.151:0/1540703496 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed381f91c0 con 0x7fed381445d0 2024-09-16T15:47:01.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.474+0000 7fed3d600640 1 --2- 172.21.15.151:0/1540703496 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fed381445d0 0x7fed3809bff0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fed3000c400 tx=0x7fed3000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:01.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.474+0000 7fed34c00640 1 -- 172.21.15.151:0/1540703496 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed30017070 con 0x7fed381445d0 2024-09-16T15:47:01.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.474+0000 7fed34c00640 1 -- 172.21.15.151:0/1540703496 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fed300053c0 con 0x7fed381445d0 2024-09-16T15:47:01.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.474+0000 7fed3f600640 1 -- 172.21.15.151:0/1540703496 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed381e5170 con 0x7fed381445d0 2024-09-16T15:47:01.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.474+0000 7fed34c00640 1 -- 172.21.15.151:0/1540703496 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed30003850 con 0x7fed381445d0 2024-09-16T15:47:01.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.474+0000 7fed3f600640 1 -- 172.21.15.151:0/1540703496 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed381e5720 con 0x7fed381445d0 2024-09-16T15:47:01.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.475+0000 7fed3f600640 1 -- 172.21.15.151:0/1540703496 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed3815f6b0 con 0x7fed381445d0 2024-09-16T15:47:01.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.476+0000 7fed34c00640 1 -- 172.21.15.151:0/1540703496 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 24) ==== 102409+0+0 (secure 0 0 0) 0x7fed3001a070 con 0x7fed381445d0 2024-09-16T15:47:01.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.477+0000 7fed34c00640 1 --2- 172.21.15.151:0/1540703496 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fed180794b0 0x7fed1807b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:01.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.478+0000 7fed34c00640 1 -- 172.21.15.151:0/1540703496 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5765+0+0 (secure 0 0 0) 0x7fed30098920 con 0x7fed381445d0 2024-09-16T15:47:01.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.478+0000 7fed3cc00640 1 --2- 172.21.15.151:0/1540703496 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fed180794b0 0x7fed1807b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:01.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.479+0000 7fed3cc00640 1 --2- 172.21.15.151:0/1540703496 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fed180794b0 0x7fed1807b970 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fed28034000 tx=0x7fed28004ab0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:01.484 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.482+0000 7fed34c00640 1 -- 172.21.15.151:0/1540703496 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fed381f91c0 con 0x7fed381445d0 2024-09-16T15:47:01.690 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.688+0000 7fed3f600640 1 -- 172.21.15.151:0/1540703496 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fed38070a20 con 0x7fed180794b0 2024-09-16T15:47:01.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.690+0000 7fed34c00640 1 -- 172.21.15.151:0/1540703496 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fed38070a20 con 0x7fed180794b0 2024-09-16T15:47:01.692 INFO:teuthology.orchestra.run.smithi151.stdout:[ 2024-09-16T15:47:01.693 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:47:01.693 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "cephfs" 2024-09-16T15:47:01.693 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:47:01.693 INFO:teuthology.orchestra.run.smithi151.stdout:] 2024-09-16T15:47:01.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.693+0000 7fed3f600640 1 -- 172.21.15.151:0/1540703496 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fed180794b0 msgr2=0x7fed1807b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:01.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.693+0000 7fed3f600640 1 --2- 172.21.15.151:0/1540703496 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fed180794b0 0x7fed1807b970 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fed28034000 tx=0x7fed28004ab0 comp rx=0 tx=0).stop 2024-09-16T15:47:01.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.693+0000 7fed3f600640 1 -- 172.21.15.151:0/1540703496 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fed381445d0 msgr2=0x7fed3809bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:01.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.693+0000 7fed3f600640 1 --2- 172.21.15.151:0/1540703496 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fed381445d0 0x7fed3809bff0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fed3000c400 tx=0x7fed3000c8c0 comp rx=0 tx=0).stop 2024-09-16T15:47:01.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.693+0000 7fed3f600640 1 -- 172.21.15.151:0/1540703496 shutdown_connections 2024-09-16T15:47:01.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.693+0000 7fed3f600640 1 --2- 172.21.15.151:0/1540703496 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fed180794b0 0x7fed1807b970 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:01.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.693+0000 7fed3f600640 1 --2- 172.21.15.151:0/1540703496 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fed381445d0 0x7fed3809bff0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:01.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.693+0000 7fed3f600640 1 --2- 172.21.15.151:0/1540703496 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fed380ccfd0 0x7fed3815e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:01.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.693+0000 7fed3f600640 1 --2- 172.21.15.151:0/1540703496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed380b4ca0 0x7fed381f8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:01.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.693+0000 7fed3f600640 1 -- 172.21.15.151:0/1540703496 >> 172.21.15.151:0/1540703496 conn(0x7fed380b1950 msgr2=0x7fed380afea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:01.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.693+0000 7fed3f600640 1 -- 172.21.15.151:0/1540703496 shutdown_connections 2024-09-16T15:47:01.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.693+0000 7fed3f600640 1 -- 172.21.15.151:0/1540703496 wait complete. 2024-09-16T15:47:01.711 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-16T15:47:01.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.871+0000 7f7fe1600640 1 -- 172.21.15.151:0/2805581418 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7fc0003a50 msgr2=0x7f7fc0003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:01.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.871+0000 7f7fe1600640 1 --2- 172.21.15.151:0/2805581418 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7fc0003a50 0x7f7fc0003e70 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc0af630 tx=0x7f7fc0005830 comp rx=0 tx=0).stop 2024-09-16T15:47:01.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.872+0000 7f7fe1600640 1 -- 172.21.15.151:0/2805581418 shutdown_connections 2024-09-16T15:47:01.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.872+0000 7f7fe1600640 1 --2- 172.21.15.151:0/2805581418 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7fc0003a50 0x7f7fc0003e70 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:01.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.872+0000 7f7fe1600640 1 -- 172.21.15.151:0/2805581418 >> 172.21.15.151:0/2805581418 conn(0x7f7fdc0b1840 msgr2=0x7f7fdc0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:01.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.872+0000 7f7fe1600640 1 -- 172.21.15.151:0/2805581418 shutdown_connections 2024-09-16T15:47:01.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.872+0000 7f7fe1600640 1 -- 172.21.15.151:0/2805581418 wait complete. 2024-09-16T15:47:01.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.873+0000 7f7fe1600640 1 Processor -- start 2024-09-16T15:47:01.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.873+0000 7f7fe1600640 1 -- start start 2024-09-16T15:47:01.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.874+0000 7f7fe1600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7fc0003a50 0x7f7fdc1466d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:01.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.874+0000 7f7fe1600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7fdc15a890 0x7f7fdc15ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:01.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.874+0000 7f7fe1600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7fdc1e7f10 0x7f7fdc1e8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:01.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.874+0000 7f7fe1600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7fdc0ce940 con 0x7f7fdc15a890 2024-09-16T15:47:01.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.874+0000 7f7fe1600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7fdc0ce7c0 con 0x7f7fc0003a50 2024-09-16T15:47:01.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.874+0000 7f7fe1600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f7fdc0cf070 con 0x7f7fdc1e7f10 2024-09-16T15:47:01.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.875+0000 7f7fda000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7fdc15a890 0x7f7fdc15ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:01.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.875+0000 7f7fda000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7fdc15a890 0x7f7fdc15ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:46250/0 (socket says 172.21.15.151:46250) 2024-09-16T15:47:01.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.875+0000 7f7fda000640 1 -- 172.21.15.151:0/1677351274 learned_addr learned my addr 172.21.15.151:0/1677351274 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:47:01.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.875+0000 7f7fdaa00640 1 --2- 172.21.15.151:0/1677351274 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7fc0003a50 0x7f7fdc1466d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:01.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.875+0000 7f7fdb400640 1 --2- 172.21.15.151:0/1677351274 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7fdc1e7f10 0x7f7fdc1e8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:01.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.875+0000 7f7fda000640 1 -- 172.21.15.151:0/1677351274 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7fdc1e7f10 msgr2=0x7f7fdc1e8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:01.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.875+0000 7f7fda000640 1 --2- 172.21.15.151:0/1677351274 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7fdc1e7f10 0x7f7fdc1e8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:01.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.875+0000 7f7fda000640 1 -- 172.21.15.151:0/1677351274 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7fc0003a50 msgr2=0x7f7fdc1466d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:01.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.875+0000 7f7fda000640 1 --2- 172.21.15.151:0/1677351274 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7fc0003a50 0x7f7fdc1466d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:01.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.875+0000 7f7fda000640 1 -- 172.21.15.151:0/1677351274 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fdc15f1c0 con 0x7f7fdc15a890 2024-09-16T15:47:01.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.875+0000 7f7fda000640 1 --2- 172.21.15.151:0/1677351274 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7fdc15a890 0x7f7fdc15ad10 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8011a10 tx=0x7f7fc8011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:01.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.876+0000 7f7fd2a00640 1 -- 172.21.15.151:0/1677351274 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc801e070 con 0x7f7fdc15a890 2024-09-16T15:47:01.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.876+0000 7f7fe1600640 1 -- 172.21.15.151:0/1677351274 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fdc15f4a0 con 0x7f7fdc15a890 2024-09-16T15:47:01.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.876+0000 7f7fd2a00640 1 -- 172.21.15.151:0/1677351274 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7fc8004460 con 0x7f7fdc15a890 2024-09-16T15:47:01.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.876+0000 7f7fd2a00640 1 -- 172.21.15.151:0/1677351274 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc800fc00 con 0x7f7fdc15a890 2024-09-16T15:47:01.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.876+0000 7f7fe1600640 1 -- 172.21.15.151:0/1677351274 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fdc1e8c80 con 0x7f7fdc15a890 2024-09-16T15:47:01.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.877+0000 7f7fe1600640 1 -- 172.21.15.151:0/1677351274 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fdc0b2b40 con 0x7f7fdc15a890 2024-09-16T15:47:01.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.878+0000 7f7fd2a00640 1 -- 172.21.15.151:0/1677351274 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 24) ==== 102409+0+0 (secure 0 0 0) 0x7f7fc8021070 con 0x7f7fdc15a890 2024-09-16T15:47:01.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.879+0000 7f7fd2a00640 1 --2- 172.21.15.151:0/1677351274 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7fbc0794b0 0x7f7fbc07b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:01.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.879+0000 7f7fd2a00640 1 -- 172.21.15.151:0/1677351274 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5765+0+0 (secure 0 0 0) 0x7f7fc8099cb0 con 0x7f7fdc15a890 2024-09-16T15:47:01.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.880+0000 7f7fdaa00640 1 --2- 172.21.15.151:0/1677351274 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7fbc0794b0 0x7f7fbc07b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:01.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.880+0000 7f7fdaa00640 1 --2- 172.21.15.151:0/1677351274 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7fbc0794b0 0x7f7fbc07b970 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc033000 tx=0x7f7fdc0750b0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:01.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:01.883+0000 7f7fd2a00640 1 -- 172.21.15.151:0/1677351274 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7fc80636a0 con 0x7f7fdc15a890 2024-09-16T15:47:02.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.069+0000 7f7fe1600640 1 -- 172.21.15.151:0/1677351274 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f7fdc0b3d20 con 0x7f7fdc15a890 2024-09-16T15:47:02.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.069+0000 7f7fd2a00640 1 -- 172.21.15.151:0/1677351274 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v37)=0 v37) ==== 122+0+0 (secure 0 0 0) 0x7f7fc80680d0 con 0x7f7fdc15a890 2024-09-16T15:47:02.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.071+0000 7f7fe1600640 1 -- 172.21.15.151:0/1677351274 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7fbc0794b0 msgr2=0x7f7fbc07b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:02.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.071+0000 7f7fe1600640 1 --2- 172.21.15.151:0/1677351274 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7fbc0794b0 0x7f7fbc07b970 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc033000 tx=0x7f7fdc0750b0 comp rx=0 tx=0).stop 2024-09-16T15:47:02.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.071+0000 7f7fe1600640 1 -- 172.21.15.151:0/1677351274 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7fdc15a890 msgr2=0x7f7fdc15ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:02.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.071+0000 7f7fe1600640 1 --2- 172.21.15.151:0/1677351274 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7fdc15a890 0x7f7fdc15ad10 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8011a10 tx=0x7f7fc8011ed0 comp rx=0 tx=0).stop 2024-09-16T15:47:02.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.072+0000 7f7fe1600640 1 -- 172.21.15.151:0/1677351274 shutdown_connections 2024-09-16T15:47:02.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.072+0000 7f7fe1600640 1 --2- 172.21.15.151:0/1677351274 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7fbc0794b0 0x7f7fbc07b970 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:02.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.072+0000 7f7fe1600640 1 --2- 172.21.15.151:0/1677351274 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7fdc1e7f10 0x7f7fdc1e8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:02.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.072+0000 7f7fe1600640 1 --2- 172.21.15.151:0/1677351274 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7fdc15a890 0x7f7fdc15ad10 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:02.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.072+0000 7f7fe1600640 1 --2- 172.21.15.151:0/1677351274 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7fc0003a50 0x7f7fdc1466d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:02.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.072+0000 7f7fe1600640 1 -- 172.21.15.151:0/1677351274 >> 172.21.15.151:0/1677351274 conn(0x7f7fdc0b1840 msgr2=0x7f7fdc0cdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:02.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.072+0000 7f7fe1600640 1 -- 172.21.15.151:0/1677351274 shutdown_connections 2024-09-16T15:47:02.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.072+0000 7f7fe1600640 1 -- 172.21.15.151:0/1677351274 wait complete. 2024-09-16T15:47:02.090 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mgr mgr/volumes/max_concurrent_clones 2024-09-16T15:47:02.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.248+0000 7f0912400640 1 -- 172.21.15.151:0/2386202637 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f090c145750 msgr2=0x7f090c073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:02.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.248+0000 7f0912400640 1 --2- 172.21.15.151:0/2386202637 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f090c145750 0x7f090c073440 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f08fc009830 tx=0x7f08fc02a480 comp rx=0 tx=0).stop 2024-09-16T15:47:02.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.248+0000 7f0912400640 1 -- 172.21.15.151:0/2386202637 shutdown_connections 2024-09-16T15:47:02.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.248+0000 7f0912400640 1 --2- 172.21.15.151:0/2386202637 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f090c0cccd0 0x7f090c0cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:02.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.248+0000 7f0912400640 1 --2- 172.21.15.151:0/2386202637 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f090c145750 0x7f090c073440 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:02.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.248+0000 7f0912400640 1 -- 172.21.15.151:0/2386202637 >> 172.21.15.151:0/2386202637 conn(0x7f090c0b1840 msgr2=0x7f090c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:02.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.248+0000 7f0912400640 1 -- 172.21.15.151:0/2386202637 shutdown_connections 2024-09-16T15:47:02.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.248+0000 7f0912400640 1 -- 172.21.15.151:0/2386202637 wait complete. 2024-09-16T15:47:02.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.250+0000 7f0912400640 1 Processor -- start 2024-09-16T15:47:02.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.250+0000 7f0912400640 1 -- start start 2024-09-16T15:47:02.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.251+0000 7f0912400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f090c06efa0 0x7f090c15e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:02.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.251+0000 7f0912400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f090c0cccd0 0x7f090c1427d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:02.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.251+0000 7f0912400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f090c145750 0x7f090c13fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:02.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.251+0000 7f0912400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f090c0ce480 con 0x7f090c145750 2024-09-16T15:47:02.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.251+0000 7f0912400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f090c0ce300 con 0x7f090c06efa0 2024-09-16T15:47:02.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.251+0000 7f0912400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f090c0ce600 con 0x7f090c0cccd0 2024-09-16T15:47:02.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.251+0000 7f090be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f090c145750 0x7f090c13fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:02.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.251+0000 7f090be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f090c145750 0x7f090c13fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:46258/0 (socket says 172.21.15.151:46258) 2024-09-16T15:47:02.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.251+0000 7f090be00640 1 -- 172.21.15.151:0/2857445243 learned_addr learned my addr 172.21.15.151:0/2857445243 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:47:02.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.251+0000 7f090b400640 1 --2- 172.21.15.151:0/2857445243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f090c06efa0 0x7f090c15e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:02.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.251+0000 7f090aa00640 1 --2- 172.21.15.151:0/2857445243 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f090c0cccd0 0x7f090c1427d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:02.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.252+0000 7f090be00640 1 -- 172.21.15.151:0/2857445243 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f090c0cccd0 msgr2=0x7f090c1427d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:02.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.252+0000 7f090be00640 1 --2- 172.21.15.151:0/2857445243 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f090c0cccd0 0x7f090c1427d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:02.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.252+0000 7f090be00640 1 -- 172.21.15.151:0/2857445243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f090c06efa0 msgr2=0x7f090c15e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:02.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.252+0000 7f090be00640 1 --2- 172.21.15.151:0/2857445243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f090c06efa0 0x7f090c15e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:02.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.252+0000 7f090be00640 1 -- 172.21.15.151:0/2857445243 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f090c0aa7b0 con 0x7f090c145750 2024-09-16T15:47:02.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.252+0000 7f090be00640 1 --2- 172.21.15.151:0/2857445243 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f090c145750 0x7f090c13fc90 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f090400eaa0 tx=0x7f090400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:02.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.252+0000 7f0903400640 1 -- 172.21.15.151:0/2857445243 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f090400cdb0 con 0x7f090c145750 2024-09-16T15:47:02.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.252+0000 7f0903400640 1 -- 172.21.15.151:0/2857445243 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0904004c10 con 0x7f090c145750 2024-09-16T15:47:02.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.252+0000 7f0912400640 1 -- 172.21.15.151:0/2857445243 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f090c15ec90 con 0x7f090c145750 2024-09-16T15:47:02.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.252+0000 7f0903400640 1 -- 172.21.15.151:0/2857445243 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0904005020 con 0x7f090c145750 2024-09-16T15:47:02.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.252+0000 7f0912400640 1 -- 172.21.15.151:0/2857445243 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f090c0ad100 con 0x7f090c145750 2024-09-16T15:47:02.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.253+0000 7f0912400640 1 -- 172.21.15.151:0/2857445243 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f090c0cdbb0 con 0x7f090c145750 2024-09-16T15:47:02.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.254+0000 7f0903400640 1 -- 172.21.15.151:0/2857445243 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 24) ==== 102409+0+0 (secure 0 0 0) 0x7f09040051c0 con 0x7f090c145750 2024-09-16T15:47:02.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.255+0000 7f0903400640 1 --2- 172.21.15.151:0/2857445243 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f08ec0794b0 0x7f08ec07b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:02.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.255+0000 7f0903400640 1 -- 172.21.15.151:0/2857445243 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5765+0+0 (secure 0 0 0) 0x7f0904099d50 con 0x7f090c145750 2024-09-16T15:47:02.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.256+0000 7f090b400640 1 --2- 172.21.15.151:0/2857445243 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f08ec0794b0 0x7f08ec07b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:02.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.256+0000 7f090b400640 1 --2- 172.21.15.151:0/2857445243 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f08ec0794b0 0x7f08ec07b970 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f08fc034000 tx=0x7f08fc004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:02.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.259+0000 7f0903400640 1 -- 172.21.15.151:0/2857445243 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0904016030 con 0x7f090c145750 2024-09-16T15:47:02.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.445+0000 7f0912400640 1 -- 172.21.15.151:0/2857445243 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"} v 0) -- 0x7f090c0cf5d0 con 0x7f090c145750 2024-09-16T15:47:02.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.446+0000 7f0903400640 1 -- 172.21.15.151:0/2857445243 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"}]=0 v37) ==== 116+0+2 (secure 0 0 0) 0x7f0904063e10 con 0x7f090c145750 2024-09-16T15:47:02.448 INFO:teuthology.orchestra.run.smithi151.stdout:4 2024-09-16T15:47:02.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.448+0000 7f0912400640 1 -- 172.21.15.151:0/2857445243 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f08ec0794b0 msgr2=0x7f08ec07b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:02.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.448+0000 7f0912400640 1 --2- 172.21.15.151:0/2857445243 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f08ec0794b0 0x7f08ec07b970 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f08fc034000 tx=0x7f08fc004730 comp rx=0 tx=0).stop 2024-09-16T15:47:02.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.448+0000 7f0912400640 1 -- 172.21.15.151:0/2857445243 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f090c145750 msgr2=0x7f090c13fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:02.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.449+0000 7f0912400640 1 --2- 172.21.15.151:0/2857445243 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f090c145750 0x7f090c13fc90 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f090400eaa0 tx=0x7f090400ef60 comp rx=0 tx=0).stop 2024-09-16T15:47:02.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.449+0000 7f0912400640 1 -- 172.21.15.151:0/2857445243 shutdown_connections 2024-09-16T15:47:02.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.449+0000 7f0912400640 1 --2- 172.21.15.151:0/2857445243 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f08ec0794b0 0x7f08ec07b970 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:02.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.449+0000 7f0912400640 1 --2- 172.21.15.151:0/2857445243 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f090c145750 0x7f090c13fc90 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:02.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.449+0000 7f0912400640 1 --2- 172.21.15.151:0/2857445243 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f090c0cccd0 0x7f090c1427d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:02.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.449+0000 7f0912400640 1 --2- 172.21.15.151:0/2857445243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f090c06efa0 0x7f090c15e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:02.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.449+0000 7f0912400640 1 -- 172.21.15.151:0/2857445243 >> 172.21.15.151:0/2857445243 conn(0x7f090c0b1840 msgr2=0x7f090c0a98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:02.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.449+0000 7f0912400640 1 -- 172.21.15.151:0/2857445243 shutdown_connections 2024-09-16T15:47:02.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.449+0000 7f0912400640 1 -- 172.21.15.151:0/2857445243 wait complete. 2024-09-16T15:47:02.467 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-16T15:47:02.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.625+0000 7ff002c00640 1 -- 172.21.15.151:0/2293138783 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feffc06efa0 msgr2=0x7feffc0cc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:02.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.625+0000 7ff002c00640 1 --2- 172.21.15.151:0/2293138783 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feffc06efa0 0x7feffc0cc5c0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7feffc0d0cd0 tx=0x7fefe802a4d0 comp rx=0 tx=0).stop 2024-09-16T15:47:02.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.625+0000 7ff002c00640 1 -- 172.21.15.151:0/2293138783 shutdown_connections 2024-09-16T15:47:02.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.625+0000 7ff002c00640 1 --2- 172.21.15.151:0/2293138783 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feffc0ccb00 0x7feffc15f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:02.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.625+0000 7ff002c00640 1 --2- 172.21.15.151:0/2293138783 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feffc06efa0 0x7feffc0cc5c0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:02.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.625+0000 7ff002c00640 1 --2- 172.21.15.151:0/2293138783 >> v2:172.21.15.167:3300/0 conn(0x7feffc145750 0x7feffc073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:02.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.625+0000 7ff002c00640 1 -- 172.21.15.151:0/2293138783 >> 172.21.15.151:0/2293138783 conn(0x7feffc0b1840 msgr2=0x7feffc0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:02.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.625+0000 7ff002c00640 1 -- 172.21.15.151:0/2293138783 shutdown_connections 2024-09-16T15:47:02.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.625+0000 7ff002c00640 1 -- 172.21.15.151:0/2293138783 wait complete. 2024-09-16T15:47:02.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.627+0000 7ff002c00640 1 Processor -- start 2024-09-16T15:47:02.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.627+0000 7ff002c00640 1 -- start start 2024-09-16T15:47:02.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.628+0000 7ff002c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feffc06efa0 0x7feffc0acc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:02.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.628+0000 7ff002c00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feffc0ccb00 0x7feffc1416f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:02.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.628+0000 7ff002c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feffc145750 0x7feffc0cb880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:02.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.628+0000 7ff002c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7feffc1602c0 con 0x7feffc145750 2024-09-16T15:47:02.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.628+0000 7ff002c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7feffc160140 con 0x7feffc06efa0 2024-09-16T15:47:02.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.628+0000 7ff002c00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7feffc160440 con 0x7feffc0ccb00 2024-09-16T15:47:02.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.628+0000 7ff000c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feffc145750 0x7feffc0cb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:02.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.628+0000 7ff000c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feffc145750 0x7feffc0cb880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:46276/0 (socket says 172.21.15.151:46276) 2024-09-16T15:47:02.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.628+0000 7ff000c00640 1 -- 172.21.15.151:0/929880726 learned_addr learned my addr 172.21.15.151:0/929880726 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:47:02.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.629+0000 7feffb400640 1 --2- 172.21.15.151:0/929880726 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feffc0ccb00 0x7feffc1416f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:02.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.629+0000 7feffbe00640 1 --2- 172.21.15.151:0/929880726 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feffc06efa0 0x7feffc0acc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:02.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.629+0000 7ff000c00640 1 -- 172.21.15.151:0/929880726 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feffc0ccb00 msgr2=0x7feffc1416f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:02.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.629+0000 7ff000c00640 1 --2- 172.21.15.151:0/929880726 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feffc0ccb00 0x7feffc1416f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:02.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.629+0000 7ff000c00640 1 -- 172.21.15.151:0/929880726 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feffc06efa0 msgr2=0x7feffc0acc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:02.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.629+0000 7ff000c00640 1 --2- 172.21.15.151:0/929880726 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feffc06efa0 0x7feffc0acc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:02.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.629+0000 7ff000c00640 1 -- 172.21.15.151:0/929880726 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feffc0ad650 con 0x7feffc145750 2024-09-16T15:47:02.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.629+0000 7feffbe00640 1 --2- 172.21.15.151:0/929880726 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feffc06efa0 0x7feffc0acc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:47:02.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.629+0000 7ff000c00640 1 --2- 172.21.15.151:0/929880726 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feffc145750 0x7feffc0cb880 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7feff400c970 tx=0x7feff400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:02.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.629+0000 7feff3e00640 1 -- 172.21.15.151:0/929880726 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff4007bb0 con 0x7feffc145750 2024-09-16T15:47:02.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.629+0000 7feff3e00640 1 -- 172.21.15.151:0/929880726 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feff400f450 con 0x7feffc145750 2024-09-16T15:47:02.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.629+0000 7feff3e00640 1 -- 172.21.15.151:0/929880726 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff4007e60 con 0x7feffc145750 2024-09-16T15:47:02.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.629+0000 7ff002c00640 1 -- 172.21.15.151:0/929880726 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feffc0ad930 con 0x7feffc145750 2024-09-16T15:47:02.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.629+0000 7ff002c00640 1 -- 172.21.15.151:0/929880726 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feffc0adef0 con 0x7feffc145750 2024-09-16T15:47:02.634 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.630+0000 7ff002c00640 1 -- 172.21.15.151:0/929880726 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feffc15f9f0 con 0x7feffc145750 2024-09-16T15:47:02.634 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.632+0000 7feff3e00640 1 -- 172.21.15.151:0/929880726 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 24) ==== 102409+0+0 (secure 0 0 0) 0x7feff400f8a0 con 0x7feffc145750 2024-09-16T15:47:02.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.633+0000 7feff3e00640 1 --2- 172.21.15.151:0/929880726 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fefdc0794b0 0x7fefdc07b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:02.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.633+0000 7feff3e00640 1 -- 172.21.15.151:0/929880726 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5765+0+0 (secure 0 0 0) 0x7feff4099b40 con 0x7feffc145750 2024-09-16T15:47:02.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.634+0000 7feffbe00640 1 --2- 172.21.15.151:0/929880726 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fefdc0794b0 0x7fefdc07b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:02.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.634+0000 7feffbe00640 1 --2- 172.21.15.151:0/929880726 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fefdc0794b0 0x7fefdc07b970 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fefec005fd0 tx=0x7fefec0082b0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:02.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.637+0000 7feff3e00640 1 -- 172.21.15.151:0/929880726 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7feff4014030 con 0x7feffc145750 2024-09-16T15:47:02.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.822+0000 7ff002c00640 1 -- 172.21.15.151:0/929880726 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7feffc0a9460 con 0x7feffc145750 2024-09-16T15:47:02.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.826+0000 7feff3e00640 1 -- 172.21.15.151:0/929880726 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v38)=0 v38) ==== 131+0+0 (secure 0 0 0) 0x7feff4063cc0 con 0x7feffc145750 2024-09-16T15:47:02.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.829+0000 7ff002c00640 1 -- 172.21.15.151:0/929880726 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fefdc0794b0 msgr2=0x7fefdc07b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:02.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.829+0000 7ff002c00640 1 --2- 172.21.15.151:0/929880726 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fefdc0794b0 0x7fefdc07b970 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fefec005fd0 tx=0x7fefec0082b0 comp rx=0 tx=0).stop 2024-09-16T15:47:02.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.829+0000 7ff002c00640 1 -- 172.21.15.151:0/929880726 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feffc145750 msgr2=0x7feffc0cb880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:02.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.829+0000 7ff002c00640 1 --2- 172.21.15.151:0/929880726 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feffc145750 0x7feffc0cb880 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7feff400c970 tx=0x7feff400ce30 comp rx=0 tx=0).stop 2024-09-16T15:47:02.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.829+0000 7ff002c00640 1 -- 172.21.15.151:0/929880726 shutdown_connections 2024-09-16T15:47:02.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.829+0000 7ff002c00640 1 --2- 172.21.15.151:0/929880726 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fefdc0794b0 0x7fefdc07b970 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:02.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.829+0000 7ff002c00640 1 --2- 172.21.15.151:0/929880726 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feffc145750 0x7feffc0cb880 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:02.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.829+0000 7ff002c00640 1 --2- 172.21.15.151:0/929880726 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feffc0ccb00 0x7feffc1416f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:02.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.829+0000 7ff002c00640 1 --2- 172.21.15.151:0/929880726 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feffc06efa0 0x7feffc0acc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:02.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.829+0000 7ff002c00640 1 -- 172.21.15.151:0/929880726 >> 172.21.15.151:0/929880726 conn(0x7feffc0b1840 msgr2=0x7feffc0af9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:02.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.829+0000 7ff002c00640 1 -- 172.21.15.151:0/929880726 shutdown_connections 2024-09-16T15:47:02.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:02.829+0000 7ff002c00640 1 -- 172.21.15.151:0/929880726 wait complete. 2024-09-16T15:47:02.847 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:47:02.846+0000 7f48f2600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:02.847 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:47:02.846+0000 7f48f2600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:02.847 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:47:02.846+0000 7f48f2600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:02.847 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:47:02.846+0000 7f48f2600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:02.847 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:47:02.846+0000 7f48f2600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:02.849 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/snapshot_clone_no_wait false 2024-09-16T15:47:03.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.009+0000 7ff53b200640 1 -- 172.21.15.151:0/3490367581 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff53c15e870 msgr2=0x7ff53c0ae5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:03.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.009+0000 7ff53b200640 1 --2- 172.21.15.151:0/3490367581 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff53c15e870 0x7ff53c0ae5e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff52c009860 tx=0x7ff52c02a4b0 comp rx=0 tx=0).stop 2024-09-16T15:47:03.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.009+0000 7ff53b200640 1 -- 172.21.15.151:0/3490367581 shutdown_connections 2024-09-16T15:47:03.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.009+0000 7ff53b200640 1 --2- 172.21.15.151:0/3490367581 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff53c15f360 0x7ff53c0739a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:03.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.009+0000 7ff53b200640 1 --2- 172.21.15.151:0/3490367581 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff53c15e870 0x7ff53c0ae5e0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:03.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.009+0000 7ff53b200640 1 -- 172.21.15.151:0/3490367581 >> 172.21.15.151:0/3490367581 conn(0x7ff53c0cf9f0 msgr2=0x7ff53c0cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:03.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.009+0000 7ff53b200640 1 -- 172.21.15.151:0/3490367581 shutdown_connections 2024-09-16T15:47:03.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.009+0000 7ff53b200640 1 -- 172.21.15.151:0/3490367581 wait complete. 2024-09-16T15:47:03.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.011+0000 7ff53b200640 1 Processor -- start 2024-09-16T15:47:03.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.011+0000 7ff53b200640 1 -- start start 2024-09-16T15:47:03.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.012+0000 7ff53b200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff53c06e880 0x7ff53c0b52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:03.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.012+0000 7ff53b200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff53c15e870 0x7ff53c14d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:03.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.012+0000 7ff53b200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff53c15f360 0x7ff53c14def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:03.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.012+0000 7ff53b200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff53c074060 con 0x7ff53c06e880 2024-09-16T15:47:03.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.012+0000 7ff53b200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff53c073ee0 con 0x7ff53c15e870 2024-09-16T15:47:03.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.012+0000 7ff53b200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7ff53c0741e0 con 0x7ff53c15f360 2024-09-16T15:47:03.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.012+0000 7ff539e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff53c06e880 0x7ff53c0b52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:03.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.012+0000 7ff539e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff53c06e880 0x7ff53c0b52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44900/0 (socket says 172.21.15.151:44900) 2024-09-16T15:47:03.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.012+0000 7ff539e00640 1 -- 172.21.15.151:0/2859087759 learned_addr learned my addr 172.21.15.151:0/2859087759 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:47:03.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.013+0000 7ff539400640 1 --2- 172.21.15.151:0/2859087759 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff53c15e870 0x7ff53c14d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:03.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.013+0000 7ff53a800640 1 --2- 172.21.15.151:0/2859087759 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff53c15f360 0x7ff53c14def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:03.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.013+0000 7ff539e00640 1 -- 172.21.15.151:0/2859087759 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff53c15f360 msgr2=0x7ff53c14def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:03.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.013+0000 7ff539e00640 1 --2- 172.21.15.151:0/2859087759 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff53c15f360 0x7ff53c14def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:03.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.013+0000 7ff539e00640 1 -- 172.21.15.151:0/2859087759 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff53c15e870 msgr2=0x7ff53c14d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:03.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.013+0000 7ff539e00640 1 --2- 172.21.15.151:0/2859087759 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff53c15e870 0x7ff53c14d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:03.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.013+0000 7ff539e00640 1 -- 172.21.15.151:0/2859087759 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff53c14e430 con 0x7ff53c06e880 2024-09-16T15:47:03.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.013+0000 7ff539e00640 1 --2- 172.21.15.151:0/2859087759 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff53c06e880 0x7ff53c0b52c0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7ff52c000c00 tx=0x7ff52c032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:03.017 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.013+0000 7ff532000640 1 -- 172.21.15.151:0/2859087759 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff52c040070 con 0x7ff53c06e880 2024-09-16T15:47:03.017 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.014+0000 7ff532000640 1 -- 172.21.15.151:0/2859087759 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff52c0038e0 con 0x7ff53c06e880 2024-09-16T15:47:03.017 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.014+0000 7ff53b200640 1 -- 172.21.15.151:0/2859087759 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff53c0b3240 con 0x7ff53c06e880 2024-09-16T15:47:03.017 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.014+0000 7ff532000640 1 -- 172.21.15.151:0/2859087759 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff52c005c90 con 0x7ff53c06e880 2024-09-16T15:47:03.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.014+0000 7ff53b200640 1 -- 172.21.15.151:0/2859087759 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff53c0b37c0 con 0x7ff53c06e880 2024-09-16T15:47:03.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.015+0000 7ff53b200640 1 -- 172.21.15.151:0/2859087759 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff53c0ae5e0 con 0x7ff53c06e880 2024-09-16T15:47:03.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.016+0000 7ff532000640 1 -- 172.21.15.151:0/2859087759 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 24) ==== 102409+0+0 (secure 0 0 0) 0x7ff52c039040 con 0x7ff53c06e880 2024-09-16T15:47:03.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.017+0000 7ff532000640 1 --2- 172.21.15.151:0/2859087759 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff51c0794b0 0x7ff51c07b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:03.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.017+0000 7ff532000640 1 -- 172.21.15.151:0/2859087759 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5765+0+0 (secure 0 0 0) 0x7ff52c0bab50 con 0x7ff53c06e880 2024-09-16T15:47:03.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.017+0000 7ff539400640 1 --2- 172.21.15.151:0/2859087759 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff51c0794b0 0x7ff51c07b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:03.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.018+0000 7ff539400640 1 --2- 172.21.15.151:0/2859087759 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff51c0794b0 0x7ff51c07b970 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff528007fd0 tx=0x7ff528001410 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:03.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.021+0000 7ff532000640 1 -- 172.21.15.151:0/2859087759 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff52c084340 con 0x7ff53c06e880 2024-09-16T15:47:03.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.207+0000 7ff53b200640 1 -- 172.21.15.151:0/2859087759 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/snapshot_clone_no_wait}] v 0) -- 0x7ff53c0c3300 con 0x7ff53c06e880 2024-09-16T15:47:03.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.213+0000 7ff532000640 1 -- 172.21.15.151:0/2859087759 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/snapshot_clone_no_wait}]=0 v39)=0 v39) ==== 136+0+0 (secure 0 0 0) 0x7ff52c088d70 con 0x7ff53c06e880 2024-09-16T15:47:03.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.215+0000 7ff53b200640 1 -- 172.21.15.151:0/2859087759 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff51c0794b0 msgr2=0x7ff51c07b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:03.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.215+0000 7ff53b200640 1 --2- 172.21.15.151:0/2859087759 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff51c0794b0 0x7ff51c07b970 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff528007fd0 tx=0x7ff528001410 comp rx=0 tx=0).stop 2024-09-16T15:47:03.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.215+0000 7ff53b200640 1 -- 172.21.15.151:0/2859087759 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff53c06e880 msgr2=0x7ff53c0b52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:03.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.215+0000 7ff53b200640 1 --2- 172.21.15.151:0/2859087759 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff53c06e880 0x7ff53c0b52c0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7ff52c000c00 tx=0x7ff52c032c10 comp rx=0 tx=0).stop 2024-09-16T15:47:03.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.215+0000 7ff53b200640 1 -- 172.21.15.151:0/2859087759 shutdown_connections 2024-09-16T15:47:03.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.215+0000 7ff53b200640 1 --2- 172.21.15.151:0/2859087759 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff51c0794b0 0x7ff51c07b970 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:03.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.215+0000 7ff53b200640 1 --2- 172.21.15.151:0/2859087759 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff53c15e870 0x7ff53c14d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:03.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.215+0000 7ff53b200640 1 --2- 172.21.15.151:0/2859087759 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff53c15f360 0x7ff53c14def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:03.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.215+0000 7ff53b200640 1 --2- 172.21.15.151:0/2859087759 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff53c06e880 0x7ff53c0b52c0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:03.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.216+0000 7ff53b200640 1 -- 172.21.15.151:0/2859087759 >> 172.21.15.151:0/2859087759 conn(0x7ff53c0cf9f0 msgr2=0x7ff53c06ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:03.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.216+0000 7ff53b200640 1 -- 172.21.15.151:0/2859087759 shutdown_connections 2024-09-16T15:47:03.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.216+0000 7ff53b200640 1 -- 172.21.15.151:0/2859087759 wait complete. 2024-09-16T15:47:03.235 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-09-16T15:47:03.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.395+0000 7f7e9e000640 1 -- 172.21.15.151:0/3137479705 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e981512b0 msgr2=0x7f7e980cefb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:03.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.395+0000 7f7e9e000640 1 --2- 172.21.15.151:0/3137479705 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e981512b0 0x7f7e980cefb0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f7e88005fd0 tx=0x7f7e8802b1a0 comp rx=0 tx=0).stop 2024-09-16T15:47:03.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.395+0000 7f7e9e000640 1 -- 172.21.15.151:0/3137479705 shutdown_connections 2024-09-16T15:47:03.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.395+0000 7f7e9e000640 1 --2- 172.21.15.151:0/3137479705 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7e980b3e20 0x7f7e980cf4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:03.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.395+0000 7f7e9e000640 1 --2- 172.21.15.151:0/3137479705 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e981512b0 0x7f7e980cefb0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:03.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.395+0000 7f7e9e000640 1 -- 172.21.15.151:0/3137479705 >> 172.21.15.151:0/3137479705 conn(0x7f7e980b18e0 msgr2=0x7f7e980b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:03.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.395+0000 7f7e9e000640 1 -- 172.21.15.151:0/3137479705 shutdown_connections 2024-09-16T15:47:03.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.395+0000 7f7e9e000640 1 -- 172.21.15.151:0/3137479705 wait complete. 2024-09-16T15:47:03.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.397+0000 7f7e9e000640 1 Processor -- start 2024-09-16T15:47:03.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.397+0000 7f7e9e000640 1 -- start start 2024-09-16T15:47:03.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.398+0000 7f7e9e000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e9806fa00 0x7f7e981f4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:03.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.398+0000 7f7e9e000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7e980b3e20 0x7f7e9815dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:03.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.398+0000 7f7e9e000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7e981512b0 0x7f7e98074050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:03.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.398+0000 7f7e9e000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7e980cc750 con 0x7f7e9806fa00 2024-09-16T15:47:03.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.398+0000 7f7e9e000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7e980cc5d0 con 0x7f7e981512b0 2024-09-16T15:47:03.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.398+0000 7f7e9e000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f7e980cc8d0 con 0x7f7e980b3e20 2024-09-16T15:47:03.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.398+0000 7f7e97400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e9806fa00 0x7f7e981f4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:03.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.398+0000 7f7e97400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e9806fa00 0x7f7e981f4db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44912/0 (socket says 172.21.15.151:44912) 2024-09-16T15:47:03.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.398+0000 7f7e97400640 1 -- 172.21.15.151:0/2287594631 learned_addr learned my addr 172.21.15.151:0/2287594631 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:47:03.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.398+0000 7f7e96a00640 1 --2- 172.21.15.151:0/2287594631 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7e980b3e20 0x7f7e9815dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:03.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.398+0000 7f7e97e00640 1 --2- 172.21.15.151:0/2287594631 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7e981512b0 0x7f7e98074050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:03.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.398+0000 7f7e97400640 1 -- 172.21.15.151:0/2287594631 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7e980b3e20 msgr2=0x7f7e9815dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:03.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.398+0000 7f7e97400640 1 --2- 172.21.15.151:0/2287594631 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7e980b3e20 0x7f7e9815dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:03.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.398+0000 7f7e97400640 1 -- 172.21.15.151:0/2287594631 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7e981512b0 msgr2=0x7f7e98074050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:03.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.398+0000 7f7e97400640 1 --2- 172.21.15.151:0/2287594631 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7e981512b0 0x7f7e98074050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:03.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.398+0000 7f7e97400640 1 -- 172.21.15.151:0/2287594631 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e9815e3e0 con 0x7f7e9806fa00 2024-09-16T15:47:03.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.398+0000 7f7e97e00640 1 --2- 172.21.15.151:0/2287594631 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7e981512b0 0x7f7e98074050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:47:03.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.399+0000 7f7e97400640 1 --2- 172.21.15.151:0/2287594631 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e9806fa00 0x7f7e981f4db0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f7e880075d0 tx=0x7f7e880038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:03.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.399+0000 7f7e8f400640 1 -- 172.21.15.151:0/2287594631 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8802bae0 con 0x7f7e9806fa00 2024-09-16T15:47:03.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.399+0000 7f7e8f400640 1 -- 172.21.15.151:0/2287594631 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7e88005020 con 0x7f7e9806fa00 2024-09-16T15:47:03.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.399+0000 7f7e8f400640 1 -- 172.21.15.151:0/2287594631 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e88005d50 con 0x7f7e9806fa00 2024-09-16T15:47:03.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.399+0000 7f7e9e000640 1 -- 172.21.15.151:0/2287594631 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e980ae810 con 0x7f7e9806fa00 2024-09-16T15:47:03.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.399+0000 7f7e9e000640 1 -- 172.21.15.151:0/2287594631 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e980aedd0 con 0x7f7e9806fa00 2024-09-16T15:47:03.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.400+0000 7f7e9e000640 1 -- 172.21.15.151:0/2287594631 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e98069360 con 0x7f7e9806fa00 2024-09-16T15:47:03.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.401+0000 7f7e8f400640 1 -- 172.21.15.151:0/2287594631 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 24) ==== 102409+0+0 (secure 0 0 0) 0x7f7e88005450 con 0x7f7e9806fa00 2024-09-16T15:47:03.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.403+0000 7f7e8f400640 1 --2- 172.21.15.151:0/2287594631 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7e780794b0 0x7f7e7807b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:03.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.403+0000 7f7e8f400640 1 -- 172.21.15.151:0/2287594631 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5765+0+0 (secure 0 0 0) 0x7f7e88042070 con 0x7f7e9806fa00 2024-09-16T15:47:03.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.403+0000 7f7e96a00640 1 --2- 172.21.15.151:0/2287594631 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7e780794b0 0x7f7e7807b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:03.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.404+0000 7f7e96a00640 1 --2- 172.21.15.151:0/2287594631 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7e780794b0 0x7f7e7807b970 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7e98159c00 tx=0x7f7e8400b390 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:03.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.407+0000 7f7e8f400640 1 -- 172.21.15.151:0/2287594631 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7e88082d50 con 0x7f7e9806fa00 2024-09-16T15:47:03.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.616+0000 7f7e9e000640 1 -- 172.21.15.151:0/2287594631 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7f7e980cd490 con 0x7f7e780794b0 2024-09-16T15:47:03.629 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:47:03.628+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:03.629 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:47:03.628+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:03.629 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:47:03.628+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:03.629 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:47:03.628+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:03.629 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:47:03.628+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:03.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.665+0000 7f7e8f400640 1 -- 172.21.15.151:0/2287594631 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7e980cd490 con 0x7f7e780794b0 2024-09-16T15:47:03.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.668+0000 7f7e9e000640 1 -- 172.21.15.151:0/2287594631 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7e780794b0 msgr2=0x7f7e7807b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:03.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.668+0000 7f7e9e000640 1 --2- 172.21.15.151:0/2287594631 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7e780794b0 0x7f7e7807b970 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7e98159c00 tx=0x7f7e8400b390 comp rx=0 tx=0).stop 2024-09-16T15:47:03.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.668+0000 7f7e9e000640 1 -- 172.21.15.151:0/2287594631 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e9806fa00 msgr2=0x7f7e981f4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:03.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.668+0000 7f7e9e000640 1 --2- 172.21.15.151:0/2287594631 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e9806fa00 0x7f7e981f4db0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f7e880075d0 tx=0x7f7e880038a0 comp rx=0 tx=0).stop 2024-09-16T15:47:03.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.668+0000 7f7e9e000640 1 -- 172.21.15.151:0/2287594631 shutdown_connections 2024-09-16T15:47:03.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.668+0000 7f7e9e000640 1 --2- 172.21.15.151:0/2287594631 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7e780794b0 0x7f7e7807b970 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:03.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.668+0000 7f7e9e000640 1 --2- 172.21.15.151:0/2287594631 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7e981512b0 0x7f7e98074050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:03.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.668+0000 7f7e9e000640 1 --2- 172.21.15.151:0/2287594631 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7e980b3e20 0x7f7e9815dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:03.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.668+0000 7f7e9e000640 1 --2- 172.21.15.151:0/2287594631 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e9806fa00 0x7f7e981f4db0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:03.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.668+0000 7f7e9e000640 1 -- 172.21.15.151:0/2287594631 >> 172.21.15.151:0/2287594631 conn(0x7f7e980b18e0 msgr2=0x7f7e980adab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:03.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.668+0000 7f7e9e000640 1 -- 172.21.15.151:0/2287594631 shutdown_connections 2024-09-16T15:47:03.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.668+0000 7f7e9e000640 1 -- 172.21.15.151:0/2287594631 wait complete. 2024-09-16T15:47:03.686 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-16T15:47:03.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.847+0000 7fd1b3200640 1 -- 172.21.15.151:0/3294218037 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd198004800 msgr2=0x7fd198004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:03.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.847+0000 7fd1b3200640 1 --2- 172.21.15.151:0/3294218037 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd198004800 0x7fd198004c80 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fd19c0016c0 tx=0x7fd198005830 comp rx=0 tx=0).stop 2024-09-16T15:47:03.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.847+0000 7fd1b3200640 1 -- 172.21.15.151:0/3294218037 shutdown_connections 2024-09-16T15:47:03.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.847+0000 7fd1b3200640 1 --2- 172.21.15.151:0/3294218037 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd198004800 0x7fd198004c80 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:03.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.847+0000 7fd1b3200640 1 -- 172.21.15.151:0/3294218037 >> 172.21.15.151:0/3294218037 conn(0x7fd1b40b1510 msgr2=0x7fd1b40b18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:03.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.847+0000 7fd1b3200640 1 -- 172.21.15.151:0/3294218037 shutdown_connections 2024-09-16T15:47:03.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.847+0000 7fd1b3200640 1 -- 172.21.15.151:0/3294218037 wait complete. 2024-09-16T15:47:03.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.849+0000 7fd1b3200640 1 Processor -- start 2024-09-16T15:47:03.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.849+0000 7fd1b3200640 1 -- start start 2024-09-16T15:47:03.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.850+0000 7fd1b3200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd198004800 0x7fd1b414d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:03.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.850+0000 7fd1b3200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd1b40c3470 0x7fd1b40c38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:03.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.850+0000 7fd1b3200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1b4159b40 0x7fd1b4159ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:03.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.850+0000 7fd1b3200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b4073ec0 con 0x7fd1b40c3470 2024-09-16T15:47:03.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.850+0000 7fd1b3200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b4073d40 con 0x7fd1b4159b40 2024-09-16T15:47:03.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.850+0000 7fd1b3200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fd1b4074040 con 0x7fd198004800 2024-09-16T15:47:03.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.850+0000 7fd1abe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd1b40c3470 0x7fd1b40c38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:03.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.850+0000 7fd1abe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd1b40c3470 0x7fd1b40c38f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44932/0 (socket says 172.21.15.151:44932) 2024-09-16T15:47:03.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.850+0000 7fd1abe00640 1 -- 172.21.15.151:0/191822677 learned_addr learned my addr 172.21.15.151:0/191822677 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:47:03.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.850+0000 7fd1b2800640 1 --2- 172.21.15.151:0/191822677 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1b4159b40 0x7fd1b4159ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:03.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.850+0000 7fd1b1e00640 1 --2- 172.21.15.151:0/191822677 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd198004800 0x7fd1b414d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:03.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.850+0000 7fd1abe00640 1 -- 172.21.15.151:0/191822677 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd198004800 msgr2=0x7fd1b414d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:03.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.850+0000 7fd1abe00640 1 --2- 172.21.15.151:0/191822677 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd198004800 0x7fd1b414d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:03.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.850+0000 7fd1abe00640 1 -- 172.21.15.151:0/191822677 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1b4159b40 msgr2=0x7fd1b4159ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:03.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.850+0000 7fd1abe00640 1 --2- 172.21.15.151:0/191822677 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1b4159b40 0x7fd1b4159ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:03.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.850+0000 7fd1abe00640 1 -- 172.21.15.151:0/191822677 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1b414ded0 con 0x7fd1b40c3470 2024-09-16T15:47:03.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.851+0000 7fd1abe00640 1 --2- 172.21.15.151:0/191822677 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd1b40c3470 0x7fd1b40c38f0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fd19c0017e0 tx=0x7fd19c0063a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:03.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.851+0000 7fd1aa000640 1 -- 172.21.15.151:0/191822677 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd19c042070 con 0x7fd1b40c3470 2024-09-16T15:47:03.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.851+0000 7fd1aa000640 1 -- 172.21.15.151:0/191822677 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd19c039070 con 0x7fd1b40c3470 2024-09-16T15:47:03.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.851+0000 7fd1b3200640 1 -- 172.21.15.151:0/191822677 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1b41f7ad0 con 0x7fd1b40c3470 2024-09-16T15:47:03.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.851+0000 7fd1aa000640 1 -- 172.21.15.151:0/191822677 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd19c003d70 con 0x7fd1b40c3470 2024-09-16T15:47:03.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.851+0000 7fd1b3200640 1 -- 172.21.15.151:0/191822677 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1b41f8090 con 0x7fd1b40c3470 2024-09-16T15:47:03.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.852+0000 7fd1b3200640 1 -- 172.21.15.151:0/191822677 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1b40cf130 con 0x7fd1b40c3470 2024-09-16T15:47:03.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.853+0000 7fd1aa000640 1 -- 172.21.15.151:0/191822677 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 24) ==== 102409+0+0 (secure 0 0 0) 0x7fd19c03b070 con 0x7fd1b40c3470 2024-09-16T15:47:03.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.854+0000 7fd1aa000640 1 --2- 172.21.15.151:0/191822677 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd1900794e0 0x7fd19007b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:03.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.854+0000 7fd1aa000640 1 -- 172.21.15.151:0/191822677 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5765+0+0 (secure 0 0 0) 0x7fd19c0b88c0 con 0x7fd1b40c3470 2024-09-16T15:47:03.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.855+0000 7fd1b1e00640 1 --2- 172.21.15.151:0/191822677 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd1900794e0 0x7fd19007b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:03.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.855+0000 7fd1b1e00640 1 --2- 172.21.15.151:0/191822677 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd1900794e0 0x7fd19007b9a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4006cd0 tx=0x7fd1a40067a0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:03.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:03.858+0000 7fd1aa000640 1 -- 172.21.15.151:0/191822677 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd19c082070 con 0x7fd1b40c3470 2024-09-16T15:47:04.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:04.071+0000 7fd1b3200640 1 -- 172.21.15.151:0/191822677 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fd1b40c3fe0 con 0x7fd1900794e0 2024-09-16T15:47:04.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:04.078+0000 7fd1aa000640 1 -- 172.21.15.151:0/191822677 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7fd1a4006d00 con 0x7fd1900794e0 2024-09-16T15:47:04.079 INFO:teuthology.orchestra.run.smithi151.stdout:/volumes/_nogroup/sv1/0be48f7c-c0a2-473c-b88e-38e220cdcc14 2024-09-16T15:47:04.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:04.079+0000 7fd1b3200640 1 -- 172.21.15.151:0/191822677 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd1900794e0 msgr2=0x7fd19007b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:04.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:04.079+0000 7fd1b3200640 1 --2- 172.21.15.151:0/191822677 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd1900794e0 0x7fd19007b9a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4006cd0 tx=0x7fd1a40067a0 comp rx=0 tx=0).stop 2024-09-16T15:47:04.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:04.079+0000 7fd1b3200640 1 -- 172.21.15.151:0/191822677 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd1b40c3470 msgr2=0x7fd1b40c38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:04.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:04.079+0000 7fd1b3200640 1 --2- 172.21.15.151:0/191822677 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd1b40c3470 0x7fd1b40c38f0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fd19c0017e0 tx=0x7fd19c0063a0 comp rx=0 tx=0).stop 2024-09-16T15:47:04.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:04.080+0000 7fd1b3200640 1 -- 172.21.15.151:0/191822677 shutdown_connections 2024-09-16T15:47:04.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:04.080+0000 7fd1b3200640 1 --2- 172.21.15.151:0/191822677 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd1900794e0 0x7fd19007b9a0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:04.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:04.080+0000 7fd1b3200640 1 --2- 172.21.15.151:0/191822677 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1b4159b40 0x7fd1b4159ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:04.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:04.080+0000 7fd1b3200640 1 --2- 172.21.15.151:0/191822677 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd1b40c3470 0x7fd1b40c38f0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:04.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:04.080+0000 7fd1b3200640 1 --2- 172.21.15.151:0/191822677 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd198004800 0x7fd1b414d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:04.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:04.080+0000 7fd1b3200640 1 -- 172.21.15.151:0/191822677 >> 172.21.15.151:0/191822677 conn(0x7fd1b40b1510 msgr2=0x7fd1b40afb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:04.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:04.080+0000 7fd1b3200640 1 -- 172.21.15.151:0/191822677 shutdown_connections 2024-09-16T15:47:04.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:04.080+0000 7fd1b3200640 1 -- 172.21.15.151:0/191822677 wait complete. 2024-09-16T15:47:04.098 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/0be48f7c-c0a2-473c-b88e-38e220cdcc14 2024-09-16T15:47:04.098 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:47:04.098 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:47:04.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:47:04.135 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/0be48f7c-c0a2-473c-b88e-38e220cdcc14/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T15:47:12.585 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records in 2024-09-16T15:47:12.585 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records out 2024-09-16T15:47:12.585 INFO:teuthology.orchestra.run.smithi151.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.3682 s, 128 MB/s 2024-09-16T15:47:12.586 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:47:12.586 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:47:12.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:47:12.621 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/0be48f7c-c0a2-473c-b88e-38e220cdcc14/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T15:47:18.577 DEBUG:teuthology.orchestra.run.smithi151:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:47:18.585 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:47:20.535 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records in 2024-09-16T15:47:20.535 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records out 2024-09-16T15:47:20.535 INFO:teuthology.orchestra.run.smithi151.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.83618 s, 137 MB/s 2024-09-16T15:47:20.536 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:47:20.536 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:47:20.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:47:20.573 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/0be48f7c-c0a2-473c-b88e-38e220cdcc14/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T15:47:28.421 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records in 2024-09-16T15:47:28.421 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records out 2024-09-16T15:47:28.421 INFO:teuthology.orchestra.run.smithi151.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.77099 s, 138 MB/s 2024-09-16T15:47:28.422 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-09-16T15:47:28.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:28.584+0000 7f839f800640 1 -- 172.21.15.151:0/2015921725 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f839806e6f0 msgr2=0x7f839806eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:28.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:28.584+0000 7f839f800640 1 --2- 172.21.15.151:0/2015921725 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f839806e6f0 0x7f839806eb70 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f8390005fd0 tx=0x7f839002a540 comp rx=0 tx=0).stop 2024-09-16T15:47:28.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:28.584+0000 7f839f800640 1 -- 172.21.15.151:0/2015921725 shutdown_connections 2024-09-16T15:47:28.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:28.584+0000 7f839f800640 1 --2- 172.21.15.151:0/2015921725 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f839806e6f0 0x7f839806eb70 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:28.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:28.584+0000 7f839f800640 1 --2- 172.21.15.151:0/2015921725 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f839815f9a0 0x7f839815fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:28.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:28.584+0000 7f839f800640 1 -- 172.21.15.151:0/2015921725 >> 172.21.15.151:0/2015921725 conn(0x7f83980b18e0 msgr2=0x7f83980b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:28.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:28.584+0000 7f839f800640 1 -- 172.21.15.151:0/2015921725 shutdown_connections 2024-09-16T15:47:28.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:28.585+0000 7f839f800640 1 -- 172.21.15.151:0/2015921725 wait complete. 2024-09-16T15:47:28.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:28.586+0000 7f839f800640 1 Processor -- start 2024-09-16T15:47:28.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:28.586+0000 7f839f800640 1 -- start start 2024-09-16T15:47:28.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:28.587+0000 7f839f800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f839806e6f0 0x7f83980ced80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:28.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:28.587+0000 7f839f800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8398073de0 0x7f83981f68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:28.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:28.587+0000 7f839f800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f839815f9a0 0x7f839814f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:28.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:28.587+0000 7f839f800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f839806f0b0 con 0x7f8398073de0 2024-09-16T15:47:28.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:28.587+0000 7f839f800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f83980cd5e0 con 0x7f839815f9a0 2024-09-16T15:47:28.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:28.587+0000 7f839f800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f839806f230 con 0x7f839806e6f0 2024-09-16T15:47:28.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:28.588+0000 7f8397e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8398073de0 0x7f83981f68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:28.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:28.588+0000 7f839ce00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f839806e6f0 0x7f83980ced80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:28.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:28.588+0000 7f8397e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8398073de0 0x7f83981f68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:48078/0 (socket says 172.21.15.151:48078) 2024-09-16T15:47:28.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:28.588+0000 7f8397e00640 1 -- 172.21.15.151:0/439186900 learned_addr learned my addr 172.21.15.151:0/439186900 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:47:28.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:28.588+0000 7f839d800640 1 --2- 172.21.15.151:0/439186900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f839815f9a0 0x7f839814f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:28.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:28.588+0000 7f8397e00640 1 -- 172.21.15.151:0/439186900 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f839806e6f0 msgr2=0x7f83980ced80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:28.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:28.588+0000 7f8397e00640 1 --2- 172.21.15.151:0/439186900 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f839806e6f0 0x7f83980ced80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:28.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:28.588+0000 7f8397e00640 1 -- 172.21.15.151:0/439186900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f839815f9a0 msgr2=0x7f839814f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:28.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:28.588+0000 7f8397e00640 1 --2- 172.21.15.151:0/439186900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f839815f9a0 0x7f839814f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:28.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:28.588+0000 7f8397e00640 1 -- 172.21.15.151:0/439186900 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83981f2a40 con 0x7f8398073de0 2024-09-16T15:47:28.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:28.588+0000 7f8397e00640 1 --2- 172.21.15.151:0/439186900 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8398073de0 0x7f83981f68e0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f838400c790 tx=0x7f838400cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:28.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:28.589+0000 7f8394c00640 1 -- 172.21.15.151:0/439186900 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8384016020 con 0x7f8398073de0 2024-09-16T15:47:28.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:28.589+0000 7f8394c00640 1 -- 172.21.15.151:0/439186900 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8384003990 con 0x7f8398073de0 2024-09-16T15:47:28.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:28.589+0000 7f8394c00640 1 -- 172.21.15.151:0/439186900 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8384006e60 con 0x7f8398073de0 2024-09-16T15:47:28.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:28.589+0000 7f839f800640 1 -- 172.21.15.151:0/439186900 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83980cf710 con 0x7f8398073de0 2024-09-16T15:47:28.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:28.589+0000 7f839f800640 1 -- 172.21.15.151:0/439186900 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83981f2ce0 con 0x7f8398073de0 2024-09-16T15:47:28.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:28.590+0000 7f839f800640 1 -- 172.21.15.151:0/439186900 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8398069360 con 0x7f8398073de0 2024-09-16T15:47:28.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:28.591+0000 7f8394c00640 1 -- 172.21.15.151:0/439186900 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 25) ==== 102511+0+0 (secure 0 0 0) 0x7f838400a850 con 0x7f8398073de0 2024-09-16T15:47:28.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:28.592+0000 7f8394c00640 1 --2- 172.21.15.151:0/439186900 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f83780795d0 0x7f837807ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:28.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:28.592+0000 7f8394c00640 1 -- 172.21.15.151:0/439186900 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5881+0+0 (secure 0 0 0) 0x7f8384099280 con 0x7f8398073de0 2024-09-16T15:47:28.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:28.592+0000 7f839ce00640 1 --2- 172.21.15.151:0/439186900 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f83780795d0 0x7f837807ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:28.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:28.593+0000 7f839ce00640 1 --2- 172.21.15.151:0/439186900 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f83780795d0 0x7f837807ba90 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8388004630 tx=0x7f838800b2e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:28.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:28.596+0000 7f8394c00640 1 -- 172.21.15.151:0/439186900 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8384020020 con 0x7f8398073de0 2024-09-16T15:47:28.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:28.809+0000 7f839f800640 1 -- 172.21.15.151:0/439186900 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7f839806eb70 con 0x7f83780795d0 2024-09-16T15:47:28.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:28.822+0000 7f8394c00640 1 -- 172.21.15.151:0/439186900 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f839806eb70 con 0x7f83780795d0 2024-09-16T15:47:28.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:28.825+0000 7f839f800640 1 -- 172.21.15.151:0/439186900 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f83780795d0 msgr2=0x7f837807ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:28.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:28.825+0000 7f839f800640 1 --2- 172.21.15.151:0/439186900 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f83780795d0 0x7f837807ba90 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8388004630 tx=0x7f838800b2e0 comp rx=0 tx=0).stop 2024-09-16T15:47:28.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:28.825+0000 7f839f800640 1 -- 172.21.15.151:0/439186900 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8398073de0 msgr2=0x7f83981f68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:28.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:28.825+0000 7f839f800640 1 --2- 172.21.15.151:0/439186900 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8398073de0 0x7f83981f68e0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f838400c790 tx=0x7f838400cc50 comp rx=0 tx=0).stop 2024-09-16T15:47:28.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:28.825+0000 7f839f800640 1 -- 172.21.15.151:0/439186900 shutdown_connections 2024-09-16T15:47:28.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:28.825+0000 7f839f800640 1 --2- 172.21.15.151:0/439186900 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f83780795d0 0x7f837807ba90 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:28.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:28.825+0000 7f839f800640 1 --2- 172.21.15.151:0/439186900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f839815f9a0 0x7f839814f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:28.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:28.825+0000 7f839f800640 1 --2- 172.21.15.151:0/439186900 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8398073de0 0x7f83981f68e0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:28.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:28.825+0000 7f839f800640 1 --2- 172.21.15.151:0/439186900 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f839806e6f0 0x7f83980ced80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:28.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:28.825+0000 7f839f800640 1 -- 172.21.15.151:0/439186900 >> 172.21.15.151:0/439186900 conn(0x7f83980b18e0 msgr2=0x7f83980ad010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:28.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:28.825+0000 7f839f800640 1 -- 172.21.15.151:0/439186900 shutdown_connections 2024-09-16T15:47:28.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:28.825+0000 7f839f800640 1 -- 172.21.15.151:0/439186900 wait complete. 2024-09-16T15:47:28.842 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-16T15:47:29.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.004+0000 7f0cc5800640 1 -- 172.21.15.151:0/1539961352 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0cc00cccd0 msgr2=0x7f0cc0073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:29.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.004+0000 7f0cc5800640 1 --2- 172.21.15.151:0/1539961352 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0cc00cccd0 0x7f0cc0073440 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f0cac009830 tx=0x7f0cac02a480 comp rx=0 tx=0).stop 2024-09-16T15:47:29.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.004+0000 7f0cc5800640 1 -- 172.21.15.151:0/1539961352 shutdown_connections 2024-09-16T15:47:29.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.004+0000 7f0cc5800640 1 --2- 172.21.15.151:0/1539961352 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0cc00cccd0 0x7f0cc0073440 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:29.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.004+0000 7f0cc5800640 1 -- 172.21.15.151:0/1539961352 >> 172.21.15.151:0/1539961352 conn(0x7f0cc00b1840 msgr2=0x7f0cc00b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:29.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.004+0000 7f0cc5800640 1 -- 172.21.15.151:0/1539961352 shutdown_connections 2024-09-16T15:47:29.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.004+0000 7f0cc5800640 1 -- 172.21.15.151:0/1539961352 wait complete. 2024-09-16T15:47:29.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.006+0000 7f0cc5800640 1 Processor -- start 2024-09-16T15:47:29.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.006+0000 7f0cc5800640 1 -- start start 2024-09-16T15:47:29.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.007+0000 7f0cc5800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0cc006efa0 0x7f0cc0159c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:29.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.007+0000 7f0cc5800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0cc00cccd0 0x7f0cc00aea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:29.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.007+0000 7f0cc5800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0cc0145750 0x7f0cc00aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:29.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.007+0000 7f0cc5800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc00ce6a0 con 0x7f0cc00cccd0 2024-09-16T15:47:29.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.007+0000 7f0cc5800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc00ce520 con 0x7f0cc0145750 2024-09-16T15:47:29.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.007+0000 7f0cc5800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f0cc00ce820 con 0x7f0cc006efa0 2024-09-16T15:47:29.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.007+0000 7f0cbe000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0cc00cccd0 0x7f0cc00aea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:29.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.007+0000 7f0cbe000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0cc00cccd0 0x7f0cc00aea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:48108/0 (socket says 172.21.15.151:48108) 2024-09-16T15:47:29.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.007+0000 7f0cbe000640 1 -- 172.21.15.151:0/4224925978 learned_addr learned my addr 172.21.15.151:0/4224925978 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:47:29.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.007+0000 7f0cbf400640 1 --2- 172.21.15.151:0/4224925978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0cc0145750 0x7f0cc00aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:29.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.007+0000 7f0cbea00640 1 --2- 172.21.15.151:0/4224925978 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0cc006efa0 0x7f0cc0159c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:29.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.008+0000 7f0cbe000640 1 -- 172.21.15.151:0/4224925978 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0cc006efa0 msgr2=0x7f0cc0159c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:29.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.008+0000 7f0cbe000640 1 --2- 172.21.15.151:0/4224925978 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0cc006efa0 0x7f0cc0159c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:29.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.008+0000 7f0cbe000640 1 -- 172.21.15.151:0/4224925978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0cc0145750 msgr2=0x7f0cc00aefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:29.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.008+0000 7f0cbe000640 1 --2- 172.21.15.151:0/4224925978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0cc0145750 0x7f0cc00aefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:29.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.008+0000 7f0cbe000640 1 -- 172.21.15.151:0/4224925978 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cc00aab70 con 0x7f0cc00cccd0 2024-09-16T15:47:29.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.008+0000 7f0cbe000640 1 --2- 172.21.15.151:0/4224925978 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0cc00cccd0 0x7f0cc00aea80 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f0cac009950 tx=0x7f0cac032d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:29.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.008+0000 7f0cb6a00640 1 -- 172.21.15.151:0/4224925978 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cac040070 con 0x7f0cc00cccd0 2024-09-16T15:47:29.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.008+0000 7f0cb6a00640 1 -- 172.21.15.151:0/4224925978 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0cac003af0 con 0x7f0cc00cccd0 2024-09-16T15:47:29.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.008+0000 7f0cc5800640 1 -- 172.21.15.151:0/4224925978 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cc00aae50 con 0x7f0cc00cccd0 2024-09-16T15:47:29.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.008+0000 7f0cb6a00640 1 -- 172.21.15.151:0/4224925978 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cac045460 con 0x7f0cc00cccd0 2024-09-16T15:47:29.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.009+0000 7f0cc5800640 1 -- 172.21.15.151:0/4224925978 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cc00ab410 con 0x7f0cc00cccd0 2024-09-16T15:47:29.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.010+0000 7f0cc5800640 1 -- 172.21.15.151:0/4224925978 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cc00cecd0 con 0x7f0cc00cccd0 2024-09-16T15:47:29.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.011+0000 7f0cb6a00640 1 -- 172.21.15.151:0/4224925978 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 25) ==== 102511+0+0 (secure 0 0 0) 0x7f0cac039040 con 0x7f0cc00cccd0 2024-09-16T15:47:29.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.012+0000 7f0cb6a00640 1 --2- 172.21.15.151:0/4224925978 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0ca0079600 0x7f0ca007bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:29.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.012+0000 7f0cb6a00640 1 -- 172.21.15.151:0/4224925978 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5881+0+0 (secure 0 0 0) 0x7f0cac0babb0 con 0x7f0cc00cccd0 2024-09-16T15:47:29.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.012+0000 7f0cbea00640 1 --2- 172.21.15.151:0/4224925978 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0ca0079600 0x7f0ca007bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:29.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.013+0000 7f0cbea00640 1 --2- 172.21.15.151:0/4224925978 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0ca0079600 0x7f0ca007bac0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f0cb000b450 tx=0x7f0cb000d040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:29.017 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.016+0000 7f0cb6a00640 1 -- 172.21.15.151:0/4224925978 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0cac084370 con 0x7f0cc00cccd0 2024-09-16T15:47:29.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.233+0000 7f0cc5800640 1 -- 172.21.15.151:0/4224925978 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f0cc015ede0 con 0x7f0ca0079600 2024-09-16T15:47:29.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.238+0000 7f0cb6a00640 1 -- 172.21.15.151:0/4224925978 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f0cc015ede0 con 0x7f0ca0079600 2024-09-16T15:47:29.239 INFO:teuthology.orchestra.run.smithi151.stdout:/volumes/_nogroup/sv1/0be48f7c-c0a2-473c-b88e-38e220cdcc14 2024-09-16T15:47:29.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.240+0000 7f0cc5800640 1 -- 172.21.15.151:0/4224925978 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0ca0079600 msgr2=0x7f0ca007bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:29.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.240+0000 7f0cc5800640 1 --2- 172.21.15.151:0/4224925978 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0ca0079600 0x7f0ca007bac0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f0cb000b450 tx=0x7f0cb000d040 comp rx=0 tx=0).stop 2024-09-16T15:47:29.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.240+0000 7f0cc5800640 1 -- 172.21.15.151:0/4224925978 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0cc00cccd0 msgr2=0x7f0cc00aea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:29.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.240+0000 7f0cc5800640 1 --2- 172.21.15.151:0/4224925978 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0cc00cccd0 0x7f0cc00aea80 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f0cac009950 tx=0x7f0cac032d50 comp rx=0 tx=0).stop 2024-09-16T15:47:29.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.241+0000 7f0cc5800640 1 -- 172.21.15.151:0/4224925978 shutdown_connections 2024-09-16T15:47:29.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.241+0000 7f0cc5800640 1 --2- 172.21.15.151:0/4224925978 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0ca0079600 0x7f0ca007bac0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:29.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.241+0000 7f0cc5800640 1 --2- 172.21.15.151:0/4224925978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0cc0145750 0x7f0cc00aefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:29.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.241+0000 7f0cc5800640 1 --2- 172.21.15.151:0/4224925978 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0cc00cccd0 0x7f0cc00aea80 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:29.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.241+0000 7f0cc5800640 1 --2- 172.21.15.151:0/4224925978 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0cc006efa0 0x7f0cc0159c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:29.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.241+0000 7f0cc5800640 1 -- 172.21.15.151:0/4224925978 >> 172.21.15.151:0/4224925978 conn(0x7f0cc00b1840 msgr2=0x7f0cc00b1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:29.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.241+0000 7f0cc5800640 1 -- 172.21.15.151:0/4224925978 shutdown_connections 2024-09-16T15:47:29.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.241+0000 7f0cc5800640 1 -- 172.21.15.151:0/4224925978 wait complete. 2024-09-16T15:47:29.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:47:29.257 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/0be48f7c-c0a2-473c-b88e-38e220cdcc14 2024-09-16T15:47:29.257 DEBUG:teuthology.orchestra.run.smithi151:> ) 2024-09-16T15:47:29.293 INFO:teuthology.orchestra.run.smithi151.stdout:# file: volumes/_nogroup/sv1/0be48f7c-c0a2-473c-b88e-38e220cdcc14 2024-09-16T15:47:29.293 INFO:teuthology.orchestra.run.smithi151.stdout:ceph.dir.rbytes="3221225472" 2024-09-16T15:47:29.293 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:47:29.294 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 exp_size = 3221225472 2024-09-16T15:47:29.294 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone74020 2024-09-16T15:47:29.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.496+0000 7f6c1f400640 1 -- 172.21.15.151:0/619757415 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c1806efa0 msgr2=0x7f6c180cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:29.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.497+0000 7f6c1f400640 1 --2- 172.21.15.151:0/619757415 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c1806efa0 0x7f6c180cdbb0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f6c10005fd0 tx=0x7f6c1002aa10 comp rx=0 tx=0).stop 2024-09-16T15:47:29.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.497+0000 7f6c1f400640 1 -- 172.21.15.151:0/619757415 shutdown_connections 2024-09-16T15:47:29.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.497+0000 7f6c1f400640 1 --2- 172.21.15.151:0/619757415 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c1806efa0 0x7f6c180cdbb0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:29.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.497+0000 7f6c1f400640 1 --2- 172.21.15.151:0/619757415 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c180cccd0 0x7f6c18073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:29.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.497+0000 7f6c1f400640 1 -- 172.21.15.151:0/619757415 >> 172.21.15.151:0/619757415 conn(0x7f6c180b1840 msgr2=0x7f6c180b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:29.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.497+0000 7f6c1f400640 1 -- 172.21.15.151:0/619757415 shutdown_connections 2024-09-16T15:47:29.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.497+0000 7f6c1f400640 1 -- 172.21.15.151:0/619757415 wait complete. 2024-09-16T15:47:29.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.498+0000 7f6c1f400640 1 Processor -- start 2024-09-16T15:47:29.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.498+0000 7f6c1f400640 1 -- start start 2024-09-16T15:47:29.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.499+0000 7f6c1f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c1806efa0 0x7f6c1815ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:29.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.499+0000 7f6c1f400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c180cccd0 0x7f6c180ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:29.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.499+0000 7f6c1f400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c18145750 0x7f6c180cacf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:29.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.499+0000 7f6c1f400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6c180ce480 con 0x7f6c1806efa0 2024-09-16T15:47:29.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.499+0000 7f6c1f400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6c180ce300 con 0x7f6c180cccd0 2024-09-16T15:47:29.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.499+0000 7f6c1f400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f6c180ce600 con 0x7f6c18145750 2024-09-16T15:47:29.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.500+0000 7f6c1ca00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c1806efa0 0x7f6c1815ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:29.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.500+0000 7f6c1ca00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c1806efa0 0x7f6c1815ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:48120/0 (socket says 172.21.15.151:48120) 2024-09-16T15:47:29.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.500+0000 7f6c1ca00640 1 -- 172.21.15.151:0/3195950872 learned_addr learned my addr 172.21.15.151:0/3195950872 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:47:29.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.500+0000 7f6c17e00640 1 --2- 172.21.15.151:0/3195950872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c180cccd0 0x7f6c180ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:29.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.500+0000 7f6c1d400640 1 --2- 172.21.15.151:0/3195950872 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c18145750 0x7f6c180cacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:29.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.500+0000 7f6c1ca00640 1 -- 172.21.15.151:0/3195950872 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c18145750 msgr2=0x7f6c180cacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:29.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.500+0000 7f6c1ca00640 1 --2- 172.21.15.151:0/3195950872 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c18145750 0x7f6c180cacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:29.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.500+0000 7f6c1ca00640 1 -- 172.21.15.151:0/3195950872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c180cccd0 msgr2=0x7f6c180ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:29.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.500+0000 7f6c1ca00640 1 --2- 172.21.15.151:0/3195950872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c180cccd0 0x7f6c180ca7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:29.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.500+0000 7f6c1ca00640 1 -- 172.21.15.151:0/3195950872 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c181f35b0 con 0x7f6c1806efa0 2024-09-16T15:47:29.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.500+0000 7f6c1ca00640 1 --2- 172.21.15.151:0/3195950872 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c1806efa0 0x7f6c1815ded0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f6c0800c3a0 tx=0x7f6c0800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:29.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.501+0000 7f6c14c00640 1 -- 172.21.15.151:0/3195950872 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c08017070 con 0x7f6c1806efa0 2024-09-16T15:47:29.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.501+0000 7f6c14c00640 1 -- 172.21.15.151:0/3195950872 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6c080053c0 con 0x7f6c1806efa0 2024-09-16T15:47:29.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.501+0000 7f6c1f400640 1 -- 172.21.15.151:0/3195950872 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c181380c0 con 0x7f6c1806efa0 2024-09-16T15:47:29.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.501+0000 7f6c14c00640 1 -- 172.21.15.151:0/3195950872 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c08003850 con 0x7f6c1806efa0 2024-09-16T15:47:29.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.501+0000 7f6c1f400640 1 -- 172.21.15.151:0/3195950872 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c180c5ba0 con 0x7f6c1806efa0 2024-09-16T15:47:29.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.502+0000 7f6c1f400640 1 -- 172.21.15.151:0/3195950872 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c180cdbb0 con 0x7f6c1806efa0 2024-09-16T15:47:29.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.503+0000 7f6c14c00640 1 -- 172.21.15.151:0/3195950872 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 25) ==== 102511+0+0 (secure 0 0 0) 0x7f6c0801a070 con 0x7f6c1806efa0 2024-09-16T15:47:29.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.504+0000 7f6c14c00640 1 --2- 172.21.15.151:0/3195950872 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6bf8079600 0x7f6bf807bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:29.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.504+0000 7f6c14c00640 1 -- 172.21.15.151:0/3195950872 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5881+0+0 (secure 0 0 0) 0x7f6c080988e0 con 0x7f6c1806efa0 2024-09-16T15:47:29.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.504+0000 7f6c17e00640 1 --2- 172.21.15.151:0/3195950872 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6bf8079600 0x7f6bf807bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:29.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.505+0000 7f6c17e00640 1 --2- 172.21.15.151:0/3195950872 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6bf8079600 0x7f6bf807bac0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6c180c7930 tx=0x7f6c04005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:29.509 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.508+0000 7f6c14c00640 1 -- 172.21.15.151:0/3195950872 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6c181f35b0 con 0x7f6c1806efa0 2024-09-16T15:47:29.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.727+0000 7f6c1f400640 1 -- 172.21.15.151:0/3195950872 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone74020", "target": ["mon-mgr", ""]}) -- 0x7f6c1806ed60 con 0x7f6bf8079600 2024-09-16T15:47:29.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.797+0000 7f6c14c00640 1 -- 172.21.15.151:0/3195950872 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6c1806ed60 con 0x7f6bf8079600 2024-09-16T15:47:29.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.799+0000 7f6c1f400640 1 -- 172.21.15.151:0/3195950872 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6bf8079600 msgr2=0x7f6bf807bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:29.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.799+0000 7f6c1f400640 1 --2- 172.21.15.151:0/3195950872 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6bf8079600 0x7f6bf807bac0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6c180c7930 tx=0x7f6c04005f90 comp rx=0 tx=0).stop 2024-09-16T15:47:29.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.799+0000 7f6c1f400640 1 -- 172.21.15.151:0/3195950872 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c1806efa0 msgr2=0x7f6c1815ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:29.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.799+0000 7f6c1f400640 1 --2- 172.21.15.151:0/3195950872 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c1806efa0 0x7f6c1815ded0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f6c0800c3a0 tx=0x7f6c0800c860 comp rx=0 tx=0).stop 2024-09-16T15:47:29.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.799+0000 7f6c1f400640 1 -- 172.21.15.151:0/3195950872 shutdown_connections 2024-09-16T15:47:29.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.799+0000 7f6c1f400640 1 --2- 172.21.15.151:0/3195950872 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6bf8079600 0x7f6bf807bac0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:29.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.799+0000 7f6c1f400640 1 --2- 172.21.15.151:0/3195950872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c180cccd0 0x7f6c180ca7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:29.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.799+0000 7f6c1f400640 1 --2- 172.21.15.151:0/3195950872 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c18145750 0x7f6c180cacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:29.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.799+0000 7f6c1f400640 1 --2- 172.21.15.151:0/3195950872 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c1806efa0 0x7f6c1815ded0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:29.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.799+0000 7f6c1f400640 1 -- 172.21.15.151:0/3195950872 >> 172.21.15.151:0/3195950872 conn(0x7f6c180b1840 msgr2=0x7f6c180aa2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:29.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.799+0000 7f6c1f400640 1 -- 172.21.15.151:0/3195950872 shutdown_connections 2024-09-16T15:47:29.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.799+0000 7f6c1f400640 1 -- 172.21.15.151:0/3195950872 wait complete. 2024-09-16T15:47:29.803 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:47:29.802+0000 7f48f6c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:29.803 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:47:29.802+0000 7f48f6c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:29.803 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:47:29.802+0000 7f48f6c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:29.803 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:47:29.802+0000 7f48f6c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:29.803 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:47:29.802+0000 7f48f6c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:29.819 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone59661 2024-09-16T15:47:29.820 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:47:29.818+0000 7f48f5800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:29.820 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:47:29.818+0000 7f48f5800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:29.820 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:47:29.818+0000 7f48f5800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:29.820 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:47:29.818+0000 7f48f5800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:29.820 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:47:29.818+0000 7f48f5800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:29.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.986+0000 7f0432400640 1 -- 172.21.15.151:0/222592000 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f042c145750 msgr2=0x7f042c073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:29.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.986+0000 7f0432400640 1 --2- 172.21.15.151:0/222592000 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f042c145750 0x7f042c073440 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f041c005fd0 tx=0x7f041c02aa10 comp rx=0 tx=0).stop 2024-09-16T15:47:29.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.986+0000 7f0432400640 1 -- 172.21.15.151:0/222592000 shutdown_connections 2024-09-16T15:47:29.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.986+0000 7f0432400640 1 --2- 172.21.15.151:0/222592000 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f042c06efa0 0x7f042c0cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:29.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.986+0000 7f0432400640 1 --2- 172.21.15.151:0/222592000 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f042c145750 0x7f042c073440 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:29.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.986+0000 7f0432400640 1 -- 172.21.15.151:0/222592000 >> 172.21.15.151:0/222592000 conn(0x7f042c0b1840 msgr2=0x7f042c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:29.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.986+0000 7f0432400640 1 -- 172.21.15.151:0/222592000 shutdown_connections 2024-09-16T15:47:29.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.986+0000 7f0432400640 1 -- 172.21.15.151:0/222592000 wait complete. 2024-09-16T15:47:29.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.988+0000 7f0432400640 1 Processor -- start 2024-09-16T15:47:29.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.988+0000 7f0432400640 1 -- start start 2024-09-16T15:47:29.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.989+0000 7f0432400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f042c06efa0 0x7f042c1f65d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:29.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.989+0000 7f0432400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f042c0ccb00 0x7f042c14f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:29.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.989+0000 7f0432400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f042c145750 0x7f042c0ae3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:29.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.989+0000 7f0432400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f042c0b26b0 con 0x7f042c0ccb00 2024-09-16T15:47:29.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.989+0000 7f0432400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f042c0cedd0 con 0x7f042c06efa0 2024-09-16T15:47:29.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.989+0000 7f0432400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f042c160100 con 0x7f042c145750 2024-09-16T15:47:29.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.989+0000 7f042aa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f042c0ccb00 0x7f042c14f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:29.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.989+0000 7f042aa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f042c0ccb00 0x7f042c14f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:48144/0 (socket says 172.21.15.151:48144) 2024-09-16T15:47:29.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.989+0000 7f042aa00640 1 -- 172.21.15.151:0/643305205 learned_addr learned my addr 172.21.15.151:0/643305205 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:47:29.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.990+0000 7f042b400640 1 --2- 172.21.15.151:0/643305205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f042c06efa0 0x7f042c1f65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:29.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.990+0000 7f042be00640 1 --2- 172.21.15.151:0/643305205 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f042c145750 0x7f042c0ae3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:29.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.990+0000 7f042aa00640 1 -- 172.21.15.151:0/643305205 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f042c145750 msgr2=0x7f042c0ae3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:29.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.990+0000 7f042aa00640 1 --2- 172.21.15.151:0/643305205 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f042c145750 0x7f042c0ae3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:29.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.990+0000 7f042aa00640 1 -- 172.21.15.151:0/643305205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f042c06efa0 msgr2=0x7f042c1f65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:29.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.990+0000 7f042aa00640 1 --2- 172.21.15.151:0/643305205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f042c06efa0 0x7f042c1f65d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:29.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.990+0000 7f042aa00640 1 -- 172.21.15.151:0/643305205 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f042c0a9e20 con 0x7f042c0ccb00 2024-09-16T15:47:29.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.990+0000 7f042aa00640 1 --2- 172.21.15.151:0/643305205 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f042c0ccb00 0x7f042c14f150 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f041800d7a0 tx=0x7f041800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:29.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.990+0000 7f0423400640 1 -- 172.21.15.151:0/643305205 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0418017070 con 0x7f042c0ccb00 2024-09-16T15:47:29.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.990+0000 7f0423400640 1 -- 172.21.15.151:0/643305205 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0418009d20 con 0x7f042c0ccb00 2024-09-16T15:47:29.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.990+0000 7f0423400640 1 -- 172.21.15.151:0/643305205 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0418005280 con 0x7f042c0ccb00 2024-09-16T15:47:29.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.990+0000 7f0432400640 1 -- 172.21.15.151:0/643305205 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f042c0aa100 con 0x7f042c0ccb00 2024-09-16T15:47:29.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.991+0000 7f0432400640 1 -- 172.21.15.151:0/643305205 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f042c0aa6c0 con 0x7f042c0ccb00 2024-09-16T15:47:29.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.991+0000 7f0432400640 1 -- 172.21.15.151:0/643305205 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f042c1605b0 con 0x7f042c0ccb00 2024-09-16T15:47:29.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.993+0000 7f0423400640 1 -- 172.21.15.151:0/643305205 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 25) ==== 102511+0+0 (secure 0 0 0) 0x7f0418005420 con 0x7f042c0ccb00 2024-09-16T15:47:29.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.994+0000 7f0423400640 1 --2- 172.21.15.151:0/643305205 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f040c0795d0 0x7f040c07ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:29.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.994+0000 7f0423400640 1 -- 172.21.15.151:0/643305205 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5881+0+0 (secure 0 0 0) 0x7f0418099730 con 0x7f042c0ccb00 2024-09-16T15:47:29.996 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.995+0000 7f042b400640 1 --2- 172.21.15.151:0/643305205 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f040c0795d0 0x7f040c07ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:29.996 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.995+0000 7f042b400640 1 --2- 172.21.15.151:0/643305205 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f040c0795d0 0x7f040c07ba90 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f041c035000 tx=0x7f041c03b040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:29.999 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:29.998+0000 7f0423400640 1 -- 172.21.15.151:0/643305205 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0418062ef0 con 0x7f042c0ccb00 2024-09-16T15:47:30.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:30.216+0000 7f0432400640 1 -- 172.21.15.151:0/643305205 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone59661", "target": ["mon-mgr", ""]}) -- 0x7f042c1f6b10 con 0x7f040c0795d0 2024-09-16T15:47:30.804 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:47:30.803+0000 7f48e5e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:30.804 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:47:30.803+0000 7f48e5e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:30.804 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:47:30.803+0000 7f48e5e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:30.805 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:47:30.803+0000 7f48e5e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:30.805 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:47:30.803+0000 7f48e5e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:30.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:30.897+0000 7f0423400640 1 -- 172.21.15.151:0/643305205 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f042c1f6b10 con 0x7f040c0795d0 2024-09-16T15:47:30.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:30.903+0000 7f0432400640 1 -- 172.21.15.151:0/643305205 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f040c0795d0 msgr2=0x7f040c07ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:30.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:30.903+0000 7f0432400640 1 --2- 172.21.15.151:0/643305205 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f040c0795d0 0x7f040c07ba90 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f041c035000 tx=0x7f041c03b040 comp rx=0 tx=0).stop 2024-09-16T15:47:30.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:30.903+0000 7f0432400640 1 -- 172.21.15.151:0/643305205 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f042c0ccb00 msgr2=0x7f042c14f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:30.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:30.903+0000 7f0432400640 1 --2- 172.21.15.151:0/643305205 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f042c0ccb00 0x7f042c14f150 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f041800d7a0 tx=0x7f041800dc60 comp rx=0 tx=0).stop 2024-09-16T15:47:30.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:30.903+0000 7f0432400640 1 -- 172.21.15.151:0/643305205 shutdown_connections 2024-09-16T15:47:30.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:30.903+0000 7f0432400640 1 --2- 172.21.15.151:0/643305205 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f040c0795d0 0x7f040c07ba90 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:30.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:30.903+0000 7f0432400640 1 --2- 172.21.15.151:0/643305205 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f042c145750 0x7f042c0ae3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:30.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:30.903+0000 7f0432400640 1 --2- 172.21.15.151:0/643305205 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f042c0ccb00 0x7f042c14f150 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:30.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:30.903+0000 7f0432400640 1 --2- 172.21.15.151:0/643305205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f042c06efa0 0x7f042c1f65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:30.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:30.903+0000 7f0432400640 1 -- 172.21.15.151:0/643305205 >> 172.21.15.151:0/643305205 conn(0x7f042c0b1840 msgr2=0x7f042c0cfbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:30.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:30.903+0000 7f0432400640 1 -- 172.21.15.151:0/643305205 shutdown_connections 2024-09-16T15:47:30.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:30.903+0000 7f0432400640 1 -- 172.21.15.151:0/643305205 wait complete. 2024-09-16T15:47:30.926 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone92092 2024-09-16T15:47:31.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:31.184+0000 7f9edf400640 1 -- 172.21.15.151:0/2826196274 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ed806efa0 msgr2=0x7f9ed80cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:31.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:31.184+0000 7f9edf400640 1 --2- 172.21.15.151:0/2826196274 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ed806efa0 0x7f9ed80cdbb0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0005fd0 tx=0x7f9ed002a970 comp rx=0 tx=0).stop 2024-09-16T15:47:31.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:31.184+0000 7f9edf400640 1 -- 172.21.15.151:0/2826196274 shutdown_connections 2024-09-16T15:47:31.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:31.184+0000 7f9edf400640 1 --2- 172.21.15.151:0/2826196274 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ed806efa0 0x7f9ed80cdbb0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:31.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:31.184+0000 7f9edf400640 1 --2- 172.21.15.151:0/2826196274 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ed80cccd0 0x7f9ed8073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:31.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:31.184+0000 7f9edf400640 1 -- 172.21.15.151:0/2826196274 >> 172.21.15.151:0/2826196274 conn(0x7f9ed80b1840 msgr2=0x7f9ed80b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:31.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:31.185+0000 7f9edf400640 1 -- 172.21.15.151:0/2826196274 shutdown_connections 2024-09-16T15:47:31.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:31.185+0000 7f9edf400640 1 -- 172.21.15.151:0/2826196274 wait complete. 2024-09-16T15:47:31.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:31.187+0000 7f9edf400640 1 Processor -- start 2024-09-16T15:47:31.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:31.187+0000 7f9edf400640 1 -- start start 2024-09-16T15:47:31.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:31.188+0000 7f9edf400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ed806efa0 0x7f9ed8156760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:31.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:31.188+0000 7f9edf400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ed80cccd0 0x7f9ed815ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:31.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:31.188+0000 7f9edf400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ed8145750 0x7f9ed813b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:31.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:31.188+0000 7f9edf400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed80ce480 con 0x7f9ed8145750 2024-09-16T15:47:31.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:31.188+0000 7f9edf400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed80ce300 con 0x7f9ed80cccd0 2024-09-16T15:47:31.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:31.188+0000 7f9edf400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f9ed80ce600 con 0x7f9ed806efa0 2024-09-16T15:47:31.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:31.188+0000 7f9edd400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ed8145750 0x7f9ed813b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:31.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:31.188+0000 7f9edd400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ed8145750 0x7f9ed813b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:48166/0 (socket says 172.21.15.151:48166) 2024-09-16T15:47:31.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:31.188+0000 7f9edd400640 1 -- 172.21.15.151:0/1190082596 learned_addr learned my addr 172.21.15.151:0/1190082596 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:47:31.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:31.188+0000 7f9edca00640 1 --2- 172.21.15.151:0/1190082596 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ed806efa0 0x7f9ed8156760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:31.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:31.188+0000 7f9ed7e00640 1 --2- 172.21.15.151:0/1190082596 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ed80cccd0 0x7f9ed815ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:31.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:31.189+0000 7f9edd400640 1 -- 172.21.15.151:0/1190082596 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ed806efa0 msgr2=0x7f9ed8156760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:31.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:31.189+0000 7f9edd400640 1 --2- 172.21.15.151:0/1190082596 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ed806efa0 0x7f9ed8156760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:31.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:31.189+0000 7f9edd400640 1 -- 172.21.15.151:0/1190082596 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ed80cccd0 msgr2=0x7f9ed815ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:31.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:31.189+0000 7f9edd400640 1 --2- 172.21.15.151:0/1190082596 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ed80cccd0 0x7f9ed815ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:31.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:31.189+0000 7f9edd400640 1 -- 172.21.15.151:0/1190082596 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ed813a070 con 0x7f9ed8145750 2024-09-16T15:47:31.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:31.190+0000 7f9edd400640 1 --2- 172.21.15.151:0/1190082596 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ed8145750 0x7f9ed813b110 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0006010 tx=0x7f9ed0003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:31.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:31.191+0000 7f9ed4c00640 1 -- 172.21.15.151:0/1190082596 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed0034070 con 0x7f9ed8145750 2024-09-16T15:47:31.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:31.191+0000 7f9ed4c00640 1 -- 172.21.15.151:0/1190082596 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ed00045c0 con 0x7f9ed8145750 2024-09-16T15:47:31.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:31.191+0000 7f9ed4c00640 1 -- 172.21.15.151:0/1190082596 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed0039450 con 0x7f9ed8145750 2024-09-16T15:47:31.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:31.192+0000 7f9edf400640 1 -- 172.21.15.151:0/1190082596 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ed813a350 con 0x7f9ed8145750 2024-09-16T15:47:31.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:31.192+0000 7f9edf400640 1 -- 172.21.15.151:0/1190082596 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ed813a910 con 0x7f9ed8145750 2024-09-16T15:47:31.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:31.193+0000 7f9edf400640 1 -- 172.21.15.151:0/1190082596 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ed80ceb90 con 0x7f9ed8145750 2024-09-16T15:47:31.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:31.196+0000 7f9ed4c00640 1 -- 172.21.15.151:0/1190082596 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 25) ==== 102511+0+0 (secure 0 0 0) 0x7f9ed0037020 con 0x7f9ed8145750 2024-09-16T15:47:31.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:31.198+0000 7f9ed4c00640 1 --2- 172.21.15.151:0/1190082596 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9eb80795d0 0x7f9eb807ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:31.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:31.198+0000 7f9ed4c00640 1 -- 172.21.15.151:0/1190082596 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5881+0+0 (secure 0 0 0) 0x7f9ed00bae60 con 0x7f9ed8145750 2024-09-16T15:47:31.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:31.201+0000 7f9ed4c00640 1 -- 172.21.15.151:0/1190082596 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9ed0053080 con 0x7f9ed8145750 2024-09-16T15:47:31.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:31.203+0000 7f9edca00640 1 --2- 172.21.15.151:0/1190082596 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9eb80795d0 0x7f9eb807ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:31.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:31.221+0000 7f9edca00640 1 --2- 172.21.15.151:0/1190082596 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9eb80795d0 0x7f9eb807ba90 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc006fd0 tx=0x7f9ecc009060 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:31.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:31.591+0000 7f9edf400640 1 -- 172.21.15.151:0/1190082596 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone92092", "target": ["mon-mgr", ""]}) -- 0x7f9ed8069a10 con 0x7f9eb80795d0 2024-09-16T15:47:31.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:31.954+0000 7f9ed4c00640 1 -- 172.21.15.151:0/1190082596 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9ed8069a10 con 0x7f9eb80795d0 2024-09-16T15:47:31.959 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:31.958+0000 7f9edf400640 1 -- 172.21.15.151:0/1190082596 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9eb80795d0 msgr2=0x7f9eb807ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:31.959 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:31.958+0000 7f9edf400640 1 --2- 172.21.15.151:0/1190082596 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9eb80795d0 0x7f9eb807ba90 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc006fd0 tx=0x7f9ecc009060 comp rx=0 tx=0).stop 2024-09-16T15:47:31.959 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:31.958+0000 7f9edf400640 1 -- 172.21.15.151:0/1190082596 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ed8145750 msgr2=0x7f9ed813b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:31.959 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:31.958+0000 7f9edf400640 1 --2- 172.21.15.151:0/1190082596 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ed8145750 0x7f9ed813b110 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0006010 tx=0x7f9ed0003f10 comp rx=0 tx=0).stop 2024-09-16T15:47:31.959 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:31.958+0000 7f9edf400640 1 -- 172.21.15.151:0/1190082596 shutdown_connections 2024-09-16T15:47:31.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:31.958+0000 7f9edf400640 1 --2- 172.21.15.151:0/1190082596 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9eb80795d0 0x7f9eb807ba90 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:31.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:31.958+0000 7f9edf400640 1 --2- 172.21.15.151:0/1190082596 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9ed80cccd0 0x7f9ed815ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:31.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:31.958+0000 7f9edf400640 1 --2- 172.21.15.151:0/1190082596 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9ed8145750 0x7f9ed813b110 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:31.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:31.958+0000 7f9edf400640 1 --2- 172.21.15.151:0/1190082596 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9ed806efa0 0x7f9ed8156760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:31.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:31.958+0000 7f9edf400640 1 -- 172.21.15.151:0/1190082596 >> 172.21.15.151:0/1190082596 conn(0x7f9ed80b1840 msgr2=0x7f9ed806f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:31.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:31.958+0000 7f9edf400640 1 -- 172.21.15.151:0/1190082596 shutdown_connections 2024-09-16T15:47:31.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:31.959+0000 7f9edf400640 1 -- 172.21.15.151:0/1190082596 wait complete. 2024-09-16T15:47:32.002 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone1123 2024-09-16T15:47:32.010 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:47:32.009+0000 7f48f6200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:32.010 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:47:32.009+0000 7f48f6200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:32.011 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:47:32.009+0000 7f48f6200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:32.011 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:47:32.009+0000 7f48f6200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:32.011 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:47:32.009+0000 7f48f6200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:32.294 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:32.293+0000 7f7e72e00640 1 -- 172.21.15.151:0/1414768110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7e6c0b4e90 msgr2=0x7f7e6c0d2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:32.294 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:32.293+0000 7f7e72e00640 1 --2- 172.21.15.151:0/1414768110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7e6c0b4e90 0x7f7e6c0d2530 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f7e64005fd0 tx=0x7f7e6402a8b0 comp rx=0 tx=0).stop 2024-09-16T15:47:32.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:32.293+0000 7f7e72e00640 1 -- 172.21.15.151:0/1414768110 shutdown_connections 2024-09-16T15:47:32.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:32.293+0000 7f7e72e00640 1 --2- 172.21.15.151:0/1414768110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7e6c0b4e90 0x7f7e6c0d2530 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:32.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:32.293+0000 7f7e72e00640 1 --2- 172.21.15.151:0/1414768110 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7e6c0d0cc0 0x7f7e6c0d10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:32.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:32.293+0000 7f7e72e00640 1 -- 172.21.15.151:0/1414768110 >> 172.21.15.151:0/1414768110 conn(0x7f7e6c0d4040 msgr2=0x7f7e6c0d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:32.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:32.293+0000 7f7e72e00640 1 -- 172.21.15.151:0/1414768110 shutdown_connections 2024-09-16T15:47:32.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:32.293+0000 7f7e72e00640 1 -- 172.21.15.151:0/1414768110 wait complete. 2024-09-16T15:47:32.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:32.296+0000 7f7e72e00640 1 Processor -- start 2024-09-16T15:47:32.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:32.296+0000 7f7e72e00640 1 -- start start 2024-09-16T15:47:32.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:32.297+0000 7f7e72e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7e6c0b4e90 0x7f7e6c1f54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:32.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:32.297+0000 7f7e72e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7e6c0d0cc0 0x7f7e6c1515b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:32.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:32.297+0000 7f7e72e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e6c0d4bd0 0x7f7e6c0cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:32.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:32.297+0000 7f7e72e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6c0d2bf0 con 0x7f7e6c0d4bd0 2024-09-16T15:47:32.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:32.297+0000 7f7e72e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6c0d2a70 con 0x7f7e6c0b4e90 2024-09-16T15:47:32.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:32.297+0000 7f7e72e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f7e6c0cea30 con 0x7f7e6c0d0cc0 2024-09-16T15:47:32.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:32.298+0000 7f7e70e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e6c0d4bd0 0x7f7e6c0cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:32.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:32.298+0000 7f7e70e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e6c0d4bd0 0x7f7e6c0cc4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:48176/0 (socket says 172.21.15.151:48176) 2024-09-16T15:47:32.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:32.298+0000 7f7e70e00640 1 -- 172.21.15.151:0/182095063 learned_addr learned my addr 172.21.15.151:0/182095063 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:47:32.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:32.298+0000 7f7e6b400640 1 --2- 172.21.15.151:0/182095063 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7e6c0d0cc0 0x7f7e6c1515b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:32.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:32.299+0000 7f7e70e00640 1 -- 172.21.15.151:0/182095063 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7e6c0d0cc0 msgr2=0x7f7e6c1515b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:32.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:32.299+0000 7f7e70e00640 1 --2- 172.21.15.151:0/182095063 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7e6c0d0cc0 0x7f7e6c1515b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:32.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:32.299+0000 7f7e70e00640 1 -- 172.21.15.151:0/182095063 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7e6c0b4e90 msgr2=0x7f7e6c1f54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:32.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:32.299+0000 7f7e70e00640 1 --2- 172.21.15.151:0/182095063 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7e6c0b4e90 0x7f7e6c1f54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:32.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:32.299+0000 7f7e70e00640 1 -- 172.21.15.151:0/182095063 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e6c0caa20 con 0x7f7e6c0d4bd0 2024-09-16T15:47:32.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:32.299+0000 7f7e70e00640 1 --2- 172.21.15.151:0/182095063 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e6c0d4bd0 0x7f7e6c0cc4b0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f7e64009b30 tx=0x7f7e640316e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:32.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:32.300+0000 7f7e63e00640 1 -- 172.21.15.151:0/182095063 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e64040070 con 0x7f7e6c0d4bd0 2024-09-16T15:47:32.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:32.300+0000 7f7e63e00640 1 -- 172.21.15.151:0/182095063 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7e64004040 con 0x7f7e6c0d4bd0 2024-09-16T15:47:32.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:32.300+0000 7f7e72e00640 1 -- 172.21.15.151:0/182095063 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e6c0cad00 con 0x7f7e6c0d4bd0 2024-09-16T15:47:32.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:32.300+0000 7f7e63e00640 1 -- 172.21.15.151:0/182095063 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e64004450 con 0x7f7e6c0d4bd0 2024-09-16T15:47:32.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:32.300+0000 7f7e72e00640 1 -- 172.21.15.151:0/182095063 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e6c1f4990 con 0x7f7e6c0d4bd0 2024-09-16T15:47:32.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:32.302+0000 7f7e72e00640 1 -- 172.21.15.151:0/182095063 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e6c0d10a0 con 0x7f7e6c0d4bd0 2024-09-16T15:47:32.304 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:32.303+0000 7f7e63e00640 1 -- 172.21.15.151:0/182095063 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 25) ==== 102511+0+0 (secure 0 0 0) 0x7f7e64039040 con 0x7f7e6c0d4bd0 2024-09-16T15:47:32.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:32.304+0000 7f7e63e00640 1 --2- 172.21.15.151:0/182095063 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7e4c079630 0x7f7e4c07baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:32.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:32.304+0000 7f7e63e00640 1 -- 172.21.15.151:0/182095063 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5881+0+0 (secure 0 0 0) 0x7f7e640bac60 con 0x7f7e6c0d4bd0 2024-09-16T15:47:32.311 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:32.310+0000 7f7e63e00640 1 -- 172.21.15.151:0/182095063 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7e640849f0 con 0x7f7e6c0d4bd0 2024-09-16T15:47:32.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:32.322+0000 7f7e6be00640 1 --2- 172.21.15.151:0/182095063 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7e4c079630 0x7f7e4c07baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:32.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:32.330+0000 7f7e6be00640 1 --2- 172.21.15.151:0/182095063 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7e4c079630 0x7f7e4c07baf0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f7e580058c0 tx=0x7f7e58005850 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:32.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:32.538+0000 7f7e63e00640 1 -- 172.21.15.151:0/182095063 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mgrmap(e 26) ==== 102664+0+0 (secure 0 0 0) 0x7f7e64080560 con 0x7f7e6c0d4bd0 2024-09-16T15:47:32.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:32.744+0000 7f7e72e00640 1 -- 172.21.15.151:0/182095063 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone1123", "target": ["mon-mgr", ""]}) -- 0x7f7e6c161ab0 con 0x7f7e4c079630 2024-09-16T15:47:34.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:34.180+0000 7f7e63e00640 1 -- 172.21.15.151:0/182095063 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7e6c161ab0 con 0x7f7e4c079630 2024-09-16T15:47:34.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:34.182+0000 7f7e72e00640 1 -- 172.21.15.151:0/182095063 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7e4c079630 msgr2=0x7f7e4c07baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:34.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:34.182+0000 7f7e72e00640 1 --2- 172.21.15.151:0/182095063 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7e4c079630 0x7f7e4c07baf0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f7e580058c0 tx=0x7f7e58005850 comp rx=0 tx=0).stop 2024-09-16T15:47:34.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:34.182+0000 7f7e72e00640 1 -- 172.21.15.151:0/182095063 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e6c0d4bd0 msgr2=0x7f7e6c0cc4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:34.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:34.182+0000 7f7e72e00640 1 --2- 172.21.15.151:0/182095063 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e6c0d4bd0 0x7f7e6c0cc4b0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f7e64009b30 tx=0x7f7e640316e0 comp rx=0 tx=0).stop 2024-09-16T15:47:34.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:34.182+0000 7f7e72e00640 1 -- 172.21.15.151:0/182095063 shutdown_connections 2024-09-16T15:47:34.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:34.182+0000 7f7e72e00640 1 --2- 172.21.15.151:0/182095063 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7e4c079630 0x7f7e4c07baf0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:34.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:34.182+0000 7f7e72e00640 1 --2- 172.21.15.151:0/182095063 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e6c0d4bd0 0x7f7e6c0cc4b0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:34.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:34.182+0000 7f7e72e00640 1 --2- 172.21.15.151:0/182095063 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7e6c0d0cc0 0x7f7e6c1515b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:34.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:34.182+0000 7f7e72e00640 1 --2- 172.21.15.151:0/182095063 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7e6c0b4e90 0x7f7e6c1f54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:34.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:34.182+0000 7f7e72e00640 1 -- 172.21.15.151:0/182095063 >> 172.21.15.151:0/182095063 conn(0x7f7e6c0d4040 msgr2=0x7f7e6c1613a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:34.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:34.183+0000 7f7e72e00640 1 -- 172.21.15.151:0/182095063 shutdown_connections 2024-09-16T15:47:34.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:34.183+0000 7f7e72e00640 1 -- 172.21.15.151:0/182095063 wait complete. 2024-09-16T15:47:34.221 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone12644 2024-09-16T15:47:34.259 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:47:34.256+0000 7f48f7600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:34.259 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:47:34.256+0000 7f48f7600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:34.259 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:47:34.256+0000 7f48f7600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:34.259 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:47:34.256+0000 7f48f7600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:34.259 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:47:34.256+0000 7f48f7600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:34.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:34.536+0000 7f9c65800640 1 -- 172.21.15.151:0/2173593658 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9c44003a00 msgr2=0x7f9c44003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:34.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:34.536+0000 7f9c65800640 1 --2- 172.21.15.151:0/2173593658 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9c44003a00 0x7f9c44003de0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c00bb20 tx=0x7f9c44005830 comp rx=0 tx=0).stop 2024-09-16T15:47:34.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:34.536+0000 7f9c5f400640 1 -- 172.21.15.151:0/2173593658 reap_dead start 2024-09-16T15:47:34.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:34.536+0000 7f9c65800640 1 -- 172.21.15.151:0/2173593658 shutdown_connections 2024-09-16T15:47:34.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:34.536+0000 7f9c65800640 1 -- 172.21.15.151:0/2173593658 >> 172.21.15.151:0/2173593658 conn(0x7f9c600b1840 msgr2=0x7f9c600b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:34.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:34.536+0000 7f9c65800640 1 -- 172.21.15.151:0/2173593658 shutdown_connections 2024-09-16T15:47:34.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:34.536+0000 7f9c65800640 1 -- 172.21.15.151:0/2173593658 wait complete. 2024-09-16T15:47:34.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:34.538+0000 7f9c65800640 1 Processor -- start 2024-09-16T15:47:34.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:34.539+0000 7f9c65800640 1 -- start start 2024-09-16T15:47:34.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:34.540+0000 7f9c65800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c44003a00 0x7f9c6015ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:34.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:34.540+0000 7f9c65800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9c60152630 0x7f9c601f4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:34.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:34.540+0000 7f9c65800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9c60152ab0 0x7f9c601f4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:34.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:34.540+0000 7f9c65800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9c600ce4a0 con 0x7f9c60152ab0 2024-09-16T15:47:34.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:34.540+0000 7f9c65800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9c60070470 con 0x7f9c44003a00 2024-09-16T15:47:34.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:34.540+0000 7f9c65800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f9c600ce620 con 0x7f9c60152630 2024-09-16T15:47:34.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:34.540+0000 7f9c5f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9c60152ab0 0x7f9c601f4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:34.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:34.540+0000 7f9c5f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9c60152ab0 0x7f9c601f4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49016/0 (socket says 172.21.15.151:49016) 2024-09-16T15:47:34.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:34.540+0000 7f9c5f400640 1 -- 172.21.15.151:0/3641480199 learned_addr learned my addr 172.21.15.151:0/3641480199 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:47:34.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:34.541+0000 7f9c5ea00640 1 --2- 172.21.15.151:0/3641480199 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c44003a00 0x7f9c6015ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:34.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:34.541+0000 7f9c5e000640 1 --2- 172.21.15.151:0/3641480199 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9c60152630 0x7f9c601f4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:34.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:34.541+0000 7f9c5f400640 1 -- 172.21.15.151:0/3641480199 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9c60152630 msgr2=0x7f9c601f4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:34.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:34.541+0000 7f9c5f400640 1 --2- 172.21.15.151:0/3641480199 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9c60152630 0x7f9c601f4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:34.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:34.541+0000 7f9c5f400640 1 -- 172.21.15.151:0/3641480199 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c44003a00 msgr2=0x7f9c6015ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:34.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:34.541+0000 7f9c5f400640 1 --2- 172.21.15.151:0/3641480199 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c44003a00 0x7f9c6015ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:34.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:34.541+0000 7f9c5f400640 1 -- 172.21.15.151:0/3641480199 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c60154f20 con 0x7f9c60152ab0 2024-09-16T15:47:34.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:34.541+0000 7f9c5e000640 1 --2- 172.21.15.151:0/3641480199 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9c60152630 0x7f9c601f4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:47:34.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:34.542+0000 7f9c5f400640 1 --2- 172.21.15.151:0/3641480199 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9c60152ab0 0x7f9c601f4fd0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f9c600cd7d0 tx=0x7f9c5800cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:34.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:34.542+0000 7f9c56a00640 1 -- 172.21.15.151:0/3641480199 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c58014070 con 0x7f9c60152ab0 2024-09-16T15:47:34.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:34.542+0000 7f9c56a00640 1 -- 172.21.15.151:0/3641480199 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9c58004e50 con 0x7f9c60152ab0 2024-09-16T15:47:34.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:34.542+0000 7f9c65800640 1 -- 172.21.15.151:0/3641480199 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c60155200 con 0x7f9c60152ab0 2024-09-16T15:47:34.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:34.542+0000 7f9c56a00640 1 -- 172.21.15.151:0/3641480199 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c58007be0 con 0x7f9c60152ab0 2024-09-16T15:47:34.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:34.542+0000 7f9c65800640 1 -- 172.21.15.151:0/3641480199 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c601557c0 con 0x7f9c60152ab0 2024-09-16T15:47:34.546 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:34.545+0000 7f9c56a00640 1 -- 172.21.15.151:0/3641480199 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 26) ==== 102664+0+0 (secure 0 0 0) 0x7f9c58005020 con 0x7f9c60152ab0 2024-09-16T15:47:34.546 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:34.545+0000 7f9c65800640 1 -- 172.21.15.151:0/3641480199 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c600cf090 con 0x7f9c60152ab0 2024-09-16T15:47:34.548 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:34.547+0000 7f9c56a00640 1 --2- 172.21.15.151:0/3641480199 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9c40079780 0x7f9c4007bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:34.549 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:34.547+0000 7f9c56a00640 1 -- 172.21.15.151:0/3641480199 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5881+0+0 (secure 0 0 0) 0x7f9c58099730 con 0x7f9c60152ab0 2024-09-16T15:47:34.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:34.556+0000 7f9c56a00640 1 -- 172.21.15.151:0/3641480199 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9c58062f10 con 0x7f9c60152ab0 2024-09-16T15:47:34.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:34.567+0000 7f9c5ea00640 1 --2- 172.21.15.151:0/3641480199 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9c40079780 0x7f9c4007bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:34.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:34.569+0000 7f9c5ea00640 1 --2- 172.21.15.151:0/3641480199 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9c40079780 0x7f9c4007bc40 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c036000 tx=0x7f9c4c03c040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:34.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:34.592+0000 7f9c56a00640 1 -- 172.21.15.151:0/3641480199 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mgrmap(e 27) ==== 102715+0+0 (secure 0 0 0) 0x7f9c580992f0 con 0x7f9c60152ab0 2024-09-16T15:47:34.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:34.959+0000 7f9c65800640 1 -- 172.21.15.151:0/3641480199 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone12644", "target": ["mon-mgr", ""]}) -- 0x7f9c600b1420 con 0x7f9c40079780 2024-09-16T15:47:36.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:36.085+0000 7f9c56a00640 1 -- 172.21.15.151:0/3641480199 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9c600b1420 con 0x7f9c40079780 2024-09-16T15:47:36.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:36.099+0000 7f9c65800640 1 -- 172.21.15.151:0/3641480199 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9c40079780 msgr2=0x7f9c4007bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:36.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:36.099+0000 7f9c65800640 1 --2- 172.21.15.151:0/3641480199 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9c40079780 0x7f9c4007bc40 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c036000 tx=0x7f9c4c03c040 comp rx=0 tx=0).stop 2024-09-16T15:47:36.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:36.099+0000 7f9c65800640 1 -- 172.21.15.151:0/3641480199 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9c60152ab0 msgr2=0x7f9c601f4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:36.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:36.099+0000 7f9c65800640 1 --2- 172.21.15.151:0/3641480199 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9c60152ab0 0x7f9c601f4fd0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f9c600cd7d0 tx=0x7f9c5800cd00 comp rx=0 tx=0).stop 2024-09-16T15:47:36.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:36.102+0000 7f9c65800640 1 -- 172.21.15.151:0/3641480199 shutdown_connections 2024-09-16T15:47:36.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:36.102+0000 7f9c65800640 1 --2- 172.21.15.151:0/3641480199 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9c40079780 0x7f9c4007bc40 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:36.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:36.102+0000 7f9c65800640 1 --2- 172.21.15.151:0/3641480199 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9c60152ab0 0x7f9c601f4fd0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:36.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:36.102+0000 7f9c65800640 1 --2- 172.21.15.151:0/3641480199 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9c60152630 0x7f9c601f4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:36.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:36.102+0000 7f9c65800640 1 --2- 172.21.15.151:0/3641480199 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c44003a00 0x7f9c6015ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:36.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:36.102+0000 7f9c65800640 1 -- 172.21.15.151:0/3641480199 >> 172.21.15.151:0/3641480199 conn(0x7f9c600b1840 msgr2=0x7f9c60151b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:36.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:36.105+0000 7f9c65800640 1 -- 172.21.15.151:0/3641480199 shutdown_connections 2024-09-16T15:47:36.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:36.105+0000 7f9c65800640 1 -- 172.21.15.151:0/3641480199 wait complete. 2024-09-16T15:47:36.146 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone93605 2024-09-16T15:47:36.439 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:36.437+0000 7f8755000640 1 -- 172.21.15.151:0/1430808765 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8750167040 msgr2=0x7f8750169430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:36.439 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:36.437+0000 7f8755000640 1 --2- 172.21.15.151:0/1430808765 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8750167040 0x7f8750169430 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f8748005fd0 tx=0x7f874802aa00 comp rx=0 tx=0).stop 2024-09-16T15:47:36.439 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:36.438+0000 7f8755000640 1 -- 172.21.15.151:0/1430808765 shutdown_connections 2024-09-16T15:47:36.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:36.438+0000 7f8755000640 1 --2- 172.21.15.151:0/1430808765 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8750167040 0x7f8750169430 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:36.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:36.438+0000 7f8755000640 1 -- 172.21.15.151:0/1430808765 >> 172.21.15.151:0/1430808765 conn(0x7f87500d3fb0 msgr2=0x7f87500d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:36.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:36.438+0000 7f8755000640 1 -- 172.21.15.151:0/1430808765 shutdown_connections 2024-09-16T15:47:36.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:36.438+0000 7f8755000640 1 -- 172.21.15.151:0/1430808765 wait complete. 2024-09-16T15:47:36.441 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:36.440+0000 7f8755000640 1 Processor -- start 2024-09-16T15:47:36.441 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:36.440+0000 7f8755000640 1 -- start start 2024-09-16T15:47:36.443 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:36.441+0000 7f8755000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87500b5630 0x7f875015fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:36.443 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:36.441+0000 7f8755000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8750161590 0x7f8750156b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:36.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:36.441+0000 7f8755000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8750167040 0x7f875015c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:36.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:36.441+0000 7f8755000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f87500cfe40 con 0x7f8750161590 2024-09-16T15:47:36.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:36.441+0000 7f8755000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f87500cfcc0 con 0x7f87500b5630 2024-09-16T15:47:36.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:36.441+0000 7f8755000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f8750169970 con 0x7f8750167040 2024-09-16T15:47:36.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:36.442+0000 7f874d400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8750161590 0x7f8750156b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:36.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:36.442+0000 7f874d400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8750161590 0x7f8750156b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49020/0 (socket says 172.21.15.151:49020) 2024-09-16T15:47:36.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:36.442+0000 7f874d400640 1 -- 172.21.15.151:0/1166561970 learned_addr learned my addr 172.21.15.151:0/1166561970 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:47:36.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:36.442+0000 7f874de00640 1 --2- 172.21.15.151:0/1166561970 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87500b5630 0x7f875015fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:36.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:36.442+0000 7f874d400640 1 -- 172.21.15.151:0/1166561970 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8750167040 msgr2=0x7f875015c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:36.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:36.442+0000 7f874e800640 1 --2- 172.21.15.151:0/1166561970 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8750167040 0x7f875015c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:36.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:36.442+0000 7f874d400640 1 --2- 172.21.15.151:0/1166561970 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8750167040 0x7f875015c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:36.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:36.442+0000 7f874d400640 1 -- 172.21.15.151:0/1166561970 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87500b5630 msgr2=0x7f875015fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:36.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:36.442+0000 7f874d400640 1 --2- 172.21.15.151:0/1166561970 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87500b5630 0x7f875015fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:36.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:36.442+0000 7f874d400640 1 -- 172.21.15.151:0/1166561970 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87500cb120 con 0x7f8750161590 2024-09-16T15:47:36.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:36.442+0000 7f874de00640 1 --2- 172.21.15.151:0/1166561970 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87500b5630 0x7f875015fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:47:36.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:36.443+0000 7f874d400640 1 --2- 172.21.15.151:0/1166561970 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8750161590 0x7f8750156b90 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f874000c410 tx=0x7f874000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:36.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:36.443+0000 7f8746000640 1 -- 172.21.15.151:0/1166561970 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8740017070 con 0x7f8750161590 2024-09-16T15:47:36.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:36.443+0000 7f8746000640 1 -- 172.21.15.151:0/1166561970 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f87400035f0 con 0x7f8750161590 2024-09-16T15:47:36.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:36.443+0000 7f8755000640 1 -- 172.21.15.151:0/1166561970 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8750160410 con 0x7f8750161590 2024-09-16T15:47:36.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:36.443+0000 7f8746000640 1 -- 172.21.15.151:0/1166561970 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8740003a80 con 0x7f8750161590 2024-09-16T15:47:36.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:36.443+0000 7f8755000640 1 -- 172.21.15.151:0/1166561970 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f875015fad0 con 0x7f8750161590 2024-09-16T15:47:36.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:36.444+0000 7f8755000640 1 -- 172.21.15.151:0/1166561970 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f875006acd0 con 0x7f8750161590 2024-09-16T15:47:36.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:36.446+0000 7f8746000640 1 -- 172.21.15.151:0/1166561970 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 27) ==== 102715+0+0 (secure 0 0 0) 0x7f8740003c20 con 0x7f8750161590 2024-09-16T15:47:36.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:36.447+0000 7f8746000640 1 --2- 172.21.15.151:0/1166561970 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f872c079810 0x7f872c07bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:36.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:36.447+0000 7f8746000640 1 -- 172.21.15.151:0/1166561970 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5881+0+0 (secure 0 0 0) 0x7f8740098b00 con 0x7f8750161590 2024-09-16T15:47:36.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:36.448+0000 7f874de00640 1 --2- 172.21.15.151:0/1166561970 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f872c079810 0x7f872c07bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:36.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:36.451+0000 7f8746000640 1 -- 172.21.15.151:0/1166561970 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8740062830 con 0x7f8750161590 2024-09-16T15:47:36.471 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:36.470+0000 7f874de00640 1 --2- 172.21.15.151:0/1166561970 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f872c079810 0x7f872c07bcd0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8738003eb0 tx=0x7f8738009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:36.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:36.673+0000 7f8746000640 1 -- 172.21.15.151:0/1166561970 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mgrmap(e 28) ==== 102766+0+0 (secure 0 0 0) 0x7f874005e3a0 con 0x7f8750161590 2024-09-16T15:47:36.833 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:36.832+0000 7f8755000640 1 -- 172.21.15.151:0/1166561970 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone93605", "target": ["mon-mgr", ""]}) -- 0x7f87500b9f30 con 0x7f872c079810 2024-09-16T15:47:37.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:37.885+0000 7f8746000640 1 -- 172.21.15.151:0/1166561970 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f87500b9f30 con 0x7f872c079810 2024-09-16T15:47:37.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:37.890+0000 7f8755000640 1 -- 172.21.15.151:0/1166561970 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f872c079810 msgr2=0x7f872c07bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:37.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:37.890+0000 7f8755000640 1 --2- 172.21.15.151:0/1166561970 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f872c079810 0x7f872c07bcd0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8738003eb0 tx=0x7f8738009040 comp rx=0 tx=0).stop 2024-09-16T15:47:37.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:37.890+0000 7f8755000640 1 -- 172.21.15.151:0/1166561970 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8750161590 msgr2=0x7f8750156b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:37.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:37.890+0000 7f8755000640 1 --2- 172.21.15.151:0/1166561970 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8750161590 0x7f8750156b90 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f874000c410 tx=0x7f874000c8d0 comp rx=0 tx=0).stop 2024-09-16T15:47:37.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:37.891+0000 7f8755000640 1 -- 172.21.15.151:0/1166561970 shutdown_connections 2024-09-16T15:47:37.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:37.891+0000 7f8755000640 1 --2- 172.21.15.151:0/1166561970 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f872c079810 0x7f872c07bcd0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:37.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:37.891+0000 7f8755000640 1 --2- 172.21.15.151:0/1166561970 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8750167040 0x7f875015c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:37.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:37.891+0000 7f8755000640 1 --2- 172.21.15.151:0/1166561970 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8750161590 0x7f8750156b90 secure :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f874000c410 tx=0x7f874000c8d0 comp rx=0 tx=0).stop 2024-09-16T15:47:37.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:37.891+0000 7f8755000640 1 --2- 172.21.15.151:0/1166561970 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87500b5630 0x7f875015fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:37.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:37.891+0000 7f8755000640 1 -- 172.21.15.151:0/1166561970 >> 172.21.15.151:0/1166561970 conn(0x7f87500d3fb0 msgr2=0x7f8750167c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:37.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:37.893+0000 7f8755000640 1 -- 172.21.15.151:0/1166561970 shutdown_connections 2024-09-16T15:47:37.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:37.893+0000 7f8755000640 1 -- 172.21.15.151:0/1166561970 wait complete. 2024-09-16T15:47:37.934 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone72556 2024-09-16T15:47:38.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:38.269+0000 7f3bbfe00640 1 -- 172.21.15.151:0/3199386850 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3bc00c5730 msgr2=0x7f3bc00c6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:38.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:38.269+0000 7f3bbfe00640 1 --2- 172.21.15.151:0/3199386850 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3bc00c5730 0x7f3bc00c6430 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3ba8009830 tx=0x7f3ba802a510 comp rx=0 tx=0).stop 2024-09-16T15:47:38.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:38.269+0000 7f3bbfe00640 1 -- 172.21.15.151:0/3199386850 shutdown_connections 2024-09-16T15:47:38.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:38.269+0000 7f3bbfe00640 1 --2- 172.21.15.151:0/3199386850 >> v2:172.21.15.167:3300/0 conn(0x7f3bc006e660 0x7f3bc006eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:38.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:38.269+0000 7f3bbfe00640 1 --2- 172.21.15.151:0/3199386850 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3bc00c5730 0x7f3bc00c6430 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:38.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:38.269+0000 7f3bbfe00640 1 -- 172.21.15.151:0/3199386850 >> 172.21.15.151:0/3199386850 conn(0x7f3bc00cc490 msgr2=0x7f3bc00cc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:38.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:38.270+0000 7f3bbfe00640 1 -- 172.21.15.151:0/3199386850 shutdown_connections 2024-09-16T15:47:38.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:38.270+0000 7f3bbfe00640 1 -- 172.21.15.151:0/3199386850 wait complete. 2024-09-16T15:47:38.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:38.274+0000 7f3bbfe00640 1 Processor -- start 2024-09-16T15:47:38.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:38.275+0000 7f3bbfe00640 1 -- start start 2024-09-16T15:47:38.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:38.276+0000 7f3bbfe00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3bc006e660 0x7f3bc01492e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:38.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:38.276+0000 7f3bbfe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3bc00c5730 0x7f3bc0153620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:38.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:38.277+0000 7f3bbfe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bc015f670 0x7f3bc0153b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:38.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:38.277+0000 7f3bbfe00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc00ca4f0 con 0x7f3bc00c5730 2024-09-16T15:47:38.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:38.277+0000 7f3bbfe00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc00ca370 con 0x7f3bc015f670 2024-09-16T15:47:38.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:38.277+0000 7f3bbfe00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f3bc006f050 con 0x7f3bc006e660 2024-09-16T15:47:38.280 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:38.277+0000 7f3bb6000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3bc00c5730 0x7f3bc0153620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:38.280 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:38.277+0000 7f3bb6000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3bc00c5730 0x7f3bc0153620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49024/0 (socket says 172.21.15.151:49024) 2024-09-16T15:47:38.281 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:38.277+0000 7f3bb6000640 1 -- 172.21.15.151:0/2907663823 learned_addr learned my addr 172.21.15.151:0/2907663823 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:47:38.281 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:38.277+0000 7f3bb6000640 1 -- 172.21.15.151:0/2907663823 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3bc006e660 msgr2=0x7f3bc01492e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T15:47:38.281 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:38.277+0000 7f3bb6000640 1 --2- 172.21.15.151:0/2907663823 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3bc006e660 0x7f3bc01492e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:38.281 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:38.277+0000 7f3bb6000640 1 -- 172.21.15.151:0/2907663823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bc015f670 msgr2=0x7f3bc0153b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:38.281 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:38.277+0000 7f3bb6000640 1 --2- 172.21.15.151:0/2907663823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bc015f670 0x7f3bc0153b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:38.281 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:38.277+0000 7f3bb6000640 1 -- 172.21.15.151:0/2907663823 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bc0148240 con 0x7f3bc00c5730 2024-09-16T15:47:38.281 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:38.278+0000 7f3bb6000640 1 --2- 172.21.15.151:0/2907663823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3bc00c5730 0x7f3bc0153620 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f3ba8009950 tx=0x7f3ba8032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:38.281 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:38.278+0000 7f3bb7400640 1 -- 172.21.15.151:0/2907663823 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba8040070 con 0x7f3bc00c5730 2024-09-16T15:47:38.281 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:38.278+0000 7f3bb7400640 1 -- 172.21.15.151:0/2907663823 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ba80038e0 con 0x7f3bc00c5730 2024-09-16T15:47:38.281 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:38.278+0000 7f3bb7400640 1 -- 172.21.15.151:0/2907663823 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba8005c90 con 0x7f3bc00c5730 2024-09-16T15:47:38.281 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:38.278+0000 7f3bbfe00640 1 -- 172.21.15.151:0/2907663823 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bc0148520 con 0x7f3bc00c5730 2024-09-16T15:47:38.281 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:38.279+0000 7f3bbfe00640 1 -- 172.21.15.151:0/2907663823 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bc0148ae0 con 0x7f3bc00c5730 2024-09-16T15:47:38.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:38.281+0000 7f3bbfe00640 1 -- 172.21.15.151:0/2907663823 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bc006a150 con 0x7f3bc00c5730 2024-09-16T15:47:38.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:38.281+0000 7f3bb7400640 1 -- 172.21.15.151:0/2907663823 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 28) ==== 102766+0+0 (secure 0 0 0) 0x7f3ba8039040 con 0x7f3bc00c5730 2024-09-16T15:47:38.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:38.282+0000 7f3bb7400640 1 --2- 172.21.15.151:0/2907663823 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3b9c0798a0 0x7f3b9c07bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:38.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:38.282+0000 7f3bb7400640 1 -- 172.21.15.151:0/2907663823 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5881+0+0 (secure 0 0 0) 0x7f3ba80babc0 con 0x7f3bc00c5730 2024-09-16T15:47:38.293 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:38.292+0000 7f3bb7400640 1 -- 172.21.15.151:0/2907663823 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f3bc006a150 con 0x7f3bc00c5730 2024-09-16T15:47:38.304 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:38.303+0000 7f3bbea00640 1 --2- 172.21.15.151:0/2907663823 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3b9c0798a0 0x7f3b9c07bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:38.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:38.369+0000 7f3bbea00640 1 --2- 172.21.15.151:0/2907663823 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3b9c0798a0 0x7f3b9c07bd60 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f3bb0001220 tx=0x7f3bb00011b0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:38.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:38.735+0000 7f3bbfe00640 1 -- 172.21.15.151:0/2907663823 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone72556", "target": ["mon-mgr", ""]}) -- 0x7f3bc0149820 con 0x7f3b9c0798a0 2024-09-16T15:47:39.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:39.871+0000 7f3bb7400640 1 -- 172.21.15.151:0/2907663823 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3bc0149820 con 0x7f3b9c0798a0 2024-09-16T15:47:39.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:39.879+0000 7f3bbfe00640 1 -- 172.21.15.151:0/2907663823 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3b9c0798a0 msgr2=0x7f3b9c07bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:39.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:39.879+0000 7f3bbfe00640 1 --2- 172.21.15.151:0/2907663823 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3b9c0798a0 0x7f3b9c07bd60 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f3bb0001220 tx=0x7f3bb00011b0 comp rx=0 tx=0).stop 2024-09-16T15:47:39.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:39.879+0000 7f3bbfe00640 1 -- 172.21.15.151:0/2907663823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3bc00c5730 msgr2=0x7f3bc0153620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:39.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:39.879+0000 7f3bbfe00640 1 --2- 172.21.15.151:0/2907663823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3bc00c5730 0x7f3bc0153620 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f3ba8009950 tx=0x7f3ba8032c10 comp rx=0 tx=0).stop 2024-09-16T15:47:39.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:39.881+0000 7f3bbfe00640 1 -- 172.21.15.151:0/2907663823 shutdown_connections 2024-09-16T15:47:39.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:39.881+0000 7f3bbfe00640 1 --2- 172.21.15.151:0/2907663823 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3b9c0798a0 0x7f3b9c07bd60 secure :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f3bb0001220 tx=0x7f3bb00011b0 comp rx=0 tx=0).stop 2024-09-16T15:47:39.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:39.881+0000 7f3bbfe00640 1 --2- 172.21.15.151:0/2907663823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bc015f670 0x7f3bc0153b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:39.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:39.881+0000 7f3bbfe00640 1 --2- 172.21.15.151:0/2907663823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3bc00c5730 0x7f3bc0153620 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:39.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:39.881+0000 7f3bbfe00640 1 --2- 172.21.15.151:0/2907663823 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3bc006e660 0x7f3bc01492e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:39.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:39.881+0000 7f3bbfe00640 1 -- 172.21.15.151:0/2907663823 >> 172.21.15.151:0/2907663823 conn(0x7f3bc00cc490 msgr2=0x7f3bc00cc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:39.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:39.884+0000 7f3bbfe00640 1 -- 172.21.15.151:0/2907663823 shutdown_connections 2024-09-16T15:47:39.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:39.884+0000 7f3bbfe00640 1 -- 172.21.15.151:0/2907663823 wait complete. 2024-09-16T15:47:39.915 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-16T15:47:40.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:40.243+0000 7fb951a00640 1 -- 172.21.15.151:0/184560543 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb930004800 msgr2=0x7fb930004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:40.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:40.243+0000 7fb951a00640 1 --2- 172.21.15.151:0/184560543 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb930004800 0x7fb930004c80 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fb93800c6c0 tx=0x7fb930005830 comp rx=0 tx=0).stop 2024-09-16T15:47:40.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:40.245+0000 7fb951a00640 1 -- 172.21.15.151:0/184560543 shutdown_connections 2024-09-16T15:47:40.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:40.246+0000 7fb951a00640 1 --2- 172.21.15.151:0/184560543 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb930004800 0x7fb930004c80 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:40.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:40.246+0000 7fb951a00640 1 -- 172.21.15.151:0/184560543 >> 172.21.15.151:0/184560543 conn(0x7fb94c0b18e0 msgr2=0x7fb94c0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:40.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:40.246+0000 7fb951a00640 1 -- 172.21.15.151:0/184560543 shutdown_connections 2024-09-16T15:47:40.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:40.246+0000 7fb951a00640 1 -- 172.21.15.151:0/184560543 wait complete. 2024-09-16T15:47:40.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:40.250+0000 7fb951a00640 1 Processor -- start 2024-09-16T15:47:40.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:40.250+0000 7fb951a00640 1 -- start start 2024-09-16T15:47:40.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:40.252+0000 7fb951a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb930004800 0x7fb94c09bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:40.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:40.252+0000 7fb951a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb94c14d730 0x7fb94c0cf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:40.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:40.252+0000 7fb951a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb94c0ae590 0x7fb94c0aea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:40.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:40.252+0000 7fb951a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb94c0cd2d0 con 0x7fb930004800 2024-09-16T15:47:40.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:40.252+0000 7fb951a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb94c072a80 con 0x7fb94c14d730 2024-09-16T15:47:40.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:40.252+0000 7fb951a00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fb94c0cd450 con 0x7fb94c0ae590 2024-09-16T15:47:40.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:40.252+0000 7fb94aa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb930004800 0x7fb94c09bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:40.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:40.253+0000 7fb94aa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb930004800 0x7fb94c09bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49048/0 (socket says 172.21.15.151:49048) 2024-09-16T15:47:40.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:40.253+0000 7fb94aa00640 1 -- 172.21.15.151:0/1626842100 learned_addr learned my addr 172.21.15.151:0/1626842100 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:47:40.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:40.253+0000 7fb94a000640 1 --2- 172.21.15.151:0/1626842100 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb94c14d730 0x7fb94c0cf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:40.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:40.253+0000 7fb94b400640 1 --2- 172.21.15.151:0/1626842100 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb94c0ae590 0x7fb94c0aea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:40.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:40.254+0000 7fb94aa00640 1 -- 172.21.15.151:0/1626842100 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb94c0ae590 msgr2=0x7fb94c0aea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:40.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:40.254+0000 7fb94aa00640 1 --2- 172.21.15.151:0/1626842100 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb94c0ae590 0x7fb94c0aea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:40.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:40.254+0000 7fb94aa00640 1 -- 172.21.15.151:0/1626842100 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb94c14d730 msgr2=0x7fb94c0cf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:40.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:40.254+0000 7fb94aa00640 1 --2- 172.21.15.151:0/1626842100 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb94c14d730 0x7fb94c0cf5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:40.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:40.254+0000 7fb94aa00640 1 -- 172.21.15.151:0/1626842100 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb94c09c1d0 con 0x7fb930004800 2024-09-16T15:47:40.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:40.254+0000 7fb94aa00640 1 --2- 172.21.15.151:0/1626842100 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb930004800 0x7fb94c09bbd0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fb93c00d630 tx=0x7fb93c00daf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:40.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:40.255+0000 7fb942a00640 1 -- 172.21.15.151:0/1626842100 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb93c017070 con 0x7fb930004800 2024-09-16T15:47:40.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:40.255+0000 7fb942a00640 1 -- 172.21.15.151:0/1626842100 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb93c005310 con 0x7fb930004800 2024-09-16T15:47:40.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:40.255+0000 7fb942a00640 1 -- 172.21.15.151:0/1626842100 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb93c0047a0 con 0x7fb930004800 2024-09-16T15:47:40.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:40.255+0000 7fb951a00640 1 -- 172.21.15.151:0/1626842100 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb94c09c4b0 con 0x7fb930004800 2024-09-16T15:47:40.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:40.255+0000 7fb951a00640 1 -- 172.21.15.151:0/1626842100 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb94c09ca70 con 0x7fb930004800 2024-09-16T15:47:40.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:40.258+0000 7fb951a00640 1 -- 172.21.15.151:0/1626842100 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb94c069360 con 0x7fb930004800 2024-09-16T15:47:40.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:40.258+0000 7fb942a00640 1 -- 172.21.15.151:0/1626842100 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 28) ==== 102766+0+0 (secure 0 0 0) 0x7fb93c004940 con 0x7fb930004800 2024-09-16T15:47:40.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:40.259+0000 7fb942a00640 1 --2- 172.21.15.151:0/1626842100 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb92c0798d0 0x7fb92c07bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:40.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:40.260+0000 7fb942a00640 1 -- 172.21.15.151:0/1626842100 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5881+0+0 (secure 0 0 0) 0x7fb93c099730 con 0x7fb930004800 2024-09-16T15:47:40.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:40.268+0000 7fb942a00640 1 -- 172.21.15.151:0/1626842100 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb93c062e80 con 0x7fb930004800 2024-09-16T15:47:40.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:40.283+0000 7fb94a000640 1 --2- 172.21.15.151:0/1626842100 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb92c0798d0 0x7fb92c07bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:40.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:40.290+0000 7fb94a000640 1 --2- 172.21.15.151:0/1626842100 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb92c0798d0 0x7fb92c07bd90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb938032000 tx=0x7fb938001020 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:40.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:40.785+0000 7fb951a00640 1 -- 172.21.15.151:0/1626842100 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fb94c161b90 con 0x7fb930004800 2024-09-16T15:47:40.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:40.785+0000 7fb942a00640 1 -- 172.21.15.151:0/1626842100 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2183 (secure 0 0 0) 0x7fb93c0678b0 con 0x7fb930004800 2024-09-16T15:47:40.787 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:47:40.787 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-16T15:47:40.787 INFO:teuthology.orchestra.run.smithi151.stdout: "fsid": "1757be18-e41a-4446-bb4f-ee4ac81c39df", 2024-09-16T15:47:40.787 INFO:teuthology.orchestra.run.smithi151.stdout: "health": { 2024-09-16T15:47:40.787 INFO:teuthology.orchestra.run.smithi151.stdout: "status": "HEALTH_OK", 2024-09-16T15:47:40.788 INFO:teuthology.orchestra.run.smithi151.stdout: "checks": {}, 2024-09-16T15:47:40.788 INFO:teuthology.orchestra.run.smithi151.stdout: "mutes": [] 2024-09-16T15:47:40.788 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:47:40.788 INFO:teuthology.orchestra.run.smithi151.stdout: "election_epoch": 4, 2024-09-16T15:47:40.788 INFO:teuthology.orchestra.run.smithi151.stdout: "quorum": [ 2024-09-16T15:47:40.788 INFO:teuthology.orchestra.run.smithi151.stdout: 0, 2024-09-16T15:47:40.788 INFO:teuthology.orchestra.run.smithi151.stdout: 1, 2024-09-16T15:47:40.788 INFO:teuthology.orchestra.run.smithi151.stdout: 2 2024-09-16T15:47:40.788 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:47:40.788 INFO:teuthology.orchestra.run.smithi151.stdout: "quorum_names": [ 2024-09-16T15:47:40.788 INFO:teuthology.orchestra.run.smithi151.stdout: "a", 2024-09-16T15:47:40.788 INFO:teuthology.orchestra.run.smithi151.stdout: "b", 2024-09-16T15:47:40.788 INFO:teuthology.orchestra.run.smithi151.stdout: "c" 2024-09-16T15:47:40.789 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:47:40.789 INFO:teuthology.orchestra.run.smithi151.stdout: "quorum_age": 524, 2024-09-16T15:47:40.789 INFO:teuthology.orchestra.run.smithi151.stdout: "monmap": { 2024-09-16T15:47:40.789 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 1, 2024-09-16T15:47:40.789 INFO:teuthology.orchestra.run.smithi151.stdout: "min_mon_release_name": "squid", 2024-09-16T15:47:40.789 INFO:teuthology.orchestra.run.smithi151.stdout: "num_mons": 3 2024-09-16T15:47:40.789 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:47:40.789 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap": { 2024-09-16T15:47:40.789 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 78, 2024-09-16T15:47:40.789 INFO:teuthology.orchestra.run.smithi151.stdout: "num_osds": 8, 2024-09-16T15:47:40.789 INFO:teuthology.orchestra.run.smithi151.stdout: "num_up_osds": 8, 2024-09-16T15:47:40.789 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_up_since": 1726501150, 2024-09-16T15:47:40.789 INFO:teuthology.orchestra.run.smithi151.stdout: "num_in_osds": 8, 2024-09-16T15:47:40.789 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_in_since": 1726501140, 2024-09-16T15:47:40.790 INFO:teuthology.orchestra.run.smithi151.stdout: "num_remapped_pgs": 0 2024-09-16T15:47:40.790 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:47:40.790 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap": { 2024-09-16T15:47:40.790 INFO:teuthology.orchestra.run.smithi151.stdout: "pgs_by_state": [ 2024-09-16T15:47:40.790 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:47:40.790 INFO:teuthology.orchestra.run.smithi151.stdout: "state_name": "active+clean", 2024-09-16T15:47:40.790 INFO:teuthology.orchestra.run.smithi151.stdout: "count": 97 2024-09-16T15:47:40.790 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:47:40.790 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:47:40.790 INFO:teuthology.orchestra.run.smithi151.stdout: "num_pgs": 97, 2024-09-16T15:47:40.790 INFO:teuthology.orchestra.run.smithi151.stdout: "num_pools": 3, 2024-09-16T15:47:40.790 INFO:teuthology.orchestra.run.smithi151.stdout: "num_objects": 1597, 2024-09-16T15:47:40.790 INFO:teuthology.orchestra.run.smithi151.stdout: "data_bytes": 6548523946, 2024-09-16T15:47:40.791 INFO:teuthology.orchestra.run.smithi151.stdout: "bytes_used": 12669362176, 2024-09-16T15:47:40.791 INFO:teuthology.orchestra.run.smithi151.stdout: "bytes_avail": 760424751104, 2024-09-16T15:47:40.791 INFO:teuthology.orchestra.run.smithi151.stdout: "bytes_total": 773094113280, 2024-09-16T15:47:40.791 INFO:teuthology.orchestra.run.smithi151.stdout: "read_bytes_sec": 288753788, 2024-09-16T15:47:40.791 INFO:teuthology.orchestra.run.smithi151.stdout: "write_bytes_sec": 358276787, 2024-09-16T15:47:40.791 INFO:teuthology.orchestra.run.smithi151.stdout: "read_op_per_sec": 70, 2024-09-16T15:47:40.791 INFO:teuthology.orchestra.run.smithi151.stdout: "write_op_per_sec": 98 2024-09-16T15:47:40.791 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:47:40.791 INFO:teuthology.orchestra.run.smithi151.stdout: "fsmap": { 2024-09-16T15:47:40.791 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 47, 2024-09-16T15:47:40.791 INFO:teuthology.orchestra.run.smithi151.stdout: "btime": "2024-09-16T15:47:01:291406+0000", 2024-09-16T15:47:40.791 INFO:teuthology.orchestra.run.smithi151.stdout: "id": 5, 2024-09-16T15:47:40.791 INFO:teuthology.orchestra.run.smithi151.stdout: "up": 1, 2024-09-16T15:47:40.791 INFO:teuthology.orchestra.run.smithi151.stdout: "in": 1, 2024-09-16T15:47:40.792 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 1, 2024-09-16T15:47:40.792 INFO:teuthology.orchestra.run.smithi151.stdout: "by_rank": [ 2024-09-16T15:47:40.792 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:47:40.792 INFO:teuthology.orchestra.run.smithi151.stdout: "filesystem_id": 5, 2024-09-16T15:47:40.792 INFO:teuthology.orchestra.run.smithi151.stdout: "rank": 0, 2024-09-16T15:47:40.792 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "a", 2024-09-16T15:47:40.792 INFO:teuthology.orchestra.run.smithi151.stdout: "status": "up:active", 2024-09-16T15:47:40.792 INFO:teuthology.orchestra.run.smithi151.stdout: "gid": 6456 2024-09-16T15:47:40.792 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:47:40.792 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:47:40.792 INFO:teuthology.orchestra.run.smithi151.stdout: "up:standby": 3 2024-09-16T15:47:40.792 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:47:40.792 INFO:teuthology.orchestra.run.smithi151.stdout: "mgrmap": { 2024-09-16T15:47:40.792 INFO:teuthology.orchestra.run.smithi151.stdout: "available": true, 2024-09-16T15:47:40.793 INFO:teuthology.orchestra.run.smithi151.stdout: "num_standbys": 1, 2024-09-16T15:47:40.793 INFO:teuthology.orchestra.run.smithi151.stdout: "modules": [ 2024-09-16T15:47:40.793 INFO:teuthology.orchestra.run.smithi151.stdout: "iostat", 2024-09-16T15:47:40.793 INFO:teuthology.orchestra.run.smithi151.stdout: "nfs", 2024-09-16T15:47:40.793 INFO:teuthology.orchestra.run.smithi151.stdout: "restful" 2024-09-16T15:47:40.793 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:47:40.793 INFO:teuthology.orchestra.run.smithi151.stdout: "services": {} 2024-09-16T15:47:40.793 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:47:40.793 INFO:teuthology.orchestra.run.smithi151.stdout: "servicemap": { 2024-09-16T15:47:40.793 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 10, 2024-09-16T15:47:40.793 INFO:teuthology.orchestra.run.smithi151.stdout: "modified": "2024-09-16T15:45:53.688171+0000", 2024-09-16T15:47:40.793 INFO:teuthology.orchestra.run.smithi151.stdout: "services": {} 2024-09-16T15:47:40.793 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:47:40.794 INFO:teuthology.orchestra.run.smithi151.stdout: "progress_events": { 2024-09-16T15:47:40.794 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr-vol-ongoing-clones": { 2024-09-16T15:47:40.794 INFO:teuthology.orchestra.run.smithi151.stdout: "message": "4 ongoing clones - average progress is 30.013% (5s)\n [========....................] (remaining: 13s)", 2024-09-16T15:47:40.794 INFO:teuthology.orchestra.run.smithi151.stdout: "progress": 0.30013000965118408, 2024-09-16T15:47:40.794 INFO:teuthology.orchestra.run.smithi151.stdout: "add_to_ceph_s": true 2024-09-16T15:47:40.794 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:47:40.794 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:47:40.794 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-16T15:47:40.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:40.789+0000 7fb951a00640 1 -- 172.21.15.151:0/1626842100 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb92c0798d0 msgr2=0x7fb92c07bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:40.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:40.789+0000 7fb951a00640 1 --2- 172.21.15.151:0/1626842100 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb92c0798d0 0x7fb92c07bd90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb938032000 tx=0x7fb938001020 comp rx=0 tx=0).stop 2024-09-16T15:47:40.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:40.789+0000 7fb951a00640 1 -- 172.21.15.151:0/1626842100 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb930004800 msgr2=0x7fb94c09bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:40.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:40.789+0000 7fb951a00640 1 --2- 172.21.15.151:0/1626842100 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb930004800 0x7fb94c09bbd0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fb93c00d630 tx=0x7fb93c00daf0 comp rx=0 tx=0).stop 2024-09-16T15:47:40.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:40.789+0000 7fb951a00640 1 -- 172.21.15.151:0/1626842100 shutdown_connections 2024-09-16T15:47:40.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:40.789+0000 7fb951a00640 1 --2- 172.21.15.151:0/1626842100 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb92c0798d0 0x7fb92c07bd90 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:40.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:40.789+0000 7fb951a00640 1 --2- 172.21.15.151:0/1626842100 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb94c14d730 0x7fb94c0cf5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:40.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:40.789+0000 7fb951a00640 1 --2- 172.21.15.151:0/1626842100 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb94c0ae590 0x7fb94c0aea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:40.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:40.789+0000 7fb951a00640 1 --2- 172.21.15.151:0/1626842100 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb930004800 0x7fb94c09bbd0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:40.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:40.789+0000 7fb951a00640 1 -- 172.21.15.151:0/1626842100 >> 172.21.15.151:0/1626842100 conn(0x7fb94c0b18e0 msgr2=0x7fb94c1609f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:40.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:40.790+0000 7fb951a00640 1 -- 172.21.15.151:0/1626842100 shutdown_connections 2024-09-16T15:47:40.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:40.790+0000 7fb951a00640 1 -- 172.21.15.151:0/1626842100 wait complete. 2024-09-16T15:47:41.821 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-16T15:47:42.135 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.134+0000 7efc74a00640 1 -- 172.21.15.151:0/3206175262 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc540051c0 msgr2=0x7efc5400dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:42.135 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.134+0000 7efc74a00640 1 --2- 172.21.15.151:0/3206175262 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc540051c0 0x7efc5400dbd0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7efc6800e3d0 tx=0x7efc54005830 comp rx=0 tx=0).stop 2024-09-16T15:47:42.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.135+0000 7efc74a00640 1 -- 172.21.15.151:0/3206175262 shutdown_connections 2024-09-16T15:47:42.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.135+0000 7efc74a00640 1 --2- 172.21.15.151:0/3206175262 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc540051c0 0x7efc5400dbd0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:42.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.135+0000 7efc74a00640 1 -- 172.21.15.151:0/3206175262 >> 172.21.15.151:0/3206175262 conn(0x7efc700b18e0 msgr2=0x7efc700b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:42.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.135+0000 7efc74a00640 1 -- 172.21.15.151:0/3206175262 shutdown_connections 2024-09-16T15:47:42.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.135+0000 7efc74a00640 1 -- 172.21.15.151:0/3206175262 wait complete. 2024-09-16T15:47:42.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.138+0000 7efc74a00640 1 Processor -- start 2024-09-16T15:47:42.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.138+0000 7efc74a00640 1 -- start start 2024-09-16T15:47:42.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.139+0000 7efc74a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efc540051c0 0x7efc70156630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:42.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.139+0000 7efc74a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc700c8f60 0x7efc7015a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:42.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.139+0000 7efc74a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc7015aed0 0x7efc701e52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:42.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.139+0000 7efc74a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efc700cc480 con 0x7efc7015aed0 2024-09-16T15:47:42.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.139+0000 7efc74a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efc700cc300 con 0x7efc700c8f60 2024-09-16T15:47:42.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.139+0000 7efc74a00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7efc700b48b0 con 0x7efc540051c0 2024-09-16T15:47:42.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.140+0000 7efc6fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc7015aed0 0x7efc701e52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:42.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.140+0000 7efc6fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc7015aed0 0x7efc701e52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49070/0 (socket says 172.21.15.151:49070) 2024-09-16T15:47:42.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.140+0000 7efc6fe00640 1 -- 172.21.15.151:0/1123510281 learned_addr learned my addr 172.21.15.151:0/1123510281 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:47:42.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.140+0000 7efc66a00640 1 --2- 172.21.15.151:0/1123510281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc700c8f60 0x7efc7015a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:42.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.141+0000 7efc6fe00640 1 -- 172.21.15.151:0/1123510281 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efc540051c0 msgr2=0x7efc70156630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:42.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.141+0000 7efc6fe00640 1 --2- 172.21.15.151:0/1123510281 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efc540051c0 0x7efc70156630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:42.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.141+0000 7efc6fe00640 1 -- 172.21.15.151:0/1123510281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc700c8f60 msgr2=0x7efc7015a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:42.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.141+0000 7efc6fe00640 1 --2- 172.21.15.151:0/1123510281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc700c8f60 0x7efc7015a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:42.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.141+0000 7efc6fe00640 1 -- 172.21.15.151:0/1123510281 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc70156fd0 con 0x7efc7015aed0 2024-09-16T15:47:42.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.142+0000 7efc6fe00640 1 --2- 172.21.15.151:0/1123510281 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc7015aed0 0x7efc701e52a0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7efc6800e4f0 tx=0x7efc68032f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:42.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.143+0000 7efc67e00640 1 -- 172.21.15.151:0/1123510281 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc68037070 con 0x7efc7015aed0 2024-09-16T15:47:42.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.143+0000 7efc67e00640 1 -- 172.21.15.151:0/1123510281 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc68038040 con 0x7efc7015aed0 2024-09-16T15:47:42.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.143+0000 7efc67e00640 1 -- 172.21.15.151:0/1123510281 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc68004d80 con 0x7efc7015aed0 2024-09-16T15:47:42.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.143+0000 7efc74a00640 1 -- 172.21.15.151:0/1123510281 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc700c93e0 con 0x7efc7015aed0 2024-09-16T15:47:42.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.143+0000 7efc74a00640 1 -- 172.21.15.151:0/1123510281 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc701f4b40 con 0x7efc7015aed0 2024-09-16T15:47:42.146 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.145+0000 7efc74a00640 1 -- 172.21.15.151:0/1123510281 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc700cc680 con 0x7efc7015aed0 2024-09-16T15:47:42.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.150+0000 7efc67e00640 1 -- 172.21.15.151:0/1123510281 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 28) ==== 102766+0+0 (secure 0 0 0) 0x7efc68046020 con 0x7efc7015aed0 2024-09-16T15:47:42.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.153+0000 7efc67e00640 1 --2- 172.21.15.151:0/1123510281 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efc4c079970 0x7efc4c07be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:42.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.154+0000 7efc67e00640 1 -- 172.21.15.151:0/1123510281 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5881+0+0 (secure 0 0 0) 0x7efc680ba840 con 0x7efc7015aed0 2024-09-16T15:47:42.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.157+0000 7efc67e00640 1 -- 172.21.15.151:0/1123510281 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7efc700c93e0 con 0x7efc7015aed0 2024-09-16T15:47:42.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.165+0000 7efc6f400640 1 --2- 172.21.15.151:0/1123510281 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efc4c079970 0x7efc4c07be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:42.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.170+0000 7efc6f400640 1 --2- 172.21.15.151:0/1123510281 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efc4c079970 0x7efc4c07be30 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7efc60000fb0 tx=0x7efc600014f0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:42.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.632+0000 7efc74a00640 1 -- 172.21.15.151:0/1123510281 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7efc700af530 con 0x7efc7015aed0 2024-09-16T15:47:42.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.632+0000 7efc67e00640 1 -- 172.21.15.151:0/1123510281 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2424 (secure 0 0 0) 0x7efc70156fd0 con 0x7efc7015aed0 2024-09-16T15:47:42.634 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:47:42.634 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-16T15:47:42.634 INFO:teuthology.orchestra.run.smithi151.stdout: "fsid": "1757be18-e41a-4446-bb4f-ee4ac81c39df", 2024-09-16T15:47:42.634 INFO:teuthology.orchestra.run.smithi151.stdout: "health": { 2024-09-16T15:47:42.634 INFO:teuthology.orchestra.run.smithi151.stdout: "status": "HEALTH_OK", 2024-09-16T15:47:42.634 INFO:teuthology.orchestra.run.smithi151.stdout: "checks": {}, 2024-09-16T15:47:42.634 INFO:teuthology.orchestra.run.smithi151.stdout: "mutes": [] 2024-09-16T15:47:42.634 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:47:42.634 INFO:teuthology.orchestra.run.smithi151.stdout: "election_epoch": 4, 2024-09-16T15:47:42.634 INFO:teuthology.orchestra.run.smithi151.stdout: "quorum": [ 2024-09-16T15:47:42.634 INFO:teuthology.orchestra.run.smithi151.stdout: 0, 2024-09-16T15:47:42.634 INFO:teuthology.orchestra.run.smithi151.stdout: 1, 2024-09-16T15:47:42.634 INFO:teuthology.orchestra.run.smithi151.stdout: 2 2024-09-16T15:47:42.634 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:47:42.635 INFO:teuthology.orchestra.run.smithi151.stdout: "quorum_names": [ 2024-09-16T15:47:42.635 INFO:teuthology.orchestra.run.smithi151.stdout: "a", 2024-09-16T15:47:42.635 INFO:teuthology.orchestra.run.smithi151.stdout: "b", 2024-09-16T15:47:42.635 INFO:teuthology.orchestra.run.smithi151.stdout: "c" 2024-09-16T15:47:42.635 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:47:42.635 INFO:teuthology.orchestra.run.smithi151.stdout: "quorum_age": 526, 2024-09-16T15:47:42.635 INFO:teuthology.orchestra.run.smithi151.stdout: "monmap": { 2024-09-16T15:47:42.635 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 1, 2024-09-16T15:47:42.635 INFO:teuthology.orchestra.run.smithi151.stdout: "min_mon_release_name": "squid", 2024-09-16T15:47:42.635 INFO:teuthology.orchestra.run.smithi151.stdout: "num_mons": 3 2024-09-16T15:47:42.635 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:47:42.635 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap": { 2024-09-16T15:47:42.635 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 78, 2024-09-16T15:47:42.635 INFO:teuthology.orchestra.run.smithi151.stdout: "num_osds": 8, 2024-09-16T15:47:42.635 INFO:teuthology.orchestra.run.smithi151.stdout: "num_up_osds": 8, 2024-09-16T15:47:42.635 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_up_since": 1726501150, 2024-09-16T15:47:42.636 INFO:teuthology.orchestra.run.smithi151.stdout: "num_in_osds": 8, 2024-09-16T15:47:42.636 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_in_since": 1726501140, 2024-09-16T15:47:42.636 INFO:teuthology.orchestra.run.smithi151.stdout: "num_remapped_pgs": 0 2024-09-16T15:47:42.636 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:47:42.636 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap": { 2024-09-16T15:47:42.636 INFO:teuthology.orchestra.run.smithi151.stdout: "pgs_by_state": [ 2024-09-16T15:47:42.636 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:47:42.636 INFO:teuthology.orchestra.run.smithi151.stdout: "state_name": "active+clean", 2024-09-16T15:47:42.636 INFO:teuthology.orchestra.run.smithi151.stdout: "count": 97 2024-09-16T15:47:42.636 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:47:42.636 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:47:42.636 INFO:teuthology.orchestra.run.smithi151.stdout: "num_pgs": 97, 2024-09-16T15:47:42.636 INFO:teuthology.orchestra.run.smithi151.stdout: "num_pools": 3, 2024-09-16T15:47:42.636 INFO:teuthology.orchestra.run.smithi151.stdout: "num_objects": 2444, 2024-09-16T15:47:42.637 INFO:teuthology.orchestra.run.smithi151.stdout: "data_bytes": 10101319650, 2024-09-16T15:47:42.637 INFO:teuthology.orchestra.run.smithi151.stdout: "bytes_used": 20047646720, 2024-09-16T15:47:42.637 INFO:teuthology.orchestra.run.smithi151.stdout: "bytes_avail": 753046466560, 2024-09-16T15:47:42.637 INFO:teuthology.orchestra.run.smithi151.stdout: "bytes_total": 773094113280, 2024-09-16T15:47:42.637 INFO:teuthology.orchestra.run.smithi151.stdout: "read_bytes_sec": 586932533, 2024-09-16T15:47:42.637 INFO:teuthology.orchestra.run.smithi151.stdout: "write_bytes_sec": 654031555, 2024-09-16T15:47:42.637 INFO:teuthology.orchestra.run.smithi151.stdout: "read_op_per_sec": 142, 2024-09-16T15:47:42.637 INFO:teuthology.orchestra.run.smithi151.stdout: "write_op_per_sec": 175 2024-09-16T15:47:42.637 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:47:42.637 INFO:teuthology.orchestra.run.smithi151.stdout: "fsmap": { 2024-09-16T15:47:42.637 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 47, 2024-09-16T15:47:42.637 INFO:teuthology.orchestra.run.smithi151.stdout: "btime": "2024-09-16T15:47:01:291406+0000", 2024-09-16T15:47:42.637 INFO:teuthology.orchestra.run.smithi151.stdout: "id": 5, 2024-09-16T15:47:42.637 INFO:teuthology.orchestra.run.smithi151.stdout: "up": 1, 2024-09-16T15:47:42.637 INFO:teuthology.orchestra.run.smithi151.stdout: "in": 1, 2024-09-16T15:47:42.637 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 1, 2024-09-16T15:47:42.638 INFO:teuthology.orchestra.run.smithi151.stdout: "by_rank": [ 2024-09-16T15:47:42.638 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:47:42.638 INFO:teuthology.orchestra.run.smithi151.stdout: "filesystem_id": 5, 2024-09-16T15:47:42.638 INFO:teuthology.orchestra.run.smithi151.stdout: "rank": 0, 2024-09-16T15:47:42.638 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "a", 2024-09-16T15:47:42.638 INFO:teuthology.orchestra.run.smithi151.stdout: "status": "up:active", 2024-09-16T15:47:42.638 INFO:teuthology.orchestra.run.smithi151.stdout: "gid": 6456 2024-09-16T15:47:42.638 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:47:42.638 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:47:42.638 INFO:teuthology.orchestra.run.smithi151.stdout: "up:standby": 3 2024-09-16T15:47:42.638 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:47:42.638 INFO:teuthology.orchestra.run.smithi151.stdout: "mgrmap": { 2024-09-16T15:47:42.638 INFO:teuthology.orchestra.run.smithi151.stdout: "available": true, 2024-09-16T15:47:42.638 INFO:teuthology.orchestra.run.smithi151.stdout: "num_standbys": 1, 2024-09-16T15:47:42.638 INFO:teuthology.orchestra.run.smithi151.stdout: "modules": [ 2024-09-16T15:47:42.639 INFO:teuthology.orchestra.run.smithi151.stdout: "iostat", 2024-09-16T15:47:42.639 INFO:teuthology.orchestra.run.smithi151.stdout: "nfs", 2024-09-16T15:47:42.639 INFO:teuthology.orchestra.run.smithi151.stdout: "restful" 2024-09-16T15:47:42.639 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:47:42.639 INFO:teuthology.orchestra.run.smithi151.stdout: "services": {} 2024-09-16T15:47:42.639 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:47:42.639 INFO:teuthology.orchestra.run.smithi151.stdout: "servicemap": { 2024-09-16T15:47:42.639 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 10, 2024-09-16T15:47:42.639 INFO:teuthology.orchestra.run.smithi151.stdout: "modified": "2024-09-16T15:45:53.688171+0000", 2024-09-16T15:47:42.639 INFO:teuthology.orchestra.run.smithi151.stdout: "services": {} 2024-09-16T15:47:42.639 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:47:42.639 INFO:teuthology.orchestra.run.smithi151.stdout: "progress_events": { 2024-09-16T15:47:42.639 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr-vol-ongoing-clones": { 2024-09-16T15:47:42.639 INFO:teuthology.orchestra.run.smithi151.stdout: "message": "4 ongoing clones - average progress is 36.068% (9s)\n [==========..................] (remaining: 16s)", 2024-09-16T15:47:42.639 INFO:teuthology.orchestra.run.smithi151.stdout: "progress": 0.36068001389503479, 2024-09-16T15:47:42.639 INFO:teuthology.orchestra.run.smithi151.stdout: "add_to_ceph_s": true 2024-09-16T15:47:42.640 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:47:42.640 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr-vol-total-clones": { 2024-09-16T15:47:42.640 INFO:teuthology.orchestra.run.smithi151.stdout: "message": "Total 6 clones - average progress is 31.337% (0s)\n [========....................] ", 2024-09-16T15:47:42.640 INFO:teuthology.orchestra.run.smithi151.stdout: "progress": 0.3133699893951416, 2024-09-16T15:47:42.640 INFO:teuthology.orchestra.run.smithi151.stdout: "add_to_ceph_s": true 2024-09-16T15:47:42.640 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:47:42.640 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:47:42.640 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-16T15:47:42.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.636+0000 7efc74a00640 1 -- 172.21.15.151:0/1123510281 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efc4c079970 msgr2=0x7efc4c07be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:42.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.636+0000 7efc74a00640 1 --2- 172.21.15.151:0/1123510281 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efc4c079970 0x7efc4c07be30 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7efc60000fb0 tx=0x7efc600014f0 comp rx=0 tx=0).stop 2024-09-16T15:47:42.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.636+0000 7efc74a00640 1 -- 172.21.15.151:0/1123510281 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc7015aed0 msgr2=0x7efc701e52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:42.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.636+0000 7efc74a00640 1 --2- 172.21.15.151:0/1123510281 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc7015aed0 0x7efc701e52a0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7efc6800e4f0 tx=0x7efc68032f80 comp rx=0 tx=0).stop 2024-09-16T15:47:42.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.637+0000 7efc74a00640 1 -- 172.21.15.151:0/1123510281 shutdown_connections 2024-09-16T15:47:42.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.637+0000 7efc74a00640 1 --2- 172.21.15.151:0/1123510281 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efc4c079970 0x7efc4c07be30 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:42.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.637+0000 7efc74a00640 1 --2- 172.21.15.151:0/1123510281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc700c8f60 0x7efc7015a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:42.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.637+0000 7efc74a00640 1 --2- 172.21.15.151:0/1123510281 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc7015aed0 0x7efc701e52a0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:42.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.637+0000 7efc74a00640 1 --2- 172.21.15.151:0/1123510281 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efc540051c0 0x7efc70156630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:42.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.637+0000 7efc74a00640 1 -- 172.21.15.151:0/1123510281 >> 172.21.15.151:0/1123510281 conn(0x7efc700b18e0 msgr2=0x7efc700cde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:42.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.643+0000 7efc74a00640 1 -- 172.21.15.151:0/1123510281 shutdown_connections 2024-09-16T15:47:42.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.644+0000 7efc74a00640 1 -- 172.21.15.151:0/1123510281 wait complete. 2024-09-16T15:47:42.675 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone74020 2024-09-16T15:47:42.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.927+0000 7f04ca200640 1 -- 172.21.15.151:0/3964564553 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f04c40b45d0 msgr2=0x7f04c406e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:42.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.927+0000 7f04ca200640 1 --2- 172.21.15.151:0/3964564553 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f04c40b45d0 0x7f04c406e860 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f04b4005fd0 tx=0x7f04b402b700 comp rx=0 tx=0).stop 2024-09-16T15:47:42.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.928+0000 7f04ca200640 1 -- 172.21.15.151:0/3964564553 shutdown_connections 2024-09-16T15:47:42.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.928+0000 7f04ca200640 1 --2- 172.21.15.151:0/3964564553 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f04c40b45d0 0x7f04c406e860 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:42.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.928+0000 7f04ca200640 1 --2- 172.21.15.151:0/3964564553 >> v2:172.21.15.151:3300/0 conn(0x7f04c41437c0 0x7f04c40cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:42.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.928+0000 7f04ca200640 1 -- 172.21.15.151:0/3964564553 >> 172.21.15.151:0/3964564553 conn(0x7f04c40b1840 msgr2=0x7f04c40b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:42.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.929+0000 7f04ca200640 1 -- 172.21.15.151:0/3964564553 shutdown_connections 2024-09-16T15:47:42.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.929+0000 7f04ca200640 1 -- 172.21.15.151:0/3964564553 wait complete. 2024-09-16T15:47:42.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.931+0000 7f04ca200640 1 Processor -- start 2024-09-16T15:47:42.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.931+0000 7f04ca200640 1 -- start start 2024-09-16T15:47:42.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.933+0000 7f04ca200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04c40b45d0 0x7f04c41f9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:42.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.933+0000 7f04ca200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f04c40b4bd0 0x7f04c40ad0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:42.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.933+0000 7f04ca200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f04c41437c0 0x7f04c40afe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:42.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.933+0000 7f04ca200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f04c406ef20 con 0x7f04c40b4bd0 2024-09-16T15:47:42.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.933+0000 7f04ca200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f04c406eda0 con 0x7f04c40b45d0 2024-09-16T15:47:42.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.933+0000 7f04ca200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f04c406f0a0 con 0x7f04c41437c0 2024-09-16T15:47:42.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.933+0000 7f04c2a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f04c40b4bd0 0x7f04c40ad0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:42.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.933+0000 7f04c2a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f04c40b4bd0 0x7f04c40ad0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:42876/0 (socket says 172.21.15.151:42876) 2024-09-16T15:47:42.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.933+0000 7f04c2a00640 1 -- 172.21.15.151:0/2980128157 learned_addr learned my addr 172.21.15.151:0/2980128157 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:47:42.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.934+0000 7f04c3400640 1 --2- 172.21.15.151:0/2980128157 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04c40b45d0 0x7f04c41f9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:42.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.934+0000 7f04c3e00640 1 --2- 172.21.15.151:0/2980128157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f04c41437c0 0x7f04c40afe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:42.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.935+0000 7f04c2a00640 1 -- 172.21.15.151:0/2980128157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f04c41437c0 msgr2=0x7f04c40afe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:42.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.935+0000 7f04c2a00640 1 --2- 172.21.15.151:0/2980128157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f04c41437c0 0x7f04c40afe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:42.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.935+0000 7f04c2a00640 1 -- 172.21.15.151:0/2980128157 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04c40b45d0 msgr2=0x7f04c41f9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:42.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.935+0000 7f04c2a00640 1 --2- 172.21.15.151:0/2980128157 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04c40b45d0 0x7f04c41f9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:42.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.935+0000 7f04c2a00640 1 -- 172.21.15.151:0/2980128157 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04c41f9c80 con 0x7f04c40b4bd0 2024-09-16T15:47:42.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.935+0000 7f04c3400640 1 --2- 172.21.15.151:0/2980128157 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04c40b45d0 0x7f04c41f9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:47:42.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.935+0000 7f04c2a00640 1 --2- 172.21.15.151:0/2980128157 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f04c40b4bd0 0x7f04c40ad0d0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f04b000c480 tx=0x7f04b000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:42.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.936+0000 7f04bb400640 1 -- 172.21.15.151:0/2980128157 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b00052d0 con 0x7f04c40b4bd0 2024-09-16T15:47:42.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.936+0000 7f04bb400640 1 -- 172.21.15.151:0/2980128157 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f04b0005470 con 0x7f04c40b4bd0 2024-09-16T15:47:42.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.936+0000 7f04bb400640 1 -- 172.21.15.151:0/2980128157 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b0012870 con 0x7f04c40b4bd0 2024-09-16T15:47:42.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.936+0000 7f04ca200640 1 -- 172.21.15.151:0/2980128157 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04c4155d20 con 0x7f04c40b4bd0 2024-09-16T15:47:42.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.936+0000 7f04ca200640 1 -- 172.21.15.151:0/2980128157 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04c40ac8d0 con 0x7f04c40b4bd0 2024-09-16T15:47:42.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.939+0000 7f04ca200640 1 -- 172.21.15.151:0/2980128157 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04c406e860 con 0x7f04c40b4bd0 2024-09-16T15:47:42.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.940+0000 7f04bb400640 1 -- 172.21.15.151:0/2980128157 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 28) ==== 102766+0+0 (secure 0 0 0) 0x7f04b00035f0 con 0x7f04c40b4bd0 2024-09-16T15:47:42.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.941+0000 7f04bb400640 1 --2- 172.21.15.151:0/2980128157 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f04a40798a0 0x7f04a407bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:42.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.942+0000 7f04bb400640 1 -- 172.21.15.151:0/2980128157 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5881+0+0 (secure 0 0 0) 0x7f04b0099250 con 0x7f04c40b4bd0 2024-09-16T15:47:42.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.951+0000 7f04bb400640 1 -- 172.21.15.151:0/2980128157 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f04b00629a0 con 0x7f04c40b4bd0 2024-09-16T15:47:42.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.983+0000 7f04c3400640 1 --2- 172.21.15.151:0/2980128157 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f04a40798a0 0x7f04a407bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:43.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:42.998+0000 7f04c3400640 1 --2- 172.21.15.151:0/2980128157 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f04a40798a0 0x7f04a407bd60 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f04c4074620 tx=0x7f04bc00a540 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:43.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:43.299+0000 7f04ca200640 1 -- 172.21.15.151:0/2980128157 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone74020", "target": ["mon-mgr", ""]}) -- 0x7f04c40cd570 con 0x7f04a40798a0 2024-09-16T15:47:43.527 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:47:43.526+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:43.527 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:47:43.526+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:43.527 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:47:43.526+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:43.527 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:47:43.526+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:43.527 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:47:43.526+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:44.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:44.533+0000 7f04bb400640 1 -- 172.21.15.151:0/2980128157 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mgrmap(e 29) ==== 102817+0+0 (secure 0 0 0) 0x7f04b005ec10 con 0x7f04c40b4bd0 2024-09-16T15:47:45.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:45.496+0000 7f04bb400640 1 -- 172.21.15.151:0/2980128157 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f04c40cd570 con 0x7f04a40798a0 2024-09-16T15:47:45.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:45.501+0000 7f04ca200640 1 -- 172.21.15.151:0/2980128157 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f04a40798a0 msgr2=0x7f04a407bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:45.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:45.501+0000 7f04ca200640 1 --2- 172.21.15.151:0/2980128157 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f04a40798a0 0x7f04a407bd60 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f04c4074620 tx=0x7f04bc00a540 comp rx=0 tx=0).stop 2024-09-16T15:47:45.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:45.501+0000 7f04ca200640 1 -- 172.21.15.151:0/2980128157 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f04c40b4bd0 msgr2=0x7f04c40ad0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:45.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:45.501+0000 7f04ca200640 1 --2- 172.21.15.151:0/2980128157 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f04c40b4bd0 0x7f04c40ad0d0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f04b000c480 tx=0x7f04b000c940 comp rx=0 tx=0).stop 2024-09-16T15:47:45.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:45.501+0000 7f04ca200640 1 -- 172.21.15.151:0/2980128157 shutdown_connections 2024-09-16T15:47:45.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:45.501+0000 7f04ca200640 1 --2- 172.21.15.151:0/2980128157 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f04a40798a0 0x7f04a407bd60 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:45.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:45.501+0000 7f04ca200640 1 --2- 172.21.15.151:0/2980128157 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f04c41437c0 0x7f04c40afe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:45.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:45.501+0000 7f04ca200640 1 --2- 172.21.15.151:0/2980128157 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f04c40b4bd0 0x7f04c40ad0d0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:45.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:45.501+0000 7f04ca200640 1 --2- 172.21.15.151:0/2980128157 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04c40b45d0 0x7f04c41f9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:45.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:45.501+0000 7f04ca200640 1 -- 172.21.15.151:0/2980128157 >> 172.21.15.151:0/2980128157 conn(0x7f04c40b1840 msgr2=0x7f04c415b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:45.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:45.502+0000 7f04ca200640 1 -- 172.21.15.151:0/2980128157 shutdown_connections 2024-09-16T15:47:45.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:45.502+0000 7f04ca200640 1 -- 172.21.15.151:0/2980128157 wait complete. 2024-09-16T15:47:45.555 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone59661 2024-09-16T15:47:45.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:45.860+0000 7f99f2a00640 1 -- 172.21.15.151:0/496960857 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99ec0cccd0 msgr2=0x7f99ec0cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:45.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:45.860+0000 7f99f2a00640 1 --2- 172.21.15.151:0/496960857 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99ec0cccd0 0x7f99ec0cdbb0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f99ec0d0cd0 tx=0x7f99e402a410 comp rx=0 tx=0).stop 2024-09-16T15:47:45.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:45.861+0000 7f99f2a00640 1 -- 172.21.15.151:0/496960857 shutdown_connections 2024-09-16T15:47:45.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:45.861+0000 7f99f2a00640 1 --2- 172.21.15.151:0/496960857 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99ec0cccd0 0x7f99ec0cdbb0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:45.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:45.861+0000 7f99f2a00640 1 --2- 172.21.15.151:0/496960857 >> v2:172.21.15.167:3300/0 conn(0x7f99ec145750 0x7f99ec073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:45.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:45.861+0000 7f99f2a00640 1 -- 172.21.15.151:0/496960857 >> 172.21.15.151:0/496960857 conn(0x7f99ec0b1840 msgr2=0x7f99ec0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:45.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:45.861+0000 7f99f2a00640 1 -- 172.21.15.151:0/496960857 shutdown_connections 2024-09-16T15:47:45.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:45.861+0000 7f99f2a00640 1 -- 172.21.15.151:0/496960857 wait complete. 2024-09-16T15:47:45.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:45.863+0000 7f99f2a00640 1 Processor -- start 2024-09-16T15:47:45.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:45.863+0000 7f99f2a00640 1 -- start start 2024-09-16T15:47:45.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:45.864+0000 7f99f2a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99ec06efa0 0x7f99ec1fd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:45.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:45.864+0000 7f99f2a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99ec0cccd0 0x7f99ec15e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:45.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:45.864+0000 7f99f2a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99ec145750 0x7f99ec0b4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:45.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:45.864+0000 7f99f2a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f99ec0ce480 con 0x7f99ec06efa0 2024-09-16T15:47:45.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:45.864+0000 7f99f2a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f99ec0ce300 con 0x7f99ec145750 2024-09-16T15:47:45.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:45.864+0000 7f99f2a00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f99ec0ce600 con 0x7f99ec0cccd0 2024-09-16T15:47:45.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:45.865+0000 7f99ebe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99ec06efa0 0x7f99ec1fd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:45.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:45.865+0000 7f99ebe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99ec06efa0 0x7f99ec1fd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:42900/0 (socket says 172.21.15.151:42900) 2024-09-16T15:47:45.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:45.865+0000 7f99ebe00640 1 -- 172.21.15.151:0/440210250 learned_addr learned my addr 172.21.15.151:0/440210250 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:47:45.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:45.865+0000 7f99f0a00640 1 --2- 172.21.15.151:0/440210250 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99ec145750 0x7f99ec0b4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:45.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:45.866+0000 7f99ebe00640 1 -- 172.21.15.151:0/440210250 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99ec0cccd0 msgr2=0x7f99ec15e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:45.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:45.866+0000 7f99ebe00640 1 --2- 172.21.15.151:0/440210250 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99ec0cccd0 0x7f99ec15e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:45.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:45.866+0000 7f99ebe00640 1 -- 172.21.15.151:0/440210250 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99ec145750 msgr2=0x7f99ec0b4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:45.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:45.866+0000 7f99ebe00640 1 --2- 172.21.15.151:0/440210250 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99ec145750 0x7f99ec0b4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:45.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:45.866+0000 7f99ebe00640 1 -- 172.21.15.151:0/440210250 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99ec15edb0 con 0x7f99ec06efa0 2024-09-16T15:47:45.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:45.867+0000 7f99ebe00640 1 --2- 172.21.15.151:0/440210250 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99ec06efa0 0x7f99ec1fd3f0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f99dc00ea20 tx=0x7f99dc00eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:45.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:45.867+0000 7f99e3e00640 1 -- 172.21.15.151:0/440210250 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99dc00cde0 con 0x7f99ec06efa0 2024-09-16T15:47:45.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:45.867+0000 7f99e3e00640 1 -- 172.21.15.151:0/440210250 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f99dc011560 con 0x7f99ec06efa0 2024-09-16T15:47:45.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:45.867+0000 7f99e3e00640 1 -- 172.21.15.151:0/440210250 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99dc011990 con 0x7f99ec06efa0 2024-09-16T15:47:45.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:45.867+0000 7f99f2a00640 1 -- 172.21.15.151:0/440210250 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99ec0aacd0 con 0x7f99ec06efa0 2024-09-16T15:47:45.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:45.871+0000 7f99f2a00640 1 -- 172.21.15.151:0/440210250 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99ec1fcbf0 con 0x7f99ec06efa0 2024-09-16T15:47:45.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:45.873+0000 7f99e3e00640 1 -- 172.21.15.151:0/440210250 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 29) ==== 102817+0+0 (secure 0 0 0) 0x7f99dc011b30 con 0x7f99ec06efa0 2024-09-16T15:47:45.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:45.874+0000 7f99f2a00640 1 -- 172.21.15.151:0/440210250 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99ec0cdbb0 con 0x7f99ec06efa0 2024-09-16T15:47:45.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:45.875+0000 7f99e3e00640 1 --2- 172.21.15.151:0/440210250 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f99cc079930 0x7f99cc07bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:45.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:45.876+0000 7f99e3e00640 1 -- 172.21.15.151:0/440210250 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5881+0+0 (secure 0 0 0) 0x7f99dc099a20 con 0x7f99ec06efa0 2024-09-16T15:47:45.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:45.882+0000 7f99e3e00640 1 -- 172.21.15.151:0/440210250 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f99dc09de20 con 0x7f99ec06efa0 2024-09-16T15:47:45.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:45.883+0000 7f99eb400640 1 --2- 172.21.15.151:0/440210250 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f99cc079930 0x7f99cc07bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:45.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:45.905+0000 7f99eb400640 1 --2- 172.21.15.151:0/440210250 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f99cc079930 0x7f99cc07bdf0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f99ec0cfd20 tx=0x7f99d8007a90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:46.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:46.328+0000 7f99f2a00640 1 -- 172.21.15.151:0/440210250 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone59661", "target": ["mon-mgr", ""]}) -- 0x7f99ec0cfa70 con 0x7f99cc079930 2024-09-16T15:47:46.526 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:47:46.524+0000 7f48efe00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-16T15:47:46.530 INFO:teuthology.orchestra.run.smithi151.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-16T15:47:46.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:46.529+0000 7f99e3e00640 1 -- 172.21.15.151:0/440210250 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f99ec0cfa70 con 0x7f99cc079930 2024-09-16T15:47:46.553 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T15:47:46.554 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone92092 2024-09-16T15:47:46.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:46.824+0000 7f9d43c00640 1 -- 172.21.15.151:0/48244277 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d3c0af190 msgr2=0x7f9d3c0b2b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:46.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:46.824+0000 7f9d43c00640 1 --2- 172.21.15.151:0/48244277 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d3c0af190 0x7f9d3c0b2b40 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c0b2470 tx=0x7f9d2802a4f0 comp rx=0 tx=0).stop 2024-09-16T15:47:46.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:46.824+0000 7f9d43c00640 1 -- 172.21.15.151:0/48244277 shutdown_connections 2024-09-16T15:47:46.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:46.824+0000 7f9d43c00640 1 --2- 172.21.15.151:0/48244277 >> v2:172.21.15.167:3300/0 conn(0x7f9d3c0cec60 0x7f9d3c0cf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:46.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:46.824+0000 7f9d43c00640 1 --2- 172.21.15.151:0/48244277 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d3c0af190 0x7f9d3c0b2b40 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:46.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:46.824+0000 7f9d43c00640 1 --2- 172.21.15.151:0/48244277 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9d3c161760 0x7f9d3c0ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:46.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:46.825+0000 7f9d43c00640 1 -- 172.21.15.151:0/48244277 >> 172.21.15.151:0/48244277 conn(0x7f9d3c0b1840 msgr2=0x7f9d3c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:46.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:46.825+0000 7f9d43c00640 1 -- 172.21.15.151:0/48244277 shutdown_connections 2024-09-16T15:47:46.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:46.825+0000 7f9d43c00640 1 -- 172.21.15.151:0/48244277 wait complete. 2024-09-16T15:47:46.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:46.827+0000 7f9d43c00640 1 Processor -- start 2024-09-16T15:47:46.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:46.828+0000 7f9d43c00640 1 -- start start 2024-09-16T15:47:46.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:46.830+0000 7f9d43c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d3c0af190 0x7f9d3c1f8f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:46.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:46.830+0000 7f9d43c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d3c0cec60 0x7f9d3c158250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:46.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:46.830+0000 7f9d43c00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9d3c161760 0x7f9d3c157500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:46.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:46.830+0000 7f9d43c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9d3c0b3200 con 0x7f9d3c0af190 2024-09-16T15:47:46.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:46.830+0000 7f9d43c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9d3c0b3080 con 0x7f9d3c0cec60 2024-09-16T15:47:46.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:46.830+0000 7f9d43c00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f9d3c162190 con 0x7f9d3c161760 2024-09-16T15:47:46.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:46.831+0000 7f9d41200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d3c0af190 0x7f9d3c1f8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:46.833 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:46.831+0000 7f9d41200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d3c0af190 0x7f9d3c1f8f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:42930/0 (socket says 172.21.15.151:42930) 2024-09-16T15:47:46.833 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:46.831+0000 7f9d41200640 1 -- 172.21.15.151:0/1955494770 learned_addr learned my addr 172.21.15.151:0/1955494770 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:47:46.833 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:46.831+0000 7f9d3be00640 1 --2- 172.21.15.151:0/1955494770 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d3c0cec60 0x7f9d3c158250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:46.833 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:46.831+0000 7f9d41c00640 1 --2- 172.21.15.151:0/1955494770 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9d3c161760 0x7f9d3c157500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:46.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:46.831+0000 7f9d41200640 1 -- 172.21.15.151:0/1955494770 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9d3c161760 msgr2=0x7f9d3c157500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:46.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:46.832+0000 7f9d41200640 1 --2- 172.21.15.151:0/1955494770 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9d3c161760 0x7f9d3c157500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:46.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:46.832+0000 7f9d41200640 1 -- 172.21.15.151:0/1955494770 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d3c0cec60 msgr2=0x7f9d3c158250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:46.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:46.832+0000 7f9d41200640 1 --2- 172.21.15.151:0/1955494770 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d3c0cec60 0x7f9d3c158250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:46.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:46.832+0000 7f9d41200640 1 -- 172.21.15.151:0/1955494770 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d3c15a0d0 con 0x7f9d3c0af190 2024-09-16T15:47:46.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:46.832+0000 7f9d41200640 1 --2- 172.21.15.151:0/1955494770 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d3c0af190 0x7f9d3c1f8f60 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f9d3000b3e0 tx=0x7f9d3000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:46.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:46.832+0000 7f9d38c00640 1 -- 172.21.15.151:0/1955494770 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d30015020 con 0x7f9d3c0af190 2024-09-16T15:47:46.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:46.832+0000 7f9d38c00640 1 -- 172.21.15.151:0/1955494770 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9d30011940 con 0x7f9d3c0af190 2024-09-16T15:47:46.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:46.833+0000 7f9d38c00640 1 -- 172.21.15.151:0/1955494770 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d30011d90 con 0x7f9d3c0af190 2024-09-16T15:47:46.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:46.833+0000 7f9d43c00640 1 -- 172.21.15.151:0/1955494770 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d3c15a3b0 con 0x7f9d3c0af190 2024-09-16T15:47:46.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:46.833+0000 7f9d43c00640 1 -- 172.21.15.151:0/1955494770 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d3c15a970 con 0x7f9d3c0af190 2024-09-16T15:47:46.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:46.835+0000 7f9d43c00640 1 -- 172.21.15.151:0/1955494770 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d3c0cf090 con 0x7f9d3c0af190 2024-09-16T15:47:46.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:46.836+0000 7f9d38c00640 1 -- 172.21.15.151:0/1955494770 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 29) ==== 102817+0+0 (secure 0 0 0) 0x7f9d30018020 con 0x7f9d3c0af190 2024-09-16T15:47:46.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:46.838+0000 7f9d38c00640 1 --2- 172.21.15.151:0/1955494770 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9d1c079930 0x7f9d1c07bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:46.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:46.838+0000 7f9d38c00640 1 -- 172.21.15.151:0/1955494770 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5881+0+0 (secure 0 0 0) 0x7f9d300991f0 con 0x7f9d3c0af190 2024-09-16T15:47:46.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:46.844+0000 7f9d3be00640 1 --2- 172.21.15.151:0/1955494770 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9d1c079930 0x7f9d1c07bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:46.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:46.844+0000 7f9d38c00640 1 -- 172.21.15.151:0/1955494770 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9d3001f030 con 0x7f9d3c0af190 2024-09-16T15:47:46.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:46.856+0000 7f9d3be00640 1 --2- 172.21.15.151:0/1955494770 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9d1c079930 0x7f9d1c07bdf0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9d28033000 tx=0x7f9d28004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:47.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:47.266+0000 7f9d43c00640 1 -- 172.21.15.151:0/1955494770 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone92092", "target": ["mon-mgr", ""]}) -- 0x7f9d3c0ae3f0 con 0x7f9d1c079930 2024-09-16T15:47:48.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:48.097+0000 7f9d38c00640 1 -- 172.21.15.151:0/1955494770 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9d3c0ae3f0 con 0x7f9d1c079930 2024-09-16T15:47:48.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:48.101+0000 7f9d43c00640 1 -- 172.21.15.151:0/1955494770 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9d1c079930 msgr2=0x7f9d1c07bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:48.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:48.101+0000 7f9d43c00640 1 --2- 172.21.15.151:0/1955494770 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9d1c079930 0x7f9d1c07bdf0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9d28033000 tx=0x7f9d28004730 comp rx=0 tx=0).stop 2024-09-16T15:47:48.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:48.101+0000 7f9d43c00640 1 -- 172.21.15.151:0/1955494770 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d3c0af190 msgr2=0x7f9d3c1f8f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:48.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:48.101+0000 7f9d43c00640 1 --2- 172.21.15.151:0/1955494770 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d3c0af190 0x7f9d3c1f8f60 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f9d3000b3e0 tx=0x7f9d3000b8a0 comp rx=0 tx=0).stop 2024-09-16T15:47:48.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:48.101+0000 7f9d43c00640 1 -- 172.21.15.151:0/1955494770 shutdown_connections 2024-09-16T15:47:48.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:48.101+0000 7f9d43c00640 1 --2- 172.21.15.151:0/1955494770 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9d1c079930 0x7f9d1c07bdf0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:48.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:48.101+0000 7f9d43c00640 1 --2- 172.21.15.151:0/1955494770 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d3c0cec60 0x7f9d3c158250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:48.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:48.101+0000 7f9d43c00640 1 --2- 172.21.15.151:0/1955494770 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9d3c161760 0x7f9d3c157500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:48.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:48.101+0000 7f9d43c00640 1 --2- 172.21.15.151:0/1955494770 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d3c0af190 0x7f9d3c1f8f60 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:48.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:48.101+0000 7f9d43c00640 1 -- 172.21.15.151:0/1955494770 >> 172.21.15.151:0/1955494770 conn(0x7f9d3c0b1840 msgr2=0x7f9d3c15e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:48.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:48.102+0000 7f9d43c00640 1 -- 172.21.15.151:0/1955494770 shutdown_connections 2024-09-16T15:47:48.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:48.102+0000 7f9d43c00640 1 -- 172.21.15.151:0/1955494770 wait complete. 2024-09-16T15:47:48.124 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone1123 2024-09-16T15:47:48.471 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:48.470+0000 7f9d04c00640 1 -- 172.21.15.151:0/134140095 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d000b4bd0 msgr2=0x7f9d000b3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:48.471 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:48.470+0000 7f9d04c00640 1 --2- 172.21.15.151:0/134140095 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d000b4bd0 0x7f9d000b3ec0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f9cec00a940 tx=0x7f9cec02b8f0 comp rx=0 tx=0).stop 2024-09-16T15:47:48.471 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:48.470+0000 7f9d04c00640 1 -- 172.21.15.151:0/134140095 shutdown_connections 2024-09-16T15:47:48.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:48.470+0000 7f9d04c00640 1 --2- 172.21.15.151:0/134140095 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d000b4bd0 0x7f9d000b3ec0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:48.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:48.470+0000 7f9d04c00640 1 -- 172.21.15.151:0/134140095 >> 172.21.15.151:0/134140095 conn(0x7f9d000b1840 msgr2=0x7f9d000b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:48.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:48.470+0000 7f9d04c00640 1 -- 172.21.15.151:0/134140095 shutdown_connections 2024-09-16T15:47:48.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:48.470+0000 7f9d04c00640 1 -- 172.21.15.151:0/134140095 wait complete. 2024-09-16T15:47:48.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:48.472+0000 7f9d04c00640 1 Processor -- start 2024-09-16T15:47:48.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:48.473+0000 7f9d04c00640 1 -- start start 2024-09-16T15:47:48.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:48.474+0000 7f9d04c00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9d000b4400 0x7f9d001f3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:48.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:48.474+0000 7f9d04c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d000b4bd0 0x7f9d0013d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:48.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:48.474+0000 7f9d04c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d001437c0 0x7f9d001f3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:48.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:48.474+0000 7f9d04c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9d0006f0f0 con 0x7f9d000b4bd0 2024-09-16T15:47:48.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:48.474+0000 7f9d04c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9d0006ef70 con 0x7f9d001437c0 2024-09-16T15:47:48.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:48.474+0000 7f9d04c00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f9d0006f270 con 0x7f9d000b4400 2024-09-16T15:47:48.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:48.475+0000 7f9cfd400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d000b4bd0 0x7f9d0013d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:48.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:48.475+0000 7f9cfd400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d000b4bd0 0x7f9d0013d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:42946/0 (socket says 172.21.15.151:42946) 2024-09-16T15:47:48.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:48.475+0000 7f9cfd400640 1 -- 172.21.15.151:0/2869689440 learned_addr learned my addr 172.21.15.151:0/2869689440 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:47:48.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:48.476+0000 7f9cfd400640 1 -- 172.21.15.151:0/2869689440 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9d000b4400 msgr2=0x7f9d001f3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:48.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:48.476+0000 7f9cfd400640 1 --2- 172.21.15.151:0/2869689440 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9d000b4400 0x7f9d001f3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:48.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:48.477+0000 7f9cfe800640 1 --2- 172.21.15.151:0/2869689440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d001437c0 0x7f9d001f3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:48.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:48.477+0000 7f9cfd400640 1 -- 172.21.15.151:0/2869689440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d001437c0 msgr2=0x7f9d001f3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:48.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:48.477+0000 7f9cfd400640 1 --2- 172.21.15.151:0/2869689440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d001437c0 0x7f9d001f3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:48.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:48.477+0000 7f9cfd400640 1 -- 172.21.15.151:0/2869689440 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d000a9460 con 0x7f9d000b4bd0 2024-09-16T15:47:48.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:48.477+0000 7f9cfe800640 1 --2- 172.21.15.151:0/2869689440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d001437c0 0x7f9d001f3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:47:48.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:48.477+0000 7f9cfd400640 1 --2- 172.21.15.151:0/2869689440 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d000b4bd0 0x7f9d0013d4e0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f9cec000c00 tx=0x7f9cec0043d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:48.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:48.478+0000 7f9cf6000640 1 -- 172.21.15.151:0/2869689440 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cec0037f0 con 0x7f9d000b4bd0 2024-09-16T15:47:48.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:48.478+0000 7f9cf6000640 1 -- 172.21.15.151:0/2869689440 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9cec02e070 con 0x7f9d000b4bd0 2024-09-16T15:47:48.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:48.478+0000 7f9cf6000640 1 -- 172.21.15.151:0/2869689440 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cec003c50 con 0x7f9d000b4bd0 2024-09-16T15:47:48.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:48.478+0000 7f9d04c00640 1 -- 172.21.15.151:0/2869689440 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d001f3680 con 0x7f9d000b4bd0 2024-09-16T15:47:48.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:48.479+0000 7f9d04c00640 1 -- 172.21.15.151:0/2869689440 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d000af2c0 con 0x7f9d000b4bd0 2024-09-16T15:47:48.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:48.482+0000 7f9d04c00640 1 -- 172.21.15.151:0/2869689440 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d0006f800 con 0x7f9d000b4bd0 2024-09-16T15:47:48.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:48.482+0000 7f9cf6000640 1 -- 172.21.15.151:0/2869689440 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 29) ==== 102817+0+0 (secure 0 0 0) 0x7f9cec032480 con 0x7f9d000b4bd0 2024-09-16T15:47:48.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:48.484+0000 7f9cf6000640 1 --2- 172.21.15.151:0/2869689440 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9ce0079980 0x7f9ce007be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:48.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:48.484+0000 7f9cf6000640 1 -- 172.21.15.151:0/2869689440 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5881+0+0 (secure 0 0 0) 0x7f9cec0badb0 con 0x7f9d000b4bd0 2024-09-16T15:47:48.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:48.491+0000 7f9cfde00640 1 --2- 172.21.15.151:0/2869689440 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9ce0079980 0x7f9ce007be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:48.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:48.493+0000 7f9cfde00640 1 --2- 172.21.15.151:0/2869689440 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9ce0079980 0x7f9ce007be40 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0000f60 tx=0x7f9cf00079b0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:48.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:48.496+0000 7f9cf6000640 1 -- 172.21.15.151:0/2869689440 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9cec084f30 con 0x7f9d000b4bd0 2024-09-16T15:47:48.729 DEBUG:teuthology.orchestra.run.smithi151:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:47:48.735 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:47:48.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:48.920+0000 7f9d04c00640 1 -- 172.21.15.151:0/2869689440 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone1123", "target": ["mon-mgr", ""]}) -- 0x7f9d000cd570 con 0x7f9ce0079980 2024-09-16T15:47:50.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:50.573+0000 7f9cf6000640 1 -- 172.21.15.151:0/2869689440 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9d000cd570 con 0x7f9ce0079980 2024-09-16T15:47:50.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:50.577+0000 7f9d04c00640 1 -- 172.21.15.151:0/2869689440 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9ce0079980 msgr2=0x7f9ce007be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:50.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:50.577+0000 7f9d04c00640 1 --2- 172.21.15.151:0/2869689440 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9ce0079980 0x7f9ce007be40 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0000f60 tx=0x7f9cf00079b0 comp rx=0 tx=0).stop 2024-09-16T15:47:50.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:50.577+0000 7f9d04c00640 1 -- 172.21.15.151:0/2869689440 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d000b4bd0 msgr2=0x7f9d0013d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:50.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:50.577+0000 7f9d04c00640 1 --2- 172.21.15.151:0/2869689440 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d000b4bd0 0x7f9d0013d4e0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f9cec000c00 tx=0x7f9cec0043d0 comp rx=0 tx=0).stop 2024-09-16T15:47:50.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:50.577+0000 7f9d04c00640 1 -- 172.21.15.151:0/2869689440 shutdown_connections 2024-09-16T15:47:50.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:50.578+0000 7f9d04c00640 1 --2- 172.21.15.151:0/2869689440 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9ce0079980 0x7f9ce007be40 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:50.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:50.578+0000 7f9d04c00640 1 --2- 172.21.15.151:0/2869689440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d001437c0 0x7f9d001f3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:50.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:50.578+0000 7f9d04c00640 1 --2- 172.21.15.151:0/2869689440 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d000b4bd0 0x7f9d0013d4e0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:50.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:50.578+0000 7f9d04c00640 1 --2- 172.21.15.151:0/2869689440 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9d000b4400 0x7f9d001f3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:50.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:50.578+0000 7f9d04c00640 1 -- 172.21.15.151:0/2869689440 >> 172.21.15.151:0/2869689440 conn(0x7f9d000b1840 msgr2=0x7f9d0015b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:50.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:50.578+0000 7f9d04c00640 1 -- 172.21.15.151:0/2869689440 shutdown_connections 2024-09-16T15:47:50.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:50.578+0000 7f9d04c00640 1 -- 172.21.15.151:0/2869689440 wait complete. 2024-09-16T15:47:50.615 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone12644 2024-09-16T15:47:50.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:50.963+0000 7f4376400640 1 -- 172.21.15.151:0/1187623486 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f43701437c0 msgr2=0x7f43700cdad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:50.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:50.963+0000 7f4376400640 1 --2- 172.21.15.151:0/1187623486 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f43701437c0 0x7f43700cdad0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f43700cd540 tx=0x7f436802b560 comp rx=0 tx=0).stop 2024-09-16T15:47:50.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:50.964+0000 7f4376400640 1 -- 172.21.15.151:0/1187623486 shutdown_connections 2024-09-16T15:47:50.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:50.964+0000 7f4376400640 1 --2- 172.21.15.151:0/1187623486 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43700b4400 0x7f437015db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:50.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:50.964+0000 7f4376400640 1 --2- 172.21.15.151:0/1187623486 >> v2:172.21.15.167:3300/0 conn(0x7f43700b4bd0 0x7f43700b3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:50.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:50.964+0000 7f4376400640 1 --2- 172.21.15.151:0/1187623486 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f43701437c0 0x7f43700cdad0 secure :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f43700cd540 tx=0x7f436802b560 comp rx=0 tx=0).stop 2024-09-16T15:47:50.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:50.964+0000 7f4376400640 1 -- 172.21.15.151:0/1187623486 >> 172.21.15.151:0/1187623486 conn(0x7f43700b1840 msgr2=0x7f43700b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:50.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:50.965+0000 7f4376400640 1 -- 172.21.15.151:0/1187623486 shutdown_connections 2024-09-16T15:47:50.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:50.966+0000 7f4376400640 1 -- 172.21.15.151:0/1187623486 wait complete. 2024-09-16T15:47:50.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:50.968+0000 7f4376400640 1 Processor -- start 2024-09-16T15:47:50.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:50.968+0000 7f4376400640 1 -- start start 2024-09-16T15:47:50.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:50.969+0000 7f4376400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43700b4400 0x7f437013f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:50.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:50.969+0000 7f4376400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43700b4bd0 0x7f43700aeca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:50.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:50.969+0000 7f4376400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f43701601a0 0x7f437013cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:50.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:50.969+0000 7f4376400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f437015e090 con 0x7f43701601a0 2024-09-16T15:47:50.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:50.969+0000 7f4376400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4370070a80 con 0x7f43700b4bd0 2024-09-16T15:47:50.972 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:50.969+0000 7f4376400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f437015e210 con 0x7f43700b4400 2024-09-16T15:47:50.972 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:50.970+0000 7f436fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f43701601a0 0x7f437013cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:50.972 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:50.970+0000 7f436fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f43701601a0 0x7f437013cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:42964/0 (socket says 172.21.15.151:42964) 2024-09-16T15:47:50.972 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:50.970+0000 7f436fe00640 1 -- 172.21.15.151:0/1065541185 learned_addr learned my addr 172.21.15.151:0/1065541185 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:47:50.972 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:50.970+0000 7f436ea00640 1 --2- 172.21.15.151:0/1065541185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43700b4bd0 0x7f43700aeca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:50.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:50.971+0000 7f436f400640 1 --2- 172.21.15.151:0/1065541185 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43700b4400 0x7f437013f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:50.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:50.972+0000 7f436ea00640 1 -- 172.21.15.151:0/1065541185 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43700b4400 msgr2=0x7f437013f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:50.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:50.972+0000 7f436ea00640 1 --2- 172.21.15.151:0/1065541185 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43700b4400 0x7f437013f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:50.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:50.972+0000 7f436ea00640 1 -- 172.21.15.151:0/1065541185 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f43701601a0 msgr2=0x7f437013cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:50.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:50.972+0000 7f436ea00640 1 --2- 172.21.15.151:0/1065541185 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f43701601a0 0x7f437013cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:50.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:50.972+0000 7f436ea00640 1 -- 172.21.15.151:0/1065541185 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43700af1e0 con 0x7f43700b4bd0 2024-09-16T15:47:50.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:50.973+0000 7f436ea00640 1 --2- 172.21.15.151:0/1065541185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43700b4bd0 0x7f43700aeca0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f436000b410 tx=0x7f436000b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:50.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:50.974+0000 7f4367400640 1 -- 172.21.15.151:0/1065541185 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4360015020 con 0x7f43700b4bd0 2024-09-16T15:47:50.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:50.974+0000 7f4367400640 1 -- 172.21.15.151:0/1065541185 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4360011940 con 0x7f43700b4bd0 2024-09-16T15:47:50.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:50.974+0000 7f4367400640 1 -- 172.21.15.151:0/1065541185 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4360011d90 con 0x7f43700b4bd0 2024-09-16T15:47:50.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:50.974+0000 7f4376400640 1 -- 172.21.15.151:0/1065541185 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4370162b80 con 0x7f43700b4bd0 2024-09-16T15:47:50.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:50.974+0000 7f4376400640 1 -- 172.21.15.151:0/1065541185 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43701e9110 con 0x7f43700b4bd0 2024-09-16T15:47:50.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:50.982+0000 7f4376400640 1 -- 172.21.15.151:0/1065541185 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f437015e660 con 0x7f43700b4bd0 2024-09-16T15:47:50.996 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:50.993+0000 7f4367400640 1 -- 172.21.15.151:0/1065541185 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 29) ==== 102817+0+0 (secure 0 0 0) 0x7f4360018020 con 0x7f43700b4bd0 2024-09-16T15:47:50.996 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:50.995+0000 7f4367400640 1 --2- 172.21.15.151:0/1065541185 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4350079960 0x7f435007be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:50.997 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:50.996+0000 7f4367400640 1 -- 172.21.15.151:0/1065541185 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5881+0+0 (secure 0 0 0) 0x7f436005eb60 con 0x7f43700b4bd0 2024-09-16T15:47:51.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:51.008+0000 7f4367400640 1 -- 172.21.15.151:0/1065541185 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f436001f070 con 0x7f43700b4bd0 2024-09-16T15:47:51.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:51.010+0000 7f436f400640 1 --2- 172.21.15.151:0/1065541185 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4350079960 0x7f435007be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:51.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:51.023+0000 7f436f400640 1 --2- 172.21.15.151:0/1065541185 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4350079960 0x7f435007be20 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f4368034000 tx=0x7f4368004ab0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:51.438 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:51.437+0000 7f4376400640 1 -- 172.21.15.151:0/1065541185 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone12644", "target": ["mon-mgr", ""]}) -- 0x7f4370162eb0 con 0x7f4350079960 2024-09-16T15:47:52.315 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:52.314+0000 7f4367400640 1 -- 172.21.15.151:0/1065541185 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4370162eb0 con 0x7f4350079960 2024-09-16T15:47:52.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:52.318+0000 7f4376400640 1 -- 172.21.15.151:0/1065541185 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4350079960 msgr2=0x7f435007be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:52.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:52.318+0000 7f4376400640 1 --2- 172.21.15.151:0/1065541185 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4350079960 0x7f435007be20 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f4368034000 tx=0x7f4368004ab0 comp rx=0 tx=0).stop 2024-09-16T15:47:52.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:52.318+0000 7f4376400640 1 -- 172.21.15.151:0/1065541185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43700b4bd0 msgr2=0x7f43700aeca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:52.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:52.318+0000 7f4376400640 1 --2- 172.21.15.151:0/1065541185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43700b4bd0 0x7f43700aeca0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f436000b410 tx=0x7f436000b8d0 comp rx=0 tx=0).stop 2024-09-16T15:47:52.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:52.318+0000 7f4376400640 1 -- 172.21.15.151:0/1065541185 shutdown_connections 2024-09-16T15:47:52.320 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:52.318+0000 7f4376400640 1 --2- 172.21.15.151:0/1065541185 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4350079960 0x7f435007be20 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:52.320 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:52.318+0000 7f4376400640 1 --2- 172.21.15.151:0/1065541185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43700b4bd0 0x7f43700aeca0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:52.320 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:52.318+0000 7f4376400640 1 --2- 172.21.15.151:0/1065541185 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f43701601a0 0x7f437013cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:52.320 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:52.318+0000 7f4376400640 1 --2- 172.21.15.151:0/1065541185 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f43700b4400 0x7f437013f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:52.320 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:52.318+0000 7f4376400640 1 -- 172.21.15.151:0/1065541185 >> 172.21.15.151:0/1065541185 conn(0x7f43700b1840 msgr2=0x7f4370072ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:52.320 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:52.318+0000 7f4376400640 1 -- 172.21.15.151:0/1065541185 shutdown_connections 2024-09-16T15:47:52.320 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:52.318+0000 7f4376400640 1 -- 172.21.15.151:0/1065541185 wait complete. 2024-09-16T15:47:52.358 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone93605 2024-09-16T15:47:52.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:52.673+0000 7f8bade00640 1 -- 172.21.15.151:0/1236634706 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b8c004090 msgr2=0x7f8b8c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:52.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:52.673+0000 7f8bade00640 1 --2- 172.21.15.151:0/1236634706 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b8c004090 0x7f8b8c004800 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f8b9400bb20 tx=0x7f8b8c005830 comp rx=0 tx=0).stop 2024-09-16T15:47:52.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:52.673+0000 7f8ba7400640 1 -- 172.21.15.151:0/1236634706 reap_dead start 2024-09-16T15:47:52.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:52.673+0000 7f8bade00640 1 -- 172.21.15.151:0/1236634706 shutdown_connections 2024-09-16T15:47:52.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:52.673+0000 7f8bade00640 1 -- 172.21.15.151:0/1236634706 >> 172.21.15.151:0/1236634706 conn(0x7f8ba80cdde0 msgr2=0x7f8ba80d0d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:52.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:52.673+0000 7f8bade00640 1 -- 172.21.15.151:0/1236634706 shutdown_connections 2024-09-16T15:47:52.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:52.673+0000 7f8bade00640 1 -- 172.21.15.151:0/1236634706 wait complete. 2024-09-16T15:47:52.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:52.675+0000 7f8bade00640 1 Processor -- start 2024-09-16T15:47:52.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:52.675+0000 7f8bade00640 1 -- start start 2024-09-16T15:47:52.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:52.676+0000 7f8bade00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b8c004090 0x7f8ba80c8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:52.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:52.676+0000 7f8bade00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ba814b1e0 0x7f8ba80b46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:52.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:52.676+0000 7f8bade00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8ba80b4bf0 0x7f8ba80aea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:52.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:52.676+0000 7f8bade00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba8160250 con 0x7f8b8c004090 2024-09-16T15:47:52.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:52.676+0000 7f8bade00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba81600d0 con 0x7f8ba814b1e0 2024-09-16T15:47:52.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:52.676+0000 7f8bade00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f8ba81603d0 con 0x7f8ba80b4bf0 2024-09-16T15:47:52.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:52.676+0000 7f8ba6a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b8c004090 0x7f8ba80c8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:52.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:52.677+0000 7f8ba6a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b8c004090 0x7f8ba80c8e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:42976/0 (socket says 172.21.15.151:42976) 2024-09-16T15:47:52.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:52.677+0000 7f8ba6a00640 1 -- 172.21.15.151:0/2524265969 learned_addr learned my addr 172.21.15.151:0/2524265969 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:47:52.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:52.677+0000 7f8ba6000640 1 --2- 172.21.15.151:0/2524265969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ba814b1e0 0x7f8ba80b46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:52.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:52.677+0000 7f8ba7400640 1 --2- 172.21.15.151:0/2524265969 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8ba80b4bf0 0x7f8ba80aea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:52.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:52.677+0000 7f8ba6a00640 1 -- 172.21.15.151:0/2524265969 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8ba80b4bf0 msgr2=0x7f8ba80aea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:52.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:52.677+0000 7f8ba6a00640 1 --2- 172.21.15.151:0/2524265969 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8ba80b4bf0 0x7f8ba80aea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:52.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:52.677+0000 7f8ba6a00640 1 -- 172.21.15.151:0/2524265969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ba814b1e0 msgr2=0x7f8ba80b46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:52.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:52.677+0000 7f8ba6a00640 1 --2- 172.21.15.151:0/2524265969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ba814b1e0 0x7f8ba80b46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:52.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:52.677+0000 7f8ba6a00640 1 -- 172.21.15.151:0/2524265969 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ba814b660 con 0x7f8b8c004090 2024-09-16T15:47:52.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:52.678+0000 7f8ba6a00640 1 --2- 172.21.15.151:0/2524265969 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b8c004090 0x7f8ba80c8e50 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f8b9800c790 tx=0x7f8b9800cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:52.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:52.678+0000 7f8b9f400640 1 -- 172.21.15.151:0/2524265969 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b98017070 con 0x7f8b8c004090 2024-09-16T15:47:52.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:52.678+0000 7f8b9f400640 1 -- 172.21.15.151:0/2524265969 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8b98005360 con 0x7f8b8c004090 2024-09-16T15:47:52.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:52.678+0000 7f8bade00640 1 -- 172.21.15.151:0/2524265969 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ba80b0bc0 con 0x7f8b8c004090 2024-09-16T15:47:52.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:52.678+0000 7f8bade00640 1 -- 172.21.15.151:0/2524265969 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ba80b1180 con 0x7f8b8c004090 2024-09-16T15:47:52.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:52.678+0000 7f8b9f400640 1 -- 172.21.15.151:0/2524265969 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b98003850 con 0x7f8b8c004090 2024-09-16T15:47:52.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:52.680+0000 7f8bade00640 1 -- 172.21.15.151:0/2524265969 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ba806f540 con 0x7f8b8c004090 2024-09-16T15:47:52.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:52.681+0000 7f8b9f400640 1 -- 172.21.15.151:0/2524265969 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 29) ==== 102817+0+0 (secure 0 0 0) 0x7f8b9801a070 con 0x7f8b8c004090 2024-09-16T15:47:52.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:52.682+0000 7f8b9f400640 1 --2- 172.21.15.151:0/2524265969 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8b88079960 0x7f8b8807be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:52.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:52.682+0000 7f8b9f400640 1 -- 172.21.15.151:0/2524265969 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5881+0+0 (secure 0 0 0) 0x7f8b98098bf0 con 0x7f8b8c004090 2024-09-16T15:47:52.685 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:52.684+0000 7f8ba6000640 1 --2- 172.21.15.151:0/2524265969 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8b88079960 0x7f8b8807be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:52.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:52.685+0000 7f8ba6000640 1 --2- 172.21.15.151:0/2524265969 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8b88079960 0x7f8b8807be20 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f8b94036000 tx=0x7f8b940046c0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:52.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:52.687+0000 7f8b9f400640 1 -- 172.21.15.151:0/2524265969 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8b980629d0 con 0x7f8b8c004090 2024-09-16T15:47:53.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:53.106+0000 7f8bade00640 1 -- 172.21.15.151:0/2524265969 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone93605", "target": ["mon-mgr", ""]}) -- 0x7f8ba80b4fd0 con 0x7f8b88079960 2024-09-16T15:47:53.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:53.571+0000 7f8b9f400640 1 -- 172.21.15.151:0/2524265969 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8ba80b4fd0 con 0x7f8b88079960 2024-09-16T15:47:53.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:53.577+0000 7f8bade00640 1 -- 172.21.15.151:0/2524265969 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8b88079960 msgr2=0x7f8b8807be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:53.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:53.577+0000 7f8bade00640 1 --2- 172.21.15.151:0/2524265969 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8b88079960 0x7f8b8807be20 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f8b94036000 tx=0x7f8b940046c0 comp rx=0 tx=0).stop 2024-09-16T15:47:53.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:53.577+0000 7f8bade00640 1 -- 172.21.15.151:0/2524265969 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b8c004090 msgr2=0x7f8ba80c8e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:53.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:53.577+0000 7f8bade00640 1 --2- 172.21.15.151:0/2524265969 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b8c004090 0x7f8ba80c8e50 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f8b9800c790 tx=0x7f8b9800cc50 comp rx=0 tx=0).stop 2024-09-16T15:47:53.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:53.577+0000 7f8bade00640 1 -- 172.21.15.151:0/2524265969 shutdown_connections 2024-09-16T15:47:53.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:53.577+0000 7f8bade00640 1 --2- 172.21.15.151:0/2524265969 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8b88079960 0x7f8b8807be20 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:53.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:53.577+0000 7f8bade00640 1 --2- 172.21.15.151:0/2524265969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ba814b1e0 0x7f8ba80b46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:53.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:53.577+0000 7f8bade00640 1 --2- 172.21.15.151:0/2524265969 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8ba80b4bf0 0x7f8ba80aea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:53.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:53.577+0000 7f8bade00640 1 --2- 172.21.15.151:0/2524265969 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b8c004090 0x7f8ba80c8e50 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:53.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:53.577+0000 7f8bade00640 1 -- 172.21.15.151:0/2524265969 >> 172.21.15.151:0/2524265969 conn(0x7f8ba80cdde0 msgr2=0x7f8ba80cfb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:53.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:53.580+0000 7f8bade00640 1 -- 172.21.15.151:0/2524265969 shutdown_connections 2024-09-16T15:47:53.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:53.580+0000 7f8bade00640 1 -- 172.21.15.151:0/2524265969 wait complete. 2024-09-16T15:47:53.616 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone72556 2024-09-16T15:47:53.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:53.863+0000 7f8606800640 1 -- 172.21.15.151:0/3687052220 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f860006eda0 msgr2=0x7f86000b1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:53.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:53.863+0000 7f8606800640 1 --2- 172.21.15.151:0/3687052220 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f860006eda0 0x7f86000b1b50 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f85ec009830 tx=0x7f85ec02a520 comp rx=0 tx=0).stop 2024-09-16T15:47:53.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:53.863+0000 7f8606800640 1 -- 172.21.15.151:0/3687052220 shutdown_connections 2024-09-16T15:47:53.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:53.863+0000 7f8606800640 1 --2- 172.21.15.151:0/3687052220 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f860006eda0 0x7f86000b1b50 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:53.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:53.863+0000 7f8606800640 1 --2- 172.21.15.151:0/3687052220 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f860015e2c0 0x7f86000b1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:53.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:53.863+0000 7f8606800640 1 -- 172.21.15.151:0/3687052220 >> 172.21.15.151:0/3687052220 conn(0x7f86000d1dd0 msgr2=0x7f86000d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:53.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:53.863+0000 7f8606800640 1 -- 172.21.15.151:0/3687052220 shutdown_connections 2024-09-16T15:47:53.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:53.863+0000 7f8606800640 1 -- 172.21.15.151:0/3687052220 wait complete. 2024-09-16T15:47:53.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:53.865+0000 7f8606800640 1 Processor -- start 2024-09-16T15:47:53.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:53.865+0000 7f8606800640 1 -- start start 2024-09-16T15:47:53.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:53.866+0000 7f8606800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f860006eda0 0x7f86001f8750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:53.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:53.866+0000 7f8606800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86000b2090 0x7f86001609c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:53.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:53.866+0000 7f8606800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f860015e2c0 0x7f86000b5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:53.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:53.866+0000 7f8606800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f86000af310 con 0x7f860006eda0 2024-09-16T15:47:53.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:53.866+0000 7f8606800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f86000af190 con 0x7f86000b2090 2024-09-16T15:47:53.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:53.866+0000 7f8606800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f86000af490 con 0x7f860015e2c0 2024-09-16T15:47:53.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:53.866+0000 7f85ff400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f860006eda0 0x7f86001f8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:53.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:53.866+0000 7f85ff400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f860006eda0 0x7f86001f8750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53176/0 (socket says 172.21.15.151:53176) 2024-09-16T15:47:53.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:53.866+0000 7f85ff400640 1 -- 172.21.15.151:0/283042414 learned_addr learned my addr 172.21.15.151:0/283042414 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:47:53.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:53.866+0000 7f85fea00640 1 --2- 172.21.15.151:0/283042414 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86000b2090 0x7f86001609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:53.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:53.867+0000 7f85ff400640 1 -- 172.21.15.151:0/283042414 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f860015e2c0 msgr2=0x7f86000b5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:53.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:53.867+0000 7f85ff400640 1 --2- 172.21.15.151:0/283042414 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f860015e2c0 0x7f86000b5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:53.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:53.867+0000 7f85ff400640 1 -- 172.21.15.151:0/283042414 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86000b2090 msgr2=0x7f86001609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:53.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:53.867+0000 7f85ff400640 1 --2- 172.21.15.151:0/283042414 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86000b2090 0x7f86001609c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:53.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:53.867+0000 7f85ff400640 1 -- 172.21.15.151:0/283042414 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86000cd640 con 0x7f860006eda0 2024-09-16T15:47:53.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:53.867+0000 7f85ff400640 1 --2- 172.21.15.151:0/283042414 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f860006eda0 0x7f86001f8750 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f85f000c970 tx=0x7f85f000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:53.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:53.868+0000 7f85f7400640 1 -- 172.21.15.151:0/283042414 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f0007bb0 con 0x7f860006eda0 2024-09-16T15:47:53.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:53.868+0000 7f85f7400640 1 -- 172.21.15.151:0/283042414 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f85f0007e40 con 0x7f860006eda0 2024-09-16T15:47:53.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:53.868+0000 7f85f7400640 1 -- 172.21.15.151:0/283042414 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f000f690 con 0x7f860006eda0 2024-09-16T15:47:53.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:53.868+0000 7f8606800640 1 -- 172.21.15.151:0/283042414 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86000cd920 con 0x7f860006eda0 2024-09-16T15:47:53.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:53.869+0000 7f8606800640 1 -- 172.21.15.151:0/283042414 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86000cdee0 con 0x7f860006eda0 2024-09-16T15:47:53.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:53.872+0000 7f8606800640 1 -- 172.21.15.151:0/283042414 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86000697c0 con 0x7f860006eda0 2024-09-16T15:47:53.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:53.875+0000 7f85f7400640 1 -- 172.21.15.151:0/283042414 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 29) ==== 102817+0+0 (secure 0 0 0) 0x7f85f000f830 con 0x7f860006eda0 2024-09-16T15:47:53.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:53.876+0000 7f85f7400640 1 --2- 172.21.15.151:0/283042414 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f85e0079930 0x7f85e007bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:53.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:53.877+0000 7f85f7400640 1 -- 172.21.15.151:0/283042414 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5881+0+0 (secure 0 0 0) 0x7f85f009a070 con 0x7f860006eda0 2024-09-16T15:47:53.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:53.879+0000 7f85fea00640 1 --2- 172.21.15.151:0/283042414 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f85e0079930 0x7f85e007bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:53.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:53.884+0000 7f85f7400640 1 -- 172.21.15.151:0/283042414 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f85f0014030 con 0x7f860006eda0 2024-09-16T15:47:53.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:53.887+0000 7f85fea00640 1 --2- 172.21.15.151:0/283042414 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f85e0079930 0x7f85e007bdf0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f85ec034000 tx=0x7f85ec004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:54.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.234+0000 7f8606800640 1 -- 172.21.15.151:0/283042414 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone72556", "target": ["mon-mgr", ""]}) -- 0x7f86000cf670 con 0x7f85e0079930 2024-09-16T15:47:54.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.401+0000 7f85f7400640 1 -- 172.21.15.151:0/283042414 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f86000cf670 con 0x7f85e0079930 2024-09-16T15:47:54.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.404+0000 7f8606800640 1 -- 172.21.15.151:0/283042414 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f85e0079930 msgr2=0x7f85e007bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:54.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.404+0000 7f8606800640 1 --2- 172.21.15.151:0/283042414 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f85e0079930 0x7f85e007bdf0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f85ec034000 tx=0x7f85ec004730 comp rx=0 tx=0).stop 2024-09-16T15:47:54.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.404+0000 7f8606800640 1 -- 172.21.15.151:0/283042414 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f860006eda0 msgr2=0x7f86001f8750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:54.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.404+0000 7f8606800640 1 --2- 172.21.15.151:0/283042414 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f860006eda0 0x7f86001f8750 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f85f000c970 tx=0x7f85f000ce30 comp rx=0 tx=0).stop 2024-09-16T15:47:54.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.404+0000 7f8606800640 1 -- 172.21.15.151:0/283042414 shutdown_connections 2024-09-16T15:47:54.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.404+0000 7f8606800640 1 --2- 172.21.15.151:0/283042414 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f85e0079930 0x7f85e007bdf0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:54.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.404+0000 7f8606800640 1 --2- 172.21.15.151:0/283042414 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86000b2090 0x7f86001609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:54.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.404+0000 7f8606800640 1 --2- 172.21.15.151:0/283042414 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f860015e2c0 0x7f86000b5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:54.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.404+0000 7f8606800640 1 --2- 172.21.15.151:0/283042414 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f860006eda0 0x7f86001f8750 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:54.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.404+0000 7f8606800640 1 -- 172.21.15.151:0/283042414 >> 172.21.15.151:0/283042414 conn(0x7f86000d1dd0 msgr2=0x7f8600071620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:54.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.404+0000 7f8606800640 1 -- 172.21.15.151:0/283042414 shutdown_connections 2024-09-16T15:47:54.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.404+0000 7f8606800640 1 -- 172.21.15.151:0/283042414 wait complete. 2024-09-16T15:47:54.422 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-16T15:47:54.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.582+0000 7fe3ab000640 1 -- 172.21.15.151:0/2429487929 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe38c003a50 msgr2=0x7fe38c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:54.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.582+0000 7fe3ab000640 1 --2- 172.21.15.151:0/2429487929 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe38c003a50 0x7fe38c003e70 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fe39800c310 tx=0x7fe38c005830 comp rx=0 tx=0).stop 2024-09-16T15:47:54.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.582+0000 7fe3ab000640 1 -- 172.21.15.151:0/2429487929 shutdown_connections 2024-09-16T15:47:54.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.582+0000 7fe3ab000640 1 --2- 172.21.15.151:0/2429487929 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe38c003a50 0x7fe38c003e70 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:54.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.582+0000 7fe3ab000640 1 -- 172.21.15.151:0/2429487929 >> 172.21.15.151:0/2429487929 conn(0x7fe3a40cfa20 msgr2=0x7fe3a40cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:54.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.582+0000 7fe3ab000640 1 -- 172.21.15.151:0/2429487929 shutdown_connections 2024-09-16T15:47:54.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.582+0000 7fe3ab000640 1 -- 172.21.15.151:0/2429487929 wait complete. 2024-09-16T15:47:54.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.583+0000 7fe3ab000640 1 Processor -- start 2024-09-16T15:47:54.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.584+0000 7fe3ab000640 1 -- start start 2024-09-16T15:47:54.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.584+0000 7fe3ab000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe38c003a50 0x7fe3a415e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:54.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.584+0000 7fe3ab000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe3a415df60 0x7fe3a41f2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:54.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.584+0000 7fe3ab000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe3a415ebb0 0x7fe3a415ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:54.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.584+0000 7fe3ab000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a40b2c80 con 0x7fe3a415df60 2024-09-16T15:47:54.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.584+0000 7fe3ab000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a40b2b00 con 0x7fe38c003a50 2024-09-16T15:47:54.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.584+0000 7fe3ab000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fe3a4073d40 con 0x7fe3a415ebb0 2024-09-16T15:47:54.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.585+0000 7fe3a9200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe3a415df60 0x7fe3a41f2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:54.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.585+0000 7fe3a9200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe3a415df60 0x7fe3a41f2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53190/0 (socket says 172.21.15.151:53190) 2024-09-16T15:47:54.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.585+0000 7fe3a9200640 1 -- 172.21.15.151:0/1728586455 learned_addr learned my addr 172.21.15.151:0/1728586455 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:47:54.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.585+0000 7fe3a9c00640 1 --2- 172.21.15.151:0/1728586455 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe38c003a50 0x7fe3a415e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:54.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.585+0000 7fe3aa600640 1 --2- 172.21.15.151:0/1728586455 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe3a415ebb0 0x7fe3a415ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:54.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.585+0000 7fe3a9200640 1 -- 172.21.15.151:0/1728586455 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe3a415ebb0 msgr2=0x7fe3a415ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:54.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.585+0000 7fe3a9200640 1 --2- 172.21.15.151:0/1728586455 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe3a415ebb0 0x7fe3a415ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:54.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.585+0000 7fe3a9200640 1 -- 172.21.15.151:0/1728586455 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe38c003a50 msgr2=0x7fe3a415e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:54.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.585+0000 7fe3a9200640 1 --2- 172.21.15.151:0/1728586455 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe38c003a50 0x7fe3a415e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:54.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.585+0000 7fe3a9200640 1 -- 172.21.15.151:0/1728586455 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3a415f110 con 0x7fe3a415df60 2024-09-16T15:47:54.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.585+0000 7fe3a9200640 1 --2- 172.21.15.151:0/1728586455 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe3a415df60 0x7fe3a41f2b70 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fe390011a70 tx=0x7fe390011f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:54.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.586+0000 7fe3a1600640 1 -- 172.21.15.151:0/1728586455 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe390004460 con 0x7fe3a415df60 2024-09-16T15:47:54.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.586+0000 7fe3a1600640 1 -- 172.21.15.151:0/1728586455 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe39000fa00 con 0x7fe3a415df60 2024-09-16T15:47:54.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.586+0000 7fe3ab000640 1 -- 172.21.15.151:0/1728586455 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3a415d480 con 0x7fe3a415df60 2024-09-16T15:47:54.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.586+0000 7fe3a1600640 1 -- 172.21.15.151:0/1728586455 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39000fe50 con 0x7fe3a415df60 2024-09-16T15:47:54.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.586+0000 7fe3ab000640 1 -- 172.21.15.151:0/1728586455 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3a41f30b0 con 0x7fe3a415df60 2024-09-16T15:47:54.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.587+0000 7fe3ab000640 1 -- 172.21.15.151:0/1728586455 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3a41f3370 con 0x7fe3a415df60 2024-09-16T15:47:54.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.588+0000 7fe3a1600640 1 -- 172.21.15.151:0/1728586455 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 29) ==== 102817+0+0 (secure 0 0 0) 0x7fe390005020 con 0x7fe3a415df60 2024-09-16T15:47:54.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.589+0000 7fe3a1600640 1 --2- 172.21.15.151:0/1728586455 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe384079930 0x7fe38407bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:54.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.589+0000 7fe3a1600640 1 -- 172.21.15.151:0/1728586455 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5881+0+0 (secure 0 0 0) 0x7fe390099f10 con 0x7fe3a415df60 2024-09-16T15:47:54.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.589+0000 7fe3a9c00640 1 --2- 172.21.15.151:0/1728586455 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe384079930 0x7fe38407bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:54.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.590+0000 7fe3a9c00640 1 --2- 172.21.15.151:0/1728586455 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe384079930 0x7fe38407bdf0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fe398033000 tx=0x7fe398038040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:54.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.593+0000 7fe3a1600640 1 -- 172.21.15.151:0/1728586455 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe390005a90 con 0x7fe3a415df60 2024-09-16T15:47:54.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.803+0000 7fe3ab000640 1 -- 172.21.15.151:0/1728586455 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fe3a40742a0 con 0x7fe384079930 2024-09-16T15:47:54.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.804+0000 7fe3a1600640 1 -- 172.21.15.151:0/1728586455 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fe3a40742a0 con 0x7fe384079930 2024-09-16T15:47:54.805 INFO:teuthology.orchestra.run.smithi151.stdout:[ 2024-09-16T15:47:54.806 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:47:54.806 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "cephfs" 2024-09-16T15:47:54.806 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:47:54.806 INFO:teuthology.orchestra.run.smithi151.stdout:] 2024-09-16T15:47:54.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.807+0000 7fe3ab000640 1 -- 172.21.15.151:0/1728586455 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe384079930 msgr2=0x7fe38407bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:54.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.807+0000 7fe3ab000640 1 --2- 172.21.15.151:0/1728586455 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe384079930 0x7fe38407bdf0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fe398033000 tx=0x7fe398038040 comp rx=0 tx=0).stop 2024-09-16T15:47:54.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.807+0000 7fe3ab000640 1 -- 172.21.15.151:0/1728586455 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe3a415df60 msgr2=0x7fe3a41f2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:54.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.807+0000 7fe3ab000640 1 --2- 172.21.15.151:0/1728586455 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe3a415df60 0x7fe3a41f2b70 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fe390011a70 tx=0x7fe390011f30 comp rx=0 tx=0).stop 2024-09-16T15:47:54.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.807+0000 7fe3ab000640 1 -- 172.21.15.151:0/1728586455 shutdown_connections 2024-09-16T15:47:54.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.807+0000 7fe3ab000640 1 --2- 172.21.15.151:0/1728586455 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe384079930 0x7fe38407bdf0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:54.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.807+0000 7fe3ab000640 1 --2- 172.21.15.151:0/1728586455 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe3a415ebb0 0x7fe3a415ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:54.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.807+0000 7fe3ab000640 1 --2- 172.21.15.151:0/1728586455 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe3a415df60 0x7fe3a41f2b70 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:54.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.807+0000 7fe3ab000640 1 --2- 172.21.15.151:0/1728586455 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe38c003a50 0x7fe3a415e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:54.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.807+0000 7fe3ab000640 1 -- 172.21.15.151:0/1728586455 >> 172.21.15.151:0/1728586455 conn(0x7fe3a40cfa20 msgr2=0x7fe3a40b3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:54.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.807+0000 7fe3ab000640 1 -- 172.21.15.151:0/1728586455 shutdown_connections 2024-09-16T15:47:54.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.807+0000 7fe3ab000640 1 -- 172.21.15.151:0/1728586455 wait complete. 2024-09-16T15:47:54.824 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-09-16T15:47:54.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.983+0000 7f9d79200640 1 -- 172.21.15.151:0/4273323655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d7415c080 msgr2=0x7f9d740cf120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:54.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.983+0000 7f9d79200640 1 --2- 172.21.15.151:0/4273323655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d7415c080 0x7f9d740cf120 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f9d7406db10 tx=0x7f9d6002a540 comp rx=0 tx=0).stop 2024-09-16T15:47:54.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.983+0000 7f9d79200640 1 -- 172.21.15.151:0/4273323655 shutdown_connections 2024-09-16T15:47:54.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.983+0000 7f9d79200640 1 --2- 172.21.15.151:0/4273323655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d7415c080 0x7f9d740cf120 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:54.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.983+0000 7f9d79200640 1 --2- 172.21.15.151:0/4273323655 >> v2:172.21.15.167:3300/0 conn(0x7f9d740b3370 0x7f9d740cebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:54.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.983+0000 7f9d79200640 1 -- 172.21.15.151:0/4273323655 >> 172.21.15.151:0/4273323655 conn(0x7f9d740b1840 msgr2=0x7f9d740b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:54.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.983+0000 7f9d79200640 1 -- 172.21.15.151:0/4273323655 shutdown_connections 2024-09-16T15:47:54.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.983+0000 7f9d79200640 1 -- 172.21.15.151:0/4273323655 wait complete. 2024-09-16T15:47:54.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.985+0000 7f9d79200640 1 Processor -- start 2024-09-16T15:47:54.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.985+0000 7f9d79200640 1 -- start start 2024-09-16T15:47:54.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.986+0000 7f9d79200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9d740b3370 0x7f9d740b0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:54.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.986+0000 7f9d79200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d7415c080 0x7f9d74139020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:54.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.986+0000 7f9d79200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d7415e980 0x7f9d740732d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:54.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.986+0000 7f9d79200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9d74164e20 con 0x7f9d7415c080 2024-09-16T15:47:54.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.986+0000 7f9d79200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9d74164ca0 con 0x7f9d7415e980 2024-09-16T15:47:54.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.986+0000 7f9d79200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f9d74164fa0 con 0x7f9d740b3370 2024-09-16T15:47:54.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.986+0000 7f9d71400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d7415c080 0x7f9d74139020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:54.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.986+0000 7f9d71400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d7415c080 0x7f9d74139020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53198/0 (socket says 172.21.15.151:53198) 2024-09-16T15:47:54.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.986+0000 7f9d71400640 1 -- 172.21.15.151:0/2686319844 learned_addr learned my addr 172.21.15.151:0/2686319844 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:47:54.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.987+0000 7f9d72800640 1 --2- 172.21.15.151:0/2686319844 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d7415e980 0x7f9d740732d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:54.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.987+0000 7f9d71e00640 1 --2- 172.21.15.151:0/2686319844 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9d740b3370 0x7f9d740b0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:54.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.987+0000 7f9d71400640 1 -- 172.21.15.151:0/2686319844 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9d740b3370 msgr2=0x7f9d740b0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:54.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.987+0000 7f9d71400640 1 --2- 172.21.15.151:0/2686319844 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9d740b3370 0x7f9d740b0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:54.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.987+0000 7f9d71400640 1 -- 172.21.15.151:0/2686319844 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d7415e980 msgr2=0x7f9d740732d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:54.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.987+0000 7f9d71400640 1 --2- 172.21.15.151:0/2686319844 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d7415e980 0x7f9d740732d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:54.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.987+0000 7f9d71400640 1 -- 172.21.15.151:0/2686319844 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d741f2970 con 0x7f9d7415c080 2024-09-16T15:47:54.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.987+0000 7f9d72800640 1 --2- 172.21.15.151:0/2686319844 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d7415e980 0x7f9d740732d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:47:54.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.987+0000 7f9d71400640 1 --2- 172.21.15.151:0/2686319844 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d7415c080 0x7f9d74139020 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f9d60009780 tx=0x7f9d60032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:54.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.987+0000 7f9d6a000640 1 -- 172.21.15.151:0/2686319844 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d60040070 con 0x7f9d7415c080 2024-09-16T15:47:54.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.987+0000 7f9d6a000640 1 -- 172.21.15.151:0/2686319844 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9d60005310 con 0x7f9d7415c080 2024-09-16T15:47:54.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.987+0000 7f9d79200640 1 -- 172.21.15.151:0/2686319844 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d74164540 con 0x7f9d7415c080 2024-09-16T15:47:54.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.987+0000 7f9d6a000640 1 -- 172.21.15.151:0/2686319844 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d600047a0 con 0x7f9d7415c080 2024-09-16T15:47:54.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.987+0000 7f9d79200640 1 -- 172.21.15.151:0/2686319844 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d74152820 con 0x7f9d7415c080 2024-09-16T15:47:54.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.989+0000 7f9d79200640 1 -- 172.21.15.151:0/2686319844 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d7415ee30 con 0x7f9d7415c080 2024-09-16T15:47:54.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.990+0000 7f9d6a000640 1 -- 172.21.15.151:0/2686319844 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 29) ==== 102817+0+0 (secure 0 0 0) 0x7f9d60039040 con 0x7f9d7415c080 2024-09-16T15:47:54.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.991+0000 7f9d6a000640 1 --2- 172.21.15.151:0/2686319844 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9d54079930 0x7f9d5407bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:54.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.991+0000 7f9d6a000640 1 -- 172.21.15.151:0/2686319844 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5881+0+0 (secure 0 0 0) 0x7f9d600ba7e0 con 0x7f9d7415c080 2024-09-16T15:47:54.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.992+0000 7f9d71e00640 1 --2- 172.21.15.151:0/2686319844 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9d54079930 0x7f9d5407bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:54.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.992+0000 7f9d71e00640 1 --2- 172.21.15.151:0/2686319844 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9d54079930 0x7f9d5407bdf0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9d64009b60 tx=0x7f9d64005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:54.997 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:54.996+0000 7f9d6a000640 1 -- 172.21.15.151:0/2686319844 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9d600848e0 con 0x7f9d7415c080 2024-09-16T15:47:55.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.204+0000 7f9d79200640 1 -- 172.21.15.151:0/2686319844 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9d74072db0 con 0x7f9d54079930 2024-09-16T15:47:55.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.215+0000 7f9d6a000640 1 -- 172.21.15.151:0/2686319844 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+380 (secure 0 0 0) 0x7f9d74072db0 con 0x7f9d54079930 2024-09-16T15:47:55.216 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:47:55.216 INFO:teuthology.orchestra.run.smithi151.stdout:[ 2024-09-16T15:47:55.216 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:47:55.216 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "subvol_clone74020" 2024-09-16T15:47:55.216 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:47:55.216 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:47:55.217 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sv1" 2024-09-16T15:47:55.217 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:47:55.217 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:47:55.217 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "subvol_clone93605" 2024-09-16T15:47:55.217 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:47:55.217 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:47:55.217 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "subvol_clone1123" 2024-09-16T15:47:55.217 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:47:55.217 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:47:55.217 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "subvol_clone12644" 2024-09-16T15:47:55.217 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:47:55.217 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:47:55.217 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "subvol_clone72556" 2024-09-16T15:47:55.217 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:47:55.218 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:47:55.218 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "subvol_clone92092" 2024-09-16T15:47:55.218 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:47:55.218 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:47:55.218 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "subvol_clone59661" 2024-09-16T15:47:55.218 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:47:55.218 INFO:teuthology.orchestra.run.smithi151.stdout:] 2024-09-16T15:47:55.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.217+0000 7f9d79200640 1 -- 172.21.15.151:0/2686319844 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9d54079930 msgr2=0x7f9d5407bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:55.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.217+0000 7f9d79200640 1 --2- 172.21.15.151:0/2686319844 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9d54079930 0x7f9d5407bdf0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9d64009b60 tx=0x7f9d64005f90 comp rx=0 tx=0).stop 2024-09-16T15:47:55.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.217+0000 7f9d79200640 1 -- 172.21.15.151:0/2686319844 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d7415c080 msgr2=0x7f9d74139020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:55.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.217+0000 7f9d79200640 1 --2- 172.21.15.151:0/2686319844 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d7415c080 0x7f9d74139020 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f9d60009780 tx=0x7f9d60032c10 comp rx=0 tx=0).stop 2024-09-16T15:47:55.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.217+0000 7f9d79200640 1 -- 172.21.15.151:0/2686319844 shutdown_connections 2024-09-16T15:47:55.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.217+0000 7f9d79200640 1 --2- 172.21.15.151:0/2686319844 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9d54079930 0x7f9d5407bdf0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:55.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.217+0000 7f9d79200640 1 --2- 172.21.15.151:0/2686319844 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d7415e980 0x7f9d740732d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:55.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.217+0000 7f9d79200640 1 --2- 172.21.15.151:0/2686319844 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d7415c080 0x7f9d74139020 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:55.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.217+0000 7f9d79200640 1 --2- 172.21.15.151:0/2686319844 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9d740b3370 0x7f9d740b0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:55.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.217+0000 7f9d79200640 1 -- 172.21.15.151:0/2686319844 >> 172.21.15.151:0/2686319844 conn(0x7f9d740b1840 msgr2=0x7f9d740740a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:55.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.217+0000 7f9d79200640 1 -- 172.21.15.151:0/2686319844 shutdown_connections 2024-09-16T15:47:55.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.218+0000 7f9d79200640 1 -- 172.21.15.151:0/2686319844 wait complete. 2024-09-16T15:47:55.235 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone74020 2024-09-16T15:47:55.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.402+0000 7f9140600640 1 -- 172.21.15.151:0/196947734 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9138073de0 msgr2=0x7f9138074240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:55.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.402+0000 7f9140600640 1 --2- 172.21.15.151:0/196947734 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9138073de0 0x7f9138074240 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9124009830 tx=0x7f912402a480 comp rx=0 tx=0).stop 2024-09-16T15:47:55.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.402+0000 7f9140600640 1 -- 172.21.15.151:0/196947734 shutdown_connections 2024-09-16T15:47:55.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.402+0000 7f9140600640 1 --2- 172.21.15.151:0/196947734 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f913806e6f0 0x7f913806eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:55.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.402+0000 7f9140600640 1 --2- 172.21.15.151:0/196947734 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9138073de0 0x7f9138074240 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:55.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.402+0000 7f9140600640 1 -- 172.21.15.151:0/196947734 >> 172.21.15.151:0/196947734 conn(0x7f91380b18e0 msgr2=0x7f91380b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:55.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.402+0000 7f9140600640 1 -- 172.21.15.151:0/196947734 shutdown_connections 2024-09-16T15:47:55.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.402+0000 7f9140600640 1 -- 172.21.15.151:0/196947734 wait complete. 2024-09-16T15:47:55.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.404+0000 7f9140600640 1 Processor -- start 2024-09-16T15:47:55.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.404+0000 7f9140600640 1 -- start start 2024-09-16T15:47:55.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.405+0000 7f9140600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f913806e6f0 0x7f91380ce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:55.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.405+0000 7f9140600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9138073de0 0x7f9138155910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:55.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.405+0000 7f9140600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f913815f9a0 0x7f9138157df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:55.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.405+0000 7f9140600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f913806f0e0 con 0x7f913815f9a0 2024-09-16T15:47:55.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.405+0000 7f9140600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f91380cd5e0 con 0x7f913806e6f0 2024-09-16T15:47:55.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.405+0000 7f9140600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f913806f260 con 0x7f9138073de0 2024-09-16T15:47:55.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.405+0000 7f913e600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f913815f9a0 0x7f9138157df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:55.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.405+0000 7f913e600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f913815f9a0 0x7f9138157df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53208/0 (socket says 172.21.15.151:53208) 2024-09-16T15:47:55.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.405+0000 7f913e600640 1 -- 172.21.15.151:0/438524323 learned_addr learned my addr 172.21.15.151:0/438524323 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:47:55.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.406+0000 7f913e600640 1 -- 172.21.15.151:0/438524323 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9138073de0 msgr2=0x7f9138155910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:55.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.406+0000 7f913e600640 1 --2- 172.21.15.151:0/438524323 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9138073de0 0x7f9138155910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:55.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.406+0000 7f913e600640 1 -- 172.21.15.151:0/438524323 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f913806e6f0 msgr2=0x7f91380ce7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:55.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.406+0000 7f913e600640 1 --2- 172.21.15.151:0/438524323 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f913806e6f0 0x7f91380ce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:55.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.406+0000 7f913e600640 1 -- 172.21.15.151:0/438524323 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91381f5ca0 con 0x7f913815f9a0 2024-09-16T15:47:55.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.406+0000 7f913e600640 1 --2- 172.21.15.151:0/438524323 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f913815f9a0 0x7f9138157df0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f913000efd0 tx=0x7f913000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:55.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.406+0000 7f9135600640 1 -- 172.21.15.151:0/438524323 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9130003b70 con 0x7f913815f9a0 2024-09-16T15:47:55.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.406+0000 7f9135600640 1 -- 172.21.15.151:0/438524323 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9130003e00 con 0x7f913815f9a0 2024-09-16T15:47:55.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.406+0000 7f9135600640 1 -- 172.21.15.151:0/438524323 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9130004280 con 0x7f913815f9a0 2024-09-16T15:47:55.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.406+0000 7f9140600640 1 -- 172.21.15.151:0/438524323 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91380cdce0 con 0x7f913815f9a0 2024-09-16T15:47:55.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.406+0000 7f9140600640 1 -- 172.21.15.151:0/438524323 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9138155e50 con 0x7f913815f9a0 2024-09-16T15:47:55.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.408+0000 7f9140600640 1 -- 172.21.15.151:0/438524323 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9138069360 con 0x7f913815f9a0 2024-09-16T15:47:55.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.409+0000 7f9135600640 1 -- 172.21.15.151:0/438524323 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 29) ==== 102817+0+0 (secure 0 0 0) 0x7f9130004420 con 0x7f913815f9a0 2024-09-16T15:47:55.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.409+0000 7f9135600640 1 --2- 172.21.15.151:0/438524323 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9118079930 0x7f911807bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:55.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.410+0000 7f9135600640 1 -- 172.21.15.151:0/438524323 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5881+0+0 (secure 0 0 0) 0x7f913009a790 con 0x7f913815f9a0 2024-09-16T15:47:55.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.410+0000 7f913dc00640 1 --2- 172.21.15.151:0/438524323 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9118079930 0x7f911807bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:55.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.411+0000 7f913dc00640 1 --2- 172.21.15.151:0/438524323 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9118079930 0x7f911807bdf0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f912c007fd0 tx=0x7f912c007900 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:55.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.413+0000 7f9135600640 1 -- 172.21.15.151:0/438524323 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f91300644f0 con 0x7f913815f9a0 2024-09-16T15:47:55.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.631+0000 7f9140600640 1 -- 172.21.15.151:0/438524323 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone74020", "target": ["mon-mgr", ""]}) -- 0x7f913806eba0 con 0x7f9118079930 2024-09-16T15:47:55.653 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.652+0000 7f9135600640 1 -- 172.21.15.151:0/438524323 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f913806eba0 con 0x7f9118079930 2024-09-16T15:47:55.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.654+0000 7f9140600640 1 -- 172.21.15.151:0/438524323 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9118079930 msgr2=0x7f911807bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:55.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.654+0000 7f9140600640 1 --2- 172.21.15.151:0/438524323 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9118079930 0x7f911807bdf0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f912c007fd0 tx=0x7f912c007900 comp rx=0 tx=0).stop 2024-09-16T15:47:55.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.654+0000 7f9140600640 1 -- 172.21.15.151:0/438524323 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f913815f9a0 msgr2=0x7f9138157df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:55.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.654+0000 7f9140600640 1 --2- 172.21.15.151:0/438524323 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f913815f9a0 0x7f9138157df0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f913000efd0 tx=0x7f913000c6a0 comp rx=0 tx=0).stop 2024-09-16T15:47:55.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.654+0000 7f9140600640 1 -- 172.21.15.151:0/438524323 shutdown_connections 2024-09-16T15:47:55.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.654+0000 7f9140600640 1 --2- 172.21.15.151:0/438524323 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9118079930 0x7f911807bdf0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:55.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.654+0000 7f9140600640 1 --2- 172.21.15.151:0/438524323 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f913815f9a0 0x7f9138157df0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:55.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.654+0000 7f9140600640 1 --2- 172.21.15.151:0/438524323 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9138073de0 0x7f9138155910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:55.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.654+0000 7f9140600640 1 --2- 172.21.15.151:0/438524323 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f913806e6f0 0x7f91380ce7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:55.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.654+0000 7f9140600640 1 -- 172.21.15.151:0/438524323 >> 172.21.15.151:0/438524323 conn(0x7f91380b18e0 msgr2=0x7f91380b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:55.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.654+0000 7f9140600640 1 -- 172.21.15.151:0/438524323 shutdown_connections 2024-09-16T15:47:55.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.654+0000 7f9140600640 1 -- 172.21.15.151:0/438524323 wait complete. 2024-09-16T15:47:55.666 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:47:55.665+0000 7f48f3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:55.666 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:47:55.665+0000 7f48f3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:55.666 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:47:55.665+0000 7f48f3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:55.666 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:47:55.665+0000 7f48f3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:55.666 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:47:55.665+0000 7f48f3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:47:55.672 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-09-16T15:47:55.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.835+0000 7f136d800640 1 -- 172.21.15.151:0/240248955 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f136815f9a0 msgr2=0x7f136815fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:55.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.835+0000 7f136d800640 1 --2- 172.21.15.151:0/240248955 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f136815f9a0 0x7f136815fd80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1358009830 tx=0x7f135802a480 comp rx=0 tx=0).stop 2024-09-16T15:47:55.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.835+0000 7f136d800640 1 -- 172.21.15.151:0/240248955 shutdown_connections 2024-09-16T15:47:55.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.835+0000 7f136d800640 1 --2- 172.21.15.151:0/240248955 >> v2:172.21.15.167:3300/0 conn(0x7f136806e6f0 0x7f136806eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:55.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.835+0000 7f136d800640 1 --2- 172.21.15.151:0/240248955 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f136815f9a0 0x7f136815fd80 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:55.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.835+0000 7f136d800640 1 -- 172.21.15.151:0/240248955 >> 172.21.15.151:0/240248955 conn(0x7f13680b18e0 msgr2=0x7f13680b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:55.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.835+0000 7f136d800640 1 -- 172.21.15.151:0/240248955 shutdown_connections 2024-09-16T15:47:55.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.835+0000 7f136d800640 1 -- 172.21.15.151:0/240248955 wait complete. 2024-09-16T15:47:55.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.837+0000 7f136d800640 1 Processor -- start 2024-09-16T15:47:55.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.837+0000 7f136d800640 1 -- start start 2024-09-16T15:47:55.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.838+0000 7f136d800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f136806e6f0 0x7f13680b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:55.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.838+0000 7f136d800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1368073de0 0x7f13681f7d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:55.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.838+0000 7f136d800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f136815f9a0 0x7f13681f8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:55.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.838+0000 7f136d800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f136806f0b0 con 0x7f1368073de0 2024-09-16T15:47:55.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.838+0000 7f136d800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f13680cd5e0 con 0x7f136806e6f0 2024-09-16T15:47:55.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.838+0000 7f136d800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f136806f230 con 0x7f136815f9a0 2024-09-16T15:47:55.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.838+0000 7f1366000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1368073de0 0x7f13681f7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:55.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.838+0000 7f1366000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1368073de0 0x7f13681f7d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53220/0 (socket says 172.21.15.151:53220) 2024-09-16T15:47:55.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.838+0000 7f1366000640 1 -- 172.21.15.151:0/3207786127 learned_addr learned my addr 172.21.15.151:0/3207786127 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:47:55.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.838+0000 7f1366a00640 1 --2- 172.21.15.151:0/3207786127 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f136806e6f0 0x7f13680b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:55.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.839+0000 7f1367400640 1 --2- 172.21.15.151:0/3207786127 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f136815f9a0 0x7f13681f8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:55.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.839+0000 7f1366000640 1 -- 172.21.15.151:0/3207786127 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f136815f9a0 msgr2=0x7f13681f8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:55.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.839+0000 7f1366000640 1 --2- 172.21.15.151:0/3207786127 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f136815f9a0 0x7f13681f8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:55.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.839+0000 7f1366000640 1 -- 172.21.15.151:0/3207786127 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f136806e6f0 msgr2=0x7f13680b2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:55.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.839+0000 7f1366000640 1 --2- 172.21.15.151:0/3207786127 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f136806e6f0 0x7f13680b2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:55.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.839+0000 7f1366000640 1 -- 172.21.15.151:0/3207786127 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13681536e0 con 0x7f1368073de0 2024-09-16T15:47:55.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.839+0000 7f1367400640 1 --2- 172.21.15.151:0/3207786127 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f136815f9a0 0x7f13681f8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:47:55.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.839+0000 7f1366000640 1 --2- 172.21.15.151:0/3207786127 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1368073de0 0x7f13681f7d40 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f135400c4d0 tx=0x7f135400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:55.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.840+0000 7f135ea00640 1 -- 172.21.15.151:0/3207786127 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1354016020 con 0x7f1368073de0 2024-09-16T15:47:55.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.840+0000 7f136d800640 1 -- 172.21.15.151:0/3207786127 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13681539c0 con 0x7f1368073de0 2024-09-16T15:47:55.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.840+0000 7f135ea00640 1 -- 172.21.15.151:0/3207786127 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1354012920 con 0x7f1368073de0 2024-09-16T15:47:55.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.840+0000 7f135ea00640 1 -- 172.21.15.151:0/3207786127 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1354012d30 con 0x7f1368073de0 2024-09-16T15:47:55.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.840+0000 7f136d800640 1 -- 172.21.15.151:0/3207786127 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1368155660 con 0x7f1368073de0 2024-09-16T15:47:55.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.841+0000 7f136d800640 1 -- 172.21.15.151:0/3207786127 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f136806eb70 con 0x7f1368073de0 2024-09-16T15:47:55.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.846+0000 7f135ea00640 1 -- 172.21.15.151:0/3207786127 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 29) ==== 102817+0+0 (secure 0 0 0) 0x7f1354019020 con 0x7f1368073de0 2024-09-16T15:47:55.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.847+0000 7f135ea00640 1 --2- 172.21.15.151:0/3207786127 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1348079930 0x7f134807bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:55.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.847+0000 7f135ea00640 1 -- 172.21.15.151:0/3207786127 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5881+0+0 (secure 0 0 0) 0x7f13540994d0 con 0x7f1368073de0 2024-09-16T15:47:55.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.848+0000 7f1366a00640 1 --2- 172.21.15.151:0/3207786127 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1348079930 0x7f134807bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:55.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.848+0000 7f1366a00640 1 --2- 172.21.15.151:0/3207786127 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1348079930 0x7f134807bdf0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f1358034000 tx=0x7f1358004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:55.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.850+0000 7f135ea00640 1 -- 172.21.15.151:0/3207786127 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1354020030 con 0x7f1368073de0 2024-09-16T15:47:55.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:55.857+0000 7f135ea00640 1 -- 172.21.15.151:0/3207786127 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mgrmap(e 30) ==== 102868+0+0 (secure 0 0 0) 0x7f135405f1e0 con 0x7f1368073de0 2024-09-16T15:47:56.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.066+0000 7f136d800640 1 -- 172.21.15.151:0/3207786127 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f13680709f0 con 0x7f1348079930 2024-09-16T15:47:56.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.080+0000 7f135ea00640 1 -- 172.21.15.151:0/3207786127 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f13680709f0 con 0x7f1348079930 2024-09-16T15:47:56.081 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:47:56.081 INFO:teuthology.orchestra.run.smithi151.stdout:[ 2024-09-16T15:47:56.082 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:47:56.082 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ss1" 2024-09-16T15:47:56.082 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:47:56.082 INFO:teuthology.orchestra.run.smithi151.stdout:] 2024-09-16T15:47:56.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.083+0000 7f136d800640 1 -- 172.21.15.151:0/3207786127 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1348079930 msgr2=0x7f134807bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:56.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.083+0000 7f136d800640 1 --2- 172.21.15.151:0/3207786127 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1348079930 0x7f134807bdf0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f1358034000 tx=0x7f1358004730 comp rx=0 tx=0).stop 2024-09-16T15:47:56.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.083+0000 7f136d800640 1 -- 172.21.15.151:0/3207786127 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1368073de0 msgr2=0x7f13681f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:56.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.083+0000 7f136d800640 1 --2- 172.21.15.151:0/3207786127 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1368073de0 0x7f13681f7d40 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f135400c4d0 tx=0x7f135400c990 comp rx=0 tx=0).stop 2024-09-16T15:47:56.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.083+0000 7f136d800640 1 -- 172.21.15.151:0/3207786127 shutdown_connections 2024-09-16T15:47:56.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.083+0000 7f136d800640 1 --2- 172.21.15.151:0/3207786127 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1348079930 0x7f134807bdf0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:56.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.083+0000 7f136d800640 1 --2- 172.21.15.151:0/3207786127 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f136815f9a0 0x7f13681f8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:56.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.083+0000 7f136d800640 1 --2- 172.21.15.151:0/3207786127 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1368073de0 0x7f13681f7d40 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:56.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.083+0000 7f136d800640 1 --2- 172.21.15.151:0/3207786127 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f136806e6f0 0x7f13680b2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:56.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.083+0000 7f136d800640 1 -- 172.21.15.151:0/3207786127 >> 172.21.15.151:0/3207786127 conn(0x7f13680b18e0 msgr2=0x7f13680acd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:56.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.083+0000 7f136d800640 1 -- 172.21.15.151:0/3207786127 shutdown_connections 2024-09-16T15:47:56.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.083+0000 7f136d800640 1 -- 172.21.15.151:0/3207786127 wait complete. 2024-09-16T15:47:56.100 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-09-16T15:47:56.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.264+0000 7f02e9800640 1 -- 172.21.15.151:0/1596633509 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02e415e720 msgr2=0x7f02e415b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:56.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.264+0000 7f02e9800640 1 --2- 172.21.15.151:0/1596633509 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02e415e720 0x7f02e415b980 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f02e4072b50 tx=0x7f02d002a5f0 comp rx=0 tx=0).stop 2024-09-16T15:47:56.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.264+0000 7f02e9800640 1 -- 172.21.15.151:0/1596633509 shutdown_connections 2024-09-16T15:47:56.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.265+0000 7f02e9800640 1 --2- 172.21.15.151:0/1596633509 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02e415e720 0x7f02e415b980 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:56.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.265+0000 7f02e9800640 1 --2- 172.21.15.151:0/1596633509 >> v2:172.21.15.167:3300/0 conn(0x7f02e40cf200 0x7f02e40caa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:56.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.265+0000 7f02e9800640 1 -- 172.21.15.151:0/1596633509 >> 172.21.15.151:0/1596633509 conn(0x7f02e40b1ba0 msgr2=0x7f02e40b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:56.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.265+0000 7f02e9800640 1 -- 172.21.15.151:0/1596633509 shutdown_connections 2024-09-16T15:47:56.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.265+0000 7f02e9800640 1 -- 172.21.15.151:0/1596633509 wait complete. 2024-09-16T15:47:56.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.266+0000 7f02e9800640 1 Processor -- start 2024-09-16T15:47:56.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.267+0000 7f02e9800640 1 -- start start 2024-09-16T15:47:56.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.268+0000 7f02e9800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f02e406fd10 0x7f02e41f47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:56.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.268+0000 7f02e9800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02e40cf200 0x7f02e40ae830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:56.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.268+0000 7f02e9800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f02e415e720 0x7f02e40aed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:56.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.268+0000 7f02e9800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f02e40cdd00 con 0x7f02e40cf200 2024-09-16T15:47:56.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.268+0000 7f02e9800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f02e40cde80 con 0x7f02e415e720 2024-09-16T15:47:56.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.268+0000 7f02e9800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f02e40ce000 con 0x7f02e406fd10 2024-09-16T15:47:56.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.268+0000 7f02e2000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02e40cf200 0x7f02e40ae830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:56.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.268+0000 7f02e2a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f02e406fd10 0x7f02e41f47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:56.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.268+0000 7f02e2000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02e40cf200 0x7f02e40ae830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53236/0 (socket says 172.21.15.151:53236) 2024-09-16T15:47:56.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.268+0000 7f02e2000640 1 -- 172.21.15.151:0/3138975890 learned_addr learned my addr 172.21.15.151:0/3138975890 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:47:56.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.268+0000 7f02e3400640 1 --2- 172.21.15.151:0/3138975890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f02e415e720 0x7f02e40aed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:56.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.268+0000 7f02e2000640 1 -- 172.21.15.151:0/3138975890 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f02e406fd10 msgr2=0x7f02e41f47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:56.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.268+0000 7f02e2000640 1 --2- 172.21.15.151:0/3138975890 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f02e406fd10 0x7f02e41f47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:56.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.268+0000 7f02e2000640 1 -- 172.21.15.151:0/3138975890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f02e415e720 msgr2=0x7f02e40aed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:56.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.268+0000 7f02e2000640 1 --2- 172.21.15.151:0/3138975890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f02e415e720 0x7f02e40aed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:56.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.268+0000 7f02e2000640 1 -- 172.21.15.151:0/3138975890 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02e41f4d30 con 0x7f02e40cf200 2024-09-16T15:47:56.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.269+0000 7f02e2000640 1 --2- 172.21.15.151:0/3138975890 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02e40cf200 0x7f02e40ae830 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f02d0000c00 tx=0x7f02d0003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:56.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.269+0000 7f02daa00640 1 -- 172.21.15.151:0/3138975890 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d003e030 con 0x7f02e40cf200 2024-09-16T15:47:56.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.269+0000 7f02daa00640 1 -- 172.21.15.151:0/3138975890 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f02d0003d10 con 0x7f02e40cf200 2024-09-16T15:47:56.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.269+0000 7f02e9800640 1 -- 172.21.15.151:0/3138975890 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02e40c6600 con 0x7f02e40cf200 2024-09-16T15:47:56.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.269+0000 7f02daa00640 1 -- 172.21.15.151:0/3138975890 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d0035040 con 0x7f02e40cf200 2024-09-16T15:47:56.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.269+0000 7f02e9800640 1 -- 172.21.15.151:0/3138975890 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02e40c4000 con 0x7f02e40cf200 2024-09-16T15:47:56.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.270+0000 7f02e9800640 1 -- 172.21.15.151:0/3138975890 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02e40ce560 con 0x7f02e40cf200 2024-09-16T15:47:56.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.272+0000 7f02daa00640 1 -- 172.21.15.151:0/3138975890 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 30) ==== 102868+0+0 (secure 0 0 0) 0x7f02d0041030 con 0x7f02e40cf200 2024-09-16T15:47:56.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.272+0000 7f02daa00640 1 --2- 172.21.15.151:0/3138975890 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f02c40799c0 0x7f02c407be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:56.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.273+0000 7f02daa00640 1 -- 172.21.15.151:0/3138975890 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5881+0+0 (secure 0 0 0) 0x7f02d00b9aa0 con 0x7f02e40cf200 2024-09-16T15:47:56.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.273+0000 7f02e2a00640 1 --2- 172.21.15.151:0/3138975890 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f02c40799c0 0x7f02c407be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:56.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.273+0000 7f02e2a00640 1 --2- 172.21.15.151:0/3138975890 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f02c40799c0 0x7f02c407be80 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f02d4009bd0 tx=0x7f02d4005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:56.278 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.277+0000 7f02daa00640 1 -- 172.21.15.151:0/3138975890 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f02e40c6600 con 0x7f02e40cf200 2024-09-16T15:47:56.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.499+0000 7f02e9800640 1 -- 172.21.15.151:0/3138975890 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f02e40b2270 con 0x7f02c40799c0 2024-09-16T15:47:56.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.528+0000 7f02daa00640 1 -- 172.21.15.151:0/3138975890 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f02e40b2270 con 0x7f02c40799c0 2024-09-16T15:47:56.529 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:47:56.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.530+0000 7f02e9800640 1 -- 172.21.15.151:0/3138975890 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f02c40799c0 msgr2=0x7f02c407be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:56.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.530+0000 7f02e9800640 1 --2- 172.21.15.151:0/3138975890 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f02c40799c0 0x7f02c407be80 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f02d4009bd0 tx=0x7f02d4005f90 comp rx=0 tx=0).stop 2024-09-16T15:47:56.532 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.530+0000 7f02e9800640 1 -- 172.21.15.151:0/3138975890 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02e40cf200 msgr2=0x7f02e40ae830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:56.532 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.530+0000 7f02e9800640 1 --2- 172.21.15.151:0/3138975890 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02e40cf200 0x7f02e40ae830 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f02d0000c00 tx=0x7f02d0003820 comp rx=0 tx=0).stop 2024-09-16T15:47:56.532 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.530+0000 7f02e9800640 1 -- 172.21.15.151:0/3138975890 shutdown_connections 2024-09-16T15:47:56.532 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.530+0000 7f02e9800640 1 --2- 172.21.15.151:0/3138975890 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f02c40799c0 0x7f02c407be80 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:56.532 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.530+0000 7f02e9800640 1 --2- 172.21.15.151:0/3138975890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f02e415e720 0x7f02e40aed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:56.532 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.530+0000 7f02e9800640 1 --2- 172.21.15.151:0/3138975890 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02e40cf200 0x7f02e40ae830 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:56.532 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.530+0000 7f02e9800640 1 --2- 172.21.15.151:0/3138975890 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f02e406fd10 0x7f02e41f47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:56.532 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.530+0000 7f02e9800640 1 -- 172.21.15.151:0/3138975890 >> 172.21.15.151:0/3138975890 conn(0x7f02e40b1ba0 msgr2=0x7f02e40739b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:56.532 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.530+0000 7f02e9800640 1 -- 172.21.15.151:0/3138975890 shutdown_connections 2024-09-16T15:47:56.532 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.530+0000 7f02e9800640 1 -- 172.21.15.151:0/3138975890 wait complete. 2024-09-16T15:47:56.548 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-09-16T15:47:56.711 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.710+0000 7f21f5600640 1 -- 172.21.15.151:0/1587380976 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f21f0161460 msgr2=0x7f21f00d22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:56.711 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.710+0000 7f21f5600640 1 --2- 172.21.15.151:0/1587380976 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f21f0161460 0x7f21f00d22f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f21e0009830 tx=0x7f21e002a530 comp rx=0 tx=0).stop 2024-09-16T15:47:56.711 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.710+0000 7f21f5600640 1 -- 172.21.15.151:0/1587380976 shutdown_connections 2024-09-16T15:47:56.711 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.710+0000 7f21f5600640 1 --2- 172.21.15.151:0/1587380976 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f21f0161460 0x7f21f00d22f0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:56.711 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.710+0000 7f21f5600640 1 -- 172.21.15.151:0/1587380976 >> 172.21.15.151:0/1587380976 conn(0x7f21f009d0b0 msgr2=0x7f21f00d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:56.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.710+0000 7f21f5600640 1 -- 172.21.15.151:0/1587380976 shutdown_connections 2024-09-16T15:47:56.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.710+0000 7f21f5600640 1 -- 172.21.15.151:0/1587380976 wait complete. 2024-09-16T15:47:56.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.712+0000 7f21f5600640 1 Processor -- start 2024-09-16T15:47:56.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.712+0000 7f21f5600640 1 -- start start 2024-09-16T15:47:56.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.712+0000 7f21f5600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f21f00b43c0 0x7f21f01f4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:56.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.713+0000 7f21f5600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f21f00b5480 0x7f21f00cea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:56.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.713+0000 7f21f5600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f21f0161460 0x7f21f01f7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:56.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.713+0000 7f21f5600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f21f00d29b0 con 0x7f21f00b5480 2024-09-16T15:47:56.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.713+0000 7f21f5600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f21f00d2830 con 0x7f21f0161460 2024-09-16T15:47:56.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.713+0000 7f21f5600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f21f00d2b30 con 0x7f21f00b43c0 2024-09-16T15:47:56.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.713+0000 7f21e7e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f21f00b5480 0x7f21f00cea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:56.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.713+0000 7f21e7e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f21f00b5480 0x7f21f00cea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53250/0 (socket says 172.21.15.151:53250) 2024-09-16T15:47:56.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.713+0000 7f21e7e00640 1 -- 172.21.15.151:0/2755467136 learned_addr learned my addr 172.21.15.151:0/2755467136 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:47:56.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.713+0000 7f21efe00640 1 --2- 172.21.15.151:0/2755467136 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f21f00b43c0 0x7f21f01f4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:56.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.713+0000 7f21f4c00640 1 --2- 172.21.15.151:0/2755467136 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f21f0161460 0x7f21f01f7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:56.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.713+0000 7f21e7e00640 1 -- 172.21.15.151:0/2755467136 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f21f00b43c0 msgr2=0x7f21f01f4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:56.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.713+0000 7f21e7e00640 1 --2- 172.21.15.151:0/2755467136 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f21f00b43c0 0x7f21f01f4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:56.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.713+0000 7f21e7e00640 1 -- 172.21.15.151:0/2755467136 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f21f0161460 msgr2=0x7f21f01f7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:56.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.713+0000 7f21e7e00640 1 --2- 172.21.15.151:0/2755467136 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f21f0161460 0x7f21f01f7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:56.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.713+0000 7f21e7e00640 1 -- 172.21.15.151:0/2755467136 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21f01f5500 con 0x7f21f00b5480 2024-09-16T15:47:56.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.714+0000 7f21e7e00640 1 --2- 172.21.15.151:0/2755467136 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f21f00b5480 0x7f21f00cea50 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f21d800c460 tx=0x7f21d800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:56.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.714+0000 7f21ecc00640 1 -- 172.21.15.151:0/2755467136 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d8017070 con 0x7f21f00b5480 2024-09-16T15:47:56.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.714+0000 7f21ecc00640 1 -- 172.21.15.151:0/2755467136 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f21d80035f0 con 0x7f21f00b5480 2024-09-16T15:47:56.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.714+0000 7f21f5600640 1 -- 172.21.15.151:0/2755467136 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21f01f8540 con 0x7f21f00b5480 2024-09-16T15:47:56.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.714+0000 7f21ecc00640 1 -- 172.21.15.151:0/2755467136 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d8003a40 con 0x7f21f00b5480 2024-09-16T15:47:56.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.714+0000 7f21f5600640 1 -- 172.21.15.151:0/2755467136 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21f01f47c0 con 0x7f21f00b5480 2024-09-16T15:47:56.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.715+0000 7f21f5600640 1 -- 172.21.15.151:0/2755467136 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21f00cf870 con 0x7f21f00b5480 2024-09-16T15:47:56.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.716+0000 7f21ecc00640 1 -- 172.21.15.151:0/2755467136 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 30) ==== 102868+0+0 (secure 0 0 0) 0x7f21d801a070 con 0x7f21f00b5480 2024-09-16T15:47:56.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.717+0000 7f21ecc00640 1 --2- 172.21.15.151:0/2755467136 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f21cc0799c0 0x7f21cc07be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:56.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.717+0000 7f21ecc00640 1 -- 172.21.15.151:0/2755467136 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5881+0+0 (secure 0 0 0) 0x7f21d8098af0 con 0x7f21f00b5480 2024-09-16T15:47:56.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.718+0000 7f21efe00640 1 --2- 172.21.15.151:0/2755467136 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f21cc0799c0 0x7f21cc07be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:56.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.718+0000 7f21efe00640 1 --2- 172.21.15.151:0/2755467136 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f21cc0799c0 0x7f21cc07be80 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f21e0034000 tx=0x7f21e0004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:56.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.721+0000 7f21ecc00640 1 -- 172.21.15.151:0/2755467136 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f21f01f5500 con 0x7f21f00b5480 2024-09-16T15:47:56.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.933+0000 7f21f5600640 1 -- 172.21.15.151:0/2755467136 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f21f01f4c60 con 0x7f21cc0799c0 2024-09-16T15:47:56.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.950+0000 7f21ecc00640 1 -- 172.21.15.151:0/2755467136 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f21f01f4c60 con 0x7f21cc0799c0 2024-09-16T15:47:56.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.953+0000 7f21f5600640 1 -- 172.21.15.151:0/2755467136 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f21cc0799c0 msgr2=0x7f21cc07be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:56.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.953+0000 7f21f5600640 1 --2- 172.21.15.151:0/2755467136 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f21cc0799c0 0x7f21cc07be80 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f21e0034000 tx=0x7f21e0004730 comp rx=0 tx=0).stop 2024-09-16T15:47:56.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.953+0000 7f21f5600640 1 -- 172.21.15.151:0/2755467136 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f21f00b5480 msgr2=0x7f21f00cea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:56.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.953+0000 7f21f5600640 1 --2- 172.21.15.151:0/2755467136 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f21f00b5480 0x7f21f00cea50 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f21d800c460 tx=0x7f21d800c920 comp rx=0 tx=0).stop 2024-09-16T15:47:56.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.953+0000 7f21f5600640 1 -- 172.21.15.151:0/2755467136 shutdown_connections 2024-09-16T15:47:56.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.953+0000 7f21f5600640 1 --2- 172.21.15.151:0/2755467136 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f21cc0799c0 0x7f21cc07be80 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:56.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.953+0000 7f21f5600640 1 --2- 172.21.15.151:0/2755467136 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f21f0161460 0x7f21f01f7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:56.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.953+0000 7f21f5600640 1 --2- 172.21.15.151:0/2755467136 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f21f00b5480 0x7f21f00cea50 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:56.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.953+0000 7f21f5600640 1 --2- 172.21.15.151:0/2755467136 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f21f00b43c0 0x7f21f01f4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:56.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.953+0000 7f21f5600640 1 -- 172.21.15.151:0/2755467136 >> 172.21.15.151:0/2755467136 conn(0x7f21f009d0b0 msgr2=0x7f21f00d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:56.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.953+0000 7f21f5600640 1 -- 172.21.15.151:0/2755467136 shutdown_connections 2024-09-16T15:47:56.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:56.953+0000 7f21f5600640 1 -- 172.21.15.151:0/2755467136 wait complete. 2024-09-16T15:47:56.973 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone93605 2024-09-16T15:47:57.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:57.137+0000 7f4e47e00640 1 -- 172.21.15.151:0/2691770785 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e4815e710 msgr2=0x7f4e480d4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:57.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:57.137+0000 7f4e47e00640 1 --2- 172.21.15.151:0/2691770785 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e4815e710 0x7f4e480d4a20 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f4e38005fd0 tx=0x7f4e3802a5d0 comp rx=0 tx=0).stop 2024-09-16T15:47:57.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:57.138+0000 7f4e47e00640 1 -- 172.21.15.151:0/2691770785 shutdown_connections 2024-09-16T15:47:57.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:57.138+0000 7f4e47e00640 1 --2- 172.21.15.151:0/2691770785 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e4815eaf0 0x7f4e4815ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:57.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:57.138+0000 7f4e47e00640 1 --2- 172.21.15.151:0/2691770785 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e4815e710 0x7f4e480d4a20 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:57.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:57.138+0000 7f4e47e00640 1 -- 172.21.15.151:0/2691770785 >> 172.21.15.151:0/2691770785 conn(0x7f4e480d3f00 msgr2=0x7f4e480d4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:57.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:57.138+0000 7f4e47e00640 1 -- 172.21.15.151:0/2691770785 shutdown_connections 2024-09-16T15:47:57.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:57.138+0000 7f4e47e00640 1 -- 172.21.15.151:0/2691770785 wait complete. 2024-09-16T15:47:57.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:57.139+0000 7f4e47e00640 1 Processor -- start 2024-09-16T15:47:57.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:57.140+0000 7f4e47e00640 1 -- start start 2024-09-16T15:47:57.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:57.140+0000 7f4e47e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e480d08d0 0x7f4e480b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:57.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:57.140+0000 7f4e47e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e4815e710 0x7f4e480b5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:57.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:57.140+0000 7f4e47e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e4815eaf0 0x7f4e480b8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:57.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:57.140+0000 7f4e47e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4e480d10f0 con 0x7f4e4815e710 2024-09-16T15:47:57.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:57.140+0000 7f4e47e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4e480d0f70 con 0x7f4e4815eaf0 2024-09-16T15:47:57.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:57.140+0000 7f4e47e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f4e480d1270 con 0x7f4e480d08d0 2024-09-16T15:47:57.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:57.141+0000 7f4e46000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e4815e710 0x7f4e480b5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:57.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:57.141+0000 7f4e46000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e4815e710 0x7f4e480b5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53262/0 (socket says 172.21.15.151:53262) 2024-09-16T15:47:57.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:57.141+0000 7f4e46a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e480d08d0 0x7f4e480b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:57.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:57.141+0000 7f4e46000640 1 -- 172.21.15.151:0/1592295955 learned_addr learned my addr 172.21.15.151:0/1592295955 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:47:57.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:57.141+0000 7f4e47400640 1 --2- 172.21.15.151:0/1592295955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e4815eaf0 0x7f4e480b8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:57.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:57.141+0000 7f4e46a00640 1 -- 172.21.15.151:0/1592295955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e4815eaf0 msgr2=0x7f4e480b8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:57.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:57.141+0000 7f4e46a00640 1 --2- 172.21.15.151:0/1592295955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e4815eaf0 0x7f4e480b8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:57.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:57.141+0000 7f4e46a00640 1 -- 172.21.15.151:0/1592295955 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e4815e710 msgr2=0x7f4e480b5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:57.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:57.141+0000 7f4e46a00640 1 --2- 172.21.15.151:0/1592295955 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e4815e710 0x7f4e480b5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:57.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:57.141+0000 7f4e46a00640 1 -- 172.21.15.151:0/1592295955 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e481f60b0 con 0x7f4e480d08d0 2024-09-16T15:47:57.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:57.141+0000 7f4e46000640 1 --2- 172.21.15.151:0/1592295955 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e4815e710 0x7f4e480b5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:47:57.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:57.141+0000 7f4e46a00640 1 --2- 172.21.15.151:0/1592295955 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e480d08d0 0x7f4e480b78b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4e38031000 tx=0x7f4e38003670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:57.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:57.142+0000 7f4e3f400640 1 -- 172.21.15.151:0/1592295955 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e3803f070 con 0x7f4e480d08d0 2024-09-16T15:47:57.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:57.142+0000 7f4e3f400640 1 -- 172.21.15.151:0/1592295955 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e38003da0 con 0x7f4e480d08d0 2024-09-16T15:47:57.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:57.142+0000 7f4e3f400640 1 -- 172.21.15.151:0/1592295955 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e38035030 con 0x7f4e480d08d0 2024-09-16T15:47:57.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:57.142+0000 7f4e47e00640 1 -- 172.21.15.151:0/1592295955 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e481f6390 con 0x7f4e480d08d0 2024-09-16T15:47:57.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:57.142+0000 7f4e47e00640 1 -- 172.21.15.151:0/1592295955 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e480b70b0 con 0x7f4e480d08d0 2024-09-16T15:47:57.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:57.143+0000 7f4e47e00640 1 -- 172.21.15.151:0/1592295955 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e4806d050 con 0x7f4e480d08d0 2024-09-16T15:47:57.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:57.144+0000 7f4e3f400640 1 -- 172.21.15.151:0/1592295955 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 30) ==== 102868+0+0 (secure 0 0 0) 0x7f4e38042070 con 0x7f4e480d08d0 2024-09-16T15:47:57.146 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:57.145+0000 7f4e3f400640 1 --2- 172.21.15.151:0/1592295955 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4e280799c0 0x7f4e2807be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:57.146 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:57.145+0000 7f4e3f400640 1 -- 172.21.15.151:0/1592295955 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(78..78 src has 1..78) ==== 5881+0+0 (secure 0 0 0) 0x7f4e380ba350 con 0x7f4e480d08d0 2024-09-16T15:47:57.146 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:57.146+0000 7f4e46000640 1 --2- 172.21.15.151:0/1592295955 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4e280799c0 0x7f4e2807be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:57.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:57.146+0000 7f4e46000640 1 --2- 172.21.15.151:0/1592295955 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4e280799c0 0x7f4e2807be80 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f4e481f9b40 tx=0x7f4e3400b480 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:57.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:57.150+0000 7f4e3f400640 1 -- 172.21.15.151:0/1592295955 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4e4806d050 con 0x7f4e480d08d0 2024-09-16T15:47:57.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:57.423+0000 7f4e47e00640 1 -- 172.21.15.151:0/1592295955 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone93605", "target": ["mon-mgr", ""]}) -- 0x7f4e480d4a20 con 0x7f4e280799c0 2024-09-16T15:47:58.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.269+0000 7f4e3f400640 1 -- 172.21.15.151:0/1592295955 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4e480d4a20 con 0x7f4e280799c0 2024-09-16T15:47:58.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.272+0000 7f4e47e00640 1 -- 172.21.15.151:0/1592295955 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4e280799c0 msgr2=0x7f4e2807be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:58.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.272+0000 7f4e47e00640 1 --2- 172.21.15.151:0/1592295955 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4e280799c0 0x7f4e2807be80 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f4e481f9b40 tx=0x7f4e3400b480 comp rx=0 tx=0).stop 2024-09-16T15:47:58.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.272+0000 7f4e47e00640 1 -- 172.21.15.151:0/1592295955 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e480d08d0 msgr2=0x7f4e480b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:58.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.272+0000 7f4e47e00640 1 --2- 172.21.15.151:0/1592295955 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e480d08d0 0x7f4e480b78b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4e38031000 tx=0x7f4e38003670 comp rx=0 tx=0).stop 2024-09-16T15:47:58.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.272+0000 7f4e47e00640 1 -- 172.21.15.151:0/1592295955 shutdown_connections 2024-09-16T15:47:58.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.272+0000 7f4e47e00640 1 --2- 172.21.15.151:0/1592295955 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4e280799c0 0x7f4e2807be80 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:58.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.272+0000 7f4e47e00640 1 --2- 172.21.15.151:0/1592295955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e4815eaf0 0x7f4e480b8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:58.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.272+0000 7f4e47e00640 1 --2- 172.21.15.151:0/1592295955 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e4815e710 0x7f4e480b5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:58.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.272+0000 7f4e47e00640 1 --2- 172.21.15.151:0/1592295955 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e480d08d0 0x7f4e480b78b0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:58.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.272+0000 7f4e47e00640 1 -- 172.21.15.151:0/1592295955 >> 172.21.15.151:0/1592295955 conn(0x7f4e480d3f00 msgr2=0x7f4e480d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:58.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.272+0000 7f4e47e00640 1 -- 172.21.15.151:0/1592295955 shutdown_connections 2024-09-16T15:47:58.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.272+0000 7f4e47e00640 1 -- 172.21.15.151:0/1592295955 wait complete. 2024-09-16T15:47:58.292 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone1123 2024-09-16T15:47:58.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.455+0000 7f6e99000640 1 -- 172.21.15.151:0/2610189800 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6e9406e6f0 msgr2=0x7f6e9406eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:58.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.455+0000 7f6e99000640 1 --2- 172.21.15.151:0/2610189800 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6e9406e6f0 0x7f6e9406eba0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c005fd0 tx=0x7f6e8c02ab20 comp rx=0 tx=0).stop 2024-09-16T15:47:58.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.455+0000 7f6e99000640 1 -- 172.21.15.151:0/2610189800 shutdown_connections 2024-09-16T15:47:58.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.455+0000 7f6e99000640 1 --2- 172.21.15.151:0/2610189800 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6e9406e6f0 0x7f6e9406eba0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:58.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.455+0000 7f6e99000640 1 --2- 172.21.15.151:0/2610189800 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e9415f910 0x7f6e9415fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:58.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.455+0000 7f6e99000640 1 -- 172.21.15.151:0/2610189800 >> 172.21.15.151:0/2610189800 conn(0x7f6e940b18e0 msgr2=0x7f6e940b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:58.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.455+0000 7f6e99000640 1 -- 172.21.15.151:0/2610189800 shutdown_connections 2024-09-16T15:47:58.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.455+0000 7f6e99000640 1 -- 172.21.15.151:0/2610189800 wait complete. 2024-09-16T15:47:58.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.457+0000 7f6e99000640 1 Processor -- start 2024-09-16T15:47:58.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.457+0000 7f6e99000640 1 -- start start 2024-09-16T15:47:58.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.458+0000 7f6e99000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e9406e6f0 0x7f6e9415f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:58.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.458+0000 7f6e99000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6e9415f910 0x7f6e940ad210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:58.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.458+0000 7f6e99000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e9413ff70 0x7f6e940ad750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:58.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.458+0000 7f6e99000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6e9406f260 con 0x7f6e9415f910 2024-09-16T15:47:58.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.458+0000 7f6e99000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6e9406f0e0 con 0x7f6e9413ff70 2024-09-16T15:47:58.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.458+0000 7f6e99000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f6e9406f3e0 con 0x7f6e9406e6f0 2024-09-16T15:47:58.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.458+0000 7f6e91e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e9406e6f0 0x7f6e9415f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:58.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.458+0000 7f6e91400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6e9415f910 0x7f6e940ad210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:58.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.458+0000 7f6e91e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e9406e6f0 0x7f6e9415f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.151:51976/0 (socket says 172.21.15.151:51976) 2024-09-16T15:47:58.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.458+0000 7f6e91e00640 1 -- 172.21.15.151:0/3237397743 learned_addr learned my addr 172.21.15.151:0/3237397743 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:47:58.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.458+0000 7f6e92800640 1 --2- 172.21.15.151:0/3237397743 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e9413ff70 0x7f6e940ad750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:58.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.459+0000 7f6e91e00640 1 -- 172.21.15.151:0/3237397743 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e9413ff70 msgr2=0x7f6e940ad750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:58.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.459+0000 7f6e91e00640 1 --2- 172.21.15.151:0/3237397743 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e9413ff70 0x7f6e940ad750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:58.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.459+0000 7f6e91e00640 1 -- 172.21.15.151:0/3237397743 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6e9415f910 msgr2=0x7f6e940ad210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:58.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.459+0000 7f6e91e00640 1 --2- 172.21.15.151:0/3237397743 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6e9415f910 0x7f6e940ad210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:58.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.459+0000 7f6e91e00640 1 -- 172.21.15.151:0/3237397743 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e9415e460 con 0x7f6e9406e6f0 2024-09-16T15:47:58.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.459+0000 7f6e91400640 1 --2- 172.21.15.151:0/3237397743 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6e9415f910 0x7f6e940ad210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:47:58.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.459+0000 7f6e91e00640 1 --2- 172.21.15.151:0/3237397743 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e9406e6f0 0x7f6e9415f260 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6e8400e790 tx=0x7f6e8400ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:58.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.459+0000 7f6e8a000640 1 -- 172.21.15.151:0/3237397743 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e8400eea0 con 0x7f6e9406e6f0 2024-09-16T15:47:58.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.459+0000 7f6e8a000640 1 -- 172.21.15.151:0/3237397743 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6e8400cdb0 con 0x7f6e9406e6f0 2024-09-16T15:47:58.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.459+0000 7f6e99000640 1 -- 172.21.15.151:0/3237397743 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e941425d0 con 0x7f6e9406e6f0 2024-09-16T15:47:58.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.459+0000 7f6e8a000640 1 -- 172.21.15.151:0/3237397743 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e84010690 con 0x7f6e9406e6f0 2024-09-16T15:47:58.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.459+0000 7f6e99000640 1 -- 172.21.15.151:0/3237397743 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e94142b90 con 0x7f6e9406e6f0 2024-09-16T15:47:58.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.461+0000 7f6e99000640 1 -- 172.21.15.151:0/3237397743 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e94069360 con 0x7f6e9406e6f0 2024-09-16T15:47:58.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.462+0000 7f6e8a000640 1 -- 172.21.15.151:0/3237397743 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 30) ==== 102868+0+0 (secure 0 0 0) 0x7f6e84010830 con 0x7f6e9406e6f0 2024-09-16T15:47:58.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.463+0000 7f6e8a000640 1 --2- 172.21.15.151:0/3237397743 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6e740799c0 0x7f6e7407be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:58.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.463+0000 7f6e8a000640 1 -- 172.21.15.151:0/3237397743 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(78..78 src has 1..78) ==== 5881+0+0 (secure 0 0 0) 0x7f6e8409a210 con 0x7f6e9406e6f0 2024-09-16T15:47:58.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.463+0000 7f6e91400640 1 --2- 172.21.15.151:0/3237397743 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6e740799c0 0x7f6e7407be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:58.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.464+0000 7f6e91400640 1 --2- 172.21.15.151:0/3237397743 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6e740799c0 0x7f6e7407be80 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6e9415dcb0 tx=0x7f6e80009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:58.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.467+0000 7f6e8a000640 1 -- 172.21.15.151:0/3237397743 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6e840638b0 con 0x7f6e9406e6f0 2024-09-16T15:47:58.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.679+0000 7f6e99000640 1 -- 172.21.15.151:0/3237397743 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone1123", "target": ["mon-mgr", ""]}) -- 0x7f6e9406eba0 con 0x7f6e740799c0 2024-09-16T15:47:58.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.703+0000 7f6e8a000640 1 -- 172.21.15.151:0/3237397743 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6e9406eba0 con 0x7f6e740799c0 2024-09-16T15:47:58.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.706+0000 7f6e99000640 1 -- 172.21.15.151:0/3237397743 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6e740799c0 msgr2=0x7f6e7407be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:58.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.706+0000 7f6e99000640 1 --2- 172.21.15.151:0/3237397743 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6e740799c0 0x7f6e7407be80 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6e9415dcb0 tx=0x7f6e80009040 comp rx=0 tx=0).stop 2024-09-16T15:47:58.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.706+0000 7f6e99000640 1 -- 172.21.15.151:0/3237397743 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e9406e6f0 msgr2=0x7f6e9415f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:58.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.706+0000 7f6e99000640 1 --2- 172.21.15.151:0/3237397743 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e9406e6f0 0x7f6e9415f260 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6e8400e790 tx=0x7f6e8400ec50 comp rx=0 tx=0).stop 2024-09-16T15:47:58.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.706+0000 7f6e99000640 1 -- 172.21.15.151:0/3237397743 shutdown_connections 2024-09-16T15:47:58.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.706+0000 7f6e99000640 1 --2- 172.21.15.151:0/3237397743 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6e740799c0 0x7f6e7407be80 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:58.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.706+0000 7f6e99000640 1 --2- 172.21.15.151:0/3237397743 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e9413ff70 0x7f6e940ad750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:58.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.706+0000 7f6e99000640 1 --2- 172.21.15.151:0/3237397743 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6e9415f910 0x7f6e940ad210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:58.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.706+0000 7f6e99000640 1 --2- 172.21.15.151:0/3237397743 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e9406e6f0 0x7f6e9415f260 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:58.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.706+0000 7f6e99000640 1 -- 172.21.15.151:0/3237397743 >> 172.21.15.151:0/3237397743 conn(0x7f6e940b18e0 msgr2=0x7f6e940cdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:58.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.706+0000 7f6e99000640 1 -- 172.21.15.151:0/3237397743 shutdown_connections 2024-09-16T15:47:58.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.706+0000 7f6e99000640 1 -- 172.21.15.151:0/3237397743 wait complete. 2024-09-16T15:47:58.726 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone12644 2024-09-16T15:47:58.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.890+0000 7f8fce400640 1 -- 172.21.15.151:0/2573264175 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8fc80b0ef0 msgr2=0x7f8fc80d2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:58.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.890+0000 7f8fce400640 1 --2- 172.21.15.151:0/2573264175 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8fc80b0ef0 0x7f8fc80d2950 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4009860 tx=0x7f8fb402a5a0 comp rx=0 tx=0).stop 2024-09-16T15:47:58.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.890+0000 7f8fce400640 1 -- 172.21.15.151:0/2573264175 shutdown_connections 2024-09-16T15:47:58.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.890+0000 7f8fce400640 1 --2- 172.21.15.151:0/2573264175 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8fc80b0ef0 0x7f8fc80d2950 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:58.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.890+0000 7f8fce400640 1 -- 172.21.15.151:0/2573264175 >> 172.21.15.151:0/2573264175 conn(0x7f8fc80d1e30 msgr2=0x7f8fc80d2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:58.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.890+0000 7f8fce400640 1 -- 172.21.15.151:0/2573264175 shutdown_connections 2024-09-16T15:47:58.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.890+0000 7f8fce400640 1 -- 172.21.15.151:0/2573264175 wait complete. 2024-09-16T15:47:58.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.891+0000 7f8fce400640 1 Processor -- start 2024-09-16T15:47:58.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.891+0000 7f8fce400640 1 -- start start 2024-09-16T15:47:58.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.892+0000 7f8fce400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8fc80aea80 0x7f8fc8152460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:58.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.892+0000 7f8fce400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fc80b0ef0 0x7f8fc80b1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:58.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.892+0000 7f8fce400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8fc80d0410 0x7f8fc815b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:58.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.892+0000 7f8fce400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc80af9b0 con 0x7f8fc80b0ef0 2024-09-16T15:47:58.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.892+0000 7f8fce400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc80af830 con 0x7f8fc80d0410 2024-09-16T15:47:58.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.892+0000 7f8fce400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f8fc80afb30 con 0x7f8fc80aea80 2024-09-16T15:47:58.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.893+0000 7f8fc6a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fc80b0ef0 0x7f8fc80b1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:58.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.893+0000 7f8fc6a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fc80b0ef0 0x7f8fc80b1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53288/0 (socket says 172.21.15.151:53288) 2024-09-16T15:47:58.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.893+0000 7f8fc6a00640 1 -- 172.21.15.151:0/3243085652 learned_addr learned my addr 172.21.15.151:0/3243085652 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:47:58.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.893+0000 7f8fc7e00640 1 --2- 172.21.15.151:0/3243085652 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8fc80d0410 0x7f8fc815b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:58.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.893+0000 7f8fc7400640 1 --2- 172.21.15.151:0/3243085652 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8fc80aea80 0x7f8fc8152460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:58.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.893+0000 7f8fc6a00640 1 -- 172.21.15.151:0/3243085652 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8fc80aea80 msgr2=0x7f8fc8152460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:58.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.893+0000 7f8fc6a00640 1 --2- 172.21.15.151:0/3243085652 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8fc80aea80 0x7f8fc8152460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:58.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.893+0000 7f8fc6a00640 1 -- 172.21.15.151:0/3243085652 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8fc80d0410 msgr2=0x7f8fc815b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:58.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.893+0000 7f8fc6a00640 1 --2- 172.21.15.151:0/3243085652 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8fc80d0410 0x7f8fc815b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:58.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.893+0000 7f8fc6a00640 1 -- 172.21.15.151:0/3243085652 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fc81f2520 con 0x7f8fc80b0ef0 2024-09-16T15:47:58.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.893+0000 7f8fc6a00640 1 --2- 172.21.15.151:0/3243085652 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fc80b0ef0 0x7f8fc80b1400 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4009980 tx=0x7f8fb4032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:58.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.894+0000 7f8fbf400640 1 -- 172.21.15.151:0/3243085652 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb4040070 con 0x7f8fc80b0ef0 2024-09-16T15:47:58.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.894+0000 7f8fbf400640 1 -- 172.21.15.151:0/3243085652 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8fb40038e0 con 0x7f8fc80b0ef0 2024-09-16T15:47:58.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.894+0000 7f8fbf400640 1 -- 172.21.15.151:0/3243085652 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb4003d30 con 0x7f8fc80b0ef0 2024-09-16T15:47:58.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.894+0000 7f8fce400640 1 -- 172.21.15.151:0/3243085652 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fc81f2800 con 0x7f8fc80b0ef0 2024-09-16T15:47:58.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.894+0000 7f8fce400640 1 -- 172.21.15.151:0/3243085652 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fc81f2dc0 con 0x7f8fc80b0ef0 2024-09-16T15:47:58.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.895+0000 7f8fce400640 1 -- 172.21.15.151:0/3243085652 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fc80d2950 con 0x7f8fc80b0ef0 2024-09-16T15:47:58.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.896+0000 7f8fbf400640 1 -- 172.21.15.151:0/3243085652 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 30) ==== 102868+0+0 (secure 0 0 0) 0x7f8fb4039040 con 0x7f8fc80b0ef0 2024-09-16T15:47:58.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.897+0000 7f8fbf400640 1 --2- 172.21.15.151:0/3243085652 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8fa80799c0 0x7f8fa807be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:58.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.897+0000 7f8fbf400640 1 -- 172.21.15.151:0/3243085652 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5881+0+0 (secure 0 0 0) 0x7f8fb40bad30 con 0x7f8fc80b0ef0 2024-09-16T15:47:58.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.897+0000 7f8fc7400640 1 --2- 172.21.15.151:0/3243085652 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8fa80799c0 0x7f8fa807be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:58.899 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.898+0000 7f8fc7400640 1 --2- 172.21.15.151:0/3243085652 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8fa80799c0 0x7f8fa807be80 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f8fb80046b0 tx=0x7f8fb8007cc0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:58.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:58.901+0000 7f8fbf400640 1 -- 172.21.15.151:0/3243085652 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8fb4084450 con 0x7f8fc80b0ef0 2024-09-16T15:47:59.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:59.111+0000 7f8fce400640 1 -- 172.21.15.151:0/3243085652 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone12644", "target": ["mon-mgr", ""]}) -- 0x7f8fc80af100 con 0x7f8fa80799c0 2024-09-16T15:47:59.130 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:59.129+0000 7f8fbf400640 1 -- 172.21.15.151:0/3243085652 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8fc80af100 con 0x7f8fa80799c0 2024-09-16T15:47:59.133 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:59.131+0000 7f8fce400640 1 -- 172.21.15.151:0/3243085652 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8fa80799c0 msgr2=0x7f8fa807be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:59.133 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:59.131+0000 7f8fce400640 1 --2- 172.21.15.151:0/3243085652 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8fa80799c0 0x7f8fa807be80 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f8fb80046b0 tx=0x7f8fb8007cc0 comp rx=0 tx=0).stop 2024-09-16T15:47:59.133 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:59.131+0000 7f8fce400640 1 -- 172.21.15.151:0/3243085652 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fc80b0ef0 msgr2=0x7f8fc80b1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:59.133 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:59.131+0000 7f8fce400640 1 --2- 172.21.15.151:0/3243085652 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fc80b0ef0 0x7f8fc80b1400 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4009980 tx=0x7f8fb4032c10 comp rx=0 tx=0).stop 2024-09-16T15:47:59.133 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:59.131+0000 7f8fce400640 1 -- 172.21.15.151:0/3243085652 shutdown_connections 2024-09-16T15:47:59.133 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:59.131+0000 7f8fce400640 1 --2- 172.21.15.151:0/3243085652 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8fa80799c0 0x7f8fa807be80 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:59.135 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:59.131+0000 7f8fce400640 1 --2- 172.21.15.151:0/3243085652 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8fc80d0410 0x7f8fc815b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:59.135 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:59.131+0000 7f8fce400640 1 --2- 172.21.15.151:0/3243085652 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fc80b0ef0 0x7f8fc80b1400 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:59.135 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:59.131+0000 7f8fce400640 1 --2- 172.21.15.151:0/3243085652 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8fc80aea80 0x7f8fc8152460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:59.135 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:59.131+0000 7f8fce400640 1 -- 172.21.15.151:0/3243085652 >> 172.21.15.151:0/3243085652 conn(0x7f8fc80d1e30 msgr2=0x7f8fc8075890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:59.135 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:59.132+0000 7f8fce400640 1 -- 172.21.15.151:0/3243085652 shutdown_connections 2024-09-16T15:47:59.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:59.132+0000 7f8fce400640 1 -- 172.21.15.151:0/3243085652 wait complete. 2024-09-16T15:47:59.150 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone72556 2024-09-16T15:47:59.316 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:59.315+0000 7f599b200640 1 -- 172.21.15.151:0/2279625492 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f599406e590 msgr2=0x7f599406fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:59.316 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:59.315+0000 7f599b200640 1 --2- 172.21.15.151:0/2279625492 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f599406e590 0x7f599406fe40 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f5994074a70 tx=0x7f598402a530 comp rx=0 tx=0).stop 2024-09-16T15:47:59.317 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:59.315+0000 7f599b200640 1 -- 172.21.15.151:0/2279625492 shutdown_connections 2024-09-16T15:47:59.317 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:59.315+0000 7f599b200640 1 --2- 172.21.15.151:0/2279625492 >> v2:172.21.15.167:3300/0 conn(0x7f5994070380 0x7f59940cb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:59.317 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:59.315+0000 7f599b200640 1 --2- 172.21.15.151:0/2279625492 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f599406e590 0x7f599406fe40 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:59.317 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:59.315+0000 7f599b200640 1 -- 172.21.15.151:0/2279625492 >> 172.21.15.151:0/2279625492 conn(0x7f59940cfa70 msgr2=0x7f59940cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:47:59.317 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:59.315+0000 7f599b200640 1 -- 172.21.15.151:0/2279625492 shutdown_connections 2024-09-16T15:47:59.317 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:59.315+0000 7f599b200640 1 -- 172.21.15.151:0/2279625492 wait complete. 2024-09-16T15:47:59.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:59.318+0000 7f599b200640 1 Processor -- start 2024-09-16T15:47:59.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:59.318+0000 7f599b200640 1 -- start start 2024-09-16T15:47:59.321 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:59.319+0000 7f599b200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f599406e590 0x7f5994151870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:59.321 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:59.319+0000 7f599b200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5994070380 0x7f599415d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:59.321 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:59.319+0000 7f599b200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f599415bda0 0x7f599415d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:59.321 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:59.319+0000 7f599b200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f59940cba90 con 0x7f5994070380 2024-09-16T15:47:59.321 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:59.319+0000 7f599b200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f59940cb910 con 0x7f599415bda0 2024-09-16T15:47:59.321 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:59.319+0000 7f599b200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f59940cbc10 con 0x7f599406e590 2024-09-16T15:47:59.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:59.321+0000 7f5999400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5994070380 0x7f599415d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:59.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:59.321+0000 7f5999400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5994070380 0x7f599415d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53300/0 (socket says 172.21.15.151:53300) 2024-09-16T15:47:59.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:59.321+0000 7f5999400640 1 -- 172.21.15.151:0/1740788727 learned_addr learned my addr 172.21.15.151:0/1740788727 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:47:59.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:59.322+0000 7f5999400640 1 -- 172.21.15.151:0/1740788727 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f599406e590 msgr2=0x7f5994151870 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:47:59.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:59.322+0000 7f5999400640 1 --2- 172.21.15.151:0/1740788727 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f599406e590 0x7f5994151870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:59.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:59.322+0000 7f5999400640 1 -- 172.21.15.151:0/1740788727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f599415bda0 msgr2=0x7f599415d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:47:59.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:59.322+0000 7f5999400640 1 --2- 172.21.15.151:0/1740788727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f599415bda0 0x7f599415d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:47:59.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:59.322+0000 7f5999400640 1 -- 172.21.15.151:0/1740788727 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59940cf650 con 0x7f5994070380 2024-09-16T15:47:59.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:59.323+0000 7f5999400640 1 --2- 172.21.15.151:0/1740788727 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5994070380 0x7f599415d310 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f5984009780 tx=0x7f5984032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:59.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:59.325+0000 7f598e000640 1 -- 172.21.15.151:0/1740788727 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5984040070 con 0x7f5994070380 2024-09-16T15:47:59.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:59.325+0000 7f598e000640 1 -- 172.21.15.151:0/1740788727 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5984005310 con 0x7f5994070380 2024-09-16T15:47:59.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:59.325+0000 7f598e000640 1 -- 172.21.15.151:0/1740788727 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59840047a0 con 0x7f5994070380 2024-09-16T15:47:59.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:59.326+0000 7f599b200640 1 -- 172.21.15.151:0/1740788727 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5994151db0 con 0x7f5994070380 2024-09-16T15:47:59.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:59.326+0000 7f599b200640 1 -- 172.21.15.151:0/1740788727 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5994155c00 con 0x7f5994070380 2024-09-16T15:47:59.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:59.327+0000 7f599b200640 1 -- 172.21.15.151:0/1740788727 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59940cc1a0 con 0x7f5994070380 2024-09-16T15:47:59.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:59.331+0000 7f598e000640 1 -- 172.21.15.151:0/1740788727 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 30) ==== 102868+0+0 (secure 0 0 0) 0x7f5984039040 con 0x7f5994070380 2024-09-16T15:47:59.333 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:59.332+0000 7f598e000640 1 --2- 172.21.15.151:0/1740788727 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5978079a10 0x7f597807bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:47:59.333 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:59.333+0000 7f598e000640 1 -- 172.21.15.151:0/1740788727 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5881+0+0 (secure 0 0 0) 0x7f59840ba950 con 0x7f5994070380 2024-09-16T15:47:59.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:59.333+0000 7f5999e00640 1 --2- 172.21.15.151:0/1740788727 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5978079a10 0x7f597807bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:47:59.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:59.335+0000 7f5999e00640 1 --2- 172.21.15.151:0/1740788727 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5978079a10 0x7f597807bed0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f59880014d0 tx=0x7f59880078a0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:47:59.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:59.340+0000 7f598e000640 1 -- 172.21.15.151:0/1740788727 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f5984084700 con 0x7f5994070380 2024-09-16T15:47:59.732 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:47:59.731+0000 7f599b200640 1 -- 172.21.15.151:0/1740788727 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone72556", "target": ["mon-mgr", ""]}) -- 0x7f5994069890 con 0x7f5978079a10 2024-09-16T15:48:03.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:03.960+0000 7f598e000640 1 -- 172.21.15.151:0/1740788727 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5994069890 con 0x7f5978079a10 2024-09-16T15:48:03.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:03.964+0000 7f599b200640 1 -- 172.21.15.151:0/1740788727 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5978079a10 msgr2=0x7f597807bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:03.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:03.964+0000 7f599b200640 1 --2- 172.21.15.151:0/1740788727 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5978079a10 0x7f597807bed0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f59880014d0 tx=0x7f59880078a0 comp rx=0 tx=0).stop 2024-09-16T15:48:03.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:03.964+0000 7f599b200640 1 -- 172.21.15.151:0/1740788727 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5994070380 msgr2=0x7f599415d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:03.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:03.964+0000 7f599b200640 1 --2- 172.21.15.151:0/1740788727 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5994070380 0x7f599415d310 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f5984009780 tx=0x7f5984032c10 comp rx=0 tx=0).stop 2024-09-16T15:48:03.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:03.964+0000 7f599b200640 1 -- 172.21.15.151:0/1740788727 shutdown_connections 2024-09-16T15:48:03.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:03.964+0000 7f599b200640 1 --2- 172.21.15.151:0/1740788727 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5978079a10 0x7f597807bed0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:03.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:03.964+0000 7f599b200640 1 --2- 172.21.15.151:0/1740788727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f599415bda0 0x7f599415d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:03.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:03.964+0000 7f599b200640 1 --2- 172.21.15.151:0/1740788727 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5994070380 0x7f599415d310 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:03.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:03.964+0000 7f599b200640 1 --2- 172.21.15.151:0/1740788727 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f599406e590 0x7f5994151870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:03.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:03.964+0000 7f599b200640 1 -- 172.21.15.151:0/1740788727 >> 172.21.15.151:0/1740788727 conn(0x7f59940cfa70 msgr2=0x7f59940cc3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:03.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:03.964+0000 7f599b200640 1 -- 172.21.15.151:0/1740788727 shutdown_connections 2024-09-16T15:48:03.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:03.964+0000 7f599b200640 1 -- 172.21.15.151:0/1740788727 wait complete. 2024-09-16T15:48:03.986 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone92092 2024-09-16T15:48:04.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.151+0000 7f804c600640 1 -- 172.21.15.151:0/3635006691 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f804415f910 msgr2=0x7f804415fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:04.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.151+0000 7f804c600640 1 --2- 172.21.15.151:0/3635006691 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f804415f910 0x7f804415fcf0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f80440af430 tx=0x7f803802a940 comp rx=0 tx=0).stop 2024-09-16T15:48:04.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.151+0000 7f804c600640 1 -- 172.21.15.151:0/3635006691 shutdown_connections 2024-09-16T15:48:04.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.151+0000 7f804c600640 1 --2- 172.21.15.151:0/3635006691 >> v2:172.21.15.167:3300/0 conn(0x7f80440733e0 0x7f80440cce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:04.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.151+0000 7f804c600640 1 --2- 172.21.15.151:0/3635006691 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f804415f910 0x7f804415fcf0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:04.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.151+0000 7f804c600640 1 -- 172.21.15.151:0/3635006691 >> 172.21.15.151:0/3635006691 conn(0x7f80440b18e0 msgr2=0x7f80440b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:04.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.151+0000 7f804c600640 1 -- 172.21.15.151:0/3635006691 shutdown_connections 2024-09-16T15:48:04.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.151+0000 7f804c600640 1 -- 172.21.15.151:0/3635006691 wait complete. 2024-09-16T15:48:04.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.153+0000 7f804c600640 1 Processor -- start 2024-09-16T15:48:04.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.153+0000 7f804c600640 1 -- start start 2024-09-16T15:48:04.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.154+0000 7f804c600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f804415c350 0x7f80440cacf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:04.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.154+0000 7f804c600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f804415f910 0x7f80441f91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:04.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.154+0000 7f804c600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80440c97c0 0x7f80440cfe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:04.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.154+0000 7f804c600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f80440cd510 con 0x7f804415f910 2024-09-16T15:48:04.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.154+0000 7f804c600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f80440cd390 con 0x7f80440c97c0 2024-09-16T15:48:04.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.154+0000 7f804c600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f804415cf10 con 0x7f804415c350 2024-09-16T15:48:04.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.154+0000 7f8049200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f804415f910 0x7f80441f91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:04.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.154+0000 7f8049200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f804415f910 0x7f80441f91a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44612/0 (socket says 172.21.15.151:44612) 2024-09-16T15:48:04.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.154+0000 7f8049200640 1 -- 172.21.15.151:0/301762114 learned_addr learned my addr 172.21.15.151:0/301762114 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:48:04.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.155+0000 7f804a600640 1 --2- 172.21.15.151:0/301762114 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80440c97c0 0x7f80440cfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:04.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.155+0000 7f8049200640 1 -- 172.21.15.151:0/301762114 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f804415c350 msgr2=0x7f80440cacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:04.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.155+0000 7f8049200640 1 --2- 172.21.15.151:0/301762114 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f804415c350 0x7f80440cacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:04.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.155+0000 7f8049200640 1 -- 172.21.15.151:0/301762114 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80440c97c0 msgr2=0x7f80440cfe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:04.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.155+0000 7f8049200640 1 --2- 172.21.15.151:0/301762114 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80440c97c0 0x7f80440cfe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:04.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.155+0000 7f8049200640 1 -- 172.21.15.151:0/301762114 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80440c9c50 con 0x7f804415f910 2024-09-16T15:48:04.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.155+0000 7f804a600640 1 --2- 172.21.15.151:0/301762114 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80440c97c0 0x7f80440cfe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:48:04.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.155+0000 7f8049200640 1 --2- 172.21.15.151:0/301762114 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f804415f910 0x7f80441f91a0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f803000d950 tx=0x7f803000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:04.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.155+0000 7f8041600640 1 -- 172.21.15.151:0/301762114 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8030017070 con 0x7f804415f910 2024-09-16T15:48:04.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.155+0000 7f8041600640 1 -- 172.21.15.151:0/301762114 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8030009d20 con 0x7f804415f910 2024-09-16T15:48:04.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.155+0000 7f8041600640 1 -- 172.21.15.151:0/301762114 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80300106c0 con 0x7f804415f910 2024-09-16T15:48:04.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.155+0000 7f804c600640 1 -- 172.21.15.151:0/301762114 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80440c9f30 con 0x7f804415f910 2024-09-16T15:48:04.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.155+0000 7f804c600640 1 -- 172.21.15.151:0/301762114 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80440ca4f0 con 0x7f804415f910 2024-09-16T15:48:04.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.157+0000 7f804c600640 1 -- 172.21.15.151:0/301762114 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8044069360 con 0x7f804415f910 2024-09-16T15:48:04.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.158+0000 7f8041600640 1 -- 172.21.15.151:0/301762114 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 30) ==== 102868+0+0 (secure 0 0 0) 0x7f8030010860 con 0x7f804415f910 2024-09-16T15:48:04.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.159+0000 7f8041600640 1 --2- 172.21.15.151:0/301762114 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f80240799c0 0x7f802407be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:04.160 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.159+0000 7f8041600640 1 -- 172.21.15.151:0/301762114 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5937+0+0 (secure 0 0 0) 0x7f8030099cb0 con 0x7f804415f910 2024-09-16T15:48:04.160 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.159+0000 7f8049c00640 1 --2- 172.21.15.151:0/301762114 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f80240799c0 0x7f802407be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:04.161 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.160+0000 7f8049c00640 1 --2- 172.21.15.151:0/301762114 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f80240799c0 0x7f802407be80 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f8038034000 tx=0x7f8038004a40 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:04.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.163+0000 7f8041600640 1 -- 172.21.15.151:0/301762114 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8030063a20 con 0x7f804415f910 2024-09-16T15:48:04.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.375+0000 7f804c600640 1 -- 172.21.15.151:0/301762114 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone92092", "target": ["mon-mgr", ""]}) -- 0x7f80440cce50 con 0x7f80240799c0 2024-09-16T15:48:04.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.398+0000 7f8041600640 1 -- 172.21.15.151:0/301762114 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f80440cce50 con 0x7f80240799c0 2024-09-16T15:48:04.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.401+0000 7f804c600640 1 -- 172.21.15.151:0/301762114 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f80240799c0 msgr2=0x7f802407be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:04.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.401+0000 7f804c600640 1 --2- 172.21.15.151:0/301762114 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f80240799c0 0x7f802407be80 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f8038034000 tx=0x7f8038004a40 comp rx=0 tx=0).stop 2024-09-16T15:48:04.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.401+0000 7f804c600640 1 -- 172.21.15.151:0/301762114 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f804415f910 msgr2=0x7f80441f91a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:04.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.401+0000 7f804c600640 1 --2- 172.21.15.151:0/301762114 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f804415f910 0x7f80441f91a0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f803000d950 tx=0x7f803000de10 comp rx=0 tx=0).stop 2024-09-16T15:48:04.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.401+0000 7f804c600640 1 -- 172.21.15.151:0/301762114 shutdown_connections 2024-09-16T15:48:04.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.401+0000 7f804c600640 1 --2- 172.21.15.151:0/301762114 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f80240799c0 0x7f802407be80 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:04.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.401+0000 7f804c600640 1 --2- 172.21.15.151:0/301762114 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80440c97c0 0x7f80440cfe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:04.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.401+0000 7f804c600640 1 --2- 172.21.15.151:0/301762114 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f804415f910 0x7f80441f91a0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:04.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.401+0000 7f804c600640 1 --2- 172.21.15.151:0/301762114 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f804415c350 0x7f80440cacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:04.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.401+0000 7f804c600640 1 -- 172.21.15.151:0/301762114 >> 172.21.15.151:0/301762114 conn(0x7f80440b18e0 msgr2=0x7f804415b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:04.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.401+0000 7f804c600640 1 -- 172.21.15.151:0/301762114 shutdown_connections 2024-09-16T15:48:04.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.401+0000 7f804c600640 1 -- 172.21.15.151:0/301762114 wait complete. 2024-09-16T15:48:04.422 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone59661 2024-09-16T15:48:04.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.585+0000 7fdaa3e00640 1 -- 172.21.15.151:0/2513048400 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda84003c60 msgr2=0x7fda84004060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:04.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.585+0000 7fdaa3e00640 1 --2- 172.21.15.151:0/2513048400 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda84003c60 0x7fda84004060 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fda9000c640 tx=0x7fda84005830 comp rx=0 tx=0).stop 2024-09-16T15:48:04.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.585+0000 7fdaa1e00640 1 -- 172.21.15.151:0/2513048400 reap_dead start 2024-09-16T15:48:04.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.585+0000 7fdaa3e00640 1 -- 172.21.15.151:0/2513048400 shutdown_connections 2024-09-16T15:48:04.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.585+0000 7fdaa3e00640 1 -- 172.21.15.151:0/2513048400 >> 172.21.15.151:0/2513048400 conn(0x7fda9c0b18e0 msgr2=0x7fda9c0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:04.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.585+0000 7fdaa3e00640 1 -- 172.21.15.151:0/2513048400 shutdown_connections 2024-09-16T15:48:04.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.585+0000 7fdaa3e00640 1 -- 172.21.15.151:0/2513048400 wait complete. 2024-09-16T15:48:04.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.587+0000 7fdaa3e00640 1 Processor -- start 2024-09-16T15:48:04.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.587+0000 7fdaa3e00640 1 -- start start 2024-09-16T15:48:04.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.588+0000 7fdaa3e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda84003c60 0x7fda9c156a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:04.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.588+0000 7fdaa3e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fda9c156070 0x7fda9c0cf4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:04.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.588+0000 7fdaa3e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fda9c157630 0x7fda9c157ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:04.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.588+0000 7fdaa3e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fda9c074240 con 0x7fda84003c60 2024-09-16T15:48:04.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.588+0000 7fdaa3e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fda9c0cd5e0 con 0x7fda9c156070 2024-09-16T15:48:04.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.588+0000 7fdaa3e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fda9c0743c0 con 0x7fda9c157630 2024-09-16T15:48:04.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.588+0000 7fdaa1400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda84003c60 0x7fda9c156a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:04.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.588+0000 7fdaa1400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda84003c60 0x7fda9c156a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44630/0 (socket says 172.21.15.151:44630) 2024-09-16T15:48:04.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.588+0000 7fdaa1400640 1 -- 172.21.15.151:0/945066345 learned_addr learned my addr 172.21.15.151:0/945066345 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:48:04.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.588+0000 7fdaa1e00640 1 --2- 172.21.15.151:0/945066345 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fda9c157630 0x7fda9c157ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:04.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.588+0000 7fdaa0a00640 1 --2- 172.21.15.151:0/945066345 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fda9c156070 0x7fda9c0cf4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:04.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.589+0000 7fdaa1400640 1 -- 172.21.15.151:0/945066345 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fda9c157630 msgr2=0x7fda9c157ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:04.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.589+0000 7fdaa1400640 1 --2- 172.21.15.151:0/945066345 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fda9c157630 0x7fda9c157ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:04.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.589+0000 7fdaa1400640 1 -- 172.21.15.151:0/945066345 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fda9c156070 msgr2=0x7fda9c0cf4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:04.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.589+0000 7fdaa1400640 1 --2- 172.21.15.151:0/945066345 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fda9c156070 0x7fda9c0cf4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:04.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.589+0000 7fdaa1400640 1 -- 172.21.15.151:0/945066345 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda9c1f2d60 con 0x7fda84003c60 2024-09-16T15:48:04.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.589+0000 7fdaa1400640 1 --2- 172.21.15.151:0/945066345 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda84003c60 0x7fda9c156a70 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fda9002b360 tx=0x7fda90004f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:04.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.589+0000 7fda99600640 1 -- 172.21.15.151:0/945066345 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda90039070 con 0x7fda84003c60 2024-09-16T15:48:04.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.589+0000 7fda99600640 1 -- 172.21.15.151:0/945066345 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fda9002b9a0 con 0x7fda84003c60 2024-09-16T15:48:04.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.589+0000 7fdaa3e00640 1 -- 172.21.15.151:0/945066345 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda9c1f3010 con 0x7fda84003c60 2024-09-16T15:48:04.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.589+0000 7fda99600640 1 -- 172.21.15.151:0/945066345 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda9002bdb0 con 0x7fda84003c60 2024-09-16T15:48:04.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.589+0000 7fdaa3e00640 1 -- 172.21.15.151:0/945066345 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda9c1f35d0 con 0x7fda84003c60 2024-09-16T15:48:04.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.591+0000 7fdaa3e00640 1 -- 172.21.15.151:0/945066345 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda9c156780 con 0x7fda84003c60 2024-09-16T15:48:04.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.592+0000 7fda99600640 1 -- 172.21.15.151:0/945066345 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 30) ==== 102868+0+0 (secure 0 0 0) 0x7fda90044020 con 0x7fda84003c60 2024-09-16T15:48:04.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.593+0000 7fda99600640 1 --2- 172.21.15.151:0/945066345 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fda7c0799f0 0x7fda7c07beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:04.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.593+0000 7fda99600640 1 -- 172.21.15.151:0/945066345 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5937+0+0 (secure 0 0 0) 0x7fda900b8e00 con 0x7fda84003c60 2024-09-16T15:48:04.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.593+0000 7fdaa0a00640 1 --2- 172.21.15.151:0/945066345 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fda7c0799f0 0x7fda7c07beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:04.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.594+0000 7fdaa0a00640 1 --2- 172.21.15.151:0/945066345 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fda7c0799f0 0x7fda7c07beb0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fda9c15de40 tx=0x7fda88005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:04.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.597+0000 7fda99600640 1 -- 172.21.15.151:0/945066345 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fda9c156780 con 0x7fda84003c60 2024-09-16T15:48:04.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.807+0000 7fdaa3e00640 1 -- 172.21.15.151:0/945066345 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone59661", "target": ["mon-mgr", ""]}) -- 0x7fda9c159d60 con 0x7fda7c0799f0 2024-09-16T15:48:04.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.829+0000 7fda99600640 1 -- 172.21.15.151:0/945066345 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fda9c159d60 con 0x7fda7c0799f0 2024-09-16T15:48:04.833 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.832+0000 7fdaa3e00640 1 -- 172.21.15.151:0/945066345 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fda7c0799f0 msgr2=0x7fda7c07beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:04.833 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.832+0000 7fdaa3e00640 1 --2- 172.21.15.151:0/945066345 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fda7c0799f0 0x7fda7c07beb0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fda9c15de40 tx=0x7fda88005f90 comp rx=0 tx=0).stop 2024-09-16T15:48:04.833 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.832+0000 7fdaa3e00640 1 -- 172.21.15.151:0/945066345 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda84003c60 msgr2=0x7fda9c156a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:04.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.832+0000 7fdaa3e00640 1 --2- 172.21.15.151:0/945066345 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda84003c60 0x7fda9c156a70 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fda9002b360 tx=0x7fda90004f20 comp rx=0 tx=0).stop 2024-09-16T15:48:04.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.832+0000 7fdaa3e00640 1 -- 172.21.15.151:0/945066345 shutdown_connections 2024-09-16T15:48:04.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.832+0000 7fdaa3e00640 1 --2- 172.21.15.151:0/945066345 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fda7c0799f0 0x7fda7c07beb0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:04.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.832+0000 7fdaa3e00640 1 --2- 172.21.15.151:0/945066345 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fda9c156070 0x7fda9c0cf4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:04.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.832+0000 7fdaa3e00640 1 --2- 172.21.15.151:0/945066345 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fda9c157630 0x7fda9c157ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:04.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.832+0000 7fdaa3e00640 1 --2- 172.21.15.151:0/945066345 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda84003c60 0x7fda9c156a70 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:04.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.832+0000 7fdaa3e00640 1 -- 172.21.15.151:0/945066345 >> 172.21.15.151:0/945066345 conn(0x7fda9c0b18e0 msgr2=0x7fda9c0b0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:04.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.832+0000 7fdaa3e00640 1 -- 172.21.15.151:0/945066345 shutdown_connections 2024-09-16T15:48:04.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04.832+0000 7fdaa3e00640 1 -- 172.21.15.151:0/945066345 wait complete. 2024-09-16T15:48:04.850 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-16T15:48:04.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:48:04.937 INFO:teuthology.orchestra.run.smithi151.stdout:2 2024-09-16T15:48:04.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:48:04.945 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-16T15:48:05.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.104+0000 7fdda7c00640 1 -- 172.21.15.151:0/3931561555 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdda006efa0 msgr2=0x7fdda00cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:05.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.104+0000 7fdda7c00640 1 --2- 172.21.15.151:0/3931561555 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdda006efa0 0x7fdda00cdbb0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fdd98005fd0 tx=0x7fdd9802a540 comp rx=0 tx=0).stop 2024-09-16T15:48:05.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.104+0000 7fdda7c00640 1 -- 172.21.15.151:0/3931561555 shutdown_connections 2024-09-16T15:48:05.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.104+0000 7fdda7c00640 1 --2- 172.21.15.151:0/3931561555 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdda006efa0 0x7fdda00cdbb0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:05.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.104+0000 7fdda7c00640 1 --2- 172.21.15.151:0/3931561555 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdda00cccd0 0x7fdda0073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:05.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.104+0000 7fdda7c00640 1 -- 172.21.15.151:0/3931561555 >> 172.21.15.151:0/3931561555 conn(0x7fdda00b1840 msgr2=0x7fdda00b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:05.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.104+0000 7fdda7c00640 1 -- 172.21.15.151:0/3931561555 shutdown_connections 2024-09-16T15:48:05.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.104+0000 7fdda7c00640 1 -- 172.21.15.151:0/3931561555 wait complete. 2024-09-16T15:48:05.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.106+0000 7fdda7c00640 1 Processor -- start 2024-09-16T15:48:05.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.106+0000 7fdda7c00640 1 -- start start 2024-09-16T15:48:05.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.107+0000 7fdda7c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdda006efa0 0x7fdda00ab000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:05.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.107+0000 7fdda7c00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdda00cccd0 0x7fdda01f7c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:05.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.107+0000 7fdda7c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdda0145750 0x7fdda01f81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:05.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.107+0000 7fdda7c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdda00ce480 con 0x7fdda0145750 2024-09-16T15:48:05.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.107+0000 7fdda7c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdda00ce300 con 0x7fdda006efa0 2024-09-16T15:48:05.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.107+0000 7fdda7c00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fdda00ce600 con 0x7fdda00cccd0 2024-09-16T15:48:05.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.107+0000 7fdda5c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdda0145750 0x7fdda01f81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:05.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.107+0000 7fdda5c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdda0145750 0x7fdda01f81c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44642/0 (socket says 172.21.15.151:44642) 2024-09-16T15:48:05.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.107+0000 7fdda5c00640 1 -- 172.21.15.151:0/442690007 learned_addr learned my addr 172.21.15.151:0/442690007 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:48:05.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.107+0000 7fdda5200640 1 --2- 172.21.15.151:0/442690007 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdda006efa0 0x7fdda00ab000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:05.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.108+0000 7fdd9fe00640 1 --2- 172.21.15.151:0/442690007 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdda00cccd0 0x7fdda01f7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:05.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.108+0000 7fdda5c00640 1 -- 172.21.15.151:0/442690007 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdda00cccd0 msgr2=0x7fdda01f7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:05.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.108+0000 7fdda5c00640 1 --2- 172.21.15.151:0/442690007 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdda00cccd0 0x7fdda01f7c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:05.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.108+0000 7fdda5c00640 1 -- 172.21.15.151:0/442690007 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdda006efa0 msgr2=0x7fdda00ab000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:05.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.108+0000 7fdda5c00640 1 --2- 172.21.15.151:0/442690007 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdda006efa0 0x7fdda00ab000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:05.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.108+0000 7fdda5c00640 1 -- 172.21.15.151:0/442690007 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdda00ab540 con 0x7fdda0145750 2024-09-16T15:48:05.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.108+0000 7fdda5c00640 1 --2- 172.21.15.151:0/442690007 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdda0145750 0x7fdda01f81c0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fdd980099f0 tx=0x7fdd980038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:05.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.108+0000 7fdd9cc00640 1 -- 172.21.15.151:0/442690007 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9803f070 con 0x7fdda0145750 2024-09-16T15:48:05.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.108+0000 7fdd9cc00640 1 -- 172.21.15.151:0/442690007 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdd980047a0 con 0x7fdda0145750 2024-09-16T15:48:05.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.108+0000 7fdda7c00640 1 -- 172.21.15.151:0/442690007 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdda01f8b00 con 0x7fdda0145750 2024-09-16T15:48:05.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.109+0000 7fdd9cc00640 1 -- 172.21.15.151:0/442690007 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd98035030 con 0x7fdda0145750 2024-09-16T15:48:05.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.109+0000 7fdda7c00640 1 -- 172.21.15.151:0/442690007 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdda00adc20 con 0x7fdda0145750 2024-09-16T15:48:05.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.110+0000 7fdda7c00640 1 -- 172.21.15.151:0/442690007 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdda015f3d0 con 0x7fdda0145750 2024-09-16T15:48:05.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.111+0000 7fdd9cc00640 1 -- 172.21.15.151:0/442690007 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 30) ==== 102868+0+0 (secure 0 0 0) 0x7fdd98042070 con 0x7fdda0145750 2024-09-16T15:48:05.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.112+0000 7fdd9cc00640 1 --2- 172.21.15.151:0/442690007 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdd800799f0 0x7fdd8007beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:05.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.112+0000 7fdd9cc00640 1 -- 172.21.15.151:0/442690007 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5937+0+0 (secure 0 0 0) 0x7fdd980ba860 con 0x7fdda0145750 2024-09-16T15:48:05.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.112+0000 7fdda5200640 1 --2- 172.21.15.151:0/442690007 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdd800799f0 0x7fdd8007beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:05.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.113+0000 7fdda5200640 1 --2- 172.21.15.151:0/442690007 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdd800799f0 0x7fdd8007beb0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fdd90001e20 tx=0x7fdd90009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:05.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.116+0000 7fdd9cc00640 1 -- 172.21.15.151:0/442690007 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fdd98083f80 con 0x7fdda0145750 2024-09-16T15:48:05.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.308+0000 7fdda7c00640 1 -- 172.21.15.151:0/442690007 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7fdda0069a10 con 0x7fdda0145750 2024-09-16T15:48:05.311 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.309+0000 7fdd9cc00640 1 -- 172.21.15.151:0/442690007 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v39)=0 v39) ==== 131+0+0 (secure 0 0 0) 0x7fdd980889b0 con 0x7fdda0145750 2024-09-16T15:48:05.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.311+0000 7fdda7c00640 1 -- 172.21.15.151:0/442690007 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdd800799f0 msgr2=0x7fdd8007beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:05.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.311+0000 7fdda7c00640 1 --2- 172.21.15.151:0/442690007 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdd800799f0 0x7fdd8007beb0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fdd90001e20 tx=0x7fdd90009040 comp rx=0 tx=0).stop 2024-09-16T15:48:05.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.311+0000 7fdda7c00640 1 -- 172.21.15.151:0/442690007 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdda0145750 msgr2=0x7fdda01f81c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:05.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.311+0000 7fdda7c00640 1 --2- 172.21.15.151:0/442690007 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdda0145750 0x7fdda01f81c0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fdd980099f0 tx=0x7fdd980038a0 comp rx=0 tx=0).stop 2024-09-16T15:48:05.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.312+0000 7fdda7c00640 1 -- 172.21.15.151:0/442690007 shutdown_connections 2024-09-16T15:48:05.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.312+0000 7fdda7c00640 1 --2- 172.21.15.151:0/442690007 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdd800799f0 0x7fdd8007beb0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:05.315 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.312+0000 7fdda7c00640 1 --2- 172.21.15.151:0/442690007 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdda0145750 0x7fdda01f81c0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:05.315 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.312+0000 7fdda7c00640 1 --2- 172.21.15.151:0/442690007 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdda00cccd0 0x7fdda01f7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:05.315 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.312+0000 7fdda7c00640 1 --2- 172.21.15.151:0/442690007 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdda006efa0 0x7fdda00ab000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:05.315 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.312+0000 7fdda7c00640 1 -- 172.21.15.151:0/442690007 >> 172.21.15.151:0/442690007 conn(0x7fdda00b1840 msgr2=0x7fdda00aa2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:05.315 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.312+0000 7fdda7c00640 1 -- 172.21.15.151:0/442690007 shutdown_connections 2024-09-16T15:48:05.315 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.312+0000 7fdda7c00640 1 -- 172.21.15.151:0/442690007 wait complete. 2024-09-16T15:48:05.330 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-16T15:48:05.330 DEBUG:teuthology.orchestra.run.smithi151:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:48:05.410 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:48:05.512 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-16T15:48:05.512 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:48:05.512 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:48:05.512 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:48:05.548 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:48:05.548 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:48:05.624 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-16T15:48:05.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.821+0000 7fc853c00640 1 -- 172.21.15.151:0/118206046 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc84c15f5a0 msgr2=0x7fc84c0b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:05.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.821+0000 7fc853c00640 1 --2- 172.21.15.151:0/118206046 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc84c15f5a0 0x7fc84c0b2440 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fc83c005fd0 tx=0x7fc83c02a540 comp rx=0 tx=0).stop 2024-09-16T15:48:05.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.821+0000 7fc853c00640 1 -- 172.21.15.151:0/118206046 shutdown_connections 2024-09-16T15:48:05.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.821+0000 7fc853c00640 1 --2- 172.21.15.151:0/118206046 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc84c0ccf00 0x7fc84c0acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:05.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.821+0000 7fc853c00640 1 --2- 172.21.15.151:0/118206046 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc84c15f5a0 0x7fc84c0b2440 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:05.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.821+0000 7fc853c00640 1 -- 172.21.15.151:0/118206046 >> 172.21.15.151:0/118206046 conn(0x7fc84c0b1950 msgr2=0x7fc84c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:05.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.821+0000 7fc853c00640 1 -- 172.21.15.151:0/118206046 shutdown_connections 2024-09-16T15:48:05.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.821+0000 7fc853c00640 1 -- 172.21.15.151:0/118206046 wait complete. 2024-09-16T15:48:05.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.823+0000 7fc853c00640 1 Processor -- start 2024-09-16T15:48:05.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.823+0000 7fc853c00640 1 -- start start 2024-09-16T15:48:05.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.824+0000 7fc853c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc84c0ad270 0x7fc84c14c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:05.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.824+0000 7fc853c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc84c0ccf00 0x7fc84c14e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:05.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.824+0000 7fc853c00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc84c15f5a0 0x7fc84c14eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:05.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.824+0000 7fc853c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc84c06f2e0 con 0x7fc84c0ccf00 2024-09-16T15:48:05.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.824+0000 7fc853c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc84c06f160 con 0x7fc84c0ad270 2024-09-16T15:48:05.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.824+0000 7fc853c00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fc84c06f460 con 0x7fc84c15f5a0 2024-09-16T15:48:05.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.824+0000 7fc84be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc84c0ccf00 0x7fc84c14e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:05.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.824+0000 7fc84be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc84c0ccf00 0x7fc84c14e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44670/0 (socket says 172.21.15.151:44670) 2024-09-16T15:48:05.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.824+0000 7fc84be00640 1 -- 172.21.15.151:0/1492939884 learned_addr learned my addr 172.21.15.151:0/1492939884 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:48:05.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.825+0000 7fc851c00640 1 --2- 172.21.15.151:0/1492939884 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc84c15f5a0 0x7fc84c14eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:05.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.825+0000 7fc851200640 1 --2- 172.21.15.151:0/1492939884 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc84c0ad270 0x7fc84c14c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:05.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.825+0000 7fc84be00640 1 -- 172.21.15.151:0/1492939884 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc84c15f5a0 msgr2=0x7fc84c14eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:05.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.825+0000 7fc84be00640 1 --2- 172.21.15.151:0/1492939884 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc84c15f5a0 0x7fc84c14eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:05.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.825+0000 7fc84be00640 1 -- 172.21.15.151:0/1492939884 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc84c0ad270 msgr2=0x7fc84c14c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:05.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.825+0000 7fc84be00640 1 --2- 172.21.15.151:0/1492939884 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc84c0ad270 0x7fc84c14c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:05.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.825+0000 7fc84be00640 1 -- 172.21.15.151:0/1492939884 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc84c14f110 con 0x7fc84c0ccf00 2024-09-16T15:48:05.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.825+0000 7fc84be00640 1 --2- 172.21.15.151:0/1492939884 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc84c0ccf00 0x7fc84c14e5c0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fc83800d950 tx=0x7fc83800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:05.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.825+0000 7fc848c00640 1 -- 172.21.15.151:0/1492939884 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc838017070 con 0x7fc84c0ccf00 2024-09-16T15:48:05.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.825+0000 7fc848c00640 1 -- 172.21.15.151:0/1492939884 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc838009d20 con 0x7fc84c0ccf00 2024-09-16T15:48:05.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.825+0000 7fc853c00640 1 -- 172.21.15.151:0/1492939884 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc84c14f3f0 con 0x7fc84c0ccf00 2024-09-16T15:48:05.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.825+0000 7fc848c00640 1 -- 172.21.15.151:0/1492939884 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8380106c0 con 0x7fc84c0ccf00 2024-09-16T15:48:05.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.825+0000 7fc853c00640 1 -- 172.21.15.151:0/1492939884 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc84c150950 con 0x7fc84c0ccf00 2024-09-16T15:48:05.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.827+0000 7fc853c00640 1 -- 172.21.15.151:0/1492939884 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc84c06f990 con 0x7fc84c0ccf00 2024-09-16T15:48:05.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.828+0000 7fc848c00640 1 -- 172.21.15.151:0/1492939884 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 30) ==== 102868+0+0 (secure 0 0 0) 0x7fc838010860 con 0x7fc84c0ccf00 2024-09-16T15:48:05.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.829+0000 7fc848c00640 1 --2- 172.21.15.151:0/1492939884 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc82c0799c0 0x7fc82c07be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:05.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.829+0000 7fc848c00640 1 -- 172.21.15.151:0/1492939884 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5937+0+0 (secure 0 0 0) 0x7fc838099c50 con 0x7fc84c0ccf00 2024-09-16T15:48:05.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.829+0000 7fc851200640 1 --2- 172.21.15.151:0/1492939884 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc82c0799c0 0x7fc82c07be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:05.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.830+0000 7fc851200640 1 --2- 172.21.15.151:0/1492939884 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc82c0799c0 0x7fc82c07be80 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fc83c033000 tx=0x7fc83c004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:05.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:05.833+0000 7fc848c00640 1 -- 172.21.15.151:0/1492939884 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc8380639c0 con 0x7fc84c0ccf00 2024-09-16T15:48:06.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:06.032+0000 7fc853c00640 1 -- 172.21.15.151:0/1492939884 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc84c15ed50 con 0x7fc84c0ccf00 2024-09-16T15:48:06.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:06.032+0000 7fc848c00640 1 -- 172.21.15.151:0/1492939884 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v79) ==== 80+0+0 (secure 0 0 0) 0x7fc8380632c0 con 0x7fc84c0ccf00 2024-09-16T15:48:06.034 INFO:teuthology.orchestra.run.smithi151.stderr:listed 0 entries 2024-09-16T15:48:06.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:06.035+0000 7fc853c00640 1 -- 172.21.15.151:0/1492939884 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc82c0799c0 msgr2=0x7fc82c07be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:06.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:06.035+0000 7fc853c00640 1 --2- 172.21.15.151:0/1492939884 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc82c0799c0 0x7fc82c07be80 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fc83c033000 tx=0x7fc83c004730 comp rx=0 tx=0).stop 2024-09-16T15:48:06.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:06.035+0000 7fc853c00640 1 -- 172.21.15.151:0/1492939884 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc84c0ccf00 msgr2=0x7fc84c14e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:06.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:06.035+0000 7fc853c00640 1 --2- 172.21.15.151:0/1492939884 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc84c0ccf00 0x7fc84c14e5c0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fc83800d950 tx=0x7fc83800de10 comp rx=0 tx=0).stop 2024-09-16T15:48:06.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:06.035+0000 7fc853c00640 1 -- 172.21.15.151:0/1492939884 shutdown_connections 2024-09-16T15:48:06.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:06.035+0000 7fc853c00640 1 --2- 172.21.15.151:0/1492939884 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc82c0799c0 0x7fc82c07be80 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:06.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:06.035+0000 7fc853c00640 1 --2- 172.21.15.151:0/1492939884 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc84c15f5a0 0x7fc84c14eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:06.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:06.035+0000 7fc853c00640 1 --2- 172.21.15.151:0/1492939884 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc84c0ccf00 0x7fc84c14e5c0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:06.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:06.035+0000 7fc853c00640 1 --2- 172.21.15.151:0/1492939884 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc84c0ad270 0x7fc84c14c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:06.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:06.035+0000 7fc853c00640 1 -- 172.21.15.151:0/1492939884 >> 172.21.15.151:0/1492939884 conn(0x7fc84c0b1950 msgr2=0x7fc84c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:06.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:06.035+0000 7fc853c00640 1 -- 172.21.15.151:0/1492939884 shutdown_connections 2024-09-16T15:48:06.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:06.035+0000 7fc853c00640 1 -- 172.21.15.151:0/1492939884 wait complete. 2024-09-16T15:48:06.053 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-16T15:48:06.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:48:06.053 DEBUG:teuthology.orchestra.run.smithi151:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-16T15:48:06.127 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi151.front.sepia.ceph.com 2024-09-16T15:48:06.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:48:06.127 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-16T15:48:06.190 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:48:06.190 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-16T15:48:06.191 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:48:06.228 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:48:06.228 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:48:06.228 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:48:06.228 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:48:06.264 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T15:48:06.264 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:48:06.264 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-16T15:48:06.264 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:48:06.304 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:48:06.304 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:48:06.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:06.550+0000 7f6ba1a00640 1 -- 172.21.15.151:0/3808092747 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6b9c1438b0 msgr2=0x7f6b9c0cc5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:06.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:06.550+0000 7f6ba1a00640 1 --2- 172.21.15.151:0/3808092747 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6b9c1438b0 0x7f6b9c0cc5f0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c009830 tx=0x7f6b8c02a4f0 comp rx=0 tx=0).stop 2024-09-16T15:48:06.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:06.550+0000 7f6ba1a00640 1 -- 172.21.15.151:0/3808092747 shutdown_connections 2024-09-16T15:48:06.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:06.550+0000 7f6ba1a00640 1 --2- 172.21.15.151:0/3808092747 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6b9c1438b0 0x7f6b9c0cc5f0 secure :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c009830 tx=0x7f6b8c02a4f0 comp rx=0 tx=0).stop 2024-09-16T15:48:06.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:06.550+0000 7f6ba1a00640 1 -- 172.21.15.151:0/3808092747 >> 172.21.15.151:0/3808092747 conn(0x7f6b9c0b1950 msgr2=0x7f6b9c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:06.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:06.551+0000 7f6ba1a00640 1 -- 172.21.15.151:0/3808092747 shutdown_connections 2024-09-16T15:48:06.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:06.551+0000 7f6ba1a00640 1 -- 172.21.15.151:0/3808092747 wait complete. 2024-09-16T15:48:06.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:06.554+0000 7f6ba1a00640 1 Processor -- start 2024-09-16T15:48:06.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:06.554+0000 7f6ba1a00640 1 -- start start 2024-09-16T15:48:06.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:06.556+0000 7f6ba1a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b9c06ff60 0x7f6b9c0cb780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:06.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:06.556+0000 7f6ba1a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b9c0ccb30 0x7f6b9c201ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:06.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:06.556+0000 7f6ba1a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6b9c202010 0x7f6b9c1e3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:06.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:06.556+0000 7f6ba1a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9c0b4610 con 0x7f6b9c202010 2024-09-16T15:48:06.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:06.556+0000 7f6ba1a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9c070b30 con 0x7f6b9c06ff60 2024-09-16T15:48:06.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:06.556+0000 7f6ba1a00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f6b9c0b4790 con 0x7f6b9c0ccb30 2024-09-16T15:48:06.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:06.556+0000 7f6b9a000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b9c0ccb30 0x7f6b9c201ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:06.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:06.556+0000 7f6b9aa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b9c06ff60 0x7f6b9c0cb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:06.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:06.556+0000 7f6b9a000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b9c0ccb30 0x7f6b9c201ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.151:46212/0 (socket says 172.21.15.151:46212) 2024-09-16T15:48:06.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:06.556+0000 7f6b9aa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b9c06ff60 0x7f6b9c0cb780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.151:53538/0 (socket says 172.21.15.151:53538) 2024-09-16T15:48:06.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:06.556+0000 7f6b9a000640 1 -- 172.21.15.151:0/1147386576 learned_addr learned my addr 172.21.15.151:0/1147386576 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:48:06.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:06.556+0000 7f6b9b400640 1 --2- 172.21.15.151:0/1147386576 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6b9c202010 0x7f6b9c1e3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:06.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:06.557+0000 7f6b9a000640 1 -- 172.21.15.151:0/1147386576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b9c06ff60 msgr2=0x7f6b9c0cb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:06.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:06.557+0000 7f6b9a000640 1 --2- 172.21.15.151:0/1147386576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b9c06ff60 0x7f6b9c0cb780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:06.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:06.557+0000 7f6b9a000640 1 -- 172.21.15.151:0/1147386576 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6b9c202010 msgr2=0x7f6b9c1e3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:06.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:06.557+0000 7f6b9a000640 1 --2- 172.21.15.151:0/1147386576 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6b9c202010 0x7f6b9c1e3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:06.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:06.557+0000 7f6b9a000640 1 -- 172.21.15.151:0/1147386576 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b9c202520 con 0x7f6b9c0ccb30 2024-09-16T15:48:06.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:06.557+0000 7f6b9b400640 1 --2- 172.21.15.151:0/1147386576 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6b9c202010 0x7f6b9c1e3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:48:06.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:06.557+0000 7f6b9a000640 1 --2- 172.21.15.151:0/1147386576 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b9c0ccb30 0x7f6b9c201ad0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6b8800c730 tx=0x7f6b8800cbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:06.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:06.557+0000 7f6b92a00640 1 -- 172.21.15.151:0/1147386576 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b88017070 con 0x7f6b9c0ccb30 2024-09-16T15:48:06.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:06.557+0000 7f6b92a00640 1 -- 172.21.15.151:0/1147386576 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6b88005380 con 0x7f6b9c0ccb30 2024-09-16T15:48:06.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:06.557+0000 7f6b92a00640 1 -- 172.21.15.151:0/1147386576 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8800a8a0 con 0x7f6b9c0ccb30 2024-09-16T15:48:06.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:06.558+0000 7f6ba1a00640 1 -- 172.21.15.151:0/1147386576 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b9c202800 con 0x7f6b9c0ccb30 2024-09-16T15:48:06.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:06.558+0000 7f6ba1a00640 1 -- 172.21.15.151:0/1147386576 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b9c0c5040 con 0x7f6b9c0ccb30 2024-09-16T15:48:06.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:06.559+0000 7f6ba1a00640 1 -- 172.21.15.151:0/1147386576 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b9c0b4c60 con 0x7f6b9c0ccb30 2024-09-16T15:48:06.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:06.561+0000 7f6b92a00640 1 -- 172.21.15.151:0/1147386576 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 30) ==== 102868+0+0 (secure 0 0 0) 0x7f6b8801a070 con 0x7f6b9c0ccb30 2024-09-16T15:48:06.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:06.563+0000 7f6b92a00640 1 --2- 172.21.15.151:0/1147386576 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6b7c0799c0 0x7f6b7c07be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:06.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:06.563+0000 7f6b92a00640 1 -- 172.21.15.151:0/1147386576 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(80..80 src has 1..80) ==== 5909+0+0 (secure 0 0 0) 0x7f6b88098d00 con 0x7f6b9c0ccb30 2024-09-16T15:48:06.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:06.564+0000 7f6b9aa00640 1 --2- 172.21.15.151:0/1147386576 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6b7c0799c0 0x7f6b7c07be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:06.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:06.565+0000 7f6b9aa00640 1 --2- 172.21.15.151:0/1147386576 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6b7c0799c0 0x7f6b7c07be80 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c034000 tx=0x7f6b8c004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:06.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:06.569+0000 7f6b92a00640 1 -- 172.21.15.151:0/1147386576 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6b9c0cdae0 con 0x7f6b9c0ccb30 2024-09-16T15:48:06.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:06.944+0000 7f6ba1a00640 1 -- 172.21.15.151:0/1147386576 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6b9c0cd430 con 0x7f6b9c0ccb30 2024-09-16T15:48:06.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:06.945+0000 7f6b92a00640 1 -- 172.21.15.151:0/1147386576 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5234 (secure 0 0 0) 0x7f6b880624e0 con 0x7f6b9c0ccb30 2024-09-16T15:48:06.948 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:48:06.948 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":47,"btime":"2024-09-16T15:47:01:291406+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5705,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3291846596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3291846596},{"type":"v1","addr":"172.21.15.167:6837","nonce":3291846596}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6175,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/2504229431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":2504229431},{"type":"v1","addr":"172.21.15.151:6834","nonce":2504229431}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6181,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/464376104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":464376104},{"type":"v1","addr":"172.21.15.167:6836","nonce":464376104}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:46:52.671228+0000","modified":"2024-09-16T15:46:55.692251+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6456},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6456":{"gid":6456,"name":"a","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.151:6835/3158835036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":3158835036},{"type":"v1","addr":"172.21.15.151:6835","nonce":3158835036}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6456,"qdb_cluster":[6456]},"id":5}]} 2024-09-16T15:48:06.948 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 47 2024-09-16T15:48:06.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:06.948+0000 7f6ba1a00640 1 -- 172.21.15.151:0/1147386576 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6b7c0799c0 msgr2=0x7f6b7c07be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:06.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:06.948+0000 7f6ba1a00640 1 --2- 172.21.15.151:0/1147386576 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6b7c0799c0 0x7f6b7c07be80 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c034000 tx=0x7f6b8c004730 comp rx=0 tx=0).stop 2024-09-16T15:48:06.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:06.948+0000 7f6ba1a00640 1 -- 172.21.15.151:0/1147386576 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b9c0ccb30 msgr2=0x7f6b9c201ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:06.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:06.948+0000 7f6ba1a00640 1 --2- 172.21.15.151:0/1147386576 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b9c0ccb30 0x7f6b9c201ad0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6b8800c730 tx=0x7f6b8800cbf0 comp rx=0 tx=0).stop 2024-09-16T15:48:06.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:06.948+0000 7f6ba1a00640 1 -- 172.21.15.151:0/1147386576 shutdown_connections 2024-09-16T15:48:06.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:06.948+0000 7f6ba1a00640 1 --2- 172.21.15.151:0/1147386576 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6b7c0799c0 0x7f6b7c07be80 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:06.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:06.948+0000 7f6ba1a00640 1 --2- 172.21.15.151:0/1147386576 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6b9c202010 0x7f6b9c1e3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:06.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:06.948+0000 7f6ba1a00640 1 --2- 172.21.15.151:0/1147386576 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b9c0ccb30 0x7f6b9c201ad0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:06.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:06.948+0000 7f6ba1a00640 1 --2- 172.21.15.151:0/1147386576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b9c06ff60 0x7f6b9c0cb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:06.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:06.948+0000 7f6ba1a00640 1 -- 172.21.15.151:0/1147386576 >> 172.21.15.151:0/1147386576 conn(0x7f6b9c0b1950 msgr2=0x7f6b9c0cbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:06.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:06.948+0000 7f6ba1a00640 1 -- 172.21.15.151:0/1147386576 shutdown_connections 2024-09-16T15:48:06.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:06.948+0000 7f6ba1a00640 1 -- 172.21.15.151:0/1147386576 wait complete. 2024-09-16T15:48:06.966 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:48:07.125 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.123+0000 7f3787800640 1 -- 172.21.15.151:0/1745424509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37800ccf00 msgr2=0x7f37800b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:07.125 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.123+0000 7f3787800640 1 --2- 172.21.15.151:0/1745424509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37800ccf00 0x7f37800b2440 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f376c009830 tx=0x7f376c02a530 comp rx=0 tx=0).stop 2024-09-16T15:48:07.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.123+0000 7f3787800640 1 -- 172.21.15.151:0/1745424509 shutdown_connections 2024-09-16T15:48:07.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.123+0000 7f3787800640 1 --2- 172.21.15.151:0/1745424509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37800ccf00 0x7f37800b2440 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:07.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.123+0000 7f3787800640 1 -- 172.21.15.151:0/1745424509 >> 172.21.15.151:0/1745424509 conn(0x7f37800b1950 msgr2=0x7f37800b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:07.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.123+0000 7f3787800640 1 -- 172.21.15.151:0/1745424509 shutdown_connections 2024-09-16T15:48:07.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.123+0000 7f3787800640 1 -- 172.21.15.151:0/1745424509 wait complete. 2024-09-16T15:48:07.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.125+0000 7f3787800640 1 Processor -- start 2024-09-16T15:48:07.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.125+0000 7f3787800640 1 -- start start 2024-09-16T15:48:07.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.126+0000 7f3787800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3780073fb0 0x7f37801f2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:07.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.126+0000 7f3787800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f37800ccf00 0x7f3780156090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:07.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.126+0000 7f3787800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f378015f5a0 0x7f37801565d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:07.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.126+0000 7f3787800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f378006f9f0 con 0x7f3780073fb0 2024-09-16T15:48:07.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.126+0000 7f3787800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f378006f870 con 0x7f378015f5a0 2024-09-16T15:48:07.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.126+0000 7f3787800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f3780074420 con 0x7f37800ccf00 2024-09-16T15:48:07.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.126+0000 7f3784e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3780073fb0 0x7f37801f2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:07.130 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.126+0000 7f3784e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3780073fb0 0x7f37801f2fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44690/0 (socket says 172.21.15.151:44690) 2024-09-16T15:48:07.130 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.126+0000 7f3784e00640 1 -- 172.21.15.151:0/4153232201 learned_addr learned my addr 172.21.15.151:0/4153232201 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:48:07.130 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.126+0000 7f377fe00640 1 --2- 172.21.15.151:0/4153232201 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f37800ccf00 0x7f3780156090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:07.130 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.126+0000 7f3785800640 1 --2- 172.21.15.151:0/4153232201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f378015f5a0 0x7f37801565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:07.130 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.126+0000 7f3784e00640 1 -- 172.21.15.151:0/4153232201 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f37800ccf00 msgr2=0x7f3780156090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:07.130 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.126+0000 7f3784e00640 1 --2- 172.21.15.151:0/4153232201 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f37800ccf00 0x7f3780156090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:07.130 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.126+0000 7f3784e00640 1 -- 172.21.15.151:0/4153232201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f378015f5a0 msgr2=0x7f37801565d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:07.130 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.126+0000 7f3784e00640 1 --2- 172.21.15.151:0/4153232201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f378015f5a0 0x7f37801565d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:07.130 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.126+0000 7f3784e00640 1 -- 172.21.15.151:0/4153232201 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37801f3520 con 0x7f3780073fb0 2024-09-16T15:48:07.130 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.127+0000 7f3784e00640 1 --2- 172.21.15.151:0/4153232201 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3780073fb0 0x7f37801f2fe0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f377000c7b0 tx=0x7f377000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:07.130 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.127+0000 7f377cc00640 1 -- 172.21.15.151:0/4153232201 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3770003b70 con 0x7f3780073fb0 2024-09-16T15:48:07.130 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.127+0000 7f377cc00640 1 -- 172.21.15.151:0/4153232201 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3770003d10 con 0x7f3780073fb0 2024-09-16T15:48:07.130 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.127+0000 7f3787800640 1 -- 172.21.15.151:0/4153232201 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37800cfdd0 con 0x7f3780073fb0 2024-09-16T15:48:07.131 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.127+0000 7f377cc00640 1 -- 172.21.15.151:0/4153232201 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f377000aab0 con 0x7f3780073fb0 2024-09-16T15:48:07.131 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.127+0000 7f3787800640 1 -- 172.21.15.151:0/4153232201 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37800d0390 con 0x7f3780073fb0 2024-09-16T15:48:07.131 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.128+0000 7f3787800640 1 -- 172.21.15.151:0/4153232201 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37800b2440 con 0x7f3780073fb0 2024-09-16T15:48:07.131 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.129+0000 7f377cc00640 1 -- 172.21.15.151:0/4153232201 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 30) ==== 102868+0+0 (secure 0 0 0) 0x7f377000ac50 con 0x7f3780073fb0 2024-09-16T15:48:07.133 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.130+0000 7f377cc00640 1 --2- 172.21.15.151:0/4153232201 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f37600799c0 0x7f376007be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:07.133 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.130+0000 7f377cc00640 1 -- 172.21.15.151:0/4153232201 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5909+0+0 (secure 0 0 0) 0x7f3770099610 con 0x7f3780073fb0 2024-09-16T15:48:07.133 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.131+0000 7f377fe00640 1 --2- 172.21.15.151:0/4153232201 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f37600799c0 0x7f376007be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:07.133 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.131+0000 7f377fe00640 1 --2- 172.21.15.151:0/4153232201 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f37600799c0 0x7f376007be80 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f376c034000 tx=0x7f376c004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:07.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.134+0000 7f377cc00640 1 -- 172.21.15.151:0/4153232201 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f3770062cb0 con 0x7f3780073fb0 2024-09-16T15:48:07.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.389+0000 7f3787800640 1 -- 172.21.15.151:0/4153232201 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f37800b1240 con 0x7f3780073fb0 2024-09-16T15:48:07.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.390+0000 7f377cc00640 1 -- 172.21.15.151:0/4153232201 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5234 (secure 0 0 0) 0x7f37700676e0 con 0x7f3780073fb0 2024-09-16T15:48:07.392 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:48:07.392 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":47,"btime":"2024-09-16T15:47:01:291406+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5705,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3291846596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3291846596},{"type":"v1","addr":"172.21.15.167:6837","nonce":3291846596}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6175,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/2504229431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":2504229431},{"type":"v1","addr":"172.21.15.151:6834","nonce":2504229431}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6181,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/464376104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":464376104},{"type":"v1","addr":"172.21.15.167:6836","nonce":464376104}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:46:52.671228+0000","modified":"2024-09-16T15:46:55.692251+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6456},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6456":{"gid":6456,"name":"a","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.151:6835/3158835036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":3158835036},{"type":"v1","addr":"172.21.15.151:6835","nonce":3158835036}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6456,"qdb_cluster":[6456]},"id":5}]} 2024-09-16T15:48:07.392 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 47 2024-09-16T15:48:07.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.394+0000 7f3787800640 1 -- 172.21.15.151:0/4153232201 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f37600799c0 msgr2=0x7f376007be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:07.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.394+0000 7f3787800640 1 --2- 172.21.15.151:0/4153232201 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f37600799c0 0x7f376007be80 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f376c034000 tx=0x7f376c004730 comp rx=0 tx=0).stop 2024-09-16T15:48:07.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.394+0000 7f3787800640 1 -- 172.21.15.151:0/4153232201 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3780073fb0 msgr2=0x7f37801f2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:07.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.394+0000 7f3787800640 1 --2- 172.21.15.151:0/4153232201 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3780073fb0 0x7f37801f2fe0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f377000c7b0 tx=0x7f377000cc70 comp rx=0 tx=0).stop 2024-09-16T15:48:07.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.394+0000 7f3787800640 1 -- 172.21.15.151:0/4153232201 shutdown_connections 2024-09-16T15:48:07.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.394+0000 7f3787800640 1 --2- 172.21.15.151:0/4153232201 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f37600799c0 0x7f376007be80 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:07.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.394+0000 7f3787800640 1 --2- 172.21.15.151:0/4153232201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f378015f5a0 0x7f37801565d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:07.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.394+0000 7f3787800640 1 --2- 172.21.15.151:0/4153232201 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f37800ccf00 0x7f3780156090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:07.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.394+0000 7f3787800640 1 --2- 172.21.15.151:0/4153232201 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3780073fb0 0x7f37801f2fe0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:07.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.394+0000 7f3787800640 1 -- 172.21.15.151:0/4153232201 >> 172.21.15.151:0/4153232201 conn(0x7f37800b1950 msgr2=0x7f3780075b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:07.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.395+0000 7f3787800640 1 -- 172.21.15.151:0/4153232201 shutdown_connections 2024-09-16T15:48:07.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.396+0000 7f3787800640 1 -- 172.21.15.151:0/4153232201 wait complete. 2024-09-16T15:48:07.422 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:48:07.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.675+0000 7f561d800640 1 -- 172.21.15.151:0/3138563371 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56181438b0 msgr2=0x7f56180cc5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:07.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.675+0000 7f561d800640 1 --2- 172.21.15.151:0/3138563371 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56181438b0 0x7f56180cc5f0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f5600005fd0 tx=0x7f560002a5d0 comp rx=0 tx=0).stop 2024-09-16T15:48:07.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.675+0000 7f561d800640 1 -- 172.21.15.151:0/3138563371 shutdown_connections 2024-09-16T15:48:07.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.675+0000 7f561d800640 1 --2- 172.21.15.151:0/3138563371 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f56180ccb30 0x7f561806f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:07.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.675+0000 7f561d800640 1 --2- 172.21.15.151:0/3138563371 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56181438b0 0x7f56180cc5f0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:07.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.675+0000 7f561d800640 1 -- 172.21.15.151:0/3138563371 >> 172.21.15.151:0/3138563371 conn(0x7f56180b1950 msgr2=0x7f56180b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:07.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.676+0000 7f561d800640 1 -- 172.21.15.151:0/3138563371 shutdown_connections 2024-09-16T15:48:07.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.677+0000 7f561d800640 1 -- 172.21.15.151:0/3138563371 wait complete. 2024-09-16T15:48:07.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.680+0000 7f561d800640 1 Processor -- start 2024-09-16T15:48:07.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.680+0000 7f561d800640 1 -- start start 2024-09-16T15:48:07.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.684+0000 7f561d800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f561806fd90 0x7f5618142b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:07.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.684+0000 7f561d800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56180ccb30 0x7f5618158f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:07.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.685+0000 7f5616a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f561806fd90 0x7f5618142b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:07.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.685+0000 7f5616a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f561806fd90 0x7f5618142b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.151:46242/0 (socket says 172.21.15.151:46242) 2024-09-16T15:48:07.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.685+0000 7f5616a00640 1 -- 172.21.15.151:0/3177473070 learned_addr learned my addr 172.21.15.151:0/3177473070 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:48:07.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.685+0000 7f5616000640 1 --2- 172.21.15.151:0/3177473070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56180ccb30 0x7f5618158f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:07.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.685+0000 7f561d800640 1 --2- 172.21.15.151:0/3177473070 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56181438b0 0x7f5618139ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:07.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.686+0000 7f561d800640 1 -- 172.21.15.151:0/3177473070 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5618070920 con 0x7f56181438b0 2024-09-16T15:48:07.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.686+0000 7f561d800640 1 -- 172.21.15.151:0/3177473070 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f56180b4f60 con 0x7f56180ccb30 2024-09-16T15:48:07.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.686+0000 7f561d800640 1 -- 172.21.15.151:0/3177473070 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f5618070aa0 con 0x7f561806fd90 2024-09-16T15:48:07.689 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.687+0000 7f5616a00640 1 -- 172.21.15.151:0/3177473070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56180ccb30 msgr2=0x7f5618158f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:07.690 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.688+0000 7f5616a00640 1 --2- 172.21.15.151:0/3177473070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56180ccb30 0x7f5618158f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:07.690 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.688+0000 7f5617400640 1 --2- 172.21.15.151:0/3177473070 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56181438b0 0x7f5618139ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:07.690 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.688+0000 7f5616a00640 1 -- 172.21.15.151:0/3177473070 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56181438b0 msgr2=0x7f5618139ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:07.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.688+0000 7f5616a00640 1 --2- 172.21.15.151:0/3177473070 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56181438b0 0x7f5618139ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:07.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.688+0000 7f5616a00640 1 -- 172.21.15.151:0/3177473070 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5618159440 con 0x7f561806fd90 2024-09-16T15:48:07.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.688+0000 7f5616000640 1 --2- 172.21.15.151:0/3177473070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56180ccb30 0x7f5618158f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:48:07.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.688+0000 7f5617400640 1 --2- 172.21.15.151:0/3177473070 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56181438b0 0x7f5618139ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:48:07.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.689+0000 7f5616a00640 1 --2- 172.21.15.151:0/3177473070 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f561806fd90 0x7f5618142b70 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5600034040 tx=0x7f56000035f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:07.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.689+0000 7f560ea00640 1 -- 172.21.15.151:0/3177473070 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f560003e030 con 0x7f561806fd90 2024-09-16T15:48:07.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.689+0000 7f560ea00640 1 -- 172.21.15.151:0/3177473070 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5600003ae0 con 0x7f561806fd90 2024-09-16T15:48:07.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.689+0000 7f560ea00640 1 -- 172.21.15.151:0/3177473070 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5600035040 con 0x7f561806fd90 2024-09-16T15:48:07.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.690+0000 7f561d800640 1 -- 172.21.15.151:0/3177473070 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5618139a00 con 0x7f561806fd90 2024-09-16T15:48:07.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.690+0000 7f561d800640 1 -- 172.21.15.151:0/3177473070 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f56180b0140 con 0x7f561806fd90 2024-09-16T15:48:07.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.693+0000 7f561d800640 1 -- 172.21.15.151:0/3177473070 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f561806f850 con 0x7f561806fd90 2024-09-16T15:48:07.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.695+0000 7f560ea00640 1 -- 172.21.15.151:0/3177473070 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 30) ==== 102868+0+0 (secure 0 0 0) 0x7f5600041030 con 0x7f561806fd90 2024-09-16T15:48:07.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.697+0000 7f560ea00640 1 --2- 172.21.15.151:0/3177473070 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f55f4079a90 0x7f55f407bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:07.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.697+0000 7f560ea00640 1 -- 172.21.15.151:0/3177473070 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(80..80 src has 1..80) ==== 5909+0+0 (secure 0 0 0) 0x7f56000b9c70 con 0x7f561806fd90 2024-09-16T15:48:07.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.698+0000 7f5616000640 1 --2- 172.21.15.151:0/3177473070 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f55f4079a90 0x7f55f407bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:07.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.699+0000 7f5616000640 1 --2- 172.21.15.151:0/3177473070 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f55f4079a90 0x7f55f407bf50 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f56180b1920 tx=0x7f5608005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:07.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.705+0000 7f560ea00640 1 -- 172.21.15.151:0/3177473070 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f5618159440 con 0x7f561806fd90 2024-09-16T15:48:07.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.898+0000 7f561d800640 1 -- 172.21.15.151:0/3177473070 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f56180cd430 con 0x7f561806fd90 2024-09-16T15:48:07.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.899+0000 7f560ea00640 1 -- 172.21.15.151:0/3177473070 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+17132 (secure 0 0 0) 0x7f5600083760 con 0x7f561806fd90 2024-09-16T15:48:07.902 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:48:07.902 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":80,"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","created":"2024-09-16T15:38:56.348535+0000","modified":"2024-09-16T15:48:06.427551+0000","last_up_change":"2024-09-16T15:39:10.681384+0000","last_in_change":"2024-09-16T15:39:00.544477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:39:11.602252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:46:50.425692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-16T15:46:51.632208+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":79,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5524076-a903-4d0a-af6a-caf474c4765d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6809","nonce":3111342741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6811","nonce":3111342741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6815","nonce":3111342741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6813","nonce":3111342741}]},"public_addr":"172.21.15.151:6809/3111342741","cluster_addr":"172.21.15.151:6811/3111342741","heartbeat_back_addr":"172.21.15.151:6815/3111342741","heartbeat_front_addr":"172.21.15.151:6813/3111342741","state":["exists","up"]},{"osd":1,"uuid":"dd25925f-0ae1-494e-8529-4dc205759db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6801","nonce":3368304087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6803","nonce":3368304087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6807","nonce":3368304087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6805","nonce":3368304087}]},"public_addr":"172.21.15.151:6801/3368304087","cluster_addr":"172.21.15.151:6803/3368304087","heartbeat_back_addr":"172.21.15.151:6807/3368304087","heartbeat_front_addr":"172.21.15.151:6805/3368304087","state":["exists","up"]},{"osd":2,"uuid":"c2f18115-013d-4da6-a0fe-cb978f4494b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6823","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6825","nonce":3079373271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6827","nonce":3079373271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6831","nonce":3079373271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6829","nonce":3079373271}]},"public_addr":"172.21.15.151:6825/3079373271","cluster_addr":"172.21.15.151:6827/3079373271","heartbeat_back_addr":"172.21.15.151:6831/3079373271","heartbeat_front_addr":"172.21.15.151:6829/3079373271","state":["exists","up"]},{"osd":3,"uuid":"36edb718-2fcb-4d88-8ddc-c321eed4279d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6817","nonce":1142509041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6819","nonce":1142509041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6824","nonce":1142509041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6821","nonce":1142509041}]},"public_addr":"172.21.15.151:6817/1142509041","cluster_addr":"172.21.15.151:6819/1142509041","heartbeat_back_addr":"172.21.15.151:6824/1142509041","heartbeat_front_addr":"172.21.15.151:6821/1142509041","state":["exists","up"]},{"osd":4,"uuid":"64d33708-cd0a-4236-abe3-1b646e82ab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6811","nonce":2735495271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6813","nonce":2735495271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6817","nonce":2735495271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6815","nonce":2735495271}]},"public_addr":"172.21.15.167:6811/2735495271","cluster_addr":"172.21.15.167:6813/2735495271","heartbeat_back_addr":"172.21.15.167:6817/2735495271","heartbeat_front_addr":"172.21.15.167:6815/2735495271","state":["exists","up"]},{"osd":5,"uuid":"f14b44a9-5910-458c-b518-09a8d0de94de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6827","nonce":1494916572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6829","nonce":1494916572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6833","nonce":1494916572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6831","nonce":1494916572}]},"public_addr":"172.21.15.167:6827/1494916572","cluster_addr":"172.21.15.167:6829/1494916572","heartbeat_back_addr":"172.21.15.167:6833/1494916572","heartbeat_front_addr":"172.21.15.167:6831/1494916572","state":["exists","up"]},{"osd":6,"uuid":"39335545-2654-4710-84b0-a9aa47cfcbba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6819","nonce":2538273273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6821","nonce":2538273273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6825","nonce":2538273273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6823","nonce":2538273273}]},"public_addr":"172.21.15.167:6819/2538273273","cluster_addr":"172.21.15.167:6821/2538273273","heartbeat_back_addr":"172.21.15.167:6825/2538273273","heartbeat_front_addr":"172.21.15.167:6823/2538273273","state":["exists","up"]},{"osd":7,"uuid":"4f8bfc81-53dd-4339-ac22-1a4007d9f963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6803","nonce":265072227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6805","nonce":265072227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6809","nonce":265072227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6807","nonce":265072227}]},"public_addr":"172.21.15.167:6803/265072227","cluster_addr":"172.21.15.167:6805/265072227","heartbeat_back_addr":"172.21.15.167:6809/265072227","heartbeat_front_addr":"172.21.15.167:6807/265072227","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.150359+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.084358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.722368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.332809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.320709+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.395153+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.935250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.801443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.a","mappings":[{"from":2,"to":7}]},{"pgid":"11.1","mappings":[{"from":1,"to":7}]},{"pgid":"11.3","mappings":[{"from":3,"to":2}]},{"pgid":"11.3e","mappings":[{"from":1,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:48:07.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.902+0000 7f561d800640 1 -- 172.21.15.151:0/3177473070 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f55f4079a90 msgr2=0x7f55f407bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:07.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.902+0000 7f561d800640 1 --2- 172.21.15.151:0/3177473070 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f55f4079a90 0x7f55f407bf50 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f56180b1920 tx=0x7f5608005f90 comp rx=0 tx=0).stop 2024-09-16T15:48:07.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.902+0000 7f561d800640 1 -- 172.21.15.151:0/3177473070 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f561806fd90 msgr2=0x7f5618142b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:07.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.902+0000 7f561d800640 1 --2- 172.21.15.151:0/3177473070 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f561806fd90 0x7f5618142b70 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5600034040 tx=0x7f56000035f0 comp rx=0 tx=0).stop 2024-09-16T15:48:07.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.902+0000 7f561d800640 1 -- 172.21.15.151:0/3177473070 shutdown_connections 2024-09-16T15:48:07.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.902+0000 7f561d800640 1 --2- 172.21.15.151:0/3177473070 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f55f4079a90 0x7f55f407bf50 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:07.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.902+0000 7f561d800640 1 --2- 172.21.15.151:0/3177473070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56180ccb30 0x7f5618158f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:07.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.902+0000 7f561d800640 1 --2- 172.21.15.151:0/3177473070 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56181438b0 0x7f5618139ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:07.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.902+0000 7f561d800640 1 --2- 172.21.15.151:0/3177473070 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f561806fd90 0x7f5618142b70 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:07.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.902+0000 7f561d800640 1 -- 172.21.15.151:0/3177473070 >> 172.21.15.151:0/3177473070 conn(0x7f56180b1950 msgr2=0x7f56180ae4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:07.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.902+0000 7f561d800640 1 -- 172.21.15.151:0/3177473070 shutdown_connections 2024-09-16T15:48:07.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:07.902+0000 7f561d800640 1 -- 172.21.15.151:0/3177473070 wait complete. 2024-09-16T15:48:07.920 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-16T15:48:07.920 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:48:08.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.077+0000 7f5a7fe00640 1 -- 172.21.15.151:0/4160177908 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a800b28d0 msgr2=0x7f5a80162730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:08.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.077+0000 7f5a7fe00640 1 --2- 172.21.15.151:0/4160177908 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a800b28d0 0x7f5a80162730 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f5a78005fd0 tx=0x7f5a7802a8b0 comp rx=0 tx=0).stop 2024-09-16T15:48:08.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.077+0000 7f5a7fe00640 1 -- 172.21.15.151:0/4160177908 shutdown_connections 2024-09-16T15:48:08.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.077+0000 7f5a7fe00640 1 --2- 172.21.15.151:0/4160177908 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a800b28d0 0x7f5a80162730 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:08.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.077+0000 7f5a7fe00640 1 --2- 172.21.15.151:0/4160177908 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a800d1ec0 0x7f5a800d22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:08.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.077+0000 7f5a7fe00640 1 -- 172.21.15.151:0/4160177908 >> 172.21.15.151:0/4160177908 conn(0x7f5a800d3de0 msgr2=0x7f5a800d41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:08.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.077+0000 7f5a7fe00640 1 -- 172.21.15.151:0/4160177908 shutdown_connections 2024-09-16T15:48:08.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.077+0000 7f5a7fe00640 1 -- 172.21.15.151:0/4160177908 wait complete. 2024-09-16T15:48:08.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.078+0000 7f5a7fe00640 1 Processor -- start 2024-09-16T15:48:08.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.079+0000 7f5a7fe00640 1 -- start start 2024-09-16T15:48:08.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.079+0000 7f5a7fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a800b28d0 0x7f5a800ca5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:08.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.079+0000 7f5a7fe00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a800b4d10 0x7f5a800cb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:08.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.079+0000 7f5a7fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a800d1ec0 0x7f5a800cfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:08.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.079+0000 7f5a7fe00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5a800b3a80 con 0x7f5a800b28d0 2024-09-16T15:48:08.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.079+0000 7f5a7fe00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5a800b3900 con 0x7f5a800d1ec0 2024-09-16T15:48:08.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.079+0000 7f5a7fe00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f5a800b3c00 con 0x7f5a800b4d10 2024-09-16T15:48:08.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.080+0000 7f5a7ea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a800b28d0 0x7f5a800ca5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:08.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.080+0000 7f5a7ea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a800b28d0 0x7f5a800ca5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44732/0 (socket says 172.21.15.151:44732) 2024-09-16T15:48:08.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.080+0000 7f5a7ea00640 1 -- 172.21.15.151:0/3308366086 learned_addr learned my addr 172.21.15.151:0/3308366086 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:48:08.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.080+0000 7f5a7e000640 1 --2- 172.21.15.151:0/3308366086 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a800b4d10 0x7f5a800cb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:08.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.080+0000 7f5a7f400640 1 --2- 172.21.15.151:0/3308366086 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a800d1ec0 0x7f5a800cfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:08.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.080+0000 7f5a7ea00640 1 -- 172.21.15.151:0/3308366086 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a800b4d10 msgr2=0x7f5a800cb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:08.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.080+0000 7f5a7ea00640 1 --2- 172.21.15.151:0/3308366086 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a800b4d10 0x7f5a800cb8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:08.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.080+0000 7f5a7ea00640 1 -- 172.21.15.151:0/3308366086 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a800d1ec0 msgr2=0x7f5a800cfcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:08.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.080+0000 7f5a7ea00640 1 --2- 172.21.15.151:0/3308366086 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a800d1ec0 0x7f5a800cfcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:08.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.080+0000 7f5a7ea00640 1 -- 172.21.15.151:0/3308366086 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a800caf50 con 0x7f5a800b28d0 2024-09-16T15:48:08.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.080+0000 7f5a7ea00640 1 --2- 172.21.15.151:0/3308366086 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a800b28d0 0x7f5a800ca5d0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f5a7000e790 tx=0x7f5a7000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:08.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.081+0000 7f5a76a00640 1 -- 172.21.15.151:0/3308366086 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a7000eea0 con 0x7f5a800b28d0 2024-09-16T15:48:08.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.081+0000 7f5a76a00640 1 -- 172.21.15.151:0/3308366086 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a7000cdb0 con 0x7f5a800b28d0 2024-09-16T15:48:08.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.081+0000 7f5a7fe00640 1 -- 172.21.15.151:0/3308366086 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a800cbde0 con 0x7f5a800b28d0 2024-09-16T15:48:08.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.081+0000 7f5a76a00640 1 -- 172.21.15.151:0/3308366086 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a70010690 con 0x7f5a800b28d0 2024-09-16T15:48:08.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.081+0000 7f5a7fe00640 1 -- 172.21.15.151:0/3308366086 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a800cb4d0 con 0x7f5a800b28d0 2024-09-16T15:48:08.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.082+0000 7f5a7fe00640 1 -- 172.21.15.151:0/3308366086 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a800d22a0 con 0x7f5a800b28d0 2024-09-16T15:48:08.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.083+0000 7f5a76a00640 1 -- 172.21.15.151:0/3308366086 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 30) ==== 102868+0+0 (secure 0 0 0) 0x7f5a70010830 con 0x7f5a800b28d0 2024-09-16T15:48:08.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.084+0000 7f5a76a00640 1 --2- 172.21.15.151:0/3308366086 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5a600799c0 0x7f5a6007be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:08.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.084+0000 7f5a76a00640 1 -- 172.21.15.151:0/3308366086 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5909+0+0 (secure 0 0 0) 0x7f5a7009a1d0 con 0x7f5a800b28d0 2024-09-16T15:48:08.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.084+0000 7f5a7e000640 1 --2- 172.21.15.151:0/3308366086 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5a600799c0 0x7f5a6007be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:08.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.085+0000 7f5a7e000640 1 --2- 172.21.15.151:0/3308366086 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5a600799c0 0x7f5a6007be80 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f5a801f7ba0 tx=0x7f5a6c007e90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:08.090 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.088+0000 7f5a76a00640 1 -- 172.21.15.151:0/3308366086 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f5a700638f0 con 0x7f5a800b28d0 2024-09-16T15:48:08.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.312+0000 7f5a7fe00640 1 -- 172.21.15.151:0/3308366086 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5a800d1510 con 0x7f5a800b28d0 2024-09-16T15:48:08.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.312+0000 7f5a76a00640 1 -- 172.21.15.151:0/3308366086 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5234 (secure 0 0 0) 0x7f5a70068320 con 0x7f5a800b28d0 2024-09-16T15:48:08.314 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:48:08.315 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":47,"btime":"2024-09-16T15:47:01:291406+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5705,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3291846596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3291846596},{"type":"v1","addr":"172.21.15.167:6837","nonce":3291846596}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6175,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/2504229431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":2504229431},{"type":"v1","addr":"172.21.15.151:6834","nonce":2504229431}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6181,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/464376104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":464376104},{"type":"v1","addr":"172.21.15.167:6836","nonce":464376104}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:46:52.671228+0000","modified":"2024-09-16T15:46:55.692251+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6456},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6456":{"gid":6456,"name":"a","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.151:6835/3158835036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":3158835036},{"type":"v1","addr":"172.21.15.151:6835","nonce":3158835036}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6456,"qdb_cluster":[6456]},"id":5}]} 2024-09-16T15:48:08.315 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 47 2024-09-16T15:48:08.317 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.315+0000 7f5a7fe00640 1 -- 172.21.15.151:0/3308366086 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5a600799c0 msgr2=0x7f5a6007be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:08.317 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.315+0000 7f5a7fe00640 1 --2- 172.21.15.151:0/3308366086 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5a600799c0 0x7f5a6007be80 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f5a801f7ba0 tx=0x7f5a6c007e90 comp rx=0 tx=0).stop 2024-09-16T15:48:08.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.315+0000 7f5a7fe00640 1 -- 172.21.15.151:0/3308366086 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a800b28d0 msgr2=0x7f5a800ca5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:08.320 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.315+0000 7f5a7fe00640 1 --2- 172.21.15.151:0/3308366086 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a800b28d0 0x7f5a800ca5d0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f5a7000e790 tx=0x7f5a7000ec50 comp rx=0 tx=0).stop 2024-09-16T15:48:08.320 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.315+0000 7f5a7fe00640 1 -- 172.21.15.151:0/3308366086 shutdown_connections 2024-09-16T15:48:08.320 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.315+0000 7f5a7fe00640 1 --2- 172.21.15.151:0/3308366086 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5a600799c0 0x7f5a6007be80 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:08.320 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.315+0000 7f5a7fe00640 1 --2- 172.21.15.151:0/3308366086 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a800d1ec0 0x7f5a800cfcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:08.320 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.315+0000 7f5a7fe00640 1 --2- 172.21.15.151:0/3308366086 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a800b4d10 0x7f5a800cb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:08.320 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.315+0000 7f5a7fe00640 1 --2- 172.21.15.151:0/3308366086 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a800b28d0 0x7f5a800ca5d0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:08.320 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.315+0000 7f5a7fe00640 1 -- 172.21.15.151:0/3308366086 >> 172.21.15.151:0/3308366086 conn(0x7f5a800d3de0 msgr2=0x7f5a800b4060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:08.320 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.315+0000 7f5a7fe00640 1 -- 172.21.15.151:0/3308366086 shutdown_connections 2024-09-16T15:48:08.320 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.315+0000 7f5a7fe00640 1 -- 172.21.15.151:0/3308366086 wait complete. 2024-09-16T15:48:08.333 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:48:08.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.489+0000 7f8471000640 1 -- 172.21.15.151:0/94583197 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f846c0ccf00 msgr2=0x7f846c0acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:08.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.489+0000 7f8471000640 1 --2- 172.21.15.151:0/94583197 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f846c0ccf00 0x7f846c0acd30 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f846c075130 tx=0x7f845802a4d0 comp rx=0 tx=0).stop 2024-09-16T15:48:08.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.489+0000 7f8471000640 1 -- 172.21.15.151:0/94583197 shutdown_connections 2024-09-16T15:48:08.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.489+0000 7f8471000640 1 --2- 172.21.15.151:0/94583197 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f846c0ccf00 0x7f846c0acd30 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:08.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.489+0000 7f8471000640 1 --2- 172.21.15.151:0/94583197 >> v2:172.21.15.167:3300/0 conn(0x7f846c15f5a0 0x7f846c0b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:08.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.489+0000 7f8471000640 1 -- 172.21.15.151:0/94583197 >> 172.21.15.151:0/94583197 conn(0x7f846c0b1950 msgr2=0x7f846c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:08.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.489+0000 7f8471000640 1 -- 172.21.15.151:0/94583197 shutdown_connections 2024-09-16T15:48:08.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.489+0000 7f8471000640 1 -- 172.21.15.151:0/94583197 wait complete. 2024-09-16T15:48:08.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.490+0000 7f8471000640 1 Processor -- start 2024-09-16T15:48:08.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.491+0000 7f8471000640 1 -- start start 2024-09-16T15:48:08.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.492+0000 7f8471000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f846c0ad270 0x7f846c1f5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:08.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.492+0000 7f8471000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f846c0ccf00 0x7f846c15c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:08.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.492+0000 7f8471000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f846c15f5a0 0x7f846c0d0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:08.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.492+0000 7f8471000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f846c06f160 con 0x7f846c15f5a0 2024-09-16T15:48:08.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.492+0000 7f8471000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f846c06f2e0 con 0x7f846c0ccf00 2024-09-16T15:48:08.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.492+0000 7f8471000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f846c06f460 con 0x7f846c0ad270 2024-09-16T15:48:08.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.492+0000 7f8469e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f846c0ad270 0x7f846c1f5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:08.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.492+0000 7f846a800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f846c15f5a0 0x7f846c0d0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:08.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.492+0000 7f8469400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f846c0ccf00 0x7f846c15c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:08.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.492+0000 7f8469400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f846c0ccf00 0x7f846c15c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.151:53586/0 (socket says 172.21.15.151:53586) 2024-09-16T15:48:08.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.492+0000 7f8469e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f846c0ad270 0x7f846c1f5be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.151:46270/0 (socket says 172.21.15.151:46270) 2024-09-16T15:48:08.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.492+0000 7f8469400640 1 -- 172.21.15.151:0/3194098636 learned_addr learned my addr 172.21.15.151:0/3194098636 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:48:08.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.492+0000 7f846a800640 1 -- 172.21.15.151:0/3194098636 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f846c0ad270 msgr2=0x7f846c1f5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:08.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.492+0000 7f846a800640 1 --2- 172.21.15.151:0/3194098636 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f846c0ad270 0x7f846c1f5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:08.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.492+0000 7f846a800640 1 -- 172.21.15.151:0/3194098636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f846c0ccf00 msgr2=0x7f846c15c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:08.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.492+0000 7f846a800640 1 --2- 172.21.15.151:0/3194098636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f846c0ccf00 0x7f846c15c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:08.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.492+0000 7f846a800640 1 -- 172.21.15.151:0/3194098636 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f846c15a960 con 0x7f846c15f5a0 2024-09-16T15:48:08.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.493+0000 7f846a800640 1 --2- 172.21.15.151:0/3194098636 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f846c15f5a0 0x7f846c0d0750 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f846400c780 tx=0x7f846400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:08.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.493+0000 7f8462000640 1 -- 172.21.15.151:0/3194098636 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8464017070 con 0x7f846c15f5a0 2024-09-16T15:48:08.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.493+0000 7f8462000640 1 -- 172.21.15.151:0/3194098636 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8464003c80 con 0x7f846c15f5a0 2024-09-16T15:48:08.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.493+0000 7f8471000640 1 -- 172.21.15.151:0/3194098636 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f846c15b380 con 0x7f846c15f5a0 2024-09-16T15:48:08.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.493+0000 7f8462000640 1 -- 172.21.15.151:0/3194098636 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84640052d0 con 0x7f846c15f5a0 2024-09-16T15:48:08.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.493+0000 7f8471000640 1 -- 172.21.15.151:0/3194098636 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f846c15b940 con 0x7f846c15f5a0 2024-09-16T15:48:08.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.494+0000 7f8471000640 1 -- 172.21.15.151:0/3194098636 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f846c06fa10 con 0x7f846c15f5a0 2024-09-16T15:48:08.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.495+0000 7f8462000640 1 -- 172.21.15.151:0/3194098636 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 30) ==== 102868+0+0 (secure 0 0 0) 0x7f846401a070 con 0x7f846c15f5a0 2024-09-16T15:48:08.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.496+0000 7f8462000640 1 --2- 172.21.15.151:0/3194098636 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f844c0799c0 0x7f844c07be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:08.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.496+0000 7f8462000640 1 -- 172.21.15.151:0/3194098636 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5909+0+0 (secure 0 0 0) 0x7f8464098e30 con 0x7f846c15f5a0 2024-09-16T15:48:08.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.497+0000 7f8469e00640 1 --2- 172.21.15.151:0/3194098636 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f844c0799c0 0x7f844c07be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:08.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.497+0000 7f8469e00640 1 --2- 172.21.15.151:0/3194098636 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f844c0799c0 0x7f844c07be80 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f846c15c900 tx=0x7f845c009060 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:08.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.501+0000 7f8462000640 1 -- 172.21.15.151:0/3194098636 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8464062550 con 0x7f846c15f5a0 2024-09-16T15:48:08.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.718+0000 7f8471000640 1 -- 172.21.15.151:0/3194098636 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f846c15cb90 con 0x7f846c15f5a0 2024-09-16T15:48:08.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.719+0000 7f8462000640 1 -- 172.21.15.151:0/3194098636 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5234 (secure 0 0 0) 0x7f8464066f80 con 0x7f846c15f5a0 2024-09-16T15:48:08.721 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:48:08.721 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":47,"btime":"2024-09-16T15:47:01:291406+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5705,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3291846596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3291846596},{"type":"v1","addr":"172.21.15.167:6837","nonce":3291846596}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6175,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/2504229431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":2504229431},{"type":"v1","addr":"172.21.15.151:6834","nonce":2504229431}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6181,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/464376104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":464376104},{"type":"v1","addr":"172.21.15.167:6836","nonce":464376104}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:46:52.671228+0000","modified":"2024-09-16T15:46:55.692251+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6456},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6456":{"gid":6456,"name":"a","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.151:6835/3158835036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":3158835036},{"type":"v1","addr":"172.21.15.151:6835","nonce":3158835036}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6456,"qdb_cluster":[6456]},"id":5}]} 2024-09-16T15:48:08.721 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 47 2024-09-16T15:48:08.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.721+0000 7f8471000640 1 -- 172.21.15.151:0/3194098636 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f844c0799c0 msgr2=0x7f844c07be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:08.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.721+0000 7f8471000640 1 --2- 172.21.15.151:0/3194098636 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f844c0799c0 0x7f844c07be80 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f846c15c900 tx=0x7f845c009060 comp rx=0 tx=0).stop 2024-09-16T15:48:08.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.721+0000 7f8471000640 1 -- 172.21.15.151:0/3194098636 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f846c15f5a0 msgr2=0x7f846c0d0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:08.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.721+0000 7f8471000640 1 --2- 172.21.15.151:0/3194098636 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f846c15f5a0 0x7f846c0d0750 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f846400c780 tx=0x7f846400cc40 comp rx=0 tx=0).stop 2024-09-16T15:48:08.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.721+0000 7f8471000640 1 -- 172.21.15.151:0/3194098636 shutdown_connections 2024-09-16T15:48:08.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.721+0000 7f8471000640 1 --2- 172.21.15.151:0/3194098636 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f844c0799c0 0x7f844c07be80 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:08.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.721+0000 7f8471000640 1 --2- 172.21.15.151:0/3194098636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f846c0ccf00 0x7f846c15c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:08.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.721+0000 7f8471000640 1 --2- 172.21.15.151:0/3194098636 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f846c15f5a0 0x7f846c0d0750 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:08.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.721+0000 7f8471000640 1 --2- 172.21.15.151:0/3194098636 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f846c0ad270 0x7f846c1f5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:08.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.721+0000 7f8471000640 1 -- 172.21.15.151:0/3194098636 >> 172.21.15.151:0/3194098636 conn(0x7f846c0b1950 msgr2=0x7f846c0b2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:08.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.721+0000 7f8471000640 1 -- 172.21.15.151:0/3194098636 shutdown_connections 2024-09-16T15:48:08.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.721+0000 7f8471000640 1 -- 172.21.15.151:0/3194098636 wait complete. 2024-09-16T15:48:08.740 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:48:08.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.898+0000 7fcfaae00640 1 -- 172.21.15.151:0/2927068456 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcfa415f5a0 msgr2=0x7fcfa40b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:08.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.898+0000 7fcfaae00640 1 --2- 172.21.15.151:0/2927068456 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcfa415f5a0 0x7fcfa40b2440 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fcf94009830 tx=0x7fcf9402a480 comp rx=0 tx=0).stop 2024-09-16T15:48:08.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.898+0000 7fcfaae00640 1 -- 172.21.15.151:0/2927068456 shutdown_connections 2024-09-16T15:48:08.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.898+0000 7fcfaae00640 1 --2- 172.21.15.151:0/2927068456 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcfa4073fb0 0x7fcfa4074440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:08.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.898+0000 7fcfaae00640 1 --2- 172.21.15.151:0/2927068456 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcfa415f5a0 0x7fcfa40b2440 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:08.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.898+0000 7fcfaae00640 1 -- 172.21.15.151:0/2927068456 >> 172.21.15.151:0/2927068456 conn(0x7fcfa40b1950 msgr2=0x7fcfa40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:08.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.898+0000 7fcfaae00640 1 -- 172.21.15.151:0/2927068456 shutdown_connections 2024-09-16T15:48:08.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.898+0000 7fcfaae00640 1 -- 172.21.15.151:0/2927068456 wait complete. 2024-09-16T15:48:08.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.900+0000 7fcfaae00640 1 Processor -- start 2024-09-16T15:48:08.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.900+0000 7fcfaae00640 1 -- start start 2024-09-16T15:48:08.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.901+0000 7fcfaae00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcfa4073fb0 0x7fcfa41f4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:08.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.901+0000 7fcfaae00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcfa40ccf00 0x7fcfa41ea480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:08.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.901+0000 7fcfaae00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfa415f5a0 0x7fcfa41555b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:08.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.901+0000 7fcfaae00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa406f2e0 con 0x7fcfa415f5a0 2024-09-16T15:48:08.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.901+0000 7fcfaae00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa406f160 con 0x7fcfa40ccf00 2024-09-16T15:48:08.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.901+0000 7fcfaae00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fcfa406f460 con 0x7fcfa4073fb0 2024-09-16T15:48:08.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.901+0000 7fcfa8e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfa415f5a0 0x7fcfa41555b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:08.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.901+0000 7fcfa8e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfa415f5a0 0x7fcfa41555b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44764/0 (socket says 172.21.15.151:44764) 2024-09-16T15:48:08.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.901+0000 7fcfa8e00640 1 -- 172.21.15.151:0/486931647 learned_addr learned my addr 172.21.15.151:0/486931647 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:48:08.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.901+0000 7fcfa3e00640 1 --2- 172.21.15.151:0/486931647 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcfa4073fb0 0x7fcfa41f4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:08.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.901+0000 7fcfa3400640 1 --2- 172.21.15.151:0/486931647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcfa40ccf00 0x7fcfa41ea480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:08.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.901+0000 7fcfa8e00640 1 -- 172.21.15.151:0/486931647 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcfa4073fb0 msgr2=0x7fcfa41f4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:08.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.901+0000 7fcfa8e00640 1 --2- 172.21.15.151:0/486931647 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcfa4073fb0 0x7fcfa41f4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:08.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.901+0000 7fcfa8e00640 1 -- 172.21.15.151:0/486931647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcfa40ccf00 msgr2=0x7fcfa41ea480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:08.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.901+0000 7fcfa8e00640 1 --2- 172.21.15.151:0/486931647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcfa40ccf00 0x7fcfa41ea480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:08.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.901+0000 7fcfa8e00640 1 -- 172.21.15.151:0/486931647 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfa41ea9c0 con 0x7fcfa415f5a0 2024-09-16T15:48:08.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.902+0000 7fcfa8e00640 1 --2- 172.21.15.151:0/486931647 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfa415f5a0 0x7fcfa41555b0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c00eb10 tx=0x7fcf9c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:08.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.902+0000 7fcf9be00640 1 -- 172.21.15.151:0/486931647 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf9c004b20 con 0x7fcfa415f5a0 2024-09-16T15:48:08.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.902+0000 7fcf9be00640 1 -- 172.21.15.151:0/486931647 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcf9c004ce0 con 0x7fcfa415f5a0 2024-09-16T15:48:08.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.902+0000 7fcf9be00640 1 -- 172.21.15.151:0/486931647 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf9c005280 con 0x7fcfa415f5a0 2024-09-16T15:48:08.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.902+0000 7fcfaae00640 1 -- 172.21.15.151:0/486931647 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfa4155af0 con 0x7fcfa415f5a0 2024-09-16T15:48:08.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.902+0000 7fcfaae00640 1 -- 172.21.15.151:0/486931647 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfa41f5590 con 0x7fcfa415f5a0 2024-09-16T15:48:08.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.903+0000 7fcfaae00640 1 -- 172.21.15.151:0/486931647 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfa40cf290 con 0x7fcfa415f5a0 2024-09-16T15:48:08.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.904+0000 7fcf9be00640 1 -- 172.21.15.151:0/486931647 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 30) ==== 102868+0+0 (secure 0 0 0) 0x7fcf9c005420 con 0x7fcfa415f5a0 2024-09-16T15:48:08.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.905+0000 7fcf9be00640 1 --2- 172.21.15.151:0/486931647 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcf840799c0 0x7fcf8407be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:08.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.905+0000 7fcf9be00640 1 -- 172.21.15.151:0/486931647 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5909+0+0 (secure 0 0 0) 0x7fcf9c09a4c0 con 0x7fcfa415f5a0 2024-09-16T15:48:08.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.906+0000 7fcfa3e00640 1 --2- 172.21.15.151:0/486931647 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcf840799c0 0x7fcf8407be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:08.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.906+0000 7fcfa3e00640 1 --2- 172.21.15.151:0/486931647 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcf840799c0 0x7fcf8407be80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fcf94034000 tx=0x7fcf94004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:08.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:08.909+0000 7fcf9be00640 1 -- 172.21.15.151:0/486931647 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fcf9c063be0 con 0x7fcfa415f5a0 2024-09-16T15:48:09.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.097+0000 7fcfaae00640 1 -- 172.21.15.151:0/486931647 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcfa41eab90 con 0x7fcfa415f5a0 2024-09-16T15:48:09.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.098+0000 7fcf9be00640 1 -- 172.21.15.151:0/486931647 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+17132 (secure 0 0 0) 0x7fcf9c09b190 con 0x7fcfa415f5a0 2024-09-16T15:48:09.101 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:48:09.101 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":80,"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","created":"2024-09-16T15:38:56.348535+0000","modified":"2024-09-16T15:48:06.427551+0000","last_up_change":"2024-09-16T15:39:10.681384+0000","last_in_change":"2024-09-16T15:39:00.544477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:39:11.602252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:46:50.425692+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-16T15:46:51.632208+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":79,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5524076-a903-4d0a-af6a-caf474c4765d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6809","nonce":3111342741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6811","nonce":3111342741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6815","nonce":3111342741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6813","nonce":3111342741}]},"public_addr":"172.21.15.151:6809/3111342741","cluster_addr":"172.21.15.151:6811/3111342741","heartbeat_back_addr":"172.21.15.151:6815/3111342741","heartbeat_front_addr":"172.21.15.151:6813/3111342741","state":["exists","up"]},{"osd":1,"uuid":"dd25925f-0ae1-494e-8529-4dc205759db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6801","nonce":3368304087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6803","nonce":3368304087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6807","nonce":3368304087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6805","nonce":3368304087}]},"public_addr":"172.21.15.151:6801/3368304087","cluster_addr":"172.21.15.151:6803/3368304087","heartbeat_back_addr":"172.21.15.151:6807/3368304087","heartbeat_front_addr":"172.21.15.151:6805/3368304087","state":["exists","up"]},{"osd":2,"uuid":"c2f18115-013d-4da6-a0fe-cb978f4494b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6823","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6825","nonce":3079373271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6827","nonce":3079373271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6831","nonce":3079373271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6829","nonce":3079373271}]},"public_addr":"172.21.15.151:6825/3079373271","cluster_addr":"172.21.15.151:6827/3079373271","heartbeat_back_addr":"172.21.15.151:6831/3079373271","heartbeat_front_addr":"172.21.15.151:6829/3079373271","state":["exists","up"]},{"osd":3,"uuid":"36edb718-2fcb-4d88-8ddc-c321eed4279d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6817","nonce":1142509041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6819","nonce":1142509041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6824","nonce":1142509041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6821","nonce":1142509041}]},"public_addr":"172.21.15.151:6817/1142509041","cluster_addr":"172.21.15.151:6819/1142509041","heartbeat_back_addr":"172.21.15.151:6824/1142509041","heartbeat_front_addr":"172.21.15.151:6821/1142509041","state":["exists","up"]},{"osd":4,"uuid":"64d33708-cd0a-4236-abe3-1b646e82ab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6811","nonce":2735495271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6813","nonce":2735495271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6817","nonce":2735495271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6815","nonce":2735495271}]},"public_addr":"172.21.15.167:6811/2735495271","cluster_addr":"172.21.15.167:6813/2735495271","heartbeat_back_addr":"172.21.15.167:6817/2735495271","heartbeat_front_addr":"172.21.15.167:6815/2735495271","state":["exists","up"]},{"osd":5,"uuid":"f14b44a9-5910-458c-b518-09a8d0de94de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6827","nonce":1494916572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6829","nonce":1494916572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6833","nonce":1494916572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6831","nonce":1494916572}]},"public_addr":"172.21.15.167:6827/1494916572","cluster_addr":"172.21.15.167:6829/1494916572","heartbeat_back_addr":"172.21.15.167:6833/1494916572","heartbeat_front_addr":"172.21.15.167:6831/1494916572","state":["exists","up"]},{"osd":6,"uuid":"39335545-2654-4710-84b0-a9aa47cfcbba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6819","nonce":2538273273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6821","nonce":2538273273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6825","nonce":2538273273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6823","nonce":2538273273}]},"public_addr":"172.21.15.167:6819/2538273273","cluster_addr":"172.21.15.167:6821/2538273273","heartbeat_back_addr":"172.21.15.167:6825/2538273273","heartbeat_front_addr":"172.21.15.167:6823/2538273273","state":["exists","up"]},{"osd":7,"uuid":"4f8bfc81-53dd-4339-ac22-1a4007d9f963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6803","nonce":265072227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6805","nonce":265072227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6809","nonce":265072227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6807","nonce":265072227}]},"public_addr":"172.21.15.167:6803/265072227","cluster_addr":"172.21.15.167:6805/265072227","heartbeat_back_addr":"172.21.15.167:6809/265072227","heartbeat_front_addr":"172.21.15.167:6807/265072227","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.150359+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.084358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.722368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.332809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.320709+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.395153+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.935250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.801443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.a","mappings":[{"from":2,"to":7}]},{"pgid":"11.1","mappings":[{"from":1,"to":7}]},{"pgid":"11.3","mappings":[{"from":3,"to":2}]},{"pgid":"11.3e","mappings":[{"from":1,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:48:09.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.100+0000 7fcfaae00640 1 -- 172.21.15.151:0/486931647 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcf840799c0 msgr2=0x7fcf8407be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:09.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.100+0000 7fcfaae00640 1 --2- 172.21.15.151:0/486931647 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcf840799c0 0x7fcf8407be80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fcf94034000 tx=0x7fcf94004730 comp rx=0 tx=0).stop 2024-09-16T15:48:09.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.100+0000 7fcfaae00640 1 -- 172.21.15.151:0/486931647 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfa415f5a0 msgr2=0x7fcfa41555b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:09.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.100+0000 7fcfaae00640 1 --2- 172.21.15.151:0/486931647 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfa415f5a0 0x7fcfa41555b0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c00eb10 tx=0x7fcf9c00c2d0 comp rx=0 tx=0).stop 2024-09-16T15:48:09.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.101+0000 7fcfaae00640 1 -- 172.21.15.151:0/486931647 shutdown_connections 2024-09-16T15:48:09.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.101+0000 7fcfaae00640 1 --2- 172.21.15.151:0/486931647 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcf840799c0 0x7fcf8407be80 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:09.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.101+0000 7fcfaae00640 1 --2- 172.21.15.151:0/486931647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcfa40ccf00 0x7fcfa41ea480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:09.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.101+0000 7fcfaae00640 1 --2- 172.21.15.151:0/486931647 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfa415f5a0 0x7fcfa41555b0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:09.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.101+0000 7fcfaae00640 1 --2- 172.21.15.151:0/486931647 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcfa4073fb0 0x7fcfa41f4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:09.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.101+0000 7fcfaae00640 1 -- 172.21.15.151:0/486931647 >> 172.21.15.151:0/486931647 conn(0x7fcfa40b1950 msgr2=0x7fcfa40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:09.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.101+0000 7fcfaae00640 1 -- 172.21.15.151:0/486931647 shutdown_connections 2024-09-16T15:48:09.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.101+0000 7fcfaae00640 1 -- 172.21.15.151:0/486931647 wait complete. 2024-09-16T15:48:09.120 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-16T15:48:09.278 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.276+0000 7fb52e800640 1 -- 172.21.15.151:0/111021428 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5280b4100 msgr2=0x7fb5280b1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:09.278 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.276+0000 7fb52e800640 1 --2- 172.21.15.151:0/111021428 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5280b4100 0x7fb5280b1770 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fb5280d4ea0 tx=0x7fb52002a7f0 comp rx=0 tx=0).stop 2024-09-16T15:48:09.278 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.276+0000 7fb52e800640 1 -- 172.21.15.151:0/111021428 shutdown_connections 2024-09-16T15:48:09.278 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.276+0000 7fb52e800640 1 --2- 172.21.15.151:0/111021428 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5280b4100 0x7fb5280b1770 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:09.278 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.276+0000 7fb52e800640 1 --2- 172.21.15.151:0/111021428 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5280b5370 0x7fb5280b3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:09.278 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.276+0000 7fb52e800640 1 -- 172.21.15.151:0/111021428 >> 172.21.15.151:0/111021428 conn(0x7fb5280d3df0 msgr2=0x7fb5280d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:09.278 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.276+0000 7fb52e800640 1 -- 172.21.15.151:0/111021428 shutdown_connections 2024-09-16T15:48:09.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.276+0000 7fb52e800640 1 -- 172.21.15.151:0/111021428 wait complete. 2024-09-16T15:48:09.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.277+0000 7fb52e800640 1 Processor -- start 2024-09-16T15:48:09.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.278+0000 7fb52e800640 1 -- start start 2024-09-16T15:48:09.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.278+0000 7fb52e800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5280b4100 0x7fb52814f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:09.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.278+0000 7fb52e800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5280b5370 0x7fb5280b6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:09.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.278+0000 7fb52e800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb528161440 0x7fb5280b7160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:09.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.278+0000 7fb52e800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb5280b2040 con 0x7fb528161440 2024-09-16T15:48:09.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.278+0000 7fb52e800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb5280b1ec0 con 0x7fb5280b5370 2024-09-16T15:48:09.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.278+0000 7fb52e800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fb5280b21c0 con 0x7fb5280b4100 2024-09-16T15:48:09.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.279+0000 7fb52de00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb528161440 0x7fb5280b7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:09.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.279+0000 7fb52de00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb528161440 0x7fb5280b7160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44774/0 (socket says 172.21.15.151:44774) 2024-09-16T15:48:09.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.279+0000 7fb52de00640 1 -- 172.21.15.151:0/2306493623 learned_addr learned my addr 172.21.15.151:0/2306493623 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:48:09.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.279+0000 7fb52d400640 1 --2- 172.21.15.151:0/2306493623 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5280b4100 0x7fb52814f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:09.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.279+0000 7fb52ca00640 1 --2- 172.21.15.151:0/2306493623 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5280b5370 0x7fb5280b6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:09.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.279+0000 7fb52de00640 1 -- 172.21.15.151:0/2306493623 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5280b4100 msgr2=0x7fb52814f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:09.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.279+0000 7fb52de00640 1 --2- 172.21.15.151:0/2306493623 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5280b4100 0x7fb52814f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:09.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.279+0000 7fb52de00640 1 -- 172.21.15.151:0/2306493623 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5280b5370 msgr2=0x7fb5280b6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:09.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.279+0000 7fb52de00640 1 --2- 172.21.15.151:0/2306493623 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5280b5370 0x7fb5280b6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:09.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.279+0000 7fb52de00640 1 -- 172.21.15.151:0/2306493623 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5280cfa60 con 0x7fb528161440 2024-09-16T15:48:09.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.279+0000 7fb52d400640 1 --2- 172.21.15.151:0/2306493623 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5280b4100 0x7fb52814f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:48:09.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.279+0000 7fb52de00640 1 --2- 172.21.15.151:0/2306493623 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb528161440 0x7fb5280b7160 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fb520035040 tx=0x7fb520003ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:09.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.280+0000 7fb525600640 1 -- 172.21.15.151:0/2306493623 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb520041020 con 0x7fb528161440 2024-09-16T15:48:09.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.280+0000 7fb525600640 1 -- 172.21.15.151:0/2306493623 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb520032c70 con 0x7fb528161440 2024-09-16T15:48:09.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.280+0000 7fb525600640 1 -- 172.21.15.151:0/2306493623 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb520037040 con 0x7fb528161440 2024-09-16T15:48:09.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.280+0000 7fb52e800640 1 -- 172.21.15.151:0/2306493623 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5280d0a10 con 0x7fb528161440 2024-09-16T15:48:09.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.280+0000 7fb52e800640 1 -- 172.21.15.151:0/2306493623 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb52806f100 con 0x7fb528161440 2024-09-16T15:48:09.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.281+0000 7fb52e800640 1 -- 172.21.15.151:0/2306493623 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5280d0800 con 0x7fb528161440 2024-09-16T15:48:09.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.282+0000 7fb525600640 1 -- 172.21.15.151:0/2306493623 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 30) ==== 102868+0+0 (secure 0 0 0) 0x7fb520044020 con 0x7fb528161440 2024-09-16T15:48:09.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.283+0000 7fb525600640 1 --2- 172.21.15.151:0/2306493623 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb5080799c0 0x7fb50807be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:09.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.283+0000 7fb525600640 1 -- 172.21.15.151:0/2306493623 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5909+0+0 (secure 0 0 0) 0x7fb5200b9f80 con 0x7fb528161440 2024-09-16T15:48:09.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.284+0000 7fb52d400640 1 --2- 172.21.15.151:0/2306493623 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb5080799c0 0x7fb50807be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:09.286 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.284+0000 7fb52d400640 1 --2- 172.21.15.151:0/2306493623 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb5080799c0 0x7fb50807be80 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb5280748e0 tx=0x7fb51c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:09.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.287+0000 7fb525600640 1 -- 172.21.15.151:0/2306493623 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb5200bfe20 con 0x7fb528161440 2024-09-16T15:48:09.509 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.507+0000 7fb52e800640 1 -- 172.21.15.151:0/2306493623 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fb5280b1770 con 0x7fb528161440 2024-09-16T15:48:09.509 INFO:tasks.ceph.mon.a.smithi151.stderr:2024-09-16T15:48:09.507+0000 7f62c3200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T15:48:09.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.509+0000 7fb525600640 1 -- 172.21.15.151:0/2306493623 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v48) ==== 161+0+0 (secure 0 0 0) 0x7fb52004b030 con 0x7fb528161440 2024-09-16T15:48:09.511 INFO:teuthology.orchestra.run.smithi151.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-16T15:48:09.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.513+0000 7fb52e800640 1 -- 172.21.15.151:0/2306493623 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb5080799c0 msgr2=0x7fb50807be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:09.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.513+0000 7fb52e800640 1 --2- 172.21.15.151:0/2306493623 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb5080799c0 0x7fb50807be80 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb5280748e0 tx=0x7fb51c009040 comp rx=0 tx=0).stop 2024-09-16T15:48:09.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.513+0000 7fb52e800640 1 -- 172.21.15.151:0/2306493623 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb528161440 msgr2=0x7fb5280b7160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:09.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.513+0000 7fb52e800640 1 --2- 172.21.15.151:0/2306493623 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb528161440 0x7fb5280b7160 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fb520035040 tx=0x7fb520003ee0 comp rx=0 tx=0).stop 2024-09-16T15:48:09.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.513+0000 7fb52e800640 1 -- 172.21.15.151:0/2306493623 shutdown_connections 2024-09-16T15:48:09.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.513+0000 7fb52e800640 1 --2- 172.21.15.151:0/2306493623 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb5080799c0 0x7fb50807be80 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:09.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.513+0000 7fb52e800640 1 --2- 172.21.15.151:0/2306493623 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb5280b5370 0x7fb5280b6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:09.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.513+0000 7fb52e800640 1 --2- 172.21.15.151:0/2306493623 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb528161440 0x7fb5280b7160 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:09.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.513+0000 7fb52e800640 1 --2- 172.21.15.151:0/2306493623 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb5280b4100 0x7fb52814f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:09.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.513+0000 7fb52e800640 1 -- 172.21.15.151:0/2306493623 >> 172.21.15.151:0/2306493623 conn(0x7fb5280d3df0 msgr2=0x7fb52806d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:09.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.513+0000 7fb52e800640 1 -- 172.21.15.151:0/2306493623 shutdown_connections 2024-09-16T15:48:09.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.513+0000 7fb52e800640 1 -- 172.21.15.151:0/2306493623 wait complete. 2024-09-16T15:48:09.534 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-16T15:48:09.582 INFO:tasks.ceph.mds.a.smithi151.stdout:starting mds.a at 2024-09-16T15:48:09.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.693+0000 7fd8d2200640 1 -- 172.21.15.151:0/3446414992 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8cc0cccd0 msgr2=0x7fd8cc073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:09.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.693+0000 7fd8d2200640 1 --2- 172.21.15.151:0/3446414992 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8cc0cccd0 0x7fd8cc073440 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8009830 tx=0x7fd8b802a480 comp rx=0 tx=0).stop 2024-09-16T15:48:09.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.693+0000 7fd8d2200640 1 -- 172.21.15.151:0/3446414992 shutdown_connections 2024-09-16T15:48:09.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.693+0000 7fd8d2200640 1 --2- 172.21.15.151:0/3446414992 >> v2:172.21.15.167:3300/0 conn(0x7fd8cc06efa0 0x7fd8cc0cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:09.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.694+0000 7fd8d2200640 1 --2- 172.21.15.151:0/3446414992 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8cc0cccd0 0x7fd8cc073440 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:09.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.694+0000 7fd8d2200640 1 -- 172.21.15.151:0/3446414992 >> 172.21.15.151:0/3446414992 conn(0x7fd8cc0b1840 msgr2=0x7fd8cc0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:09.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.694+0000 7fd8d2200640 1 -- 172.21.15.151:0/3446414992 shutdown_connections 2024-09-16T15:48:09.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.694+0000 7fd8d2200640 1 -- 172.21.15.151:0/3446414992 wait complete. 2024-09-16T15:48:09.697 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.695+0000 7fd8d2200640 1 Processor -- start 2024-09-16T15:48:09.697 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.695+0000 7fd8d2200640 1 -- start start 2024-09-16T15:48:09.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.696+0000 7fd8d2200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8cc06efa0 0x7fd8cc1f4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:09.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.696+0000 7fd8d2200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8cc0cccd0 0x7fd8cc1f88d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:09.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.696+0000 7fd8d2200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8cc145750 0x7fd8cc0afe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:09.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.696+0000 7fd8d2200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd8cc0ce480 con 0x7fd8cc0cccd0 2024-09-16T15:48:09.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.696+0000 7fd8d2200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd8cc0ce300 con 0x7fd8cc145750 2024-09-16T15:48:09.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.696+0000 7fd8d2200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fd8cc0ce600 con 0x7fd8cc06efa0 2024-09-16T15:48:09.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.696+0000 7fd8caa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8cc0cccd0 0x7fd8cc1f88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:09.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.697+0000 7fd8caa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8cc0cccd0 0x7fd8cc1f88d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44808/0 (socket says 172.21.15.151:44808) 2024-09-16T15:48:09.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.697+0000 7fd8caa00640 1 -- 172.21.15.151:0/292689824 learned_addr learned my addr 172.21.15.151:0/292689824 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:48:09.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.697+0000 7fd8cb400640 1 --2- 172.21.15.151:0/292689824 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8cc06efa0 0x7fd8cc1f4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:09.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.697+0000 7fd8cbe00640 1 --2- 172.21.15.151:0/292689824 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8cc145750 0x7fd8cc0afe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:09.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.697+0000 7fd8caa00640 1 -- 172.21.15.151:0/292689824 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8cc06efa0 msgr2=0x7fd8cc1f4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:09.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.697+0000 7fd8caa00640 1 --2- 172.21.15.151:0/292689824 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8cc06efa0 0x7fd8cc1f4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:09.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.697+0000 7fd8caa00640 1 -- 172.21.15.151:0/292689824 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8cc145750 msgr2=0x7fd8cc0afe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:09.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.697+0000 7fd8caa00640 1 --2- 172.21.15.151:0/292689824 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8cc145750 0x7fd8cc0afe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:09.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.697+0000 7fd8caa00640 1 -- 172.21.15.151:0/292689824 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8cc1f5390 con 0x7fd8cc0cccd0 2024-09-16T15:48:09.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.697+0000 7fd8caa00640 1 --2- 172.21.15.151:0/292689824 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8cc0cccd0 0x7fd8cc1f88d0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fd8b802aa70 tx=0x7fd8b8032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:09.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.697+0000 7fd8c3400640 1 -- 172.21.15.151:0/292689824 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b8040070 con 0x7fd8cc0cccd0 2024-09-16T15:48:09.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.698+0000 7fd8d2200640 1 -- 172.21.15.151:0/292689824 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8cc1f5670 con 0x7fd8cc0cccd0 2024-09-16T15:48:09.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.698+0000 7fd8c3400640 1 -- 172.21.15.151:0/292689824 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd8b8005310 con 0x7fd8cc0cccd0 2024-09-16T15:48:09.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.698+0000 7fd8c3400640 1 -- 172.21.15.151:0/292689824 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b8005c90 con 0x7fd8cc0cccd0 2024-09-16T15:48:09.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.698+0000 7fd8d2200640 1 -- 172.21.15.151:0/292689824 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8cc0c4e30 con 0x7fd8cc0cccd0 2024-09-16T15:48:09.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.699+0000 7fd8d2200640 1 -- 172.21.15.151:0/292689824 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8cc0c5030 con 0x7fd8cc0cccd0 2024-09-16T15:48:09.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.700+0000 7fd8c3400640 1 -- 172.21.15.151:0/292689824 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 30) ==== 102868+0+0 (secure 0 0 0) 0x7fd8b8039040 con 0x7fd8cc0cccd0 2024-09-16T15:48:09.703 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.701+0000 7fd8c3400640 1 --2- 172.21.15.151:0/292689824 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd8ac0799f0 0x7fd8ac07beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:09.703 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.701+0000 7fd8c3400640 1 -- 172.21.15.151:0/292689824 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5967+0+0 (secure 0 0 0) 0x7fd8b80ba8d0 con 0x7fd8cc0cccd0 2024-09-16T15:48:09.703 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.701+0000 7fd8cb400640 1 --2- 172.21.15.151:0/292689824 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd8ac0799f0 0x7fd8ac07beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:09.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.702+0000 7fd8cb400640 1 --2- 172.21.15.151:0/292689824 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd8ac0799f0 0x7fd8ac07beb0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc0046b0 tx=0x7fd8bc009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:09.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.705+0000 7fd8c3400640 1 -- 172.21.15.151:0/292689824 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd8cc0c5030 con 0x7fd8cc0cccd0 2024-09-16T15:48:09.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:09.930+0000 7fd8d2200640 1 -- 172.21.15.151:0/292689824 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fd8cc0b3db0 con 0x7fd8cc0cccd0 2024-09-16T15:48:10.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:10.511+0000 7fd8c3400640 1 -- 172.21.15.151:0/292689824 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v49) ==== 104+0+0 (secure 0 0 0) 0x7fd8b80845c0 con 0x7fd8cc0cccd0 2024-09-16T15:48:10.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:10.514+0000 7fd8d2200640 1 -- 172.21.15.151:0/292689824 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd8ac0799f0 msgr2=0x7fd8ac07beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:10.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:10.514+0000 7fd8d2200640 1 --2- 172.21.15.151:0/292689824 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd8ac0799f0 0x7fd8ac07beb0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc0046b0 tx=0x7fd8bc009040 comp rx=0 tx=0).stop 2024-09-16T15:48:10.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:10.514+0000 7fd8d2200640 1 -- 172.21.15.151:0/292689824 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8cc0cccd0 msgr2=0x7fd8cc1f88d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:10.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:10.514+0000 7fd8d2200640 1 --2- 172.21.15.151:0/292689824 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8cc0cccd0 0x7fd8cc1f88d0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fd8b802aa70 tx=0x7fd8b8032c10 comp rx=0 tx=0).stop 2024-09-16T15:48:10.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:10.514+0000 7fd8d2200640 1 -- 172.21.15.151:0/292689824 shutdown_connections 2024-09-16T15:48:10.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:10.514+0000 7fd8d2200640 1 --2- 172.21.15.151:0/292689824 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd8ac0799f0 0x7fd8ac07beb0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:10.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:10.514+0000 7fd8d2200640 1 --2- 172.21.15.151:0/292689824 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8cc145750 0x7fd8cc0afe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:10.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:10.514+0000 7fd8d2200640 1 --2- 172.21.15.151:0/292689824 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8cc0cccd0 0x7fd8cc1f88d0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:10.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:10.514+0000 7fd8d2200640 1 --2- 172.21.15.151:0/292689824 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8cc06efa0 0x7fd8cc1f4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:10.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:10.514+0000 7fd8d2200640 1 -- 172.21.15.151:0/292689824 >> 172.21.15.151:0/292689824 conn(0x7fd8cc0b1840 msgr2=0x7fd8cc0cbe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:10.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:10.514+0000 7fd8d2200640 1 -- 172.21.15.151:0/292689824 shutdown_connections 2024-09-16T15:48:10.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:10.514+0000 7fd8d2200640 1 -- 172.21.15.151:0/292689824 wait complete. 2024-09-16T15:48:10.535 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-16T15:48:10.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:10.696+0000 7f4e9b400640 1 -- 172.21.15.151:0/1722119823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e9415cff0 msgr2=0x7f4e9406e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:10.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:10.696+0000 7f4e9b400640 1 --2- 172.21.15.151:0/1722119823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e9415cff0 0x7f4e9406e6f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f4e80009860 tx=0x7f4e8002a540 comp rx=0 tx=0).stop 2024-09-16T15:48:10.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:10.697+0000 7f4e9b400640 1 -- 172.21.15.151:0/1722119823 shutdown_connections 2024-09-16T15:48:10.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:10.697+0000 7f4e9b400640 1 --2- 172.21.15.151:0/1722119823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e9415cff0 0x7f4e9406e6f0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:10.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:10.697+0000 7f4e9b400640 1 --2- 172.21.15.151:0/1722119823 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e94159aa0 0x7f4e940741a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:10.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:10.697+0000 7f4e9b400640 1 -- 172.21.15.151:0/1722119823 >> 172.21.15.151:0/1722119823 conn(0x7f4e940b18e0 msgr2=0x7f4e940b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:10.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:10.697+0000 7f4e9b400640 1 -- 172.21.15.151:0/1722119823 shutdown_connections 2024-09-16T15:48:10.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:10.697+0000 7f4e9b400640 1 -- 172.21.15.151:0/1722119823 wait complete. 2024-09-16T15:48:10.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:10.698+0000 7f4e9b400640 1 Processor -- start 2024-09-16T15:48:10.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:10.698+0000 7f4e9b400640 1 -- start start 2024-09-16T15:48:10.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:10.699+0000 7f4e9b400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e9406f220 0x7f4e940cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:10.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:10.699+0000 7f4e9b400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e94159aa0 0x7f4e94140f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:10.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:10.699+0000 7f4e9b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e9415cff0 0x7f4e94144170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:10.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:10.699+0000 7f4e9b400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4e9406fa60 con 0x7f4e9415cff0 2024-09-16T15:48:10.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:10.699+0000 7f4e9b400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4e9406f8e0 con 0x7f4e94159aa0 2024-09-16T15:48:10.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:10.699+0000 7f4e9b400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f4e940cc8a0 con 0x7f4e9406f220 2024-09-16T15:48:10.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:10.700+0000 7f4e99400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e9415cff0 0x7f4e94144170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:10.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:10.700+0000 7f4e99400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e9415cff0 0x7f4e94144170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44814/0 (socket says 172.21.15.151:44814) 2024-09-16T15:48:10.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:10.700+0000 7f4e99400640 1 -- 172.21.15.151:0/1918775373 learned_addr learned my addr 172.21.15.151:0/1918775373 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:48:10.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:10.700+0000 7f4e98a00640 1 --2- 172.21.15.151:0/1918775373 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e9406f220 0x7f4e940cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:10.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:10.700+0000 7f4e93e00640 1 --2- 172.21.15.151:0/1918775373 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e94159aa0 0x7f4e94140f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:10.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:10.700+0000 7f4e99400640 1 -- 172.21.15.151:0/1918775373 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e9406f220 msgr2=0x7f4e940cdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:10.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:10.700+0000 7f4e99400640 1 --2- 172.21.15.151:0/1918775373 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e9406f220 0x7f4e940cdad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:10.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:10.700+0000 7f4e99400640 1 -- 172.21.15.151:0/1918775373 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e94159aa0 msgr2=0x7f4e94140f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:10.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:10.700+0000 7f4e99400640 1 --2- 172.21.15.151:0/1918775373 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e94159aa0 0x7f4e94140f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:10.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:10.700+0000 7f4e99400640 1 -- 172.21.15.151:0/1918775373 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e940ce190 con 0x7f4e9415cff0 2024-09-16T15:48:10.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:10.701+0000 7f4e99400640 1 --2- 172.21.15.151:0/1918775373 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e9415cff0 0x7f4e94144170 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c00c7a0 tx=0x7f4e8c00cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:10.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:10.701+0000 7f4e90c00640 1 -- 172.21.15.151:0/1918775373 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c003b70 con 0x7f4e9415cff0 2024-09-16T15:48:10.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:10.701+0000 7f4e90c00640 1 -- 172.21.15.151:0/1918775373 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e8c003d10 con 0x7f4e9415cff0 2024-09-16T15:48:10.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:10.701+0000 7f4e90c00640 1 -- 172.21.15.151:0/1918775373 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c005330 con 0x7f4e9415cff0 2024-09-16T15:48:10.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:10.701+0000 7f4e9b400640 1 -- 172.21.15.151:0/1918775373 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e941f4e90 con 0x7f4e9415cff0 2024-09-16T15:48:10.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:10.701+0000 7f4e9b400640 1 -- 172.21.15.151:0/1918775373 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e941f5450 con 0x7f4e9415cff0 2024-09-16T15:48:10.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:10.702+0000 7f4e9b400640 1 -- 172.21.15.151:0/1918775373 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e94069360 con 0x7f4e9415cff0 2024-09-16T15:48:10.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:10.703+0000 7f4e90c00640 1 -- 172.21.15.151:0/1918775373 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 30) ==== 102868+0+0 (secure 0 0 0) 0x7f4e8c0054d0 con 0x7f4e9415cff0 2024-09-16T15:48:10.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:10.704+0000 7f4e90c00640 1 --2- 172.21.15.151:0/1918775373 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4e740799c0 0x7f4e7407be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:10.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:10.704+0000 7f4e90c00640 1 -- 172.21.15.151:0/1918775373 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5967+0+0 (secure 0 0 0) 0x7f4e8c0994e0 con 0x7f4e9415cff0 2024-09-16T15:48:10.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:10.705+0000 7f4e98a00640 1 --2- 172.21.15.151:0/1918775373 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4e740799c0 0x7f4e7407be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:10.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:10.705+0000 7f4e98a00640 1 --2- 172.21.15.151:0/1918775373 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4e740799c0 0x7f4e7407be80 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f4e94158b30 tx=0x7f4e84009260 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:10.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:10.708+0000 7f4e90c00640 1 -- 172.21.15.151:0/1918775373 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4e8c0632b0 con 0x7f4e9415cff0 2024-09-16T15:48:10.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:10.895+0000 7f4e9b400640 1 -- 172.21.15.151:0/1918775373 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4e940741a0 con 0x7f4e9415cff0 2024-09-16T15:48:11.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:11.898+0000 7f4e90c00640 1 -- 172.21.15.151:0/1918775373 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v82) ==== 181+0+0 (secure 0 0 0) 0x7f4e8c062bb0 con 0x7f4e9415cff0 2024-09-16T15:48:11.901 INFO:teuthology.orchestra.run.smithi151.stderr:pool 'cephfs_metadata' removed 2024-09-16T15:48:11.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:11.902+0000 7f4e9b400640 1 -- 172.21.15.151:0/1918775373 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4e740799c0 msgr2=0x7f4e7407be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:11.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:11.902+0000 7f4e9b400640 1 --2- 172.21.15.151:0/1918775373 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4e740799c0 0x7f4e7407be80 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f4e94158b30 tx=0x7f4e84009260 comp rx=0 tx=0).stop 2024-09-16T15:48:11.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:11.902+0000 7f4e9b400640 1 -- 172.21.15.151:0/1918775373 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e9415cff0 msgr2=0x7f4e94144170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:11.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:11.902+0000 7f4e9b400640 1 --2- 172.21.15.151:0/1918775373 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e9415cff0 0x7f4e94144170 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c00c7a0 tx=0x7f4e8c00cba0 comp rx=0 tx=0).stop 2024-09-16T15:48:11.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:11.902+0000 7f4e9b400640 1 -- 172.21.15.151:0/1918775373 shutdown_connections 2024-09-16T15:48:11.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:11.902+0000 7f4e9b400640 1 --2- 172.21.15.151:0/1918775373 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4e740799c0 0x7f4e7407be80 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:11.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:11.902+0000 7f4e9b400640 1 --2- 172.21.15.151:0/1918775373 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e94159aa0 0x7f4e94140f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:11.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:11.902+0000 7f4e9b400640 1 --2- 172.21.15.151:0/1918775373 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e9415cff0 0x7f4e94144170 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:11.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:11.902+0000 7f4e9b400640 1 --2- 172.21.15.151:0/1918775373 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e9406f220 0x7f4e940cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:11.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:11.902+0000 7f4e9b400640 1 -- 172.21.15.151:0/1918775373 >> 172.21.15.151:0/1918775373 conn(0x7f4e940b18e0 msgr2=0x7f4e9415b770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:11.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:11.903+0000 7f4e9b400640 1 -- 172.21.15.151:0/1918775373 shutdown_connections 2024-09-16T15:48:11.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:11.903+0000 7f4e9b400640 1 -- 172.21.15.151:0/1918775373 wait complete. 2024-09-16T15:48:11.922 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-16T15:48:12.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:12.084+0000 7fa15f800640 1 -- 172.21.15.151:0/3923753032 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa15806f220 msgr2=0x7fa15806f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:12.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:12.084+0000 7fa15f800640 1 --2- 172.21.15.151:0/3923753032 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa15806f220 0x7fa15806f6d0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fa150009ca0 tx=0x7fa15002aa30 comp rx=0 tx=0).stop 2024-09-16T15:48:12.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:12.084+0000 7fa15f800640 1 -- 172.21.15.151:0/3923753032 shutdown_connections 2024-09-16T15:48:12.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:12.084+0000 7fa15f800640 1 --2- 172.21.15.151:0/3923753032 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa15806f220 0x7fa15806f6d0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:12.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:12.084+0000 7fa15f800640 1 --2- 172.21.15.151:0/3923753032 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa158159aa0 0x7fa1580741a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:12.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:12.084+0000 7fa15f800640 1 -- 172.21.15.151:0/3923753032 >> 172.21.15.151:0/3923753032 conn(0x7fa1580b18e0 msgr2=0x7fa1580b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:12.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:12.084+0000 7fa15f800640 1 -- 172.21.15.151:0/3923753032 shutdown_connections 2024-09-16T15:48:12.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:12.084+0000 7fa15f800640 1 -- 172.21.15.151:0/3923753032 wait complete. 2024-09-16T15:48:12.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:12.085+0000 7fa15f800640 1 Processor -- start 2024-09-16T15:48:12.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:12.086+0000 7fa15f800640 1 -- start start 2024-09-16T15:48:12.090 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:12.087+0000 7fa15f800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa15806f220 0x7fa1580b3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:12.090 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:12.087+0000 7fa15f800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa158159aa0 0x7fa1580cf2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:12.090 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:12.087+0000 7fa15f800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa15815cff0 0x7fa15814e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:12.090 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:12.087+0000 7fa15f800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa1580cc520 con 0x7fa158159aa0 2024-09-16T15:48:12.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:12.087+0000 7fa15f800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa1580cc3a0 con 0x7fa15815cff0 2024-09-16T15:48:12.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:12.087+0000 7fa15f800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fa1580cc6a0 con 0x7fa15806f220 2024-09-16T15:48:12.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:12.087+0000 7fa157e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa158159aa0 0x7fa1580cf2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:12.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:12.087+0000 7fa157e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa158159aa0 0x7fa1580cf2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44840/0 (socket says 172.21.15.151:44840) 2024-09-16T15:48:12.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:12.087+0000 7fa157e00640 1 -- 172.21.15.151:0/4156835205 learned_addr learned my addr 172.21.15.151:0/4156835205 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:48:12.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:12.087+0000 7fa15ce00640 1 --2- 172.21.15.151:0/4156835205 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa15806f220 0x7fa1580b3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:12.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:12.087+0000 7fa15d800640 1 --2- 172.21.15.151:0/4156835205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa15815cff0 0x7fa15814e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:12.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:12.087+0000 7fa157e00640 1 -- 172.21.15.151:0/4156835205 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa15806f220 msgr2=0x7fa1580b3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:12.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:12.087+0000 7fa157e00640 1 --2- 172.21.15.151:0/4156835205 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa15806f220 0x7fa1580b3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:12.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:12.087+0000 7fa157e00640 1 -- 172.21.15.151:0/4156835205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa15815cff0 msgr2=0x7fa15814e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:12.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:12.087+0000 7fa157e00640 1 --2- 172.21.15.151:0/4156835205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa15815cff0 0x7fa15814e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:12.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:12.087+0000 7fa157e00640 1 -- 172.21.15.151:0/4156835205 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1580d0130 con 0x7fa158159aa0 2024-09-16T15:48:12.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:12.087+0000 7fa15d800640 1 --2- 172.21.15.151:0/4156835205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa15815cff0 0x7fa15814e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:48:12.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:12.088+0000 7fa157e00640 1 --2- 172.21.15.151:0/4156835205 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa158159aa0 0x7fa1580cf2d0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fa14400b3c0 tx=0x7fa14400b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:12.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:12.088+0000 7fa154c00640 1 -- 172.21.15.151:0/4156835205 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa144015020 con 0x7fa158159aa0 2024-09-16T15:48:12.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:12.088+0000 7fa154c00640 1 -- 172.21.15.151:0/4156835205 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa144011920 con 0x7fa158159aa0 2024-09-16T15:48:12.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:12.088+0000 7fa15f800640 1 -- 172.21.15.151:0/4156835205 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1580b3ba0 con 0x7fa158159aa0 2024-09-16T15:48:12.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:12.088+0000 7fa154c00640 1 -- 172.21.15.151:0/4156835205 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa144011db0 con 0x7fa158159aa0 2024-09-16T15:48:12.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:12.088+0000 7fa15f800640 1 -- 172.21.15.151:0/4156835205 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1580cfe50 con 0x7fa158159aa0 2024-09-16T15:48:12.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:12.089+0000 7fa15f800640 1 -- 172.21.15.151:0/4156835205 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1580ccba0 con 0x7fa158159aa0 2024-09-16T15:48:12.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:12.090+0000 7fa154c00640 1 -- 172.21.15.151:0/4156835205 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 30) ==== 102868+0+0 (secure 0 0 0) 0x7fa144018020 con 0x7fa158159aa0 2024-09-16T15:48:12.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:12.091+0000 7fa154c00640 1 --2- 172.21.15.151:0/4156835205 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa1380799f0 0x7fa13807beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:12.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:12.091+0000 7fa154c00640 1 -- 172.21.15.151:0/4156835205 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5500+0+0 (secure 0 0 0) 0x7fa1440994c0 con 0x7fa158159aa0 2024-09-16T15:48:12.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:12.092+0000 7fa15ce00640 1 --2- 172.21.15.151:0/4156835205 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa1380799f0 0x7fa13807beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:12.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:12.093+0000 7fa15ce00640 1 --2- 172.21.15.151:0/4156835205 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa1380799f0 0x7fa13807beb0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fa14800abe0 tx=0x7fa14800a2d0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:12.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:12.095+0000 7fa154c00640 1 -- 172.21.15.151:0/4156835205 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa14401f030 con 0x7fa158159aa0 2024-09-16T15:48:12.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:12.283+0000 7fa15f800640 1 -- 172.21.15.151:0/4156835205 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa158155cb0 con 0x7fa158159aa0 2024-09-16T15:48:12.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:12.906+0000 7fa154c00640 1 -- 172.21.15.151:0/4156835205 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v83) ==== 169+0+0 (secure 0 0 0) 0x7fa144063230 con 0x7fa158159aa0 2024-09-16T15:48:12.909 INFO:teuthology.orchestra.run.smithi151.stderr:pool 'cephfs_data' removed 2024-09-16T15:48:12.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:12.911+0000 7fa15f800640 1 -- 172.21.15.151:0/4156835205 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa1380799f0 msgr2=0x7fa13807beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:12.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:12.911+0000 7fa15f800640 1 --2- 172.21.15.151:0/4156835205 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa1380799f0 0x7fa13807beb0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fa14800abe0 tx=0x7fa14800a2d0 comp rx=0 tx=0).stop 2024-09-16T15:48:12.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:12.911+0000 7fa15f800640 1 -- 172.21.15.151:0/4156835205 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa158159aa0 msgr2=0x7fa1580cf2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:12.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:12.911+0000 7fa15f800640 1 --2- 172.21.15.151:0/4156835205 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa158159aa0 0x7fa1580cf2d0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fa14400b3c0 tx=0x7fa14400b7c0 comp rx=0 tx=0).stop 2024-09-16T15:48:12.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:12.911+0000 7fa15f800640 1 -- 172.21.15.151:0/4156835205 shutdown_connections 2024-09-16T15:48:12.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:12.911+0000 7fa15f800640 1 --2- 172.21.15.151:0/4156835205 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa1380799f0 0x7fa13807beb0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:12.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:12.911+0000 7fa15f800640 1 --2- 172.21.15.151:0/4156835205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa15815cff0 0x7fa15814e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:12.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:12.911+0000 7fa15f800640 1 --2- 172.21.15.151:0/4156835205 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa158159aa0 0x7fa1580cf2d0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:12.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:12.911+0000 7fa15f800640 1 --2- 172.21.15.151:0/4156835205 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa15806f220 0x7fa1580b3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:12.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:12.912+0000 7fa15f800640 1 -- 172.21.15.151:0/4156835205 >> 172.21.15.151:0/4156835205 conn(0x7fa1580b18e0 msgr2=0x7fa1580aebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:12.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:12.912+0000 7fa15f800640 1 -- 172.21.15.151:0/4156835205 shutdown_connections 2024-09-16T15:48:12.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:12.912+0000 7fa15f800640 1 -- 172.21.15.151:0/4156835205 wait complete. 2024-09-16T15:48:12.934 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-16T15:48:13.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.092+0000 7f6ac7e00640 1 -- 172.21.15.151:0/3058952477 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ac80b34b0 msgr2=0x7f6ac80b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:13.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.092+0000 7f6ac7e00640 1 --2- 172.21.15.151:0/3058952477 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ac80b34b0 0x7f6ac80b3940 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f6ac80b6930 tx=0x7f6ab402a4d0 comp rx=0 tx=0).stop 2024-09-16T15:48:13.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.092+0000 7f6ac7e00640 1 -- 172.21.15.151:0/3058952477 shutdown_connections 2024-09-16T15:48:13.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.092+0000 7f6ac7e00640 1 --2- 172.21.15.151:0/3058952477 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ac80b3e80 0x7f6ac80d1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:13.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.092+0000 7f6ac7e00640 1 --2- 172.21.15.151:0/3058952477 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ac80b34b0 0x7f6ac80b3940 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:13.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.092+0000 7f6ac7e00640 1 --2- 172.21.15.151:0/3058952477 >> v2:172.21.15.167:3300/0 conn(0x7f6ac8161440 0x7f6ac80b5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:13.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.092+0000 7f6ac7e00640 1 -- 172.21.15.151:0/3058952477 >> 172.21.15.151:0/3058952477 conn(0x7f6ac80d3df0 msgr2=0x7f6ac80d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:13.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.092+0000 7f6ac7e00640 1 -- 172.21.15.151:0/3058952477 shutdown_connections 2024-09-16T15:48:13.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.092+0000 7f6ac7e00640 1 -- 172.21.15.151:0/3058952477 wait complete. 2024-09-16T15:48:13.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.093+0000 7f6ac7e00640 1 Processor -- start 2024-09-16T15:48:13.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.094+0000 7f6ac7e00640 1 -- start start 2024-09-16T15:48:13.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.094+0000 7f6ac7e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ac80b34b0 0x7f6ac806ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:13.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.094+0000 7f6ac7e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ac80b3e80 0x7f6ac814b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:13.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.094+0000 7f6ac7e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ac8161440 0x7f6ac80d0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:13.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.095+0000 7f6ac7e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6ac80d2420 con 0x7f6ac8161440 2024-09-16T15:48:13.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.095+0000 7f6ac7e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6ac80d22a0 con 0x7f6ac80b3e80 2024-09-16T15:48:13.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.095+0000 7f6ac7e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f6ac80d25a0 con 0x7f6ac80b34b0 2024-09-16T15:48:13.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.095+0000 7f6ac7400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ac8161440 0x7f6ac80d0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:13.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.095+0000 7f6ac7400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ac8161440 0x7f6ac80d0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47442/0 (socket says 172.21.15.151:47442) 2024-09-16T15:48:13.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.095+0000 7f6ac7400640 1 -- 172.21.15.151:0/1923320933 learned_addr learned my addr 172.21.15.151:0/1923320933 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:48:13.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.095+0000 7f6ac6a00640 1 --2- 172.21.15.151:0/1923320933 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ac80b34b0 0x7f6ac806ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:13.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.095+0000 7f6ac6000640 1 --2- 172.21.15.151:0/1923320933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ac80b3e80 0x7f6ac814b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:13.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.095+0000 7f6ac7400640 1 -- 172.21.15.151:0/1923320933 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ac80b34b0 msgr2=0x7f6ac806ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:13.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.095+0000 7f6ac7400640 1 --2- 172.21.15.151:0/1923320933 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ac80b34b0 0x7f6ac806ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:13.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.095+0000 7f6ac7400640 1 -- 172.21.15.151:0/1923320933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ac80b3e80 msgr2=0x7f6ac814b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:13.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.095+0000 7f6ac7400640 1 --2- 172.21.15.151:0/1923320933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ac80b3e80 0x7f6ac814b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:13.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.095+0000 7f6ac7400640 1 -- 172.21.15.151:0/1923320933 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ac80cd0b0 con 0x7f6ac8161440 2024-09-16T15:48:13.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.096+0000 7f6ac7400640 1 --2- 172.21.15.151:0/1923320933 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ac8161440 0x7f6ac80d0a60 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f6ac000e980 tx=0x7f6ac000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:13.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.096+0000 7f6abea00640 1 -- 172.21.15.151:0/1923320933 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac0004d20 con 0x7f6ac8161440 2024-09-16T15:48:13.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.096+0000 7f6abea00640 1 -- 172.21.15.151:0/1923320933 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6ac000cb20 con 0x7f6ac8161440 2024-09-16T15:48:13.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.096+0000 7f6abea00640 1 -- 172.21.15.151:0/1923320933 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac0005020 con 0x7f6ac8161440 2024-09-16T15:48:13.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.096+0000 7f6ac7e00640 1 -- 172.21.15.151:0/1923320933 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ac8157880 con 0x7f6ac8161440 2024-09-16T15:48:13.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.096+0000 7f6ac7e00640 1 -- 172.21.15.151:0/1923320933 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ac80cd630 con 0x7f6ac8161440 2024-09-16T15:48:13.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.097+0000 7f6ac7e00640 1 -- 172.21.15.151:0/1923320933 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ac80d2b30 con 0x7f6ac8161440 2024-09-16T15:48:13.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.098+0000 7f6abea00640 1 -- 172.21.15.151:0/1923320933 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 30) ==== 102868+0+0 (secure 0 0 0) 0x7f6ac00051c0 con 0x7f6ac8161440 2024-09-16T15:48:13.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.099+0000 7f6abea00640 1 --2- 172.21.15.151:0/1923320933 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6aa80799c0 0x7f6aa807be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:13.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.099+0000 7f6abea00640 1 -- 172.21.15.151:0/1923320933 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 4999+0+0 (secure 0 0 0) 0x7f6ac009a080 con 0x7f6ac8161440 2024-09-16T15:48:13.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.099+0000 7f6ac6a00640 1 --2- 172.21.15.151:0/1923320933 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6aa80799c0 0x7f6aa807be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:13.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.100+0000 7f6ac6a00640 1 --2- 172.21.15.151:0/1923320933 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6aa80799c0 0x7f6aa807be80 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f6ab800bb50 tx=0x7f6ab800b2e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:13.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.103+0000 7f6abea00640 1 -- 172.21.15.151:0/1923320933 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6ac0016030 con 0x7f6ac8161440 2024-09-16T15:48:13.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.294+0000 7f6ac7e00640 1 -- 172.21.15.151:0/1923320933 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f6ac80cdc80 con 0x7f6ac8161440 2024-09-16T15:48:13.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.299+0000 7f6abea00640 1 -- 172.21.15.151:0/1923320933 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v40) ==== 71+0+0 (secure 0 0 0) 0x7f6ac00637a0 con 0x7f6ac8161440 2024-09-16T15:48:13.304 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.301+0000 7f6ac7e00640 1 -- 172.21.15.151:0/1923320933 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6aa80799c0 msgr2=0x7f6aa807be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:13.304 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.301+0000 7f6ac7e00640 1 --2- 172.21.15.151:0/1923320933 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6aa80799c0 0x7f6aa807be80 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f6ab800bb50 tx=0x7f6ab800b2e0 comp rx=0 tx=0).stop 2024-09-16T15:48:13.304 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.301+0000 7f6ac7e00640 1 -- 172.21.15.151:0/1923320933 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ac8161440 msgr2=0x7f6ac80d0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:13.304 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.301+0000 7f6ac7e00640 1 --2- 172.21.15.151:0/1923320933 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ac8161440 0x7f6ac80d0a60 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f6ac000e980 tx=0x7f6ac000ee40 comp rx=0 tx=0).stop 2024-09-16T15:48:13.304 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.302+0000 7f6ac7e00640 1 -- 172.21.15.151:0/1923320933 shutdown_connections 2024-09-16T15:48:13.304 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.302+0000 7f6ac7e00640 1 --2- 172.21.15.151:0/1923320933 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6aa80799c0 0x7f6aa807be80 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:13.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.302+0000 7f6ac7e00640 1 --2- 172.21.15.151:0/1923320933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ac80b3e80 0x7f6ac814b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:13.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.302+0000 7f6ac7e00640 1 --2- 172.21.15.151:0/1923320933 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ac8161440 0x7f6ac80d0a60 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:13.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.302+0000 7f6ac7e00640 1 --2- 172.21.15.151:0/1923320933 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6ac80b34b0 0x7f6ac806ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:13.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.302+0000 7f6ac7e00640 1 -- 172.21.15.151:0/1923320933 >> 172.21.15.151:0/1923320933 conn(0x7f6ac80d3df0 msgr2=0x7f6ac80d41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:13.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.302+0000 7f6ac7e00640 1 -- 172.21.15.151:0/1923320933 shutdown_connections 2024-09-16T15:48:13.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.302+0000 7f6ac7e00640 1 -- 172.21.15.151:0/1923320933 wait complete. 2024-09-16T15:48:13.320 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_more_than_cloner_threads' 2024-09-16T15:48:13.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.477+0000 7f2f8fc00640 1 -- 172.21.15.151:0/2064396517 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2f700039e0 msgr2=0x7f2f70003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:13.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.477+0000 7f2f8fc00640 1 --2- 172.21.15.151:0/2064396517 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2f700039e0 0x7f2f70003dc0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c001770 tx=0x7f2f70005830 comp rx=0 tx=0).stop 2024-09-16T15:48:13.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.477+0000 7f2f8dc00640 1 -- 172.21.15.151:0/2064396517 reap_dead start 2024-09-16T15:48:13.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.477+0000 7f2f8fc00640 1 -- 172.21.15.151:0/2064396517 shutdown_connections 2024-09-16T15:48:13.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.477+0000 7f2f8fc00640 1 -- 172.21.15.151:0/2064396517 >> 172.21.15.151:0/2064396517 conn(0x7f2f8806d230 msgr2=0x7f2f880688b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:13.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.477+0000 7f2f8fc00640 1 -- 172.21.15.151:0/2064396517 shutdown_connections 2024-09-16T15:48:13.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.477+0000 7f2f8fc00640 1 -- 172.21.15.151:0/2064396517 wait complete. 2024-09-16T15:48:13.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.478+0000 7f2f8fc00640 1 Processor -- start 2024-09-16T15:48:13.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.479+0000 7f2f8fc00640 1 -- start start 2024-09-16T15:48:13.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.479+0000 7f2f8fc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2f700039e0 0x7f2f8806f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:13.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.479+0000 7f2f8fc00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f88157a40 0x7f2f881f41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:13.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.480+0000 7f2f8fc00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2f881f87c0 0x7f2f8815a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:13.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.480+0000 7f2f8fc00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2f88071260 con 0x7f2f700039e0 2024-09-16T15:48:13.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.480+0000 7f2f8fc00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2f880b79b0 con 0x7f2f88157a40 2024-09-16T15:48:13.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.480+0000 7f2f8fc00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f2f880713e0 con 0x7f2f881f87c0 2024-09-16T15:48:13.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.480+0000 7f2f8d200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2f700039e0 0x7f2f8806f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:13.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.480+0000 7f2f8d200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2f700039e0 0x7f2f8806f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47464/0 (socket says 172.21.15.151:47464) 2024-09-16T15:48:13.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.480+0000 7f2f8d200640 1 -- 172.21.15.151:0/2360671701 learned_addr learned my addr 172.21.15.151:0/2360671701 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:48:13.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.480+0000 7f2f8dc00640 1 --2- 172.21.15.151:0/2360671701 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2f881f87c0 0x7f2f8815a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:13.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.480+0000 7f2f87e00640 1 --2- 172.21.15.151:0/2360671701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f88157a40 0x7f2f881f41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:13.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.480+0000 7f2f8d200640 1 -- 172.21.15.151:0/2360671701 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2f881f87c0 msgr2=0x7f2f8815a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:13.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.480+0000 7f2f8d200640 1 --2- 172.21.15.151:0/2360671701 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2f881f87c0 0x7f2f8815a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:13.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.480+0000 7f2f8d200640 1 -- 172.21.15.151:0/2360671701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f88157a40 msgr2=0x7f2f881f41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:13.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.480+0000 7f2f8d200640 1 --2- 172.21.15.151:0/2360671701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f88157a40 0x7f2f881f41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:13.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.480+0000 7f2f8d200640 1 -- 172.21.15.151:0/2360671701 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f880b3140 con 0x7f2f700039e0 2024-09-16T15:48:13.484 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.480+0000 7f2f8d200640 1 --2- 172.21.15.151:0/2360671701 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2f700039e0 0x7f2f8806f130 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c001770 tx=0x7f2f7c0060e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:13.484 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.481+0000 7f2f84c00640 1 -- 172.21.15.151:0/2360671701 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f7c037040 con 0x7f2f700039e0 2024-09-16T15:48:13.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.481+0000 7f2f84c00640 1 -- 172.21.15.151:0/2360671701 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2f7c041070 con 0x7f2f700039e0 2024-09-16T15:48:13.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.481+0000 7f2f8fc00640 1 -- 172.21.15.151:0/2360671701 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f8806f730 con 0x7f2f700039e0 2024-09-16T15:48:13.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.481+0000 7f2f84c00640 1 -- 172.21.15.151:0/2360671701 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f7c003a30 con 0x7f2f700039e0 2024-09-16T15:48:13.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.481+0000 7f2f8fc00640 1 -- 172.21.15.151:0/2360671701 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f880b33e0 con 0x7f2f700039e0 2024-09-16T15:48:13.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.482+0000 7f2f8fc00640 1 -- 172.21.15.151:0/2360671701 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f8806fc50 con 0x7f2f700039e0 2024-09-16T15:48:13.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.483+0000 7f2f84c00640 1 -- 172.21.15.151:0/2360671701 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 30) ==== 102868+0+0 (secure 0 0 0) 0x7f2f7c003bd0 con 0x7f2f700039e0 2024-09-16T15:48:13.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.484+0000 7f2f84c00640 1 --2- 172.21.15.151:0/2360671701 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2f680799f0 0x7f2f6807beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:13.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.484+0000 7f2f84c00640 1 -- 172.21.15.151:0/2360671701 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 4999+0+0 (secure 0 0 0) 0x7f2f7c0b9940 con 0x7f2f700039e0 2024-09-16T15:48:13.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.485+0000 7f2f87e00640 1 --2- 172.21.15.151:0/2360671701 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2f680799f0 0x7f2f6807beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:13.487 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.485+0000 7f2f87e00640 1 --2- 172.21.15.151:0/2360671701 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2f680799f0 0x7f2f6807beb0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2f8806fe60 tx=0x7f2f74001080 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:13.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.488+0000 7f2f84c00640 1 -- 172.21.15.151:0/2360671701 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2f8806fc50 con 0x7f2f700039e0 2024-09-16T15:48:13.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.757+0000 7f2f8fc00640 1 -- 172.21.15.151:0/2360671701 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_more_than_cloner_threads"]} v 0) -- 0x7f2f880cea60 con 0x7f2f700039e0 2024-09-16T15:48:13.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.908+0000 7f2f84c00640 1 -- 172.21.15.151:0/2360671701 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_clones_more_than_cloner_threads"]}]=0 v398) ==== 167+0+0 (secure 0 0 0) 0x7f2f7c083060 con 0x7f2f700039e0 2024-09-16T15:48:13.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.911+0000 7f2f8fc00640 1 -- 172.21.15.151:0/2360671701 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2f680799f0 msgr2=0x7f2f6807beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:13.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.911+0000 7f2f8fc00640 1 --2- 172.21.15.151:0/2360671701 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2f680799f0 0x7f2f6807beb0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2f8806fe60 tx=0x7f2f74001080 comp rx=0 tx=0).stop 2024-09-16T15:48:13.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.911+0000 7f2f8fc00640 1 -- 172.21.15.151:0/2360671701 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2f700039e0 msgr2=0x7f2f8806f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:13.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.911+0000 7f2f8fc00640 1 --2- 172.21.15.151:0/2360671701 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2f700039e0 0x7f2f8806f130 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c001770 tx=0x7f2f7c0060e0 comp rx=0 tx=0).stop 2024-09-16T15:48:13.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.911+0000 7f2f8fc00640 1 -- 172.21.15.151:0/2360671701 shutdown_connections 2024-09-16T15:48:13.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.911+0000 7f2f8fc00640 1 --2- 172.21.15.151:0/2360671701 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2f680799f0 0x7f2f6807beb0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:13.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.911+0000 7f2f8fc00640 1 --2- 172.21.15.151:0/2360671701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f88157a40 0x7f2f881f41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:13.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.911+0000 7f2f8fc00640 1 --2- 172.21.15.151:0/2360671701 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2f881f87c0 0x7f2f8815a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:13.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.911+0000 7f2f8fc00640 1 --2- 172.21.15.151:0/2360671701 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2f700039e0 0x7f2f8806f130 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:13.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.911+0000 7f2f8fc00640 1 -- 172.21.15.151:0/2360671701 >> 172.21.15.151:0/2360671701 conn(0x7f2f8806d230 msgr2=0x7f2f880688b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:13.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.911+0000 7f2f8fc00640 1 -- 172.21.15.151:0/2360671701 shutdown_connections 2024-09-16T15:48:13.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:13.911+0000 7f2f8fc00640 1 -- 172.21.15.151:0/2360671701 wait complete. 2024-09-16T15:48:13.929 INFO:tasks.cephfs_test_runner:Test that 2 progress bars are printed in output of "ceph status" ... ok 2024-09-16T15:48:13.930 INFO:tasks.cephfs_test_runner:Starting test: test_progress_drops_when_new_jobs_are_added (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that progress indicated by progress bar for ongoing+pending clones 2024-09-16T15:48:13.930 INFO:tasks.cephfs_test_runner:test_progress_drops_when_new_jobs_are_added (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-09-16T15:48:13.930 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:48:14.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.087+0000 7f0118000640 1 -- 172.21.15.151:0/3868733205 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0110073fb0 msgr2=0x7f0110074460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:14.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.087+0000 7f0118000640 1 --2- 172.21.15.151:0/3868733205 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0110073fb0 0x7f0110074460 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f0108005fd0 tx=0x7f010802a4a0 comp rx=0 tx=0).stop 2024-09-16T15:48:14.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.087+0000 7f0118000640 1 -- 172.21.15.151:0/3868733205 shutdown_connections 2024-09-16T15:48:14.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.087+0000 7f0118000640 1 --2- 172.21.15.151:0/3868733205 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0110073fb0 0x7f0110074460 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:14.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.087+0000 7f0118000640 1 --2- 172.21.15.151:0/3868733205 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f011015f5a0 0x7f01100b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:14.090 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.087+0000 7f0118000640 1 -- 172.21.15.151:0/3868733205 >> 172.21.15.151:0/3868733205 conn(0x7f01100b1950 msgr2=0x7f01100b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:14.090 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.087+0000 7f0118000640 1 -- 172.21.15.151:0/3868733205 shutdown_connections 2024-09-16T15:48:14.090 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.087+0000 7f0118000640 1 -- 172.21.15.151:0/3868733205 wait complete. 2024-09-16T15:48:14.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.088+0000 7f0118000640 1 Processor -- start 2024-09-16T15:48:14.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.089+0000 7f0118000640 1 -- start start 2024-09-16T15:48:14.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.089+0000 7f0118000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0110073fb0 0x7f01101425b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:14.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.089+0000 7f0118000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01100ccf00 0x7f01100cb710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:14.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.089+0000 7f0118000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f011015f5a0 0x7f01100b0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:14.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.089+0000 7f0118000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f011006f2e0 con 0x7f011015f5a0 2024-09-16T15:48:14.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.089+0000 7f0118000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f011006f160 con 0x7f01100ccf00 2024-09-16T15:48:14.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.089+0000 7f0118000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f011006f460 con 0x7f0110073fb0 2024-09-16T15:48:14.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.090+0000 7f0116000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f011015f5a0 0x7f01100b0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:14.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.090+0000 7f0116000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f011015f5a0 0x7f01100b0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47478/0 (socket says 172.21.15.151:47478) 2024-09-16T15:48:14.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.090+0000 7f0116000640 1 -- 172.21.15.151:0/3926088124 learned_addr learned my addr 172.21.15.151:0/3926088124 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:48:14.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.090+0000 7f0115600640 1 --2- 172.21.15.151:0/3926088124 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0110073fb0 0x7f01101425b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:14.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.090+0000 7f0114c00640 1 --2- 172.21.15.151:0/3926088124 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01100ccf00 0x7f01100cb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:14.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.090+0000 7f0116000640 1 -- 172.21.15.151:0/3926088124 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0110073fb0 msgr2=0x7f01101425b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:14.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.090+0000 7f0116000640 1 --2- 172.21.15.151:0/3926088124 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0110073fb0 0x7f01101425b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:14.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.090+0000 7f0116000640 1 -- 172.21.15.151:0/3926088124 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01100ccf00 msgr2=0x7f01100cb710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:14.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.090+0000 7f0116000640 1 --2- 172.21.15.151:0/3926088124 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01100ccf00 0x7f01100cb710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:14.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.090+0000 7f0116000640 1 -- 172.21.15.151:0/3926088124 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0110142e50 con 0x7f011015f5a0 2024-09-16T15:48:14.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.090+0000 7f0114c00640 1 --2- 172.21.15.151:0/3926088124 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01100ccf00 0x7f01100cb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:48:14.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.090+0000 7f0116000640 1 --2- 172.21.15.151:0/3926088124 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f011015f5a0 0x7f01100b0290 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f01080099f0 tx=0x7f01080038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:14.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.091+0000 7f010d600640 1 -- 172.21.15.151:0/3926088124 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f010803f070 con 0x7f011015f5a0 2024-09-16T15:48:14.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.091+0000 7f010d600640 1 -- 172.21.15.151:0/3926088124 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f01080047a0 con 0x7f011015f5a0 2024-09-16T15:48:14.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.091+0000 7f010d600640 1 -- 172.21.15.151:0/3926088124 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0108035030 con 0x7f011015f5a0 2024-09-16T15:48:14.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.091+0000 7f0118000640 1 -- 172.21.15.151:0/3926088124 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0110143130 con 0x7f011015f5a0 2024-09-16T15:48:14.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.091+0000 7f0118000640 1 -- 172.21.15.151:0/3926088124 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01101405c0 con 0x7f011015f5a0 2024-09-16T15:48:14.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.092+0000 7f0118000640 1 -- 172.21.15.151:0/3926088124 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f011006f9f0 con 0x7f011015f5a0 2024-09-16T15:48:14.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.093+0000 7f010d600640 1 -- 172.21.15.151:0/3926088124 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 30) ==== 102868+0+0 (secure 0 0 0) 0x7f0108042070 con 0x7f011015f5a0 2024-09-16T15:48:14.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.094+0000 7f010d600640 1 --2- 172.21.15.151:0/3926088124 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f00f00799c0 0x7f00f007be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:14.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.094+0000 7f010d600640 1 -- 172.21.15.151:0/3926088124 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 4999+0+0 (secure 0 0 0) 0x7f01080ba540 con 0x7f011015f5a0 2024-09-16T15:48:14.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.095+0000 7f0115600640 1 --2- 172.21.15.151:0/3926088124 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f00f00799c0 0x7f00f007be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:14.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.095+0000 7f0115600640 1 --2- 172.21.15.151:0/3926088124 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f00f00799c0 0x7f00f007be80 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f0104005fd0 tx=0x7f01040095e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:14.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.098+0000 7f010d600640 1 -- 172.21.15.151:0/3926088124 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0108083cc0 con 0x7f011015f5a0 2024-09-16T15:48:14.287 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.285+0000 7f0118000640 1 -- 172.21.15.151:0/3926088124 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f01100ae710 con 0x7f011015f5a0 2024-09-16T15:48:14.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.286+0000 7f010d600640 1 -- 172.21.15.151:0/3926088124 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+13182 (secure 0 0 0) 0x7f01080886f0 con 0x7f011015f5a0 2024-09-16T15:48:14.288 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:48:14.289 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":83,"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","created":"2024-09-16T15:38:56.348535+0000","modified":"2024-09-16T15:48:12.900472+0000","last_up_change":"2024-09-16T15:39:10.681384+0000","last_in_change":"2024-09-16T15:39:00.544477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:39:11.602252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5524076-a903-4d0a-af6a-caf474c4765d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6809","nonce":3111342741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6811","nonce":3111342741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6815","nonce":3111342741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6813","nonce":3111342741}]},"public_addr":"172.21.15.151:6809/3111342741","cluster_addr":"172.21.15.151:6811/3111342741","heartbeat_back_addr":"172.21.15.151:6815/3111342741","heartbeat_front_addr":"172.21.15.151:6813/3111342741","state":["exists","up"]},{"osd":1,"uuid":"dd25925f-0ae1-494e-8529-4dc205759db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6801","nonce":3368304087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6803","nonce":3368304087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6807","nonce":3368304087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6805","nonce":3368304087}]},"public_addr":"172.21.15.151:6801/3368304087","cluster_addr":"172.21.15.151:6803/3368304087","heartbeat_back_addr":"172.21.15.151:6807/3368304087","heartbeat_front_addr":"172.21.15.151:6805/3368304087","state":["exists","up"]},{"osd":2,"uuid":"c2f18115-013d-4da6-a0fe-cb978f4494b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6823","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6825","nonce":3079373271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6827","nonce":3079373271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6831","nonce":3079373271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6829","nonce":3079373271}]},"public_addr":"172.21.15.151:6825/3079373271","cluster_addr":"172.21.15.151:6827/3079373271","heartbeat_back_addr":"172.21.15.151:6831/3079373271","heartbeat_front_addr":"172.21.15.151:6829/3079373271","state":["exists","up"]},{"osd":3,"uuid":"36edb718-2fcb-4d88-8ddc-c321eed4279d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6817","nonce":1142509041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6819","nonce":1142509041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6824","nonce":1142509041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6821","nonce":1142509041}]},"public_addr":"172.21.15.151:6817/1142509041","cluster_addr":"172.21.15.151:6819/1142509041","heartbeat_back_addr":"172.21.15.151:6824/1142509041","heartbeat_front_addr":"172.21.15.151:6821/1142509041","state":["exists","up"]},{"osd":4,"uuid":"64d33708-cd0a-4236-abe3-1b646e82ab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6811","nonce":2735495271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6813","nonce":2735495271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6817","nonce":2735495271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6815","nonce":2735495271}]},"public_addr":"172.21.15.167:6811/2735495271","cluster_addr":"172.21.15.167:6813/2735495271","heartbeat_back_addr":"172.21.15.167:6817/2735495271","heartbeat_front_addr":"172.21.15.167:6815/2735495271","state":["exists","up"]},{"osd":5,"uuid":"f14b44a9-5910-458c-b518-09a8d0de94de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6827","nonce":1494916572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6829","nonce":1494916572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6833","nonce":1494916572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6831","nonce":1494916572}]},"public_addr":"172.21.15.167:6827/1494916572","cluster_addr":"172.21.15.167:6829/1494916572","heartbeat_back_addr":"172.21.15.167:6833/1494916572","heartbeat_front_addr":"172.21.15.167:6831/1494916572","state":["exists","up"]},{"osd":6,"uuid":"39335545-2654-4710-84b0-a9aa47cfcbba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6819","nonce":2538273273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6821","nonce":2538273273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6825","nonce":2538273273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6823","nonce":2538273273}]},"public_addr":"172.21.15.167:6819/2538273273","cluster_addr":"172.21.15.167:6821/2538273273","heartbeat_back_addr":"172.21.15.167:6825/2538273273","heartbeat_front_addr":"172.21.15.167:6823/2538273273","state":["exists","up"]},{"osd":7,"uuid":"4f8bfc81-53dd-4339-ac22-1a4007d9f963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6803","nonce":265072227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6805","nonce":265072227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6809","nonce":265072227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6807","nonce":265072227}]},"public_addr":"172.21.15.167:6803/265072227","cluster_addr":"172.21.15.167:6805/265072227","heartbeat_back_addr":"172.21.15.167:6809/265072227","heartbeat_front_addr":"172.21.15.167:6807/265072227","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.150359+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.084358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.722368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.332809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.320709+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.395153+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.935250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.801443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.151:6835/3158835036":"2024-09-17T15:48:09.509102+0000","172.21.15.151:6833/3158835036":"2024-09-17T15:48:09.509102+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:48:14.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.288+0000 7f0118000640 1 -- 172.21.15.151:0/3926088124 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f00f00799c0 msgr2=0x7f00f007be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:14.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.288+0000 7f0118000640 1 --2- 172.21.15.151:0/3926088124 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f00f00799c0 0x7f00f007be80 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f0104005fd0 tx=0x7f01040095e0 comp rx=0 tx=0).stop 2024-09-16T15:48:14.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.288+0000 7f0118000640 1 -- 172.21.15.151:0/3926088124 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f011015f5a0 msgr2=0x7f01100b0290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:14.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.288+0000 7f0118000640 1 --2- 172.21.15.151:0/3926088124 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f011015f5a0 0x7f01100b0290 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f01080099f0 tx=0x7f01080038a0 comp rx=0 tx=0).stop 2024-09-16T15:48:14.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.289+0000 7f0118000640 1 -- 172.21.15.151:0/3926088124 shutdown_connections 2024-09-16T15:48:14.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.289+0000 7f0118000640 1 --2- 172.21.15.151:0/3926088124 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f00f00799c0 0x7f00f007be80 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:14.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.289+0000 7f0118000640 1 --2- 172.21.15.151:0/3926088124 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01100ccf00 0x7f01100cb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:14.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.289+0000 7f0118000640 1 --2- 172.21.15.151:0/3926088124 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f011015f5a0 0x7f01100b0290 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:14.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.289+0000 7f0118000640 1 --2- 172.21.15.151:0/3926088124 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0110073fb0 0x7f01101425b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:14.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.289+0000 7f0118000640 1 -- 172.21.15.151:0/3926088124 >> 172.21.15.151:0/3926088124 conn(0x7f01100b1950 msgr2=0x7f011015e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:14.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.289+0000 7f0118000640 1 -- 172.21.15.151:0/3926088124 shutdown_connections 2024-09-16T15:48:14.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.289+0000 7f0118000640 1 -- 172.21.15.151:0/3926088124 wait complete. 2024-09-16T15:48:14.306 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T15:39:11.602252+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-16T15:48:14.307 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-16T15:48:14.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.465+0000 7faedf400640 1 -- 172.21.15.151:0/2349625638 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faed806efa0 msgr2=0x7faed8073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:14.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.465+0000 7faedf400640 1 --2- 172.21.15.151:0/2349625638 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faed806efa0 0x7faed8073440 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7faed0005fd0 tx=0x7faed002a820 comp rx=0 tx=0).stop 2024-09-16T15:48:14.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.466+0000 7faedf400640 1 -- 172.21.15.151:0/2349625638 shutdown_connections 2024-09-16T15:48:14.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.466+0000 7faedf400640 1 --2- 172.21.15.151:0/2349625638 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faed806efa0 0x7faed8073440 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:14.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.466+0000 7faedf400640 1 -- 172.21.15.151:0/2349625638 >> 172.21.15.151:0/2349625638 conn(0x7faed80b1840 msgr2=0x7faed80b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:14.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.466+0000 7faedf400640 1 -- 172.21.15.151:0/2349625638 shutdown_connections 2024-09-16T15:48:14.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.466+0000 7faedf400640 1 -- 172.21.15.151:0/2349625638 wait complete. 2024-09-16T15:48:14.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.467+0000 7faedf400640 1 Processor -- start 2024-09-16T15:48:14.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.467+0000 7faedf400640 1 -- start start 2024-09-16T15:48:14.471 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.468+0000 7faedf400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faed806efa0 0x7faed80c4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:14.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.468+0000 7faedf400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faed80cccd0 0x7faed8140ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:14.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.468+0000 7faedf400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faed8145750 0x7faed80b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:14.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.468+0000 7faedf400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7faed80ce6d0 con 0x7faed80cccd0 2024-09-16T15:48:14.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.468+0000 7faedf400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7faed80ce550 con 0x7faed806efa0 2024-09-16T15:48:14.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.468+0000 7faedf400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7faed80ce850 con 0x7faed8145750 2024-09-16T15:48:14.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.469+0000 7faecfe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faed80cccd0 0x7faed8140ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:14.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.469+0000 7faecfe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faed80cccd0 0x7faed8140ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47490/0 (socket says 172.21.15.151:47490) 2024-09-16T15:48:14.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.469+0000 7faecfe00640 1 -- 172.21.15.151:0/3661606883 learned_addr learned my addr 172.21.15.151:0/3661606883 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:48:14.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.469+0000 7faedd400640 1 --2- 172.21.15.151:0/3661606883 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faed8145750 0x7faed80b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:14.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.469+0000 7faedca00640 1 --2- 172.21.15.151:0/3661606883 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faed806efa0 0x7faed80c4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:14.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.469+0000 7faecfe00640 1 -- 172.21.15.151:0/3661606883 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faed8145750 msgr2=0x7faed80b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:14.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.469+0000 7faecfe00640 1 --2- 172.21.15.151:0/3661606883 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faed8145750 0x7faed80b31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:14.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.469+0000 7faecfe00640 1 -- 172.21.15.151:0/3661606883 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faed806efa0 msgr2=0x7faed80c4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:14.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.469+0000 7faecfe00640 1 --2- 172.21.15.151:0/3661606883 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faed806efa0 0x7faed80c4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:14.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.469+0000 7faecfe00640 1 -- 172.21.15.151:0/3661606883 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faed8141650 con 0x7faed80cccd0 2024-09-16T15:48:14.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.469+0000 7faecfe00640 1 --2- 172.21.15.151:0/3661606883 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faed80cccd0 0x7faed8140ee0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7faec000c460 tx=0x7faec000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:14.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.470+0000 7faed5600640 1 -- 172.21.15.151:0/3661606883 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec0016020 con 0x7faed80cccd0 2024-09-16T15:48:14.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.470+0000 7faed5600640 1 -- 172.21.15.151:0/3661606883 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faec0012960 con 0x7faed80cccd0 2024-09-16T15:48:14.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.470+0000 7faed5600640 1 -- 172.21.15.151:0/3661606883 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec0012db0 con 0x7faed80cccd0 2024-09-16T15:48:14.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.470+0000 7faedf400640 1 -- 172.21.15.151:0/3661606883 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faed8141930 con 0x7faed80cccd0 2024-09-16T15:48:14.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.470+0000 7faedf400640 1 -- 172.21.15.151:0/3661606883 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faed8141ef0 con 0x7faed80cccd0 2024-09-16T15:48:14.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.471+0000 7faedf400640 1 -- 172.21.15.151:0/3661606883 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faed80ced20 con 0x7faed80cccd0 2024-09-16T15:48:14.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.472+0000 7faed5600640 1 -- 172.21.15.151:0/3661606883 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 30) ==== 102868+0+0 (secure 0 0 0) 0x7faec0019020 con 0x7faed80cccd0 2024-09-16T15:48:14.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.473+0000 7faed5600640 1 --2- 172.21.15.151:0/3661606883 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7faeb40799c0 0x7faeb407be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:14.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.473+0000 7faed5600640 1 -- 172.21.15.151:0/3661606883 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 4999+0+0 (secure 0 0 0) 0x7faec0099570 con 0x7faed80cccd0 2024-09-16T15:48:14.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.473+0000 7faedca00640 1 --2- 172.21.15.151:0/3661606883 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7faeb40799c0 0x7faeb407be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:14.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.474+0000 7faedca00640 1 --2- 172.21.15.151:0/3661606883 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7faeb40799c0 0x7faeb407be80 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7faec8007fd0 tx=0x7faec8009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:14.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.477+0000 7faed5600640 1 -- 172.21.15.151:0/3661606883 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7faec0020030 con 0x7faed80cccd0 2024-09-16T15:48:14.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.661+0000 7faedf400640 1 -- 172.21.15.151:0/3661606883 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7faed806abf0 con 0x7faed80cccd0 2024-09-16T15:48:14.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.662+0000 7faed5600640 1 -- 172.21.15.151:0/3661606883 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) ==== 93+0+10 (secure 0 0 0) 0x7faec00632e0 con 0x7faed80cccd0 2024-09-16T15:48:14.664 INFO:teuthology.orchestra.run.smithi151.stdout:pg_num: 1 2024-09-16T15:48:14.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.664+0000 7faedf400640 1 -- 172.21.15.151:0/3661606883 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7faeb40799c0 msgr2=0x7faeb407be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:14.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.664+0000 7faedf400640 1 --2- 172.21.15.151:0/3661606883 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7faeb40799c0 0x7faeb407be80 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7faec8007fd0 tx=0x7faec8009040 comp rx=0 tx=0).stop 2024-09-16T15:48:14.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.664+0000 7faedf400640 1 -- 172.21.15.151:0/3661606883 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faed80cccd0 msgr2=0x7faed8140ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:14.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.664+0000 7faedf400640 1 --2- 172.21.15.151:0/3661606883 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faed80cccd0 0x7faed8140ee0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7faec000c460 tx=0x7faec000c920 comp rx=0 tx=0).stop 2024-09-16T15:48:14.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.664+0000 7faedf400640 1 -- 172.21.15.151:0/3661606883 shutdown_connections 2024-09-16T15:48:14.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.664+0000 7faedf400640 1 --2- 172.21.15.151:0/3661606883 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7faeb40799c0 0x7faeb407be80 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:14.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.664+0000 7faedf400640 1 --2- 172.21.15.151:0/3661606883 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faed8145750 0x7faed80b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:14.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.664+0000 7faedf400640 1 --2- 172.21.15.151:0/3661606883 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faed80cccd0 0x7faed8140ee0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:14.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.664+0000 7faedf400640 1 --2- 172.21.15.151:0/3661606883 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faed806efa0 0x7faed80c4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:14.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.664+0000 7faedf400640 1 -- 172.21.15.151:0/3661606883 >> 172.21.15.151:0/3661606883 conn(0x7faed80b1840 msgr2=0x7faed80b1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:14.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.664+0000 7faedf400640 1 -- 172.21.15.151:0/3661606883 shutdown_connections 2024-09-16T15:48:14.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.664+0000 7faedf400640 1 -- 172.21.15.151:0/3661606883 wait complete. 2024-09-16T15:48:14.683 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_drops_when_new_jobs_are_added' 2024-09-16T15:48:14.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.839+0000 7f97e9000640 1 -- 172.21.15.151:0/1198552109 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f97e4073fb0 msgr2=0x7f97e4074440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:14.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.839+0000 7f97e9000640 1 --2- 172.21.15.151:0/1198552109 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f97e4073fb0 0x7f97e4074440 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f97dc005fd0 tx=0x7f97dc02aa10 comp rx=0 tx=0).stop 2024-09-16T15:48:14.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.839+0000 7f97e9000640 1 -- 172.21.15.151:0/1198552109 shutdown_connections 2024-09-16T15:48:14.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.839+0000 7f97e9000640 1 --2- 172.21.15.151:0/1198552109 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f97e4073fb0 0x7f97e4074440 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:14.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.839+0000 7f97e9000640 1 --2- 172.21.15.151:0/1198552109 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97e40ccf00 0x7f97e40b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:14.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.839+0000 7f97e9000640 1 -- 172.21.15.151:0/1198552109 >> 172.21.15.151:0/1198552109 conn(0x7f97e40b1950 msgr2=0x7f97e40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:14.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.839+0000 7f97e9000640 1 -- 172.21.15.151:0/1198552109 shutdown_connections 2024-09-16T15:48:14.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.839+0000 7f97e9000640 1 -- 172.21.15.151:0/1198552109 wait complete. 2024-09-16T15:48:14.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.841+0000 7f97e9000640 1 Processor -- start 2024-09-16T15:48:14.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.841+0000 7f97e9000640 1 -- start start 2024-09-16T15:48:14.845 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.842+0000 7f97e9000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97e4073fb0 0x7f97e41f3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:14.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.842+0000 7f97e9000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97e40ccf00 0x7f97e41f5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:14.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.842+0000 7f97e9000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f97e415f5a0 0x7f97e40cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:14.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.842+0000 7f97e9000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f97e406f2e0 con 0x7f97e415f5a0 2024-09-16T15:48:14.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.842+0000 7f97e9000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f97e406f160 con 0x7f97e4073fb0 2024-09-16T15:48:14.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.842+0000 7f97e9000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f97e406f460 con 0x7f97e40ccf00 2024-09-16T15:48:14.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.842+0000 7f97e2800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f97e415f5a0 0x7f97e40cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:14.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.842+0000 7f97e2800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f97e415f5a0 0x7f97e40cffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47498/0 (socket says 172.21.15.151:47498) 2024-09-16T15:48:14.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.842+0000 7f97e2800640 1 -- 172.21.15.151:0/1525690447 learned_addr learned my addr 172.21.15.151:0/1525690447 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:48:14.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.843+0000 7f97e1400640 1 --2- 172.21.15.151:0/1525690447 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97e40ccf00 0x7f97e41f5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:14.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.843+0000 7f97e1e00640 1 --2- 172.21.15.151:0/1525690447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97e4073fb0 0x7f97e41f3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:14.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.843+0000 7f97e2800640 1 -- 172.21.15.151:0/1525690447 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97e40ccf00 msgr2=0x7f97e41f5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:14.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.843+0000 7f97e2800640 1 --2- 172.21.15.151:0/1525690447 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97e40ccf00 0x7f97e41f5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:14.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.843+0000 7f97e2800640 1 -- 172.21.15.151:0/1525690447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97e4073fb0 msgr2=0x7f97e41f3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:14.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.843+0000 7f97e2800640 1 --2- 172.21.15.151:0/1525690447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97e4073fb0 0x7f97e41f3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:14.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.843+0000 7f97e2800640 1 -- 172.21.15.151:0/1525690447 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97e41590f0 con 0x7f97e415f5a0 2024-09-16T15:48:14.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.843+0000 7f97e1400640 1 --2- 172.21.15.151:0/1525690447 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97e40ccf00 0x7f97e41f5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:48:14.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.843+0000 7f97e2800640 1 --2- 172.21.15.151:0/1525690447 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f97e415f5a0 0x7f97e40cffd0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f97dc009d90 tx=0x7f97dc003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:14.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.843+0000 7f97da000640 1 -- 172.21.15.151:0/1525690447 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97dc047070 con 0x7f97e415f5a0 2024-09-16T15:48:14.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.843+0000 7f97da000640 1 -- 172.21.15.151:0/1525690447 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f97dc004250 con 0x7f97e415f5a0 2024-09-16T15:48:14.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.843+0000 7f97e9000640 1 -- 172.21.15.151:0/1525690447 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97e41f3670 con 0x7f97e415f5a0 2024-09-16T15:48:14.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.844+0000 7f97da000640 1 -- 172.21.15.151:0/1525690447 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97dc033050 con 0x7f97e415f5a0 2024-09-16T15:48:14.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.844+0000 7f97e9000640 1 -- 172.21.15.151:0/1525690447 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97e415ba70 con 0x7f97e415f5a0 2024-09-16T15:48:14.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.845+0000 7f97e9000640 1 -- 172.21.15.151:0/1525690447 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97e415c510 con 0x7f97e415f5a0 2024-09-16T15:48:14.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.846+0000 7f97da000640 1 -- 172.21.15.151:0/1525690447 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 30) ==== 102868+0+0 (secure 0 0 0) 0x7f97dc04a070 con 0x7f97e415f5a0 2024-09-16T15:48:14.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.847+0000 7f97da000640 1 --2- 172.21.15.151:0/1525690447 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f97c40799c0 0x7f97c407be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:14.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.847+0000 7f97da000640 1 -- 172.21.15.151:0/1525690447 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 4999+0+0 (secure 0 0 0) 0x7f97dc0ba980 con 0x7f97e415f5a0 2024-09-16T15:48:14.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.847+0000 7f97e1e00640 1 --2- 172.21.15.151:0/1525690447 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f97c40799c0 0x7f97c407be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:14.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.848+0000 7f97e1e00640 1 --2- 172.21.15.151:0/1525690447 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f97c40799c0 0x7f97c407be80 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f97e40cdf40 tx=0x7f97d4009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:14.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:14.851+0000 7f97da000640 1 -- 172.21.15.151:0/1525690447 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f97e415c510 con 0x7f97e415f5a0 2024-09-16T15:48:15.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:15.107+0000 7f97e9000640 1 -- 172.21.15.151:0/1525690447 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_drops_when_new_jobs_are_added"]} v 0) -- 0x7f97e40ae9f0 con 0x7f97e415f5a0 2024-09-16T15:48:15.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:15.917+0000 7f97da000640 1 -- 172.21.15.151:0/1525690447 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_drops_when_new_jobs_are_added"]}]=0 v400) ==== 177+0+0 (secure 0 0 0) 0x7f97e41f3670 con 0x7f97e415f5a0 2024-09-16T15:48:15.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:15.920+0000 7f97e9000640 1 -- 172.21.15.151:0/1525690447 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f97c40799c0 msgr2=0x7f97c407be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:15.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:15.920+0000 7f97e9000640 1 --2- 172.21.15.151:0/1525690447 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f97c40799c0 0x7f97c407be80 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f97e40cdf40 tx=0x7f97d4009040 comp rx=0 tx=0).stop 2024-09-16T15:48:15.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:15.920+0000 7f97e9000640 1 -- 172.21.15.151:0/1525690447 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f97e415f5a0 msgr2=0x7f97e40cffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:15.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:15.920+0000 7f97e9000640 1 --2- 172.21.15.151:0/1525690447 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f97e415f5a0 0x7f97e40cffd0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f97dc009d90 tx=0x7f97dc003f10 comp rx=0 tx=0).stop 2024-09-16T15:48:15.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:15.920+0000 7f97e9000640 1 -- 172.21.15.151:0/1525690447 shutdown_connections 2024-09-16T15:48:15.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:15.920+0000 7f97e9000640 1 --2- 172.21.15.151:0/1525690447 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f97c40799c0 0x7f97c407be80 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:15.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:15.920+0000 7f97e9000640 1 --2- 172.21.15.151:0/1525690447 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f97e415f5a0 0x7f97e40cffd0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:15.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:15.920+0000 7f97e9000640 1 --2- 172.21.15.151:0/1525690447 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f97e40ccf00 0x7f97e41f5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:15.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:15.920+0000 7f97e9000640 1 --2- 172.21.15.151:0/1525690447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97e4073fb0 0x7f97e41f3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:15.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:15.920+0000 7f97e9000640 1 -- 172.21.15.151:0/1525690447 >> 172.21.15.151:0/1525690447 conn(0x7f97e40b1950 msgr2=0x7f97e409d300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:15.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:15.920+0000 7f97e9000640 1 -- 172.21.15.151:0/1525690447 shutdown_connections 2024-09-16T15:48:15.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:15.920+0000 7f97e9000640 1 -- 172.21.15.151:0/1525690447 wait complete. 2024-09-16T15:48:15.940 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-16T15:48:16.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.097+0000 7f1b99200640 1 -- 172.21.15.151:0/3688568796 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1b940ad270 msgr2=0x7f1b94073fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:16.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.097+0000 7f1b99200640 1 --2- 172.21.15.151:0/3688568796 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1b940ad270 0x7f1b94073fb0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f1b940723f0 tx=0x7f1b8c02a980 comp rx=0 tx=0).stop 2024-09-16T15:48:16.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.097+0000 7f1b99200640 1 -- 172.21.15.151:0/3688568796 shutdown_connections 2024-09-16T15:48:16.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.097+0000 7f1b99200640 1 --2- 172.21.15.151:0/3688568796 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1b940ad270 0x7f1b94073fb0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:16.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.097+0000 7f1b99200640 1 --2- 172.21.15.151:0/3688568796 >> v2:172.21.15.167:3300/0 conn(0x7f1b940ccf00 0x7f1b940acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:16.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.097+0000 7f1b99200640 1 --2- 172.21.15.151:0/3688568796 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1b9415f5a0 0x7f1b940b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:16.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.097+0000 7f1b99200640 1 -- 172.21.15.151:0/3688568796 >> 172.21.15.151:0/3688568796 conn(0x7f1b940b1950 msgr2=0x7f1b940b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:16.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.097+0000 7f1b99200640 1 -- 172.21.15.151:0/3688568796 shutdown_connections 2024-09-16T15:48:16.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.097+0000 7f1b99200640 1 -- 172.21.15.151:0/3688568796 wait complete. 2024-09-16T15:48:16.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.099+0000 7f1b99200640 1 Processor -- start 2024-09-16T15:48:16.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.099+0000 7f1b99200640 1 -- start start 2024-09-16T15:48:16.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.100+0000 7f1b99200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1b940ad270 0x7f1b940d0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:16.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.100+0000 7f1b99200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b940ccf00 0x7f1b941f27b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:16.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.100+0000 7f1b99200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1b9415f5a0 0x7f1b94156530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:16.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.100+0000 7f1b99200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1b9406f160 con 0x7f1b9415f5a0 2024-09-16T15:48:16.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.100+0000 7f1b99200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1b94074780 con 0x7f1b940ccf00 2024-09-16T15:48:16.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.100+0000 7f1b99200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f1b9406f2e0 con 0x7f1b940ad270 2024-09-16T15:48:16.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.100+0000 7f1b92800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1b9415f5a0 0x7f1b94156530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:16.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.100+0000 7f1b92800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1b9415f5a0 0x7f1b94156530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47506/0 (socket says 172.21.15.151:47506) 2024-09-16T15:48:16.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.100+0000 7f1b92800640 1 -- 172.21.15.151:0/442480332 learned_addr learned my addr 172.21.15.151:0/442480332 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:48:16.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.100+0000 7f1b91e00640 1 --2- 172.21.15.151:0/442480332 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1b940ad270 0x7f1b940d0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:16.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.100+0000 7f1b91400640 1 --2- 172.21.15.151:0/442480332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b940ccf00 0x7f1b941f27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:16.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.101+0000 7f1b92800640 1 -- 172.21.15.151:0/442480332 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1b940ad270 msgr2=0x7f1b940d0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:16.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.101+0000 7f1b92800640 1 --2- 172.21.15.151:0/442480332 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1b940ad270 0x7f1b940d0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:16.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.101+0000 7f1b92800640 1 -- 172.21.15.151:0/442480332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b940ccf00 msgr2=0x7f1b941f27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:16.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.101+0000 7f1b92800640 1 --2- 172.21.15.151:0/442480332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b940ccf00 0x7f1b941f27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:16.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.101+0000 7f1b92800640 1 -- 172.21.15.151:0/442480332 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b941f2ff0 con 0x7f1b9415f5a0 2024-09-16T15:48:16.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.101+0000 7f1b92800640 1 --2- 172.21.15.151:0/442480332 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1b9415f5a0 0x7f1b94156530 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c009d60 tx=0x7f1b8c003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:16.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.101+0000 7f1b8a000640 1 -- 172.21.15.151:0/442480332 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c034050 con 0x7f1b9415f5a0 2024-09-16T15:48:16.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.101+0000 7f1b8a000640 1 -- 172.21.15.151:0/442480332 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1b8c004450 con 0x7f1b9415f5a0 2024-09-16T15:48:16.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.101+0000 7f1b99200640 1 -- 172.21.15.151:0/442480332 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b941f32d0 con 0x7f1b9415f5a0 2024-09-16T15:48:16.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.101+0000 7f1b8a000640 1 -- 172.21.15.151:0/442480332 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c043440 con 0x7f1b9415f5a0 2024-09-16T15:48:16.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.101+0000 7f1b99200640 1 -- 172.21.15.151:0/442480332 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b941f2cf0 con 0x7f1b9415f5a0 2024-09-16T15:48:16.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.102+0000 7f1b99200640 1 -- 172.21.15.151:0/442480332 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b9406f870 con 0x7f1b9415f5a0 2024-09-16T15:48:16.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.104+0000 7f1b8a000640 1 -- 172.21.15.151:0/442480332 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 30) ==== 102868+0+0 (secure 0 0 0) 0x7f1b8c037020 con 0x7f1b9415f5a0 2024-09-16T15:48:16.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.104+0000 7f1b8a000640 1 --2- 172.21.15.151:0/442480332 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1b740799c0 0x7f1b7407be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:16.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.105+0000 7f1b8a000640 1 -- 172.21.15.151:0/442480332 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 4999+0+0 (secure 0 0 0) 0x7f1b8c0bab80 con 0x7f1b9415f5a0 2024-09-16T15:48:16.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.105+0000 7f1b91e00640 1 --2- 172.21.15.151:0/442480332 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1b740799c0 0x7f1b7407be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:16.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.106+0000 7f1b91e00640 1 --2- 172.21.15.151:0/442480332 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1b740799c0 0x7f1b7407be80 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f1b84004630 tx=0x7f1b8400b2e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:16.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.109+0000 7f1b8a000640 1 -- 172.21.15.151:0/442480332 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1b8c084220 con 0x7f1b9415f5a0 2024-09-16T15:48:16.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.294+0000 7f1b99200640 1 -- 172.21.15.151:0/442480332 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f1b94155cc0 con 0x7f1b9415f5a0 2024-09-16T15:48:16.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.294+0000 7f1b8a000640 1 -- 172.21.15.151:0/442480332 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v40) ==== 59+0+1998 (secure 0 0 0) 0x7f1b8c088c50 con 0x7f1b9415f5a0 2024-09-16T15:48:16.297 INFO:teuthology.orchestra.run.smithi151.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-16T15:48:16.297 INFO:teuthology.orchestra.run.smithi151.stdout:mon advanced mon_op_complaint_time 120 2024-09-16T15:48:16.297 INFO:teuthology.orchestra.run.smithi151.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-16T15:48:16.297 INFO:teuthology.orchestra.run.smithi151.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-16T15:48:16.297 INFO:teuthology.orchestra.run.smithi151.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3795.338492 2024-09-16T15:48:16.297 INFO:teuthology.orchestra.run.smithi151.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 4030.355715 2024-09-16T15:48:16.297 INFO:teuthology.orchestra.run.smithi151.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3817.533640 2024-09-16T15:48:16.297 INFO:teuthology.orchestra.run.smithi151.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3950.656136 2024-09-16T15:48:16.297 INFO:teuthology.orchestra.run.smithi151.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4449.408052 2024-09-16T15:48:16.297 INFO:teuthology.orchestra.run.smithi151.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4439.932296 2024-09-16T15:48:16.298 INFO:teuthology.orchestra.run.smithi151.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4566.875579 2024-09-16T15:48:16.298 INFO:teuthology.orchestra.run.smithi151.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 6320.073632 2024-09-16T15:48:16.298 INFO:teuthology.orchestra.run.smithi151.stdout:mds advanced debug_mds 20/20 2024-09-16T15:48:16.298 INFO:teuthology.orchestra.run.smithi151.stdout:mds advanced debug_mds_balancer 20/20 2024-09-16T15:48:16.298 INFO:teuthology.orchestra.run.smithi151.stdout:mds advanced debug_ms 2/2 2024-09-16T15:48:16.298 INFO:teuthology.orchestra.run.smithi151.stdout:mds dev mds_debug_frag true 2024-09-16T15:48:16.298 INFO:teuthology.orchestra.run.smithi151.stdout:mds dev mds_debug_scatterstat true 2024-09-16T15:48:16.298 INFO:teuthology.orchestra.run.smithi151.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-16T15:48:16.298 INFO:teuthology.orchestra.run.smithi151.stdout:mds dev mds_verify_scatter true 2024-09-16T15:48:16.298 INFO:teuthology.orchestra.run.smithi151.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-16T15:48:16.298 INFO:teuthology.orchestra.run.smithi151.stdout:mds advanced rados_mon_op_timeout 900 2024-09-16T15:48:16.298 INFO:teuthology.orchestra.run.smithi151.stdout:mds advanced rados_osd_op_timeout 900 2024-09-16T15:48:16.298 INFO:teuthology.orchestra.run.smithi151.stdout:client advanced client_mount_timeout 600 2024-09-16T15:48:16.299 INFO:teuthology.orchestra.run.smithi151.stdout:client advanced debug_client 20/20 2024-09-16T15:48:16.299 INFO:teuthology.orchestra.run.smithi151.stdout:client advanced debug_ms 1/1 2024-09-16T15:48:16.299 INFO:teuthology.orchestra.run.smithi151.stdout:client advanced rados_mon_op_timeout 900 2024-09-16T15:48:16.299 INFO:teuthology.orchestra.run.smithi151.stdout:client advanced rados_osd_op_timeout 900 2024-09-16T15:48:16.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.297+0000 7f1b99200640 1 -- 172.21.15.151:0/442480332 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1b740799c0 msgr2=0x7f1b7407be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:16.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.297+0000 7f1b99200640 1 --2- 172.21.15.151:0/442480332 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1b740799c0 0x7f1b7407be80 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f1b84004630 tx=0x7f1b8400b2e0 comp rx=0 tx=0).stop 2024-09-16T15:48:16.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.297+0000 7f1b99200640 1 -- 172.21.15.151:0/442480332 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1b9415f5a0 msgr2=0x7f1b94156530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:16.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.297+0000 7f1b99200640 1 --2- 172.21.15.151:0/442480332 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1b9415f5a0 0x7f1b94156530 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c009d60 tx=0x7f1b8c003f10 comp rx=0 tx=0).stop 2024-09-16T15:48:16.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.297+0000 7f1b99200640 1 -- 172.21.15.151:0/442480332 shutdown_connections 2024-09-16T15:48:16.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.297+0000 7f1b99200640 1 --2- 172.21.15.151:0/442480332 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1b740799c0 0x7f1b7407be80 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:16.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.297+0000 7f1b99200640 1 --2- 172.21.15.151:0/442480332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b940ccf00 0x7f1b941f27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:16.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.297+0000 7f1b99200640 1 --2- 172.21.15.151:0/442480332 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1b9415f5a0 0x7f1b94156530 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:16.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.297+0000 7f1b99200640 1 --2- 172.21.15.151:0/442480332 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1b940ad270 0x7f1b940d0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:16.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.297+0000 7f1b99200640 1 -- 172.21.15.151:0/442480332 >> 172.21.15.151:0/442480332 conn(0x7f1b940b1950 msgr2=0x7f1b9409cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:16.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.297+0000 7f1b99200640 1 -- 172.21.15.151:0/442480332 shutdown_connections 2024-09-16T15:48:16.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.297+0000 7f1b99200640 1 -- 172.21.15.151:0/442480332 wait complete. 2024-09-16T15:48:16.316 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-16T15:48:16.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.474+0000 7f4a10000640 1 -- 172.21.15.151:0/2852855808 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a080cccd0 msgr2=0x7f4a08073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:16.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.474+0000 7f4a10000640 1 --2- 172.21.15.151:0/2852855808 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a080cccd0 0x7f4a08073440 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f49f4009860 tx=0x7f49f402a510 comp rx=0 tx=0).stop 2024-09-16T15:48:16.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.475+0000 7f4a10000640 1 -- 172.21.15.151:0/2852855808 shutdown_connections 2024-09-16T15:48:16.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.475+0000 7f4a10000640 1 --2- 172.21.15.151:0/2852855808 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a080cccd0 0x7f4a08073440 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:16.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.475+0000 7f4a10000640 1 -- 172.21.15.151:0/2852855808 >> 172.21.15.151:0/2852855808 conn(0x7f4a080b1840 msgr2=0x7f4a080b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:16.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.475+0000 7f4a10000640 1 -- 172.21.15.151:0/2852855808 shutdown_connections 2024-09-16T15:48:16.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.475+0000 7f4a10000640 1 -- 172.21.15.151:0/2852855808 wait complete. 2024-09-16T15:48:16.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.476+0000 7f4a10000640 1 Processor -- start 2024-09-16T15:48:16.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.476+0000 7f4a10000640 1 -- start start 2024-09-16T15:48:16.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.477+0000 7f4a10000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a0806efa0 0x7f4a081573b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:16.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.477+0000 7f4a10000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a080cccd0 0x7f4a081578f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:16.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.477+0000 7f4a10000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a08145750 0x7f4a080cb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:16.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.477+0000 7f4a10000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4a080ce6d0 con 0x7f4a0806efa0 2024-09-16T15:48:16.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.477+0000 7f4a10000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4a080ce550 con 0x7f4a08145750 2024-09-16T15:48:16.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.477+0000 7f4a10000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f4a080ce850 con 0x7f4a080cccd0 2024-09-16T15:48:16.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.478+0000 7f4a0d600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a0806efa0 0x7f4a081573b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:16.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.478+0000 7f4a0d600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a0806efa0 0x7f4a081573b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47508/0 (socket says 172.21.15.151:47508) 2024-09-16T15:48:16.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.478+0000 7f4a0d600640 1 -- 172.21.15.151:0/3375658099 learned_addr learned my addr 172.21.15.151:0/3375658099 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:48:16.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.478+0000 7f4a0cc00640 1 --2- 172.21.15.151:0/3375658099 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a080cccd0 0x7f4a081578f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:16.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.478+0000 7f4a0e000640 1 --2- 172.21.15.151:0/3375658099 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a08145750 0x7f4a080cb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:16.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.478+0000 7f4a0d600640 1 -- 172.21.15.151:0/3375658099 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a080cccd0 msgr2=0x7f4a081578f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:16.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.478+0000 7f4a0d600640 1 --2- 172.21.15.151:0/3375658099 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a080cccd0 0x7f4a081578f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:16.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.478+0000 7f4a0d600640 1 -- 172.21.15.151:0/3375658099 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a08145750 msgr2=0x7f4a080cb7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:16.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.478+0000 7f4a0d600640 1 --2- 172.21.15.151:0/3375658099 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a08145750 0x7f4a080cb7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:16.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.478+0000 7f4a0d600640 1 -- 172.21.15.151:0/3375658099 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a080b3b30 con 0x7f4a0806efa0 2024-09-16T15:48:16.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.478+0000 7f4a0e000640 1 --2- 172.21.15.151:0/3375658099 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a08145750 0x7f4a080cb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:48:16.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.478+0000 7f4a0cc00640 1 --2- 172.21.15.151:0/3375658099 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a080cccd0 0x7f4a081578f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:48:16.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.478+0000 7f4a0d600640 1 --2- 172.21.15.151:0/3375658099 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a0806efa0 0x7f4a081573b0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f49fc00c3f0 tx=0x7f49fc00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:16.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.479+0000 7f4a05600640 1 -- 172.21.15.151:0/3375658099 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49fc017070 con 0x7f4a0806efa0 2024-09-16T15:48:16.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.479+0000 7f4a05600640 1 -- 172.21.15.151:0/3375658099 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f49fc0035f0 con 0x7f4a0806efa0 2024-09-16T15:48:16.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.479+0000 7f4a10000640 1 -- 172.21.15.151:0/3375658099 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a080b3e10 con 0x7f4a0806efa0 2024-09-16T15:48:16.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.479+0000 7f4a05600640 1 -- 172.21.15.151:0/3375658099 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49fc003a40 con 0x7f4a0806efa0 2024-09-16T15:48:16.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.479+0000 7f4a10000640 1 -- 172.21.15.151:0/3375658099 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a080b43d0 con 0x7f4a0806efa0 2024-09-16T15:48:16.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.480+0000 7f4a10000640 1 -- 172.21.15.151:0/3375658099 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a080ced00 con 0x7f4a0806efa0 2024-09-16T15:48:16.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.481+0000 7f4a05600640 1 -- 172.21.15.151:0/3375658099 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 30) ==== 102868+0+0 (secure 0 0 0) 0x7f49fc01a070 con 0x7f4a0806efa0 2024-09-16T15:48:16.484 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.482+0000 7f4a05600640 1 --2- 172.21.15.151:0/3375658099 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f49e80799f0 0x7f49e807beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:16.484 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.482+0000 7f4a05600640 1 -- 172.21.15.151:0/3375658099 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 4999+0+0 (secure 0 0 0) 0x7f49fc098b70 con 0x7f4a0806efa0 2024-09-16T15:48:16.484 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.482+0000 7f4a0cc00640 1 --2- 172.21.15.151:0/3375658099 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f49e80799f0 0x7f49e807beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:16.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.483+0000 7f4a0cc00640 1 --2- 172.21.15.151:0/3375658099 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f49e80799f0 0x7f49e807beb0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f49f4034000 tx=0x7f49f4004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:16.488 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.486+0000 7f4a05600640 1 -- 172.21.15.151:0/3375658099 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f49fc062860 con 0x7f4a0806efa0 2024-09-16T15:48:16.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.670+0000 7f4a10000640 1 -- 172.21.15.151:0/3375658099 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f4a080af9a0 con 0x7f4a0806efa0 2024-09-16T15:48:16.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.676+0000 7f4a05600640 1 -- 172.21.15.151:0/3375658099 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v41)=0 v41) ==== 122+0+0 (secure 0 0 0) 0x7f49fc062160 con 0x7f4a0806efa0 2024-09-16T15:48:16.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.678+0000 7f4a10000640 1 -- 172.21.15.151:0/3375658099 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f49e80799f0 msgr2=0x7f49e807beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:16.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.678+0000 7f4a10000640 1 --2- 172.21.15.151:0/3375658099 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f49e80799f0 0x7f49e807beb0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f49f4034000 tx=0x7f49f4004730 comp rx=0 tx=0).stop 2024-09-16T15:48:16.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.678+0000 7f4a10000640 1 -- 172.21.15.151:0/3375658099 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a0806efa0 msgr2=0x7f4a081573b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:16.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.678+0000 7f4a10000640 1 --2- 172.21.15.151:0/3375658099 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a0806efa0 0x7f4a081573b0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f49fc00c3f0 tx=0x7f49fc00c8b0 comp rx=0 tx=0).stop 2024-09-16T15:48:16.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.678+0000 7f4a10000640 1 -- 172.21.15.151:0/3375658099 shutdown_connections 2024-09-16T15:48:16.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.678+0000 7f4a10000640 1 --2- 172.21.15.151:0/3375658099 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f49e80799f0 0x7f49e807beb0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:16.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.678+0000 7f4a10000640 1 --2- 172.21.15.151:0/3375658099 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a08145750 0x7f4a080cb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:16.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.678+0000 7f4a10000640 1 --2- 172.21.15.151:0/3375658099 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4a080cccd0 0x7f4a081578f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:16.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.678+0000 7f4a10000640 1 --2- 172.21.15.151:0/3375658099 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a0806efa0 0x7f4a081573b0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:16.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.678+0000 7f4a10000640 1 -- 172.21.15.151:0/3375658099 >> 172.21.15.151:0/3375658099 conn(0x7f4a080b1840 msgr2=0x7f4a0806f410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:16.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.678+0000 7f4a10000640 1 -- 172.21.15.151:0/3375658099 shutdown_connections 2024-09-16T15:48:16.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:16.678+0000 7f4a10000640 1 -- 172.21.15.151:0/3375658099 wait complete. 2024-09-16T15:48:16.696 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-16T15:48:16.696 DEBUG:teuthology.orchestra.run.smithi151:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:48:16.791 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:48:16.881 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-16T15:48:16.881 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:48:16.881 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:48:16.917 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T15:48:16.917 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-16T15:48:16.917 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:48:16.955 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:48:16.955 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:48:16.955 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:48:16.992 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T15:48:16.992 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-16T15:48:16.992 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:48:17.031 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:48:17.031 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-16T15:48:17.032 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:48:17.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.189+0000 7f28ab200640 1 -- 172.21.15.151:0/2354751864 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f28a415f5a0 msgr2=0x7f28a40b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:17.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.189+0000 7f28ab200640 1 --2- 172.21.15.151:0/2354751864 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f28a415f5a0 0x7f28a40b2440 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2894009830 tx=0x7f289402a480 comp rx=0 tx=0).stop 2024-09-16T15:48:17.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.189+0000 7f28ab200640 1 -- 172.21.15.151:0/2354751864 shutdown_connections 2024-09-16T15:48:17.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.189+0000 7f28ab200640 1 --2- 172.21.15.151:0/2354751864 >> v2:172.21.15.167:3300/0 conn(0x7f28a4073fb0 0x7f28a4074460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:17.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.189+0000 7f28ab200640 1 --2- 172.21.15.151:0/2354751864 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f28a415f5a0 0x7f28a40b2440 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:17.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.189+0000 7f28ab200640 1 -- 172.21.15.151:0/2354751864 >> 172.21.15.151:0/2354751864 conn(0x7f28a40b1950 msgr2=0x7f28a40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:17.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.189+0000 7f28ab200640 1 -- 172.21.15.151:0/2354751864 shutdown_connections 2024-09-16T15:48:17.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.190+0000 7f28ab200640 1 -- 172.21.15.151:0/2354751864 wait complete. 2024-09-16T15:48:17.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.191+0000 7f28ab200640 1 Processor -- start 2024-09-16T15:48:17.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.191+0000 7f28ab200640 1 -- start start 2024-09-16T15:48:17.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.192+0000 7f28ab200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28a4073fb0 0x7f28a4155420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:17.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.192+0000 7f28ab200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f28a40ccf00 0x7f28a41f51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:17.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.192+0000 7f28ab200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28a415f5a0 0x7f28a40b1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:17.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.192+0000 7f28ab200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f28a406f2e0 con 0x7f28a4073fb0 2024-09-16T15:48:17.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.192+0000 7f28ab200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f28a406f160 con 0x7f28a415f5a0 2024-09-16T15:48:17.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.192+0000 7f28ab200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f28a406f460 con 0x7f28a40ccf00 2024-09-16T15:48:17.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.192+0000 7f28a3e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28a4073fb0 0x7f28a4155420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:17.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.192+0000 7f28a3e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28a4073fb0 0x7f28a4155420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47522/0 (socket says 172.21.15.151:47522) 2024-09-16T15:48:17.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.192+0000 7f28a3e00640 1 -- 172.21.15.151:0/1385267289 learned_addr learned my addr 172.21.15.151:0/1385267289 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:48:17.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.193+0000 7f289be00640 1 --2- 172.21.15.151:0/1385267289 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f28a40ccf00 0x7f28a41f51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:17.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.193+0000 7f28a9200640 1 --2- 172.21.15.151:0/1385267289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28a415f5a0 0x7f28a40b1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:17.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.193+0000 7f28a3e00640 1 -- 172.21.15.151:0/1385267289 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f28a40ccf00 msgr2=0x7f28a41f51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:17.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.193+0000 7f28a3e00640 1 --2- 172.21.15.151:0/1385267289 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f28a40ccf00 0x7f28a41f51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:17.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.193+0000 7f28a3e00640 1 -- 172.21.15.151:0/1385267289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28a415f5a0 msgr2=0x7f28a40b1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:17.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.193+0000 7f28a3e00640 1 --2- 172.21.15.151:0/1385267289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28a415f5a0 0x7f28a40b1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:17.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.193+0000 7f28a3e00640 1 -- 172.21.15.151:0/1385267289 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28a40cfaf0 con 0x7f28a4073fb0 2024-09-16T15:48:17.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.193+0000 7f28a3e00640 1 --2- 172.21.15.151:0/1385267289 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28a4073fb0 0x7f28a4155420 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f2894009950 tx=0x7f2894033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:17.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.193+0000 7f28a0c00640 1 -- 172.21.15.151:0/1385267289 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2894005c90 con 0x7f28a4073fb0 2024-09-16T15:48:17.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.193+0000 7f28a0c00640 1 -- 172.21.15.151:0/1385267289 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2894031e50 con 0x7f28a4073fb0 2024-09-16T15:48:17.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.193+0000 7f28a0c00640 1 -- 172.21.15.151:0/1385267289 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2894005260 con 0x7f28a4073fb0 2024-09-16T15:48:17.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.194+0000 7f28ab200640 1 -- 172.21.15.151:0/1385267289 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28a40ca1c0 con 0x7f28a4073fb0 2024-09-16T15:48:17.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.194+0000 7f28ab200640 1 -- 172.21.15.151:0/1385267289 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28a40ca780 con 0x7f28a4073fb0 2024-09-16T15:48:17.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.195+0000 7f28ab200640 1 -- 172.21.15.151:0/1385267289 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28a406f9f0 con 0x7f28a4073fb0 2024-09-16T15:48:17.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.196+0000 7f28a0c00640 1 -- 172.21.15.151:0/1385267289 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 30) ==== 102868+0+0 (secure 0 0 0) 0x7f2894005400 con 0x7f28a4073fb0 2024-09-16T15:48:17.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.197+0000 7f28a0c00640 1 --2- 172.21.15.151:0/1385267289 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f28800799c0 0x7f288007be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:17.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.197+0000 7f28a0c00640 1 -- 172.21.15.151:0/1385267289 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 4999+0+0 (secure 0 0 0) 0x7f28940ba460 con 0x7f28a4073fb0 2024-09-16T15:48:17.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.197+0000 7f289be00640 1 --2- 172.21.15.151:0/1385267289 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f28800799c0 0x7f288007be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:17.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.198+0000 7f289be00640 1 --2- 172.21.15.151:0/1385267289 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f28800799c0 0x7f288007be80 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f288c007fd0 tx=0x7f288c001410 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:17.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.201+0000 7f28a0c00640 1 -- 172.21.15.151:0/1385267289 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f289403f030 con 0x7f28a4073fb0 2024-09-16T15:48:17.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.420+0000 7f28ab200640 1 -- 172.21.15.151:0/1385267289 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f28a40b0650 con 0x7f28a4073fb0 2024-09-16T15:48:17.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.421+0000 7f28a0c00640 1 -- 172.21.15.151:0/1385267289 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+3920 (secure 0 0 0) 0x7f2894083dc0 con 0x7f28a4073fb0 2024-09-16T15:48:17.423 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:48:17.423 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":49,"btime":"2024-09-16T15:48:10:511229+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5705,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/3291846596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3291846596},{"type":"v1","addr":"172.21.15.167:6837","nonce":3291846596}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6175,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/2504229431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":2504229431},{"type":"v1","addr":"172.21.15.151:6834","nonce":2504229431}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6181,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/464376104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":464376104},{"type":"v1","addr":"172.21.15.167:6836","nonce":464376104}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6939,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/3816266630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":3816266630},{"type":"v1","addr":"172.21.15.151:6835","nonce":3816266630}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49}],"filesystems":[]} 2024-09-16T15:48:17.424 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 49 2024-09-16T15:48:17.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.423+0000 7f28ab200640 1 -- 172.21.15.151:0/1385267289 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f28800799c0 msgr2=0x7f288007be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:17.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.423+0000 7f28ab200640 1 --2- 172.21.15.151:0/1385267289 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f28800799c0 0x7f288007be80 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f288c007fd0 tx=0x7f288c001410 comp rx=0 tx=0).stop 2024-09-16T15:48:17.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.423+0000 7f28ab200640 1 -- 172.21.15.151:0/1385267289 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28a4073fb0 msgr2=0x7f28a4155420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:17.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.423+0000 7f28ab200640 1 --2- 172.21.15.151:0/1385267289 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28a4073fb0 0x7f28a4155420 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f2894009950 tx=0x7f2894033c20 comp rx=0 tx=0).stop 2024-09-16T15:48:17.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.423+0000 7f28ab200640 1 -- 172.21.15.151:0/1385267289 shutdown_connections 2024-09-16T15:48:17.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.423+0000 7f28ab200640 1 --2- 172.21.15.151:0/1385267289 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f28800799c0 0x7f288007be80 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:17.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.423+0000 7f28ab200640 1 --2- 172.21.15.151:0/1385267289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28a415f5a0 0x7f28a40b1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:17.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.423+0000 7f28ab200640 1 --2- 172.21.15.151:0/1385267289 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f28a40ccf00 0x7f28a41f51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:17.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.423+0000 7f28ab200640 1 --2- 172.21.15.151:0/1385267289 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28a4073fb0 0x7f28a4155420 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:17.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.423+0000 7f28ab200640 1 -- 172.21.15.151:0/1385267289 >> 172.21.15.151:0/1385267289 conn(0x7f28a40b1950 msgr2=0x7f28a40b4810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:17.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.423+0000 7f28ab200640 1 -- 172.21.15.151:0/1385267289 shutdown_connections 2024-09-16T15:48:17.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.423+0000 7f28ab200640 1 -- 172.21.15.151:0/1385267289 wait complete. 2024-09-16T15:48:17.442 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-16T15:48:17.442 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-16T15:48:17.442 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-16T15:48:17.443 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:48:17.443 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-16T15:48:17.443 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-16T15:48:17.443 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-16T15:48:17.443 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:48:17.443 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-16T15:48:17.443 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-16T15:48:17.444 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-16T15:48:17.444 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:48:17.444 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-16T15:48:17.444 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-16T15:48:17.444 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-16T15:48:17.444 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:48:17.456 INFO:tasks.ceph.mds.a:Stopped 2024-09-16T15:48:17.456 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-16T15:48:17.458 INFO:tasks.ceph.mds.c:Stopped 2024-09-16T15:48:17.458 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-16T15:48:17.465 INFO:tasks.ceph.mds.d:Stopped 2024-09-16T15:48:17.465 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-16T15:48:17.466 INFO:tasks.ceph.mds.b:Stopped 2024-09-16T15:48:17.466 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-16T15:48:17.501 INFO:tasks.ceph.mds.a:Started 2024-09-16T15:48:17.502 DEBUG:teuthology.parallel:result is None 2024-09-16T15:48:17.504 INFO:tasks.ceph.mds.c:Started 2024-09-16T15:48:17.504 DEBUG:teuthology.parallel:result is None 2024-09-16T15:48:17.510 INFO:tasks.ceph.mds.d:Started 2024-09-16T15:48:17.510 DEBUG:teuthology.parallel:result is None 2024-09-16T15:48:17.512 INFO:tasks.ceph.mds.b:Started 2024-09-16T15:48:17.512 DEBUG:teuthology.parallel:result is None 2024-09-16T15:48:17.513 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-16T15:48:17.633 INFO:tasks.ceph.mds.a.smithi151.stdout:starting mds.a at 2024-09-16T15:48:17.633 INFO:tasks.ceph.mds.c.smithi151.stdout:starting mds.c at 2024-09-16T15:48:17.641 INFO:tasks.ceph.mds.d.smithi167.stdout:starting mds.d at 2024-09-16T15:48:17.641 INFO:tasks.ceph.mds.b.smithi167.stdout:starting mds.b at 2024-09-16T15:48:17.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.680+0000 7fe231a00640 1 -- 172.21.15.151:0/538861751 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe22c1438b0 msgr2=0x7fe22c0cc5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:17.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.680+0000 7fe231a00640 1 --2- 172.21.15.151:0/538861751 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe22c1438b0 0x7fe22c0cc5f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe21c00a970 tx=0x7fe21c02b850 comp rx=0 tx=0).stop 2024-09-16T15:48:17.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.680+0000 7fe231a00640 1 -- 172.21.15.151:0/538861751 shutdown_connections 2024-09-16T15:48:17.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.680+0000 7fe231a00640 1 --2- 172.21.15.151:0/538861751 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe22c1438b0 0x7fe22c0cc5f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:17.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.680+0000 7fe231a00640 1 -- 172.21.15.151:0/538861751 >> 172.21.15.151:0/538861751 conn(0x7fe22c0b1950 msgr2=0x7fe22c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:17.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.680+0000 7fe231a00640 1 -- 172.21.15.151:0/538861751 shutdown_connections 2024-09-16T15:48:17.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.680+0000 7fe231a00640 1 -- 172.21.15.151:0/538861751 wait complete. 2024-09-16T15:48:17.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.682+0000 7fe231a00640 1 Processor -- start 2024-09-16T15:48:17.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.682+0000 7fe231a00640 1 -- start start 2024-09-16T15:48:17.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.683+0000 7fe231a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe22c06ff60 0x7fe22c13a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:17.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.683+0000 7fe231a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe22c0ccb30 0x7fe22c1f1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:17.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.683+0000 7fe231a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe22c1438b0 0x7fe22c1f2050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:17.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.683+0000 7fe231a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe22c0b4610 con 0x7fe22c06ff60 2024-09-16T15:48:17.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.683+0000 7fe231a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe22c070b30 con 0x7fe22c0ccb30 2024-09-16T15:48:17.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.683+0000 7fe231a00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fe22c0b4790 con 0x7fe22c1438b0 2024-09-16T15:48:17.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.683+0000 7fe22aa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe22c06ff60 0x7fe22c13a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:17.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.683+0000 7fe22aa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe22c06ff60 0x7fe22c13a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47568/0 (socket says 172.21.15.151:47568) 2024-09-16T15:48:17.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.683+0000 7fe22aa00640 1 -- 172.21.15.151:0/3016055438 learned_addr learned my addr 172.21.15.151:0/3016055438 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:48:17.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.683+0000 7fe223e00640 1 --2- 172.21.15.151:0/3016055438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe22c0ccb30 0x7fe22c1f1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:17.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.683+0000 7fe22b400640 1 --2- 172.21.15.151:0/3016055438 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe22c1438b0 0x7fe22c1f2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:17.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.683+0000 7fe22aa00640 1 -- 172.21.15.151:0/3016055438 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe22c1438b0 msgr2=0x7fe22c1f2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:17.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.683+0000 7fe22aa00640 1 --2- 172.21.15.151:0/3016055438 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe22c1438b0 0x7fe22c1f2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:17.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.683+0000 7fe22aa00640 1 -- 172.21.15.151:0/3016055438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe22c0ccb30 msgr2=0x7fe22c1f1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:17.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.683+0000 7fe22aa00640 1 --2- 172.21.15.151:0/3016055438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe22c0ccb30 0x7fe22c1f1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:17.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.683+0000 7fe22aa00640 1 -- 172.21.15.151:0/3016055438 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe22c09c460 con 0x7fe22c06ff60 2024-09-16T15:48:17.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.684+0000 7fe22aa00640 1 --2- 172.21.15.151:0/3016055438 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe22c06ff60 0x7fe22c13a140 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fe21c00aa60 tx=0x7fe21c0336c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:17.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.684+0000 7fe222a00640 1 -- 172.21.15.151:0/3016055438 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21c042070 con 0x7fe22c06ff60 2024-09-16T15:48:17.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.684+0000 7fe222a00640 1 -- 172.21.15.151:0/3016055438 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe21c033e00 con 0x7fe22c06ff60 2024-09-16T15:48:17.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.684+0000 7fe222a00640 1 -- 172.21.15.151:0/3016055438 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21c005c90 con 0x7fe22c06ff60 2024-09-16T15:48:17.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.684+0000 7fe231a00640 1 -- 172.21.15.151:0/3016055438 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe22c09c740 con 0x7fe22c06ff60 2024-09-16T15:48:17.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.684+0000 7fe231a00640 1 -- 172.21.15.151:0/3016055438 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe22c09cd00 con 0x7fe22c06ff60 2024-09-16T15:48:17.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.685+0000 7fe231a00640 1 -- 172.21.15.151:0/3016055438 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe22c0cc5f0 con 0x7fe22c06ff60 2024-09-16T15:48:17.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.686+0000 7fe222a00640 1 -- 172.21.15.151:0/3016055438 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 30) ==== 102868+0+0 (secure 0 0 0) 0x7fe21c03a040 con 0x7fe22c06ff60 2024-09-16T15:48:17.689 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.687+0000 7fe222a00640 1 --2- 172.21.15.151:0/3016055438 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe2080799c0 0x7fe20807be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:17.689 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.687+0000 7fe222a00640 1 -- 172.21.15.151:0/3016055438 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 4999+0+0 (secure 0 0 0) 0x7fe21c0ba560 con 0x7fe22c06ff60 2024-09-16T15:48:17.689 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.688+0000 7fe223e00640 1 --2- 172.21.15.151:0/3016055438 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe2080799c0 0x7fe20807be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:17.690 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.688+0000 7fe223e00640 1 --2- 172.21.15.151:0/3016055438 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe2080799c0 0x7fe20807be80 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe22c0c5600 tx=0x7fe214001db0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:17.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.691+0000 7fe222a00640 1 -- 172.21.15.151:0/3016055438 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe21c083c00 con 0x7fe22c06ff60 2024-09-16T15:48:17.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.887+0000 7fe231a00640 1 -- 172.21.15.151:0/3016055438 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fe22c0cd430 con 0x7fe22c06ff60 2024-09-16T15:48:17.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.934+0000 7fe222a00640 1 -- 172.21.15.151:0/3016055438 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v85) ==== 97+0+0 (secure 0 0 0) 0x7fe21c088630 con 0x7fe22c06ff60 2024-09-16T15:48:17.936 INFO:teuthology.orchestra.run.smithi151.stderr: removed all blocklist entries 2024-09-16T15:48:17.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.937+0000 7fe231a00640 1 -- 172.21.15.151:0/3016055438 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe2080799c0 msgr2=0x7fe20807be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:17.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.937+0000 7fe231a00640 1 --2- 172.21.15.151:0/3016055438 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe2080799c0 0x7fe20807be80 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe22c0c5600 tx=0x7fe214001db0 comp rx=0 tx=0).stop 2024-09-16T15:48:17.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.937+0000 7fe231a00640 1 -- 172.21.15.151:0/3016055438 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe22c06ff60 msgr2=0x7fe22c13a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:17.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.937+0000 7fe231a00640 1 --2- 172.21.15.151:0/3016055438 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe22c06ff60 0x7fe22c13a140 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fe21c00aa60 tx=0x7fe21c0336c0 comp rx=0 tx=0).stop 2024-09-16T15:48:17.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.937+0000 7fe231a00640 1 -- 172.21.15.151:0/3016055438 shutdown_connections 2024-09-16T15:48:17.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.937+0000 7fe231a00640 1 --2- 172.21.15.151:0/3016055438 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe2080799c0 0x7fe20807be80 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:17.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.937+0000 7fe231a00640 1 --2- 172.21.15.151:0/3016055438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe22c0ccb30 0x7fe22c1f1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:17.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.937+0000 7fe231a00640 1 --2- 172.21.15.151:0/3016055438 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe22c1438b0 0x7fe22c1f2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:17.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.937+0000 7fe231a00640 1 --2- 172.21.15.151:0/3016055438 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe22c06ff60 0x7fe22c13a140 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:17.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.937+0000 7fe231a00640 1 -- 172.21.15.151:0/3016055438 >> 172.21.15.151:0/3016055438 conn(0x7fe22c0b1950 msgr2=0x7fe22c15c770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:17.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.937+0000 7fe231a00640 1 -- 172.21.15.151:0/3016055438 shutdown_connections 2024-09-16T15:48:17.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:17.937+0000 7fe231a00640 1 -- 172.21.15.151:0/3016055438 wait complete. 2024-09-16T15:48:17.957 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-16T15:48:18.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.112+0000 7f2546400640 1 -- 172.21.15.151:0/1305668135 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f254015f5a0 msgr2=0x7f25400b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:18.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.112+0000 7f2546400640 1 --2- 172.21.15.151:0/1305668135 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f254015f5a0 0x7f25400b2440 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f2530005fd0 tx=0x7f253002aa10 comp rx=0 tx=0).stop 2024-09-16T15:48:18.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.112+0000 7f2546400640 1 -- 172.21.15.151:0/1305668135 shutdown_connections 2024-09-16T15:48:18.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.112+0000 7f2546400640 1 --2- 172.21.15.151:0/1305668135 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2540073fb0 0x7f2540074440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:18.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.112+0000 7f2546400640 1 --2- 172.21.15.151:0/1305668135 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f254015f5a0 0x7f25400b2440 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:18.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.112+0000 7f2546400640 1 -- 172.21.15.151:0/1305668135 >> 172.21.15.151:0/1305668135 conn(0x7f25400b1950 msgr2=0x7f25400b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:18.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.112+0000 7f2546400640 1 -- 172.21.15.151:0/1305668135 shutdown_connections 2024-09-16T15:48:18.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.112+0000 7f2546400640 1 -- 172.21.15.151:0/1305668135 wait complete. 2024-09-16T15:48:18.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.114+0000 7f2546400640 1 Processor -- start 2024-09-16T15:48:18.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.114+0000 7f2546400640 1 -- start start 2024-09-16T15:48:18.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.115+0000 7f2546400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2540073fb0 0x7f254014f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:18.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.115+0000 7f2546400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f25400ccf00 0x7f2540151390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:18.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.115+0000 7f2546400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f254015f5a0 0x7f25400cef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:18.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.115+0000 7f2546400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f254006f2e0 con 0x7f25400ccf00 2024-09-16T15:48:18.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.115+0000 7f2546400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f254006f160 con 0x7f254015f5a0 2024-09-16T15:48:18.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.115+0000 7f2546400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f254006f460 con 0x7f2540073fb0 2024-09-16T15:48:18.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.115+0000 7f253ea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f25400ccf00 0x7f2540151390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:18.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.115+0000 7f253f400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2540073fb0 0x7f254014f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:18.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.115+0000 7f253ea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f25400ccf00 0x7f2540151390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47586/0 (socket says 172.21.15.151:47586) 2024-09-16T15:48:18.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.115+0000 7f253ea00640 1 -- 172.21.15.151:0/3870426730 learned_addr learned my addr 172.21.15.151:0/3870426730 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:48:18.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.115+0000 7f253fe00640 1 --2- 172.21.15.151:0/3870426730 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f254015f5a0 0x7f25400cef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:18.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.116+0000 7f253ea00640 1 -- 172.21.15.151:0/3870426730 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2540073fb0 msgr2=0x7f254014f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:18.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.116+0000 7f253ea00640 1 --2- 172.21.15.151:0/3870426730 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2540073fb0 0x7f254014f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:18.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.116+0000 7f253ea00640 1 -- 172.21.15.151:0/3870426730 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f254015f5a0 msgr2=0x7f25400cef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:18.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.116+0000 7f253ea00640 1 --2- 172.21.15.151:0/3870426730 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f254015f5a0 0x7f25400cef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:18.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.116+0000 7f253ea00640 1 -- 172.21.15.151:0/3870426730 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25400c4e10 con 0x7f25400ccf00 2024-09-16T15:48:18.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.116+0000 7f253ea00640 1 --2- 172.21.15.151:0/3870426730 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f25400ccf00 0x7f2540151390 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f252c00c450 tx=0x7f252c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:18.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.116+0000 7f2537400640 1 -- 172.21.15.151:0/3870426730 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f252c017070 con 0x7f25400ccf00 2024-09-16T15:48:18.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.116+0000 7f2537400640 1 -- 172.21.15.151:0/3870426730 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f252c0053c0 con 0x7f25400ccf00 2024-09-16T15:48:18.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.116+0000 7f2537400640 1 -- 172.21.15.151:0/3870426730 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f252c003850 con 0x7f25400ccf00 2024-09-16T15:48:18.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.116+0000 7f2546400640 1 -- 172.21.15.151:0/3870426730 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25400c50f0 con 0x7f25400ccf00 2024-09-16T15:48:18.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.117+0000 7f2546400640 1 -- 172.21.15.151:0/3870426730 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25401518d0 con 0x7f25400ccf00 2024-09-16T15:48:18.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.118+0000 7f2546400640 1 -- 172.21.15.151:0/3870426730 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f254006f980 con 0x7f25400ccf00 2024-09-16T15:48:18.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.119+0000 7f2537400640 1 -- 172.21.15.151:0/3870426730 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 30) ==== 102868+0+0 (secure 0 0 0) 0x7f252c01a070 con 0x7f25400ccf00 2024-09-16T15:48:18.122 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.120+0000 7f2537400640 1 --2- 172.21.15.151:0/3870426730 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f25200799c0 0x7f252007be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:18.122 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.120+0000 7f2537400640 1 -- 172.21.15.151:0/3870426730 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 4913+0+0 (secure 0 0 0) 0x7f252c098b50 con 0x7f25400ccf00 2024-09-16T15:48:18.122 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.120+0000 7f253f400640 1 --2- 172.21.15.151:0/3870426730 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f25200799c0 0x7f252007be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:18.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.121+0000 7f253f400640 1 --2- 172.21.15.151:0/3870426730 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f25200799c0 0x7f252007be80 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f2530035000 tx=0x7f253003b040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:18.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.124+0000 7f2537400640 1 -- 172.21.15.151:0/3870426730 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f252c062270 con 0x7f25400ccf00 2024-09-16T15:48:18.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.377+0000 7f2546400640 1 -- 172.21.15.151:0/3870426730 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f2540154df0 con 0x7f25400ccf00 2024-09-16T15:48:18.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.377+0000 7f2537400640 1 -- 172.21.15.151:0/3870426730 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) ==== 80+0+5304 (secure 0 0 0) 0x7f252c066ca0 con 0x7f25400ccf00 2024-09-16T15:48:18.379 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:48:18.379 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-16T15:48:18.379 INFO:teuthology.orchestra.run.smithi151.stdout: "auth_dump": [ 2024-09-16T15:48:18.380 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:48:18.380 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "mds.a", 2024-09-16T15:48:18.380 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDsUOhmJVK6AhAANfmTPS6kKn41LSD/MuyW9g==", 2024-09-16T15:48:18.380 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:48:18.380 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:48:18.380 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow *", 2024-09-16T15:48:18.380 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow *", 2024-09-16T15:48:18.380 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:48:18.380 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:48:18.380 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:48:18.380 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:48:18.380 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "mds.b", 2024-09-16T15:48:18.381 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDsUOhmX1R3FhAAUQgUsE/6wUyokXcZuBuIWg==", 2024-09-16T15:48:18.381 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:48:18.381 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:48:18.381 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow *", 2024-09-16T15:48:18.381 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow *", 2024-09-16T15:48:18.381 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:48:18.381 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:48:18.381 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:48:18.381 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:48:18.381 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "mds.c", 2024-09-16T15:48:18.381 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDsUOhmYGCKDBAAY2EFNmiFNf1x2so9wxp4YQ==", 2024-09-16T15:48:18.381 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:48:18.381 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:48:18.381 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow *", 2024-09-16T15:48:18.382 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow *", 2024-09-16T15:48:18.382 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:48:18.382 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:48:18.382 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:48:18.382 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:48:18.382 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "mds.d", 2024-09-16T15:48:18.382 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDsUOhmfCk4IBAAx12Gw2QudQ2nrPb8VlE0SA==", 2024-09-16T15:48:18.382 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:48:18.382 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:48:18.382 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow *", 2024-09-16T15:48:18.382 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow *", 2024-09-16T15:48:18.382 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:48:18.382 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:48:18.382 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:48:18.383 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:48:18.383 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.0", 2024-09-16T15:48:18.383 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDvUOhmKji5NxAAn/E3YQR7jeMnBTLBXc1G6A==", 2024-09-16T15:48:18.383 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:48:18.383 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:48:18.383 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:48:18.383 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:48:18.383 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:48:18.383 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:48:18.383 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:48:18.383 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.1", 2024-09-16T15:48:18.383 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDzUOhmmBrNBhAA0Uwh4Fam2bmKKvz5jAQBWw==", 2024-09-16T15:48:18.383 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:48:18.383 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:48:18.384 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:48:18.384 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:48:18.384 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:48:18.384 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:48:18.384 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:48:18.384 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.2", 2024-09-16T15:48:18.384 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQD2UOhmOV6GERAAjD+PQLTZrCFR1669UEEIAQ==", 2024-09-16T15:48:18.384 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:48:18.384 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:48:18.384 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:48:18.384 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:48:18.384 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:48:18.384 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:48:18.385 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:48:18.385 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.3", 2024-09-16T15:48:18.385 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQD5UOhmCRrdGRAAQC0bqWX6N1jniVGEQ2aAvg==", 2024-09-16T15:48:18.385 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:48:18.385 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:48:18.385 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:48:18.385 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:48:18.385 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:48:18.385 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:48:18.385 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:48:18.385 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.4", 2024-09-16T15:48:18.385 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQD/UOhmXI/nGRAA7t6XveKvrWgqy6rHuMQSZQ==", 2024-09-16T15:48:18.385 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:48:18.386 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:48:18.386 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:48:18.386 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:48:18.386 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:48:18.386 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:48:18.386 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:48:18.386 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.5", 2024-09-16T15:48:18.386 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQACUehmKM//IxAAsoFpiiTXNs//v3YXeUf/Iw==", 2024-09-16T15:48:18.386 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:48:18.386 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:48:18.386 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:48:18.386 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:48:18.386 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:48:18.386 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:48:18.386 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:48:18.387 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.6", 2024-09-16T15:48:18.387 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQAFUehmlo/FLRAAp5mvUSWCaBbm85QyTZY0eA==", 2024-09-16T15:48:18.387 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:48:18.387 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:48:18.387 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:48:18.387 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:48:18.387 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:48:18.387 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:48:18.387 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:48:18.387 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.7", 2024-09-16T15:48:18.387 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQAIUehmR32JNxAA27antOIKot1nScQbLZDf5g==", 2024-09-16T15:48:18.387 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:48:18.387 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:48:18.387 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:48:18.387 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:48:18.387 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:48:18.388 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:48:18.388 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:48:18.388 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "client.0", 2024-09-16T15:48:18.388 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDsUOhmf5PBKBAAKE4Y4zbA4h07IBNjPkyptw==", 2024-09-16T15:48:18.388 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:48:18.388 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:48:18.388 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow r", 2024-09-16T15:48:18.388 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-16T15:48:18.388 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:48:18.388 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:48:18.388 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:48:18.388 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "client.1", 2024-09-16T15:48:18.388 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDsUOhm15emNRAAwhK7QmA2c9IYGn04ERvY0g==", 2024-09-16T15:48:18.388 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:48:18.389 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:48:18.389 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow r", 2024-09-16T15:48:18.389 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow rw", 2024-09-16T15:48:18.389 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow rwx" 2024-09-16T15:48:18.389 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:48:18.389 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:48:18.389 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:48:18.389 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "client.2", 2024-09-16T15:48:18.389 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDsUOhm6zUyLxAAkzeiIQTuNZkMM+Oit9rC5A==", 2024-09-16T15:48:18.389 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:48:18.389 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:48:18.389 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow r", 2024-09-16T15:48:18.389 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow rw", 2024-09-16T15:48:18.389 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow rwx" 2024-09-16T15:48:18.389 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:48:18.390 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:48:18.390 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:48:18.390 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "client.3", 2024-09-16T15:48:18.390 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDtUOhmD82EABAAeDf82/hPxpPfAUk8oj5h/Q==", 2024-09-16T15:48:18.390 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:48:18.390 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:48:18.390 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow r", 2024-09-16T15:48:18.390 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow rw", 2024-09-16T15:48:18.390 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow rwx" 2024-09-16T15:48:18.390 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:48:18.390 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:48:18.390 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:48:18.390 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "client.admin", 2024-09-16T15:48:18.390 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDrUOhmQFmVDRAA8NKysbAe475xvg4CPAcFyw==", 2024-09-16T15:48:18.390 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:48:18.390 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow *", 2024-09-16T15:48:18.391 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow *", 2024-09-16T15:48:18.391 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow *", 2024-09-16T15:48:18.391 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:48:18.391 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:48:18.391 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:48:18.391 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:48:18.391 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "mgr.x", 2024-09-16T15:48:18.391 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDrUOhmloW3NxAA8UoLUfrw3wf1nzKbmtK/jw==", 2024-09-16T15:48:18.391 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:48:18.391 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow *", 2024-09-16T15:48:18.391 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile mgr", 2024-09-16T15:48:18.391 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:48:18.391 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:48:18.391 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:48:18.391 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:48:18.392 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "mgr.y", 2024-09-16T15:48:18.392 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDrUOhm1uZOLxAATgjA79vYf0psqQVo03sdDA==", 2024-09-16T15:48:18.392 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:48:18.392 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow *", 2024-09-16T15:48:18.392 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile mgr", 2024-09-16T15:48:18.392 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:48:18.392 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:48:18.392 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:48:18.392 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-09-16T15:48:18.392 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-16T15:48:18.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.379+0000 7f2546400640 1 -- 172.21.15.151:0/3870426730 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f25200799c0 msgr2=0x7f252007be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:18.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.380+0000 7f2546400640 1 --2- 172.21.15.151:0/3870426730 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f25200799c0 0x7f252007be80 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f2530035000 tx=0x7f253003b040 comp rx=0 tx=0).stop 2024-09-16T15:48:18.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.380+0000 7f2546400640 1 -- 172.21.15.151:0/3870426730 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f25400ccf00 msgr2=0x7f2540151390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:18.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.380+0000 7f2546400640 1 --2- 172.21.15.151:0/3870426730 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f25400ccf00 0x7f2540151390 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f252c00c450 tx=0x7f252c00c910 comp rx=0 tx=0).stop 2024-09-16T15:48:18.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.380+0000 7f2546400640 1 -- 172.21.15.151:0/3870426730 shutdown_connections 2024-09-16T15:48:18.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.380+0000 7f2546400640 1 --2- 172.21.15.151:0/3870426730 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f25200799c0 0x7f252007be80 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:18.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.380+0000 7f2546400640 1 --2- 172.21.15.151:0/3870426730 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f254015f5a0 0x7f25400cef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:18.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.380+0000 7f2546400640 1 --2- 172.21.15.151:0/3870426730 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f25400ccf00 0x7f2540151390 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:18.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.380+0000 7f2546400640 1 --2- 172.21.15.151:0/3870426730 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2540073fb0 0x7f254014f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:18.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.380+0000 7f2546400640 1 -- 172.21.15.151:0/3870426730 >> 172.21.15.151:0/3870426730 conn(0x7f25400b1950 msgr2=0x7f25400b0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:18.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.380+0000 7f2546400640 1 -- 172.21.15.151:0/3870426730 shutdown_connections 2024-09-16T15:48:18.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.380+0000 7f2546400640 1 -- 172.21.15.151:0/3870426730 wait complete. 2024-09-16T15:48:18.399 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-16T15:48:18.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.555+0000 7fe7a2000640 1 -- 172.21.15.151:0/3512326273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe79c0b2440 msgr2=0x7fe79c06f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:18.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.555+0000 7fe7a2000640 1 --2- 172.21.15.151:0/3512326273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe79c0b2440 0x7fe79c06f160 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fe794005fd0 tx=0x7fe79402a8b0 comp rx=0 tx=0).stop 2024-09-16T15:48:18.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.555+0000 7fe7a2000640 1 -- 172.21.15.151:0/3512326273 shutdown_connections 2024-09-16T15:48:18.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.555+0000 7fe7a2000640 1 --2- 172.21.15.151:0/3512326273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe79c0b2440 0x7fe79c06f160 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:18.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.555+0000 7fe7a2000640 1 -- 172.21.15.151:0/3512326273 >> 172.21.15.151:0/3512326273 conn(0x7fe79c0b1950 msgr2=0x7fe79c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:18.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.555+0000 7fe7a2000640 1 -- 172.21.15.151:0/3512326273 shutdown_connections 2024-09-16T15:48:18.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.555+0000 7fe7a2000640 1 -- 172.21.15.151:0/3512326273 wait complete. 2024-09-16T15:48:18.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.557+0000 7fe7a2000640 1 Processor -- start 2024-09-16T15:48:18.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.557+0000 7fe7a2000640 1 -- start start 2024-09-16T15:48:18.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.558+0000 7fe7a2000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe79c0b2440 0x7fe79c09c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:18.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.558+0000 7fe7a2000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe79c0ccf00 0x7fe79c09ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:18.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.558+0000 7fe7a2000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe79c15f5a0 0x7fe79c0d00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:18.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.558+0000 7fe7a2000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe79c074840 con 0x7fe79c0b2440 2024-09-16T15:48:18.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.558+0000 7fe7a2000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe79c0746c0 con 0x7fe79c15f5a0 2024-09-16T15:48:18.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.558+0000 7fe7a2000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fe79c06f6a0 con 0x7fe79c0ccf00 2024-09-16T15:48:18.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.558+0000 7fe79b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe79c0b2440 0x7fe79c09c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:18.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.558+0000 7fe79b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe79c0b2440 0x7fe79c09c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47618/0 (socket says 172.21.15.151:47618) 2024-09-16T15:48:18.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.558+0000 7fe79b400640 1 -- 172.21.15.151:0/2349889537 learned_addr learned my addr 172.21.15.151:0/2349889537 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:48:18.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.558+0000 7fe79be00640 1 --2- 172.21.15.151:0/2349889537 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe79c15f5a0 0x7fe79c0d00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:18.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.558+0000 7fe793e00640 1 --2- 172.21.15.151:0/2349889537 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe79c0ccf00 0x7fe79c09ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:18.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.559+0000 7fe79b400640 1 -- 172.21.15.151:0/2349889537 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe79c0ccf00 msgr2=0x7fe79c09ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:18.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.559+0000 7fe79b400640 1 --2- 172.21.15.151:0/2349889537 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe79c0ccf00 0x7fe79c09ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:18.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.559+0000 7fe79b400640 1 -- 172.21.15.151:0/2349889537 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe79c15f5a0 msgr2=0x7fe79c0d00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:18.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.559+0000 7fe79b400640 1 --2- 172.21.15.151:0/2349889537 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe79c15f5a0 0x7fe79c0d00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:18.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.559+0000 7fe79b400640 1 -- 172.21.15.151:0/2349889537 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe79c09b6e0 con 0x7fe79c0b2440 2024-09-16T15:48:18.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.559+0000 7fe793e00640 1 --2- 172.21.15.151:0/2349889537 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe79c0ccf00 0x7fe79c09ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:48:18.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.559+0000 7fe79b400640 1 --2- 172.21.15.151:0/2349889537 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe79c0b2440 0x7fe79c09c780 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fe78c00c430 tx=0x7fe78c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:18.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.559+0000 7fe793400640 1 -- 172.21.15.151:0/2349889537 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78c011850 con 0x7fe79c0b2440 2024-09-16T15:48:18.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.559+0000 7fe793400640 1 -- 172.21.15.151:0/2349889537 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe78c017070 con 0x7fe79c0b2440 2024-09-16T15:48:18.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.559+0000 7fe793400640 1 -- 172.21.15.151:0/2349889537 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78c011da0 con 0x7fe79c0b2440 2024-09-16T15:48:18.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.559+0000 7fe7a2000640 1 -- 172.21.15.151:0/2349889537 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe79c09b9c0 con 0x7fe79c0b2440 2024-09-16T15:48:18.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.559+0000 7fe7a2000640 1 -- 172.21.15.151:0/2349889537 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe79c09bf80 con 0x7fe79c0b2440 2024-09-16T15:48:18.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.560+0000 7fe7a2000640 1 -- 172.21.15.151:0/2349889537 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe79c0b1440 con 0x7fe79c0b2440 2024-09-16T15:48:18.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.562+0000 7fe793400640 1 -- 172.21.15.151:0/2349889537 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 30) ==== 102868+0+0 (secure 0 0 0) 0x7fe78c01c460 con 0x7fe79c0b2440 2024-09-16T15:48:18.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.562+0000 7fe793400640 1 --2- 172.21.15.151:0/2349889537 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe7780799c0 0x7fe77807be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:18.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.563+0000 7fe793400640 1 -- 172.21.15.151:0/2349889537 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 4913+0+0 (secure 0 0 0) 0x7fe78c099590 con 0x7fe79c0b2440 2024-09-16T15:48:18.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.563+0000 7fe793e00640 1 --2- 172.21.15.151:0/2349889537 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe7780799c0 0x7fe77807be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:18.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.563+0000 7fe793e00640 1 --2- 172.21.15.151:0/2349889537 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe7780799c0 0x7fe77807be80 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fe79c071270 tx=0x7fe784009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:18.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.566+0000 7fe793400640 1 -- 172.21.15.151:0/2349889537 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe79c0b1440 con 0x7fe79c0b2440 2024-09-16T15:48:18.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.753+0000 7fe7a2000640 1 -- 172.21.15.151:0/2349889537 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fe79c06f160 con 0x7fe79c0b2440 2024-09-16T15:48:18.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.753+0000 7fe793400640 1 -- 172.21.15.151:0/2349889537 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v87) ==== 84+0+65 (secure 0 0 0) 0x7fe78c063300 con 0x7fe79c0b2440 2024-09-16T15:48:18.755 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:48:18.755 INFO:teuthology.orchestra.run.smithi151.stdout:[ 2024-09-16T15:48:18.755 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:48:18.756 INFO:teuthology.orchestra.run.smithi151.stdout: "poolnum": 1, 2024-09-16T15:48:18.756 INFO:teuthology.orchestra.run.smithi151.stdout: "poolname": ".mgr" 2024-09-16T15:48:18.756 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:48:18.756 INFO:teuthology.orchestra.run.smithi151.stdout:] 2024-09-16T15:48:18.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.755+0000 7fe7a2000640 1 -- 172.21.15.151:0/2349889537 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe7780799c0 msgr2=0x7fe77807be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:18.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.755+0000 7fe7a2000640 1 --2- 172.21.15.151:0/2349889537 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe7780799c0 0x7fe77807be80 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fe79c071270 tx=0x7fe784009040 comp rx=0 tx=0).stop 2024-09-16T15:48:18.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.755+0000 7fe7a2000640 1 -- 172.21.15.151:0/2349889537 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe79c0b2440 msgr2=0x7fe79c09c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:18.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.755+0000 7fe7a2000640 1 --2- 172.21.15.151:0/2349889537 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe79c0b2440 0x7fe79c09c780 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fe78c00c430 tx=0x7fe78c00c8f0 comp rx=0 tx=0).stop 2024-09-16T15:48:18.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.756+0000 7fe7a2000640 1 -- 172.21.15.151:0/2349889537 shutdown_connections 2024-09-16T15:48:18.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.756+0000 7fe7a2000640 1 --2- 172.21.15.151:0/2349889537 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe7780799c0 0x7fe77807be80 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:18.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.756+0000 7fe7a2000640 1 --2- 172.21.15.151:0/2349889537 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe79c15f5a0 0x7fe79c0d00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:18.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.756+0000 7fe7a2000640 1 --2- 172.21.15.151:0/2349889537 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe79c0ccf00 0x7fe79c09ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:18.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.756+0000 7fe7a2000640 1 --2- 172.21.15.151:0/2349889537 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe79c0b2440 0x7fe79c09c780 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:18.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.756+0000 7fe7a2000640 1 -- 172.21.15.151:0/2349889537 >> 172.21.15.151:0/2349889537 conn(0x7fe79c0b1950 msgr2=0x7fe79c0ae5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:18.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.756+0000 7fe7a2000640 1 -- 172.21.15.151:0/2349889537 shutdown_connections 2024-09-16T15:48:18.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.756+0000 7fe7a2000640 1 -- 172.21.15.151:0/2349889537 wait complete. 2024-09-16T15:48:18.774 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-16T15:48:18.774 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-16T15:48:18.934 DEBUG:teuthology.orchestra.run.smithi151:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:48:18.939 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:48:18.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.937+0000 7f10f5200640 1 -- 172.21.15.151:0/3734546249 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10f00d0fe0 msgr2=0x7f10f015ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:18.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.937+0000 7f10f5200640 1 --2- 172.21.15.151:0/3734546249 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10f00d0fe0 0x7f10f015ef50 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f10e8005fd0 tx=0x7f10e802aaa0 comp rx=0 tx=0).stop 2024-09-16T15:48:18.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.937+0000 7f10f5200640 1 -- 172.21.15.151:0/3734546249 shutdown_connections 2024-09-16T15:48:18.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.937+0000 7f10f5200640 1 --2- 172.21.15.151:0/3734546249 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10f00d0fe0 0x7f10f015ef50 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:18.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.937+0000 7f10f5200640 1 --2- 172.21.15.151:0/3734546249 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f10f015e710 0x7f10f00d4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:18.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.937+0000 7f10f5200640 1 -- 172.21.15.151:0/3734546249 >> 172.21.15.151:0/3734546249 conn(0x7f10f00d3f00 msgr2=0x7f10f00d4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:18.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.937+0000 7f10f5200640 1 -- 172.21.15.151:0/3734546249 shutdown_connections 2024-09-16T15:48:18.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.937+0000 7f10f5200640 1 -- 172.21.15.151:0/3734546249 wait complete. 2024-09-16T15:48:18.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.939+0000 7f10f5200640 1 Processor -- start 2024-09-16T15:48:18.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.939+0000 7f10f5200640 1 -- start start 2024-09-16T15:48:18.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.940+0000 7f10f5200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10f00d0fe0 0x7f10f00b4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:18.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.940+0000 7f10f5200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10f015e710 0x7f10f00b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:18.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.940+0000 7f10f5200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f10f015eaf0 0x7f10f01f73f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:18.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.940+0000 7f10f5200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f10f00d1e60 con 0x7f10f015e710 2024-09-16T15:48:18.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.940+0000 7f10f5200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f10f00d1ce0 con 0x7f10f00d0fe0 2024-09-16T15:48:18.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.940+0000 7f10f5200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f10f00d1fe0 con 0x7f10f015eaf0 2024-09-16T15:48:18.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.940+0000 7f10eea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10f015e710 0x7f10f00b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:18.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.940+0000 7f10eea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10f015e710 0x7f10f00b3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47648/0 (socket says 172.21.15.151:47648) 2024-09-16T15:48:18.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.940+0000 7f10eea00640 1 -- 172.21.15.151:0/2993201262 learned_addr learned my addr 172.21.15.151:0/2993201262 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:48:18.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.940+0000 7f10ef400640 1 --2- 172.21.15.151:0/2993201262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10f00d0fe0 0x7f10f00b4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:18.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.940+0000 7f10efe00640 1 --2- 172.21.15.151:0/2993201262 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f10f015eaf0 0x7f10f01f73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:18.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.941+0000 7f10eea00640 1 -- 172.21.15.151:0/2993201262 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f10f015eaf0 msgr2=0x7f10f01f73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:18.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.941+0000 7f10eea00640 1 --2- 172.21.15.151:0/2993201262 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f10f015eaf0 0x7f10f01f73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:18.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.941+0000 7f10eea00640 1 -- 172.21.15.151:0/2993201262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10f00d0fe0 msgr2=0x7f10f00b4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:18.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.941+0000 7f10eea00640 1 --2- 172.21.15.151:0/2993201262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10f00d0fe0 0x7f10f00b4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:18.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.941+0000 7f10eea00640 1 -- 172.21.15.151:0/2993201262 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10f01f7930 con 0x7f10f015e710 2024-09-16T15:48:18.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.941+0000 7f10eea00640 1 --2- 172.21.15.151:0/2993201262 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10f015e710 0x7f10f00b3ca0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f10dc00c420 tx=0x7f10dc00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:18.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.941+0000 7f10e7400640 1 -- 172.21.15.151:0/2993201262 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10dc017070 con 0x7f10f015e710 2024-09-16T15:48:18.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.941+0000 7f10e7400640 1 -- 172.21.15.151:0/2993201262 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f10dc0053c0 con 0x7f10f015e710 2024-09-16T15:48:18.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.941+0000 7f10e7400640 1 -- 172.21.15.151:0/2993201262 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10dc003850 con 0x7f10f015e710 2024-09-16T15:48:18.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.941+0000 7f10f5200640 1 -- 172.21.15.151:0/2993201262 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10f00b5730 con 0x7f10f015e710 2024-09-16T15:48:18.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.941+0000 7f10f5200640 1 -- 172.21.15.151:0/2993201262 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10f00b4e90 con 0x7f10f015e710 2024-09-16T15:48:18.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.943+0000 7f10f5200640 1 -- 172.21.15.151:0/2993201262 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10f015ef50 con 0x7f10f015e710 2024-09-16T15:48:18.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.944+0000 7f10e7400640 1 -- 172.21.15.151:0/2993201262 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 30) ==== 102868+0+0 (secure 0 0 0) 0x7f10dc01a070 con 0x7f10f015e710 2024-09-16T15:48:18.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.946+0000 7f10e7400640 1 --2- 172.21.15.151:0/2993201262 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f10d00799c0 0x7f10d007be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:18.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.946+0000 7f10e7400640 1 -- 172.21.15.151:0/2993201262 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 4913+0+0 (secure 0 0 0) 0x7f10dc098b90 con 0x7f10f015e710 2024-09-16T15:48:18.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.946+0000 7f10ef400640 1 --2- 172.21.15.151:0/2993201262 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f10d00799c0 0x7f10d007be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:18.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.947+0000 7f10ef400640 1 --2- 172.21.15.151:0/2993201262 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f10d00799c0 0x7f10d007be80 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f10e0009bd0 tx=0x7f10e0005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:18.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:18.950+0000 7f10e7400640 1 -- 172.21.15.151:0/2993201262 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f10dc0622b0 con 0x7f10f015e710 2024-09-16T15:48:19.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:19.139+0000 7f10f5200640 1 -- 172.21.15.151:0/2993201262 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f10f00b9e80 con 0x7f10f015e710 2024-09-16T15:48:19.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:19.944+0000 7f10e7400640 1 -- 172.21.15.151:0/2993201262 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v88) ==== 138+0+0 (secure 0 0 0) 0x7f10dc066ce0 con 0x7f10f015e710 2024-09-16T15:48:19.947 INFO:teuthology.orchestra.run.smithi151.stderr:pool 'cephfs_metadata' created 2024-09-16T15:48:19.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:19.947+0000 7f10f5200640 1 -- 172.21.15.151:0/2993201262 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f10d00799c0 msgr2=0x7f10d007be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:19.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:19.947+0000 7f10f5200640 1 --2- 172.21.15.151:0/2993201262 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f10d00799c0 0x7f10d007be80 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f10e0009bd0 tx=0x7f10e0005f90 comp rx=0 tx=0).stop 2024-09-16T15:48:19.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:19.947+0000 7f10f5200640 1 -- 172.21.15.151:0/2993201262 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10f015e710 msgr2=0x7f10f00b3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:19.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:19.947+0000 7f10f5200640 1 --2- 172.21.15.151:0/2993201262 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10f015e710 0x7f10f00b3ca0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f10dc00c420 tx=0x7f10dc00c8e0 comp rx=0 tx=0).stop 2024-09-16T15:48:19.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:19.947+0000 7f10f5200640 1 -- 172.21.15.151:0/2993201262 shutdown_connections 2024-09-16T15:48:19.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:19.947+0000 7f10f5200640 1 --2- 172.21.15.151:0/2993201262 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f10d00799c0 0x7f10d007be80 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:19.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:19.947+0000 7f10f5200640 1 --2- 172.21.15.151:0/2993201262 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f10f015eaf0 0x7f10f01f73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:19.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:19.947+0000 7f10f5200640 1 --2- 172.21.15.151:0/2993201262 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10f015e710 0x7f10f00b3ca0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:19.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:19.947+0000 7f10f5200640 1 --2- 172.21.15.151:0/2993201262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10f00d0fe0 0x7f10f00b4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:19.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:19.947+0000 7f10f5200640 1 -- 172.21.15.151:0/2993201262 >> 172.21.15.151:0/2993201262 conn(0x7f10f00d3f00 msgr2=0x7f10f015c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:19.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:19.948+0000 7f10f5200640 1 -- 172.21.15.151:0/2993201262 shutdown_connections 2024-09-16T15:48:19.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:19.948+0000 7f10f5200640 1 -- 172.21.15.151:0/2993201262 wait complete. 2024-09-16T15:48:19.974 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-16T15:48:20.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:20.141+0000 7f6dbac00640 1 -- 172.21.15.151:0/565983879 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6db415f670 msgr2=0x7f6db40c6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:20.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:20.141+0000 7f6dbac00640 1 --2- 172.21.15.151:0/565983879 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6db415f670 0x7f6db40c6430 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f6db4075410 tx=0x7f6da802aa10 comp rx=0 tx=0).stop 2024-09-16T15:48:20.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:20.141+0000 7f6dbac00640 1 -- 172.21.15.151:0/565983879 shutdown_connections 2024-09-16T15:48:20.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:20.141+0000 7f6dbac00640 1 --2- 172.21.15.151:0/565983879 >> v2:172.21.15.167:3300/0 conn(0x7f6db415c0f0 0x7f6db415c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:20.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:20.141+0000 7f6dbac00640 1 --2- 172.21.15.151:0/565983879 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6db40c5730 0x7f6db415fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:20.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:20.141+0000 7f6dbac00640 1 --2- 172.21.15.151:0/565983879 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6db415f670 0x7f6db40c6430 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:20.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:20.141+0000 7f6dbac00640 1 -- 172.21.15.151:0/565983879 >> 172.21.15.151:0/565983879 conn(0x7f6db40cc490 msgr2=0x7f6db40cc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:20.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:20.141+0000 7f6dbac00640 1 -- 172.21.15.151:0/565983879 shutdown_connections 2024-09-16T15:48:20.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:20.141+0000 7f6dbac00640 1 -- 172.21.15.151:0/565983879 wait complete. 2024-09-16T15:48:20.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:20.143+0000 7f6dbac00640 1 Processor -- start 2024-09-16T15:48:20.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:20.143+0000 7f6dbac00640 1 -- start start 2024-09-16T15:48:20.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:20.144+0000 7f6dbac00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6db40c5730 0x7f6db40c6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:20.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:20.144+0000 7f6dbac00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6db415c0f0 0x7f6db41515f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:20.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:20.144+0000 7f6dbac00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6db415f670 0x7f6db4145520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:20.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:20.144+0000 7f6dbac00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6db40cd2c0 con 0x7f6db415c0f0 2024-09-16T15:48:20.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:20.144+0000 7f6dbac00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6db40cd140 con 0x7f6db40c5730 2024-09-16T15:48:20.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:20.144+0000 7f6dbac00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f6db40ca370 con 0x7f6db415f670 2024-09-16T15:48:20.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:20.144+0000 7f6db8e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6db415c0f0 0x7f6db41515f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:20.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:20.144+0000 7f6db8e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6db415c0f0 0x7f6db41515f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47662/0 (socket says 172.21.15.151:47662) 2024-09-16T15:48:20.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:20.144+0000 7f6db8e00640 1 -- 172.21.15.151:0/3683702769 learned_addr learned my addr 172.21.15.151:0/3683702769 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:48:20.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:20.144+0000 7f6db9800640 1 --2- 172.21.15.151:0/3683702769 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6db40c5730 0x7f6db40c6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:20.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:20.144+0000 7f6dba200640 1 --2- 172.21.15.151:0/3683702769 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6db415f670 0x7f6db4145520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:20.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:20.145+0000 7f6db8e00640 1 -- 172.21.15.151:0/3683702769 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6db415f670 msgr2=0x7f6db4145520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:20.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:20.145+0000 7f6db8e00640 1 --2- 172.21.15.151:0/3683702769 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6db415f670 0x7f6db4145520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:20.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:20.145+0000 7f6db8e00640 1 -- 172.21.15.151:0/3683702769 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6db40c5730 msgr2=0x7f6db40c6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:20.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:20.145+0000 7f6db8e00640 1 --2- 172.21.15.151:0/3683702769 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6db40c5730 0x7f6db40c6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:20.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:20.145+0000 7f6db8e00640 1 -- 172.21.15.151:0/3683702769 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6db4145a60 con 0x7f6db415c0f0 2024-09-16T15:48:20.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:20.145+0000 7f6db8e00640 1 --2- 172.21.15.151:0/3683702769 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6db415c0f0 0x7f6db41515f0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f6da000d720 tx=0x7f6da000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:20.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:20.145+0000 7f6db1600640 1 -- 172.21.15.151:0/3683702769 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da0017070 con 0x7f6db415c0f0 2024-09-16T15:48:20.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:20.145+0000 7f6db1600640 1 -- 172.21.15.151:0/3683702769 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6da0009d20 con 0x7f6db415c0f0 2024-09-16T15:48:20.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:20.145+0000 7f6dbac00640 1 -- 172.21.15.151:0/3683702769 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6db40b2a00 con 0x7f6db415c0f0 2024-09-16T15:48:20.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:20.145+0000 7f6db1600640 1 -- 172.21.15.151:0/3683702769 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da0004280 con 0x7f6db415c0f0 2024-09-16T15:48:20.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:20.145+0000 7f6dbac00640 1 -- 172.21.15.151:0/3683702769 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6db40b2fc0 con 0x7f6db415c0f0 2024-09-16T15:48:20.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:20.146+0000 7f6dbac00640 1 -- 172.21.15.151:0/3683702769 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6db415cd10 con 0x7f6db415c0f0 2024-09-16T15:48:20.150 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:20.148+0000 7f6db1600640 1 -- 172.21.15.151:0/3683702769 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 30) ==== 102868+0+0 (secure 0 0 0) 0x7f6da0005020 con 0x7f6db415c0f0 2024-09-16T15:48:20.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:20.149+0000 7f6db1600640 1 --2- 172.21.15.151:0/3683702769 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6d940799c0 0x7f6d9407be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:20.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:20.149+0000 7f6db1600640 1 -- 172.21.15.151:0/3683702769 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5283+0+0 (secure 0 0 0) 0x7f6da0099b00 con 0x7f6db415c0f0 2024-09-16T15:48:20.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:20.149+0000 7f6db9800640 1 --2- 172.21.15.151:0/3683702769 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6d940799c0 0x7f6d9407be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:20.152 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:20.150+0000 7f6db9800640 1 --2- 172.21.15.151:0/3683702769 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6d940799c0 0x7f6d9407be80 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f6da8035000 tx=0x7f6da803b040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:20.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:20.153+0000 7f6db1600640 1 -- 172.21.15.151:0/3683702769 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6da0020020 con 0x7f6db415c0f0 2024-09-16T15:48:20.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:20.340+0000 7f6dbac00640 1 -- 172.21.15.151:0/3683702769 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f6db40b0360 con 0x7f6db415c0f0 2024-09-16T15:48:20.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:20.947+0000 7f6db1600640 1 -- 172.21.15.151:0/3683702769 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v89) ==== 170+0+0 (secure 0 0 0) 0x7f6db4145a60 con 0x7f6db415c0f0 2024-09-16T15:48:20.950 INFO:teuthology.orchestra.run.smithi151.stderr:pool 'cephfs_data' created 2024-09-16T15:48:20.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:20.951+0000 7f6dbac00640 1 -- 172.21.15.151:0/3683702769 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6d940799c0 msgr2=0x7f6d9407be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:20.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:20.951+0000 7f6dbac00640 1 --2- 172.21.15.151:0/3683702769 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6d940799c0 0x7f6d9407be80 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f6da8035000 tx=0x7f6da803b040 comp rx=0 tx=0).stop 2024-09-16T15:48:20.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:20.951+0000 7f6dbac00640 1 -- 172.21.15.151:0/3683702769 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6db415c0f0 msgr2=0x7f6db41515f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:20.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:20.951+0000 7f6dbac00640 1 --2- 172.21.15.151:0/3683702769 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6db415c0f0 0x7f6db41515f0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f6da000d720 tx=0x7f6da000dbe0 comp rx=0 tx=0).stop 2024-09-16T15:48:20.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:20.951+0000 7f6dbac00640 1 -- 172.21.15.151:0/3683702769 shutdown_connections 2024-09-16T15:48:20.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:20.951+0000 7f6dbac00640 1 --2- 172.21.15.151:0/3683702769 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6d940799c0 0x7f6d9407be80 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:20.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:20.951+0000 7f6dbac00640 1 --2- 172.21.15.151:0/3683702769 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6db415f670 0x7f6db4145520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:20.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:20.951+0000 7f6dbac00640 1 --2- 172.21.15.151:0/3683702769 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6db415c0f0 0x7f6db41515f0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:20.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:20.951+0000 7f6dbac00640 1 --2- 172.21.15.151:0/3683702769 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6db40c5730 0x7f6db40c6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:20.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:20.951+0000 7f6dbac00640 1 -- 172.21.15.151:0/3683702769 >> 172.21.15.151:0/3683702769 conn(0x7f6db40cc490 msgr2=0x7f6db4152050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:20.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:20.951+0000 7f6dbac00640 1 -- 172.21.15.151:0/3683702769 shutdown_connections 2024-09-16T15:48:20.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:20.951+0000 7f6dbac00640 1 -- 172.21.15.151:0/3683702769 wait complete. 2024-09-16T15:48:20.982 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-16T15:48:21.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:21.139+0000 7f898c400640 1 -- 172.21.15.151:0/585029247 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f898406efa0 msgr2=0x7f89840cc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:21.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:21.139+0000 7f898c400640 1 --2- 172.21.15.151:0/585029247 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f898406efa0 0x7f89840cc5c0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f8970005fd0 tx=0x7f897002a540 comp rx=0 tx=0).stop 2024-09-16T15:48:21.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:21.139+0000 7f898c400640 1 -- 172.21.15.151:0/585029247 shutdown_connections 2024-09-16T15:48:21.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:21.139+0000 7f898c400640 1 --2- 172.21.15.151:0/585029247 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f898406efa0 0x7f89840cc5c0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:21.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:21.139+0000 7f898c400640 1 --2- 172.21.15.151:0/585029247 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8984145750 0x7f8984073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:21.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:21.139+0000 7f898c400640 1 -- 172.21.15.151:0/585029247 >> 172.21.15.151:0/585029247 conn(0x7f89840b1840 msgr2=0x7f89840b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:21.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:21.140+0000 7f898c400640 1 -- 172.21.15.151:0/585029247 shutdown_connections 2024-09-16T15:48:21.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:21.140+0000 7f898c400640 1 -- 172.21.15.151:0/585029247 wait complete. 2024-09-16T15:48:21.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:21.141+0000 7f898c400640 1 Processor -- start 2024-09-16T15:48:21.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:21.141+0000 7f898c400640 1 -- start start 2024-09-16T15:48:21.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:21.142+0000 7f898c400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f898406efa0 0x7f89840b3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:21.146 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:21.142+0000 7f898c400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89840ccb00 0x7f898414b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:21.146 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:21.142+0000 7f898c400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8984145750 0x7f89841f42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:21.146 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:21.142+0000 7f898c400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f89840b2650 con 0x7f89840ccb00 2024-09-16T15:48:21.146 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:21.142+0000 7f898c400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f89840cedd0 con 0x7f8984145750 2024-09-16T15:48:21.146 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:21.142+0000 7f898c400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f8984160100 con 0x7f898406efa0 2024-09-16T15:48:21.146 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:21.142+0000 7f8989000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89840ccb00 0x7f898414b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:21.146 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:21.142+0000 7f8989000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89840ccb00 0x7f898414b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47678/0 (socket says 172.21.15.151:47678) 2024-09-16T15:48:21.146 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:21.142+0000 7f8989000640 1 -- 172.21.15.151:0/2971506253 learned_addr learned my addr 172.21.15.151:0/2971506253 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:48:21.146 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:21.143+0000 7f8989a00640 1 --2- 172.21.15.151:0/2971506253 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f898406efa0 0x7f89840b3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:21.146 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:21.143+0000 7f898a400640 1 --2- 172.21.15.151:0/2971506253 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8984145750 0x7f89841f42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:21.146 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:21.143+0000 7f8989000640 1 -- 172.21.15.151:0/2971506253 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f898406efa0 msgr2=0x7f89840b3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:21.146 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:21.143+0000 7f8989000640 1 --2- 172.21.15.151:0/2971506253 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f898406efa0 0x7f89840b3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:21.146 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:21.143+0000 7f8989000640 1 -- 172.21.15.151:0/2971506253 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8984145750 msgr2=0x7f89841f42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:21.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:21.143+0000 7f8989000640 1 --2- 172.21.15.151:0/2971506253 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8984145750 0x7f89841f42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:21.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:21.143+0000 7f8989000640 1 -- 172.21.15.151:0/2971506253 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f898414bb60 con 0x7f89840ccb00 2024-09-16T15:48:21.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:21.143+0000 7f8989a00640 1 --2- 172.21.15.151:0/2971506253 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f898406efa0 0x7f89840b3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:48:21.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:21.143+0000 7f898a400640 1 --2- 172.21.15.151:0/2971506253 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8984145750 0x7f89841f42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:48:21.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:21.143+0000 7f8989000640 1 --2- 172.21.15.151:0/2971506253 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89840ccb00 0x7f898414b3c0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f8970009850 tx=0x7f89700038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:21.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:21.143+0000 7f8981600640 1 -- 172.21.15.151:0/2971506253 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897003f070 con 0x7f89840ccb00 2024-09-16T15:48:21.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:21.144+0000 7f898c400640 1 -- 172.21.15.151:0/2971506253 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f898414be40 con 0x7f89840ccb00 2024-09-16T15:48:21.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:21.143+0000 7f8981600640 1 -- 172.21.15.151:0/2971506253 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f89700047a0 con 0x7f89840ccb00 2024-09-16T15:48:21.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:21.144+0000 7f8981600640 1 -- 172.21.15.151:0/2971506253 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8970035030 con 0x7f89840ccb00 2024-09-16T15:48:21.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:21.144+0000 7f898c400640 1 -- 172.21.15.151:0/2971506253 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f898414c400 con 0x7f89840ccb00 2024-09-16T15:48:21.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:21.145+0000 7f898c400640 1 -- 172.21.15.151:0/2971506253 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89841605b0 con 0x7f89840ccb00 2024-09-16T15:48:21.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:21.147+0000 7f8981600640 1 -- 172.21.15.151:0/2971506253 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 30) ==== 102868+0+0 (secure 0 0 0) 0x7f8970042070 con 0x7f89840ccb00 2024-09-16T15:48:21.150 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:21.148+0000 7f8981600640 1 --2- 172.21.15.151:0/2971506253 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f89640799f0 0x7f896407beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:21.150 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:21.148+0000 7f8981600640 1 -- 172.21.15.151:0/2971506253 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5665+0+0 (secure 0 0 0) 0x7f89700bab60 con 0x7f89840ccb00 2024-09-16T15:48:21.150 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:21.149+0000 7f8989a00640 1 --2- 172.21.15.151:0/2971506253 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f89640799f0 0x7f896407beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:21.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:21.149+0000 7f8989a00640 1 --2- 172.21.15.151:0/2971506253 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f89640799f0 0x7f896407beb0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f8978005fd0 tx=0x7f8978009340 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:21.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:21.151+0000 7f8981600640 1 -- 172.21.15.151:0/2971506253 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8970084930 con 0x7f89840ccb00 2024-09-16T15:48:21.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:21.372+0000 7f898c400640 1 -- 172.21.15.151:0/2971506253 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f8984073710 con 0x7f89840ccb00 2024-09-16T15:48:21.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:21.972+0000 7f8981600640 1 -- 172.21.15.151:0/2971506253 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v52) ==== 174+0+0 (secure 0 0 0) 0x7f8970084230 con 0x7f89840ccb00 2024-09-16T15:48:21.975 INFO:teuthology.orchestra.run.smithi151.stderr:new fs with metadata pool 12 and data pool 13 2024-09-16T15:48:21.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:21.982+0000 7f898c400640 1 -- 172.21.15.151:0/2971506253 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f89640799f0 msgr2=0x7f896407beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:21.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:21.982+0000 7f898c400640 1 --2- 172.21.15.151:0/2971506253 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f89640799f0 0x7f896407beb0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f8978005fd0 tx=0x7f8978009340 comp rx=0 tx=0).stop 2024-09-16T15:48:21.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:21.982+0000 7f898c400640 1 -- 172.21.15.151:0/2971506253 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89840ccb00 msgr2=0x7f898414b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:21.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:21.982+0000 7f898c400640 1 --2- 172.21.15.151:0/2971506253 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89840ccb00 0x7f898414b3c0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f8970009850 tx=0x7f89700038a0 comp rx=0 tx=0).stop 2024-09-16T15:48:21.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:21.983+0000 7f898c400640 1 -- 172.21.15.151:0/2971506253 shutdown_connections 2024-09-16T15:48:21.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:21.983+0000 7f898c400640 1 --2- 172.21.15.151:0/2971506253 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f89640799f0 0x7f896407beb0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:21.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:21.983+0000 7f898c400640 1 --2- 172.21.15.151:0/2971506253 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8984145750 0x7f89841f42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:21.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:21.983+0000 7f898c400640 1 --2- 172.21.15.151:0/2971506253 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89840ccb00 0x7f898414b3c0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:21.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:21.983+0000 7f898c400640 1 --2- 172.21.15.151:0/2971506253 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f898406efa0 0x7f89840b3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:21.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:21.983+0000 7f898c400640 1 -- 172.21.15.151:0/2971506253 >> 172.21.15.151:0/2971506253 conn(0x7f89840b1840 msgr2=0x7f89840cfbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:21.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:21.984+0000 7f898c400640 1 -- 172.21.15.151:0/2971506253 shutdown_connections 2024-09-16T15:48:21.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:21.986+0000 7f898c400640 1 -- 172.21.15.151:0/2971506253 wait complete. 2024-09-16T15:48:22.017 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:48:22.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.175+0000 7fb369a00640 1 -- 172.21.15.151:0/3741686864 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb348004d40 msgr2=0x7fb3480051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:22.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.175+0000 7fb369a00640 1 --2- 172.21.15.151:0/3741686864 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb348004d40 0x7fb3480051c0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fb35c00c610 tx=0x7fb348005830 comp rx=0 tx=0).stop 2024-09-16T15:48:22.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.175+0000 7fb369000640 1 -- 172.21.15.151:0/3741686864 reap_dead start 2024-09-16T15:48:22.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.175+0000 7fb369a00640 1 -- 172.21.15.151:0/3741686864 shutdown_connections 2024-09-16T15:48:22.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.175+0000 7fb369a00640 1 -- 172.21.15.151:0/3741686864 >> 172.21.15.151:0/3741686864 conn(0x7fb3640d3df0 msgr2=0x7fb3640d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:22.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.175+0000 7fb369a00640 1 -- 172.21.15.151:0/3741686864 shutdown_connections 2024-09-16T15:48:22.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.175+0000 7fb369a00640 1 -- 172.21.15.151:0/3741686864 wait complete. 2024-09-16T15:48:22.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.176+0000 7fb369a00640 1 Processor -- start 2024-09-16T15:48:22.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.177+0000 7fb369a00640 1 -- start start 2024-09-16T15:48:22.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.177+0000 7fb369a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb348004d40 0x7fb3640cbe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:22.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.177+0000 7fb369a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3640b9670 0x7fb3640b8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:22.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.177+0000 7fb369a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3640cd670 0x7fb3641f7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:22.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.177+0000 7fb369a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb3640b2900 con 0x7fb3640cd670 2024-09-16T15:48:22.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.177+0000 7fb369a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb3640b2780 con 0x7fb3640b9670 2024-09-16T15:48:22.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.177+0000 7fb369a00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fb3640b2a80 con 0x7fb348004d40 2024-09-16T15:48:22.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.178+0000 7fb369000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3640cd670 0x7fb3641f7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:22.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.178+0000 7fb369000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3640cd670 0x7fb3641f7b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47700/0 (socket says 172.21.15.151:47700) 2024-09-16T15:48:22.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.178+0000 7fb369000640 1 -- 172.21.15.151:0/244780782 learned_addr learned my addr 172.21.15.151:0/244780782 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:48:22.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.178+0000 7fb35be00640 1 --2- 172.21.15.151:0/244780782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3640b9670 0x7fb3640b8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:22.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.178+0000 7fb363e00640 1 --2- 172.21.15.151:0/244780782 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb348004d40 0x7fb3640cbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:22.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.178+0000 7fb369000640 1 -- 172.21.15.151:0/244780782 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb348004d40 msgr2=0x7fb3640cbe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:22.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.178+0000 7fb369000640 1 --2- 172.21.15.151:0/244780782 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb348004d40 0x7fb3640cbe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:22.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.178+0000 7fb369000640 1 -- 172.21.15.151:0/244780782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3640b9670 msgr2=0x7fb3640b8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:22.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.178+0000 7fb369000640 1 --2- 172.21.15.151:0/244780782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3640b9670 0x7fb3640b8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:22.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.178+0000 7fb369000640 1 -- 172.21.15.151:0/244780782 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3640b9af0 con 0x7fb3640cd670 2024-09-16T15:48:22.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.178+0000 7fb35be00640 1 --2- 172.21.15.151:0/244780782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3640b9670 0x7fb3640b8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:48:22.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.178+0000 7fb369000640 1 --2- 172.21.15.151:0/244780782 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3640cd670 0x7fb3641f7b20 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fb35c004b90 tx=0x7fb35c004bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:22.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.179+0000 7fb360c00640 1 -- 172.21.15.151:0/244780782 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb35c02bc10 con 0x7fb3640cd670 2024-09-16T15:48:22.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.179+0000 7fb360c00640 1 -- 172.21.15.151:0/244780782 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb35c006070 con 0x7fb3640cd670 2024-09-16T15:48:22.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.179+0000 7fb360c00640 1 -- 172.21.15.151:0/244780782 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb35c006480 con 0x7fb3640cd670 2024-09-16T15:48:22.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.179+0000 7fb369a00640 1 -- 172.21.15.151:0/244780782 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3640cb040 con 0x7fb3640cd670 2024-09-16T15:48:22.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.179+0000 7fb369a00640 1 -- 172.21.15.151:0/244780782 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3640cb600 con 0x7fb3640cd670 2024-09-16T15:48:22.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.180+0000 7fb369a00640 1 -- 172.21.15.151:0/244780782 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3640b5370 con 0x7fb3640cd670 2024-09-16T15:48:22.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.181+0000 7fb360c00640 1 -- 172.21.15.151:0/244780782 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 30) ==== 102868+0+0 (secure 0 0 0) 0x7fb35c006620 con 0x7fb3640cd670 2024-09-16T15:48:22.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.182+0000 7fb360c00640 1 --2- 172.21.15.151:0/244780782 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb3400799f0 0x7fb34007beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:22.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.182+0000 7fb360c00640 1 -- 172.21.15.151:0/244780782 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7fb35c0b9590 con 0x7fb3640cd670 2024-09-16T15:48:22.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.183+0000 7fb363e00640 1 --2- 172.21.15.151:0/244780782 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb3400799f0 0x7fb34007beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:22.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.183+0000 7fb363e00640 1 --2- 172.21.15.151:0/244780782 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb3400799f0 0x7fb34007beb0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fb354009cd0 tx=0x7fb354005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:22.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.186+0000 7fb360c00640 1 -- 172.21.15.151:0/244780782 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb3640b9af0 con 0x7fb3640cd670 2024-09-16T15:48:22.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.369+0000 7fb369a00640 1 -- 172.21.15.151:0/244780782 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb3640d2b40 con 0x7fb3640cd670 2024-09-16T15:48:22.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.370+0000 7fb360c00640 1 -- 172.21.15.151:0/244780782 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+16892 (secure 0 0 0) 0x7fb35c083650 con 0x7fb3640cd670 2024-09-16T15:48:22.373 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:48:22.373 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":90,"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","created":"2024-09-16T15:38:56.348535+0000","modified":"2024-09-16T15:48:21.374120+0000","last_up_change":"2024-09-16T15:39:10.681384+0000","last_in_change":"2024-09-16T15:39:00.544477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:39:11.602252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:48:19.150224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-16T15:48:20.351519+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5524076-a903-4d0a-af6a-caf474c4765d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6809","nonce":3111342741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6811","nonce":3111342741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6815","nonce":3111342741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6813","nonce":3111342741}]},"public_addr":"172.21.15.151:6809/3111342741","cluster_addr":"172.21.15.151:6811/3111342741","heartbeat_back_addr":"172.21.15.151:6815/3111342741","heartbeat_front_addr":"172.21.15.151:6813/3111342741","state":["exists","up"]},{"osd":1,"uuid":"dd25925f-0ae1-494e-8529-4dc205759db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6801","nonce":3368304087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6803","nonce":3368304087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6807","nonce":3368304087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6805","nonce":3368304087}]},"public_addr":"172.21.15.151:6801/3368304087","cluster_addr":"172.21.15.151:6803/3368304087","heartbeat_back_addr":"172.21.15.151:6807/3368304087","heartbeat_front_addr":"172.21.15.151:6805/3368304087","state":["exists","up"]},{"osd":2,"uuid":"c2f18115-013d-4da6-a0fe-cb978f4494b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6823","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6825","nonce":3079373271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6827","nonce":3079373271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6831","nonce":3079373271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6829","nonce":3079373271}]},"public_addr":"172.21.15.151:6825/3079373271","cluster_addr":"172.21.15.151:6827/3079373271","heartbeat_back_addr":"172.21.15.151:6831/3079373271","heartbeat_front_addr":"172.21.15.151:6829/3079373271","state":["exists","up"]},{"osd":3,"uuid":"36edb718-2fcb-4d88-8ddc-c321eed4279d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6817","nonce":1142509041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6819","nonce":1142509041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6824","nonce":1142509041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6821","nonce":1142509041}]},"public_addr":"172.21.15.151:6817/1142509041","cluster_addr":"172.21.15.151:6819/1142509041","heartbeat_back_addr":"172.21.15.151:6824/1142509041","heartbeat_front_addr":"172.21.15.151:6821/1142509041","state":["exists","up"]},{"osd":4,"uuid":"64d33708-cd0a-4236-abe3-1b646e82ab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6811","nonce":2735495271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6813","nonce":2735495271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6817","nonce":2735495271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6815","nonce":2735495271}]},"public_addr":"172.21.15.167:6811/2735495271","cluster_addr":"172.21.15.167:6813/2735495271","heartbeat_back_addr":"172.21.15.167:6817/2735495271","heartbeat_front_addr":"172.21.15.167:6815/2735495271","state":["exists","up"]},{"osd":5,"uuid":"f14b44a9-5910-458c-b518-09a8d0de94de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6827","nonce":1494916572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6829","nonce":1494916572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6833","nonce":1494916572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6831","nonce":1494916572}]},"public_addr":"172.21.15.167:6827/1494916572","cluster_addr":"172.21.15.167:6829/1494916572","heartbeat_back_addr":"172.21.15.167:6833/1494916572","heartbeat_front_addr":"172.21.15.167:6831/1494916572","state":["exists","up"]},{"osd":6,"uuid":"39335545-2654-4710-84b0-a9aa47cfcbba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6819","nonce":2538273273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6821","nonce":2538273273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6825","nonce":2538273273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6823","nonce":2538273273}]},"public_addr":"172.21.15.167:6819/2538273273","cluster_addr":"172.21.15.167:6821/2538273273","heartbeat_back_addr":"172.21.15.167:6825/2538273273","heartbeat_front_addr":"172.21.15.167:6823/2538273273","state":["exists","up"]},{"osd":7,"uuid":"4f8bfc81-53dd-4339-ac22-1a4007d9f963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6803","nonce":265072227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6805","nonce":265072227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6809","nonce":265072227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6807","nonce":265072227}]},"public_addr":"172.21.15.167:6803/265072227","cluster_addr":"172.21.15.167:6805/265072227","heartbeat_back_addr":"172.21.15.167:6809/265072227","heartbeat_front_addr":"172.21.15.167:6807/265072227","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.150359+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.084358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.722368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.332809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.320709+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.395153+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.935250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.801443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:48:22.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.372+0000 7fb369a00640 1 -- 172.21.15.151:0/244780782 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb3400799f0 msgr2=0x7fb34007beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:22.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.372+0000 7fb369a00640 1 --2- 172.21.15.151:0/244780782 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb3400799f0 0x7fb34007beb0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fb354009cd0 tx=0x7fb354005f90 comp rx=0 tx=0).stop 2024-09-16T15:48:22.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.372+0000 7fb369a00640 1 -- 172.21.15.151:0/244780782 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3640cd670 msgr2=0x7fb3641f7b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:22.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.372+0000 7fb369a00640 1 --2- 172.21.15.151:0/244780782 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3640cd670 0x7fb3641f7b20 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fb35c004b90 tx=0x7fb35c004bc0 comp rx=0 tx=0).stop 2024-09-16T15:48:22.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.373+0000 7fb369a00640 1 -- 172.21.15.151:0/244780782 shutdown_connections 2024-09-16T15:48:22.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.373+0000 7fb369a00640 1 --2- 172.21.15.151:0/244780782 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb3400799f0 0x7fb34007beb0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:22.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.373+0000 7fb369a00640 1 --2- 172.21.15.151:0/244780782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3640b9670 0x7fb3640b8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:22.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.373+0000 7fb369a00640 1 --2- 172.21.15.151:0/244780782 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3640cd670 0x7fb3641f7b20 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:22.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.373+0000 7fb369a00640 1 --2- 172.21.15.151:0/244780782 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb348004d40 0x7fb3640cbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:22.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.373+0000 7fb369a00640 1 -- 172.21.15.151:0/244780782 >> 172.21.15.151:0/244780782 conn(0x7fb3640d3df0 msgr2=0x7fb3640d19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:22.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.373+0000 7fb369a00640 1 -- 172.21.15.151:0/244780782 shutdown_connections 2024-09-16T15:48:22.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.373+0000 7fb369a00640 1 -- 172.21.15.151:0/244780782 wait complete. 2024-09-16T15:48:22.392 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:48:22.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.547+0000 7f24b4200640 1 -- 172.21.15.151:0/2229642481 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24ac15f5a0 msgr2=0x7f24ac0b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:22.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.547+0000 7f24b4200640 1 --2- 172.21.15.151:0/2229642481 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24ac15f5a0 0x7f24ac0b2440 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f24a0009be0 tx=0x7f24a002a820 comp rx=0 tx=0).stop 2024-09-16T15:48:22.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.548+0000 7f24b4200640 1 -- 172.21.15.151:0/2229642481 shutdown_connections 2024-09-16T15:48:22.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.548+0000 7f24b4200640 1 --2- 172.21.15.151:0/2229642481 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24ac15f5a0 0x7f24ac0b2440 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:22.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.548+0000 7f24b4200640 1 -- 172.21.15.151:0/2229642481 >> 172.21.15.151:0/2229642481 conn(0x7f24ac0b1950 msgr2=0x7f24ac0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:22.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.548+0000 7f24b4200640 1 -- 172.21.15.151:0/2229642481 shutdown_connections 2024-09-16T15:48:22.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.548+0000 7f24b4200640 1 -- 172.21.15.151:0/2229642481 wait complete. 2024-09-16T15:48:22.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.549+0000 7f24b4200640 1 Processor -- start 2024-09-16T15:48:22.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.549+0000 7f24b4200640 1 -- start start 2024-09-16T15:48:22.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.550+0000 7f24b4200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f24ac073fb0 0x7f24ac146af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:22.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.550+0000 7f24b4200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24ac0ccf00 0x7f24ac0cb860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:22.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.550+0000 7f24b4200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24ac15f5a0 0x7f24ac0cbda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:22.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.550+0000 7f24b4200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f24ac06ef40 con 0x7f24ac073fb0 2024-09-16T15:48:22.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.550+0000 7f24b4200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f24ac06fa80 con 0x7f24ac15f5a0 2024-09-16T15:48:22.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.550+0000 7f24b4200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f24ac074460 con 0x7f24ac0ccf00 2024-09-16T15:48:22.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.551+0000 7f24b1800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f24ac073fb0 0x7f24ac146af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:22.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.551+0000 7f24b1800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f24ac073fb0 0x7f24ac146af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47710/0 (socket says 172.21.15.151:47710) 2024-09-16T15:48:22.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.551+0000 7f24b1800640 1 -- 172.21.15.151:0/1936296722 learned_addr learned my addr 172.21.15.151:0/1936296722 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:48:22.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.551+0000 7f24b0e00640 1 --2- 172.21.15.151:0/1936296722 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24ac0ccf00 0x7f24ac0cb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:22.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.551+0000 7f24b2200640 1 --2- 172.21.15.151:0/1936296722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24ac15f5a0 0x7f24ac0cbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:22.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.551+0000 7f24b1800640 1 -- 172.21.15.151:0/1936296722 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24ac0ccf00 msgr2=0x7f24ac0cb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:22.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.551+0000 7f24b1800640 1 --2- 172.21.15.151:0/1936296722 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24ac0ccf00 0x7f24ac0cb860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:22.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.551+0000 7f24b1800640 1 -- 172.21.15.151:0/1936296722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24ac15f5a0 msgr2=0x7f24ac0cbda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:22.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.551+0000 7f24b1800640 1 --2- 172.21.15.151:0/1936296722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24ac15f5a0 0x7f24ac0cbda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:22.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.551+0000 7f24b1800640 1 -- 172.21.15.151:0/1936296722 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24ac1f5640 con 0x7f24ac073fb0 2024-09-16T15:48:22.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.551+0000 7f24b2200640 1 --2- 172.21.15.151:0/1936296722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24ac15f5a0 0x7f24ac0cbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:48:22.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.551+0000 7f24b0e00640 1 --2- 172.21.15.151:0/1936296722 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24ac0ccf00 0x7f24ac0cb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:48:22.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.551+0000 7f24b1800640 1 --2- 172.21.15.151:0/1936296722 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f24ac073fb0 0x7f24ac146af0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f24a0009be0 tx=0x7f24a0030650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:22.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.552+0000 7f24a9600640 1 -- 172.21.15.151:0/1936296722 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a0041030 con 0x7f24ac073fb0 2024-09-16T15:48:22.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.552+0000 7f24a9600640 1 -- 172.21.15.151:0/1936296722 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f24a0003970 con 0x7f24ac073fb0 2024-09-16T15:48:22.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.552+0000 7f24a9600640 1 -- 172.21.15.151:0/1936296722 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a0036040 con 0x7f24ac073fb0 2024-09-16T15:48:22.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.552+0000 7f24b4200640 1 -- 172.21.15.151:0/1936296722 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24ac0cebd0 con 0x7f24ac073fb0 2024-09-16T15:48:22.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.552+0000 7f24b4200640 1 -- 172.21.15.151:0/1936296722 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24ac1f5920 con 0x7f24ac073fb0 2024-09-16T15:48:22.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.553+0000 7f24b4200640 1 -- 172.21.15.151:0/1936296722 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24ac0b2440 con 0x7f24ac073fb0 2024-09-16T15:48:22.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.554+0000 7f24a9600640 1 -- 172.21.15.151:0/1936296722 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 30) ==== 102868+0+0 (secure 0 0 0) 0x7f24a0044070 con 0x7f24ac073fb0 2024-09-16T15:48:22.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.555+0000 7f24a9600640 1 --2- 172.21.15.151:0/1936296722 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f248c0799c0 0x7f248c07be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:22.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.555+0000 7f24a9600640 1 -- 172.21.15.151:0/1936296722 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f24a00ba5f0 con 0x7f24ac073fb0 2024-09-16T15:48:22.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.555+0000 7f24b0e00640 1 --2- 172.21.15.151:0/1936296722 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f248c0799c0 0x7f248c07be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:22.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.556+0000 7f24b0e00640 1 --2- 172.21.15.151:0/1936296722 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f248c0799c0 0x7f248c07be80 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2498001220 tx=0x7f2498009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:22.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.559+0000 7f24a9600640 1 -- 172.21.15.151:0/1936296722 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f24a0084360 con 0x7f24ac073fb0 2024-09-16T15:48:22.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.746+0000 7f24b4200640 1 -- 172.21.15.151:0/1936296722 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f24ac0b2140 con 0x7f24ac073fb0 2024-09-16T15:48:22.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.747+0000 7f24a9600640 1 -- 172.21.15.151:0/1936296722 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+16892 (secure 0 0 0) 0x7f24a0083c60 con 0x7f24ac073fb0 2024-09-16T15:48:22.749 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:48:22.749 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":90,"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","created":"2024-09-16T15:38:56.348535+0000","modified":"2024-09-16T15:48:21.374120+0000","last_up_change":"2024-09-16T15:39:10.681384+0000","last_in_change":"2024-09-16T15:39:00.544477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:39:11.602252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:48:19.150224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-16T15:48:20.351519+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5524076-a903-4d0a-af6a-caf474c4765d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6809","nonce":3111342741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6811","nonce":3111342741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6815","nonce":3111342741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6813","nonce":3111342741}]},"public_addr":"172.21.15.151:6809/3111342741","cluster_addr":"172.21.15.151:6811/3111342741","heartbeat_back_addr":"172.21.15.151:6815/3111342741","heartbeat_front_addr":"172.21.15.151:6813/3111342741","state":["exists","up"]},{"osd":1,"uuid":"dd25925f-0ae1-494e-8529-4dc205759db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6801","nonce":3368304087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6803","nonce":3368304087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6807","nonce":3368304087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6805","nonce":3368304087}]},"public_addr":"172.21.15.151:6801/3368304087","cluster_addr":"172.21.15.151:6803/3368304087","heartbeat_back_addr":"172.21.15.151:6807/3368304087","heartbeat_front_addr":"172.21.15.151:6805/3368304087","state":["exists","up"]},{"osd":2,"uuid":"c2f18115-013d-4da6-a0fe-cb978f4494b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6823","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6825","nonce":3079373271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6827","nonce":3079373271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6831","nonce":3079373271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6829","nonce":3079373271}]},"public_addr":"172.21.15.151:6825/3079373271","cluster_addr":"172.21.15.151:6827/3079373271","heartbeat_back_addr":"172.21.15.151:6831/3079373271","heartbeat_front_addr":"172.21.15.151:6829/3079373271","state":["exists","up"]},{"osd":3,"uuid":"36edb718-2fcb-4d88-8ddc-c321eed4279d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6817","nonce":1142509041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6819","nonce":1142509041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6824","nonce":1142509041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6821","nonce":1142509041}]},"public_addr":"172.21.15.151:6817/1142509041","cluster_addr":"172.21.15.151:6819/1142509041","heartbeat_back_addr":"172.21.15.151:6824/1142509041","heartbeat_front_addr":"172.21.15.151:6821/1142509041","state":["exists","up"]},{"osd":4,"uuid":"64d33708-cd0a-4236-abe3-1b646e82ab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6811","nonce":2735495271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6813","nonce":2735495271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6817","nonce":2735495271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6815","nonce":2735495271}]},"public_addr":"172.21.15.167:6811/2735495271","cluster_addr":"172.21.15.167:6813/2735495271","heartbeat_back_addr":"172.21.15.167:6817/2735495271","heartbeat_front_addr":"172.21.15.167:6815/2735495271","state":["exists","up"]},{"osd":5,"uuid":"f14b44a9-5910-458c-b518-09a8d0de94de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6827","nonce":1494916572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6829","nonce":1494916572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6833","nonce":1494916572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6831","nonce":1494916572}]},"public_addr":"172.21.15.167:6827/1494916572","cluster_addr":"172.21.15.167:6829/1494916572","heartbeat_back_addr":"172.21.15.167:6833/1494916572","heartbeat_front_addr":"172.21.15.167:6831/1494916572","state":["exists","up"]},{"osd":6,"uuid":"39335545-2654-4710-84b0-a9aa47cfcbba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6819","nonce":2538273273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6821","nonce":2538273273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6825","nonce":2538273273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6823","nonce":2538273273}]},"public_addr":"172.21.15.167:6819/2538273273","cluster_addr":"172.21.15.167:6821/2538273273","heartbeat_back_addr":"172.21.15.167:6825/2538273273","heartbeat_front_addr":"172.21.15.167:6823/2538273273","state":["exists","up"]},{"osd":7,"uuid":"4f8bfc81-53dd-4339-ac22-1a4007d9f963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6803","nonce":265072227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6805","nonce":265072227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6809","nonce":265072227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6807","nonce":265072227}]},"public_addr":"172.21.15.167:6803/265072227","cluster_addr":"172.21.15.167:6805/265072227","heartbeat_back_addr":"172.21.15.167:6809/265072227","heartbeat_front_addr":"172.21.15.167:6807/265072227","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.150359+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.084358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.722368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.332809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.320709+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.395153+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.935250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.801443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:48:22.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.749+0000 7f24b4200640 1 -- 172.21.15.151:0/1936296722 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f248c0799c0 msgr2=0x7f248c07be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:22.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.749+0000 7f24b4200640 1 --2- 172.21.15.151:0/1936296722 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f248c0799c0 0x7f248c07be80 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2498001220 tx=0x7f2498009040 comp rx=0 tx=0).stop 2024-09-16T15:48:22.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.749+0000 7f24b4200640 1 -- 172.21.15.151:0/1936296722 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f24ac073fb0 msgr2=0x7f24ac146af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:22.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.749+0000 7f24b4200640 1 --2- 172.21.15.151:0/1936296722 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f24ac073fb0 0x7f24ac146af0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f24a0009be0 tx=0x7f24a0030650 comp rx=0 tx=0).stop 2024-09-16T15:48:22.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.749+0000 7f24b4200640 1 -- 172.21.15.151:0/1936296722 shutdown_connections 2024-09-16T15:48:22.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.749+0000 7f24b4200640 1 --2- 172.21.15.151:0/1936296722 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f248c0799c0 0x7f248c07be80 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:22.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.749+0000 7f24b4200640 1 --2- 172.21.15.151:0/1936296722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24ac15f5a0 0x7f24ac0cbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:22.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.749+0000 7f24b4200640 1 --2- 172.21.15.151:0/1936296722 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f24ac0ccf00 0x7f24ac0cb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:22.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.749+0000 7f24b4200640 1 --2- 172.21.15.151:0/1936296722 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f24ac073fb0 0x7f24ac146af0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:22.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.749+0000 7f24b4200640 1 -- 172.21.15.151:0/1936296722 >> 172.21.15.151:0/1936296722 conn(0x7f24ac0b1950 msgr2=0x7f24ac09d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:22.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.749+0000 7f24b4200640 1 -- 172.21.15.151:0/1936296722 shutdown_connections 2024-09-16T15:48:22.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.749+0000 7f24b4200640 1 -- 172.21.15.151:0/1936296722 wait complete. 2024-09-16T15:48:22.767 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-16T15:48:22.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.927+0000 7f9f7be00640 1 -- 172.21.15.151:0/2944465323 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9f7c0b3450 msgr2=0x7f9f7c0b3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:22.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.927+0000 7f9f7be00640 1 --2- 172.21.15.151:0/2944465323 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9f7c0b3450 0x7f9f7c0b3890 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c08df50 tx=0x7f9f6802a4b0 comp rx=0 tx=0).stop 2024-09-16T15:48:22.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.927+0000 7f9f7be00640 1 -- 172.21.15.151:0/2944465323 shutdown_connections 2024-09-16T15:48:22.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.927+0000 7f9f7be00640 1 --2- 172.21.15.151:0/2944465323 >> v2:172.21.15.167:3300/0 conn(0x7f9f7c0cf4b0 0x7f9f7c0cf900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:22.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.927+0000 7f9f7be00640 1 --2- 172.21.15.151:0/2944465323 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9f7c0b3450 0x7f9f7c0b3890 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:22.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.927+0000 7f9f7be00640 1 -- 172.21.15.151:0/2944465323 >> 172.21.15.151:0/2944465323 conn(0x7f9f7c0d3d50 msgr2=0x7f9f7c0d4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:22.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.927+0000 7f9f7be00640 1 -- 172.21.15.151:0/2944465323 shutdown_connections 2024-09-16T15:48:22.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.927+0000 7f9f7be00640 1 -- 172.21.15.151:0/2944465323 wait complete. 2024-09-16T15:48:22.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.928+0000 7f9f7be00640 1 Processor -- start 2024-09-16T15:48:22.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.929+0000 7f9f7be00640 1 -- start start 2024-09-16T15:48:22.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.929+0000 7f9f7be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9f7c0b3450 0x7f9f7c15fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:22.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.929+0000 7f9f7be00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f7c0cf4b0 0x7f9f7c06e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:22.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.929+0000 7f9f7be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f7c15df60 0x7f9f7c06e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:22.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.929+0000 7f9f7be00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9f7c0d2330 con 0x7f9f7c0b3450 2024-09-16T15:48:22.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.929+0000 7f9f7be00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9f7c0cea70 con 0x7f9f7c15df60 2024-09-16T15:48:22.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.929+0000 7f9f7be00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f9f7c0d24b0 con 0x7f9f7c0cf4b0 2024-09-16T15:48:22.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.930+0000 7f9f7aa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9f7c0b3450 0x7f9f7c15fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:22.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.930+0000 7f9f7aa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9f7c0b3450 0x7f9f7c15fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:42644/0 (socket says 172.21.15.151:42644) 2024-09-16T15:48:22.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.930+0000 7f9f7aa00640 1 -- 172.21.15.151:0/1596520411 learned_addr learned my addr 172.21.15.151:0/1596520411 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:48:22.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.930+0000 7f9f7a000640 1 --2- 172.21.15.151:0/1596520411 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f7c0cf4b0 0x7f9f7c06e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:22.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.930+0000 7f9f7b400640 1 --2- 172.21.15.151:0/1596520411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f7c15df60 0x7f9f7c06e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:22.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.930+0000 7f9f7aa00640 1 -- 172.21.15.151:0/1596520411 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f7c0cf4b0 msgr2=0x7f9f7c06e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:22.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.930+0000 7f9f7aa00640 1 --2- 172.21.15.151:0/1596520411 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f7c0cf4b0 0x7f9f7c06e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:22.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.930+0000 7f9f7aa00640 1 -- 172.21.15.151:0/1596520411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f7c15df60 msgr2=0x7f9f7c06e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:22.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.930+0000 7f9f7aa00640 1 --2- 172.21.15.151:0/1596520411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f7c15df60 0x7f9f7c06e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:22.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.930+0000 7f9f7aa00640 1 -- 172.21.15.151:0/1596520411 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f7c1f8840 con 0x7f9f7c0b3450 2024-09-16T15:48:22.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.930+0000 7f9f7b400640 1 --2- 172.21.15.151:0/1596520411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f7c15df60 0x7f9f7c06e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:48:22.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.930+0000 7f9f7aa00640 1 --2- 172.21.15.151:0/1596520411 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9f7c0b3450 0x7f9f7c15fd90 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c00c3a0 tx=0x7f9f6c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:22.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.931+0000 7f9f72a00640 1 -- 172.21.15.151:0/1596520411 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c011790 con 0x7f9f7c0b3450 2024-09-16T15:48:22.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.931+0000 7f9f72a00640 1 -- 172.21.15.151:0/1596520411 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9f6c017070 con 0x7f9f7c0b3450 2024-09-16T15:48:22.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.931+0000 7f9f7be00640 1 -- 172.21.15.151:0/1596520411 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f7c1f4750 con 0x7f9f7c0b3450 2024-09-16T15:48:22.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.931+0000 7f9f72a00640 1 -- 172.21.15.151:0/1596520411 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c011cc0 con 0x7f9f7c0b3450 2024-09-16T15:48:22.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.931+0000 7f9f7be00640 1 -- 172.21.15.151:0/1596520411 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f7c1f4d10 con 0x7f9f7c0b3450 2024-09-16T15:48:22.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.932+0000 7f9f7be00640 1 -- 172.21.15.151:0/1596520411 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f7c0d2a40 con 0x7f9f7c0b3450 2024-09-16T15:48:22.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.933+0000 7f9f72a00640 1 -- 172.21.15.151:0/1596520411 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 30) ==== 102868+0+0 (secure 0 0 0) 0x7f9f6c01c460 con 0x7f9f7c0b3450 2024-09-16T15:48:22.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.934+0000 7f9f72a00640 1 --2- 172.21.15.151:0/1596520411 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9f5c0799c0 0x7f9f5c07be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:22.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.934+0000 7f9f72a00640 1 -- 172.21.15.151:0/1596520411 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5765+0+0 (secure 0 0 0) 0x7f9f6c09a040 con 0x7f9f7c0b3450 2024-09-16T15:48:22.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.935+0000 7f9f7a000640 1 --2- 172.21.15.151:0/1596520411 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9f5c0799c0 0x7f9f5c07be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:22.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.935+0000 7f9f7a000640 1 --2- 172.21.15.151:0/1596520411 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9f5c0799c0 0x7f9f5c07be80 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f9f68034000 tx=0x7f9f68004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:22.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:22.937+0000 7f9f72a00640 1 -- 172.21.15.151:0/1596520411 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9f6c063df0 con 0x7f9f7c0b3450 2024-09-16T15:48:23.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:23.160+0000 7f9f7be00640 1 -- 172.21.15.151:0/1596520411 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f9f7c0b4b50 con 0x7f9f7c0b3450 2024-09-16T15:48:23.947 INFO:tasks.ceph.mds.c.smithi151.stderr:2024-09-16T15:48:23.945+0000 7f4a24600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-16T15:48:23.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:23.991+0000 7f9f72a00640 1 -- 172.21.15.151:0/1596520411 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v55) ==== 118+0+0 (secure 0 0 0) 0x7f9f6c01ae40 con 0x7f9f7c0b3450 2024-09-16T15:48:23.996 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:23.993+0000 7f9f7be00640 1 -- 172.21.15.151:0/1596520411 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9f5c0799c0 msgr2=0x7f9f5c07be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:23.996 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:23.993+0000 7f9f7be00640 1 --2- 172.21.15.151:0/1596520411 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9f5c0799c0 0x7f9f5c07be80 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f9f68034000 tx=0x7f9f68004730 comp rx=0 tx=0).stop 2024-09-16T15:48:23.996 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:23.993+0000 7f9f7be00640 1 -- 172.21.15.151:0/1596520411 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9f7c0b3450 msgr2=0x7f9f7c15fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:23.996 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:23.993+0000 7f9f7be00640 1 --2- 172.21.15.151:0/1596520411 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9f7c0b3450 0x7f9f7c15fd90 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c00c3a0 tx=0x7f9f6c00c860 comp rx=0 tx=0).stop 2024-09-16T15:48:23.996 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:23.993+0000 7f9f7be00640 1 -- 172.21.15.151:0/1596520411 shutdown_connections 2024-09-16T15:48:23.996 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:23.993+0000 7f9f7be00640 1 --2- 172.21.15.151:0/1596520411 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9f5c0799c0 0x7f9f5c07be80 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:23.996 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:23.993+0000 7f9f7be00640 1 --2- 172.21.15.151:0/1596520411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f7c15df60 0x7f9f7c06e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:23.996 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:23.993+0000 7f9f7be00640 1 --2- 172.21.15.151:0/1596520411 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f7c0cf4b0 0x7f9f7c06e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:23.997 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:23.993+0000 7f9f7be00640 1 --2- 172.21.15.151:0/1596520411 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9f7c0b3450 0x7f9f7c15fd90 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:23.997 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:23.993+0000 7f9f7be00640 1 -- 172.21.15.151:0/1596520411 >> 172.21.15.151:0/1596520411 conn(0x7f9f7c0d3d50 msgr2=0x7f9f7c0b1720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:23.997 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:23.994+0000 7f9f7be00640 1 -- 172.21.15.151:0/1596520411 shutdown_connections 2024-09-16T15:48:23.997 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:23.994+0000 7f9f7be00640 1 -- 172.21.15.151:0/1596520411 wait complete. 2024-09-16T15:48:24.012 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-16T15:48:24.013 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-16T15:48:24.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:24.173+0000 7fc194200640 1 -- 172.21.15.151:0/3297620374 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc174004050 msgr2=0x7fc174004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:24.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:24.173+0000 7fc194200640 1 --2- 172.21.15.151:0/3297620374 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc174004050 0x7fc174004800 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fc1780016c0 tx=0x7fc174005830 comp rx=0 tx=0).stop 2024-09-16T15:48:24.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:24.173+0000 7fc192200640 1 -- 172.21.15.151:0/3297620374 reap_dead start 2024-09-16T15:48:24.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:24.173+0000 7fc194200640 1 -- 172.21.15.151:0/3297620374 shutdown_connections 2024-09-16T15:48:24.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:24.173+0000 7fc194200640 1 -- 172.21.15.151:0/3297620374 >> 172.21.15.151:0/3297620374 conn(0x7fc18c0b1840 msgr2=0x7fc18c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:24.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:24.174+0000 7fc194200640 1 -- 172.21.15.151:0/3297620374 shutdown_connections 2024-09-16T15:48:24.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:24.174+0000 7fc194200640 1 -- 172.21.15.151:0/3297620374 wait complete. 2024-09-16T15:48:24.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:24.175+0000 7fc194200640 1 Processor -- start 2024-09-16T15:48:24.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:24.175+0000 7fc194200640 1 -- start start 2024-09-16T15:48:24.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:24.176+0000 7fc194200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc174004050 0x7fc18c1413c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:24.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:24.176+0000 7fc194200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc18c15e7f0 0x7fc18c1f4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:24.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:24.176+0000 7fc194200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc18c0c6950 0x7fc18c0c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:24.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:24.176+0000 7fc194200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc18c0ce2c0 con 0x7fc174004050 2024-09-16T15:48:24.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:24.176+0000 7fc194200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc18c0736d0 con 0x7fc18c0c6950 2024-09-16T15:48:24.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:24.176+0000 7fc194200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fc18c0ce440 con 0x7fc18c15e7f0 2024-09-16T15:48:24.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:24.176+0000 7fc191800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc174004050 0x7fc18c1413c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:24.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:24.176+0000 7fc191800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc174004050 0x7fc18c1413c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:42664/0 (socket says 172.21.15.151:42664) 2024-09-16T15:48:24.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:24.176+0000 7fc191800640 1 -- 172.21.15.151:0/2374928175 learned_addr learned my addr 172.21.15.151:0/2374928175 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:48:24.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:24.177+0000 7fc191800640 1 -- 172.21.15.151:0/2374928175 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc18c15e7f0 msgr2=0x7fc18c1f4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:24.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:24.177+0000 7fc191800640 1 --2- 172.21.15.151:0/2374928175 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc18c15e7f0 0x7fc18c1f4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:24.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:24.177+0000 7fc191800640 1 -- 172.21.15.151:0/2374928175 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc18c0c6950 msgr2=0x7fc18c0c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:24.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:24.177+0000 7fc192200640 1 --2- 172.21.15.151:0/2374928175 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc18c0c6950 0x7fc18c0c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:24.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:24.177+0000 7fc191800640 1 --2- 172.21.15.151:0/2374928175 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc18c0c6950 0x7fc18c0c6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:24.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:24.177+0000 7fc191800640 1 -- 172.21.15.151:0/2374928175 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc18c1f4540 con 0x7fc174004050 2024-09-16T15:48:24.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:24.177+0000 7fc192200640 1 --2- 172.21.15.151:0/2374928175 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc18c0c6950 0x7fc18c0c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:48:24.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:24.177+0000 7fc191800640 1 --2- 172.21.15.151:0/2374928175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc174004050 0x7fc18c1413c0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fc180006340 tx=0x7fc180006740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:24.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:24.177+0000 7fc189600640 1 -- 172.21.15.151:0/2374928175 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18000fb40 con 0x7fc174004050 2024-09-16T15:48:24.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:24.177+0000 7fc189600640 1 -- 172.21.15.151:0/2374928175 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc180016020 con 0x7fc174004050 2024-09-16T15:48:24.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:24.177+0000 7fc194200640 1 -- 172.21.15.151:0/2374928175 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc18c0c5470 con 0x7fc174004050 2024-09-16T15:48:24.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:24.177+0000 7fc189600640 1 -- 172.21.15.151:0/2374928175 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc180011430 con 0x7fc174004050 2024-09-16T15:48:24.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:24.178+0000 7fc194200640 1 -- 172.21.15.151:0/2374928175 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc18c0c5710 con 0x7fc174004050 2024-09-16T15:48:24.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:24.179+0000 7fc194200640 1 -- 172.21.15.151:0/2374928175 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc18c0ce9a0 con 0x7fc174004050 2024-09-16T15:48:24.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:24.180+0000 7fc189600640 1 -- 172.21.15.151:0/2374928175 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 30) ==== 102868+0+0 (secure 0 0 0) 0x7fc1800115d0 con 0x7fc174004050 2024-09-16T15:48:24.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:24.181+0000 7fc189600640 1 --2- 172.21.15.151:0/2374928175 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc16c0799c0 0x7fc16c07be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:24.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:24.181+0000 7fc189600640 1 -- 172.21.15.151:0/2374928175 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7fc180099fb0 con 0x7fc174004050 2024-09-16T15:48:24.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:24.181+0000 7fc190e00640 1 --2- 172.21.15.151:0/2374928175 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc16c0799c0 0x7fc16c07be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:24.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:24.182+0000 7fc190e00640 1 --2- 172.21.15.151:0/2374928175 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc16c0799c0 0x7fc16c07be80 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fc178032000 tx=0x7fc178001120 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:24.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:24.185+0000 7fc189600640 1 -- 172.21.15.151:0/2374928175 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc180020030 con 0x7fc174004050 2024-09-16T15:48:24.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:24.406+0000 7fc194200640 1 -- 172.21.15.151:0/2374928175 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fc18c0c77c0 con 0x7fc174004050 2024-09-16T15:48:24.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:24.992+0000 7fc189600640 1 -- 172.21.15.151:0/2374928175 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v56) ==== 122+0+0 (secure 0 0 0) 0x7fc18c1f4540 con 0x7fc174004050 2024-09-16T15:48:24.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:24.995+0000 7fc194200640 1 -- 172.21.15.151:0/2374928175 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc16c0799c0 msgr2=0x7fc16c07be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:24.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:24.995+0000 7fc194200640 1 --2- 172.21.15.151:0/2374928175 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc16c0799c0 0x7fc16c07be80 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fc178032000 tx=0x7fc178001120 comp rx=0 tx=0).stop 2024-09-16T15:48:24.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:24.995+0000 7fc194200640 1 -- 172.21.15.151:0/2374928175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc174004050 msgr2=0x7fc18c1413c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:24.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:24.995+0000 7fc194200640 1 --2- 172.21.15.151:0/2374928175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc174004050 0x7fc18c1413c0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fc180006340 tx=0x7fc180006740 comp rx=0 tx=0).stop 2024-09-16T15:48:24.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:24.995+0000 7fc194200640 1 -- 172.21.15.151:0/2374928175 shutdown_connections 2024-09-16T15:48:24.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:24.995+0000 7fc194200640 1 --2- 172.21.15.151:0/2374928175 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc16c0799c0 0x7fc16c07be80 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:24.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:24.995+0000 7fc194200640 1 --2- 172.21.15.151:0/2374928175 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc18c0c6950 0x7fc18c0c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:24.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:24.995+0000 7fc194200640 1 --2- 172.21.15.151:0/2374928175 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc18c15e7f0 0x7fc18c1f4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:24.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:24.995+0000 7fc194200640 1 --2- 172.21.15.151:0/2374928175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc174004050 0x7fc18c1413c0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:24.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:24.995+0000 7fc194200640 1 -- 172.21.15.151:0/2374928175 >> 172.21.15.151:0/2374928175 conn(0x7fc18c0b1840 msgr2=0x7fc18c0ad870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:24.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:24.995+0000 7fc194200640 1 -- 172.21.15.151:0/2374928175 shutdown_connections 2024-09-16T15:48:24.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:24.996+0000 7fc194200640 1 -- 172.21.15.151:0/2374928175 wait complete. 2024-09-16T15:48:25.015 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-16T15:48:25.015 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:48:25.174 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.172+0000 7f32bf800640 1 -- 172.21.15.151:0/2635223214 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f32b80cf300 msgr2=0x7f32b80729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:25.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.172+0000 7f32bf800640 1 --2- 172.21.15.151:0/2635223214 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f32b80cf300 0x7f32b80729a0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f32a4009860 tx=0x7f32a402a5f0 comp rx=0 tx=0).stop 2024-09-16T15:48:25.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.172+0000 7f32bf800640 1 -- 172.21.15.151:0/2635223214 shutdown_connections 2024-09-16T15:48:25.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.172+0000 7f32bf800640 1 --2- 172.21.15.151:0/2635223214 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f32b80cf300 0x7f32b80729a0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:25.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.172+0000 7f32bf800640 1 -- 172.21.15.151:0/2635223214 >> 172.21.15.151:0/2635223214 conn(0x7f32b80b1950 msgr2=0x7f32b80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:25.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.172+0000 7f32bf800640 1 -- 172.21.15.151:0/2635223214 shutdown_connections 2024-09-16T15:48:25.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.172+0000 7f32bf800640 1 -- 172.21.15.151:0/2635223214 wait complete. 2024-09-16T15:48:25.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.174+0000 7f32bf800640 1 Processor -- start 2024-09-16T15:48:25.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.174+0000 7f32bf800640 1 -- start start 2024-09-16T15:48:25.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.175+0000 7f32bf800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32b80ccfd0 0x7f32b80c4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:25.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.175+0000 7f32bf800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f32b80cf300 0x7f32b8141710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:25.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.175+0000 7f32bf800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f32b81445d0 0x7f32b806fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:25.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.175+0000 7f32bf800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f32b80735f0 con 0x7f32b81445d0 2024-09-16T15:48:25.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.175+0000 7f32bf800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f32b80b4f60 con 0x7f32b80ccfd0 2024-09-16T15:48:25.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.175+0000 7f32bf800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f32b8073770 con 0x7f32b80cf300 2024-09-16T15:48:25.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.175+0000 7f32bd800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f32b81445d0 0x7f32b806fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:25.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.175+0000 7f32bd800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f32b81445d0 0x7f32b806fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:42684/0 (socket says 172.21.15.151:42684) 2024-09-16T15:48:25.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.175+0000 7f32bd800640 1 -- 172.21.15.151:0/197195889 learned_addr learned my addr 172.21.15.151:0/197195889 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:48:25.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.175+0000 7f32bce00640 1 --2- 172.21.15.151:0/197195889 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32b80ccfd0 0x7f32b80c4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:25.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.176+0000 7f32b7e00640 1 --2- 172.21.15.151:0/197195889 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f32b80cf300 0x7f32b8141710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:25.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.176+0000 7f32bd800640 1 -- 172.21.15.151:0/197195889 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f32b80cf300 msgr2=0x7f32b8141710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:25.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.176+0000 7f32bd800640 1 --2- 172.21.15.151:0/197195889 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f32b80cf300 0x7f32b8141710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:25.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.176+0000 7f32bd800640 1 -- 172.21.15.151:0/197195889 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32b80ccfd0 msgr2=0x7f32b80c4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:25.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.176+0000 7f32bd800640 1 --2- 172.21.15.151:0/197195889 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32b80ccfd0 0x7f32b80c4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:25.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.176+0000 7f32bd800640 1 -- 172.21.15.151:0/197195889 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32b81f5220 con 0x7f32b81445d0 2024-09-16T15:48:25.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.176+0000 7f32b7e00640 1 --2- 172.21.15.151:0/197195889 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f32b80cf300 0x7f32b8141710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:48:25.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.176+0000 7f32bd800640 1 --2- 172.21.15.151:0/197195889 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f32b81445d0 0x7f32b806fe10 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f32b000cbc0 tx=0x7f32b000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:25.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.176+0000 7f32b4c00640 1 -- 172.21.15.151:0/197195889 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b0016020 con 0x7f32b81445d0 2024-09-16T15:48:25.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.176+0000 7f32b4c00640 1 -- 172.21.15.151:0/197195889 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f32b000ae30 con 0x7f32b81445d0 2024-09-16T15:48:25.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.176+0000 7f32b4c00640 1 -- 172.21.15.151:0/197195889 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b0012660 con 0x7f32b81445d0 2024-09-16T15:48:25.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.176+0000 7f32bf800640 1 -- 172.21.15.151:0/197195889 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32b81f5500 con 0x7f32b81445d0 2024-09-16T15:48:25.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.177+0000 7f32bf800640 1 -- 172.21.15.151:0/197195889 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32b81f5ac0 con 0x7f32b81445d0 2024-09-16T15:48:25.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.177+0000 7f32bf800640 1 -- 172.21.15.151:0/197195889 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32b809d120 con 0x7f32b81445d0 2024-09-16T15:48:25.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.179+0000 7f32b4c00640 1 -- 172.21.15.151:0/197195889 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 30) ==== 102868+0+0 (secure 0 0 0) 0x7f32b0019020 con 0x7f32b81445d0 2024-09-16T15:48:25.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.179+0000 7f32b4c00640 1 --2- 172.21.15.151:0/197195889 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f32980799c0 0x7f329807be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:25.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.180+0000 7f32b4c00640 1 -- 172.21.15.151:0/197195889 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7f32b0099b10 con 0x7f32b81445d0 2024-09-16T15:48:25.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.180+0000 7f32bce00640 1 --2- 172.21.15.151:0/197195889 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f32980799c0 0x7f329807be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:25.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.181+0000 7f32bce00640 1 --2- 172.21.15.151:0/197195889 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f32980799c0 0x7f329807be80 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f32a8001720 tx=0x7f32a8009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:25.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.184+0000 7f32b4c00640 1 -- 172.21.15.151:0/197195889 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f32b809d120 con 0x7f32b81445d0 2024-09-16T15:48:25.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.403+0000 7f32bf800640 1 -- 172.21.15.151:0/197195889 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f32b81f5e00 con 0x7f32b81445d0 2024-09-16T15:48:25.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.403+0000 7f32b4c00640 1 -- 172.21.15.151:0/197195889 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+5234 (secure 0 0 0) 0x7f32b0020030 con 0x7f32b81445d0 2024-09-16T15:48:25.406 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:48:25.406 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":56,"btime":"2024-09-16T15:48:24:992606+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6134,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/743718626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":743718626},{"type":"v1","addr":"172.21.15.167:6836","nonce":743718626}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6137,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1262689373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1262689373},{"type":"v1","addr":"172.21.15.167:6837","nonce":1262689373}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6670,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/1127792363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1127792363},{"type":"v1","addr":"172.21.15.151:6834","nonce":1127792363}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:48:21.374084+0000","modified":"2024-09-16T15:48:24.408969+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6673},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6673":{"gid":6673,"name":"c","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.151:6835/1518702790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":1518702790},{"type":"v1","addr":"172.21.15.151:6835","nonce":1518702790}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6673,"qdb_cluster":[6673]},"id":6}]} 2024-09-16T15:48:25.406 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 56 2024-09-16T15:48:25.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.406+0000 7f32bf800640 1 -- 172.21.15.151:0/197195889 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f32980799c0 msgr2=0x7f329807be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:25.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.406+0000 7f32bf800640 1 --2- 172.21.15.151:0/197195889 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f32980799c0 0x7f329807be80 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f32a8001720 tx=0x7f32a8009040 comp rx=0 tx=0).stop 2024-09-16T15:48:25.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.406+0000 7f32bf800640 1 -- 172.21.15.151:0/197195889 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f32b81445d0 msgr2=0x7f32b806fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:25.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.406+0000 7f32bf800640 1 --2- 172.21.15.151:0/197195889 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f32b81445d0 0x7f32b806fe10 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f32b000cbc0 tx=0x7f32b000a430 comp rx=0 tx=0).stop 2024-09-16T15:48:25.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.406+0000 7f32bf800640 1 -- 172.21.15.151:0/197195889 shutdown_connections 2024-09-16T15:48:25.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.406+0000 7f32bf800640 1 --2- 172.21.15.151:0/197195889 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f32980799c0 0x7f329807be80 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:25.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.406+0000 7f32bf800640 1 --2- 172.21.15.151:0/197195889 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f32b81445d0 0x7f32b806fe10 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:25.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.406+0000 7f32bf800640 1 --2- 172.21.15.151:0/197195889 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f32b80cf300 0x7f32b8141710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:25.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.406+0000 7f32bf800640 1 --2- 172.21.15.151:0/197195889 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32b80ccfd0 0x7f32b80c4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:25.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.406+0000 7f32bf800640 1 -- 172.21.15.151:0/197195889 >> 172.21.15.151:0/197195889 conn(0x7f32b80b1950 msgr2=0x7f32b809ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:25.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.406+0000 7f32bf800640 1 -- 172.21.15.151:0/197195889 shutdown_connections 2024-09-16T15:48:25.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.406+0000 7f32bf800640 1 -- 172.21.15.151:0/197195889 wait complete. 2024-09-16T15:48:25.424 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:48:25.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.582+0000 7fe9ade00640 1 -- 172.21.15.151:0/59790968 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe990004d40 msgr2=0x7fe9900051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:25.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.582+0000 7fe9ade00640 1 --2- 172.21.15.151:0/59790968 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe990004d40 0x7fe9900051c0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fe9a000c770 tx=0x7fe990005830 comp rx=0 tx=0).stop 2024-09-16T15:48:25.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.582+0000 7fe9ad400640 1 -- 172.21.15.151:0/59790968 reap_dead start 2024-09-16T15:48:25.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.582+0000 7fe9ade00640 1 -- 172.21.15.151:0/59790968 shutdown_connections 2024-09-16T15:48:25.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.582+0000 7fe9ade00640 1 -- 172.21.15.151:0/59790968 >> 172.21.15.151:0/59790968 conn(0x7fe9a80d3df0 msgr2=0x7fe9a80d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:25.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.582+0000 7fe9ade00640 1 -- 172.21.15.151:0/59790968 shutdown_connections 2024-09-16T15:48:25.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.582+0000 7fe9ade00640 1 -- 172.21.15.151:0/59790968 wait complete. 2024-09-16T15:48:25.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.583+0000 7fe9ade00640 1 Processor -- start 2024-09-16T15:48:25.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.584+0000 7fe9ade00640 1 -- start start 2024-09-16T15:48:25.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.584+0000 7fe9ade00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe990004d40 0x7fe9a81f3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:25.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.585+0000 7fe9ade00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9a8159e70 0x7fe9a80d0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:25.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.585+0000 7fe9ade00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9a80d0f40 0x7fe9a81f7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:25.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.585+0000 7fe9ade00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a80b1750 con 0x7fe9a8159e70 2024-09-16T15:48:25.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.585+0000 7fe9ade00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a80b15d0 con 0x7fe9a80d0f40 2024-09-16T15:48:25.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.585+0000 7fe9ade00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fe9a80b18d0 con 0x7fe990004d40 2024-09-16T15:48:25.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.585+0000 7fe9a7e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9a8159e70 0x7fe9a80d0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:25.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.585+0000 7fe9a7e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9a8159e70 0x7fe9a80d0a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:42716/0 (socket says 172.21.15.151:42716) 2024-09-16T15:48:25.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.585+0000 7fe9a7e00640 1 -- 172.21.15.151:0/1970499486 learned_addr learned my addr 172.21.15.151:0/1970499486 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:48:25.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.585+0000 7fe9aca00640 1 --2- 172.21.15.151:0/1970499486 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe990004d40 0x7fe9a81f3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:25.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.585+0000 7fe9ad400640 1 --2- 172.21.15.151:0/1970499486 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9a80d0f40 0x7fe9a81f7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:25.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.585+0000 7fe9a7e00640 1 -- 172.21.15.151:0/1970499486 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe990004d40 msgr2=0x7fe9a81f3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:25.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.585+0000 7fe9a7e00640 1 --2- 172.21.15.151:0/1970499486 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe990004d40 0x7fe9a81f3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:25.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.585+0000 7fe9a7e00640 1 -- 172.21.15.151:0/1970499486 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9a80d0f40 msgr2=0x7fe9a81f7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:25.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.585+0000 7fe9a7e00640 1 --2- 172.21.15.151:0/1970499486 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9a80d0f40 0x7fe9a81f7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:25.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.585+0000 7fe9a7e00640 1 -- 172.21.15.151:0/1970499486 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9a81590d0 con 0x7fe9a8159e70 2024-09-16T15:48:25.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.586+0000 7fe9a7e00640 1 --2- 172.21.15.151:0/1970499486 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9a8159e70 0x7fe9a80d0a00 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fe994011b10 tx=0x7fe994011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:25.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.586+0000 7fe9a4c00640 1 -- 172.21.15.151:0/1970499486 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99401e070 con 0x7fe9a8159e70 2024-09-16T15:48:25.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.586+0000 7fe9a4c00640 1 -- 172.21.15.151:0/1970499486 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe994004460 con 0x7fe9a8159e70 2024-09-16T15:48:25.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.586+0000 7fe9ade00640 1 -- 172.21.15.151:0/1970499486 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9a81f8a20 con 0x7fe9a8159e70 2024-09-16T15:48:25.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.586+0000 7fe9a4c00640 1 -- 172.21.15.151:0/1970499486 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99400fb80 con 0x7fe9a8159e70 2024-09-16T15:48:25.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.586+0000 7fe9ade00640 1 -- 172.21.15.151:0/1970499486 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9a8159670 con 0x7fe9a8159e70 2024-09-16T15:48:25.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.587+0000 7fe9ade00640 1 -- 172.21.15.151:0/1970499486 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9a80b4e00 con 0x7fe9a8159e70 2024-09-16T15:48:25.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.588+0000 7fe9a4c00640 1 -- 172.21.15.151:0/1970499486 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 30) ==== 102868+0+0 (secure 0 0 0) 0x7fe994021070 con 0x7fe9a8159e70 2024-09-16T15:48:25.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.589+0000 7fe9a4c00640 1 --2- 172.21.15.151:0/1970499486 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe9880799f0 0x7fe98807beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:25.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.589+0000 7fe9a4c00640 1 -- 172.21.15.151:0/1970499486 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7fe99409a0b0 con 0x7fe9a8159e70 2024-09-16T15:48:25.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.590+0000 7fe9aca00640 1 --2- 172.21.15.151:0/1970499486 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe9880799f0 0x7fe98807beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:25.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.590+0000 7fe9aca00640 1 --2- 172.21.15.151:0/1970499486 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe9880799f0 0x7fe98807beb0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fe99c009cd0 tx=0x7fe99c005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:25.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.593+0000 7fe9a4c00640 1 -- 172.21.15.151:0/1970499486 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe994063830 con 0x7fe9a8159e70 2024-09-16T15:48:25.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.778+0000 7fe9ade00640 1 -- 172.21.15.151:0/1970499486 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe9a815fac0 con 0x7fe9a8159e70 2024-09-16T15:48:25.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.779+0000 7fe9a4c00640 1 -- 172.21.15.151:0/1970499486 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+16879 (secure 0 0 0) 0x7fe994068260 con 0x7fe9a8159e70 2024-09-16T15:48:25.781 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:48:25.782 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":91,"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","created":"2024-09-16T15:38:56.348535+0000","modified":"2024-09-16T15:48:22.972235+0000","last_up_change":"2024-09-16T15:39:10.681384+0000","last_in_change":"2024-09-16T15:39:00.544477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:39:11.602252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:48:19.150224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-16T15:48:20.351519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5524076-a903-4d0a-af6a-caf474c4765d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6809","nonce":3111342741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6811","nonce":3111342741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6815","nonce":3111342741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6813","nonce":3111342741}]},"public_addr":"172.21.15.151:6809/3111342741","cluster_addr":"172.21.15.151:6811/3111342741","heartbeat_back_addr":"172.21.15.151:6815/3111342741","heartbeat_front_addr":"172.21.15.151:6813/3111342741","state":["exists","up"]},{"osd":1,"uuid":"dd25925f-0ae1-494e-8529-4dc205759db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6801","nonce":3368304087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6803","nonce":3368304087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6807","nonce":3368304087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6805","nonce":3368304087}]},"public_addr":"172.21.15.151:6801/3368304087","cluster_addr":"172.21.15.151:6803/3368304087","heartbeat_back_addr":"172.21.15.151:6807/3368304087","heartbeat_front_addr":"172.21.15.151:6805/3368304087","state":["exists","up"]},{"osd":2,"uuid":"c2f18115-013d-4da6-a0fe-cb978f4494b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6823","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6825","nonce":3079373271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6827","nonce":3079373271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6831","nonce":3079373271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6829","nonce":3079373271}]},"public_addr":"172.21.15.151:6825/3079373271","cluster_addr":"172.21.15.151:6827/3079373271","heartbeat_back_addr":"172.21.15.151:6831/3079373271","heartbeat_front_addr":"172.21.15.151:6829/3079373271","state":["exists","up"]},{"osd":3,"uuid":"36edb718-2fcb-4d88-8ddc-c321eed4279d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6817","nonce":1142509041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6819","nonce":1142509041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6824","nonce":1142509041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6821","nonce":1142509041}]},"public_addr":"172.21.15.151:6817/1142509041","cluster_addr":"172.21.15.151:6819/1142509041","heartbeat_back_addr":"172.21.15.151:6824/1142509041","heartbeat_front_addr":"172.21.15.151:6821/1142509041","state":["exists","up"]},{"osd":4,"uuid":"64d33708-cd0a-4236-abe3-1b646e82ab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6811","nonce":2735495271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6813","nonce":2735495271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6817","nonce":2735495271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6815","nonce":2735495271}]},"public_addr":"172.21.15.167:6811/2735495271","cluster_addr":"172.21.15.167:6813/2735495271","heartbeat_back_addr":"172.21.15.167:6817/2735495271","heartbeat_front_addr":"172.21.15.167:6815/2735495271","state":["exists","up"]},{"osd":5,"uuid":"f14b44a9-5910-458c-b518-09a8d0de94de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6827","nonce":1494916572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6829","nonce":1494916572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6833","nonce":1494916572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6831","nonce":1494916572}]},"public_addr":"172.21.15.167:6827/1494916572","cluster_addr":"172.21.15.167:6829/1494916572","heartbeat_back_addr":"172.21.15.167:6833/1494916572","heartbeat_front_addr":"172.21.15.167:6831/1494916572","state":["exists","up"]},{"osd":6,"uuid":"39335545-2654-4710-84b0-a9aa47cfcbba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6819","nonce":2538273273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6821","nonce":2538273273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6825","nonce":2538273273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6823","nonce":2538273273}]},"public_addr":"172.21.15.167:6819/2538273273","cluster_addr":"172.21.15.167:6821/2538273273","heartbeat_back_addr":"172.21.15.167:6825/2538273273","heartbeat_front_addr":"172.21.15.167:6823/2538273273","state":["exists","up"]},{"osd":7,"uuid":"4f8bfc81-53dd-4339-ac22-1a4007d9f963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6803","nonce":265072227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6805","nonce":265072227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6809","nonce":265072227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6807","nonce":265072227}]},"public_addr":"172.21.15.167:6803/265072227","cluster_addr":"172.21.15.167:6805/265072227","heartbeat_back_addr":"172.21.15.167:6809/265072227","heartbeat_front_addr":"172.21.15.167:6807/265072227","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.150359+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.084358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.722368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.332809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.320709+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.395153+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.935250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.801443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:48:25.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.781+0000 7fe9ade00640 1 -- 172.21.15.151:0/1970499486 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe9880799f0 msgr2=0x7fe98807beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:25.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.781+0000 7fe9ade00640 1 --2- 172.21.15.151:0/1970499486 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe9880799f0 0x7fe98807beb0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fe99c009cd0 tx=0x7fe99c005f90 comp rx=0 tx=0).stop 2024-09-16T15:48:25.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.781+0000 7fe9ade00640 1 -- 172.21.15.151:0/1970499486 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9a8159e70 msgr2=0x7fe9a80d0a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:25.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.781+0000 7fe9ade00640 1 --2- 172.21.15.151:0/1970499486 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9a8159e70 0x7fe9a80d0a00 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fe994011b10 tx=0x7fe994011f10 comp rx=0 tx=0).stop 2024-09-16T15:48:25.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.781+0000 7fe9ade00640 1 -- 172.21.15.151:0/1970499486 shutdown_connections 2024-09-16T15:48:25.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.781+0000 7fe9ade00640 1 --2- 172.21.15.151:0/1970499486 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe9880799f0 0x7fe98807beb0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:25.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.781+0000 7fe9ade00640 1 --2- 172.21.15.151:0/1970499486 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9a80d0f40 0x7fe9a81f7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:25.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.781+0000 7fe9ade00640 1 --2- 172.21.15.151:0/1970499486 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe9a8159e70 0x7fe9a80d0a00 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:25.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.781+0000 7fe9ade00640 1 --2- 172.21.15.151:0/1970499486 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe990004d40 0x7fe9a81f3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:25.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.781+0000 7fe9ade00640 1 -- 172.21.15.151:0/1970499486 >> 172.21.15.151:0/1970499486 conn(0x7fe9a80d3df0 msgr2=0x7fe9a81605e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:25.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.782+0000 7fe9ade00640 1 -- 172.21.15.151:0/1970499486 shutdown_connections 2024-09-16T15:48:25.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.782+0000 7fe9ade00640 1 -- 172.21.15.151:0/1970499486 wait complete. 2024-09-16T15:48:25.800 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T15:48:25.801 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:48:25.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.955+0000 7f0c46c00640 1 -- 172.21.15.151:0/3759280987 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c401445d0 msgr2=0x7f0c400cebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:25.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.955+0000 7f0c46c00640 1 --2- 172.21.15.151:0/3759280987 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c401445d0 0x7f0c400cebf0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f0c4015b380 tx=0x7f0c3002a5f0 comp rx=0 tx=0).stop 2024-09-16T15:48:25.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.955+0000 7f0c46c00640 1 -- 172.21.15.151:0/3759280987 shutdown_connections 2024-09-16T15:48:25.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.955+0000 7f0c46c00640 1 --2- 172.21.15.151:0/3759280987 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c400b4ca0 0x7f0c4015e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:25.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.955+0000 7f0c46c00640 1 --2- 172.21.15.151:0/3759280987 >> v2:172.21.15.167:3300/0 conn(0x7f0c400ccfd0 0x7f0c400cf130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:25.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.955+0000 7f0c46c00640 1 --2- 172.21.15.151:0/3759280987 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c401445d0 0x7f0c400cebf0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:25.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.955+0000 7f0c46c00640 1 -- 172.21.15.151:0/3759280987 >> 172.21.15.151:0/3759280987 conn(0x7f0c400b1950 msgr2=0x7f0c400b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:25.959 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.955+0000 7f0c46c00640 1 -- 172.21.15.151:0/3759280987 shutdown_connections 2024-09-16T15:48:25.959 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.955+0000 7f0c46c00640 1 -- 172.21.15.151:0/3759280987 wait complete. 2024-09-16T15:48:25.959 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.957+0000 7f0c46c00640 1 Processor -- start 2024-09-16T15:48:25.959 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.957+0000 7f0c46c00640 1 -- start start 2024-09-16T15:48:25.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.958+0000 7f0c46c00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c400b4ca0 0x7f0c40141000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:25.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.958+0000 7f0c46c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c400ccfd0 0x7f0c4013f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:25.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.958+0000 7f0c46c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c401445d0 0x7f0c400adc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:25.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.958+0000 7f0c46c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0c4015f240 con 0x7f0c400ccfd0 2024-09-16T15:48:25.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.958+0000 7f0c46c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0c4015f0c0 con 0x7f0c401445d0 2024-09-16T15:48:25.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.958+0000 7f0c46c00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f0c4015f3c0 con 0x7f0c400b4ca0 2024-09-16T15:48:25.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.958+0000 7f0c3f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c400ccfd0 0x7f0c4013f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:25.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.958+0000 7f0c3f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c400ccfd0 0x7f0c4013f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:42736/0 (socket says 172.21.15.151:42736) 2024-09-16T15:48:25.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.958+0000 7f0c3f400640 1 -- 172.21.15.151:0/770849871 learned_addr learned my addr 172.21.15.151:0/770849871 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:48:25.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.959+0000 7f0c3fe00640 1 --2- 172.21.15.151:0/770849871 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c400b4ca0 0x7f0c40141000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:25.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.959+0000 7f0c44c00640 1 --2- 172.21.15.151:0/770849871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c401445d0 0x7f0c400adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:25.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.959+0000 7f0c3f400640 1 -- 172.21.15.151:0/770849871 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c400b4ca0 msgr2=0x7f0c40141000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:25.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.959+0000 7f0c3f400640 1 --2- 172.21.15.151:0/770849871 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c400b4ca0 0x7f0c40141000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:25.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.959+0000 7f0c3f400640 1 -- 172.21.15.151:0/770849871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c401445d0 msgr2=0x7f0c400adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:25.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.959+0000 7f0c3f400640 1 --2- 172.21.15.151:0/770849871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c401445d0 0x7f0c400adc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:25.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.959+0000 7f0c3f400640 1 -- 172.21.15.151:0/770849871 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c400acfe0 con 0x7f0c400ccfd0 2024-09-16T15:48:25.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.959+0000 7f0c3f400640 1 --2- 172.21.15.151:0/770849871 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c400ccfd0 0x7f0c4013f8d0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c00d950 tx=0x7f0c2c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:25.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.959+0000 7f0c37e00640 1 -- 172.21.15.151:0/770849871 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c017070 con 0x7f0c400ccfd0 2024-09-16T15:48:25.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.959+0000 7f0c37e00640 1 -- 172.21.15.151:0/770849871 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0c2c009d20 con 0x7f0c400ccfd0 2024-09-16T15:48:25.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.959+0000 7f0c46c00640 1 -- 172.21.15.151:0/770849871 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c4013ff60 con 0x7f0c400ccfd0 2024-09-16T15:48:25.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.959+0000 7f0c37e00640 1 -- 172.21.15.151:0/770849871 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c0106c0 con 0x7f0c400ccfd0 2024-09-16T15:48:25.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.959+0000 7f0c46c00640 1 -- 172.21.15.151:0/770849871 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c40140800 con 0x7f0c400ccfd0 2024-09-16T15:48:25.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.960+0000 7f0c46c00640 1 -- 172.21.15.151:0/770849871 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c4015e970 con 0x7f0c400ccfd0 2024-09-16T15:48:25.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.961+0000 7f0c37e00640 1 -- 172.21.15.151:0/770849871 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 30) ==== 102868+0+0 (secure 0 0 0) 0x7f0c2c010860 con 0x7f0c400ccfd0 2024-09-16T15:48:25.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.962+0000 7f0c37e00640 1 --2- 172.21.15.151:0/770849871 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0c200799c0 0x7f0c2007be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:25.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.962+0000 7f0c37e00640 1 -- 172.21.15.151:0/770849871 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7f0c2c099c50 con 0x7f0c400ccfd0 2024-09-16T15:48:25.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.963+0000 7f0c3fe00640 1 --2- 172.21.15.151:0/770849871 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0c200799c0 0x7f0c2007be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:25.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.963+0000 7f0c3fe00640 1 --2- 172.21.15.151:0/770849871 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0c200799c0 0x7f0c2007be80 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f0c30033000 tx=0x7f0c30004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:25.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:25.966+0000 7f0c37e00640 1 -- 172.21.15.151:0/770849871 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0c2c0639c0 con 0x7f0c400ccfd0 2024-09-16T15:48:26.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.152+0000 7f0c46c00640 1 -- 172.21.15.151:0/770849871 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0c40070a20 con 0x7f0c200799c0 2024-09-16T15:48:26.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.157+0000 7f0c37e00640 1 -- 172.21.15.151:0/770849871 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247443 (secure 0 0 0) 0x7f0c40070a20 con 0x7f0c200799c0 2024-09-16T15:48:26.163 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:48:26.165 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-09-16T15:48:26.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.161+0000 7f0c46c00640 1 -- 172.21.15.151:0/770849871 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0c200799c0 msgr2=0x7f0c2007be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:26.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.161+0000 7f0c46c00640 1 --2- 172.21.15.151:0/770849871 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0c200799c0 0x7f0c2007be80 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f0c30033000 tx=0x7f0c30004730 comp rx=0 tx=0).stop 2024-09-16T15:48:26.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.162+0000 7f0c46c00640 1 -- 172.21.15.151:0/770849871 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c400ccfd0 msgr2=0x7f0c4013f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:26.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.162+0000 7f0c46c00640 1 --2- 172.21.15.151:0/770849871 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c400ccfd0 0x7f0c4013f8d0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c00d950 tx=0x7f0c2c00de10 comp rx=0 tx=0).stop 2024-09-16T15:48:26.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.162+0000 7f0c46c00640 1 -- 172.21.15.151:0/770849871 shutdown_connections 2024-09-16T15:48:26.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.162+0000 7f0c46c00640 1 --2- 172.21.15.151:0/770849871 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0c200799c0 0x7f0c2007be80 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:26.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.162+0000 7f0c46c00640 1 --2- 172.21.15.151:0/770849871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c401445d0 0x7f0c400adc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:26.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.162+0000 7f0c46c00640 1 --2- 172.21.15.151:0/770849871 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c400ccfd0 0x7f0c4013f8d0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:26.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.162+0000 7f0c46c00640 1 --2- 172.21.15.151:0/770849871 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c400b4ca0 0x7f0c40141000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:26.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.162+0000 7f0c46c00640 1 -- 172.21.15.151:0/770849871 >> 172.21.15.151:0/770849871 conn(0x7f0c400b1950 msgr2=0x7f0c400af880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:26.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.162+0000 7f0c46c00640 1 -- 172.21.15.151:0/770849871 shutdown_connections 2024-09-16T15:48:26.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.162+0000 7f0c46c00640 1 -- 172.21.15.151:0/770849871 wait complete. 2024-09-16T15:48:26.181 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":362,"stamp":"2024-09-16T15:48:25.749544+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":108,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78,"ondisk_log_size":78,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":489404,"kb_used_data":9596,"kb_used_omap":12,"kb_used_meta":479603,"kb_avail":754485316,"statfs":{"total":773094113280,"available":772592963584,"internally_reserved":0,"allocated":9826304,"data_stored":4902599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12738,"internal_metadata":491114046},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.801460"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.964779+0000","last_change":"2024-09-16T15:48:20.964779+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.964779+0000","last_undegraded":"2024-09-16T15:48:20.964779+0000","last_fullsized":"2024-09-16T15:48:20.964779+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.956223+0000","last_change":"2024-09-16T15:48:20.956223+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.956223+0000","last_undegraded":"2024-09-16T15:48:20.956223+0000","last_fullsized":"2024-09-16T15:48:20.956223+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.966554+0000","last_change":"2024-09-16T15:48:20.966554+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.966554+0000","last_undegraded":"2024-09-16T15:48:20.966554+0000","last_fullsized":"2024-09-16T15:48:20.966554+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.958260+0000","last_change":"2024-09-16T15:48:20.958260+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.958260+0000","last_undegraded":"2024-09-16T15:48:20.958260+0000","last_fullsized":"2024-09-16T15:48:20.958260+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.961437+0000","last_change":"2024-09-16T15:48:20.961437+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.961437+0000","last_undegraded":"2024-09-16T15:48:20.961437+0000","last_fullsized":"2024-09-16T15:48:20.961437+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.964352+0000","last_change":"2024-09-16T15:48:20.964352+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.964352+0000","last_undegraded":"2024-09-16T15:48:20.964352+0000","last_fullsized":"2024-09-16T15:48:20.964352+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.956641+0000","last_change":"2024-09-16T15:48:20.956641+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.956641+0000","last_undegraded":"2024-09-16T15:48:20.956641+0000","last_fullsized":"2024-09-16T15:48:20.956641+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.961810+0000","last_change":"2024-09-16T15:48:20.961810+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.961810+0000","last_undegraded":"2024-09-16T15:48:20.961810+0000","last_fullsized":"2024-09-16T15:48:20.961810+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.964381+0000","last_change":"2024-09-16T15:48:20.964381+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.964381+0000","last_undegraded":"2024-09-16T15:48:20.964381+0000","last_fullsized":"2024-09-16T15:48:20.964381+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.964822+0000","last_change":"2024-09-16T15:48:20.964822+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.964822+0000","last_undegraded":"2024-09-16T15:48:20.964822+0000","last_fullsized":"2024-09-16T15:48:20.964822+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.962582+0000","last_change":"2024-09-16T15:48:20.962582+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.962582+0000","last_undegraded":"2024-09-16T15:48:20.962582+0000","last_fullsized":"2024-09-16T15:48:20.962582+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.963218+0000","last_change":"2024-09-16T15:48:20.963218+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.963218+0000","last_undegraded":"2024-09-16T15:48:20.963218+0000","last_fullsized":"2024-09-16T15:48:20.963218+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.959497+0000","last_change":"2024-09-16T15:48:20.959497+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.959497+0000","last_undegraded":"2024-09-16T15:48:20.959497+0000","last_fullsized":"2024-09-16T15:48:20.959497+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.351800+0000","last_change":"2024-09-16T15:48:21.991329+0000","last_active":"2024-09-16T15:48:23.351800+0000","last_peered":"2024-09-16T15:48:23.351800+0000","last_clean":"2024-09-16T15:48:23.351800+0000","last_became_active":"2024-09-16T15:48:21.990123+0000","last_became_peered":"2024-09-16T15:48:21.990123+0000","last_unstale":"2024-09-16T15:48:23.351800+0000","last_undegraded":"2024-09-16T15:48:23.351800+0000","last_fullsized":"2024-09-16T15:48:23.351800+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:01:55.713464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.956053+0000","last_change":"2024-09-16T15:48:20.956053+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.956053+0000","last_undegraded":"2024-09-16T15:48:20.956053+0000","last_fullsized":"2024-09-16T15:48:20.956053+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.964568+0000","last_change":"2024-09-16T15:48:20.964568+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.964568+0000","last_undegraded":"2024-09-16T15:48:20.964568+0000","last_fullsized":"2024-09-16T15:48:20.964568+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.960600+0000","last_change":"2024-09-16T15:48:20.960600+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.960600+0000","last_undegraded":"2024-09-16T15:48:20.960600+0000","last_fullsized":"2024-09-16T15:48:20.960600+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.965863+0000","last_change":"2024-09-16T15:48:20.965863+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.965863+0000","last_undegraded":"2024-09-16T15:48:20.965863+0000","last_fullsized":"2024-09-16T15:48:20.965863+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352178+0000","last_change":"2024-09-16T15:48:21.996135+0000","last_active":"2024-09-16T15:48:23.352178+0000","last_peered":"2024-09-16T15:48:23.352178+0000","last_clean":"2024-09-16T15:48:23.352178+0000","last_became_active":"2024-09-16T15:48:21.995671+0000","last_became_peered":"2024-09-16T15:48:21.995671+0000","last_unstale":"2024-09-16T15:48:23.352178+0000","last_undegraded":"2024-09-16T15:48:23.352178+0000","last_fullsized":"2024-09-16T15:48:23.352178+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:13:55.927762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352319+0000","last_change":"2024-09-16T15:48:21.994572+0000","last_active":"2024-09-16T15:48:23.352319+0000","last_peered":"2024-09-16T15:48:23.352319+0000","last_clean":"2024-09-16T15:48:23.352319+0000","last_became_active":"2024-09-16T15:48:21.994305+0000","last_became_peered":"2024-09-16T15:48:21.994305+0000","last_unstale":"2024-09-16T15:48:23.352319+0000","last_undegraded":"2024-09-16T15:48:23.352319+0000","last_fullsized":"2024-09-16T15:48:23.352319+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:27:24.090121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.958863+0000","last_change":"2024-09-16T15:48:20.958863+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.958863+0000","last_undegraded":"2024-09-16T15:48:20.958863+0000","last_fullsized":"2024-09-16T15:48:20.958863+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.351839+0000","last_change":"2024-09-16T15:48:21.988902+0000","last_active":"2024-09-16T15:48:23.351839+0000","last_peered":"2024-09-16T15:48:23.351839+0000","last_clean":"2024-09-16T15:48:23.351839+0000","last_became_active":"2024-09-16T15:48:21.988613+0000","last_became_peered":"2024-09-16T15:48:21.988613+0000","last_unstale":"2024-09-16T15:48:23.351839+0000","last_undegraded":"2024-09-16T15:48:23.351839+0000","last_fullsized":"2024-09-16T15:48:23.351839+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:05:00.466068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.962242+0000","last_change":"2024-09-16T15:48:20.962242+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.962242+0000","last_undegraded":"2024-09-16T15:48:20.962242+0000","last_fullsized":"2024-09-16T15:48:20.962242+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.962313+0000","last_change":"2024-09-16T15:48:20.962313+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.962313+0000","last_undegraded":"2024-09-16T15:48:20.962313+0000","last_fullsized":"2024-09-16T15:48:20.962313+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.966477+0000","last_change":"2024-09-16T15:48:20.966477+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.966477+0000","last_undegraded":"2024-09-16T15:48:20.966477+0000","last_fullsized":"2024-09-16T15:48:20.966477+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.957649+0000","last_change":"2024-09-16T15:48:20.957649+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.957649+0000","last_undegraded":"2024-09-16T15:48:20.957649+0000","last_fullsized":"2024-09-16T15:48:20.957649+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.958211+0000","last_change":"2024-09-16T15:48:20.958211+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.958211+0000","last_undegraded":"2024-09-16T15:48:20.958211+0000","last_fullsized":"2024-09-16T15:48:20.958211+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.963150+0000","last_change":"2024-09-16T15:48:20.963150+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.963150+0000","last_undegraded":"2024-09-16T15:48:20.963150+0000","last_fullsized":"2024-09-16T15:48:20.963150+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.957984+0000","last_change":"2024-09-16T15:48:20.957984+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.957984+0000","last_undegraded":"2024-09-16T15:48:20.957984+0000","last_fullsized":"2024-09-16T15:48:20.957984+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352274+0000","last_change":"2024-09-16T15:48:21.991963+0000","last_active":"2024-09-16T15:48:23.352274+0000","last_peered":"2024-09-16T15:48:23.352274+0000","last_clean":"2024-09-16T15:48:23.352274+0000","last_became_active":"2024-09-16T15:48:21.991698+0000","last_became_peered":"2024-09-16T15:48:21.991698+0000","last_unstale":"2024-09-16T15:48:23.352274+0000","last_undegraded":"2024-09-16T15:48:23.352274+0000","last_fullsized":"2024-09-16T15:48:23.352274+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:21:15.053411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.961609+0000","last_change":"2024-09-16T15:48:20.961609+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.961609+0000","last_undegraded":"2024-09-16T15:48:20.961609+0000","last_fullsized":"2024-09-16T15:48:20.961609+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.962070+0000","last_change":"2024-09-16T15:48:20.962070+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.962070+0000","last_undegraded":"2024-09-16T15:48:20.962070+0000","last_fullsized":"2024-09-16T15:48:20.962070+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.961053+0000","last_change":"2024-09-16T15:48:20.961053+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.961053+0000","last_undegraded":"2024-09-16T15:48:20.961053+0000","last_fullsized":"2024-09-16T15:48:20.961053+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.977633+0000","last_change":"2024-09-16T15:48:20.975503+0000","last_active":"2024-09-16T15:48:20.977633+0000","last_peered":"2024-09-16T15:48:20.977633+0000","last_clean":"2024-09-16T15:48:20.977633+0000","last_became_active":"2024-09-16T15:48:20.973819+0000","last_became_peered":"2024-09-16T15:48:20.973819+0000","last_unstale":"2024-09-16T15:48:20.977633+0000","last_undegraded":"2024-09-16T15:48:20.977633+0000","last_fullsized":"2024-09-16T15:48:20.977633+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:39:26.482165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.963995+0000","last_change":"2024-09-16T15:48:20.963995+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.963995+0000","last_undegraded":"2024-09-16T15:48:20.963995+0000","last_fullsized":"2024-09-16T15:48:20.963995+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.976680+0000","last_change":"2024-09-16T15:48:20.974951+0000","last_active":"2024-09-16T15:48:20.976680+0000","last_peered":"2024-09-16T15:48:20.976680+0000","last_clean":"2024-09-16T15:48:20.976680+0000","last_became_active":"2024-09-16T15:48:20.974688+0000","last_became_peered":"2024-09-16T15:48:20.974688+0000","last_unstale":"2024-09-16T15:48:20.976680+0000","last_undegraded":"2024-09-16T15:48:20.976680+0000","last_fullsized":"2024-09-16T15:48:20.976680+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:40:29.438048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.960641+0000","last_change":"2024-09-16T15:48:20.960641+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.960641+0000","last_undegraded":"2024-09-16T15:48:20.960641+0000","last_fullsized":"2024-09-16T15:48:20.960641+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.981008+0000","last_change":"2024-09-16T15:48:20.978653+0000","last_active":"2024-09-16T15:48:20.981008+0000","last_peered":"2024-09-16T15:48:20.981008+0000","last_clean":"2024-09-16T15:48:20.981008+0000","last_became_active":"2024-09-16T15:48:20.976364+0000","last_became_peered":"2024-09-16T15:48:20.976364+0000","last_unstale":"2024-09-16T15:48:20.981008+0000","last_undegraded":"2024-09-16T15:48:20.981008+0000","last_fullsized":"2024-09-16T15:48:20.981008+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:01:37.109321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.961183+0000","last_change":"2024-09-16T15:48:20.961183+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.961183+0000","last_undegraded":"2024-09-16T15:48:20.961183+0000","last_fullsized":"2024-09-16T15:48:20.961183+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.969107+0000","last_change":"2024-09-16T15:48:20.967236+0000","last_active":"2024-09-16T15:48:20.969107+0000","last_peered":"2024-09-16T15:48:20.969107+0000","last_clean":"2024-09-16T15:48:20.969107+0000","last_became_active":"2024-09-16T15:48:20.966954+0000","last_became_peered":"2024-09-16T15:48:20.966954+0000","last_unstale":"2024-09-16T15:48:20.969107+0000","last_undegraded":"2024-09-16T15:48:20.969107+0000","last_fullsized":"2024-09-16T15:48:20.969107+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:02:43.930006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.959326+0000","last_change":"2024-09-16T15:48:20.959326+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.959326+0000","last_undegraded":"2024-09-16T15:48:20.959326+0000","last_fullsized":"2024-09-16T15:48:20.959326+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.975825+0000","last_change":"2024-09-16T15:48:20.975041+0000","last_active":"2024-09-16T15:48:20.975825+0000","last_peered":"2024-09-16T15:48:20.975825+0000","last_clean":"2024-09-16T15:48:20.975825+0000","last_became_active":"2024-09-16T15:48:20.971719+0000","last_became_peered":"2024-09-16T15:48:20.971719+0000","last_unstale":"2024-09-16T15:48:20.975825+0000","last_undegraded":"2024-09-16T15:48:20.975825+0000","last_fullsized":"2024-09-16T15:48:20.975825+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:32:44.623249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.962285+0000","last_change":"2024-09-16T15:48:20.962285+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.962285+0000","last_undegraded":"2024-09-16T15:48:20.962285+0000","last_fullsized":"2024-09-16T15:48:20.962285+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.976080+0000","last_change":"2024-09-16T15:48:20.971997+0000","last_active":"2024-09-16T15:48:20.976080+0000","last_peered":"2024-09-16T15:48:20.976080+0000","last_clean":"2024-09-16T15:48:20.976080+0000","last_became_active":"2024-09-16T15:48:20.971230+0000","last_became_peered":"2024-09-16T15:48:20.971230+0000","last_unstale":"2024-09-16T15:48:20.976080+0000","last_undegraded":"2024-09-16T15:48:20.976080+0000","last_fullsized":"2024-09-16T15:48:20.976080+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:20:05.425588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.962335+0000","last_change":"2024-09-16T15:48:20.962335+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.962335+0000","last_undegraded":"2024-09-16T15:48:20.962335+0000","last_fullsized":"2024-09-16T15:48:20.962335+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.351663+0000","last_change":"2024-09-16T15:48:20.977058+0000","last_active":"2024-09-16T15:48:23.351663+0000","last_peered":"2024-09-16T15:48:23.351663+0000","last_clean":"2024-09-16T15:48:23.351663+0000","last_became_active":"2024-09-16T15:48:20.975591+0000","last_became_peered":"2024-09-16T15:48:20.975591+0000","last_unstale":"2024-09-16T15:48:23.351663+0000","last_undegraded":"2024-09-16T15:48:23.351663+0000","last_fullsized":"2024-09-16T15:48:23.351663+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:58:51.576740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.351992+0000","last_change":"2024-09-16T15:48:21.987074+0000","last_active":"2024-09-16T15:48:23.351992+0000","last_peered":"2024-09-16T15:48:23.351992+0000","last_clean":"2024-09-16T15:48:23.351992+0000","last_became_active":"2024-09-16T15:48:21.986779+0000","last_became_peered":"2024-09-16T15:48:21.986779+0000","last_unstale":"2024-09-16T15:48:23.351992+0000","last_undegraded":"2024-09-16T15:48:23.351992+0000","last_fullsized":"2024-09-16T15:48:23.351992+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:57:02.685046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.351536+0000","last_change":"2024-09-16T15:48:20.977637+0000","last_active":"2024-09-16T15:48:23.351536+0000","last_peered":"2024-09-16T15:48:23.351536+0000","last_clean":"2024-09-16T15:48:23.351536+0000","last_became_active":"2024-09-16T15:48:20.977367+0000","last_became_peered":"2024-09-16T15:48:20.977367+0000","last_unstale":"2024-09-16T15:48:23.351536+0000","last_undegraded":"2024-09-16T15:48:23.351536+0000","last_fullsized":"2024-09-16T15:48:23.351536+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:57:12.992776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352256+0000","last_change":"2024-09-16T15:48:21.990047+0000","last_active":"2024-09-16T15:48:23.352256+0000","last_peered":"2024-09-16T15:48:23.352256+0000","last_clean":"2024-09-16T15:48:23.352256+0000","last_became_active":"2024-09-16T15:48:21.989735+0000","last_became_peered":"2024-09-16T15:48:21.989735+0000","last_unstale":"2024-09-16T15:48:23.352256+0000","last_undegraded":"2024-09-16T15:48:23.352256+0000","last_fullsized":"2024-09-16T15:48:23.352256+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:50:37.046745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.971713+0000","last_change":"2024-09-16T15:48:20.969036+0000","last_active":"2024-09-16T15:48:20.971713+0000","last_peered":"2024-09-16T15:48:20.971713+0000","last_clean":"2024-09-16T15:48:20.971713+0000","last_became_active":"2024-09-16T15:48:20.968771+0000","last_became_peered":"2024-09-16T15:48:20.968771+0000","last_unstale":"2024-09-16T15:48:20.971713+0000","last_undegraded":"2024-09-16T15:48:20.971713+0000","last_fullsized":"2024-09-16T15:48:20.971713+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:27:21.822056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.962027+0000","last_change":"2024-09-16T15:48:20.962027+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.962027+0000","last_undegraded":"2024-09-16T15:48:20.962027+0000","last_fullsized":"2024-09-16T15:48:20.962027+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.969598+0000","last_change":"2024-09-16T15:48:20.968246+0000","last_active":"2024-09-16T15:48:20.969598+0000","last_peered":"2024-09-16T15:48:20.969598+0000","last_clean":"2024-09-16T15:48:20.969598+0000","last_became_active":"2024-09-16T15:48:20.967963+0000","last_became_peered":"2024-09-16T15:48:20.967963+0000","last_unstale":"2024-09-16T15:48:20.969598+0000","last_undegraded":"2024-09-16T15:48:20.969598+0000","last_fullsized":"2024-09-16T15:48:20.969598+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:23:26.528353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.967777+0000","last_change":"2024-09-16T15:48:20.967777+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.967777+0000","last_undegraded":"2024-09-16T15:48:20.967777+0000","last_fullsized":"2024-09-16T15:48:20.967777+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.977951+0000","last_change":"2024-09-16T15:48:20.975763+0000","last_active":"2024-09-16T15:48:20.977951+0000","last_peered":"2024-09-16T15:48:20.977951+0000","last_clean":"2024-09-16T15:48:20.977951+0000","last_became_active":"2024-09-16T15:48:20.974952+0000","last_became_peered":"2024-09-16T15:48:20.974952+0000","last_unstale":"2024-09-16T15:48:20.977951+0000","last_undegraded":"2024-09-16T15:48:20.977951+0000","last_fullsized":"2024-09-16T15:48:20.977951+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:37:00.965428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.958573+0000","last_change":"2024-09-16T15:48:20.958573+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.958573+0000","last_undegraded":"2024-09-16T15:48:20.958573+0000","last_fullsized":"2024-09-16T15:48:20.958573+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.974235+0000","last_change":"2024-09-16T15:48:20.972430+0000","last_active":"2024-09-16T15:48:20.974235+0000","last_peered":"2024-09-16T15:48:20.974235+0000","last_clean":"2024-09-16T15:48:20.974235+0000","last_became_active":"2024-09-16T15:48:20.972159+0000","last_became_peered":"2024-09-16T15:48:20.972159+0000","last_unstale":"2024-09-16T15:48:20.974235+0000","last_undegraded":"2024-09-16T15:48:20.974235+0000","last_fullsized":"2024-09-16T15:48:20.974235+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:02:58.958514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.965797+0000","last_change":"2024-09-16T15:48:20.965797+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.965797+0000","last_undegraded":"2024-09-16T15:48:20.965797+0000","last_fullsized":"2024-09-16T15:48:20.965797+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.969032+0000","last_change":"2024-09-16T15:48:20.965556+0000","last_active":"2024-09-16T15:48:20.969032+0000","last_peered":"2024-09-16T15:48:20.969032+0000","last_clean":"2024-09-16T15:48:20.969032+0000","last_became_active":"2024-09-16T15:48:20.965196+0000","last_became_peered":"2024-09-16T15:48:20.965196+0000","last_unstale":"2024-09-16T15:48:20.969032+0000","last_undegraded":"2024-09-16T15:48:20.969032+0000","last_fullsized":"2024-09-16T15:48:20.969032+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:57:22.438388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.959493+0000","last_change":"2024-09-16T15:48:20.959493+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.959493+0000","last_undegraded":"2024-09-16T15:48:20.959493+0000","last_fullsized":"2024-09-16T15:48:20.959493+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.970792+0000","last_change":"2024-09-16T15:48:20.969210+0000","last_active":"2024-09-16T15:48:20.970792+0000","last_peered":"2024-09-16T15:48:20.970792+0000","last_clean":"2024-09-16T15:48:20.970792+0000","last_became_active":"2024-09-16T15:48:20.968946+0000","last_became_peered":"2024-09-16T15:48:20.968946+0000","last_unstale":"2024-09-16T15:48:20.970792+0000","last_undegraded":"2024-09-16T15:48:20.970792+0000","last_fullsized":"2024-09-16T15:48:20.970792+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:43:02.173876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352100+0000","last_change":"2024-09-16T15:48:21.997110+0000","last_active":"2024-09-16T15:48:23.352100+0000","last_peered":"2024-09-16T15:48:23.352100+0000","last_clean":"2024-09-16T15:48:23.352100+0000","last_became_active":"2024-09-16T15:48:21.996855+0000","last_became_peered":"2024-09-16T15:48:21.996855+0000","last_unstale":"2024-09-16T15:48:23.352100+0000","last_undegraded":"2024-09-16T15:48:23.352100+0000","last_fullsized":"2024-09-16T15:48:23.352100+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:44:31.136306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.970157+0000","last_change":"2024-09-16T15:48:20.969040+0000","last_active":"2024-09-16T15:48:20.970157+0000","last_peered":"2024-09-16T15:48:20.970157+0000","last_clean":"2024-09-16T15:48:20.970157+0000","last_became_active":"2024-09-16T15:48:20.968541+0000","last_became_peered":"2024-09-16T15:48:20.968541+0000","last_unstale":"2024-09-16T15:48:20.970157+0000","last_undegraded":"2024-09-16T15:48:20.970157+0000","last_fullsized":"2024-09-16T15:48:20.970157+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:02:23.941065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.964413+0000","last_change":"2024-09-16T15:48:20.964413+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.964413+0000","last_undegraded":"2024-09-16T15:48:20.964413+0000","last_fullsized":"2024-09-16T15:48:20.964413+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.979658+0000","last_change":"2024-09-16T15:48:20.975861+0000","last_active":"2024-09-16T15:48:20.979658+0000","last_peered":"2024-09-16T15:48:20.979658+0000","last_clean":"2024-09-16T15:48:20.979658+0000","last_became_active":"2024-09-16T15:48:20.974746+0000","last_became_peered":"2024-09-16T15:48:20.974746+0000","last_unstale":"2024-09-16T15:48:20.979658+0000","last_undegraded":"2024-09-16T15:48:20.979658+0000","last_fullsized":"2024-09-16T15:48:20.979658+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:13.847700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.960635+0000","last_change":"2024-09-16T15:48:20.960635+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.960635+0000","last_undegraded":"2024-09-16T15:48:20.960635+0000","last_fullsized":"2024-09-16T15:48:20.960635+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.977666+0000","last_change":"2024-09-16T15:48:20.975777+0000","last_active":"2024-09-16T15:48:20.977666+0000","last_peered":"2024-09-16T15:48:20.977666+0000","last_clean":"2024-09-16T15:48:20.977666+0000","last_became_active":"2024-09-16T15:48:20.974162+0000","last_became_peered":"2024-09-16T15:48:20.974162+0000","last_unstale":"2024-09-16T15:48:20.977666+0000","last_undegraded":"2024-09-16T15:48:20.977666+0000","last_fullsized":"2024-09-16T15:48:20.977666+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:34:52.666788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.961693+0000","last_change":"2024-09-16T15:48:20.961693+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.961693+0000","last_undegraded":"2024-09-16T15:48:20.961693+0000","last_fullsized":"2024-09-16T15:48:20.961693+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"90'2","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352068+0000","last_change":"2024-09-16T15:48:20.975158+0000","last_active":"2024-09-16T15:48:23.352068+0000","last_peered":"2024-09-16T15:48:23.352068+0000","last_clean":"2024-09-16T15:48:23.352068+0000","last_became_active":"2024-09-16T15:48:20.974854+0000","last_became_peered":"2024-09-16T15:48:20.974854+0000","last_unstale":"2024-09-16T15:48:23.352068+0000","last_undegraded":"2024-09-16T15:48:23.352068+0000","last_fullsized":"2024-09-16T15:48:23.352068+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:41:44.873886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.958552+0000","last_change":"2024-09-16T15:48:20.958552+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.958552+0000","last_undegraded":"2024-09-16T15:48:20.958552+0000","last_fullsized":"2024-09-16T15:48:20.958552+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352234+0000","last_change":"2024-09-16T15:48:20.978031+0000","last_active":"2024-09-16T15:48:23.352234+0000","last_peered":"2024-09-16T15:48:23.352234+0000","last_clean":"2024-09-16T15:48:23.352234+0000","last_became_active":"2024-09-16T15:48:20.976491+0000","last_became_peered":"2024-09-16T15:48:20.976491+0000","last_unstale":"2024-09-16T15:48:23.352234+0000","last_undegraded":"2024-09-16T15:48:23.352234+0000","last_fullsized":"2024-09-16T15:48:23.352234+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:57.979104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.956433+0000","last_change":"2024-09-16T15:48:20.956433+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.956433+0000","last_undegraded":"2024-09-16T15:48:20.956433+0000","last_fullsized":"2024-09-16T15:48:20.956433+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.b","version":"90'2","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.351800+0000","last_change":"2024-09-16T15:48:20.976944+0000","last_active":"2024-09-16T15:48:23.351800+0000","last_peered":"2024-09-16T15:48:23.351800+0000","last_clean":"2024-09-16T15:48:23.351800+0000","last_became_active":"2024-09-16T15:48:20.976649+0000","last_became_peered":"2024-09-16T15:48:20.976649+0000","last_unstale":"2024-09-16T15:48:23.351800+0000","last_undegraded":"2024-09-16T15:48:23.351800+0000","last_fullsized":"2024-09-16T15:48:23.351800+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:33:01.893406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.957113+0000","last_change":"2024-09-16T15:48:20.957113+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.957113+0000","last_undegraded":"2024-09-16T15:48:20.957113+0000","last_fullsized":"2024-09-16T15:48:20.957113+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.973153+0000","last_change":"2024-09-16T15:48:20.970601+0000","last_active":"2024-09-16T15:48:20.973153+0000","last_peered":"2024-09-16T15:48:20.973153+0000","last_clean":"2024-09-16T15:48:20.973153+0000","last_became_active":"2024-09-16T15:48:20.970272+0000","last_became_peered":"2024-09-16T15:48:20.970272+0000","last_unstale":"2024-09-16T15:48:20.973153+0000","last_undegraded":"2024-09-16T15:48:20.973153+0000","last_fullsized":"2024-09-16T15:48:20.973153+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:31:13.471665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.957294+0000","last_change":"2024-09-16T15:48:20.957294+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.957294+0000","last_undegraded":"2024-09-16T15:48:20.957294+0000","last_fullsized":"2024-09-16T15:48:20.957294+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.973815+0000","last_change":"2024-09-16T15:48:20.971710+0000","last_active":"2024-09-16T15:48:20.973815+0000","last_peered":"2024-09-16T15:48:20.973815+0000","last_clean":"2024-09-16T15:48:20.973815+0000","last_became_active":"2024-09-16T15:48:20.971360+0000","last_became_peered":"2024-09-16T15:48:20.971360+0000","last_unstale":"2024-09-16T15:48:20.973815+0000","last_undegraded":"2024-09-16T15:48:20.973815+0000","last_fullsized":"2024-09-16T15:48:20.973815+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:53:28.706338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.959074+0000","last_change":"2024-09-16T15:48:20.959074+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.959074+0000","last_undegraded":"2024-09-16T15:48:20.959074+0000","last_fullsized":"2024-09-16T15:48:20.959074+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.351750+0000","last_change":"2024-09-16T15:48:20.980185+0000","last_active":"2024-09-16T15:48:23.351750+0000","last_peered":"2024-09-16T15:48:23.351750+0000","last_clean":"2024-09-16T15:48:23.351750+0000","last_became_active":"2024-09-16T15:48:20.979875+0000","last_became_peered":"2024-09-16T15:48:20.979875+0000","last_unstale":"2024-09-16T15:48:23.351750+0000","last_undegraded":"2024-09-16T15:48:23.351750+0000","last_fullsized":"2024-09-16T15:48:23.351750+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:56:29.587401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.957180+0000","last_change":"2024-09-16T15:48:20.957180+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.957180+0000","last_undegraded":"2024-09-16T15:48:20.957180+0000","last_fullsized":"2024-09-16T15:48:20.957180+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"90'2","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352056+0000","last_change":"2024-09-16T15:48:20.978563+0000","last_active":"2024-09-16T15:48:23.352056+0000","last_peered":"2024-09-16T15:48:23.352056+0000","last_clean":"2024-09-16T15:48:23.352056+0000","last_became_active":"2024-09-16T15:48:20.978188+0000","last_became_peered":"2024-09-16T15:48:20.978188+0000","last_unstale":"2024-09-16T15:48:23.352056+0000","last_undegraded":"2024-09-16T15:48:23.352056+0000","last_fullsized":"2024-09-16T15:48:23.352056+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:28:27.584567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.957910+0000","last_change":"2024-09-16T15:48:20.957910+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.957910+0000","last_undegraded":"2024-09-16T15:48:20.957910+0000","last_fullsized":"2024-09-16T15:48:20.957910+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.979361+0000","last_change":"2024-09-16T15:48:20.975435+0000","last_active":"2024-09-16T15:48:20.979361+0000","last_peered":"2024-09-16T15:48:20.979361+0000","last_clean":"2024-09-16T15:48:20.979361+0000","last_became_active":"2024-09-16T15:48:20.975091+0000","last_became_peered":"2024-09-16T15:48:20.975091+0000","last_unstale":"2024-09-16T15:48:20.979361+0000","last_undegraded":"2024-09-16T15:48:20.979361+0000","last_fullsized":"2024-09-16T15:48:20.979361+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:43:06.025743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.957792+0000","last_change":"2024-09-16T15:48:20.957792+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.957792+0000","last_undegraded":"2024-09-16T15:48:20.957792+0000","last_fullsized":"2024-09-16T15:48:20.957792+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.975126+0000","last_change":"2024-09-16T15:48:20.974118+0000","last_active":"2024-09-16T15:48:20.975126+0000","last_peered":"2024-09-16T15:48:20.975126+0000","last_clean":"2024-09-16T15:48:20.975126+0000","last_became_active":"2024-09-16T15:48:20.973841+0000","last_became_peered":"2024-09-16T15:48:20.973841+0000","last_unstale":"2024-09-16T15:48:20.975126+0000","last_undegraded":"2024-09-16T15:48:20.975126+0000","last_fullsized":"2024-09-16T15:48:20.975126+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:09:11.275872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.960754+0000","last_change":"2024-09-16T15:48:20.960754+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.960754+0000","last_undegraded":"2024-09-16T15:48:20.960754+0000","last_fullsized":"2024-09-16T15:48:20.960754+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.976606+0000","last_change":"2024-09-16T15:48:20.974113+0000","last_active":"2024-09-16T15:48:20.976606+0000","last_peered":"2024-09-16T15:48:20.976606+0000","last_clean":"2024-09-16T15:48:20.976606+0000","last_became_active":"2024-09-16T15:48:20.973851+0000","last_became_peered":"2024-09-16T15:48:20.973851+0000","last_unstale":"2024-09-16T15:48:20.976606+0000","last_undegraded":"2024-09-16T15:48:20.976606+0000","last_fullsized":"2024-09-16T15:48:20.976606+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:14:35.885189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.956848+0000","last_change":"2024-09-16T15:48:20.956848+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.956848+0000","last_undegraded":"2024-09-16T15:48:20.956848+0000","last_fullsized":"2024-09-16T15:48:20.956848+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.972184+0000","last_change":"2024-09-16T15:48:20.968528+0000","last_active":"2024-09-16T15:48:20.972184+0000","last_peered":"2024-09-16T15:48:20.972184+0000","last_clean":"2024-09-16T15:48:20.972184+0000","last_became_active":"2024-09-16T15:48:20.967944+0000","last_became_peered":"2024-09-16T15:48:20.967944+0000","last_unstale":"2024-09-16T15:48:20.972184+0000","last_undegraded":"2024-09-16T15:48:20.972184+0000","last_fullsized":"2024-09-16T15:48:20.972184+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:24:23.675488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.962709+0000","last_change":"2024-09-16T15:48:20.962709+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.962709+0000","last_undegraded":"2024-09-16T15:48:20.962709+0000","last_fullsized":"2024-09-16T15:48:20.962709+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.966248+0000","last_change":"2024-09-16T15:48:20.963312+0000","last_active":"2024-09-16T15:48:20.966248+0000","last_peered":"2024-09-16T15:48:20.966248+0000","last_clean":"2024-09-16T15:48:20.966248+0000","last_became_active":"2024-09-16T15:48:20.963034+0000","last_became_peered":"2024-09-16T15:48:20.963034+0000","last_unstale":"2024-09-16T15:48:20.966248+0000","last_undegraded":"2024-09-16T15:48:20.966248+0000","last_fullsized":"2024-09-16T15:48:20.966248+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:39:53.503131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.956619+0000","last_change":"2024-09-16T15:48:20.956619+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.956619+0000","last_undegraded":"2024-09-16T15:48:20.956619+0000","last_fullsized":"2024-09-16T15:48:20.956619+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.978757+0000","last_change":"2024-09-16T15:48:20.978028+0000","last_active":"2024-09-16T15:48:20.978757+0000","last_peered":"2024-09-16T15:48:20.978757+0000","last_clean":"2024-09-16T15:48:20.978757+0000","last_became_active":"2024-09-16T15:48:20.977556+0000","last_became_peered":"2024-09-16T15:48:20.977556+0000","last_unstale":"2024-09-16T15:48:20.978757+0000","last_undegraded":"2024-09-16T15:48:20.978757+0000","last_fullsized":"2024-09-16T15:48:20.978757+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:34:32.400938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.958982+0000","last_change":"2024-09-16T15:48:20.958982+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.958982+0000","last_undegraded":"2024-09-16T15:48:20.958982+0000","last_fullsized":"2024-09-16T15:48:20.958982+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.976881+0000","last_change":"2024-09-16T15:48:20.976801+0000","last_active":"2024-09-16T15:48:20.976881+0000","last_peered":"2024-09-16T15:48:20.976881+0000","last_clean":"2024-09-16T15:48:20.976881+0000","last_became_active":"2024-09-16T15:48:20.975672+0000","last_became_peered":"2024-09-16T15:48:20.975672+0000","last_unstale":"2024-09-16T15:48:20.976881+0000","last_undegraded":"2024-09-16T15:48:20.976881+0000","last_fullsized":"2024-09-16T15:48:20.976881+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:59:53.683980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.965048+0000","last_change":"2024-09-16T15:48:20.965048+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.965048+0000","last_undegraded":"2024-09-16T15:48:20.965048+0000","last_fullsized":"2024-09-16T15:48:20.965048+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.977310+0000","last_change":"2024-09-16T15:48:20.974835+0000","last_active":"2024-09-16T15:48:20.977310+0000","last_peered":"2024-09-16T15:48:20.977310+0000","last_clean":"2024-09-16T15:48:20.977310+0000","last_became_active":"2024-09-16T15:48:20.974577+0000","last_became_peered":"2024-09-16T15:48:20.974577+0000","last_unstale":"2024-09-16T15:48:20.977310+0000","last_undegraded":"2024-09-16T15:48:20.977310+0000","last_fullsized":"2024-09-16T15:48:20.977310+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:29:48.935182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":238,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.955843+0000","last_change":"2024-09-16T15:39:12.698813+0000","last_active":"2024-09-16T15:48:20.955843+0000","last_peered":"2024-09-16T15:48:20.955843+0000","last_clean":"2024-09-16T15:48:20.955843+0000","last_became_active":"2024-09-16T15:39:12.697695+0000","last_became_peered":"2024-09-16T15:39:12.697695+0000","last_unstale":"2024-09-16T15:48:20.955843+0000","last_undegraded":"2024-09-16T15:48:20.955843+0000","last_fullsized":"2024-09-16T15:48:20.955843+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_clean_scrub_stamp":"2024-09-16T15:39:11.684967+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:02.061081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509555,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64108,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307732,"statfs":{"total":96636764160,"available":96571117568,"internally_reserved":0,"allocated":1073152,"data_stored":461993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":64551368},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509554,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63352,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94308488,"statfs":{"total":96636764160,"available":96571891712,"internally_reserved":0,"allocated":1667072,"data_stored":1052361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":63175111},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509553,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64124,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307716,"statfs":{"total":96636764160,"available":96571101184,"internally_reserved":0,"allocated":1085440,"data_stored":468324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509553,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58616,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313224,"statfs":{"total":96636764160,"available":96576741376,"internally_reserved":0,"allocated":1073152,"data_stored":461993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":58915273},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509553,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58612,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313228,"statfs":{"total":96636764160,"available":96576745472,"internally_reserved":0,"allocated":1073152,"data_stored":461993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509554,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59184,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312656,"statfs":{"total":96636764160,"available":96576159744,"internally_reserved":0,"allocated":1667072,"data_stored":1052361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509552,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58604,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313236,"statfs":{"total":96636764160,"available":96576753664,"internally_reserved":0,"allocated":1073152,"data_stored":461993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542260,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62804,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94309036,"statfs":{"total":96636764160,"available":96572452864,"internally_reserved":0,"allocated":1114112,"data_stored":481581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":63175116},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:48:26.185 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:48:26.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.377+0000 7fd721400640 1 -- 172.21.15.151:0/2280156778 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd71c0ccf00 msgr2=0x7fd71c0b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:26.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.377+0000 7fd721400640 1 --2- 172.21.15.151:0/2280156778 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd71c0ccf00 0x7fd71c0b2440 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fd708009830 tx=0x7fd70802a480 comp rx=0 tx=0).stop 2024-09-16T15:48:26.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.377+0000 7fd721400640 1 -- 172.21.15.151:0/2280156778 shutdown_connections 2024-09-16T15:48:26.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.377+0000 7fd721400640 1 --2- 172.21.15.151:0/2280156778 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd71c0ccf00 0x7fd71c0b2440 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:26.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.377+0000 7fd721400640 1 -- 172.21.15.151:0/2280156778 >> 172.21.15.151:0/2280156778 conn(0x7fd71c0b1950 msgr2=0x7fd71c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:26.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.377+0000 7fd721400640 1 -- 172.21.15.151:0/2280156778 shutdown_connections 2024-09-16T15:48:26.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.377+0000 7fd721400640 1 -- 172.21.15.151:0/2280156778 wait complete. 2024-09-16T15:48:26.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.379+0000 7fd721400640 1 Processor -- start 2024-09-16T15:48:26.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.379+0000 7fd721400640 1 -- start start 2024-09-16T15:48:26.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.380+0000 7fd721400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd71c073fb0 0x7fd71c154e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:26.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.380+0000 7fd721400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd71c0ccf00 0x7fd71c1f4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:26.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.380+0000 7fd721400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd71c15f5a0 0x7fd71c0ca9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:26.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.380+0000 7fd721400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd71c06f9f0 con 0x7fd71c073fb0 2024-09-16T15:48:26.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.380+0000 7fd721400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd71c06f870 con 0x7fd71c0ccf00 2024-09-16T15:48:26.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.380+0000 7fd721400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fd71c074420 con 0x7fd71c15f5a0 2024-09-16T15:48:26.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.380+0000 7fd71a800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd71c073fb0 0x7fd71c154e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:26.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.380+0000 7fd71a800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd71c073fb0 0x7fd71c154e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:42750/0 (socket says 172.21.15.151:42750) 2024-09-16T15:48:26.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.380+0000 7fd71a800640 1 -- 172.21.15.151:0/2829104287 learned_addr learned my addr 172.21.15.151:0/2829104287 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:48:26.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.380+0000 7fd719e00640 1 --2- 172.21.15.151:0/2829104287 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd71c0ccf00 0x7fd71c1f4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:26.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.380+0000 7fd71b200640 1 --2- 172.21.15.151:0/2829104287 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd71c15f5a0 0x7fd71c0ca9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:26.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.381+0000 7fd71a800640 1 -- 172.21.15.151:0/2829104287 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd71c15f5a0 msgr2=0x7fd71c0ca9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:26.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.381+0000 7fd71a800640 1 --2- 172.21.15.151:0/2829104287 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd71c15f5a0 0x7fd71c0ca9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:26.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.381+0000 7fd71a800640 1 -- 172.21.15.151:0/2829104287 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd71c0ccf00 msgr2=0x7fd71c1f4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:26.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.381+0000 7fd71a800640 1 --2- 172.21.15.151:0/2829104287 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd71c0ccf00 0x7fd71c1f4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:26.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.381+0000 7fd71a800640 1 -- 172.21.15.151:0/2829104287 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd71c1f57f0 con 0x7fd71c073fb0 2024-09-16T15:48:26.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.381+0000 7fd71a800640 1 --2- 172.21.15.151:0/2829104287 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd71c073fb0 0x7fd71c154e60 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fd70c00c3c0 tx=0x7fd70c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:26.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.381+0000 7fd712a00640 1 -- 172.21.15.151:0/2829104287 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd70c017070 con 0x7fd71c073fb0 2024-09-16T15:48:26.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.381+0000 7fd712a00640 1 -- 172.21.15.151:0/2829104287 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd70c0035f0 con 0x7fd71c073fb0 2024-09-16T15:48:26.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.381+0000 7fd721400640 1 -- 172.21.15.151:0/2829104287 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd71c15e0a0 con 0x7fd71c073fb0 2024-09-16T15:48:26.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.381+0000 7fd712a00640 1 -- 172.21.15.151:0/2829104287 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd70c003a40 con 0x7fd71c073fb0 2024-09-16T15:48:26.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.381+0000 7fd721400640 1 -- 172.21.15.151:0/2829104287 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd71c0c9410 con 0x7fd71c073fb0 2024-09-16T15:48:26.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.382+0000 7fd721400640 1 -- 172.21.15.151:0/2829104287 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd71c0ce7a0 con 0x7fd71c073fb0 2024-09-16T15:48:26.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.383+0000 7fd712a00640 1 -- 172.21.15.151:0/2829104287 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 30) ==== 102868+0+0 (secure 0 0 0) 0x7fd70c01a070 con 0x7fd71c073fb0 2024-09-16T15:48:26.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.385+0000 7fd712a00640 1 --2- 172.21.15.151:0/2829104287 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd6fc0799c0 0x7fd6fc07be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:26.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.385+0000 7fd712a00640 1 -- 172.21.15.151:0/2829104287 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7fd70c098ac0 con 0x7fd71c073fb0 2024-09-16T15:48:26.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.385+0000 7fd719e00640 1 --2- 172.21.15.151:0/2829104287 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd6fc0799c0 0x7fd6fc07be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:26.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.386+0000 7fd719e00640 1 --2- 172.21.15.151:0/2829104287 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd6fc0799c0 0x7fd6fc07be80 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fd708034000 tx=0x7fd708004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:26.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.389+0000 7fd712a00640 1 -- 172.21.15.151:0/2829104287 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd71c15e0a0 con 0x7fd71c073fb0 2024-09-16T15:48:26.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.571+0000 7fd721400640 1 -- 172.21.15.151:0/2829104287 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd71c1f4f50 con 0x7fd6fc0799c0 2024-09-16T15:48:26.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.577+0000 7fd712a00640 1 -- 172.21.15.151:0/2829104287 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247443 (secure 0 0 0) 0x7fd71c1f4f50 con 0x7fd6fc0799c0 2024-09-16T15:48:26.582 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:48:26.583 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-09-16T15:48:26.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.580+0000 7fd721400640 1 -- 172.21.15.151:0/2829104287 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd6fc0799c0 msgr2=0x7fd6fc07be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:26.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.580+0000 7fd721400640 1 --2- 172.21.15.151:0/2829104287 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd6fc0799c0 0x7fd6fc07be80 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fd708034000 tx=0x7fd708004730 comp rx=0 tx=0).stop 2024-09-16T15:48:26.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.580+0000 7fd721400640 1 -- 172.21.15.151:0/2829104287 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd71c073fb0 msgr2=0x7fd71c154e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:26.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.580+0000 7fd721400640 1 --2- 172.21.15.151:0/2829104287 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd71c073fb0 0x7fd71c154e60 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fd70c00c3c0 tx=0x7fd70c00c880 comp rx=0 tx=0).stop 2024-09-16T15:48:26.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.580+0000 7fd721400640 1 -- 172.21.15.151:0/2829104287 shutdown_connections 2024-09-16T15:48:26.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.580+0000 7fd721400640 1 --2- 172.21.15.151:0/2829104287 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd6fc0799c0 0x7fd6fc07be80 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:26.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.580+0000 7fd721400640 1 --2- 172.21.15.151:0/2829104287 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd71c0ccf00 0x7fd71c1f4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:26.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.580+0000 7fd721400640 1 --2- 172.21.15.151:0/2829104287 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd71c15f5a0 0x7fd71c0ca9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:26.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.580+0000 7fd721400640 1 --2- 172.21.15.151:0/2829104287 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd71c073fb0 0x7fd71c154e60 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:26.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.580+0000 7fd721400640 1 -- 172.21.15.151:0/2829104287 >> 172.21.15.151:0/2829104287 conn(0x7fd71c0b1950 msgr2=0x7fd71c15d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:26.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.580+0000 7fd721400640 1 -- 172.21.15.151:0/2829104287 shutdown_connections 2024-09-16T15:48:26.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.581+0000 7fd721400640 1 -- 172.21.15.151:0/2829104287 wait complete. 2024-09-16T15:48:26.599 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":362,"stamp":"2024-09-16T15:48:25.749544+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":108,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78,"ondisk_log_size":78,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":489404,"kb_used_data":9596,"kb_used_omap":12,"kb_used_meta":479603,"kb_avail":754485316,"statfs":{"total":773094113280,"available":772592963584,"internally_reserved":0,"allocated":9826304,"data_stored":4902599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12738,"internal_metadata":491114046},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.801460"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.964779+0000","last_change":"2024-09-16T15:48:20.964779+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.964779+0000","last_undegraded":"2024-09-16T15:48:20.964779+0000","last_fullsized":"2024-09-16T15:48:20.964779+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.956223+0000","last_change":"2024-09-16T15:48:20.956223+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.956223+0000","last_undegraded":"2024-09-16T15:48:20.956223+0000","last_fullsized":"2024-09-16T15:48:20.956223+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.966554+0000","last_change":"2024-09-16T15:48:20.966554+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.966554+0000","last_undegraded":"2024-09-16T15:48:20.966554+0000","last_fullsized":"2024-09-16T15:48:20.966554+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.958260+0000","last_change":"2024-09-16T15:48:20.958260+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.958260+0000","last_undegraded":"2024-09-16T15:48:20.958260+0000","last_fullsized":"2024-09-16T15:48:20.958260+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.961437+0000","last_change":"2024-09-16T15:48:20.961437+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.961437+0000","last_undegraded":"2024-09-16T15:48:20.961437+0000","last_fullsized":"2024-09-16T15:48:20.961437+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.964352+0000","last_change":"2024-09-16T15:48:20.964352+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.964352+0000","last_undegraded":"2024-09-16T15:48:20.964352+0000","last_fullsized":"2024-09-16T15:48:20.964352+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.956641+0000","last_change":"2024-09-16T15:48:20.956641+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.956641+0000","last_undegraded":"2024-09-16T15:48:20.956641+0000","last_fullsized":"2024-09-16T15:48:20.956641+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.961810+0000","last_change":"2024-09-16T15:48:20.961810+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.961810+0000","last_undegraded":"2024-09-16T15:48:20.961810+0000","last_fullsized":"2024-09-16T15:48:20.961810+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.964381+0000","last_change":"2024-09-16T15:48:20.964381+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.964381+0000","last_undegraded":"2024-09-16T15:48:20.964381+0000","last_fullsized":"2024-09-16T15:48:20.964381+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.964822+0000","last_change":"2024-09-16T15:48:20.964822+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.964822+0000","last_undegraded":"2024-09-16T15:48:20.964822+0000","last_fullsized":"2024-09-16T15:48:20.964822+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.962582+0000","last_change":"2024-09-16T15:48:20.962582+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.962582+0000","last_undegraded":"2024-09-16T15:48:20.962582+0000","last_fullsized":"2024-09-16T15:48:20.962582+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.963218+0000","last_change":"2024-09-16T15:48:20.963218+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.963218+0000","last_undegraded":"2024-09-16T15:48:20.963218+0000","last_fullsized":"2024-09-16T15:48:20.963218+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.959497+0000","last_change":"2024-09-16T15:48:20.959497+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.959497+0000","last_undegraded":"2024-09-16T15:48:20.959497+0000","last_fullsized":"2024-09-16T15:48:20.959497+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.351800+0000","last_change":"2024-09-16T15:48:21.991329+0000","last_active":"2024-09-16T15:48:23.351800+0000","last_peered":"2024-09-16T15:48:23.351800+0000","last_clean":"2024-09-16T15:48:23.351800+0000","last_became_active":"2024-09-16T15:48:21.990123+0000","last_became_peered":"2024-09-16T15:48:21.990123+0000","last_unstale":"2024-09-16T15:48:23.351800+0000","last_undegraded":"2024-09-16T15:48:23.351800+0000","last_fullsized":"2024-09-16T15:48:23.351800+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:01:55.713464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.956053+0000","last_change":"2024-09-16T15:48:20.956053+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.956053+0000","last_undegraded":"2024-09-16T15:48:20.956053+0000","last_fullsized":"2024-09-16T15:48:20.956053+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.964568+0000","last_change":"2024-09-16T15:48:20.964568+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.964568+0000","last_undegraded":"2024-09-16T15:48:20.964568+0000","last_fullsized":"2024-09-16T15:48:20.964568+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.960600+0000","last_change":"2024-09-16T15:48:20.960600+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.960600+0000","last_undegraded":"2024-09-16T15:48:20.960600+0000","last_fullsized":"2024-09-16T15:48:20.960600+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.965863+0000","last_change":"2024-09-16T15:48:20.965863+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.965863+0000","last_undegraded":"2024-09-16T15:48:20.965863+0000","last_fullsized":"2024-09-16T15:48:20.965863+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352178+0000","last_change":"2024-09-16T15:48:21.996135+0000","last_active":"2024-09-16T15:48:23.352178+0000","last_peered":"2024-09-16T15:48:23.352178+0000","last_clean":"2024-09-16T15:48:23.352178+0000","last_became_active":"2024-09-16T15:48:21.995671+0000","last_became_peered":"2024-09-16T15:48:21.995671+0000","last_unstale":"2024-09-16T15:48:23.352178+0000","last_undegraded":"2024-09-16T15:48:23.352178+0000","last_fullsized":"2024-09-16T15:48:23.352178+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:13:55.927762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352319+0000","last_change":"2024-09-16T15:48:21.994572+0000","last_active":"2024-09-16T15:48:23.352319+0000","last_peered":"2024-09-16T15:48:23.352319+0000","last_clean":"2024-09-16T15:48:23.352319+0000","last_became_active":"2024-09-16T15:48:21.994305+0000","last_became_peered":"2024-09-16T15:48:21.994305+0000","last_unstale":"2024-09-16T15:48:23.352319+0000","last_undegraded":"2024-09-16T15:48:23.352319+0000","last_fullsized":"2024-09-16T15:48:23.352319+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:27:24.090121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.958863+0000","last_change":"2024-09-16T15:48:20.958863+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.958863+0000","last_undegraded":"2024-09-16T15:48:20.958863+0000","last_fullsized":"2024-09-16T15:48:20.958863+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.351839+0000","last_change":"2024-09-16T15:48:21.988902+0000","last_active":"2024-09-16T15:48:23.351839+0000","last_peered":"2024-09-16T15:48:23.351839+0000","last_clean":"2024-09-16T15:48:23.351839+0000","last_became_active":"2024-09-16T15:48:21.988613+0000","last_became_peered":"2024-09-16T15:48:21.988613+0000","last_unstale":"2024-09-16T15:48:23.351839+0000","last_undegraded":"2024-09-16T15:48:23.351839+0000","last_fullsized":"2024-09-16T15:48:23.351839+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:05:00.466068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.962242+0000","last_change":"2024-09-16T15:48:20.962242+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.962242+0000","last_undegraded":"2024-09-16T15:48:20.962242+0000","last_fullsized":"2024-09-16T15:48:20.962242+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.962313+0000","last_change":"2024-09-16T15:48:20.962313+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.962313+0000","last_undegraded":"2024-09-16T15:48:20.962313+0000","last_fullsized":"2024-09-16T15:48:20.962313+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.966477+0000","last_change":"2024-09-16T15:48:20.966477+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.966477+0000","last_undegraded":"2024-09-16T15:48:20.966477+0000","last_fullsized":"2024-09-16T15:48:20.966477+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.957649+0000","last_change":"2024-09-16T15:48:20.957649+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.957649+0000","last_undegraded":"2024-09-16T15:48:20.957649+0000","last_fullsized":"2024-09-16T15:48:20.957649+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.958211+0000","last_change":"2024-09-16T15:48:20.958211+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.958211+0000","last_undegraded":"2024-09-16T15:48:20.958211+0000","last_fullsized":"2024-09-16T15:48:20.958211+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.963150+0000","last_change":"2024-09-16T15:48:20.963150+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.963150+0000","last_undegraded":"2024-09-16T15:48:20.963150+0000","last_fullsized":"2024-09-16T15:48:20.963150+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.957984+0000","last_change":"2024-09-16T15:48:20.957984+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.957984+0000","last_undegraded":"2024-09-16T15:48:20.957984+0000","last_fullsized":"2024-09-16T15:48:20.957984+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352274+0000","last_change":"2024-09-16T15:48:21.991963+0000","last_active":"2024-09-16T15:48:23.352274+0000","last_peered":"2024-09-16T15:48:23.352274+0000","last_clean":"2024-09-16T15:48:23.352274+0000","last_became_active":"2024-09-16T15:48:21.991698+0000","last_became_peered":"2024-09-16T15:48:21.991698+0000","last_unstale":"2024-09-16T15:48:23.352274+0000","last_undegraded":"2024-09-16T15:48:23.352274+0000","last_fullsized":"2024-09-16T15:48:23.352274+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:21:15.053411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.961609+0000","last_change":"2024-09-16T15:48:20.961609+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.961609+0000","last_undegraded":"2024-09-16T15:48:20.961609+0000","last_fullsized":"2024-09-16T15:48:20.961609+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.962070+0000","last_change":"2024-09-16T15:48:20.962070+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.962070+0000","last_undegraded":"2024-09-16T15:48:20.962070+0000","last_fullsized":"2024-09-16T15:48:20.962070+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.961053+0000","last_change":"2024-09-16T15:48:20.961053+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.961053+0000","last_undegraded":"2024-09-16T15:48:20.961053+0000","last_fullsized":"2024-09-16T15:48:20.961053+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.977633+0000","last_change":"2024-09-16T15:48:20.975503+0000","last_active":"2024-09-16T15:48:20.977633+0000","last_peered":"2024-09-16T15:48:20.977633+0000","last_clean":"2024-09-16T15:48:20.977633+0000","last_became_active":"2024-09-16T15:48:20.973819+0000","last_became_peered":"2024-09-16T15:48:20.973819+0000","last_unstale":"2024-09-16T15:48:20.977633+0000","last_undegraded":"2024-09-16T15:48:20.977633+0000","last_fullsized":"2024-09-16T15:48:20.977633+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:39:26.482165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.963995+0000","last_change":"2024-09-16T15:48:20.963995+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.963995+0000","last_undegraded":"2024-09-16T15:48:20.963995+0000","last_fullsized":"2024-09-16T15:48:20.963995+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.976680+0000","last_change":"2024-09-16T15:48:20.974951+0000","last_active":"2024-09-16T15:48:20.976680+0000","last_peered":"2024-09-16T15:48:20.976680+0000","last_clean":"2024-09-16T15:48:20.976680+0000","last_became_active":"2024-09-16T15:48:20.974688+0000","last_became_peered":"2024-09-16T15:48:20.974688+0000","last_unstale":"2024-09-16T15:48:20.976680+0000","last_undegraded":"2024-09-16T15:48:20.976680+0000","last_fullsized":"2024-09-16T15:48:20.976680+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:40:29.438048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.960641+0000","last_change":"2024-09-16T15:48:20.960641+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.960641+0000","last_undegraded":"2024-09-16T15:48:20.960641+0000","last_fullsized":"2024-09-16T15:48:20.960641+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.981008+0000","last_change":"2024-09-16T15:48:20.978653+0000","last_active":"2024-09-16T15:48:20.981008+0000","last_peered":"2024-09-16T15:48:20.981008+0000","last_clean":"2024-09-16T15:48:20.981008+0000","last_became_active":"2024-09-16T15:48:20.976364+0000","last_became_peered":"2024-09-16T15:48:20.976364+0000","last_unstale":"2024-09-16T15:48:20.981008+0000","last_undegraded":"2024-09-16T15:48:20.981008+0000","last_fullsized":"2024-09-16T15:48:20.981008+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:01:37.109321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.961183+0000","last_change":"2024-09-16T15:48:20.961183+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.961183+0000","last_undegraded":"2024-09-16T15:48:20.961183+0000","last_fullsized":"2024-09-16T15:48:20.961183+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.969107+0000","last_change":"2024-09-16T15:48:20.967236+0000","last_active":"2024-09-16T15:48:20.969107+0000","last_peered":"2024-09-16T15:48:20.969107+0000","last_clean":"2024-09-16T15:48:20.969107+0000","last_became_active":"2024-09-16T15:48:20.966954+0000","last_became_peered":"2024-09-16T15:48:20.966954+0000","last_unstale":"2024-09-16T15:48:20.969107+0000","last_undegraded":"2024-09-16T15:48:20.969107+0000","last_fullsized":"2024-09-16T15:48:20.969107+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:02:43.930006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.959326+0000","last_change":"2024-09-16T15:48:20.959326+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.959326+0000","last_undegraded":"2024-09-16T15:48:20.959326+0000","last_fullsized":"2024-09-16T15:48:20.959326+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.975825+0000","last_change":"2024-09-16T15:48:20.975041+0000","last_active":"2024-09-16T15:48:20.975825+0000","last_peered":"2024-09-16T15:48:20.975825+0000","last_clean":"2024-09-16T15:48:20.975825+0000","last_became_active":"2024-09-16T15:48:20.971719+0000","last_became_peered":"2024-09-16T15:48:20.971719+0000","last_unstale":"2024-09-16T15:48:20.975825+0000","last_undegraded":"2024-09-16T15:48:20.975825+0000","last_fullsized":"2024-09-16T15:48:20.975825+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:32:44.623249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.962285+0000","last_change":"2024-09-16T15:48:20.962285+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.962285+0000","last_undegraded":"2024-09-16T15:48:20.962285+0000","last_fullsized":"2024-09-16T15:48:20.962285+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.976080+0000","last_change":"2024-09-16T15:48:20.971997+0000","last_active":"2024-09-16T15:48:20.976080+0000","last_peered":"2024-09-16T15:48:20.976080+0000","last_clean":"2024-09-16T15:48:20.976080+0000","last_became_active":"2024-09-16T15:48:20.971230+0000","last_became_peered":"2024-09-16T15:48:20.971230+0000","last_unstale":"2024-09-16T15:48:20.976080+0000","last_undegraded":"2024-09-16T15:48:20.976080+0000","last_fullsized":"2024-09-16T15:48:20.976080+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:20:05.425588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.962335+0000","last_change":"2024-09-16T15:48:20.962335+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.962335+0000","last_undegraded":"2024-09-16T15:48:20.962335+0000","last_fullsized":"2024-09-16T15:48:20.962335+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.351663+0000","last_change":"2024-09-16T15:48:20.977058+0000","last_active":"2024-09-16T15:48:23.351663+0000","last_peered":"2024-09-16T15:48:23.351663+0000","last_clean":"2024-09-16T15:48:23.351663+0000","last_became_active":"2024-09-16T15:48:20.975591+0000","last_became_peered":"2024-09-16T15:48:20.975591+0000","last_unstale":"2024-09-16T15:48:23.351663+0000","last_undegraded":"2024-09-16T15:48:23.351663+0000","last_fullsized":"2024-09-16T15:48:23.351663+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:58:51.576740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.351992+0000","last_change":"2024-09-16T15:48:21.987074+0000","last_active":"2024-09-16T15:48:23.351992+0000","last_peered":"2024-09-16T15:48:23.351992+0000","last_clean":"2024-09-16T15:48:23.351992+0000","last_became_active":"2024-09-16T15:48:21.986779+0000","last_became_peered":"2024-09-16T15:48:21.986779+0000","last_unstale":"2024-09-16T15:48:23.351992+0000","last_undegraded":"2024-09-16T15:48:23.351992+0000","last_fullsized":"2024-09-16T15:48:23.351992+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:57:02.685046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.351536+0000","last_change":"2024-09-16T15:48:20.977637+0000","last_active":"2024-09-16T15:48:23.351536+0000","last_peered":"2024-09-16T15:48:23.351536+0000","last_clean":"2024-09-16T15:48:23.351536+0000","last_became_active":"2024-09-16T15:48:20.977367+0000","last_became_peered":"2024-09-16T15:48:20.977367+0000","last_unstale":"2024-09-16T15:48:23.351536+0000","last_undegraded":"2024-09-16T15:48:23.351536+0000","last_fullsized":"2024-09-16T15:48:23.351536+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:57:12.992776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352256+0000","last_change":"2024-09-16T15:48:21.990047+0000","last_active":"2024-09-16T15:48:23.352256+0000","last_peered":"2024-09-16T15:48:23.352256+0000","last_clean":"2024-09-16T15:48:23.352256+0000","last_became_active":"2024-09-16T15:48:21.989735+0000","last_became_peered":"2024-09-16T15:48:21.989735+0000","last_unstale":"2024-09-16T15:48:23.352256+0000","last_undegraded":"2024-09-16T15:48:23.352256+0000","last_fullsized":"2024-09-16T15:48:23.352256+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:50:37.046745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.971713+0000","last_change":"2024-09-16T15:48:20.969036+0000","last_active":"2024-09-16T15:48:20.971713+0000","last_peered":"2024-09-16T15:48:20.971713+0000","last_clean":"2024-09-16T15:48:20.971713+0000","last_became_active":"2024-09-16T15:48:20.968771+0000","last_became_peered":"2024-09-16T15:48:20.968771+0000","last_unstale":"2024-09-16T15:48:20.971713+0000","last_undegraded":"2024-09-16T15:48:20.971713+0000","last_fullsized":"2024-09-16T15:48:20.971713+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:27:21.822056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.962027+0000","last_change":"2024-09-16T15:48:20.962027+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.962027+0000","last_undegraded":"2024-09-16T15:48:20.962027+0000","last_fullsized":"2024-09-16T15:48:20.962027+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.969598+0000","last_change":"2024-09-16T15:48:20.968246+0000","last_active":"2024-09-16T15:48:20.969598+0000","last_peered":"2024-09-16T15:48:20.969598+0000","last_clean":"2024-09-16T15:48:20.969598+0000","last_became_active":"2024-09-16T15:48:20.967963+0000","last_became_peered":"2024-09-16T15:48:20.967963+0000","last_unstale":"2024-09-16T15:48:20.969598+0000","last_undegraded":"2024-09-16T15:48:20.969598+0000","last_fullsized":"2024-09-16T15:48:20.969598+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:23:26.528353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.967777+0000","last_change":"2024-09-16T15:48:20.967777+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.967777+0000","last_undegraded":"2024-09-16T15:48:20.967777+0000","last_fullsized":"2024-09-16T15:48:20.967777+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.977951+0000","last_change":"2024-09-16T15:48:20.975763+0000","last_active":"2024-09-16T15:48:20.977951+0000","last_peered":"2024-09-16T15:48:20.977951+0000","last_clean":"2024-09-16T15:48:20.977951+0000","last_became_active":"2024-09-16T15:48:20.974952+0000","last_became_peered":"2024-09-16T15:48:20.974952+0000","last_unstale":"2024-09-16T15:48:20.977951+0000","last_undegraded":"2024-09-16T15:48:20.977951+0000","last_fullsized":"2024-09-16T15:48:20.977951+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:37:00.965428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.958573+0000","last_change":"2024-09-16T15:48:20.958573+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.958573+0000","last_undegraded":"2024-09-16T15:48:20.958573+0000","last_fullsized":"2024-09-16T15:48:20.958573+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.974235+0000","last_change":"2024-09-16T15:48:20.972430+0000","last_active":"2024-09-16T15:48:20.974235+0000","last_peered":"2024-09-16T15:48:20.974235+0000","last_clean":"2024-09-16T15:48:20.974235+0000","last_became_active":"2024-09-16T15:48:20.972159+0000","last_became_peered":"2024-09-16T15:48:20.972159+0000","last_unstale":"2024-09-16T15:48:20.974235+0000","last_undegraded":"2024-09-16T15:48:20.974235+0000","last_fullsized":"2024-09-16T15:48:20.974235+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:02:58.958514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.965797+0000","last_change":"2024-09-16T15:48:20.965797+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.965797+0000","last_undegraded":"2024-09-16T15:48:20.965797+0000","last_fullsized":"2024-09-16T15:48:20.965797+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.969032+0000","last_change":"2024-09-16T15:48:20.965556+0000","last_active":"2024-09-16T15:48:20.969032+0000","last_peered":"2024-09-16T15:48:20.969032+0000","last_clean":"2024-09-16T15:48:20.969032+0000","last_became_active":"2024-09-16T15:48:20.965196+0000","last_became_peered":"2024-09-16T15:48:20.965196+0000","last_unstale":"2024-09-16T15:48:20.969032+0000","last_undegraded":"2024-09-16T15:48:20.969032+0000","last_fullsized":"2024-09-16T15:48:20.969032+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:57:22.438388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.959493+0000","last_change":"2024-09-16T15:48:20.959493+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.959493+0000","last_undegraded":"2024-09-16T15:48:20.959493+0000","last_fullsized":"2024-09-16T15:48:20.959493+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.970792+0000","last_change":"2024-09-16T15:48:20.969210+0000","last_active":"2024-09-16T15:48:20.970792+0000","last_peered":"2024-09-16T15:48:20.970792+0000","last_clean":"2024-09-16T15:48:20.970792+0000","last_became_active":"2024-09-16T15:48:20.968946+0000","last_became_peered":"2024-09-16T15:48:20.968946+0000","last_unstale":"2024-09-16T15:48:20.970792+0000","last_undegraded":"2024-09-16T15:48:20.970792+0000","last_fullsized":"2024-09-16T15:48:20.970792+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:43:02.173876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352100+0000","last_change":"2024-09-16T15:48:21.997110+0000","last_active":"2024-09-16T15:48:23.352100+0000","last_peered":"2024-09-16T15:48:23.352100+0000","last_clean":"2024-09-16T15:48:23.352100+0000","last_became_active":"2024-09-16T15:48:21.996855+0000","last_became_peered":"2024-09-16T15:48:21.996855+0000","last_unstale":"2024-09-16T15:48:23.352100+0000","last_undegraded":"2024-09-16T15:48:23.352100+0000","last_fullsized":"2024-09-16T15:48:23.352100+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:44:31.136306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.970157+0000","last_change":"2024-09-16T15:48:20.969040+0000","last_active":"2024-09-16T15:48:20.970157+0000","last_peered":"2024-09-16T15:48:20.970157+0000","last_clean":"2024-09-16T15:48:20.970157+0000","last_became_active":"2024-09-16T15:48:20.968541+0000","last_became_peered":"2024-09-16T15:48:20.968541+0000","last_unstale":"2024-09-16T15:48:20.970157+0000","last_undegraded":"2024-09-16T15:48:20.970157+0000","last_fullsized":"2024-09-16T15:48:20.970157+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:02:23.941065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.964413+0000","last_change":"2024-09-16T15:48:20.964413+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.964413+0000","last_undegraded":"2024-09-16T15:48:20.964413+0000","last_fullsized":"2024-09-16T15:48:20.964413+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.979658+0000","last_change":"2024-09-16T15:48:20.975861+0000","last_active":"2024-09-16T15:48:20.979658+0000","last_peered":"2024-09-16T15:48:20.979658+0000","last_clean":"2024-09-16T15:48:20.979658+0000","last_became_active":"2024-09-16T15:48:20.974746+0000","last_became_peered":"2024-09-16T15:48:20.974746+0000","last_unstale":"2024-09-16T15:48:20.979658+0000","last_undegraded":"2024-09-16T15:48:20.979658+0000","last_fullsized":"2024-09-16T15:48:20.979658+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:13.847700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.960635+0000","last_change":"2024-09-16T15:48:20.960635+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.960635+0000","last_undegraded":"2024-09-16T15:48:20.960635+0000","last_fullsized":"2024-09-16T15:48:20.960635+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.977666+0000","last_change":"2024-09-16T15:48:20.975777+0000","last_active":"2024-09-16T15:48:20.977666+0000","last_peered":"2024-09-16T15:48:20.977666+0000","last_clean":"2024-09-16T15:48:20.977666+0000","last_became_active":"2024-09-16T15:48:20.974162+0000","last_became_peered":"2024-09-16T15:48:20.974162+0000","last_unstale":"2024-09-16T15:48:20.977666+0000","last_undegraded":"2024-09-16T15:48:20.977666+0000","last_fullsized":"2024-09-16T15:48:20.977666+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:34:52.666788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.961693+0000","last_change":"2024-09-16T15:48:20.961693+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.961693+0000","last_undegraded":"2024-09-16T15:48:20.961693+0000","last_fullsized":"2024-09-16T15:48:20.961693+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"90'2","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352068+0000","last_change":"2024-09-16T15:48:20.975158+0000","last_active":"2024-09-16T15:48:23.352068+0000","last_peered":"2024-09-16T15:48:23.352068+0000","last_clean":"2024-09-16T15:48:23.352068+0000","last_became_active":"2024-09-16T15:48:20.974854+0000","last_became_peered":"2024-09-16T15:48:20.974854+0000","last_unstale":"2024-09-16T15:48:23.352068+0000","last_undegraded":"2024-09-16T15:48:23.352068+0000","last_fullsized":"2024-09-16T15:48:23.352068+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:41:44.873886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.958552+0000","last_change":"2024-09-16T15:48:20.958552+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.958552+0000","last_undegraded":"2024-09-16T15:48:20.958552+0000","last_fullsized":"2024-09-16T15:48:20.958552+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352234+0000","last_change":"2024-09-16T15:48:20.978031+0000","last_active":"2024-09-16T15:48:23.352234+0000","last_peered":"2024-09-16T15:48:23.352234+0000","last_clean":"2024-09-16T15:48:23.352234+0000","last_became_active":"2024-09-16T15:48:20.976491+0000","last_became_peered":"2024-09-16T15:48:20.976491+0000","last_unstale":"2024-09-16T15:48:23.352234+0000","last_undegraded":"2024-09-16T15:48:23.352234+0000","last_fullsized":"2024-09-16T15:48:23.352234+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:57.979104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.956433+0000","last_change":"2024-09-16T15:48:20.956433+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.956433+0000","last_undegraded":"2024-09-16T15:48:20.956433+0000","last_fullsized":"2024-09-16T15:48:20.956433+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.b","version":"90'2","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.351800+0000","last_change":"2024-09-16T15:48:20.976944+0000","last_active":"2024-09-16T15:48:23.351800+0000","last_peered":"2024-09-16T15:48:23.351800+0000","last_clean":"2024-09-16T15:48:23.351800+0000","last_became_active":"2024-09-16T15:48:20.976649+0000","last_became_peered":"2024-09-16T15:48:20.976649+0000","last_unstale":"2024-09-16T15:48:23.351800+0000","last_undegraded":"2024-09-16T15:48:23.351800+0000","last_fullsized":"2024-09-16T15:48:23.351800+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:33:01.893406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.957113+0000","last_change":"2024-09-16T15:48:20.957113+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.957113+0000","last_undegraded":"2024-09-16T15:48:20.957113+0000","last_fullsized":"2024-09-16T15:48:20.957113+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.973153+0000","last_change":"2024-09-16T15:48:20.970601+0000","last_active":"2024-09-16T15:48:20.973153+0000","last_peered":"2024-09-16T15:48:20.973153+0000","last_clean":"2024-09-16T15:48:20.973153+0000","last_became_active":"2024-09-16T15:48:20.970272+0000","last_became_peered":"2024-09-16T15:48:20.970272+0000","last_unstale":"2024-09-16T15:48:20.973153+0000","last_undegraded":"2024-09-16T15:48:20.973153+0000","last_fullsized":"2024-09-16T15:48:20.973153+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:31:13.471665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.957294+0000","last_change":"2024-09-16T15:48:20.957294+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.957294+0000","last_undegraded":"2024-09-16T15:48:20.957294+0000","last_fullsized":"2024-09-16T15:48:20.957294+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.973815+0000","last_change":"2024-09-16T15:48:20.971710+0000","last_active":"2024-09-16T15:48:20.973815+0000","last_peered":"2024-09-16T15:48:20.973815+0000","last_clean":"2024-09-16T15:48:20.973815+0000","last_became_active":"2024-09-16T15:48:20.971360+0000","last_became_peered":"2024-09-16T15:48:20.971360+0000","last_unstale":"2024-09-16T15:48:20.973815+0000","last_undegraded":"2024-09-16T15:48:20.973815+0000","last_fullsized":"2024-09-16T15:48:20.973815+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:53:28.706338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.959074+0000","last_change":"2024-09-16T15:48:20.959074+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.959074+0000","last_undegraded":"2024-09-16T15:48:20.959074+0000","last_fullsized":"2024-09-16T15:48:20.959074+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.351750+0000","last_change":"2024-09-16T15:48:20.980185+0000","last_active":"2024-09-16T15:48:23.351750+0000","last_peered":"2024-09-16T15:48:23.351750+0000","last_clean":"2024-09-16T15:48:23.351750+0000","last_became_active":"2024-09-16T15:48:20.979875+0000","last_became_peered":"2024-09-16T15:48:20.979875+0000","last_unstale":"2024-09-16T15:48:23.351750+0000","last_undegraded":"2024-09-16T15:48:23.351750+0000","last_fullsized":"2024-09-16T15:48:23.351750+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:56:29.587401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.957180+0000","last_change":"2024-09-16T15:48:20.957180+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.957180+0000","last_undegraded":"2024-09-16T15:48:20.957180+0000","last_fullsized":"2024-09-16T15:48:20.957180+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"90'2","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352056+0000","last_change":"2024-09-16T15:48:20.978563+0000","last_active":"2024-09-16T15:48:23.352056+0000","last_peered":"2024-09-16T15:48:23.352056+0000","last_clean":"2024-09-16T15:48:23.352056+0000","last_became_active":"2024-09-16T15:48:20.978188+0000","last_became_peered":"2024-09-16T15:48:20.978188+0000","last_unstale":"2024-09-16T15:48:23.352056+0000","last_undegraded":"2024-09-16T15:48:23.352056+0000","last_fullsized":"2024-09-16T15:48:23.352056+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:28:27.584567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.957910+0000","last_change":"2024-09-16T15:48:20.957910+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.957910+0000","last_undegraded":"2024-09-16T15:48:20.957910+0000","last_fullsized":"2024-09-16T15:48:20.957910+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.979361+0000","last_change":"2024-09-16T15:48:20.975435+0000","last_active":"2024-09-16T15:48:20.979361+0000","last_peered":"2024-09-16T15:48:20.979361+0000","last_clean":"2024-09-16T15:48:20.979361+0000","last_became_active":"2024-09-16T15:48:20.975091+0000","last_became_peered":"2024-09-16T15:48:20.975091+0000","last_unstale":"2024-09-16T15:48:20.979361+0000","last_undegraded":"2024-09-16T15:48:20.979361+0000","last_fullsized":"2024-09-16T15:48:20.979361+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:43:06.025743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.957792+0000","last_change":"2024-09-16T15:48:20.957792+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.957792+0000","last_undegraded":"2024-09-16T15:48:20.957792+0000","last_fullsized":"2024-09-16T15:48:20.957792+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.975126+0000","last_change":"2024-09-16T15:48:20.974118+0000","last_active":"2024-09-16T15:48:20.975126+0000","last_peered":"2024-09-16T15:48:20.975126+0000","last_clean":"2024-09-16T15:48:20.975126+0000","last_became_active":"2024-09-16T15:48:20.973841+0000","last_became_peered":"2024-09-16T15:48:20.973841+0000","last_unstale":"2024-09-16T15:48:20.975126+0000","last_undegraded":"2024-09-16T15:48:20.975126+0000","last_fullsized":"2024-09-16T15:48:20.975126+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:09:11.275872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.960754+0000","last_change":"2024-09-16T15:48:20.960754+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.960754+0000","last_undegraded":"2024-09-16T15:48:20.960754+0000","last_fullsized":"2024-09-16T15:48:20.960754+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.976606+0000","last_change":"2024-09-16T15:48:20.974113+0000","last_active":"2024-09-16T15:48:20.976606+0000","last_peered":"2024-09-16T15:48:20.976606+0000","last_clean":"2024-09-16T15:48:20.976606+0000","last_became_active":"2024-09-16T15:48:20.973851+0000","last_became_peered":"2024-09-16T15:48:20.973851+0000","last_unstale":"2024-09-16T15:48:20.976606+0000","last_undegraded":"2024-09-16T15:48:20.976606+0000","last_fullsized":"2024-09-16T15:48:20.976606+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:14:35.885189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.956848+0000","last_change":"2024-09-16T15:48:20.956848+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.956848+0000","last_undegraded":"2024-09-16T15:48:20.956848+0000","last_fullsized":"2024-09-16T15:48:20.956848+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.972184+0000","last_change":"2024-09-16T15:48:20.968528+0000","last_active":"2024-09-16T15:48:20.972184+0000","last_peered":"2024-09-16T15:48:20.972184+0000","last_clean":"2024-09-16T15:48:20.972184+0000","last_became_active":"2024-09-16T15:48:20.967944+0000","last_became_peered":"2024-09-16T15:48:20.967944+0000","last_unstale":"2024-09-16T15:48:20.972184+0000","last_undegraded":"2024-09-16T15:48:20.972184+0000","last_fullsized":"2024-09-16T15:48:20.972184+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:24:23.675488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.962709+0000","last_change":"2024-09-16T15:48:20.962709+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.962709+0000","last_undegraded":"2024-09-16T15:48:20.962709+0000","last_fullsized":"2024-09-16T15:48:20.962709+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.966248+0000","last_change":"2024-09-16T15:48:20.963312+0000","last_active":"2024-09-16T15:48:20.966248+0000","last_peered":"2024-09-16T15:48:20.966248+0000","last_clean":"2024-09-16T15:48:20.966248+0000","last_became_active":"2024-09-16T15:48:20.963034+0000","last_became_peered":"2024-09-16T15:48:20.963034+0000","last_unstale":"2024-09-16T15:48:20.966248+0000","last_undegraded":"2024-09-16T15:48:20.966248+0000","last_fullsized":"2024-09-16T15:48:20.966248+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:39:53.503131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.956619+0000","last_change":"2024-09-16T15:48:20.956619+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.956619+0000","last_undegraded":"2024-09-16T15:48:20.956619+0000","last_fullsized":"2024-09-16T15:48:20.956619+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.978757+0000","last_change":"2024-09-16T15:48:20.978028+0000","last_active":"2024-09-16T15:48:20.978757+0000","last_peered":"2024-09-16T15:48:20.978757+0000","last_clean":"2024-09-16T15:48:20.978757+0000","last_became_active":"2024-09-16T15:48:20.977556+0000","last_became_peered":"2024-09-16T15:48:20.977556+0000","last_unstale":"2024-09-16T15:48:20.978757+0000","last_undegraded":"2024-09-16T15:48:20.978757+0000","last_fullsized":"2024-09-16T15:48:20.978757+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:34:32.400938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.958982+0000","last_change":"2024-09-16T15:48:20.958982+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.958982+0000","last_undegraded":"2024-09-16T15:48:20.958982+0000","last_fullsized":"2024-09-16T15:48:20.958982+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.976881+0000","last_change":"2024-09-16T15:48:20.976801+0000","last_active":"2024-09-16T15:48:20.976881+0000","last_peered":"2024-09-16T15:48:20.976881+0000","last_clean":"2024-09-16T15:48:20.976881+0000","last_became_active":"2024-09-16T15:48:20.975672+0000","last_became_peered":"2024-09-16T15:48:20.975672+0000","last_unstale":"2024-09-16T15:48:20.976881+0000","last_undegraded":"2024-09-16T15:48:20.976881+0000","last_fullsized":"2024-09-16T15:48:20.976881+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:59:53.683980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.965048+0000","last_change":"2024-09-16T15:48:20.965048+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.965048+0000","last_undegraded":"2024-09-16T15:48:20.965048+0000","last_fullsized":"2024-09-16T15:48:20.965048+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.977310+0000","last_change":"2024-09-16T15:48:20.974835+0000","last_active":"2024-09-16T15:48:20.977310+0000","last_peered":"2024-09-16T15:48:20.977310+0000","last_clean":"2024-09-16T15:48:20.977310+0000","last_became_active":"2024-09-16T15:48:20.974577+0000","last_became_peered":"2024-09-16T15:48:20.974577+0000","last_unstale":"2024-09-16T15:48:20.977310+0000","last_undegraded":"2024-09-16T15:48:20.977310+0000","last_fullsized":"2024-09-16T15:48:20.977310+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:29:48.935182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":238,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.955843+0000","last_change":"2024-09-16T15:39:12.698813+0000","last_active":"2024-09-16T15:48:20.955843+0000","last_peered":"2024-09-16T15:48:20.955843+0000","last_clean":"2024-09-16T15:48:20.955843+0000","last_became_active":"2024-09-16T15:39:12.697695+0000","last_became_peered":"2024-09-16T15:39:12.697695+0000","last_unstale":"2024-09-16T15:48:20.955843+0000","last_undegraded":"2024-09-16T15:48:20.955843+0000","last_fullsized":"2024-09-16T15:48:20.955843+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_clean_scrub_stamp":"2024-09-16T15:39:11.684967+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:02.061081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509555,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64108,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307732,"statfs":{"total":96636764160,"available":96571117568,"internally_reserved":0,"allocated":1073152,"data_stored":461993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":64551368},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509554,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63352,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94308488,"statfs":{"total":96636764160,"available":96571891712,"internally_reserved":0,"allocated":1667072,"data_stored":1052361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":63175111},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509553,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64124,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307716,"statfs":{"total":96636764160,"available":96571101184,"internally_reserved":0,"allocated":1085440,"data_stored":468324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509553,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58616,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313224,"statfs":{"total":96636764160,"available":96576741376,"internally_reserved":0,"allocated":1073152,"data_stored":461993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":58915273},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509553,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58612,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313228,"statfs":{"total":96636764160,"available":96576745472,"internally_reserved":0,"allocated":1073152,"data_stored":461993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509554,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59184,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312656,"statfs":{"total":96636764160,"available":96576159744,"internally_reserved":0,"allocated":1667072,"data_stored":1052361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509552,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58604,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313236,"statfs":{"total":96636764160,"available":96576753664,"internally_reserved":0,"allocated":1073152,"data_stored":461993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542260,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62804,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94309036,"statfs":{"total":96636764160,"available":96572452864,"internally_reserved":0,"allocated":1114112,"data_stored":481581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":63175116},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:48:26.603 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:48:26.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.796+0000 7f5a9dc00640 1 -- 172.21.15.151:0/217481287 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a7c0039e0 msgr2=0x7f5a7c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:26.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.796+0000 7f5a9dc00640 1 --2- 172.21.15.151:0/217481287 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a7c0039e0 0x7f5a7c003dc0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f5a88009eb0 tx=0x7f5a7c005830 comp rx=0 tx=0).stop 2024-09-16T15:48:26.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.796+0000 7f5a97400640 1 -- 172.21.15.151:0/217481287 reap_dead start 2024-09-16T15:48:26.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.796+0000 7f5a9dc00640 1 -- 172.21.15.151:0/217481287 shutdown_connections 2024-09-16T15:48:26.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.796+0000 7f5a9dc00640 1 -- 172.21.15.151:0/217481287 >> 172.21.15.151:0/217481287 conn(0x7f5a980b1950 msgr2=0x7f5a980b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:26.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.796+0000 7f5a9dc00640 1 -- 172.21.15.151:0/217481287 shutdown_connections 2024-09-16T15:48:26.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.796+0000 7f5a9dc00640 1 -- 172.21.15.151:0/217481287 wait complete. 2024-09-16T15:48:26.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.798+0000 7f5a9dc00640 1 Processor -- start 2024-09-16T15:48:26.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.798+0000 7f5a9dc00640 1 -- start start 2024-09-16T15:48:26.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.799+0000 7f5a9dc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a7c0039e0 0x7f5a980cc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:26.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.799+0000 7f5a9dc00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a980cde60 0x7f5a980ce2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:26.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.799+0000 7f5a9dc00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a981f4910 0x7f5a981f4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:26.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.799+0000 7f5a9dc00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5a9806f9f0 con 0x7f5a7c0039e0 2024-09-16T15:48:26.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.799+0000 7f5a9dc00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5a9806f870 con 0x7f5a980cde60 2024-09-16T15:48:26.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.799+0000 7f5a9dc00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f5a980b2440 con 0x7f5a981f4910 2024-09-16T15:48:26.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.799+0000 7f5a96a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a7c0039e0 0x7f5a980cc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:26.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.799+0000 7f5a96a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a7c0039e0 0x7f5a980cc7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:42770/0 (socket says 172.21.15.151:42770) 2024-09-16T15:48:26.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.799+0000 7f5a96a00640 1 -- 172.21.15.151:0/3790238746 learned_addr learned my addr 172.21.15.151:0/3790238746 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:48:26.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.799+0000 7f5a96000640 1 --2- 172.21.15.151:0/3790238746 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a980cde60 0x7f5a980ce2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:26.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.799+0000 7f5a97400640 1 --2- 172.21.15.151:0/3790238746 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a981f4910 0x7f5a981f4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:26.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.799+0000 7f5a96a00640 1 -- 172.21.15.151:0/3790238746 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a981f4910 msgr2=0x7f5a981f4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:26.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.799+0000 7f5a96a00640 1 --2- 172.21.15.151:0/3790238746 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a981f4910 0x7f5a981f4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:26.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.799+0000 7f5a96a00640 1 -- 172.21.15.151:0/3790238746 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a980cde60 msgr2=0x7f5a980ce2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:26.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.799+0000 7f5a96a00640 1 --2- 172.21.15.151:0/3790238746 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a980cde60 0x7f5a980ce2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:26.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.799+0000 7f5a96a00640 1 -- 172.21.15.151:0/3790238746 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a980b2ef0 con 0x7f5a7c0039e0 2024-09-16T15:48:26.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.800+0000 7f5a96a00640 1 --2- 172.21.15.151:0/3790238746 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a7c0039e0 0x7f5a980cc7c0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f5a8802b810 tx=0x7f5a88033b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:26.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.800+0000 7f5a8ea00640 1 -- 172.21.15.151:0/3790238746 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a88042020 con 0x7f5a7c0039e0 2024-09-16T15:48:26.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.800+0000 7f5a8ea00640 1 -- 172.21.15.151:0/3790238746 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a8802bbe0 con 0x7f5a7c0039e0 2024-09-16T15:48:26.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.800+0000 7f5a8ea00640 1 -- 172.21.15.151:0/3790238746 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a8803d6b0 con 0x7f5a7c0039e0 2024-09-16T15:48:26.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.800+0000 7f5a9dc00640 1 -- 172.21.15.151:0/3790238746 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a980b31d0 con 0x7f5a7c0039e0 2024-09-16T15:48:26.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.800+0000 7f5a9dc00640 1 -- 172.21.15.151:0/3790238746 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a980b3790 con 0x7f5a7c0039e0 2024-09-16T15:48:26.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.801+0000 7f5a9dc00640 1 -- 172.21.15.151:0/3790238746 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a98074460 con 0x7f5a7c0039e0 2024-09-16T15:48:26.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.802+0000 7f5a8ea00640 1 -- 172.21.15.151:0/3790238746 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 30) ==== 102868+0+0 (secure 0 0 0) 0x7f5a88045020 con 0x7f5a7c0039e0 2024-09-16T15:48:26.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.803+0000 7f5a8ea00640 1 --2- 172.21.15.151:0/3790238746 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5a780799f0 0x7f5a7807beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:26.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.803+0000 7f5a8ea00640 1 -- 172.21.15.151:0/3790238746 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7f5a880bb190 con 0x7f5a7c0039e0 2024-09-16T15:48:26.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.804+0000 7f5a96000640 1 --2- 172.21.15.151:0/3790238746 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5a780799f0 0x7f5a7807beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:26.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.805+0000 7f5a96000640 1 --2- 172.21.15.151:0/3790238746 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5a780799f0 0x7f5a7807beb0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f5a84006930 tx=0x7f5a840068c0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:26.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.808+0000 7f5a8ea00640 1 -- 172.21.15.151:0/3790238746 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f5a8804c020 con 0x7f5a7c0039e0 2024-09-16T15:48:26.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.992+0000 7f5a9dc00640 1 -- 172.21.15.151:0/3790238746 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5a9809d8a0 con 0x7f5a780799f0 2024-09-16T15:48:26.999 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:26.997+0000 7f5a8ea00640 1 -- 172.21.15.151:0/3790238746 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247443 (secure 0 0 0) 0x7f5a9809d8a0 con 0x7f5a780799f0 2024-09-16T15:48:27.002 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:48:27.003 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-09-16T15:48:27.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.001+0000 7f5a9dc00640 1 -- 172.21.15.151:0/3790238746 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5a780799f0 msgr2=0x7f5a7807beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:27.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.001+0000 7f5a9dc00640 1 --2- 172.21.15.151:0/3790238746 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5a780799f0 0x7f5a7807beb0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f5a84006930 tx=0x7f5a840068c0 comp rx=0 tx=0).stop 2024-09-16T15:48:27.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.001+0000 7f5a9dc00640 1 -- 172.21.15.151:0/3790238746 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a7c0039e0 msgr2=0x7f5a980cc7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:27.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.001+0000 7f5a9dc00640 1 --2- 172.21.15.151:0/3790238746 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a7c0039e0 0x7f5a980cc7c0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f5a8802b810 tx=0x7f5a88033b20 comp rx=0 tx=0).stop 2024-09-16T15:48:27.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.002+0000 7f5a9dc00640 1 -- 172.21.15.151:0/3790238746 shutdown_connections 2024-09-16T15:48:27.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.002+0000 7f5a9dc00640 1 --2- 172.21.15.151:0/3790238746 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5a780799f0 0x7f5a7807beb0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:27.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.002+0000 7f5a9dc00640 1 --2- 172.21.15.151:0/3790238746 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a980cde60 0x7f5a980ce2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:27.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.002+0000 7f5a9dc00640 1 --2- 172.21.15.151:0/3790238746 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5a981f4910 0x7f5a981f4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:27.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.002+0000 7f5a9dc00640 1 --2- 172.21.15.151:0/3790238746 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a7c0039e0 0x7f5a980cc7c0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:27.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.002+0000 7f5a9dc00640 1 -- 172.21.15.151:0/3790238746 >> 172.21.15.151:0/3790238746 conn(0x7f5a980b1950 msgr2=0x7f5a9815da30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:27.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.002+0000 7f5a9dc00640 1 -- 172.21.15.151:0/3790238746 shutdown_connections 2024-09-16T15:48:27.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.002+0000 7f5a9dc00640 1 -- 172.21.15.151:0/3790238746 wait complete. 2024-09-16T15:48:27.020 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":362,"stamp":"2024-09-16T15:48:25.749544+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":108,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78,"ondisk_log_size":78,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":489404,"kb_used_data":9596,"kb_used_omap":12,"kb_used_meta":479603,"kb_avail":754485316,"statfs":{"total":773094113280,"available":772592963584,"internally_reserved":0,"allocated":9826304,"data_stored":4902599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12738,"internal_metadata":491114046},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.801460"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.964779+0000","last_change":"2024-09-16T15:48:20.964779+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.964779+0000","last_undegraded":"2024-09-16T15:48:20.964779+0000","last_fullsized":"2024-09-16T15:48:20.964779+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.956223+0000","last_change":"2024-09-16T15:48:20.956223+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.956223+0000","last_undegraded":"2024-09-16T15:48:20.956223+0000","last_fullsized":"2024-09-16T15:48:20.956223+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.966554+0000","last_change":"2024-09-16T15:48:20.966554+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.966554+0000","last_undegraded":"2024-09-16T15:48:20.966554+0000","last_fullsized":"2024-09-16T15:48:20.966554+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.958260+0000","last_change":"2024-09-16T15:48:20.958260+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.958260+0000","last_undegraded":"2024-09-16T15:48:20.958260+0000","last_fullsized":"2024-09-16T15:48:20.958260+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.961437+0000","last_change":"2024-09-16T15:48:20.961437+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.961437+0000","last_undegraded":"2024-09-16T15:48:20.961437+0000","last_fullsized":"2024-09-16T15:48:20.961437+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.964352+0000","last_change":"2024-09-16T15:48:20.964352+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.964352+0000","last_undegraded":"2024-09-16T15:48:20.964352+0000","last_fullsized":"2024-09-16T15:48:20.964352+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.956641+0000","last_change":"2024-09-16T15:48:20.956641+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.956641+0000","last_undegraded":"2024-09-16T15:48:20.956641+0000","last_fullsized":"2024-09-16T15:48:20.956641+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.961810+0000","last_change":"2024-09-16T15:48:20.961810+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.961810+0000","last_undegraded":"2024-09-16T15:48:20.961810+0000","last_fullsized":"2024-09-16T15:48:20.961810+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.964381+0000","last_change":"2024-09-16T15:48:20.964381+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.964381+0000","last_undegraded":"2024-09-16T15:48:20.964381+0000","last_fullsized":"2024-09-16T15:48:20.964381+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.964822+0000","last_change":"2024-09-16T15:48:20.964822+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.964822+0000","last_undegraded":"2024-09-16T15:48:20.964822+0000","last_fullsized":"2024-09-16T15:48:20.964822+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.962582+0000","last_change":"2024-09-16T15:48:20.962582+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.962582+0000","last_undegraded":"2024-09-16T15:48:20.962582+0000","last_fullsized":"2024-09-16T15:48:20.962582+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.963218+0000","last_change":"2024-09-16T15:48:20.963218+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.963218+0000","last_undegraded":"2024-09-16T15:48:20.963218+0000","last_fullsized":"2024-09-16T15:48:20.963218+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.959497+0000","last_change":"2024-09-16T15:48:20.959497+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.959497+0000","last_undegraded":"2024-09-16T15:48:20.959497+0000","last_fullsized":"2024-09-16T15:48:20.959497+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.351800+0000","last_change":"2024-09-16T15:48:21.991329+0000","last_active":"2024-09-16T15:48:23.351800+0000","last_peered":"2024-09-16T15:48:23.351800+0000","last_clean":"2024-09-16T15:48:23.351800+0000","last_became_active":"2024-09-16T15:48:21.990123+0000","last_became_peered":"2024-09-16T15:48:21.990123+0000","last_unstale":"2024-09-16T15:48:23.351800+0000","last_undegraded":"2024-09-16T15:48:23.351800+0000","last_fullsized":"2024-09-16T15:48:23.351800+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:01:55.713464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.956053+0000","last_change":"2024-09-16T15:48:20.956053+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.956053+0000","last_undegraded":"2024-09-16T15:48:20.956053+0000","last_fullsized":"2024-09-16T15:48:20.956053+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.964568+0000","last_change":"2024-09-16T15:48:20.964568+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.964568+0000","last_undegraded":"2024-09-16T15:48:20.964568+0000","last_fullsized":"2024-09-16T15:48:20.964568+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.960600+0000","last_change":"2024-09-16T15:48:20.960600+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.960600+0000","last_undegraded":"2024-09-16T15:48:20.960600+0000","last_fullsized":"2024-09-16T15:48:20.960600+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.965863+0000","last_change":"2024-09-16T15:48:20.965863+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.965863+0000","last_undegraded":"2024-09-16T15:48:20.965863+0000","last_fullsized":"2024-09-16T15:48:20.965863+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352178+0000","last_change":"2024-09-16T15:48:21.996135+0000","last_active":"2024-09-16T15:48:23.352178+0000","last_peered":"2024-09-16T15:48:23.352178+0000","last_clean":"2024-09-16T15:48:23.352178+0000","last_became_active":"2024-09-16T15:48:21.995671+0000","last_became_peered":"2024-09-16T15:48:21.995671+0000","last_unstale":"2024-09-16T15:48:23.352178+0000","last_undegraded":"2024-09-16T15:48:23.352178+0000","last_fullsized":"2024-09-16T15:48:23.352178+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:13:55.927762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352319+0000","last_change":"2024-09-16T15:48:21.994572+0000","last_active":"2024-09-16T15:48:23.352319+0000","last_peered":"2024-09-16T15:48:23.352319+0000","last_clean":"2024-09-16T15:48:23.352319+0000","last_became_active":"2024-09-16T15:48:21.994305+0000","last_became_peered":"2024-09-16T15:48:21.994305+0000","last_unstale":"2024-09-16T15:48:23.352319+0000","last_undegraded":"2024-09-16T15:48:23.352319+0000","last_fullsized":"2024-09-16T15:48:23.352319+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:27:24.090121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.958863+0000","last_change":"2024-09-16T15:48:20.958863+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.958863+0000","last_undegraded":"2024-09-16T15:48:20.958863+0000","last_fullsized":"2024-09-16T15:48:20.958863+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.351839+0000","last_change":"2024-09-16T15:48:21.988902+0000","last_active":"2024-09-16T15:48:23.351839+0000","last_peered":"2024-09-16T15:48:23.351839+0000","last_clean":"2024-09-16T15:48:23.351839+0000","last_became_active":"2024-09-16T15:48:21.988613+0000","last_became_peered":"2024-09-16T15:48:21.988613+0000","last_unstale":"2024-09-16T15:48:23.351839+0000","last_undegraded":"2024-09-16T15:48:23.351839+0000","last_fullsized":"2024-09-16T15:48:23.351839+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:05:00.466068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.962242+0000","last_change":"2024-09-16T15:48:20.962242+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.962242+0000","last_undegraded":"2024-09-16T15:48:20.962242+0000","last_fullsized":"2024-09-16T15:48:20.962242+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.962313+0000","last_change":"2024-09-16T15:48:20.962313+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.962313+0000","last_undegraded":"2024-09-16T15:48:20.962313+0000","last_fullsized":"2024-09-16T15:48:20.962313+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.966477+0000","last_change":"2024-09-16T15:48:20.966477+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.966477+0000","last_undegraded":"2024-09-16T15:48:20.966477+0000","last_fullsized":"2024-09-16T15:48:20.966477+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.957649+0000","last_change":"2024-09-16T15:48:20.957649+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.957649+0000","last_undegraded":"2024-09-16T15:48:20.957649+0000","last_fullsized":"2024-09-16T15:48:20.957649+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.958211+0000","last_change":"2024-09-16T15:48:20.958211+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.958211+0000","last_undegraded":"2024-09-16T15:48:20.958211+0000","last_fullsized":"2024-09-16T15:48:20.958211+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.963150+0000","last_change":"2024-09-16T15:48:20.963150+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.963150+0000","last_undegraded":"2024-09-16T15:48:20.963150+0000","last_fullsized":"2024-09-16T15:48:20.963150+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.957984+0000","last_change":"2024-09-16T15:48:20.957984+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.957984+0000","last_undegraded":"2024-09-16T15:48:20.957984+0000","last_fullsized":"2024-09-16T15:48:20.957984+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352274+0000","last_change":"2024-09-16T15:48:21.991963+0000","last_active":"2024-09-16T15:48:23.352274+0000","last_peered":"2024-09-16T15:48:23.352274+0000","last_clean":"2024-09-16T15:48:23.352274+0000","last_became_active":"2024-09-16T15:48:21.991698+0000","last_became_peered":"2024-09-16T15:48:21.991698+0000","last_unstale":"2024-09-16T15:48:23.352274+0000","last_undegraded":"2024-09-16T15:48:23.352274+0000","last_fullsized":"2024-09-16T15:48:23.352274+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:21:15.053411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.961609+0000","last_change":"2024-09-16T15:48:20.961609+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.961609+0000","last_undegraded":"2024-09-16T15:48:20.961609+0000","last_fullsized":"2024-09-16T15:48:20.961609+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.962070+0000","last_change":"2024-09-16T15:48:20.962070+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.962070+0000","last_undegraded":"2024-09-16T15:48:20.962070+0000","last_fullsized":"2024-09-16T15:48:20.962070+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.961053+0000","last_change":"2024-09-16T15:48:20.961053+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.961053+0000","last_undegraded":"2024-09-16T15:48:20.961053+0000","last_fullsized":"2024-09-16T15:48:20.961053+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.977633+0000","last_change":"2024-09-16T15:48:20.975503+0000","last_active":"2024-09-16T15:48:20.977633+0000","last_peered":"2024-09-16T15:48:20.977633+0000","last_clean":"2024-09-16T15:48:20.977633+0000","last_became_active":"2024-09-16T15:48:20.973819+0000","last_became_peered":"2024-09-16T15:48:20.973819+0000","last_unstale":"2024-09-16T15:48:20.977633+0000","last_undegraded":"2024-09-16T15:48:20.977633+0000","last_fullsized":"2024-09-16T15:48:20.977633+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:39:26.482165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.963995+0000","last_change":"2024-09-16T15:48:20.963995+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.963995+0000","last_undegraded":"2024-09-16T15:48:20.963995+0000","last_fullsized":"2024-09-16T15:48:20.963995+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.976680+0000","last_change":"2024-09-16T15:48:20.974951+0000","last_active":"2024-09-16T15:48:20.976680+0000","last_peered":"2024-09-16T15:48:20.976680+0000","last_clean":"2024-09-16T15:48:20.976680+0000","last_became_active":"2024-09-16T15:48:20.974688+0000","last_became_peered":"2024-09-16T15:48:20.974688+0000","last_unstale":"2024-09-16T15:48:20.976680+0000","last_undegraded":"2024-09-16T15:48:20.976680+0000","last_fullsized":"2024-09-16T15:48:20.976680+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:40:29.438048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.960641+0000","last_change":"2024-09-16T15:48:20.960641+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.960641+0000","last_undegraded":"2024-09-16T15:48:20.960641+0000","last_fullsized":"2024-09-16T15:48:20.960641+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.981008+0000","last_change":"2024-09-16T15:48:20.978653+0000","last_active":"2024-09-16T15:48:20.981008+0000","last_peered":"2024-09-16T15:48:20.981008+0000","last_clean":"2024-09-16T15:48:20.981008+0000","last_became_active":"2024-09-16T15:48:20.976364+0000","last_became_peered":"2024-09-16T15:48:20.976364+0000","last_unstale":"2024-09-16T15:48:20.981008+0000","last_undegraded":"2024-09-16T15:48:20.981008+0000","last_fullsized":"2024-09-16T15:48:20.981008+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:01:37.109321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.961183+0000","last_change":"2024-09-16T15:48:20.961183+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.961183+0000","last_undegraded":"2024-09-16T15:48:20.961183+0000","last_fullsized":"2024-09-16T15:48:20.961183+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.969107+0000","last_change":"2024-09-16T15:48:20.967236+0000","last_active":"2024-09-16T15:48:20.969107+0000","last_peered":"2024-09-16T15:48:20.969107+0000","last_clean":"2024-09-16T15:48:20.969107+0000","last_became_active":"2024-09-16T15:48:20.966954+0000","last_became_peered":"2024-09-16T15:48:20.966954+0000","last_unstale":"2024-09-16T15:48:20.969107+0000","last_undegraded":"2024-09-16T15:48:20.969107+0000","last_fullsized":"2024-09-16T15:48:20.969107+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:02:43.930006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.959326+0000","last_change":"2024-09-16T15:48:20.959326+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.959326+0000","last_undegraded":"2024-09-16T15:48:20.959326+0000","last_fullsized":"2024-09-16T15:48:20.959326+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.975825+0000","last_change":"2024-09-16T15:48:20.975041+0000","last_active":"2024-09-16T15:48:20.975825+0000","last_peered":"2024-09-16T15:48:20.975825+0000","last_clean":"2024-09-16T15:48:20.975825+0000","last_became_active":"2024-09-16T15:48:20.971719+0000","last_became_peered":"2024-09-16T15:48:20.971719+0000","last_unstale":"2024-09-16T15:48:20.975825+0000","last_undegraded":"2024-09-16T15:48:20.975825+0000","last_fullsized":"2024-09-16T15:48:20.975825+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:32:44.623249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.962285+0000","last_change":"2024-09-16T15:48:20.962285+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.962285+0000","last_undegraded":"2024-09-16T15:48:20.962285+0000","last_fullsized":"2024-09-16T15:48:20.962285+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.976080+0000","last_change":"2024-09-16T15:48:20.971997+0000","last_active":"2024-09-16T15:48:20.976080+0000","last_peered":"2024-09-16T15:48:20.976080+0000","last_clean":"2024-09-16T15:48:20.976080+0000","last_became_active":"2024-09-16T15:48:20.971230+0000","last_became_peered":"2024-09-16T15:48:20.971230+0000","last_unstale":"2024-09-16T15:48:20.976080+0000","last_undegraded":"2024-09-16T15:48:20.976080+0000","last_fullsized":"2024-09-16T15:48:20.976080+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:20:05.425588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.962335+0000","last_change":"2024-09-16T15:48:20.962335+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.962335+0000","last_undegraded":"2024-09-16T15:48:20.962335+0000","last_fullsized":"2024-09-16T15:48:20.962335+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.351663+0000","last_change":"2024-09-16T15:48:20.977058+0000","last_active":"2024-09-16T15:48:23.351663+0000","last_peered":"2024-09-16T15:48:23.351663+0000","last_clean":"2024-09-16T15:48:23.351663+0000","last_became_active":"2024-09-16T15:48:20.975591+0000","last_became_peered":"2024-09-16T15:48:20.975591+0000","last_unstale":"2024-09-16T15:48:23.351663+0000","last_undegraded":"2024-09-16T15:48:23.351663+0000","last_fullsized":"2024-09-16T15:48:23.351663+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:58:51.576740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.351992+0000","last_change":"2024-09-16T15:48:21.987074+0000","last_active":"2024-09-16T15:48:23.351992+0000","last_peered":"2024-09-16T15:48:23.351992+0000","last_clean":"2024-09-16T15:48:23.351992+0000","last_became_active":"2024-09-16T15:48:21.986779+0000","last_became_peered":"2024-09-16T15:48:21.986779+0000","last_unstale":"2024-09-16T15:48:23.351992+0000","last_undegraded":"2024-09-16T15:48:23.351992+0000","last_fullsized":"2024-09-16T15:48:23.351992+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:57:02.685046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.351536+0000","last_change":"2024-09-16T15:48:20.977637+0000","last_active":"2024-09-16T15:48:23.351536+0000","last_peered":"2024-09-16T15:48:23.351536+0000","last_clean":"2024-09-16T15:48:23.351536+0000","last_became_active":"2024-09-16T15:48:20.977367+0000","last_became_peered":"2024-09-16T15:48:20.977367+0000","last_unstale":"2024-09-16T15:48:23.351536+0000","last_undegraded":"2024-09-16T15:48:23.351536+0000","last_fullsized":"2024-09-16T15:48:23.351536+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:57:12.992776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352256+0000","last_change":"2024-09-16T15:48:21.990047+0000","last_active":"2024-09-16T15:48:23.352256+0000","last_peered":"2024-09-16T15:48:23.352256+0000","last_clean":"2024-09-16T15:48:23.352256+0000","last_became_active":"2024-09-16T15:48:21.989735+0000","last_became_peered":"2024-09-16T15:48:21.989735+0000","last_unstale":"2024-09-16T15:48:23.352256+0000","last_undegraded":"2024-09-16T15:48:23.352256+0000","last_fullsized":"2024-09-16T15:48:23.352256+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:50:37.046745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.971713+0000","last_change":"2024-09-16T15:48:20.969036+0000","last_active":"2024-09-16T15:48:20.971713+0000","last_peered":"2024-09-16T15:48:20.971713+0000","last_clean":"2024-09-16T15:48:20.971713+0000","last_became_active":"2024-09-16T15:48:20.968771+0000","last_became_peered":"2024-09-16T15:48:20.968771+0000","last_unstale":"2024-09-16T15:48:20.971713+0000","last_undegraded":"2024-09-16T15:48:20.971713+0000","last_fullsized":"2024-09-16T15:48:20.971713+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:27:21.822056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.962027+0000","last_change":"2024-09-16T15:48:20.962027+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.962027+0000","last_undegraded":"2024-09-16T15:48:20.962027+0000","last_fullsized":"2024-09-16T15:48:20.962027+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.969598+0000","last_change":"2024-09-16T15:48:20.968246+0000","last_active":"2024-09-16T15:48:20.969598+0000","last_peered":"2024-09-16T15:48:20.969598+0000","last_clean":"2024-09-16T15:48:20.969598+0000","last_became_active":"2024-09-16T15:48:20.967963+0000","last_became_peered":"2024-09-16T15:48:20.967963+0000","last_unstale":"2024-09-16T15:48:20.969598+0000","last_undegraded":"2024-09-16T15:48:20.969598+0000","last_fullsized":"2024-09-16T15:48:20.969598+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:23:26.528353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.967777+0000","last_change":"2024-09-16T15:48:20.967777+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.967777+0000","last_undegraded":"2024-09-16T15:48:20.967777+0000","last_fullsized":"2024-09-16T15:48:20.967777+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.977951+0000","last_change":"2024-09-16T15:48:20.975763+0000","last_active":"2024-09-16T15:48:20.977951+0000","last_peered":"2024-09-16T15:48:20.977951+0000","last_clean":"2024-09-16T15:48:20.977951+0000","last_became_active":"2024-09-16T15:48:20.974952+0000","last_became_peered":"2024-09-16T15:48:20.974952+0000","last_unstale":"2024-09-16T15:48:20.977951+0000","last_undegraded":"2024-09-16T15:48:20.977951+0000","last_fullsized":"2024-09-16T15:48:20.977951+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:37:00.965428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.958573+0000","last_change":"2024-09-16T15:48:20.958573+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.958573+0000","last_undegraded":"2024-09-16T15:48:20.958573+0000","last_fullsized":"2024-09-16T15:48:20.958573+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.974235+0000","last_change":"2024-09-16T15:48:20.972430+0000","last_active":"2024-09-16T15:48:20.974235+0000","last_peered":"2024-09-16T15:48:20.974235+0000","last_clean":"2024-09-16T15:48:20.974235+0000","last_became_active":"2024-09-16T15:48:20.972159+0000","last_became_peered":"2024-09-16T15:48:20.972159+0000","last_unstale":"2024-09-16T15:48:20.974235+0000","last_undegraded":"2024-09-16T15:48:20.974235+0000","last_fullsized":"2024-09-16T15:48:20.974235+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:02:58.958514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.965797+0000","last_change":"2024-09-16T15:48:20.965797+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.965797+0000","last_undegraded":"2024-09-16T15:48:20.965797+0000","last_fullsized":"2024-09-16T15:48:20.965797+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.969032+0000","last_change":"2024-09-16T15:48:20.965556+0000","last_active":"2024-09-16T15:48:20.969032+0000","last_peered":"2024-09-16T15:48:20.969032+0000","last_clean":"2024-09-16T15:48:20.969032+0000","last_became_active":"2024-09-16T15:48:20.965196+0000","last_became_peered":"2024-09-16T15:48:20.965196+0000","last_unstale":"2024-09-16T15:48:20.969032+0000","last_undegraded":"2024-09-16T15:48:20.969032+0000","last_fullsized":"2024-09-16T15:48:20.969032+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:57:22.438388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.959493+0000","last_change":"2024-09-16T15:48:20.959493+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.959493+0000","last_undegraded":"2024-09-16T15:48:20.959493+0000","last_fullsized":"2024-09-16T15:48:20.959493+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.970792+0000","last_change":"2024-09-16T15:48:20.969210+0000","last_active":"2024-09-16T15:48:20.970792+0000","last_peered":"2024-09-16T15:48:20.970792+0000","last_clean":"2024-09-16T15:48:20.970792+0000","last_became_active":"2024-09-16T15:48:20.968946+0000","last_became_peered":"2024-09-16T15:48:20.968946+0000","last_unstale":"2024-09-16T15:48:20.970792+0000","last_undegraded":"2024-09-16T15:48:20.970792+0000","last_fullsized":"2024-09-16T15:48:20.970792+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:43:02.173876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352100+0000","last_change":"2024-09-16T15:48:21.997110+0000","last_active":"2024-09-16T15:48:23.352100+0000","last_peered":"2024-09-16T15:48:23.352100+0000","last_clean":"2024-09-16T15:48:23.352100+0000","last_became_active":"2024-09-16T15:48:21.996855+0000","last_became_peered":"2024-09-16T15:48:21.996855+0000","last_unstale":"2024-09-16T15:48:23.352100+0000","last_undegraded":"2024-09-16T15:48:23.352100+0000","last_fullsized":"2024-09-16T15:48:23.352100+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:44:31.136306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.970157+0000","last_change":"2024-09-16T15:48:20.969040+0000","last_active":"2024-09-16T15:48:20.970157+0000","last_peered":"2024-09-16T15:48:20.970157+0000","last_clean":"2024-09-16T15:48:20.970157+0000","last_became_active":"2024-09-16T15:48:20.968541+0000","last_became_peered":"2024-09-16T15:48:20.968541+0000","last_unstale":"2024-09-16T15:48:20.970157+0000","last_undegraded":"2024-09-16T15:48:20.970157+0000","last_fullsized":"2024-09-16T15:48:20.970157+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:02:23.941065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.964413+0000","last_change":"2024-09-16T15:48:20.964413+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.964413+0000","last_undegraded":"2024-09-16T15:48:20.964413+0000","last_fullsized":"2024-09-16T15:48:20.964413+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.979658+0000","last_change":"2024-09-16T15:48:20.975861+0000","last_active":"2024-09-16T15:48:20.979658+0000","last_peered":"2024-09-16T15:48:20.979658+0000","last_clean":"2024-09-16T15:48:20.979658+0000","last_became_active":"2024-09-16T15:48:20.974746+0000","last_became_peered":"2024-09-16T15:48:20.974746+0000","last_unstale":"2024-09-16T15:48:20.979658+0000","last_undegraded":"2024-09-16T15:48:20.979658+0000","last_fullsized":"2024-09-16T15:48:20.979658+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:13.847700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.960635+0000","last_change":"2024-09-16T15:48:20.960635+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.960635+0000","last_undegraded":"2024-09-16T15:48:20.960635+0000","last_fullsized":"2024-09-16T15:48:20.960635+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.977666+0000","last_change":"2024-09-16T15:48:20.975777+0000","last_active":"2024-09-16T15:48:20.977666+0000","last_peered":"2024-09-16T15:48:20.977666+0000","last_clean":"2024-09-16T15:48:20.977666+0000","last_became_active":"2024-09-16T15:48:20.974162+0000","last_became_peered":"2024-09-16T15:48:20.974162+0000","last_unstale":"2024-09-16T15:48:20.977666+0000","last_undegraded":"2024-09-16T15:48:20.977666+0000","last_fullsized":"2024-09-16T15:48:20.977666+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:34:52.666788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.961693+0000","last_change":"2024-09-16T15:48:20.961693+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.961693+0000","last_undegraded":"2024-09-16T15:48:20.961693+0000","last_fullsized":"2024-09-16T15:48:20.961693+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"90'2","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352068+0000","last_change":"2024-09-16T15:48:20.975158+0000","last_active":"2024-09-16T15:48:23.352068+0000","last_peered":"2024-09-16T15:48:23.352068+0000","last_clean":"2024-09-16T15:48:23.352068+0000","last_became_active":"2024-09-16T15:48:20.974854+0000","last_became_peered":"2024-09-16T15:48:20.974854+0000","last_unstale":"2024-09-16T15:48:23.352068+0000","last_undegraded":"2024-09-16T15:48:23.352068+0000","last_fullsized":"2024-09-16T15:48:23.352068+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:41:44.873886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.958552+0000","last_change":"2024-09-16T15:48:20.958552+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.958552+0000","last_undegraded":"2024-09-16T15:48:20.958552+0000","last_fullsized":"2024-09-16T15:48:20.958552+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352234+0000","last_change":"2024-09-16T15:48:20.978031+0000","last_active":"2024-09-16T15:48:23.352234+0000","last_peered":"2024-09-16T15:48:23.352234+0000","last_clean":"2024-09-16T15:48:23.352234+0000","last_became_active":"2024-09-16T15:48:20.976491+0000","last_became_peered":"2024-09-16T15:48:20.976491+0000","last_unstale":"2024-09-16T15:48:23.352234+0000","last_undegraded":"2024-09-16T15:48:23.352234+0000","last_fullsized":"2024-09-16T15:48:23.352234+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:57.979104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.956433+0000","last_change":"2024-09-16T15:48:20.956433+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.956433+0000","last_undegraded":"2024-09-16T15:48:20.956433+0000","last_fullsized":"2024-09-16T15:48:20.956433+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.b","version":"90'2","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.351800+0000","last_change":"2024-09-16T15:48:20.976944+0000","last_active":"2024-09-16T15:48:23.351800+0000","last_peered":"2024-09-16T15:48:23.351800+0000","last_clean":"2024-09-16T15:48:23.351800+0000","last_became_active":"2024-09-16T15:48:20.976649+0000","last_became_peered":"2024-09-16T15:48:20.976649+0000","last_unstale":"2024-09-16T15:48:23.351800+0000","last_undegraded":"2024-09-16T15:48:23.351800+0000","last_fullsized":"2024-09-16T15:48:23.351800+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:33:01.893406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.957113+0000","last_change":"2024-09-16T15:48:20.957113+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.957113+0000","last_undegraded":"2024-09-16T15:48:20.957113+0000","last_fullsized":"2024-09-16T15:48:20.957113+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.973153+0000","last_change":"2024-09-16T15:48:20.970601+0000","last_active":"2024-09-16T15:48:20.973153+0000","last_peered":"2024-09-16T15:48:20.973153+0000","last_clean":"2024-09-16T15:48:20.973153+0000","last_became_active":"2024-09-16T15:48:20.970272+0000","last_became_peered":"2024-09-16T15:48:20.970272+0000","last_unstale":"2024-09-16T15:48:20.973153+0000","last_undegraded":"2024-09-16T15:48:20.973153+0000","last_fullsized":"2024-09-16T15:48:20.973153+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:31:13.471665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.957294+0000","last_change":"2024-09-16T15:48:20.957294+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.957294+0000","last_undegraded":"2024-09-16T15:48:20.957294+0000","last_fullsized":"2024-09-16T15:48:20.957294+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.973815+0000","last_change":"2024-09-16T15:48:20.971710+0000","last_active":"2024-09-16T15:48:20.973815+0000","last_peered":"2024-09-16T15:48:20.973815+0000","last_clean":"2024-09-16T15:48:20.973815+0000","last_became_active":"2024-09-16T15:48:20.971360+0000","last_became_peered":"2024-09-16T15:48:20.971360+0000","last_unstale":"2024-09-16T15:48:20.973815+0000","last_undegraded":"2024-09-16T15:48:20.973815+0000","last_fullsized":"2024-09-16T15:48:20.973815+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:53:28.706338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.959074+0000","last_change":"2024-09-16T15:48:20.959074+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.959074+0000","last_undegraded":"2024-09-16T15:48:20.959074+0000","last_fullsized":"2024-09-16T15:48:20.959074+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.351750+0000","last_change":"2024-09-16T15:48:20.980185+0000","last_active":"2024-09-16T15:48:23.351750+0000","last_peered":"2024-09-16T15:48:23.351750+0000","last_clean":"2024-09-16T15:48:23.351750+0000","last_became_active":"2024-09-16T15:48:20.979875+0000","last_became_peered":"2024-09-16T15:48:20.979875+0000","last_unstale":"2024-09-16T15:48:23.351750+0000","last_undegraded":"2024-09-16T15:48:23.351750+0000","last_fullsized":"2024-09-16T15:48:23.351750+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:56:29.587401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.957180+0000","last_change":"2024-09-16T15:48:20.957180+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.957180+0000","last_undegraded":"2024-09-16T15:48:20.957180+0000","last_fullsized":"2024-09-16T15:48:20.957180+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"90'2","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352056+0000","last_change":"2024-09-16T15:48:20.978563+0000","last_active":"2024-09-16T15:48:23.352056+0000","last_peered":"2024-09-16T15:48:23.352056+0000","last_clean":"2024-09-16T15:48:23.352056+0000","last_became_active":"2024-09-16T15:48:20.978188+0000","last_became_peered":"2024-09-16T15:48:20.978188+0000","last_unstale":"2024-09-16T15:48:23.352056+0000","last_undegraded":"2024-09-16T15:48:23.352056+0000","last_fullsized":"2024-09-16T15:48:23.352056+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:28:27.584567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.957910+0000","last_change":"2024-09-16T15:48:20.957910+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.957910+0000","last_undegraded":"2024-09-16T15:48:20.957910+0000","last_fullsized":"2024-09-16T15:48:20.957910+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.979361+0000","last_change":"2024-09-16T15:48:20.975435+0000","last_active":"2024-09-16T15:48:20.979361+0000","last_peered":"2024-09-16T15:48:20.979361+0000","last_clean":"2024-09-16T15:48:20.979361+0000","last_became_active":"2024-09-16T15:48:20.975091+0000","last_became_peered":"2024-09-16T15:48:20.975091+0000","last_unstale":"2024-09-16T15:48:20.979361+0000","last_undegraded":"2024-09-16T15:48:20.979361+0000","last_fullsized":"2024-09-16T15:48:20.979361+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:43:06.025743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.957792+0000","last_change":"2024-09-16T15:48:20.957792+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.957792+0000","last_undegraded":"2024-09-16T15:48:20.957792+0000","last_fullsized":"2024-09-16T15:48:20.957792+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.975126+0000","last_change":"2024-09-16T15:48:20.974118+0000","last_active":"2024-09-16T15:48:20.975126+0000","last_peered":"2024-09-16T15:48:20.975126+0000","last_clean":"2024-09-16T15:48:20.975126+0000","last_became_active":"2024-09-16T15:48:20.973841+0000","last_became_peered":"2024-09-16T15:48:20.973841+0000","last_unstale":"2024-09-16T15:48:20.975126+0000","last_undegraded":"2024-09-16T15:48:20.975126+0000","last_fullsized":"2024-09-16T15:48:20.975126+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:09:11.275872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.960754+0000","last_change":"2024-09-16T15:48:20.960754+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.960754+0000","last_undegraded":"2024-09-16T15:48:20.960754+0000","last_fullsized":"2024-09-16T15:48:20.960754+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.976606+0000","last_change":"2024-09-16T15:48:20.974113+0000","last_active":"2024-09-16T15:48:20.976606+0000","last_peered":"2024-09-16T15:48:20.976606+0000","last_clean":"2024-09-16T15:48:20.976606+0000","last_became_active":"2024-09-16T15:48:20.973851+0000","last_became_peered":"2024-09-16T15:48:20.973851+0000","last_unstale":"2024-09-16T15:48:20.976606+0000","last_undegraded":"2024-09-16T15:48:20.976606+0000","last_fullsized":"2024-09-16T15:48:20.976606+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:14:35.885189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.956848+0000","last_change":"2024-09-16T15:48:20.956848+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.956848+0000","last_undegraded":"2024-09-16T15:48:20.956848+0000","last_fullsized":"2024-09-16T15:48:20.956848+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.972184+0000","last_change":"2024-09-16T15:48:20.968528+0000","last_active":"2024-09-16T15:48:20.972184+0000","last_peered":"2024-09-16T15:48:20.972184+0000","last_clean":"2024-09-16T15:48:20.972184+0000","last_became_active":"2024-09-16T15:48:20.967944+0000","last_became_peered":"2024-09-16T15:48:20.967944+0000","last_unstale":"2024-09-16T15:48:20.972184+0000","last_undegraded":"2024-09-16T15:48:20.972184+0000","last_fullsized":"2024-09-16T15:48:20.972184+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:24:23.675488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.962709+0000","last_change":"2024-09-16T15:48:20.962709+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.962709+0000","last_undegraded":"2024-09-16T15:48:20.962709+0000","last_fullsized":"2024-09-16T15:48:20.962709+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.966248+0000","last_change":"2024-09-16T15:48:20.963312+0000","last_active":"2024-09-16T15:48:20.966248+0000","last_peered":"2024-09-16T15:48:20.966248+0000","last_clean":"2024-09-16T15:48:20.966248+0000","last_became_active":"2024-09-16T15:48:20.963034+0000","last_became_peered":"2024-09-16T15:48:20.963034+0000","last_unstale":"2024-09-16T15:48:20.966248+0000","last_undegraded":"2024-09-16T15:48:20.966248+0000","last_fullsized":"2024-09-16T15:48:20.966248+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:39:53.503131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.956619+0000","last_change":"2024-09-16T15:48:20.956619+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.956619+0000","last_undegraded":"2024-09-16T15:48:20.956619+0000","last_fullsized":"2024-09-16T15:48:20.956619+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.978757+0000","last_change":"2024-09-16T15:48:20.978028+0000","last_active":"2024-09-16T15:48:20.978757+0000","last_peered":"2024-09-16T15:48:20.978757+0000","last_clean":"2024-09-16T15:48:20.978757+0000","last_became_active":"2024-09-16T15:48:20.977556+0000","last_became_peered":"2024-09-16T15:48:20.977556+0000","last_unstale":"2024-09-16T15:48:20.978757+0000","last_undegraded":"2024-09-16T15:48:20.978757+0000","last_fullsized":"2024-09-16T15:48:20.978757+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:34:32.400938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.958982+0000","last_change":"2024-09-16T15:48:20.958982+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.958982+0000","last_undegraded":"2024-09-16T15:48:20.958982+0000","last_fullsized":"2024-09-16T15:48:20.958982+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.976881+0000","last_change":"2024-09-16T15:48:20.976801+0000","last_active":"2024-09-16T15:48:20.976881+0000","last_peered":"2024-09-16T15:48:20.976881+0000","last_clean":"2024-09-16T15:48:20.976881+0000","last_became_active":"2024-09-16T15:48:20.975672+0000","last_became_peered":"2024-09-16T15:48:20.975672+0000","last_unstale":"2024-09-16T15:48:20.976881+0000","last_undegraded":"2024-09-16T15:48:20.976881+0000","last_fullsized":"2024-09-16T15:48:20.976881+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:59:53.683980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-09-16T15:48:20.965048+0000","last_change":"2024-09-16T15:48:20.965048+0000","last_active":"2024-09-16T15:48:20.946152+0000","last_peered":"2024-09-16T15:48:20.946152+0000","last_clean":"2024-09-16T15:48:20.946152+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:48:20.965048+0000","last_undegraded":"2024-09-16T15:48:20.965048+0000","last_fullsized":"2024-09-16T15:48:20.965048+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":10,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.977310+0000","last_change":"2024-09-16T15:48:20.974835+0000","last_active":"2024-09-16T15:48:20.977310+0000","last_peered":"2024-09-16T15:48:20.977310+0000","last_clean":"2024-09-16T15:48:20.977310+0000","last_became_active":"2024-09-16T15:48:20.974577+0000","last_became_peered":"2024-09-16T15:48:20.974577+0000","last_unstale":"2024-09-16T15:48:20.977310+0000","last_undegraded":"2024-09-16T15:48:20.977310+0000","last_fullsized":"2024-09-16T15:48:20.977310+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:29:48.935182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":238,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-16T15:48:20.955843+0000","last_change":"2024-09-16T15:39:12.698813+0000","last_active":"2024-09-16T15:48:20.955843+0000","last_peered":"2024-09-16T15:48:20.955843+0000","last_clean":"2024-09-16T15:48:20.955843+0000","last_became_active":"2024-09-16T15:39:12.697695+0000","last_became_peered":"2024-09-16T15:39:12.697695+0000","last_unstale":"2024-09-16T15:48:20.955843+0000","last_undegraded":"2024-09-16T15:48:20.955843+0000","last_fullsized":"2024-09-16T15:48:20.955843+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_clean_scrub_stamp":"2024-09-16T15:39:11.684967+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:02.061081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509555,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64108,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307732,"statfs":{"total":96636764160,"available":96571117568,"internally_reserved":0,"allocated":1073152,"data_stored":461993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":64551368},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509554,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63352,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94308488,"statfs":{"total":96636764160,"available":96571891712,"internally_reserved":0,"allocated":1667072,"data_stored":1052361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":63175111},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509553,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64124,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307716,"statfs":{"total":96636764160,"available":96571101184,"internally_reserved":0,"allocated":1085440,"data_stored":468324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509553,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58616,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313224,"statfs":{"total":96636764160,"available":96576741376,"internally_reserved":0,"allocated":1073152,"data_stored":461993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":58915273},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509553,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58612,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313228,"statfs":{"total":96636764160,"available":96576745472,"internally_reserved":0,"allocated":1073152,"data_stored":461993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509554,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59184,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312656,"statfs":{"total":96636764160,"available":96576159744,"internally_reserved":0,"allocated":1667072,"data_stored":1052361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509552,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58604,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313236,"statfs":{"total":96636764160,"available":96576753664,"internally_reserved":0,"allocated":1073152,"data_stored":461993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542260,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62804,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94309036,"statfs":{"total":96636764160,"available":96572452864,"internally_reserved":0,"allocated":1114112,"data_stored":481581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":63175116},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:48:27.024 INFO:tasks.ceph.ceph_manager.ceph:PG 13.3f is not active+clean 2024-09-16T15:48:27.024 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:48:20.964779+0000', 'last_change': '2024-09-16T15:48:20.964779+0000', 'last_active': '2024-09-16T15:48:20.946152+0000', 'last_peered': '2024-09-16T15:48:20.946152+0000', 'last_clean': '2024-09-16T15:48:20.946152+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:48:20.964779+0000', 'last_undegraded': '2024-09-16T15:48:20.964779+0000', 'last_fullsized': '2024-09-16T15:48:20.964779+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_clean_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-16T15:48:27.024 INFO:tasks.ceph.ceph_manager.ceph:PG 13.3e is not active+clean 2024-09-16T15:48:27.024 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:48:20.956223+0000', 'last_change': '2024-09-16T15:48:20.956223+0000', 'last_active': '2024-09-16T15:48:20.946152+0000', 'last_peered': '2024-09-16T15:48:20.946152+0000', 'last_clean': '2024-09-16T15:48:20.946152+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:48:20.956223+0000', 'last_undegraded': '2024-09-16T15:48:20.956223+0000', 'last_fullsized': '2024-09-16T15:48:20.956223+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_clean_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-16T15:48:27.024 INFO:tasks.ceph.ceph_manager.ceph:PG 13.3d is not active+clean 2024-09-16T15:48:27.024 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:48:20.966554+0000', 'last_change': '2024-09-16T15:48:20.966554+0000', 'last_active': '2024-09-16T15:48:20.946152+0000', 'last_peered': '2024-09-16T15:48:20.946152+0000', 'last_clean': '2024-09-16T15:48:20.946152+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:48:20.966554+0000', 'last_undegraded': '2024-09-16T15:48:20.966554+0000', 'last_fullsized': '2024-09-16T15:48:20.966554+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_clean_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T15:48:27.024 INFO:tasks.ceph.ceph_manager.ceph:PG 13.3c is not active+clean 2024-09-16T15:48:27.024 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:48:20.958260+0000', 'last_change': '2024-09-16T15:48:20.958260+0000', 'last_active': '2024-09-16T15:48:20.946152+0000', 'last_peered': '2024-09-16T15:48:20.946152+0000', 'last_clean': '2024-09-16T15:48:20.946152+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:48:20.958260+0000', 'last_undegraded': '2024-09-16T15:48:20.958260+0000', 'last_fullsized': '2024-09-16T15:48:20.958260+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_clean_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-16T15:48:27.025 INFO:tasks.ceph.ceph_manager.ceph:PG 13.3b is not active+clean 2024-09-16T15:48:27.025 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:48:20.961437+0000', 'last_change': '2024-09-16T15:48:20.961437+0000', 'last_active': '2024-09-16T15:48:20.946152+0000', 'last_peered': '2024-09-16T15:48:20.946152+0000', 'last_clean': '2024-09-16T15:48:20.946152+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:48:20.961437+0000', 'last_undegraded': '2024-09-16T15:48:20.961437+0000', 'last_fullsized': '2024-09-16T15:48:20.961437+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_clean_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T15:48:27.025 INFO:tasks.ceph.ceph_manager.ceph:PG 13.3a is not active+clean 2024-09-16T15:48:27.025 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:48:20.964352+0000', 'last_change': '2024-09-16T15:48:20.964352+0000', 'last_active': '2024-09-16T15:48:20.946152+0000', 'last_peered': '2024-09-16T15:48:20.946152+0000', 'last_clean': '2024-09-16T15:48:20.946152+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:48:20.964352+0000', 'last_undegraded': '2024-09-16T15:48:20.964352+0000', 'last_fullsized': '2024-09-16T15:48:20.964352+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_clean_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T15:48:27.025 INFO:tasks.ceph.ceph_manager.ceph:PG 13.39 is not active+clean 2024-09-16T15:48:27.025 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:48:20.956641+0000', 'last_change': '2024-09-16T15:48:20.956641+0000', 'last_active': '2024-09-16T15:48:20.946152+0000', 'last_peered': '2024-09-16T15:48:20.946152+0000', 'last_clean': '2024-09-16T15:48:20.946152+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:48:20.956641+0000', 'last_undegraded': '2024-09-16T15:48:20.956641+0000', 'last_fullsized': '2024-09-16T15:48:20.956641+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_clean_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-16T15:48:27.025 INFO:tasks.ceph.ceph_manager.ceph:PG 13.38 is not active+clean 2024-09-16T15:48:27.025 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:48:20.961810+0000', 'last_change': '2024-09-16T15:48:20.961810+0000', 'last_active': '2024-09-16T15:48:20.946152+0000', 'last_peered': '2024-09-16T15:48:20.946152+0000', 'last_clean': '2024-09-16T15:48:20.946152+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:48:20.961810+0000', 'last_undegraded': '2024-09-16T15:48:20.961810+0000', 'last_fullsized': '2024-09-16T15:48:20.961810+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_clean_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T15:48:27.025 INFO:tasks.ceph.ceph_manager.ceph:PG 13.37 is not active+clean 2024-09-16T15:48:27.025 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:48:20.964381+0000', 'last_change': '2024-09-16T15:48:20.964381+0000', 'last_active': '2024-09-16T15:48:20.946152+0000', 'last_peered': '2024-09-16T15:48:20.946152+0000', 'last_clean': '2024-09-16T15:48:20.946152+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:48:20.964381+0000', 'last_undegraded': '2024-09-16T15:48:20.964381+0000', 'last_fullsized': '2024-09-16T15:48:20.964381+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_clean_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-16T15:48:27.026 INFO:tasks.ceph.ceph_manager.ceph:PG 13.36 is not active+clean 2024-09-16T15:48:27.026 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:48:20.964822+0000', 'last_change': '2024-09-16T15:48:20.964822+0000', 'last_active': '2024-09-16T15:48:20.946152+0000', 'last_peered': '2024-09-16T15:48:20.946152+0000', 'last_clean': '2024-09-16T15:48:20.946152+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:48:20.964822+0000', 'last_undegraded': '2024-09-16T15:48:20.964822+0000', 'last_fullsized': '2024-09-16T15:48:20.964822+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_clean_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T15:48:27.026 INFO:tasks.ceph.ceph_manager.ceph:PG 13.35 is not active+clean 2024-09-16T15:48:27.026 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:48:20.962582+0000', 'last_change': '2024-09-16T15:48:20.962582+0000', 'last_active': '2024-09-16T15:48:20.946152+0000', 'last_peered': '2024-09-16T15:48:20.946152+0000', 'last_clean': '2024-09-16T15:48:20.946152+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:48:20.962582+0000', 'last_undegraded': '2024-09-16T15:48:20.962582+0000', 'last_fullsized': '2024-09-16T15:48:20.962582+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_clean_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-16T15:48:27.026 INFO:tasks.ceph.ceph_manager.ceph:PG 13.34 is not active+clean 2024-09-16T15:48:27.026 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:48:20.963218+0000', 'last_change': '2024-09-16T15:48:20.963218+0000', 'last_active': '2024-09-16T15:48:20.946152+0000', 'last_peered': '2024-09-16T15:48:20.946152+0000', 'last_clean': '2024-09-16T15:48:20.946152+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:48:20.963218+0000', 'last_undegraded': '2024-09-16T15:48:20.963218+0000', 'last_fullsized': '2024-09-16T15:48:20.963218+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_clean_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T15:48:27.026 INFO:tasks.ceph.ceph_manager.ceph:PG 13.33 is not active+clean 2024-09-16T15:48:27.026 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:48:20.959497+0000', 'last_change': '2024-09-16T15:48:20.959497+0000', 'last_active': '2024-09-16T15:48:20.946152+0000', 'last_peered': '2024-09-16T15:48:20.946152+0000', 'last_clean': '2024-09-16T15:48:20.946152+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:48:20.959497+0000', 'last_undegraded': '2024-09-16T15:48:20.959497+0000', 'last_fullsized': '2024-09-16T15:48:20.959497+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_clean_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T15:48:27.026 INFO:tasks.ceph.ceph_manager.ceph:PG 13.31 is not active+clean 2024-09-16T15:48:27.026 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:48:20.956053+0000', 'last_change': '2024-09-16T15:48:20.956053+0000', 'last_active': '2024-09-16T15:48:20.946152+0000', 'last_peered': '2024-09-16T15:48:20.946152+0000', 'last_clean': '2024-09-16T15:48:20.946152+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:48:20.956053+0000', 'last_undegraded': '2024-09-16T15:48:20.956053+0000', 'last_fullsized': '2024-09-16T15:48:20.956053+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_clean_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-16T15:48:27.026 INFO:tasks.ceph.ceph_manager.ceph:PG 13.30 is not active+clean 2024-09-16T15:48:27.026 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:48:20.964568+0000', 'last_change': '2024-09-16T15:48:20.964568+0000', 'last_active': '2024-09-16T15:48:20.946152+0000', 'last_peered': '2024-09-16T15:48:20.946152+0000', 'last_clean': '2024-09-16T15:48:20.946152+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:48:20.964568+0000', 'last_undegraded': '2024-09-16T15:48:20.964568+0000', 'last_fullsized': '2024-09-16T15:48:20.964568+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_clean_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T15:48:27.026 INFO:tasks.ceph.ceph_manager.ceph:PG 13.2f is not active+clean 2024-09-16T15:48:27.027 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:48:20.960600+0000', 'last_change': '2024-09-16T15:48:20.960600+0000', 'last_active': '2024-09-16T15:48:20.946152+0000', 'last_peered': '2024-09-16T15:48:20.946152+0000', 'last_clean': '2024-09-16T15:48:20.946152+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:48:20.960600+0000', 'last_undegraded': '2024-09-16T15:48:20.960600+0000', 'last_fullsized': '2024-09-16T15:48:20.960600+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_clean_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T15:48:27.027 INFO:tasks.ceph.ceph_manager.ceph:PG 13.2e is not active+clean 2024-09-16T15:48:27.027 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:48:20.965863+0000', 'last_change': '2024-09-16T15:48:20.965863+0000', 'last_active': '2024-09-16T15:48:20.946152+0000', 'last_peered': '2024-09-16T15:48:20.946152+0000', 'last_clean': '2024-09-16T15:48:20.946152+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:48:20.965863+0000', 'last_undegraded': '2024-09-16T15:48:20.965863+0000', 'last_fullsized': '2024-09-16T15:48:20.965863+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_clean_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-16T15:48:27.027 INFO:tasks.ceph.ceph_manager.ceph:PG 13.2b is not active+clean 2024-09-16T15:48:27.027 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:48:20.958863+0000', 'last_change': '2024-09-16T15:48:20.958863+0000', 'last_active': '2024-09-16T15:48:20.946152+0000', 'last_peered': '2024-09-16T15:48:20.946152+0000', 'last_clean': '2024-09-16T15:48:20.946152+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:48:20.958863+0000', 'last_undegraded': '2024-09-16T15:48:20.958863+0000', 'last_fullsized': '2024-09-16T15:48:20.958863+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_clean_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-16T15:48:27.027 INFO:tasks.ceph.ceph_manager.ceph:PG 13.29 is not active+clean 2024-09-16T15:48:27.027 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:48:20.962242+0000', 'last_change': '2024-09-16T15:48:20.962242+0000', 'last_active': '2024-09-16T15:48:20.946152+0000', 'last_peered': '2024-09-16T15:48:20.946152+0000', 'last_clean': '2024-09-16T15:48:20.946152+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:48:20.962242+0000', 'last_undegraded': '2024-09-16T15:48:20.962242+0000', 'last_fullsized': '2024-09-16T15:48:20.962242+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_clean_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T15:48:27.027 INFO:tasks.ceph.ceph_manager.ceph:PG 13.28 is not active+clean 2024-09-16T15:48:27.027 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:48:20.962313+0000', 'last_change': '2024-09-16T15:48:20.962313+0000', 'last_active': '2024-09-16T15:48:20.946152+0000', 'last_peered': '2024-09-16T15:48:20.946152+0000', 'last_clean': '2024-09-16T15:48:20.946152+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:48:20.962313+0000', 'last_undegraded': '2024-09-16T15:48:20.962313+0000', 'last_fullsized': '2024-09-16T15:48:20.962313+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_clean_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-16T15:48:27.027 INFO:tasks.ceph.ceph_manager.ceph:PG 13.27 is not active+clean 2024-09-16T15:48:27.027 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:48:20.966477+0000', 'last_change': '2024-09-16T15:48:20.966477+0000', 'last_active': '2024-09-16T15:48:20.946152+0000', 'last_peered': '2024-09-16T15:48:20.946152+0000', 'last_clean': '2024-09-16T15:48:20.946152+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:48:20.966477+0000', 'last_undegraded': '2024-09-16T15:48:20.966477+0000', 'last_fullsized': '2024-09-16T15:48:20.966477+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_clean_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-16T15:48:27.027 INFO:tasks.ceph.ceph_manager.ceph:PG 13.26 is not active+clean 2024-09-16T15:48:27.028 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:48:20.957649+0000', 'last_change': '2024-09-16T15:48:20.957649+0000', 'last_active': '2024-09-16T15:48:20.946152+0000', 'last_peered': '2024-09-16T15:48:20.946152+0000', 'last_clean': '2024-09-16T15:48:20.946152+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:48:20.957649+0000', 'last_undegraded': '2024-09-16T15:48:20.957649+0000', 'last_fullsized': '2024-09-16T15:48:20.957649+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_clean_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T15:48:27.028 INFO:tasks.ceph.ceph_manager.ceph:PG 13.25 is not active+clean 2024-09-16T15:48:27.028 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:48:20.958211+0000', 'last_change': '2024-09-16T15:48:20.958211+0000', 'last_active': '2024-09-16T15:48:20.946152+0000', 'last_peered': '2024-09-16T15:48:20.946152+0000', 'last_clean': '2024-09-16T15:48:20.946152+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:48:20.958211+0000', 'last_undegraded': '2024-09-16T15:48:20.958211+0000', 'last_fullsized': '2024-09-16T15:48:20.958211+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_clean_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T15:48:27.028 INFO:tasks.ceph.ceph_manager.ceph:PG 13.24 is not active+clean 2024-09-16T15:48:27.028 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:48:20.963150+0000', 'last_change': '2024-09-16T15:48:20.963150+0000', 'last_active': '2024-09-16T15:48:20.946152+0000', 'last_peered': '2024-09-16T15:48:20.946152+0000', 'last_clean': '2024-09-16T15:48:20.946152+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:48:20.963150+0000', 'last_undegraded': '2024-09-16T15:48:20.963150+0000', 'last_fullsized': '2024-09-16T15:48:20.963150+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_clean_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-16T15:48:27.028 INFO:tasks.ceph.ceph_manager.ceph:PG 13.23 is not active+clean 2024-09-16T15:48:27.028 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:48:20.957984+0000', 'last_change': '2024-09-16T15:48:20.957984+0000', 'last_active': '2024-09-16T15:48:20.946152+0000', 'last_peered': '2024-09-16T15:48:20.946152+0000', 'last_clean': '2024-09-16T15:48:20.946152+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:48:20.957984+0000', 'last_undegraded': '2024-09-16T15:48:20.957984+0000', 'last_fullsized': '2024-09-16T15:48:20.957984+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_clean_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-16T15:48:27.028 INFO:tasks.ceph.ceph_manager.ceph:PG 13.21 is not active+clean 2024-09-16T15:48:27.028 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:48:20.961609+0000', 'last_change': '2024-09-16T15:48:20.961609+0000', 'last_active': '2024-09-16T15:48:20.946152+0000', 'last_peered': '2024-09-16T15:48:20.946152+0000', 'last_clean': '2024-09-16T15:48:20.946152+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:48:20.961609+0000', 'last_undegraded': '2024-09-16T15:48:20.961609+0000', 'last_fullsized': '2024-09-16T15:48:20.961609+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_clean_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T15:48:27.028 INFO:tasks.ceph.ceph_manager.ceph:PG 13.20 is not active+clean 2024-09-16T15:48:27.028 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:48:20.962070+0000', 'last_change': '2024-09-16T15:48:20.962070+0000', 'last_active': '2024-09-16T15:48:20.946152+0000', 'last_peered': '2024-09-16T15:48:20.946152+0000', 'last_clean': '2024-09-16T15:48:20.946152+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:48:20.962070+0000', 'last_undegraded': '2024-09-16T15:48:20.962070+0000', 'last_fullsized': '2024-09-16T15:48:20.962070+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_clean_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T15:48:27.028 INFO:tasks.ceph.ceph_manager.ceph:PG 13.1e is not active+clean 2024-09-16T15:48:27.029 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:48:20.961053+0000', 'last_change': '2024-09-16T15:48:20.961053+0000', 'last_active': '2024-09-16T15:48:20.946152+0000', 'last_peered': '2024-09-16T15:48:20.946152+0000', 'last_clean': '2024-09-16T15:48:20.946152+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:48:20.961053+0000', 'last_undegraded': '2024-09-16T15:48:20.961053+0000', 'last_fullsized': '2024-09-16T15:48:20.961053+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_clean_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T15:48:27.029 INFO:tasks.ceph.ceph_manager.ceph:PG 13.1f is not active+clean 2024-09-16T15:48:27.029 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:48:20.963995+0000', 'last_change': '2024-09-16T15:48:20.963995+0000', 'last_active': '2024-09-16T15:48:20.946152+0000', 'last_peered': '2024-09-16T15:48:20.946152+0000', 'last_clean': '2024-09-16T15:48:20.946152+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:48:20.963995+0000', 'last_undegraded': '2024-09-16T15:48:20.963995+0000', 'last_fullsized': '2024-09-16T15:48:20.963995+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_clean_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-16T15:48:27.029 INFO:tasks.ceph.ceph_manager.ceph:PG 13.1c is not active+clean 2024-09-16T15:48:27.029 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:48:20.960641+0000', 'last_change': '2024-09-16T15:48:20.960641+0000', 'last_active': '2024-09-16T15:48:20.946152+0000', 'last_peered': '2024-09-16T15:48:20.946152+0000', 'last_clean': '2024-09-16T15:48:20.946152+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:48:20.960641+0000', 'last_undegraded': '2024-09-16T15:48:20.960641+0000', 'last_fullsized': '2024-09-16T15:48:20.960641+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_clean_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T15:48:27.029 INFO:tasks.ceph.ceph_manager.ceph:PG 13.1d is not active+clean 2024-09-16T15:48:27.029 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:48:20.961183+0000', 'last_change': '2024-09-16T15:48:20.961183+0000', 'last_active': '2024-09-16T15:48:20.946152+0000', 'last_peered': '2024-09-16T15:48:20.946152+0000', 'last_clean': '2024-09-16T15:48:20.946152+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:48:20.961183+0000', 'last_undegraded': '2024-09-16T15:48:20.961183+0000', 'last_fullsized': '2024-09-16T15:48:20.961183+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_clean_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-16T15:48:27.029 INFO:tasks.ceph.ceph_manager.ceph:PG 13.1a is not active+clean 2024-09-16T15:48:27.029 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:48:20.959326+0000', 'last_change': '2024-09-16T15:48:20.959326+0000', 'last_active': '2024-09-16T15:48:20.946152+0000', 'last_peered': '2024-09-16T15:48:20.946152+0000', 'last_clean': '2024-09-16T15:48:20.946152+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:48:20.959326+0000', 'last_undegraded': '2024-09-16T15:48:20.959326+0000', 'last_fullsized': '2024-09-16T15:48:20.959326+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_clean_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-16T15:48:27.029 INFO:tasks.ceph.ceph_manager.ceph:PG 13.1b is not active+clean 2024-09-16T15:48:27.029 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:48:20.962285+0000', 'last_change': '2024-09-16T15:48:20.962285+0000', 'last_active': '2024-09-16T15:48:20.946152+0000', 'last_peered': '2024-09-16T15:48:20.946152+0000', 'last_clean': '2024-09-16T15:48:20.946152+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:48:20.962285+0000', 'last_undegraded': '2024-09-16T15:48:20.962285+0000', 'last_fullsized': '2024-09-16T15:48:20.962285+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_clean_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T15:48:27.030 INFO:tasks.ceph.ceph_manager.ceph:PG 13.18 is not active+clean 2024-09-16T15:48:27.030 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:48:20.962335+0000', 'last_change': '2024-09-16T15:48:20.962335+0000', 'last_active': '2024-09-16T15:48:20.946152+0000', 'last_peered': '2024-09-16T15:48:20.946152+0000', 'last_clean': '2024-09-16T15:48:20.946152+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:48:20.962335+0000', 'last_undegraded': '2024-09-16T15:48:20.962335+0000', 'last_fullsized': '2024-09-16T15:48:20.962335+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_clean_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-16T15:48:27.030 INFO:tasks.ceph.ceph_manager.ceph:PG 13.17 is not active+clean 2024-09-16T15:48:27.030 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:48:20.962027+0000', 'last_change': '2024-09-16T15:48:20.962027+0000', 'last_active': '2024-09-16T15:48:20.946152+0000', 'last_peered': '2024-09-16T15:48:20.946152+0000', 'last_clean': '2024-09-16T15:48:20.946152+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:48:20.962027+0000', 'last_undegraded': '2024-09-16T15:48:20.962027+0000', 'last_fullsized': '2024-09-16T15:48:20.962027+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_clean_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-16T15:48:27.030 INFO:tasks.ceph.ceph_manager.ceph:PG 13.14 is not active+clean 2024-09-16T15:48:27.030 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:48:20.967777+0000', 'last_change': '2024-09-16T15:48:20.967777+0000', 'last_active': '2024-09-16T15:48:20.946152+0000', 'last_peered': '2024-09-16T15:48:20.946152+0000', 'last_clean': '2024-09-16T15:48:20.946152+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:48:20.967777+0000', 'last_undegraded': '2024-09-16T15:48:20.967777+0000', 'last_fullsized': '2024-09-16T15:48:20.967777+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_clean_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-16T15:48:27.030 INFO:tasks.ceph.ceph_manager.ceph:PG 13.15 is not active+clean 2024-09-16T15:48:27.030 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:48:20.958573+0000', 'last_change': '2024-09-16T15:48:20.958573+0000', 'last_active': '2024-09-16T15:48:20.946152+0000', 'last_peered': '2024-09-16T15:48:20.946152+0000', 'last_clean': '2024-09-16T15:48:20.946152+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:48:20.958573+0000', 'last_undegraded': '2024-09-16T15:48:20.958573+0000', 'last_fullsized': '2024-09-16T15:48:20.958573+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_clean_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T15:48:27.030 INFO:tasks.ceph.ceph_manager.ceph:PG 13.12 is not active+clean 2024-09-16T15:48:27.030 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:48:20.965797+0000', 'last_change': '2024-09-16T15:48:20.965797+0000', 'last_active': '2024-09-16T15:48:20.946152+0000', 'last_peered': '2024-09-16T15:48:20.946152+0000', 'last_clean': '2024-09-16T15:48:20.946152+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:48:20.965797+0000', 'last_undegraded': '2024-09-16T15:48:20.965797+0000', 'last_fullsized': '2024-09-16T15:48:20.965797+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_clean_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-16T15:48:27.030 INFO:tasks.ceph.ceph_manager.ceph:PG 13.13 is not active+clean 2024-09-16T15:48:27.030 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:48:20.959493+0000', 'last_change': '2024-09-16T15:48:20.959493+0000', 'last_active': '2024-09-16T15:48:20.946152+0000', 'last_peered': '2024-09-16T15:48:20.946152+0000', 'last_clean': '2024-09-16T15:48:20.946152+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:48:20.959493+0000', 'last_undegraded': '2024-09-16T15:48:20.959493+0000', 'last_fullsized': '2024-09-16T15:48:20.959493+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_clean_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-16T15:48:27.031 INFO:tasks.ceph.ceph_manager.ceph:PG 13.11 is not active+clean 2024-09-16T15:48:27.031 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:48:20.964413+0000', 'last_change': '2024-09-16T15:48:20.964413+0000', 'last_active': '2024-09-16T15:48:20.946152+0000', 'last_peered': '2024-09-16T15:48:20.946152+0000', 'last_clean': '2024-09-16T15:48:20.946152+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:48:20.964413+0000', 'last_undegraded': '2024-09-16T15:48:20.964413+0000', 'last_fullsized': '2024-09-16T15:48:20.964413+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_clean_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-16T15:48:27.031 INFO:tasks.ceph.ceph_manager.ceph:PG 13.e is not active+clean 2024-09-16T15:48:27.031 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:48:20.960635+0000', 'last_change': '2024-09-16T15:48:20.960635+0000', 'last_active': '2024-09-16T15:48:20.946152+0000', 'last_peered': '2024-09-16T15:48:20.946152+0000', 'last_clean': '2024-09-16T15:48:20.946152+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:48:20.960635+0000', 'last_undegraded': '2024-09-16T15:48:20.960635+0000', 'last_fullsized': '2024-09-16T15:48:20.960635+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_clean_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T15:48:27.031 INFO:tasks.ceph.ceph_manager.ceph:PG 13.f is not active+clean 2024-09-16T15:48:27.031 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:48:20.961693+0000', 'last_change': '2024-09-16T15:48:20.961693+0000', 'last_active': '2024-09-16T15:48:20.946152+0000', 'last_peered': '2024-09-16T15:48:20.946152+0000', 'last_clean': '2024-09-16T15:48:20.946152+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:48:20.961693+0000', 'last_undegraded': '2024-09-16T15:48:20.961693+0000', 'last_fullsized': '2024-09-16T15:48:20.961693+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_clean_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T15:48:27.031 INFO:tasks.ceph.ceph_manager.ceph:PG 13.d is not active+clean 2024-09-16T15:48:27.031 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:48:20.958552+0000', 'last_change': '2024-09-16T15:48:20.958552+0000', 'last_active': '2024-09-16T15:48:20.946152+0000', 'last_peered': '2024-09-16T15:48:20.946152+0000', 'last_clean': '2024-09-16T15:48:20.946152+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:48:20.958552+0000', 'last_undegraded': '2024-09-16T15:48:20.958552+0000', 'last_fullsized': '2024-09-16T15:48:20.958552+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_clean_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T15:48:27.031 INFO:tasks.ceph.ceph_manager.ceph:PG 13.a is not active+clean 2024-09-16T15:48:27.031 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:48:20.956433+0000', 'last_change': '2024-09-16T15:48:20.956433+0000', 'last_active': '2024-09-16T15:48:20.946152+0000', 'last_peered': '2024-09-16T15:48:20.946152+0000', 'last_clean': '2024-09-16T15:48:20.946152+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:48:20.956433+0000', 'last_undegraded': '2024-09-16T15:48:20.956433+0000', 'last_fullsized': '2024-09-16T15:48:20.956433+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_clean_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-16T15:48:27.031 INFO:tasks.ceph.ceph_manager.ceph:PG 13.b is not active+clean 2024-09-16T15:48:27.031 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:48:20.957113+0000', 'last_change': '2024-09-16T15:48:20.957113+0000', 'last_active': '2024-09-16T15:48:20.946152+0000', 'last_peered': '2024-09-16T15:48:20.946152+0000', 'last_clean': '2024-09-16T15:48:20.946152+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:48:20.957113+0000', 'last_undegraded': '2024-09-16T15:48:20.957113+0000', 'last_fullsized': '2024-09-16T15:48:20.957113+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_clean_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T15:48:27.032 INFO:tasks.ceph.ceph_manager.ceph:PG 13.8 is not active+clean 2024-09-16T15:48:27.032 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:48:20.957294+0000', 'last_change': '2024-09-16T15:48:20.957294+0000', 'last_active': '2024-09-16T15:48:20.946152+0000', 'last_peered': '2024-09-16T15:48:20.946152+0000', 'last_clean': '2024-09-16T15:48:20.946152+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:48:20.957294+0000', 'last_undegraded': '2024-09-16T15:48:20.957294+0000', 'last_fullsized': '2024-09-16T15:48:20.957294+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_clean_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-16T15:48:27.032 INFO:tasks.ceph.ceph_manager.ceph:PG 13.9 is not active+clean 2024-09-16T15:48:27.032 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:48:20.959074+0000', 'last_change': '2024-09-16T15:48:20.959074+0000', 'last_active': '2024-09-16T15:48:20.946152+0000', 'last_peered': '2024-09-16T15:48:20.946152+0000', 'last_clean': '2024-09-16T15:48:20.946152+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:48:20.959074+0000', 'last_undegraded': '2024-09-16T15:48:20.959074+0000', 'last_fullsized': '2024-09-16T15:48:20.959074+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_clean_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T15:48:27.032 INFO:tasks.ceph.ceph_manager.ceph:PG 13.6 is not active+clean 2024-09-16T15:48:27.032 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:48:20.957180+0000', 'last_change': '2024-09-16T15:48:20.957180+0000', 'last_active': '2024-09-16T15:48:20.946152+0000', 'last_peered': '2024-09-16T15:48:20.946152+0000', 'last_clean': '2024-09-16T15:48:20.946152+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:48:20.957180+0000', 'last_undegraded': '2024-09-16T15:48:20.957180+0000', 'last_fullsized': '2024-09-16T15:48:20.957180+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_clean_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T15:48:27.032 INFO:tasks.ceph.ceph_manager.ceph:PG 13.7 is not active+clean 2024-09-16T15:48:27.032 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:48:20.957910+0000', 'last_change': '2024-09-16T15:48:20.957910+0000', 'last_active': '2024-09-16T15:48:20.946152+0000', 'last_peered': '2024-09-16T15:48:20.946152+0000', 'last_clean': '2024-09-16T15:48:20.946152+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:48:20.957910+0000', 'last_undegraded': '2024-09-16T15:48:20.957910+0000', 'last_fullsized': '2024-09-16T15:48:20.957910+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_clean_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T15:48:27.032 INFO:tasks.ceph.ceph_manager.ceph:PG 13.4 is not active+clean 2024-09-16T15:48:27.032 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:48:20.957792+0000', 'last_change': '2024-09-16T15:48:20.957792+0000', 'last_active': '2024-09-16T15:48:20.946152+0000', 'last_peered': '2024-09-16T15:48:20.946152+0000', 'last_clean': '2024-09-16T15:48:20.946152+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:48:20.957792+0000', 'last_undegraded': '2024-09-16T15:48:20.957792+0000', 'last_fullsized': '2024-09-16T15:48:20.957792+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_clean_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T15:48:27.032 INFO:tasks.ceph.ceph_manager.ceph:PG 13.5 is not active+clean 2024-09-16T15:48:27.033 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:48:20.960754+0000', 'last_change': '2024-09-16T15:48:20.960754+0000', 'last_active': '2024-09-16T15:48:20.946152+0000', 'last_peered': '2024-09-16T15:48:20.946152+0000', 'last_clean': '2024-09-16T15:48:20.946152+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:48:20.960754+0000', 'last_undegraded': '2024-09-16T15:48:20.960754+0000', 'last_fullsized': '2024-09-16T15:48:20.960754+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_clean_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T15:48:27.033 INFO:tasks.ceph.ceph_manager.ceph:PG 13.2 is not active+clean 2024-09-16T15:48:27.033 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:48:20.956848+0000', 'last_change': '2024-09-16T15:48:20.956848+0000', 'last_active': '2024-09-16T15:48:20.946152+0000', 'last_peered': '2024-09-16T15:48:20.946152+0000', 'last_clean': '2024-09-16T15:48:20.946152+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:48:20.956848+0000', 'last_undegraded': '2024-09-16T15:48:20.956848+0000', 'last_fullsized': '2024-09-16T15:48:20.956848+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_clean_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-16T15:48:27.033 INFO:tasks.ceph.ceph_manager.ceph:PG 13.3 is not active+clean 2024-09-16T15:48:27.033 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:48:20.962709+0000', 'last_change': '2024-09-16T15:48:20.962709+0000', 'last_active': '2024-09-16T15:48:20.946152+0000', 'last_peered': '2024-09-16T15:48:20.946152+0000', 'last_clean': '2024-09-16T15:48:20.946152+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:48:20.962709+0000', 'last_undegraded': '2024-09-16T15:48:20.962709+0000', 'last_fullsized': '2024-09-16T15:48:20.962709+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_clean_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-16T15:48:27.033 INFO:tasks.ceph.ceph_manager.ceph:PG 13.0 is not active+clean 2024-09-16T15:48:27.033 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:48:20.956619+0000', 'last_change': '2024-09-16T15:48:20.956619+0000', 'last_active': '2024-09-16T15:48:20.946152+0000', 'last_peered': '2024-09-16T15:48:20.946152+0000', 'last_clean': '2024-09-16T15:48:20.946152+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:48:20.956619+0000', 'last_undegraded': '2024-09-16T15:48:20.956619+0000', 'last_fullsized': '2024-09-16T15:48:20.956619+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_clean_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-16T15:48:27.033 INFO:tasks.ceph.ceph_manager.ceph:PG 13.1 is not active+clean 2024-09-16T15:48:27.033 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:48:20.958982+0000', 'last_change': '2024-09-16T15:48:20.958982+0000', 'last_active': '2024-09-16T15:48:20.946152+0000', 'last_peered': '2024-09-16T15:48:20.946152+0000', 'last_clean': '2024-09-16T15:48:20.946152+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:48:20.958982+0000', 'last_undegraded': '2024-09-16T15:48:20.958982+0000', 'last_fullsized': '2024-09-16T15:48:20.958982+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_clean_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-16T15:48:27.033 INFO:tasks.ceph.ceph_manager.ceph:PG 13.c is not active+clean 2024-09-16T15:48:27.033 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '13.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:48:20.965048+0000', 'last_change': '2024-09-16T15:48:20.965048+0000', 'last_active': '2024-09-16T15:48:20.946152+0000', 'last_peered': '2024-09-16T15:48:20.946152+0000', 'last_clean': '2024-09-16T15:48:20.946152+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:48:20.965048+0000', 'last_undegraded': '2024-09-16T15:48:20.965048+0000', 'last_fullsized': '2024-09-16T15:48:20.965048+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'last_clean_scrub_stamp': '2024-09-16T15:48:20.946152+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-16T15:48:27.034 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T15:48:27.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.217+0000 7fc0d5a00640 1 -- 172.21.15.151:0/576651950 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc0d015cff0 msgr2=0x7fc0d006e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:27.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.217+0000 7fc0d5a00640 1 --2- 172.21.15.151:0/576651950 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc0d015cff0 0x7fc0d006e6f0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fc0d0075110 tx=0x7fc0bc02a560 comp rx=0 tx=0).stop 2024-09-16T15:48:27.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.217+0000 7fc0d5a00640 1 -- 172.21.15.151:0/576651950 shutdown_connections 2024-09-16T15:48:27.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.217+0000 7fc0d5a00640 1 --2- 172.21.15.151:0/576651950 >> v2:172.21.15.167:3300/0 conn(0x7fc0d006f220 0x7fc0d006f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:27.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.217+0000 7fc0d5a00640 1 --2- 172.21.15.151:0/576651950 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc0d015cff0 0x7fc0d006e6f0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:27.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.217+0000 7fc0d5a00640 1 -- 172.21.15.151:0/576651950 >> 172.21.15.151:0/576651950 conn(0x7fc0d00b18e0 msgr2=0x7fc0d00b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:27.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.217+0000 7fc0d5a00640 1 -- 172.21.15.151:0/576651950 shutdown_connections 2024-09-16T15:48:27.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.217+0000 7fc0d5a00640 1 -- 172.21.15.151:0/576651950 wait complete. 2024-09-16T15:48:27.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.219+0000 7fc0d5a00640 1 Processor -- start 2024-09-16T15:48:27.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.219+0000 7fc0d5a00640 1 -- start start 2024-09-16T15:48:27.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.220+0000 7fc0d5a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc0d006f220 0x7fc0d00cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:27.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.220+0000 7fc0d5a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0d0159aa0 0x7fc0d00ce6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:27.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.220+0000 7fc0d5a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0d015cff0 0x7fc0d0149150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:27.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.220+0000 7fc0d5a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d00cc520 con 0x7fc0d006f220 2024-09-16T15:48:27.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.220+0000 7fc0d5a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d00cc3a0 con 0x7fc0d015cff0 2024-09-16T15:48:27.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.220+0000 7fc0d5a00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fc0d00cc6a0 con 0x7fc0d0159aa0 2024-09-16T15:48:27.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.220+0000 7fc0cea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc0d006f220 0x7fc0d00cdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:27.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.220+0000 7fc0cea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc0d006f220 0x7fc0d00cdb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:42786/0 (socket says 172.21.15.151:42786) 2024-09-16T15:48:27.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.220+0000 7fc0cea00640 1 -- 172.21.15.151:0/2019734700 learned_addr learned my addr 172.21.15.151:0/2019734700 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:48:27.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.220+0000 7fc0ce000640 1 --2- 172.21.15.151:0/2019734700 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0d0159aa0 0x7fc0d00ce6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:27.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.220+0000 7fc0cf400640 1 --2- 172.21.15.151:0/2019734700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0d015cff0 0x7fc0d0149150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:27.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.221+0000 7fc0cea00640 1 -- 172.21.15.151:0/2019734700 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0d0159aa0 msgr2=0x7fc0d00ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:27.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.221+0000 7fc0cea00640 1 --2- 172.21.15.151:0/2019734700 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0d0159aa0 0x7fc0d00ce6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:27.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.221+0000 7fc0cea00640 1 -- 172.21.15.151:0/2019734700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0d015cff0 msgr2=0x7fc0d0149150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:27.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.221+0000 7fc0cea00640 1 --2- 172.21.15.151:0/2019734700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0d015cff0 0x7fc0d0149150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:27.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.221+0000 7fc0cea00640 1 -- 172.21.15.151:0/2019734700 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0d0157090 con 0x7fc0d006f220 2024-09-16T15:48:27.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.221+0000 7fc0ce000640 1 --2- 172.21.15.151:0/2019734700 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0d0159aa0 0x7fc0d00ce6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:48:27.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.221+0000 7fc0cea00640 1 --2- 172.21.15.151:0/2019734700 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc0d006f220 0x7fc0d00cdb40 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fc0c000c430 tx=0x7fc0c000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:27.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.221+0000 7fc0c6a00640 1 -- 172.21.15.151:0/2019734700 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c0011790 con 0x7fc0d006f220 2024-09-16T15:48:27.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.221+0000 7fc0c6a00640 1 -- 172.21.15.151:0/2019734700 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc0c0017070 con 0x7fc0d006f220 2024-09-16T15:48:27.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.221+0000 7fc0c6a00640 1 -- 172.21.15.151:0/2019734700 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c0011d00 con 0x7fc0d006f220 2024-09-16T15:48:27.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.221+0000 7fc0d5a00640 1 -- 172.21.15.151:0/2019734700 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0d00ae160 con 0x7fc0d006f220 2024-09-16T15:48:27.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.221+0000 7fc0d5a00640 1 -- 172.21.15.151:0/2019734700 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0d00ae720 con 0x7fc0d006f220 2024-09-16T15:48:27.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.223+0000 7fc0d5a00640 1 -- 172.21.15.151:0/2019734700 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0d0069360 con 0x7fc0d006f220 2024-09-16T15:48:27.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.224+0000 7fc0c6a00640 1 -- 172.21.15.151:0/2019734700 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 30) ==== 102868+0+0 (secure 0 0 0) 0x7fc0c001c460 con 0x7fc0d006f220 2024-09-16T15:48:27.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.225+0000 7fc0c6a00640 1 --2- 172.21.15.151:0/2019734700 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc0b00799c0 0x7fc0b007be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:27.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.225+0000 7fc0c6a00640 1 -- 172.21.15.151:0/2019734700 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7fc0c00996b0 con 0x7fc0d006f220 2024-09-16T15:48:27.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.225+0000 7fc0ce000640 1 --2- 172.21.15.151:0/2019734700 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc0b00799c0 0x7fc0b007be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:27.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.226+0000 7fc0ce000640 1 --2- 172.21.15.151:0/2019734700 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc0b00799c0 0x7fc0b007be80 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc033000 tx=0x7fc0bc004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:27.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.229+0000 7fc0c6a00640 1 -- 172.21.15.151:0/2019734700 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc0c0063420 con 0x7fc0d006f220 2024-09-16T15:48:27.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.486+0000 7fc0d5a00640 1 -- 172.21.15.151:0/2019734700 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fc0d00cbf70 con 0x7fc0d006f220 2024-09-16T15:48:27.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.487+0000 7fc0c6a00640 1 -- 172.21.15.151:0/2019734700 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1138 (secure 0 0 0) 0x7fc0c0062d20 con 0x7fc0d006f220 2024-09-16T15:48:27.489 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:48:27.489 INFO:teuthology.orchestra.run.smithi151.stdout:{"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":571,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":91,"num_osds":8,"num_up_osds":8,"osd_up_since":1726501150,"num_in_osds":8,"osd_in_since":1726501140,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":56},{"state_name":"active+clean","count":41}],"num_pgs":97,"num_pools":3,"num_objects":5,"data_bytes":590368,"bytes_used":501149696,"bytes_avail":772592963584,"bytes_total":773094113280,"inactive_pgs_ratio":0.57731956243515015,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":1},"fsmap":{"epoch":56,"btime":"2024-09-16T15:48:24:992606+0000","id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"c","status":"up:active","gid":6673}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":13,"modified":"2024-09-16T15:48:01.742446+0000","services":{}},"progress_events":{}} 2024-09-16T15:48:27.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.489+0000 7fc0d5a00640 1 -- 172.21.15.151:0/2019734700 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc0b00799c0 msgr2=0x7fc0b007be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:27.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.489+0000 7fc0d5a00640 1 --2- 172.21.15.151:0/2019734700 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc0b00799c0 0x7fc0b007be80 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc033000 tx=0x7fc0bc004730 comp rx=0 tx=0).stop 2024-09-16T15:48:27.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.489+0000 7fc0d5a00640 1 -- 172.21.15.151:0/2019734700 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc0d006f220 msgr2=0x7fc0d00cdb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:27.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.489+0000 7fc0d5a00640 1 --2- 172.21.15.151:0/2019734700 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc0d006f220 0x7fc0d00cdb40 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fc0c000c430 tx=0x7fc0c000c8f0 comp rx=0 tx=0).stop 2024-09-16T15:48:27.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.489+0000 7fc0d5a00640 1 -- 172.21.15.151:0/2019734700 shutdown_connections 2024-09-16T15:48:27.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.489+0000 7fc0d5a00640 1 --2- 172.21.15.151:0/2019734700 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc0b00799c0 0x7fc0b007be80 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:27.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.489+0000 7fc0d5a00640 1 --2- 172.21.15.151:0/2019734700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0d015cff0 0x7fc0d0149150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:27.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.489+0000 7fc0d5a00640 1 --2- 172.21.15.151:0/2019734700 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0d0159aa0 0x7fc0d00ce6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:27.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.489+0000 7fc0d5a00640 1 --2- 172.21.15.151:0/2019734700 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc0d006f220 0x7fc0d00cdb40 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:27.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.489+0000 7fc0d5a00640 1 -- 172.21.15.151:0/2019734700 >> 172.21.15.151:0/2019734700 conn(0x7fc0d00b18e0 msgr2=0x7fc0d015a660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:27.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.490+0000 7fc0d5a00640 1 -- 172.21.15.151:0/2019734700 shutdown_connections 2024-09-16T15:48:27.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.490+0000 7fc0d5a00640 1 -- 172.21.15.151:0/2019734700 wait complete. 2024-09-16T15:48:27.507 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-16T15:48:27.507 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:48:27.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.664+0000 7f1cf5600640 1 -- 172.21.15.151:0/3499959919 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1cf0073e00 msgr2=0x7f1cf0168e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:27.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.664+0000 7f1cf5600640 1 --2- 172.21.15.151:0/3499959919 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1cf0073e00 0x7f1cf0168e10 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f1cf00b4890 tx=0x7f1ce802a800 comp rx=0 tx=0).stop 2024-09-16T15:48:27.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.664+0000 7f1cf5600640 1 -- 172.21.15.151:0/3499959919 shutdown_connections 2024-09-16T15:48:27.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.664+0000 7f1cf5600640 1 --2- 172.21.15.151:0/3499959919 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1cf0073e00 0x7f1cf0168e10 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:27.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.664+0000 7f1cf5600640 1 -- 172.21.15.151:0/3499959919 >> 172.21.15.151:0/3499959919 conn(0x7f1cf00b1950 msgr2=0x7f1cf00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:27.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.664+0000 7f1cf5600640 1 -- 172.21.15.151:0/3499959919 shutdown_connections 2024-09-16T15:48:27.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.664+0000 7f1cf5600640 1 -- 172.21.15.151:0/3499959919 wait complete. 2024-09-16T15:48:27.668 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.666+0000 7f1cf5600640 1 Processor -- start 2024-09-16T15:48:27.668 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.666+0000 7f1cf5600640 1 -- start start 2024-09-16T15:48:27.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.667+0000 7f1cf5600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1cf0073e00 0x7f1cf014a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:27.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.667+0000 7f1cf5600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1cf00af3d0 0x7f1cf015bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:27.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.667+0000 7f1cf5600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1cf0161550 0x7f1cf0153d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:27.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.667+0000 7f1cf5600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf01694d0 con 0x7f1cf00af3d0 2024-09-16T15:48:27.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.667+0000 7f1cf5600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf0169350 con 0x7f1cf0073e00 2024-09-16T15:48:27.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.667+0000 7f1cf5600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f1cf0169650 con 0x7f1cf0161550 2024-09-16T15:48:27.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.667+0000 7f1cee000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1cf00af3d0 0x7f1cf015bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:27.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.667+0000 7f1cee000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1cf00af3d0 0x7f1cf015bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:42800/0 (socket says 172.21.15.151:42800) 2024-09-16T15:48:27.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.667+0000 7f1cee000640 1 -- 172.21.15.151:0/2659679856 learned_addr learned my addr 172.21.15.151:0/2659679856 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:48:27.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.667+0000 7f1ceea00640 1 --2- 172.21.15.151:0/2659679856 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1cf0073e00 0x7f1cf014a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:27.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.667+0000 7f1cef400640 1 --2- 172.21.15.151:0/2659679856 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1cf0161550 0x7f1cf0153d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:27.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.668+0000 7f1cee000640 1 -- 172.21.15.151:0/2659679856 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1cf0161550 msgr2=0x7f1cf0153d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:27.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.668+0000 7f1cee000640 1 --2- 172.21.15.151:0/2659679856 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1cf0161550 0x7f1cf0153d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:27.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.668+0000 7f1cee000640 1 -- 172.21.15.151:0/2659679856 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1cf0073e00 msgr2=0x7f1cf014a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:27.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.668+0000 7f1cee000640 1 --2- 172.21.15.151:0/2659679856 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1cf0073e00 0x7f1cf014a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:27.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.668+0000 7f1cee000640 1 -- 172.21.15.151:0/2659679856 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cf015c620 con 0x7f1cf00af3d0 2024-09-16T15:48:27.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.668+0000 7f1cef400640 1 --2- 172.21.15.151:0/2659679856 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1cf0161550 0x7f1cf0153d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:48:27.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.668+0000 7f1cee000640 1 --2- 172.21.15.151:0/2659679856 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1cf00af3d0 0x7f1cf015bcd0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc00c450 tx=0x7f1cdc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:27.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.668+0000 7f1ce6a00640 1 -- 172.21.15.151:0/2659679856 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cdc016020 con 0x7f1cf00af3d0 2024-09-16T15:48:27.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.668+0000 7f1ce6a00640 1 -- 172.21.15.151:0/2659679856 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1cdc012920 con 0x7f1cf00af3d0 2024-09-16T15:48:27.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.668+0000 7f1ce6a00640 1 -- 172.21.15.151:0/2659679856 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cdc012d70 con 0x7f1cf00af3d0 2024-09-16T15:48:27.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.668+0000 7f1cf5600640 1 -- 172.21.15.151:0/2659679856 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cf015c900 con 0x7f1cf00af3d0 2024-09-16T15:48:27.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.668+0000 7f1cf5600640 1 -- 172.21.15.151:0/2659679856 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cf015cec0 con 0x7f1cf00af3d0 2024-09-16T15:48:27.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.670+0000 7f1cf5600640 1 -- 172.21.15.151:0/2659679856 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cf006f710 con 0x7f1cf00af3d0 2024-09-16T15:48:27.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.671+0000 7f1ce6a00640 1 -- 172.21.15.151:0/2659679856 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 30) ==== 102868+0+0 (secure 0 0 0) 0x7f1cdc019020 con 0x7f1cf00af3d0 2024-09-16T15:48:27.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.672+0000 7f1ce6a00640 1 --2- 172.21.15.151:0/2659679856 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1cd00799c0 0x7f1cd007be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:27.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.672+0000 7f1ce6a00640 1 -- 172.21.15.151:0/2659679856 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7f1cdc099720 con 0x7f1cf00af3d0 2024-09-16T15:48:27.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.672+0000 7f1ceea00640 1 --2- 172.21.15.151:0/2659679856 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1cd00799c0 0x7f1cd007be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:27.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.673+0000 7f1ceea00640 1 --2- 172.21.15.151:0/2659679856 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1cd00799c0 0x7f1cd007be80 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f1ce0006fd0 tx=0x7f1ce0009060 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:27.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.676+0000 7f1ce6a00640 1 -- 172.21.15.151:0/2659679856 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1cdc020020 con 0x7f1cf00af3d0 2024-09-16T15:48:27.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.863+0000 7f1cf5600640 1 -- 172.21.15.151:0/2659679856 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1cf00acf80 con 0x7f1cd00799c0 2024-09-16T15:48:27.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.868+0000 7f1ce6a00640 1 -- 172.21.15.151:0/2659679856 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252073 (secure 0 0 0) 0x7f1cf00acf80 con 0x7f1cd00799c0 2024-09-16T15:48:27.873 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:48:27.874 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-09-16T15:48:27.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.872+0000 7f1cf5600640 1 -- 172.21.15.151:0/2659679856 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1cd00799c0 msgr2=0x7f1cd007be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:27.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.872+0000 7f1cf5600640 1 --2- 172.21.15.151:0/2659679856 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1cd00799c0 0x7f1cd007be80 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f1ce0006fd0 tx=0x7f1ce0009060 comp rx=0 tx=0).stop 2024-09-16T15:48:27.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.872+0000 7f1cf5600640 1 -- 172.21.15.151:0/2659679856 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1cf00af3d0 msgr2=0x7f1cf015bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:27.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.872+0000 7f1cf5600640 1 --2- 172.21.15.151:0/2659679856 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1cf00af3d0 0x7f1cf015bcd0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc00c450 tx=0x7f1cdc00c910 comp rx=0 tx=0).stop 2024-09-16T15:48:27.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.872+0000 7f1cf5600640 1 -- 172.21.15.151:0/2659679856 shutdown_connections 2024-09-16T15:48:27.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.872+0000 7f1cf5600640 1 --2- 172.21.15.151:0/2659679856 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1cd00799c0 0x7f1cd007be80 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:27.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.872+0000 7f1cf5600640 1 --2- 172.21.15.151:0/2659679856 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1cf0161550 0x7f1cf0153d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:27.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.872+0000 7f1cf5600640 1 --2- 172.21.15.151:0/2659679856 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1cf00af3d0 0x7f1cf015bcd0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:27.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.872+0000 7f1cf5600640 1 --2- 172.21.15.151:0/2659679856 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1cf0073e00 0x7f1cf014a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:27.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.872+0000 7f1cf5600640 1 -- 172.21.15.151:0/2659679856 >> 172.21.15.151:0/2659679856 conn(0x7f1cf00b1950 msgr2=0x7f1cf0161c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:27.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.872+0000 7f1cf5600640 1 -- 172.21.15.151:0/2659679856 shutdown_connections 2024-09-16T15:48:27.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:27.872+0000 7f1cf5600640 1 -- 172.21.15.151:0/2659679856 wait complete. 2024-09-16T15:48:27.890 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":363,"stamp":"2024-09-16T15:48:27.750000+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":173,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":489692,"kb_used_data":9896,"kb_used_omap":12,"kb_used_meta":479603,"kb_avail":754485028,"statfs":{"total":773094113280,"available":772592668672,"internally_reserved":0,"allocated":10133504,"data_stored":5037644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12738,"internal_metadata":491114046},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.796721"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.353070+0000","last_change":"2024-09-16T15:48:21.998960+0000","last_active":"2024-09-16T15:48:23.353070+0000","last_peered":"2024-09-16T15:48:23.353070+0000","last_clean":"2024-09-16T15:48:23.353070+0000","last_became_active":"2024-09-16T15:48:21.997542+0000","last_became_peered":"2024-09-16T15:48:21.997542+0000","last_unstale":"2024-09-16T15:48:23.353070+0000","last_undegraded":"2024-09-16T15:48:23.353070+0000","last_fullsized":"2024-09-16T15:48:23.353070+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:01:01.819932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.981239+0000","last_change":"2024-09-16T15:48:21.978226+0000","last_active":"2024-09-16T15:48:22.981239+0000","last_peered":"2024-09-16T15:48:22.981239+0000","last_clean":"2024-09-16T15:48:22.981239+0000","last_became_active":"2024-09-16T15:48:21.977983+0000","last_became_peered":"2024-09-16T15:48:21.977983+0000","last_unstale":"2024-09-16T15:48:22.981239+0000","last_undegraded":"2024-09-16T15:48:22.981239+0000","last_fullsized":"2024-09-16T15:48:22.981239+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:39:54.510565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352016+0000","last_change":"2024-09-16T15:48:21.984845+0000","last_active":"2024-09-16T15:48:23.352016+0000","last_peered":"2024-09-16T15:48:23.352016+0000","last_clean":"2024-09-16T15:48:23.352016+0000","last_became_active":"2024-09-16T15:48:21.984556+0000","last_became_peered":"2024-09-16T15:48:21.984556+0000","last_unstale":"2024-09-16T15:48:23.352016+0000","last_undegraded":"2024-09-16T15:48:23.352016+0000","last_fullsized":"2024-09-16T15:48:23.352016+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:07:20.943835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.980513+0000","last_change":"2024-09-16T15:48:21.984821+0000","last_active":"2024-09-16T15:48:22.980513+0000","last_peered":"2024-09-16T15:48:22.980513+0000","last_clean":"2024-09-16T15:48:22.980513+0000","last_became_active":"2024-09-16T15:48:21.984402+0000","last_became_peered":"2024-09-16T15:48:21.984402+0000","last_unstale":"2024-09-16T15:48:22.980513+0000","last_undegraded":"2024-09-16T15:48:22.980513+0000","last_fullsized":"2024-09-16T15:48:22.980513+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:04:46.727567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.975730+0000","last_change":"2024-09-16T15:48:21.995863+0000","last_active":"2024-09-16T15:48:22.975730+0000","last_peered":"2024-09-16T15:48:22.975730+0000","last_clean":"2024-09-16T15:48:22.975730+0000","last_became_active":"2024-09-16T15:48:21.994582+0000","last_became_peered":"2024-09-16T15:48:21.994582+0000","last_unstale":"2024-09-16T15:48:22.975730+0000","last_undegraded":"2024-09-16T15:48:22.975730+0000","last_fullsized":"2024-09-16T15:48:22.975730+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:54:53.550842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.351087+0000","last_change":"2024-09-16T15:48:21.990586+0000","last_active":"2024-09-16T15:48:23.351087+0000","last_peered":"2024-09-16T15:48:23.351087+0000","last_clean":"2024-09-16T15:48:23.351087+0000","last_became_active":"2024-09-16T15:48:21.987414+0000","last_became_peered":"2024-09-16T15:48:21.987414+0000","last_unstale":"2024-09-16T15:48:23.351087+0000","last_undegraded":"2024-09-16T15:48:23.351087+0000","last_fullsized":"2024-09-16T15:48:23.351087+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:32:12.555452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.979914+0000","last_change":"2024-09-16T15:48:21.981046+0000","last_active":"2024-09-16T15:48:22.979914+0000","last_peered":"2024-09-16T15:48:22.979914+0000","last_clean":"2024-09-16T15:48:22.979914+0000","last_became_active":"2024-09-16T15:48:21.980750+0000","last_became_peered":"2024-09-16T15:48:21.980750+0000","last_unstale":"2024-09-16T15:48:22.979914+0000","last_undegraded":"2024-09-16T15:48:22.979914+0000","last_fullsized":"2024-09-16T15:48:22.979914+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:03:48.288898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.351538+0000","last_change":"2024-09-16T15:48:21.994790+0000","last_active":"2024-09-16T15:48:23.351538+0000","last_peered":"2024-09-16T15:48:23.351538+0000","last_clean":"2024-09-16T15:48:23.351538+0000","last_became_active":"2024-09-16T15:48:21.993406+0000","last_became_peered":"2024-09-16T15:48:21.993406+0000","last_unstale":"2024-09-16T15:48:23.351538+0000","last_undegraded":"2024-09-16T15:48:23.351538+0000","last_fullsized":"2024-09-16T15:48:23.351538+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:19:07.343152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.353136+0000","last_change":"2024-09-16T15:48:21.988921+0000","last_active":"2024-09-16T15:48:23.353136+0000","last_peered":"2024-09-16T15:48:23.353136+0000","last_clean":"2024-09-16T15:48:23.353136+0000","last_became_active":"2024-09-16T15:48:21.987388+0000","last_became_peered":"2024-09-16T15:48:21.987388+0000","last_unstale":"2024-09-16T15:48:23.353136+0000","last_undegraded":"2024-09-16T15:48:23.353136+0000","last_fullsized":"2024-09-16T15:48:23.353136+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:16:58.292432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352286+0000","last_change":"2024-09-16T15:48:21.981255+0000","last_active":"2024-09-16T15:48:23.352286+0000","last_peered":"2024-09-16T15:48:23.352286+0000","last_clean":"2024-09-16T15:48:23.352286+0000","last_became_active":"2024-09-16T15:48:21.980926+0000","last_became_peered":"2024-09-16T15:48:21.980926+0000","last_unstale":"2024-09-16T15:48:23.352286+0000","last_undegraded":"2024-09-16T15:48:23.352286+0000","last_fullsized":"2024-09-16T15:48:23.352286+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:57:23.445821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.991376+0000","last_change":"2024-09-16T15:48:21.990558+0000","last_active":"2024-09-16T15:48:22.991376+0000","last_peered":"2024-09-16T15:48:22.991376+0000","last_clean":"2024-09-16T15:48:22.991376+0000","last_became_active":"2024-09-16T15:48:21.990293+0000","last_became_peered":"2024-09-16T15:48:21.990293+0000","last_unstale":"2024-09-16T15:48:22.991376+0000","last_undegraded":"2024-09-16T15:48:22.991376+0000","last_fullsized":"2024-09-16T15:48:22.991376+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:21:37.652682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352398+0000","last_change":"2024-09-16T15:48:21.987737+0000","last_active":"2024-09-16T15:48:23.352398+0000","last_peered":"2024-09-16T15:48:23.352398+0000","last_clean":"2024-09-16T15:48:23.352398+0000","last_became_active":"2024-09-16T15:48:21.985591+0000","last_became_peered":"2024-09-16T15:48:21.985591+0000","last_unstale":"2024-09-16T15:48:23.352398+0000","last_undegraded":"2024-09-16T15:48:23.352398+0000","last_fullsized":"2024-09-16T15:48:23.352398+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:47:53.347929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.975990+0000","last_change":"2024-09-16T15:48:21.989759+0000","last_active":"2024-09-16T15:48:22.975990+0000","last_peered":"2024-09-16T15:48:22.975990+0000","last_clean":"2024-09-16T15:48:22.975990+0000","last_became_active":"2024-09-16T15:48:21.989495+0000","last_became_peered":"2024-09-16T15:48:21.989495+0000","last_unstale":"2024-09-16T15:48:22.975990+0000","last_undegraded":"2024-09-16T15:48:22.975990+0000","last_fullsized":"2024-09-16T15:48:22.975990+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:36:39.039493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.351800+0000","last_change":"2024-09-16T15:48:21.991329+0000","last_active":"2024-09-16T15:48:23.351800+0000","last_peered":"2024-09-16T15:48:23.351800+0000","last_clean":"2024-09-16T15:48:23.351800+0000","last_became_active":"2024-09-16T15:48:21.990123+0000","last_became_peered":"2024-09-16T15:48:21.990123+0000","last_unstale":"2024-09-16T15:48:23.351800+0000","last_undegraded":"2024-09-16T15:48:23.351800+0000","last_fullsized":"2024-09-16T15:48:23.351800+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:01:55.713464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.980177+0000","last_change":"2024-09-16T15:48:21.986388+0000","last_active":"2024-09-16T15:48:22.980177+0000","last_peered":"2024-09-16T15:48:22.980177+0000","last_clean":"2024-09-16T15:48:22.980177+0000","last_became_active":"2024-09-16T15:48:21.986033+0000","last_became_peered":"2024-09-16T15:48:21.986033+0000","last_unstale":"2024-09-16T15:48:22.980177+0000","last_undegraded":"2024-09-16T15:48:22.980177+0000","last_fullsized":"2024-09-16T15:48:22.980177+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:43:55.728934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.351045+0000","last_change":"2024-09-16T15:48:21.982014+0000","last_active":"2024-09-16T15:48:23.351045+0000","last_peered":"2024-09-16T15:48:23.351045+0000","last_clean":"2024-09-16T15:48:23.351045+0000","last_became_active":"2024-09-16T15:48:21.981772+0000","last_became_peered":"2024-09-16T15:48:21.981772+0000","last_unstale":"2024-09-16T15:48:23.351045+0000","last_undegraded":"2024-09-16T15:48:23.351045+0000","last_fullsized":"2024-09-16T15:48:23.351045+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:38:23.345237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.353074+0000","last_change":"2024-09-16T15:48:21.986477+0000","last_active":"2024-09-16T15:48:23.353074+0000","last_peered":"2024-09-16T15:48:23.353074+0000","last_clean":"2024-09-16T15:48:23.353074+0000","last_became_active":"2024-09-16T15:48:21.984068+0000","last_became_peered":"2024-09-16T15:48:21.984068+0000","last_unstale":"2024-09-16T15:48:23.353074+0000","last_undegraded":"2024-09-16T15:48:23.353074+0000","last_fullsized":"2024-09-16T15:48:23.353074+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:12:07.405991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.991694+0000","last_change":"2024-09-16T15:48:21.993871+0000","last_active":"2024-09-16T15:48:22.991694+0000","last_peered":"2024-09-16T15:48:22.991694+0000","last_clean":"2024-09-16T15:48:22.991694+0000","last_became_active":"2024-09-16T15:48:21.992631+0000","last_became_peered":"2024-09-16T15:48:21.992631+0000","last_unstale":"2024-09-16T15:48:22.991694+0000","last_undegraded":"2024-09-16T15:48:22.991694+0000","last_fullsized":"2024-09-16T15:48:22.991694+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:02:59.965947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352178+0000","last_change":"2024-09-16T15:48:21.996135+0000","last_active":"2024-09-16T15:48:23.352178+0000","last_peered":"2024-09-16T15:48:23.352178+0000","last_clean":"2024-09-16T15:48:23.352178+0000","last_became_active":"2024-09-16T15:48:21.995671+0000","last_became_peered":"2024-09-16T15:48:21.995671+0000","last_unstale":"2024-09-16T15:48:23.352178+0000","last_undegraded":"2024-09-16T15:48:23.352178+0000","last_fullsized":"2024-09-16T15:48:23.352178+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:13:55.927762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352319+0000","last_change":"2024-09-16T15:48:21.994572+0000","last_active":"2024-09-16T15:48:23.352319+0000","last_peered":"2024-09-16T15:48:23.352319+0000","last_clean":"2024-09-16T15:48:23.352319+0000","last_became_active":"2024-09-16T15:48:21.994305+0000","last_became_peered":"2024-09-16T15:48:21.994305+0000","last_unstale":"2024-09-16T15:48:23.352319+0000","last_undegraded":"2024-09-16T15:48:23.352319+0000","last_fullsized":"2024-09-16T15:48:23.352319+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:27:24.090121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.979793+0000","last_change":"2024-09-16T15:48:21.976952+0000","last_active":"2024-09-16T15:48:22.979793+0000","last_peered":"2024-09-16T15:48:22.979793+0000","last_clean":"2024-09-16T15:48:22.979793+0000","last_became_active":"2024-09-16T15:48:21.976547+0000","last_became_peered":"2024-09-16T15:48:21.976547+0000","last_unstale":"2024-09-16T15:48:22.979793+0000","last_undegraded":"2024-09-16T15:48:22.979793+0000","last_fullsized":"2024-09-16T15:48:22.979793+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:29:49.942615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.351839+0000","last_change":"2024-09-16T15:48:21.988902+0000","last_active":"2024-09-16T15:48:23.351839+0000","last_peered":"2024-09-16T15:48:23.351839+0000","last_clean":"2024-09-16T15:48:23.351839+0000","last_became_active":"2024-09-16T15:48:21.988613+0000","last_became_peered":"2024-09-16T15:48:21.988613+0000","last_unstale":"2024-09-16T15:48:23.351839+0000","last_undegraded":"2024-09-16T15:48:23.351839+0000","last_fullsized":"2024-09-16T15:48:23.351839+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:05:00.466068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.351748+0000","last_change":"2024-09-16T15:48:21.985758+0000","last_active":"2024-09-16T15:48:23.351748+0000","last_peered":"2024-09-16T15:48:23.351748+0000","last_clean":"2024-09-16T15:48:23.351748+0000","last_became_active":"2024-09-16T15:48:21.985459+0000","last_became_peered":"2024-09-16T15:48:21.985459+0000","last_unstale":"2024-09-16T15:48:23.351748+0000","last_undegraded":"2024-09-16T15:48:23.351748+0000","last_fullsized":"2024-09-16T15:48:23.351748+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:38:23.345237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.991387+0000","last_change":"2024-09-16T15:48:21.990276+0000","last_active":"2024-09-16T15:48:22.991387+0000","last_peered":"2024-09-16T15:48:22.991387+0000","last_clean":"2024-09-16T15:48:22.991387+0000","last_became_active":"2024-09-16T15:48:21.989558+0000","last_became_peered":"2024-09-16T15:48:21.989558+0000","last_unstale":"2024-09-16T15:48:22.991387+0000","last_undegraded":"2024-09-16T15:48:22.991387+0000","last_fullsized":"2024-09-16T15:48:22.991387+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:48:19.694340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.991799+0000","last_change":"2024-09-16T15:48:21.990934+0000","last_active":"2024-09-16T15:48:22.991799+0000","last_peered":"2024-09-16T15:48:22.991799+0000","last_clean":"2024-09-16T15:48:22.991799+0000","last_became_active":"2024-09-16T15:48:21.990668+0000","last_became_peered":"2024-09-16T15:48:21.990668+0000","last_unstale":"2024-09-16T15:48:22.991799+0000","last_undegraded":"2024-09-16T15:48:22.991799+0000","last_fullsized":"2024-09-16T15:48:22.991799+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:09:12.283305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.976198+0000","last_change":"2024-09-16T15:48:21.980489+0000","last_active":"2024-09-16T15:48:22.976198+0000","last_peered":"2024-09-16T15:48:22.976198+0000","last_clean":"2024-09-16T15:48:22.976198+0000","last_became_active":"2024-09-16T15:48:21.980231+0000","last_became_peered":"2024-09-16T15:48:21.980231+0000","last_unstale":"2024-09-16T15:48:22.976198+0000","last_undegraded":"2024-09-16T15:48:22.976198+0000","last_fullsized":"2024-09-16T15:48:22.976198+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:52:21.377556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.976037+0000","last_change":"2024-09-16T15:48:21.985490+0000","last_active":"2024-09-16T15:48:22.976037+0000","last_peered":"2024-09-16T15:48:22.976037+0000","last_clean":"2024-09-16T15:48:22.976037+0000","last_became_active":"2024-09-16T15:48:21.985234+0000","last_became_peered":"2024-09-16T15:48:21.985234+0000","last_unstale":"2024-09-16T15:48:22.976037+0000","last_undegraded":"2024-09-16T15:48:22.976037+0000","last_fullsized":"2024-09-16T15:48:22.976037+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:55:04.959013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352909+0000","last_change":"2024-09-16T15:48:21.984672+0000","last_active":"2024-09-16T15:48:23.352909+0000","last_peered":"2024-09-16T15:48:23.352909+0000","last_clean":"2024-09-16T15:48:23.352909+0000","last_became_active":"2024-09-16T15:48:21.984399+0000","last_became_peered":"2024-09-16T15:48:21.984399+0000","last_unstale":"2024-09-16T15:48:23.352909+0000","last_undegraded":"2024-09-16T15:48:23.352909+0000","last_fullsized":"2024-09-16T15:48:23.352909+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:09:02.201894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.980034+0000","last_change":"2024-09-16T15:48:21.973853+0000","last_active":"2024-09-16T15:48:22.980034+0000","last_peered":"2024-09-16T15:48:22.980034+0000","last_clean":"2024-09-16T15:48:22.980034+0000","last_became_active":"2024-09-16T15:48:21.973569+0000","last_became_peered":"2024-09-16T15:48:21.973569+0000","last_unstale":"2024-09-16T15:48:22.980034+0000","last_undegraded":"2024-09-16T15:48:22.980034+0000","last_fullsized":"2024-09-16T15:48:22.980034+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:39:27.489599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352274+0000","last_change":"2024-09-16T15:48:21.991963+0000","last_active":"2024-09-16T15:48:23.352274+0000","last_peered":"2024-09-16T15:48:23.352274+0000","last_clean":"2024-09-16T15:48:23.352274+0000","last_became_active":"2024-09-16T15:48:21.991698+0000","last_became_peered":"2024-09-16T15:48:21.991698+0000","last_unstale":"2024-09-16T15:48:23.352274+0000","last_undegraded":"2024-09-16T15:48:23.352274+0000","last_fullsized":"2024-09-16T15:48:23.352274+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:21:15.053411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.350962+0000","last_change":"2024-09-16T15:48:21.987538+0000","last_active":"2024-09-16T15:48:23.350962+0000","last_peered":"2024-09-16T15:48:23.350962+0000","last_clean":"2024-09-16T15:48:23.350962+0000","last_became_active":"2024-09-16T15:48:21.987288+0000","last_became_peered":"2024-09-16T15:48:21.987288+0000","last_unstale":"2024-09-16T15:48:23.350962+0000","last_undegraded":"2024-09-16T15:48:23.350962+0000","last_fullsized":"2024-09-16T15:48:23.350962+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:19:07.343152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.351252+0000","last_change":"2024-09-16T15:48:21.988285+0000","last_active":"2024-09-16T15:48:23.351252+0000","last_peered":"2024-09-16T15:48:23.351252+0000","last_clean":"2024-09-16T15:48:23.351252+0000","last_became_active":"2024-09-16T15:48:21.988029+0000","last_became_peered":"2024-09-16T15:48:21.988029+0000","last_unstale":"2024-09-16T15:48:23.351252+0000","last_undegraded":"2024-09-16T15:48:23.351252+0000","last_fullsized":"2024-09-16T15:48:23.351252+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:16:05.042149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.353150+0000","last_change":"2024-09-16T15:48:21.981145+0000","last_active":"2024-09-16T15:48:23.353150+0000","last_peered":"2024-09-16T15:48:23.353150+0000","last_clean":"2024-09-16T15:48:23.353150+0000","last_became_active":"2024-09-16T15:48:21.980840+0000","last_became_peered":"2024-09-16T15:48:21.980840+0000","last_unstale":"2024-09-16T15:48:23.353150+0000","last_undegraded":"2024-09-16T15:48:23.353150+0000","last_fullsized":"2024-09-16T15:48:23.353150+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:57:23.445821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"90'4","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.976472+0000","last_change":"2024-09-16T15:48:20.975503+0000","last_active":"2024-09-16T15:48:22.976472+0000","last_peered":"2024-09-16T15:48:22.976472+0000","last_clean":"2024-09-16T15:48:22.976472+0000","last_became_active":"2024-09-16T15:48:20.973819+0000","last_became_peered":"2024-09-16T15:48:20.973819+0000","last_unstale":"2024-09-16T15:48:22.976472+0000","last_undegraded":"2024-09-16T15:48:22.976472+0000","last_fullsized":"2024-09-16T15:48:22.976472+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:36:41.387478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.353295+0000","last_change":"2024-09-16T15:48:21.997837+0000","last_active":"2024-09-16T15:48:23.353295+0000","last_peered":"2024-09-16T15:48:23.353295+0000","last_clean":"2024-09-16T15:48:23.353295+0000","last_became_active":"2024-09-16T15:48:21.997316+0000","last_became_peered":"2024-09-16T15:48:21.997316+0000","last_unstale":"2024-09-16T15:48:23.353295+0000","last_undegraded":"2024-09-16T15:48:23.353295+0000","last_fullsized":"2024-09-16T15:48:23.353295+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:48:19.694340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"90'1","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352734+0000","last_change":"2024-09-16T15:48:20.974951+0000","last_active":"2024-09-16T15:48:23.352734+0000","last_peered":"2024-09-16T15:48:23.352734+0000","last_clean":"2024-09-16T15:48:23.352734+0000","last_became_active":"2024-09-16T15:48:20.974688+0000","last_became_peered":"2024-09-16T15:48:20.974688+0000","last_unstale":"2024-09-16T15:48:23.352734+0000","last_undegraded":"2024-09-16T15:48:23.352734+0000","last_fullsized":"2024-09-16T15:48:23.352734+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:54:06.530879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352482+0000","last_change":"2024-09-16T15:48:21.990313+0000","last_active":"2024-09-16T15:48:23.352482+0000","last_peered":"2024-09-16T15:48:23.352482+0000","last_clean":"2024-09-16T15:48:23.352482+0000","last_became_active":"2024-09-16T15:48:21.988356+0000","last_became_peered":"2024-09-16T15:48:21.988356+0000","last_unstale":"2024-09-16T15:48:23.352482+0000","last_undegraded":"2024-09-16T15:48:23.352482+0000","last_fullsized":"2024-09-16T15:48:23.352482+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:16:05.042149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"90'2","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.991491+0000","last_change":"2024-09-16T15:48:20.978653+0000","last_active":"2024-09-16T15:48:22.991491+0000","last_peered":"2024-09-16T15:48:22.991491+0000","last_clean":"2024-09-16T15:48:22.991491+0000","last_became_active":"2024-09-16T15:48:20.976364+0000","last_became_peered":"2024-09-16T15:48:20.976364+0000","last_unstale":"2024-09-16T15:48:22.991491+0000","last_undegraded":"2024-09-16T15:48:22.991491+0000","last_fullsized":"2024-09-16T15:48:22.991491+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:23:03.943492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.353301+0000","last_change":"2024-09-16T15:48:21.998745+0000","last_active":"2024-09-16T15:48:23.353301+0000","last_peered":"2024-09-16T15:48:23.353301+0000","last_clean":"2024-09-16T15:48:23.353301+0000","last_became_active":"2024-09-16T15:48:21.997785+0000","last_became_peered":"2024-09-16T15:48:21.997785+0000","last_unstale":"2024-09-16T15:48:23.353301+0000","last_undegraded":"2024-09-16T15:48:23.353301+0000","last_fullsized":"2024-09-16T15:48:23.353301+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:21:37.652682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"90'2","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.980371+0000","last_change":"2024-09-16T15:48:20.967236+0000","last_active":"2024-09-16T15:48:22.980371+0000","last_peered":"2024-09-16T15:48:22.980371+0000","last_clean":"2024-09-16T15:48:22.980371+0000","last_became_active":"2024-09-16T15:48:20.966954+0000","last_became_peered":"2024-09-16T15:48:20.966954+0000","last_unstale":"2024-09-16T15:48:22.980371+0000","last_undegraded":"2024-09-16T15:48:22.980371+0000","last_fullsized":"2024-09-16T15:48:22.980371+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:06:56.198600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.980724+0000","last_change":"2024-09-16T15:48:21.993949+0000","last_active":"2024-09-16T15:48:22.980724+0000","last_peered":"2024-09-16T15:48:22.980724+0000","last_clean":"2024-09-16T15:48:22.980724+0000","last_became_active":"2024-09-16T15:48:21.993530+0000","last_became_peered":"2024-09-16T15:48:21.993530+0000","last_unstale":"2024-09-16T15:48:22.980724+0000","last_undegraded":"2024-09-16T15:48:22.980724+0000","last_fullsized":"2024-09-16T15:48:22.980724+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:44:18.885274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1b","version":"90'1","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352248+0000","last_change":"2024-09-16T15:48:20.975041+0000","last_active":"2024-09-16T15:48:23.352248+0000","last_peered":"2024-09-16T15:48:23.352248+0000","last_clean":"2024-09-16T15:48:23.352248+0000","last_became_active":"2024-09-16T15:48:20.971719+0000","last_became_peered":"2024-09-16T15:48:20.971719+0000","last_unstale":"2024-09-16T15:48:23.352248+0000","last_undegraded":"2024-09-16T15:48:23.352248+0000","last_fullsized":"2024-09-16T15:48:23.352248+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:13:03.816294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.351424+0000","last_change":"2024-09-16T15:48:21.988933+0000","last_active":"2024-09-16T15:48:23.351424+0000","last_peered":"2024-09-16T15:48:23.351424+0000","last_clean":"2024-09-16T15:48:23.351424+0000","last_became_active":"2024-09-16T15:48:21.988681+0000","last_became_peered":"2024-09-16T15:48:21.988681+0000","last_unstale":"2024-09-16T15:48:23.351424+0000","last_undegraded":"2024-09-16T15:48:23.351424+0000","last_fullsized":"2024-09-16T15:48:23.351424+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:47:53.347929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1a","version":"90'1","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.351139+0000","last_change":"2024-09-16T15:48:20.971997+0000","last_active":"2024-09-16T15:48:23.351139+0000","last_peered":"2024-09-16T15:48:23.351139+0000","last_clean":"2024-09-16T15:48:23.351139+0000","last_became_active":"2024-09-16T15:48:20.971230+0000","last_became_peered":"2024-09-16T15:48:20.971230+0000","last_unstale":"2024-09-16T15:48:23.351139+0000","last_undegraded":"2024-09-16T15:48:23.351139+0000","last_fullsized":"2024-09-16T15:48:23.351139+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:13:03.816294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.351819+0000","last_change":"2024-09-16T15:48:21.996922+0000","last_active":"2024-09-16T15:48:23.351819+0000","last_peered":"2024-09-16T15:48:23.351819+0000","last_clean":"2024-09-16T15:48:23.351819+0000","last_became_active":"2024-09-16T15:48:21.996633+0000","last_became_peered":"2024-09-16T15:48:21.996633+0000","last_unstale":"2024-09-16T15:48:23.351819+0000","last_undegraded":"2024-09-16T15:48:23.351819+0000","last_fullsized":"2024-09-16T15:48:23.351819+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:17:26.941297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.351663+0000","last_change":"2024-09-16T15:48:20.977058+0000","last_active":"2024-09-16T15:48:23.351663+0000","last_peered":"2024-09-16T15:48:23.351663+0000","last_clean":"2024-09-16T15:48:23.351663+0000","last_became_active":"2024-09-16T15:48:20.975591+0000","last_became_peered":"2024-09-16T15:48:20.975591+0000","last_unstale":"2024-09-16T15:48:23.351663+0000","last_undegraded":"2024-09-16T15:48:23.351663+0000","last_fullsized":"2024-09-16T15:48:23.351663+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:58:51.576740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.351992+0000","last_change":"2024-09-16T15:48:21.987074+0000","last_active":"2024-09-16T15:48:23.351992+0000","last_peered":"2024-09-16T15:48:23.351992+0000","last_clean":"2024-09-16T15:48:23.351992+0000","last_became_active":"2024-09-16T15:48:21.986779+0000","last_became_peered":"2024-09-16T15:48:21.986779+0000","last_unstale":"2024-09-16T15:48:23.351992+0000","last_undegraded":"2024-09-16T15:48:23.351992+0000","last_fullsized":"2024-09-16T15:48:23.351992+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:57:02.685046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.351536+0000","last_change":"2024-09-16T15:48:20.977637+0000","last_active":"2024-09-16T15:48:23.351536+0000","last_peered":"2024-09-16T15:48:23.351536+0000","last_clean":"2024-09-16T15:48:23.351536+0000","last_became_active":"2024-09-16T15:48:20.977367+0000","last_became_peered":"2024-09-16T15:48:20.977367+0000","last_unstale":"2024-09-16T15:48:23.351536+0000","last_undegraded":"2024-09-16T15:48:23.351536+0000","last_fullsized":"2024-09-16T15:48:23.351536+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:57:12.992776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352256+0000","last_change":"2024-09-16T15:48:21.990047+0000","last_active":"2024-09-16T15:48:23.352256+0000","last_peered":"2024-09-16T15:48:23.352256+0000","last_clean":"2024-09-16T15:48:23.352256+0000","last_became_active":"2024-09-16T15:48:21.989735+0000","last_became_peered":"2024-09-16T15:48:21.989735+0000","last_unstale":"2024-09-16T15:48:23.352256+0000","last_undegraded":"2024-09-16T15:48:23.352256+0000","last_fullsized":"2024-09-16T15:48:23.352256+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:50:37.046745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.353336+0000","last_change":"2024-09-16T15:48:20.969036+0000","last_active":"2024-09-16T15:48:23.353336+0000","last_peered":"2024-09-16T15:48:23.353336+0000","last_clean":"2024-09-16T15:48:23.353336+0000","last_became_active":"2024-09-16T15:48:20.968771+0000","last_became_peered":"2024-09-16T15:48:20.968771+0000","last_unstale":"2024-09-16T15:48:23.353336+0000","last_undegraded":"2024-09-16T15:48:23.353336+0000","last_fullsized":"2024-09-16T15:48:23.353336+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:47:02.673874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.353252+0000","last_change":"2024-09-16T15:48:21.997579+0000","last_active":"2024-09-16T15:48:23.353252+0000","last_peered":"2024-09-16T15:48:23.353252+0000","last_clean":"2024-09-16T15:48:23.353252+0000","last_became_active":"2024-09-16T15:48:21.997157+0000","last_became_peered":"2024-09-16T15:48:21.997157+0000","last_unstale":"2024-09-16T15:48:23.353252+0000","last_undegraded":"2024-09-16T15:48:23.353252+0000","last_fullsized":"2024-09-16T15:48:23.353252+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:17:00.610181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352849+0000","last_change":"2024-09-16T15:48:20.968246+0000","last_active":"2024-09-16T15:48:23.352849+0000","last_peered":"2024-09-16T15:48:23.352849+0000","last_clean":"2024-09-16T15:48:23.352849+0000","last_became_active":"2024-09-16T15:48:20.967963+0000","last_became_peered":"2024-09-16T15:48:20.967963+0000","last_unstale":"2024-09-16T15:48:23.352849+0000","last_undegraded":"2024-09-16T15:48:23.352849+0000","last_fullsized":"2024-09-16T15:48:23.352849+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:54:40.467435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.991369+0000","last_change":"2024-09-16T15:48:21.990983+0000","last_active":"2024-09-16T15:48:22.991369+0000","last_peered":"2024-09-16T15:48:22.991369+0000","last_clean":"2024-09-16T15:48:22.991369+0000","last_became_active":"2024-09-16T15:48:21.990728+0000","last_became_peered":"2024-09-16T15:48:21.990728+0000","last_unstale":"2024-09-16T15:48:22.991369+0000","last_undegraded":"2024-09-16T15:48:22.991369+0000","last_fullsized":"2024-09-16T15:48:22.991369+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:45:22.428267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.15","version":"90'3","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.353396+0000","last_change":"2024-09-16T15:48:20.975763+0000","last_active":"2024-09-16T15:48:23.353396+0000","last_peered":"2024-09-16T15:48:23.353396+0000","last_clean":"2024-09-16T15:48:23.353396+0000","last_became_active":"2024-09-16T15:48:20.974952+0000","last_became_peered":"2024-09-16T15:48:20.974952+0000","last_unstale":"2024-09-16T15:48:23.353396+0000","last_undegraded":"2024-09-16T15:48:23.353396+0000","last_fullsized":"2024-09-16T15:48:23.353396+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:31:15.757130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.975956+0000","last_change":"2024-09-16T15:48:21.995765+0000","last_active":"2024-09-16T15:48:22.975956+0000","last_peered":"2024-09-16T15:48:22.975956+0000","last_clean":"2024-09-16T15:48:22.975956+0000","last_became_active":"2024-09-16T15:48:21.995445+0000","last_became_peered":"2024-09-16T15:48:21.995445+0000","last_unstale":"2024-09-16T15:48:22.975956+0000","last_undegraded":"2024-09-16T15:48:22.975956+0000","last_fullsized":"2024-09-16T15:48:22.975956+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:33:32.580089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"90'6","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352532+0000","last_change":"2024-09-16T15:48:20.972430+0000","last_active":"2024-09-16T15:48:23.352532+0000","last_peered":"2024-09-16T15:48:23.352532+0000","last_clean":"2024-09-16T15:48:23.352532+0000","last_became_active":"2024-09-16T15:48:20.972159+0000","last_became_peered":"2024-09-16T15:48:20.972159+0000","last_unstale":"2024-09-16T15:48:23.352532+0000","last_undegraded":"2024-09-16T15:48:23.352532+0000","last_fullsized":"2024-09-16T15:48:23.352532+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:54:06.530879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.991316+0000","last_change":"2024-09-16T15:48:21.990590+0000","last_active":"2024-09-16T15:48:22.991316+0000","last_peered":"2024-09-16T15:48:22.991316+0000","last_clean":"2024-09-16T15:48:22.991316+0000","last_became_active":"2024-09-16T15:48:21.989821+0000","last_became_peered":"2024-09-16T15:48:21.989821+0000","last_unstale":"2024-09-16T15:48:22.991316+0000","last_undegraded":"2024-09-16T15:48:22.991316+0000","last_fullsized":"2024-09-16T15:48:22.991316+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:17:00.610181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.13","version":"90'2","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.980206+0000","last_change":"2024-09-16T15:48:20.965556+0000","last_active":"2024-09-16T15:48:22.980206+0000","last_peered":"2024-09-16T15:48:22.980206+0000","last_clean":"2024-09-16T15:48:22.980206+0000","last_became_active":"2024-09-16T15:48:20.965196+0000","last_became_peered":"2024-09-16T15:48:20.965196+0000","last_unstale":"2024-09-16T15:48:22.980206+0000","last_undegraded":"2024-09-16T15:48:22.980206+0000","last_fullsized":"2024-09-16T15:48:22.980206+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:05.970213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352808+0000","last_change":"2024-09-16T15:48:21.993233+0000","last_active":"2024-09-16T15:48:23.352808+0000","last_peered":"2024-09-16T15:48:23.352808+0000","last_clean":"2024-09-16T15:48:23.352808+0000","last_became_active":"2024-09-16T15:48:21.992915+0000","last_became_peered":"2024-09-16T15:48:21.992915+0000","last_unstale":"2024-09-16T15:48:23.352808+0000","last_undegraded":"2024-09-16T15:48:23.352808+0000","last_fullsized":"2024-09-16T15:48:23.352808+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:36:34.887933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.980104+0000","last_change":"2024-09-16T15:48:20.969210+0000","last_active":"2024-09-16T15:48:22.980104+0000","last_peered":"2024-09-16T15:48:22.980104+0000","last_clean":"2024-09-16T15:48:22.980104+0000","last_became_active":"2024-09-16T15:48:20.968946+0000","last_became_peered":"2024-09-16T15:48:20.968946+0000","last_unstale":"2024-09-16T15:48:22.980104+0000","last_undegraded":"2024-09-16T15:48:22.980104+0000","last_fullsized":"2024-09-16T15:48:22.980104+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:40:35.123939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352100+0000","last_change":"2024-09-16T15:48:21.997110+0000","last_active":"2024-09-16T15:48:23.352100+0000","last_peered":"2024-09-16T15:48:23.352100+0000","last_clean":"2024-09-16T15:48:23.352100+0000","last_became_active":"2024-09-16T15:48:21.996855+0000","last_became_peered":"2024-09-16T15:48:21.996855+0000","last_unstale":"2024-09-16T15:48:23.352100+0000","last_undegraded":"2024-09-16T15:48:23.352100+0000","last_fullsized":"2024-09-16T15:48:23.352100+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:44:31.136306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.350997+0000","last_change":"2024-09-16T15:48:20.969040+0000","last_active":"2024-09-16T15:48:23.350997+0000","last_peered":"2024-09-16T15:48:23.350997+0000","last_clean":"2024-09-16T15:48:23.350997+0000","last_became_active":"2024-09-16T15:48:20.968541+0000","last_became_peered":"2024-09-16T15:48:20.968541+0000","last_unstale":"2024-09-16T15:48:23.350997+0000","last_undegraded":"2024-09-16T15:48:23.350997+0000","last_fullsized":"2024-09-16T15:48:23.350997+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:32:44.623249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.991116+0000","last_change":"2024-09-16T15:48:21.989170+0000","last_active":"2024-09-16T15:48:22.991116+0000","last_peered":"2024-09-16T15:48:22.991116+0000","last_clean":"2024-09-16T15:48:22.991116+0000","last_became_active":"2024-09-16T15:48:21.988901+0000","last_became_peered":"2024-09-16T15:48:21.988901+0000","last_unstale":"2024-09-16T15:48:22.991116+0000","last_undegraded":"2024-09-16T15:48:22.991116+0000","last_fullsized":"2024-09-16T15:48:22.991116+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:16:58.292432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"90'8","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.991610+0000","last_change":"2024-09-16T15:48:20.975861+0000","last_active":"2024-09-16T15:48:22.991610+0000","last_peered":"2024-09-16T15:48:22.991610+0000","last_clean":"2024-09-16T15:48:22.991610+0000","last_became_active":"2024-09-16T15:48:20.974746+0000","last_became_peered":"2024-09-16T15:48:20.974746+0000","last_unstale":"2024-09-16T15:48:22.991610+0000","last_undegraded":"2024-09-16T15:48:22.991610+0000","last_fullsized":"2024-09-16T15:48:22.991610+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:11:13.839123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.353333+0000","last_change":"2024-09-16T15:48:21.990395+0000","last_active":"2024-09-16T15:48:23.353333+0000","last_peered":"2024-09-16T15:48:23.353333+0000","last_clean":"2024-09-16T15:48:23.353333+0000","last_became_active":"2024-09-16T15:48:21.990144+0000","last_became_peered":"2024-09-16T15:48:21.990144+0000","last_unstale":"2024-09-16T15:48:23.353333+0000","last_undegraded":"2024-09-16T15:48:23.353333+0000","last_fullsized":"2024-09-16T15:48:23.353333+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:02:44.937439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"90'1","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.976584+0000","last_change":"2024-09-16T15:48:20.975777+0000","last_active":"2024-09-16T15:48:22.976584+0000","last_peered":"2024-09-16T15:48:22.976584+0000","last_clean":"2024-09-16T15:48:22.976584+0000","last_became_active":"2024-09-16T15:48:20.974162+0000","last_became_peered":"2024-09-16T15:48:20.974162+0000","last_unstale":"2024-09-16T15:48:22.976584+0000","last_undegraded":"2024-09-16T15:48:22.976584+0000","last_fullsized":"2024-09-16T15:48:22.976584+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:44:17.877841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352625+0000","last_change":"2024-09-16T15:48:21.986577+0000","last_active":"2024-09-16T15:48:23.352625+0000","last_peered":"2024-09-16T15:48:23.352625+0000","last_clean":"2024-09-16T15:48:23.352625+0000","last_became_active":"2024-09-16T15:48:21.986333+0000","last_became_peered":"2024-09-16T15:48:21.986333+0000","last_unstale":"2024-09-16T15:48:23.352625+0000","last_undegraded":"2024-09-16T15:48:23.352625+0000","last_fullsized":"2024-09-16T15:48:23.352625+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:02:44.937439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"90'2","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352068+0000","last_change":"2024-09-16T15:48:20.975158+0000","last_active":"2024-09-16T15:48:23.352068+0000","last_peered":"2024-09-16T15:48:23.352068+0000","last_clean":"2024-09-16T15:48:23.352068+0000","last_became_active":"2024-09-16T15:48:20.974854+0000","last_became_peered":"2024-09-16T15:48:20.974854+0000","last_unstale":"2024-09-16T15:48:23.352068+0000","last_undegraded":"2024-09-16T15:48:23.352068+0000","last_fullsized":"2024-09-16T15:48:23.352068+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:41:44.873886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352765+0000","last_change":"2024-09-16T15:48:21.991352+0000","last_active":"2024-09-16T15:48:23.352765+0000","last_peered":"2024-09-16T15:48:23.352765+0000","last_clean":"2024-09-16T15:48:23.352765+0000","last_became_active":"2024-09-16T15:48:21.991096+0000","last_became_peered":"2024-09-16T15:48:21.991096+0000","last_unstale":"2024-09-16T15:48:23.352765+0000","last_undegraded":"2024-09-16T15:48:23.352765+0000","last_fullsized":"2024-09-16T15:48:23.352765+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:07:20.943835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352234+0000","last_change":"2024-09-16T15:48:20.978031+0000","last_active":"2024-09-16T15:48:23.352234+0000","last_peered":"2024-09-16T15:48:23.352234+0000","last_clean":"2024-09-16T15:48:23.352234+0000","last_became_active":"2024-09-16T15:48:20.976491+0000","last_became_peered":"2024-09-16T15:48:20.976491+0000","last_unstale":"2024-09-16T15:48:23.352234+0000","last_undegraded":"2024-09-16T15:48:23.352234+0000","last_fullsized":"2024-09-16T15:48:23.352234+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:57.979104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.979914+0000","last_change":"2024-09-16T15:48:21.993773+0000","last_active":"2024-09-16T15:48:22.979914+0000","last_peered":"2024-09-16T15:48:22.979914+0000","last_clean":"2024-09-16T15:48:22.979914+0000","last_became_active":"2024-09-16T15:48:21.993443+0000","last_became_peered":"2024-09-16T15:48:21.993443+0000","last_unstale":"2024-09-16T15:48:22.979914+0000","last_undegraded":"2024-09-16T15:48:22.979914+0000","last_fullsized":"2024-09-16T15:48:22.979914+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:30:15.463014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.b","version":"90'2","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.351800+0000","last_change":"2024-09-16T15:48:20.976944+0000","last_active":"2024-09-16T15:48:23.351800+0000","last_peered":"2024-09-16T15:48:23.351800+0000","last_clean":"2024-09-16T15:48:23.351800+0000","last_became_active":"2024-09-16T15:48:20.976649+0000","last_became_peered":"2024-09-16T15:48:20.976649+0000","last_unstale":"2024-09-16T15:48:23.351800+0000","last_undegraded":"2024-09-16T15:48:23.351800+0000","last_fullsized":"2024-09-16T15:48:23.351800+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:33:01.893406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.976168+0000","last_change":"2024-09-16T15:48:21.994077+0000","last_active":"2024-09-16T15:48:22.976168+0000","last_peered":"2024-09-16T15:48:22.976168+0000","last_clean":"2024-09-16T15:48:22.976168+0000","last_became_active":"2024-09-16T15:48:21.992548+0000","last_became_peered":"2024-09-16T15:48:21.992548+0000","last_unstale":"2024-09-16T15:48:22.976168+0000","last_undegraded":"2024-09-16T15:48:22.976168+0000","last_fullsized":"2024-09-16T15:48:22.976168+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:25:24.140562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.980214+0000","last_change":"2024-09-16T15:48:20.970601+0000","last_active":"2024-09-16T15:48:22.980214+0000","last_peered":"2024-09-16T15:48:22.980214+0000","last_clean":"2024-09-16T15:48:22.980214+0000","last_became_active":"2024-09-16T15:48:20.970272+0000","last_became_peered":"2024-09-16T15:48:20.970272+0000","last_unstale":"2024-09-16T15:48:22.980214+0000","last_undegraded":"2024-09-16T15:48:22.980214+0000","last_fullsized":"2024-09-16T15:48:22.980214+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:42:55.329422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.980294+0000","last_change":"2024-09-16T15:48:21.978436+0000","last_active":"2024-09-16T15:48:22.980294+0000","last_peered":"2024-09-16T15:48:22.980294+0000","last_clean":"2024-09-16T15:48:22.980294+0000","last_became_active":"2024-09-16T15:48:21.977384+0000","last_became_peered":"2024-09-16T15:48:21.977384+0000","last_unstale":"2024-09-16T15:48:22.980294+0000","last_undegraded":"2024-09-16T15:48:22.980294+0000","last_fullsized":"2024-09-16T15:48:22.980294+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:34:53.674221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.991107+0000","last_change":"2024-09-16T15:48:20.971710+0000","last_active":"2024-09-16T15:48:22.991107+0000","last_peered":"2024-09-16T15:48:22.991107+0000","last_clean":"2024-09-16T15:48:22.991107+0000","last_became_active":"2024-09-16T15:48:20.971360+0000","last_became_peered":"2024-09-16T15:48:20.971360+0000","last_unstale":"2024-09-16T15:48:22.991107+0000","last_undegraded":"2024-09-16T15:48:22.991107+0000","last_fullsized":"2024-09-16T15:48:22.991107+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:02:25.192877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.351953+0000","last_change":"2024-09-16T15:48:21.975033+0000","last_active":"2024-09-16T15:48:23.351953+0000","last_peered":"2024-09-16T15:48:23.351953+0000","last_clean":"2024-09-16T15:48:23.351953+0000","last_became_active":"2024-09-16T15:48:21.974636+0000","last_became_peered":"2024-09-16T15:48:21.974636+0000","last_unstale":"2024-09-16T15:48:23.351953+0000","last_undegraded":"2024-09-16T15:48:23.351953+0000","last_fullsized":"2024-09-16T15:48:23.351953+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:32:12.555452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.351750+0000","last_change":"2024-09-16T15:48:20.980185+0000","last_active":"2024-09-16T15:48:23.351750+0000","last_peered":"2024-09-16T15:48:23.351750+0000","last_clean":"2024-09-16T15:48:23.351750+0000","last_became_active":"2024-09-16T15:48:20.979875+0000","last_became_peered":"2024-09-16T15:48:20.979875+0000","last_unstale":"2024-09-16T15:48:23.351750+0000","last_undegraded":"2024-09-16T15:48:23.351750+0000","last_fullsized":"2024-09-16T15:48:23.351750+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:56:29.587401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.976320+0000","last_change":"2024-09-16T15:48:21.988229+0000","last_active":"2024-09-16T15:48:22.976320+0000","last_peered":"2024-09-16T15:48:22.976320+0000","last_clean":"2024-09-16T15:48:22.976320+0000","last_became_active":"2024-09-16T15:48:21.987857+0000","last_became_peered":"2024-09-16T15:48:21.987857+0000","last_unstale":"2024-09-16T15:48:22.976320+0000","last_undegraded":"2024-09-16T15:48:22.976320+0000","last_fullsized":"2024-09-16T15:48:22.976320+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:40:50.356201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"90'2","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352056+0000","last_change":"2024-09-16T15:48:20.978563+0000","last_active":"2024-09-16T15:48:23.352056+0000","last_peered":"2024-09-16T15:48:23.352056+0000","last_clean":"2024-09-16T15:48:23.352056+0000","last_became_active":"2024-09-16T15:48:20.978188+0000","last_became_peered":"2024-09-16T15:48:20.978188+0000","last_unstale":"2024-09-16T15:48:23.352056+0000","last_undegraded":"2024-09-16T15:48:23.352056+0000","last_fullsized":"2024-09-16T15:48:23.352056+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:28:27.584567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.976444+0000","last_change":"2024-09-16T15:48:21.983190+0000","last_active":"2024-09-16T15:48:22.976444+0000","last_peered":"2024-09-16T15:48:22.976444+0000","last_clean":"2024-09-16T15:48:22.976444+0000","last_became_active":"2024-09-16T15:48:21.982916+0000","last_became_peered":"2024-09-16T15:48:21.982916+0000","last_unstale":"2024-09-16T15:48:22.976444+0000","last_undegraded":"2024-09-16T15:48:22.976444+0000","last_fullsized":"2024-09-16T15:48:22.976444+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:55:24.014926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"91'1","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.991689+0000","last_change":"2024-09-16T15:48:20.975435+0000","last_active":"2024-09-16T15:48:22.991689+0000","last_peered":"2024-09-16T15:48:22.991689+0000","last_clean":"2024-09-16T15:48:22.991689+0000","last_became_active":"2024-09-16T15:48:20.975091+0000","last_became_peered":"2024-09-16T15:48:20.975091+0000","last_unstale":"2024-09-16T15:48:22.991689+0000","last_undegraded":"2024-09-16T15:48:22.991689+0000","last_fullsized":"2024-09-16T15:48:22.991689+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:50:09.821843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.975941+0000","last_change":"2024-09-16T15:48:21.988044+0000","last_active":"2024-09-16T15:48:22.975941+0000","last_peered":"2024-09-16T15:48:22.975941+0000","last_clean":"2024-09-16T15:48:22.975941+0000","last_became_active":"2024-09-16T15:48:21.987788+0000","last_became_peered":"2024-09-16T15:48:21.987788+0000","last_unstale":"2024-09-16T15:48:22.975941+0000","last_undegraded":"2024-09-16T15:48:22.975941+0000","last_fullsized":"2024-09-16T15:48:22.975941+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:54:15.406878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352674+0000","last_change":"2024-09-16T15:48:20.974118+0000","last_active":"2024-09-16T15:48:23.352674+0000","last_peered":"2024-09-16T15:48:23.352674+0000","last_clean":"2024-09-16T15:48:23.352674+0000","last_became_active":"2024-09-16T15:48:20.973841+0000","last_became_peered":"2024-09-16T15:48:20.973841+0000","last_unstale":"2024-09-16T15:48:23.352674+0000","last_undegraded":"2024-09-16T15:48:23.352674+0000","last_fullsized":"2024-09-16T15:48:23.352674+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:54:40.467435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352250+0000","last_change":"2024-09-16T15:48:21.989418+0000","last_active":"2024-09-16T15:48:23.352250+0000","last_peered":"2024-09-16T15:48:23.352250+0000","last_clean":"2024-09-16T15:48:23.352250+0000","last_became_active":"2024-09-16T15:48:21.989166+0000","last_became_peered":"2024-09-16T15:48:21.989166+0000","last_unstale":"2024-09-16T15:48:23.352250+0000","last_undegraded":"2024-09-16T15:48:23.352250+0000","last_fullsized":"2024-09-16T15:48:23.352250+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:12:07.405991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.351511+0000","last_change":"2024-09-16T15:48:20.974113+0000","last_active":"2024-09-16T15:48:23.351511+0000","last_peered":"2024-09-16T15:48:23.351511+0000","last_clean":"2024-09-16T15:48:23.351511+0000","last_became_active":"2024-09-16T15:48:20.973851+0000","last_became_peered":"2024-09-16T15:48:20.973851+0000","last_unstale":"2024-09-16T15:48:23.351511+0000","last_undegraded":"2024-09-16T15:48:23.351511+0000","last_fullsized":"2024-09-16T15:48:23.351511+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:55:33.505049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352485+0000","last_change":"2024-09-16T15:48:21.997253+0000","last_active":"2024-09-16T15:48:23.352485+0000","last_peered":"2024-09-16T15:48:23.352485+0000","last_clean":"2024-09-16T15:48:23.352485+0000","last_became_active":"2024-09-16T15:48:21.996568+0000","last_became_peered":"2024-09-16T15:48:21.996568+0000","last_unstale":"2024-09-16T15:48:23.352485+0000","last_undegraded":"2024-09-16T15:48:23.352485+0000","last_fullsized":"2024-09-16T15:48:23.352485+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:53:59.460386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.980860+0000","last_change":"2024-09-16T15:48:20.968528+0000","last_active":"2024-09-16T15:48:22.980860+0000","last_peered":"2024-09-16T15:48:22.980860+0000","last_clean":"2024-09-16T15:48:22.980860+0000","last_became_active":"2024-09-16T15:48:20.967944+0000","last_became_peered":"2024-09-16T15:48:20.967944+0000","last_unstale":"2024-09-16T15:48:22.980860+0000","last_undegraded":"2024-09-16T15:48:22.980860+0000","last_fullsized":"2024-09-16T15:48:22.980860+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:16:08.669040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.991351+0000","last_change":"2024-09-16T15:48:21.991566+0000","last_active":"2024-09-16T15:48:22.991351+0000","last_peered":"2024-09-16T15:48:22.991351+0000","last_clean":"2024-09-16T15:48:22.991351+0000","last_became_active":"2024-09-16T15:48:21.991274+0000","last_became_peered":"2024-09-16T15:48:21.991274+0000","last_unstale":"2024-09-16T15:48:22.991351+0000","last_undegraded":"2024-09-16T15:48:22.991351+0000","last_fullsized":"2024-09-16T15:48:22.991351+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:23:27.535786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.351253+0000","last_change":"2024-09-16T15:48:20.963312+0000","last_active":"2024-09-16T15:48:23.351253+0000","last_peered":"2024-09-16T15:48:23.351253+0000","last_clean":"2024-09-16T15:48:23.351253+0000","last_became_active":"2024-09-16T15:48:20.963034+0000","last_became_peered":"2024-09-16T15:48:20.963034+0000","last_unstale":"2024-09-16T15:48:23.351253+0000","last_undegraded":"2024-09-16T15:48:23.351253+0000","last_fullsized":"2024-09-16T15:48:23.351253+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:59:53.683980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352072+0000","last_change":"2024-09-16T15:48:21.996871+0000","last_active":"2024-09-16T15:48:23.352072+0000","last_peered":"2024-09-16T15:48:23.352072+0000","last_clean":"2024-09-16T15:48:23.352072+0000","last_became_active":"2024-09-16T15:48:21.996556+0000","last_became_peered":"2024-09-16T15:48:21.996556+0000","last_unstale":"2024-09-16T15:48:23.352072+0000","last_undegraded":"2024-09-16T15:48:23.352072+0000","last_fullsized":"2024-09-16T15:48:23.352072+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:56:48.853231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352292+0000","last_change":"2024-09-16T15:48:20.978028+0000","last_active":"2024-09-16T15:48:23.352292+0000","last_peered":"2024-09-16T15:48:23.352292+0000","last_clean":"2024-09-16T15:48:23.352292+0000","last_became_active":"2024-09-16T15:48:20.977556+0000","last_became_peered":"2024-09-16T15:48:20.977556+0000","last_unstale":"2024-09-16T15:48:23.352292+0000","last_undegraded":"2024-09-16T15:48:23.352292+0000","last_fullsized":"2024-09-16T15:48:23.352292+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:35:16.664943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352088+0000","last_change":"2024-09-16T15:48:21.992228+0000","last_active":"2024-09-16T15:48:23.352088+0000","last_peered":"2024-09-16T15:48:23.352088+0000","last_clean":"2024-09-16T15:48:23.352088+0000","last_became_active":"2024-09-16T15:48:21.991894+0000","last_became_peered":"2024-09-16T15:48:21.991894+0000","last_unstale":"2024-09-16T15:48:23.352088+0000","last_undegraded":"2024-09-16T15:48:23.352088+0000","last_fullsized":"2024-09-16T15:48:23.352088+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:16:09.676473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.351704+0000","last_change":"2024-09-16T15:48:20.976801+0000","last_active":"2024-09-16T15:48:23.351704+0000","last_peered":"2024-09-16T15:48:23.351704+0000","last_clean":"2024-09-16T15:48:23.351704+0000","last_became_active":"2024-09-16T15:48:20.975672+0000","last_became_peered":"2024-09-16T15:48:20.975672+0000","last_unstale":"2024-09-16T15:48:23.351704+0000","last_undegraded":"2024-09-16T15:48:23.351704+0000","last_fullsized":"2024-09-16T15:48:23.351704+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:55:33.505049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.991410+0000","last_change":"2024-09-16T15:48:21.989919+0000","last_active":"2024-09-16T15:48:22.991410+0000","last_peered":"2024-09-16T15:48:22.991410+0000","last_clean":"2024-09-16T15:48:22.991410+0000","last_became_active":"2024-09-16T15:48:21.989572+0000","last_became_peered":"2024-09-16T15:48:21.989572+0000","last_unstale":"2024-09-16T15:48:22.991410+0000","last_undegraded":"2024-09-16T15:48:22.991410+0000","last_fullsized":"2024-09-16T15:48:22.991410+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:57:07.183635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"90'1","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.976109+0000","last_change":"2024-09-16T15:48:20.974835+0000","last_active":"2024-09-16T15:48:22.976109+0000","last_peered":"2024-09-16T15:48:22.976109+0000","last_clean":"2024-09-16T15:48:22.976109+0000","last_became_active":"2024-09-16T15:48:20.974577+0000","last_became_peered":"2024-09-16T15:48:20.974577+0000","last_unstale":"2024-09-16T15:48:22.976109+0000","last_undegraded":"2024-09-16T15:48:22.976109+0000","last_fullsized":"2024-09-16T15:48:22.976109+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:04:45.720134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":242,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352225+0000","last_change":"2024-09-16T15:39:12.698813+0000","last_active":"2024-09-16T15:48:23.352225+0000","last_peered":"2024-09-16T15:48:23.352225+0000","last_clean":"2024-09-16T15:48:23.352225+0000","last_became_active":"2024-09-16T15:39:12.697695+0000","last_became_peered":"2024-09-16T15:39:12.697695+0000","last_unstale":"2024-09-16T15:48:23.352225+0000","last_undegraded":"2024-09-16T15:48:23.352225+0000","last_fullsized":"2024-09-16T15:48:23.352225+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_clean_scrub_stamp":"2024-09-16T15:39:11.684967+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:02.061081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509556,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64152,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307688,"statfs":{"total":96636764160,"available":96571072512,"internally_reserved":0,"allocated":1118208,"data_stored":482183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":64551368},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509555,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63404,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94308436,"statfs":{"total":96636764160,"available":96571838464,"internally_reserved":0,"allocated":1720320,"data_stored":1073525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":63175111},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509554,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64152,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307688,"statfs":{"total":96636764160,"available":96571072512,"internally_reserved":0,"allocated":1114112,"data_stored":481545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509554,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58660,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313180,"statfs":{"total":96636764160,"available":96576696320,"internally_reserved":0,"allocated":1122304,"data_stored":482185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":58915273},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509554,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58640,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313200,"statfs":{"total":96636764160,"available":96576716800,"internally_reserved":0,"allocated":1110016,"data_stored":481511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509555,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59224,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312616,"statfs":{"total":96636764160,"available":96576118784,"internally_reserved":0,"allocated":1708032,"data_stored":1071969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509553,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58656,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313184,"statfs":{"total":96636764160,"available":96576700416,"internally_reserved":0,"allocated":1126400,"data_stored":483145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542260,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62804,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94309036,"statfs":{"total":96636764160,"available":96572452864,"internally_reserved":0,"allocated":1114112,"data_stored":481581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":63175116},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:48:30.894 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:48:31.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.061+0000 7f8484600640 1 -- 172.21.15.151:0/2830015382 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f847c15f5a0 msgr2=0x7f847c0b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:31.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.061+0000 7f8484600640 1 --2- 172.21.15.151:0/2830015382 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f847c15f5a0 0x7f847c0b2440 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f847c06dac0 tx=0x7f847002a4b0 comp rx=0 tx=0).stop 2024-09-16T15:48:31.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.062+0000 7f8484600640 1 -- 172.21.15.151:0/2830015382 shutdown_connections 2024-09-16T15:48:31.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.062+0000 7f8484600640 1 --2- 172.21.15.151:0/2830015382 >> v2:172.21.15.167:3300/0 conn(0x7f847c073fb0 0x7f847c074420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:31.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.062+0000 7f8484600640 1 --2- 172.21.15.151:0/2830015382 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f847c15f5a0 0x7f847c0b2440 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:31.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.062+0000 7f8484600640 1 -- 172.21.15.151:0/2830015382 >> 172.21.15.151:0/2830015382 conn(0x7f847c0b1950 msgr2=0x7f847c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:31.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.062+0000 7f8484600640 1 -- 172.21.15.151:0/2830015382 shutdown_connections 2024-09-16T15:48:31.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.062+0000 7f8484600640 1 -- 172.21.15.151:0/2830015382 wait complete. 2024-09-16T15:48:31.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.063+0000 7f8484600640 1 Processor -- start 2024-09-16T15:48:31.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.063+0000 7f8484600640 1 -- start start 2024-09-16T15:48:31.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.064+0000 7f8484600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f847c073fb0 0x7f847c0b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:31.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.064+0000 7f8484600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f847c0ccf00 0x7f847c0b3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:31.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.064+0000 7f8484600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f847c15f5a0 0x7f847c156090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:31.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.064+0000 7f8484600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f847c06f2e0 con 0x7f847c0ccf00 2024-09-16T15:48:31.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.064+0000 7f8484600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f847c06f160 con 0x7f847c073fb0 2024-09-16T15:48:31.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.064+0000 7f8484600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f847c06f460 con 0x7f847c15f5a0 2024-09-16T15:48:31.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.065+0000 7f8481200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f847c0ccf00 0x7f847c0b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:31.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.065+0000 7f8481200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f847c0ccf00 0x7f847c0b3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:42822/0 (socket says 172.21.15.151:42822) 2024-09-16T15:48:31.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.065+0000 7f8481200640 1 -- 172.21.15.151:0/2146148146 learned_addr learned my addr 172.21.15.151:0/2146148146 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:48:31.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.065+0000 7f8481c00640 1 --2- 172.21.15.151:0/2146148146 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f847c073fb0 0x7f847c0b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:31.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.065+0000 7f8482600640 1 --2- 172.21.15.151:0/2146148146 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f847c15f5a0 0x7f847c156090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:31.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.065+0000 7f8481200640 1 -- 172.21.15.151:0/2146148146 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f847c15f5a0 msgr2=0x7f847c156090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:31.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.065+0000 7f8481200640 1 --2- 172.21.15.151:0/2146148146 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f847c15f5a0 0x7f847c156090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:31.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.065+0000 7f8481200640 1 -- 172.21.15.151:0/2146148146 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f847c073fb0 msgr2=0x7f847c0b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:31.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.065+0000 7f8481200640 1 --2- 172.21.15.151:0/2146148146 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f847c073fb0 0x7f847c0b2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:31.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.065+0000 7f8481200640 1 -- 172.21.15.151:0/2146148146 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f847c0b38c0 con 0x7f847c0ccf00 2024-09-16T15:48:31.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.065+0000 7f8482600640 1 --2- 172.21.15.151:0/2146148146 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f847c15f5a0 0x7f847c156090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:48:31.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.065+0000 7f8481200640 1 --2- 172.21.15.151:0/2146148146 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f847c0ccf00 0x7f847c0b3290 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f846800c4c0 tx=0x7f846800c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:31.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.065+0000 7f8479600640 1 -- 172.21.15.151:0/2146148146 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8468016020 con 0x7f847c0ccf00 2024-09-16T15:48:31.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.066+0000 7f8479600640 1 -- 172.21.15.151:0/2146148146 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8468012940 con 0x7f847c0ccf00 2024-09-16T15:48:31.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.066+0000 7f8484600640 1 -- 172.21.15.151:0/2146148146 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f847c0b3ba0 con 0x7f847c0ccf00 2024-09-16T15:48:31.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.066+0000 7f8479600640 1 -- 172.21.15.151:0/2146148146 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8468012d90 con 0x7f847c0ccf00 2024-09-16T15:48:31.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.066+0000 7f8484600640 1 -- 172.21.15.151:0/2146148146 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f847c0b4160 con 0x7f847c0ccf00 2024-09-16T15:48:31.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.067+0000 7f8484600640 1 -- 172.21.15.151:0/2146148146 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f847c06f9f0 con 0x7f847c0ccf00 2024-09-16T15:48:31.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.068+0000 7f8479600640 1 -- 172.21.15.151:0/2146148146 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 30) ==== 102868+0+0 (secure 0 0 0) 0x7f8468019020 con 0x7f847c0ccf00 2024-09-16T15:48:31.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.069+0000 7f8479600640 1 --2- 172.21.15.151:0/2146148146 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f845c0799c0 0x7f845c07be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:31.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.069+0000 7f8479600640 1 -- 172.21.15.151:0/2146148146 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7f84680995b0 con 0x7f847c0ccf00 2024-09-16T15:48:31.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.069+0000 7f8481c00640 1 --2- 172.21.15.151:0/2146148146 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f845c0799c0 0x7f845c07be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:31.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.070+0000 7f8481c00640 1 --2- 172.21.15.151:0/2146148146 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f845c0799c0 0x7f845c07be80 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8470034000 tx=0x7f8470004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:31.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.073+0000 7f8479600640 1 -- 172.21.15.151:0/2146148146 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8468020030 con 0x7f847c0ccf00 2024-09-16T15:48:31.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.253+0000 7f8484600640 1 -- 172.21.15.151:0/2146148146 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f847c09de60 con 0x7f845c0799c0 2024-09-16T15:48:31.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.258+0000 7f8479600640 1 -- 172.21.15.151:0/2146148146 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252073 (secure 0 0 0) 0x7f847c09de60 con 0x7f845c0799c0 2024-09-16T15:48:31.263 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:48:31.264 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-09-16T15:48:31.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.262+0000 7f8484600640 1 -- 172.21.15.151:0/2146148146 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f845c0799c0 msgr2=0x7f845c07be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:31.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.262+0000 7f8484600640 1 --2- 172.21.15.151:0/2146148146 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f845c0799c0 0x7f845c07be80 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8470034000 tx=0x7f8470004730 comp rx=0 tx=0).stop 2024-09-16T15:48:31.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.262+0000 7f8484600640 1 -- 172.21.15.151:0/2146148146 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f847c0ccf00 msgr2=0x7f847c0b3290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:31.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.262+0000 7f8484600640 1 --2- 172.21.15.151:0/2146148146 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f847c0ccf00 0x7f847c0b3290 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f846800c4c0 tx=0x7f846800c980 comp rx=0 tx=0).stop 2024-09-16T15:48:31.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.262+0000 7f8484600640 1 -- 172.21.15.151:0/2146148146 shutdown_connections 2024-09-16T15:48:31.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.262+0000 7f8484600640 1 --2- 172.21.15.151:0/2146148146 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f845c0799c0 0x7f845c07be80 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:31.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.262+0000 7f8484600640 1 --2- 172.21.15.151:0/2146148146 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f847c15f5a0 0x7f847c156090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:31.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.262+0000 7f8484600640 1 --2- 172.21.15.151:0/2146148146 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f847c0ccf00 0x7f847c0b3290 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:31.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.262+0000 7f8484600640 1 --2- 172.21.15.151:0/2146148146 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f847c073fb0 0x7f847c0b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:31.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.262+0000 7f8484600640 1 -- 172.21.15.151:0/2146148146 >> 172.21.15.151:0/2146148146 conn(0x7f847c0b1950 msgr2=0x7f847c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:31.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.262+0000 7f8484600640 1 -- 172.21.15.151:0/2146148146 shutdown_connections 2024-09-16T15:48:31.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.262+0000 7f8484600640 1 -- 172.21.15.151:0/2146148146 wait complete. 2024-09-16T15:48:31.280 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":364,"stamp":"2024-09-16T15:48:29.750663+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":173,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":489692,"kb_used_data":9896,"kb_used_omap":12,"kb_used_meta":479603,"kb_avail":754485028,"statfs":{"total":773094113280,"available":772592668672,"internally_reserved":0,"allocated":10133504,"data_stored":5037644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12738,"internal_metadata":491114046},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002123"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.353070+0000","last_change":"2024-09-16T15:48:21.998960+0000","last_active":"2024-09-16T15:48:23.353070+0000","last_peered":"2024-09-16T15:48:23.353070+0000","last_clean":"2024-09-16T15:48:23.353070+0000","last_became_active":"2024-09-16T15:48:21.997542+0000","last_became_peered":"2024-09-16T15:48:21.997542+0000","last_unstale":"2024-09-16T15:48:23.353070+0000","last_undegraded":"2024-09-16T15:48:23.353070+0000","last_fullsized":"2024-09-16T15:48:23.353070+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:01:01.819932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.981239+0000","last_change":"2024-09-16T15:48:21.978226+0000","last_active":"2024-09-16T15:48:22.981239+0000","last_peered":"2024-09-16T15:48:22.981239+0000","last_clean":"2024-09-16T15:48:22.981239+0000","last_became_active":"2024-09-16T15:48:21.977983+0000","last_became_peered":"2024-09-16T15:48:21.977983+0000","last_unstale":"2024-09-16T15:48:22.981239+0000","last_undegraded":"2024-09-16T15:48:22.981239+0000","last_fullsized":"2024-09-16T15:48:22.981239+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:39:54.510565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352016+0000","last_change":"2024-09-16T15:48:21.984845+0000","last_active":"2024-09-16T15:48:23.352016+0000","last_peered":"2024-09-16T15:48:23.352016+0000","last_clean":"2024-09-16T15:48:23.352016+0000","last_became_active":"2024-09-16T15:48:21.984556+0000","last_became_peered":"2024-09-16T15:48:21.984556+0000","last_unstale":"2024-09-16T15:48:23.352016+0000","last_undegraded":"2024-09-16T15:48:23.352016+0000","last_fullsized":"2024-09-16T15:48:23.352016+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:07:20.943835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.980513+0000","last_change":"2024-09-16T15:48:21.984821+0000","last_active":"2024-09-16T15:48:22.980513+0000","last_peered":"2024-09-16T15:48:22.980513+0000","last_clean":"2024-09-16T15:48:22.980513+0000","last_became_active":"2024-09-16T15:48:21.984402+0000","last_became_peered":"2024-09-16T15:48:21.984402+0000","last_unstale":"2024-09-16T15:48:22.980513+0000","last_undegraded":"2024-09-16T15:48:22.980513+0000","last_fullsized":"2024-09-16T15:48:22.980513+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:04:46.727567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.975730+0000","last_change":"2024-09-16T15:48:21.995863+0000","last_active":"2024-09-16T15:48:22.975730+0000","last_peered":"2024-09-16T15:48:22.975730+0000","last_clean":"2024-09-16T15:48:22.975730+0000","last_became_active":"2024-09-16T15:48:21.994582+0000","last_became_peered":"2024-09-16T15:48:21.994582+0000","last_unstale":"2024-09-16T15:48:22.975730+0000","last_undegraded":"2024-09-16T15:48:22.975730+0000","last_fullsized":"2024-09-16T15:48:22.975730+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:54:53.550842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.351087+0000","last_change":"2024-09-16T15:48:21.990586+0000","last_active":"2024-09-16T15:48:23.351087+0000","last_peered":"2024-09-16T15:48:23.351087+0000","last_clean":"2024-09-16T15:48:23.351087+0000","last_became_active":"2024-09-16T15:48:21.987414+0000","last_became_peered":"2024-09-16T15:48:21.987414+0000","last_unstale":"2024-09-16T15:48:23.351087+0000","last_undegraded":"2024-09-16T15:48:23.351087+0000","last_fullsized":"2024-09-16T15:48:23.351087+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:32:12.555452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.979914+0000","last_change":"2024-09-16T15:48:21.981046+0000","last_active":"2024-09-16T15:48:22.979914+0000","last_peered":"2024-09-16T15:48:22.979914+0000","last_clean":"2024-09-16T15:48:22.979914+0000","last_became_active":"2024-09-16T15:48:21.980750+0000","last_became_peered":"2024-09-16T15:48:21.980750+0000","last_unstale":"2024-09-16T15:48:22.979914+0000","last_undegraded":"2024-09-16T15:48:22.979914+0000","last_fullsized":"2024-09-16T15:48:22.979914+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:03:48.288898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.351538+0000","last_change":"2024-09-16T15:48:21.994790+0000","last_active":"2024-09-16T15:48:23.351538+0000","last_peered":"2024-09-16T15:48:23.351538+0000","last_clean":"2024-09-16T15:48:23.351538+0000","last_became_active":"2024-09-16T15:48:21.993406+0000","last_became_peered":"2024-09-16T15:48:21.993406+0000","last_unstale":"2024-09-16T15:48:23.351538+0000","last_undegraded":"2024-09-16T15:48:23.351538+0000","last_fullsized":"2024-09-16T15:48:23.351538+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:19:07.343152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.353136+0000","last_change":"2024-09-16T15:48:21.988921+0000","last_active":"2024-09-16T15:48:23.353136+0000","last_peered":"2024-09-16T15:48:23.353136+0000","last_clean":"2024-09-16T15:48:23.353136+0000","last_became_active":"2024-09-16T15:48:21.987388+0000","last_became_peered":"2024-09-16T15:48:21.987388+0000","last_unstale":"2024-09-16T15:48:23.353136+0000","last_undegraded":"2024-09-16T15:48:23.353136+0000","last_fullsized":"2024-09-16T15:48:23.353136+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:16:58.292432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352286+0000","last_change":"2024-09-16T15:48:21.981255+0000","last_active":"2024-09-16T15:48:23.352286+0000","last_peered":"2024-09-16T15:48:23.352286+0000","last_clean":"2024-09-16T15:48:23.352286+0000","last_became_active":"2024-09-16T15:48:21.980926+0000","last_became_peered":"2024-09-16T15:48:21.980926+0000","last_unstale":"2024-09-16T15:48:23.352286+0000","last_undegraded":"2024-09-16T15:48:23.352286+0000","last_fullsized":"2024-09-16T15:48:23.352286+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:57:23.445821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.991376+0000","last_change":"2024-09-16T15:48:21.990558+0000","last_active":"2024-09-16T15:48:22.991376+0000","last_peered":"2024-09-16T15:48:22.991376+0000","last_clean":"2024-09-16T15:48:22.991376+0000","last_became_active":"2024-09-16T15:48:21.990293+0000","last_became_peered":"2024-09-16T15:48:21.990293+0000","last_unstale":"2024-09-16T15:48:22.991376+0000","last_undegraded":"2024-09-16T15:48:22.991376+0000","last_fullsized":"2024-09-16T15:48:22.991376+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:21:37.652682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352398+0000","last_change":"2024-09-16T15:48:21.987737+0000","last_active":"2024-09-16T15:48:23.352398+0000","last_peered":"2024-09-16T15:48:23.352398+0000","last_clean":"2024-09-16T15:48:23.352398+0000","last_became_active":"2024-09-16T15:48:21.985591+0000","last_became_peered":"2024-09-16T15:48:21.985591+0000","last_unstale":"2024-09-16T15:48:23.352398+0000","last_undegraded":"2024-09-16T15:48:23.352398+0000","last_fullsized":"2024-09-16T15:48:23.352398+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:47:53.347929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.975990+0000","last_change":"2024-09-16T15:48:21.989759+0000","last_active":"2024-09-16T15:48:22.975990+0000","last_peered":"2024-09-16T15:48:22.975990+0000","last_clean":"2024-09-16T15:48:22.975990+0000","last_became_active":"2024-09-16T15:48:21.989495+0000","last_became_peered":"2024-09-16T15:48:21.989495+0000","last_unstale":"2024-09-16T15:48:22.975990+0000","last_undegraded":"2024-09-16T15:48:22.975990+0000","last_fullsized":"2024-09-16T15:48:22.975990+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:36:39.039493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.351800+0000","last_change":"2024-09-16T15:48:21.991329+0000","last_active":"2024-09-16T15:48:23.351800+0000","last_peered":"2024-09-16T15:48:23.351800+0000","last_clean":"2024-09-16T15:48:23.351800+0000","last_became_active":"2024-09-16T15:48:21.990123+0000","last_became_peered":"2024-09-16T15:48:21.990123+0000","last_unstale":"2024-09-16T15:48:23.351800+0000","last_undegraded":"2024-09-16T15:48:23.351800+0000","last_fullsized":"2024-09-16T15:48:23.351800+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:01:55.713464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.980177+0000","last_change":"2024-09-16T15:48:21.986388+0000","last_active":"2024-09-16T15:48:22.980177+0000","last_peered":"2024-09-16T15:48:22.980177+0000","last_clean":"2024-09-16T15:48:22.980177+0000","last_became_active":"2024-09-16T15:48:21.986033+0000","last_became_peered":"2024-09-16T15:48:21.986033+0000","last_unstale":"2024-09-16T15:48:22.980177+0000","last_undegraded":"2024-09-16T15:48:22.980177+0000","last_fullsized":"2024-09-16T15:48:22.980177+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:43:55.728934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.351045+0000","last_change":"2024-09-16T15:48:21.982014+0000","last_active":"2024-09-16T15:48:23.351045+0000","last_peered":"2024-09-16T15:48:23.351045+0000","last_clean":"2024-09-16T15:48:23.351045+0000","last_became_active":"2024-09-16T15:48:21.981772+0000","last_became_peered":"2024-09-16T15:48:21.981772+0000","last_unstale":"2024-09-16T15:48:23.351045+0000","last_undegraded":"2024-09-16T15:48:23.351045+0000","last_fullsized":"2024-09-16T15:48:23.351045+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:38:23.345237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.353074+0000","last_change":"2024-09-16T15:48:21.986477+0000","last_active":"2024-09-16T15:48:23.353074+0000","last_peered":"2024-09-16T15:48:23.353074+0000","last_clean":"2024-09-16T15:48:23.353074+0000","last_became_active":"2024-09-16T15:48:21.984068+0000","last_became_peered":"2024-09-16T15:48:21.984068+0000","last_unstale":"2024-09-16T15:48:23.353074+0000","last_undegraded":"2024-09-16T15:48:23.353074+0000","last_fullsized":"2024-09-16T15:48:23.353074+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:12:07.405991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.991694+0000","last_change":"2024-09-16T15:48:21.993871+0000","last_active":"2024-09-16T15:48:22.991694+0000","last_peered":"2024-09-16T15:48:22.991694+0000","last_clean":"2024-09-16T15:48:22.991694+0000","last_became_active":"2024-09-16T15:48:21.992631+0000","last_became_peered":"2024-09-16T15:48:21.992631+0000","last_unstale":"2024-09-16T15:48:22.991694+0000","last_undegraded":"2024-09-16T15:48:22.991694+0000","last_fullsized":"2024-09-16T15:48:22.991694+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:02:59.965947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352178+0000","last_change":"2024-09-16T15:48:21.996135+0000","last_active":"2024-09-16T15:48:23.352178+0000","last_peered":"2024-09-16T15:48:23.352178+0000","last_clean":"2024-09-16T15:48:23.352178+0000","last_became_active":"2024-09-16T15:48:21.995671+0000","last_became_peered":"2024-09-16T15:48:21.995671+0000","last_unstale":"2024-09-16T15:48:23.352178+0000","last_undegraded":"2024-09-16T15:48:23.352178+0000","last_fullsized":"2024-09-16T15:48:23.352178+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:13:55.927762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352319+0000","last_change":"2024-09-16T15:48:21.994572+0000","last_active":"2024-09-16T15:48:23.352319+0000","last_peered":"2024-09-16T15:48:23.352319+0000","last_clean":"2024-09-16T15:48:23.352319+0000","last_became_active":"2024-09-16T15:48:21.994305+0000","last_became_peered":"2024-09-16T15:48:21.994305+0000","last_unstale":"2024-09-16T15:48:23.352319+0000","last_undegraded":"2024-09-16T15:48:23.352319+0000","last_fullsized":"2024-09-16T15:48:23.352319+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:27:24.090121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.979793+0000","last_change":"2024-09-16T15:48:21.976952+0000","last_active":"2024-09-16T15:48:22.979793+0000","last_peered":"2024-09-16T15:48:22.979793+0000","last_clean":"2024-09-16T15:48:22.979793+0000","last_became_active":"2024-09-16T15:48:21.976547+0000","last_became_peered":"2024-09-16T15:48:21.976547+0000","last_unstale":"2024-09-16T15:48:22.979793+0000","last_undegraded":"2024-09-16T15:48:22.979793+0000","last_fullsized":"2024-09-16T15:48:22.979793+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:29:49.942615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.351839+0000","last_change":"2024-09-16T15:48:21.988902+0000","last_active":"2024-09-16T15:48:23.351839+0000","last_peered":"2024-09-16T15:48:23.351839+0000","last_clean":"2024-09-16T15:48:23.351839+0000","last_became_active":"2024-09-16T15:48:21.988613+0000","last_became_peered":"2024-09-16T15:48:21.988613+0000","last_unstale":"2024-09-16T15:48:23.351839+0000","last_undegraded":"2024-09-16T15:48:23.351839+0000","last_fullsized":"2024-09-16T15:48:23.351839+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:05:00.466068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.351748+0000","last_change":"2024-09-16T15:48:21.985758+0000","last_active":"2024-09-16T15:48:23.351748+0000","last_peered":"2024-09-16T15:48:23.351748+0000","last_clean":"2024-09-16T15:48:23.351748+0000","last_became_active":"2024-09-16T15:48:21.985459+0000","last_became_peered":"2024-09-16T15:48:21.985459+0000","last_unstale":"2024-09-16T15:48:23.351748+0000","last_undegraded":"2024-09-16T15:48:23.351748+0000","last_fullsized":"2024-09-16T15:48:23.351748+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:38:23.345237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.991387+0000","last_change":"2024-09-16T15:48:21.990276+0000","last_active":"2024-09-16T15:48:22.991387+0000","last_peered":"2024-09-16T15:48:22.991387+0000","last_clean":"2024-09-16T15:48:22.991387+0000","last_became_active":"2024-09-16T15:48:21.989558+0000","last_became_peered":"2024-09-16T15:48:21.989558+0000","last_unstale":"2024-09-16T15:48:22.991387+0000","last_undegraded":"2024-09-16T15:48:22.991387+0000","last_fullsized":"2024-09-16T15:48:22.991387+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:48:19.694340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.991799+0000","last_change":"2024-09-16T15:48:21.990934+0000","last_active":"2024-09-16T15:48:22.991799+0000","last_peered":"2024-09-16T15:48:22.991799+0000","last_clean":"2024-09-16T15:48:22.991799+0000","last_became_active":"2024-09-16T15:48:21.990668+0000","last_became_peered":"2024-09-16T15:48:21.990668+0000","last_unstale":"2024-09-16T15:48:22.991799+0000","last_undegraded":"2024-09-16T15:48:22.991799+0000","last_fullsized":"2024-09-16T15:48:22.991799+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:09:12.283305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.976198+0000","last_change":"2024-09-16T15:48:21.980489+0000","last_active":"2024-09-16T15:48:22.976198+0000","last_peered":"2024-09-16T15:48:22.976198+0000","last_clean":"2024-09-16T15:48:22.976198+0000","last_became_active":"2024-09-16T15:48:21.980231+0000","last_became_peered":"2024-09-16T15:48:21.980231+0000","last_unstale":"2024-09-16T15:48:22.976198+0000","last_undegraded":"2024-09-16T15:48:22.976198+0000","last_fullsized":"2024-09-16T15:48:22.976198+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:52:21.377556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.976037+0000","last_change":"2024-09-16T15:48:21.985490+0000","last_active":"2024-09-16T15:48:22.976037+0000","last_peered":"2024-09-16T15:48:22.976037+0000","last_clean":"2024-09-16T15:48:22.976037+0000","last_became_active":"2024-09-16T15:48:21.985234+0000","last_became_peered":"2024-09-16T15:48:21.985234+0000","last_unstale":"2024-09-16T15:48:22.976037+0000","last_undegraded":"2024-09-16T15:48:22.976037+0000","last_fullsized":"2024-09-16T15:48:22.976037+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:55:04.959013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352909+0000","last_change":"2024-09-16T15:48:21.984672+0000","last_active":"2024-09-16T15:48:23.352909+0000","last_peered":"2024-09-16T15:48:23.352909+0000","last_clean":"2024-09-16T15:48:23.352909+0000","last_became_active":"2024-09-16T15:48:21.984399+0000","last_became_peered":"2024-09-16T15:48:21.984399+0000","last_unstale":"2024-09-16T15:48:23.352909+0000","last_undegraded":"2024-09-16T15:48:23.352909+0000","last_fullsized":"2024-09-16T15:48:23.352909+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:09:02.201894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.980034+0000","last_change":"2024-09-16T15:48:21.973853+0000","last_active":"2024-09-16T15:48:22.980034+0000","last_peered":"2024-09-16T15:48:22.980034+0000","last_clean":"2024-09-16T15:48:22.980034+0000","last_became_active":"2024-09-16T15:48:21.973569+0000","last_became_peered":"2024-09-16T15:48:21.973569+0000","last_unstale":"2024-09-16T15:48:22.980034+0000","last_undegraded":"2024-09-16T15:48:22.980034+0000","last_fullsized":"2024-09-16T15:48:22.980034+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:39:27.489599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352274+0000","last_change":"2024-09-16T15:48:21.991963+0000","last_active":"2024-09-16T15:48:23.352274+0000","last_peered":"2024-09-16T15:48:23.352274+0000","last_clean":"2024-09-16T15:48:23.352274+0000","last_became_active":"2024-09-16T15:48:21.991698+0000","last_became_peered":"2024-09-16T15:48:21.991698+0000","last_unstale":"2024-09-16T15:48:23.352274+0000","last_undegraded":"2024-09-16T15:48:23.352274+0000","last_fullsized":"2024-09-16T15:48:23.352274+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:21:15.053411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.350962+0000","last_change":"2024-09-16T15:48:21.987538+0000","last_active":"2024-09-16T15:48:23.350962+0000","last_peered":"2024-09-16T15:48:23.350962+0000","last_clean":"2024-09-16T15:48:23.350962+0000","last_became_active":"2024-09-16T15:48:21.987288+0000","last_became_peered":"2024-09-16T15:48:21.987288+0000","last_unstale":"2024-09-16T15:48:23.350962+0000","last_undegraded":"2024-09-16T15:48:23.350962+0000","last_fullsized":"2024-09-16T15:48:23.350962+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:19:07.343152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.351252+0000","last_change":"2024-09-16T15:48:21.988285+0000","last_active":"2024-09-16T15:48:23.351252+0000","last_peered":"2024-09-16T15:48:23.351252+0000","last_clean":"2024-09-16T15:48:23.351252+0000","last_became_active":"2024-09-16T15:48:21.988029+0000","last_became_peered":"2024-09-16T15:48:21.988029+0000","last_unstale":"2024-09-16T15:48:23.351252+0000","last_undegraded":"2024-09-16T15:48:23.351252+0000","last_fullsized":"2024-09-16T15:48:23.351252+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:16:05.042149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.353150+0000","last_change":"2024-09-16T15:48:21.981145+0000","last_active":"2024-09-16T15:48:23.353150+0000","last_peered":"2024-09-16T15:48:23.353150+0000","last_clean":"2024-09-16T15:48:23.353150+0000","last_became_active":"2024-09-16T15:48:21.980840+0000","last_became_peered":"2024-09-16T15:48:21.980840+0000","last_unstale":"2024-09-16T15:48:23.353150+0000","last_undegraded":"2024-09-16T15:48:23.353150+0000","last_fullsized":"2024-09-16T15:48:23.353150+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:57:23.445821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"90'4","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.976472+0000","last_change":"2024-09-16T15:48:20.975503+0000","last_active":"2024-09-16T15:48:22.976472+0000","last_peered":"2024-09-16T15:48:22.976472+0000","last_clean":"2024-09-16T15:48:22.976472+0000","last_became_active":"2024-09-16T15:48:20.973819+0000","last_became_peered":"2024-09-16T15:48:20.973819+0000","last_unstale":"2024-09-16T15:48:22.976472+0000","last_undegraded":"2024-09-16T15:48:22.976472+0000","last_fullsized":"2024-09-16T15:48:22.976472+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:36:41.387478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.353295+0000","last_change":"2024-09-16T15:48:21.997837+0000","last_active":"2024-09-16T15:48:23.353295+0000","last_peered":"2024-09-16T15:48:23.353295+0000","last_clean":"2024-09-16T15:48:23.353295+0000","last_became_active":"2024-09-16T15:48:21.997316+0000","last_became_peered":"2024-09-16T15:48:21.997316+0000","last_unstale":"2024-09-16T15:48:23.353295+0000","last_undegraded":"2024-09-16T15:48:23.353295+0000","last_fullsized":"2024-09-16T15:48:23.353295+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:48:19.694340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"90'1","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352734+0000","last_change":"2024-09-16T15:48:20.974951+0000","last_active":"2024-09-16T15:48:23.352734+0000","last_peered":"2024-09-16T15:48:23.352734+0000","last_clean":"2024-09-16T15:48:23.352734+0000","last_became_active":"2024-09-16T15:48:20.974688+0000","last_became_peered":"2024-09-16T15:48:20.974688+0000","last_unstale":"2024-09-16T15:48:23.352734+0000","last_undegraded":"2024-09-16T15:48:23.352734+0000","last_fullsized":"2024-09-16T15:48:23.352734+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:54:06.530879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352482+0000","last_change":"2024-09-16T15:48:21.990313+0000","last_active":"2024-09-16T15:48:23.352482+0000","last_peered":"2024-09-16T15:48:23.352482+0000","last_clean":"2024-09-16T15:48:23.352482+0000","last_became_active":"2024-09-16T15:48:21.988356+0000","last_became_peered":"2024-09-16T15:48:21.988356+0000","last_unstale":"2024-09-16T15:48:23.352482+0000","last_undegraded":"2024-09-16T15:48:23.352482+0000","last_fullsized":"2024-09-16T15:48:23.352482+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:16:05.042149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"90'2","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.991491+0000","last_change":"2024-09-16T15:48:20.978653+0000","last_active":"2024-09-16T15:48:22.991491+0000","last_peered":"2024-09-16T15:48:22.991491+0000","last_clean":"2024-09-16T15:48:22.991491+0000","last_became_active":"2024-09-16T15:48:20.976364+0000","last_became_peered":"2024-09-16T15:48:20.976364+0000","last_unstale":"2024-09-16T15:48:22.991491+0000","last_undegraded":"2024-09-16T15:48:22.991491+0000","last_fullsized":"2024-09-16T15:48:22.991491+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:23:03.943492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.353301+0000","last_change":"2024-09-16T15:48:21.998745+0000","last_active":"2024-09-16T15:48:23.353301+0000","last_peered":"2024-09-16T15:48:23.353301+0000","last_clean":"2024-09-16T15:48:23.353301+0000","last_became_active":"2024-09-16T15:48:21.997785+0000","last_became_peered":"2024-09-16T15:48:21.997785+0000","last_unstale":"2024-09-16T15:48:23.353301+0000","last_undegraded":"2024-09-16T15:48:23.353301+0000","last_fullsized":"2024-09-16T15:48:23.353301+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:21:37.652682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"90'2","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.980371+0000","last_change":"2024-09-16T15:48:20.967236+0000","last_active":"2024-09-16T15:48:22.980371+0000","last_peered":"2024-09-16T15:48:22.980371+0000","last_clean":"2024-09-16T15:48:22.980371+0000","last_became_active":"2024-09-16T15:48:20.966954+0000","last_became_peered":"2024-09-16T15:48:20.966954+0000","last_unstale":"2024-09-16T15:48:22.980371+0000","last_undegraded":"2024-09-16T15:48:22.980371+0000","last_fullsized":"2024-09-16T15:48:22.980371+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:06:56.198600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.980724+0000","last_change":"2024-09-16T15:48:21.993949+0000","last_active":"2024-09-16T15:48:22.980724+0000","last_peered":"2024-09-16T15:48:22.980724+0000","last_clean":"2024-09-16T15:48:22.980724+0000","last_became_active":"2024-09-16T15:48:21.993530+0000","last_became_peered":"2024-09-16T15:48:21.993530+0000","last_unstale":"2024-09-16T15:48:22.980724+0000","last_undegraded":"2024-09-16T15:48:22.980724+0000","last_fullsized":"2024-09-16T15:48:22.980724+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:44:18.885274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1b","version":"90'1","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352248+0000","last_change":"2024-09-16T15:48:20.975041+0000","last_active":"2024-09-16T15:48:23.352248+0000","last_peered":"2024-09-16T15:48:23.352248+0000","last_clean":"2024-09-16T15:48:23.352248+0000","last_became_active":"2024-09-16T15:48:20.971719+0000","last_became_peered":"2024-09-16T15:48:20.971719+0000","last_unstale":"2024-09-16T15:48:23.352248+0000","last_undegraded":"2024-09-16T15:48:23.352248+0000","last_fullsized":"2024-09-16T15:48:23.352248+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:13:03.816294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.351424+0000","last_change":"2024-09-16T15:48:21.988933+0000","last_active":"2024-09-16T15:48:23.351424+0000","last_peered":"2024-09-16T15:48:23.351424+0000","last_clean":"2024-09-16T15:48:23.351424+0000","last_became_active":"2024-09-16T15:48:21.988681+0000","last_became_peered":"2024-09-16T15:48:21.988681+0000","last_unstale":"2024-09-16T15:48:23.351424+0000","last_undegraded":"2024-09-16T15:48:23.351424+0000","last_fullsized":"2024-09-16T15:48:23.351424+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:47:53.347929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1a","version":"90'1","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.351139+0000","last_change":"2024-09-16T15:48:20.971997+0000","last_active":"2024-09-16T15:48:23.351139+0000","last_peered":"2024-09-16T15:48:23.351139+0000","last_clean":"2024-09-16T15:48:23.351139+0000","last_became_active":"2024-09-16T15:48:20.971230+0000","last_became_peered":"2024-09-16T15:48:20.971230+0000","last_unstale":"2024-09-16T15:48:23.351139+0000","last_undegraded":"2024-09-16T15:48:23.351139+0000","last_fullsized":"2024-09-16T15:48:23.351139+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:13:03.816294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.351819+0000","last_change":"2024-09-16T15:48:21.996922+0000","last_active":"2024-09-16T15:48:23.351819+0000","last_peered":"2024-09-16T15:48:23.351819+0000","last_clean":"2024-09-16T15:48:23.351819+0000","last_became_active":"2024-09-16T15:48:21.996633+0000","last_became_peered":"2024-09-16T15:48:21.996633+0000","last_unstale":"2024-09-16T15:48:23.351819+0000","last_undegraded":"2024-09-16T15:48:23.351819+0000","last_fullsized":"2024-09-16T15:48:23.351819+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:17:26.941297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.351663+0000","last_change":"2024-09-16T15:48:20.977058+0000","last_active":"2024-09-16T15:48:23.351663+0000","last_peered":"2024-09-16T15:48:23.351663+0000","last_clean":"2024-09-16T15:48:23.351663+0000","last_became_active":"2024-09-16T15:48:20.975591+0000","last_became_peered":"2024-09-16T15:48:20.975591+0000","last_unstale":"2024-09-16T15:48:23.351663+0000","last_undegraded":"2024-09-16T15:48:23.351663+0000","last_fullsized":"2024-09-16T15:48:23.351663+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:58:51.576740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.351992+0000","last_change":"2024-09-16T15:48:21.987074+0000","last_active":"2024-09-16T15:48:23.351992+0000","last_peered":"2024-09-16T15:48:23.351992+0000","last_clean":"2024-09-16T15:48:23.351992+0000","last_became_active":"2024-09-16T15:48:21.986779+0000","last_became_peered":"2024-09-16T15:48:21.986779+0000","last_unstale":"2024-09-16T15:48:23.351992+0000","last_undegraded":"2024-09-16T15:48:23.351992+0000","last_fullsized":"2024-09-16T15:48:23.351992+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:57:02.685046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.351536+0000","last_change":"2024-09-16T15:48:20.977637+0000","last_active":"2024-09-16T15:48:23.351536+0000","last_peered":"2024-09-16T15:48:23.351536+0000","last_clean":"2024-09-16T15:48:23.351536+0000","last_became_active":"2024-09-16T15:48:20.977367+0000","last_became_peered":"2024-09-16T15:48:20.977367+0000","last_unstale":"2024-09-16T15:48:23.351536+0000","last_undegraded":"2024-09-16T15:48:23.351536+0000","last_fullsized":"2024-09-16T15:48:23.351536+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:57:12.992776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352256+0000","last_change":"2024-09-16T15:48:21.990047+0000","last_active":"2024-09-16T15:48:23.352256+0000","last_peered":"2024-09-16T15:48:23.352256+0000","last_clean":"2024-09-16T15:48:23.352256+0000","last_became_active":"2024-09-16T15:48:21.989735+0000","last_became_peered":"2024-09-16T15:48:21.989735+0000","last_unstale":"2024-09-16T15:48:23.352256+0000","last_undegraded":"2024-09-16T15:48:23.352256+0000","last_fullsized":"2024-09-16T15:48:23.352256+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:50:37.046745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.353336+0000","last_change":"2024-09-16T15:48:20.969036+0000","last_active":"2024-09-16T15:48:23.353336+0000","last_peered":"2024-09-16T15:48:23.353336+0000","last_clean":"2024-09-16T15:48:23.353336+0000","last_became_active":"2024-09-16T15:48:20.968771+0000","last_became_peered":"2024-09-16T15:48:20.968771+0000","last_unstale":"2024-09-16T15:48:23.353336+0000","last_undegraded":"2024-09-16T15:48:23.353336+0000","last_fullsized":"2024-09-16T15:48:23.353336+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:47:02.673874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.353252+0000","last_change":"2024-09-16T15:48:21.997579+0000","last_active":"2024-09-16T15:48:23.353252+0000","last_peered":"2024-09-16T15:48:23.353252+0000","last_clean":"2024-09-16T15:48:23.353252+0000","last_became_active":"2024-09-16T15:48:21.997157+0000","last_became_peered":"2024-09-16T15:48:21.997157+0000","last_unstale":"2024-09-16T15:48:23.353252+0000","last_undegraded":"2024-09-16T15:48:23.353252+0000","last_fullsized":"2024-09-16T15:48:23.353252+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:17:00.610181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352849+0000","last_change":"2024-09-16T15:48:20.968246+0000","last_active":"2024-09-16T15:48:23.352849+0000","last_peered":"2024-09-16T15:48:23.352849+0000","last_clean":"2024-09-16T15:48:23.352849+0000","last_became_active":"2024-09-16T15:48:20.967963+0000","last_became_peered":"2024-09-16T15:48:20.967963+0000","last_unstale":"2024-09-16T15:48:23.352849+0000","last_undegraded":"2024-09-16T15:48:23.352849+0000","last_fullsized":"2024-09-16T15:48:23.352849+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:54:40.467435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.991369+0000","last_change":"2024-09-16T15:48:21.990983+0000","last_active":"2024-09-16T15:48:22.991369+0000","last_peered":"2024-09-16T15:48:22.991369+0000","last_clean":"2024-09-16T15:48:22.991369+0000","last_became_active":"2024-09-16T15:48:21.990728+0000","last_became_peered":"2024-09-16T15:48:21.990728+0000","last_unstale":"2024-09-16T15:48:22.991369+0000","last_undegraded":"2024-09-16T15:48:22.991369+0000","last_fullsized":"2024-09-16T15:48:22.991369+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:45:22.428267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.15","version":"90'3","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.353396+0000","last_change":"2024-09-16T15:48:20.975763+0000","last_active":"2024-09-16T15:48:23.353396+0000","last_peered":"2024-09-16T15:48:23.353396+0000","last_clean":"2024-09-16T15:48:23.353396+0000","last_became_active":"2024-09-16T15:48:20.974952+0000","last_became_peered":"2024-09-16T15:48:20.974952+0000","last_unstale":"2024-09-16T15:48:23.353396+0000","last_undegraded":"2024-09-16T15:48:23.353396+0000","last_fullsized":"2024-09-16T15:48:23.353396+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:31:15.757130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.975956+0000","last_change":"2024-09-16T15:48:21.995765+0000","last_active":"2024-09-16T15:48:22.975956+0000","last_peered":"2024-09-16T15:48:22.975956+0000","last_clean":"2024-09-16T15:48:22.975956+0000","last_became_active":"2024-09-16T15:48:21.995445+0000","last_became_peered":"2024-09-16T15:48:21.995445+0000","last_unstale":"2024-09-16T15:48:22.975956+0000","last_undegraded":"2024-09-16T15:48:22.975956+0000","last_fullsized":"2024-09-16T15:48:22.975956+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:33:32.580089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"90'6","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352532+0000","last_change":"2024-09-16T15:48:20.972430+0000","last_active":"2024-09-16T15:48:23.352532+0000","last_peered":"2024-09-16T15:48:23.352532+0000","last_clean":"2024-09-16T15:48:23.352532+0000","last_became_active":"2024-09-16T15:48:20.972159+0000","last_became_peered":"2024-09-16T15:48:20.972159+0000","last_unstale":"2024-09-16T15:48:23.352532+0000","last_undegraded":"2024-09-16T15:48:23.352532+0000","last_fullsized":"2024-09-16T15:48:23.352532+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:54:06.530879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.991316+0000","last_change":"2024-09-16T15:48:21.990590+0000","last_active":"2024-09-16T15:48:22.991316+0000","last_peered":"2024-09-16T15:48:22.991316+0000","last_clean":"2024-09-16T15:48:22.991316+0000","last_became_active":"2024-09-16T15:48:21.989821+0000","last_became_peered":"2024-09-16T15:48:21.989821+0000","last_unstale":"2024-09-16T15:48:22.991316+0000","last_undegraded":"2024-09-16T15:48:22.991316+0000","last_fullsized":"2024-09-16T15:48:22.991316+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:17:00.610181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.13","version":"90'2","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.980206+0000","last_change":"2024-09-16T15:48:20.965556+0000","last_active":"2024-09-16T15:48:22.980206+0000","last_peered":"2024-09-16T15:48:22.980206+0000","last_clean":"2024-09-16T15:48:22.980206+0000","last_became_active":"2024-09-16T15:48:20.965196+0000","last_became_peered":"2024-09-16T15:48:20.965196+0000","last_unstale":"2024-09-16T15:48:22.980206+0000","last_undegraded":"2024-09-16T15:48:22.980206+0000","last_fullsized":"2024-09-16T15:48:22.980206+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:05.970213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352808+0000","last_change":"2024-09-16T15:48:21.993233+0000","last_active":"2024-09-16T15:48:23.352808+0000","last_peered":"2024-09-16T15:48:23.352808+0000","last_clean":"2024-09-16T15:48:23.352808+0000","last_became_active":"2024-09-16T15:48:21.992915+0000","last_became_peered":"2024-09-16T15:48:21.992915+0000","last_unstale":"2024-09-16T15:48:23.352808+0000","last_undegraded":"2024-09-16T15:48:23.352808+0000","last_fullsized":"2024-09-16T15:48:23.352808+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:36:34.887933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.980104+0000","last_change":"2024-09-16T15:48:20.969210+0000","last_active":"2024-09-16T15:48:22.980104+0000","last_peered":"2024-09-16T15:48:22.980104+0000","last_clean":"2024-09-16T15:48:22.980104+0000","last_became_active":"2024-09-16T15:48:20.968946+0000","last_became_peered":"2024-09-16T15:48:20.968946+0000","last_unstale":"2024-09-16T15:48:22.980104+0000","last_undegraded":"2024-09-16T15:48:22.980104+0000","last_fullsized":"2024-09-16T15:48:22.980104+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:40:35.123939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352100+0000","last_change":"2024-09-16T15:48:21.997110+0000","last_active":"2024-09-16T15:48:23.352100+0000","last_peered":"2024-09-16T15:48:23.352100+0000","last_clean":"2024-09-16T15:48:23.352100+0000","last_became_active":"2024-09-16T15:48:21.996855+0000","last_became_peered":"2024-09-16T15:48:21.996855+0000","last_unstale":"2024-09-16T15:48:23.352100+0000","last_undegraded":"2024-09-16T15:48:23.352100+0000","last_fullsized":"2024-09-16T15:48:23.352100+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:44:31.136306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.350997+0000","last_change":"2024-09-16T15:48:20.969040+0000","last_active":"2024-09-16T15:48:23.350997+0000","last_peered":"2024-09-16T15:48:23.350997+0000","last_clean":"2024-09-16T15:48:23.350997+0000","last_became_active":"2024-09-16T15:48:20.968541+0000","last_became_peered":"2024-09-16T15:48:20.968541+0000","last_unstale":"2024-09-16T15:48:23.350997+0000","last_undegraded":"2024-09-16T15:48:23.350997+0000","last_fullsized":"2024-09-16T15:48:23.350997+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:32:44.623249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.991116+0000","last_change":"2024-09-16T15:48:21.989170+0000","last_active":"2024-09-16T15:48:22.991116+0000","last_peered":"2024-09-16T15:48:22.991116+0000","last_clean":"2024-09-16T15:48:22.991116+0000","last_became_active":"2024-09-16T15:48:21.988901+0000","last_became_peered":"2024-09-16T15:48:21.988901+0000","last_unstale":"2024-09-16T15:48:22.991116+0000","last_undegraded":"2024-09-16T15:48:22.991116+0000","last_fullsized":"2024-09-16T15:48:22.991116+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:16:58.292432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"90'8","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.991610+0000","last_change":"2024-09-16T15:48:20.975861+0000","last_active":"2024-09-16T15:48:22.991610+0000","last_peered":"2024-09-16T15:48:22.991610+0000","last_clean":"2024-09-16T15:48:22.991610+0000","last_became_active":"2024-09-16T15:48:20.974746+0000","last_became_peered":"2024-09-16T15:48:20.974746+0000","last_unstale":"2024-09-16T15:48:22.991610+0000","last_undegraded":"2024-09-16T15:48:22.991610+0000","last_fullsized":"2024-09-16T15:48:22.991610+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:11:13.839123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.353333+0000","last_change":"2024-09-16T15:48:21.990395+0000","last_active":"2024-09-16T15:48:23.353333+0000","last_peered":"2024-09-16T15:48:23.353333+0000","last_clean":"2024-09-16T15:48:23.353333+0000","last_became_active":"2024-09-16T15:48:21.990144+0000","last_became_peered":"2024-09-16T15:48:21.990144+0000","last_unstale":"2024-09-16T15:48:23.353333+0000","last_undegraded":"2024-09-16T15:48:23.353333+0000","last_fullsized":"2024-09-16T15:48:23.353333+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:02:44.937439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"90'1","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.976584+0000","last_change":"2024-09-16T15:48:20.975777+0000","last_active":"2024-09-16T15:48:22.976584+0000","last_peered":"2024-09-16T15:48:22.976584+0000","last_clean":"2024-09-16T15:48:22.976584+0000","last_became_active":"2024-09-16T15:48:20.974162+0000","last_became_peered":"2024-09-16T15:48:20.974162+0000","last_unstale":"2024-09-16T15:48:22.976584+0000","last_undegraded":"2024-09-16T15:48:22.976584+0000","last_fullsized":"2024-09-16T15:48:22.976584+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:44:17.877841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352625+0000","last_change":"2024-09-16T15:48:21.986577+0000","last_active":"2024-09-16T15:48:23.352625+0000","last_peered":"2024-09-16T15:48:23.352625+0000","last_clean":"2024-09-16T15:48:23.352625+0000","last_became_active":"2024-09-16T15:48:21.986333+0000","last_became_peered":"2024-09-16T15:48:21.986333+0000","last_unstale":"2024-09-16T15:48:23.352625+0000","last_undegraded":"2024-09-16T15:48:23.352625+0000","last_fullsized":"2024-09-16T15:48:23.352625+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:02:44.937439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"90'2","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352068+0000","last_change":"2024-09-16T15:48:20.975158+0000","last_active":"2024-09-16T15:48:23.352068+0000","last_peered":"2024-09-16T15:48:23.352068+0000","last_clean":"2024-09-16T15:48:23.352068+0000","last_became_active":"2024-09-16T15:48:20.974854+0000","last_became_peered":"2024-09-16T15:48:20.974854+0000","last_unstale":"2024-09-16T15:48:23.352068+0000","last_undegraded":"2024-09-16T15:48:23.352068+0000","last_fullsized":"2024-09-16T15:48:23.352068+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:41:44.873886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352765+0000","last_change":"2024-09-16T15:48:21.991352+0000","last_active":"2024-09-16T15:48:23.352765+0000","last_peered":"2024-09-16T15:48:23.352765+0000","last_clean":"2024-09-16T15:48:23.352765+0000","last_became_active":"2024-09-16T15:48:21.991096+0000","last_became_peered":"2024-09-16T15:48:21.991096+0000","last_unstale":"2024-09-16T15:48:23.352765+0000","last_undegraded":"2024-09-16T15:48:23.352765+0000","last_fullsized":"2024-09-16T15:48:23.352765+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:07:20.943835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352234+0000","last_change":"2024-09-16T15:48:20.978031+0000","last_active":"2024-09-16T15:48:23.352234+0000","last_peered":"2024-09-16T15:48:23.352234+0000","last_clean":"2024-09-16T15:48:23.352234+0000","last_became_active":"2024-09-16T15:48:20.976491+0000","last_became_peered":"2024-09-16T15:48:20.976491+0000","last_unstale":"2024-09-16T15:48:23.352234+0000","last_undegraded":"2024-09-16T15:48:23.352234+0000","last_fullsized":"2024-09-16T15:48:23.352234+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:49:57.979104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.979914+0000","last_change":"2024-09-16T15:48:21.993773+0000","last_active":"2024-09-16T15:48:22.979914+0000","last_peered":"2024-09-16T15:48:22.979914+0000","last_clean":"2024-09-16T15:48:22.979914+0000","last_became_active":"2024-09-16T15:48:21.993443+0000","last_became_peered":"2024-09-16T15:48:21.993443+0000","last_unstale":"2024-09-16T15:48:22.979914+0000","last_undegraded":"2024-09-16T15:48:22.979914+0000","last_fullsized":"2024-09-16T15:48:22.979914+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:30:15.463014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.b","version":"90'2","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.351800+0000","last_change":"2024-09-16T15:48:20.976944+0000","last_active":"2024-09-16T15:48:23.351800+0000","last_peered":"2024-09-16T15:48:23.351800+0000","last_clean":"2024-09-16T15:48:23.351800+0000","last_became_active":"2024-09-16T15:48:20.976649+0000","last_became_peered":"2024-09-16T15:48:20.976649+0000","last_unstale":"2024-09-16T15:48:23.351800+0000","last_undegraded":"2024-09-16T15:48:23.351800+0000","last_fullsized":"2024-09-16T15:48:23.351800+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:33:01.893406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.976168+0000","last_change":"2024-09-16T15:48:21.994077+0000","last_active":"2024-09-16T15:48:22.976168+0000","last_peered":"2024-09-16T15:48:22.976168+0000","last_clean":"2024-09-16T15:48:22.976168+0000","last_became_active":"2024-09-16T15:48:21.992548+0000","last_became_peered":"2024-09-16T15:48:21.992548+0000","last_unstale":"2024-09-16T15:48:22.976168+0000","last_undegraded":"2024-09-16T15:48:22.976168+0000","last_fullsized":"2024-09-16T15:48:22.976168+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:25:24.140562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.980214+0000","last_change":"2024-09-16T15:48:20.970601+0000","last_active":"2024-09-16T15:48:22.980214+0000","last_peered":"2024-09-16T15:48:22.980214+0000","last_clean":"2024-09-16T15:48:22.980214+0000","last_became_active":"2024-09-16T15:48:20.970272+0000","last_became_peered":"2024-09-16T15:48:20.970272+0000","last_unstale":"2024-09-16T15:48:22.980214+0000","last_undegraded":"2024-09-16T15:48:22.980214+0000","last_fullsized":"2024-09-16T15:48:22.980214+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:42:55.329422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.980294+0000","last_change":"2024-09-16T15:48:21.978436+0000","last_active":"2024-09-16T15:48:22.980294+0000","last_peered":"2024-09-16T15:48:22.980294+0000","last_clean":"2024-09-16T15:48:22.980294+0000","last_became_active":"2024-09-16T15:48:21.977384+0000","last_became_peered":"2024-09-16T15:48:21.977384+0000","last_unstale":"2024-09-16T15:48:22.980294+0000","last_undegraded":"2024-09-16T15:48:22.980294+0000","last_fullsized":"2024-09-16T15:48:22.980294+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:34:53.674221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.991107+0000","last_change":"2024-09-16T15:48:20.971710+0000","last_active":"2024-09-16T15:48:22.991107+0000","last_peered":"2024-09-16T15:48:22.991107+0000","last_clean":"2024-09-16T15:48:22.991107+0000","last_became_active":"2024-09-16T15:48:20.971360+0000","last_became_peered":"2024-09-16T15:48:20.971360+0000","last_unstale":"2024-09-16T15:48:22.991107+0000","last_undegraded":"2024-09-16T15:48:22.991107+0000","last_fullsized":"2024-09-16T15:48:22.991107+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:02:25.192877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.351953+0000","last_change":"2024-09-16T15:48:21.975033+0000","last_active":"2024-09-16T15:48:23.351953+0000","last_peered":"2024-09-16T15:48:23.351953+0000","last_clean":"2024-09-16T15:48:23.351953+0000","last_became_active":"2024-09-16T15:48:21.974636+0000","last_became_peered":"2024-09-16T15:48:21.974636+0000","last_unstale":"2024-09-16T15:48:23.351953+0000","last_undegraded":"2024-09-16T15:48:23.351953+0000","last_fullsized":"2024-09-16T15:48:23.351953+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:32:12.555452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.351750+0000","last_change":"2024-09-16T15:48:20.980185+0000","last_active":"2024-09-16T15:48:23.351750+0000","last_peered":"2024-09-16T15:48:23.351750+0000","last_clean":"2024-09-16T15:48:23.351750+0000","last_became_active":"2024-09-16T15:48:20.979875+0000","last_became_peered":"2024-09-16T15:48:20.979875+0000","last_unstale":"2024-09-16T15:48:23.351750+0000","last_undegraded":"2024-09-16T15:48:23.351750+0000","last_fullsized":"2024-09-16T15:48:23.351750+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:56:29.587401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.976320+0000","last_change":"2024-09-16T15:48:21.988229+0000","last_active":"2024-09-16T15:48:22.976320+0000","last_peered":"2024-09-16T15:48:22.976320+0000","last_clean":"2024-09-16T15:48:22.976320+0000","last_became_active":"2024-09-16T15:48:21.987857+0000","last_became_peered":"2024-09-16T15:48:21.987857+0000","last_unstale":"2024-09-16T15:48:22.976320+0000","last_undegraded":"2024-09-16T15:48:22.976320+0000","last_fullsized":"2024-09-16T15:48:22.976320+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:40:50.356201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"90'2","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352056+0000","last_change":"2024-09-16T15:48:20.978563+0000","last_active":"2024-09-16T15:48:23.352056+0000","last_peered":"2024-09-16T15:48:23.352056+0000","last_clean":"2024-09-16T15:48:23.352056+0000","last_became_active":"2024-09-16T15:48:20.978188+0000","last_became_peered":"2024-09-16T15:48:20.978188+0000","last_unstale":"2024-09-16T15:48:23.352056+0000","last_undegraded":"2024-09-16T15:48:23.352056+0000","last_fullsized":"2024-09-16T15:48:23.352056+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:28:27.584567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.976444+0000","last_change":"2024-09-16T15:48:21.983190+0000","last_active":"2024-09-16T15:48:22.976444+0000","last_peered":"2024-09-16T15:48:22.976444+0000","last_clean":"2024-09-16T15:48:22.976444+0000","last_became_active":"2024-09-16T15:48:21.982916+0000","last_became_peered":"2024-09-16T15:48:21.982916+0000","last_unstale":"2024-09-16T15:48:22.976444+0000","last_undegraded":"2024-09-16T15:48:22.976444+0000","last_fullsized":"2024-09-16T15:48:22.976444+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:55:24.014926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"91'1","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.991689+0000","last_change":"2024-09-16T15:48:20.975435+0000","last_active":"2024-09-16T15:48:22.991689+0000","last_peered":"2024-09-16T15:48:22.991689+0000","last_clean":"2024-09-16T15:48:22.991689+0000","last_became_active":"2024-09-16T15:48:20.975091+0000","last_became_peered":"2024-09-16T15:48:20.975091+0000","last_unstale":"2024-09-16T15:48:22.991689+0000","last_undegraded":"2024-09-16T15:48:22.991689+0000","last_fullsized":"2024-09-16T15:48:22.991689+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:50:09.821843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.975941+0000","last_change":"2024-09-16T15:48:21.988044+0000","last_active":"2024-09-16T15:48:22.975941+0000","last_peered":"2024-09-16T15:48:22.975941+0000","last_clean":"2024-09-16T15:48:22.975941+0000","last_became_active":"2024-09-16T15:48:21.987788+0000","last_became_peered":"2024-09-16T15:48:21.987788+0000","last_unstale":"2024-09-16T15:48:22.975941+0000","last_undegraded":"2024-09-16T15:48:22.975941+0000","last_fullsized":"2024-09-16T15:48:22.975941+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:54:15.406878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352674+0000","last_change":"2024-09-16T15:48:20.974118+0000","last_active":"2024-09-16T15:48:23.352674+0000","last_peered":"2024-09-16T15:48:23.352674+0000","last_clean":"2024-09-16T15:48:23.352674+0000","last_became_active":"2024-09-16T15:48:20.973841+0000","last_became_peered":"2024-09-16T15:48:20.973841+0000","last_unstale":"2024-09-16T15:48:23.352674+0000","last_undegraded":"2024-09-16T15:48:23.352674+0000","last_fullsized":"2024-09-16T15:48:23.352674+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:54:40.467435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352250+0000","last_change":"2024-09-16T15:48:21.989418+0000","last_active":"2024-09-16T15:48:23.352250+0000","last_peered":"2024-09-16T15:48:23.352250+0000","last_clean":"2024-09-16T15:48:23.352250+0000","last_became_active":"2024-09-16T15:48:21.989166+0000","last_became_peered":"2024-09-16T15:48:21.989166+0000","last_unstale":"2024-09-16T15:48:23.352250+0000","last_undegraded":"2024-09-16T15:48:23.352250+0000","last_fullsized":"2024-09-16T15:48:23.352250+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:12:07.405991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.351511+0000","last_change":"2024-09-16T15:48:20.974113+0000","last_active":"2024-09-16T15:48:23.351511+0000","last_peered":"2024-09-16T15:48:23.351511+0000","last_clean":"2024-09-16T15:48:23.351511+0000","last_became_active":"2024-09-16T15:48:20.973851+0000","last_became_peered":"2024-09-16T15:48:20.973851+0000","last_unstale":"2024-09-16T15:48:23.351511+0000","last_undegraded":"2024-09-16T15:48:23.351511+0000","last_fullsized":"2024-09-16T15:48:23.351511+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:55:33.505049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352485+0000","last_change":"2024-09-16T15:48:21.997253+0000","last_active":"2024-09-16T15:48:23.352485+0000","last_peered":"2024-09-16T15:48:23.352485+0000","last_clean":"2024-09-16T15:48:23.352485+0000","last_became_active":"2024-09-16T15:48:21.996568+0000","last_became_peered":"2024-09-16T15:48:21.996568+0000","last_unstale":"2024-09-16T15:48:23.352485+0000","last_undegraded":"2024-09-16T15:48:23.352485+0000","last_fullsized":"2024-09-16T15:48:23.352485+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:53:59.460386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.980860+0000","last_change":"2024-09-16T15:48:20.968528+0000","last_active":"2024-09-16T15:48:22.980860+0000","last_peered":"2024-09-16T15:48:22.980860+0000","last_clean":"2024-09-16T15:48:22.980860+0000","last_became_active":"2024-09-16T15:48:20.967944+0000","last_became_peered":"2024-09-16T15:48:20.967944+0000","last_unstale":"2024-09-16T15:48:22.980860+0000","last_undegraded":"2024-09-16T15:48:22.980860+0000","last_fullsized":"2024-09-16T15:48:22.980860+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:16:08.669040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.991351+0000","last_change":"2024-09-16T15:48:21.991566+0000","last_active":"2024-09-16T15:48:22.991351+0000","last_peered":"2024-09-16T15:48:22.991351+0000","last_clean":"2024-09-16T15:48:22.991351+0000","last_became_active":"2024-09-16T15:48:21.991274+0000","last_became_peered":"2024-09-16T15:48:21.991274+0000","last_unstale":"2024-09-16T15:48:22.991351+0000","last_undegraded":"2024-09-16T15:48:22.991351+0000","last_fullsized":"2024-09-16T15:48:22.991351+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:23:27.535786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.351253+0000","last_change":"2024-09-16T15:48:20.963312+0000","last_active":"2024-09-16T15:48:23.351253+0000","last_peered":"2024-09-16T15:48:23.351253+0000","last_clean":"2024-09-16T15:48:23.351253+0000","last_became_active":"2024-09-16T15:48:20.963034+0000","last_became_peered":"2024-09-16T15:48:20.963034+0000","last_unstale":"2024-09-16T15:48:23.351253+0000","last_undegraded":"2024-09-16T15:48:23.351253+0000","last_fullsized":"2024-09-16T15:48:23.351253+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:59:53.683980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352072+0000","last_change":"2024-09-16T15:48:21.996871+0000","last_active":"2024-09-16T15:48:23.352072+0000","last_peered":"2024-09-16T15:48:23.352072+0000","last_clean":"2024-09-16T15:48:23.352072+0000","last_became_active":"2024-09-16T15:48:21.996556+0000","last_became_peered":"2024-09-16T15:48:21.996556+0000","last_unstale":"2024-09-16T15:48:23.352072+0000","last_undegraded":"2024-09-16T15:48:23.352072+0000","last_fullsized":"2024-09-16T15:48:23.352072+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:56:48.853231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352292+0000","last_change":"2024-09-16T15:48:20.978028+0000","last_active":"2024-09-16T15:48:23.352292+0000","last_peered":"2024-09-16T15:48:23.352292+0000","last_clean":"2024-09-16T15:48:23.352292+0000","last_became_active":"2024-09-16T15:48:20.977556+0000","last_became_peered":"2024-09-16T15:48:20.977556+0000","last_unstale":"2024-09-16T15:48:23.352292+0000","last_undegraded":"2024-09-16T15:48:23.352292+0000","last_fullsized":"2024-09-16T15:48:23.352292+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:35:16.664943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352088+0000","last_change":"2024-09-16T15:48:21.992228+0000","last_active":"2024-09-16T15:48:23.352088+0000","last_peered":"2024-09-16T15:48:23.352088+0000","last_clean":"2024-09-16T15:48:23.352088+0000","last_became_active":"2024-09-16T15:48:21.991894+0000","last_became_peered":"2024-09-16T15:48:21.991894+0000","last_unstale":"2024-09-16T15:48:23.352088+0000","last_undegraded":"2024-09-16T15:48:23.352088+0000","last_fullsized":"2024-09-16T15:48:23.352088+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:16:09.676473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.351704+0000","last_change":"2024-09-16T15:48:20.976801+0000","last_active":"2024-09-16T15:48:23.351704+0000","last_peered":"2024-09-16T15:48:23.351704+0000","last_clean":"2024-09-16T15:48:23.351704+0000","last_became_active":"2024-09-16T15:48:20.975672+0000","last_became_peered":"2024-09-16T15:48:20.975672+0000","last_unstale":"2024-09-16T15:48:23.351704+0000","last_undegraded":"2024-09-16T15:48:23.351704+0000","last_fullsized":"2024-09-16T15:48:23.351704+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:55:33.505049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.991410+0000","last_change":"2024-09-16T15:48:21.989919+0000","last_active":"2024-09-16T15:48:22.991410+0000","last_peered":"2024-09-16T15:48:22.991410+0000","last_clean":"2024-09-16T15:48:22.991410+0000","last_became_active":"2024-09-16T15:48:21.989572+0000","last_became_peered":"2024-09-16T15:48:21.989572+0000","last_unstale":"2024-09-16T15:48:22.991410+0000","last_undegraded":"2024-09-16T15:48:22.991410+0000","last_fullsized":"2024-09-16T15:48:22.991410+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:20.946152+0000","last_clean_scrub_stamp":"2024-09-16T15:48:20.946152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:57:07.183635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"90'1","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:22.976109+0000","last_change":"2024-09-16T15:48:20.974835+0000","last_active":"2024-09-16T15:48:22.976109+0000","last_peered":"2024-09-16T15:48:22.976109+0000","last_clean":"2024-09-16T15:48:22.976109+0000","last_became_active":"2024-09-16T15:48:20.974577+0000","last_became_peered":"2024-09-16T15:48:20.974577+0000","last_unstale":"2024-09-16T15:48:22.976109+0000","last_undegraded":"2024-09-16T15:48:22.976109+0000","last_fullsized":"2024-09-16T15:48:22.976109+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:48:19.938719+0000","last_clean_scrub_stamp":"2024-09-16T15:48:19.938719+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:04:45.720134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":242,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-16T15:48:23.352225+0000","last_change":"2024-09-16T15:39:12.698813+0000","last_active":"2024-09-16T15:48:23.352225+0000","last_peered":"2024-09-16T15:48:23.352225+0000","last_clean":"2024-09-16T15:48:23.352225+0000","last_became_active":"2024-09-16T15:39:12.697695+0000","last_became_peered":"2024-09-16T15:39:12.697695+0000","last_unstale":"2024-09-16T15:48:23.352225+0000","last_undegraded":"2024-09-16T15:48:23.352225+0000","last_fullsized":"2024-09-16T15:48:23.352225+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_clean_scrub_stamp":"2024-09-16T15:39:11.684967+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:02.061081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509556,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64152,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307688,"statfs":{"total":96636764160,"available":96571072512,"internally_reserved":0,"allocated":1118208,"data_stored":482183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":64551368},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509555,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63404,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94308436,"statfs":{"total":96636764160,"available":96571838464,"internally_reserved":0,"allocated":1720320,"data_stored":1073525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":63175111},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509554,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64152,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307688,"statfs":{"total":96636764160,"available":96571072512,"internally_reserved":0,"allocated":1114112,"data_stored":481545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509554,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58660,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313180,"statfs":{"total":96636764160,"available":96576696320,"internally_reserved":0,"allocated":1122304,"data_stored":482185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":58915273},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509554,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58640,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313200,"statfs":{"total":96636764160,"available":96576716800,"internally_reserved":0,"allocated":1110016,"data_stored":481511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509555,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59224,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312616,"statfs":{"total":96636764160,"available":96576118784,"internally_reserved":0,"allocated":1708032,"data_stored":1071969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509553,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58656,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313184,"statfs":{"total":96636764160,"available":96576700416,"internally_reserved":0,"allocated":1126400,"data_stored":483145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542260,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62804,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94309036,"statfs":{"total":96636764160,"available":96572452864,"internally_reserved":0,"allocated":1114112,"data_stored":481581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":63175116},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:48:31.284 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T15:48:31.284 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-16T15:48:31.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.483+0000 7f4d69200640 1 -- 172.21.15.151:0/4130437 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d48004d40 msgr2=0x7f4d480051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:31.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.483+0000 7f4d69200640 1 --2- 172.21.15.151:0/4130437 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d48004d40 0x7f4d480051c0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c00c290 tx=0x7f4d48005830 comp rx=0 tx=0).stop 2024-09-16T15:48:31.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.483+0000 7f4d62800640 1 -- 172.21.15.151:0/4130437 reap_dead start 2024-09-16T15:48:31.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.483+0000 7f4d69200640 1 -- 172.21.15.151:0/4130437 shutdown_connections 2024-09-16T15:48:31.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.483+0000 7f4d69200640 1 -- 172.21.15.151:0/4130437 >> 172.21.15.151:0/4130437 conn(0x7f4d640b1840 msgr2=0x7f4d640b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:31.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.484+0000 7f4d69200640 1 -- 172.21.15.151:0/4130437 shutdown_connections 2024-09-16T15:48:31.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.484+0000 7f4d69200640 1 -- 172.21.15.151:0/4130437 wait complete. 2024-09-16T15:48:31.487 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.485+0000 7f4d69200640 1 Processor -- start 2024-09-16T15:48:31.487 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.485+0000 7f4d69200640 1 -- start start 2024-09-16T15:48:31.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.486+0000 7f4d69200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d48004d40 0x7f4d6414bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:31.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.486+0000 7f4d69200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d6414e700 0x7f4d641f4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:31.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.486+0000 7f4d69200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4d640cadf0 0x7f4d640b09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:31.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.486+0000 7f4d69200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4d640ce4a0 con 0x7f4d48004d40 2024-09-16T15:48:31.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.486+0000 7f4d69200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4d640ce620 con 0x7f4d6414e700 2024-09-16T15:48:31.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.486+0000 7f4d69200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f4d640ce7a0 con 0x7f4d640cadf0 2024-09-16T15:48:31.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.486+0000 7f4d61e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d48004d40 0x7f4d6414bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:31.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.486+0000 7f4d61e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d48004d40 0x7f4d6414bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:42834/0 (socket says 172.21.15.151:42834) 2024-09-16T15:48:31.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.486+0000 7f4d61e00640 1 -- 172.21.15.151:0/3006609328 learned_addr learned my addr 172.21.15.151:0/3006609328 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:48:31.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.487+0000 7f4d61400640 1 --2- 172.21.15.151:0/3006609328 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d6414e700 0x7f4d641f4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:31.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.487+0000 7f4d62800640 1 --2- 172.21.15.151:0/3006609328 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4d640cadf0 0x7f4d640b09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:31.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.487+0000 7f4d61e00640 1 -- 172.21.15.151:0/3006609328 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4d640cadf0 msgr2=0x7f4d640b09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:31.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.487+0000 7f4d61e00640 1 --2- 172.21.15.151:0/3006609328 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4d640cadf0 0x7f4d640b09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:31.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.487+0000 7f4d61e00640 1 -- 172.21.15.151:0/3006609328 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d6414e700 msgr2=0x7f4d641f4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:31.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.487+0000 7f4d61e00640 1 --2- 172.21.15.151:0/3006609328 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d6414e700 0x7f4d641f4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:31.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.487+0000 7f4d61e00640 1 -- 172.21.15.151:0/3006609328 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d6414ecc0 con 0x7f4d48004d40 2024-09-16T15:48:31.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.487+0000 7f4d61e00640 1 --2- 172.21.15.151:0/3006609328 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d48004d40 0x7f4d6414bdd0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f4d54011ce0 tx=0x7f4d5400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:31.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.487+0000 7f4d5a000640 1 -- 172.21.15.151:0/3006609328 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d54003d80 con 0x7f4d48004d40 2024-09-16T15:48:31.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.487+0000 7f4d5a000640 1 -- 172.21.15.151:0/3006609328 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4d54004b20 con 0x7f4d48004d40 2024-09-16T15:48:31.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.487+0000 7f4d5a000640 1 -- 172.21.15.151:0/3006609328 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d54004050 con 0x7f4d48004d40 2024-09-16T15:48:31.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.487+0000 7f4d69200640 1 -- 172.21.15.151:0/3006609328 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d6414efa0 con 0x7f4d48004d40 2024-09-16T15:48:31.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.488+0000 7f4d69200640 1 -- 172.21.15.151:0/3006609328 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d6414f560 con 0x7f4d48004d40 2024-09-16T15:48:31.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.489+0000 7f4d69200640 1 -- 172.21.15.151:0/3006609328 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d640ce980 con 0x7f4d48004d40 2024-09-16T15:48:31.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.490+0000 7f4d5a000640 1 -- 172.21.15.151:0/3006609328 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 30) ==== 102868+0+0 (secure 0 0 0) 0x7f4d54011d10 con 0x7f4d48004d40 2024-09-16T15:48:31.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.491+0000 7f4d5a000640 1 --2- 172.21.15.151:0/3006609328 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4d440799c0 0x7f4d4407be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:31.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.491+0000 7f4d5a000640 1 -- 172.21.15.151:0/3006609328 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7f4d5409a710 con 0x7f4d48004d40 2024-09-16T15:48:31.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.491+0000 7f4d61400640 1 --2- 172.21.15.151:0/3006609328 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4d440799c0 0x7f4d4407be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:31.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.492+0000 7f4d61400640 1 --2- 172.21.15.151:0/3006609328 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4d440799c0 0x7f4d4407be80 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f4d640b4d90 tx=0x7f4d50005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:31.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.495+0000 7f4d5a000640 1 -- 172.21.15.151:0/3006609328 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4d54064480 con 0x7f4d48004d40 2024-09-16T15:48:31.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.748+0000 7f4d69200640 1 -- 172.21.15.151:0/3006609328 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f4d640ce010 con 0x7f4d48004d40 2024-09-16T15:48:31.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.751+0000 7f4d5a000640 1 -- 172.21.15.151:0/3006609328 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) ==== 190+0+147 (secure 0 0 0) 0x7f4d54063d80 con 0x7f4d48004d40 2024-09-16T15:48:31.753 INFO:teuthology.orchestra.run.smithi151.stdout:[client.0] 2024-09-16T15:48:31.753 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQDsUOhmf5PBKBAAKE4Y4zbA4h07IBNjPkyptw== 2024-09-16T15:48:31.753 INFO:teuthology.orchestra.run.smithi151.stdout: caps mds = "allow" 2024-09-16T15:48:31.753 INFO:teuthology.orchestra.run.smithi151.stdout: caps mon = "allow r" 2024-09-16T15:48:31.753 INFO:teuthology.orchestra.run.smithi151.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-16T15:48:31.753 INFO:teuthology.orchestra.run.smithi151.stderr:updated caps for client.0 2024-09-16T15:48:31.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.753+0000 7f4d69200640 1 -- 172.21.15.151:0/3006609328 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4d440799c0 msgr2=0x7f4d4407be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:31.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.753+0000 7f4d69200640 1 --2- 172.21.15.151:0/3006609328 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4d440799c0 0x7f4d4407be80 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f4d640b4d90 tx=0x7f4d50005f90 comp rx=0 tx=0).stop 2024-09-16T15:48:31.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.753+0000 7f4d69200640 1 -- 172.21.15.151:0/3006609328 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d48004d40 msgr2=0x7f4d6414bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:31.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.753+0000 7f4d69200640 1 --2- 172.21.15.151:0/3006609328 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d48004d40 0x7f4d6414bdd0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f4d54011ce0 tx=0x7f4d5400f790 comp rx=0 tx=0).stop 2024-09-16T15:48:31.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.754+0000 7f4d69200640 1 -- 172.21.15.151:0/3006609328 shutdown_connections 2024-09-16T15:48:31.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.754+0000 7f4d69200640 1 --2- 172.21.15.151:0/3006609328 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4d440799c0 0x7f4d4407be80 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:31.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.754+0000 7f4d69200640 1 --2- 172.21.15.151:0/3006609328 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d6414e700 0x7f4d641f4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:31.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.754+0000 7f4d69200640 1 --2- 172.21.15.151:0/3006609328 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4d640cadf0 0x7f4d640b09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:31.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.754+0000 7f4d69200640 1 --2- 172.21.15.151:0/3006609328 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d48004d40 0x7f4d6414bdd0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:31.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.754+0000 7f4d69200640 1 -- 172.21.15.151:0/3006609328 >> 172.21.15.151:0/3006609328 conn(0x7f4d640b1840 msgr2=0x7f4d64073e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:31.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.754+0000 7f4d69200640 1 -- 172.21.15.151:0/3006609328 shutdown_connections 2024-09-16T15:48:31.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.754+0000 7f4d69200640 1 -- 172.21.15.151:0/3006609328 wait complete. 2024-09-16T15:48:31.771 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:48:31.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.927+0000 7f5677e00640 1 -- 172.21.15.151:0/1816353527 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56780aecc0 msgr2=0x7f567815dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:31.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.927+0000 7f5677e00640 1 --2- 172.21.15.151:0/1816353527 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56780aecc0 0x7f567815dc30 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f567806e840 tx=0x7f566402a540 comp rx=0 tx=0).stop 2024-09-16T15:48:31.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.927+0000 7f5677e00640 1 -- 172.21.15.151:0/1816353527 shutdown_connections 2024-09-16T15:48:31.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.927+0000 7f5677e00640 1 --2- 172.21.15.151:0/1816353527 >> v2:172.21.15.167:3300/0 conn(0x7f56780701e0 0x7f567815e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:31.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.927+0000 7f5677e00640 1 --2- 172.21.15.151:0/1816353527 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56780aecc0 0x7f567815dc30 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:31.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.927+0000 7f5677e00640 1 -- 172.21.15.151:0/1816353527 >> 172.21.15.151:0/1816353527 conn(0x7f56780cfa20 msgr2=0x7f56780cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:31.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.927+0000 7f5677e00640 1 -- 172.21.15.151:0/1816353527 shutdown_connections 2024-09-16T15:48:31.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.928+0000 7f5677e00640 1 -- 172.21.15.151:0/1816353527 wait complete. 2024-09-16T15:48:31.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.929+0000 7f5677e00640 1 Processor -- start 2024-09-16T15:48:31.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.929+0000 7f5677e00640 1 -- start start 2024-09-16T15:48:31.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.930+0000 7f5677e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56780701e0 0x7f56781554e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:31.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.930+0000 7f5677e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f56780aecc0 0x7f5678156c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:31.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.930+0000 7f5677e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56780caa50 0x7f56781577d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:31.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.930+0000 7f5677e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5678073370 con 0x7f56780701e0 2024-09-16T15:48:31.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.930+0000 7f5677e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f56780731f0 con 0x7f56780caa50 2024-09-16T15:48:31.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.930+0000 7f5677e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f56780734f0 con 0x7f56780aecc0 2024-09-16T15:48:31.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.930+0000 7f5676a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56780701e0 0x7f56781554e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:31.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.930+0000 7f5676a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56780701e0 0x7f56781554e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:42848/0 (socket says 172.21.15.151:42848) 2024-09-16T15:48:31.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.930+0000 7f5676a00640 1 -- 172.21.15.151:0/3650040876 learned_addr learned my addr 172.21.15.151:0/3650040876 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:48:31.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.931+0000 7f5677400640 1 --2- 172.21.15.151:0/3650040876 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56780caa50 0x7f56781577d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:31.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.931+0000 7f5676000640 1 --2- 172.21.15.151:0/3650040876 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f56780aecc0 0x7f5678156c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:31.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.931+0000 7f5676a00640 1 -- 172.21.15.151:0/3650040876 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f56780aecc0 msgr2=0x7f5678156c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:31.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.931+0000 7f5676a00640 1 --2- 172.21.15.151:0/3650040876 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f56780aecc0 0x7f5678156c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:31.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.931+0000 7f5676a00640 1 -- 172.21.15.151:0/3650040876 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56780caa50 msgr2=0x7f56781577d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:31.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.931+0000 7f5676a00640 1 --2- 172.21.15.151:0/3650040876 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56780caa50 0x7f56781577d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:31.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.931+0000 7f5676a00640 1 -- 172.21.15.151:0/3650040876 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5678155e80 con 0x7f56780701e0 2024-09-16T15:48:31.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.931+0000 7f5676a00640 1 --2- 172.21.15.151:0/3650040876 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56780701e0 0x7f56781554e0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f566800cbc0 tx=0x7f566800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:31.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.931+0000 7f566ea00640 1 -- 172.21.15.151:0/3650040876 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5668016020 con 0x7f56780701e0 2024-09-16T15:48:31.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.931+0000 7f566ea00640 1 -- 172.21.15.151:0/3650040876 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f566800ad90 con 0x7f56780701e0 2024-09-16T15:48:31.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.931+0000 7f5677e00640 1 -- 172.21.15.151:0/3650040876 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5678156120 con 0x7f56780701e0 2024-09-16T15:48:31.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.932+0000 7f566ea00640 1 -- 172.21.15.151:0/3650040876 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5668012680 con 0x7f56780701e0 2024-09-16T15:48:31.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.932+0000 7f5677e00640 1 -- 172.21.15.151:0/3650040876 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f567815c610 con 0x7f56780701e0 2024-09-16T15:48:31.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.933+0000 7f5677e00640 1 -- 172.21.15.151:0/3650040876 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5678156840 con 0x7f56780701e0 2024-09-16T15:48:31.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.934+0000 7f566ea00640 1 -- 172.21.15.151:0/3650040876 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 30) ==== 102868+0+0 (secure 0 0 0) 0x7f5668019020 con 0x7f56780701e0 2024-09-16T15:48:31.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.935+0000 7f566ea00640 1 --2- 172.21.15.151:0/3650040876 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f56580799c0 0x7f565807be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:31.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.935+0000 7f566ea00640 1 -- 172.21.15.151:0/3650040876 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7f5668099e40 con 0x7f56780701e0 2024-09-16T15:48:31.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.935+0000 7f5676000640 1 --2- 172.21.15.151:0/3650040876 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f56580799c0 0x7f565807be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:31.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.936+0000 7f5676000640 1 --2- 172.21.15.151:0/3650040876 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f56580799c0 0x7f565807be80 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5664034000 tx=0x7f5664004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:31.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:31.939+0000 7f566ea00640 1 -- 172.21.15.151:0/3650040876 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f5668020020 con 0x7f56780701e0 2024-09-16T15:48:32.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.160+0000 7f5677e00640 1 -- 172.21.15.151:0/3650040876 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5678073a10 con 0x7f56780701e0 2024-09-16T15:48:32.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.161+0000 7f566ea00640 1 -- 172.21.15.151:0/3650040876 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+5234 (secure 0 0 0) 0x7f5668063bb0 con 0x7f56780701e0 2024-09-16T15:48:32.163 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:48:32.163 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":56,"btime":"2024-09-16T15:48:24:992606+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6134,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/743718626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":743718626},{"type":"v1","addr":"172.21.15.167:6836","nonce":743718626}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6137,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1262689373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1262689373},{"type":"v1","addr":"172.21.15.167:6837","nonce":1262689373}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6670,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/1127792363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1127792363},{"type":"v1","addr":"172.21.15.151:6834","nonce":1127792363}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:48:21.374084+0000","modified":"2024-09-16T15:48:24.408969+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6673},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6673":{"gid":6673,"name":"c","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.151:6835/1518702790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":1518702790},{"type":"v1","addr":"172.21.15.151:6835","nonce":1518702790}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6673,"qdb_cluster":[6673]},"id":6}]} 2024-09-16T15:48:32.163 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 56 2024-09-16T15:48:32.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.163+0000 7f5677e00640 1 -- 172.21.15.151:0/3650040876 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f56580799c0 msgr2=0x7f565807be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:32.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.163+0000 7f5677e00640 1 --2- 172.21.15.151:0/3650040876 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f56580799c0 0x7f565807be80 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5664034000 tx=0x7f5664004730 comp rx=0 tx=0).stop 2024-09-16T15:48:32.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.163+0000 7f5677e00640 1 -- 172.21.15.151:0/3650040876 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56780701e0 msgr2=0x7f56781554e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:32.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.163+0000 7f5677e00640 1 --2- 172.21.15.151:0/3650040876 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56780701e0 0x7f56781554e0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f566800cbc0 tx=0x7f566800a060 comp rx=0 tx=0).stop 2024-09-16T15:48:32.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.163+0000 7f5677e00640 1 -- 172.21.15.151:0/3650040876 shutdown_connections 2024-09-16T15:48:32.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.163+0000 7f5677e00640 1 --2- 172.21.15.151:0/3650040876 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f56580799c0 0x7f565807be80 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:32.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.163+0000 7f5677e00640 1 --2- 172.21.15.151:0/3650040876 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56780caa50 0x7f56781577d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:32.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.163+0000 7f5677e00640 1 --2- 172.21.15.151:0/3650040876 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f56780aecc0 0x7f5678156c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:32.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.163+0000 7f5677e00640 1 --2- 172.21.15.151:0/3650040876 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56780701e0 0x7f56781554e0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:32.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.163+0000 7f5677e00640 1 -- 172.21.15.151:0/3650040876 >> 172.21.15.151:0/3650040876 conn(0x7f56780cfa20 msgr2=0x7f56780ce3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:32.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.163+0000 7f5677e00640 1 -- 172.21.15.151:0/3650040876 shutdown_connections 2024-09-16T15:48:32.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.163+0000 7f5677e00640 1 -- 172.21.15.151:0/3650040876 wait complete. 2024-09-16T15:48:32.583 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 56, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-16T15:48:21.374084+0000', 'modified': '2024-09-16T15:48:24.408969+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6673}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6673': {'gid': 6673, 'name': 'c', 'rank': 0, 'incarnation': 52, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.151:6835/1518702790', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6833', 'nonce': 1518702790}, {'type': 'v1', 'addr': '172.21.15.151:6835', 'nonce': 1518702790}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6673, 'qdb_cluster': [6673]} 2024-09-16T15:48:32.583 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-16T15:48:32.584 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-16T15:48:32.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.743+0000 7fe193a00640 1 -- 172.21.15.151:0/2491830119 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe18c0ccf00 msgr2=0x7fe18c0acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:32.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.743+0000 7fe193a00640 1 --2- 172.21.15.151:0/2491830119 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe18c0ccf00 0x7fe18c0acd30 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fe178005fd0 tx=0x7fe17802a540 comp rx=0 tx=0).stop 2024-09-16T15:48:32.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.743+0000 7fe193a00640 1 -- 172.21.15.151:0/2491830119 shutdown_connections 2024-09-16T15:48:32.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.743+0000 7fe193a00640 1 --2- 172.21.15.151:0/2491830119 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe18c0ccf00 0x7fe18c0acd30 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:32.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.743+0000 7fe193a00640 1 --2- 172.21.15.151:0/2491830119 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe18c15f5a0 0x7fe18c0b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:32.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.743+0000 7fe193a00640 1 -- 172.21.15.151:0/2491830119 >> 172.21.15.151:0/2491830119 conn(0x7fe18c0b1950 msgr2=0x7fe18c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:32.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.743+0000 7fe193a00640 1 -- 172.21.15.151:0/2491830119 shutdown_connections 2024-09-16T15:48:32.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.743+0000 7fe193a00640 1 -- 172.21.15.151:0/2491830119 wait complete. 2024-09-16T15:48:32.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.745+0000 7fe193a00640 1 Processor -- start 2024-09-16T15:48:32.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.745+0000 7fe193a00640 1 -- start start 2024-09-16T15:48:32.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.746+0000 7fe193a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe18c0ad270 0x7fe18c1f4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:32.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.746+0000 7fe193a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe18c0ccf00 0x7fe18c148450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:32.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.746+0000 7fe193a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe18c15f5a0 0x7fe18c09cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:32.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.746+0000 7fe193a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe18c06f2e0 con 0x7fe18c0ad270 2024-09-16T15:48:32.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.746+0000 7fe193a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe18c06f160 con 0x7fe18c15f5a0 2024-09-16T15:48:32.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.746+0000 7fe193a00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fe18c06f460 con 0x7fe18c0ccf00 2024-09-16T15:48:32.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.746+0000 7fe191000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe18c0ad270 0x7fe18c1f4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:32.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.746+0000 7fe191000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe18c0ad270 0x7fe18c1f4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:42872/0 (socket says 172.21.15.151:42872) 2024-09-16T15:48:32.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.746+0000 7fe191000640 1 -- 172.21.15.151:0/3458789661 learned_addr learned my addr 172.21.15.151:0/3458789661 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:48:32.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.746+0000 7fe18be00640 1 --2- 172.21.15.151:0/3458789661 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe18c0ccf00 0x7fe18c148450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:32.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.746+0000 7fe191a00640 1 --2- 172.21.15.151:0/3458789661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe18c15f5a0 0x7fe18c09cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:32.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.747+0000 7fe191000640 1 -- 172.21.15.151:0/3458789661 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe18c0ccf00 msgr2=0x7fe18c148450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:32.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.747+0000 7fe191000640 1 --2- 172.21.15.151:0/3458789661 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe18c0ccf00 0x7fe18c148450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:32.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.747+0000 7fe191000640 1 -- 172.21.15.151:0/3458789661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe18c15f5a0 msgr2=0x7fe18c09cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:32.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.747+0000 7fe191000640 1 --2- 172.21.15.151:0/3458789661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe18c15f5a0 0x7fe18c09cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:32.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.747+0000 7fe191000640 1 -- 172.21.15.151:0/3458789661 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe18c1563c0 con 0x7fe18c0ad270 2024-09-16T15:48:32.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.747+0000 7fe191000640 1 --2- 172.21.15.151:0/3458789661 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe18c0ad270 0x7fe18c1f4c60 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fe17c00d6d0 tx=0x7fe17c00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:32.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.747+0000 7fe188c00640 1 -- 172.21.15.151:0/3458789661 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe17c017070 con 0x7fe18c0ad270 2024-09-16T15:48:32.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.747+0000 7fe188c00640 1 -- 172.21.15.151:0/3458789661 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe17c005330 con 0x7fe18c0ad270 2024-09-16T15:48:32.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.747+0000 7fe188c00640 1 -- 172.21.15.151:0/3458789661 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe17c009820 con 0x7fe18c0ad270 2024-09-16T15:48:32.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.747+0000 7fe193a00640 1 -- 172.21.15.151:0/3458789661 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe18c0cf360 con 0x7fe18c0ad270 2024-09-16T15:48:32.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.747+0000 7fe193a00640 1 -- 172.21.15.151:0/3458789661 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe18c156090 con 0x7fe18c0ad270 2024-09-16T15:48:32.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.750+0000 7fe188c00640 1 -- 172.21.15.151:0/3458789661 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 30) ==== 102868+0+0 (secure 0 0 0) 0x7fe17c009bd0 con 0x7fe18c0ad270 2024-09-16T15:48:32.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.750+0000 7fe188c00640 1 --2- 172.21.15.151:0/3458789661 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe16c0799c0 0x7fe16c07be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:32.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.751+0000 7fe188c00640 1 -- 172.21.15.151:0/3458789661 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7fe17c099b50 con 0x7fe18c0ad270 2024-09-16T15:48:32.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.751+0000 7fe1969bd740 1 Processor -- start 2024-09-16T15:48:32.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.751+0000 7fe1969bd740 1 -- start start 2024-09-16T15:48:32.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.751+0000 7fe18be00640 1 --2- 172.21.15.151:0/3458789661 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe16c0799c0 0x7fe16c07be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:32.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.752+0000 7fe1969bd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5595cb0888b0 0x5595cb088c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:32.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.752+0000 7fe1969bd740 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5595cb043a20 0x5595cb087390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:32.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.752+0000 7fe1969bd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5595cb043fa0 0x5595cb0878d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:32.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.752+0000 7fe1969bd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5595cafc6430 con 0x5595cb043fa0 2024-09-16T15:48:32.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.752+0000 7fe1969bd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5595cafb65d0 con 0x5595cb0888b0 2024-09-16T15:48:32.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.752+0000 7fe1969bd740 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x5595cafa5a00 con 0x5595cb043a20 2024-09-16T15:48:32.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.752+0000 7fe18be00640 1 --2- 172.21.15.151:0/3458789661 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe16c0799c0 0x7fe16c07be80 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe178033000 tx=0x7fe178004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:32.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.752+0000 7fe191a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5595cb043fa0 0x5595cb0878d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:32.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.752+0000 7fe191a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5595cb043fa0 0x5595cb0878d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:42878/0 (socket says 172.21.15.151:42878) 2024-09-16T15:48:32.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.752+0000 7fe191a00640 1 -- 172.21.15.151:0/2609681788 learned_addr learned my addr 172.21.15.151:0/2609681788 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:48:32.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.752+0000 7fe18be00640 1 --2- 172.21.15.151:0/2609681788 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5595cb043a20 0x5595cb087390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:32.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.752+0000 7fe191000640 1 --2- 172.21.15.151:0/2609681788 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5595cb0888b0 0x5595cb088c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:32.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.752+0000 7fe191a00640 1 -- 172.21.15.151:0/2609681788 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5595cb043a20 msgr2=0x5595cb087390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:48:32.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.752+0000 7fe191a00640 1 --2- 172.21.15.151:0/2609681788 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5595cb043a20 0x5595cb087390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:32.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.752+0000 7fe191a00640 1 -- 172.21.15.151:0/2609681788 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5595cb0888b0 msgr2=0x5595cb088c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:48:32.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.753+0000 7fe191a00640 1 --2- 172.21.15.151:0/2609681788 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5595cb0888b0 0x5595cb088c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:32.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.753+0000 7fe191a00640 1 -- 172.21.15.151:0/2609681788 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595caf6e520 con 0x5595cb043fa0 2024-09-16T15:48:32.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.753+0000 7fe191000640 1 --2- 172.21.15.151:0/2609681788 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5595cb0888b0 0x5595cb088c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:48:32.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.753+0000 7fe191a00640 1 --2- 172.21.15.151:0/2609681788 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5595cb043fa0 0x5595cb0878d0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fe184001c30 tx=0x7fe1840020f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c5031cb38abf335 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:32.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.753+0000 7fe181600640 1 -- 172.21.15.151:0/2609681788 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe184019070 con 0x5595cb043fa0 2024-09-16T15:48:32.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.753+0000 7fe181600640 1 -- 172.21.15.151:0/2609681788 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe184003b80 con 0x5595cb043fa0 2024-09-16T15:48:32.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.753+0000 7fe181600640 1 -- 172.21.15.151:0/2609681788 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18400e860 con 0x5595cb043fa0 2024-09-16T15:48:32.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.754+0000 7fe1969bd740 1 -- 172.21.15.151:0/2609681788 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5595cb043fa0 msgr2=0x5595cb0878d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:32.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.754+0000 7fe1969bd740 1 --2- 172.21.15.151:0/2609681788 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5595cb043fa0 0x5595cb0878d0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fe184001c30 tx=0x7fe1840020f0 comp rx=0 tx=0).stop 2024-09-16T15:48:32.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.754+0000 7fe1969bd740 1 -- 172.21.15.151:0/2609681788 shutdown_connections 2024-09-16T15:48:32.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.754+0000 7fe1969bd740 1 --2- 172.21.15.151:0/2609681788 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5595cb043fa0 0x5595cb0878d0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:32.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.754+0000 7fe1969bd740 1 --2- 172.21.15.151:0/2609681788 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5595cb043a20 0x5595cb087390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:32.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.754+0000 7fe1969bd740 1 --2- 172.21.15.151:0/2609681788 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5595cb0888b0 0x5595cb088c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:32.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.754+0000 7fe1969bd740 1 -- 172.21.15.151:0/2609681788 >> 172.21.15.151:0/2609681788 conn(0x5595cb03eff0 msgr2=0x5595cb042a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:32.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.754+0000 7fe1969bd740 1 -- 172.21.15.151:0/2609681788 shutdown_connections 2024-09-16T15:48:32.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.754+0000 7fe1969bd740 1 -- 172.21.15.151:0/2609681788 wait complete. 2024-09-16T15:48:32.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.755+0000 7fe1969bd740 1 -- start start 2024-09-16T15:48:32.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.755+0000 7fe1969bd740 1 Processor -- start 2024-09-16T15:48:32.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.756+0000 7fe1969bd740 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5595cb043fa0 0x5595cb098720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:32.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.756+0000 7fe1969bd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5595cb0888b0 0x5595cb067180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:32.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.756+0000 7fe1969bd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5595cb09b1e0 0x5595cb07e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:32.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.756+0000 7fe1969bd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5595caf4c330 con 0x5595cb0888b0 2024-09-16T15:48:32.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.756+0000 7fe1969bd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5595cafa9810 con 0x5595cb09b1e0 2024-09-16T15:48:32.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.756+0000 7fe1969bd740 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x5595cad16620 con 0x5595cb043fa0 2024-09-16T15:48:32.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.756+0000 7fe1969bd740 10 client.0 _ll_register_callbacks cb 0x5595caf51420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T15:48:32.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.756+0000 7fe18be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5595cb0888b0 0x5595cb067180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:32.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.756+0000 7fe18be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5595cb0888b0 0x5595cb067180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:42880/0 (socket says 172.21.15.151:42880) 2024-09-16T15:48:32.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.756+0000 7fe18be00640 1 -- 172.21.15.151:0/2059465054 learned_addr learned my addr 172.21.15.151:0/2059465054 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:48:32.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.756+0000 7fe191000640 1 --2- 172.21.15.151:0/2059465054 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5595cb043fa0 0x5595cb098720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:32.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.756+0000 7fe191a00640 1 --2- 172.21.15.151:0/2059465054 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5595cb09b1e0 0x5595cb07e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:32.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.756+0000 7fe18be00640 1 -- 172.21.15.151:0/2059465054 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5595cb043fa0 msgr2=0x5595cb098720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:48:32.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.756+0000 7fe18be00640 1 --2- 172.21.15.151:0/2059465054 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5595cb043fa0 0x5595cb098720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:32.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.756+0000 7fe18be00640 1 -- 172.21.15.151:0/2059465054 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5595cb09b1e0 msgr2=0x5595cb07e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:48:32.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.756+0000 7fe18be00640 1 --2- 172.21.15.151:0/2059465054 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5595cb09b1e0 0x5595cb07e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:32.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.756+0000 7fe18be00640 1 -- 172.21.15.151:0/2059465054 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595cad07850 con 0x5595cb0888b0 2024-09-16T15:48:32.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.757+0000 7fe18be00640 1 --2- 172.21.15.151:0/2059465054 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5595cb0888b0 0x5595cb067180 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fe178004650 tx=0x7fe178009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6df1b54af7a581b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:32.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.757+0000 7fe176000640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-16T15:48:32.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.757+0000 7fe176000640 1 -- 172.21.15.151:0/2059465054 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe17804c070 con 0x5595cb0888b0 2024-09-16T15:48:32.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.757+0000 7fe1969bd740 1 -- 172.21.15.151:0/2059465054 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe184003b80 con 0x5595cb0888b0 2024-09-16T15:48:32.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.757+0000 7fe176000640 1 -- 172.21.15.151:0/2059465054 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7fe17803bac0 con 0x5595cb0888b0 2024-09-16T15:48:32.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.757+0000 7fe176000640 1 -- 172.21.15.151:0/2059465054 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7fe178057050 con 0x5595cb0888b0 2024-09-16T15:48:32.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.757+0000 7fe1969bd740 10 client.7182 fetch_fsmap learned FSMap version 56 2024-09-16T15:48:32.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.757+0000 7fe1969bd740 1 -- 172.21.15.151:0/2059465054 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) -- 0x5595cad7e6b0 con 0x5595cb0888b0 2024-09-16T15:48:32.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.757+0000 7fe176000640 1 -- 172.21.15.151:0/2059465054 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe184003b80 con 0x5595cb0888b0 2024-09-16T15:48:32.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.757+0000 7fe176000640 1 -- 172.21.15.151:0/2059465054 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 56) ==== 3502+0+0 (secure 0 0 0) 0x7fe17805a020 con 0x5595cb0888b0 2024-09-16T15:48:32.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.758+0000 7fe1969bd740 10 client.7182 fetch_fsmap finished waiting for FSMap version 56 2024-09-16T15:48:32.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.758+0000 7fe1969bd740 10 client.7182 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-16T15:48:32.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.758+0000 7fe1969bd740 20 client.7182 populate_metadata read hostname 'smithi151' 2024-09-16T15:48:32.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.758+0000 7fe1969bd740 1 --2- 172.21.15.151:0/2059465054 >> [v2:172.21.15.151:6833/1518702790,v1:172.21.15.151:6835/1518702790] conn(0x5595cb081f10 0x5595cb0822f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:32.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.758+0000 7fe1969bd740 4 client.7182 mds_command: new command op to 6673 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:48:32.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.758+0000 7fe1969bd740 1 -- 172.21.15.151:0/2059465054 --> [v2:172.21.15.151:6833/1518702790,v1:172.21.15.151:6835/1518702790] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5595cafb8520 con 0x5595cb081f10 2024-09-16T15:48:32.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.758+0000 7fe191000640 1 --2- 172.21.15.151:0/2059465054 >> [v2:172.21.15.151:6833/1518702790,v1:172.21.15.151:6835/1518702790] conn(0x5595cb081f10 0x5595cb0822f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:32.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.758+0000 7fe191000640 1 --2- 172.21.15.151:0/2059465054 >> [v2:172.21.15.151:6833/1518702790,v1:172.21.15.151:6835/1518702790] conn(0x5595cb081f10 0x5595cb0822f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af3b3950f575c559 server_cookie=21511f60a48218de in_seq=0 out_seq=0 2024-09-16T15:48:32.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.758+0000 7fe176000640 10 client.7182 ms_handle_connect on v2:172.21.15.151:6833/1518702790 2024-09-16T15:48:32.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.759+0000 7fe176000640 1 -- 172.21.15.151:0/2059465054 <== mds.0 v2:172.21.15.151:6833/1518702790 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5595cafb8520 con 0x5595cb081f10 2024-09-16T15:48:32.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.759+0000 7fe176000640 10 client.7182 handle_command_reply: tid=0 2024-09-16T15:48:32.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.759+0000 7fe1969bd740 1 client.7182 shutdown 2024-09-16T15:48:32.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.759+0000 7fe1969bd740 1 -- 172.21.15.151:0/2059465054 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5595cb0888b0 msgr2=0x5595cb067180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:32.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.759+0000 7fe1969bd740 1 --2- 172.21.15.151:0/2059465054 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5595cb0888b0 0x5595cb067180 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fe178004650 tx=0x7fe178009260 comp rx=0 tx=0).stop 2024-09-16T15:48:32.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.759+0000 7fe1969bd740 1 -- 172.21.15.151:0/2059465054 shutdown_connections 2024-09-16T15:48:32.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.759+0000 7fe1969bd740 1 --2- 172.21.15.151:0/2059465054 >> [v2:172.21.15.151:6833/1518702790,v1:172.21.15.151:6835/1518702790] conn(0x5595cb081f10 0x5595cb0822f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:32.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.759+0000 7fe1969bd740 1 --2- 172.21.15.151:0/2059465054 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5595cb09b1e0 0x5595cb07e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:32.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.759+0000 7fe1969bd740 1 --2- 172.21.15.151:0/2059465054 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5595cb0888b0 0x5595cb067180 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:32.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.759+0000 7fe1969bd740 1 --2- 172.21.15.151:0/2059465054 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5595cb043fa0 0x5595cb098720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:32.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.759+0000 7fe176000640 0 client.7182 ms_handle_reset on v2:172.21.15.151:6833/1518702790 2024-09-16T15:48:32.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.759+0000 7fe1969bd740 1 -- 172.21.15.151:0/2059465054 >> 172.21.15.151:0/2059465054 conn(0x5595cb043a20 msgr2=0x5595cb07de90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:32.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.760+0000 7fe1969bd740 1 -- 172.21.15.151:0/2059465054 shutdown_connections 2024-09-16T15:48:32.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.760+0000 7fe1969bd740 1 -- 172.21.15.151:0/2059465054 wait complete. 2024-09-16T15:48:32.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.760+0000 7fe1969bd740 20 client.7182 trim_cache size 0 max 16384 2024-09-16T15:48:32.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.778+0000 7fe1969bd740 1 Processor -- start 2024-09-16T15:48:32.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.779+0000 7fe1969bd740 1 -- start start 2024-09-16T15:48:32.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.779+0000 7fe1969bd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5595cb043fa0 0x5595cb0966f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:32.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.779+0000 7fe1969bd740 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5595cb0888b0 0x5595cb096c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:32.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.779+0000 7fe1969bd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5595cb09b1e0 0x5595cb0ab6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:32.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.779+0000 7fe1969bd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe18c2004a0 con 0x5595cb043fa0 2024-09-16T15:48:32.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.779+0000 7fe1969bd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe18c0cde80 con 0x5595cb09b1e0 2024-09-16T15:48:32.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.779+0000 7fe1969bd740 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fe18c0ce4d0 con 0x5595cb0888b0 2024-09-16T15:48:32.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.780+0000 7fe191000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5595cb043fa0 0x5595cb0966f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:32.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.780+0000 7fe18be00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5595cb0888b0 0x5595cb096c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:32.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.780+0000 7fe18be00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5595cb0888b0 0x5595cb096c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.151:36746/0 (socket says 172.21.15.151:36746) 2024-09-16T15:48:32.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.780+0000 7fe191000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5595cb043fa0 0x5595cb0966f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:42888/0 (socket says 172.21.15.151:42888) 2024-09-16T15:48:32.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.780+0000 7fe18be00640 1 -- 172.21.15.151:0/736590493 learned_addr learned my addr 172.21.15.151:0/736590493 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:48:32.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.780+0000 7fe191a00640 1 --2- 172.21.15.151:0/736590493 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5595cb09b1e0 0x5595cb0ab6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:32.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.780+0000 7fe191000640 1 -- 172.21.15.151:0/736590493 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5595cb0888b0 msgr2=0x5595cb096c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:48:32.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.780+0000 7fe191000640 1 --2- 172.21.15.151:0/736590493 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5595cb0888b0 0x5595cb096c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:32.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.780+0000 7fe191000640 1 -- 172.21.15.151:0/736590493 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5595cb09b1e0 msgr2=0x5595cb0ab6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:48:32.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.780+0000 7fe191000640 1 --2- 172.21.15.151:0/736590493 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5595cb09b1e0 0x5595cb0ab6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:32.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.780+0000 7fe191000640 1 -- 172.21.15.151:0/736590493 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595cb080d60 con 0x5595cb043fa0 2024-09-16T15:48:32.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.780+0000 7fe191000640 1 --2- 172.21.15.151:0/736590493 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5595cb043fa0 0x5595cb0966f0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fe17c099a60 tx=0x7fe17c06af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fc5377ada67c5dc server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:32.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.781+0000 7fe180c00640 1 -- 172.21.15.151:0/736590493 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe17c020040 con 0x5595cb043fa0 2024-09-16T15:48:32.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.781+0000 7fe180c00640 1 -- 172.21.15.151:0/736590493 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe17c067ca0 con 0x5595cb043fa0 2024-09-16T15:48:32.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.781+0000 7fe180c00640 1 -- 172.21.15.151:0/736590493 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe17c06cb50 con 0x5595cb043fa0 2024-09-16T15:48:32.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.781+0000 7fe1969bd740 1 -- 172.21.15.151:0/736590493 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5595cb043fa0 msgr2=0x5595cb0966f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:32.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.781+0000 7fe1969bd740 1 --2- 172.21.15.151:0/736590493 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5595cb043fa0 0x5595cb0966f0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fe17c099a60 tx=0x7fe17c06af90 comp rx=0 tx=0).stop 2024-09-16T15:48:32.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.781+0000 7fe1969bd740 1 -- 172.21.15.151:0/736590493 shutdown_connections 2024-09-16T15:48:32.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.781+0000 7fe1969bd740 1 --2- 172.21.15.151:0/736590493 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5595cb09b1e0 0x5595cb0ab6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:32.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.781+0000 7fe1969bd740 1 --2- 172.21.15.151:0/736590493 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5595cb0888b0 0x5595cb096c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:32.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.781+0000 7fe1969bd740 1 --2- 172.21.15.151:0/736590493 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5595cb043fa0 0x5595cb0966f0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:32.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.781+0000 7fe1969bd740 1 -- 172.21.15.151:0/736590493 >> 172.21.15.151:0/736590493 conn(0x5595cb043a20 msgr2=0x5595cb06c260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:32.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.781+0000 7fe1969bd740 1 -- 172.21.15.151:0/736590493 shutdown_connections 2024-09-16T15:48:32.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.781+0000 7fe1969bd740 1 -- 172.21.15.151:0/736590493 wait complete. 2024-09-16T15:48:32.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.782+0000 7fe1969bd740 1 -- start start 2024-09-16T15:48:32.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.782+0000 7fe1969bd740 1 Processor -- start 2024-09-16T15:48:32.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.783+0000 7fe1969bd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5595cb043fa0 0x5595cb1060a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:32.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.783+0000 7fe1969bd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5595cb0888b0 0x5595cb108b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:32.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.783+0000 7fe1969bd740 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5595cb09b1e0 0x5595cb087b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:32.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.783+0000 7fe1969bd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5595cb0676c0 con 0x5595cb0888b0 2024-09-16T15:48:32.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.783+0000 7fe1969bd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5595cb07f080 con 0x5595cb043fa0 2024-09-16T15:48:32.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.783+0000 7fe1969bd740 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x5595cb080f50 con 0x5595cb09b1e0 2024-09-16T15:48:32.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.783+0000 7fe1969bd740 10 client.0 _ll_register_callbacks cb 0x7fe178003750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T15:48:32.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.783+0000 7fe18be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5595cb0888b0 0x5595cb108b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:32.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.783+0000 7fe18be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5595cb0888b0 0x5595cb108b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:42898/0 (socket says 172.21.15.151:42898) 2024-09-16T15:48:32.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.783+0000 7fe18be00640 1 -- 172.21.15.151:0/1290501197 learned_addr learned my addr 172.21.15.151:0/1290501197 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:48:32.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.784+0000 7fe191000640 1 --2- 172.21.15.151:0/1290501197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5595cb043fa0 0x5595cb1060a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:32.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.784+0000 7fe191a00640 1 --2- 172.21.15.151:0/1290501197 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5595cb09b1e0 0x5595cb087b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:32.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.784+0000 7fe18be00640 1 -- 172.21.15.151:0/1290501197 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5595cb09b1e0 msgr2=0x5595cb087b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:48:32.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.784+0000 7fe18be00640 1 --2- 172.21.15.151:0/1290501197 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5595cb09b1e0 0x5595cb087b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:32.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.784+0000 7fe18be00640 1 -- 172.21.15.151:0/1290501197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5595cb043fa0 msgr2=0x5595cb1060a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:48:32.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.784+0000 7fe18be00640 1 --2- 172.21.15.151:0/1290501197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5595cb043fa0 0x5595cb1060a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:32.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.784+0000 7fe18be00640 1 -- 172.21.15.151:0/1290501197 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595cb097010 con 0x5595cb0888b0 2024-09-16T15:48:32.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.784+0000 7fe191a00640 1 --2- 172.21.15.151:0/1290501197 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5595cb09b1e0 0x5595cb087b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:48:32.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.784+0000 7fe191000640 1 --2- 172.21.15.151:0/1290501197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5595cb043fa0 0x5595cb1060a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:48:32.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.784+0000 7fe18be00640 1 --2- 172.21.15.151:0/1290501197 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5595cb0888b0 0x5595cb108b50 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fe178009850 tx=0x7fe178051670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee72350b5e2bfe8c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:32.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.784+0000 7fe176000640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-16T15:48:32.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.784+0000 7fe176000640 1 -- 172.21.15.151:0/1290501197 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe178060060 con 0x5595cb0888b0 2024-09-16T15:48:32.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.784+0000 7fe176000640 1 -- 172.21.15.151:0/1290501197 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7fe178051bc0 con 0x5595cb0888b0 2024-09-16T15:48:32.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.784+0000 7fe1969bd740 1 -- 172.21.15.151:0/1290501197 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe17c067ca0 con 0x5595cb0888b0 2024-09-16T15:48:32.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.784+0000 7fe176000640 1 -- 172.21.15.151:0/1290501197 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7fe178066050 con 0x5595cb0888b0 2024-09-16T15:48:32.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.784+0000 7fe1969bd740 10 client.7188 fetch_fsmap learned FSMap version 56 2024-09-16T15:48:32.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.784+0000 7fe1969bd740 1 -- 172.21.15.151:0/1290501197 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) -- 0x5595cb097f20 con 0x5595cb0888b0 2024-09-16T15:48:32.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.785+0000 7fe176000640 1 -- 172.21.15.151:0/1290501197 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe17c067ca0 con 0x5595cb0888b0 2024-09-16T15:48:32.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.785+0000 7fe176000640 1 -- 172.21.15.151:0/1290501197 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 56) ==== 3502+0+0 (secure 0 0 0) 0x7fe178069020 con 0x5595cb0888b0 2024-09-16T15:48:32.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.785+0000 7fe1969bd740 10 client.7188 fetch_fsmap finished waiting for FSMap version 56 2024-09-16T15:48:32.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.785+0000 7fe1969bd740 10 client.7188 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-16T15:48:32.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.785+0000 7fe1969bd740 20 client.7188 populate_metadata read hostname 'smithi151' 2024-09-16T15:48:32.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.785+0000 7fe1969bd740 1 --2- 172.21.15.151:0/1290501197 >> [v2:172.21.15.151:6833/1518702790,v1:172.21.15.151:6835/1518702790] conn(0x5595cb042830 0x5595cb0822f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:32.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.785+0000 7fe1969bd740 4 client.7188 mds_command: new command op to 6673 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T15:48:32.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.785+0000 7fe1969bd740 1 -- 172.21.15.151:0/1290501197 --> [v2:172.21.15.151:6833/1518702790,v1:172.21.15.151:6835/1518702790] -- command(tid 0: {"prefix": "status"}) -- 0x5595cafb3fd0 con 0x5595cb042830 2024-09-16T15:48:32.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.785+0000 7fe191000640 1 --2- 172.21.15.151:0/1290501197 >> [v2:172.21.15.151:6833/1518702790,v1:172.21.15.151:6835/1518702790] conn(0x5595cb042830 0x5595cb0822f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:32.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.785+0000 7fe191000640 1 --2- 172.21.15.151:0/1290501197 >> [v2:172.21.15.151:6833/1518702790,v1:172.21.15.151:6835/1518702790] conn(0x5595cb042830 0x5595cb0822f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ab61d7e09bcaf88 server_cookie=56b842245b1d2b9a in_seq=0 out_seq=0 2024-09-16T15:48:32.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.785+0000 7fe176000640 10 client.7188 ms_handle_connect on v2:172.21.15.151:6833/1518702790 2024-09-16T15:48:32.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.786+0000 7fe176000640 1 -- 172.21.15.151:0/1290501197 <== mds.0 v2:172.21.15.151:6833/1518702790 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x5595cafb3fd0 con 0x5595cb042830 2024-09-16T15:48:32.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.786+0000 7fe176000640 10 client.7188 handle_command_reply: tid=0 2024-09-16T15:48:32.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.786+0000 7fe1969bd740 1 client.7188 shutdown 2024-09-16T15:48:32.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.786+0000 7fe1969bd740 1 -- 172.21.15.151:0/1290501197 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5595cb0888b0 msgr2=0x5595cb108b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:32.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.786+0000 7fe1969bd740 1 --2- 172.21.15.151:0/1290501197 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5595cb0888b0 0x5595cb108b50 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fe178009850 tx=0x7fe178051670 comp rx=0 tx=0).stop 2024-09-16T15:48:32.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.786+0000 7fe1969bd740 1 -- 172.21.15.151:0/1290501197 shutdown_connections 2024-09-16T15:48:32.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.786+0000 7fe1969bd740 1 --2- 172.21.15.151:0/1290501197 >> [v2:172.21.15.151:6833/1518702790,v1:172.21.15.151:6835/1518702790] conn(0x5595cb042830 0x5595cb0822f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:32.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.786+0000 7fe1969bd740 1 --2- 172.21.15.151:0/1290501197 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5595cb09b1e0 0x5595cb087b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:32.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.786+0000 7fe1969bd740 1 --2- 172.21.15.151:0/1290501197 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5595cb0888b0 0x5595cb108b50 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:32.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.786+0000 7fe176000640 0 client.7188 ms_handle_reset on v2:172.21.15.151:6833/1518702790 2024-09-16T15:48:32.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.786+0000 7fe1969bd740 1 --2- 172.21.15.151:0/1290501197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5595cb043fa0 0x5595cb1060a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:32.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.786+0000 7fe1969bd740 1 -- 172.21.15.151:0/1290501197 >> 172.21.15.151:0/1290501197 conn(0x5595cb043a20 msgr2=0x5595cb083510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:32.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.786+0000 7fe1969bd740 1 -- 172.21.15.151:0/1290501197 shutdown_connections 2024-09-16T15:48:32.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.786+0000 7fe1969bd740 1 -- 172.21.15.151:0/1290501197 wait complete. 2024-09-16T15:48:32.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.786+0000 7fe1969bd740 20 client.7188 trim_cache size 0 max 16384 2024-09-16T15:48:32.793 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-16T15:48:32.793 INFO:teuthology.orchestra.run.smithi151.stdout: "cluster_fsid": "1757be18-e41a-4446-bb4f-ee4ac81c39df", 2024-09-16T15:48:32.793 INFO:teuthology.orchestra.run.smithi151.stdout: "whoami": 0, 2024-09-16T15:48:32.793 INFO:teuthology.orchestra.run.smithi151.stdout: "id": 6673, 2024-09-16T15:48:32.793 INFO:teuthology.orchestra.run.smithi151.stdout: "want_state": "up:active", 2024-09-16T15:48:32.793 INFO:teuthology.orchestra.run.smithi151.stdout: "state": "up:active", 2024-09-16T15:48:32.793 INFO:teuthology.orchestra.run.smithi151.stdout: "fs_name": "cephfs", 2024-09-16T15:48:32.793 INFO:teuthology.orchestra.run.smithi151.stdout: "rank_uptime": 10.812711179000001, 2024-09-16T15:48:32.793 INFO:teuthology.orchestra.run.smithi151.stdout: "mdsmap_epoch": 56, 2024-09-16T15:48:32.793 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_epoch": 91, 2024-09-16T15:48:32.793 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_epoch_barrier": 91, 2024-09-16T15:48:32.793 INFO:teuthology.orchestra.run.smithi151.stdout: "uptime": 15.150685908 2024-09-16T15:48:32.794 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-16T15:48:32.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.787+0000 7fe193a00640 1 -- 172.21.15.151:0/3458789661 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe16c0799c0 msgr2=0x7fe16c07be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:32.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.787+0000 7fe193a00640 1 --2- 172.21.15.151:0/3458789661 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe16c0799c0 0x7fe16c07be80 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe178033000 tx=0x7fe178004730 comp rx=0 tx=0).stop 2024-09-16T15:48:32.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.787+0000 7fe193a00640 1 -- 172.21.15.151:0/3458789661 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe18c0ad270 msgr2=0x7fe18c1f4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:32.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.787+0000 7fe193a00640 1 --2- 172.21.15.151:0/3458789661 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe18c0ad270 0x7fe18c1f4c60 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fe17c00d6d0 tx=0x7fe17c00db90 comp rx=0 tx=0).stop 2024-09-16T15:48:32.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.787+0000 7fe193a00640 1 -- 172.21.15.151:0/3458789661 shutdown_connections 2024-09-16T15:48:32.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.787+0000 7fe193a00640 1 --2- 172.21.15.151:0/3458789661 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe16c0799c0 0x7fe16c07be80 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:32.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.787+0000 7fe193a00640 1 --2- 172.21.15.151:0/3458789661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe18c15f5a0 0x7fe18c09cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:32.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.787+0000 7fe193a00640 1 --2- 172.21.15.151:0/3458789661 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe18c0ccf00 0x7fe18c148450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:32.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.787+0000 7fe193a00640 1 --2- 172.21.15.151:0/3458789661 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe18c0ad270 0x7fe18c1f4c60 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:32.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.787+0000 7fe193a00640 1 -- 172.21.15.151:0/3458789661 >> 172.21.15.151:0/3458789661 conn(0x7fe18c0b1950 msgr2=0x7fe18c0b0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:32.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.787+0000 7fe193a00640 1 -- 172.21.15.151:0/3458789661 shutdown_connections 2024-09-16T15:48:32.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.787+0000 7fe193a00640 1 -- 172.21.15.151:0/3458789661 wait complete. 2024-09-16T15:48:32.805 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-16T15:48:32.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.962+0000 7f8102600640 1 -- 172.21.15.151:0/2315490553 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f80e0003aa0 msgr2=0x7f80e0003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:32.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.962+0000 7f8102600640 1 --2- 172.21.15.151:0/2315490553 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f80e0003aa0 0x7f80e0003ec0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f80ec009b20 tx=0x7f80e0005830 comp rx=0 tx=0).stop 2024-09-16T15:48:32.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.962+0000 7f8102600640 1 -- 172.21.15.151:0/2315490553 shutdown_connections 2024-09-16T15:48:32.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.962+0000 7f8102600640 1 --2- 172.21.15.151:0/2315490553 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f80e0003aa0 0x7f80e0003ec0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:32.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.962+0000 7f8102600640 1 -- 172.21.15.151:0/2315490553 >> 172.21.15.151:0/2315490553 conn(0x7f80fc0cfa20 msgr2=0x7f80fc0cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:32.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.963+0000 7f8102600640 1 -- 172.21.15.151:0/2315490553 shutdown_connections 2024-09-16T15:48:32.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.963+0000 7f8102600640 1 -- 172.21.15.151:0/2315490553 wait complete. 2024-09-16T15:48:32.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.964+0000 7f8102600640 1 Processor -- start 2024-09-16T15:48:32.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.964+0000 7f8102600640 1 -- start start 2024-09-16T15:48:32.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.965+0000 7f8102600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f80e0003aa0 0x7f80fc1f46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:32.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.965+0000 7f8102600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80fc1f8250 0x7f80fc0b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:32.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.965+0000 7f8102600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80fc1f86d0 0x7f80fc0b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:32.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.965+0000 7f8102600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f80fc15ddb0 con 0x7f80e0003aa0 2024-09-16T15:48:32.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.965+0000 7f8102600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f80fc15dc30 con 0x7f80fc1f86d0 2024-09-16T15:48:32.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.965+0000 7f8102600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f80fc15df30 con 0x7f80fc1f8250 2024-09-16T15:48:32.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.966+0000 7f8101200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f80e0003aa0 0x7f80fc1f46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:32.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.966+0000 7f8101200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f80e0003aa0 0x7f80fc1f46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:51850/0 (socket says 172.21.15.151:51850) 2024-09-16T15:48:32.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.966+0000 7f8101200640 1 -- 172.21.15.151:0/4250262217 learned_addr learned my addr 172.21.15.151:0/4250262217 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:48:32.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.966+0000 7f8101200640 1 -- 172.21.15.151:0/4250262217 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80fc1f8250 msgr2=0x7f80fc0b28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:32.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.966+0000 7f8101200640 1 --2- 172.21.15.151:0/4250262217 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80fc1f8250 0x7f80fc0b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:32.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.966+0000 7f8101200640 1 -- 172.21.15.151:0/4250262217 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80fc1f86d0 msgr2=0x7f80fc0b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:32.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.966+0000 7f8101200640 1 --2- 172.21.15.151:0/4250262217 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80fc1f86d0 0x7f80fc0b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:32.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.966+0000 7f8101200640 1 -- 172.21.15.151:0/4250262217 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80fc1f8c30 con 0x7f80e0003aa0 2024-09-16T15:48:32.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.966+0000 7f8101200640 1 --2- 172.21.15.151:0/4250262217 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f80e0003aa0 0x7f80fc1f46b0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f80ec009c40 tx=0x7f80ec0336f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:32.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.966+0000 7f80f8c00640 1 -- 172.21.15.151:0/4250262217 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ec038040 con 0x7f80e0003aa0 2024-09-16T15:48:32.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.967+0000 7f80f8c00640 1 -- 172.21.15.151:0/4250262217 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f80ec041070 con 0x7f80e0003aa0 2024-09-16T15:48:32.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.967+0000 7f80f8c00640 1 -- 172.21.15.151:0/4250262217 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ec03c430 con 0x7f80e0003aa0 2024-09-16T15:48:32.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.967+0000 7f8102600640 1 -- 172.21.15.151:0/4250262217 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80fc1f8f10 con 0x7f80e0003aa0 2024-09-16T15:48:32.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.967+0000 7f8102600640 1 -- 172.21.15.151:0/4250262217 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80fc1f5e20 con 0x7f80e0003aa0 2024-09-16T15:48:32.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.968+0000 7f8102600640 1 -- 172.21.15.151:0/4250262217 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80fc1f9150 con 0x7f80e0003aa0 2024-09-16T15:48:32.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.969+0000 7f80f8c00640 1 -- 172.21.15.151:0/4250262217 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 30) ==== 102868+0+0 (secure 0 0 0) 0x7f80ec02b870 con 0x7f80e0003aa0 2024-09-16T15:48:32.972 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.970+0000 7f80f8c00640 1 --2- 172.21.15.151:0/4250262217 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f80dc0799f0 0x7f80dc07beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:32.972 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.970+0000 7f80f8c00640 1 -- 172.21.15.151:0/4250262217 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7f80ec0ba060 con 0x7f80e0003aa0 2024-09-16T15:48:32.972 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.970+0000 7f80fbe00640 1 --2- 172.21.15.151:0/4250262217 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f80dc0799f0 0x7f80dc07beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:32.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.971+0000 7f80fbe00640 1 --2- 172.21.15.151:0/4250262217 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f80dc0799f0 0x7f80dc07beb0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f80e8000fb0 tx=0x7f80e8001be0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:32.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:32.974+0000 7f80f8c00640 1 -- 172.21.15.151:0/4250262217 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f80fc1f9150 con 0x7f80e0003aa0 2024-09-16T15:48:33.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:33.195+0000 7f8102600640 1 -- 172.21.15.151:0/4250262217 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f80fc15e430 con 0x7f80e0003aa0 2024-09-16T15:48:33.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:33.195+0000 7f80f8c00640 1 -- 172.21.15.151:0/4250262217 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v56) ==== 53+0+73 (secure 0 0 0) 0x7f80ec083dd0 con 0x7f80e0003aa0 2024-09-16T15:48:33.197 INFO:teuthology.orchestra.run.smithi151.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-16T15:48:33.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:33.198+0000 7f8102600640 1 -- 172.21.15.151:0/4250262217 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f80dc0799f0 msgr2=0x7f80dc07beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:33.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:33.198+0000 7f8102600640 1 --2- 172.21.15.151:0/4250262217 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f80dc0799f0 0x7f80dc07beb0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f80e8000fb0 tx=0x7f80e8001be0 comp rx=0 tx=0).stop 2024-09-16T15:48:33.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:33.198+0000 7f8102600640 1 -- 172.21.15.151:0/4250262217 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f80e0003aa0 msgr2=0x7f80fc1f46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:33.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:33.198+0000 7f8102600640 1 --2- 172.21.15.151:0/4250262217 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f80e0003aa0 0x7f80fc1f46b0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f80ec009c40 tx=0x7f80ec0336f0 comp rx=0 tx=0).stop 2024-09-16T15:48:33.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:33.198+0000 7f8102600640 1 -- 172.21.15.151:0/4250262217 shutdown_connections 2024-09-16T15:48:33.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:33.198+0000 7f8102600640 1 --2- 172.21.15.151:0/4250262217 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f80dc0799f0 0x7f80dc07beb0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:33.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:33.198+0000 7f8102600640 1 --2- 172.21.15.151:0/4250262217 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f80fc1f86d0 0x7f80fc0b2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:33.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:33.198+0000 7f8102600640 1 --2- 172.21.15.151:0/4250262217 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f80fc1f8250 0x7f80fc0b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:33.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:33.198+0000 7f8102600640 1 --2- 172.21.15.151:0/4250262217 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f80e0003aa0 0x7f80fc1f46b0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:33.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:33.198+0000 7f8102600640 1 -- 172.21.15.151:0/4250262217 >> 172.21.15.151:0/4250262217 conn(0x7f80fc0cfa20 msgr2=0x7f80fc0af300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:33.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:33.198+0000 7f8102600640 1 -- 172.21.15.151:0/4250262217 shutdown_connections 2024-09-16T15:48:33.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:33.198+0000 7f8102600640 1 -- 172.21.15.151:0/4250262217 wait complete. 2024-09-16T15:48:33.217 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-16T15:48:33.217 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T15:48:33.217 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi151.front.sepia.ceph.com 2024-09-16T15:48:33.217 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-16T15:48:33.217 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-16T15:48:33.217 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-16T15:48:33.217 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-16T15:48:33.217 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T15:48:33.217 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T15:48:33.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:48:33.218 DEBUG:teuthology.orchestra.run.smithi151:> ip addr 2024-09-16T15:48:33.253 INFO:teuthology.orchestra.run.smithi151.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T15:48:33.253 INFO:teuthology.orchestra.run.smithi151.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T15:48:33.254 INFO:teuthology.orchestra.run.smithi151.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T15:48:33.254 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:48:33.254 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 ::1/128 scope host 2024-09-16T15:48:33.254 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:48:33.254 INFO:teuthology.orchestra.run.smithi151.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:48:33.254 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:da:6e:0c brd ff:ff:ff:ff:ff:ff 2024-09-16T15:48:33.254 INFO:teuthology.orchestra.run.smithi151.stdout: altname enp7s0f0 2024-09-16T15:48:33.254 INFO:teuthology.orchestra.run.smithi151.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:48:33.254 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:da:6e:0d brd ff:ff:ff:ff:ff:ff 2024-09-16T15:48:33.254 INFO:teuthology.orchestra.run.smithi151.stdout: altname enp7s0f1 2024-09-16T15:48:33.254 INFO:teuthology.orchestra.run.smithi151.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:48:33.254 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:88:6f:62 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:48:33.254 INFO:teuthology.orchestra.run.smithi151.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T15:48:33.255 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:88:6f:63 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:48:33.255 INFO:teuthology.orchestra.run.smithi151.stdout: inet 172.21.15.151/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-16T15:48:33.255 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft 42176sec preferred_lft 42176sec 2024-09-16T15:48:33.255 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 fe80::ec4:7aff:fe88:6f63/64 scope link noprefixroute 2024-09-16T15:48:33.255 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:48:33.255 INFO:teuthology.orchestra.run.smithi151.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-16T15:48:33.255 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 62:3f:3b:48:ad:f8 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:48:33.255 INFO:teuthology.orchestra.run.smithi151.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-16T15:48:33.255 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:48:33.255 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 fe80::40fa:7fff:fedf:2a00/64 scope link 2024-09-16T15:48:33.255 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:48:33.255 INFO:teuthology.orchestra.run.smithi151.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T15:48:33.255 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 62:3f:3b:48:ad:f8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T15:48:33.255 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 fe80::603f:3bff:fe48:adf8/64 scope link 2024-09-16T15:48:33.255 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:48:33.256 INFO:teuthology.orchestra.run.smithi151.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T15:48:33.256 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether b6:ac:fe:34:a7:66 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-16T15:48:33.256 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 fe80::b4ac:feff:fe34:a766/64 scope link 2024-09-16T15:48:33.256 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:48:33.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:48:33.256 DEBUG:teuthology.orchestra.run.smithi151:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T15:48:33.347 INFO:teuthology.orchestra.run.smithi151.stdout:1 2024-09-16T15:48:33.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:48:33.351 DEBUG:teuthology.orchestra.run.smithi151:> route 2024-09-16T15:48:33.386 INFO:teuthology.orchestra.run.smithi151.stdout:Kernel IP routing table 2024-09-16T15:48:33.386 INFO:teuthology.orchestra.run.smithi151.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T15:48:33.386 INFO:teuthology.orchestra.run.smithi151.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-16T15:48:33.386 INFO:teuthology.orchestra.run.smithi151.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-16T15:48:33.387 INFO:teuthology.orchestra.run.smithi151.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T15:48:33.387 DEBUG:teuthology.orchestra.run.smithi151:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T15:48:33.387 DEBUG:teuthology.orchestra.run.smithi151:> set -e 2024-09-16T15:48:33.387 DEBUG:teuthology.orchestra.run.smithi151:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-16T15:48:33.387 DEBUG:teuthology.orchestra.run.smithi151:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-16T15:48:33.387 DEBUG:teuthology.orchestra.run.smithi151:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-16T15:48:33.387 DEBUG:teuthology.orchestra.run.smithi151:> ') 2024-09-16T15:48:33.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:48:33.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:48:33.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:48:33.584 DEBUG:teuthology.orchestra.run.smithi151:> ip netns list 2024-09-16T15:48:33.620 INFO:teuthology.orchestra.run.smithi151.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-16T15:48:33.620 INFO:teuthology.orchestra.run.smithi151.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-16T15:48:33.620 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T15:48:33.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:48:33.620 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip link set brx.0 up 2024-09-16T15:48:33.716 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:48:33.716 DEBUG:teuthology.orchestra.run.smithi151:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-16T15:48:33.751 INFO:teuthology.orchestra.run.smithi151.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-16T15:48:33.752 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:48:33.752 DEBUG:teuthology.orchestra.run.smithi151:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-16T15:48:33.826 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-16T15:48:33.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:48:33.827 DEBUG:teuthology.orchestra.run.smithi151:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-16T15:48:33.955 INFO:teuthology.orchestra.run.smithi151.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-16T15:48:33.955 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-16T15:48:33.955 INFO:teuthology.orchestra.run.smithi151.stdout:invalid new device string format 2024-09-16T15:48:33.971 INFO:teuthology.orchestra.run.smithi151.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-16T15:48:33.971 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-16T15:48:33.971 INFO:teuthology.orchestra.run.smithi151.stdout:invalid new device string format 2024-09-16T15:48:33.971 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: resolved to: "172.21.15.151:3300,172.21.15.167:3300,172.21.15.167:3301" 2024-09-16T15:48:33.971 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-16T15:48:33.993 INFO:teuthology.orchestra.run.smithi151.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-16T15:48:33.993 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-16T15:48:33.993 INFO:teuthology.orchestra.run.smithi151.stdout:invalid new device string format 2024-09-16T15:48:33.994 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: resolved to: "172.21.15.151:3300,172.21.15.167:3300,172.21.15.167:3301" 2024-09-16T15:48:33.994 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-16T15:48:33.994 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: resolved to: "172.21.15.151:6789,172.21.15.167:6789,172.21.15.167:6790" 2024-09-16T15:48:33.994 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.151:6789,172.21.15.167:6789,172.21.15.167:6790:/ 2024-09-16T15:48:33.994 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=1757be18-e41a-4446-bb4f-ee4ac81c39df" will pass to kernel 2024-09-16T15:48:33.995 INFO:teuthology.orchestra.run.smithi151.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-16T15:48:33.995 INFO:teuthology.orchestra.run.smithi151.stdout: You just mounted a file system that supports labels which does not 2024-09-16T15:48:33.995 INFO:teuthology.orchestra.run.smithi151.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-16T15:48:33.995 INFO:teuthology.orchestra.run.smithi151.stdout: applications will generate AVC messages and not be allowed access to 2024-09-16T15:48:33.995 INFO:teuthology.orchestra.run.smithi151.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-16T15:48:33.998 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-16T15:48:33.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:48:33.998 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-16T15:48:34.054 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T15:48:34.134 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:48:34.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:48:34.151 INFO:teuthology.orchestra.run.smithi151.stdout:client6799 2024-09-16T15:48:34.152 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:48:34.152 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:48:34.188 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-16T15:48:34.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:34.388+0000 7f2d14a00640 1 -- 172.21.15.151:0/3394859028 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d1015cff0 msgr2=0x7f2d1006e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:34.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:34.388+0000 7f2d14a00640 1 --2- 172.21.15.151:0/3394859028 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d1015cff0 0x7f2d1006e6f0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f2d100b2680 tx=0x7f2cfc02a560 comp rx=0 tx=0).stop 2024-09-16T15:48:34.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:34.388+0000 7f2d14a00640 1 -- 172.21.15.151:0/3394859028 shutdown_connections 2024-09-16T15:48:34.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:34.388+0000 7f2d14a00640 1 --2- 172.21.15.151:0/3394859028 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d1006f220 0x7f2d1006f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:34.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:34.388+0000 7f2d14a00640 1 --2- 172.21.15.151:0/3394859028 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d1015cff0 0x7f2d1006e6f0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:34.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:34.388+0000 7f2d14a00640 1 --2- 172.21.15.151:0/3394859028 >> v2:172.21.15.167:3300/0 conn(0x7f2d10159aa0 0x7f2d100741a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:34.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:34.388+0000 7f2d14a00640 1 -- 172.21.15.151:0/3394859028 >> 172.21.15.151:0/3394859028 conn(0x7f2d100b18e0 msgr2=0x7f2d100b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:34.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:34.389+0000 7f2d14a00640 1 -- 172.21.15.151:0/3394859028 shutdown_connections 2024-09-16T15:48:34.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:34.389+0000 7f2d14a00640 1 -- 172.21.15.151:0/3394859028 wait complete. 2024-09-16T15:48:34.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:34.390+0000 7f2d14a00640 1 Processor -- start 2024-09-16T15:48:34.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:34.390+0000 7f2d14a00640 1 -- start start 2024-09-16T15:48:34.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:34.391+0000 7f2d14a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d1006f220 0x7f2d1014c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:34.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:34.391+0000 7f2d14a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d10159aa0 0x7f2d100af850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:34.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:34.391+0000 7f2d14a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d1015cff0 0x7f2d100afd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:34.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:34.391+0000 7f2d14a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2d100cc310 con 0x7f2d10159aa0 2024-09-16T15:48:34.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:34.391+0000 7f2d14a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2d100cc190 con 0x7f2d1006f220 2024-09-16T15:48:34.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:34.391+0000 7f2d14a00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f2d100cc490 con 0x7f2d1015cff0 2024-09-16T15:48:34.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:34.392+0000 7f2d0d400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d10159aa0 0x7f2d100af850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:34.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:34.392+0000 7f2d0d400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d10159aa0 0x7f2d100af850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:51870/0 (socket says 172.21.15.151:51870) 2024-09-16T15:48:34.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:34.392+0000 7f2d0d400640 1 -- 172.21.15.151:0/3369342100 learned_addr learned my addr 172.21.15.151:0/3369342100 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:48:34.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:34.392+0000 7f2d0de00640 1 --2- 172.21.15.151:0/3369342100 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d1006f220 0x7f2d1014c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:34.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:34.392+0000 7f2d0e800640 1 --2- 172.21.15.151:0/3369342100 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d1015cff0 0x7f2d100afd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:34.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:34.392+0000 7f2d0d400640 1 -- 172.21.15.151:0/3369342100 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d1015cff0 msgr2=0x7f2d100afd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:34.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:34.392+0000 7f2d0d400640 1 --2- 172.21.15.151:0/3369342100 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d1015cff0 0x7f2d100afd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:34.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:34.392+0000 7f2d0d400640 1 -- 172.21.15.151:0/3369342100 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d1006f220 msgr2=0x7f2d1014c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:34.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:34.392+0000 7f2d0d400640 1 --2- 172.21.15.151:0/3369342100 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d1006f220 0x7f2d1014c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:34.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:34.392+0000 7f2d0d400640 1 -- 172.21.15.151:0/3369342100 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d100cabd0 con 0x7f2d10159aa0 2024-09-16T15:48:34.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:34.392+0000 7f2d0d400640 1 --2- 172.21.15.151:0/3369342100 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d10159aa0 0x7f2d100af850 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc034040 tx=0x7f2cfc003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:34.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:34.393+0000 7f2d06000640 1 -- 172.21.15.151:0/3369342100 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc03e030 con 0x7f2d10159aa0 2024-09-16T15:48:34.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:34.393+0000 7f2d06000640 1 -- 172.21.15.151:0/3369342100 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2cfc003e00 con 0x7f2d10159aa0 2024-09-16T15:48:34.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:34.393+0000 7f2d06000640 1 -- 172.21.15.151:0/3369342100 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc035040 con 0x7f2d10159aa0 2024-09-16T15:48:34.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:34.393+0000 7f2d14a00640 1 -- 172.21.15.151:0/3369342100 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d100caeb0 con 0x7f2d10159aa0 2024-09-16T15:48:34.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:34.393+0000 7f2d14a00640 1 -- 172.21.15.151:0/3369342100 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d100cb470 con 0x7f2d10159aa0 2024-09-16T15:48:34.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:34.394+0000 7f2d14a00640 1 -- 172.21.15.151:0/3369342100 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d100741a0 con 0x7f2d10159aa0 2024-09-16T15:48:34.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:34.395+0000 7f2d06000640 1 -- 172.21.15.151:0/3369342100 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 30) ==== 102868+0+0 (secure 0 0 0) 0x7f2cfc041030 con 0x7f2d10159aa0 2024-09-16T15:48:34.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:34.396+0000 7f2d06000640 1 --2- 172.21.15.151:0/3369342100 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2cf00799c0 0x7f2cf007be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:34.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:34.396+0000 7f2d06000640 1 -- 172.21.15.151:0/3369342100 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7f2cfc0b9b30 con 0x7f2d10159aa0 2024-09-16T15:48:34.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:34.396+0000 7f2d0de00640 1 --2- 172.21.15.151:0/3369342100 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2cf00799c0 0x7f2cf007be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:34.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:34.397+0000 7f2d0de00640 1 --2- 172.21.15.151:0/3369342100 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2cf00799c0 0x7f2cf007be80 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f2d0000bbe0 tx=0x7f2d0000b390 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:34.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:34.400+0000 7f2d06000640 1 -- 172.21.15.151:0/3369342100 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2cfc0835a0 con 0x7f2d10159aa0 2024-09-16T15:48:34.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:34.620+0000 7f2d14a00640 1 -- 172.21.15.151:0/3369342100 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f2d10070980 con 0x7f2d10159aa0 2024-09-16T15:48:35.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.038+0000 7f2d06000640 1 -- 172.21.15.151:0/3369342100 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v57) ==== 136+0+0 (secure 0 0 0) 0x7f2cfc087fd0 con 0x7f2d10159aa0 2024-09-16T15:48:35.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.041+0000 7f2d14a00640 1 -- 172.21.15.151:0/3369342100 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2cf00799c0 msgr2=0x7f2cf007be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:35.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.041+0000 7f2d14a00640 1 --2- 172.21.15.151:0/3369342100 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2cf00799c0 0x7f2cf007be80 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f2d0000bbe0 tx=0x7f2d0000b390 comp rx=0 tx=0).stop 2024-09-16T15:48:35.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.041+0000 7f2d14a00640 1 -- 172.21.15.151:0/3369342100 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d10159aa0 msgr2=0x7f2d100af850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:35.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.041+0000 7f2d14a00640 1 --2- 172.21.15.151:0/3369342100 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d10159aa0 0x7f2d100af850 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc034040 tx=0x7f2cfc003910 comp rx=0 tx=0).stop 2024-09-16T15:48:35.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.041+0000 7f2d14a00640 1 -- 172.21.15.151:0/3369342100 shutdown_connections 2024-09-16T15:48:35.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.041+0000 7f2d14a00640 1 --2- 172.21.15.151:0/3369342100 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2cf00799c0 0x7f2cf007be80 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:35.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.041+0000 7f2d14a00640 1 --2- 172.21.15.151:0/3369342100 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d1015cff0 0x7f2d100afd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:35.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.041+0000 7f2d14a00640 1 --2- 172.21.15.151:0/3369342100 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d10159aa0 0x7f2d100af850 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:35.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.041+0000 7f2d14a00640 1 --2- 172.21.15.151:0/3369342100 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d1006f220 0x7f2d1014c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:35.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.041+0000 7f2d14a00640 1 -- 172.21.15.151:0/3369342100 >> 172.21.15.151:0/3369342100 conn(0x7f2d100b18e0 msgr2=0x7f2d1015a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:35.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.041+0000 7f2d14a00640 1 -- 172.21.15.151:0/3369342100 shutdown_connections 2024-09-16T15:48:35.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.041+0000 7f2d14a00640 1 -- 172.21.15.151:0/3369342100 wait complete. 2024-09-16T15:48:35.059 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-16T15:48:35.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.219+0000 7f47c5000640 1 -- 172.21.15.151:0/3498384595 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f47c00ccf00 msgr2=0x7f47c00b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:35.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.219+0000 7f47c5000640 1 --2- 172.21.15.151:0/3498384595 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f47c00ccf00 0x7f47c00b2440 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f47ac005fd0 tx=0x7f47ac02a540 comp rx=0 tx=0).stop 2024-09-16T15:48:35.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.219+0000 7f47c5000640 1 -- 172.21.15.151:0/3498384595 shutdown_connections 2024-09-16T15:48:35.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.219+0000 7f47c5000640 1 --2- 172.21.15.151:0/3498384595 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f47c0073fb0 0x7f47c0074440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:35.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.219+0000 7f47c5000640 1 --2- 172.21.15.151:0/3498384595 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f47c00ccf00 0x7f47c00b2440 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:35.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.219+0000 7f47c5000640 1 -- 172.21.15.151:0/3498384595 >> 172.21.15.151:0/3498384595 conn(0x7f47c00b1950 msgr2=0x7f47c00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:35.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.219+0000 7f47c5000640 1 -- 172.21.15.151:0/3498384595 shutdown_connections 2024-09-16T15:48:35.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.219+0000 7f47c5000640 1 -- 172.21.15.151:0/3498384595 wait complete. 2024-09-16T15:48:35.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.221+0000 7f47c5000640 1 Processor -- start 2024-09-16T15:48:35.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.221+0000 7f47c5000640 1 -- start start 2024-09-16T15:48:35.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.222+0000 7f47c5000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47c0073fb0 0x7f47c0156090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:35.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.222+0000 7f47c5000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f47c00ccf00 0x7f47c015ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:35.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.222+0000 7f47c5000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f47c015f5a0 0x7f47c015b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:35.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.222+0000 7f47c5000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f47c006f2e0 con 0x7f47c015f5a0 2024-09-16T15:48:35.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.222+0000 7f47c5000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f47c006f160 con 0x7f47c0073fb0 2024-09-16T15:48:35.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.222+0000 7f47c5000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f47c006f460 con 0x7f47c00ccf00 2024-09-16T15:48:35.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.222+0000 7f47be800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f47c015f5a0 0x7f47c015b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:35.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.222+0000 7f47be800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f47c015f5a0 0x7f47c015b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:51886/0 (socket says 172.21.15.151:51886) 2024-09-16T15:48:35.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.222+0000 7f47be800640 1 -- 172.21.15.151:0/3769815817 learned_addr learned my addr 172.21.15.151:0/3769815817 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:48:35.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.222+0000 7f47bd400640 1 --2- 172.21.15.151:0/3769815817 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f47c00ccf00 0x7f47c015ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:35.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.222+0000 7f47bde00640 1 --2- 172.21.15.151:0/3769815817 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47c0073fb0 0x7f47c0156090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:35.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.222+0000 7f47be800640 1 -- 172.21.15.151:0/3769815817 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f47c00ccf00 msgr2=0x7f47c015ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:35.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.222+0000 7f47be800640 1 --2- 172.21.15.151:0/3769815817 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f47c00ccf00 0x7f47c015ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:35.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.222+0000 7f47be800640 1 -- 172.21.15.151:0/3769815817 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47c0073fb0 msgr2=0x7f47c0156090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:35.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.222+0000 7f47be800640 1 --2- 172.21.15.151:0/3769815817 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47c0073fb0 0x7f47c0156090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:35.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.222+0000 7f47be800640 1 -- 172.21.15.151:0/3769815817 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47c0157a30 con 0x7f47c015f5a0 2024-09-16T15:48:35.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.223+0000 7f47be800640 1 --2- 172.21.15.151:0/3769815817 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f47c015f5a0 0x7f47c015b3a0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f47b800eaa0 tx=0x7f47b800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:35.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.223+0000 7f47b6000640 1 -- 172.21.15.151:0/3769815817 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b800cdb0 con 0x7f47c015f5a0 2024-09-16T15:48:35.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.223+0000 7f47b6000640 1 -- 172.21.15.151:0/3769815817 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f47b8004c10 con 0x7f47c015f5a0 2024-09-16T15:48:35.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.223+0000 7f47c5000640 1 -- 172.21.15.151:0/3769815817 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47c0156990 con 0x7f47c015f5a0 2024-09-16T15:48:35.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.223+0000 7f47b6000640 1 -- 172.21.15.151:0/3769815817 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b8005020 con 0x7f47c015f5a0 2024-09-16T15:48:35.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.223+0000 7f47c5000640 1 -- 172.21.15.151:0/3769815817 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47c0156f50 con 0x7f47c015f5a0 2024-09-16T15:48:35.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.224+0000 7f47c5000640 1 -- 172.21.15.151:0/3769815817 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47c006f980 con 0x7f47c015f5a0 2024-09-16T15:48:35.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.225+0000 7f47b6000640 1 -- 172.21.15.151:0/3769815817 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 30) ==== 102868+0+0 (secure 0 0 0) 0x7f47b80051c0 con 0x7f47c015f5a0 2024-09-16T15:48:35.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.226+0000 7f47b6000640 1 --2- 172.21.15.151:0/3769815817 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f47a00799c0 0x7f47a007be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:35.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.226+0000 7f47b6000640 1 -- 172.21.15.151:0/3769815817 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7f47b809a110 con 0x7f47c015f5a0 2024-09-16T15:48:35.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.227+0000 7f47bde00640 1 --2- 172.21.15.151:0/3769815817 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f47a00799c0 0x7f47a007be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:35.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.227+0000 7f47bde00640 1 --2- 172.21.15.151:0/3769815817 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f47a00799c0 0x7f47a007be80 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f47b0001720 tx=0x7f47b0009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:35.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.230+0000 7f47b6000640 1 -- 172.21.15.151:0/3769815817 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f47b8016030 con 0x7f47c015f5a0 2024-09-16T15:48:35.438 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.437+0000 7f47c5000640 1 -- 172.21.15.151:0/3769815817 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f47c00cc9b0 con 0x7f47a00799c0 2024-09-16T15:48:35.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.439+0000 7f47b6000640 1 -- 172.21.15.151:0/3769815817 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f47c00cc9b0 con 0x7f47a00799c0 2024-09-16T15:48:35.440 INFO:teuthology.orchestra.run.smithi151.stdout:[ 2024-09-16T15:48:35.441 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:48:35.441 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "cephfs" 2024-09-16T15:48:35.441 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:48:35.441 INFO:teuthology.orchestra.run.smithi151.stdout:] 2024-09-16T15:48:35.443 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.441+0000 7f47c5000640 1 -- 172.21.15.151:0/3769815817 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f47a00799c0 msgr2=0x7f47a007be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:35.443 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.441+0000 7f47c5000640 1 --2- 172.21.15.151:0/3769815817 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f47a00799c0 0x7f47a007be80 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f47b0001720 tx=0x7f47b0009040 comp rx=0 tx=0).stop 2024-09-16T15:48:35.443 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.442+0000 7f47c5000640 1 -- 172.21.15.151:0/3769815817 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f47c015f5a0 msgr2=0x7f47c015b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:35.443 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.442+0000 7f47c5000640 1 --2- 172.21.15.151:0/3769815817 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f47c015f5a0 0x7f47c015b3a0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f47b800eaa0 tx=0x7f47b800ef60 comp rx=0 tx=0).stop 2024-09-16T15:48:35.443 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.442+0000 7f47c5000640 1 -- 172.21.15.151:0/3769815817 shutdown_connections 2024-09-16T15:48:35.443 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.442+0000 7f47c5000640 1 --2- 172.21.15.151:0/3769815817 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f47a00799c0 0x7f47a007be80 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:35.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.442+0000 7f47c5000640 1 --2- 172.21.15.151:0/3769815817 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f47c015f5a0 0x7f47c015b3a0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:35.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.442+0000 7f47c5000640 1 --2- 172.21.15.151:0/3769815817 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f47c00ccf00 0x7f47c015ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:35.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.442+0000 7f47c5000640 1 --2- 172.21.15.151:0/3769815817 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47c0073fb0 0x7f47c0156090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:35.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.442+0000 7f47c5000640 1 -- 172.21.15.151:0/3769815817 >> 172.21.15.151:0/3769815817 conn(0x7f47c00b1950 msgr2=0x7f47c00ae5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:35.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.442+0000 7f47c5000640 1 -- 172.21.15.151:0/3769815817 shutdown_connections 2024-09-16T15:48:35.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.442+0000 7f47c5000640 1 -- 172.21.15.151:0/3769815817 wait complete. 2024-09-16T15:48:35.458 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-16T15:48:35.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.619+0000 7fb941e00640 1 -- 172.21.15.151:0/3551580707 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb93c0cccd0 msgr2=0x7fb93c0cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:35.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.619+0000 7fb941e00640 1 --2- 172.21.15.151:0/3551580707 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb93c0cccd0 0x7fb93c0cdbb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb934009830 tx=0x7fb93402a480 comp rx=0 tx=0).stop 2024-09-16T15:48:35.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.619+0000 7fb941e00640 1 -- 172.21.15.151:0/3551580707 shutdown_connections 2024-09-16T15:48:35.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.619+0000 7fb941e00640 1 --2- 172.21.15.151:0/3551580707 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb93c0cccd0 0x7fb93c0cdbb0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:35.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.619+0000 7fb941e00640 1 --2- 172.21.15.151:0/3551580707 >> v2:172.21.15.167:3300/0 conn(0x7fb93c145750 0x7fb93c073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:35.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.619+0000 7fb941e00640 1 -- 172.21.15.151:0/3551580707 >> 172.21.15.151:0/3551580707 conn(0x7fb93c0b1840 msgr2=0x7fb93c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:35.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.619+0000 7fb941e00640 1 -- 172.21.15.151:0/3551580707 shutdown_connections 2024-09-16T15:48:35.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.619+0000 7fb941e00640 1 -- 172.21.15.151:0/3551580707 wait complete. 2024-09-16T15:48:35.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.620+0000 7fb941e00640 1 Processor -- start 2024-09-16T15:48:35.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.620+0000 7fb941e00640 1 -- start start 2024-09-16T15:48:35.624 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.621+0000 7fb941e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb93c06efa0 0x7fb93c136f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:35.624 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.621+0000 7fb941e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb93c0cccd0 0x7fb93c0b0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:35.624 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.621+0000 7fb941e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb93c145750 0x7fb93c0b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:35.624 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.621+0000 7fb941e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb93c0ce480 con 0x7fb93c145750 2024-09-16T15:48:35.624 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.621+0000 7fb941e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb93c0ce300 con 0x7fb93c06efa0 2024-09-16T15:48:35.624 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.621+0000 7fb941e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fb93c0ce600 con 0x7fb93c0cccd0 2024-09-16T15:48:35.624 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.622+0000 7fb93b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb93c145750 0x7fb93c0b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:35.624 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.622+0000 7fb93b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb93c145750 0x7fb93c0b0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:51900/0 (socket says 172.21.15.151:51900) 2024-09-16T15:48:35.624 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.622+0000 7fb93b400640 1 -- 172.21.15.151:0/2368116893 learned_addr learned my addr 172.21.15.151:0/2368116893 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:48:35.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.622+0000 7fb93a000640 1 --2- 172.21.15.151:0/2368116893 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb93c0cccd0 0x7fb93c0b0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:35.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.622+0000 7fb93aa00640 1 --2- 172.21.15.151:0/2368116893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb93c06efa0 0x7fb93c136f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:35.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.622+0000 7fb93b400640 1 -- 172.21.15.151:0/2368116893 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb93c0cccd0 msgr2=0x7fb93c0b0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:35.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.622+0000 7fb93b400640 1 --2- 172.21.15.151:0/2368116893 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb93c0cccd0 0x7fb93c0b0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:35.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.622+0000 7fb93b400640 1 -- 172.21.15.151:0/2368116893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb93c06efa0 msgr2=0x7fb93c136f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:35.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.622+0000 7fb93b400640 1 --2- 172.21.15.151:0/2368116893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb93c06efa0 0x7fb93c136f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:35.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.622+0000 7fb93b400640 1 -- 172.21.15.151:0/2368116893 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb93c1374d0 con 0x7fb93c145750 2024-09-16T15:48:35.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.622+0000 7fb93b400640 1 --2- 172.21.15.151:0/2368116893 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb93c145750 0x7fb93c0b0fd0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fb934009920 tx=0x7fb934032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:35.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.623+0000 7fb933400640 1 -- 172.21.15.151:0/2368116893 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb934040070 con 0x7fb93c145750 2024-09-16T15:48:35.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.623+0000 7fb933400640 1 -- 172.21.15.151:0/2368116893 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb934005310 con 0x7fb93c145750 2024-09-16T15:48:35.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.623+0000 7fb941e00640 1 -- 172.21.15.151:0/2368116893 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb93c1394b0 con 0x7fb93c145750 2024-09-16T15:48:35.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.623+0000 7fb933400640 1 -- 172.21.15.151:0/2368116893 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb934005c90 con 0x7fb93c145750 2024-09-16T15:48:35.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.623+0000 7fb941e00640 1 -- 172.21.15.151:0/2368116893 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb93c139a70 con 0x7fb93c145750 2024-09-16T15:48:35.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.624+0000 7fb941e00640 1 -- 172.21.15.151:0/2368116893 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb93c0cdbb0 con 0x7fb93c145750 2024-09-16T15:48:35.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.625+0000 7fb933400640 1 -- 172.21.15.151:0/2368116893 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 30) ==== 102868+0+0 (secure 0 0 0) 0x7fb934039040 con 0x7fb93c145750 2024-09-16T15:48:35.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.626+0000 7fb933400640 1 --2- 172.21.15.151:0/2368116893 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb91c0799c0 0x7fb91c07be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:35.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.626+0000 7fb933400640 1 -- 172.21.15.151:0/2368116893 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7fb9340bab90 con 0x7fb93c145750 2024-09-16T15:48:35.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.626+0000 7fb93aa00640 1 --2- 172.21.15.151:0/2368116893 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb91c0799c0 0x7fb91c07be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:35.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.627+0000 7fb93aa00640 1 --2- 172.21.15.151:0/2368116893 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb91c0799c0 0x7fb91c07be80 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fb93c0b4bd0 tx=0x7fb92c009500 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:35.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.630+0000 7fb933400640 1 -- 172.21.15.151:0/2368116893 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb934084880 con 0x7fb93c145750 2024-09-16T15:48:35.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.815+0000 7fb941e00640 1 -- 172.21.15.151:0/2368116893 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fb93c0b0330 con 0x7fb93c145750 2024-09-16T15:48:35.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.815+0000 7fb933400640 1 -- 172.21.15.151:0/2368116893 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v41)=0 v41) ==== 122+0+0 (secure 0 0 0) 0x7fb934084180 con 0x7fb93c145750 2024-09-16T15:48:35.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.818+0000 7fb941e00640 1 -- 172.21.15.151:0/2368116893 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb91c0799c0 msgr2=0x7fb91c07be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:35.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.818+0000 7fb941e00640 1 --2- 172.21.15.151:0/2368116893 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb91c0799c0 0x7fb91c07be80 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fb93c0b4bd0 tx=0x7fb92c009500 comp rx=0 tx=0).stop 2024-09-16T15:48:35.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.818+0000 7fb941e00640 1 -- 172.21.15.151:0/2368116893 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb93c145750 msgr2=0x7fb93c0b0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:35.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.818+0000 7fb941e00640 1 --2- 172.21.15.151:0/2368116893 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb93c145750 0x7fb93c0b0fd0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fb934009920 tx=0x7fb934032c10 comp rx=0 tx=0).stop 2024-09-16T15:48:35.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.818+0000 7fb941e00640 1 -- 172.21.15.151:0/2368116893 shutdown_connections 2024-09-16T15:48:35.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.818+0000 7fb941e00640 1 --2- 172.21.15.151:0/2368116893 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb91c0799c0 0x7fb91c07be80 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:35.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.818+0000 7fb941e00640 1 --2- 172.21.15.151:0/2368116893 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb93c145750 0x7fb93c0b0fd0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:35.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.818+0000 7fb941e00640 1 --2- 172.21.15.151:0/2368116893 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb93c0cccd0 0x7fb93c0b0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:35.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.818+0000 7fb941e00640 1 --2- 172.21.15.151:0/2368116893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb93c06efa0 0x7fb93c136f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:35.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.818+0000 7fb941e00640 1 -- 172.21.15.151:0/2368116893 >> 172.21.15.151:0/2368116893 conn(0x7fb93c0b1840 msgr2=0x7fb93c0a9460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:35.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.818+0000 7fb941e00640 1 -- 172.21.15.151:0/2368116893 shutdown_connections 2024-09-16T15:48:35.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.818+0000 7fb941e00640 1 -- 172.21.15.151:0/2368116893 wait complete. 2024-09-16T15:48:35.836 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mgr mgr/volumes/max_concurrent_clones 2024-09-16T15:48:35.997 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.995+0000 7f395e200640 1 -- 172.21.15.151:0/2790246042 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f393c003fd0 msgr2=0x7f393c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:35.997 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.995+0000 7f395e200640 1 --2- 172.21.15.151:0/2790246042 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f393c003fd0 0x7f393c004800 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f394400c290 tx=0x7f393c005830 comp rx=0 tx=0).stop 2024-09-16T15:48:35.997 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.995+0000 7f3957e00640 1 -- 172.21.15.151:0/2790246042 reap_dead start 2024-09-16T15:48:35.997 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.995+0000 7f395e200640 1 -- 172.21.15.151:0/2790246042 shutdown_connections 2024-09-16T15:48:35.997 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.995+0000 7f395e200640 1 -- 172.21.15.151:0/2790246042 >> 172.21.15.151:0/2790246042 conn(0x7f39580b1840 msgr2=0x7f39580b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:35.997 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.995+0000 7f395e200640 1 -- 172.21.15.151:0/2790246042 shutdown_connections 2024-09-16T15:48:35.997 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.995+0000 7f395e200640 1 -- 172.21.15.151:0/2790246042 wait complete. 2024-09-16T15:48:35.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.997+0000 7f395e200640 1 Processor -- start 2024-09-16T15:48:35.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.997+0000 7f395e200640 1 -- start start 2024-09-16T15:48:36.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.998+0000 7f395e200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f393c003fd0 0x7f39581f76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:36.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.998+0000 7f395e200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3958155de0 0x7f395815a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:36.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.998+0000 7f395e200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3958154720 0x7f395815fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:36.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.998+0000 7f395e200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f39580ce4a0 con 0x7f393c003fd0 2024-09-16T15:48:36.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.998+0000 7f395e200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3958070470 con 0x7f3958154720 2024-09-16T15:48:36.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.998+0000 7f395e200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f39580ce620 con 0x7f3958155de0 2024-09-16T15:48:36.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.998+0000 7f3957400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f393c003fd0 0x7f39581f76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:36.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.998+0000 7f3957400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f393c003fd0 0x7f39581f76f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:51912/0 (socket says 172.21.15.151:51912) 2024-09-16T15:48:36.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.998+0000 7f3957400640 1 -- 172.21.15.151:0/1125246304 learned_addr learned my addr 172.21.15.151:0/1125246304 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:48:36.001 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.999+0000 7f3956a00640 1 --2- 172.21.15.151:0/1125246304 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3958155de0 0x7f395815a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:36.001 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.999+0000 7f3957e00640 1 --2- 172.21.15.151:0/1125246304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3958154720 0x7f395815fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:36.001 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.999+0000 7f3957400640 1 -- 172.21.15.151:0/1125246304 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3958155de0 msgr2=0x7f395815a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:36.001 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.999+0000 7f3957400640 1 --2- 172.21.15.151:0/1125246304 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3958155de0 0x7f395815a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:36.001 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.999+0000 7f3957400640 1 -- 172.21.15.151:0/1125246304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3958154720 msgr2=0x7f395815fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:36.001 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.999+0000 7f3957400640 1 --2- 172.21.15.151:0/1125246304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3958154720 0x7f395815fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:36.001 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.999+0000 7f3957400640 1 -- 172.21.15.151:0/1125246304 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39581f7c30 con 0x7f393c003fd0 2024-09-16T15:48:36.001 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.999+0000 7f3957400640 1 --2- 172.21.15.151:0/1125246304 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f393c003fd0 0x7f39581f76f0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f3948011a50 tx=0x7f3948011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:36.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.999+0000 7f394f400640 1 -- 172.21.15.151:0/1125246304 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394801f070 con 0x7f393c003fd0 2024-09-16T15:48:36.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.999+0000 7f394f400640 1 -- 172.21.15.151:0/1125246304 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f394800fe80 con 0x7f393c003fd0 2024-09-16T15:48:36.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.999+0000 7f395e200640 1 -- 172.21.15.151:0/1125246304 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39581562a0 con 0x7f393c003fd0 2024-09-16T15:48:36.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:35.999+0000 7f394f400640 1 -- 172.21.15.151:0/1125246304 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3948012690 con 0x7f393c003fd0 2024-09-16T15:48:36.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.000+0000 7f395e200640 1 -- 172.21.15.151:0/1125246304 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3958158d70 con 0x7f393c003fd0 2024-09-16T15:48:36.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.001+0000 7f395e200640 1 -- 172.21.15.151:0/1125246304 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39580cf090 con 0x7f393c003fd0 2024-09-16T15:48:36.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.002+0000 7f394f400640 1 -- 172.21.15.151:0/1125246304 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 30) ==== 102868+0+0 (secure 0 0 0) 0x7f3948022070 con 0x7f393c003fd0 2024-09-16T15:48:36.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.003+0000 7f394f400640 1 --2- 172.21.15.151:0/1125246304 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f39380799c0 0x7f393807be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:36.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.003+0000 7f394f400640 1 -- 172.21.15.151:0/1125246304 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7f394809a260 con 0x7f393c003fd0 2024-09-16T15:48:36.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.003+0000 7f3956a00640 1 --2- 172.21.15.151:0/1125246304 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f39380799c0 0x7f393807be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:36.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.004+0000 7f3956a00640 1 --2- 172.21.15.151:0/1125246304 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f39380799c0 0x7f393807be80 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f3944033000 tx=0x7f39580750b0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:36.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.007+0000 7f394f400640 1 -- 172.21.15.151:0/1125246304 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f3948063980 con 0x7f393c003fd0 2024-09-16T15:48:36.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.196+0000 7f395e200640 1 -- 172.21.15.151:0/1125246304 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"} v 0) -- 0x7f3958160e80 con 0x7f393c003fd0 2024-09-16T15:48:36.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.196+0000 7f394f400640 1 -- 172.21.15.151:0/1125246304 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"}]=0 v41) ==== 116+0+2 (secure 0 0 0) 0x7f39480683b0 con 0x7f393c003fd0 2024-09-16T15:48:36.197 INFO:teuthology.orchestra.run.smithi151.stdout:4 2024-09-16T15:48:36.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.199+0000 7f395e200640 1 -- 172.21.15.151:0/1125246304 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f39380799c0 msgr2=0x7f393807be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:36.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.199+0000 7f395e200640 1 --2- 172.21.15.151:0/1125246304 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f39380799c0 0x7f393807be80 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f3944033000 tx=0x7f39580750b0 comp rx=0 tx=0).stop 2024-09-16T15:48:36.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.199+0000 7f395e200640 1 -- 172.21.15.151:0/1125246304 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f393c003fd0 msgr2=0x7f39581f76f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:36.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.199+0000 7f395e200640 1 --2- 172.21.15.151:0/1125246304 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f393c003fd0 0x7f39581f76f0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f3948011a50 tx=0x7f3948011f10 comp rx=0 tx=0).stop 2024-09-16T15:48:36.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.199+0000 7f395e200640 1 -- 172.21.15.151:0/1125246304 shutdown_connections 2024-09-16T15:48:36.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.199+0000 7f395e200640 1 --2- 172.21.15.151:0/1125246304 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f39380799c0 0x7f393807be80 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:36.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.199+0000 7f395e200640 1 --2- 172.21.15.151:0/1125246304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3958154720 0x7f395815fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:36.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.199+0000 7f395e200640 1 --2- 172.21.15.151:0/1125246304 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3958155de0 0x7f395815a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:36.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.199+0000 7f395e200640 1 --2- 172.21.15.151:0/1125246304 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f393c003fd0 0x7f39581f76f0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:36.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.199+0000 7f395e200640 1 -- 172.21.15.151:0/1125246304 >> 172.21.15.151:0/1125246304 conn(0x7f39580b1840 msgr2=0x7f3958073eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:36.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.199+0000 7f395e200640 1 -- 172.21.15.151:0/1125246304 shutdown_connections 2024-09-16T15:48:36.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.199+0000 7f395e200640 1 -- 172.21.15.151:0/1125246304 wait complete. 2024-09-16T15:48:36.216 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-16T15:48:36.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.374+0000 7efef4e00640 1 -- 172.21.15.151:0/2860179779 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efef006efa0 msgr2=0x7efef0073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:36.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.374+0000 7efef4e00640 1 --2- 172.21.15.151:0/2860179779 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efef006efa0 0x7efef0073440 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7efee8005fd0 tx=0x7efee802a8b0 comp rx=0 tx=0).stop 2024-09-16T15:48:36.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.374+0000 7efef4e00640 1 -- 172.21.15.151:0/2860179779 shutdown_connections 2024-09-16T15:48:36.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.374+0000 7efef4e00640 1 --2- 172.21.15.151:0/2860179779 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efef006efa0 0x7efef0073440 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:36.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.374+0000 7efef4e00640 1 -- 172.21.15.151:0/2860179779 >> 172.21.15.151:0/2860179779 conn(0x7efef00b1840 msgr2=0x7efef00b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:36.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.374+0000 7efef4e00640 1 -- 172.21.15.151:0/2860179779 shutdown_connections 2024-09-16T15:48:36.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.374+0000 7efef4e00640 1 -- 172.21.15.151:0/2860179779 wait complete. 2024-09-16T15:48:36.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.376+0000 7efef4e00640 1 Processor -- start 2024-09-16T15:48:36.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.376+0000 7efef4e00640 1 -- start start 2024-09-16T15:48:36.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.377+0000 7efef4e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efef006efa0 0x7efef01f8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:36.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.377+0000 7efef4e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efef00cccd0 0x7efef00b0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:36.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.377+0000 7efef4e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efef0145750 0x7efef00b0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:36.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.377+0000 7efef4e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efef00ce6a0 con 0x7efef0145750 2024-09-16T15:48:36.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.377+0000 7efef4e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efef00ce520 con 0x7efef00cccd0 2024-09-16T15:48:36.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.377+0000 7efef4e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7efef00ce820 con 0x7efef006efa0 2024-09-16T15:48:36.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.378+0000 7efeee800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efef0145750 0x7efef00b0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:36.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.378+0000 7efeee800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efef0145750 0x7efef00b0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:51928/0 (socket says 172.21.15.151:51928) 2024-09-16T15:48:36.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.378+0000 7efeee800640 1 -- 172.21.15.151:0/242524138 learned_addr learned my addr 172.21.15.151:0/242524138 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:48:36.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.378+0000 7efeede00640 1 --2- 172.21.15.151:0/242524138 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efef006efa0 0x7efef01f8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:36.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.378+0000 7efeed400640 1 --2- 172.21.15.151:0/242524138 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efef00cccd0 0x7efef00b0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:36.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.378+0000 7efeee800640 1 -- 172.21.15.151:0/242524138 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efef006efa0 msgr2=0x7efef01f8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:36.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.378+0000 7efeee800640 1 --2- 172.21.15.151:0/242524138 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efef006efa0 0x7efef01f8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:36.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.378+0000 7efeee800640 1 -- 172.21.15.151:0/242524138 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efef00cccd0 msgr2=0x7efef00b0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:36.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.378+0000 7efeee800640 1 --2- 172.21.15.151:0/242524138 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efef00cccd0 0x7efef00b0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:36.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.378+0000 7efeee800640 1 -- 172.21.15.151:0/242524138 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efef01f4820 con 0x7efef0145750 2024-09-16T15:48:36.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.378+0000 7efeee800640 1 --2- 172.21.15.151:0/242524138 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efef0145750 0x7efef00b0e40 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7efee8006010 tx=0x7efee8003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:36.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.379+0000 7efee6000640 1 -- 172.21.15.151:0/242524138 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee8040070 con 0x7efef0145750 2024-09-16T15:48:36.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.379+0000 7efee6000640 1 -- 172.21.15.151:0/242524138 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efee8004330 con 0x7efef0145750 2024-09-16T15:48:36.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.379+0000 7efee6000640 1 -- 172.21.15.151:0/242524138 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee8031640 con 0x7efef0145750 2024-09-16T15:48:36.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.379+0000 7efef4e00640 1 -- 172.21.15.151:0/242524138 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efef01f4ae0 con 0x7efef0145750 2024-09-16T15:48:36.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.379+0000 7efef4e00640 1 -- 172.21.15.151:0/242524138 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efef01f50a0 con 0x7efef0145750 2024-09-16T15:48:36.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.380+0000 7efef4e00640 1 -- 172.21.15.151:0/242524138 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efef00cecd0 con 0x7efef0145750 2024-09-16T15:48:36.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.381+0000 7efee6000640 1 -- 172.21.15.151:0/242524138 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 30) ==== 102868+0+0 (secure 0 0 0) 0x7efee8039040 con 0x7efef0145750 2024-09-16T15:48:36.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.382+0000 7efee6000640 1 --2- 172.21.15.151:0/242524138 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efed00799c0 0x7efed007be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:36.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.382+0000 7efee6000640 1 -- 172.21.15.151:0/242524138 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7efee80bae60 con 0x7efef0145750 2024-09-16T15:48:36.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.382+0000 7efeede00640 1 --2- 172.21.15.151:0/242524138 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efed00799c0 0x7efed007be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:36.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.383+0000 7efeede00640 1 --2- 172.21.15.151:0/242524138 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efed00799c0 0x7efed007be80 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7efee00046b0 tx=0x7efee0007e90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:36.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.386+0000 7efee6000640 1 -- 172.21.15.151:0/242524138 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7efee8084580 con 0x7efef0145750 2024-09-16T15:48:36.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.569+0000 7efef4e00640 1 -- 172.21.15.151:0/242524138 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7efef00b03e0 con 0x7efef0145750 2024-09-16T15:48:36.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.573+0000 7efee6000640 1 -- 172.21.15.151:0/242524138 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v42)=0 v42) ==== 131+0+0 (secure 0 0 0) 0x7efee8088fb0 con 0x7efef0145750 2024-09-16T15:48:36.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.576+0000 7efef4e00640 1 -- 172.21.15.151:0/242524138 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efed00799c0 msgr2=0x7efed007be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:36.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.576+0000 7efef4e00640 1 --2- 172.21.15.151:0/242524138 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efed00799c0 0x7efed007be80 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7efee00046b0 tx=0x7efee0007e90 comp rx=0 tx=0).stop 2024-09-16T15:48:36.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.576+0000 7efef4e00640 1 -- 172.21.15.151:0/242524138 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efef0145750 msgr2=0x7efef00b0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:36.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.576+0000 7efef4e00640 1 --2- 172.21.15.151:0/242524138 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efef0145750 0x7efef00b0e40 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7efee8006010 tx=0x7efee8003ba0 comp rx=0 tx=0).stop 2024-09-16T15:48:36.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.576+0000 7efef4e00640 1 -- 172.21.15.151:0/242524138 shutdown_connections 2024-09-16T15:48:36.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.576+0000 7efef4e00640 1 --2- 172.21.15.151:0/242524138 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efed00799c0 0x7efed007be80 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:36.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.576+0000 7efef4e00640 1 --2- 172.21.15.151:0/242524138 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efef00cccd0 0x7efef00b0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:36.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.576+0000 7efef4e00640 1 --2- 172.21.15.151:0/242524138 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efef0145750 0x7efef00b0e40 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:36.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.576+0000 7efef4e00640 1 --2- 172.21.15.151:0/242524138 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efef006efa0 0x7efef01f8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:36.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.576+0000 7efef4e00640 1 -- 172.21.15.151:0/242524138 >> 172.21.15.151:0/242524138 conn(0x7efef00b1840 msgr2=0x7efef0141b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:36.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.576+0000 7efef4e00640 1 -- 172.21.15.151:0/242524138 shutdown_connections 2024-09-16T15:48:36.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.576+0000 7efef4e00640 1 -- 172.21.15.151:0/242524138 wait complete. 2024-09-16T15:48:36.593 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/snapshot_clone_no_wait false 2024-09-16T15:48:36.594 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:48:36.593+0000 7f48f1c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:48:36.594 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:48:36.593+0000 7f48f1c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:48:36.594 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:48:36.593+0000 7f48f1c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:48:36.594 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:48:36.593+0000 7f48f1c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:48:36.595 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:48:36.593+0000 7f48f1c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:48:36.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.753+0000 7ffa52000640 1 -- 172.21.15.151:0/3755223936 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa30004d40 msgr2=0x7ffa300051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:36.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.753+0000 7ffa52000640 1 --2- 172.21.15.151:0/3755223936 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa30004d40 0x7ffa300051c0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7ffa44009ed0 tx=0x7ffa30005830 comp rx=0 tx=0).stop 2024-09-16T15:48:36.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.753+0000 7ffa4be00640 1 -- 172.21.15.151:0/3755223936 reap_dead start 2024-09-16T15:48:36.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.753+0000 7ffa52000640 1 -- 172.21.15.151:0/3755223936 shutdown_connections 2024-09-16T15:48:36.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.753+0000 7ffa52000640 1 -- 172.21.15.151:0/3755223936 >> 172.21.15.151:0/3755223936 conn(0x7ffa4c0b1840 msgr2=0x7ffa4c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:36.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.753+0000 7ffa52000640 1 -- 172.21.15.151:0/3755223936 shutdown_connections 2024-09-16T15:48:36.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.753+0000 7ffa52000640 1 -- 172.21.15.151:0/3755223936 wait complete. 2024-09-16T15:48:36.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.755+0000 7ffa52000640 1 Processor -- start 2024-09-16T15:48:36.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.755+0000 7ffa52000640 1 -- start start 2024-09-16T15:48:36.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.756+0000 7ffa52000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa30004d40 0x7ffa4c0b3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:36.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.756+0000 7ffa52000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffa4c13c050 0x7ffa4c15e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:36.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.756+0000 7ffa52000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa4c13ca20 0x7ffa4c13d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:36.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.756+0000 7ffa52000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4c0737d0 con 0x7ffa30004d40 2024-09-16T15:48:36.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.756+0000 7ffa52000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4c073650 con 0x7ffa4c13ca20 2024-09-16T15:48:36.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.756+0000 7ffa52000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7ffa4c0ce0f0 con 0x7ffa4c13c050 2024-09-16T15:48:36.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.756+0000 7ffa4b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa30004d40 0x7ffa4c0b3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:36.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.756+0000 7ffa4b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa30004d40 0x7ffa4c0b3130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:51952/0 (socket says 172.21.15.151:51952) 2024-09-16T15:48:36.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.756+0000 7ffa4b400640 1 -- 172.21.15.151:0/829643683 learned_addr learned my addr 172.21.15.151:0/829643683 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:48:36.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.756+0000 7ffa4be00640 1 --2- 172.21.15.151:0/829643683 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa4c13ca20 0x7ffa4c13d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:36.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.756+0000 7ffa4aa00640 1 --2- 172.21.15.151:0/829643683 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffa4c13c050 0x7ffa4c15e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:36.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.756+0000 7ffa4b400640 1 -- 172.21.15.151:0/829643683 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffa4c13c050 msgr2=0x7ffa4c15e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:36.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.756+0000 7ffa4b400640 1 --2- 172.21.15.151:0/829643683 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffa4c13c050 0x7ffa4c15e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:36.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.756+0000 7ffa4b400640 1 -- 172.21.15.151:0/829643683 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa4c13ca20 msgr2=0x7ffa4c13d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:36.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.756+0000 7ffa4b400640 1 --2- 172.21.15.151:0/829643683 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa4c13ca20 0x7ffa4c13d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:36.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.756+0000 7ffa4b400640 1 -- 172.21.15.151:0/829643683 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa4c13fac0 con 0x7ffa30004d40 2024-09-16T15:48:36.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.757+0000 7ffa4b400640 1 --2- 172.21.15.151:0/829643683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa30004d40 0x7ffa4c0b3130 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c011ab0 tx=0x7ffa3c011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:36.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.757+0000 7ffa43400640 1 -- 172.21.15.151:0/829643683 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c01e070 con 0x7ffa30004d40 2024-09-16T15:48:36.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.757+0000 7ffa43400640 1 -- 172.21.15.151:0/829643683 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffa3c004460 con 0x7ffa30004d40 2024-09-16T15:48:36.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.757+0000 7ffa43400640 1 -- 172.21.15.151:0/829643683 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c00fc60 con 0x7ffa30004d40 2024-09-16T15:48:36.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.757+0000 7ffa52000640 1 -- 172.21.15.151:0/829643683 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa4c0ac610 con 0x7ffa30004d40 2024-09-16T15:48:36.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.757+0000 7ffa52000640 1 -- 172.21.15.151:0/829643683 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa4c0acbd0 con 0x7ffa30004d40 2024-09-16T15:48:36.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.758+0000 7ffa52000640 1 -- 172.21.15.151:0/829643683 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa4c0cdbb0 con 0x7ffa30004d40 2024-09-16T15:48:36.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.759+0000 7ffa43400640 1 -- 172.21.15.151:0/829643683 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 30) ==== 102868+0+0 (secure 0 0 0) 0x7ffa3c021070 con 0x7ffa30004d40 2024-09-16T15:48:36.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.760+0000 7ffa43400640 1 --2- 172.21.15.151:0/829643683 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ffa2c0799f0 0x7ffa2c07beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:36.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.760+0000 7ffa43400640 1 -- 172.21.15.151:0/829643683 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7ffa3c09a1b0 con 0x7ffa30004d40 2024-09-16T15:48:36.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.761+0000 7ffa4aa00640 1 --2- 172.21.15.151:0/829643683 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ffa2c0799f0 0x7ffa2c07beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:36.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.762+0000 7ffa4aa00640 1 --2- 172.21.15.151:0/829643683 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ffa2c0799f0 0x7ffa2c07beb0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7ffa38000fb0 tx=0x7ffa38006720 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:36.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.765+0000 7ffa43400640 1 -- 172.21.15.151:0/829643683 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ffa3c0638d0 con 0x7ffa30004d40 2024-09-16T15:48:36.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.950+0000 7ffa52000640 1 -- 172.21.15.151:0/829643683 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/snapshot_clone_no_wait}] v 0) -- 0x7ffa4c0cf580 con 0x7ffa30004d40 2024-09-16T15:48:36.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.954+0000 7ffa43400640 1 -- 172.21.15.151:0/829643683 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/snapshot_clone_no_wait}]=0 v43)=0 v43) ==== 136+0+0 (secure 0 0 0) 0x7ffa3c068300 con 0x7ffa30004d40 2024-09-16T15:48:36.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.957+0000 7ffa52000640 1 -- 172.21.15.151:0/829643683 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ffa2c0799f0 msgr2=0x7ffa2c07beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:36.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.957+0000 7ffa52000640 1 --2- 172.21.15.151:0/829643683 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ffa2c0799f0 0x7ffa2c07beb0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7ffa38000fb0 tx=0x7ffa38006720 comp rx=0 tx=0).stop 2024-09-16T15:48:36.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.957+0000 7ffa52000640 1 -- 172.21.15.151:0/829643683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa30004d40 msgr2=0x7ffa4c0b3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:36.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.957+0000 7ffa52000640 1 --2- 172.21.15.151:0/829643683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa30004d40 0x7ffa4c0b3130 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c011ab0 tx=0x7ffa3c011f70 comp rx=0 tx=0).stop 2024-09-16T15:48:36.959 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.957+0000 7ffa52000640 1 -- 172.21.15.151:0/829643683 shutdown_connections 2024-09-16T15:48:36.959 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.957+0000 7ffa52000640 1 --2- 172.21.15.151:0/829643683 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ffa2c0799f0 0x7ffa2c07beb0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:36.959 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.957+0000 7ffa52000640 1 --2- 172.21.15.151:0/829643683 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa4c13ca20 0x7ffa4c13d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:36.959 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.957+0000 7ffa52000640 1 --2- 172.21.15.151:0/829643683 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ffa4c13c050 0x7ffa4c15e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:36.959 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.957+0000 7ffa52000640 1 --2- 172.21.15.151:0/829643683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa30004d40 0x7ffa4c0b3130 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:36.959 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.957+0000 7ffa52000640 1 -- 172.21.15.151:0/829643683 >> 172.21.15.151:0/829643683 conn(0x7ffa4c0b1840 msgr2=0x7ffa4c143520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:36.959 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.957+0000 7ffa52000640 1 -- 172.21.15.151:0/829643683 shutdown_connections 2024-09-16T15:48:36.959 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:36.957+0000 7ffa52000640 1 -- 172.21.15.151:0/829643683 wait complete. 2024-09-16T15:48:36.976 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-09-16T15:48:37.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.138+0000 7f907aa00640 1 -- 172.21.15.151:0/1284703852 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f90740d2b50 msgr2=0x7f90740d0500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:37.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.138+0000 7f907aa00640 1 --2- 172.21.15.151:0/1284703852 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f90740d2b50 0x7f90740d0500 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f90740caef0 tx=0x7f906c02a8b0 comp rx=0 tx=0).stop 2024-09-16T15:48:37.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.138+0000 7f907aa00640 1 -- 172.21.15.151:0/1284703852 shutdown_connections 2024-09-16T15:48:37.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.138+0000 7f907aa00640 1 --2- 172.21.15.151:0/1284703852 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f90740d2b50 0x7f90740d0500 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:37.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.138+0000 7f907aa00640 1 --2- 172.21.15.151:0/1284703852 >> v2:172.21.15.167:3300/0 conn(0x7f90740cf470 0x7f90740d2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:37.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.138+0000 7f907aa00640 1 -- 172.21.15.151:0/1284703852 >> 172.21.15.151:0/1284703852 conn(0x7f907409d180 msgr2=0x7f90740d3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:37.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.138+0000 7f907aa00640 1 -- 172.21.15.151:0/1284703852 shutdown_connections 2024-09-16T15:48:37.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.138+0000 7f907aa00640 1 -- 172.21.15.151:0/1284703852 wait complete. 2024-09-16T15:48:37.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.140+0000 7f907aa00640 1 Processor -- start 2024-09-16T15:48:37.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.140+0000 7f907aa00640 1 -- start start 2024-09-16T15:48:37.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.141+0000 7f907aa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f90740cf470 0x7f90740a1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:37.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.141+0000 7f907aa00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90740d2b50 0x7f90741f79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:37.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.141+0000 7f907aa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f907415dfb0 0x7f90741f7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:37.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.141+0000 7f907aa00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f90740b6d10 con 0x7f90740cf470 2024-09-16T15:48:37.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.141+0000 7f907aa00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f90740d0e70 con 0x7f907415dfb0 2024-09-16T15:48:37.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.141+0000 7f907aa00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f90740b6e90 con 0x7f90740d2b50 2024-09-16T15:48:37.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.141+0000 7f9079600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f90740cf470 0x7f90740a1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:37.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.141+0000 7f9079600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f90740cf470 0x7f90740a1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:51970/0 (socket says 172.21.15.151:51970) 2024-09-16T15:48:37.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.141+0000 7f9079600640 1 -- 172.21.15.151:0/3134482308 learned_addr learned my addr 172.21.15.151:0/3134482308 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:48:37.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.141+0000 7f9078c00640 1 --2- 172.21.15.151:0/3134482308 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90740d2b50 0x7f90741f79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:37.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.141+0000 7f907a000640 1 --2- 172.21.15.151:0/3134482308 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f907415dfb0 0x7f90741f7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:37.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.141+0000 7f9079600640 1 -- 172.21.15.151:0/3134482308 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90740d2b50 msgr2=0x7f90741f79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:37.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.141+0000 7f9079600640 1 --2- 172.21.15.151:0/3134482308 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90740d2b50 0x7f90741f79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:37.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.141+0000 7f9079600640 1 -- 172.21.15.151:0/3134482308 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f907415dfb0 msgr2=0x7f90741f7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:37.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.141+0000 7f9079600640 1 --2- 172.21.15.151:0/3134482308 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f907415dfb0 0x7f90741f7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:37.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.141+0000 7f9079600640 1 -- 172.21.15.151:0/3134482308 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90741f5610 con 0x7f90740cf470 2024-09-16T15:48:37.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.142+0000 7f9079600640 1 --2- 172.21.15.151:0/3134482308 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f90740cf470 0x7f90740a1790 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f906800c3c0 tx=0x7f906800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:37.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.142+0000 7f9071600640 1 -- 172.21.15.151:0/3134482308 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9068017070 con 0x7f90740cf470 2024-09-16T15:48:37.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.142+0000 7f9071600640 1 -- 172.21.15.151:0/3134482308 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f90680035f0 con 0x7f90740cf470 2024-09-16T15:48:37.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.142+0000 7f907aa00640 1 -- 172.21.15.151:0/3134482308 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90741f58f0 con 0x7f90740cf470 2024-09-16T15:48:37.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.142+0000 7f9071600640 1 -- 172.21.15.151:0/3134482308 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9068003a40 con 0x7f90740cf470 2024-09-16T15:48:37.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.142+0000 7f907aa00640 1 -- 172.21.15.151:0/3134482308 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90740ccb80 con 0x7f90740cf470 2024-09-16T15:48:37.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.143+0000 7f907aa00640 1 -- 172.21.15.151:0/3134482308 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90740b73b0 con 0x7f90740cf470 2024-09-16T15:48:37.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.144+0000 7f9071600640 1 -- 172.21.15.151:0/3134482308 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 30) ==== 102868+0+0 (secure 0 0 0) 0x7f906801a070 con 0x7f90740cf470 2024-09-16T15:48:37.146 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.145+0000 7f9071600640 1 --2- 172.21.15.151:0/3134482308 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f90540799c0 0x7f905407be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:37.146 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.145+0000 7f9071600640 1 -- 172.21.15.151:0/3134482308 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7f9068098b10 con 0x7f90740cf470 2024-09-16T15:48:37.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.146+0000 7f9078c00640 1 --2- 172.21.15.151:0/3134482308 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f90540799c0 0x7f905407be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:37.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.146+0000 7f9078c00640 1 --2- 172.21.15.151:0/3134482308 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f90540799c0 0x7f905407be80 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f90740cdbb0 tx=0x7f906000b390 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:37.150 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.149+0000 7f9071600640 1 -- 172.21.15.151:0/3134482308 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f90680621b0 con 0x7f90740cf470 2024-09-16T15:48:37.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.360+0000 7f907aa00640 1 -- 172.21.15.151:0/3134482308 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7f907408df30 con 0x7f90540799c0 2024-09-16T15:48:37.373 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:48:37.371+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:48:37.373 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:48:37.371+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:48:37.373 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:48:37.371+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:48:37.373 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:48:37.371+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:48:37.373 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:48:37.371+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:48:37.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.412+0000 7f9071600640 1 -- 172.21.15.151:0/3134482308 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f907408df30 con 0x7f90540799c0 2024-09-16T15:48:37.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.414+0000 7f907aa00640 1 -- 172.21.15.151:0/3134482308 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f90540799c0 msgr2=0x7f905407be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:37.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.414+0000 7f907aa00640 1 --2- 172.21.15.151:0/3134482308 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f90540799c0 0x7f905407be80 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f90740cdbb0 tx=0x7f906000b390 comp rx=0 tx=0).stop 2024-09-16T15:48:37.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.415+0000 7f907aa00640 1 -- 172.21.15.151:0/3134482308 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f90740cf470 msgr2=0x7f90740a1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:37.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.415+0000 7f907aa00640 1 --2- 172.21.15.151:0/3134482308 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f90740cf470 0x7f90740a1790 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f906800c3c0 tx=0x7f906800c880 comp rx=0 tx=0).stop 2024-09-16T15:48:37.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.415+0000 7f907aa00640 1 -- 172.21.15.151:0/3134482308 shutdown_connections 2024-09-16T15:48:37.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.415+0000 7f907aa00640 1 --2- 172.21.15.151:0/3134482308 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f90540799c0 0x7f905407be80 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:37.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.415+0000 7f907aa00640 1 --2- 172.21.15.151:0/3134482308 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f907415dfb0 0x7f90741f7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:37.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.415+0000 7f907aa00640 1 --2- 172.21.15.151:0/3134482308 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f90740d2b50 0x7f90741f79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:37.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.415+0000 7f907aa00640 1 --2- 172.21.15.151:0/3134482308 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f90740cf470 0x7f90740a1790 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:37.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.415+0000 7f907aa00640 1 -- 172.21.15.151:0/3134482308 >> 172.21.15.151:0/3134482308 conn(0x7f907409d180 msgr2=0x7f90740d4790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:37.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.415+0000 7f907aa00640 1 -- 172.21.15.151:0/3134482308 shutdown_connections 2024-09-16T15:48:37.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.415+0000 7f907aa00640 1 -- 172.21.15.151:0/3134482308 wait complete. 2024-09-16T15:48:37.432 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-16T15:48:37.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.592+0000 7f5287e00640 1 -- 172.21.15.151:0/2664675482 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f528815f640 msgr2=0x7f5288072340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:37.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.592+0000 7f5287e00640 1 --2- 172.21.15.151:0/2664675482 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f528815f640 0x7f5288072340 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5280009be0 tx=0x7f528002a820 comp rx=0 tx=0).stop 2024-09-16T15:48:37.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.592+0000 7f5287e00640 1 -- 172.21.15.151:0/2664675482 shutdown_connections 2024-09-16T15:48:37.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.592+0000 7f5287e00640 1 --2- 172.21.15.151:0/2664675482 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f528815f640 0x7f5288072340 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:37.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.592+0000 7f5287e00640 1 -- 172.21.15.151:0/2664675482 >> 172.21.15.151:0/2664675482 conn(0x7f52880d1bf0 msgr2=0x7f52880d1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:37.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.592+0000 7f5287e00640 1 -- 172.21.15.151:0/2664675482 shutdown_connections 2024-09-16T15:48:37.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.592+0000 7f5287e00640 1 -- 172.21.15.151:0/2664675482 wait complete. 2024-09-16T15:48:37.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.594+0000 7f5287e00640 1 Processor -- start 2024-09-16T15:48:37.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.594+0000 7f5287e00640 1 -- start start 2024-09-16T15:48:37.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.595+0000 7f5287e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f52880d0400 0x7f528815e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:37.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.595+0000 7f5287e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f528815f260 0x7f52880b20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:37.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.595+0000 7f5287e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f528815f640 0x7f52880cb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:37.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.595+0000 7f5287e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f52880af230 con 0x7f52880d0400 2024-09-16T15:48:37.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.595+0000 7f5287e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f52880af0b0 con 0x7f528815f260 2024-09-16T15:48:37.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.595+0000 7f5287e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f52880af3b0 con 0x7f528815f640 2024-09-16T15:48:37.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.595+0000 7f5286a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f52880d0400 0x7f528815e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:37.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.595+0000 7f5286a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f52880d0400 0x7f528815e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:51976/0 (socket says 172.21.15.151:51976) 2024-09-16T15:48:37.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.595+0000 7f5286a00640 1 -- 172.21.15.151:0/3314299466 learned_addr learned my addr 172.21.15.151:0/3314299466 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:48:37.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.595+0000 7f5286000640 1 --2- 172.21.15.151:0/3314299466 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f528815f260 0x7f52880b20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:37.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.595+0000 7f5287400640 1 --2- 172.21.15.151:0/3314299466 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f528815f640 0x7f52880cb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:37.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.595+0000 7f5286a00640 1 -- 172.21.15.151:0/3314299466 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f528815f640 msgr2=0x7f52880cb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:37.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.595+0000 7f5286a00640 1 --2- 172.21.15.151:0/3314299466 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f528815f640 0x7f52880cb0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:37.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.595+0000 7f5286a00640 1 -- 172.21.15.151:0/3314299466 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f528815f260 msgr2=0x7f52880b20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:37.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.595+0000 7f5286a00640 1 --2- 172.21.15.151:0/3314299466 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f528815f260 0x7f52880b20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:37.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.595+0000 7f5286a00640 1 -- 172.21.15.151:0/3314299466 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52880b2a00 con 0x7f52880d0400 2024-09-16T15:48:37.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.596+0000 7f5286a00640 1 --2- 172.21.15.151:0/3314299466 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f52880d0400 0x7f528815e0c0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f527800c700 tx=0x7f527800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:37.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.596+0000 7f527ea00640 1 -- 172.21.15.151:0/3314299466 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5278016020 con 0x7f52880d0400 2024-09-16T15:48:37.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.596+0000 7f527ea00640 1 -- 172.21.15.151:0/3314299466 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f527800a930 con 0x7f52880d0400 2024-09-16T15:48:37.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.596+0000 7f5287e00640 1 -- 172.21.15.151:0/3314299466 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f528809be20 con 0x7f52880d0400 2024-09-16T15:48:37.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.596+0000 7f527ea00640 1 -- 172.21.15.151:0/3314299466 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f527800ad40 con 0x7f52880d0400 2024-09-16T15:48:37.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.596+0000 7f5287e00640 1 -- 172.21.15.151:0/3314299466 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f528809c3e0 con 0x7f52880d0400 2024-09-16T15:48:37.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.597+0000 7f5287e00640 1 -- 172.21.15.151:0/3314299466 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5288072340 con 0x7f52880d0400 2024-09-16T15:48:37.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.598+0000 7f527ea00640 1 -- 172.21.15.151:0/3314299466 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 30) ==== 102868+0+0 (secure 0 0 0) 0x7f5278019020 con 0x7f52880d0400 2024-09-16T15:48:37.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.599+0000 7f527ea00640 1 --2- 172.21.15.151:0/3314299466 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f52680799c0 0x7f526807be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:48:37.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.599+0000 7f527ea00640 1 -- 172.21.15.151:0/3314299466 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7f52780994b0 con 0x7f52880d0400 2024-09-16T15:48:37.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.600+0000 7f5286000640 1 --2- 172.21.15.151:0/3314299466 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f52680799c0 0x7f526807be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:48:37.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.600+0000 7f5286000640 1 --2- 172.21.15.151:0/3314299466 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f52680799c0 0x7f526807be80 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f52880b2d40 tx=0x7f5274009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:48:37.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.603+0000 7f527ea00640 1 -- 172.21.15.151:0/3314299466 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f5278020030 con 0x7f52880d0400 2024-09-16T15:48:37.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.818+0000 7f5287e00640 1 -- 172.21.15.151:0/3314299466 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f528815e600 con 0x7f52680799c0 2024-09-16T15:48:37.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.824+0000 7f527ea00640 1 -- 172.21.15.151:0/3314299466 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f527400f700 con 0x7f52680799c0 2024-09-16T15:48:37.825 INFO:teuthology.orchestra.run.smithi151.stdout:/volumes/_nogroup/sv1/36ce568d-e1cb-4659-a3f7-f6d7ceec4389 2024-09-16T15:48:37.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.827+0000 7f5287e00640 1 -- 172.21.15.151:0/3314299466 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f52680799c0 msgr2=0x7f526807be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:37.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.827+0000 7f5287e00640 1 --2- 172.21.15.151:0/3314299466 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f52680799c0 0x7f526807be80 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f52880b2d40 tx=0x7f5274009040 comp rx=0 tx=0).stop 2024-09-16T15:48:37.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.827+0000 7f5287e00640 1 -- 172.21.15.151:0/3314299466 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f52880d0400 msgr2=0x7f528815e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:48:37.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.827+0000 7f5287e00640 1 --2- 172.21.15.151:0/3314299466 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f52880d0400 0x7f528815e0c0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f527800c700 tx=0x7f527800cbc0 comp rx=0 tx=0).stop 2024-09-16T15:48:37.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.827+0000 7f5287e00640 1 -- 172.21.15.151:0/3314299466 shutdown_connections 2024-09-16T15:48:37.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.827+0000 7f5287e00640 1 --2- 172.21.15.151:0/3314299466 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f52680799c0 0x7f526807be80 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:37.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.827+0000 7f5287e00640 1 --2- 172.21.15.151:0/3314299466 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f528815f260 0x7f52880b20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:37.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.827+0000 7f5287e00640 1 --2- 172.21.15.151:0/3314299466 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f528815f640 0x7f52880cb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:37.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.827+0000 7f5287e00640 1 --2- 172.21.15.151:0/3314299466 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f52880d0400 0x7f528815e0c0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:48:37.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.827+0000 7f5287e00640 1 -- 172.21.15.151:0/3314299466 >> 172.21.15.151:0/3314299466 conn(0x7f52880d1bf0 msgr2=0x7f52880c77d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:48:37.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.827+0000 7f5287e00640 1 -- 172.21.15.151:0/3314299466 shutdown_connections 2024-09-16T15:48:37.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:48:37.827+0000 7f5287e00640 1 -- 172.21.15.151:0/3314299466 wait complete. 2024-09-16T15:48:37.844 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/36ce568d-e1cb-4659-a3f7-f6d7ceec4389 2024-09-16T15:48:37.844 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:48:37.844 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:48:37.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:48:37.880 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/36ce568d-e1cb-4659-a3f7-f6d7ceec4389/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T15:48:45.858 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records in 2024-09-16T15:48:45.859 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records out 2024-09-16T15:48:45.859 INFO:teuthology.orchestra.run.smithi151.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.89663 s, 136 MB/s 2024-09-16T15:48:45.860 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:48:45.860 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:48:45.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:48:45.895 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/36ce568d-e1cb-4659-a3f7-f6d7ceec4389/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T15:48:49.034 DEBUG:teuthology.orchestra.run.smithi151:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:48:49.039 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:48:53.910 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records in 2024-09-16T15:48:53.910 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records out 2024-09-16T15:48:53.910 INFO:teuthology.orchestra.run.smithi151.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.93782 s, 135 MB/s 2024-09-16T15:48:53.912 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:48:53.912 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:48:53.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:48:53.948 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/36ce568d-e1cb-4659-a3f7-f6d7ceec4389/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T15:49:02.366 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records in 2024-09-16T15:49:02.366 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records out 2024-09-16T15:49:02.366 INFO:teuthology.orchestra.run.smithi151.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.34085 s, 129 MB/s 2024-09-16T15:49:02.367 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-09-16T15:49:02.532 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.530+0000 7fae54800640 1 -- 172.21.15.151:0/1471286553 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae4c06e6f0 msgr2=0x7fae4c06eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:02.532 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.530+0000 7fae54800640 1 --2- 172.21.15.151:0/1471286553 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae4c06e6f0 0x7fae4c06eb70 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fae48005fd0 tx=0x7fae4802aa10 comp rx=0 tx=0).stop 2024-09-16T15:49:02.532 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.530+0000 7fae54800640 1 -- 172.21.15.151:0/1471286553 shutdown_connections 2024-09-16T15:49:02.532 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.530+0000 7fae54800640 1 --2- 172.21.15.151:0/1471286553 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae4c06e6f0 0x7fae4c06eb70 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:02.532 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.530+0000 7fae54800640 1 --2- 172.21.15.151:0/1471286553 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fae4c15f9a0 0x7fae4c15fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:02.532 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.530+0000 7fae54800640 1 -- 172.21.15.151:0/1471286553 >> 172.21.15.151:0/1471286553 conn(0x7fae4c0b18e0 msgr2=0x7fae4c0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:49:02.532 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.531+0000 7fae54800640 1 -- 172.21.15.151:0/1471286553 shutdown_connections 2024-09-16T15:49:02.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.531+0000 7fae54800640 1 -- 172.21.15.151:0/1471286553 wait complete. 2024-09-16T15:49:02.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.532+0000 7fae54800640 1 Processor -- start 2024-09-16T15:49:02.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.532+0000 7fae54800640 1 -- start start 2024-09-16T15:49:02.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.533+0000 7fae54800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae4c06e6f0 0x7fae4c1476d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:02.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.533+0000 7fae54800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae4c073de0 0x7fae4c15cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:02.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.533+0000 7fae54800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fae4c15f9a0 0x7fae4c15e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:02.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.533+0000 7fae54800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fae4c06f0b0 con 0x7fae4c073de0 2024-09-16T15:49:02.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.533+0000 7fae54800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fae4c0cd5e0 con 0x7fae4c06e6f0 2024-09-16T15:49:02.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.533+0000 7fae54800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fae4c06f230 con 0x7fae4c15f9a0 2024-09-16T15:49:02.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.533+0000 7fae51400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae4c073de0 0x7fae4c15cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:02.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.534+0000 7fae52800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fae4c15f9a0 0x7fae4c15e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:02.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.534+0000 7fae52800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fae4c15f9a0 0x7fae4c15e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.151:58178/0 (socket says 172.21.15.151:58178) 2024-09-16T15:49:02.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.534+0000 7fae52800640 1 -- 172.21.15.151:0/2930738823 learned_addr learned my addr 172.21.15.151:0/2930738823 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:49:02.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.534+0000 7fae51400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae4c073de0 0x7fae4c15cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33530/0 (socket says 172.21.15.151:33530) 2024-09-16T15:49:02.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.534+0000 7fae51e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae4c06e6f0 0x7fae4c1476d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:02.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.534+0000 7fae52800640 1 -- 172.21.15.151:0/2930738823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae4c06e6f0 msgr2=0x7fae4c1476d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:02.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.534+0000 7fae52800640 1 --2- 172.21.15.151:0/2930738823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae4c06e6f0 0x7fae4c1476d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:02.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.534+0000 7fae52800640 1 -- 172.21.15.151:0/2930738823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae4c073de0 msgr2=0x7fae4c15cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:02.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.534+0000 7fae52800640 1 --2- 172.21.15.151:0/2930738823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae4c073de0 0x7fae4c15cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:02.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.534+0000 7fae52800640 1 -- 172.21.15.151:0/2930738823 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae4c147c10 con 0x7fae4c15f9a0 2024-09-16T15:49:02.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.534+0000 7fae51e00640 1 --2- 172.21.15.151:0/2930738823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae4c06e6f0 0x7fae4c1476d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:49:02.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.534+0000 7fae52800640 1 --2- 172.21.15.151:0/2930738823 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fae4c15f9a0 0x7fae4c15e800 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fae48006010 tx=0x7fae48003ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:49:02.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.535+0000 7fae46000640 1 -- 172.21.15.151:0/2930738823 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae48047070 con 0x7fae4c15f9a0 2024-09-16T15:49:02.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.535+0000 7fae46000640 1 -- 172.21.15.151:0/2930738823 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fae4802f670 con 0x7fae4c15f9a0 2024-09-16T15:49:02.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.535+0000 7fae54800640 1 -- 172.21.15.151:0/2930738823 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae4c0c9f70 con 0x7fae4c15f9a0 2024-09-16T15:49:02.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.535+0000 7fae46000640 1 -- 172.21.15.151:0/2930738823 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae48033050 con 0x7fae4c15f9a0 2024-09-16T15:49:02.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.535+0000 7fae54800640 1 -- 172.21.15.151:0/2930738823 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fae4c0ca530 con 0x7fae4c15f9a0 2024-09-16T15:49:02.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.536+0000 7fae54800640 1 -- 172.21.15.151:0/2930738823 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae4c069360 con 0x7fae4c15f9a0 2024-09-16T15:49:02.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.537+0000 7fae46000640 1 -- 172.21.15.151:0/2930738823 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 31) ==== 102970+0+0 (secure 0 0 0) 0x7fae4804a070 con 0x7fae4c15f9a0 2024-09-16T15:49:02.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.538+0000 7fae46000640 1 --2- 172.21.15.151:0/2930738823 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fae30079ae0 0x7fae3007bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:02.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.538+0000 7fae46000640 1 -- 172.21.15.151:0/2930738823 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7fae480ba9f0 con 0x7fae4c15f9a0 2024-09-16T15:49:02.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.538+0000 7fae51e00640 1 --2- 172.21.15.151:0/2930738823 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fae30079ae0 0x7fae3007bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:02.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.539+0000 7fae51e00640 1 --2- 172.21.15.151:0/2930738823 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fae30079ae0 0x7fae3007bfa0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fae40009f20 tx=0x7fae40005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:49:02.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.542+0000 7fae46000640 1 -- 172.21.15.151:0/2930738823 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fae4c147c10 con 0x7fae4c15f9a0 2024-09-16T15:49:02.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.754+0000 7fae54800640 1 -- 172.21.15.151:0/2930738823 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7fae4c0cd7b0 con 0x7fae30079ae0 2024-09-16T15:49:02.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.769+0000 7fae46000640 1 -- 172.21.15.151:0/2930738823 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fae4c0cd7b0 con 0x7fae30079ae0 2024-09-16T15:49:02.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.771+0000 7fae54800640 1 -- 172.21.15.151:0/2930738823 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fae30079ae0 msgr2=0x7fae3007bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:02.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.771+0000 7fae54800640 1 --2- 172.21.15.151:0/2930738823 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fae30079ae0 0x7fae3007bfa0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fae40009f20 tx=0x7fae40005f90 comp rx=0 tx=0).stop 2024-09-16T15:49:02.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.771+0000 7fae54800640 1 -- 172.21.15.151:0/2930738823 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fae4c15f9a0 msgr2=0x7fae4c15e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:02.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.771+0000 7fae54800640 1 --2- 172.21.15.151:0/2930738823 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fae4c15f9a0 0x7fae4c15e800 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fae48006010 tx=0x7fae48003ef0 comp rx=0 tx=0).stop 2024-09-16T15:49:02.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.771+0000 7fae54800640 1 -- 172.21.15.151:0/2930738823 shutdown_connections 2024-09-16T15:49:02.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.771+0000 7fae54800640 1 --2- 172.21.15.151:0/2930738823 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fae30079ae0 0x7fae3007bfa0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:02.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.771+0000 7fae54800640 1 --2- 172.21.15.151:0/2930738823 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fae4c15f9a0 0x7fae4c15e800 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:02.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.771+0000 7fae54800640 1 --2- 172.21.15.151:0/2930738823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae4c073de0 0x7fae4c15cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:02.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.771+0000 7fae54800640 1 --2- 172.21.15.151:0/2930738823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae4c06e6f0 0x7fae4c1476d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:02.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.771+0000 7fae54800640 1 -- 172.21.15.151:0/2930738823 >> 172.21.15.151:0/2930738823 conn(0x7fae4c0b18e0 msgr2=0x7fae4c0acd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:49:02.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.772+0000 7fae54800640 1 -- 172.21.15.151:0/2930738823 shutdown_connections 2024-09-16T15:49:02.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.772+0000 7fae54800640 1 -- 172.21.15.151:0/2930738823 wait complete. 2024-09-16T15:49:02.789 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-16T15:49:02.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.948+0000 7fdaf9a00640 1 -- 172.21.15.151:0/3976594740 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdaf406efa0 msgr2=0x7fdaf40cc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:02.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.948+0000 7fdaf9a00640 1 --2- 172.21.15.151:0/3976594740 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdaf406efa0 0x7fdaf40cc5c0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fdaf40d0cd0 tx=0x7fdadc02a4a0 comp rx=0 tx=0).stop 2024-09-16T15:49:02.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.948+0000 7fdaf9a00640 1 -- 172.21.15.151:0/3976594740 shutdown_connections 2024-09-16T15:49:02.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.948+0000 7fdaf9a00640 1 --2- 172.21.15.151:0/3976594740 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdaf406efa0 0x7fdaf40cc5c0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:02.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.948+0000 7fdaf9a00640 1 --2- 172.21.15.151:0/3976594740 >> v2:172.21.15.167:3300/0 conn(0x7fdaf4145750 0x7fdaf4073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:02.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.948+0000 7fdaf9a00640 1 -- 172.21.15.151:0/3976594740 >> 172.21.15.151:0/3976594740 conn(0x7fdaf40b1840 msgr2=0x7fdaf40b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:49:02.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.948+0000 7fdaf9a00640 1 -- 172.21.15.151:0/3976594740 shutdown_connections 2024-09-16T15:49:02.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.948+0000 7fdaf9a00640 1 -- 172.21.15.151:0/3976594740 wait complete. 2024-09-16T15:49:02.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.949+0000 7fdaf9a00640 1 Processor -- start 2024-09-16T15:49:02.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.949+0000 7fdaf9a00640 1 -- start start 2024-09-16T15:49:02.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.950+0000 7fdaf9a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdaf406efa0 0x7fdaf41e66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:02.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.950+0000 7fdaf9a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdaf40ccb00 0x7fdaf40aeac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:02.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.950+0000 7fdaf9a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdaf4145750 0x7fdaf40af000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:02.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.950+0000 7fdaf9a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf40ced40 con 0x7fdaf40ccb00 2024-09-16T15:49:02.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.950+0000 7fdaf9a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf40cebc0 con 0x7fdaf4145750 2024-09-16T15:49:02.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.950+0000 7fdaf9a00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fdaf40ceec0 con 0x7fdaf406efa0 2024-09-16T15:49:02.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.951+0000 7fdaebe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdaf40ccb00 0x7fdaf40aeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:02.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.951+0000 7fdaebe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdaf40ccb00 0x7fdaf40aeac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41476/0 (socket says 172.21.15.151:41476) 2024-09-16T15:49:02.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.951+0000 7fdaebe00640 1 -- 172.21.15.151:0/3199661346 learned_addr learned my addr 172.21.15.151:0/3199661346 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:49:02.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.951+0000 7fdaf2a00640 1 --2- 172.21.15.151:0/3199661346 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdaf406efa0 0x7fdaf41e66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:02.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.951+0000 7fdaf3400640 1 --2- 172.21.15.151:0/3199661346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdaf4145750 0x7fdaf40af000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:02.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.951+0000 7fdaebe00640 1 -- 172.21.15.151:0/3199661346 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdaf406efa0 msgr2=0x7fdaf41e66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:02.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.951+0000 7fdaebe00640 1 --2- 172.21.15.151:0/3199661346 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdaf406efa0 0x7fdaf41e66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:02.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.951+0000 7fdaebe00640 1 -- 172.21.15.151:0/3199661346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdaf4145750 msgr2=0x7fdaf40af000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:02.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.951+0000 7fdaebe00640 1 --2- 172.21.15.151:0/3199661346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdaf4145750 0x7fdaf40af000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:02.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.951+0000 7fdaebe00640 1 -- 172.21.15.151:0/3199661346 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaf41e5620 con 0x7fdaf40ccb00 2024-09-16T15:49:02.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.951+0000 7fdaebe00640 1 --2- 172.21.15.151:0/3199661346 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdaf40ccb00 0x7fdaf40aeac0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fdadc02aa90 tx=0x7fdadc032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:49:02.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.952+0000 7fdaeaa00640 1 -- 172.21.15.151:0/3199661346 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdadc040070 con 0x7fdaf40ccb00 2024-09-16T15:49:02.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.952+0000 7fdaeaa00640 1 -- 172.21.15.151:0/3199661346 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdadc005310 con 0x7fdaf40ccb00 2024-09-16T15:49:02.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.952+0000 7fdaeaa00640 1 -- 172.21.15.151:0/3199661346 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdadc005c90 con 0x7fdaf40ccb00 2024-09-16T15:49:02.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.952+0000 7fdaf9a00640 1 -- 172.21.15.151:0/3199661346 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaf41e5900 con 0x7fdaf40ccb00 2024-09-16T15:49:02.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.952+0000 7fdaf9a00640 1 -- 172.21.15.151:0/3199661346 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaf41e61a0 con 0x7fdaf40ccb00 2024-09-16T15:49:02.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.953+0000 7fdaf9a00640 1 -- 172.21.15.151:0/3199661346 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaf406ac80 con 0x7fdaf40ccb00 2024-09-16T15:49:02.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.954+0000 7fdaeaa00640 1 -- 172.21.15.151:0/3199661346 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 31) ==== 102970+0+0 (secure 0 0 0) 0x7fdadc039040 con 0x7fdaf40ccb00 2024-09-16T15:49:02.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.955+0000 7fdaeaa00640 1 --2- 172.21.15.151:0/3199661346 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdad0079ae0 0x7fdad007bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:02.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.955+0000 7fdaeaa00640 1 -- 172.21.15.151:0/3199661346 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7fdadc0ba730 con 0x7fdaf40ccb00 2024-09-16T15:49:02.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.955+0000 7fdaf2a00640 1 --2- 172.21.15.151:0/3199661346 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdad0079ae0 0x7fdad007bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:02.957 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.956+0000 7fdaf2a00640 1 --2- 172.21.15.151:0/3199661346 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdad0079ae0 0x7fdad007bfa0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fdaf41e75f0 tx=0x7fdae4005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:49:02.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:02.959+0000 7fdaeaa00640 1 -- 172.21.15.151:0/3199661346 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fdaf406ac80 con 0x7fdaf40ccb00 2024-09-16T15:49:03.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:03.171+0000 7fdaf9a00640 1 -- 172.21.15.151:0/3199661346 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fdaf40cc5c0 con 0x7fdad0079ae0 2024-09-16T15:49:03.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:03.176+0000 7fdaeaa00640 1 -- 172.21.15.151:0/3199661346 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7fdaf40cc5c0 con 0x7fdad0079ae0 2024-09-16T15:49:03.177 INFO:teuthology.orchestra.run.smithi151.stdout:/volumes/_nogroup/sv1/36ce568d-e1cb-4659-a3f7-f6d7ceec4389 2024-09-16T15:49:03.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:03.179+0000 7fdaf9a00640 1 -- 172.21.15.151:0/3199661346 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdad0079ae0 msgr2=0x7fdad007bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:03.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:03.179+0000 7fdaf9a00640 1 --2- 172.21.15.151:0/3199661346 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdad0079ae0 0x7fdad007bfa0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fdaf41e75f0 tx=0x7fdae4005f90 comp rx=0 tx=0).stop 2024-09-16T15:49:03.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:03.179+0000 7fdaf9a00640 1 -- 172.21.15.151:0/3199661346 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdaf40ccb00 msgr2=0x7fdaf40aeac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:03.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:03.179+0000 7fdaf9a00640 1 --2- 172.21.15.151:0/3199661346 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdaf40ccb00 0x7fdaf40aeac0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fdadc02aa90 tx=0x7fdadc032c10 comp rx=0 tx=0).stop 2024-09-16T15:49:03.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:03.179+0000 7fdaf9a00640 1 -- 172.21.15.151:0/3199661346 shutdown_connections 2024-09-16T15:49:03.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:03.179+0000 7fdaf9a00640 1 --2- 172.21.15.151:0/3199661346 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdad0079ae0 0x7fdad007bfa0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:03.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:03.179+0000 7fdaf9a00640 1 --2- 172.21.15.151:0/3199661346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdaf4145750 0x7fdaf40af000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:03.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:03.179+0000 7fdaf9a00640 1 --2- 172.21.15.151:0/3199661346 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdaf40ccb00 0x7fdaf40aeac0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:03.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:03.179+0000 7fdaf9a00640 1 --2- 172.21.15.151:0/3199661346 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdaf406efa0 0x7fdaf41e66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:03.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:03.179+0000 7fdaf9a00640 1 -- 172.21.15.151:0/3199661346 >> 172.21.15.151:0/3199661346 conn(0x7fdaf40b1840 msgr2=0x7fdaf40af750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:49:03.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:03.179+0000 7fdaf9a00640 1 -- 172.21.15.151:0/3199661346 shutdown_connections 2024-09-16T15:49:03.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:03.179+0000 7fdaf9a00640 1 -- 172.21.15.151:0/3199661346 wait complete. 2024-09-16T15:49:03.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:49:03.196 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/36ce568d-e1cb-4659-a3f7-f6d7ceec4389 2024-09-16T15:49:03.196 DEBUG:teuthology.orchestra.run.smithi151:> ) 2024-09-16T15:49:03.231 INFO:teuthology.orchestra.run.smithi151.stdout:# file: volumes/_nogroup/sv1/36ce568d-e1cb-4659-a3f7-f6d7ceec4389 2024-09-16T15:49:03.231 INFO:teuthology.orchestra.run.smithi151.stdout:ceph.dir.rbytes="2972172288" 2024-09-16T15:49:03.231 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:49:03.232 INFO:tasks.cephfs.test_volumes:r_size = 2972172288 exp_size = 3221225472 2024-09-16T15:49:05.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:49:05.233 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/36ce568d-e1cb-4659-a3f7-f6d7ceec4389 2024-09-16T15:49:05.234 DEBUG:teuthology.orchestra.run.smithi151:> ) 2024-09-16T15:49:05.274 INFO:teuthology.orchestra.run.smithi151.stdout:# file: volumes/_nogroup/sv1/36ce568d-e1cb-4659-a3f7-f6d7ceec4389 2024-09-16T15:49:05.274 INFO:teuthology.orchestra.run.smithi151.stdout:ceph.dir.rbytes="2972172288" 2024-09-16T15:49:05.275 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:49:05.275 INFO:tasks.cephfs.test_volumes:r_size = 2972172288 exp_size = 3221225472 2024-09-16T15:49:07.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:49:07.276 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/36ce568d-e1cb-4659-a3f7-f6d7ceec4389 2024-09-16T15:49:07.276 DEBUG:teuthology.orchestra.run.smithi151:> ) 2024-09-16T15:49:07.317 INFO:teuthology.orchestra.run.smithi151.stdout:# file: volumes/_nogroup/sv1/36ce568d-e1cb-4659-a3f7-f6d7ceec4389 2024-09-16T15:49:07.317 INFO:teuthology.orchestra.run.smithi151.stdout:ceph.dir.rbytes="2972172288" 2024-09-16T15:49:07.318 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:49:07.318 INFO:tasks.cephfs.test_volumes:r_size = 2972172288 exp_size = 3221225472 2024-09-16T15:49:09.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:49:09.319 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/36ce568d-e1cb-4659-a3f7-f6d7ceec4389 2024-09-16T15:49:09.320 DEBUG:teuthology.orchestra.run.smithi151:> ) 2024-09-16T15:49:09.360 INFO:teuthology.orchestra.run.smithi151.stdout:# file: volumes/_nogroup/sv1/36ce568d-e1cb-4659-a3f7-f6d7ceec4389 2024-09-16T15:49:09.361 INFO:teuthology.orchestra.run.smithi151.stdout:ceph.dir.rbytes="3221225472" 2024-09-16T15:49:09.361 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:49:09.362 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 exp_size = 3221225472 2024-09-16T15:49:09.362 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone55130 2024-09-16T15:49:09.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:09.571+0000 7f8b9fe00640 1 -- 172.21.15.151:0/3717051009 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b9806efa0 msgr2=0x7f8b980cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:09.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:09.571+0000 7f8b9fe00640 1 --2- 172.21.15.151:0/3717051009 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b9806efa0 0x7f8b980cdbb0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f8b980af780 tx=0x7f8b9002a4a0 comp rx=0 tx=0).stop 2024-09-16T15:49:09.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:09.572+0000 7f8b9fe00640 1 -- 172.21.15.151:0/3717051009 shutdown_connections 2024-09-16T15:49:09.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:09.572+0000 7f8b9fe00640 1 --2- 172.21.15.151:0/3717051009 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b9806efa0 0x7f8b980cdbb0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:09.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:09.572+0000 7f8b9fe00640 1 --2- 172.21.15.151:0/3717051009 >> v2:172.21.15.167:3300/0 conn(0x7f8b980cccd0 0x7f8b98073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:09.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:09.572+0000 7f8b9fe00640 1 -- 172.21.15.151:0/3717051009 >> 172.21.15.151:0/3717051009 conn(0x7f8b980b1840 msgr2=0x7f8b980b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:49:09.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:09.572+0000 7f8b9fe00640 1 -- 172.21.15.151:0/3717051009 shutdown_connections 2024-09-16T15:49:09.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:09.572+0000 7f8b9fe00640 1 -- 172.21.15.151:0/3717051009 wait complete. 2024-09-16T15:49:09.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:09.573+0000 7f8b9fe00640 1 Processor -- start 2024-09-16T15:49:09.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:09.573+0000 7f8b9fe00640 1 -- start start 2024-09-16T15:49:09.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:09.574+0000 7f8b9fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b9806efa0 0x7f8b98154bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:09.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:09.574+0000 7f8b9fe00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b980cccd0 0x7f8b980cc490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:09.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:09.574+0000 7f8b9fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b98145750 0x7f8b981f7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:09.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:09.574+0000 7f8b9fe00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8b980ce480 con 0x7f8b98145750 2024-09-16T15:49:09.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:09.574+0000 7f8b9fe00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8b980ce300 con 0x7f8b9806efa0 2024-09-16T15:49:09.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:09.574+0000 7f8b9fe00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f8b980ce600 con 0x7f8b980cccd0 2024-09-16T15:49:09.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:09.575+0000 7f8b9de00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b98145750 0x7f8b981f7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:09.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:09.575+0000 7f8b9de00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b98145750 0x7f8b981f7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41498/0 (socket says 172.21.15.151:41498) 2024-09-16T15:49:09.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:09.575+0000 7f8b9de00640 1 -- 172.21.15.151:0/388980592 learned_addr learned my addr 172.21.15.151:0/388980592 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:49:09.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:09.575+0000 7f8b9ca00640 1 --2- 172.21.15.151:0/388980592 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b980cccd0 0x7f8b980cc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:09.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:09.575+0000 7f8b9d400640 1 --2- 172.21.15.151:0/388980592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b9806efa0 0x7f8b98154bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:09.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:09.575+0000 7f8b9de00640 1 -- 172.21.15.151:0/388980592 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b980cccd0 msgr2=0x7f8b980cc490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:09.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:09.575+0000 7f8b9de00640 1 --2- 172.21.15.151:0/388980592 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b980cccd0 0x7f8b980cc490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:09.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:09.575+0000 7f8b9de00640 1 -- 172.21.15.151:0/388980592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b9806efa0 msgr2=0x7f8b98154bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:09.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:09.575+0000 7f8b9de00640 1 --2- 172.21.15.151:0/388980592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b9806efa0 0x7f8b98154bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:09.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:09.575+0000 7f8b9de00640 1 -- 172.21.15.151:0/388980592 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b9813ff30 con 0x7f8b98145750 2024-09-16T15:49:09.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:09.575+0000 7f8b9de00640 1 --2- 172.21.15.151:0/388980592 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b98145750 0x7f8b981f7d20 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f8b90009920 tx=0x7f8b90032c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:49:09.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:09.576+0000 7f8b95600640 1 -- 172.21.15.151:0/388980592 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b90040070 con 0x7f8b98145750 2024-09-16T15:49:09.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:09.576+0000 7f8b95600640 1 -- 172.21.15.151:0/388980592 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8b90005310 con 0x7f8b98145750 2024-09-16T15:49:09.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:09.576+0000 7f8b95600640 1 -- 172.21.15.151:0/388980592 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b900047a0 con 0x7f8b98145750 2024-09-16T15:49:09.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:09.576+0000 7f8b9fe00640 1 -- 172.21.15.151:0/388980592 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b98140210 con 0x7f8b98145750 2024-09-16T15:49:09.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:09.576+0000 7f8b9fe00640 1 -- 172.21.15.151:0/388980592 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b981407d0 con 0x7f8b98145750 2024-09-16T15:49:09.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:09.577+0000 7f8b9fe00640 1 -- 172.21.15.151:0/388980592 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b980cdbb0 con 0x7f8b98145750 2024-09-16T15:49:09.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:09.578+0000 7f8b95600640 1 -- 172.21.15.151:0/388980592 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 31) ==== 102970+0+0 (secure 0 0 0) 0x7f8b90039040 con 0x7f8b98145750 2024-09-16T15:49:09.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:09.579+0000 7f8b95600640 1 --2- 172.21.15.151:0/388980592 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8b78079b10 0x7f8b7807bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:09.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:09.579+0000 7f8b95600640 1 -- 172.21.15.151:0/388980592 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7f8b900ba540 con 0x7f8b98145750 2024-09-16T15:49:09.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:09.580+0000 7f8b9d400640 1 --2- 172.21.15.151:0/388980592 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8b78079b10 0x7f8b7807bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:09.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:09.580+0000 7f8b9d400640 1 --2- 172.21.15.151:0/388980592 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8b78079b10 0x7f8b7807bfd0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c002200 tx=0x7f8b8c0019f0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:49:09.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:09.584+0000 7f8b95600640 1 -- 172.21.15.151:0/388980592 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8b90083fb0 con 0x7f8b98145750 2024-09-16T15:49:09.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:09.801+0000 7f8b9fe00640 1 -- 172.21.15.151:0/388980592 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone55130", "target": ["mon-mgr", ""]}) -- 0x7f8b980b2e90 con 0x7f8b78079b10 2024-09-16T15:49:09.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:09.871+0000 7f8b95600640 1 -- 172.21.15.151:0/388980592 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8b980b2e90 con 0x7f8b78079b10 2024-09-16T15:49:09.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:09.873+0000 7f8b9fe00640 1 -- 172.21.15.151:0/388980592 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8b78079b10 msgr2=0x7f8b7807bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:09.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:09.873+0000 7f8b9fe00640 1 --2- 172.21.15.151:0/388980592 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8b78079b10 0x7f8b7807bfd0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c002200 tx=0x7f8b8c0019f0 comp rx=0 tx=0).stop 2024-09-16T15:49:09.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:09.873+0000 7f8b9fe00640 1 -- 172.21.15.151:0/388980592 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b98145750 msgr2=0x7f8b981f7d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:09.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:09.873+0000 7f8b9fe00640 1 --2- 172.21.15.151:0/388980592 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b98145750 0x7f8b981f7d20 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f8b90009920 tx=0x7f8b90032c00 comp rx=0 tx=0).stop 2024-09-16T15:49:09.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:09.873+0000 7f8b9fe00640 1 -- 172.21.15.151:0/388980592 shutdown_connections 2024-09-16T15:49:09.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:09.873+0000 7f8b9fe00640 1 --2- 172.21.15.151:0/388980592 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8b78079b10 0x7f8b7807bfd0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:09.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:09.873+0000 7f8b9fe00640 1 --2- 172.21.15.151:0/388980592 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b98145750 0x7f8b981f7d20 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:09.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:09.873+0000 7f8b9fe00640 1 --2- 172.21.15.151:0/388980592 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b980cccd0 0x7f8b980cc490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:09.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:09.873+0000 7f8b9fe00640 1 --2- 172.21.15.151:0/388980592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b9806efa0 0x7f8b98154bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:09.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:09.873+0000 7f8b9fe00640 1 -- 172.21.15.151:0/388980592 >> 172.21.15.151:0/388980592 conn(0x7f8b980b1840 msgr2=0x7f8b980b3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:49:09.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:09.874+0000 7f8b9fe00640 1 -- 172.21.15.151:0/388980592 shutdown_connections 2024-09-16T15:49:09.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:09.874+0000 7f8b9fe00640 1 -- 172.21.15.151:0/388980592 wait complete. 2024-09-16T15:49:09.876 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:49:09.875+0000 7f48f7600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:49:09.876 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:49:09.875+0000 7f48f7600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:49:09.876 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:49:09.875+0000 7f48f7600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:49:09.876 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:49:09.875+0000 7f48f7600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:49:09.876 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:49:09.875+0000 7f48f7600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:49:09.892 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone23331 2024-09-16T15:49:09.893 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:49:09.891+0000 7f48f6c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:49:09.893 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:49:09.891+0000 7f48f6c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:49:09.893 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:49:09.891+0000 7f48f6c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:49:09.893 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:49:09.891+0000 7f48f6c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:49:09.893 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:49:09.891+0000 7f48f6c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:49:10.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:10.064+0000 7fc225200640 1 -- 172.21.15.151:0/3659200832 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc220145750 msgr2=0x7fc220073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:10.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:10.064+0000 7fc225200640 1 --2- 172.21.15.151:0/3659200832 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc220145750 0x7fc220073440 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc210009be0 tx=0x7fc21002a800 comp rx=0 tx=0).stop 2024-09-16T15:49:10.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:10.064+0000 7fc225200640 1 -- 172.21.15.151:0/3659200832 shutdown_connections 2024-09-16T15:49:10.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:10.064+0000 7fc225200640 1 --2- 172.21.15.151:0/3659200832 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc220145750 0x7fc220073440 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:10.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:10.064+0000 7fc225200640 1 -- 172.21.15.151:0/3659200832 >> 172.21.15.151:0/3659200832 conn(0x7fc2200b1840 msgr2=0x7fc2200b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:49:10.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:10.064+0000 7fc225200640 1 -- 172.21.15.151:0/3659200832 shutdown_connections 2024-09-16T15:49:10.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:10.064+0000 7fc225200640 1 -- 172.21.15.151:0/3659200832 wait complete. 2024-09-16T15:49:10.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:10.066+0000 7fc225200640 1 Processor -- start 2024-09-16T15:49:10.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:10.066+0000 7fc225200640 1 -- start start 2024-09-16T15:49:10.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:10.067+0000 7fc225200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc22006efa0 0x7fc2200b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:10.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:10.067+0000 7fc225200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc2200cccd0 0x7fc2200aed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:10.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:10.067+0000 7fc225200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc220145750 0x7fc220154600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:10.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:10.067+0000 7fc225200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc2200ce700 con 0x7fc220145750 2024-09-16T15:49:10.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:10.067+0000 7fc225200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc2200ce580 con 0x7fc22006efa0 2024-09-16T15:49:10.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:10.067+0000 7fc225200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fc2200ce880 con 0x7fc2200cccd0 2024-09-16T15:49:10.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:10.067+0000 7fc21e800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc220145750 0x7fc220154600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:10.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:10.067+0000 7fc215400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc2200cccd0 0x7fc2200aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:10.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:10.067+0000 7fc21de00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc22006efa0 0x7fc2200b41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:10.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:10.067+0000 7fc21e800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc220145750 0x7fc220154600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41512/0 (socket says 172.21.15.151:41512) 2024-09-16T15:49:10.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:10.067+0000 7fc21e800640 1 -- 172.21.15.151:0/2948232392 learned_addr learned my addr 172.21.15.151:0/2948232392 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:49:10.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:10.068+0000 7fc21de00640 1 -- 172.21.15.151:0/2948232392 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc2200cccd0 msgr2=0x7fc2200aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:10.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:10.068+0000 7fc21de00640 1 --2- 172.21.15.151:0/2948232392 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc2200cccd0 0x7fc2200aed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:10.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:10.068+0000 7fc21de00640 1 -- 172.21.15.151:0/2948232392 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc220145750 msgr2=0x7fc220154600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:10.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:10.068+0000 7fc21de00640 1 --2- 172.21.15.151:0/2948232392 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc220145750 0x7fc220154600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:10.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:10.068+0000 7fc21de00640 1 -- 172.21.15.151:0/2948232392 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2200ac080 con 0x7fc22006efa0 2024-09-16T15:49:10.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:10.068+0000 7fc21e800640 1 --2- 172.21.15.151:0/2948232392 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc220145750 0x7fc220154600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:49:10.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:10.068+0000 7fc21de00640 1 --2- 172.21.15.151:0/2948232392 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc22006efa0 0x7fc2200b41d0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fc210009000 tx=0x7fc210032d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:49:10.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:10.068+0000 7fc216a00640 1 -- 172.21.15.151:0/2948232392 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc210041020 con 0x7fc22006efa0 2024-09-16T15:49:10.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:10.068+0000 7fc216a00640 1 -- 172.21.15.151:0/2948232392 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc210004330 con 0x7fc22006efa0 2024-09-16T15:49:10.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:10.068+0000 7fc216a00640 1 -- 172.21.15.151:0/2948232392 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc210036040 con 0x7fc22006efa0 2024-09-16T15:49:10.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:10.068+0000 7fc225200640 1 -- 172.21.15.151:0/2948232392 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2200b36f0 con 0x7fc22006efa0 2024-09-16T15:49:10.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:10.068+0000 7fc225200640 1 -- 172.21.15.151:0/2948232392 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2200b3cb0 con 0x7fc22006efa0 2024-09-16T15:49:10.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:10.069+0000 7fc225200640 1 -- 172.21.15.151:0/2948232392 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2200ced70 con 0x7fc22006efa0 2024-09-16T15:49:10.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:10.071+0000 7fc216a00640 1 -- 172.21.15.151:0/2948232392 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 31) ==== 102970+0+0 (secure 0 0 0) 0x7fc210044020 con 0x7fc22006efa0 2024-09-16T15:49:10.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:10.071+0000 7fc216a00640 1 --2- 172.21.15.151:0/2948232392 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc1fc079ae0 0x7fc1fc07bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:10.073 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:10.072+0000 7fc216a00640 1 -- 172.21.15.151:0/2948232392 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7fc2100ba6e0 con 0x7fc22006efa0 2024-09-16T15:49:10.073 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:10.072+0000 7fc215400640 1 --2- 172.21.15.151:0/2948232392 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc1fc079ae0 0x7fc1fc07bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:10.073 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:10.072+0000 7fc215400640 1 --2- 172.21.15.151:0/2948232392 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc1fc079ae0 0x7fc1fc07bfa0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fc208007fd0 tx=0x7fc208009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:49:10.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:10.076+0000 7fc216a00640 1 -- 172.21.15.151:0/2948232392 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc21004b030 con 0x7fc22006efa0 2024-09-16T15:49:10.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:10.294+0000 7fc225200640 1 -- 172.21.15.151:0/2948232392 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone23331", "target": ["mon-mgr", ""]}) -- 0x7fc22015e950 con 0x7fc1fc079ae0 2024-09-16T15:49:10.877 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:49:10.876+0000 7f488dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:49:10.877 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:49:10.876+0000 7f488dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:49:10.877 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:49:10.876+0000 7f488dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:49:10.878 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:49:10.876+0000 7f488dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:49:10.878 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:49:10.876+0000 7f488dc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:49:10.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:10.968+0000 7fc216a00640 1 -- 172.21.15.151:0/2948232392 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc2080014d0 con 0x7fc1fc079ae0 2024-09-16T15:49:10.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:10.971+0000 7fc225200640 1 -- 172.21.15.151:0/2948232392 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc1fc079ae0 msgr2=0x7fc1fc07bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:10.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:10.971+0000 7fc225200640 1 --2- 172.21.15.151:0/2948232392 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc1fc079ae0 0x7fc1fc07bfa0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fc208007fd0 tx=0x7fc208009040 comp rx=0 tx=0).stop 2024-09-16T15:49:10.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:10.971+0000 7fc225200640 1 -- 172.21.15.151:0/2948232392 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc22006efa0 msgr2=0x7fc2200b41d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:10.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:10.971+0000 7fc225200640 1 --2- 172.21.15.151:0/2948232392 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc22006efa0 0x7fc2200b41d0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fc210009000 tx=0x7fc210032d10 comp rx=0 tx=0).stop 2024-09-16T15:49:10.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:10.971+0000 7fc225200640 1 -- 172.21.15.151:0/2948232392 shutdown_connections 2024-09-16T15:49:10.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:10.971+0000 7fc225200640 1 --2- 172.21.15.151:0/2948232392 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc1fc079ae0 0x7fc1fc07bfa0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:10.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:10.971+0000 7fc225200640 1 --2- 172.21.15.151:0/2948232392 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc220145750 0x7fc220154600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:10.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:10.971+0000 7fc225200640 1 --2- 172.21.15.151:0/2948232392 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc2200cccd0 0x7fc2200aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:10.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:10.971+0000 7fc225200640 1 --2- 172.21.15.151:0/2948232392 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc22006efa0 0x7fc2200b41d0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:10.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:10.971+0000 7fc225200640 1 -- 172.21.15.151:0/2948232392 >> 172.21.15.151:0/2948232392 conn(0x7fc2200b1840 msgr2=0x7fc2200b0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:49:10.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:10.971+0000 7fc225200640 1 -- 172.21.15.151:0/2948232392 shutdown_connections 2024-09-16T15:49:10.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:10.972+0000 7fc225200640 1 -- 172.21.15.151:0/2948232392 wait complete. 2024-09-16T15:49:10.994 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone13712 2024-09-16T15:49:11.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:11.242+0000 7fee25400640 1 -- 172.21.15.151:0/3936388678 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee200b4bd0 msgr2=0x7fee201437c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:11.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:11.242+0000 7fee25400640 1 --2- 172.21.15.151:0/3936388678 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee200b4bd0 0x7fee201437c0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fee18005fd0 tx=0x7fee1802aa00 comp rx=0 tx=0).stop 2024-09-16T15:49:11.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:11.242+0000 7fee25400640 1 -- 172.21.15.151:0/3936388678 shutdown_connections 2024-09-16T15:49:11.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:11.242+0000 7fee25400640 1 --2- 172.21.15.151:0/3936388678 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee20070870 0x7fee200b3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:11.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:11.242+0000 7fee25400640 1 --2- 172.21.15.151:0/3936388678 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee200b4bd0 0x7fee201437c0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:11.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:11.242+0000 7fee25400640 1 -- 172.21.15.151:0/3936388678 >> 172.21.15.151:0/3936388678 conn(0x7fee200b1840 msgr2=0x7fee200b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:49:11.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:11.243+0000 7fee25400640 1 -- 172.21.15.151:0/3936388678 shutdown_connections 2024-09-16T15:49:11.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:11.243+0000 7fee25400640 1 -- 172.21.15.151:0/3936388678 wait complete. 2024-09-16T15:49:11.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:11.245+0000 7fee25400640 1 Processor -- start 2024-09-16T15:49:11.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:11.246+0000 7fee25400640 1 -- start start 2024-09-16T15:49:11.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:11.247+0000 7fee25400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee20070870 0x7fee201500e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:11.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:11.247+0000 7fee25400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee200b4400 0x7fee201f6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:11.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:11.247+0000 7fee25400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee200b4bd0 0x7fee201f6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:11.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:11.247+0000 7fee25400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fee200cde60 con 0x7fee20070870 2024-09-16T15:49:11.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:11.247+0000 7fee25400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fee200cdce0 con 0x7fee200b4400 2024-09-16T15:49:11.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:11.247+0000 7fee25400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fee200cdfe0 con 0x7fee200b4bd0 2024-09-16T15:49:11.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:11.248+0000 7fee1e800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee20070870 0x7fee201500e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:11.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:11.248+0000 7fee1e800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee20070870 0x7fee201500e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41532/0 (socket says 172.21.15.151:41532) 2024-09-16T15:49:11.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:11.248+0000 7fee1e800640 1 -- 172.21.15.151:0/2652970694 learned_addr learned my addr 172.21.15.151:0/2652970694 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:49:11.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:11.248+0000 7fee1f200640 1 --2- 172.21.15.151:0/2652970694 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee200b4bd0 0x7fee201f6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:11.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:11.249+0000 7fee1e800640 1 -- 172.21.15.151:0/2652970694 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee200b4bd0 msgr2=0x7fee201f6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:11.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:11.249+0000 7fee1e800640 1 --2- 172.21.15.151:0/2652970694 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee200b4bd0 0x7fee201f6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:11.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:11.249+0000 7fee1e800640 1 -- 172.21.15.151:0/2652970694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee200b4400 msgr2=0x7fee201f6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:11.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:11.249+0000 7fee1e800640 1 --2- 172.21.15.151:0/2652970694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee200b4400 0x7fee201f6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:11.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:11.249+0000 7fee1e800640 1 -- 172.21.15.151:0/2652970694 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee2013d060 con 0x7fee20070870 2024-09-16T15:49:11.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:11.253+0000 7fee1e800640 1 --2- 172.21.15.151:0/2652970694 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee20070870 0x7fee201500e0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fee18000f80 tx=0x7fee18003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:49:11.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:11.254+0000 7fee16a00640 1 -- 172.21.15.151:0/2652970694 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee18046070 con 0x7fee20070870 2024-09-16T15:49:11.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:11.254+0000 7fee16a00640 1 -- 172.21.15.151:0/2652970694 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fee180045c0 con 0x7fee20070870 2024-09-16T15:49:11.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:11.254+0000 7fee16a00640 1 -- 172.21.15.151:0/2652970694 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee18033070 con 0x7fee20070870 2024-09-16T15:49:11.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:11.254+0000 7fee25400640 1 -- 172.21.15.151:0/2652970694 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee2013e550 con 0x7fee20070870 2024-09-16T15:49:11.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:11.254+0000 7fee25400640 1 -- 172.21.15.151:0/2652970694 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee20142e30 con 0x7fee20070870 2024-09-16T15:49:11.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:11.256+0000 7fee25400640 1 -- 172.21.15.151:0/2652970694 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee200b3ec0 con 0x7fee20070870 2024-09-16T15:49:11.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:11.259+0000 7fee16a00640 1 -- 172.21.15.151:0/2652970694 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 31) ==== 102970+0+0 (secure 0 0 0) 0x7fee18049070 con 0x7fee20070870 2024-09-16T15:49:11.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:11.265+0000 7fee16a00640 1 --2- 172.21.15.151:0/2652970694 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fee00079b60 0x7fee0007c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:11.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:11.265+0000 7fee16a00640 1 -- 172.21.15.151:0/2652970694 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7fee180ba500 con 0x7fee20070870 2024-09-16T15:49:11.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:11.270+0000 7fee1de00640 1 --2- 172.21.15.151:0/2652970694 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fee00079b60 0x7fee0007c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:11.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:11.273+0000 7fee16a00640 1 -- 172.21.15.151:0/2652970694 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fee1803e070 con 0x7fee20070870 2024-09-16T15:49:11.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:11.274+0000 7fee1de00640 1 --2- 172.21.15.151:0/2652970694 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fee00079b60 0x7fee0007c020 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fee201417b0 tx=0x7fee0c006c90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:49:11.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:11.656+0000 7fee25400640 1 -- 172.21.15.151:0/2652970694 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone13712", "target": ["mon-mgr", ""]}) -- 0x7fee200cd570 con 0x7fee00079b60 2024-09-16T15:49:12.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:12.273+0000 7fee16a00640 1 -- 172.21.15.151:0/2652970694 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mgrmap(e 32) ==== 103123+0+0 (secure 0 0 0) 0x7fee18036070 con 0x7fee20070870 2024-09-16T15:49:12.293 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:12.292+0000 7fee16a00640 1 -- 172.21.15.151:0/2652970694 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fee200cd570 con 0x7fee00079b60 2024-09-16T15:49:12.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:12.295+0000 7fee25400640 1 -- 172.21.15.151:0/2652970694 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fee00079b60 msgr2=0x7fee0007c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:12.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:12.295+0000 7fee25400640 1 --2- 172.21.15.151:0/2652970694 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fee00079b60 0x7fee0007c020 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fee201417b0 tx=0x7fee0c006c90 comp rx=0 tx=0).stop 2024-09-16T15:49:12.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:12.295+0000 7fee25400640 1 -- 172.21.15.151:0/2652970694 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee20070870 msgr2=0x7fee201500e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:12.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:12.295+0000 7fee25400640 1 --2- 172.21.15.151:0/2652970694 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee20070870 0x7fee201500e0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fee18000f80 tx=0x7fee18003f90 comp rx=0 tx=0).stop 2024-09-16T15:49:12.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:12.295+0000 7fee25400640 1 -- 172.21.15.151:0/2652970694 shutdown_connections 2024-09-16T15:49:12.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:12.295+0000 7fee25400640 1 --2- 172.21.15.151:0/2652970694 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fee00079b60 0x7fee0007c020 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:12.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:12.295+0000 7fee25400640 1 --2- 172.21.15.151:0/2652970694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee200b4400 0x7fee201f6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:12.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:12.295+0000 7fee25400640 1 --2- 172.21.15.151:0/2652970694 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee200b4bd0 0x7fee201f6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:12.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:12.295+0000 7fee25400640 1 --2- 172.21.15.151:0/2652970694 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee20070870 0x7fee201500e0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:12.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:12.295+0000 7fee25400640 1 -- 172.21.15.151:0/2652970694 >> 172.21.15.151:0/2652970694 conn(0x7fee200b1840 msgr2=0x7fee2015ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:49:12.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:12.295+0000 7fee25400640 1 -- 172.21.15.151:0/2652970694 shutdown_connections 2024-09-16T15:49:12.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:12.295+0000 7fee25400640 1 -- 172.21.15.151:0/2652970694 wait complete. 2024-09-16T15:49:12.316 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone53393 2024-09-16T15:49:12.415 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:49:12.413+0000 7f48f6200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:49:12.416 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:49:12.413+0000 7f48f6200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:49:12.416 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:49:12.413+0000 7f48f6200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:49:12.416 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:49:12.413+0000 7f48f6200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:49:12.416 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:49:12.413+0000 7f48f6200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:49:12.660 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:12.658+0000 7fc812800640 1 -- 172.21.15.151:0/634273541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc80c0af170 msgr2=0x7fc80c0cdad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:12.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:12.658+0000 7fc812800640 1 --2- 172.21.15.151:0/634273541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc80c0af170 0x7fc80c0cdad0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc009860 tx=0x7fc7fc02a540 comp rx=0 tx=0).stop 2024-09-16T15:49:12.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:12.659+0000 7fc812800640 1 -- 172.21.15.151:0/634273541 shutdown_connections 2024-09-16T15:49:12.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:12.659+0000 7fc812800640 1 --2- 172.21.15.151:0/634273541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc80c0af170 0x7fc80c0cdad0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:12.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:12.659+0000 7fc812800640 1 --2- 172.21.15.151:0/634273541 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc80c161490 0x7fc80c0b3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:12.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:12.659+0000 7fc812800640 1 -- 172.21.15.151:0/634273541 >> 172.21.15.151:0/634273541 conn(0x7fc80c0b1840 msgr2=0x7fc80c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:49:12.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:12.666+0000 7fc812800640 1 -- 172.21.15.151:0/634273541 shutdown_connections 2024-09-16T15:49:12.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:12.667+0000 7fc812800640 1 -- 172.21.15.151:0/634273541 wait complete. 2024-09-16T15:49:12.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:12.672+0000 7fc812800640 1 Processor -- start 2024-09-16T15:49:12.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:12.672+0000 7fc812800640 1 -- start start 2024-09-16T15:49:12.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:12.674+0000 7fc812800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc80c0af170 0x7fc80c15e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:12.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:12.674+0000 7fc812800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc80c0b4370 0x7fc80c1f9680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:12.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:12.675+0000 7fc812800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc80c161490 0x7fc80c1f9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:12.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:12.675+0000 7fc812800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c161bb0 con 0x7fc80c0b4370 2024-09-16T15:49:12.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:12.675+0000 7fc812800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c161d30 con 0x7fc80c161490 2024-09-16T15:49:12.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:12.675+0000 7fc812800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fc80c161eb0 con 0x7fc80c0af170 2024-09-16T15:49:12.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:12.675+0000 7fc80aa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc80c0b4370 0x7fc80c1f9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:12.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:12.675+0000 7fc80aa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc80c0b4370 0x7fc80c1f9680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41538/0 (socket says 172.21.15.151:41538) 2024-09-16T15:49:12.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:12.675+0000 7fc80aa00640 1 -- 172.21.15.151:0/38337666 learned_addr learned my addr 172.21.15.151:0/38337666 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:49:12.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:12.675+0000 7fc80b400640 1 --2- 172.21.15.151:0/38337666 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc80c0af170 0x7fc80c15e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:12.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:12.675+0000 7fc80be00640 1 --2- 172.21.15.151:0/38337666 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc80c161490 0x7fc80c1f9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:12.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:12.676+0000 7fc80aa00640 1 -- 172.21.15.151:0/38337666 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc80c0af170 msgr2=0x7fc80c15e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:12.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:12.676+0000 7fc80aa00640 1 --2- 172.21.15.151:0/38337666 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc80c0af170 0x7fc80c15e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:12.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:12.676+0000 7fc80aa00640 1 -- 172.21.15.151:0/38337666 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc80c161490 msgr2=0x7fc80c1f9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:12.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:12.676+0000 7fc80aa00640 1 --2- 172.21.15.151:0/38337666 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc80c161490 0x7fc80c1f9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:12.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:12.676+0000 7fc80aa00640 1 -- 172.21.15.151:0/38337666 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc80c0ad690 con 0x7fc80c0b4370 2024-09-16T15:49:12.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:12.676+0000 7fc80aa00640 1 --2- 172.21.15.151:0/38337666 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc80c0b4370 0x7fc80c1f9680 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc000c00 tx=0x7fc7fc032b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:49:12.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:12.678+0000 7fc803400640 1 -- 172.21.15.151:0/38337666 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc040070 con 0x7fc80c0b4370 2024-09-16T15:49:12.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:12.678+0000 7fc803400640 1 -- 172.21.15.151:0/38337666 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc7fc003af0 con 0x7fc80c0b4370 2024-09-16T15:49:12.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:12.678+0000 7fc812800640 1 -- 172.21.15.151:0/38337666 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc80c0ad930 con 0x7fc80c0b4370 2024-09-16T15:49:12.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:12.678+0000 7fc803400640 1 -- 172.21.15.151:0/38337666 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc032e40 con 0x7fc80c0b4370 2024-09-16T15:49:12.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:12.679+0000 7fc812800640 1 -- 172.21.15.151:0/38337666 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc80c0adef0 con 0x7fc80c0b4370 2024-09-16T15:49:12.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:12.682+0000 7fc812800640 1 -- 172.21.15.151:0/38337666 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc80c0ce310 con 0x7fc80c0b4370 2024-09-16T15:49:12.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:12.682+0000 7fc803400640 1 -- 172.21.15.151:0/38337666 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 32) ==== 103123+0+0 (secure 0 0 0) 0x7fc7fc039040 con 0x7fc80c0b4370 2024-09-16T15:49:12.685 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:12.683+0000 7fc803400640 1 --2- 172.21.15.151:0/38337666 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc7ec079c90 0x7fc7ec07c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:12.685 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:12.684+0000 7fc803400640 1 -- 172.21.15.151:0/38337666 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7fc7fc0baa30 con 0x7fc80c0b4370 2024-09-16T15:49:12.685 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:12.684+0000 7fc80b400640 1 --2- 172.21.15.151:0/38337666 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc7ec079c90 0x7fc7ec07c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:12.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:12.686+0000 7fc80b400640 1 --2- 172.21.15.151:0/38337666 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc7ec079c90 0x7fc7ec07c150 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fc80c074470 tx=0x7fc7f8009260 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:49:12.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:12.689+0000 7fc803400640 1 -- 172.21.15.151:0/38337666 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc7fc084af0 con 0x7fc80c0b4370 2024-09-16T15:49:13.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:13.073+0000 7fc812800640 1 -- 172.21.15.151:0/38337666 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone53393", "target": ["mon-mgr", ""]}) -- 0x7fc80c0b3e30 con 0x7fc7ec079c90 2024-09-16T15:49:14.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:14.306+0000 7fc803400640 1 -- 172.21.15.151:0/38337666 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mgrmap(e 33) ==== 103174+0+0 (secure 0 0 0) 0x7fc7fc0806f0 con 0x7fc80c0b4370 2024-09-16T15:49:14.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:14.727+0000 7fc803400640 1 -- 172.21.15.151:0/38337666 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc80c0b3e30 con 0x7fc7ec079c90 2024-09-16T15:49:14.732 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:14.731+0000 7fc812800640 1 -- 172.21.15.151:0/38337666 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc7ec079c90 msgr2=0x7fc7ec07c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:14.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:14.731+0000 7fc812800640 1 --2- 172.21.15.151:0/38337666 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc7ec079c90 0x7fc7ec07c150 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fc80c074470 tx=0x7fc7f8009260 comp rx=0 tx=0).stop 2024-09-16T15:49:14.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:14.731+0000 7fc812800640 1 -- 172.21.15.151:0/38337666 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc80c0b4370 msgr2=0x7fc80c1f9680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:14.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:14.731+0000 7fc812800640 1 --2- 172.21.15.151:0/38337666 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc80c0b4370 0x7fc80c1f9680 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc000c00 tx=0x7fc7fc032b90 comp rx=0 tx=0).stop 2024-09-16T15:49:14.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:14.732+0000 7fc812800640 1 -- 172.21.15.151:0/38337666 shutdown_connections 2024-09-16T15:49:14.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:14.732+0000 7fc812800640 1 --2- 172.21.15.151:0/38337666 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc7ec079c90 0x7fc7ec07c150 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:14.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:14.732+0000 7fc812800640 1 --2- 172.21.15.151:0/38337666 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc80c161490 0x7fc80c1f9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:14.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:14.732+0000 7fc812800640 1 --2- 172.21.15.151:0/38337666 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc80c0b4370 0x7fc80c1f9680 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:14.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:14.732+0000 7fc812800640 1 --2- 172.21.15.151:0/38337666 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc80c0af170 0x7fc80c15e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:14.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:14.732+0000 7fc812800640 1 -- 172.21.15.151:0/38337666 >> 172.21.15.151:0/38337666 conn(0x7fc80c0b1840 msgr2=0x7fc80c0af7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:49:14.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:14.732+0000 7fc812800640 1 -- 172.21.15.151:0/38337666 shutdown_connections 2024-09-16T15:49:14.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:14.732+0000 7fc812800640 1 -- 172.21.15.151:0/38337666 wait complete. 2024-09-16T15:49:14.762 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone15034 2024-09-16T15:49:14.839 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:49:14.837+0000 7f48f5800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:49:14.839 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:49:14.837+0000 7f48f5800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:49:14.839 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:49:14.837+0000 7f48f5800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:49:14.839 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:49:14.837+0000 7f48f5800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:49:14.839 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:49:14.837+0000 7f48f5800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:49:15.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:15.074+0000 7f4ca1800640 1 -- 172.21.15.151:0/90919390 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4c9c0ce930 msgr2=0x7f4c9c0d0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:15.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:15.074+0000 7f4ca1800640 1 --2- 172.21.15.151:0/90919390 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4c9c0ce930 0x7f4c9c0d0d50 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f4c84009830 tx=0x7f4c8402a3f0 comp rx=0 tx=0).stop 2024-09-16T15:49:15.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:15.076+0000 7f4ca1800640 1 -- 172.21.15.151:0/90919390 shutdown_connections 2024-09-16T15:49:15.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:15.076+0000 7f4ca1800640 1 --2- 172.21.15.151:0/90919390 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c9c161a90 0x7f4c9c0b16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:15.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:15.076+0000 7f4ca1800640 1 --2- 172.21.15.151:0/90919390 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4c9c0ce930 0x7f4c9c0d0d50 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:15.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:15.076+0000 7f4ca1800640 1 -- 172.21.15.151:0/90919390 >> 172.21.15.151:0/90919390 conn(0x7f4c9c0d1e40 msgr2=0x7f4c9c0d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:49:15.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:15.079+0000 7f4ca1800640 1 -- 172.21.15.151:0/90919390 shutdown_connections 2024-09-16T15:49:15.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:15.080+0000 7f4ca1800640 1 -- 172.21.15.151:0/90919390 wait complete. 2024-09-16T15:49:15.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:15.082+0000 7f4ca1800640 1 Processor -- start 2024-09-16T15:49:15.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:15.083+0000 7f4ca1800640 1 -- start start 2024-09-16T15:49:15.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:15.084+0000 7f4ca1800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c9c0ce930 0x7f4c9c14e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:15.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:15.084+0000 7f4ca1800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4c9c0d1940 0x7f4c9c14ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:15.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:15.084+0000 7f4ca1800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c9c161a90 0x7f4c9c0b4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:15.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:15.084+0000 7f4ca1800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9c0b0d30 con 0x7f4c9c0d1940 2024-09-16T15:49:15.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:15.084+0000 7f4ca1800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9c0cef20 con 0x7f4c9c0ce930 2024-09-16T15:49:15.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:15.084+0000 7f4ca1800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f4c9c0b0eb0 con 0x7f4c9c161a90 2024-09-16T15:49:15.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:15.085+0000 7f4c9a000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4c9c0d1940 0x7f4c9c14ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:15.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:15.085+0000 7f4c9a000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4c9c0d1940 0x7f4c9c14ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:40044/0 (socket says 172.21.15.151:40044) 2024-09-16T15:49:15.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:15.085+0000 7f4c9b400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c9c161a90 0x7f4c9c0b4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:15.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:15.085+0000 7f4c9a000640 1 -- 172.21.15.151:0/850943350 learned_addr learned my addr 172.21.15.151:0/850943350 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:49:15.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:15.086+0000 7f4c9aa00640 1 --2- 172.21.15.151:0/850943350 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c9c0ce930 0x7f4c9c14e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:15.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:15.086+0000 7f4c9a000640 1 -- 172.21.15.151:0/850943350 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c9c161a90 msgr2=0x7f4c9c0b4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:15.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:15.086+0000 7f4c9a000640 1 --2- 172.21.15.151:0/850943350 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c9c161a90 0x7f4c9c0b4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:15.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:15.086+0000 7f4c9a000640 1 -- 172.21.15.151:0/850943350 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c9c0ce930 msgr2=0x7f4c9c14e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:15.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:15.086+0000 7f4c9a000640 1 --2- 172.21.15.151:0/850943350 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c9c0ce930 0x7f4c9c14e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:15.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:15.086+0000 7f4c9a000640 1 -- 172.21.15.151:0/850943350 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c9c0b6740 con 0x7f4c9c0d1940 2024-09-16T15:49:15.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:15.087+0000 7f4c9a000640 1 --2- 172.21.15.151:0/850943350 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4c9c0d1940 0x7f4c9c14ae20 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c00c490 tx=0x7f4c8c00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:49:15.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:15.087+0000 7f4c92a00640 1 -- 172.21.15.151:0/850943350 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c016020 con 0x7f4c9c0d1940 2024-09-16T15:49:15.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:15.087+0000 7f4c92a00640 1 -- 172.21.15.151:0/850943350 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4c8c012940 con 0x7f4c9c0d1940 2024-09-16T15:49:15.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:15.087+0000 7f4c92a00640 1 -- 172.21.15.151:0/850943350 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c012d90 con 0x7f4c9c0d1940 2024-09-16T15:49:15.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:15.087+0000 7f4ca1800640 1 -- 172.21.15.151:0/850943350 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c9c0b51b0 con 0x7f4c9c0d1940 2024-09-16T15:49:15.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:15.087+0000 7f4ca1800640 1 -- 172.21.15.151:0/850943350 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c9c0b6cc0 con 0x7f4c9c0d1940 2024-09-16T15:49:15.090 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:15.089+0000 7f4ca1800640 1 -- 172.21.15.151:0/850943350 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c9c0b16b0 con 0x7f4c9c0d1940 2024-09-16T15:49:15.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:15.091+0000 7f4c92a00640 1 -- 172.21.15.151:0/850943350 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 33) ==== 103174+0+0 (secure 0 0 0) 0x7f4c8c019020 con 0x7f4c9c0d1940 2024-09-16T15:49:15.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:15.093+0000 7f4c92a00640 1 --2- 172.21.15.151:0/850943350 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4c78079d70 0x7f4c7807c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:15.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:15.093+0000 7f4c92a00640 1 -- 172.21.15.151:0/850943350 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7f4c8c098f00 con 0x7f4c9c0d1940 2024-09-16T15:49:15.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:15.103+0000 7f4c92a00640 1 -- 172.21.15.151:0/850943350 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4c8c020070 con 0x7f4c9c0d1940 2024-09-16T15:49:15.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:15.119+0000 7f4c9aa00640 1 --2- 172.21.15.151:0/850943350 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4c78079d70 0x7f4c7807c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:15.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:15.122+0000 7f4c9aa00640 1 --2- 172.21.15.151:0/850943350 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4c78079d70 0x7f4c7807c230 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f4c84034000 tx=0x7f4c84004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:49:15.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:15.482+0000 7f4ca1800640 1 -- 172.21.15.151:0/850943350 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone15034", "target": ["mon-mgr", ""]}) -- 0x7f4c9c070f50 con 0x7f4c78079d70 2024-09-16T15:49:16.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:16.378+0000 7f4c92a00640 1 -- 172.21.15.151:0/850943350 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mgrmap(e 34) ==== 103225+0+0 (secure 0 0 0) 0x7f4c8c05ec20 con 0x7f4c9c0d1940 2024-09-16T15:49:17.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:17.078+0000 7f4c92a00640 1 -- 172.21.15.151:0/850943350 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4c9c070f50 con 0x7f4c78079d70 2024-09-16T15:49:17.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:17.081+0000 7f4ca1800640 1 -- 172.21.15.151:0/850943350 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4c78079d70 msgr2=0x7f4c7807c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:17.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:17.081+0000 7f4ca1800640 1 --2- 172.21.15.151:0/850943350 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4c78079d70 0x7f4c7807c230 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f4c84034000 tx=0x7f4c84004730 comp rx=0 tx=0).stop 2024-09-16T15:49:17.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:17.081+0000 7f4ca1800640 1 -- 172.21.15.151:0/850943350 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4c9c0d1940 msgr2=0x7f4c9c14ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:17.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:17.081+0000 7f4ca1800640 1 --2- 172.21.15.151:0/850943350 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4c9c0d1940 0x7f4c9c14ae20 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c00c490 tx=0x7f4c8c00c950 comp rx=0 tx=0).stop 2024-09-16T15:49:17.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:17.081+0000 7f4ca1800640 1 -- 172.21.15.151:0/850943350 shutdown_connections 2024-09-16T15:49:17.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:17.081+0000 7f4ca1800640 1 --2- 172.21.15.151:0/850943350 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4c78079d70 0x7f4c7807c230 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:17.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:17.081+0000 7f4ca1800640 1 --2- 172.21.15.151:0/850943350 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c9c161a90 0x7f4c9c0b4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:17.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:17.081+0000 7f4ca1800640 1 --2- 172.21.15.151:0/850943350 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4c9c0d1940 0x7f4c9c14ae20 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:17.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:17.081+0000 7f4ca1800640 1 --2- 172.21.15.151:0/850943350 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c9c0ce930 0x7f4c9c14e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:17.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:17.081+0000 7f4ca1800640 1 -- 172.21.15.151:0/850943350 >> 172.21.15.151:0/850943350 conn(0x7f4c9c0d1e40 msgr2=0x7f4c9c075a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:49:17.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:17.081+0000 7f4ca1800640 1 -- 172.21.15.151:0/850943350 shutdown_connections 2024-09-16T15:49:17.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:17.081+0000 7f4ca1800640 1 -- 172.21.15.151:0/850943350 wait complete. 2024-09-16T15:49:17.113 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-16T15:49:17.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:17.425+0000 7f2c25000640 1 -- 172.21.15.151:0/482355985 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c20161b90 msgr2=0x7f2c20162040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:17.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:17.426+0000 7f2c25000640 1 --2- 172.21.15.151:0/482355985 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c20161b90 0x7f2c20162040 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f2c2006aa00 tx=0x7f2c1802a930 comp rx=0 tx=0).stop 2024-09-16T15:49:17.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:17.426+0000 7f2c25000640 1 -- 172.21.15.151:0/482355985 shutdown_connections 2024-09-16T15:49:17.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:17.426+0000 7f2c25000640 1 --2- 172.21.15.151:0/482355985 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c20161b90 0x7f2c20162040 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:17.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:17.426+0000 7f2c25000640 1 --2- 172.21.15.151:0/482355985 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c200b4570 0x7f2c200cd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:17.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:17.426+0000 7f2c25000640 1 --2- 172.21.15.151:0/482355985 >> v2:172.21.15.167:3300/0 conn(0x7f2c2015bcf0 0x7f2c200727c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:17.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:17.426+0000 7f2c25000640 1 -- 172.21.15.151:0/482355985 >> 172.21.15.151:0/482355985 conn(0x7f2c200b18e0 msgr2=0x7f2c200b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:49:17.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:17.426+0000 7f2c25000640 1 -- 172.21.15.151:0/482355985 shutdown_connections 2024-09-16T15:49:17.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:17.427+0000 7f2c25000640 1 -- 172.21.15.151:0/482355985 wait complete. 2024-09-16T15:49:17.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:17.429+0000 7f2c25000640 1 Processor -- start 2024-09-16T15:49:17.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:17.430+0000 7f2c25000640 1 -- start start 2024-09-16T15:49:17.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:17.431+0000 7f2c25000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c200b4570 0x7f2c201515b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:17.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:17.431+0000 7f2c25000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c2015bcf0 0x7f2c2014f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:17.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:17.431+0000 7f2c25000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c20161b90 0x7f2c2009c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:17.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:17.431+0000 7f2c25000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2006f590 con 0x7f2c2015bcf0 2024-09-16T15:49:17.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:17.431+0000 7f2c25000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2006f410 con 0x7f2c20161b90 2024-09-16T15:49:17.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:17.431+0000 7f2c25000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f2c2006f710 con 0x7f2c200b4570 2024-09-16T15:49:17.433 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:17.431+0000 7f2c1d400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c2015bcf0 0x7f2c2014f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:17.433 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:17.431+0000 7f2c1d400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c2015bcf0 0x7f2c2014f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:40062/0 (socket says 172.21.15.151:40062) 2024-09-16T15:49:17.433 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:17.431+0000 7f2c1d400640 1 -- 172.21.15.151:0/536878140 learned_addr learned my addr 172.21.15.151:0/536878140 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:49:17.433 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:17.432+0000 7f2c1de00640 1 --2- 172.21.15.151:0/536878140 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c200b4570 0x7f2c201515b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:17.434 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:17.432+0000 7f2c1e800640 1 --2- 172.21.15.151:0/536878140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c20161b90 0x7f2c2009c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:17.434 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:17.432+0000 7f2c1d400640 1 -- 172.21.15.151:0/536878140 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c200b4570 msgr2=0x7f2c201515b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:17.434 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:17.432+0000 7f2c1d400640 1 --2- 172.21.15.151:0/536878140 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c200b4570 0x7f2c201515b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:17.434 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:17.432+0000 7f2c1d400640 1 -- 172.21.15.151:0/536878140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c20161b90 msgr2=0x7f2c2009c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:17.434 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:17.432+0000 7f2c1d400640 1 --2- 172.21.15.151:0/536878140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c20161b90 0x7f2c2009c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:17.434 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:17.432+0000 7f2c1d400640 1 -- 172.21.15.151:0/536878140 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c20158250 con 0x7f2c2015bcf0 2024-09-16T15:49:17.434 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:17.432+0000 7f2c1de00640 1 --2- 172.21.15.151:0/536878140 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c200b4570 0x7f2c201515b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:49:17.434 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:17.433+0000 7f2c1d400640 1 --2- 172.21.15.151:0/536878140 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c2015bcf0 0x7f2c2014f760 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c00b480 tx=0x7f2c0c00b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:49:17.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:17.433+0000 7f2c16000640 1 -- 172.21.15.151:0/536878140 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c0c014070 con 0x7f2c2015bcf0 2024-09-16T15:49:17.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:17.433+0000 7f2c16000640 1 -- 172.21.15.151:0/536878140 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2c0c010960 con 0x7f2c2015bcf0 2024-09-16T15:49:17.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:17.434+0000 7f2c16000640 1 -- 172.21.15.151:0/536878140 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c0c010db0 con 0x7f2c2015bcf0 2024-09-16T15:49:17.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:17.434+0000 7f2c25000640 1 -- 172.21.15.151:0/536878140 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c20158530 con 0x7f2c2015bcf0 2024-09-16T15:49:17.438 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:17.434+0000 7f2c25000640 1 -- 172.21.15.151:0/536878140 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c20158af0 con 0x7f2c2015bcf0 2024-09-16T15:49:17.438 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:17.436+0000 7f2c25000640 1 -- 172.21.15.151:0/536878140 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c20069360 con 0x7f2c2015bcf0 2024-09-16T15:49:17.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:17.447+0000 7f2c16000640 1 -- 172.21.15.151:0/536878140 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 34) ==== 103225+0+0 (secure 0 0 0) 0x7f2c0c017070 con 0x7f2c2015bcf0 2024-09-16T15:49:17.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:17.449+0000 7f2c16000640 1 --2- 172.21.15.151:0/536878140 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2c00079db0 0x7f2c0007c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:17.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:17.449+0000 7f2c16000640 1 -- 172.21.15.151:0/536878140 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7f2c0c099ce0 con 0x7f2c2015bcf0 2024-09-16T15:49:17.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:17.451+0000 7f2c16000640 1 -- 172.21.15.151:0/536878140 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2c0c0639e0 con 0x7f2c2015bcf0 2024-09-16T15:49:17.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:17.469+0000 7f2c1de00640 1 --2- 172.21.15.151:0/536878140 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2c00079db0 0x7f2c0007c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:17.484 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:17.482+0000 7f2c1de00640 1 --2- 172.21.15.151:0/536878140 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2c00079db0 0x7f2c0007c270 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f2c10009930 tx=0x7f2c10008040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:49:17.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:17.909+0000 7f2c25000640 1 -- 172.21.15.151:0/536878140 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f2c200727c0 con 0x7f2c2015bcf0 2024-09-16T15:49:17.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:17.909+0000 7f2c16000640 1 -- 172.21.15.151:0/536878140 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2176 (secure 0 0 0) 0x7f2c0c0632e0 con 0x7f2c2015bcf0 2024-09-16T15:49:17.910 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:49:17.911 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-16T15:49:17.911 INFO:teuthology.orchestra.run.smithi151.stdout: "fsid": "1757be18-e41a-4446-bb4f-ee4ac81c39df", 2024-09-16T15:49:17.911 INFO:teuthology.orchestra.run.smithi151.stdout: "health": { 2024-09-16T15:49:17.911 INFO:teuthology.orchestra.run.smithi151.stdout: "status": "HEALTH_OK", 2024-09-16T15:49:17.911 INFO:teuthology.orchestra.run.smithi151.stdout: "checks": {}, 2024-09-16T15:49:17.911 INFO:teuthology.orchestra.run.smithi151.stdout: "mutes": [] 2024-09-16T15:49:17.911 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:49:17.911 INFO:teuthology.orchestra.run.smithi151.stdout: "election_epoch": 4, 2024-09-16T15:49:17.911 INFO:teuthology.orchestra.run.smithi151.stdout: "quorum": [ 2024-09-16T15:49:17.911 INFO:teuthology.orchestra.run.smithi151.stdout: 0, 2024-09-16T15:49:17.911 INFO:teuthology.orchestra.run.smithi151.stdout: 1, 2024-09-16T15:49:17.911 INFO:teuthology.orchestra.run.smithi151.stdout: 2 2024-09-16T15:49:17.911 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:49:17.912 INFO:teuthology.orchestra.run.smithi151.stdout: "quorum_names": [ 2024-09-16T15:49:17.912 INFO:teuthology.orchestra.run.smithi151.stdout: "a", 2024-09-16T15:49:17.912 INFO:teuthology.orchestra.run.smithi151.stdout: "b", 2024-09-16T15:49:17.912 INFO:teuthology.orchestra.run.smithi151.stdout: "c" 2024-09-16T15:49:17.912 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:49:17.912 INFO:teuthology.orchestra.run.smithi151.stdout: "quorum_age": 621, 2024-09-16T15:49:17.912 INFO:teuthology.orchestra.run.smithi151.stdout: "monmap": { 2024-09-16T15:49:17.912 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 1, 2024-09-16T15:49:17.912 INFO:teuthology.orchestra.run.smithi151.stdout: "min_mon_release_name": "squid", 2024-09-16T15:49:17.912 INFO:teuthology.orchestra.run.smithi151.stdout: "num_mons": 3 2024-09-16T15:49:17.912 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:49:17.912 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap": { 2024-09-16T15:49:17.912 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 91, 2024-09-16T15:49:17.912 INFO:teuthology.orchestra.run.smithi151.stdout: "num_osds": 8, 2024-09-16T15:49:17.912 INFO:teuthology.orchestra.run.smithi151.stdout: "num_up_osds": 8, 2024-09-16T15:49:17.913 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_up_since": 1726501150, 2024-09-16T15:49:17.913 INFO:teuthology.orchestra.run.smithi151.stdout: "num_in_osds": 8, 2024-09-16T15:49:17.913 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_in_since": 1726501140, 2024-09-16T15:49:17.913 INFO:teuthology.orchestra.run.smithi151.stdout: "num_remapped_pgs": 0 2024-09-16T15:49:17.913 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:49:17.913 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap": { 2024-09-16T15:49:17.913 INFO:teuthology.orchestra.run.smithi151.stdout: "pgs_by_state": [ 2024-09-16T15:49:17.913 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:49:17.913 INFO:teuthology.orchestra.run.smithi151.stdout: "state_name": "active+clean", 2024-09-16T15:49:17.913 INFO:teuthology.orchestra.run.smithi151.stdout: "count": 97 2024-09-16T15:49:17.913 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:49:17.913 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:49:17.913 INFO:teuthology.orchestra.run.smithi151.stdout: "num_pgs": 97, 2024-09-16T15:49:17.913 INFO:teuthology.orchestra.run.smithi151.stdout: "num_pools": 3, 2024-09-16T15:49:17.914 INFO:teuthology.orchestra.run.smithi151.stdout: "num_objects": 868, 2024-09-16T15:49:17.914 INFO:teuthology.orchestra.run.smithi151.stdout: "data_bytes": 3515733944, 2024-09-16T15:49:17.914 INFO:teuthology.orchestra.run.smithi151.stdout: "bytes_used": 7387537408, 2024-09-16T15:49:17.914 INFO:teuthology.orchestra.run.smithi151.stdout: "bytes_avail": 765706575872, 2024-09-16T15:49:17.914 INFO:teuthology.orchestra.run.smithi151.stdout: "bytes_total": 773094113280, 2024-09-16T15:49:17.914 INFO:teuthology.orchestra.run.smithi151.stdout: "read_bytes_sec": 28615166, 2024-09-16T15:49:17.914 INFO:teuthology.orchestra.run.smithi151.stdout: "write_bytes_sec": 62476835, 2024-09-16T15:49:17.914 INFO:teuthology.orchestra.run.smithi151.stdout: "read_op_per_sec": 7, 2024-09-16T15:49:17.914 INFO:teuthology.orchestra.run.smithi151.stdout: "write_op_per_sec": 17 2024-09-16T15:49:17.914 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:49:17.914 INFO:teuthology.orchestra.run.smithi151.stdout: "fsmap": { 2024-09-16T15:49:17.914 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 57, 2024-09-16T15:49:17.914 INFO:teuthology.orchestra.run.smithi151.stdout: "btime": "2024-09-16T15:48:35:029906+0000", 2024-09-16T15:49:17.914 INFO:teuthology.orchestra.run.smithi151.stdout: "id": 6, 2024-09-16T15:49:17.914 INFO:teuthology.orchestra.run.smithi151.stdout: "up": 1, 2024-09-16T15:49:17.915 INFO:teuthology.orchestra.run.smithi151.stdout: "in": 1, 2024-09-16T15:49:17.915 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 1, 2024-09-16T15:49:17.915 INFO:teuthology.orchestra.run.smithi151.stdout: "by_rank": [ 2024-09-16T15:49:17.915 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:49:17.915 INFO:teuthology.orchestra.run.smithi151.stdout: "filesystem_id": 6, 2024-09-16T15:49:17.915 INFO:teuthology.orchestra.run.smithi151.stdout: "rank": 0, 2024-09-16T15:49:17.915 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "c", 2024-09-16T15:49:17.915 INFO:teuthology.orchestra.run.smithi151.stdout: "status": "up:active", 2024-09-16T15:49:17.915 INFO:teuthology.orchestra.run.smithi151.stdout: "gid": 6673 2024-09-16T15:49:17.915 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:49:17.915 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:49:17.915 INFO:teuthology.orchestra.run.smithi151.stdout: "up:standby": 3 2024-09-16T15:49:17.915 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:49:17.915 INFO:teuthology.orchestra.run.smithi151.stdout: "mgrmap": { 2024-09-16T15:49:17.916 INFO:teuthology.orchestra.run.smithi151.stdout: "available": true, 2024-09-16T15:49:17.916 INFO:teuthology.orchestra.run.smithi151.stdout: "num_standbys": 1, 2024-09-16T15:49:17.916 INFO:teuthology.orchestra.run.smithi151.stdout: "modules": [ 2024-09-16T15:49:17.916 INFO:teuthology.orchestra.run.smithi151.stdout: "iostat", 2024-09-16T15:49:17.916 INFO:teuthology.orchestra.run.smithi151.stdout: "nfs", 2024-09-16T15:49:17.916 INFO:teuthology.orchestra.run.smithi151.stdout: "restful" 2024-09-16T15:49:17.916 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:49:17.916 INFO:teuthology.orchestra.run.smithi151.stdout: "services": {} 2024-09-16T15:49:17.916 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:49:17.916 INFO:teuthology.orchestra.run.smithi151.stdout: "servicemap": { 2024-09-16T15:49:17.916 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 13, 2024-09-16T15:49:17.916 INFO:teuthology.orchestra.run.smithi151.stdout: "modified": "2024-09-16T15:48:01.742446+0000", 2024-09-16T15:49:17.916 INFO:teuthology.orchestra.run.smithi151.stdout: "services": {} 2024-09-16T15:49:17.916 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:49:17.917 INFO:teuthology.orchestra.run.smithi151.stdout: "progress_events": { 2024-09-16T15:49:17.917 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr-vol-ongoing-clones": { 2024-09-16T15:49:17.917 INFO:teuthology.orchestra.run.smithi151.stdout: "message": "2 ongoing clones - average progress is 37.76% (3s)\n [==========..................] (remaining: 6s)", 2024-09-16T15:49:17.917 INFO:teuthology.orchestra.run.smithi151.stdout: "progress": 0.37760001420974731, 2024-09-16T15:49:17.917 INFO:teuthology.orchestra.run.smithi151.stdout: "add_to_ceph_s": true 2024-09-16T15:49:17.917 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:49:17.917 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:49:17.917 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-16T15:49:17.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:17.912+0000 7f2c25000640 1 -- 172.21.15.151:0/536878140 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2c00079db0 msgr2=0x7f2c0007c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:17.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:17.912+0000 7f2c25000640 1 --2- 172.21.15.151:0/536878140 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2c00079db0 0x7f2c0007c270 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f2c10009930 tx=0x7f2c10008040 comp rx=0 tx=0).stop 2024-09-16T15:49:17.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:17.913+0000 7f2c25000640 1 -- 172.21.15.151:0/536878140 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c2015bcf0 msgr2=0x7f2c2014f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:17.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:17.913+0000 7f2c25000640 1 --2- 172.21.15.151:0/536878140 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c2015bcf0 0x7f2c2014f760 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c00b480 tx=0x7f2c0c00b940 comp rx=0 tx=0).stop 2024-09-16T15:49:17.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:17.913+0000 7f2c25000640 1 -- 172.21.15.151:0/536878140 shutdown_connections 2024-09-16T15:49:17.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:17.913+0000 7f2c25000640 1 --2- 172.21.15.151:0/536878140 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2c00079db0 0x7f2c0007c270 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:17.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:17.913+0000 7f2c25000640 1 --2- 172.21.15.151:0/536878140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c20161b90 0x7f2c2009c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:17.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:17.913+0000 7f2c25000640 1 --2- 172.21.15.151:0/536878140 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c2015bcf0 0x7f2c2014f760 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:17.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:17.913+0000 7f2c25000640 1 --2- 172.21.15.151:0/536878140 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c200b4570 0x7f2c201515b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:17.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:17.913+0000 7f2c25000640 1 -- 172.21.15.151:0/536878140 >> 172.21.15.151:0/536878140 conn(0x7f2c200b18e0 msgr2=0x7f2c200cfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:49:17.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:17.913+0000 7f2c25000640 1 -- 172.21.15.151:0/536878140 shutdown_connections 2024-09-16T15:49:17.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:17.913+0000 7f2c25000640 1 -- 172.21.15.151:0/536878140 wait complete. 2024-09-16T15:49:18.947 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-16T15:49:19.132 DEBUG:teuthology.orchestra.run.smithi151:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:49:19.139 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:49:19.287 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:19.286+0000 7f53b6c00640 1 -- 172.21.15.151:0/716894605 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53b01616e0 msgr2=0x7f53b00b41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:19.287 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:19.286+0000 7f53b6c00640 1 --2- 172.21.15.151:0/716894605 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53b01616e0 0x7f53b00b41c0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f53b00cb060 tx=0x7f53a002a440 comp rx=0 tx=0).stop 2024-09-16T15:49:19.287 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:19.286+0000 7f53b6c00640 1 -- 172.21.15.151:0/716894605 shutdown_connections 2024-09-16T15:49:19.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:19.286+0000 7f53b6c00640 1 --2- 172.21.15.151:0/716894605 >> v2:172.21.15.167:3300/0 conn(0x7f53b00b4ae0 0x7f53b00d2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:19.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:19.286+0000 7f53b6c00640 1 --2- 172.21.15.151:0/716894605 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f53b00d1ee0 0x7f53b00d2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:19.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:19.286+0000 7f53b6c00640 1 --2- 172.21.15.151:0/716894605 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53b01616e0 0x7f53b00b41c0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:19.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:19.286+0000 7f53b6c00640 1 -- 172.21.15.151:0/716894605 >> 172.21.15.151:0/716894605 conn(0x7f53b00d3fe0 msgr2=0x7f53b00d43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:49:19.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:19.286+0000 7f53b6c00640 1 -- 172.21.15.151:0/716894605 shutdown_connections 2024-09-16T15:49:19.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:19.286+0000 7f53b6c00640 1 -- 172.21.15.151:0/716894605 wait complete. 2024-09-16T15:49:19.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:19.288+0000 7f53b6c00640 1 Processor -- start 2024-09-16T15:49:19.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:19.288+0000 7f53b6c00640 1 -- start start 2024-09-16T15:49:19.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:19.289+0000 7f53b6c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53b00b4ae0 0x7f53b006e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:19.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:19.289+0000 7f53b6c00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f53b00d1ee0 0x7f53b009e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:19.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:19.289+0000 7f53b6c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53b01616e0 0x7f53b01f58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:19.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:19.289+0000 7f53b6c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f53b0161c30 con 0x7f53b00b4ae0 2024-09-16T15:49:19.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:19.289+0000 7f53b6c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f53b00b4910 con 0x7f53b01616e0 2024-09-16T15:49:19.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:19.289+0000 7f53b6c00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f53b00d03b0 con 0x7f53b00d1ee0 2024-09-16T15:49:19.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:19.289+0000 7f53afe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53b00b4ae0 0x7f53b006e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:19.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:19.289+0000 7f53afe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53b00b4ae0 0x7f53b006e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:40074/0 (socket says 172.21.15.151:40074) 2024-09-16T15:49:19.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:19.289+0000 7f53afe00640 1 -- 172.21.15.151:0/885690053 learned_addr learned my addr 172.21.15.151:0/885690053 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:49:19.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:19.290+0000 7f53afe00640 1 -- 172.21.15.151:0/885690053 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f53b00d1ee0 msgr2=0x7f53b009e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:19.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:19.290+0000 7f53afe00640 1 --2- 172.21.15.151:0/885690053 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f53b00d1ee0 0x7f53b009e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:19.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:19.290+0000 7f53afe00640 1 -- 172.21.15.151:0/885690053 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53b01616e0 msgr2=0x7f53b01f58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:19.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:19.290+0000 7f53afe00640 1 --2- 172.21.15.151:0/885690053 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53b01616e0 0x7f53b01f58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:19.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:19.290+0000 7f53afe00640 1 -- 172.21.15.151:0/885690053 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53b00a04e0 con 0x7f53b00b4ae0 2024-09-16T15:49:19.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:19.290+0000 7f53afe00640 1 --2- 172.21.15.151:0/885690053 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53b00b4ae0 0x7f53b006e540 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f53a0034040 tx=0x7f53a0003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:49:19.293 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:19.290+0000 7f53acc00640 1 -- 172.21.15.151:0/885690053 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a003e030 con 0x7f53b00b4ae0 2024-09-16T15:49:19.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:19.290+0000 7f53acc00640 1 -- 172.21.15.151:0/885690053 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f53a0003e00 con 0x7f53b00b4ae0 2024-09-16T15:49:19.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:19.290+0000 7f53b6c00640 1 -- 172.21.15.151:0/885690053 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53b00a07c0 con 0x7f53b00b4ae0 2024-09-16T15:49:19.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:19.291+0000 7f53acc00640 1 -- 172.21.15.151:0/885690053 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a0035040 con 0x7f53b00b4ae0 2024-09-16T15:49:19.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:19.291+0000 7f53b6c00640 1 -- 172.21.15.151:0/885690053 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53b00a0d80 con 0x7f53b00b4ae0 2024-09-16T15:49:19.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:19.292+0000 7f53b6c00640 1 -- 172.21.15.151:0/885690053 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53b00b41c0 con 0x7f53b00b4ae0 2024-09-16T15:49:19.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:19.294+0000 7f53acc00640 1 -- 172.21.15.151:0/885690053 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 34) ==== 103225+0+0 (secure 0 0 0) 0x7f53a0041030 con 0x7f53b00b4ae0 2024-09-16T15:49:19.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:19.296+0000 7f53acc00640 1 --2- 172.21.15.151:0/885690053 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f538c079db0 0x7f538c07c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:19.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:19.296+0000 7f53acc00640 1 -- 172.21.15.151:0/885690053 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7f53a00ba0d0 con 0x7f53b00b4ae0 2024-09-16T15:49:19.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:19.300+0000 7f53acc00640 1 -- 172.21.15.151:0/885690053 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f53a0083d90 con 0x7f53b00b4ae0 2024-09-16T15:49:19.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:19.360+0000 7f53a7e00640 1 --2- 172.21.15.151:0/885690053 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f538c079db0 0x7f538c07c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:19.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:19.369+0000 7f53a7e00640 1 --2- 172.21.15.151:0/885690053 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f538c079db0 0x7f538c07c270 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f53b006dc20 tx=0x7f5398006c90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:49:19.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:19.750+0000 7f53b6c00640 1 -- 172.21.15.151:0/885690053 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f53b00b3280 con 0x7f53b00b4ae0 2024-09-16T15:49:19.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:19.750+0000 7f53acc00640 1 -- 172.21.15.151:0/885690053 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2181 (secure 0 0 0) 0x7f53a0083690 con 0x7f53b00b4ae0 2024-09-16T15:49:19.751 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:49:19.752 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-16T15:49:19.752 INFO:teuthology.orchestra.run.smithi151.stdout: "fsid": "1757be18-e41a-4446-bb4f-ee4ac81c39df", 2024-09-16T15:49:19.752 INFO:teuthology.orchestra.run.smithi151.stdout: "health": { 2024-09-16T15:49:19.752 INFO:teuthology.orchestra.run.smithi151.stdout: "status": "HEALTH_OK", 2024-09-16T15:49:19.752 INFO:teuthology.orchestra.run.smithi151.stdout: "checks": {}, 2024-09-16T15:49:19.752 INFO:teuthology.orchestra.run.smithi151.stdout: "mutes": [] 2024-09-16T15:49:19.752 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:49:19.752 INFO:teuthology.orchestra.run.smithi151.stdout: "election_epoch": 4, 2024-09-16T15:49:19.752 INFO:teuthology.orchestra.run.smithi151.stdout: "quorum": [ 2024-09-16T15:49:19.752 INFO:teuthology.orchestra.run.smithi151.stdout: 0, 2024-09-16T15:49:19.752 INFO:teuthology.orchestra.run.smithi151.stdout: 1, 2024-09-16T15:49:19.752 INFO:teuthology.orchestra.run.smithi151.stdout: 2 2024-09-16T15:49:19.753 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:49:19.753 INFO:teuthology.orchestra.run.smithi151.stdout: "quorum_names": [ 2024-09-16T15:49:19.753 INFO:teuthology.orchestra.run.smithi151.stdout: "a", 2024-09-16T15:49:19.753 INFO:teuthology.orchestra.run.smithi151.stdout: "b", 2024-09-16T15:49:19.753 INFO:teuthology.orchestra.run.smithi151.stdout: "c" 2024-09-16T15:49:19.753 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:49:19.753 INFO:teuthology.orchestra.run.smithi151.stdout: "quorum_age": 623, 2024-09-16T15:49:19.753 INFO:teuthology.orchestra.run.smithi151.stdout: "monmap": { 2024-09-16T15:49:19.753 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 1, 2024-09-16T15:49:19.753 INFO:teuthology.orchestra.run.smithi151.stdout: "min_mon_release_name": "squid", 2024-09-16T15:49:19.753 INFO:teuthology.orchestra.run.smithi151.stdout: "num_mons": 3 2024-09-16T15:49:19.753 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:49:19.753 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap": { 2024-09-16T15:49:19.753 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 91, 2024-09-16T15:49:19.753 INFO:teuthology.orchestra.run.smithi151.stdout: "num_osds": 8, 2024-09-16T15:49:19.754 INFO:teuthology.orchestra.run.smithi151.stdout: "num_up_osds": 8, 2024-09-16T15:49:19.754 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_up_since": 1726501150, 2024-09-16T15:49:19.754 INFO:teuthology.orchestra.run.smithi151.stdout: "num_in_osds": 8, 2024-09-16T15:49:19.754 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_in_since": 1726501140, 2024-09-16T15:49:19.754 INFO:teuthology.orchestra.run.smithi151.stdout: "num_remapped_pgs": 0 2024-09-16T15:49:19.754 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:49:19.754 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap": { 2024-09-16T15:49:19.754 INFO:teuthology.orchestra.run.smithi151.stdout: "pgs_by_state": [ 2024-09-16T15:49:19.754 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:49:19.754 INFO:teuthology.orchestra.run.smithi151.stdout: "state_name": "active+clean", 2024-09-16T15:49:19.754 INFO:teuthology.orchestra.run.smithi151.stdout: "count": 97 2024-09-16T15:49:19.754 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:49:19.754 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:49:19.754 INFO:teuthology.orchestra.run.smithi151.stdout: "num_pgs": 97, 2024-09-16T15:49:19.755 INFO:teuthology.orchestra.run.smithi151.stdout: "num_pools": 3, 2024-09-16T15:49:19.755 INFO:teuthology.orchestra.run.smithi151.stdout: "num_objects": 1599, 2024-09-16T15:49:19.755 INFO:teuthology.orchestra.run.smithi151.stdout: "data_bytes": 6565249202, 2024-09-16T15:49:19.755 INFO:teuthology.orchestra.run.smithi151.stdout: "bytes_used": 13202341888, 2024-09-16T15:49:19.755 INFO:teuthology.orchestra.run.smithi151.stdout: "bytes_avail": 759891771392, 2024-09-16T15:49:19.755 INFO:teuthology.orchestra.run.smithi151.stdout: "bytes_total": 773094113280, 2024-09-16T15:49:19.755 INFO:teuthology.orchestra.run.smithi151.stdout: "read_bytes_sec": 294483040, 2024-09-16T15:49:19.755 INFO:teuthology.orchestra.run.smithi151.stdout: "write_bytes_sec": 309522640, 2024-09-16T15:49:19.755 INFO:teuthology.orchestra.run.smithi151.stdout: "read_op_per_sec": 71, 2024-09-16T15:49:19.755 INFO:teuthology.orchestra.run.smithi151.stdout: "write_op_per_sec": 85 2024-09-16T15:49:19.755 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:49:19.755 INFO:teuthology.orchestra.run.smithi151.stdout: "fsmap": { 2024-09-16T15:49:19.755 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 57, 2024-09-16T15:49:19.755 INFO:teuthology.orchestra.run.smithi151.stdout: "btime": "2024-09-16T15:48:35:029906+0000", 2024-09-16T15:49:19.755 INFO:teuthology.orchestra.run.smithi151.stdout: "id": 6, 2024-09-16T15:49:19.755 INFO:teuthology.orchestra.run.smithi151.stdout: "up": 1, 2024-09-16T15:49:19.756 INFO:teuthology.orchestra.run.smithi151.stdout: "in": 1, 2024-09-16T15:49:19.756 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 1, 2024-09-16T15:49:19.756 INFO:teuthology.orchestra.run.smithi151.stdout: "by_rank": [ 2024-09-16T15:49:19.756 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:49:19.756 INFO:teuthology.orchestra.run.smithi151.stdout: "filesystem_id": 6, 2024-09-16T15:49:19.756 INFO:teuthology.orchestra.run.smithi151.stdout: "rank": 0, 2024-09-16T15:49:19.756 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "c", 2024-09-16T15:49:19.756 INFO:teuthology.orchestra.run.smithi151.stdout: "status": "up:active", 2024-09-16T15:49:19.756 INFO:teuthology.orchestra.run.smithi151.stdout: "gid": 6673 2024-09-16T15:49:19.756 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:49:19.756 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:49:19.756 INFO:teuthology.orchestra.run.smithi151.stdout: "up:standby": 3 2024-09-16T15:49:19.756 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:49:19.756 INFO:teuthology.orchestra.run.smithi151.stdout: "mgrmap": { 2024-09-16T15:49:19.757 INFO:teuthology.orchestra.run.smithi151.stdout: "available": true, 2024-09-16T15:49:19.757 INFO:teuthology.orchestra.run.smithi151.stdout: "num_standbys": 1, 2024-09-16T15:49:19.757 INFO:teuthology.orchestra.run.smithi151.stdout: "modules": [ 2024-09-16T15:49:19.757 INFO:teuthology.orchestra.run.smithi151.stdout: "iostat", 2024-09-16T15:49:19.757 INFO:teuthology.orchestra.run.smithi151.stdout: "nfs", 2024-09-16T15:49:19.757 INFO:teuthology.orchestra.run.smithi151.stdout: "restful" 2024-09-16T15:49:19.757 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:49:19.757 INFO:teuthology.orchestra.run.smithi151.stdout: "services": {} 2024-09-16T15:49:19.757 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:49:19.757 INFO:teuthology.orchestra.run.smithi151.stdout: "servicemap": { 2024-09-16T15:49:19.757 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 13, 2024-09-16T15:49:19.757 INFO:teuthology.orchestra.run.smithi151.stdout: "modified": "2024-09-16T15:48:01.742446+0000", 2024-09-16T15:49:19.757 INFO:teuthology.orchestra.run.smithi151.stdout: "services": {} 2024-09-16T15:49:19.757 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:49:19.758 INFO:teuthology.orchestra.run.smithi151.stdout: "progress_events": { 2024-09-16T15:49:19.758 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr-vol-ongoing-clones": { 2024-09-16T15:49:19.758 INFO:teuthology.orchestra.run.smithi151.stdout: "message": "2 ongoing clones - average progress is 37.76% (3s)\n [==========..................] (remaining: 6s)", 2024-09-16T15:49:19.758 INFO:teuthology.orchestra.run.smithi151.stdout: "progress": 0.37760001420974731, 2024-09-16T15:49:19.758 INFO:teuthology.orchestra.run.smithi151.stdout: "add_to_ceph_s": true 2024-09-16T15:49:19.758 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:49:19.758 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:49:19.758 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-16T15:49:19.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:19.753+0000 7f53b6c00640 1 -- 172.21.15.151:0/885690053 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f538c079db0 msgr2=0x7f538c07c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:19.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:19.753+0000 7f53b6c00640 1 --2- 172.21.15.151:0/885690053 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f538c079db0 0x7f538c07c270 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f53b006dc20 tx=0x7f5398006c90 comp rx=0 tx=0).stop 2024-09-16T15:49:19.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:19.753+0000 7f53b6c00640 1 -- 172.21.15.151:0/885690053 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53b00b4ae0 msgr2=0x7f53b006e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:19.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:19.753+0000 7f53b6c00640 1 --2- 172.21.15.151:0/885690053 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53b00b4ae0 0x7f53b006e540 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f53a0034040 tx=0x7f53a0003910 comp rx=0 tx=0).stop 2024-09-16T15:49:19.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:19.753+0000 7f53b6c00640 1 -- 172.21.15.151:0/885690053 shutdown_connections 2024-09-16T15:49:19.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:19.753+0000 7f53b6c00640 1 --2- 172.21.15.151:0/885690053 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f538c079db0 0x7f538c07c270 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:19.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:19.753+0000 7f53b6c00640 1 --2- 172.21.15.151:0/885690053 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53b01616e0 0x7f53b01f58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:19.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:19.753+0000 7f53b6c00640 1 --2- 172.21.15.151:0/885690053 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f53b00d1ee0 0x7f53b009e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:19.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:19.753+0000 7f53b6c00640 1 --2- 172.21.15.151:0/885690053 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f53b00b4ae0 0x7f53b006e540 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:19.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:19.753+0000 7f53b6c00640 1 -- 172.21.15.151:0/885690053 >> 172.21.15.151:0/885690053 conn(0x7f53b00d3fe0 msgr2=0x7f53b00b0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:49:19.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:19.754+0000 7f53b6c00640 1 -- 172.21.15.151:0/885690053 shutdown_connections 2024-09-16T15:49:19.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:19.754+0000 7f53b6c00640 1 -- 172.21.15.151:0/885690053 wait complete. 2024-09-16T15:49:20.783 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-16T15:49:21.054 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:21.053+0000 7f0c7da00640 1 -- 172.21.15.151:0/547205452 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c5c004800 msgr2=0x7f0c5c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:21.054 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:21.053+0000 7f0c7da00640 1 --2- 172.21.15.151:0/547205452 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c5c004800 0x7f0c5c004c80 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f0c6000c6e0 tx=0x7f0c5c005830 comp rx=0 tx=0).stop 2024-09-16T15:49:21.054 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:21.053+0000 7f0c7da00640 1 -- 172.21.15.151:0/547205452 shutdown_connections 2024-09-16T15:49:21.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:21.053+0000 7f0c7da00640 1 --2- 172.21.15.151:0/547205452 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c5c004800 0x7f0c5c004c80 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:21.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:21.053+0000 7f0c7da00640 1 -- 172.21.15.151:0/547205452 >> 172.21.15.151:0/547205452 conn(0x7f0c780b18e0 msgr2=0x7f0c780b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:49:21.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:21.053+0000 7f0c7da00640 1 -- 172.21.15.151:0/547205452 shutdown_connections 2024-09-16T15:49:21.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:21.054+0000 7f0c7da00640 1 -- 172.21.15.151:0/547205452 wait complete. 2024-09-16T15:49:21.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:21.055+0000 7f0c7da00640 1 Processor -- start 2024-09-16T15:49:21.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:21.056+0000 7f0c7da00640 1 -- start start 2024-09-16T15:49:21.058 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:21.057+0000 7f0c7da00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c5c004800 0x7f0c780ae400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:21.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:21.057+0000 7f0c7da00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c781f5e00 0x7f0c780cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:21.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:21.057+0000 7f0c7da00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c781522c0 0x7f0c7815b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:21.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:21.057+0000 7f0c7da00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0c780cd1e0 con 0x7f0c781522c0 2024-09-16T15:49:21.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:21.057+0000 7f0c7da00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0c780cd060 con 0x7f0c781f5e00 2024-09-16T15:49:21.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:21.057+0000 7f0c7da00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f0c780cd360 con 0x7f0c5c004800 2024-09-16T15:49:21.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:21.057+0000 7f0c77400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c781522c0 0x7f0c7815b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:21.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:21.058+0000 7f0c77400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c781522c0 0x7f0c7815b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:40092/0 (socket says 172.21.15.151:40092) 2024-09-16T15:49:21.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:21.058+0000 7f0c77400640 1 -- 172.21.15.151:0/2238686422 learned_addr learned my addr 172.21.15.151:0/2238686422 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:49:21.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:21.058+0000 7f0c6e000640 1 --2- 172.21.15.151:0/2238686422 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c781f5e00 0x7f0c780cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:21.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:21.058+0000 7f0c77400640 1 -- 172.21.15.151:0/2238686422 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c5c004800 msgr2=0x7f0c780ae400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:21.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:21.058+0000 7f0c76a00640 1 --2- 172.21.15.151:0/2238686422 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c5c004800 0x7f0c780ae400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:21.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:21.058+0000 7f0c77400640 1 --2- 172.21.15.151:0/2238686422 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c5c004800 0x7f0c780ae400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:21.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:21.058+0000 7f0c77400640 1 -- 172.21.15.151:0/2238686422 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c781f5e00 msgr2=0x7f0c780cb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:21.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:21.058+0000 7f0c77400640 1 --2- 172.21.15.151:0/2238686422 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c781f5e00 0x7f0c780cb6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:21.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:21.058+0000 7f0c77400640 1 -- 172.21.15.151:0/2238686422 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c780ae9e0 con 0x7f0c781522c0 2024-09-16T15:49:21.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:21.059+0000 7f0c76a00640 1 --2- 172.21.15.151:0/2238686422 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c5c004800 0x7f0c780ae400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:49:21.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:21.059+0000 7f0c77400640 1 --2- 172.21.15.151:0/2238686422 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c781522c0 0x7f0c7815b370 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f0c70001350 tx=0x7f0c7000dd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:49:21.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:21.059+0000 7f0c6f400640 1 -- 172.21.15.151:0/2238686422 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c70003da0 con 0x7f0c781522c0 2024-09-16T15:49:21.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:21.059+0000 7f0c6f400640 1 -- 172.21.15.151:0/2238686422 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0c7000b8f0 con 0x7f0c781522c0 2024-09-16T15:49:21.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:21.059+0000 7f0c7da00640 1 -- 172.21.15.151:0/2238686422 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c780aecc0 con 0x7f0c781522c0 2024-09-16T15:49:21.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:21.059+0000 7f0c6f400640 1 -- 172.21.15.151:0/2238686422 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c7000bce0 con 0x7f0c781522c0 2024-09-16T15:49:21.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:21.059+0000 7f0c7da00640 1 -- 172.21.15.151:0/2238686422 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c781f58e0 con 0x7f0c781522c0 2024-09-16T15:49:21.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:21.062+0000 7f0c7da00640 1 -- 172.21.15.151:0/2238686422 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c78069360 con 0x7f0c781522c0 2024-09-16T15:49:21.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:21.065+0000 7f0c6f400640 1 -- 172.21.15.151:0/2238686422 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 34) ==== 103225+0+0 (secure 0 0 0) 0x7f0c70006070 con 0x7f0c781522c0 2024-09-16T15:49:21.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:21.067+0000 7f0c6f400640 1 --2- 172.21.15.151:0/2238686422 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0c54079de0 0x7f0c5407c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:21.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:21.068+0000 7f0c6f400640 1 -- 172.21.15.151:0/2238686422 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7f0c70099b00 con 0x7f0c781522c0 2024-09-16T15:49:21.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:21.075+0000 7f0c6f400640 1 -- 172.21.15.151:0/2238686422 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0c70063bc0 con 0x7f0c781522c0 2024-09-16T15:49:21.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:21.085+0000 7f0c76a00640 1 --2- 172.21.15.151:0/2238686422 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0c54079de0 0x7f0c5407c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:21.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:21.086+0000 7f0c76a00640 1 --2- 172.21.15.151:0/2238686422 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0c54079de0 0x7f0c5407c2a0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f0c68005d00 tx=0x7f0c68005e70 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:49:21.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:21.581+0000 7f0c7da00640 1 -- 172.21.15.151:0/2238686422 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f0c781f53f0 con 0x7f0c781522c0 2024-09-16T15:49:21.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:21.592+0000 7f0c6f400640 1 -- 172.21.15.151:0/2238686422 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2634 (secure 0 0 0) 0x7f0c700634c0 con 0x7f0c781522c0 2024-09-16T15:49:21.594 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:49:21.594 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-16T15:49:21.594 INFO:teuthology.orchestra.run.smithi151.stdout: "fsid": "1757be18-e41a-4446-bb4f-ee4ac81c39df", 2024-09-16T15:49:21.594 INFO:teuthology.orchestra.run.smithi151.stdout: "health": { 2024-09-16T15:49:21.594 INFO:teuthology.orchestra.run.smithi151.stdout: "status": "HEALTH_OK", 2024-09-16T15:49:21.594 INFO:teuthology.orchestra.run.smithi151.stdout: "checks": {}, 2024-09-16T15:49:21.594 INFO:teuthology.orchestra.run.smithi151.stdout: "mutes": [] 2024-09-16T15:49:21.594 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:49:21.594 INFO:teuthology.orchestra.run.smithi151.stdout: "election_epoch": 4, 2024-09-16T15:49:21.594 INFO:teuthology.orchestra.run.smithi151.stdout: "quorum": [ 2024-09-16T15:49:21.594 INFO:teuthology.orchestra.run.smithi151.stdout: 0, 2024-09-16T15:49:21.594 INFO:teuthology.orchestra.run.smithi151.stdout: 1, 2024-09-16T15:49:21.594 INFO:teuthology.orchestra.run.smithi151.stdout: 2 2024-09-16T15:49:21.594 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:49:21.595 INFO:teuthology.orchestra.run.smithi151.stdout: "quorum_names": [ 2024-09-16T15:49:21.595 INFO:teuthology.orchestra.run.smithi151.stdout: "a", 2024-09-16T15:49:21.595 INFO:teuthology.orchestra.run.smithi151.stdout: "b", 2024-09-16T15:49:21.595 INFO:teuthology.orchestra.run.smithi151.stdout: "c" 2024-09-16T15:49:21.595 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:49:21.595 INFO:teuthology.orchestra.run.smithi151.stdout: "quorum_age": 625, 2024-09-16T15:49:21.595 INFO:teuthology.orchestra.run.smithi151.stdout: "monmap": { 2024-09-16T15:49:21.595 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 1, 2024-09-16T15:49:21.595 INFO:teuthology.orchestra.run.smithi151.stdout: "min_mon_release_name": "squid", 2024-09-16T15:49:21.595 INFO:teuthology.orchestra.run.smithi151.stdout: "num_mons": 3 2024-09-16T15:49:21.595 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:49:21.595 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap": { 2024-09-16T15:49:21.595 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 91, 2024-09-16T15:49:21.595 INFO:teuthology.orchestra.run.smithi151.stdout: "num_osds": 8, 2024-09-16T15:49:21.595 INFO:teuthology.orchestra.run.smithi151.stdout: "num_up_osds": 8, 2024-09-16T15:49:21.595 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_up_since": 1726501150, 2024-09-16T15:49:21.596 INFO:teuthology.orchestra.run.smithi151.stdout: "num_in_osds": 8, 2024-09-16T15:49:21.596 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_in_since": 1726501140, 2024-09-16T15:49:21.596 INFO:teuthology.orchestra.run.smithi151.stdout: "num_remapped_pgs": 0 2024-09-16T15:49:21.596 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:49:21.596 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap": { 2024-09-16T15:49:21.596 INFO:teuthology.orchestra.run.smithi151.stdout: "pgs_by_state": [ 2024-09-16T15:49:21.596 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:49:21.596 INFO:teuthology.orchestra.run.smithi151.stdout: "state_name": "active+clean", 2024-09-16T15:49:21.596 INFO:teuthology.orchestra.run.smithi151.stdout: "count": 97 2024-09-16T15:49:21.596 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:49:21.596 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:49:21.596 INFO:teuthology.orchestra.run.smithi151.stdout: "num_pgs": 97, 2024-09-16T15:49:21.596 INFO:teuthology.orchestra.run.smithi151.stdout: "num_pools": 3, 2024-09-16T15:49:21.597 INFO:teuthology.orchestra.run.smithi151.stdout: "num_objects": 1599, 2024-09-16T15:49:21.597 INFO:teuthology.orchestra.run.smithi151.stdout: "data_bytes": 6565249202, 2024-09-16T15:49:21.597 INFO:teuthology.orchestra.run.smithi151.stdout: "bytes_used": 13202341888, 2024-09-16T15:49:21.597 INFO:teuthology.orchestra.run.smithi151.stdout: "bytes_avail": 759891771392, 2024-09-16T15:49:21.597 INFO:teuthology.orchestra.run.smithi151.stdout: "bytes_total": 773094113280, 2024-09-16T15:49:21.597 INFO:teuthology.orchestra.run.smithi151.stdout: "read_bytes_sec": 294477978, 2024-09-16T15:49:21.597 INFO:teuthology.orchestra.run.smithi151.stdout: "write_bytes_sec": 278115331, 2024-09-16T15:49:21.597 INFO:teuthology.orchestra.run.smithi151.stdout: "read_op_per_sec": 71, 2024-09-16T15:49:21.597 INFO:teuthology.orchestra.run.smithi151.stdout: "write_op_per_sec": 77 2024-09-16T15:49:21.597 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:49:21.597 INFO:teuthology.orchestra.run.smithi151.stdout: "fsmap": { 2024-09-16T15:49:21.597 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 57, 2024-09-16T15:49:21.597 INFO:teuthology.orchestra.run.smithi151.stdout: "btime": "2024-09-16T15:48:35:029906+0000", 2024-09-16T15:49:21.597 INFO:teuthology.orchestra.run.smithi151.stdout: "id": 6, 2024-09-16T15:49:21.597 INFO:teuthology.orchestra.run.smithi151.stdout: "up": 1, 2024-09-16T15:49:21.598 INFO:teuthology.orchestra.run.smithi151.stdout: "in": 1, 2024-09-16T15:49:21.598 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 1, 2024-09-16T15:49:21.598 INFO:teuthology.orchestra.run.smithi151.stdout: "by_rank": [ 2024-09-16T15:49:21.598 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:49:21.598 INFO:teuthology.orchestra.run.smithi151.stdout: "filesystem_id": 6, 2024-09-16T15:49:21.598 INFO:teuthology.orchestra.run.smithi151.stdout: "rank": 0, 2024-09-16T15:49:21.598 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "c", 2024-09-16T15:49:21.598 INFO:teuthology.orchestra.run.smithi151.stdout: "status": "up:active", 2024-09-16T15:49:21.598 INFO:teuthology.orchestra.run.smithi151.stdout: "gid": 6673 2024-09-16T15:49:21.598 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:49:21.598 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:49:21.598 INFO:teuthology.orchestra.run.smithi151.stdout: "up:standby": 3 2024-09-16T15:49:21.598 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:49:21.598 INFO:teuthology.orchestra.run.smithi151.stdout: "mgrmap": { 2024-09-16T15:49:21.598 INFO:teuthology.orchestra.run.smithi151.stdout: "available": true, 2024-09-16T15:49:21.599 INFO:teuthology.orchestra.run.smithi151.stdout: "num_standbys": 1, 2024-09-16T15:49:21.599 INFO:teuthology.orchestra.run.smithi151.stdout: "modules": [ 2024-09-16T15:49:21.599 INFO:teuthology.orchestra.run.smithi151.stdout: "iostat", 2024-09-16T15:49:21.599 INFO:teuthology.orchestra.run.smithi151.stdout: "nfs", 2024-09-16T15:49:21.599 INFO:teuthology.orchestra.run.smithi151.stdout: "restful" 2024-09-16T15:49:21.599 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:49:21.599 INFO:teuthology.orchestra.run.smithi151.stdout: "services": {} 2024-09-16T15:49:21.599 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:49:21.599 INFO:teuthology.orchestra.run.smithi151.stdout: "servicemap": { 2024-09-16T15:49:21.599 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 14, 2024-09-16T15:49:21.599 INFO:teuthology.orchestra.run.smithi151.stdout: "modified": "2024-09-16T15:49:19.782478+0000", 2024-09-16T15:49:21.599 INFO:teuthology.orchestra.run.smithi151.stdout: "services": { 2024-09-16T15:49:21.599 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": { 2024-09-16T15:49:21.599 INFO:teuthology.orchestra.run.smithi151.stdout: "daemons": { 2024-09-16T15:49:21.599 INFO:teuthology.orchestra.run.smithi151.stdout: "summary": "", 2024-09-16T15:49:21.600 INFO:teuthology.orchestra.run.smithi151.stdout: "c": { 2024-09-16T15:49:21.600 INFO:teuthology.orchestra.run.smithi151.stdout: "start_epoch": 0, 2024-09-16T15:49:21.600 INFO:teuthology.orchestra.run.smithi151.stdout: "start_stamp": "0.000000", 2024-09-16T15:49:21.600 INFO:teuthology.orchestra.run.smithi151.stdout: "gid": 0, 2024-09-16T15:49:21.600 INFO:teuthology.orchestra.run.smithi151.stdout: "addr": "(unrecognized address family 0)/0", 2024-09-16T15:49:21.600 INFO:teuthology.orchestra.run.smithi151.stdout: "metadata": {}, 2024-09-16T15:49:21.600 INFO:teuthology.orchestra.run.smithi151.stdout: "task_status": {} 2024-09-16T15:49:21.600 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:49:21.600 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:49:21.600 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:49:21.600 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:49:21.600 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:49:21.600 INFO:teuthology.orchestra.run.smithi151.stdout: "progress_events": { 2024-09-16T15:49:21.600 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr-vol-ongoing-clones": { 2024-09-16T15:49:21.600 INFO:teuthology.orchestra.run.smithi151.stdout: "message": "2 ongoing clones - average progress is 37.76% (3s)\n [==========..................] (remaining: 6s)", 2024-09-16T15:49:21.601 INFO:teuthology.orchestra.run.smithi151.stdout: "progress": 0.37760001420974731, 2024-09-16T15:49:21.601 INFO:teuthology.orchestra.run.smithi151.stdout: "add_to_ceph_s": true 2024-09-16T15:49:21.601 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:49:21.601 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:49:21.601 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-16T15:49:21.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:21.598+0000 7f0c7da00640 1 -- 172.21.15.151:0/2238686422 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0c54079de0 msgr2=0x7f0c5407c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:21.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:21.598+0000 7f0c7da00640 1 --2- 172.21.15.151:0/2238686422 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0c54079de0 0x7f0c5407c2a0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f0c68005d00 tx=0x7f0c68005e70 comp rx=0 tx=0).stop 2024-09-16T15:49:21.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:21.599+0000 7f0c7da00640 1 -- 172.21.15.151:0/2238686422 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c781522c0 msgr2=0x7f0c7815b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:21.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:21.599+0000 7f0c7da00640 1 --2- 172.21.15.151:0/2238686422 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c781522c0 0x7f0c7815b370 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f0c70001350 tx=0x7f0c7000dd80 comp rx=0 tx=0).stop 2024-09-16T15:49:21.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:21.599+0000 7f0c7da00640 1 -- 172.21.15.151:0/2238686422 shutdown_connections 2024-09-16T15:49:21.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:21.599+0000 7f0c7da00640 1 --2- 172.21.15.151:0/2238686422 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0c54079de0 0x7f0c5407c2a0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:21.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:21.599+0000 7f0c7da00640 1 --2- 172.21.15.151:0/2238686422 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c781f5e00 0x7f0c780cb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:21.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:21.599+0000 7f0c7da00640 1 --2- 172.21.15.151:0/2238686422 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c781522c0 0x7f0c7815b370 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:21.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:21.599+0000 7f0c7da00640 1 --2- 172.21.15.151:0/2238686422 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c5c004800 0x7f0c780ae400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:21.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:21.599+0000 7f0c7da00640 1 -- 172.21.15.151:0/2238686422 >> 172.21.15.151:0/2238686422 conn(0x7f0c780b18e0 msgr2=0x7f0c780b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:49:21.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:21.600+0000 7f0c7da00640 1 -- 172.21.15.151:0/2238686422 shutdown_connections 2024-09-16T15:49:21.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:21.600+0000 7f0c7da00640 1 -- 172.21.15.151:0/2238686422 wait complete. 2024-09-16T15:49:22.624 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-16T15:49:22.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:22.905+0000 7f76a6600640 1 -- 172.21.15.151:0/1235224270 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f76a01512b0 msgr2=0x7f76a006fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:22.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:22.905+0000 7f76a6600640 1 --2- 172.21.15.151:0/1235224270 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f76a01512b0 0x7f76a006fa00 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f76a0151690 tx=0x7f769002b900 comp rx=0 tx=0).stop 2024-09-16T15:49:22.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:22.905+0000 7f76a6600640 1 -- 172.21.15.151:0/1235224270 shutdown_connections 2024-09-16T15:49:22.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:22.905+0000 7f76a6600640 1 --2- 172.21.15.151:0/1235224270 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f76a00b3e20 0x7f76a00cc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:22.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:22.905+0000 7f76a6600640 1 --2- 172.21.15.151:0/1235224270 >> v2:172.21.15.167:3300/0 conn(0x7f76a00708b0 0x7f76a006ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:22.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:22.905+0000 7f76a6600640 1 --2- 172.21.15.151:0/1235224270 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f76a01512b0 0x7f76a006fa00 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:22.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:22.905+0000 7f76a6600640 1 -- 172.21.15.151:0/1235224270 >> 172.21.15.151:0/1235224270 conn(0x7f76a00b18e0 msgr2=0x7f76a00b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:49:22.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:22.906+0000 7f76a6600640 1 -- 172.21.15.151:0/1235224270 shutdown_connections 2024-09-16T15:49:22.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:22.906+0000 7f76a6600640 1 -- 172.21.15.151:0/1235224270 wait complete. 2024-09-16T15:49:22.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:22.908+0000 7f76a6600640 1 Processor -- start 2024-09-16T15:49:22.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:22.908+0000 7f76a6600640 1 -- start start 2024-09-16T15:49:22.912 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:22.910+0000 7f76a6600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f76a00708b0 0x7f76a01f37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:22.912 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:22.910+0000 7f76a6600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76a00b3e20 0x7f76a015df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:22.912 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:22.910+0000 7f76a6600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f76a01512b0 0x7f76a01f62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:22.912 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:22.910+0000 7f76a6600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f76a00cd020 con 0x7f76a00708b0 2024-09-16T15:49:22.912 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:22.910+0000 7f76a6600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f76a00ccea0 con 0x7f76a00b3e20 2024-09-16T15:49:22.912 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:22.910+0000 7f76a6600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f76a00cefb0 con 0x7f76a01512b0 2024-09-16T15:49:22.912 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:22.910+0000 7f769f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f76a00708b0 0x7f76a01f37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:22.912 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:22.910+0000 7f769f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f76a00708b0 0x7f76a01f37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52872/0 (socket says 172.21.15.151:52872) 2024-09-16T15:49:22.912 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:22.910+0000 7f769f400640 1 -- 172.21.15.151:0/487988739 learned_addr learned my addr 172.21.15.151:0/487988739 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:49:22.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:22.911+0000 7f769f400640 1 -- 172.21.15.151:0/487988739 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f76a01512b0 msgr2=0x7f76a01f62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:22.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:22.911+0000 7f769f400640 1 --2- 172.21.15.151:0/487988739 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f76a01512b0 0x7f76a01f62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:22.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:22.911+0000 7f769f400640 1 -- 172.21.15.151:0/487988739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76a00b3e20 msgr2=0x7f76a015df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:22.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:22.911+0000 7f769f400640 1 --2- 172.21.15.151:0/487988739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76a00b3e20 0x7f76a015df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:22.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:22.911+0000 7f769f400640 1 -- 172.21.15.151:0/487988739 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76a015e440 con 0x7f76a00708b0 2024-09-16T15:49:22.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:22.911+0000 7f769f400640 1 --2- 172.21.15.151:0/487988739 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f76a00708b0 0x7f76a01f37c0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f7690000f80 tx=0x7f7690003f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:49:22.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:22.912+0000 7f7697400640 1 -- 172.21.15.151:0/487988739 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7690035050 con 0x7f76a00708b0 2024-09-16T15:49:22.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:22.912+0000 7f7697400640 1 -- 172.21.15.151:0/487988739 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f76900045c0 con 0x7f76a00708b0 2024-09-16T15:49:22.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:22.912+0000 7f7697400640 1 -- 172.21.15.151:0/487988739 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7690045480 con 0x7f76a00708b0 2024-09-16T15:49:22.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:22.912+0000 7f76a6600640 1 -- 172.21.15.151:0/487988739 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76a01f6d40 con 0x7f76a00708b0 2024-09-16T15:49:22.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:22.912+0000 7f76a6600640 1 -- 172.21.15.151:0/487988739 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76a01f2fc0 con 0x7f76a00708b0 2024-09-16T15:49:22.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:22.914+0000 7f76a6600640 1 -- 172.21.15.151:0/487988739 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76a00cc340 con 0x7f76a00708b0 2024-09-16T15:49:22.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:22.917+0000 7f7697400640 1 -- 172.21.15.151:0/487988739 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 34) ==== 103225+0+0 (secure 0 0 0) 0x7f7690038020 con 0x7f76a00708b0 2024-09-16T15:49:22.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:22.918+0000 7f7697400640 1 --2- 172.21.15.151:0/487988739 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7680079e30 0x7f768007c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:22.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:22.918+0000 7f7697400640 1 -- 172.21.15.151:0/487988739 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7f76900bb060 con 0x7f76a00708b0 2024-09-16T15:49:22.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:22.924+0000 7f7697400640 1 -- 172.21.15.151:0/487988739 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f769004a060 con 0x7f76a00708b0 2024-09-16T15:49:22.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:22.964+0000 7f769ea00640 1 --2- 172.21.15.151:0/487988739 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7680079e30 0x7f768007c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:23.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:23.048+0000 7f769ea00640 1 --2- 172.21.15.151:0/487988739 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7680079e30 0x7f768007c2f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f768c005fd0 tx=0x7f768c005850 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:49:23.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:23.402+0000 7f76a6600640 1 -- 172.21.15.151:0/487988739 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f76a006f700 con 0x7f76a00708b0 2024-09-16T15:49:23.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:23.403+0000 7f7697400640 1 -- 172.21.15.151:0/487988739 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2638 (secure 0 0 0) 0x7f7690084d60 con 0x7f76a00708b0 2024-09-16T15:49:23.405 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:49:23.406 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-16T15:49:23.406 INFO:teuthology.orchestra.run.smithi151.stdout: "fsid": "1757be18-e41a-4446-bb4f-ee4ac81c39df", 2024-09-16T15:49:23.406 INFO:teuthology.orchestra.run.smithi151.stdout: "health": { 2024-09-16T15:49:23.406 INFO:teuthology.orchestra.run.smithi151.stdout: "status": "HEALTH_OK", 2024-09-16T15:49:23.406 INFO:teuthology.orchestra.run.smithi151.stdout: "checks": {}, 2024-09-16T15:49:23.406 INFO:teuthology.orchestra.run.smithi151.stdout: "mutes": [] 2024-09-16T15:49:23.406 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:49:23.406 INFO:teuthology.orchestra.run.smithi151.stdout: "election_epoch": 4, 2024-09-16T15:49:23.406 INFO:teuthology.orchestra.run.smithi151.stdout: "quorum": [ 2024-09-16T15:49:23.406 INFO:teuthology.orchestra.run.smithi151.stdout: 0, 2024-09-16T15:49:23.406 INFO:teuthology.orchestra.run.smithi151.stdout: 1, 2024-09-16T15:49:23.406 INFO:teuthology.orchestra.run.smithi151.stdout: 2 2024-09-16T15:49:23.406 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:49:23.407 INFO:teuthology.orchestra.run.smithi151.stdout: "quorum_names": [ 2024-09-16T15:49:23.407 INFO:teuthology.orchestra.run.smithi151.stdout: "a", 2024-09-16T15:49:23.407 INFO:teuthology.orchestra.run.smithi151.stdout: "b", 2024-09-16T15:49:23.407 INFO:teuthology.orchestra.run.smithi151.stdout: "c" 2024-09-16T15:49:23.407 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:49:23.407 INFO:teuthology.orchestra.run.smithi151.stdout: "quorum_age": 627, 2024-09-16T15:49:23.407 INFO:teuthology.orchestra.run.smithi151.stdout: "monmap": { 2024-09-16T15:49:23.407 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 1, 2024-09-16T15:49:23.407 INFO:teuthology.orchestra.run.smithi151.stdout: "min_mon_release_name": "squid", 2024-09-16T15:49:23.407 INFO:teuthology.orchestra.run.smithi151.stdout: "num_mons": 3 2024-09-16T15:49:23.407 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:49:23.407 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap": { 2024-09-16T15:49:23.407 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 91, 2024-09-16T15:49:23.407 INFO:teuthology.orchestra.run.smithi151.stdout: "num_osds": 8, 2024-09-16T15:49:23.408 INFO:teuthology.orchestra.run.smithi151.stdout: "num_up_osds": 8, 2024-09-16T15:49:23.408 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_up_since": 1726501150, 2024-09-16T15:49:23.408 INFO:teuthology.orchestra.run.smithi151.stdout: "num_in_osds": 8, 2024-09-16T15:49:23.408 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_in_since": 1726501140, 2024-09-16T15:49:23.408 INFO:teuthology.orchestra.run.smithi151.stdout: "num_remapped_pgs": 0 2024-09-16T15:49:23.408 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:49:23.408 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap": { 2024-09-16T15:49:23.408 INFO:teuthology.orchestra.run.smithi151.stdout: "pgs_by_state": [ 2024-09-16T15:49:23.408 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:49:23.408 INFO:teuthology.orchestra.run.smithi151.stdout: "state_name": "active+clean", 2024-09-16T15:49:23.408 INFO:teuthology.orchestra.run.smithi151.stdout: "count": 97 2024-09-16T15:49:23.408 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:49:23.408 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:49:23.408 INFO:teuthology.orchestra.run.smithi151.stdout: "num_pgs": 97, 2024-09-16T15:49:23.409 INFO:teuthology.orchestra.run.smithi151.stdout: "num_pools": 3, 2024-09-16T15:49:23.409 INFO:teuthology.orchestra.run.smithi151.stdout: "num_objects": 2450, 2024-09-16T15:49:23.409 INFO:teuthology.orchestra.run.smithi151.stdout: "data_bytes": 10139003637, 2024-09-16T15:49:23.409 INFO:teuthology.orchestra.run.smithi151.stdout: "bytes_used": 20324343808, 2024-09-16T15:49:23.409 INFO:teuthology.orchestra.run.smithi151.stdout: "bytes_avail": 752769769472, 2024-09-16T15:49:23.409 INFO:teuthology.orchestra.run.smithi151.stdout: "bytes_total": 773094113280, 2024-09-16T15:49:23.409 INFO:teuthology.orchestra.run.smithi151.stdout: "read_bytes_sec": 589965740, 2024-09-16T15:49:23.409 INFO:teuthology.orchestra.run.smithi151.stdout: "write_bytes_sec": 575367522, 2024-09-16T15:49:23.409 INFO:teuthology.orchestra.run.smithi151.stdout: "read_op_per_sec": 142, 2024-09-16T15:49:23.409 INFO:teuthology.orchestra.run.smithi151.stdout: "write_op_per_sec": 155 2024-09-16T15:49:23.409 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:49:23.409 INFO:teuthology.orchestra.run.smithi151.stdout: "fsmap": { 2024-09-16T15:49:23.409 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 57, 2024-09-16T15:49:23.410 INFO:teuthology.orchestra.run.smithi151.stdout: "btime": "2024-09-16T15:48:35:029906+0000", 2024-09-16T15:49:23.410 INFO:teuthology.orchestra.run.smithi151.stdout: "id": 6, 2024-09-16T15:49:23.410 INFO:teuthology.orchestra.run.smithi151.stdout: "up": 1, 2024-09-16T15:49:23.410 INFO:teuthology.orchestra.run.smithi151.stdout: "in": 1, 2024-09-16T15:49:23.410 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 1, 2024-09-16T15:49:23.410 INFO:teuthology.orchestra.run.smithi151.stdout: "by_rank": [ 2024-09-16T15:49:23.410 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:49:23.410 INFO:teuthology.orchestra.run.smithi151.stdout: "filesystem_id": 6, 2024-09-16T15:49:23.410 INFO:teuthology.orchestra.run.smithi151.stdout: "rank": 0, 2024-09-16T15:49:23.410 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "c", 2024-09-16T15:49:23.410 INFO:teuthology.orchestra.run.smithi151.stdout: "status": "up:active", 2024-09-16T15:49:23.410 INFO:teuthology.orchestra.run.smithi151.stdout: "gid": 6673 2024-09-16T15:49:23.410 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:49:23.410 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:49:23.411 INFO:teuthology.orchestra.run.smithi151.stdout: "up:standby": 3 2024-09-16T15:49:23.411 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:49:23.411 INFO:teuthology.orchestra.run.smithi151.stdout: "mgrmap": { 2024-09-16T15:49:23.411 INFO:teuthology.orchestra.run.smithi151.stdout: "available": true, 2024-09-16T15:49:23.411 INFO:teuthology.orchestra.run.smithi151.stdout: "num_standbys": 1, 2024-09-16T15:49:23.411 INFO:teuthology.orchestra.run.smithi151.stdout: "modules": [ 2024-09-16T15:49:23.411 INFO:teuthology.orchestra.run.smithi151.stdout: "iostat", 2024-09-16T15:49:23.411 INFO:teuthology.orchestra.run.smithi151.stdout: "nfs", 2024-09-16T15:49:23.411 INFO:teuthology.orchestra.run.smithi151.stdout: "restful" 2024-09-16T15:49:23.411 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:49:23.411 INFO:teuthology.orchestra.run.smithi151.stdout: "services": {} 2024-09-16T15:49:23.411 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:49:23.411 INFO:teuthology.orchestra.run.smithi151.stdout: "servicemap": { 2024-09-16T15:49:23.411 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 14, 2024-09-16T15:49:23.412 INFO:teuthology.orchestra.run.smithi151.stdout: "modified": "2024-09-16T15:49:19.782478+0000", 2024-09-16T15:49:23.412 INFO:teuthology.orchestra.run.smithi151.stdout: "services": { 2024-09-16T15:49:23.412 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": { 2024-09-16T15:49:23.412 INFO:teuthology.orchestra.run.smithi151.stdout: "daemons": { 2024-09-16T15:49:23.412 INFO:teuthology.orchestra.run.smithi151.stdout: "summary": "", 2024-09-16T15:49:23.412 INFO:teuthology.orchestra.run.smithi151.stdout: "c": { 2024-09-16T15:49:23.412 INFO:teuthology.orchestra.run.smithi151.stdout: "start_epoch": 0, 2024-09-16T15:49:23.412 INFO:teuthology.orchestra.run.smithi151.stdout: "start_stamp": "0.000000", 2024-09-16T15:49:23.412 INFO:teuthology.orchestra.run.smithi151.stdout: "gid": 0, 2024-09-16T15:49:23.412 INFO:teuthology.orchestra.run.smithi151.stdout: "addr": "(unrecognized address family 0)/0", 2024-09-16T15:49:23.412 INFO:teuthology.orchestra.run.smithi151.stdout: "metadata": {}, 2024-09-16T15:49:23.412 INFO:teuthology.orchestra.run.smithi151.stdout: "task_status": {} 2024-09-16T15:49:23.412 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:49:23.413 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:49:23.413 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:49:23.413 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:49:23.413 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:49:23.413 INFO:teuthology.orchestra.run.smithi151.stdout: "progress_events": { 2024-09-16T15:49:23.413 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr-vol-ongoing-clones": { 2024-09-16T15:49:23.413 INFO:teuthology.orchestra.run.smithi151.stdout: "message": "4 ongoing clones - average progress is 47.266% (8s)\n [=============...............] (remaining: 9s)", 2024-09-16T15:49:23.413 INFO:teuthology.orchestra.run.smithi151.stdout: "progress": 0.47266000509262085, 2024-09-16T15:49:23.413 INFO:teuthology.orchestra.run.smithi151.stdout: "add_to_ceph_s": true 2024-09-16T15:49:23.413 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:49:23.413 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:49:23.413 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-16T15:49:23.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:23.409+0000 7f76a6600640 1 -- 172.21.15.151:0/487988739 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7680079e30 msgr2=0x7f768007c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:23.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:23.409+0000 7f76a6600640 1 --2- 172.21.15.151:0/487988739 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7680079e30 0x7f768007c2f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f768c005fd0 tx=0x7f768c005850 comp rx=0 tx=0).stop 2024-09-16T15:49:23.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:23.409+0000 7f76a6600640 1 -- 172.21.15.151:0/487988739 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f76a00708b0 msgr2=0x7f76a01f37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:23.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:23.409+0000 7f76a6600640 1 --2- 172.21.15.151:0/487988739 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f76a00708b0 0x7f76a01f37c0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f7690000f80 tx=0x7f7690003f30 comp rx=0 tx=0).stop 2024-09-16T15:49:23.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:23.411+0000 7f76a6600640 1 -- 172.21.15.151:0/487988739 shutdown_connections 2024-09-16T15:49:23.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:23.411+0000 7f76a6600640 1 --2- 172.21.15.151:0/487988739 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7680079e30 0x7f768007c2f0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:23.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:23.411+0000 7f76a6600640 1 --2- 172.21.15.151:0/487988739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76a00b3e20 0x7f76a015df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:23.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:23.411+0000 7f76a6600640 1 --2- 172.21.15.151:0/487988739 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f76a01512b0 0x7f76a01f62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:23.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:23.411+0000 7f76a6600640 1 --2- 172.21.15.151:0/487988739 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f76a00708b0 0x7f76a01f37c0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:23.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:23.411+0000 7f76a6600640 1 -- 172.21.15.151:0/487988739 >> 172.21.15.151:0/487988739 conn(0x7f76a00b18e0 msgr2=0x7f76a00b0440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:49:23.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:23.417+0000 7f76a6600640 1 -- 172.21.15.151:0/487988739 shutdown_connections 2024-09-16T15:49:23.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:23.417+0000 7f76a6600640 1 -- 172.21.15.151:0/487988739 wait complete. 2024-09-16T15:49:24.458 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-16T15:49:24.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:24.714+0000 7f1880e00640 1 -- 172.21.15.151:0/2234250281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f187c0708d0 msgr2=0x7f187c06eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:24.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:24.714+0000 7f1880e00640 1 --2- 172.21.15.151:0/2234250281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f187c0708d0 0x7f187c06eba0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f187c069720 tx=0x7f187402a830 comp rx=0 tx=0).stop 2024-09-16T15:49:24.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:24.714+0000 7f1880e00640 1 -- 172.21.15.151:0/2234250281 shutdown_connections 2024-09-16T15:49:24.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:24.714+0000 7f1880e00640 1 --2- 172.21.15.151:0/2234250281 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f187c0708d0 0x7f187c06eba0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:24.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:24.714+0000 7f1880e00640 1 --2- 172.21.15.151:0/2234250281 >> v2:172.21.15.151:3300/0 conn(0x7f187c0ccc50 0x7f187c15afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:24.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:24.714+0000 7f1880e00640 1 -- 172.21.15.151:0/2234250281 >> 172.21.15.151:0/2234250281 conn(0x7f187c0b18e0 msgr2=0x7f187c0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:49:24.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:24.714+0000 7f1880e00640 1 -- 172.21.15.151:0/2234250281 shutdown_connections 2024-09-16T15:49:24.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:24.714+0000 7f1880e00640 1 -- 172.21.15.151:0/2234250281 wait complete. 2024-09-16T15:49:24.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:24.717+0000 7f1880e00640 1 Processor -- start 2024-09-16T15:49:24.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:24.717+0000 7f1880e00640 1 -- start start 2024-09-16T15:49:24.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:24.719+0000 7f1880e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f187c0708d0 0x7f187c14c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:24.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:24.719+0000 7f1880e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f187c0ccc50 0x7f187c14bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:24.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:24.719+0000 7f1880e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f187c09aa00 0x7f187c09aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:24.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:24.719+0000 7f1880e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f187c06f0e0 con 0x7f187c0ccc50 2024-09-16T15:49:24.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:24.719+0000 7f1880e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f187c0b4200 con 0x7f187c09aa00 2024-09-16T15:49:24.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:24.719+0000 7f1880e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f187c06f260 con 0x7f187c0708d0 2024-09-16T15:49:24.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:24.719+0000 7f187b400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f187c0708d0 0x7f187c14c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:24.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:24.719+0000 7f187b400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f187c0708d0 0x7f187c14c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.151:45744/0 (socket says 172.21.15.151:45744) 2024-09-16T15:49:24.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:24.719+0000 7f187b400640 1 -- 172.21.15.151:0/274212150 learned_addr learned my addr 172.21.15.151:0/274212150 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:49:24.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:24.720+0000 7f187be00640 1 --2- 172.21.15.151:0/274212150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f187c09aa00 0x7f187c09aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:24.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:24.720+0000 7f187aa00640 1 --2- 172.21.15.151:0/274212150 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f187c0ccc50 0x7f187c14bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:24.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:24.720+0000 7f187b400640 1 -- 172.21.15.151:0/274212150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f187c09aa00 msgr2=0x7f187c09aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:24.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:24.720+0000 7f187b400640 1 --2- 172.21.15.151:0/274212150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f187c09aa00 0x7f187c09aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:24.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:24.720+0000 7f187b400640 1 -- 172.21.15.151:0/274212150 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f187c0ccc50 msgr2=0x7f187c14bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:24.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:24.720+0000 7f187b400640 1 --2- 172.21.15.151:0/274212150 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f187c0ccc50 0x7f187c14bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:24.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:24.720+0000 7f187b400640 1 -- 172.21.15.151:0/274212150 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f187c1520c0 con 0x7f187c0708d0 2024-09-16T15:49:24.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:24.721+0000 7f187b400640 1 --2- 172.21.15.151:0/274212150 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f187c0708d0 0x7f187c14c340 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f186c00c370 tx=0x7f186c00c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:49:24.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:24.721+0000 7f1873400640 1 -- 172.21.15.151:0/274212150 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f186c006890 con 0x7f187c0708d0 2024-09-16T15:49:24.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:24.721+0000 7f1880e00640 1 -- 172.21.15.151:0/274212150 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f187c09b780 con 0x7f187c0708d0 2024-09-16T15:49:24.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:24.721+0000 7f1873400640 1 -- 172.21.15.151:0/274212150 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f186c006b20 con 0x7f187c0708d0 2024-09-16T15:49:24.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:24.721+0000 7f1880e00640 1 -- 172.21.15.151:0/274212150 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f187c152360 con 0x7f187c0708d0 2024-09-16T15:49:24.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:24.721+0000 7f1873400640 1 -- 172.21.15.151:0/274212150 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f186c007020 con 0x7f187c0708d0 2024-09-16T15:49:24.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:24.723+0000 7f1880e00640 1 -- 172.21.15.151:0/274212150 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f187c06eba0 con 0x7f187c0708d0 2024-09-16T15:49:24.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:24.724+0000 7f1873400640 1 -- 172.21.15.151:0/274212150 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 34) ==== 103225+0+0 (secure 0 0 0) 0x7f186c019020 con 0x7f187c0708d0 2024-09-16T15:49:24.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:24.725+0000 7f1873400640 1 --2- 172.21.15.151:0/274212150 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f185c079de0 0x7f185c07c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:24.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:24.725+0000 7f1873400640 1 -- 172.21.15.151:0/274212150 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7f186c099140 con 0x7f187c0708d0 2024-09-16T15:49:24.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:24.726+0000 7f187aa00640 1 --2- 172.21.15.151:0/274212150 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f185c079de0 0x7f185c07c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:24.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:24.727+0000 7f187aa00640 1 --2- 172.21.15.151:0/274212150 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f185c079de0 0x7f185c07c2a0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f187c0d0bb0 tx=0x7f1868005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:49:24.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:24.734+0000 7f1873400640 1 -- 172.21.15.151:0/274212150 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f186c062d80 con 0x7f187c0708d0 2024-09-16T15:49:25.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:25.196+0000 7f1880e00640 1 -- 172.21.15.151:0/274212150 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f187c0cd220 con 0x7f187c0708d0 2024-09-16T15:49:25.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:25.199+0000 7f1873400640 1 -- 172.21.15.151:0/274212150 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2638 (secure 0 0 0) 0x7f186c062680 con 0x7f187c0708d0 2024-09-16T15:49:25.201 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:49:25.201 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-16T15:49:25.201 INFO:teuthology.orchestra.run.smithi151.stdout: "fsid": "1757be18-e41a-4446-bb4f-ee4ac81c39df", 2024-09-16T15:49:25.201 INFO:teuthology.orchestra.run.smithi151.stdout: "health": { 2024-09-16T15:49:25.201 INFO:teuthology.orchestra.run.smithi151.stdout: "status": "HEALTH_OK", 2024-09-16T15:49:25.201 INFO:teuthology.orchestra.run.smithi151.stdout: "checks": {}, 2024-09-16T15:49:25.201 INFO:teuthology.orchestra.run.smithi151.stdout: "mutes": [] 2024-09-16T15:49:25.201 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:49:25.201 INFO:teuthology.orchestra.run.smithi151.stdout: "election_epoch": 4, 2024-09-16T15:49:25.202 INFO:teuthology.orchestra.run.smithi151.stdout: "quorum": [ 2024-09-16T15:49:25.202 INFO:teuthology.orchestra.run.smithi151.stdout: 0, 2024-09-16T15:49:25.202 INFO:teuthology.orchestra.run.smithi151.stdout: 1, 2024-09-16T15:49:25.202 INFO:teuthology.orchestra.run.smithi151.stdout: 2 2024-09-16T15:49:25.202 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:49:25.202 INFO:teuthology.orchestra.run.smithi151.stdout: "quorum_names": [ 2024-09-16T15:49:25.202 INFO:teuthology.orchestra.run.smithi151.stdout: "a", 2024-09-16T15:49:25.202 INFO:teuthology.orchestra.run.smithi151.stdout: "b", 2024-09-16T15:49:25.202 INFO:teuthology.orchestra.run.smithi151.stdout: "c" 2024-09-16T15:49:25.202 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:49:25.202 INFO:teuthology.orchestra.run.smithi151.stdout: "quorum_age": 628, 2024-09-16T15:49:25.202 INFO:teuthology.orchestra.run.smithi151.stdout: "monmap": { 2024-09-16T15:49:25.202 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 1, 2024-09-16T15:49:25.202 INFO:teuthology.orchestra.run.smithi151.stdout: "min_mon_release_name": "squid", 2024-09-16T15:49:25.202 INFO:teuthology.orchestra.run.smithi151.stdout: "num_mons": 3 2024-09-16T15:49:25.203 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:49:25.203 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap": { 2024-09-16T15:49:25.203 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 91, 2024-09-16T15:49:25.203 INFO:teuthology.orchestra.run.smithi151.stdout: "num_osds": 8, 2024-09-16T15:49:25.203 INFO:teuthology.orchestra.run.smithi151.stdout: "num_up_osds": 8, 2024-09-16T15:49:25.203 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_up_since": 1726501150, 2024-09-16T15:49:25.203 INFO:teuthology.orchestra.run.smithi151.stdout: "num_in_osds": 8, 2024-09-16T15:49:25.203 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_in_since": 1726501140, 2024-09-16T15:49:25.203 INFO:teuthology.orchestra.run.smithi151.stdout: "num_remapped_pgs": 0 2024-09-16T15:49:25.203 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:49:25.203 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap": { 2024-09-16T15:49:25.203 INFO:teuthology.orchestra.run.smithi151.stdout: "pgs_by_state": [ 2024-09-16T15:49:25.203 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:49:25.203 INFO:teuthology.orchestra.run.smithi151.stdout: "state_name": "active+clean", 2024-09-16T15:49:25.203 INFO:teuthology.orchestra.run.smithi151.stdout: "count": 97 2024-09-16T15:49:25.204 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:49:25.204 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:49:25.204 INFO:teuthology.orchestra.run.smithi151.stdout: "num_pgs": 97, 2024-09-16T15:49:25.204 INFO:teuthology.orchestra.run.smithi151.stdout: "num_pools": 3, 2024-09-16T15:49:25.204 INFO:teuthology.orchestra.run.smithi151.stdout: "num_objects": 2450, 2024-09-16T15:49:25.204 INFO:teuthology.orchestra.run.smithi151.stdout: "data_bytes": 10139003637, 2024-09-16T15:49:25.204 INFO:teuthology.orchestra.run.smithi151.stdout: "bytes_used": 20324343808, 2024-09-16T15:49:25.204 INFO:teuthology.orchestra.run.smithi151.stdout: "bytes_avail": 752769769472, 2024-09-16T15:49:25.204 INFO:teuthology.orchestra.run.smithi151.stdout: "bytes_total": 773094113280, 2024-09-16T15:49:25.204 INFO:teuthology.orchestra.run.smithi151.stdout: "read_bytes_sec": 585937753, 2024-09-16T15:49:25.204 INFO:teuthology.orchestra.run.smithi151.stdout: "write_bytes_sec": 574115226, 2024-09-16T15:49:25.204 INFO:teuthology.orchestra.run.smithi151.stdout: "read_op_per_sec": 141, 2024-09-16T15:49:25.204 INFO:teuthology.orchestra.run.smithi151.stdout: "write_op_per_sec": 152 2024-09-16T15:49:25.204 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:49:25.204 INFO:teuthology.orchestra.run.smithi151.stdout: "fsmap": { 2024-09-16T15:49:25.205 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 57, 2024-09-16T15:49:25.205 INFO:teuthology.orchestra.run.smithi151.stdout: "btime": "2024-09-16T15:48:35:029906+0000", 2024-09-16T15:49:25.205 INFO:teuthology.orchestra.run.smithi151.stdout: "id": 6, 2024-09-16T15:49:25.205 INFO:teuthology.orchestra.run.smithi151.stdout: "up": 1, 2024-09-16T15:49:25.205 INFO:teuthology.orchestra.run.smithi151.stdout: "in": 1, 2024-09-16T15:49:25.205 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 1, 2024-09-16T15:49:25.205 INFO:teuthology.orchestra.run.smithi151.stdout: "by_rank": [ 2024-09-16T15:49:25.205 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:49:25.205 INFO:teuthology.orchestra.run.smithi151.stdout: "filesystem_id": 6, 2024-09-16T15:49:25.205 INFO:teuthology.orchestra.run.smithi151.stdout: "rank": 0, 2024-09-16T15:49:25.205 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "c", 2024-09-16T15:49:25.205 INFO:teuthology.orchestra.run.smithi151.stdout: "status": "up:active", 2024-09-16T15:49:25.205 INFO:teuthology.orchestra.run.smithi151.stdout: "gid": 6673 2024-09-16T15:49:25.205 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:49:25.206 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:49:25.206 INFO:teuthology.orchestra.run.smithi151.stdout: "up:standby": 3 2024-09-16T15:49:25.206 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:49:25.206 INFO:teuthology.orchestra.run.smithi151.stdout: "mgrmap": { 2024-09-16T15:49:25.206 INFO:teuthology.orchestra.run.smithi151.stdout: "available": true, 2024-09-16T15:49:25.206 INFO:teuthology.orchestra.run.smithi151.stdout: "num_standbys": 1, 2024-09-16T15:49:25.206 INFO:teuthology.orchestra.run.smithi151.stdout: "modules": [ 2024-09-16T15:49:25.206 INFO:teuthology.orchestra.run.smithi151.stdout: "iostat", 2024-09-16T15:49:25.206 INFO:teuthology.orchestra.run.smithi151.stdout: "nfs", 2024-09-16T15:49:25.206 INFO:teuthology.orchestra.run.smithi151.stdout: "restful" 2024-09-16T15:49:25.206 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:49:25.206 INFO:teuthology.orchestra.run.smithi151.stdout: "services": {} 2024-09-16T15:49:25.206 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:49:25.206 INFO:teuthology.orchestra.run.smithi151.stdout: "servicemap": { 2024-09-16T15:49:25.206 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 14, 2024-09-16T15:49:25.207 INFO:teuthology.orchestra.run.smithi151.stdout: "modified": "2024-09-16T15:49:19.782478+0000", 2024-09-16T15:49:25.207 INFO:teuthology.orchestra.run.smithi151.stdout: "services": { 2024-09-16T15:49:25.207 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": { 2024-09-16T15:49:25.207 INFO:teuthology.orchestra.run.smithi151.stdout: "daemons": { 2024-09-16T15:49:25.207 INFO:teuthology.orchestra.run.smithi151.stdout: "summary": "", 2024-09-16T15:49:25.207 INFO:teuthology.orchestra.run.smithi151.stdout: "c": { 2024-09-16T15:49:25.207 INFO:teuthology.orchestra.run.smithi151.stdout: "start_epoch": 0, 2024-09-16T15:49:25.207 INFO:teuthology.orchestra.run.smithi151.stdout: "start_stamp": "0.000000", 2024-09-16T15:49:25.207 INFO:teuthology.orchestra.run.smithi151.stdout: "gid": 0, 2024-09-16T15:49:25.207 INFO:teuthology.orchestra.run.smithi151.stdout: "addr": "(unrecognized address family 0)/0", 2024-09-16T15:49:25.207 INFO:teuthology.orchestra.run.smithi151.stdout: "metadata": {}, 2024-09-16T15:49:25.207 INFO:teuthology.orchestra.run.smithi151.stdout: "task_status": {} 2024-09-16T15:49:25.207 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:49:25.208 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:49:25.208 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:49:25.208 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:49:25.208 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:49:25.208 INFO:teuthology.orchestra.run.smithi151.stdout: "progress_events": { 2024-09-16T15:49:25.208 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr-vol-ongoing-clones": { 2024-09-16T15:49:25.208 INFO:teuthology.orchestra.run.smithi151.stdout: "message": "4 ongoing clones - average progress is 47.266% (8s)\n [=============...............] (remaining: 9s)", 2024-09-16T15:49:25.208 INFO:teuthology.orchestra.run.smithi151.stdout: "progress": 0.47266000509262085, 2024-09-16T15:49:25.208 INFO:teuthology.orchestra.run.smithi151.stdout: "add_to_ceph_s": true 2024-09-16T15:49:25.208 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:49:25.208 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:49:25.208 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-16T15:49:25.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:25.204+0000 7f1880e00640 1 -- 172.21.15.151:0/274212150 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f185c079de0 msgr2=0x7f185c07c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:25.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:25.204+0000 7f1880e00640 1 --2- 172.21.15.151:0/274212150 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f185c079de0 0x7f185c07c2a0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f187c0d0bb0 tx=0x7f1868005f90 comp rx=0 tx=0).stop 2024-09-16T15:49:25.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:25.204+0000 7f1880e00640 1 -- 172.21.15.151:0/274212150 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f187c0708d0 msgr2=0x7f187c14c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:25.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:25.204+0000 7f1880e00640 1 --2- 172.21.15.151:0/274212150 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f187c0708d0 0x7f187c14c340 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f186c00c370 tx=0x7f186c00c830 comp rx=0 tx=0).stop 2024-09-16T15:49:25.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:25.204+0000 7f1880e00640 1 -- 172.21.15.151:0/274212150 shutdown_connections 2024-09-16T15:49:25.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:25.204+0000 7f1880e00640 1 --2- 172.21.15.151:0/274212150 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f185c079de0 0x7f185c07c2a0 secure :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f187c0d0bb0 tx=0x7f1868005f90 comp rx=0 tx=0).stop 2024-09-16T15:49:25.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:25.204+0000 7f1880e00640 1 --2- 172.21.15.151:0/274212150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f187c09aa00 0x7f187c09aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:25.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:25.204+0000 7f1880e00640 1 --2- 172.21.15.151:0/274212150 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f187c0ccc50 0x7f187c14bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:25.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:25.204+0000 7f1880e00640 1 --2- 172.21.15.151:0/274212150 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f187c0708d0 0x7f187c14c340 secure :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f186c00c370 tx=0x7f186c00c830 comp rx=0 tx=0).stop 2024-09-16T15:49:25.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:25.204+0000 7f1880e00640 1 -- 172.21.15.151:0/274212150 >> 172.21.15.151:0/274212150 conn(0x7f187c0b18e0 msgr2=0x7f187c0aed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:49:25.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:25.205+0000 7f1880e00640 1 -- 172.21.15.151:0/274212150 shutdown_connections 2024-09-16T15:49:25.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:25.205+0000 7f1880e00640 1 -- 172.21.15.151:0/274212150 wait complete. 2024-09-16T15:49:26.247 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-16T15:49:26.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:26.550+0000 7fc18e400640 1 -- 172.21.15.151:0/1082904203 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1880d1b70 msgr2=0x7fc1880d2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:26.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:26.550+0000 7fc18e400640 1 --2- 172.21.15.151:0/1082904203 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1880d1b70 0x7fc1880d2730 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fc188069740 tx=0x7fc17402a530 comp rx=0 tx=0).stop 2024-09-16T15:49:26.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:26.550+0000 7fc18e400640 1 -- 172.21.15.151:0/1082904203 shutdown_connections 2024-09-16T15:49:26.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:26.550+0000 7fc18e400640 1 --2- 172.21.15.151:0/1082904203 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1880d1b70 0x7fc1880d2730 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:26.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:26.550+0000 7fc18e400640 1 -- 172.21.15.151:0/1082904203 >> 172.21.15.151:0/1082904203 conn(0x7fc1880d3fa0 msgr2=0x7fc1880d43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:49:26.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:26.551+0000 7fc18e400640 1 -- 172.21.15.151:0/1082904203 shutdown_connections 2024-09-16T15:49:26.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:26.551+0000 7fc18e400640 1 -- 172.21.15.151:0/1082904203 wait complete. 2024-09-16T15:49:26.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:26.553+0000 7fc18e400640 1 Processor -- start 2024-09-16T15:49:26.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:26.554+0000 7fc18e400640 1 -- start start 2024-09-16T15:49:26.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:26.555+0000 7fc18e400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1880b7720 0x7fc1880cc590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:26.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:26.555+0000 7fc18e400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1880d1b70 0x7fc188154310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:26.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:26.555+0000 7fc18e400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc18815e1b0 0x7fc1880c9820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:26.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:26.555+0000 7fc18e400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc1880b83c0 con 0x7fc1880b7720 2024-09-16T15:49:26.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:26.555+0000 7fc18e400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc1880b8240 con 0x7fc1880d1b70 2024-09-16T15:49:26.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:26.555+0000 7fc18e400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fc1880b8540 con 0x7fc18815e1b0 2024-09-16T15:49:26.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:26.555+0000 7fc187400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1880b7720 0x7fc1880cc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:26.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:26.555+0000 7fc187400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1880b7720 0x7fc1880cc590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52928/0 (socket says 172.21.15.151:52928) 2024-09-16T15:49:26.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:26.555+0000 7fc187400640 1 -- 172.21.15.151:0/3044559442 learned_addr learned my addr 172.21.15.151:0/3044559442 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:49:26.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:26.556+0000 7fc187400640 1 -- 172.21.15.151:0/3044559442 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc18815e1b0 msgr2=0x7fc1880c9820 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T15:49:26.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:26.556+0000 7fc187400640 1 --2- 172.21.15.151:0/3044559442 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc18815e1b0 0x7fc1880c9820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:26.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:26.556+0000 7fc187400640 1 -- 172.21.15.151:0/3044559442 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1880d1b70 msgr2=0x7fc188154310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:26.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:26.556+0000 7fc187400640 1 --2- 172.21.15.151:0/3044559442 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1880d1b70 0x7fc188154310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:26.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:26.556+0000 7fc187400640 1 -- 172.21.15.151:0/3044559442 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1881f8590 con 0x7fc1880b7720 2024-09-16T15:49:26.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:26.556+0000 7fc187400640 1 --2- 172.21.15.151:0/3044559442 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1880b7720 0x7fc1880cc590 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fc17800b410 tx=0x7fc17800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:49:26.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:26.557+0000 7fc17f400640 1 -- 172.21.15.151:0/3044559442 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc178015020 con 0x7fc1880b7720 2024-09-16T15:49:26.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:26.557+0000 7fc17f400640 1 -- 172.21.15.151:0/3044559442 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc178011920 con 0x7fc1880b7720 2024-09-16T15:49:26.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:26.557+0000 7fc17f400640 1 -- 172.21.15.151:0/3044559442 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc178011d70 con 0x7fc1880b7720 2024-09-16T15:49:26.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:26.557+0000 7fc18e400640 1 -- 172.21.15.151:0/3044559442 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1881f8870 con 0x7fc1880b7720 2024-09-16T15:49:26.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:26.558+0000 7fc18e400640 1 -- 172.21.15.151:0/3044559442 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1881f8e30 con 0x7fc1880b7720 2024-09-16T15:49:26.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:26.559+0000 7fc18e400640 1 -- 172.21.15.151:0/3044559442 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1880b7b30 con 0x7fc1880b7720 2024-09-16T15:49:26.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:26.560+0000 7fc17f400640 1 -- 172.21.15.151:0/3044559442 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 34) ==== 103225+0+0 (secure 0 0 0) 0x7fc178018020 con 0x7fc1880b7720 2024-09-16T15:49:26.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:26.562+0000 7fc17f400640 1 --2- 172.21.15.151:0/3044559442 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc168079db0 0x7fc16807c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:26.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:26.562+0000 7fc17f400640 1 -- 172.21.15.151:0/3044559442 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7fc178099a20 con 0x7fc1880b7720 2024-09-16T15:49:26.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:26.566+0000 7fc17f400640 1 -- 172.21.15.151:0/3044559442 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc17801f020 con 0x7fc1880b7720 2024-09-16T15:49:26.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:26.581+0000 7fc186a00640 1 --2- 172.21.15.151:0/3044559442 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc168079db0 0x7fc16807c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:26.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:26.588+0000 7fc186a00640 1 --2- 172.21.15.151:0/3044559442 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc168079db0 0x7fc16807c270 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc174033000 tx=0x7fc1740046c0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:49:27.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:27.027+0000 7fc18e400640 1 -- 172.21.15.151:0/3044559442 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fc1880d1210 con 0x7fc1880b7720 2024-09-16T15:49:27.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:27.028+0000 7fc17f400640 1 -- 172.21.15.151:0/3044559442 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2879 (secure 0 0 0) 0x7fc1780636e0 con 0x7fc1880b7720 2024-09-16T15:49:27.031 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:49:27.031 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-16T15:49:27.031 INFO:teuthology.orchestra.run.smithi151.stdout: "fsid": "1757be18-e41a-4446-bb4f-ee4ac81c39df", 2024-09-16T15:49:27.031 INFO:teuthology.orchestra.run.smithi151.stdout: "health": { 2024-09-16T15:49:27.031 INFO:teuthology.orchestra.run.smithi151.stdout: "status": "HEALTH_OK", 2024-09-16T15:49:27.031 INFO:teuthology.orchestra.run.smithi151.stdout: "checks": {}, 2024-09-16T15:49:27.031 INFO:teuthology.orchestra.run.smithi151.stdout: "mutes": [] 2024-09-16T15:49:27.031 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:49:27.031 INFO:teuthology.orchestra.run.smithi151.stdout: "election_epoch": 4, 2024-09-16T15:49:27.032 INFO:teuthology.orchestra.run.smithi151.stdout: "quorum": [ 2024-09-16T15:49:27.032 INFO:teuthology.orchestra.run.smithi151.stdout: 0, 2024-09-16T15:49:27.032 INFO:teuthology.orchestra.run.smithi151.stdout: 1, 2024-09-16T15:49:27.032 INFO:teuthology.orchestra.run.smithi151.stdout: 2 2024-09-16T15:49:27.032 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:49:27.032 INFO:teuthology.orchestra.run.smithi151.stdout: "quorum_names": [ 2024-09-16T15:49:27.032 INFO:teuthology.orchestra.run.smithi151.stdout: "a", 2024-09-16T15:49:27.032 INFO:teuthology.orchestra.run.smithi151.stdout: "b", 2024-09-16T15:49:27.032 INFO:teuthology.orchestra.run.smithi151.stdout: "c" 2024-09-16T15:49:27.032 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:49:27.032 INFO:teuthology.orchestra.run.smithi151.stdout: "quorum_age": 630, 2024-09-16T15:49:27.032 INFO:teuthology.orchestra.run.smithi151.stdout: "monmap": { 2024-09-16T15:49:27.032 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 1, 2024-09-16T15:49:27.032 INFO:teuthology.orchestra.run.smithi151.stdout: "min_mon_release_name": "squid", 2024-09-16T15:49:27.033 INFO:teuthology.orchestra.run.smithi151.stdout: "num_mons": 3 2024-09-16T15:49:27.033 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:49:27.033 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap": { 2024-09-16T15:49:27.033 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 91, 2024-09-16T15:49:27.033 INFO:teuthology.orchestra.run.smithi151.stdout: "num_osds": 8, 2024-09-16T15:49:27.033 INFO:teuthology.orchestra.run.smithi151.stdout: "num_up_osds": 8, 2024-09-16T15:49:27.033 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_up_since": 1726501150, 2024-09-16T15:49:27.033 INFO:teuthology.orchestra.run.smithi151.stdout: "num_in_osds": 8, 2024-09-16T15:49:27.033 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_in_since": 1726501140, 2024-09-16T15:49:27.033 INFO:teuthology.orchestra.run.smithi151.stdout: "num_remapped_pgs": 0 2024-09-16T15:49:27.033 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:49:27.033 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap": { 2024-09-16T15:49:27.033 INFO:teuthology.orchestra.run.smithi151.stdout: "pgs_by_state": [ 2024-09-16T15:49:27.033 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:49:27.033 INFO:teuthology.orchestra.run.smithi151.stdout: "state_name": "active+clean", 2024-09-16T15:49:27.033 INFO:teuthology.orchestra.run.smithi151.stdout: "count": 97 2024-09-16T15:49:27.034 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:49:27.034 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:49:27.034 INFO:teuthology.orchestra.run.smithi151.stdout: "num_pgs": 97, 2024-09-16T15:49:27.034 INFO:teuthology.orchestra.run.smithi151.stdout: "num_pools": 3, 2024-09-16T15:49:27.034 INFO:teuthology.orchestra.run.smithi151.stdout: "num_objects": 2561, 2024-09-16T15:49:27.034 INFO:teuthology.orchestra.run.smithi151.stdout: "data_bytes": 10604571381, 2024-09-16T15:49:27.034 INFO:teuthology.orchestra.run.smithi151.stdout: "bytes_used": 21091901440, 2024-09-16T15:49:27.034 INFO:teuthology.orchestra.run.smithi151.stdout: "bytes_avail": 752002211840, 2024-09-16T15:49:27.034 INFO:teuthology.orchestra.run.smithi151.stdout: "bytes_total": 773094113280, 2024-09-16T15:49:27.034 INFO:teuthology.orchestra.run.smithi151.stdout: "read_bytes_sec": 625084865, 2024-09-16T15:49:27.034 INFO:teuthology.orchestra.run.smithi151.stdout: "write_bytes_sec": 612912162, 2024-09-16T15:49:27.034 INFO:teuthology.orchestra.run.smithi151.stdout: "read_op_per_sec": 150, 2024-09-16T15:49:27.034 INFO:teuthology.orchestra.run.smithi151.stdout: "write_op_per_sec": 161 2024-09-16T15:49:27.034 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:49:27.034 INFO:teuthology.orchestra.run.smithi151.stdout: "fsmap": { 2024-09-16T15:49:27.035 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 57, 2024-09-16T15:49:27.035 INFO:teuthology.orchestra.run.smithi151.stdout: "btime": "2024-09-16T15:48:35:029906+0000", 2024-09-16T15:49:27.035 INFO:teuthology.orchestra.run.smithi151.stdout: "id": 6, 2024-09-16T15:49:27.035 INFO:teuthology.orchestra.run.smithi151.stdout: "up": 1, 2024-09-16T15:49:27.035 INFO:teuthology.orchestra.run.smithi151.stdout: "in": 1, 2024-09-16T15:49:27.035 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 1, 2024-09-16T15:49:27.035 INFO:teuthology.orchestra.run.smithi151.stdout: "by_rank": [ 2024-09-16T15:49:27.035 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:49:27.035 INFO:teuthology.orchestra.run.smithi151.stdout: "filesystem_id": 6, 2024-09-16T15:49:27.035 INFO:teuthology.orchestra.run.smithi151.stdout: "rank": 0, 2024-09-16T15:49:27.035 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "c", 2024-09-16T15:49:27.035 INFO:teuthology.orchestra.run.smithi151.stdout: "status": "up:active", 2024-09-16T15:49:27.035 INFO:teuthology.orchestra.run.smithi151.stdout: "gid": 6673 2024-09-16T15:49:27.035 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:49:27.035 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:49:27.035 INFO:teuthology.orchestra.run.smithi151.stdout: "up:standby": 3 2024-09-16T15:49:27.036 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:49:27.036 INFO:teuthology.orchestra.run.smithi151.stdout: "mgrmap": { 2024-09-16T15:49:27.036 INFO:teuthology.orchestra.run.smithi151.stdout: "available": true, 2024-09-16T15:49:27.036 INFO:teuthology.orchestra.run.smithi151.stdout: "num_standbys": 1, 2024-09-16T15:49:27.036 INFO:teuthology.orchestra.run.smithi151.stdout: "modules": [ 2024-09-16T15:49:27.036 INFO:teuthology.orchestra.run.smithi151.stdout: "iostat", 2024-09-16T15:49:27.036 INFO:teuthology.orchestra.run.smithi151.stdout: "nfs", 2024-09-16T15:49:27.036 INFO:teuthology.orchestra.run.smithi151.stdout: "restful" 2024-09-16T15:49:27.036 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:49:27.036 INFO:teuthology.orchestra.run.smithi151.stdout: "services": {} 2024-09-16T15:49:27.036 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:49:27.036 INFO:teuthology.orchestra.run.smithi151.stdout: "servicemap": { 2024-09-16T15:49:27.036 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 14, 2024-09-16T15:49:27.036 INFO:teuthology.orchestra.run.smithi151.stdout: "modified": "2024-09-16T15:49:19.782478+0000", 2024-09-16T15:49:27.036 INFO:teuthology.orchestra.run.smithi151.stdout: "services": { 2024-09-16T15:49:27.037 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": { 2024-09-16T15:49:27.037 INFO:teuthology.orchestra.run.smithi151.stdout: "daemons": { 2024-09-16T15:49:27.037 INFO:teuthology.orchestra.run.smithi151.stdout: "summary": "", 2024-09-16T15:49:27.037 INFO:teuthology.orchestra.run.smithi151.stdout: "c": { 2024-09-16T15:49:27.037 INFO:teuthology.orchestra.run.smithi151.stdout: "start_epoch": 0, 2024-09-16T15:49:27.037 INFO:teuthology.orchestra.run.smithi151.stdout: "start_stamp": "0.000000", 2024-09-16T15:49:27.037 INFO:teuthology.orchestra.run.smithi151.stdout: "gid": 0, 2024-09-16T15:49:27.037 INFO:teuthology.orchestra.run.smithi151.stdout: "addr": "(unrecognized address family 0)/0", 2024-09-16T15:49:27.037 INFO:teuthology.orchestra.run.smithi151.stdout: "metadata": {}, 2024-09-16T15:49:27.037 INFO:teuthology.orchestra.run.smithi151.stdout: "task_status": {} 2024-09-16T15:49:27.037 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:49:27.037 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:49:27.037 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:49:27.037 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:49:27.037 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:49:27.037 INFO:teuthology.orchestra.run.smithi151.stdout: "progress_events": { 2024-09-16T15:49:27.038 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr-vol-ongoing-clones": { 2024-09-16T15:49:27.038 INFO:teuthology.orchestra.run.smithi151.stdout: "message": "4 ongoing clones - average progress is 44.922% (13s)\n [============................] (remaining: 16s)", 2024-09-16T15:49:27.038 INFO:teuthology.orchestra.run.smithi151.stdout: "progress": 0.44922000169754028, 2024-09-16T15:49:27.038 INFO:teuthology.orchestra.run.smithi151.stdout: "add_to_ceph_s": true 2024-09-16T15:49:27.038 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:49:27.038 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr-vol-total-clones": { 2024-09-16T15:49:27.038 INFO:teuthology.orchestra.run.smithi151.stdout: "message": "Total 5 clones - average progress is 55.937% (0s)\n [===============.............] ", 2024-09-16T15:49:27.038 INFO:teuthology.orchestra.run.smithi151.stdout: "progress": 0.55936998128890991, 2024-09-16T15:49:27.038 INFO:teuthology.orchestra.run.smithi151.stdout: "add_to_ceph_s": true 2024-09-16T15:49:27.038 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:49:27.038 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:49:27.038 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-16T15:49:27.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:27.036+0000 7fc18e400640 1 -- 172.21.15.151:0/3044559442 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc168079db0 msgr2=0x7fc16807c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:27.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:27.036+0000 7fc18e400640 1 --2- 172.21.15.151:0/3044559442 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc168079db0 0x7fc16807c270 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc174033000 tx=0x7fc1740046c0 comp rx=0 tx=0).stop 2024-09-16T15:49:27.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:27.036+0000 7fc18e400640 1 -- 172.21.15.151:0/3044559442 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1880b7720 msgr2=0x7fc1880cc590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:27.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:27.036+0000 7fc18e400640 1 --2- 172.21.15.151:0/3044559442 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1880b7720 0x7fc1880cc590 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fc17800b410 tx=0x7fc17800b8d0 comp rx=0 tx=0).stop 2024-09-16T15:49:27.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:27.037+0000 7fc18e400640 1 -- 172.21.15.151:0/3044559442 shutdown_connections 2024-09-16T15:49:27.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:27.037+0000 7fc18e400640 1 --2- 172.21.15.151:0/3044559442 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc168079db0 0x7fc16807c270 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:27.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:27.037+0000 7fc18e400640 1 --2- 172.21.15.151:0/3044559442 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1880d1b70 0x7fc188154310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:27.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:27.037+0000 7fc18e400640 1 --2- 172.21.15.151:0/3044559442 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc18815e1b0 0x7fc1880c9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:27.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:27.037+0000 7fc18e400640 1 --2- 172.21.15.151:0/3044559442 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1880b7720 0x7fc1880cc590 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:27.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:27.037+0000 7fc18e400640 1 -- 172.21.15.151:0/3044559442 >> 172.21.15.151:0/3044559442 conn(0x7fc1880d3fa0 msgr2=0x7fc1880d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:49:27.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:27.037+0000 7fc18e400640 1 -- 172.21.15.151:0/3044559442 shutdown_connections 2024-09-16T15:49:27.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:27.038+0000 7fc18e400640 1 -- 172.21.15.151:0/3044559442 wait complete. 2024-09-16T15:49:27.066 INFO:tasks.cephfs.test_volumes:pevs - {'mgr-vol-ongoing-clones': {'message': '4 ongoing clones - average progress is 44.922% (13s)\n [============................] (remaining: 16s)', 'progress': 0.4492200016975403, 'add_to_ceph_s': True}, 'mgr-vol-total-clones': {'message': 'Total 5 clones - average progress is 55.937% (0s)\n [===============.............] ', 'progress': 0.5593699812889099, 'add_to_ceph_s': True}} 2024-09-16T15:49:27.067 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone77745 2024-09-16T15:49:27.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:27.365+0000 7fb6d2000640 1 -- 172.21.15.151:0/27499163 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb6cc06efa0 msgr2=0x7fb6cc0cc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:27.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:27.365+0000 7fb6d2000640 1 --2- 172.21.15.151:0/27499163 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb6cc06efa0 0x7fb6cc0cc5c0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc0cc550 tx=0x7fb6b402a4d0 comp rx=0 tx=0).stop 2024-09-16T15:49:27.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:27.365+0000 7fb6d2000640 1 -- 172.21.15.151:0/27499163 shutdown_connections 2024-09-16T15:49:27.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:27.365+0000 7fb6d2000640 1 --2- 172.21.15.151:0/27499163 >> v2:172.21.15.167:3300/0 conn(0x7fb6cc0ccb00 0x7fb6cc15f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:27.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:27.365+0000 7fb6d2000640 1 --2- 172.21.15.151:0/27499163 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb6cc06efa0 0x7fb6cc0cc5c0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:27.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:27.365+0000 7fb6d2000640 1 --2- 172.21.15.151:0/27499163 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb6cc145750 0x7fb6cc073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:27.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:27.365+0000 7fb6d2000640 1 -- 172.21.15.151:0/27499163 >> 172.21.15.151:0/27499163 conn(0x7fb6cc0b1840 msgr2=0x7fb6cc0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:49:27.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:27.365+0000 7fb6d2000640 1 -- 172.21.15.151:0/27499163 shutdown_connections 2024-09-16T15:49:27.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:27.365+0000 7fb6d2000640 1 -- 172.21.15.151:0/27499163 wait complete. 2024-09-16T15:49:27.368 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:27.367+0000 7fb6d2000640 1 Processor -- start 2024-09-16T15:49:27.368 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:27.367+0000 7fb6d2000640 1 -- start start 2024-09-16T15:49:27.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:27.369+0000 7fb6d2000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6cc06efa0 0x7fb6cc1f4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:27.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:27.369+0000 7fb6d2000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb6cc0ccb00 0x7fb6cc15e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:27.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:27.369+0000 7fb6d2000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb6cc145750 0x7fb6cc143750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:27.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:27.369+0000 7fb6d2000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb6cc1602c0 con 0x7fb6cc145750 2024-09-16T15:49:27.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:27.369+0000 7fb6d2000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb6cc160140 con 0x7fb6cc06efa0 2024-09-16T15:49:27.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:27.369+0000 7fb6d2000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fb6cc160440 con 0x7fb6cc0ccb00 2024-09-16T15:49:27.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:27.369+0000 7fb6cbe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb6cc145750 0x7fb6cc143750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:27.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:27.369+0000 7fb6c3e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb6cc0ccb00 0x7fb6cc15e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:27.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:27.369+0000 7fb6cbe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb6cc145750 0x7fb6cc143750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52950/0 (socket says 172.21.15.151:52950) 2024-09-16T15:49:27.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:27.369+0000 7fb6cbe00640 1 -- 172.21.15.151:0/2808375615 learned_addr learned my addr 172.21.15.151:0/2808375615 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:49:27.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:27.370+0000 7fb6c3e00640 1 -- 172.21.15.151:0/2808375615 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6cc06efa0 msgr2=0x7fb6cc1f4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:27.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:27.370+0000 7fb6c3e00640 1 --2- 172.21.15.151:0/2808375615 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6cc06efa0 0x7fb6cc1f4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:27.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:27.370+0000 7fb6c3e00640 1 -- 172.21.15.151:0/2808375615 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb6cc145750 msgr2=0x7fb6cc143750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:27.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:27.370+0000 7fb6c3e00640 1 --2- 172.21.15.151:0/2808375615 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb6cc145750 0x7fb6cc143750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:27.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:27.370+0000 7fb6c3e00640 1 -- 172.21.15.151:0/2808375615 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6cc0b3170 con 0x7fb6cc0ccb00 2024-09-16T15:49:27.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:27.371+0000 7fb6c3e00640 1 --2- 172.21.15.151:0/2808375615 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb6cc0ccb00 0x7fb6cc15e670 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb6b4031000 tx=0x7fb6b4003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:49:27.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:27.372+0000 7fb6c3400640 1 -- 172.21.15.151:0/2808375615 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b403f070 con 0x7fb6cc0ccb00 2024-09-16T15:49:27.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:27.372+0000 7fb6c3400640 1 -- 172.21.15.151:0/2808375615 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb6b4004a90 con 0x7fb6cc0ccb00 2024-09-16T15:49:27.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:27.372+0000 7fb6c3400640 1 -- 172.21.15.151:0/2808375615 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b4035030 con 0x7fb6cc0ccb00 2024-09-16T15:49:27.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:27.373+0000 7fb6d2000640 1 -- 172.21.15.151:0/2808375615 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6cc0b3450 con 0x7fb6cc0ccb00 2024-09-16T15:49:27.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:27.373+0000 7fb6d2000640 1 -- 172.21.15.151:0/2808375615 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6cc0b3a10 con 0x7fb6cc0ccb00 2024-09-16T15:49:27.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:27.374+0000 7fb6d2000640 1 -- 172.21.15.151:0/2808375615 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6cc15f9a0 con 0x7fb6cc0ccb00 2024-09-16T15:49:27.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:27.377+0000 7fb6c3400640 1 -- 172.21.15.151:0/2808375615 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 34) ==== 103225+0+0 (secure 0 0 0) 0x7fb6b4042070 con 0x7fb6cc0ccb00 2024-09-16T15:49:27.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:27.378+0000 7fb6c3400640 1 --2- 172.21.15.151:0/2808375615 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb6a8079db0 0x7fb6a807c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:27.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:27.378+0000 7fb6c3400640 1 -- 172.21.15.151:0/2808375615 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7fb6b40badc0 con 0x7fb6cc0ccb00 2024-09-16T15:49:27.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:27.382+0000 7fb6cb400640 1 --2- 172.21.15.151:0/2808375615 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb6a8079db0 0x7fb6a807c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:27.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:27.389+0000 7fb6c3400640 1 -- 172.21.15.151:0/2808375615 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb6b4084430 con 0x7fb6cc0ccb00 2024-09-16T15:49:27.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:27.472+0000 7fb6cb400640 1 --2- 172.21.15.151:0/2808375615 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb6a8079db0 0x7fb6a807c270 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc009670 tx=0x7fb6bc006bf0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:49:27.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:27.824+0000 7fb6d2000640 1 -- 172.21.15.151:0/2808375615 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone77745", "target": ["mon-mgr", ""]}) -- 0x7fb6cc069a10 con 0x7fb6a8079db0 2024-09-16T15:49:28.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:28.811+0000 7fb6c3400640 1 -- 172.21.15.151:0/2808375615 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb6cc069a10 con 0x7fb6a8079db0 2024-09-16T15:49:28.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:28.815+0000 7fb6d2000640 1 -- 172.21.15.151:0/2808375615 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb6a8079db0 msgr2=0x7fb6a807c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:28.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:28.815+0000 7fb6d2000640 1 --2- 172.21.15.151:0/2808375615 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb6a8079db0 0x7fb6a807c270 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc009670 tx=0x7fb6bc006bf0 comp rx=0 tx=0).stop 2024-09-16T15:49:28.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:28.815+0000 7fb6d2000640 1 -- 172.21.15.151:0/2808375615 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb6cc0ccb00 msgr2=0x7fb6cc15e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:28.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:28.815+0000 7fb6d2000640 1 --2- 172.21.15.151:0/2808375615 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb6cc0ccb00 0x7fb6cc15e670 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb6b4031000 tx=0x7fb6b4003b90 comp rx=0 tx=0).stop 2024-09-16T15:49:28.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:28.815+0000 7fb6d2000640 1 -- 172.21.15.151:0/2808375615 shutdown_connections 2024-09-16T15:49:28.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:28.815+0000 7fb6d2000640 1 --2- 172.21.15.151:0/2808375615 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb6a8079db0 0x7fb6a807c270 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:28.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:28.815+0000 7fb6d2000640 1 --2- 172.21.15.151:0/2808375615 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb6cc145750 0x7fb6cc143750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:28.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:28.815+0000 7fb6d2000640 1 --2- 172.21.15.151:0/2808375615 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb6cc0ccb00 0x7fb6cc15e670 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:28.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:28.815+0000 7fb6d2000640 1 --2- 172.21.15.151:0/2808375615 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6cc06efa0 0x7fb6cc1f4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:28.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:28.815+0000 7fb6d2000640 1 -- 172.21.15.151:0/2808375615 >> 172.21.15.151:0/2808375615 conn(0x7fb6cc0b1840 msgr2=0x7fb6cc06f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:49:28.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:28.815+0000 7fb6d2000640 1 -- 172.21.15.151:0/2808375615 shutdown_connections 2024-09-16T15:49:28.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:28.815+0000 7fb6d2000640 1 -- 172.21.15.151:0/2808375615 wait complete. 2024-09-16T15:49:28.845 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone22126 2024-09-16T15:49:29.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:29.155+0000 7f2070800640 1 -- 172.21.15.151:0/173150239 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f206806efa0 msgr2=0x7f2068073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:29.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:29.155+0000 7f2070800640 1 --2- 172.21.15.151:0/173150239 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f206806efa0 0x7f2068073440 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f2058005fd0 tx=0x7f205802a4a0 comp rx=0 tx=0).stop 2024-09-16T15:49:29.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:29.155+0000 7f2070800640 1 -- 172.21.15.151:0/173150239 shutdown_connections 2024-09-16T15:49:29.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:29.155+0000 7f2070800640 1 --2- 172.21.15.151:0/173150239 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f206806efa0 0x7f2068073440 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:29.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:29.155+0000 7f2070800640 1 -- 172.21.15.151:0/173150239 >> 172.21.15.151:0/173150239 conn(0x7f20680b1840 msgr2=0x7f20680b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:49:29.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:29.156+0000 7f2070800640 1 -- 172.21.15.151:0/173150239 shutdown_connections 2024-09-16T15:49:29.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:29.156+0000 7f2070800640 1 -- 172.21.15.151:0/173150239 wait complete. 2024-09-16T15:49:29.160 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:29.158+0000 7f2070800640 1 Processor -- start 2024-09-16T15:49:29.160 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:29.158+0000 7f2070800640 1 -- start start 2024-09-16T15:49:29.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:29.159+0000 7f2070800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f206806efa0 0x7f2068140850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:29.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:29.159+0000 7f2070800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f20680cccd0 0x7f20681f8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:29.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:29.159+0000 7f2070800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2068145750 0x7f20681fc990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:29.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:29.159+0000 7f2070800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f20680ce6a0 con 0x7f206806efa0 2024-09-16T15:49:29.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:29.159+0000 7f2070800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f20680ce520 con 0x7f2068145750 2024-09-16T15:49:29.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:29.159+0000 7f2070800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f20680ce820 con 0x7f20680cccd0 2024-09-16T15:49:29.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:29.159+0000 7f206de00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f206806efa0 0x7f2068140850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:29.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:29.159+0000 7f206de00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f206806efa0 0x7f2068140850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52974/0 (socket says 172.21.15.151:52974) 2024-09-16T15:49:29.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:29.159+0000 7f206de00640 1 -- 172.21.15.151:0/3638657019 learned_addr learned my addr 172.21.15.151:0/3638657019 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:49:29.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:29.159+0000 7f206e800640 1 --2- 172.21.15.151:0/3638657019 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2068145750 0x7f20681fc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:29.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:29.159+0000 7f206d400640 1 --2- 172.21.15.151:0/3638657019 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f20680cccd0 0x7f20681f8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:29.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:29.160+0000 7f206de00640 1 -- 172.21.15.151:0/3638657019 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f20680cccd0 msgr2=0x7f20681f8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:29.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:29.160+0000 7f206de00640 1 --2- 172.21.15.151:0/3638657019 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f20680cccd0 0x7f20681f8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:29.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:29.160+0000 7f206de00640 1 -- 172.21.15.151:0/3638657019 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2068145750 msgr2=0x7f20681fc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:29.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:29.160+0000 7f206de00640 1 --2- 172.21.15.151:0/3638657019 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2068145750 0x7f20681fc990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:29.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:29.160+0000 7f206de00640 1 -- 172.21.15.151:0/3638657019 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20680a9620 con 0x7f206806efa0 2024-09-16T15:49:29.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:29.160+0000 7f206de00640 1 --2- 172.21.15.151:0/3638657019 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f206806efa0 0x7f2068140850 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f205c00b420 tx=0x7f205c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:49:29.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:29.160+0000 7f2062000640 1 -- 172.21.15.151:0/3638657019 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f205c011830 con 0x7f206806efa0 2024-09-16T15:49:29.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:29.160+0000 7f2070800640 1 -- 172.21.15.151:0/3638657019 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f206815e710 con 0x7f206806efa0 2024-09-16T15:49:29.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:29.161+0000 7f2062000640 1 -- 172.21.15.151:0/3638657019 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f205c016070 con 0x7f206806efa0 2024-09-16T15:49:29.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:29.161+0000 7f2062000640 1 -- 172.21.15.151:0/3638657019 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f205c011d60 con 0x7f206806efa0 2024-09-16T15:49:29.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:29.161+0000 7f2070800640 1 -- 172.21.15.151:0/3638657019 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f206815ce50 con 0x7f206806efa0 2024-09-16T15:49:29.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:29.162+0000 7f2070800640 1 -- 172.21.15.151:0/3638657019 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20680d0af0 con 0x7f206806efa0 2024-09-16T15:49:29.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:29.165+0000 7f2062000640 1 -- 172.21.15.151:0/3638657019 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 34) ==== 103225+0+0 (secure 0 0 0) 0x7f205c01b460 con 0x7f206806efa0 2024-09-16T15:49:29.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:29.167+0000 7f2062000640 1 --2- 172.21.15.151:0/3638657019 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f204c079db0 0x7f204c07c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:29.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:29.168+0000 7f2062000640 1 -- 172.21.15.151:0/3638657019 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7f205c0995a0 con 0x7f206806efa0 2024-09-16T15:49:29.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:29.170+0000 7f206d400640 1 --2- 172.21.15.151:0/3638657019 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f204c079db0 0x7f204c07c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:29.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:29.175+0000 7f2062000640 1 -- 172.21.15.151:0/3638657019 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f20680d0af0 con 0x7f206806efa0 2024-09-16T15:49:29.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:29.180+0000 7f206d400640 1 --2- 172.21.15.151:0/3638657019 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f204c079db0 0x7f204c07c270 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f206813a3d0 tx=0x7f2064009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:49:29.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:29.561+0000 7f2070800640 1 -- 172.21.15.151:0/3638657019 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone22126", "target": ["mon-mgr", ""]}) -- 0x7f20680ced50 con 0x7f204c079db0 2024-09-16T15:49:30.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:30.593+0000 7f2062000640 1 -- 172.21.15.151:0/3638657019 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f20680ced50 con 0x7f204c079db0 2024-09-16T15:49:30.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:30.596+0000 7f2070800640 1 -- 172.21.15.151:0/3638657019 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f204c079db0 msgr2=0x7f204c07c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:30.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:30.596+0000 7f2070800640 1 --2- 172.21.15.151:0/3638657019 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f204c079db0 0x7f204c07c270 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f206813a3d0 tx=0x7f2064009040 comp rx=0 tx=0).stop 2024-09-16T15:49:30.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:30.596+0000 7f2070800640 1 -- 172.21.15.151:0/3638657019 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f206806efa0 msgr2=0x7f2068140850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:30.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:30.596+0000 7f2070800640 1 --2- 172.21.15.151:0/3638657019 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f206806efa0 0x7f2068140850 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f205c00b420 tx=0x7f205c00b8e0 comp rx=0 tx=0).stop 2024-09-16T15:49:30.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:30.596+0000 7f2070800640 1 -- 172.21.15.151:0/3638657019 shutdown_connections 2024-09-16T15:49:30.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:30.596+0000 7f2070800640 1 --2- 172.21.15.151:0/3638657019 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f204c079db0 0x7f204c07c270 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:30.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:30.596+0000 7f2070800640 1 --2- 172.21.15.151:0/3638657019 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2068145750 0x7f20681fc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:30.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:30.596+0000 7f2070800640 1 --2- 172.21.15.151:0/3638657019 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f20680cccd0 0x7f20681f8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:30.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:30.596+0000 7f2070800640 1 --2- 172.21.15.151:0/3638657019 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f206806efa0 0x7f2068140850 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:30.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:30.596+0000 7f2070800640 1 -- 172.21.15.151:0/3638657019 >> 172.21.15.151:0/3638657019 conn(0x7f20680b1840 msgr2=0x7f20680b35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:49:30.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:30.596+0000 7f2070800640 1 -- 172.21.15.151:0/3638657019 shutdown_connections 2024-09-16T15:49:30.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:30.596+0000 7f2070800640 1 -- 172.21.15.151:0/3638657019 wait complete. 2024-09-16T15:49:30.617 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone81887 2024-09-16T15:49:30.926 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:30.923+0000 7fa072a00640 1 -- 172.21.15.151:0/509728512 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa06c161590 msgr2=0x7fa06c0b6e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:30.926 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:30.923+0000 7fa072a00640 1 --2- 172.21.15.151:0/509728512 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa06c161590 0x7fa06c0b6e00 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fa058013f30 tx=0x7fa058032ac0 comp rx=0 tx=0).stop 2024-09-16T15:49:30.926 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:30.923+0000 7fa072a00640 1 -- 172.21.15.151:0/509728512 shutdown_connections 2024-09-16T15:49:30.926 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:30.923+0000 7fa072a00640 1 --2- 172.21.15.151:0/509728512 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa06c0b5630 0x7fa06c0cf5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:30.926 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:30.923+0000 7fa072a00640 1 --2- 172.21.15.151:0/509728512 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa06c161590 0x7fa06c0b6e00 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:30.926 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:30.923+0000 7fa072a00640 1 -- 172.21.15.151:0/509728512 >> 172.21.15.151:0/509728512 conn(0x7fa06c0d3fb0 msgr2=0x7fa06c0d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:49:30.926 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:30.923+0000 7fa072a00640 1 -- 172.21.15.151:0/509728512 shutdown_connections 2024-09-16T15:49:30.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:30.923+0000 7fa072a00640 1 -- 172.21.15.151:0/509728512 wait complete. 2024-09-16T15:49:30.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:30.928+0000 7fa072a00640 1 Processor -- start 2024-09-16T15:49:30.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:30.928+0000 7fa072a00640 1 -- start start 2024-09-16T15:49:30.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:30.931+0000 7fa072a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa06c0b5630 0x7fa06c1f1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:30.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:30.931+0000 7fa072a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa06c0b7340 0x7fa06c0cb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:30.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:30.931+0000 7fa072a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa06c161590 0x7fa06c073580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:30.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:30.931+0000 7fa072a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa06c0cfe80 con 0x7fa06c0b5630 2024-09-16T15:49:30.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:30.931+0000 7fa072a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa06c0cfd00 con 0x7fa06c0b7340 2024-09-16T15:49:30.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:30.931+0000 7fa072a00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fa06c162430 con 0x7fa06c161590 2024-09-16T15:49:30.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:30.931+0000 7fa06be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa06c0b5630 0x7fa06c1f1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:30.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:30.931+0000 7fa06be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa06c0b5630 0x7fa06c1f1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52996/0 (socket says 172.21.15.151:52996) 2024-09-16T15:49:30.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:30.931+0000 7fa06be00640 1 -- 172.21.15.151:0/2593462388 learned_addr learned my addr 172.21.15.151:0/2593462388 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:49:30.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:30.932+0000 7fa070a00640 1 --2- 172.21.15.151:0/2593462388 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa06c161590 0x7fa06c073580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:30.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:30.933+0000 7fa06be00640 1 -- 172.21.15.151:0/2593462388 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa06c161590 msgr2=0x7fa06c073580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:30.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:30.933+0000 7fa06be00640 1 --2- 172.21.15.151:0/2593462388 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa06c161590 0x7fa06c073580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:30.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:30.933+0000 7fa06be00640 1 -- 172.21.15.151:0/2593462388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa06c0b7340 msgr2=0x7fa06c0cb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:30.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:30.933+0000 7fa06be00640 1 --2- 172.21.15.151:0/2593462388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa06c0b7340 0x7fa06c0cb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:30.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:30.933+0000 7fa06be00640 1 -- 172.21.15.151:0/2593462388 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa06c1f1940 con 0x7fa06c0b5630 2024-09-16T15:49:30.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:30.933+0000 7fa06be00640 1 --2- 172.21.15.151:0/2593462388 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa06c0b5630 0x7fa06c1f1400 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fa058000c00 tx=0x7fa0580043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:49:30.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:30.933+0000 7fa063e00640 1 -- 172.21.15.151:0/2593462388 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05804c070 con 0x7fa06c0b5630 2024-09-16T15:49:30.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:30.934+0000 7fa072a00640 1 -- 172.21.15.151:0/2593462388 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa06c0cc6e0 con 0x7fa06c0b5630 2024-09-16T15:49:30.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:30.934+0000 7fa072a00640 1 -- 172.21.15.151:0/2593462388 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa06c0ccf20 con 0x7fa06c0b5630 2024-09-16T15:49:30.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:30.934+0000 7fa063e00640 1 -- 172.21.15.151:0/2593462388 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa058003a00 con 0x7fa06c0b5630 2024-09-16T15:49:30.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:30.934+0000 7fa063e00640 1 -- 172.21.15.151:0/2593462388 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa058003e30 con 0x7fa06c0b5630 2024-09-16T15:49:30.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:30.936+0000 7fa072a00640 1 -- 172.21.15.151:0/2593462388 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa06c1628b0 con 0x7fa06c0b5630 2024-09-16T15:49:30.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:30.940+0000 7fa063e00640 1 -- 172.21.15.151:0/2593462388 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 34) ==== 103225+0+0 (secure 0 0 0) 0x7fa058034760 con 0x7fa06c0b5630 2024-09-16T15:49:30.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:30.941+0000 7fa063e00640 1 --2- 172.21.15.151:0/2593462388 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa04c079db0 0x7fa04c07c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:30.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:30.942+0000 7fa063e00640 1 -- 172.21.15.151:0/2593462388 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7fa0580c2140 con 0x7fa06c0b5630 2024-09-16T15:49:30.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:30.950+0000 7fa063e00640 1 -- 172.21.15.151:0/2593462388 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa06c0cc6e0 con 0x7fa06c0b5630 2024-09-16T15:49:30.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:30.951+0000 7fa06b400640 1 --2- 172.21.15.151:0/2593462388 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa04c079db0 0x7fa04c07c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:30.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:30.958+0000 7fa06b400640 1 --2- 172.21.15.151:0/2593462388 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa04c079db0 0x7fa04c07c270 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa06c0b96e0 tx=0x7fa064009230 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:49:31.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:31.352+0000 7fa072a00640 1 -- 172.21.15.151:0/2593462388 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone81887", "target": ["mon-mgr", ""]}) -- 0x7fa06c0d1d80 con 0x7fa04c079db0 2024-09-16T15:49:32.280 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:32.278+0000 7fa063e00640 1 -- 172.21.15.151:0/2593462388 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa06c0d1d80 con 0x7fa04c079db0 2024-09-16T15:49:32.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:32.281+0000 7fa072a00640 1 -- 172.21.15.151:0/2593462388 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa04c079db0 msgr2=0x7fa04c07c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:32.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:32.281+0000 7fa072a00640 1 --2- 172.21.15.151:0/2593462388 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa04c079db0 0x7fa04c07c270 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa06c0b96e0 tx=0x7fa064009230 comp rx=0 tx=0).stop 2024-09-16T15:49:32.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:32.281+0000 7fa072a00640 1 -- 172.21.15.151:0/2593462388 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa06c0b5630 msgr2=0x7fa06c1f1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:32.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:32.281+0000 7fa072a00640 1 --2- 172.21.15.151:0/2593462388 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa06c0b5630 0x7fa06c1f1400 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fa058000c00 tx=0x7fa0580043b0 comp rx=0 tx=0).stop 2024-09-16T15:49:32.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:32.281+0000 7fa072a00640 1 -- 172.21.15.151:0/2593462388 shutdown_connections 2024-09-16T15:49:32.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:32.281+0000 7fa072a00640 1 --2- 172.21.15.151:0/2593462388 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa04c079db0 0x7fa04c07c270 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:32.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:32.281+0000 7fa072a00640 1 --2- 172.21.15.151:0/2593462388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa06c0b7340 0x7fa06c0cb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:32.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:32.281+0000 7fa072a00640 1 --2- 172.21.15.151:0/2593462388 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa06c161590 0x7fa06c073580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:32.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:32.282+0000 7fa072a00640 1 --2- 172.21.15.151:0/2593462388 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa06c0b5630 0x7fa06c1f1400 secure :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fa058000c00 tx=0x7fa0580043b0 comp rx=0 tx=0).stop 2024-09-16T15:49:32.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:32.282+0000 7fa072a00640 1 -- 172.21.15.151:0/2593462388 >> 172.21.15.151:0/2593462388 conn(0x7fa06c0d3fb0 msgr2=0x7fa06c15fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:49:32.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:32.283+0000 7fa072a00640 1 -- 172.21.15.151:0/2593462388 shutdown_connections 2024-09-16T15:49:32.286 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:32.284+0000 7fa072a00640 1 -- 172.21.15.151:0/2593462388 wait complete. 2024-09-16T15:49:32.328 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone57948 2024-09-16T15:49:32.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:32.673+0000 7f384a000640 1 -- 172.21.15.151:0/4292403952 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f38440b3370 msgr2=0x7f38440cebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:32.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:32.673+0000 7f384a000640 1 --2- 172.21.15.151:0/4292403952 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f38440b3370 0x7f38440cebe0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f38440cd7c0 tx=0x7f383402a560 comp rx=0 tx=0).stop 2024-09-16T15:49:32.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:32.673+0000 7f384a000640 1 -- 172.21.15.151:0/4292403952 shutdown_connections 2024-09-16T15:49:32.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:32.673+0000 7f384a000640 1 --2- 172.21.15.151:0/4292403952 >> v2:172.21.15.167:3300/0 conn(0x7f384415f090 0x7f38440cf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:32.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:32.673+0000 7f384a000640 1 --2- 172.21.15.151:0/4292403952 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f38440b3370 0x7f38440cebe0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:32.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:32.673+0000 7f384a000640 1 -- 172.21.15.151:0/4292403952 >> 172.21.15.151:0/4292403952 conn(0x7f38440b1840 msgr2=0x7f38440b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:49:32.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:32.674+0000 7f384a000640 1 -- 172.21.15.151:0/4292403952 shutdown_connections 2024-09-16T15:49:32.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:32.674+0000 7f384a000640 1 -- 172.21.15.151:0/4292403952 wait complete. 2024-09-16T15:49:32.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:32.676+0000 7f384a000640 1 Processor -- start 2024-09-16T15:49:32.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:32.676+0000 7f384a000640 1 -- start start 2024-09-16T15:49:32.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:32.678+0000 7f384a000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38440b3370 0x7f3844140d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:32.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:32.678+0000 7f384a000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f384415c080 0x7f3844153e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:32.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:32.678+0000 7f384a000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f384415f090 0x7f3844163e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:32.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:32.678+0000 7f384a000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3844164e20 con 0x7f384415f090 2024-09-16T15:49:32.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:32.678+0000 7f384a000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3844164ca0 con 0x7f38440b3370 2024-09-16T15:49:32.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:32.678+0000 7f384a000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f3844164fa0 con 0x7f384415c080 2024-09-16T15:49:32.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:32.678+0000 7f3843e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f384415f090 0x7f3844163e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:32.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:32.679+0000 7f3843e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f384415f090 0x7f3844163e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53018/0 (socket says 172.21.15.151:53018) 2024-09-16T15:49:32.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:32.679+0000 7f3843e00640 1 -- 172.21.15.151:0/2947655070 learned_addr learned my addr 172.21.15.151:0/2947655070 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:49:32.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:32.679+0000 7f383aa00640 1 --2- 172.21.15.151:0/2947655070 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f384415c080 0x7f3844153e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:32.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:32.679+0000 7f3843400640 1 --2- 172.21.15.151:0/2947655070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38440b3370 0x7f3844140d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:32.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:32.679+0000 7f383aa00640 1 -- 172.21.15.151:0/2947655070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38440b3370 msgr2=0x7f3844140d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:32.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:32.679+0000 7f383aa00640 1 --2- 172.21.15.151:0/2947655070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38440b3370 0x7f3844140d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:32.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:32.680+0000 7f383aa00640 1 -- 172.21.15.151:0/2947655070 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f384415f090 msgr2=0x7f3844163e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:32.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:32.680+0000 7f383aa00640 1 --2- 172.21.15.151:0/2947655070 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f384415f090 0x7f3844163e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:32.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:32.680+0000 7f383aa00640 1 -- 172.21.15.151:0/2947655070 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38441525c0 con 0x7f384415c080 2024-09-16T15:49:32.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:32.680+0000 7f3843e00640 1 --2- 172.21.15.151:0/2947655070 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f384415f090 0x7f3844163e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:49:32.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:32.680+0000 7f383aa00640 1 --2- 172.21.15.151:0/2947655070 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f384415c080 0x7f3844153e30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f38440b5010 tx=0x7f382c00c7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:49:32.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:32.682+0000 7f383be00640 1 -- 172.21.15.151:0/2947655070 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f382c017070 con 0x7f384415c080 2024-09-16T15:49:32.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:32.682+0000 7f383be00640 1 -- 172.21.15.151:0/2947655070 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f382c0035f0 con 0x7f384415c080 2024-09-16T15:49:32.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:32.682+0000 7f383be00640 1 -- 172.21.15.151:0/2947655070 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f382c003a80 con 0x7f384415c080 2024-09-16T15:49:32.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:32.684+0000 7f384a000640 1 -- 172.21.15.151:0/2947655070 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38441528a0 con 0x7f384415c080 2024-09-16T15:49:32.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:32.684+0000 7f384a000640 1 -- 172.21.15.151:0/2947655070 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3844152e60 con 0x7f384415c080 2024-09-16T15:49:32.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:32.686+0000 7f384a000640 1 -- 172.21.15.151:0/2947655070 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f384415e980 con 0x7f384415c080 2024-09-16T15:49:32.694 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:32.691+0000 7f383be00640 1 -- 172.21.15.151:0/2947655070 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 34) ==== 103225+0+0 (secure 0 0 0) 0x7f382c003c20 con 0x7f384415c080 2024-09-16T15:49:32.694 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:32.692+0000 7f383be00640 1 --2- 172.21.15.151:0/2947655070 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3820079db0 0x7f382007c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:32.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:32.693+0000 7f383be00640 1 -- 172.21.15.151:0/2947655070 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7f382c0990a0 con 0x7f384415c080 2024-09-16T15:49:32.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:32.699+0000 7f383be00640 1 -- 172.21.15.151:0/2947655070 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f382c020070 con 0x7f384415c080 2024-09-16T15:49:32.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:32.703+0000 7f3843400640 1 --2- 172.21.15.151:0/2947655070 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3820079db0 0x7f382007c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:32.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:32.709+0000 7f3843400640 1 --2- 172.21.15.151:0/2947655070 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3820079db0 0x7f382007c270 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f3834033000 tx=0x7f3834004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:49:33.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:33.115+0000 7f384a000640 1 -- 172.21.15.151:0/2947655070 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone57948", "target": ["mon-mgr", ""]}) -- 0x7f38440cdf50 con 0x7f3820079db0 2024-09-16T15:49:34.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:34.929+0000 7f383be00640 1 -- 172.21.15.151:0/2947655070 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f38440cdf50 con 0x7f3820079db0 2024-09-16T15:49:34.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:34.933+0000 7f384a000640 1 -- 172.21.15.151:0/2947655070 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3820079db0 msgr2=0x7f382007c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:34.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:34.933+0000 7f384a000640 1 --2- 172.21.15.151:0/2947655070 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3820079db0 0x7f382007c270 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f3834033000 tx=0x7f3834004730 comp rx=0 tx=0).stop 2024-09-16T15:49:34.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:34.933+0000 7f384a000640 1 -- 172.21.15.151:0/2947655070 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f384415c080 msgr2=0x7f3844153e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:34.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:34.933+0000 7f384a000640 1 --2- 172.21.15.151:0/2947655070 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f384415c080 0x7f3844153e30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f38440b5010 tx=0x7f382c00c7f0 comp rx=0 tx=0).stop 2024-09-16T15:49:34.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:34.933+0000 7f384a000640 1 -- 172.21.15.151:0/2947655070 shutdown_connections 2024-09-16T15:49:34.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:34.933+0000 7f384a000640 1 --2- 172.21.15.151:0/2947655070 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3820079db0 0x7f382007c270 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:34.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:34.933+0000 7f384a000640 1 --2- 172.21.15.151:0/2947655070 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f384415f090 0x7f3844163e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:34.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:34.933+0000 7f384a000640 1 --2- 172.21.15.151:0/2947655070 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f384415c080 0x7f3844153e30 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:34.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:34.933+0000 7f384a000640 1 --2- 172.21.15.151:0/2947655070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38440b3370 0x7f3844140d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:34.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:34.933+0000 7f384a000640 1 -- 172.21.15.151:0/2947655070 >> 172.21.15.151:0/2947655070 conn(0x7f38440b1840 msgr2=0x7f384413cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:49:34.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:34.933+0000 7f384a000640 1 -- 172.21.15.151:0/2947655070 shutdown_connections 2024-09-16T15:49:34.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:34.933+0000 7f384a000640 1 -- 172.21.15.151:0/2947655070 wait complete. 2024-09-16T15:49:34.958 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone35339 2024-09-16T15:49:35.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:35.258+0000 7f4e40a00640 1 -- 172.21.15.151:0/594868114 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e3c0cd0a0 msgr2=0x7f4e3c0c4ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:35.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:35.258+0000 7f4e40a00640 1 --2- 172.21.15.151:0/594868114 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e3c0cd0a0 0x7f4e3c0c4ae0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f4e28005fd0 tx=0x7f4e2802a5f0 comp rx=0 tx=0).stop 2024-09-16T15:49:35.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:35.259+0000 7f4e40a00640 1 -- 172.21.15.151:0/594868114 shutdown_connections 2024-09-16T15:49:35.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:35.259+0000 7f4e40a00640 1 --2- 172.21.15.151:0/594868114 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e3c0cd0a0 0x7f4e3c0c4ae0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:35.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:35.259+0000 7f4e40a00640 1 --2- 172.21.15.151:0/594868114 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e3c147740 0x7f4e3c0cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:35.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:35.259+0000 7f4e40a00640 1 -- 172.21.15.151:0/594868114 >> 172.21.15.151:0/594868114 conn(0x7f4e3c0cc490 msgr2=0x7f4e3c0cc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:49:35.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:35.259+0000 7f4e40a00640 1 -- 172.21.15.151:0/594868114 shutdown_connections 2024-09-16T15:49:35.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:35.259+0000 7f4e40a00640 1 -- 172.21.15.151:0/594868114 wait complete. 2024-09-16T15:49:35.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:35.262+0000 7f4e40a00640 1 Processor -- start 2024-09-16T15:49:35.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:35.263+0000 7f4e40a00640 1 -- start start 2024-09-16T15:49:35.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:35.264+0000 7f4e40a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e3c0cdbb0 0x7f4e3c143320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:35.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:35.264+0000 7f4e40a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e3c147740 0x7f4e3c0b50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:35.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:35.264+0000 7f4e40a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e3c144cd0 0x7f4e3c0b5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:35.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:35.264+0000 7f4e40a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3c0c5470 con 0x7f4e3c0cdbb0 2024-09-16T15:49:35.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:35.264+0000 7f4e40a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3c0c52f0 con 0x7f4e3c144cd0 2024-09-16T15:49:35.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:35.264+0000 7f4e40a00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f4e3c162030 con 0x7f4e3c147740 2024-09-16T15:49:35.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:35.264+0000 7f4e3a800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e3c0cdbb0 0x7f4e3c143320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:35.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:35.264+0000 7f4e3a800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e3c0cdbb0 0x7f4e3c143320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:51230/0 (socket says 172.21.15.151:51230) 2024-09-16T15:49:35.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:35.264+0000 7f4e3a800640 1 -- 172.21.15.151:0/2977418518 learned_addr learned my addr 172.21.15.151:0/2977418518 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:49:35.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:35.265+0000 7f4e39e00640 1 --2- 172.21.15.151:0/2977418518 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e3c147740 0x7f4e3c0b50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:35.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:35.265+0000 7f4e3a800640 1 -- 172.21.15.151:0/2977418518 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e3c147740 msgr2=0x7f4e3c0b50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:35.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:35.265+0000 7f4e3a800640 1 --2- 172.21.15.151:0/2977418518 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e3c147740 0x7f4e3c0b50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:35.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:35.265+0000 7f4e3a800640 1 -- 172.21.15.151:0/2977418518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e3c144cd0 msgr2=0x7f4e3c0b5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:35.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:35.265+0000 7f4e3a800640 1 --2- 172.21.15.151:0/2977418518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e3c144cd0 0x7f4e3c0b5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:35.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:35.265+0000 7f4e3a800640 1 -- 172.21.15.151:0/2977418518 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e3c143ca0 con 0x7f4e3c0cdbb0 2024-09-16T15:49:35.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:35.265+0000 7f4e39e00640 1 --2- 172.21.15.151:0/2977418518 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e3c147740 0x7f4e3c0b50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:49:35.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:35.266+0000 7f4e3a800640 1 --2- 172.21.15.151:0/2977418518 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e3c0cdbb0 0x7f4e3c143320 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c00d770 tx=0x7f4e2c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:49:35.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:35.266+0000 7f4e32a00640 1 -- 172.21.15.151:0/2977418518 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e2c017070 con 0x7f4e3c0cdbb0 2024-09-16T15:49:35.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:35.267+0000 7f4e32a00640 1 -- 172.21.15.151:0/2977418518 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e2c009d20 con 0x7f4e3c0cdbb0 2024-09-16T15:49:35.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:35.267+0000 7f4e32a00640 1 -- 172.21.15.151:0/2977418518 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e2c0106c0 con 0x7f4e3c0cdbb0 2024-09-16T15:49:35.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:35.267+0000 7f4e40a00640 1 -- 172.21.15.151:0/2977418518 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e3c1fd190 con 0x7f4e3c0cdbb0 2024-09-16T15:49:35.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:35.267+0000 7f4e40a00640 1 -- 172.21.15.151:0/2977418518 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e3c1441f0 con 0x7f4e3c0cdbb0 2024-09-16T15:49:35.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:35.268+0000 7f4e40a00640 1 -- 172.21.15.151:0/2977418518 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e3c1624e0 con 0x7f4e3c0cdbb0 2024-09-16T15:49:35.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:35.270+0000 7f4e32a00640 1 -- 172.21.15.151:0/2977418518 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 34) ==== 103225+0+0 (secure 0 0 0) 0x7f4e2c01a070 con 0x7f4e3c0cdbb0 2024-09-16T15:49:35.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:35.271+0000 7f4e32a00640 1 --2- 172.21.15.151:0/2977418518 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4e1c079db0 0x7f4e1c07c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:35.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:35.271+0000 7f4e32a00640 1 -- 172.21.15.151:0/2977418518 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7f4e2c099b70 con 0x7f4e3c0cdbb0 2024-09-16T15:49:35.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:35.280+0000 7f4e32a00640 1 -- 172.21.15.151:0/2977418518 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4e2c063870 con 0x7f4e3c0cdbb0 2024-09-16T15:49:35.293 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:35.291+0000 7f4e39e00640 1 --2- 172.21.15.151:0/2977418518 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4e1c079db0 0x7f4e1c07c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:35.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:35.299+0000 7f4e39e00640 1 --2- 172.21.15.151:0/2977418518 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4e1c079db0 0x7f4e1c07c270 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f4e28032000 tx=0x7f4e28004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:49:35.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:35.690+0000 7f4e40a00640 1 -- 172.21.15.151:0/2977418518 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone35339", "target": ["mon-mgr", ""]}) -- 0x7f4e3c0aede0 con 0x7f4e1c079db0 2024-09-16T15:49:36.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:36.788+0000 7f4e32a00640 1 -- 172.21.15.151:0/2977418518 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4e3c0aede0 con 0x7f4e1c079db0 2024-09-16T15:49:36.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:36.797+0000 7f4e40a00640 1 -- 172.21.15.151:0/2977418518 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4e1c079db0 msgr2=0x7f4e1c07c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:36.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:36.797+0000 7f4e40a00640 1 --2- 172.21.15.151:0/2977418518 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4e1c079db0 0x7f4e1c07c270 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f4e28032000 tx=0x7f4e28004730 comp rx=0 tx=0).stop 2024-09-16T15:49:36.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:36.797+0000 7f4e40a00640 1 -- 172.21.15.151:0/2977418518 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e3c0cdbb0 msgr2=0x7f4e3c143320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:36.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:36.797+0000 7f4e40a00640 1 --2- 172.21.15.151:0/2977418518 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e3c0cdbb0 0x7f4e3c143320 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c00d770 tx=0x7f4e2c00dc30 comp rx=0 tx=0).stop 2024-09-16T15:49:36.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:36.800+0000 7f4e40a00640 1 -- 172.21.15.151:0/2977418518 shutdown_connections 2024-09-16T15:49:36.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:36.800+0000 7f4e40a00640 1 --2- 172.21.15.151:0/2977418518 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4e1c079db0 0x7f4e1c07c270 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:36.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:36.800+0000 7f4e40a00640 1 --2- 172.21.15.151:0/2977418518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e3c144cd0 0x7f4e3c0b5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:36.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:36.800+0000 7f4e40a00640 1 --2- 172.21.15.151:0/2977418518 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e3c147740 0x7f4e3c0b50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:36.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:36.800+0000 7f4e40a00640 1 --2- 172.21.15.151:0/2977418518 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e3c0cdbb0 0x7f4e3c143320 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:36.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:36.800+0000 7f4e40a00640 1 -- 172.21.15.151:0/2977418518 >> 172.21.15.151:0/2977418518 conn(0x7f4e3c0cc490 msgr2=0x7f4e3c0743a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:49:36.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:36.801+0000 7f4e40a00640 1 -- 172.21.15.151:0/2977418518 shutdown_connections 2024-09-16T15:49:36.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:36.801+0000 7f4e40a00640 1 -- 172.21.15.151:0/2977418518 wait complete. 2024-09-16T15:49:36.851 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone872410 2024-09-16T15:49:37.150 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:37.147+0000 7f268e200640 1 -- 172.21.15.151:0/1736432214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f268815be10 msgr2=0x7f26880688b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:37.150 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:37.147+0000 7f268e200640 1 --2- 172.21.15.151:0/1736432214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f268815be10 0x7f26880688b0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f2678005fd0 tx=0x7f267802a4a0 comp rx=0 tx=0).stop 2024-09-16T15:49:37.150 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:37.148+0000 7f268e200640 1 -- 172.21.15.151:0/1736432214 shutdown_connections 2024-09-16T15:49:37.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:37.148+0000 7f268e200640 1 --2- 172.21.15.151:0/1736432214 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26880ce350 0x7f2688068df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:37.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:37.148+0000 7f268e200640 1 --2- 172.21.15.151:0/1736432214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f268815be10 0x7f26880688b0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:37.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:37.148+0000 7f268e200640 1 -- 172.21.15.151:0/1736432214 >> 172.21.15.151:0/1736432214 conn(0x7f26880cd830 msgr2=0x7f26880cdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:49:37.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:37.148+0000 7f268e200640 1 -- 172.21.15.151:0/1736432214 shutdown_connections 2024-09-16T15:49:37.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:37.148+0000 7f268e200640 1 -- 172.21.15.151:0/1736432214 wait complete. 2024-09-16T15:49:37.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:37.151+0000 7f268e200640 1 Processor -- start 2024-09-16T15:49:37.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:37.151+0000 7f268e200640 1 -- start start 2024-09-16T15:49:37.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:37.154+0000 7f268e200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2688070810 0x7f26880d0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:37.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:37.154+0000 7f268e200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26880ce350 0x7f2688159e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:37.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:37.154+0000 7f268e200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f268815be10 0x7f26880a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:37.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:37.154+0000 7f268e200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f26880cbb10 con 0x7f2688070810 2024-09-16T15:49:37.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:37.154+0000 7f268e200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f268806e0b0 con 0x7f26880ce350 2024-09-16T15:49:37.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:37.154+0000 7f268e200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f26880cbc90 con 0x7f268815be10 2024-09-16T15:49:37.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:37.155+0000 7f2687400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2688070810 0x7f26880d0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:37.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:37.155+0000 7f2687400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2688070810 0x7f26880d0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:51234/0 (socket says 172.21.15.151:51234) 2024-09-16T15:49:37.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:37.155+0000 7f2687400640 1 -- 172.21.15.151:0/2464313308 learned_addr learned my addr 172.21.15.151:0/2464313308 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:49:37.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:37.156+0000 7f267fe00640 1 --2- 172.21.15.151:0/2464313308 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26880ce350 0x7f2688159e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:37.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:37.157+0000 7f2687e00640 1 --2- 172.21.15.151:0/2464313308 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f268815be10 0x7f26880a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:37.160 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:37.157+0000 7f2687400640 1 -- 172.21.15.151:0/2464313308 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f268815be10 msgr2=0x7f26880a16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:37.160 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:37.157+0000 7f2687400640 1 --2- 172.21.15.151:0/2464313308 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f268815be10 0x7f26880a16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:37.160 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:37.157+0000 7f2687400640 1 -- 172.21.15.151:0/2464313308 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26880ce350 msgr2=0x7f2688159e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:37.160 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:37.157+0000 7f2687400640 1 --2- 172.21.15.151:0/2464313308 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26880ce350 0x7f2688159e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:37.160 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:37.157+0000 7f2687400640 1 -- 172.21.15.151:0/2464313308 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26880ceab0 con 0x7f2688070810 2024-09-16T15:49:37.160 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:37.158+0000 7f2687400640 1 --2- 172.21.15.151:0/2464313308 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2688070810 0x7f26880d0540 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f2678034040 tx=0x7f2678003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:49:37.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:37.158+0000 7f267f400640 1 -- 172.21.15.151:0/2464313308 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267803e030 con 0x7f2688070810 2024-09-16T15:49:37.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:37.158+0000 7f267f400640 1 -- 172.21.15.151:0/2464313308 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2678003d10 con 0x7f2688070810 2024-09-16T15:49:37.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:37.158+0000 7f268e200640 1 -- 172.21.15.151:0/2464313308 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26880ced90 con 0x7f2688070810 2024-09-16T15:49:37.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:37.158+0000 7f268e200640 1 -- 172.21.15.151:0/2464313308 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26880cf350 con 0x7f2688070810 2024-09-16T15:49:37.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:37.159+0000 7f267f400640 1 -- 172.21.15.151:0/2464313308 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2678035040 con 0x7f2688070810 2024-09-16T15:49:37.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:37.160+0000 7f268e200640 1 -- 172.21.15.151:0/2464313308 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2660005290 con 0x7f2688070810 2024-09-16T15:49:37.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:37.163+0000 7f267f400640 1 -- 172.21.15.151:0/2464313308 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 34) ==== 103225+0+0 (secure 0 0 0) 0x7f2678041030 con 0x7f2688070810 2024-09-16T15:49:37.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:37.164+0000 7f267f400640 1 --2- 172.21.15.151:0/2464313308 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2664079db0 0x7f266407c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:37.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:37.164+0000 7f267f400640 1 -- 172.21.15.151:0/2464313308 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7f26780ba4a0 con 0x7f2688070810 2024-09-16T15:49:37.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:37.171+0000 7f267f400640 1 -- 172.21.15.151:0/2464313308 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2678083b50 con 0x7f2688070810 2024-09-16T15:49:37.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:37.204+0000 7f267fe00640 1 --2- 172.21.15.151:0/2464313308 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2664079db0 0x7f266407c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:37.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:37.246+0000 7f267fe00640 1 --2- 172.21.15.151:0/2464313308 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2664079db0 0x7f266407c270 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f268809f090 tx=0x7f2670005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:49:37.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:37.639+0000 7f268e200640 1 -- 172.21.15.151:0/2464313308 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone872410", "target": ["mon-mgr", ""]}) -- 0x7f2660000c80 con 0x7f2664079db0 2024-09-16T15:49:39.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:39.199+0000 7f267f400640 1 -- 172.21.15.151:0/2464313308 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2660000c80 con 0x7f2664079db0 2024-09-16T15:49:39.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:39.204+0000 7f268e200640 1 -- 172.21.15.151:0/2464313308 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2664079db0 msgr2=0x7f266407c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:39.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:39.204+0000 7f268e200640 1 --2- 172.21.15.151:0/2464313308 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2664079db0 0x7f266407c270 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f268809f090 tx=0x7f2670005f90 comp rx=0 tx=0).stop 2024-09-16T15:49:39.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:39.204+0000 7f268e200640 1 -- 172.21.15.151:0/2464313308 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2688070810 msgr2=0x7f26880d0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:39.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:39.204+0000 7f268e200640 1 --2- 172.21.15.151:0/2464313308 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2688070810 0x7f26880d0540 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f2678034040 tx=0x7f2678003820 comp rx=0 tx=0).stop 2024-09-16T15:49:39.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:39.207+0000 7f268e200640 1 -- 172.21.15.151:0/2464313308 shutdown_connections 2024-09-16T15:49:39.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:39.207+0000 7f268e200640 1 --2- 172.21.15.151:0/2464313308 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2664079db0 0x7f266407c270 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:39.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:39.207+0000 7f268e200640 1 --2- 172.21.15.151:0/2464313308 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26880ce350 0x7f2688159e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:39.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:39.207+0000 7f268e200640 1 --2- 172.21.15.151:0/2464313308 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f268815be10 0x7f26880a16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:39.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:39.207+0000 7f268e200640 1 --2- 172.21.15.151:0/2464313308 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2688070810 0x7f26880d0540 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:39.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:39.207+0000 7f268e200640 1 -- 172.21.15.151:0/2464313308 >> 172.21.15.151:0/2464313308 conn(0x7f26880cd830 msgr2=0x7f26880b1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:49:39.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:39.207+0000 7f268e200640 1 -- 172.21.15.151:0/2464313308 shutdown_connections 2024-09-16T15:49:39.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:39.207+0000 7f268e200640 1 -- 172.21.15.151:0/2464313308 wait complete. 2024-09-16T15:49:39.232 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone556011 2024-09-16T15:49:39.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:39.589+0000 7f864f400640 1 -- 172.21.15.151:0/3893578840 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86500ae440 con 0x7f86480e39e0 2024-09-16T15:49:39.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:39.589+0000 7f8656600640 1 -- 172.21.15.151:0/3893578840 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86480e39e0 msgr2=0x7f8648052ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:39.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:39.589+0000 7f8656600640 1 --2- 172.21.15.151:0/3893578840 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86480e39e0 0x7f8648052ea0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f8650052430 tx=0x7f86500a5e50 comp rx=0 tx=0).stop 2024-09-16T15:49:39.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:39.589+0000 7f8656600640 1 -- 172.21.15.151:0/3893578840 shutdown_connections 2024-09-16T15:49:39.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:39.590+0000 7f8656600640 1 --2- 172.21.15.151:0/3893578840 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86480e39e0 0x7f8648052ea0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:39.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:39.590+0000 7f8656600640 1 --2- 172.21.15.151:0/3893578840 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f864803ab10 0x7f8648037940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:39.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:39.590+0000 7f8656600640 1 -- 172.21.15.151:0/3893578840 >> 172.21.15.151:0/3893578840 conn(0x7f8648055540 msgr2=0x7f8648055920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:49:39.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:39.593+0000 7f8656600640 1 -- 172.21.15.151:0/3893578840 shutdown_connections 2024-09-16T15:49:39.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:39.593+0000 7f8656600640 1 -- 172.21.15.151:0/3893578840 wait complete. 2024-09-16T15:49:39.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:39.597+0000 7f8656600640 1 Processor -- start 2024-09-16T15:49:39.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:39.598+0000 7f8656600640 1 -- start start 2024-09-16T15:49:39.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:39.599+0000 7f8656600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f864803ab10 0x7f86480dff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:39.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:39.599+0000 7f8656600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86480e2a60 0x7f86480e64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:39.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:39.599+0000 7f8656600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86480e39e0 0x7f86480e6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:39.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:39.599+0000 7f8656600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f86480537b0 con 0x7f86480e39e0 2024-09-16T15:49:39.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:39.599+0000 7f8656600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8648053630 con 0x7f86480e2a60 2024-09-16T15:49:39.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:39.599+0000 7f8656600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f8648053930 con 0x7f864803ab10 2024-09-16T15:49:39.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:39.602+0000 7f8655c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86480e39e0 0x7f86480e6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:39.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:39.602+0000 7f8655c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86480e39e0 0x7f86480e6a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:51242/0 (socket says 172.21.15.151:51242) 2024-09-16T15:49:39.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:39.603+0000 7f8655c00640 1 -- 172.21.15.151:0/3866068799 learned_addr learned my addr 172.21.15.151:0/3866068799 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:49:39.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:39.603+0000 7f864fe00640 1 --2- 172.21.15.151:0/3866068799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86480e2a60 0x7f86480e64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:39.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:39.603+0000 7f8655200640 1 --2- 172.21.15.151:0/3866068799 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f864803ab10 0x7f86480dff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:39.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:39.604+0000 7f8655c00640 1 -- 172.21.15.151:0/3866068799 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f864803ab10 msgr2=0x7f86480dff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:39.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:39.604+0000 7f8655c00640 1 --2- 172.21.15.151:0/3866068799 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f864803ab10 0x7f86480dff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:39.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:39.604+0000 7f8655c00640 1 -- 172.21.15.151:0/3866068799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86480e2a60 msgr2=0x7f86480e64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:39.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:39.604+0000 7f8655c00640 1 --2- 172.21.15.151:0/3866068799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86480e2a60 0x7f86480e64e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:39.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:39.604+0000 7f8655c00640 1 -- 172.21.15.151:0/3866068799 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8648180e10 con 0x7f86480e39e0 2024-09-16T15:49:39.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:39.604+0000 7f8655c00640 1 --2- 172.21.15.151:0/3866068799 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86480e39e0 0x7f86480e6a20 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f8650052890 tx=0x7f8650063a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:49:39.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:39.605+0000 7f864cc00640 1 -- 172.21.15.151:0/3866068799 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86500b3040 con 0x7f86480e39e0 2024-09-16T15:49:39.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:39.605+0000 7f864cc00640 1 -- 172.21.15.151:0/3866068799 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8650063c70 con 0x7f86480e39e0 2024-09-16T15:49:39.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:39.605+0000 7f864cc00640 1 -- 172.21.15.151:0/3866068799 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86500b96a0 con 0x7f86480e39e0 2024-09-16T15:49:39.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:39.606+0000 7f8656600640 1 -- 172.21.15.151:0/3866068799 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86481810f0 con 0x7f86480e39e0 2024-09-16T15:49:39.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:39.606+0000 7f8656600640 1 -- 172.21.15.151:0/3866068799 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86481816b0 con 0x7f86480e39e0 2024-09-16T15:49:39.611 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:39.608+0000 7f8656600640 1 -- 172.21.15.151:0/3866068799 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8648037e90 con 0x7f86480e39e0 2024-09-16T15:49:39.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:39.611+0000 7f864cc00640 1 -- 172.21.15.151:0/3866068799 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 34) ==== 103225+0+0 (secure 0 0 0) 0x7f86500c0020 con 0x7f86480e39e0 2024-09-16T15:49:39.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:39.613+0000 7f864cc00640 1 --2- 172.21.15.151:0/3866068799 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8630079e00 0x7f863007c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:39.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:39.614+0000 7f864cc00640 1 -- 172.21.15.151:0/3866068799 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7f8650135860 con 0x7f86480e39e0 2024-09-16T15:49:39.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:39.618+0000 7f864cc00640 1 -- 172.21.15.151:0/3866068799 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f86500ff560 con 0x7f86480e39e0 2024-09-16T15:49:39.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:39.631+0000 7f8655200640 1 --2- 172.21.15.151:0/3866068799 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8630079e00 0x7f863007c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:39.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:39.661+0000 7f8655200640 1 --2- 172.21.15.151:0/3866068799 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8630079e00 0x7f863007c2c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f8644001610 tx=0x7f8644009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:49:39.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:39.990+0000 7f8656600640 1 -- 172.21.15.151:0/3866068799 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone556011", "target": ["mon-mgr", ""]}) -- 0x7f8648180bc0 con 0x7f8630079e00 2024-09-16T15:49:41.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:41.080+0000 7f864cc00640 1 -- 172.21.15.151:0/3866068799 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8648180bc0 con 0x7f8630079e00 2024-09-16T15:49:41.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:41.088+0000 7f8656600640 1 -- 172.21.15.151:0/3866068799 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8630079e00 msgr2=0x7f863007c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:41.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:41.088+0000 7f8656600640 1 --2- 172.21.15.151:0/3866068799 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8630079e00 0x7f863007c2c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f8644001610 tx=0x7f8644009040 comp rx=0 tx=0).stop 2024-09-16T15:49:41.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:41.088+0000 7f8656600640 1 -- 172.21.15.151:0/3866068799 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86480e39e0 msgr2=0x7f86480e6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:41.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:41.088+0000 7f8656600640 1 --2- 172.21.15.151:0/3866068799 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86480e39e0 0x7f86480e6a20 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f8650052890 tx=0x7f8650063a80 comp rx=0 tx=0).stop 2024-09-16T15:49:41.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:41.091+0000 7f8656600640 1 -- 172.21.15.151:0/3866068799 shutdown_connections 2024-09-16T15:49:41.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:41.091+0000 7f8656600640 1 --2- 172.21.15.151:0/3866068799 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8630079e00 0x7f863007c2c0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:41.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:41.091+0000 7f8656600640 1 --2- 172.21.15.151:0/3866068799 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86480e2a60 0x7f86480e64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:41.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:41.091+0000 7f8656600640 1 --2- 172.21.15.151:0/3866068799 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86480e39e0 0x7f86480e6a20 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:41.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:41.091+0000 7f8656600640 1 --2- 172.21.15.151:0/3866068799 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f864803ab10 0x7f86480dff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:41.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:41.091+0000 7f8656600640 1 -- 172.21.15.151:0/3866068799 >> 172.21.15.151:0/3866068799 conn(0x7f8648055540 msgr2=0x7f864803d830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:49:41.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:41.091+0000 7f8656600640 1 -- 172.21.15.151:0/3866068799 shutdown_connections 2024-09-16T15:49:41.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:41.092+0000 7f8656600640 1 -- 172.21.15.151:0/3866068799 wait complete. 2024-09-16T15:49:41.127 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone515512 2024-09-16T15:49:41.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:41.396+0000 7f40cb200640 1 -- 172.21.15.151:0/2754521368 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f40ac003a80 msgr2=0x7f40ac003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:41.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:41.396+0000 7f40cb200640 1 --2- 172.21.15.151:0/2754521368 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f40ac003a80 0x7f40ac003ea0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f40b8001690 tx=0x7f40ac005830 comp rx=0 tx=0).stop 2024-09-16T15:49:41.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:41.396+0000 7f40ca800640 1 -- 172.21.15.151:0/2754521368 reap_dead start 2024-09-16T15:49:41.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:41.396+0000 7f40cb200640 1 -- 172.21.15.151:0/2754521368 shutdown_connections 2024-09-16T15:49:41.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:41.396+0000 7f40cb200640 1 -- 172.21.15.151:0/2754521368 >> 172.21.15.151:0/2754521368 conn(0x7f40c40d3e60 msgr2=0x7f40c40d4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:49:41.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:41.396+0000 7f40cb200640 1 -- 172.21.15.151:0/2754521368 shutdown_connections 2024-09-16T15:49:41.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:41.396+0000 7f40cb200640 1 -- 172.21.15.151:0/2754521368 wait complete. 2024-09-16T15:49:41.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:41.399+0000 7f40cb200640 1 Processor -- start 2024-09-16T15:49:41.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:41.399+0000 7f40cb200640 1 -- start start 2024-09-16T15:49:41.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:41.401+0000 7f40cb200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f40ac003a80 0x7f40c40cd370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:41.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:41.401+0000 7f40cb200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f40c414dd70 0x7f40c414d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:41.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:41.401+0000 7f40cb200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40c414d680 0x7f40c41f7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:41.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:41.401+0000 7f40cb200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f40c40b82e0 con 0x7f40c414dd70 2024-09-16T15:49:41.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:41.401+0000 7f40cb200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f40c40b8160 con 0x7f40c414d680 2024-09-16T15:49:41.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:41.401+0000 7f40cb200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f40c40b8460 con 0x7f40ac003a80 2024-09-16T15:49:41.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:41.401+0000 7f40c9400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f40c414dd70 0x7f40c414d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:41.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:41.401+0000 7f40c9400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f40c414dd70 0x7f40c414d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:51270/0 (socket says 172.21.15.151:51270) 2024-09-16T15:49:41.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:41.401+0000 7f40c9400640 1 -- 172.21.15.151:0/3684869577 learned_addr learned my addr 172.21.15.151:0/3684869577 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:49:41.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:41.402+0000 7f40ca800640 1 --2- 172.21.15.151:0/3684869577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40c414d680 0x7f40c41f7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:41.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:41.402+0000 7f40c9400640 1 -- 172.21.15.151:0/3684869577 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f40ac003a80 msgr2=0x7f40c40cd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:41.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:41.402+0000 7f40c9400640 1 --2- 172.21.15.151:0/3684869577 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f40ac003a80 0x7f40c40cd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:41.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:41.402+0000 7f40c9400640 1 -- 172.21.15.151:0/3684869577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40c414d680 msgr2=0x7f40c41f7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:41.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:41.402+0000 7f40c9400640 1 --2- 172.21.15.151:0/3684869577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40c414d680 0x7f40c41f7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:41.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:41.402+0000 7f40c9400640 1 -- 172.21.15.151:0/3684869577 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40c409ffe0 con 0x7f40c414dd70 2024-09-16T15:49:41.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:41.403+0000 7f40c9400640 1 --2- 172.21.15.151:0/3684869577 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f40c414dd70 0x7f40c414d140 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f40b4006680 tx=0x7f40b40066b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:49:41.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:41.403+0000 7f40ca800640 1 --2- 172.21.15.151:0/3684869577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40c414d680 0x7f40c41f7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:49:41.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:41.403+0000 7f40be000640 1 -- 172.21.15.151:0/3684869577 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b400f7e0 con 0x7f40c414dd70 2024-09-16T15:49:41.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:41.403+0000 7f40be000640 1 -- 172.21.15.151:0/3684869577 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f40b4016020 con 0x7f40c414dd70 2024-09-16T15:49:41.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:41.403+0000 7f40be000640 1 -- 172.21.15.151:0/3684869577 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b400fc20 con 0x7f40c414dd70 2024-09-16T15:49:41.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:41.403+0000 7f40cb200640 1 -- 172.21.15.151:0/3684869577 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40c40a02c0 con 0x7f40c414dd70 2024-09-16T15:49:41.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:41.403+0000 7f40cb200640 1 -- 172.21.15.151:0/3684869577 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40c40cd8b0 con 0x7f40c414dd70 2024-09-16T15:49:41.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:41.405+0000 7f40cb200640 1 -- 172.21.15.151:0/3684869577 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40c40b89f0 con 0x7f40c414dd70 2024-09-16T15:49:41.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:41.408+0000 7f40be000640 1 -- 172.21.15.151:0/3684869577 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 34) ==== 103225+0+0 (secure 0 0 0) 0x7f40b4011430 con 0x7f40c414dd70 2024-09-16T15:49:41.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:41.409+0000 7f40be000640 1 --2- 172.21.15.151:0/3684869577 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f40a8079db0 0x7f40a807c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:41.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:41.410+0000 7f40be000640 1 -- 172.21.15.151:0/3684869577 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7f40b409a680 con 0x7f40c414dd70 2024-09-16T15:49:41.419 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:41.417+0000 7f40c9e00640 1 --2- 172.21.15.151:0/3684869577 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f40a8079db0 0x7f40a807c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:41.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:41.417+0000 7f40be000640 1 -- 172.21.15.151:0/3684869577 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f40b4020020 con 0x7f40c414dd70 2024-09-16T15:49:41.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:41.425+0000 7f40c9e00640 1 --2- 172.21.15.151:0/3684869577 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f40a8079db0 0x7f40a807c270 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f40b8032000 tx=0x7f40b8001120 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:49:41.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:41.840+0000 7f40cb200640 1 -- 172.21.15.151:0/3684869577 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone515512", "target": ["mon-mgr", ""]}) -- 0x7f40c41f87f0 con 0x7f40a8079db0 2024-09-16T15:49:43.509 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:43.507+0000 7f40be000640 1 -- 172.21.15.151:0/3684869577 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f40c41f87f0 con 0x7f40a8079db0 2024-09-16T15:49:43.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:43.515+0000 7f40cb200640 1 -- 172.21.15.151:0/3684869577 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f40a8079db0 msgr2=0x7f40a807c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:43.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:43.515+0000 7f40cb200640 1 --2- 172.21.15.151:0/3684869577 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f40a8079db0 0x7f40a807c270 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f40b8032000 tx=0x7f40b8001120 comp rx=0 tx=0).stop 2024-09-16T15:49:43.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:43.515+0000 7f40cb200640 1 -- 172.21.15.151:0/3684869577 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f40c414dd70 msgr2=0x7f40c414d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:43.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:43.515+0000 7f40cb200640 1 --2- 172.21.15.151:0/3684869577 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f40c414dd70 0x7f40c414d140 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f40b4006680 tx=0x7f40b40066b0 comp rx=0 tx=0).stop 2024-09-16T15:49:43.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:43.515+0000 7f40cb200640 1 -- 172.21.15.151:0/3684869577 shutdown_connections 2024-09-16T15:49:43.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:43.515+0000 7f40cb200640 1 --2- 172.21.15.151:0/3684869577 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f40a8079db0 0x7f40a807c270 secure :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f40b8032000 tx=0x7f40b8001120 comp rx=0 tx=0).stop 2024-09-16T15:49:43.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:43.515+0000 7f40cb200640 1 --2- 172.21.15.151:0/3684869577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40c414d680 0x7f40c41f7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:43.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:43.515+0000 7f40cb200640 1 --2- 172.21.15.151:0/3684869577 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f40c414dd70 0x7f40c414d140 secure :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f40b4006680 tx=0x7f40b40066b0 comp rx=0 tx=0).stop 2024-09-16T15:49:43.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:43.515+0000 7f40cb200640 1 --2- 172.21.15.151:0/3684869577 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f40ac003a80 0x7f40c40cd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:43.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:43.515+0000 7f40cb200640 1 -- 172.21.15.151:0/3684869577 >> 172.21.15.151:0/3684869577 conn(0x7f40c40d3e60 msgr2=0x7f40c4160b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:49:43.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:43.522+0000 7f40cb200640 1 -- 172.21.15.151:0/3684869577 shutdown_connections 2024-09-16T15:49:43.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:43.525+0000 7f40cb200640 1 -- 172.21.15.151:0/3684869577 wait complete. 2024-09-16T15:49:43.578 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone277913 2024-09-16T15:49:43.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:43.940+0000 7f302b400640 1 -- 172.21.15.151:0/1653611561 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3014031bf0 con 0x7f302c0af170 2024-09-16T15:49:43.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:43.941+0000 7f3033800640 1 -- 172.21.15.151:0/1653611561 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f302c0af170 msgr2=0x7f302c0cdad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:43.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:43.941+0000 7f3033800640 1 --2- 172.21.15.151:0/1653611561 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f302c0af170 0x7f302c0cdad0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3014009830 tx=0x7f301402a4a0 comp rx=0 tx=0).stop 2024-09-16T15:49:43.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:43.942+0000 7f3033800640 1 -- 172.21.15.151:0/1653611561 shutdown_connections 2024-09-16T15:49:43.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:43.942+0000 7f3033800640 1 --2- 172.21.15.151:0/1653611561 >> v2:172.21.15.167:3300/0 conn(0x7f302c161bb0 0x7f302c0ce010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:43.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:43.942+0000 7f3033800640 1 --2- 172.21.15.151:0/1653611561 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f302c0af170 0x7f302c0cdad0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:43.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:43.942+0000 7f3033800640 1 -- 172.21.15.151:0/1653611561 >> 172.21.15.151:0/1653611561 conn(0x7f302c0b1840 msgr2=0x7f302c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:49:43.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:43.943+0000 7f3033800640 1 -- 172.21.15.151:0/1653611561 shutdown_connections 2024-09-16T15:49:43.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:43.943+0000 7f3033800640 1 -- 172.21.15.151:0/1653611561 wait complete. 2024-09-16T15:49:43.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:43.949+0000 7f3033800640 1 Processor -- start 2024-09-16T15:49:43.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:43.950+0000 7f3033800640 1 -- start start 2024-09-16T15:49:43.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:43.951+0000 7f3033800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f302c0af170 0x7f302c1f8760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:43.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:43.951+0000 7f3033800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f302c161490 0x7f302c0b0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:43.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:43.951+0000 7f3033800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f302c161bb0 0x7f302c146d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:43.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:43.951+0000 7f3033800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f302c06fa20 con 0x7f302c0af170 2024-09-16T15:49:43.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:43.951+0000 7f3033800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f302c06f8a0 con 0x7f302c161490 2024-09-16T15:49:43.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:43.951+0000 7f3033800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f302c06fba0 con 0x7f302c161bb0 2024-09-16T15:49:43.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:43.953+0000 7f3031800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f302c161bb0 0x7f302c146d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:43.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:43.953+0000 7f3031800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f302c161bb0 0x7f302c146d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.151:51660/0 (socket says 172.21.15.151:51660) 2024-09-16T15:49:43.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:43.953+0000 7f3031800640 1 -- 172.21.15.151:0/1684993307 learned_addr learned my addr 172.21.15.151:0/1684993307 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:49:43.957 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:43.953+0000 7f3030e00640 1 --2- 172.21.15.151:0/1684993307 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f302c0af170 0x7f302c1f8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:43.957 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:43.954+0000 7f302be00640 1 --2- 172.21.15.151:0/1684993307 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f302c161490 0x7f302c0b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:43.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:43.955+0000 7f3030e00640 1 -- 172.21.15.151:0/1684993307 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f302c161bb0 msgr2=0x7f302c146d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:43.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:43.955+0000 7f3030e00640 1 --2- 172.21.15.151:0/1684993307 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f302c161bb0 0x7f302c146d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:43.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:43.955+0000 7f3030e00640 1 -- 172.21.15.151:0/1684993307 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f302c161490 msgr2=0x7f302c0b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:43.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:43.955+0000 7f3030e00640 1 --2- 172.21.15.151:0/1684993307 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f302c161490 0x7f302c0b0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:43.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:43.955+0000 7f3030e00640 1 -- 172.21.15.151:0/1684993307 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f302c14aa60 con 0x7f302c0af170 2024-09-16T15:49:43.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:43.957+0000 7f3030e00640 1 --2- 172.21.15.151:0/1684993307 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f302c0af170 0x7f302c1f8760 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f3024001fa0 tx=0x7f302400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:49:43.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:43.958+0000 7f3028c00640 1 -- 172.21.15.151:0/1684993307 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3024004a20 con 0x7f302c0af170 2024-09-16T15:49:43.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:43.958+0000 7f3028c00640 1 -- 172.21.15.151:0/1684993307 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3024004cb0 con 0x7f302c0af170 2024-09-16T15:49:43.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:43.958+0000 7f3028c00640 1 -- 172.21.15.151:0/1684993307 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3024005290 con 0x7f302c0af170 2024-09-16T15:49:43.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:43.958+0000 7f3033800640 1 -- 172.21.15.151:0/1684993307 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f302c149ca0 con 0x7f302c0af170 2024-09-16T15:49:43.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:43.959+0000 7f3033800640 1 -- 172.21.15.151:0/1684993307 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f302c14a260 con 0x7f302c0af170 2024-09-16T15:49:43.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:43.964+0000 7f3033800640 1 -- 172.21.15.151:0/1684993307 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f302c070000 con 0x7f302c0af170 2024-09-16T15:49:43.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:43.967+0000 7f3028c00640 1 -- 172.21.15.151:0/1684993307 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 34) ==== 103225+0+0 (secure 0 0 0) 0x7f3024005430 con 0x7f302c0af170 2024-09-16T15:49:43.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:43.968+0000 7f3028c00640 1 --2- 172.21.15.151:0/1684993307 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3008079db0 0x7f300807c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:43.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:43.968+0000 7f3028c00640 1 -- 172.21.15.151:0/1684993307 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7f302409a450 con 0x7f302c0af170 2024-09-16T15:49:43.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:43.977+0000 7f3028c00640 1 -- 172.21.15.151:0/1684993307 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f3024063ac0 con 0x7f302c0af170 2024-09-16T15:49:43.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:43.980+0000 7f302be00640 1 --2- 172.21.15.151:0/1684993307 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3008079db0 0x7f300807c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:43.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:43.988+0000 7f302be00640 1 --2- 172.21.15.151:0/1684993307 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3008079db0 0x7f300807c270 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f3014034000 tx=0x7f3014004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:49:44.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:44.464+0000 7f3033800640 1 -- 172.21.15.151:0/1684993307 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone277913", "target": ["mon-mgr", ""]}) -- 0x7f302c0adbf0 con 0x7f3008079db0 2024-09-16T15:49:45.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:45.734+0000 7f3028c00640 1 -- 172.21.15.151:0/1684993307 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f302c0adbf0 con 0x7f3008079db0 2024-09-16T15:49:45.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:45.740+0000 7f3033800640 1 -- 172.21.15.151:0/1684993307 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3008079db0 msgr2=0x7f300807c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:45.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:45.740+0000 7f3033800640 1 --2- 172.21.15.151:0/1684993307 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3008079db0 0x7f300807c270 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f3014034000 tx=0x7f3014004730 comp rx=0 tx=0).stop 2024-09-16T15:49:45.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:45.740+0000 7f3033800640 1 -- 172.21.15.151:0/1684993307 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f302c0af170 msgr2=0x7f302c1f8760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:45.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:45.740+0000 7f3033800640 1 --2- 172.21.15.151:0/1684993307 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f302c0af170 0x7f302c1f8760 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f3024001fa0 tx=0x7f302400a430 comp rx=0 tx=0).stop 2024-09-16T15:49:45.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:45.742+0000 7f3033800640 1 -- 172.21.15.151:0/1684993307 shutdown_connections 2024-09-16T15:49:45.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:45.742+0000 7f3033800640 1 --2- 172.21.15.151:0/1684993307 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3008079db0 0x7f300807c270 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:45.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:45.742+0000 7f3033800640 1 --2- 172.21.15.151:0/1684993307 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f302c161490 0x7f302c0b0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:45.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:45.742+0000 7f3033800640 1 --2- 172.21.15.151:0/1684993307 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f302c161bb0 0x7f302c146d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:45.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:45.742+0000 7f3033800640 1 --2- 172.21.15.151:0/1684993307 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f302c0af170 0x7f302c1f8760 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:45.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:45.742+0000 7f3033800640 1 -- 172.21.15.151:0/1684993307 >> 172.21.15.151:0/1684993307 conn(0x7f302c0b1840 msgr2=0x7f302c073e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:49:45.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:45.743+0000 7f3033800640 1 -- 172.21.15.151:0/1684993307 shutdown_connections 2024-09-16T15:49:45.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:45.743+0000 7f3033800640 1 -- 172.21.15.151:0/1684993307 wait complete. 2024-09-16T15:49:45.776 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone880714 2024-09-16T15:49:46.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:46.091+0000 7ff4cb200640 1 -- 172.21.15.151:0/4216241731 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff4b0004800 msgr2=0x7ff4b0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:46.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:46.091+0000 7ff4cb200640 1 --2- 172.21.15.151:0/4216241731 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff4b0004800 0x7ff4b0004c80 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc00c710 tx=0x7ff4b0005830 comp rx=0 tx=0).stop 2024-09-16T15:49:46.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:46.091+0000 7ff4ca800640 1 -- 172.21.15.151:0/4216241731 reap_dead start 2024-09-16T15:49:46.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:46.091+0000 7ff4cb200640 1 -- 172.21.15.151:0/4216241731 shutdown_connections 2024-09-16T15:49:46.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:46.091+0000 7ff4cb200640 1 -- 172.21.15.151:0/4216241731 >> 172.21.15.151:0/4216241731 conn(0x7ff4cc0d3e60 msgr2=0x7ff4cc0d4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:49:46.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:46.092+0000 7ff4cb200640 1 -- 172.21.15.151:0/4216241731 shutdown_connections 2024-09-16T15:49:46.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:46.092+0000 7ff4cb200640 1 -- 172.21.15.151:0/4216241731 wait complete. 2024-09-16T15:49:46.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:46.097+0000 7ff4cb200640 1 Processor -- start 2024-09-16T15:49:46.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:46.097+0000 7ff4cb200640 1 -- start start 2024-09-16T15:49:46.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:46.099+0000 7ff4cb200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4b0004800 0x7ff4cc0c9c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:46.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:46.099+0000 7ff4cb200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff4cc15d590 0x7ff4cc0c9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:46.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:46.099+0000 7ff4cb200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4cc0ccc50 0x7ff4cc0cd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:46.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:46.099+0000 7ff4cb200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff4cc0b82e0 con 0x7ff4cc15d590 2024-09-16T15:49:46.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:46.099+0000 7ff4cb200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff4cc0b8160 con 0x7ff4b0004800 2024-09-16T15:49:46.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:46.099+0000 7ff4cb200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7ff4cc0b8460 con 0x7ff4cc0ccc50 2024-09-16T15:49:46.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:46.099+0000 7ff4c3e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff4cc15d590 0x7ff4cc0c9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:46.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:46.099+0000 7ff4c3e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff4cc15d590 0x7ff4cc0c9370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:54918/0 (socket says 172.21.15.151:54918) 2024-09-16T15:49:46.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:46.099+0000 7ff4c3e00640 1 -- 172.21.15.151:0/3300973887 learned_addr learned my addr 172.21.15.151:0/3300973887 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:49:46.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:46.100+0000 7ff4ca800640 1 --2- 172.21.15.151:0/3300973887 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4cc0ccc50 0x7ff4cc0cd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:46.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:46.100+0000 7ff4c9e00640 1 --2- 172.21.15.151:0/3300973887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4b0004800 0x7ff4cc0c9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:46.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:46.100+0000 7ff4c3e00640 1 -- 172.21.15.151:0/3300973887 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4cc0ccc50 msgr2=0x7ff4cc0cd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:46.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:46.100+0000 7ff4c3e00640 1 --2- 172.21.15.151:0/3300973887 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4cc0ccc50 0x7ff4cc0cd0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:46.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:46.100+0000 7ff4c3e00640 1 -- 172.21.15.151:0/3300973887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4b0004800 msgr2=0x7ff4cc0c9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:46.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:46.100+0000 7ff4c3e00640 1 --2- 172.21.15.151:0/3300973887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4b0004800 0x7ff4cc0c9c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:46.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:46.100+0000 7ff4c3e00640 1 -- 172.21.15.151:0/3300973887 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4cc1f8960 con 0x7ff4cc15d590 2024-09-16T15:49:46.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:46.100+0000 7ff4c9e00640 1 --2- 172.21.15.151:0/3300973887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4b0004800 0x7ff4cc0c9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:49:46.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:46.101+0000 7ff4c3e00640 1 --2- 172.21.15.151:0/3300973887 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff4cc15d590 0x7ff4cc0c9370 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc00c630 tx=0x7ff4bc006310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:49:46.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:46.101+0000 7ff4c2000640 1 -- 172.21.15.151:0/3300973887 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc004a30 con 0x7ff4cc15d590 2024-09-16T15:49:46.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:46.101+0000 7ff4c2000640 1 -- 172.21.15.151:0/3300973887 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff4bc039070 con 0x7ff4cc15d590 2024-09-16T15:49:46.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:46.101+0000 7ff4cb200640 1 -- 172.21.15.151:0/3300973887 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4cc1fc380 con 0x7ff4cc15d590 2024-09-16T15:49:46.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:46.101+0000 7ff4c2000640 1 -- 172.21.15.151:0/3300973887 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc006500 con 0x7ff4cc15d590 2024-09-16T15:49:46.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:46.101+0000 7ff4cb200640 1 -- 172.21.15.151:0/3300973887 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4cc1fc940 con 0x7ff4cc15d590 2024-09-16T15:49:46.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:46.103+0000 7ff4cb200640 1 -- 172.21.15.151:0/3300973887 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4cc0b89a0 con 0x7ff4cc15d590 2024-09-16T15:49:46.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:46.104+0000 7ff4c2000640 1 -- 172.21.15.151:0/3300973887 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 34) ==== 103225+0+0 (secure 0 0 0) 0x7ff4bc03b070 con 0x7ff4cc15d590 2024-09-16T15:49:46.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:46.105+0000 7ff4c2000640 1 --2- 172.21.15.151:0/3300973887 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff4a8079de0 0x7ff4a807c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:46.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:46.106+0000 7ff4c2000640 1 -- 172.21.15.151:0/3300973887 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7ff4bc0b8bb0 con 0x7ff4cc15d590 2024-09-16T15:49:46.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:46.112+0000 7ff4c2000640 1 -- 172.21.15.151:0/3300973887 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff4bc082c70 con 0x7ff4cc15d590 2024-09-16T15:49:46.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:46.188+0000 7ff4c9e00640 1 --2- 172.21.15.151:0/3300973887 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff4a8079de0 0x7ff4a807c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:46.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:46.250+0000 7ff4c9e00640 1 --2- 172.21.15.151:0/3300973887 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff4a8079de0 0x7ff4a807c2a0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ff4b4001920 tx=0x7ff4b4009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:49:46.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:46.499+0000 7ff4cb200640 1 -- 172.21.15.151:0/3300973887 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone880714", "target": ["mon-mgr", ""]}) -- 0x7ff4cc1f8b30 con 0x7ff4a8079de0 2024-09-16T15:49:46.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:46.917+0000 7ff4c2000640 1 -- 172.21.15.151:0/3300973887 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff4cc1f8b30 con 0x7ff4a8079de0 2024-09-16T15:49:46.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:46.920+0000 7ff4cb200640 1 -- 172.21.15.151:0/3300973887 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff4a8079de0 msgr2=0x7ff4a807c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:46.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:46.920+0000 7ff4cb200640 1 --2- 172.21.15.151:0/3300973887 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff4a8079de0 0x7ff4a807c2a0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ff4b4001920 tx=0x7ff4b4009040 comp rx=0 tx=0).stop 2024-09-16T15:49:46.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:46.920+0000 7ff4cb200640 1 -- 172.21.15.151:0/3300973887 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff4cc15d590 msgr2=0x7ff4cc0c9370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:46.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:46.920+0000 7ff4cb200640 1 --2- 172.21.15.151:0/3300973887 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff4cc15d590 0x7ff4cc0c9370 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc00c630 tx=0x7ff4bc006310 comp rx=0 tx=0).stop 2024-09-16T15:49:46.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:46.920+0000 7ff4cb200640 1 -- 172.21.15.151:0/3300973887 shutdown_connections 2024-09-16T15:49:46.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:46.920+0000 7ff4cb200640 1 --2- 172.21.15.151:0/3300973887 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff4a8079de0 0x7ff4a807c2a0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:46.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:46.920+0000 7ff4cb200640 1 --2- 172.21.15.151:0/3300973887 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4cc0ccc50 0x7ff4cc0cd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:46.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:46.920+0000 7ff4cb200640 1 --2- 172.21.15.151:0/3300973887 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff4cc15d590 0x7ff4cc0c9370 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:46.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:46.920+0000 7ff4cb200640 1 --2- 172.21.15.151:0/3300973887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4b0004800 0x7ff4cc0c9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:46.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:46.920+0000 7ff4cb200640 1 -- 172.21.15.151:0/3300973887 >> 172.21.15.151:0/3300973887 conn(0x7ff4cc0d3e60 msgr2=0x7ff4cc0d0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:49:46.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:46.920+0000 7ff4cb200640 1 -- 172.21.15.151:0/3300973887 shutdown_connections 2024-09-16T15:49:46.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:46.920+0000 7ff4cb200640 1 -- 172.21.15.151:0/3300973887 wait complete. 2024-09-16T15:49:46.960 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone139815 2024-09-16T15:49:47.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:47.293+0000 7f8ba3200640 1 -- 172.21.15.151:0/4244344048 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ba40cf110 msgr2=0x7f8ba40d2890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:47.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:47.293+0000 7f8ba3200640 1 --2- 172.21.15.151:0/4244344048 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ba40cf110 0x7f8ba40d2890 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c005fd0 tx=0x7f8b8c02a4a0 comp rx=0 tx=0).stop 2024-09-16T15:49:47.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:47.293+0000 7f8ba3200640 1 -- 172.21.15.151:0/4244344048 shutdown_connections 2024-09-16T15:49:47.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:47.293+0000 7f8ba3200640 1 --2- 172.21.15.151:0/4244344048 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ba40cf110 0x7f8ba40d2890 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:47.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:47.293+0000 7f8ba3200640 1 --2- 172.21.15.151:0/4244344048 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8ba40ced30 0x7f8ba40b4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:47.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:47.293+0000 7f8ba3200640 1 -- 172.21.15.151:0/4244344048 >> 172.21.15.151:0/4244344048 conn(0x7f8ba40d3e20 msgr2=0x7f8ba40d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:49:47.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:47.293+0000 7f8ba3200640 1 -- 172.21.15.151:0/4244344048 shutdown_connections 2024-09-16T15:49:47.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:47.293+0000 7f8ba3200640 1 -- 172.21.15.151:0/4244344048 wait complete. 2024-09-16T15:49:47.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:47.296+0000 7f8ba3200640 1 Processor -- start 2024-09-16T15:49:47.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:47.296+0000 7f8ba3200640 1 -- start start 2024-09-16T15:49:47.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:47.297+0000 7f8ba3200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ba40b6c50 0x7f8ba4155bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:47.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:47.297+0000 7f8ba3200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ba40ced30 0x7f8ba40c9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:47.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:47.297+0000 7f8ba3200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8ba40cf110 0x7f8ba41f7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:47.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:47.297+0000 7f8ba3200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba40d07c0 con 0x7f8ba40b6c50 2024-09-16T15:49:47.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:47.297+0000 7f8ba3200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba40d0640 con 0x7f8ba40ced30 2024-09-16T15:49:47.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:47.297+0000 7f8ba3200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f8ba40d0940 con 0x7f8ba40cf110 2024-09-16T15:49:47.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:47.297+0000 7f8ba1e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ba40b6c50 0x7f8ba4155bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:47.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:47.297+0000 7f8ba1e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ba40b6c50 0x7f8ba4155bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:54942/0 (socket says 172.21.15.151:54942) 2024-09-16T15:49:47.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:47.297+0000 7f8ba1e00640 1 -- 172.21.15.151:0/3353363879 learned_addr learned my addr 172.21.15.151:0/3353363879 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:49:47.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:47.298+0000 7f8ba1e00640 1 -- 172.21.15.151:0/3353363879 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8ba40cf110 msgr2=0x7f8ba41f7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:47.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:47.298+0000 7f8ba1e00640 1 --2- 172.21.15.151:0/3353363879 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8ba40cf110 0x7f8ba41f7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:47.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:47.298+0000 7f8ba1e00640 1 -- 172.21.15.151:0/3353363879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ba40ced30 msgr2=0x7f8ba40c9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:47.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:47.298+0000 7f8ba1e00640 1 --2- 172.21.15.151:0/3353363879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ba40ced30 0x7f8ba40c9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:47.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:47.298+0000 7f8ba1e00640 1 -- 172.21.15.151:0/3353363879 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ba415b2b0 con 0x7f8ba40b6c50 2024-09-16T15:49:47.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:47.298+0000 7f8ba1e00640 1 --2- 172.21.15.151:0/3353363879 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ba40b6c50 0x7f8ba4155bd0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f8b9400db50 tx=0x7f8b9400b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:49:47.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:47.298+0000 7f8b9aa00640 1 -- 172.21.15.151:0/3353363879 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b94017070 con 0x7f8ba40b6c50 2024-09-16T15:49:47.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:47.299+0000 7f8b9aa00640 1 -- 172.21.15.151:0/3353363879 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8b9400de50 con 0x7f8ba40b6c50 2024-09-16T15:49:47.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:47.299+0000 7f8ba3200640 1 -- 172.21.15.151:0/3353363879 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ba415b590 con 0x7f8ba40b6c50 2024-09-16T15:49:47.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:47.299+0000 7f8b9aa00640 1 -- 172.21.15.151:0/3353363879 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b940106a0 con 0x7f8ba40b6c50 2024-09-16T15:49:47.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:47.299+0000 7f8ba3200640 1 -- 172.21.15.151:0/3353363879 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ba415bb50 con 0x7f8ba40b6c50 2024-09-16T15:49:47.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:47.300+0000 7f8ba3200640 1 -- 172.21.15.151:0/3353363879 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ba40b4920 con 0x7f8ba40b6c50 2024-09-16T15:49:47.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:47.301+0000 7f8b9aa00640 1 -- 172.21.15.151:0/3353363879 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 34) ==== 103225+0+0 (secure 0 0 0) 0x7f8b9401a070 con 0x7f8ba40b6c50 2024-09-16T15:49:47.304 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:47.302+0000 7f8b9aa00640 1 --2- 172.21.15.151:0/3353363879 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8b80079db0 0x7f8b8007c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:47.304 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:47.302+0000 7f8b9aa00640 1 -- 172.21.15.151:0/3353363879 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7f8b9409a050 con 0x7f8ba40b6c50 2024-09-16T15:49:47.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:47.308+0000 7f8b9aa00640 1 -- 172.21.15.151:0/3353363879 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8b94063c90 con 0x7f8ba40b6c50 2024-09-16T15:49:47.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:47.330+0000 7f8b99400640 1 --2- 172.21.15.151:0/3353363879 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8b80079db0 0x7f8b8007c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:47.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:47.354+0000 7f8b99400640 1 --2- 172.21.15.151:0/3353363879 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8b80079db0 0x7f8b8007c270 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c033000 tx=0x7f8b8c0046c0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:49:47.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:47.694+0000 7f8ba3200640 1 -- 172.21.15.151:0/3353363879 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone139815", "target": ["mon-mgr", ""]}) -- 0x7f8ba415b060 con 0x7f8b80079db0 2024-09-16T15:49:48.205 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:48.203+0000 7f8b9aa00640 1 -- 172.21.15.151:0/3353363879 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8ba415b060 con 0x7f8b80079db0 2024-09-16T15:49:48.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:48.209+0000 7f8ba3200640 1 -- 172.21.15.151:0/3353363879 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8b80079db0 msgr2=0x7f8b8007c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:48.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:48.209+0000 7f8ba3200640 1 --2- 172.21.15.151:0/3353363879 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8b80079db0 0x7f8b8007c270 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c033000 tx=0x7f8b8c0046c0 comp rx=0 tx=0).stop 2024-09-16T15:49:48.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:48.209+0000 7f8ba3200640 1 -- 172.21.15.151:0/3353363879 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ba40b6c50 msgr2=0x7f8ba4155bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:48.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:48.209+0000 7f8ba3200640 1 --2- 172.21.15.151:0/3353363879 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ba40b6c50 0x7f8ba4155bd0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f8b9400db50 tx=0x7f8b9400b630 comp rx=0 tx=0).stop 2024-09-16T15:49:48.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:48.209+0000 7f8ba3200640 1 -- 172.21.15.151:0/3353363879 shutdown_connections 2024-09-16T15:49:48.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:48.209+0000 7f8ba3200640 1 --2- 172.21.15.151:0/3353363879 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8b80079db0 0x7f8b8007c270 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:48.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:48.209+0000 7f8ba3200640 1 --2- 172.21.15.151:0/3353363879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ba40ced30 0x7f8ba40c9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:48.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:48.209+0000 7f8ba3200640 1 --2- 172.21.15.151:0/3353363879 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8ba40cf110 0x7f8ba41f7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:48.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:48.209+0000 7f8ba3200640 1 --2- 172.21.15.151:0/3353363879 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ba40b6c50 0x7f8ba4155bd0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:48.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:48.209+0000 7f8ba3200640 1 -- 172.21.15.151:0/3353363879 >> 172.21.15.151:0/3353363879 conn(0x7f8ba40d3e20 msgr2=0x7f8ba40d1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:49:48.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:48.209+0000 7f8ba3200640 1 -- 172.21.15.151:0/3353363879 shutdown_connections 2024-09-16T15:49:48.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:48.210+0000 7f8ba3200640 1 -- 172.21.15.151:0/3353363879 wait complete. 2024-09-16T15:49:48.246 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone446916 2024-09-16T15:49:48.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:48.553+0000 7f42a7e00640 1 -- 172.21.15.151:0/678717553 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4288004800 msgr2=0x7f4288004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:48.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:48.553+0000 7f42a7e00640 1 --2- 172.21.15.151:0/678717553 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4288004800 0x7f4288004c80 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f428c00bc80 tx=0x7f4288005830 comp rx=0 tx=0).stop 2024-09-16T15:49:48.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:48.553+0000 7f42a5e00640 1 -- 172.21.15.151:0/678717553 reap_dead start 2024-09-16T15:49:48.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:48.553+0000 7f42a7e00640 1 -- 172.21.15.151:0/678717553 shutdown_connections 2024-09-16T15:49:48.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:48.553+0000 7f42a7e00640 1 -- 172.21.15.151:0/678717553 >> 172.21.15.151:0/678717553 conn(0x7f42a00b1840 msgr2=0x7f42a00b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:49:48.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:48.554+0000 7f42a7e00640 1 -- 172.21.15.151:0/678717553 shutdown_connections 2024-09-16T15:49:48.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:48.554+0000 7f42a7e00640 1 -- 172.21.15.151:0/678717553 wait complete. 2024-09-16T15:49:48.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:48.556+0000 7f42a7e00640 1 Processor -- start 2024-09-16T15:49:48.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:48.557+0000 7f42a7e00640 1 -- start start 2024-09-16T15:49:48.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:48.558+0000 7f42a7e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4288004800 0x7f42a01f8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:48.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:48.558+0000 7f42a7e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f42a0154d80 0x7f42a014f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:48.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:48.558+0000 7f42a7e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42a015dc20 0x7f42a0150540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:48.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:48.558+0000 7f42a7e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f42a00cf210 con 0x7f42a0154d80 2024-09-16T15:49:48.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:48.558+0000 7f42a7e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f42a00cf090 con 0x7f4288004800 2024-09-16T15:49:48.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:48.558+0000 7f42a7e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f42a00cf390 con 0x7f42a015dc20 2024-09-16T15:49:48.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:48.558+0000 7f42a4a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f42a0154d80 0x7f42a014f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:48.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:48.558+0000 7f42a4a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f42a0154d80 0x7f42a014f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:54970/0 (socket says 172.21.15.151:54970) 2024-09-16T15:49:48.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:48.558+0000 7f42a4a00640 1 -- 172.21.15.151:0/3202156587 learned_addr learned my addr 172.21.15.151:0/3202156587 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:49:48.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:48.559+0000 7f42a5e00640 1 --2- 172.21.15.151:0/3202156587 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42a015dc20 0x7f42a0150540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:48.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:48.559+0000 7f42a5400640 1 --2- 172.21.15.151:0/3202156587 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4288004800 0x7f42a01f8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:48.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:48.559+0000 7f42a4a00640 1 -- 172.21.15.151:0/3202156587 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42a015dc20 msgr2=0x7f42a0150540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:48.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:48.559+0000 7f42a4a00640 1 --2- 172.21.15.151:0/3202156587 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42a015dc20 0x7f42a0150540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:48.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:48.559+0000 7f42a4a00640 1 -- 172.21.15.151:0/3202156587 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4288004800 msgr2=0x7f42a01f8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:48.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:48.559+0000 7f42a4a00640 1 --2- 172.21.15.151:0/3202156587 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4288004800 0x7f42a01f8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:48.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:48.559+0000 7f42a4a00640 1 -- 172.21.15.151:0/3202156587 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42a01f4c40 con 0x7f42a0154d80 2024-09-16T15:49:48.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:48.560+0000 7f42a4a00640 1 --2- 172.21.15.151:0/3202156587 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f42a0154d80 0x7f42a014f450 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f428c02f040 tx=0x7f428c02cff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:49:48.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:48.560+0000 7f429d600640 1 -- 172.21.15.151:0/3202156587 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f428c034040 con 0x7f42a0154d80 2024-09-16T15:49:48.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:48.560+0000 7f429d600640 1 -- 172.21.15.151:0/3202156587 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f428c005020 con 0x7f42a0154d80 2024-09-16T15:49:48.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:48.560+0000 7f429d600640 1 -- 172.21.15.151:0/3202156587 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f428c005450 con 0x7f42a0154d80 2024-09-16T15:49:48.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:48.560+0000 7f42a7e00640 1 -- 172.21.15.151:0/3202156587 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42a01f8700 con 0x7f42a0154d80 2024-09-16T15:49:48.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:48.560+0000 7f42a7e00640 1 -- 172.21.15.151:0/3202156587 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42a01f8cc0 con 0x7f42a0154d80 2024-09-16T15:49:48.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:48.562+0000 7f42a7e00640 1 -- 172.21.15.151:0/3202156587 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42a00b2b40 con 0x7f42a0154d80 2024-09-16T15:49:48.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:48.564+0000 7f429d600640 1 -- 172.21.15.151:0/3202156587 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 34) ==== 103225+0+0 (secure 0 0 0) 0x7f428c037070 con 0x7f42a0154d80 2024-09-16T15:49:48.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:48.565+0000 7f429d600640 1 --2- 172.21.15.151:0/3202156587 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4280079de0 0x7f428007c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:48.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:48.566+0000 7f429d600640 1 -- 172.21.15.151:0/3202156587 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7f428c0b9eb0 con 0x7f42a0154d80 2024-09-16T15:49:48.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:48.573+0000 7f429d600640 1 -- 172.21.15.151:0/3202156587 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f428c083580 con 0x7f42a0154d80 2024-09-16T15:49:48.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:48.583+0000 7f42a5400640 1 --2- 172.21.15.151:0/3202156587 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4280079de0 0x7f428007c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:48.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:48.593+0000 7f42a5400640 1 --2- 172.21.15.151:0/3202156587 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4280079de0 0x7f428007c2a0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f4298001a70 tx=0x7f4298006a60 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:49:49.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:49.009+0000 7f42a7e00640 1 -- 172.21.15.151:0/3202156587 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone446916", "target": ["mon-mgr", ""]}) -- 0x7f42a00b4c90 con 0x7f4280079de0 2024-09-16T15:49:49.358 DEBUG:teuthology.orchestra.run.smithi151:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:49:49.367 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:49:49.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:49.873+0000 7f429d600640 1 -- 172.21.15.151:0/3202156587 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f42a00b4c90 con 0x7f4280079de0 2024-09-16T15:49:49.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:49.877+0000 7f42a7e00640 1 -- 172.21.15.151:0/3202156587 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4280079de0 msgr2=0x7f428007c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:49.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:49.877+0000 7f42a7e00640 1 --2- 172.21.15.151:0/3202156587 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4280079de0 0x7f428007c2a0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f4298001a70 tx=0x7f4298006a60 comp rx=0 tx=0).stop 2024-09-16T15:49:49.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:49.877+0000 7f42a7e00640 1 -- 172.21.15.151:0/3202156587 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f42a0154d80 msgr2=0x7f42a014f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:49.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:49.877+0000 7f42a7e00640 1 --2- 172.21.15.151:0/3202156587 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f42a0154d80 0x7f42a014f450 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f428c02f040 tx=0x7f428c02cff0 comp rx=0 tx=0).stop 2024-09-16T15:49:49.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:49.877+0000 7f42a7e00640 1 -- 172.21.15.151:0/3202156587 shutdown_connections 2024-09-16T15:49:49.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:49.877+0000 7f42a7e00640 1 --2- 172.21.15.151:0/3202156587 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4280079de0 0x7f428007c2a0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:49.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:49.877+0000 7f42a7e00640 1 --2- 172.21.15.151:0/3202156587 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42a015dc20 0x7f42a0150540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:49.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:49.877+0000 7f42a7e00640 1 --2- 172.21.15.151:0/3202156587 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f42a0154d80 0x7f42a014f450 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:49.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:49.877+0000 7f42a7e00640 1 --2- 172.21.15.151:0/3202156587 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4288004800 0x7f42a01f8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:49.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:49.877+0000 7f42a7e00640 1 -- 172.21.15.151:0/3202156587 >> 172.21.15.151:0/3202156587 conn(0x7f42a00b1840 msgr2=0x7f42a00ad870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:49:49.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:49.878+0000 7f42a7e00640 1 -- 172.21.15.151:0/3202156587 shutdown_connections 2024-09-16T15:49:49.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:49.878+0000 7f42a7e00640 1 -- 172.21.15.151:0/3202156587 wait complete. 2024-09-16T15:49:49.914 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone962717 2024-09-16T15:49:50.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:50.248+0000 7f792c400640 1 -- 172.21.15.151:0/1569307545 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79240b2db0 msgr2=0x7f792415eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:50.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:50.248+0000 7f792c400640 1 --2- 172.21.15.151:0/1569307545 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79240b2db0 0x7f792415eb00 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f7918005fd0 tx=0x7f791802a4a0 comp rx=0 tx=0).stop 2024-09-16T15:49:50.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:50.248+0000 7f792c400640 1 -- 172.21.15.151:0/1569307545 shutdown_connections 2024-09-16T15:49:50.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:50.248+0000 7f792c400640 1 --2- 172.21.15.151:0/1569307545 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79240cebe0 0x7f79240cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:50.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:50.248+0000 7f792c400640 1 --2- 172.21.15.151:0/1569307545 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79240b2db0 0x7f792415eb00 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:50.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:50.248+0000 7f792c400640 1 -- 172.21.15.151:0/1569307545 >> 172.21.15.151:0/1569307545 conn(0x7f79240b1840 msgr2=0x7f79240b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:49:50.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:50.249+0000 7f792c400640 1 -- 172.21.15.151:0/1569307545 shutdown_connections 2024-09-16T15:49:50.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:50.249+0000 7f792c400640 1 -- 172.21.15.151:0/1569307545 wait complete. 2024-09-16T15:49:50.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:50.251+0000 7f792c400640 1 Processor -- start 2024-09-16T15:49:50.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:50.251+0000 7f792c400640 1 -- start start 2024-09-16T15:49:50.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:50.252+0000 7f792c400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79240b2db0 0x7f7924151ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:50.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:50.252+0000 7f792c400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79240b3190 0x7f79241597a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:50.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:50.252+0000 7f792c400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79240cebe0 0x7f7924153d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:50.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:50.252+0000 7f792c400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f79240cf710 con 0x7f79240cebe0 2024-09-16T15:49:50.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:50.252+0000 7f792c400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f79240cf590 con 0x7f79240b2db0 2024-09-16T15:49:50.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:50.252+0000 7f792c400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f79240cf890 con 0x7f79240b3190 2024-09-16T15:49:50.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:50.253+0000 7f792a400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79240cebe0 0x7f7924153d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:50.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:50.253+0000 7f7929000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79240b3190 0x7f79241597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:50.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:50.253+0000 7f7929a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79240b2db0 0x7f7924151ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:50.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:50.253+0000 7f792a400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79240cebe0 0x7f7924153d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:54980/0 (socket says 172.21.15.151:54980) 2024-09-16T15:49:50.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:50.253+0000 7f792a400640 1 -- 172.21.15.151:0/2973824615 learned_addr learned my addr 172.21.15.151:0/2973824615 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:49:50.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:50.254+0000 7f792a400640 1 -- 172.21.15.151:0/2973824615 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79240b3190 msgr2=0x7f79241597a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:50.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:50.254+0000 7f792a400640 1 --2- 172.21.15.151:0/2973824615 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79240b3190 0x7f79241597a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:50.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:50.254+0000 7f792a400640 1 -- 172.21.15.151:0/2973824615 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79240b2db0 msgr2=0x7f7924151ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:50.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:50.254+0000 7f792a400640 1 --2- 172.21.15.151:0/2973824615 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79240b2db0 0x7f7924151ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:50.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:50.254+0000 7f792a400640 1 -- 172.21.15.151:0/2973824615 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79241f46d0 con 0x7f79240cebe0 2024-09-16T15:49:50.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:50.254+0000 7f792a400640 1 --2- 172.21.15.151:0/2973824615 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79240cebe0 0x7f7924153d00 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f791c00eab0 tx=0x7f791c00ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:49:50.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:50.254+0000 7f7921600640 1 -- 172.21.15.151:0/2973824615 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f791c00ce40 con 0x7f79240cebe0 2024-09-16T15:49:50.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:50.254+0000 7f7921600640 1 -- 172.21.15.151:0/2973824615 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f791c004b20 con 0x7f79240cebe0 2024-09-16T15:49:50.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:50.254+0000 7f7921600640 1 -- 172.21.15.151:0/2973824615 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f791c005020 con 0x7f79240cebe0 2024-09-16T15:49:50.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:50.255+0000 7f792c400640 1 -- 172.21.15.151:0/2973824615 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79241f80f0 con 0x7f79240cebe0 2024-09-16T15:49:50.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:50.255+0000 7f792c400640 1 -- 172.21.15.151:0/2973824615 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79241f86b0 con 0x7f79240cebe0 2024-09-16T15:49:50.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:50.258+0000 7f792c400640 1 -- 172.21.15.151:0/2973824615 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79240cf050 con 0x7f79240cebe0 2024-09-16T15:49:50.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:50.258+0000 7f7921600640 1 -- 172.21.15.151:0/2973824615 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 34) ==== 103225+0+0 (secure 0 0 0) 0x7f791c0051c0 con 0x7f79240cebe0 2024-09-16T15:49:50.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:50.260+0000 7f7921600640 1 --2- 172.21.15.151:0/2973824615 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7904079db0 0x7f790407c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:50.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:50.260+0000 7f7921600640 1 -- 172.21.15.151:0/2973824615 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7f791c09a060 con 0x7f79240cebe0 2024-09-16T15:49:50.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:50.266+0000 7f7921600640 1 -- 172.21.15.151:0/2973824615 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f791c063b30 con 0x7f79240cebe0 2024-09-16T15:49:50.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:50.271+0000 7f7929a00640 1 --2- 172.21.15.151:0/2973824615 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7904079db0 0x7f790407c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:50.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:50.281+0000 7f7929a00640 1 --2- 172.21.15.151:0/2973824615 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7904079db0 0x7f790407c270 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f7918033000 tx=0x7f7918004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:49:50.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:50.721+0000 7f792c400640 1 -- 172.21.15.151:0/2973824615 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone962717", "target": ["mon-mgr", ""]}) -- 0x7f7924069a10 con 0x7f7904079db0 2024-09-16T15:49:51.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:51.813+0000 7f7921600640 1 -- 172.21.15.151:0/2973824615 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7924069a10 con 0x7f7904079db0 2024-09-16T15:49:51.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:51.819+0000 7f792c400640 1 -- 172.21.15.151:0/2973824615 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7904079db0 msgr2=0x7f790407c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:51.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:51.819+0000 7f792c400640 1 --2- 172.21.15.151:0/2973824615 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7904079db0 0x7f790407c270 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f7918033000 tx=0x7f7918004730 comp rx=0 tx=0).stop 2024-09-16T15:49:51.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:51.819+0000 7f792c400640 1 -- 172.21.15.151:0/2973824615 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79240cebe0 msgr2=0x7f7924153d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:51.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:51.819+0000 7f792c400640 1 --2- 172.21.15.151:0/2973824615 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79240cebe0 0x7f7924153d00 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f791c00eab0 tx=0x7f791c00ef70 comp rx=0 tx=0).stop 2024-09-16T15:49:51.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:51.819+0000 7f792c400640 1 -- 172.21.15.151:0/2973824615 shutdown_connections 2024-09-16T15:49:51.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:51.819+0000 7f792c400640 1 --2- 172.21.15.151:0/2973824615 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7904079db0 0x7f790407c270 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:51.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:51.819+0000 7f792c400640 1 --2- 172.21.15.151:0/2973824615 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79240cebe0 0x7f7924153d00 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:51.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:51.819+0000 7f792c400640 1 --2- 172.21.15.151:0/2973824615 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79240b3190 0x7f79241597a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:51.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:51.819+0000 7f792c400640 1 --2- 172.21.15.151:0/2973824615 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79240b2db0 0x7f7924151ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:51.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:51.819+0000 7f792c400640 1 -- 172.21.15.151:0/2973824615 >> 172.21.15.151:0/2973824615 conn(0x7f79240b1840 msgr2=0x7f79240ae380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:49:51.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:51.819+0000 7f792c400640 1 -- 172.21.15.151:0/2973824615 shutdown_connections 2024-09-16T15:49:51.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:51.819+0000 7f792c400640 1 -- 172.21.15.151:0/2973824615 wait complete. 2024-09-16T15:49:51.853 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone499718 2024-09-16T15:49:52.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:52.163+0000 7f9e46a00640 1 -- 172.21.15.151:0/2812856456 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e380047a0 con 0x7f9e480ce6c0 2024-09-16T15:49:52.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:52.164+0000 7f9e4d800640 1 -- 172.21.15.151:0/2812856456 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9e480ce6c0 msgr2=0x7f9e480cb010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:52.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:52.164+0000 7f9e4d800640 1 --2- 172.21.15.151:0/2812856456 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9e480ce6c0 0x7f9e480cb010 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f9e480c6850 tx=0x7f9e3802a4f0 comp rx=0 tx=0).stop 2024-09-16T15:49:52.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:52.168+0000 7f9e4d800640 1 -- 172.21.15.151:0/2812856456 shutdown_connections 2024-09-16T15:49:52.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:52.168+0000 7f9e4d800640 1 --2- 172.21.15.151:0/2812856456 >> v2:172.21.15.167:3300/0 conn(0x7f9e480cb550 0x7f9e480cb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:52.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:52.168+0000 7f9e4d800640 1 --2- 172.21.15.151:0/2812856456 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9e480ce6c0 0x7f9e480cb010 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:52.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:52.168+0000 7f9e4d800640 1 -- 172.21.15.151:0/2812856456 >> 172.21.15.151:0/2812856456 conn(0x7f9e480cc490 msgr2=0x7f9e480cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:49:52.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:52.168+0000 7f9e4d800640 1 -- 172.21.15.151:0/2812856456 shutdown_connections 2024-09-16T15:49:52.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:52.168+0000 7f9e4d800640 1 -- 172.21.15.151:0/2812856456 wait complete. 2024-09-16T15:49:52.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:52.174+0000 7f9e4d800640 1 Processor -- start 2024-09-16T15:49:52.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:52.177+0000 7f9e4d800640 1 -- start start 2024-09-16T15:49:52.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:52.187+0000 7f9e4d800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9e480cb550 0x7f9e481e5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:52.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:52.187+0000 7f9e4d800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9e480ce2e0 0x7f9e480c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:52.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:52.187+0000 7f9e4d800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e480ce6c0 0x7f9e480c7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:52.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:52.191+0000 7f9e4d800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4815aed0 con 0x7f9e480ce2e0 2024-09-16T15:49:52.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:52.191+0000 7f9e4d800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4815ad50 con 0x7f9e480ce6c0 2024-09-16T15:49:52.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:52.192+0000 7f9e4d800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f9e480cbf20 con 0x7f9e480cb550 2024-09-16T15:49:52.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:52.192+0000 7f9e47e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9e480cb550 0x7f9e481e5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:52.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:52.192+0000 7f9e47400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9e480ce2e0 0x7f9e480c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:52.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:52.192+0000 7f9e47e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9e480cb550 0x7f9e481e5700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.151:51764/0 (socket says 172.21.15.151:51764) 2024-09-16T15:49:52.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:52.192+0000 7f9e47400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9e480ce2e0 0x7f9e480c6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:54992/0 (socket says 172.21.15.151:54992) 2024-09-16T15:49:52.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:52.192+0000 7f9e47e00640 1 -- 172.21.15.151:0/944174081 learned_addr learned my addr 172.21.15.151:0/944174081 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:49:52.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:52.192+0000 7f9e4ce00640 1 --2- 172.21.15.151:0/944174081 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e480ce6c0 0x7f9e480c7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:52.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:52.194+0000 7f9e47400640 1 -- 172.21.15.151:0/944174081 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9e480cb550 msgr2=0x7f9e481e5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:52.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:52.194+0000 7f9e47400640 1 --2- 172.21.15.151:0/944174081 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9e480cb550 0x7f9e481e5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:52.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:52.194+0000 7f9e47400640 1 -- 172.21.15.151:0/944174081 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e480ce6c0 msgr2=0x7f9e480c7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:52.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:52.194+0000 7f9e47400640 1 --2- 172.21.15.151:0/944174081 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e480ce6c0 0x7f9e480c7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:52.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:52.194+0000 7f9e47400640 1 -- 172.21.15.151:0/944174081 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e4813f750 con 0x7f9e480ce2e0 2024-09-16T15:49:52.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:52.195+0000 7f9e47400640 1 --2- 172.21.15.151:0/944174081 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9e480ce2e0 0x7f9e480c6c10 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f9e38031000 tx=0x7f9e38003930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:49:52.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:52.197+0000 7f9e3fe00640 1 -- 172.21.15.151:0/944174081 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e3803e030 con 0x7f9e480ce2e0 2024-09-16T15:49:52.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:52.198+0000 7f9e4d800640 1 -- 172.21.15.151:0/944174081 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e4813fa30 con 0x7f9e480ce2e0 2024-09-16T15:49:52.205 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:52.199+0000 7f9e4d800640 1 -- 172.21.15.151:0/944174081 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e4813fff0 con 0x7f9e480ce2e0 2024-09-16T15:49:52.205 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:52.202+0000 7f9e3fe00640 1 -- 172.21.15.151:0/944174081 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9e38003e20 con 0x7f9e480ce2e0 2024-09-16T15:49:52.205 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:52.202+0000 7f9e3fe00640 1 -- 172.21.15.151:0/944174081 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e38035040 con 0x7f9e480ce2e0 2024-09-16T15:49:52.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:52.206+0000 7f9e3fe00640 1 -- 172.21.15.151:0/944174081 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 34) ==== 103225+0+0 (secure 0 0 0) 0x7f9e38041030 con 0x7f9e480ce2e0 2024-09-16T15:49:52.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:52.208+0000 7f9e4d800640 1 -- 172.21.15.151:0/944174081 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e480cb9e0 con 0x7f9e480ce2e0 2024-09-16T15:49:52.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:52.213+0000 7f9e3fe00640 1 --2- 172.21.15.151:0/944174081 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9e28079e30 0x7f9e2807c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:52.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:52.215+0000 7f9e3fe00640 1 -- 172.21.15.151:0/944174081 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7f9e380b96d0 con 0x7f9e480ce2e0 2024-09-16T15:49:52.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:52.216+0000 7f9e47e00640 1 --2- 172.21.15.151:0/944174081 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9e28079e30 0x7f9e2807c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:52.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:52.222+0000 7f9e3fe00640 1 -- 172.21.15.151:0/944174081 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9e38082ba0 con 0x7f9e480ce2e0 2024-09-16T15:49:52.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:52.226+0000 7f9e47e00640 1 --2- 172.21.15.151:0/944174081 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9e28079e30 0x7f9e2807c2f0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f9e480c8a30 tx=0x7f9e34009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:49:52.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:52.667+0000 7f9e4d800640 1 -- 172.21.15.151:0/944174081 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone499718", "target": ["mon-mgr", ""]}) -- 0x7f9e480cd080 con 0x7f9e28079e30 2024-09-16T15:49:55.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:55.699+0000 7f9e3fe00640 1 -- 172.21.15.151:0/944174081 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9e480cd080 con 0x7f9e28079e30 2024-09-16T15:49:55.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:55.703+0000 7f9e4d800640 1 -- 172.21.15.151:0/944174081 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9e28079e30 msgr2=0x7f9e2807c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:55.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:55.703+0000 7f9e4d800640 1 --2- 172.21.15.151:0/944174081 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9e28079e30 0x7f9e2807c2f0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f9e480c8a30 tx=0x7f9e34009040 comp rx=0 tx=0).stop 2024-09-16T15:49:55.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:55.703+0000 7f9e4d800640 1 -- 172.21.15.151:0/944174081 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9e480ce2e0 msgr2=0x7f9e480c6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:55.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:55.703+0000 7f9e4d800640 1 --2- 172.21.15.151:0/944174081 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9e480ce2e0 0x7f9e480c6c10 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f9e38031000 tx=0x7f9e38003930 comp rx=0 tx=0).stop 2024-09-16T15:49:55.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:55.703+0000 7f9e4d800640 1 -- 172.21.15.151:0/944174081 shutdown_connections 2024-09-16T15:49:55.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:55.703+0000 7f9e4d800640 1 --2- 172.21.15.151:0/944174081 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9e28079e30 0x7f9e2807c2f0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:55.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:55.703+0000 7f9e4d800640 1 --2- 172.21.15.151:0/944174081 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e480ce6c0 0x7f9e480c7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:55.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:55.703+0000 7f9e4d800640 1 --2- 172.21.15.151:0/944174081 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9e480ce2e0 0x7f9e480c6c10 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:55.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:55.703+0000 7f9e4d800640 1 --2- 172.21.15.151:0/944174081 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9e480cb550 0x7f9e481e5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:55.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:55.703+0000 7f9e4d800640 1 -- 172.21.15.151:0/944174081 >> 172.21.15.151:0/944174081 conn(0x7f9e480cc490 msgr2=0x7f9e480b3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:49:55.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:55.704+0000 7f9e4d800640 1 -- 172.21.15.151:0/944174081 shutdown_connections 2024-09-16T15:49:55.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:55.704+0000 7f9e4d800640 1 -- 172.21.15.151:0/944174081 wait complete. 2024-09-16T15:49:55.737 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone237619 2024-09-16T15:49:56.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:56.034+0000 7f86e3200640 1 -- 172.21.15.151:0/3502607464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86dc0b4bd0 msgr2=0x7f86dc0b3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:56.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:56.034+0000 7f86e3200640 1 --2- 172.21.15.151:0/3502607464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86dc0b4bd0 0x7f86dc0b3ec0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f86dc0cd280 tx=0x7f86cc02a560 comp rx=0 tx=0).stop 2024-09-16T15:49:56.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:56.035+0000 7f86e3200640 1 -- 172.21.15.151:0/3502607464 shutdown_connections 2024-09-16T15:49:56.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:56.035+0000 7f86e3200640 1 --2- 172.21.15.151:0/3502607464 >> v2:172.21.15.167:3300/0 conn(0x7f86dc070870 0x7f86dc0b4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:56.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:56.035+0000 7f86e3200640 1 --2- 172.21.15.151:0/3502607464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86dc0b4bd0 0x7f86dc0b3ec0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:56.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:56.035+0000 7f86e3200640 1 -- 172.21.15.151:0/3502607464 >> 172.21.15.151:0/3502607464 conn(0x7f86dc0b1840 msgr2=0x7f86dc0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:49:56.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:56.036+0000 7f86e3200640 1 -- 172.21.15.151:0/3502607464 shutdown_connections 2024-09-16T15:49:56.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:56.036+0000 7f86e3200640 1 -- 172.21.15.151:0/3502607464 wait complete. 2024-09-16T15:49:56.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:56.039+0000 7f86e3200640 1 Processor -- start 2024-09-16T15:49:56.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:56.039+0000 7f86e3200640 1 -- start start 2024-09-16T15:49:56.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:56.040+0000 7f86e3200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f86dc070870 0x7f86dc1f2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:56.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:56.040+0000 7f86e3200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86dc0b4bd0 0x7f86dc139bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:56.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:56.040+0000 7f86e3200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86dc1437c0 0x7f86dc1502f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:56.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:56.040+0000 7f86e3200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f86dc165470 con 0x7f86dc0b4bd0 2024-09-16T15:49:56.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:56.040+0000 7f86e3200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f86dc1652f0 con 0x7f86dc1437c0 2024-09-16T15:49:56.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:56.040+0000 7f86e3200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f86dc1655f0 con 0x7f86dc070870 2024-09-16T15:49:56.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:56.041+0000 7f86e1200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86dc1437c0 0x7f86dc1502f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:56.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:56.041+0000 7f86e1200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86dc1437c0 0x7f86dc1502f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.151:53850/0 (socket says 172.21.15.151:53850) 2024-09-16T15:49:56.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:56.041+0000 7f86e1200640 1 -- 172.21.15.151:0/148629474 learned_addr learned my addr 172.21.15.151:0/148629474 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:49:56.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:56.041+0000 7f86db400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86dc0b4bd0 0x7f86dc139bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:56.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:56.041+0000 7f86dbe00640 1 --2- 172.21.15.151:0/148629474 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f86dc070870 0x7f86dc1f2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:56.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:56.042+0000 7f86e1200640 1 -- 172.21.15.151:0/148629474 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f86dc070870 msgr2=0x7f86dc1f2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:56.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:56.042+0000 7f86e1200640 1 --2- 172.21.15.151:0/148629474 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f86dc070870 0x7f86dc1f2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:56.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:56.042+0000 7f86e1200640 1 -- 172.21.15.151:0/148629474 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86dc0b4bd0 msgr2=0x7f86dc139bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:56.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:56.043+0000 7f86e1200640 1 --2- 172.21.15.151:0/148629474 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86dc0b4bd0 0x7f86dc139bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:56.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:56.043+0000 7f86e1200640 1 -- 172.21.15.151:0/148629474 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86dc142b60 con 0x7f86dc1437c0 2024-09-16T15:49:56.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:56.043+0000 7f86db400640 1 --2- 172.21.15.151:0/148629474 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86dc0b4bd0 0x7f86dc139bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:49:56.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:56.043+0000 7f86e1200640 1 --2- 172.21.15.151:0/148629474 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86dc1437c0 0x7f86dc1502f0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f86d400b970 tx=0x7f86d400be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:49:56.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:56.044+0000 7f86d3e00640 1 -- 172.21.15.151:0/148629474 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d4019070 con 0x7f86dc1437c0 2024-09-16T15:49:56.047 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:56.044+0000 7f86d3e00640 1 -- 172.21.15.151:0/148629474 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f86d400cc50 con 0x7f86dc1437c0 2024-09-16T15:49:56.047 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:56.044+0000 7f86d3e00640 1 -- 172.21.15.151:0/148629474 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d40146a0 con 0x7f86dc1437c0 2024-09-16T15:49:56.047 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:56.044+0000 7f86e3200640 1 -- 172.21.15.151:0/148629474 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86dc142e40 con 0x7f86dc1437c0 2024-09-16T15:49:56.047 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:56.044+0000 7f86e3200640 1 -- 172.21.15.151:0/148629474 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86dc1f2a60 con 0x7f86dc1437c0 2024-09-16T15:49:56.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:56.047+0000 7f86e3200640 1 -- 172.21.15.151:0/148629474 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86dc165ac0 con 0x7f86dc1437c0 2024-09-16T15:49:56.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:56.049+0000 7f86d3e00640 1 -- 172.21.15.151:0/148629474 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 34) ==== 103225+0+0 (secure 0 0 0) 0x7f86d401c070 con 0x7f86dc1437c0 2024-09-16T15:49:56.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:56.051+0000 7f86d3e00640 1 --2- 172.21.15.151:0/148629474 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f86bc079db0 0x7f86bc07c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:49:56.054 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:56.052+0000 7f86d3e00640 1 -- 172.21.15.151:0/148629474 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7f86d40996c0 con 0x7f86dc1437c0 2024-09-16T15:49:56.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:56.070+0000 7f86d3e00640 1 -- 172.21.15.151:0/148629474 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f86d4062d90 con 0x7f86dc1437c0 2024-09-16T15:49:56.090 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:56.088+0000 7f86dbe00640 1 --2- 172.21.15.151:0/148629474 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f86bc079db0 0x7f86bc07c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:49:56.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:56.104+0000 7f86dbe00640 1 --2- 172.21.15.151:0/148629474 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f86bc079db0 0x7f86bc07c270 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f86c8006fd0 tx=0x7f86c8008040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:49:56.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:56.553+0000 7f86e3200640 1 -- 172.21.15.151:0/148629474 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone237619", "target": ["mon-mgr", ""]}) -- 0x7f86dc143010 con 0x7f86bc079db0 2024-09-16T15:49:58.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:58.075+0000 7f86d3e00640 1 -- 172.21.15.151:0/148629474 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f86dc143010 con 0x7f86bc079db0 2024-09-16T15:49:58.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:58.083+0000 7f86e3200640 1 -- 172.21.15.151:0/148629474 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f86bc079db0 msgr2=0x7f86bc07c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:58.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:58.083+0000 7f86e3200640 1 --2- 172.21.15.151:0/148629474 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f86bc079db0 0x7f86bc07c270 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f86c8006fd0 tx=0x7f86c8008040 comp rx=0 tx=0).stop 2024-09-16T15:49:58.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:58.083+0000 7f86e3200640 1 -- 172.21.15.151:0/148629474 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86dc1437c0 msgr2=0x7f86dc1502f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:49:58.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:58.083+0000 7f86e3200640 1 --2- 172.21.15.151:0/148629474 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86dc1437c0 0x7f86dc1502f0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f86d400b970 tx=0x7f86d400be30 comp rx=0 tx=0).stop 2024-09-16T15:49:58.090 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:58.089+0000 7f86e3200640 1 -- 172.21.15.151:0/148629474 shutdown_connections 2024-09-16T15:49:58.090 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:58.089+0000 7f86e3200640 1 --2- 172.21.15.151:0/148629474 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f86bc079db0 0x7f86bc07c270 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:58.090 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:58.089+0000 7f86e3200640 1 --2- 172.21.15.151:0/148629474 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86dc1437c0 0x7f86dc1502f0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:58.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:58.089+0000 7f86e3200640 1 --2- 172.21.15.151:0/148629474 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86dc0b4bd0 0x7f86dc139bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:58.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:58.089+0000 7f86e3200640 1 --2- 172.21.15.151:0/148629474 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f86dc070870 0x7f86dc1f2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:49:58.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:58.089+0000 7f86e3200640 1 -- 172.21.15.151:0/148629474 >> 172.21.15.151:0/148629474 conn(0x7f86dc0b1840 msgr2=0x7f86dc06e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:49:58.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:58.091+0000 7f86e3200640 1 -- 172.21.15.151:0/148629474 shutdown_connections 2024-09-16T15:49:58.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:49:58.091+0000 7f86e3200640 1 -- 172.21.15.151:0/148629474 wait complete. 2024-09-16T15:50:00.146 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-16T15:50:00.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:00.450+0000 7f2c7d600640 1 -- 172.21.15.151:0/1496512823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c780ccba0 msgr2=0x7f2c78073280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:00.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:00.450+0000 7f2c7d600640 1 --2- 172.21.15.151:0/1496512823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c780ccba0 0x7f2c78073280 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f2c7806fc20 tx=0x7f2c7002a7e0 comp rx=0 tx=0).stop 2024-09-16T15:50:00.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:00.451+0000 7f2c7d600640 1 -- 172.21.15.151:0/1496512823 shutdown_connections 2024-09-16T15:50:00.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:00.451+0000 7f2c7d600640 1 --2- 172.21.15.151:0/1496512823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c780ccba0 0x7f2c78073280 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:00.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:00.451+0000 7f2c7d600640 1 --2- 172.21.15.151:0/1496512823 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c7806e9b0 0x7f2c780cc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:00.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:00.451+0000 7f2c7d600640 1 -- 172.21.15.151:0/1496512823 >> 172.21.15.151:0/1496512823 conn(0x7f2c780b18e0 msgr2=0x7f2c780b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:00.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:00.451+0000 7f2c7d600640 1 -- 172.21.15.151:0/1496512823 shutdown_connections 2024-09-16T15:50:00.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:00.451+0000 7f2c7d600640 1 -- 172.21.15.151:0/1496512823 wait complete. 2024-09-16T15:50:00.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:00.453+0000 7f2c7d600640 1 Processor -- start 2024-09-16T15:50:00.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:00.453+0000 7f2c7d600640 1 -- start start 2024-09-16T15:50:00.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:00.454+0000 7f2c7d600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c7806e9b0 0x7f2c78148810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:00.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:00.454+0000 7f2c7d600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c780ccba0 0x7f2c7809d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:00.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:00.454+0000 7f2c7d600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c781563a0 0x7f2c7814bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:00.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:00.455+0000 7f2c7d600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2c78073c10 con 0x7f2c7806e9b0 2024-09-16T15:50:00.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:00.455+0000 7f2c7d600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2c78073a90 con 0x7f2c780ccba0 2024-09-16T15:50:00.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:00.455+0000 7f2c7d600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f2c78073d90 con 0x7f2c781563a0 2024-09-16T15:50:00.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:00.455+0000 7f2c77e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c7806e9b0 0x7f2c78148810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:00.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:00.455+0000 7f2c77e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c7806e9b0 0x7f2c78148810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52614/0 (socket says 172.21.15.151:52614) 2024-09-16T15:50:00.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:00.455+0000 7f2c77e00640 1 -- 172.21.15.151:0/2750288939 learned_addr learned my addr 172.21.15.151:0/2750288939 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:00.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:00.455+0000 7f2c77e00640 1 -- 172.21.15.151:0/2750288939 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c781563a0 msgr2=0x7f2c7814bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:00.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:00.455+0000 7f2c77e00640 1 --2- 172.21.15.151:0/2750288939 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c781563a0 0x7f2c7814bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:00.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:00.455+0000 7f2c77e00640 1 -- 172.21.15.151:0/2750288939 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c780ccba0 msgr2=0x7f2c7809d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:00.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:00.455+0000 7f2c77e00640 1 --2- 172.21.15.151:0/2750288939 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c780ccba0 0x7f2c7809d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:00.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:00.455+0000 7f2c77e00640 1 -- 172.21.15.151:0/2750288939 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c7814c060 con 0x7f2c7806e9b0 2024-09-16T15:50:00.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:00.456+0000 7f2c77e00640 1 --2- 172.21.15.151:0/2750288939 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c7806e9b0 0x7f2c78148810 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f2c6800b420 tx=0x7f2c6800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:00.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:00.456+0000 7f2c6fe00640 1 -- 172.21.15.151:0/2750288939 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c68011830 con 0x7f2c7806e9b0 2024-09-16T15:50:00.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:00.456+0000 7f2c6fe00640 1 -- 172.21.15.151:0/2750288939 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2c68016070 con 0x7f2c7806e9b0 2024-09-16T15:50:00.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:00.456+0000 7f2c6fe00640 1 -- 172.21.15.151:0/2750288939 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c68011d80 con 0x7f2c7806e9b0 2024-09-16T15:50:00.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:00.456+0000 7f2c7d600640 1 -- 172.21.15.151:0/2750288939 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c780cf820 con 0x7f2c7806e9b0 2024-09-16T15:50:00.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:00.457+0000 7f2c7d600640 1 -- 172.21.15.151:0/2750288939 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c781490d0 con 0x7f2c7806e9b0 2024-09-16T15:50:00.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:00.458+0000 7f2c7d600640 1 -- 172.21.15.151:0/2750288939 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c7815f0f0 con 0x7f2c7806e9b0 2024-09-16T15:50:00.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:00.459+0000 7f2c6fe00640 1 -- 172.21.15.151:0/2750288939 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 34) ==== 103225+0+0 (secure 0 0 0) 0x7f2c6801b460 con 0x7f2c7806e9b0 2024-09-16T15:50:00.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:00.461+0000 7f2c6fe00640 1 --2- 172.21.15.151:0/2750288939 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2c58079db0 0x7f2c5807c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:00.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:00.461+0000 7f2c6fe00640 1 -- 172.21.15.151:0/2750288939 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7f2c68099780 con 0x7f2c7806e9b0 2024-09-16T15:50:00.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:00.471+0000 7f2c6fe00640 1 -- 172.21.15.151:0/2750288939 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2c7815f0f0 con 0x7f2c7806e9b0 2024-09-16T15:50:00.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:00.477+0000 7f2c77400640 1 --2- 172.21.15.151:0/2750288939 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2c58079db0 0x7f2c5807c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:00.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:00.492+0000 7f2c77400640 1 --2- 172.21.15.151:0/2750288939 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2c58079db0 0x7f2c5807c270 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f2c64005fd0 tx=0x7f2c64005850 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:00.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:00.931+0000 7f2c7d600640 1 -- 172.21.15.151:0/2750288939 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f2c7806aaf0 con 0x7f2c7806e9b0 2024-09-16T15:50:00.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:00.932+0000 7f2c6fe00640 1 -- 172.21.15.151:0/2750288939 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2443 (secure 0 0 0) 0x7f2c7806aaf0 con 0x7f2c7806e9b0 2024-09-16T15:50:00.933 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:50:00.933 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-16T15:50:00.933 INFO:teuthology.orchestra.run.smithi151.stdout: "fsid": "1757be18-e41a-4446-bb4f-ee4ac81c39df", 2024-09-16T15:50:00.933 INFO:teuthology.orchestra.run.smithi151.stdout: "health": { 2024-09-16T15:50:00.933 INFO:teuthology.orchestra.run.smithi151.stdout: "status": "HEALTH_OK", 2024-09-16T15:50:00.933 INFO:teuthology.orchestra.run.smithi151.stdout: "checks": {}, 2024-09-16T15:50:00.933 INFO:teuthology.orchestra.run.smithi151.stdout: "mutes": [] 2024-09-16T15:50:00.933 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:50:00.933 INFO:teuthology.orchestra.run.smithi151.stdout: "election_epoch": 4, 2024-09-16T15:50:00.933 INFO:teuthology.orchestra.run.smithi151.stdout: "quorum": [ 2024-09-16T15:50:00.933 INFO:teuthology.orchestra.run.smithi151.stdout: 0, 2024-09-16T15:50:00.933 INFO:teuthology.orchestra.run.smithi151.stdout: 1, 2024-09-16T15:50:00.934 INFO:teuthology.orchestra.run.smithi151.stdout: 2 2024-09-16T15:50:00.934 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:50:00.934 INFO:teuthology.orchestra.run.smithi151.stdout: "quorum_names": [ 2024-09-16T15:50:00.934 INFO:teuthology.orchestra.run.smithi151.stdout: "a", 2024-09-16T15:50:00.934 INFO:teuthology.orchestra.run.smithi151.stdout: "b", 2024-09-16T15:50:00.934 INFO:teuthology.orchestra.run.smithi151.stdout: "c" 2024-09-16T15:50:00.934 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:50:00.934 INFO:teuthology.orchestra.run.smithi151.stdout: "quorum_age": 664, 2024-09-16T15:50:00.934 INFO:teuthology.orchestra.run.smithi151.stdout: "monmap": { 2024-09-16T15:50:00.934 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 1, 2024-09-16T15:50:00.934 INFO:teuthology.orchestra.run.smithi151.stdout: "min_mon_release_name": "squid", 2024-09-16T15:50:00.934 INFO:teuthology.orchestra.run.smithi151.stdout: "num_mons": 3 2024-09-16T15:50:00.934 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:50:00.934 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap": { 2024-09-16T15:50:00.934 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 91, 2024-09-16T15:50:00.935 INFO:teuthology.orchestra.run.smithi151.stdout: "num_osds": 8, 2024-09-16T15:50:00.935 INFO:teuthology.orchestra.run.smithi151.stdout: "num_up_osds": 8, 2024-09-16T15:50:00.935 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_up_since": 1726501150, 2024-09-16T15:50:00.935 INFO:teuthology.orchestra.run.smithi151.stdout: "num_in_osds": 8, 2024-09-16T15:50:00.935 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_in_since": 1726501140, 2024-09-16T15:50:00.935 INFO:teuthology.orchestra.run.smithi151.stdout: "num_remapped_pgs": 0 2024-09-16T15:50:00.935 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:50:00.935 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap": { 2024-09-16T15:50:00.935 INFO:teuthology.orchestra.run.smithi151.stdout: "pgs_by_state": [ 2024-09-16T15:50:00.935 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:50:00.935 INFO:teuthology.orchestra.run.smithi151.stdout: "state_name": "active+clean", 2024-09-16T15:50:00.935 INFO:teuthology.orchestra.run.smithi151.stdout: "count": 97 2024-09-16T15:50:00.935 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:50:00.935 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:50:00.935 INFO:teuthology.orchestra.run.smithi151.stdout: "num_pgs": 97, 2024-09-16T15:50:00.935 INFO:teuthology.orchestra.run.smithi151.stdout: "num_pools": 3, 2024-09-16T15:50:00.936 INFO:teuthology.orchestra.run.smithi151.stdout: "num_objects": 8127, 2024-09-16T15:50:00.936 INFO:teuthology.orchestra.run.smithi151.stdout: "data_bytes": 33876526370, 2024-09-16T15:50:00.936 INFO:teuthology.orchestra.run.smithi151.stdout: "bytes_used": 67671183360, 2024-09-16T15:50:00.936 INFO:teuthology.orchestra.run.smithi151.stdout: "bytes_avail": 705422929920, 2024-09-16T15:50:00.936 INFO:teuthology.orchestra.run.smithi151.stdout: "bytes_total": 773094113280, 2024-09-16T15:50:00.936 INFO:teuthology.orchestra.run.smithi151.stdout: "read_bytes_sec": 542613597, 2024-09-16T15:50:00.936 INFO:teuthology.orchestra.run.smithi151.stdout: "write_bytes_sec": 546862973, 2024-09-16T15:50:00.936 INFO:teuthology.orchestra.run.smithi151.stdout: "read_op_per_sec": 132, 2024-09-16T15:50:00.936 INFO:teuthology.orchestra.run.smithi151.stdout: "write_op_per_sec": 148 2024-09-16T15:50:00.936 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:50:00.936 INFO:teuthology.orchestra.run.smithi151.stdout: "fsmap": { 2024-09-16T15:50:00.936 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 57, 2024-09-16T15:50:00.936 INFO:teuthology.orchestra.run.smithi151.stdout: "btime": "2024-09-16T15:48:35:029906+0000", 2024-09-16T15:50:00.936 INFO:teuthology.orchestra.run.smithi151.stdout: "id": 6, 2024-09-16T15:50:00.936 INFO:teuthology.orchestra.run.smithi151.stdout: "up": 1, 2024-09-16T15:50:00.937 INFO:teuthology.orchestra.run.smithi151.stdout: "in": 1, 2024-09-16T15:50:00.937 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 1, 2024-09-16T15:50:00.937 INFO:teuthology.orchestra.run.smithi151.stdout: "by_rank": [ 2024-09-16T15:50:00.937 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:50:00.937 INFO:teuthology.orchestra.run.smithi151.stdout: "filesystem_id": 6, 2024-09-16T15:50:00.937 INFO:teuthology.orchestra.run.smithi151.stdout: "rank": 0, 2024-09-16T15:50:00.937 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "c", 2024-09-16T15:50:00.937 INFO:teuthology.orchestra.run.smithi151.stdout: "status": "up:active", 2024-09-16T15:50:00.937 INFO:teuthology.orchestra.run.smithi151.stdout: "gid": 6673 2024-09-16T15:50:00.937 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:50:00.937 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:50:00.937 INFO:teuthology.orchestra.run.smithi151.stdout: "up:standby": 3 2024-09-16T15:50:00.937 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:50:00.937 INFO:teuthology.orchestra.run.smithi151.stdout: "mgrmap": { 2024-09-16T15:50:00.937 INFO:teuthology.orchestra.run.smithi151.stdout: "available": true, 2024-09-16T15:50:00.937 INFO:teuthology.orchestra.run.smithi151.stdout: "num_standbys": 1, 2024-09-16T15:50:00.938 INFO:teuthology.orchestra.run.smithi151.stdout: "modules": [ 2024-09-16T15:50:00.938 INFO:teuthology.orchestra.run.smithi151.stdout: "iostat", 2024-09-16T15:50:00.938 INFO:teuthology.orchestra.run.smithi151.stdout: "nfs", 2024-09-16T15:50:00.938 INFO:teuthology.orchestra.run.smithi151.stdout: "restful" 2024-09-16T15:50:00.938 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:50:00.938 INFO:teuthology.orchestra.run.smithi151.stdout: "services": {} 2024-09-16T15:50:00.938 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:50:00.938 INFO:teuthology.orchestra.run.smithi151.stdout: "servicemap": { 2024-09-16T15:50:00.938 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 15, 2024-09-16T15:50:00.938 INFO:teuthology.orchestra.run.smithi151.stdout: "modified": "2024-09-16T15:49:29.793029+0000", 2024-09-16T15:50:00.938 INFO:teuthology.orchestra.run.smithi151.stdout: "services": {} 2024-09-16T15:50:00.938 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:50:00.938 INFO:teuthology.orchestra.run.smithi151.stdout: "progress_events": { 2024-09-16T15:50:00.938 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr-vol-ongoing-clones": { 2024-09-16T15:50:00.938 INFO:teuthology.orchestra.run.smithi151.stdout: "message": "4 ongoing clones - average progress is 46.289% (39s)\n [============................] (remaining: 45s)", 2024-09-16T15:50:00.939 INFO:teuthology.orchestra.run.smithi151.stdout: "progress": 0.46288999915122986, 2024-09-16T15:50:00.939 INFO:teuthology.orchestra.run.smithi151.stdout: "add_to_ceph_s": true 2024-09-16T15:50:00.939 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:50:00.939 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr-vol-total-clones": { 2024-09-16T15:50:00.939 INFO:teuthology.orchestra.run.smithi151.stdout: "message": "Total 8 clones - average progress is 28.451% (26s)\n [=======.....................] (remaining: 66s)", 2024-09-16T15:50:00.939 INFO:teuthology.orchestra.run.smithi151.stdout: "progress": 0.28450998663902283, 2024-09-16T15:50:00.939 INFO:teuthology.orchestra.run.smithi151.stdout: "add_to_ceph_s": true 2024-09-16T15:50:00.939 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:50:00.939 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:50:00.939 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-16T15:50:00.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:00.935+0000 7f2c7d600640 1 -- 172.21.15.151:0/2750288939 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2c58079db0 msgr2=0x7f2c5807c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:00.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:00.935+0000 7f2c7d600640 1 --2- 172.21.15.151:0/2750288939 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2c58079db0 0x7f2c5807c270 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f2c64005fd0 tx=0x7f2c64005850 comp rx=0 tx=0).stop 2024-09-16T15:50:00.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:00.935+0000 7f2c7d600640 1 -- 172.21.15.151:0/2750288939 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c7806e9b0 msgr2=0x7f2c78148810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:00.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:00.935+0000 7f2c7d600640 1 --2- 172.21.15.151:0/2750288939 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c7806e9b0 0x7f2c78148810 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f2c6800b420 tx=0x7f2c6800b8e0 comp rx=0 tx=0).stop 2024-09-16T15:50:00.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:00.935+0000 7f2c7d600640 1 -- 172.21.15.151:0/2750288939 shutdown_connections 2024-09-16T15:50:00.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:00.935+0000 7f2c7d600640 1 --2- 172.21.15.151:0/2750288939 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2c58079db0 0x7f2c5807c270 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:00.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:00.935+0000 7f2c7d600640 1 --2- 172.21.15.151:0/2750288939 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c780ccba0 0x7f2c7809d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:00.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:00.935+0000 7f2c7d600640 1 --2- 172.21.15.151:0/2750288939 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c781563a0 0x7f2c7814bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:00.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:00.935+0000 7f2c7d600640 1 --2- 172.21.15.151:0/2750288939 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c7806e9b0 0x7f2c78148810 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:00.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:00.935+0000 7f2c7d600640 1 -- 172.21.15.151:0/2750288939 >> 172.21.15.151:0/2750288939 conn(0x7f2c780b18e0 msgr2=0x7f2c7815d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:00.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:00.935+0000 7f2c7d600640 1 -- 172.21.15.151:0/2750288939 shutdown_connections 2024-09-16T15:50:00.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:00.935+0000 7f2c7d600640 1 -- 172.21.15.151:0/2750288939 wait complete. 2024-09-16T15:50:00.954 INFO:tasks.cephfs.test_volumes:pevs - {'mgr-vol-ongoing-clones': {'message': '4 ongoing clones - average progress is 46.289% (39s)\n [============................] (remaining: 45s)', 'progress': 0.46288999915122986, 'add_to_ceph_s': True}, 'mgr-vol-total-clones': {'message': 'Total 8 clones - average progress is 28.451% (26s)\n [=======.....................] (remaining: 66s)', 'progress': 0.2845099866390228, 'add_to_ceph_s': True}} 2024-09-16T15:50:00.954 INFO:tasks.cephfs.test_volumes:new_onpen_p is less than onpen_p. 2024-09-16T15:50:00.954 INFO:tasks.cephfs.test_volumes:new_onpen_p = 0.2845099866390228; onpen_p = 0.5593699812889099 2024-09-16T15:50:00.954 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone55130 2024-09-16T15:50:01.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:01.230+0000 7feebde00640 1 -- 172.21.15.151:0/775267249 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feeb80cccd0 msgr2=0x7feeb8073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:01.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:01.230+0000 7feebde00640 1 --2- 172.21.15.151:0/775267249 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feeb80cccd0 0x7feeb8073440 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7feea4009830 tx=0x7feea402a480 comp rx=0 tx=0).stop 2024-09-16T15:50:01.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:01.230+0000 7feebde00640 1 -- 172.21.15.151:0/775267249 shutdown_connections 2024-09-16T15:50:01.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:01.230+0000 7feebde00640 1 --2- 172.21.15.151:0/775267249 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feeb80cccd0 0x7feeb8073440 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:01.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:01.230+0000 7feebde00640 1 -- 172.21.15.151:0/775267249 >> 172.21.15.151:0/775267249 conn(0x7feeb80b1840 msgr2=0x7feeb80b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:01.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:01.230+0000 7feebde00640 1 -- 172.21.15.151:0/775267249 shutdown_connections 2024-09-16T15:50:01.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:01.230+0000 7feebde00640 1 -- 172.21.15.151:0/775267249 wait complete. 2024-09-16T15:50:01.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:01.233+0000 7feebde00640 1 Processor -- start 2024-09-16T15:50:01.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:01.233+0000 7feebde00640 1 -- start start 2024-09-16T15:50:01.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:01.235+0000 7feebde00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feeb806efa0 0x7feeb81f8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:01.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:01.235+0000 7feebde00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feeb80cccd0 0x7feeb81f57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:01.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:01.235+0000 7feebde00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feeb8145750 0x7feeb81e8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:01.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:01.235+0000 7feebde00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7feeb80ce6a0 con 0x7feeb8145750 2024-09-16T15:50:01.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:01.235+0000 7feebde00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7feeb80ce520 con 0x7feeb80cccd0 2024-09-16T15:50:01.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:01.235+0000 7feebde00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7feeb80ce820 con 0x7feeb806efa0 2024-09-16T15:50:01.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:01.236+0000 7feeb7400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feeb8145750 0x7feeb81e8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:01.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:01.236+0000 7feeb7400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feeb8145750 0x7feeb81e8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52632/0 (socket says 172.21.15.151:52632) 2024-09-16T15:50:01.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:01.236+0000 7feeb7400640 1 -- 172.21.15.151:0/2483014026 learned_addr learned my addr 172.21.15.151:0/2483014026 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:01.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:01.236+0000 7feeb6000640 1 --2- 172.21.15.151:0/2483014026 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feeb80cccd0 0x7feeb81f57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:01.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:01.236+0000 7feeb6a00640 1 --2- 172.21.15.151:0/2483014026 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feeb806efa0 0x7feeb81f8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:01.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:01.236+0000 7feeb7400640 1 -- 172.21.15.151:0/2483014026 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feeb806efa0 msgr2=0x7feeb81f8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:01.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:01.236+0000 7feeb7400640 1 --2- 172.21.15.151:0/2483014026 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feeb806efa0 0x7feeb81f8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:01.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:01.236+0000 7feeb7400640 1 -- 172.21.15.151:0/2483014026 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feeb80cccd0 msgr2=0x7feeb81f57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:01.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:01.236+0000 7feeb7400640 1 --2- 172.21.15.151:0/2483014026 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feeb80cccd0 0x7feeb81f57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:01.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:01.236+0000 7feeb7400640 1 -- 172.21.15.151:0/2483014026 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feeb81f4ff0 con 0x7feeb8145750 2024-09-16T15:50:01.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:01.237+0000 7feeb7400640 1 --2- 172.21.15.151:0/2483014026 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feeb8145750 0x7feeb81e8aa0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7feeb000c390 tx=0x7feeb000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:01.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:01.237+0000 7feeaf400640 1 -- 172.21.15.151:0/2483014026 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb0017070 con 0x7feeb8145750 2024-09-16T15:50:01.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:01.237+0000 7feeaf400640 1 -- 172.21.15.151:0/2483014026 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feeb00035f0 con 0x7feeb8145750 2024-09-16T15:50:01.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:01.237+0000 7feeaf400640 1 -- 172.21.15.151:0/2483014026 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb0003a40 con 0x7feeb8145750 2024-09-16T15:50:01.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:01.237+0000 7feebde00640 1 -- 172.21.15.151:0/2483014026 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feeb80c73a0 con 0x7feeb8145750 2024-09-16T15:50:01.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:01.237+0000 7feebde00640 1 -- 172.21.15.151:0/2483014026 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feeb81f5290 con 0x7feeb8145750 2024-09-16T15:50:01.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:01.239+0000 7feebde00640 1 -- 172.21.15.151:0/2483014026 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feeb80aa620 con 0x7feeb8145750 2024-09-16T15:50:01.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:01.240+0000 7feeaf400640 1 -- 172.21.15.151:0/2483014026 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 34) ==== 103225+0+0 (secure 0 0 0) 0x7feeb0003be0 con 0x7feeb8145750 2024-09-16T15:50:01.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:01.241+0000 7feeaf400640 1 --2- 172.21.15.151:0/2483014026 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fee98079db0 0x7fee9807c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:01.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:01.241+0000 7feeaf400640 1 -- 172.21.15.151:0/2483014026 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7feeb0098c90 con 0x7feeb8145750 2024-09-16T15:50:01.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:01.245+0000 7feeaf400640 1 -- 172.21.15.151:0/2483014026 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7feeb0062990 con 0x7feeb8145750 2024-09-16T15:50:01.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:01.260+0000 7feeb6a00640 1 --2- 172.21.15.151:0/2483014026 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fee98079db0 0x7fee9807c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:01.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:01.267+0000 7feeb6a00640 1 --2- 172.21.15.151:0/2483014026 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fee98079db0 0x7fee9807c270 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7feea80020b0 tx=0x7feea8009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:01.653 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:01.652+0000 7feebde00640 1 -- 172.21.15.151:0/2483014026 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone55130", "target": ["mon-mgr", ""]}) -- 0x7feeb8069a10 con 0x7fee98079db0 2024-09-16T15:50:01.752 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:50:01.751+0000 7f48efe00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-16T15:50:01.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:01.756+0000 7feeaf400640 1 -- 172.21.15.151:0/2483014026 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7feeb8069a10 con 0x7fee98079db0 2024-09-16T15:50:01.757 INFO:teuthology.orchestra.run.smithi151.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-16T15:50:01.771 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T15:50:01.772 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone23331 2024-09-16T15:50:02.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:02.033+0000 7fc3c7200640 1 -- 172.21.15.151:0/1087858406 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc3c0070650 msgr2=0x7fc3c00ce1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:02.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:02.033+0000 7fc3c7200640 1 --2- 172.21.15.151:0/1087858406 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc3c0070650 0x7fc3c00ce1e0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fc3c00cb970 tx=0x7fc3b002a540 comp rx=0 tx=0).stop 2024-09-16T15:50:02.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:02.033+0000 7fc3c7200640 1 -- 172.21.15.151:0/1087858406 shutdown_connections 2024-09-16T15:50:02.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:02.033+0000 7fc3c7200640 1 --2- 172.21.15.151:0/1087858406 >> v2:172.21.15.167:3300/0 conn(0x7fc3c006e940 0x7fc3c006edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:02.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:02.033+0000 7fc3c7200640 1 --2- 172.21.15.151:0/1087858406 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc3c0070650 0x7fc3c00ce1e0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:02.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:02.033+0000 7fc3c7200640 1 -- 172.21.15.151:0/1087858406 >> 172.21.15.151:0/1087858406 conn(0x7fc3c00cf9f0 msgr2=0x7fc3c00cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:02.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:02.033+0000 7fc3c7200640 1 -- 172.21.15.151:0/1087858406 shutdown_connections 2024-09-16T15:50:02.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:02.033+0000 7fc3c7200640 1 -- 172.21.15.151:0/1087858406 wait complete. 2024-09-16T15:50:02.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:02.035+0000 7fc3c7200640 1 Processor -- start 2024-09-16T15:50:02.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:02.035+0000 7fc3c7200640 1 -- start start 2024-09-16T15:50:02.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:02.036+0000 7fc3c7200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc3c006e940 0x7fc3c01537c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:02.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:02.036+0000 7fc3c7200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3c0070650 0x7fc3c0158ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:02.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:02.036+0000 7fc3c7200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc3c015dc00 0x7fc3c0159220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:02.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:02.036+0000 7fc3c7200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c006f680 con 0x7fc3c006e940 2024-09-16T15:50:02.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:02.036+0000 7fc3c7200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c006f500 con 0x7fc3c0070650 2024-09-16T15:50:02.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:02.036+0000 7fc3c7200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fc3c006f800 con 0x7fc3c015dc00 2024-09-16T15:50:02.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:02.036+0000 7fc3c5e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc3c006e940 0x7fc3c01537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:02.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:02.036+0000 7fc3c5e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc3c006e940 0x7fc3c01537c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52646/0 (socket says 172.21.15.151:52646) 2024-09-16T15:50:02.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:02.036+0000 7fc3c5e00640 1 -- 172.21.15.151:0/3463032015 learned_addr learned my addr 172.21.15.151:0/3463032015 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:02.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:02.037+0000 7fc3c5e00640 1 -- 172.21.15.151:0/3463032015 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc3c015dc00 msgr2=0x7fc3c0159220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:02.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:02.037+0000 7fc3c5e00640 1 --2- 172.21.15.151:0/3463032015 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc3c015dc00 0x7fc3c0159220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:02.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:02.037+0000 7fc3c5e00640 1 -- 172.21.15.151:0/3463032015 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3c0070650 msgr2=0x7fc3c0158ce0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T15:50:02.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:02.037+0000 7fc3c5e00640 1 --2- 172.21.15.151:0/3463032015 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc3c0070650 0x7fc3c0158ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:02.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:02.037+0000 7fc3c5e00640 1 -- 172.21.15.151:0/3463032015 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3c0073120 con 0x7fc3c006e940 2024-09-16T15:50:02.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:02.037+0000 7fc3c5e00640 1 --2- 172.21.15.151:0/3463032015 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc3c006e940 0x7fc3c01537c0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fc3b400c3a0 tx=0x7fc3b400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:02.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:02.037+0000 7fc3ba000640 1 -- 172.21.15.151:0/3463032015 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b4011790 con 0x7fc3c006e940 2024-09-16T15:50:02.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:02.037+0000 7fc3ba000640 1 -- 172.21.15.151:0/3463032015 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc3b4017070 con 0x7fc3c006e940 2024-09-16T15:50:02.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:02.037+0000 7fc3c7200640 1 -- 172.21.15.151:0/3463032015 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3c00c5870 con 0x7fc3c006e940 2024-09-16T15:50:02.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:02.037+0000 7fc3ba000640 1 -- 172.21.15.151:0/3463032015 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b4011cc0 con 0x7fc3c006e940 2024-09-16T15:50:02.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:02.037+0000 7fc3c7200640 1 -- 172.21.15.151:0/3463032015 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3c00c5e30 con 0x7fc3c006e940 2024-09-16T15:50:02.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:02.038+0000 7fc3c7200640 1 -- 172.21.15.151:0/3463032015 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3c006edb0 con 0x7fc3c006e940 2024-09-16T15:50:02.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:02.040+0000 7fc3ba000640 1 -- 172.21.15.151:0/3463032015 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 34) ==== 103225+0+0 (secure 0 0 0) 0x7fc3b401c460 con 0x7fc3c006e940 2024-09-16T15:50:02.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:02.041+0000 7fc3ba000640 1 --2- 172.21.15.151:0/3463032015 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc3a4079db0 0x7fc3a407c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:02.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:02.041+0000 7fc3ba000640 1 -- 172.21.15.151:0/3463032015 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7fc3b4099800 con 0x7fc3c006e940 2024-09-16T15:50:02.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:02.045+0000 7fc3c5400640 1 --2- 172.21.15.151:0/3463032015 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc3a4079db0 0x7fc3a407c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:02.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:02.046+0000 7fc3ba000640 1 -- 172.21.15.151:0/3463032015 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc3b40634c0 con 0x7fc3c006e940 2024-09-16T15:50:02.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:02.051+0000 7fc3c5400640 1 --2- 172.21.15.151:0/3463032015 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc3a4079db0 0x7fc3a407c270 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fc3b0034000 tx=0x7fc3b0004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:02.471 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:02.470+0000 7fc3c7200640 1 -- 172.21.15.151:0/3463032015 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone23331", "target": ["mon-mgr", ""]}) -- 0x7fc3c01541a0 con 0x7fc3a4079db0 2024-09-16T15:50:02.637 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:50:02.636+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:50:02.637 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:50:02.636+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:50:02.637 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:50:02.636+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:50:02.637 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:50:02.636+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:50:02.638 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:50:02.636+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:50:02.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:02.732+0000 7fc3ba000640 1 -- 172.21.15.151:0/3463032015 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mgrmap(e 35) ==== 103174+0+0 (secure 0 0 0) 0x7fc3b405f030 con 0x7fc3c006e940 2024-09-16T15:50:03.244 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:50:03.243+0000 7f48efe00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-16T15:50:03.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:03.246+0000 7fc3ba000640 1 -- 172.21.15.151:0/3463032015 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7fc3b0004300 con 0x7fc3a4079db0 2024-09-16T15:50:03.247 INFO:teuthology.orchestra.run.smithi151.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-16T15:50:03.268 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T15:50:03.268 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone13712 2024-09-16T15:50:03.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:03.637+0000 7f666fc00640 1 -- 172.21.15.151:0/4196341034 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f666815c030 msgr2=0x7f66680d02d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:03.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:03.637+0000 7f666fc00640 1 --2- 172.21.15.151:0/4196341034 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f666815c030 0x7f66680d02d0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f6658009830 tx=0x7f665802a4a0 comp rx=0 tx=0).stop 2024-09-16T15:50:03.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:03.637+0000 7f666fc00640 1 -- 172.21.15.151:0/4196341034 shutdown_connections 2024-09-16T15:50:03.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:03.637+0000 7f666fc00640 1 --2- 172.21.15.151:0/4196341034 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66680d2990 0x7f66680d0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:03.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:03.637+0000 7f666fc00640 1 --2- 172.21.15.151:0/4196341034 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f666815c030 0x7f66680d02d0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:03.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:03.637+0000 7f666fc00640 1 -- 172.21.15.151:0/4196341034 >> 172.21.15.151:0/4196341034 conn(0x7f66680d1d90 msgr2=0x7f66680d21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:03.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:03.639+0000 7f666fc00640 1 -- 172.21.15.151:0/4196341034 shutdown_connections 2024-09-16T15:50:03.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:03.639+0000 7f666fc00640 1 -- 172.21.15.151:0/4196341034 wait complete. 2024-09-16T15:50:03.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:03.642+0000 7f666fc00640 1 Processor -- start 2024-09-16T15:50:03.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:03.642+0000 7f666fc00640 1 -- start start 2024-09-16T15:50:03.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:03.644+0000 7f666fc00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66680d2990 0x7f666813ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:03.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:03.644+0000 7f666fc00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f666815bc50 0x7f66680b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:03.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:03.644+0000 7f666fc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f666815c030 0x7f66680c7a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:03.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:03.644+0000 7f666fc00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f66680b2030 con 0x7f666815c030 2024-09-16T15:50:03.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:03.644+0000 7f666fc00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f66680b1eb0 con 0x7f66680d2990 2024-09-16T15:50:03.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:03.644+0000 7f666fc00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f66680b21b0 con 0x7f666815bc50 2024-09-16T15:50:03.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:03.644+0000 7f666dc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f666815c030 0x7f66680c7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:03.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:03.644+0000 7f666dc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f666815c030 0x7f66680c7a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:57126/0 (socket says 172.21.15.151:57126) 2024-09-16T15:50:03.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:03.644+0000 7f666dc00640 1 -- 172.21.15.151:0/2176233292 learned_addr learned my addr 172.21.15.151:0/2176233292 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:03.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:03.645+0000 7f666dc00640 1 -- 172.21.15.151:0/2176233292 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f666815bc50 msgr2=0x7f66680b50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:03.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:03.645+0000 7f666d200640 1 --2- 172.21.15.151:0/2176233292 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66680d2990 0x7f666813ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:03.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:03.645+0000 7f666dc00640 1 --2- 172.21.15.151:0/2176233292 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f666815bc50 0x7f66680b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:03.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:03.645+0000 7f666dc00640 1 -- 172.21.15.151:0/2176233292 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66680d2990 msgr2=0x7f666813ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:03.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:03.645+0000 7f666dc00640 1 --2- 172.21.15.151:0/2176233292 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66680d2990 0x7f666813ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:03.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:03.645+0000 7f666dc00640 1 -- 172.21.15.151:0/2176233292 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66680b70b0 con 0x7f666815c030 2024-09-16T15:50:03.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:03.645+0000 7f666d200640 1 --2- 172.21.15.151:0/2176233292 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f66680d2990 0x7f666813ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:50:03.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:03.645+0000 7f666dc00640 1 --2- 172.21.15.151:0/2176233292 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f666815c030 0x7f66680c7a90 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f665400d750 tx=0x7f665400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:03.648 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:03.646+0000 7f6664c00640 1 -- 172.21.15.151:0/2176233292 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6654017070 con 0x7f666815c030 2024-09-16T15:50:03.648 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:03.646+0000 7f6664c00640 1 -- 172.21.15.151:0/2176233292 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6654009ac0 con 0x7f666815c030 2024-09-16T15:50:03.648 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:03.646+0000 7f6664c00640 1 -- 172.21.15.151:0/2176233292 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6654005020 con 0x7f666815c030 2024-09-16T15:50:03.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:03.646+0000 7f666fc00640 1 -- 172.21.15.151:0/2176233292 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66680b7390 con 0x7f666815c030 2024-09-16T15:50:03.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:03.646+0000 7f666fc00640 1 -- 172.21.15.151:0/2176233292 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66680b7950 con 0x7f666815c030 2024-09-16T15:50:03.650 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:03.649+0000 7f666fc00640 1 -- 172.21.15.151:0/2176233292 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66680b7b90 con 0x7f666815c030 2024-09-16T15:50:03.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:03.650+0000 7f6664c00640 1 -- 172.21.15.151:0/2176233292 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 35) ==== 103174+0+0 (secure 0 0 0) 0x7f66540051c0 con 0x7f666815c030 2024-09-16T15:50:03.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:03.651+0000 7f6664c00640 1 --2- 172.21.15.151:0/2176233292 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6648079d20 0x7f664807c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:03.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:03.651+0000 7f6664c00640 1 -- 172.21.15.151:0/2176233292 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7f66540999e0 con 0x7f666815c030 2024-09-16T15:50:03.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:03.655+0000 7f666d200640 1 --2- 172.21.15.151:0/2176233292 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6648079d20 0x7f664807c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:03.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:03.659+0000 7f666d200640 1 --2- 172.21.15.151:0/2176233292 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6648079d20 0x7f664807c1e0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f6660004a30 tx=0x7f6660009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:03.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:03.661+0000 7f6664c00640 1 -- 172.21.15.151:0/2176233292 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f66680b7b90 con 0x7f666815c030 2024-09-16T15:50:04.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:04.142+0000 7f666fc00640 1 -- 172.21.15.151:0/2176233292 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone13712", "target": ["mon-mgr", ""]}) -- 0x7f66680b0990 con 0x7f6648079d20 2024-09-16T15:50:04.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:04.795+0000 7f6664c00640 1 -- 172.21.15.151:0/2176233292 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f66680b0990 con 0x7f6648079d20 2024-09-16T15:50:04.797 INFO:teuthology.orchestra.run.smithi151.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-16T15:50:04.803 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:50:04.792+0000 7f48efe00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-16T15:50:04.811 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T15:50:04.811 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone53393 2024-09-16T15:50:05.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:05.130+0000 7ff261a00640 1 -- 172.21.15.151:0/1876228147 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff25c161460 msgr2=0x7ff25c0d21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:05.133 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:05.130+0000 7ff261a00640 1 --2- 172.21.15.151:0/1876228147 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff25c161460 0x7ff25c0d21b0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7ff25c0b9ea0 tx=0x7ff25402a4f0 comp rx=0 tx=0).stop 2024-09-16T15:50:05.133 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:05.131+0000 7ff261a00640 1 -- 172.21.15.151:0/1876228147 shutdown_connections 2024-09-16T15:50:05.133 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:05.131+0000 7ff261a00640 1 --2- 172.21.15.151:0/1876228147 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff25c0b5270 0x7ff25c0b33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:05.133 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:05.131+0000 7ff261a00640 1 --2- 172.21.15.151:0/1876228147 >> v2:172.21.15.167:3300/0 conn(0x7ff25c0b4d40 0x7ff25c0d26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:05.133 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:05.131+0000 7ff261a00640 1 --2- 172.21.15.151:0/1876228147 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff25c161460 0x7ff25c0d21b0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:05.133 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:05.131+0000 7ff261a00640 1 -- 172.21.15.151:0/1876228147 >> 172.21.15.151:0/1876228147 conn(0x7ff25c09d0b0 msgr2=0x7ff25c0d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:05.133 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:05.131+0000 7ff261a00640 1 -- 172.21.15.151:0/1876228147 shutdown_connections 2024-09-16T15:50:05.133 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:05.131+0000 7ff261a00640 1 -- 172.21.15.151:0/1876228147 wait complete. 2024-09-16T15:50:05.135 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:05.134+0000 7ff261a00640 1 Processor -- start 2024-09-16T15:50:05.135 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:05.134+0000 7ff261a00640 1 -- start start 2024-09-16T15:50:05.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:05.136+0000 7ff261a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff25c0b4d40 0x7ff25c09e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:05.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:05.136+0000 7ff261a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff25c0b5270 0x7ff25c09ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:05.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:05.136+0000 7ff261a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff25c161460 0x7ff25c1f4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:05.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:05.136+0000 7ff261a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff25c0b38f0 con 0x7ff25c161460 2024-09-16T15:50:05.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:05.136+0000 7ff261a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff25c0d1220 con 0x7ff25c0b4d40 2024-09-16T15:50:05.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:05.136+0000 7ff261a00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7ff25c0b3a70 con 0x7ff25c0b5270 2024-09-16T15:50:05.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:05.136+0000 7ff261000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff25c161460 0x7ff25c1f4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:05.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:05.137+0000 7ff261000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff25c161460 0x7ff25c1f4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:57144/0 (socket says 172.21.15.151:57144) 2024-09-16T15:50:05.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:05.137+0000 7ff261000640 1 -- 172.21.15.151:0/2606329227 learned_addr learned my addr 172.21.15.151:0/2606329227 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:05.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:05.137+0000 7ff261000640 1 -- 172.21.15.151:0/2606329227 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff25c0b5270 msgr2=0x7ff25c09ecb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T15:50:05.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:05.137+0000 7ff261000640 1 --2- 172.21.15.151:0/2606329227 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff25c0b5270 0x7ff25c09ecb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:05.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:05.137+0000 7ff261000640 1 -- 172.21.15.151:0/2606329227 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff25c0b4d40 msgr2=0x7ff25c09e050 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T15:50:05.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:05.137+0000 7ff261000640 1 --2- 172.21.15.151:0/2606329227 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff25c0b4d40 0x7ff25c09e050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:05.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:05.137+0000 7ff261000640 1 -- 172.21.15.151:0/2606329227 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff25c1519f0 con 0x7ff25c161460 2024-09-16T15:50:05.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:05.138+0000 7ff261000640 1 --2- 172.21.15.151:0/2606329227 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff25c161460 0x7ff25c1f4740 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7ff24800b420 tx=0x7ff24800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:05.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:05.139+0000 7ff253e00640 1 -- 172.21.15.151:0/2606329227 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff248011830 con 0x7ff25c161460 2024-09-16T15:50:05.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:05.139+0000 7ff261a00640 1 -- 172.21.15.151:0/2606329227 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff25c1f5000 con 0x7ff25c161460 2024-09-16T15:50:05.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:05.139+0000 7ff261a00640 1 -- 172.21.15.151:0/2606329227 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff25c1f55c0 con 0x7ff25c161460 2024-09-16T15:50:05.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:05.140+0000 7ff253e00640 1 -- 172.21.15.151:0/2606329227 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff248016070 con 0x7ff25c161460 2024-09-16T15:50:05.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:05.140+0000 7ff253e00640 1 -- 172.21.15.151:0/2606329227 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff248011bd0 con 0x7ff25c161460 2024-09-16T15:50:05.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:05.141+0000 7ff261a00640 1 -- 172.21.15.151:0/2606329227 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff234005290 con 0x7ff25c161460 2024-09-16T15:50:05.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:05.154+0000 7ff253e00640 1 -- 172.21.15.151:0/2606329227 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 36) ==== 103225+0+0 (secure 0 0 0) 0x7ff24801b460 con 0x7ff25c161460 2024-09-16T15:50:05.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:05.155+0000 7ff253e00640 1 --2- 172.21.15.151:0/2606329227 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff23c079db0 0x7ff23c07c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:05.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:05.156+0000 7ff253e00640 1 -- 172.21.15.151:0/2606329227 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7ff24809a270 con 0x7ff25c161460 2024-09-16T15:50:05.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:05.162+0000 7ff253e00640 1 -- 172.21.15.151:0/2606329227 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff248063f30 con 0x7ff25c161460 2024-09-16T15:50:05.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:05.173+0000 7ff25be00640 1 --2- 172.21.15.151:0/2606329227 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff23c079db0 0x7ff23c07c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:05.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:05.216+0000 7ff25be00640 1 --2- 172.21.15.151:0/2606329227 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff23c079db0 0x7ff23c07c270 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7ff254033000 tx=0x7ff254004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:05.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:05.627+0000 7ff261a00640 1 -- 172.21.15.151:0/2606329227 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone53393", "target": ["mon-mgr", ""]}) -- 0x7ff234000bc0 con 0x7ff23c079db0 2024-09-16T15:50:05.986 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:50:05.983+0000 7f48efe00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-16T15:50:05.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:05.989+0000 7ff253e00640 1 -- 172.21.15.151:0/2606329227 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7ff234000bc0 con 0x7ff23c079db0 2024-09-16T15:50:05.990 INFO:teuthology.orchestra.run.smithi151.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-16T15:50:06.003 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T15:50:06.003 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone15034 2024-09-16T15:50:06.312 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:06.310+0000 7f0eca600640 1 -- 172.21.15.151:0/4274379522 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ec40cd6f0 msgr2=0x7f0ec40d01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:06.312 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:06.310+0000 7f0eca600640 1 --2- 172.21.15.151:0/4274379522 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ec40cd6f0 0x7f0ec40d01d0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0005fd0 tx=0x7f0eb002a610 comp rx=0 tx=0).stop 2024-09-16T15:50:06.312 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:06.311+0000 7f0eca600640 1 -- 172.21.15.151:0/4274379522 shutdown_connections 2024-09-16T15:50:06.312 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:06.311+0000 7f0eca600640 1 --2- 172.21.15.151:0/4274379522 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ec40cd6f0 0x7f0ec40d01d0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:06.312 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:06.311+0000 7f0eca600640 1 --2- 172.21.15.151:0/4274379522 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ec415bf10 0x7f0ec41604b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:06.312 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:06.311+0000 7f0eca600640 1 -- 172.21.15.151:0/4274379522 >> 172.21.15.151:0/4274379522 conn(0x7f0ec40d1e20 msgr2=0x7f0ec40d2230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:06.313 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:06.312+0000 7f0eca600640 1 -- 172.21.15.151:0/4274379522 shutdown_connections 2024-09-16T15:50:06.313 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:06.312+0000 7f0eca600640 1 -- 172.21.15.151:0/4274379522 wait complete. 2024-09-16T15:50:06.316 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:06.315+0000 7f0eca600640 1 Processor -- start 2024-09-16T15:50:06.316 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:06.316+0000 7f0eca600640 1 -- start start 2024-09-16T15:50:06.318 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:06.317+0000 7f0eca600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ec40cd6f0 0x7f0ec41f2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:06.318 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:06.317+0000 7f0eca600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ec40d0710 0x7f0ec40cab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:06.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:06.317+0000 7f0eca600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ec415bf10 0x7f0ec4157ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:06.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:06.317+0000 7f0eca600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec40cdcb0 con 0x7f0ec40d0710 2024-09-16T15:50:06.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:06.317+0000 7f0eca600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec40cdb30 con 0x7f0ec415bf10 2024-09-16T15:50:06.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:06.317+0000 7f0eca600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f0ec40cf9f0 con 0x7f0ec40cd6f0 2024-09-16T15:50:06.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:06.318+0000 7f0ec2a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ec40d0710 0x7f0ec40cab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:06.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:06.318+0000 7f0ec2a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ec40d0710 0x7f0ec40cab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:57146/0 (socket says 172.21.15.151:57146) 2024-09-16T15:50:06.320 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:06.318+0000 7f0ec2a00640 1 -- 172.21.15.151:0/1894597757 learned_addr learned my addr 172.21.15.151:0/1894597757 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:06.320 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:06.318+0000 7f0ec3400640 1 --2- 172.21.15.151:0/1894597757 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ec40cd6f0 0x7f0ec41f2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:06.320 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:06.319+0000 7f0ec2a00640 1 -- 172.21.15.151:0/1894597757 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ec40cd6f0 msgr2=0x7f0ec41f2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:06.320 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:06.319+0000 7f0ec2a00640 1 --2- 172.21.15.151:0/1894597757 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0ec40cd6f0 0x7f0ec41f2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:06.320 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:06.319+0000 7f0ec2a00640 1 -- 172.21.15.151:0/1894597757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ec415bf10 msgr2=0x7f0ec4157ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:06.320 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:06.319+0000 7f0ec2a00640 1 --2- 172.21.15.151:0/1894597757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ec415bf10 0x7f0ec4157ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:06.320 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:06.319+0000 7f0ec2a00640 1 -- 172.21.15.151:0/1894597757 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ec41521e0 con 0x7f0ec40d0710 2024-09-16T15:50:06.321 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:06.319+0000 7f0ec2a00640 1 --2- 172.21.15.151:0/1894597757 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ec40d0710 0x7f0ec40cab60 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0009850 tx=0x7f0eb00042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:06.321 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:06.320+0000 7f0ebb400640 1 -- 172.21.15.151:0/1894597757 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb0041070 con 0x7f0ec40d0710 2024-09-16T15:50:06.321 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:06.320+0000 7f0ebb400640 1 -- 172.21.15.151:0/1894597757 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0eb0031940 con 0x7f0ec40d0710 2024-09-16T15:50:06.321 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:06.320+0000 7f0ebb400640 1 -- 172.21.15.151:0/1894597757 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb0031d50 con 0x7f0ec40d0710 2024-09-16T15:50:06.321 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:06.320+0000 7f0eca600640 1 -- 172.21.15.151:0/1894597757 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ec41524c0 con 0x7f0ec40d0710 2024-09-16T15:50:06.321 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:06.320+0000 7f0eca600640 1 -- 172.21.15.151:0/1894597757 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ec4152a80 con 0x7f0ec40d0710 2024-09-16T15:50:06.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:06.321+0000 7f0eca600640 1 -- 172.21.15.151:0/1894597757 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ec40d01d0 con 0x7f0ec40d0710 2024-09-16T15:50:06.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:06.323+0000 7f0ebb400640 1 -- 172.21.15.151:0/1894597757 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 36) ==== 103225+0+0 (secure 0 0 0) 0x7f0eb0044070 con 0x7f0ec40d0710 2024-09-16T15:50:06.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:06.326+0000 7f0ebb400640 1 --2- 172.21.15.151:0/1894597757 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0ea4079db0 0x7f0ea407c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:06.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:06.328+0000 7f0ebb400640 1 -- 172.21.15.151:0/1894597757 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7f0eb00ba9f0 con 0x7f0ec40d0710 2024-09-16T15:50:06.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:06.334+0000 7f0ebb400640 1 -- 172.21.15.151:0/1894597757 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0eb00846b0 con 0x7f0ec40d0710 2024-09-16T15:50:06.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:06.373+0000 7f0ec3400640 1 --2- 172.21.15.151:0/1894597757 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0ea4079db0 0x7f0ea407c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:06.434 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:06.430+0000 7f0ec3400640 1 --2- 172.21.15.151:0/1894597757 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0ea4079db0 0x7f0ea407c270 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4005d70 tx=0x7f0eb4005d00 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:06.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:06.828+0000 7f0eca600640 1 -- 172.21.15.151:0/1894597757 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone15034", "target": ["mon-mgr", ""]}) -- 0x7f0ec4069630 con 0x7f0ea4079db0 2024-09-16T15:50:07.144 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:50:07.142+0000 7f48efe00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-16T15:50:07.146 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:07.145+0000 7f0ebb400640 1 -- 172.21.15.151:0/1894597757 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f0ec4069630 con 0x7f0ea4079db0 2024-09-16T15:50:07.146 INFO:teuthology.orchestra.run.smithi151.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-16T15:50:07.164 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T15:50:07.164 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone77745 2024-09-16T15:50:07.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:07.468+0000 7f1fee600640 1 -- 172.21.15.151:0/273358027 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1fe80af170 msgr2=0x7f1fe80cdad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:07.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:07.468+0000 7f1fee600640 1 --2- 172.21.15.151:0/273358027 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1fe80af170 0x7f1fe80cdad0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f1fe80d0c10 tx=0x7f1fd802a560 comp rx=0 tx=0).stop 2024-09-16T15:50:07.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:07.468+0000 7f1fee600640 1 -- 172.21.15.151:0/273358027 shutdown_connections 2024-09-16T15:50:07.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:07.468+0000 7f1fee600640 1 --2- 172.21.15.151:0/273358027 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1fe80b4370 0x7f1fe80ce010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:07.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:07.468+0000 7f1fee600640 1 --2- 172.21.15.151:0/273358027 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1fe80af170 0x7f1fe80cdad0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:07.471 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:07.468+0000 7f1fee600640 1 --2- 172.21.15.151:0/273358027 >> v2:172.21.15.167:3300/0 conn(0x7f1fe8161490 0x7f1fe80b3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:07.471 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:07.468+0000 7f1fee600640 1 -- 172.21.15.151:0/273358027 >> 172.21.15.151:0/273358027 conn(0x7f1fe80b1840 msgr2=0x7f1fe80b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:07.471 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:07.469+0000 7f1fee600640 1 -- 172.21.15.151:0/273358027 shutdown_connections 2024-09-16T15:50:07.471 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:07.469+0000 7f1fee600640 1 -- 172.21.15.151:0/273358027 wait complete. 2024-09-16T15:50:07.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:07.472+0000 7f1fee600640 1 Processor -- start 2024-09-16T15:50:07.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:07.473+0000 7f1fee600640 1 -- start start 2024-09-16T15:50:07.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:07.475+0000 7f1fee600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1fe80b4370 0x7f1fe815d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:07.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:07.475+0000 7f1fee600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1fe8161490 0x7f1fe81f9b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:07.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:07.475+0000 7f1fee600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1fe815e6c0 0x7f1fe8156870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:07.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:07.475+0000 7f1fee600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe806f8a0 con 0x7f1fe80b4370 2024-09-16T15:50:07.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:07.475+0000 7f1fee600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe8161dc0 con 0x7f1fe8161490 2024-09-16T15:50:07.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:07.475+0000 7f1fee600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f1fe806fa20 con 0x7f1fe815e6c0 2024-09-16T15:50:07.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:07.476+0000 7f1fe7400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1fe80b4370 0x7f1fe815d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:07.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:07.476+0000 7f1fe7400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1fe80b4370 0x7f1fe815d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:57168/0 (socket says 172.21.15.151:57168) 2024-09-16T15:50:07.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:07.476+0000 7f1fe6a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1fe8161490 0x7f1fe81f9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:07.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:07.476+0000 7f1fe7400640 1 -- 172.21.15.151:0/1444457073 learned_addr learned my addr 172.21.15.151:0/1444457073 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:07.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:07.477+0000 7f1fe7e00640 1 --2- 172.21.15.151:0/1444457073 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1fe815e6c0 0x7f1fe8156870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:07.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:07.477+0000 7f1fe7e00640 1 -- 172.21.15.151:0/1444457073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1fe8161490 msgr2=0x7f1fe81f9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:07.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:07.477+0000 7f1fe7e00640 1 --2- 172.21.15.151:0/1444457073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1fe8161490 0x7f1fe81f9b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:07.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:07.477+0000 7f1fe7e00640 1 -- 172.21.15.151:0/1444457073 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1fe80b4370 msgr2=0x7f1fe815d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:07.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:07.477+0000 7f1fe7e00640 1 --2- 172.21.15.151:0/1444457073 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1fe80b4370 0x7f1fe815d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:07.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:07.477+0000 7f1fe7e00640 1 -- 172.21.15.151:0/1444457073 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fe81fa050 con 0x7f1fe815e6c0 2024-09-16T15:50:07.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:07.478+0000 7f1fe7400640 1 --2- 172.21.15.151:0/1444457073 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1fe80b4370 0x7f1fe815d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:50:07.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:07.478+0000 7f1fe7e00640 1 --2- 172.21.15.151:0/1444457073 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1fe815e6c0 0x7f1fe8156870 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1fe000be50 tx=0x7f1fe000bf40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:07.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:07.479+0000 7f1fdf400640 1 -- 172.21.15.151:0/1444457073 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe0019070 con 0x7f1fe815e6c0 2024-09-16T15:50:07.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:07.479+0000 7f1fdf400640 1 -- 172.21.15.151:0/1444457073 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1fe0014440 con 0x7f1fe815e6c0 2024-09-16T15:50:07.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:07.479+0000 7f1fee600640 1 -- 172.21.15.151:0/1444457073 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fe815c6a0 con 0x7f1fe815e6c0 2024-09-16T15:50:07.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:07.479+0000 7f1fee600640 1 -- 172.21.15.151:0/1444457073 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fe815cc60 con 0x7f1fe815e6c0 2024-09-16T15:50:07.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:07.479+0000 7f1fdf400640 1 -- 172.21.15.151:0/1444457073 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe0014890 con 0x7f1fe815e6c0 2024-09-16T15:50:07.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:07.483+0000 7f1fee600640 1 -- 172.21.15.151:0/1444457073 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fc0005290 con 0x7f1fe815e6c0 2024-09-16T15:50:07.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:07.484+0000 7f1fdf400640 1 -- 172.21.15.151:0/1444457073 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 36) ==== 103225+0+0 (secure 0 0 0) 0x7f1fe001c070 con 0x7f1fe815e6c0 2024-09-16T15:50:07.487 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:07.486+0000 7f1fdf400640 1 --2- 172.21.15.151:0/1444457073 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1fc8079db0 0x7f1fc807c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:07.488 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:07.487+0000 7f1fdf400640 1 -- 172.21.15.151:0/1444457073 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7f1fe0099830 con 0x7f1fe815e6c0 2024-09-16T15:50:07.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:07.493+0000 7f1fe7400640 1 --2- 172.21.15.151:0/1444457073 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1fc8079db0 0x7f1fc807c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:07.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:07.495+0000 7f1fdf400640 1 -- 172.21.15.151:0/1444457073 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1fe0062ea0 con 0x7f1fe815e6c0 2024-09-16T15:50:07.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:07.501+0000 7f1fe7400640 1 --2- 172.21.15.151:0/1444457073 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1fc8079db0 0x7f1fc807c270 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f1fd40058c0 tx=0x7f1fd400b410 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:07.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:07.882+0000 7f1fee600640 1 -- 172.21.15.151:0/1444457073 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone77745", "target": ["mon-mgr", ""]}) -- 0x7f1fc0000bc0 con 0x7f1fc8079db0 2024-09-16T15:50:08.132 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:50:08.130+0000 7f48efe00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-16T15:50:08.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:08.131+0000 7f1fdf400640 1 -- 172.21.15.151:0/1444457073 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f1fc0000bc0 con 0x7f1fc8079db0 2024-09-16T15:50:08.132 INFO:teuthology.orchestra.run.smithi151.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-16T15:50:08.148 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T15:50:08.148 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone22126 2024-09-16T15:50:08.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:08.454+0000 7f5132800640 1 -- 172.21.15.151:0/2620141958 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f512c0ce010 msgr2=0x7f512c0ce480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:08.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:08.454+0000 7f5132800640 1 --2- 172.21.15.151:0/2620141958 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f512c0ce010 0x7f512c0ce480 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f512c069430 tx=0x7f512402aa50 comp rx=0 tx=0).stop 2024-09-16T15:50:08.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:08.454+0000 7f5132800640 1 -- 172.21.15.151:0/2620141958 shutdown_connections 2024-09-16T15:50:08.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:08.454+0000 7f5132800640 1 --2- 172.21.15.151:0/2620141958 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f512c0ce010 0x7f512c0ce480 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:08.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:08.454+0000 7f5132800640 1 --2- 172.21.15.151:0/2620141958 >> v2:172.21.15.167:3300/0 conn(0x7f512c15dc00 0x7f512c0cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:08.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:08.454+0000 7f5132800640 1 --2- 172.21.15.151:0/2620141958 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f512c0ca2b0 0x7f512c0ca690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:08.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:08.455+0000 7f5132800640 1 -- 172.21.15.151:0/2620141958 >> 172.21.15.151:0/2620141958 conn(0x7f512c0cf9f0 msgr2=0x7f512c0cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:08.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:08.455+0000 7f5132800640 1 -- 172.21.15.151:0/2620141958 shutdown_connections 2024-09-16T15:50:08.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:08.455+0000 7f5132800640 1 -- 172.21.15.151:0/2620141958 wait complete. 2024-09-16T15:50:08.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:08.458+0000 7f5132800640 1 Processor -- start 2024-09-16T15:50:08.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:08.459+0000 7f5132800640 1 -- start start 2024-09-16T15:50:08.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:08.460+0000 7f5132800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f512c0ca2b0 0x7f512c0c6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:08.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:08.460+0000 7f5132800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f512c0ce010 0x7f512c0c7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:08.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:08.461+0000 7f5132800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f512c15dc00 0x7f512c153ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:08.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:08.461+0000 7f5132800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f512c15a9a0 con 0x7f512c0ce010 2024-09-16T15:50:08.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:08.461+0000 7f5132800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f512c15a820 con 0x7f512c15dc00 2024-09-16T15:50:08.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:08.461+0000 7f5132800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f512c15ab20 con 0x7f512c0ca2b0 2024-09-16T15:50:08.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:08.461+0000 7f5130a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f512c0ce010 0x7f512c0c7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:08.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:08.461+0000 7f5130a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f512c0ce010 0x7f512c0c7360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:57188/0 (socket says 172.21.15.151:57188) 2024-09-16T15:50:08.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:08.461+0000 7f5130a00640 1 -- 172.21.15.151:0/3850096102 learned_addr learned my addr 172.21.15.151:0/3850096102 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:08.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:08.462+0000 7f5131e00640 1 --2- 172.21.15.151:0/3850096102 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f512c15dc00 0x7f512c153ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:08.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:08.462+0000 7f5131400640 1 --2- 172.21.15.151:0/3850096102 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f512c0ca2b0 0x7f512c0c6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:08.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:08.462+0000 7f5130a00640 1 -- 172.21.15.151:0/3850096102 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f512c0ca2b0 msgr2=0x7f512c0c6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:08.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:08.462+0000 7f5130a00640 1 --2- 172.21.15.151:0/3850096102 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f512c0ca2b0 0x7f512c0c6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:08.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:08.462+0000 7f5130a00640 1 -- 172.21.15.151:0/3850096102 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f512c15dc00 msgr2=0x7f512c153ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:08.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:08.462+0000 7f5130a00640 1 --2- 172.21.15.151:0/3850096102 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f512c15dc00 0x7f512c153ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:08.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:08.462+0000 7f5130a00640 1 -- 172.21.15.151:0/3850096102 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f512c0c7b90 con 0x7f512c0ce010 2024-09-16T15:50:08.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:08.462+0000 7f5131400640 1 --2- 172.21.15.151:0/3850096102 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f512c0ca2b0 0x7f512c0c6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:50:08.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:08.462+0000 7f5130a00640 1 --2- 172.21.15.151:0/3850096102 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f512c0ce010 0x7f512c0c7360 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f511800d740 tx=0x7f511800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:08.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:08.463+0000 7f5129600640 1 -- 172.21.15.151:0/3850096102 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5118017070 con 0x7f512c0ce010 2024-09-16T15:50:08.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:08.463+0000 7f5129600640 1 -- 172.21.15.151:0/3850096102 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5118009d20 con 0x7f512c0ce010 2024-09-16T15:50:08.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:08.463+0000 7f5129600640 1 -- 172.21.15.151:0/3850096102 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51180106c0 con 0x7f512c0ce010 2024-09-16T15:50:08.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:08.463+0000 7f5132800640 1 -- 172.21.15.151:0/3850096102 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f512c1f4800 con 0x7f512c0ce010 2024-09-16T15:50:08.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:08.463+0000 7f5132800640 1 -- 172.21.15.151:0/3850096102 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f512c1f4dc0 con 0x7f512c0ce010 2024-09-16T15:50:08.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:08.465+0000 7f5132800640 1 -- 172.21.15.151:0/3850096102 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f512c0ce480 con 0x7f512c0ce010 2024-09-16T15:50:08.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:08.467+0000 7f5129600640 1 -- 172.21.15.151:0/3850096102 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 36) ==== 103225+0+0 (secure 0 0 0) 0x7f511801a070 con 0x7f512c0ce010 2024-09-16T15:50:08.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:08.469+0000 7f5129600640 1 --2- 172.21.15.151:0/3850096102 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f510c079e30 0x7f510c07c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:08.471 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:08.470+0000 7f5129600640 1 -- 172.21.15.151:0/3850096102 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7f5118099af0 con 0x7f512c0ce010 2024-09-16T15:50:08.484 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:08.480+0000 7f5129600640 1 -- 172.21.15.151:0/3850096102 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f511809fe20 con 0x7f512c0ce010 2024-09-16T15:50:08.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:08.533+0000 7f5131400640 1 --2- 172.21.15.151:0/3850096102 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f510c079e30 0x7f510c07c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:08.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:08.540+0000 7f5131400640 1 --2- 172.21.15.151:0/3850096102 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f510c079e30 0x7f510c07c2f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5120006fd0 tx=0x7f5120008040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:08.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:08.883+0000 7f5132800640 1 -- 172.21.15.151:0/3850096102 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone22126", "target": ["mon-mgr", ""]}) -- 0x7f512c159ba0 con 0x7f510c079e30 2024-09-16T15:50:09.327 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:50:09.326+0000 7f48efe00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-16T15:50:09.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:09.329+0000 7f5129600640 1 -- 172.21.15.151:0/3850096102 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f512c159ba0 con 0x7f510c079e30 2024-09-16T15:50:09.330 INFO:teuthology.orchestra.run.smithi151.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-16T15:50:09.350 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T15:50:09.351 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone81887 2024-09-16T15:50:09.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:09.736+0000 7fa41ac00640 1 -- 172.21.15.151:0/1569641470 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa41415fbe0 msgr2=0x7fa41415e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:09.738 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:09.736+0000 7fa41ac00640 1 --2- 172.21.15.151:0/1569641470 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa41415fbe0 0x7fa41415e080 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fa4140b2e60 tx=0x7fa40802a7c0 comp rx=0 tx=0).stop 2024-09-16T15:50:09.738 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:09.736+0000 7fa41ac00640 1 -- 172.21.15.151:0/1569641470 shutdown_connections 2024-09-16T15:50:09.738 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:09.737+0000 7fa41ac00640 1 --2- 172.21.15.151:0/1569641470 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa41415fbe0 0x7fa41415e080 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:09.738 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:09.737+0000 7fa41ac00640 1 -- 172.21.15.151:0/1569641470 >> 172.21.15.151:0/1569641470 conn(0x7fa4140b5680 msgr2=0x7fa4140b5a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:09.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:09.740+0000 7fa41ac00640 1 -- 172.21.15.151:0/1569641470 shutdown_connections 2024-09-16T15:50:09.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:09.740+0000 7fa41ac00640 1 -- 172.21.15.151:0/1569641470 wait complete. 2024-09-16T15:50:09.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:09.743+0000 7fa41ac00640 1 Processor -- start 2024-09-16T15:50:09.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:09.744+0000 7fa41ac00640 1 -- start start 2024-09-16T15:50:09.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:09.745+0000 7fa41ac00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4140b1880 0x7fa4141f5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:09.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:09.746+0000 7fa41ac00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa41415f790 0x7fa414159450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:09.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:09.746+0000 7fa41ac00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa41415fbe0 0x7fa414155fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:09.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:09.746+0000 7fa41ac00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa41415e740 con 0x7fa41415fbe0 2024-09-16T15:50:09.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:09.746+0000 7fa41ac00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa41415e5c0 con 0x7fa4140b1880 2024-09-16T15:50:09.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:09.746+0000 7fa41ac00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fa41415e8c0 con 0x7fa41415f790 2024-09-16T15:50:09.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:09.746+0000 7fa41a200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa41415fbe0 0x7fa414155fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:09.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:09.746+0000 7fa41a200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa41415fbe0 0x7fa414155fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:57200/0 (socket says 172.21.15.151:57200) 2024-09-16T15:50:09.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:09.747+0000 7fa41a200640 1 -- 172.21.15.151:0/292924613 learned_addr learned my addr 172.21.15.151:0/292924613 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:09.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:09.747+0000 7fa41a200640 1 -- 172.21.15.151:0/292924613 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa41415f790 msgr2=0x7fa414159450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:09.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:09.747+0000 7fa41a200640 1 --2- 172.21.15.151:0/292924613 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa41415f790 0x7fa414159450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:09.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:09.747+0000 7fa41a200640 1 -- 172.21.15.151:0/292924613 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4140b1880 msgr2=0x7fa4141f5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:09.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:09.747+0000 7fa41a200640 1 --2- 172.21.15.151:0/292924613 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4140b1880 0x7fa4141f5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:09.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:09.747+0000 7fa41a200640 1 -- 172.21.15.151:0/292924613 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa414159990 con 0x7fa41415fbe0 2024-09-16T15:50:09.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:09.748+0000 7fa41a200640 1 --2- 172.21.15.151:0/292924613 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa41415fbe0 0x7fa414155fa0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fa408034040 tx=0x7fa408003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:09.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:09.751+0000 7fa411600640 1 -- 172.21.15.151:0/292924613 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa408040070 con 0x7fa41415fbe0 2024-09-16T15:50:09.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:09.752+0000 7fa411600640 1 -- 172.21.15.151:0/292924613 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa408004540 con 0x7fa41415fbe0 2024-09-16T15:50:09.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:09.752+0000 7fa411600640 1 -- 172.21.15.151:0/292924613 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40803c420 con 0x7fa41415fbe0 2024-09-16T15:50:09.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:09.752+0000 7fa41ac00640 1 -- 172.21.15.151:0/292924613 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4140b4d70 con 0x7fa41415fbe0 2024-09-16T15:50:09.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:09.752+0000 7fa41ac00640 1 -- 172.21.15.151:0/292924613 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa414157a30 con 0x7fa41415fbe0 2024-09-16T15:50:09.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:09.757+0000 7fa41ac00640 1 -- 172.21.15.151:0/292924613 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3f0005290 con 0x7fa41415fbe0 2024-09-16T15:50:09.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:09.761+0000 7fa411600640 1 -- 172.21.15.151:0/292924613 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 36) ==== 103225+0+0 (secure 0 0 0) 0x7fa408043070 con 0x7fa41415fbe0 2024-09-16T15:50:09.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:09.763+0000 7fa411600640 1 --2- 172.21.15.151:0/292924613 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa3f4079e00 0x7fa3f407c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:09.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:09.764+0000 7fa411600640 1 -- 172.21.15.151:0/292924613 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7fa4080bac20 con 0x7fa41415fbe0 2024-09-16T15:50:09.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:09.773+0000 7fa419800640 1 --2- 172.21.15.151:0/292924613 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa3f4079e00 0x7fa3f407c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:09.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:09.774+0000 7fa411600640 1 -- 172.21.15.151:0/292924613 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa408084920 con 0x7fa41415fbe0 2024-09-16T15:50:09.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:09.795+0000 7fa419800640 1 --2- 172.21.15.151:0/292924613 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa3f4079e00 0x7fa3f407c2c0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa40c000f60 tx=0x7fa40c007990 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:10.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:10.242+0000 7fa41ac00640 1 -- 172.21.15.151:0/292924613 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone81887", "target": ["mon-mgr", ""]}) -- 0x7fa3f0000bc0 con 0x7fa3f4079e00 2024-09-16T15:50:10.706 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:50:10.703+0000 7f48efe00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-16T15:50:10.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:10.706+0000 7fa411600640 1 -- 172.21.15.151:0/292924613 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7fa3f0000bc0 con 0x7fa3f4079e00 2024-09-16T15:50:10.707 INFO:teuthology.orchestra.run.smithi151.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-16T15:50:10.723 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T15:50:10.724 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone57948 2024-09-16T15:50:11.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:11.013+0000 7fb6ac800640 1 -- 172.21.15.151:0/1491250336 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6a40d15e0 msgr2=0x7fb6a40d2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:11.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:11.013+0000 7fb6ac800640 1 --2- 172.21.15.151:0/1491250336 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6a40d15e0 0x7fb6a40d2530 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fb698009830 tx=0x7fb69802a480 comp rx=0 tx=0).stop 2024-09-16T15:50:11.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:11.014+0000 7fb6ac800640 1 -- 172.21.15.151:0/1491250336 shutdown_connections 2024-09-16T15:50:11.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:11.014+0000 7fb6ac800640 1 --2- 172.21.15.151:0/1491250336 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6a40d15e0 0x7fb6a40d2530 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:11.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:11.014+0000 7fb6ac800640 1 --2- 172.21.15.151:0/1491250336 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb6a40d0cc0 0x7fb6a40d10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:11.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:11.014+0000 7fb6ac800640 1 -- 172.21.15.151:0/1491250336 >> 172.21.15.151:0/1491250336 conn(0x7fb6a40d4040 msgr2=0x7fb6a40d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:11.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:11.014+0000 7fb6ac800640 1 -- 172.21.15.151:0/1491250336 shutdown_connections 2024-09-16T15:50:11.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:11.015+0000 7fb6ac800640 1 -- 172.21.15.151:0/1491250336 wait complete. 2024-09-16T15:50:11.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:11.017+0000 7fb6ac800640 1 Processor -- start 2024-09-16T15:50:11.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:11.017+0000 7fb6ac800640 1 -- start start 2024-09-16T15:50:11.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:11.018+0000 7fb6ac800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6a40b4e90 0x7fb6a415baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:11.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:11.018+0000 7fb6ac800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb6a40d0cc0 0x7fb6a414f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:11.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:11.018+0000 7fb6ac800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb6a40d15e0 0x7fb6a4158ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:11.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:11.018+0000 7fb6ac800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a40cedc0 con 0x7fb6a40d0cc0 2024-09-16T15:50:11.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:11.018+0000 7fb6ac800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a40cec40 con 0x7fb6a40b4e90 2024-09-16T15:50:11.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:11.018+0000 7fb6ac800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fb6a40cef40 con 0x7fb6a40d15e0 2024-09-16T15:50:11.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:11.019+0000 7fb6a9400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb6a40d0cc0 0x7fb6a414f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:11.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:11.019+0000 7fb6a9400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb6a40d0cc0 0x7fb6a414f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:57206/0 (socket says 172.21.15.151:57206) 2024-09-16T15:50:11.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:11.019+0000 7fb6a9400640 1 -- 172.21.15.151:0/1578732585 learned_addr learned my addr 172.21.15.151:0/1578732585 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:11.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:11.019+0000 7fb6a9e00640 1 --2- 172.21.15.151:0/1578732585 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6a40b4e90 0x7fb6a415baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:11.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:11.019+0000 7fb6aa800640 1 --2- 172.21.15.151:0/1578732585 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb6a40d15e0 0x7fb6a4158ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:11.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:11.020+0000 7fb6a9400640 1 -- 172.21.15.151:0/1578732585 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb6a40d15e0 msgr2=0x7fb6a4158ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:11.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:11.020+0000 7fb6a9400640 1 --2- 172.21.15.151:0/1578732585 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb6a40d15e0 0x7fb6a4158ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:11.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:11.020+0000 7fb6a9400640 1 -- 172.21.15.151:0/1578732585 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6a40b4e90 msgr2=0x7fb6a415baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:11.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:11.020+0000 7fb6a9400640 1 --2- 172.21.15.151:0/1578732585 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6a40b4e90 0x7fb6a415baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:11.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:11.020+0000 7fb6a9400640 1 -- 172.21.15.151:0/1578732585 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6a414faf0 con 0x7fb6a40d0cc0 2024-09-16T15:50:11.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:11.020+0000 7fb6a9400640 1 --2- 172.21.15.151:0/1578732585 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb6a40d0cc0 0x7fb6a414f5b0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fb69400c500 tx=0x7fb69400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:11.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:11.020+0000 7fb69e000640 1 -- 172.21.15.151:0/1578732585 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6940052d0 con 0x7fb6a40d0cc0 2024-09-16T15:50:11.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:11.021+0000 7fb69e000640 1 -- 172.21.15.151:0/1578732585 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb694005470 con 0x7fb6a40d0cc0 2024-09-16T15:50:11.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:11.021+0000 7fb69e000640 1 -- 172.21.15.151:0/1578732585 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb694012870 con 0x7fb6a40d0cc0 2024-09-16T15:50:11.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:11.021+0000 7fb6ac800640 1 -- 172.21.15.151:0/1578732585 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6a415aa00 con 0x7fb6a40d0cc0 2024-09-16T15:50:11.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:11.021+0000 7fb6ac800640 1 -- 172.21.15.151:0/1578732585 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6a415afc0 con 0x7fb6a40d0cc0 2024-09-16T15:50:11.024 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:11.023+0000 7fb6ac800640 1 -- 172.21.15.151:0/1578732585 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6a40d10a0 con 0x7fb6a40d0cc0 2024-09-16T15:50:11.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:11.025+0000 7fb69e000640 1 -- 172.21.15.151:0/1578732585 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 36) ==== 103225+0+0 (secure 0 0 0) 0x7fb6940035f0 con 0x7fb6a40d0cc0 2024-09-16T15:50:11.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:11.026+0000 7fb69e000640 1 --2- 172.21.15.151:0/1578732585 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb688079de0 0x7fb68807c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:11.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:11.026+0000 7fb69e000640 1 -- 172.21.15.151:0/1578732585 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7fb694099b90 con 0x7fb6a40d0cc0 2024-09-16T15:50:11.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:11.030+0000 7fb69e000640 1 -- 172.21.15.151:0/1578732585 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb694020060 con 0x7fb6a40d0cc0 2024-09-16T15:50:11.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:11.055+0000 7fb6a9e00640 1 --2- 172.21.15.151:0/1578732585 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb688079de0 0x7fb68807c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:11.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:11.104+0000 7fb6a9e00640 1 --2- 172.21.15.151:0/1578732585 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb688079de0 0x7fb68807c2a0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0005fd0 tx=0x7fb6a0009480 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:11.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:11.435+0000 7fb6ac800640 1 -- 172.21.15.151:0/1578732585 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone57948", "target": ["mon-mgr", ""]}) -- 0x7fb6a41593e0 con 0x7fb688079de0 2024-09-16T15:50:12.815 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:50:12.809+0000 7f48efe00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-16T15:50:12.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:12.821+0000 7fb69e000640 1 -- 172.21.15.151:0/1578732585 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7fb6a41593e0 con 0x7fb688079de0 2024-09-16T15:50:12.825 INFO:teuthology.orchestra.run.smithi151.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-16T15:50:12.839 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T15:50:12.840 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone35339 2024-09-16T15:50:13.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:13.113+0000 7f89b7200640 1 -- 172.21.15.151:0/3600820280 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89b0161760 msgr2=0x7f89b00ce4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:13.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:13.113+0000 7f89b7200640 1 --2- 172.21.15.151:0/3600820280 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89b0161760 0x7f89b00ce4a0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f89b00b4a60 tx=0x7f89a002a560 comp rx=0 tx=0).stop 2024-09-16T15:50:13.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:13.113+0000 7f89b7200640 1 -- 172.21.15.151:0/3600820280 shutdown_connections 2024-09-16T15:50:13.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:13.113+0000 7f89b7200640 1 --2- 172.21.15.151:0/3600820280 >> v2:172.21.15.167:3300/0 conn(0x7f89b00cec60 0x7f89b00cf0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:13.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:13.113+0000 7f89b7200640 1 --2- 172.21.15.151:0/3600820280 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f89b00af190 0x7f89b00b2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:13.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:13.113+0000 7f89b7200640 1 --2- 172.21.15.151:0/3600820280 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89b0161760 0x7f89b00ce4a0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:13.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:13.113+0000 7f89b7200640 1 -- 172.21.15.151:0/3600820280 >> 172.21.15.151:0/3600820280 conn(0x7f89b00b1840 msgr2=0x7f89b00b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:13.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:13.113+0000 7f89b7200640 1 -- 172.21.15.151:0/3600820280 shutdown_connections 2024-09-16T15:50:13.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:13.114+0000 7f89b7200640 1 -- 172.21.15.151:0/3600820280 wait complete. 2024-09-16T15:50:13.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:13.117+0000 7f89b7200640 1 Processor -- start 2024-09-16T15:50:13.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:13.118+0000 7f89b7200640 1 -- start start 2024-09-16T15:50:13.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:13.119+0000 7f89b7200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89b00af190 0x7f89b01eab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:13.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:13.119+0000 7f89b7200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89b00cec60 0x7f89b015f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:13.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:13.119+0000 7f89b7200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f89b0161760 0x7f89b01f7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:13.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:13.119+0000 7f89b7200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f89b00b3200 con 0x7f89b00cec60 2024-09-16T15:50:13.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:13.119+0000 7f89b7200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f89b00b3080 con 0x7f89b00af190 2024-09-16T15:50:13.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:13.119+0000 7f89b7200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f89b0162190 con 0x7f89b0161760 2024-09-16T15:50:13.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:13.120+0000 7f89af400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89b00cec60 0x7f89b015f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:13.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:13.120+0000 7f89af400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89b00cec60 0x7f89b015f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41856/0 (socket says 172.21.15.151:41856) 2024-09-16T15:50:13.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:13.120+0000 7f89af400640 1 -- 172.21.15.151:0/3717671614 learned_addr learned my addr 172.21.15.151:0/3717671614 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:13.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:13.121+0000 7f89af400640 1 -- 172.21.15.151:0/3717671614 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f89b0161760 msgr2=0x7f89b01f7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:13.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:13.121+0000 7f89af400640 1 --2- 172.21.15.151:0/3717671614 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f89b0161760 0x7f89b01f7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:13.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:13.121+0000 7f89af400640 1 -- 172.21.15.151:0/3717671614 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89b00af190 msgr2=0x7f89b01eab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:13.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:13.121+0000 7f89af400640 1 --2- 172.21.15.151:0/3717671614 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89b00af190 0x7f89b01eab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:13.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:13.121+0000 7f89af400640 1 -- 172.21.15.151:0/3717671614 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89b015fbb0 con 0x7f89b00cec60 2024-09-16T15:50:13.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:13.122+0000 7f89af400640 1 --2- 172.21.15.151:0/3717671614 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f89b00cec60 0x7f89b015f130 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f899c00d780 tx=0x7f899c00dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:13.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:13.122+0000 7f89a7e00640 1 -- 172.21.15.151:0/3717671614 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f899c017070 con 0x7f89b00cec60 2024-09-16T15:50:13.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:13.123+0000 7f89a7e00640 1 -- 172.21.15.151:0/3717671614 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f899c00de90 con 0x7f89b00cec60 2024-09-16T15:50:13.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:13.123+0000 7f89a7e00640 1 -- 172.21.15.151:0/3717671614 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f899c005020 con 0x7f89b00cec60 2024-09-16T15:50:13.125 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:13.123+0000 7f89b7200640 1 -- 172.21.15.151:0/3717671614 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89b015fe90 con 0x7f89b00cec60 2024-09-16T15:50:13.125 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:13.123+0000 7f89b7200640 1 -- 172.21.15.151:0/3717671614 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89b0160450 con 0x7f89b00cec60 2024-09-16T15:50:13.134 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:13.127+0000 7f89b7200640 1 -- 172.21.15.151:0/3717671614 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8988005290 con 0x7f89b00cec60 2024-09-16T15:50:13.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:13.143+0000 7f89a7e00640 1 -- 172.21.15.151:0/3717671614 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 36) ==== 103225+0+0 (secure 0 0 0) 0x7f899c0051c0 con 0x7f89b00cec60 2024-09-16T15:50:13.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:13.151+0000 7f89a7e00640 1 --2- 172.21.15.151:0/3717671614 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8990079e80 0x7f899007c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:13.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:13.152+0000 7f89a7e00640 1 -- 172.21.15.151:0/3717671614 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7f899c099860 con 0x7f89b00cec60 2024-09-16T15:50:13.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:13.155+0000 7f89a7e00640 1 -- 172.21.15.151:0/3717671614 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f899c0633a0 con 0x7f89b00cec60 2024-09-16T15:50:13.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:13.172+0000 7f89afe00640 1 --2- 172.21.15.151:0/3717671614 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8990079e80 0x7f899007c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:13.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:13.178+0000 7f89afe00640 1 --2- 172.21.15.151:0/3717671614 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8990079e80 0x7f899007c340 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f89a0033000 tx=0x7f89a00046c0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:13.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:13.516+0000 7f89b7200640 1 -- 172.21.15.151:0/3717671614 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone35339", "target": ["mon-mgr", ""]}) -- 0x7f8988000bc0 con 0x7f8990079e80 2024-09-16T15:50:14.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:14.363+0000 7f89a7e00640 1 -- 172.21.15.151:0/3717671614 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f8988000bc0 con 0x7f8990079e80 2024-09-16T15:50:14.365 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:50:14.361+0000 7f48efe00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-16T15:50:14.366 INFO:teuthology.orchestra.run.smithi151.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-16T15:50:14.393 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T15:50:14.394 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone872410 2024-09-16T15:50:14.685 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:14.678+0000 7fc9e0a00640 1 -- 172.21.15.151:0/446901051 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc9dc15bec0 msgr2=0x7fc9dc06fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:14.685 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:14.678+0000 7fc9e0a00640 1 --2- 172.21.15.151:0/446901051 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc9dc15bec0 0x7fc9dc06fce0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc005fd0 tx=0x7fc9cc02aa10 comp rx=0 tx=0).stop 2024-09-16T15:50:14.685 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:14.678+0000 7fc9e0a00640 1 -- 172.21.15.151:0/446901051 shutdown_connections 2024-09-16T15:50:14.685 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:14.678+0000 7fc9e0a00640 1 --2- 172.21.15.151:0/446901051 >> v2:172.21.15.167:3300/0 conn(0x7fc9dc0cbc00 0x7fc9dc073ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:14.685 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:14.678+0000 7fc9e0a00640 1 --2- 172.21.15.151:0/446901051 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc9dc15bec0 0x7fc9dc06fce0 secure :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc005fd0 tx=0x7fc9cc02aa10 comp rx=0 tx=0).stop 2024-09-16T15:50:14.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:14.678+0000 7fc9e0a00640 1 -- 172.21.15.151:0/446901051 >> 172.21.15.151:0/446901051 conn(0x7fc9dc0cf9f0 msgr2=0x7fc9dc0cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:14.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:14.682+0000 7fc9e0a00640 1 -- 172.21.15.151:0/446901051 shutdown_connections 2024-09-16T15:50:14.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:14.683+0000 7fc9e0a00640 1 -- 172.21.15.151:0/446901051 wait complete. 2024-09-16T15:50:14.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:14.687+0000 7fc9e0a00640 1 Processor -- start 2024-09-16T15:50:14.690 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:14.687+0000 7fc9e0a00640 1 -- start start 2024-09-16T15:50:14.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:14.689+0000 7fc9e0a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc9dc072560 0x7fc9dc144760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:14.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:14.689+0000 7fc9e0a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc9dc0cbc00 0x7fc9dc1fc720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:14.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:14.689+0000 7fc9e0a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc9dc1581b0 0x7fc9dc1fcc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:14.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:14.689+0000 7fc9e0a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc9dc0ae990 con 0x7fc9dc1581b0 2024-09-16T15:50:14.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:14.689+0000 7fc9e0a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc9dc0ae810 con 0x7fc9dc072560 2024-09-16T15:50:14.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:14.689+0000 7fc9e0a00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fc9dc0aeb10 con 0x7fc9dc0cbc00 2024-09-16T15:50:14.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:14.692+0000 7fc9db400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc9dc072560 0x7fc9dc144760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:14.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:14.692+0000 7fc9db400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc9dc072560 0x7fc9dc144760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.151:53504/0 (socket says 172.21.15.151:53504) 2024-09-16T15:50:14.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:14.692+0000 7fc9db400640 1 -- 172.21.15.151:0/110980808 learned_addr learned my addr 172.21.15.151:0/110980808 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:14.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:14.692+0000 7fc9d2a00640 1 --2- 172.21.15.151:0/110980808 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc9dc0cbc00 0x7fc9dc1fc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:14.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:14.693+0000 7fc9dbe00640 1 --2- 172.21.15.151:0/110980808 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc9dc1581b0 0x7fc9dc1fcc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:14.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:14.694+0000 7fc9dbe00640 1 -- 172.21.15.151:0/110980808 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc9dc0cbc00 msgr2=0x7fc9dc1fc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:14.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:14.694+0000 7fc9dbe00640 1 --2- 172.21.15.151:0/110980808 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc9dc0cbc00 0x7fc9dc1fc720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:14.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:14.694+0000 7fc9dbe00640 1 -- 172.21.15.151:0/110980808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc9dc072560 msgr2=0x7fc9dc144760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:14.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:14.694+0000 7fc9dbe00640 1 --2- 172.21.15.151:0/110980808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc9dc072560 0x7fc9dc144760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:14.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:14.694+0000 7fc9dbe00640 1 -- 172.21.15.151:0/110980808 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9dc144ca0 con 0x7fc9dc1581b0 2024-09-16T15:50:14.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:14.697+0000 7fc9dbe00640 1 --2- 172.21.15.151:0/110980808 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc9dc1581b0 0x7fc9dc1fcc60 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fc9d400b390 tx=0x7fc9d400b850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:14.703 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:14.702+0000 7fc9d3e00640 1 -- 172.21.15.151:0/110980808 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d4015020 con 0x7fc9dc1581b0 2024-09-16T15:50:14.703 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:14.702+0000 7fc9d3e00640 1 -- 172.21.15.151:0/110980808 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc9d4011a00 con 0x7fc9dc1581b0 2024-09-16T15:50:14.703 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:14.702+0000 7fc9d3e00640 1 -- 172.21.15.151:0/110980808 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d4011e50 con 0x7fc9dc1581b0 2024-09-16T15:50:14.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:14.703+0000 7fc9e0a00640 1 -- 172.21.15.151:0/110980808 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9dc15def0 con 0x7fc9dc1581b0 2024-09-16T15:50:14.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:14.703+0000 7fc9e0a00640 1 -- 172.21.15.151:0/110980808 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9dc15e4b0 con 0x7fc9dc1581b0 2024-09-16T15:50:14.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:14.705+0000 7fc9e0a00640 1 -- 172.21.15.151:0/110980808 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9dc0af000 con 0x7fc9dc1581b0 2024-09-16T15:50:14.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:14.709+0000 7fc9d3e00640 1 -- 172.21.15.151:0/110980808 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 36) ==== 103225+0+0 (secure 0 0 0) 0x7fc9d4018020 con 0x7fc9dc1581b0 2024-09-16T15:50:14.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:14.712+0000 7fc9d3e00640 1 --2- 172.21.15.151:0/110980808 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc9b807a020 0x7fc9b807c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:14.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:14.712+0000 7fc9d3e00640 1 -- 172.21.15.151:0/110980808 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7fc9d4099f80 con 0x7fc9dc1581b0 2024-09-16T15:50:14.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:14.718+0000 7fc9db400640 1 --2- 172.21.15.151:0/110980808 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc9b807a020 0x7fc9b807c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:14.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:14.719+0000 7fc9d3e00640 1 -- 172.21.15.151:0/110980808 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc9d401f020 con 0x7fc9dc1581b0 2024-09-16T15:50:14.739 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:14.738+0000 7fc9db400640 1 --2- 172.21.15.151:0/110980808 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc9b807a020 0x7fc9b807c4e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc000f80 tx=0x7fc9cc03a040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:15.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:15.136+0000 7fc9e0a00640 1 -- 172.21.15.151:0/110980808 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone872410", "target": ["mon-mgr", ""]}) -- 0x7fc9dc0c6840 con 0x7fc9b807a020 2024-09-16T15:50:15.290 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:50:15.289+0000 7f48efe00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-16T15:50:15.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:15.291+0000 7fc9d3e00640 1 -- 172.21.15.151:0/110980808 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7fc9dc0c6840 con 0x7fc9b807a020 2024-09-16T15:50:15.292 INFO:teuthology.orchestra.run.smithi151.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-16T15:50:15.311 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T15:50:15.312 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone556011 2024-09-16T15:50:15.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:15.580+0000 7fb442400640 1 -- 172.21.15.151:0/2718765302 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb43c161460 msgr2=0x7fb43c0d22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:15.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:15.580+0000 7fb442400640 1 --2- 172.21.15.151:0/2718765302 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb43c161460 0x7fb43c0d22f0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fb42c009830 tx=0x7fb42c02a480 comp rx=0 tx=0).stop 2024-09-16T15:50:15.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:15.581+0000 7fb442400640 1 -- 172.21.15.151:0/2718765302 shutdown_connections 2024-09-16T15:50:15.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:15.581+0000 7fb442400640 1 --2- 172.21.15.151:0/2718765302 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb43c161460 0x7fb43c0d22f0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:15.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:15.581+0000 7fb442400640 1 -- 172.21.15.151:0/2718765302 >> 172.21.15.151:0/2718765302 conn(0x7fb43c09d0b0 msgr2=0x7fb43c0d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:15.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:15.581+0000 7fb442400640 1 -- 172.21.15.151:0/2718765302 shutdown_connections 2024-09-16T15:50:15.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:15.581+0000 7fb442400640 1 -- 172.21.15.151:0/2718765302 wait complete. 2024-09-16T15:50:15.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:15.583+0000 7fb442400640 1 Processor -- start 2024-09-16T15:50:15.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:15.583+0000 7fb442400640 1 -- start start 2024-09-16T15:50:15.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:15.584+0000 7fb442400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb43c0b43c0 0x7fb43c1581c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:15.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:15.584+0000 7fb442400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb43c0b5480 0x7fb43c15a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:15.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:15.584+0000 7fb442400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb43c161460 0x7fb43c15cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:15.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:15.584+0000 7fb442400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb43c0d29b0 con 0x7fb43c0b5480 2024-09-16T15:50:15.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:15.584+0000 7fb442400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb43c0d2830 con 0x7fb43c161460 2024-09-16T15:50:15.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:15.584+0000 7fb442400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fb43c0d2b30 con 0x7fb43c0b43c0 2024-09-16T15:50:15.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:15.584+0000 7fb43be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb43c0b5480 0x7fb43c15a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:15.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:15.584+0000 7fb43be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb43c0b5480 0x7fb43c15a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41882/0 (socket says 172.21.15.151:41882) 2024-09-16T15:50:15.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:15.584+0000 7fb43be00640 1 -- 172.21.15.151:0/3605041521 learned_addr learned my addr 172.21.15.151:0/3605041521 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:15.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:15.585+0000 7fb43be00640 1 -- 172.21.15.151:0/3605041521 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb43c0b43c0 msgr2=0x7fb43c1581c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:15.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:15.585+0000 7fb43be00640 1 --2- 172.21.15.151:0/3605041521 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb43c0b43c0 0x7fb43c1581c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:15.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:15.585+0000 7fb43be00640 1 -- 172.21.15.151:0/3605041521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb43c161460 msgr2=0x7fb43c15cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:15.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:15.585+0000 7fb43be00640 1 --2- 172.21.15.151:0/3605041521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb43c161460 0x7fb43c15cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:15.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:15.585+0000 7fb43be00640 1 -- 172.21.15.151:0/3605041521 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb43c159b60 con 0x7fb43c0b5480 2024-09-16T15:50:15.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:15.585+0000 7fb43be00640 1 --2- 172.21.15.151:0/3605041521 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb43c0b5480 0x7fb43c15a260 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fb42800c400 tx=0x7fb42800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:15.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:15.586+0000 7fb438c00640 1 -- 172.21.15.151:0/3605041521 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb428017070 con 0x7fb43c0b5480 2024-09-16T15:50:15.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:15.586+0000 7fb442400640 1 -- 172.21.15.151:0/3605041521 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb43c158ac0 con 0x7fb43c0b5480 2024-09-16T15:50:15.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:15.586+0000 7fb438c00640 1 -- 172.21.15.151:0/3605041521 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb4280035f0 con 0x7fb43c0b5480 2024-09-16T15:50:15.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:15.586+0000 7fb438c00640 1 -- 172.21.15.151:0/3605041521 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb428003a40 con 0x7fb43c0b5480 2024-09-16T15:50:15.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:15.586+0000 7fb442400640 1 -- 172.21.15.151:0/3605041521 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb43c159080 con 0x7fb43c0b5480 2024-09-16T15:50:15.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:15.587+0000 7fb442400640 1 -- 172.21.15.151:0/3605041521 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb43c0cf870 con 0x7fb43c0b5480 2024-09-16T15:50:15.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:15.590+0000 7fb438c00640 1 -- 172.21.15.151:0/3605041521 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 36) ==== 103225+0+0 (secure 0 0 0) 0x7fb42801a070 con 0x7fb43c0b5480 2024-09-16T15:50:15.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:15.593+0000 7fb438c00640 1 --2- 172.21.15.151:0/3605041521 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb41c079de0 0x7fb41c07c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:15.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:15.593+0000 7fb438c00640 1 -- 172.21.15.151:0/3605041521 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7fb428099330 con 0x7fb43c0b5480 2024-09-16T15:50:15.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:15.596+0000 7fb438c00640 1 -- 172.21.15.151:0/3605041521 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb428021070 con 0x7fb43c0b5480 2024-09-16T15:50:15.610 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:15.609+0000 7fb441000640 1 --2- 172.21.15.151:0/3605041521 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb41c079de0 0x7fb41c07c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:15.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:15.724+0000 7fb441000640 1 --2- 172.21.15.151:0/3605041521 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb41c079de0 0x7fb41c07c2a0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb42c034000 tx=0x7fb42c0046c0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:16.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:16.018+0000 7fb442400640 1 -- 172.21.15.151:0/3605041521 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone556011", "target": ["mon-mgr", ""]}) -- 0x7fb43c074670 con 0x7fb41c079de0 2024-09-16T15:50:16.105 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:50:16.102+0000 7f48efe00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-16T15:50:16.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:16.113+0000 7fb438c00640 1 -- 172.21.15.151:0/3605041521 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7fb43c074670 con 0x7fb41c079de0 2024-09-16T15:50:16.114 INFO:teuthology.orchestra.run.smithi151.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-16T15:50:16.125 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T15:50:16.126 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone515512 2024-09-16T15:50:16.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:16.389+0000 7fe1f3000640 1 -- 172.21.15.151:0/3613283062 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1ec145750 msgr2=0x7fe1ec073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:16.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:16.389+0000 7fe1f3000640 1 --2- 172.21.15.151:0/3613283062 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1ec145750 0x7fe1ec073440 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec0d0cd0 tx=0x7fe1dc02a400 comp rx=0 tx=0).stop 2024-09-16T15:50:16.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:16.390+0000 7fe1f3000640 1 -- 172.21.15.151:0/3613283062 shutdown_connections 2024-09-16T15:50:16.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:16.390+0000 7fe1f3000640 1 --2- 172.21.15.151:0/3613283062 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1ec0ccb00 0x7fe1ec15f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:16.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:16.390+0000 7fe1f3000640 1 --2- 172.21.15.151:0/3613283062 >> v2:172.21.15.151:3300/0 conn(0x7fe1ec06efa0 0x7fe1ec0cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:16.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:16.390+0000 7fe1f3000640 1 --2- 172.21.15.151:0/3613283062 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1ec145750 0x7fe1ec073440 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:16.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:16.390+0000 7fe1f3000640 1 -- 172.21.15.151:0/3613283062 >> 172.21.15.151:0/3613283062 conn(0x7fe1ec0b1840 msgr2=0x7fe1ec0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:16.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:16.392+0000 7fe1f3000640 1 -- 172.21.15.151:0/3613283062 shutdown_connections 2024-09-16T15:50:16.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:16.393+0000 7fe1f3000640 1 -- 172.21.15.151:0/3613283062 wait complete. 2024-09-16T15:50:16.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:16.398+0000 7fe1f3000640 1 Processor -- start 2024-09-16T15:50:16.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:16.402+0000 7fe1f3000640 1 -- start start 2024-09-16T15:50:16.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:16.405+0000 7fe1f3000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1ec06efa0 0x7fe1ec0aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:16.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:16.405+0000 7fe1f3000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1ec0ccb00 0x7fe1ec1f9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:16.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:16.405+0000 7fe1f3000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe1ec145750 0x7fe1ec1f9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:16.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:16.405+0000 7fe1f3000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ec1602c0 con 0x7fe1ec145750 2024-09-16T15:50:16.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:16.405+0000 7fe1f3000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ec160140 con 0x7fe1ec06efa0 2024-09-16T15:50:16.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:16.405+0000 7fe1f3000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fe1ec160440 con 0x7fe1ec0ccb00 2024-09-16T15:50:16.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:16.406+0000 7fe1f1000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe1ec145750 0x7fe1ec1f9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:16.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:16.407+0000 7fe1e3400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1ec0ccb00 0x7fe1ec1f9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:16.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:16.407+0000 7fe1e3400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1ec0ccb00 0x7fe1ec1f9560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.151:45428/0 (socket says 172.21.15.151:45428) 2024-09-16T15:50:16.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:16.407+0000 7fe1e3400640 1 -- 172.21.15.151:0/4198607495 learned_addr learned my addr 172.21.15.151:0/4198607495 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:16.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:16.408+0000 7fe1ebe00640 1 --2- 172.21.15.151:0/4198607495 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1ec06efa0 0x7fe1ec0aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:16.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:16.411+0000 7fe1e3400640 1 -- 172.21.15.151:0/4198607495 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1ec06efa0 msgr2=0x7fe1ec0aeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:16.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:16.411+0000 7fe1e3400640 1 --2- 172.21.15.151:0/4198607495 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1ec06efa0 0x7fe1ec0aeb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:16.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:16.411+0000 7fe1e3400640 1 -- 172.21.15.151:0/4198607495 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe1ec145750 msgr2=0x7fe1ec1f9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:16.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:16.411+0000 7fe1e3400640 1 --2- 172.21.15.151:0/4198607495 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe1ec145750 0x7fe1ec1f9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:16.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:16.411+0000 7fe1e3400640 1 -- 172.21.15.151:0/4198607495 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1ec0af3a0 con 0x7fe1ec0ccb00 2024-09-16T15:50:16.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:16.415+0000 7fe1e3400640 1 --2- 172.21.15.151:0/4198607495 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1ec0ccb00 0x7fe1ec1f9560 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe1d400d7a0 tx=0x7fe1d400dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:16.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:16.419+0000 7fe1e8c00640 1 -- 172.21.15.151:0/4198607495 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d4017070 con 0x7fe1ec0ccb00 2024-09-16T15:50:16.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:16.419+0000 7fe1e8c00640 1 -- 172.21.15.151:0/4198607495 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe1d4009d20 con 0x7fe1ec0ccb00 2024-09-16T15:50:16.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:16.419+0000 7fe1f3000640 1 -- 172.21.15.151:0/4198607495 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1ec13eb30 con 0x7fe1ec0ccb00 2024-09-16T15:50:16.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:16.419+0000 7fe1f3000640 1 -- 172.21.15.151:0/4198607495 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1ec13f0f0 con 0x7fe1ec0ccb00 2024-09-16T15:50:16.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:16.420+0000 7fe1e8c00640 1 -- 172.21.15.151:0/4198607495 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d40106c0 con 0x7fe1ec0ccb00 2024-09-16T15:50:16.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:16.424+0000 7fe1f3000640 1 -- 172.21.15.151:0/4198607495 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1c4005290 con 0x7fe1ec0ccb00 2024-09-16T15:50:16.434 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:16.433+0000 7fe1e8c00640 1 -- 172.21.15.151:0/4198607495 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 36) ==== 103225+0+0 (secure 0 0 0) 0x7fe1d401a070 con 0x7fe1ec0ccb00 2024-09-16T15:50:16.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:16.435+0000 7fe1e8c00640 1 --2- 172.21.15.151:0/4198607495 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe1c8079e00 0x7fe1c807c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:16.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:16.435+0000 7fe1e8c00640 1 -- 172.21.15.151:0/4198607495 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7fe1d4099dd0 con 0x7fe1ec0ccb00 2024-09-16T15:50:16.438 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:16.437+0000 7fe1ebe00640 1 --2- 172.21.15.151:0/4198607495 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe1c8079e00 0x7fe1c807c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:16.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:16.445+0000 7fe1e8c00640 1 -- 172.21.15.151:0/4198607495 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe1d4063ad0 con 0x7fe1ec0ccb00 2024-09-16T15:50:16.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:16.453+0000 7fe1ebe00640 1 --2- 172.21.15.151:0/4198607495 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe1c8079e00 0x7fe1c807c2c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc034000 tx=0x7fe1dc004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:16.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:16.840+0000 7fe1f3000640 1 -- 172.21.15.151:0/4198607495 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone515512", "target": ["mon-mgr", ""]}) -- 0x7fe1c4000bc0 con 0x7fe1c8079e00 2024-09-16T15:50:18.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:18.207+0000 7fe1e8c00640 1 -- 172.21.15.151:0/4198607495 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe1c4000bc0 con 0x7fe1c8079e00 2024-09-16T15:50:18.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:18.210+0000 7fe1f3000640 1 -- 172.21.15.151:0/4198607495 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe1c8079e00 msgr2=0x7fe1c807c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:18.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:18.210+0000 7fe1f3000640 1 --2- 172.21.15.151:0/4198607495 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe1c8079e00 0x7fe1c807c2c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc034000 tx=0x7fe1dc004730 comp rx=0 tx=0).stop 2024-09-16T15:50:18.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:18.210+0000 7fe1f3000640 1 -- 172.21.15.151:0/4198607495 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1ec0ccb00 msgr2=0x7fe1ec1f9560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:18.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:18.210+0000 7fe1f3000640 1 --2- 172.21.15.151:0/4198607495 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1ec0ccb00 0x7fe1ec1f9560 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe1d400d7a0 tx=0x7fe1d400dc60 comp rx=0 tx=0).stop 2024-09-16T15:50:18.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:18.210+0000 7fe1f3000640 1 -- 172.21.15.151:0/4198607495 shutdown_connections 2024-09-16T15:50:18.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:18.210+0000 7fe1f3000640 1 --2- 172.21.15.151:0/4198607495 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe1c8079e00 0x7fe1c807c2c0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:18.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:18.210+0000 7fe1f3000640 1 --2- 172.21.15.151:0/4198607495 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe1ec145750 0x7fe1ec1f9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:18.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:18.210+0000 7fe1f3000640 1 --2- 172.21.15.151:0/4198607495 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe1ec0ccb00 0x7fe1ec1f9560 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:18.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:18.210+0000 7fe1f3000640 1 --2- 172.21.15.151:0/4198607495 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe1ec06efa0 0x7fe1ec0aeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:18.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:18.210+0000 7fe1f3000640 1 -- 172.21.15.151:0/4198607495 >> 172.21.15.151:0/4198607495 conn(0x7fe1ec0b1840 msgr2=0x7fe1ec138740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:18.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:18.210+0000 7fe1f3000640 1 -- 172.21.15.151:0/4198607495 shutdown_connections 2024-09-16T15:50:18.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:18.211+0000 7fe1f3000640 1 -- 172.21.15.151:0/4198607495 wait complete. 2024-09-16T15:50:18.250 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone277913 2024-09-16T15:50:18.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:18.593+0000 7fea27e00640 1 -- 172.21.15.151:0/1131503197 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea28071a20 msgr2=0x7fea2806d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:18.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:18.593+0000 7fea27e00640 1 --2- 172.21.15.151:0/1131503197 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea28071a20 0x7fea2806d0b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fea20005fd0 tx=0x7fea2002aa90 comp rx=0 tx=0).stop 2024-09-16T15:50:18.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:18.593+0000 7fea27e00640 1 -- 172.21.15.151:0/1131503197 shutdown_connections 2024-09-16T15:50:18.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:18.593+0000 7fea27e00640 1 --2- 172.21.15.151:0/1131503197 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea28071a20 0x7fea2806d0b0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:18.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:18.593+0000 7fea27e00640 1 --2- 172.21.15.151:0/1131503197 >> v2:172.21.15.167:3300/0 conn(0x7fea2815cae0 0x7fea2806cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:18.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:18.593+0000 7fea27e00640 1 -- 172.21.15.151:0/1131503197 >> 172.21.15.151:0/1131503197 conn(0x7fea280d0b20 msgr2=0x7fea280d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:18.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:18.594+0000 7fea27e00640 1 -- 172.21.15.151:0/1131503197 shutdown_connections 2024-09-16T15:50:18.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:18.594+0000 7fea27e00640 1 -- 172.21.15.151:0/1131503197 wait complete. 2024-09-16T15:50:18.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:18.597+0000 7fea27e00640 1 Processor -- start 2024-09-16T15:50:18.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:18.597+0000 7fea27e00640 1 -- start start 2024-09-16T15:50:18.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:18.599+0000 7fea27e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea28071a20 0x7fea280c6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:18.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:18.600+0000 7fea27e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea280c41b0 0x7fea280c72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:18.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:18.600+0000 7fea27e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea2815cae0 0x7fea280c8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:18.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:18.600+0000 7fea27e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fea2815c2f0 con 0x7fea28071a20 2024-09-16T15:50:18.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:18.600+0000 7fea27e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fea2815fde0 con 0x7fea2815cae0 2024-09-16T15:50:18.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:18.600+0000 7fea27e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fea280c7bf0 con 0x7fea280c41b0 2024-09-16T15:50:18.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:18.600+0000 7fea26a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea28071a20 0x7fea280c6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:18.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:18.601+0000 7fea26a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea28071a20 0x7fea280c6d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41916/0 (socket says 172.21.15.151:41916) 2024-09-16T15:50:18.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:18.601+0000 7fea26a00640 1 -- 172.21.15.151:0/2004562729 learned_addr learned my addr 172.21.15.151:0/2004562729 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:18.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:18.601+0000 7fea26000640 1 --2- 172.21.15.151:0/2004562729 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea280c41b0 0x7fea280c72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:18.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:18.601+0000 7fea27400640 1 --2- 172.21.15.151:0/2004562729 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea2815cae0 0x7fea280c8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:18.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:18.601+0000 7fea26a00640 1 -- 172.21.15.151:0/2004562729 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea280c41b0 msgr2=0x7fea280c72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:18.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:18.601+0000 7fea26a00640 1 --2- 172.21.15.151:0/2004562729 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea280c41b0 0x7fea280c72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:18.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:18.601+0000 7fea26a00640 1 -- 172.21.15.151:0/2004562729 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea2815cae0 msgr2=0x7fea280c8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:18.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:18.601+0000 7fea26a00640 1 --2- 172.21.15.151:0/2004562729 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea2815cae0 0x7fea280c8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:18.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:18.601+0000 7fea26a00640 1 -- 172.21.15.151:0/2004562729 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea280b21a0 con 0x7fea28071a20 2024-09-16T15:50:18.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:18.603+0000 7fea26a00640 1 --2- 172.21.15.151:0/2004562729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea28071a20 0x7fea280c6d70 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fea1800b4a0 tx=0x7fea1800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:18.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:18.603+0000 7fea1f400640 1 -- 172.21.15.151:0/2004562729 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea18014070 con 0x7fea28071a20 2024-09-16T15:50:18.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:18.604+0000 7fea1f400640 1 -- 172.21.15.151:0/2004562729 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fea18010700 con 0x7fea28071a20 2024-09-16T15:50:18.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:18.604+0000 7fea1f400640 1 -- 172.21.15.151:0/2004562729 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea18010b50 con 0x7fea28071a20 2024-09-16T15:50:18.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:18.604+0000 7fea27e00640 1 -- 172.21.15.151:0/2004562729 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea280c7e40 con 0x7fea28071a20 2024-09-16T15:50:18.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:18.604+0000 7fea27e00640 1 -- 172.21.15.151:0/2004562729 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea280b6540 con 0x7fea28071a20 2024-09-16T15:50:18.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:18.607+0000 7fea27e00640 1 -- 172.21.15.151:0/2004562729 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea28160280 con 0x7fea28071a20 2024-09-16T15:50:18.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:18.615+0000 7fea1f400640 1 -- 172.21.15.151:0/2004562729 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 36) ==== 103225+0+0 (secure 0 0 0) 0x7fea18017070 con 0x7fea28071a20 2024-09-16T15:50:18.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:18.616+0000 7fea1f400640 1 --2- 172.21.15.151:0/2004562729 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fea08079e80 0x7fea0807c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:18.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:18.617+0000 7fea1f400640 1 -- 172.21.15.151:0/2004562729 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7fea18099870 con 0x7fea28071a20 2024-09-16T15:50:18.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:18.618+0000 7fea1f400640 1 -- 172.21.15.151:0/2004562729 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fea18063530 con 0x7fea28071a20 2024-09-16T15:50:18.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:18.634+0000 7fea26000640 1 --2- 172.21.15.151:0/2004562729 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fea08079e80 0x7fea0807c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:18.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:18.641+0000 7fea26000640 1 --2- 172.21.15.151:0/2004562729 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fea08079e80 0x7fea0807c340 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fea14007fd0 tx=0x7fea14009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:19.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:19.030+0000 7fea27e00640 1 -- 172.21.15.151:0/2004562729 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone277913", "target": ["mon-mgr", ""]}) -- 0x7fea280d0900 con 0x7fea08079e80 2024-09-16T15:50:19.586 DEBUG:teuthology.orchestra.run.smithi151:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:50:19.591 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:50:20.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:20.163+0000 7fea1f400640 1 -- 172.21.15.151:0/2004562729 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fea280d0900 con 0x7fea08079e80 2024-09-16T15:50:20.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:20.167+0000 7fea27e00640 1 -- 172.21.15.151:0/2004562729 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fea08079e80 msgr2=0x7fea0807c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:20.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:20.167+0000 7fea27e00640 1 --2- 172.21.15.151:0/2004562729 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fea08079e80 0x7fea0807c340 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fea14007fd0 tx=0x7fea14009040 comp rx=0 tx=0).stop 2024-09-16T15:50:20.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:20.167+0000 7fea27e00640 1 -- 172.21.15.151:0/2004562729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea28071a20 msgr2=0x7fea280c6d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:20.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:20.167+0000 7fea27e00640 1 --2- 172.21.15.151:0/2004562729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea28071a20 0x7fea280c6d70 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fea1800b4a0 tx=0x7fea1800b960 comp rx=0 tx=0).stop 2024-09-16T15:50:20.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:20.167+0000 7fea27e00640 1 -- 172.21.15.151:0/2004562729 shutdown_connections 2024-09-16T15:50:20.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:20.167+0000 7fea27e00640 1 --2- 172.21.15.151:0/2004562729 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fea08079e80 0x7fea0807c340 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:20.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:20.167+0000 7fea27e00640 1 --2- 172.21.15.151:0/2004562729 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea2815cae0 0x7fea280c8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:20.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:20.167+0000 7fea27e00640 1 --2- 172.21.15.151:0/2004562729 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fea280c41b0 0x7fea280c72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:20.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:20.167+0000 7fea27e00640 1 --2- 172.21.15.151:0/2004562729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea28071a20 0x7fea280c6d70 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:20.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:20.167+0000 7fea27e00640 1 -- 172.21.15.151:0/2004562729 >> 172.21.15.151:0/2004562729 conn(0x7fea280d0b20 msgr2=0x7fea280d0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:20.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:20.167+0000 7fea27e00640 1 -- 172.21.15.151:0/2004562729 shutdown_connections 2024-09-16T15:50:20.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:20.167+0000 7fea27e00640 1 -- 172.21.15.151:0/2004562729 wait complete. 2024-09-16T15:50:20.205 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone880714 2024-09-16T15:50:20.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:20.568+0000 7f4d84600640 1 -- 172.21.15.151:0/3663313756 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d7404f2c0 msgr2=0x7f4d74039480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:20.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:20.569+0000 7f4d84600640 1 --2- 172.21.15.151:0/3663313756 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d7404f2c0 0x7f4d74039480 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c06b110 tx=0x7f4d7c0a5cf0 comp rx=0 tx=0).stop 2024-09-16T15:50:20.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:20.569+0000 7f4d84600640 1 -- 172.21.15.151:0/3663313756 shutdown_connections 2024-09-16T15:50:20.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:20.569+0000 7f4d84600640 1 --2- 172.21.15.151:0/3663313756 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d7404f2c0 0x7f4d74039480 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:20.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:20.569+0000 7f4d84600640 1 --2- 172.21.15.151:0/3663313756 >> v2:172.21.15.167:3300/0 conn(0x7f4d740071b0 0x7f4d7404ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:20.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:20.570+0000 7f4d84600640 1 --2- 172.21.15.151:0/3663313756 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4d740e22f0 0x7f4d74053e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:20.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:20.570+0000 7f4d84600640 1 -- 172.21.15.151:0/3663313756 >> 172.21.15.151:0/3663313756 conn(0x7f4d7401d940 msgr2=0x7f4d74054730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:20.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:20.570+0000 7f4d84600640 1 -- 172.21.15.151:0/3663313756 shutdown_connections 2024-09-16T15:50:20.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:20.571+0000 7f4d84600640 1 -- 172.21.15.151:0/3663313756 wait complete. 2024-09-16T15:50:20.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:20.575+0000 7f4d84600640 1 Processor -- start 2024-09-16T15:50:20.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:20.575+0000 7f4d84600640 1 -- start start 2024-09-16T15:50:20.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:20.578+0000 7f4d84600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d740071b0 0x7f4d74050af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:20.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:20.578+0000 7f4d84600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d7404f2c0 0x7f4d74051030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:20.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:20.578+0000 7f4d84600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4d740e22f0 0x7f4d7404d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:20.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:20.578+0000 7f4d84600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4d74054510 con 0x7f4d7404f2c0 2024-09-16T15:50:20.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:20.578+0000 7f4d84600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4d74054390 con 0x7f4d740071b0 2024-09-16T15:50:20.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:20.578+0000 7f4d84600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f4d740399c0 con 0x7f4d740e22f0 2024-09-16T15:50:20.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:20.581+0000 7f4d81200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d7404f2c0 0x7f4d74051030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:20.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:20.582+0000 7f4d81200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d7404f2c0 0x7f4d74051030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41938/0 (socket says 172.21.15.151:41938) 2024-09-16T15:50:20.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:20.582+0000 7f4d81200640 1 -- 172.21.15.151:0/3944097408 learned_addr learned my addr 172.21.15.151:0/3944097408 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:20.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:20.585+0000 7f4d81200640 1 -- 172.21.15.151:0/3944097408 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4d740e22f0 msgr2=0x7f4d7404d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:20.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:20.585+0000 7f4d81200640 1 --2- 172.21.15.151:0/3944097408 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4d740e22f0 0x7f4d7404d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:20.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:20.585+0000 7f4d81200640 1 -- 172.21.15.151:0/3944097408 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d740071b0 msgr2=0x7f4d74050af0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T15:50:20.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:20.585+0000 7f4d81200640 1 --2- 172.21.15.151:0/3944097408 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d740071b0 0x7f4d74050af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:20.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:20.585+0000 7f4d81200640 1 -- 172.21.15.151:0/3944097408 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d7404e6e0 con 0x7f4d7404f2c0 2024-09-16T15:50:20.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:20.589+0000 7f4d81200640 1 --2- 172.21.15.151:0/3944097408 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d7404f2c0 0x7f4d74051030 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f4d68010ef0 tx=0x7f4d680094e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:20.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:20.594+0000 7f4d79600640 1 -- 172.21.15.151:0/3944097408 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d6801e070 con 0x7f4d7404f2c0 2024-09-16T15:50:20.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:20.594+0000 7f4d84600640 1 -- 172.21.15.151:0/3944097408 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d7404a580 con 0x7f4d7404f2c0 2024-09-16T15:50:20.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:20.596+0000 7f4d84600640 1 -- 172.21.15.151:0/3944097408 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d7404ab40 con 0x7f4d7404f2c0 2024-09-16T15:50:20.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:20.595+0000 7f4d79600640 1 -- 172.21.15.151:0/3944097408 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4d68009d10 con 0x7f4d7404f2c0 2024-09-16T15:50:20.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:20.601+0000 7f4d79600640 1 -- 172.21.15.151:0/3944097408 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d6801e070 con 0x7f4d7404f2c0 2024-09-16T15:50:20.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:20.605+0000 7f4d84600640 1 -- 172.21.15.151:0/3944097408 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d74053e50 con 0x7f4d7404f2c0 2024-09-16T15:50:20.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:20.624+0000 7f4d79600640 1 -- 172.21.15.151:0/3944097408 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 36) ==== 103225+0+0 (secure 0 0 0) 0x7f4d68021070 con 0x7f4d7404f2c0 2024-09-16T15:50:20.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:20.631+0000 7f4d79600640 1 --2- 172.21.15.151:0/3944097408 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4d5c079fb0 0x7f4d5c07c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:20.634 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:20.633+0000 7f4d79600640 1 -- 172.21.15.151:0/3944097408 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7f4d680a17c0 con 0x7f4d7404f2c0 2024-09-16T15:50:20.634 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:20.633+0000 7f4d79600640 1 -- 172.21.15.151:0/3944097408 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4d68010070 con 0x7f4d7404f2c0 2024-09-16T15:50:20.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:20.657+0000 7f4d81c00640 1 --2- 172.21.15.151:0/3944097408 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4d5c079fb0 0x7f4d5c07c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:20.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:20.667+0000 7f4d81c00640 1 --2- 172.21.15.151:0/3944097408 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4d5c079fb0 0x7f4d5c07c470 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f4d700095f0 tx=0x7f4d70006b70 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:21.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:21.140+0000 7f4d84600640 1 -- 172.21.15.151:0/3944097408 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone880714", "target": ["mon-mgr", ""]}) -- 0x7f4d74001010 con 0x7f4d5c079fb0 2024-09-16T15:50:22.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:22.876+0000 7f4d79600640 1 -- 172.21.15.151:0/3944097408 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4d74001010 con 0x7f4d5c079fb0 2024-09-16T15:50:22.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:22.880+0000 7f4d84600640 1 -- 172.21.15.151:0/3944097408 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4d5c079fb0 msgr2=0x7f4d5c07c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:22.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:22.880+0000 7f4d84600640 1 --2- 172.21.15.151:0/3944097408 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4d5c079fb0 0x7f4d5c07c470 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f4d700095f0 tx=0x7f4d70006b70 comp rx=0 tx=0).stop 2024-09-16T15:50:22.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:22.880+0000 7f4d84600640 1 -- 172.21.15.151:0/3944097408 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d7404f2c0 msgr2=0x7f4d74051030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:22.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:22.880+0000 7f4d84600640 1 --2- 172.21.15.151:0/3944097408 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d7404f2c0 0x7f4d74051030 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f4d68010ef0 tx=0x7f4d680094e0 comp rx=0 tx=0).stop 2024-09-16T15:50:22.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:22.881+0000 7f4d84600640 1 -- 172.21.15.151:0/3944097408 shutdown_connections 2024-09-16T15:50:22.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:22.881+0000 7f4d84600640 1 --2- 172.21.15.151:0/3944097408 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4d5c079fb0 0x7f4d5c07c470 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:22.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:22.881+0000 7f4d84600640 1 --2- 172.21.15.151:0/3944097408 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4d740e22f0 0x7f4d7404d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:22.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:22.881+0000 7f4d84600640 1 --2- 172.21.15.151:0/3944097408 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d7404f2c0 0x7f4d74051030 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:22.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:22.881+0000 7f4d84600640 1 --2- 172.21.15.151:0/3944097408 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d740071b0 0x7f4d74050af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:22.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:22.881+0000 7f4d84600640 1 -- 172.21.15.151:0/3944097408 >> 172.21.15.151:0/3944097408 conn(0x7f4d7401d940 msgr2=0x7f4d740dda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:22.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:22.881+0000 7f4d84600640 1 -- 172.21.15.151:0/3944097408 shutdown_connections 2024-09-16T15:50:22.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:22.881+0000 7f4d84600640 1 -- 172.21.15.151:0/3944097408 wait complete. 2024-09-16T15:50:22.917 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone139815 2024-09-16T15:50:23.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:23.213+0000 7f8bec800640 1 -- 172.21.15.151:0/2915172761 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8be40b4400 msgr2=0x7f8be415db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:23.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:23.213+0000 7f8bec800640 1 --2- 172.21.15.151:0/2915172761 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8be40b4400 0x7f8be415db50 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f8be40d0cd0 tx=0x7f8be002b560 comp rx=0 tx=0).stop 2024-09-16T15:50:23.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:23.214+0000 7f8bec800640 1 -- 172.21.15.151:0/2915172761 shutdown_connections 2024-09-16T15:50:23.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:23.214+0000 7f8bec800640 1 --2- 172.21.15.151:0/2915172761 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8be40b4400 0x7f8be415db50 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:23.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:23.214+0000 7f8bec800640 1 --2- 172.21.15.151:0/2915172761 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8be40b4bd0 0x7f8be40b3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:23.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:23.214+0000 7f8bec800640 1 --2- 172.21.15.151:0/2915172761 >> v2:172.21.15.167:3300/0 conn(0x7f8be41437c0 0x7f8be40cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:23.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:23.214+0000 7f8bec800640 1 -- 172.21.15.151:0/2915172761 >> 172.21.15.151:0/2915172761 conn(0x7f8be40b1840 msgr2=0x7f8be40b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:23.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:23.214+0000 7f8bec800640 1 -- 172.21.15.151:0/2915172761 shutdown_connections 2024-09-16T15:50:23.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:23.214+0000 7f8bec800640 1 -- 172.21.15.151:0/2915172761 wait complete. 2024-09-16T15:50:23.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:23.217+0000 7f8bec800640 1 Processor -- start 2024-09-16T15:50:23.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:23.218+0000 7f8bec800640 1 -- start start 2024-09-16T15:50:23.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:23.219+0000 7f8bec800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8be40b4400 0x7f8be41f9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:23.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:23.219+0000 7f8bec800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8be40b4bd0 0x7f8be4159880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:23.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:23.219+0000 7f8bec800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8be41437c0 0x7f8be41622e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:23.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:23.219+0000 7f8bec800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8be415e090 con 0x7f8be40b4400 2024-09-16T15:50:23.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:23.219+0000 7f8bec800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8be4070a80 con 0x7f8be41437c0 2024-09-16T15:50:23.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:23.219+0000 7f8bec800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f8be415e210 con 0x7f8be40b4bd0 2024-09-16T15:50:23.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:23.219+0000 7f8be9e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8be40b4400 0x7f8be41f9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:23.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:23.219+0000 7f8be9e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8be40b4400 0x7f8be41f9870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:35258/0 (socket says 172.21.15.151:35258) 2024-09-16T15:50:23.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:23.219+0000 7f8be9e00640 1 -- 172.21.15.151:0/134805339 learned_addr learned my addr 172.21.15.151:0/134805339 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:23.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:23.220+0000 7f8be9400640 1 --2- 172.21.15.151:0/134805339 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8be40b4bd0 0x7f8be4159880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:23.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:23.220+0000 7f8bea800640 1 --2- 172.21.15.151:0/134805339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8be41437c0 0x7f8be41622e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:23.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:23.221+0000 7f8be9e00640 1 -- 172.21.15.151:0/134805339 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8be40b4bd0 msgr2=0x7f8be4159880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:23.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:23.221+0000 7f8be9e00640 1 --2- 172.21.15.151:0/134805339 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8be40b4bd0 0x7f8be4159880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:23.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:23.221+0000 7f8be9e00640 1 -- 172.21.15.151:0/134805339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8be41437c0 msgr2=0x7f8be41622e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:23.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:23.221+0000 7f8be9e00640 1 --2- 172.21.15.151:0/134805339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8be41437c0 0x7f8be41622e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:23.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:23.221+0000 7f8be9e00640 1 -- 172.21.15.151:0/134805339 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8be4156df0 con 0x7f8be40b4400 2024-09-16T15:50:23.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:23.221+0000 7f8be9e00640 1 --2- 172.21.15.151:0/134805339 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8be40b4400 0x7f8be41f9870 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f8bd800c9a0 tx=0x7f8bd800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:23.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:23.222+0000 7f8bde000640 1 -- 172.21.15.151:0/134805339 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd8007bb0 con 0x7f8be40b4400 2024-09-16T15:50:23.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:23.222+0000 7f8bde000640 1 -- 172.21.15.151:0/134805339 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8bd8007e40 con 0x7f8be40b4400 2024-09-16T15:50:23.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:23.222+0000 7f8bde000640 1 -- 172.21.15.151:0/134805339 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd800f690 con 0x7f8be40b4400 2024-09-16T15:50:23.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:23.222+0000 7f8bec800640 1 -- 172.21.15.151:0/134805339 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8be41f9db0 con 0x7f8be40b4400 2024-09-16T15:50:23.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:23.222+0000 7f8bec800640 1 -- 172.21.15.151:0/134805339 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8be415a5b0 con 0x7f8be40b4400 2024-09-16T15:50:23.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:23.224+0000 7f8bec800640 1 -- 172.21.15.151:0/134805339 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8be415e660 con 0x7f8be40b4400 2024-09-16T15:50:23.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:23.226+0000 7f8bde000640 1 -- 172.21.15.151:0/134805339 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 36) ==== 103225+0+0 (secure 0 0 0) 0x7f8bd8018400 con 0x7f8be40b4400 2024-09-16T15:50:23.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:23.228+0000 7f8bde000640 1 --2- 172.21.15.151:0/134805339 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8bc8079e00 0x7f8bc807c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:23.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:23.229+0000 7f8bde000640 1 -- 172.21.15.151:0/134805339 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7f8bd809a260 con 0x7f8be40b4400 2024-09-16T15:50:23.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:23.237+0000 7f8bde000640 1 -- 172.21.15.151:0/134805339 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8bd8014030 con 0x7f8be40b4400 2024-09-16T15:50:23.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:23.275+0000 7f8be9400640 1 --2- 172.21.15.151:0/134805339 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8bc8079e00 0x7f8bc807c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:23.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:23.353+0000 7f8be9400640 1 --2- 172.21.15.151:0/134805339 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8bc8079e00 0x7f8bc807c2c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f8be41426d0 tx=0x7f8bd4005e50 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:23.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:23.701+0000 7f8bec800640 1 -- 172.21.15.151:0/134805339 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone139815", "target": ["mon-mgr", ""]}) -- 0x7f8be415aeb0 con 0x7f8bc8079e00 2024-09-16T15:50:25.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:25.150+0000 7f8bde000640 1 -- 172.21.15.151:0/134805339 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8be415aeb0 con 0x7f8bc8079e00 2024-09-16T15:50:25.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:25.154+0000 7f8bec800640 1 -- 172.21.15.151:0/134805339 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8bc8079e00 msgr2=0x7f8bc807c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:25.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:25.154+0000 7f8bec800640 1 --2- 172.21.15.151:0/134805339 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8bc8079e00 0x7f8bc807c2c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f8be41426d0 tx=0x7f8bd4005e50 comp rx=0 tx=0).stop 2024-09-16T15:50:25.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:25.154+0000 7f8bec800640 1 -- 172.21.15.151:0/134805339 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8be40b4400 msgr2=0x7f8be41f9870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:25.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:25.154+0000 7f8bec800640 1 --2- 172.21.15.151:0/134805339 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8be40b4400 0x7f8be41f9870 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f8bd800c9a0 tx=0x7f8bd800ce60 comp rx=0 tx=0).stop 2024-09-16T15:50:25.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:25.155+0000 7f8bec800640 1 -- 172.21.15.151:0/134805339 shutdown_connections 2024-09-16T15:50:25.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:25.155+0000 7f8bec800640 1 --2- 172.21.15.151:0/134805339 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8bc8079e00 0x7f8bc807c2c0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:25.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:25.155+0000 7f8bec800640 1 --2- 172.21.15.151:0/134805339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8be41437c0 0x7f8be41622e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:25.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:25.155+0000 7f8bec800640 1 --2- 172.21.15.151:0/134805339 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8be40b4bd0 0x7f8be4159880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:25.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:25.155+0000 7f8bec800640 1 --2- 172.21.15.151:0/134805339 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8be40b4400 0x7f8be41f9870 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:25.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:25.155+0000 7f8bec800640 1 -- 172.21.15.151:0/134805339 >> 172.21.15.151:0/134805339 conn(0x7f8be40b1840 msgr2=0x7f8be415b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:25.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:25.155+0000 7f8bec800640 1 -- 172.21.15.151:0/134805339 shutdown_connections 2024-09-16T15:50:25.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:25.155+0000 7f8bec800640 1 -- 172.21.15.151:0/134805339 wait complete. 2024-09-16T15:50:25.190 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone446916 2024-09-16T15:50:25.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:25.512+0000 7f2deb200640 1 -- 172.21.15.151:0/4163998877 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2dd0004800 msgr2=0x7f2dd0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:25.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:25.512+0000 7f2deb200640 1 --2- 172.21.15.151:0/4163998877 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2dd0004800 0x7f2dd0004c80 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f2dd800c310 tx=0x7f2dd0005830 comp rx=0 tx=0).stop 2024-09-16T15:50:25.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:25.512+0000 7f2deb200640 1 -- 172.21.15.151:0/4163998877 shutdown_connections 2024-09-16T15:50:25.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:25.512+0000 7f2deb200640 1 --2- 172.21.15.151:0/4163998877 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2dd0004800 0x7f2dd0004c80 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:25.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:25.512+0000 7f2deb200640 1 -- 172.21.15.151:0/4163998877 >> 172.21.15.151:0/4163998877 conn(0x7f2dec09d0b0 msgr2=0x7f2dec0d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:25.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:25.512+0000 7f2deb200640 1 -- 172.21.15.151:0/4163998877 shutdown_connections 2024-09-16T15:50:25.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:25.513+0000 7f2deb200640 1 -- 172.21.15.151:0/4163998877 wait complete. 2024-09-16T15:50:25.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:25.518+0000 7f2deb200640 1 Processor -- start 2024-09-16T15:50:25.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:25.518+0000 7f2deb200640 1 -- start start 2024-09-16T15:50:25.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:25.520+0000 7f2deb200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2dec1f81f0 0x7f2dec1f5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:25.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:25.520+0000 7f2deb200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2dec0c7920 0x7f2dec1f85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:25.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:25.521+0000 7f2deb200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2dec1f8b10 0x7f2dec0cc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:25.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:25.521+0000 7f2deb200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2dec0b50c0 con 0x7f2dec0c7920 2024-09-16T15:50:25.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:25.521+0000 7f2de9400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2dec0c7920 0x7f2dec1f85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:25.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:25.521+0000 7f2deb200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2dec0b4f40 con 0x7f2dec1f8b10 2024-09-16T15:50:25.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:25.521+0000 7f2deb200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f2dec0d22f0 con 0x7f2dec1f81f0 2024-09-16T15:50:25.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:25.521+0000 7f2de9400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2dec0c7920 0x7f2dec1f85d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:35270/0 (socket says 172.21.15.151:35270) 2024-09-16T15:50:25.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:25.521+0000 7f2de9400640 1 -- 172.21.15.151:0/1688146091 learned_addr learned my addr 172.21.15.151:0/1688146091 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:25.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:25.523+0000 7f2dea800640 1 --2- 172.21.15.151:0/1688146091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2dec1f8b10 0x7f2dec0cc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:25.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:25.523+0000 7f2de9400640 1 -- 172.21.15.151:0/1688146091 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2dec1f81f0 msgr2=0x7f2dec1f5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:25.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:25.523+0000 7f2de9400640 1 --2- 172.21.15.151:0/1688146091 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2dec1f81f0 0x7f2dec1f5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:25.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:25.523+0000 7f2de9400640 1 -- 172.21.15.151:0/1688146091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2dec1f8b10 msgr2=0x7f2dec0cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:25.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:25.523+0000 7f2de9400640 1 --2- 172.21.15.151:0/1688146091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2dec1f8b10 0x7f2dec0cc100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:25.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:25.523+0000 7f2de9400640 1 -- 172.21.15.151:0/1688146091 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dec0cd080 con 0x7f2dec0c7920 2024-09-16T15:50:25.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:25.523+0000 7f2dea800640 1 --2- 172.21.15.151:0/1688146091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2dec1f8b10 0x7f2dec0cc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:50:25.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:25.525+0000 7f2de9400640 1 --2- 172.21.15.151:0/1688146091 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2dec0c7920 0x7f2dec1f85d0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f2dd802b190 tx=0x7f2dd8001720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:25.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:25.526+0000 7f2de2000640 1 -- 172.21.15.151:0/1688146091 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd8006600 con 0x7f2dec0c7920 2024-09-16T15:50:25.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:25.526+0000 7f2de2000640 1 -- 172.21.15.151:0/1688146091 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2dd8003a00 con 0x7f2dec0c7920 2024-09-16T15:50:25.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:25.526+0000 7f2de2000640 1 -- 172.21.15.151:0/1688146091 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd8003e50 con 0x7f2dec0c7920 2024-09-16T15:50:25.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:25.527+0000 7f2deb200640 1 -- 172.21.15.151:0/1688146091 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dec1f6130 con 0x7f2dec0c7920 2024-09-16T15:50:25.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:25.528+0000 7f2deb200640 1 -- 172.21.15.151:0/1688146091 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dec1f66f0 con 0x7f2dec0c7920 2024-09-16T15:50:25.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:25.530+0000 7f2deb200640 1 -- 172.21.15.151:0/1688146091 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2dec0ce820 con 0x7f2dec0c7920 2024-09-16T15:50:25.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:25.531+0000 7f2de2000640 1 -- 172.21.15.151:0/1688146091 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 36) ==== 103225+0+0 (secure 0 0 0) 0x7f2dd802b560 con 0x7f2dec0c7920 2024-09-16T15:50:25.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:25.534+0000 7f2de2000640 1 --2- 172.21.15.151:0/1688146091 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2dcc079f80 0x7f2dcc07c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:25.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:25.535+0000 7f2de2000640 1 -- 172.21.15.151:0/1688146091 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7f2dd8006d30 con 0x7f2dec0c7920 2024-09-16T15:50:25.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:25.538+0000 7f2de9e00640 1 --2- 172.21.15.151:0/1688146091 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2dcc079f80 0x7f2dcc07c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:25.546 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:25.545+0000 7f2de2000640 1 -- 172.21.15.151:0/1688146091 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2dd8082e10 con 0x7f2dec0c7920 2024-09-16T15:50:25.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:25.569+0000 7f2de9e00640 1 --2- 172.21.15.151:0/1688146091 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2dcc079f80 0x7f2dcc07c440 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f2dec052150 tx=0x7f2ddc0079b0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:25.997 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:25.995+0000 7f2deb200640 1 -- 172.21.15.151:0/1688146091 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone446916", "target": ["mon-mgr", ""]}) -- 0x7f2dec069d80 con 0x7f2dcc079f80 2024-09-16T15:50:27.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:27.453+0000 7f2de2000640 1 -- 172.21.15.151:0/1688146091 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2ddc005030 con 0x7f2dcc079f80 2024-09-16T15:50:27.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:27.458+0000 7f2deb200640 1 -- 172.21.15.151:0/1688146091 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2dcc079f80 msgr2=0x7f2dcc07c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:27.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:27.458+0000 7f2deb200640 1 --2- 172.21.15.151:0/1688146091 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2dcc079f80 0x7f2dcc07c440 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f2dec052150 tx=0x7f2ddc0079b0 comp rx=0 tx=0).stop 2024-09-16T15:50:27.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:27.458+0000 7f2deb200640 1 -- 172.21.15.151:0/1688146091 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2dec0c7920 msgr2=0x7f2dec1f85d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:27.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:27.458+0000 7f2deb200640 1 --2- 172.21.15.151:0/1688146091 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2dec0c7920 0x7f2dec1f85d0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f2dd802b190 tx=0x7f2dd8001720 comp rx=0 tx=0).stop 2024-09-16T15:50:27.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:27.460+0000 7f2deb200640 1 -- 172.21.15.151:0/1688146091 shutdown_connections 2024-09-16T15:50:27.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:27.460+0000 7f2deb200640 1 --2- 172.21.15.151:0/1688146091 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2dcc079f80 0x7f2dcc07c440 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:27.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:27.460+0000 7f2deb200640 1 --2- 172.21.15.151:0/1688146091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2dec1f8b10 0x7f2dec0cc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:27.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:27.460+0000 7f2deb200640 1 --2- 172.21.15.151:0/1688146091 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2dec0c7920 0x7f2dec1f85d0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:27.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:27.460+0000 7f2deb200640 1 --2- 172.21.15.151:0/1688146091 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2dec1f81f0 0x7f2dec1f5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:27.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:27.460+0000 7f2deb200640 1 -- 172.21.15.151:0/1688146091 >> 172.21.15.151:0/1688146091 conn(0x7f2dec09d0b0 msgr2=0x7f2dec0cea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:27.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:27.461+0000 7f2deb200640 1 -- 172.21.15.151:0/1688146091 shutdown_connections 2024-09-16T15:50:27.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:27.461+0000 7f2deb200640 1 -- 172.21.15.151:0/1688146091 wait complete. 2024-09-16T15:50:27.489 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone962717 2024-09-16T15:50:27.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:27.871+0000 7fc765800640 1 -- 172.21.15.151:0/2080779298 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7601437c0 msgr2=0x7fc7600cdad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:27.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:27.871+0000 7fc765800640 1 --2- 172.21.15.151:0/2080779298 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7601437c0 0x7fc7600cdad0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fc7600cd540 tx=0x7fc75002b1e0 comp rx=0 tx=0).stop 2024-09-16T15:50:27.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:27.872+0000 7fc765800640 1 -- 172.21.15.151:0/2080779298 shutdown_connections 2024-09-16T15:50:27.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:27.872+0000 7fc765800640 1 --2- 172.21.15.151:0/2080779298 >> v2:172.21.15.167:3300/0 conn(0x7fc7600b4bd0 0x7fc7600b3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:27.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:27.872+0000 7fc765800640 1 --2- 172.21.15.151:0/2080779298 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7601437c0 0x7fc7600cdad0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:27.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:27.872+0000 7fc765800640 1 -- 172.21.15.151:0/2080779298 >> 172.21.15.151:0/2080779298 conn(0x7fc7600b1840 msgr2=0x7fc7600b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:27.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:27.872+0000 7fc765800640 1 -- 172.21.15.151:0/2080779298 shutdown_connections 2024-09-16T15:50:27.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:27.872+0000 7fc765800640 1 -- 172.21.15.151:0/2080779298 wait complete. 2024-09-16T15:50:27.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:27.875+0000 7fc765800640 1 Processor -- start 2024-09-16T15:50:27.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:27.876+0000 7fc765800640 1 -- start start 2024-09-16T15:50:27.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:27.877+0000 7fc765800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7600b4400 0x7fc760139cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:27.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:27.878+0000 7fc765800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7600b4bd0 0x7fc76013a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:27.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:27.878+0000 7fc765800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7601437c0 0x7fc7601f40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:27.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:27.878+0000 7fc765800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc76006f870 con 0x7fc7600b4bd0 2024-09-16T15:50:27.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:27.878+0000 7fc765800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc760070a80 con 0x7fc7600b4400 2024-09-16T15:50:27.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:27.878+0000 7fc765800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fc76006f9f0 con 0x7fc7601437c0 2024-09-16T15:50:27.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:27.879+0000 7fc75f400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7601437c0 0x7fc7601f40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:27.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:27.879+0000 7fc75f400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7601437c0 0x7fc7601f40c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.151:49758/0 (socket says 172.21.15.151:49758) 2024-09-16T15:50:27.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:27.879+0000 7fc75f400640 1 -- 172.21.15.151:0/1321194426 learned_addr learned my addr 172.21.15.151:0/1321194426 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:27.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:27.880+0000 7fc75ea00640 1 --2- 172.21.15.151:0/1321194426 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7600b4400 0x7fc760139cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:27.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:27.880+0000 7fc757e00640 1 --2- 172.21.15.151:0/1321194426 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7600b4bd0 0x7fc76013a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:27.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:27.881+0000 7fc75ea00640 1 -- 172.21.15.151:0/1321194426 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7601437c0 msgr2=0x7fc7601f40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:27.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:27.881+0000 7fc75ea00640 1 --2- 172.21.15.151:0/1321194426 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7601437c0 0x7fc7601f40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:27.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:27.881+0000 7fc75ea00640 1 -- 172.21.15.151:0/1321194426 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7600b4bd0 msgr2=0x7fc76013a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:27.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:27.881+0000 7fc75ea00640 1 --2- 172.21.15.151:0/1321194426 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7600b4bd0 0x7fc76013a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:27.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:27.881+0000 7fc75ea00640 1 -- 172.21.15.151:0/1321194426 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc760156940 con 0x7fc7600b4400 2024-09-16T15:50:27.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:27.881+0000 7fc757e00640 1 --2- 172.21.15.151:0/1321194426 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7600b4bd0 0x7fc76013a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:50:27.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:27.882+0000 7fc75ea00640 1 --2- 172.21.15.151:0/1321194426 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7600b4400 0x7fc760139cb0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fc750031000 tx=0x7fc750003880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:27.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:27.883+0000 7fc756a00640 1 -- 172.21.15.151:0/1321194426 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc750005d50 con 0x7fc7600b4400 2024-09-16T15:50:27.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:27.883+0000 7fc756a00640 1 -- 172.21.15.151:0/1321194426 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc750005020 con 0x7fc7600b4400 2024-09-16T15:50:27.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:27.883+0000 7fc756a00640 1 -- 172.21.15.151:0/1321194426 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc750005470 con 0x7fc7600b4400 2024-09-16T15:50:27.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:27.883+0000 7fc765800640 1 -- 172.21.15.151:0/1321194426 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc760156c20 con 0x7fc7600b4400 2024-09-16T15:50:27.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:27.883+0000 7fc765800640 1 -- 172.21.15.151:0/1321194426 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc760134bb0 con 0x7fc7600b4400 2024-09-16T15:50:27.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:27.886+0000 7fc765800640 1 -- 172.21.15.151:0/1321194426 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7600b3ec0 con 0x7fc7600b4400 2024-09-16T15:50:27.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:27.892+0000 7fc756a00640 1 -- 172.21.15.151:0/1321194426 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 36) ==== 103225+0+0 (secure 0 0 0) 0x7fc7500047a0 con 0x7fc7600b4400 2024-09-16T15:50:27.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:27.893+0000 7fc756a00640 1 --2- 172.21.15.151:0/1321194426 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc73c079e30 0x7fc73c07c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:27.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:27.893+0000 7fc756a00640 1 -- 172.21.15.151:0/1321194426 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7fc7500b9640 con 0x7fc7600b4400 2024-09-16T15:50:27.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:27.904+0000 7fc756a00640 1 -- 172.21.15.151:0/1321194426 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc750083340 con 0x7fc7600b4400 2024-09-16T15:50:27.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:27.926+0000 7fc757e00640 1 --2- 172.21.15.151:0/1321194426 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc73c079e30 0x7fc73c07c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:27.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:27.944+0000 7fc757e00640 1 --2- 172.21.15.151:0/1321194426 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc73c079e30 0x7fc73c07c2f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fc748006fd0 tx=0x7fc748008040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:28.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:28.372+0000 7fc765800640 1 -- 172.21.15.151:0/1321194426 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone962717", "target": ["mon-mgr", ""]}) -- 0x7fc7600cd570 con 0x7fc73c079e30 2024-09-16T15:50:29.488 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:29.486+0000 7fc756a00640 1 -- 172.21.15.151:0/1321194426 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc7600cd570 con 0x7fc73c079e30 2024-09-16T15:50:29.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:29.494+0000 7fc765800640 1 -- 172.21.15.151:0/1321194426 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc73c079e30 msgr2=0x7fc73c07c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:29.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:29.494+0000 7fc765800640 1 --2- 172.21.15.151:0/1321194426 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc73c079e30 0x7fc73c07c2f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fc748006fd0 tx=0x7fc748008040 comp rx=0 tx=0).stop 2024-09-16T15:50:29.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:29.494+0000 7fc765800640 1 -- 172.21.15.151:0/1321194426 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7600b4400 msgr2=0x7fc760139cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:29.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:29.494+0000 7fc765800640 1 --2- 172.21.15.151:0/1321194426 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7600b4400 0x7fc760139cb0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fc750031000 tx=0x7fc750003880 comp rx=0 tx=0).stop 2024-09-16T15:50:29.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:29.495+0000 7fc765800640 1 -- 172.21.15.151:0/1321194426 shutdown_connections 2024-09-16T15:50:29.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:29.495+0000 7fc765800640 1 --2- 172.21.15.151:0/1321194426 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc73c079e30 0x7fc73c07c2f0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:29.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:29.495+0000 7fc765800640 1 --2- 172.21.15.151:0/1321194426 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7601437c0 0x7fc7601f40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:29.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:29.495+0000 7fc765800640 1 --2- 172.21.15.151:0/1321194426 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7600b4bd0 0x7fc76013a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:29.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:29.495+0000 7fc765800640 1 --2- 172.21.15.151:0/1321194426 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7600b4400 0x7fc760139cb0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:29.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:29.495+0000 7fc765800640 1 -- 172.21.15.151:0/1321194426 >> 172.21.15.151:0/1321194426 conn(0x7fc7600b1840 msgr2=0x7fc7600aeab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:29.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:29.495+0000 7fc765800640 1 -- 172.21.15.151:0/1321194426 shutdown_connections 2024-09-16T15:50:29.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:29.495+0000 7fc765800640 1 -- 172.21.15.151:0/1321194426 wait complete. 2024-09-16T15:50:29.524 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone499718 2024-09-16T15:50:29.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:29.794+0000 7fbac0e00640 1 -- 172.21.15.151:0/2360148074 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbabc0cec60 msgr2=0x7fbabc0cf070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:29.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:29.794+0000 7fbac0e00640 1 --2- 172.21.15.151:0/2360148074 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbabc0cec60 0x7fbabc0cf070 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fbabc0b2470 tx=0x7fbab402a540 comp rx=0 tx=0).stop 2024-09-16T15:50:29.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:29.794+0000 7fbac0e00640 1 -- 172.21.15.151:0/2360148074 shutdown_connections 2024-09-16T15:50:29.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:29.794+0000 7fbac0e00640 1 --2- 172.21.15.151:0/2360148074 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbabc0cec60 0x7fbabc0cf070 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:29.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:29.794+0000 7fbac0e00640 1 --2- 172.21.15.151:0/2360148074 >> v2:172.21.15.167:3300/0 conn(0x7fbabc0af190 0x7fbabc0b2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:29.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:29.794+0000 7fbac0e00640 1 -- 172.21.15.151:0/2360148074 >> 172.21.15.151:0/2360148074 conn(0x7fbabc0b1840 msgr2=0x7fbabc0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:29.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:29.794+0000 7fbac0e00640 1 -- 172.21.15.151:0/2360148074 shutdown_connections 2024-09-16T15:50:29.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:29.794+0000 7fbac0e00640 1 -- 172.21.15.151:0/2360148074 wait complete. 2024-09-16T15:50:29.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:29.796+0000 7fbac0e00640 1 Processor -- start 2024-09-16T15:50:29.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:29.797+0000 7fbac0e00640 1 -- start start 2024-09-16T15:50:29.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:29.801+0000 7fbac0e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbabc0af190 0x7fbabc15b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:29.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:29.801+0000 7fbac0e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbabc0cec60 0x7fbabc1484c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:29.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:29.802+0000 7fbac0e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbabc161760 0x7fbabc159020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:29.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:29.802+0000 7fbac0e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbabc162310 con 0x7fbabc0cec60 2024-09-16T15:50:29.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:29.802+0000 7fbac0e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbabc162190 con 0x7fbabc161760 2024-09-16T15:50:29.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:29.802+0000 7fbac0e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fbabc162490 con 0x7fbabc0af190 2024-09-16T15:50:29.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:29.802+0000 7fbab9400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbabc0cec60 0x7fbabc1484c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:29.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:29.802+0000 7fbab9400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbabc0cec60 0x7fbabc1484c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:35310/0 (socket says 172.21.15.151:35310) 2024-09-16T15:50:29.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:29.802+0000 7fbab9400640 1 -- 172.21.15.151:0/2945373645 learned_addr learned my addr 172.21.15.151:0/2945373645 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:29.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:29.803+0000 7fbaba800640 1 --2- 172.21.15.151:0/2945373645 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbabc161760 0x7fbabc159020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:29.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:29.803+0000 7fbab9e00640 1 --2- 172.21.15.151:0/2945373645 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbabc0af190 0x7fbabc15b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:29.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:29.805+0000 7fbab9e00640 1 -- 172.21.15.151:0/2945373645 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbabc161760 msgr2=0x7fbabc159020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:29.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:29.805+0000 7fbab9e00640 1 --2- 172.21.15.151:0/2945373645 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbabc161760 0x7fbabc159020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:29.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:29.805+0000 7fbab9e00640 1 -- 172.21.15.151:0/2945373645 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbabc0cec60 msgr2=0x7fbabc1484c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:29.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:29.805+0000 7fbab9e00640 1 --2- 172.21.15.151:0/2945373645 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbabc0cec60 0x7fbabc1484c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:29.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:29.805+0000 7fbab9e00640 1 -- 172.21.15.151:0/2945373645 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbabc15c190 con 0x7fbabc0af190 2024-09-16T15:50:29.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:29.805+0000 7fbaba800640 1 --2- 172.21.15.151:0/2945373645 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbabc161760 0x7fbabc159020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:50:29.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:29.805+0000 7fbab9400640 1 --2- 172.21.15.151:0/2945373645 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbabc0cec60 0x7fbabc1484c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:50:29.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:29.806+0000 7fbab9e00640 1 --2- 172.21.15.151:0/2945373645 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbabc0af190 0x7fbabc15b080 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbaac00cae0 tx=0x7fbaac00a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:29.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:29.809+0000 7fbab2000640 1 -- 172.21.15.151:0/2945373645 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaac016020 con 0x7fbabc0af190 2024-09-16T15:50:29.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:29.809+0000 7fbab2000640 1 -- 172.21.15.151:0/2945373645 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbaac00acd0 con 0x7fbabc0af190 2024-09-16T15:50:29.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:29.809+0000 7fbab2000640 1 -- 172.21.15.151:0/2945373645 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaac012680 con 0x7fbabc0af190 2024-09-16T15:50:29.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:29.809+0000 7fbac0e00640 1 -- 172.21.15.151:0/2945373645 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbabc15c430 con 0x7fbabc0af190 2024-09-16T15:50:29.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:29.810+0000 7fbac0e00640 1 -- 172.21.15.151:0/2945373645 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbabc15c9f0 con 0x7fbabc0af190 2024-09-16T15:50:29.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:29.813+0000 7fbac0e00640 1 -- 172.21.15.151:0/2945373645 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbabc06ef50 con 0x7fbabc0af190 2024-09-16T15:50:29.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:29.815+0000 7fbab2000640 1 -- 172.21.15.151:0/2945373645 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 36) ==== 103225+0+0 (secure 0 0 0) 0x7fbaac019020 con 0x7fbabc0af190 2024-09-16T15:50:29.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:29.817+0000 7fbab2000640 1 --2- 172.21.15.151:0/2945373645 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fba98079de0 0x7fba9807c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:29.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:29.818+0000 7fbab2000640 1 -- 172.21.15.151:0/2945373645 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7fbaac099c70 con 0x7fbabc0af190 2024-09-16T15:50:29.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:29.831+0000 7fbab9400640 1 --2- 172.21.15.151:0/2945373645 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fba98079de0 0x7fba9807c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:29.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:29.833+0000 7fbab2000640 1 -- 172.21.15.151:0/2945373645 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fbabc06ef50 con 0x7fbabc0af190 2024-09-16T15:50:29.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:29.834+0000 7fbab9400640 1 --2- 172.21.15.151:0/2945373645 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fba98079de0 0x7fba9807c2a0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fbabc158600 tx=0x7fbaa400a380 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:30.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:30.224+0000 7fbac0e00640 1 -- 172.21.15.151:0/2945373645 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone499718", "target": ["mon-mgr", ""]}) -- 0x7fbabc0b2b40 con 0x7fba98079de0 2024-09-16T15:50:30.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:30.642+0000 7fbab2000640 1 -- 172.21.15.151:0/2945373645 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fbabc0b2b40 con 0x7fba98079de0 2024-09-16T15:50:30.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:30.645+0000 7fbac0e00640 1 -- 172.21.15.151:0/2945373645 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fba98079de0 msgr2=0x7fba9807c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:30.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:30.645+0000 7fbac0e00640 1 --2- 172.21.15.151:0/2945373645 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fba98079de0 0x7fba9807c2a0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fbabc158600 tx=0x7fbaa400a380 comp rx=0 tx=0).stop 2024-09-16T15:50:30.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:30.645+0000 7fbac0e00640 1 -- 172.21.15.151:0/2945373645 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbabc0af190 msgr2=0x7fbabc15b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:30.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:30.645+0000 7fbac0e00640 1 --2- 172.21.15.151:0/2945373645 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbabc0af190 0x7fbabc15b080 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbaac00cae0 tx=0x7fbaac00a060 comp rx=0 tx=0).stop 2024-09-16T15:50:30.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:30.645+0000 7fbac0e00640 1 -- 172.21.15.151:0/2945373645 shutdown_connections 2024-09-16T15:50:30.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:30.645+0000 7fbac0e00640 1 --2- 172.21.15.151:0/2945373645 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fba98079de0 0x7fba9807c2a0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:30.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:30.645+0000 7fbac0e00640 1 --2- 172.21.15.151:0/2945373645 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbabc161760 0x7fbabc159020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:30.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:30.645+0000 7fbac0e00640 1 --2- 172.21.15.151:0/2945373645 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbabc0cec60 0x7fbabc1484c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:30.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:30.645+0000 7fbac0e00640 1 --2- 172.21.15.151:0/2945373645 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbabc0af190 0x7fbabc15b080 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:30.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:30.645+0000 7fbac0e00640 1 -- 172.21.15.151:0/2945373645 >> 172.21.15.151:0/2945373645 conn(0x7fbabc0b1840 msgr2=0x7fbabc0b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:30.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:30.646+0000 7fbac0e00640 1 -- 172.21.15.151:0/2945373645 shutdown_connections 2024-09-16T15:50:30.648 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:30.646+0000 7fbac0e00640 1 -- 172.21.15.151:0/2945373645 wait complete. 2024-09-16T15:50:30.677 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone237619 2024-09-16T15:50:30.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:30.941+0000 7f3400400640 1 -- 172.21.15.151:0/2878566608 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f33e00039e0 msgr2=0x7f33e0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:30.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:30.941+0000 7f3400400640 1 --2- 172.21.15.151:0/2878566608 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f33e00039e0 0x7f33e0003de0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f33ec00c260 tx=0x7f33e0005830 comp rx=0 tx=0).stop 2024-09-16T15:50:30.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:30.941+0000 7f33fe400640 1 -- 172.21.15.151:0/2878566608 reap_dead start 2024-09-16T15:50:30.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:30.941+0000 7f3400400640 1 -- 172.21.15.151:0/2878566608 shutdown_connections 2024-09-16T15:50:30.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:30.941+0000 7f3400400640 1 -- 172.21.15.151:0/2878566608 >> 172.21.15.151:0/2878566608 conn(0x7f33f80b1840 msgr2=0x7f33f80b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:30.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:30.941+0000 7f3400400640 1 -- 172.21.15.151:0/2878566608 shutdown_connections 2024-09-16T15:50:30.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:30.941+0000 7f3400400640 1 -- 172.21.15.151:0/2878566608 wait complete. 2024-09-16T15:50:30.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:30.943+0000 7f3400400640 1 Processor -- start 2024-09-16T15:50:30.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:30.943+0000 7f3400400640 1 -- start start 2024-09-16T15:50:30.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:30.945+0000 7f3400400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33e00039e0 0x7f33f8160870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:30.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:30.945+0000 7f3400400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33f8143f80 0x7f33f81443e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:30.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:30.945+0000 7f3400400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f33f815dc20 0x7f33f815e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:30.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:30.945+0000 7f3400400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f33f80cf0b0 con 0x7f33f815dc20 2024-09-16T15:50:30.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:30.945+0000 7f3400400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f33f80cf230 con 0x7f33f8143f80 2024-09-16T15:50:30.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:30.945+0000 7f3400400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f33f80cf3b0 con 0x7f33e00039e0 2024-09-16T15:50:30.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:30.945+0000 7f33fe400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f33f815dc20 0x7f33f815e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:30.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:30.945+0000 7f33fe400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f33f815dc20 0x7f33f815e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:35338/0 (socket says 172.21.15.151:35338) 2024-09-16T15:50:30.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:30.945+0000 7f33fe400640 1 -- 172.21.15.151:0/2165665666 learned_addr learned my addr 172.21.15.151:0/2165665666 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:30.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:30.946+0000 7f33fe400640 1 -- 172.21.15.151:0/2165665666 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33e00039e0 msgr2=0x7f33f8160870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:30.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:30.946+0000 7f33fe400640 1 --2- 172.21.15.151:0/2165665666 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33e00039e0 0x7f33f8160870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:30.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:30.946+0000 7f33fe400640 1 -- 172.21.15.151:0/2165665666 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33f8143f80 msgr2=0x7f33f81443e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:30.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:30.946+0000 7f33fe400640 1 --2- 172.21.15.151:0/2165665666 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33f8143f80 0x7f33f81443e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:30.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:30.946+0000 7f33fe400640 1 -- 172.21.15.151:0/2165665666 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33f815faf0 con 0x7f33f815dc20 2024-09-16T15:50:30.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:30.946+0000 7f33fe400640 1 --2- 172.21.15.151:0/2165665666 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f33f815dc20 0x7f33f815e0a0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f33f80b4810 tx=0x7f33f000db30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:30.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:30.946+0000 7f33f5600640 1 -- 172.21.15.151:0/2165665666 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f0014020 con 0x7f33f815dc20 2024-09-16T15:50:30.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:30.946+0000 7f33f5600640 1 -- 172.21.15.151:0/2165665666 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f33f0004cb0 con 0x7f33f815dc20 2024-09-16T15:50:30.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:30.946+0000 7f3400400640 1 -- 172.21.15.151:0/2165665666 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33f81f5360 con 0x7f33f815dc20 2024-09-16T15:50:30.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:30.946+0000 7f33f5600640 1 -- 172.21.15.151:0/2165665666 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f0004e60 con 0x7f33f815dc20 2024-09-16T15:50:30.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:30.946+0000 7f3400400640 1 -- 172.21.15.151:0/2165665666 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33f8160070 con 0x7f33f815dc20 2024-09-16T15:50:30.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:30.949+0000 7f3400400640 1 -- 172.21.15.151:0/2165665666 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33f80b2d50 con 0x7f33f815dc20 2024-09-16T15:50:30.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:30.951+0000 7f33f5600640 1 -- 172.21.15.151:0/2165665666 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 36) ==== 103225+0+0 (secure 0 0 0) 0x7f33f0017020 con 0x7f33f815dc20 2024-09-16T15:50:30.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:30.953+0000 7f33f5600640 1 --2- 172.21.15.151:0/2165665666 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f33d8079db0 0x7f33d807c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:30.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:30.953+0000 7f33f5600640 1 -- 172.21.15.151:0/2165665666 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7f33f0099ea0 con 0x7f33f815dc20 2024-09-16T15:50:30.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:30.954+0000 7f33fda00640 1 --2- 172.21.15.151:0/2165665666 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f33d8079db0 0x7f33d807c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:30.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:30.960+0000 7f33fda00640 1 --2- 172.21.15.151:0/2165665666 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f33d8079db0 0x7f33d807c270 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f33ec033000 tx=0x7f33f8070080 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:30.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:30.961+0000 7f33f5600640 1 -- 172.21.15.151:0/2165665666 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f33f001e020 con 0x7f33f815dc20 2024-09-16T15:50:31.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:31.353+0000 7f3400400640 1 -- 172.21.15.151:0/2165665666 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone237619", "target": ["mon-mgr", ""]}) -- 0x7f33f8157ce0 con 0x7f33d8079db0 2024-09-16T15:50:31.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:31.537+0000 7f33f5600640 1 -- 172.21.15.151:0/2165665666 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f33f8157ce0 con 0x7f33d8079db0 2024-09-16T15:50:31.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:31.540+0000 7f3400400640 1 -- 172.21.15.151:0/2165665666 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f33d8079db0 msgr2=0x7f33d807c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:31.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:31.540+0000 7f3400400640 1 --2- 172.21.15.151:0/2165665666 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f33d8079db0 0x7f33d807c270 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f33ec033000 tx=0x7f33f8070080 comp rx=0 tx=0).stop 2024-09-16T15:50:31.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:31.540+0000 7f3400400640 1 -- 172.21.15.151:0/2165665666 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f33f815dc20 msgr2=0x7f33f815e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:31.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:31.540+0000 7f3400400640 1 --2- 172.21.15.151:0/2165665666 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f33f815dc20 0x7f33f815e0a0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f33f80b4810 tx=0x7f33f000db30 comp rx=0 tx=0).stop 2024-09-16T15:50:31.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:31.540+0000 7f3400400640 1 -- 172.21.15.151:0/2165665666 shutdown_connections 2024-09-16T15:50:31.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:31.540+0000 7f3400400640 1 --2- 172.21.15.151:0/2165665666 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f33d8079db0 0x7f33d807c270 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:31.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:31.540+0000 7f3400400640 1 --2- 172.21.15.151:0/2165665666 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33f8143f80 0x7f33f81443e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:31.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:31.540+0000 7f3400400640 1 --2- 172.21.15.151:0/2165665666 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f33f815dc20 0x7f33f815e0a0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:31.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:31.540+0000 7f3400400640 1 --2- 172.21.15.151:0/2165665666 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33e00039e0 0x7f33f8160870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:31.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:31.540+0000 7f3400400640 1 -- 172.21.15.151:0/2165665666 >> 172.21.15.151:0/2165665666 conn(0x7f33f80b1840 msgr2=0x7f33f80b3f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:31.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:31.540+0000 7f3400400640 1 -- 172.21.15.151:0/2165665666 shutdown_connections 2024-09-16T15:50:31.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:31.540+0000 7f3400400640 1 -- 172.21.15.151:0/2165665666 wait complete. 2024-09-16T15:50:31.557 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-16T15:50:31.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:31.717+0000 7f910dc00640 1 -- 172.21.15.151:0/3116642592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91080af3d0 msgr2=0x7f910806f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:31.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:31.717+0000 7f910dc00640 1 --2- 172.21.15.151:0/3116642592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91080af3d0 0x7f910806f660 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f90f4009830 tx=0x7f90f402a4b0 comp rx=0 tx=0).stop 2024-09-16T15:50:31.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:31.717+0000 7f910dc00640 1 -- 172.21.15.151:0/3116642592 shutdown_connections 2024-09-16T15:50:31.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:31.717+0000 7f910dc00640 1 --2- 172.21.15.151:0/3116642592 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91080af3d0 0x7f910806f660 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:31.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:31.717+0000 7f910dc00640 1 -- 172.21.15.151:0/3116642592 >> 172.21.15.151:0/3116642592 conn(0x7f91080b1950 msgr2=0x7f91080b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:31.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:31.717+0000 7f910dc00640 1 -- 172.21.15.151:0/3116642592 shutdown_connections 2024-09-16T15:50:31.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:31.717+0000 7f910dc00640 1 -- 172.21.15.151:0/3116642592 wait complete. 2024-09-16T15:50:31.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:31.718+0000 7f910dc00640 1 Processor -- start 2024-09-16T15:50:31.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:31.719+0000 7f910dc00640 1 -- start start 2024-09-16T15:50:31.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:31.719+0000 7f910dc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9108072df0 0x7f910814f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:31.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:31.719+0000 7f910dc00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f91080af3d0 0x7f91081594e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:31.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:31.719+0000 7f910dc00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9108161550 0x7f9108156a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:31.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:31.720+0000 7f910dc00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9108073950 con 0x7f9108072df0 2024-09-16T15:50:31.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:31.720+0000 7f910dc00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f910806fdb0 con 0x7f9108161550 2024-09-16T15:50:31.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:31.720+0000 7f910dc00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f9108073ad0 con 0x7f91080af3d0 2024-09-16T15:50:31.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:31.720+0000 7f9106a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9108072df0 0x7f910814f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:31.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:31.720+0000 7f9106a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9108072df0 0x7f910814f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:35352/0 (socket says 172.21.15.151:35352) 2024-09-16T15:50:31.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:31.720+0000 7f9106a00640 1 -- 172.21.15.151:0/1825534854 learned_addr learned my addr 172.21.15.151:0/1825534854 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:31.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:31.720+0000 7f9107400640 1 --2- 172.21.15.151:0/1825534854 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9108161550 0x7f9108156a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:31.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:31.720+0000 7f9106000640 1 --2- 172.21.15.151:0/1825534854 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f91080af3d0 0x7f91081594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:31.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:31.720+0000 7f9106a00640 1 -- 172.21.15.151:0/1825534854 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f91080af3d0 msgr2=0x7f91081594e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:31.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:31.720+0000 7f9106a00640 1 --2- 172.21.15.151:0/1825534854 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f91080af3d0 0x7f91081594e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:31.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:31.720+0000 7f9106a00640 1 -- 172.21.15.151:0/1825534854 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9108161550 msgr2=0x7f9108156a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:31.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:31.720+0000 7f9106a00640 1 --2- 172.21.15.151:0/1825534854 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9108161550 0x7f9108156a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:31.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:31.720+0000 7f9106a00640 1 -- 172.21.15.151:0/1825534854 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9108159a20 con 0x7f9108072df0 2024-09-16T15:50:31.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:31.720+0000 7f9106a00640 1 --2- 172.21.15.151:0/1825534854 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9108072df0 0x7f910814f1e0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f90f800c500 tx=0x7f90f800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:31.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:31.721+0000 7f90fea00640 1 -- 172.21.15.151:0/1825534854 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f8012830 con 0x7f9108072df0 2024-09-16T15:50:31.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:31.721+0000 7f90fea00640 1 -- 172.21.15.151:0/1825534854 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f90f8017070 con 0x7f9108072df0 2024-09-16T15:50:31.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:31.721+0000 7f910dc00640 1 -- 172.21.15.151:0/1825534854 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91081557f0 con 0x7f9108072df0 2024-09-16T15:50:31.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:31.721+0000 7f90fea00640 1 -- 172.21.15.151:0/1825534854 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f8012d60 con 0x7f9108072df0 2024-09-16T15:50:31.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:31.721+0000 7f910dc00640 1 -- 172.21.15.151:0/1825534854 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9108156210 con 0x7f9108072df0 2024-09-16T15:50:31.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:31.722+0000 7f910dc00640 1 -- 172.21.15.151:0/1825534854 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9108073fc0 con 0x7f9108072df0 2024-09-16T15:50:31.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:31.723+0000 7f90fea00640 1 -- 172.21.15.151:0/1825534854 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 36) ==== 103225+0+0 (secure 0 0 0) 0x7f90f801c460 con 0x7f9108072df0 2024-09-16T15:50:31.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:31.724+0000 7f90fea00640 1 --2- 172.21.15.151:0/1825534854 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f90e8079db0 0x7f90e807c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:31.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:31.724+0000 7f90fea00640 1 -- 172.21.15.151:0/1825534854 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7f90f80997e0 con 0x7f9108072df0 2024-09-16T15:50:31.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:31.724+0000 7f9106000640 1 --2- 172.21.15.151:0/1825534854 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f90e8079db0 0x7f90e807c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:31.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:31.725+0000 7f9106000640 1 --2- 172.21.15.151:0/1825534854 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f90e8079db0 0x7f90e807c270 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f90f4034000 tx=0x7f90f4004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:31.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:31.728+0000 7f90fea00640 1 -- 172.21.15.151:0/1825534854 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f90f8062dd0 con 0x7f9108072df0 2024-09-16T15:50:31.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:31.938+0000 7f910dc00640 1 -- 172.21.15.151:0/1825534854 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f91080aec30 con 0x7f90e8079db0 2024-09-16T15:50:31.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:31.940+0000 7f90fea00640 1 -- 172.21.15.151:0/1825534854 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f91080aec30 con 0x7f90e8079db0 2024-09-16T15:50:31.941 INFO:teuthology.orchestra.run.smithi151.stdout:[ 2024-09-16T15:50:31.941 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:50:31.941 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "cephfs" 2024-09-16T15:50:31.941 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:50:31.941 INFO:teuthology.orchestra.run.smithi151.stdout:] 2024-09-16T15:50:31.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:31.942+0000 7f910dc00640 1 -- 172.21.15.151:0/1825534854 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f90e8079db0 msgr2=0x7f90e807c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:31.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:31.942+0000 7f910dc00640 1 --2- 172.21.15.151:0/1825534854 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f90e8079db0 0x7f90e807c270 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f90f4034000 tx=0x7f90f4004730 comp rx=0 tx=0).stop 2024-09-16T15:50:31.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:31.942+0000 7f910dc00640 1 -- 172.21.15.151:0/1825534854 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9108072df0 msgr2=0x7f910814f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:31.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:31.942+0000 7f910dc00640 1 --2- 172.21.15.151:0/1825534854 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9108072df0 0x7f910814f1e0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f90f800c500 tx=0x7f90f800c9c0 comp rx=0 tx=0).stop 2024-09-16T15:50:31.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:31.942+0000 7f910dc00640 1 -- 172.21.15.151:0/1825534854 shutdown_connections 2024-09-16T15:50:31.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:31.942+0000 7f910dc00640 1 --2- 172.21.15.151:0/1825534854 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f90e8079db0 0x7f90e807c270 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:31.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:31.942+0000 7f910dc00640 1 --2- 172.21.15.151:0/1825534854 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9108161550 0x7f9108156a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:31.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:31.942+0000 7f910dc00640 1 --2- 172.21.15.151:0/1825534854 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f91080af3d0 0x7f91081594e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:31.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:31.942+0000 7f910dc00640 1 --2- 172.21.15.151:0/1825534854 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9108072df0 0x7f910814f1e0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:31.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:31.942+0000 7f910dc00640 1 -- 172.21.15.151:0/1825534854 >> 172.21.15.151:0/1825534854 conn(0x7f91080b1950 msgr2=0x7f9108073240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:31.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:31.942+0000 7f910dc00640 1 -- 172.21.15.151:0/1825534854 shutdown_connections 2024-09-16T15:50:31.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:31.942+0000 7f910dc00640 1 -- 172.21.15.151:0/1825534854 wait complete. 2024-09-16T15:50:31.959 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-09-16T15:50:32.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.123+0000 7f228d400640 1 -- 172.21.15.151:0/2823318681 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2288145750 msgr2=0x7f2288073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:32.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.123+0000 7f228d400640 1 --2- 172.21.15.151:0/2823318681 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2288145750 0x7f2288073440 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2278009be0 tx=0x7f227802a890 comp rx=0 tx=0).stop 2024-09-16T15:50:32.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.123+0000 7f228d400640 1 -- 172.21.15.151:0/2823318681 shutdown_connections 2024-09-16T15:50:32.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.123+0000 7f228d400640 1 --2- 172.21.15.151:0/2823318681 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2288145750 0x7f2288073440 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:32.125 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.123+0000 7f228d400640 1 -- 172.21.15.151:0/2823318681 >> 172.21.15.151:0/2823318681 conn(0x7f22880b1840 msgr2=0x7f22880b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:32.125 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.123+0000 7f228d400640 1 -- 172.21.15.151:0/2823318681 shutdown_connections 2024-09-16T15:50:32.125 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.123+0000 7f228d400640 1 -- 172.21.15.151:0/2823318681 wait complete. 2024-09-16T15:50:32.125 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.124+0000 7f228d400640 1 Processor -- start 2024-09-16T15:50:32.125 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.124+0000 7f228d400640 1 -- start start 2024-09-16T15:50:32.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.125+0000 7f228d400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f228806efa0 0x7f228814e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:32.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.125+0000 7f228d400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f22880cccd0 0x7f22881e9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:32.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.125+0000 7f228d400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2288145750 0x7f22881ea110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:32.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.125+0000 7f228d400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f22880ce6d0 con 0x7f2288145750 2024-09-16T15:50:32.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.125+0000 7f228d400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f22880ce550 con 0x7f228806efa0 2024-09-16T15:50:32.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.125+0000 7f228d400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f22880ce850 con 0x7f22880cccd0 2024-09-16T15:50:32.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.126+0000 7f2287200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2288145750 0x7f22881ea110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:32.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.126+0000 7f2287200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2288145750 0x7f22881ea110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:35368/0 (socket says 172.21.15.151:35368) 2024-09-16T15:50:32.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.126+0000 7f2287200640 1 -- 172.21.15.151:0/2897911650 learned_addr learned my addr 172.21.15.151:0/2897911650 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:32.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.126+0000 7f2286800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f228806efa0 0x7f228814e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:32.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.126+0000 7f2285e00640 1 --2- 172.21.15.151:0/2897911650 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f22880cccd0 0x7f22881e9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:32.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.126+0000 7f2287200640 1 -- 172.21.15.151:0/2897911650 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f22880cccd0 msgr2=0x7f22881e9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:32.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.126+0000 7f2287200640 1 --2- 172.21.15.151:0/2897911650 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f22880cccd0 0x7f22881e9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:32.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.126+0000 7f2287200640 1 -- 172.21.15.151:0/2897911650 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f228806efa0 msgr2=0x7f228814e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:32.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.126+0000 7f2287200640 1 --2- 172.21.15.151:0/2897911650 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f228806efa0 0x7f228814e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:32.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.126+0000 7f2287200640 1 -- 172.21.15.151:0/2897911650 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22881e5df0 con 0x7f2288145750 2024-09-16T15:50:32.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.126+0000 7f2286800640 1 --2- 172.21.15.151:0/2897911650 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f228806efa0 0x7f228814e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:50:32.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.126+0000 7f2287200640 1 --2- 172.21.15.151:0/2897911650 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2288145750 0x7f22881ea110 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f228000c410 tx=0x7f228000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:32.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.127+0000 7f227ea00640 1 -- 172.21.15.151:0/2897911650 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2280011790 con 0x7f2288145750 2024-09-16T15:50:32.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.127+0000 7f227ea00640 1 -- 172.21.15.151:0/2897911650 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2280017070 con 0x7f2288145750 2024-09-16T15:50:32.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.127+0000 7f227ea00640 1 -- 172.21.15.151:0/2897911650 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2280011d20 con 0x7f2288145750 2024-09-16T15:50:32.130 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.127+0000 7f228d400640 1 -- 172.21.15.151:0/2897911650 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22881e60d0 con 0x7f2288145750 2024-09-16T15:50:32.130 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.127+0000 7f228d400640 1 -- 172.21.15.151:0/2897911650 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22881e6690 con 0x7f2288145750 2024-09-16T15:50:32.130 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.128+0000 7f228d400640 1 -- 172.21.15.151:0/2897911650 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22880ced00 con 0x7f2288145750 2024-09-16T15:50:32.130 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.130+0000 7f227ea00640 1 -- 172.21.15.151:0/2897911650 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 36) ==== 103225+0+0 (secure 0 0 0) 0x7f228001c460 con 0x7f2288145750 2024-09-16T15:50:32.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.131+0000 7f227ea00640 1 --2- 172.21.15.151:0/2897911650 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2264079db0 0x7f226407c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:32.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.131+0000 7f227ea00640 1 -- 172.21.15.151:0/2897911650 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7f2280099700 con 0x7f2288145750 2024-09-16T15:50:32.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.131+0000 7f2286800640 1 --2- 172.21.15.151:0/2897911650 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2264079db0 0x7f226407c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:32.133 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.132+0000 7f2286800640 1 --2- 172.21.15.151:0/2897911650 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2264079db0 0x7f226407c270 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2278034000 tx=0x7f2278004ab0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:32.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.136+0000 7f227ea00640 1 -- 172.21.15.151:0/2897911650 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2280062dd0 con 0x7f2288145750 2024-09-16T15:50:32.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.351+0000 7f228d400640 1 -- 172.21.15.151:0/2897911650 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f228815ec50 con 0x7f2264079db0 2024-09-16T15:50:32.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.365+0000 7f227ea00640 1 -- 172.21.15.151:0/2897911650 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+1028 (secure 0 0 0) 0x7f2278003bd0 con 0x7f2264079db0 2024-09-16T15:50:32.366 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:50:32.366 INFO:teuthology.orchestra.run.smithi151.stdout:[ 2024-09-16T15:50:32.367 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:50:32.367 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "subvol_clone13712" 2024-09-16T15:50:32.367 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:50:32.367 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:50:32.367 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "subvol_clone499718" 2024-09-16T15:50:32.367 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:50:32.367 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:50:32.367 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "subvol_clone55130" 2024-09-16T15:50:32.367 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:50:32.367 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:50:32.368 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "subvol_clone962717" 2024-09-16T15:50:32.368 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:50:32.368 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:50:32.368 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sv1" 2024-09-16T15:50:32.368 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:50:32.368 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:50:32.368 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "subvol_clone872410" 2024-09-16T15:50:32.368 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:50:32.368 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:50:32.368 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "subvol_clone22126" 2024-09-16T15:50:32.369 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:50:32.369 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:50:32.369 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "subvol_clone81887" 2024-09-16T15:50:32.369 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:50:32.369 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:50:32.369 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "subvol_clone556011" 2024-09-16T15:50:32.369 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:50:32.369 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:50:32.369 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "subvol_clone277913" 2024-09-16T15:50:32.369 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:50:32.369 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:50:32.369 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "subvol_clone53393" 2024-09-16T15:50:32.369 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:50:32.370 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:50:32.370 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "subvol_clone139815" 2024-09-16T15:50:32.370 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:50:32.370 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:50:32.370 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "subvol_clone515512" 2024-09-16T15:50:32.370 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:50:32.370 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:50:32.370 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "subvol_clone237619" 2024-09-16T15:50:32.370 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:50:32.370 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:50:32.370 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "subvol_clone35339" 2024-09-16T15:50:32.370 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:50:32.370 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:50:32.370 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "subvol_clone57948" 2024-09-16T15:50:32.371 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:50:32.371 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:50:32.371 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "subvol_clone15034" 2024-09-16T15:50:32.371 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:50:32.371 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:50:32.371 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "subvol_clone880714" 2024-09-16T15:50:32.371 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:50:32.371 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:50:32.371 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "subvol_clone77745" 2024-09-16T15:50:32.371 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:50:32.371 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:50:32.371 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "subvol_clone23331" 2024-09-16T15:50:32.371 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:50:32.372 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:50:32.372 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "subvol_clone446916" 2024-09-16T15:50:32.372 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:50:32.372 INFO:teuthology.orchestra.run.smithi151.stdout:] 2024-09-16T15:50:32.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.367+0000 7f228d400640 1 -- 172.21.15.151:0/2897911650 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2264079db0 msgr2=0x7f226407c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:32.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.367+0000 7f228d400640 1 --2- 172.21.15.151:0/2897911650 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2264079db0 0x7f226407c270 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2278034000 tx=0x7f2278004ab0 comp rx=0 tx=0).stop 2024-09-16T15:50:32.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.367+0000 7f228d400640 1 -- 172.21.15.151:0/2897911650 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2288145750 msgr2=0x7f22881ea110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:32.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.367+0000 7f228d400640 1 --2- 172.21.15.151:0/2897911650 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2288145750 0x7f22881ea110 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f228000c410 tx=0x7f228000c8d0 comp rx=0 tx=0).stop 2024-09-16T15:50:32.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.368+0000 7f228d400640 1 -- 172.21.15.151:0/2897911650 shutdown_connections 2024-09-16T15:50:32.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.368+0000 7f228d400640 1 --2- 172.21.15.151:0/2897911650 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2264079db0 0x7f226407c270 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:32.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.368+0000 7f228d400640 1 --2- 172.21.15.151:0/2897911650 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2288145750 0x7f22881ea110 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:32.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.368+0000 7f228d400640 1 --2- 172.21.15.151:0/2897911650 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f22880cccd0 0x7f22881e9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:32.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.368+0000 7f228d400640 1 --2- 172.21.15.151:0/2897911650 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f228806efa0 0x7f228814e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:32.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.368+0000 7f228d400640 1 -- 172.21.15.151:0/2897911650 >> 172.21.15.151:0/2897911650 conn(0x7f22880b1840 msgr2=0x7f228806f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:32.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.368+0000 7f228d400640 1 -- 172.21.15.151:0/2897911650 shutdown_connections 2024-09-16T15:50:32.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.368+0000 7f228d400640 1 -- 172.21.15.151:0/2897911650 wait complete. 2024-09-16T15:50:32.385 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone13712 2024-09-16T15:50:32.549 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.548+0000 7fe416400640 1 -- 172.21.15.151:0/665251693 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe41015bda0 msgr2=0x7fe410073010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:32.549 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.548+0000 7fe416400640 1 --2- 172.21.15.151:0/665251693 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe41015bda0 0x7fe410073010 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fe404009830 tx=0x7fe40402a480 comp rx=0 tx=0).stop 2024-09-16T15:50:32.549 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.548+0000 7fe416400640 1 -- 172.21.15.151:0/665251693 shutdown_connections 2024-09-16T15:50:32.549 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.548+0000 7fe416400640 1 --2- 172.21.15.151:0/665251693 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe410070550 0x7fe4100cb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:32.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.548+0000 7fe416400640 1 --2- 172.21.15.151:0/665251693 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe41015bda0 0x7fe410073010 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:32.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.548+0000 7fe416400640 1 -- 172.21.15.151:0/665251693 >> 172.21.15.151:0/665251693 conn(0x7fe4100cfa70 msgr2=0x7fe4100cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:32.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.548+0000 7fe416400640 1 -- 172.21.15.151:0/665251693 shutdown_connections 2024-09-16T15:50:32.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.548+0000 7fe416400640 1 -- 172.21.15.151:0/665251693 wait complete. 2024-09-16T15:50:32.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.550+0000 7fe416400640 1 Processor -- start 2024-09-16T15:50:32.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.550+0000 7fe416400640 1 -- start start 2024-09-16T15:50:32.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.551+0000 7fe416400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe41006e590 0x7fe4101f5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:32.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.551+0000 7fe416400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe410070550 0x7fe41015ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:32.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.551+0000 7fe416400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe41015bda0 0x7fe41015d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:32.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.551+0000 7fe416400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe4100cba90 con 0x7fe41006e590 2024-09-16T15:50:32.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.551+0000 7fe416400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe4100cb910 con 0x7fe410070550 2024-09-16T15:50:32.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.551+0000 7fe416400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fe4100cbc10 con 0x7fe41015bda0 2024-09-16T15:50:32.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.551+0000 7fe415000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe41006e590 0x7fe4101f5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:32.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.551+0000 7fe415000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe41006e590 0x7fe4101f5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:35386/0 (socket says 172.21.15.151:35386) 2024-09-16T15:50:32.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.551+0000 7fe415000640 1 -- 172.21.15.151:0/111732982 learned_addr learned my addr 172.21.15.151:0/111732982 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:32.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.551+0000 7fe415a00640 1 --2- 172.21.15.151:0/111732982 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe41015bda0 0x7fe41015d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:32.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.551+0000 7fe40fe00640 1 --2- 172.21.15.151:0/111732982 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe410070550 0x7fe41015ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:32.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.551+0000 7fe415000640 1 -- 172.21.15.151:0/111732982 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe41015bda0 msgr2=0x7fe41015d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:32.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.551+0000 7fe415000640 1 --2- 172.21.15.151:0/111732982 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe41015bda0 0x7fe41015d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:32.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.551+0000 7fe415000640 1 -- 172.21.15.151:0/111732982 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe410070550 msgr2=0x7fe41015ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:32.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.551+0000 7fe415000640 1 --2- 172.21.15.151:0/111732982 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe410070550 0x7fe41015ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:32.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.551+0000 7fe415000640 1 -- 172.21.15.151:0/111732982 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe41015c4a0 con 0x7fe41006e590 2024-09-16T15:50:32.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.551+0000 7fe415a00640 1 --2- 172.21.15.151:0/111732982 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe41015bda0 0x7fe41015d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:50:32.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.552+0000 7fe415000640 1 --2- 172.21.15.151:0/111732982 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe41006e590 0x7fe4101f5e10 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fe404000c00 tx=0x7fe404033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:32.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.552+0000 7fe40cc00640 1 -- 172.21.15.151:0/111732982 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe404031d60 con 0x7fe41006e590 2024-09-16T15:50:32.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.552+0000 7fe40cc00640 1 -- 172.21.15.151:0/111732982 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe404033e10 con 0x7fe41006e590 2024-09-16T15:50:32.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.552+0000 7fe40cc00640 1 -- 172.21.15.151:0/111732982 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4040332b0 con 0x7fe41006e590 2024-09-16T15:50:32.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.552+0000 7fe416400640 1 -- 172.21.15.151:0/111732982 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe41015c780 con 0x7fe41006e590 2024-09-16T15:50:32.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.552+0000 7fe416400640 1 -- 172.21.15.151:0/111732982 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe41015cd40 con 0x7fe41006e590 2024-09-16T15:50:32.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.553+0000 7fe416400640 1 -- 172.21.15.151:0/111732982 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4100cc960 con 0x7fe41006e590 2024-09-16T15:50:32.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.554+0000 7fe40cc00640 1 -- 172.21.15.151:0/111732982 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 36) ==== 103225+0+0 (secure 0 0 0) 0x7fe404033450 con 0x7fe41006e590 2024-09-16T15:50:32.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.555+0000 7fe40cc00640 1 --2- 172.21.15.151:0/111732982 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe3f0079db0 0x7fe3f007c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:32.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.555+0000 7fe40cc00640 1 -- 172.21.15.151:0/111732982 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7fe4040bad10 con 0x7fe41006e590 2024-09-16T15:50:32.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.555+0000 7fe40fe00640 1 --2- 172.21.15.151:0/111732982 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe3f0079db0 0x7fe3f007c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:32.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.556+0000 7fe40fe00640 1 --2- 172.21.15.151:0/111732982 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe3f0079db0 0x7fe3f007c270 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fe4100aebc0 tx=0x7fe3fc009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:32.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.559+0000 7fe40cc00640 1 -- 172.21.15.151:0/111732982 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe40403f030 con 0x7fe41006e590 2024-09-16T15:50:32.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.776+0000 7fe416400640 1 -- 172.21.15.151:0/111732982 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone13712", "target": ["mon-mgr", ""]}) -- 0x7fe41015a1c0 con 0x7fe3f0079db0 2024-09-16T15:50:32.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.796+0000 7fe40cc00640 1 -- 172.21.15.151:0/111732982 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe41015a1c0 con 0x7fe3f0079db0 2024-09-16T15:50:32.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.798+0000 7fe416400640 1 -- 172.21.15.151:0/111732982 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe3f0079db0 msgr2=0x7fe3f007c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:32.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.798+0000 7fe416400640 1 --2- 172.21.15.151:0/111732982 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe3f0079db0 0x7fe3f007c270 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fe4100aebc0 tx=0x7fe3fc009040 comp rx=0 tx=0).stop 2024-09-16T15:50:32.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.798+0000 7fe416400640 1 -- 172.21.15.151:0/111732982 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe41006e590 msgr2=0x7fe4101f5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:32.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.798+0000 7fe416400640 1 --2- 172.21.15.151:0/111732982 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe41006e590 0x7fe4101f5e10 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fe404000c00 tx=0x7fe404033c20 comp rx=0 tx=0).stop 2024-09-16T15:50:32.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.798+0000 7fe416400640 1 -- 172.21.15.151:0/111732982 shutdown_connections 2024-09-16T15:50:32.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.798+0000 7fe416400640 1 --2- 172.21.15.151:0/111732982 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe3f0079db0 0x7fe3f007c270 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:32.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.798+0000 7fe416400640 1 --2- 172.21.15.151:0/111732982 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe410070550 0x7fe41015ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:32.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.798+0000 7fe416400640 1 --2- 172.21.15.151:0/111732982 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe41015bda0 0x7fe41015d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:32.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.798+0000 7fe416400640 1 --2- 172.21.15.151:0/111732982 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe41006e590 0x7fe4101f5e10 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:32.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.798+0000 7fe416400640 1 -- 172.21.15.151:0/111732982 >> 172.21.15.151:0/111732982 conn(0x7fe4100cfa70 msgr2=0x7fe4100b2a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:32.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.799+0000 7fe416400640 1 -- 172.21.15.151:0/111732982 shutdown_connections 2024-09-16T15:50:32.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.799+0000 7fe416400640 1 -- 172.21.15.151:0/111732982 wait complete. 2024-09-16T15:50:32.801 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:50:32.799+0000 7f48f3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:50:32.801 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:50:32.799+0000 7f48f3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:50:32.801 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:50:32.799+0000 7f48f3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:50:32.801 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:50:32.799+0000 7f48f3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:50:32.801 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:50:32.799+0000 7f48f3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:50:32.816 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone499718 2024-09-16T15:50:32.817 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:50:32.815+0000 7f48f3a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:50:32.817 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:50:32.815+0000 7f48f3a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:50:32.817 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:50:32.815+0000 7f48f3a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:50:32.817 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:50:32.815+0000 7f48f3a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:50:32.817 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:50:32.815+0000 7f48f3a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:50:32.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.983+0000 7f9cf9e00640 1 -- 172.21.15.151:0/3633568076 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9cf406f220 msgr2=0x7f9cf406f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:32.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.983+0000 7f9cf9e00640 1 --2- 172.21.15.151:0/3633568076 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9cf406f220 0x7f9cf406f6b0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f9cec005fd0 tx=0x7f9cec02a8b0 comp rx=0 tx=0).stop 2024-09-16T15:50:32.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.983+0000 7f9cf9e00640 1 -- 172.21.15.151:0/3633568076 shutdown_connections 2024-09-16T15:50:32.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.983+0000 7f9cf9e00640 1 --2- 172.21.15.151:0/3633568076 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9cf406f220 0x7f9cf406f6b0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:32.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.983+0000 7f9cf9e00640 1 -- 172.21.15.151:0/3633568076 >> 172.21.15.151:0/3633568076 conn(0x7f9cf40b18e0 msgr2=0x7f9cf40b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:32.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.983+0000 7f9cf9e00640 1 -- 172.21.15.151:0/3633568076 shutdown_connections 2024-09-16T15:50:32.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.983+0000 7f9cf9e00640 1 -- 172.21.15.151:0/3633568076 wait complete. 2024-09-16T15:50:32.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.984+0000 7f9cf9e00640 1 Processor -- start 2024-09-16T15:50:32.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.985+0000 7f9cf9e00640 1 -- start start 2024-09-16T15:50:32.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.985+0000 7f9cf9e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9cf406f220 0x7f9cf4149610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:32.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.985+0000 7f9cf9e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cf4159aa0 0x7f9cf4154350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:32.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.985+0000 7f9cf9e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9cf415cff0 0x7f9cf4154890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:32.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.986+0000 7f9cf9e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf40cc740 con 0x7f9cf406f220 2024-09-16T15:50:32.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.986+0000 7f9cf9e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf40cc5c0 con 0x7f9cf4159aa0 2024-09-16T15:50:32.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.986+0000 7f9cf9e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f9cf40cc8c0 con 0x7f9cf415cff0 2024-09-16T15:50:32.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.986+0000 7f9cf2a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9cf406f220 0x7f9cf4149610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:32.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.986+0000 7f9cf2a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9cf406f220 0x7f9cf4149610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53834/0 (socket says 172.21.15.151:53834) 2024-09-16T15:50:32.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.986+0000 7f9cf2a00640 1 -- 172.21.15.151:0/3637583531 learned_addr learned my addr 172.21.15.151:0/3637583531 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:32.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.986+0000 7f9cf2000640 1 --2- 172.21.15.151:0/3637583531 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cf4159aa0 0x7f9cf4154350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:32.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.986+0000 7f9cf3400640 1 --2- 172.21.15.151:0/3637583531 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9cf415cff0 0x7f9cf4154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:32.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.986+0000 7f9cf2a00640 1 -- 172.21.15.151:0/3637583531 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9cf415cff0 msgr2=0x7f9cf4154890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:32.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.986+0000 7f9cf2a00640 1 --2- 172.21.15.151:0/3637583531 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9cf415cff0 0x7f9cf4154890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:32.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.986+0000 7f9cf2a00640 1 -- 172.21.15.151:0/3637583531 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cf4159aa0 msgr2=0x7f9cf4154350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:32.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.986+0000 7f9cf2a00640 1 --2- 172.21.15.151:0/3637583531 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cf4159aa0 0x7f9cf4154350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:32.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.986+0000 7f9cf2a00640 1 -- 172.21.15.151:0/3637583531 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cf40cdb70 con 0x7f9cf406f220 2024-09-16T15:50:32.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.987+0000 7f9cf2a00640 1 --2- 172.21.15.151:0/3637583531 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9cf406f220 0x7f9cf4149610 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f9ce400c490 tx=0x7f9ce400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:32.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.987+0000 7f9ceb400640 1 -- 172.21.15.151:0/3637583531 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce4016020 con 0x7f9cf406f220 2024-09-16T15:50:32.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.987+0000 7f9ceb400640 1 -- 172.21.15.151:0/3637583531 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ce4012940 con 0x7f9cf406f220 2024-09-16T15:50:32.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.987+0000 7f9ceb400640 1 -- 172.21.15.151:0/3637583531 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce4012d90 con 0x7f9cf406f220 2024-09-16T15:50:32.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.987+0000 7f9cf9e00640 1 -- 172.21.15.151:0/3637583531 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cf40cde10 con 0x7f9cf406f220 2024-09-16T15:50:32.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.987+0000 7f9cf9e00640 1 -- 172.21.15.151:0/3637583531 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cf40ce3d0 con 0x7f9cf406f220 2024-09-16T15:50:32.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.988+0000 7f9cf9e00640 1 -- 172.21.15.151:0/3637583531 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cf406f6b0 con 0x7f9cf406f220 2024-09-16T15:50:32.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.989+0000 7f9ceb400640 1 -- 172.21.15.151:0/3637583531 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 36) ==== 103225+0+0 (secure 0 0 0) 0x7f9ce4019020 con 0x7f9cf406f220 2024-09-16T15:50:32.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.990+0000 7f9ceb400640 1 --2- 172.21.15.151:0/3637583531 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9cd4079de0 0x7f9cd407c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:32.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.990+0000 7f9ceb400640 1 -- 172.21.15.151:0/3637583531 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7f9ce4099500 con 0x7f9cf406f220 2024-09-16T15:50:32.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.990+0000 7f9cf2000640 1 --2- 172.21.15.151:0/3637583531 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9cd4079de0 0x7f9cd407c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:32.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.991+0000 7f9cf2000640 1 --2- 172.21.15.151:0/3637583531 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9cd4079de0 0x7f9cd407c2a0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4155610 tx=0x7f9ce0009060 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:32.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:32.994+0000 7f9ceb400640 1 -- 172.21.15.151:0/3637583531 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9ce4020030 con 0x7f9cf406f220 2024-09-16T15:50:33.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:33.211+0000 7f9cf9e00640 1 -- 172.21.15.151:0/3637583531 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone499718", "target": ["mon-mgr", ""]}) -- 0x7f9cf415a8c0 con 0x7f9cd4079de0 2024-09-16T15:50:33.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:33.875+0000 7f9ceb400640 1 -- 172.21.15.151:0/3637583531 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9cf415a8c0 con 0x7f9cd4079de0 2024-09-16T15:50:33.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:33.877+0000 7f9cf9e00640 1 -- 172.21.15.151:0/3637583531 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9cd4079de0 msgr2=0x7f9cd407c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:33.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:33.877+0000 7f9cf9e00640 1 --2- 172.21.15.151:0/3637583531 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9cd4079de0 0x7f9cd407c2a0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4155610 tx=0x7f9ce0009060 comp rx=0 tx=0).stop 2024-09-16T15:50:33.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:33.877+0000 7f9cf9e00640 1 -- 172.21.15.151:0/3637583531 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9cf406f220 msgr2=0x7f9cf4149610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:33.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:33.877+0000 7f9cf9e00640 1 --2- 172.21.15.151:0/3637583531 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9cf406f220 0x7f9cf4149610 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f9ce400c490 tx=0x7f9ce400c950 comp rx=0 tx=0).stop 2024-09-16T15:50:33.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:33.878+0000 7f9cf9e00640 1 -- 172.21.15.151:0/3637583531 shutdown_connections 2024-09-16T15:50:33.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:33.878+0000 7f9cf9e00640 1 --2- 172.21.15.151:0/3637583531 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9cd4079de0 0x7f9cd407c2a0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:33.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:33.878+0000 7f9cf9e00640 1 --2- 172.21.15.151:0/3637583531 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9cf4159aa0 0x7f9cf4154350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:33.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:33.878+0000 7f9cf9e00640 1 --2- 172.21.15.151:0/3637583531 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9cf415cff0 0x7f9cf4154890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:33.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:33.878+0000 7f9cf9e00640 1 --2- 172.21.15.151:0/3637583531 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9cf406f220 0x7f9cf4149610 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:33.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:33.878+0000 7f9cf9e00640 1 -- 172.21.15.151:0/3637583531 >> 172.21.15.151:0/3637583531 conn(0x7f9cf40b18e0 msgr2=0x7f9cf415b290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:33.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:33.878+0000 7f9cf9e00640 1 -- 172.21.15.151:0/3637583531 shutdown_connections 2024-09-16T15:50:33.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:33.878+0000 7f9cf9e00640 1 -- 172.21.15.151:0/3637583531 wait complete. 2024-09-16T15:50:33.898 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone55130 2024-09-16T15:50:34.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.060+0000 7f6001000640 1 -- 172.21.15.151:0/1921920209 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ffc15f9a0 msgr2=0x7f5ffc15fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:34.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.060+0000 7f6001000640 1 --2- 172.21.15.151:0/1921920209 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ffc15f9a0 0x7f5ffc15fd80 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f5fec009830 tx=0x7f5fec02a4d0 comp rx=0 tx=0).stop 2024-09-16T15:50:34.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.060+0000 7f6001000640 1 -- 172.21.15.151:0/1921920209 shutdown_connections 2024-09-16T15:50:34.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.060+0000 7f6001000640 1 --2- 172.21.15.151:0/1921920209 >> v2:172.21.15.167:3300/0 conn(0x7f5ffc06e6f0 0x7f5ffc06eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:34.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.060+0000 7f6001000640 1 --2- 172.21.15.151:0/1921920209 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ffc15f9a0 0x7f5ffc15fd80 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:34.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.060+0000 7f6001000640 1 -- 172.21.15.151:0/1921920209 >> 172.21.15.151:0/1921920209 conn(0x7f5ffc0b18e0 msgr2=0x7f5ffc0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:34.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.060+0000 7f6001000640 1 -- 172.21.15.151:0/1921920209 shutdown_connections 2024-09-16T15:50:34.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.060+0000 7f6001000640 1 -- 172.21.15.151:0/1921920209 wait complete. 2024-09-16T15:50:34.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.062+0000 7f6001000640 1 Processor -- start 2024-09-16T15:50:34.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.062+0000 7f6001000640 1 -- start start 2024-09-16T15:50:34.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.063+0000 7f6001000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5ffc06e6f0 0x7f5ffc1f5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:34.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.063+0000 7f6001000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ffc073de0 0x7f5ffc0d0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:34.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.063+0000 7f6001000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ffc15f9a0 0x7f5ffc15ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:34.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.063+0000 7f6001000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5ffc0cd550 con 0x7f5ffc15f9a0 2024-09-16T15:50:34.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.063+0000 7f6001000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5ffc0cd3d0 con 0x7f5ffc073de0 2024-09-16T15:50:34.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.063+0000 7f6001000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f5ffc06f0e0 con 0x7f5ffc06e6f0 2024-09-16T15:50:34.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.063+0000 7f5ffa800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ffc15f9a0 0x7f5ffc15ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:34.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.063+0000 7f5ffa800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ffc15f9a0 0x7f5ffc15ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53848/0 (socket says 172.21.15.151:53848) 2024-09-16T15:50:34.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.063+0000 7f5ffa800640 1 -- 172.21.15.151:0/3098672993 learned_addr learned my addr 172.21.15.151:0/3098672993 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:34.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.064+0000 7f5ff9400640 1 --2- 172.21.15.151:0/3098672993 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ffc073de0 0x7f5ffc0d0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:34.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.064+0000 7f5ff9e00640 1 --2- 172.21.15.151:0/3098672993 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5ffc06e6f0 0x7f5ffc1f5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:34.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.064+0000 7f5ffa800640 1 -- 172.21.15.151:0/3098672993 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5ffc06e6f0 msgr2=0x7f5ffc1f5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:34.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.064+0000 7f5ffa800640 1 --2- 172.21.15.151:0/3098672993 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5ffc06e6f0 0x7f5ffc1f5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:34.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.064+0000 7f5ffa800640 1 -- 172.21.15.151:0/3098672993 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ffc073de0 msgr2=0x7f5ffc0d0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:34.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.064+0000 7f5ffa800640 1 --2- 172.21.15.151:0/3098672993 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ffc073de0 0x7f5ffc0d0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:34.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.064+0000 7f5ffa800640 1 -- 172.21.15.151:0/3098672993 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ffc0cdc00 con 0x7f5ffc15f9a0 2024-09-16T15:50:34.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.064+0000 7f5ffa800640 1 --2- 172.21.15.151:0/3098672993 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ffc15f9a0 0x7f5ffc15ca90 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f5ff400eaa0 tx=0x7f5ff400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:34.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.064+0000 7f5ff2000640 1 -- 172.21.15.151:0/3098672993 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff400cdb0 con 0x7f5ffc15f9a0 2024-09-16T15:50:34.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.064+0000 7f5ff2000640 1 -- 172.21.15.151:0/3098672993 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ff4004c10 con 0x7f5ffc15f9a0 2024-09-16T15:50:34.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.064+0000 7f6001000640 1 -- 172.21.15.151:0/3098672993 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ffc0d0970 con 0x7f5ffc15f9a0 2024-09-16T15:50:34.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.064+0000 7f5ff2000640 1 -- 172.21.15.151:0/3098672993 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff4005020 con 0x7f5ffc15f9a0 2024-09-16T15:50:34.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.064+0000 7f6001000640 1 -- 172.21.15.151:0/3098672993 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ffc0ce180 con 0x7f5ffc15f9a0 2024-09-16T15:50:34.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.066+0000 7f6001000640 1 -- 172.21.15.151:0/3098672993 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ffc06a360 con 0x7f5ffc15f9a0 2024-09-16T15:50:34.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.067+0000 7f5ff2000640 1 -- 172.21.15.151:0/3098672993 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 36) ==== 103225+0+0 (secure 0 0 0) 0x7f5ff40051c0 con 0x7f5ffc15f9a0 2024-09-16T15:50:34.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.068+0000 7f5ff2000640 1 --2- 172.21.15.151:0/3098672993 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5fdc079db0 0x7f5fdc07c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:34.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.068+0000 7f5ff2000640 1 -- 172.21.15.151:0/3098672993 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7f5ff409a0e0 con 0x7f5ffc15f9a0 2024-09-16T15:50:34.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.068+0000 7f5ff9e00640 1 --2- 172.21.15.151:0/3098672993 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5fdc079db0 0x7f5fdc07c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:34.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.069+0000 7f5ff9e00640 1 --2- 172.21.15.151:0/3098672993 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5fdc079db0 0x7f5fdc07c270 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f5fec034000 tx=0x7f5fec004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:34.073 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.072+0000 7f5ff2000640 1 -- 172.21.15.151:0/3098672993 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f5ffc06a360 con 0x7f5ffc15f9a0 2024-09-16T15:50:34.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.283+0000 7f6001000640 1 -- 172.21.15.151:0/3098672993 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone55130", "target": ["mon-mgr", ""]}) -- 0x7f5ffc09dd00 con 0x7f5fdc079db0 2024-09-16T15:50:34.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.304+0000 7f5ff2000640 1 -- 172.21.15.151:0/3098672993 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5ffc09dd00 con 0x7f5fdc079db0 2024-09-16T15:50:34.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.306+0000 7f6001000640 1 -- 172.21.15.151:0/3098672993 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5fdc079db0 msgr2=0x7f5fdc07c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:34.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.306+0000 7f6001000640 1 --2- 172.21.15.151:0/3098672993 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5fdc079db0 0x7f5fdc07c270 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f5fec034000 tx=0x7f5fec004730 comp rx=0 tx=0).stop 2024-09-16T15:50:34.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.306+0000 7f6001000640 1 -- 172.21.15.151:0/3098672993 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ffc15f9a0 msgr2=0x7f5ffc15ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:34.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.307+0000 7f6001000640 1 --2- 172.21.15.151:0/3098672993 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ffc15f9a0 0x7f5ffc15ca90 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f5ff400eaa0 tx=0x7f5ff400ef60 comp rx=0 tx=0).stop 2024-09-16T15:50:34.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.307+0000 7f6001000640 1 -- 172.21.15.151:0/3098672993 shutdown_connections 2024-09-16T15:50:34.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.307+0000 7f6001000640 1 --2- 172.21.15.151:0/3098672993 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5fdc079db0 0x7f5fdc07c270 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:34.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.307+0000 7f6001000640 1 --2- 172.21.15.151:0/3098672993 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ffc073de0 0x7f5ffc0d0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:34.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.307+0000 7f6001000640 1 --2- 172.21.15.151:0/3098672993 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ffc15f9a0 0x7f5ffc15ca90 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:34.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.307+0000 7f6001000640 1 --2- 172.21.15.151:0/3098672993 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5ffc06e6f0 0x7f5ffc1f5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:34.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.307+0000 7f6001000640 1 -- 172.21.15.151:0/3098672993 >> 172.21.15.151:0/3098672993 conn(0x7f5ffc0b18e0 msgr2=0x7f5ffc0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:34.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.307+0000 7f6001000640 1 -- 172.21.15.151:0/3098672993 shutdown_connections 2024-09-16T15:50:34.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.307+0000 7f6001000640 1 -- 172.21.15.151:0/3098672993 wait complete. 2024-09-16T15:50:34.324 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone962717 2024-09-16T15:50:34.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.490+0000 7fe123c00640 1 -- 172.21.15.151:0/2533949662 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe11c15f9a0 msgr2=0x7fe11c15fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:34.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.490+0000 7fe123c00640 1 --2- 172.21.15.151:0/2533949662 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe11c15f9a0 0x7fe11c15fd80 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fe10c005fd0 tx=0x7fe10c02aa10 comp rx=0 tx=0).stop 2024-09-16T15:50:34.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.490+0000 7fe123c00640 1 -- 172.21.15.151:0/2533949662 shutdown_connections 2024-09-16T15:50:34.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.490+0000 7fe123c00640 1 --2- 172.21.15.151:0/2533949662 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe11c06e6f0 0x7fe11c06eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:34.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.490+0000 7fe123c00640 1 --2- 172.21.15.151:0/2533949662 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe11c15f9a0 0x7fe11c15fd80 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:34.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.490+0000 7fe123c00640 1 -- 172.21.15.151:0/2533949662 >> 172.21.15.151:0/2533949662 conn(0x7fe11c0b18e0 msgr2=0x7fe11c0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:34.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.490+0000 7fe123c00640 1 -- 172.21.15.151:0/2533949662 shutdown_connections 2024-09-16T15:50:34.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.490+0000 7fe123c00640 1 -- 172.21.15.151:0/2533949662 wait complete. 2024-09-16T15:50:34.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.491+0000 7fe123c00640 1 Processor -- start 2024-09-16T15:50:34.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.492+0000 7fe123c00640 1 -- start start 2024-09-16T15:50:34.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.492+0000 7fe123c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe11c06e6f0 0x7fe11c09b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:34.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.492+0000 7fe123c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe11c073de0 0x7fe11c0ceae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:34.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.493+0000 7fe123c00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe11c15f9a0 0x7fe11c0cf020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:34.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.493+0000 7fe123c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe11c06f0b0 con 0x7fe11c073de0 2024-09-16T15:50:34.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.493+0000 7fe123c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe11c0cd5e0 con 0x7fe11c06e6f0 2024-09-16T15:50:34.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.493+0000 7fe123c00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fe11c06f230 con 0x7fe11c15f9a0 2024-09-16T15:50:34.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.493+0000 7fe11be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe11c073de0 0x7fe11c0ceae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:34.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.493+0000 7fe11be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe11c073de0 0x7fe11c0ceae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53868/0 (socket says 172.21.15.151:53868) 2024-09-16T15:50:34.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.493+0000 7fe11be00640 1 -- 172.21.15.151:0/580453884 learned_addr learned my addr 172.21.15.151:0/580453884 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:34.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.493+0000 7fe121c00640 1 --2- 172.21.15.151:0/580453884 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe11c15f9a0 0x7fe11c0cf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:34.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.493+0000 7fe121200640 1 --2- 172.21.15.151:0/580453884 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe11c06e6f0 0x7fe11c09b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:34.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.493+0000 7fe11be00640 1 -- 172.21.15.151:0/580453884 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe11c15f9a0 msgr2=0x7fe11c0cf020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:34.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.493+0000 7fe11be00640 1 --2- 172.21.15.151:0/580453884 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe11c15f9a0 0x7fe11c0cf020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:34.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.493+0000 7fe11be00640 1 -- 172.21.15.151:0/580453884 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe11c06e6f0 msgr2=0x7fe11c09b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:34.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.493+0000 7fe11be00640 1 --2- 172.21.15.151:0/580453884 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe11c06e6f0 0x7fe11c09b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:34.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.493+0000 7fe11be00640 1 -- 172.21.15.151:0/580453884 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe11c09baf0 con 0x7fe11c073de0 2024-09-16T15:50:34.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.494+0000 7fe11be00640 1 --2- 172.21.15.151:0/580453884 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe11c073de0 0x7fe11c0ceae0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fe10800c450 tx=0x7fe10800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:34.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.494+0000 7fe118c00640 1 -- 172.21.15.151:0/580453884 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1080052d0 con 0x7fe11c073de0 2024-09-16T15:50:34.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.494+0000 7fe118c00640 1 -- 172.21.15.151:0/580453884 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe108005470 con 0x7fe11c073de0 2024-09-16T15:50:34.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.494+0000 7fe123c00640 1 -- 172.21.15.151:0/580453884 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe11c09c2d0 con 0x7fe11c073de0 2024-09-16T15:50:34.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.494+0000 7fe118c00640 1 -- 172.21.15.151:0/580453884 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe108012870 con 0x7fe11c073de0 2024-09-16T15:50:34.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.494+0000 7fe123c00640 1 -- 172.21.15.151:0/580453884 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe11c09c890 con 0x7fe11c073de0 2024-09-16T15:50:34.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.495+0000 7fe123c00640 1 -- 172.21.15.151:0/580453884 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe11c069360 con 0x7fe11c073de0 2024-09-16T15:50:34.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.496+0000 7fe118c00640 1 -- 172.21.15.151:0/580453884 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7fe1080035f0 con 0x7fe11c073de0 2024-09-16T15:50:34.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.497+0000 7fe118c00640 1 --2- 172.21.15.151:0/580453884 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe0fc079f00 0x7fe0fc07c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:34.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.497+0000 7fe118c00640 1 -- 172.21.15.151:0/580453884 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7fe108099070 con 0x7fe11c073de0 2024-09-16T15:50:34.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.497+0000 7fe121200640 1 --2- 172.21.15.151:0/580453884 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe0fc079f00 0x7fe0fc07c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:34.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.498+0000 7fe121200640 1 --2- 172.21.15.151:0/580453884 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe0fc079f00 0x7fe0fc07c3c0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fe10c035000 tx=0x7fe10c03b040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:34.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.501+0000 7fe118c00640 1 -- 172.21.15.151:0/580453884 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe108062af0 con 0x7fe11c073de0 2024-09-16T15:50:34.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.713+0000 7fe123c00640 1 -- 172.21.15.151:0/580453884 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone962717", "target": ["mon-mgr", ""]}) -- 0x7fe11c06a360 con 0x7fe0fc079f00 2024-09-16T15:50:34.739 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.738+0000 7fe118c00640 1 -- 172.21.15.151:0/580453884 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe11c06a360 con 0x7fe0fc079f00 2024-09-16T15:50:34.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.741+0000 7fe123c00640 1 -- 172.21.15.151:0/580453884 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe0fc079f00 msgr2=0x7fe0fc07c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:34.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.741+0000 7fe123c00640 1 --2- 172.21.15.151:0/580453884 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe0fc079f00 0x7fe0fc07c3c0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fe10c035000 tx=0x7fe10c03b040 comp rx=0 tx=0).stop 2024-09-16T15:50:34.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.741+0000 7fe123c00640 1 -- 172.21.15.151:0/580453884 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe11c073de0 msgr2=0x7fe11c0ceae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:34.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.741+0000 7fe123c00640 1 --2- 172.21.15.151:0/580453884 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe11c073de0 0x7fe11c0ceae0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fe10800c450 tx=0x7fe10800c910 comp rx=0 tx=0).stop 2024-09-16T15:50:34.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.741+0000 7fe123c00640 1 -- 172.21.15.151:0/580453884 shutdown_connections 2024-09-16T15:50:34.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.741+0000 7fe123c00640 1 --2- 172.21.15.151:0/580453884 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe0fc079f00 0x7fe0fc07c3c0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:34.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.741+0000 7fe123c00640 1 --2- 172.21.15.151:0/580453884 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe11c15f9a0 0x7fe11c0cf020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:34.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.741+0000 7fe123c00640 1 --2- 172.21.15.151:0/580453884 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe11c073de0 0x7fe11c0ceae0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:34.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.741+0000 7fe123c00640 1 --2- 172.21.15.151:0/580453884 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe11c06e6f0 0x7fe11c09b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:34.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.741+0000 7fe123c00640 1 -- 172.21.15.151:0/580453884 >> 172.21.15.151:0/580453884 conn(0x7fe11c0b18e0 msgr2=0x7fe11c0aed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:34.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.741+0000 7fe123c00640 1 -- 172.21.15.151:0/580453884 shutdown_connections 2024-09-16T15:50:34.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.741+0000 7fe123c00640 1 -- 172.21.15.151:0/580453884 wait complete. 2024-09-16T15:50:34.761 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-09-16T15:50:34.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.924+0000 7f8fd2200640 1 -- 172.21.15.151:0/1919284299 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8fcc0b4350 msgr2=0x7f8fcc06f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:34.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.924+0000 7f8fd2200640 1 --2- 172.21.15.151:0/1919284299 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8fcc0b4350 0x7f8fcc06f3e0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4009830 tx=0x7f8fb402a540 comp rx=0 tx=0).stop 2024-09-16T15:50:34.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.924+0000 7f8fd2200640 1 -- 172.21.15.151:0/1919284299 shutdown_connections 2024-09-16T15:50:34.926 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.924+0000 7f8fd2200640 1 --2- 172.21.15.151:0/1919284299 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8fcc0b4350 0x7f8fcc06f3e0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:34.926 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.924+0000 7f8fd2200640 1 -- 172.21.15.151:0/1919284299 >> 172.21.15.151:0/1919284299 conn(0x7f8fcc0b18e0 msgr2=0x7f8fcc0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:34.926 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.924+0000 7f8fd2200640 1 -- 172.21.15.151:0/1919284299 shutdown_connections 2024-09-16T15:50:34.926 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.924+0000 7f8fd2200640 1 -- 172.21.15.151:0/1919284299 wait complete. 2024-09-16T15:50:34.926 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.925+0000 7f8fd2200640 1 Processor -- start 2024-09-16T15:50:34.926 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.926+0000 7f8fd2200640 1 -- start start 2024-09-16T15:50:34.928 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.926+0000 7f8fd2200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8fcc06f920 0x7f8fcc09d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:34.928 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.927+0000 7f8fd2200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fcc0b4350 0x7f8fcc152990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:34.928 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.927+0000 7f8fd2200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8fcc1550c0 0x7f8fcc152ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:34.928 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.927+0000 7f8fd2200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8fcc15ed20 con 0x7f8fcc0b4350 2024-09-16T15:50:34.928 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.927+0000 7f8fd2200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8fcc15eba0 con 0x7f8fcc06f920 2024-09-16T15:50:34.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.927+0000 7f8fd2200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f8fcc15eea0 con 0x7f8fcc1550c0 2024-09-16T15:50:34.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.927+0000 7f8fc2a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fcc0b4350 0x7f8fcc152990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:34.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.927+0000 7f8fc2a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fcc0b4350 0x7f8fcc152990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53876/0 (socket says 172.21.15.151:53876) 2024-09-16T15:50:34.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.927+0000 7f8fc2a00640 1 -- 172.21.15.151:0/1328332124 learned_addr learned my addr 172.21.15.151:0/1328332124 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:34.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.927+0000 7f8fcbe00640 1 --2- 172.21.15.151:0/1328332124 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8fcc1550c0 0x7f8fcc152ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:34.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.927+0000 7f8fcb400640 1 --2- 172.21.15.151:0/1328332124 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8fcc06f920 0x7f8fcc09d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:34.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.927+0000 7f8fc2a00640 1 -- 172.21.15.151:0/1328332124 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8fcc1550c0 msgr2=0x7f8fcc152ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:34.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.927+0000 7f8fc2a00640 1 --2- 172.21.15.151:0/1328332124 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8fcc1550c0 0x7f8fcc152ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:34.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.927+0000 7f8fc2a00640 1 -- 172.21.15.151:0/1328332124 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8fcc06f920 msgr2=0x7f8fcc09d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:34.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.927+0000 7f8fc2a00640 1 --2- 172.21.15.151:0/1328332124 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8fcc06f920 0x7f8fcc09d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:34.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.927+0000 7f8fc2a00640 1 -- 172.21.15.151:0/1328332124 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fcc09d910 con 0x7f8fcc0b4350 2024-09-16T15:50:34.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.928+0000 7f8fc2a00640 1 --2- 172.21.15.151:0/1328332124 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fcc0b4350 0x7f8fcc152990 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4009950 tx=0x7f8fb4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:34.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.928+0000 7f8fc3e00640 1 -- 172.21.15.151:0/1328332124 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb4005c90 con 0x7f8fcc0b4350 2024-09-16T15:50:34.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.928+0000 7f8fc3e00640 1 -- 172.21.15.151:0/1328332124 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8fb4005020 con 0x7f8fcc0b4350 2024-09-16T15:50:34.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.928+0000 7f8fd2200640 1 -- 172.21.15.151:0/1328332124 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fcc153c80 con 0x7f8fcc0b4350 2024-09-16T15:50:34.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.928+0000 7f8fc3e00640 1 -- 172.21.15.151:0/1328332124 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb4005430 con 0x7f8fcc0b4350 2024-09-16T15:50:34.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.928+0000 7f8fd2200640 1 -- 172.21.15.151:0/1328332124 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fcc15f5b0 con 0x7f8fcc0b4350 2024-09-16T15:50:34.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.929+0000 7f8fd2200640 1 -- 172.21.15.151:0/1328332124 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fcc069360 con 0x7f8fcc0b4350 2024-09-16T15:50:34.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.930+0000 7f8fc3e00640 1 -- 172.21.15.151:0/1328332124 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7f8fb4039040 con 0x7f8fcc0b4350 2024-09-16T15:50:34.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.932+0000 7f8fc3e00640 1 --2- 172.21.15.151:0/1328332124 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8fa8079f00 0x7f8fa807c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:34.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.932+0000 7f8fc3e00640 1 -- 172.21.15.151:0/1328332124 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7f8fb40ba500 con 0x7f8fcc0b4350 2024-09-16T15:50:34.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.932+0000 7f8fcb400640 1 --2- 172.21.15.151:0/1328332124 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8fa8079f00 0x7f8fa807c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:34.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.933+0000 7f8fcb400640 1 --2- 172.21.15.151:0/1328332124 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8fa8079f00 0x7f8fa807c3c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc001a00 tx=0x7f8fbc001990 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:34.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:34.936+0000 7f8fc3e00640 1 -- 172.21.15.151:0/1328332124 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8fb4040030 con 0x7f8fcc0b4350 2024-09-16T15:50:35.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.167+0000 7f8fd2200640 1 -- 172.21.15.151:0/1328332124 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8fcc06a360 con 0x7f8fa8079f00 2024-09-16T15:50:35.194 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:50:35.195 INFO:teuthology.orchestra.run.smithi151.stdout:[ 2024-09-16T15:50:35.195 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:50:35.195 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ss1" 2024-09-16T15:50:35.195 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:50:35.195 INFO:teuthology.orchestra.run.smithi151.stdout:] 2024-09-16T15:50:35.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.193+0000 7f8fc3e00640 1 -- 172.21.15.151:0/1328332124 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f8fcc06a360 con 0x7f8fa8079f00 2024-09-16T15:50:35.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.196+0000 7f8fd2200640 1 -- 172.21.15.151:0/1328332124 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8fa8079f00 msgr2=0x7f8fa807c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:35.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.196+0000 7f8fd2200640 1 --2- 172.21.15.151:0/1328332124 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8fa8079f00 0x7f8fa807c3c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc001a00 tx=0x7f8fbc001990 comp rx=0 tx=0).stop 2024-09-16T15:50:35.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.196+0000 7f8fd2200640 1 -- 172.21.15.151:0/1328332124 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fcc0b4350 msgr2=0x7f8fcc152990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:35.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.196+0000 7f8fd2200640 1 --2- 172.21.15.151:0/1328332124 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fcc0b4350 0x7f8fcc152990 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4009950 tx=0x7f8fb4003c10 comp rx=0 tx=0).stop 2024-09-16T15:50:35.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.197+0000 7f8fd2200640 1 -- 172.21.15.151:0/1328332124 shutdown_connections 2024-09-16T15:50:35.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.197+0000 7f8fd2200640 1 --2- 172.21.15.151:0/1328332124 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8fa8079f00 0x7f8fa807c3c0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:35.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.197+0000 7f8fd2200640 1 --2- 172.21.15.151:0/1328332124 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8fcc1550c0 0x7f8fcc152ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:35.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.197+0000 7f8fd2200640 1 --2- 172.21.15.151:0/1328332124 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fcc0b4350 0x7f8fcc152990 secure :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4009950 tx=0x7f8fb4003c10 comp rx=0 tx=0).stop 2024-09-16T15:50:35.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.197+0000 7f8fd2200640 1 --2- 172.21.15.151:0/1328332124 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8fcc06f920 0x7f8fcc09d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:35.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.197+0000 7f8fd2200640 1 -- 172.21.15.151:0/1328332124 >> 172.21.15.151:0/1328332124 conn(0x7f8fcc0b18e0 msgr2=0x7f8fcc0727c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:35.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.197+0000 7f8fd2200640 1 -- 172.21.15.151:0/1328332124 shutdown_connections 2024-09-16T15:50:35.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.197+0000 7f8fd2200640 1 -- 172.21.15.151:0/1328332124 wait complete. 2024-09-16T15:50:35.226 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-09-16T15:50:35.488 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.487+0000 7f27d2800640 1 -- 172.21.15.151:0/3483430950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27cc1437c0 msgr2=0x7f27cc0cdad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:35.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.487+0000 7f27d2800640 1 --2- 172.21.15.151:0/3483430950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27cc1437c0 0x7f27cc0cdad0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f27c400afd0 tx=0x7f27c402b5b0 comp rx=0 tx=0).stop 2024-09-16T15:50:35.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.487+0000 7f27d2800640 1 -- 172.21.15.151:0/3483430950 shutdown_connections 2024-09-16T15:50:35.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.487+0000 7f27d2800640 1 --2- 172.21.15.151:0/3483430950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27cc1437c0 0x7f27cc0cdad0 secure :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f27c400afd0 tx=0x7f27c402b5b0 comp rx=0 tx=0).stop 2024-09-16T15:50:35.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.487+0000 7f27d2800640 1 -- 172.21.15.151:0/3483430950 >> 172.21.15.151:0/3483430950 conn(0x7f27cc0b1840 msgr2=0x7f27cc0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:35.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.490+0000 7f27d2800640 1 -- 172.21.15.151:0/3483430950 shutdown_connections 2024-09-16T15:50:35.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.490+0000 7f27d2800640 1 -- 172.21.15.151:0/3483430950 wait complete. 2024-09-16T15:50:35.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.492+0000 7f27d2800640 1 Processor -- start 2024-09-16T15:50:35.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.492+0000 7f27d2800640 1 -- start start 2024-09-16T15:50:35.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.493+0000 7f27d2800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27cc0b45d0 0x7f27cc13df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:35.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.493+0000 7f27d2800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f27cc0b4bd0 0x7f27cc153620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:35.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.493+0000 7f27d2800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27cc138780 0x7f27cc140750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:35.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.493+0000 7f27d2800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f27cc06f0f0 con 0x7f27cc0b4bd0 2024-09-16T15:50:35.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.493+0000 7f27d2800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f27cc06ef70 con 0x7f27cc138780 2024-09-16T15:50:35.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.493+0000 7f27d2800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f27cc06f270 con 0x7f27cc0b45d0 2024-09-16T15:50:35.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.494+0000 7f27cb400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27cc0b45d0 0x7f27cc13df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:35.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.494+0000 7f27cb400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27cc0b45d0 0x7f27cc13df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.151:39834/0 (socket says 172.21.15.151:39834) 2024-09-16T15:50:35.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.494+0000 7f27cb400640 1 -- 172.21.15.151:0/3881984801 learned_addr learned my addr 172.21.15.151:0/3881984801 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:35.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.494+0000 7f27cbe00640 1 --2- 172.21.15.151:0/3881984801 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27cc138780 0x7f27cc140750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:35.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.494+0000 7f27caa00640 1 --2- 172.21.15.151:0/3881984801 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f27cc0b4bd0 0x7f27cc153620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:35.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.495+0000 7f27cbe00640 1 -- 172.21.15.151:0/3881984801 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27cc0b45d0 msgr2=0x7f27cc13df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:35.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.495+0000 7f27cbe00640 1 --2- 172.21.15.151:0/3881984801 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27cc0b45d0 0x7f27cc13df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:35.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.495+0000 7f27cbe00640 1 -- 172.21.15.151:0/3881984801 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f27cc0b4bd0 msgr2=0x7f27cc153620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:35.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.495+0000 7f27cbe00640 1 --2- 172.21.15.151:0/3881984801 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f27cc0b4bd0 0x7f27cc153620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:35.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.496+0000 7f27cbe00640 1 -- 172.21.15.151:0/3881984801 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27cc13d350 con 0x7f27cc138780 2024-09-16T15:50:35.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.496+0000 7f27caa00640 1 --2- 172.21.15.151:0/3881984801 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f27cc0b4bd0 0x7f27cc153620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:50:35.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.496+0000 7f27cbe00640 1 --2- 172.21.15.151:0/3881984801 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27cc138780 0x7f27cc140750 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f27bc00c340 tx=0x7f27bc00c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:35.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.497+0000 7f27c3400640 1 -- 172.21.15.151:0/3881984801 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27bc011790 con 0x7f27cc138780 2024-09-16T15:50:35.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.497+0000 7f27c3400640 1 -- 172.21.15.151:0/3881984801 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f27bc017070 con 0x7f27cc138780 2024-09-16T15:50:35.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.497+0000 7f27c3400640 1 -- 172.21.15.151:0/3881984801 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27bc011d20 con 0x7f27cc138780 2024-09-16T15:50:35.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.497+0000 7f27d2800640 1 -- 172.21.15.151:0/3881984801 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27cc13cd90 con 0x7f27cc138780 2024-09-16T15:50:35.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.497+0000 7f27d2800640 1 -- 172.21.15.151:0/3881984801 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27cc153d50 con 0x7f27cc138780 2024-09-16T15:50:35.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.499+0000 7f27d2800640 1 -- 172.21.15.151:0/3881984801 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27a4005290 con 0x7f27cc138780 2024-09-16T15:50:35.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.501+0000 7f27c3400640 1 -- 172.21.15.151:0/3881984801 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7f27bc01c460 con 0x7f27cc138780 2024-09-16T15:50:35.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.503+0000 7f27c3400640 1 --2- 172.21.15.151:0/3881984801 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f27ac079fa0 0x7f27ac07c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:35.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.503+0000 7f27c3400640 1 -- 172.21.15.151:0/3881984801 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7f27bc099230 con 0x7f27cc138780 2024-09-16T15:50:35.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.504+0000 7f27cb400640 1 --2- 172.21.15.151:0/3881984801 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f27ac079fa0 0x7f27ac07c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:35.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.506+0000 7f27cb400640 1 --2- 172.21.15.151:0/3881984801 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f27ac079fa0 0x7f27ac07c460 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f27c4007510 tx=0x7f27c4004ab0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:35.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.509+0000 7f27c3400640 1 -- 172.21.15.151:0/3881984801 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f27bc063380 con 0x7f27cc138780 2024-09-16T15:50:35.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.761+0000 7f27d2800640 1 -- 172.21.15.151:0/3881984801 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f27a4000c70 con 0x7f27ac079fa0 2024-09-16T15:50:35.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.789+0000 7f27c3400640 1 -- 172.21.15.151:0/3881984801 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f27a4000c70 con 0x7f27ac079fa0 2024-09-16T15:50:35.790 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:50:35.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.791+0000 7f27d2800640 1 -- 172.21.15.151:0/3881984801 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f27ac079fa0 msgr2=0x7f27ac07c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:35.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.791+0000 7f27d2800640 1 --2- 172.21.15.151:0/3881984801 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f27ac079fa0 0x7f27ac07c460 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f27c4007510 tx=0x7f27c4004ab0 comp rx=0 tx=0).stop 2024-09-16T15:50:35.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.791+0000 7f27d2800640 1 -- 172.21.15.151:0/3881984801 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27cc138780 msgr2=0x7f27cc140750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:35.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.791+0000 7f27d2800640 1 --2- 172.21.15.151:0/3881984801 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27cc138780 0x7f27cc140750 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f27bc00c340 tx=0x7f27bc00c800 comp rx=0 tx=0).stop 2024-09-16T15:50:35.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.791+0000 7f27d2800640 1 -- 172.21.15.151:0/3881984801 shutdown_connections 2024-09-16T15:50:35.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.791+0000 7f27d2800640 1 --2- 172.21.15.151:0/3881984801 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f27ac079fa0 0x7f27ac07c460 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:35.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.791+0000 7f27d2800640 1 --2- 172.21.15.151:0/3881984801 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27cc138780 0x7f27cc140750 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:35.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.791+0000 7f27d2800640 1 --2- 172.21.15.151:0/3881984801 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f27cc0b4bd0 0x7f27cc153620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:35.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.791+0000 7f27d2800640 1 --2- 172.21.15.151:0/3881984801 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f27cc0b45d0 0x7f27cc13df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:35.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.791+0000 7f27d2800640 1 -- 172.21.15.151:0/3881984801 >> 172.21.15.151:0/3881984801 conn(0x7f27cc0b1840 msgr2=0x7f27cc15c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:35.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.791+0000 7f27d2800640 1 -- 172.21.15.151:0/3881984801 shutdown_connections 2024-09-16T15:50:35.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.791+0000 7f27d2800640 1 -- 172.21.15.151:0/3881984801 wait complete. 2024-09-16T15:50:35.809 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-09-16T15:50:35.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.972+0000 7f2d2c600640 1 -- 172.21.15.151:0/786242203 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d2406efa0 msgr2=0x7f2d240cc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:35.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.972+0000 7f2d2c600640 1 --2- 172.21.15.151:0/786242203 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d2406efa0 0x7f2d240cc5c0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f2d240d0cd0 tx=0x7f2d1002a4d0 comp rx=0 tx=0).stop 2024-09-16T15:50:35.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.972+0000 7f2d2c600640 1 -- 172.21.15.151:0/786242203 shutdown_connections 2024-09-16T15:50:35.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.972+0000 7f2d2c600640 1 --2- 172.21.15.151:0/786242203 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d240ccb00 0x7f2d2415f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:35.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.972+0000 7f2d2c600640 1 --2- 172.21.15.151:0/786242203 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d2406efa0 0x7f2d240cc5c0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:35.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.972+0000 7f2d2c600640 1 --2- 172.21.15.151:0/786242203 >> v2:172.21.15.167:3300/0 conn(0x7f2d24145750 0x7f2d24073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:35.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.972+0000 7f2d2c600640 1 -- 172.21.15.151:0/786242203 >> 172.21.15.151:0/786242203 conn(0x7f2d240b1840 msgr2=0x7f2d240b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:35.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.972+0000 7f2d2c600640 1 -- 172.21.15.151:0/786242203 shutdown_connections 2024-09-16T15:50:35.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.972+0000 7f2d2c600640 1 -- 172.21.15.151:0/786242203 wait complete. 2024-09-16T15:50:35.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.973+0000 7f2d2c600640 1 Processor -- start 2024-09-16T15:50:35.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.974+0000 7f2d2c600640 1 -- start start 2024-09-16T15:50:35.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.974+0000 7f2d2c600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d2406efa0 0x7f2d24140220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:35.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.974+0000 7f2d2c600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d240ccb00 0x7f2d241439b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:35.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.975+0000 7f2d2c600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d24145750 0x7f2d240cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:35.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.975+0000 7f2d2c600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2d241602c0 con 0x7f2d2406efa0 2024-09-16T15:50:35.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.975+0000 7f2d2c600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2d24160140 con 0x7f2d24145750 2024-09-16T15:50:35.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.975+0000 7f2d2c600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f2d24160440 con 0x7f2d240ccb00 2024-09-16T15:50:35.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.975+0000 7f2d29c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d2406efa0 0x7f2d24140220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:35.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.975+0000 7f2d29c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d2406efa0 0x7f2d24140220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53908/0 (socket says 172.21.15.151:53908) 2024-09-16T15:50:35.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.975+0000 7f2d29c00640 1 -- 172.21.15.151:0/3312865506 learned_addr learned my addr 172.21.15.151:0/3312865506 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:35.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.975+0000 7f2d2a600640 1 --2- 172.21.15.151:0/3312865506 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d24145750 0x7f2d240cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:35.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.975+0000 7f2d29200640 1 --2- 172.21.15.151:0/3312865506 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d240ccb00 0x7f2d241439b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:35.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.975+0000 7f2d29c00640 1 -- 172.21.15.151:0/3312865506 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d240ccb00 msgr2=0x7f2d241439b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:35.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.975+0000 7f2d29c00640 1 --2- 172.21.15.151:0/3312865506 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d240ccb00 0x7f2d241439b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:35.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.975+0000 7f2d29c00640 1 -- 172.21.15.151:0/3312865506 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d24145750 msgr2=0x7f2d240cfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:35.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.975+0000 7f2d29c00640 1 --2- 172.21.15.151:0/3312865506 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d24145750 0x7f2d240cfe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:35.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.975+0000 7f2d29c00640 1 -- 172.21.15.151:0/3312865506 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d241f4c50 con 0x7f2d2406efa0 2024-09-16T15:50:35.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.975+0000 7f2d29200640 1 --2- 172.21.15.151:0/3312865506 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d240ccb00 0x7f2d241439b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:50:35.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.975+0000 7f2d29c00640 1 --2- 172.21.15.151:0/3312865506 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d2406efa0 0x7f2d24140220 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f2d1800ea50 tx=0x7f2d1800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:35.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.976+0000 7f2d21600640 1 -- 172.21.15.151:0/3312865506 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1800cdc0 con 0x7f2d2406efa0 2024-09-16T15:50:35.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.976+0000 7f2d21600640 1 -- 172.21.15.151:0/3312865506 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2d18011560 con 0x7f2d2406efa0 2024-09-16T15:50:35.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.976+0000 7f2d21600640 1 -- 172.21.15.151:0/3312865506 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d18011990 con 0x7f2d2406efa0 2024-09-16T15:50:35.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.976+0000 7f2d2c600640 1 -- 172.21.15.151:0/3312865506 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d241f4f30 con 0x7f2d2406efa0 2024-09-16T15:50:35.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.976+0000 7f2d2c600640 1 -- 172.21.15.151:0/3312865506 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d241f54f0 con 0x7f2d2406efa0 2024-09-16T15:50:35.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.977+0000 7f2d2c600640 1 -- 172.21.15.151:0/3312865506 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d2415f9f0 con 0x7f2d2406efa0 2024-09-16T15:50:35.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.978+0000 7f2d21600640 1 -- 172.21.15.151:0/3312865506 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7f2d18011b30 con 0x7f2d2406efa0 2024-09-16T15:50:35.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.979+0000 7f2d21600640 1 --2- 172.21.15.151:0/3312865506 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2d04079ed0 0x7f2d0407c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:35.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.979+0000 7f2d21600640 1 -- 172.21.15.151:0/3312865506 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7f2d1809a2b0 con 0x7f2d2406efa0 2024-09-16T15:50:35.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.980+0000 7f2d29200640 1 --2- 172.21.15.151:0/3312865506 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2d04079ed0 0x7f2d0407c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:35.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.980+0000 7f2d29200640 1 --2- 172.21.15.151:0/3312865506 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2d04079ed0 0x7f2d0407c390 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f2d10033000 tx=0x7f2d10004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:35.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:35.983+0000 7f2d21600640 1 -- 172.21.15.151:0/3312865506 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2d18016030 con 0x7f2d2406efa0 2024-09-16T15:50:36.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:36.256+0000 7f2d2c600640 1 -- 172.21.15.151:0/3312865506 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f2d240b4570 con 0x7f2d04079ed0 2024-09-16T15:50:36.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:36.290+0000 7f2d21600640 1 -- 172.21.15.151:0/3312865506 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2d240b4570 con 0x7f2d04079ed0 2024-09-16T15:50:36.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:36.294+0000 7f2d2c600640 1 -- 172.21.15.151:0/3312865506 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2d04079ed0 msgr2=0x7f2d0407c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:36.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:36.294+0000 7f2d2c600640 1 --2- 172.21.15.151:0/3312865506 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2d04079ed0 0x7f2d0407c390 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f2d10033000 tx=0x7f2d10004730 comp rx=0 tx=0).stop 2024-09-16T15:50:36.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:36.294+0000 7f2d2c600640 1 -- 172.21.15.151:0/3312865506 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d2406efa0 msgr2=0x7f2d24140220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:36.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:36.294+0000 7f2d2c600640 1 --2- 172.21.15.151:0/3312865506 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d2406efa0 0x7f2d24140220 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f2d1800ea50 tx=0x7f2d1800ef10 comp rx=0 tx=0).stop 2024-09-16T15:50:36.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:36.294+0000 7f2d2c600640 1 -- 172.21.15.151:0/3312865506 shutdown_connections 2024-09-16T15:50:36.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:36.294+0000 7f2d2c600640 1 --2- 172.21.15.151:0/3312865506 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2d04079ed0 0x7f2d0407c390 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:36.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:36.294+0000 7f2d2c600640 1 --2- 172.21.15.151:0/3312865506 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d24145750 0x7f2d240cfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:36.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:36.294+0000 7f2d2c600640 1 --2- 172.21.15.151:0/3312865506 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d240ccb00 0x7f2d241439b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:36.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:36.294+0000 7f2d2c600640 1 --2- 172.21.15.151:0/3312865506 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d2406efa0 0x7f2d24140220 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:36.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:36.294+0000 7f2d2c600640 1 -- 172.21.15.151:0/3312865506 >> 172.21.15.151:0/3312865506 conn(0x7f2d240b1840 msgr2=0x7f2d240b3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:36.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:36.295+0000 7f2d2c600640 1 -- 172.21.15.151:0/3312865506 shutdown_connections 2024-09-16T15:50:36.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:36.295+0000 7f2d2c600640 1 -- 172.21.15.151:0/3312865506 wait complete. 2024-09-16T15:50:36.324 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone872410 2024-09-16T15:50:36.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:36.586+0000 7f6907800640 1 -- 172.21.15.151:0/88304448 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69000b3e20 msgr2=0x7f69000cf4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:36.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:36.586+0000 7f6907800640 1 --2- 172.21.15.151:0/88304448 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69000b3e20 0x7f69000cf4f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f6900151690 tx=0x7f68ec02a560 comp rx=0 tx=0).stop 2024-09-16T15:50:36.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:36.587+0000 7f6907800640 1 -- 172.21.15.151:0/88304448 shutdown_connections 2024-09-16T15:50:36.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:36.587+0000 7f6907800640 1 --2- 172.21.15.151:0/88304448 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69000b3e20 0x7f69000cf4f0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:36.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:36.587+0000 7f6907800640 1 --2- 172.21.15.151:0/88304448 >> v2:172.21.15.151:3300/0 conn(0x7f69001512b0 0x7f69000cefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:36.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:36.587+0000 7f6907800640 1 -- 172.21.15.151:0/88304448 >> 172.21.15.151:0/88304448 conn(0x7f69000b18e0 msgr2=0x7f69000b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:36.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:36.587+0000 7f6907800640 1 -- 172.21.15.151:0/88304448 shutdown_connections 2024-09-16T15:50:36.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:36.587+0000 7f6907800640 1 -- 172.21.15.151:0/88304448 wait complete. 2024-09-16T15:50:36.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:36.589+0000 7f6907800640 1 Processor -- start 2024-09-16T15:50:36.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:36.589+0000 7f6907800640 1 -- start start 2024-09-16T15:50:36.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:36.590+0000 7f6907800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f690006fa00 0x7f690015c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:36.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:36.590+0000 7f6907800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69000b3e20 0x7f6900144c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:36.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:36.590+0000 7f6907800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f69001512b0 0x7f69000738c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:36.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:36.590+0000 7f6907800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f69000cc6d0 con 0x7f69000b3e20 2024-09-16T15:50:36.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:36.590+0000 7f6907800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f69000cc550 con 0x7f690006fa00 2024-09-16T15:50:36.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:36.590+0000 7f6907800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f69000cc850 con 0x7f69001512b0 2024-09-16T15:50:36.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:36.591+0000 7f68ffe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69000b3e20 0x7f6900144c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:36.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:36.591+0000 7f68ffe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69000b3e20 0x7f6900144c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53932/0 (socket says 172.21.15.151:53932) 2024-09-16T15:50:36.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:36.591+0000 7f68ffe00640 1 -- 172.21.15.151:0/2371397359 learned_addr learned my addr 172.21.15.151:0/2371397359 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:36.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:36.592+0000 7f68ffe00640 1 -- 172.21.15.151:0/2371397359 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f69001512b0 msgr2=0x7f69000738c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:36.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:36.592+0000 7f68ffe00640 1 --2- 172.21.15.151:0/2371397359 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f69001512b0 0x7f69000738c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:36.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:36.592+0000 7f68ffe00640 1 -- 172.21.15.151:0/2371397359 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f690006fa00 msgr2=0x7f690015c2c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T15:50:36.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:36.592+0000 7f68ffe00640 1 --2- 172.21.15.151:0/2371397359 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f690006fa00 0x7f690015c2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:36.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:36.592+0000 7f68ffe00640 1 -- 172.21.15.151:0/2371397359 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6900073e00 con 0x7f69000b3e20 2024-09-16T15:50:36.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:36.592+0000 7f68ffe00640 1 --2- 172.21.15.151:0/2371397359 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69000b3e20 0x7f6900144c10 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f68ec009850 tx=0x7f68ec003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:36.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:36.593+0000 7f68fcc00640 1 -- 172.21.15.151:0/2371397359 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ec03f070 con 0x7f69000b3e20 2024-09-16T15:50:36.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:36.593+0000 7f68fcc00640 1 -- 172.21.15.151:0/2371397359 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f68ec0047a0 con 0x7f69000b3e20 2024-09-16T15:50:36.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:36.593+0000 7f68fcc00640 1 -- 172.21.15.151:0/2371397359 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ec035030 con 0x7f69000b3e20 2024-09-16T15:50:36.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:36.593+0000 7f6907800640 1 -- 172.21.15.151:0/2371397359 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69000740a0 con 0x7f69000b3e20 2024-09-16T15:50:36.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:36.593+0000 7f6907800640 1 -- 172.21.15.151:0/2371397359 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6900149810 con 0x7f69000b3e20 2024-09-16T15:50:36.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:36.595+0000 7f6907800640 1 -- 172.21.15.151:0/2371397359 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6900069360 con 0x7f69000b3e20 2024-09-16T15:50:36.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:36.596+0000 7f68fcc00640 1 -- 172.21.15.151:0/2371397359 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7f68ec042070 con 0x7f69000b3e20 2024-09-16T15:50:36.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:36.598+0000 7f68fcc00640 1 --2- 172.21.15.151:0/2371397359 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f68e0079f20 0x7f68e007c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:36.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:36.598+0000 7f68fcc00640 1 -- 172.21.15.151:0/2371397359 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7f68ec0ba4a0 con 0x7f69000b3e20 2024-09-16T15:50:36.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:36.599+0000 7f6904e00640 1 --2- 172.21.15.151:0/2371397359 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f68e0079f20 0x7f68e007c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:36.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:36.600+0000 7f6904e00640 1 --2- 172.21.15.151:0/2371397359 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f68e0079f20 0x7f68e007c3e0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f68f4005fd0 tx=0x7f68f400b2f0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:36.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:36.604+0000 7f68fcc00640 1 -- 172.21.15.151:0/2371397359 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f68ec0845f0 con 0x7f69000b3e20 2024-09-16T15:50:36.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:36.899+0000 7f6907800640 1 -- 172.21.15.151:0/2371397359 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone872410", "target": ["mon-mgr", ""]}) -- 0x7f69000cccd0 con 0x7f68e0079f20 2024-09-16T15:50:36.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:36.920+0000 7f68fcc00640 1 -- 172.21.15.151:0/2371397359 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f69000cccd0 con 0x7f68e0079f20 2024-09-16T15:50:36.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:36.923+0000 7f6907800640 1 -- 172.21.15.151:0/2371397359 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f68e0079f20 msgr2=0x7f68e007c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:36.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:36.923+0000 7f6907800640 1 --2- 172.21.15.151:0/2371397359 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f68e0079f20 0x7f68e007c3e0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f68f4005fd0 tx=0x7f68f400b2f0 comp rx=0 tx=0).stop 2024-09-16T15:50:36.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:36.923+0000 7f6907800640 1 -- 172.21.15.151:0/2371397359 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69000b3e20 msgr2=0x7f6900144c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:36.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:36.923+0000 7f6907800640 1 --2- 172.21.15.151:0/2371397359 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69000b3e20 0x7f6900144c10 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f68ec009850 tx=0x7f68ec003910 comp rx=0 tx=0).stop 2024-09-16T15:50:36.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:36.923+0000 7f6907800640 1 -- 172.21.15.151:0/2371397359 shutdown_connections 2024-09-16T15:50:36.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:36.923+0000 7f6907800640 1 --2- 172.21.15.151:0/2371397359 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f68e0079f20 0x7f68e007c3e0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:36.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:36.923+0000 7f6907800640 1 --2- 172.21.15.151:0/2371397359 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f69001512b0 0x7f69000738c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:36.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:36.923+0000 7f6907800640 1 --2- 172.21.15.151:0/2371397359 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69000b3e20 0x7f6900144c10 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:36.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:36.923+0000 7f6907800640 1 --2- 172.21.15.151:0/2371397359 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f690006fa00 0x7f690015c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:36.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:36.923+0000 7f6907800640 1 -- 172.21.15.151:0/2371397359 >> 172.21.15.151:0/2371397359 conn(0x7f69000b18e0 msgr2=0x7f69000b0510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:36.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:36.923+0000 7f6907800640 1 -- 172.21.15.151:0/2371397359 shutdown_connections 2024-09-16T15:50:36.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:36.923+0000 7f6907800640 1 -- 172.21.15.151:0/2371397359 wait complete. 2024-09-16T15:50:36.942 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone22126 2024-09-16T15:50:37.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.108+0000 7f7851600640 1 -- 172.21.15.151:0/3812484438 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f784c15b700 msgr2=0x7f784c0ccca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:37.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.108+0000 7f7851600640 1 --2- 172.21.15.151:0/3812484438 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f784c15b700 0x7f784c0ccca0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7834005fd0 tx=0x7f783402a5c0 comp rx=0 tx=0).stop 2024-09-16T15:50:37.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.108+0000 7f7851600640 1 -- 172.21.15.151:0/3812484438 shutdown_connections 2024-09-16T15:50:37.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.108+0000 7f7851600640 1 --2- 172.21.15.151:0/3812484438 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f784c15b700 0x7f784c0ccca0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:37.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.108+0000 7f7851600640 1 --2- 172.21.15.151:0/3812484438 >> v2:172.21.15.167:3300/0 conn(0x7f784c158900 0x7f784c15b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:37.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.108+0000 7f7851600640 1 -- 172.21.15.151:0/3812484438 >> 172.21.15.151:0/3812484438 conn(0x7f784c0b18e0 msgr2=0x7f784c0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:37.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.108+0000 7f7851600640 1 -- 172.21.15.151:0/3812484438 shutdown_connections 2024-09-16T15:50:37.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.108+0000 7f7851600640 1 -- 172.21.15.151:0/3812484438 wait complete. 2024-09-16T15:50:37.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.110+0000 7f7851600640 1 Processor -- start 2024-09-16T15:50:37.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.110+0000 7f7851600640 1 -- start start 2024-09-16T15:50:37.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.111+0000 7f7851600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f784c0cd1e0 0x7f784c1f4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:37.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.111+0000 7f7851600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f784c158900 0x7f784c09dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:37.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.111+0000 7f7851600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f784c15b700 0x7f784c0727c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:37.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.111+0000 7f7851600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f784c0ced60 con 0x7f784c158900 2024-09-16T15:50:37.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.111+0000 7f7851600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f784c0cebe0 con 0x7f784c15b700 2024-09-16T15:50:37.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.111+0000 7f7851600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f784c0ceee0 con 0x7f784c0cd1e0 2024-09-16T15:50:37.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.111+0000 7f7842000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f784c158900 0x7f784c09dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:37.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.111+0000 7f7842000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f784c158900 0x7f784c09dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53934/0 (socket says 172.21.15.151:53934) 2024-09-16T15:50:37.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.111+0000 7f7842000640 1 -- 172.21.15.151:0/3923198452 learned_addr learned my addr 172.21.15.151:0/3923198452 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:37.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.111+0000 7f784aa00640 1 --2- 172.21.15.151:0/3923198452 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f784c0cd1e0 0x7f784c1f4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:37.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.111+0000 7f784b400640 1 --2- 172.21.15.151:0/3923198452 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f784c15b700 0x7f784c0727c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:37.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.111+0000 7f7842000640 1 -- 172.21.15.151:0/3923198452 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f784c0cd1e0 msgr2=0x7f784c1f4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:37.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.111+0000 7f7842000640 1 --2- 172.21.15.151:0/3923198452 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f784c0cd1e0 0x7f784c1f4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:37.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.112+0000 7f7842000640 1 -- 172.21.15.151:0/3923198452 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f784c15b700 msgr2=0x7f784c0727c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:37.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.112+0000 7f7842000640 1 --2- 172.21.15.151:0/3923198452 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f784c15b700 0x7f784c0727c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:37.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.112+0000 7f7842000640 1 -- 172.21.15.151:0/3923198452 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f784c1f5350 con 0x7f784c158900 2024-09-16T15:50:37.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.112+0000 7f7842000640 1 --2- 172.21.15.151:0/3923198452 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f784c158900 0x7f784c09dc70 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f7834034040 tx=0x7f78340038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:37.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.112+0000 7f7843400640 1 -- 172.21.15.151:0/3923198452 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f783403e030 con 0x7f784c158900 2024-09-16T15:50:37.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.112+0000 7f7843400640 1 -- 172.21.15.151:0/3923198452 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7834003d90 con 0x7f784c158900 2024-09-16T15:50:37.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.112+0000 7f7851600640 1 -- 172.21.15.151:0/3923198452 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f784c1f5630 con 0x7f784c158900 2024-09-16T15:50:37.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.112+0000 7f7843400640 1 -- 172.21.15.151:0/3923198452 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7834035040 con 0x7f784c158900 2024-09-16T15:50:37.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.112+0000 7f7851600640 1 -- 172.21.15.151:0/3923198452 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f784c1e8ee0 con 0x7f784c158900 2024-09-16T15:50:37.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.113+0000 7f7851600640 1 -- 172.21.15.151:0/3923198452 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f784c069360 con 0x7f784c158900 2024-09-16T15:50:37.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.114+0000 7f7843400640 1 -- 172.21.15.151:0/3923198452 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7f7834041030 con 0x7f784c158900 2024-09-16T15:50:37.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.115+0000 7f7843400640 1 --2- 172.21.15.151:0/3923198452 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7828079ed0 0x7f782807c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:37.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.116+0000 7f7843400640 1 -- 172.21.15.151:0/3923198452 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7f78340b9fd0 con 0x7f784c158900 2024-09-16T15:50:37.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.116+0000 7f784aa00640 1 --2- 172.21.15.151:0/3923198452 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7828079ed0 0x7f782807c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:37.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.117+0000 7f784aa00640 1 --2- 172.21.15.151:0/3923198452 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7828079ed0 0x7f782807c390 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f783c0095f0 tx=0x7f783c005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:37.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.120+0000 7f7843400640 1 -- 172.21.15.151:0/3923198452 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7834083a50 con 0x7f784c158900 2024-09-16T15:50:37.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.429+0000 7f7851600640 1 -- 172.21.15.151:0/3923198452 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone22126", "target": ["mon-mgr", ""]}) -- 0x7f784c15e340 con 0x7f7828079ed0 2024-09-16T15:50:37.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.449+0000 7f7843400640 1 -- 172.21.15.151:0/3923198452 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f784c15e340 con 0x7f7828079ed0 2024-09-16T15:50:37.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.452+0000 7f7851600640 1 -- 172.21.15.151:0/3923198452 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7828079ed0 msgr2=0x7f782807c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:37.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.452+0000 7f7851600640 1 --2- 172.21.15.151:0/3923198452 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7828079ed0 0x7f782807c390 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f783c0095f0 tx=0x7f783c005f90 comp rx=0 tx=0).stop 2024-09-16T15:50:37.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.452+0000 7f7851600640 1 -- 172.21.15.151:0/3923198452 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f784c158900 msgr2=0x7f784c09dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:37.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.452+0000 7f7851600640 1 --2- 172.21.15.151:0/3923198452 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f784c158900 0x7f784c09dc70 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f7834034040 tx=0x7f78340038a0 comp rx=0 tx=0).stop 2024-09-16T15:50:37.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.452+0000 7f7851600640 1 -- 172.21.15.151:0/3923198452 shutdown_connections 2024-09-16T15:50:37.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.452+0000 7f7851600640 1 --2- 172.21.15.151:0/3923198452 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7828079ed0 0x7f782807c390 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:37.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.452+0000 7f7851600640 1 --2- 172.21.15.151:0/3923198452 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f784c15b700 0x7f784c0727c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:37.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.452+0000 7f7851600640 1 --2- 172.21.15.151:0/3923198452 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f784c158900 0x7f784c09dc70 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:37.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.452+0000 7f7851600640 1 --2- 172.21.15.151:0/3923198452 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f784c0cd1e0 0x7f784c1f4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:37.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.452+0000 7f7851600640 1 -- 172.21.15.151:0/3923198452 >> 172.21.15.151:0/3923198452 conn(0x7f784c0b18e0 msgr2=0x7f784c0cdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:37.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.452+0000 7f7851600640 1 -- 172.21.15.151:0/3923198452 shutdown_connections 2024-09-16T15:50:37.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.452+0000 7f7851600640 1 -- 172.21.15.151:0/3923198452 wait complete. 2024-09-16T15:50:37.471 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone81887 2024-09-16T15:50:37.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.636+0000 7f6049400640 1 -- 172.21.15.151:0/3986999547 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60440d2920 msgr2=0x7f60440d47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:37.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.636+0000 7f6049400640 1 --2- 172.21.15.151:0/3986999547 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60440d2920 0x7f60440d47e0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f60440b3770 tx=0x7f602c02a5f0 comp rx=0 tx=0).stop 2024-09-16T15:50:37.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.636+0000 7f6049400640 1 -- 172.21.15.151:0/3986999547 shutdown_connections 2024-09-16T15:50:37.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.636+0000 7f6049400640 1 --2- 172.21.15.151:0/3986999547 >> v2:172.21.15.167:3300/0 conn(0x7f60440b5200 0x7f60440b1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:37.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.636+0000 7f6049400640 1 --2- 172.21.15.151:0/3986999547 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60440d2920 0x7f60440d47e0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:37.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.636+0000 7f6049400640 1 -- 172.21.15.151:0/3986999547 >> 172.21.15.151:0/3986999547 conn(0x7f604409d180 msgr2=0x7f60440d3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:37.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.636+0000 7f6049400640 1 -- 172.21.15.151:0/3986999547 shutdown_connections 2024-09-16T15:50:37.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.636+0000 7f6049400640 1 -- 172.21.15.151:0/3986999547 wait complete. 2024-09-16T15:50:37.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.637+0000 7f6049400640 1 Processor -- start 2024-09-16T15:50:37.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.638+0000 7f6049400640 1 -- start start 2024-09-16T15:50:37.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.638+0000 7f6049400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60440b5200 0x7f60441f4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:37.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.638+0000 7f6049400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f60440d2920 0x7f60440b7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:37.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.639+0000 7f6049400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60441623b0 0x7f60440b9010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:37.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.639+0000 7f6049400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f60440b2230 con 0x7f60440b5200 2024-09-16T15:50:37.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.639+0000 7f6049400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f60440b20b0 con 0x7f60441623b0 2024-09-16T15:50:37.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.639+0000 7f6049400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f60440b23b0 con 0x7f60440d2920 2024-09-16T15:50:37.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.639+0000 7f6043e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60440b5200 0x7f60441f4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:37.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.639+0000 7f6043e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60440b5200 0x7f60441f4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53952/0 (socket says 172.21.15.151:53952) 2024-09-16T15:50:37.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.639+0000 7f6043e00640 1 -- 172.21.15.151:0/28237841 learned_addr learned my addr 172.21.15.151:0/28237841 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:37.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.639+0000 7f6048a00640 1 --2- 172.21.15.151:0/28237841 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60441623b0 0x7f60440b9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:37.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.639+0000 7f603be00640 1 --2- 172.21.15.151:0/28237841 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f60440d2920 0x7f60440b7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:37.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.639+0000 7f6043e00640 1 -- 172.21.15.151:0/28237841 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f60440d2920 msgr2=0x7f60440b7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:37.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.639+0000 7f6043e00640 1 --2- 172.21.15.151:0/28237841 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f60440d2920 0x7f60440b7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:37.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.639+0000 7f6043e00640 1 -- 172.21.15.151:0/28237841 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60441623b0 msgr2=0x7f60440b9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:37.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.639+0000 7f6043e00640 1 --2- 172.21.15.151:0/28237841 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60441623b0 0x7f60440b9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:37.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.639+0000 7f6043e00640 1 -- 172.21.15.151:0/28237841 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60440cf7e0 con 0x7f60440b5200 2024-09-16T15:50:37.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.639+0000 7f6043e00640 1 --2- 172.21.15.151:0/28237841 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60440b5200 0x7f60441f4b30 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f603400c3a0 tx=0x7f603400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:37.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.640+0000 7f6040c00640 1 -- 172.21.15.151:0/28237841 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6034017070 con 0x7f60440b5200 2024-09-16T15:50:37.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.640+0000 7f6040c00640 1 -- 172.21.15.151:0/28237841 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f60340053c0 con 0x7f60440b5200 2024-09-16T15:50:37.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.640+0000 7f6049400640 1 -- 172.21.15.151:0/28237841 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60441f56b0 con 0x7f60440b5200 2024-09-16T15:50:37.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.640+0000 7f6040c00640 1 -- 172.21.15.151:0/28237841 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6034003850 con 0x7f60440b5200 2024-09-16T15:50:37.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.640+0000 7f6049400640 1 -- 172.21.15.151:0/28237841 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60441f5070 con 0x7f60440b5200 2024-09-16T15:50:37.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.641+0000 7f6049400640 1 -- 172.21.15.151:0/28237841 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60440b1960 con 0x7f60440b5200 2024-09-16T15:50:37.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.642+0000 7f6040c00640 1 -- 172.21.15.151:0/28237841 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7f603401a070 con 0x7f60440b5200 2024-09-16T15:50:37.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.643+0000 7f6040c00640 1 --2- 172.21.15.151:0/28237841 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6020079ed0 0x7f602007c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:37.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.643+0000 7f6040c00640 1 -- 172.21.15.151:0/28237841 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5765+0+0 (secure 0 0 0) 0x7f60340988e0 con 0x7f60440b5200 2024-09-16T15:50:37.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.644+0000 7f603be00640 1 --2- 172.21.15.151:0/28237841 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6020079ed0 0x7f602007c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:37.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.644+0000 7f603be00640 1 --2- 172.21.15.151:0/28237841 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6020079ed0 0x7f602007c390 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f602c033000 tx=0x7f602c004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:37.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.647+0000 7f6040c00640 1 -- 172.21.15.151:0/28237841 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6034062400 con 0x7f60440b5200 2024-09-16T15:50:37.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.869+0000 7f6049400640 1 -- 172.21.15.151:0/28237841 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone81887", "target": ["mon-mgr", ""]}) -- 0x7f604408d600 con 0x7f6020079ed0 2024-09-16T15:50:37.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.888+0000 7f6040c00640 1 -- 172.21.15.151:0/28237841 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f604408d600 con 0x7f6020079ed0 2024-09-16T15:50:37.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.891+0000 7f6049400640 1 -- 172.21.15.151:0/28237841 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6020079ed0 msgr2=0x7f602007c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:37.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.891+0000 7f6049400640 1 --2- 172.21.15.151:0/28237841 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6020079ed0 0x7f602007c390 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f602c033000 tx=0x7f602c004730 comp rx=0 tx=0).stop 2024-09-16T15:50:37.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.891+0000 7f6049400640 1 -- 172.21.15.151:0/28237841 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60440b5200 msgr2=0x7f60441f4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:37.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.891+0000 7f6049400640 1 --2- 172.21.15.151:0/28237841 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60440b5200 0x7f60441f4b30 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f603400c3a0 tx=0x7f603400c860 comp rx=0 tx=0).stop 2024-09-16T15:50:37.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.891+0000 7f6049400640 1 -- 172.21.15.151:0/28237841 shutdown_connections 2024-09-16T15:50:37.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.891+0000 7f6049400640 1 --2- 172.21.15.151:0/28237841 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6020079ed0 0x7f602007c390 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:37.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.891+0000 7f6049400640 1 --2- 172.21.15.151:0/28237841 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f60441623b0 0x7f60440b9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:37.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.891+0000 7f6049400640 1 --2- 172.21.15.151:0/28237841 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f60440d2920 0x7f60440b7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:37.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.891+0000 7f6049400640 1 --2- 172.21.15.151:0/28237841 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60440b5200 0x7f60441f4b30 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:37.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.891+0000 7f6049400640 1 -- 172.21.15.151:0/28237841 >> 172.21.15.151:0/28237841 conn(0x7f604409d180 msgr2=0x7f60440cb230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:37.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.891+0000 7f6049400640 1 -- 172.21.15.151:0/28237841 shutdown_connections 2024-09-16T15:50:37.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:37.891+0000 7f6049400640 1 -- 172.21.15.151:0/28237841 wait complete. 2024-09-16T15:50:37.913 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone556011 2024-09-16T15:50:38.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.084+0000 7f33e3000640 1 -- 172.21.15.151:0/720829609 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f33dc06e590 msgr2=0x7f33dc06fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:38.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.084+0000 7f33e3000640 1 --2- 172.21.15.151:0/720829609 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f33dc06e590 0x7f33dc06fe40 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f33dc070ae0 tx=0x7f33c802a5f0 comp rx=0 tx=0).stop 2024-09-16T15:50:38.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.084+0000 7f33e3000640 1 -- 172.21.15.151:0/720829609 shutdown_connections 2024-09-16T15:50:38.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.084+0000 7f33e3000640 1 --2- 172.21.15.151:0/720829609 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f33dc06e590 0x7f33dc06fe40 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:38.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.084+0000 7f33e3000640 1 --2- 172.21.15.151:0/720829609 >> v2:172.21.15.167:3300/0 conn(0x7f33dc15bda0 0x7f33dc073010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:38.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.084+0000 7f33e3000640 1 -- 172.21.15.151:0/720829609 >> 172.21.15.151:0/720829609 conn(0x7f33dc0cfa70 msgr2=0x7f33dc0cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:38.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.084+0000 7f33e3000640 1 -- 172.21.15.151:0/720829609 shutdown_connections 2024-09-16T15:50:38.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.084+0000 7f33e3000640 1 -- 172.21.15.151:0/720829609 wait complete. 2024-09-16T15:50:38.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.086+0000 7f33e3000640 1 Processor -- start 2024-09-16T15:50:38.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.086+0000 7f33e3000640 1 -- start start 2024-09-16T15:50:38.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.087+0000 7f33e3000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f33dc06e590 0x7f33dc1514e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:38.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.087+0000 7f33e3000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33dc070380 0x7f33dc0ce990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:38.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.087+0000 7f33e3000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33dc15bda0 0x7f33dc0c8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:38.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.087+0000 7f33e3000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f33dc0cc350 con 0x7f33dc06e590 2024-09-16T15:50:38.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.087+0000 7f33e3000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f33dc0cc1d0 con 0x7f33dc15bda0 2024-09-16T15:50:38.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.087+0000 7f33e3000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f33dc0740e0 con 0x7f33dc070380 2024-09-16T15:50:38.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.087+0000 7f33e1c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f33dc06e590 0x7f33dc1514e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:38.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.087+0000 7f33e1c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f33dc06e590 0x7f33dc1514e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53958/0 (socket says 172.21.15.151:53958) 2024-09-16T15:50:38.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.087+0000 7f33e1c00640 1 -- 172.21.15.151:0/1432216048 learned_addr learned my addr 172.21.15.151:0/1432216048 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:38.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.087+0000 7f33e2600640 1 --2- 172.21.15.151:0/1432216048 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33dc15bda0 0x7f33dc0c8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:38.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.087+0000 7f33e1200640 1 --2- 172.21.15.151:0/1432216048 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33dc070380 0x7f33dc0ce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:38.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.088+0000 7f33e1c00640 1 -- 172.21.15.151:0/1432216048 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33dc070380 msgr2=0x7f33dc0ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:38.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.088+0000 7f33e1c00640 1 --2- 172.21.15.151:0/1432216048 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33dc070380 0x7f33dc0ce990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:38.090 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.088+0000 7f33e1c00640 1 -- 172.21.15.151:0/1432216048 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33dc15bda0 msgr2=0x7f33dc0c8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:38.090 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.088+0000 7f33e1c00640 1 --2- 172.21.15.151:0/1432216048 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33dc15bda0 0x7f33dc0c8d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:38.090 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.088+0000 7f33e1c00640 1 -- 172.21.15.151:0/1432216048 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33dc0cf3e0 con 0x7f33dc06e590 2024-09-16T15:50:38.090 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.088+0000 7f33e1c00640 1 --2- 172.21.15.151:0/1432216048 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f33dc06e590 0x7f33dc1514e0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f33d000d6d0 tx=0x7f33d000db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:38.090 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.088+0000 7f33d9600640 1 -- 172.21.15.151:0/1432216048 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d0017070 con 0x7f33dc06e590 2024-09-16T15:50:38.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.088+0000 7f33d9600640 1 -- 172.21.15.151:0/1432216048 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f33d0005330 con 0x7f33dc06e590 2024-09-16T15:50:38.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.088+0000 7f33d9600640 1 -- 172.21.15.151:0/1432216048 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d0009820 con 0x7f33dc06e590 2024-09-16T15:50:38.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.088+0000 7f33e3000640 1 -- 172.21.15.151:0/1432216048 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33dc15c4a0 con 0x7f33dc06e590 2024-09-16T15:50:38.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.088+0000 7f33e3000640 1 -- 172.21.15.151:0/1432216048 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33dc15ca60 con 0x7f33dc06e590 2024-09-16T15:50:38.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.089+0000 7f33e3000640 1 -- 172.21.15.151:0/1432216048 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33dc074590 con 0x7f33dc06e590 2024-09-16T15:50:38.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.091+0000 7f33d9600640 1 -- 172.21.15.151:0/1432216048 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7f33d0009bd0 con 0x7f33dc06e590 2024-09-16T15:50:38.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.091+0000 7f33d9600640 1 --2- 172.21.15.151:0/1432216048 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f33bc079ed0 0x7f33bc07c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:38.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.092+0000 7f33d9600640 1 -- 172.21.15.151:0/1432216048 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5821+0+0 (secure 0 0 0) 0x7f33d0099900 con 0x7f33dc06e590 2024-09-16T15:50:38.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.092+0000 7f33e1200640 1 --2- 172.21.15.151:0/1432216048 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f33bc079ed0 0x7f33bc07c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:38.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.093+0000 7f33e1200640 1 --2- 172.21.15.151:0/1432216048 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f33bc079ed0 0x7f33bc07c390 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f33c8033000 tx=0x7f33c8004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:38.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.096+0000 7f33d9600640 1 -- 172.21.15.151:0/1432216048 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f33d00a2070 con 0x7f33dc06e590 2024-09-16T15:50:38.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.473+0000 7f33e3000640 1 -- 172.21.15.151:0/1432216048 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone556011", "target": ["mon-mgr", ""]}) -- 0x7f33dc0b3f10 con 0x7f33bc079ed0 2024-09-16T15:50:38.509 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.508+0000 7f33d9600640 1 -- 172.21.15.151:0/1432216048 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f33dc0b3f10 con 0x7f33bc079ed0 2024-09-16T15:50:38.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.512+0000 7f33e3000640 1 -- 172.21.15.151:0/1432216048 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f33bc079ed0 msgr2=0x7f33bc07c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:38.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.512+0000 7f33e3000640 1 --2- 172.21.15.151:0/1432216048 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f33bc079ed0 0x7f33bc07c390 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f33c8033000 tx=0x7f33c8004730 comp rx=0 tx=0).stop 2024-09-16T15:50:38.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.512+0000 7f33e3000640 1 -- 172.21.15.151:0/1432216048 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f33dc06e590 msgr2=0x7f33dc1514e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:38.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.512+0000 7f33e3000640 1 --2- 172.21.15.151:0/1432216048 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f33dc06e590 0x7f33dc1514e0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f33d000d6d0 tx=0x7f33d000db90 comp rx=0 tx=0).stop 2024-09-16T15:50:38.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.512+0000 7f33e3000640 1 -- 172.21.15.151:0/1432216048 shutdown_connections 2024-09-16T15:50:38.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.512+0000 7f33e3000640 1 --2- 172.21.15.151:0/1432216048 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f33bc079ed0 0x7f33bc07c390 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:38.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.512+0000 7f33e3000640 1 --2- 172.21.15.151:0/1432216048 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f33dc15bda0 0x7f33dc0c8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:38.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.512+0000 7f33e3000640 1 --2- 172.21.15.151:0/1432216048 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f33dc070380 0x7f33dc0ce990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:38.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.512+0000 7f33e3000640 1 --2- 172.21.15.151:0/1432216048 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f33dc06e590 0x7f33dc1514e0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:38.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.512+0000 7f33e3000640 1 -- 172.21.15.151:0/1432216048 >> 172.21.15.151:0/1432216048 conn(0x7f33dc0cfa70 msgr2=0x7f33dc0b2a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:38.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.512+0000 7f33e3000640 1 -- 172.21.15.151:0/1432216048 shutdown_connections 2024-09-16T15:50:38.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.513+0000 7f33e3000640 1 -- 172.21.15.151:0/1432216048 wait complete. 2024-09-16T15:50:38.540 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone277913 2024-09-16T15:50:38.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.719+0000 7fde27200640 1 -- 172.21.15.151:0/3855578229 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde28073cf0 msgr2=0x7fde2806ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:38.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.719+0000 7fde27200640 1 --2- 172.21.15.151:0/3855578229 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde28073cf0 0x7fde2806ff30 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fde2815e640 tx=0x7fde2002aac0 comp rx=0 tx=0).stop 2024-09-16T15:50:38.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.719+0000 7fde27200640 1 -- 172.21.15.151:0/3855578229 shutdown_connections 2024-09-16T15:50:38.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.719+0000 7fde27200640 1 --2- 172.21.15.151:0/3855578229 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde28073cf0 0x7fde2806ff30 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:38.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.719+0000 7fde27200640 1 --2- 172.21.15.151:0/3855578229 >> v2:172.21.15.167:3300/0 conn(0x7fde280d07b0 0x7fde28072d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:38.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.719+0000 7fde27200640 1 -- 172.21.15.151:0/3855578229 >> 172.21.15.151:0/3855578229 conn(0x7fde280cfa70 msgr2=0x7fde280cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:38.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.719+0000 7fde27200640 1 -- 172.21.15.151:0/3855578229 shutdown_connections 2024-09-16T15:50:38.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.719+0000 7fde27200640 1 -- 172.21.15.151:0/3855578229 wait complete. 2024-09-16T15:50:38.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.721+0000 7fde27200640 1 Processor -- start 2024-09-16T15:50:38.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.721+0000 7fde27200640 1 -- start start 2024-09-16T15:50:38.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.722+0000 7fde27200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde28073cf0 0x7fde280c5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:38.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.722+0000 7fde27200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde2815abd0 0x7fde280c32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:38.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.722+0000 7fde27200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde281f48d0 0x7fde2814be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:38.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.722+0000 7fde27200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fde280cd7c0 con 0x7fde281f48d0 2024-09-16T15:50:38.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.722+0000 7fde27200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fde280cd640 con 0x7fde28073cf0 2024-09-16T15:50:38.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.722+0000 7fde27200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fde280cd940 con 0x7fde2815abd0 2024-09-16T15:50:38.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.722+0000 7fde26800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde281f48d0 0x7fde2814be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:38.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.722+0000 7fde26800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde281f48d0 0x7fde2814be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53962/0 (socket says 172.21.15.151:53962) 2024-09-16T15:50:38.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.722+0000 7fde26800640 1 -- 172.21.15.151:0/1513728235 learned_addr learned my addr 172.21.15.151:0/1513728235 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:38.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.722+0000 7fde25e00640 1 --2- 172.21.15.151:0/1513728235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde28073cf0 0x7fde280c5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:38.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.722+0000 7fde25400640 1 --2- 172.21.15.151:0/1513728235 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde2815abd0 0x7fde280c32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:38.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.722+0000 7fde26800640 1 -- 172.21.15.151:0/1513728235 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde2815abd0 msgr2=0x7fde280c32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:38.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.722+0000 7fde26800640 1 --2- 172.21.15.151:0/1513728235 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde2815abd0 0x7fde280c32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:38.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.722+0000 7fde26800640 1 -- 172.21.15.151:0/1513728235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde28073cf0 msgr2=0x7fde280c5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:38.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.722+0000 7fde26800640 1 --2- 172.21.15.151:0/1513728235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde28073cf0 0x7fde280c5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:38.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.722+0000 7fde26800640 1 -- 172.21.15.151:0/1513728235 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde2814c380 con 0x7fde281f48d0 2024-09-16T15:50:38.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.723+0000 7fde26800640 1 --2- 172.21.15.151:0/1513728235 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde281f48d0 0x7fde2814be40 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fde2002b0b0 tx=0x7fde20003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:38.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.723+0000 7fde1e000640 1 -- 172.21.15.151:0/1513728235 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde20044070 con 0x7fde281f48d0 2024-09-16T15:50:38.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.723+0000 7fde1e000640 1 -- 172.21.15.151:0/1513728235 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fde20004250 con 0x7fde281f48d0 2024-09-16T15:50:38.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.723+0000 7fde1e000640 1 -- 172.21.15.151:0/1513728235 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde20004680 con 0x7fde281f48d0 2024-09-16T15:50:38.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.723+0000 7fde27200640 1 -- 172.21.15.151:0/1513728235 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde281f4db0 con 0x7fde281f48d0 2024-09-16T15:50:38.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.723+0000 7fde27200640 1 -- 172.21.15.151:0/1513728235 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde281f5370 con 0x7fde281f48d0 2024-09-16T15:50:38.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.724+0000 7fde27200640 1 -- 172.21.15.151:0/1513728235 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde2806ff30 con 0x7fde281f48d0 2024-09-16T15:50:38.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.725+0000 7fde1e000640 1 -- 172.21.15.151:0/1513728235 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7fde2002c840 con 0x7fde281f48d0 2024-09-16T15:50:38.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.726+0000 7fde1e000640 1 --2- 172.21.15.151:0/1513728235 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fde08079ed0 0x7fde0807c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:38.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.726+0000 7fde1e000640 1 -- 172.21.15.151:0/1513728235 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5821+0+0 (secure 0 0 0) 0x7fde200b9f80 con 0x7fde281f48d0 2024-09-16T15:50:38.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.727+0000 7fde25e00640 1 --2- 172.21.15.151:0/1513728235 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fde08079ed0 0x7fde0807c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:38.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.727+0000 7fde25e00640 1 --2- 172.21.15.151:0/1513728235 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fde08079ed0 0x7fde0807c390 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fde18001720 tx=0x7fde18009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:38.732 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.731+0000 7fde1e000640 1 -- 172.21.15.151:0/1513728235 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fde20032030 con 0x7fde281f48d0 2024-09-16T15:50:38.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.943+0000 7fde27200640 1 -- 172.21.15.151:0/1513728235 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone277913", "target": ["mon-mgr", ""]}) -- 0x7fde280ce160 con 0x7fde08079ed0 2024-09-16T15:50:38.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.962+0000 7fde1e000640 1 -- 172.21.15.151:0/1513728235 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fde280ce160 con 0x7fde08079ed0 2024-09-16T15:50:38.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.964+0000 7fde27200640 1 -- 172.21.15.151:0/1513728235 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fde08079ed0 msgr2=0x7fde0807c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:38.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.964+0000 7fde27200640 1 --2- 172.21.15.151:0/1513728235 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fde08079ed0 0x7fde0807c390 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fde18001720 tx=0x7fde18009040 comp rx=0 tx=0).stop 2024-09-16T15:50:38.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.964+0000 7fde27200640 1 -- 172.21.15.151:0/1513728235 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde281f48d0 msgr2=0x7fde2814be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:38.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.964+0000 7fde27200640 1 --2- 172.21.15.151:0/1513728235 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde281f48d0 0x7fde2814be40 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fde2002b0b0 tx=0x7fde20003f10 comp rx=0 tx=0).stop 2024-09-16T15:50:38.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.965+0000 7fde27200640 1 -- 172.21.15.151:0/1513728235 shutdown_connections 2024-09-16T15:50:38.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.965+0000 7fde27200640 1 --2- 172.21.15.151:0/1513728235 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fde08079ed0 0x7fde0807c390 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:38.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.965+0000 7fde27200640 1 --2- 172.21.15.151:0/1513728235 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde281f48d0 0x7fde2814be40 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:38.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.965+0000 7fde27200640 1 --2- 172.21.15.151:0/1513728235 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde2815abd0 0x7fde280c32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:38.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.965+0000 7fde27200640 1 --2- 172.21.15.151:0/1513728235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde28073cf0 0x7fde280c5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:38.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.965+0000 7fde27200640 1 -- 172.21.15.151:0/1513728235 >> 172.21.15.151:0/1513728235 conn(0x7fde280cfa70 msgr2=0x7fde280b48b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:38.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.965+0000 7fde27200640 1 -- 172.21.15.151:0/1513728235 shutdown_connections 2024-09-16T15:50:38.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:38.965+0000 7fde27200640 1 -- 172.21.15.151:0/1513728235 wait complete. 2024-09-16T15:50:38.983 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone53393 2024-09-16T15:50:39.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.165+0000 7fcdac400640 1 -- 172.21.15.151:0/645955604 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcda40708b0 msgr2=0x7fcda40cc880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:39.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.165+0000 7fcdac400640 1 --2- 172.21.15.151:0/645955604 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcda40708b0 0x7fcda40cc880 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fcda4151690 tx=0x7fcd9c032e30 comp rx=0 tx=0).stop 2024-09-16T15:50:39.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.165+0000 7fcdac400640 1 -- 172.21.15.151:0/645955604 shutdown_connections 2024-09-16T15:50:39.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.165+0000 7fcdac400640 1 --2- 172.21.15.151:0/645955604 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcda40708b0 0x7fcda40cc880 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:39.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.165+0000 7fcdac400640 1 --2- 172.21.15.151:0/645955604 >> v2:172.21.15.167:3300/0 conn(0x7fcda40cf6c0 0x7fcda40cc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:39.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.165+0000 7fcdac400640 1 -- 172.21.15.151:0/645955604 >> 172.21.15.151:0/645955604 conn(0x7fcda40b18e0 msgr2=0x7fcda40b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:39.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.166+0000 7fcdac400640 1 -- 172.21.15.151:0/645955604 shutdown_connections 2024-09-16T15:50:39.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.166+0000 7fcdac400640 1 -- 172.21.15.151:0/645955604 wait complete. 2024-09-16T15:50:39.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.169+0000 7fcdac400640 1 Processor -- start 2024-09-16T15:50:39.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.169+0000 7fcdac400640 1 -- start start 2024-09-16T15:50:39.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.170+0000 7fcdac400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcda40708b0 0x7fcda409d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:39.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.170+0000 7fcdac400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcda40cf6c0 0x7fcda413ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:39.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.170+0000 7fcdac400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcda41512b0 0x7fcda409aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:39.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.170+0000 7fcdac400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcda406fb80 con 0x7fcda40cf6c0 2024-09-16T15:50:39.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.170+0000 7fcdac400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcda406fa00 con 0x7fcda40708b0 2024-09-16T15:50:39.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.170+0000 7fcdac400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fcda406fd00 con 0x7fcda41512b0 2024-09-16T15:50:39.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.171+0000 7fcda9a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcda40708b0 0x7fcda409d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:39.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.171+0000 7fcda9000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcda40cf6c0 0x7fcda413ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:39.174 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.171+0000 7fcda9a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcda40708b0 0x7fcda409d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.151:56648/0 (socket says 172.21.15.151:56648) 2024-09-16T15:50:39.174 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.171+0000 7fcda9000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcda40cf6c0 0x7fcda413ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53982/0 (socket says 172.21.15.151:53982) 2024-09-16T15:50:39.174 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.171+0000 7fcda9a00640 1 -- 172.21.15.151:0/1515334869 learned_addr learned my addr 172.21.15.151:0/1515334869 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:39.174 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.172+0000 7fcda9000640 1 -- 172.21.15.151:0/1515334869 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcda41512b0 msgr2=0x7fcda409aa00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T15:50:39.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.172+0000 7fcda9000640 1 --2- 172.21.15.151:0/1515334869 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcda41512b0 0x7fcda409aa00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:39.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.172+0000 7fcda9000640 1 -- 172.21.15.151:0/1515334869 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcda40708b0 msgr2=0x7fcda409d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:39.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.172+0000 7fcda9000640 1 --2- 172.21.15.151:0/1515334869 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcda40708b0 0x7fcda409d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:39.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.172+0000 7fcda9000640 1 -- 172.21.15.151:0/1515334869 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcda409cc50 con 0x7fcda40cf6c0 2024-09-16T15:50:39.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.172+0000 7fcda9000640 1 --2- 172.21.15.151:0/1515334869 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcda40cf6c0 0x7fcda413ef60 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fcd9000e9f0 tx=0x7fcd9000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:39.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.173+0000 7fcda1600640 1 -- 172.21.15.151:0/1515334869 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd9000cc60 con 0x7fcda40cf6c0 2024-09-16T15:50:39.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.173+0000 7fcda1600640 1 -- 172.21.15.151:0/1515334869 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd90010430 con 0x7fcda40cf6c0 2024-09-16T15:50:39.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.173+0000 7fcda1600640 1 -- 172.21.15.151:0/1515334869 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd9000cc60 con 0x7fcda40cf6c0 2024-09-16T15:50:39.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.174+0000 7fcdac400640 1 -- 172.21.15.151:0/1515334869 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcda409cf10 con 0x7fcda40cf6c0 2024-09-16T15:50:39.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.175+0000 7fcdac400640 1 -- 172.21.15.151:0/1515334869 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcda409d4d0 con 0x7fcda40cf6c0 2024-09-16T15:50:39.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.178+0000 7fcdac400640 1 -- 172.21.15.151:0/1515334869 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcda4069360 con 0x7fcda40cf6c0 2024-09-16T15:50:39.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.178+0000 7fcda1600640 1 -- 172.21.15.151:0/1515334869 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7fcd90010600 con 0x7fcda40cf6c0 2024-09-16T15:50:39.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.180+0000 7fcda1600640 1 --2- 172.21.15.151:0/1515334869 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcd8408b0e0 0x7fcd8408d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:39.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.180+0000 7fcda1600640 1 -- 172.21.15.151:0/1515334869 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5821+0+0 (secure 0 0 0) 0x7fcd90099f90 con 0x7fcda40cf6c0 2024-09-16T15:50:39.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.182+0000 7fcda9a00640 1 --2- 172.21.15.151:0/1515334869 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcd8408b0e0 0x7fcd8408d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:39.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.183+0000 7fcda9a00640 1 --2- 172.21.15.151:0/1515334869 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcd8408b0e0 0x7fcd8408d5a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fcda4151f60 tx=0x7fcd98009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:39.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.188+0000 7fcda1600640 1 -- 172.21.15.151:0/1515334869 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fcd90063c80 con 0x7fcda40cf6c0 2024-09-16T15:50:39.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.601+0000 7fcdac400640 1 -- 172.21.15.151:0/1515334869 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone53393", "target": ["mon-mgr", ""]}) -- 0x7fcda409c9f0 con 0x7fcd8408b0e0 2024-09-16T15:50:39.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.636+0000 7fcda1600640 1 -- 172.21.15.151:0/1515334869 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcd98001cc0 con 0x7fcd8408b0e0 2024-09-16T15:50:39.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.640+0000 7fcdac400640 1 -- 172.21.15.151:0/1515334869 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcd8408b0e0 msgr2=0x7fcd8408d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:39.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.640+0000 7fcdac400640 1 --2- 172.21.15.151:0/1515334869 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcd8408b0e0 0x7fcd8408d5a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fcda4151f60 tx=0x7fcd98009040 comp rx=0 tx=0).stop 2024-09-16T15:50:39.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.640+0000 7fcdac400640 1 -- 172.21.15.151:0/1515334869 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcda40cf6c0 msgr2=0x7fcda413ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:39.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.640+0000 7fcdac400640 1 --2- 172.21.15.151:0/1515334869 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcda40cf6c0 0x7fcda413ef60 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fcd9000e9f0 tx=0x7fcd9000eeb0 comp rx=0 tx=0).stop 2024-09-16T15:50:39.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.640+0000 7fcdac400640 1 -- 172.21.15.151:0/1515334869 shutdown_connections 2024-09-16T15:50:39.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.640+0000 7fcdac400640 1 --2- 172.21.15.151:0/1515334869 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcd8408b0e0 0x7fcd8408d5a0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:39.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.640+0000 7fcdac400640 1 --2- 172.21.15.151:0/1515334869 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcda41512b0 0x7fcda409aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:39.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.640+0000 7fcdac400640 1 --2- 172.21.15.151:0/1515334869 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcda40cf6c0 0x7fcda413ef60 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:39.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.640+0000 7fcdac400640 1 --2- 172.21.15.151:0/1515334869 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcda40708b0 0x7fcda409d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:39.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.640+0000 7fcdac400640 1 -- 172.21.15.151:0/1515334869 >> 172.21.15.151:0/1515334869 conn(0x7fcda40b18e0 msgr2=0x7fcda40b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:39.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.640+0000 7fcdac400640 1 -- 172.21.15.151:0/1515334869 shutdown_connections 2024-09-16T15:50:39.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.640+0000 7fcdac400640 1 -- 172.21.15.151:0/1515334869 wait complete. 2024-09-16T15:50:39.667 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone139815 2024-09-16T15:50:39.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.960+0000 7fc18fe00640 1 -- 172.21.15.151:0/2436826752 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc18815f9a0 msgr2=0x7fc18815fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:39.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.960+0000 7fc18fe00640 1 --2- 172.21.15.151:0/2436826752 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc18815f9a0 0x7fc18815fd80 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fc17c009900 tx=0x7fc17c02a620 comp rx=0 tx=0).stop 2024-09-16T15:50:39.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.960+0000 7fc18fe00640 1 -- 172.21.15.151:0/2436826752 shutdown_connections 2024-09-16T15:50:39.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.960+0000 7fc18fe00640 1 --2- 172.21.15.151:0/2436826752 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc18815f9a0 0x7fc18815fd80 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:39.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.960+0000 7fc18fe00640 1 -- 172.21.15.151:0/2436826752 >> 172.21.15.151:0/2436826752 conn(0x7fc1880b18e0 msgr2=0x7fc1880b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:39.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.961+0000 7fc18fe00640 1 -- 172.21.15.151:0/2436826752 shutdown_connections 2024-09-16T15:50:39.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.962+0000 7fc18fe00640 1 -- 172.21.15.151:0/2436826752 wait complete. 2024-09-16T15:50:39.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.964+0000 7fc18fe00640 1 Processor -- start 2024-09-16T15:50:39.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.964+0000 7fc18fe00640 1 -- start start 2024-09-16T15:50:39.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.965+0000 7fc18fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc18806e6f0 0x7fc1881486e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:39.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.965+0000 7fc18fe00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc188073de0 0x7fc188154f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:39.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.965+0000 7fc18fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc18815f9a0 0x7fc1881554c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:39.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.965+0000 7fc18fe00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc1880cd550 con 0x7fc18806e6f0 2024-09-16T15:50:39.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.965+0000 7fc18fe00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc1880cd3d0 con 0x7fc18815f9a0 2024-09-16T15:50:39.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.965+0000 7fc18fe00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fc18806f2b0 con 0x7fc188073de0 2024-09-16T15:50:39.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.966+0000 7fc18ca00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc188073de0 0x7fc188154f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:39.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.966+0000 7fc18ca00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc188073de0 0x7fc188154f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.151:39920/0 (socket says 172.21.15.151:39920) 2024-09-16T15:50:39.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.966+0000 7fc18ca00640 1 -- 172.21.15.151:0/2267775046 learned_addr learned my addr 172.21.15.151:0/2267775046 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:39.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.966+0000 7fc18d400640 1 --2- 172.21.15.151:0/2267775046 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc18806e6f0 0x7fc1881486e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:39.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.966+0000 7fc18de00640 1 --2- 172.21.15.151:0/2267775046 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc18815f9a0 0x7fc1881554c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:39.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.966+0000 7fc18ca00640 1 -- 172.21.15.151:0/2267775046 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc18815f9a0 msgr2=0x7fc1881554c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:39.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.966+0000 7fc18ca00640 1 --2- 172.21.15.151:0/2267775046 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc18815f9a0 0x7fc1881554c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:39.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.966+0000 7fc18ca00640 1 -- 172.21.15.151:0/2267775046 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc18806e6f0 msgr2=0x7fc1881486e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:39.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.966+0000 7fc18ca00640 1 --2- 172.21.15.151:0/2267775046 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc18806e6f0 0x7fc1881486e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:39.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.967+0000 7fc18ca00640 1 -- 172.21.15.151:0/2267775046 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc18814c420 con 0x7fc188073de0 2024-09-16T15:50:39.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.967+0000 7fc18ca00640 1 --2- 172.21.15.151:0/2267775046 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc188073de0 0x7fc188154f80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc17400c500 tx=0x7fc17400c9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:39.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.967+0000 7fc185600640 1 -- 172.21.15.151:0/2267775046 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1740052d0 con 0x7fc188073de0 2024-09-16T15:50:39.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.968+0000 7fc185600640 1 -- 172.21.15.151:0/2267775046 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc174005470 con 0x7fc188073de0 2024-09-16T15:50:39.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.968+0000 7fc185600640 1 -- 172.21.15.151:0/2267775046 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc174012870 con 0x7fc188073de0 2024-09-16T15:50:39.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.968+0000 7fc18fe00640 1 -- 172.21.15.151:0/2267775046 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc18814c700 con 0x7fc188073de0 2024-09-16T15:50:39.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.968+0000 7fc18fe00640 1 -- 172.21.15.151:0/2267775046 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc18814ccc0 con 0x7fc188073de0 2024-09-16T15:50:39.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.970+0000 7fc18fe00640 1 -- 172.21.15.151:0/2267775046 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc188069360 con 0x7fc188073de0 2024-09-16T15:50:39.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.972+0000 7fc185600640 1 -- 172.21.15.151:0/2267775046 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7fc1740035f0 con 0x7fc188073de0 2024-09-16T15:50:39.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.973+0000 7fc185600640 1 --2- 172.21.15.151:0/2267775046 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc168079f20 0x7fc16807c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:39.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.974+0000 7fc185600640 1 -- 172.21.15.151:0/2267775046 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(92..92 src has 1..92) ==== 5821+0+0 (secure 0 0 0) 0x7fc174099190 con 0x7fc188073de0 2024-09-16T15:50:39.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.975+0000 7fc18d400640 1 --2- 172.21.15.151:0/2267775046 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc168079f20 0x7fc16807c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:39.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.976+0000 7fc18d400640 1 --2- 172.21.15.151:0/2267775046 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc168079f20 0x7fc16807c3e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc17c033000 tx=0x7fc17c004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:39.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:39.981+0000 7fc185600640 1 -- 172.21.15.151:0/2267775046 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc174062c10 con 0x7fc188073de0 2024-09-16T15:50:40.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.262+0000 7fc18fe00640 1 -- 172.21.15.151:0/2267775046 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone139815", "target": ["mon-mgr", ""]}) -- 0x7fc1880cd7b0 con 0x7fc168079f20 2024-09-16T15:50:40.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.296+0000 7fc185600640 1 -- 172.21.15.151:0/2267775046 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc1880cd7b0 con 0x7fc168079f20 2024-09-16T15:50:40.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.299+0000 7fc18fe00640 1 -- 172.21.15.151:0/2267775046 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc168079f20 msgr2=0x7fc16807c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:40.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.299+0000 7fc18fe00640 1 --2- 172.21.15.151:0/2267775046 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc168079f20 0x7fc16807c3e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc17c033000 tx=0x7fc17c004730 comp rx=0 tx=0).stop 2024-09-16T15:50:40.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.299+0000 7fc18fe00640 1 -- 172.21.15.151:0/2267775046 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc188073de0 msgr2=0x7fc188154f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:40.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.299+0000 7fc18fe00640 1 --2- 172.21.15.151:0/2267775046 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc188073de0 0x7fc188154f80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc17400c500 tx=0x7fc17400c9c0 comp rx=0 tx=0).stop 2024-09-16T15:50:40.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.300+0000 7fc18fe00640 1 -- 172.21.15.151:0/2267775046 shutdown_connections 2024-09-16T15:50:40.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.300+0000 7fc18fe00640 1 --2- 172.21.15.151:0/2267775046 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc168079f20 0x7fc16807c3e0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:40.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.300+0000 7fc18fe00640 1 --2- 172.21.15.151:0/2267775046 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc18815f9a0 0x7fc1881554c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:40.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.300+0000 7fc18fe00640 1 --2- 172.21.15.151:0/2267775046 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc188073de0 0x7fc188154f80 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:40.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.300+0000 7fc18fe00640 1 --2- 172.21.15.151:0/2267775046 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc18806e6f0 0x7fc1881486e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:40.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.300+0000 7fc18fe00640 1 -- 172.21.15.151:0/2267775046 >> 172.21.15.151:0/2267775046 conn(0x7fc1880b18e0 msgr2=0x7fc18815a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:40.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.300+0000 7fc18fe00640 1 -- 172.21.15.151:0/2267775046 shutdown_connections 2024-09-16T15:50:40.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.300+0000 7fc18fe00640 1 -- 172.21.15.151:0/2267775046 wait complete. 2024-09-16T15:50:40.328 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone515512 2024-09-16T15:50:40.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.597+0000 7f5eee000640 1 -- 172.21.15.151:0/4110576504 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee800a580 con 0x7f5ef00b3e20 2024-09-16T15:50:40.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.598+0000 7f5ef6400640 1 -- 172.21.15.151:0/4110576504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ef00b3e20 msgr2=0x7f5ef006ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:40.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.598+0000 7f5ef6400640 1 --2- 172.21.15.151:0/4110576504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ef00b3e20 0x7f5ef006ff40 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0072480 tx=0x7f5ee802b5f0 comp rx=0 tx=0).stop 2024-09-16T15:50:40.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.598+0000 7f5ef6400640 1 -- 172.21.15.151:0/4110576504 shutdown_connections 2024-09-16T15:50:40.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.598+0000 7f5ef6400640 1 --2- 172.21.15.151:0/4110576504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ef00b3e20 0x7f5ef006ff40 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:40.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.598+0000 7f5ef6400640 1 --2- 172.21.15.151:0/4110576504 >> v2:172.21.15.151:3300/0 conn(0x7f5ef00cf6c0 0x7f5ef006fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:40.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.598+0000 7f5ef6400640 1 -- 172.21.15.151:0/4110576504 >> 172.21.15.151:0/4110576504 conn(0x7f5ef00b18e0 msgr2=0x7f5ef00b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:40.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.599+0000 7f5ef6400640 1 -- 172.21.15.151:0/4110576504 shutdown_connections 2024-09-16T15:50:40.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.599+0000 7f5ef6400640 1 -- 172.21.15.151:0/4110576504 wait complete. 2024-09-16T15:50:40.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.601+0000 7f5ef6400640 1 Processor -- start 2024-09-16T15:50:40.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.602+0000 7f5ef6400640 1 -- start start 2024-09-16T15:50:40.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.603+0000 7f5ef6400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ef00b3e20 0x7f5ef009d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:40.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.603+0000 7f5ef6400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5ef00cf6c0 0x7f5ef00aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:40.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.604+0000 7f5ef6400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ef01512b0 0x7f5ef01fc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:40.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.604+0000 7f5ef6400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef00cc6d0 con 0x7f5ef01512b0 2024-09-16T15:50:40.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.604+0000 7f5ef6400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef00cc550 con 0x7f5ef00b3e20 2024-09-16T15:50:40.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.604+0000 7f5ef6400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f5ef00cc850 con 0x7f5ef00cf6c0 2024-09-16T15:50:40.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.604+0000 7f5eeea00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5ef00cf6c0 0x7f5ef00aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:40.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.604+0000 7f5eeea00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5ef00cf6c0 0x7f5ef00aeb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.151:39934/0 (socket says 172.21.15.151:39934) 2024-09-16T15:50:40.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.604+0000 7f5eeea00640 1 -- 172.21.15.151:0/956569372 learned_addr learned my addr 172.21.15.151:0/956569372 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:40.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.604+0000 7f5eef400640 1 --2- 172.21.15.151:0/956569372 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ef00b3e20 0x7f5ef009d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:40.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.604+0000 7f5eefe00640 1 --2- 172.21.15.151:0/956569372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ef01512b0 0x7f5ef01fc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:40.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.605+0000 7f5eeea00640 1 -- 172.21.15.151:0/956569372 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ef00b3e20 msgr2=0x7f5ef009d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:40.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.605+0000 7f5eeea00640 1 --2- 172.21.15.151:0/956569372 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ef00b3e20 0x7f5ef009d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:40.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.605+0000 7f5eeea00640 1 -- 172.21.15.151:0/956569372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ef01512b0 msgr2=0x7f5ef01fc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:40.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.605+0000 7f5eeea00640 1 --2- 172.21.15.151:0/956569372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ef01512b0 0x7f5ef01fc8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:40.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.605+0000 7f5eeea00640 1 -- 172.21.15.151:0/956569372 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ef00ae040 con 0x7f5ef00cf6c0 2024-09-16T15:50:40.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.605+0000 7f5eefe00640 1 --2- 172.21.15.151:0/956569372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ef01512b0 0x7f5ef01fc8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:50:40.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.606+0000 7f5eeea00640 1 --2- 172.21.15.151:0/956569372 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5ef00cf6c0 0x7f5ef00aeb70 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5edc00ea20 tx=0x7f5edc00eee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:40.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.606+0000 7f5ee7400640 1 -- 172.21.15.151:0/956569372 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5edc00cd80 con 0x7f5ef00cf6c0 2024-09-16T15:50:40.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.606+0000 7f5ee7400640 1 -- 172.21.15.151:0/956569372 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5edc010430 con 0x7f5ef00cf6c0 2024-09-16T15:50:40.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.606+0000 7f5ef6400640 1 -- 172.21.15.151:0/956569372 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ef01fd630 con 0x7f5ef00cf6c0 2024-09-16T15:50:40.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.607+0000 7f5ef6400640 1 -- 172.21.15.151:0/956569372 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ef014a2a0 con 0x7f5ef00cf6c0 2024-09-16T15:50:40.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.607+0000 7f5ee7400640 1 -- 172.21.15.151:0/956569372 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5edc010840 con 0x7f5ef00cf6c0 2024-09-16T15:50:40.610 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.609+0000 7f5ef6400640 1 -- 172.21.15.151:0/956569372 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ec8005290 con 0x7f5ef00cf6c0 2024-09-16T15:50:40.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.612+0000 7f5ee7400640 1 -- 172.21.15.151:0/956569372 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7f5edc0109e0 con 0x7f5ef00cf6c0 2024-09-16T15:50:40.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.614+0000 7f5ee7400640 1 --2- 172.21.15.151:0/956569372 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5ed0079fa0 0x7f5ed007c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:40.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.614+0000 7f5ee7400640 1 -- 172.21.15.151:0/956569372 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(92..92 src has 1..92) ==== 5821+0+0 (secure 0 0 0) 0x7f5edc09a240 con 0x7f5ef00cf6c0 2024-09-16T15:50:40.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.614+0000 7f5eef400640 1 --2- 172.21.15.151:0/956569372 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5ed0079fa0 0x7f5ed007c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:40.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.615+0000 7f5eef400640 1 --2- 172.21.15.151:0/956569372 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5ed0079fa0 0x7f5ed007c460 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0001640 tx=0x7f5ee0009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:40.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.616+0000 7f5ee7400640 1 -- 172.21.15.151:0/956569372 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f5edc064390 con 0x7f5ef00cf6c0 2024-09-16T15:50:40.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.974+0000 7f5ef6400640 1 -- 172.21.15.151:0/956569372 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone515512", "target": ["mon-mgr", ""]}) -- 0x7f5ec8000bc0 con 0x7f5ed0079fa0 2024-09-16T15:50:40.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.990+0000 7f5ee7400640 1 -- 172.21.15.151:0/956569372 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5ec8000bc0 con 0x7f5ed0079fa0 2024-09-16T15:50:40.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.993+0000 7f5ef6400640 1 -- 172.21.15.151:0/956569372 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5ed0079fa0 msgr2=0x7f5ed007c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:40.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.993+0000 7f5ef6400640 1 --2- 172.21.15.151:0/956569372 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5ed0079fa0 0x7f5ed007c460 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0001640 tx=0x7f5ee0009040 comp rx=0 tx=0).stop 2024-09-16T15:50:40.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.993+0000 7f5ef6400640 1 -- 172.21.15.151:0/956569372 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5ef00cf6c0 msgr2=0x7f5ef00aeb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:40.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.993+0000 7f5ef6400640 1 --2- 172.21.15.151:0/956569372 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5ef00cf6c0 0x7f5ef00aeb70 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5edc00ea20 tx=0x7f5edc00eee0 comp rx=0 tx=0).stop 2024-09-16T15:50:40.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.993+0000 7f5ef6400640 1 -- 172.21.15.151:0/956569372 shutdown_connections 2024-09-16T15:50:40.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.993+0000 7f5ef6400640 1 --2- 172.21.15.151:0/956569372 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5ed0079fa0 0x7f5ed007c460 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:40.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.993+0000 7f5ef6400640 1 --2- 172.21.15.151:0/956569372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ef01512b0 0x7f5ef01fc8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:40.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.993+0000 7f5ef6400640 1 --2- 172.21.15.151:0/956569372 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5ef00cf6c0 0x7f5ef00aeb70 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:40.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.993+0000 7f5ef6400640 1 --2- 172.21.15.151:0/956569372 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ef00b3e20 0x7f5ef009d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:40.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.993+0000 7f5ef6400640 1 -- 172.21.15.151:0/956569372 >> 172.21.15.151:0/956569372 conn(0x7f5ef00b18e0 msgr2=0x7f5ef00b0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:40.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.993+0000 7f5ef6400640 1 -- 172.21.15.151:0/956569372 shutdown_connections 2024-09-16T15:50:40.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:40.993+0000 7f5ef6400640 1 -- 172.21.15.151:0/956569372 wait complete. 2024-09-16T15:50:41.015 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone237619 2024-09-16T15:50:41.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.207+0000 7fe549000640 1 -- 172.21.15.151:0/1662058527 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe5441620e0 msgr2=0x7fe5440b0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:41.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.207+0000 7fe549000640 1 --2- 172.21.15.151:0/1662058527 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe5441620e0 0x7fe5440b0c30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe52c009900 tx=0x7fe52c02a650 comp rx=0 tx=0).stop 2024-09-16T15:50:41.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.208+0000 7fe549000640 1 -- 172.21.15.151:0/1662058527 shutdown_connections 2024-09-16T15:50:41.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.208+0000 7fe549000640 1 --2- 172.21.15.151:0/1662058527 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe5441620e0 0x7fe5440b0c30 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:41.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.208+0000 7fe549000640 1 -- 172.21.15.151:0/1662058527 >> 172.21.15.151:0/1662058527 conn(0x7fe5440d1e40 msgr2=0x7fe5440d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:41.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.208+0000 7fe549000640 1 -- 172.21.15.151:0/1662058527 shutdown_connections 2024-09-16T15:50:41.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.208+0000 7fe549000640 1 -- 172.21.15.151:0/1662058527 wait complete. 2024-09-16T15:50:41.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.210+0000 7fe549000640 1 Processor -- start 2024-09-16T15:50:41.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.210+0000 7fe549000640 1 -- start start 2024-09-16T15:50:41.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.211+0000 7fe549000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe5440d1a00 0x7fe544149e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:41.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.211+0000 7fe549000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5441620e0 0x7fe5440b5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:41.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.211+0000 7fe549000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe5441624c0 0x7fe5441549d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:41.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.211+0000 7fe549000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe54416d990 con 0x7fe5441624c0 2024-09-16T15:50:41.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.211+0000 7fe549000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe5440b01b0 con 0x7fe5441620e0 2024-09-16T15:50:41.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.211+0000 7fe549000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fe54416db10 con 0x7fe5440d1a00 2024-09-16T15:50:41.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.212+0000 7fe541e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe5440d1a00 0x7fe544149e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:41.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.212+0000 7fe542800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe5441624c0 0x7fe5441549d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:41.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.212+0000 7fe541e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe5440d1a00 0x7fe544149e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.151:39962/0 (socket says 172.21.15.151:39962) 2024-09-16T15:50:41.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.212+0000 7fe541e00640 1 -- 172.21.15.151:0/534353111 learned_addr learned my addr 172.21.15.151:0/534353111 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:41.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.212+0000 7fe541e00640 1 -- 172.21.15.151:0/534353111 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5441620e0 msgr2=0x7fe5440b5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:41.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.212+0000 7fe53be00640 1 --2- 172.21.15.151:0/534353111 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5441620e0 0x7fe5440b5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:41.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.212+0000 7fe541e00640 1 --2- 172.21.15.151:0/534353111 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5441620e0 0x7fe5440b5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:41.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.212+0000 7fe541e00640 1 -- 172.21.15.151:0/534353111 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe5441624c0 msgr2=0x7fe5441549d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:41.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.212+0000 7fe541e00640 1 --2- 172.21.15.151:0/534353111 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe5441624c0 0x7fe5441549d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:41.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.212+0000 7fe541e00640 1 -- 172.21.15.151:0/534353111 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe54414a860 con 0x7fe5440d1a00 2024-09-16T15:50:41.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.212+0000 7fe53be00640 1 --2- 172.21.15.151:0/534353111 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5441620e0 0x7fe5440b5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:50:41.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.212+0000 7fe542800640 1 --2- 172.21.15.151:0/534353111 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe5441624c0 0x7fe5441549d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:50:41.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.213+0000 7fe541e00640 1 --2- 172.21.15.151:0/534353111 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe5440d1a00 0x7fe544149e10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe52c009230 tx=0x7fe52c0038a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:41.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.213+0000 7fe53a000640 1 -- 172.21.15.151:0/534353111 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52c03f070 con 0x7fe5440d1a00 2024-09-16T15:50:41.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.213+0000 7fe53a000640 1 -- 172.21.15.151:0/534353111 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe52c0047a0 con 0x7fe5440d1a00 2024-09-16T15:50:41.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.213+0000 7fe53a000640 1 -- 172.21.15.151:0/534353111 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52c035030 con 0x7fe5440d1a00 2024-09-16T15:50:41.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.213+0000 7fe549000640 1 -- 172.21.15.151:0/534353111 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe54414ab40 con 0x7fe5440d1a00 2024-09-16T15:50:41.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.214+0000 7fe549000640 1 -- 172.21.15.151:0/534353111 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe54414a350 con 0x7fe5440d1a00 2024-09-16T15:50:41.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.215+0000 7fe549000640 1 -- 172.21.15.151:0/534353111 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5440b0c30 con 0x7fe5440d1a00 2024-09-16T15:50:41.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.217+0000 7fe53a000640 1 -- 172.21.15.151:0/534353111 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7fe52c042070 con 0x7fe5440d1a00 2024-09-16T15:50:41.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.218+0000 7fe53a000640 1 --2- 172.21.15.151:0/534353111 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe520079fd0 0x7fe52007c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:41.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.218+0000 7fe53a000640 1 -- 172.21.15.151:0/534353111 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(92..92 src has 1..92) ==== 5821+0+0 (secure 0 0 0) 0x7fe52c0ba290 con 0x7fe5440d1a00 2024-09-16T15:50:41.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.219+0000 7fe53be00640 1 --2- 172.21.15.151:0/534353111 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe520079fd0 0x7fe52007c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:41.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.221+0000 7fe53be00640 1 --2- 172.21.15.151:0/534353111 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe520079fd0 0x7fe52007c490 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fe54414ce10 tx=0x7fe534009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:41.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.225+0000 7fe53a000640 1 -- 172.21.15.151:0/534353111 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe52c083d90 con 0x7fe5440d1a00 2024-09-16T15:50:41.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.636+0000 7fe549000640 1 -- 172.21.15.151:0/534353111 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone237619", "target": ["mon-mgr", ""]}) -- 0x7fe544074bf0 con 0x7fe520079fd0 2024-09-16T15:50:41.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.676+0000 7fe53a000640 1 -- 172.21.15.151:0/534353111 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe544074bf0 con 0x7fe520079fd0 2024-09-16T15:50:41.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.680+0000 7fe549000640 1 -- 172.21.15.151:0/534353111 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe520079fd0 msgr2=0x7fe52007c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:41.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.680+0000 7fe549000640 1 --2- 172.21.15.151:0/534353111 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe520079fd0 0x7fe52007c490 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fe54414ce10 tx=0x7fe534009040 comp rx=0 tx=0).stop 2024-09-16T15:50:41.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.680+0000 7fe549000640 1 -- 172.21.15.151:0/534353111 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe5440d1a00 msgr2=0x7fe544149e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:41.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.680+0000 7fe549000640 1 --2- 172.21.15.151:0/534353111 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe5440d1a00 0x7fe544149e10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe52c009230 tx=0x7fe52c0038a0 comp rx=0 tx=0).stop 2024-09-16T15:50:41.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.680+0000 7fe549000640 1 -- 172.21.15.151:0/534353111 shutdown_connections 2024-09-16T15:50:41.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.680+0000 7fe549000640 1 --2- 172.21.15.151:0/534353111 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe520079fd0 0x7fe52007c490 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:41.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.680+0000 7fe549000640 1 --2- 172.21.15.151:0/534353111 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5441620e0 0x7fe5440b5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:41.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.680+0000 7fe549000640 1 --2- 172.21.15.151:0/534353111 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe5441624c0 0x7fe5441549d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:41.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.680+0000 7fe549000640 1 --2- 172.21.15.151:0/534353111 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe5440d1a00 0x7fe544149e10 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:41.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.680+0000 7fe549000640 1 -- 172.21.15.151:0/534353111 >> 172.21.15.151:0/534353111 conn(0x7fe5440d1e40 msgr2=0x7fe5440b3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:41.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.682+0000 7fe549000640 1 -- 172.21.15.151:0/534353111 shutdown_connections 2024-09-16T15:50:41.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.682+0000 7fe549000640 1 -- 172.21.15.151:0/534353111 wait complete. 2024-09-16T15:50:41.711 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone35339 2024-09-16T15:50:41.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.907+0000 7fddfce00640 1 -- 172.21.15.151:0/749871117 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fddf80cce30 msgr2=0x7fddf8070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:41.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.907+0000 7fddfce00640 1 --2- 172.21.15.151:0/749871117 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fddf80cce30 0x7fddf8070340 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fddf80d1520 tx=0x7fdde402a560 comp rx=0 tx=0).stop 2024-09-16T15:50:41.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.908+0000 7fddfce00640 1 -- 172.21.15.151:0/749871117 shutdown_connections 2024-09-16T15:50:41.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.908+0000 7fddfce00640 1 --2- 172.21.15.151:0/749871117 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fddf80cce30 0x7fddf8070340 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:41.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.908+0000 7fddfce00640 1 --2- 172.21.15.151:0/749871117 >> v2:172.21.15.167:3300/0 conn(0x7fddf8156460 0x7fddf8160280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:41.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.908+0000 7fddfce00640 1 -- 172.21.15.151:0/749871117 >> 172.21.15.151:0/749871117 conn(0x7fddf806cb70 msgr2=0x7fddf806cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:41.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.908+0000 7fddfce00640 1 -- 172.21.15.151:0/749871117 shutdown_connections 2024-09-16T15:50:41.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.908+0000 7fddfce00640 1 -- 172.21.15.151:0/749871117 wait complete. 2024-09-16T15:50:41.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.909+0000 7fddfce00640 1 Processor -- start 2024-09-16T15:50:41.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.909+0000 7fddfce00640 1 -- start start 2024-09-16T15:50:41.912 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.910+0000 7fddfce00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fddf8070880 0x7fddf81f6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:41.912 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.910+0000 7fddfce00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fddf80cce30 0x7fddf8140c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:41.912 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.910+0000 7fddfce00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fddf8156460 0x7fddf80b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:41.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.910+0000 7fddfce00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fddf8073780 con 0x7fddf80cce30 2024-09-16T15:50:41.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.910+0000 7fddfce00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fddf815d430 con 0x7fddf8070880 2024-09-16T15:50:41.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.910+0000 7fddfce00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fddf8073900 con 0x7fddf8156460 2024-09-16T15:50:41.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.911+0000 7fddf5e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fddf8070880 0x7fddf81f6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:41.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.911+0000 7fddf5400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fddf80cce30 0x7fddf8140c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:41.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.911+0000 7fddf5e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fddf8070880 0x7fddf81f6340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.151:56706/0 (socket says 172.21.15.151:56706) 2024-09-16T15:50:41.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.911+0000 7fddf5e00640 1 -- 172.21.15.151:0/3206823242 learned_addr learned my addr 172.21.15.151:0/3206823242 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:41.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.911+0000 7fddf6800640 1 --2- 172.21.15.151:0/3206823242 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fddf8156460 0x7fddf80b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:41.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.911+0000 7fddf5e00640 1 -- 172.21.15.151:0/3206823242 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fddf8156460 msgr2=0x7fddf80b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:41.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.911+0000 7fddf5e00640 1 --2- 172.21.15.151:0/3206823242 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fddf8156460 0x7fddf80b1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:41.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.911+0000 7fddf5e00640 1 -- 172.21.15.151:0/3206823242 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fddf80cce30 msgr2=0x7fddf8140c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:41.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.911+0000 7fddf5e00640 1 --2- 172.21.15.151:0/3206823242 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fddf80cce30 0x7fddf8140c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:41.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.911+0000 7fddf5e00640 1 -- 172.21.15.151:0/3206823242 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddf81f72a0 con 0x7fddf8070880 2024-09-16T15:50:41.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.911+0000 7fddf5400640 1 --2- 172.21.15.151:0/3206823242 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fddf80cce30 0x7fddf8140c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:50:41.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.911+0000 7fddf5e00640 1 --2- 172.21.15.151:0/3206823242 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fddf8070880 0x7fddf81f6340 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fdde800e7c0 tx=0x7fdde800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:41.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.912+0000 7fddee000640 1 -- 172.21.15.151:0/3206823242 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde800ccc0 con 0x7fddf8070880 2024-09-16T15:50:41.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.912+0000 7fddee000640 1 -- 172.21.15.151:0/3206823242 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdde8010430 con 0x7fddf8070880 2024-09-16T15:50:41.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.912+0000 7fddee000640 1 -- 172.21.15.151:0/3206823242 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde8010860 con 0x7fddf8070880 2024-09-16T15:50:41.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.912+0000 7fddfce00640 1 -- 172.21.15.151:0/3206823242 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddf81f6a00 con 0x7fddf8070880 2024-09-16T15:50:41.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.912+0000 7fddfce00640 1 -- 172.21.15.151:0/3206823242 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddf81f6fc0 con 0x7fddf8070880 2024-09-16T15:50:41.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.913+0000 7fddfce00640 1 -- 172.21.15.151:0/3206823242 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddf8073e20 con 0x7fddf8070880 2024-09-16T15:50:41.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.914+0000 7fddee000640 1 -- 172.21.15.151:0/3206823242 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7fdde8010a00 con 0x7fddf8070880 2024-09-16T15:50:41.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.915+0000 7fddee000640 1 --2- 172.21.15.151:0/3206823242 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fddd8079ed0 0x7fddd807c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:41.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.915+0000 7fddee000640 1 -- 172.21.15.151:0/3206823242 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5821+0+0 (secure 0 0 0) 0x7fdde8099fc0 con 0x7fddf8070880 2024-09-16T15:50:41.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.915+0000 7fddf5400640 1 --2- 172.21.15.151:0/3206823242 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fddd8079ed0 0x7fddd807c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:41.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.916+0000 7fddf5400640 1 --2- 172.21.15.151:0/3206823242 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fddd8079ed0 0x7fddd807c390 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fdde4033000 tx=0x7fdde4004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:41.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:41.920+0000 7fddee000640 1 -- 172.21.15.151:0/3206823242 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fdde8063a40 con 0x7fddf8070880 2024-09-16T15:50:42.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.131+0000 7fddfce00640 1 -- 172.21.15.151:0/3206823242 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone35339", "target": ["mon-mgr", ""]}) -- 0x7fddf80d1100 con 0x7fddd8079ed0 2024-09-16T15:50:42.161 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.160+0000 7fddee000640 1 -- 172.21.15.151:0/3206823242 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fddf80d1100 con 0x7fddd8079ed0 2024-09-16T15:50:42.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.165+0000 7fddfce00640 1 -- 172.21.15.151:0/3206823242 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fddd8079ed0 msgr2=0x7fddd807c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:42.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.165+0000 7fddfce00640 1 --2- 172.21.15.151:0/3206823242 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fddd8079ed0 0x7fddd807c390 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fdde4033000 tx=0x7fdde4004730 comp rx=0 tx=0).stop 2024-09-16T15:50:42.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.165+0000 7fddfce00640 1 -- 172.21.15.151:0/3206823242 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fddf8070880 msgr2=0x7fddf81f6340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:42.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.165+0000 7fddfce00640 1 --2- 172.21.15.151:0/3206823242 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fddf8070880 0x7fddf81f6340 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fdde800e7c0 tx=0x7fdde800ec80 comp rx=0 tx=0).stop 2024-09-16T15:50:42.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.165+0000 7fddfce00640 1 -- 172.21.15.151:0/3206823242 shutdown_connections 2024-09-16T15:50:42.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.165+0000 7fddfce00640 1 --2- 172.21.15.151:0/3206823242 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fddd8079ed0 0x7fddd807c390 secure :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fdde4033000 tx=0x7fdde4004730 comp rx=0 tx=0).stop 2024-09-16T15:50:42.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.165+0000 7fddfce00640 1 --2- 172.21.15.151:0/3206823242 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fddf8156460 0x7fddf80b1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:42.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.165+0000 7fddfce00640 1 --2- 172.21.15.151:0/3206823242 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fddf80cce30 0x7fddf8140c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:42.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.165+0000 7fddfce00640 1 --2- 172.21.15.151:0/3206823242 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fddf8070880 0x7fddf81f6340 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:42.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.165+0000 7fddfce00640 1 -- 172.21.15.151:0/3206823242 >> 172.21.15.151:0/3206823242 conn(0x7fddf806cb70 msgr2=0x7fddf81553f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:42.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.166+0000 7fddfce00640 1 -- 172.21.15.151:0/3206823242 shutdown_connections 2024-09-16T15:50:42.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.166+0000 7fddfce00640 1 -- 172.21.15.151:0/3206823242 wait complete. 2024-09-16T15:50:42.194 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone57948 2024-09-16T15:50:42.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.413+0000 7f889b000640 1 -- 172.21.15.151:0/1342696768 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8894070380 msgr2=0x7f88940cb1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:42.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.413+0000 7f889b000640 1 --2- 172.21.15.151:0/1342696768 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8894070380 0x7f88940cb1c0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f889406f8d0 tx=0x7f888c02aab0 comp rx=0 tx=0).stop 2024-09-16T15:50:42.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.413+0000 7f889b000640 1 -- 172.21.15.151:0/1342696768 shutdown_connections 2024-09-16T15:50:42.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.413+0000 7f889b000640 1 --2- 172.21.15.151:0/1342696768 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8894070380 0x7f88940cb1c0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:42.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.413+0000 7f889b000640 1 --2- 172.21.15.151:0/1342696768 >> v2:172.21.15.167:3300/0 conn(0x7f889406e590 0x7f889406fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:42.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.413+0000 7f889b000640 1 -- 172.21.15.151:0/1342696768 >> 172.21.15.151:0/1342696768 conn(0x7f88940cfa70 msgr2=0x7f88940cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:42.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.413+0000 7f889b000640 1 -- 172.21.15.151:0/1342696768 shutdown_connections 2024-09-16T15:50:42.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.413+0000 7f889b000640 1 -- 172.21.15.151:0/1342696768 wait complete. 2024-09-16T15:50:42.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.415+0000 7f889b000640 1 Processor -- start 2024-09-16T15:50:42.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.415+0000 7f889b000640 1 -- start start 2024-09-16T15:50:42.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.416+0000 7f889b000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f889406e590 0x7f889415cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:42.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.416+0000 7f889b000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8894070380 0x7f88940ce900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:42.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.416+0000 7f889b000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f889415bda0 0x7f88940c8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:42.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.416+0000 7f889b000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f88940cba90 con 0x7f889415bda0 2024-09-16T15:50:42.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.416+0000 7f889b000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f88940cb910 con 0x7f8894070380 2024-09-16T15:50:42.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.416+0000 7f889b000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f88940cbc10 con 0x7f889406e590 2024-09-16T15:50:42.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.416+0000 7f889a600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f889415bda0 0x7f88940c8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:42.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.416+0000 7f889a600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f889415bda0 0x7f88940c8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:54044/0 (socket says 172.21.15.151:54044) 2024-09-16T15:50:42.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.416+0000 7f889a600640 1 -- 172.21.15.151:0/1467184971 learned_addr learned my addr 172.21.15.151:0/1467184971 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:42.419 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.416+0000 7f8899c00640 1 --2- 172.21.15.151:0/1467184971 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f889406e590 0x7f889415cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:42.419 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.416+0000 7f8899200640 1 --2- 172.21.15.151:0/1467184971 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8894070380 0x7f88940ce900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:42.419 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.417+0000 7f889a600640 1 -- 172.21.15.151:0/1467184971 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f889406e590 msgr2=0x7f889415cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:42.419 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.417+0000 7f889a600640 1 --2- 172.21.15.151:0/1467184971 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f889406e590 0x7f889415cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:42.419 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.417+0000 7f889a600640 1 -- 172.21.15.151:0/1467184971 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8894070380 msgr2=0x7f88940ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:42.419 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.417+0000 7f889a600640 1 --2- 172.21.15.151:0/1467184971 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8894070380 0x7f88940ce900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:42.419 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.417+0000 7f889a600640 1 -- 172.21.15.151:0/1467184971 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f889415d1e0 con 0x7f889415bda0 2024-09-16T15:50:42.419 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.417+0000 7f889a600640 1 --2- 172.21.15.151:0/1467184971 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f889415bda0 0x7f88940c8260 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f888c009d60 tx=0x7f888c003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:42.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.417+0000 7f8891600640 1 -- 172.21.15.151:0/1467184971 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888c046070 con 0x7f889415bda0 2024-09-16T15:50:42.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.417+0000 7f8891600640 1 -- 172.21.15.151:0/1467184971 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f888c003e50 con 0x7f889415bda0 2024-09-16T15:50:42.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.417+0000 7f8891600640 1 -- 172.21.15.151:0/1467184971 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888c0044b0 con 0x7f889415bda0 2024-09-16T15:50:42.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.417+0000 7f889b000640 1 -- 172.21.15.151:0/1467184971 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88940c8bd0 con 0x7f889415bda0 2024-09-16T15:50:42.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.418+0000 7f889b000640 1 -- 172.21.15.151:0/1467184971 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f889415c4a0 con 0x7f889415bda0 2024-09-16T15:50:42.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.418+0000 7f889b000640 1 -- 172.21.15.151:0/1467184971 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88940cc130 con 0x7f889415bda0 2024-09-16T15:50:42.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.420+0000 7f8891600640 1 -- 172.21.15.151:0/1467184971 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7f888c049070 con 0x7f889415bda0 2024-09-16T15:50:42.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.420+0000 7f8891600640 1 --2- 172.21.15.151:0/1467184971 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8874079f00 0x7f887407c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:42.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.421+0000 7f8891600640 1 -- 172.21.15.151:0/1467184971 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5821+0+0 (secure 0 0 0) 0x7f888c0ba4d0 con 0x7f889415bda0 2024-09-16T15:50:42.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.421+0000 7f8899c00640 1 --2- 172.21.15.151:0/1467184971 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8874079f00 0x7f887407c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:42.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.421+0000 7f8899c00640 1 --2- 172.21.15.151:0/1467184971 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8874079f00 0x7f887407c3c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f8888001720 tx=0x7f8888009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:42.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.425+0000 7f8891600640 1 -- 172.21.15.151:0/1467184971 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f888c032030 con 0x7f889415bda0 2024-09-16T15:50:42.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.636+0000 7f889b000640 1 -- 172.21.15.151:0/1467184971 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone57948", "target": ["mon-mgr", ""]}) -- 0x7f88940cd210 con 0x7f8874079f00 2024-09-16T15:50:42.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.663+0000 7f8891600640 1 -- 172.21.15.151:0/1467184971 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f88940cd210 con 0x7f8874079f00 2024-09-16T15:50:42.668 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.666+0000 7f889b000640 1 -- 172.21.15.151:0/1467184971 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8874079f00 msgr2=0x7f887407c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:42.668 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.666+0000 7f889b000640 1 --2- 172.21.15.151:0/1467184971 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8874079f00 0x7f887407c3c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f8888001720 tx=0x7f8888009040 comp rx=0 tx=0).stop 2024-09-16T15:50:42.668 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.666+0000 7f889b000640 1 -- 172.21.15.151:0/1467184971 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f889415bda0 msgr2=0x7f88940c8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:42.668 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.666+0000 7f889b000640 1 --2- 172.21.15.151:0/1467184971 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f889415bda0 0x7f88940c8260 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f888c009d60 tx=0x7f888c003ce0 comp rx=0 tx=0).stop 2024-09-16T15:50:42.668 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.666+0000 7f889b000640 1 -- 172.21.15.151:0/1467184971 shutdown_connections 2024-09-16T15:50:42.668 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.666+0000 7f889b000640 1 --2- 172.21.15.151:0/1467184971 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8874079f00 0x7f887407c3c0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:42.668 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.666+0000 7f889b000640 1 --2- 172.21.15.151:0/1467184971 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8894070380 0x7f88940ce900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:42.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.666+0000 7f889b000640 1 --2- 172.21.15.151:0/1467184971 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f889415bda0 0x7f88940c8260 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:42.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.666+0000 7f889b000640 1 --2- 172.21.15.151:0/1467184971 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f889406e590 0x7f889415cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:42.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.666+0000 7f889b000640 1 -- 172.21.15.151:0/1467184971 >> 172.21.15.151:0/1467184971 conn(0x7f88940cfa70 msgr2=0x7f88940ad880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:42.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.667+0000 7f889b000640 1 -- 172.21.15.151:0/1467184971 shutdown_connections 2024-09-16T15:50:42.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.667+0000 7f889b000640 1 -- 172.21.15.151:0/1467184971 wait complete. 2024-09-16T15:50:42.684 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone15034 2024-09-16T15:50:42.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.850+0000 7fcd07200640 1 -- 172.21.15.151:0/3983131231 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd000b2ff0 msgr2=0x7fcd000743d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:42.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.850+0000 7fcd07200640 1 --2- 172.21.15.151:0/3983131231 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd000b2ff0 0x7fcd000743d0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fccf0009860 tx=0x7fccf002a4b0 comp rx=0 tx=0).stop 2024-09-16T15:50:42.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.850+0000 7fcd07200640 1 -- 172.21.15.151:0/3983131231 shutdown_connections 2024-09-16T15:50:42.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.850+0000 7fcd07200640 1 --2- 172.21.15.151:0/3983131231 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd000b2ff0 0x7fcd000743d0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:42.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.850+0000 7fcd07200640 1 --2- 172.21.15.151:0/3983131231 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd000b4bd0 0x7fcd0015ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:42.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.850+0000 7fcd07200640 1 -- 172.21.15.151:0/3983131231 >> 172.21.15.151:0/3983131231 conn(0x7fcd000b1cf0 msgr2=0x7fcd000b20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:42.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.850+0000 7fcd07200640 1 -- 172.21.15.151:0/3983131231 shutdown_connections 2024-09-16T15:50:42.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.851+0000 7fcd07200640 1 -- 172.21.15.151:0/3983131231 wait complete. 2024-09-16T15:50:42.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.852+0000 7fcd07200640 1 Processor -- start 2024-09-16T15:50:42.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.852+0000 7fcd07200640 1 -- start start 2024-09-16T15:50:42.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.853+0000 7fcd07200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd000b2ff0 0x7fcd001504e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:42.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.853+0000 7fcd07200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd000b4bd0 0x7fcd00155620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:42.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.853+0000 7fcd07200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd000b5bb0 0x7fcd001e7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:42.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.853+0000 7fcd07200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0015ecf0 con 0x7fcd000b2ff0 2024-09-16T15:50:42.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.853+0000 7fcd07200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcd00074d00 con 0x7fcd000b4bd0 2024-09-16T15:50:42.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.853+0000 7fcd07200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fcd0015ee70 con 0x7fcd000b5bb0 2024-09-16T15:50:42.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.853+0000 7fcd05e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd000b2ff0 0x7fcd001504e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:42.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.853+0000 7fcd05e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd000b2ff0 0x7fcd001504e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:45768/0 (socket says 172.21.15.151:45768) 2024-09-16T15:50:42.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.854+0000 7fcd05e00640 1 -- 172.21.15.151:0/214773595 learned_addr learned my addr 172.21.15.151:0/214773595 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:42.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.854+0000 7fcd06800640 1 --2- 172.21.15.151:0/214773595 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd000b5bb0 0x7fcd001e7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:42.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.854+0000 7fcd05400640 1 --2- 172.21.15.151:0/214773595 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd000b4bd0 0x7fcd00155620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:42.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.854+0000 7fcd05e00640 1 -- 172.21.15.151:0/214773595 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd000b5bb0 msgr2=0x7fcd001e7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:42.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.854+0000 7fcd05e00640 1 --2- 172.21.15.151:0/214773595 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd000b5bb0 0x7fcd001e7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:42.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.854+0000 7fcd05e00640 1 -- 172.21.15.151:0/214773595 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd000b4bd0 msgr2=0x7fcd00155620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:42.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.854+0000 7fcd05e00640 1 --2- 172.21.15.151:0/214773595 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd000b4bd0 0x7fcd00155620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:42.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.854+0000 7fcd05e00640 1 -- 172.21.15.151:0/214773595 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd001f6860 con 0x7fcd000b2ff0 2024-09-16T15:50:42.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.854+0000 7fcd05400640 1 --2- 172.21.15.151:0/214773595 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd000b4bd0 0x7fcd00155620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:50:42.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.854+0000 7fcd05e00640 1 --2- 172.21.15.151:0/214773595 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd000b2ff0 0x7fcd001504e0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fccf400ea20 tx=0x7fccf400eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:42.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.854+0000 7fccfa000640 1 -- 172.21.15.151:0/214773595 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf400cd80 con 0x7fcd000b2ff0 2024-09-16T15:50:42.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.854+0000 7fccfa000640 1 -- 172.21.15.151:0/214773595 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fccf4010430 con 0x7fcd000b2ff0 2024-09-16T15:50:42.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.854+0000 7fcd07200640 1 -- 172.21.15.151:0/214773595 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd001f6b40 con 0x7fcd000b2ff0 2024-09-16T15:50:42.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.855+0000 7fccfa000640 1 -- 172.21.15.151:0/214773595 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf4010840 con 0x7fcd000b2ff0 2024-09-16T15:50:42.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.855+0000 7fcd07200640 1 -- 172.21.15.151:0/214773595 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd001f7100 con 0x7fcd000b2ff0 2024-09-16T15:50:42.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.856+0000 7fcd07200640 1 -- 172.21.15.151:0/214773595 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd0015f390 con 0x7fcd000b2ff0 2024-09-16T15:50:42.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.857+0000 7fccfa000640 1 -- 172.21.15.151:0/214773595 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7fccf40109e0 con 0x7fcd000b2ff0 2024-09-16T15:50:42.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.858+0000 7fccfa000640 1 --2- 172.21.15.151:0/214773595 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcce4079ed0 0x7fcce407c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:42.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.858+0000 7fccfa000640 1 -- 172.21.15.151:0/214773595 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5821+0+0 (secure 0 0 0) 0x7fccf409a1d0 con 0x7fcd000b2ff0 2024-09-16T15:50:42.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.858+0000 7fcd05400640 1 --2- 172.21.15.151:0/214773595 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcce4079ed0 0x7fcce407c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:42.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.859+0000 7fcd05400640 1 --2- 172.21.15.151:0/214773595 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcce4079ed0 0x7fcce407c390 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fccf0034000 tx=0x7fccf0004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:42.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:42.862+0000 7fccfa000640 1 -- 172.21.15.151:0/214773595 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fccf40642a0 con 0x7fcd000b2ff0 2024-09-16T15:50:43.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.073+0000 7fcd07200640 1 -- 172.21.15.151:0/214773595 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone15034", "target": ["mon-mgr", ""]}) -- 0x7fcd000afa40 con 0x7fcce4079ed0 2024-09-16T15:50:43.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.092+0000 7fccfa000640 1 -- 172.21.15.151:0/214773595 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcd000afa40 con 0x7fcce4079ed0 2024-09-16T15:50:43.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.095+0000 7fcd07200640 1 -- 172.21.15.151:0/214773595 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcce4079ed0 msgr2=0x7fcce407c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:43.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.095+0000 7fcd07200640 1 --2- 172.21.15.151:0/214773595 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcce4079ed0 0x7fcce407c390 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fccf0034000 tx=0x7fccf0004730 comp rx=0 tx=0).stop 2024-09-16T15:50:43.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.095+0000 7fcd07200640 1 -- 172.21.15.151:0/214773595 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd000b2ff0 msgr2=0x7fcd001504e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:43.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.095+0000 7fcd07200640 1 --2- 172.21.15.151:0/214773595 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd000b2ff0 0x7fcd001504e0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fccf400ea20 tx=0x7fccf400eee0 comp rx=0 tx=0).stop 2024-09-16T15:50:43.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.095+0000 7fcd07200640 1 -- 172.21.15.151:0/214773595 shutdown_connections 2024-09-16T15:50:43.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.096+0000 7fcd07200640 1 --2- 172.21.15.151:0/214773595 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcce4079ed0 0x7fcce407c390 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:43.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.096+0000 7fcd07200640 1 --2- 172.21.15.151:0/214773595 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd000b4bd0 0x7fcd00155620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:43.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.096+0000 7fcd07200640 1 --2- 172.21.15.151:0/214773595 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcd000b5bb0 0x7fcd001e7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:43.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.096+0000 7fcd07200640 1 --2- 172.21.15.151:0/214773595 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd000b2ff0 0x7fcd001504e0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:43.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.096+0000 7fcd07200640 1 -- 172.21.15.151:0/214773595 >> 172.21.15.151:0/214773595 conn(0x7fcd000b1cf0 msgr2=0x7fcd000b20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:43.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.096+0000 7fcd07200640 1 -- 172.21.15.151:0/214773595 shutdown_connections 2024-09-16T15:50:43.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.096+0000 7fcd07200640 1 -- 172.21.15.151:0/214773595 wait complete. 2024-09-16T15:50:43.113 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone880714 2024-09-16T15:50:43.368 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.367+0000 7f564e600640 1 -- 172.21.15.151:0/2094079511 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f56481512b0 msgr2=0x7f56480cefb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:43.368 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.367+0000 7f564e600640 1 --2- 172.21.15.151:0/2094079511 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f56481512b0 0x7f56480cefb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f563800b9c0 tx=0x7f563802c200 comp rx=0 tx=0).stop 2024-09-16T15:50:43.368 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.367+0000 7f564e600640 1 -- 172.21.15.151:0/2094079511 shutdown_connections 2024-09-16T15:50:43.368 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.367+0000 7f564e600640 1 --2- 172.21.15.151:0/2094079511 >> v2:172.21.15.151:3300/0 conn(0x7f56480b3e20 0x7f56480cf4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:43.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.367+0000 7f564e600640 1 --2- 172.21.15.151:0/2094079511 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f56481512b0 0x7f56480cefb0 secure :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f563800b9c0 tx=0x7f563802c200 comp rx=0 tx=0).stop 2024-09-16T15:50:43.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.367+0000 7f564e600640 1 -- 172.21.15.151:0/2094079511 >> 172.21.15.151:0/2094079511 conn(0x7f56480b18e0 msgr2=0x7f56480b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:43.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.369+0000 7f564e600640 1 -- 172.21.15.151:0/2094079511 shutdown_connections 2024-09-16T15:50:43.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.369+0000 7f564e600640 1 -- 172.21.15.151:0/2094079511 wait complete. 2024-09-16T15:50:43.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.372+0000 7f564e600640 1 Processor -- start 2024-09-16T15:50:43.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.372+0000 7f564e600640 1 -- start start 2024-09-16T15:50:43.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.374+0000 7f564e600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56480708b0 0x7f5648150a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:43.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.374+0000 7f564e600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56480b3e20 0x7f564815e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:43.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.375+0000 7f564e600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5648153890 0x7f5648153d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:43.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.375+0000 7f564e600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f56480cc7b0 con 0x7f56480b3e20 2024-09-16T15:50:43.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.375+0000 7f564e600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f56480cc630 con 0x7f56480708b0 2024-09-16T15:50:43.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.375+0000 7f564e600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f56480cc930 con 0x7f5648153890 2024-09-16T15:50:43.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.375+0000 7f5646a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56480b3e20 0x7f564815e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:43.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.375+0000 7f5647400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56480708b0 0x7f5648150a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:43.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.375+0000 7f5647400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56480708b0 0x7f5648150a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.151:47648/0 (socket says 172.21.15.151:47648) 2024-09-16T15:50:43.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.376+0000 7f5647e00640 1 --2- 172.21.15.151:0/2703658617 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5648153890 0x7f5648153d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:43.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.376+0000 7f5647400640 1 -- 172.21.15.151:0/2703658617 learned_addr learned my addr 172.21.15.151:0/2703658617 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:43.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.376+0000 7f5647400640 1 -- 172.21.15.151:0/2703658617 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5648153890 msgr2=0x7f5648153d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:43.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.376+0000 7f5647400640 1 --2- 172.21.15.151:0/2703658617 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5648153890 0x7f5648153d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:43.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.376+0000 7f5647400640 1 -- 172.21.15.151:0/2703658617 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56480b3e20 msgr2=0x7f564815e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:43.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.377+0000 7f5647400640 1 --2- 172.21.15.151:0/2703658617 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56480b3e20 0x7f564815e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:43.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.377+0000 7f5647400640 1 -- 172.21.15.151:0/2703658617 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f564809acc0 con 0x7f56480708b0 2024-09-16T15:50:43.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.377+0000 7f5646a00640 1 --2- 172.21.15.151:0/2703658617 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56480b3e20 0x7f564815e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:50:43.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.377+0000 7f5647400640 1 --2- 172.21.15.151:0/2703658617 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56480708b0 0x7f5648150a10 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f563800bae0 tx=0x7f5638032570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:43.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.378+0000 7f563f400640 1 -- 172.21.15.151:0/2703658617 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5638043070 con 0x7f56480708b0 2024-09-16T15:50:43.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.378+0000 7f563f400640 1 -- 172.21.15.151:0/2703658617 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5638004130 con 0x7f56480708b0 2024-09-16T15:50:43.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.378+0000 7f563f400640 1 -- 172.21.15.151:0/2703658617 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5638039040 con 0x7f56480708b0 2024-09-16T15:50:43.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.378+0000 7f564e600640 1 -- 172.21.15.151:0/2703658617 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f564815ea70 con 0x7f56480708b0 2024-09-16T15:50:43.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.378+0000 7f564e600640 1 -- 172.21.15.151:0/2703658617 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f564809d0c0 con 0x7f56480708b0 2024-09-16T15:50:43.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.380+0000 7f564e600640 1 -- 172.21.15.151:0/2703658617 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5648069360 con 0x7f56480708b0 2024-09-16T15:50:43.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.386+0000 7f563f400640 1 -- 172.21.15.151:0/2703658617 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7f5638046070 con 0x7f56480708b0 2024-09-16T15:50:43.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.387+0000 7f563f400640 1 --2- 172.21.15.151:0/2703658617 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5628079fd0 0x7f562807c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:43.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.387+0000 7f563f400640 1 -- 172.21.15.151:0/2703658617 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5821+0+0 (secure 0 0 0) 0x7f56380b9930 con 0x7f56480708b0 2024-09-16T15:50:43.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.388+0000 7f5646a00640 1 --2- 172.21.15.151:0/2703658617 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5628079fd0 0x7f562807c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:43.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.388+0000 7f563f400640 1 -- 172.21.15.151:0/2703658617 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f563804d080 con 0x7f56480708b0 2024-09-16T15:50:43.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.389+0000 7f5646a00640 1 --2- 172.21.15.151:0/2703658617 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5628079fd0 0x7f562807c490 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f56480b4f10 tx=0x7f5634007e50 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:43.739 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.738+0000 7f564e600640 1 -- 172.21.15.151:0/2703658617 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone880714", "target": ["mon-mgr", ""]}) -- 0x7f5648154260 con 0x7f5628079fd0 2024-09-16T15:50:43.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.756+0000 7f563f400640 1 -- 172.21.15.151:0/2703658617 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5634005470 con 0x7f5628079fd0 2024-09-16T15:50:43.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.759+0000 7f564e600640 1 -- 172.21.15.151:0/2703658617 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5628079fd0 msgr2=0x7f562807c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:43.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.759+0000 7f564e600640 1 --2- 172.21.15.151:0/2703658617 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5628079fd0 0x7f562807c490 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f56480b4f10 tx=0x7f5634007e50 comp rx=0 tx=0).stop 2024-09-16T15:50:43.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.759+0000 7f564e600640 1 -- 172.21.15.151:0/2703658617 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56480708b0 msgr2=0x7f5648150a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:43.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.759+0000 7f564e600640 1 --2- 172.21.15.151:0/2703658617 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56480708b0 0x7f5648150a10 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f563800bae0 tx=0x7f5638032570 comp rx=0 tx=0).stop 2024-09-16T15:50:43.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.759+0000 7f564e600640 1 -- 172.21.15.151:0/2703658617 shutdown_connections 2024-09-16T15:50:43.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.759+0000 7f564e600640 1 --2- 172.21.15.151:0/2703658617 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5628079fd0 0x7f562807c490 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:43.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.759+0000 7f564e600640 1 --2- 172.21.15.151:0/2703658617 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5648153890 0x7f5648153d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:43.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.759+0000 7f564e600640 1 --2- 172.21.15.151:0/2703658617 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56480b3e20 0x7f564815e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:43.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.759+0000 7f564e600640 1 --2- 172.21.15.151:0/2703658617 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56480708b0 0x7f5648150a10 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:43.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.759+0000 7f564e600640 1 -- 172.21.15.151:0/2703658617 >> 172.21.15.151:0/2703658617 conn(0x7f56480b18e0 msgr2=0x7f56480af750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:43.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.759+0000 7f564e600640 1 -- 172.21.15.151:0/2703658617 shutdown_connections 2024-09-16T15:50:43.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.759+0000 7f564e600640 1 -- 172.21.15.151:0/2703658617 wait complete. 2024-09-16T15:50:43.778 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone77745 2024-09-16T15:50:43.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.942+0000 7f4ed2e00640 1 -- 172.21.15.151:0/3508946906 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ecc159aa0 msgr2=0x7f4ecc0741a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:43.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.942+0000 7f4ed2e00640 1 --2- 172.21.15.151:0/3508946906 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ecc159aa0 0x7f4ecc0741a0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc009830 tx=0x7f4ebc02a5b0 comp rx=0 tx=0).stop 2024-09-16T15:50:43.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.942+0000 7f4ed2e00640 1 -- 172.21.15.151:0/3508946906 shutdown_connections 2024-09-16T15:50:43.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.942+0000 7f4ed2e00640 1 --2- 172.21.15.151:0/3508946906 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ecc159aa0 0x7f4ecc0741a0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:43.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.942+0000 7f4ed2e00640 1 -- 172.21.15.151:0/3508946906 >> 172.21.15.151:0/3508946906 conn(0x7f4ecc0b18e0 msgr2=0x7f4ecc0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:43.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.942+0000 7f4ed2e00640 1 -- 172.21.15.151:0/3508946906 shutdown_connections 2024-09-16T15:50:43.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.942+0000 7f4ed2e00640 1 -- 172.21.15.151:0/3508946906 wait complete. 2024-09-16T15:50:43.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.944+0000 7f4ed2e00640 1 Processor -- start 2024-09-16T15:50:43.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.944+0000 7f4ed2e00640 1 -- start start 2024-09-16T15:50:43.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.945+0000 7f4ed2e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ecc06f220 0x7f4ecc1f8f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:43.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.945+0000 7f4ed2e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ecc159aa0 0x7f4ecc1547b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:43.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.945+0000 7f4ed2e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4ecc15cff0 0x7f4ecc1ee960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:43.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.945+0000 7f4ed2e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4ecc06f6b0 con 0x7f4ecc06f220 2024-09-16T15:50:43.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.945+0000 7f4ed2e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4ecc0ccc50 con 0x7f4ecc159aa0 2024-09-16T15:50:43.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.945+0000 7f4ed2e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f4ecc06f830 con 0x7f4ecc15cff0 2024-09-16T15:50:43.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.945+0000 7f4ecbe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ecc06f220 0x7f4ecc1f8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:43.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.945+0000 7f4ecbe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ecc06f220 0x7f4ecc1f8f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:45822/0 (socket says 172.21.15.151:45822) 2024-09-16T15:50:43.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.945+0000 7f4ecbe00640 1 -- 172.21.15.151:0/4069310704 learned_addr learned my addr 172.21.15.151:0/4069310704 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:43.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.945+0000 7f4ed0e00640 1 --2- 172.21.15.151:0/4069310704 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4ecc15cff0 0x7f4ecc1ee960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:43.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.945+0000 7f4ecb400640 1 --2- 172.21.15.151:0/4069310704 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ecc159aa0 0x7f4ecc1547b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:43.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.946+0000 7f4ecbe00640 1 -- 172.21.15.151:0/4069310704 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4ecc15cff0 msgr2=0x7f4ecc1ee960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:43.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.946+0000 7f4ecbe00640 1 --2- 172.21.15.151:0/4069310704 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4ecc15cff0 0x7f4ecc1ee960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:43.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.946+0000 7f4ecbe00640 1 -- 172.21.15.151:0/4069310704 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ecc159aa0 msgr2=0x7f4ecc1547b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:43.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.946+0000 7f4ecbe00640 1 --2- 172.21.15.151:0/4069310704 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ecc159aa0 0x7f4ecc1547b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:43.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.946+0000 7f4ecbe00640 1 -- 172.21.15.151:0/4069310704 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ecc1f5420 con 0x7f4ecc06f220 2024-09-16T15:50:43.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.946+0000 7f4ecbe00640 1 --2- 172.21.15.151:0/4069310704 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ecc06f220 0x7f4ecc1f8f20 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc009950 tx=0x7f4ebc032c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:43.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.946+0000 7f4ec3e00640 1 -- 172.21.15.151:0/4069310704 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc040070 con 0x7f4ecc06f220 2024-09-16T15:50:43.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.946+0000 7f4ec3e00640 1 -- 172.21.15.151:0/4069310704 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4ebc005310 con 0x7f4ecc06f220 2024-09-16T15:50:43.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.946+0000 7f4ed2e00640 1 -- 172.21.15.151:0/4069310704 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ecc1f5700 con 0x7f4ecc06f220 2024-09-16T15:50:43.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.946+0000 7f4ec3e00640 1 -- 172.21.15.151:0/4069310704 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc0047a0 con 0x7f4ecc06f220 2024-09-16T15:50:43.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.946+0000 7f4ed2e00640 1 -- 172.21.15.151:0/4069310704 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ecc1f5cc0 con 0x7f4ecc06f220 2024-09-16T15:50:43.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.947+0000 7f4ed2e00640 1 -- 172.21.15.151:0/4069310704 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ecc0cc190 con 0x7f4ecc06f220 2024-09-16T15:50:43.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.948+0000 7f4ec3e00640 1 -- 172.21.15.151:0/4069310704 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7f4ebc039040 con 0x7f4ecc06f220 2024-09-16T15:50:43.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.949+0000 7f4ec3e00640 1 --2- 172.21.15.151:0/4069310704 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4eac079ed0 0x7f4eac07c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:43.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.950+0000 7f4ec3e00640 1 -- 172.21.15.151:0/4069310704 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5821+0+0 (secure 0 0 0) 0x7f4ebc0ba840 con 0x7f4ecc06f220 2024-09-16T15:50:43.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.950+0000 7f4ecb400640 1 --2- 172.21.15.151:0/4069310704 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4eac079ed0 0x7f4eac07c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:43.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.950+0000 7f4ecb400640 1 --2- 172.21.15.151:0/4069310704 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4eac079ed0 0x7f4eac07c390 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc159470 tx=0x7f4eb8009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:43.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:43.954+0000 7f4ec3e00640 1 -- 172.21.15.151:0/4069310704 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4ebc0842c0 con 0x7f4ecc06f220 2024-09-16T15:50:44.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:44.184+0000 7f4ed2e00640 1 -- 172.21.15.151:0/4069310704 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone77745", "target": ["mon-mgr", ""]}) -- 0x7f4ecc0b2160 con 0x7f4eac079ed0 2024-09-16T15:50:44.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:44.215+0000 7f4ec3e00640 1 -- 172.21.15.151:0/4069310704 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4ecc0b2160 con 0x7f4eac079ed0 2024-09-16T15:50:44.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:44.220+0000 7f4ed2e00640 1 -- 172.21.15.151:0/4069310704 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4eac079ed0 msgr2=0x7f4eac07c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:44.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:44.220+0000 7f4ed2e00640 1 --2- 172.21.15.151:0/4069310704 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4eac079ed0 0x7f4eac07c390 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc159470 tx=0x7f4eb8009040 comp rx=0 tx=0).stop 2024-09-16T15:50:44.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:44.220+0000 7f4ed2e00640 1 -- 172.21.15.151:0/4069310704 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ecc06f220 msgr2=0x7f4ecc1f8f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:44.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:44.220+0000 7f4ed2e00640 1 --2- 172.21.15.151:0/4069310704 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ecc06f220 0x7f4ecc1f8f20 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc009950 tx=0x7f4ebc032c00 comp rx=0 tx=0).stop 2024-09-16T15:50:44.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:44.220+0000 7f4ed2e00640 1 -- 172.21.15.151:0/4069310704 shutdown_connections 2024-09-16T15:50:44.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:44.220+0000 7f4ed2e00640 1 --2- 172.21.15.151:0/4069310704 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4eac079ed0 0x7f4eac07c390 secure :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc159470 tx=0x7f4eb8009040 comp rx=0 tx=0).stop 2024-09-16T15:50:44.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:44.220+0000 7f4ed2e00640 1 --2- 172.21.15.151:0/4069310704 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ecc159aa0 0x7f4ecc1547b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:44.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:44.220+0000 7f4ed2e00640 1 --2- 172.21.15.151:0/4069310704 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4ecc15cff0 0x7f4ecc1ee960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:44.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:44.220+0000 7f4ed2e00640 1 --2- 172.21.15.151:0/4069310704 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ecc06f220 0x7f4ecc1f8f20 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:44.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:44.220+0000 7f4ed2e00640 1 -- 172.21.15.151:0/4069310704 >> 172.21.15.151:0/4069310704 conn(0x7f4ecc0b18e0 msgr2=0x7f4ecc15ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:44.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:44.221+0000 7f4ed2e00640 1 -- 172.21.15.151:0/4069310704 shutdown_connections 2024-09-16T15:50:44.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:44.221+0000 7f4ed2e00640 1 -- 172.21.15.151:0/4069310704 wait complete. 2024-09-16T15:50:44.253 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone23331 2024-09-16T15:50:44.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:44.528+0000 7f956dc00640 1 -- 172.21.15.151:0/181398892 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95680cf170 msgr2=0x7f95680aec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:44.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:44.528+0000 7f956dc00640 1 --2- 172.21.15.151:0/181398892 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95680cf170 0x7f95680aec10 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f9560005fd0 tx=0x7f956002a8a0 comp rx=0 tx=0).stop 2024-09-16T15:50:44.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:44.528+0000 7f956dc00640 1 -- 172.21.15.151:0/181398892 shutdown_connections 2024-09-16T15:50:44.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:44.528+0000 7f956dc00640 1 --2- 172.21.15.151:0/181398892 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95680cf170 0x7f95680aec10 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:44.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:44.528+0000 7f956dc00640 1 --2- 172.21.15.151:0/181398892 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f956815aed0 0x7f95680ce530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:44.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:44.528+0000 7f956dc00640 1 -- 172.21.15.151:0/181398892 >> 172.21.15.151:0/181398892 conn(0x7f95680cfa70 msgr2=0x7f95680cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:44.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:44.528+0000 7f956dc00640 1 -- 172.21.15.151:0/181398892 shutdown_connections 2024-09-16T15:50:44.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:44.529+0000 7f956dc00640 1 -- 172.21.15.151:0/181398892 wait complete. 2024-09-16T15:50:44.532 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:44.531+0000 7f956dc00640 1 Processor -- start 2024-09-16T15:50:44.532 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:44.531+0000 7f956dc00640 1 -- start start 2024-09-16T15:50:44.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:44.532+0000 7f956dc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f95680706b0 0x7f956814b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:44.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:44.532+0000 7f956dc00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95680cf170 0x7f95680b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:44.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:44.532+0000 7f956dc00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f956815aed0 0x7f9568153160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:44.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:44.532+0000 7f956dc00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f95680730a0 con 0x7f95680706b0 2024-09-16T15:50:44.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:44.532+0000 7f956dc00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9568072f20 con 0x7f95680cf170 2024-09-16T15:50:44.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:44.532+0000 7f956dc00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f9568073220 con 0x7f956815aed0 2024-09-16T15:50:44.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:44.533+0000 7f9567e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f95680706b0 0x7f956814b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:44.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:44.533+0000 7f9567e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f95680706b0 0x7f956814b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:45828/0 (socket says 172.21.15.151:45828) 2024-09-16T15:50:44.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:44.533+0000 7f9567e00640 1 -- 172.21.15.151:0/1925503112 learned_addr learned my addr 172.21.15.151:0/1925503112 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:44.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:44.533+0000 7f956d200640 1 --2- 172.21.15.151:0/1925503112 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f956815aed0 0x7f9568153160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:44.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:44.533+0000 7f9567400640 1 --2- 172.21.15.151:0/1925503112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95680cf170 0x7f95680b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:44.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:44.534+0000 7f9567e00640 1 -- 172.21.15.151:0/1925503112 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f956815aed0 msgr2=0x7f9568153160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:44.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:44.534+0000 7f9567e00640 1 --2- 172.21.15.151:0/1925503112 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f956815aed0 0x7f9568153160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:44.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:44.534+0000 7f9567e00640 1 -- 172.21.15.151:0/1925503112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95680cf170 msgr2=0x7f95680b3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:44.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:44.534+0000 7f9567e00640 1 --2- 172.21.15.151:0/1925503112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95680cf170 0x7f95680b3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:44.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:44.534+0000 7f9567e00640 1 -- 172.21.15.151:0/1925503112 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95680b48d0 con 0x7f95680706b0 2024-09-16T15:50:44.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:44.534+0000 7f9567400640 1 --2- 172.21.15.151:0/1925503112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95680cf170 0x7f95680b3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:50:44.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:44.534+0000 7f9567e00640 1 --2- 172.21.15.151:0/1925503112 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f95680706b0 0x7f956814b850 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f955400d740 tx=0x7f955400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:44.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:44.534+0000 7f955fe00640 1 -- 172.21.15.151:0/1925503112 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9554017070 con 0x7f95680706b0 2024-09-16T15:50:44.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:44.535+0000 7f955fe00640 1 -- 172.21.15.151:0/1925503112 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9554009d20 con 0x7f95680706b0 2024-09-16T15:50:44.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:44.535+0000 7f955fe00640 1 -- 172.21.15.151:0/1925503112 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95540106c0 con 0x7f95680706b0 2024-09-16T15:50:44.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:44.535+0000 7f956dc00640 1 -- 172.21.15.151:0/1925503112 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95680b4bb0 con 0x7f95680706b0 2024-09-16T15:50:44.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:44.535+0000 7f956dc00640 1 -- 172.21.15.151:0/1925503112 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95680b5450 con 0x7f95680706b0 2024-09-16T15:50:44.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:44.537+0000 7f956dc00640 1 -- 172.21.15.151:0/1925503112 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95680b59f0 con 0x7f95680706b0 2024-09-16T15:50:44.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:44.539+0000 7f955fe00640 1 -- 172.21.15.151:0/1925503112 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7f955401a070 con 0x7f95680706b0 2024-09-16T15:50:44.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:44.540+0000 7f955fe00640 1 --2- 172.21.15.151:0/1925503112 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9548079f50 0x7f954807c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:44.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:44.541+0000 7f955fe00640 1 -- 172.21.15.151:0/1925503112 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5821+0+0 (secure 0 0 0) 0x7f9554099670 con 0x7f95680706b0 2024-09-16T15:50:44.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:44.541+0000 7f9567400640 1 --2- 172.21.15.151:0/1925503112 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9548079f50 0x7f954807c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:44.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:44.542+0000 7f9567400640 1 --2- 172.21.15.151:0/1925503112 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9548079f50 0x7f954807c410 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f95680c6bf0 tx=0x7f9558007e90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:44.548 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:44.547+0000 7f955fe00640 1 -- 172.21.15.151:0/1925503112 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9554063170 con 0x7f95680706b0 2024-09-16T15:50:44.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:44.947+0000 7f956dc00640 1 -- 172.21.15.151:0/1925503112 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone23331", "target": ["mon-mgr", ""]}) -- 0x7f95680add90 con 0x7f9548079f50 2024-09-16T15:50:44.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:44.967+0000 7f955fe00640 1 -- 172.21.15.151:0/1925503112 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f95680add90 con 0x7f9548079f50 2024-09-16T15:50:44.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:44.970+0000 7f956dc00640 1 -- 172.21.15.151:0/1925503112 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9548079f50 msgr2=0x7f954807c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:44.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:44.970+0000 7f956dc00640 1 --2- 172.21.15.151:0/1925503112 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9548079f50 0x7f954807c410 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f95680c6bf0 tx=0x7f9558007e90 comp rx=0 tx=0).stop 2024-09-16T15:50:44.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:44.970+0000 7f956dc00640 1 -- 172.21.15.151:0/1925503112 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f95680706b0 msgr2=0x7f956814b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:44.972 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:44.970+0000 7f956dc00640 1 --2- 172.21.15.151:0/1925503112 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f95680706b0 0x7f956814b850 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f955400d740 tx=0x7f955400dc00 comp rx=0 tx=0).stop 2024-09-16T15:50:44.972 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:44.970+0000 7f956dc00640 1 -- 172.21.15.151:0/1925503112 shutdown_connections 2024-09-16T15:50:44.972 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:44.970+0000 7f956dc00640 1 --2- 172.21.15.151:0/1925503112 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9548079f50 0x7f954807c410 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:44.972 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:44.970+0000 7f956dc00640 1 --2- 172.21.15.151:0/1925503112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95680cf170 0x7f95680b3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:44.972 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:44.970+0000 7f956dc00640 1 --2- 172.21.15.151:0/1925503112 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f956815aed0 0x7f9568153160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:44.972 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:44.970+0000 7f956dc00640 1 --2- 172.21.15.151:0/1925503112 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f95680706b0 0x7f956814b850 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:44.972 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:44.970+0000 7f956dc00640 1 -- 172.21.15.151:0/1925503112 >> 172.21.15.151:0/1925503112 conn(0x7f95680cfa70 msgr2=0x7f9568072450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:44.972 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:44.970+0000 7f956dc00640 1 -- 172.21.15.151:0/1925503112 shutdown_connections 2024-09-16T15:50:44.972 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:44.970+0000 7f956dc00640 1 -- 172.21.15.151:0/1925503112 wait complete. 2024-09-16T15:50:44.990 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone446916 2024-09-16T15:50:45.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.166+0000 7f4e99c00640 1 -- 172.21.15.151:0/3990942314 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e9415bda0 msgr2=0x7f4e94073010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:45.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.166+0000 7f4e99c00640 1 --2- 172.21.15.151:0/3990942314 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e9415bda0 0x7f4e94073010 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f4e84009830 tx=0x7f4e8402a5d0 comp rx=0 tx=0).stop 2024-09-16T15:50:45.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.167+0000 7f4e99c00640 1 -- 172.21.15.151:0/3990942314 shutdown_connections 2024-09-16T15:50:45.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.167+0000 7f4e99c00640 1 --2- 172.21.15.151:0/3990942314 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e9415bda0 0x7f4e94073010 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:45.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.167+0000 7f4e99c00640 1 -- 172.21.15.151:0/3990942314 >> 172.21.15.151:0/3990942314 conn(0x7f4e940cfa70 msgr2=0x7f4e940cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:45.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.168+0000 7f4e99c00640 1 -- 172.21.15.151:0/3990942314 shutdown_connections 2024-09-16T15:50:45.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.169+0000 7f4e99c00640 1 -- 172.21.15.151:0/3990942314 wait complete. 2024-09-16T15:50:45.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.171+0000 7f4e99c00640 1 Processor -- start 2024-09-16T15:50:45.174 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.172+0000 7f4e99c00640 1 -- start start 2024-09-16T15:50:45.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.176+0000 7f4e99c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e9406e590 0x7f4e94151030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:45.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.176+0000 7f4e99c00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e94070550 0x7f4e940c5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:45.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.176+0000 7f4e99c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e9415bda0 0x7f4e9415dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:45.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.176+0000 7f4e93400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e94070550 0x7f4e940c5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:45.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.176+0000 7f4e93400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e94070550 0x7f4e940c5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.151:56232/0 (socket says 172.21.15.151:56232) 2024-09-16T15:50:45.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.177+0000 7f4e93e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e9406e590 0x7f4e94151030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:45.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.177+0000 7f4e99c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4e940cbc60 con 0x7f4e9406e590 2024-09-16T15:50:45.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.177+0000 7f4e99c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4e940cbae0 con 0x7f4e9415bda0 2024-09-16T15:50:45.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.177+0000 7f4e99c00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f4e940cbde0 con 0x7f4e94070550 2024-09-16T15:50:45.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.177+0000 7f4e93e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e9406e590 0x7f4e94151030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:45832/0 (socket says 172.21.15.151:45832) 2024-09-16T15:50:45.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.177+0000 7f4e93e00640 1 -- 172.21.15.151:0/1594558613 learned_addr learned my addr 172.21.15.151:0/1594558613 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:45.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.178+0000 7f4e99200640 1 --2- 172.21.15.151:0/1594558613 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e9415bda0 0x7f4e9415dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:45.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.178+0000 7f4e93400640 1 -- 172.21.15.151:0/1594558613 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e9415bda0 msgr2=0x7f4e9415dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:45.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.178+0000 7f4e93400640 1 --2- 172.21.15.151:0/1594558613 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e9415bda0 0x7f4e9415dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:45.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.178+0000 7f4e93400640 1 -- 172.21.15.151:0/1594558613 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e9406e590 msgr2=0x7f4e94151030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:45.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.178+0000 7f4e93400640 1 --2- 172.21.15.151:0/1594558613 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e9406e590 0x7f4e94151030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:45.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.178+0000 7f4e93400640 1 -- 172.21.15.151:0/1594558613 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e940c63e0 con 0x7f4e94070550 2024-09-16T15:50:45.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.178+0000 7f4e99200640 1 --2- 172.21.15.151:0/1594558613 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e9415bda0 0x7f4e9415dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:50:45.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.178+0000 7f4e93e00640 1 --2- 172.21.15.151:0/1594558613 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e9406e590 0x7f4e94151030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:50:45.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.179+0000 7f4e93400640 1 --2- 172.21.15.151:0/1594558613 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e94070550 0x7f4e940c5ea0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4e8000c460 tx=0x7f4e8000c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:45.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.180+0000 7f4e8be00640 1 -- 172.21.15.151:0/1594558613 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e800052d0 con 0x7f4e94070550 2024-09-16T15:50:45.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.180+0000 7f4e8be00640 1 -- 172.21.15.151:0/1594558613 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e80005490 con 0x7f4e94070550 2024-09-16T15:50:45.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.180+0000 7f4e8be00640 1 -- 172.21.15.151:0/1594558613 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e80012870 con 0x7f4e94070550 2024-09-16T15:50:45.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.180+0000 7f4e99c00640 1 -- 172.21.15.151:0/1594558613 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e9415f4a0 con 0x7f4e94070550 2024-09-16T15:50:45.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.181+0000 7f4e99c00640 1 -- 172.21.15.151:0/1594558613 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e9415eee0 con 0x7f4e94070550 2024-09-16T15:50:45.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.184+0000 7f4e99c00640 1 -- 172.21.15.151:0/1594558613 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e94050f80 con 0x7f4e94070550 2024-09-16T15:50:45.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.185+0000 7f4e8be00640 1 -- 172.21.15.151:0/1594558613 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7f4e80012a10 con 0x7f4e94070550 2024-09-16T15:50:45.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.187+0000 7f4e8be00640 1 --2- 172.21.15.151:0/1594558613 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4e74079f00 0x7f4e7407c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:45.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.187+0000 7f4e8be00640 1 -- 172.21.15.151:0/1594558613 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(92..92 src has 1..92) ==== 5821+0+0 (secure 0 0 0) 0x7f4e80099070 con 0x7f4e94070550 2024-09-16T15:50:45.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.187+0000 7f4e93e00640 1 --2- 172.21.15.151:0/1594558613 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4e74079f00 0x7f4e7407c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:45.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.189+0000 7f4e93e00640 1 --2- 172.21.15.151:0/1594558613 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4e74079f00 0x7f4e7407c3c0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f4e84035040 tx=0x7f4e84004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:45.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.194+0000 7f4e8be00640 1 -- 172.21.15.151:0/1594558613 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4e94050f80 con 0x7f4e94070550 2024-09-16T15:50:45.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.576+0000 7f4e99c00640 1 -- 172.21.15.151:0/1594558613 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone446916", "target": ["mon-mgr", ""]}) -- 0x7f4e94072e00 con 0x7f4e74079f00 2024-09-16T15:50:45.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.594+0000 7f4e8be00640 1 -- 172.21.15.151:0/1594558613 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4e94072e00 con 0x7f4e74079f00 2024-09-16T15:50:45.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.596+0000 7f4e99c00640 1 -- 172.21.15.151:0/1594558613 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4e74079f00 msgr2=0x7f4e7407c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:45.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.596+0000 7f4e99c00640 1 --2- 172.21.15.151:0/1594558613 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4e74079f00 0x7f4e7407c3c0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f4e84035040 tx=0x7f4e84004730 comp rx=0 tx=0).stop 2024-09-16T15:50:45.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.596+0000 7f4e99c00640 1 -- 172.21.15.151:0/1594558613 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e94070550 msgr2=0x7f4e940c5ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:45.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.596+0000 7f4e99c00640 1 --2- 172.21.15.151:0/1594558613 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e94070550 0x7f4e940c5ea0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4e8000c460 tx=0x7f4e8000c920 comp rx=0 tx=0).stop 2024-09-16T15:50:45.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.596+0000 7f4e99c00640 1 -- 172.21.15.151:0/1594558613 shutdown_connections 2024-09-16T15:50:45.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.596+0000 7f4e99c00640 1 --2- 172.21.15.151:0/1594558613 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4e74079f00 0x7f4e7407c3c0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:45.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.596+0000 7f4e99c00640 1 --2- 172.21.15.151:0/1594558613 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e9415bda0 0x7f4e9415dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:45.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.596+0000 7f4e99c00640 1 --2- 172.21.15.151:0/1594558613 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4e94070550 0x7f4e940c5ea0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:45.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.596+0000 7f4e99c00640 1 --2- 172.21.15.151:0/1594558613 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4e9406e590 0x7f4e94151030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:45.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.596+0000 7f4e99c00640 1 -- 172.21.15.151:0/1594558613 >> 172.21.15.151:0/1594558613 conn(0x7f4e940cfa70 msgr2=0x7f4e940cda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:45.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.596+0000 7f4e99c00640 1 -- 172.21.15.151:0/1594558613 shutdown_connections 2024-09-16T15:50:45.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.596+0000 7f4e99c00640 1 -- 172.21.15.151:0/1594558613 wait complete. 2024-09-16T15:50:45.614 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-16T15:50:45.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:50:45.701 INFO:teuthology.orchestra.run.smithi151.stdout:2 2024-09-16T15:50:45.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:50:45.709 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-16T15:50:45.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.871+0000 7f110b000640 1 -- 172.21.15.151:0/2766503022 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f110406efa0 msgr2=0x7f11040cc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:45.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.871+0000 7f110b000640 1 --2- 172.21.15.151:0/2766503022 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f110406efa0 0x7f11040cc5c0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f11040d0cd0 tx=0x7f10f002a560 comp rx=0 tx=0).stop 2024-09-16T15:50:45.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.871+0000 7f110b000640 1 -- 172.21.15.151:0/2766503022 shutdown_connections 2024-09-16T15:50:45.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.871+0000 7f110b000640 1 --2- 172.21.15.151:0/2766503022 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f110406efa0 0x7f11040cc5c0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:45.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.871+0000 7f110b000640 1 --2- 172.21.15.151:0/2766503022 >> v2:172.21.15.167:3300/0 conn(0x7f1104145750 0x7f1104073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:45.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.871+0000 7f110b000640 1 -- 172.21.15.151:0/2766503022 >> 172.21.15.151:0/2766503022 conn(0x7f11040b1840 msgr2=0x7f11040b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:45.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.871+0000 7f110b000640 1 -- 172.21.15.151:0/2766503022 shutdown_connections 2024-09-16T15:50:45.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.871+0000 7f110b000640 1 -- 172.21.15.151:0/2766503022 wait complete. 2024-09-16T15:50:45.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.873+0000 7f110b000640 1 Processor -- start 2024-09-16T15:50:45.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.873+0000 7f110b000640 1 -- start start 2024-09-16T15:50:45.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.874+0000 7f110b000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f110406efa0 0x7f11041e50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:45.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.874+0000 7f110b000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11040ccb00 0x7f11040b4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:45.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.874+0000 7f110b000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1104145750 0x7f11041e6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:45.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.874+0000 7f110b000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f11040ced40 con 0x7f1104145750 2024-09-16T15:50:45.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.874+0000 7f110b000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f11040cebc0 con 0x7f110406efa0 2024-09-16T15:50:45.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.874+0000 7f110b000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f11040ceec0 con 0x7f11040ccb00 2024-09-16T15:50:45.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.874+0000 7f1109000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1104145750 0x7f11041e6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:45.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.874+0000 7f1103400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11040ccb00 0x7f11040b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:45.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.874+0000 7f1103e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f110406efa0 0x7f11041e50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:45.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.874+0000 7f1103400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11040ccb00 0x7f11040b4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.151:56242/0 (socket says 172.21.15.151:56242) 2024-09-16T15:50:45.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.874+0000 7f1103400640 1 -- 172.21.15.151:0/434130875 learned_addr learned my addr 172.21.15.151:0/434130875 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:45.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.874+0000 7f1109000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1104145750 0x7f11041e6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:45846/0 (socket says 172.21.15.151:45846) 2024-09-16T15:50:45.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.874+0000 7f1103e00640 1 -- 172.21.15.151:0/434130875 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11040ccb00 msgr2=0x7f11040b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:45.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.874+0000 7f1103e00640 1 --2- 172.21.15.151:0/434130875 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11040ccb00 0x7f11040b4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:45.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.874+0000 7f1103e00640 1 -- 172.21.15.151:0/434130875 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1104145750 msgr2=0x7f11041e6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:45.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.874+0000 7f1103e00640 1 --2- 172.21.15.151:0/434130875 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1104145750 0x7f11041e6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:45.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.874+0000 7f1103e00640 1 -- 172.21.15.151:0/434130875 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11041e5630 con 0x7f110406efa0 2024-09-16T15:50:45.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.875+0000 7f1109000640 1 --2- 172.21.15.151:0/434130875 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1104145750 0x7f11041e6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:50:45.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.875+0000 7f1103e00640 1 --2- 172.21.15.151:0/434130875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f110406efa0 0x7f11041e50f0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f10f400e6d0 tx=0x7f10f400eb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:45.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.875+0000 7f10fbe00640 1 -- 172.21.15.151:0/434130875 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f400cab0 con 0x7f110406efa0 2024-09-16T15:50:45.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.875+0000 7f10fbe00640 1 -- 172.21.15.151:0/434130875 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f10f400cd40 con 0x7f110406efa0 2024-09-16T15:50:45.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.875+0000 7f10fbe00640 1 -- 172.21.15.151:0/434130875 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f4010690 con 0x7f110406efa0 2024-09-16T15:50:45.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.875+0000 7f110b000640 1 -- 172.21.15.151:0/434130875 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11041f85f0 con 0x7f110406efa0 2024-09-16T15:50:45.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.875+0000 7f110b000640 1 -- 172.21.15.151:0/434130875 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11041f8bb0 con 0x7f110406efa0 2024-09-16T15:50:45.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.876+0000 7f110b000640 1 -- 172.21.15.151:0/434130875 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11040cc5c0 con 0x7f110406efa0 2024-09-16T15:50:45.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.877+0000 7f10fbe00640 1 -- 172.21.15.151:0/434130875 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7f10f4010830 con 0x7f110406efa0 2024-09-16T15:50:45.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.878+0000 7f10fbe00640 1 --2- 172.21.15.151:0/434130875 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f10e4079ed0 0x7f10e407c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:45.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.878+0000 7f10fbe00640 1 -- 172.21.15.151:0/434130875 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5821+0+0 (secure 0 0 0) 0x7f10f4099f70 con 0x7f110406efa0 2024-09-16T15:50:45.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.879+0000 7f1103400640 1 --2- 172.21.15.151:0/434130875 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f10e4079ed0 0x7f10e407c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:45.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.879+0000 7f1103400640 1 --2- 172.21.15.151:0/434130875 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f10e4079ed0 0x7f10e407c390 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f11040b4bb0 tx=0x7f10f0004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:45.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:45.883+0000 7f10fbe00640 1 -- 172.21.15.151:0/434130875 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f10f40639f0 con 0x7f110406efa0 2024-09-16T15:50:46.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:46.074+0000 7f110b000640 1 -- 172.21.15.151:0/434130875 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f11040b4c30 con 0x7f110406efa0 2024-09-16T15:50:46.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:46.076+0000 7f10fbe00640 1 -- 172.21.15.151:0/434130875 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v43)=0 v43) ==== 131+0+0 (secure 0 0 0) 0x7f10f4068420 con 0x7f110406efa0 2024-09-16T15:50:46.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:46.078+0000 7f110b000640 1 -- 172.21.15.151:0/434130875 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f10e4079ed0 msgr2=0x7f10e407c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:46.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:46.078+0000 7f110b000640 1 --2- 172.21.15.151:0/434130875 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f10e4079ed0 0x7f10e407c390 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f11040b4bb0 tx=0x7f10f0004730 comp rx=0 tx=0).stop 2024-09-16T15:50:46.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:46.078+0000 7f110b000640 1 -- 172.21.15.151:0/434130875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f110406efa0 msgr2=0x7f11041e50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:46.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:46.078+0000 7f110b000640 1 --2- 172.21.15.151:0/434130875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f110406efa0 0x7f11041e50f0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f10f400e6d0 tx=0x7f10f400eb90 comp rx=0 tx=0).stop 2024-09-16T15:50:46.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:46.078+0000 7f110b000640 1 -- 172.21.15.151:0/434130875 shutdown_connections 2024-09-16T15:50:46.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:46.078+0000 7f110b000640 1 --2- 172.21.15.151:0/434130875 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f10e4079ed0 0x7f10e407c390 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:46.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:46.078+0000 7f110b000640 1 --2- 172.21.15.151:0/434130875 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1104145750 0x7f11041e6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:46.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:46.078+0000 7f110b000640 1 --2- 172.21.15.151:0/434130875 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f11040ccb00 0x7f11040b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:46.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:46.078+0000 7f110b000640 1 --2- 172.21.15.151:0/434130875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f110406efa0 0x7f11041e50f0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:46.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:46.079+0000 7f110b000640 1 -- 172.21.15.151:0/434130875 >> 172.21.15.151:0/434130875 conn(0x7f11040b1840 msgr2=0x7f11040b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:46.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:46.079+0000 7f110b000640 1 -- 172.21.15.151:0/434130875 shutdown_connections 2024-09-16T15:50:46.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:46.079+0000 7f110b000640 1 -- 172.21.15.151:0/434130875 wait complete. 2024-09-16T15:50:46.095 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-16T15:50:46.095 DEBUG:teuthology.orchestra.run.smithi151:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:50:46.179 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:50:46.297 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-16T15:50:46.297 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:50:46.298 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:50:46.298 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:50:46.356 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:50:46.357 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:50:46.419 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-16T15:50:46.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:46.671+0000 7fcb5fe00640 1 -- 172.21.15.151:0/1630046719 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb5806fd90 msgr2=0x7fcb580b3f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:46.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:46.671+0000 7fcb5fe00640 1 --2- 172.21.15.151:0/1630046719 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb5806fd90 0x7fcb580b3f00 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fcb50005fd0 tx=0x7fcb5002aaa0 comp rx=0 tx=0).stop 2024-09-16T15:50:46.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:46.672+0000 7fcb5fe00640 1 -- 172.21.15.151:0/1630046719 shutdown_connections 2024-09-16T15:50:46.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:46.672+0000 7fcb5fe00640 1 --2- 172.21.15.151:0/1630046719 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb5806fd90 0x7fcb580b3f00 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:46.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:46.672+0000 7fcb5fe00640 1 --2- 172.21.15.151:0/1630046719 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb580ccd00 0x7fcb5806f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:46.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:46.672+0000 7fcb5fe00640 1 -- 172.21.15.151:0/1630046719 >> 172.21.15.151:0/1630046719 conn(0x7fcb580b1950 msgr2=0x7fcb580b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:46.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:46.672+0000 7fcb5fe00640 1 -- 172.21.15.151:0/1630046719 shutdown_connections 2024-09-16T15:50:46.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:46.673+0000 7fcb5fe00640 1 -- 172.21.15.151:0/1630046719 wait complete. 2024-09-16T15:50:46.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:46.676+0000 7fcb5fe00640 1 Processor -- start 2024-09-16T15:50:46.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:46.676+0000 7fcb5fe00640 1 -- start start 2024-09-16T15:50:46.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:46.678+0000 7fcb5fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb5806fd90 0x7fcb58156d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:46.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:46.678+0000 7fcb5fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb580ccd00 0x7fcb5809d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:46.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:46.678+0000 7fcb5fe00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb581438b0 0x7fcb5809b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:46.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:46.678+0000 7fcb5fe00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcb580b4440 con 0x7fcb5806fd90 2024-09-16T15:50:46.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:46.678+0000 7fcb5fe00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcb58070b30 con 0x7fcb580ccd00 2024-09-16T15:50:46.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:46.678+0000 7fcb5fe00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fcb580b45c0 con 0x7fcb581438b0 2024-09-16T15:50:46.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:46.678+0000 7fcb5ca00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb580ccd00 0x7fcb5809d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:46.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:46.678+0000 7fcb5ca00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb580ccd00 0x7fcb5809d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.151:47696/0 (socket says 172.21.15.151:47696) 2024-09-16T15:50:46.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:46.678+0000 7fcb5ca00640 1 -- 172.21.15.151:0/3799585467 learned_addr learned my addr 172.21.15.151:0/3799585467 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:46.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:46.678+0000 7fcb5de00640 1 --2- 172.21.15.151:0/3799585467 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb581438b0 0x7fcb5809b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:46.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:46.679+0000 7fcb5d400640 1 --2- 172.21.15.151:0/3799585467 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb5806fd90 0x7fcb58156d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:46.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:46.679+0000 7fcb5ca00640 1 -- 172.21.15.151:0/3799585467 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb581438b0 msgr2=0x7fcb5809b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:46.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:46.679+0000 7fcb5ca00640 1 --2- 172.21.15.151:0/3799585467 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb581438b0 0x7fcb5809b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:46.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:46.679+0000 7fcb5ca00640 1 -- 172.21.15.151:0/3799585467 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb5806fd90 msgr2=0x7fcb58156d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:46.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:46.679+0000 7fcb5ca00640 1 --2- 172.21.15.151:0/3799585467 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb5806fd90 0x7fcb58156d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:46.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:46.679+0000 7fcb5ca00640 1 -- 172.21.15.151:0/3799585467 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb581690b0 con 0x7fcb580ccd00 2024-09-16T15:50:46.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:46.679+0000 7fcb5d400640 1 --2- 172.21.15.151:0/3799585467 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb5806fd90 0x7fcb58156d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:50:46.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:46.680+0000 7fcb5ca00640 1 --2- 172.21.15.151:0/3799585467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb580ccd00 0x7fcb5809d3c0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fcb4400d770 tx=0x7fcb4400dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:46.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:46.681+0000 7fcb55600640 1 -- 172.21.15.151:0/3799585467 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb44017070 con 0x7fcb580ccd00 2024-09-16T15:50:46.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:46.681+0000 7fcb55600640 1 -- 172.21.15.151:0/3799585467 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcb44009d20 con 0x7fcb580ccd00 2024-09-16T15:50:46.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:46.681+0000 7fcb55600640 1 -- 172.21.15.151:0/3799585467 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb440106c0 con 0x7fcb580ccd00 2024-09-16T15:50:46.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:46.681+0000 7fcb5fe00640 1 -- 172.21.15.151:0/3799585467 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb5813b210 con 0x7fcb580ccd00 2024-09-16T15:50:46.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:46.681+0000 7fcb5fe00640 1 -- 172.21.15.151:0/3799585467 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb5813b7d0 con 0x7fcb580ccd00 2024-09-16T15:50:46.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:46.682+0000 7fcb5fe00640 1 -- 172.21.15.151:0/3799585467 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb580b31d0 con 0x7fcb580ccd00 2024-09-16T15:50:46.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:46.685+0000 7fcb55600640 1 -- 172.21.15.151:0/3799585467 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7fcb4401a070 con 0x7fcb580ccd00 2024-09-16T15:50:46.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:46.686+0000 7fcb55600640 1 --2- 172.21.15.151:0/3799585467 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcb38079ed0 0x7fcb3807c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:46.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:46.687+0000 7fcb55600640 1 -- 172.21.15.151:0/3799585467 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5793+0+0 (secure 0 0 0) 0x7fcb44099c10 con 0x7fcb580ccd00 2024-09-16T15:50:46.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:46.687+0000 7fcb5d400640 1 --2- 172.21.15.151:0/3799585467 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcb38079ed0 0x7fcb3807c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:46.689 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:46.688+0000 7fcb5d400640 1 --2- 172.21.15.151:0/3799585467 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcb38079ed0 0x7fcb3807c390 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c0016a0 tx=0x7fcb4c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:46.694 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:46.693+0000 7fcb55600640 1 -- 172.21.15.151:0/3799585467 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fcb580b31d0 con 0x7fcb580ccd00 2024-09-16T15:50:46.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:46.979+0000 7fcb5fe00640 1 -- 172.21.15.151:0/3799585467 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fcb580b3f00 con 0x7fcb580ccd00 2024-09-16T15:50:46.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:46.980+0000 7fcb55600640 1 -- 172.21.15.151:0/3799585467 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v93) ==== 80+0+0 (secure 0 0 0) 0x7fcb580b3f00 con 0x7fcb580ccd00 2024-09-16T15:50:46.981 INFO:teuthology.orchestra.run.smithi151.stderr:listed 0 entries 2024-09-16T15:50:46.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:46.982+0000 7fcb5fe00640 1 -- 172.21.15.151:0/3799585467 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcb38079ed0 msgr2=0x7fcb3807c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:46.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:46.982+0000 7fcb5fe00640 1 --2- 172.21.15.151:0/3799585467 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcb38079ed0 0x7fcb3807c390 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c0016a0 tx=0x7fcb4c009040 comp rx=0 tx=0).stop 2024-09-16T15:50:46.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:46.982+0000 7fcb5fe00640 1 -- 172.21.15.151:0/3799585467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb580ccd00 msgr2=0x7fcb5809d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:46.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:46.982+0000 7fcb5fe00640 1 --2- 172.21.15.151:0/3799585467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb580ccd00 0x7fcb5809d3c0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fcb4400d770 tx=0x7fcb4400dc30 comp rx=0 tx=0).stop 2024-09-16T15:50:46.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:46.982+0000 7fcb5fe00640 1 -- 172.21.15.151:0/3799585467 shutdown_connections 2024-09-16T15:50:46.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:46.982+0000 7fcb5fe00640 1 --2- 172.21.15.151:0/3799585467 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcb38079ed0 0x7fcb3807c390 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:46.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:46.982+0000 7fcb5fe00640 1 --2- 172.21.15.151:0/3799585467 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb580ccd00 0x7fcb5809d3c0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:46.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:46.982+0000 7fcb5fe00640 1 --2- 172.21.15.151:0/3799585467 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb581438b0 0x7fcb5809b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:46.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:46.982+0000 7fcb5fe00640 1 --2- 172.21.15.151:0/3799585467 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb5806fd90 0x7fcb58156d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:46.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:46.982+0000 7fcb5fe00640 1 -- 172.21.15.151:0/3799585467 >> 172.21.15.151:0/3799585467 conn(0x7fcb580b1950 msgr2=0x7fcb5815bfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:46.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:46.982+0000 7fcb5fe00640 1 -- 172.21.15.151:0/3799585467 shutdown_connections 2024-09-16T15:50:46.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:46.982+0000 7fcb5fe00640 1 -- 172.21.15.151:0/3799585467 wait complete. 2024-09-16T15:50:46.999 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-16T15:50:46.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:50:46.999 DEBUG:teuthology.orchestra.run.smithi151:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-16T15:50:47.079 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi151.front.sepia.ceph.com 2024-09-16T15:50:47.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:50:47.080 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-16T15:50:47.138 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:50:47.139 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-16T15:50:47.139 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:50:47.189 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:50:47.189 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:50:47.189 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:50:47.189 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:50:47.248 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T15:50:47.248 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:50:47.248 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-16T15:50:47.248 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:50:47.303 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:50:47.303 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:50:47.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:47.572+0000 7feed9e00640 1 -- 172.21.15.151:0/3511021998 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feed4073fb0 msgr2=0x7feed4074440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:47.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:47.572+0000 7feed9e00640 1 --2- 172.21.15.151:0/3511021998 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feed4073fb0 0x7feed4074440 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7feec4005fd0 tx=0x7feec402a540 comp rx=0 tx=0).stop 2024-09-16T15:50:47.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:47.572+0000 7feed9e00640 1 -- 172.21.15.151:0/3511021998 shutdown_connections 2024-09-16T15:50:47.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:47.572+0000 7feed9e00640 1 --2- 172.21.15.151:0/3511021998 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feed4073fb0 0x7feed4074440 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:47.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:47.573+0000 7feed9e00640 1 --2- 172.21.15.151:0/3511021998 >> v2:172.21.15.167:3300/0 conn(0x7feed40ccf00 0x7feed40b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:47.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:47.573+0000 7feed9e00640 1 -- 172.21.15.151:0/3511021998 >> 172.21.15.151:0/3511021998 conn(0x7feed40b1950 msgr2=0x7feed40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:47.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:47.573+0000 7feed9e00640 1 -- 172.21.15.151:0/3511021998 shutdown_connections 2024-09-16T15:50:47.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:47.574+0000 7feed9e00640 1 -- 172.21.15.151:0/3511021998 wait complete. 2024-09-16T15:50:47.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:47.577+0000 7feed9e00640 1 Processor -- start 2024-09-16T15:50:47.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:47.578+0000 7feed9e00640 1 -- start start 2024-09-16T15:50:47.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:47.580+0000 7feed9e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feed4073fb0 0x7feed41fca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:47.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:47.580+0000 7feed9e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feed40ccf00 0x7feed40b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:47.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:47.580+0000 7feed9e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feed415f5a0 0x7feed414c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:47.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:47.580+0000 7feed9e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7feed406f2e0 con 0x7feed40ccf00 2024-09-16T15:50:47.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:47.580+0000 7feed9e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7feed406f160 con 0x7feed415f5a0 2024-09-16T15:50:47.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:47.580+0000 7feed9e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7feed406f460 con 0x7feed4073fb0 2024-09-16T15:50:47.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:47.581+0000 7feed3400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feed415f5a0 0x7feed414c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:47.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:47.581+0000 7feed3400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feed415f5a0 0x7feed414c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.151:47722/0 (socket says 172.21.15.151:47722) 2024-09-16T15:50:47.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:47.581+0000 7feed3400640 1 -- 172.21.15.151:0/1500145061 learned_addr learned my addr 172.21.15.151:0/1500145061 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:47.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:47.582+0000 7feed2a00640 1 --2- 172.21.15.151:0/1500145061 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feed4073fb0 0x7feed41fca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:47.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:47.582+0000 7feed3400640 1 -- 172.21.15.151:0/1500145061 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feed4073fb0 msgr2=0x7feed41fca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:47.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:47.582+0000 7feed3400640 1 --2- 172.21.15.151:0/1500145061 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feed4073fb0 0x7feed41fca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:47.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:47.582+0000 7feed3400640 1 -- 172.21.15.151:0/1500145061 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feed40ccf00 msgr2=0x7feed40b2d50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T15:50:47.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:47.582+0000 7feed2000640 1 --2- 172.21.15.151:0/1500145061 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feed40ccf00 0x7feed40b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:47.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:47.583+0000 7feed3400640 1 --2- 172.21.15.151:0/1500145061 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feed40ccf00 0x7feed40b2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:47.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:47.583+0000 7feed3400640 1 -- 172.21.15.151:0/1500145061 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feed41fcfe0 con 0x7feed415f5a0 2024-09-16T15:50:47.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:47.583+0000 7feed2a00640 1 --2- 172.21.15.151:0/1500145061 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feed4073fb0 0x7feed41fca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:50:47.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:47.583+0000 7feed2000640 1 --2- 172.21.15.151:0/1500145061 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feed40ccf00 0x7feed40b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:50:47.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:47.586+0000 7feed3400640 1 --2- 172.21.15.151:0/1500145061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feed415f5a0 0x7feed414c400 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7feec40099f0 tx=0x7feec40056f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:47.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:47.586+0000 7feecb400640 1 -- 172.21.15.151:0/1500145061 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec403f070 con 0x7feed415f5a0 2024-09-16T15:50:47.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:47.586+0000 7feecb400640 1 -- 172.21.15.151:0/1500145061 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feec40035f0 con 0x7feed415f5a0 2024-09-16T15:50:47.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:47.586+0000 7feecb400640 1 -- 172.21.15.151:0/1500145061 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec4035030 con 0x7feed415f5a0 2024-09-16T15:50:47.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:47.588+0000 7feed9e00640 1 -- 172.21.15.151:0/1500145061 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feed41fd2c0 con 0x7feed415f5a0 2024-09-16T15:50:47.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:47.589+0000 7feed9e00640 1 -- 172.21.15.151:0/1500145061 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feed41fd880 con 0x7feed415f5a0 2024-09-16T15:50:47.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:47.591+0000 7feed9e00640 1 -- 172.21.15.151:0/1500145061 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feed40acfc0 con 0x7feed415f5a0 2024-09-16T15:50:47.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:47.592+0000 7feecb400640 1 -- 172.21.15.151:0/1500145061 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7feec4042070 con 0x7feed415f5a0 2024-09-16T15:50:47.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:47.594+0000 7feecb400640 1 --2- 172.21.15.151:0/1500145061 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7feeb4079fd0 0x7feeb407c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:47.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:47.594+0000 7feecb400640 1 -- 172.21.15.151:0/1500145061 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5793+0+0 (secure 0 0 0) 0x7feec40ba610 con 0x7feed415f5a0 2024-09-16T15:50:47.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:47.595+0000 7feed2a00640 1 --2- 172.21.15.151:0/1500145061 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7feeb4079fd0 0x7feeb407c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:47.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:47.596+0000 7feed2a00640 1 --2- 172.21.15.151:0/1500145061 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7feeb4079fd0 0x7feeb407c490 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7feecc001d10 tx=0x7feecc009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:47.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:47.601+0000 7feecb400640 1 -- 172.21.15.151:0/1500145061 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7feec40846e0 con 0x7feed415f5a0 2024-09-16T15:50:47.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:47.902+0000 7feed9e00640 1 -- 172.21.15.151:0/1500145061 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feed409cb20 con 0x7feed415f5a0 2024-09-16T15:50:47.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:47.903+0000 7feecb400640 1 -- 172.21.15.151:0/1500145061 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+5234 (secure 0 0 0) 0x7feec4083fe0 con 0x7feed415f5a0 2024-09-16T15:50:47.904 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:50:47.904 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":57,"btime":"2024-09-16T15:48:35:029906+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6134,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/743718626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":743718626},{"type":"v1","addr":"172.21.15.167:6836","nonce":743718626}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6137,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1262689373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1262689373},{"type":"v1","addr":"172.21.15.167:6837","nonce":1262689373}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6670,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/1127792363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1127792363},{"type":"v1","addr":"172.21.15.151:6834","nonce":1127792363}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:48:21.374084+0000","modified":"2024-09-16T15:48:24.408969+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6673},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6673":{"gid":6673,"name":"c","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.151:6835/1518702790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":1518702790},{"type":"v1","addr":"172.21.15.151:6835","nonce":1518702790}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6673,"qdb_cluster":[6673]},"id":6}]} 2024-09-16T15:50:47.905 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 57 2024-09-16T15:50:47.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:47.905+0000 7feed9e00640 1 -- 172.21.15.151:0/1500145061 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7feeb4079fd0 msgr2=0x7feeb407c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:47.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:47.905+0000 7feed9e00640 1 --2- 172.21.15.151:0/1500145061 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7feeb4079fd0 0x7feeb407c490 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7feecc001d10 tx=0x7feecc009040 comp rx=0 tx=0).stop 2024-09-16T15:50:47.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:47.905+0000 7feed9e00640 1 -- 172.21.15.151:0/1500145061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feed415f5a0 msgr2=0x7feed414c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:47.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:47.905+0000 7feed9e00640 1 --2- 172.21.15.151:0/1500145061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feed415f5a0 0x7feed414c400 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7feec40099f0 tx=0x7feec40056f0 comp rx=0 tx=0).stop 2024-09-16T15:50:47.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:47.905+0000 7feed9e00640 1 -- 172.21.15.151:0/1500145061 shutdown_connections 2024-09-16T15:50:47.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:47.905+0000 7feed9e00640 1 --2- 172.21.15.151:0/1500145061 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7feeb4079fd0 0x7feeb407c490 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:47.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:47.905+0000 7feed9e00640 1 --2- 172.21.15.151:0/1500145061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feed415f5a0 0x7feed414c400 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:47.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:47.905+0000 7feed9e00640 1 --2- 172.21.15.151:0/1500145061 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feed40ccf00 0x7feed40b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:47.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:47.905+0000 7feed9e00640 1 --2- 172.21.15.151:0/1500145061 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feed4073fb0 0x7feed41fca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:47.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:47.905+0000 7feed9e00640 1 -- 172.21.15.151:0/1500145061 >> 172.21.15.151:0/1500145061 conn(0x7feed40b1950 msgr2=0x7feed406f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:47.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:47.905+0000 7feed9e00640 1 -- 172.21.15.151:0/1500145061 shutdown_connections 2024-09-16T15:50:47.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:47.906+0000 7feed9e00640 1 -- 172.21.15.151:0/1500145061 wait complete. 2024-09-16T15:50:47.923 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:50:48.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.081+0000 7f136f800640 1 -- 172.21.15.151:0/3734093178 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13680ccf00 msgr2=0x7f13680acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:48.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.081+0000 7f136f800640 1 --2- 172.21.15.151:0/3734093178 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13680ccf00 0x7f13680acd30 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f1354009830 tx=0x7f135402a480 comp rx=0 tx=0).stop 2024-09-16T15:50:48.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.081+0000 7f136f800640 1 -- 172.21.15.151:0/3734093178 shutdown_connections 2024-09-16T15:50:48.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.081+0000 7f136f800640 1 --2- 172.21.15.151:0/3734093178 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13680ccf00 0x7f13680acd30 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:48.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.081+0000 7f136f800640 1 --2- 172.21.15.151:0/3734093178 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f136815f5a0 0x7f13680b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:48.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.081+0000 7f136f800640 1 -- 172.21.15.151:0/3734093178 >> 172.21.15.151:0/3734093178 conn(0x7f13680b1950 msgr2=0x7f13680b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:48.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.081+0000 7f136f800640 1 -- 172.21.15.151:0/3734093178 shutdown_connections 2024-09-16T15:50:48.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.081+0000 7f136f800640 1 -- 172.21.15.151:0/3734093178 wait complete. 2024-09-16T15:50:48.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.083+0000 7f136f800640 1 Processor -- start 2024-09-16T15:50:48.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.083+0000 7f136f800640 1 -- start start 2024-09-16T15:50:48.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.084+0000 7f136f800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13680ad270 0x7f13681520a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:48.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.084+0000 7f136f800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13680ccf00 0x7f13681f8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:48.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.084+0000 7f136f800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f136815f5a0 0x7f136809c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:48.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.084+0000 7f136f800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f136806f2e0 con 0x7f136815f5a0 2024-09-16T15:50:48.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.084+0000 7f136f800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f136806f160 con 0x7f13680ad270 2024-09-16T15:50:48.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.084+0000 7f136f800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f136806f460 con 0x7f13680ccf00 2024-09-16T15:50:48.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.084+0000 7f136d800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f136815f5a0 0x7f136809c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:48.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.084+0000 7f136d800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f136815f5a0 0x7f136809c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:45896/0 (socket says 172.21.15.151:45896) 2024-09-16T15:50:48.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.084+0000 7f136d800640 1 -- 172.21.15.151:0/3965423627 learned_addr learned my addr 172.21.15.151:0/3965423627 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:48.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.085+0000 7f1367e00640 1 --2- 172.21.15.151:0/3965423627 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13680ccf00 0x7f13681f8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:48.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.085+0000 7f136ce00640 1 --2- 172.21.15.151:0/3965423627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13680ad270 0x7f13681520a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:48.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.085+0000 7f136d800640 1 -- 172.21.15.151:0/3965423627 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13680ccf00 msgr2=0x7f13681f8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:48.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.085+0000 7f136d800640 1 --2- 172.21.15.151:0/3965423627 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13680ccf00 0x7f13681f8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:48.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.085+0000 7f136d800640 1 -- 172.21.15.151:0/3965423627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13680ad270 msgr2=0x7f13681520a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:48.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.085+0000 7f136d800640 1 --2- 172.21.15.151:0/3965423627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13680ad270 0x7f13681520a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:48.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.085+0000 7f136d800640 1 -- 172.21.15.151:0/3965423627 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13680b36b0 con 0x7f136815f5a0 2024-09-16T15:50:48.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.085+0000 7f136ce00640 1 --2- 172.21.15.151:0/3965423627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13680ad270 0x7f13681520a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:50:48.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.085+0000 7f136d800640 1 --2- 172.21.15.151:0/3965423627 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f136815f5a0 0x7f136809c130 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f136000cb60 tx=0x7f136000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:48.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.085+0000 7f1364c00640 1 -- 172.21.15.151:0/3965423627 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1360016020 con 0x7f136815f5a0 2024-09-16T15:50:48.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.085+0000 7f1364c00640 1 -- 172.21.15.151:0/3965423627 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f136000ae30 con 0x7f136815f5a0 2024-09-16T15:50:48.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.085+0000 7f136f800640 1 -- 172.21.15.151:0/3965423627 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13680b3990 con 0x7f136815f5a0 2024-09-16T15:50:48.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.085+0000 7f1364c00640 1 -- 172.21.15.151:0/3965423627 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1360012680 con 0x7f136815f5a0 2024-09-16T15:50:48.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.086+0000 7f136f800640 1 -- 172.21.15.151:0/3965423627 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13681f8c50 con 0x7f136815f5a0 2024-09-16T15:50:48.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.087+0000 7f136f800640 1 -- 172.21.15.151:0/3965423627 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f136809ccd0 con 0x7f136815f5a0 2024-09-16T15:50:48.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.088+0000 7f1364c00640 1 -- 172.21.15.151:0/3965423627 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7f1360019020 con 0x7f136815f5a0 2024-09-16T15:50:48.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.089+0000 7f1364c00640 1 --2- 172.21.15.151:0/3965423627 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1348079ed0 0x7f134807c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:48.090 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.089+0000 7f1364c00640 1 -- 172.21.15.151:0/3965423627 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5793+0+0 (secure 0 0 0) 0x7f1360099d50 con 0x7f136815f5a0 2024-09-16T15:50:48.090 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.089+0000 7f136ce00640 1 --2- 172.21.15.151:0/3965423627 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1348079ed0 0x7f134807c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:48.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.090+0000 7f136ce00640 1 --2- 172.21.15.151:0/3965423627 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1348079ed0 0x7f134807c390 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f135c009b80 tx=0x7f135c005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:48.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.093+0000 7f1364c00640 1 -- 172.21.15.151:0/3965423627 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1360020020 con 0x7f136815f5a0 2024-09-16T15:50:48.320 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.319+0000 7f136f800640 1 -- 172.21.15.151:0/3965423627 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f136815d410 con 0x7f136815f5a0 2024-09-16T15:50:48.321 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.320+0000 7f1364c00640 1 -- 172.21.15.151:0/3965423627 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+5234 (secure 0 0 0) 0x7f1360063da0 con 0x7f136815f5a0 2024-09-16T15:50:48.321 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:50:48.321 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":57,"btime":"2024-09-16T15:48:35:029906+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6134,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/743718626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":743718626},{"type":"v1","addr":"172.21.15.167:6836","nonce":743718626}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6137,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1262689373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1262689373},{"type":"v1","addr":"172.21.15.167:6837","nonce":1262689373}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6670,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/1127792363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1127792363},{"type":"v1","addr":"172.21.15.151:6834","nonce":1127792363}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:48:21.374084+0000","modified":"2024-09-16T15:48:24.408969+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6673},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6673":{"gid":6673,"name":"c","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.151:6835/1518702790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":1518702790},{"type":"v1","addr":"172.21.15.151:6835","nonce":1518702790}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6673,"qdb_cluster":[6673]},"id":6}]} 2024-09-16T15:50:48.321 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 57 2024-09-16T15:50:48.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.322+0000 7f136f800640 1 -- 172.21.15.151:0/3965423627 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1348079ed0 msgr2=0x7f134807c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:48.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.322+0000 7f136f800640 1 --2- 172.21.15.151:0/3965423627 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1348079ed0 0x7f134807c390 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f135c009b80 tx=0x7f135c005f90 comp rx=0 tx=0).stop 2024-09-16T15:50:48.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.323+0000 7f136f800640 1 -- 172.21.15.151:0/3965423627 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f136815f5a0 msgr2=0x7f136809c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:48.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.323+0000 7f136f800640 1 --2- 172.21.15.151:0/3965423627 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f136815f5a0 0x7f136809c130 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f136000cb60 tx=0x7f136000a430 comp rx=0 tx=0).stop 2024-09-16T15:50:48.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.323+0000 7f136f800640 1 -- 172.21.15.151:0/3965423627 shutdown_connections 2024-09-16T15:50:48.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.323+0000 7f136f800640 1 --2- 172.21.15.151:0/3965423627 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1348079ed0 0x7f134807c390 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:48.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.323+0000 7f136f800640 1 --2- 172.21.15.151:0/3965423627 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f136815f5a0 0x7f136809c130 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:48.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.323+0000 7f136f800640 1 --2- 172.21.15.151:0/3965423627 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13680ccf00 0x7f13681f8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:48.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.323+0000 7f136f800640 1 --2- 172.21.15.151:0/3965423627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13680ad270 0x7f13681520a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:48.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.323+0000 7f136f800640 1 -- 172.21.15.151:0/3965423627 >> 172.21.15.151:0/3965423627 conn(0x7f13680b1950 msgr2=0x7f13681e72f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:48.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.323+0000 7f136f800640 1 -- 172.21.15.151:0/3965423627 shutdown_connections 2024-09-16T15:50:48.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.323+0000 7f136f800640 1 -- 172.21.15.151:0/3965423627 wait complete. 2024-09-16T15:50:48.341 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:50:48.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.497+0000 7f67cc600640 1 -- 172.21.15.151:0/2621046896 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f67c40ccf00 msgr2=0x7f67c40acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:48.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.497+0000 7f67cc600640 1 --2- 172.21.15.151:0/2621046896 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f67c40ccf00 0x7f67c40acd30 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f67b0005fd0 tx=0x7f67b002a540 comp rx=0 tx=0).stop 2024-09-16T15:50:48.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.497+0000 7f67cc600640 1 -- 172.21.15.151:0/2621046896 shutdown_connections 2024-09-16T15:50:48.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.497+0000 7f67cc600640 1 --2- 172.21.15.151:0/2621046896 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f67c40ccf00 0x7f67c40acd30 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:48.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.497+0000 7f67cc600640 1 --2- 172.21.15.151:0/2621046896 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f67c415f5a0 0x7f67c40b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:48.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.497+0000 7f67cc600640 1 -- 172.21.15.151:0/2621046896 >> 172.21.15.151:0/2621046896 conn(0x7f67c40b1950 msgr2=0x7f67c40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:48.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.497+0000 7f67cc600640 1 -- 172.21.15.151:0/2621046896 shutdown_connections 2024-09-16T15:50:48.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.498+0000 7f67cc600640 1 -- 172.21.15.151:0/2621046896 wait complete. 2024-09-16T15:50:48.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.499+0000 7f67cc600640 1 Processor -- start 2024-09-16T15:50:48.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.499+0000 7f67cc600640 1 -- start start 2024-09-16T15:50:48.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.500+0000 7f67cc600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f67c40ad270 0x7f67c4146430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:48.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.500+0000 7f67cc600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67c40ccf00 0x7f67c415da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:48.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.500+0000 7f67cc600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f67c415f5a0 0x7f67c415df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:48.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.500+0000 7f67cc600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f67c406f2e0 con 0x7f67c40ad270 2024-09-16T15:50:48.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.500+0000 7f67cc600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f67c406f160 con 0x7f67c40ccf00 2024-09-16T15:50:48.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.500+0000 7f67cc600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f67c406f460 con 0x7f67c415f5a0 2024-09-16T15:50:48.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.500+0000 7f67c9c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f67c40ad270 0x7f67c4146430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:48.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.500+0000 7f67c9c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f67c40ad270 0x7f67c4146430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:45914/0 (socket says 172.21.15.151:45914) 2024-09-16T15:50:48.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.500+0000 7f67c9c00640 1 -- 172.21.15.151:0/3831794868 learned_addr learned my addr 172.21.15.151:0/3831794868 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:48.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.501+0000 7f67c9200640 1 --2- 172.21.15.151:0/3831794868 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67c40ccf00 0x7f67c415da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:48.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.501+0000 7f67ca600640 1 --2- 172.21.15.151:0/3831794868 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f67c415f5a0 0x7f67c415df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:48.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.501+0000 7f67c9c00640 1 -- 172.21.15.151:0/3831794868 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f67c415f5a0 msgr2=0x7f67c415df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:48.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.501+0000 7f67c9c00640 1 --2- 172.21.15.151:0/3831794868 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f67c415f5a0 0x7f67c415df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:48.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.501+0000 7f67c9c00640 1 -- 172.21.15.151:0/3831794868 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67c40ccf00 msgr2=0x7f67c415da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:48.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.501+0000 7f67c9c00640 1 --2- 172.21.15.151:0/3831794868 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67c40ccf00 0x7f67c415da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:48.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.501+0000 7f67c9c00640 1 -- 172.21.15.151:0/3831794868 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67c40b3d60 con 0x7f67c40ad270 2024-09-16T15:50:48.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.501+0000 7f67c9c00640 1 --2- 172.21.15.151:0/3831794868 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f67c40ad270 0x7f67c4146430 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f67b800d700 tx=0x7f67b800dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:48.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.501+0000 7f67c1600640 1 -- 172.21.15.151:0/3831794868 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b8017070 con 0x7f67c40ad270 2024-09-16T15:50:48.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.501+0000 7f67c1600640 1 -- 172.21.15.151:0/3831794868 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f67b8005310 con 0x7f67c40ad270 2024-09-16T15:50:48.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.501+0000 7f67cc600640 1 -- 172.21.15.151:0/3831794868 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67c40b2fa0 con 0x7f67c40ad270 2024-09-16T15:50:48.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.501+0000 7f67c1600640 1 -- 172.21.15.151:0/3831794868 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b8009820 con 0x7f67c40ad270 2024-09-16T15:50:48.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.501+0000 7f67cc600640 1 -- 172.21.15.151:0/3831794868 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67c40b3560 con 0x7f67c40ad270 2024-09-16T15:50:48.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.502+0000 7f67cc600640 1 -- 172.21.15.151:0/3831794868 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67c406f980 con 0x7f67c40ad270 2024-09-16T15:50:48.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.504+0000 7f67c1600640 1 -- 172.21.15.151:0/3831794868 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7f67b80099c0 con 0x7f67c40ad270 2024-09-16T15:50:48.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.504+0000 7f67c1600640 1 --2- 172.21.15.151:0/3831794868 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f67a4079ed0 0x7f67a407c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:48.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.505+0000 7f67c1600640 1 -- 172.21.15.151:0/3831794868 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5793+0+0 (secure 0 0 0) 0x7f67b8099790 con 0x7f67c40ad270 2024-09-16T15:50:48.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.505+0000 7f67c9200640 1 --2- 172.21.15.151:0/3831794868 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f67a4079ed0 0x7f67a407c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:48.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.506+0000 7f67c9200640 1 --2- 172.21.15.151:0/3831794868 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f67a4079ed0 0x7f67a407c390 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f67b0033000 tx=0x7f67b0004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:48.509 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.509+0000 7f67c1600640 1 -- 172.21.15.151:0/3831794868 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f67b8063860 con 0x7f67c40ad270 2024-09-16T15:50:48.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.695+0000 7f67cc600640 1 -- 172.21.15.151:0/3831794868 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f67c409b820 con 0x7f67c40ad270 2024-09-16T15:50:48.697 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.696+0000 7f67c1600640 1 -- 172.21.15.151:0/3831794868 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+16945 (secure 0 0 0) 0x7f67b8063160 con 0x7f67c40ad270 2024-09-16T15:50:48.698 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:50:48.698 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":93,"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","created":"2024-09-16T15:38:56.348535+0000","modified":"2024-09-16T15:50:46.473983+0000","last_up_change":"2024-09-16T15:39:10.681384+0000","last_in_change":"2024-09-16T15:39:00.544477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:39:11.602252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:48:19.150224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-16T15:48:20.351519+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":92,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5524076-a903-4d0a-af6a-caf474c4765d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6809","nonce":3111342741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6811","nonce":3111342741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6815","nonce":3111342741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6813","nonce":3111342741}]},"public_addr":"172.21.15.151:6809/3111342741","cluster_addr":"172.21.15.151:6811/3111342741","heartbeat_back_addr":"172.21.15.151:6815/3111342741","heartbeat_front_addr":"172.21.15.151:6813/3111342741","state":["exists","up"]},{"osd":1,"uuid":"dd25925f-0ae1-494e-8529-4dc205759db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6801","nonce":3368304087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6803","nonce":3368304087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6807","nonce":3368304087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6805","nonce":3368304087}]},"public_addr":"172.21.15.151:6801/3368304087","cluster_addr":"172.21.15.151:6803/3368304087","heartbeat_back_addr":"172.21.15.151:6807/3368304087","heartbeat_front_addr":"172.21.15.151:6805/3368304087","state":["exists","up"]},{"osd":2,"uuid":"c2f18115-013d-4da6-a0fe-cb978f4494b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6823","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6825","nonce":3079373271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6827","nonce":3079373271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6831","nonce":3079373271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6829","nonce":3079373271}]},"public_addr":"172.21.15.151:6825/3079373271","cluster_addr":"172.21.15.151:6827/3079373271","heartbeat_back_addr":"172.21.15.151:6831/3079373271","heartbeat_front_addr":"172.21.15.151:6829/3079373271","state":["exists","up"]},{"osd":3,"uuid":"36edb718-2fcb-4d88-8ddc-c321eed4279d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6817","nonce":1142509041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6819","nonce":1142509041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6824","nonce":1142509041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6821","nonce":1142509041}]},"public_addr":"172.21.15.151:6817/1142509041","cluster_addr":"172.21.15.151:6819/1142509041","heartbeat_back_addr":"172.21.15.151:6824/1142509041","heartbeat_front_addr":"172.21.15.151:6821/1142509041","state":["exists","up"]},{"osd":4,"uuid":"64d33708-cd0a-4236-abe3-1b646e82ab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6811","nonce":2735495271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6813","nonce":2735495271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6817","nonce":2735495271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6815","nonce":2735495271}]},"public_addr":"172.21.15.167:6811/2735495271","cluster_addr":"172.21.15.167:6813/2735495271","heartbeat_back_addr":"172.21.15.167:6817/2735495271","heartbeat_front_addr":"172.21.15.167:6815/2735495271","state":["exists","up"]},{"osd":5,"uuid":"f14b44a9-5910-458c-b518-09a8d0de94de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6827","nonce":1494916572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6829","nonce":1494916572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6833","nonce":1494916572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6831","nonce":1494916572}]},"public_addr":"172.21.15.167:6827/1494916572","cluster_addr":"172.21.15.167:6829/1494916572","heartbeat_back_addr":"172.21.15.167:6833/1494916572","heartbeat_front_addr":"172.21.15.167:6831/1494916572","state":["exists","up"]},{"osd":6,"uuid":"39335545-2654-4710-84b0-a9aa47cfcbba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6819","nonce":2538273273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6821","nonce":2538273273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6825","nonce":2538273273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6823","nonce":2538273273}]},"public_addr":"172.21.15.167:6819/2538273273","cluster_addr":"172.21.15.167:6821/2538273273","heartbeat_back_addr":"172.21.15.167:6825/2538273273","heartbeat_front_addr":"172.21.15.167:6823/2538273273","state":["exists","up"]},{"osd":7,"uuid":"4f8bfc81-53dd-4339-ac22-1a4007d9f963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6803","nonce":265072227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6805","nonce":265072227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6809","nonce":265072227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6807","nonce":265072227}]},"public_addr":"172.21.15.167:6803/265072227","cluster_addr":"172.21.15.167:6805/265072227","heartbeat_back_addr":"172.21.15.167:6809/265072227","heartbeat_front_addr":"172.21.15.167:6807/265072227","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.150359+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.084358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.722368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.332809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.320709+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.395153+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.935250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.801443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":13,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:50:48.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.699+0000 7f67cc600640 1 -- 172.21.15.151:0/3831794868 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f67a4079ed0 msgr2=0x7f67a407c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:48.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.699+0000 7f67cc600640 1 --2- 172.21.15.151:0/3831794868 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f67a4079ed0 0x7f67a407c390 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f67b0033000 tx=0x7f67b0004730 comp rx=0 tx=0).stop 2024-09-16T15:50:48.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.699+0000 7f67cc600640 1 -- 172.21.15.151:0/3831794868 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f67c40ad270 msgr2=0x7f67c4146430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:48.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.699+0000 7f67cc600640 1 --2- 172.21.15.151:0/3831794868 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f67c40ad270 0x7f67c4146430 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f67b800d700 tx=0x7f67b800dbc0 comp rx=0 tx=0).stop 2024-09-16T15:50:48.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.699+0000 7f67cc600640 1 -- 172.21.15.151:0/3831794868 shutdown_connections 2024-09-16T15:50:48.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.699+0000 7f67cc600640 1 --2- 172.21.15.151:0/3831794868 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f67a4079ed0 0x7f67a407c390 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:48.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.699+0000 7f67cc600640 1 --2- 172.21.15.151:0/3831794868 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67c40ccf00 0x7f67c415da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:48.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.699+0000 7f67cc600640 1 --2- 172.21.15.151:0/3831794868 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f67c415f5a0 0x7f67c415df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:48.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.699+0000 7f67cc600640 1 --2- 172.21.15.151:0/3831794868 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f67c40ad270 0x7f67c4146430 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:48.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.699+0000 7f67cc600640 1 -- 172.21.15.151:0/3831794868 >> 172.21.15.151:0/3831794868 conn(0x7f67c40b1950 msgr2=0x7f67c409c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:48.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.699+0000 7f67cc600640 1 -- 172.21.15.151:0/3831794868 shutdown_connections 2024-09-16T15:50:48.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.699+0000 7f67cc600640 1 -- 172.21.15.151:0/3831794868 wait complete. 2024-09-16T15:50:48.716 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-16T15:50:48.716 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:50:48.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.875+0000 7f6372400640 1 -- 172.21.15.151:0/3542176930 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63500052e0 msgr2=0x7f635000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:48.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.875+0000 7f6372400640 1 --2- 172.21.15.151:0/3542176930 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63500052e0 0x7f635000b9c0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f6364005fd0 tx=0x7f6350005830 comp rx=0 tx=0).stop 2024-09-16T15:50:48.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.875+0000 7f6372400640 1 -- 172.21.15.151:0/3542176930 shutdown_connections 2024-09-16T15:50:48.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.875+0000 7f6372400640 1 --2- 172.21.15.151:0/3542176930 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63500052e0 0x7f635000b9c0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:48.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.875+0000 7f6372400640 1 -- 172.21.15.151:0/3542176930 >> 172.21.15.151:0/3542176930 conn(0x7f636c0b1950 msgr2=0x7f636c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:48.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.875+0000 7f6372400640 1 -- 172.21.15.151:0/3542176930 shutdown_connections 2024-09-16T15:50:48.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.875+0000 7f6372400640 1 -- 172.21.15.151:0/3542176930 wait complete. 2024-09-16T15:50:48.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.877+0000 7f6372400640 1 Processor -- start 2024-09-16T15:50:48.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.877+0000 7f6372400640 1 -- start start 2024-09-16T15:50:48.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.878+0000 7f6372400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63500052e0 0x7f636c0cc710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:48.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.878+0000 7f6372400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f636c14b4a0 0x7f636c0c7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:48.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.878+0000 7f6372400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f636c14b920 0x7f636c0c8420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:48.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.878+0000 7f6372400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f636c0ad0c0 con 0x7f636c14b920 2024-09-16T15:50:48.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.878+0000 7f6372400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f636c0acf40 con 0x7f63500052e0 2024-09-16T15:50:48.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.878+0000 7f6372400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f636c0ad240 con 0x7f636c14b4a0 2024-09-16T15:50:48.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.878+0000 7f636be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f636c14b920 0x7f636c0c8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:48.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.878+0000 7f636b400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63500052e0 0x7f636c0cc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:48.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.878+0000 7f636b400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63500052e0 0x7f636c0cc710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.151:47774/0 (socket says 172.21.15.151:47774) 2024-09-16T15:50:48.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.878+0000 7f636b400640 1 -- 172.21.15.151:0/949278765 learned_addr learned my addr 172.21.15.151:0/949278765 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:48.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.878+0000 7f6363e00640 1 --2- 172.21.15.151:0/949278765 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f636c14b4a0 0x7f636c0c7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:48.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.878+0000 7f636b400640 1 -- 172.21.15.151:0/949278765 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f636c14b4a0 msgr2=0x7f636c0c7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:48.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.878+0000 7f636b400640 1 --2- 172.21.15.151:0/949278765 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f636c14b4a0 0x7f636c0c7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:48.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.878+0000 7f636b400640 1 -- 172.21.15.151:0/949278765 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f636c14b920 msgr2=0x7f636c0c8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:48.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.878+0000 7f636b400640 1 --2- 172.21.15.151:0/949278765 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f636c14b920 0x7f636c0c8420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:48.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.878+0000 7f636b400640 1 -- 172.21.15.151:0/949278765 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f636c0ccc50 con 0x7f63500052e0 2024-09-16T15:50:48.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.878+0000 7f636be00640 1 --2- 172.21.15.151:0/949278765 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f636c14b920 0x7f636c0c8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:50:48.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.879+0000 7f636b400640 1 --2- 172.21.15.151:0/949278765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63500052e0 0x7f636c0cc710 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f635c003ed0 tx=0x7f635c001630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:48.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.879+0000 7f6363400640 1 -- 172.21.15.151:0/949278765 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f635c015070 con 0x7f63500052e0 2024-09-16T15:50:48.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.879+0000 7f6363400640 1 -- 172.21.15.151:0/949278765 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f635c011840 con 0x7f63500052e0 2024-09-16T15:50:48.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.879+0000 7f6372400640 1 -- 172.21.15.151:0/949278765 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f636c15b750 con 0x7f63500052e0 2024-09-16T15:50:48.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.879+0000 7f6363400640 1 -- 172.21.15.151:0/949278765 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f635c011c90 con 0x7f63500052e0 2024-09-16T15:50:48.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.879+0000 7f6372400640 1 -- 172.21.15.151:0/949278765 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f636c15bd10 con 0x7f63500052e0 2024-09-16T15:50:48.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.880+0000 7f6372400640 1 -- 172.21.15.151:0/949278765 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f636c073fb0 con 0x7f63500052e0 2024-09-16T15:50:48.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.881+0000 7f6363400640 1 -- 172.21.15.151:0/949278765 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7f635c018070 con 0x7f63500052e0 2024-09-16T15:50:48.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.882+0000 7f6363400640 1 --2- 172.21.15.151:0/949278765 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6348079ed0 0x7f634807c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:48.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.882+0000 7f6363400640 1 -- 172.21.15.151:0/949278765 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5793+0+0 (secure 0 0 0) 0x7f635c09a120 con 0x7f63500052e0 2024-09-16T15:50:48.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.883+0000 7f6363e00640 1 --2- 172.21.15.151:0/949278765 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6348079ed0 0x7f634807c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:48.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.883+0000 7f6363e00640 1 --2- 172.21.15.151:0/949278765 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6348079ed0 0x7f634807c390 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f6354001220 tx=0x7f63540067a0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:48.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:48.887+0000 7f6363400640 1 -- 172.21.15.151:0/949278765 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f635c063c40 con 0x7f63500052e0 2024-09-16T15:50:49.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.107+0000 7f6372400640 1 -- 172.21.15.151:0/949278765 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f636c0b1310 con 0x7f63500052e0 2024-09-16T15:50:49.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.108+0000 7f6363400640 1 -- 172.21.15.151:0/949278765 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+5234 (secure 0 0 0) 0x7f635c068670 con 0x7f63500052e0 2024-09-16T15:50:49.110 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:50:49.110 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":57,"btime":"2024-09-16T15:48:35:029906+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6134,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/743718626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":743718626},{"type":"v1","addr":"172.21.15.167:6836","nonce":743718626}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6137,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1262689373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1262689373},{"type":"v1","addr":"172.21.15.167:6837","nonce":1262689373}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6670,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/1127792363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1127792363},{"type":"v1","addr":"172.21.15.151:6834","nonce":1127792363}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:48:21.374084+0000","modified":"2024-09-16T15:48:24.408969+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6673},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6673":{"gid":6673,"name":"c","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.151:6835/1518702790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":1518702790},{"type":"v1","addr":"172.21.15.151:6835","nonce":1518702790}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6673,"qdb_cluster":[6673]},"id":6}]} 2024-09-16T15:50:49.110 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 57 2024-09-16T15:50:49.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.111+0000 7f6372400640 1 -- 172.21.15.151:0/949278765 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6348079ed0 msgr2=0x7f634807c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:49.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.111+0000 7f6372400640 1 --2- 172.21.15.151:0/949278765 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6348079ed0 0x7f634807c390 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f6354001220 tx=0x7f63540067a0 comp rx=0 tx=0).stop 2024-09-16T15:50:49.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.111+0000 7f6372400640 1 -- 172.21.15.151:0/949278765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63500052e0 msgr2=0x7f636c0cc710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:49.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.111+0000 7f6372400640 1 --2- 172.21.15.151:0/949278765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63500052e0 0x7f636c0cc710 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f635c003ed0 tx=0x7f635c001630 comp rx=0 tx=0).stop 2024-09-16T15:50:49.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.111+0000 7f6372400640 1 -- 172.21.15.151:0/949278765 shutdown_connections 2024-09-16T15:50:49.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.111+0000 7f6372400640 1 --2- 172.21.15.151:0/949278765 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6348079ed0 0x7f634807c390 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:49.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.111+0000 7f6372400640 1 --2- 172.21.15.151:0/949278765 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f636c14b920 0x7f636c0c8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:49.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.111+0000 7f6372400640 1 --2- 172.21.15.151:0/949278765 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f636c14b4a0 0x7f636c0c7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:49.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.111+0000 7f6372400640 1 --2- 172.21.15.151:0/949278765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63500052e0 0x7f636c0cc710 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:49.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.111+0000 7f6372400640 1 -- 172.21.15.151:0/949278765 >> 172.21.15.151:0/949278765 conn(0x7f636c0b1950 msgr2=0x7f636c15e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:49.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.111+0000 7f6372400640 1 -- 172.21.15.151:0/949278765 shutdown_connections 2024-09-16T15:50:49.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.111+0000 7f6372400640 1 -- 172.21.15.151:0/949278765 wait complete. 2024-09-16T15:50:49.130 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:50:49.287 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.286+0000 7ff9b0c00640 1 -- 172.21.15.151:0/2465408648 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff990003a00 msgr2=0x7ff990003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:49.287 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.286+0000 7ff9b0c00640 1 --2- 172.21.15.151:0/2465408648 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff990003a00 0x7ff990003e20 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7ff99c00bb20 tx=0x7ff990005830 comp rx=0 tx=0).stop 2024-09-16T15:50:49.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.286+0000 7ff9aa800640 1 -- 172.21.15.151:0/2465408648 reap_dead start 2024-09-16T15:50:49.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.286+0000 7ff9b0c00640 1 -- 172.21.15.151:0/2465408648 shutdown_connections 2024-09-16T15:50:49.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.286+0000 7ff9b0c00640 1 -- 172.21.15.151:0/2465408648 >> 172.21.15.151:0/2465408648 conn(0x7ff9ac0b1950 msgr2=0x7ff9ac0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:49.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.286+0000 7ff9b0c00640 1 -- 172.21.15.151:0/2465408648 shutdown_connections 2024-09-16T15:50:49.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.286+0000 7ff9b0c00640 1 -- 172.21.15.151:0/2465408648 wait complete. 2024-09-16T15:50:49.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.288+0000 7ff9b0c00640 1 Processor -- start 2024-09-16T15:50:49.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.288+0000 7ff9b0c00640 1 -- start start 2024-09-16T15:50:49.290 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.289+0000 7ff9b0c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff990003a00 0x7ff9ac15b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:49.290 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.289+0000 7ff9b0c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff9ac14e5d0 0x7ff9ac14aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:49.290 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.289+0000 7ff9b0c00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff9ac1596c0 0x7ff9ac0d0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:49.290 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.289+0000 7ff9b0c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ac0745c0 con 0x7ff990003a00 2024-09-16T15:50:49.290 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.289+0000 7ff9b0c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ac074440 con 0x7ff9ac14e5d0 2024-09-16T15:50:49.290 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.289+0000 7ff9b0c00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7ff9ac074740 con 0x7ff9ac1596c0 2024-09-16T15:50:49.290 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.289+0000 7ff9a9e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff990003a00 0x7ff9ac15b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:49.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.289+0000 7ff9a9e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff990003a00 0x7ff9ac15b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:45940/0 (socket says 172.21.15.151:45940) 2024-09-16T15:50:49.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.289+0000 7ff9a9e00640 1 -- 172.21.15.151:0/2702410695 learned_addr learned my addr 172.21.15.151:0/2702410695 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:49.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.289+0000 7ff9a9400640 1 --2- 172.21.15.151:0/2702410695 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff9ac14e5d0 0x7ff9ac14aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:49.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.289+0000 7ff9a9e00640 1 -- 172.21.15.151:0/2702410695 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff9ac1596c0 msgr2=0x7ff9ac0d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:49.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.289+0000 7ff9a9e00640 1 --2- 172.21.15.151:0/2702410695 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff9ac1596c0 0x7ff9ac0d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:49.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.289+0000 7ff9a9e00640 1 -- 172.21.15.151:0/2702410695 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff9ac14e5d0 msgr2=0x7ff9ac14aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:49.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.289+0000 7ff9a9e00640 1 --2- 172.21.15.151:0/2702410695 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff9ac14e5d0 0x7ff9ac14aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:49.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.289+0000 7ff9a9e00640 1 -- 172.21.15.151:0/2702410695 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9ac159fc0 con 0x7ff990003a00 2024-09-16T15:50:49.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.290+0000 7ff9a9e00640 1 --2- 172.21.15.151:0/2702410695 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff990003a00 0x7ff9ac15b060 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7ff99c034040 tx=0x7ff99c00b7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:49.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.290+0000 7ff9a2000640 1 -- 172.21.15.151:0/2702410695 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99c040070 con 0x7ff990003a00 2024-09-16T15:50:49.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.290+0000 7ff9a2000640 1 -- 172.21.15.151:0/2702410695 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff99c004230 con 0x7ff990003a00 2024-09-16T15:50:49.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.290+0000 7ff9a2000640 1 -- 172.21.15.151:0/2702410695 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99c035040 con 0x7ff990003a00 2024-09-16T15:50:49.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.290+0000 7ff9b0c00640 1 -- 172.21.15.151:0/2702410695 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9ac15a2a0 con 0x7ff990003a00 2024-09-16T15:50:49.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.290+0000 7ff9b0c00640 1 -- 172.21.15.151:0/2702410695 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9ac15a860 con 0x7ff990003a00 2024-09-16T15:50:49.293 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.291+0000 7ff9b0c00640 1 -- 172.21.15.151:0/2702410695 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9ac0afa90 con 0x7ff990003a00 2024-09-16T15:50:49.293 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.292+0000 7ff9a2000640 1 -- 172.21.15.151:0/2702410695 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7ff99c038020 con 0x7ff990003a00 2024-09-16T15:50:49.294 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.293+0000 7ff9a2000640 1 --2- 172.21.15.151:0/2702410695 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff98c079f00 0x7ff98c07c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:49.294 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.293+0000 7ff9a2000640 1 -- 172.21.15.151:0/2702410695 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5793+0+0 (secure 0 0 0) 0x7ff99c0ba140 con 0x7ff990003a00 2024-09-16T15:50:49.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.294+0000 7ff9a9400640 1 --2- 172.21.15.151:0/2702410695 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff98c079f00 0x7ff98c07c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:49.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.295+0000 7ff9a9400640 1 --2- 172.21.15.151:0/2702410695 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff98c079f00 0x7ff98c07c3c0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7ff998001240 tx=0x7ff998001110 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:49.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.297+0000 7ff9a2000640 1 -- 172.21.15.151:0/2702410695 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff9ac0afa90 con 0x7ff990003a00 2024-09-16T15:50:49.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.517+0000 7ff9b0c00640 1 -- 172.21.15.151:0/2702410695 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff9ac09cf30 con 0x7ff990003a00 2024-09-16T15:50:49.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.518+0000 7ff9a2000640 1 -- 172.21.15.151:0/2702410695 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+5234 (secure 0 0 0) 0x7ff99c0842b0 con 0x7ff990003a00 2024-09-16T15:50:49.519 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:50:49.519 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":57,"btime":"2024-09-16T15:48:35:029906+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6134,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/743718626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":743718626},{"type":"v1","addr":"172.21.15.167:6836","nonce":743718626}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6137,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1262689373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1262689373},{"type":"v1","addr":"172.21.15.167:6837","nonce":1262689373}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6670,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/1127792363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1127792363},{"type":"v1","addr":"172.21.15.151:6834","nonce":1127792363}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:48:21.374084+0000","modified":"2024-09-16T15:48:24.408969+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6673},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6673":{"gid":6673,"name":"c","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.151:6835/1518702790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":1518702790},{"type":"v1","addr":"172.21.15.151:6835","nonce":1518702790}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6673,"qdb_cluster":[6673]},"id":6}]} 2024-09-16T15:50:49.520 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 57 2024-09-16T15:50:49.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.520+0000 7ff9b0c00640 1 -- 172.21.15.151:0/2702410695 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff98c079f00 msgr2=0x7ff98c07c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:49.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.520+0000 7ff9b0c00640 1 --2- 172.21.15.151:0/2702410695 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff98c079f00 0x7ff98c07c3c0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7ff998001240 tx=0x7ff998001110 comp rx=0 tx=0).stop 2024-09-16T15:50:49.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.520+0000 7ff9b0c00640 1 -- 172.21.15.151:0/2702410695 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff990003a00 msgr2=0x7ff9ac15b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:49.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.520+0000 7ff9b0c00640 1 --2- 172.21.15.151:0/2702410695 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff990003a00 0x7ff9ac15b060 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7ff99c034040 tx=0x7ff99c00b7d0 comp rx=0 tx=0).stop 2024-09-16T15:50:49.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.520+0000 7ff9b0c00640 1 -- 172.21.15.151:0/2702410695 shutdown_connections 2024-09-16T15:50:49.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.520+0000 7ff9b0c00640 1 --2- 172.21.15.151:0/2702410695 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff98c079f00 0x7ff98c07c3c0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:49.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.520+0000 7ff9b0c00640 1 --2- 172.21.15.151:0/2702410695 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff9ac14e5d0 0x7ff9ac14aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:49.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.520+0000 7ff9b0c00640 1 --2- 172.21.15.151:0/2702410695 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff9ac1596c0 0x7ff9ac0d0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:49.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.520+0000 7ff9b0c00640 1 --2- 172.21.15.151:0/2702410695 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff990003a00 0x7ff9ac15b060 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:49.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.520+0000 7ff9b0c00640 1 -- 172.21.15.151:0/2702410695 >> 172.21.15.151:0/2702410695 conn(0x7ff9ac0b1950 msgr2=0x7ff9ac06f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:49.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.520+0000 7ff9b0c00640 1 -- 172.21.15.151:0/2702410695 shutdown_connections 2024-09-16T15:50:49.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.521+0000 7ff9b0c00640 1 -- 172.21.15.151:0/2702410695 wait complete. 2024-09-16T15:50:49.537 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:50:49.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.694+0000 7fc850a00640 1 -- 172.21.15.151:0/1423366233 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc84c073fb0 msgr2=0x7fc84c074420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:49.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.694+0000 7fc850a00640 1 --2- 172.21.15.151:0/1423366233 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc84c073fb0 0x7fc84c074420 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fc84c0cfbb0 tx=0x7fc84402a820 comp rx=0 tx=0).stop 2024-09-16T15:50:49.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.694+0000 7fc850a00640 1 -- 172.21.15.151:0/1423366233 shutdown_connections 2024-09-16T15:50:49.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.694+0000 7fc850a00640 1 --2- 172.21.15.151:0/1423366233 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc84c073fb0 0x7fc84c074420 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:49.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.694+0000 7fc850a00640 1 --2- 172.21.15.151:0/1423366233 >> v2:172.21.15.167:3300/0 conn(0x7fc84c0ccf00 0x7fc84c0b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:49.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.694+0000 7fc850a00640 1 -- 172.21.15.151:0/1423366233 >> 172.21.15.151:0/1423366233 conn(0x7fc84c0b1950 msgr2=0x7fc84c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:49.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.694+0000 7fc850a00640 1 -- 172.21.15.151:0/1423366233 shutdown_connections 2024-09-16T15:50:49.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.694+0000 7fc850a00640 1 -- 172.21.15.151:0/1423366233 wait complete. 2024-09-16T15:50:49.697 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.696+0000 7fc850a00640 1 Processor -- start 2024-09-16T15:50:49.697 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.696+0000 7fc850a00640 1 -- start start 2024-09-16T15:50:49.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.697+0000 7fc850a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc84c073fb0 0x7fc84c0cba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:49.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.697+0000 7fc850a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc84c0ccf00 0x7fc84c1f86c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:49.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.697+0000 7fc850a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc84c15f5a0 0x7fc84c0cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:49.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.697+0000 7fc850a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc84c06f2e0 con 0x7fc84c0ccf00 2024-09-16T15:50:49.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.697+0000 7fc850a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc84c06f160 con 0x7fc84c15f5a0 2024-09-16T15:50:49.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.697+0000 7fc850a00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fc84c06f460 con 0x7fc84c073fb0 2024-09-16T15:50:49.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.697+0000 7fc849400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc84c0ccf00 0x7fc84c1f86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:49.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.697+0000 7fc849400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc84c0ccf00 0x7fc84c1f86c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:45960/0 (socket says 172.21.15.151:45960) 2024-09-16T15:50:49.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.697+0000 7fc849400640 1 -- 172.21.15.151:0/3288747758 learned_addr learned my addr 172.21.15.151:0/3288747758 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:49.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.697+0000 7fc84a800640 1 --2- 172.21.15.151:0/3288747758 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc84c15f5a0 0x7fc84c0cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:49.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.697+0000 7fc849e00640 1 --2- 172.21.15.151:0/3288747758 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc84c073fb0 0x7fc84c0cba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:49.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.698+0000 7fc849400640 1 -- 172.21.15.151:0/3288747758 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc84c073fb0 msgr2=0x7fc84c0cba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:49.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.698+0000 7fc849400640 1 --2- 172.21.15.151:0/3288747758 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc84c073fb0 0x7fc84c0cba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:49.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.698+0000 7fc849400640 1 -- 172.21.15.151:0/3288747758 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc84c15f5a0 msgr2=0x7fc84c0cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:49.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.698+0000 7fc849400640 1 --2- 172.21.15.151:0/3288747758 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc84c15f5a0 0x7fc84c0cdae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:49.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.698+0000 7fc849400640 1 -- 172.21.15.151:0/3288747758 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc84c0ce5f0 con 0x7fc84c0ccf00 2024-09-16T15:50:49.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.698+0000 7fc84a800640 1 --2- 172.21.15.151:0/3288747758 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc84c15f5a0 0x7fc84c0cdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:50:49.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.698+0000 7fc849e00640 1 --2- 172.21.15.151:0/3288747758 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc84c073fb0 0x7fc84c0cba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:50:49.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.698+0000 7fc849400640 1 --2- 172.21.15.151:0/3288747758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc84c0ccf00 0x7fc84c1f86c0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fc83800d950 tx=0x7fc83800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:49.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.698+0000 7fc842000640 1 -- 172.21.15.151:0/3288747758 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc838017070 con 0x7fc84c0ccf00 2024-09-16T15:50:49.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.698+0000 7fc842000640 1 -- 172.21.15.151:0/3288747758 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc838009d20 con 0x7fc84c0ccf00 2024-09-16T15:50:49.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.698+0000 7fc842000640 1 -- 172.21.15.151:0/3288747758 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8380106c0 con 0x7fc84c0ccf00 2024-09-16T15:50:49.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.698+0000 7fc850a00640 1 -- 172.21.15.151:0/3288747758 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc84c0ce310 con 0x7fc84c0ccf00 2024-09-16T15:50:49.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.699+0000 7fc850a00640 1 -- 172.21.15.151:0/3288747758 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc84c1f7ec0 con 0x7fc84c0ccf00 2024-09-16T15:50:49.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.700+0000 7fc850a00640 1 -- 172.21.15.151:0/3288747758 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc84c06f9f0 con 0x7fc84c0ccf00 2024-09-16T15:50:49.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.701+0000 7fc842000640 1 -- 172.21.15.151:0/3288747758 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7fc838010860 con 0x7fc84c0ccf00 2024-09-16T15:50:49.703 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.702+0000 7fc842000640 1 --2- 172.21.15.151:0/3288747758 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc82c079f00 0x7fc82c07c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:49.703 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.702+0000 7fc842000640 1 -- 172.21.15.151:0/3288747758 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5793+0+0 (secure 0 0 0) 0x7fc8380999d0 con 0x7fc84c0ccf00 2024-09-16T15:50:49.703 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.702+0000 7fc849e00640 1 --2- 172.21.15.151:0/3288747758 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc82c079f00 0x7fc82c07c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:49.703 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.703+0000 7fc849e00640 1 --2- 172.21.15.151:0/3288747758 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc82c079f00 0x7fc82c07c3c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fc83c000f60 tx=0x7fc83c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:49.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.706+0000 7fc842000640 1 -- 172.21.15.151:0/3288747758 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc838063b40 con 0x7fc84c0ccf00 2024-09-16T15:50:49.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.895+0000 7fc850a00640 1 -- 172.21.15.151:0/3288747758 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc84c0acdd0 con 0x7fc84c0ccf00 2024-09-16T15:50:49.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.896+0000 7fc842000640 1 -- 172.21.15.151:0/3288747758 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+16945 (secure 0 0 0) 0x7fc838063440 con 0x7fc84c0ccf00 2024-09-16T15:50:49.898 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:50:49.898 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":93,"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","created":"2024-09-16T15:38:56.348535+0000","modified":"2024-09-16T15:50:46.473983+0000","last_up_change":"2024-09-16T15:39:10.681384+0000","last_in_change":"2024-09-16T15:39:00.544477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:39:11.602252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:48:19.150224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-16T15:48:20.351519+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":92,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5524076-a903-4d0a-af6a-caf474c4765d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6809","nonce":3111342741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6811","nonce":3111342741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6815","nonce":3111342741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6813","nonce":3111342741}]},"public_addr":"172.21.15.151:6809/3111342741","cluster_addr":"172.21.15.151:6811/3111342741","heartbeat_back_addr":"172.21.15.151:6815/3111342741","heartbeat_front_addr":"172.21.15.151:6813/3111342741","state":["exists","up"]},{"osd":1,"uuid":"dd25925f-0ae1-494e-8529-4dc205759db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6801","nonce":3368304087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6803","nonce":3368304087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6807","nonce":3368304087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6805","nonce":3368304087}]},"public_addr":"172.21.15.151:6801/3368304087","cluster_addr":"172.21.15.151:6803/3368304087","heartbeat_back_addr":"172.21.15.151:6807/3368304087","heartbeat_front_addr":"172.21.15.151:6805/3368304087","state":["exists","up"]},{"osd":2,"uuid":"c2f18115-013d-4da6-a0fe-cb978f4494b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6823","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6825","nonce":3079373271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6827","nonce":3079373271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6831","nonce":3079373271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6829","nonce":3079373271}]},"public_addr":"172.21.15.151:6825/3079373271","cluster_addr":"172.21.15.151:6827/3079373271","heartbeat_back_addr":"172.21.15.151:6831/3079373271","heartbeat_front_addr":"172.21.15.151:6829/3079373271","state":["exists","up"]},{"osd":3,"uuid":"36edb718-2fcb-4d88-8ddc-c321eed4279d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6817","nonce":1142509041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6819","nonce":1142509041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6824","nonce":1142509041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6821","nonce":1142509041}]},"public_addr":"172.21.15.151:6817/1142509041","cluster_addr":"172.21.15.151:6819/1142509041","heartbeat_back_addr":"172.21.15.151:6824/1142509041","heartbeat_front_addr":"172.21.15.151:6821/1142509041","state":["exists","up"]},{"osd":4,"uuid":"64d33708-cd0a-4236-abe3-1b646e82ab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6811","nonce":2735495271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6813","nonce":2735495271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6817","nonce":2735495271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6815","nonce":2735495271}]},"public_addr":"172.21.15.167:6811/2735495271","cluster_addr":"172.21.15.167:6813/2735495271","heartbeat_back_addr":"172.21.15.167:6817/2735495271","heartbeat_front_addr":"172.21.15.167:6815/2735495271","state":["exists","up"]},{"osd":5,"uuid":"f14b44a9-5910-458c-b518-09a8d0de94de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6827","nonce":1494916572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6829","nonce":1494916572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6833","nonce":1494916572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6831","nonce":1494916572}]},"public_addr":"172.21.15.167:6827/1494916572","cluster_addr":"172.21.15.167:6829/1494916572","heartbeat_back_addr":"172.21.15.167:6833/1494916572","heartbeat_front_addr":"172.21.15.167:6831/1494916572","state":["exists","up"]},{"osd":6,"uuid":"39335545-2654-4710-84b0-a9aa47cfcbba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6819","nonce":2538273273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6821","nonce":2538273273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6825","nonce":2538273273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6823","nonce":2538273273}]},"public_addr":"172.21.15.167:6819/2538273273","cluster_addr":"172.21.15.167:6821/2538273273","heartbeat_back_addr":"172.21.15.167:6825/2538273273","heartbeat_front_addr":"172.21.15.167:6823/2538273273","state":["exists","up"]},{"osd":7,"uuid":"4f8bfc81-53dd-4339-ac22-1a4007d9f963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6803","nonce":265072227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6805","nonce":265072227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6809","nonce":265072227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6807","nonce":265072227}]},"public_addr":"172.21.15.167:6803/265072227","cluster_addr":"172.21.15.167:6805/265072227","heartbeat_back_addr":"172.21.15.167:6809/265072227","heartbeat_front_addr":"172.21.15.167:6807/265072227","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.150359+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.084358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.722368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.332809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.320709+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.395153+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.935250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.801443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":13,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:50:49.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.898+0000 7fc850a00640 1 -- 172.21.15.151:0/3288747758 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc82c079f00 msgr2=0x7fc82c07c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:49.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.899+0000 7fc850a00640 1 --2- 172.21.15.151:0/3288747758 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc82c079f00 0x7fc82c07c3c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fc83c000f60 tx=0x7fc83c009040 comp rx=0 tx=0).stop 2024-09-16T15:50:49.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.899+0000 7fc850a00640 1 -- 172.21.15.151:0/3288747758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc84c0ccf00 msgr2=0x7fc84c1f86c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:49.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.899+0000 7fc850a00640 1 --2- 172.21.15.151:0/3288747758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc84c0ccf00 0x7fc84c1f86c0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fc83800d950 tx=0x7fc83800de10 comp rx=0 tx=0).stop 2024-09-16T15:50:49.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.899+0000 7fc850a00640 1 -- 172.21.15.151:0/3288747758 shutdown_connections 2024-09-16T15:50:49.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.899+0000 7fc850a00640 1 --2- 172.21.15.151:0/3288747758 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc82c079f00 0x7fc82c07c3c0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:49.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.899+0000 7fc850a00640 1 --2- 172.21.15.151:0/3288747758 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc84c15f5a0 0x7fc84c0cdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:49.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.899+0000 7fc850a00640 1 --2- 172.21.15.151:0/3288747758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc84c0ccf00 0x7fc84c1f86c0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:49.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.899+0000 7fc850a00640 1 --2- 172.21.15.151:0/3288747758 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc84c073fb0 0x7fc84c0cba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:49.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.899+0000 7fc850a00640 1 -- 172.21.15.151:0/3288747758 >> 172.21.15.151:0/3288747758 conn(0x7fc84c0b1950 msgr2=0x7fc84c0b2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:49.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.899+0000 7fc850a00640 1 -- 172.21.15.151:0/3288747758 shutdown_connections 2024-09-16T15:50:49.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:49.899+0000 7fc850a00640 1 -- 172.21.15.151:0/3288747758 wait complete. 2024-09-16T15:50:49.916 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-16T15:50:50.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.074+0000 7fabc8800640 1 -- 172.21.15.151:0/4052622622 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fabc0073fb0 msgr2=0x7fabc0074440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:50.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.074+0000 7fabc8800640 1 --2- 172.21.15.151:0/4052622622 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fabc0073fb0 0x7fabc0074440 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fabbc005fd0 tx=0x7fabbc02aa00 comp rx=0 tx=0).stop 2024-09-16T15:50:50.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.074+0000 7fabc8800640 1 -- 172.21.15.151:0/4052622622 shutdown_connections 2024-09-16T15:50:50.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.074+0000 7fabc8800640 1 --2- 172.21.15.151:0/4052622622 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fabc0073fb0 0x7fabc0074440 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:50.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.074+0000 7fabc8800640 1 --2- 172.21.15.151:0/4052622622 >> v2:172.21.15.167:3300/0 conn(0x7fabc00ccf00 0x7fabc00b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:50.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.074+0000 7fabc8800640 1 -- 172.21.15.151:0/4052622622 >> 172.21.15.151:0/4052622622 conn(0x7fabc00b1950 msgr2=0x7fabc00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:50.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.074+0000 7fabc8800640 1 -- 172.21.15.151:0/4052622622 shutdown_connections 2024-09-16T15:50:50.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.074+0000 7fabc8800640 1 -- 172.21.15.151:0/4052622622 wait complete. 2024-09-16T15:50:50.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.076+0000 7fabc8800640 1 Processor -- start 2024-09-16T15:50:50.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.076+0000 7fabc8800640 1 -- start start 2024-09-16T15:50:50.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.077+0000 7fabc8800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fabc0073fb0 0x7fabc01f1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:50.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.077+0000 7fabc8800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fabc00ccf00 0x7fabc0159610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:50.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.077+0000 7fabc8800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fabc015f5a0 0x7fabc00b4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:50.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.077+0000 7fabc8800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fabc006f2e0 con 0x7fabc00ccf00 2024-09-16T15:50:50.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.077+0000 7fabc8800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fabc006f160 con 0x7fabc0073fb0 2024-09-16T15:50:50.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.077+0000 7fabc8800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fabc006f460 con 0x7fabc015f5a0 2024-09-16T15:50:50.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.077+0000 7fabc5400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fabc00ccf00 0x7fabc0159610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:50.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.077+0000 7fabc5400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fabc00ccf00 0x7fabc0159610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:45964/0 (socket says 172.21.15.151:45964) 2024-09-16T15:50:50.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.077+0000 7fabc5400640 1 -- 172.21.15.151:0/2585129814 learned_addr learned my addr 172.21.15.151:0/2585129814 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:50.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.077+0000 7fabc5e00640 1 --2- 172.21.15.151:0/2585129814 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fabc0073fb0 0x7fabc01f1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:50.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.077+0000 7fabc6800640 1 --2- 172.21.15.151:0/2585129814 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fabc015f5a0 0x7fabc00b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:50.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.077+0000 7fabc5400640 1 -- 172.21.15.151:0/2585129814 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fabc015f5a0 msgr2=0x7fabc00b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:50.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.077+0000 7fabc5400640 1 --2- 172.21.15.151:0/2585129814 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fabc015f5a0 0x7fabc00b4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:50.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.077+0000 7fabc5400640 1 -- 172.21.15.151:0/2585129814 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fabc0073fb0 msgr2=0x7fabc01f1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:50.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.077+0000 7fabc5400640 1 --2- 172.21.15.151:0/2585129814 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fabc0073fb0 0x7fabc01f1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:50.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.077+0000 7fabc5400640 1 -- 172.21.15.151:0/2585129814 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabc01f8f00 con 0x7fabc00ccf00 2024-09-16T15:50:50.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.078+0000 7fabc5400640 1 --2- 172.21.15.151:0/2585129814 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fabc00ccf00 0x7fabc0159610 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fabb000ea60 tx=0x7fabb000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:50.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.078+0000 7fabba000640 1 -- 172.21.15.151:0/2585129814 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb000cde0 con 0x7fabc00ccf00 2024-09-16T15:50:50.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.078+0000 7fabba000640 1 -- 172.21.15.151:0/2585129814 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fabb0004020 con 0x7fabc00ccf00 2024-09-16T15:50:50.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.078+0000 7fabc8800640 1 -- 172.21.15.151:0/2585129814 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabc00cebd0 con 0x7fabc00ccf00 2024-09-16T15:50:50.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.078+0000 7fabba000640 1 -- 172.21.15.151:0/2585129814 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb0004450 con 0x7fabc00ccf00 2024-09-16T15:50:50.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.078+0000 7fabc8800640 1 -- 172.21.15.151:0/2585129814 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabc00cf470 con 0x7fabc00ccf00 2024-09-16T15:50:50.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.079+0000 7fabc8800640 1 -- 172.21.15.151:0/2585129814 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabc00cefc0 con 0x7fabc00ccf00 2024-09-16T15:50:50.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.080+0000 7fabba000640 1 -- 172.21.15.151:0/2585129814 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7fabb00035f0 con 0x7fabc00ccf00 2024-09-16T15:50:50.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.081+0000 7fabba000640 1 --2- 172.21.15.151:0/2585129814 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7faba4079f00 0x7faba407c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:50.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.081+0000 7fabba000640 1 -- 172.21.15.151:0/2585129814 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5793+0+0 (secure 0 0 0) 0x7fabb009a0a0 con 0x7fabc00ccf00 2024-09-16T15:50:50.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.082+0000 7fabc5e00640 1 --2- 172.21.15.151:0/2585129814 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7faba4079f00 0x7faba407c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:50.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.083+0000 7fabc5e00640 1 --2- 172.21.15.151:0/2585129814 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7faba4079f00 0x7faba407c3c0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fabb40015e0 tx=0x7fabb4009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:50.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.085+0000 7fabba000640 1 -- 172.21.15.151:0/2585129814 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fabb0064170 con 0x7fabc00ccf00 2024-09-16T15:50:50.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.309+0000 7fabc8800640 1 -- 172.21.15.151:0/2585129814 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fabc01f2470 con 0x7fabc00ccf00 2024-09-16T15:50:50.311 INFO:tasks.ceph.mon.a.smithi151.stderr:2024-09-16T15:50:50.310+0000 7f62c3200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T15:50:50.313 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.312+0000 7fabba000640 1 -- 172.21.15.151:0/2585129814 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v58) ==== 161+0+0 (secure 0 0 0) 0x7fabb0063a70 con 0x7fabc00ccf00 2024-09-16T15:50:50.314 INFO:teuthology.orchestra.run.smithi151.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-16T15:50:50.317 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.316+0000 7fabc8800640 1 -- 172.21.15.151:0/2585129814 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7faba4079f00 msgr2=0x7faba407c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:50.317 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.316+0000 7fabc8800640 1 --2- 172.21.15.151:0/2585129814 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7faba4079f00 0x7faba407c3c0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fabb40015e0 tx=0x7fabb4009040 comp rx=0 tx=0).stop 2024-09-16T15:50:50.317 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.316+0000 7fabc8800640 1 -- 172.21.15.151:0/2585129814 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fabc00ccf00 msgr2=0x7fabc0159610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:50.317 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.316+0000 7fabc8800640 1 --2- 172.21.15.151:0/2585129814 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fabc00ccf00 0x7fabc0159610 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fabb000ea60 tx=0x7fabb000ef20 comp rx=0 tx=0).stop 2024-09-16T15:50:50.317 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.316+0000 7fabc8800640 1 -- 172.21.15.151:0/2585129814 shutdown_connections 2024-09-16T15:50:50.317 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.316+0000 7fabc8800640 1 --2- 172.21.15.151:0/2585129814 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7faba4079f00 0x7faba407c3c0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:50.318 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.316+0000 7fabc8800640 1 --2- 172.21.15.151:0/2585129814 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fabc015f5a0 0x7fabc00b4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:50.318 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.316+0000 7fabc8800640 1 --2- 172.21.15.151:0/2585129814 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fabc00ccf00 0x7fabc0159610 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:50.318 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.316+0000 7fabc8800640 1 --2- 172.21.15.151:0/2585129814 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fabc0073fb0 0x7fabc01f1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:50.318 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.316+0000 7fabc8800640 1 -- 172.21.15.151:0/2585129814 >> 172.21.15.151:0/2585129814 conn(0x7fabc00b1950 msgr2=0x7fabc00acd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:50.318 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.316+0000 7fabc8800640 1 -- 172.21.15.151:0/2585129814 shutdown_connections 2024-09-16T15:50:50.318 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.316+0000 7fabc8800640 1 -- 172.21.15.151:0/2585129814 wait complete. 2024-09-16T15:50:50.334 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-16T15:50:50.384 INFO:tasks.ceph.mds.c.smithi151.stdout:starting mds.c at 2024-09-16T15:50:50.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.494+0000 7f1032000640 1 -- 172.21.15.151:0/4112441094 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f102c0b3450 msgr2=0x7f102c0b3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:50.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.494+0000 7f1032000640 1 --2- 172.21.15.151:0/4112441094 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f102c0b3450 0x7f102c0b3890 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f1018005fd0 tx=0x7f101802a540 comp rx=0 tx=0).stop 2024-09-16T15:50:50.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.494+0000 7f1032000640 1 -- 172.21.15.151:0/4112441094 shutdown_connections 2024-09-16T15:50:50.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.494+0000 7f1032000640 1 --2- 172.21.15.151:0/4112441094 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f102c0cf4b0 0x7f102c0cf940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:50.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.494+0000 7f1032000640 1 --2- 172.21.15.151:0/4112441094 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f102c0b3450 0x7f102c0b3890 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:50.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.494+0000 7f1032000640 1 -- 172.21.15.151:0/4112441094 >> 172.21.15.151:0/4112441094 conn(0x7f102c0d3d50 msgr2=0x7f102c0d4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:50.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.494+0000 7f1032000640 1 -- 172.21.15.151:0/4112441094 shutdown_connections 2024-09-16T15:50:50.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.494+0000 7f1032000640 1 -- 172.21.15.151:0/4112441094 wait complete. 2024-09-16T15:50:50.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.496+0000 7f1032000640 1 Processor -- start 2024-09-16T15:50:50.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.496+0000 7f1032000640 1 -- start start 2024-09-16T15:50:50.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.497+0000 7f1032000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f102c0b3450 0x7f102c0b8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:50.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.497+0000 7f1032000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f102c0cf4b0 0x7f102c160fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:50.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.497+0000 7f1032000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f102c15df60 0x7f102c15f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:50.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.497+0000 7f1032000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f102c0d24b0 con 0x7f102c0cf4b0 2024-09-16T15:50:50.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.497+0000 7f1032000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f102c0d2330 con 0x7f102c15df60 2024-09-16T15:50:50.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.497+0000 7f1032000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f102c0d2630 con 0x7f102c0b3450 2024-09-16T15:50:50.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.497+0000 7f102be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f102c0cf4b0 0x7f102c160fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:50.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.497+0000 7f102be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f102c0cf4b0 0x7f102c160fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:46020/0 (socket says 172.21.15.151:46020) 2024-09-16T15:50:50.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.497+0000 7f102be00640 1 -- 172.21.15.151:0/3378311677 learned_addr learned my addr 172.21.15.151:0/3378311677 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:50.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.497+0000 7f1030c00640 1 --2- 172.21.15.151:0/3378311677 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f102c0b3450 0x7f102c0b8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:50.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.497+0000 7f1031600640 1 --2- 172.21.15.151:0/3378311677 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f102c15df60 0x7f102c15f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:50.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.497+0000 7f102be00640 1 -- 172.21.15.151:0/3378311677 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f102c0b3450 msgr2=0x7f102c0b8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:50.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.497+0000 7f102be00640 1 --2- 172.21.15.151:0/3378311677 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f102c0b3450 0x7f102c0b8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:50.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.497+0000 7f102be00640 1 -- 172.21.15.151:0/3378311677 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f102c15df60 msgr2=0x7f102c15f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:50.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.497+0000 7f102be00640 1 --2- 172.21.15.151:0/3378311677 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f102c15df60 0x7f102c15f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:50.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.497+0000 7f102be00640 1 -- 172.21.15.151:0/3378311677 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f102c1f7d10 con 0x7f102c0cf4b0 2024-09-16T15:50:50.501 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.498+0000 7f102be00640 1 --2- 172.21.15.151:0/3378311677 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f102c0cf4b0 0x7f102c160fc0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f1018009850 tx=0x7f10180038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:50.501 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.498+0000 7f1028c00640 1 -- 172.21.15.151:0/3378311677 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f101803f070 con 0x7f102c0cf4b0 2024-09-16T15:50:50.501 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.498+0000 7f1028c00640 1 -- 172.21.15.151:0/3378311677 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f10180047a0 con 0x7f102c0cf4b0 2024-09-16T15:50:50.501 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.498+0000 7f1028c00640 1 -- 172.21.15.151:0/3378311677 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1018035030 con 0x7f102c0cf4b0 2024-09-16T15:50:50.501 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.498+0000 7f1032000640 1 -- 172.21.15.151:0/3378311677 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f102c1f7ff0 con 0x7f102c0cf4b0 2024-09-16T15:50:50.501 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.498+0000 7f1032000640 1 -- 172.21.15.151:0/3378311677 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f102c1f85b0 con 0x7f102c0cf4b0 2024-09-16T15:50:50.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.499+0000 7f1032000640 1 -- 172.21.15.151:0/3378311677 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f102c0d2bc0 con 0x7f102c0cf4b0 2024-09-16T15:50:50.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.500+0000 7f1028c00640 1 -- 172.21.15.151:0/3378311677 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7f1018042070 con 0x7f102c0cf4b0 2024-09-16T15:50:50.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.501+0000 7f1028c00640 1 --2- 172.21.15.151:0/3378311677 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f100c079ed0 0x7f100c07c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:50.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.501+0000 7f1028c00640 1 -- 172.21.15.151:0/3378311677 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5851+0+0 (secure 0 0 0) 0x7f10180ba500 con 0x7f102c0cf4b0 2024-09-16T15:50:50.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.502+0000 7f1030c00640 1 --2- 172.21.15.151:0/3378311677 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f100c079ed0 0x7f100c07c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:50.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.503+0000 7f1030c00640 1 --2- 172.21.15.151:0/3378311677 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f100c079ed0 0x7f100c07c390 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f101c007fd0 tx=0x7f101c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:50.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.506+0000 7f1028c00640 1 -- 172.21.15.151:0/3378311677 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f10180845d0 con 0x7f102c0cf4b0 2024-09-16T15:50:50.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:50.728+0000 7f1032000640 1 -- 172.21.15.151:0/3378311677 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f102c0a1c70 con 0x7f102c0cf4b0 2024-09-16T15:50:50.841 DEBUG:teuthology.orchestra.run.smithi151:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:50:50.846 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:50:51.315 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:51.313+0000 7f1028c00640 1 -- 172.21.15.151:0/3378311677 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v59) ==== 104+0+0 (secure 0 0 0) 0x7f1018083ed0 con 0x7f102c0cf4b0 2024-09-16T15:50:51.318 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:51.316+0000 7f1032000640 1 -- 172.21.15.151:0/3378311677 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f100c079ed0 msgr2=0x7f100c07c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:51.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:51.316+0000 7f1032000640 1 --2- 172.21.15.151:0/3378311677 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f100c079ed0 0x7f100c07c390 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f101c007fd0 tx=0x7f101c009040 comp rx=0 tx=0).stop 2024-09-16T15:50:51.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:51.316+0000 7f1032000640 1 -- 172.21.15.151:0/3378311677 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f102c0cf4b0 msgr2=0x7f102c160fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:51.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:51.316+0000 7f1032000640 1 --2- 172.21.15.151:0/3378311677 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f102c0cf4b0 0x7f102c160fc0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f1018009850 tx=0x7f10180038a0 comp rx=0 tx=0).stop 2024-09-16T15:50:51.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:51.316+0000 7f1032000640 1 -- 172.21.15.151:0/3378311677 shutdown_connections 2024-09-16T15:50:51.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:51.316+0000 7f1032000640 1 --2- 172.21.15.151:0/3378311677 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f100c079ed0 0x7f100c07c390 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:51.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:51.316+0000 7f1032000640 1 --2- 172.21.15.151:0/3378311677 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f102c15df60 0x7f102c15f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:51.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:51.316+0000 7f1032000640 1 --2- 172.21.15.151:0/3378311677 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f102c0cf4b0 0x7f102c160fc0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:51.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:51.316+0000 7f1032000640 1 --2- 172.21.15.151:0/3378311677 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f102c0b3450 0x7f102c0b8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:51.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:51.316+0000 7f1032000640 1 -- 172.21.15.151:0/3378311677 >> 172.21.15.151:0/3378311677 conn(0x7f102c0d3d50 msgr2=0x7f102c0b0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:51.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:51.316+0000 7f1032000640 1 -- 172.21.15.151:0/3378311677 shutdown_connections 2024-09-16T15:50:51.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:51.317+0000 7f1032000640 1 -- 172.21.15.151:0/3378311677 wait complete. 2024-09-16T15:50:51.336 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-16T15:50:51.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:51.498+0000 7ff0dbe00640 1 -- 172.21.15.151:0/2135249816 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff0dc06e590 msgr2=0x7ff0dc06fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:51.501 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:51.498+0000 7ff0dbe00640 1 --2- 172.21.15.151:0/2135249816 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff0dc06e590 0x7ff0dc06fe40 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4009830 tx=0x7ff0c402a510 comp rx=0 tx=0).stop 2024-09-16T15:50:51.501 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:51.498+0000 7ff0dbe00640 1 -- 172.21.15.151:0/2135249816 shutdown_connections 2024-09-16T15:50:51.501 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:51.498+0000 7ff0dbe00640 1 --2- 172.21.15.151:0/2135249816 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff0dc06e590 0x7ff0dc06fe40 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:51.501 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:51.498+0000 7ff0dbe00640 1 --2- 172.21.15.151:0/2135249816 >> v2:172.21.15.167:3300/0 conn(0x7ff0dc15bda0 0x7ff0dc073010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:51.501 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:51.498+0000 7ff0dbe00640 1 -- 172.21.15.151:0/2135249816 >> 172.21.15.151:0/2135249816 conn(0x7ff0dc0cfa70 msgr2=0x7ff0dc0cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:51.501 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:51.498+0000 7ff0dbe00640 1 -- 172.21.15.151:0/2135249816 shutdown_connections 2024-09-16T15:50:51.501 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:51.498+0000 7ff0dbe00640 1 -- 172.21.15.151:0/2135249816 wait complete. 2024-09-16T15:50:51.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:51.500+0000 7ff0dbe00640 1 Processor -- start 2024-09-16T15:50:51.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:51.500+0000 7ff0dbe00640 1 -- start start 2024-09-16T15:50:51.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:51.501+0000 7ff0dbe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0dc06e590 0x7ff0dc0b5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:51.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:51.501+0000 7ff0dbe00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff0dc070380 0x7ff0dc1f4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:51.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:51.501+0000 7ff0dbe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0dc15bda0 0x7ff0dc158260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:51.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:51.501+0000 7ff0dbe00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff0dc0cc350 con 0x7ff0dc06e590 2024-09-16T15:50:51.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:51.501+0000 7ff0dbe00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff0dc0cc1d0 con 0x7ff0dc15bda0 2024-09-16T15:50:51.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:51.501+0000 7ff0dbe00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7ff0dc0740e0 con 0x7ff0dc070380 2024-09-16T15:50:51.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:51.501+0000 7ff0daa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0dc06e590 0x7ff0dc0b5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:51.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:51.501+0000 7ff0daa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0dc06e590 0x7ff0dc0b5490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:46036/0 (socket says 172.21.15.151:46036) 2024-09-16T15:50:51.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:51.501+0000 7ff0daa00640 1 -- 172.21.15.151:0/4120483073 learned_addr learned my addr 172.21.15.151:0/4120483073 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:51.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:51.501+0000 7ff0d3e00640 1 --2- 172.21.15.151:0/4120483073 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff0dc070380 0x7ff0dc1f4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:51.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:51.501+0000 7ff0db400640 1 --2- 172.21.15.151:0/4120483073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0dc15bda0 0x7ff0dc158260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:51.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:51.502+0000 7ff0daa00640 1 -- 172.21.15.151:0/4120483073 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff0dc070380 msgr2=0x7ff0dc1f4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:51.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:51.502+0000 7ff0daa00640 1 --2- 172.21.15.151:0/4120483073 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff0dc070380 0x7ff0dc1f4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:51.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:51.502+0000 7ff0daa00640 1 -- 172.21.15.151:0/4120483073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0dc15bda0 msgr2=0x7ff0dc158260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:51.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:51.502+0000 7ff0daa00640 1 --2- 172.21.15.151:0/4120483073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0dc15bda0 0x7ff0dc158260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:51.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:51.502+0000 7ff0daa00640 1 -- 172.21.15.151:0/4120483073 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0dc0b59d0 con 0x7ff0dc06e590 2024-09-16T15:50:51.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:51.502+0000 7ff0db400640 1 --2- 172.21.15.151:0/4120483073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0dc15bda0 0x7ff0dc158260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:50:51.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:51.502+0000 7ff0daa00640 1 --2- 172.21.15.151:0/4120483073 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0dc06e590 0x7ff0dc0b5490 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc00e9f0 tx=0x7ff0cc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:51.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:51.502+0000 7ff0d2a00640 1 -- 172.21.15.151:0/4120483073 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc00cd80 con 0x7ff0dc06e590 2024-09-16T15:50:51.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:51.502+0000 7ff0d2a00640 1 -- 172.21.15.151:0/4120483073 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff0cc004020 con 0x7ff0dc06e590 2024-09-16T15:50:51.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:51.502+0000 7ff0dbe00640 1 -- 172.21.15.151:0/4120483073 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0dc15a360 con 0x7ff0dc06e590 2024-09-16T15:50:51.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:51.502+0000 7ff0d2a00640 1 -- 172.21.15.151:0/4120483073 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc004470 con 0x7ff0dc06e590 2024-09-16T15:50:51.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:51.502+0000 7ff0dbe00640 1 -- 172.21.15.151:0/4120483073 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0dc15a920 con 0x7ff0dc06e590 2024-09-16T15:50:51.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:51.503+0000 7ff0dbe00640 1 -- 172.21.15.151:0/4120483073 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0dc074610 con 0x7ff0dc06e590 2024-09-16T15:50:51.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:51.504+0000 7ff0d2a00640 1 -- 172.21.15.151:0/4120483073 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7ff0cc0035f0 con 0x7ff0dc06e590 2024-09-16T15:50:51.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:51.505+0000 7ff0d2a00640 1 --2- 172.21.15.151:0/4120483073 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff0b8079f00 0x7ff0b807c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:51.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:51.506+0000 7ff0d2a00640 1 -- 172.21.15.151:0/4120483073 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5851+0+0 (secure 0 0 0) 0x7ff0cc09a040 con 0x7ff0dc06e590 2024-09-16T15:50:51.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:51.506+0000 7ff0d3e00640 1 --2- 172.21.15.151:0/4120483073 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff0b8079f00 0x7ff0b807c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:51.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:51.507+0000 7ff0d3e00640 1 --2- 172.21.15.151:0/4120483073 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff0b8079f00 0x7ff0b807c3c0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4034000 tx=0x7ff0c4004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:51.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:51.510+0000 7ff0d2a00640 1 -- 172.21.15.151:0/4120483073 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff0cc064090 con 0x7ff0dc06e590 2024-09-16T15:50:51.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:51.705+0000 7ff0dbe00640 1 -- 172.21.15.151:0/4120483073 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff0dc0b0170 con 0x7ff0dc06e590 2024-09-16T15:50:52.317 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:52.315+0000 7ff0d2a00640 1 -- 172.21.15.151:0/4120483073 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v95) ==== 181+0+0 (secure 0 0 0) 0x7ff0cc063990 con 0x7ff0dc06e590 2024-09-16T15:50:52.318 INFO:teuthology.orchestra.run.smithi151.stderr:pool 'cephfs_metadata' removed 2024-09-16T15:50:52.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:52.319+0000 7ff0dbe00640 1 -- 172.21.15.151:0/4120483073 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff0b8079f00 msgr2=0x7ff0b807c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:52.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:52.319+0000 7ff0dbe00640 1 --2- 172.21.15.151:0/4120483073 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff0b8079f00 0x7ff0b807c3c0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4034000 tx=0x7ff0c4004730 comp rx=0 tx=0).stop 2024-09-16T15:50:52.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:52.319+0000 7ff0dbe00640 1 -- 172.21.15.151:0/4120483073 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0dc06e590 msgr2=0x7ff0dc0b5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:52.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:52.319+0000 7ff0dbe00640 1 --2- 172.21.15.151:0/4120483073 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0dc06e590 0x7ff0dc0b5490 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc00e9f0 tx=0x7ff0cc00eeb0 comp rx=0 tx=0).stop 2024-09-16T15:50:52.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:52.320+0000 7ff0dbe00640 1 -- 172.21.15.151:0/4120483073 shutdown_connections 2024-09-16T15:50:52.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:52.320+0000 7ff0dbe00640 1 --2- 172.21.15.151:0/4120483073 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff0b8079f00 0x7ff0b807c3c0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:52.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:52.320+0000 7ff0dbe00640 1 --2- 172.21.15.151:0/4120483073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0dc15bda0 0x7ff0dc158260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:52.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:52.320+0000 7ff0dbe00640 1 --2- 172.21.15.151:0/4120483073 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff0dc070380 0x7ff0dc1f4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:52.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:52.320+0000 7ff0dbe00640 1 --2- 172.21.15.151:0/4120483073 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0dc06e590 0x7ff0dc0b5490 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:52.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:52.320+0000 7ff0dbe00640 1 -- 172.21.15.151:0/4120483073 >> 172.21.15.151:0/4120483073 conn(0x7ff0dc0cfa70 msgr2=0x7ff0dc0c90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:52.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:52.320+0000 7ff0dbe00640 1 -- 172.21.15.151:0/4120483073 shutdown_connections 2024-09-16T15:50:52.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:52.320+0000 7ff0dbe00640 1 -- 172.21.15.151:0/4120483073 wait complete. 2024-09-16T15:50:52.341 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-16T15:50:52.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:52.510+0000 7fdc7f200640 1 -- 172.21.15.151:0/4081734920 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdc8015f200 msgr2=0x7fdc800d4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:52.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:52.510+0000 7fdc7f200640 1 --2- 172.21.15.151:0/4081734920 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdc8015f200 0x7fdc800d4a20 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c005fd0 tx=0x7fdc6c02a5d0 comp rx=0 tx=0).stop 2024-09-16T15:50:52.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:52.510+0000 7fdc7f200640 1 -- 172.21.15.151:0/4081734920 shutdown_connections 2024-09-16T15:50:52.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:52.510+0000 7fdc7f200640 1 --2- 172.21.15.151:0/4081734920 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc800d08d0 0x7fdc800d0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:52.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:52.510+0000 7fdc7f200640 1 --2- 172.21.15.151:0/4081734920 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdc8015f200 0x7fdc800d4a20 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:52.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:52.510+0000 7fdc7f200640 1 -- 172.21.15.151:0/4081734920 >> 172.21.15.151:0/4081734920 conn(0x7fdc800d3f00 msgr2=0x7fdc800d4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:52.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:52.510+0000 7fdc7f200640 1 -- 172.21.15.151:0/4081734920 shutdown_connections 2024-09-16T15:50:52.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:52.510+0000 7fdc7f200640 1 -- 172.21.15.151:0/4081734920 wait complete. 2024-09-16T15:50:52.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:52.512+0000 7fdc7f200640 1 Processor -- start 2024-09-16T15:50:52.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:52.512+0000 7fdc7f200640 1 -- start start 2024-09-16T15:50:52.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:52.513+0000 7fdc7f200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc800d08d0 0x7fdc801f4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:52.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:52.513+0000 7fdc7f200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdc8015e710 0x7fdc800cafd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:52.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:52.513+0000 7fdc7f200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc8015f200 0x7fdc800cbe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:52.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:52.513+0000 7fdc7f200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdc800d1e60 con 0x7fdc8015e710 2024-09-16T15:50:52.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:52.513+0000 7fdc7f200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdc800d1ce0 con 0x7fdc8015f200 2024-09-16T15:50:52.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:52.513+0000 7fdc7f200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fdc800d1fe0 con 0x7fdc800d08d0 2024-09-16T15:50:52.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:52.513+0000 7fdc7d400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdc8015e710 0x7fdc800cafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:52.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:52.513+0000 7fdc7d400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdc8015e710 0x7fdc800cafd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:46060/0 (socket says 172.21.15.151:46060) 2024-09-16T15:50:52.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:52.513+0000 7fdc7d400640 1 -- 172.21.15.151:0/159992282 learned_addr learned my addr 172.21.15.151:0/159992282 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:52.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:52.513+0000 7fdc7e800640 1 --2- 172.21.15.151:0/159992282 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc8015f200 0x7fdc800cbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:52.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:52.513+0000 7fdc7de00640 1 --2- 172.21.15.151:0/159992282 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc800d08d0 0x7fdc801f4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:52.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:52.513+0000 7fdc7d400640 1 -- 172.21.15.151:0/159992282 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc800d08d0 msgr2=0x7fdc801f4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:52.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:52.513+0000 7fdc7d400640 1 --2- 172.21.15.151:0/159992282 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc800d08d0 0x7fdc801f4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:52.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:52.513+0000 7fdc7d400640 1 -- 172.21.15.151:0/159992282 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc8015f200 msgr2=0x7fdc800cbe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:52.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:52.513+0000 7fdc7d400640 1 --2- 172.21.15.151:0/159992282 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc8015f200 0x7fdc800cbe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:52.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:52.513+0000 7fdc7d400640 1 -- 172.21.15.151:0/159992282 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc80151950 con 0x7fdc8015e710 2024-09-16T15:50:52.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:52.514+0000 7fdc7d400640 1 --2- 172.21.15.151:0/159992282 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdc8015e710 0x7fdc800cafd0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c006010 tx=0x7fdc6c0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:52.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:52.514+0000 7fdc76000640 1 -- 172.21.15.151:0/159992282 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c03f070 con 0x7fdc8015e710 2024-09-16T15:50:52.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:52.514+0000 7fdc76000640 1 -- 172.21.15.151:0/159992282 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdc6c0047a0 con 0x7fdc8015e710 2024-09-16T15:50:52.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:52.514+0000 7fdc76000640 1 -- 172.21.15.151:0/159992282 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c035030 con 0x7fdc8015e710 2024-09-16T15:50:52.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:52.514+0000 7fdc7f200640 1 -- 172.21.15.151:0/159992282 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc80151c30 con 0x7fdc8015e710 2024-09-16T15:50:52.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:52.514+0000 7fdc7f200640 1 -- 172.21.15.151:0/159992282 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc801521f0 con 0x7fdc8015e710 2024-09-16T15:50:52.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:52.515+0000 7fdc7f200640 1 -- 172.21.15.151:0/159992282 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc80069a80 con 0x7fdc8015e710 2024-09-16T15:50:52.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:52.516+0000 7fdc76000640 1 -- 172.21.15.151:0/159992282 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7fdc6c042070 con 0x7fdc8015e710 2024-09-16T15:50:52.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:52.518+0000 7fdc76000640 1 --2- 172.21.15.151:0/159992282 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdc60079ed0 0x7fdc6007c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:52.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:52.518+0000 7fdc76000640 1 -- 172.21.15.151:0/159992282 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5413+0+0 (secure 0 0 0) 0x7fdc6c0ba310 con 0x7fdc8015e710 2024-09-16T15:50:52.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:52.518+0000 7fdc7de00640 1 --2- 172.21.15.151:0/159992282 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdc60079ed0 0x7fdc6007c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:52.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:52.519+0000 7fdc7de00640 1 --2- 172.21.15.151:0/159992282 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdc60079ed0 0x7fdc6007c390 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fdc80161250 tx=0x7fdc70009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:52.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:52.522+0000 7fdc76000640 1 -- 172.21.15.151:0/159992282 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fdc80069a80 con 0x7fdc8015e710 2024-09-16T15:50:52.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:52.713+0000 7fdc7f200640 1 -- 172.21.15.151:0/159992282 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fdc800d0d10 con 0x7fdc8015e710 2024-09-16T15:50:53.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.322+0000 7fdc76000640 1 -- 172.21.15.151:0/159992282 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v96) ==== 169+0+0 (secure 0 0 0) 0x7fdc6c083d90 con 0x7fdc8015e710 2024-09-16T15:50:53.325 INFO:teuthology.orchestra.run.smithi151.stderr:pool 'cephfs_data' removed 2024-09-16T15:50:53.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.327+0000 7fdc7f200640 1 -- 172.21.15.151:0/159992282 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdc60079ed0 msgr2=0x7fdc6007c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:53.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.327+0000 7fdc7f200640 1 --2- 172.21.15.151:0/159992282 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdc60079ed0 0x7fdc6007c390 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fdc80161250 tx=0x7fdc70009040 comp rx=0 tx=0).stop 2024-09-16T15:50:53.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.327+0000 7fdc7f200640 1 -- 172.21.15.151:0/159992282 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdc8015e710 msgr2=0x7fdc800cafd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:53.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.327+0000 7fdc7f200640 1 --2- 172.21.15.151:0/159992282 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdc8015e710 0x7fdc800cafd0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c006010 tx=0x7fdc6c0038a0 comp rx=0 tx=0).stop 2024-09-16T15:50:53.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.327+0000 7fdc7f200640 1 -- 172.21.15.151:0/159992282 shutdown_connections 2024-09-16T15:50:53.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.327+0000 7fdc7f200640 1 --2- 172.21.15.151:0/159992282 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdc60079ed0 0x7fdc6007c390 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:53.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.327+0000 7fdc7f200640 1 --2- 172.21.15.151:0/159992282 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc8015f200 0x7fdc800cbe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:53.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.327+0000 7fdc7f200640 1 --2- 172.21.15.151:0/159992282 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdc8015e710 0x7fdc800cafd0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:53.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.327+0000 7fdc7f200640 1 --2- 172.21.15.151:0/159992282 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdc800d08d0 0x7fdc801f4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:53.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.328+0000 7fdc7f200640 1 -- 172.21.15.151:0/159992282 >> 172.21.15.151:0/159992282 conn(0x7fdc800d3f00 msgr2=0x7fdc8015c490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:53.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.328+0000 7fdc7f200640 1 -- 172.21.15.151:0/159992282 shutdown_connections 2024-09-16T15:50:53.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.328+0000 7fdc7f200640 1 -- 172.21.15.151:0/159992282 wait complete. 2024-09-16T15:50:53.360 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-16T15:50:53.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.520+0000 7fb658000640 1 -- 172.21.15.151:0/2877806588 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb65015f5a0 msgr2=0x7fb6500b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:53.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.520+0000 7fb658000640 1 --2- 172.21.15.151:0/2877806588 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb65015f5a0 0x7fb6500b2440 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fb644005fd0 tx=0x7fb64402a540 comp rx=0 tx=0).stop 2024-09-16T15:50:53.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.520+0000 7fb658000640 1 -- 172.21.15.151:0/2877806588 shutdown_connections 2024-09-16T15:50:53.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.520+0000 7fb658000640 1 --2- 172.21.15.151:0/2877806588 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb65015f5a0 0x7fb6500b2440 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:53.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.520+0000 7fb658000640 1 -- 172.21.15.151:0/2877806588 >> 172.21.15.151:0/2877806588 conn(0x7fb6500b1950 msgr2=0x7fb6500b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:53.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.521+0000 7fb658000640 1 -- 172.21.15.151:0/2877806588 shutdown_connections 2024-09-16T15:50:53.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.521+0000 7fb658000640 1 -- 172.21.15.151:0/2877806588 wait complete. 2024-09-16T15:50:53.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.522+0000 7fb658000640 1 Processor -- start 2024-09-16T15:50:53.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.522+0000 7fb658000640 1 -- start start 2024-09-16T15:50:53.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.523+0000 7fb658000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb650073fb0 0x7fb650145fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:53.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.523+0000 7fb658000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6500ccf00 0x7fb6501468c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:53.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.523+0000 7fb658000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb65015f5a0 0x7fb6500b3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:53.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.523+0000 7fb658000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb65006f9f0 con 0x7fb65015f5a0 2024-09-16T15:50:53.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.523+0000 7fb658000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb65006f870 con 0x7fb6500ccf00 2024-09-16T15:50:53.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.523+0000 7fb658000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fb650074420 con 0x7fb650073fb0 2024-09-16T15:50:53.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.524+0000 7fb656000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb65015f5a0 0x7fb6500b3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:53.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.524+0000 7fb656000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb65015f5a0 0x7fb6500b3c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:54170/0 (socket says 172.21.15.151:54170) 2024-09-16T15:50:53.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.524+0000 7fb656000640 1 -- 172.21.15.151:0/205915046 learned_addr learned my addr 172.21.15.151:0/205915046 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:53.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.524+0000 7fb655600640 1 --2- 172.21.15.151:0/205915046 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb650073fb0 0x7fb650145fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:53.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.524+0000 7fb654c00640 1 --2- 172.21.15.151:0/205915046 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6500ccf00 0x7fb6501468c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:53.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.524+0000 7fb656000640 1 -- 172.21.15.151:0/205915046 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb650073fb0 msgr2=0x7fb650145fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:53.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.524+0000 7fb656000640 1 --2- 172.21.15.151:0/205915046 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb650073fb0 0x7fb650145fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:53.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.524+0000 7fb656000640 1 -- 172.21.15.151:0/205915046 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6500ccf00 msgr2=0x7fb6501468c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:53.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.524+0000 7fb656000640 1 --2- 172.21.15.151:0/205915046 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6500ccf00 0x7fb6501468c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:53.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.524+0000 7fb656000640 1 -- 172.21.15.151:0/205915046 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6501f57d0 con 0x7fb65015f5a0 2024-09-16T15:50:53.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.524+0000 7fb656000640 1 --2- 172.21.15.151:0/205915046 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb65015f5a0 0x7fb6500b3c20 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fb64800c780 tx=0x7fb64800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:53.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.525+0000 7fb64d600640 1 -- 172.21.15.151:0/205915046 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb648017070 con 0x7fb65015f5a0 2024-09-16T15:50:53.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.525+0000 7fb64d600640 1 -- 172.21.15.151:0/205915046 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb648005070 con 0x7fb65015f5a0 2024-09-16T15:50:53.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.525+0000 7fb64d600640 1 -- 172.21.15.151:0/205915046 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb648005500 con 0x7fb65015f5a0 2024-09-16T15:50:53.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.525+0000 7fb658000640 1 -- 172.21.15.151:0/205915046 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6500b2e60 con 0x7fb65015f5a0 2024-09-16T15:50:53.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.525+0000 7fb658000640 1 -- 172.21.15.151:0/205915046 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6500b3420 con 0x7fb65015f5a0 2024-09-16T15:50:53.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.526+0000 7fb658000640 1 -- 172.21.15.151:0/205915046 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6500b2440 con 0x7fb65015f5a0 2024-09-16T15:50:53.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.527+0000 7fb64d600640 1 -- 172.21.15.151:0/205915046 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7fb64801a070 con 0x7fb65015f5a0 2024-09-16T15:50:53.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.528+0000 7fb64d600640 1 --2- 172.21.15.151:0/205915046 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb630079ed0 0x7fb63007c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:53.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.528+0000 7fb64d600640 1 -- 172.21.15.151:0/205915046 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 4999+0+0 (secure 0 0 0) 0x7fb648098c60 con 0x7fb65015f5a0 2024-09-16T15:50:53.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.528+0000 7fb655600640 1 --2- 172.21.15.151:0/205915046 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb630079ed0 0x7fb63007c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:53.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.529+0000 7fb655600640 1 --2- 172.21.15.151:0/205915046 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb630079ed0 0x7fb63007c390 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fb644033000 tx=0x7fb644004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:53.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.532+0000 7fb64d600640 1 -- 172.21.15.151:0/205915046 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb6480626e0 con 0x7fb65015f5a0 2024-09-16T15:50:53.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.726+0000 7fb658000640 1 -- 172.21.15.151:0/205915046 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7fb6500aced0 con 0x7fb65015f5a0 2024-09-16T15:50:53.732 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.730+0000 7fb64d600640 1 -- 172.21.15.151:0/205915046 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v44) ==== 71+0+0 (secure 0 0 0) 0x7fb648067110 con 0x7fb65015f5a0 2024-09-16T15:50:53.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.733+0000 7fb658000640 1 -- 172.21.15.151:0/205915046 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb630079ed0 msgr2=0x7fb63007c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:53.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.733+0000 7fb658000640 1 --2- 172.21.15.151:0/205915046 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb630079ed0 0x7fb63007c390 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fb644033000 tx=0x7fb644004730 comp rx=0 tx=0).stop 2024-09-16T15:50:53.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.733+0000 7fb658000640 1 -- 172.21.15.151:0/205915046 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb65015f5a0 msgr2=0x7fb6500b3c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:53.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.733+0000 7fb658000640 1 --2- 172.21.15.151:0/205915046 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb65015f5a0 0x7fb6500b3c20 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fb64800c780 tx=0x7fb64800cc40 comp rx=0 tx=0).stop 2024-09-16T15:50:53.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.733+0000 7fb658000640 1 -- 172.21.15.151:0/205915046 shutdown_connections 2024-09-16T15:50:53.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.733+0000 7fb658000640 1 --2- 172.21.15.151:0/205915046 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb630079ed0 0x7fb63007c390 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:53.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.733+0000 7fb658000640 1 --2- 172.21.15.151:0/205915046 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6500ccf00 0x7fb6501468c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:53.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.733+0000 7fb658000640 1 --2- 172.21.15.151:0/205915046 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb65015f5a0 0x7fb6500b3c20 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:53.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.733+0000 7fb658000640 1 --2- 172.21.15.151:0/205915046 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb650073fb0 0x7fb650145fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:53.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.733+0000 7fb658000640 1 -- 172.21.15.151:0/205915046 >> 172.21.15.151:0/205915046 conn(0x7fb6500b1950 msgr2=0x7fb6500ad710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:53.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.733+0000 7fb658000640 1 -- 172.21.15.151:0/205915046 shutdown_connections 2024-09-16T15:50:53.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.733+0000 7fb658000640 1 -- 172.21.15.151:0/205915046 wait complete. 2024-09-16T15:50:53.750 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_drops_when_new_jobs_are_added' 2024-09-16T15:50:53.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.922+0000 7fa0f4600640 1 -- 172.21.15.151:0/796300670 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0ec161550 msgr2=0x7fa0ec06ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:53.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.922+0000 7fa0f4600640 1 --2- 172.21.15.151:0/796300670 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0ec161550 0x7fa0ec06ef50 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0005fd0 tx=0x7fa0e002aa10 comp rx=0 tx=0).stop 2024-09-16T15:50:53.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.922+0000 7fa0f4600640 1 -- 172.21.15.151:0/796300670 shutdown_connections 2024-09-16T15:50:53.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.922+0000 7fa0f4600640 1 --2- 172.21.15.151:0/796300670 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0ec161550 0x7fa0ec06ef50 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:53.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.922+0000 7fa0f4600640 1 -- 172.21.15.151:0/796300670 >> 172.21.15.151:0/796300670 conn(0x7fa0ec0b1950 msgr2=0x7fa0ec0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:53.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.922+0000 7fa0f4600640 1 -- 172.21.15.151:0/796300670 shutdown_connections 2024-09-16T15:50:53.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.922+0000 7fa0f4600640 1 -- 172.21.15.151:0/796300670 wait complete. 2024-09-16T15:50:53.926 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.924+0000 7fa0f4600640 1 Processor -- start 2024-09-16T15:50:53.926 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.924+0000 7fa0f4600640 1 -- start start 2024-09-16T15:50:53.928 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.925+0000 7fa0f4600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0ec072df0 0x7fa0ec146d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:53.928 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.925+0000 7fa0f4600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0ec0af3d0 0x7fa0ec15fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:53.928 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.925+0000 7fa0f4600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0ec161550 0x7fa0ec160340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:53.928 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.925+0000 7fa0f4600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ec0739b0 con 0x7fa0ec072df0 2024-09-16T15:50:53.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.925+0000 7fa0f4600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ec06fdb0 con 0x7fa0ec161550 2024-09-16T15:50:53.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.925+0000 7fa0f4600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fa0ec073b30 con 0x7fa0ec0af3d0 2024-09-16T15:50:53.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.925+0000 7fa0f1c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0ec072df0 0x7fa0ec146d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:53.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.925+0000 7fa0f1c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0ec072df0 0x7fa0ec146d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:54194/0 (socket says 172.21.15.151:54194) 2024-09-16T15:50:53.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.925+0000 7fa0f1c00640 1 -- 172.21.15.151:0/4099022283 learned_addr learned my addr 172.21.15.151:0/4099022283 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:53.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.926+0000 7fa0f1c00640 1 -- 172.21.15.151:0/4099022283 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0ec0af3d0 msgr2=0x7fa0ec15fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:53.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.926+0000 7fa0f2600640 1 --2- 172.21.15.151:0/4099022283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0ec161550 0x7fa0ec160340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:53.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.926+0000 7fa0f1200640 1 --2- 172.21.15.151:0/4099022283 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0ec0af3d0 0x7fa0ec15fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:53.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.926+0000 7fa0f1c00640 1 --2- 172.21.15.151:0/4099022283 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0ec0af3d0 0x7fa0ec15fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:53.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.926+0000 7fa0f1c00640 1 -- 172.21.15.151:0/4099022283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0ec161550 msgr2=0x7fa0ec160340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:53.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.926+0000 7fa0f1c00640 1 --2- 172.21.15.151:0/4099022283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0ec161550 0x7fa0ec160340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:53.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.926+0000 7fa0f1c00640 1 -- 172.21.15.151:0/4099022283 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0ec14a600 con 0x7fa0ec072df0 2024-09-16T15:50:53.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.926+0000 7fa0f2600640 1 --2- 172.21.15.151:0/4099022283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0ec161550 0x7fa0ec160340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:50:53.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.926+0000 7fa0f1200640 1 --2- 172.21.15.151:0/4099022283 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0ec0af3d0 0x7fa0ec15fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:50:53.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.926+0000 7fa0f1c00640 1 --2- 172.21.15.151:0/4099022283 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0ec072df0 0x7fa0ec146d80 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0000f80 tx=0x7fa0e0003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:53.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.926+0000 7fa0e9600640 1 -- 172.21.15.151:0/4099022283 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e0046070 con 0x7fa0ec072df0 2024-09-16T15:50:53.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.926+0000 7fa0e9600640 1 -- 172.21.15.151:0/4099022283 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa0e00046c0 con 0x7fa0ec072df0 2024-09-16T15:50:53.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.926+0000 7fa0e9600640 1 -- 172.21.15.151:0/4099022283 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e0033070 con 0x7fa0ec072df0 2024-09-16T15:50:53.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.926+0000 7fa0f4600640 1 -- 172.21.15.151:0/4099022283 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0ec1f8090 con 0x7fa0ec072df0 2024-09-16T15:50:53.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.926+0000 7fa0f4600640 1 -- 172.21.15.151:0/4099022283 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0ec1f8650 con 0x7fa0ec072df0 2024-09-16T15:50:53.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.927+0000 7fa0f4600640 1 -- 172.21.15.151:0/4099022283 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0ec074040 con 0x7fa0ec072df0 2024-09-16T15:50:53.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.928+0000 7fa0e9600640 1 -- 172.21.15.151:0/4099022283 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7fa0e0036070 con 0x7fa0ec072df0 2024-09-16T15:50:53.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.929+0000 7fa0e9600640 1 --2- 172.21.15.151:0/4099022283 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa0cc079ed0 0x7fa0cc07c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:53.931 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.930+0000 7fa0e9600640 1 -- 172.21.15.151:0/4099022283 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 4999+0+0 (secure 0 0 0) 0x7fa0e00ba580 con 0x7fa0ec072df0 2024-09-16T15:50:53.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.930+0000 7fa0f1200640 1 --2- 172.21.15.151:0/4099022283 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa0cc079ed0 0x7fa0cc07c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:53.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.931+0000 7fa0f1200640 1 --2- 172.21.15.151:0/4099022283 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa0cc079ed0 0x7fa0cc07c390 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec160f60 tx=0x7fa0d8009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:53.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:53.934+0000 7fa0e9600640 1 -- 172.21.15.151:0/4099022283 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa0e003e070 con 0x7fa0ec072df0 2024-09-16T15:50:54.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.187+0000 7fa0f4600640 1 -- 172.21.15.151:0/4099022283 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_drops_when_new_jobs_are_added"]} v 0) -- 0x7fa0ec0b1000 con 0x7fa0ec072df0 2024-09-16T15:50:54.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.324+0000 7fa0e9600640 1 -- 172.21.15.151:0/4099022283 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_drops_when_new_jobs_are_added"]}]=0 v526) ==== 174+0+0 (secure 0 0 0) 0x7fa0e0084650 con 0x7fa0ec072df0 2024-09-16T15:50:54.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.326+0000 7fa0f4600640 1 -- 172.21.15.151:0/4099022283 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa0cc079ed0 msgr2=0x7fa0cc07c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:54.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.326+0000 7fa0f4600640 1 --2- 172.21.15.151:0/4099022283 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa0cc079ed0 0x7fa0cc07c390 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec160f60 tx=0x7fa0d8009040 comp rx=0 tx=0).stop 2024-09-16T15:50:54.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.326+0000 7fa0f4600640 1 -- 172.21.15.151:0/4099022283 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0ec072df0 msgr2=0x7fa0ec146d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:54.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.326+0000 7fa0f4600640 1 --2- 172.21.15.151:0/4099022283 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0ec072df0 0x7fa0ec146d80 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0000f80 tx=0x7fa0e0003f90 comp rx=0 tx=0).stop 2024-09-16T15:50:54.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.326+0000 7fa0f4600640 1 -- 172.21.15.151:0/4099022283 shutdown_connections 2024-09-16T15:50:54.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.326+0000 7fa0f4600640 1 --2- 172.21.15.151:0/4099022283 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa0cc079ed0 0x7fa0cc07c390 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:54.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.326+0000 7fa0f4600640 1 --2- 172.21.15.151:0/4099022283 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0ec161550 0x7fa0ec160340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:54.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.326+0000 7fa0f4600640 1 --2- 172.21.15.151:0/4099022283 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa0ec0af3d0 0x7fa0ec15fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:54.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.326+0000 7fa0f4600640 1 --2- 172.21.15.151:0/4099022283 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0ec072df0 0x7fa0ec146d80 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:54.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.326+0000 7fa0f4600640 1 -- 172.21.15.151:0/4099022283 >> 172.21.15.151:0/4099022283 conn(0x7fa0ec0b1950 msgr2=0x7fa0ec0b3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:54.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.326+0000 7fa0f4600640 1 -- 172.21.15.151:0/4099022283 shutdown_connections 2024-09-16T15:50:54.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.326+0000 7fa0f4600640 1 -- 172.21.15.151:0/4099022283 wait complete. 2024-09-16T15:50:54.345 INFO:tasks.cephfs_test_runner:Test that progress indicated by progress bar for ongoing+pending clones ... ok 2024-09-16T15:50:54.346 INFO:tasks.cephfs_test_runner:Starting test: test_progress_is_printed_in_clone_status_output (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that the command "ceph fs clone status" prints progress stats 2024-09-16T15:50:54.346 INFO:tasks.cephfs_test_runner:test_progress_is_printed_in_clone_status_output (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-09-16T15:50:54.346 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:50:54.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.505+0000 7f0a34400640 1 -- 172.21.15.151:0/2483978438 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a2c0ccf00 msgr2=0x7f0a2c0b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:54.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.505+0000 7f0a34400640 1 --2- 172.21.15.151:0/2483978438 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a2c0ccf00 0x7f0a2c0b2440 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f0a18005fd0 tx=0x7f0a1802a540 comp rx=0 tx=0).stop 2024-09-16T15:50:54.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.505+0000 7f0a34400640 1 -- 172.21.15.151:0/2483978438 shutdown_connections 2024-09-16T15:50:54.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.505+0000 7f0a34400640 1 --2- 172.21.15.151:0/2483978438 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a2c073fb0 0x7f0a2c074440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:54.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.505+0000 7f0a34400640 1 --2- 172.21.15.151:0/2483978438 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a2c0ccf00 0x7f0a2c0b2440 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:54.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.505+0000 7f0a34400640 1 -- 172.21.15.151:0/2483978438 >> 172.21.15.151:0/2483978438 conn(0x7f0a2c0b1950 msgr2=0x7f0a2c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:54.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.505+0000 7f0a34400640 1 -- 172.21.15.151:0/2483978438 shutdown_connections 2024-09-16T15:50:54.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.505+0000 7f0a34400640 1 -- 172.21.15.151:0/2483978438 wait complete. 2024-09-16T15:50:54.509 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.507+0000 7f0a34400640 1 Processor -- start 2024-09-16T15:50:54.509 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.507+0000 7f0a34400640 1 -- start start 2024-09-16T15:50:54.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.508+0000 7f0a34400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a2c073fb0 0x7f0a2c0d06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:54.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.508+0000 7f0a34400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a2c0ccf00 0x7f0a2c0cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:54.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.508+0000 7f0a34400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a2c15f5a0 0x7f0a2c1f81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:54.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.508+0000 7f0a34400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0a2c06f2e0 con 0x7f0a2c0ccf00 2024-09-16T15:50:54.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.508+0000 7f0a34400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0a2c06f160 con 0x7f0a2c073fb0 2024-09-16T15:50:54.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.508+0000 7f0a34400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f0a2c06f460 con 0x7f0a2c15f5a0 2024-09-16T15:50:54.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.508+0000 7f0a31000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a2c0ccf00 0x7f0a2c0cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:54.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.508+0000 7f0a31000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a2c0ccf00 0x7f0a2c0cdae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:54212/0 (socket says 172.21.15.151:54212) 2024-09-16T15:50:54.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.508+0000 7f0a31000640 1 -- 172.21.15.151:0/1463012135 learned_addr learned my addr 172.21.15.151:0/1463012135 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:54.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.508+0000 7f0a32400640 1 --2- 172.21.15.151:0/1463012135 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a2c15f5a0 0x7f0a2c1f81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:54.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.508+0000 7f0a31a00640 1 --2- 172.21.15.151:0/1463012135 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a2c073fb0 0x7f0a2c0d06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:54.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.509+0000 7f0a31000640 1 -- 172.21.15.151:0/1463012135 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a2c15f5a0 msgr2=0x7f0a2c1f81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:54.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.509+0000 7f0a31000640 1 --2- 172.21.15.151:0/1463012135 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a2c15f5a0 0x7f0a2c1f81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:54.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.509+0000 7f0a31000640 1 -- 172.21.15.151:0/1463012135 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a2c073fb0 msgr2=0x7f0a2c0d06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:54.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.509+0000 7f0a31000640 1 --2- 172.21.15.151:0/1463012135 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a2c073fb0 0x7f0a2c0d06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:54.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.509+0000 7f0a31000640 1 -- 172.21.15.151:0/1463012135 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a2c0cf6a0 con 0x7f0a2c0ccf00 2024-09-16T15:50:54.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.510+0000 7f0a31000640 1 --2- 172.21.15.151:0/1463012135 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a2c0ccf00 0x7f0a2c0cdae0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f0a18000c00 tx=0x7f0a18003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:54.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.510+0000 7f0a29600640 1 -- 172.21.15.151:0/1463012135 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a1803f070 con 0x7f0a2c0ccf00 2024-09-16T15:50:54.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.510+0000 7f0a29600640 1 -- 172.21.15.151:0/1463012135 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0a180047a0 con 0x7f0a2c0ccf00 2024-09-16T15:50:54.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.510+0000 7f0a34400640 1 -- 172.21.15.151:0/1463012135 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a2c0cf940 con 0x7f0a2c0ccf00 2024-09-16T15:50:54.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.510+0000 7f0a29600640 1 -- 172.21.15.151:0/1463012135 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a18035030 con 0x7f0a2c0ccf00 2024-09-16T15:50:54.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.510+0000 7f0a34400640 1 -- 172.21.15.151:0/1463012135 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a2c0cff00 con 0x7f0a2c0ccf00 2024-09-16T15:50:54.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.511+0000 7f0a34400640 1 -- 172.21.15.151:0/1463012135 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a2c0d0340 con 0x7f0a2c0ccf00 2024-09-16T15:50:54.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.512+0000 7f0a29600640 1 -- 172.21.15.151:0/1463012135 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7f0a18042070 con 0x7f0a2c0ccf00 2024-09-16T15:50:54.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.513+0000 7f0a29600640 1 --2- 172.21.15.151:0/1463012135 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0a0c079ed0 0x7f0a0c07c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:54.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.513+0000 7f0a29600640 1 -- 172.21.15.151:0/1463012135 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 4999+0+0 (secure 0 0 0) 0x7f0a180ba8a0 con 0x7f0a2c0ccf00 2024-09-16T15:50:54.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.513+0000 7f0a31a00640 1 --2- 172.21.15.151:0/1463012135 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0a0c079ed0 0x7f0a0c07c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:54.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.514+0000 7f0a31a00640 1 --2- 172.21.15.151:0/1463012135 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0a0c079ed0 0x7f0a0c07c390 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f0a20001720 tx=0x7f0a20009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:54.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.517+0000 7f0a29600640 1 -- 172.21.15.151:0/1463012135 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0a180842a0 con 0x7f0a2c0ccf00 2024-09-16T15:50:54.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.703+0000 7f0a34400640 1 -- 172.21.15.151:0/1463012135 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0a2c06f980 con 0x7f0a2c0ccf00 2024-09-16T15:50:54.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.703+0000 7f0a29600640 1 -- 172.21.15.151:0/1463012135 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+13182 (secure 0 0 0) 0x7f0a18088cd0 con 0x7f0a2c0ccf00 2024-09-16T15:50:54.706 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:50:54.706 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":96,"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","created":"2024-09-16T15:38:56.348535+0000","modified":"2024-09-16T15:50:53.317846+0000","last_up_change":"2024-09-16T15:39:10.681384+0000","last_in_change":"2024-09-16T15:39:00.544477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:39:11.602252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5524076-a903-4d0a-af6a-caf474c4765d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6809","nonce":3111342741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6811","nonce":3111342741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6815","nonce":3111342741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6813","nonce":3111342741}]},"public_addr":"172.21.15.151:6809/3111342741","cluster_addr":"172.21.15.151:6811/3111342741","heartbeat_back_addr":"172.21.15.151:6815/3111342741","heartbeat_front_addr":"172.21.15.151:6813/3111342741","state":["exists","up"]},{"osd":1,"uuid":"dd25925f-0ae1-494e-8529-4dc205759db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6801","nonce":3368304087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6803","nonce":3368304087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6807","nonce":3368304087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6805","nonce":3368304087}]},"public_addr":"172.21.15.151:6801/3368304087","cluster_addr":"172.21.15.151:6803/3368304087","heartbeat_back_addr":"172.21.15.151:6807/3368304087","heartbeat_front_addr":"172.21.15.151:6805/3368304087","state":["exists","up"]},{"osd":2,"uuid":"c2f18115-013d-4da6-a0fe-cb978f4494b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6823","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6825","nonce":3079373271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6827","nonce":3079373271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6831","nonce":3079373271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6829","nonce":3079373271}]},"public_addr":"172.21.15.151:6825/3079373271","cluster_addr":"172.21.15.151:6827/3079373271","heartbeat_back_addr":"172.21.15.151:6831/3079373271","heartbeat_front_addr":"172.21.15.151:6829/3079373271","state":["exists","up"]},{"osd":3,"uuid":"36edb718-2fcb-4d88-8ddc-c321eed4279d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6817","nonce":1142509041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6819","nonce":1142509041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6824","nonce":1142509041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6821","nonce":1142509041}]},"public_addr":"172.21.15.151:6817/1142509041","cluster_addr":"172.21.15.151:6819/1142509041","heartbeat_back_addr":"172.21.15.151:6824/1142509041","heartbeat_front_addr":"172.21.15.151:6821/1142509041","state":["exists","up"]},{"osd":4,"uuid":"64d33708-cd0a-4236-abe3-1b646e82ab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6811","nonce":2735495271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6813","nonce":2735495271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6817","nonce":2735495271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6815","nonce":2735495271}]},"public_addr":"172.21.15.167:6811/2735495271","cluster_addr":"172.21.15.167:6813/2735495271","heartbeat_back_addr":"172.21.15.167:6817/2735495271","heartbeat_front_addr":"172.21.15.167:6815/2735495271","state":["exists","up"]},{"osd":5,"uuid":"f14b44a9-5910-458c-b518-09a8d0de94de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6827","nonce":1494916572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6829","nonce":1494916572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6833","nonce":1494916572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6831","nonce":1494916572}]},"public_addr":"172.21.15.167:6827/1494916572","cluster_addr":"172.21.15.167:6829/1494916572","heartbeat_back_addr":"172.21.15.167:6833/1494916572","heartbeat_front_addr":"172.21.15.167:6831/1494916572","state":["exists","up"]},{"osd":6,"uuid":"39335545-2654-4710-84b0-a9aa47cfcbba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6819","nonce":2538273273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6821","nonce":2538273273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6825","nonce":2538273273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6823","nonce":2538273273}]},"public_addr":"172.21.15.167:6819/2538273273","cluster_addr":"172.21.15.167:6821/2538273273","heartbeat_back_addr":"172.21.15.167:6825/2538273273","heartbeat_front_addr":"172.21.15.167:6823/2538273273","state":["exists","up"]},{"osd":7,"uuid":"4f8bfc81-53dd-4339-ac22-1a4007d9f963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6803","nonce":265072227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6805","nonce":265072227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6809","nonce":265072227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6807","nonce":265072227}]},"public_addr":"172.21.15.167:6803/265072227","cluster_addr":"172.21.15.167:6805/265072227","heartbeat_back_addr":"172.21.15.167:6809/265072227","heartbeat_front_addr":"172.21.15.167:6807/265072227","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.150359+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.084358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.722368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.332809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.320709+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.395153+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.935250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.801443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.151:6835/1518702790":"2024-09-17T15:50:50.310724+0000","172.21.15.151:6833/1518702790":"2024-09-17T15:50:50.310724+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:50:54.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.706+0000 7f0a34400640 1 -- 172.21.15.151:0/1463012135 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0a0c079ed0 msgr2=0x7f0a0c07c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:54.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.706+0000 7f0a34400640 1 --2- 172.21.15.151:0/1463012135 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0a0c079ed0 0x7f0a0c07c390 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f0a20001720 tx=0x7f0a20009040 comp rx=0 tx=0).stop 2024-09-16T15:50:54.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.706+0000 7f0a34400640 1 -- 172.21.15.151:0/1463012135 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a2c0ccf00 msgr2=0x7f0a2c0cdae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:54.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.706+0000 7f0a34400640 1 --2- 172.21.15.151:0/1463012135 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a2c0ccf00 0x7f0a2c0cdae0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f0a18000c00 tx=0x7f0a18003910 comp rx=0 tx=0).stop 2024-09-16T15:50:54.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.706+0000 7f0a34400640 1 -- 172.21.15.151:0/1463012135 shutdown_connections 2024-09-16T15:50:54.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.706+0000 7f0a34400640 1 --2- 172.21.15.151:0/1463012135 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0a0c079ed0 0x7f0a0c07c390 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:54.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.706+0000 7f0a34400640 1 --2- 172.21.15.151:0/1463012135 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a2c15f5a0 0x7f0a2c1f81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:54.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.706+0000 7f0a34400640 1 --2- 172.21.15.151:0/1463012135 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a2c0ccf00 0x7f0a2c0cdae0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:54.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.706+0000 7f0a34400640 1 --2- 172.21.15.151:0/1463012135 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a2c073fb0 0x7f0a2c0d06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:54.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.706+0000 7f0a34400640 1 -- 172.21.15.151:0/1463012135 >> 172.21.15.151:0/1463012135 conn(0x7f0a2c0b1950 msgr2=0x7f0a2c15cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:54.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.706+0000 7f0a34400640 1 -- 172.21.15.151:0/1463012135 shutdown_connections 2024-09-16T15:50:54.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.706+0000 7f0a34400640 1 -- 172.21.15.151:0/1463012135 wait complete. 2024-09-16T15:50:54.724 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T15:39:11.602252+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-16T15:50:54.725 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-16T15:50:54.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.885+0000 7f479f400640 1 -- 172.21.15.151:0/2340645105 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f47980b3370 msgr2=0x7f47980cebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:54.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.885+0000 7f479f400640 1 --2- 172.21.15.151:0/2340645105 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f47980b3370 0x7f47980cebe0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f4788005fd0 tx=0x7f478802aaa0 comp rx=0 tx=0).stop 2024-09-16T15:50:54.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.885+0000 7f479f400640 1 -- 172.21.15.151:0/2340645105 shutdown_connections 2024-09-16T15:50:54.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.885+0000 7f479f400640 1 --2- 172.21.15.151:0/2340645105 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f479815c080 0x7f47980cf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:54.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.885+0000 7f479f400640 1 --2- 172.21.15.151:0/2340645105 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f47980b3370 0x7f47980cebe0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:54.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.885+0000 7f479f400640 1 -- 172.21.15.151:0/2340645105 >> 172.21.15.151:0/2340645105 conn(0x7f47980b1840 msgr2=0x7f47980b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:54.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.885+0000 7f479f400640 1 -- 172.21.15.151:0/2340645105 shutdown_connections 2024-09-16T15:50:54.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.885+0000 7f479f400640 1 -- 172.21.15.151:0/2340645105 wait complete. 2024-09-16T15:50:54.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.886+0000 7f479f400640 1 Processor -- start 2024-09-16T15:50:54.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.886+0000 7f479f400640 1 -- start start 2024-09-16T15:50:54.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.887+0000 7f479f400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47980b3370 0x7f47981381a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:54.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.887+0000 7f479f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f479815c080 0x7f47980738f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:54.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.887+0000 7f479f400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f479815e980 0x7f4798073e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:54.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.887+0000 7f479f400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4798164e20 con 0x7f479815c080 2024-09-16T15:50:54.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.887+0000 7f479f400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4798164ca0 con 0x7f47980b3370 2024-09-16T15:50:54.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.887+0000 7f479f400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f4798164fa0 con 0x7f479815e980 2024-09-16T15:50:54.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.888+0000 7f4797e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f479815c080 0x7f47980738f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:54.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.888+0000 7f4797e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f479815c080 0x7f47980738f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:54240/0 (socket says 172.21.15.151:54240) 2024-09-16T15:50:54.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.888+0000 7f4797e00640 1 -- 172.21.15.151:0/776655068 learned_addr learned my addr 172.21.15.151:0/776655068 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:54.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.888+0000 7f479ca00640 1 --2- 172.21.15.151:0/776655068 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47980b3370 0x7f47981381a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:54.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.888+0000 7f4797e00640 1 -- 172.21.15.151:0/776655068 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f479815e980 msgr2=0x7f4798073e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:54.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.888+0000 7f479d400640 1 --2- 172.21.15.151:0/776655068 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f479815e980 0x7f4798073e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:54.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.888+0000 7f4797e00640 1 --2- 172.21.15.151:0/776655068 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f479815e980 0x7f4798073e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:54.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.888+0000 7f4797e00640 1 -- 172.21.15.151:0/776655068 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47980b3370 msgr2=0x7f47981381a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:54.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.888+0000 7f4797e00640 1 --2- 172.21.15.151:0/776655068 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47980b3370 0x7f47981381a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:54.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.888+0000 7f4797e00640 1 -- 172.21.15.151:0/776655068 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47981386e0 con 0x7f479815c080 2024-09-16T15:50:54.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.888+0000 7f479d400640 1 --2- 172.21.15.151:0/776655068 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f479815e980 0x7f4798073e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:50:54.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.888+0000 7f479ca00640 1 --2- 172.21.15.151:0/776655068 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47980b3370 0x7f47981381a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:50:54.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.888+0000 7f4797e00640 1 --2- 172.21.15.151:0/776655068 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f479815c080 0x7f47980738f0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f478400b4d0 tx=0x7f478400b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:54.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.888+0000 7f4794c00640 1 -- 172.21.15.151:0/776655068 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4784014070 con 0x7f479815c080 2024-09-16T15:50:54.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.888+0000 7f4794c00640 1 -- 172.21.15.151:0/776655068 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4784010960 con 0x7f479815c080 2024-09-16T15:50:54.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.888+0000 7f479f400640 1 -- 172.21.15.151:0/776655068 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4798074460 con 0x7f479815c080 2024-09-16T15:50:54.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.889+0000 7f4794c00640 1 -- 172.21.15.151:0/776655068 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4784010db0 con 0x7f479815c080 2024-09-16T15:50:54.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.889+0000 7f479f400640 1 -- 172.21.15.151:0/776655068 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47981f8750 con 0x7f479815c080 2024-09-16T15:50:54.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.890+0000 7f479f400640 1 -- 172.21.15.151:0/776655068 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f479815ee10 con 0x7f479815c080 2024-09-16T15:50:54.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.891+0000 7f4794c00640 1 -- 172.21.15.151:0/776655068 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7f4784017070 con 0x7f479815c080 2024-09-16T15:50:54.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.892+0000 7f4794c00640 1 --2- 172.21.15.151:0/776655068 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4778079ed0 0x7f477807c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:54.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.892+0000 7f4794c00640 1 -- 172.21.15.151:0/776655068 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 4999+0+0 (secure 0 0 0) 0x7f4784099410 con 0x7f479815c080 2024-09-16T15:50:54.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.892+0000 7f479ca00640 1 --2- 172.21.15.151:0/776655068 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4778079ed0 0x7f477807c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:54.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.893+0000 7f479ca00640 1 --2- 172.21.15.151:0/776655068 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4778079ed0 0x7f477807c390 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f4788035000 tx=0x7f4788004ab0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:54.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:54.896+0000 7f4794c00640 1 -- 172.21.15.151:0/776655068 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f47981386e0 con 0x7f479815c080 2024-09-16T15:50:55.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:55.078+0000 7f479f400640 1 -- 172.21.15.151:0/776655068 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f47981395b0 con 0x7f479815c080 2024-09-16T15:50:55.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:55.078+0000 7f4794c00640 1 -- 172.21.15.151:0/776655068 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) ==== 93+0+10 (secure 0 0 0) 0x7f47840634e0 con 0x7f479815c080 2024-09-16T15:50:55.080 INFO:teuthology.orchestra.run.smithi151.stdout:pg_num: 1 2024-09-16T15:50:55.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:55.081+0000 7f479f400640 1 -- 172.21.15.151:0/776655068 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4778079ed0 msgr2=0x7f477807c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:55.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:55.081+0000 7f479f400640 1 --2- 172.21.15.151:0/776655068 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4778079ed0 0x7f477807c390 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f4788035000 tx=0x7f4788004ab0 comp rx=0 tx=0).stop 2024-09-16T15:50:55.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:55.081+0000 7f479f400640 1 -- 172.21.15.151:0/776655068 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f479815c080 msgr2=0x7f47980738f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:55.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:55.081+0000 7f479f400640 1 --2- 172.21.15.151:0/776655068 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f479815c080 0x7f47980738f0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f478400b4d0 tx=0x7f478400b990 comp rx=0 tx=0).stop 2024-09-16T15:50:55.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:55.081+0000 7f479f400640 1 -- 172.21.15.151:0/776655068 shutdown_connections 2024-09-16T15:50:55.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:55.081+0000 7f479f400640 1 --2- 172.21.15.151:0/776655068 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4778079ed0 0x7f477807c390 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:55.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:55.081+0000 7f479f400640 1 --2- 172.21.15.151:0/776655068 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f479815e980 0x7f4798073e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:55.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:55.081+0000 7f479f400640 1 --2- 172.21.15.151:0/776655068 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f479815c080 0x7f47980738f0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:55.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:55.081+0000 7f479f400640 1 --2- 172.21.15.151:0/776655068 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47980b3370 0x7f47981381a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:55.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:55.081+0000 7f479f400640 1 -- 172.21.15.151:0/776655068 >> 172.21.15.151:0/776655068 conn(0x7f47980b1840 msgr2=0x7f47980aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:55.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:55.081+0000 7f479f400640 1 -- 172.21.15.151:0/776655068 shutdown_connections 2024-09-16T15:50:55.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:55.081+0000 7f479f400640 1 -- 172.21.15.151:0/776655068 wait complete. 2024-09-16T15:50:55.100 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_is_printed_in_clone_status_output' 2024-09-16T15:50:55.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:55.256+0000 7f0177e00640 1 -- 172.21.15.151:0/4280240907 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f015c003a50 msgr2=0x7f015c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:55.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:55.256+0000 7f0177e00640 1 --2- 172.21.15.151:0/4280240907 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f015c003a50 0x7f015c003e70 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f0168006fb0 tx=0x7f015c005830 comp rx=0 tx=0).stop 2024-09-16T15:50:55.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:55.256+0000 7f0177400640 1 -- 172.21.15.151:0/4280240907 reap_dead start 2024-09-16T15:50:55.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:55.256+0000 7f0177e00640 1 -- 172.21.15.151:0/4280240907 shutdown_connections 2024-09-16T15:50:55.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:55.256+0000 7f0177e00640 1 -- 172.21.15.151:0/4280240907 >> 172.21.15.151:0/4280240907 conn(0x7f01780cfa20 msgr2=0x7f01780cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:55.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:55.256+0000 7f0177e00640 1 -- 172.21.15.151:0/4280240907 shutdown_connections 2024-09-16T15:50:55.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:55.256+0000 7f0177e00640 1 -- 172.21.15.151:0/4280240907 wait complete. 2024-09-16T15:50:55.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:55.258+0000 7f0177e00640 1 Processor -- start 2024-09-16T15:50:55.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:55.258+0000 7f0177e00640 1 -- start start 2024-09-16T15:50:55.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:55.259+0000 7f0177e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f015c003a50 0x7f01780b1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:55.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:55.259+0000 7f0177e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01780b0940 0x7f01781511d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:55.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:55.259+0000 7f0177e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01781f8310 0x7f017815a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:55.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:55.259+0000 7f0177e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0178074660 con 0x7f015c003a50 2024-09-16T15:50:55.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:55.259+0000 7f0177e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f017806f320 con 0x7f01780b0940 2024-09-16T15:50:55.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:55.259+0000 7f0177e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f01780abb60 con 0x7f01781f8310 2024-09-16T15:50:55.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:55.259+0000 7f0176a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f015c003a50 0x7f01780b1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:55.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:55.259+0000 7f0176a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f015c003a50 0x7f01780b1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:54264/0 (socket says 172.21.15.151:54264) 2024-09-16T15:50:55.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:55.259+0000 7f0176a00640 1 -- 172.21.15.151:0/3352766736 learned_addr learned my addr 172.21.15.151:0/3352766736 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:55.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:55.260+0000 7f0176a00640 1 -- 172.21.15.151:0/3352766736 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01781f8310 msgr2=0x7f017815a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:55.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:55.260+0000 7f0176a00640 1 --2- 172.21.15.151:0/3352766736 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01781f8310 0x7f017815a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:55.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:55.260+0000 7f0176a00640 1 -- 172.21.15.151:0/3352766736 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01780b0940 msgr2=0x7f01781511d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:55.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:55.260+0000 7f0176a00640 1 --2- 172.21.15.151:0/3352766736 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01780b0940 0x7f01781511d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:55.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:55.260+0000 7f0176a00640 1 -- 172.21.15.151:0/3352766736 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01780c9460 con 0x7f015c003a50 2024-09-16T15:50:55.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:55.260+0000 7f0176a00640 1 --2- 172.21.15.151:0/3352766736 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f015c003a50 0x7f01780b1080 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f016802b0b0 tx=0x7f0168001d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:55.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:55.260+0000 7f016f400640 1 -- 172.21.15.151:0/3352766736 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0168031040 con 0x7f015c003a50 2024-09-16T15:50:55.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:55.260+0000 7f016f400640 1 -- 172.21.15.151:0/3352766736 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0168006540 con 0x7f015c003a50 2024-09-16T15:50:55.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:55.260+0000 7f0177e00640 1 -- 172.21.15.151:0/3352766736 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01780c9700 con 0x7f015c003a50 2024-09-16T15:50:55.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:55.260+0000 7f016f400640 1 -- 172.21.15.151:0/3352766736 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0168003a40 con 0x7f015c003a50 2024-09-16T15:50:55.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:55.260+0000 7f0177e00640 1 -- 172.21.15.151:0/3352766736 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01781f7b10 con 0x7f015c003a50 2024-09-16T15:50:55.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:55.261+0000 7f0177e00640 1 -- 172.21.15.151:0/3352766736 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01780b28f0 con 0x7f015c003a50 2024-09-16T15:50:55.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:55.262+0000 7f016f400640 1 -- 172.21.15.151:0/3352766736 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7f0168034070 con 0x7f015c003a50 2024-09-16T15:50:55.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:55.263+0000 7f016f400640 1 --2- 172.21.15.151:0/3352766736 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0158079ed0 0x7f015807c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:55.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:55.264+0000 7f016f400640 1 -- 172.21.15.151:0/3352766736 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 4999+0+0 (secure 0 0 0) 0x7f01680b9640 con 0x7f015c003a50 2024-09-16T15:50:55.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:55.264+0000 7f0176000640 1 --2- 172.21.15.151:0/3352766736 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0158079ed0 0x7f015807c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:55.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:55.264+0000 7f0176000640 1 --2- 172.21.15.151:0/3352766736 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0158079ed0 0x7f015807c390 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f0164005bc0 tx=0x7f0164005ab0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:55.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:55.267+0000 7f016f400640 1 -- 172.21.15.151:0/3352766736 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0168083790 con 0x7f015c003a50 2024-09-16T15:50:55.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:55.524+0000 7f0177e00640 1 -- 172.21.15.151:0/3352766736 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_is_printed_in_clone_status_output"]} v 0) -- 0x7f0178159dd0 con 0x7f015c003a50 2024-09-16T15:50:56.333 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.331+0000 7f016f400640 1 -- 172.21.15.151:0/3352766736 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_is_printed_in_clone_status_output"]}]=0 v528) ==== 181+0+0 (secure 0 0 0) 0x7f0168083090 con 0x7f015c003a50 2024-09-16T15:50:56.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.334+0000 7f0177e00640 1 -- 172.21.15.151:0/3352766736 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0158079ed0 msgr2=0x7f015807c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:56.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.334+0000 7f0177e00640 1 --2- 172.21.15.151:0/3352766736 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0158079ed0 0x7f015807c390 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f0164005bc0 tx=0x7f0164005ab0 comp rx=0 tx=0).stop 2024-09-16T15:50:56.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.334+0000 7f0177e00640 1 -- 172.21.15.151:0/3352766736 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f015c003a50 msgr2=0x7f01780b1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:56.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.334+0000 7f0177e00640 1 --2- 172.21.15.151:0/3352766736 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f015c003a50 0x7f01780b1080 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f016802b0b0 tx=0x7f0168001d70 comp rx=0 tx=0).stop 2024-09-16T15:50:56.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.335+0000 7f0177e00640 1 -- 172.21.15.151:0/3352766736 shutdown_connections 2024-09-16T15:50:56.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.335+0000 7f0177e00640 1 --2- 172.21.15.151:0/3352766736 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0158079ed0 0x7f015807c390 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:56.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.335+0000 7f0177e00640 1 --2- 172.21.15.151:0/3352766736 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01780b0940 0x7f01781511d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:56.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.335+0000 7f0177e00640 1 --2- 172.21.15.151:0/3352766736 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01781f8310 0x7f017815a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:56.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.335+0000 7f0177e00640 1 --2- 172.21.15.151:0/3352766736 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f015c003a50 0x7f01780b1080 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:56.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.335+0000 7f0177e00640 1 -- 172.21.15.151:0/3352766736 >> 172.21.15.151:0/3352766736 conn(0x7f01780cfa20 msgr2=0x7f0178158460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:56.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.335+0000 7f0177e00640 1 -- 172.21.15.151:0/3352766736 shutdown_connections 2024-09-16T15:50:56.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.335+0000 7f0177e00640 1 -- 172.21.15.151:0/3352766736 wait complete. 2024-09-16T15:50:56.355 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-16T15:50:56.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.514+0000 7f4af2600640 1 -- 172.21.15.151:0/3811882217 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4aec073fb0 msgr2=0x7f4aec074440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:56.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.514+0000 7f4af2600640 1 --2- 172.21.15.151:0/3811882217 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4aec073fb0 0x7f4aec074440 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f4ae4005fd0 tx=0x7f4ae402aa10 comp rx=0 tx=0).stop 2024-09-16T15:50:56.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.514+0000 7f4af2600640 1 -- 172.21.15.151:0/3811882217 shutdown_connections 2024-09-16T15:50:56.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.514+0000 7f4af2600640 1 --2- 172.21.15.151:0/3811882217 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4aec073fb0 0x7f4aec074440 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:56.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.514+0000 7f4af2600640 1 --2- 172.21.15.151:0/3811882217 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4aec0ccf00 0x7f4aec0b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:56.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.514+0000 7f4af2600640 1 -- 172.21.15.151:0/3811882217 >> 172.21.15.151:0/3811882217 conn(0x7f4aec0b1950 msgr2=0x7f4aec0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:56.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.514+0000 7f4af2600640 1 -- 172.21.15.151:0/3811882217 shutdown_connections 2024-09-16T15:50:56.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.514+0000 7f4af2600640 1 -- 172.21.15.151:0/3811882217 wait complete. 2024-09-16T15:50:56.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.516+0000 7f4af2600640 1 Processor -- start 2024-09-16T15:50:56.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.516+0000 7f4af2600640 1 -- start start 2024-09-16T15:50:56.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.517+0000 7f4af2600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4aec073fb0 0x7f4aec1f5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:56.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.517+0000 7f4af2600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4aec0ccf00 0x7f4aec1f3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:56.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.517+0000 7f4af2600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4aec15f5a0 0x7f4aec14df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:56.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.517+0000 7f4af2600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4aec06f2e0 con 0x7f4aec0ccf00 2024-09-16T15:50:56.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.517+0000 7f4af2600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4aec06f160 con 0x7f4aec073fb0 2024-09-16T15:50:56.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.517+0000 7f4af2600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f4aec06f460 con 0x7f4aec15f5a0 2024-09-16T15:50:56.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.517+0000 7f4ae2a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4aec0ccf00 0x7f4aec1f3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:56.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.517+0000 7f4ae2a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4aec0ccf00 0x7f4aec1f3850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:54278/0 (socket says 172.21.15.151:54278) 2024-09-16T15:50:56.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.517+0000 7f4ae2a00640 1 -- 172.21.15.151:0/3604634415 learned_addr learned my addr 172.21.15.151:0/3604634415 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:56.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.517+0000 7f4aebe00640 1 --2- 172.21.15.151:0/3604634415 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4aec15f5a0 0x7f4aec14df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:56.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.517+0000 7f4aeb400640 1 --2- 172.21.15.151:0/3604634415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4aec073fb0 0x7f4aec1f5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:56.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.517+0000 7f4ae2a00640 1 -- 172.21.15.151:0/3604634415 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4aec15f5a0 msgr2=0x7f4aec14df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:56.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.517+0000 7f4ae2a00640 1 --2- 172.21.15.151:0/3604634415 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4aec15f5a0 0x7f4aec14df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:56.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.517+0000 7f4ae2a00640 1 -- 172.21.15.151:0/3604634415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4aec073fb0 msgr2=0x7f4aec1f5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:56.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.517+0000 7f4ae2a00640 1 --2- 172.21.15.151:0/3604634415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4aec073fb0 0x7f4aec1f5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:56.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.517+0000 7f4ae2a00640 1 -- 172.21.15.151:0/3604634415 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4aec1f3d90 con 0x7f4aec0ccf00 2024-09-16T15:50:56.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.518+0000 7f4ae2a00640 1 --2- 172.21.15.151:0/3604634415 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4aec0ccf00 0x7f4aec1f3850 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f4ad400d950 tx=0x7f4ad400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:56.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.518+0000 7f4ae3e00640 1 -- 172.21.15.151:0/3604634415 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad4017070 con 0x7f4aec0ccf00 2024-09-16T15:50:56.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.518+0000 7f4ae3e00640 1 -- 172.21.15.151:0/3604634415 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4ad4009d20 con 0x7f4aec0ccf00 2024-09-16T15:50:56.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.518+0000 7f4ae3e00640 1 -- 172.21.15.151:0/3604634415 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad40106c0 con 0x7f4aec0ccf00 2024-09-16T15:50:56.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.518+0000 7f4af2600640 1 -- 172.21.15.151:0/3604634415 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4aec14e940 con 0x7f4aec0ccf00 2024-09-16T15:50:56.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.518+0000 7f4af2600640 1 -- 172.21.15.151:0/3604634415 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4aec0cf060 con 0x7f4aec0ccf00 2024-09-16T15:50:56.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.519+0000 7f4af2600640 1 -- 172.21.15.151:0/3604634415 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4aec06f9f0 con 0x7f4aec0ccf00 2024-09-16T15:50:56.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.520+0000 7f4ae3e00640 1 -- 172.21.15.151:0/3604634415 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7f4ad4010860 con 0x7f4aec0ccf00 2024-09-16T15:50:56.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.521+0000 7f4ae3e00640 1 --2- 172.21.15.151:0/3604634415 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4ac8079ed0 0x7f4ac807c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:56.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.521+0000 7f4ae3e00640 1 -- 172.21.15.151:0/3604634415 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 4999+0+0 (secure 0 0 0) 0x7f4ad4099a00 con 0x7f4aec0ccf00 2024-09-16T15:50:56.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.522+0000 7f4aeb400640 1 --2- 172.21.15.151:0/3604634415 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4ac8079ed0 0x7f4ac807c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:56.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.522+0000 7f4aeb400640 1 --2- 172.21.15.151:0/3604634415 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4ac8079ed0 0x7f4ac807c390 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f4adc000f60 tx=0x7f4adc009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:56.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.525+0000 7f4ae3e00640 1 -- 172.21.15.151:0/3604634415 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4ad4063480 con 0x7f4aec0ccf00 2024-09-16T15:50:56.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.711+0000 7f4af2600640 1 -- 172.21.15.151:0/3604634415 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f4aec154de0 con 0x7f4aec0ccf00 2024-09-16T15:50:56.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.711+0000 7f4ae3e00640 1 -- 172.21.15.151:0/3604634415 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v44) ==== 59+0+1998 (secure 0 0 0) 0x7f4ad4067eb0 con 0x7f4aec0ccf00 2024-09-16T15:50:56.713 INFO:teuthology.orchestra.run.smithi151.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-16T15:50:56.714 INFO:teuthology.orchestra.run.smithi151.stdout:mon advanced mon_op_complaint_time 120 2024-09-16T15:50:56.714 INFO:teuthology.orchestra.run.smithi151.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-16T15:50:56.714 INFO:teuthology.orchestra.run.smithi151.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-16T15:50:56.714 INFO:teuthology.orchestra.run.smithi151.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3795.338492 2024-09-16T15:50:56.714 INFO:teuthology.orchestra.run.smithi151.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 4030.355715 2024-09-16T15:50:56.714 INFO:teuthology.orchestra.run.smithi151.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3817.533640 2024-09-16T15:50:56.714 INFO:teuthology.orchestra.run.smithi151.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3950.656136 2024-09-16T15:50:56.714 INFO:teuthology.orchestra.run.smithi151.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4449.408052 2024-09-16T15:50:56.714 INFO:teuthology.orchestra.run.smithi151.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4439.932296 2024-09-16T15:50:56.714 INFO:teuthology.orchestra.run.smithi151.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4566.875579 2024-09-16T15:50:56.714 INFO:teuthology.orchestra.run.smithi151.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 6320.073632 2024-09-16T15:50:56.714 INFO:teuthology.orchestra.run.smithi151.stdout:mds advanced debug_mds 20/20 2024-09-16T15:50:56.714 INFO:teuthology.orchestra.run.smithi151.stdout:mds advanced debug_mds_balancer 20/20 2024-09-16T15:50:56.715 INFO:teuthology.orchestra.run.smithi151.stdout:mds advanced debug_ms 2/2 2024-09-16T15:50:56.715 INFO:teuthology.orchestra.run.smithi151.stdout:mds dev mds_debug_frag true 2024-09-16T15:50:56.715 INFO:teuthology.orchestra.run.smithi151.stdout:mds dev mds_debug_scatterstat true 2024-09-16T15:50:56.715 INFO:teuthology.orchestra.run.smithi151.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-16T15:50:56.715 INFO:teuthology.orchestra.run.smithi151.stdout:mds dev mds_verify_scatter true 2024-09-16T15:50:56.715 INFO:teuthology.orchestra.run.smithi151.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-16T15:50:56.715 INFO:teuthology.orchestra.run.smithi151.stdout:mds advanced rados_mon_op_timeout 900 2024-09-16T15:50:56.715 INFO:teuthology.orchestra.run.smithi151.stdout:mds advanced rados_osd_op_timeout 900 2024-09-16T15:50:56.715 INFO:teuthology.orchestra.run.smithi151.stdout:client advanced client_mount_timeout 600 2024-09-16T15:50:56.715 INFO:teuthology.orchestra.run.smithi151.stdout:client advanced debug_client 20/20 2024-09-16T15:50:56.715 INFO:teuthology.orchestra.run.smithi151.stdout:client advanced debug_ms 1/1 2024-09-16T15:50:56.715 INFO:teuthology.orchestra.run.smithi151.stdout:client advanced rados_mon_op_timeout 900 2024-09-16T15:50:56.715 INFO:teuthology.orchestra.run.smithi151.stdout:client advanced rados_osd_op_timeout 900 2024-09-16T15:50:56.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.714+0000 7f4af2600640 1 -- 172.21.15.151:0/3604634415 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4ac8079ed0 msgr2=0x7f4ac807c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:56.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.714+0000 7f4af2600640 1 --2- 172.21.15.151:0/3604634415 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4ac8079ed0 0x7f4ac807c390 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f4adc000f60 tx=0x7f4adc009040 comp rx=0 tx=0).stop 2024-09-16T15:50:56.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.714+0000 7f4af2600640 1 -- 172.21.15.151:0/3604634415 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4aec0ccf00 msgr2=0x7f4aec1f3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:56.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.714+0000 7f4af2600640 1 --2- 172.21.15.151:0/3604634415 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4aec0ccf00 0x7f4aec1f3850 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f4ad400d950 tx=0x7f4ad400de10 comp rx=0 tx=0).stop 2024-09-16T15:50:56.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.714+0000 7f4af2600640 1 -- 172.21.15.151:0/3604634415 shutdown_connections 2024-09-16T15:50:56.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.714+0000 7f4af2600640 1 --2- 172.21.15.151:0/3604634415 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4ac8079ed0 0x7f4ac807c390 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:56.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.714+0000 7f4af2600640 1 --2- 172.21.15.151:0/3604634415 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4aec15f5a0 0x7f4aec14df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:56.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.714+0000 7f4af2600640 1 --2- 172.21.15.151:0/3604634415 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4aec0ccf00 0x7f4aec1f3850 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:56.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.714+0000 7f4af2600640 1 --2- 172.21.15.151:0/3604634415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4aec073fb0 0x7f4aec1f5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:56.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.714+0000 7f4af2600640 1 -- 172.21.15.151:0/3604634415 >> 172.21.15.151:0/3604634415 conn(0x7f4aec0b1950 msgr2=0x7f4aec0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:56.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.714+0000 7f4af2600640 1 -- 172.21.15.151:0/3604634415 shutdown_connections 2024-09-16T15:50:56.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.714+0000 7f4af2600640 1 -- 172.21.15.151:0/3604634415 wait complete. 2024-09-16T15:50:56.733 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-16T15:50:56.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.891+0000 7f4511e00640 1 -- 172.21.15.151:0/3606733855 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f450c15e0c0 msgr2=0x7f450c0ae290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:56.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.891+0000 7f4511e00640 1 --2- 172.21.15.151:0/3606733855 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f450c15e0c0 0x7f450c0ae290 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f450c070a70 tx=0x7f44fc02a560 comp rx=0 tx=0).stop 2024-09-16T15:50:56.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.891+0000 7f4511e00640 1 -- 172.21.15.151:0/3606733855 shutdown_connections 2024-09-16T15:50:56.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.891+0000 7f4511e00640 1 --2- 172.21.15.151:0/3606733855 >> v2:172.21.15.167:3300/0 conn(0x7f450c0cdfa0 0x7f450c0ca7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:56.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.891+0000 7f4511e00640 1 --2- 172.21.15.151:0/3606733855 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f450c15e0c0 0x7f450c0ae290 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:56.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.891+0000 7f4511e00640 1 -- 172.21.15.151:0/3606733855 >> 172.21.15.151:0/3606733855 conn(0x7f450c0cf9f0 msgr2=0x7f450c0cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:56.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.891+0000 7f4511e00640 1 -- 172.21.15.151:0/3606733855 shutdown_connections 2024-09-16T15:50:56.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.891+0000 7f4511e00640 1 -- 172.21.15.151:0/3606733855 wait complete. 2024-09-16T15:50:56.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.893+0000 7f4511e00640 1 Processor -- start 2024-09-16T15:50:56.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.893+0000 7f4511e00640 1 -- start start 2024-09-16T15:50:56.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.894+0000 7f4511e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f450c06ff60 0x7f450c0b5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:56.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.894+0000 7f4511e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f450c0cdfa0 0x7f450c151d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:56.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.894+0000 7f4511e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f450c15e0c0 0x7f450c15c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:56.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.894+0000 7f4511e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f450c0cb1a0 con 0x7f450c15e0c0 2024-09-16T15:50:56.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.894+0000 7f4511e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f450c0cb020 con 0x7f450c0cdfa0 2024-09-16T15:50:56.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.894+0000 7f4511e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f450c0cb320 con 0x7f450c06ff60 2024-09-16T15:50:56.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.894+0000 7f4511400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f450c15e0c0 0x7f450c15c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:56.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.894+0000 7f4511400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f450c15e0c0 0x7f450c15c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:54298/0 (socket says 172.21.15.151:54298) 2024-09-16T15:50:56.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.894+0000 7f4511400640 1 -- 172.21.15.151:0/2148632005 learned_addr learned my addr 172.21.15.151:0/2148632005 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:56.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.894+0000 7f450be00640 1 --2- 172.21.15.151:0/2148632005 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f450c0cdfa0 0x7f450c151d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:56.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.894+0000 7f4510a00640 1 --2- 172.21.15.151:0/2148632005 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f450c06ff60 0x7f450c0b5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:56.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.895+0000 7f4511400640 1 -- 172.21.15.151:0/2148632005 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f450c06ff60 msgr2=0x7f450c0b5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:56.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.895+0000 7f4511400640 1 --2- 172.21.15.151:0/2148632005 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f450c06ff60 0x7f450c0b5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:56.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.895+0000 7f4511400640 1 -- 172.21.15.151:0/2148632005 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f450c0cdfa0 msgr2=0x7f450c151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:56.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.895+0000 7f4511400640 1 --2- 172.21.15.151:0/2148632005 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f450c0cdfa0 0x7f450c151d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:56.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.895+0000 7f4511400640 1 -- 172.21.15.151:0/2148632005 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f450c1f4bc0 con 0x7f450c15e0c0 2024-09-16T15:50:56.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.895+0000 7f4511400640 1 --2- 172.21.15.151:0/2148632005 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f450c15e0c0 0x7f450c15c340 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f450400ead0 tx=0x7f450400ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:56.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.895+0000 7f4508c00640 1 -- 172.21.15.151:0/2148632005 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450400cde0 con 0x7f450c15e0c0 2024-09-16T15:50:56.899 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.895+0000 7f4508c00640 1 -- 172.21.15.151:0/2148632005 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4504004c10 con 0x7f450c15e0c0 2024-09-16T15:50:56.899 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.895+0000 7f4508c00640 1 -- 172.21.15.151:0/2148632005 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4504005020 con 0x7f450c15e0c0 2024-09-16T15:50:56.899 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.895+0000 7f4511e00640 1 -- 172.21.15.151:0/2148632005 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f450c1f4ea0 con 0x7f450c15e0c0 2024-09-16T15:50:56.899 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.895+0000 7f4511e00640 1 -- 172.21.15.151:0/2148632005 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f450c1f5460 con 0x7f450c15e0c0 2024-09-16T15:50:56.899 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.896+0000 7f4511e00640 1 -- 172.21.15.151:0/2148632005 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f450c1f4930 con 0x7f450c15e0c0 2024-09-16T15:50:56.899 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.898+0000 7f4508c00640 1 -- 172.21.15.151:0/2148632005 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7f45040051c0 con 0x7f450c15e0c0 2024-09-16T15:50:56.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.899+0000 7f4508c00640 1 --2- 172.21.15.151:0/2148632005 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f44ec079ed0 0x7f44ec07c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:56.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.899+0000 7f4508c00640 1 -- 172.21.15.151:0/2148632005 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 4999+0+0 (secure 0 0 0) 0x7f450409a080 con 0x7f450c15e0c0 2024-09-16T15:50:56.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.899+0000 7f4510a00640 1 --2- 172.21.15.151:0/2148632005 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f44ec079ed0 0x7f44ec07c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:56.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.900+0000 7f4510a00640 1 --2- 172.21.15.151:0/2148632005 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f44ec079ed0 0x7f44ec07c390 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f44fc033000 tx=0x7f44fc004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:56.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:56.903+0000 7f4508c00640 1 -- 172.21.15.151:0/2148632005 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4504016030 con 0x7f450c15e0c0 2024-09-16T15:50:57.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:57.084+0000 7f4511e00640 1 -- 172.21.15.151:0/2148632005 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f450c1f5660 con 0x7f450c15e0c0 2024-09-16T15:50:57.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:57.088+0000 7f4508c00640 1 -- 172.21.15.151:0/2148632005 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v45)=0 v45) ==== 122+0+0 (secure 0 0 0) 0x7f4504063b00 con 0x7f450c15e0c0 2024-09-16T15:50:57.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:57.090+0000 7f4511e00640 1 -- 172.21.15.151:0/2148632005 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f44ec079ed0 msgr2=0x7f44ec07c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:57.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:57.090+0000 7f4511e00640 1 --2- 172.21.15.151:0/2148632005 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f44ec079ed0 0x7f44ec07c390 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f44fc033000 tx=0x7f44fc004730 comp rx=0 tx=0).stop 2024-09-16T15:50:57.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:57.090+0000 7f4511e00640 1 -- 172.21.15.151:0/2148632005 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f450c15e0c0 msgr2=0x7f450c15c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:57.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:57.090+0000 7f4511e00640 1 --2- 172.21.15.151:0/2148632005 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f450c15e0c0 0x7f450c15c340 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f450400ead0 tx=0x7f450400ef90 comp rx=0 tx=0).stop 2024-09-16T15:50:57.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:57.090+0000 7f4511e00640 1 -- 172.21.15.151:0/2148632005 shutdown_connections 2024-09-16T15:50:57.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:57.090+0000 7f4511e00640 1 --2- 172.21.15.151:0/2148632005 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f44ec079ed0 0x7f44ec07c390 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:57.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:57.090+0000 7f4511e00640 1 --2- 172.21.15.151:0/2148632005 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f450c0cdfa0 0x7f450c151d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:57.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:57.090+0000 7f4511e00640 1 --2- 172.21.15.151:0/2148632005 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f450c15e0c0 0x7f450c15c340 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:57.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:57.090+0000 7f4511e00640 1 --2- 172.21.15.151:0/2148632005 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f450c06ff60 0x7f450c0b5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:57.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:57.090+0000 7f4511e00640 1 -- 172.21.15.151:0/2148632005 >> 172.21.15.151:0/2148632005 conn(0x7f450c0cf9f0 msgr2=0x7f450c0cc8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:57.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:57.090+0000 7f4511e00640 1 -- 172.21.15.151:0/2148632005 shutdown_connections 2024-09-16T15:50:57.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:57.090+0000 7f4511e00640 1 -- 172.21.15.151:0/2148632005 wait complete. 2024-09-16T15:50:57.108 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-16T15:50:57.109 DEBUG:teuthology.orchestra.run.smithi151:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:50:57.202 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:50:57.300 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-16T15:50:57.300 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:50:57.300 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:50:57.335 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T15:50:57.335 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-16T15:50:57.335 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:50:57.373 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:50:57.373 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:50:57.373 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:50:57.410 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T15:50:57.410 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-16T15:50:57.410 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:50:57.448 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:50:57.448 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-16T15:50:57.449 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:50:57.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:57.607+0000 7f88ace00640 1 -- 172.21.15.151:0/1919576672 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f888c003a00 msgr2=0x7f888c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:57.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:57.607+0000 7f88ace00640 1 --2- 172.21.15.151:0/1919576672 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f888c003a00 0x7f888c003e20 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f8898009ed0 tx=0x7f888c005830 comp rx=0 tx=0).stop 2024-09-16T15:50:57.610 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:57.607+0000 7f88a6800640 1 -- 172.21.15.151:0/1919576672 reap_dead start 2024-09-16T15:50:57.610 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:57.607+0000 7f88ace00640 1 -- 172.21.15.151:0/1919576672 shutdown_connections 2024-09-16T15:50:57.610 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:57.607+0000 7f88ace00640 1 -- 172.21.15.151:0/1919576672 >> 172.21.15.151:0/1919576672 conn(0x7f88a80b1950 msgr2=0x7f88a80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:57.610 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:57.607+0000 7f88ace00640 1 -- 172.21.15.151:0/1919576672 shutdown_connections 2024-09-16T15:50:57.610 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:57.607+0000 7f88ace00640 1 -- 172.21.15.151:0/1919576672 wait complete. 2024-09-16T15:50:57.611 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:57.609+0000 7f88ace00640 1 Processor -- start 2024-09-16T15:50:57.611 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:57.609+0000 7f88ace00640 1 -- start start 2024-09-16T15:50:57.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:57.610+0000 7f88ace00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f888c003a00 0x7f88a815a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:57.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:57.610+0000 7f88ace00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f88a8159a60 0x7f88a80cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:57.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:57.610+0000 7f88ace00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88a815c0c0 0x7f88a815c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:57.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:57.610+0000 7f88ace00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f88a80b25c0 con 0x7f88a8159a60 2024-09-16T15:50:57.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:57.610+0000 7f88ace00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f88a80b2440 con 0x7f88a815c0c0 2024-09-16T15:50:57.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:57.610+0000 7f88ace00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f88a80b2740 con 0x7f888c003a00 2024-09-16T15:50:57.614 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:57.610+0000 7f88a5400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f88a8159a60 0x7f88a80cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:57.614 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:57.610+0000 7f88a5400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f88a8159a60 0x7f88a80cc4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:54304/0 (socket says 172.21.15.151:54304) 2024-09-16T15:50:57.614 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:57.610+0000 7f88a5400640 1 -- 172.21.15.151:0/3034332280 learned_addr learned my addr 172.21.15.151:0/3034332280 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:57.614 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:57.610+0000 7f88a5e00640 1 --2- 172.21.15.151:0/3034332280 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f888c003a00 0x7f88a815a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:57.614 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:57.610+0000 7f88a6800640 1 --2- 172.21.15.151:0/3034332280 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88a815c0c0 0x7f88a815c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:57.614 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:57.611+0000 7f88a5400640 1 -- 172.21.15.151:0/3034332280 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f888c003a00 msgr2=0x7f88a815a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:57.614 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:57.611+0000 7f88a5400640 1 --2- 172.21.15.151:0/3034332280 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f888c003a00 0x7f88a815a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:57.614 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:57.611+0000 7f88a5400640 1 -- 172.21.15.151:0/3034332280 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88a815c0c0 msgr2=0x7f88a815c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:57.614 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:57.611+0000 7f88a5400640 1 --2- 172.21.15.151:0/3034332280 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88a815c0c0 0x7f88a815c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:57.614 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:57.611+0000 7f88a5400640 1 -- 172.21.15.151:0/3034332280 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88a80cc9e0 con 0x7f88a8159a60 2024-09-16T15:50:57.614 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:57.611+0000 7f88a5400640 1 --2- 172.21.15.151:0/3034332280 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f88a8159a60 0x7f88a80cc4a0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f8894006660 tx=0x7f88940039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:57.614 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:57.611+0000 7f889e000640 1 -- 172.21.15.151:0/3034332280 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8894015070 con 0x7f88a8159a60 2024-09-16T15:50:57.614 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:57.611+0000 7f889e000640 1 -- 172.21.15.151:0/3034332280 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8894011900 con 0x7f88a8159a60 2024-09-16T15:50:57.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:57.611+0000 7f88ace00640 1 -- 172.21.15.151:0/3034332280 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88a80b3800 con 0x7f88a8159a60 2024-09-16T15:50:57.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:57.611+0000 7f889e000640 1 -- 172.21.15.151:0/3034332280 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8894011d30 con 0x7f88a8159a60 2024-09-16T15:50:57.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:57.611+0000 7f88ace00640 1 -- 172.21.15.151:0/3034332280 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88a80b3d80 con 0x7f88a8159a60 2024-09-16T15:50:57.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:57.612+0000 7f88ace00640 1 -- 172.21.15.151:0/3034332280 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88a8074460 con 0x7f88a8159a60 2024-09-16T15:50:57.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:57.613+0000 7f889e000640 1 -- 172.21.15.151:0/3034332280 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7f8894018070 con 0x7f88a8159a60 2024-09-16T15:50:57.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:57.615+0000 7f889e000640 1 --2- 172.21.15.151:0/3034332280 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8888079f00 0x7f888807c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:57.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:57.615+0000 7f889e000640 1 -- 172.21.15.151:0/3034332280 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 4999+0+0 (secure 0 0 0) 0x7f889409a300 con 0x7f88a8159a60 2024-09-16T15:50:57.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:57.615+0000 7f88a5e00640 1 --2- 172.21.15.151:0/3034332280 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8888079f00 0x7f888807c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:57.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:57.616+0000 7f88a5e00640 1 --2- 172.21.15.151:0/3034332280 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8888079f00 0x7f888807c3c0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f8898035000 tx=0x7f889802d040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:57.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:57.619+0000 7f889e000640 1 -- 172.21.15.151:0/3034332280 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8894063d80 con 0x7f88a8159a60 2024-09-16T15:50:57.840 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:57.838+0000 7f88ace00640 1 -- 172.21.15.151:0/3034332280 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f88a80cf990 con 0x7f88a8159a60 2024-09-16T15:50:57.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:57.839+0000 7f889e000640 1 -- 172.21.15.151:0/3034332280 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+3917 (secure 0 0 0) 0x7f88940687b0 con 0x7f88a8159a60 2024-09-16T15:50:57.841 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:50:57.841 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":59,"btime":"2024-09-16T15:50:51:313495+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6134,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/743718626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":743718626},{"type":"v1","addr":"172.21.15.167:6836","nonce":743718626}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6137,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1262689373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1262689373},{"type":"v1","addr":"172.21.15.167:6837","nonce":1262689373}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6670,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/1127792363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1127792363},{"type":"v1","addr":"172.21.15.151:6834","nonce":1127792363}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":7809,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/128499555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":128499555},{"type":"v1","addr":"172.21.15.151:6835","nonce":128499555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":59}],"filesystems":[]} 2024-09-16T15:50:57.841 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 59 2024-09-16T15:50:57.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:57.841+0000 7f88ace00640 1 -- 172.21.15.151:0/3034332280 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8888079f00 msgr2=0x7f888807c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:57.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:57.841+0000 7f88ace00640 1 --2- 172.21.15.151:0/3034332280 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8888079f00 0x7f888807c3c0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f8898035000 tx=0x7f889802d040 comp rx=0 tx=0).stop 2024-09-16T15:50:57.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:57.841+0000 7f88ace00640 1 -- 172.21.15.151:0/3034332280 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f88a8159a60 msgr2=0x7f88a80cc4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:57.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:57.841+0000 7f88ace00640 1 --2- 172.21.15.151:0/3034332280 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f88a8159a60 0x7f88a80cc4a0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f8894006660 tx=0x7f88940039c0 comp rx=0 tx=0).stop 2024-09-16T15:50:57.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:57.841+0000 7f88ace00640 1 -- 172.21.15.151:0/3034332280 shutdown_connections 2024-09-16T15:50:57.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:57.841+0000 7f88ace00640 1 --2- 172.21.15.151:0/3034332280 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8888079f00 0x7f888807c3c0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:57.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:57.841+0000 7f88ace00640 1 --2- 172.21.15.151:0/3034332280 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88a815c0c0 0x7f88a815c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:57.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:57.841+0000 7f88ace00640 1 --2- 172.21.15.151:0/3034332280 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f88a8159a60 0x7f88a80cc4a0 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:57.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:57.841+0000 7f88ace00640 1 --2- 172.21.15.151:0/3034332280 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f888c003a00 0x7f88a815a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:57.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:57.841+0000 7f88ace00640 1 -- 172.21.15.151:0/3034332280 >> 172.21.15.151:0/3034332280 conn(0x7f88a80b1950 msgr2=0x7f88a80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:57.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:57.841+0000 7f88ace00640 1 -- 172.21.15.151:0/3034332280 shutdown_connections 2024-09-16T15:50:57.844 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:57.841+0000 7f88ace00640 1 -- 172.21.15.151:0/3034332280 wait complete. 2024-09-16T15:50:57.860 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-16T15:50:57.860 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-16T15:50:57.860 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-16T15:50:57.860 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:50:57.860 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-16T15:50:57.860 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-16T15:50:57.861 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-16T15:50:57.861 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:50:57.861 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-16T15:50:57.861 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-16T15:50:57.861 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-16T15:50:57.861 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:50:57.861 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-16T15:50:57.861 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-16T15:50:57.862 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-16T15:50:57.862 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:50:57.876 INFO:tasks.ceph.mds.c:Stopped 2024-09-16T15:50:57.876 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-16T15:50:57.877 INFO:tasks.ceph.mds.a:Stopped 2024-09-16T15:50:57.877 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-16T15:50:57.878 INFO:tasks.ceph.mds.b:Stopped 2024-09-16T15:50:57.878 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-16T15:50:57.879 INFO:tasks.ceph.mds.d:Stopped 2024-09-16T15:50:57.879 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-16T15:50:57.920 INFO:tasks.ceph.mds.c:Started 2024-09-16T15:50:57.920 DEBUG:teuthology.parallel:result is None 2024-09-16T15:50:57.922 INFO:tasks.ceph.mds.b:Started 2024-09-16T15:50:57.922 DEBUG:teuthology.parallel:result is None 2024-09-16T15:50:57.922 INFO:tasks.ceph.mds.a:Started 2024-09-16T15:50:57.923 DEBUG:teuthology.parallel:result is None 2024-09-16T15:50:57.924 INFO:tasks.ceph.mds.d:Started 2024-09-16T15:50:57.924 DEBUG:teuthology.parallel:result is None 2024-09-16T15:50:57.925 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-16T15:50:58.052 INFO:tasks.ceph.mds.a.smithi151.stdout:starting mds.a at 2024-09-16T15:50:58.052 INFO:tasks.ceph.mds.c.smithi151.stdout:starting mds.c at 2024-09-16T15:50:58.053 INFO:tasks.ceph.mds.d.smithi167.stdout:starting mds.d at 2024-09-16T15:50:58.054 INFO:tasks.ceph.mds.b.smithi167.stdout:starting mds.b at 2024-09-16T15:50:58.080 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-09-16T15:50:58.080 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-09-16T15:50:58.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.090+0000 7ff6f2200640 1 -- 172.21.15.151:0/584667548 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff6ec0d1ec0 msgr2=0x7ff6ec0d22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:58.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.090+0000 7ff6f2200640 1 --2- 172.21.15.151:0/584667548 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff6ec0d1ec0 0x7ff6ec0d22a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff6e0009860 tx=0x7ff6e002a5a0 comp rx=0 tx=0).stop 2024-09-16T15:50:58.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.091+0000 7ff6f2200640 1 -- 172.21.15.151:0/584667548 shutdown_connections 2024-09-16T15:50:58.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.091+0000 7ff6f2200640 1 --2- 172.21.15.151:0/584667548 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff6ec0d1ec0 0x7ff6ec0d22a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:58.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.091+0000 7ff6f2200640 1 -- 172.21.15.151:0/584667548 >> 172.21.15.151:0/584667548 conn(0x7ff6ec0d3de0 msgr2=0x7ff6ec0d41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:58.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.091+0000 7ff6f2200640 1 -- 172.21.15.151:0/584667548 shutdown_connections 2024-09-16T15:50:58.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.091+0000 7ff6f2200640 1 -- 172.21.15.151:0/584667548 wait complete. 2024-09-16T15:50:58.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.092+0000 7ff6f2200640 1 Processor -- start 2024-09-16T15:50:58.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.092+0000 7ff6f2200640 1 -- start start 2024-09-16T15:50:58.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.093+0000 7ff6f2200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6ec0b28d0 0x7ff6ec154420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:58.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.093+0000 7ff6f2200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff6ec0b4d10 0x7ff6ec0c7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:58.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.093+0000 7ff6f2200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6ec0d1ec0 0x7ff6ec0c8270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:58.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.093+0000 7ff6f2200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ec1692e0 con 0x7ff6ec0d1ec0 2024-09-16T15:50:58.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.093+0000 7ff6f2200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff6ec0b4a50 con 0x7ff6ec0b28d0 2024-09-16T15:50:58.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.093+0000 7ff6f2200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7ff6ec169460 con 0x7ff6ec0b4d10 2024-09-16T15:50:58.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.094+0000 7ff6f1800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6ec0d1ec0 0x7ff6ec0c8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:58.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.094+0000 7ff6f1800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6ec0d1ec0 0x7ff6ec0c8270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:54324/0 (socket says 172.21.15.151:54324) 2024-09-16T15:50:58.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.094+0000 7ff6f1800640 1 -- 172.21.15.151:0/1014276748 learned_addr learned my addr 172.21.15.151:0/1014276748 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:58.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.094+0000 7ff6ebe00640 1 --2- 172.21.15.151:0/1014276748 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff6ec0b4d10 0x7ff6ec0c7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:58.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.094+0000 7ff6f0e00640 1 --2- 172.21.15.151:0/1014276748 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6ec0b28d0 0x7ff6ec154420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:58.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.094+0000 7ff6f1800640 1 -- 172.21.15.151:0/1014276748 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff6ec0b4d10 msgr2=0x7ff6ec0c7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:58.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.094+0000 7ff6f1800640 1 --2- 172.21.15.151:0/1014276748 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff6ec0b4d10 0x7ff6ec0c7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:58.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.094+0000 7ff6f1800640 1 -- 172.21.15.151:0/1014276748 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6ec0b28d0 msgr2=0x7ff6ec154420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:58.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.094+0000 7ff6f1800640 1 --2- 172.21.15.151:0/1014276748 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6ec0b28d0 0x7ff6ec154420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:58.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.094+0000 7ff6f1800640 1 -- 172.21.15.151:0/1014276748 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6ec154960 con 0x7ff6ec0d1ec0 2024-09-16T15:50:58.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.094+0000 7ff6f1800640 1 --2- 172.21.15.151:0/1014276748 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6ec0d1ec0 0x7ff6ec0c8270 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7ff6e400c770 tx=0x7ff6e400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:58.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.094+0000 7ff6e8c00640 1 -- 172.21.15.151:0/1014276748 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e4003b70 con 0x7ff6ec0d1ec0 2024-09-16T15:50:58.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.094+0000 7ff6e8c00640 1 -- 172.21.15.151:0/1014276748 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff6e4003d10 con 0x7ff6ec0d1ec0 2024-09-16T15:50:58.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.095+0000 7ff6f2200640 1 -- 172.21.15.151:0/1014276748 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6ec0c8e50 con 0x7ff6ec0d1ec0 2024-09-16T15:50:58.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.095+0000 7ff6e8c00640 1 -- 172.21.15.151:0/1014276748 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e400aab0 con 0x7ff6ec0d1ec0 2024-09-16T15:50:58.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.095+0000 7ff6f2200640 1 -- 172.21.15.151:0/1014276748 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6ec0c96d0 con 0x7ff6ec0d1ec0 2024-09-16T15:50:58.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.096+0000 7ff6f2200640 1 -- 172.21.15.151:0/1014276748 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6ec0c9220 con 0x7ff6ec0d1ec0 2024-09-16T15:50:58.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.097+0000 7ff6e8c00640 1 -- 172.21.15.151:0/1014276748 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7ff6e400ac50 con 0x7ff6ec0d1ec0 2024-09-16T15:50:58.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.098+0000 7ff6e8c00640 1 --2- 172.21.15.151:0/1014276748 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff6cc079ed0 0x7ff6cc07c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:58.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.098+0000 7ff6e8c00640 1 -- 172.21.15.151:0/1014276748 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 4999+0+0 (secure 0 0 0) 0x7ff6e4099590 con 0x7ff6ec0d1ec0 2024-09-16T15:50:58.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.098+0000 7ff6f0e00640 1 --2- 172.21.15.151:0/1014276748 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff6cc079ed0 0x7ff6cc07c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:58.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.099+0000 7ff6f0e00640 1 --2- 172.21.15.151:0/1014276748 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff6cc079ed0 0x7ff6cc07c390 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ff6e0034000 tx=0x7ff6e0004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:58.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.102+0000 7ff6e8c00640 1 -- 172.21.15.151:0/1014276748 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff6e4063660 con 0x7ff6ec0d1ec0 2024-09-16T15:50:58.304 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.302+0000 7ff6f2200640 1 -- 172.21.15.151:0/1014276748 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7ff6ec0cac70 con 0x7ff6ec0d1ec0 2024-09-16T15:50:58.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.347+0000 7ff6e8c00640 1 -- 172.21.15.151:0/1014276748 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v98) ==== 97+0+0 (secure 0 0 0) 0x7ff6e4062f60 con 0x7ff6ec0d1ec0 2024-09-16T15:50:58.350 INFO:teuthology.orchestra.run.smithi151.stderr: removed all blocklist entries 2024-09-16T15:50:58.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.352+0000 7ff6f2200640 1 -- 172.21.15.151:0/1014276748 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff6cc079ed0 msgr2=0x7ff6cc07c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:58.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.352+0000 7ff6f2200640 1 --2- 172.21.15.151:0/1014276748 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff6cc079ed0 0x7ff6cc07c390 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ff6e0034000 tx=0x7ff6e0004730 comp rx=0 tx=0).stop 2024-09-16T15:50:58.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.352+0000 7ff6f2200640 1 -- 172.21.15.151:0/1014276748 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6ec0d1ec0 msgr2=0x7ff6ec0c8270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:58.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.352+0000 7ff6f2200640 1 --2- 172.21.15.151:0/1014276748 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6ec0d1ec0 0x7ff6ec0c8270 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7ff6e400c770 tx=0x7ff6e400cc30 comp rx=0 tx=0).stop 2024-09-16T15:50:58.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.352+0000 7ff6f2200640 1 -- 172.21.15.151:0/1014276748 shutdown_connections 2024-09-16T15:50:58.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.352+0000 7ff6f2200640 1 --2- 172.21.15.151:0/1014276748 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff6cc079ed0 0x7ff6cc07c390 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:58.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.352+0000 7ff6f2200640 1 --2- 172.21.15.151:0/1014276748 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff6ec0d1ec0 0x7ff6ec0c8270 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:58.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.352+0000 7ff6f2200640 1 --2- 172.21.15.151:0/1014276748 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff6ec0b4d10 0x7ff6ec0c7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:58.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.352+0000 7ff6f2200640 1 --2- 172.21.15.151:0/1014276748 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff6ec0b28d0 0x7ff6ec154420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:58.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.352+0000 7ff6f2200640 1 -- 172.21.15.151:0/1014276748 >> 172.21.15.151:0/1014276748 conn(0x7ff6ec0d3de0 msgr2=0x7ff6ec06e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:58.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.352+0000 7ff6f2200640 1 -- 172.21.15.151:0/1014276748 shutdown_connections 2024-09-16T15:50:58.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.352+0000 7ff6f2200640 1 -- 172.21.15.151:0/1014276748 wait complete. 2024-09-16T15:50:58.373 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-16T15:50:58.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.532+0000 7f7169e00640 1 -- 172.21.15.151:0/2210288810 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71641445d0 msgr2=0x7f71640cebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:58.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.532+0000 7f7169e00640 1 --2- 172.21.15.151:0/2210288810 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71641445d0 0x7f71640cebf0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f7154005fd0 tx=0x7f715402a670 comp rx=0 tx=0).stop 2024-09-16T15:50:58.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.532+0000 7f7169e00640 1 -- 172.21.15.151:0/2210288810 shutdown_connections 2024-09-16T15:50:58.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.532+0000 7f7169e00640 1 --2- 172.21.15.151:0/2210288810 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f71640ccfd0 0x7f71640cf130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:58.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.532+0000 7f7169e00640 1 --2- 172.21.15.151:0/2210288810 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71641445d0 0x7f71640cebf0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:58.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.532+0000 7f7169e00640 1 -- 172.21.15.151:0/2210288810 >> 172.21.15.151:0/2210288810 conn(0x7f71640b1950 msgr2=0x7f71640b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:58.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.532+0000 7f7169e00640 1 -- 172.21.15.151:0/2210288810 shutdown_connections 2024-09-16T15:50:58.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.532+0000 7f7169e00640 1 -- 172.21.15.151:0/2210288810 wait complete. 2024-09-16T15:50:58.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.533+0000 7f7169e00640 1 Processor -- start 2024-09-16T15:50:58.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.533+0000 7f7169e00640 1 -- start start 2024-09-16T15:50:58.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.534+0000 7f7169e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71640b4ca0 0x7f716413b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:58.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.534+0000 7f7169e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71640ccfd0 0x7f71640c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:58.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.534+0000 7f7169e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f71641445d0 0x7f71640cddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:58.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.534+0000 7f7169e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f716415f080 con 0x7f71640ccfd0 2024-09-16T15:50:58.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.534+0000 7f7169e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7164073bc0 con 0x7f71640b4ca0 2024-09-16T15:50:58.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.534+0000 7f7169e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f716415f200 con 0x7f71641445d0 2024-09-16T15:50:58.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.535+0000 7f7162000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71640ccfd0 0x7f71640c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:58.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.535+0000 7f7162000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71640ccfd0 0x7f71640c9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:54336/0 (socket says 172.21.15.151:54336) 2024-09-16T15:50:58.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.535+0000 7f7162000640 1 -- 172.21.15.151:0/337821975 learned_addr learned my addr 172.21.15.151:0/337821975 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:58.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.535+0000 7f7163400640 1 --2- 172.21.15.151:0/337821975 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f71641445d0 0x7f71640cddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:58.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.535+0000 7f7162a00640 1 --2- 172.21.15.151:0/337821975 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71640b4ca0 0x7f716413b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:58.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.535+0000 7f7162000640 1 -- 172.21.15.151:0/337821975 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f71641445d0 msgr2=0x7f71640cddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:58.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.535+0000 7f7162000640 1 --2- 172.21.15.151:0/337821975 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f71641445d0 0x7f71640cddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:58.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.535+0000 7f7162000640 1 -- 172.21.15.151:0/337821975 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71640b4ca0 msgr2=0x7f716413b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:58.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.535+0000 7f7162000640 1 --2- 172.21.15.151:0/337821975 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71640b4ca0 0x7f716413b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:58.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.535+0000 7f7162000640 1 -- 172.21.15.151:0/337821975 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f716406ff80 con 0x7f71640ccfd0 2024-09-16T15:50:58.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.535+0000 7f7162000640 1 --2- 172.21.15.151:0/337821975 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71640ccfd0 0x7f71640c9d00 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f715000d980 tx=0x7f715000de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:58.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.536+0000 7f715b400640 1 -- 172.21.15.151:0/337821975 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7150017070 con 0x7f71640ccfd0 2024-09-16T15:50:58.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.536+0000 7f715b400640 1 -- 172.21.15.151:0/337821975 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7150009d20 con 0x7f71640ccfd0 2024-09-16T15:50:58.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.536+0000 7f7169e00640 1 -- 172.21.15.151:0/337821975 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71640c9220 con 0x7f71640ccfd0 2024-09-16T15:50:58.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.536+0000 7f715b400640 1 -- 172.21.15.151:0/337821975 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71500106c0 con 0x7f71640ccfd0 2024-09-16T15:50:58.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.536+0000 7f7169e00640 1 -- 172.21.15.151:0/337821975 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71640ca240 con 0x7f71640ccfd0 2024-09-16T15:50:58.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.537+0000 7f7169e00640 1 -- 172.21.15.151:0/337821975 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f716415f6b0 con 0x7f71640ccfd0 2024-09-16T15:50:58.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.538+0000 7f715b400640 1 -- 172.21.15.151:0/337821975 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7f7150010860 con 0x7f71640ccfd0 2024-09-16T15:50:58.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.539+0000 7f715b400640 1 --2- 172.21.15.151:0/337821975 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7144079ed0 0x7f714407c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:58.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.539+0000 7f715b400640 1 -- 172.21.15.151:0/337821975 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 4913+0+0 (secure 0 0 0) 0x7f71500999e0 con 0x7f71640ccfd0 2024-09-16T15:50:58.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.539+0000 7f7162a00640 1 --2- 172.21.15.151:0/337821975 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7144079ed0 0x7f714407c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:58.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.540+0000 7f7162a00640 1 --2- 172.21.15.151:0/337821975 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7144079ed0 0x7f714407c390 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f7154033000 tx=0x7f7154004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:58.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.543+0000 7f715b400640 1 -- 172.21.15.151:0/337821975 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7150063460 con 0x7f71640ccfd0 2024-09-16T15:50:58.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.794+0000 7f7169e00640 1 -- 172.21.15.151:0/337821975 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f7164070a20 con 0x7f71640ccfd0 2024-09-16T15:50:58.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.795+0000 7f715b400640 1 -- 172.21.15.151:0/337821975 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) ==== 80+0+5304 (secure 0 0 0) 0x7f7150067e90 con 0x7f71640ccfd0 2024-09-16T15:50:58.797 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:50:58.797 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-16T15:50:58.797 INFO:teuthology.orchestra.run.smithi151.stdout: "auth_dump": [ 2024-09-16T15:50:58.797 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:50:58.797 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "mds.a", 2024-09-16T15:50:58.797 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDsUOhmJVK6AhAANfmTPS6kKn41LSD/MuyW9g==", 2024-09-16T15:50:58.797 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:50:58.797 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:50:58.797 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow *", 2024-09-16T15:50:58.797 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow *", 2024-09-16T15:50:58.797 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:50:58.798 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:50:58.798 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:50:58.798 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:50:58.798 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "mds.b", 2024-09-16T15:50:58.798 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDsUOhmX1R3FhAAUQgUsE/6wUyokXcZuBuIWg==", 2024-09-16T15:50:58.798 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:50:58.798 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:50:58.798 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow *", 2024-09-16T15:50:58.798 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow *", 2024-09-16T15:50:58.798 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:50:58.798 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:50:58.798 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:50:58.798 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:50:58.798 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "mds.c", 2024-09-16T15:50:58.798 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDsUOhmYGCKDBAAY2EFNmiFNf1x2so9wxp4YQ==", 2024-09-16T15:50:58.798 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:50:58.799 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:50:58.799 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow *", 2024-09-16T15:50:58.799 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow *", 2024-09-16T15:50:58.799 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:50:58.799 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:50:58.799 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:50:58.799 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:50:58.799 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "mds.d", 2024-09-16T15:50:58.799 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDsUOhmfCk4IBAAx12Gw2QudQ2nrPb8VlE0SA==", 2024-09-16T15:50:58.799 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:50:58.799 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:50:58.799 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow *", 2024-09-16T15:50:58.799 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow *", 2024-09-16T15:50:58.799 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:50:58.800 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:50:58.800 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:50:58.800 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:50:58.800 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.0", 2024-09-16T15:50:58.800 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDvUOhmKji5NxAAn/E3YQR7jeMnBTLBXc1G6A==", 2024-09-16T15:50:58.800 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:50:58.800 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:50:58.800 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:50:58.800 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:50:58.800 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:50:58.800 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:50:58.800 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:50:58.800 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.1", 2024-09-16T15:50:58.800 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDzUOhmmBrNBhAA0Uwh4Fam2bmKKvz5jAQBWw==", 2024-09-16T15:50:58.800 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:50:58.801 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:50:58.801 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:50:58.801 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:50:58.801 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:50:58.801 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:50:58.801 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:50:58.801 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.2", 2024-09-16T15:50:58.801 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQD2UOhmOV6GERAAjD+PQLTZrCFR1669UEEIAQ==", 2024-09-16T15:50:58.801 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:50:58.801 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:50:58.801 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:50:58.801 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:50:58.801 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:50:58.801 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:50:58.802 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:50:58.802 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.3", 2024-09-16T15:50:58.802 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQD5UOhmCRrdGRAAQC0bqWX6N1jniVGEQ2aAvg==", 2024-09-16T15:50:58.802 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:50:58.802 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:50:58.802 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:50:58.802 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:50:58.802 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:50:58.802 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:50:58.802 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:50:58.802 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.4", 2024-09-16T15:50:58.802 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQD/UOhmXI/nGRAA7t6XveKvrWgqy6rHuMQSZQ==", 2024-09-16T15:50:58.802 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:50:58.802 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:50:58.802 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:50:58.803 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:50:58.803 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:50:58.803 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:50:58.803 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:50:58.803 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.5", 2024-09-16T15:50:58.803 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQACUehmKM//IxAAsoFpiiTXNs//v3YXeUf/Iw==", 2024-09-16T15:50:58.803 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:50:58.803 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:50:58.803 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:50:58.803 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:50:58.803 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:50:58.803 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:50:58.803 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:50:58.803 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.6", 2024-09-16T15:50:58.803 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQAFUehmlo/FLRAAp5mvUSWCaBbm85QyTZY0eA==", 2024-09-16T15:50:58.804 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:50:58.804 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:50:58.804 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:50:58.804 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:50:58.804 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:50:58.804 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:50:58.804 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:50:58.804 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.7", 2024-09-16T15:50:58.804 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQAIUehmR32JNxAA27antOIKot1nScQbLZDf5g==", 2024-09-16T15:50:58.804 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:50:58.804 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:50:58.804 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:50:58.804 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:50:58.804 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:50:58.804 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:50:58.805 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:50:58.805 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "client.0", 2024-09-16T15:50:58.805 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDsUOhmf5PBKBAAKE4Y4zbA4h07IBNjPkyptw==", 2024-09-16T15:50:58.805 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:50:58.805 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:50:58.805 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow r", 2024-09-16T15:50:58.805 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-16T15:50:58.805 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:50:58.805 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:50:58.805 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:50:58.805 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "client.1", 2024-09-16T15:50:58.805 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDsUOhm15emNRAAwhK7QmA2c9IYGn04ERvY0g==", 2024-09-16T15:50:58.805 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:50:58.805 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:50:58.805 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow r", 2024-09-16T15:50:58.806 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow rw", 2024-09-16T15:50:58.806 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow rwx" 2024-09-16T15:50:58.806 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:50:58.806 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:50:58.806 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:50:58.806 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "client.2", 2024-09-16T15:50:58.806 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDsUOhm6zUyLxAAkzeiIQTuNZkMM+Oit9rC5A==", 2024-09-16T15:50:58.806 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:50:58.806 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:50:58.806 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow r", 2024-09-16T15:50:58.806 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow rw", 2024-09-16T15:50:58.806 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow rwx" 2024-09-16T15:50:58.806 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:50:58.806 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:50:58.806 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:50:58.807 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "client.3", 2024-09-16T15:50:58.807 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDtUOhmD82EABAAeDf82/hPxpPfAUk8oj5h/Q==", 2024-09-16T15:50:58.807 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:50:58.807 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:50:58.807 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow r", 2024-09-16T15:50:58.807 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow rw", 2024-09-16T15:50:58.807 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow rwx" 2024-09-16T15:50:58.807 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:50:58.807 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:50:58.807 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:50:58.807 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "client.admin", 2024-09-16T15:50:58.807 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDrUOhmQFmVDRAA8NKysbAe475xvg4CPAcFyw==", 2024-09-16T15:50:58.807 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:50:58.807 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow *", 2024-09-16T15:50:58.807 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow *", 2024-09-16T15:50:58.808 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow *", 2024-09-16T15:50:58.808 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:50:58.808 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:50:58.808 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:50:58.808 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:50:58.808 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "mgr.x", 2024-09-16T15:50:58.808 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDrUOhmloW3NxAA8UoLUfrw3wf1nzKbmtK/jw==", 2024-09-16T15:50:58.808 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:50:58.808 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow *", 2024-09-16T15:50:58.808 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile mgr", 2024-09-16T15:50:58.808 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:50:58.808 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:50:58.808 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:50:58.808 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:50:58.808 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "mgr.y", 2024-09-16T15:50:58.809 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDrUOhm1uZOLxAATgjA79vYf0psqQVo03sdDA==", 2024-09-16T15:50:58.809 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:50:58.809 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow *", 2024-09-16T15:50:58.809 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile mgr", 2024-09-16T15:50:58.809 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:50:58.809 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:50:58.809 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:50:58.809 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-09-16T15:50:58.809 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-16T15:50:58.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.797+0000 7f7169e00640 1 -- 172.21.15.151:0/337821975 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7144079ed0 msgr2=0x7f714407c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:58.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.797+0000 7f7169e00640 1 --2- 172.21.15.151:0/337821975 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7144079ed0 0x7f714407c390 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f7154033000 tx=0x7f7154004730 comp rx=0 tx=0).stop 2024-09-16T15:50:58.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.797+0000 7f7169e00640 1 -- 172.21.15.151:0/337821975 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71640ccfd0 msgr2=0x7f71640c9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:58.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.797+0000 7f7169e00640 1 --2- 172.21.15.151:0/337821975 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71640ccfd0 0x7f71640c9d00 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f715000d980 tx=0x7f715000de40 comp rx=0 tx=0).stop 2024-09-16T15:50:58.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.797+0000 7f7169e00640 1 -- 172.21.15.151:0/337821975 shutdown_connections 2024-09-16T15:50:58.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.797+0000 7f7169e00640 1 --2- 172.21.15.151:0/337821975 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7144079ed0 0x7f714407c390 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:58.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.797+0000 7f7169e00640 1 --2- 172.21.15.151:0/337821975 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f71641445d0 0x7f71640cddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:58.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.797+0000 7f7169e00640 1 --2- 172.21.15.151:0/337821975 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71640ccfd0 0x7f71640c9d00 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:58.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.797+0000 7f7169e00640 1 --2- 172.21.15.151:0/337821975 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71640b4ca0 0x7f716413b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:58.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.797+0000 7f7169e00640 1 -- 172.21.15.151:0/337821975 >> 172.21.15.151:0/337821975 conn(0x7f71640b1950 msgr2=0x7f71640af880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:58.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.797+0000 7f7169e00640 1 -- 172.21.15.151:0/337821975 shutdown_connections 2024-09-16T15:50:58.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.797+0000 7f7169e00640 1 -- 172.21.15.151:0/337821975 wait complete. 2024-09-16T15:50:58.815 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-16T15:50:58.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.972+0000 7f8ea9200640 1 -- 172.21.15.151:0/534152260 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ea40ad880 msgr2=0x7f8ea40adcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:58.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.972+0000 7f8ea9200640 1 --2- 172.21.15.151:0/534152260 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ea40ad880 0x7f8ea40adcd0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c009830 tx=0x7f8e8c02a510 comp rx=0 tx=0).stop 2024-09-16T15:50:58.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.972+0000 7f8ea9200640 1 -- 172.21.15.151:0/534152260 shutdown_connections 2024-09-16T15:50:58.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.972+0000 7f8ea9200640 1 --2- 172.21.15.151:0/534152260 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8ea40cd5f0 0x7f8ea40cda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:58.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.972+0000 7f8ea9200640 1 --2- 172.21.15.151:0/534152260 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ea40ad880 0x7f8ea40adcd0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:58.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.972+0000 7f8ea9200640 1 -- 172.21.15.151:0/534152260 >> 172.21.15.151:0/534152260 conn(0x7f8ea40cfa20 msgr2=0x7f8ea40cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:58.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.972+0000 7f8ea9200640 1 -- 172.21.15.151:0/534152260 shutdown_connections 2024-09-16T15:50:58.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.972+0000 7f8ea9200640 1 -- 172.21.15.151:0/534152260 wait complete. 2024-09-16T15:50:58.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.974+0000 7f8ea9200640 1 Processor -- start 2024-09-16T15:50:58.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.974+0000 7f8ea9200640 1 -- start start 2024-09-16T15:50:58.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.975+0000 7f8ea9200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8ea406fb90 0x7f8ea4154330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:58.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.975+0000 7f8ea9200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ea40ad880 0x7f8ea41f50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:58.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.975+0000 7f8ea9200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ea40cd5f0 0x7f8ea406e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:58.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.975+0000 7f8ea9200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea415edf0 con 0x7f8ea40ad880 2024-09-16T15:50:58.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.975+0000 7f8ea9200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea415ec70 con 0x7f8ea40cd5f0 2024-09-16T15:50:58.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.975+0000 7f8ea9200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f8ea415ef70 con 0x7f8ea406fb90 2024-09-16T15:50:58.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.975+0000 7f8e9be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ea40ad880 0x7f8ea41f50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:58.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.975+0000 7f8e9be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ea40ad880 0x7f8ea41f50f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:54352/0 (socket says 172.21.15.151:54352) 2024-09-16T15:50:58.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.975+0000 7f8e9be00640 1 -- 172.21.15.151:0/15664860 learned_addr learned my addr 172.21.15.151:0/15664860 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:58.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.975+0000 7f8ea3400640 1 --2- 172.21.15.151:0/15664860 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8ea406fb90 0x7f8ea4154330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:58.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.975+0000 7f8ea3e00640 1 --2- 172.21.15.151:0/15664860 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ea40cd5f0 0x7f8ea406e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:58.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.976+0000 7f8e9be00640 1 -- 172.21.15.151:0/15664860 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8ea406fb90 msgr2=0x7f8ea4154330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:58.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.976+0000 7f8e9be00640 1 --2- 172.21.15.151:0/15664860 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8ea406fb90 0x7f8ea4154330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:58.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.976+0000 7f8e9be00640 1 -- 172.21.15.151:0/15664860 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ea40cd5f0 msgr2=0x7f8ea406e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:58.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.976+0000 7f8e9be00640 1 --2- 172.21.15.151:0/15664860 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ea40cd5f0 0x7f8ea406e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:58.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.976+0000 7f8e9be00640 1 -- 172.21.15.151:0/15664860 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ea4147750 con 0x7f8ea40ad880 2024-09-16T15:50:58.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.976+0000 7f8ea3400640 1 --2- 172.21.15.151:0/15664860 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8ea406fb90 0x7f8ea4154330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:50:58.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.976+0000 7f8ea3e00640 1 --2- 172.21.15.151:0/15664860 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ea40cd5f0 0x7f8ea406e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:50:58.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.976+0000 7f8e9be00640 1 --2- 172.21.15.151:0/15664860 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ea40ad880 0x7f8ea41f50f0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c009950 tx=0x7f8e8c032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:58.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.976+0000 7f8e9b400640 1 -- 172.21.15.151:0/15664860 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e8c040070 con 0x7f8ea40ad880 2024-09-16T15:50:58.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.977+0000 7f8e9b400640 1 -- 172.21.15.151:0/15664860 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8e8c005310 con 0x7f8ea40ad880 2024-09-16T15:50:58.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.977+0000 7f8ea9200640 1 -- 172.21.15.151:0/15664860 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ea40b3fc0 con 0x7f8ea40ad880 2024-09-16T15:50:58.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.977+0000 7f8e9b400640 1 -- 172.21.15.151:0/15664860 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e8c005c90 con 0x7f8ea40ad880 2024-09-16T15:50:58.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.977+0000 7f8ea9200640 1 -- 172.21.15.151:0/15664860 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ea40b4580 con 0x7f8ea40ad880 2024-09-16T15:50:58.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.978+0000 7f8ea9200640 1 -- 172.21.15.151:0/15664860 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ea4160160 con 0x7f8ea40ad880 2024-09-16T15:50:58.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.979+0000 7f8e9b400640 1 -- 172.21.15.151:0/15664860 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7f8e8c039040 con 0x7f8ea40ad880 2024-09-16T15:50:58.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.980+0000 7f8e9b400640 1 --2- 172.21.15.151:0/15664860 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8e80079f00 0x7f8e8007c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:58.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.980+0000 7f8e9b400640 1 -- 172.21.15.151:0/15664860 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 4913+0+0 (secure 0 0 0) 0x7f8e8c0ba580 con 0x7f8ea40ad880 2024-09-16T15:50:58.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.980+0000 7f8ea3400640 1 --2- 172.21.15.151:0/15664860 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8e80079f00 0x7f8e8007c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:58.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.981+0000 7f8ea3400640 1 --2- 172.21.15.151:0/15664860 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8e80079f00 0x7f8e8007c3c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f8e94007f20 tx=0x7f8e94007eb0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:58.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:58.988+0000 7f8e9b400640 1 -- 172.21.15.151:0/15664860 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8ea4160160 con 0x7f8ea40ad880 2024-09-16T15:50:59.174 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:59.172+0000 7f8ea9200640 1 -- 172.21.15.151:0/15664860 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f8ea40adcd0 con 0x7f8ea40ad880 2024-09-16T15:50:59.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:59.173+0000 7f8e9b400640 1 -- 172.21.15.151:0/15664860 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v100) ==== 84+0+65 (secure 0 0 0) 0x7f8ea4147750 con 0x7f8ea40ad880 2024-09-16T15:50:59.175 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:50:59.175 INFO:teuthology.orchestra.run.smithi151.stdout:[ 2024-09-16T15:50:59.175 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:50:59.175 INFO:teuthology.orchestra.run.smithi151.stdout: "poolnum": 1, 2024-09-16T15:50:59.175 INFO:teuthology.orchestra.run.smithi151.stdout: "poolname": ".mgr" 2024-09-16T15:50:59.175 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:50:59.175 INFO:teuthology.orchestra.run.smithi151.stdout:] 2024-09-16T15:50:59.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:59.175+0000 7f8ea9200640 1 -- 172.21.15.151:0/15664860 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8e80079f00 msgr2=0x7f8e8007c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:59.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:59.175+0000 7f8ea9200640 1 --2- 172.21.15.151:0/15664860 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8e80079f00 0x7f8e8007c3c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f8e94007f20 tx=0x7f8e94007eb0 comp rx=0 tx=0).stop 2024-09-16T15:50:59.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:59.175+0000 7f8ea9200640 1 -- 172.21.15.151:0/15664860 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ea40ad880 msgr2=0x7f8ea41f50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:59.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:59.175+0000 7f8ea9200640 1 --2- 172.21.15.151:0/15664860 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ea40ad880 0x7f8ea41f50f0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c009950 tx=0x7f8e8c032c10 comp rx=0 tx=0).stop 2024-09-16T15:50:59.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:59.175+0000 7f8ea9200640 1 -- 172.21.15.151:0/15664860 shutdown_connections 2024-09-16T15:50:59.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:59.175+0000 7f8ea9200640 1 --2- 172.21.15.151:0/15664860 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8e80079f00 0x7f8e8007c3c0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:59.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:59.175+0000 7f8ea9200640 1 --2- 172.21.15.151:0/15664860 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8ea40cd5f0 0x7f8ea406e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:59.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:59.175+0000 7f8ea9200640 1 --2- 172.21.15.151:0/15664860 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ea40ad880 0x7f8ea41f50f0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:59.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:59.175+0000 7f8ea9200640 1 --2- 172.21.15.151:0/15664860 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8ea406fb90 0x7f8ea4154330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:59.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:59.175+0000 7f8ea9200640 1 -- 172.21.15.151:0/15664860 >> 172.21.15.151:0/15664860 conn(0x7f8ea40cfa20 msgr2=0x7f8ea4070100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:59.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:59.175+0000 7f8ea9200640 1 -- 172.21.15.151:0/15664860 shutdown_connections 2024-09-16T15:50:59.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:59.175+0000 7f8ea9200640 1 -- 172.21.15.151:0/15664860 wait complete. 2024-09-16T15:50:59.193 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-16T15:50:59.193 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-16T15:50:59.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:59.357+0000 7f88c2800640 1 -- 172.21.15.151:0/2153241856 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f88bc15e710 msgr2=0x7f88bc0d4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:59.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:59.357+0000 7f88c2800640 1 --2- 172.21.15.151:0/2153241856 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f88bc15e710 0x7f88bc0d4a20 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f88b0005fd0 tx=0x7f88b002a670 comp rx=0 tx=0).stop 2024-09-16T15:50:59.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:59.358+0000 7f88c2800640 1 -- 172.21.15.151:0/2153241856 shutdown_connections 2024-09-16T15:50:59.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:59.358+0000 7f88c2800640 1 --2- 172.21.15.151:0/2153241856 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f88bc15e710 0x7f88bc0d4a20 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:59.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:59.358+0000 7f88c2800640 1 -- 172.21.15.151:0/2153241856 >> 172.21.15.151:0/2153241856 conn(0x7f88bc0d3f00 msgr2=0x7f88bc0d4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:50:59.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:59.358+0000 7f88c2800640 1 -- 172.21.15.151:0/2153241856 shutdown_connections 2024-09-16T15:50:59.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:59.358+0000 7f88c2800640 1 -- 172.21.15.151:0/2153241856 wait complete. 2024-09-16T15:50:59.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:59.359+0000 7f88c2800640 1 Processor -- start 2024-09-16T15:50:59.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:59.359+0000 7f88c2800640 1 -- start start 2024-09-16T15:50:59.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:59.360+0000 7f88c2800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88bc0d0fe0 0x7f88bc14dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:59.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:59.360+0000 7f88c2800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f88bc15e710 0x7f88bc1510a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:59.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:59.360+0000 7f88c2800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f88bc15eaf0 0x7f88bc1f7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:59.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:59.360+0000 7f88c2800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f88bc0d21e0 con 0x7f88bc15eaf0 2024-09-16T15:50:59.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:59.360+0000 7f88c2800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f88bc15f380 con 0x7f88bc0d0fe0 2024-09-16T15:50:59.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:59.360+0000 7f88c2800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f88bc0d2360 con 0x7f88bc15e710 2024-09-16T15:50:59.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:59.361+0000 7f88c1e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f88bc15eaf0 0x7f88bc1f7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:59.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:59.361+0000 7f88c1e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f88bc15eaf0 0x7f88bc1f7700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:54374/0 (socket says 172.21.15.151:54374) 2024-09-16T15:50:59.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:59.361+0000 7f88c1e00640 1 -- 172.21.15.151:0/3888358711 learned_addr learned my addr 172.21.15.151:0/3888358711 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:50:59.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:59.361+0000 7f88c1400640 1 --2- 172.21.15.151:0/3888358711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88bc0d0fe0 0x7f88bc14dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:59.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:59.361+0000 7f88c0a00640 1 --2- 172.21.15.151:0/3888358711 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f88bc15e710 0x7f88bc1510a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:59.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:59.361+0000 7f88c1e00640 1 -- 172.21.15.151:0/3888358711 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f88bc15e710 msgr2=0x7f88bc1510a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:59.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:59.361+0000 7f88c1e00640 1 --2- 172.21.15.151:0/3888358711 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f88bc15e710 0x7f88bc1510a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:59.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:59.361+0000 7f88c1e00640 1 -- 172.21.15.151:0/3888358711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88bc0d0fe0 msgr2=0x7f88bc14dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:50:59.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:59.361+0000 7f88c1e00640 1 --2- 172.21.15.151:0/3888358711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88bc0d0fe0 0x7f88bc14dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:50:59.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:59.361+0000 7f88c1e00640 1 -- 172.21.15.151:0/3888358711 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88bc1f7c40 con 0x7f88bc15eaf0 2024-09-16T15:50:59.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:59.361+0000 7f88c1e00640 1 --2- 172.21.15.151:0/3888358711 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f88bc15eaf0 0x7f88bc1f7700 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f88b400c730 tx=0x7f88b400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:59.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:59.362+0000 7f88b9600640 1 -- 172.21.15.151:0/3888358711 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b4003b70 con 0x7f88bc15eaf0 2024-09-16T15:50:59.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:59.362+0000 7f88b9600640 1 -- 172.21.15.151:0/3888358711 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f88b4003d10 con 0x7f88bc15eaf0 2024-09-16T15:50:59.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:59.362+0000 7f88b9600640 1 -- 172.21.15.151:0/3888358711 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b400aab0 con 0x7f88bc15eaf0 2024-09-16T15:50:59.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:59.362+0000 7f88c2800640 1 -- 172.21.15.151:0/3888358711 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88bc0b6dd0 con 0x7f88bc15eaf0 2024-09-16T15:50:59.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:59.362+0000 7f88c2800640 1 -- 172.21.15.151:0/3888358711 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88bc0b7070 con 0x7f88bc15eaf0 2024-09-16T15:50:59.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:59.363+0000 7f88c2800640 1 -- 172.21.15.151:0/3888358711 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88bc0d2810 con 0x7f88bc15eaf0 2024-09-16T15:50:59.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:59.364+0000 7f88b9600640 1 -- 172.21.15.151:0/3888358711 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7f88b400ac50 con 0x7f88bc15eaf0 2024-09-16T15:50:59.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:59.365+0000 7f88b9600640 1 --2- 172.21.15.151:0/3888358711 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f889c079ed0 0x7f889c07c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:50:59.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:59.365+0000 7f88b9600640 1 -- 172.21.15.151:0/3888358711 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 4913+0+0 (secure 0 0 0) 0x7f88b4099380 con 0x7f88bc15eaf0 2024-09-16T15:50:59.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:59.365+0000 7f88c1400640 1 --2- 172.21.15.151:0/3888358711 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f889c079ed0 0x7f889c07c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:50:59.368 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:59.366+0000 7f88c1400640 1 --2- 172.21.15.151:0/3888358711 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f889c079ed0 0x7f889c07c390 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f88b0033000 tx=0x7f88b0004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:50:59.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:59.369+0000 7f88b9600640 1 -- 172.21.15.151:0/3888358711 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f88bc1f7c40 con 0x7f88bc15eaf0 2024-09-16T15:50:59.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:50:59.556+0000 7f88c2800640 1 -- 172.21.15.151:0/3888358711 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f88bc15d160 con 0x7f88bc15eaf0 2024-09-16T15:51:00.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:00.354+0000 7f88b9600640 1 -- 172.21.15.151:0/3888358711 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v101) ==== 138+0+0 (secure 0 0 0) 0x7f88b4062e00 con 0x7f88bc15eaf0 2024-09-16T15:51:00.356 INFO:teuthology.orchestra.run.smithi151.stderr:pool 'cephfs_metadata' created 2024-09-16T15:51:00.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:00.357+0000 7f88c2800640 1 -- 172.21.15.151:0/3888358711 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f889c079ed0 msgr2=0x7f889c07c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:00.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:00.357+0000 7f88c2800640 1 --2- 172.21.15.151:0/3888358711 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f889c079ed0 0x7f889c07c390 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f88b0033000 tx=0x7f88b0004730 comp rx=0 tx=0).stop 2024-09-16T15:51:00.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:00.357+0000 7f88c2800640 1 -- 172.21.15.151:0/3888358711 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f88bc15eaf0 msgr2=0x7f88bc1f7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:00.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:00.357+0000 7f88c2800640 1 --2- 172.21.15.151:0/3888358711 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f88bc15eaf0 0x7f88bc1f7700 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f88b400c730 tx=0x7f88b400cbf0 comp rx=0 tx=0).stop 2024-09-16T15:51:00.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:00.357+0000 7f88c2800640 1 -- 172.21.15.151:0/3888358711 shutdown_connections 2024-09-16T15:51:00.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:00.357+0000 7f88c2800640 1 --2- 172.21.15.151:0/3888358711 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f889c079ed0 0x7f889c07c390 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:00.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:00.357+0000 7f88c2800640 1 --2- 172.21.15.151:0/3888358711 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f88bc15eaf0 0x7f88bc1f7700 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:00.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:00.357+0000 7f88c2800640 1 --2- 172.21.15.151:0/3888358711 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f88bc15e710 0x7f88bc1510a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:00.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:00.357+0000 7f88c2800640 1 --2- 172.21.15.151:0/3888358711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f88bc0d0fe0 0x7f88bc14dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:00.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:00.357+0000 7f88c2800640 1 -- 172.21.15.151:0/3888358711 >> 172.21.15.151:0/3888358711 conn(0x7f88bc0d3f00 msgr2=0x7f88bc15d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:00.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:00.357+0000 7f88c2800640 1 -- 172.21.15.151:0/3888358711 shutdown_connections 2024-09-16T15:51:00.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:00.357+0000 7f88c2800640 1 -- 172.21.15.151:0/3888358711 wait complete. 2024-09-16T15:51:00.379 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-16T15:51:00.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:00.566+0000 7f1163600640 1 -- 172.21.15.151:0/1115492461 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1140005210 msgr2=0x7f11400079a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:00.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:00.566+0000 7f1163600640 1 --2- 172.21.15.151:0/1115492461 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1140005210 0x7f11400079a0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f115c072730 tx=0x7f1140005830 comp rx=0 tx=0).stop 2024-09-16T15:51:00.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:00.567+0000 7f1163600640 1 -- 172.21.15.151:0/1115492461 shutdown_connections 2024-09-16T15:51:00.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:00.567+0000 7f1163600640 1 --2- 172.21.15.151:0/1115492461 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1140005210 0x7f11400079a0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:00.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:00.567+0000 7f1163600640 1 -- 172.21.15.151:0/1115492461 >> 172.21.15.151:0/1115492461 conn(0x7f115c0b1840 msgr2=0x7f115c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:00.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:00.567+0000 7f1163600640 1 -- 172.21.15.151:0/1115492461 shutdown_connections 2024-09-16T15:51:00.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:00.567+0000 7f1163600640 1 -- 172.21.15.151:0/1115492461 wait complete. 2024-09-16T15:51:00.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:00.568+0000 7f1163600640 1 Processor -- start 2024-09-16T15:51:00.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:00.568+0000 7f1163600640 1 -- start start 2024-09-16T15:51:00.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:00.569+0000 7f1163600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1140005210 0x7f115c073b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:00.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:00.569+0000 7f1163600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f115c16a000 0x7f115c16a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:00.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:00.569+0000 7f1163600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f115c1535d0 0x7f115c153a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:00.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:00.569+0000 7f1163600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f115c0b3e80 con 0x7f115c16a000 2024-09-16T15:51:00.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:00.569+0000 7f1163600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f115c0b3d00 con 0x7f115c1535d0 2024-09-16T15:51:00.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:00.569+0000 7f1163600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f115c0b4000 con 0x7f1140005210 2024-09-16T15:51:00.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:00.570+0000 7f1153e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f115c16a000 0x7f115c16a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:00.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:00.570+0000 7f1153e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f115c16a000 0x7f115c16a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:54396/0 (socket says 172.21.15.151:54396) 2024-09-16T15:51:00.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:00.570+0000 7f1153e00640 1 -- 172.21.15.151:0/3531387001 learned_addr learned my addr 172.21.15.151:0/3531387001 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:51:00.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:00.570+0000 7f1161600640 1 --2- 172.21.15.151:0/3531387001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f115c1535d0 0x7f115c153a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:00.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:00.570+0000 7f1160c00640 1 --2- 172.21.15.151:0/3531387001 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1140005210 0x7f115c073b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:00.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:00.570+0000 7f1153e00640 1 -- 172.21.15.151:0/3531387001 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1140005210 msgr2=0x7f115c073b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:00.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:00.570+0000 7f1153e00640 1 --2- 172.21.15.151:0/3531387001 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1140005210 0x7f115c073b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:00.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:00.570+0000 7f1153e00640 1 -- 172.21.15.151:0/3531387001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f115c1535d0 msgr2=0x7f115c153a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:00.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:00.570+0000 7f1153e00640 1 --2- 172.21.15.151:0/3531387001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f115c1535d0 0x7f115c153a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:00.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:00.570+0000 7f1153e00640 1 -- 172.21.15.151:0/3531387001 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f115c201590 con 0x7f115c16a000 2024-09-16T15:51:00.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:00.570+0000 7f1160c00640 1 --2- 172.21.15.151:0/3531387001 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1140005210 0x7f115c073b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:51:00.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:00.570+0000 7f1153e00640 1 --2- 172.21.15.151:0/3531387001 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f115c16a000 0x7f115c16a480 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f1144003de0 tx=0x7f1144001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:00.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:00.570+0000 7f1159600640 1 -- 172.21.15.151:0/3531387001 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1144001d40 con 0x7f115c16a000 2024-09-16T15:51:00.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:00.571+0000 7f1159600640 1 -- 172.21.15.151:0/3531387001 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1144016020 con 0x7f115c16a000 2024-09-16T15:51:00.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:00.571+0000 7f1159600640 1 -- 172.21.15.151:0/3531387001 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1144011880 con 0x7f115c16a000 2024-09-16T15:51:00.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:00.571+0000 7f1163600640 1 -- 172.21.15.151:0/3531387001 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f115c201870 con 0x7f115c16a000 2024-09-16T15:51:00.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:00.571+0000 7f1163600640 1 -- 172.21.15.151:0/3531387001 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f115c201e30 con 0x7f115c16a000 2024-09-16T15:51:00.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:00.572+0000 7f1163600640 1 -- 172.21.15.151:0/3531387001 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f115c0b1230 con 0x7f115c16a000 2024-09-16T15:51:00.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:00.573+0000 7f1159600640 1 -- 172.21.15.151:0/3531387001 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7f1144011a20 con 0x7f115c16a000 2024-09-16T15:51:00.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:00.574+0000 7f1159600640 1 --2- 172.21.15.151:0/3531387001 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1138079f00 0x7f113807c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:00.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:00.574+0000 7f1159600640 1 -- 172.21.15.151:0/3531387001 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5283+0+0 (secure 0 0 0) 0x7f114409a1c0 con 0x7f115c16a000 2024-09-16T15:51:00.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:00.574+0000 7f1160c00640 1 --2- 172.21.15.151:0/3531387001 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1138079f00 0x7f113807c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:00.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:00.575+0000 7f1160c00640 1 --2- 172.21.15.151:0/3531387001 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1138079f00 0x7f113807c3c0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f114c001860 tx=0x7f114c001c70 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:00.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:00.579+0000 7f1159600640 1 -- 172.21.15.151:0/3531387001 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f115c0b1230 con 0x7f115c16a000 2024-09-16T15:51:00.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:00.764+0000 7f1163600640 1 -- 172.21.15.151:0/3531387001 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f115c06f860 con 0x7f115c16a000 2024-09-16T15:51:01.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:01.357+0000 7f1159600640 1 -- 172.21.15.151:0/3531387001 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v102) ==== 170+0+0 (secure 0 0 0) 0x7f1144020020 con 0x7f115c16a000 2024-09-16T15:51:01.360 INFO:teuthology.orchestra.run.smithi151.stderr:pool 'cephfs_data' created 2024-09-16T15:51:01.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:01.360+0000 7f1163600640 1 -- 172.21.15.151:0/3531387001 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1138079f00 msgr2=0x7f113807c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:01.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:01.360+0000 7f1163600640 1 --2- 172.21.15.151:0/3531387001 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1138079f00 0x7f113807c3c0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f114c001860 tx=0x7f114c001c70 comp rx=0 tx=0).stop 2024-09-16T15:51:01.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:01.360+0000 7f1163600640 1 -- 172.21.15.151:0/3531387001 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f115c16a000 msgr2=0x7f115c16a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:01.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:01.360+0000 7f1163600640 1 --2- 172.21.15.151:0/3531387001 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f115c16a000 0x7f115c16a480 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f1144003de0 tx=0x7f1144001630 comp rx=0 tx=0).stop 2024-09-16T15:51:01.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:01.361+0000 7f1163600640 1 -- 172.21.15.151:0/3531387001 shutdown_connections 2024-09-16T15:51:01.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:01.361+0000 7f1163600640 1 --2- 172.21.15.151:0/3531387001 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1138079f00 0x7f113807c3c0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:01.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:01.361+0000 7f1163600640 1 --2- 172.21.15.151:0/3531387001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f115c1535d0 0x7f115c153a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:01.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:01.361+0000 7f1163600640 1 --2- 172.21.15.151:0/3531387001 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f115c16a000 0x7f115c16a480 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:01.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:01.361+0000 7f1163600640 1 --2- 172.21.15.151:0/3531387001 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1140005210 0x7f115c073b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:01.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:01.361+0000 7f1163600640 1 -- 172.21.15.151:0/3531387001 >> 172.21.15.151:0/3531387001 conn(0x7f115c0b1840 msgr2=0x7f115c0b0780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:01.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:01.361+0000 7f1163600640 1 -- 172.21.15.151:0/3531387001 shutdown_connections 2024-09-16T15:51:01.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:01.361+0000 7f1163600640 1 -- 172.21.15.151:0/3531387001 wait complete. 2024-09-16T15:51:01.394 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-16T15:51:01.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:01.555+0000 7fe255c00640 1 -- 172.21.15.151:0/3936508906 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe2500cf830 msgr2=0x7fe2500b33b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:01.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:01.555+0000 7fe255c00640 1 --2- 172.21.15.151:0/3936508906 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe2500cf830 0x7fe2500b33b0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fe238009830 tx=0x7fe23802a480 comp rx=0 tx=0).stop 2024-09-16T15:51:01.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:01.555+0000 7fe255c00640 1 -- 172.21.15.151:0/3936508906 shutdown_connections 2024-09-16T15:51:01.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:01.555+0000 7fe255c00640 1 --2- 172.21.15.151:0/3936508906 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe2500cf830 0x7fe2500b33b0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:01.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:01.555+0000 7fe255c00640 1 -- 172.21.15.151:0/3936508906 >> 172.21.15.151:0/3936508906 conn(0x7fe25009d0b0 msgr2=0x7fe2500d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:01.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:01.555+0000 7fe255c00640 1 -- 172.21.15.151:0/3936508906 shutdown_connections 2024-09-16T15:51:01.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:01.555+0000 7fe255c00640 1 -- 172.21.15.151:0/3936508906 wait complete. 2024-09-16T15:51:01.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:01.556+0000 7fe255c00640 1 Processor -- start 2024-09-16T15:51:01.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:01.557+0000 7fe255c00640 1 -- start start 2024-09-16T15:51:01.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:01.557+0000 7fe255c00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe2500cf830 0x7fe25015d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:01.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:01.557+0000 7fe255c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2500d2a00 0x7fe2500c82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:01.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:01.557+0000 7fe255c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe250161460 0x7fe2500ca2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:01.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:01.557+0000 7fe255c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe2500b50c0 con 0x7fe250161460 2024-09-16T15:51:01.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:01.557+0000 7fe255c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe2500b4f40 con 0x7fe2500d2a00 2024-09-16T15:51:01.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:01.557+0000 7fe255c00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fe2500b4000 con 0x7fe2500cf830 2024-09-16T15:51:01.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:01.558+0000 7fe255200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe250161460 0x7fe2500ca2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:01.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:01.558+0000 7fe24fe00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe2500cf830 0x7fe25015d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:01.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:01.558+0000 7fe247e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2500d2a00 0x7fe2500c82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:01.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:01.558+0000 7fe255200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe250161460 0x7fe2500ca2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:54422/0 (socket says 172.21.15.151:54422) 2024-09-16T15:51:01.561 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:01.558+0000 7fe255200640 1 -- 172.21.15.151:0/2570334329 learned_addr learned my addr 172.21.15.151:0/2570334329 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:51:01.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:01.558+0000 7fe255200640 1 -- 172.21.15.151:0/2570334329 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe2500cf830 msgr2=0x7fe25015d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:01.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:01.558+0000 7fe255200640 1 --2- 172.21.15.151:0/2570334329 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe2500cf830 0x7fe25015d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:01.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:01.558+0000 7fe255200640 1 -- 172.21.15.151:0/2570334329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2500d2a00 msgr2=0x7fe2500c82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:01.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:01.558+0000 7fe255200640 1 --2- 172.21.15.151:0/2570334329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2500d2a00 0x7fe2500c82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:01.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:01.558+0000 7fe255200640 1 -- 172.21.15.151:0/2570334329 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe25015dbf0 con 0x7fe250161460 2024-09-16T15:51:01.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:01.558+0000 7fe247e00640 1 --2- 172.21.15.151:0/2570334329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2500d2a00 0x7fe2500c82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:51:01.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:01.559+0000 7fe255200640 1 --2- 172.21.15.151:0/2570334329 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe250161460 0x7fe2500ca2b0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fe24800c780 tx=0x7fe24800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:01.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:01.559+0000 7fe24cc00640 1 -- 172.21.15.151:0/2570334329 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe248004b60 con 0x7fe250161460 2024-09-16T15:51:01.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:01.559+0000 7fe24cc00640 1 -- 172.21.15.151:0/2570334329 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe248004df0 con 0x7fe250161460 2024-09-16T15:51:01.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:01.559+0000 7fe24cc00640 1 -- 172.21.15.151:0/2570334329 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24800aa70 con 0x7fe250161460 2024-09-16T15:51:01.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:01.559+0000 7fe255c00640 1 -- 172.21.15.151:0/2570334329 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2501f5230 con 0x7fe250161460 2024-09-16T15:51:01.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:01.559+0000 7fe255c00640 1 -- 172.21.15.151:0/2570334329 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2501f57f0 con 0x7fe250161460 2024-09-16T15:51:01.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:01.560+0000 7fe255c00640 1 -- 172.21.15.151:0/2570334329 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2500b4590 con 0x7fe250161460 2024-09-16T15:51:01.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:01.562+0000 7fe24cc00640 1 -- 172.21.15.151:0/2570334329 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7fe24800ac10 con 0x7fe250161460 2024-09-16T15:51:01.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:01.563+0000 7fe24cc00640 1 --2- 172.21.15.151:0/2570334329 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe22c079ed0 0x7fe22c07c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:01.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:01.563+0000 7fe24cc00640 1 -- 172.21.15.151:0/2570334329 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5665+0+0 (secure 0 0 0) 0x7fe248099f20 con 0x7fe250161460 2024-09-16T15:51:01.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:01.564+0000 7fe24fe00640 1 --2- 172.21.15.151:0/2570334329 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe22c079ed0 0x7fe22c07c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:01.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:01.564+0000 7fe24fe00640 1 --2- 172.21.15.151:0/2570334329 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe22c079ed0 0x7fe22c07c390 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fe240001700 tx=0x7fe240007e90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:01.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:01.567+0000 7fe24cc00640 1 -- 172.21.15.151:0/2570334329 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe248020070 con 0x7fe250161460 2024-09-16T15:51:01.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:01.787+0000 7fe255c00640 1 -- 172.21.15.151:0/2570334329 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fe2500b2c70 con 0x7fe250161460 2024-09-16T15:51:02.388 INFO:teuthology.orchestra.run.smithi151.stderr:new fs with metadata pool 14 and data pool 15 2024-09-16T15:51:02.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.385+0000 7fe24cc00640 1 -- 172.21.15.151:0/2570334329 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v62) ==== 174+0+0 (secure 0 0 0) 0x7fe248063a20 con 0x7fe250161460 2024-09-16T15:51:02.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.391+0000 7fe255c00640 1 -- 172.21.15.151:0/2570334329 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe22c079ed0 msgr2=0x7fe22c07c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:02.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.391+0000 7fe255c00640 1 --2- 172.21.15.151:0/2570334329 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe22c079ed0 0x7fe22c07c390 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fe240001700 tx=0x7fe240007e90 comp rx=0 tx=0).stop 2024-09-16T15:51:02.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.391+0000 7fe255c00640 1 -- 172.21.15.151:0/2570334329 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe250161460 msgr2=0x7fe2500ca2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:02.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.391+0000 7fe255c00640 1 --2- 172.21.15.151:0/2570334329 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe250161460 0x7fe2500ca2b0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fe24800c780 tx=0x7fe24800cc40 comp rx=0 tx=0).stop 2024-09-16T15:51:02.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.392+0000 7fe255c00640 1 -- 172.21.15.151:0/2570334329 shutdown_connections 2024-09-16T15:51:02.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.392+0000 7fe255c00640 1 --2- 172.21.15.151:0/2570334329 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe22c079ed0 0x7fe22c07c390 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:02.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.392+0000 7fe255c00640 1 --2- 172.21.15.151:0/2570334329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2500d2a00 0x7fe2500c82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:02.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.392+0000 7fe255c00640 1 --2- 172.21.15.151:0/2570334329 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe250161460 0x7fe2500ca2b0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:02.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.392+0000 7fe255c00640 1 --2- 172.21.15.151:0/2570334329 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe2500cf830 0x7fe25015d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:02.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.393+0000 7fe255c00640 1 -- 172.21.15.151:0/2570334329 >> 172.21.15.151:0/2570334329 conn(0x7fe25009d0b0 msgr2=0x7fe2500b58e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:02.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.393+0000 7fe255c00640 1 -- 172.21.15.151:0/2570334329 shutdown_connections 2024-09-16T15:51:02.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.393+0000 7fe255c00640 1 -- 172.21.15.151:0/2570334329 wait complete. 2024-09-16T15:51:02.423 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:51:02.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.578+0000 7ff507000640 1 -- 172.21.15.151:0/3543270178 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff50015f5a0 msgr2=0x7ff5000b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:02.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.578+0000 7ff507000640 1 --2- 172.21.15.151:0/3543270178 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff50015f5a0 0x7ff5000b2440 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0005fd0 tx=0x7ff4f002aa10 comp rx=0 tx=0).stop 2024-09-16T15:51:02.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.579+0000 7ff507000640 1 -- 172.21.15.151:0/3543270178 shutdown_connections 2024-09-16T15:51:02.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.579+0000 7ff507000640 1 --2- 172.21.15.151:0/3543270178 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff5000ccf00 0x7ff5000acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:02.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.579+0000 7ff507000640 1 --2- 172.21.15.151:0/3543270178 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff50015f5a0 0x7ff5000b2440 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:02.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.579+0000 7ff507000640 1 -- 172.21.15.151:0/3543270178 >> 172.21.15.151:0/3543270178 conn(0x7ff5000b1950 msgr2=0x7ff5000b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:02.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.579+0000 7ff507000640 1 -- 172.21.15.151:0/3543270178 shutdown_connections 2024-09-16T15:51:02.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.579+0000 7ff507000640 1 -- 172.21.15.151:0/3543270178 wait complete. 2024-09-16T15:51:02.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.580+0000 7ff507000640 1 Processor -- start 2024-09-16T15:51:02.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.580+0000 7ff507000640 1 -- start start 2024-09-16T15:51:02.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.581+0000 7ff507000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff5000ad270 0x7ff5000cfd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:02.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.581+0000 7ff507000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5000ccf00 0x7ff5000cb450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:02.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.581+0000 7ff507000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff50015f5a0 0x7ff500149280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:02.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.581+0000 7ff507000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff50006f2e0 con 0x7ff5000ad270 2024-09-16T15:51:02.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.581+0000 7ff507000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff50006f160 con 0x7ff5000ccf00 2024-09-16T15:51:02.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.581+0000 7ff507000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7ff50006f460 con 0x7ff50015f5a0 2024-09-16T15:51:02.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.581+0000 7ff4ffe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff5000ad270 0x7ff5000cfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:02.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.582+0000 7ff4ffe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff5000ad270 0x7ff5000cfd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:54442/0 (socket says 172.21.15.151:54442) 2024-09-16T15:51:02.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.582+0000 7ff4ffe00640 1 -- 172.21.15.151:0/3465414325 learned_addr learned my addr 172.21.15.151:0/3465414325 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:51:02.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.582+0000 7ff505000640 1 --2- 172.21.15.151:0/3465414325 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff50015f5a0 0x7ff500149280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:02.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.582+0000 7ff4ff400640 1 --2- 172.21.15.151:0/3465414325 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5000ccf00 0x7ff5000cb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:02.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.582+0000 7ff4ffe00640 1 -- 172.21.15.151:0/3465414325 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff50015f5a0 msgr2=0x7ff500149280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:02.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.582+0000 7ff4ffe00640 1 --2- 172.21.15.151:0/3465414325 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff50015f5a0 0x7ff500149280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:02.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.582+0000 7ff4ffe00640 1 -- 172.21.15.151:0/3465414325 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5000ccf00 msgr2=0x7ff5000cb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:02.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.582+0000 7ff4ffe00640 1 --2- 172.21.15.151:0/3465414325 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5000ccf00 0x7ff5000cb450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:02.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.582+0000 7ff4ffe00640 1 -- 172.21.15.151:0/3465414325 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff50014fe70 con 0x7ff5000ad270 2024-09-16T15:51:02.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.582+0000 7ff4ffe00640 1 --2- 172.21.15.151:0/3465414325 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff5000ad270 0x7ff5000cfd80 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0009bf0 tx=0x7ff4f0003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:02.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.582+0000 7ff4f7e00640 1 -- 172.21.15.151:0/3465414325 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f0033050 con 0x7ff5000ad270 2024-09-16T15:51:02.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.582+0000 7ff4f7e00640 1 -- 172.21.15.151:0/3465414325 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff4f0004450 con 0x7ff5000ad270 2024-09-16T15:51:02.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.583+0000 7ff507000640 1 -- 172.21.15.151:0/3465414325 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5000ce080 con 0x7ff5000ad270 2024-09-16T15:51:02.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.583+0000 7ff4f7e00640 1 -- 172.21.15.151:0/3465414325 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f0004840 con 0x7ff5000ad270 2024-09-16T15:51:02.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.583+0000 7ff507000640 1 -- 172.21.15.151:0/3465414325 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5000ce640 con 0x7ff5000ad270 2024-09-16T15:51:02.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.584+0000 7ff507000640 1 -- 172.21.15.151:0/3465414325 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff50006f9f0 con 0x7ff5000ad270 2024-09-16T15:51:02.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.585+0000 7ff4f7e00640 1 -- 172.21.15.151:0/3465414325 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7ff4f0036070 con 0x7ff5000ad270 2024-09-16T15:51:02.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.586+0000 7ff4f7e00640 1 --2- 172.21.15.151:0/3465414325 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff4e0079ed0 0x7ff4e007c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:02.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.586+0000 7ff4f7e00640 1 -- 172.21.15.151:0/3465414325 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5765+0+0 (secure 0 0 0) 0x7ff4f00baa80 con 0x7ff5000ad270 2024-09-16T15:51:02.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.586+0000 7ff4ff400640 1 --2- 172.21.15.151:0/3465414325 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff4e0079ed0 0x7ff4e007c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:02.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.587+0000 7ff4ff400640 1 --2- 172.21.15.151:0/3465414325 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff4e0079ed0 0x7ff4e007c390 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec005fd0 tx=0x7ff4ec005850 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:02.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.590+0000 7ff4f7e00640 1 -- 172.21.15.151:0/3465414325 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff4f0084b50 con 0x7ff5000ad270 2024-09-16T15:51:02.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.773+0000 7ff507000640 1 -- 172.21.15.151:0/3465414325 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff50015ed50 con 0x7ff5000ad270 2024-09-16T15:51:02.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.774+0000 7ff4f7e00640 1 -- 172.21.15.151:0/3465414325 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) ==== 74+0+16839 (secure 0 0 0) 0x7ff4f0084450 con 0x7ff5000ad270 2024-09-16T15:51:02.777 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:51:02.777 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":103,"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","created":"2024-09-16T15:38:56.348535+0000","modified":"2024-09-16T15:51:01.789415+0000","last_up_change":"2024-09-16T15:39:10.681384+0000","last_in_change":"2024-09-16T15:39:00.544477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:39:11.602252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:50:59.568133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-16T15:51:00.775162+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5524076-a903-4d0a-af6a-caf474c4765d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6809","nonce":3111342741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6811","nonce":3111342741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6815","nonce":3111342741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6813","nonce":3111342741}]},"public_addr":"172.21.15.151:6809/3111342741","cluster_addr":"172.21.15.151:6811/3111342741","heartbeat_back_addr":"172.21.15.151:6815/3111342741","heartbeat_front_addr":"172.21.15.151:6813/3111342741","state":["exists","up"]},{"osd":1,"uuid":"dd25925f-0ae1-494e-8529-4dc205759db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6801","nonce":3368304087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6803","nonce":3368304087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6807","nonce":3368304087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6805","nonce":3368304087}]},"public_addr":"172.21.15.151:6801/3368304087","cluster_addr":"172.21.15.151:6803/3368304087","heartbeat_back_addr":"172.21.15.151:6807/3368304087","heartbeat_front_addr":"172.21.15.151:6805/3368304087","state":["exists","up"]},{"osd":2,"uuid":"c2f18115-013d-4da6-a0fe-cb978f4494b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6823","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6825","nonce":3079373271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6827","nonce":3079373271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6831","nonce":3079373271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6829","nonce":3079373271}]},"public_addr":"172.21.15.151:6825/3079373271","cluster_addr":"172.21.15.151:6827/3079373271","heartbeat_back_addr":"172.21.15.151:6831/3079373271","heartbeat_front_addr":"172.21.15.151:6829/3079373271","state":["exists","up"]},{"osd":3,"uuid":"36edb718-2fcb-4d88-8ddc-c321eed4279d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6817","nonce":1142509041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6819","nonce":1142509041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6824","nonce":1142509041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6821","nonce":1142509041}]},"public_addr":"172.21.15.151:6817/1142509041","cluster_addr":"172.21.15.151:6819/1142509041","heartbeat_back_addr":"172.21.15.151:6824/1142509041","heartbeat_front_addr":"172.21.15.151:6821/1142509041","state":["exists","up"]},{"osd":4,"uuid":"64d33708-cd0a-4236-abe3-1b646e82ab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6811","nonce":2735495271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6813","nonce":2735495271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6817","nonce":2735495271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6815","nonce":2735495271}]},"public_addr":"172.21.15.167:6811/2735495271","cluster_addr":"172.21.15.167:6813/2735495271","heartbeat_back_addr":"172.21.15.167:6817/2735495271","heartbeat_front_addr":"172.21.15.167:6815/2735495271","state":["exists","up"]},{"osd":5,"uuid":"f14b44a9-5910-458c-b518-09a8d0de94de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6827","nonce":1494916572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6829","nonce":1494916572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6833","nonce":1494916572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6831","nonce":1494916572}]},"public_addr":"172.21.15.167:6827/1494916572","cluster_addr":"172.21.15.167:6829/1494916572","heartbeat_back_addr":"172.21.15.167:6833/1494916572","heartbeat_front_addr":"172.21.15.167:6831/1494916572","state":["exists","up"]},{"osd":6,"uuid":"39335545-2654-4710-84b0-a9aa47cfcbba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6819","nonce":2538273273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6821","nonce":2538273273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6825","nonce":2538273273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6823","nonce":2538273273}]},"public_addr":"172.21.15.167:6819/2538273273","cluster_addr":"172.21.15.167:6821/2538273273","heartbeat_back_addr":"172.21.15.167:6825/2538273273","heartbeat_front_addr":"172.21.15.167:6823/2538273273","state":["exists","up"]},{"osd":7,"uuid":"4f8bfc81-53dd-4339-ac22-1a4007d9f963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6803","nonce":265072227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6805","nonce":265072227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6809","nonce":265072227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6807","nonce":265072227}]},"public_addr":"172.21.15.167:6803/265072227","cluster_addr":"172.21.15.167:6805/265072227","heartbeat_back_addr":"172.21.15.167:6809/265072227","heartbeat_front_addr":"172.21.15.167:6807/265072227","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.150359+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.084358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.722368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.332809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.320709+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.395153+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.935250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.801443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:51:02.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.777+0000 7ff507000640 1 -- 172.21.15.151:0/3465414325 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff4e0079ed0 msgr2=0x7ff4e007c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:02.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.777+0000 7ff507000640 1 --2- 172.21.15.151:0/3465414325 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff4e0079ed0 0x7ff4e007c390 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec005fd0 tx=0x7ff4ec005850 comp rx=0 tx=0).stop 2024-09-16T15:51:02.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.777+0000 7ff507000640 1 -- 172.21.15.151:0/3465414325 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff5000ad270 msgr2=0x7ff5000cfd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:02.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.777+0000 7ff507000640 1 --2- 172.21.15.151:0/3465414325 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff5000ad270 0x7ff5000cfd80 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0009bf0 tx=0x7ff4f0003f90 comp rx=0 tx=0).stop 2024-09-16T15:51:02.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.777+0000 7ff507000640 1 -- 172.21.15.151:0/3465414325 shutdown_connections 2024-09-16T15:51:02.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.777+0000 7ff507000640 1 --2- 172.21.15.151:0/3465414325 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff4e0079ed0 0x7ff4e007c390 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:02.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.777+0000 7ff507000640 1 --2- 172.21.15.151:0/3465414325 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5000ccf00 0x7ff5000cb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:02.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.777+0000 7ff507000640 1 --2- 172.21.15.151:0/3465414325 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff50015f5a0 0x7ff500149280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:02.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.777+0000 7ff507000640 1 --2- 172.21.15.151:0/3465414325 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff5000ad270 0x7ff5000cfd80 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:02.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.777+0000 7ff507000640 1 -- 172.21.15.151:0/3465414325 >> 172.21.15.151:0/3465414325 conn(0x7ff5000b1950 msgr2=0x7ff50009d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:02.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.777+0000 7ff507000640 1 -- 172.21.15.151:0/3465414325 shutdown_connections 2024-09-16T15:51:02.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.777+0000 7ff507000640 1 -- 172.21.15.151:0/3465414325 wait complete. 2024-09-16T15:51:02.796 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:51:02.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.950+0000 7f585e600640 1 -- 172.21.15.151:0/1595846092 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5858073fb0 msgr2=0x7f5858074420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:02.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.950+0000 7f585e600640 1 --2- 172.21.15.151:0/1595846092 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5858073fb0 0x7f5858074420 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f58580cfbb0 tx=0x7f585002a820 comp rx=0 tx=0).stop 2024-09-16T15:51:02.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.950+0000 7f585e600640 1 -- 172.21.15.151:0/1595846092 shutdown_connections 2024-09-16T15:51:02.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.950+0000 7f585e600640 1 --2- 172.21.15.151:0/1595846092 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5858073fb0 0x7f5858074420 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:02.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.950+0000 7f585e600640 1 --2- 172.21.15.151:0/1595846092 >> v2:172.21.15.167:3300/0 conn(0x7f58580ccf00 0x7f58580b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:02.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.950+0000 7f585e600640 1 -- 172.21.15.151:0/1595846092 >> 172.21.15.151:0/1595846092 conn(0x7f58580b1950 msgr2=0x7f58580b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:02.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.950+0000 7f585e600640 1 -- 172.21.15.151:0/1595846092 shutdown_connections 2024-09-16T15:51:02.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.950+0000 7f585e600640 1 -- 172.21.15.151:0/1595846092 wait complete. 2024-09-16T15:51:02.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.952+0000 7f585e600640 1 Processor -- start 2024-09-16T15:51:02.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.952+0000 7f585e600640 1 -- start start 2024-09-16T15:51:02.957 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.953+0000 7f585e600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5858073fb0 0x7f58580d0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:02.957 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.953+0000 7f585e600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f58580ccf00 0x7f585814c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:02.957 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.953+0000 7f585e600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f585815f5a0 0x7f58581f4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:02.957 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.953+0000 7f585e600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f585806f2e0 con 0x7f585815f5a0 2024-09-16T15:51:02.957 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.953+0000 7f585e600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f585806f160 con 0x7f5858073fb0 2024-09-16T15:51:02.957 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.953+0000 7f585e600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f585806f460 con 0x7f58580ccf00 2024-09-16T15:51:02.957 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.953+0000 7f5857e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f585815f5a0 0x7f58581f4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:02.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.954+0000 7f5857e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f585815f5a0 0x7f58581f4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52558/0 (socket says 172.21.15.151:52558) 2024-09-16T15:51:02.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.954+0000 7f5857e00640 1 -- 172.21.15.151:0/2164951278 learned_addr learned my addr 172.21.15.151:0/2164951278 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:51:02.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.954+0000 7f5857400640 1 --2- 172.21.15.151:0/2164951278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5858073fb0 0x7f58580d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:02.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.954+0000 7f5856a00640 1 --2- 172.21.15.151:0/2164951278 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f58580ccf00 0x7f585814c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:02.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.954+0000 7f5857e00640 1 -- 172.21.15.151:0/2164951278 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f58580ccf00 msgr2=0x7f585814c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:02.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.954+0000 7f5857e00640 1 --2- 172.21.15.151:0/2164951278 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f58580ccf00 0x7f585814c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:02.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.954+0000 7f5857e00640 1 -- 172.21.15.151:0/2164951278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5858073fb0 msgr2=0x7f58580d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:02.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.954+0000 7f5857e00640 1 --2- 172.21.15.151:0/2164951278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5858073fb0 0x7f58580d0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:02.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.954+0000 7f5857e00640 1 -- 172.21.15.151:0/2164951278 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58581f52b0 con 0x7f585815f5a0 2024-09-16T15:51:02.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.954+0000 7f5856a00640 1 --2- 172.21.15.151:0/2164951278 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f58580ccf00 0x7f585814c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:51:02.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.954+0000 7f5857e00640 1 --2- 172.21.15.151:0/2164951278 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f585815f5a0 0x7f58581f4a10 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f585002ae10 tx=0x7f5850032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:02.959 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.954+0000 7f584f400640 1 -- 172.21.15.151:0/2164951278 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5850037040 con 0x7f585815f5a0 2024-09-16T15:51:02.959 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.954+0000 7f584f400640 1 -- 172.21.15.151:0/2164951278 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5850042070 con 0x7f585815f5a0 2024-09-16T15:51:02.959 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.954+0000 7f585e600640 1 -- 172.21.15.151:0/2164951278 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58581f5590 con 0x7f585815f5a0 2024-09-16T15:51:02.959 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.955+0000 7f584f400640 1 -- 172.21.15.151:0/2164951278 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5850003e60 con 0x7f585815f5a0 2024-09-16T15:51:02.959 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.955+0000 7f585e600640 1 -- 172.21.15.151:0/2164951278 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58581f5e30 con 0x7f585815f5a0 2024-09-16T15:51:02.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.956+0000 7f585e600640 1 -- 172.21.15.151:0/2164951278 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58581f5c20 con 0x7f585815f5a0 2024-09-16T15:51:02.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.957+0000 7f584f400640 1 -- 172.21.15.151:0/2164951278 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7f58500042a0 con 0x7f585815f5a0 2024-09-16T15:51:02.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.958+0000 7f584f400640 1 --2- 172.21.15.151:0/2164951278 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5838079ed0 0x7f583807c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:02.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.958+0000 7f584f400640 1 -- 172.21.15.151:0/2164951278 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5765+0+0 (secure 0 0 0) 0x7f58500ba290 con 0x7f585815f5a0 2024-09-16T15:51:02.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.958+0000 7f5857400640 1 --2- 172.21.15.151:0/2164951278 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5838079ed0 0x7f583807c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:02.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.959+0000 7f5857400640 1 --2- 172.21.15.151:0/2164951278 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5838079ed0 0x7f583807c390 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f5848000f60 tx=0x7f5848009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:02.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:02.962+0000 7f584f400640 1 -- 172.21.15.151:0/2164951278 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f5850084360 con 0x7f585815f5a0 2024-09-16T15:51:03.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:03.147+0000 7f585e600640 1 -- 172.21.15.151:0/2164951278 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f585806f9f0 con 0x7f585815f5a0 2024-09-16T15:51:03.150 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:03.148+0000 7f584f400640 1 -- 172.21.15.151:0/2164951278 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) ==== 74+0+16839 (secure 0 0 0) 0x7f5850083c60 con 0x7f585815f5a0 2024-09-16T15:51:03.151 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:51:03.151 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":103,"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","created":"2024-09-16T15:38:56.348535+0000","modified":"2024-09-16T15:51:01.789415+0000","last_up_change":"2024-09-16T15:39:10.681384+0000","last_in_change":"2024-09-16T15:39:00.544477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:39:11.602252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:50:59.568133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-16T15:51:00.775162+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5524076-a903-4d0a-af6a-caf474c4765d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6809","nonce":3111342741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6811","nonce":3111342741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6815","nonce":3111342741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6813","nonce":3111342741}]},"public_addr":"172.21.15.151:6809/3111342741","cluster_addr":"172.21.15.151:6811/3111342741","heartbeat_back_addr":"172.21.15.151:6815/3111342741","heartbeat_front_addr":"172.21.15.151:6813/3111342741","state":["exists","up"]},{"osd":1,"uuid":"dd25925f-0ae1-494e-8529-4dc205759db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6801","nonce":3368304087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6803","nonce":3368304087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6807","nonce":3368304087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6805","nonce":3368304087}]},"public_addr":"172.21.15.151:6801/3368304087","cluster_addr":"172.21.15.151:6803/3368304087","heartbeat_back_addr":"172.21.15.151:6807/3368304087","heartbeat_front_addr":"172.21.15.151:6805/3368304087","state":["exists","up"]},{"osd":2,"uuid":"c2f18115-013d-4da6-a0fe-cb978f4494b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6823","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6825","nonce":3079373271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6827","nonce":3079373271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6831","nonce":3079373271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6829","nonce":3079373271}]},"public_addr":"172.21.15.151:6825/3079373271","cluster_addr":"172.21.15.151:6827/3079373271","heartbeat_back_addr":"172.21.15.151:6831/3079373271","heartbeat_front_addr":"172.21.15.151:6829/3079373271","state":["exists","up"]},{"osd":3,"uuid":"36edb718-2fcb-4d88-8ddc-c321eed4279d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6817","nonce":1142509041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6819","nonce":1142509041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6824","nonce":1142509041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6821","nonce":1142509041}]},"public_addr":"172.21.15.151:6817/1142509041","cluster_addr":"172.21.15.151:6819/1142509041","heartbeat_back_addr":"172.21.15.151:6824/1142509041","heartbeat_front_addr":"172.21.15.151:6821/1142509041","state":["exists","up"]},{"osd":4,"uuid":"64d33708-cd0a-4236-abe3-1b646e82ab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6811","nonce":2735495271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6813","nonce":2735495271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6817","nonce":2735495271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6815","nonce":2735495271}]},"public_addr":"172.21.15.167:6811/2735495271","cluster_addr":"172.21.15.167:6813/2735495271","heartbeat_back_addr":"172.21.15.167:6817/2735495271","heartbeat_front_addr":"172.21.15.167:6815/2735495271","state":["exists","up"]},{"osd":5,"uuid":"f14b44a9-5910-458c-b518-09a8d0de94de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6827","nonce":1494916572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6829","nonce":1494916572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6833","nonce":1494916572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6831","nonce":1494916572}]},"public_addr":"172.21.15.167:6827/1494916572","cluster_addr":"172.21.15.167:6829/1494916572","heartbeat_back_addr":"172.21.15.167:6833/1494916572","heartbeat_front_addr":"172.21.15.167:6831/1494916572","state":["exists","up"]},{"osd":6,"uuid":"39335545-2654-4710-84b0-a9aa47cfcbba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6819","nonce":2538273273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6821","nonce":2538273273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6825","nonce":2538273273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6823","nonce":2538273273}]},"public_addr":"172.21.15.167:6819/2538273273","cluster_addr":"172.21.15.167:6821/2538273273","heartbeat_back_addr":"172.21.15.167:6825/2538273273","heartbeat_front_addr":"172.21.15.167:6823/2538273273","state":["exists","up"]},{"osd":7,"uuid":"4f8bfc81-53dd-4339-ac22-1a4007d9f963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6803","nonce":265072227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6805","nonce":265072227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6809","nonce":265072227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6807","nonce":265072227}]},"public_addr":"172.21.15.167:6803/265072227","cluster_addr":"172.21.15.167:6805/265072227","heartbeat_back_addr":"172.21.15.167:6809/265072227","heartbeat_front_addr":"172.21.15.167:6807/265072227","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.150359+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.084358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.722368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.332809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.320709+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.395153+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.935250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.801443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:51:03.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:03.150+0000 7f585e600640 1 -- 172.21.15.151:0/2164951278 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5838079ed0 msgr2=0x7f583807c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:03.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:03.150+0000 7f585e600640 1 --2- 172.21.15.151:0/2164951278 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5838079ed0 0x7f583807c390 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f5848000f60 tx=0x7f5848009040 comp rx=0 tx=0).stop 2024-09-16T15:51:03.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:03.150+0000 7f585e600640 1 -- 172.21.15.151:0/2164951278 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f585815f5a0 msgr2=0x7f58581f4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:03.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:03.150+0000 7f585e600640 1 --2- 172.21.15.151:0/2164951278 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f585815f5a0 0x7f58581f4a10 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f585002ae10 tx=0x7f5850032d10 comp rx=0 tx=0).stop 2024-09-16T15:51:03.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:03.150+0000 7f585e600640 1 -- 172.21.15.151:0/2164951278 shutdown_connections 2024-09-16T15:51:03.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:03.150+0000 7f585e600640 1 --2- 172.21.15.151:0/2164951278 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5838079ed0 0x7f583807c390 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:03.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:03.150+0000 7f585e600640 1 --2- 172.21.15.151:0/2164951278 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f585815f5a0 0x7f58581f4a10 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:03.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:03.150+0000 7f585e600640 1 --2- 172.21.15.151:0/2164951278 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f58580ccf00 0x7f585814c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:03.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:03.150+0000 7f585e600640 1 --2- 172.21.15.151:0/2164951278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5858073fb0 0x7f58580d0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:03.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:03.150+0000 7f585e600640 1 -- 172.21.15.151:0/2164951278 >> 172.21.15.151:0/2164951278 conn(0x7f58580b1950 msgr2=0x7f5858155870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:03.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:03.151+0000 7f585e600640 1 -- 172.21.15.151:0/2164951278 shutdown_connections 2024-09-16T15:51:03.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:03.151+0000 7f585e600640 1 -- 172.21.15.151:0/2164951278 wait complete. 2024-09-16T15:51:03.170 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-16T15:51:03.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:03.329+0000 7f2542c00640 1 -- 172.21.15.151:0/3486203335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f253c15df50 msgr2=0x7f253c0ce930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:03.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:03.329+0000 7f2542c00640 1 --2- 172.21.15.151:0/3486203335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f253c15df50 0x7f253c0ce930 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f253c0d27b0 tx=0x7f253002aa80 comp rx=0 tx=0).stop 2024-09-16T15:51:03.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:03.329+0000 7f2542c00640 1 -- 172.21.15.151:0/3486203335 shutdown_connections 2024-09-16T15:51:03.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:03.329+0000 7f2542c00640 1 --2- 172.21.15.151:0/3486203335 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f253c0cf580 0x7f253c0c7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:03.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:03.329+0000 7f2542c00640 1 --2- 172.21.15.151:0/3486203335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f253c15df50 0x7f253c0ce930 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:03.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:03.329+0000 7f2542c00640 1 -- 172.21.15.151:0/3486203335 >> 172.21.15.151:0/3486203335 conn(0x7f253c0d3e60 msgr2=0x7f253c0d4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:03.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:03.329+0000 7f2542c00640 1 -- 172.21.15.151:0/3486203335 shutdown_connections 2024-09-16T15:51:03.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:03.329+0000 7f2542c00640 1 -- 172.21.15.151:0/3486203335 wait complete. 2024-09-16T15:51:03.333 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:03.331+0000 7f2542c00640 1 Processor -- start 2024-09-16T15:51:03.333 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:03.331+0000 7f2542c00640 1 -- start start 2024-09-16T15:51:03.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:03.332+0000 7f2542c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f253c0cf580 0x7f253c0caf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:03.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:03.332+0000 7f2542c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f253c0d4980 0x7f253c1f4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:03.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:03.332+0000 7f2542c00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f253c15df50 0x7f253c0a1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:03.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:03.332+0000 7f2542c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f253c0c86a0 con 0x7f253c0d4980 2024-09-16T15:51:03.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:03.332+0000 7f2542c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f253c0c8520 con 0x7f253c0cf580 2024-09-16T15:51:03.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:03.332+0000 7f2542c00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f253c0c8820 con 0x7f253c15df50 2024-09-16T15:51:03.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:03.333+0000 7f2540e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f253c0d4980 0x7f253c1f4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:03.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:03.333+0000 7f2541800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f253c0cf580 0x7f253c0caf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:03.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:03.333+0000 7f2540e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f253c0d4980 0x7f253c1f4e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52574/0 (socket says 172.21.15.151:52574) 2024-09-16T15:51:03.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:03.333+0000 7f2542200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f253c15df50 0x7f253c0a1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:03.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:03.333+0000 7f2540e00640 1 -- 172.21.15.151:0/290408045 learned_addr learned my addr 172.21.15.151:0/290408045 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:51:03.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:03.333+0000 7f2540e00640 1 -- 172.21.15.151:0/290408045 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f253c15df50 msgr2=0x7f253c0a1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:03.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:03.333+0000 7f2540e00640 1 --2- 172.21.15.151:0/290408045 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f253c15df50 0x7f253c0a1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:03.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:03.333+0000 7f2540e00640 1 -- 172.21.15.151:0/290408045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f253c0cf580 msgr2=0x7f253c0caf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:03.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:03.333+0000 7f2540e00640 1 --2- 172.21.15.151:0/290408045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f253c0cf580 0x7f253c0caf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:03.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:03.333+0000 7f2540e00640 1 -- 172.21.15.151:0/290408045 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f253c1f5340 con 0x7f253c0d4980 2024-09-16T15:51:03.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:03.333+0000 7f2541800640 1 --2- 172.21.15.151:0/290408045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f253c0cf580 0x7f253c0caf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:51:03.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:03.333+0000 7f2540e00640 1 --2- 172.21.15.151:0/290408045 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f253c0d4980 0x7f253c1f4e00 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f252800c450 tx=0x7f252800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:03.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:03.334+0000 7f2539600640 1 -- 172.21.15.151:0/290408045 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2528016020 con 0x7f253c0d4980 2024-09-16T15:51:03.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:03.334+0000 7f2539600640 1 -- 172.21.15.151:0/290408045 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2528005400 con 0x7f253c0d4980 2024-09-16T15:51:03.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:03.334+0000 7f2539600640 1 -- 172.21.15.151:0/290408045 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2528007280 con 0x7f253c0d4980 2024-09-16T15:51:03.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:03.334+0000 7f2542c00640 1 -- 172.21.15.151:0/290408045 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f253c0cb550 con 0x7f253c0d4980 2024-09-16T15:51:03.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:03.334+0000 7f2542c00640 1 -- 172.21.15.151:0/290408045 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f253c0cbb10 con 0x7f253c0d4980 2024-09-16T15:51:03.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:03.335+0000 7f2542c00640 1 -- 172.21.15.151:0/290408045 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f253c0c7dd0 con 0x7f253c0d4980 2024-09-16T15:51:03.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:03.336+0000 7f2539600640 1 -- 172.21.15.151:0/290408045 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7f2528007420 con 0x7f253c0d4980 2024-09-16T15:51:03.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:03.337+0000 7f2539600640 1 --2- 172.21.15.151:0/290408045 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f251c079ed0 0x7f251c07c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:03.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:03.337+0000 7f2539600640 1 -- 172.21.15.151:0/290408045 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5765+0+0 (secure 0 0 0) 0x7f2528098e80 con 0x7f253c0d4980 2024-09-16T15:51:03.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:03.338+0000 7f2541800640 1 --2- 172.21.15.151:0/290408045 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f251c079ed0 0x7f251c07c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:03.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:03.338+0000 7f2541800640 1 --2- 172.21.15.151:0/290408045 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f251c079ed0 0x7f251c07c390 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f2530035000 tx=0x7f2530038040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:03.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:03.341+0000 7f2539600640 1 -- 172.21.15.151:0/290408045 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2528020020 con 0x7f253c0d4980 2024-09-16T15:51:03.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:03.566+0000 7f2542c00640 1 -- 172.21.15.151:0/290408045 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f253c1f5600 con 0x7f253c0d4980 2024-09-16T15:51:04.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:04.401+0000 7f2539600640 1 -- 172.21.15.151:0/290408045 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v65) ==== 118+0+0 (secure 0 0 0) 0x7f253c1f5340 con 0x7f253c0d4980 2024-09-16T15:51:04.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:04.404+0000 7f2542c00640 1 -- 172.21.15.151:0/290408045 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f251c079ed0 msgr2=0x7f251c07c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:04.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:04.404+0000 7f2542c00640 1 --2- 172.21.15.151:0/290408045 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f251c079ed0 0x7f251c07c390 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f2530035000 tx=0x7f2530038040 comp rx=0 tx=0).stop 2024-09-16T15:51:04.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:04.404+0000 7f2542c00640 1 -- 172.21.15.151:0/290408045 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f253c0d4980 msgr2=0x7f253c1f4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:04.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:04.404+0000 7f2542c00640 1 --2- 172.21.15.151:0/290408045 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f253c0d4980 0x7f253c1f4e00 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f252800c450 tx=0x7f252800c910 comp rx=0 tx=0).stop 2024-09-16T15:51:04.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:04.405+0000 7f2542c00640 1 -- 172.21.15.151:0/290408045 shutdown_connections 2024-09-16T15:51:04.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:04.405+0000 7f2542c00640 1 --2- 172.21.15.151:0/290408045 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f251c079ed0 0x7f251c07c390 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:04.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:04.405+0000 7f2542c00640 1 --2- 172.21.15.151:0/290408045 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f253c15df50 0x7f253c0a1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:04.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:04.405+0000 7f2542c00640 1 --2- 172.21.15.151:0/290408045 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f253c0d4980 0x7f253c1f4e00 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:04.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:04.405+0000 7f2542c00640 1 --2- 172.21.15.151:0/290408045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f253c0cf580 0x7f253c0caf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:04.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:04.405+0000 7f2542c00640 1 -- 172.21.15.151:0/290408045 >> 172.21.15.151:0/290408045 conn(0x7f253c0d3e60 msgr2=0x7f253c06df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:04.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:04.405+0000 7f2542c00640 1 -- 172.21.15.151:0/290408045 shutdown_connections 2024-09-16T15:51:04.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:04.405+0000 7f2542c00640 1 -- 172.21.15.151:0/290408045 wait complete. 2024-09-16T15:51:04.425 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-16T15:51:04.425 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-16T15:51:04.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:04.585+0000 7f1c47800640 1 -- 172.21.15.151:0/3845102253 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c400cccd0 msgr2=0x7f1c400cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:04.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:04.585+0000 7f1c47800640 1 --2- 172.21.15.151:0/3845102253 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c400cccd0 0x7f1c400cdbb0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f1c400d0cd0 tx=0x7f1c3802a820 comp rx=0 tx=0).stop 2024-09-16T15:51:04.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:04.585+0000 7f1c47800640 1 -- 172.21.15.151:0/3845102253 shutdown_connections 2024-09-16T15:51:04.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:04.585+0000 7f1c47800640 1 --2- 172.21.15.151:0/3845102253 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c400cccd0 0x7f1c400cdbb0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:04.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:04.585+0000 7f1c47800640 1 --2- 172.21.15.151:0/3845102253 >> v2:172.21.15.167:3300/0 conn(0x7f1c40145750 0x7f1c40073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:04.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:04.585+0000 7f1c47800640 1 -- 172.21.15.151:0/3845102253 >> 172.21.15.151:0/3845102253 conn(0x7f1c400b1840 msgr2=0x7f1c400b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:04.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:04.585+0000 7f1c47800640 1 -- 172.21.15.151:0/3845102253 shutdown_connections 2024-09-16T15:51:04.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:04.585+0000 7f1c47800640 1 -- 172.21.15.151:0/3845102253 wait complete. 2024-09-16T15:51:04.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:04.586+0000 7f1c47800640 1 Processor -- start 2024-09-16T15:51:04.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:04.587+0000 7f1c47800640 1 -- start start 2024-09-16T15:51:04.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:04.587+0000 7f1c47800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1c4006efa0 0x7f1c401f7a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:04.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:04.587+0000 7f1c47800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c400cccd0 0x7f1c4013d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:04.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:04.588+0000 7f1c47800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c40145750 0x7f1c4013da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:04.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:04.588+0000 7f1c47800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1c400ce480 con 0x7f1c400cccd0 2024-09-16T15:51:04.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:04.588+0000 7f1c47800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1c400ce300 con 0x7f1c40145750 2024-09-16T15:51:04.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:04.588+0000 7f1c47800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f1c400ce600 con 0x7f1c4006efa0 2024-09-16T15:51:04.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:04.588+0000 7f1c37e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c400cccd0 0x7f1c4013d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:04.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:04.588+0000 7f1c37e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c400cccd0 0x7f1c4013d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52586/0 (socket says 172.21.15.151:52586) 2024-09-16T15:51:04.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:04.588+0000 7f1c37e00640 1 -- 172.21.15.151:0/3090775782 learned_addr learned my addr 172.21.15.151:0/3090775782 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:51:04.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:04.588+0000 7f1c44e00640 1 --2- 172.21.15.151:0/3090775782 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1c4006efa0 0x7f1c401f7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:04.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:04.588+0000 7f1c45800640 1 --2- 172.21.15.151:0/3090775782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c40145750 0x7f1c4013da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:04.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:04.588+0000 7f1c37e00640 1 -- 172.21.15.151:0/3090775782 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1c4006efa0 msgr2=0x7f1c401f7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:04.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:04.588+0000 7f1c37e00640 1 --2- 172.21.15.151:0/3090775782 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1c4006efa0 0x7f1c401f7a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:04.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:04.588+0000 7f1c37e00640 1 -- 172.21.15.151:0/3090775782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c40145750 msgr2=0x7f1c4013da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:04.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:04.588+0000 7f1c37e00640 1 --2- 172.21.15.151:0/3090775782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c40145750 0x7f1c4013da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:04.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:04.588+0000 7f1c37e00640 1 -- 172.21.15.151:0/3090775782 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c400c51f0 con 0x7f1c400cccd0 2024-09-16T15:51:04.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:04.588+0000 7f1c44e00640 1 --2- 172.21.15.151:0/3090775782 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1c4006efa0 0x7f1c401f7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:51:04.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:04.588+0000 7f1c45800640 1 --2- 172.21.15.151:0/3090775782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c40145750 0x7f1c4013da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:51:04.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:04.588+0000 7f1c37e00640 1 --2- 172.21.15.151:0/3090775782 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c400cccd0 0x7f1c4013d4e0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f1c2800c450 tx=0x7f1c2800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:04.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:04.589+0000 7f1c3d600640 1 -- 172.21.15.151:0/3090775782 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c28016020 con 0x7f1c400cccd0 2024-09-16T15:51:04.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:04.589+0000 7f1c3d600640 1 -- 172.21.15.151:0/3090775782 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1c28012920 con 0x7f1c400cccd0 2024-09-16T15:51:04.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:04.589+0000 7f1c47800640 1 -- 172.21.15.151:0/3090775782 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c401f7f70 con 0x7f1c400cccd0 2024-09-16T15:51:04.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:04.589+0000 7f1c3d600640 1 -- 172.21.15.151:0/3090775782 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c28012d70 con 0x7f1c400cccd0 2024-09-16T15:51:04.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:04.589+0000 7f1c47800640 1 -- 172.21.15.151:0/3090775782 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c400c4f10 con 0x7f1c400cccd0 2024-09-16T15:51:04.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:04.590+0000 7f1c47800640 1 -- 172.21.15.151:0/3090775782 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c400cdbb0 con 0x7f1c400cccd0 2024-09-16T15:51:04.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:04.591+0000 7f1c3d600640 1 -- 172.21.15.151:0/3090775782 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7f1c28019020 con 0x7f1c400cccd0 2024-09-16T15:51:04.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:04.593+0000 7f1c3d600640 1 --2- 172.21.15.151:0/3090775782 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1c1c079f00 0x7f1c1c07c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:04.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:04.593+0000 7f1c3d600640 1 -- 172.21.15.151:0/3090775782 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5765+0+0 (secure 0 0 0) 0x7f1c280993e0 con 0x7f1c400cccd0 2024-09-16T15:51:04.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:04.593+0000 7f1c44e00640 1 --2- 172.21.15.151:0/3090775782 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1c1c079f00 0x7f1c1c07c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:04.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:04.594+0000 7f1c44e00640 1 --2- 172.21.15.151:0/3090775782 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1c1c079f00 0x7f1c1c07c3c0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f1c30009aa0 tx=0x7f1c30005db0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:04.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:04.597+0000 7f1c3d600640 1 -- 172.21.15.151:0/3090775782 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1c28020030 con 0x7f1c400cccd0 2024-09-16T15:51:04.795 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-09-16T15:51:04.795 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-09-16T15:51:04.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:04.818+0000 7f1c47800640 1 -- 172.21.15.151:0/3090775782 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f1c400cf1a0 con 0x7f1c400cccd0 2024-09-16T15:51:05.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.410+0000 7f1c3d600640 1 -- 172.21.15.151:0/3090775782 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v66) ==== 122+0+0 (secure 0 0 0) 0x7f1c280634b0 con 0x7f1c400cccd0 2024-09-16T15:51:05.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.414+0000 7f1c47800640 1 -- 172.21.15.151:0/3090775782 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1c1c079f00 msgr2=0x7f1c1c07c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:05.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.414+0000 7f1c47800640 1 --2- 172.21.15.151:0/3090775782 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1c1c079f00 0x7f1c1c07c3c0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f1c30009aa0 tx=0x7f1c30005db0 comp rx=0 tx=0).stop 2024-09-16T15:51:05.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.414+0000 7f1c47800640 1 -- 172.21.15.151:0/3090775782 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c400cccd0 msgr2=0x7f1c4013d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:05.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.414+0000 7f1c47800640 1 --2- 172.21.15.151:0/3090775782 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c400cccd0 0x7f1c4013d4e0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f1c2800c450 tx=0x7f1c2800c910 comp rx=0 tx=0).stop 2024-09-16T15:51:05.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.414+0000 7f1c47800640 1 -- 172.21.15.151:0/3090775782 shutdown_connections 2024-09-16T15:51:05.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.414+0000 7f1c47800640 1 --2- 172.21.15.151:0/3090775782 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1c1c079f00 0x7f1c1c07c3c0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:05.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.414+0000 7f1c47800640 1 --2- 172.21.15.151:0/3090775782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c40145750 0x7f1c4013da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:05.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.414+0000 7f1c47800640 1 --2- 172.21.15.151:0/3090775782 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c400cccd0 0x7f1c4013d4e0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:05.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.414+0000 7f1c47800640 1 --2- 172.21.15.151:0/3090775782 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1c4006efa0 0x7f1c401f7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:05.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.414+0000 7f1c47800640 1 -- 172.21.15.151:0/3090775782 >> 172.21.15.151:0/3090775782 conn(0x7f1c400b1840 msgr2=0x7f1c400a9790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:05.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.414+0000 7f1c47800640 1 -- 172.21.15.151:0/3090775782 shutdown_connections 2024-09-16T15:51:05.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.414+0000 7f1c47800640 1 -- 172.21.15.151:0/3090775782 wait complete. 2024-09-16T15:51:05.433 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-16T15:51:05.434 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:51:05.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.589+0000 7fb489c00640 1 -- 172.21.15.151:0/797487088 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4840b4d10 msgr2=0x7fb484162730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:05.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.589+0000 7fb489c00640 1 --2- 172.21.15.151:0/797487088 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4840b4d10 0x7fb484162730 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fb47c005fd0 tx=0x7fb47c02a8b0 comp rx=0 tx=0).stop 2024-09-16T15:51:05.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.590+0000 7fb489c00640 1 -- 172.21.15.151:0/797487088 shutdown_connections 2024-09-16T15:51:05.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.590+0000 7fb489c00640 1 --2- 172.21.15.151:0/797487088 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4840b4d10 0x7fb484162730 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:05.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.590+0000 7fb489c00640 1 --2- 172.21.15.151:0/797487088 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4840b28d0 0x7fb4840d22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:05.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.590+0000 7fb489c00640 1 -- 172.21.15.151:0/797487088 >> 172.21.15.151:0/797487088 conn(0x7fb4840d3de0 msgr2=0x7fb4840d41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:05.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.590+0000 7fb489c00640 1 -- 172.21.15.151:0/797487088 shutdown_connections 2024-09-16T15:51:05.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.590+0000 7fb489c00640 1 -- 172.21.15.151:0/797487088 wait complete. 2024-09-16T15:51:05.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.591+0000 7fb489c00640 1 Processor -- start 2024-09-16T15:51:05.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.591+0000 7fb489c00640 1 -- start start 2024-09-16T15:51:05.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.592+0000 7fb489c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4840b28d0 0x7fb4841f8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:05.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.592+0000 7fb489c00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4840b4d10 0x7fb4841f4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:05.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.592+0000 7fb489c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4840d1ec0 0x7fb4841502b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:05.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.592+0000 7fb489c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb4840b3a80 con 0x7fb4840b28d0 2024-09-16T15:51:05.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.592+0000 7fb489c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb4840b3900 con 0x7fb4840d1ec0 2024-09-16T15:51:05.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.592+0000 7fb489c00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fb4840b3c00 con 0x7fb4840b4d10 2024-09-16T15:51:05.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.592+0000 7fb483e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4840b28d0 0x7fb4841f8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:05.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.592+0000 7fb483e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4840b28d0 0x7fb4841f8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52604/0 (socket says 172.21.15.151:52604) 2024-09-16T15:51:05.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.593+0000 7fb483e00640 1 -- 172.21.15.151:0/1209909843 learned_addr learned my addr 172.21.15.151:0/1209909843 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:51:05.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.593+0000 7fb489200640 1 --2- 172.21.15.151:0/1209909843 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4840d1ec0 0x7fb4841502b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:05.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.593+0000 7fb483400640 1 --2- 172.21.15.151:0/1209909843 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4840b4d10 0x7fb4841f4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:05.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.593+0000 7fb483e00640 1 -- 172.21.15.151:0/1209909843 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4840b4d10 msgr2=0x7fb4841f4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:05.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.593+0000 7fb483e00640 1 --2- 172.21.15.151:0/1209909843 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4840b4d10 0x7fb4841f4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:05.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.593+0000 7fb483e00640 1 -- 172.21.15.151:0/1209909843 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4840d1ec0 msgr2=0x7fb4841502b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:05.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.593+0000 7fb483e00640 1 --2- 172.21.15.151:0/1209909843 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4840d1ec0 0x7fb4841502b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:05.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.593+0000 7fb483e00640 1 -- 172.21.15.151:0/1209909843 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb48415fd60 con 0x7fb4840b28d0 2024-09-16T15:51:05.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.593+0000 7fb483e00640 1 --2- 172.21.15.151:0/1209909843 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4840b28d0 0x7fb4841f8710 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fb47400c3c0 tx=0x7fb47400c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:05.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.593+0000 7fb47be00640 1 -- 172.21.15.151:0/1209909843 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb474017070 con 0x7fb4840b28d0 2024-09-16T15:51:05.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.593+0000 7fb47be00640 1 -- 172.21.15.151:0/1209909843 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb4740035f0 con 0x7fb4840b28d0 2024-09-16T15:51:05.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.593+0000 7fb489c00640 1 -- 172.21.15.151:0/1209909843 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4841f4e20 con 0x7fb4840b28d0 2024-09-16T15:51:05.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.594+0000 7fb47be00640 1 -- 172.21.15.151:0/1209909843 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb474003a80 con 0x7fb4840b28d0 2024-09-16T15:51:05.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.594+0000 7fb489c00640 1 -- 172.21.15.151:0/1209909843 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4841f53e0 con 0x7fb4840b28d0 2024-09-16T15:51:05.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.595+0000 7fb489c00640 1 -- 172.21.15.151:0/1209909843 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb484160130 con 0x7fb4840b28d0 2024-09-16T15:51:05.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.596+0000 7fb47be00640 1 -- 172.21.15.151:0/1209909843 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7fb47401a070 con 0x7fb4840b28d0 2024-09-16T15:51:05.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.597+0000 7fb47be00640 1 --2- 172.21.15.151:0/1209909843 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb464079ed0 0x7fb46407c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:05.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.597+0000 7fb47be00640 1 -- 172.21.15.151:0/1209909843 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5765+0+0 (secure 0 0 0) 0x7fb4740988e0 con 0x7fb4840b28d0 2024-09-16T15:51:05.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.597+0000 7fb483400640 1 --2- 172.21.15.151:0/1209909843 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb464079ed0 0x7fb46407c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:05.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.598+0000 7fb483400640 1 --2- 172.21.15.151:0/1209909843 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb464079ed0 0x7fb46407c390 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fb4840ce1e0 tx=0x7fb470009110 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:05.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.601+0000 7fb47be00640 1 -- 172.21.15.151:0/1209909843 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb48415fd60 con 0x7fb4840b28d0 2024-09-16T15:51:05.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.820+0000 7fb489c00640 1 -- 172.21.15.151:0/1209909843 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb4840d22a0 con 0x7fb4840b28d0 2024-09-16T15:51:05.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.821+0000 7fb47be00640 1 -- 172.21.15.151:0/1209909843 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+5234 (secure 0 0 0) 0x7fb474062360 con 0x7fb4840b28d0 2024-09-16T15:51:05.823 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:51:05.823 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":66,"btime":"2024-09-16T15:51:05:403851+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6680,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2638998637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":2638998637},{"type":"v1","addr":"172.21.15.151:6835","nonce":2638998637}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":7330,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3105097249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3105097249},{"type":"v1","addr":"172.21.15.167:6836","nonce":3105097249}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":7333,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/613324294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":613324294},{"type":"v1","addr":"172.21.15.167:6837","nonce":613324294}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:51:01.789380+0000","modified":"2024-09-16T15:51:04.820629+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7899},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7899":{"gid":7899,"name":"c","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.151:6834/1174862515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1174862515},{"type":"v1","addr":"172.21.15.151:6834","nonce":1174862515}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7899,"qdb_cluster":[7899]},"id":7}]} 2024-09-16T15:51:05.823 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 66 2024-09-16T15:51:05.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.823+0000 7fb489c00640 1 -- 172.21.15.151:0/1209909843 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb464079ed0 msgr2=0x7fb46407c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:05.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.823+0000 7fb489c00640 1 --2- 172.21.15.151:0/1209909843 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb464079ed0 0x7fb46407c390 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fb4840ce1e0 tx=0x7fb470009110 comp rx=0 tx=0).stop 2024-09-16T15:51:05.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.823+0000 7fb489c00640 1 -- 172.21.15.151:0/1209909843 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4840b28d0 msgr2=0x7fb4841f8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:05.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.823+0000 7fb489c00640 1 --2- 172.21.15.151:0/1209909843 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4840b28d0 0x7fb4841f8710 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fb47400c3c0 tx=0x7fb47400c7c0 comp rx=0 tx=0).stop 2024-09-16T15:51:05.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.823+0000 7fb489c00640 1 -- 172.21.15.151:0/1209909843 shutdown_connections 2024-09-16T15:51:05.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.823+0000 7fb489c00640 1 --2- 172.21.15.151:0/1209909843 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb464079ed0 0x7fb46407c390 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:05.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.823+0000 7fb489c00640 1 --2- 172.21.15.151:0/1209909843 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4840d1ec0 0x7fb4841502b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:05.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.823+0000 7fb489c00640 1 --2- 172.21.15.151:0/1209909843 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4840b4d10 0x7fb4841f4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:05.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.823+0000 7fb489c00640 1 --2- 172.21.15.151:0/1209909843 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4840b28d0 0x7fb4841f8710 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:05.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.823+0000 7fb489c00640 1 -- 172.21.15.151:0/1209909843 >> 172.21.15.151:0/1209909843 conn(0x7fb4840d3de0 msgr2=0x7fb48415e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:05.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.824+0000 7fb489c00640 1 -- 172.21.15.151:0/1209909843 shutdown_connections 2024-09-16T15:51:05.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.824+0000 7fb489c00640 1 -- 172.21.15.151:0/1209909843 wait complete. 2024-09-16T15:51:05.841 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:51:05.999 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.997+0000 7f63df200640 1 -- 172.21.15.151:0/430166340 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63e00725d0 msgr2=0x7f63e00b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:05.999 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.997+0000 7f63df200640 1 --2- 172.21.15.151:0/430166340 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63e00725d0 0x7f63e00b28f0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f63cc009830 tx=0x7f63cc02a5c0 comp rx=0 tx=0).stop 2024-09-16T15:51:06.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.997+0000 7f63df200640 1 -- 172.21.15.151:0/430166340 shutdown_connections 2024-09-16T15:51:06.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.997+0000 7f63df200640 1 --2- 172.21.15.151:0/430166340 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63e00725d0 0x7f63e00b28f0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:06.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.997+0000 7f63df200640 1 -- 172.21.15.151:0/430166340 >> 172.21.15.151:0/430166340 conn(0x7f63e00cfa20 msgr2=0x7f63e00cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:06.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.997+0000 7f63df200640 1 -- 172.21.15.151:0/430166340 shutdown_connections 2024-09-16T15:51:06.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.997+0000 7f63df200640 1 -- 172.21.15.151:0/430166340 wait complete. 2024-09-16T15:51:06.001 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.999+0000 7f63df200640 1 Processor -- start 2024-09-16T15:51:06.001 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:05.999+0000 7f63df200640 1 -- start start 2024-09-16T15:51:06.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.000+0000 7f63df200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f63e00d0540 0x7f63e00cc430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:06.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.000+0000 7f63df200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63e015bc30 0x7f63e00c56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:06.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.000+0000 7f63df200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63e00b4590 0x7f63e00b4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:06.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.000+0000 7f63df200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f63e0074660 con 0x7f63e015bc30 2024-09-16T15:51:06.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.000+0000 7f63df200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f63e006fa30 con 0x7f63e00b4590 2024-09-16T15:51:06.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.000+0000 7f63df200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f63e01f8300 con 0x7f63e00d0540 2024-09-16T15:51:06.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.000+0000 7f63dd400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63e015bc30 0x7f63e00c56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:06.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.000+0000 7f63dd400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63e015bc30 0x7f63e00c56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52608/0 (socket says 172.21.15.151:52608) 2024-09-16T15:51:06.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.000+0000 7f63dd400640 1 -- 172.21.15.151:0/857976154 learned_addr learned my addr 172.21.15.151:0/857976154 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:51:06.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.000+0000 7f63dde00640 1 --2- 172.21.15.151:0/857976154 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f63e00d0540 0x7f63e00cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:06.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.000+0000 7f63de800640 1 --2- 172.21.15.151:0/857976154 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63e00b4590 0x7f63e00b4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:06.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.001+0000 7f63dd400640 1 -- 172.21.15.151:0/857976154 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f63e00d0540 msgr2=0x7f63e00cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:06.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.001+0000 7f63dd400640 1 --2- 172.21.15.151:0/857976154 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f63e00d0540 0x7f63e00cc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:06.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.001+0000 7f63dd400640 1 -- 172.21.15.151:0/857976154 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63e00b4590 msgr2=0x7f63e00b4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:06.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.001+0000 7f63dd400640 1 --2- 172.21.15.151:0/857976154 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63e00b4590 0x7f63e00b4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:06.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.001+0000 7f63dd400640 1 -- 172.21.15.151:0/857976154 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63e01f85e0 con 0x7f63e015bc30 2024-09-16T15:51:06.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.001+0000 7f63dde00640 1 --2- 172.21.15.151:0/857976154 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f63e00d0540 0x7f63e00cc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:51:06.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.001+0000 7f63de800640 1 --2- 172.21.15.151:0/857976154 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63e00b4590 0x7f63e00b4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:51:06.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.001+0000 7f63dd400640 1 --2- 172.21.15.151:0/857976154 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63e015bc30 0x7f63e00c56a0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f63cc000c00 tx=0x7f63cc032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:06.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.001+0000 7f63d6000640 1 -- 172.21.15.151:0/857976154 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63cc040070 con 0x7f63e015bc30 2024-09-16T15:51:06.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.001+0000 7f63d6000640 1 -- 172.21.15.151:0/857976154 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f63cc0038e0 con 0x7f63e015bc30 2024-09-16T15:51:06.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.001+0000 7f63d6000640 1 -- 172.21.15.151:0/857976154 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63cc003d30 con 0x7f63e015bc30 2024-09-16T15:51:06.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.001+0000 7f63df200640 1 -- 172.21.15.151:0/857976154 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63e01f88c0 con 0x7f63e015bc30 2024-09-16T15:51:06.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.001+0000 7f63df200640 1 -- 172.21.15.151:0/857976154 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63e01f8e80 con 0x7f63e015bc30 2024-09-16T15:51:06.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.002+0000 7f63df200640 1 -- 172.21.15.151:0/857976154 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63e00b28f0 con 0x7f63e015bc30 2024-09-16T15:51:06.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.004+0000 7f63d6000640 1 -- 172.21.15.151:0/857976154 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7f63cc039040 con 0x7f63e015bc30 2024-09-16T15:51:06.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.004+0000 7f63d6000640 1 --2- 172.21.15.151:0/857976154 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f63c0079f00 0x7f63c007c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:06.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.005+0000 7f63d6000640 1 -- 172.21.15.151:0/857976154 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5765+0+0 (secure 0 0 0) 0x7f63cc0bab60 con 0x7f63e015bc30 2024-09-16T15:51:06.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.005+0000 7f63dde00640 1 --2- 172.21.15.151:0/857976154 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f63c0079f00 0x7f63c007c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:06.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.006+0000 7f63dde00640 1 --2- 172.21.15.151:0/857976154 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f63c0079f00 0x7f63c007c3c0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f63d00012e0 tx=0x7f63d0009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:06.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.009+0000 7f63d6000640 1 -- 172.21.15.151:0/857976154 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f63cc084cd0 con 0x7f63e015bc30 2024-09-16T15:51:06.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.193+0000 7f63df200640 1 -- 172.21.15.151:0/857976154 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f63e01e9c60 con 0x7f63e015bc30 2024-09-16T15:51:06.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.194+0000 7f63d6000640 1 -- 172.21.15.151:0/857976154 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) ==== 74+0+16826 (secure 0 0 0) 0x7f63cc0845d0 con 0x7f63e015bc30 2024-09-16T15:51:06.197 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:51:06.197 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":104,"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","created":"2024-09-16T15:38:56.348535+0000","modified":"2024-09-16T15:51:03.372350+0000","last_up_change":"2024-09-16T15:39:10.681384+0000","last_in_change":"2024-09-16T15:39:00.544477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:39:11.602252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:50:59.568133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-16T15:51:00.775162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5524076-a903-4d0a-af6a-caf474c4765d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6809","nonce":3111342741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6811","nonce":3111342741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6815","nonce":3111342741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6813","nonce":3111342741}]},"public_addr":"172.21.15.151:6809/3111342741","cluster_addr":"172.21.15.151:6811/3111342741","heartbeat_back_addr":"172.21.15.151:6815/3111342741","heartbeat_front_addr":"172.21.15.151:6813/3111342741","state":["exists","up"]},{"osd":1,"uuid":"dd25925f-0ae1-494e-8529-4dc205759db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6801","nonce":3368304087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6803","nonce":3368304087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6807","nonce":3368304087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6805","nonce":3368304087}]},"public_addr":"172.21.15.151:6801/3368304087","cluster_addr":"172.21.15.151:6803/3368304087","heartbeat_back_addr":"172.21.15.151:6807/3368304087","heartbeat_front_addr":"172.21.15.151:6805/3368304087","state":["exists","up"]},{"osd":2,"uuid":"c2f18115-013d-4da6-a0fe-cb978f4494b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6823","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6825","nonce":3079373271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6827","nonce":3079373271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6831","nonce":3079373271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6829","nonce":3079373271}]},"public_addr":"172.21.15.151:6825/3079373271","cluster_addr":"172.21.15.151:6827/3079373271","heartbeat_back_addr":"172.21.15.151:6831/3079373271","heartbeat_front_addr":"172.21.15.151:6829/3079373271","state":["exists","up"]},{"osd":3,"uuid":"36edb718-2fcb-4d88-8ddc-c321eed4279d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6817","nonce":1142509041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6819","nonce":1142509041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6824","nonce":1142509041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6821","nonce":1142509041}]},"public_addr":"172.21.15.151:6817/1142509041","cluster_addr":"172.21.15.151:6819/1142509041","heartbeat_back_addr":"172.21.15.151:6824/1142509041","heartbeat_front_addr":"172.21.15.151:6821/1142509041","state":["exists","up"]},{"osd":4,"uuid":"64d33708-cd0a-4236-abe3-1b646e82ab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6811","nonce":2735495271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6813","nonce":2735495271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6817","nonce":2735495271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6815","nonce":2735495271}]},"public_addr":"172.21.15.167:6811/2735495271","cluster_addr":"172.21.15.167:6813/2735495271","heartbeat_back_addr":"172.21.15.167:6817/2735495271","heartbeat_front_addr":"172.21.15.167:6815/2735495271","state":["exists","up"]},{"osd":5,"uuid":"f14b44a9-5910-458c-b518-09a8d0de94de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6827","nonce":1494916572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6829","nonce":1494916572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6833","nonce":1494916572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6831","nonce":1494916572}]},"public_addr":"172.21.15.167:6827/1494916572","cluster_addr":"172.21.15.167:6829/1494916572","heartbeat_back_addr":"172.21.15.167:6833/1494916572","heartbeat_front_addr":"172.21.15.167:6831/1494916572","state":["exists","up"]},{"osd":6,"uuid":"39335545-2654-4710-84b0-a9aa47cfcbba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6819","nonce":2538273273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6821","nonce":2538273273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6825","nonce":2538273273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6823","nonce":2538273273}]},"public_addr":"172.21.15.167:6819/2538273273","cluster_addr":"172.21.15.167:6821/2538273273","heartbeat_back_addr":"172.21.15.167:6825/2538273273","heartbeat_front_addr":"172.21.15.167:6823/2538273273","state":["exists","up"]},{"osd":7,"uuid":"4f8bfc81-53dd-4339-ac22-1a4007d9f963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6803","nonce":265072227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6805","nonce":265072227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6809","nonce":265072227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6807","nonce":265072227}]},"public_addr":"172.21.15.167:6803/265072227","cluster_addr":"172.21.15.167:6805/265072227","heartbeat_back_addr":"172.21.15.167:6809/265072227","heartbeat_front_addr":"172.21.15.167:6807/265072227","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.150359+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.084358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.722368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.332809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.320709+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.395153+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.935250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.801443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:51:06.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.196+0000 7f63df200640 1 -- 172.21.15.151:0/857976154 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f63c0079f00 msgr2=0x7f63c007c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:06.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.196+0000 7f63df200640 1 --2- 172.21.15.151:0/857976154 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f63c0079f00 0x7f63c007c3c0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f63d00012e0 tx=0x7f63d0009040 comp rx=0 tx=0).stop 2024-09-16T15:51:06.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.196+0000 7f63df200640 1 -- 172.21.15.151:0/857976154 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63e015bc30 msgr2=0x7f63e00c56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:06.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.196+0000 7f63df200640 1 --2- 172.21.15.151:0/857976154 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63e015bc30 0x7f63e00c56a0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f63cc000c00 tx=0x7f63cc032c10 comp rx=0 tx=0).stop 2024-09-16T15:51:06.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.197+0000 7f63df200640 1 -- 172.21.15.151:0/857976154 shutdown_connections 2024-09-16T15:51:06.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.197+0000 7f63df200640 1 --2- 172.21.15.151:0/857976154 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f63c0079f00 0x7f63c007c3c0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:06.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.197+0000 7f63df200640 1 --2- 172.21.15.151:0/857976154 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63e00b4590 0x7f63e00b4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:06.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.197+0000 7f63df200640 1 --2- 172.21.15.151:0/857976154 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63e015bc30 0x7f63e00c56a0 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:06.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.197+0000 7f63df200640 1 --2- 172.21.15.151:0/857976154 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f63e00d0540 0x7f63e00cc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:06.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.197+0000 7f63df200640 1 -- 172.21.15.151:0/857976154 >> 172.21.15.151:0/857976154 conn(0x7f63e00cfa20 msgr2=0x7f63e0152e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:06.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.197+0000 7f63df200640 1 -- 172.21.15.151:0/857976154 shutdown_connections 2024-09-16T15:51:06.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.197+0000 7f63df200640 1 -- 172.21.15.151:0/857976154 wait complete. 2024-09-16T15:51:06.215 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T15:51:06.215 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:51:06.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.370+0000 7ff5fce00640 1 -- 172.21.15.151:0/2870081333 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff5f815f5a0 msgr2=0x7ff5f80b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:06.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.370+0000 7ff5fce00640 1 --2- 172.21.15.151:0/2870081333 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff5f815f5a0 0x7ff5f80b2440 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8005fd0 tx=0x7ff5e802a4a0 comp rx=0 tx=0).stop 2024-09-16T15:51:06.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.370+0000 7ff5fce00640 1 -- 172.21.15.151:0/2870081333 shutdown_connections 2024-09-16T15:51:06.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.370+0000 7ff5fce00640 1 --2- 172.21.15.151:0/2870081333 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff5f8073fb0 0x7ff5f8074440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:06.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.370+0000 7ff5fce00640 1 --2- 172.21.15.151:0/2870081333 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff5f815f5a0 0x7ff5f80b2440 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:06.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.370+0000 7ff5fce00640 1 -- 172.21.15.151:0/2870081333 >> 172.21.15.151:0/2870081333 conn(0x7ff5f80b1950 msgr2=0x7ff5f80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:06.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.371+0000 7ff5fce00640 1 -- 172.21.15.151:0/2870081333 shutdown_connections 2024-09-16T15:51:06.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.371+0000 7ff5fce00640 1 -- 172.21.15.151:0/2870081333 wait complete. 2024-09-16T15:51:06.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.372+0000 7ff5fce00640 1 Processor -- start 2024-09-16T15:51:06.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.372+0000 7ff5fce00640 1 -- start start 2024-09-16T15:51:06.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.373+0000 7ff5fce00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff5f8073fb0 0x7ff5f80cc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:06.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.373+0000 7ff5fce00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5f80ccf00 0x7ff5f8150890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:06.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.373+0000 7ff5fce00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff5f815f5a0 0x7ff5f81f80c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:06.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.373+0000 7ff5fce00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f806f2e0 con 0x7ff5f815f5a0 2024-09-16T15:51:06.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.373+0000 7ff5fce00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f806f160 con 0x7ff5f80ccf00 2024-09-16T15:51:06.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.373+0000 7ff5fce00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7ff5f806f460 con 0x7ff5f8073fb0 2024-09-16T15:51:06.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.373+0000 7ff5f6800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff5f815f5a0 0x7ff5f81f80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:06.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.373+0000 7ff5f5400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5f80ccf00 0x7ff5f8150890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:06.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.373+0000 7ff5f5e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff5f8073fb0 0x7ff5f80cc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:06.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.374+0000 7ff5f6800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff5f815f5a0 0x7ff5f81f80c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52620/0 (socket says 172.21.15.151:52620) 2024-09-16T15:51:06.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.374+0000 7ff5f5e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff5f8073fb0 0x7ff5f80cc1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.151:36432/0 (socket says 172.21.15.151:36432) 2024-09-16T15:51:06.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.374+0000 7ff5f5400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5f80ccf00 0x7ff5f8150890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.151:39906/0 (socket says 172.21.15.151:39906) 2024-09-16T15:51:06.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.374+0000 7ff5f6800640 1 -- 172.21.15.151:0/2320569723 learned_addr learned my addr 172.21.15.151:0/2320569723 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:51:06.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.374+0000 7ff5f6800640 1 -- 172.21.15.151:0/2320569723 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff5f8073fb0 msgr2=0x7ff5f80cc1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:06.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.374+0000 7ff5f6800640 1 --2- 172.21.15.151:0/2320569723 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff5f8073fb0 0x7ff5f80cc1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:06.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.374+0000 7ff5f6800640 1 -- 172.21.15.151:0/2320569723 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5f80ccf00 msgr2=0x7ff5f8150890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:06.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.374+0000 7ff5f6800640 1 --2- 172.21.15.151:0/2320569723 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5f80ccf00 0x7ff5f8150890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:06.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.374+0000 7ff5f6800640 1 -- 172.21.15.151:0/2320569723 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5f80ca280 con 0x7ff5f815f5a0 2024-09-16T15:51:06.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.374+0000 7ff5f6800640 1 --2- 172.21.15.151:0/2320569723 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff5f815f5a0 0x7ff5f81f80c0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7ff5f000b970 tx=0x7ff5f000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:06.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.374+0000 7ff5ee000640 1 -- 172.21.15.151:0/2320569723 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f0019070 con 0x7ff5f815f5a0 2024-09-16T15:51:06.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.375+0000 7ff5ee000640 1 -- 172.21.15.151:0/2320569723 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff5f000cc50 con 0x7ff5f815f5a0 2024-09-16T15:51:06.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.375+0000 7ff5ee000640 1 -- 172.21.15.151:0/2320569723 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f00146a0 con 0x7ff5f815f5a0 2024-09-16T15:51:06.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.375+0000 7ff5fce00640 1 -- 172.21.15.151:0/2320569723 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5f80c9520 con 0x7ff5f815f5a0 2024-09-16T15:51:06.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.375+0000 7ff5fce00640 1 -- 172.21.15.151:0/2320569723 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5f80c9f10 con 0x7ff5f815f5a0 2024-09-16T15:51:06.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.376+0000 7ff5fce00640 1 -- 172.21.15.151:0/2320569723 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5f80c9a40 con 0x7ff5f815f5a0 2024-09-16T15:51:06.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.377+0000 7ff5ee000640 1 -- 172.21.15.151:0/2320569723 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7ff5f001c070 con 0x7ff5f815f5a0 2024-09-16T15:51:06.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.379+0000 7ff5ee000640 1 --2- 172.21.15.151:0/2320569723 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff5d8079ed0 0x7ff5d807c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:06.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.379+0000 7ff5ee000640 1 -- 172.21.15.151:0/2320569723 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5765+0+0 (secure 0 0 0) 0x7ff5f00992b0 con 0x7ff5f815f5a0 2024-09-16T15:51:06.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.379+0000 7ff5f5e00640 1 --2- 172.21.15.151:0/2320569723 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff5d8079ed0 0x7ff5d807c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:06.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.380+0000 7ff5f5e00640 1 --2- 172.21.15.151:0/2320569723 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff5d8079ed0 0x7ff5d807c390 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8033000 tx=0x7ff5e8004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:06.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.383+0000 7ff5ee000640 1 -- 172.21.15.151:0/2320569723 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff5f80ca280 con 0x7ff5f815f5a0 2024-09-16T15:51:06.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.571+0000 7ff5fce00640 1 -- 172.21.15.151:0/2320569723 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff5f806f9a0 con 0x7ff5d8079ed0 2024-09-16T15:51:06.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.577+0000 7ff5ee000640 1 -- 172.21.15.151:0/2320569723 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+248219 (secure 0 0 0) 0x7ff5f806f9a0 con 0x7ff5d8079ed0 2024-09-16T15:51:06.582 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:51:06.583 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-09-16T15:51:06.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.581+0000 7ff5fce00640 1 -- 172.21.15.151:0/2320569723 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff5d8079ed0 msgr2=0x7ff5d807c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:06.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.581+0000 7ff5fce00640 1 --2- 172.21.15.151:0/2320569723 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff5d8079ed0 0x7ff5d807c390 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8033000 tx=0x7ff5e8004730 comp rx=0 tx=0).stop 2024-09-16T15:51:06.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.581+0000 7ff5fce00640 1 -- 172.21.15.151:0/2320569723 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff5f815f5a0 msgr2=0x7ff5f81f80c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:06.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.581+0000 7ff5fce00640 1 --2- 172.21.15.151:0/2320569723 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff5f815f5a0 0x7ff5f81f80c0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7ff5f000b970 tx=0x7ff5f000be30 comp rx=0 tx=0).stop 2024-09-16T15:51:06.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.581+0000 7ff5fce00640 1 -- 172.21.15.151:0/2320569723 shutdown_connections 2024-09-16T15:51:06.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.581+0000 7ff5fce00640 1 --2- 172.21.15.151:0/2320569723 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff5d8079ed0 0x7ff5d807c390 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:06.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.581+0000 7ff5fce00640 1 --2- 172.21.15.151:0/2320569723 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5f80ccf00 0x7ff5f8150890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:06.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.581+0000 7ff5fce00640 1 --2- 172.21.15.151:0/2320569723 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff5f815f5a0 0x7ff5f81f80c0 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:06.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.581+0000 7ff5fce00640 1 --2- 172.21.15.151:0/2320569723 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff5f8073fb0 0x7ff5f80cc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:06.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.581+0000 7ff5fce00640 1 -- 172.21.15.151:0/2320569723 >> 172.21.15.151:0/2320569723 conn(0x7ff5f80b1950 msgr2=0x7ff5f80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:06.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.581+0000 7ff5fce00640 1 -- 172.21.15.151:0/2320569723 shutdown_connections 2024-09-16T15:51:06.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.581+0000 7ff5fce00640 1 -- 172.21.15.151:0/2320569723 wait complete. 2024-09-16T15:51:06.599 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":455,"stamp":"2024-09-16T15:51:05.880418+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590950,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":121,"num_write_kb":1629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":89,"ondisk_log_size":89,"up":188,"acting":188,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":27,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1065132,"kb_used_data":10920,"kb_used_omap":12,"kb_used_meta":1054003,"kb_avail":753909588,"statfs":{"total":773094113280,"available":772003418112,"internally_reserved":0,"allocated":11182080,"data_stored":5520116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12738,"internal_metadata":1079299646},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":582,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.518821"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T15:51:02.391024+0000","last_change":"2024-09-16T15:51:02.386431+0000","last_active":"2024-09-16T15:51:02.391024+0000","last_peered":"2024-09-16T15:51:02.391024+0000","last_clean":"2024-09-16T15:51:02.391024+0000","last_became_active":"2024-09-16T15:51:02.386126+0000","last_became_peered":"2024-09-16T15:51:02.386126+0000","last_unstale":"2024-09-16T15:51:02.391024+0000","last_undegraded":"2024-09-16T15:51:02.391024+0000","last_fullsized":"2024-09-16T15:51:02.391024+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:25:09.365288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T15:51:02.400467+0000","last_change":"2024-09-16T15:51:02.388621+0000","last_active":"2024-09-16T15:51:02.400467+0000","last_peered":"2024-09-16T15:51:02.400467+0000","last_clean":"2024-09-16T15:51:02.400467+0000","last_became_active":"2024-09-16T15:51:02.388294+0000","last_became_peered":"2024-09-16T15:51:02.388294+0000","last_unstale":"2024-09-16T15:51:02.400467+0000","last_undegraded":"2024-09-16T15:51:02.400467+0000","last_fullsized":"2024-09-16T15:51:02.400467+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:43:28.544765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.369847+0000","last_change":"2024-09-16T15:51:01.369847+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.369847+0000","last_undegraded":"2024-09-16T15:51:01.369847+0000","last_fullsized":"2024-09-16T15:51:01.369847+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T15:51:02.387475+0000","last_change":"2024-09-16T15:51:02.385219+0000","last_active":"2024-09-16T15:51:02.387475+0000","last_peered":"2024-09-16T15:51:02.387475+0000","last_clean":"2024-09-16T15:51:02.387475+0000","last_became_active":"2024-09-16T15:51:02.384310+0000","last_became_peered":"2024-09-16T15:51:02.384310+0000","last_unstale":"2024-09-16T15:51:02.387475+0000","last_undegraded":"2024-09-16T15:51:02.387475+0000","last_fullsized":"2024-09-16T15:51:02.387475+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:48:47.559516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.374870+0000","last_change":"2024-09-16T15:51:01.374870+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.374870+0000","last_undegraded":"2024-09-16T15:51:01.374870+0000","last_fullsized":"2024-09-16T15:51:01.374870+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.374553+0000","last_change":"2024-09-16T15:51:01.374553+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.374553+0000","last_undegraded":"2024-09-16T15:51:01.374553+0000","last_fullsized":"2024-09-16T15:51:01.374553+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.372119+0000","last_change":"2024-09-16T15:51:01.372119+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.372119+0000","last_undegraded":"2024-09-16T15:51:01.372119+0000","last_fullsized":"2024-09-16T15:51:01.372119+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.370748+0000","last_change":"2024-09-16T15:51:01.370748+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.370748+0000","last_undegraded":"2024-09-16T15:51:01.370748+0000","last_fullsized":"2024-09-16T15:51:01.370748+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.373077+0000","last_change":"2024-09-16T15:51:01.373077+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.373077+0000","last_undegraded":"2024-09-16T15:51:01.373077+0000","last_fullsized":"2024-09-16T15:51:01.373077+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:51:01.355970+0000","last_change":"2024-09-16T15:51:01.355970+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.355970+0000","last_undegraded":"2024-09-16T15:51:01.355970+0000","last_fullsized":"2024-09-16T15:51:01.355970+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.374982+0000","last_change":"2024-09-16T15:51:01.374982+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.374982+0000","last_undegraded":"2024-09-16T15:51:01.374982+0000","last_fullsized":"2024-09-16T15:51:01.374982+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.373278+0000","last_change":"2024-09-16T15:51:01.373278+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.373278+0000","last_undegraded":"2024-09-16T15:51:01.373278+0000","last_fullsized":"2024-09-16T15:51:01.373278+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309716+0000","last_change":"2024-09-16T15:51:02.407269+0000","last_active":"2024-09-16T15:51:04.309716+0000","last_peered":"2024-09-16T15:51:04.309716+0000","last_clean":"2024-09-16T15:51:04.309716+0000","last_became_active":"2024-09-16T15:51:02.406943+0000","last_became_peered":"2024-09-16T15:51:02.406943+0000","last_unstale":"2024-09-16T15:51:04.309716+0000","last_undegraded":"2024-09-16T15:51:04.309716+0000","last_fullsized":"2024-09-16T15:51:04.309716+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:29:33.561032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T15:51:02.405765+0000","last_change":"2024-09-16T15:51:02.398129+0000","last_active":"2024-09-16T15:51:02.405765+0000","last_peered":"2024-09-16T15:51:02.405765+0000","last_clean":"2024-09-16T15:51:02.405765+0000","last_became_active":"2024-09-16T15:51:02.397834+0000","last_became_peered":"2024-09-16T15:51:02.397834+0000","last_unstale":"2024-09-16T15:51:02.405765+0000","last_undegraded":"2024-09-16T15:51:02.405765+0000","last_fullsized":"2024-09-16T15:51:02.405765+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:35:17.388414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T15:51:02.404562+0000","last_change":"2024-09-16T15:51:02.396682+0000","last_active":"2024-09-16T15:51:02.404562+0000","last_peered":"2024-09-16T15:51:02.404562+0000","last_clean":"2024-09-16T15:51:02.404562+0000","last_became_active":"2024-09-16T15:51:02.396412+0000","last_became_peered":"2024-09-16T15:51:02.396412+0000","last_unstale":"2024-09-16T15:51:02.404562+0000","last_undegraded":"2024-09-16T15:51:02.404562+0000","last_fullsized":"2024-09-16T15:51:02.404562+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:52:38.771302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.371468+0000","last_change":"2024-09-16T15:51:01.371468+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.371468+0000","last_undegraded":"2024-09-16T15:51:01.371468+0000","last_fullsized":"2024-09-16T15:51:01.371468+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.379608+0000","last_change":"2024-09-16T15:51:01.379608+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.379608+0000","last_undegraded":"2024-09-16T15:51:01.379608+0000","last_fullsized":"2024-09-16T15:51:01.379608+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T15:51:02.404957+0000","last_change":"2024-09-16T15:51:02.399241+0000","last_active":"2024-09-16T15:51:02.404957+0000","last_peered":"2024-09-16T15:51:02.404957+0000","last_clean":"2024-09-16T15:51:02.404957+0000","last_became_active":"2024-09-16T15:51:02.398973+0000","last_became_peered":"2024-09-16T15:51:02.398973+0000","last_unstale":"2024-09-16T15:51:02.404957+0000","last_undegraded":"2024-09-16T15:51:02.404957+0000","last_fullsized":"2024-09-16T15:51:02.404957+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:46:06.817591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.372608+0000","last_change":"2024-09-16T15:51:01.372608+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.372608+0000","last_undegraded":"2024-09-16T15:51:01.372608+0000","last_fullsized":"2024-09-16T15:51:01.372608+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.370913+0000","last_change":"2024-09-16T15:51:01.370913+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.370913+0000","last_undegraded":"2024-09-16T15:51:01.370913+0000","last_fullsized":"2024-09-16T15:51:01.370913+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.371225+0000","last_change":"2024-09-16T15:51:01.371225+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.371225+0000","last_undegraded":"2024-09-16T15:51:01.371225+0000","last_fullsized":"2024-09-16T15:51:01.371225+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.368817+0000","last_change":"2024-09-16T15:51:01.368817+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.368817+0000","last_undegraded":"2024-09-16T15:51:01.368817+0000","last_fullsized":"2024-09-16T15:51:01.368817+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T15:51:02.405411+0000","last_change":"2024-09-16T15:51:02.399544+0000","last_active":"2024-09-16T15:51:02.405411+0000","last_peered":"2024-09-16T15:51:02.405411+0000","last_clean":"2024-09-16T15:51:02.405411+0000","last_became_active":"2024-09-16T15:51:02.396515+0000","last_became_peered":"2024-09-16T15:51:02.396515+0000","last_unstale":"2024-09-16T15:51:02.405411+0000","last_undegraded":"2024-09-16T15:51:02.405411+0000","last_fullsized":"2024-09-16T15:51:02.405411+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:05:52.491919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.371334+0000","last_change":"2024-09-16T15:51:01.371334+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.371334+0000","last_undegraded":"2024-09-16T15:51:01.371334+0000","last_fullsized":"2024-09-16T15:51:01.371334+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:51:01.355970+0000","last_change":"2024-09-16T15:51:01.355970+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.355970+0000","last_undegraded":"2024-09-16T15:51:01.355970+0000","last_fullsized":"2024-09-16T15:51:01.355970+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309644+0000","last_change":"2024-09-16T15:51:02.401422+0000","last_active":"2024-09-16T15:51:04.309644+0000","last_peered":"2024-09-16T15:51:04.309644+0000","last_clean":"2024-09-16T15:51:04.309644+0000","last_became_active":"2024-09-16T15:51:02.400804+0000","last_became_peered":"2024-09-16T15:51:02.400804+0000","last_unstale":"2024-09-16T15:51:04.309644+0000","last_undegraded":"2024-09-16T15:51:04.309644+0000","last_fullsized":"2024-09-16T15:51:04.309644+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:54:09.898078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.371083+0000","last_change":"2024-09-16T15:51:01.371083+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.371083+0000","last_undegraded":"2024-09-16T15:51:01.371083+0000","last_fullsized":"2024-09-16T15:51:01.371083+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.368057+0000","last_change":"2024-09-16T15:51:01.368057+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.368057+0000","last_undegraded":"2024-09-16T15:51:01.368057+0000","last_fullsized":"2024-09-16T15:51:01.368057+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309979+0000","last_change":"2024-09-16T15:51:02.405833+0000","last_active":"2024-09-16T15:51:04.309979+0000","last_peered":"2024-09-16T15:51:04.309979+0000","last_clean":"2024-09-16T15:51:04.309979+0000","last_became_active":"2024-09-16T15:51:02.404364+0000","last_became_peered":"2024-09-16T15:51:02.404364+0000","last_unstale":"2024-09-16T15:51:04.309979+0000","last_undegraded":"2024-09-16T15:51:04.309979+0000","last_fullsized":"2024-09-16T15:51:04.309979+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:47:54.421341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309062+0000","last_change":"2024-09-16T15:51:02.402965+0000","last_active":"2024-09-16T15:51:04.309062+0000","last_peered":"2024-09-16T15:51:04.309062+0000","last_clean":"2024-09-16T15:51:04.309062+0000","last_became_active":"2024-09-16T15:51:02.401107+0000","last_became_peered":"2024-09-16T15:51:02.401107+0000","last_unstale":"2024-09-16T15:51:04.309062+0000","last_undegraded":"2024-09-16T15:51:04.309062+0000","last_fullsized":"2024-09-16T15:51:04.309062+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:24:38.932170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.368886+0000","last_change":"2024-09-16T15:51:01.368886+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.368886+0000","last_undegraded":"2024-09-16T15:51:01.368886+0000","last_fullsized":"2024-09-16T15:51:01.368886+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T15:51:02.387143+0000","last_change":"2024-09-16T15:51:02.384379+0000","last_active":"2024-09-16T15:51:02.387143+0000","last_peered":"2024-09-16T15:51:02.387143+0000","last_clean":"2024-09-16T15:51:02.387143+0000","last_became_active":"2024-09-16T15:51:02.384012+0000","last_became_peered":"2024-09-16T15:51:02.384012+0000","last_unstale":"2024-09-16T15:51:02.387143+0000","last_undegraded":"2024-09-16T15:51:02.387143+0000","last_fullsized":"2024-09-16T15:51:02.387143+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:04:09.954852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.382918+0000","last_change":"2024-09-16T15:51:01.382918+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.382918+0000","last_undegraded":"2024-09-16T15:51:01.382918+0000","last_fullsized":"2024-09-16T15:51:01.382918+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1f","version":"103'4","reported_seq":19,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309998+0000","last_change":"2024-09-16T15:51:01.375200+0000","last_active":"2024-09-16T15:51:04.309998+0000","last_peered":"2024-09-16T15:51:04.309998+0000","last_clean":"2024-09-16T15:51:04.309998+0000","last_became_active":"2024-09-16T15:51:01.374919+0000","last_became_peered":"2024-09-16T15:51:01.374919+0000","last_unstale":"2024-09-16T15:51:04.309998+0000","last_undegraded":"2024-09-16T15:51:04.309998+0000","last_fullsized":"2024-09-16T15:51:04.309998+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:36:44.338407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.310065+0000","last_change":"2024-09-16T15:51:02.402988+0000","last_active":"2024-09-16T15:51:04.310065+0000","last_peered":"2024-09-16T15:51:04.310065+0000","last_clean":"2024-09-16T15:51:04.310065+0000","last_became_active":"2024-09-16T15:51:02.402708+0000","last_became_peered":"2024-09-16T15:51:02.402708+0000","last_unstale":"2024-09-16T15:51:04.310065+0000","last_undegraded":"2024-09-16T15:51:04.310065+0000","last_fullsized":"2024-09-16T15:51:04.310065+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:14:48.952444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.392214+0000","last_change":"2024-09-16T15:51:01.391174+0000","last_active":"2024-09-16T15:51:01.392214+0000","last_peered":"2024-09-16T15:51:01.392214+0000","last_clean":"2024-09-16T15:51:01.392214+0000","last_became_active":"2024-09-16T15:51:01.390904+0000","last_became_peered":"2024-09-16T15:51:01.390904+0000","last_unstale":"2024-09-16T15:51:01.392214+0000","last_undegraded":"2024-09-16T15:51:01.392214+0000","last_fullsized":"2024-09-16T15:51:01.392214+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:15:52.955571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T15:51:02.386883+0000","last_change":"2024-09-16T15:51:02.382876+0000","last_active":"2024-09-16T15:51:02.386883+0000","last_peered":"2024-09-16T15:51:02.386883+0000","last_clean":"2024-09-16T15:51:02.386883+0000","last_became_active":"2024-09-16T15:51:02.382617+0000","last_became_peered":"2024-09-16T15:51:02.382617+0000","last_unstale":"2024-09-16T15:51:02.386883+0000","last_undegraded":"2024-09-16T15:51:02.386883+0000","last_fullsized":"2024-09-16T15:51:02.386883+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:19:48.933205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.390016+0000","last_change":"2024-09-16T15:51:01.388774+0000","last_active":"2024-09-16T15:51:01.390016+0000","last_peered":"2024-09-16T15:51:01.390016+0000","last_clean":"2024-09-16T15:51:01.390016+0000","last_became_active":"2024-09-16T15:51:01.388500+0000","last_became_peered":"2024-09-16T15:51:01.388500+0000","last_unstale":"2024-09-16T15:51:01.390016+0000","last_undegraded":"2024-09-16T15:51:01.390016+0000","last_fullsized":"2024-09-16T15:51:01.390016+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:08:59.734743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:51:01.355970+0000","last_change":"2024-09-16T15:51:01.355970+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.355970+0000","last_undegraded":"2024-09-16T15:51:01.355970+0000","last_fullsized":"2024-09-16T15:51:01.355970+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1c","version":"103'2","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T15:51:02.414797+0000","last_change":"2024-09-16T15:51:01.390985+0000","last_active":"2024-09-16T15:51:02.414797+0000","last_peered":"2024-09-16T15:51:02.414797+0000","last_clean":"2024-09-16T15:51:02.414797+0000","last_became_active":"2024-09-16T15:51:01.390463+0000","last_became_peered":"2024-09-16T15:51:01.390463+0000","last_unstale":"2024-09-16T15:51:02.414797+0000","last_undegraded":"2024-09-16T15:51:02.414797+0000","last_fullsized":"2024-09-16T15:51:02.414797+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:57:05.221628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.370455+0000","last_change":"2024-09-16T15:51:01.370455+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.370455+0000","last_undegraded":"2024-09-16T15:51:01.370455+0000","last_fullsized":"2024-09-16T15:51:01.370455+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1b","version":"103'1","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T15:51:02.411975+0000","last_change":"2024-09-16T15:51:01.387614+0000","last_active":"2024-09-16T15:51:02.411975+0000","last_peered":"2024-09-16T15:51:02.411975+0000","last_clean":"2024-09-16T15:51:02.411975+0000","last_became_active":"2024-09-16T15:51:01.387163+0000","last_became_peered":"2024-09-16T15:51:01.387163+0000","last_unstale":"2024-09-16T15:51:02.411975+0000","last_undegraded":"2024-09-16T15:51:02.411975+0000","last_fullsized":"2024-09-16T15:51:02.411975+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:48:24.298645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.370317+0000","last_change":"2024-09-16T15:51:01.370317+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.370317+0000","last_undegraded":"2024-09-16T15:51:01.370317+0000","last_fullsized":"2024-09-16T15:51:01.370317+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.383140+0000","last_change":"2024-09-16T15:51:01.378848+0000","last_active":"2024-09-16T15:51:01.383140+0000","last_peered":"2024-09-16T15:51:01.383140+0000","last_clean":"2024-09-16T15:51:01.383140+0000","last_became_active":"2024-09-16T15:51:01.377621+0000","last_became_peered":"2024-09-16T15:51:01.377621+0000","last_unstale":"2024-09-16T15:51:01.383140+0000","last_undegraded":"2024-09-16T15:51:01.383140+0000","last_fullsized":"2024-09-16T15:51:01.383140+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:28:49.058738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.308778+0000","last_change":"2024-09-16T15:51:02.405495+0000","last_active":"2024-09-16T15:51:04.308778+0000","last_peered":"2024-09-16T15:51:04.308778+0000","last_clean":"2024-09-16T15:51:04.308778+0000","last_became_active":"2024-09-16T15:51:02.405222+0000","last_became_peered":"2024-09-16T15:51:02.405222+0000","last_unstale":"2024-09-16T15:51:04.308778+0000","last_undegraded":"2024-09-16T15:51:04.308778+0000","last_fullsized":"2024-09-16T15:51:04.308778+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:22:02.790695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.378694+0000","last_change":"2024-09-16T15:51:01.372791+0000","last_active":"2024-09-16T15:51:01.378694+0000","last_peered":"2024-09-16T15:51:01.378694+0000","last_clean":"2024-09-16T15:51:01.378694+0000","last_became_active":"2024-09-16T15:51:01.372493+0000","last_became_peered":"2024-09-16T15:51:01.372493+0000","last_unstale":"2024-09-16T15:51:01.378694+0000","last_undegraded":"2024-09-16T15:51:01.378694+0000","last_fullsized":"2024-09-16T15:51:01.378694+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:24:18.852411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.369717+0000","last_change":"2024-09-16T15:51:01.369717+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.369717+0000","last_undegraded":"2024-09-16T15:51:01.369717+0000","last_fullsized":"2024-09-16T15:51:01.369717+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.389654+0000","last_change":"2024-09-16T15:51:01.386550+0000","last_active":"2024-09-16T15:51:01.389654+0000","last_peered":"2024-09-16T15:51:01.389654+0000","last_clean":"2024-09-16T15:51:01.389654+0000","last_became_active":"2024-09-16T15:51:01.386275+0000","last_became_peered":"2024-09-16T15:51:01.386275+0000","last_unstale":"2024-09-16T15:51:01.389654+0000","last_undegraded":"2024-09-16T15:51:01.389654+0000","last_fullsized":"2024-09-16T15:51:01.389654+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:33:03.973467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T15:51:02.391064+0000","last_change":"2024-09-16T15:51:02.386731+0000","last_active":"2024-09-16T15:51:02.391064+0000","last_peered":"2024-09-16T15:51:02.391064+0000","last_clean":"2024-09-16T15:51:02.391064+0000","last_became_active":"2024-09-16T15:51:02.384022+0000","last_became_peered":"2024-09-16T15:51:02.384022+0000","last_unstale":"2024-09-16T15:51:02.391064+0000","last_undegraded":"2024-09-16T15:51:02.391064+0000","last_fullsized":"2024-09-16T15:51:02.391064+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:10:52.384143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.381974+0000","last_change":"2024-09-16T15:51:01.378467+0000","last_active":"2024-09-16T15:51:01.381974+0000","last_peered":"2024-09-16T15:51:01.381974+0000","last_clean":"2024-09-16T15:51:01.381974+0000","last_became_active":"2024-09-16T15:51:01.378205+0000","last_became_peered":"2024-09-16T15:51:01.378205+0000","last_unstale":"2024-09-16T15:51:01.381974+0000","last_undegraded":"2024-09-16T15:51:01.381974+0000","last_fullsized":"2024-09-16T15:51:01.381974+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:53:22.512677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.368015+0000","last_change":"2024-09-16T15:51:01.368015+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.368015+0000","last_undegraded":"2024-09-16T15:51:01.368015+0000","last_fullsized":"2024-09-16T15:51:01.368015+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T15:51:02.384082+0000","last_change":"2024-09-16T15:51:01.389590+0000","last_active":"2024-09-16T15:51:02.384082+0000","last_peered":"2024-09-16T15:51:02.384082+0000","last_clean":"2024-09-16T15:51:02.384082+0000","last_became_active":"2024-09-16T15:51:01.389186+0000","last_became_peered":"2024-09-16T15:51:01.389186+0000","last_unstale":"2024-09-16T15:51:02.384082+0000","last_undegraded":"2024-09-16T15:51:02.384082+0000","last_fullsized":"2024-09-16T15:51:02.384082+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:22:23.409721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.369935+0000","last_change":"2024-09-16T15:51:01.369935+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.369935+0000","last_undegraded":"2024-09-16T15:51:01.369935+0000","last_fullsized":"2024-09-16T15:51:01.369935+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.382259+0000","last_change":"2024-09-16T15:51:01.379431+0000","last_active":"2024-09-16T15:51:01.382259+0000","last_peered":"2024-09-16T15:51:01.382259+0000","last_clean":"2024-09-16T15:51:01.382259+0000","last_became_active":"2024-09-16T15:51:01.379201+0000","last_became_peered":"2024-09-16T15:51:01.379201+0000","last_unstale":"2024-09-16T15:51:01.382259+0000","last_undegraded":"2024-09-16T15:51:01.382259+0000","last_fullsized":"2024-09-16T15:51:01.382259+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:40:55.915776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.367826+0000","last_change":"2024-09-16T15:51:01.367826+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.367826+0000","last_undegraded":"2024-09-16T15:51:01.367826+0000","last_fullsized":"2024-09-16T15:51:01.367826+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.380263+0000","last_change":"2024-09-16T15:51:01.378141+0000","last_active":"2024-09-16T15:51:01.380263+0000","last_peered":"2024-09-16T15:51:01.380263+0000","last_clean":"2024-09-16T15:51:01.380263+0000","last_became_active":"2024-09-16T15:51:01.377875+0000","last_became_peered":"2024-09-16T15:51:01.377875+0000","last_unstale":"2024-09-16T15:51:01.380263+0000","last_undegraded":"2024-09-16T15:51:01.380263+0000","last_fullsized":"2024-09-16T15:51:01.380263+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:12:01.247706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T15:51:02.390885+0000","last_change":"2024-09-16T15:51:02.387140+0000","last_active":"2024-09-16T15:51:02.390885+0000","last_peered":"2024-09-16T15:51:02.390885+0000","last_clean":"2024-09-16T15:51:02.390885+0000","last_became_active":"2024-09-16T15:51:02.385974+0000","last_became_peered":"2024-09-16T15:51:02.385974+0000","last_unstale":"2024-09-16T15:51:02.390885+0000","last_undegraded":"2024-09-16T15:51:02.390885+0000","last_fullsized":"2024-09-16T15:51:02.390885+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:48:40.504107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.387346+0000","last_change":"2024-09-16T15:51:01.384546+0000","last_active":"2024-09-16T15:51:01.387346+0000","last_peered":"2024-09-16T15:51:01.387346+0000","last_clean":"2024-09-16T15:51:01.387346+0000","last_became_active":"2024-09-16T15:51:01.384288+0000","last_became_peered":"2024-09-16T15:51:01.384288+0000","last_unstale":"2024-09-16T15:51:01.387346+0000","last_undegraded":"2024-09-16T15:51:01.387346+0000","last_fullsized":"2024-09-16T15:51:01.387346+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:01:08.875366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.368875+0000","last_change":"2024-09-16T15:51:01.368875+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.368875+0000","last_undegraded":"2024-09-16T15:51:01.368875+0000","last_fullsized":"2024-09-16T15:51:01.368875+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T15:51:02.380598+0000","last_change":"2024-09-16T15:51:01.384749+0000","last_active":"2024-09-16T15:51:02.380598+0000","last_peered":"2024-09-16T15:51:02.380598+0000","last_clean":"2024-09-16T15:51:02.380598+0000","last_became_active":"2024-09-16T15:51:01.384433+0000","last_became_peered":"2024-09-16T15:51:01.384433+0000","last_unstale":"2024-09-16T15:51:02.380598+0000","last_undegraded":"2024-09-16T15:51:02.380598+0000","last_fullsized":"2024-09-16T15:51:02.380598+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:43:12.283062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.368901+0000","last_change":"2024-09-16T15:51:01.368901+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.368901+0000","last_undegraded":"2024-09-16T15:51:01.368901+0000","last_fullsized":"2024-09-16T15:51:01.368901+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.380285+0000","last_change":"2024-09-16T15:51:01.378352+0000","last_active":"2024-09-16T15:51:01.380285+0000","last_peered":"2024-09-16T15:51:01.380285+0000","last_clean":"2024-09-16T15:51:01.380285+0000","last_became_active":"2024-09-16T15:51:01.376625+0000","last_became_peered":"2024-09-16T15:51:01.376625+0000","last_unstale":"2024-09-16T15:51:01.380285+0000","last_undegraded":"2024-09-16T15:51:01.380285+0000","last_fullsized":"2024-09-16T15:51:01.380285+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:16:13.783024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.367556+0000","last_change":"2024-09-16T15:51:01.367556+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.367556+0000","last_undegraded":"2024-09-16T15:51:01.367556+0000","last_fullsized":"2024-09-16T15:51:01.367556+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.10","version":"103'8","reported_seq":21,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T15:51:02.415518+0000","last_change":"2024-09-16T15:51:01.383280+0000","last_active":"2024-09-16T15:51:02.415518+0000","last_peered":"2024-09-16T15:51:02.415518+0000","last_clean":"2024-09-16T15:51:02.415518+0000","last_became_active":"2024-09-16T15:51:01.381141+0000","last_became_peered":"2024-09-16T15:51:01.381141+0000","last_unstale":"2024-09-16T15:51:02.415518+0000","last_undegraded":"2024-09-16T15:51:02.415518+0000","last_fullsized":"2024-09-16T15:51:02.415518+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:10:44.749157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.367595+0000","last_change":"2024-09-16T15:51:01.367595+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.367595+0000","last_undegraded":"2024-09-16T15:51:01.367595+0000","last_fullsized":"2024-09-16T15:51:01.367595+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.391952+0000","last_change":"2024-09-16T15:51:01.391228+0000","last_active":"2024-09-16T15:51:01.391952+0000","last_peered":"2024-09-16T15:51:01.391952+0000","last_clean":"2024-09-16T15:51:01.391952+0000","last_became_active":"2024-09-16T15:51:01.390919+0000","last_became_peered":"2024-09-16T15:51:01.390919+0000","last_unstale":"2024-09-16T15:51:01.391952+0000","last_undegraded":"2024-09-16T15:51:01.391952+0000","last_fullsized":"2024-09-16T15:51:01.391952+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:16:20.956001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.367237+0000","last_change":"2024-09-16T15:51:01.367237+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.367237+0000","last_undegraded":"2024-09-16T15:51:01.367237+0000","last_fullsized":"2024-09-16T15:51:01.367237+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.382149+0000","last_change":"2024-09-16T15:51:01.377467+0000","last_active":"2024-09-16T15:51:01.382149+0000","last_peered":"2024-09-16T15:51:01.382149+0000","last_clean":"2024-09-16T15:51:01.382149+0000","last_became_active":"2024-09-16T15:51:01.377180+0000","last_became_peered":"2024-09-16T15:51:01.377180+0000","last_unstale":"2024-09-16T15:51:01.382149+0000","last_undegraded":"2024-09-16T15:51:01.382149+0000","last_fullsized":"2024-09-16T15:51:01.382149+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:07:47.076318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.368491+0000","last_change":"2024-09-16T15:51:01.368491+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.368491+0000","last_undegraded":"2024-09-16T15:51:01.368491+0000","last_fullsized":"2024-09-16T15:51:01.368491+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.387295+0000","last_change":"2024-09-16T15:51:01.385783+0000","last_active":"2024-09-16T15:51:01.387295+0000","last_peered":"2024-09-16T15:51:01.387295+0000","last_clean":"2024-09-16T15:51:01.387295+0000","last_became_active":"2024-09-16T15:51:01.385500+0000","last_became_peered":"2024-09-16T15:51:01.385500+0000","last_unstale":"2024-09-16T15:51:01.387295+0000","last_undegraded":"2024-09-16T15:51:01.387295+0000","last_fullsized":"2024-09-16T15:51:01.387295+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:53:45.571107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.380115+0000","last_change":"2024-09-16T15:51:01.380115+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.380115+0000","last_undegraded":"2024-09-16T15:51:01.380115+0000","last_fullsized":"2024-09-16T15:51:01.380115+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"103'2","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T15:51:02.412322+0000","last_change":"2024-09-16T15:51:01.391626+0000","last_active":"2024-09-16T15:51:02.412322+0000","last_peered":"2024-09-16T15:51:02.412322+0000","last_clean":"2024-09-16T15:51:02.412322+0000","last_became_active":"2024-09-16T15:51:01.391322+0000","last_became_peered":"2024-09-16T15:51:01.391322+0000","last_unstale":"2024-09-16T15:51:02.412322+0000","last_undegraded":"2024-09-16T15:51:02.412322+0000","last_fullsized":"2024-09-16T15:51:02.412322+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:13:41.121392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.376646+0000","last_change":"2024-09-16T15:51:01.376646+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.376646+0000","last_undegraded":"2024-09-16T15:51:01.376646+0000","last_fullsized":"2024-09-16T15:51:01.376646+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.380454+0000","last_change":"2024-09-16T15:51:01.378559+0000","last_active":"2024-09-16T15:51:01.380454+0000","last_peered":"2024-09-16T15:51:01.380454+0000","last_clean":"2024-09-16T15:51:01.380454+0000","last_became_active":"2024-09-16T15:51:01.377521+0000","last_became_peered":"2024-09-16T15:51:01.377521+0000","last_unstale":"2024-09-16T15:51:01.380454+0000","last_undegraded":"2024-09-16T15:51:01.380454+0000","last_fullsized":"2024-09-16T15:51:01.380454+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:45:48.115715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T15:51:02.405451+0000","last_change":"2024-09-16T15:51:02.400056+0000","last_active":"2024-09-16T15:51:02.405451+0000","last_peered":"2024-09-16T15:51:02.405451+0000","last_clean":"2024-09-16T15:51:02.405451+0000","last_became_active":"2024-09-16T15:51:02.397362+0000","last_became_peered":"2024-09-16T15:51:02.397362+0000","last_unstale":"2024-09-16T15:51:02.405451+0000","last_undegraded":"2024-09-16T15:51:02.405451+0000","last_fullsized":"2024-09-16T15:51:02.405451+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:52:34.894816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-09-16T15:51:00.676479+0000","last_change":"2024-09-16T15:51:00.676479+0000","last_active":"2024-09-16T15:51:00.350216+0000","last_peered":"2024-09-16T15:51:00.350216+0000","last_clean":"2024-09-16T15:51:00.350216+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:00.676479+0000","last_undegraded":"2024-09-16T15:51:00.676479+0000","last_fullsized":"2024-09-16T15:51:00.676479+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.367293+0000","last_change":"2024-09-16T15:51:01.367293+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.367293+0000","last_undegraded":"2024-09-16T15:51:01.367293+0000","last_fullsized":"2024-09-16T15:51:01.367293+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.388344+0000","last_change":"2024-09-16T15:51:01.386549+0000","last_active":"2024-09-16T15:51:01.388344+0000","last_peered":"2024-09-16T15:51:01.388344+0000","last_clean":"2024-09-16T15:51:01.388344+0000","last_became_active":"2024-09-16T15:51:01.386253+0000","last_became_peered":"2024-09-16T15:51:01.386253+0000","last_unstale":"2024-09-16T15:51:01.388344+0000","last_undegraded":"2024-09-16T15:51:01.388344+0000","last_fullsized":"2024-09-16T15:51:01.388344+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:16:55.241704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.382489+0000","last_change":"2024-09-16T15:51:01.382489+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.382489+0000","last_undegraded":"2024-09-16T15:51:01.382489+0000","last_fullsized":"2024-09-16T15:51:01.382489+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.381276+0000","last_change":"2024-09-16T15:51:01.379987+0000","last_active":"2024-09-16T15:51:01.381276+0000","last_peered":"2024-09-16T15:51:01.381276+0000","last_clean":"2024-09-16T15:51:01.381276+0000","last_became_active":"2024-09-16T15:51:01.379714+0000","last_became_peered":"2024-09-16T15:51:01.379714+0000","last_unstale":"2024-09-16T15:51:01.381276+0000","last_undegraded":"2024-09-16T15:51:01.381276+0000","last_fullsized":"2024-09-16T15:51:01.381276+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:59:46.751381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.379213+0000","last_change":"2024-09-16T15:51:01.379213+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.379213+0000","last_undegraded":"2024-09-16T15:51:01.379213+0000","last_fullsized":"2024-09-16T15:51:01.379213+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.392581+0000","last_change":"2024-09-16T15:51:01.391293+0000","last_active":"2024-09-16T15:51:01.392581+0000","last_peered":"2024-09-16T15:51:01.392581+0000","last_clean":"2024-09-16T15:51:01.392581+0000","last_became_active":"2024-09-16T15:51:01.388187+0000","last_became_peered":"2024-09-16T15:51:01.388187+0000","last_unstale":"2024-09-16T15:51:01.392581+0000","last_undegraded":"2024-09-16T15:51:01.392581+0000","last_fullsized":"2024-09-16T15:51:01.392581+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:41:28.514384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.371328+0000","last_change":"2024-09-16T15:51:01.371328+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.371328+0000","last_undegraded":"2024-09-16T15:51:01.371328+0000","last_fullsized":"2024-09-16T15:51:01.371328+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.387376+0000","last_change":"2024-09-16T15:51:01.381796+0000","last_active":"2024-09-16T15:51:01.387376+0000","last_peered":"2024-09-16T15:51:01.387376+0000","last_clean":"2024-09-16T15:51:01.387376+0000","last_became_active":"2024-09-16T15:51:01.381524+0000","last_became_peered":"2024-09-16T15:51:01.381524+0000","last_unstale":"2024-09-16T15:51:01.387376+0000","last_undegraded":"2024-09-16T15:51:01.387376+0000","last_fullsized":"2024-09-16T15:51:01.387376+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:29:47.623958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.368671+0000","last_change":"2024-09-16T15:51:01.368671+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.368671+0000","last_undegraded":"2024-09-16T15:51:01.368671+0000","last_fullsized":"2024-09-16T15:51:01.368671+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.383212+0000","last_change":"2024-09-16T15:51:01.380670+0000","last_active":"2024-09-16T15:51:01.383212+0000","last_peered":"2024-09-16T15:51:01.383212+0000","last_clean":"2024-09-16T15:51:01.383212+0000","last_became_active":"2024-09-16T15:51:01.380213+0000","last_became_peered":"2024-09-16T15:51:01.380213+0000","last_unstale":"2024-09-16T15:51:01.383212+0000","last_undegraded":"2024-09-16T15:51:01.383212+0000","last_fullsized":"2024-09-16T15:51:01.383212+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:54:34.716483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.379707+0000","last_change":"2024-09-16T15:51:01.379707+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.379707+0000","last_undegraded":"2024-09-16T15:51:01.379707+0000","last_fullsized":"2024-09-16T15:51:01.379707+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.390441+0000","last_change":"2024-09-16T15:51:01.388916+0000","last_active":"2024-09-16T15:51:01.390441+0000","last_peered":"2024-09-16T15:51:01.390441+0000","last_clean":"2024-09-16T15:51:01.390441+0000","last_became_active":"2024-09-16T15:51:01.388644+0000","last_became_peered":"2024-09-16T15:51:01.388644+0000","last_unstale":"2024-09-16T15:51:01.390441+0000","last_undegraded":"2024-09-16T15:51:01.390441+0000","last_fullsized":"2024-09-16T15:51:01.390441+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:38:33.558776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.368076+0000","last_change":"2024-09-16T15:51:01.368076+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.368076+0000","last_undegraded":"2024-09-16T15:51:01.368076+0000","last_fullsized":"2024-09-16T15:51:01.368076+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.388815+0000","last_change":"2024-09-16T15:51:01.387695+0000","last_active":"2024-09-16T15:51:01.388815+0000","last_peered":"2024-09-16T15:51:01.388815+0000","last_clean":"2024-09-16T15:51:01.388815+0000","last_became_active":"2024-09-16T15:51:01.387455+0000","last_became_peered":"2024-09-16T15:51:01.387455+0000","last_unstale":"2024-09-16T15:51:01.388815+0000","last_undegraded":"2024-09-16T15:51:01.388815+0000","last_fullsized":"2024-09-16T15:51:01.388815+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:23:23.665770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.378487+0000","last_change":"2024-09-16T15:51:01.378487+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.378487+0000","last_undegraded":"2024-09-16T15:51:01.378487+0000","last_fullsized":"2024-09-16T15:51:01.378487+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.308686+0000","last_change":"2024-09-16T15:51:01.384206+0000","last_active":"2024-09-16T15:51:04.308686+0000","last_peered":"2024-09-16T15:51:04.308686+0000","last_clean":"2024-09-16T15:51:04.308686+0000","last_became_active":"2024-09-16T15:51:01.383873+0000","last_became_peered":"2024-09-16T15:51:01.383873+0000","last_unstale":"2024-09-16T15:51:04.308686+0000","last_undegraded":"2024-09-16T15:51:04.308686+0000","last_fullsized":"2024-09-16T15:51:04.308686+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:15:03.468370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.367780+0000","last_change":"2024-09-16T15:51:01.367780+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.367780+0000","last_undegraded":"2024-09-16T15:51:01.367780+0000","last_fullsized":"2024-09-16T15:51:01.367780+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.392779+0000","last_change":"2024-09-16T15:51:01.390952+0000","last_active":"2024-09-16T15:51:01.392779+0000","last_peered":"2024-09-16T15:51:01.392779+0000","last_clean":"2024-09-16T15:51:01.392779+0000","last_became_active":"2024-09-16T15:51:01.390640+0000","last_became_peered":"2024-09-16T15:51:01.390640+0000","last_unstale":"2024-09-16T15:51:01.392779+0000","last_undegraded":"2024-09-16T15:51:01.392779+0000","last_fullsized":"2024-09-16T15:51:01.392779+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:04:41.524063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.370302+0000","last_change":"2024-09-16T15:51:01.370302+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.370302+0000","last_undegraded":"2024-09-16T15:51:01.370302+0000","last_fullsized":"2024-09-16T15:51:01.370302+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.388861+0000","last_change":"2024-09-16T15:51:01.388328+0000","last_active":"2024-09-16T15:51:01.388861+0000","last_peered":"2024-09-16T15:51:01.388861+0000","last_clean":"2024-09-16T15:51:01.388861+0000","last_became_active":"2024-09-16T15:51:01.388066+0000","last_became_peered":"2024-09-16T15:51:01.388066+0000","last_unstale":"2024-09-16T15:51:01.388861+0000","last_undegraded":"2024-09-16T15:51:01.388861+0000","last_fullsized":"2024-09-16T15:51:01.388861+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:22:46.801645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":262,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.367319+0000","last_change":"2024-09-16T15:39:12.698813+0000","last_active":"2024-09-16T15:51:01.367319+0000","last_peered":"2024-09-16T15:51:01.367319+0000","last_clean":"2024-09-16T15:51:01.367319+0000","last_became_active":"2024-09-16T15:39:12.697695+0000","last_became_peered":"2024-09-16T15:39:12.697695+0000","last_unstale":"2024-09-16T15:51:01.367319+0000","last_undegraded":"2024-09-16T15:51:01.367319+0000","last_fullsized":"2024-09-16T15:51:01.367319+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_clean_scrub_stamp":"2024-09-16T15:39:11.684967+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:02.061081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":122,"acting":122,"num_store_stats":7},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":582,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509587,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":148960,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222880,"statfs":{"total":96636764160,"available":96484229120,"internally_reserved":0,"allocated":1253376,"data_stored":544601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":151255496},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509586,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131488,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240352,"statfs":{"total":96636764160,"available":96502120448,"internally_reserved":0,"allocated":1847296,"data_stored":1134969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509585,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144800,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94227040,"statfs":{"total":96636764160,"available":96488488960,"internally_reserved":0,"allocated":1241088,"data_stored":538270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509585,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126752,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94245088,"statfs":{"total":96636764160,"available":96506970112,"internally_reserved":0,"allocated":1253376,"data_stored":544601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":128514505},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509585,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126740,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94245100,"statfs":{"total":96636764160,"available":96506982400,"internally_reserved":0,"allocated":1241088,"data_stored":538270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":128514503},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509586,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123160,"kb_used_data":1792,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248680,"statfs":{"total":96636764160,"available":96510648320,"internally_reserved":0,"allocated":1835008,"data_stored":1128638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":124254663},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509584,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":140612,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94231228,"statfs":{"total":96636764160,"available":96492777472,"internally_reserved":0,"allocated":1228800,"data_stored":532397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":142735815},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542292,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122620,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94249220,"statfs":{"total":96636764160,"available":96511201280,"internally_reserved":0,"allocated":1282048,"data_stored":558370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":124254668},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:51:06.604 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:51:06.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.795+0000 7f2dbd400640 1 -- 172.21.15.151:0/2461027218 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2db815f5a0 msgr2=0x7f2db80b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:06.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.795+0000 7f2dbd400640 1 --2- 172.21.15.151:0/2461027218 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2db815f5a0 0x7f2db80b2440 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f2da8005fd0 tx=0x7f2da802a540 comp rx=0 tx=0).stop 2024-09-16T15:51:06.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.795+0000 7f2dbd400640 1 -- 172.21.15.151:0/2461027218 shutdown_connections 2024-09-16T15:51:06.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.796+0000 7f2dbd400640 1 --2- 172.21.15.151:0/2461027218 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2db8073fb0 0x7f2db8074440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:06.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.796+0000 7f2dbd400640 1 --2- 172.21.15.151:0/2461027218 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2db815f5a0 0x7f2db80b2440 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:06.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.796+0000 7f2dbd400640 1 -- 172.21.15.151:0/2461027218 >> 172.21.15.151:0/2461027218 conn(0x7f2db80b1950 msgr2=0x7f2db80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:06.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.796+0000 7f2dbd400640 1 -- 172.21.15.151:0/2461027218 shutdown_connections 2024-09-16T15:51:06.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.796+0000 7f2dbd400640 1 -- 172.21.15.151:0/2461027218 wait complete. 2024-09-16T15:51:06.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.797+0000 7f2dbd400640 1 Processor -- start 2024-09-16T15:51:06.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.797+0000 7f2dbd400640 1 -- start start 2024-09-16T15:51:06.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.798+0000 7f2dbd400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2db8073fb0 0x7f2db8152860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:06.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.798+0000 7f2dbd400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2db80ccf00 0x7f2db814ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:06.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.798+0000 7f2dbd400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2db815f5a0 0x7f2db81f7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:06.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.798+0000 7f2dbd400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2db806f2e0 con 0x7f2db8073fb0 2024-09-16T15:51:06.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.798+0000 7f2dbd400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2db806f160 con 0x7f2db80ccf00 2024-09-16T15:51:06.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.798+0000 7f2dbd400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f2db806f460 con 0x7f2db815f5a0 2024-09-16T15:51:06.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.799+0000 7f2db6800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2db8073fb0 0x7f2db8152860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:06.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.799+0000 7f2db6800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2db8073fb0 0x7f2db8152860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52644/0 (socket says 172.21.15.151:52644) 2024-09-16T15:51:06.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.799+0000 7f2db6800640 1 -- 172.21.15.151:0/1023116259 learned_addr learned my addr 172.21.15.151:0/1023116259 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:51:06.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.799+0000 7f2db5e00640 1 --2- 172.21.15.151:0/1023116259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2db80ccf00 0x7f2db814ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:06.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.799+0000 7f2db7200640 1 --2- 172.21.15.151:0/1023116259 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2db815f5a0 0x7f2db81f7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:06.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.799+0000 7f2db6800640 1 -- 172.21.15.151:0/1023116259 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2db815f5a0 msgr2=0x7f2db81f7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:06.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.799+0000 7f2db6800640 1 --2- 172.21.15.151:0/1023116259 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2db815f5a0 0x7f2db81f7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:06.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.799+0000 7f2db6800640 1 -- 172.21.15.151:0/1023116259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2db80ccf00 msgr2=0x7f2db814ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:06.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.799+0000 7f2db6800640 1 --2- 172.21.15.151:0/1023116259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2db80ccf00 0x7f2db814ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:06.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.799+0000 7f2db6800640 1 -- 172.21.15.151:0/1023116259 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2db81530a0 con 0x7f2db8073fb0 2024-09-16T15:51:06.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.799+0000 7f2db6800640 1 --2- 172.21.15.151:0/1023116259 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2db8073fb0 0x7f2db8152860 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f2da8034040 tx=0x7f2da80038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:06.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.800+0000 7f2daea00640 1 -- 172.21.15.151:0/1023116259 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da803e030 con 0x7f2db8073fb0 2024-09-16T15:51:06.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.800+0000 7f2daea00640 1 -- 172.21.15.151:0/1023116259 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2da8003d90 con 0x7f2db8073fb0 2024-09-16T15:51:06.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.800+0000 7f2dbd400640 1 -- 172.21.15.151:0/1023116259 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2db8153380 con 0x7f2db8073fb0 2024-09-16T15:51:06.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.800+0000 7f2daea00640 1 -- 172.21.15.151:0/1023116259 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da8035040 con 0x7f2db8073fb0 2024-09-16T15:51:06.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.800+0000 7f2dbd400640 1 -- 172.21.15.151:0/1023116259 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2db8153940 con 0x7f2db8073fb0 2024-09-16T15:51:06.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.801+0000 7f2dbd400640 1 -- 172.21.15.151:0/1023116259 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2db806f980 con 0x7f2db8073fb0 2024-09-16T15:51:06.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.802+0000 7f2daea00640 1 -- 172.21.15.151:0/1023116259 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7f2da8041030 con 0x7f2db8073fb0 2024-09-16T15:51:06.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.803+0000 7f2daea00640 1 --2- 172.21.15.151:0/1023116259 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2d98079ed0 0x7f2d9807c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:06.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.803+0000 7f2daea00640 1 -- 172.21.15.151:0/1023116259 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5765+0+0 (secure 0 0 0) 0x7f2da80ba020 con 0x7f2db8073fb0 2024-09-16T15:51:06.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.804+0000 7f2db5e00640 1 --2- 172.21.15.151:0/1023116259 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2d98079ed0 0x7f2d9807c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:06.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.804+0000 7f2db5e00640 1 --2- 172.21.15.151:0/1023116259 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2d98079ed0 0x7f2d9807c390 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f2db815e840 tx=0x7f2da4009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:06.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.807+0000 7f2daea00640 1 -- 172.21.15.151:0/1023116259 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2da8083aa0 con 0x7f2db8073fb0 2024-09-16T15:51:06.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.990+0000 7f2dbd400640 1 -- 172.21.15.151:0/1023116259 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2db8154e50 con 0x7f2d98079ed0 2024-09-16T15:51:06.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:06.996+0000 7f2daea00640 1 -- 172.21.15.151:0/1023116259 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+248219 (secure 0 0 0) 0x7f2db8154e50 con 0x7f2d98079ed0 2024-09-16T15:51:07.001 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:51:07.002 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-09-16T15:51:07.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.000+0000 7f2dbd400640 1 -- 172.21.15.151:0/1023116259 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2d98079ed0 msgr2=0x7f2d9807c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:07.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.000+0000 7f2dbd400640 1 --2- 172.21.15.151:0/1023116259 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2d98079ed0 0x7f2d9807c390 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f2db815e840 tx=0x7f2da4009040 comp rx=0 tx=0).stop 2024-09-16T15:51:07.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.000+0000 7f2dbd400640 1 -- 172.21.15.151:0/1023116259 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2db8073fb0 msgr2=0x7f2db8152860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:07.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.000+0000 7f2dbd400640 1 --2- 172.21.15.151:0/1023116259 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2db8073fb0 0x7f2db8152860 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f2da8034040 tx=0x7f2da80038a0 comp rx=0 tx=0).stop 2024-09-16T15:51:07.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.000+0000 7f2dbd400640 1 -- 172.21.15.151:0/1023116259 shutdown_connections 2024-09-16T15:51:07.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.000+0000 7f2dbd400640 1 --2- 172.21.15.151:0/1023116259 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2d98079ed0 0x7f2d9807c390 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:07.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.000+0000 7f2dbd400640 1 --2- 172.21.15.151:0/1023116259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2db80ccf00 0x7f2db814ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:07.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.000+0000 7f2dbd400640 1 --2- 172.21.15.151:0/1023116259 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2db815f5a0 0x7f2db81f7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:07.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.000+0000 7f2dbd400640 1 --2- 172.21.15.151:0/1023116259 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2db8073fb0 0x7f2db8152860 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:07.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.000+0000 7f2dbd400640 1 -- 172.21.15.151:0/1023116259 >> 172.21.15.151:0/1023116259 conn(0x7f2db80b1950 msgr2=0x7f2db8154030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:07.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.000+0000 7f2dbd400640 1 -- 172.21.15.151:0/1023116259 shutdown_connections 2024-09-16T15:51:07.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.000+0000 7f2dbd400640 1 -- 172.21.15.151:0/1023116259 wait complete. 2024-09-16T15:51:07.018 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":455,"stamp":"2024-09-16T15:51:05.880418+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590950,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":121,"num_write_kb":1629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":89,"ondisk_log_size":89,"up":188,"acting":188,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":27,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1065132,"kb_used_data":10920,"kb_used_omap":12,"kb_used_meta":1054003,"kb_avail":753909588,"statfs":{"total":773094113280,"available":772003418112,"internally_reserved":0,"allocated":11182080,"data_stored":5520116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12738,"internal_metadata":1079299646},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":582,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.518821"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T15:51:02.391024+0000","last_change":"2024-09-16T15:51:02.386431+0000","last_active":"2024-09-16T15:51:02.391024+0000","last_peered":"2024-09-16T15:51:02.391024+0000","last_clean":"2024-09-16T15:51:02.391024+0000","last_became_active":"2024-09-16T15:51:02.386126+0000","last_became_peered":"2024-09-16T15:51:02.386126+0000","last_unstale":"2024-09-16T15:51:02.391024+0000","last_undegraded":"2024-09-16T15:51:02.391024+0000","last_fullsized":"2024-09-16T15:51:02.391024+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:25:09.365288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T15:51:02.400467+0000","last_change":"2024-09-16T15:51:02.388621+0000","last_active":"2024-09-16T15:51:02.400467+0000","last_peered":"2024-09-16T15:51:02.400467+0000","last_clean":"2024-09-16T15:51:02.400467+0000","last_became_active":"2024-09-16T15:51:02.388294+0000","last_became_peered":"2024-09-16T15:51:02.388294+0000","last_unstale":"2024-09-16T15:51:02.400467+0000","last_undegraded":"2024-09-16T15:51:02.400467+0000","last_fullsized":"2024-09-16T15:51:02.400467+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:43:28.544765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.369847+0000","last_change":"2024-09-16T15:51:01.369847+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.369847+0000","last_undegraded":"2024-09-16T15:51:01.369847+0000","last_fullsized":"2024-09-16T15:51:01.369847+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T15:51:02.387475+0000","last_change":"2024-09-16T15:51:02.385219+0000","last_active":"2024-09-16T15:51:02.387475+0000","last_peered":"2024-09-16T15:51:02.387475+0000","last_clean":"2024-09-16T15:51:02.387475+0000","last_became_active":"2024-09-16T15:51:02.384310+0000","last_became_peered":"2024-09-16T15:51:02.384310+0000","last_unstale":"2024-09-16T15:51:02.387475+0000","last_undegraded":"2024-09-16T15:51:02.387475+0000","last_fullsized":"2024-09-16T15:51:02.387475+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:48:47.559516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.374870+0000","last_change":"2024-09-16T15:51:01.374870+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.374870+0000","last_undegraded":"2024-09-16T15:51:01.374870+0000","last_fullsized":"2024-09-16T15:51:01.374870+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.374553+0000","last_change":"2024-09-16T15:51:01.374553+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.374553+0000","last_undegraded":"2024-09-16T15:51:01.374553+0000","last_fullsized":"2024-09-16T15:51:01.374553+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.372119+0000","last_change":"2024-09-16T15:51:01.372119+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.372119+0000","last_undegraded":"2024-09-16T15:51:01.372119+0000","last_fullsized":"2024-09-16T15:51:01.372119+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.370748+0000","last_change":"2024-09-16T15:51:01.370748+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.370748+0000","last_undegraded":"2024-09-16T15:51:01.370748+0000","last_fullsized":"2024-09-16T15:51:01.370748+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.373077+0000","last_change":"2024-09-16T15:51:01.373077+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.373077+0000","last_undegraded":"2024-09-16T15:51:01.373077+0000","last_fullsized":"2024-09-16T15:51:01.373077+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:51:01.355970+0000","last_change":"2024-09-16T15:51:01.355970+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.355970+0000","last_undegraded":"2024-09-16T15:51:01.355970+0000","last_fullsized":"2024-09-16T15:51:01.355970+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.374982+0000","last_change":"2024-09-16T15:51:01.374982+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.374982+0000","last_undegraded":"2024-09-16T15:51:01.374982+0000","last_fullsized":"2024-09-16T15:51:01.374982+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.373278+0000","last_change":"2024-09-16T15:51:01.373278+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.373278+0000","last_undegraded":"2024-09-16T15:51:01.373278+0000","last_fullsized":"2024-09-16T15:51:01.373278+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309716+0000","last_change":"2024-09-16T15:51:02.407269+0000","last_active":"2024-09-16T15:51:04.309716+0000","last_peered":"2024-09-16T15:51:04.309716+0000","last_clean":"2024-09-16T15:51:04.309716+0000","last_became_active":"2024-09-16T15:51:02.406943+0000","last_became_peered":"2024-09-16T15:51:02.406943+0000","last_unstale":"2024-09-16T15:51:04.309716+0000","last_undegraded":"2024-09-16T15:51:04.309716+0000","last_fullsized":"2024-09-16T15:51:04.309716+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:29:33.561032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T15:51:02.405765+0000","last_change":"2024-09-16T15:51:02.398129+0000","last_active":"2024-09-16T15:51:02.405765+0000","last_peered":"2024-09-16T15:51:02.405765+0000","last_clean":"2024-09-16T15:51:02.405765+0000","last_became_active":"2024-09-16T15:51:02.397834+0000","last_became_peered":"2024-09-16T15:51:02.397834+0000","last_unstale":"2024-09-16T15:51:02.405765+0000","last_undegraded":"2024-09-16T15:51:02.405765+0000","last_fullsized":"2024-09-16T15:51:02.405765+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:35:17.388414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T15:51:02.404562+0000","last_change":"2024-09-16T15:51:02.396682+0000","last_active":"2024-09-16T15:51:02.404562+0000","last_peered":"2024-09-16T15:51:02.404562+0000","last_clean":"2024-09-16T15:51:02.404562+0000","last_became_active":"2024-09-16T15:51:02.396412+0000","last_became_peered":"2024-09-16T15:51:02.396412+0000","last_unstale":"2024-09-16T15:51:02.404562+0000","last_undegraded":"2024-09-16T15:51:02.404562+0000","last_fullsized":"2024-09-16T15:51:02.404562+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:52:38.771302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.371468+0000","last_change":"2024-09-16T15:51:01.371468+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.371468+0000","last_undegraded":"2024-09-16T15:51:01.371468+0000","last_fullsized":"2024-09-16T15:51:01.371468+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.379608+0000","last_change":"2024-09-16T15:51:01.379608+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.379608+0000","last_undegraded":"2024-09-16T15:51:01.379608+0000","last_fullsized":"2024-09-16T15:51:01.379608+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T15:51:02.404957+0000","last_change":"2024-09-16T15:51:02.399241+0000","last_active":"2024-09-16T15:51:02.404957+0000","last_peered":"2024-09-16T15:51:02.404957+0000","last_clean":"2024-09-16T15:51:02.404957+0000","last_became_active":"2024-09-16T15:51:02.398973+0000","last_became_peered":"2024-09-16T15:51:02.398973+0000","last_unstale":"2024-09-16T15:51:02.404957+0000","last_undegraded":"2024-09-16T15:51:02.404957+0000","last_fullsized":"2024-09-16T15:51:02.404957+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:46:06.817591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.372608+0000","last_change":"2024-09-16T15:51:01.372608+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.372608+0000","last_undegraded":"2024-09-16T15:51:01.372608+0000","last_fullsized":"2024-09-16T15:51:01.372608+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.370913+0000","last_change":"2024-09-16T15:51:01.370913+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.370913+0000","last_undegraded":"2024-09-16T15:51:01.370913+0000","last_fullsized":"2024-09-16T15:51:01.370913+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.371225+0000","last_change":"2024-09-16T15:51:01.371225+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.371225+0000","last_undegraded":"2024-09-16T15:51:01.371225+0000","last_fullsized":"2024-09-16T15:51:01.371225+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.368817+0000","last_change":"2024-09-16T15:51:01.368817+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.368817+0000","last_undegraded":"2024-09-16T15:51:01.368817+0000","last_fullsized":"2024-09-16T15:51:01.368817+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T15:51:02.405411+0000","last_change":"2024-09-16T15:51:02.399544+0000","last_active":"2024-09-16T15:51:02.405411+0000","last_peered":"2024-09-16T15:51:02.405411+0000","last_clean":"2024-09-16T15:51:02.405411+0000","last_became_active":"2024-09-16T15:51:02.396515+0000","last_became_peered":"2024-09-16T15:51:02.396515+0000","last_unstale":"2024-09-16T15:51:02.405411+0000","last_undegraded":"2024-09-16T15:51:02.405411+0000","last_fullsized":"2024-09-16T15:51:02.405411+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:05:52.491919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.371334+0000","last_change":"2024-09-16T15:51:01.371334+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.371334+0000","last_undegraded":"2024-09-16T15:51:01.371334+0000","last_fullsized":"2024-09-16T15:51:01.371334+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:51:01.355970+0000","last_change":"2024-09-16T15:51:01.355970+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.355970+0000","last_undegraded":"2024-09-16T15:51:01.355970+0000","last_fullsized":"2024-09-16T15:51:01.355970+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309644+0000","last_change":"2024-09-16T15:51:02.401422+0000","last_active":"2024-09-16T15:51:04.309644+0000","last_peered":"2024-09-16T15:51:04.309644+0000","last_clean":"2024-09-16T15:51:04.309644+0000","last_became_active":"2024-09-16T15:51:02.400804+0000","last_became_peered":"2024-09-16T15:51:02.400804+0000","last_unstale":"2024-09-16T15:51:04.309644+0000","last_undegraded":"2024-09-16T15:51:04.309644+0000","last_fullsized":"2024-09-16T15:51:04.309644+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:54:09.898078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.371083+0000","last_change":"2024-09-16T15:51:01.371083+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.371083+0000","last_undegraded":"2024-09-16T15:51:01.371083+0000","last_fullsized":"2024-09-16T15:51:01.371083+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.368057+0000","last_change":"2024-09-16T15:51:01.368057+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.368057+0000","last_undegraded":"2024-09-16T15:51:01.368057+0000","last_fullsized":"2024-09-16T15:51:01.368057+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309979+0000","last_change":"2024-09-16T15:51:02.405833+0000","last_active":"2024-09-16T15:51:04.309979+0000","last_peered":"2024-09-16T15:51:04.309979+0000","last_clean":"2024-09-16T15:51:04.309979+0000","last_became_active":"2024-09-16T15:51:02.404364+0000","last_became_peered":"2024-09-16T15:51:02.404364+0000","last_unstale":"2024-09-16T15:51:04.309979+0000","last_undegraded":"2024-09-16T15:51:04.309979+0000","last_fullsized":"2024-09-16T15:51:04.309979+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:47:54.421341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309062+0000","last_change":"2024-09-16T15:51:02.402965+0000","last_active":"2024-09-16T15:51:04.309062+0000","last_peered":"2024-09-16T15:51:04.309062+0000","last_clean":"2024-09-16T15:51:04.309062+0000","last_became_active":"2024-09-16T15:51:02.401107+0000","last_became_peered":"2024-09-16T15:51:02.401107+0000","last_unstale":"2024-09-16T15:51:04.309062+0000","last_undegraded":"2024-09-16T15:51:04.309062+0000","last_fullsized":"2024-09-16T15:51:04.309062+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:24:38.932170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.368886+0000","last_change":"2024-09-16T15:51:01.368886+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.368886+0000","last_undegraded":"2024-09-16T15:51:01.368886+0000","last_fullsized":"2024-09-16T15:51:01.368886+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T15:51:02.387143+0000","last_change":"2024-09-16T15:51:02.384379+0000","last_active":"2024-09-16T15:51:02.387143+0000","last_peered":"2024-09-16T15:51:02.387143+0000","last_clean":"2024-09-16T15:51:02.387143+0000","last_became_active":"2024-09-16T15:51:02.384012+0000","last_became_peered":"2024-09-16T15:51:02.384012+0000","last_unstale":"2024-09-16T15:51:02.387143+0000","last_undegraded":"2024-09-16T15:51:02.387143+0000","last_fullsized":"2024-09-16T15:51:02.387143+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:04:09.954852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.382918+0000","last_change":"2024-09-16T15:51:01.382918+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.382918+0000","last_undegraded":"2024-09-16T15:51:01.382918+0000","last_fullsized":"2024-09-16T15:51:01.382918+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1f","version":"103'4","reported_seq":19,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309998+0000","last_change":"2024-09-16T15:51:01.375200+0000","last_active":"2024-09-16T15:51:04.309998+0000","last_peered":"2024-09-16T15:51:04.309998+0000","last_clean":"2024-09-16T15:51:04.309998+0000","last_became_active":"2024-09-16T15:51:01.374919+0000","last_became_peered":"2024-09-16T15:51:01.374919+0000","last_unstale":"2024-09-16T15:51:04.309998+0000","last_undegraded":"2024-09-16T15:51:04.309998+0000","last_fullsized":"2024-09-16T15:51:04.309998+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:36:44.338407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.310065+0000","last_change":"2024-09-16T15:51:02.402988+0000","last_active":"2024-09-16T15:51:04.310065+0000","last_peered":"2024-09-16T15:51:04.310065+0000","last_clean":"2024-09-16T15:51:04.310065+0000","last_became_active":"2024-09-16T15:51:02.402708+0000","last_became_peered":"2024-09-16T15:51:02.402708+0000","last_unstale":"2024-09-16T15:51:04.310065+0000","last_undegraded":"2024-09-16T15:51:04.310065+0000","last_fullsized":"2024-09-16T15:51:04.310065+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:14:48.952444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.392214+0000","last_change":"2024-09-16T15:51:01.391174+0000","last_active":"2024-09-16T15:51:01.392214+0000","last_peered":"2024-09-16T15:51:01.392214+0000","last_clean":"2024-09-16T15:51:01.392214+0000","last_became_active":"2024-09-16T15:51:01.390904+0000","last_became_peered":"2024-09-16T15:51:01.390904+0000","last_unstale":"2024-09-16T15:51:01.392214+0000","last_undegraded":"2024-09-16T15:51:01.392214+0000","last_fullsized":"2024-09-16T15:51:01.392214+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:15:52.955571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T15:51:02.386883+0000","last_change":"2024-09-16T15:51:02.382876+0000","last_active":"2024-09-16T15:51:02.386883+0000","last_peered":"2024-09-16T15:51:02.386883+0000","last_clean":"2024-09-16T15:51:02.386883+0000","last_became_active":"2024-09-16T15:51:02.382617+0000","last_became_peered":"2024-09-16T15:51:02.382617+0000","last_unstale":"2024-09-16T15:51:02.386883+0000","last_undegraded":"2024-09-16T15:51:02.386883+0000","last_fullsized":"2024-09-16T15:51:02.386883+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:19:48.933205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.390016+0000","last_change":"2024-09-16T15:51:01.388774+0000","last_active":"2024-09-16T15:51:01.390016+0000","last_peered":"2024-09-16T15:51:01.390016+0000","last_clean":"2024-09-16T15:51:01.390016+0000","last_became_active":"2024-09-16T15:51:01.388500+0000","last_became_peered":"2024-09-16T15:51:01.388500+0000","last_unstale":"2024-09-16T15:51:01.390016+0000","last_undegraded":"2024-09-16T15:51:01.390016+0000","last_fullsized":"2024-09-16T15:51:01.390016+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:08:59.734743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:51:01.355970+0000","last_change":"2024-09-16T15:51:01.355970+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.355970+0000","last_undegraded":"2024-09-16T15:51:01.355970+0000","last_fullsized":"2024-09-16T15:51:01.355970+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1c","version":"103'2","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T15:51:02.414797+0000","last_change":"2024-09-16T15:51:01.390985+0000","last_active":"2024-09-16T15:51:02.414797+0000","last_peered":"2024-09-16T15:51:02.414797+0000","last_clean":"2024-09-16T15:51:02.414797+0000","last_became_active":"2024-09-16T15:51:01.390463+0000","last_became_peered":"2024-09-16T15:51:01.390463+0000","last_unstale":"2024-09-16T15:51:02.414797+0000","last_undegraded":"2024-09-16T15:51:02.414797+0000","last_fullsized":"2024-09-16T15:51:02.414797+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:57:05.221628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.370455+0000","last_change":"2024-09-16T15:51:01.370455+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.370455+0000","last_undegraded":"2024-09-16T15:51:01.370455+0000","last_fullsized":"2024-09-16T15:51:01.370455+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1b","version":"103'1","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T15:51:02.411975+0000","last_change":"2024-09-16T15:51:01.387614+0000","last_active":"2024-09-16T15:51:02.411975+0000","last_peered":"2024-09-16T15:51:02.411975+0000","last_clean":"2024-09-16T15:51:02.411975+0000","last_became_active":"2024-09-16T15:51:01.387163+0000","last_became_peered":"2024-09-16T15:51:01.387163+0000","last_unstale":"2024-09-16T15:51:02.411975+0000","last_undegraded":"2024-09-16T15:51:02.411975+0000","last_fullsized":"2024-09-16T15:51:02.411975+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:48:24.298645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.370317+0000","last_change":"2024-09-16T15:51:01.370317+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.370317+0000","last_undegraded":"2024-09-16T15:51:01.370317+0000","last_fullsized":"2024-09-16T15:51:01.370317+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.383140+0000","last_change":"2024-09-16T15:51:01.378848+0000","last_active":"2024-09-16T15:51:01.383140+0000","last_peered":"2024-09-16T15:51:01.383140+0000","last_clean":"2024-09-16T15:51:01.383140+0000","last_became_active":"2024-09-16T15:51:01.377621+0000","last_became_peered":"2024-09-16T15:51:01.377621+0000","last_unstale":"2024-09-16T15:51:01.383140+0000","last_undegraded":"2024-09-16T15:51:01.383140+0000","last_fullsized":"2024-09-16T15:51:01.383140+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:28:49.058738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.308778+0000","last_change":"2024-09-16T15:51:02.405495+0000","last_active":"2024-09-16T15:51:04.308778+0000","last_peered":"2024-09-16T15:51:04.308778+0000","last_clean":"2024-09-16T15:51:04.308778+0000","last_became_active":"2024-09-16T15:51:02.405222+0000","last_became_peered":"2024-09-16T15:51:02.405222+0000","last_unstale":"2024-09-16T15:51:04.308778+0000","last_undegraded":"2024-09-16T15:51:04.308778+0000","last_fullsized":"2024-09-16T15:51:04.308778+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:22:02.790695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.378694+0000","last_change":"2024-09-16T15:51:01.372791+0000","last_active":"2024-09-16T15:51:01.378694+0000","last_peered":"2024-09-16T15:51:01.378694+0000","last_clean":"2024-09-16T15:51:01.378694+0000","last_became_active":"2024-09-16T15:51:01.372493+0000","last_became_peered":"2024-09-16T15:51:01.372493+0000","last_unstale":"2024-09-16T15:51:01.378694+0000","last_undegraded":"2024-09-16T15:51:01.378694+0000","last_fullsized":"2024-09-16T15:51:01.378694+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:24:18.852411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.369717+0000","last_change":"2024-09-16T15:51:01.369717+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.369717+0000","last_undegraded":"2024-09-16T15:51:01.369717+0000","last_fullsized":"2024-09-16T15:51:01.369717+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.389654+0000","last_change":"2024-09-16T15:51:01.386550+0000","last_active":"2024-09-16T15:51:01.389654+0000","last_peered":"2024-09-16T15:51:01.389654+0000","last_clean":"2024-09-16T15:51:01.389654+0000","last_became_active":"2024-09-16T15:51:01.386275+0000","last_became_peered":"2024-09-16T15:51:01.386275+0000","last_unstale":"2024-09-16T15:51:01.389654+0000","last_undegraded":"2024-09-16T15:51:01.389654+0000","last_fullsized":"2024-09-16T15:51:01.389654+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:33:03.973467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T15:51:02.391064+0000","last_change":"2024-09-16T15:51:02.386731+0000","last_active":"2024-09-16T15:51:02.391064+0000","last_peered":"2024-09-16T15:51:02.391064+0000","last_clean":"2024-09-16T15:51:02.391064+0000","last_became_active":"2024-09-16T15:51:02.384022+0000","last_became_peered":"2024-09-16T15:51:02.384022+0000","last_unstale":"2024-09-16T15:51:02.391064+0000","last_undegraded":"2024-09-16T15:51:02.391064+0000","last_fullsized":"2024-09-16T15:51:02.391064+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:10:52.384143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.381974+0000","last_change":"2024-09-16T15:51:01.378467+0000","last_active":"2024-09-16T15:51:01.381974+0000","last_peered":"2024-09-16T15:51:01.381974+0000","last_clean":"2024-09-16T15:51:01.381974+0000","last_became_active":"2024-09-16T15:51:01.378205+0000","last_became_peered":"2024-09-16T15:51:01.378205+0000","last_unstale":"2024-09-16T15:51:01.381974+0000","last_undegraded":"2024-09-16T15:51:01.381974+0000","last_fullsized":"2024-09-16T15:51:01.381974+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:53:22.512677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.368015+0000","last_change":"2024-09-16T15:51:01.368015+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.368015+0000","last_undegraded":"2024-09-16T15:51:01.368015+0000","last_fullsized":"2024-09-16T15:51:01.368015+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T15:51:02.384082+0000","last_change":"2024-09-16T15:51:01.389590+0000","last_active":"2024-09-16T15:51:02.384082+0000","last_peered":"2024-09-16T15:51:02.384082+0000","last_clean":"2024-09-16T15:51:02.384082+0000","last_became_active":"2024-09-16T15:51:01.389186+0000","last_became_peered":"2024-09-16T15:51:01.389186+0000","last_unstale":"2024-09-16T15:51:02.384082+0000","last_undegraded":"2024-09-16T15:51:02.384082+0000","last_fullsized":"2024-09-16T15:51:02.384082+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:22:23.409721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.369935+0000","last_change":"2024-09-16T15:51:01.369935+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.369935+0000","last_undegraded":"2024-09-16T15:51:01.369935+0000","last_fullsized":"2024-09-16T15:51:01.369935+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.382259+0000","last_change":"2024-09-16T15:51:01.379431+0000","last_active":"2024-09-16T15:51:01.382259+0000","last_peered":"2024-09-16T15:51:01.382259+0000","last_clean":"2024-09-16T15:51:01.382259+0000","last_became_active":"2024-09-16T15:51:01.379201+0000","last_became_peered":"2024-09-16T15:51:01.379201+0000","last_unstale":"2024-09-16T15:51:01.382259+0000","last_undegraded":"2024-09-16T15:51:01.382259+0000","last_fullsized":"2024-09-16T15:51:01.382259+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:40:55.915776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.367826+0000","last_change":"2024-09-16T15:51:01.367826+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.367826+0000","last_undegraded":"2024-09-16T15:51:01.367826+0000","last_fullsized":"2024-09-16T15:51:01.367826+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.380263+0000","last_change":"2024-09-16T15:51:01.378141+0000","last_active":"2024-09-16T15:51:01.380263+0000","last_peered":"2024-09-16T15:51:01.380263+0000","last_clean":"2024-09-16T15:51:01.380263+0000","last_became_active":"2024-09-16T15:51:01.377875+0000","last_became_peered":"2024-09-16T15:51:01.377875+0000","last_unstale":"2024-09-16T15:51:01.380263+0000","last_undegraded":"2024-09-16T15:51:01.380263+0000","last_fullsized":"2024-09-16T15:51:01.380263+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:12:01.247706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T15:51:02.390885+0000","last_change":"2024-09-16T15:51:02.387140+0000","last_active":"2024-09-16T15:51:02.390885+0000","last_peered":"2024-09-16T15:51:02.390885+0000","last_clean":"2024-09-16T15:51:02.390885+0000","last_became_active":"2024-09-16T15:51:02.385974+0000","last_became_peered":"2024-09-16T15:51:02.385974+0000","last_unstale":"2024-09-16T15:51:02.390885+0000","last_undegraded":"2024-09-16T15:51:02.390885+0000","last_fullsized":"2024-09-16T15:51:02.390885+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:48:40.504107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.387346+0000","last_change":"2024-09-16T15:51:01.384546+0000","last_active":"2024-09-16T15:51:01.387346+0000","last_peered":"2024-09-16T15:51:01.387346+0000","last_clean":"2024-09-16T15:51:01.387346+0000","last_became_active":"2024-09-16T15:51:01.384288+0000","last_became_peered":"2024-09-16T15:51:01.384288+0000","last_unstale":"2024-09-16T15:51:01.387346+0000","last_undegraded":"2024-09-16T15:51:01.387346+0000","last_fullsized":"2024-09-16T15:51:01.387346+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:01:08.875366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.368875+0000","last_change":"2024-09-16T15:51:01.368875+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.368875+0000","last_undegraded":"2024-09-16T15:51:01.368875+0000","last_fullsized":"2024-09-16T15:51:01.368875+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T15:51:02.380598+0000","last_change":"2024-09-16T15:51:01.384749+0000","last_active":"2024-09-16T15:51:02.380598+0000","last_peered":"2024-09-16T15:51:02.380598+0000","last_clean":"2024-09-16T15:51:02.380598+0000","last_became_active":"2024-09-16T15:51:01.384433+0000","last_became_peered":"2024-09-16T15:51:01.384433+0000","last_unstale":"2024-09-16T15:51:02.380598+0000","last_undegraded":"2024-09-16T15:51:02.380598+0000","last_fullsized":"2024-09-16T15:51:02.380598+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:43:12.283062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.368901+0000","last_change":"2024-09-16T15:51:01.368901+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.368901+0000","last_undegraded":"2024-09-16T15:51:01.368901+0000","last_fullsized":"2024-09-16T15:51:01.368901+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.380285+0000","last_change":"2024-09-16T15:51:01.378352+0000","last_active":"2024-09-16T15:51:01.380285+0000","last_peered":"2024-09-16T15:51:01.380285+0000","last_clean":"2024-09-16T15:51:01.380285+0000","last_became_active":"2024-09-16T15:51:01.376625+0000","last_became_peered":"2024-09-16T15:51:01.376625+0000","last_unstale":"2024-09-16T15:51:01.380285+0000","last_undegraded":"2024-09-16T15:51:01.380285+0000","last_fullsized":"2024-09-16T15:51:01.380285+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:16:13.783024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.367556+0000","last_change":"2024-09-16T15:51:01.367556+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.367556+0000","last_undegraded":"2024-09-16T15:51:01.367556+0000","last_fullsized":"2024-09-16T15:51:01.367556+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.10","version":"103'8","reported_seq":21,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T15:51:02.415518+0000","last_change":"2024-09-16T15:51:01.383280+0000","last_active":"2024-09-16T15:51:02.415518+0000","last_peered":"2024-09-16T15:51:02.415518+0000","last_clean":"2024-09-16T15:51:02.415518+0000","last_became_active":"2024-09-16T15:51:01.381141+0000","last_became_peered":"2024-09-16T15:51:01.381141+0000","last_unstale":"2024-09-16T15:51:02.415518+0000","last_undegraded":"2024-09-16T15:51:02.415518+0000","last_fullsized":"2024-09-16T15:51:02.415518+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:10:44.749157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.367595+0000","last_change":"2024-09-16T15:51:01.367595+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.367595+0000","last_undegraded":"2024-09-16T15:51:01.367595+0000","last_fullsized":"2024-09-16T15:51:01.367595+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.391952+0000","last_change":"2024-09-16T15:51:01.391228+0000","last_active":"2024-09-16T15:51:01.391952+0000","last_peered":"2024-09-16T15:51:01.391952+0000","last_clean":"2024-09-16T15:51:01.391952+0000","last_became_active":"2024-09-16T15:51:01.390919+0000","last_became_peered":"2024-09-16T15:51:01.390919+0000","last_unstale":"2024-09-16T15:51:01.391952+0000","last_undegraded":"2024-09-16T15:51:01.391952+0000","last_fullsized":"2024-09-16T15:51:01.391952+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:16:20.956001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.367237+0000","last_change":"2024-09-16T15:51:01.367237+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.367237+0000","last_undegraded":"2024-09-16T15:51:01.367237+0000","last_fullsized":"2024-09-16T15:51:01.367237+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.382149+0000","last_change":"2024-09-16T15:51:01.377467+0000","last_active":"2024-09-16T15:51:01.382149+0000","last_peered":"2024-09-16T15:51:01.382149+0000","last_clean":"2024-09-16T15:51:01.382149+0000","last_became_active":"2024-09-16T15:51:01.377180+0000","last_became_peered":"2024-09-16T15:51:01.377180+0000","last_unstale":"2024-09-16T15:51:01.382149+0000","last_undegraded":"2024-09-16T15:51:01.382149+0000","last_fullsized":"2024-09-16T15:51:01.382149+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:07:47.076318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.368491+0000","last_change":"2024-09-16T15:51:01.368491+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.368491+0000","last_undegraded":"2024-09-16T15:51:01.368491+0000","last_fullsized":"2024-09-16T15:51:01.368491+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.387295+0000","last_change":"2024-09-16T15:51:01.385783+0000","last_active":"2024-09-16T15:51:01.387295+0000","last_peered":"2024-09-16T15:51:01.387295+0000","last_clean":"2024-09-16T15:51:01.387295+0000","last_became_active":"2024-09-16T15:51:01.385500+0000","last_became_peered":"2024-09-16T15:51:01.385500+0000","last_unstale":"2024-09-16T15:51:01.387295+0000","last_undegraded":"2024-09-16T15:51:01.387295+0000","last_fullsized":"2024-09-16T15:51:01.387295+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:53:45.571107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.380115+0000","last_change":"2024-09-16T15:51:01.380115+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.380115+0000","last_undegraded":"2024-09-16T15:51:01.380115+0000","last_fullsized":"2024-09-16T15:51:01.380115+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"103'2","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T15:51:02.412322+0000","last_change":"2024-09-16T15:51:01.391626+0000","last_active":"2024-09-16T15:51:02.412322+0000","last_peered":"2024-09-16T15:51:02.412322+0000","last_clean":"2024-09-16T15:51:02.412322+0000","last_became_active":"2024-09-16T15:51:01.391322+0000","last_became_peered":"2024-09-16T15:51:01.391322+0000","last_unstale":"2024-09-16T15:51:02.412322+0000","last_undegraded":"2024-09-16T15:51:02.412322+0000","last_fullsized":"2024-09-16T15:51:02.412322+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:13:41.121392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.376646+0000","last_change":"2024-09-16T15:51:01.376646+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.376646+0000","last_undegraded":"2024-09-16T15:51:01.376646+0000","last_fullsized":"2024-09-16T15:51:01.376646+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.380454+0000","last_change":"2024-09-16T15:51:01.378559+0000","last_active":"2024-09-16T15:51:01.380454+0000","last_peered":"2024-09-16T15:51:01.380454+0000","last_clean":"2024-09-16T15:51:01.380454+0000","last_became_active":"2024-09-16T15:51:01.377521+0000","last_became_peered":"2024-09-16T15:51:01.377521+0000","last_unstale":"2024-09-16T15:51:01.380454+0000","last_undegraded":"2024-09-16T15:51:01.380454+0000","last_fullsized":"2024-09-16T15:51:01.380454+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:45:48.115715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T15:51:02.405451+0000","last_change":"2024-09-16T15:51:02.400056+0000","last_active":"2024-09-16T15:51:02.405451+0000","last_peered":"2024-09-16T15:51:02.405451+0000","last_clean":"2024-09-16T15:51:02.405451+0000","last_became_active":"2024-09-16T15:51:02.397362+0000","last_became_peered":"2024-09-16T15:51:02.397362+0000","last_unstale":"2024-09-16T15:51:02.405451+0000","last_undegraded":"2024-09-16T15:51:02.405451+0000","last_fullsized":"2024-09-16T15:51:02.405451+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:52:34.894816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-09-16T15:51:00.676479+0000","last_change":"2024-09-16T15:51:00.676479+0000","last_active":"2024-09-16T15:51:00.350216+0000","last_peered":"2024-09-16T15:51:00.350216+0000","last_clean":"2024-09-16T15:51:00.350216+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:00.676479+0000","last_undegraded":"2024-09-16T15:51:00.676479+0000","last_fullsized":"2024-09-16T15:51:00.676479+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.367293+0000","last_change":"2024-09-16T15:51:01.367293+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.367293+0000","last_undegraded":"2024-09-16T15:51:01.367293+0000","last_fullsized":"2024-09-16T15:51:01.367293+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.388344+0000","last_change":"2024-09-16T15:51:01.386549+0000","last_active":"2024-09-16T15:51:01.388344+0000","last_peered":"2024-09-16T15:51:01.388344+0000","last_clean":"2024-09-16T15:51:01.388344+0000","last_became_active":"2024-09-16T15:51:01.386253+0000","last_became_peered":"2024-09-16T15:51:01.386253+0000","last_unstale":"2024-09-16T15:51:01.388344+0000","last_undegraded":"2024-09-16T15:51:01.388344+0000","last_fullsized":"2024-09-16T15:51:01.388344+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:16:55.241704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.382489+0000","last_change":"2024-09-16T15:51:01.382489+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.382489+0000","last_undegraded":"2024-09-16T15:51:01.382489+0000","last_fullsized":"2024-09-16T15:51:01.382489+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.381276+0000","last_change":"2024-09-16T15:51:01.379987+0000","last_active":"2024-09-16T15:51:01.381276+0000","last_peered":"2024-09-16T15:51:01.381276+0000","last_clean":"2024-09-16T15:51:01.381276+0000","last_became_active":"2024-09-16T15:51:01.379714+0000","last_became_peered":"2024-09-16T15:51:01.379714+0000","last_unstale":"2024-09-16T15:51:01.381276+0000","last_undegraded":"2024-09-16T15:51:01.381276+0000","last_fullsized":"2024-09-16T15:51:01.381276+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:59:46.751381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.379213+0000","last_change":"2024-09-16T15:51:01.379213+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.379213+0000","last_undegraded":"2024-09-16T15:51:01.379213+0000","last_fullsized":"2024-09-16T15:51:01.379213+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.392581+0000","last_change":"2024-09-16T15:51:01.391293+0000","last_active":"2024-09-16T15:51:01.392581+0000","last_peered":"2024-09-16T15:51:01.392581+0000","last_clean":"2024-09-16T15:51:01.392581+0000","last_became_active":"2024-09-16T15:51:01.388187+0000","last_became_peered":"2024-09-16T15:51:01.388187+0000","last_unstale":"2024-09-16T15:51:01.392581+0000","last_undegraded":"2024-09-16T15:51:01.392581+0000","last_fullsized":"2024-09-16T15:51:01.392581+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:41:28.514384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.371328+0000","last_change":"2024-09-16T15:51:01.371328+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.371328+0000","last_undegraded":"2024-09-16T15:51:01.371328+0000","last_fullsized":"2024-09-16T15:51:01.371328+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.387376+0000","last_change":"2024-09-16T15:51:01.381796+0000","last_active":"2024-09-16T15:51:01.387376+0000","last_peered":"2024-09-16T15:51:01.387376+0000","last_clean":"2024-09-16T15:51:01.387376+0000","last_became_active":"2024-09-16T15:51:01.381524+0000","last_became_peered":"2024-09-16T15:51:01.381524+0000","last_unstale":"2024-09-16T15:51:01.387376+0000","last_undegraded":"2024-09-16T15:51:01.387376+0000","last_fullsized":"2024-09-16T15:51:01.387376+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:29:47.623958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.368671+0000","last_change":"2024-09-16T15:51:01.368671+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.368671+0000","last_undegraded":"2024-09-16T15:51:01.368671+0000","last_fullsized":"2024-09-16T15:51:01.368671+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.383212+0000","last_change":"2024-09-16T15:51:01.380670+0000","last_active":"2024-09-16T15:51:01.383212+0000","last_peered":"2024-09-16T15:51:01.383212+0000","last_clean":"2024-09-16T15:51:01.383212+0000","last_became_active":"2024-09-16T15:51:01.380213+0000","last_became_peered":"2024-09-16T15:51:01.380213+0000","last_unstale":"2024-09-16T15:51:01.383212+0000","last_undegraded":"2024-09-16T15:51:01.383212+0000","last_fullsized":"2024-09-16T15:51:01.383212+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:54:34.716483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.379707+0000","last_change":"2024-09-16T15:51:01.379707+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.379707+0000","last_undegraded":"2024-09-16T15:51:01.379707+0000","last_fullsized":"2024-09-16T15:51:01.379707+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.390441+0000","last_change":"2024-09-16T15:51:01.388916+0000","last_active":"2024-09-16T15:51:01.390441+0000","last_peered":"2024-09-16T15:51:01.390441+0000","last_clean":"2024-09-16T15:51:01.390441+0000","last_became_active":"2024-09-16T15:51:01.388644+0000","last_became_peered":"2024-09-16T15:51:01.388644+0000","last_unstale":"2024-09-16T15:51:01.390441+0000","last_undegraded":"2024-09-16T15:51:01.390441+0000","last_fullsized":"2024-09-16T15:51:01.390441+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:38:33.558776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.368076+0000","last_change":"2024-09-16T15:51:01.368076+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.368076+0000","last_undegraded":"2024-09-16T15:51:01.368076+0000","last_fullsized":"2024-09-16T15:51:01.368076+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.388815+0000","last_change":"2024-09-16T15:51:01.387695+0000","last_active":"2024-09-16T15:51:01.388815+0000","last_peered":"2024-09-16T15:51:01.388815+0000","last_clean":"2024-09-16T15:51:01.388815+0000","last_became_active":"2024-09-16T15:51:01.387455+0000","last_became_peered":"2024-09-16T15:51:01.387455+0000","last_unstale":"2024-09-16T15:51:01.388815+0000","last_undegraded":"2024-09-16T15:51:01.388815+0000","last_fullsized":"2024-09-16T15:51:01.388815+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:23:23.665770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.378487+0000","last_change":"2024-09-16T15:51:01.378487+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.378487+0000","last_undegraded":"2024-09-16T15:51:01.378487+0000","last_fullsized":"2024-09-16T15:51:01.378487+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.308686+0000","last_change":"2024-09-16T15:51:01.384206+0000","last_active":"2024-09-16T15:51:04.308686+0000","last_peered":"2024-09-16T15:51:04.308686+0000","last_clean":"2024-09-16T15:51:04.308686+0000","last_became_active":"2024-09-16T15:51:01.383873+0000","last_became_peered":"2024-09-16T15:51:01.383873+0000","last_unstale":"2024-09-16T15:51:04.308686+0000","last_undegraded":"2024-09-16T15:51:04.308686+0000","last_fullsized":"2024-09-16T15:51:04.308686+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:15:03.468370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.367780+0000","last_change":"2024-09-16T15:51:01.367780+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.367780+0000","last_undegraded":"2024-09-16T15:51:01.367780+0000","last_fullsized":"2024-09-16T15:51:01.367780+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.392779+0000","last_change":"2024-09-16T15:51:01.390952+0000","last_active":"2024-09-16T15:51:01.392779+0000","last_peered":"2024-09-16T15:51:01.392779+0000","last_clean":"2024-09-16T15:51:01.392779+0000","last_became_active":"2024-09-16T15:51:01.390640+0000","last_became_peered":"2024-09-16T15:51:01.390640+0000","last_unstale":"2024-09-16T15:51:01.392779+0000","last_undegraded":"2024-09-16T15:51:01.392779+0000","last_fullsized":"2024-09-16T15:51:01.392779+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:04:41.524063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.370302+0000","last_change":"2024-09-16T15:51:01.370302+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.370302+0000","last_undegraded":"2024-09-16T15:51:01.370302+0000","last_fullsized":"2024-09-16T15:51:01.370302+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.388861+0000","last_change":"2024-09-16T15:51:01.388328+0000","last_active":"2024-09-16T15:51:01.388861+0000","last_peered":"2024-09-16T15:51:01.388861+0000","last_clean":"2024-09-16T15:51:01.388861+0000","last_became_active":"2024-09-16T15:51:01.388066+0000","last_became_peered":"2024-09-16T15:51:01.388066+0000","last_unstale":"2024-09-16T15:51:01.388861+0000","last_undegraded":"2024-09-16T15:51:01.388861+0000","last_fullsized":"2024-09-16T15:51:01.388861+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:22:46.801645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":262,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.367319+0000","last_change":"2024-09-16T15:39:12.698813+0000","last_active":"2024-09-16T15:51:01.367319+0000","last_peered":"2024-09-16T15:51:01.367319+0000","last_clean":"2024-09-16T15:51:01.367319+0000","last_became_active":"2024-09-16T15:39:12.697695+0000","last_became_peered":"2024-09-16T15:39:12.697695+0000","last_unstale":"2024-09-16T15:51:01.367319+0000","last_undegraded":"2024-09-16T15:51:01.367319+0000","last_fullsized":"2024-09-16T15:51:01.367319+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_clean_scrub_stamp":"2024-09-16T15:39:11.684967+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:02.061081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":122,"acting":122,"num_store_stats":7},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":582,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509587,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":148960,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222880,"statfs":{"total":96636764160,"available":96484229120,"internally_reserved":0,"allocated":1253376,"data_stored":544601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":151255496},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509586,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131488,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240352,"statfs":{"total":96636764160,"available":96502120448,"internally_reserved":0,"allocated":1847296,"data_stored":1134969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509585,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144800,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94227040,"statfs":{"total":96636764160,"available":96488488960,"internally_reserved":0,"allocated":1241088,"data_stored":538270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509585,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126752,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94245088,"statfs":{"total":96636764160,"available":96506970112,"internally_reserved":0,"allocated":1253376,"data_stored":544601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":128514505},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509585,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126740,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94245100,"statfs":{"total":96636764160,"available":96506982400,"internally_reserved":0,"allocated":1241088,"data_stored":538270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":128514503},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509586,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123160,"kb_used_data":1792,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248680,"statfs":{"total":96636764160,"available":96510648320,"internally_reserved":0,"allocated":1835008,"data_stored":1128638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":124254663},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509584,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":140612,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94231228,"statfs":{"total":96636764160,"available":96492777472,"internally_reserved":0,"allocated":1228800,"data_stored":532397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":142735815},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542292,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122620,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94249220,"statfs":{"total":96636764160,"available":96511201280,"internally_reserved":0,"allocated":1282048,"data_stored":558370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":124254668},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:51:07.021 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:51:07.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.215+0000 7f7c25400640 1 -- 172.21.15.151:0/2614219919 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c200ccf00 msgr2=0x7f7c200b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:07.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.215+0000 7f7c25400640 1 --2- 172.21.15.151:0/2614219919 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c200ccf00 0x7f7c200b2440 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f7c08005fd0 tx=0x7f7c0802a540 comp rx=0 tx=0).stop 2024-09-16T15:51:07.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.215+0000 7f7c25400640 1 -- 172.21.15.151:0/2614219919 shutdown_connections 2024-09-16T15:51:07.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.215+0000 7f7c25400640 1 --2- 172.21.15.151:0/2614219919 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7c20073fb0 0x7f7c20074440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:07.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.215+0000 7f7c25400640 1 --2- 172.21.15.151:0/2614219919 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c200ccf00 0x7f7c200b2440 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:07.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.215+0000 7f7c25400640 1 -- 172.21.15.151:0/2614219919 >> 172.21.15.151:0/2614219919 conn(0x7f7c200b1950 msgr2=0x7f7c200b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:07.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.215+0000 7f7c25400640 1 -- 172.21.15.151:0/2614219919 shutdown_connections 2024-09-16T15:51:07.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.215+0000 7f7c25400640 1 -- 172.21.15.151:0/2614219919 wait complete. 2024-09-16T15:51:07.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.216+0000 7f7c25400640 1 Processor -- start 2024-09-16T15:51:07.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.217+0000 7f7c25400640 1 -- start start 2024-09-16T15:51:07.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.217+0000 7f7c25400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c20073fb0 0x7f7c200d0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:07.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.217+0000 7f7c25400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7c200ccf00 0x7f7c20151f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:07.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.217+0000 7f7c25400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c2015f5a0 0x7f7c20158260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:07.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.217+0000 7f7c25400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7c2006f2e0 con 0x7f7c2015f5a0 2024-09-16T15:51:07.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.218+0000 7f7c25400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7c2006f160 con 0x7f7c20073fb0 2024-09-16T15:51:07.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.218+0000 7f7c25400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f7c2006f460 con 0x7f7c200ccf00 2024-09-16T15:51:07.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.218+0000 7f7c1f200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c2015f5a0 0x7f7c20158260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:07.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.218+0000 7f7c17e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7c200ccf00 0x7f7c20151f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:07.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.218+0000 7f7c17e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7c200ccf00 0x7f7c20151f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.151:36474/0 (socket says 172.21.15.151:36474) 2024-09-16T15:51:07.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.218+0000 7f7c17e00640 1 -- 172.21.15.151:0/4118557324 learned_addr learned my addr 172.21.15.151:0/4118557324 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:51:07.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.218+0000 7f7c1f200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c2015f5a0 0x7f7c20158260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52662/0 (socket says 172.21.15.151:52662) 2024-09-16T15:51:07.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.218+0000 7f7c1e800640 1 --2- 172.21.15.151:0/4118557324 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c20073fb0 0x7f7c200d0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:07.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.218+0000 7f7c17e00640 1 -- 172.21.15.151:0/4118557324 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c20073fb0 msgr2=0x7f7c200d0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:07.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.218+0000 7f7c17e00640 1 --2- 172.21.15.151:0/4118557324 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c20073fb0 0x7f7c200d0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:07.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.218+0000 7f7c17e00640 1 -- 172.21.15.151:0/4118557324 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c2015f5a0 msgr2=0x7f7c20158260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:07.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.218+0000 7f7c17e00640 1 --2- 172.21.15.151:0/4118557324 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c2015f5a0 0x7f7c20158260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:07.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.218+0000 7f7c17e00640 1 -- 172.21.15.151:0/4118557324 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c201f4bc0 con 0x7f7c200ccf00 2024-09-16T15:51:07.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.218+0000 7f7c1f200640 1 --2- 172.21.15.151:0/4118557324 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c2015f5a0 0x7f7c20158260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:51:07.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.219+0000 7f7c17e00640 1 --2- 172.21.15.151:0/4118557324 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7c200ccf00 0x7f7c20151f40 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7c08009850 tx=0x7f7c080038a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:07.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.219+0000 7f7c16a00640 1 -- 172.21.15.151:0/4118557324 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0803f070 con 0x7f7c200ccf00 2024-09-16T15:51:07.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.219+0000 7f7c16a00640 1 -- 172.21.15.151:0/4118557324 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7c080047a0 con 0x7f7c200ccf00 2024-09-16T15:51:07.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.219+0000 7f7c25400640 1 -- 172.21.15.151:0/4118557324 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c201f4ea0 con 0x7f7c200ccf00 2024-09-16T15:51:07.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.219+0000 7f7c16a00640 1 -- 172.21.15.151:0/4118557324 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c08035030 con 0x7f7c200ccf00 2024-09-16T15:51:07.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.219+0000 7f7c25400640 1 -- 172.21.15.151:0/4118557324 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c201f5460 con 0x7f7c200ccf00 2024-09-16T15:51:07.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.220+0000 7f7c25400640 1 -- 172.21.15.151:0/4118557324 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c2006f980 con 0x7f7c200ccf00 2024-09-16T15:51:07.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.222+0000 7f7c16a00640 1 -- 172.21.15.151:0/4118557324 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7f7c08042070 con 0x7f7c200ccf00 2024-09-16T15:51:07.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.222+0000 7f7c16a00640 1 --2- 172.21.15.151:0/4118557324 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7bfc079ed0 0x7f7bfc07c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:07.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.223+0000 7f7c16a00640 1 -- 172.21.15.151:0/4118557324 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(104..104 src has 1..104) ==== 5765+0+0 (secure 0 0 0) 0x7f7c080ba900 con 0x7f7c200ccf00 2024-09-16T15:51:07.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.223+0000 7f7c1e800640 1 --2- 172.21.15.151:0/4118557324 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7bfc079ed0 0x7f7bfc07c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:07.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.224+0000 7f7c1e800640 1 --2- 172.21.15.151:0/4118557324 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7bfc079ed0 0x7f7bfc07c390 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f7c201559e0 tx=0x7f7c10009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:07.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.226+0000 7f7c16a00640 1 -- 172.21.15.151:0/4118557324 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7c08084400 con 0x7f7c200ccf00 2024-09-16T15:51:07.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.412+0000 7f7c25400640 1 -- 172.21.15.151:0/4118557324 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7c201591c0 con 0x7f7bfc079ed0 2024-09-16T15:51:07.419 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.417+0000 7f7c16a00640 1 -- 172.21.15.151:0/4118557324 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+248219 (secure 0 0 0) 0x7f7c201591c0 con 0x7f7bfc079ed0 2024-09-16T15:51:07.424 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:51:07.425 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-09-16T15:51:07.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.422+0000 7f7c25400640 1 -- 172.21.15.151:0/4118557324 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7bfc079ed0 msgr2=0x7f7bfc07c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:07.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.422+0000 7f7c25400640 1 --2- 172.21.15.151:0/4118557324 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7bfc079ed0 0x7f7bfc07c390 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f7c201559e0 tx=0x7f7c10009040 comp rx=0 tx=0).stop 2024-09-16T15:51:07.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.422+0000 7f7c25400640 1 -- 172.21.15.151:0/4118557324 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7c200ccf00 msgr2=0x7f7c20151f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:07.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.422+0000 7f7c25400640 1 --2- 172.21.15.151:0/4118557324 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7c200ccf00 0x7f7c20151f40 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7c08009850 tx=0x7f7c080038a0 comp rx=0 tx=0).stop 2024-09-16T15:51:07.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.422+0000 7f7c25400640 1 -- 172.21.15.151:0/4118557324 shutdown_connections 2024-09-16T15:51:07.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.422+0000 7f7c25400640 1 --2- 172.21.15.151:0/4118557324 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7bfc079ed0 0x7f7bfc07c390 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:07.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.422+0000 7f7c25400640 1 --2- 172.21.15.151:0/4118557324 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c2015f5a0 0x7f7c20158260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:07.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.422+0000 7f7c25400640 1 --2- 172.21.15.151:0/4118557324 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7c200ccf00 0x7f7c20151f40 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:07.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.422+0000 7f7c25400640 1 --2- 172.21.15.151:0/4118557324 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c20073fb0 0x7f7c200d0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:07.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.422+0000 7f7c25400640 1 -- 172.21.15.151:0/4118557324 >> 172.21.15.151:0/4118557324 conn(0x7f7c200b1950 msgr2=0x7f7c200b10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:07.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.422+0000 7f7c25400640 1 -- 172.21.15.151:0/4118557324 shutdown_connections 2024-09-16T15:51:07.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.422+0000 7f7c25400640 1 -- 172.21.15.151:0/4118557324 wait complete. 2024-09-16T15:51:07.440 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":455,"stamp":"2024-09-16T15:51:05.880418+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590950,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":121,"num_write_kb":1629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":89,"ondisk_log_size":89,"up":188,"acting":188,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":27,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1065132,"kb_used_data":10920,"kb_used_omap":12,"kb_used_meta":1054003,"kb_avail":753909588,"statfs":{"total":773094113280,"available":772003418112,"internally_reserved":0,"allocated":11182080,"data_stored":5520116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12738,"internal_metadata":1079299646},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":582,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.518821"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T15:51:02.391024+0000","last_change":"2024-09-16T15:51:02.386431+0000","last_active":"2024-09-16T15:51:02.391024+0000","last_peered":"2024-09-16T15:51:02.391024+0000","last_clean":"2024-09-16T15:51:02.391024+0000","last_became_active":"2024-09-16T15:51:02.386126+0000","last_became_peered":"2024-09-16T15:51:02.386126+0000","last_unstale":"2024-09-16T15:51:02.391024+0000","last_undegraded":"2024-09-16T15:51:02.391024+0000","last_fullsized":"2024-09-16T15:51:02.391024+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:25:09.365288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T15:51:02.400467+0000","last_change":"2024-09-16T15:51:02.388621+0000","last_active":"2024-09-16T15:51:02.400467+0000","last_peered":"2024-09-16T15:51:02.400467+0000","last_clean":"2024-09-16T15:51:02.400467+0000","last_became_active":"2024-09-16T15:51:02.388294+0000","last_became_peered":"2024-09-16T15:51:02.388294+0000","last_unstale":"2024-09-16T15:51:02.400467+0000","last_undegraded":"2024-09-16T15:51:02.400467+0000","last_fullsized":"2024-09-16T15:51:02.400467+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:43:28.544765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.369847+0000","last_change":"2024-09-16T15:51:01.369847+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.369847+0000","last_undegraded":"2024-09-16T15:51:01.369847+0000","last_fullsized":"2024-09-16T15:51:01.369847+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T15:51:02.387475+0000","last_change":"2024-09-16T15:51:02.385219+0000","last_active":"2024-09-16T15:51:02.387475+0000","last_peered":"2024-09-16T15:51:02.387475+0000","last_clean":"2024-09-16T15:51:02.387475+0000","last_became_active":"2024-09-16T15:51:02.384310+0000","last_became_peered":"2024-09-16T15:51:02.384310+0000","last_unstale":"2024-09-16T15:51:02.387475+0000","last_undegraded":"2024-09-16T15:51:02.387475+0000","last_fullsized":"2024-09-16T15:51:02.387475+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:48:47.559516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.374870+0000","last_change":"2024-09-16T15:51:01.374870+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.374870+0000","last_undegraded":"2024-09-16T15:51:01.374870+0000","last_fullsized":"2024-09-16T15:51:01.374870+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.374553+0000","last_change":"2024-09-16T15:51:01.374553+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.374553+0000","last_undegraded":"2024-09-16T15:51:01.374553+0000","last_fullsized":"2024-09-16T15:51:01.374553+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.372119+0000","last_change":"2024-09-16T15:51:01.372119+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.372119+0000","last_undegraded":"2024-09-16T15:51:01.372119+0000","last_fullsized":"2024-09-16T15:51:01.372119+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.370748+0000","last_change":"2024-09-16T15:51:01.370748+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.370748+0000","last_undegraded":"2024-09-16T15:51:01.370748+0000","last_fullsized":"2024-09-16T15:51:01.370748+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.373077+0000","last_change":"2024-09-16T15:51:01.373077+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.373077+0000","last_undegraded":"2024-09-16T15:51:01.373077+0000","last_fullsized":"2024-09-16T15:51:01.373077+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:51:01.355970+0000","last_change":"2024-09-16T15:51:01.355970+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.355970+0000","last_undegraded":"2024-09-16T15:51:01.355970+0000","last_fullsized":"2024-09-16T15:51:01.355970+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.374982+0000","last_change":"2024-09-16T15:51:01.374982+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.374982+0000","last_undegraded":"2024-09-16T15:51:01.374982+0000","last_fullsized":"2024-09-16T15:51:01.374982+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.373278+0000","last_change":"2024-09-16T15:51:01.373278+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.373278+0000","last_undegraded":"2024-09-16T15:51:01.373278+0000","last_fullsized":"2024-09-16T15:51:01.373278+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309716+0000","last_change":"2024-09-16T15:51:02.407269+0000","last_active":"2024-09-16T15:51:04.309716+0000","last_peered":"2024-09-16T15:51:04.309716+0000","last_clean":"2024-09-16T15:51:04.309716+0000","last_became_active":"2024-09-16T15:51:02.406943+0000","last_became_peered":"2024-09-16T15:51:02.406943+0000","last_unstale":"2024-09-16T15:51:04.309716+0000","last_undegraded":"2024-09-16T15:51:04.309716+0000","last_fullsized":"2024-09-16T15:51:04.309716+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:29:33.561032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T15:51:02.405765+0000","last_change":"2024-09-16T15:51:02.398129+0000","last_active":"2024-09-16T15:51:02.405765+0000","last_peered":"2024-09-16T15:51:02.405765+0000","last_clean":"2024-09-16T15:51:02.405765+0000","last_became_active":"2024-09-16T15:51:02.397834+0000","last_became_peered":"2024-09-16T15:51:02.397834+0000","last_unstale":"2024-09-16T15:51:02.405765+0000","last_undegraded":"2024-09-16T15:51:02.405765+0000","last_fullsized":"2024-09-16T15:51:02.405765+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:35:17.388414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T15:51:02.404562+0000","last_change":"2024-09-16T15:51:02.396682+0000","last_active":"2024-09-16T15:51:02.404562+0000","last_peered":"2024-09-16T15:51:02.404562+0000","last_clean":"2024-09-16T15:51:02.404562+0000","last_became_active":"2024-09-16T15:51:02.396412+0000","last_became_peered":"2024-09-16T15:51:02.396412+0000","last_unstale":"2024-09-16T15:51:02.404562+0000","last_undegraded":"2024-09-16T15:51:02.404562+0000","last_fullsized":"2024-09-16T15:51:02.404562+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:52:38.771302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.371468+0000","last_change":"2024-09-16T15:51:01.371468+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.371468+0000","last_undegraded":"2024-09-16T15:51:01.371468+0000","last_fullsized":"2024-09-16T15:51:01.371468+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.379608+0000","last_change":"2024-09-16T15:51:01.379608+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.379608+0000","last_undegraded":"2024-09-16T15:51:01.379608+0000","last_fullsized":"2024-09-16T15:51:01.379608+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T15:51:02.404957+0000","last_change":"2024-09-16T15:51:02.399241+0000","last_active":"2024-09-16T15:51:02.404957+0000","last_peered":"2024-09-16T15:51:02.404957+0000","last_clean":"2024-09-16T15:51:02.404957+0000","last_became_active":"2024-09-16T15:51:02.398973+0000","last_became_peered":"2024-09-16T15:51:02.398973+0000","last_unstale":"2024-09-16T15:51:02.404957+0000","last_undegraded":"2024-09-16T15:51:02.404957+0000","last_fullsized":"2024-09-16T15:51:02.404957+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:46:06.817591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.372608+0000","last_change":"2024-09-16T15:51:01.372608+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.372608+0000","last_undegraded":"2024-09-16T15:51:01.372608+0000","last_fullsized":"2024-09-16T15:51:01.372608+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.370913+0000","last_change":"2024-09-16T15:51:01.370913+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.370913+0000","last_undegraded":"2024-09-16T15:51:01.370913+0000","last_fullsized":"2024-09-16T15:51:01.370913+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.371225+0000","last_change":"2024-09-16T15:51:01.371225+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.371225+0000","last_undegraded":"2024-09-16T15:51:01.371225+0000","last_fullsized":"2024-09-16T15:51:01.371225+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.368817+0000","last_change":"2024-09-16T15:51:01.368817+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.368817+0000","last_undegraded":"2024-09-16T15:51:01.368817+0000","last_fullsized":"2024-09-16T15:51:01.368817+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T15:51:02.405411+0000","last_change":"2024-09-16T15:51:02.399544+0000","last_active":"2024-09-16T15:51:02.405411+0000","last_peered":"2024-09-16T15:51:02.405411+0000","last_clean":"2024-09-16T15:51:02.405411+0000","last_became_active":"2024-09-16T15:51:02.396515+0000","last_became_peered":"2024-09-16T15:51:02.396515+0000","last_unstale":"2024-09-16T15:51:02.405411+0000","last_undegraded":"2024-09-16T15:51:02.405411+0000","last_fullsized":"2024-09-16T15:51:02.405411+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:05:52.491919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.371334+0000","last_change":"2024-09-16T15:51:01.371334+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.371334+0000","last_undegraded":"2024-09-16T15:51:01.371334+0000","last_fullsized":"2024-09-16T15:51:01.371334+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:51:01.355970+0000","last_change":"2024-09-16T15:51:01.355970+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.355970+0000","last_undegraded":"2024-09-16T15:51:01.355970+0000","last_fullsized":"2024-09-16T15:51:01.355970+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309644+0000","last_change":"2024-09-16T15:51:02.401422+0000","last_active":"2024-09-16T15:51:04.309644+0000","last_peered":"2024-09-16T15:51:04.309644+0000","last_clean":"2024-09-16T15:51:04.309644+0000","last_became_active":"2024-09-16T15:51:02.400804+0000","last_became_peered":"2024-09-16T15:51:02.400804+0000","last_unstale":"2024-09-16T15:51:04.309644+0000","last_undegraded":"2024-09-16T15:51:04.309644+0000","last_fullsized":"2024-09-16T15:51:04.309644+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:54:09.898078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.371083+0000","last_change":"2024-09-16T15:51:01.371083+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.371083+0000","last_undegraded":"2024-09-16T15:51:01.371083+0000","last_fullsized":"2024-09-16T15:51:01.371083+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.368057+0000","last_change":"2024-09-16T15:51:01.368057+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.368057+0000","last_undegraded":"2024-09-16T15:51:01.368057+0000","last_fullsized":"2024-09-16T15:51:01.368057+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309979+0000","last_change":"2024-09-16T15:51:02.405833+0000","last_active":"2024-09-16T15:51:04.309979+0000","last_peered":"2024-09-16T15:51:04.309979+0000","last_clean":"2024-09-16T15:51:04.309979+0000","last_became_active":"2024-09-16T15:51:02.404364+0000","last_became_peered":"2024-09-16T15:51:02.404364+0000","last_unstale":"2024-09-16T15:51:04.309979+0000","last_undegraded":"2024-09-16T15:51:04.309979+0000","last_fullsized":"2024-09-16T15:51:04.309979+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:47:54.421341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309062+0000","last_change":"2024-09-16T15:51:02.402965+0000","last_active":"2024-09-16T15:51:04.309062+0000","last_peered":"2024-09-16T15:51:04.309062+0000","last_clean":"2024-09-16T15:51:04.309062+0000","last_became_active":"2024-09-16T15:51:02.401107+0000","last_became_peered":"2024-09-16T15:51:02.401107+0000","last_unstale":"2024-09-16T15:51:04.309062+0000","last_undegraded":"2024-09-16T15:51:04.309062+0000","last_fullsized":"2024-09-16T15:51:04.309062+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:24:38.932170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.368886+0000","last_change":"2024-09-16T15:51:01.368886+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.368886+0000","last_undegraded":"2024-09-16T15:51:01.368886+0000","last_fullsized":"2024-09-16T15:51:01.368886+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T15:51:02.387143+0000","last_change":"2024-09-16T15:51:02.384379+0000","last_active":"2024-09-16T15:51:02.387143+0000","last_peered":"2024-09-16T15:51:02.387143+0000","last_clean":"2024-09-16T15:51:02.387143+0000","last_became_active":"2024-09-16T15:51:02.384012+0000","last_became_peered":"2024-09-16T15:51:02.384012+0000","last_unstale":"2024-09-16T15:51:02.387143+0000","last_undegraded":"2024-09-16T15:51:02.387143+0000","last_fullsized":"2024-09-16T15:51:02.387143+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:04:09.954852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.382918+0000","last_change":"2024-09-16T15:51:01.382918+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.382918+0000","last_undegraded":"2024-09-16T15:51:01.382918+0000","last_fullsized":"2024-09-16T15:51:01.382918+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1f","version":"103'4","reported_seq":19,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309998+0000","last_change":"2024-09-16T15:51:01.375200+0000","last_active":"2024-09-16T15:51:04.309998+0000","last_peered":"2024-09-16T15:51:04.309998+0000","last_clean":"2024-09-16T15:51:04.309998+0000","last_became_active":"2024-09-16T15:51:01.374919+0000","last_became_peered":"2024-09-16T15:51:01.374919+0000","last_unstale":"2024-09-16T15:51:04.309998+0000","last_undegraded":"2024-09-16T15:51:04.309998+0000","last_fullsized":"2024-09-16T15:51:04.309998+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:36:44.338407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.310065+0000","last_change":"2024-09-16T15:51:02.402988+0000","last_active":"2024-09-16T15:51:04.310065+0000","last_peered":"2024-09-16T15:51:04.310065+0000","last_clean":"2024-09-16T15:51:04.310065+0000","last_became_active":"2024-09-16T15:51:02.402708+0000","last_became_peered":"2024-09-16T15:51:02.402708+0000","last_unstale":"2024-09-16T15:51:04.310065+0000","last_undegraded":"2024-09-16T15:51:04.310065+0000","last_fullsized":"2024-09-16T15:51:04.310065+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:14:48.952444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.392214+0000","last_change":"2024-09-16T15:51:01.391174+0000","last_active":"2024-09-16T15:51:01.392214+0000","last_peered":"2024-09-16T15:51:01.392214+0000","last_clean":"2024-09-16T15:51:01.392214+0000","last_became_active":"2024-09-16T15:51:01.390904+0000","last_became_peered":"2024-09-16T15:51:01.390904+0000","last_unstale":"2024-09-16T15:51:01.392214+0000","last_undegraded":"2024-09-16T15:51:01.392214+0000","last_fullsized":"2024-09-16T15:51:01.392214+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:15:52.955571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T15:51:02.386883+0000","last_change":"2024-09-16T15:51:02.382876+0000","last_active":"2024-09-16T15:51:02.386883+0000","last_peered":"2024-09-16T15:51:02.386883+0000","last_clean":"2024-09-16T15:51:02.386883+0000","last_became_active":"2024-09-16T15:51:02.382617+0000","last_became_peered":"2024-09-16T15:51:02.382617+0000","last_unstale":"2024-09-16T15:51:02.386883+0000","last_undegraded":"2024-09-16T15:51:02.386883+0000","last_fullsized":"2024-09-16T15:51:02.386883+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:19:48.933205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.390016+0000","last_change":"2024-09-16T15:51:01.388774+0000","last_active":"2024-09-16T15:51:01.390016+0000","last_peered":"2024-09-16T15:51:01.390016+0000","last_clean":"2024-09-16T15:51:01.390016+0000","last_became_active":"2024-09-16T15:51:01.388500+0000","last_became_peered":"2024-09-16T15:51:01.388500+0000","last_unstale":"2024-09-16T15:51:01.390016+0000","last_undegraded":"2024-09-16T15:51:01.390016+0000","last_fullsized":"2024-09-16T15:51:01.390016+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:08:59.734743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:51:01.355970+0000","last_change":"2024-09-16T15:51:01.355970+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.355970+0000","last_undegraded":"2024-09-16T15:51:01.355970+0000","last_fullsized":"2024-09-16T15:51:01.355970+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1c","version":"103'2","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T15:51:02.414797+0000","last_change":"2024-09-16T15:51:01.390985+0000","last_active":"2024-09-16T15:51:02.414797+0000","last_peered":"2024-09-16T15:51:02.414797+0000","last_clean":"2024-09-16T15:51:02.414797+0000","last_became_active":"2024-09-16T15:51:01.390463+0000","last_became_peered":"2024-09-16T15:51:01.390463+0000","last_unstale":"2024-09-16T15:51:02.414797+0000","last_undegraded":"2024-09-16T15:51:02.414797+0000","last_fullsized":"2024-09-16T15:51:02.414797+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:57:05.221628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.370455+0000","last_change":"2024-09-16T15:51:01.370455+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.370455+0000","last_undegraded":"2024-09-16T15:51:01.370455+0000","last_fullsized":"2024-09-16T15:51:01.370455+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1b","version":"103'1","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T15:51:02.411975+0000","last_change":"2024-09-16T15:51:01.387614+0000","last_active":"2024-09-16T15:51:02.411975+0000","last_peered":"2024-09-16T15:51:02.411975+0000","last_clean":"2024-09-16T15:51:02.411975+0000","last_became_active":"2024-09-16T15:51:01.387163+0000","last_became_peered":"2024-09-16T15:51:01.387163+0000","last_unstale":"2024-09-16T15:51:02.411975+0000","last_undegraded":"2024-09-16T15:51:02.411975+0000","last_fullsized":"2024-09-16T15:51:02.411975+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:48:24.298645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.370317+0000","last_change":"2024-09-16T15:51:01.370317+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.370317+0000","last_undegraded":"2024-09-16T15:51:01.370317+0000","last_fullsized":"2024-09-16T15:51:01.370317+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.383140+0000","last_change":"2024-09-16T15:51:01.378848+0000","last_active":"2024-09-16T15:51:01.383140+0000","last_peered":"2024-09-16T15:51:01.383140+0000","last_clean":"2024-09-16T15:51:01.383140+0000","last_became_active":"2024-09-16T15:51:01.377621+0000","last_became_peered":"2024-09-16T15:51:01.377621+0000","last_unstale":"2024-09-16T15:51:01.383140+0000","last_undegraded":"2024-09-16T15:51:01.383140+0000","last_fullsized":"2024-09-16T15:51:01.383140+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:28:49.058738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.308778+0000","last_change":"2024-09-16T15:51:02.405495+0000","last_active":"2024-09-16T15:51:04.308778+0000","last_peered":"2024-09-16T15:51:04.308778+0000","last_clean":"2024-09-16T15:51:04.308778+0000","last_became_active":"2024-09-16T15:51:02.405222+0000","last_became_peered":"2024-09-16T15:51:02.405222+0000","last_unstale":"2024-09-16T15:51:04.308778+0000","last_undegraded":"2024-09-16T15:51:04.308778+0000","last_fullsized":"2024-09-16T15:51:04.308778+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:22:02.790695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.378694+0000","last_change":"2024-09-16T15:51:01.372791+0000","last_active":"2024-09-16T15:51:01.378694+0000","last_peered":"2024-09-16T15:51:01.378694+0000","last_clean":"2024-09-16T15:51:01.378694+0000","last_became_active":"2024-09-16T15:51:01.372493+0000","last_became_peered":"2024-09-16T15:51:01.372493+0000","last_unstale":"2024-09-16T15:51:01.378694+0000","last_undegraded":"2024-09-16T15:51:01.378694+0000","last_fullsized":"2024-09-16T15:51:01.378694+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:24:18.852411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.369717+0000","last_change":"2024-09-16T15:51:01.369717+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.369717+0000","last_undegraded":"2024-09-16T15:51:01.369717+0000","last_fullsized":"2024-09-16T15:51:01.369717+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.389654+0000","last_change":"2024-09-16T15:51:01.386550+0000","last_active":"2024-09-16T15:51:01.389654+0000","last_peered":"2024-09-16T15:51:01.389654+0000","last_clean":"2024-09-16T15:51:01.389654+0000","last_became_active":"2024-09-16T15:51:01.386275+0000","last_became_peered":"2024-09-16T15:51:01.386275+0000","last_unstale":"2024-09-16T15:51:01.389654+0000","last_undegraded":"2024-09-16T15:51:01.389654+0000","last_fullsized":"2024-09-16T15:51:01.389654+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:33:03.973467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T15:51:02.391064+0000","last_change":"2024-09-16T15:51:02.386731+0000","last_active":"2024-09-16T15:51:02.391064+0000","last_peered":"2024-09-16T15:51:02.391064+0000","last_clean":"2024-09-16T15:51:02.391064+0000","last_became_active":"2024-09-16T15:51:02.384022+0000","last_became_peered":"2024-09-16T15:51:02.384022+0000","last_unstale":"2024-09-16T15:51:02.391064+0000","last_undegraded":"2024-09-16T15:51:02.391064+0000","last_fullsized":"2024-09-16T15:51:02.391064+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:10:52.384143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.381974+0000","last_change":"2024-09-16T15:51:01.378467+0000","last_active":"2024-09-16T15:51:01.381974+0000","last_peered":"2024-09-16T15:51:01.381974+0000","last_clean":"2024-09-16T15:51:01.381974+0000","last_became_active":"2024-09-16T15:51:01.378205+0000","last_became_peered":"2024-09-16T15:51:01.378205+0000","last_unstale":"2024-09-16T15:51:01.381974+0000","last_undegraded":"2024-09-16T15:51:01.381974+0000","last_fullsized":"2024-09-16T15:51:01.381974+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:53:22.512677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.368015+0000","last_change":"2024-09-16T15:51:01.368015+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.368015+0000","last_undegraded":"2024-09-16T15:51:01.368015+0000","last_fullsized":"2024-09-16T15:51:01.368015+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T15:51:02.384082+0000","last_change":"2024-09-16T15:51:01.389590+0000","last_active":"2024-09-16T15:51:02.384082+0000","last_peered":"2024-09-16T15:51:02.384082+0000","last_clean":"2024-09-16T15:51:02.384082+0000","last_became_active":"2024-09-16T15:51:01.389186+0000","last_became_peered":"2024-09-16T15:51:01.389186+0000","last_unstale":"2024-09-16T15:51:02.384082+0000","last_undegraded":"2024-09-16T15:51:02.384082+0000","last_fullsized":"2024-09-16T15:51:02.384082+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:22:23.409721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.369935+0000","last_change":"2024-09-16T15:51:01.369935+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.369935+0000","last_undegraded":"2024-09-16T15:51:01.369935+0000","last_fullsized":"2024-09-16T15:51:01.369935+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.382259+0000","last_change":"2024-09-16T15:51:01.379431+0000","last_active":"2024-09-16T15:51:01.382259+0000","last_peered":"2024-09-16T15:51:01.382259+0000","last_clean":"2024-09-16T15:51:01.382259+0000","last_became_active":"2024-09-16T15:51:01.379201+0000","last_became_peered":"2024-09-16T15:51:01.379201+0000","last_unstale":"2024-09-16T15:51:01.382259+0000","last_undegraded":"2024-09-16T15:51:01.382259+0000","last_fullsized":"2024-09-16T15:51:01.382259+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:40:55.915776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.367826+0000","last_change":"2024-09-16T15:51:01.367826+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.367826+0000","last_undegraded":"2024-09-16T15:51:01.367826+0000","last_fullsized":"2024-09-16T15:51:01.367826+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.380263+0000","last_change":"2024-09-16T15:51:01.378141+0000","last_active":"2024-09-16T15:51:01.380263+0000","last_peered":"2024-09-16T15:51:01.380263+0000","last_clean":"2024-09-16T15:51:01.380263+0000","last_became_active":"2024-09-16T15:51:01.377875+0000","last_became_peered":"2024-09-16T15:51:01.377875+0000","last_unstale":"2024-09-16T15:51:01.380263+0000","last_undegraded":"2024-09-16T15:51:01.380263+0000","last_fullsized":"2024-09-16T15:51:01.380263+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:12:01.247706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T15:51:02.390885+0000","last_change":"2024-09-16T15:51:02.387140+0000","last_active":"2024-09-16T15:51:02.390885+0000","last_peered":"2024-09-16T15:51:02.390885+0000","last_clean":"2024-09-16T15:51:02.390885+0000","last_became_active":"2024-09-16T15:51:02.385974+0000","last_became_peered":"2024-09-16T15:51:02.385974+0000","last_unstale":"2024-09-16T15:51:02.390885+0000","last_undegraded":"2024-09-16T15:51:02.390885+0000","last_fullsized":"2024-09-16T15:51:02.390885+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:48:40.504107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.387346+0000","last_change":"2024-09-16T15:51:01.384546+0000","last_active":"2024-09-16T15:51:01.387346+0000","last_peered":"2024-09-16T15:51:01.387346+0000","last_clean":"2024-09-16T15:51:01.387346+0000","last_became_active":"2024-09-16T15:51:01.384288+0000","last_became_peered":"2024-09-16T15:51:01.384288+0000","last_unstale":"2024-09-16T15:51:01.387346+0000","last_undegraded":"2024-09-16T15:51:01.387346+0000","last_fullsized":"2024-09-16T15:51:01.387346+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:01:08.875366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.368875+0000","last_change":"2024-09-16T15:51:01.368875+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.368875+0000","last_undegraded":"2024-09-16T15:51:01.368875+0000","last_fullsized":"2024-09-16T15:51:01.368875+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T15:51:02.380598+0000","last_change":"2024-09-16T15:51:01.384749+0000","last_active":"2024-09-16T15:51:02.380598+0000","last_peered":"2024-09-16T15:51:02.380598+0000","last_clean":"2024-09-16T15:51:02.380598+0000","last_became_active":"2024-09-16T15:51:01.384433+0000","last_became_peered":"2024-09-16T15:51:01.384433+0000","last_unstale":"2024-09-16T15:51:02.380598+0000","last_undegraded":"2024-09-16T15:51:02.380598+0000","last_fullsized":"2024-09-16T15:51:02.380598+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:43:12.283062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.368901+0000","last_change":"2024-09-16T15:51:01.368901+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.368901+0000","last_undegraded":"2024-09-16T15:51:01.368901+0000","last_fullsized":"2024-09-16T15:51:01.368901+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.380285+0000","last_change":"2024-09-16T15:51:01.378352+0000","last_active":"2024-09-16T15:51:01.380285+0000","last_peered":"2024-09-16T15:51:01.380285+0000","last_clean":"2024-09-16T15:51:01.380285+0000","last_became_active":"2024-09-16T15:51:01.376625+0000","last_became_peered":"2024-09-16T15:51:01.376625+0000","last_unstale":"2024-09-16T15:51:01.380285+0000","last_undegraded":"2024-09-16T15:51:01.380285+0000","last_fullsized":"2024-09-16T15:51:01.380285+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:16:13.783024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.367556+0000","last_change":"2024-09-16T15:51:01.367556+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.367556+0000","last_undegraded":"2024-09-16T15:51:01.367556+0000","last_fullsized":"2024-09-16T15:51:01.367556+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.10","version":"103'8","reported_seq":21,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T15:51:02.415518+0000","last_change":"2024-09-16T15:51:01.383280+0000","last_active":"2024-09-16T15:51:02.415518+0000","last_peered":"2024-09-16T15:51:02.415518+0000","last_clean":"2024-09-16T15:51:02.415518+0000","last_became_active":"2024-09-16T15:51:01.381141+0000","last_became_peered":"2024-09-16T15:51:01.381141+0000","last_unstale":"2024-09-16T15:51:02.415518+0000","last_undegraded":"2024-09-16T15:51:02.415518+0000","last_fullsized":"2024-09-16T15:51:02.415518+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:10:44.749157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.367595+0000","last_change":"2024-09-16T15:51:01.367595+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.367595+0000","last_undegraded":"2024-09-16T15:51:01.367595+0000","last_fullsized":"2024-09-16T15:51:01.367595+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.391952+0000","last_change":"2024-09-16T15:51:01.391228+0000","last_active":"2024-09-16T15:51:01.391952+0000","last_peered":"2024-09-16T15:51:01.391952+0000","last_clean":"2024-09-16T15:51:01.391952+0000","last_became_active":"2024-09-16T15:51:01.390919+0000","last_became_peered":"2024-09-16T15:51:01.390919+0000","last_unstale":"2024-09-16T15:51:01.391952+0000","last_undegraded":"2024-09-16T15:51:01.391952+0000","last_fullsized":"2024-09-16T15:51:01.391952+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:16:20.956001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.367237+0000","last_change":"2024-09-16T15:51:01.367237+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.367237+0000","last_undegraded":"2024-09-16T15:51:01.367237+0000","last_fullsized":"2024-09-16T15:51:01.367237+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.382149+0000","last_change":"2024-09-16T15:51:01.377467+0000","last_active":"2024-09-16T15:51:01.382149+0000","last_peered":"2024-09-16T15:51:01.382149+0000","last_clean":"2024-09-16T15:51:01.382149+0000","last_became_active":"2024-09-16T15:51:01.377180+0000","last_became_peered":"2024-09-16T15:51:01.377180+0000","last_unstale":"2024-09-16T15:51:01.382149+0000","last_undegraded":"2024-09-16T15:51:01.382149+0000","last_fullsized":"2024-09-16T15:51:01.382149+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:07:47.076318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.368491+0000","last_change":"2024-09-16T15:51:01.368491+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.368491+0000","last_undegraded":"2024-09-16T15:51:01.368491+0000","last_fullsized":"2024-09-16T15:51:01.368491+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.387295+0000","last_change":"2024-09-16T15:51:01.385783+0000","last_active":"2024-09-16T15:51:01.387295+0000","last_peered":"2024-09-16T15:51:01.387295+0000","last_clean":"2024-09-16T15:51:01.387295+0000","last_became_active":"2024-09-16T15:51:01.385500+0000","last_became_peered":"2024-09-16T15:51:01.385500+0000","last_unstale":"2024-09-16T15:51:01.387295+0000","last_undegraded":"2024-09-16T15:51:01.387295+0000","last_fullsized":"2024-09-16T15:51:01.387295+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:53:45.571107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.380115+0000","last_change":"2024-09-16T15:51:01.380115+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.380115+0000","last_undegraded":"2024-09-16T15:51:01.380115+0000","last_fullsized":"2024-09-16T15:51:01.380115+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"103'2","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T15:51:02.412322+0000","last_change":"2024-09-16T15:51:01.391626+0000","last_active":"2024-09-16T15:51:02.412322+0000","last_peered":"2024-09-16T15:51:02.412322+0000","last_clean":"2024-09-16T15:51:02.412322+0000","last_became_active":"2024-09-16T15:51:01.391322+0000","last_became_peered":"2024-09-16T15:51:01.391322+0000","last_unstale":"2024-09-16T15:51:02.412322+0000","last_undegraded":"2024-09-16T15:51:02.412322+0000","last_fullsized":"2024-09-16T15:51:02.412322+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:13:41.121392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.376646+0000","last_change":"2024-09-16T15:51:01.376646+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.376646+0000","last_undegraded":"2024-09-16T15:51:01.376646+0000","last_fullsized":"2024-09-16T15:51:01.376646+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.380454+0000","last_change":"2024-09-16T15:51:01.378559+0000","last_active":"2024-09-16T15:51:01.380454+0000","last_peered":"2024-09-16T15:51:01.380454+0000","last_clean":"2024-09-16T15:51:01.380454+0000","last_became_active":"2024-09-16T15:51:01.377521+0000","last_became_peered":"2024-09-16T15:51:01.377521+0000","last_unstale":"2024-09-16T15:51:01.380454+0000","last_undegraded":"2024-09-16T15:51:01.380454+0000","last_fullsized":"2024-09-16T15:51:01.380454+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:45:48.115715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-09-16T15:51:02.405451+0000","last_change":"2024-09-16T15:51:02.400056+0000","last_active":"2024-09-16T15:51:02.405451+0000","last_peered":"2024-09-16T15:51:02.405451+0000","last_clean":"2024-09-16T15:51:02.405451+0000","last_became_active":"2024-09-16T15:51:02.397362+0000","last_became_peered":"2024-09-16T15:51:02.397362+0000","last_unstale":"2024-09-16T15:51:02.405451+0000","last_undegraded":"2024-09-16T15:51:02.405451+0000","last_fullsized":"2024-09-16T15:51:02.405451+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:52:34.894816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":101,"state":"creating+peering","last_fresh":"2024-09-16T15:51:00.676479+0000","last_change":"2024-09-16T15:51:00.676479+0000","last_active":"2024-09-16T15:51:00.350216+0000","last_peered":"2024-09-16T15:51:00.350216+0000","last_clean":"2024-09-16T15:51:00.350216+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:00.676479+0000","last_undegraded":"2024-09-16T15:51:00.676479+0000","last_fullsized":"2024-09-16T15:51:00.676479+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.367293+0000","last_change":"2024-09-16T15:51:01.367293+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.367293+0000","last_undegraded":"2024-09-16T15:51:01.367293+0000","last_fullsized":"2024-09-16T15:51:01.367293+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.388344+0000","last_change":"2024-09-16T15:51:01.386549+0000","last_active":"2024-09-16T15:51:01.388344+0000","last_peered":"2024-09-16T15:51:01.388344+0000","last_clean":"2024-09-16T15:51:01.388344+0000","last_became_active":"2024-09-16T15:51:01.386253+0000","last_became_peered":"2024-09-16T15:51:01.386253+0000","last_unstale":"2024-09-16T15:51:01.388344+0000","last_undegraded":"2024-09-16T15:51:01.388344+0000","last_fullsized":"2024-09-16T15:51:01.388344+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:16:55.241704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.382489+0000","last_change":"2024-09-16T15:51:01.382489+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.382489+0000","last_undegraded":"2024-09-16T15:51:01.382489+0000","last_fullsized":"2024-09-16T15:51:01.382489+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.381276+0000","last_change":"2024-09-16T15:51:01.379987+0000","last_active":"2024-09-16T15:51:01.381276+0000","last_peered":"2024-09-16T15:51:01.381276+0000","last_clean":"2024-09-16T15:51:01.381276+0000","last_became_active":"2024-09-16T15:51:01.379714+0000","last_became_peered":"2024-09-16T15:51:01.379714+0000","last_unstale":"2024-09-16T15:51:01.381276+0000","last_undegraded":"2024-09-16T15:51:01.381276+0000","last_fullsized":"2024-09-16T15:51:01.381276+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:59:46.751381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.379213+0000","last_change":"2024-09-16T15:51:01.379213+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.379213+0000","last_undegraded":"2024-09-16T15:51:01.379213+0000","last_fullsized":"2024-09-16T15:51:01.379213+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.392581+0000","last_change":"2024-09-16T15:51:01.391293+0000","last_active":"2024-09-16T15:51:01.392581+0000","last_peered":"2024-09-16T15:51:01.392581+0000","last_clean":"2024-09-16T15:51:01.392581+0000","last_became_active":"2024-09-16T15:51:01.388187+0000","last_became_peered":"2024-09-16T15:51:01.388187+0000","last_unstale":"2024-09-16T15:51:01.392581+0000","last_undegraded":"2024-09-16T15:51:01.392581+0000","last_fullsized":"2024-09-16T15:51:01.392581+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:41:28.514384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.371328+0000","last_change":"2024-09-16T15:51:01.371328+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.371328+0000","last_undegraded":"2024-09-16T15:51:01.371328+0000","last_fullsized":"2024-09-16T15:51:01.371328+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.387376+0000","last_change":"2024-09-16T15:51:01.381796+0000","last_active":"2024-09-16T15:51:01.387376+0000","last_peered":"2024-09-16T15:51:01.387376+0000","last_clean":"2024-09-16T15:51:01.387376+0000","last_became_active":"2024-09-16T15:51:01.381524+0000","last_became_peered":"2024-09-16T15:51:01.381524+0000","last_unstale":"2024-09-16T15:51:01.387376+0000","last_undegraded":"2024-09-16T15:51:01.387376+0000","last_fullsized":"2024-09-16T15:51:01.387376+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:29:47.623958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.368671+0000","last_change":"2024-09-16T15:51:01.368671+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.368671+0000","last_undegraded":"2024-09-16T15:51:01.368671+0000","last_fullsized":"2024-09-16T15:51:01.368671+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.383212+0000","last_change":"2024-09-16T15:51:01.380670+0000","last_active":"2024-09-16T15:51:01.383212+0000","last_peered":"2024-09-16T15:51:01.383212+0000","last_clean":"2024-09-16T15:51:01.383212+0000","last_became_active":"2024-09-16T15:51:01.380213+0000","last_became_peered":"2024-09-16T15:51:01.380213+0000","last_unstale":"2024-09-16T15:51:01.383212+0000","last_undegraded":"2024-09-16T15:51:01.383212+0000","last_fullsized":"2024-09-16T15:51:01.383212+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:54:34.716483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.379707+0000","last_change":"2024-09-16T15:51:01.379707+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.379707+0000","last_undegraded":"2024-09-16T15:51:01.379707+0000","last_fullsized":"2024-09-16T15:51:01.379707+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.390441+0000","last_change":"2024-09-16T15:51:01.388916+0000","last_active":"2024-09-16T15:51:01.390441+0000","last_peered":"2024-09-16T15:51:01.390441+0000","last_clean":"2024-09-16T15:51:01.390441+0000","last_became_active":"2024-09-16T15:51:01.388644+0000","last_became_peered":"2024-09-16T15:51:01.388644+0000","last_unstale":"2024-09-16T15:51:01.390441+0000","last_undegraded":"2024-09-16T15:51:01.390441+0000","last_fullsized":"2024-09-16T15:51:01.390441+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:38:33.558776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.368076+0000","last_change":"2024-09-16T15:51:01.368076+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.368076+0000","last_undegraded":"2024-09-16T15:51:01.368076+0000","last_fullsized":"2024-09-16T15:51:01.368076+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.388815+0000","last_change":"2024-09-16T15:51:01.387695+0000","last_active":"2024-09-16T15:51:01.388815+0000","last_peered":"2024-09-16T15:51:01.388815+0000","last_clean":"2024-09-16T15:51:01.388815+0000","last_became_active":"2024-09-16T15:51:01.387455+0000","last_became_peered":"2024-09-16T15:51:01.387455+0000","last_unstale":"2024-09-16T15:51:01.388815+0000","last_undegraded":"2024-09-16T15:51:01.388815+0000","last_fullsized":"2024-09-16T15:51:01.388815+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:23:23.665770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.378487+0000","last_change":"2024-09-16T15:51:01.378487+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.378487+0000","last_undegraded":"2024-09-16T15:51:01.378487+0000","last_fullsized":"2024-09-16T15:51:01.378487+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.308686+0000","last_change":"2024-09-16T15:51:01.384206+0000","last_active":"2024-09-16T15:51:04.308686+0000","last_peered":"2024-09-16T15:51:04.308686+0000","last_clean":"2024-09-16T15:51:04.308686+0000","last_became_active":"2024-09-16T15:51:01.383873+0000","last_became_peered":"2024-09-16T15:51:01.383873+0000","last_unstale":"2024-09-16T15:51:04.308686+0000","last_undegraded":"2024-09-16T15:51:04.308686+0000","last_fullsized":"2024-09-16T15:51:04.308686+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:15:03.468370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.367780+0000","last_change":"2024-09-16T15:51:01.367780+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.367780+0000","last_undegraded":"2024-09-16T15:51:01.367780+0000","last_fullsized":"2024-09-16T15:51:01.367780+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.392779+0000","last_change":"2024-09-16T15:51:01.390952+0000","last_active":"2024-09-16T15:51:01.392779+0000","last_peered":"2024-09-16T15:51:01.392779+0000","last_clean":"2024-09-16T15:51:01.392779+0000","last_became_active":"2024-09-16T15:51:01.390640+0000","last_became_peered":"2024-09-16T15:51:01.390640+0000","last_unstale":"2024-09-16T15:51:01.392779+0000","last_undegraded":"2024-09-16T15:51:01.392779+0000","last_fullsized":"2024-09-16T15:51:01.392779+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:04:41.524063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":1,"reported_epoch":102,"state":"creating+peering","last_fresh":"2024-09-16T15:51:01.370302+0000","last_change":"2024-09-16T15:51:01.370302+0000","last_active":"2024-09-16T15:51:01.355970+0000","last_peered":"2024-09-16T15:51:01.355970+0000","last_clean":"2024-09-16T15:51:01.355970+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:51:01.370302+0000","last_undegraded":"2024-09-16T15:51:01.370302+0000","last_fullsized":"2024-09-16T15:51:01.370302+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":10,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.388861+0000","last_change":"2024-09-16T15:51:01.388328+0000","last_active":"2024-09-16T15:51:01.388861+0000","last_peered":"2024-09-16T15:51:01.388861+0000","last_clean":"2024-09-16T15:51:01.388861+0000","last_became_active":"2024-09-16T15:51:01.388066+0000","last_became_peered":"2024-09-16T15:51:01.388066+0000","last_unstale":"2024-09-16T15:51:01.388861+0000","last_undegraded":"2024-09-16T15:51:01.388861+0000","last_fullsized":"2024-09-16T15:51:01.388861+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:22:46.801645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":262,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-16T15:51:01.367319+0000","last_change":"2024-09-16T15:39:12.698813+0000","last_active":"2024-09-16T15:51:01.367319+0000","last_peered":"2024-09-16T15:51:01.367319+0000","last_clean":"2024-09-16T15:51:01.367319+0000","last_became_active":"2024-09-16T15:39:12.697695+0000","last_became_peered":"2024-09-16T15:39:12.697695+0000","last_unstale":"2024-09-16T15:51:01.367319+0000","last_undegraded":"2024-09-16T15:51:01.367319+0000","last_fullsized":"2024-09-16T15:51:01.367319+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_clean_scrub_stamp":"2024-09-16T15:39:11.684967+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:02.061081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":122,"acting":122,"num_store_stats":7},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":582,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509587,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":148960,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222880,"statfs":{"total":96636764160,"available":96484229120,"internally_reserved":0,"allocated":1253376,"data_stored":544601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":151255496},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509586,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131488,"kb_used_data":1804,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240352,"statfs":{"total":96636764160,"available":96502120448,"internally_reserved":0,"allocated":1847296,"data_stored":1134969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509585,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144800,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94227040,"statfs":{"total":96636764160,"available":96488488960,"internally_reserved":0,"allocated":1241088,"data_stored":538270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509585,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126752,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94245088,"statfs":{"total":96636764160,"available":96506970112,"internally_reserved":0,"allocated":1253376,"data_stored":544601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":128514505},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509585,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126740,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94245100,"statfs":{"total":96636764160,"available":96506982400,"internally_reserved":0,"allocated":1241088,"data_stored":538270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":128514503},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509586,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123160,"kb_used_data":1792,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248680,"statfs":{"total":96636764160,"available":96510648320,"internally_reserved":0,"allocated":1835008,"data_stored":1128638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":124254663},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509584,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":140612,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94231228,"statfs":{"total":96636764160,"available":96492777472,"internally_reserved":0,"allocated":1228800,"data_stored":532397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":142735815},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542292,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122620,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94249220,"statfs":{"total":96636764160,"available":96511201280,"internally_reserved":0,"allocated":1282048,"data_stored":558370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":124254668},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:51:07.443 INFO:tasks.ceph.ceph_manager.ceph:PG 15.3d is not active+clean 2024-09-16T15:51:07.443 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 102, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:51:01.369847+0000', 'last_change': '2024-09-16T15:51:01.369847+0000', 'last_active': '2024-09-16T15:51:01.355970+0000', 'last_peered': '2024-09-16T15:51:01.355970+0000', 'last_clean': '2024-09-16T15:51:01.355970+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:51:01.369847+0000', 'last_undegraded': '2024-09-16T15:51:01.369847+0000', 'last_fullsized': '2024-09-16T15:51:01.369847+0000', 'mapping_epoch': 102, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 102, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_clean_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-16T15:51:07.444 INFO:tasks.ceph.ceph_manager.ceph:PG 15.3b is not active+clean 2024-09-16T15:51:07.444 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 102, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:51:01.374870+0000', 'last_change': '2024-09-16T15:51:01.374870+0000', 'last_active': '2024-09-16T15:51:01.355970+0000', 'last_peered': '2024-09-16T15:51:01.355970+0000', 'last_clean': '2024-09-16T15:51:01.355970+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:51:01.374870+0000', 'last_undegraded': '2024-09-16T15:51:01.374870+0000', 'last_fullsized': '2024-09-16T15:51:01.374870+0000', 'mapping_epoch': 102, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 102, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_clean_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T15:51:07.444 INFO:tasks.ceph.ceph_manager.ceph:PG 15.3a is not active+clean 2024-09-16T15:51:07.444 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 102, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:51:01.374553+0000', 'last_change': '2024-09-16T15:51:01.374553+0000', 'last_active': '2024-09-16T15:51:01.355970+0000', 'last_peered': '2024-09-16T15:51:01.355970+0000', 'last_clean': '2024-09-16T15:51:01.355970+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:51:01.374553+0000', 'last_undegraded': '2024-09-16T15:51:01.374553+0000', 'last_fullsized': '2024-09-16T15:51:01.374553+0000', 'mapping_epoch': 102, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 102, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_clean_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T15:51:07.444 INFO:tasks.ceph.ceph_manager.ceph:PG 15.39 is not active+clean 2024-09-16T15:51:07.444 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 102, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:51:01.372119+0000', 'last_change': '2024-09-16T15:51:01.372119+0000', 'last_active': '2024-09-16T15:51:01.355970+0000', 'last_peered': '2024-09-16T15:51:01.355970+0000', 'last_clean': '2024-09-16T15:51:01.355970+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:51:01.372119+0000', 'last_undegraded': '2024-09-16T15:51:01.372119+0000', 'last_fullsized': '2024-09-16T15:51:01.372119+0000', 'mapping_epoch': 102, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 102, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_clean_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T15:51:07.444 INFO:tasks.ceph.ceph_manager.ceph:PG 15.38 is not active+clean 2024-09-16T15:51:07.444 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 102, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:51:01.370748+0000', 'last_change': '2024-09-16T15:51:01.370748+0000', 'last_active': '2024-09-16T15:51:01.355970+0000', 'last_peered': '2024-09-16T15:51:01.355970+0000', 'last_clean': '2024-09-16T15:51:01.355970+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:51:01.370748+0000', 'last_undegraded': '2024-09-16T15:51:01.370748+0000', 'last_fullsized': '2024-09-16T15:51:01.370748+0000', 'mapping_epoch': 102, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 102, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_clean_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-16T15:51:07.444 INFO:tasks.ceph.ceph_manager.ceph:PG 15.37 is not active+clean 2024-09-16T15:51:07.444 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 102, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:51:01.373077+0000', 'last_change': '2024-09-16T15:51:01.373077+0000', 'last_active': '2024-09-16T15:51:01.355970+0000', 'last_peered': '2024-09-16T15:51:01.355970+0000', 'last_clean': '2024-09-16T15:51:01.355970+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:51:01.373077+0000', 'last_undegraded': '2024-09-16T15:51:01.373077+0000', 'last_fullsized': '2024-09-16T15:51:01.373077+0000', 'mapping_epoch': 102, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 102, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_clean_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T15:51:07.444 INFO:tasks.ceph.ceph_manager.ceph:PG 15.36 is not active+clean 2024-09-16T15:51:07.445 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:51:01.355970+0000', 'last_change': '2024-09-16T15:51:01.355970+0000', 'last_active': '2024-09-16T15:51:01.355970+0000', 'last_peered': '2024-09-16T15:51:01.355970+0000', 'last_clean': '2024-09-16T15:51:01.355970+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:51:01.355970+0000', 'last_undegraded': '2024-09-16T15:51:01.355970+0000', 'last_fullsized': '2024-09-16T15:51:01.355970+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_clean_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:51:07.445 INFO:tasks.ceph.ceph_manager.ceph:PG 15.35 is not active+clean 2024-09-16T15:51:07.445 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 102, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:51:01.374982+0000', 'last_change': '2024-09-16T15:51:01.374982+0000', 'last_active': '2024-09-16T15:51:01.355970+0000', 'last_peered': '2024-09-16T15:51:01.355970+0000', 'last_clean': '2024-09-16T15:51:01.355970+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:51:01.374982+0000', 'last_undegraded': '2024-09-16T15:51:01.374982+0000', 'last_fullsized': '2024-09-16T15:51:01.374982+0000', 'mapping_epoch': 102, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 102, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_clean_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T15:51:07.445 INFO:tasks.ceph.ceph_manager.ceph:PG 15.34 is not active+clean 2024-09-16T15:51:07.445 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 102, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:51:01.373278+0000', 'last_change': '2024-09-16T15:51:01.373278+0000', 'last_active': '2024-09-16T15:51:01.355970+0000', 'last_peered': '2024-09-16T15:51:01.355970+0000', 'last_clean': '2024-09-16T15:51:01.355970+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:51:01.373278+0000', 'last_undegraded': '2024-09-16T15:51:01.373278+0000', 'last_fullsized': '2024-09-16T15:51:01.373278+0000', 'mapping_epoch': 102, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 102, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_clean_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-16T15:51:07.445 INFO:tasks.ceph.ceph_manager.ceph:PG 15.30 is not active+clean 2024-09-16T15:51:07.445 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 102, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:51:01.371468+0000', 'last_change': '2024-09-16T15:51:01.371468+0000', 'last_active': '2024-09-16T15:51:01.355970+0000', 'last_peered': '2024-09-16T15:51:01.355970+0000', 'last_clean': '2024-09-16T15:51:01.355970+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:51:01.371468+0000', 'last_undegraded': '2024-09-16T15:51:01.371468+0000', 'last_fullsized': '2024-09-16T15:51:01.371468+0000', 'mapping_epoch': 102, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 102, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_clean_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T15:51:07.445 INFO:tasks.ceph.ceph_manager.ceph:PG 15.2f is not active+clean 2024-09-16T15:51:07.445 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 102, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:51:01.379608+0000', 'last_change': '2024-09-16T15:51:01.379608+0000', 'last_active': '2024-09-16T15:51:01.355970+0000', 'last_peered': '2024-09-16T15:51:01.355970+0000', 'last_clean': '2024-09-16T15:51:01.355970+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:51:01.379608+0000', 'last_undegraded': '2024-09-16T15:51:01.379608+0000', 'last_fullsized': '2024-09-16T15:51:01.379608+0000', 'mapping_epoch': 102, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 102, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_clean_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-16T15:51:07.445 INFO:tasks.ceph.ceph_manager.ceph:PG 15.2d is not active+clean 2024-09-16T15:51:07.445 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 102, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:51:01.372608+0000', 'last_change': '2024-09-16T15:51:01.372608+0000', 'last_active': '2024-09-16T15:51:01.355970+0000', 'last_peered': '2024-09-16T15:51:01.355970+0000', 'last_clean': '2024-09-16T15:51:01.355970+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:51:01.372608+0000', 'last_undegraded': '2024-09-16T15:51:01.372608+0000', 'last_fullsized': '2024-09-16T15:51:01.372608+0000', 'mapping_epoch': 102, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 102, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_clean_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-16T15:51:07.446 INFO:tasks.ceph.ceph_manager.ceph:PG 15.2c is not active+clean 2024-09-16T15:51:07.446 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 102, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:51:01.370913+0000', 'last_change': '2024-09-16T15:51:01.370913+0000', 'last_active': '2024-09-16T15:51:01.355970+0000', 'last_peered': '2024-09-16T15:51:01.355970+0000', 'last_clean': '2024-09-16T15:51:01.355970+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:51:01.370913+0000', 'last_undegraded': '2024-09-16T15:51:01.370913+0000', 'last_fullsized': '2024-09-16T15:51:01.370913+0000', 'mapping_epoch': 102, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 102, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_clean_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-16T15:51:07.446 INFO:tasks.ceph.ceph_manager.ceph:PG 15.2b is not active+clean 2024-09-16T15:51:07.446 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 102, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:51:01.371225+0000', 'last_change': '2024-09-16T15:51:01.371225+0000', 'last_active': '2024-09-16T15:51:01.355970+0000', 'last_peered': '2024-09-16T15:51:01.355970+0000', 'last_clean': '2024-09-16T15:51:01.355970+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:51:01.371225+0000', 'last_undegraded': '2024-09-16T15:51:01.371225+0000', 'last_fullsized': '2024-09-16T15:51:01.371225+0000', 'mapping_epoch': 102, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 102, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_clean_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-16T15:51:07.446 INFO:tasks.ceph.ceph_manager.ceph:PG 15.2a is not active+clean 2024-09-16T15:51:07.446 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 102, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:51:01.368817+0000', 'last_change': '2024-09-16T15:51:01.368817+0000', 'last_active': '2024-09-16T15:51:01.355970+0000', 'last_peered': '2024-09-16T15:51:01.355970+0000', 'last_clean': '2024-09-16T15:51:01.355970+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:51:01.368817+0000', 'last_undegraded': '2024-09-16T15:51:01.368817+0000', 'last_fullsized': '2024-09-16T15:51:01.368817+0000', 'mapping_epoch': 102, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 102, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_clean_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-16T15:51:07.446 INFO:tasks.ceph.ceph_manager.ceph:PG 15.28 is not active+clean 2024-09-16T15:51:07.446 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 102, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:51:01.371334+0000', 'last_change': '2024-09-16T15:51:01.371334+0000', 'last_active': '2024-09-16T15:51:01.355970+0000', 'last_peered': '2024-09-16T15:51:01.355970+0000', 'last_clean': '2024-09-16T15:51:01.355970+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:51:01.371334+0000', 'last_undegraded': '2024-09-16T15:51:01.371334+0000', 'last_fullsized': '2024-09-16T15:51:01.371334+0000', 'mapping_epoch': 102, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 102, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_clean_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T15:51:07.446 INFO:tasks.ceph.ceph_manager.ceph:PG 15.27 is not active+clean 2024-09-16T15:51:07.446 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:51:01.355970+0000', 'last_change': '2024-09-16T15:51:01.355970+0000', 'last_active': '2024-09-16T15:51:01.355970+0000', 'last_peered': '2024-09-16T15:51:01.355970+0000', 'last_clean': '2024-09-16T15:51:01.355970+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:51:01.355970+0000', 'last_undegraded': '2024-09-16T15:51:01.355970+0000', 'last_fullsized': '2024-09-16T15:51:01.355970+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_clean_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:51:07.446 INFO:tasks.ceph.ceph_manager.ceph:PG 15.25 is not active+clean 2024-09-16T15:51:07.446 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 102, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:51:01.371083+0000', 'last_change': '2024-09-16T15:51:01.371083+0000', 'last_active': '2024-09-16T15:51:01.355970+0000', 'last_peered': '2024-09-16T15:51:01.355970+0000', 'last_clean': '2024-09-16T15:51:01.355970+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:51:01.371083+0000', 'last_undegraded': '2024-09-16T15:51:01.371083+0000', 'last_fullsized': '2024-09-16T15:51:01.371083+0000', 'mapping_epoch': 102, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 102, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_clean_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T15:51:07.447 INFO:tasks.ceph.ceph_manager.ceph:PG 15.24 is not active+clean 2024-09-16T15:51:07.447 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 102, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:51:01.368057+0000', 'last_change': '2024-09-16T15:51:01.368057+0000', 'last_active': '2024-09-16T15:51:01.355970+0000', 'last_peered': '2024-09-16T15:51:01.355970+0000', 'last_clean': '2024-09-16T15:51:01.355970+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:51:01.368057+0000', 'last_undegraded': '2024-09-16T15:51:01.368057+0000', 'last_fullsized': '2024-09-16T15:51:01.368057+0000', 'mapping_epoch': 102, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 102, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_clean_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T15:51:07.447 INFO:tasks.ceph.ceph_manager.ceph:PG 15.21 is not active+clean 2024-09-16T15:51:07.447 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 102, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:51:01.368886+0000', 'last_change': '2024-09-16T15:51:01.368886+0000', 'last_active': '2024-09-16T15:51:01.355970+0000', 'last_peered': '2024-09-16T15:51:01.355970+0000', 'last_clean': '2024-09-16T15:51:01.355970+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:51:01.368886+0000', 'last_undegraded': '2024-09-16T15:51:01.368886+0000', 'last_fullsized': '2024-09-16T15:51:01.368886+0000', 'mapping_epoch': 102, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 102, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_clean_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-16T15:51:07.447 INFO:tasks.ceph.ceph_manager.ceph:PG 15.1e is not active+clean 2024-09-16T15:51:07.447 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 102, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:51:01.382918+0000', 'last_change': '2024-09-16T15:51:01.382918+0000', 'last_active': '2024-09-16T15:51:01.355970+0000', 'last_peered': '2024-09-16T15:51:01.355970+0000', 'last_clean': '2024-09-16T15:51:01.355970+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:51:01.382918+0000', 'last_undegraded': '2024-09-16T15:51:01.382918+0000', 'last_fullsized': '2024-09-16T15:51:01.382918+0000', 'mapping_epoch': 102, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 102, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_clean_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-16T15:51:07.447 INFO:tasks.ceph.ceph_manager.ceph:PG 15.1d is not active+clean 2024-09-16T15:51:07.447 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-16T15:51:01.355970+0000', 'last_change': '2024-09-16T15:51:01.355970+0000', 'last_active': '2024-09-16T15:51:01.355970+0000', 'last_peered': '2024-09-16T15:51:01.355970+0000', 'last_clean': '2024-09-16T15:51:01.355970+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:51:01.355970+0000', 'last_undegraded': '2024-09-16T15:51:01.355970+0000', 'last_fullsized': '2024-09-16T15:51:01.355970+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_clean_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-16T15:51:07.447 INFO:tasks.ceph.ceph_manager.ceph:PG 15.1a is not active+clean 2024-09-16T15:51:07.447 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 102, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:51:01.370455+0000', 'last_change': '2024-09-16T15:51:01.370455+0000', 'last_active': '2024-09-16T15:51:01.355970+0000', 'last_peered': '2024-09-16T15:51:01.355970+0000', 'last_clean': '2024-09-16T15:51:01.355970+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:51:01.370455+0000', 'last_undegraded': '2024-09-16T15:51:01.370455+0000', 'last_fullsized': '2024-09-16T15:51:01.370455+0000', 'mapping_epoch': 102, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 102, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_clean_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-16T15:51:07.447 INFO:tasks.ceph.ceph_manager.ceph:PG 15.1b is not active+clean 2024-09-16T15:51:07.447 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 102, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:51:01.370317+0000', 'last_change': '2024-09-16T15:51:01.370317+0000', 'last_active': '2024-09-16T15:51:01.355970+0000', 'last_peered': '2024-09-16T15:51:01.355970+0000', 'last_clean': '2024-09-16T15:51:01.355970+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:51:01.370317+0000', 'last_undegraded': '2024-09-16T15:51:01.370317+0000', 'last_fullsized': '2024-09-16T15:51:01.370317+0000', 'mapping_epoch': 102, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 102, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_clean_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T15:51:07.448 INFO:tasks.ceph.ceph_manager.ceph:PG 15.19 is not active+clean 2024-09-16T15:51:07.448 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 102, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:51:01.369717+0000', 'last_change': '2024-09-16T15:51:01.369717+0000', 'last_active': '2024-09-16T15:51:01.355970+0000', 'last_peered': '2024-09-16T15:51:01.355970+0000', 'last_clean': '2024-09-16T15:51:01.355970+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:51:01.369717+0000', 'last_undegraded': '2024-09-16T15:51:01.369717+0000', 'last_fullsized': '2024-09-16T15:51:01.369717+0000', 'mapping_epoch': 102, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 102, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_clean_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-16T15:51:07.448 INFO:tasks.ceph.ceph_manager.ceph:PG 15.17 is not active+clean 2024-09-16T15:51:07.448 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 102, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:51:01.368015+0000', 'last_change': '2024-09-16T15:51:01.368015+0000', 'last_active': '2024-09-16T15:51:01.355970+0000', 'last_peered': '2024-09-16T15:51:01.355970+0000', 'last_clean': '2024-09-16T15:51:01.355970+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:51:01.368015+0000', 'last_undegraded': '2024-09-16T15:51:01.368015+0000', 'last_fullsized': '2024-09-16T15:51:01.368015+0000', 'mapping_epoch': 102, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 102, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_clean_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T15:51:07.448 INFO:tasks.ceph.ceph_manager.ceph:PG 15.14 is not active+clean 2024-09-16T15:51:07.448 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 102, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:51:01.369935+0000', 'last_change': '2024-09-16T15:51:01.369935+0000', 'last_active': '2024-09-16T15:51:01.355970+0000', 'last_peered': '2024-09-16T15:51:01.355970+0000', 'last_clean': '2024-09-16T15:51:01.355970+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:51:01.369935+0000', 'last_undegraded': '2024-09-16T15:51:01.369935+0000', 'last_fullsized': '2024-09-16T15:51:01.369935+0000', 'mapping_epoch': 102, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 102, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_clean_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T15:51:07.448 INFO:tasks.ceph.ceph_manager.ceph:PG 15.15 is not active+clean 2024-09-16T15:51:07.448 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 102, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:51:01.367826+0000', 'last_change': '2024-09-16T15:51:01.367826+0000', 'last_active': '2024-09-16T15:51:01.355970+0000', 'last_peered': '2024-09-16T15:51:01.355970+0000', 'last_clean': '2024-09-16T15:51:01.355970+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:51:01.367826+0000', 'last_undegraded': '2024-09-16T15:51:01.367826+0000', 'last_fullsized': '2024-09-16T15:51:01.367826+0000', 'mapping_epoch': 102, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 102, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_clean_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T15:51:07.448 INFO:tasks.ceph.ceph_manager.ceph:PG 15.13 is not active+clean 2024-09-16T15:51:07.448 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 102, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:51:01.368875+0000', 'last_change': '2024-09-16T15:51:01.368875+0000', 'last_active': '2024-09-16T15:51:01.355970+0000', 'last_peered': '2024-09-16T15:51:01.355970+0000', 'last_clean': '2024-09-16T15:51:01.355970+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:51:01.368875+0000', 'last_undegraded': '2024-09-16T15:51:01.368875+0000', 'last_fullsized': '2024-09-16T15:51:01.368875+0000', 'mapping_epoch': 102, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 102, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_clean_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-16T15:51:07.448 INFO:tasks.ceph.ceph_manager.ceph:PG 15.10 is not active+clean 2024-09-16T15:51:07.448 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 102, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:51:01.368901+0000', 'last_change': '2024-09-16T15:51:01.368901+0000', 'last_active': '2024-09-16T15:51:01.355970+0000', 'last_peered': '2024-09-16T15:51:01.355970+0000', 'last_clean': '2024-09-16T15:51:01.355970+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:51:01.368901+0000', 'last_undegraded': '2024-09-16T15:51:01.368901+0000', 'last_fullsized': '2024-09-16T15:51:01.368901+0000', 'mapping_epoch': 102, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 102, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_clean_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T15:51:07.449 INFO:tasks.ceph.ceph_manager.ceph:PG 15.11 is not active+clean 2024-09-16T15:51:07.449 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 102, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:51:01.367556+0000', 'last_change': '2024-09-16T15:51:01.367556+0000', 'last_active': '2024-09-16T15:51:01.355970+0000', 'last_peered': '2024-09-16T15:51:01.355970+0000', 'last_clean': '2024-09-16T15:51:01.355970+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:51:01.367556+0000', 'last_undegraded': '2024-09-16T15:51:01.367556+0000', 'last_fullsized': '2024-09-16T15:51:01.367556+0000', 'mapping_epoch': 102, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 102, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_clean_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-16T15:51:07.449 INFO:tasks.ceph.ceph_manager.ceph:PG 15.f is not active+clean 2024-09-16T15:51:07.449 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 102, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:51:01.367595+0000', 'last_change': '2024-09-16T15:51:01.367595+0000', 'last_active': '2024-09-16T15:51:01.355970+0000', 'last_peered': '2024-09-16T15:51:01.355970+0000', 'last_clean': '2024-09-16T15:51:01.355970+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:51:01.367595+0000', 'last_undegraded': '2024-09-16T15:51:01.367595+0000', 'last_fullsized': '2024-09-16T15:51:01.367595+0000', 'mapping_epoch': 102, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 102, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_clean_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T15:51:07.449 INFO:tasks.ceph.ceph_manager.ceph:PG 15.c is not active+clean 2024-09-16T15:51:07.449 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 102, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:51:01.367237+0000', 'last_change': '2024-09-16T15:51:01.367237+0000', 'last_active': '2024-09-16T15:51:01.355970+0000', 'last_peered': '2024-09-16T15:51:01.355970+0000', 'last_clean': '2024-09-16T15:51:01.355970+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:51:01.367237+0000', 'last_undegraded': '2024-09-16T15:51:01.367237+0000', 'last_fullsized': '2024-09-16T15:51:01.367237+0000', 'mapping_epoch': 102, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 102, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_clean_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T15:51:07.449 INFO:tasks.ceph.ceph_manager.ceph:PG 15.d is not active+clean 2024-09-16T15:51:07.449 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 102, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:51:01.368491+0000', 'last_change': '2024-09-16T15:51:01.368491+0000', 'last_active': '2024-09-16T15:51:01.355970+0000', 'last_peered': '2024-09-16T15:51:01.355970+0000', 'last_clean': '2024-09-16T15:51:01.355970+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:51:01.368491+0000', 'last_undegraded': '2024-09-16T15:51:01.368491+0000', 'last_fullsized': '2024-09-16T15:51:01.368491+0000', 'mapping_epoch': 102, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 102, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_clean_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-16T15:51:07.449 INFO:tasks.ceph.ceph_manager.ceph:PG 15.a is not active+clean 2024-09-16T15:51:07.449 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 102, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:51:01.380115+0000', 'last_change': '2024-09-16T15:51:01.380115+0000', 'last_active': '2024-09-16T15:51:01.355970+0000', 'last_peered': '2024-09-16T15:51:01.355970+0000', 'last_clean': '2024-09-16T15:51:01.355970+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:51:01.380115+0000', 'last_undegraded': '2024-09-16T15:51:01.380115+0000', 'last_fullsized': '2024-09-16T15:51:01.380115+0000', 'mapping_epoch': 102, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 102, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_clean_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-16T15:51:07.449 INFO:tasks.ceph.ceph_manager.ceph:PG 15.b is not active+clean 2024-09-16T15:51:07.449 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 102, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:51:01.376646+0000', 'last_change': '2024-09-16T15:51:01.376646+0000', 'last_active': '2024-09-16T15:51:01.355970+0000', 'last_peered': '2024-09-16T15:51:01.355970+0000', 'last_clean': '2024-09-16T15:51:01.355970+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:51:01.376646+0000', 'last_undegraded': '2024-09-16T15:51:01.376646+0000', 'last_fullsized': '2024-09-16T15:51:01.376646+0000', 'mapping_epoch': 102, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 102, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_clean_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-16T15:51:07.450 INFO:tasks.ceph.ceph_manager.ceph:PG 14.9 is not active+clean 2024-09-16T15:51:07.450 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '14.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 101, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:51:00.676479+0000', 'last_change': '2024-09-16T15:51:00.676479+0000', 'last_active': '2024-09-16T15:51:00.350216+0000', 'last_peered': '2024-09-16T15:51:00.350216+0000', 'last_clean': '2024-09-16T15:51:00.350216+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:51:00.676479+0000', 'last_undegraded': '2024-09-16T15:51:00.676479+0000', 'last_fullsized': '2024-09-16T15:51:00.676479+0000', 'mapping_epoch': 101, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 101, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:51:00.350216+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:51:00.350216+0000', 'last_clean_scrub_stamp': '2024-09-16T15:51:00.350216+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T15:51:07.450 INFO:tasks.ceph.ceph_manager.ceph:PG 15.9 is not active+clean 2024-09-16T15:51:07.450 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 102, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:51:01.367293+0000', 'last_change': '2024-09-16T15:51:01.367293+0000', 'last_active': '2024-09-16T15:51:01.355970+0000', 'last_peered': '2024-09-16T15:51:01.355970+0000', 'last_clean': '2024-09-16T15:51:01.355970+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:51:01.367293+0000', 'last_undegraded': '2024-09-16T15:51:01.367293+0000', 'last_fullsized': '2024-09-16T15:51:01.367293+0000', 'mapping_epoch': 102, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 102, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_clean_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T15:51:07.450 INFO:tasks.ceph.ceph_manager.ceph:PG 15.6 is not active+clean 2024-09-16T15:51:07.450 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 102, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:51:01.382489+0000', 'last_change': '2024-09-16T15:51:01.382489+0000', 'last_active': '2024-09-16T15:51:01.355970+0000', 'last_peered': '2024-09-16T15:51:01.355970+0000', 'last_clean': '2024-09-16T15:51:01.355970+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:51:01.382489+0000', 'last_undegraded': '2024-09-16T15:51:01.382489+0000', 'last_fullsized': '2024-09-16T15:51:01.382489+0000', 'mapping_epoch': 102, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 102, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_clean_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-16T15:51:07.450 INFO:tasks.ceph.ceph_manager.ceph:PG 15.7 is not active+clean 2024-09-16T15:51:07.450 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 102, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:51:01.379213+0000', 'last_change': '2024-09-16T15:51:01.379213+0000', 'last_active': '2024-09-16T15:51:01.355970+0000', 'last_peered': '2024-09-16T15:51:01.355970+0000', 'last_clean': '2024-09-16T15:51:01.355970+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:51:01.379213+0000', 'last_undegraded': '2024-09-16T15:51:01.379213+0000', 'last_fullsized': '2024-09-16T15:51:01.379213+0000', 'mapping_epoch': 102, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 102, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_clean_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-16T15:51:07.450 INFO:tasks.ceph.ceph_manager.ceph:PG 15.4 is not active+clean 2024-09-16T15:51:07.450 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 102, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:51:01.371328+0000', 'last_change': '2024-09-16T15:51:01.371328+0000', 'last_active': '2024-09-16T15:51:01.355970+0000', 'last_peered': '2024-09-16T15:51:01.355970+0000', 'last_clean': '2024-09-16T15:51:01.355970+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:51:01.371328+0000', 'last_undegraded': '2024-09-16T15:51:01.371328+0000', 'last_fullsized': '2024-09-16T15:51:01.371328+0000', 'mapping_epoch': 102, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 102, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_clean_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-16T15:51:07.450 INFO:tasks.ceph.ceph_manager.ceph:PG 15.5 is not active+clean 2024-09-16T15:51:07.450 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 102, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:51:01.368671+0000', 'last_change': '2024-09-16T15:51:01.368671+0000', 'last_active': '2024-09-16T15:51:01.355970+0000', 'last_peered': '2024-09-16T15:51:01.355970+0000', 'last_clean': '2024-09-16T15:51:01.355970+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:51:01.368671+0000', 'last_undegraded': '2024-09-16T15:51:01.368671+0000', 'last_fullsized': '2024-09-16T15:51:01.368671+0000', 'mapping_epoch': 102, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 102, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_clean_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T15:51:07.450 INFO:tasks.ceph.ceph_manager.ceph:PG 15.2 is not active+clean 2024-09-16T15:51:07.451 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 102, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:51:01.379707+0000', 'last_change': '2024-09-16T15:51:01.379707+0000', 'last_active': '2024-09-16T15:51:01.355970+0000', 'last_peered': '2024-09-16T15:51:01.355970+0000', 'last_clean': '2024-09-16T15:51:01.355970+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:51:01.379707+0000', 'last_undegraded': '2024-09-16T15:51:01.379707+0000', 'last_fullsized': '2024-09-16T15:51:01.379707+0000', 'mapping_epoch': 102, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 102, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_clean_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-16T15:51:07.451 INFO:tasks.ceph.ceph_manager.ceph:PG 15.3 is not active+clean 2024-09-16T15:51:07.451 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 102, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:51:01.368076+0000', 'last_change': '2024-09-16T15:51:01.368076+0000', 'last_active': '2024-09-16T15:51:01.355970+0000', 'last_peered': '2024-09-16T15:51:01.355970+0000', 'last_clean': '2024-09-16T15:51:01.355970+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:51:01.368076+0000', 'last_undegraded': '2024-09-16T15:51:01.368076+0000', 'last_fullsized': '2024-09-16T15:51:01.368076+0000', 'mapping_epoch': 102, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 102, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_clean_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T15:51:07.451 INFO:tasks.ceph.ceph_manager.ceph:PG 15.0 is not active+clean 2024-09-16T15:51:07.451 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 102, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:51:01.378487+0000', 'last_change': '2024-09-16T15:51:01.378487+0000', 'last_active': '2024-09-16T15:51:01.355970+0000', 'last_peered': '2024-09-16T15:51:01.355970+0000', 'last_clean': '2024-09-16T15:51:01.355970+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:51:01.378487+0000', 'last_undegraded': '2024-09-16T15:51:01.378487+0000', 'last_fullsized': '2024-09-16T15:51:01.378487+0000', 'mapping_epoch': 102, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 102, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_clean_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-16T15:51:07.451 INFO:tasks.ceph.ceph_manager.ceph:PG 15.1 is not active+clean 2024-09-16T15:51:07.451 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 102, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:51:01.367780+0000', 'last_change': '2024-09-16T15:51:01.367780+0000', 'last_active': '2024-09-16T15:51:01.355970+0000', 'last_peered': '2024-09-16T15:51:01.355970+0000', 'last_clean': '2024-09-16T15:51:01.355970+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:51:01.367780+0000', 'last_undegraded': '2024-09-16T15:51:01.367780+0000', 'last_fullsized': '2024-09-16T15:51:01.367780+0000', 'mapping_epoch': 102, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 102, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_clean_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-16T15:51:07.451 INFO:tasks.ceph.ceph_manager.ceph:PG 15.e is not active+clean 2024-09-16T15:51:07.451 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '15.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 102, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:51:01.370302+0000', 'last_change': '2024-09-16T15:51:01.370302+0000', 'last_active': '2024-09-16T15:51:01.355970+0000', 'last_peered': '2024-09-16T15:51:01.355970+0000', 'last_clean': '2024-09-16T15:51:01.355970+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:51:01.370302+0000', 'last_undegraded': '2024-09-16T15:51:01.370302+0000', 'last_fullsized': '2024-09-16T15:51:01.370302+0000', 'mapping_epoch': 102, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 102, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'last_clean_scrub_stamp': '2024-09-16T15:51:01.355970+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-16T15:51:07.452 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T15:51:07.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.634+0000 7fd887600640 1 -- 172.21.15.151:0/2353175731 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd88015cff0 msgr2=0x7fd88006e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:07.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.634+0000 7fd887600640 1 --2- 172.21.15.151:0/2353175731 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd88015cff0 0x7fd88006e6f0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fd8800b2680 tx=0x7fd86c02a560 comp rx=0 tx=0).stop 2024-09-16T15:51:07.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.634+0000 7fd887600640 1 -- 172.21.15.151:0/2353175731 shutdown_connections 2024-09-16T15:51:07.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.634+0000 7fd887600640 1 --2- 172.21.15.151:0/2353175731 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd88006f220 0x7fd88006f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:07.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.634+0000 7fd887600640 1 --2- 172.21.15.151:0/2353175731 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd88015cff0 0x7fd88006e6f0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:07.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.634+0000 7fd887600640 1 --2- 172.21.15.151:0/2353175731 >> v2:172.21.15.167:3300/0 conn(0x7fd880159aa0 0x7fd8800741a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:07.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.634+0000 7fd887600640 1 -- 172.21.15.151:0/2353175731 >> 172.21.15.151:0/2353175731 conn(0x7fd8800b18e0 msgr2=0x7fd8800b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:07.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.634+0000 7fd887600640 1 -- 172.21.15.151:0/2353175731 shutdown_connections 2024-09-16T15:51:07.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.634+0000 7fd887600640 1 -- 172.21.15.151:0/2353175731 wait complete. 2024-09-16T15:51:07.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.636+0000 7fd887600640 1 Processor -- start 2024-09-16T15:51:07.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.636+0000 7fd887600640 1 -- start start 2024-09-16T15:51:07.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.637+0000 7fd887600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd88006f220 0x7fd8800cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:07.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.637+0000 7fd887600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd880159aa0 0x7fd8800b0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:07.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.637+0000 7fd887600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd88015cff0 0x7fd8800b0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:07.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.637+0000 7fd887600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd8800cc310 con 0x7fd880159aa0 2024-09-16T15:51:07.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.637+0000 7fd887600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd8800cc190 con 0x7fd88015cff0 2024-09-16T15:51:07.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.637+0000 7fd887600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fd8800cc490 con 0x7fd88006f220 2024-09-16T15:51:07.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.637+0000 7fd87fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd880159aa0 0x7fd8800b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:07.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.637+0000 7fd87fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd880159aa0 0x7fd8800b0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52688/0 (socket says 172.21.15.151:52688) 2024-09-16T15:51:07.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.637+0000 7fd87fe00640 1 -- 172.21.15.151:0/3967979885 learned_addr learned my addr 172.21.15.151:0/3967979885 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:51:07.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.637+0000 7fd884c00640 1 --2- 172.21.15.151:0/3967979885 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd88006f220 0x7fd8800cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:07.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.637+0000 7fd885600640 1 --2- 172.21.15.151:0/3967979885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd88015cff0 0x7fd8800b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:07.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.637+0000 7fd87fe00640 1 -- 172.21.15.151:0/3967979885 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd88006f220 msgr2=0x7fd8800cdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:07.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.638+0000 7fd87fe00640 1 --2- 172.21.15.151:0/3967979885 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd88006f220 0x7fd8800cdad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:07.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.638+0000 7fd87fe00640 1 -- 172.21.15.151:0/3967979885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd88015cff0 msgr2=0x7fd8800b0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:07.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.638+0000 7fd87fe00640 1 --2- 172.21.15.151:0/3967979885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd88015cff0 0x7fd8800b0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:07.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.638+0000 7fd87fe00640 1 -- 172.21.15.151:0/3967979885 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8800ce010 con 0x7fd880159aa0 2024-09-16T15:51:07.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.638+0000 7fd87fe00640 1 --2- 172.21.15.151:0/3967979885 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd880159aa0 0x7fd8800b0430 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fd86c034040 tx=0x7fd86c0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:07.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.638+0000 7fd87cc00640 1 -- 172.21.15.151:0/3967979885 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86c03e030 con 0x7fd880159aa0 2024-09-16T15:51:07.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.638+0000 7fd87cc00640 1 -- 172.21.15.151:0/3967979885 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd86c0047a0 con 0x7fd880159aa0 2024-09-16T15:51:07.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.638+0000 7fd87cc00640 1 -- 172.21.15.151:0/3967979885 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86c035040 con 0x7fd880159aa0 2024-09-16T15:51:07.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.638+0000 7fd887600640 1 -- 172.21.15.151:0/3967979885 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8800b30c0 con 0x7fd880159aa0 2024-09-16T15:51:07.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.638+0000 7fd887600640 1 -- 172.21.15.151:0/3967979885 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8800b3680 con 0x7fd880159aa0 2024-09-16T15:51:07.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.639+0000 7fd887600640 1 -- 172.21.15.151:0/3967979885 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8800741a0 con 0x7fd880159aa0 2024-09-16T15:51:07.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.640+0000 7fd87cc00640 1 -- 172.21.15.151:0/3967979885 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7fd86c041030 con 0x7fd880159aa0 2024-09-16T15:51:07.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.641+0000 7fd87cc00640 1 --2- 172.21.15.151:0/3967979885 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd860079ed0 0x7fd86007c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:07.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.642+0000 7fd87cc00640 1 -- 172.21.15.151:0/3967979885 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5765+0+0 (secure 0 0 0) 0x7fd86c0b9d70 con 0x7fd880159aa0 2024-09-16T15:51:07.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.642+0000 7fd884c00640 1 --2- 172.21.15.151:0/3967979885 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd860079ed0 0x7fd86007c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:07.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.642+0000 7fd884c00640 1 --2- 172.21.15.151:0/3967979885 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd860079ed0 0x7fd86007c390 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fd88015ef60 tx=0x7fd87000a2d0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:07.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.645+0000 7fd87cc00640 1 -- 172.21.15.151:0/3967979885 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd86c083e40 con 0x7fd880159aa0 2024-09-16T15:51:07.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.898+0000 7fd887600640 1 -- 172.21.15.151:0/3967979885 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fd8800b1dd0 con 0x7fd880159aa0 2024-09-16T15:51:07.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.899+0000 7fd87cc00640 1 -- 172.21.15.151:0/3967979885 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1219 (secure 0 0 0) 0x7fd86c083740 con 0x7fd880159aa0 2024-09-16T15:51:07.901 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:51:07.901 INFO:teuthology.orchestra.run.smithi151.stdout:{"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":731,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":104,"num_osds":8,"num_up_osds":8,"osd_up_since":1726501150,"num_in_osds":8,"osd_in_since":1726501140,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":50},{"state_name":"creating+peering","count":44},{"state_name":"unknown","count":3}],"num_pgs":97,"num_pools":3,"num_objects":10,"data_bytes":590950,"bytes_used":1090695168,"bytes_avail":772003418112,"bytes_total":773094113280,"unknown_pgs_ratio":0.030927835032343864,"inactive_pgs_ratio":0.45360824465751648,"write_bytes_sec":185,"read_op_per_sec":0,"write_op_per_sec":3},"fsmap":{"epoch":66,"btime":"2024-09-16T15:51:05:403851+0000","id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":7899}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":15,"modified":"2024-09-16T15:49:29.793029+0000","services":{}},"progress_events":{}} 2024-09-16T15:51:07.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.901+0000 7fd887600640 1 -- 172.21.15.151:0/3967979885 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd860079ed0 msgr2=0x7fd86007c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:07.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.901+0000 7fd887600640 1 --2- 172.21.15.151:0/3967979885 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd860079ed0 0x7fd86007c390 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fd88015ef60 tx=0x7fd87000a2d0 comp rx=0 tx=0).stop 2024-09-16T15:51:07.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.901+0000 7fd887600640 1 -- 172.21.15.151:0/3967979885 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd880159aa0 msgr2=0x7fd8800b0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:07.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.901+0000 7fd887600640 1 --2- 172.21.15.151:0/3967979885 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd880159aa0 0x7fd8800b0430 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fd86c034040 tx=0x7fd86c0039e0 comp rx=0 tx=0).stop 2024-09-16T15:51:07.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.901+0000 7fd887600640 1 -- 172.21.15.151:0/3967979885 shutdown_connections 2024-09-16T15:51:07.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.901+0000 7fd887600640 1 --2- 172.21.15.151:0/3967979885 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd860079ed0 0x7fd86007c390 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:07.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.901+0000 7fd887600640 1 --2- 172.21.15.151:0/3967979885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd88015cff0 0x7fd8800b0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:07.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.901+0000 7fd887600640 1 --2- 172.21.15.151:0/3967979885 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd880159aa0 0x7fd8800b0430 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:07.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.901+0000 7fd887600640 1 --2- 172.21.15.151:0/3967979885 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd88006f220 0x7fd8800cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:07.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.901+0000 7fd887600640 1 -- 172.21.15.151:0/3967979885 >> 172.21.15.151:0/3967979885 conn(0x7fd8800b18e0 msgr2=0x7fd88015adc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:07.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.901+0000 7fd887600640 1 -- 172.21.15.151:0/3967979885 shutdown_connections 2024-09-16T15:51:07.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:07.901+0000 7fd887600640 1 -- 172.21.15.151:0/3967979885 wait complete. 2024-09-16T15:51:07.919 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-16T15:51:07.920 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:51:08.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:08.076+0000 7fb94f200640 1 -- 172.21.15.151:0/1315684734 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb92c004800 msgr2=0x7fb92c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:08.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:08.076+0000 7fb94f200640 1 --2- 172.21.15.151:0/1315684734 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb92c004800 0x7fb92c004c80 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fb93400c310 tx=0x7fb92c005830 comp rx=0 tx=0).stop 2024-09-16T15:51:08.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:08.076+0000 7fb94d200640 1 -- 172.21.15.151:0/1315684734 reap_dead start 2024-09-16T15:51:08.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:08.076+0000 7fb94f200640 1 -- 172.21.15.151:0/1315684734 shutdown_connections 2024-09-16T15:51:08.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:08.076+0000 7fb94f200640 1 -- 172.21.15.151:0/1315684734 >> 172.21.15.151:0/1315684734 conn(0x7fb9480b1950 msgr2=0x7fb9480b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:08.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:08.076+0000 7fb94f200640 1 -- 172.21.15.151:0/1315684734 shutdown_connections 2024-09-16T15:51:08.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:08.076+0000 7fb94f200640 1 -- 172.21.15.151:0/1315684734 wait complete. 2024-09-16T15:51:08.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:08.078+0000 7fb94f200640 1 Processor -- start 2024-09-16T15:51:08.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:08.078+0000 7fb94f200640 1 -- start start 2024-09-16T15:51:08.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:08.079+0000 7fb94f200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb92c004800 0x7fb94815dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:08.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:08.079+0000 7fb94f200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb94813b5d0 0x7fb9480acd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:08.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:08.079+0000 7fb94f200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb94815e450 0x7fb9480ad270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:08.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:08.079+0000 7fb94f200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb9480730b0 con 0x7fb92c004800 2024-09-16T15:51:08.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:08.079+0000 7fb94f200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb9480b4eb0 con 0x7fb94813b5d0 2024-09-16T15:51:08.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:08.079+0000 7fb94f200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fb948073230 con 0x7fb94815e450 2024-09-16T15:51:08.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:08.079+0000 7fb947e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb92c004800 0x7fb94815dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:08.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:08.079+0000 7fb947e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb92c004800 0x7fb94815dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52698/0 (socket says 172.21.15.151:52698) 2024-09-16T15:51:08.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:08.079+0000 7fb947e00640 1 -- 172.21.15.151:0/3929778991 learned_addr learned my addr 172.21.15.151:0/3929778991 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:51:08.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:08.080+0000 7fb947400640 1 --2- 172.21.15.151:0/3929778991 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb94813b5d0 0x7fb9480acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:08.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:08.080+0000 7fb94d200640 1 --2- 172.21.15.151:0/3929778991 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb94815e450 0x7fb9480ad270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:08.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:08.080+0000 7fb947e00640 1 -- 172.21.15.151:0/3929778991 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb94815e450 msgr2=0x7fb9480ad270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:08.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:08.080+0000 7fb947e00640 1 --2- 172.21.15.151:0/3929778991 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb94815e450 0x7fb9480ad270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:08.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:08.080+0000 7fb947e00640 1 -- 172.21.15.151:0/3929778991 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb94813b5d0 msgr2=0x7fb9480acd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:08.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:08.080+0000 7fb947e00640 1 --2- 172.21.15.151:0/3929778991 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb94813b5d0 0x7fb9480acd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:08.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:08.080+0000 7fb947e00640 1 -- 172.21.15.151:0/3929778991 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb94813bad0 con 0x7fb92c004800 2024-09-16T15:51:08.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:08.080+0000 7fb947400640 1 --2- 172.21.15.151:0/3929778991 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb94813b5d0 0x7fb9480acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:51:08.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:08.080+0000 7fb947e00640 1 --2- 172.21.15.151:0/3929778991 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb92c004800 0x7fb94815dd90 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fb938003f50 tx=0x7fb9380016f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:08.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:08.080+0000 7fb93fe00640 1 -- 172.21.15.151:0/3929778991 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb938011640 con 0x7fb92c004800 2024-09-16T15:51:08.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:08.080+0000 7fb93fe00640 1 -- 172.21.15.151:0/3929778991 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb938016020 con 0x7fb92c004800 2024-09-16T15:51:08.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:08.080+0000 7fb93fe00640 1 -- 172.21.15.151:0/3929778991 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb938011a60 con 0x7fb92c004800 2024-09-16T15:51:08.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:08.080+0000 7fb94f200640 1 -- 172.21.15.151:0/3929778991 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb94813bdb0 con 0x7fb92c004800 2024-09-16T15:51:08.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:08.080+0000 7fb94f200640 1 -- 172.21.15.151:0/3929778991 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb94813c370 con 0x7fb92c004800 2024-09-16T15:51:08.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:08.082+0000 7fb94f200640 1 -- 172.21.15.151:0/3929778991 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9480737f0 con 0x7fb92c004800 2024-09-16T15:51:08.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:08.083+0000 7fb93fe00640 1 -- 172.21.15.151:0/3929778991 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7fb938011c00 con 0x7fb92c004800 2024-09-16T15:51:08.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:08.084+0000 7fb93fe00640 1 --2- 172.21.15.151:0/3929778991 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb928079f00 0x7fb92807c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:08.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:08.084+0000 7fb93fe00640 1 -- 172.21.15.151:0/3929778991 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5765+0+0 (secure 0 0 0) 0x7fb93809a1c0 con 0x7fb92c004800 2024-09-16T15:51:08.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:08.084+0000 7fb947400640 1 --2- 172.21.15.151:0/3929778991 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb928079f00 0x7fb92807c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:08.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:08.085+0000 7fb947400640 1 --2- 172.21.15.151:0/3929778991 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb928079f00 0x7fb92807c3c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fb934033000 tx=0x7fb9480b2e20 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:08.090 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:08.088+0000 7fb93fe00640 1 -- 172.21.15.151:0/3929778991 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb938020020 con 0x7fb92c004800 2024-09-16T15:51:08.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:08.270+0000 7fb94f200640 1 -- 172.21.15.151:0/3929778991 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb94815f840 con 0x7fb928079f00 2024-09-16T15:51:08.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:08.274+0000 7fb93fe00640 1 -- 172.21.15.151:0/3929778991 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252499 (secure 0 0 0) 0x7fb934081a70 con 0x7fb928079f00 2024-09-16T15:51:08.279 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:51:08.280 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-09-16T15:51:08.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:08.278+0000 7fb94f200640 1 -- 172.21.15.151:0/3929778991 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb928079f00 msgr2=0x7fb92807c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:08.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:08.278+0000 7fb94f200640 1 --2- 172.21.15.151:0/3929778991 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb928079f00 0x7fb92807c3c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fb934033000 tx=0x7fb9480b2e20 comp rx=0 tx=0).stop 2024-09-16T15:51:08.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:08.278+0000 7fb94f200640 1 -- 172.21.15.151:0/3929778991 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb92c004800 msgr2=0x7fb94815dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:08.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:08.278+0000 7fb94f200640 1 --2- 172.21.15.151:0/3929778991 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb92c004800 0x7fb94815dd90 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fb938003f50 tx=0x7fb9380016f0 comp rx=0 tx=0).stop 2024-09-16T15:51:08.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:08.278+0000 7fb94f200640 1 -- 172.21.15.151:0/3929778991 shutdown_connections 2024-09-16T15:51:08.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:08.278+0000 7fb94f200640 1 --2- 172.21.15.151:0/3929778991 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb928079f00 0x7fb92807c3c0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:08.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:08.278+0000 7fb94f200640 1 --2- 172.21.15.151:0/3929778991 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb94813b5d0 0x7fb9480acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:08.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:08.278+0000 7fb94f200640 1 --2- 172.21.15.151:0/3929778991 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb94815e450 0x7fb9480ad270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:08.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:08.278+0000 7fb94f200640 1 --2- 172.21.15.151:0/3929778991 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb92c004800 0x7fb94815dd90 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:08.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:08.278+0000 7fb94f200640 1 -- 172.21.15.151:0/3929778991 >> 172.21.15.151:0/3929778991 conn(0x7fb9480b1950 msgr2=0x7fb948073a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:08.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:08.278+0000 7fb94f200640 1 -- 172.21.15.151:0/3929778991 shutdown_connections 2024-09-16T15:51:08.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:08.278+0000 7fb94f200640 1 -- 172.21.15.151:0/3929778991 wait complete. 2024-09-16T15:51:08.297 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":456,"stamp":"2024-09-16T15:51:07.880931+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":168,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1065416,"kb_used_data":11208,"kb_used_omap":12,"kb_used_meta":1054003,"kb_avail":753909304,"statfs":{"total":773094113280,"available":772003127296,"internally_reserved":0,"allocated":11476992,"data_stored":5647860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12738,"internal_metadata":1079299646},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.517887"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.310177+0000","last_change":"2024-09-16T15:51:02.386431+0000","last_active":"2024-09-16T15:51:04.310177+0000","last_peered":"2024-09-16T15:51:04.310177+0000","last_clean":"2024-09-16T15:51:04.310177+0000","last_became_active":"2024-09-16T15:51:02.386126+0000","last_became_peered":"2024-09-16T15:51:02.386126+0000","last_unstale":"2024-09-16T15:51:04.310177+0000","last_undegraded":"2024-09-16T15:51:04.310177+0000","last_fullsized":"2024-09-16T15:51:04.310177+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:21.600849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309583+0000","last_change":"2024-09-16T15:51:02.388621+0000","last_active":"2024-09-16T15:51:04.309583+0000","last_peered":"2024-09-16T15:51:04.309583+0000","last_clean":"2024-09-16T15:51:04.309583+0000","last_became_active":"2024-09-16T15:51:02.388294+0000","last_became_peered":"2024-09-16T15:51:02.388294+0000","last_unstale":"2024-09-16T15:51:04.309583+0000","last_undegraded":"2024-09-16T15:51:04.309583+0000","last_fullsized":"2024-09-16T15:51:04.309583+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:44:02.181311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:03.381077+0000","last_change":"2024-09-16T15:51:02.396532+0000","last_active":"2024-09-16T15:51:03.381077+0000","last_peered":"2024-09-16T15:51:03.381077+0000","last_clean":"2024-09-16T15:51:03.381077+0000","last_became_active":"2024-09-16T15:51:02.396235+0000","last_became_peered":"2024-09-16T15:51:02.396235+0000","last_unstale":"2024-09-16T15:51:03.381077+0000","last_undegraded":"2024-09-16T15:51:03.381077+0000","last_fullsized":"2024-09-16T15:51:03.381077+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:39:19.449312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309522+0000","last_change":"2024-09-16T15:51:02.385219+0000","last_active":"2024-09-16T15:51:04.309522+0000","last_peered":"2024-09-16T15:51:04.309522+0000","last_clean":"2024-09-16T15:51:04.309522+0000","last_became_active":"2024-09-16T15:51:02.384310+0000","last_became_peered":"2024-09-16T15:51:02.384310+0000","last_unstale":"2024-09-16T15:51:04.309522+0000","last_undegraded":"2024-09-16T15:51:04.309522+0000","last_fullsized":"2024-09-16T15:51:04.309522+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:24:11.153158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309076+0000","last_change":"2024-09-16T15:51:02.397330+0000","last_active":"2024-09-16T15:51:04.309076+0000","last_peered":"2024-09-16T15:51:04.309076+0000","last_clean":"2024-09-16T15:51:04.309076+0000","last_became_active":"2024-09-16T15:51:02.397058+0000","last_became_peered":"2024-09-16T15:51:02.397058+0000","last_unstale":"2024-09-16T15:51:04.309076+0000","last_undegraded":"2024-09-16T15:51:04.309076+0000","last_fullsized":"2024-09-16T15:51:04.309076+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:21:04.406320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.310287+0000","last_change":"2024-09-16T15:51:02.403013+0000","last_active":"2024-09-16T15:51:04.310287+0000","last_peered":"2024-09-16T15:51:04.310287+0000","last_clean":"2024-09-16T15:51:04.310287+0000","last_became_active":"2024-09-16T15:51:02.402778+0000","last_became_peered":"2024-09-16T15:51:02.402778+0000","last_unstale":"2024-09-16T15:51:04.310287+0000","last_undegraded":"2024-09-16T15:51:04.310287+0000","last_fullsized":"2024-09-16T15:51:04.310287+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:27:46.534033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.308977+0000","last_change":"2024-09-16T15:51:02.389718+0000","last_active":"2024-09-16T15:51:04.308977+0000","last_peered":"2024-09-16T15:51:04.308977+0000","last_clean":"2024-09-16T15:51:04.308977+0000","last_became_active":"2024-09-16T15:51:02.386922+0000","last_became_peered":"2024-09-16T15:51:02.386922+0000","last_unstale":"2024-09-16T15:51:04.308977+0000","last_undegraded":"2024-09-16T15:51:04.308977+0000","last_fullsized":"2024-09-16T15:51:04.308977+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:40:46.193203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:03.379704+0000","last_change":"2024-09-16T15:51:02.392522+0000","last_active":"2024-09-16T15:51:03.379704+0000","last_peered":"2024-09-16T15:51:03.379704+0000","last_clean":"2024-09-16T15:51:03.379704+0000","last_became_active":"2024-09-16T15:51:02.392253+0000","last_became_peered":"2024-09-16T15:51:02.392253+0000","last_unstale":"2024-09-16T15:51:03.379704+0000","last_undegraded":"2024-09-16T15:51:03.379704+0000","last_fullsized":"2024-09-16T15:51:03.379704+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:10:47.555331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309588+0000","last_change":"2024-09-16T15:51:02.397324+0000","last_active":"2024-09-16T15:51:04.309588+0000","last_peered":"2024-09-16T15:51:04.309588+0000","last_clean":"2024-09-16T15:51:04.309588+0000","last_became_active":"2024-09-16T15:51:02.397074+0000","last_became_peered":"2024-09-16T15:51:02.397074+0000","last_unstale":"2024-09-16T15:51:04.309588+0000","last_undegraded":"2024-09-16T15:51:04.309588+0000","last_fullsized":"2024-09-16T15:51:04.309588+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:19:02.723695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:03.378265+0000","last_change":"2024-09-16T15:51:02.385658+0000","last_active":"2024-09-16T15:51:03.378265+0000","last_peered":"2024-09-16T15:51:03.378265+0000","last_clean":"2024-09-16T15:51:03.378265+0000","last_became_active":"2024-09-16T15:51:02.384691+0000","last_became_peered":"2024-09-16T15:51:02.384691+0000","last_unstale":"2024-09-16T15:51:03.378265+0000","last_undegraded":"2024-09-16T15:51:03.378265+0000","last_fullsized":"2024-09-16T15:51:03.378265+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:21:46.321024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309172+0000","last_change":"2024-09-16T15:51:02.406193+0000","last_active":"2024-09-16T15:51:04.309172+0000","last_peered":"2024-09-16T15:51:04.309172+0000","last_clean":"2024-09-16T15:51:04.309172+0000","last_became_active":"2024-09-16T15:51:02.402890+0000","last_became_peered":"2024-09-16T15:51:02.402890+0000","last_unstale":"2024-09-16T15:51:04.309172+0000","last_undegraded":"2024-09-16T15:51:04.309172+0000","last_fullsized":"2024-09-16T15:51:04.309172+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:31:52.533744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.310084+0000","last_change":"2024-09-16T15:51:02.393952+0000","last_active":"2024-09-16T15:51:04.310084+0000","last_peered":"2024-09-16T15:51:04.310084+0000","last_clean":"2024-09-16T15:51:04.310084+0000","last_became_active":"2024-09-16T15:51:02.393190+0000","last_became_peered":"2024-09-16T15:51:02.393190+0000","last_unstale":"2024-09-16T15:51:04.310084+0000","last_undegraded":"2024-09-16T15:51:04.310084+0000","last_fullsized":"2024-09-16T15:51:04.310084+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:56:55.816696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309716+0000","last_change":"2024-09-16T15:51:02.407269+0000","last_active":"2024-09-16T15:51:04.309716+0000","last_peered":"2024-09-16T15:51:04.309716+0000","last_clean":"2024-09-16T15:51:04.309716+0000","last_became_active":"2024-09-16T15:51:02.406943+0000","last_became_peered":"2024-09-16T15:51:02.406943+0000","last_unstale":"2024-09-16T15:51:04.309716+0000","last_undegraded":"2024-09-16T15:51:04.309716+0000","last_fullsized":"2024-09-16T15:51:04.309716+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:29:33.561032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.308960+0000","last_change":"2024-09-16T15:51:02.398129+0000","last_active":"2024-09-16T15:51:04.308960+0000","last_peered":"2024-09-16T15:51:04.308960+0000","last_clean":"2024-09-16T15:51:04.308960+0000","last_became_active":"2024-09-16T15:51:02.397834+0000","last_became_peered":"2024-09-16T15:51:02.397834+0000","last_unstale":"2024-09-16T15:51:04.308960+0000","last_undegraded":"2024-09-16T15:51:04.308960+0000","last_fullsized":"2024-09-16T15:51:04.308960+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:15:15.665992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309257+0000","last_change":"2024-09-16T15:51:02.396682+0000","last_active":"2024-09-16T15:51:04.309257+0000","last_peered":"2024-09-16T15:51:04.309257+0000","last_clean":"2024-09-16T15:51:04.309257+0000","last_became_active":"2024-09-16T15:51:02.396412+0000","last_became_peered":"2024-09-16T15:51:02.396412+0000","last_unstale":"2024-09-16T15:51:04.309257+0000","last_undegraded":"2024-09-16T15:51:04.309257+0000","last_fullsized":"2024-09-16T15:51:04.309257+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:56:52.705232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309717+0000","last_change":"2024-09-16T15:51:02.396610+0000","last_active":"2024-09-16T15:51:04.309717+0000","last_peered":"2024-09-16T15:51:04.309717+0000","last_clean":"2024-09-16T15:51:04.309717+0000","last_became_active":"2024-09-16T15:51:02.396178+0000","last_became_peered":"2024-09-16T15:51:02.396178+0000","last_unstale":"2024-09-16T15:51:04.309717+0000","last_undegraded":"2024-09-16T15:51:04.309717+0000","last_fullsized":"2024-09-16T15:51:04.309717+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:27:42.235783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:03.381293+0000","last_change":"2024-09-16T15:51:02.400919+0000","last_active":"2024-09-16T15:51:03.381293+0000","last_peered":"2024-09-16T15:51:03.381293+0000","last_clean":"2024-09-16T15:51:03.381293+0000","last_became_active":"2024-09-16T15:51:02.398907+0000","last_became_peered":"2024-09-16T15:51:02.398907+0000","last_unstale":"2024-09-16T15:51:03.381293+0000","last_undegraded":"2024-09-16T15:51:03.381293+0000","last_fullsized":"2024-09-16T15:51:03.381293+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:10:06.599715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.310010+0000","last_change":"2024-09-16T15:51:02.399241+0000","last_active":"2024-09-16T15:51:04.310010+0000","last_peered":"2024-09-16T15:51:04.310010+0000","last_clean":"2024-09-16T15:51:04.310010+0000","last_became_active":"2024-09-16T15:51:02.398973+0000","last_became_peered":"2024-09-16T15:51:02.398973+0000","last_unstale":"2024-09-16T15:51:04.310010+0000","last_undegraded":"2024-09-16T15:51:04.310010+0000","last_fullsized":"2024-09-16T15:51:04.310010+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:21:46.321024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:03.381167+0000","last_change":"2024-09-16T15:51:02.385117+0000","last_active":"2024-09-16T15:51:03.381167+0000","last_peered":"2024-09-16T15:51:03.381167+0000","last_clean":"2024-09-16T15:51:03.381167+0000","last_became_active":"2024-09-16T15:51:02.384813+0000","last_became_peered":"2024-09-16T15:51:02.384813+0000","last_unstale":"2024-09-16T15:51:03.381167+0000","last_undegraded":"2024-09-16T15:51:03.381167+0000","last_fullsized":"2024-09-16T15:51:03.381167+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:21:16.736819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.310289+0000","last_change":"2024-09-16T15:51:02.404614+0000","last_active":"2024-09-16T15:51:04.310289+0000","last_peered":"2024-09-16T15:51:04.310289+0000","last_clean":"2024-09-16T15:51:04.310289+0000","last_became_active":"2024-09-16T15:51:02.399859+0000","last_became_peered":"2024-09-16T15:51:02.399859+0000","last_unstale":"2024-09-16T15:51:04.310289+0000","last_undegraded":"2024-09-16T15:51:04.310289+0000","last_fullsized":"2024-09-16T15:51:04.310289+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:39:19.449312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:03.380958+0000","last_change":"2024-09-16T15:51:02.393101+0000","last_active":"2024-09-16T15:51:03.380958+0000","last_peered":"2024-09-16T15:51:03.380958+0000","last_clean":"2024-09-16T15:51:03.380958+0000","last_became_active":"2024-09-16T15:51:02.392793+0000","last_became_peered":"2024-09-16T15:51:02.392793+0000","last_unstale":"2024-09-16T15:51:03.380958+0000","last_undegraded":"2024-09-16T15:51:03.380958+0000","last_fullsized":"2024-09-16T15:51:03.380958+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:28:04.550380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:03.379789+0000","last_change":"2024-09-16T15:51:02.398357+0000","last_active":"2024-09-16T15:51:03.379789+0000","last_peered":"2024-09-16T15:51:03.379789+0000","last_clean":"2024-09-16T15:51:03.379789+0000","last_became_active":"2024-09-16T15:51:02.398089+0000","last_became_peered":"2024-09-16T15:51:02.398089+0000","last_unstale":"2024-09-16T15:51:03.379789+0000","last_undegraded":"2024-09-16T15:51:03.379789+0000","last_fullsized":"2024-09-16T15:51:03.379789+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:43:25.312945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309505+0000","last_change":"2024-09-16T15:51:02.399544+0000","last_active":"2024-09-16T15:51:04.309505+0000","last_peered":"2024-09-16T15:51:04.309505+0000","last_clean":"2024-09-16T15:51:04.309505+0000","last_became_active":"2024-09-16T15:51:02.396515+0000","last_became_peered":"2024-09-16T15:51:02.396515+0000","last_unstale":"2024-09-16T15:51:04.309505+0000","last_undegraded":"2024-09-16T15:51:04.309505+0000","last_fullsized":"2024-09-16T15:51:04.309505+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:54:35.722237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.308818+0000","last_change":"2024-09-16T15:51:02.389471+0000","last_active":"2024-09-16T15:51:04.308818+0000","last_peered":"2024-09-16T15:51:04.308818+0000","last_clean":"2024-09-16T15:51:04.308818+0000","last_became_active":"2024-09-16T15:51:02.389208+0000","last_became_peered":"2024-09-16T15:51:02.389208+0000","last_unstale":"2024-09-16T15:51:04.308818+0000","last_undegraded":"2024-09-16T15:51:04.308818+0000","last_fullsized":"2024-09-16T15:51:04.308818+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:41:29.520138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:03.378481+0000","last_change":"2024-09-16T15:51:02.391207+0000","last_active":"2024-09-16T15:51:03.378481+0000","last_peered":"2024-09-16T15:51:03.378481+0000","last_clean":"2024-09-16T15:51:03.378481+0000","last_became_active":"2024-09-16T15:51:02.390909+0000","last_became_peered":"2024-09-16T15:51:02.390909+0000","last_unstale":"2024-09-16T15:51:03.378481+0000","last_undegraded":"2024-09-16T15:51:03.378481+0000","last_fullsized":"2024-09-16T15:51:03.378481+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:21.600849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309644+0000","last_change":"2024-09-16T15:51:02.401422+0000","last_active":"2024-09-16T15:51:04.309644+0000","last_peered":"2024-09-16T15:51:04.309644+0000","last_clean":"2024-09-16T15:51:04.309644+0000","last_became_active":"2024-09-16T15:51:02.400804+0000","last_became_peered":"2024-09-16T15:51:02.400804+0000","last_unstale":"2024-09-16T15:51:04.309644+0000","last_undegraded":"2024-09-16T15:51:04.309644+0000","last_fullsized":"2024-09-16T15:51:04.309644+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:54:09.898078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309724+0000","last_change":"2024-09-16T15:51:02.404620+0000","last_active":"2024-09-16T15:51:04.309724+0000","last_peered":"2024-09-16T15:51:04.309724+0000","last_clean":"2024-09-16T15:51:04.309724+0000","last_became_active":"2024-09-16T15:51:02.404361+0000","last_became_peered":"2024-09-16T15:51:02.404361+0000","last_unstale":"2024-09-16T15:51:04.309724+0000","last_undegraded":"2024-09-16T15:51:04.309724+0000","last_fullsized":"2024-09-16T15:51:04.309724+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:00:20.598220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.308994+0000","last_change":"2024-09-16T15:51:02.403010+0000","last_active":"2024-09-16T15:51:04.308994+0000","last_peered":"2024-09-16T15:51:04.308994+0000","last_clean":"2024-09-16T15:51:04.308994+0000","last_became_active":"2024-09-16T15:51:02.402725+0000","last_became_peered":"2024-09-16T15:51:02.402725+0000","last_unstale":"2024-09-16T15:51:04.308994+0000","last_undegraded":"2024-09-16T15:51:04.308994+0000","last_fullsized":"2024-09-16T15:51:04.308994+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:30:13.508956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309979+0000","last_change":"2024-09-16T15:51:02.405833+0000","last_active":"2024-09-16T15:51:04.309979+0000","last_peered":"2024-09-16T15:51:04.309979+0000","last_clean":"2024-09-16T15:51:04.309979+0000","last_became_active":"2024-09-16T15:51:02.404364+0000","last_became_peered":"2024-09-16T15:51:02.404364+0000","last_unstale":"2024-09-16T15:51:04.309979+0000","last_undegraded":"2024-09-16T15:51:04.309979+0000","last_fullsized":"2024-09-16T15:51:04.309979+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:47:54.421341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309062+0000","last_change":"2024-09-16T15:51:02.402965+0000","last_active":"2024-09-16T15:51:04.309062+0000","last_peered":"2024-09-16T15:51:04.309062+0000","last_clean":"2024-09-16T15:51:04.309062+0000","last_became_active":"2024-09-16T15:51:02.401107+0000","last_became_peered":"2024-09-16T15:51:02.401107+0000","last_unstale":"2024-09-16T15:51:04.309062+0000","last_undegraded":"2024-09-16T15:51:04.309062+0000","last_fullsized":"2024-09-16T15:51:04.309062+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:24:38.932170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:03.380690+0000","last_change":"2024-09-16T15:51:02.392272+0000","last_active":"2024-09-16T15:51:03.380690+0000","last_peered":"2024-09-16T15:51:03.380690+0000","last_clean":"2024-09-16T15:51:03.380690+0000","last_became_active":"2024-09-16T15:51:02.392008+0000","last_became_peered":"2024-09-16T15:51:02.392008+0000","last_unstale":"2024-09-16T15:51:03.380690+0000","last_undegraded":"2024-09-16T15:51:03.380690+0000","last_fullsized":"2024-09-16T15:51:03.380690+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:56:55.816696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309158+0000","last_change":"2024-09-16T15:51:02.384379+0000","last_active":"2024-09-16T15:51:04.309158+0000","last_peered":"2024-09-16T15:51:04.309158+0000","last_clean":"2024-09-16T15:51:04.309158+0000","last_became_active":"2024-09-16T15:51:02.384012+0000","last_became_peered":"2024-09-16T15:51:02.384012+0000","last_unstale":"2024-09-16T15:51:04.309158+0000","last_undegraded":"2024-09-16T15:51:04.309158+0000","last_fullsized":"2024-09-16T15:51:04.309158+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:12:54.477349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:03.381453+0000","last_change":"2024-09-16T15:51:02.388400+0000","last_active":"2024-09-16T15:51:03.381453+0000","last_peered":"2024-09-16T15:51:03.381453+0000","last_clean":"2024-09-16T15:51:03.381453+0000","last_became_active":"2024-09-16T15:51:02.388145+0000","last_became_peered":"2024-09-16T15:51:02.388145+0000","last_unstale":"2024-09-16T15:51:03.381453+0000","last_undegraded":"2024-09-16T15:51:03.381453+0000","last_fullsized":"2024-09-16T15:51:03.381453+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:01:13.753963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1f","version":"103'4","reported_seq":19,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309998+0000","last_change":"2024-09-16T15:51:01.375200+0000","last_active":"2024-09-16T15:51:04.309998+0000","last_peered":"2024-09-16T15:51:04.309998+0000","last_clean":"2024-09-16T15:51:04.309998+0000","last_became_active":"2024-09-16T15:51:01.374919+0000","last_became_peered":"2024-09-16T15:51:01.374919+0000","last_unstale":"2024-09-16T15:51:04.309998+0000","last_undegraded":"2024-09-16T15:51:04.309998+0000","last_fullsized":"2024-09-16T15:51:04.309998+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:36:44.338407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.310065+0000","last_change":"2024-09-16T15:51:02.402988+0000","last_active":"2024-09-16T15:51:04.310065+0000","last_peered":"2024-09-16T15:51:04.310065+0000","last_clean":"2024-09-16T15:51:04.310065+0000","last_became_active":"2024-09-16T15:51:02.402708+0000","last_became_peered":"2024-09-16T15:51:02.402708+0000","last_unstale":"2024-09-16T15:51:04.310065+0000","last_undegraded":"2024-09-16T15:51:04.310065+0000","last_fullsized":"2024-09-16T15:51:04.310065+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:14:48.952444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1e","version":"103'1","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309267+0000","last_change":"2024-09-16T15:51:01.391174+0000","last_active":"2024-09-16T15:51:04.309267+0000","last_peered":"2024-09-16T15:51:04.309267+0000","last_clean":"2024-09-16T15:51:04.309267+0000","last_became_active":"2024-09-16T15:51:01.390904+0000","last_became_peered":"2024-09-16T15:51:01.390904+0000","last_unstale":"2024-09-16T15:51:04.309267+0000","last_undegraded":"2024-09-16T15:51:04.309267+0000","last_fullsized":"2024-09-16T15:51:04.309267+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:00:46.351712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309807+0000","last_change":"2024-09-16T15:51:02.382876+0000","last_active":"2024-09-16T15:51:04.309807+0000","last_peered":"2024-09-16T15:51:04.309807+0000","last_clean":"2024-09-16T15:51:04.309807+0000","last_became_active":"2024-09-16T15:51:02.382617+0000","last_became_peered":"2024-09-16T15:51:02.382617+0000","last_unstale":"2024-09-16T15:51:04.309807+0000","last_undegraded":"2024-09-16T15:51:04.309807+0000","last_fullsized":"2024-09-16T15:51:04.309807+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:58:47.475064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"103'2","reported_seq":17,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:03.381228+0000","last_change":"2024-09-16T15:51:01.388774+0000","last_active":"2024-09-16T15:51:03.381228+0000","last_peered":"2024-09-16T15:51:03.381228+0000","last_clean":"2024-09-16T15:51:03.381228+0000","last_became_active":"2024-09-16T15:51:01.388500+0000","last_became_peered":"2024-09-16T15:51:01.388500+0000","last_unstale":"2024-09-16T15:51:03.381228+0000","last_undegraded":"2024-09-16T15:51:03.381228+0000","last_fullsized":"2024-09-16T15:51:03.381228+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:46:58.289338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:03.378280+0000","last_change":"2024-09-16T15:51:02.386042+0000","last_active":"2024-09-16T15:51:03.378280+0000","last_peered":"2024-09-16T15:51:03.378280+0000","last_clean":"2024-09-16T15:51:03.378280+0000","last_became_active":"2024-09-16T15:51:02.385803+0000","last_became_peered":"2024-09-16T15:51:02.385803+0000","last_unstale":"2024-09-16T15:51:03.378280+0000","last_undegraded":"2024-09-16T15:51:03.378280+0000","last_fullsized":"2024-09-16T15:51:03.378280+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:30:51.230580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"103'2","reported_seq":17,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309747+0000","last_change":"2024-09-16T15:51:01.390985+0000","last_active":"2024-09-16T15:51:04.309747+0000","last_peered":"2024-09-16T15:51:04.309747+0000","last_clean":"2024-09-16T15:51:04.309747+0000","last_became_active":"2024-09-16T15:51:01.390463+0000","last_became_peered":"2024-09-16T15:51:01.390463+0000","last_unstale":"2024-09-16T15:51:04.309747+0000","last_undegraded":"2024-09-16T15:51:04.309747+0000","last_fullsized":"2024-09-16T15:51:04.309747+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:57:05.221628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.310415+0000","last_change":"2024-09-16T15:51:02.387425+0000","last_active":"2024-09-16T15:51:04.310415+0000","last_peered":"2024-09-16T15:51:04.310415+0000","last_clean":"2024-09-16T15:51:04.310415+0000","last_became_active":"2024-09-16T15:51:02.386275+0000","last_became_peered":"2024-09-16T15:51:02.386275+0000","last_unstale":"2024-09-16T15:51:04.310415+0000","last_undegraded":"2024-09-16T15:51:04.310415+0000","last_fullsized":"2024-09-16T15:51:04.310415+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:57:45.368832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1b","version":"103'1","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309180+0000","last_change":"2024-09-16T15:51:01.387614+0000","last_active":"2024-09-16T15:51:04.309180+0000","last_peered":"2024-09-16T15:51:04.309180+0000","last_clean":"2024-09-16T15:51:04.309180+0000","last_became_active":"2024-09-16T15:51:01.387163+0000","last_became_peered":"2024-09-16T15:51:01.387163+0000","last_unstale":"2024-09-16T15:51:04.309180+0000","last_undegraded":"2024-09-16T15:51:04.309180+0000","last_fullsized":"2024-09-16T15:51:04.309180+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:48:24.298645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309297+0000","last_change":"2024-09-16T15:51:02.404828+0000","last_active":"2024-09-16T15:51:04.309297+0000","last_peered":"2024-09-16T15:51:04.309297+0000","last_clean":"2024-09-16T15:51:04.309297+0000","last_became_active":"2024-09-16T15:51:02.403724+0000","last_became_peered":"2024-09-16T15:51:02.403724+0000","last_unstale":"2024-09-16T15:51:04.309297+0000","last_undegraded":"2024-09-16T15:51:04.309297+0000","last_fullsized":"2024-09-16T15:51:04.309297+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:55:45.502743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1a","version":"103'1","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:03.380176+0000","last_change":"2024-09-16T15:51:01.378848+0000","last_active":"2024-09-16T15:51:03.380176+0000","last_peered":"2024-09-16T15:51:03.380176+0000","last_clean":"2024-09-16T15:51:03.380176+0000","last_became_active":"2024-09-16T15:51:01.377621+0000","last_became_peered":"2024-09-16T15:51:01.377621+0000","last_unstale":"2024-09-16T15:51:03.380176+0000","last_undegraded":"2024-09-16T15:51:03.380176+0000","last_fullsized":"2024-09-16T15:51:03.380176+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:44:22.903859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.308778+0000","last_change":"2024-09-16T15:51:02.405495+0000","last_active":"2024-09-16T15:51:04.308778+0000","last_peered":"2024-09-16T15:51:04.308778+0000","last_clean":"2024-09-16T15:51:04.308778+0000","last_became_active":"2024-09-16T15:51:02.405222+0000","last_became_peered":"2024-09-16T15:51:02.405222+0000","last_unstale":"2024-09-16T15:51:04.308778+0000","last_undegraded":"2024-09-16T15:51:04.308778+0000","last_fullsized":"2024-09-16T15:51:04.308778+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:22:02.790695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309131+0000","last_change":"2024-09-16T15:51:01.372791+0000","last_active":"2024-09-16T15:51:04.309131+0000","last_peered":"2024-09-16T15:51:04.309131+0000","last_clean":"2024-09-16T15:51:04.309131+0000","last_became_active":"2024-09-16T15:51:01.372493+0000","last_became_peered":"2024-09-16T15:51:01.372493+0000","last_unstale":"2024-09-16T15:51:04.309131+0000","last_undegraded":"2024-09-16T15:51:04.309131+0000","last_fullsized":"2024-09-16T15:51:04.309131+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:01:27.263927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.308738+0000","last_change":"2024-09-16T15:51:02.386313+0000","last_active":"2024-09-16T15:51:04.308738+0000","last_peered":"2024-09-16T15:51:04.308738+0000","last_clean":"2024-09-16T15:51:04.308738+0000","last_became_active":"2024-09-16T15:51:02.385942+0000","last_became_peered":"2024-09-16T15:51:02.385942+0000","last_unstale":"2024-09-16T15:51:04.308738+0000","last_undegraded":"2024-09-16T15:51:04.308738+0000","last_fullsized":"2024-09-16T15:51:04.308738+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:29:23.387660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:03.379691+0000","last_change":"2024-09-16T15:51:01.386550+0000","last_active":"2024-09-16T15:51:03.379691+0000","last_peered":"2024-09-16T15:51:03.379691+0000","last_clean":"2024-09-16T15:51:03.379691+0000","last_became_active":"2024-09-16T15:51:01.386275+0000","last_became_peered":"2024-09-16T15:51:01.386275+0000","last_unstale":"2024-09-16T15:51:03.379691+0000","last_undegraded":"2024-09-16T15:51:03.379691+0000","last_fullsized":"2024-09-16T15:51:03.379691+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:42:26.502435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.310033+0000","last_change":"2024-09-16T15:51:02.386731+0000","last_active":"2024-09-16T15:51:04.310033+0000","last_peered":"2024-09-16T15:51:04.310033+0000","last_clean":"2024-09-16T15:51:04.310033+0000","last_became_active":"2024-09-16T15:51:02.384022+0000","last_became_peered":"2024-09-16T15:51:02.384022+0000","last_unstale":"2024-09-16T15:51:04.310033+0000","last_undegraded":"2024-09-16T15:51:04.310033+0000","last_fullsized":"2024-09-16T15:51:04.310033+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:30:51.230580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:03.380909+0000","last_change":"2024-09-16T15:51:01.378467+0000","last_active":"2024-09-16T15:51:03.380909+0000","last_peered":"2024-09-16T15:51:03.380909+0000","last_clean":"2024-09-16T15:51:03.380909+0000","last_became_active":"2024-09-16T15:51:01.378205+0000","last_became_peered":"2024-09-16T15:51:01.378205+0000","last_unstale":"2024-09-16T15:51:03.380909+0000","last_undegraded":"2024-09-16T15:51:03.380909+0000","last_fullsized":"2024-09-16T15:51:03.380909+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:39:21.798976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309818+0000","last_change":"2024-09-16T15:51:02.399038+0000","last_active":"2024-09-16T15:51:04.309818+0000","last_peered":"2024-09-16T15:51:04.309818+0000","last_clean":"2024-09-16T15:51:04.309818+0000","last_became_active":"2024-09-16T15:51:02.398752+0000","last_became_peered":"2024-09-16T15:51:02.398752+0000","last_unstale":"2024-09-16T15:51:04.309818+0000","last_undegraded":"2024-09-16T15:51:04.309818+0000","last_fullsized":"2024-09-16T15:51:04.309818+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:56:24.131464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.310208+0000","last_change":"2024-09-16T15:51:01.389590+0000","last_active":"2024-09-16T15:51:04.310208+0000","last_peered":"2024-09-16T15:51:04.310208+0000","last_clean":"2024-09-16T15:51:04.310208+0000","last_became_active":"2024-09-16T15:51:01.389186+0000","last_became_peered":"2024-09-16T15:51:01.389186+0000","last_unstale":"2024-09-16T15:51:04.310208+0000","last_undegraded":"2024-09-16T15:51:04.310208+0000","last_fullsized":"2024-09-16T15:51:04.310208+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:22:23.409721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309112+0000","last_change":"2024-09-16T15:51:02.391455+0000","last_active":"2024-09-16T15:51:04.309112+0000","last_peered":"2024-09-16T15:51:04.309112+0000","last_clean":"2024-09-16T15:51:04.309112+0000","last_became_active":"2024-09-16T15:51:02.391250+0000","last_became_peered":"2024-09-16T15:51:02.391250+0000","last_unstale":"2024-09-16T15:51:04.309112+0000","last_undegraded":"2024-09-16T15:51:04.309112+0000","last_fullsized":"2024-09-16T15:51:04.309112+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:17:21.134677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.15","version":"103'3","reported_seq":18,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309817+0000","last_change":"2024-09-16T15:51:01.379431+0000","last_active":"2024-09-16T15:51:04.309817+0000","last_peered":"2024-09-16T15:51:04.309817+0000","last_clean":"2024-09-16T15:51:04.309817+0000","last_became_active":"2024-09-16T15:51:01.379201+0000","last_became_peered":"2024-09-16T15:51:01.379201+0000","last_unstale":"2024-09-16T15:51:04.309817+0000","last_undegraded":"2024-09-16T15:51:04.309817+0000","last_fullsized":"2024-09-16T15:51:04.309817+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:04:50.965816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309521+0000","last_change":"2024-09-16T15:51:02.400930+0000","last_active":"2024-09-16T15:51:04.309521+0000","last_peered":"2024-09-16T15:51:04.309521+0000","last_clean":"2024-09-16T15:51:04.309521+0000","last_became_active":"2024-09-16T15:51:02.400622+0000","last_became_peered":"2024-09-16T15:51:02.400622+0000","last_unstale":"2024-09-16T15:51:04.309521+0000","last_undegraded":"2024-09-16T15:51:04.309521+0000","last_fullsized":"2024-09-16T15:51:04.309521+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:12:08.141906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.14","version":"103'6","reported_seq":21,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.310440+0000","last_change":"2024-09-16T15:51:01.378141+0000","last_active":"2024-09-16T15:51:04.310440+0000","last_peered":"2024-09-16T15:51:04.310440+0000","last_clean":"2024-09-16T15:51:04.310440+0000","last_became_active":"2024-09-16T15:51:01.377875+0000","last_became_peered":"2024-09-16T15:51:01.377875+0000","last_unstale":"2024-09-16T15:51:04.310440+0000","last_undegraded":"2024-09-16T15:51:04.310440+0000","last_fullsized":"2024-09-16T15:51:04.310440+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:48:33.457678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309046+0000","last_change":"2024-09-16T15:51:02.387140+0000","last_active":"2024-09-16T15:51:04.309046+0000","last_peered":"2024-09-16T15:51:04.309046+0000","last_clean":"2024-09-16T15:51:04.309046+0000","last_became_active":"2024-09-16T15:51:02.385974+0000","last_became_peered":"2024-09-16T15:51:02.385974+0000","last_unstale":"2024-09-16T15:51:04.309046+0000","last_undegraded":"2024-09-16T15:51:04.309046+0000","last_fullsized":"2024-09-16T15:51:04.309046+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:37:03.319352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"103'2","reported_seq":17,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:03.379867+0000","last_change":"2024-09-16T15:51:01.384546+0000","last_active":"2024-09-16T15:51:03.379867+0000","last_peered":"2024-09-16T15:51:03.379867+0000","last_clean":"2024-09-16T15:51:03.379867+0000","last_became_active":"2024-09-16T15:51:01.384288+0000","last_became_peered":"2024-09-16T15:51:01.384288+0000","last_unstale":"2024-09-16T15:51:03.379867+0000","last_undegraded":"2024-09-16T15:51:03.379867+0000","last_fullsized":"2024-09-16T15:51:03.379867+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:56:23.125710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:03.380503+0000","last_change":"2024-09-16T15:51:02.396326+0000","last_active":"2024-09-16T15:51:03.380503+0000","last_peered":"2024-09-16T15:51:03.380503+0000","last_clean":"2024-09-16T15:51:03.380503+0000","last_became_active":"2024-09-16T15:51:02.394643+0000","last_became_peered":"2024-09-16T15:51:02.394643+0000","last_unstale":"2024-09-16T15:51:03.380503+0000","last_undegraded":"2024-09-16T15:51:03.380503+0000","last_fullsized":"2024-09-16T15:51:03.380503+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:29:23.387660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309143+0000","last_change":"2024-09-16T15:51:01.384749+0000","last_active":"2024-09-16T15:51:04.309143+0000","last_peered":"2024-09-16T15:51:04.309143+0000","last_clean":"2024-09-16T15:51:04.309143+0000","last_became_active":"2024-09-16T15:51:01.384433+0000","last_became_peered":"2024-09-16T15:51:01.384433+0000","last_unstale":"2024-09-16T15:51:04.309143+0000","last_undegraded":"2024-09-16T15:51:04.309143+0000","last_fullsized":"2024-09-16T15:51:04.309143+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:43:12.283062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309922+0000","last_change":"2024-09-16T15:51:02.391709+0000","last_active":"2024-09-16T15:51:04.309922+0000","last_peered":"2024-09-16T15:51:04.309922+0000","last_clean":"2024-09-16T15:51:04.309922+0000","last_became_active":"2024-09-16T15:51:02.391379+0000","last_became_peered":"2024-09-16T15:51:02.391379+0000","last_unstale":"2024-09-16T15:51:04.309922+0000","last_undegraded":"2024-09-16T15:51:04.309922+0000","last_fullsized":"2024-09-16T15:51:04.309922+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:46:45.927933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309481+0000","last_change":"2024-09-16T15:51:01.378352+0000","last_active":"2024-09-16T15:51:04.309481+0000","last_peered":"2024-09-16T15:51:04.309481+0000","last_clean":"2024-09-16T15:51:04.309481+0000","last_became_active":"2024-09-16T15:51:01.376625+0000","last_became_peered":"2024-09-16T15:51:01.376625+0000","last_unstale":"2024-09-16T15:51:04.309481+0000","last_undegraded":"2024-09-16T15:51:04.309481+0000","last_fullsized":"2024-09-16T15:51:04.309481+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:07:26.131631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:03.380970+0000","last_change":"2024-09-16T15:51:02.397905+0000","last_active":"2024-09-16T15:51:03.380970+0000","last_peered":"2024-09-16T15:51:03.380970+0000","last_clean":"2024-09-16T15:51:03.380970+0000","last_became_active":"2024-09-16T15:51:02.397623+0000","last_became_peered":"2024-09-16T15:51:02.397623+0000","last_unstale":"2024-09-16T15:51:03.380970+0000","last_undegraded":"2024-09-16T15:51:03.380970+0000","last_fullsized":"2024-09-16T15:51:03.380970+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:36:12.989907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.10","version":"103'8","reported_seq":23,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309643+0000","last_change":"2024-09-16T15:51:01.383280+0000","last_active":"2024-09-16T15:51:04.309643+0000","last_peered":"2024-09-16T15:51:04.309643+0000","last_clean":"2024-09-16T15:51:04.309643+0000","last_became_active":"2024-09-16T15:51:01.381141+0000","last_became_peered":"2024-09-16T15:51:01.381141+0000","last_unstale":"2024-09-16T15:51:04.309643+0000","last_undegraded":"2024-09-16T15:51:04.309643+0000","last_fullsized":"2024-09-16T15:51:04.309643+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:10:44.749157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.310001+0000","last_change":"2024-09-16T15:51:02.389522+0000","last_active":"2024-09-16T15:51:04.310001+0000","last_peered":"2024-09-16T15:51:04.310001+0000","last_clean":"2024-09-16T15:51:04.310001+0000","last_became_active":"2024-09-16T15:51:02.389260+0000","last_became_peered":"2024-09-16T15:51:02.389260+0000","last_unstale":"2024-09-16T15:51:04.310001+0000","last_undegraded":"2024-09-16T15:51:04.310001+0000","last_fullsized":"2024-09-16T15:51:04.310001+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:21:10.883240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.e","version":"103'2","reported_seq":17,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309924+0000","last_change":"2024-09-16T15:51:01.391228+0000","last_active":"2024-09-16T15:51:04.309924+0000","last_peered":"2024-09-16T15:51:04.309924+0000","last_clean":"2024-09-16T15:51:04.309924+0000","last_became_active":"2024-09-16T15:51:01.390919+0000","last_became_peered":"2024-09-16T15:51:01.390919+0000","last_unstale":"2024-09-16T15:51:04.309924+0000","last_undegraded":"2024-09-16T15:51:04.309924+0000","last_fullsized":"2024-09-16T15:51:04.309924+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:28:48.291775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309186+0000","last_change":"2024-09-16T15:51:02.388901+0000","last_active":"2024-09-16T15:51:04.309186+0000","last_peered":"2024-09-16T15:51:04.309186+0000","last_clean":"2024-09-16T15:51:04.309186+0000","last_became_active":"2024-09-16T15:51:02.388312+0000","last_became_peered":"2024-09-16T15:51:02.388312+0000","last_unstale":"2024-09-16T15:51:04.309186+0000","last_undegraded":"2024-09-16T15:51:04.309186+0000","last_fullsized":"2024-09-16T15:51:04.309186+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:42:27.508189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"103'1","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309927+0000","last_change":"2024-09-16T15:51:01.377467+0000","last_active":"2024-09-16T15:51:04.309927+0000","last_peered":"2024-09-16T15:51:04.309927+0000","last_clean":"2024-09-16T15:51:04.309927+0000","last_became_active":"2024-09-16T15:51:01.377180+0000","last_became_peered":"2024-09-16T15:51:01.377180+0000","last_unstale":"2024-09-16T15:51:04.309927+0000","last_undegraded":"2024-09-16T15:51:04.309927+0000","last_fullsized":"2024-09-16T15:51:04.309927+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:15:29.048646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.310133+0000","last_change":"2024-09-16T15:51:02.401565+0000","last_active":"2024-09-16T15:51:04.310133+0000","last_peered":"2024-09-16T15:51:04.310133+0000","last_clean":"2024-09-16T15:51:04.310133+0000","last_became_active":"2024-09-16T15:51:02.401304+0000","last_became_peered":"2024-09-16T15:51:02.401304+0000","last_unstale":"2024-09-16T15:51:04.310133+0000","last_undegraded":"2024-09-16T15:51:04.310133+0000","last_fullsized":"2024-09-16T15:51:04.310133+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:36:12.989907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:03.381005+0000","last_change":"2024-09-16T15:51:01.385783+0000","last_active":"2024-09-16T15:51:03.381005+0000","last_peered":"2024-09-16T15:51:03.381005+0000","last_clean":"2024-09-16T15:51:03.381005+0000","last_became_active":"2024-09-16T15:51:01.385500+0000","last_became_peered":"2024-09-16T15:51:01.385500+0000","last_unstale":"2024-09-16T15:51:03.381005+0000","last_undegraded":"2024-09-16T15:51:03.381005+0000","last_fullsized":"2024-09-16T15:51:03.381005+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:30:12.503202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:03.379982+0000","last_change":"2024-09-16T15:51:02.399523+0000","last_active":"2024-09-16T15:51:03.379982+0000","last_peered":"2024-09-16T15:51:03.379982+0000","last_clean":"2024-09-16T15:51:03.379982+0000","last_became_active":"2024-09-16T15:51:02.399256+0000","last_became_peered":"2024-09-16T15:51:02.399256+0000","last_unstale":"2024-09-16T15:51:03.379982+0000","last_undegraded":"2024-09-16T15:51:03.379982+0000","last_fullsized":"2024-09-16T15:51:03.379982+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:59:14.234290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"103'2","reported_seq":17,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309313+0000","last_change":"2024-09-16T15:51:01.391626+0000","last_active":"2024-09-16T15:51:04.309313+0000","last_peered":"2024-09-16T15:51:04.309313+0000","last_clean":"2024-09-16T15:51:04.309313+0000","last_became_active":"2024-09-16T15:51:01.391322+0000","last_became_peered":"2024-09-16T15:51:01.391322+0000","last_unstale":"2024-09-16T15:51:04.309313+0000","last_undegraded":"2024-09-16T15:51:04.309313+0000","last_fullsized":"2024-09-16T15:51:04.309313+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:13:41.121392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:03.379709+0000","last_change":"2024-09-16T15:51:02.405123+0000","last_active":"2024-09-16T15:51:03.379709+0000","last_peered":"2024-09-16T15:51:03.379709+0000","last_clean":"2024-09-16T15:51:03.379709+0000","last_became_active":"2024-09-16T15:51:02.404678+0000","last_became_peered":"2024-09-16T15:51:02.404678+0000","last_unstale":"2024-09-16T15:51:03.379709+0000","last_undegraded":"2024-09-16T15:51:03.379709+0000","last_fullsized":"2024-09-16T15:51:03.379709+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:03:32.011151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.308961+0000","last_change":"2024-09-16T15:51:01.378559+0000","last_active":"2024-09-16T15:51:04.308961+0000","last_peered":"2024-09-16T15:51:04.308961+0000","last_clean":"2024-09-16T15:51:04.308961+0000","last_became_active":"2024-09-16T15:51:01.377521+0000","last_became_peered":"2024-09-16T15:51:01.377521+0000","last_unstale":"2024-09-16T15:51:04.308961+0000","last_undegraded":"2024-09-16T15:51:04.308961+0000","last_fullsized":"2024-09-16T15:51:04.308961+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:29:18.126495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309424+0000","last_change":"2024-09-16T15:51:02.400056+0000","last_active":"2024-09-16T15:51:04.309424+0000","last_peered":"2024-09-16T15:51:04.309424+0000","last_clean":"2024-09-16T15:51:04.309424+0000","last_became_active":"2024-09-16T15:51:02.397362+0000","last_became_peered":"2024-09-16T15:51:02.397362+0000","last_unstale":"2024-09-16T15:51:04.309424+0000","last_undegraded":"2024-09-16T15:51:04.309424+0000","last_fullsized":"2024-09-16T15:51:04.309424+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:45:49.121469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:03.378116+0000","last_change":"2024-09-16T15:51:01.372918+0000","last_active":"2024-09-16T15:51:03.378116+0000","last_peered":"2024-09-16T15:51:03.378116+0000","last_clean":"2024-09-16T15:51:03.378116+0000","last_became_active":"2024-09-16T15:51:01.372464+0000","last_became_peered":"2024-09-16T15:51:01.372464+0000","last_unstale":"2024-09-16T15:51:03.378116+0000","last_undegraded":"2024-09-16T15:51:03.378116+0000","last_fullsized":"2024-09-16T15:51:03.378116+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:54:34.716483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.310129+0000","last_change":"2024-09-16T15:51:02.406721+0000","last_active":"2024-09-16T15:51:04.310129+0000","last_peered":"2024-09-16T15:51:04.310129+0000","last_clean":"2024-09-16T15:51:04.310129+0000","last_became_active":"2024-09-16T15:51:02.405161+0000","last_became_peered":"2024-09-16T15:51:02.405161+0000","last_unstale":"2024-09-16T15:51:04.310129+0000","last_undegraded":"2024-09-16T15:51:04.310129+0000","last_fullsized":"2024-09-16T15:51:04.310129+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:14:26.465332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:03.379912+0000","last_change":"2024-09-16T15:51:01.386549+0000","last_active":"2024-09-16T15:51:03.379912+0000","last_peered":"2024-09-16T15:51:03.379912+0000","last_clean":"2024-09-16T15:51:03.379912+0000","last_became_active":"2024-09-16T15:51:01.386253+0000","last_became_peered":"2024-09-16T15:51:01.386253+0000","last_unstale":"2024-09-16T15:51:03.379912+0000","last_undegraded":"2024-09-16T15:51:03.379912+0000","last_fullsized":"2024-09-16T15:51:03.379912+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:32:44.402539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:03.381369+0000","last_change":"2024-09-16T15:51:02.396005+0000","last_active":"2024-09-16T15:51:03.381369+0000","last_peered":"2024-09-16T15:51:03.381369+0000","last_clean":"2024-09-16T15:51:03.381369+0000","last_became_active":"2024-09-16T15:51:02.395727+0000","last_became_peered":"2024-09-16T15:51:02.395727+0000","last_unstale":"2024-09-16T15:51:03.381369+0000","last_undegraded":"2024-09-16T15:51:03.381369+0000","last_fullsized":"2024-09-16T15:51:03.381369+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:28:17.080952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.7","version":"103'2","reported_seq":17,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.310547+0000","last_change":"2024-09-16T15:51:01.379987+0000","last_active":"2024-09-16T15:51:04.310547+0000","last_peered":"2024-09-16T15:51:04.310547+0000","last_clean":"2024-09-16T15:51:04.310547+0000","last_became_active":"2024-09-16T15:51:01.379714+0000","last_became_peered":"2024-09-16T15:51:01.379714+0000","last_unstale":"2024-09-16T15:51:04.310547+0000","last_undegraded":"2024-09-16T15:51:04.310547+0000","last_fullsized":"2024-09-16T15:51:04.310547+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:39:21.798976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:03.381327+0000","last_change":"2024-09-16T15:51:02.384154+0000","last_active":"2024-09-16T15:51:03.381327+0000","last_peered":"2024-09-16T15:51:03.381327+0000","last_clean":"2024-09-16T15:51:03.381327+0000","last_became_active":"2024-09-16T15:51:02.383885+0000","last_became_peered":"2024-09-16T15:51:02.383885+0000","last_unstale":"2024-09-16T15:51:03.381327+0000","last_undegraded":"2024-09-16T15:51:03.381327+0000","last_fullsized":"2024-09-16T15:51:03.381327+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:31:37.796930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.6","version":"104'1","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:03.381671+0000","last_change":"2024-09-16T15:51:01.391293+0000","last_active":"2024-09-16T15:51:03.381671+0000","last_peered":"2024-09-16T15:51:03.381671+0000","last_clean":"2024-09-16T15:51:03.381671+0000","last_became_active":"2024-09-16T15:51:01.388187+0000","last_became_peered":"2024-09-16T15:51:01.388187+0000","last_unstale":"2024-09-16T15:51:03.381671+0000","last_undegraded":"2024-09-16T15:51:03.381671+0000","last_fullsized":"2024-09-16T15:51:03.381671+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:00:19.592466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:03.380574+0000","last_change":"2024-09-16T15:51:02.399564+0000","last_active":"2024-09-16T15:51:03.380574+0000","last_peered":"2024-09-16T15:51:03.380574+0000","last_clean":"2024-09-16T15:51:03.380574+0000","last_became_active":"2024-09-16T15:51:02.397436+0000","last_became_peered":"2024-09-16T15:51:02.397436+0000","last_unstale":"2024-09-16T15:51:03.380574+0000","last_undegraded":"2024-09-16T15:51:03.380574+0000","last_fullsized":"2024-09-16T15:51:03.380574+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:57:33.960661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:03.381184+0000","last_change":"2024-09-16T15:51:01.381796+0000","last_active":"2024-09-16T15:51:03.381184+0000","last_peered":"2024-09-16T15:51:03.381184+0000","last_clean":"2024-09-16T15:51:03.381184+0000","last_became_active":"2024-09-16T15:51:01.381524+0000","last_became_peered":"2024-09-16T15:51:01.381524+0000","last_unstale":"2024-09-16T15:51:03.381184+0000","last_undegraded":"2024-09-16T15:51:03.381184+0000","last_fullsized":"2024-09-16T15:51:03.381184+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:03:22.300390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309648+0000","last_change":"2024-09-16T15:51:02.389579+0000","last_active":"2024-09-16T15:51:04.309648+0000","last_peered":"2024-09-16T15:51:04.309648+0000","last_clean":"2024-09-16T15:51:04.309648+0000","last_became_active":"2024-09-16T15:51:02.389279+0000","last_became_peered":"2024-09-16T15:51:02.389279+0000","last_unstale":"2024-09-16T15:51:04.309648+0000","last_undegraded":"2024-09-16T15:51:04.309648+0000","last_fullsized":"2024-09-16T15:51:04.309648+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:32:17.641313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309944+0000","last_change":"2024-09-16T15:51:01.380670+0000","last_active":"2024-09-16T15:51:04.309944+0000","last_peered":"2024-09-16T15:51:04.309944+0000","last_clean":"2024-09-16T15:51:04.309944+0000","last_became_active":"2024-09-16T15:51:01.380213+0000","last_became_peered":"2024-09-16T15:51:01.380213+0000","last_unstale":"2024-09-16T15:51:04.309944+0000","last_undegraded":"2024-09-16T15:51:04.309944+0000","last_fullsized":"2024-09-16T15:51:04.309944+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:07:53.419216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:03.379649+0000","last_change":"2024-09-16T15:51:02.398432+0000","last_active":"2024-09-16T15:51:03.379649+0000","last_peered":"2024-09-16T15:51:03.379649+0000","last_clean":"2024-09-16T15:51:03.379649+0000","last_became_active":"2024-09-16T15:51:02.397631+0000","last_became_peered":"2024-09-16T15:51:02.397631+0000","last_unstale":"2024-09-16T15:51:03.379649+0000","last_undegraded":"2024-09-16T15:51:03.379649+0000","last_fullsized":"2024-09-16T15:51:03.379649+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:52:13.451127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:03.380697+0000","last_change":"2024-09-16T15:51:01.388916+0000","last_active":"2024-09-16T15:51:03.380697+0000","last_peered":"2024-09-16T15:51:03.380697+0000","last_clean":"2024-09-16T15:51:03.380697+0000","last_became_active":"2024-09-16T15:51:01.388644+0000","last_became_peered":"2024-09-16T15:51:01.388644+0000","last_unstale":"2024-09-16T15:51:03.380697+0000","last_undegraded":"2024-09-16T15:51:03.380697+0000","last_fullsized":"2024-09-16T15:51:03.380697+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:32:29.346680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309492+0000","last_change":"2024-09-16T15:51:02.401892+0000","last_active":"2024-09-16T15:51:04.309492+0000","last_peered":"2024-09-16T15:51:04.309492+0000","last_clean":"2024-09-16T15:51:04.309492+0000","last_became_active":"2024-09-16T15:51:02.401633+0000","last_became_peered":"2024-09-16T15:51:02.401633+0000","last_unstale":"2024-09-16T15:51:04.309492+0000","last_undegraded":"2024-09-16T15:51:04.309492+0000","last_fullsized":"2024-09-16T15:51:04.309492+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:34:03.971706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309051+0000","last_change":"2024-09-16T15:51:01.387695+0000","last_active":"2024-09-16T15:51:04.309051+0000","last_peered":"2024-09-16T15:51:04.309051+0000","last_clean":"2024-09-16T15:51:04.309051+0000","last_became_active":"2024-09-16T15:51:01.387455+0000","last_became_peered":"2024-09-16T15:51:01.387455+0000","last_unstale":"2024-09-16T15:51:04.309051+0000","last_undegraded":"2024-09-16T15:51:04.309051+0000","last_fullsized":"2024-09-16T15:51:04.309051+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:28:29.328131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:03.380005+0000","last_change":"2024-09-16T15:51:02.389201+0000","last_active":"2024-09-16T15:51:03.380005+0000","last_peered":"2024-09-16T15:51:03.380005+0000","last_clean":"2024-09-16T15:51:03.380005+0000","last_became_active":"2024-09-16T15:51:02.388954+0000","last_became_peered":"2024-09-16T15:51:02.388954+0000","last_unstale":"2024-09-16T15:51:03.380005+0000","last_undegraded":"2024-09-16T15:51:03.380005+0000","last_fullsized":"2024-09-16T15:51:03.380005+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:02:03.707466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.308686+0000","last_change":"2024-09-16T15:51:01.384206+0000","last_active":"2024-09-16T15:51:04.308686+0000","last_peered":"2024-09-16T15:51:04.308686+0000","last_clean":"2024-09-16T15:51:04.308686+0000","last_became_active":"2024-09-16T15:51:01.383873+0000","last_became_peered":"2024-09-16T15:51:01.383873+0000","last_unstale":"2024-09-16T15:51:04.308686+0000","last_undegraded":"2024-09-16T15:51:04.308686+0000","last_fullsized":"2024-09-16T15:51:04.308686+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:15:03.468370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.308953+0000","last_change":"2024-09-16T15:51:02.392882+0000","last_active":"2024-09-16T15:51:04.308953+0000","last_peered":"2024-09-16T15:51:04.308953+0000","last_clean":"2024-09-16T15:51:04.308953+0000","last_became_active":"2024-09-16T15:51:02.392610+0000","last_became_peered":"2024-09-16T15:51:02.392610+0000","last_unstale":"2024-09-16T15:51:04.308953+0000","last_undegraded":"2024-09-16T15:51:04.308953+0000","last_fullsized":"2024-09-16T15:51:04.308953+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:57:33.960661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:03.380013+0000","last_change":"2024-09-16T15:51:01.390952+0000","last_active":"2024-09-16T15:51:03.380013+0000","last_peered":"2024-09-16T15:51:03.380013+0000","last_clean":"2024-09-16T15:51:03.380013+0000","last_became_active":"2024-09-16T15:51:01.390640+0000","last_became_peered":"2024-09-16T15:51:01.390640+0000","last_unstale":"2024-09-16T15:51:03.380013+0000","last_undegraded":"2024-09-16T15:51:03.380013+0000","last_fullsized":"2024-09-16T15:51:03.380013+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:29:00.354401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:03.381424+0000","last_change":"2024-09-16T15:51:02.404581+0000","last_active":"2024-09-16T15:51:03.381424+0000","last_peered":"2024-09-16T15:51:03.381424+0000","last_clean":"2024-09-16T15:51:03.381424+0000","last_became_active":"2024-09-16T15:51:02.404256+0000","last_became_peered":"2024-09-16T15:51:02.404256+0000","last_unstale":"2024-09-16T15:51:03.381424+0000","last_undegraded":"2024-09-16T15:51:03.381424+0000","last_fullsized":"2024-09-16T15:51:03.381424+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:57:45.368832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.f","version":"103'1","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.310516+0000","last_change":"2024-09-16T15:51:01.388328+0000","last_active":"2024-09-16T15:51:04.310516+0000","last_peered":"2024-09-16T15:51:04.310516+0000","last_clean":"2024-09-16T15:51:04.310516+0000","last_became_active":"2024-09-16T15:51:01.388066+0000","last_became_peered":"2024-09-16T15:51:01.388066+0000","last_unstale":"2024-09-16T15:51:04.310516+0000","last_undegraded":"2024-09-16T15:51:04.310516+0000","last_fullsized":"2024-09-16T15:51:04.310516+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:46:58.289338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":266,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.308775+0000","last_change":"2024-09-16T15:39:12.698813+0000","last_active":"2024-09-16T15:51:04.308775+0000","last_peered":"2024-09-16T15:51:04.308775+0000","last_clean":"2024-09-16T15:51:04.308775+0000","last_became_active":"2024-09-16T15:39:12.697695+0000","last_became_peered":"2024-09-16T15:39:12.697695+0000","last_unstale":"2024-09-16T15:51:04.308775+0000","last_undegraded":"2024-09-16T15:51:04.308775+0000","last_fullsized":"2024-09-16T15:51:04.308775+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_clean_scrub_stamp":"2024-09-16T15:39:11.684967+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:02.061081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509588,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":148984,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222856,"statfs":{"total":96636764160,"available":96484204544,"internally_reserved":0,"allocated":1277952,"data_stored":557788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":151255496},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509587,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131540,"kb_used_data":1848,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240300,"statfs":{"total":96636764160,"available":96502067200,"internally_reserved":0,"allocated":1892352,"data_stored":1149824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509586,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144832,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94227008,"statfs":{"total":96636764160,"available":96488456192,"internally_reserved":0,"allocated":1282048,"data_stored":558370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509586,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126788,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94245052,"statfs":{"total":96636764160,"available":96506933248,"internally_reserved":0,"allocated":1290240,"data_stored":557982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":128514505},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509586,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126776,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94245064,"statfs":{"total":96636764160,"available":96506945536,"internally_reserved":0,"allocated":1282048,"data_stored":557858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":128514503},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509587,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123212,"kb_used_data":1848,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248628,"statfs":{"total":96636764160,"available":96510595072,"internally_reserved":0,"allocated":1892352,"data_stored":1149880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":124254663},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509585,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":140664,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94231176,"statfs":{"total":96636764160,"available":96492724224,"internally_reserved":0,"allocated":1277952,"data_stored":557788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":142735815},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542292,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122620,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94249220,"statfs":{"total":96636764160,"available":96511201280,"internally_reserved":0,"allocated":1282048,"data_stored":558370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":124254668},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:51:08.354 INFO:tasks.ceph.mds.c.smithi151.stderr:2024-09-16T15:51:08.353+0000 7f0e82e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-16T15:51:11.301 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:51:11.471 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.468+0000 7f0ddce00640 1 -- 172.21.15.151:0/1971414686 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0dd80ccfd0 msgr2=0x7f0dd8072ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:11.471 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.468+0000 7f0ddce00640 1 --2- 172.21.15.151:0/1971414686 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0dd80ccfd0 0x7f0dd8072ee0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f0dd0009ca0 tx=0x7f0dd002ab50 comp rx=0 tx=0).stop 2024-09-16T15:51:11.471 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.469+0000 7f0ddce00640 1 -- 172.21.15.151:0/1971414686 shutdown_connections 2024-09-16T15:51:11.471 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.469+0000 7f0ddce00640 1 --2- 172.21.15.151:0/1971414686 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0dd80ccfd0 0x7f0dd8072ee0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:11.471 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.469+0000 7f0ddce00640 1 --2- 172.21.15.151:0/1971414686 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0dd80cf300 0x7f0dd80729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:11.471 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.469+0000 7f0ddce00640 1 -- 172.21.15.151:0/1971414686 >> 172.21.15.151:0/1971414686 conn(0x7f0dd80b1950 msgr2=0x7f0dd80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:11.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.469+0000 7f0ddce00640 1 -- 172.21.15.151:0/1971414686 shutdown_connections 2024-09-16T15:51:11.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.469+0000 7f0ddce00640 1 -- 172.21.15.151:0/1971414686 wait complete. 2024-09-16T15:51:11.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.470+0000 7f0ddce00640 1 Processor -- start 2024-09-16T15:51:11.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.470+0000 7f0ddce00640 1 -- start start 2024-09-16T15:51:11.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.471+0000 7f0ddce00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0dd80ccfd0 0x7f0dd80b0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:11.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.471+0000 7f0ddce00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0dd80cf300 0x7f0dd80b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:11.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.471+0000 7f0ddce00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0dd81445d0 0x7f0dd81f5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:11.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.471+0000 7f0ddce00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd8073420 con 0x7f0dd80ccfd0 2024-09-16T15:51:11.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.471+0000 7f0ddce00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd80b4eb0 con 0x7f0dd81445d0 2024-09-16T15:51:11.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.471+0000 7f0ddce00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f0dd80735a0 con 0x7f0dd80cf300 2024-09-16T15:51:11.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.471+0000 7f0dd5e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0dd80ccfd0 0x7f0dd80b0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:11.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.472+0000 7f0dd5e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0dd80ccfd0 0x7f0dd80b0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52714/0 (socket says 172.21.15.151:52714) 2024-09-16T15:51:11.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.472+0000 7f0dd5e00640 1 -- 172.21.15.151:0/1039037677 learned_addr learned my addr 172.21.15.151:0/1039037677 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:51:11.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.472+0000 7f0dd6800640 1 --2- 172.21.15.151:0/1039037677 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0dd81445d0 0x7f0dd81f5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:11.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.472+0000 7f0dd5400640 1 --2- 172.21.15.151:0/1039037677 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0dd80cf300 0x7f0dd80b0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:11.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.472+0000 7f0dd5e00640 1 -- 172.21.15.151:0/1039037677 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0dd80cf300 msgr2=0x7f0dd80b0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:11.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.472+0000 7f0dd5e00640 1 --2- 172.21.15.151:0/1039037677 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0dd80cf300 0x7f0dd80b0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:11.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.472+0000 7f0dd5e00640 1 -- 172.21.15.151:0/1039037677 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0dd81445d0 msgr2=0x7f0dd81f5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:11.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.472+0000 7f0dd5e00640 1 --2- 172.21.15.151:0/1039037677 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0dd81445d0 0x7f0dd81f5ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:11.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.472+0000 7f0dd5e00640 1 -- 172.21.15.151:0/1039037677 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dd81f29c0 con 0x7f0dd80ccfd0 2024-09-16T15:51:11.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.472+0000 7f0dd6800640 1 --2- 172.21.15.151:0/1039037677 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0dd81445d0 0x7f0dd81f5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:51:11.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.472+0000 7f0dd5e00640 1 --2- 172.21.15.151:0/1039037677 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0dd80ccfd0 0x7f0dd80b0910 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f0dc800c3c0 tx=0x7f0dc800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:11.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.472+0000 7f0dce000640 1 -- 172.21.15.151:0/1039037677 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc8017070 con 0x7f0dd80ccfd0 2024-09-16T15:51:11.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.472+0000 7f0dce000640 1 -- 172.21.15.151:0/1039037677 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0dc80035f0 con 0x7f0dd80ccfd0 2024-09-16T15:51:11.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.472+0000 7f0dce000640 1 -- 172.21.15.151:0/1039037677 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc8003a80 con 0x7f0dd80ccfd0 2024-09-16T15:51:11.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.473+0000 7f0ddce00640 1 -- 172.21.15.151:0/1039037677 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dd81f2c60 con 0x7f0dd80ccfd0 2024-09-16T15:51:11.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.473+0000 7f0ddce00640 1 -- 172.21.15.151:0/1039037677 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dd81f3220 con 0x7f0dd80ccfd0 2024-09-16T15:51:11.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.474+0000 7f0ddce00640 1 -- 172.21.15.151:0/1039037677 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dd8073a00 con 0x7f0dd80ccfd0 2024-09-16T15:51:11.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.475+0000 7f0dce000640 1 -- 172.21.15.151:0/1039037677 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7f0dc801a070 con 0x7f0dd80ccfd0 2024-09-16T15:51:11.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.476+0000 7f0dce000640 1 --2- 172.21.15.151:0/1039037677 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0db8079ed0 0x7f0db807c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:11.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.476+0000 7f0dce000640 1 -- 172.21.15.151:0/1039037677 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5765+0+0 (secure 0 0 0) 0x7f0dc8098970 con 0x7f0dd80ccfd0 2024-09-16T15:51:11.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.476+0000 7f0dd5400640 1 --2- 172.21.15.151:0/1039037677 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0db8079ed0 0x7f0db807c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:11.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.477+0000 7f0dd5400640 1 --2- 172.21.15.151:0/1039037677 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0db8079ed0 0x7f0db807c390 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f0dd80cd780 tx=0x7f0dc4005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:11.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.480+0000 7f0dce000640 1 -- 172.21.15.151:0/1039037677 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0dc8062a40 con 0x7f0dd80ccfd0 2024-09-16T15:51:11.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.663+0000 7f0ddce00640 1 -- 172.21.15.151:0/1039037677 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0dd80ce690 con 0x7f0db8079ed0 2024-09-16T15:51:11.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.668+0000 7f0dce000640 1 -- 172.21.15.151:0/1039037677 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252499 (secure 0 0 0) 0x7f0dd80ce690 con 0x7f0db8079ed0 2024-09-16T15:51:11.673 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:51:11.674 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-09-16T15:51:11.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.671+0000 7f0ddce00640 1 -- 172.21.15.151:0/1039037677 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0db8079ed0 msgr2=0x7f0db807c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:11.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.671+0000 7f0ddce00640 1 --2- 172.21.15.151:0/1039037677 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0db8079ed0 0x7f0db807c390 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f0dd80cd780 tx=0x7f0dc4005f90 comp rx=0 tx=0).stop 2024-09-16T15:51:11.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.671+0000 7f0ddce00640 1 -- 172.21.15.151:0/1039037677 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0dd80ccfd0 msgr2=0x7f0dd80b0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:11.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.671+0000 7f0ddce00640 1 --2- 172.21.15.151:0/1039037677 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0dd80ccfd0 0x7f0dd80b0910 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f0dc800c3c0 tx=0x7f0dc800c880 comp rx=0 tx=0).stop 2024-09-16T15:51:11.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.672+0000 7f0ddce00640 1 -- 172.21.15.151:0/1039037677 shutdown_connections 2024-09-16T15:51:11.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.672+0000 7f0ddce00640 1 --2- 172.21.15.151:0/1039037677 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0db8079ed0 0x7f0db807c390 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:11.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.672+0000 7f0ddce00640 1 --2- 172.21.15.151:0/1039037677 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0dd81445d0 0x7f0dd81f5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:11.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.672+0000 7f0ddce00640 1 --2- 172.21.15.151:0/1039037677 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0dd80cf300 0x7f0dd80b0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:11.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.672+0000 7f0ddce00640 1 --2- 172.21.15.151:0/1039037677 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0dd80ccfd0 0x7f0dd80b0910 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:11.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.672+0000 7f0ddce00640 1 -- 172.21.15.151:0/1039037677 >> 172.21.15.151:0/1039037677 conn(0x7f0dd80b1950 msgr2=0x7f0dd81420f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:11.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.672+0000 7f0ddce00640 1 -- 172.21.15.151:0/1039037677 shutdown_connections 2024-09-16T15:51:11.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.672+0000 7f0ddce00640 1 -- 172.21.15.151:0/1039037677 wait complete. 2024-09-16T15:51:11.689 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":457,"stamp":"2024-09-16T15:51:09.881637+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":168,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1065416,"kb_used_data":11208,"kb_used_omap":12,"kb_used_meta":1054003,"kb_avail":753909304,"statfs":{"total":773094113280,"available":772003127296,"internally_reserved":0,"allocated":11476992,"data_stored":5647860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12738,"internal_metadata":1079299646},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002306"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.310177+0000","last_change":"2024-09-16T15:51:02.386431+0000","last_active":"2024-09-16T15:51:04.310177+0000","last_peered":"2024-09-16T15:51:04.310177+0000","last_clean":"2024-09-16T15:51:04.310177+0000","last_became_active":"2024-09-16T15:51:02.386126+0000","last_became_peered":"2024-09-16T15:51:02.386126+0000","last_unstale":"2024-09-16T15:51:04.310177+0000","last_undegraded":"2024-09-16T15:51:04.310177+0000","last_fullsized":"2024-09-16T15:51:04.310177+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:21.600849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309583+0000","last_change":"2024-09-16T15:51:02.388621+0000","last_active":"2024-09-16T15:51:04.309583+0000","last_peered":"2024-09-16T15:51:04.309583+0000","last_clean":"2024-09-16T15:51:04.309583+0000","last_became_active":"2024-09-16T15:51:02.388294+0000","last_became_peered":"2024-09-16T15:51:02.388294+0000","last_unstale":"2024-09-16T15:51:04.309583+0000","last_undegraded":"2024-09-16T15:51:04.309583+0000","last_fullsized":"2024-09-16T15:51:04.309583+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:44:02.181311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:03.381077+0000","last_change":"2024-09-16T15:51:02.396532+0000","last_active":"2024-09-16T15:51:03.381077+0000","last_peered":"2024-09-16T15:51:03.381077+0000","last_clean":"2024-09-16T15:51:03.381077+0000","last_became_active":"2024-09-16T15:51:02.396235+0000","last_became_peered":"2024-09-16T15:51:02.396235+0000","last_unstale":"2024-09-16T15:51:03.381077+0000","last_undegraded":"2024-09-16T15:51:03.381077+0000","last_fullsized":"2024-09-16T15:51:03.381077+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:39:19.449312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309522+0000","last_change":"2024-09-16T15:51:02.385219+0000","last_active":"2024-09-16T15:51:04.309522+0000","last_peered":"2024-09-16T15:51:04.309522+0000","last_clean":"2024-09-16T15:51:04.309522+0000","last_became_active":"2024-09-16T15:51:02.384310+0000","last_became_peered":"2024-09-16T15:51:02.384310+0000","last_unstale":"2024-09-16T15:51:04.309522+0000","last_undegraded":"2024-09-16T15:51:04.309522+0000","last_fullsized":"2024-09-16T15:51:04.309522+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:24:11.153158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309076+0000","last_change":"2024-09-16T15:51:02.397330+0000","last_active":"2024-09-16T15:51:04.309076+0000","last_peered":"2024-09-16T15:51:04.309076+0000","last_clean":"2024-09-16T15:51:04.309076+0000","last_became_active":"2024-09-16T15:51:02.397058+0000","last_became_peered":"2024-09-16T15:51:02.397058+0000","last_unstale":"2024-09-16T15:51:04.309076+0000","last_undegraded":"2024-09-16T15:51:04.309076+0000","last_fullsized":"2024-09-16T15:51:04.309076+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:21:04.406320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.310287+0000","last_change":"2024-09-16T15:51:02.403013+0000","last_active":"2024-09-16T15:51:04.310287+0000","last_peered":"2024-09-16T15:51:04.310287+0000","last_clean":"2024-09-16T15:51:04.310287+0000","last_became_active":"2024-09-16T15:51:02.402778+0000","last_became_peered":"2024-09-16T15:51:02.402778+0000","last_unstale":"2024-09-16T15:51:04.310287+0000","last_undegraded":"2024-09-16T15:51:04.310287+0000","last_fullsized":"2024-09-16T15:51:04.310287+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:27:46.534033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.308977+0000","last_change":"2024-09-16T15:51:02.389718+0000","last_active":"2024-09-16T15:51:04.308977+0000","last_peered":"2024-09-16T15:51:04.308977+0000","last_clean":"2024-09-16T15:51:04.308977+0000","last_became_active":"2024-09-16T15:51:02.386922+0000","last_became_peered":"2024-09-16T15:51:02.386922+0000","last_unstale":"2024-09-16T15:51:04.308977+0000","last_undegraded":"2024-09-16T15:51:04.308977+0000","last_fullsized":"2024-09-16T15:51:04.308977+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:40:46.193203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:03.379704+0000","last_change":"2024-09-16T15:51:02.392522+0000","last_active":"2024-09-16T15:51:03.379704+0000","last_peered":"2024-09-16T15:51:03.379704+0000","last_clean":"2024-09-16T15:51:03.379704+0000","last_became_active":"2024-09-16T15:51:02.392253+0000","last_became_peered":"2024-09-16T15:51:02.392253+0000","last_unstale":"2024-09-16T15:51:03.379704+0000","last_undegraded":"2024-09-16T15:51:03.379704+0000","last_fullsized":"2024-09-16T15:51:03.379704+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:10:47.555331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309588+0000","last_change":"2024-09-16T15:51:02.397324+0000","last_active":"2024-09-16T15:51:04.309588+0000","last_peered":"2024-09-16T15:51:04.309588+0000","last_clean":"2024-09-16T15:51:04.309588+0000","last_became_active":"2024-09-16T15:51:02.397074+0000","last_became_peered":"2024-09-16T15:51:02.397074+0000","last_unstale":"2024-09-16T15:51:04.309588+0000","last_undegraded":"2024-09-16T15:51:04.309588+0000","last_fullsized":"2024-09-16T15:51:04.309588+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:19:02.723695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:03.378265+0000","last_change":"2024-09-16T15:51:02.385658+0000","last_active":"2024-09-16T15:51:03.378265+0000","last_peered":"2024-09-16T15:51:03.378265+0000","last_clean":"2024-09-16T15:51:03.378265+0000","last_became_active":"2024-09-16T15:51:02.384691+0000","last_became_peered":"2024-09-16T15:51:02.384691+0000","last_unstale":"2024-09-16T15:51:03.378265+0000","last_undegraded":"2024-09-16T15:51:03.378265+0000","last_fullsized":"2024-09-16T15:51:03.378265+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:21:46.321024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309172+0000","last_change":"2024-09-16T15:51:02.406193+0000","last_active":"2024-09-16T15:51:04.309172+0000","last_peered":"2024-09-16T15:51:04.309172+0000","last_clean":"2024-09-16T15:51:04.309172+0000","last_became_active":"2024-09-16T15:51:02.402890+0000","last_became_peered":"2024-09-16T15:51:02.402890+0000","last_unstale":"2024-09-16T15:51:04.309172+0000","last_undegraded":"2024-09-16T15:51:04.309172+0000","last_fullsized":"2024-09-16T15:51:04.309172+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:31:52.533744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.310084+0000","last_change":"2024-09-16T15:51:02.393952+0000","last_active":"2024-09-16T15:51:04.310084+0000","last_peered":"2024-09-16T15:51:04.310084+0000","last_clean":"2024-09-16T15:51:04.310084+0000","last_became_active":"2024-09-16T15:51:02.393190+0000","last_became_peered":"2024-09-16T15:51:02.393190+0000","last_unstale":"2024-09-16T15:51:04.310084+0000","last_undegraded":"2024-09-16T15:51:04.310084+0000","last_fullsized":"2024-09-16T15:51:04.310084+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:56:55.816696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309716+0000","last_change":"2024-09-16T15:51:02.407269+0000","last_active":"2024-09-16T15:51:04.309716+0000","last_peered":"2024-09-16T15:51:04.309716+0000","last_clean":"2024-09-16T15:51:04.309716+0000","last_became_active":"2024-09-16T15:51:02.406943+0000","last_became_peered":"2024-09-16T15:51:02.406943+0000","last_unstale":"2024-09-16T15:51:04.309716+0000","last_undegraded":"2024-09-16T15:51:04.309716+0000","last_fullsized":"2024-09-16T15:51:04.309716+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:29:33.561032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.308960+0000","last_change":"2024-09-16T15:51:02.398129+0000","last_active":"2024-09-16T15:51:04.308960+0000","last_peered":"2024-09-16T15:51:04.308960+0000","last_clean":"2024-09-16T15:51:04.308960+0000","last_became_active":"2024-09-16T15:51:02.397834+0000","last_became_peered":"2024-09-16T15:51:02.397834+0000","last_unstale":"2024-09-16T15:51:04.308960+0000","last_undegraded":"2024-09-16T15:51:04.308960+0000","last_fullsized":"2024-09-16T15:51:04.308960+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:15:15.665992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309257+0000","last_change":"2024-09-16T15:51:02.396682+0000","last_active":"2024-09-16T15:51:04.309257+0000","last_peered":"2024-09-16T15:51:04.309257+0000","last_clean":"2024-09-16T15:51:04.309257+0000","last_became_active":"2024-09-16T15:51:02.396412+0000","last_became_peered":"2024-09-16T15:51:02.396412+0000","last_unstale":"2024-09-16T15:51:04.309257+0000","last_undegraded":"2024-09-16T15:51:04.309257+0000","last_fullsized":"2024-09-16T15:51:04.309257+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:56:52.705232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309717+0000","last_change":"2024-09-16T15:51:02.396610+0000","last_active":"2024-09-16T15:51:04.309717+0000","last_peered":"2024-09-16T15:51:04.309717+0000","last_clean":"2024-09-16T15:51:04.309717+0000","last_became_active":"2024-09-16T15:51:02.396178+0000","last_became_peered":"2024-09-16T15:51:02.396178+0000","last_unstale":"2024-09-16T15:51:04.309717+0000","last_undegraded":"2024-09-16T15:51:04.309717+0000","last_fullsized":"2024-09-16T15:51:04.309717+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:27:42.235783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:03.381293+0000","last_change":"2024-09-16T15:51:02.400919+0000","last_active":"2024-09-16T15:51:03.381293+0000","last_peered":"2024-09-16T15:51:03.381293+0000","last_clean":"2024-09-16T15:51:03.381293+0000","last_became_active":"2024-09-16T15:51:02.398907+0000","last_became_peered":"2024-09-16T15:51:02.398907+0000","last_unstale":"2024-09-16T15:51:03.381293+0000","last_undegraded":"2024-09-16T15:51:03.381293+0000","last_fullsized":"2024-09-16T15:51:03.381293+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:10:06.599715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.310010+0000","last_change":"2024-09-16T15:51:02.399241+0000","last_active":"2024-09-16T15:51:04.310010+0000","last_peered":"2024-09-16T15:51:04.310010+0000","last_clean":"2024-09-16T15:51:04.310010+0000","last_became_active":"2024-09-16T15:51:02.398973+0000","last_became_peered":"2024-09-16T15:51:02.398973+0000","last_unstale":"2024-09-16T15:51:04.310010+0000","last_undegraded":"2024-09-16T15:51:04.310010+0000","last_fullsized":"2024-09-16T15:51:04.310010+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:21:46.321024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:03.381167+0000","last_change":"2024-09-16T15:51:02.385117+0000","last_active":"2024-09-16T15:51:03.381167+0000","last_peered":"2024-09-16T15:51:03.381167+0000","last_clean":"2024-09-16T15:51:03.381167+0000","last_became_active":"2024-09-16T15:51:02.384813+0000","last_became_peered":"2024-09-16T15:51:02.384813+0000","last_unstale":"2024-09-16T15:51:03.381167+0000","last_undegraded":"2024-09-16T15:51:03.381167+0000","last_fullsized":"2024-09-16T15:51:03.381167+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:21:16.736819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.310289+0000","last_change":"2024-09-16T15:51:02.404614+0000","last_active":"2024-09-16T15:51:04.310289+0000","last_peered":"2024-09-16T15:51:04.310289+0000","last_clean":"2024-09-16T15:51:04.310289+0000","last_became_active":"2024-09-16T15:51:02.399859+0000","last_became_peered":"2024-09-16T15:51:02.399859+0000","last_unstale":"2024-09-16T15:51:04.310289+0000","last_undegraded":"2024-09-16T15:51:04.310289+0000","last_fullsized":"2024-09-16T15:51:04.310289+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:39:19.449312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:03.380958+0000","last_change":"2024-09-16T15:51:02.393101+0000","last_active":"2024-09-16T15:51:03.380958+0000","last_peered":"2024-09-16T15:51:03.380958+0000","last_clean":"2024-09-16T15:51:03.380958+0000","last_became_active":"2024-09-16T15:51:02.392793+0000","last_became_peered":"2024-09-16T15:51:02.392793+0000","last_unstale":"2024-09-16T15:51:03.380958+0000","last_undegraded":"2024-09-16T15:51:03.380958+0000","last_fullsized":"2024-09-16T15:51:03.380958+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:28:04.550380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:03.379789+0000","last_change":"2024-09-16T15:51:02.398357+0000","last_active":"2024-09-16T15:51:03.379789+0000","last_peered":"2024-09-16T15:51:03.379789+0000","last_clean":"2024-09-16T15:51:03.379789+0000","last_became_active":"2024-09-16T15:51:02.398089+0000","last_became_peered":"2024-09-16T15:51:02.398089+0000","last_unstale":"2024-09-16T15:51:03.379789+0000","last_undegraded":"2024-09-16T15:51:03.379789+0000","last_fullsized":"2024-09-16T15:51:03.379789+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:43:25.312945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309505+0000","last_change":"2024-09-16T15:51:02.399544+0000","last_active":"2024-09-16T15:51:04.309505+0000","last_peered":"2024-09-16T15:51:04.309505+0000","last_clean":"2024-09-16T15:51:04.309505+0000","last_became_active":"2024-09-16T15:51:02.396515+0000","last_became_peered":"2024-09-16T15:51:02.396515+0000","last_unstale":"2024-09-16T15:51:04.309505+0000","last_undegraded":"2024-09-16T15:51:04.309505+0000","last_fullsized":"2024-09-16T15:51:04.309505+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:54:35.722237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.308818+0000","last_change":"2024-09-16T15:51:02.389471+0000","last_active":"2024-09-16T15:51:04.308818+0000","last_peered":"2024-09-16T15:51:04.308818+0000","last_clean":"2024-09-16T15:51:04.308818+0000","last_became_active":"2024-09-16T15:51:02.389208+0000","last_became_peered":"2024-09-16T15:51:02.389208+0000","last_unstale":"2024-09-16T15:51:04.308818+0000","last_undegraded":"2024-09-16T15:51:04.308818+0000","last_fullsized":"2024-09-16T15:51:04.308818+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:41:29.520138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:03.378481+0000","last_change":"2024-09-16T15:51:02.391207+0000","last_active":"2024-09-16T15:51:03.378481+0000","last_peered":"2024-09-16T15:51:03.378481+0000","last_clean":"2024-09-16T15:51:03.378481+0000","last_became_active":"2024-09-16T15:51:02.390909+0000","last_became_peered":"2024-09-16T15:51:02.390909+0000","last_unstale":"2024-09-16T15:51:03.378481+0000","last_undegraded":"2024-09-16T15:51:03.378481+0000","last_fullsized":"2024-09-16T15:51:03.378481+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:21.600849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309644+0000","last_change":"2024-09-16T15:51:02.401422+0000","last_active":"2024-09-16T15:51:04.309644+0000","last_peered":"2024-09-16T15:51:04.309644+0000","last_clean":"2024-09-16T15:51:04.309644+0000","last_became_active":"2024-09-16T15:51:02.400804+0000","last_became_peered":"2024-09-16T15:51:02.400804+0000","last_unstale":"2024-09-16T15:51:04.309644+0000","last_undegraded":"2024-09-16T15:51:04.309644+0000","last_fullsized":"2024-09-16T15:51:04.309644+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:54:09.898078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309724+0000","last_change":"2024-09-16T15:51:02.404620+0000","last_active":"2024-09-16T15:51:04.309724+0000","last_peered":"2024-09-16T15:51:04.309724+0000","last_clean":"2024-09-16T15:51:04.309724+0000","last_became_active":"2024-09-16T15:51:02.404361+0000","last_became_peered":"2024-09-16T15:51:02.404361+0000","last_unstale":"2024-09-16T15:51:04.309724+0000","last_undegraded":"2024-09-16T15:51:04.309724+0000","last_fullsized":"2024-09-16T15:51:04.309724+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:00:20.598220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.308994+0000","last_change":"2024-09-16T15:51:02.403010+0000","last_active":"2024-09-16T15:51:04.308994+0000","last_peered":"2024-09-16T15:51:04.308994+0000","last_clean":"2024-09-16T15:51:04.308994+0000","last_became_active":"2024-09-16T15:51:02.402725+0000","last_became_peered":"2024-09-16T15:51:02.402725+0000","last_unstale":"2024-09-16T15:51:04.308994+0000","last_undegraded":"2024-09-16T15:51:04.308994+0000","last_fullsized":"2024-09-16T15:51:04.308994+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:30:13.508956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309979+0000","last_change":"2024-09-16T15:51:02.405833+0000","last_active":"2024-09-16T15:51:04.309979+0000","last_peered":"2024-09-16T15:51:04.309979+0000","last_clean":"2024-09-16T15:51:04.309979+0000","last_became_active":"2024-09-16T15:51:02.404364+0000","last_became_peered":"2024-09-16T15:51:02.404364+0000","last_unstale":"2024-09-16T15:51:04.309979+0000","last_undegraded":"2024-09-16T15:51:04.309979+0000","last_fullsized":"2024-09-16T15:51:04.309979+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:47:54.421341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309062+0000","last_change":"2024-09-16T15:51:02.402965+0000","last_active":"2024-09-16T15:51:04.309062+0000","last_peered":"2024-09-16T15:51:04.309062+0000","last_clean":"2024-09-16T15:51:04.309062+0000","last_became_active":"2024-09-16T15:51:02.401107+0000","last_became_peered":"2024-09-16T15:51:02.401107+0000","last_unstale":"2024-09-16T15:51:04.309062+0000","last_undegraded":"2024-09-16T15:51:04.309062+0000","last_fullsized":"2024-09-16T15:51:04.309062+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:24:38.932170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:03.380690+0000","last_change":"2024-09-16T15:51:02.392272+0000","last_active":"2024-09-16T15:51:03.380690+0000","last_peered":"2024-09-16T15:51:03.380690+0000","last_clean":"2024-09-16T15:51:03.380690+0000","last_became_active":"2024-09-16T15:51:02.392008+0000","last_became_peered":"2024-09-16T15:51:02.392008+0000","last_unstale":"2024-09-16T15:51:03.380690+0000","last_undegraded":"2024-09-16T15:51:03.380690+0000","last_fullsized":"2024-09-16T15:51:03.380690+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:56:55.816696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309158+0000","last_change":"2024-09-16T15:51:02.384379+0000","last_active":"2024-09-16T15:51:04.309158+0000","last_peered":"2024-09-16T15:51:04.309158+0000","last_clean":"2024-09-16T15:51:04.309158+0000","last_became_active":"2024-09-16T15:51:02.384012+0000","last_became_peered":"2024-09-16T15:51:02.384012+0000","last_unstale":"2024-09-16T15:51:04.309158+0000","last_undegraded":"2024-09-16T15:51:04.309158+0000","last_fullsized":"2024-09-16T15:51:04.309158+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:12:54.477349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:03.381453+0000","last_change":"2024-09-16T15:51:02.388400+0000","last_active":"2024-09-16T15:51:03.381453+0000","last_peered":"2024-09-16T15:51:03.381453+0000","last_clean":"2024-09-16T15:51:03.381453+0000","last_became_active":"2024-09-16T15:51:02.388145+0000","last_became_peered":"2024-09-16T15:51:02.388145+0000","last_unstale":"2024-09-16T15:51:03.381453+0000","last_undegraded":"2024-09-16T15:51:03.381453+0000","last_fullsized":"2024-09-16T15:51:03.381453+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:01:13.753963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1f","version":"103'4","reported_seq":19,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309998+0000","last_change":"2024-09-16T15:51:01.375200+0000","last_active":"2024-09-16T15:51:04.309998+0000","last_peered":"2024-09-16T15:51:04.309998+0000","last_clean":"2024-09-16T15:51:04.309998+0000","last_became_active":"2024-09-16T15:51:01.374919+0000","last_became_peered":"2024-09-16T15:51:01.374919+0000","last_unstale":"2024-09-16T15:51:04.309998+0000","last_undegraded":"2024-09-16T15:51:04.309998+0000","last_fullsized":"2024-09-16T15:51:04.309998+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:36:44.338407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.310065+0000","last_change":"2024-09-16T15:51:02.402988+0000","last_active":"2024-09-16T15:51:04.310065+0000","last_peered":"2024-09-16T15:51:04.310065+0000","last_clean":"2024-09-16T15:51:04.310065+0000","last_became_active":"2024-09-16T15:51:02.402708+0000","last_became_peered":"2024-09-16T15:51:02.402708+0000","last_unstale":"2024-09-16T15:51:04.310065+0000","last_undegraded":"2024-09-16T15:51:04.310065+0000","last_fullsized":"2024-09-16T15:51:04.310065+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:14:48.952444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1e","version":"103'1","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309267+0000","last_change":"2024-09-16T15:51:01.391174+0000","last_active":"2024-09-16T15:51:04.309267+0000","last_peered":"2024-09-16T15:51:04.309267+0000","last_clean":"2024-09-16T15:51:04.309267+0000","last_became_active":"2024-09-16T15:51:01.390904+0000","last_became_peered":"2024-09-16T15:51:01.390904+0000","last_unstale":"2024-09-16T15:51:04.309267+0000","last_undegraded":"2024-09-16T15:51:04.309267+0000","last_fullsized":"2024-09-16T15:51:04.309267+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:00:46.351712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309807+0000","last_change":"2024-09-16T15:51:02.382876+0000","last_active":"2024-09-16T15:51:04.309807+0000","last_peered":"2024-09-16T15:51:04.309807+0000","last_clean":"2024-09-16T15:51:04.309807+0000","last_became_active":"2024-09-16T15:51:02.382617+0000","last_became_peered":"2024-09-16T15:51:02.382617+0000","last_unstale":"2024-09-16T15:51:04.309807+0000","last_undegraded":"2024-09-16T15:51:04.309807+0000","last_fullsized":"2024-09-16T15:51:04.309807+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:58:47.475064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"103'2","reported_seq":17,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:03.381228+0000","last_change":"2024-09-16T15:51:01.388774+0000","last_active":"2024-09-16T15:51:03.381228+0000","last_peered":"2024-09-16T15:51:03.381228+0000","last_clean":"2024-09-16T15:51:03.381228+0000","last_became_active":"2024-09-16T15:51:01.388500+0000","last_became_peered":"2024-09-16T15:51:01.388500+0000","last_unstale":"2024-09-16T15:51:03.381228+0000","last_undegraded":"2024-09-16T15:51:03.381228+0000","last_fullsized":"2024-09-16T15:51:03.381228+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:46:58.289338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:03.378280+0000","last_change":"2024-09-16T15:51:02.386042+0000","last_active":"2024-09-16T15:51:03.378280+0000","last_peered":"2024-09-16T15:51:03.378280+0000","last_clean":"2024-09-16T15:51:03.378280+0000","last_became_active":"2024-09-16T15:51:02.385803+0000","last_became_peered":"2024-09-16T15:51:02.385803+0000","last_unstale":"2024-09-16T15:51:03.378280+0000","last_undegraded":"2024-09-16T15:51:03.378280+0000","last_fullsized":"2024-09-16T15:51:03.378280+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:30:51.230580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"103'2","reported_seq":17,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309747+0000","last_change":"2024-09-16T15:51:01.390985+0000","last_active":"2024-09-16T15:51:04.309747+0000","last_peered":"2024-09-16T15:51:04.309747+0000","last_clean":"2024-09-16T15:51:04.309747+0000","last_became_active":"2024-09-16T15:51:01.390463+0000","last_became_peered":"2024-09-16T15:51:01.390463+0000","last_unstale":"2024-09-16T15:51:04.309747+0000","last_undegraded":"2024-09-16T15:51:04.309747+0000","last_fullsized":"2024-09-16T15:51:04.309747+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:57:05.221628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.310415+0000","last_change":"2024-09-16T15:51:02.387425+0000","last_active":"2024-09-16T15:51:04.310415+0000","last_peered":"2024-09-16T15:51:04.310415+0000","last_clean":"2024-09-16T15:51:04.310415+0000","last_became_active":"2024-09-16T15:51:02.386275+0000","last_became_peered":"2024-09-16T15:51:02.386275+0000","last_unstale":"2024-09-16T15:51:04.310415+0000","last_undegraded":"2024-09-16T15:51:04.310415+0000","last_fullsized":"2024-09-16T15:51:04.310415+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:57:45.368832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1b","version":"103'1","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309180+0000","last_change":"2024-09-16T15:51:01.387614+0000","last_active":"2024-09-16T15:51:04.309180+0000","last_peered":"2024-09-16T15:51:04.309180+0000","last_clean":"2024-09-16T15:51:04.309180+0000","last_became_active":"2024-09-16T15:51:01.387163+0000","last_became_peered":"2024-09-16T15:51:01.387163+0000","last_unstale":"2024-09-16T15:51:04.309180+0000","last_undegraded":"2024-09-16T15:51:04.309180+0000","last_fullsized":"2024-09-16T15:51:04.309180+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:48:24.298645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309297+0000","last_change":"2024-09-16T15:51:02.404828+0000","last_active":"2024-09-16T15:51:04.309297+0000","last_peered":"2024-09-16T15:51:04.309297+0000","last_clean":"2024-09-16T15:51:04.309297+0000","last_became_active":"2024-09-16T15:51:02.403724+0000","last_became_peered":"2024-09-16T15:51:02.403724+0000","last_unstale":"2024-09-16T15:51:04.309297+0000","last_undegraded":"2024-09-16T15:51:04.309297+0000","last_fullsized":"2024-09-16T15:51:04.309297+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:55:45.502743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1a","version":"103'1","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:03.380176+0000","last_change":"2024-09-16T15:51:01.378848+0000","last_active":"2024-09-16T15:51:03.380176+0000","last_peered":"2024-09-16T15:51:03.380176+0000","last_clean":"2024-09-16T15:51:03.380176+0000","last_became_active":"2024-09-16T15:51:01.377621+0000","last_became_peered":"2024-09-16T15:51:01.377621+0000","last_unstale":"2024-09-16T15:51:03.380176+0000","last_undegraded":"2024-09-16T15:51:03.380176+0000","last_fullsized":"2024-09-16T15:51:03.380176+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:44:22.903859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.308778+0000","last_change":"2024-09-16T15:51:02.405495+0000","last_active":"2024-09-16T15:51:04.308778+0000","last_peered":"2024-09-16T15:51:04.308778+0000","last_clean":"2024-09-16T15:51:04.308778+0000","last_became_active":"2024-09-16T15:51:02.405222+0000","last_became_peered":"2024-09-16T15:51:02.405222+0000","last_unstale":"2024-09-16T15:51:04.308778+0000","last_undegraded":"2024-09-16T15:51:04.308778+0000","last_fullsized":"2024-09-16T15:51:04.308778+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:22:02.790695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309131+0000","last_change":"2024-09-16T15:51:01.372791+0000","last_active":"2024-09-16T15:51:04.309131+0000","last_peered":"2024-09-16T15:51:04.309131+0000","last_clean":"2024-09-16T15:51:04.309131+0000","last_became_active":"2024-09-16T15:51:01.372493+0000","last_became_peered":"2024-09-16T15:51:01.372493+0000","last_unstale":"2024-09-16T15:51:04.309131+0000","last_undegraded":"2024-09-16T15:51:04.309131+0000","last_fullsized":"2024-09-16T15:51:04.309131+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:01:27.263927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.308738+0000","last_change":"2024-09-16T15:51:02.386313+0000","last_active":"2024-09-16T15:51:04.308738+0000","last_peered":"2024-09-16T15:51:04.308738+0000","last_clean":"2024-09-16T15:51:04.308738+0000","last_became_active":"2024-09-16T15:51:02.385942+0000","last_became_peered":"2024-09-16T15:51:02.385942+0000","last_unstale":"2024-09-16T15:51:04.308738+0000","last_undegraded":"2024-09-16T15:51:04.308738+0000","last_fullsized":"2024-09-16T15:51:04.308738+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:29:23.387660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:03.379691+0000","last_change":"2024-09-16T15:51:01.386550+0000","last_active":"2024-09-16T15:51:03.379691+0000","last_peered":"2024-09-16T15:51:03.379691+0000","last_clean":"2024-09-16T15:51:03.379691+0000","last_became_active":"2024-09-16T15:51:01.386275+0000","last_became_peered":"2024-09-16T15:51:01.386275+0000","last_unstale":"2024-09-16T15:51:03.379691+0000","last_undegraded":"2024-09-16T15:51:03.379691+0000","last_fullsized":"2024-09-16T15:51:03.379691+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:42:26.502435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.310033+0000","last_change":"2024-09-16T15:51:02.386731+0000","last_active":"2024-09-16T15:51:04.310033+0000","last_peered":"2024-09-16T15:51:04.310033+0000","last_clean":"2024-09-16T15:51:04.310033+0000","last_became_active":"2024-09-16T15:51:02.384022+0000","last_became_peered":"2024-09-16T15:51:02.384022+0000","last_unstale":"2024-09-16T15:51:04.310033+0000","last_undegraded":"2024-09-16T15:51:04.310033+0000","last_fullsized":"2024-09-16T15:51:04.310033+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:30:51.230580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:03.380909+0000","last_change":"2024-09-16T15:51:01.378467+0000","last_active":"2024-09-16T15:51:03.380909+0000","last_peered":"2024-09-16T15:51:03.380909+0000","last_clean":"2024-09-16T15:51:03.380909+0000","last_became_active":"2024-09-16T15:51:01.378205+0000","last_became_peered":"2024-09-16T15:51:01.378205+0000","last_unstale":"2024-09-16T15:51:03.380909+0000","last_undegraded":"2024-09-16T15:51:03.380909+0000","last_fullsized":"2024-09-16T15:51:03.380909+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:39:21.798976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309818+0000","last_change":"2024-09-16T15:51:02.399038+0000","last_active":"2024-09-16T15:51:04.309818+0000","last_peered":"2024-09-16T15:51:04.309818+0000","last_clean":"2024-09-16T15:51:04.309818+0000","last_became_active":"2024-09-16T15:51:02.398752+0000","last_became_peered":"2024-09-16T15:51:02.398752+0000","last_unstale":"2024-09-16T15:51:04.309818+0000","last_undegraded":"2024-09-16T15:51:04.309818+0000","last_fullsized":"2024-09-16T15:51:04.309818+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:56:24.131464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.310208+0000","last_change":"2024-09-16T15:51:01.389590+0000","last_active":"2024-09-16T15:51:04.310208+0000","last_peered":"2024-09-16T15:51:04.310208+0000","last_clean":"2024-09-16T15:51:04.310208+0000","last_became_active":"2024-09-16T15:51:01.389186+0000","last_became_peered":"2024-09-16T15:51:01.389186+0000","last_unstale":"2024-09-16T15:51:04.310208+0000","last_undegraded":"2024-09-16T15:51:04.310208+0000","last_fullsized":"2024-09-16T15:51:04.310208+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:22:23.409721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309112+0000","last_change":"2024-09-16T15:51:02.391455+0000","last_active":"2024-09-16T15:51:04.309112+0000","last_peered":"2024-09-16T15:51:04.309112+0000","last_clean":"2024-09-16T15:51:04.309112+0000","last_became_active":"2024-09-16T15:51:02.391250+0000","last_became_peered":"2024-09-16T15:51:02.391250+0000","last_unstale":"2024-09-16T15:51:04.309112+0000","last_undegraded":"2024-09-16T15:51:04.309112+0000","last_fullsized":"2024-09-16T15:51:04.309112+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:17:21.134677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.15","version":"103'3","reported_seq":18,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309817+0000","last_change":"2024-09-16T15:51:01.379431+0000","last_active":"2024-09-16T15:51:04.309817+0000","last_peered":"2024-09-16T15:51:04.309817+0000","last_clean":"2024-09-16T15:51:04.309817+0000","last_became_active":"2024-09-16T15:51:01.379201+0000","last_became_peered":"2024-09-16T15:51:01.379201+0000","last_unstale":"2024-09-16T15:51:04.309817+0000","last_undegraded":"2024-09-16T15:51:04.309817+0000","last_fullsized":"2024-09-16T15:51:04.309817+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:04:50.965816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309521+0000","last_change":"2024-09-16T15:51:02.400930+0000","last_active":"2024-09-16T15:51:04.309521+0000","last_peered":"2024-09-16T15:51:04.309521+0000","last_clean":"2024-09-16T15:51:04.309521+0000","last_became_active":"2024-09-16T15:51:02.400622+0000","last_became_peered":"2024-09-16T15:51:02.400622+0000","last_unstale":"2024-09-16T15:51:04.309521+0000","last_undegraded":"2024-09-16T15:51:04.309521+0000","last_fullsized":"2024-09-16T15:51:04.309521+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:12:08.141906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.14","version":"103'6","reported_seq":21,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.310440+0000","last_change":"2024-09-16T15:51:01.378141+0000","last_active":"2024-09-16T15:51:04.310440+0000","last_peered":"2024-09-16T15:51:04.310440+0000","last_clean":"2024-09-16T15:51:04.310440+0000","last_became_active":"2024-09-16T15:51:01.377875+0000","last_became_peered":"2024-09-16T15:51:01.377875+0000","last_unstale":"2024-09-16T15:51:04.310440+0000","last_undegraded":"2024-09-16T15:51:04.310440+0000","last_fullsized":"2024-09-16T15:51:04.310440+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:48:33.457678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309046+0000","last_change":"2024-09-16T15:51:02.387140+0000","last_active":"2024-09-16T15:51:04.309046+0000","last_peered":"2024-09-16T15:51:04.309046+0000","last_clean":"2024-09-16T15:51:04.309046+0000","last_became_active":"2024-09-16T15:51:02.385974+0000","last_became_peered":"2024-09-16T15:51:02.385974+0000","last_unstale":"2024-09-16T15:51:04.309046+0000","last_undegraded":"2024-09-16T15:51:04.309046+0000","last_fullsized":"2024-09-16T15:51:04.309046+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:37:03.319352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"103'2","reported_seq":17,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:03.379867+0000","last_change":"2024-09-16T15:51:01.384546+0000","last_active":"2024-09-16T15:51:03.379867+0000","last_peered":"2024-09-16T15:51:03.379867+0000","last_clean":"2024-09-16T15:51:03.379867+0000","last_became_active":"2024-09-16T15:51:01.384288+0000","last_became_peered":"2024-09-16T15:51:01.384288+0000","last_unstale":"2024-09-16T15:51:03.379867+0000","last_undegraded":"2024-09-16T15:51:03.379867+0000","last_fullsized":"2024-09-16T15:51:03.379867+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:56:23.125710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:03.380503+0000","last_change":"2024-09-16T15:51:02.396326+0000","last_active":"2024-09-16T15:51:03.380503+0000","last_peered":"2024-09-16T15:51:03.380503+0000","last_clean":"2024-09-16T15:51:03.380503+0000","last_became_active":"2024-09-16T15:51:02.394643+0000","last_became_peered":"2024-09-16T15:51:02.394643+0000","last_unstale":"2024-09-16T15:51:03.380503+0000","last_undegraded":"2024-09-16T15:51:03.380503+0000","last_fullsized":"2024-09-16T15:51:03.380503+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:29:23.387660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309143+0000","last_change":"2024-09-16T15:51:01.384749+0000","last_active":"2024-09-16T15:51:04.309143+0000","last_peered":"2024-09-16T15:51:04.309143+0000","last_clean":"2024-09-16T15:51:04.309143+0000","last_became_active":"2024-09-16T15:51:01.384433+0000","last_became_peered":"2024-09-16T15:51:01.384433+0000","last_unstale":"2024-09-16T15:51:04.309143+0000","last_undegraded":"2024-09-16T15:51:04.309143+0000","last_fullsized":"2024-09-16T15:51:04.309143+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:43:12.283062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309922+0000","last_change":"2024-09-16T15:51:02.391709+0000","last_active":"2024-09-16T15:51:04.309922+0000","last_peered":"2024-09-16T15:51:04.309922+0000","last_clean":"2024-09-16T15:51:04.309922+0000","last_became_active":"2024-09-16T15:51:02.391379+0000","last_became_peered":"2024-09-16T15:51:02.391379+0000","last_unstale":"2024-09-16T15:51:04.309922+0000","last_undegraded":"2024-09-16T15:51:04.309922+0000","last_fullsized":"2024-09-16T15:51:04.309922+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:46:45.927933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309481+0000","last_change":"2024-09-16T15:51:01.378352+0000","last_active":"2024-09-16T15:51:04.309481+0000","last_peered":"2024-09-16T15:51:04.309481+0000","last_clean":"2024-09-16T15:51:04.309481+0000","last_became_active":"2024-09-16T15:51:01.376625+0000","last_became_peered":"2024-09-16T15:51:01.376625+0000","last_unstale":"2024-09-16T15:51:04.309481+0000","last_undegraded":"2024-09-16T15:51:04.309481+0000","last_fullsized":"2024-09-16T15:51:04.309481+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:07:26.131631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:03.380970+0000","last_change":"2024-09-16T15:51:02.397905+0000","last_active":"2024-09-16T15:51:03.380970+0000","last_peered":"2024-09-16T15:51:03.380970+0000","last_clean":"2024-09-16T15:51:03.380970+0000","last_became_active":"2024-09-16T15:51:02.397623+0000","last_became_peered":"2024-09-16T15:51:02.397623+0000","last_unstale":"2024-09-16T15:51:03.380970+0000","last_undegraded":"2024-09-16T15:51:03.380970+0000","last_fullsized":"2024-09-16T15:51:03.380970+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:36:12.989907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.10","version":"103'8","reported_seq":23,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309643+0000","last_change":"2024-09-16T15:51:01.383280+0000","last_active":"2024-09-16T15:51:04.309643+0000","last_peered":"2024-09-16T15:51:04.309643+0000","last_clean":"2024-09-16T15:51:04.309643+0000","last_became_active":"2024-09-16T15:51:01.381141+0000","last_became_peered":"2024-09-16T15:51:01.381141+0000","last_unstale":"2024-09-16T15:51:04.309643+0000","last_undegraded":"2024-09-16T15:51:04.309643+0000","last_fullsized":"2024-09-16T15:51:04.309643+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:10:44.749157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.310001+0000","last_change":"2024-09-16T15:51:02.389522+0000","last_active":"2024-09-16T15:51:04.310001+0000","last_peered":"2024-09-16T15:51:04.310001+0000","last_clean":"2024-09-16T15:51:04.310001+0000","last_became_active":"2024-09-16T15:51:02.389260+0000","last_became_peered":"2024-09-16T15:51:02.389260+0000","last_unstale":"2024-09-16T15:51:04.310001+0000","last_undegraded":"2024-09-16T15:51:04.310001+0000","last_fullsized":"2024-09-16T15:51:04.310001+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:21:10.883240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.e","version":"103'2","reported_seq":17,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309924+0000","last_change":"2024-09-16T15:51:01.391228+0000","last_active":"2024-09-16T15:51:04.309924+0000","last_peered":"2024-09-16T15:51:04.309924+0000","last_clean":"2024-09-16T15:51:04.309924+0000","last_became_active":"2024-09-16T15:51:01.390919+0000","last_became_peered":"2024-09-16T15:51:01.390919+0000","last_unstale":"2024-09-16T15:51:04.309924+0000","last_undegraded":"2024-09-16T15:51:04.309924+0000","last_fullsized":"2024-09-16T15:51:04.309924+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:28:48.291775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309186+0000","last_change":"2024-09-16T15:51:02.388901+0000","last_active":"2024-09-16T15:51:04.309186+0000","last_peered":"2024-09-16T15:51:04.309186+0000","last_clean":"2024-09-16T15:51:04.309186+0000","last_became_active":"2024-09-16T15:51:02.388312+0000","last_became_peered":"2024-09-16T15:51:02.388312+0000","last_unstale":"2024-09-16T15:51:04.309186+0000","last_undegraded":"2024-09-16T15:51:04.309186+0000","last_fullsized":"2024-09-16T15:51:04.309186+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:42:27.508189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"103'1","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309927+0000","last_change":"2024-09-16T15:51:01.377467+0000","last_active":"2024-09-16T15:51:04.309927+0000","last_peered":"2024-09-16T15:51:04.309927+0000","last_clean":"2024-09-16T15:51:04.309927+0000","last_became_active":"2024-09-16T15:51:01.377180+0000","last_became_peered":"2024-09-16T15:51:01.377180+0000","last_unstale":"2024-09-16T15:51:04.309927+0000","last_undegraded":"2024-09-16T15:51:04.309927+0000","last_fullsized":"2024-09-16T15:51:04.309927+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:15:29.048646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.310133+0000","last_change":"2024-09-16T15:51:02.401565+0000","last_active":"2024-09-16T15:51:04.310133+0000","last_peered":"2024-09-16T15:51:04.310133+0000","last_clean":"2024-09-16T15:51:04.310133+0000","last_became_active":"2024-09-16T15:51:02.401304+0000","last_became_peered":"2024-09-16T15:51:02.401304+0000","last_unstale":"2024-09-16T15:51:04.310133+0000","last_undegraded":"2024-09-16T15:51:04.310133+0000","last_fullsized":"2024-09-16T15:51:04.310133+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:36:12.989907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:03.381005+0000","last_change":"2024-09-16T15:51:01.385783+0000","last_active":"2024-09-16T15:51:03.381005+0000","last_peered":"2024-09-16T15:51:03.381005+0000","last_clean":"2024-09-16T15:51:03.381005+0000","last_became_active":"2024-09-16T15:51:01.385500+0000","last_became_peered":"2024-09-16T15:51:01.385500+0000","last_unstale":"2024-09-16T15:51:03.381005+0000","last_undegraded":"2024-09-16T15:51:03.381005+0000","last_fullsized":"2024-09-16T15:51:03.381005+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:30:12.503202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:03.379982+0000","last_change":"2024-09-16T15:51:02.399523+0000","last_active":"2024-09-16T15:51:03.379982+0000","last_peered":"2024-09-16T15:51:03.379982+0000","last_clean":"2024-09-16T15:51:03.379982+0000","last_became_active":"2024-09-16T15:51:02.399256+0000","last_became_peered":"2024-09-16T15:51:02.399256+0000","last_unstale":"2024-09-16T15:51:03.379982+0000","last_undegraded":"2024-09-16T15:51:03.379982+0000","last_fullsized":"2024-09-16T15:51:03.379982+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:59:14.234290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"103'2","reported_seq":17,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309313+0000","last_change":"2024-09-16T15:51:01.391626+0000","last_active":"2024-09-16T15:51:04.309313+0000","last_peered":"2024-09-16T15:51:04.309313+0000","last_clean":"2024-09-16T15:51:04.309313+0000","last_became_active":"2024-09-16T15:51:01.391322+0000","last_became_peered":"2024-09-16T15:51:01.391322+0000","last_unstale":"2024-09-16T15:51:04.309313+0000","last_undegraded":"2024-09-16T15:51:04.309313+0000","last_fullsized":"2024-09-16T15:51:04.309313+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:13:41.121392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:03.379709+0000","last_change":"2024-09-16T15:51:02.405123+0000","last_active":"2024-09-16T15:51:03.379709+0000","last_peered":"2024-09-16T15:51:03.379709+0000","last_clean":"2024-09-16T15:51:03.379709+0000","last_became_active":"2024-09-16T15:51:02.404678+0000","last_became_peered":"2024-09-16T15:51:02.404678+0000","last_unstale":"2024-09-16T15:51:03.379709+0000","last_undegraded":"2024-09-16T15:51:03.379709+0000","last_fullsized":"2024-09-16T15:51:03.379709+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:03:32.011151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.308961+0000","last_change":"2024-09-16T15:51:01.378559+0000","last_active":"2024-09-16T15:51:04.308961+0000","last_peered":"2024-09-16T15:51:04.308961+0000","last_clean":"2024-09-16T15:51:04.308961+0000","last_became_active":"2024-09-16T15:51:01.377521+0000","last_became_peered":"2024-09-16T15:51:01.377521+0000","last_unstale":"2024-09-16T15:51:04.308961+0000","last_undegraded":"2024-09-16T15:51:04.308961+0000","last_fullsized":"2024-09-16T15:51:04.308961+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:29:18.126495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309424+0000","last_change":"2024-09-16T15:51:02.400056+0000","last_active":"2024-09-16T15:51:04.309424+0000","last_peered":"2024-09-16T15:51:04.309424+0000","last_clean":"2024-09-16T15:51:04.309424+0000","last_became_active":"2024-09-16T15:51:02.397362+0000","last_became_peered":"2024-09-16T15:51:02.397362+0000","last_unstale":"2024-09-16T15:51:04.309424+0000","last_undegraded":"2024-09-16T15:51:04.309424+0000","last_fullsized":"2024-09-16T15:51:04.309424+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:45:49.121469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:03.378116+0000","last_change":"2024-09-16T15:51:01.372918+0000","last_active":"2024-09-16T15:51:03.378116+0000","last_peered":"2024-09-16T15:51:03.378116+0000","last_clean":"2024-09-16T15:51:03.378116+0000","last_became_active":"2024-09-16T15:51:01.372464+0000","last_became_peered":"2024-09-16T15:51:01.372464+0000","last_unstale":"2024-09-16T15:51:03.378116+0000","last_undegraded":"2024-09-16T15:51:03.378116+0000","last_fullsized":"2024-09-16T15:51:03.378116+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:54:34.716483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.310129+0000","last_change":"2024-09-16T15:51:02.406721+0000","last_active":"2024-09-16T15:51:04.310129+0000","last_peered":"2024-09-16T15:51:04.310129+0000","last_clean":"2024-09-16T15:51:04.310129+0000","last_became_active":"2024-09-16T15:51:02.405161+0000","last_became_peered":"2024-09-16T15:51:02.405161+0000","last_unstale":"2024-09-16T15:51:04.310129+0000","last_undegraded":"2024-09-16T15:51:04.310129+0000","last_fullsized":"2024-09-16T15:51:04.310129+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:14:26.465332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:03.379912+0000","last_change":"2024-09-16T15:51:01.386549+0000","last_active":"2024-09-16T15:51:03.379912+0000","last_peered":"2024-09-16T15:51:03.379912+0000","last_clean":"2024-09-16T15:51:03.379912+0000","last_became_active":"2024-09-16T15:51:01.386253+0000","last_became_peered":"2024-09-16T15:51:01.386253+0000","last_unstale":"2024-09-16T15:51:03.379912+0000","last_undegraded":"2024-09-16T15:51:03.379912+0000","last_fullsized":"2024-09-16T15:51:03.379912+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:32:44.402539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:03.381369+0000","last_change":"2024-09-16T15:51:02.396005+0000","last_active":"2024-09-16T15:51:03.381369+0000","last_peered":"2024-09-16T15:51:03.381369+0000","last_clean":"2024-09-16T15:51:03.381369+0000","last_became_active":"2024-09-16T15:51:02.395727+0000","last_became_peered":"2024-09-16T15:51:02.395727+0000","last_unstale":"2024-09-16T15:51:03.381369+0000","last_undegraded":"2024-09-16T15:51:03.381369+0000","last_fullsized":"2024-09-16T15:51:03.381369+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:28:17.080952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.7","version":"103'2","reported_seq":17,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.310547+0000","last_change":"2024-09-16T15:51:01.379987+0000","last_active":"2024-09-16T15:51:04.310547+0000","last_peered":"2024-09-16T15:51:04.310547+0000","last_clean":"2024-09-16T15:51:04.310547+0000","last_became_active":"2024-09-16T15:51:01.379714+0000","last_became_peered":"2024-09-16T15:51:01.379714+0000","last_unstale":"2024-09-16T15:51:04.310547+0000","last_undegraded":"2024-09-16T15:51:04.310547+0000","last_fullsized":"2024-09-16T15:51:04.310547+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:39:21.798976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:03.381327+0000","last_change":"2024-09-16T15:51:02.384154+0000","last_active":"2024-09-16T15:51:03.381327+0000","last_peered":"2024-09-16T15:51:03.381327+0000","last_clean":"2024-09-16T15:51:03.381327+0000","last_became_active":"2024-09-16T15:51:02.383885+0000","last_became_peered":"2024-09-16T15:51:02.383885+0000","last_unstale":"2024-09-16T15:51:03.381327+0000","last_undegraded":"2024-09-16T15:51:03.381327+0000","last_fullsized":"2024-09-16T15:51:03.381327+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:31:37.796930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.6","version":"104'1","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:03.381671+0000","last_change":"2024-09-16T15:51:01.391293+0000","last_active":"2024-09-16T15:51:03.381671+0000","last_peered":"2024-09-16T15:51:03.381671+0000","last_clean":"2024-09-16T15:51:03.381671+0000","last_became_active":"2024-09-16T15:51:01.388187+0000","last_became_peered":"2024-09-16T15:51:01.388187+0000","last_unstale":"2024-09-16T15:51:03.381671+0000","last_undegraded":"2024-09-16T15:51:03.381671+0000","last_fullsized":"2024-09-16T15:51:03.381671+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:00:19.592466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:03.380574+0000","last_change":"2024-09-16T15:51:02.399564+0000","last_active":"2024-09-16T15:51:03.380574+0000","last_peered":"2024-09-16T15:51:03.380574+0000","last_clean":"2024-09-16T15:51:03.380574+0000","last_became_active":"2024-09-16T15:51:02.397436+0000","last_became_peered":"2024-09-16T15:51:02.397436+0000","last_unstale":"2024-09-16T15:51:03.380574+0000","last_undegraded":"2024-09-16T15:51:03.380574+0000","last_fullsized":"2024-09-16T15:51:03.380574+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:57:33.960661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:03.381184+0000","last_change":"2024-09-16T15:51:01.381796+0000","last_active":"2024-09-16T15:51:03.381184+0000","last_peered":"2024-09-16T15:51:03.381184+0000","last_clean":"2024-09-16T15:51:03.381184+0000","last_became_active":"2024-09-16T15:51:01.381524+0000","last_became_peered":"2024-09-16T15:51:01.381524+0000","last_unstale":"2024-09-16T15:51:03.381184+0000","last_undegraded":"2024-09-16T15:51:03.381184+0000","last_fullsized":"2024-09-16T15:51:03.381184+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:03:22.300390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309648+0000","last_change":"2024-09-16T15:51:02.389579+0000","last_active":"2024-09-16T15:51:04.309648+0000","last_peered":"2024-09-16T15:51:04.309648+0000","last_clean":"2024-09-16T15:51:04.309648+0000","last_became_active":"2024-09-16T15:51:02.389279+0000","last_became_peered":"2024-09-16T15:51:02.389279+0000","last_unstale":"2024-09-16T15:51:04.309648+0000","last_undegraded":"2024-09-16T15:51:04.309648+0000","last_fullsized":"2024-09-16T15:51:04.309648+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:32:17.641313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309944+0000","last_change":"2024-09-16T15:51:01.380670+0000","last_active":"2024-09-16T15:51:04.309944+0000","last_peered":"2024-09-16T15:51:04.309944+0000","last_clean":"2024-09-16T15:51:04.309944+0000","last_became_active":"2024-09-16T15:51:01.380213+0000","last_became_peered":"2024-09-16T15:51:01.380213+0000","last_unstale":"2024-09-16T15:51:04.309944+0000","last_undegraded":"2024-09-16T15:51:04.309944+0000","last_fullsized":"2024-09-16T15:51:04.309944+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:07:53.419216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:03.379649+0000","last_change":"2024-09-16T15:51:02.398432+0000","last_active":"2024-09-16T15:51:03.379649+0000","last_peered":"2024-09-16T15:51:03.379649+0000","last_clean":"2024-09-16T15:51:03.379649+0000","last_became_active":"2024-09-16T15:51:02.397631+0000","last_became_peered":"2024-09-16T15:51:02.397631+0000","last_unstale":"2024-09-16T15:51:03.379649+0000","last_undegraded":"2024-09-16T15:51:03.379649+0000","last_fullsized":"2024-09-16T15:51:03.379649+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:52:13.451127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:03.380697+0000","last_change":"2024-09-16T15:51:01.388916+0000","last_active":"2024-09-16T15:51:03.380697+0000","last_peered":"2024-09-16T15:51:03.380697+0000","last_clean":"2024-09-16T15:51:03.380697+0000","last_became_active":"2024-09-16T15:51:01.388644+0000","last_became_peered":"2024-09-16T15:51:01.388644+0000","last_unstale":"2024-09-16T15:51:03.380697+0000","last_undegraded":"2024-09-16T15:51:03.380697+0000","last_fullsized":"2024-09-16T15:51:03.380697+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:32:29.346680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309492+0000","last_change":"2024-09-16T15:51:02.401892+0000","last_active":"2024-09-16T15:51:04.309492+0000","last_peered":"2024-09-16T15:51:04.309492+0000","last_clean":"2024-09-16T15:51:04.309492+0000","last_became_active":"2024-09-16T15:51:02.401633+0000","last_became_peered":"2024-09-16T15:51:02.401633+0000","last_unstale":"2024-09-16T15:51:04.309492+0000","last_undegraded":"2024-09-16T15:51:04.309492+0000","last_fullsized":"2024-09-16T15:51:04.309492+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:34:03.971706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.309051+0000","last_change":"2024-09-16T15:51:01.387695+0000","last_active":"2024-09-16T15:51:04.309051+0000","last_peered":"2024-09-16T15:51:04.309051+0000","last_clean":"2024-09-16T15:51:04.309051+0000","last_became_active":"2024-09-16T15:51:01.387455+0000","last_became_peered":"2024-09-16T15:51:01.387455+0000","last_unstale":"2024-09-16T15:51:04.309051+0000","last_undegraded":"2024-09-16T15:51:04.309051+0000","last_fullsized":"2024-09-16T15:51:04.309051+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:28:29.328131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:03.380005+0000","last_change":"2024-09-16T15:51:02.389201+0000","last_active":"2024-09-16T15:51:03.380005+0000","last_peered":"2024-09-16T15:51:03.380005+0000","last_clean":"2024-09-16T15:51:03.380005+0000","last_became_active":"2024-09-16T15:51:02.388954+0000","last_became_peered":"2024-09-16T15:51:02.388954+0000","last_unstale":"2024-09-16T15:51:03.380005+0000","last_undegraded":"2024-09-16T15:51:03.380005+0000","last_fullsized":"2024-09-16T15:51:03.380005+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:02:03.707466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.308686+0000","last_change":"2024-09-16T15:51:01.384206+0000","last_active":"2024-09-16T15:51:04.308686+0000","last_peered":"2024-09-16T15:51:04.308686+0000","last_clean":"2024-09-16T15:51:04.308686+0000","last_became_active":"2024-09-16T15:51:01.383873+0000","last_became_peered":"2024-09-16T15:51:01.383873+0000","last_unstale":"2024-09-16T15:51:04.308686+0000","last_undegraded":"2024-09-16T15:51:04.308686+0000","last_fullsized":"2024-09-16T15:51:04.308686+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:15:03.468370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.308953+0000","last_change":"2024-09-16T15:51:02.392882+0000","last_active":"2024-09-16T15:51:04.308953+0000","last_peered":"2024-09-16T15:51:04.308953+0000","last_clean":"2024-09-16T15:51:04.308953+0000","last_became_active":"2024-09-16T15:51:02.392610+0000","last_became_peered":"2024-09-16T15:51:02.392610+0000","last_unstale":"2024-09-16T15:51:04.308953+0000","last_undegraded":"2024-09-16T15:51:04.308953+0000","last_fullsized":"2024-09-16T15:51:04.308953+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:57:33.960661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:03.380013+0000","last_change":"2024-09-16T15:51:01.390952+0000","last_active":"2024-09-16T15:51:03.380013+0000","last_peered":"2024-09-16T15:51:03.380013+0000","last_clean":"2024-09-16T15:51:03.380013+0000","last_became_active":"2024-09-16T15:51:01.390640+0000","last_became_peered":"2024-09-16T15:51:01.390640+0000","last_unstale":"2024-09-16T15:51:03.380013+0000","last_undegraded":"2024-09-16T15:51:03.380013+0000","last_fullsized":"2024-09-16T15:51:03.380013+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:29:00.354401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:03.381424+0000","last_change":"2024-09-16T15:51:02.404581+0000","last_active":"2024-09-16T15:51:03.381424+0000","last_peered":"2024-09-16T15:51:03.381424+0000","last_clean":"2024-09-16T15:51:03.381424+0000","last_became_active":"2024-09-16T15:51:02.404256+0000","last_became_peered":"2024-09-16T15:51:02.404256+0000","last_unstale":"2024-09-16T15:51:03.381424+0000","last_undegraded":"2024-09-16T15:51:03.381424+0000","last_fullsized":"2024-09-16T15:51:03.381424+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:01.355970+0000","last_clean_scrub_stamp":"2024-09-16T15:51:01.355970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:57:45.368832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.f","version":"103'1","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.310516+0000","last_change":"2024-09-16T15:51:01.388328+0000","last_active":"2024-09-16T15:51:04.310516+0000","last_peered":"2024-09-16T15:51:04.310516+0000","last_clean":"2024-09-16T15:51:04.310516+0000","last_became_active":"2024-09-16T15:51:01.388066+0000","last_became_peered":"2024-09-16T15:51:01.388066+0000","last_unstale":"2024-09-16T15:51:04.310516+0000","last_undegraded":"2024-09-16T15:51:04.310516+0000","last_fullsized":"2024-09-16T15:51:04.310516+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:51:00.350216+0000","last_clean_scrub_stamp":"2024-09-16T15:51:00.350216+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:46:58.289338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":266,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-16T15:51:04.308775+0000","last_change":"2024-09-16T15:39:12.698813+0000","last_active":"2024-09-16T15:51:04.308775+0000","last_peered":"2024-09-16T15:51:04.308775+0000","last_clean":"2024-09-16T15:51:04.308775+0000","last_became_active":"2024-09-16T15:39:12.697695+0000","last_became_peered":"2024-09-16T15:39:12.697695+0000","last_unstale":"2024-09-16T15:51:04.308775+0000","last_undegraded":"2024-09-16T15:51:04.308775+0000","last_fullsized":"2024-09-16T15:51:04.308775+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_clean_scrub_stamp":"2024-09-16T15:39:11.684967+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:02.061081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509588,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":148984,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222856,"statfs":{"total":96636764160,"available":96484204544,"internally_reserved":0,"allocated":1277952,"data_stored":557788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":151255496},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509587,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131540,"kb_used_data":1848,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240300,"statfs":{"total":96636764160,"available":96502067200,"internally_reserved":0,"allocated":1892352,"data_stored":1149824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509586,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":144832,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94227008,"statfs":{"total":96636764160,"available":96488456192,"internally_reserved":0,"allocated":1282048,"data_stored":558370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":146995653},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509586,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126788,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94245052,"statfs":{"total":96636764160,"available":96506933248,"internally_reserved":0,"allocated":1290240,"data_stored":557982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":128514505},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509586,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126776,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94245064,"statfs":{"total":96636764160,"available":96506945536,"internally_reserved":0,"allocated":1282048,"data_stored":557858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":128514503},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509587,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123212,"kb_used_data":1848,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248628,"statfs":{"total":96636764160,"available":96510595072,"internally_reserved":0,"allocated":1892352,"data_stored":1149880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":124254663},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509585,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":140664,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94231176,"statfs":{"total":96636764160,"available":96492724224,"internally_reserved":0,"allocated":1277952,"data_stored":557788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":142735815},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542292,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122620,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94249220,"statfs":{"total":96636764160,"available":96511201280,"internally_reserved":0,"allocated":1282048,"data_stored":558370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":124254668},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:51:11.693 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T15:51:11.693 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-16T15:51:11.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.892+0000 7f18df400640 1 -- 172.21.15.151:0/180824131 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f18d80af190 msgr2=0x7f18d80b2b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:11.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.892+0000 7f18df400640 1 --2- 172.21.15.151:0/180824131 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f18d80af190 0x7f18d80b2b40 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f18d80b2470 tx=0x7f18c402a4d0 comp rx=0 tx=0).stop 2024-09-16T15:51:11.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.892+0000 7f18df400640 1 -- 172.21.15.151:0/180824131 shutdown_connections 2024-09-16T15:51:11.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.892+0000 7f18df400640 1 --2- 172.21.15.151:0/180824131 >> v2:172.21.15.167:3300/0 conn(0x7f18d80cec60 0x7f18d80cf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:11.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.892+0000 7f18df400640 1 --2- 172.21.15.151:0/180824131 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f18d80af190 0x7f18d80b2b40 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:11.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.892+0000 7f18df400640 1 --2- 172.21.15.151:0/180824131 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f18d8161760 0x7f18d80ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:11.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.892+0000 7f18df400640 1 -- 172.21.15.151:0/180824131 >> 172.21.15.151:0/180824131 conn(0x7f18d80b1840 msgr2=0x7f18d80b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:11.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.893+0000 7f18df400640 1 -- 172.21.15.151:0/180824131 shutdown_connections 2024-09-16T15:51:11.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.893+0000 7f18df400640 1 -- 172.21.15.151:0/180824131 wait complete. 2024-09-16T15:51:11.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.894+0000 7f18df400640 1 Processor -- start 2024-09-16T15:51:11.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.894+0000 7f18df400640 1 -- start start 2024-09-16T15:51:11.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.895+0000 7f18df400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18d80af190 0x7f18d80cd200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:11.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.895+0000 7f18df400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f18d80cec60 0x7f18d815fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:11.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.895+0000 7f18df400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f18d8161760 0x7f18d81602e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:11.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.895+0000 7f18df400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f18d80b3200 con 0x7f18d80cec60 2024-09-16T15:51:11.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.895+0000 7f18df400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f18d80b3080 con 0x7f18d80af190 2024-09-16T15:51:11.899 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.895+0000 7f18df400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f18d8162190 con 0x7f18d8161760 2024-09-16T15:51:11.899 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.895+0000 7f18d7e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f18d80cec60 0x7f18d815fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:11.899 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.895+0000 7f18d7e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f18d80cec60 0x7f18d815fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52738/0 (socket says 172.21.15.151:52738) 2024-09-16T15:51:11.899 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.895+0000 7f18d7e00640 1 -- 172.21.15.151:0/19942774 learned_addr learned my addr 172.21.15.151:0/19942774 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:51:11.899 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.896+0000 7f18dca00640 1 --2- 172.21.15.151:0/19942774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18d80af190 0x7f18d80cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:11.899 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.896+0000 7f18dd400640 1 --2- 172.21.15.151:0/19942774 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f18d8161760 0x7f18d81602e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:11.899 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.896+0000 7f18d7e00640 1 -- 172.21.15.151:0/19942774 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f18d8161760 msgr2=0x7f18d81602e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:11.899 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.896+0000 7f18d7e00640 1 --2- 172.21.15.151:0/19942774 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f18d8161760 0x7f18d81602e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:11.899 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.896+0000 7f18d7e00640 1 -- 172.21.15.151:0/19942774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18d80af190 msgr2=0x7f18d80cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:11.899 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.896+0000 7f18d7e00640 1 --2- 172.21.15.151:0/19942774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18d80af190 0x7f18d80cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:11.899 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.896+0000 7f18d7e00640 1 -- 172.21.15.151:0/19942774 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18d8152420 con 0x7f18d80cec60 2024-09-16T15:51:11.899 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.896+0000 7f18d7e00640 1 --2- 172.21.15.151:0/19942774 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f18d80cec60 0x7f18d815fda0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f18c4000c00 tx=0x7f18c4003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:11.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.896+0000 7f18d4c00640 1 -- 172.21.15.151:0/19942774 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c403f070 con 0x7f18d80cec60 2024-09-16T15:51:11.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.897+0000 7f18d4c00640 1 -- 172.21.15.151:0/19942774 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f18c40047a0 con 0x7f18d80cec60 2024-09-16T15:51:11.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.897+0000 7f18d4c00640 1 -- 172.21.15.151:0/19942774 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c4035030 con 0x7f18d80cec60 2024-09-16T15:51:11.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.897+0000 7f18df400640 1 -- 172.21.15.151:0/19942774 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18d8152690 con 0x7f18d80cec60 2024-09-16T15:51:11.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.897+0000 7f18df400640 1 -- 172.21.15.151:0/19942774 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18d8152c50 con 0x7f18d80cec60 2024-09-16T15:51:11.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.898+0000 7f18df400640 1 -- 172.21.15.151:0/19942774 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18d80cf090 con 0x7f18d80cec60 2024-09-16T15:51:11.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.899+0000 7f18d4c00640 1 -- 172.21.15.151:0/19942774 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7f18c4042070 con 0x7f18d80cec60 2024-09-16T15:51:11.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.900+0000 7f18d4c00640 1 --2- 172.21.15.151:0/19942774 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f18b8079ed0 0x7f18b807c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:11.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.900+0000 7f18d4c00640 1 -- 172.21.15.151:0/19942774 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5765+0+0 (secure 0 0 0) 0x7f18c40ba400 con 0x7f18d80cec60 2024-09-16T15:51:11.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.900+0000 7f18dca00640 1 --2- 172.21.15.151:0/19942774 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f18b8079ed0 0x7f18b807c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:11.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.901+0000 7f18dca00640 1 --2- 172.21.15.151:0/19942774 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f18b8079ed0 0x7f18b807c390 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f18c8005fd0 tx=0x7f18c80095e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:11.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:11.904+0000 7f18d4c00640 1 -- 172.21.15.151:0/19942774 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f18c4083e90 con 0x7f18d80cec60 2024-09-16T15:51:12.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:12.156+0000 7f18df400640 1 -- 172.21.15.151:0/19942774 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f18d80ae6b0 con 0x7f18d80cec60 2024-09-16T15:51:12.160 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:12.158+0000 7f18d4c00640 1 -- 172.21.15.151:0/19942774 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) ==== 190+0+147 (secure 0 0 0) 0x7f18c40888c0 con 0x7f18d80cec60 2024-09-16T15:51:12.160 INFO:teuthology.orchestra.run.smithi151.stdout:[client.0] 2024-09-16T15:51:12.161 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQDsUOhmf5PBKBAAKE4Y4zbA4h07IBNjPkyptw== 2024-09-16T15:51:12.161 INFO:teuthology.orchestra.run.smithi151.stdout: caps mds = "allow" 2024-09-16T15:51:12.161 INFO:teuthology.orchestra.run.smithi151.stdout: caps mon = "allow r" 2024-09-16T15:51:12.161 INFO:teuthology.orchestra.run.smithi151.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-16T15:51:12.161 INFO:teuthology.orchestra.run.smithi151.stderr:updated caps for client.0 2024-09-16T15:51:12.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:12.161+0000 7f18df400640 1 -- 172.21.15.151:0/19942774 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f18b8079ed0 msgr2=0x7f18b807c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:12.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:12.161+0000 7f18df400640 1 --2- 172.21.15.151:0/19942774 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f18b8079ed0 0x7f18b807c390 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f18c8005fd0 tx=0x7f18c80095e0 comp rx=0 tx=0).stop 2024-09-16T15:51:12.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:12.161+0000 7f18df400640 1 -- 172.21.15.151:0/19942774 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f18d80cec60 msgr2=0x7f18d815fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:12.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:12.161+0000 7f18df400640 1 --2- 172.21.15.151:0/19942774 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f18d80cec60 0x7f18d815fda0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f18c4000c00 tx=0x7f18c4003910 comp rx=0 tx=0).stop 2024-09-16T15:51:12.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:12.162+0000 7f18df400640 1 -- 172.21.15.151:0/19942774 shutdown_connections 2024-09-16T15:51:12.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:12.162+0000 7f18df400640 1 --2- 172.21.15.151:0/19942774 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f18b8079ed0 0x7f18b807c390 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:12.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:12.162+0000 7f18df400640 1 --2- 172.21.15.151:0/19942774 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f18d8161760 0x7f18d81602e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:12.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:12.162+0000 7f18df400640 1 --2- 172.21.15.151:0/19942774 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f18d80cec60 0x7f18d815fda0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:12.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:12.162+0000 7f18df400640 1 --2- 172.21.15.151:0/19942774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18d80af190 0x7f18d80cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:12.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:12.162+0000 7f18df400640 1 -- 172.21.15.151:0/19942774 >> 172.21.15.151:0/19942774 conn(0x7f18d80b1840 msgr2=0x7f18d815dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:12.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:12.162+0000 7f18df400640 1 -- 172.21.15.151:0/19942774 shutdown_connections 2024-09-16T15:51:12.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:12.162+0000 7f18df400640 1 -- 172.21.15.151:0/19942774 wait complete. 2024-09-16T15:51:12.180 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:51:12.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:12.335+0000 7f858e600640 1 -- 172.21.15.151:0/690027323 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8588073fb0 msgr2=0x7f8588074420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:12.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:12.335+0000 7f858e600640 1 --2- 172.21.15.151:0/690027323 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8588073fb0 0x7f8588074420 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f8588075130 tx=0x7f858002a9a0 comp rx=0 tx=0).stop 2024-09-16T15:51:12.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:12.335+0000 7f858e600640 1 -- 172.21.15.151:0/690027323 shutdown_connections 2024-09-16T15:51:12.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:12.335+0000 7f858e600640 1 --2- 172.21.15.151:0/690027323 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8588073fb0 0x7f8588074420 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:12.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:12.335+0000 7f858e600640 1 --2- 172.21.15.151:0/690027323 >> v2:172.21.15.167:3300/0 conn(0x7f858815f5a0 0x7f85880b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:12.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:12.335+0000 7f858e600640 1 -- 172.21.15.151:0/690027323 >> 172.21.15.151:0/690027323 conn(0x7f85880b1950 msgr2=0x7f85880b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:12.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:12.336+0000 7f858e600640 1 -- 172.21.15.151:0/690027323 shutdown_connections 2024-09-16T15:51:12.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:12.336+0000 7f858e600640 1 -- 172.21.15.151:0/690027323 wait complete. 2024-09-16T15:51:12.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:12.337+0000 7f858e600640 1 Processor -- start 2024-09-16T15:51:12.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:12.337+0000 7f858e600640 1 -- start start 2024-09-16T15:51:12.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:12.338+0000 7f858e600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8588073fb0 0x7f85881f4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:12.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:12.338+0000 7f858e600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f85880ccf00 0x7f858815e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:12.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:12.338+0000 7f858e600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f858815f5a0 0x7f8588156b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:12.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:12.338+0000 7f858e600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f858806f2e0 con 0x7f85880ccf00 2024-09-16T15:51:12.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:12.338+0000 7f858e600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f858806f160 con 0x7f8588073fb0 2024-09-16T15:51:12.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:12.338+0000 7f858e600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f858806f460 con 0x7f858815f5a0 2024-09-16T15:51:12.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:12.338+0000 7f8586a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f85880ccf00 0x7f858815e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:12.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:12.339+0000 7f8586a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f85880ccf00 0x7f858815e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52750/0 (socket says 172.21.15.151:52750) 2024-09-16T15:51:12.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:12.339+0000 7f8586a00640 1 -- 172.21.15.151:0/349477286 learned_addr learned my addr 172.21.15.151:0/349477286 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:51:12.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:12.339+0000 7f8587400640 1 --2- 172.21.15.151:0/349477286 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8588073fb0 0x7f85881f4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:12.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:12.339+0000 7f8587e00640 1 --2- 172.21.15.151:0/349477286 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f858815f5a0 0x7f8588156b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:12.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:12.339+0000 7f8586a00640 1 -- 172.21.15.151:0/349477286 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f858815f5a0 msgr2=0x7f8588156b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:12.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:12.339+0000 7f8586a00640 1 --2- 172.21.15.151:0/349477286 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f858815f5a0 0x7f8588156b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:12.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:12.339+0000 7f8586a00640 1 -- 172.21.15.151:0/349477286 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8588073fb0 msgr2=0x7f85881f4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:12.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:12.339+0000 7f8586a00640 1 --2- 172.21.15.151:0/349477286 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8588073fb0 0x7f85881f4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:12.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:12.339+0000 7f8586a00640 1 -- 172.21.15.151:0/349477286 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85880cb9e0 con 0x7f85880ccf00 2024-09-16T15:51:12.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:12.339+0000 7f8586a00640 1 --2- 172.21.15.151:0/349477286 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f85880ccf00 0x7f858815e1d0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f857400c430 tx=0x7f857400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:12.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:12.339+0000 7f857f400640 1 -- 172.21.15.151:0/349477286 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8574011790 con 0x7f85880ccf00 2024-09-16T15:51:12.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:12.340+0000 7f857f400640 1 -- 172.21.15.151:0/349477286 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8574017070 con 0x7f85880ccf00 2024-09-16T15:51:12.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:12.340+0000 7f857f400640 1 -- 172.21.15.151:0/349477286 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8574011d00 con 0x7f85880ccf00 2024-09-16T15:51:12.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:12.340+0000 7f858e600640 1 -- 172.21.15.151:0/349477286 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85880cbcc0 con 0x7f85880ccf00 2024-09-16T15:51:12.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:12.340+0000 7f858e600640 1 -- 172.21.15.151:0/349477286 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85880cc560 con 0x7f85880ccf00 2024-09-16T15:51:12.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:12.341+0000 7f858e600640 1 -- 172.21.15.151:0/349477286 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85880cc760 con 0x7f85880ccf00 2024-09-16T15:51:12.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:12.342+0000 7f857f400640 1 -- 172.21.15.151:0/349477286 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7f857401c460 con 0x7f85880ccf00 2024-09-16T15:51:12.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:12.343+0000 7f857f400640 1 --2- 172.21.15.151:0/349477286 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8568079ed0 0x7f856807c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:12.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:12.343+0000 7f857f400640 1 -- 172.21.15.151:0/349477286 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5765+0+0 (secure 0 0 0) 0x7f8574099e60 con 0x7f85880ccf00 2024-09-16T15:51:12.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:12.344+0000 7f8587400640 1 --2- 172.21.15.151:0/349477286 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8568079ed0 0x7f856807c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:12.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:12.344+0000 7f8587400640 1 --2- 172.21.15.151:0/349477286 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8568079ed0 0x7f856807c390 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f8578005fd0 tx=0x7f857800a250 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:12.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:12.346+0000 7f857f400640 1 -- 172.21.15.151:0/349477286 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f85880cc760 con 0x7f85880ccf00 2024-09-16T15:51:12.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:12.570+0000 7f858e600640 1 -- 172.21.15.151:0/349477286 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f85880cbe90 con 0x7f85880ccf00 2024-09-16T15:51:12.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:12.570+0000 7f857f400640 1 -- 172.21.15.151:0/349477286 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+5234 (secure 0 0 0) 0x7f8574063960 con 0x7f85880ccf00 2024-09-16T15:51:12.573 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:51:12.573 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":66,"btime":"2024-09-16T15:51:05:403851+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6680,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2638998637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":2638998637},{"type":"v1","addr":"172.21.15.151:6835","nonce":2638998637}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":7330,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3105097249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3105097249},{"type":"v1","addr":"172.21.15.167:6836","nonce":3105097249}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":7333,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/613324294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":613324294},{"type":"v1","addr":"172.21.15.167:6837","nonce":613324294}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:51:01.789380+0000","modified":"2024-09-16T15:51:04.820629+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7899},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7899":{"gid":7899,"name":"c","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.151:6834/1174862515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1174862515},{"type":"v1","addr":"172.21.15.151:6834","nonce":1174862515}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7899,"qdb_cluster":[7899]},"id":7}]} 2024-09-16T15:51:12.573 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 66 2024-09-16T15:51:12.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:12.573+0000 7f858e600640 1 -- 172.21.15.151:0/349477286 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8568079ed0 msgr2=0x7f856807c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:12.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:12.573+0000 7f858e600640 1 --2- 172.21.15.151:0/349477286 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8568079ed0 0x7f856807c390 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f8578005fd0 tx=0x7f857800a250 comp rx=0 tx=0).stop 2024-09-16T15:51:12.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:12.573+0000 7f858e600640 1 -- 172.21.15.151:0/349477286 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f85880ccf00 msgr2=0x7f858815e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:12.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:12.573+0000 7f858e600640 1 --2- 172.21.15.151:0/349477286 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f85880ccf00 0x7f858815e1d0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f857400c430 tx=0x7f857400c8f0 comp rx=0 tx=0).stop 2024-09-16T15:51:12.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:12.573+0000 7f858e600640 1 -- 172.21.15.151:0/349477286 shutdown_connections 2024-09-16T15:51:12.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:12.573+0000 7f858e600640 1 --2- 172.21.15.151:0/349477286 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8568079ed0 0x7f856807c390 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:12.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:12.573+0000 7f858e600640 1 --2- 172.21.15.151:0/349477286 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f858815f5a0 0x7f8588156b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:12.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:12.573+0000 7f858e600640 1 --2- 172.21.15.151:0/349477286 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f85880ccf00 0x7f858815e1d0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:12.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:12.573+0000 7f858e600640 1 --2- 172.21.15.151:0/349477286 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8588073fb0 0x7f85881f4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:12.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:12.573+0000 7f858e600640 1 -- 172.21.15.151:0/349477286 >> 172.21.15.151:0/349477286 conn(0x7f85880b1950 msgr2=0x7f85880cf1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:12.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:12.573+0000 7f858e600640 1 -- 172.21.15.151:0/349477286 shutdown_connections 2024-09-16T15:51:12.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:12.573+0000 7f858e600640 1 -- 172.21.15.151:0/349477286 wait complete. 2024-09-16T15:51:12.994 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 66, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-16T15:51:01.789380+0000', 'modified': '2024-09-16T15:51:04.820629+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7899}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7899': {'gid': 7899, 'name': 'c', 'rank': 0, 'incarnation': 62, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.151:6834/1174862515', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6832', 'nonce': 1174862515}, {'type': 'v1', 'addr': '172.21.15.151:6834', 'nonce': 1174862515}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7899, 'qdb_cluster': [7899]} 2024-09-16T15:51:12.994 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-16T15:51:12.994 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-16T15:51:13.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.152+0000 7f6a02e00640 1 -- 172.21.15.151:0/2221136302 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69fc0cbdc0 msgr2=0x7f69fc0cd5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:13.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.152+0000 7f6a02e00640 1 --2- 172.21.15.151:0/2221136302 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69fc0cbdc0 0x7f69fc0cd5f0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f69e8005fd0 tx=0x7f69e802a5c0 comp rx=0 tx=0).stop 2024-09-16T15:51:13.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.152+0000 7f6a02e00640 1 -- 172.21.15.151:0/2221136302 shutdown_connections 2024-09-16T15:51:13.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.152+0000 7f6a02e00640 1 --2- 172.21.15.151:0/2221136302 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f69fc0cdb30 0x7f69fc15a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:13.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.152+0000 7f6a02e00640 1 --2- 172.21.15.151:0/2221136302 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69fc0cbdc0 0x7f69fc0cd5f0 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:13.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.152+0000 7f6a02e00640 1 -- 172.21.15.151:0/2221136302 >> 172.21.15.151:0/2221136302 conn(0x7f69fc0cfa20 msgr2=0x7f69fc0cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:13.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.152+0000 7f6a02e00640 1 -- 172.21.15.151:0/2221136302 shutdown_connections 2024-09-16T15:51:13.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.152+0000 7f6a02e00640 1 -- 172.21.15.151:0/2221136302 wait complete. 2024-09-16T15:51:13.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.153+0000 7f6a02e00640 1 Processor -- start 2024-09-16T15:51:13.155 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.154+0000 7f6a02e00640 1 -- start start 2024-09-16T15:51:13.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.154+0000 7f6a02e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f69fc0706c0 0x7f69fc153210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:13.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.155+0000 7f6a02e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69fc0cbdc0 0x7f69fc1f7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:13.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.155+0000 7f6a02e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69fc0cdb30 0x7f69fc1f8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:13.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.155+0000 7f6a02e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f69fc15aef0 con 0x7f69fc0cbdc0 2024-09-16T15:51:13.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.155+0000 7f6a02e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f69fc15ad70 con 0x7f69fc0cdb30 2024-09-16T15:51:13.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.155+0000 7f6a02e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f69fc15b070 con 0x7f69fc0706c0 2024-09-16T15:51:13.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.155+0000 7f6a01000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69fc0cbdc0 0x7f69fc1f7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:13.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.155+0000 7f6a01000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69fc0cbdc0 0x7f69fc1f7b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52758/0 (socket says 172.21.15.151:52758) 2024-09-16T15:51:13.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.155+0000 7f6a01000640 1 -- 172.21.15.151:0/3340929232 learned_addr learned my addr 172.21.15.151:0/3340929232 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:51:13.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.155+0000 7f6a01a00640 1 --2- 172.21.15.151:0/3340929232 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f69fc0706c0 0x7f69fc153210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:13.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.155+0000 7f6a02400640 1 --2- 172.21.15.151:0/3340929232 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69fc0cdb30 0x7f69fc1f8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:13.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.155+0000 7f6a01000640 1 -- 172.21.15.151:0/3340929232 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f69fc0706c0 msgr2=0x7f69fc153210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:13.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.155+0000 7f6a01000640 1 --2- 172.21.15.151:0/3340929232 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f69fc0706c0 0x7f69fc153210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:13.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.155+0000 7f6a01000640 1 -- 172.21.15.151:0/3340929232 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69fc0cdb30 msgr2=0x7f69fc1f8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:13.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.155+0000 7f6a01000640 1 --2- 172.21.15.151:0/3340929232 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69fc0cdb30 0x7f69fc1f8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:13.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.155+0000 7f6a01000640 1 -- 172.21.15.151:0/3340929232 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69fc0c7cd0 con 0x7f69fc0cbdc0 2024-09-16T15:51:13.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.156+0000 7f6a01000640 1 --2- 172.21.15.151:0/3340929232 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69fc0cbdc0 0x7f69fc1f7b00 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f69e8009850 tx=0x7f69e80038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:13.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.156+0000 7f69f9600640 1 -- 172.21.15.151:0/3340929232 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e803f070 con 0x7f69fc0cbdc0 2024-09-16T15:51:13.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.156+0000 7f69f9600640 1 -- 172.21.15.151:0/3340929232 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f69e80047a0 con 0x7f69fc0cbdc0 2024-09-16T15:51:13.160 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.156+0000 7f69f9600640 1 -- 172.21.15.151:0/3340929232 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e8035030 con 0x7f69fc0cbdc0 2024-09-16T15:51:13.160 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.156+0000 7f6a02e00640 1 -- 172.21.15.151:0/3340929232 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69fc0b4e00 con 0x7f69fc0cbdc0 2024-09-16T15:51:13.160 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.156+0000 7f6a02e00640 1 -- 172.21.15.151:0/3340929232 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69fc0b53c0 con 0x7f69fc0cbdc0 2024-09-16T15:51:13.161 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.159+0000 7f69f9600640 1 -- 172.21.15.151:0/3340929232 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7f69e8042070 con 0x7f69fc0cbdc0 2024-09-16T15:51:13.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.160+0000 7f69f9600640 1 --2- 172.21.15.151:0/3340929232 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f69dc079f00 0x7f69dc07c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:13.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.160+0000 7f6a073d1740 1 Processor -- start 2024-09-16T15:51:13.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.160+0000 7f69f9600640 1 -- 172.21.15.151:0/3340929232 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5765+0+0 (secure 0 0 0) 0x7f69e80ba420 con 0x7f69fc0cbdc0 2024-09-16T15:51:13.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.160+0000 7f6a073d1740 1 -- start start 2024-09-16T15:51:13.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.161+0000 7f6a01a00640 1 --2- 172.21.15.151:0/3340929232 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f69dc079f00 0x7f69dc07c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:13.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.161+0000 7f6a073d1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556029b08aa0 0x556029b08e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:13.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.161+0000 7f6a073d1740 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556029ac3c10 0x556029b07580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:13.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.161+0000 7f6a073d1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556029ac4190 0x556029b07ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:13.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.161+0000 7f6a073d1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x556029a46430 con 0x556029ac4190 2024-09-16T15:51:13.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.161+0000 7f6a073d1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556029a365d0 con 0x556029b08aa0 2024-09-16T15:51:13.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.161+0000 7f6a073d1740 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x556029a25a00 con 0x556029ac3c10 2024-09-16T15:51:13.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.161+0000 7f6a02400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556029ac4190 0x556029b07ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:13.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.161+0000 7f6a02400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556029ac4190 0x556029b07ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52768/0 (socket says 172.21.15.151:52768) 2024-09-16T15:51:13.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.161+0000 7f6a01a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556029b08aa0 0x556029b08e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:13.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.161+0000 7f6a02400640 1 -- 172.21.15.151:0/3511711147 learned_addr learned my addr 172.21.15.151:0/3511711147 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:51:13.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.162+0000 7f6a01000640 1 --2- 172.21.15.151:0/3511711147 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556029ac3c10 0x556029b07580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:13.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.162+0000 7f6a01a00640 1 --2- 172.21.15.151:0/3340929232 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f69dc079f00 0x7f69dc07c3c0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f69f00017c0 tx=0x7f69f0009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:13.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.162+0000 7f6a02400640 1 -- 172.21.15.151:0/3511711147 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556029ac3c10 msgr2=0x556029b07580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:51:13.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.162+0000 7f6a02400640 1 --2- 172.21.15.151:0/3511711147 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556029ac3c10 0x556029b07580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:13.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.162+0000 7f6a02400640 1 -- 172.21.15.151:0/3511711147 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556029b08aa0 msgr2=0x556029b08e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:51:13.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.162+0000 7f6a02400640 1 --2- 172.21.15.151:0/3511711147 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556029b08aa0 0x556029b08e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:13.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.162+0000 7f6a02400640 1 -- 172.21.15.151:0/3511711147 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560299ee520 con 0x556029ac4190 2024-09-16T15:51:13.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.162+0000 7f6a02400640 1 --2- 172.21.15.151:0/3511711147 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556029ac4190 0x556029b07ac0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f69f400ae90 tx=0x7f69f400e820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c7f9209849c78b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:13.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.162+0000 7f69ee000640 1 -- 172.21.15.151:0/3511711147 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f401c070 con 0x556029ac4190 2024-09-16T15:51:13.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.162+0000 7f69ee000640 1 -- 172.21.15.151:0/3511711147 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f69f400d650 con 0x556029ac4190 2024-09-16T15:51:13.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.162+0000 7f69ee000640 1 -- 172.21.15.151:0/3511711147 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f400da40 con 0x556029ac4190 2024-09-16T15:51:13.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.163+0000 7f6a073d1740 1 -- 172.21.15.151:0/3511711147 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556029ac4190 msgr2=0x556029b07ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:13.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.163+0000 7f6a073d1740 1 --2- 172.21.15.151:0/3511711147 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556029ac4190 0x556029b07ac0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f69f400ae90 tx=0x7f69f400e820 comp rx=0 tx=0).stop 2024-09-16T15:51:13.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.163+0000 7f6a073d1740 1 -- 172.21.15.151:0/3511711147 shutdown_connections 2024-09-16T15:51:13.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.163+0000 7f6a073d1740 1 --2- 172.21.15.151:0/3511711147 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556029ac4190 0x556029b07ac0 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:13.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.163+0000 7f6a073d1740 1 --2- 172.21.15.151:0/3511711147 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556029ac3c10 0x556029b07580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:13.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.163+0000 7f6a073d1740 1 --2- 172.21.15.151:0/3511711147 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556029b08aa0 0x556029b08e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:13.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.163+0000 7f6a073d1740 1 -- 172.21.15.151:0/3511711147 >> 172.21.15.151:0/3511711147 conn(0x556029abf1e0 msgr2=0x556029ac2c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:13.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.163+0000 7f6a073d1740 1 -- 172.21.15.151:0/3511711147 shutdown_connections 2024-09-16T15:51:13.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.163+0000 7f6a073d1740 1 -- 172.21.15.151:0/3511711147 wait complete. 2024-09-16T15:51:13.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.164+0000 7f6a073d1740 1 -- start start 2024-09-16T15:51:13.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.164+0000 7f6a073d1740 1 Processor -- start 2024-09-16T15:51:13.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.165+0000 7f6a073d1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556029ac4190 0x556029aec180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:13.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.165+0000 7f6a073d1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556029b08aa0 0x556029b00030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:13.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.165+0000 7f6a073d1740 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556029aea9e0 0x556029b078e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:13.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.165+0000 7f6a073d1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5560299cc330 con 0x556029ac4190 2024-09-16T15:51:13.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.165+0000 7f6a073d1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556029a29810 con 0x556029b08aa0 2024-09-16T15:51:13.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.165+0000 7f6a073d1740 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x556029796620 con 0x556029aea9e0 2024-09-16T15:51:13.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.165+0000 7f6a073d1740 10 client.0 _ll_register_callbacks cb 0x5560299d1420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T15:51:13.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.165+0000 7f6a01a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556029ac4190 0x556029aec180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:13.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.165+0000 7f6a01a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556029ac4190 0x556029aec180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52778/0 (socket says 172.21.15.151:52778) 2024-09-16T15:51:13.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.165+0000 7f6a01a00640 1 -- 172.21.15.151:0/1326201776 learned_addr learned my addr 172.21.15.151:0/1326201776 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:51:13.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.166+0000 7f6a01a00640 1 -- 172.21.15.151:0/1326201776 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556029aea9e0 msgr2=0x556029b078e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:51:13.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.166+0000 7f6a01a00640 1 --2- 172.21.15.151:0/1326201776 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556029aea9e0 0x556029b078e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:13.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.166+0000 7f6a01000640 1 --2- 172.21.15.151:0/1326201776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556029b08aa0 0x556029b00030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:13.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.166+0000 7f6a01a00640 1 -- 172.21.15.151:0/1326201776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556029b08aa0 msgr2=0x556029b00030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:51:13.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.166+0000 7f6a01a00640 1 --2- 172.21.15.151:0/1326201776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556029b08aa0 0x556029b00030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:13.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.166+0000 7f6a01a00640 1 -- 172.21.15.151:0/1326201776 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556029787850 con 0x556029ac4190 2024-09-16T15:51:13.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.166+0000 7f6a01a00640 1 --2- 172.21.15.151:0/1326201776 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556029ac4190 0x556029aec180 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f69f00156f0 tx=0x7f69f0000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d564ffe5de2815dd server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:13.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.166+0000 7f69e2a00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-16T15:51:13.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.166+0000 7f69e2a00640 1 -- 172.21.15.151:0/1326201776 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f0026060 con 0x556029ac4190 2024-09-16T15:51:13.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.166+0000 7f6a073d1740 1 -- 172.21.15.151:0/1326201776 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69f400d650 con 0x556029ac4190 2024-09-16T15:51:13.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.166+0000 7f69e2a00640 1 -- 172.21.15.151:0/1326201776 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 5765+0+0 (secure 0 0 0) 0x7f69f0014810 con 0x556029ac4190 2024-09-16T15:51:13.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.166+0000 7f69e2a00640 1 -- 172.21.15.151:0/1326201776 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f69f002c050 con 0x556029ac4190 2024-09-16T15:51:13.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.166+0000 7f6a073d1740 10 client.8058 fetch_fsmap learned FSMap version 66 2024-09-16T15:51:13.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.166+0000 7f6a073d1740 1 -- 172.21.15.151:0/1326201776 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x5560297fe6b0 con 0x556029ac4190 2024-09-16T15:51:13.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.167+0000 7f69e2a00640 1 -- 172.21.15.151:0/1326201776 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f69f400d650 con 0x556029ac4190 2024-09-16T15:51:13.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.167+0000 7f69e2a00640 1 -- 172.21.15.151:0/1326201776 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 66) ==== 3502+0+0 (secure 0 0 0) 0x7f69f002f020 con 0x556029ac4190 2024-09-16T15:51:13.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.167+0000 7f6a073d1740 10 client.8058 fetch_fsmap finished waiting for FSMap version 66 2024-09-16T15:51:13.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.167+0000 7f6a073d1740 10 client.8058 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-16T15:51:13.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.167+0000 7f6a073d1740 20 client.8058 populate_metadata read hostname 'smithi151' 2024-09-16T15:51:13.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.167+0000 7f6a073d1740 1 --2- 172.21.15.151:0/1326201776 >> [v2:172.21.15.151:6832/1174862515,v1:172.21.15.151:6834/1174862515] conn(0x556029aff040 0x556029aff420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:13.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.167+0000 7f6a073d1740 4 client.8058 mds_command: new command op to 7899 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:51:13.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.167+0000 7f6a073d1740 1 -- 172.21.15.151:0/1326201776 --> [v2:172.21.15.151:6832/1174862515,v1:172.21.15.151:6834/1174862515] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556029a38520 con 0x556029aff040 2024-09-16T15:51:13.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.167+0000 7f6a01000640 1 --2- 172.21.15.151:0/1326201776 >> [v2:172.21.15.151:6832/1174862515,v1:172.21.15.151:6834/1174862515] conn(0x556029aff040 0x556029aff420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:13.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.168+0000 7f6a01000640 1 --2- 172.21.15.151:0/1326201776 >> [v2:172.21.15.151:6832/1174862515,v1:172.21.15.151:6834/1174862515] conn(0x556029aff040 0x556029aff420 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a48f8446822f05de server_cookie=d72eb8451bd00028 in_seq=0 out_seq=0 2024-09-16T15:51:13.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.168+0000 7f69e2a00640 10 client.8058 ms_handle_connect on v2:172.21.15.151:6832/1174862515 2024-09-16T15:51:13.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.169+0000 7f69e2a00640 1 -- 172.21.15.151:0/1326201776 <== mds.0 v2:172.21.15.151:6832/1174862515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556029a38520 con 0x556029aff040 2024-09-16T15:51:13.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.169+0000 7f69e2a00640 10 client.8058 handle_command_reply: tid=0 2024-09-16T15:51:13.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.169+0000 7f6a073d1740 1 client.8058 shutdown 2024-09-16T15:51:13.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.169+0000 7f6a073d1740 1 -- 172.21.15.151:0/1326201776 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556029ac4190 msgr2=0x556029aec180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:13.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.169+0000 7f6a073d1740 1 --2- 172.21.15.151:0/1326201776 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556029ac4190 0x556029aec180 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f69f00156f0 tx=0x7f69f0000ed0 comp rx=0 tx=0).stop 2024-09-16T15:51:13.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.169+0000 7f6a073d1740 1 -- 172.21.15.151:0/1326201776 shutdown_connections 2024-09-16T15:51:13.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.169+0000 7f6a073d1740 1 --2- 172.21.15.151:0/1326201776 >> [v2:172.21.15.151:6832/1174862515,v1:172.21.15.151:6834/1174862515] conn(0x556029aff040 0x556029aff420 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:13.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.169+0000 7f6a073d1740 1 --2- 172.21.15.151:0/1326201776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556029b08aa0 0x556029b00030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:13.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.169+0000 7f6a073d1740 1 --2- 172.21.15.151:0/1326201776 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556029aea9e0 0x556029b078e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:13.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.169+0000 7f6a073d1740 1 --2- 172.21.15.151:0/1326201776 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556029ac4190 0x556029aec180 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:13.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.169+0000 7f6a073d1740 1 -- 172.21.15.151:0/1326201776 >> 172.21.15.151:0/1326201776 conn(0x556029ac3c10 msgr2=0x556029abc910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:13.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.169+0000 7f69e2a00640 0 client.8058 ms_handle_reset on v2:172.21.15.151:6832/1174862515 2024-09-16T15:51:13.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.169+0000 7f6a073d1740 1 -- 172.21.15.151:0/1326201776 shutdown_connections 2024-09-16T15:51:13.174 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.169+0000 7f6a073d1740 1 -- 172.21.15.151:0/1326201776 wait complete. 2024-09-16T15:51:13.174 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.169+0000 7f6a073d1740 20 client.8058 trim_cache size 0 max 16384 2024-09-16T15:51:13.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.188+0000 7f6a073d1740 1 Processor -- start 2024-09-16T15:51:13.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.188+0000 7f6a073d1740 1 -- start start 2024-09-16T15:51:13.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.189+0000 7f6a073d1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556029ac4190 0x556029ae4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:13.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.189+0000 7f6a073d1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556029aea9e0 0x556029b2b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:13.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.189+0000 7f6a073d1740 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556029b08aa0 0x556029b2b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:13.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.189+0000 7f6a073d1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f69fc200160 con 0x556029ac4190 2024-09-16T15:51:13.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.189+0000 7f6a073d1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f69fc0c8f60 con 0x556029aea9e0 2024-09-16T15:51:13.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.189+0000 7f6a073d1740 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f69fc0c95b0 con 0x556029b08aa0 2024-09-16T15:51:13.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.189+0000 7f6a01a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556029ac4190 0x556029ae4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:13.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.189+0000 7f6a01a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556029ac4190 0x556029ae4dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52790/0 (socket says 172.21.15.151:52790) 2024-09-16T15:51:13.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.189+0000 7f6a01a00640 1 -- 172.21.15.151:0/3734247805 learned_addr learned my addr 172.21.15.151:0/3734247805 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:51:13.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.189+0000 7f6a01000640 1 --2- 172.21.15.151:0/3734247805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556029aea9e0 0x556029b2b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:13.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.189+0000 7f6a02400640 1 --2- 172.21.15.151:0/3734247805 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556029b08aa0 0x556029b2b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:13.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.189+0000 7f6a01a00640 1 -- 172.21.15.151:0/3734247805 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556029b08aa0 msgr2=0x556029b2b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:51:13.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.189+0000 7f6a01a00640 1 --2- 172.21.15.151:0/3734247805 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556029b08aa0 0x556029b2b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:13.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.189+0000 7f6a01a00640 1 -- 172.21.15.151:0/3734247805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556029aea9e0 msgr2=0x556029b2b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:51:13.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.189+0000 7f6a01a00640 1 --2- 172.21.15.151:0/3734247805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556029aea9e0 0x556029b2b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:13.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.189+0000 7f6a01a00640 1 -- 172.21.15.151:0/3734247805 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556029b08750 con 0x556029ac4190 2024-09-16T15:51:13.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.189+0000 7f6a01000640 1 --2- 172.21.15.151:0/3734247805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556029aea9e0 0x556029b2b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:51:13.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.189+0000 7f6a02400640 1 --2- 172.21.15.151:0/3734247805 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556029b08aa0 0x556029b2b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:51:13.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.190+0000 7f6a01a00640 1 --2- 172.21.15.151:0/3734247805 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556029ac4190 0x556029ae4dd0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f69f0011010 tx=0x7f69f0007f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db11930651544494 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:13.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.190+0000 7f69ed600640 1 -- 172.21.15.151:0/3734247805 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f0034070 con 0x556029ac4190 2024-09-16T15:51:13.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.190+0000 7f69ed600640 1 -- 172.21.15.151:0/3734247805 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f69f001c040 con 0x556029ac4190 2024-09-16T15:51:13.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.190+0000 7f69ed600640 1 -- 172.21.15.151:0/3734247805 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f001b040 con 0x556029ac4190 2024-09-16T15:51:13.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.190+0000 7f6a073d1740 1 -- 172.21.15.151:0/3734247805 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556029ac4190 msgr2=0x556029ae4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:13.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.190+0000 7f6a073d1740 1 --2- 172.21.15.151:0/3734247805 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556029ac4190 0x556029ae4dd0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f69f0011010 tx=0x7f69f0007f20 comp rx=0 tx=0).stop 2024-09-16T15:51:13.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.191+0000 7f6a073d1740 1 -- 172.21.15.151:0/3734247805 shutdown_connections 2024-09-16T15:51:13.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.191+0000 7f6a073d1740 1 --2- 172.21.15.151:0/3734247805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556029aea9e0 0x556029b2b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:13.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.191+0000 7f6a073d1740 1 --2- 172.21.15.151:0/3734247805 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556029b08aa0 0x556029b2b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:13.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.191+0000 7f6a073d1740 1 --2- 172.21.15.151:0/3734247805 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556029ac4190 0x556029ae4dd0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:13.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.191+0000 7f6a073d1740 1 -- 172.21.15.151:0/3734247805 >> 172.21.15.151:0/3734247805 conn(0x556029ac3c10 msgr2=0x556029b02510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:13.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.191+0000 7f6a073d1740 1 -- 172.21.15.151:0/3734247805 shutdown_connections 2024-09-16T15:51:13.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.191+0000 7f6a073d1740 1 -- 172.21.15.151:0/3734247805 wait complete. 2024-09-16T15:51:13.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.191+0000 7f6a073d1740 1 -- start start 2024-09-16T15:51:13.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.191+0000 7f6a073d1740 1 Processor -- start 2024-09-16T15:51:13.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.192+0000 7f6a073d1740 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556029ac4190 0x556029b8c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:13.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.192+0000 7f6a073d1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556029aea9e0 0x556029abc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:13.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.192+0000 7f6a073d1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556029b08aa0 0x556029b6eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:13.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.192+0000 7f6a073d1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x556029ad80d0 con 0x556029b08aa0 2024-09-16T15:51:13.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.192+0000 7f6a073d1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556029ad8520 con 0x556029aea9e0 2024-09-16T15:51:13.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.192+0000 7f6a073d1740 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x556029aeca90 con 0x556029ac4190 2024-09-16T15:51:13.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.192+0000 7f6a073d1740 10 client.0 _ll_register_callbacks cb 0x7f69f000ff10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T15:51:13.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.193+0000 7f6a02400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556029b08aa0 0x556029b6eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:13.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.193+0000 7f6a02400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556029b08aa0 0x556029b6eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52796/0 (socket says 172.21.15.151:52796) 2024-09-16T15:51:13.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.193+0000 7f6a02400640 1 -- 172.21.15.151:0/734246071 learned_addr learned my addr 172.21.15.151:0/734246071 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:51:13.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.193+0000 7f6a01000640 1 --2- 172.21.15.151:0/734246071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556029aea9e0 0x556029abc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:13.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.193+0000 7f6a01a00640 1 --2- 172.21.15.151:0/734246071 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556029ac4190 0x556029b8c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:13.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.193+0000 7f6a02400640 1 -- 172.21.15.151:0/734246071 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556029ac4190 msgr2=0x556029b8c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:51:13.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.193+0000 7f6a02400640 1 --2- 172.21.15.151:0/734246071 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556029ac4190 0x556029b8c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:13.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.193+0000 7f6a02400640 1 -- 172.21.15.151:0/734246071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556029aea9e0 msgr2=0x556029abc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:51:13.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.193+0000 7f6a02400640 1 --2- 172.21.15.151:0/734246071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556029aea9e0 0x556029abc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:13.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.193+0000 7f6a02400640 1 -- 172.21.15.151:0/734246071 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556029b07150 con 0x556029b08aa0 2024-09-16T15:51:13.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.193+0000 7f6a01a00640 1 --2- 172.21.15.151:0/734246071 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556029ac4190 0x556029b8c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:51:13.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.193+0000 7f6a02400640 1 --2- 172.21.15.151:0/734246071 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556029b08aa0 0x556029b6eab0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f69f401a040 tx=0x7f69f400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a88dbffe423b3381 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:13.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.193+0000 7f69e2a00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-16T15:51:13.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.194+0000 7f69e2a00640 1 -- 172.21.15.151:0/734246071 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f401c040 con 0x556029b08aa0 2024-09-16T15:51:13.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.194+0000 7f69e2a00640 1 -- 172.21.15.151:0/734246071 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 5765+0+0 (secure 0 0 0) 0x7f69f4014040 con 0x556029b08aa0 2024-09-16T15:51:13.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.194+0000 7f6a073d1740 1 -- 172.21.15.151:0/734246071 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69f001c040 con 0x556029b08aa0 2024-09-16T15:51:13.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.194+0000 7f69e2a00640 1 -- 172.21.15.151:0/734246071 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f69f402e050 con 0x556029b08aa0 2024-09-16T15:51:13.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.194+0000 7f6a073d1740 10 client.8064 fetch_fsmap learned FSMap version 66 2024-09-16T15:51:13.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.194+0000 7f6a073d1740 1 -- 172.21.15.151:0/734246071 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x556029b8c960 con 0x556029b08aa0 2024-09-16T15:51:13.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.194+0000 7f69e2a00640 1 -- 172.21.15.151:0/734246071 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f69f001c040 con 0x556029b08aa0 2024-09-16T15:51:13.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.194+0000 7f69e2a00640 1 -- 172.21.15.151:0/734246071 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 66) ==== 3502+0+0 (secure 0 0 0) 0x7f69f4031020 con 0x556029b08aa0 2024-09-16T15:51:13.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.194+0000 7f6a073d1740 10 client.8064 fetch_fsmap finished waiting for FSMap version 66 2024-09-16T15:51:13.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.194+0000 7f6a073d1740 10 client.8064 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-16T15:51:13.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.194+0000 7f6a073d1740 20 client.8064 populate_metadata read hostname 'smithi151' 2024-09-16T15:51:13.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.194+0000 7f6a073d1740 1 --2- 172.21.15.151:0/734246071 >> [v2:172.21.15.151:6832/1174862515,v1:172.21.15.151:6834/1174862515] conn(0x556029b2b6d0 0x556029ac0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:13.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.194+0000 7f6a073d1740 4 client.8064 mds_command: new command op to 7899 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T15:51:13.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.194+0000 7f6a073d1740 1 -- 172.21.15.151:0/734246071 --> [v2:172.21.15.151:6832/1174862515,v1:172.21.15.151:6834/1174862515] -- command(tid 0: {"prefix": "status"}) -- 0x556029aeb450 con 0x556029b2b6d0 2024-09-16T15:51:13.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.194+0000 7f6a01a00640 1 --2- 172.21.15.151:0/734246071 >> [v2:172.21.15.151:6832/1174862515,v1:172.21.15.151:6834/1174862515] conn(0x556029b2b6d0 0x556029ac0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:13.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.195+0000 7f69e2a00640 10 client.8064 ms_handle_connect on v2:172.21.15.151:6832/1174862515 2024-09-16T15:51:13.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.195+0000 7f6a01a00640 1 --2- 172.21.15.151:0/734246071 >> [v2:172.21.15.151:6832/1174862515,v1:172.21.15.151:6834/1174862515] conn(0x556029b2b6d0 0x556029ac0660 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ec48c15588117d server_cookie=e9e9c04abafb771b in_seq=0 out_seq=0 2024-09-16T15:51:13.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.195+0000 7f69e2a00640 1 -- 172.21.15.151:0/734246071 <== mds.0 v2:172.21.15.151:6832/1174862515 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x556029aeb450 con 0x556029b2b6d0 2024-09-16T15:51:13.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.195+0000 7f69e2a00640 10 client.8064 handle_command_reply: tid=0 2024-09-16T15:51:13.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.195+0000 7f6a073d1740 1 client.8064 shutdown 2024-09-16T15:51:13.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.195+0000 7f6a073d1740 1 -- 172.21.15.151:0/734246071 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556029b08aa0 msgr2=0x556029b6eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:13.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.195+0000 7f6a073d1740 1 --2- 172.21.15.151:0/734246071 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556029b08aa0 0x556029b6eab0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f69f401a040 tx=0x7f69f400dcf0 comp rx=0 tx=0).stop 2024-09-16T15:51:13.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.195+0000 7f6a073d1740 1 -- 172.21.15.151:0/734246071 shutdown_connections 2024-09-16T15:51:13.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.195+0000 7f6a073d1740 1 --2- 172.21.15.151:0/734246071 >> [v2:172.21.15.151:6832/1174862515,v1:172.21.15.151:6834/1174862515] conn(0x556029b2b6d0 0x556029ac0660 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:13.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.195+0000 7f6a073d1740 1 --2- 172.21.15.151:0/734246071 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556029aea9e0 0x556029abc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:13.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.195+0000 7f6a073d1740 1 --2- 172.21.15.151:0/734246071 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556029b08aa0 0x556029b6eab0 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:13.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.195+0000 7f6a073d1740 1 --2- 172.21.15.151:0/734246071 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556029ac4190 0x556029b8c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:13.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.195+0000 7f69e2a00640 0 client.8064 ms_handle_reset on v2:172.21.15.151:6832/1174862515 2024-09-16T15:51:13.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.195+0000 7f6a073d1740 1 -- 172.21.15.151:0/734246071 >> 172.21.15.151:0/734246071 conn(0x556029ac3c10 msgr2=0x556029aebd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:13.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.196+0000 7f6a073d1740 1 -- 172.21.15.151:0/734246071 shutdown_connections 2024-09-16T15:51:13.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.196+0000 7f6a073d1740 1 -- 172.21.15.151:0/734246071 wait complete. 2024-09-16T15:51:13.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.196+0000 7f6a073d1740 20 client.8064 trim_cache size 0 max 16384 2024-09-16T15:51:13.202 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-16T15:51:13.202 INFO:teuthology.orchestra.run.smithi151.stdout: "cluster_fsid": "1757be18-e41a-4446-bb4f-ee4ac81c39df", 2024-09-16T15:51:13.202 INFO:teuthology.orchestra.run.smithi151.stdout: "whoami": 0, 2024-09-16T15:51:13.202 INFO:teuthology.orchestra.run.smithi151.stdout: "id": 7899, 2024-09-16T15:51:13.202 INFO:teuthology.orchestra.run.smithi151.stdout: "want_state": "up:active", 2024-09-16T15:51:13.202 INFO:teuthology.orchestra.run.smithi151.stdout: "state": "up:active", 2024-09-16T15:51:13.202 INFO:teuthology.orchestra.run.smithi151.stdout: "fs_name": "cephfs", 2024-09-16T15:51:13.203 INFO:teuthology.orchestra.run.smithi151.stdout: "rank_uptime": 10.808201431000001, 2024-09-16T15:51:13.203 INFO:teuthology.orchestra.run.smithi151.stdout: "mdsmap_epoch": 66, 2024-09-16T15:51:13.203 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_epoch": 104, 2024-09-16T15:51:13.203 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_epoch_barrier": 104, 2024-09-16T15:51:13.203 INFO:teuthology.orchestra.run.smithi151.stdout: "uptime": 15.142300906999999 2024-09-16T15:51:13.203 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-16T15:51:13.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.196+0000 7f6a02e00640 1 -- 172.21.15.151:0/3340929232 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f69dc079f00 msgr2=0x7f69dc07c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:13.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.196+0000 7f6a02e00640 1 --2- 172.21.15.151:0/3340929232 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f69dc079f00 0x7f69dc07c3c0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f69f00017c0 tx=0x7f69f0009040 comp rx=0 tx=0).stop 2024-09-16T15:51:13.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.196+0000 7f6a02e00640 1 -- 172.21.15.151:0/3340929232 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69fc0cbdc0 msgr2=0x7f69fc1f7b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:13.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.196+0000 7f6a02e00640 1 --2- 172.21.15.151:0/3340929232 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69fc0cbdc0 0x7f69fc1f7b00 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f69e8009850 tx=0x7f69e80038a0 comp rx=0 tx=0).stop 2024-09-16T15:51:13.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.196+0000 7f6a02e00640 1 -- 172.21.15.151:0/3340929232 shutdown_connections 2024-09-16T15:51:13.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.196+0000 7f6a02e00640 1 --2- 172.21.15.151:0/3340929232 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f69dc079f00 0x7f69dc07c3c0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:13.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.196+0000 7f6a02e00640 1 --2- 172.21.15.151:0/3340929232 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69fc0cdb30 0x7f69fc1f8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:13.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.196+0000 7f6a02e00640 1 --2- 172.21.15.151:0/3340929232 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69fc0cbdc0 0x7f69fc1f7b00 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:13.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.196+0000 7f6a02e00640 1 --2- 172.21.15.151:0/3340929232 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f69fc0706c0 0x7f69fc153210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:13.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.196+0000 7f6a02e00640 1 -- 172.21.15.151:0/3340929232 >> 172.21.15.151:0/3340929232 conn(0x7f69fc0cfa20 msgr2=0x7f69fc0ce040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:13.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.196+0000 7f6a02e00640 1 -- 172.21.15.151:0/3340929232 shutdown_connections 2024-09-16T15:51:13.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.197+0000 7f6a02e00640 1 -- 172.21.15.151:0/3340929232 wait complete. 2024-09-16T15:51:13.215 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-16T15:51:13.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.372+0000 7fc8cbe00640 1 -- 172.21.15.151:0/442697561 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8cc1612d0 msgr2=0x7fc8cc0b4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:13.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.372+0000 7fc8cbe00640 1 --2- 172.21.15.151:0/442697561 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8cc1612d0 0x7fc8cc0b4e00 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc0c7390 tx=0x7fc8bc02a560 comp rx=0 tx=0).stop 2024-09-16T15:51:13.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.372+0000 7fc8cbe00640 1 -- 172.21.15.151:0/442697561 shutdown_connections 2024-09-16T15:51:13.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.372+0000 7fc8cbe00640 1 --2- 172.21.15.151:0/442697561 >> v2:172.21.15.167:3300/0 conn(0x7fc8cc0d4980 0x7fc8cc0d1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:13.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.372+0000 7fc8cbe00640 1 --2- 172.21.15.151:0/442697561 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8cc1612d0 0x7fc8cc0b4e00 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:13.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.372+0000 7fc8cbe00640 1 -- 172.21.15.151:0/442697561 >> 172.21.15.151:0/442697561 conn(0x7fc8cc0d3df0 msgr2=0x7fc8cc0d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:13.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.372+0000 7fc8cbe00640 1 -- 172.21.15.151:0/442697561 shutdown_connections 2024-09-16T15:51:13.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.372+0000 7fc8cbe00640 1 -- 172.21.15.151:0/442697561 wait complete. 2024-09-16T15:51:13.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.373+0000 7fc8cbe00640 1 Processor -- start 2024-09-16T15:51:13.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.374+0000 7fc8cbe00640 1 -- start start 2024-09-16T15:51:13.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.374+0000 7fc8cbe00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc8cc0d4980 0x7fc8cc1575e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:13.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.374+0000 7fc8cbe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8cc1612d0 0x7fc8cc0c9280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:13.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.375+0000 7fc8cbe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8cc1616b0 0x7fc8cc0b8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:13.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.375+0000 7fc8cbe00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc8cc0b1b60 con 0x7fc8cc1616b0 2024-09-16T15:51:13.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.375+0000 7fc8cbe00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc8cc0b19e0 con 0x7fc8cc1612d0 2024-09-16T15:51:13.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.375+0000 7fc8cbe00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fc8cc0b1ce0 con 0x7fc8cc0d4980 2024-09-16T15:51:13.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.375+0000 7fc8cb400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8cc1616b0 0x7fc8cc0b8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:13.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.375+0000 7fc8cb400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8cc1616b0 0x7fc8cc0b8c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52816/0 (socket says 172.21.15.151:52816) 2024-09-16T15:51:13.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.375+0000 7fc8cb400640 1 -- 172.21.15.151:0/1413703851 learned_addr learned my addr 172.21.15.151:0/1413703851 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:51:13.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.375+0000 7fc8c2000640 1 --2- 172.21.15.151:0/1413703851 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8cc1612d0 0x7fc8cc0c9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:13.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.375+0000 7fc8caa00640 1 --2- 172.21.15.151:0/1413703851 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc8cc0d4980 0x7fc8cc1575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:13.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.375+0000 7fc8cb400640 1 -- 172.21.15.151:0/1413703851 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc8cc0d4980 msgr2=0x7fc8cc1575e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:13.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.375+0000 7fc8cb400640 1 --2- 172.21.15.151:0/1413703851 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc8cc0d4980 0x7fc8cc1575e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:13.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.375+0000 7fc8cb400640 1 -- 172.21.15.151:0/1413703851 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8cc1612d0 msgr2=0x7fc8cc0c9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:13.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.375+0000 7fc8cb400640 1 --2- 172.21.15.151:0/1413703851 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8cc1612d0 0x7fc8cc0c9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:13.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.375+0000 7fc8cb400640 1 -- 172.21.15.151:0/1413703851 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8cc1583f0 con 0x7fc8cc1616b0 2024-09-16T15:51:13.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.376+0000 7fc8cb400640 1 --2- 172.21.15.151:0/1413703851 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8cc1616b0 0x7fc8cc0b8c50 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fc8c400c730 tx=0x7fc8c400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:13.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.376+0000 7fc8c3400640 1 -- 172.21.15.151:0/1413703851 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c4017070 con 0x7fc8cc1616b0 2024-09-16T15:51:13.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.376+0000 7fc8c3400640 1 -- 172.21.15.151:0/1413703851 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc8c4003c60 con 0x7fc8cc1616b0 2024-09-16T15:51:13.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.376+0000 7fc8c3400640 1 -- 172.21.15.151:0/1413703851 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c40052d0 con 0x7fc8cc1616b0 2024-09-16T15:51:13.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.376+0000 7fc8cbe00640 1 -- 172.21.15.151:0/1413703851 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8cc158690 con 0x7fc8cc1616b0 2024-09-16T15:51:13.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.376+0000 7fc8cbe00640 1 -- 172.21.15.151:0/1413703851 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8cc158c50 con 0x7fc8cc1616b0 2024-09-16T15:51:13.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.377+0000 7fc8cbe00640 1 -- 172.21.15.151:0/1413703851 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8cc0b4e00 con 0x7fc8cc1616b0 2024-09-16T15:51:13.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.378+0000 7fc8c3400640 1 -- 172.21.15.151:0/1413703851 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7fc8c401a070 con 0x7fc8cc1616b0 2024-09-16T15:51:13.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.379+0000 7fc8c3400640 1 --2- 172.21.15.151:0/1413703851 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc8a8079ed0 0x7fc8a807c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:13.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.379+0000 7fc8c3400640 1 -- 172.21.15.151:0/1413703851 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5765+0+0 (secure 0 0 0) 0x7fc8c4098d60 con 0x7fc8cc1616b0 2024-09-16T15:51:13.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.380+0000 7fc8caa00640 1 --2- 172.21.15.151:0/1413703851 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc8a8079ed0 0x7fc8a807c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:13.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.381+0000 7fc8caa00640 1 --2- 172.21.15.151:0/1413703851 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc8a8079ed0 0x7fc8a807c390 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc033000 tx=0x7fc8bc004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:13.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.384+0000 7fc8c3400640 1 -- 172.21.15.151:0/1413703851 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc8cc0d2cc0 con 0x7fc8cc1616b0 2024-09-16T15:51:13.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.606+0000 7fc8cbe00640 1 -- 172.21.15.151:0/1413703851 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc8cc0c7400 con 0x7fc8cc1616b0 2024-09-16T15:51:13.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.607+0000 7fc8c3400640 1 -- 172.21.15.151:0/1413703851 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v66) ==== 53+0+73 (secure 0 0 0) 0x7fc8c40627e0 con 0x7fc8cc1616b0 2024-09-16T15:51:13.609 INFO:teuthology.orchestra.run.smithi151.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-16T15:51:13.611 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.609+0000 7fc8cbe00640 1 -- 172.21.15.151:0/1413703851 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc8a8079ed0 msgr2=0x7fc8a807c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:13.611 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.609+0000 7fc8cbe00640 1 --2- 172.21.15.151:0/1413703851 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc8a8079ed0 0x7fc8a807c390 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc033000 tx=0x7fc8bc004730 comp rx=0 tx=0).stop 2024-09-16T15:51:13.611 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.609+0000 7fc8cbe00640 1 -- 172.21.15.151:0/1413703851 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8cc1616b0 msgr2=0x7fc8cc0b8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:13.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.609+0000 7fc8cbe00640 1 --2- 172.21.15.151:0/1413703851 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8cc1616b0 0x7fc8cc0b8c50 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fc8c400c730 tx=0x7fc8c400cbf0 comp rx=0 tx=0).stop 2024-09-16T15:51:13.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.609+0000 7fc8cbe00640 1 -- 172.21.15.151:0/1413703851 shutdown_connections 2024-09-16T15:51:13.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.609+0000 7fc8cbe00640 1 --2- 172.21.15.151:0/1413703851 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc8a8079ed0 0x7fc8a807c390 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:13.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.609+0000 7fc8cbe00640 1 --2- 172.21.15.151:0/1413703851 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8cc1612d0 0x7fc8cc0c9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:13.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.609+0000 7fc8cbe00640 1 --2- 172.21.15.151:0/1413703851 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8cc1616b0 0x7fc8cc0b8c50 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:13.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.609+0000 7fc8cbe00640 1 --2- 172.21.15.151:0/1413703851 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc8cc0d4980 0x7fc8cc1575e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:13.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.609+0000 7fc8cbe00640 1 -- 172.21.15.151:0/1413703851 >> 172.21.15.151:0/1413703851 conn(0x7fc8cc0d3df0 msgr2=0x7fc8cc0b2a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:13.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.609+0000 7fc8cbe00640 1 -- 172.21.15.151:0/1413703851 shutdown_connections 2024-09-16T15:51:13.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13.609+0000 7fc8cbe00640 1 -- 172.21.15.151:0/1413703851 wait complete. 2024-09-16T15:51:13.627 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-16T15:51:13.627 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T15:51:13.628 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi151.front.sepia.ceph.com 2024-09-16T15:51:13.628 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-16T15:51:13.628 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-16T15:51:13.628 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-16T15:51:13.628 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-16T15:51:13.628 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T15:51:13.628 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T15:51:13.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:51:13.628 DEBUG:teuthology.orchestra.run.smithi151:> ip addr 2024-09-16T15:51:13.664 INFO:teuthology.orchestra.run.smithi151.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T15:51:13.664 INFO:teuthology.orchestra.run.smithi151.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T15:51:13.664 INFO:teuthology.orchestra.run.smithi151.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T15:51:13.664 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:51:13.664 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 ::1/128 scope host 2024-09-16T15:51:13.665 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:51:13.665 INFO:teuthology.orchestra.run.smithi151.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:51:13.665 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:da:6e:0c brd ff:ff:ff:ff:ff:ff 2024-09-16T15:51:13.665 INFO:teuthology.orchestra.run.smithi151.stdout: altname enp7s0f0 2024-09-16T15:51:13.665 INFO:teuthology.orchestra.run.smithi151.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:51:13.665 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:da:6e:0d brd ff:ff:ff:ff:ff:ff 2024-09-16T15:51:13.665 INFO:teuthology.orchestra.run.smithi151.stdout: altname enp7s0f1 2024-09-16T15:51:13.665 INFO:teuthology.orchestra.run.smithi151.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:51:13.665 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:88:6f:62 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:51:13.665 INFO:teuthology.orchestra.run.smithi151.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T15:51:13.665 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:88:6f:63 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:51:13.665 INFO:teuthology.orchestra.run.smithi151.stdout: inet 172.21.15.151/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-16T15:51:13.665 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft 42016sec preferred_lft 42016sec 2024-09-16T15:51:13.665 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 fe80::ec4:7aff:fe88:6f63/64 scope link noprefixroute 2024-09-16T15:51:13.665 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:51:13.666 INFO:teuthology.orchestra.run.smithi151.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-16T15:51:13.666 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 62:3f:3b:48:ad:f8 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:51:13.666 INFO:teuthology.orchestra.run.smithi151.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-16T15:51:13.666 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:51:13.666 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 fe80::40fa:7fff:fedf:2a00/64 scope link 2024-09-16T15:51:13.666 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:51:13.666 INFO:teuthology.orchestra.run.smithi151.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T15:51:13.666 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 62:3f:3b:48:ad:f8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T15:51:13.666 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 fe80::603f:3bff:fe48:adf8/64 scope link 2024-09-16T15:51:13.666 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:51:13.666 INFO:teuthology.orchestra.run.smithi151.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T15:51:13.666 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether b6:ac:fe:34:a7:66 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-16T15:51:13.666 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 fe80::b4ac:feff:fe34:a766/64 scope link 2024-09-16T15:51:13.666 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:51:13.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:51:13.667 DEBUG:teuthology.orchestra.run.smithi151:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T15:51:13.758 INFO:teuthology.orchestra.run.smithi151.stdout:1 2024-09-16T15:51:13.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:51:13.761 DEBUG:teuthology.orchestra.run.smithi151:> route 2024-09-16T15:51:13.798 INFO:teuthology.orchestra.run.smithi151.stdout:Kernel IP routing table 2024-09-16T15:51:13.798 INFO:teuthology.orchestra.run.smithi151.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T15:51:13.798 INFO:teuthology.orchestra.run.smithi151.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-16T15:51:13.798 INFO:teuthology.orchestra.run.smithi151.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-16T15:51:13.798 INFO:teuthology.orchestra.run.smithi151.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T15:51:13.798 DEBUG:teuthology.orchestra.run.smithi151:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T15:51:13.798 DEBUG:teuthology.orchestra.run.smithi151:> set -e 2024-09-16T15:51:13.798 DEBUG:teuthology.orchestra.run.smithi151:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-16T15:51:13.798 DEBUG:teuthology.orchestra.run.smithi151:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-16T15:51:13.798 DEBUG:teuthology.orchestra.run.smithi151:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-16T15:51:13.799 DEBUG:teuthology.orchestra.run.smithi151:> ') 2024-09-16T15:51:13.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:51:13.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:51:13.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:51:13.995 DEBUG:teuthology.orchestra.run.smithi151:> ip netns list 2024-09-16T15:51:14.030 INFO:teuthology.orchestra.run.smithi151.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-16T15:51:14.030 INFO:teuthology.orchestra.run.smithi151.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-16T15:51:14.031 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T15:51:14.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:51:14.031 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip link set brx.0 up 2024-09-16T15:51:14.127 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:51:14.127 DEBUG:teuthology.orchestra.run.smithi151:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-16T15:51:14.162 INFO:teuthology.orchestra.run.smithi151.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-16T15:51:14.162 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:51:14.162 DEBUG:teuthology.orchestra.run.smithi151:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-16T15:51:14.237 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-16T15:51:14.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:51:14.238 DEBUG:teuthology.orchestra.run.smithi151:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-16T15:51:14.368 INFO:teuthology.orchestra.run.smithi151.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-16T15:51:14.368 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-16T15:51:14.368 INFO:teuthology.orchestra.run.smithi151.stdout:invalid new device string format 2024-09-16T15:51:14.384 INFO:teuthology.orchestra.run.smithi151.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-16T15:51:14.384 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-16T15:51:14.384 INFO:teuthology.orchestra.run.smithi151.stdout:invalid new device string format 2024-09-16T15:51:14.384 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: resolved to: "172.21.15.151:3300,172.21.15.167:3300,172.21.15.167:3301" 2024-09-16T15:51:14.385 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-16T15:51:14.409 INFO:teuthology.orchestra.run.smithi151.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-16T15:51:14.409 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-16T15:51:14.410 INFO:teuthology.orchestra.run.smithi151.stdout:invalid new device string format 2024-09-16T15:51:14.410 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: resolved to: "172.21.15.151:3300,172.21.15.167:3300,172.21.15.167:3301" 2024-09-16T15:51:14.410 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-16T15:51:14.410 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: resolved to: "172.21.15.151:6789,172.21.15.167:6789,172.21.15.167:6790" 2024-09-16T15:51:14.410 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.151:6789,172.21.15.167:6789,172.21.15.167:6790:/ 2024-09-16T15:51:14.410 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=1757be18-e41a-4446-bb4f-ee4ac81c39df" will pass to kernel 2024-09-16T15:51:14.412 INFO:teuthology.orchestra.run.smithi151.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-16T15:51:14.412 INFO:teuthology.orchestra.run.smithi151.stdout: You just mounted a file system that supports labels which does not 2024-09-16T15:51:14.413 INFO:teuthology.orchestra.run.smithi151.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-16T15:51:14.413 INFO:teuthology.orchestra.run.smithi151.stdout: applications will generate AVC messages and not be allowed access to 2024-09-16T15:51:14.413 INFO:teuthology.orchestra.run.smithi151.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-16T15:51:14.415 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-16T15:51:14.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:51:14.416 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-16T15:51:14.472 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T15:51:14.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:14 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:51:14.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:51:14.568 INFO:teuthology.orchestra.run.smithi151.stdout:client7456 2024-09-16T15:51:14.568 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:51:14.569 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:51:14.604 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-16T15:51:14.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:14.804+0000 7fa9e1400640 1 -- 172.21.15.151:0/2647890612 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9c0004800 msgr2=0x7fa9c0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:14.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:14.804+0000 7fa9e1400640 1 --2- 172.21.15.151:0/2647890612 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9c0004800 0x7fa9c0004c80 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fa9c800c290 tx=0x7fa9c0005830 comp rx=0 tx=0).stop 2024-09-16T15:51:14.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:14.804+0000 7fa9e0a00640 1 -- 172.21.15.151:0/2647890612 reap_dead start 2024-09-16T15:51:14.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:14.804+0000 7fa9e1400640 1 -- 172.21.15.151:0/2647890612 shutdown_connections 2024-09-16T15:51:14.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:14.804+0000 7fa9e1400640 1 -- 172.21.15.151:0/2647890612 >> 172.21.15.151:0/2647890612 conn(0x7fa9dc0b1510 msgr2=0x7fa9dc0b18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:14.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:14.804+0000 7fa9e1400640 1 -- 172.21.15.151:0/2647890612 shutdown_connections 2024-09-16T15:51:14.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:14.804+0000 7fa9e1400640 1 -- 172.21.15.151:0/2647890612 wait complete. 2024-09-16T15:51:14.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:14.806+0000 7fa9e1400640 1 Processor -- start 2024-09-16T15:51:14.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:14.806+0000 7fa9e1400640 1 -- start start 2024-09-16T15:51:14.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:14.807+0000 7fa9e1400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9c0004800 0x7fa9dc14de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:14.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:14.807+0000 7fa9e1400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9dc14aa80 0x7fa9dc15d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:14.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:14.807+0000 7fa9e1400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9dc0c9100 0x7fa9dc0c9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:14.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:14.807+0000 7fa9e1400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa9dc0cf770 con 0x7fa9c0004800 2024-09-16T15:51:14.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:14.807+0000 7fa9e1400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa9dc0cf5f0 con 0x7fa9dc0c9100 2024-09-16T15:51:14.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:14.807+0000 7fa9e1400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fa9dc0b2810 con 0x7fa9dc14aa80 2024-09-16T15:51:14.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:14.807+0000 7fa9dbe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9c0004800 0x7fa9dc14de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:14.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:14.807+0000 7fa9dbe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9c0004800 0x7fa9dc14de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52826/0 (socket says 172.21.15.151:52826) 2024-09-16T15:51:14.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:14.807+0000 7fa9dbe00640 1 -- 172.21.15.151:0/1859971516 learned_addr learned my addr 172.21.15.151:0/1859971516 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:51:14.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:14.807+0000 7fa9db400640 1 --2- 172.21.15.151:0/1859971516 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9dc14aa80 0x7fa9dc15d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:14.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:14.807+0000 7fa9e0a00640 1 --2- 172.21.15.151:0/1859971516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9dc0c9100 0x7fa9dc0c9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:14.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:14.807+0000 7fa9dbe00640 1 -- 172.21.15.151:0/1859971516 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9dc14aa80 msgr2=0x7fa9dc15d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:14.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:14.807+0000 7fa9dbe00640 1 --2- 172.21.15.151:0/1859971516 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9dc14aa80 0x7fa9dc15d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:14.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:14.807+0000 7fa9dbe00640 1 -- 172.21.15.151:0/1859971516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9dc0c9100 msgr2=0x7fa9dc0c9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:14.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:14.807+0000 7fa9dbe00640 1 --2- 172.21.15.151:0/1859971516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9dc0c9100 0x7fa9dc0c9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:14.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:14.807+0000 7fa9dbe00640 1 -- 172.21.15.151:0/1859971516 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9dc0c9ac0 con 0x7fa9c0004800 2024-09-16T15:51:14.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:14.808+0000 7fa9dbe00640 1 --2- 172.21.15.151:0/1859971516 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9c0004800 0x7fa9dc14de10 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc011b10 tx=0x7fa9cc011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:14.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:14.808+0000 7fa9d3e00640 1 -- 172.21.15.151:0/1859971516 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc01e070 con 0x7fa9c0004800 2024-09-16T15:51:14.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:14.808+0000 7fa9d3e00640 1 -- 172.21.15.151:0/1859971516 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa9cc004460 con 0x7fa9c0004800 2024-09-16T15:51:14.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:14.808+0000 7fa9e1400640 1 -- 172.21.15.151:0/1859971516 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9dc14e350 con 0x7fa9c0004800 2024-09-16T15:51:14.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:14.808+0000 7fa9d3e00640 1 -- 172.21.15.151:0/1859971516 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc00fc40 con 0x7fa9c0004800 2024-09-16T15:51:14.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:14.808+0000 7fa9e1400640 1 -- 172.21.15.151:0/1859971516 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9dc15c790 con 0x7fa9c0004800 2024-09-16T15:51:14.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:14.809+0000 7fa9e1400640 1 -- 172.21.15.151:0/1859971516 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9dc073d40 con 0x7fa9c0004800 2024-09-16T15:51:14.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:14.810+0000 7fa9d3e00640 1 -- 172.21.15.151:0/1859971516 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7fa9cc021070 con 0x7fa9c0004800 2024-09-16T15:51:14.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:14.811+0000 7fa9d3e00640 1 --2- 172.21.15.151:0/1859971516 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa9bc079ed0 0x7fa9bc07c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:14.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:14.811+0000 7fa9d3e00640 1 -- 172.21.15.151:0/1859971516 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5765+0+0 (secure 0 0 0) 0x7fa9cc099f70 con 0x7fa9c0004800 2024-09-16T15:51:14.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:14.812+0000 7fa9db400640 1 --2- 172.21.15.151:0/1859971516 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa9bc079ed0 0x7fa9bc07c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:14.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:14.812+0000 7fa9db400640 1 --2- 172.21.15.151:0/1859971516 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa9bc079ed0 0x7fa9bc07c390 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8033000 tx=0x7fa9c8038040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:14.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:14.815+0000 7fa9d3e00640 1 -- 172.21.15.151:0/1859971516 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa9cc0639f0 con 0x7fa9c0004800 2024-09-16T15:51:15.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:15.035+0000 7fa9e1400640 1 -- 172.21.15.151:0/1859971516 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fa9dc069d20 con 0x7fa9c0004800 2024-09-16T15:51:15.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:15.448+0000 7fa9d3e00640 1 -- 172.21.15.151:0/1859971516 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v67) ==== 136+0+0 (secure 0 0 0) 0x7fa9cc068420 con 0x7fa9c0004800 2024-09-16T15:51:15.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:15.450+0000 7fa9e1400640 1 -- 172.21.15.151:0/1859971516 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa9bc079ed0 msgr2=0x7fa9bc07c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:15.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:15.450+0000 7fa9e1400640 1 --2- 172.21.15.151:0/1859971516 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa9bc079ed0 0x7fa9bc07c390 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8033000 tx=0x7fa9c8038040 comp rx=0 tx=0).stop 2024-09-16T15:51:15.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:15.450+0000 7fa9e1400640 1 -- 172.21.15.151:0/1859971516 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9c0004800 msgr2=0x7fa9dc14de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:15.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:15.451+0000 7fa9e1400640 1 --2- 172.21.15.151:0/1859971516 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9c0004800 0x7fa9dc14de10 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc011b10 tx=0x7fa9cc011f10 comp rx=0 tx=0).stop 2024-09-16T15:51:15.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:15.451+0000 7fa9e1400640 1 -- 172.21.15.151:0/1859971516 shutdown_connections 2024-09-16T15:51:15.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:15.451+0000 7fa9e1400640 1 --2- 172.21.15.151:0/1859971516 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa9bc079ed0 0x7fa9bc07c390 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:15.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:15.451+0000 7fa9e1400640 1 --2- 172.21.15.151:0/1859971516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9dc0c9100 0x7fa9dc0c9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:15.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:15.451+0000 7fa9e1400640 1 --2- 172.21.15.151:0/1859971516 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9dc14aa80 0x7fa9dc15d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:15.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:15.451+0000 7fa9e1400640 1 --2- 172.21.15.151:0/1859971516 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9c0004800 0x7fa9dc14de10 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:15.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:15.451+0000 7fa9e1400640 1 -- 172.21.15.151:0/1859971516 >> 172.21.15.151:0/1859971516 conn(0x7fa9dc0b1510 msgr2=0x7fa9dc0b4d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:15.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:15.451+0000 7fa9e1400640 1 -- 172.21.15.151:0/1859971516 shutdown_connections 2024-09-16T15:51:15.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:15.451+0000 7fa9e1400640 1 -- 172.21.15.151:0/1859971516 wait complete. 2024-09-16T15:51:15.471 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-16T15:51:15.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:15.628+0000 7f5bf1800640 1 -- 172.21.15.151:0/2452072641 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5bd0003ad0 msgr2=0x7f5bd0003ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:15.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:15.628+0000 7f5bf1800640 1 --2- 172.21.15.151:0/2452072641 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5bd0003ad0 0x7f5bd0003ef0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc006f50 tx=0x7f5bd0005830 comp rx=0 tx=0).stop 2024-09-16T15:51:15.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:15.628+0000 7f5bf1800640 1 -- 172.21.15.151:0/2452072641 shutdown_connections 2024-09-16T15:51:15.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:15.628+0000 7f5bf1800640 1 --2- 172.21.15.151:0/2452072641 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5bd0003ad0 0x7f5bd0003ef0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:15.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:15.628+0000 7f5bf1800640 1 -- 172.21.15.151:0/2452072641 >> 172.21.15.151:0/2452072641 conn(0x7f5bec0b1950 msgr2=0x7f5bec0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:15.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:15.629+0000 7f5bf1800640 1 -- 172.21.15.151:0/2452072641 shutdown_connections 2024-09-16T15:51:15.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:15.629+0000 7f5bf1800640 1 -- 172.21.15.151:0/2452072641 wait complete. 2024-09-16T15:51:15.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:15.630+0000 7f5bf1800640 1 Processor -- start 2024-09-16T15:51:15.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:15.630+0000 7f5bf1800640 1 -- start start 2024-09-16T15:51:15.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:15.631+0000 7f5bf1800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5bd0003ad0 0x7f5bec1f8e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:15.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:15.631+0000 7f5bf1800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5bec13f570 0x7f5bec0ce080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:15.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:15.631+0000 7f5bf1800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5bec0ce5c0 0x7f5bec09c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:15.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:15.631+0000 7f5bf1800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5bec15e970 con 0x7f5bec13f570 2024-09-16T15:51:15.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:15.631+0000 7f5bf1800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5bec0b4eb0 con 0x7f5bec0ce5c0 2024-09-16T15:51:15.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:15.631+0000 7f5bf1800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f5bec15eaf0 con 0x7f5bd0003ad0 2024-09-16T15:51:15.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:15.631+0000 7f5bea000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5bec13f570 0x7f5bec0ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:15.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:15.631+0000 7f5bea000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5bec13f570 0x7f5bec0ce080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52844/0 (socket says 172.21.15.151:52844) 2024-09-16T15:51:15.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:15.631+0000 7f5bea000640 1 -- 172.21.15.151:0/3514942189 learned_addr learned my addr 172.21.15.151:0/3514942189 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:51:15.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:15.632+0000 7f5beaa00640 1 --2- 172.21.15.151:0/3514942189 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5bd0003ad0 0x7f5bec1f8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:15.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:15.632+0000 7f5beb400640 1 --2- 172.21.15.151:0/3514942189 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5bec0ce5c0 0x7f5bec09c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:15.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:15.632+0000 7f5bea000640 1 -- 172.21.15.151:0/3514942189 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5bd0003ad0 msgr2=0x7f5bec1f8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:15.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:15.632+0000 7f5bea000640 1 --2- 172.21.15.151:0/3514942189 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5bd0003ad0 0x7f5bec1f8e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:15.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:15.632+0000 7f5bea000640 1 -- 172.21.15.151:0/3514942189 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5bec0ce5c0 msgr2=0x7f5bec09c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:15.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:15.632+0000 7f5bea000640 1 --2- 172.21.15.151:0/3514942189 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5bec0ce5c0 0x7f5bec09c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:15.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:15.632+0000 7f5bea000640 1 -- 172.21.15.151:0/3514942189 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bec1f9360 con 0x7f5bec13f570 2024-09-16T15:51:15.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:15.632+0000 7f5beb400640 1 --2- 172.21.15.151:0/3514942189 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5bec0ce5c0 0x7f5bec09c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:51:15.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:15.632+0000 7f5beaa00640 1 --2- 172.21.15.151:0/3514942189 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5bd0003ad0 0x7f5bec1f8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:51:15.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:15.632+0000 7f5bea000640 1 --2- 172.21.15.151:0/3514942189 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5bec13f570 0x7f5bec0ce080 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8003e40 tx=0x7f5bd8001850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:15.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:15.632+0000 7f5be2a00640 1 -- 172.21.15.151:0/3514942189 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd8015070 con 0x7f5bec13f570 2024-09-16T15:51:15.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:15.632+0000 7f5be2a00640 1 -- 172.21.15.151:0/3514942189 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5bd8011900 con 0x7f5bec13f570 2024-09-16T15:51:15.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:15.632+0000 7f5be2a00640 1 -- 172.21.15.151:0/3514942189 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd80012e0 con 0x7f5bec13f570 2024-09-16T15:51:15.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:15.632+0000 7f5bf1800640 1 -- 172.21.15.151:0/3514942189 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bec0cb7c0 con 0x7f5bec13f570 2024-09-16T15:51:15.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:15.633+0000 7f5bf1800640 1 -- 172.21.15.151:0/3514942189 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bec1f8620 con 0x7f5bec13f570 2024-09-16T15:51:15.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:15.634+0000 7f5bf1800640 1 -- 172.21.15.151:0/3514942189 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bec15f000 con 0x7f5bec13f570 2024-09-16T15:51:15.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:15.635+0000 7f5be2a00640 1 -- 172.21.15.151:0/3514942189 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7f5bd8018070 con 0x7f5bec13f570 2024-09-16T15:51:15.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:15.636+0000 7f5be2a00640 1 --2- 172.21.15.151:0/3514942189 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5bcc079f00 0x7f5bcc07c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:15.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:15.636+0000 7f5be2a00640 1 -- 172.21.15.151:0/3514942189 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5765+0+0 (secure 0 0 0) 0x7f5bd809a160 con 0x7f5bec13f570 2024-09-16T15:51:15.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:15.636+0000 7f5beaa00640 1 --2- 172.21.15.151:0/3514942189 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5bcc079f00 0x7f5bcc07c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:15.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:15.637+0000 7f5beaa00640 1 --2- 172.21.15.151:0/3514942189 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5bcc079f00 0x7f5bcc07c3c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc033000 tx=0x7f5bec0cd6f0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:15.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:15.639+0000 7f5be2a00640 1 -- 172.21.15.151:0/3514942189 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f5bd80642f0 con 0x7f5bec13f570 2024-09-16T15:51:15.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:15.845+0000 7f5bf1800640 1 -- 172.21.15.151:0/3514942189 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f5bec1f8460 con 0x7f5bcc079f00 2024-09-16T15:51:15.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:15.848+0000 7f5be2a00640 1 -- 172.21.15.151:0/3514942189 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f5bdc001570 con 0x7f5bcc079f00 2024-09-16T15:51:15.849 INFO:teuthology.orchestra.run.smithi151.stdout:[ 2024-09-16T15:51:15.850 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:51:15.850 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "cephfs" 2024-09-16T15:51:15.850 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:51:15.850 INFO:teuthology.orchestra.run.smithi151.stdout:] 2024-09-16T15:51:15.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:15.849+0000 7f5bf1800640 1 -- 172.21.15.151:0/3514942189 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5bcc079f00 msgr2=0x7f5bcc07c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:15.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:15.849+0000 7f5bf1800640 1 --2- 172.21.15.151:0/3514942189 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5bcc079f00 0x7f5bcc07c3c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc033000 tx=0x7f5bec0cd6f0 comp rx=0 tx=0).stop 2024-09-16T15:51:15.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:15.849+0000 7f5bf1800640 1 -- 172.21.15.151:0/3514942189 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5bec13f570 msgr2=0x7f5bec0ce080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:15.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:15.849+0000 7f5bf1800640 1 --2- 172.21.15.151:0/3514942189 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5bec13f570 0x7f5bec0ce080 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8003e40 tx=0x7f5bd8001850 comp rx=0 tx=0).stop 2024-09-16T15:51:15.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:15.850+0000 7f5bf1800640 1 -- 172.21.15.151:0/3514942189 shutdown_connections 2024-09-16T15:51:15.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:15.850+0000 7f5bf1800640 1 --2- 172.21.15.151:0/3514942189 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5bcc079f00 0x7f5bcc07c3c0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:15.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:15.850+0000 7f5bf1800640 1 --2- 172.21.15.151:0/3514942189 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5bec0ce5c0 0x7f5bec09c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:15.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:15.850+0000 7f5bf1800640 1 --2- 172.21.15.151:0/3514942189 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5bec13f570 0x7f5bec0ce080 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:15.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:15.850+0000 7f5bf1800640 1 --2- 172.21.15.151:0/3514942189 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5bd0003ad0 0x7f5bec1f8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:15.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:15.850+0000 7f5bf1800640 1 -- 172.21.15.151:0/3514942189 >> 172.21.15.151:0/3514942189 conn(0x7f5bec0b1950 msgr2=0x7f5bec0b0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:15.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:15.850+0000 7f5bf1800640 1 -- 172.21.15.151:0/3514942189 shutdown_connections 2024-09-16T15:51:15.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:15.850+0000 7f5bf1800640 1 -- 172.21.15.151:0/3514942189 wait complete. 2024-09-16T15:51:15.868 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-16T15:51:16.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.025+0000 7fb513e00640 1 -- 172.21.15.151:0/528357129 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb50c15ba90 msgr2=0x7fb50c15ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:16.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.025+0000 7fb513e00640 1 --2- 172.21.15.151:0/528357129 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb50c15ba90 0x7fb50c15ef90 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fb50c072360 tx=0x7fb4fc02a560 comp rx=0 tx=0).stop 2024-09-16T15:51:16.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.026+0000 7fb513e00640 1 -- 172.21.15.151:0/528357129 shutdown_connections 2024-09-16T15:51:16.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.026+0000 7fb513e00640 1 --2- 172.21.15.151:0/528357129 >> v2:172.21.15.167:3300/0 conn(0x7fb50c06cb70 0x7fb50c0741e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:16.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.026+0000 7fb513e00640 1 --2- 172.21.15.151:0/528357129 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb50c15ba90 0x7fb50c15ef90 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:16.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.026+0000 7fb513e00640 1 -- 172.21.15.151:0/528357129 >> 172.21.15.151:0/528357129 conn(0x7fb50c0d0d40 msgr2=0x7fb50c0d1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:16.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.026+0000 7fb513e00640 1 -- 172.21.15.151:0/528357129 shutdown_connections 2024-09-16T15:51:16.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.026+0000 7fb513e00640 1 -- 172.21.15.151:0/528357129 wait complete. 2024-09-16T15:51:16.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.027+0000 7fb513e00640 1 Processor -- start 2024-09-16T15:51:16.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.027+0000 7fb513e00640 1 -- start start 2024-09-16T15:51:16.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.028+0000 7fb513e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb50c06cb70 0x7fb50c152850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:16.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.028+0000 7fb513e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb50c074720 0x7fb50c1551f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:16.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.028+0000 7fb513e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb50c15ba90 0x7fb50c1f85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:16.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.028+0000 7fb513e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb50c0cefa0 con 0x7fb50c15ba90 2024-09-16T15:51:16.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.028+0000 7fb513e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb50c0cee20 con 0x7fb50c074720 2024-09-16T15:51:16.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.028+0000 7fb513e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fb50c0cf120 con 0x7fb50c06cb70 2024-09-16T15:51:16.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.029+0000 7fb511e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb50c15ba90 0x7fb50c1f85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:16.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.029+0000 7fb511e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb50c15ba90 0x7fb50c1f85d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52860/0 (socket says 172.21.15.151:52860) 2024-09-16T15:51:16.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.029+0000 7fb511e00640 1 -- 172.21.15.151:0/3117261249 learned_addr learned my addr 172.21.15.151:0/3117261249 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:51:16.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.029+0000 7fb511400640 1 --2- 172.21.15.151:0/3117261249 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb50c06cb70 0x7fb50c152850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:16.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.029+0000 7fb510a00640 1 --2- 172.21.15.151:0/3117261249 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb50c074720 0x7fb50c1551f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:16.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.029+0000 7fb511e00640 1 -- 172.21.15.151:0/3117261249 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb50c06cb70 msgr2=0x7fb50c152850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:16.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.029+0000 7fb511e00640 1 --2- 172.21.15.151:0/3117261249 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb50c06cb70 0x7fb50c152850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:16.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.029+0000 7fb511e00640 1 -- 172.21.15.151:0/3117261249 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb50c074720 msgr2=0x7fb50c1551f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:16.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.029+0000 7fb511e00640 1 --2- 172.21.15.151:0/3117261249 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb50c074720 0x7fb50c1551f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:16.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.029+0000 7fb511e00640 1 -- 172.21.15.151:0/3117261249 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb50c155b90 con 0x7fb50c15ba90 2024-09-16T15:51:16.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.029+0000 7fb510a00640 1 --2- 172.21.15.151:0/3117261249 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb50c074720 0x7fb50c1551f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:51:16.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.029+0000 7fb511e00640 1 --2- 172.21.15.151:0/3117261249 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb50c15ba90 0x7fb50c1f85d0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fb50400c780 tx=0x7fb50400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:16.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.030+0000 7fb509600640 1 -- 172.21.15.151:0/3117261249 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb504017070 con 0x7fb50c15ba90 2024-09-16T15:51:16.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.030+0000 7fb509600640 1 -- 172.21.15.151:0/3117261249 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb50400ce30 con 0x7fb50c15ba90 2024-09-16T15:51:16.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.030+0000 7fb509600640 1 -- 172.21.15.151:0/3117261249 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5040052d0 con 0x7fb50c15ba90 2024-09-16T15:51:16.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.030+0000 7fb513e00640 1 -- 172.21.15.151:0/3117261249 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb50c0adee0 con 0x7fb50c15ba90 2024-09-16T15:51:16.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.030+0000 7fb513e00640 1 -- 172.21.15.151:0/3117261249 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb50c152d90 con 0x7fb50c15ba90 2024-09-16T15:51:16.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.031+0000 7fb513e00640 1 -- 172.21.15.151:0/3117261249 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb50c0cf650 con 0x7fb50c15ba90 2024-09-16T15:51:16.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.032+0000 7fb509600640 1 -- 172.21.15.151:0/3117261249 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7fb504005470 con 0x7fb50c15ba90 2024-09-16T15:51:16.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.033+0000 7fb509600640 1 --2- 172.21.15.151:0/3117261249 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb4ec079ed0 0x7fb4ec07c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:16.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.033+0000 7fb509600640 1 -- 172.21.15.151:0/3117261249 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5765+0+0 (secure 0 0 0) 0x7fb504098bf0 con 0x7fb50c15ba90 2024-09-16T15:51:16.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.034+0000 7fb511400640 1 --2- 172.21.15.151:0/3117261249 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb4ec079ed0 0x7fb4ec07c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:16.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.034+0000 7fb511400640 1 --2- 172.21.15.151:0/3117261249 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb4ec079ed0 0x7fb4ec07c390 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc033000 tx=0x7fb4fc004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:16.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.037+0000 7fb509600640 1 -- 172.21.15.151:0/3117261249 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb504062670 con 0x7fb50c15ba90 2024-09-16T15:51:16.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.224+0000 7fb513e00640 1 -- 172.21.15.151:0/3117261249 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fb50c074ba0 con 0x7fb50c15ba90 2024-09-16T15:51:16.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.225+0000 7fb509600640 1 -- 172.21.15.151:0/3117261249 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v45)=0 v45) ==== 122+0+0 (secure 0 0 0) 0x7fb5040670a0 con 0x7fb50c15ba90 2024-09-16T15:51:16.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.227+0000 7fb513e00640 1 -- 172.21.15.151:0/3117261249 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb4ec079ed0 msgr2=0x7fb4ec07c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:16.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.227+0000 7fb513e00640 1 --2- 172.21.15.151:0/3117261249 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb4ec079ed0 0x7fb4ec07c390 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc033000 tx=0x7fb4fc004730 comp rx=0 tx=0).stop 2024-09-16T15:51:16.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.227+0000 7fb513e00640 1 -- 172.21.15.151:0/3117261249 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb50c15ba90 msgr2=0x7fb50c1f85d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:16.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.227+0000 7fb513e00640 1 --2- 172.21.15.151:0/3117261249 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb50c15ba90 0x7fb50c1f85d0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fb50400c780 tx=0x7fb50400cc40 comp rx=0 tx=0).stop 2024-09-16T15:51:16.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.228+0000 7fb513e00640 1 -- 172.21.15.151:0/3117261249 shutdown_connections 2024-09-16T15:51:16.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.228+0000 7fb513e00640 1 --2- 172.21.15.151:0/3117261249 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb4ec079ed0 0x7fb4ec07c390 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:16.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.228+0000 7fb513e00640 1 --2- 172.21.15.151:0/3117261249 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb50c074720 0x7fb50c1551f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:16.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.228+0000 7fb513e00640 1 --2- 172.21.15.151:0/3117261249 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb50c15ba90 0x7fb50c1f85d0 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:16.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.228+0000 7fb513e00640 1 --2- 172.21.15.151:0/3117261249 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb50c06cb70 0x7fb50c152850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:16.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.228+0000 7fb513e00640 1 -- 172.21.15.151:0/3117261249 >> 172.21.15.151:0/3117261249 conn(0x7fb50c0d0d40 msgr2=0x7fb50c070b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:16.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.228+0000 7fb513e00640 1 -- 172.21.15.151:0/3117261249 shutdown_connections 2024-09-16T15:51:16.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.228+0000 7fb513e00640 1 -- 172.21.15.151:0/3117261249 wait complete. 2024-09-16T15:51:16.246 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mgr mgr/volumes/max_concurrent_clones 2024-09-16T15:51:16.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.406+0000 7fa85b200640 1 -- 172.21.15.151:0/2403006505 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa840004800 msgr2=0x7fa840004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:16.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.406+0000 7fa85b200640 1 --2- 172.21.15.151:0/2403006505 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa840004800 0x7fa840004c80 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fa85c0b5880 tx=0x7fa840005830 comp rx=0 tx=0).stop 2024-09-16T15:51:16.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.406+0000 7fa85b200640 1 -- 172.21.15.151:0/2403006505 shutdown_connections 2024-09-16T15:51:16.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.406+0000 7fa85b200640 1 --2- 172.21.15.151:0/2403006505 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa840004800 0x7fa840004c80 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:16.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.406+0000 7fa85b200640 1 -- 172.21.15.151:0/2403006505 >> 172.21.15.151:0/2403006505 conn(0x7fa85c09d0b0 msgr2=0x7fa85c0d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:16.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.406+0000 7fa85b200640 1 -- 172.21.15.151:0/2403006505 shutdown_connections 2024-09-16T15:51:16.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.406+0000 7fa85b200640 1 -- 172.21.15.151:0/2403006505 wait complete. 2024-09-16T15:51:16.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.408+0000 7fa85b200640 1 Processor -- start 2024-09-16T15:51:16.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.408+0000 7fa85b200640 1 -- start start 2024-09-16T15:51:16.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.409+0000 7fa85b200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa840004800 0x7fa85c1569d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:16.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.409+0000 7fa85b200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa85c0b10c0 0x7fa85c0b1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:16.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.409+0000 7fa85b200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa85c0b87c0 0x7fa85c0b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:16.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.409+0000 7fa85b200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa85c0d22f0 con 0x7fa85c0b87c0 2024-09-16T15:51:16.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.409+0000 7fa85b200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa85c0b51a0 con 0x7fa840004800 2024-09-16T15:51:16.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.409+0000 7fa85b200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fa85c0d2470 con 0x7fa85c0b10c0 2024-09-16T15:51:16.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.409+0000 7fa85a800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa85c0b87c0 0x7fa85c0b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:16.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.409+0000 7fa859e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa840004800 0x7fa85c1569d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:16.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.409+0000 7fa85a800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa85c0b87c0 0x7fa85c0b8c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52876/0 (socket says 172.21.15.151:52876) 2024-09-16T15:51:16.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.409+0000 7fa85a800640 1 -- 172.21.15.151:0/243107804 learned_addr learned my addr 172.21.15.151:0/243107804 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:51:16.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.409+0000 7fa859400640 1 --2- 172.21.15.151:0/243107804 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa85c0b10c0 0x7fa85c0b1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:16.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.409+0000 7fa85a800640 1 -- 172.21.15.151:0/243107804 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa85c0b10c0 msgr2=0x7fa85c0b1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:16.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.409+0000 7fa85a800640 1 --2- 172.21.15.151:0/243107804 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa85c0b10c0 0x7fa85c0b1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:16.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.409+0000 7fa85a800640 1 -- 172.21.15.151:0/243107804 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa840004800 msgr2=0x7fa85c1569d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:16.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.409+0000 7fa85a800640 1 --2- 172.21.15.151:0/243107804 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa840004800 0x7fa85c1569d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:16.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.409+0000 7fa85a800640 1 -- 172.21.15.151:0/243107804 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa85c0b94b0 con 0x7fa85c0b87c0 2024-09-16T15:51:16.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.409+0000 7fa859e00640 1 --2- 172.21.15.151:0/243107804 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa840004800 0x7fa85c1569d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:51:16.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.410+0000 7fa85a800640 1 --2- 172.21.15.151:0/243107804 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa85c0b87c0 0x7fa85c0b8c40 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fa85c0b4eb0 tx=0x7fa854001b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:16.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.410+0000 7fa852000640 1 -- 172.21.15.151:0/243107804 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa854011870 con 0x7fa85c0b87c0 2024-09-16T15:51:16.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.410+0000 7fa852000640 1 -- 172.21.15.151:0/243107804 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa854017070 con 0x7fa85c0b87c0 2024-09-16T15:51:16.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.410+0000 7fa852000640 1 -- 172.21.15.151:0/243107804 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa854011de0 con 0x7fa85c0b87c0 2024-09-16T15:51:16.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.410+0000 7fa85b200640 1 -- 172.21.15.151:0/243107804 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa85c1f4980 con 0x7fa85c0b87c0 2024-09-16T15:51:16.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.410+0000 7fa85b200640 1 -- 172.21.15.151:0/243107804 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa85c0b9180 con 0x7fa85c0b87c0 2024-09-16T15:51:16.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.411+0000 7fa85b200640 1 -- 172.21.15.151:0/243107804 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa85c08d480 con 0x7fa85c0b87c0 2024-09-16T15:51:16.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.412+0000 7fa852000640 1 -- 172.21.15.151:0/243107804 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7fa85401c460 con 0x7fa85c0b87c0 2024-09-16T15:51:16.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.413+0000 7fa852000640 1 --2- 172.21.15.151:0/243107804 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa83c079ed0 0x7fa83c07c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:16.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.413+0000 7fa852000640 1 -- 172.21.15.151:0/243107804 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5765+0+0 (secure 0 0 0) 0x7fa854099da0 con 0x7fa85c0b87c0 2024-09-16T15:51:16.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.414+0000 7fa859e00640 1 --2- 172.21.15.151:0/243107804 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa83c079ed0 0x7fa83c07c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:16.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.415+0000 7fa859e00640 1 --2- 172.21.15.151:0/243107804 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa83c079ed0 0x7fa83c07c390 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fa84c012640 tx=0x7fa84c005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:16.419 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.417+0000 7fa852000640 1 -- 172.21.15.151:0/243107804 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa85c08d480 con 0x7fa85c0b87c0 2024-09-16T15:51:16.610 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.608+0000 7fa85b200640 1 -- 172.21.15.151:0/243107804 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"} v 0) -- 0x7fa85c0d2940 con 0x7fa85c0b87c0 2024-09-16T15:51:16.610 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.608+0000 7fa852000640 1 -- 172.21.15.151:0/243107804 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"}]=0 v45) ==== 116+0+2 (secure 0 0 0) 0x7fa854063820 con 0x7fa85c0b87c0 2024-09-16T15:51:16.610 INFO:teuthology.orchestra.run.smithi151.stdout:4 2024-09-16T15:51:16.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.611+0000 7fa85b200640 1 -- 172.21.15.151:0/243107804 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa83c079ed0 msgr2=0x7fa83c07c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:16.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.611+0000 7fa85b200640 1 --2- 172.21.15.151:0/243107804 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa83c079ed0 0x7fa83c07c390 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fa84c012640 tx=0x7fa84c005f90 comp rx=0 tx=0).stop 2024-09-16T15:51:16.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.611+0000 7fa85b200640 1 -- 172.21.15.151:0/243107804 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa85c0b87c0 msgr2=0x7fa85c0b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:16.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.611+0000 7fa85b200640 1 --2- 172.21.15.151:0/243107804 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa85c0b87c0 0x7fa85c0b8c40 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fa85c0b4eb0 tx=0x7fa854001b80 comp rx=0 tx=0).stop 2024-09-16T15:51:16.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.611+0000 7fa85b200640 1 -- 172.21.15.151:0/243107804 shutdown_connections 2024-09-16T15:51:16.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.611+0000 7fa85b200640 1 --2- 172.21.15.151:0/243107804 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa83c079ed0 0x7fa83c07c390 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:16.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.611+0000 7fa85b200640 1 --2- 172.21.15.151:0/243107804 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa85c0b87c0 0x7fa85c0b8c40 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:16.614 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.611+0000 7fa85b200640 1 --2- 172.21.15.151:0/243107804 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa85c0b10c0 0x7fa85c0b1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:16.614 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.611+0000 7fa85b200640 1 --2- 172.21.15.151:0/243107804 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa840004800 0x7fa85c1569d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:16.614 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.611+0000 7fa85b200640 1 -- 172.21.15.151:0/243107804 >> 172.21.15.151:0/243107804 conn(0x7fa85c09d0b0 msgr2=0x7fa85c0cbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:16.614 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.611+0000 7fa85b200640 1 -- 172.21.15.151:0/243107804 shutdown_connections 2024-09-16T15:51:16.614 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.611+0000 7fa85b200640 1 -- 172.21.15.151:0/243107804 wait complete. 2024-09-16T15:51:16.630 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-16T15:51:16.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.788+0000 7fb9db200640 1 -- 172.21.15.151:0/3618226770 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb9c0004800 msgr2=0x7fb9c0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:16.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.788+0000 7fb9db200640 1 --2- 172.21.15.151:0/3618226770 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb9c0004800 0x7fb9c0004c80 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fb9c8009b20 tx=0x7fb9c0005830 comp rx=0 tx=0).stop 2024-09-16T15:51:16.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.788+0000 7fb9da800640 1 -- 172.21.15.151:0/3618226770 reap_dead start 2024-09-16T15:51:16.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.788+0000 7fb9db200640 1 -- 172.21.15.151:0/3618226770 shutdown_connections 2024-09-16T15:51:16.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.788+0000 7fb9db200640 1 -- 172.21.15.151:0/3618226770 >> 172.21.15.151:0/3618226770 conn(0x7fb9dc09d230 msgr2=0x7fb9dc0d3f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:16.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.788+0000 7fb9db200640 1 -- 172.21.15.151:0/3618226770 shutdown_connections 2024-09-16T15:51:16.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.788+0000 7fb9db200640 1 -- 172.21.15.151:0/3618226770 wait complete. 2024-09-16T15:51:16.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.789+0000 7fb9db200640 1 Processor -- start 2024-09-16T15:51:16.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.790+0000 7fb9db200640 1 -- start start 2024-09-16T15:51:16.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.790+0000 7fb9db200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb9c0004800 0x7fb9dc14f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:16.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.790+0000 7fb9db200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb9dc1f8630 0x7fb9dc0c7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:16.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.790+0000 7fb9db200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9dc0cc870 0x7fb9dc0b7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:16.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.790+0000 7fb9db200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb9dc0cecb0 con 0x7fb9dc1f8630 2024-09-16T15:51:16.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.790+0000 7fb9db200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb9dc0ceb30 con 0x7fb9dc0cc870 2024-09-16T15:51:16.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.790+0000 7fb9db200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fb9dc0b2380 con 0x7fb9c0004800 2024-09-16T15:51:16.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.791+0000 7fb9d9400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb9dc1f8630 0x7fb9dc0c7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:16.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.791+0000 7fb9d9400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb9dc1f8630 0x7fb9dc0c7ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52892/0 (socket says 172.21.15.151:52892) 2024-09-16T15:51:16.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.791+0000 7fb9d9400640 1 -- 172.21.15.151:0/543915771 learned_addr learned my addr 172.21.15.151:0/543915771 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:51:16.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.791+0000 7fb9d9e00640 1 --2- 172.21.15.151:0/543915771 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb9c0004800 0x7fb9dc14f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:16.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.791+0000 7fb9da800640 1 --2- 172.21.15.151:0/543915771 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9dc0cc870 0x7fb9dc0b7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:16.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.791+0000 7fb9d9400640 1 -- 172.21.15.151:0/543915771 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb9c0004800 msgr2=0x7fb9dc14f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:16.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.791+0000 7fb9d9400640 1 --2- 172.21.15.151:0/543915771 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb9c0004800 0x7fb9dc14f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:16.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.791+0000 7fb9d9400640 1 -- 172.21.15.151:0/543915771 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9dc0cc870 msgr2=0x7fb9dc0b7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:16.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.791+0000 7fb9d9400640 1 --2- 172.21.15.151:0/543915771 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9dc0cc870 0x7fb9dc0b7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:16.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.791+0000 7fb9d9400640 1 -- 172.21.15.151:0/543915771 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9dc1f4fe0 con 0x7fb9dc1f8630 2024-09-16T15:51:16.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.791+0000 7fb9da800640 1 --2- 172.21.15.151:0/543915771 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9dc0cc870 0x7fb9dc0b7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:51:16.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.791+0000 7fb9d9400640 1 --2- 172.21.15.151:0/543915771 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb9dc1f8630 0x7fb9dc0c7ab0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fb9c8009c40 tx=0x7fb9c8033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:16.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.792+0000 7fb9d2000640 1 -- 172.21.15.151:0/543915771 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c8038040 con 0x7fb9dc1f8630 2024-09-16T15:51:16.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.792+0000 7fb9d2000640 1 -- 172.21.15.151:0/543915771 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb9c8043070 con 0x7fb9dc1f8630 2024-09-16T15:51:16.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.792+0000 7fb9d2000640 1 -- 172.21.15.151:0/543915771 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c8033c50 con 0x7fb9dc1f8630 2024-09-16T15:51:16.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.792+0000 7fb9db200640 1 -- 172.21.15.151:0/543915771 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9dc1f8b30 con 0x7fb9dc1f8630 2024-09-16T15:51:16.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.792+0000 7fb9db200640 1 -- 172.21.15.151:0/543915771 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9dc1f90f0 con 0x7fb9dc1f8630 2024-09-16T15:51:16.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.793+0000 7fb9db200640 1 -- 172.21.15.151:0/543915771 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9dc0b2880 con 0x7fb9dc1f8630 2024-09-16T15:51:16.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.794+0000 7fb9d2000640 1 -- 172.21.15.151:0/543915771 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7fb9c802b8a0 con 0x7fb9dc1f8630 2024-09-16T15:51:16.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.795+0000 7fb9d2000640 1 --2- 172.21.15.151:0/543915771 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb9bc079ed0 0x7fb9bc07c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:16.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.795+0000 7fb9d2000640 1 -- 172.21.15.151:0/543915771 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5765+0+0 (secure 0 0 0) 0x7fb9c80ba550 con 0x7fb9dc1f8630 2024-09-16T15:51:16.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.796+0000 7fb9d9e00640 1 --2- 172.21.15.151:0/543915771 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb9bc079ed0 0x7fb9bc07c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:16.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.796+0000 7fb9d9e00640 1 --2- 172.21.15.151:0/543915771 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb9bc079ed0 0x7fb9bc07c390 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc0ca940 tx=0x7fb9cc001be0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:16.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.800+0000 7fb9d2000640 1 -- 172.21.15.151:0/543915771 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb9c8083fd0 con 0x7fb9dc1f8630 2024-09-16T15:51:16.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.983+0000 7fb9db200640 1 -- 172.21.15.151:0/543915771 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7fb9dc0d17e0 con 0x7fb9dc1f8630 2024-09-16T15:51:16.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.987+0000 7fb9d2000640 1 -- 172.21.15.151:0/543915771 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v46)=0 v46) ==== 131+0+0 (secure 0 0 0) 0x7fb9c8088a00 con 0x7fb9dc1f8630 2024-09-16T15:51:16.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.990+0000 7fb9db200640 1 -- 172.21.15.151:0/543915771 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb9bc079ed0 msgr2=0x7fb9bc07c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:16.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.990+0000 7fb9db200640 1 --2- 172.21.15.151:0/543915771 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb9bc079ed0 0x7fb9bc07c390 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc0ca940 tx=0x7fb9cc001be0 comp rx=0 tx=0).stop 2024-09-16T15:51:16.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.990+0000 7fb9db200640 1 -- 172.21.15.151:0/543915771 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb9dc1f8630 msgr2=0x7fb9dc0c7ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:16.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.990+0000 7fb9db200640 1 --2- 172.21.15.151:0/543915771 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb9dc1f8630 0x7fb9dc0c7ab0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fb9c8009c40 tx=0x7fb9c8033670 comp rx=0 tx=0).stop 2024-09-16T15:51:16.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.990+0000 7fb9db200640 1 -- 172.21.15.151:0/543915771 shutdown_connections 2024-09-16T15:51:16.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.990+0000 7fb9db200640 1 --2- 172.21.15.151:0/543915771 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb9bc079ed0 0x7fb9bc07c390 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:16.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.990+0000 7fb9db200640 1 --2- 172.21.15.151:0/543915771 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb9dc0cc870 0x7fb9dc0b7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:16.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.990+0000 7fb9db200640 1 --2- 172.21.15.151:0/543915771 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb9dc1f8630 0x7fb9dc0c7ab0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:16.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.990+0000 7fb9db200640 1 --2- 172.21.15.151:0/543915771 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb9c0004800 0x7fb9dc14f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:16.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.990+0000 7fb9db200640 1 -- 172.21.15.151:0/543915771 >> 172.21.15.151:0/543915771 conn(0x7fb9dc09d230 msgr2=0x7fb9dc0cee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:16.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.990+0000 7fb9db200640 1 -- 172.21.15.151:0/543915771 shutdown_connections 2024-09-16T15:51:16.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:16.990+0000 7fb9db200640 1 -- 172.21.15.151:0/543915771 wait complete. 2024-09-16T15:51:17.008 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-09-16T15:51:17.010 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:51:17.009+0000 7f48f3a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:51:17.010 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:51:17.009+0000 7f48f3a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:51:17.010 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:51:17.009+0000 7f48f3a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:51:17.010 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:51:17.009+0000 7f48f3a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:51:17.010 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:51:17.009+0000 7f48f3a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:51:17.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.167+0000 7f2ef8a00640 1 -- 172.21.15.151:0/3398617947 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2ef415a230 msgr2=0x7f2ef406f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:17.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.167+0000 7f2ef8a00640 1 --2- 172.21.15.151:0/3398617947 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2ef415a230 0x7f2ef406f830 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f2ef40ae9c0 tx=0x7f2ee402a8f0 comp rx=0 tx=0).stop 2024-09-16T15:51:17.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.167+0000 7f2ef8a00640 1 -- 172.21.15.151:0/3398617947 shutdown_connections 2024-09-16T15:51:17.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.167+0000 7f2ef8a00640 1 --2- 172.21.15.151:0/3398617947 >> v2:172.21.15.167:3300/0 conn(0x7f2ef40ca7d0 0x7f2ef4072c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:17.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.167+0000 7f2ef8a00640 1 --2- 172.21.15.151:0/3398617947 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2ef415a230 0x7f2ef406f830 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:17.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.167+0000 7f2ef8a00640 1 -- 172.21.15.151:0/3398617947 >> 172.21.15.151:0/3398617947 conn(0x7f2ef40cfa70 msgr2=0x7f2ef40cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:17.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.167+0000 7f2ef8a00640 1 -- 172.21.15.151:0/3398617947 shutdown_connections 2024-09-16T15:51:17.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.167+0000 7f2ef8a00640 1 -- 172.21.15.151:0/3398617947 wait complete. 2024-09-16T15:51:17.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.169+0000 7f2ef8a00640 1 Processor -- start 2024-09-16T15:51:17.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.169+0000 7f2ef8a00640 1 -- start start 2024-09-16T15:51:17.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.170+0000 7f2ef8a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ef4074540 0x7f2ef4153b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:17.174 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.170+0000 7f2ef8a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2ef40ca7d0 0x7f2ef40c8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:17.174 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.170+0000 7f2ef8a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2ef415a230 0x7f2ef40b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:17.174 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.170+0000 7f2ef8a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef406e920 con 0x7f2ef40ca7d0 2024-09-16T15:51:17.174 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.170+0000 7f2ef8a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef406e7a0 con 0x7f2ef4074540 2024-09-16T15:51:17.174 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.170+0000 7f2ef8a00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f2ef406eaa0 con 0x7f2ef415a230 2024-09-16T15:51:17.174 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.170+0000 7f2ef1e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2ef40ca7d0 0x7f2ef40c8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:17.174 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.170+0000 7f2ef1e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2ef40ca7d0 0x7f2ef40c8c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52906/0 (socket says 172.21.15.151:52906) 2024-09-16T15:51:17.174 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.170+0000 7f2ef1e00640 1 -- 172.21.15.151:0/2753681461 learned_addr learned my addr 172.21.15.151:0/2753681461 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:51:17.174 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.170+0000 7f2ef2800640 1 --2- 172.21.15.151:0/2753681461 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ef4074540 0x7f2ef4153b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:17.174 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.171+0000 7f2ef3200640 1 --2- 172.21.15.151:0/2753681461 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2ef415a230 0x7f2ef40b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:17.174 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.171+0000 7f2ef1e00640 1 -- 172.21.15.151:0/2753681461 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2ef415a230 msgr2=0x7f2ef40b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:17.174 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.171+0000 7f2ef1e00640 1 --2- 172.21.15.151:0/2753681461 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2ef415a230 0x7f2ef40b28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:17.174 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.171+0000 7f2ef1e00640 1 -- 172.21.15.151:0/2753681461 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ef4074540 msgr2=0x7f2ef4153b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:17.174 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.171+0000 7f2ef1e00640 1 --2- 172.21.15.151:0/2753681461 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ef4074540 0x7f2ef4153b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:17.174 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.171+0000 7f2ef1e00640 1 -- 172.21.15.151:0/2753681461 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ef40c79c0 con 0x7f2ef40ca7d0 2024-09-16T15:51:17.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.171+0000 7f2ef1e00640 1 --2- 172.21.15.151:0/2753681461 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2ef40ca7d0 0x7f2ef40c8c70 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f2ee000c460 tx=0x7f2ee000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:17.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.171+0000 7f2eeaa00640 1 -- 172.21.15.151:0/2753681461 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee0016020 con 0x7f2ef40ca7d0 2024-09-16T15:51:17.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.171+0000 7f2eeaa00640 1 -- 172.21.15.151:0/2753681461 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ee0012960 con 0x7f2ef40ca7d0 2024-09-16T15:51:17.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.171+0000 7f2eeaa00640 1 -- 172.21.15.151:0/2753681461 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee0012db0 con 0x7f2ef40ca7d0 2024-09-16T15:51:17.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.171+0000 7f2ef8a00640 1 -- 172.21.15.151:0/2753681461 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ef40c8190 con 0x7f2ef40ca7d0 2024-09-16T15:51:17.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.172+0000 7f2ef8a00640 1 -- 172.21.15.151:0/2753681461 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ef4154040 con 0x7f2ef40ca7d0 2024-09-16T15:51:17.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.173+0000 7f2ef8a00640 1 -- 172.21.15.151:0/2753681461 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ef40c7f80 con 0x7f2ef40ca7d0 2024-09-16T15:51:17.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.174+0000 7f2eeaa00640 1 -- 172.21.15.151:0/2753681461 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7f2ee0019020 con 0x7f2ef40ca7d0 2024-09-16T15:51:17.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.175+0000 7f2eeaa00640 1 --2- 172.21.15.151:0/2753681461 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2ed4079ed0 0x7f2ed407c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:17.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.175+0000 7f2eeaa00640 1 -- 172.21.15.151:0/2753681461 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5765+0+0 (secure 0 0 0) 0x7f2ee0099270 con 0x7f2ef40ca7d0 2024-09-16T15:51:17.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.175+0000 7f2ef2800640 1 --2- 172.21.15.151:0/2753681461 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2ed4079ed0 0x7f2ed407c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:17.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.176+0000 7f2ef2800640 1 --2- 172.21.15.151:0/2753681461 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2ed4079ed0 0x7f2ed407c390 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4035000 tx=0x7f2ee4004ab0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:17.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.179+0000 7f2eeaa00640 1 -- 172.21.15.151:0/2753681461 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2ee0020030 con 0x7f2ef40ca7d0 2024-09-16T15:51:17.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.389+0000 7f2ef8a00640 1 -- 172.21.15.151:0/2753681461 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7f2ef40d0910 con 0x7f2ed4079ed0 2024-09-16T15:51:17.402 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:51:17.401+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:51:17.402 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:51:17.401+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:51:17.402 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:51:17.401+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:51:17.402 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:51:17.401+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:51:17.402 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:51:17.401+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:51:17.442 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.439+0000 7f2eeaa00640 1 -- 172.21.15.151:0/2753681461 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2ef40d0910 con 0x7f2ed4079ed0 2024-09-16T15:51:17.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.442+0000 7f2ef8a00640 1 -- 172.21.15.151:0/2753681461 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2ed4079ed0 msgr2=0x7f2ed407c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:17.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.442+0000 7f2ef8a00640 1 --2- 172.21.15.151:0/2753681461 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2ed4079ed0 0x7f2ed407c390 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4035000 tx=0x7f2ee4004ab0 comp rx=0 tx=0).stop 2024-09-16T15:51:17.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.442+0000 7f2ef8a00640 1 -- 172.21.15.151:0/2753681461 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2ef40ca7d0 msgr2=0x7f2ef40c8c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:17.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.442+0000 7f2ef8a00640 1 --2- 172.21.15.151:0/2753681461 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2ef40ca7d0 0x7f2ef40c8c70 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f2ee000c460 tx=0x7f2ee000c920 comp rx=0 tx=0).stop 2024-09-16T15:51:17.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.442+0000 7f2ef8a00640 1 -- 172.21.15.151:0/2753681461 shutdown_connections 2024-09-16T15:51:17.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.442+0000 7f2ef8a00640 1 --2- 172.21.15.151:0/2753681461 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2ed4079ed0 0x7f2ed407c390 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:17.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.442+0000 7f2ef8a00640 1 --2- 172.21.15.151:0/2753681461 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2ef415a230 0x7f2ef40b28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:17.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.442+0000 7f2ef8a00640 1 --2- 172.21.15.151:0/2753681461 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2ef40ca7d0 0x7f2ef40c8c70 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:17.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.442+0000 7f2ef8a00640 1 --2- 172.21.15.151:0/2753681461 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ef4074540 0x7f2ef4153b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:17.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.442+0000 7f2ef8a00640 1 -- 172.21.15.151:0/2753681461 >> 172.21.15.151:0/2753681461 conn(0x7f2ef40cfa70 msgr2=0x7f2ef40af760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:17.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.443+0000 7f2ef8a00640 1 -- 172.21.15.151:0/2753681461 shutdown_connections 2024-09-16T15:51:17.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.443+0000 7f2ef8a00640 1 -- 172.21.15.151:0/2753681461 wait complete. 2024-09-16T15:51:17.460 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-16T15:51:17.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.620+0000 7fa4a9400640 1 -- 172.21.15.151:0/80242776 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa488003fd0 msgr2=0x7fa488004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:17.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.620+0000 7fa4a9400640 1 --2- 172.21.15.151:0/80242776 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa488003fd0 0x7fa488004800 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fa490009b50 tx=0x7fa488005830 comp rx=0 tx=0).stop 2024-09-16T15:51:17.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.620+0000 7fa4a3200640 1 -- 172.21.15.151:0/80242776 reap_dead start 2024-09-16T15:51:17.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.620+0000 7fa4a9400640 1 -- 172.21.15.151:0/80242776 shutdown_connections 2024-09-16T15:51:17.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.620+0000 7fa4a9400640 1 -- 172.21.15.151:0/80242776 >> 172.21.15.151:0/80242776 conn(0x7fa4a40b1840 msgr2=0x7fa4a40b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:17.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.620+0000 7fa4a9400640 1 -- 172.21.15.151:0/80242776 shutdown_connections 2024-09-16T15:51:17.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.620+0000 7fa4a9400640 1 -- 172.21.15.151:0/80242776 wait complete. 2024-09-16T15:51:17.624 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.622+0000 7fa4a9400640 1 Processor -- start 2024-09-16T15:51:17.624 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.622+0000 7fa4a9400640 1 -- start start 2024-09-16T15:51:17.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.623+0000 7fa4a9400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa488003fd0 0x7fa4a40ac680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:17.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.623+0000 7fa4a9400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4a413b870 0x7fa4a413bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:17.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.623+0000 7fa4a9400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4a40aa860 0x7fa4a40aace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:17.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.623+0000 7fa4a9400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a40ced40 con 0x7fa4a40aa860 2024-09-16T15:51:17.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.623+0000 7fa4a9400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a40cebc0 con 0x7fa4a413b870 2024-09-16T15:51:17.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.623+0000 7fa4a9400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fa4a40ceec0 con 0x7fa488003fd0 2024-09-16T15:51:17.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.623+0000 7fa4a3200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4a40aa860 0x7fa4a40aace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:17.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.623+0000 7fa4a3200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4a40aa860 0x7fa4a40aace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52932/0 (socket says 172.21.15.151:52932) 2024-09-16T15:51:17.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.623+0000 7fa4a3200640 1 -- 172.21.15.151:0/74795514 learned_addr learned my addr 172.21.15.151:0/74795514 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:51:17.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.623+0000 7fa4a1e00640 1 --2- 172.21.15.151:0/74795514 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4a413b870 0x7fa4a413bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:17.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.623+0000 7fa4a2800640 1 --2- 172.21.15.151:0/74795514 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa488003fd0 0x7fa4a40ac680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:17.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.623+0000 7fa4a3200640 1 -- 172.21.15.151:0/74795514 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa488003fd0 msgr2=0x7fa4a40ac680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:17.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.623+0000 7fa4a3200640 1 --2- 172.21.15.151:0/74795514 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa488003fd0 0x7fa4a40ac680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:17.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.623+0000 7fa4a3200640 1 -- 172.21.15.151:0/74795514 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4a413b870 msgr2=0x7fa4a413bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:17.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.623+0000 7fa4a3200640 1 --2- 172.21.15.151:0/74795514 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4a413b870 0x7fa4a413bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:17.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.623+0000 7fa4a3200640 1 -- 172.21.15.151:0/74795514 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4a413c3d0 con 0x7fa4a40aa860 2024-09-16T15:51:17.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.624+0000 7fa4a3200640 1 --2- 172.21.15.151:0/74795514 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4a40aa860 0x7fa4a40aace0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fa49c002230 tx=0x7fa49c00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:17.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.624+0000 7fa49aa00640 1 -- 172.21.15.151:0/74795514 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49c015070 con 0x7fa4a40aa860 2024-09-16T15:51:17.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.624+0000 7fa49aa00640 1 -- 172.21.15.151:0/74795514 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa49c00ccb0 con 0x7fa4a40aa860 2024-09-16T15:51:17.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.624+0000 7fa4a9400640 1 -- 172.21.15.151:0/74795514 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4a40ab590 con 0x7fa4a40aa860 2024-09-16T15:51:17.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.624+0000 7fa49aa00640 1 -- 172.21.15.151:0/74795514 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49c00eea0 con 0x7fa4a40aa860 2024-09-16T15:51:17.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.624+0000 7fa4a9400640 1 -- 172.21.15.151:0/74795514 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4a40abb50 con 0x7fa4a40aa860 2024-09-16T15:51:17.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.625+0000 7fa4a9400640 1 -- 172.21.15.151:0/74795514 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4a4073440 con 0x7fa4a40aa860 2024-09-16T15:51:17.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.626+0000 7fa49aa00640 1 -- 172.21.15.151:0/74795514 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 37) ==== 103327+0+0 (secure 0 0 0) 0x7fa49c018070 con 0x7fa4a40aa860 2024-09-16T15:51:17.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.627+0000 7fa49aa00640 1 --2- 172.21.15.151:0/74795514 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa484079f00 0x7fa48407c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:17.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.627+0000 7fa49aa00640 1 -- 172.21.15.151:0/74795514 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5765+0+0 (secure 0 0 0) 0x7fa49c099ba0 con 0x7fa4a40aa860 2024-09-16T15:51:17.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.628+0000 7fa4a2800640 1 --2- 172.21.15.151:0/74795514 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa484079f00 0x7fa48407c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:17.630 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.628+0000 7fa4a2800640 1 --2- 172.21.15.151:0/74795514 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa484079f00 0x7fa48407c3c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fa494009ba0 tx=0x7fa494005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:17.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.631+0000 7fa49aa00640 1 -- 172.21.15.151:0/74795514 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa49c063c70 con 0x7fa4a40aa860 2024-09-16T15:51:17.841 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.839+0000 7fa4a9400640 1 -- 172.21.15.151:0/74795514 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fa4a4144cf0 con 0x7fa484079f00 2024-09-16T15:51:17.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.846+0000 7fa49aa00640 1 -- 172.21.15.151:0/74795514 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7fa4a4144cf0 con 0x7fa484079f00 2024-09-16T15:51:17.847 INFO:teuthology.orchestra.run.smithi151.stdout:/volumes/_nogroup/sv1/f862b7fa-4987-48e2-806c-1c057bd9c92a 2024-09-16T15:51:17.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.847+0000 7fa4a9400640 1 -- 172.21.15.151:0/74795514 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa484079f00 msgr2=0x7fa48407c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:17.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.847+0000 7fa4a9400640 1 --2- 172.21.15.151:0/74795514 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa484079f00 0x7fa48407c3c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fa494009ba0 tx=0x7fa494005f90 comp rx=0 tx=0).stop 2024-09-16T15:51:17.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.847+0000 7fa4a9400640 1 -- 172.21.15.151:0/74795514 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4a40aa860 msgr2=0x7fa4a40aace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:17.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.847+0000 7fa4a9400640 1 --2- 172.21.15.151:0/74795514 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4a40aa860 0x7fa4a40aace0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fa49c002230 tx=0x7fa49c00ec50 comp rx=0 tx=0).stop 2024-09-16T15:51:17.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.847+0000 7fa4a9400640 1 -- 172.21.15.151:0/74795514 shutdown_connections 2024-09-16T15:51:17.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.847+0000 7fa4a9400640 1 --2- 172.21.15.151:0/74795514 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa484079f00 0x7fa48407c3c0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:17.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.847+0000 7fa4a9400640 1 --2- 172.21.15.151:0/74795514 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4a413b870 0x7fa4a413bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:17.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.847+0000 7fa4a9400640 1 --2- 172.21.15.151:0/74795514 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4a40aa860 0x7fa4a40aace0 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:17.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.847+0000 7fa4a9400640 1 --2- 172.21.15.151:0/74795514 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa488003fd0 0x7fa4a40ac680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:17.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.847+0000 7fa4a9400640 1 -- 172.21.15.151:0/74795514 >> 172.21.15.151:0/74795514 conn(0x7fa4a40b1840 msgr2=0x7fa4a40a9460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:17.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.847+0000 7fa4a9400640 1 -- 172.21.15.151:0/74795514 shutdown_connections 2024-09-16T15:51:17.851 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:17.847+0000 7fa4a9400640 1 -- 172.21.15.151:0/74795514 wait complete. 2024-09-16T15:51:17.868 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/f862b7fa-4987-48e2-806c-1c057bd9c92a 2024-09-16T15:51:17.868 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:51:17.868 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:51:17.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:51:17.903 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/f862b7fa-4987-48e2-806c-1c057bd9c92a/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T15:51:20.939 DEBUG:teuthology.orchestra.run.smithi151:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:51:20.944 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:51:25.672 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records in 2024-09-16T15:51:25.672 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records out 2024-09-16T15:51:25.672 INFO:teuthology.orchestra.run.smithi151.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.68537 s, 140 MB/s 2024-09-16T15:51:25.673 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:51:25.673 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:51:25.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:51:25.709 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/f862b7fa-4987-48e2-806c-1c057bd9c92a/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T15:51:33.835 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records in 2024-09-16T15:51:33.835 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records out 2024-09-16T15:51:33.835 INFO:teuthology.orchestra.run.smithi151.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.04828 s, 133 MB/s 2024-09-16T15:51:33.836 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:51:33.836 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:51:33.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:51:33.871 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/f862b7fa-4987-48e2-806c-1c057bd9c92a/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T15:51:41.847 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records in 2024-09-16T15:51:41.847 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records out 2024-09-16T15:51:41.847 INFO:teuthology.orchestra.run.smithi151.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.89808 s, 136 MB/s 2024-09-16T15:51:41.849 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-09-16T15:51:42.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.013+0000 7f1979c00640 1 -- 172.21.15.151:0/3313234801 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1974159aa0 msgr2=0x7f19740741a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:42.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.013+0000 7f1979c00640 1 --2- 172.21.15.151:0/3313234801 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1974159aa0 0x7f19740741a0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f1974070850 tx=0x7f196402a5f0 comp rx=0 tx=0).stop 2024-09-16T15:51:42.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.013+0000 7f1979c00640 1 -- 172.21.15.151:0/3313234801 shutdown_connections 2024-09-16T15:51:42.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.013+0000 7f1979c00640 1 --2- 172.21.15.151:0/3313234801 >> v2:172.21.15.167:3300/0 conn(0x7f197415cff0 0x7f197406e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:42.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.013+0000 7f1979c00640 1 --2- 172.21.15.151:0/3313234801 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1974159aa0 0x7f19740741a0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:42.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.013+0000 7f1979c00640 1 -- 172.21.15.151:0/3313234801 >> 172.21.15.151:0/3313234801 conn(0x7f19740b18e0 msgr2=0x7f19740b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:42.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.013+0000 7f1979c00640 1 -- 172.21.15.151:0/3313234801 shutdown_connections 2024-09-16T15:51:42.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.013+0000 7f1979c00640 1 -- 172.21.15.151:0/3313234801 wait complete. 2024-09-16T15:51:42.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.014+0000 7f1979c00640 1 Processor -- start 2024-09-16T15:51:42.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.014+0000 7f1979c00640 1 -- start start 2024-09-16T15:51:42.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.015+0000 7f1979c00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f197406f220 0x7f19740ce3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:42.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.015+0000 7f1979c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1974159aa0 0x7f197415dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:42.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.015+0000 7f1979c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f197415cff0 0x7f197415e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:42.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.015+0000 7f1979c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f19740ccc30 con 0x7f197415cff0 2024-09-16T15:51:42.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.015+0000 7f1979c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f19740ccab0 con 0x7f1974159aa0 2024-09-16T15:51:42.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.015+0000 7f1979c00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f19740ccdb0 con 0x7f197406f220 2024-09-16T15:51:42.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.016+0000 7f1973400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f197415cff0 0x7f197415e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:42.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.016+0000 7f1973400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f197415cff0 0x7f197415e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36448/0 (socket says 172.21.15.151:36448) 2024-09-16T15:51:42.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.016+0000 7f1973400640 1 -- 172.21.15.151:0/2637811140 learned_addr learned my addr 172.21.15.151:0/2637811140 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:51:42.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.016+0000 7f1972a00640 1 --2- 172.21.15.151:0/2637811140 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f197406f220 0x7f19740ce3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:42.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.016+0000 7f1972000640 1 --2- 172.21.15.151:0/2637811140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1974159aa0 0x7f197415dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:42.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.016+0000 7f1973400640 1 -- 172.21.15.151:0/2637811140 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f197406f220 msgr2=0x7f19740ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:42.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.016+0000 7f1973400640 1 --2- 172.21.15.151:0/2637811140 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f197406f220 0x7f19740ce3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:42.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.016+0000 7f1973400640 1 -- 172.21.15.151:0/2637811140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1974159aa0 msgr2=0x7f197415dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:42.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.016+0000 7f1973400640 1 --2- 172.21.15.151:0/2637811140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1974159aa0 0x7f197415dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:42.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.016+0000 7f1973400640 1 -- 172.21.15.151:0/2637811140 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19740ce900 con 0x7f197415cff0 2024-09-16T15:51:42.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.016+0000 7f1973400640 1 --2- 172.21.15.151:0/2637811140 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f197415cff0 0x7f197415e1c0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f196c00c770 tx=0x7f196c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:42.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.017+0000 7f196aa00640 1 -- 172.21.15.151:0/2637811140 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196c003da0 con 0x7f197415cff0 2024-09-16T15:51:42.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.017+0000 7f196aa00640 1 -- 172.21.15.151:0/2637811140 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f196c005070 con 0x7f197415cff0 2024-09-16T15:51:42.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.017+0000 7f196aa00640 1 -- 172.21.15.151:0/2637811140 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196c005500 con 0x7f197415cff0 2024-09-16T15:51:42.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.017+0000 7f1979c00640 1 -- 172.21.15.151:0/2637811140 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f197409ce80 con 0x7f197415cff0 2024-09-16T15:51:42.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.017+0000 7f1979c00640 1 -- 172.21.15.151:0/2637811140 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19741f60c0 con 0x7f197415cff0 2024-09-16T15:51:42.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.018+0000 7f1979c00640 1 -- 172.21.15.151:0/2637811140 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1974069360 con 0x7f197415cff0 2024-09-16T15:51:42.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.019+0000 7f196aa00640 1 -- 172.21.15.151:0/2637811140 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 38) ==== 103429+0+0 (secure 0 0 0) 0x7f196c0056a0 con 0x7f197415cff0 2024-09-16T15:51:42.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.020+0000 7f196aa00640 1 --2- 172.21.15.151:0/2637811140 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1954079ff0 0x7f195407c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:42.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.020+0000 7f196aa00640 1 -- 172.21.15.151:0/2637811140 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5765+0+0 (secure 0 0 0) 0x7f196c099470 con 0x7f197415cff0 2024-09-16T15:51:42.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.021+0000 7f1972a00640 1 --2- 172.21.15.151:0/2637811140 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1954079ff0 0x7f195407c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:42.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.021+0000 7f1972a00640 1 --2- 172.21.15.151:0/2637811140 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1954079ff0 0x7f195407c4b0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f1964033000 tx=0x7f1964004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:42.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.025+0000 7f196aa00640 1 -- 172.21.15.151:0/2637811140 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f19740ce900 con 0x7f197415cff0 2024-09-16T15:51:42.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.240+0000 7f1979c00640 1 -- 172.21.15.151:0/2637811140 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7f197415f130 con 0x7f1954079ff0 2024-09-16T15:51:42.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.255+0000 7f196aa00640 1 -- 172.21.15.151:0/2637811140 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f197415f130 con 0x7f1954079ff0 2024-09-16T15:51:42.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.257+0000 7f1979c00640 1 -- 172.21.15.151:0/2637811140 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1954079ff0 msgr2=0x7f195407c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:42.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.257+0000 7f1979c00640 1 --2- 172.21.15.151:0/2637811140 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1954079ff0 0x7f195407c4b0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f1964033000 tx=0x7f1964004730 comp rx=0 tx=0).stop 2024-09-16T15:51:42.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.257+0000 7f1979c00640 1 -- 172.21.15.151:0/2637811140 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f197415cff0 msgr2=0x7f197415e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:42.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.257+0000 7f1979c00640 1 --2- 172.21.15.151:0/2637811140 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f197415cff0 0x7f197415e1c0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f196c00c770 tx=0x7f196c00cc30 comp rx=0 tx=0).stop 2024-09-16T15:51:42.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.257+0000 7f1979c00640 1 -- 172.21.15.151:0/2637811140 shutdown_connections 2024-09-16T15:51:42.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.257+0000 7f1979c00640 1 --2- 172.21.15.151:0/2637811140 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1954079ff0 0x7f195407c4b0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:42.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.257+0000 7f1979c00640 1 --2- 172.21.15.151:0/2637811140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1974159aa0 0x7f197415dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:42.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.257+0000 7f1979c00640 1 --2- 172.21.15.151:0/2637811140 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f197415cff0 0x7f197415e1c0 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:42.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.257+0000 7f1979c00640 1 --2- 172.21.15.151:0/2637811140 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f197406f220 0x7f19740ce3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:42.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.257+0000 7f1979c00640 1 -- 172.21.15.151:0/2637811140 >> 172.21.15.151:0/2637811140 conn(0x7f19740b18e0 msgr2=0x7f19740aed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:42.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.257+0000 7f1979c00640 1 -- 172.21.15.151:0/2637811140 shutdown_connections 2024-09-16T15:51:42.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.257+0000 7f1979c00640 1 -- 172.21.15.151:0/2637811140 wait complete. 2024-09-16T15:51:42.274 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-16T15:51:42.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.439+0000 7f8128c00640 1 -- 172.21.15.151:0/527807041 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f812406e580 msgr2=0x7f812406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:42.441 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.439+0000 7f8128c00640 1 --2- 172.21.15.151:0/527807041 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f812406e580 0x7f812406e9a0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f810c005fd0 tx=0x7f810c02a540 comp rx=0 tx=0).stop 2024-09-16T15:51:42.441 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.439+0000 7f8128c00640 1 -- 172.21.15.151:0/527807041 shutdown_connections 2024-09-16T15:51:42.441 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.439+0000 7f8128c00640 1 --2- 172.21.15.151:0/527807041 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f812406e580 0x7f812406e9a0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:42.441 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.439+0000 7f8128c00640 1 --2- 172.21.15.151:0/527807041 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8124154c90 0x7f81240cc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:42.441 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.439+0000 7f8128c00640 1 -- 172.21.15.151:0/527807041 >> 172.21.15.151:0/527807041 conn(0x7f81240b1510 msgr2=0x7f81240b18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:42.441 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.439+0000 7f8128c00640 1 -- 172.21.15.151:0/527807041 shutdown_connections 2024-09-16T15:51:42.441 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.439+0000 7f8128c00640 1 -- 172.21.15.151:0/527807041 wait complete. 2024-09-16T15:51:42.442 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.441+0000 7f8128c00640 1 Processor -- start 2024-09-16T15:51:42.442 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.441+0000 7f8128c00640 1 -- start start 2024-09-16T15:51:42.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.442+0000 7f8128c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f812406e580 0x7f81240739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:42.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.442+0000 7f8128c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f81240ad880 0x7f8124143880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:42.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.442+0000 7f8128c00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8124154c90 0x7f8124143dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:42.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.442+0000 7f8128c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f81240ca340 con 0x7f81240ad880 2024-09-16T15:51:42.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.442+0000 7f8128c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f81240ca1c0 con 0x7f812406e580 2024-09-16T15:51:42.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.442+0000 7f8128c00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f81240ca4c0 con 0x7f8124154c90 2024-09-16T15:51:42.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.442+0000 7f811be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f81240ad880 0x7f8124143880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:42.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.442+0000 7f811be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f81240ad880 0x7f8124143880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36468/0 (socket says 172.21.15.151:36468) 2024-09-16T15:51:42.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.442+0000 7f811be00640 1 -- 172.21.15.151:0/276494905 learned_addr learned my addr 172.21.15.151:0/276494905 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:51:42.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.442+0000 7f8123400640 1 --2- 172.21.15.151:0/276494905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f812406e580 0x7f81240739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:42.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.442+0000 7f8123e00640 1 --2- 172.21.15.151:0/276494905 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8124154c90 0x7f8124143dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:42.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.443+0000 7f811be00640 1 -- 172.21.15.151:0/276494905 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8124154c90 msgr2=0x7f8124143dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:42.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.443+0000 7f811be00640 1 --2- 172.21.15.151:0/276494905 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8124154c90 0x7f8124143dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:42.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.443+0000 7f811be00640 1 -- 172.21.15.151:0/276494905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f812406e580 msgr2=0x7f81240739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:42.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.443+0000 7f811be00640 1 --2- 172.21.15.151:0/276494905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f812406e580 0x7f81240739c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:42.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.443+0000 7f811be00640 1 -- 172.21.15.151:0/276494905 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8124144600 con 0x7f81240ad880 2024-09-16T15:51:42.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.443+0000 7f811be00640 1 --2- 172.21.15.151:0/276494905 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f81240ad880 0x7f8124143880 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f810c000c00 tx=0x7f810c0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:42.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.443+0000 7f811b400640 1 -- 172.21.15.151:0/276494905 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810c03f070 con 0x7f81240ad880 2024-09-16T15:51:42.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.443+0000 7f811b400640 1 -- 172.21.15.151:0/276494905 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f810c0047a0 con 0x7f81240ad880 2024-09-16T15:51:42.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.443+0000 7f811b400640 1 -- 172.21.15.151:0/276494905 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810c035030 con 0x7f81240ad880 2024-09-16T15:51:42.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.443+0000 7f8128c00640 1 -- 172.21.15.151:0/276494905 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81241f7bd0 con 0x7f81240ad880 2024-09-16T15:51:42.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.443+0000 7f8128c00640 1 -- 172.21.15.151:0/276494905 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81241f8190 con 0x7f81240ad880 2024-09-16T15:51:42.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.444+0000 7f8128c00640 1 -- 172.21.15.151:0/276494905 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81241f84c0 con 0x7f81240ad880 2024-09-16T15:51:42.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.445+0000 7f811b400640 1 -- 172.21.15.151:0/276494905 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 38) ==== 103429+0+0 (secure 0 0 0) 0x7f810c042070 con 0x7f81240ad880 2024-09-16T15:51:42.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.446+0000 7f811b400640 1 --2- 172.21.15.151:0/276494905 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f810007a020 0x7f810007c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:42.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.446+0000 7f811b400640 1 -- 172.21.15.151:0/276494905 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5765+0+0 (secure 0 0 0) 0x7f810c0ba820 con 0x7f81240ad880 2024-09-16T15:51:42.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.447+0000 7f8123400640 1 --2- 172.21.15.151:0/276494905 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f810007a020 0x7f810007c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:42.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.447+0000 7f8123400640 1 --2- 172.21.15.151:0/276494905 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f810007a020 0x7f810007c4e0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f8114009b30 tx=0x7f8114005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:42.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.450+0000 7f811b400640 1 -- 172.21.15.151:0/276494905 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f810c0842a0 con 0x7f81240ad880 2024-09-16T15:51:42.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.663+0000 7f8128c00640 1 -- 172.21.15.151:0/276494905 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f8124069d20 con 0x7f810007a020 2024-09-16T15:51:42.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.669+0000 7f811b400640 1 -- 172.21.15.151:0/276494905 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f8124069d20 con 0x7f810007a020 2024-09-16T15:51:42.670 INFO:teuthology.orchestra.run.smithi151.stdout:/volumes/_nogroup/sv1/f862b7fa-4987-48e2-806c-1c057bd9c92a 2024-09-16T15:51:42.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.672+0000 7f8128c00640 1 -- 172.21.15.151:0/276494905 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f810007a020 msgr2=0x7f810007c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:42.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.672+0000 7f8128c00640 1 --2- 172.21.15.151:0/276494905 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f810007a020 0x7f810007c4e0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f8114009b30 tx=0x7f8114005f90 comp rx=0 tx=0).stop 2024-09-16T15:51:42.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.672+0000 7f8128c00640 1 -- 172.21.15.151:0/276494905 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f81240ad880 msgr2=0x7f8124143880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:42.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.672+0000 7f8128c00640 1 --2- 172.21.15.151:0/276494905 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f81240ad880 0x7f8124143880 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f810c000c00 tx=0x7f810c0038a0 comp rx=0 tx=0).stop 2024-09-16T15:51:42.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.672+0000 7f8128c00640 1 -- 172.21.15.151:0/276494905 shutdown_connections 2024-09-16T15:51:42.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.672+0000 7f8128c00640 1 --2- 172.21.15.151:0/276494905 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f810007a020 0x7f810007c4e0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:42.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.672+0000 7f8128c00640 1 --2- 172.21.15.151:0/276494905 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8124154c90 0x7f8124143dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:42.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.672+0000 7f8128c00640 1 --2- 172.21.15.151:0/276494905 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f81240ad880 0x7f8124143880 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:42.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.672+0000 7f8128c00640 1 --2- 172.21.15.151:0/276494905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f812406e580 0x7f81240739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:42.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.672+0000 7f8128c00640 1 -- 172.21.15.151:0/276494905 >> 172.21.15.151:0/276494905 conn(0x7f81240b1510 msgr2=0x7f81240cb060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:42.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.672+0000 7f8128c00640 1 -- 172.21.15.151:0/276494905 shutdown_connections 2024-09-16T15:51:42.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:42.672+0000 7f8128c00640 1 -- 172.21.15.151:0/276494905 wait complete. 2024-09-16T15:51:42.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:51:42.688 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/f862b7fa-4987-48e2-806c-1c057bd9c92a 2024-09-16T15:51:42.689 DEBUG:teuthology.orchestra.run.smithi151:> ) 2024-09-16T15:51:42.724 INFO:teuthology.orchestra.run.smithi151.stdout:# file: volumes/_nogroup/sv1/f862b7fa-4987-48e2-806c-1c057bd9c92a 2024-09-16T15:51:42.724 INFO:teuthology.orchestra.run.smithi151.stdout:ceph.dir.rbytes="3055190016" 2024-09-16T15:51:42.724 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:51:42.725 INFO:tasks.cephfs.test_volumes:r_size = 3055190016 exp_size = 3221225472 2024-09-16T15:51:44.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:51:44.729 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/f862b7fa-4987-48e2-806c-1c057bd9c92a 2024-09-16T15:51:44.729 DEBUG:teuthology.orchestra.run.smithi151:> ) 2024-09-16T15:51:44.770 INFO:teuthology.orchestra.run.smithi151.stdout:# file: volumes/_nogroup/sv1/f862b7fa-4987-48e2-806c-1c057bd9c92a 2024-09-16T15:51:44.770 INFO:teuthology.orchestra.run.smithi151.stdout:ceph.dir.rbytes="3221225472" 2024-09-16T15:51:44.770 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:51:44.771 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 exp_size = 3221225472 2024-09-16T15:51:44.772 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 ss1clone1 2024-09-16T15:51:44.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:44.982+0000 7fa8aa800640 1 -- 172.21.15.151:0/3052417469 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa8a40ccb00 msgr2=0x7fa8a415f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:44.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:44.982+0000 7fa8aa800640 1 --2- 172.21.15.151:0/3052417469 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa8a40ccb00 0x7fa8a415f9f0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fa8a40d0cd0 tx=0x7fa89c02a980 comp rx=0 tx=0).stop 2024-09-16T15:51:44.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:44.982+0000 7fa8aa800640 1 -- 172.21.15.151:0/3052417469 shutdown_connections 2024-09-16T15:51:44.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:44.982+0000 7fa8aa800640 1 --2- 172.21.15.151:0/3052417469 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa8a40ccb00 0x7fa8a415f9f0 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:44.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:44.982+0000 7fa8aa800640 1 --2- 172.21.15.151:0/3052417469 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8a406efa0 0x7fa8a40cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:44.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:44.982+0000 7fa8aa800640 1 --2- 172.21.15.151:0/3052417469 >> v2:172.21.15.167:3300/0 conn(0x7fa8a4145750 0x7fa8a4073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:44.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:44.982+0000 7fa8aa800640 1 -- 172.21.15.151:0/3052417469 >> 172.21.15.151:0/3052417469 conn(0x7fa8a40b1840 msgr2=0x7fa8a40b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:44.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:44.982+0000 7fa8aa800640 1 -- 172.21.15.151:0/3052417469 shutdown_connections 2024-09-16T15:51:44.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:44.983+0000 7fa8aa800640 1 -- 172.21.15.151:0/3052417469 wait complete. 2024-09-16T15:51:44.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:44.984+0000 7fa8aa800640 1 Processor -- start 2024-09-16T15:51:44.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:44.984+0000 7fa8aa800640 1 -- start start 2024-09-16T15:51:44.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:44.985+0000 7fa8aa800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8a406efa0 0x7fa8a41539f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:44.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:44.985+0000 7fa8aa800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa8a40ccb00 0x7fa8a413d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:44.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:44.985+0000 7fa8aa800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8a4145750 0x7fa8a4141d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:44.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:44.985+0000 7fa8aa800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a41600b0 con 0x7fa8a40ccb00 2024-09-16T15:51:44.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:44.985+0000 7fa8aa800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a415ff30 con 0x7fa8a406efa0 2024-09-16T15:51:44.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:44.985+0000 7fa8aa800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fa8a4160230 con 0x7fa8a4145750 2024-09-16T15:51:44.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:44.986+0000 7fa8a3400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8a406efa0 0x7fa8a41539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:44.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:44.986+0000 7fa8a3400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8a406efa0 0x7fa8a41539f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.151:48392/0 (socket says 172.21.15.151:48392) 2024-09-16T15:51:44.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:44.986+0000 7fa8a3400640 1 -- 172.21.15.151:0/2346448589 learned_addr learned my addr 172.21.15.151:0/2346448589 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:51:44.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:44.986+0000 7fa8a2a00640 1 --2- 172.21.15.151:0/2346448589 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa8a40ccb00 0x7fa8a413d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:44.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:44.986+0000 7fa8a3e00640 1 --2- 172.21.15.151:0/2346448589 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8a4145750 0x7fa8a4141d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:44.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:44.986+0000 7fa8a3400640 1 -- 172.21.15.151:0/2346448589 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8a4145750 msgr2=0x7fa8a4141d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:44.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:44.986+0000 7fa8a3400640 1 --2- 172.21.15.151:0/2346448589 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8a4145750 0x7fa8a4141d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:44.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:44.986+0000 7fa8a3400640 1 -- 172.21.15.151:0/2346448589 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa8a40ccb00 msgr2=0x7fa8a413d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:44.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:44.986+0000 7fa8a3400640 1 --2- 172.21.15.151:0/2346448589 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa8a40ccb00 0x7fa8a413d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:44.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:44.986+0000 7fa8a3400640 1 -- 172.21.15.151:0/2346448589 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8a41f5150 con 0x7fa8a406efa0 2024-09-16T15:51:44.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:44.987+0000 7fa8a3400640 1 --2- 172.21.15.151:0/2346448589 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8a406efa0 0x7fa8a41539f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa89400e790 tx=0x7fa89400ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:44.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:44.987+0000 7fa89b400640 1 -- 172.21.15.151:0/2346448589 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa89400eea0 con 0x7fa8a406efa0 2024-09-16T15:51:44.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:44.987+0000 7fa89b400640 1 -- 172.21.15.151:0/2346448589 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa89400cdb0 con 0x7fa8a406efa0 2024-09-16T15:51:44.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:44.987+0000 7fa89b400640 1 -- 172.21.15.151:0/2346448589 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa894010690 con 0x7fa8a406efa0 2024-09-16T15:51:44.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:44.987+0000 7fa8aa800640 1 -- 172.21.15.151:0/2346448589 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8a41f4e70 con 0x7fa8a406efa0 2024-09-16T15:51:44.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:44.987+0000 7fa8aa800640 1 -- 172.21.15.151:0/2346448589 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8a41531f0 con 0x7fa8a406efa0 2024-09-16T15:51:44.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:44.988+0000 7fa8aa800640 1 -- 172.21.15.151:0/2346448589 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8a4160710 con 0x7fa8a406efa0 2024-09-16T15:51:44.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:44.990+0000 7fa89b400640 1 -- 172.21.15.151:0/2346448589 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 38) ==== 103429+0+0 (secure 0 0 0) 0x7fa894010830 con 0x7fa8a406efa0 2024-09-16T15:51:44.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:44.991+0000 7fa89b400640 1 --2- 172.21.15.151:0/2346448589 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa884079ff0 0x7fa88407c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:44.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:44.991+0000 7fa89b400640 1 -- 172.21.15.151:0/2346448589 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5765+0+0 (secure 0 0 0) 0x7fa89409a880 con 0x7fa8a406efa0 2024-09-16T15:51:44.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:44.991+0000 7fa8a2a00640 1 --2- 172.21.15.151:0/2346448589 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa884079ff0 0x7fa88407c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:44.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:44.992+0000 7fa8a2a00640 1 --2- 172.21.15.151:0/2346448589 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa884079ff0 0x7fa88407c4b0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4153f30 tx=0x7fa890005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:44.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:44.994+0000 7fa89b400640 1 -- 172.21.15.151:0/2346448589 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa894064380 con 0x7fa8a406efa0 2024-09-16T15:51:45.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:45.213+0000 7fa8aa800640 1 -- 172.21.15.151:0/2346448589 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7fa8a41f7b30 con 0x7fa884079ff0 2024-09-16T15:51:45.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:45.284+0000 7fa89b400640 1 -- 172.21.15.151:0/2346448589 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa8a41f7b30 con 0x7fa884079ff0 2024-09-16T15:51:45.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:45.287+0000 7fa8aa800640 1 -- 172.21.15.151:0/2346448589 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa884079ff0 msgr2=0x7fa88407c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:45.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:45.287+0000 7fa8aa800640 1 --2- 172.21.15.151:0/2346448589 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa884079ff0 0x7fa88407c4b0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4153f30 tx=0x7fa890005f90 comp rx=0 tx=0).stop 2024-09-16T15:51:45.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:45.287+0000 7fa8aa800640 1 -- 172.21.15.151:0/2346448589 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8a406efa0 msgr2=0x7fa8a41539f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:45.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:45.287+0000 7fa8aa800640 1 --2- 172.21.15.151:0/2346448589 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8a406efa0 0x7fa8a41539f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa89400e790 tx=0x7fa89400ec50 comp rx=0 tx=0).stop 2024-09-16T15:51:45.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:45.287+0000 7fa8aa800640 1 -- 172.21.15.151:0/2346448589 shutdown_connections 2024-09-16T15:51:45.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:45.287+0000 7fa8aa800640 1 --2- 172.21.15.151:0/2346448589 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa884079ff0 0x7fa88407c4b0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:45.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:45.287+0000 7fa8aa800640 1 --2- 172.21.15.151:0/2346448589 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa8a4145750 0x7fa8a4141d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:45.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:45.287+0000 7fa8aa800640 1 --2- 172.21.15.151:0/2346448589 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa8a40ccb00 0x7fa8a413d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:45.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:45.287+0000 7fa8aa800640 1 --2- 172.21.15.151:0/2346448589 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa8a406efa0 0x7fa8a41539f0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:45.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:45.287+0000 7fa8aa800640 1 -- 172.21.15.151:0/2346448589 >> 172.21.15.151:0/2346448589 conn(0x7fa8a40b1840 msgr2=0x7fa8a40b00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:45.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:45.287+0000 7fa8aa800640 1 -- 172.21.15.151:0/2346448589 shutdown_connections 2024-09-16T15:51:45.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:45.287+0000 7fa8aa800640 1 -- 172.21.15.151:0/2346448589 wait complete. 2024-09-16T15:51:45.289 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:51:45.288+0000 7f48f5800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:51:45.289 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:51:45.288+0000 7f48f5800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:51:45.289 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:51:45.288+0000 7f48f5800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:51:45.290 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:51:45.288+0000 7f48f5800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:51:45.290 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:51:45.288+0000 7f48f5800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:51:45.306 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-09-16T15:51:45.307 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:51:45.306+0000 7f48f6c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:51:45.307 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:51:45.306+0000 7f48f6c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:51:45.307 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:51:45.306+0000 7f48f6c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:51:45.307 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:51:45.306+0000 7f48f6c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:51:45.307 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:51:45.306+0000 7f48f6c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:51:45.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:45.471+0000 7f6783400640 1 -- 172.21.15.151:0/126364847 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f677c145750 msgr2=0x7f677c073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:45.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:45.471+0000 7f6783400640 1 --2- 172.21.15.151:0/126364847 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f677c145750 0x7f677c073440 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f677c0cfb70 tx=0x7f676c02a4d0 comp rx=0 tx=0).stop 2024-09-16T15:51:45.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:45.471+0000 7f6783400640 1 -- 172.21.15.151:0/126364847 shutdown_connections 2024-09-16T15:51:45.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:45.471+0000 7f6783400640 1 --2- 172.21.15.151:0/126364847 >> v2:172.21.15.167:3300/0 conn(0x7f677c0cccd0 0x7f677c0cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:45.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:45.471+0000 7f6783400640 1 --2- 172.21.15.151:0/126364847 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f677c145750 0x7f677c073440 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:45.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:45.471+0000 7f6783400640 1 -- 172.21.15.151:0/126364847 >> 172.21.15.151:0/126364847 conn(0x7f677c0b1840 msgr2=0x7f677c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:45.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:45.471+0000 7f6783400640 1 -- 172.21.15.151:0/126364847 shutdown_connections 2024-09-16T15:51:45.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:45.471+0000 7f6783400640 1 -- 172.21.15.151:0/126364847 wait complete. 2024-09-16T15:51:45.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:45.473+0000 7f6783400640 1 Processor -- start 2024-09-16T15:51:45.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:45.473+0000 7f6783400640 1 -- start start 2024-09-16T15:51:45.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:45.474+0000 7f6783400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f677c06efa0 0x7f677c142aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:45.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:45.474+0000 7f6783400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f677c0cccd0 0x7f677c13bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:45.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:45.474+0000 7f6783400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f677c145750 0x7f677c13d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:45.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:45.474+0000 7f6783400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f677c0ce480 con 0x7f677c0cccd0 2024-09-16T15:51:45.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:45.474+0000 7f6783400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f677c0ce300 con 0x7f677c06efa0 2024-09-16T15:51:45.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:45.474+0000 7f6783400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f677c0ce600 con 0x7f677c145750 2024-09-16T15:51:45.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:45.474+0000 7f6780a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f677c06efa0 0x7f677c142aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:45.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:45.474+0000 7f677be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f677c0cccd0 0x7f677c13bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:45.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:45.475+0000 7f6780a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f677c06efa0 0x7f677c142aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.151:48398/0 (socket says 172.21.15.151:48398) 2024-09-16T15:51:45.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:45.475+0000 7f6780a00640 1 -- 172.21.15.151:0/3050179332 learned_addr learned my addr 172.21.15.151:0/3050179332 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:51:45.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:45.475+0000 7f6781400640 1 --2- 172.21.15.151:0/3050179332 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f677c145750 0x7f677c13d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:45.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:45.475+0000 7f677be00640 1 -- 172.21.15.151:0/3050179332 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f677c145750 msgr2=0x7f677c13d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:45.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:45.475+0000 7f677be00640 1 --2- 172.21.15.151:0/3050179332 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f677c145750 0x7f677c13d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:45.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:45.475+0000 7f677be00640 1 -- 172.21.15.151:0/3050179332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f677c06efa0 msgr2=0x7f677c142aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:45.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:45.475+0000 7f677be00640 1 --2- 172.21.15.151:0/3050179332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f677c06efa0 0x7f677c142aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:45.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:45.475+0000 7f677be00640 1 -- 172.21.15.151:0/3050179332 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f677c0c4c50 con 0x7f677c0cccd0 2024-09-16T15:51:45.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:45.475+0000 7f6780a00640 1 --2- 172.21.15.151:0/3050179332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f677c06efa0 0x7f677c142aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:51:45.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:45.475+0000 7f677be00640 1 --2- 172.21.15.151:0/3050179332 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f677c0cccd0 0x7f677c13bcd0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f676800c410 tx=0x7f676800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:45.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:45.475+0000 7f6778c00640 1 -- 172.21.15.151:0/3050179332 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6768017070 con 0x7f677c0cccd0 2024-09-16T15:51:45.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:45.476+0000 7f6778c00640 1 -- 172.21.15.151:0/3050179332 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6768005320 con 0x7f677c0cccd0 2024-09-16T15:51:45.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:45.476+0000 7f6778c00640 1 -- 172.21.15.151:0/3050179332 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6768004840 con 0x7f677c0cccd0 2024-09-16T15:51:45.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:45.476+0000 7f6783400640 1 -- 172.21.15.151:0/3050179332 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f677c0c4f30 con 0x7f677c0cccd0 2024-09-16T15:51:45.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:45.476+0000 7f6783400640 1 -- 172.21.15.151:0/3050179332 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f677c152780 con 0x7f677c0cccd0 2024-09-16T15:51:45.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:45.477+0000 7f6783400640 1 -- 172.21.15.151:0/3050179332 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f677c0cdbb0 con 0x7f677c0cccd0 2024-09-16T15:51:45.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:45.478+0000 7f6778c00640 1 -- 172.21.15.151:0/3050179332 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 38) ==== 103429+0+0 (secure 0 0 0) 0x7f676801a070 con 0x7f677c0cccd0 2024-09-16T15:51:45.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:45.479+0000 7f6778c00640 1 --2- 172.21.15.151:0/3050179332 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f675c079ff0 0x7f675c07c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:45.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:45.479+0000 7f6778c00640 1 -- 172.21.15.151:0/3050179332 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5765+0+0 (secure 0 0 0) 0x7f6768098920 con 0x7f677c0cccd0 2024-09-16T15:51:45.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:45.479+0000 7f6780a00640 1 --2- 172.21.15.151:0/3050179332 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f675c079ff0 0x7f675c07c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:45.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:45.480+0000 7f6780a00640 1 --2- 172.21.15.151:0/3050179332 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f675c079ff0 0x7f675c07c4b0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f676c009230 tx=0x7f676c004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:45.484 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:45.483+0000 7f6778c00640 1 -- 172.21.15.151:0/3050179332 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f67680623a0 con 0x7f677c0cccd0 2024-09-16T15:51:45.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:45.703+0000 7f6783400640 1 -- 172.21.15.151:0/3050179332 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f677c06ed80 con 0x7f675c079ff0 2024-09-16T15:51:46.291 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:51:46.290+0000 7f4877e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:51:46.291 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:51:46.290+0000 7f4877e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:51:46.291 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:51:46.290+0000 7f4877e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:51:46.291 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:51:46.290+0000 7f4877e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:51:46.292 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:51:46.290+0000 7f4877e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:51:46.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:46.322+0000 7f6778c00640 1 -- 172.21.15.151:0/3050179332 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+278 (secure 0 0 0) 0x7f677c06ed80 con 0x7f675c079ff0 2024-09-16T15:51:46.323 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-16T15:51:46.323 INFO:teuthology.orchestra.run.smithi151.stdout: "status": { 2024-09-16T15:51:46.323 INFO:teuthology.orchestra.run.smithi151.stdout: "state": "in-progress", 2024-09-16T15:51:46.323 INFO:teuthology.orchestra.run.smithi151.stdout: "source": { 2024-09-16T15:51:46.323 INFO:teuthology.orchestra.run.smithi151.stdout: "volume": "cephfs", 2024-09-16T15:51:46.323 INFO:teuthology.orchestra.run.smithi151.stdout: "subvolume": "sv1", 2024-09-16T15:51:46.323 INFO:teuthology.orchestra.run.smithi151.stdout: "snapshot": "ss1" 2024-09-16T15:51:46.323 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:51:46.323 INFO:teuthology.orchestra.run.smithi151.stdout: "progress_report": { 2024-09-16T15:51:46.323 INFO:teuthology.orchestra.run.smithi151.stdout: "percentage cloned": "0.0%", 2024-09-16T15:51:46.324 INFO:teuthology.orchestra.run.smithi151.stdout: "amount cloned": "184/3.0G", 2024-09-16T15:51:46.324 INFO:teuthology.orchestra.run.smithi151.stdout: "files cloned": "3/4" 2024-09-16T15:51:46.324 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:51:46.324 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:51:46.324 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-16T15:51:46.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:46.324+0000 7f6783400640 1 -- 172.21.15.151:0/3050179332 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f675c079ff0 msgr2=0x7f675c07c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:46.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:46.324+0000 7f6783400640 1 --2- 172.21.15.151:0/3050179332 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f675c079ff0 0x7f675c07c4b0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f676c009230 tx=0x7f676c004730 comp rx=0 tx=0).stop 2024-09-16T15:51:46.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:46.324+0000 7f6783400640 1 -- 172.21.15.151:0/3050179332 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f677c0cccd0 msgr2=0x7f677c13bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:46.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:46.324+0000 7f6783400640 1 --2- 172.21.15.151:0/3050179332 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f677c0cccd0 0x7f677c13bcd0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f676800c410 tx=0x7f676800c8d0 comp rx=0 tx=0).stop 2024-09-16T15:51:46.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:46.324+0000 7f6783400640 1 -- 172.21.15.151:0/3050179332 shutdown_connections 2024-09-16T15:51:46.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:46.324+0000 7f6783400640 1 --2- 172.21.15.151:0/3050179332 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f675c079ff0 0x7f675c07c4b0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:46.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:46.324+0000 7f6783400640 1 --2- 172.21.15.151:0/3050179332 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f677c145750 0x7f677c13d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:46.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:46.324+0000 7f6783400640 1 --2- 172.21.15.151:0/3050179332 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f677c0cccd0 0x7f677c13bcd0 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:46.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:46.324+0000 7f6783400640 1 --2- 172.21.15.151:0/3050179332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f677c06efa0 0x7f677c142aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:46.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:46.324+0000 7f6783400640 1 -- 172.21.15.151:0/3050179332 >> 172.21.15.151:0/3050179332 conn(0x7f677c0b1840 msgr2=0x7f677c0a9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:46.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:46.324+0000 7f6783400640 1 -- 172.21.15.151:0/3050179332 shutdown_connections 2024-09-16T15:51:46.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:46.325+0000 7f6783400640 1 -- 172.21.15.151:0/3050179332 wait complete. 2024-09-16T15:51:46.343 DEBUG:tasks.cephfs.test_volumes:current clone state = in-progress 2024-09-16T15:51:46.343 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-09-16T15:51:46.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:46.563+0000 7f504f200640 1 -- 172.21.15.151:0/236246570 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f50500cefc0 msgr2=0x7f5050073d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:46.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:46.563+0000 7f504f200640 1 --2- 172.21.15.151:0/236246570 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f50500cefc0 0x7f5050073d40 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f5038005fd0 tx=0x7f503802a530 comp rx=0 tx=0).stop 2024-09-16T15:51:46.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:46.564+0000 7f504f200640 1 -- 172.21.15.151:0/236246570 shutdown_connections 2024-09-16T15:51:46.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:46.564+0000 7f504f200640 1 --2- 172.21.15.151:0/236246570 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f50500cefc0 0x7f5050073d40 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:46.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:46.564+0000 7f504f200640 1 --2- 172.21.15.151:0/236246570 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f505015bb90 0x7f50500b2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:46.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:46.564+0000 7f504f200640 1 -- 172.21.15.151:0/236246570 >> 172.21.15.151:0/236246570 conn(0x7f50500b1510 msgr2=0x7f50500b18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:46.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:46.564+0000 7f504f200640 1 -- 172.21.15.151:0/236246570 shutdown_connections 2024-09-16T15:51:46.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:46.564+0000 7f504f200640 1 -- 172.21.15.151:0/236246570 wait complete. 2024-09-16T15:51:46.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:46.566+0000 7f504f200640 1 Processor -- start 2024-09-16T15:51:46.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:46.566+0000 7f504f200640 1 -- start start 2024-09-16T15:51:46.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:46.569+0000 7f504f200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50500cefc0 0x7f50501570a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:46.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:46.569+0000 7f504f200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50500cf3e0 0x7f505015e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:46.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:46.569+0000 7f504f200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f505015bb90 0x7f505014a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:46.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:46.569+0000 7f504f200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5050074610 con 0x7f505015bb90 2024-09-16T15:51:46.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:46.569+0000 7f504f200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5050074490 con 0x7f50500cf3e0 2024-09-16T15:51:46.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:46.569+0000 7f504f200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f5050074790 con 0x7f50500cefc0 2024-09-16T15:51:46.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:46.569+0000 7f504e800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f505015bb90 0x7f505014a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:46.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:46.569+0000 7f504e800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f505015bb90 0x7f505014a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:38644/0 (socket says 172.21.15.151:38644) 2024-09-16T15:51:46.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:46.569+0000 7f504e800640 1 -- 172.21.15.151:0/3907244034 learned_addr learned my addr 172.21.15.151:0/3907244034 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:51:46.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:46.570+0000 7f504de00640 1 --2- 172.21.15.151:0/3907244034 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50500cefc0 0x7f50501570a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:46.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:46.570+0000 7f5045400640 1 --2- 172.21.15.151:0/3907244034 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50500cf3e0 0x7f505015e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:46.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:46.570+0000 7f504e800640 1 -- 172.21.15.151:0/3907244034 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50500cefc0 msgr2=0x7f50501570a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:46.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:46.570+0000 7f504e800640 1 --2- 172.21.15.151:0/3907244034 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50500cefc0 0x7f50501570a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:46.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:46.570+0000 7f504e800640 1 -- 172.21.15.151:0/3907244034 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50500cf3e0 msgr2=0x7f505015e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:46.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:46.570+0000 7f504e800640 1 --2- 172.21.15.151:0/3907244034 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50500cf3e0 0x7f505015e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:46.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:46.570+0000 7f504e800640 1 -- 172.21.15.151:0/3907244034 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f505014ae10 con 0x7f505015bb90 2024-09-16T15:51:46.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:46.570+0000 7f5045400640 1 --2- 172.21.15.151:0/3907244034 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50500cf3e0 0x7f505015e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:51:46.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:46.571+0000 7f504e800640 1 --2- 172.21.15.151:0/3907244034 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f505015bb90 0x7f505014a8d0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f504800c780 tx=0x7f504800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:46.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:46.574+0000 7f5046a00640 1 -- 172.21.15.151:0/3907244034 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5048017070 con 0x7f505015bb90 2024-09-16T15:51:46.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:46.575+0000 7f5046a00640 1 -- 172.21.15.151:0/3907244034 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5048005070 con 0x7f505015bb90 2024-09-16T15:51:46.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:46.575+0000 7f5046a00640 1 -- 172.21.15.151:0/3907244034 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5048005500 con 0x7f505015bb90 2024-09-16T15:51:46.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:46.575+0000 7f504f200640 1 -- 172.21.15.151:0/3907244034 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f505015efa0 con 0x7f505015bb90 2024-09-16T15:51:46.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:46.575+0000 7f504f200640 1 -- 172.21.15.151:0/3907244034 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50501f1650 con 0x7f505015bb90 2024-09-16T15:51:46.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:46.576+0000 7f504f200640 1 -- 172.21.15.151:0/3907244034 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f505006f710 con 0x7f505015bb90 2024-09-16T15:51:46.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:46.578+0000 7f5046a00640 1 -- 172.21.15.151:0/3907244034 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 38) ==== 103429+0+0 (secure 0 0 0) 0x7f504801a070 con 0x7f505015bb90 2024-09-16T15:51:46.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:46.580+0000 7f5046a00640 1 --2- 172.21.15.151:0/3907244034 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f502c07a040 0x7f502c07c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:46.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:46.580+0000 7f5046a00640 1 -- 172.21.15.151:0/3907244034 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5765+0+0 (secure 0 0 0) 0x7f5048098c10 con 0x7f505015bb90 2024-09-16T15:51:46.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:46.585+0000 7f504de00640 1 --2- 172.21.15.151:0/3907244034 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f502c07a040 0x7f502c07c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:46.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:46.586+0000 7f5046a00640 1 -- 172.21.15.151:0/3907244034 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f5048062e60 con 0x7f505015bb90 2024-09-16T15:51:46.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:46.588+0000 7f504de00640 1 --2- 172.21.15.151:0/3907244034 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f502c07a040 0x7f502c07c500 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f504000b810 tx=0x7f5040008040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:46.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:46.607+0000 7f5046a00640 1 -- 172.21.15.151:0/3907244034 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mgrmap(e 39) ==== 103531+0+0 (secure 0 0 0) 0x7f50480056a0 con 0x7f505015bb90 2024-09-16T15:51:46.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:46.898+0000 7f504f200640 1 -- 172.21.15.151:0/3907244034 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f5050069d20 con 0x7f502c07a040 2024-09-16T15:51:46.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:46.959+0000 7f5046a00640 1 -- 172.21.15.151:0/3907244034 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+278 (secure 0 0 0) 0x7f5050069d20 con 0x7f502c07a040 2024-09-16T15:51:46.960 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-16T15:51:46.960 INFO:teuthology.orchestra.run.smithi151.stdout: "status": { 2024-09-16T15:51:46.960 INFO:teuthology.orchestra.run.smithi151.stdout: "state": "in-progress", 2024-09-16T15:51:46.960 INFO:teuthology.orchestra.run.smithi151.stdout: "source": { 2024-09-16T15:51:46.961 INFO:teuthology.orchestra.run.smithi151.stdout: "volume": "cephfs", 2024-09-16T15:51:46.961 INFO:teuthology.orchestra.run.smithi151.stdout: "subvolume": "sv1", 2024-09-16T15:51:46.961 INFO:teuthology.orchestra.run.smithi151.stdout: "snapshot": "ss1" 2024-09-16T15:51:46.961 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:51:46.961 INFO:teuthology.orchestra.run.smithi151.stdout: "progress_report": { 2024-09-16T15:51:46.961 INFO:teuthology.orchestra.run.smithi151.stdout: "percentage cloned": "0.0%", 2024-09-16T15:51:46.961 INFO:teuthology.orchestra.run.smithi151.stdout: "amount cloned": "184/3.0G", 2024-09-16T15:51:46.961 INFO:teuthology.orchestra.run.smithi151.stdout: "files cloned": "4/4" 2024-09-16T15:51:46.961 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:51:46.961 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:51:46.961 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-16T15:51:46.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:46.962+0000 7f504f200640 1 -- 172.21.15.151:0/3907244034 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f502c07a040 msgr2=0x7f502c07c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:46.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:46.962+0000 7f504f200640 1 --2- 172.21.15.151:0/3907244034 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f502c07a040 0x7f502c07c500 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f504000b810 tx=0x7f5040008040 comp rx=0 tx=0).stop 2024-09-16T15:51:46.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:46.962+0000 7f504f200640 1 -- 172.21.15.151:0/3907244034 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f505015bb90 msgr2=0x7f505014a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:46.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:46.962+0000 7f504f200640 1 --2- 172.21.15.151:0/3907244034 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f505015bb90 0x7f505014a8d0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f504800c780 tx=0x7f504800cc40 comp rx=0 tx=0).stop 2024-09-16T15:51:46.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:46.962+0000 7f504f200640 1 -- 172.21.15.151:0/3907244034 shutdown_connections 2024-09-16T15:51:46.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:46.962+0000 7f504f200640 1 --2- 172.21.15.151:0/3907244034 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f502c07a040 0x7f502c07c500 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:46.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:46.962+0000 7f504f200640 1 --2- 172.21.15.151:0/3907244034 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f50500cf3e0 0x7f505015e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:46.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:46.962+0000 7f504f200640 1 --2- 172.21.15.151:0/3907244034 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f505015bb90 0x7f505014a8d0 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:46.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:46.962+0000 7f504f200640 1 --2- 172.21.15.151:0/3907244034 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50500cefc0 0x7f50501570a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:46.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:46.962+0000 7f504f200640 1 -- 172.21.15.151:0/3907244034 >> 172.21.15.151:0/3907244034 conn(0x7f50500b1510 msgr2=0x7f50500af2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:46.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:46.962+0000 7f504f200640 1 -- 172.21.15.151:0/3907244034 shutdown_connections 2024-09-16T15:51:46.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:46.962+0000 7f504f200640 1 -- 172.21.15.151:0/3907244034 wait complete. 2024-09-16T15:51:46.988 DEBUG:tasks.cephfs.test_volumes:percentage cloned = 0.0% 2024-09-16T15:51:47.989 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-09-16T15:51:48.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:48.233+0000 7f8526800640 1 -- 172.21.15.151:0/3503980732 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f85200af170 msgr2=0x7f85200cdad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:48.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:48.233+0000 7f8526800640 1 --2- 172.21.15.151:0/3503980732 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f85200af170 0x7f85200cdad0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f85200d0c10 tx=0x7f850c02a580 comp rx=0 tx=0).stop 2024-09-16T15:51:48.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:48.233+0000 7f8526800640 1 -- 172.21.15.151:0/3503980732 shutdown_connections 2024-09-16T15:51:48.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:48.233+0000 7f8526800640 1 --2- 172.21.15.151:0/3503980732 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f85200af170 0x7f85200cdad0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:48.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:48.233+0000 7f8526800640 1 --2- 172.21.15.151:0/3503980732 >> v2:172.21.15.167:3300/0 conn(0x7f8520161490 0x7f85200b3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:48.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:48.233+0000 7f8526800640 1 -- 172.21.15.151:0/3503980732 >> 172.21.15.151:0/3503980732 conn(0x7f85200b1840 msgr2=0x7f85200b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:48.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:48.233+0000 7f8526800640 1 -- 172.21.15.151:0/3503980732 shutdown_connections 2024-09-16T15:51:48.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:48.234+0000 7f8526800640 1 -- 172.21.15.151:0/3503980732 wait complete. 2024-09-16T15:51:48.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:48.236+0000 7f8526800640 1 Processor -- start 2024-09-16T15:51:48.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:48.236+0000 7f8526800640 1 -- start start 2024-09-16T15:51:48.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:48.237+0000 7f8526800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f85200af170 0x7f852015ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:48.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:48.237+0000 7f8526800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f85200b4370 0x7f85200c54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:48.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:48.237+0000 7f8526800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8520161490 0x7f852015b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:48.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:48.237+0000 7f8526800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8520161bb0 con 0x7f85200b4370 2024-09-16T15:51:48.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:48.237+0000 7f8526800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f85200701c0 con 0x7f8520161490 2024-09-16T15:51:48.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:48.237+0000 7f8526800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f8520161d30 con 0x7f85200af170 2024-09-16T15:51:48.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:48.237+0000 7f851ea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f85200b4370 0x7f85200c54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:48.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:48.237+0000 7f851ea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f85200b4370 0x7f85200c54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:38656/0 (socket says 172.21.15.151:38656) 2024-09-16T15:51:48.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:48.237+0000 7f851ea00640 1 -- 172.21.15.151:0/3653368790 learned_addr learned my addr 172.21.15.151:0/3653368790 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:51:48.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:48.238+0000 7f851fe00640 1 --2- 172.21.15.151:0/3653368790 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8520161490 0x7f852015b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:48.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:48.238+0000 7f851f400640 1 --2- 172.21.15.151:0/3653368790 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f85200af170 0x7f852015ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:48.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:48.238+0000 7f851ea00640 1 -- 172.21.15.151:0/3653368790 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f85200af170 msgr2=0x7f852015ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:48.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:48.238+0000 7f851ea00640 1 --2- 172.21.15.151:0/3653368790 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f85200af170 0x7f852015ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:48.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:48.238+0000 7f851ea00640 1 -- 172.21.15.151:0/3653368790 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8520161490 msgr2=0x7f852015b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:48.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:48.238+0000 7f851ea00640 1 --2- 172.21.15.151:0/3653368790 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8520161490 0x7f852015b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:48.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:48.238+0000 7f851ea00640 1 -- 172.21.15.151:0/3653368790 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f852015b070 con 0x7f85200b4370 2024-09-16T15:51:48.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:48.238+0000 7f851f400640 1 --2- 172.21.15.151:0/3653368790 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f85200af170 0x7f852015ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:51:48.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:48.239+0000 7f851ea00640 1 --2- 172.21.15.151:0/3653368790 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f85200b4370 0x7f85200c54c0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f850c009850 tx=0x7f850c0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:48.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:48.239+0000 7f8517400640 1 -- 172.21.15.151:0/3653368790 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850c03f070 con 0x7f85200b4370 2024-09-16T15:51:48.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:48.239+0000 7f8517400640 1 -- 172.21.15.151:0/3653368790 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f850c004890 con 0x7f85200b4370 2024-09-16T15:51:48.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:48.239+0000 7f8526800640 1 -- 172.21.15.151:0/3653368790 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85200c4c50 con 0x7f85200b4370 2024-09-16T15:51:48.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:48.239+0000 7f8517400640 1 -- 172.21.15.151:0/3653368790 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850c035030 con 0x7f85200b4370 2024-09-16T15:51:48.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:48.240+0000 7f8526800640 1 -- 172.21.15.151:0/3653368790 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85200c5210 con 0x7f85200b4370 2024-09-16T15:51:48.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:48.241+0000 7f8526800640 1 -- 172.21.15.151:0/3653368790 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8520070be0 con 0x7f85200b4370 2024-09-16T15:51:48.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:48.242+0000 7f8517400640 1 -- 172.21.15.151:0/3653368790 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 39) ==== 103531+0+0 (secure 0 0 0) 0x7f850c042070 con 0x7f85200b4370 2024-09-16T15:51:48.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:48.243+0000 7f8517400640 1 --2- 172.21.15.151:0/3653368790 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f850007a110 0x7f850007c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:48.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:48.243+0000 7f8517400640 1 -- 172.21.15.151:0/3653368790 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5765+0+0 (secure 0 0 0) 0x7f850c0ba6b0 con 0x7f85200b4370 2024-09-16T15:51:48.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:48.244+0000 7f851f400640 1 --2- 172.21.15.151:0/3653368790 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f850007a110 0x7f850007c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:48.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:48.244+0000 7f851f400640 1 --2- 172.21.15.151:0/3653368790 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f850007a110 0x7f850007c5d0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f8510006fd0 tx=0x7f8510008040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:48.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:48.248+0000 7f8517400640 1 -- 172.21.15.151:0/3653368790 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8520070be0 con 0x7f85200b4370 2024-09-16T15:51:48.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:48.472+0000 7f8526800640 1 -- 172.21.15.151:0/3653368790 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f85200ce870 con 0x7f850007a110 2024-09-16T15:51:48.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:48.489+0000 7f8517400640 1 -- 172.21.15.151:0/3653368790 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+282 (secure 0 0 0) 0x7f85200ce870 con 0x7f850007a110 2024-09-16T15:51:48.490 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-16T15:51:48.490 INFO:teuthology.orchestra.run.smithi151.stdout: "status": { 2024-09-16T15:51:48.490 INFO:teuthology.orchestra.run.smithi151.stdout: "state": "in-progress", 2024-09-16T15:51:48.490 INFO:teuthology.orchestra.run.smithi151.stdout: "source": { 2024-09-16T15:51:48.491 INFO:teuthology.orchestra.run.smithi151.stdout: "volume": "cephfs", 2024-09-16T15:51:48.491 INFO:teuthology.orchestra.run.smithi151.stdout: "subvolume": "sv1", 2024-09-16T15:51:48.491 INFO:teuthology.orchestra.run.smithi151.stdout: "snapshot": "ss1" 2024-09-16T15:51:48.491 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:51:48.491 INFO:teuthology.orchestra.run.smithi151.stdout: "progress_report": { 2024-09-16T15:51:48.491 INFO:teuthology.orchestra.run.smithi151.stdout: "percentage cloned": "23.958%", 2024-09-16T15:51:48.491 INFO:teuthology.orchestra.run.smithi151.stdout: "amount cloned": "736M/3.0G", 2024-09-16T15:51:48.491 INFO:teuthology.orchestra.run.smithi151.stdout: "files cloned": "4/4" 2024-09-16T15:51:48.491 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:51:48.491 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:51:48.491 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-16T15:51:48.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:48.491+0000 7f8526800640 1 -- 172.21.15.151:0/3653368790 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f850007a110 msgr2=0x7f850007c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:48.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:48.491+0000 7f8526800640 1 --2- 172.21.15.151:0/3653368790 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f850007a110 0x7f850007c5d0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f8510006fd0 tx=0x7f8510008040 comp rx=0 tx=0).stop 2024-09-16T15:51:48.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:48.491+0000 7f8526800640 1 -- 172.21.15.151:0/3653368790 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f85200b4370 msgr2=0x7f85200c54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:48.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:48.491+0000 7f8526800640 1 --2- 172.21.15.151:0/3653368790 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f85200b4370 0x7f85200c54c0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f850c009850 tx=0x7f850c0039e0 comp rx=0 tx=0).stop 2024-09-16T15:51:48.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:48.491+0000 7f8526800640 1 -- 172.21.15.151:0/3653368790 shutdown_connections 2024-09-16T15:51:48.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:48.491+0000 7f8526800640 1 --2- 172.21.15.151:0/3653368790 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f850007a110 0x7f850007c5d0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:48.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:48.491+0000 7f8526800640 1 --2- 172.21.15.151:0/3653368790 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8520161490 0x7f852015b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:48.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:48.491+0000 7f8526800640 1 --2- 172.21.15.151:0/3653368790 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f85200b4370 0x7f85200c54c0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:48.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:48.491+0000 7f8526800640 1 --2- 172.21.15.151:0/3653368790 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f85200af170 0x7f852015ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:48.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:48.491+0000 7f8526800640 1 -- 172.21.15.151:0/3653368790 >> 172.21.15.151:0/3653368790 conn(0x7f85200b1840 msgr2=0x7f852015de60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:48.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:48.491+0000 7f8526800640 1 -- 172.21.15.151:0/3653368790 shutdown_connections 2024-09-16T15:51:48.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:48.491+0000 7f8526800640 1 -- 172.21.15.151:0/3653368790 wait complete. 2024-09-16T15:51:48.508 DEBUG:tasks.cephfs.test_volumes:percentage cloned = 23.958% 2024-09-16T15:51:49.509 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-09-16T15:51:49.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:49.674+0000 7f28e0600640 1 -- 172.21.15.151:0/1139452556 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28d806efa0 msgr2=0x7f28d8073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:49.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:49.674+0000 7f28e0600640 1 --2- 172.21.15.151:0/1139452556 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28d806efa0 0x7f28d8073440 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f28d0005fd0 tx=0x7f28d002a820 comp rx=0 tx=0).stop 2024-09-16T15:51:49.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:49.675+0000 7f28e0600640 1 -- 172.21.15.151:0/1139452556 shutdown_connections 2024-09-16T15:51:49.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:49.675+0000 7f28e0600640 1 --2- 172.21.15.151:0/1139452556 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28d806efa0 0x7f28d8073440 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:49.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:49.675+0000 7f28e0600640 1 -- 172.21.15.151:0/1139452556 >> 172.21.15.151:0/1139452556 conn(0x7f28d80b1840 msgr2=0x7f28d80b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:49.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:49.675+0000 7f28e0600640 1 -- 172.21.15.151:0/1139452556 shutdown_connections 2024-09-16T15:51:49.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:49.675+0000 7f28e0600640 1 -- 172.21.15.151:0/1139452556 wait complete. 2024-09-16T15:51:49.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:49.676+0000 7f28e0600640 1 Processor -- start 2024-09-16T15:51:49.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:49.676+0000 7f28e0600640 1 -- start start 2024-09-16T15:51:49.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:49.677+0000 7f28e0600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28d806efa0 0x7f28d815e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:49.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:49.677+0000 7f28e0600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28d80cccd0 0x7f28d80c60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:49.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:49.677+0000 7f28e0600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f28d8145750 0x7f28d81f7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:49.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:49.677+0000 7f28e0600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f28d80ce6d0 con 0x7f28d806efa0 2024-09-16T15:51:49.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:49.677+0000 7f28e0600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f28d80ce550 con 0x7f28d80cccd0 2024-09-16T15:51:49.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:49.677+0000 7f28e0600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f28d80ce850 con 0x7f28d8145750 2024-09-16T15:51:49.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:49.678+0000 7f28ddc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28d806efa0 0x7f28d815e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:49.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:49.678+0000 7f28ddc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28d806efa0 0x7f28d815e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:38668/0 (socket says 172.21.15.151:38668) 2024-09-16T15:51:49.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:49.678+0000 7f28ddc00640 1 -- 172.21.15.151:0/314135892 learned_addr learned my addr 172.21.15.151:0/314135892 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:51:49.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:49.678+0000 7f28dd200640 1 --2- 172.21.15.151:0/314135892 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28d80cccd0 0x7f28d80c60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:49.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:49.678+0000 7f28de600640 1 --2- 172.21.15.151:0/314135892 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f28d8145750 0x7f28d81f7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:49.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:49.678+0000 7f28ddc00640 1 -- 172.21.15.151:0/314135892 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f28d8145750 msgr2=0x7f28d81f7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:49.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:49.678+0000 7f28ddc00640 1 --2- 172.21.15.151:0/314135892 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f28d8145750 0x7f28d81f7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:49.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:49.678+0000 7f28ddc00640 1 -- 172.21.15.151:0/314135892 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28d80cccd0 msgr2=0x7f28d80c60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:49.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:49.678+0000 7f28ddc00640 1 --2- 172.21.15.151:0/314135892 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28d80cccd0 0x7f28d80c60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:49.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:49.678+0000 7f28ddc00640 1 -- 172.21.15.151:0/314135892 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28d80c69d0 con 0x7f28d806efa0 2024-09-16T15:51:49.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:49.678+0000 7f28dd200640 1 --2- 172.21.15.151:0/314135892 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28d80cccd0 0x7f28d80c60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:51:49.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:49.678+0000 7f28de600640 1 --2- 172.21.15.151:0/314135892 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f28d8145750 0x7f28d81f7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:51:49.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:49.678+0000 7f28ddc00640 1 --2- 172.21.15.151:0/314135892 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28d806efa0 0x7f28d815e570 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f28cc00c430 tx=0x7f28cc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:49.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:49.679+0000 7f28d5600640 1 -- 172.21.15.151:0/314135892 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28cc011790 con 0x7f28d806efa0 2024-09-16T15:51:49.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:49.679+0000 7f28d5600640 1 -- 172.21.15.151:0/314135892 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f28cc017070 con 0x7f28d806efa0 2024-09-16T15:51:49.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:49.679+0000 7f28e0600640 1 -- 172.21.15.151:0/314135892 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28d80c6cb0 con 0x7f28d806efa0 2024-09-16T15:51:49.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:49.679+0000 7f28d5600640 1 -- 172.21.15.151:0/314135892 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28cc011ce0 con 0x7f28d806efa0 2024-09-16T15:51:49.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:49.679+0000 7f28e0600640 1 -- 172.21.15.151:0/314135892 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28d80c7270 con 0x7f28d806efa0 2024-09-16T15:51:49.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:49.680+0000 7f28e0600640 1 -- 172.21.15.151:0/314135892 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28d80ced00 con 0x7f28d806efa0 2024-09-16T15:51:49.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:49.681+0000 7f28d5600640 1 -- 172.21.15.151:0/314135892 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 40) ==== 103582+0+0 (secure 0 0 0) 0x7f28cc01c460 con 0x7f28d806efa0 2024-09-16T15:51:49.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:49.682+0000 7f28d5600640 1 --2- 172.21.15.151:0/314135892 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f28b807a1d0 0x7f28b807c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:49.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:49.683+0000 7f28d5600640 1 -- 172.21.15.151:0/314135892 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5765+0+0 (secure 0 0 0) 0x7f28cc099550 con 0x7f28d806efa0 2024-09-16T15:51:49.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:49.683+0000 7f28dd200640 1 --2- 172.21.15.151:0/314135892 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f28b807a1d0 0x7f28b807c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:49.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:49.684+0000 7f28dd200640 1 --2- 172.21.15.151:0/314135892 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f28b807a1d0 0x7f28b807c690 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f28d80c7f40 tx=0x7f28c4009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:49.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:49.687+0000 7f28d5600640 1 -- 172.21.15.151:0/314135892 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f28cc063620 con 0x7f28d806efa0 2024-09-16T15:51:49.907 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:49.906+0000 7f28e0600640 1 -- 172.21.15.151:0/314135892 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f28d80d0430 con 0x7f28b807a1d0 2024-09-16T15:51:49.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:49.929+0000 7f28d5600640 1 -- 172.21.15.151:0/314135892 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+282 (secure 0 0 0) 0x7f28d80d0430 con 0x7f28b807a1d0 2024-09-16T15:51:49.930 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-16T15:51:49.930 INFO:teuthology.orchestra.run.smithi151.stdout: "status": { 2024-09-16T15:51:49.930 INFO:teuthology.orchestra.run.smithi151.stdout: "state": "in-progress", 2024-09-16T15:51:49.930 INFO:teuthology.orchestra.run.smithi151.stdout: "source": { 2024-09-16T15:51:49.930 INFO:teuthology.orchestra.run.smithi151.stdout: "volume": "cephfs", 2024-09-16T15:51:49.930 INFO:teuthology.orchestra.run.smithi151.stdout: "subvolume": "sv1", 2024-09-16T15:51:49.930 INFO:teuthology.orchestra.run.smithi151.stdout: "snapshot": "ss1" 2024-09-16T15:51:49.930 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:51:49.930 INFO:teuthology.orchestra.run.smithi151.stdout: "progress_report": { 2024-09-16T15:51:49.931 INFO:teuthology.orchestra.run.smithi151.stdout: "percentage cloned": "23.958%", 2024-09-16T15:51:49.931 INFO:teuthology.orchestra.run.smithi151.stdout: "amount cloned": "736M/3.0G", 2024-09-16T15:51:49.931 INFO:teuthology.orchestra.run.smithi151.stdout: "files cloned": "4/4" 2024-09-16T15:51:49.931 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:51:49.931 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:51:49.931 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-16T15:51:49.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:49.931+0000 7f28e0600640 1 -- 172.21.15.151:0/314135892 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f28b807a1d0 msgr2=0x7f28b807c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:49.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:49.931+0000 7f28e0600640 1 --2- 172.21.15.151:0/314135892 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f28b807a1d0 0x7f28b807c690 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f28d80c7f40 tx=0x7f28c4009040 comp rx=0 tx=0).stop 2024-09-16T15:51:49.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:49.931+0000 7f28e0600640 1 -- 172.21.15.151:0/314135892 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28d806efa0 msgr2=0x7f28d815e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:49.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:49.931+0000 7f28e0600640 1 --2- 172.21.15.151:0/314135892 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28d806efa0 0x7f28d815e570 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f28cc00c430 tx=0x7f28cc00c8f0 comp rx=0 tx=0).stop 2024-09-16T15:51:49.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:49.931+0000 7f28e0600640 1 -- 172.21.15.151:0/314135892 shutdown_connections 2024-09-16T15:51:49.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:49.931+0000 7f28e0600640 1 --2- 172.21.15.151:0/314135892 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f28b807a1d0 0x7f28b807c690 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:49.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:49.931+0000 7f28e0600640 1 --2- 172.21.15.151:0/314135892 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28d80cccd0 0x7f28d80c60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:49.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:49.931+0000 7f28e0600640 1 --2- 172.21.15.151:0/314135892 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f28d8145750 0x7f28d81f7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:49.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:49.931+0000 7f28e0600640 1 --2- 172.21.15.151:0/314135892 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28d806efa0 0x7f28d815e570 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:49.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:49.931+0000 7f28e0600640 1 -- 172.21.15.151:0/314135892 >> 172.21.15.151:0/314135892 conn(0x7f28d80b1840 msgr2=0x7f28d80af780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:49.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:49.931+0000 7f28e0600640 1 -- 172.21.15.151:0/314135892 shutdown_connections 2024-09-16T15:51:49.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:49.931+0000 7f28e0600640 1 -- 172.21.15.151:0/314135892 wait complete. 2024-09-16T15:51:49.948 DEBUG:tasks.cephfs.test_volumes:percentage cloned = 23.958% 2024-09-16T15:51:50.949 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-09-16T15:51:51.045 DEBUG:teuthology.orchestra.run.smithi151:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:51:51.050 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:51:51.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:51.123+0000 7f015d200640 1 -- 172.21.15.151:0/726204789 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01580cbc00 msgr2=0x7f0158073ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:51.125 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:51.123+0000 7f015d200640 1 --2- 172.21.15.151:0/726204789 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01580cbc00 0x7f0158073ef0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f01580c9ff0 tx=0x7f015002a8b0 comp rx=0 tx=0).stop 2024-09-16T15:51:51.125 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:51.123+0000 7f015d200640 1 -- 172.21.15.151:0/726204789 shutdown_connections 2024-09-16T15:51:51.125 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:51.123+0000 7f015d200640 1 --2- 172.21.15.151:0/726204789 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01580cbc00 0x7f0158073ef0 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:51.125 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:51.123+0000 7f015d200640 1 --2- 172.21.15.151:0/726204789 >> v2:172.21.15.167:3300/0 conn(0x7f015815bec0 0x7f015806fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:51.125 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:51.123+0000 7f015d200640 1 -- 172.21.15.151:0/726204789 >> 172.21.15.151:0/726204789 conn(0x7f01580cf9f0 msgr2=0x7f01580cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:51.125 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:51.123+0000 7f015d200640 1 -- 172.21.15.151:0/726204789 shutdown_connections 2024-09-16T15:51:51.125 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:51.123+0000 7f015d200640 1 -- 172.21.15.151:0/726204789 wait complete. 2024-09-16T15:51:51.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:51.125+0000 7f015d200640 1 Processor -- start 2024-09-16T15:51:51.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:51.125+0000 7f015d200640 1 -- start start 2024-09-16T15:51:51.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:51.126+0000 7f015d200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0158072560 0x7f01581489d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:51.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:51.126+0000 7f015d200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01580cbc00 0x7f01580a9470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:51.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:51.126+0000 7f015d200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f015815bec0 0x7f01581f8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:51.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:51.126+0000 7f015d200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f01580ae930 con 0x7f0158072560 2024-09-16T15:51:51.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:51.126+0000 7f015d200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f01580ae7b0 con 0x7f015815bec0 2024-09-16T15:51:51.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:51.126+0000 7f015d200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f01580aeab0 con 0x7f01580cbc00 2024-09-16T15:51:51.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:51.126+0000 7f0157400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0158072560 0x7f01581489d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:51.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:51.126+0000 7f0157400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0158072560 0x7f01581489d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:38696/0 (socket says 172.21.15.151:38696) 2024-09-16T15:51:51.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:51.126+0000 7f0157400640 1 -- 172.21.15.151:0/3756060439 learned_addr learned my addr 172.21.15.151:0/3756060439 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:51:51.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:51.126+0000 7f0157e00640 1 --2- 172.21.15.151:0/3756060439 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f015815bec0 0x7f01581f8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:51.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:51.126+0000 7f0156a00640 1 --2- 172.21.15.151:0/3756060439 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01580cbc00 0x7f01580a9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:51.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:51.127+0000 7f0157400640 1 -- 172.21.15.151:0/3756060439 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01580cbc00 msgr2=0x7f01580a9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:51.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:51.127+0000 7f0157400640 1 --2- 172.21.15.151:0/3756060439 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01580cbc00 0x7f01580a9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:51.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:51.127+0000 7f0157400640 1 -- 172.21.15.151:0/3756060439 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f015815bec0 msgr2=0x7f01581f8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:51.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:51.127+0000 7f0157400640 1 --2- 172.21.15.151:0/3756060439 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f015815bec0 0x7f01581f8060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:51.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:51.127+0000 7f0157400640 1 -- 172.21.15.151:0/3756060439 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01580ce170 con 0x7f0158072560 2024-09-16T15:51:51.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:51.127+0000 7f0157400640 1 --2- 172.21.15.151:0/3756060439 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0158072560 0x7f01581489d0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f014800d740 tx=0x7f014800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:51.130 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:51.127+0000 7f014f400640 1 -- 172.21.15.151:0/3756060439 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0148017070 con 0x7f0158072560 2024-09-16T15:51:51.130 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:51.127+0000 7f014f400640 1 -- 172.21.15.151:0/3756060439 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0148009d20 con 0x7f0158072560 2024-09-16T15:51:51.130 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:51.127+0000 7f014f400640 1 -- 172.21.15.151:0/3756060439 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01480106c0 con 0x7f0158072560 2024-09-16T15:51:51.130 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:51.127+0000 7f015d200640 1 -- 172.21.15.151:0/3756060439 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01580ce450 con 0x7f0158072560 2024-09-16T15:51:51.130 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:51.127+0000 7f015d200640 1 -- 172.21.15.151:0/3756060439 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01580cea10 con 0x7f0158072560 2024-09-16T15:51:51.130 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:51.128+0000 7f015d200640 1 -- 172.21.15.151:0/3756060439 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f015806e7d0 con 0x7f0158072560 2024-09-16T15:51:51.130 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:51.130+0000 7f014f400640 1 -- 172.21.15.151:0/3756060439 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 40) ==== 103582+0+0 (secure 0 0 0) 0x7f014801a070 con 0x7f0158072560 2024-09-16T15:51:51.131 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:51.130+0000 7f014f400640 1 --2- 172.21.15.151:0/3756060439 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f013807a1d0 0x7f013807c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:51.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:51.131+0000 7f014f400640 1 -- 172.21.15.151:0/3756060439 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5765+0+0 (secure 0 0 0) 0x7f0148099830 con 0x7f0158072560 2024-09-16T15:51:51.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:51.131+0000 7f0156a00640 1 --2- 172.21.15.151:0/3756060439 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f013807a1d0 0x7f013807c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:51.133 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:51.132+0000 7f0156a00640 1 --2- 172.21.15.151:0/3756060439 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f013807a1d0 0x7f013807c690 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f01440017c0 tx=0x7f0144007e90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:51.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:51.135+0000 7f014f400640 1 -- 172.21.15.151:0/3756060439 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f015806e7d0 con 0x7f0158072560 2024-09-16T15:51:51.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:51.355+0000 7f015d200640 1 -- 172.21.15.151:0/3756060439 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f0158072040 con 0x7f013807a1d0 2024-09-16T15:51:51.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:51.377+0000 7f014f400640 1 -- 172.21.15.151:0/3756060439 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+282 (secure 0 0 0) 0x7f0158072040 con 0x7f013807a1d0 2024-09-16T15:51:51.378 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-16T15:51:51.378 INFO:teuthology.orchestra.run.smithi151.stdout: "status": { 2024-09-16T15:51:51.378 INFO:teuthology.orchestra.run.smithi151.stdout: "state": "in-progress", 2024-09-16T15:51:51.378 INFO:teuthology.orchestra.run.smithi151.stdout: "source": { 2024-09-16T15:51:51.378 INFO:teuthology.orchestra.run.smithi151.stdout: "volume": "cephfs", 2024-09-16T15:51:51.378 INFO:teuthology.orchestra.run.smithi151.stdout: "subvolume": "sv1", 2024-09-16T15:51:51.378 INFO:teuthology.orchestra.run.smithi151.stdout: "snapshot": "ss1" 2024-09-16T15:51:51.379 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:51:51.379 INFO:teuthology.orchestra.run.smithi151.stdout: "progress_report": { 2024-09-16T15:51:51.379 INFO:teuthology.orchestra.run.smithi151.stdout: "percentage cloned": "23.958%", 2024-09-16T15:51:51.379 INFO:teuthology.orchestra.run.smithi151.stdout: "amount cloned": "736M/3.0G", 2024-09-16T15:51:51.379 INFO:teuthology.orchestra.run.smithi151.stdout: "files cloned": "4/4" 2024-09-16T15:51:51.379 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:51:51.379 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:51:51.379 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-16T15:51:51.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:51.379+0000 7f015d200640 1 -- 172.21.15.151:0/3756060439 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f013807a1d0 msgr2=0x7f013807c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:51.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:51.379+0000 7f015d200640 1 --2- 172.21.15.151:0/3756060439 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f013807a1d0 0x7f013807c690 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f01440017c0 tx=0x7f0144007e90 comp rx=0 tx=0).stop 2024-09-16T15:51:51.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:51.379+0000 7f015d200640 1 -- 172.21.15.151:0/3756060439 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0158072560 msgr2=0x7f01581489d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:51.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:51.379+0000 7f015d200640 1 --2- 172.21.15.151:0/3756060439 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0158072560 0x7f01581489d0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f014800d740 tx=0x7f014800dc00 comp rx=0 tx=0).stop 2024-09-16T15:51:51.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:51.379+0000 7f015d200640 1 -- 172.21.15.151:0/3756060439 shutdown_connections 2024-09-16T15:51:51.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:51.379+0000 7f015d200640 1 --2- 172.21.15.151:0/3756060439 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f013807a1d0 0x7f013807c690 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:51.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:51.380+0000 7f015d200640 1 --2- 172.21.15.151:0/3756060439 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f015815bec0 0x7f01581f8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:51.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:51.380+0000 7f015d200640 1 --2- 172.21.15.151:0/3756060439 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01580cbc00 0x7f01580a9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:51.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:51.380+0000 7f015d200640 1 --2- 172.21.15.151:0/3756060439 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0158072560 0x7f01581489d0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:51.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:51.380+0000 7f015d200640 1 -- 172.21.15.151:0/3756060439 >> 172.21.15.151:0/3756060439 conn(0x7f01580cf9f0 msgr2=0x7f01580cfdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:51.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:51.380+0000 7f015d200640 1 -- 172.21.15.151:0/3756060439 shutdown_connections 2024-09-16T15:51:51.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:51.380+0000 7f015d200640 1 -- 172.21.15.151:0/3756060439 wait complete. 2024-09-16T15:51:51.397 DEBUG:tasks.cephfs.test_volumes:percentage cloned = 23.958% 2024-09-16T15:51:52.397 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-09-16T15:51:52.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:52.560+0000 7f35af200640 1 -- 172.21.15.151:0/2312656243 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f35a8145750 msgr2=0x7f35a8073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:52.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:52.560+0000 7f35af200640 1 --2- 172.21.15.151:0/2312656243 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f35a8145750 0x7f35a8073440 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f35a80af750 tx=0x7f359802a4d0 comp rx=0 tx=0).stop 2024-09-16T15:51:52.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:52.560+0000 7f35af200640 1 -- 172.21.15.151:0/2312656243 shutdown_connections 2024-09-16T15:51:52.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:52.560+0000 7f35af200640 1 --2- 172.21.15.151:0/2312656243 >> v2:172.21.15.167:3300/0 conn(0x7f35a806efa0 0x7f35a80cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:52.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:52.560+0000 7f35af200640 1 --2- 172.21.15.151:0/2312656243 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f35a8145750 0x7f35a8073440 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:52.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:52.560+0000 7f35af200640 1 -- 172.21.15.151:0/2312656243 >> 172.21.15.151:0/2312656243 conn(0x7f35a80b1840 msgr2=0x7f35a80b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:52.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:52.560+0000 7f35af200640 1 -- 172.21.15.151:0/2312656243 shutdown_connections 2024-09-16T15:51:52.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:52.560+0000 7f35af200640 1 -- 172.21.15.151:0/2312656243 wait complete. 2024-09-16T15:51:52.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:52.562+0000 7f35af200640 1 Processor -- start 2024-09-16T15:51:52.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:52.562+0000 7f35af200640 1 -- start start 2024-09-16T15:51:52.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:52.563+0000 7f35af200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f35a806efa0 0x7f35a81315f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:52.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:52.563+0000 7f35af200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f35a80ccb00 0x7f35a81f5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:52.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:52.563+0000 7f35af200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35a8145750 0x7f35a80ae640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:52.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:52.563+0000 7f35af200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f35a80ced40 con 0x7f35a80ccb00 2024-09-16T15:51:52.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:52.563+0000 7f35af200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f35a80cebc0 con 0x7f35a8145750 2024-09-16T15:51:52.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:52.563+0000 7f35af200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f35a80ceec0 con 0x7f35a806efa0 2024-09-16T15:51:52.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:52.563+0000 7f35a7400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f35a80ccb00 0x7f35a81f5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:52.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:52.563+0000 7f35a7400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f35a80ccb00 0x7f35a81f5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:38712/0 (socket says 172.21.15.151:38712) 2024-09-16T15:51:52.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:52.563+0000 7f35a7400640 1 -- 172.21.15.151:0/133824388 learned_addr learned my addr 172.21.15.151:0/133824388 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:51:52.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:52.564+0000 7f35a7e00640 1 --2- 172.21.15.151:0/133824388 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f35a806efa0 0x7f35a81315f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:52.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:52.564+0000 7f35ad200640 1 --2- 172.21.15.151:0/133824388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35a8145750 0x7f35a80ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:52.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:52.564+0000 7f35a7400640 1 -- 172.21.15.151:0/133824388 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f35a806efa0 msgr2=0x7f35a81315f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:52.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:52.564+0000 7f35a7400640 1 --2- 172.21.15.151:0/133824388 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f35a806efa0 0x7f35a81315f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:52.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:52.564+0000 7f35a7400640 1 -- 172.21.15.151:0/133824388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35a8145750 msgr2=0x7f35a80ae640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:52.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:52.564+0000 7f35a7400640 1 --2- 172.21.15.151:0/133824388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35a8145750 0x7f35a80ae640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:52.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:52.564+0000 7f35a7400640 1 -- 172.21.15.151:0/133824388 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35a81f4a60 con 0x7f35a80ccb00 2024-09-16T15:51:52.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:52.564+0000 7f35a7e00640 1 --2- 172.21.15.151:0/133824388 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f35a806efa0 0x7f35a81315f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:51:52.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:52.564+0000 7f35a7400640 1 --2- 172.21.15.151:0/133824388 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f35a80ccb00 0x7f35a81f5240 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f359400c920 tx=0x7f359400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:52.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:52.564+0000 7f359fe00640 1 -- 172.21.15.151:0/133824388 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3594007c70 con 0x7f35a80ccb00 2024-09-16T15:51:52.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:52.564+0000 7f359fe00640 1 -- 172.21.15.151:0/133824388 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f359400f450 con 0x7f35a80ccb00 2024-09-16T15:51:52.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:52.564+0000 7f359fe00640 1 -- 172.21.15.151:0/133824388 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359400f8a0 con 0x7f35a80ccb00 2024-09-16T15:51:52.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:52.564+0000 7f35af200640 1 -- 172.21.15.151:0/133824388 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35a8131e20 con 0x7f35a80ccb00 2024-09-16T15:51:52.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:52.565+0000 7f35af200640 1 -- 172.21.15.151:0/133824388 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35a81323e0 con 0x7f35a80ccb00 2024-09-16T15:51:52.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:52.565+0000 7f35af200640 1 -- 172.21.15.151:0/133824388 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35a80704f0 con 0x7f35a80ccb00 2024-09-16T15:51:52.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:52.567+0000 7f359fe00640 1 -- 172.21.15.151:0/133824388 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 40) ==== 103582+0+0 (secure 0 0 0) 0x7f359400fa40 con 0x7f35a80ccb00 2024-09-16T15:51:52.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:52.568+0000 7f359fe00640 1 --2- 172.21.15.151:0/133824388 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f358807a1d0 0x7f358807c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:52.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:52.568+0000 7f359fe00640 1 -- 172.21.15.151:0/133824388 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5765+0+0 (secure 0 0 0) 0x7f359409a040 con 0x7f35a80ccb00 2024-09-16T15:51:52.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:52.568+0000 7f35a7e00640 1 --2- 172.21.15.151:0/133824388 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f358807a1d0 0x7f358807c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:52.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:52.569+0000 7f35a7e00640 1 --2- 172.21.15.151:0/133824388 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f358807a1d0 0x7f358807c690 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f3598033000 tx=0x7f3598004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:52.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:52.572+0000 7f359fe00640 1 -- 172.21.15.151:0/133824388 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f35a80704f0 con 0x7f35a80ccb00 2024-09-16T15:51:52.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:52.794+0000 7f35af200640 1 -- 172.21.15.151:0/133824388 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f35a80ae080 con 0x7f358807a1d0 2024-09-16T15:51:52.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:52.818+0000 7f359fe00640 1 -- 172.21.15.151:0/133824388 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+282 (secure 0 0 0) 0x7f35a80ae080 con 0x7f358807a1d0 2024-09-16T15:51:52.819 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-16T15:51:52.819 INFO:teuthology.orchestra.run.smithi151.stdout: "status": { 2024-09-16T15:51:52.819 INFO:teuthology.orchestra.run.smithi151.stdout: "state": "in-progress", 2024-09-16T15:51:52.819 INFO:teuthology.orchestra.run.smithi151.stdout: "source": { 2024-09-16T15:51:52.819 INFO:teuthology.orchestra.run.smithi151.stdout: "volume": "cephfs", 2024-09-16T15:51:52.819 INFO:teuthology.orchestra.run.smithi151.stdout: "subvolume": "sv1", 2024-09-16T15:51:52.820 INFO:teuthology.orchestra.run.smithi151.stdout: "snapshot": "ss1" 2024-09-16T15:51:52.820 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:51:52.820 INFO:teuthology.orchestra.run.smithi151.stdout: "progress_report": { 2024-09-16T15:51:52.820 INFO:teuthology.orchestra.run.smithi151.stdout: "percentage cloned": "23.958%", 2024-09-16T15:51:52.820 INFO:teuthology.orchestra.run.smithi151.stdout: "amount cloned": "736M/3.0G", 2024-09-16T15:51:52.820 INFO:teuthology.orchestra.run.smithi151.stdout: "files cloned": "4/4" 2024-09-16T15:51:52.820 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:51:52.820 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:51:52.820 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-16T15:51:52.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:52.820+0000 7f35af200640 1 -- 172.21.15.151:0/133824388 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f358807a1d0 msgr2=0x7f358807c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:52.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:52.820+0000 7f35af200640 1 --2- 172.21.15.151:0/133824388 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f358807a1d0 0x7f358807c690 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f3598033000 tx=0x7f3598004730 comp rx=0 tx=0).stop 2024-09-16T15:51:52.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:52.820+0000 7f35af200640 1 -- 172.21.15.151:0/133824388 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f35a80ccb00 msgr2=0x7f35a81f5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:52.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:52.820+0000 7f35af200640 1 --2- 172.21.15.151:0/133824388 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f35a80ccb00 0x7f35a81f5240 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f359400c920 tx=0x7f359400cde0 comp rx=0 tx=0).stop 2024-09-16T15:51:52.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:52.821+0000 7f35af200640 1 -- 172.21.15.151:0/133824388 shutdown_connections 2024-09-16T15:51:52.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:52.821+0000 7f35af200640 1 --2- 172.21.15.151:0/133824388 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f358807a1d0 0x7f358807c690 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:52.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:52.821+0000 7f35af200640 1 --2- 172.21.15.151:0/133824388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35a8145750 0x7f35a80ae640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:52.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:52.821+0000 7f35af200640 1 --2- 172.21.15.151:0/133824388 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f35a80ccb00 0x7f35a81f5240 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:52.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:52.821+0000 7f35af200640 1 --2- 172.21.15.151:0/133824388 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f35a806efa0 0x7f35a81315f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:52.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:52.821+0000 7f35af200640 1 -- 172.21.15.151:0/133824388 >> 172.21.15.151:0/133824388 conn(0x7f35a80b1840 msgr2=0x7f35a80cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:52.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:52.821+0000 7f35af200640 1 -- 172.21.15.151:0/133824388 shutdown_connections 2024-09-16T15:51:52.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:52.821+0000 7f35af200640 1 -- 172.21.15.151:0/133824388 wait complete. 2024-09-16T15:51:52.837 DEBUG:tasks.cephfs.test_volumes:percentage cloned = 23.958% 2024-09-16T15:51:53.839 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-09-16T15:51:54.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:54.067+0000 7fd888c00640 1 -- 172.21.15.151:0/1308396855 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8840b4e70 msgr2=0x7fd8841625b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:54.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:54.067+0000 7fd888c00640 1 --2- 172.21.15.151:0/1308396855 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8840b4e70 0x7fd8841625b0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fd87c005fd0 tx=0x7fd87c02a990 comp rx=0 tx=0).stop 2024-09-16T15:51:54.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:54.067+0000 7fd888c00640 1 -- 172.21.15.151:0/1308396855 shutdown_connections 2024-09-16T15:51:54.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:54.067+0000 7fd888c00640 1 --2- 172.21.15.151:0/1308396855 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8840b4e70 0x7fd8841625b0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:54.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:54.067+0000 7fd888c00640 1 --2- 172.21.15.151:0/1308396855 >> v2:172.21.15.167:3300/0 conn(0x7fd8840d2000 0x7fd8840d23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:54.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:54.067+0000 7fd888c00640 1 -- 172.21.15.151:0/1308396855 >> 172.21.15.151:0/1308396855 conn(0x7fd8840d3e20 msgr2=0x7fd8840d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:54.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:54.068+0000 7fd888c00640 1 -- 172.21.15.151:0/1308396855 shutdown_connections 2024-09-16T15:51:54.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:54.068+0000 7fd888c00640 1 -- 172.21.15.151:0/1308396855 wait complete. 2024-09-16T15:51:54.073 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:54.072+0000 7fd888c00640 1 Processor -- start 2024-09-16T15:51:54.073 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:54.072+0000 7fd888c00640 1 -- start start 2024-09-16T15:51:54.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:54.073+0000 7fd888c00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8840b4e70 0x7fd88414b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:54.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:54.073+0000 7fd888c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8840d2000 0x7fd8841f4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:54.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:54.073+0000 7fd888c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8840d2920 0x7fd8840b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:54.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:54.073+0000 7fd888c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd8840b1a10 con 0x7fd8840d2920 2024-09-16T15:51:54.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:54.073+0000 7fd888c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd8840b1890 con 0x7fd8840d2000 2024-09-16T15:51:54.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:54.073+0000 7fd888c00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fd8840b1b90 con 0x7fd8840b4e70 2024-09-16T15:51:54.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:54.074+0000 7fd883400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8840b4e70 0x7fd88414b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:54.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:54.074+0000 7fd883e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8840d2920 0x7fd8840b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:54.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:54.074+0000 7fd883400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8840b4e70 0x7fd88414b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.151:43198/0 (socket says 172.21.15.151:43198) 2024-09-16T15:51:54.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:54.074+0000 7fd883400640 1 -- 172.21.15.151:0/3490068841 learned_addr learned my addr 172.21.15.151:0/3490068841 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:51:54.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:54.074+0000 7fd883e00640 1 -- 172.21.15.151:0/3490068841 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8840b4e70 msgr2=0x7fd88414b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:54.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:54.074+0000 7fd883e00640 1 --2- 172.21.15.151:0/3490068841 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8840b4e70 0x7fd88414b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:54.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:54.074+0000 7fd883e00640 1 -- 172.21.15.151:0/3490068841 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8840d2000 msgr2=0x7fd8841f4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:54.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:54.074+0000 7fd883e00640 1 --2- 172.21.15.151:0/3490068841 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8840d2000 0x7fd8841f4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:54.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:54.074+0000 7fd883e00640 1 -- 172.21.15.151:0/3490068841 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8840c80d0 con 0x7fd8840d2920 2024-09-16T15:51:54.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:54.075+0000 7fd883e00640 1 --2- 172.21.15.151:0/3490068841 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8840d2920 0x7fd8840b89e0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fd87c009000 tx=0x7fd87c031b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:54.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:54.075+0000 7fd87b400640 1 -- 172.21.15.151:0/3490068841 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd87c004240 con 0x7fd8840d2920 2024-09-16T15:51:54.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:54.076+0000 7fd87b400640 1 -- 172.21.15.151:0/3490068841 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd87c0044f0 con 0x7fd8840d2920 2024-09-16T15:51:54.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:54.076+0000 7fd87b400640 1 -- 172.21.15.151:0/3490068841 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd87c045480 con 0x7fd8840d2920 2024-09-16T15:51:54.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:54.077+0000 7fd888c00640 1 -- 172.21.15.151:0/3490068841 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8840ca9b0 con 0x7fd8840d2920 2024-09-16T15:51:54.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:54.077+0000 7fd888c00640 1 -- 172.21.15.151:0/3490068841 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8840c7d60 con 0x7fd8840d2920 2024-09-16T15:51:54.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:54.078+0000 7fd888c00640 1 -- 172.21.15.151:0/3490068841 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8840d23e0 con 0x7fd8840d2920 2024-09-16T15:51:54.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:54.079+0000 7fd87b400640 1 -- 172.21.15.151:0/3490068841 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 40) ==== 103582+0+0 (secure 0 0 0) 0x7fd87c039440 con 0x7fd8840d2920 2024-09-16T15:51:54.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:54.081+0000 7fd87b400640 1 --2- 172.21.15.151:0/3490068841 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd86407a1a0 0x7fd86407c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:54.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:54.081+0000 7fd87b400640 1 -- 172.21.15.151:0/3490068841 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5765+0+0 (secure 0 0 0) 0x7fd87c0baf60 con 0x7fd8840d2920 2024-09-16T15:51:54.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:54.082+0000 7fd883400640 1 --2- 172.21.15.151:0/3490068841 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd86407a1a0 0x7fd86407c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:54.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:54.085+0000 7fd883400640 1 --2- 172.21.15.151:0/3490068841 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd86407a1a0 0x7fd86407c660 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fd874009c20 tx=0x7fd874005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:54.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:54.085+0000 7fd87b400640 1 -- 172.21.15.151:0/3490068841 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd87c0850d0 con 0x7fd8840d2920 2024-09-16T15:51:54.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:54.420+0000 7fd888c00640 1 -- 172.21.15.151:0/3490068841 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7fd8840b8740 con 0x7fd86407a1a0 2024-09-16T15:51:54.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:54.513+0000 7fd87b400640 1 -- 172.21.15.151:0/3490068841 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+282 (secure 0 0 0) 0x7fd8840b8740 con 0x7fd86407a1a0 2024-09-16T15:51:54.514 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-16T15:51:54.514 INFO:teuthology.orchestra.run.smithi151.stdout: "status": { 2024-09-16T15:51:54.514 INFO:teuthology.orchestra.run.smithi151.stdout: "state": "in-progress", 2024-09-16T15:51:54.514 INFO:teuthology.orchestra.run.smithi151.stdout: "source": { 2024-09-16T15:51:54.514 INFO:teuthology.orchestra.run.smithi151.stdout: "volume": "cephfs", 2024-09-16T15:51:54.514 INFO:teuthology.orchestra.run.smithi151.stdout: "subvolume": "sv1", 2024-09-16T15:51:54.515 INFO:teuthology.orchestra.run.smithi151.stdout: "snapshot": "ss1" 2024-09-16T15:51:54.515 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:51:54.515 INFO:teuthology.orchestra.run.smithi151.stdout: "progress_report": { 2024-09-16T15:51:54.515 INFO:teuthology.orchestra.run.smithi151.stdout: "percentage cloned": "57.292%", 2024-09-16T15:51:54.515 INFO:teuthology.orchestra.run.smithi151.stdout: "amount cloned": "1.7G/3.0G", 2024-09-16T15:51:54.515 INFO:teuthology.orchestra.run.smithi151.stdout: "files cloned": "5/4" 2024-09-16T15:51:54.515 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:51:54.515 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:51:54.515 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-16T15:51:54.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:54.516+0000 7fd888c00640 1 -- 172.21.15.151:0/3490068841 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd86407a1a0 msgr2=0x7fd86407c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:54.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:54.516+0000 7fd888c00640 1 --2- 172.21.15.151:0/3490068841 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd86407a1a0 0x7fd86407c660 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fd874009c20 tx=0x7fd874005f90 comp rx=0 tx=0).stop 2024-09-16T15:51:54.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:54.516+0000 7fd888c00640 1 -- 172.21.15.151:0/3490068841 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8840d2920 msgr2=0x7fd8840b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:54.517 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:54.516+0000 7fd888c00640 1 --2- 172.21.15.151:0/3490068841 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8840d2920 0x7fd8840b89e0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fd87c009000 tx=0x7fd87c031b20 comp rx=0 tx=0).stop 2024-09-16T15:51:54.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:54.516+0000 7fd888c00640 1 -- 172.21.15.151:0/3490068841 shutdown_connections 2024-09-16T15:51:54.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:54.516+0000 7fd888c00640 1 --2- 172.21.15.151:0/3490068841 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd86407a1a0 0x7fd86407c660 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:54.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:54.516+0000 7fd888c00640 1 --2- 172.21.15.151:0/3490068841 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8840d2000 0x7fd8841f4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:54.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:54.516+0000 7fd888c00640 1 --2- 172.21.15.151:0/3490068841 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8840d2920 0x7fd8840b89e0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:54.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:54.516+0000 7fd888c00640 1 --2- 172.21.15.151:0/3490068841 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8840b4e70 0x7fd88414b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:54.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:54.516+0000 7fd888c00640 1 -- 172.21.15.151:0/3490068841 >> 172.21.15.151:0/3490068841 conn(0x7fd8840d3e20 msgr2=0x7fd88406e990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:54.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:54.516+0000 7fd888c00640 1 -- 172.21.15.151:0/3490068841 shutdown_connections 2024-09-16T15:51:54.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:54.516+0000 7fd888c00640 1 -- 172.21.15.151:0/3490068841 wait complete. 2024-09-16T15:51:54.535 DEBUG:tasks.cephfs.test_volumes:percentage cloned = 57.292% 2024-09-16T15:51:55.537 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-09-16T15:51:55.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:55.794+0000 7fbff6800640 1 -- 172.21.15.151:0/2312564915 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbff00cf2f0 msgr2=0x7fbff015e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:55.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:55.794+0000 7fbff6800640 1 --2- 172.21.15.151:0/2312564915 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbff00cf2f0 0x7fbff015e980 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc005fd0 tx=0x7fbfdc02a680 comp rx=0 tx=0).stop 2024-09-16T15:51:55.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:55.794+0000 7fbff6800640 1 -- 172.21.15.151:0/2312564915 shutdown_connections 2024-09-16T15:51:55.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:55.794+0000 7fbff6800640 1 --2- 172.21.15.151:0/2312564915 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbff00cf2f0 0x7fbff015e980 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:55.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:55.795+0000 7fbff6800640 1 -- 172.21.15.151:0/2312564915 >> 172.21.15.151:0/2312564915 conn(0x7fbff00b1840 msgr2=0x7fbff00b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:55.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:55.795+0000 7fbff6800640 1 -- 172.21.15.151:0/2312564915 shutdown_connections 2024-09-16T15:51:55.796 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:55.795+0000 7fbff6800640 1 -- 172.21.15.151:0/2312564915 wait complete. 2024-09-16T15:51:55.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:55.797+0000 7fbff6800640 1 Processor -- start 2024-09-16T15:51:55.800 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:55.798+0000 7fbff6800640 1 -- start start 2024-09-16T15:51:55.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:55.800+0000 7fbff6800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbff00b3370 0x7fbff0139f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:55.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:55.800+0000 7fbff6800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbff00cf2f0 0x7fbff0073f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:55.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:55.800+0000 7fbff6800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbff015c080 0x7fbff0159df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:55.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:55.800+0000 7fbff6800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbff0166080 con 0x7fbff00cf2f0 2024-09-16T15:51:55.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:55.800+0000 7fbff6800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbff0165f00 con 0x7fbff015c080 2024-09-16T15:51:55.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:55.800+0000 7fbff6800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fbff0166200 con 0x7fbff00b3370 2024-09-16T15:51:55.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:55.801+0000 7fbfefe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbff015c080 0x7fbff0159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:55.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:55.801+0000 7fbfefe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbff015c080 0x7fbff0159df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.151:39454/0 (socket says 172.21.15.151:39454) 2024-09-16T15:51:55.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:55.801+0000 7fbfefe00640 1 -- 172.21.15.151:0/776153844 learned_addr learned my addr 172.21.15.151:0/776153844 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:51:55.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:55.802+0000 7fbfef400640 1 --2- 172.21.15.151:0/776153844 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbff00b3370 0x7fbff0139f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:55.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:55.804+0000 7fbfefe00640 1 -- 172.21.15.151:0/776153844 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbff00b3370 msgr2=0x7fbff0139f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:55.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:55.804+0000 7fbfeea00640 1 --2- 172.21.15.151:0/776153844 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbff00cf2f0 0x7fbff0073f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:55.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:55.808+0000 7fbfefe00640 1 --2- 172.21.15.151:0/776153844 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbff00b3370 0x7fbff0139f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:55.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:55.808+0000 7fbfefe00640 1 -- 172.21.15.151:0/776153844 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbff00cf2f0 msgr2=0x7fbff0073f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:55.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:55.808+0000 7fbfefe00640 1 --2- 172.21.15.151:0/776153844 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbff00cf2f0 0x7fbff0073f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:55.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:55.808+0000 7fbfefe00640 1 -- 172.21.15.151:0/776153844 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbff01f8350 con 0x7fbff015c080 2024-09-16T15:51:55.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:55.809+0000 7fbfeea00640 1 --2- 172.21.15.151:0/776153844 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbff00cf2f0 0x7fbff0073f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:51:55.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:55.810+0000 7fbfef400640 1 --2- 172.21.15.151:0/776153844 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbff00b3370 0x7fbff0139f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:51:55.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:55.811+0000 7fbfefe00640 1 --2- 172.21.15.151:0/776153844 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbff015c080 0x7fbff0159df0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8002230 tx=0x7fbfe800cb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:55.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:55.812+0000 7fbfe7400640 1 -- 172.21.15.151:0/776153844 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe8003b70 con 0x7fbff015c080 2024-09-16T15:51:55.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:55.813+0000 7fbff6800640 1 -- 172.21.15.151:0/776153844 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbff01f8630 con 0x7fbff015c080 2024-09-16T15:51:55.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:55.813+0000 7fbfe7400640 1 -- 172.21.15.151:0/776153844 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbfe8003d10 con 0x7fbff015c080 2024-09-16T15:51:55.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:55.813+0000 7fbfe7400640 1 -- 172.21.15.151:0/776153844 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe8005330 con 0x7fbff015c080 2024-09-16T15:51:55.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:55.813+0000 7fbff6800640 1 -- 172.21.15.151:0/776153844 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbff01f8bf0 con 0x7fbff015c080 2024-09-16T15:51:55.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:55.815+0000 7fbff6800640 1 -- 172.21.15.151:0/776153844 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbff00cebe0 con 0x7fbff015c080 2024-09-16T15:51:55.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:55.819+0000 7fbfe7400640 1 -- 172.21.15.151:0/776153844 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 40) ==== 103582+0+0 (secure 0 0 0) 0x7fbfe80056e0 con 0x7fbff015c080 2024-09-16T15:51:55.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:55.820+0000 7fbfe7400640 1 --2- 172.21.15.151:0/776153844 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbfd007a3a0 0x7fbfd007c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:55.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:55.820+0000 7fbfe7400640 1 -- 172.21.15.151:0/776153844 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5765+0+0 (secure 0 0 0) 0x7fbfe80050d0 con 0x7fbff015c080 2024-09-16T15:51:55.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:55.820+0000 7fbfef400640 1 --2- 172.21.15.151:0/776153844 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbfd007a3a0 0x7fbfd007c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:55.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:55.822+0000 7fbfef400640 1 --2- 172.21.15.151:0/776153844 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbfd007a3a0 0x7fbfd007c860 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fbfe0007fd0 tx=0x7fbfe0009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:55.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:55.828+0000 7fbfe7400640 1 -- 172.21.15.151:0/776153844 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fbfe8064190 con 0x7fbff015c080 2024-09-16T15:51:56.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:56.148+0000 7fbff6800640 1 -- 172.21.15.151:0/776153844 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7fbff01f8e30 con 0x7fbfd007a3a0 2024-09-16T15:51:56.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:56.221+0000 7fbfe7400640 1 -- 172.21.15.151:0/776153844 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+282 (secure 0 0 0) 0x7fbfe00013c0 con 0x7fbfd007a3a0 2024-09-16T15:51:56.223 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-16T15:51:56.223 INFO:teuthology.orchestra.run.smithi151.stdout: "status": { 2024-09-16T15:51:56.223 INFO:teuthology.orchestra.run.smithi151.stdout: "state": "in-progress", 2024-09-16T15:51:56.223 INFO:teuthology.orchestra.run.smithi151.stdout: "source": { 2024-09-16T15:51:56.223 INFO:teuthology.orchestra.run.smithi151.stdout: "volume": "cephfs", 2024-09-16T15:51:56.223 INFO:teuthology.orchestra.run.smithi151.stdout: "subvolume": "sv1", 2024-09-16T15:51:56.223 INFO:teuthology.orchestra.run.smithi151.stdout: "snapshot": "ss1" 2024-09-16T15:51:56.223 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:51:56.223 INFO:teuthology.orchestra.run.smithi151.stdout: "progress_report": { 2024-09-16T15:51:56.223 INFO:teuthology.orchestra.run.smithi151.stdout: "percentage cloned": "78.906%", 2024-09-16T15:51:56.223 INFO:teuthology.orchestra.run.smithi151.stdout: "amount cloned": "2.3G/3.0G", 2024-09-16T15:51:56.224 INFO:teuthology.orchestra.run.smithi151.stdout: "files cloned": "6/4" 2024-09-16T15:51:56.224 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:51:56.224 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:51:56.224 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-16T15:51:56.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:56.224+0000 7fbff6800640 1 -- 172.21.15.151:0/776153844 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbfd007a3a0 msgr2=0x7fbfd007c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:56.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:56.224+0000 7fbff6800640 1 --2- 172.21.15.151:0/776153844 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbfd007a3a0 0x7fbfd007c860 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fbfe0007fd0 tx=0x7fbfe0009040 comp rx=0 tx=0).stop 2024-09-16T15:51:56.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:56.224+0000 7fbff6800640 1 -- 172.21.15.151:0/776153844 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbff015c080 msgr2=0x7fbff0159df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:56.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:56.224+0000 7fbff6800640 1 --2- 172.21.15.151:0/776153844 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbff015c080 0x7fbff0159df0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8002230 tx=0x7fbfe800cb90 comp rx=0 tx=0).stop 2024-09-16T15:51:56.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:56.224+0000 7fbff6800640 1 -- 172.21.15.151:0/776153844 shutdown_connections 2024-09-16T15:51:56.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:56.224+0000 7fbff6800640 1 --2- 172.21.15.151:0/776153844 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbfd007a3a0 0x7fbfd007c860 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:56.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:56.224+0000 7fbff6800640 1 --2- 172.21.15.151:0/776153844 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbff015c080 0x7fbff0159df0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:56.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:56.224+0000 7fbff6800640 1 --2- 172.21.15.151:0/776153844 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbff00cf2f0 0x7fbff0073f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:56.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:56.224+0000 7fbff6800640 1 --2- 172.21.15.151:0/776153844 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbff00b3370 0x7fbff0139f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:56.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:56.224+0000 7fbff6800640 1 -- 172.21.15.151:0/776153844 >> 172.21.15.151:0/776153844 conn(0x7fbff00b1840 msgr2=0x7fbff013d590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:56.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:56.224+0000 7fbff6800640 1 -- 172.21.15.151:0/776153844 shutdown_connections 2024-09-16T15:51:56.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:56.224+0000 7fbff6800640 1 -- 172.21.15.151:0/776153844 wait complete. 2024-09-16T15:51:56.249 DEBUG:tasks.cephfs.test_volumes:percentage cloned = 78.906% 2024-09-16T15:51:57.249 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-09-16T15:51:57.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:57.413+0000 7fbab8800640 1 -- 172.21.15.151:0/3777703504 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbab00cec60 msgr2=0x7fbab00cf090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:57.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:57.413+0000 7fbab8800640 1 --2- 172.21.15.151:0/3777703504 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbab00cec60 0x7fbab00cf090 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbaac005fd0 tx=0x7fbaac02a8b0 comp rx=0 tx=0).stop 2024-09-16T15:51:57.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:57.413+0000 7fbab8800640 1 -- 172.21.15.151:0/3777703504 shutdown_connections 2024-09-16T15:51:57.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:57.413+0000 7fbab8800640 1 --2- 172.21.15.151:0/3777703504 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbab00cec60 0x7fbab00cf090 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:57.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:57.413+0000 7fbab8800640 1 --2- 172.21.15.151:0/3777703504 >> v2:172.21.15.167:3300/0 conn(0x7fbab00af190 0x7fbab00b2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:57.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:57.413+0000 7fbab8800640 1 -- 172.21.15.151:0/3777703504 >> 172.21.15.151:0/3777703504 conn(0x7fbab00b1840 msgr2=0x7fbab00b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:57.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:57.413+0000 7fbab8800640 1 -- 172.21.15.151:0/3777703504 shutdown_connections 2024-09-16T15:51:57.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:57.413+0000 7fbab8800640 1 -- 172.21.15.151:0/3777703504 wait complete. 2024-09-16T15:51:57.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:57.414+0000 7fbab8800640 1 Processor -- start 2024-09-16T15:51:57.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:57.414+0000 7fbab8800640 1 -- start start 2024-09-16T15:51:57.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:57.415+0000 7fbab8800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbab00af190 0x7fbab01f48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:57.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:57.415+0000 7fbab8800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbab00cec60 0x7fbab0158250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:57.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:57.415+0000 7fbab8800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbab0161760 0x7fbab00af660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:57.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:57.415+0000 7fbab8800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbab0162310 con 0x7fbab00af190 2024-09-16T15:51:57.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:57.415+0000 7fbab8800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbab0162190 con 0x7fbab00cec60 2024-09-16T15:51:57.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:57.415+0000 7fbab8800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fbab0162490 con 0x7fbab0161760 2024-09-16T15:51:57.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:57.416+0000 7fbab5e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbab00af190 0x7fbab01f48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:57.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:57.416+0000 7fbab5400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbab00cec60 0x7fbab0158250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:57.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:57.416+0000 7fbab5400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbab00cec60 0x7fbab0158250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.151:39466/0 (socket says 172.21.15.151:39466) 2024-09-16T15:51:57.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:57.416+0000 7fbab5400640 1 -- 172.21.15.151:0/1543507066 learned_addr learned my addr 172.21.15.151:0/1543507066 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:51:57.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:57.416+0000 7fbab6800640 1 --2- 172.21.15.151:0/1543507066 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbab0161760 0x7fbab00af660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:57.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:57.416+0000 7fbab5e00640 1 -- 172.21.15.151:0/1543507066 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbab0161760 msgr2=0x7fbab00af660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:57.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:57.416+0000 7fbab5e00640 1 --2- 172.21.15.151:0/1543507066 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbab0161760 0x7fbab00af660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:57.419 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:57.416+0000 7fbab5e00640 1 -- 172.21.15.151:0/1543507066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbab00cec60 msgr2=0x7fbab0158250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:57.419 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:57.416+0000 7fbab5e00640 1 --2- 172.21.15.151:0/1543507066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbab00cec60 0x7fbab0158250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:57.419 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:57.416+0000 7fbab5e00640 1 -- 172.21.15.151:0/1543507066 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbab00b0850 con 0x7fbab00af190 2024-09-16T15:51:57.419 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:57.417+0000 7fbab5e00640 1 --2- 172.21.15.151:0/1543507066 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbab00af190 0x7fbab01f48b0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fbaa400c3f0 tx=0x7fbaa400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:57.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:57.417+0000 7fbaaa000640 1 -- 172.21.15.151:0/1543507066 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa4011790 con 0x7fbab00af190 2024-09-16T15:51:57.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:57.417+0000 7fbaaa000640 1 -- 172.21.15.151:0/1543507066 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbaa4017070 con 0x7fbab00af190 2024-09-16T15:51:57.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:57.417+0000 7fbab8800640 1 -- 172.21.15.151:0/1543507066 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbab01f4f90 con 0x7fbab00af190 2024-09-16T15:51:57.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:57.417+0000 7fbaaa000640 1 -- 172.21.15.151:0/1543507066 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa4011ce0 con 0x7fbab00af190 2024-09-16T15:51:57.420 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:57.417+0000 7fbab8800640 1 -- 172.21.15.151:0/1543507066 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbab01f5550 con 0x7fbab00af190 2024-09-16T15:51:57.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:57.418+0000 7fbab8800640 1 -- 172.21.15.151:0/1543507066 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbab01f5790 con 0x7fbab00af190 2024-09-16T15:51:57.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:57.419+0000 7fbaaa000640 1 -- 172.21.15.151:0/1543507066 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 40) ==== 103582+0+0 (secure 0 0 0) 0x7fbaa401c460 con 0x7fbab00af190 2024-09-16T15:51:57.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:57.420+0000 7fbaaa000640 1 --2- 172.21.15.151:0/1543507066 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fba9407a1a0 0x7fba9407c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:57.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:57.420+0000 7fbaaa000640 1 -- 172.21.15.151:0/1543507066 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5765+0+0 (secure 0 0 0) 0x7fbaa4099280 con 0x7fbab00af190 2024-09-16T15:51:57.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:57.420+0000 7fbab5400640 1 --2- 172.21.15.151:0/1543507066 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fba9407a1a0 0x7fba9407c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:57.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:57.421+0000 7fbab5400640 1 --2- 172.21.15.151:0/1543507066 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fba9407a1a0 0x7fba9407c660 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fbab00b0d00 tx=0x7fbaa0005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:57.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:57.424+0000 7fbaaa000640 1 -- 172.21.15.151:0/1543507066 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fbab01f5790 con 0x7fbab00af190 2024-09-16T15:51:57.648 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:57.647+0000 7fbab8800640 1 -- 172.21.15.151:0/1543507066 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7fbab0162920 con 0x7fba9407a1a0 2024-09-16T15:51:58.068 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.067+0000 7fbaaa000640 1 -- 172.21.15.151:0/1543507066 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+46 (secure 0 0 0) 0x7fbab0162920 con 0x7fba9407a1a0 2024-09-16T15:51:58.068 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-16T15:51:58.069 INFO:teuthology.orchestra.run.smithi151.stdout: "status": { 2024-09-16T15:51:58.069 INFO:teuthology.orchestra.run.smithi151.stdout: "state": "complete" 2024-09-16T15:51:58.069 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:51:58.069 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-16T15:51:58.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.070+0000 7fbab8800640 1 -- 172.21.15.151:0/1543507066 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fba9407a1a0 msgr2=0x7fba9407c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:58.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.070+0000 7fbab8800640 1 --2- 172.21.15.151:0/1543507066 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fba9407a1a0 0x7fba9407c660 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fbab00b0d00 tx=0x7fbaa0005f90 comp rx=0 tx=0).stop 2024-09-16T15:51:58.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.070+0000 7fbab8800640 1 -- 172.21.15.151:0/1543507066 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbab00af190 msgr2=0x7fbab01f48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:58.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.070+0000 7fbab8800640 1 --2- 172.21.15.151:0/1543507066 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbab00af190 0x7fbab01f48b0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fbaa400c3f0 tx=0x7fbaa400c8b0 comp rx=0 tx=0).stop 2024-09-16T15:51:58.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.070+0000 7fbab8800640 1 -- 172.21.15.151:0/1543507066 shutdown_connections 2024-09-16T15:51:58.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.070+0000 7fbab8800640 1 --2- 172.21.15.151:0/1543507066 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fba9407a1a0 0x7fba9407c660 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:58.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.070+0000 7fbab8800640 1 --2- 172.21.15.151:0/1543507066 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbab00cec60 0x7fbab0158250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:58.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.070+0000 7fbab8800640 1 --2- 172.21.15.151:0/1543507066 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbab0161760 0x7fbab00af660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:58.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.070+0000 7fbab8800640 1 --2- 172.21.15.151:0/1543507066 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbab00af190 0x7fbab01f48b0 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:58.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.070+0000 7fbab8800640 1 -- 172.21.15.151:0/1543507066 >> 172.21.15.151:0/1543507066 conn(0x7fbab00b1840 msgr2=0x7fbab0073a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:58.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.070+0000 7fbab8800640 1 -- 172.21.15.151:0/1543507066 shutdown_connections 2024-09-16T15:51:58.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.070+0000 7fbab8800640 1 -- 172.21.15.151:0/1543507066 wait complete. 2024-09-16T15:51:58.088 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-09-16T15:51:58.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.246+0000 7f48cba00640 1 -- 172.21.15.151:0/214385177 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f48c40d2960 msgr2=0x7f48c4160340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:58.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.246+0000 7f48cba00640 1 --2- 172.21.15.151:0/214385177 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f48c40d2960 0x7f48c4160340 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f48c40b7ef0 tx=0x7f48b002a5f0 comp rx=0 tx=0).stop 2024-09-16T15:51:58.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.247+0000 7f48cba00640 1 -- 172.21.15.151:0/214385177 shutdown_connections 2024-09-16T15:51:58.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.247+0000 7f48cba00640 1 --2- 172.21.15.151:0/214385177 >> v2:172.21.15.167:3300/0 conn(0x7f48c40b0b30 0x7f48c4160880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:58.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.247+0000 7f48cba00640 1 --2- 172.21.15.151:0/214385177 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f48c40d2960 0x7f48c4160340 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:58.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.247+0000 7f48cba00640 1 --2- 172.21.15.151:0/214385177 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48c40d00b0 0x7f48c415fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:58.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.247+0000 7f48cba00640 1 -- 172.21.15.151:0/214385177 >> 172.21.15.151:0/214385177 conn(0x7f48c40d1e40 msgr2=0x7f48c40d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:58.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.247+0000 7f48cba00640 1 -- 172.21.15.151:0/214385177 shutdown_connections 2024-09-16T15:51:58.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.247+0000 7f48cba00640 1 -- 172.21.15.151:0/214385177 wait complete. 2024-09-16T15:51:58.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.248+0000 7f48cba00640 1 Processor -- start 2024-09-16T15:51:58.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.249+0000 7f48cba00640 1 -- start start 2024-09-16T15:51:58.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.249+0000 7f48cba00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48c40b0b30 0x7f48c4150c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:58.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.249+0000 7f48cba00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f48c40d00b0 0x7f48c4152b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:58.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.250+0000 7f48cba00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48c40d2960 0x7f48c40cb930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:58.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.250+0000 7f48cba00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f48c40b1960 con 0x7f48c40d00b0 2024-09-16T15:51:58.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.250+0000 7f48cba00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f48c40b17e0 con 0x7f48c40d2960 2024-09-16T15:51:58.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.250+0000 7f48cba00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f48c40b1ae0 con 0x7f48c40b0b30 2024-09-16T15:51:58.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.250+0000 7f48c3e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f48c40d00b0 0x7f48c4152b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:58.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.250+0000 7f48c3e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f48c40d00b0 0x7f48c4152b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:60698/0 (socket says 172.21.15.151:60698) 2024-09-16T15:51:58.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.250+0000 7f48c3e00640 1 -- 172.21.15.151:0/2019657544 learned_addr learned my addr 172.21.15.151:0/2019657544 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:51:58.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.250+0000 7f48c9a00640 1 --2- 172.21.15.151:0/2019657544 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48c40d2960 0x7f48c40cb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:58.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.250+0000 7f48c9000640 1 --2- 172.21.15.151:0/2019657544 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48c40b0b30 0x7f48c4150c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:58.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.250+0000 7f48c3e00640 1 -- 172.21.15.151:0/2019657544 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48c40b0b30 msgr2=0x7f48c4150c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:58.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.250+0000 7f48c3e00640 1 --2- 172.21.15.151:0/2019657544 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48c40b0b30 0x7f48c4150c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:58.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.250+0000 7f48c3e00640 1 -- 172.21.15.151:0/2019657544 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48c40d2960 msgr2=0x7f48c40cb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:58.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.250+0000 7f48c3e00640 1 --2- 172.21.15.151:0/2019657544 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48c40d2960 0x7f48c40cb930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:58.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.250+0000 7f48c3e00640 1 -- 172.21.15.151:0/2019657544 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48c406de60 con 0x7f48c40d00b0 2024-09-16T15:51:58.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.251+0000 7f48c3e00640 1 --2- 172.21.15.151:0/2019657544 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f48c40d00b0 0x7f48c4152b10 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f48b0034040 tx=0x7f48b00039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:58.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.251+0000 7f48c0c00640 1 -- 172.21.15.151:0/2019657544 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b003e030 con 0x7f48c40d00b0 2024-09-16T15:51:58.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.251+0000 7f48c0c00640 1 -- 172.21.15.151:0/2019657544 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f48b00047a0 con 0x7f48c40d00b0 2024-09-16T15:51:58.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.251+0000 7f48c0c00640 1 -- 172.21.15.151:0/2019657544 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b0035040 con 0x7f48c40d00b0 2024-09-16T15:51:58.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.251+0000 7f48cba00640 1 -- 172.21.15.151:0/2019657544 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48c406e140 con 0x7f48c40d00b0 2024-09-16T15:51:58.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.251+0000 7f48cba00640 1 -- 172.21.15.151:0/2019657544 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48c40cbe70 con 0x7f48c40d00b0 2024-09-16T15:51:58.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.252+0000 7f48cba00640 1 -- 172.21.15.151:0/2019657544 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48c40d19e0 con 0x7f48c40d00b0 2024-09-16T15:51:58.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.253+0000 7f48c0c00640 1 -- 172.21.15.151:0/2019657544 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 40) ==== 103582+0+0 (secure 0 0 0) 0x7f48b0041030 con 0x7f48c40d00b0 2024-09-16T15:51:58.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.254+0000 7f48c0c00640 1 --2- 172.21.15.151:0/2019657544 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f48a407a1a0 0x7f48a407c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:58.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.254+0000 7f48c0c00640 1 -- 172.21.15.151:0/2019657544 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5765+0+0 (secure 0 0 0) 0x7f48b00b9c80 con 0x7f48c40d00b0 2024-09-16T15:51:58.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.255+0000 7f48c9000640 1 --2- 172.21.15.151:0/2019657544 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f48a407a1a0 0x7f48a407c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:58.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.256+0000 7f48c9000640 1 --2- 172.21.15.151:0/2019657544 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f48a407a1a0 0x7f48a407c660 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f48b8005d80 tx=0x7f48b800b390 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:58.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.259+0000 7f48c0c00640 1 -- 172.21.15.151:0/2019657544 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f48c40d19e0 con 0x7f48c40d00b0 2024-09-16T15:51:58.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.485+0000 7f48cba00640 1 -- 172.21.15.151:0/2019657544 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f48c40b2070 con 0x7f48a407a1a0 2024-09-16T15:51:58.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.492+0000 7f48c0c00640 1 -- 172.21.15.151:0/2019657544 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+46 (secure 0 0 0) 0x7f48c40b2070 con 0x7f48a407a1a0 2024-09-16T15:51:58.493 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-16T15:51:58.493 INFO:teuthology.orchestra.run.smithi151.stdout: "status": { 2024-09-16T15:51:58.493 INFO:teuthology.orchestra.run.smithi151.stdout: "state": "complete" 2024-09-16T15:51:58.493 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:51:58.493 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-16T15:51:58.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.494+0000 7f48cba00640 1 -- 172.21.15.151:0/2019657544 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f48a407a1a0 msgr2=0x7f48a407c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:58.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.494+0000 7f48cba00640 1 --2- 172.21.15.151:0/2019657544 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f48a407a1a0 0x7f48a407c660 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f48b8005d80 tx=0x7f48b800b390 comp rx=0 tx=0).stop 2024-09-16T15:51:58.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.494+0000 7f48cba00640 1 -- 172.21.15.151:0/2019657544 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f48c40d00b0 msgr2=0x7f48c4152b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:58.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.494+0000 7f48cba00640 1 --2- 172.21.15.151:0/2019657544 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f48c40d00b0 0x7f48c4152b10 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f48b0034040 tx=0x7f48b00039e0 comp rx=0 tx=0).stop 2024-09-16T15:51:58.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.494+0000 7f48cba00640 1 -- 172.21.15.151:0/2019657544 shutdown_connections 2024-09-16T15:51:58.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.494+0000 7f48cba00640 1 --2- 172.21.15.151:0/2019657544 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f48a407a1a0 0x7f48a407c660 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:58.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.494+0000 7f48cba00640 1 --2- 172.21.15.151:0/2019657544 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48c40d2960 0x7f48c40cb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:58.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.494+0000 7f48cba00640 1 --2- 172.21.15.151:0/2019657544 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f48c40d00b0 0x7f48c4152b10 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:58.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.494+0000 7f48cba00640 1 --2- 172.21.15.151:0/2019657544 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48c40b0b30 0x7f48c4150c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:58.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.494+0000 7f48cba00640 1 -- 172.21.15.151:0/2019657544 >> 172.21.15.151:0/2019657544 conn(0x7f48c40d1e40 msgr2=0x7f48c40b55f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:58.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.494+0000 7f48cba00640 1 -- 172.21.15.151:0/2019657544 shutdown_connections 2024-09-16T15:51:58.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.494+0000 7f48cba00640 1 -- 172.21.15.151:0/2019657544 wait complete. 2024-09-16T15:51:58.511 DEBUG:tasks.cephfs.test_volumes:current clone state = complete 2024-09-16T15:51:58.511 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-16T15:51:58.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.671+0000 7f1b19000640 1 -- 172.21.15.151:0/3832886866 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1b1415f5a0 msgr2=0x7f1b140b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:58.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.671+0000 7f1b19000640 1 --2- 172.21.15.151:0/3832886866 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1b1415f5a0 0x7f1b140b2440 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1b04009be0 tx=0x7f1b0402a820 comp rx=0 tx=0).stop 2024-09-16T15:51:58.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.671+0000 7f1b19000640 1 -- 172.21.15.151:0/3832886866 shutdown_connections 2024-09-16T15:51:58.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.671+0000 7f1b19000640 1 --2- 172.21.15.151:0/3832886866 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1b1415f5a0 0x7f1b140b2440 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:58.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.671+0000 7f1b19000640 1 -- 172.21.15.151:0/3832886866 >> 172.21.15.151:0/3832886866 conn(0x7f1b140b1950 msgr2=0x7f1b140b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:58.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.671+0000 7f1b19000640 1 -- 172.21.15.151:0/3832886866 shutdown_connections 2024-09-16T15:51:58.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.671+0000 7f1b19000640 1 -- 172.21.15.151:0/3832886866 wait complete. 2024-09-16T15:51:58.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.673+0000 7f1b19000640 1 Processor -- start 2024-09-16T15:51:58.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.673+0000 7f1b19000640 1 -- start start 2024-09-16T15:51:58.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.674+0000 7f1b19000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b14073fb0 0x7f1b140c90f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:58.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.674+0000 7f1b19000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1b140ccf00 0x7f1b14142d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:58.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.674+0000 7f1b19000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1b1415f5a0 0x7f1b140ca8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:58.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.674+0000 7f1b19000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1b1406ef40 con 0x7f1b1415f5a0 2024-09-16T15:51:58.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.674+0000 7f1b19000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1b1406fa80 con 0x7f1b14073fb0 2024-09-16T15:51:58.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.674+0000 7f1b19000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f1b14074460 con 0x7f1b140ccf00 2024-09-16T15:51:58.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.674+0000 7f1b12800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1b1415f5a0 0x7f1b140ca8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:58.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.674+0000 7f1b12800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1b1415f5a0 0x7f1b140ca8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:60710/0 (socket says 172.21.15.151:60710) 2024-09-16T15:51:58.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.674+0000 7f1b11e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b14073fb0 0x7f1b140c90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:58.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.674+0000 7f1b12800640 1 -- 172.21.15.151:0/2215781998 learned_addr learned my addr 172.21.15.151:0/2215781998 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:51:58.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.674+0000 7f1b11400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1b140ccf00 0x7f1b14142d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:58.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.674+0000 7f1b12800640 1 -- 172.21.15.151:0/2215781998 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1b140ccf00 msgr2=0x7f1b14142d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:58.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.674+0000 7f1b12800640 1 --2- 172.21.15.151:0/2215781998 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1b140ccf00 0x7f1b14142d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:58.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.674+0000 7f1b12800640 1 -- 172.21.15.151:0/2215781998 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b14073fb0 msgr2=0x7f1b140c90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:58.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.674+0000 7f1b12800640 1 --2- 172.21.15.151:0/2215781998 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b14073fb0 0x7f1b140c90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:58.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.674+0000 7f1b12800640 1 -- 172.21.15.151:0/2215781998 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b14143680 con 0x7f1b1415f5a0 2024-09-16T15:51:58.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.675+0000 7f1b12800640 1 --2- 172.21.15.151:0/2215781998 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1b1415f5a0 0x7f1b140ca8c0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c00c3a0 tx=0x7f1b0c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:58.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.675+0000 7f1b0a000640 1 -- 172.21.15.151:0/2215781998 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c017070 con 0x7f1b1415f5a0 2024-09-16T15:51:58.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.675+0000 7f1b0a000640 1 -- 172.21.15.151:0/2215781998 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1b0c0053e0 con 0x7f1b1415f5a0 2024-09-16T15:51:58.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.675+0000 7f1b0a000640 1 -- 172.21.15.151:0/2215781998 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c003850 con 0x7f1b1415f5a0 2024-09-16T15:51:58.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.675+0000 7f1b19000640 1 -- 172.21.15.151:0/2215781998 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b14143c10 con 0x7f1b1415f5a0 2024-09-16T15:51:58.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.675+0000 7f1b19000640 1 -- 172.21.15.151:0/2215781998 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b141441d0 con 0x7f1b1415f5a0 2024-09-16T15:51:58.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.676+0000 7f1b19000640 1 -- 172.21.15.151:0/2215781998 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b140b2440 con 0x7f1b1415f5a0 2024-09-16T15:51:58.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.677+0000 7f1b0a000640 1 -- 172.21.15.151:0/2215781998 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 40) ==== 103582+0+0 (secure 0 0 0) 0x7f1b0c01a070 con 0x7f1b1415f5a0 2024-09-16T15:51:58.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.678+0000 7f1b0a000640 1 --2- 172.21.15.151:0/2215781998 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1af407a1a0 0x7f1af407c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:58.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.678+0000 7f1b0a000640 1 -- 172.21.15.151:0/2215781998 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5765+0+0 (secure 0 0 0) 0x7f1b0c098ae0 con 0x7f1b1415f5a0 2024-09-16T15:51:58.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.679+0000 7f1b11e00640 1 --2- 172.21.15.151:0/2215781998 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1af407a1a0 0x7f1af407c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:58.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.679+0000 7f1b11e00640 1 --2- 172.21.15.151:0/2215781998 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1af407a1a0 0x7f1af407c660 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f1b04033000 tx=0x7f1b04004ab0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:58.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.682+0000 7f1b0a000640 1 -- 172.21.15.151:0/2215781998 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1b0c062560 con 0x7f1b1415f5a0 2024-09-16T15:51:58.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.891+0000 7f1b19000640 1 -- 172.21.15.151:0/2215781998 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f1b1409d6f0 con 0x7f1af407a1a0 2024-09-16T15:51:58.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.893+0000 7f1b0a000640 1 -- 172.21.15.151:0/2215781998 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f1b1409d6f0 con 0x7f1af407a1a0 2024-09-16T15:51:58.894 INFO:teuthology.orchestra.run.smithi151.stdout:[ 2024-09-16T15:51:58.894 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:51:58.894 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "cephfs" 2024-09-16T15:51:58.894 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:51:58.894 INFO:teuthology.orchestra.run.smithi151.stdout:] 2024-09-16T15:51:58.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.895+0000 7f1b19000640 1 -- 172.21.15.151:0/2215781998 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1af407a1a0 msgr2=0x7f1af407c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:58.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.895+0000 7f1b19000640 1 --2- 172.21.15.151:0/2215781998 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1af407a1a0 0x7f1af407c660 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f1b04033000 tx=0x7f1b04004ab0 comp rx=0 tx=0).stop 2024-09-16T15:51:58.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.895+0000 7f1b19000640 1 -- 172.21.15.151:0/2215781998 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1b1415f5a0 msgr2=0x7f1b140ca8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:58.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.895+0000 7f1b19000640 1 --2- 172.21.15.151:0/2215781998 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1b1415f5a0 0x7f1b140ca8c0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c00c3a0 tx=0x7f1b0c00c860 comp rx=0 tx=0).stop 2024-09-16T15:51:58.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.895+0000 7f1b19000640 1 -- 172.21.15.151:0/2215781998 shutdown_connections 2024-09-16T15:51:58.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.895+0000 7f1b19000640 1 --2- 172.21.15.151:0/2215781998 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1af407a1a0 0x7f1af407c660 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:58.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.895+0000 7f1b19000640 1 --2- 172.21.15.151:0/2215781998 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1b1415f5a0 0x7f1b140ca8c0 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:58.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.895+0000 7f1b19000640 1 --2- 172.21.15.151:0/2215781998 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1b140ccf00 0x7f1b14142d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:58.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.895+0000 7f1b19000640 1 --2- 172.21.15.151:0/2215781998 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b14073fb0 0x7f1b140c90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:58.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.895+0000 7f1b19000640 1 -- 172.21.15.151:0/2215781998 >> 172.21.15.151:0/2215781998 conn(0x7f1b140b1950 msgr2=0x7f1b14154170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:58.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.895+0000 7f1b19000640 1 -- 172.21.15.151:0/2215781998 shutdown_connections 2024-09-16T15:51:58.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:58.895+0000 7f1b19000640 1 -- 172.21.15.151:0/2215781998 wait complete. 2024-09-16T15:51:58.913 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-09-16T15:51:59.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.074+0000 7f0bdbe00640 1 -- 172.21.15.151:0/1921604078 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0bdc161460 msgr2=0x7f0bdc0d22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:59.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.074+0000 7f0bdbe00640 1 --2- 172.21.15.151:0/1921604078 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0bdc161460 0x7f0bdc0d22f0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc0ce5f0 tx=0x7f0bcc02a4a0 comp rx=0 tx=0).stop 2024-09-16T15:51:59.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.075+0000 7f0bdbe00640 1 -- 172.21.15.151:0/1921604078 shutdown_connections 2024-09-16T15:51:59.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.075+0000 7f0bdbe00640 1 --2- 172.21.15.151:0/1921604078 >> v2:172.21.15.167:3300/0 conn(0x7f0bdc0b43c0 0x7f0bdc0d2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:59.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.075+0000 7f0bdbe00640 1 --2- 172.21.15.151:0/1921604078 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0bdc161460 0x7f0bdc0d22f0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:59.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.075+0000 7f0bdbe00640 1 -- 172.21.15.151:0/1921604078 >> 172.21.15.151:0/1921604078 conn(0x7f0bdc09d0b0 msgr2=0x7f0bdc0d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:59.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.075+0000 7f0bdbe00640 1 -- 172.21.15.151:0/1921604078 shutdown_connections 2024-09-16T15:51:59.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.075+0000 7f0bdbe00640 1 -- 172.21.15.151:0/1921604078 wait complete. 2024-09-16T15:51:59.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.076+0000 7f0bdbe00640 1 Processor -- start 2024-09-16T15:51:59.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.076+0000 7f0bdbe00640 1 -- start start 2024-09-16T15:51:59.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.077+0000 7f0bdbe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0bdc0b43c0 0x7f0bdc1e6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:59.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.077+0000 7f0bdbe00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0bdc0b5480 0x7f0bdc154880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:59.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.077+0000 7f0bdbe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0bdc161460 0x7f0bdc1e9160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:59.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.077+0000 7f0bdbe00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0bdc0cef30 con 0x7f0bdc161460 2024-09-16T15:51:59.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.077+0000 7f0bdbe00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0bdc0b5150 con 0x7f0bdc0b43c0 2024-09-16T15:51:59.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.077+0000 7f0bdbe00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f0bdc0cf0b0 con 0x7f0bdc0b5480 2024-09-16T15:51:59.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.078+0000 7f0bdb400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0bdc161460 0x7f0bdc1e9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:59.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.078+0000 7f0bdb400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0bdc161460 0x7f0bdc1e9160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:60730/0 (socket says 172.21.15.151:60730) 2024-09-16T15:51:59.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.078+0000 7f0bdb400640 1 -- 172.21.15.151:0/427870024 learned_addr learned my addr 172.21.15.151:0/427870024 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:51:59.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.078+0000 7f0bdaa00640 1 --2- 172.21.15.151:0/427870024 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0bdc0b43c0 0x7f0bdc1e6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:59.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.078+0000 7f0bda000640 1 --2- 172.21.15.151:0/427870024 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0bdc0b5480 0x7f0bdc154880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:59.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.078+0000 7f0bdb400640 1 -- 172.21.15.151:0/427870024 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0bdc0b5480 msgr2=0x7f0bdc154880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:59.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.078+0000 7f0bdb400640 1 --2- 172.21.15.151:0/427870024 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0bdc0b5480 0x7f0bdc154880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:59.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.078+0000 7f0bdb400640 1 -- 172.21.15.151:0/427870024 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0bdc0b43c0 msgr2=0x7f0bdc1e6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:59.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.078+0000 7f0bdb400640 1 --2- 172.21.15.151:0/427870024 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0bdc0b43c0 0x7f0bdc1e6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:59.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.078+0000 7f0bdb400640 1 -- 172.21.15.151:0/427870024 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bdc155300 con 0x7f0bdc161460 2024-09-16T15:51:59.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.078+0000 7f0bdb400640 1 --2- 172.21.15.151:0/427870024 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0bdc161460 0x7f0bdc1e9160 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f0bd400ccb0 tx=0x7f0bd400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:59.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.078+0000 7f0bd2a00640 1 -- 172.21.15.151:0/427870024 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd400f450 con 0x7f0bdc161460 2024-09-16T15:51:59.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.079+0000 7f0bd2a00640 1 -- 172.21.15.151:0/427870024 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0bd400f700 con 0x7f0bdc161460 2024-09-16T15:51:59.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.079+0000 7f0bdbe00640 1 -- 172.21.15.151:0/427870024 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bdc1555e0 con 0x7f0bdc161460 2024-09-16T15:51:59.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.079+0000 7f0bd2a00640 1 -- 172.21.15.151:0/427870024 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd400fb90 con 0x7f0bdc161460 2024-09-16T15:51:59.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.079+0000 7f0bdbe00640 1 -- 172.21.15.151:0/427870024 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bdc155e80 con 0x7f0bdc161460 2024-09-16T15:51:59.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.080+0000 7f0bdbe00640 1 -- 172.21.15.151:0/427870024 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bdc0cf590 con 0x7f0bdc161460 2024-09-16T15:51:59.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.081+0000 7f0bd2a00640 1 -- 172.21.15.151:0/427870024 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 40) ==== 103582+0+0 (secure 0 0 0) 0x7f0bd40213f0 con 0x7f0bdc161460 2024-09-16T15:51:59.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.082+0000 7f0bd2a00640 1 --2- 172.21.15.151:0/427870024 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0bbc07a1a0 0x7f0bbc07c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:59.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.082+0000 7f0bd2a00640 1 -- 172.21.15.151:0/427870024 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5765+0+0 (secure 0 0 0) 0x7f0bd409a230 con 0x7f0bdc161460 2024-09-16T15:51:59.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.082+0000 7f0bdaa00640 1 --2- 172.21.15.151:0/427870024 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0bbc07a1a0 0x7f0bbc07c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:59.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.083+0000 7f0bdaa00640 1 --2- 172.21.15.151:0/427870024 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0bbc07a1a0 0x7f0bbc07c660 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc034000 tx=0x7f0bcc004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:59.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.086+0000 7f0bd2a00640 1 -- 172.21.15.151:0/427870024 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0bd4014030 con 0x7f0bdc161460 2024-09-16T15:51:59.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.295+0000 7f0bdbe00640 1 -- 172.21.15.151:0/427870024 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0bdc156050 con 0x7f0bbc07a1a0 2024-09-16T15:51:59.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.300+0000 7f0bd2a00640 1 -- 172.21.15.151:0/427870024 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+79 (secure 0 0 0) 0x7f0bdc156050 con 0x7f0bbc07a1a0 2024-09-16T15:51:59.302 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:51:59.302 INFO:teuthology.orchestra.run.smithi151.stdout:[ 2024-09-16T15:51:59.302 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:51:59.302 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sv1" 2024-09-16T15:51:59.302 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:51:59.302 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:51:59.302 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ss1clone1" 2024-09-16T15:51:59.302 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:51:59.302 INFO:teuthology.orchestra.run.smithi151.stdout:] 2024-09-16T15:51:59.304 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.302+0000 7f0bdbe00640 1 -- 172.21.15.151:0/427870024 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0bbc07a1a0 msgr2=0x7f0bbc07c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:59.304 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.302+0000 7f0bdbe00640 1 --2- 172.21.15.151:0/427870024 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0bbc07a1a0 0x7f0bbc07c660 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc034000 tx=0x7f0bcc004730 comp rx=0 tx=0).stop 2024-09-16T15:51:59.304 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.303+0000 7f0bdbe00640 1 -- 172.21.15.151:0/427870024 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0bdc161460 msgr2=0x7f0bdc1e9160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:59.304 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.303+0000 7f0bdbe00640 1 --2- 172.21.15.151:0/427870024 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0bdc161460 0x7f0bdc1e9160 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f0bd400ccb0 tx=0x7f0bd400a620 comp rx=0 tx=0).stop 2024-09-16T15:51:59.304 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.303+0000 7f0bdbe00640 1 -- 172.21.15.151:0/427870024 shutdown_connections 2024-09-16T15:51:59.304 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.303+0000 7f0bdbe00640 1 --2- 172.21.15.151:0/427870024 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0bbc07a1a0 0x7f0bbc07c660 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:59.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.303+0000 7f0bdbe00640 1 --2- 172.21.15.151:0/427870024 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0bdc161460 0x7f0bdc1e9160 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:59.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.303+0000 7f0bdbe00640 1 --2- 172.21.15.151:0/427870024 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0bdc0b5480 0x7f0bdc154880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:59.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.303+0000 7f0bdbe00640 1 --2- 172.21.15.151:0/427870024 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0bdc0b43c0 0x7f0bdc1e6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:59.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.303+0000 7f0bdbe00640 1 -- 172.21.15.151:0/427870024 >> 172.21.15.151:0/427870024 conn(0x7f0bdc09d0b0 msgr2=0x7f0bdc0cc9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:59.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.303+0000 7f0bdbe00640 1 -- 172.21.15.151:0/427870024 shutdown_connections 2024-09-16T15:51:59.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.303+0000 7f0bdbe00640 1 -- 172.21.15.151:0/427870024 wait complete. 2024-09-16T15:51:59.320 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-09-16T15:51:59.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.484+0000 7f1193800640 1 -- 172.21.15.151:0/1030341011 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1174004800 msgr2=0x7f1174004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:59.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.484+0000 7f1193800640 1 --2- 172.21.15.151:0/1030341011 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1174004800 0x7f1174004c80 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f1178009b50 tx=0x7f1174005830 comp rx=0 tx=0).stop 2024-09-16T15:51:59.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.484+0000 7f1193800640 1 -- 172.21.15.151:0/1030341011 shutdown_connections 2024-09-16T15:51:59.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.484+0000 7f1193800640 1 --2- 172.21.15.151:0/1030341011 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1174004800 0x7f1174004c80 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:59.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.484+0000 7f1193800640 1 -- 172.21.15.151:0/1030341011 >> 172.21.15.151:0/1030341011 conn(0x7f118c0b18e0 msgr2=0x7f118c0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:59.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.484+0000 7f1193800640 1 -- 172.21.15.151:0/1030341011 shutdown_connections 2024-09-16T15:51:59.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.484+0000 7f1193800640 1 -- 172.21.15.151:0/1030341011 wait complete. 2024-09-16T15:51:59.487 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.486+0000 7f1193800640 1 Processor -- start 2024-09-16T15:51:59.487 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.486+0000 7f1193800640 1 -- start start 2024-09-16T15:51:59.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.487+0000 7f1193800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1174004800 0x7f118c0cf370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:59.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.487+0000 7f1193800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f118c1f54a0 0x7f118c154430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:59.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.487+0000 7f1193800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f118c09bdf0 0x7f118c09c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:59.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.487+0000 7f1193800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f118c074320 con 0x7f118c09bdf0 2024-09-16T15:51:59.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.487+0000 7f1193800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f118c0741a0 con 0x7f1174004800 2024-09-16T15:51:59.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.487+0000 7f1193800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f118c0744a0 con 0x7f118c1f54a0 2024-09-16T15:51:59.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.487+0000 7f1191800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f118c09bdf0 0x7f118c09c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:59.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.487+0000 7f1191800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f118c09bdf0 0x7f118c09c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:60740/0 (socket says 172.21.15.151:60740) 2024-09-16T15:51:59.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.487+0000 7f1191800640 1 -- 172.21.15.151:0/1786289687 learned_addr learned my addr 172.21.15.151:0/1786289687 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:51:59.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.487+0000 7f118be00640 1 --2- 172.21.15.151:0/1786289687 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f118c1f54a0 0x7f118c154430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:59.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.487+0000 7f1190e00640 1 --2- 172.21.15.151:0/1786289687 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1174004800 0x7f118c0cf370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:59.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.487+0000 7f1191800640 1 -- 172.21.15.151:0/1786289687 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f118c1f54a0 msgr2=0x7f118c154430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:59.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.487+0000 7f1191800640 1 --2- 172.21.15.151:0/1786289687 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f118c1f54a0 0x7f118c154430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:59.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.487+0000 7f1191800640 1 -- 172.21.15.151:0/1786289687 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1174004800 msgr2=0x7f118c0cf370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:59.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.487+0000 7f1191800640 1 --2- 172.21.15.151:0/1786289687 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1174004800 0x7f118c0cf370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:59.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.487+0000 7f1191800640 1 -- 172.21.15.151:0/1786289687 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f118c0cdad0 con 0x7f118c09bdf0 2024-09-16T15:51:59.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.488+0000 7f1191800640 1 --2- 172.21.15.151:0/1786289687 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f118c09bdf0 0x7f118c09c270 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f1184002230 tx=0x7f118400eae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:59.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.488+0000 7f1188c00640 1 -- 172.21.15.151:0/1786289687 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1184015070 con 0x7f118c09bdf0 2024-09-16T15:51:59.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.488+0000 7f1188c00640 1 -- 172.21.15.151:0/1786289687 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f118400ca60 con 0x7f118c09bdf0 2024-09-16T15:51:59.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.488+0000 7f1188c00640 1 -- 172.21.15.151:0/1786289687 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118400ce70 con 0x7f118c09bdf0 2024-09-16T15:51:59.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.488+0000 7f1193800640 1 -- 172.21.15.151:0/1786289687 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f118c0cddb0 con 0x7f118c09bdf0 2024-09-16T15:51:59.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.488+0000 7f1193800640 1 -- 172.21.15.151:0/1786289687 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f118c0ce370 con 0x7f118c09bdf0 2024-09-16T15:51:59.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.489+0000 7f1193800640 1 -- 172.21.15.151:0/1786289687 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f118c06e7d0 con 0x7f118c09bdf0 2024-09-16T15:51:59.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.490+0000 7f1188c00640 1 -- 172.21.15.151:0/1786289687 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 40) ==== 103582+0+0 (secure 0 0 0) 0x7f1184018070 con 0x7f118c09bdf0 2024-09-16T15:51:59.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.491+0000 7f1188c00640 1 --2- 172.21.15.151:0/1786289687 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f116c07a1a0 0x7f116c07c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:59.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.491+0000 7f1188c00640 1 -- 172.21.15.151:0/1786289687 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5765+0+0 (secure 0 0 0) 0x7f1184099e80 con 0x7f118c09bdf0 2024-09-16T15:51:59.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.491+0000 7f1190e00640 1 --2- 172.21.15.151:0/1786289687 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f116c07a1a0 0x7f116c07c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:59.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.492+0000 7f1190e00640 1 --2- 172.21.15.151:0/1786289687 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f116c07a1a0 0x7f116c07c660 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f1180000c00 tx=0x7f1180006ce0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:59.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.495+0000 7f1188c00640 1 -- 172.21.15.151:0/1786289687 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1184063900 con 0x7f118c09bdf0 2024-09-16T15:51:59.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.713+0000 7f1193800640 1 -- 172.21.15.151:0/1786289687 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f118c070960 con 0x7f116c07a1a0 2024-09-16T15:51:59.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.729+0000 7f1188c00640 1 -- 172.21.15.151:0/1786289687 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f118c070960 con 0x7f116c07a1a0 2024-09-16T15:51:59.730 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:51:59.731 INFO:teuthology.orchestra.run.smithi151.stdout:[ 2024-09-16T15:51:59.731 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:51:59.731 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ss1" 2024-09-16T15:51:59.731 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:51:59.731 INFO:teuthology.orchestra.run.smithi151.stdout:] 2024-09-16T15:51:59.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.731+0000 7f1193800640 1 -- 172.21.15.151:0/1786289687 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f116c07a1a0 msgr2=0x7f116c07c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:59.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.731+0000 7f1193800640 1 --2- 172.21.15.151:0/1786289687 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f116c07a1a0 0x7f116c07c660 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f1180000c00 tx=0x7f1180006ce0 comp rx=0 tx=0).stop 2024-09-16T15:51:59.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.732+0000 7f1193800640 1 -- 172.21.15.151:0/1786289687 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f118c09bdf0 msgr2=0x7f118c09c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:59.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.732+0000 7f1193800640 1 --2- 172.21.15.151:0/1786289687 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f118c09bdf0 0x7f118c09c270 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f1184002230 tx=0x7f118400eae0 comp rx=0 tx=0).stop 2024-09-16T15:51:59.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.732+0000 7f1193800640 1 -- 172.21.15.151:0/1786289687 shutdown_connections 2024-09-16T15:51:59.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.732+0000 7f1193800640 1 --2- 172.21.15.151:0/1786289687 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f116c07a1a0 0x7f116c07c660 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:59.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.732+0000 7f1193800640 1 --2- 172.21.15.151:0/1786289687 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f118c09bdf0 0x7f118c09c270 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:59.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.732+0000 7f1193800640 1 --2- 172.21.15.151:0/1786289687 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f118c1f54a0 0x7f118c154430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:59.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.732+0000 7f1193800640 1 --2- 172.21.15.151:0/1786289687 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1174004800 0x7f118c0cf370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:59.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.732+0000 7f1193800640 1 -- 172.21.15.151:0/1786289687 >> 172.21.15.151:0/1786289687 conn(0x7f118c0b18e0 msgr2=0x7f118c0b0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:59.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.732+0000 7f1193800640 1 -- 172.21.15.151:0/1786289687 shutdown_connections 2024-09-16T15:51:59.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.732+0000 7f1193800640 1 -- 172.21.15.151:0/1786289687 wait complete. 2024-09-16T15:51:59.749 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-09-16T15:51:59.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.913+0000 7f5196e00640 1 -- 172.21.15.151:0/2112776823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5178004d40 msgr2=0x7f51780051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:59.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.913+0000 7f5196e00640 1 --2- 172.21.15.151:0/2112776823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5178004d40 0x7f51780051c0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f5188005fd0 tx=0x7f5178005830 comp rx=0 tx=0).stop 2024-09-16T15:51:59.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.913+0000 7f5196400640 1 -- 172.21.15.151:0/2112776823 reap_dead start 2024-09-16T15:51:59.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.913+0000 7f5196e00640 1 -- 172.21.15.151:0/2112776823 shutdown_connections 2024-09-16T15:51:59.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.913+0000 7f5196e00640 1 -- 172.21.15.151:0/2112776823 >> 172.21.15.151:0/2112776823 conn(0x7f51900d3e60 msgr2=0x7f51900d4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:51:59.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.913+0000 7f5196e00640 1 -- 172.21.15.151:0/2112776823 shutdown_connections 2024-09-16T15:51:59.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.913+0000 7f5196e00640 1 -- 172.21.15.151:0/2112776823 wait complete. 2024-09-16T15:51:59.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.915+0000 7f5196e00640 1 Processor -- start 2024-09-16T15:51:59.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.915+0000 7f5196e00640 1 -- start start 2024-09-16T15:51:59.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.916+0000 7f5196e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5178004d40 0x7f51901f5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:59.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.916+0000 7f5196e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51900cba90 0x7f519009e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:59.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.916+0000 7f5196e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51900c9670 0x7f51901f17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:59.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.916+0000 7f5196e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f51900d1310 con 0x7f5178004d40 2024-09-16T15:51:59.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.916+0000 7f5196e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f51900d1190 con 0x7f51900c9670 2024-09-16T15:51:59.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.916+0000 7f5196e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f51900d1490 con 0x7f51900cba90 2024-09-16T15:51:59.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.916+0000 7f5195a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5178004d40 0x7f51901f5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:59.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.916+0000 7f5195a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5178004d40 0x7f51901f5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:60768/0 (socket says 172.21.15.151:60768) 2024-09-16T15:51:59.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.916+0000 7f5195a00640 1 -- 172.21.15.151:0/1198087677 learned_addr learned my addr 172.21.15.151:0/1198087677 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:51:59.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.916+0000 7f5196400640 1 --2- 172.21.15.151:0/1198087677 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51900c9670 0x7f51901f17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:59.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.916+0000 7f5195000640 1 --2- 172.21.15.151:0/1198087677 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51900cba90 0x7f519009e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:59.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.916+0000 7f5195a00640 1 -- 172.21.15.151:0/1198087677 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51900cba90 msgr2=0x7f519009e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:59.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.916+0000 7f5195a00640 1 --2- 172.21.15.151:0/1198087677 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51900cba90 0x7f519009e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:59.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.916+0000 7f5195a00640 1 -- 172.21.15.151:0/1198087677 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51900c9670 msgr2=0x7f51901f17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:51:59.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.916+0000 7f5195a00640 1 --2- 172.21.15.151:0/1198087677 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51900c9670 0x7f51901f17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:51:59.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.916+0000 7f5195a00640 1 -- 172.21.15.151:0/1198087677 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f519009ee60 con 0x7f5178004d40 2024-09-16T15:51:59.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.916+0000 7f5195000640 1 --2- 172.21.15.151:0/1198087677 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51900cba90 0x7f519009e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:51:59.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.917+0000 7f5195a00640 1 --2- 172.21.15.151:0/1198087677 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5178004d40 0x7f51901f5400 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f5184011ae0 tx=0x7f5184009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:59.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.917+0000 7f518d600640 1 -- 172.21.15.151:0/1198087677 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518400fcb0 con 0x7f5178004d40 2024-09-16T15:51:59.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.917+0000 7f518d600640 1 -- 172.21.15.151:0/1198087677 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f518400fe50 con 0x7f5178004d40 2024-09-16T15:51:59.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.917+0000 7f518d600640 1 -- 172.21.15.151:0/1198087677 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5184004280 con 0x7f5178004d40 2024-09-16T15:51:59.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.917+0000 7f5196e00640 1 -- 172.21.15.151:0/1198087677 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f519009f0e0 con 0x7f5178004d40 2024-09-16T15:51:59.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.917+0000 7f5196e00640 1 -- 172.21.15.151:0/1198087677 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51900cb2b0 con 0x7f5178004d40 2024-09-16T15:51:59.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.918+0000 7f5196e00640 1 -- 172.21.15.151:0/1198087677 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51900d0c50 con 0x7f5178004d40 2024-09-16T15:51:59.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.919+0000 7f518d600640 1 -- 172.21.15.151:0/1198087677 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 40) ==== 103582+0+0 (secure 0 0 0) 0x7f51840035f0 con 0x7f5178004d40 2024-09-16T15:51:59.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.920+0000 7f518d600640 1 --2- 172.21.15.151:0/1198087677 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f517007a200 0x7f517007c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:51:59.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.920+0000 7f518d600640 1 -- 172.21.15.151:0/1198087677 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5765+0+0 (secure 0 0 0) 0x7f518409a130 con 0x7f5178004d40 2024-09-16T15:51:59.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.921+0000 7f5195000640 1 --2- 172.21.15.151:0/1198087677 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f517007a200 0x7f517007c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:51:59.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.922+0000 7f5195000640 1 --2- 172.21.15.151:0/1198087677 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f517007a200 0x7f517007c6c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f517c000c00 tx=0x7f517c0015f0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:51:59.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:51:59.924+0000 7f518d600640 1 -- 172.21.15.151:0/1198087677 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f519009ee60 con 0x7f5178004d40 2024-09-16T15:52:00.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.141+0000 7f5196e00640 1 -- 172.21.15.151:0/1198087677 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f51900d26f0 con 0x7f517007a200 2024-09-16T15:52:00.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.174+0000 7f518d600640 1 -- 172.21.15.151:0/1198087677 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f51900d26f0 con 0x7f517007a200 2024-09-16T15:52:00.175 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:52:00.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.177+0000 7f5196e00640 1 -- 172.21.15.151:0/1198087677 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f517007a200 msgr2=0x7f517007c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:00.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.177+0000 7f5196e00640 1 --2- 172.21.15.151:0/1198087677 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f517007a200 0x7f517007c6c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f517c000c00 tx=0x7f517c0015f0 comp rx=0 tx=0).stop 2024-09-16T15:52:00.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.177+0000 7f5196e00640 1 -- 172.21.15.151:0/1198087677 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5178004d40 msgr2=0x7f51901f5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:00.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.177+0000 7f5196e00640 1 --2- 172.21.15.151:0/1198087677 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5178004d40 0x7f51901f5400 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f5184011ae0 tx=0x7f5184009f10 comp rx=0 tx=0).stop 2024-09-16T15:52:00.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.177+0000 7f5196e00640 1 -- 172.21.15.151:0/1198087677 shutdown_connections 2024-09-16T15:52:00.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.177+0000 7f5196e00640 1 --2- 172.21.15.151:0/1198087677 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f517007a200 0x7f517007c6c0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:00.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.177+0000 7f5196e00640 1 --2- 172.21.15.151:0/1198087677 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51900c9670 0x7f51901f17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:00.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.177+0000 7f5196e00640 1 --2- 172.21.15.151:0/1198087677 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51900cba90 0x7f519009e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:00.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.177+0000 7f5196e00640 1 --2- 172.21.15.151:0/1198087677 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5178004d40 0x7f51901f5400 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:00.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.177+0000 7f5196e00640 1 -- 172.21.15.151:0/1198087677 >> 172.21.15.151:0/1198087677 conn(0x7f51900d3e60 msgr2=0x7f51900b5f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:00.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.177+0000 7f5196e00640 1 -- 172.21.15.151:0/1198087677 shutdown_connections 2024-09-16T15:52:00.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.177+0000 7f5196e00640 1 -- 172.21.15.151:0/1198087677 wait complete. 2024-09-16T15:52:00.194 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-09-16T15:52:00.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.355+0000 7f1850600640 1 -- 172.21.15.151:0/3254466460 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f184806efa0 msgr2=0x7f18480cc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:00.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.355+0000 7f1850600640 1 --2- 172.21.15.151:0/3254466460 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f184806efa0 0x7f18480cc5c0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f1834005fd0 tx=0x7f183402a540 comp rx=0 tx=0).stop 2024-09-16T15:52:00.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.355+0000 7f1850600640 1 -- 172.21.15.151:0/3254466460 shutdown_connections 2024-09-16T15:52:00.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.355+0000 7f1850600640 1 --2- 172.21.15.151:0/3254466460 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f184806efa0 0x7f18480cc5c0 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:00.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.355+0000 7f1850600640 1 --2- 172.21.15.151:0/3254466460 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1848145750 0x7f1848073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:00.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.355+0000 7f1850600640 1 -- 172.21.15.151:0/3254466460 >> 172.21.15.151:0/3254466460 conn(0x7f18480b1840 msgr2=0x7f18480b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:00.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.356+0000 7f1850600640 1 -- 172.21.15.151:0/3254466460 shutdown_connections 2024-09-16T15:52:00.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.356+0000 7f1850600640 1 -- 172.21.15.151:0/3254466460 wait complete. 2024-09-16T15:52:00.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.357+0000 7f1850600640 1 Processor -- start 2024-09-16T15:52:00.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.357+0000 7f1850600640 1 -- start start 2024-09-16T15:52:00.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.358+0000 7f1850600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f184806efa0 0x7f18480b3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:00.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.358+0000 7f1850600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f18480ccb00 0x7f184814b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:00.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.358+0000 7f1850600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1848145750 0x7f18481f42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:00.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.358+0000 7f1850600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f18480b2650 con 0x7f18480ccb00 2024-09-16T15:52:00.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.358+0000 7f1850600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f18480cedd0 con 0x7f1848145750 2024-09-16T15:52:00.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.358+0000 7f1850600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f1848160100 con 0x7f184806efa0 2024-09-16T15:52:00.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.359+0000 7f184d200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f18480ccb00 0x7f184814b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:00.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.359+0000 7f184d200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f18480ccb00 0x7f184814b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:60778/0 (socket says 172.21.15.151:60778) 2024-09-16T15:52:00.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.359+0000 7f184d200640 1 -- 172.21.15.151:0/2036154888 learned_addr learned my addr 172.21.15.151:0/2036154888 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:52:00.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.359+0000 7f184e600640 1 --2- 172.21.15.151:0/2036154888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1848145750 0x7f18481f42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:00.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.359+0000 7f184dc00640 1 --2- 172.21.15.151:0/2036154888 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f184806efa0 0x7f18480b3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:00.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.359+0000 7f184d200640 1 -- 172.21.15.151:0/2036154888 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f184806efa0 msgr2=0x7f18480b3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:00.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.359+0000 7f184d200640 1 --2- 172.21.15.151:0/2036154888 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f184806efa0 0x7f18480b3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:00.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.359+0000 7f184d200640 1 -- 172.21.15.151:0/2036154888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1848145750 msgr2=0x7f18481f42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:00.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.359+0000 7f184d200640 1 --2- 172.21.15.151:0/2036154888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1848145750 0x7f18481f42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:00.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.359+0000 7f184d200640 1 -- 172.21.15.151:0/2036154888 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f184814bb60 con 0x7f18480ccb00 2024-09-16T15:52:00.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.360+0000 7f184d200640 1 --2- 172.21.15.151:0/2036154888 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f18480ccb00 0x7f184814b3c0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f1834009850 tx=0x7f18340038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:00.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.360+0000 7f1845600640 1 -- 172.21.15.151:0/2036154888 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183403f070 con 0x7f18480ccb00 2024-09-16T15:52:00.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.360+0000 7f1845600640 1 -- 172.21.15.151:0/2036154888 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f18340047a0 con 0x7f18480ccb00 2024-09-16T15:52:00.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.360+0000 7f1845600640 1 -- 172.21.15.151:0/2036154888 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1834035030 con 0x7f18480ccb00 2024-09-16T15:52:00.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.360+0000 7f1850600640 1 -- 172.21.15.151:0/2036154888 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f184814be40 con 0x7f18480ccb00 2024-09-16T15:52:00.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.360+0000 7f1850600640 1 -- 172.21.15.151:0/2036154888 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f184814c400 con 0x7f18480ccb00 2024-09-16T15:52:00.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.361+0000 7f1850600640 1 -- 172.21.15.151:0/2036154888 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18481605b0 con 0x7f18480ccb00 2024-09-16T15:52:00.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.362+0000 7f1845600640 1 -- 172.21.15.151:0/2036154888 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 40) ==== 103582+0+0 (secure 0 0 0) 0x7f1834042070 con 0x7f18480ccb00 2024-09-16T15:52:00.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.363+0000 7f1845600640 1 --2- 172.21.15.151:0/2036154888 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f182807a1d0 0x7f182807c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:00.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.363+0000 7f1845600640 1 -- 172.21.15.151:0/2036154888 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5765+0+0 (secure 0 0 0) 0x7f18340ba550 con 0x7f18480ccb00 2024-09-16T15:52:00.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.364+0000 7f184dc00640 1 --2- 172.21.15.151:0/2036154888 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f182807a1d0 0x7f182807c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:00.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.364+0000 7f184dc00640 1 --2- 172.21.15.151:0/2036154888 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f182807a1d0 0x7f182807c690 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f183c005fd0 tx=0x7f183c009660 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:00.368 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.367+0000 7f1845600640 1 -- 172.21.15.151:0/2036154888 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1834083fd0 con 0x7f18480ccb00 2024-09-16T15:52:00.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.585+0000 7f1850600640 1 -- 172.21.15.151:0/2036154888 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f1848073710 con 0x7f182807a1d0 2024-09-16T15:52:00.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.608+0000 7f1845600640 1 -- 172.21.15.151:0/2036154888 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1848073710 con 0x7f182807a1d0 2024-09-16T15:52:00.611 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.610+0000 7f1850600640 1 -- 172.21.15.151:0/2036154888 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f182807a1d0 msgr2=0x7f182807c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:00.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.610+0000 7f1850600640 1 --2- 172.21.15.151:0/2036154888 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f182807a1d0 0x7f182807c690 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f183c005fd0 tx=0x7f183c009660 comp rx=0 tx=0).stop 2024-09-16T15:52:00.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.610+0000 7f1850600640 1 -- 172.21.15.151:0/2036154888 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f18480ccb00 msgr2=0x7f184814b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:00.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.610+0000 7f1850600640 1 --2- 172.21.15.151:0/2036154888 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f18480ccb00 0x7f184814b3c0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f1834009850 tx=0x7f18340038a0 comp rx=0 tx=0).stop 2024-09-16T15:52:00.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.610+0000 7f1850600640 1 -- 172.21.15.151:0/2036154888 shutdown_connections 2024-09-16T15:52:00.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.610+0000 7f1850600640 1 --2- 172.21.15.151:0/2036154888 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f182807a1d0 0x7f182807c690 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:00.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.610+0000 7f1850600640 1 --2- 172.21.15.151:0/2036154888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1848145750 0x7f18481f42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:00.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.610+0000 7f1850600640 1 --2- 172.21.15.151:0/2036154888 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f18480ccb00 0x7f184814b3c0 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:00.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.610+0000 7f1850600640 1 --2- 172.21.15.151:0/2036154888 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f184806efa0 0x7f18480b3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:00.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.610+0000 7f1850600640 1 -- 172.21.15.151:0/2036154888 >> 172.21.15.151:0/2036154888 conn(0x7f18480b1840 msgr2=0x7f18480cfbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:00.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.610+0000 7f1850600640 1 -- 172.21.15.151:0/2036154888 shutdown_connections 2024-09-16T15:52:00.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.610+0000 7f1850600640 1 -- 172.21.15.151:0/2036154888 wait complete. 2024-09-16T15:52:00.623 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:52:00.622+0000 7f48f3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:52:00.623 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:52:00.622+0000 7f48f3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:52:00.623 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:52:00.622+0000 7f48f3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:52:00.623 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:52:00.622+0000 7f48f3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:52:00.623 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:52:00.622+0000 7f48f3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:52:00.628 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs ss1clone1 2024-09-16T15:52:00.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.792+0000 7f775ea00640 1 -- 172.21.15.151:0/2505765130 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77580cde60 msgr2=0x7f77580b3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:00.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.792+0000 7f775ea00640 1 --2- 172.21.15.151:0/2505765130 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77580cde60 0x7f77580b3dd0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f7750005fd0 tx=0x7f775002a7a0 comp rx=0 tx=0).stop 2024-09-16T15:52:00.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.792+0000 7f775ea00640 1 -- 172.21.15.151:0/2505765130 shutdown_connections 2024-09-16T15:52:00.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.792+0000 7f775ea00640 1 --2- 172.21.15.151:0/2505765130 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77580cde60 0x7f77580b3dd0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:00.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.792+0000 7f775ea00640 1 --2- 172.21.15.151:0/2505765130 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f77580cea50 0x7f77580b3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:00.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.792+0000 7f775ea00640 1 -- 172.21.15.151:0/2505765130 >> 172.21.15.151:0/2505765130 conn(0x7f77580d0290 msgr2=0x7f77580d06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:00.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.793+0000 7f775ea00640 1 -- 172.21.15.151:0/2505765130 shutdown_connections 2024-09-16T15:52:00.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.793+0000 7f775ea00640 1 -- 172.21.15.151:0/2505765130 wait complete. 2024-09-16T15:52:00.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.794+0000 7f775ea00640 1 Processor -- start 2024-09-16T15:52:00.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.794+0000 7f775ea00640 1 -- start start 2024-09-16T15:52:00.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.795+0000 7f775ea00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f77580cde60 0x7f775815b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:00.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.795+0000 7f775ea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f77580cea50 0x7f775815c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:00.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.795+0000 7f775ea00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f775815e190 0x7f775815c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:00.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.795+0000 7f775ea00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f775815e6f0 con 0x7f77580cea50 2024-09-16T15:52:00.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.795+0000 7f775ea00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f775815e570 con 0x7f775815e190 2024-09-16T15:52:00.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.795+0000 7f775ea00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f77580b4310 con 0x7f77580cde60 2024-09-16T15:52:00.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.795+0000 7f775cc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f77580cea50 0x7f775815c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:00.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.796+0000 7f775cc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f77580cea50 0x7f775815c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:60790/0 (socket says 172.21.15.151:60790) 2024-09-16T15:52:00.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.796+0000 7f775cc00640 1 -- 172.21.15.151:0/4053415500 learned_addr learned my addr 172.21.15.151:0/4053415500 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:52:00.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.796+0000 7f775d600640 1 --2- 172.21.15.151:0/4053415500 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f77580cde60 0x7f775815b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:00.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.796+0000 7f775e000640 1 --2- 172.21.15.151:0/4053415500 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f775815e190 0x7f775815c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:00.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.796+0000 7f775cc00640 1 -- 172.21.15.151:0/4053415500 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f77580cde60 msgr2=0x7f775815b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:00.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.796+0000 7f775cc00640 1 --2- 172.21.15.151:0/4053415500 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f77580cde60 0x7f775815b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:00.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.796+0000 7f775cc00640 1 -- 172.21.15.151:0/4053415500 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f775815e190 msgr2=0x7f775815c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:00.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.796+0000 7f775cc00640 1 --2- 172.21.15.151:0/4053415500 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f775815e190 0x7f775815c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:00.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.796+0000 7f775cc00640 1 -- 172.21.15.151:0/4053415500 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77580c55b0 con 0x7f77580cea50 2024-09-16T15:52:00.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.796+0000 7f775cc00640 1 --2- 172.21.15.151:0/4053415500 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f77580cea50 0x7f775815c020 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f774400ea20 tx=0x7f774400eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:00.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.797+0000 7f7755600640 1 -- 172.21.15.151:0/4053415500 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774400cd80 con 0x7f77580cea50 2024-09-16T15:52:00.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.797+0000 7f7755600640 1 -- 172.21.15.151:0/4053415500 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7744010430 con 0x7f77580cea50 2024-09-16T15:52:00.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.797+0000 7f7755600640 1 -- 172.21.15.151:0/4053415500 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7744010840 con 0x7f77580cea50 2024-09-16T15:52:00.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.797+0000 7f775ea00640 1 -- 172.21.15.151:0/4053415500 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77580c5890 con 0x7f77580cea50 2024-09-16T15:52:00.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.797+0000 7f775ea00640 1 -- 172.21.15.151:0/4053415500 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77581e4750 con 0x7f77580cea50 2024-09-16T15:52:00.799 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.798+0000 7f775ea00640 1 -- 172.21.15.151:0/4053415500 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77580b4850 con 0x7f77580cea50 2024-09-16T15:52:00.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.800+0000 7f7755600640 1 -- 172.21.15.151:0/4053415500 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 40) ==== 103582+0+0 (secure 0 0 0) 0x7f77440109e0 con 0x7f77580cea50 2024-09-16T15:52:00.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.801+0000 7f7755600640 1 --2- 172.21.15.151:0/4053415500 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f773807a1a0 0x7f773807c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:00.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.801+0000 7f7755600640 1 -- 172.21.15.151:0/4053415500 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5765+0+0 (secure 0 0 0) 0x7f774409a280 con 0x7f77580cea50 2024-09-16T15:52:00.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.802+0000 7f775d600640 1 --2- 172.21.15.151:0/4053415500 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f773807a1a0 0x7f773807c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:00.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.802+0000 7f775d600640 1 --2- 172.21.15.151:0/4053415500 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f773807a1a0 0x7f773807c660 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f774c007960 tx=0x7f774c001d50 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:00.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:00.805+0000 7f7755600640 1 -- 172.21.15.151:0/4053415500 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7744063d00 con 0x7f77580cea50 2024-09-16T15:52:01.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:01.170+0000 7f775ea00640 1 -- 172.21.15.151:0/4053415500 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f77580703d0 con 0x7f773807a1a0 2024-09-16T15:52:01.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:01.186+0000 7f7755600640 1 -- 172.21.15.151:0/4053415500 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f77580703d0 con 0x7f773807a1a0 2024-09-16T15:52:01.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:01.189+0000 7f775ea00640 1 -- 172.21.15.151:0/4053415500 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f773807a1a0 msgr2=0x7f773807c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:01.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:01.189+0000 7f775ea00640 1 --2- 172.21.15.151:0/4053415500 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f773807a1a0 0x7f773807c660 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f774c007960 tx=0x7f774c001d50 comp rx=0 tx=0).stop 2024-09-16T15:52:01.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:01.189+0000 7f775ea00640 1 -- 172.21.15.151:0/4053415500 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f77580cea50 msgr2=0x7f775815c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:01.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:01.189+0000 7f775ea00640 1 --2- 172.21.15.151:0/4053415500 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f77580cea50 0x7f775815c020 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f774400ea20 tx=0x7f774400eee0 comp rx=0 tx=0).stop 2024-09-16T15:52:01.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:01.189+0000 7f775ea00640 1 -- 172.21.15.151:0/4053415500 shutdown_connections 2024-09-16T15:52:01.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:01.189+0000 7f775ea00640 1 --2- 172.21.15.151:0/4053415500 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f773807a1a0 0x7f773807c660 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:01.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:01.189+0000 7f775ea00640 1 --2- 172.21.15.151:0/4053415500 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f775815e190 0x7f775815c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:01.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:01.189+0000 7f775ea00640 1 --2- 172.21.15.151:0/4053415500 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f77580cea50 0x7f775815c020 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:01.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:01.189+0000 7f775ea00640 1 --2- 172.21.15.151:0/4053415500 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f77580cde60 0x7f775815b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:01.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:01.189+0000 7f775ea00640 1 -- 172.21.15.151:0/4053415500 >> 172.21.15.151:0/4053415500 conn(0x7f77580d0290 msgr2=0x7f775815d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:01.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:01.190+0000 7f775ea00640 1 -- 172.21.15.151:0/4053415500 shutdown_connections 2024-09-16T15:52:01.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:01.190+0000 7f775ea00640 1 -- 172.21.15.151:0/4053415500 wait complete. 2024-09-16T15:52:01.200 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:52:01.199+0000 7f48f2600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:52:01.200 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:52:01.199+0000 7f48f2600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:52:01.200 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:52:01.199+0000 7f48f2600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:52:01.200 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:52:01.199+0000 7f48f2600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:52:01.200 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:52:01.199+0000 7f48f2600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:52:01.210 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-16T15:52:01.505 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:01 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:52:01.512 INFO:teuthology.orchestra.run.smithi151.stdout:2 2024-09-16T15:52:01.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:52:01.521 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-16T15:52:01.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:01.682+0000 7f1505a00640 1 -- 172.21.15.151:0/1878685901 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1500145750 msgr2=0x7f1500073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:01.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:01.682+0000 7f1505a00640 1 --2- 172.21.15.151:0/1878685901 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1500145750 0x7f1500073440 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f14f0005fd0 tx=0x7f14f002a540 comp rx=0 tx=0).stop 2024-09-16T15:52:01.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:01.682+0000 7f1505a00640 1 -- 172.21.15.151:0/1878685901 shutdown_connections 2024-09-16T15:52:01.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:01.682+0000 7f1505a00640 1 --2- 172.21.15.151:0/1878685901 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1500145750 0x7f1500073440 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:01.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:01.682+0000 7f1505a00640 1 -- 172.21.15.151:0/1878685901 >> 172.21.15.151:0/1878685901 conn(0x7f15000b1840 msgr2=0x7f15000b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:01.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:01.682+0000 7f1505a00640 1 -- 172.21.15.151:0/1878685901 shutdown_connections 2024-09-16T15:52:01.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:01.682+0000 7f1505a00640 1 -- 172.21.15.151:0/1878685901 wait complete. 2024-09-16T15:52:01.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:01.684+0000 7f1505a00640 1 Processor -- start 2024-09-16T15:52:01.685 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:01.684+0000 7f1505a00640 1 -- start start 2024-09-16T15:52:01.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:01.684+0000 7f1505a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f150006efa0 0x7f150013e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:01.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:01.685+0000 7f1505a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15000cccd0 0x7f15001f7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:01.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:01.685+0000 7f1505a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1500145750 0x7f15001f8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:01.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:01.685+0000 7f1505a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f15000ce6a0 con 0x7f15000cccd0 2024-09-16T15:52:01.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:01.685+0000 7f1505a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f15000ce520 con 0x7f150006efa0 2024-09-16T15:52:01.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:01.685+0000 7f1505a00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f15000ce820 con 0x7f1500145750 2024-09-16T15:52:01.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:01.685+0000 7f14fe000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15000cccd0 0x7f15001f7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:01.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:01.685+0000 7f14fe000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15000cccd0 0x7f15001f7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:60808/0 (socket says 172.21.15.151:60808) 2024-09-16T15:52:01.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:01.685+0000 7f14fe000640 1 -- 172.21.15.151:0/2184335880 learned_addr learned my addr 172.21.15.151:0/2184335880 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:52:01.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:01.685+0000 7f14fea00640 1 --2- 172.21.15.151:0/2184335880 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f150006efa0 0x7f150013e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:01.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:01.685+0000 7f14ff400640 1 --2- 172.21.15.151:0/2184335880 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1500145750 0x7f15001f8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:01.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:01.685+0000 7f14fe000640 1 -- 172.21.15.151:0/2184335880 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1500145750 msgr2=0x7f15001f8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:01.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:01.685+0000 7f14fe000640 1 --2- 172.21.15.151:0/2184335880 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1500145750 0x7f15001f8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:01.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:01.685+0000 7f14fe000640 1 -- 172.21.15.151:0/2184335880 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f150006efa0 msgr2=0x7f150013e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:01.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:01.685+0000 7f14fe000640 1 --2- 172.21.15.151:0/2184335880 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f150006efa0 0x7f150013e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:01.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:01.685+0000 7f14fe000640 1 -- 172.21.15.151:0/2184335880 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15000b45d0 con 0x7f15000cccd0 2024-09-16T15:52:01.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:01.685+0000 7f14fe000640 1 --2- 172.21.15.151:0/2184335880 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15000cccd0 0x7f15001f7be0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f14ec00c430 tx=0x7f14ec00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:01.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:01.686+0000 7f14f6a00640 1 -- 172.21.15.151:0/2184335880 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ec011790 con 0x7f15000cccd0 2024-09-16T15:52:01.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:01.686+0000 7f14f6a00640 1 -- 172.21.15.151:0/2184335880 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f14ec017070 con 0x7f15000cccd0 2024-09-16T15:52:01.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:01.686+0000 7f1505a00640 1 -- 172.21.15.151:0/2184335880 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15000b42f0 con 0x7f15000cccd0 2024-09-16T15:52:01.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:01.686+0000 7f14f6a00640 1 -- 172.21.15.151:0/2184335880 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ec011d00 con 0x7f15000cccd0 2024-09-16T15:52:01.689 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:01.686+0000 7f1505a00640 1 -- 172.21.15.151:0/2184335880 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15001f5aa0 con 0x7f15000cccd0 2024-09-16T15:52:01.689 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:01.687+0000 7f1505a00640 1 -- 172.21.15.151:0/2184335880 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15000ced10 con 0x7f15000cccd0 2024-09-16T15:52:01.689 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:01.688+0000 7f14f6a00640 1 -- 172.21.15.151:0/2184335880 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 40) ==== 103582+0+0 (secure 0 0 0) 0x7f14ec01c460 con 0x7f15000cccd0 2024-09-16T15:52:01.690 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:01.689+0000 7f14f6a00640 1 --2- 172.21.15.151:0/2184335880 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f14e007a1a0 0x7f14e007c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:01.690 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:01.689+0000 7f14f6a00640 1 -- 172.21.15.151:0/2184335880 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5765+0+0 (secure 0 0 0) 0x7f14ec099610 con 0x7f15000cccd0 2024-09-16T15:52:01.690 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:01.690+0000 7f14fea00640 1 --2- 172.21.15.151:0/2184335880 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f14e007a1a0 0x7f14e007c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:01.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:01.690+0000 7f14fea00640 1 --2- 172.21.15.151:0/2184335880 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f14e007a1a0 0x7f14e007c660 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f14f0033000 tx=0x7f14f0004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:01.694 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:01.693+0000 7f14f6a00640 1 -- 172.21.15.151:0/2184335880 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f14ec0636e0 con 0x7f15000cccd0 2024-09-16T15:52:01.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:01.882+0000 7f1505a00640 1 -- 172.21.15.151:0/2184335880 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f1500069a10 con 0x7f15000cccd0 2024-09-16T15:52:01.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:01.883+0000 7f14f6a00640 1 -- 172.21.15.151:0/2184335880 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v46)=0 v46) ==== 131+0+0 (secure 0 0 0) 0x7f14ec062fe0 con 0x7f15000cccd0 2024-09-16T15:52:01.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:01.885+0000 7f1505a00640 1 -- 172.21.15.151:0/2184335880 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f14e007a1a0 msgr2=0x7f14e007c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:01.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:01.885+0000 7f1505a00640 1 --2- 172.21.15.151:0/2184335880 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f14e007a1a0 0x7f14e007c660 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f14f0033000 tx=0x7f14f0004730 comp rx=0 tx=0).stop 2024-09-16T15:52:01.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:01.885+0000 7f1505a00640 1 -- 172.21.15.151:0/2184335880 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15000cccd0 msgr2=0x7f15001f7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:01.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:01.885+0000 7f1505a00640 1 --2- 172.21.15.151:0/2184335880 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15000cccd0 0x7f15001f7be0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f14ec00c430 tx=0x7f14ec00c8f0 comp rx=0 tx=0).stop 2024-09-16T15:52:01.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:01.885+0000 7f1505a00640 1 -- 172.21.15.151:0/2184335880 shutdown_connections 2024-09-16T15:52:01.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:01.885+0000 7f1505a00640 1 --2- 172.21.15.151:0/2184335880 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f14e007a1a0 0x7f14e007c660 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:01.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:01.885+0000 7f1505a00640 1 --2- 172.21.15.151:0/2184335880 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1500145750 0x7f15001f8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:01.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:01.885+0000 7f1505a00640 1 --2- 172.21.15.151:0/2184335880 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15000cccd0 0x7f15001f7be0 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:01.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:01.885+0000 7f1505a00640 1 --2- 172.21.15.151:0/2184335880 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f150006efa0 0x7f150013e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:01.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:01.885+0000 7f1505a00640 1 -- 172.21.15.151:0/2184335880 >> 172.21.15.151:0/2184335880 conn(0x7f15000b1840 msgr2=0x7f15000a94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:01.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:01.885+0000 7f1505a00640 1 -- 172.21.15.151:0/2184335880 shutdown_connections 2024-09-16T15:52:01.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:01.885+0000 7f1505a00640 1 -- 172.21.15.151:0/2184335880 wait complete. 2024-09-16T15:52:01.903 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-16T15:52:01.903 DEBUG:teuthology.orchestra.run.smithi151:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:52:01.994 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:52:02.081 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-16T15:52:02.081 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:52:02.081 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:52:02.081 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:52:02.117 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:52:02.117 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:52:02.192 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-16T15:52:02.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:02.390+0000 7f6720c00640 1 -- 172.21.15.151:0/1114534864 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6700004d40 msgr2=0x7f67000051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:02.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:02.390+0000 7f6720c00640 1 --2- 172.21.15.151:0/1114534864 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6700004d40 0x7f67000051c0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f671400e3d0 tx=0x7f6700005830 comp rx=0 tx=0).stop 2024-09-16T15:52:02.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:02.390+0000 7f671a800640 1 -- 172.21.15.151:0/1114534864 reap_dead start 2024-09-16T15:52:02.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:02.390+0000 7f6720c00640 1 -- 172.21.15.151:0/1114534864 shutdown_connections 2024-09-16T15:52:02.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:02.390+0000 7f6720c00640 1 -- 172.21.15.151:0/1114534864 >> 172.21.15.151:0/1114534864 conn(0x7f671c0b1950 msgr2=0x7f671c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:02.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:02.390+0000 7f6720c00640 1 -- 172.21.15.151:0/1114534864 shutdown_connections 2024-09-16T15:52:02.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:02.390+0000 7f6720c00640 1 -- 172.21.15.151:0/1114534864 wait complete. 2024-09-16T15:52:02.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:02.392+0000 7f6720c00640 1 Processor -- start 2024-09-16T15:52:02.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:02.392+0000 7f6720c00640 1 -- start start 2024-09-16T15:52:02.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:02.393+0000 7f6720c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6700004d40 0x7f671c0d02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:02.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:02.393+0000 7f6720c00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f671c1f67b0 0x7f671c155620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:02.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:02.393+0000 7f6720c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f671c1f2580 0x7f671c0afe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:02.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:02.393+0000 7f6720c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f671c0b25c0 con 0x7f6700004d40 2024-09-16T15:52:02.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:02.393+0000 7f6720c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f671c0b2440 con 0x7f671c1f2580 2024-09-16T15:52:02.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:02.393+0000 7f6720c00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f671c0b2740 con 0x7f671c1f67b0 2024-09-16T15:52:02.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:02.393+0000 7f6719e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6700004d40 0x7f671c0d02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:02.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:02.393+0000 7f6719e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6700004d40 0x7f671c0d02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:60828/0 (socket says 172.21.15.151:60828) 2024-09-16T15:52:02.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:02.393+0000 7f6719e00640 1 -- 172.21.15.151:0/592533401 learned_addr learned my addr 172.21.15.151:0/592533401 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:52:02.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:02.393+0000 7f6719400640 1 --2- 172.21.15.151:0/592533401 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f671c1f67b0 0x7f671c155620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:02.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:02.393+0000 7f671a800640 1 --2- 172.21.15.151:0/592533401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f671c1f2580 0x7f671c0afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:02.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:02.393+0000 7f6719e00640 1 -- 172.21.15.151:0/592533401 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f671c1f67b0 msgr2=0x7f671c155620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:02.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:02.393+0000 7f6719e00640 1 --2- 172.21.15.151:0/592533401 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f671c1f67b0 0x7f671c155620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:02.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:02.393+0000 7f6719e00640 1 -- 172.21.15.151:0/592533401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f671c1f2580 msgr2=0x7f671c0afe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:02.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:02.393+0000 7f6719e00640 1 --2- 172.21.15.151:0/592533401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f671c1f2580 0x7f671c0afe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:02.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:02.393+0000 7f6719e00640 1 -- 172.21.15.151:0/592533401 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f671c1f2f40 con 0x7f6700004d40 2024-09-16T15:52:02.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:02.393+0000 7f6719400640 1 --2- 172.21.15.151:0/592533401 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f671c1f67b0 0x7f671c155620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:52:02.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:02.393+0000 7f671a800640 1 --2- 172.21.15.151:0/592533401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f671c1f2580 0x7f671c0afe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:52:02.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:02.394+0000 7f6719e00640 1 --2- 172.21.15.151:0/592533401 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6700004d40 0x7f671c0d02a0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f670c011ae0 tx=0x7f670c009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:02.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:02.394+0000 7f6712000640 1 -- 172.21.15.151:0/592533401 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f670c01e070 con 0x7f6700004d40 2024-09-16T15:52:02.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:02.394+0000 7f6712000640 1 -- 172.21.15.151:0/592533401 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f670c00fe70 con 0x7f6700004d40 2024-09-16T15:52:02.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:02.394+0000 7f6712000640 1 -- 172.21.15.151:0/592533401 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f670c004290 con 0x7f6700004d40 2024-09-16T15:52:02.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:02.394+0000 7f6720c00640 1 -- 172.21.15.151:0/592533401 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f671c1f3220 con 0x7f6700004d40 2024-09-16T15:52:02.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:02.394+0000 7f6720c00640 1 -- 172.21.15.151:0/592533401 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f671c0cfd80 con 0x7f6700004d40 2024-09-16T15:52:02.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:02.395+0000 7f6720c00640 1 -- 172.21.15.151:0/592533401 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f671c1f3460 con 0x7f6700004d40 2024-09-16T15:52:02.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:02.396+0000 7f6712000640 1 -- 172.21.15.151:0/592533401 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 40) ==== 103582+0+0 (secure 0 0 0) 0x7f670c021070 con 0x7f6700004d40 2024-09-16T15:52:02.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:02.397+0000 7f6712000640 1 --2- 172.21.15.151:0/592533401 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f66fc07a1a0 0x7f66fc07c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:02.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:02.397+0000 7f6712000640 1 -- 172.21.15.151:0/592533401 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5765+0+0 (secure 0 0 0) 0x7f670c09a190 con 0x7f6700004d40 2024-09-16T15:52:02.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:02.398+0000 7f6719400640 1 --2- 172.21.15.151:0/592533401 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f66fc07a1a0 0x7f66fc07c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:02.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:02.399+0000 7f6719400640 1 --2- 172.21.15.151:0/592533401 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f66fc07a1a0 0x7f66fc07c660 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f671c1f5ad0 tx=0x7f6708000f50 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:02.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:02.402+0000 7f6712000640 1 -- 172.21.15.151:0/592533401 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f671c1f3460 con 0x7f6700004d40 2024-09-16T15:52:02.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:02.734+0000 7f6720c00640 1 -- 172.21.15.151:0/592533401 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f671c074460 con 0x7f6700004d40 2024-09-16T15:52:02.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:02.735+0000 7f6712000640 1 -- 172.21.15.151:0/592533401 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v104) ==== 80+0+0 (secure 0 0 0) 0x7f670c063d70 con 0x7f6700004d40 2024-09-16T15:52:02.737 INFO:teuthology.orchestra.run.smithi151.stderr:listed 0 entries 2024-09-16T15:52:02.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:02.739+0000 7f6720c00640 1 -- 172.21.15.151:0/592533401 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f66fc07a1a0 msgr2=0x7f66fc07c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:02.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:02.739+0000 7f6720c00640 1 --2- 172.21.15.151:0/592533401 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f66fc07a1a0 0x7f66fc07c660 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f671c1f5ad0 tx=0x7f6708000f50 comp rx=0 tx=0).stop 2024-09-16T15:52:02.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:02.739+0000 7f6720c00640 1 -- 172.21.15.151:0/592533401 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6700004d40 msgr2=0x7f671c0d02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:02.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:02.739+0000 7f6720c00640 1 --2- 172.21.15.151:0/592533401 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6700004d40 0x7f671c0d02a0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f670c011ae0 tx=0x7f670c009e90 comp rx=0 tx=0).stop 2024-09-16T15:52:02.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:02.741+0000 7f6720c00640 1 -- 172.21.15.151:0/592533401 shutdown_connections 2024-09-16T15:52:02.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:02.741+0000 7f6720c00640 1 --2- 172.21.15.151:0/592533401 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f66fc07a1a0 0x7f66fc07c660 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:02.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:02.741+0000 7f6720c00640 1 --2- 172.21.15.151:0/592533401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f671c1f2580 0x7f671c0afe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:02.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:02.741+0000 7f6720c00640 1 --2- 172.21.15.151:0/592533401 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f671c1f67b0 0x7f671c155620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:02.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:02.741+0000 7f6720c00640 1 --2- 172.21.15.151:0/592533401 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6700004d40 0x7f671c0d02a0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:02.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:02.741+0000 7f6720c00640 1 -- 172.21.15.151:0/592533401 >> 172.21.15.151:0/592533401 conn(0x7f671c0b1950 msgr2=0x7f671c0ae5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:02.743 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:02.742+0000 7f6720c00640 1 -- 172.21.15.151:0/592533401 shutdown_connections 2024-09-16T15:52:02.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:02.742+0000 7f6720c00640 1 -- 172.21.15.151:0/592533401 wait complete. 2024-09-16T15:52:02.772 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-16T15:52:02.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:52:02.772 DEBUG:teuthology.orchestra.run.smithi151:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-16T15:52:02.879 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi151.front.sepia.ceph.com 2024-09-16T15:52:02.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:52:02.880 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-16T15:52:02.924 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:52:02.924 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-16T15:52:02.924 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:52:02.959 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:52:02.959 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:52:02.959 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:52:02.959 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:52:02.999 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T15:52:02.999 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:52:02.999 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-16T15:52:02.999 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:52:03.035 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:52:03.036 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:52:03.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.198+0000 7f5542a00640 1 -- 172.21.15.151:0/1923672774 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5520004080 msgr2=0x7f5520004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:03.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.198+0000 7f5542a00640 1 --2- 172.21.15.151:0/1923672774 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5520004080 0x7f5520004800 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f552800bb20 tx=0x7f5520005830 comp rx=0 tx=0).stop 2024-09-16T15:52:03.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.198+0000 7f5540a00640 1 -- 172.21.15.151:0/1923672774 reap_dead start 2024-09-16T15:52:03.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.198+0000 7f5542a00640 1 -- 172.21.15.151:0/1923672774 shutdown_connections 2024-09-16T15:52:03.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.198+0000 7f5542a00640 1 -- 172.21.15.151:0/1923672774 >> 172.21.15.151:0/1923672774 conn(0x7f553c0b1950 msgr2=0x7f553c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:03.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.198+0000 7f5542a00640 1 -- 172.21.15.151:0/1923672774 shutdown_connections 2024-09-16T15:52:03.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.198+0000 7f5542a00640 1 -- 172.21.15.151:0/1923672774 wait complete. 2024-09-16T15:52:03.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.200+0000 7f5542a00640 1 Processor -- start 2024-09-16T15:52:03.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.200+0000 7f5542a00640 1 -- start start 2024-09-16T15:52:03.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.201+0000 7f5542a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5520004080 0x7f553c156090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:03.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.201+0000 7f5542a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f553c0cade0 0x7f553c147790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:03.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.201+0000 7f5542a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f553c1f50a0 0x7f553c0af780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:03.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.201+0000 7f5542a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f553c0b25c0 con 0x7f5520004080 2024-09-16T15:52:03.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.201+0000 7f5542a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f553c0b2440 con 0x7f553c0cade0 2024-09-16T15:52:03.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.201+0000 7f5542a00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f553c0b2740 con 0x7f553c1f50a0 2024-09-16T15:52:03.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.201+0000 7f553be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5520004080 0x7f553c156090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:03.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.201+0000 7f553be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5520004080 0x7f553c156090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41132/0 (socket says 172.21.15.151:41132) 2024-09-16T15:52:03.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.201+0000 7f553be00640 1 -- 172.21.15.151:0/3521937864 learned_addr learned my addr 172.21.15.151:0/3521937864 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:52:03.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.201+0000 7f553b400640 1 --2- 172.21.15.151:0/3521937864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f553c0cade0 0x7f553c147790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:03.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.201+0000 7f5540a00640 1 --2- 172.21.15.151:0/3521937864 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f553c1f50a0 0x7f553c0af780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:03.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.201+0000 7f553be00640 1 -- 172.21.15.151:0/3521937864 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f553c1f50a0 msgr2=0x7f553c0af780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:03.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.201+0000 7f553be00640 1 --2- 172.21.15.151:0/3521937864 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f553c1f50a0 0x7f553c0af780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:03.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.201+0000 7f553be00640 1 -- 172.21.15.151:0/3521937864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f553c0cade0 msgr2=0x7f553c147790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:03.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.201+0000 7f553be00640 1 --2- 172.21.15.151:0/3521937864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f553c0cade0 0x7f553c147790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:03.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.201+0000 7f553be00640 1 -- 172.21.15.151:0/3521937864 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f553c156720 con 0x7f5520004080 2024-09-16T15:52:03.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.202+0000 7f553be00640 1 --2- 172.21.15.151:0/3521937864 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5520004080 0x7f553c156090 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f552c0062f0 tx=0x7f552c0067b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:03.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.202+0000 7f5533e00640 1 -- 172.21.15.151:0/3521937864 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552c015070 con 0x7f5520004080 2024-09-16T15:52:03.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.202+0000 7f5533e00640 1 -- 172.21.15.151:0/3521937864 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f552c00fd60 con 0x7f5520004080 2024-09-16T15:52:03.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.202+0000 7f5542a00640 1 -- 172.21.15.151:0/3521937864 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f553c0ca020 con 0x7f5520004080 2024-09-16T15:52:03.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.202+0000 7f5533e00640 1 -- 172.21.15.151:0/3521937864 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552c011690 con 0x7f5520004080 2024-09-16T15:52:03.205 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.202+0000 7f5542a00640 1 -- 172.21.15.151:0/3521937864 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f553c0ca5e0 con 0x7f5520004080 2024-09-16T15:52:03.205 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.203+0000 7f5542a00640 1 -- 172.21.15.151:0/3521937864 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f553c074440 con 0x7f5520004080 2024-09-16T15:52:03.205 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.205+0000 7f5533e00640 1 -- 172.21.15.151:0/3521937864 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 41) ==== 103684+0+0 (secure 0 0 0) 0x7f552c011830 con 0x7f5520004080 2024-09-16T15:52:03.207 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.206+0000 7f5533e00640 1 --2- 172.21.15.151:0/3521937864 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f551c07a2c0 0x7f551c07c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:03.207 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.206+0000 7f5533e00640 1 -- 172.21.15.151:0/3521937864 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5765+0+0 (secure 0 0 0) 0x7f552c09a7a0 con 0x7f5520004080 2024-09-16T15:52:03.207 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.206+0000 7f553b400640 1 --2- 172.21.15.151:0/3521937864 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f551c07a2c0 0x7f551c07c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:03.207 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.206+0000 7f553b400640 1 --2- 172.21.15.151:0/3521937864 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f551c07a2c0 0x7f551c07c780 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f5528036000 tx=0x7f552803c040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:03.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.210+0000 7f5533e00640 1 -- 172.21.15.151:0/3521937864 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f552c064220 con 0x7f5520004080 2024-09-16T15:52:03.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.430+0000 7f5542a00640 1 -- 172.21.15.151:0/3521937864 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f553c0ad070 con 0x7f5520004080 2024-09-16T15:52:03.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.431+0000 7f5533e00640 1 -- 172.21.15.151:0/3521937864 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5234 (secure 0 0 0) 0x7f552c068c50 con 0x7f5520004080 2024-09-16T15:52:03.432 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:52:03.432 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":67,"btime":"2024-09-16T15:51:15:447976+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6680,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2638998637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":2638998637},{"type":"v1","addr":"172.21.15.151:6835","nonce":2638998637}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":7330,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3105097249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3105097249},{"type":"v1","addr":"172.21.15.167:6836","nonce":3105097249}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":7333,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/613324294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":613324294},{"type":"v1","addr":"172.21.15.167:6837","nonce":613324294}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:51:01.789380+0000","modified":"2024-09-16T15:51:04.820629+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7899},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7899":{"gid":7899,"name":"c","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.151:6834/1174862515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1174862515},{"type":"v1","addr":"172.21.15.151:6834","nonce":1174862515}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7899,"qdb_cluster":[7899]},"id":7}]} 2024-09-16T15:52:03.433 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 67 2024-09-16T15:52:03.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.433+0000 7f5542a00640 1 -- 172.21.15.151:0/3521937864 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f551c07a2c0 msgr2=0x7f551c07c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:03.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.433+0000 7f5542a00640 1 --2- 172.21.15.151:0/3521937864 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f551c07a2c0 0x7f551c07c780 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f5528036000 tx=0x7f552803c040 comp rx=0 tx=0).stop 2024-09-16T15:52:03.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.433+0000 7f5542a00640 1 -- 172.21.15.151:0/3521937864 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5520004080 msgr2=0x7f553c156090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:03.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.433+0000 7f5542a00640 1 --2- 172.21.15.151:0/3521937864 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5520004080 0x7f553c156090 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f552c0062f0 tx=0x7f552c0067b0 comp rx=0 tx=0).stop 2024-09-16T15:52:03.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.433+0000 7f5542a00640 1 -- 172.21.15.151:0/3521937864 shutdown_connections 2024-09-16T15:52:03.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.433+0000 7f5542a00640 1 --2- 172.21.15.151:0/3521937864 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f551c07a2c0 0x7f551c07c780 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:03.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.434+0000 7f5542a00640 1 --2- 172.21.15.151:0/3521937864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f553c0cade0 0x7f553c147790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:03.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.434+0000 7f5542a00640 1 --2- 172.21.15.151:0/3521937864 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f553c1f50a0 0x7f553c0af780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:03.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.434+0000 7f5542a00640 1 --2- 172.21.15.151:0/3521937864 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5520004080 0x7f553c156090 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:03.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.434+0000 7f5542a00640 1 -- 172.21.15.151:0/3521937864 >> 172.21.15.151:0/3521937864 conn(0x7f553c0b1950 msgr2=0x7f553c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:03.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.434+0000 7f5542a00640 1 -- 172.21.15.151:0/3521937864 shutdown_connections 2024-09-16T15:52:03.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.434+0000 7f5542a00640 1 -- 172.21.15.151:0/3521937864 wait complete. 2024-09-16T15:52:03.456 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:52:03.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.709+0000 7fc105600640 1 -- 172.21.15.151:0/2209030886 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1000d2b40 msgr2=0x7fc1000d03b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:03.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.709+0000 7fc105600640 1 --2- 172.21.15.151:0/2209030886 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1000d2b40 0x7fc1000d03b0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fc1000b6bd0 tx=0x7fc0f802a7a0 comp rx=0 tx=0).stop 2024-09-16T15:52:03.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.709+0000 7fc105600640 1 -- 172.21.15.151:0/2209030886 shutdown_connections 2024-09-16T15:52:03.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.709+0000 7fc105600640 1 --2- 172.21.15.151:0/2209030886 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1000d2b40 0x7fc1000d03b0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:03.711 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.709+0000 7fc105600640 1 --2- 172.21.15.151:0/2209030886 >> v2:172.21.15.167:3300/0 conn(0x7fc1001616e0 0x7fc1000b41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:03.711 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.709+0000 7fc105600640 1 -- 172.21.15.151:0/2209030886 >> 172.21.15.151:0/2209030886 conn(0x7fc1000d3fe0 msgr2=0x7fc1000d43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:03.711 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.709+0000 7fc105600640 1 -- 172.21.15.151:0/2209030886 shutdown_connections 2024-09-16T15:52:03.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.709+0000 7fc105600640 1 -- 172.21.15.151:0/2209030886 wait complete. 2024-09-16T15:52:03.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.712+0000 7fc105600640 1 Processor -- start 2024-09-16T15:52:03.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.712+0000 7fc105600640 1 -- start start 2024-09-16T15:52:03.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.714+0000 7fc105600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1000d1ee0 0x7fc1000cdf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:03.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.714+0000 7fc105600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1000d2b40 0x7fc10015dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:03.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.714+0000 7fc105600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1001616e0 0x7fc1000cbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:03.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.714+0000 7fc105600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc1000b4910 con 0x7fc1000d2b40 2024-09-16T15:52:03.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.714+0000 7fc105600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc100161c60 con 0x7fc1001616e0 2024-09-16T15:52:03.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.714+0000 7fc105600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fc1000b4ae0 con 0x7fc1000d1ee0 2024-09-16T15:52:03.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.714+0000 7fc0fe000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1000d2b40 0x7fc10015dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:03.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.714+0000 7fc0fe000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1000d2b40 0x7fc10015dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41146/0 (socket says 172.21.15.151:41146) 2024-09-16T15:52:03.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.714+0000 7fc0fe000640 1 -- 172.21.15.151:0/2336448789 learned_addr learned my addr 172.21.15.151:0/2336448789 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:52:03.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.714+0000 7fc0ff400640 1 --2- 172.21.15.151:0/2336448789 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1001616e0 0x7fc1000cbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:03.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.714+0000 7fc0fea00640 1 --2- 172.21.15.151:0/2336448789 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1000d1ee0 0x7fc1000cdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:03.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.715+0000 7fc0fe000640 1 -- 172.21.15.151:0/2336448789 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1000d1ee0 msgr2=0x7fc1000cdf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:03.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.715+0000 7fc0fe000640 1 --2- 172.21.15.151:0/2336448789 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1000d1ee0 0x7fc1000cdf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:03.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.715+0000 7fc0fe000640 1 -- 172.21.15.151:0/2336448789 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1001616e0 msgr2=0x7fc1000cbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:03.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.715+0000 7fc0fe000640 1 --2- 172.21.15.151:0/2336448789 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1001616e0 0x7fc1000cbfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:03.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.715+0000 7fc0fe000640 1 -- 172.21.15.151:0/2336448789 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc10015ebe0 con 0x7fc1000d2b40 2024-09-16T15:52:03.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.715+0000 7fc0fe000640 1 --2- 172.21.15.151:0/2336448789 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1000d2b40 0x7fc10015dea0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec00c450 tx=0x7fc0ec00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:03.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.716+0000 7fc0f6a00640 1 -- 172.21.15.151:0/2336448789 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0ec016020 con 0x7fc1000d2b40 2024-09-16T15:52:03.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.716+0000 7fc0f6a00640 1 -- 172.21.15.151:0/2336448789 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc0ec012920 con 0x7fc1000d2b40 2024-09-16T15:52:03.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.716+0000 7fc0f6a00640 1 -- 172.21.15.151:0/2336448789 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0ec012d70 con 0x7fc1000d2b40 2024-09-16T15:52:03.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.716+0000 7fc105600640 1 -- 172.21.15.151:0/2336448789 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc10015e900 con 0x7fc1000d2b40 2024-09-16T15:52:03.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.716+0000 7fc105600640 1 -- 172.21.15.151:0/2336448789 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1000b7c30 con 0x7fc1000d2b40 2024-09-16T15:52:03.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.718+0000 7fc105600640 1 -- 172.21.15.151:0/2336448789 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1000b41c0 con 0x7fc1000d2b40 2024-09-16T15:52:03.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.721+0000 7fc0f6a00640 1 -- 172.21.15.151:0/2336448789 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 41) ==== 103684+0+0 (secure 0 0 0) 0x7fc0ec019020 con 0x7fc1000d2b40 2024-09-16T15:52:03.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.723+0000 7fc0f6a00640 1 --2- 172.21.15.151:0/2336448789 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc0e007a3c0 0x7fc0e007c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:03.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.723+0000 7fc0f6a00640 1 -- 172.21.15.151:0/2336448789 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5765+0+0 (secure 0 0 0) 0x7fc0ec099bd0 con 0x7fc1000d2b40 2024-09-16T15:52:03.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.723+0000 7fc0fea00640 1 --2- 172.21.15.151:0/2336448789 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc0e007a3c0 0x7fc0e007c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:03.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.725+0000 7fc0fea00640 1 --2- 172.21.15.151:0/2336448789 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc0e007a3c0 0x7fc0e007c880 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fc1000b5510 tx=0x7fc0f000b2e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:03.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.727+0000 7fc0f6a00640 1 -- 172.21.15.151:0/2336448789 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc0ec0636d0 con 0x7fc1000d2b40 2024-09-16T15:52:03.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.993+0000 7fc105600640 1 -- 172.21.15.151:0/2336448789 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc100074080 con 0x7fc1000d2b40 2024-09-16T15:52:03.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.994+0000 7fc0f6a00640 1 -- 172.21.15.151:0/2336448789 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5234 (secure 0 0 0) 0x7fc0ec068100 con 0x7fc1000d2b40 2024-09-16T15:52:03.995 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:52:03.995 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":67,"btime":"2024-09-16T15:51:15:447976+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6680,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2638998637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":2638998637},{"type":"v1","addr":"172.21.15.151:6835","nonce":2638998637}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":7330,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3105097249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3105097249},{"type":"v1","addr":"172.21.15.167:6836","nonce":3105097249}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":7333,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/613324294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":613324294},{"type":"v1","addr":"172.21.15.167:6837","nonce":613324294}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:51:01.789380+0000","modified":"2024-09-16T15:51:04.820629+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7899},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7899":{"gid":7899,"name":"c","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.151:6834/1174862515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1174862515},{"type":"v1","addr":"172.21.15.151:6834","nonce":1174862515}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7899,"qdb_cluster":[7899]},"id":7}]} 2024-09-16T15:52:03.995 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 67 2024-09-16T15:52:03.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.996+0000 7fc105600640 1 -- 172.21.15.151:0/2336448789 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc0e007a3c0 msgr2=0x7fc0e007c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:03.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.996+0000 7fc105600640 1 --2- 172.21.15.151:0/2336448789 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc0e007a3c0 0x7fc0e007c880 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fc1000b5510 tx=0x7fc0f000b2e0 comp rx=0 tx=0).stop 2024-09-16T15:52:03.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.996+0000 7fc105600640 1 -- 172.21.15.151:0/2336448789 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1000d2b40 msgr2=0x7fc10015dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:03.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.996+0000 7fc105600640 1 --2- 172.21.15.151:0/2336448789 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1000d2b40 0x7fc10015dea0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec00c450 tx=0x7fc0ec00c910 comp rx=0 tx=0).stop 2024-09-16T15:52:03.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.996+0000 7fc105600640 1 -- 172.21.15.151:0/2336448789 shutdown_connections 2024-09-16T15:52:03.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.996+0000 7fc105600640 1 --2- 172.21.15.151:0/2336448789 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc0e007a3c0 0x7fc0e007c880 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:03.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.996+0000 7fc105600640 1 --2- 172.21.15.151:0/2336448789 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1001616e0 0x7fc1000cbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:03.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.996+0000 7fc105600640 1 --2- 172.21.15.151:0/2336448789 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1000d2b40 0x7fc10015dea0 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:03.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.996+0000 7fc105600640 1 --2- 172.21.15.151:0/2336448789 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc1000d1ee0 0x7fc1000cdf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:03.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.996+0000 7fc105600640 1 -- 172.21.15.151:0/2336448789 >> 172.21.15.151:0/2336448789 conn(0x7fc1000d3fe0 msgr2=0x7fc10006da70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:03.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.996+0000 7fc105600640 1 -- 172.21.15.151:0/2336448789 shutdown_connections 2024-09-16T15:52:03.999 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:03.996+0000 7fc105600640 1 -- 172.21.15.151:0/2336448789 wait complete. 2024-09-16T15:52:04.014 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:52:04.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.170+0000 7f7355400640 1 -- 172.21.15.151:0/1530067309 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73500725d0 msgr2=0x7f73500b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:04.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.170+0000 7f7355400640 1 --2- 172.21.15.151:0/1530067309 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73500725d0 0x7f73500b28f0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f733c005fd0 tx=0x7f733c02a5d0 comp rx=0 tx=0).stop 2024-09-16T15:52:04.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.170+0000 7f7355400640 1 -- 172.21.15.151:0/1530067309 shutdown_connections 2024-09-16T15:52:04.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.170+0000 7f7355400640 1 --2- 172.21.15.151:0/1530067309 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73500725d0 0x7f73500b28f0 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:04.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.170+0000 7f7355400640 1 -- 172.21.15.151:0/1530067309 >> 172.21.15.151:0/1530067309 conn(0x7f73500cfa20 msgr2=0x7f73500cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:04.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.170+0000 7f7355400640 1 -- 172.21.15.151:0/1530067309 shutdown_connections 2024-09-16T15:52:04.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.170+0000 7f7355400640 1 -- 172.21.15.151:0/1530067309 wait complete. 2024-09-16T15:52:04.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.172+0000 7f7355400640 1 Processor -- start 2024-09-16T15:52:04.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.172+0000 7f7355400640 1 -- start start 2024-09-16T15:52:04.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.173+0000 7f7355400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73500d0540 0x7f73500c90c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:04.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.173+0000 7f7355400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f735015bc30 0x7f73500af640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:04.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.173+0000 7f7355400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73500afb80 0x7f73501f8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:04.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.173+0000 7f7355400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7350074660 con 0x7f73500afb80 2024-09-16T15:52:04.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.173+0000 7f7355400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f735006fa30 con 0x7f735015bc30 2024-09-16T15:52:04.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.173+0000 7f7355400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f73501f47d0 con 0x7f73500d0540 2024-09-16T15:52:04.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.173+0000 7f7354a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73500afb80 0x7f73501f8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:04.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.173+0000 7f7354a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73500afb80 0x7f73501f8080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41156/0 (socket says 172.21.15.151:41156) 2024-09-16T15:52:04.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.173+0000 7f7354a00640 1 -- 172.21.15.151:0/1423679935 learned_addr learned my addr 172.21.15.151:0/1423679935 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:52:04.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.173+0000 7f734f400640 1 --2- 172.21.15.151:0/1423679935 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f735015bc30 0x7f73500af640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:04.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.173+0000 7f734fe00640 1 --2- 172.21.15.151:0/1423679935 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73500d0540 0x7f73500c90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:04.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.173+0000 7f7354a00640 1 -- 172.21.15.151:0/1423679935 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73500d0540 msgr2=0x7f73500c90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:04.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.173+0000 7f7354a00640 1 --2- 172.21.15.151:0/1423679935 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73500d0540 0x7f73500c90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:04.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.173+0000 7f7354a00640 1 -- 172.21.15.151:0/1423679935 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f735015bc30 msgr2=0x7f73500af640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:04.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.173+0000 7f7354a00640 1 --2- 172.21.15.151:0/1423679935 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f735015bc30 0x7f73500af640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:04.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.173+0000 7f7354a00640 1 -- 172.21.15.151:0/1423679935 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73500b0030 con 0x7f73500afb80 2024-09-16T15:52:04.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.174+0000 7f7354a00640 1 --2- 172.21.15.151:0/1423679935 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73500afb80 0x7f73501f8080 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f734800c410 tx=0x7f734800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:04.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.174+0000 7f7347e00640 1 -- 172.21.15.151:0/1423679935 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7348011790 con 0x7f73500afb80 2024-09-16T15:52:04.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.174+0000 7f7347e00640 1 -- 172.21.15.151:0/1423679935 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7348017070 con 0x7f73500afb80 2024-09-16T15:52:04.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.174+0000 7f7355400640 1 -- 172.21.15.151:0/1423679935 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73501f4a50 con 0x7f73500afb80 2024-09-16T15:52:04.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.174+0000 7f7347e00640 1 -- 172.21.15.151:0/1423679935 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7348011d00 con 0x7f73500afb80 2024-09-16T15:52:04.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.174+0000 7f7355400640 1 -- 172.21.15.151:0/1423679935 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73501f5010 con 0x7f73500afb80 2024-09-16T15:52:04.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.175+0000 7f7355400640 1 -- 172.21.15.151:0/1423679935 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73500b28f0 con 0x7f73500afb80 2024-09-16T15:52:04.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.176+0000 7f7347e00640 1 -- 172.21.15.151:0/1423679935 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 41) ==== 103684+0+0 (secure 0 0 0) 0x7f734801c460 con 0x7f73500afb80 2024-09-16T15:52:04.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.177+0000 7f7347e00640 1 --2- 172.21.15.151:0/1423679935 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f733007a2c0 0x7f733007c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:04.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.177+0000 7f7347e00640 1 -- 172.21.15.151:0/1423679935 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5821+0+0 (secure 0 0 0) 0x7f7348099480 con 0x7f73500afb80 2024-09-16T15:52:04.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.178+0000 7f734fe00640 1 --2- 172.21.15.151:0/1423679935 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f733007a2c0 0x7f733007c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:04.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.179+0000 7f734fe00640 1 --2- 172.21.15.151:0/1423679935 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f733007a2c0 0x7f733007c780 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f73500c8de0 tx=0x7f7340009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:04.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.181+0000 7f7347e00640 1 -- 172.21.15.151:0/1423679935 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7348062f00 con 0x7f73500afb80 2024-09-16T15:52:04.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.368+0000 7f7355400640 1 -- 172.21.15.151:0/1423679935 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f735006e840 con 0x7f73500afb80 2024-09-16T15:52:04.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.369+0000 7f7347e00640 1 -- 172.21.15.151:0/1423679935 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) ==== 74+0+16938 (secure 0 0 0) 0x7f7348067930 con 0x7f73500afb80 2024-09-16T15:52:04.371 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:52:04.371 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":105,"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","created":"2024-09-16T15:38:56.348535+0000","modified":"2024-09-16T15:52:03.741443+0000","last_up_change":"2024-09-16T15:39:10.681384+0000","last_in_change":"2024-09-16T15:39:00.544477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:39:11.602252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:50:59.568133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-16T15:51:00.775162+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":105,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5524076-a903-4d0a-af6a-caf474c4765d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6809","nonce":3111342741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6811","nonce":3111342741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6815","nonce":3111342741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6813","nonce":3111342741}]},"public_addr":"172.21.15.151:6809/3111342741","cluster_addr":"172.21.15.151:6811/3111342741","heartbeat_back_addr":"172.21.15.151:6815/3111342741","heartbeat_front_addr":"172.21.15.151:6813/3111342741","state":["exists","up"]},{"osd":1,"uuid":"dd25925f-0ae1-494e-8529-4dc205759db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6801","nonce":3368304087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6803","nonce":3368304087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6807","nonce":3368304087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6805","nonce":3368304087}]},"public_addr":"172.21.15.151:6801/3368304087","cluster_addr":"172.21.15.151:6803/3368304087","heartbeat_back_addr":"172.21.15.151:6807/3368304087","heartbeat_front_addr":"172.21.15.151:6805/3368304087","state":["exists","up"]},{"osd":2,"uuid":"c2f18115-013d-4da6-a0fe-cb978f4494b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6823","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6825","nonce":3079373271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6827","nonce":3079373271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6831","nonce":3079373271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6829","nonce":3079373271}]},"public_addr":"172.21.15.151:6825/3079373271","cluster_addr":"172.21.15.151:6827/3079373271","heartbeat_back_addr":"172.21.15.151:6831/3079373271","heartbeat_front_addr":"172.21.15.151:6829/3079373271","state":["exists","up"]},{"osd":3,"uuid":"36edb718-2fcb-4d88-8ddc-c321eed4279d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6817","nonce":1142509041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6819","nonce":1142509041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6824","nonce":1142509041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6821","nonce":1142509041}]},"public_addr":"172.21.15.151:6817/1142509041","cluster_addr":"172.21.15.151:6819/1142509041","heartbeat_back_addr":"172.21.15.151:6824/1142509041","heartbeat_front_addr":"172.21.15.151:6821/1142509041","state":["exists","up"]},{"osd":4,"uuid":"64d33708-cd0a-4236-abe3-1b646e82ab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6811","nonce":2735495271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6813","nonce":2735495271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6817","nonce":2735495271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6815","nonce":2735495271}]},"public_addr":"172.21.15.167:6811/2735495271","cluster_addr":"172.21.15.167:6813/2735495271","heartbeat_back_addr":"172.21.15.167:6817/2735495271","heartbeat_front_addr":"172.21.15.167:6815/2735495271","state":["exists","up"]},{"osd":5,"uuid":"f14b44a9-5910-458c-b518-09a8d0de94de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6827","nonce":1494916572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6829","nonce":1494916572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6833","nonce":1494916572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6831","nonce":1494916572}]},"public_addr":"172.21.15.167:6827/1494916572","cluster_addr":"172.21.15.167:6829/1494916572","heartbeat_back_addr":"172.21.15.167:6833/1494916572","heartbeat_front_addr":"172.21.15.167:6831/1494916572","state":["exists","up"]},{"osd":6,"uuid":"39335545-2654-4710-84b0-a9aa47cfcbba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6819","nonce":2538273273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6821","nonce":2538273273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6825","nonce":2538273273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6823","nonce":2538273273}]},"public_addr":"172.21.15.167:6819/2538273273","cluster_addr":"172.21.15.167:6821/2538273273","heartbeat_back_addr":"172.21.15.167:6825/2538273273","heartbeat_front_addr":"172.21.15.167:6823/2538273273","state":["exists","up"]},{"osd":7,"uuid":"4f8bfc81-53dd-4339-ac22-1a4007d9f963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6803","nonce":265072227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6805","nonce":265072227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6809","nonce":265072227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6807","nonce":265072227}]},"public_addr":"172.21.15.167:6803/265072227","cluster_addr":"172.21.15.167:6805/265072227","heartbeat_back_addr":"172.21.15.167:6809/265072227","heartbeat_front_addr":"172.21.15.167:6807/265072227","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.150359+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.084358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.722368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.332809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.320709+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.395153+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.935250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.801443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:52:04.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.372+0000 7f7355400640 1 -- 172.21.15.151:0/1423679935 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f733007a2c0 msgr2=0x7f733007c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:04.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.372+0000 7f7355400640 1 --2- 172.21.15.151:0/1423679935 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f733007a2c0 0x7f733007c780 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f73500c8de0 tx=0x7f7340009040 comp rx=0 tx=0).stop 2024-09-16T15:52:04.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.372+0000 7f7355400640 1 -- 172.21.15.151:0/1423679935 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73500afb80 msgr2=0x7f73501f8080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:04.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.372+0000 7f7355400640 1 --2- 172.21.15.151:0/1423679935 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73500afb80 0x7f73501f8080 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f734800c410 tx=0x7f734800c8d0 comp rx=0 tx=0).stop 2024-09-16T15:52:04.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.372+0000 7f7355400640 1 -- 172.21.15.151:0/1423679935 shutdown_connections 2024-09-16T15:52:04.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.372+0000 7f7355400640 1 --2- 172.21.15.151:0/1423679935 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f733007a2c0 0x7f733007c780 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:04.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.372+0000 7f7355400640 1 --2- 172.21.15.151:0/1423679935 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f735015bc30 0x7f73500af640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:04.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.372+0000 7f7355400640 1 --2- 172.21.15.151:0/1423679935 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f73500afb80 0x7f73501f8080 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:04.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.372+0000 7f7355400640 1 --2- 172.21.15.151:0/1423679935 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f73500d0540 0x7f73500c90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:04.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.372+0000 7f7355400640 1 -- 172.21.15.151:0/1423679935 >> 172.21.15.151:0/1423679935 conn(0x7f73500cfa20 msgr2=0x7f73500cef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:04.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.372+0000 7f7355400640 1 -- 172.21.15.151:0/1423679935 shutdown_connections 2024-09-16T15:52:04.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.372+0000 7f7355400640 1 -- 172.21.15.151:0/1423679935 wait complete. 2024-09-16T15:52:04.389 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-16T15:52:04.389 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:52:04.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.546+0000 7fd071400640 1 -- 172.21.15.151:0/497872903 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd06c15f5a0 msgr2=0x7fd06c0b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:04.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.546+0000 7fd071400640 1 --2- 172.21.15.151:0/497872903 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd06c15f5a0 0x7fd06c0b2440 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fd06c06dac0 tx=0x7fd05c02a560 comp rx=0 tx=0).stop 2024-09-16T15:52:04.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.546+0000 7fd071400640 1 -- 172.21.15.151:0/497872903 shutdown_connections 2024-09-16T15:52:04.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.546+0000 7fd071400640 1 --2- 172.21.15.151:0/497872903 >> v2:172.21.15.167:3300/0 conn(0x7fd06c073fb0 0x7fd06c074420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:04.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.546+0000 7fd071400640 1 --2- 172.21.15.151:0/497872903 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd06c15f5a0 0x7fd06c0b2440 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:04.548 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.546+0000 7fd071400640 1 -- 172.21.15.151:0/497872903 >> 172.21.15.151:0/497872903 conn(0x7fd06c0b1950 msgr2=0x7fd06c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:04.548 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.546+0000 7fd071400640 1 -- 172.21.15.151:0/497872903 shutdown_connections 2024-09-16T15:52:04.548 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.546+0000 7fd071400640 1 -- 172.21.15.151:0/497872903 wait complete. 2024-09-16T15:52:04.549 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.548+0000 7fd071400640 1 Processor -- start 2024-09-16T15:52:04.549 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.548+0000 7fd071400640 1 -- start start 2024-09-16T15:52:04.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.549+0000 7fd071400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd06c073fb0 0x7fd06c1f5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:04.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.549+0000 7fd071400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd06c0ccf00 0x7fd06c147540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:04.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.549+0000 7fd071400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd06c15f5a0 0x7fd06c0c9f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:04.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.549+0000 7fd071400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd06c06f2e0 con 0x7fd06c073fb0 2024-09-16T15:52:04.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.549+0000 7fd071400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd06c06f160 con 0x7fd06c15f5a0 2024-09-16T15:52:04.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.549+0000 7fd071400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fd06c06f460 con 0x7fd06c0ccf00 2024-09-16T15:52:04.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.549+0000 7fd06a800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd06c073fb0 0x7fd06c1f5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:04.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.549+0000 7fd06a800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd06c073fb0 0x7fd06c1f5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41174/0 (socket says 172.21.15.151:41174) 2024-09-16T15:52:04.551 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.549+0000 7fd06a800640 1 -- 172.21.15.151:0/3136662586 learned_addr learned my addr 172.21.15.151:0/3136662586 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:52:04.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.549+0000 7fd069e00640 1 --2- 172.21.15.151:0/3136662586 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd06c0ccf00 0x7fd06c147540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:04.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.549+0000 7fd06b200640 1 --2- 172.21.15.151:0/3136662586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd06c15f5a0 0x7fd06c0c9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:04.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.549+0000 7fd06a800640 1 -- 172.21.15.151:0/3136662586 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd06c0ccf00 msgr2=0x7fd06c147540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:04.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.549+0000 7fd06a800640 1 --2- 172.21.15.151:0/3136662586 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd06c0ccf00 0x7fd06c147540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:04.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.549+0000 7fd06a800640 1 -- 172.21.15.151:0/3136662586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd06c15f5a0 msgr2=0x7fd06c0c9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:04.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.549+0000 7fd06a800640 1 --2- 172.21.15.151:0/3136662586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd06c15f5a0 0x7fd06c0c9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:04.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.549+0000 7fd06a800640 1 -- 172.21.15.151:0/3136662586 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd06c0b2f60 con 0x7fd06c073fb0 2024-09-16T15:52:04.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.550+0000 7fd06a800640 1 --2- 172.21.15.151:0/3136662586 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd06c073fb0 0x7fd06c1f5340 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fd05c034040 tx=0x7fd05c003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:04.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.550+0000 7fd062a00640 1 -- 172.21.15.151:0/3136662586 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05c03e030 con 0x7fd06c073fb0 2024-09-16T15:52:04.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.550+0000 7fd062a00640 1 -- 172.21.15.151:0/3136662586 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd05c003e00 con 0x7fd06c073fb0 2024-09-16T15:52:04.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.550+0000 7fd071400640 1 -- 172.21.15.151:0/3136662586 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd06c0b3240 con 0x7fd06c073fb0 2024-09-16T15:52:04.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.550+0000 7fd062a00640 1 -- 172.21.15.151:0/3136662586 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05c035040 con 0x7fd06c073fb0 2024-09-16T15:52:04.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.550+0000 7fd071400640 1 -- 172.21.15.151:0/3136662586 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd06c0b3800 con 0x7fd06c073fb0 2024-09-16T15:52:04.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.551+0000 7fd071400640 1 -- 172.21.15.151:0/3136662586 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd06c06f9f0 con 0x7fd06c073fb0 2024-09-16T15:52:04.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.552+0000 7fd062a00640 1 -- 172.21.15.151:0/3136662586 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 41) ==== 103684+0+0 (secure 0 0 0) 0x7fd05c041030 con 0x7fd06c073fb0 2024-09-16T15:52:04.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.554+0000 7fd062a00640 1 --2- 172.21.15.151:0/3136662586 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd04c07a2c0 0x7fd04c07c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:04.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.554+0000 7fd062a00640 1 -- 172.21.15.151:0/3136662586 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5821+0+0 (secure 0 0 0) 0x7fd05c0b9fb0 con 0x7fd06c073fb0 2024-09-16T15:52:04.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.554+0000 7fd069e00640 1 --2- 172.21.15.151:0/3136662586 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd04c07a2c0 0x7fd04c07c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:04.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.555+0000 7fd069e00640 1 --2- 172.21.15.151:0/3136662586 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd04c07a2c0 0x7fd04c07c780 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fd06c09df60 tx=0x7fd058009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:04.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.558+0000 7fd062a00640 1 -- 172.21.15.151:0/3136662586 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd05c084000 con 0x7fd06c073fb0 2024-09-16T15:52:04.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.785+0000 7fd071400640 1 -- 172.21.15.151:0/3136662586 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd06c1556d0 con 0x7fd06c073fb0 2024-09-16T15:52:04.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.786+0000 7fd062a00640 1 -- 172.21.15.151:0/3136662586 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5234 (secure 0 0 0) 0x7fd05c083900 con 0x7fd06c073fb0 2024-09-16T15:52:04.787 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:52:04.787 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":67,"btime":"2024-09-16T15:51:15:447976+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6680,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2638998637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":2638998637},{"type":"v1","addr":"172.21.15.151:6835","nonce":2638998637}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":7330,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3105097249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3105097249},{"type":"v1","addr":"172.21.15.167:6836","nonce":3105097249}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":7333,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/613324294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":613324294},{"type":"v1","addr":"172.21.15.167:6837","nonce":613324294}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:51:01.789380+0000","modified":"2024-09-16T15:51:04.820629+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7899},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7899":{"gid":7899,"name":"c","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.151:6834/1174862515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1174862515},{"type":"v1","addr":"172.21.15.151:6834","nonce":1174862515}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7899,"qdb_cluster":[7899]},"id":7}]} 2024-09-16T15:52:04.788 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 67 2024-09-16T15:52:04.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.789+0000 7fd071400640 1 -- 172.21.15.151:0/3136662586 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd04c07a2c0 msgr2=0x7fd04c07c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:04.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.789+0000 7fd071400640 1 --2- 172.21.15.151:0/3136662586 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd04c07a2c0 0x7fd04c07c780 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fd06c09df60 tx=0x7fd058009040 comp rx=0 tx=0).stop 2024-09-16T15:52:04.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.789+0000 7fd071400640 1 -- 172.21.15.151:0/3136662586 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd06c073fb0 msgr2=0x7fd06c1f5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:04.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.789+0000 7fd071400640 1 --2- 172.21.15.151:0/3136662586 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd06c073fb0 0x7fd06c1f5340 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fd05c034040 tx=0x7fd05c003910 comp rx=0 tx=0).stop 2024-09-16T15:52:04.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.789+0000 7fd071400640 1 -- 172.21.15.151:0/3136662586 shutdown_connections 2024-09-16T15:52:04.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.789+0000 7fd071400640 1 --2- 172.21.15.151:0/3136662586 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd04c07a2c0 0x7fd04c07c780 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:04.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.789+0000 7fd071400640 1 --2- 172.21.15.151:0/3136662586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd06c15f5a0 0x7fd06c0c9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:04.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.789+0000 7fd071400640 1 --2- 172.21.15.151:0/3136662586 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd06c0ccf00 0x7fd06c147540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:04.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.789+0000 7fd071400640 1 --2- 172.21.15.151:0/3136662586 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd06c073fb0 0x7fd06c1f5340 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:04.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.789+0000 7fd071400640 1 -- 172.21.15.151:0/3136662586 >> 172.21.15.151:0/3136662586 conn(0x7fd06c0b1950 msgr2=0x7fd06c0cf000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:04.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.789+0000 7fd071400640 1 -- 172.21.15.151:0/3136662586 shutdown_connections 2024-09-16T15:52:04.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.789+0000 7fd071400640 1 -- 172.21.15.151:0/3136662586 wait complete. 2024-09-16T15:52:04.807 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:52:04.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.964+0000 7f5ce4a00640 1 -- 172.21.15.151:0/1762261868 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ce0073fb0 msgr2=0x7f5ce0074460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:04.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.964+0000 7f5ce4a00640 1 --2- 172.21.15.151:0/1762261868 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ce0073fb0 0x7f5ce0074460 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f5cd8005fd0 tx=0x7f5cd802aa10 comp rx=0 tx=0).stop 2024-09-16T15:52:04.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.964+0000 7f5ce4a00640 1 -- 172.21.15.151:0/1762261868 shutdown_connections 2024-09-16T15:52:04.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.964+0000 7f5ce4a00640 1 --2- 172.21.15.151:0/1762261868 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ce0073fb0 0x7f5ce0074460 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:04.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.964+0000 7f5ce4a00640 1 --2- 172.21.15.151:0/1762261868 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5ce015f5a0 0x7f5ce00b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:04.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.964+0000 7f5ce4a00640 1 -- 172.21.15.151:0/1762261868 >> 172.21.15.151:0/1762261868 conn(0x7f5ce00b1950 msgr2=0x7f5ce00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:04.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.964+0000 7f5ce4a00640 1 -- 172.21.15.151:0/1762261868 shutdown_connections 2024-09-16T15:52:04.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.964+0000 7f5ce4a00640 1 -- 172.21.15.151:0/1762261868 wait complete. 2024-09-16T15:52:04.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.966+0000 7f5ce4a00640 1 Processor -- start 2024-09-16T15:52:04.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.966+0000 7f5ce4a00640 1 -- start start 2024-09-16T15:52:04.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.967+0000 7f5ce4a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ce0073fb0 0x7f5ce0149690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:04.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.967+0000 7f5ce4a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5ce00ccf00 0x7f5ce00d03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:04.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.967+0000 7f5ce4a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ce015f5a0 0x7f5ce009ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:04.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.967+0000 7f5ce4a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce006f2e0 con 0x7f5ce0073fb0 2024-09-16T15:52:04.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.967+0000 7f5ce4a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce006f160 con 0x7f5ce015f5a0 2024-09-16T15:52:04.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.967+0000 7f5ce4a00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f5ce006f460 con 0x7f5ce00ccf00 2024-09-16T15:52:04.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.967+0000 7f5cdde00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ce0073fb0 0x7f5ce0149690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:04.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.967+0000 7f5cdde00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ce0073fb0 0x7f5ce0149690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41178/0 (socket says 172.21.15.151:41178) 2024-09-16T15:52:04.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.967+0000 7f5cdde00640 1 -- 172.21.15.151:0/2112450899 learned_addr learned my addr 172.21.15.151:0/2112450899 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:52:04.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.967+0000 7f5cdd400640 1 --2- 172.21.15.151:0/2112450899 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5ce00ccf00 0x7f5ce00d03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:04.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.967+0000 7f5cde800640 1 --2- 172.21.15.151:0/2112450899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ce015f5a0 0x7f5ce009ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:04.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.967+0000 7f5cdde00640 1 -- 172.21.15.151:0/2112450899 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5ce00ccf00 msgr2=0x7f5ce00d03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:04.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.967+0000 7f5cdde00640 1 --2- 172.21.15.151:0/2112450899 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5ce00ccf00 0x7f5ce00d03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:04.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.967+0000 7f5cdde00640 1 -- 172.21.15.151:0/2112450899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ce015f5a0 msgr2=0x7f5ce009ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:04.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.967+0000 7f5cdde00640 1 --2- 172.21.15.151:0/2112450899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ce015f5a0 0x7f5ce009ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:04.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.967+0000 7f5cdde00640 1 -- 172.21.15.151:0/2112450899 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ce0149bd0 con 0x7f5ce0073fb0 2024-09-16T15:52:04.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.968+0000 7f5cdde00640 1 --2- 172.21.15.151:0/2112450899 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ce0073fb0 0x7f5ce0149690 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f5cd000e790 tx=0x7f5cd000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:04.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.968+0000 7f5cd6000640 1 -- 172.21.15.151:0/2112450899 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd000eea0 con 0x7f5ce0073fb0 2024-09-16T15:52:04.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.968+0000 7f5cd6000640 1 -- 172.21.15.151:0/2112450899 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5cd000cdb0 con 0x7f5ce0073fb0 2024-09-16T15:52:04.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.968+0000 7f5ce4a00640 1 -- 172.21.15.151:0/2112450899 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ce01f82b0 con 0x7f5ce0073fb0 2024-09-16T15:52:04.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.968+0000 7f5cd6000640 1 -- 172.21.15.151:0/2112450899 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd0010690 con 0x7f5ce0073fb0 2024-09-16T15:52:04.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.968+0000 7f5ce4a00640 1 -- 172.21.15.151:0/2112450899 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ce01f8870 con 0x7f5ce0073fb0 2024-09-16T15:52:04.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.969+0000 7f5ce4a00640 1 -- 172.21.15.151:0/2112450899 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ce006f9f0 con 0x7f5ce0073fb0 2024-09-16T15:52:04.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.970+0000 7f5cd6000640 1 -- 172.21.15.151:0/2112450899 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 41) ==== 103684+0+0 (secure 0 0 0) 0x7f5cd0010830 con 0x7f5ce0073fb0 2024-09-16T15:52:04.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.972+0000 7f5cd6000640 1 --2- 172.21.15.151:0/2112450899 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5cc007a2c0 0x7f5cc007c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:04.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.972+0000 7f5cd6000640 1 -- 172.21.15.151:0/2112450899 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5909+0+0 (secure 0 0 0) 0x7f5cd009a160 con 0x7f5ce0073fb0 2024-09-16T15:52:04.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.972+0000 7f5cdd400640 1 --2- 172.21.15.151:0/2112450899 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5cc007a2c0 0x7f5cc007c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:04.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.973+0000 7f5cdd400640 1 --2- 172.21.15.151:0/2112450899 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5cc007a2c0 0x7f5cc007c780 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f5ce01f4b60 tx=0x7f5ccc009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:04.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:04.976+0000 7f5cd6000640 1 -- 172.21.15.151:0/2112450899 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f5ce0149bd0 con 0x7f5ce0073fb0 2024-09-16T15:52:05.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.199+0000 7f5ce4a00640 1 -- 172.21.15.151:0/2112450899 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5ce015dea0 con 0x7f5ce0073fb0 2024-09-16T15:52:05.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.199+0000 7f5cd6000640 1 -- 172.21.15.151:0/2112450899 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5234 (secure 0 0 0) 0x7f5cd0063be0 con 0x7f5ce0073fb0 2024-09-16T15:52:05.200 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:52:05.201 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":67,"btime":"2024-09-16T15:51:15:447976+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6680,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2638998637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":2638998637},{"type":"v1","addr":"172.21.15.151:6835","nonce":2638998637}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":7330,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3105097249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3105097249},{"type":"v1","addr":"172.21.15.167:6836","nonce":3105097249}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":7333,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/613324294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":613324294},{"type":"v1","addr":"172.21.15.167:6837","nonce":613324294}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:51:01.789380+0000","modified":"2024-09-16T15:51:04.820629+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7899},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7899":{"gid":7899,"name":"c","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.151:6834/1174862515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1174862515},{"type":"v1","addr":"172.21.15.151:6834","nonce":1174862515}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7899,"qdb_cluster":[7899]},"id":7}]} 2024-09-16T15:52:05.201 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 67 2024-09-16T15:52:05.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.202+0000 7f5ce4a00640 1 -- 172.21.15.151:0/2112450899 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5cc007a2c0 msgr2=0x7f5cc007c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:05.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.202+0000 7f5ce4a00640 1 --2- 172.21.15.151:0/2112450899 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5cc007a2c0 0x7f5cc007c780 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f5ce01f4b60 tx=0x7f5ccc009040 comp rx=0 tx=0).stop 2024-09-16T15:52:05.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.202+0000 7f5ce4a00640 1 -- 172.21.15.151:0/2112450899 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ce0073fb0 msgr2=0x7f5ce0149690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:05.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.202+0000 7f5ce4a00640 1 --2- 172.21.15.151:0/2112450899 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ce0073fb0 0x7f5ce0149690 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f5cd000e790 tx=0x7f5cd000ec50 comp rx=0 tx=0).stop 2024-09-16T15:52:05.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.202+0000 7f5ce4a00640 1 -- 172.21.15.151:0/2112450899 shutdown_connections 2024-09-16T15:52:05.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.202+0000 7f5ce4a00640 1 --2- 172.21.15.151:0/2112450899 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f5cc007a2c0 0x7f5cc007c780 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:05.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.202+0000 7f5ce4a00640 1 --2- 172.21.15.151:0/2112450899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ce015f5a0 0x7f5ce009ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:05.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.202+0000 7f5ce4a00640 1 --2- 172.21.15.151:0/2112450899 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5ce00ccf00 0x7f5ce00d03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:05.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.202+0000 7f5ce4a00640 1 --2- 172.21.15.151:0/2112450899 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ce0073fb0 0x7f5ce0149690 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:05.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.202+0000 7f5ce4a00640 1 -- 172.21.15.151:0/2112450899 >> 172.21.15.151:0/2112450899 conn(0x7f5ce00b1950 msgr2=0x7f5ce015e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:05.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.202+0000 7f5ce4a00640 1 -- 172.21.15.151:0/2112450899 shutdown_connections 2024-09-16T15:52:05.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.202+0000 7f5ce4a00640 1 -- 172.21.15.151:0/2112450899 wait complete. 2024-09-16T15:52:05.219 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:52:05.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.376+0000 7f76b6600640 1 -- 172.21.15.151:0/1903766428 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f76b00ccf00 msgr2=0x7f76b00b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:05.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.376+0000 7f76b6600640 1 --2- 172.21.15.151:0/1903766428 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f76b00ccf00 0x7f76b00b2440 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f769c005fd0 tx=0x7f769c02a5f0 comp rx=0 tx=0).stop 2024-09-16T15:52:05.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.376+0000 7f76b6600640 1 -- 172.21.15.151:0/1903766428 shutdown_connections 2024-09-16T15:52:05.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.376+0000 7f76b6600640 1 --2- 172.21.15.151:0/1903766428 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f76b00ccf00 0x7f76b00b2440 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:05.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.376+0000 7f76b6600640 1 -- 172.21.15.151:0/1903766428 >> 172.21.15.151:0/1903766428 conn(0x7f76b00b1950 msgr2=0x7f76b00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:05.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.376+0000 7f76b6600640 1 -- 172.21.15.151:0/1903766428 shutdown_connections 2024-09-16T15:52:05.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.376+0000 7f76b6600640 1 -- 172.21.15.151:0/1903766428 wait complete. 2024-09-16T15:52:05.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.378+0000 7f76b6600640 1 Processor -- start 2024-09-16T15:52:05.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.378+0000 7f76b6600640 1 -- start start 2024-09-16T15:52:05.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.379+0000 7f76b6600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76b0073fb0 0x7f76b00b0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:05.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.379+0000 7f76b6600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f76b00ccf00 0x7f76b015ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:05.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.379+0000 7f76b6600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f76b015f5a0 0x7f76b00b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:05.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.379+0000 7f76b6600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f76b006f9f0 con 0x7f76b015f5a0 2024-09-16T15:52:05.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.379+0000 7f76b6600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f76b006f870 con 0x7f76b0073fb0 2024-09-16T15:52:05.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.379+0000 7f76b6600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f76b0074420 con 0x7f76b00ccf00 2024-09-16T15:52:05.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.379+0000 7f76afe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f76b015f5a0 0x7f76b00b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:05.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.379+0000 7f76afe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f76b015f5a0 0x7f76b00b2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41198/0 (socket says 172.21.15.151:41198) 2024-09-16T15:52:05.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.379+0000 7f76afe00640 1 -- 172.21.15.151:0/662839107 learned_addr learned my addr 172.21.15.151:0/662839107 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:52:05.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.379+0000 7f76aea00640 1 --2- 172.21.15.151:0/662839107 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f76b00ccf00 0x7f76b015ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:05.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.379+0000 7f76af400640 1 --2- 172.21.15.151:0/662839107 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76b0073fb0 0x7f76b00b0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:05.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.380+0000 7f76afe00640 1 -- 172.21.15.151:0/662839107 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f76b00ccf00 msgr2=0x7f76b015ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:05.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.380+0000 7f76afe00640 1 --2- 172.21.15.151:0/662839107 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f76b00ccf00 0x7f76b015ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:05.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.380+0000 7f76afe00640 1 -- 172.21.15.151:0/662839107 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76b0073fb0 msgr2=0x7f76b00b0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:05.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.380+0000 7f76afe00640 1 --2- 172.21.15.151:0/662839107 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76b0073fb0 0x7f76b00b0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:05.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.380+0000 7f76afe00640 1 -- 172.21.15.151:0/662839107 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76b015ac80 con 0x7f76b015f5a0 2024-09-16T15:52:05.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.380+0000 7f76afe00640 1 --2- 172.21.15.151:0/662839107 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f76b015f5a0 0x7f76b00b2d50 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f76a800c7a0 tx=0x7f76a800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:05.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.381+0000 7f76a7400640 1 -- 172.21.15.151:0/662839107 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a8003b70 con 0x7f76b015f5a0 2024-09-16T15:52:05.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.381+0000 7f76a7400640 1 -- 172.21.15.151:0/662839107 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f76a8003d10 con 0x7f76b015f5a0 2024-09-16T15:52:05.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.381+0000 7f76a7400640 1 -- 172.21.15.151:0/662839107 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a800aab0 con 0x7f76b015f5a0 2024-09-16T15:52:05.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.381+0000 7f76b6600640 1 -- 172.21.15.151:0/662839107 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76b015af60 con 0x7f76b015f5a0 2024-09-16T15:52:05.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.381+0000 7f76b6600640 1 -- 172.21.15.151:0/662839107 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76b015b520 con 0x7f76b015f5a0 2024-09-16T15:52:05.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.382+0000 7f76b6600640 1 -- 172.21.15.151:0/662839107 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76b00b2440 con 0x7f76b015f5a0 2024-09-16T15:52:05.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.383+0000 7f76a7400640 1 -- 172.21.15.151:0/662839107 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 41) ==== 103684+0+0 (secure 0 0 0) 0x7f76a800ac50 con 0x7f76b015f5a0 2024-09-16T15:52:05.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.384+0000 7f76a7400640 1 --2- 172.21.15.151:0/662839107 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f769007a2c0 0x7f769007c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:05.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.384+0000 7f76a7400640 1 -- 172.21.15.151:0/662839107 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5909+0+0 (secure 0 0 0) 0x7f76a8099540 con 0x7f76b015f5a0 2024-09-16T15:52:05.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.384+0000 7f76af400640 1 --2- 172.21.15.151:0/662839107 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f769007a2c0 0x7f769007c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:05.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.385+0000 7f76af400640 1 --2- 172.21.15.151:0/662839107 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f769007a2c0 0x7f769007c780 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f76a0000f60 tx=0x7f76a0009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:05.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.388+0000 7f76a7400640 1 -- 172.21.15.151:0/662839107 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f76a8062fc0 con 0x7f76b015f5a0 2024-09-16T15:52:05.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.574+0000 7f76b6600640 1 -- 172.21.15.151:0/662839107 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f76b009ddf0 con 0x7f76b015f5a0 2024-09-16T15:52:05.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.575+0000 7f76a7400640 1 -- 172.21.15.151:0/662839107 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) ==== 74+0+17086 (secure 0 0 0) 0x7f76a80679f0 con 0x7f76b015f5a0 2024-09-16T15:52:05.577 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:52:05.577 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":106,"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","created":"2024-09-16T15:38:56.348535+0000","modified":"2024-09-16T15:52:04.744438+0000","last_up_change":"2024-09-16T15:39:10.681384+0000","last_in_change":"2024-09-16T15:39:00.544477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:39:11.602252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:50:59.568133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-16T15:51:00.775162+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":105,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5524076-a903-4d0a-af6a-caf474c4765d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6809","nonce":3111342741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6811","nonce":3111342741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6815","nonce":3111342741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6813","nonce":3111342741}]},"public_addr":"172.21.15.151:6809/3111342741","cluster_addr":"172.21.15.151:6811/3111342741","heartbeat_back_addr":"172.21.15.151:6815/3111342741","heartbeat_front_addr":"172.21.15.151:6813/3111342741","state":["exists","up"]},{"osd":1,"uuid":"dd25925f-0ae1-494e-8529-4dc205759db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6801","nonce":3368304087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6803","nonce":3368304087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6807","nonce":3368304087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6805","nonce":3368304087}]},"public_addr":"172.21.15.151:6801/3368304087","cluster_addr":"172.21.15.151:6803/3368304087","heartbeat_back_addr":"172.21.15.151:6807/3368304087","heartbeat_front_addr":"172.21.15.151:6805/3368304087","state":["exists","up"]},{"osd":2,"uuid":"c2f18115-013d-4da6-a0fe-cb978f4494b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6823","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6825","nonce":3079373271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6827","nonce":3079373271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6831","nonce":3079373271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6829","nonce":3079373271}]},"public_addr":"172.21.15.151:6825/3079373271","cluster_addr":"172.21.15.151:6827/3079373271","heartbeat_back_addr":"172.21.15.151:6831/3079373271","heartbeat_front_addr":"172.21.15.151:6829/3079373271","state":["exists","up"]},{"osd":3,"uuid":"36edb718-2fcb-4d88-8ddc-c321eed4279d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6817","nonce":1142509041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6819","nonce":1142509041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6824","nonce":1142509041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6821","nonce":1142509041}]},"public_addr":"172.21.15.151:6817/1142509041","cluster_addr":"172.21.15.151:6819/1142509041","heartbeat_back_addr":"172.21.15.151:6824/1142509041","heartbeat_front_addr":"172.21.15.151:6821/1142509041","state":["exists","up"]},{"osd":4,"uuid":"64d33708-cd0a-4236-abe3-1b646e82ab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6811","nonce":2735495271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6813","nonce":2735495271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6817","nonce":2735495271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6815","nonce":2735495271}]},"public_addr":"172.21.15.167:6811/2735495271","cluster_addr":"172.21.15.167:6813/2735495271","heartbeat_back_addr":"172.21.15.167:6817/2735495271","heartbeat_front_addr":"172.21.15.167:6815/2735495271","state":["exists","up"]},{"osd":5,"uuid":"f14b44a9-5910-458c-b518-09a8d0de94de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6827","nonce":1494916572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6829","nonce":1494916572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6833","nonce":1494916572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6831","nonce":1494916572}]},"public_addr":"172.21.15.167:6827/1494916572","cluster_addr":"172.21.15.167:6829/1494916572","heartbeat_back_addr":"172.21.15.167:6833/1494916572","heartbeat_front_addr":"172.21.15.167:6831/1494916572","state":["exists","up"]},{"osd":6,"uuid":"39335545-2654-4710-84b0-a9aa47cfcbba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6819","nonce":2538273273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6821","nonce":2538273273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6825","nonce":2538273273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6823","nonce":2538273273}]},"public_addr":"172.21.15.167:6819/2538273273","cluster_addr":"172.21.15.167:6821/2538273273","heartbeat_back_addr":"172.21.15.167:6825/2538273273","heartbeat_front_addr":"172.21.15.167:6823/2538273273","state":["exists","up"]},{"osd":7,"uuid":"4f8bfc81-53dd-4339-ac22-1a4007d9f963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6803","nonce":265072227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6805","nonce":265072227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6809","nonce":265072227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6807","nonce":265072227}]},"public_addr":"172.21.15.167:6803/265072227","cluster_addr":"172.21.15.167:6805/265072227","heartbeat_back_addr":"172.21.15.167:6809/265072227","heartbeat_front_addr":"172.21.15.167:6807/265072227","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.150359+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.084358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.722368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.332809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.320709+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.395153+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.935250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.801443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.2","mappings":[{"from":5,"to":3}]},{"pgid":"14.5","mappings":[{"from":5,"to":0}]},{"pgid":"14.12","mappings":[{"from":6,"to":0}]},{"pgid":"14.1c","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:52:05.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.578+0000 7f76b6600640 1 -- 172.21.15.151:0/662839107 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f769007a2c0 msgr2=0x7f769007c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:05.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.578+0000 7f76b6600640 1 --2- 172.21.15.151:0/662839107 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f769007a2c0 0x7f769007c780 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f76a0000f60 tx=0x7f76a0009040 comp rx=0 tx=0).stop 2024-09-16T15:52:05.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.578+0000 7f76b6600640 1 -- 172.21.15.151:0/662839107 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f76b015f5a0 msgr2=0x7f76b00b2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:05.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.578+0000 7f76b6600640 1 --2- 172.21.15.151:0/662839107 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f76b015f5a0 0x7f76b00b2d50 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f76a800c7a0 tx=0x7f76a800cc60 comp rx=0 tx=0).stop 2024-09-16T15:52:05.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.578+0000 7f76b6600640 1 -- 172.21.15.151:0/662839107 shutdown_connections 2024-09-16T15:52:05.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.578+0000 7f76b6600640 1 --2- 172.21.15.151:0/662839107 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f769007a2c0 0x7f769007c780 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:05.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.578+0000 7f76b6600640 1 --2- 172.21.15.151:0/662839107 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f76b015f5a0 0x7f76b00b2d50 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:05.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.578+0000 7f76b6600640 1 --2- 172.21.15.151:0/662839107 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f76b00ccf00 0x7f76b015ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:05.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.578+0000 7f76b6600640 1 --2- 172.21.15.151:0/662839107 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76b0073fb0 0x7f76b00b0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:05.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.578+0000 7f76b6600640 1 -- 172.21.15.151:0/662839107 >> 172.21.15.151:0/662839107 conn(0x7f76b00b1950 msgr2=0x7f76b015d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:05.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.578+0000 7f76b6600640 1 -- 172.21.15.151:0/662839107 shutdown_connections 2024-09-16T15:52:05.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.578+0000 7f76b6600640 1 -- 172.21.15.151:0/662839107 wait complete. 2024-09-16T15:52:05.596 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-16T15:52:05.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.756+0000 7fe395400640 1 -- 172.21.15.151:0/1273797843 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe3900d1ec0 msgr2=0x7fe3900d22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:05.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.756+0000 7fe395400640 1 --2- 172.21.15.151:0/1273797843 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe3900d1ec0 0x7fe3900d22a0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fe3900c72f0 tx=0x7fe38002a5f0 comp rx=0 tx=0).stop 2024-09-16T15:52:05.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.757+0000 7fe395400640 1 -- 172.21.15.151:0/1273797843 shutdown_connections 2024-09-16T15:52:05.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.757+0000 7fe395400640 1 --2- 172.21.15.151:0/1273797843 >> v2:172.21.15.167:3300/0 conn(0x7fe3900b28d0 0x7fe390162730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:05.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.757+0000 7fe395400640 1 --2- 172.21.15.151:0/1273797843 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe3900d1ec0 0x7fe3900d22a0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:05.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.757+0000 7fe395400640 1 -- 172.21.15.151:0/1273797843 >> 172.21.15.151:0/1273797843 conn(0x7fe3900d3de0 msgr2=0x7fe3900d41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:05.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.757+0000 7fe395400640 1 -- 172.21.15.151:0/1273797843 shutdown_connections 2024-09-16T15:52:05.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.757+0000 7fe395400640 1 -- 172.21.15.151:0/1273797843 wait complete. 2024-09-16T15:52:05.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.760+0000 7fe395400640 1 Processor -- start 2024-09-16T15:52:05.761 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.760+0000 7fe395400640 1 -- start start 2024-09-16T15:52:05.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.761+0000 7fe395400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe3900b28d0 0x7fe39014d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:05.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.761+0000 7fe395400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3900b4d10 0x7fe39014a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:05.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.761+0000 7fe395400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe3900d1ec0 0x7fe3901f7e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:05.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.761+0000 7fe395400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe3900b3a80 con 0x7fe3900b28d0 2024-09-16T15:52:05.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.761+0000 7fe395400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe3900b3900 con 0x7fe3900b4d10 2024-09-16T15:52:05.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.761+0000 7fe395400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fe3900b3c00 con 0x7fe3900d1ec0 2024-09-16T15:52:05.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.761+0000 7fe38fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe3900b28d0 0x7fe39014d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:05.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.761+0000 7fe38fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe3900b28d0 0x7fe39014d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41204/0 (socket says 172.21.15.151:41204) 2024-09-16T15:52:05.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.761+0000 7fe38fe00640 1 -- 172.21.15.151:0/2228489787 learned_addr learned my addr 172.21.15.151:0/2228489787 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:52:05.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.761+0000 7fe38f400640 1 --2- 172.21.15.151:0/2228489787 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3900b4d10 0x7fe39014a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:05.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.761+0000 7fe394a00640 1 --2- 172.21.15.151:0/2228489787 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe3900d1ec0 0x7fe3901f7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:05.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.762+0000 7fe38fe00640 1 -- 172.21.15.151:0/2228489787 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe3900d1ec0 msgr2=0x7fe3901f7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:05.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.762+0000 7fe38fe00640 1 --2- 172.21.15.151:0/2228489787 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe3900d1ec0 0x7fe3901f7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:05.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.762+0000 7fe38fe00640 1 -- 172.21.15.151:0/2228489787 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3900b4d10 msgr2=0x7fe39014a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:05.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.762+0000 7fe38fe00640 1 --2- 172.21.15.151:0/2228489787 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3900b4d10 0x7fe39014a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:05.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.762+0000 7fe38fe00640 1 -- 172.21.15.151:0/2228489787 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3901f8750 con 0x7fe3900b28d0 2024-09-16T15:52:05.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.762+0000 7fe394a00640 1 --2- 172.21.15.151:0/2228489787 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe3900d1ec0 0x7fe3901f7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:52:05.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.762+0000 7fe38fe00640 1 --2- 172.21.15.151:0/2228489787 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe3900b28d0 0x7fe39014d5d0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fe380034040 tx=0x7fe380003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:05.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.764+0000 7fe387e00640 1 -- 172.21.15.151:0/2228489787 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe38003e030 con 0x7fe3900b28d0 2024-09-16T15:52:05.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.764+0000 7fe387e00640 1 -- 172.21.15.151:0/2228489787 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe380003e00 con 0x7fe3900b28d0 2024-09-16T15:52:05.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.764+0000 7fe395400640 1 -- 172.21.15.151:0/2228489787 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3901f8a30 con 0x7fe3900b28d0 2024-09-16T15:52:05.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.764+0000 7fe387e00640 1 -- 172.21.15.151:0/2228489787 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe380035040 con 0x7fe3900b28d0 2024-09-16T15:52:05.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.764+0000 7fe395400640 1 -- 172.21.15.151:0/2228489787 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe390149d30 con 0x7fe3900b28d0 2024-09-16T15:52:05.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.765+0000 7fe395400640 1 -- 172.21.15.151:0/2228489787 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3900d22a0 con 0x7fe3900b28d0 2024-09-16T15:52:05.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.766+0000 7fe387e00640 1 -- 172.21.15.151:0/2228489787 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 41) ==== 103684+0+0 (secure 0 0 0) 0x7fe380041030 con 0x7fe3900b28d0 2024-09-16T15:52:05.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.767+0000 7fe387e00640 1 --2- 172.21.15.151:0/2228489787 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe37007a310 0x7fe37007c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:05.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.768+0000 7fe387e00640 1 -- 172.21.15.151:0/2228489787 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5909+0+0 (secure 0 0 0) 0x7fe3800b9e00 con 0x7fe3900b28d0 2024-09-16T15:52:05.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.768+0000 7fe38f400640 1 --2- 172.21.15.151:0/2228489787 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe37007a310 0x7fe37007c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:05.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.769+0000 7fe38f400640 1 --2- 172.21.15.151:0/2228489787 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe37007a310 0x7fe37007c7d0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fe3900b9310 tx=0x7fe37c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:05.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.771+0000 7fe387e00640 1 -- 172.21.15.151:0/2228489787 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe380083f50 con 0x7fe3900b28d0 2024-09-16T15:52:05.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.994+0000 7fe395400640 1 -- 172.21.15.151:0/2228489787 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fe3900a1d00 con 0x7fe3900b28d0 2024-09-16T15:52:05.996 INFO:tasks.ceph.mon.a.smithi151.stderr:2024-09-16T15:52:05.995+0000 7f62c3200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T15:52:05.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:05.997+0000 7fe387e00640 1 -- 172.21.15.151:0/2228489787 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v68) ==== 161+0+0 (secure 0 0 0) 0x7fe380083850 con 0x7fe3900b28d0 2024-09-16T15:52:05.998 INFO:teuthology.orchestra.run.smithi151.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-16T15:52:06.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:06.001+0000 7fe395400640 1 -- 172.21.15.151:0/2228489787 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe37007a310 msgr2=0x7fe37007c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:06.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:06.001+0000 7fe395400640 1 --2- 172.21.15.151:0/2228489787 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe37007a310 0x7fe37007c7d0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fe3900b9310 tx=0x7fe37c009040 comp rx=0 tx=0).stop 2024-09-16T15:52:06.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:06.001+0000 7fe395400640 1 -- 172.21.15.151:0/2228489787 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe3900b28d0 msgr2=0x7fe39014d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:06.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:06.001+0000 7fe395400640 1 --2- 172.21.15.151:0/2228489787 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe3900b28d0 0x7fe39014d5d0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fe380034040 tx=0x7fe380003910 comp rx=0 tx=0).stop 2024-09-16T15:52:06.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:06.001+0000 7fe395400640 1 -- 172.21.15.151:0/2228489787 shutdown_connections 2024-09-16T15:52:06.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:06.001+0000 7fe395400640 1 --2- 172.21.15.151:0/2228489787 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe37007a310 0x7fe37007c7d0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:06.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:06.001+0000 7fe395400640 1 --2- 172.21.15.151:0/2228489787 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3900b4d10 0x7fe39014a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:06.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:06.001+0000 7fe395400640 1 --2- 172.21.15.151:0/2228489787 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe3900d1ec0 0x7fe3901f7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:06.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:06.001+0000 7fe395400640 1 --2- 172.21.15.151:0/2228489787 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe3900b28d0 0x7fe39014d5d0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:06.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:06.001+0000 7fe395400640 1 -- 172.21.15.151:0/2228489787 >> 172.21.15.151:0/2228489787 conn(0x7fe3900d3de0 msgr2=0x7fe3900b40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:06.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:06.001+0000 7fe395400640 1 -- 172.21.15.151:0/2228489787 shutdown_connections 2024-09-16T15:52:06.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:06.001+0000 7fe395400640 1 -- 172.21.15.151:0/2228489787 wait complete. 2024-09-16T15:52:06.021 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-16T15:52:06.074 INFO:tasks.ceph.mds.c.smithi151.stdout:starting mds.c at 2024-09-16T15:52:06.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:06.184+0000 7f389de00640 1 -- 172.21.15.151:0/2785814086 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f387c003a30 msgr2=0x7f387c003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:06.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:06.184+0000 7f389de00640 1 --2- 172.21.15.151:0/2785814086 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f387c003a30 0x7f387c003e50 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f388800c610 tx=0x7f387c005830 comp rx=0 tx=0).stop 2024-09-16T15:52:06.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:06.184+0000 7f3897400640 1 -- 172.21.15.151:0/2785814086 reap_dead start 2024-09-16T15:52:06.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:06.184+0000 7f389de00640 1 -- 172.21.15.151:0/2785814086 shutdown_connections 2024-09-16T15:52:06.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:06.184+0000 7f389de00640 1 -- 172.21.15.151:0/2785814086 >> 172.21.15.151:0/2785814086 conn(0x7f38980b1840 msgr2=0x7f38980b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:06.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:06.185+0000 7f389de00640 1 -- 172.21.15.151:0/2785814086 shutdown_connections 2024-09-16T15:52:06.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:06.185+0000 7f389de00640 1 -- 172.21.15.151:0/2785814086 wait complete. 2024-09-16T15:52:06.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:06.186+0000 7f389de00640 1 Processor -- start 2024-09-16T15:52:06.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:06.186+0000 7f389de00640 1 -- start start 2024-09-16T15:52:06.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:06.187+0000 7f389de00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f387c003a30 0x7f3898151ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:06.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:06.187+0000 7f389de00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38981521a0 0x7f38981f8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:06.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:06.187+0000 7f389de00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3898159940 0x7f3898159dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:06.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:06.187+0000 7f389de00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f38980cf420 con 0x7f387c003a30 2024-09-16T15:52:06.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:06.187+0000 7f389de00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f38980cf2a0 con 0x7f38981521a0 2024-09-16T15:52:06.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:06.187+0000 7f389de00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f38980cf5a0 con 0x7f3898159940 2024-09-16T15:52:06.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:06.187+0000 7f3896a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f387c003a30 0x7f3898151ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:06.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:06.187+0000 7f3896a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f387c003a30 0x7f3898151ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41246/0 (socket says 172.21.15.151:41246) 2024-09-16T15:52:06.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:06.187+0000 7f3896a00640 1 -- 172.21.15.151:0/3732873649 learned_addr learned my addr 172.21.15.151:0/3732873649 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:52:06.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:06.188+0000 7f3896000640 1 --2- 172.21.15.151:0/3732873649 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38981521a0 0x7f38981f8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:06.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:06.188+0000 7f3896a00640 1 -- 172.21.15.151:0/3732873649 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3898159940 msgr2=0x7f3898159dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:06.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:06.188+0000 7f3896a00640 1 --2- 172.21.15.151:0/3732873649 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3898159940 0x7f3898159dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:06.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:06.188+0000 7f3896a00640 1 -- 172.21.15.151:0/3732873649 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38981521a0 msgr2=0x7f38981f8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:06.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:06.188+0000 7f3896a00640 1 --2- 172.21.15.151:0/3732873649 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38981521a0 0x7f38981f8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:06.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:06.188+0000 7f3896a00640 1 -- 172.21.15.151:0/3732873649 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38981f8870 con 0x7f387c003a30 2024-09-16T15:52:06.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:06.188+0000 7f3896000640 1 --2- 172.21.15.151:0/3732873649 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38981521a0 0x7f38981f8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:52:06.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:06.188+0000 7f3896a00640 1 --2- 172.21.15.151:0/3732873649 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f387c003a30 0x7f3898151ac0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f388800c730 tx=0x7f3888004bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:06.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:06.188+0000 7f388f400640 1 -- 172.21.15.151:0/3732873649 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3888042070 con 0x7f387c003a30 2024-09-16T15:52:06.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:06.188+0000 7f388f400640 1 -- 172.21.15.151:0/3732873649 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f388802ba70 con 0x7f387c003a30 2024-09-16T15:52:06.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:06.188+0000 7f389de00640 1 -- 172.21.15.151:0/3732873649 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38981f8b50 con 0x7f387c003a30 2024-09-16T15:52:06.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:06.188+0000 7f388f400640 1 -- 172.21.15.151:0/3732873649 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3888039070 con 0x7f387c003a30 2024-09-16T15:52:06.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:06.188+0000 7f389de00640 1 -- 172.21.15.151:0/3732873649 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3898157fc0 con 0x7f387c003a30 2024-09-16T15:52:06.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:06.190+0000 7f389de00640 1 -- 172.21.15.151:0/3732873649 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38980b2b40 con 0x7f387c003a30 2024-09-16T15:52:06.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:06.191+0000 7f388f400640 1 -- 172.21.15.151:0/3732873649 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 41) ==== 103684+0+0 (secure 0 0 0) 0x7f3888006050 con 0x7f387c003a30 2024-09-16T15:52:06.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:06.191+0000 7f388f400640 1 --2- 172.21.15.151:0/3732873649 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f387807a2f0 0x7f387807c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:06.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:06.192+0000 7f388f400640 1 -- 172.21.15.151:0/3732873649 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5995+0+0 (secure 0 0 0) 0x7f38880b94e0 con 0x7f387c003a30 2024-09-16T15:52:06.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:06.192+0000 7f3896000640 1 --2- 172.21.15.151:0/3732873649 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f387807a2f0 0x7f387807c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:06.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:06.193+0000 7f3896000640 1 --2- 172.21.15.151:0/3732873649 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f387807a2f0 0x7f387807c7b0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f3884000c00 tx=0x7f3884006b70 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:06.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:06.196+0000 7f388f400640 1 -- 172.21.15.151:0/3732873649 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f38880835b0 con 0x7f387c003a30 2024-09-16T15:52:06.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:06.421+0000 7f389de00640 1 -- 172.21.15.151:0/3732873649 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f3898075210 con 0x7f387c003a30 2024-09-16T15:52:07.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:06.999+0000 7f388f400640 1 -- 172.21.15.151:0/3732873649 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v69) ==== 104+0+0 (secure 0 0 0) 0x7f3888082eb0 con 0x7f387c003a30 2024-09-16T15:52:07.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:07.002+0000 7f389de00640 1 -- 172.21.15.151:0/3732873649 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f387807a2f0 msgr2=0x7f387807c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:07.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:07.002+0000 7f389de00640 1 --2- 172.21.15.151:0/3732873649 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f387807a2f0 0x7f387807c7b0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f3884000c00 tx=0x7f3884006b70 comp rx=0 tx=0).stop 2024-09-16T15:52:07.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:07.002+0000 7f389de00640 1 -- 172.21.15.151:0/3732873649 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f387c003a30 msgr2=0x7f3898151ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:07.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:07.002+0000 7f389de00640 1 --2- 172.21.15.151:0/3732873649 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f387c003a30 0x7f3898151ac0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f388800c730 tx=0x7f3888004bb0 comp rx=0 tx=0).stop 2024-09-16T15:52:07.003 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:07.002+0000 7f389de00640 1 -- 172.21.15.151:0/3732873649 shutdown_connections 2024-09-16T15:52:07.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:07.002+0000 7f389de00640 1 --2- 172.21.15.151:0/3732873649 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f387807a2f0 0x7f387807c7b0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:07.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:07.002+0000 7f389de00640 1 --2- 172.21.15.151:0/3732873649 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38981521a0 0x7f38981f8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:07.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:07.002+0000 7f389de00640 1 --2- 172.21.15.151:0/3732873649 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3898159940 0x7f3898159dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:07.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:07.002+0000 7f389de00640 1 --2- 172.21.15.151:0/3732873649 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f387c003a30 0x7f3898151ac0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:07.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:07.002+0000 7f389de00640 1 -- 172.21.15.151:0/3732873649 >> 172.21.15.151:0/3732873649 conn(0x7f38980b1840 msgr2=0x7f3898073cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:07.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:07.002+0000 7f389de00640 1 -- 172.21.15.151:0/3732873649 shutdown_connections 2024-09-16T15:52:07.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:07.002+0000 7f389de00640 1 -- 172.21.15.151:0/3732873649 wait complete. 2024-09-16T15:52:07.021 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-16T15:52:07.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:07.182+0000 7f4173400640 1 -- 172.21.15.151:0/1412872063 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f416c06ee00 msgr2=0x7f416c06f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:07.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:07.182+0000 7f4173400640 1 --2- 172.21.15.151:0/1412872063 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f416c06ee00 0x7f416c06f300 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f416c0ad3c0 tx=0x7f415802a530 comp rx=0 tx=0).stop 2024-09-16T15:52:07.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:07.182+0000 7f4173400640 1 -- 172.21.15.151:0/1412872063 shutdown_connections 2024-09-16T15:52:07.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:07.182+0000 7f4173400640 1 --2- 172.21.15.151:0/1412872063 >> v2:172.21.15.167:3300/0 conn(0x7f416c0b4770 0x7f416c0ad6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:07.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:07.182+0000 7f4173400640 1 --2- 172.21.15.151:0/1412872063 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f416c06ee00 0x7f416c06f300 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:07.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:07.182+0000 7f4173400640 1 -- 172.21.15.151:0/1412872063 >> 172.21.15.151:0/1412872063 conn(0x7f416c0b18e0 msgr2=0x7f416c0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:07.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:07.182+0000 7f4173400640 1 -- 172.21.15.151:0/1412872063 shutdown_connections 2024-09-16T15:52:07.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:07.182+0000 7f4173400640 1 -- 172.21.15.151:0/1412872063 wait complete. 2024-09-16T15:52:07.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:07.184+0000 7f4173400640 1 Processor -- start 2024-09-16T15:52:07.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:07.184+0000 7f4173400640 1 -- start start 2024-09-16T15:52:07.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:07.185+0000 7f4173400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f416c06ee00 0x7f416c15ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:07.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:07.185+0000 7f4173400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f416c0b4770 0x7f416c14f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:07.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:07.185+0000 7f4173400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f416c15dfc0 0x7f416c1575a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:07.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:07.185+0000 7f4173400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f416c0ae0b0 con 0x7f416c0b4770 2024-09-16T15:52:07.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:07.185+0000 7f4173400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f416c0adf30 con 0x7f416c15dfc0 2024-09-16T15:52:07.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:07.185+0000 7f4173400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f416c0ae230 con 0x7f416c06ee00 2024-09-16T15:52:07.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:07.185+0000 7f416be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f416c0b4770 0x7f416c14f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:07.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:07.185+0000 7f416be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f416c0b4770 0x7f416c14f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41276/0 (socket says 172.21.15.151:41276) 2024-09-16T15:52:07.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:07.185+0000 7f416be00640 1 -- 172.21.15.151:0/3924361158 learned_addr learned my addr 172.21.15.151:0/3924361158 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:52:07.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:07.186+0000 7f416be00640 1 -- 172.21.15.151:0/3924361158 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f416c06ee00 msgr2=0x7f416c15ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:07.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:07.186+0000 7f416be00640 1 --2- 172.21.15.151:0/3924361158 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f416c06ee00 0x7f416c15ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:07.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:07.186+0000 7f416be00640 1 -- 172.21.15.151:0/3924361158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f416c15dfc0 msgr2=0x7f416c1575a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:07.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:07.186+0000 7f416be00640 1 --2- 172.21.15.151:0/3924361158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f416c15dfc0 0x7f416c1575a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:07.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:07.186+0000 7f416be00640 1 -- 172.21.15.151:0/3924361158 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f416c15b3a0 con 0x7f416c0b4770 2024-09-16T15:52:07.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:07.186+0000 7f416be00640 1 --2- 172.21.15.151:0/3924361158 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f416c0b4770 0x7f416c14f540 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f4158009780 tx=0x7f4158032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:07.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:07.186+0000 7f4168c00640 1 -- 172.21.15.151:0/3924361158 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4158040070 con 0x7f416c0b4770 2024-09-16T15:52:07.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:07.186+0000 7f4168c00640 1 -- 172.21.15.151:0/3924361158 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4158005310 con 0x7f416c0b4770 2024-09-16T15:52:07.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:07.186+0000 7f4173400640 1 -- 172.21.15.151:0/3924361158 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f416c15b680 con 0x7f416c0b4770 2024-09-16T15:52:07.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:07.186+0000 7f4168c00640 1 -- 172.21.15.151:0/3924361158 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41580047a0 con 0x7f416c0b4770 2024-09-16T15:52:07.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:07.187+0000 7f4173400640 1 -- 172.21.15.151:0/3924361158 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f416c157080 con 0x7f416c0b4770 2024-09-16T15:52:07.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:07.188+0000 7f4173400640 1 -- 172.21.15.151:0/3924361158 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f416c069360 con 0x7f416c0b4770 2024-09-16T15:52:07.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:07.189+0000 7f4168c00640 1 -- 172.21.15.151:0/3924361158 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 41) ==== 103684+0+0 (secure 0 0 0) 0x7f4158039040 con 0x7f416c0b4770 2024-09-16T15:52:07.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:07.190+0000 7f4168c00640 1 --2- 172.21.15.151:0/3924361158 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f414c07a2c0 0x7f414c07c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:07.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:07.190+0000 7f4168c00640 1 -- 172.21.15.151:0/3924361158 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5995+0+0 (secure 0 0 0) 0x7f41580baad0 con 0x7f416c0b4770 2024-09-16T15:52:07.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:07.191+0000 7f4170a00640 1 --2- 172.21.15.151:0/3924361158 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f414c07a2c0 0x7f414c07c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:07.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:07.191+0000 7f4170a00640 1 --2- 172.21.15.151:0/3924361158 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f414c07a2c0 0x7f414c07c780 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f415c007fd0 tx=0x7f415c007900 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:07.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:07.195+0000 7f4168c00640 1 -- 172.21.15.151:0/3924361158 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4158084ba0 con 0x7f416c0b4770 2024-09-16T15:52:07.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:07.378+0000 7f4173400640 1 -- 172.21.15.151:0/3924361158 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f416c0ae6e0 con 0x7f416c0b4770 2024-09-16T15:52:08.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:08.001+0000 7f4168c00640 1 -- 172.21.15.151:0/3924361158 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v109) ==== 181+0+0 (secure 0 0 0) 0x7f41580844a0 con 0x7f416c0b4770 2024-09-16T15:52:08.002 INFO:teuthology.orchestra.run.smithi151.stderr:pool 'cephfs_metadata' removed 2024-09-16T15:52:08.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:08.004+0000 7f4173400640 1 -- 172.21.15.151:0/3924361158 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f414c07a2c0 msgr2=0x7f414c07c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:08.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:08.004+0000 7f4173400640 1 --2- 172.21.15.151:0/3924361158 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f414c07a2c0 0x7f414c07c780 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f415c007fd0 tx=0x7f415c007900 comp rx=0 tx=0).stop 2024-09-16T15:52:08.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:08.004+0000 7f4173400640 1 -- 172.21.15.151:0/3924361158 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f416c0b4770 msgr2=0x7f416c14f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:08.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:08.004+0000 7f4173400640 1 --2- 172.21.15.151:0/3924361158 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f416c0b4770 0x7f416c14f540 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f4158009780 tx=0x7f4158032c10 comp rx=0 tx=0).stop 2024-09-16T15:52:08.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:08.004+0000 7f4173400640 1 -- 172.21.15.151:0/3924361158 shutdown_connections 2024-09-16T15:52:08.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:08.004+0000 7f4173400640 1 --2- 172.21.15.151:0/3924361158 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f414c07a2c0 0x7f414c07c780 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:08.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:08.004+0000 7f4173400640 1 --2- 172.21.15.151:0/3924361158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f416c15dfc0 0x7f416c1575a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:08.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:08.004+0000 7f4173400640 1 --2- 172.21.15.151:0/3924361158 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f416c0b4770 0x7f416c14f540 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:08.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:08.004+0000 7f4173400640 1 --2- 172.21.15.151:0/3924361158 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f416c06ee00 0x7f416c15ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:08.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:08.004+0000 7f4173400640 1 -- 172.21.15.151:0/3924361158 >> 172.21.15.151:0/3924361158 conn(0x7f416c0b18e0 msgr2=0x7f416c0727c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:08.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:08.004+0000 7f4173400640 1 -- 172.21.15.151:0/3924361158 shutdown_connections 2024-09-16T15:52:08.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:08.005+0000 7f4173400640 1 -- 172.21.15.151:0/3924361158 wait complete. 2024-09-16T15:52:08.024 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-16T15:52:08.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:08.185+0000 7f47f4a00640 1 -- 172.21.15.151:0/2544323263 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f47f006e590 msgr2=0x7f47f006fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:08.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:08.185+0000 7f47f4a00640 1 --2- 172.21.15.151:0/2544323263 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f47f006e590 0x7f47f006fe40 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f47f0070ae0 tx=0x7f47d802a560 comp rx=0 tx=0).stop 2024-09-16T15:52:08.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:08.185+0000 7f47f4a00640 1 -- 172.21.15.151:0/2544323263 shutdown_connections 2024-09-16T15:52:08.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:08.185+0000 7f47f4a00640 1 --2- 172.21.15.151:0/2544323263 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f47f0070380 0x7f47f00739d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:08.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:08.185+0000 7f47f4a00640 1 --2- 172.21.15.151:0/2544323263 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f47f006e590 0x7f47f006fe40 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:08.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:08.185+0000 7f47f4a00640 1 --2- 172.21.15.151:0/2544323263 >> v2:172.21.15.167:3300/0 conn(0x7f47f015bda0 0x7f47f0073010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:08.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:08.185+0000 7f47f4a00640 1 -- 172.21.15.151:0/2544323263 >> 172.21.15.151:0/2544323263 conn(0x7f47f00cfa70 msgr2=0x7f47f00cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:08.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:08.185+0000 7f47f4a00640 1 -- 172.21.15.151:0/2544323263 shutdown_connections 2024-09-16T15:52:08.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:08.185+0000 7f47f4a00640 1 -- 172.21.15.151:0/2544323263 wait complete. 2024-09-16T15:52:08.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:08.187+0000 7f47f4a00640 1 Processor -- start 2024-09-16T15:52:08.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:08.187+0000 7f47f4a00640 1 -- start start 2024-09-16T15:52:08.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:08.188+0000 7f47f4a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47f006e590 0x7f47f015c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:08.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:08.188+0000 7f47f4a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f47f0070380 0x7f47f01f7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:08.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:08.188+0000 7f47f4a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f47f015bda0 0x7f47f01f8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:08.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:08.188+0000 7f47f4a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f47f0074090 con 0x7f47f015bda0 2024-09-16T15:52:08.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:08.188+0000 7f47f4a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f47f0073f10 con 0x7f47f006e590 2024-09-16T15:52:08.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:08.188+0000 7f47f4a00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f47f0074210 con 0x7f47f0070380 2024-09-16T15:52:08.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:08.188+0000 7f47efe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f47f015bda0 0x7f47f01f8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:08.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:08.188+0000 7f47efe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f47f015bda0 0x7f47f01f8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41298/0 (socket says 172.21.15.151:41298) 2024-09-16T15:52:08.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:08.188+0000 7f47efe00640 1 -- 172.21.15.151:0/754582898 learned_addr learned my addr 172.21.15.151:0/754582898 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:52:08.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:08.188+0000 7f47ef400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47f006e590 0x7f47f015c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:08.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:08.188+0000 7f47e7e00640 1 --2- 172.21.15.151:0/754582898 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f47f0070380 0x7f47f01f7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:08.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:08.188+0000 7f47efe00640 1 -- 172.21.15.151:0/754582898 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f47f0070380 msgr2=0x7f47f01f7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:08.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:08.188+0000 7f47efe00640 1 --2- 172.21.15.151:0/754582898 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f47f0070380 0x7f47f01f7ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:08.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:08.188+0000 7f47efe00640 1 -- 172.21.15.151:0/754582898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47f006e590 msgr2=0x7f47f015c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:08.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:08.188+0000 7f47efe00640 1 --2- 172.21.15.151:0/754582898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47f006e590 0x7f47f015c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:08.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:08.188+0000 7f47efe00640 1 -- 172.21.15.151:0/754582898 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47f015dfe0 con 0x7f47f015bda0 2024-09-16T15:52:08.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:08.189+0000 7f47efe00640 1 --2- 172.21.15.151:0/754582898 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f47f015bda0 0x7f47f01f8000 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f47e800da90 tx=0x7f47e800df50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:08.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:08.189+0000 7f47e7400640 1 -- 172.21.15.151:0/754582898 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e8017070 con 0x7f47f015bda0 2024-09-16T15:52:08.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:08.189+0000 7f47e7400640 1 -- 172.21.15.151:0/754582898 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f47e8003970 con 0x7f47f015bda0 2024-09-16T15:52:08.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:08.189+0000 7f47f4a00640 1 -- 172.21.15.151:0/754582898 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47f015e2c0 con 0x7f47f015bda0 2024-09-16T15:52:08.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:08.189+0000 7f47e7400640 1 -- 172.21.15.151:0/754582898 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e8003d80 con 0x7f47f015bda0 2024-09-16T15:52:08.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:08.189+0000 7f47f4a00640 1 -- 172.21.15.151:0/754582898 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47f015e880 con 0x7f47f015bda0 2024-09-16T15:52:08.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:08.190+0000 7f47f4a00640 1 -- 172.21.15.151:0/754582898 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47f0074740 con 0x7f47f015bda0 2024-09-16T15:52:08.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:08.192+0000 7f47e7400640 1 -- 172.21.15.151:0/754582898 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 41) ==== 103684+0+0 (secure 0 0 0) 0x7f47e8003f20 con 0x7f47f015bda0 2024-09-16T15:52:08.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:08.192+0000 7f47e7400640 1 --2- 172.21.15.151:0/754582898 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f47cc07a2f0 0x7f47cc07c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:08.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:08.193+0000 7f47e7400640 1 -- 172.21.15.151:0/754582898 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5441+0+0 (secure 0 0 0) 0x7f47e8099b70 con 0x7f47f015bda0 2024-09-16T15:52:08.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:08.193+0000 7f47ef400640 1 --2- 172.21.15.151:0/754582898 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f47cc07a2f0 0x7f47cc07c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:08.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:08.193+0000 7f47ef400640 1 --2- 172.21.15.151:0/754582898 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f47cc07a2f0 0x7f47cc07c7b0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f47e0005fd0 tx=0x7f47e000b2e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:08.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:08.197+0000 7f47e7400640 1 -- 172.21.15.151:0/754582898 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f47e80635f0 con 0x7f47f015bda0 2024-09-16T15:52:08.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:08.382+0000 7f47f4a00640 1 -- 172.21.15.151:0/754582898 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f47f00ae7c0 con 0x7f47f015bda0 2024-09-16T15:52:09.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.012+0000 7f47e7400640 1 -- 172.21.15.151:0/754582898 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v110) ==== 169+0+0 (secure 0 0 0) 0x7f47e806a070 con 0x7f47f015bda0 2024-09-16T15:52:09.014 INFO:teuthology.orchestra.run.smithi151.stderr:pool 'cephfs_data' removed 2024-09-16T15:52:09.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.016+0000 7f47f4a00640 1 -- 172.21.15.151:0/754582898 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f47cc07a2f0 msgr2=0x7f47cc07c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:09.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.016+0000 7f47f4a00640 1 --2- 172.21.15.151:0/754582898 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f47cc07a2f0 0x7f47cc07c7b0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f47e0005fd0 tx=0x7f47e000b2e0 comp rx=0 tx=0).stop 2024-09-16T15:52:09.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.016+0000 7f47f4a00640 1 -- 172.21.15.151:0/754582898 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f47f015bda0 msgr2=0x7f47f01f8000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:09.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.016+0000 7f47f4a00640 1 --2- 172.21.15.151:0/754582898 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f47f015bda0 0x7f47f01f8000 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f47e800da90 tx=0x7f47e800df50 comp rx=0 tx=0).stop 2024-09-16T15:52:09.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.016+0000 7f47f4a00640 1 -- 172.21.15.151:0/754582898 shutdown_connections 2024-09-16T15:52:09.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.016+0000 7f47f4a00640 1 --2- 172.21.15.151:0/754582898 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f47cc07a2f0 0x7f47cc07c7b0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:09.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.016+0000 7f47f4a00640 1 --2- 172.21.15.151:0/754582898 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f47f015bda0 0x7f47f01f8000 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:09.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.016+0000 7f47f4a00640 1 --2- 172.21.15.151:0/754582898 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f47f0070380 0x7f47f01f7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:09.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.016+0000 7f47f4a00640 1 --2- 172.21.15.151:0/754582898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47f006e590 0x7f47f015c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:09.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.016+0000 7f47f4a00640 1 -- 172.21.15.151:0/754582898 >> 172.21.15.151:0/754582898 conn(0x7f47f00cfa70 msgr2=0x7f47f00b0a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:09.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.017+0000 7f47f4a00640 1 -- 172.21.15.151:0/754582898 shutdown_connections 2024-09-16T15:52:09.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.017+0000 7f47f4a00640 1 -- 172.21.15.151:0/754582898 wait complete. 2024-09-16T15:52:09.037 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-16T15:52:09.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.195+0000 7f986e400640 1 -- 172.21.15.151:0/2881686670 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f986815f5a0 msgr2=0x7f98680b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:09.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.195+0000 7f986e400640 1 --2- 172.21.15.151:0/2881686670 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f986815f5a0 0x7f98680b2440 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f98680cfbb0 tx=0x7f985802a4b0 comp rx=0 tx=0).stop 2024-09-16T15:52:09.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.195+0000 7f986e400640 1 -- 172.21.15.151:0/2881686670 shutdown_connections 2024-09-16T15:52:09.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.195+0000 7f986e400640 1 --2- 172.21.15.151:0/2881686670 >> v2:172.21.15.167:3300/0 conn(0x7f98680ccf00 0x7f98680acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:09.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.195+0000 7f986e400640 1 --2- 172.21.15.151:0/2881686670 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f986815f5a0 0x7f98680b2440 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:09.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.195+0000 7f986e400640 1 -- 172.21.15.151:0/2881686670 >> 172.21.15.151:0/2881686670 conn(0x7f98680b1950 msgr2=0x7f98680b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:09.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.195+0000 7f986e400640 1 -- 172.21.15.151:0/2881686670 shutdown_connections 2024-09-16T15:52:09.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.195+0000 7f986e400640 1 -- 172.21.15.151:0/2881686670 wait complete. 2024-09-16T15:52:09.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.197+0000 7f986e400640 1 Processor -- start 2024-09-16T15:52:09.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.197+0000 7f986e400640 1 -- start start 2024-09-16T15:52:09.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.198+0000 7f986e400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f98680ad270 0x7f98681547b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:09.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.198+0000 7f986e400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98680ccf00 0x7f98680cf680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:09.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.198+0000 7f986e400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f986815f5a0 0x7f98680cfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:09.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.198+0000 7f986e400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f986806f160 con 0x7f986815f5a0 2024-09-16T15:52:09.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.198+0000 7f986e400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f986806f2e0 con 0x7f98680ccf00 2024-09-16T15:52:09.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.198+0000 7f986e400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f986806f460 con 0x7f98680ad270 2024-09-16T15:52:09.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.198+0000 7f9867e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f986815f5a0 0x7f98680cfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:09.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.198+0000 7f9867e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f986815f5a0 0x7f98680cfbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41314/0 (socket says 172.21.15.151:41314) 2024-09-16T15:52:09.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.198+0000 7f9867e00640 1 -- 172.21.15.151:0/3138000037 learned_addr learned my addr 172.21.15.151:0/3138000037 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:52:09.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.198+0000 7f9867400640 1 --2- 172.21.15.151:0/3138000037 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f98680ad270 0x7f98681547b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:09.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.198+0000 7f9866a00640 1 --2- 172.21.15.151:0/3138000037 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98680ccf00 0x7f98680cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:09.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.199+0000 7f9867e00640 1 -- 172.21.15.151:0/3138000037 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f98680ad270 msgr2=0x7f98681547b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:09.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.199+0000 7f9867e00640 1 --2- 172.21.15.151:0/3138000037 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f98680ad270 0x7f98681547b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:09.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.199+0000 7f9867e00640 1 -- 172.21.15.151:0/3138000037 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98680ccf00 msgr2=0x7f98680cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:09.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.199+0000 7f9867e00640 1 --2- 172.21.15.151:0/3138000037 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98680ccf00 0x7f98680cf680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:09.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.199+0000 7f9867e00640 1 -- 172.21.15.151:0/3138000037 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9868156d80 con 0x7f986815f5a0 2024-09-16T15:52:09.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.199+0000 7f9867e00640 1 --2- 172.21.15.151:0/3138000037 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f986815f5a0 0x7f98680cfbc0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f986000c770 tx=0x7f986000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:09.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.199+0000 7f985f400640 1 -- 172.21.15.151:0/3138000037 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9860017070 con 0x7f986815f5a0 2024-09-16T15:52:09.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.199+0000 7f985f400640 1 -- 172.21.15.151:0/3138000037 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9860003ca0 con 0x7f986815f5a0 2024-09-16T15:52:09.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.199+0000 7f985f400640 1 -- 172.21.15.151:0/3138000037 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98600052d0 con 0x7f986815f5a0 2024-09-16T15:52:09.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.199+0000 7f986e400640 1 -- 172.21.15.151:0/3138000037 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98681565e0 con 0x7f986815f5a0 2024-09-16T15:52:09.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.199+0000 7f986e400640 1 -- 172.21.15.151:0/3138000037 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98680c9500 con 0x7f986815f5a0 2024-09-16T15:52:09.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.201+0000 7f986e400640 1 -- 172.21.15.151:0/3138000037 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f986806f9f0 con 0x7f986815f5a0 2024-09-16T15:52:09.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.202+0000 7f985f400640 1 -- 172.21.15.151:0/3138000037 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 41) ==== 103684+0+0 (secure 0 0 0) 0x7f986001a070 con 0x7f986815f5a0 2024-09-16T15:52:09.205 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.203+0000 7f985f400640 1 --2- 172.21.15.151:0/3138000037 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f984807a2c0 0x7f984807c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:09.205 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.203+0000 7f985f400640 1 -- 172.21.15.151:0/3138000037 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5027+0+0 (secure 0 0 0) 0x7f9860098cb0 con 0x7f986815f5a0 2024-09-16T15:52:09.205 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.203+0000 7f9867400640 1 --2- 172.21.15.151:0/3138000037 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f984807a2c0 0x7f984807c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:09.205 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.204+0000 7f9867400640 1 --2- 172.21.15.151:0/3138000037 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f984807a2c0 0x7f984807c780 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f9858034000 tx=0x7f9858004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:09.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.207+0000 7f985f400640 1 -- 172.21.15.151:0/3138000037 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9868156d80 con 0x7f986815f5a0 2024-09-16T15:52:09.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.391+0000 7f986e400640 1 -- 172.21.15.151:0/3138000037 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f9868155660 con 0x7f986815f5a0 2024-09-16T15:52:09.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.395+0000 7f985f400640 1 -- 172.21.15.151:0/3138000037 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v47) ==== 71+0+0 (secure 0 0 0) 0x7f98600627d0 con 0x7f986815f5a0 2024-09-16T15:52:09.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.398+0000 7f986e400640 1 -- 172.21.15.151:0/3138000037 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f984807a2c0 msgr2=0x7f984807c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:09.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.398+0000 7f986e400640 1 --2- 172.21.15.151:0/3138000037 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f984807a2c0 0x7f984807c780 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f9858034000 tx=0x7f9858004730 comp rx=0 tx=0).stop 2024-09-16T15:52:09.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.398+0000 7f986e400640 1 -- 172.21.15.151:0/3138000037 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f986815f5a0 msgr2=0x7f98680cfbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:09.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.398+0000 7f986e400640 1 --2- 172.21.15.151:0/3138000037 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f986815f5a0 0x7f98680cfbc0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f986000c770 tx=0x7f986000cb70 comp rx=0 tx=0).stop 2024-09-16T15:52:09.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.398+0000 7f986e400640 1 -- 172.21.15.151:0/3138000037 shutdown_connections 2024-09-16T15:52:09.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.398+0000 7f986e400640 1 --2- 172.21.15.151:0/3138000037 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f984807a2c0 0x7f984807c780 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:09.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.398+0000 7f986e400640 1 --2- 172.21.15.151:0/3138000037 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98680ccf00 0x7f98680cf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:09.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.398+0000 7f986e400640 1 --2- 172.21.15.151:0/3138000037 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f986815f5a0 0x7f98680cfbc0 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:09.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.398+0000 7f986e400640 1 --2- 172.21.15.151:0/3138000037 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f98680ad270 0x7f98681547b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:09.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.398+0000 7f986e400640 1 -- 172.21.15.151:0/3138000037 >> 172.21.15.151:0/3138000037 conn(0x7f98680b1950 msgr2=0x7f98680b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:09.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.398+0000 7f986e400640 1 -- 172.21.15.151:0/3138000037 shutdown_connections 2024-09-16T15:52:09.401 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.398+0000 7f986e400640 1 -- 172.21.15.151:0/3138000037 wait complete. 2024-09-16T15:52:09.415 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_is_printed_in_clone_status_output' 2024-09-16T15:52:09.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.573+0000 7f8853800640 1 -- 172.21.15.151:0/1247087705 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f884c0b2440 msgr2=0x7f884c06f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:09.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.573+0000 7f8853800640 1 --2- 172.21.15.151:0/1247087705 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f884c0b2440 0x7f884c06f160 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f8844005fd0 tx=0x7f884402a820 comp rx=0 tx=0).stop 2024-09-16T15:52:09.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.574+0000 7f8853800640 1 -- 172.21.15.151:0/1247087705 shutdown_connections 2024-09-16T15:52:09.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.574+0000 7f8853800640 1 --2- 172.21.15.151:0/1247087705 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f884c0b2440 0x7f884c06f160 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:09.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.574+0000 7f8853800640 1 -- 172.21.15.151:0/1247087705 >> 172.21.15.151:0/1247087705 conn(0x7f884c0b1950 msgr2=0x7f884c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:09.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.574+0000 7f8853800640 1 -- 172.21.15.151:0/1247087705 shutdown_connections 2024-09-16T15:52:09.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.574+0000 7f8853800640 1 -- 172.21.15.151:0/1247087705 wait complete. 2024-09-16T15:52:09.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.575+0000 7f8853800640 1 Processor -- start 2024-09-16T15:52:09.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.575+0000 7f8853800640 1 -- start start 2024-09-16T15:52:09.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.576+0000 7f8853800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f884c0b2440 0x7f884c1f7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:09.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.576+0000 7f8853800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f884c0ccf00 0x7f884c156780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:09.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.576+0000 7f8853800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f884c15f5a0 0x7f884c156cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:09.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.576+0000 7f8853800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f884c074840 con 0x7f884c0b2440 2024-09-16T15:52:09.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.576+0000 7f8853800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f884c0746c0 con 0x7f884c15f5a0 2024-09-16T15:52:09.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.576+0000 7f8853800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f884c06f6a0 con 0x7f884c0ccf00 2024-09-16T15:52:09.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.577+0000 7f8850e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f884c0b2440 0x7f884c1f7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:09.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.577+0000 7f8850e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f884c0b2440 0x7f884c1f7330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41330/0 (socket says 172.21.15.151:41330) 2024-09-16T15:52:09.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.577+0000 7f8850e00640 1 -- 172.21.15.151:0/781604469 learned_addr learned my addr 172.21.15.151:0/781604469 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:52:09.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.577+0000 7f8851800640 1 --2- 172.21.15.151:0/781604469 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f884c15f5a0 0x7f884c156cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:09.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.577+0000 7f884be00640 1 --2- 172.21.15.151:0/781604469 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f884c0ccf00 0x7f884c156780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:09.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.577+0000 7f8850e00640 1 -- 172.21.15.151:0/781604469 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f884c0ccf00 msgr2=0x7f884c156780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:09.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.577+0000 7f8850e00640 1 --2- 172.21.15.151:0/781604469 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f884c0ccf00 0x7f884c156780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:09.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.577+0000 7f8850e00640 1 -- 172.21.15.151:0/781604469 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f884c15f5a0 msgr2=0x7f884c156cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:09.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.577+0000 7f8850e00640 1 --2- 172.21.15.151:0/781604469 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f884c15f5a0 0x7f884c156cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:09.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.577+0000 7f8850e00640 1 -- 172.21.15.151:0/781604469 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f884c1f7870 con 0x7f884c0b2440 2024-09-16T15:52:09.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.577+0000 7f884be00640 1 --2- 172.21.15.151:0/781604469 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f884c0ccf00 0x7f884c156780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:52:09.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.577+0000 7f8850e00640 1 --2- 172.21.15.151:0/781604469 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f884c0b2440 0x7f884c1f7330 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f883c00c430 tx=0x7f883c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:09.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.578+0000 7f8848c00640 1 -- 172.21.15.151:0/781604469 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883c011790 con 0x7f884c0b2440 2024-09-16T15:52:09.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.578+0000 7f8848c00640 1 -- 172.21.15.151:0/781604469 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f883c017070 con 0x7f884c0b2440 2024-09-16T15:52:09.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.578+0000 7f8848c00640 1 -- 172.21.15.151:0/781604469 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883c011ce0 con 0x7f884c0b2440 2024-09-16T15:52:09.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.578+0000 7f8853800640 1 -- 172.21.15.151:0/781604469 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f884c0cafc0 con 0x7f884c0b2440 2024-09-16T15:52:09.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.578+0000 7f8853800640 1 -- 172.21.15.151:0/781604469 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f884c0cb580 con 0x7f884c0b2440 2024-09-16T15:52:09.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.579+0000 7f8853800640 1 -- 172.21.15.151:0/781604469 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f884c06f160 con 0x7f884c0b2440 2024-09-16T15:52:09.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.580+0000 7f8848c00640 1 -- 172.21.15.151:0/781604469 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 41) ==== 103684+0+0 (secure 0 0 0) 0x7f883c01c460 con 0x7f884c0b2440 2024-09-16T15:52:09.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.581+0000 7f8848c00640 1 --2- 172.21.15.151:0/781604469 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f882c07a2c0 0x7f882c07c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:09.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.581+0000 7f8848c00640 1 -- 172.21.15.151:0/781604469 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5027+0+0 (secure 0 0 0) 0x7f883c0995f0 con 0x7f884c0b2440 2024-09-16T15:52:09.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.581+0000 7f884be00640 1 --2- 172.21.15.151:0/781604469 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f882c07a2c0 0x7f882c07c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:09.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.582+0000 7f884be00640 1 --2- 172.21.15.151:0/781604469 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f882c07a2c0 0x7f882c07c780 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f884c0cbe90 tx=0x7f8838009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:09.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.585+0000 7f8848c00640 1 -- 172.21.15.151:0/781604469 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f883c0636c0 con 0x7f884c0b2440 2024-09-16T15:52:09.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:09.848+0000 7f8853800640 1 -- 172.21.15.151:0/781604469 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_is_printed_in_clone_status_output"]} v 0) -- 0x7f884c069ac0 con 0x7f884c0b2440 2024-09-16T15:52:10.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.014+0000 7f8848c00640 1 -- 172.21.15.151:0/781604469 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_progress_is_printed_in_clone_status_output"]}]=0 v584) ==== 178+0+0 (secure 0 0 0) 0x7f883c062fc0 con 0x7f884c0b2440 2024-09-16T15:52:10.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.017+0000 7f8853800640 1 -- 172.21.15.151:0/781604469 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f882c07a2c0 msgr2=0x7f882c07c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:10.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.017+0000 7f8853800640 1 --2- 172.21.15.151:0/781604469 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f882c07a2c0 0x7f882c07c780 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f884c0cbe90 tx=0x7f8838009040 comp rx=0 tx=0).stop 2024-09-16T15:52:10.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.017+0000 7f8853800640 1 -- 172.21.15.151:0/781604469 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f884c0b2440 msgr2=0x7f884c1f7330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:10.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.017+0000 7f8853800640 1 --2- 172.21.15.151:0/781604469 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f884c0b2440 0x7f884c1f7330 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f883c00c430 tx=0x7f883c00c8f0 comp rx=0 tx=0).stop 2024-09-16T15:52:10.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.017+0000 7f8853800640 1 -- 172.21.15.151:0/781604469 shutdown_connections 2024-09-16T15:52:10.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.017+0000 7f8853800640 1 --2- 172.21.15.151:0/781604469 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f882c07a2c0 0x7f882c07c780 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:10.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.017+0000 7f8853800640 1 --2- 172.21.15.151:0/781604469 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f884c15f5a0 0x7f884c156cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:10.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.017+0000 7f8853800640 1 --2- 172.21.15.151:0/781604469 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f884c0ccf00 0x7f884c156780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:10.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.017+0000 7f8853800640 1 --2- 172.21.15.151:0/781604469 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f884c0b2440 0x7f884c1f7330 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:10.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.017+0000 7f8853800640 1 -- 172.21.15.151:0/781604469 >> 172.21.15.151:0/781604469 conn(0x7f884c0b1950 msgr2=0x7f884c0acd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:10.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.017+0000 7f8853800640 1 -- 172.21.15.151:0/781604469 shutdown_connections 2024-09-16T15:52:10.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.017+0000 7f8853800640 1 -- 172.21.15.151:0/781604469 wait complete. 2024-09-16T15:52:10.039 INFO:tasks.cephfs_test_runner:Test that the command "ceph fs clone status" prints progress stats ... ok 2024-09-16T15:52:10.039 INFO:tasks.cephfs_test_runner:Starting test: test_when_clones_cancelled_are_equal_to_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that progress bars, that printed for 3 ongoing clone jobs, are 2024-09-16T15:52:10.039 INFO:tasks.cephfs_test_runner:test_when_clones_cancelled_are_equal_to_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-09-16T15:52:10.039 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:52:10.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.196+0000 7f64ea800640 1 -- 172.21.15.151:0/3125402869 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f64e415f5a0 msgr2=0x7f64e40b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:10.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.196+0000 7f64ea800640 1 --2- 172.21.15.151:0/3125402869 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f64e415f5a0 0x7f64e40b2440 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f64d4009be0 tx=0x7f64d402a890 comp rx=0 tx=0).stop 2024-09-16T15:52:10.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.196+0000 7f64ea800640 1 -- 172.21.15.151:0/3125402869 shutdown_connections 2024-09-16T15:52:10.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.196+0000 7f64ea800640 1 --2- 172.21.15.151:0/3125402869 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f64e415f5a0 0x7f64e40b2440 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:10.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.196+0000 7f64ea800640 1 -- 172.21.15.151:0/3125402869 >> 172.21.15.151:0/3125402869 conn(0x7f64e40b1950 msgr2=0x7f64e40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:10.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.196+0000 7f64ea800640 1 -- 172.21.15.151:0/3125402869 shutdown_connections 2024-09-16T15:52:10.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.196+0000 7f64ea800640 1 -- 172.21.15.151:0/3125402869 wait complete. 2024-09-16T15:52:10.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.197+0000 7f64ea800640 1 Processor -- start 2024-09-16T15:52:10.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.198+0000 7f64ea800640 1 -- start start 2024-09-16T15:52:10.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.198+0000 7f64ea800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f64e4073fb0 0x7f64e4149e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:10.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.198+0000 7f64ea800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64e40ccf00 0x7f64e41f7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:10.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.198+0000 7f64ea800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f64e415f5a0 0x7f64e4146a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:10.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.198+0000 7f64ea800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f64e406ef30 con 0x7f64e4073fb0 2024-09-16T15:52:10.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.198+0000 7f64ea800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f64e406fa80 con 0x7f64e40ccf00 2024-09-16T15:52:10.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.198+0000 7f64ea800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f64e4074460 con 0x7f64e415f5a0 2024-09-16T15:52:10.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.199+0000 7f64e3400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f64e4073fb0 0x7f64e4149e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:10.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.199+0000 7f64e3400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f64e4073fb0 0x7f64e4149e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41340/0 (socket says 172.21.15.151:41340) 2024-09-16T15:52:10.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.199+0000 7f64e3400640 1 -- 172.21.15.151:0/2810660061 learned_addr learned my addr 172.21.15.151:0/2810660061 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:52:10.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.199+0000 7f64e2a00640 1 --2- 172.21.15.151:0/2810660061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64e40ccf00 0x7f64e41f7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:10.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.199+0000 7f64e3e00640 1 --2- 172.21.15.151:0/2810660061 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f64e415f5a0 0x7f64e4146a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:10.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.199+0000 7f64e3400640 1 -- 172.21.15.151:0/2810660061 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f64e415f5a0 msgr2=0x7f64e4146a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:10.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.199+0000 7f64e3400640 1 --2- 172.21.15.151:0/2810660061 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f64e415f5a0 0x7f64e4146a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:10.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.199+0000 7f64e3400640 1 -- 172.21.15.151:0/2810660061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64e40ccf00 msgr2=0x7f64e41f7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:10.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.199+0000 7f64e3400640 1 --2- 172.21.15.151:0/2810660061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64e40ccf00 0x7f64e41f7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:10.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.199+0000 7f64e3400640 1 -- 172.21.15.151:0/2810660061 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64e41f8490 con 0x7f64e4073fb0 2024-09-16T15:52:10.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.199+0000 7f64e3400640 1 --2- 172.21.15.151:0/2810660061 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f64e4073fb0 0x7f64e4149e90 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f64d4009d00 tx=0x7f64d4033cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:10.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.200+0000 7f64db400640 1 -- 172.21.15.151:0/2810660061 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d4031d80 con 0x7f64e4073fb0 2024-09-16T15:52:10.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.200+0000 7f64db400640 1 -- 172.21.15.151:0/2810660061 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f64d4003a60 con 0x7f64e4073fb0 2024-09-16T15:52:10.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.200+0000 7f64ea800640 1 -- 172.21.15.151:0/2810660061 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64e41f8770 con 0x7f64e4073fb0 2024-09-16T15:52:10.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.200+0000 7f64db400640 1 -- 172.21.15.151:0/2810660061 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d40330d0 con 0x7f64e4073fb0 2024-09-16T15:52:10.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.200+0000 7f64ea800640 1 -- 172.21.15.151:0/2810660061 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64e41474b0 con 0x7f64e4073fb0 2024-09-16T15:52:10.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.201+0000 7f64ea800640 1 -- 172.21.15.151:0/2810660061 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64e40b2440 con 0x7f64e4073fb0 2024-09-16T15:52:10.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.202+0000 7f64db400640 1 -- 172.21.15.151:0/2810660061 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 41) ==== 103684+0+0 (secure 0 0 0) 0x7f64d4033270 con 0x7f64e4073fb0 2024-09-16T15:52:10.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.203+0000 7f64db400640 1 --2- 172.21.15.151:0/2810660061 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f64c407a2c0 0x7f64c407c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:10.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.203+0000 7f64db400640 1 -- 172.21.15.151:0/2810660061 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5027+0+0 (secure 0 0 0) 0x7f64d40ba8a0 con 0x7f64e4073fb0 2024-09-16T15:52:10.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.203+0000 7f64e2a00640 1 --2- 172.21.15.151:0/2810660061 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f64c407a2c0 0x7f64c407c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:10.205 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.204+0000 7f64e2a00640 1 --2- 172.21.15.151:0/2810660061 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f64c407a2c0 0x7f64c407c780 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f64e4157600 tx=0x7f64d0009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:10.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.207+0000 7f64db400640 1 -- 172.21.15.151:0/2810660061 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f64d403f030 con 0x7f64e4073fb0 2024-09-16T15:52:10.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.392+0000 7f64ea800640 1 -- 172.21.15.151:0/2810660061 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f64e406aaf0 con 0x7f64e4073fb0 2024-09-16T15:52:10.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.393+0000 7f64db400640 1 -- 172.21.15.151:0/2810660061 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) ==== 74+0+13236 (secure 0 0 0) 0x7f64d4031790 con 0x7f64e4073fb0 2024-09-16T15:52:10.395 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:52:10.395 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":110,"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","created":"2024-09-16T15:38:56.348535+0000","modified":"2024-09-16T15:52:09.002212+0000","last_up_change":"2024-09-16T15:39:10.681384+0000","last_in_change":"2024-09-16T15:39:00.544477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:39:11.602252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5524076-a903-4d0a-af6a-caf474c4765d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6809","nonce":3111342741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6811","nonce":3111342741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6815","nonce":3111342741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6813","nonce":3111342741}]},"public_addr":"172.21.15.151:6809/3111342741","cluster_addr":"172.21.15.151:6811/3111342741","heartbeat_back_addr":"172.21.15.151:6815/3111342741","heartbeat_front_addr":"172.21.15.151:6813/3111342741","state":["exists","up"]},{"osd":1,"uuid":"dd25925f-0ae1-494e-8529-4dc205759db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6801","nonce":3368304087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6803","nonce":3368304087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6807","nonce":3368304087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6805","nonce":3368304087}]},"public_addr":"172.21.15.151:6801/3368304087","cluster_addr":"172.21.15.151:6803/3368304087","heartbeat_back_addr":"172.21.15.151:6807/3368304087","heartbeat_front_addr":"172.21.15.151:6805/3368304087","state":["exists","up"]},{"osd":2,"uuid":"c2f18115-013d-4da6-a0fe-cb978f4494b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6823","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6825","nonce":3079373271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6827","nonce":3079373271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6831","nonce":3079373271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6829","nonce":3079373271}]},"public_addr":"172.21.15.151:6825/3079373271","cluster_addr":"172.21.15.151:6827/3079373271","heartbeat_back_addr":"172.21.15.151:6831/3079373271","heartbeat_front_addr":"172.21.15.151:6829/3079373271","state":["exists","up"]},{"osd":3,"uuid":"36edb718-2fcb-4d88-8ddc-c321eed4279d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6817","nonce":1142509041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6819","nonce":1142509041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6824","nonce":1142509041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6821","nonce":1142509041}]},"public_addr":"172.21.15.151:6817/1142509041","cluster_addr":"172.21.15.151:6819/1142509041","heartbeat_back_addr":"172.21.15.151:6824/1142509041","heartbeat_front_addr":"172.21.15.151:6821/1142509041","state":["exists","up"]},{"osd":4,"uuid":"64d33708-cd0a-4236-abe3-1b646e82ab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6811","nonce":2735495271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6813","nonce":2735495271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6817","nonce":2735495271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6815","nonce":2735495271}]},"public_addr":"172.21.15.167:6811/2735495271","cluster_addr":"172.21.15.167:6813/2735495271","heartbeat_back_addr":"172.21.15.167:6817/2735495271","heartbeat_front_addr":"172.21.15.167:6815/2735495271","state":["exists","up"]},{"osd":5,"uuid":"f14b44a9-5910-458c-b518-09a8d0de94de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6827","nonce":1494916572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6829","nonce":1494916572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6833","nonce":1494916572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6831","nonce":1494916572}]},"public_addr":"172.21.15.167:6827/1494916572","cluster_addr":"172.21.15.167:6829/1494916572","heartbeat_back_addr":"172.21.15.167:6833/1494916572","heartbeat_front_addr":"172.21.15.167:6831/1494916572","state":["exists","up"]},{"osd":6,"uuid":"39335545-2654-4710-84b0-a9aa47cfcbba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6819","nonce":2538273273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6821","nonce":2538273273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6825","nonce":2538273273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6823","nonce":2538273273}]},"public_addr":"172.21.15.167:6819/2538273273","cluster_addr":"172.21.15.167:6821/2538273273","heartbeat_back_addr":"172.21.15.167:6825/2538273273","heartbeat_front_addr":"172.21.15.167:6823/2538273273","state":["exists","up"]},{"osd":7,"uuid":"4f8bfc81-53dd-4339-ac22-1a4007d9f963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6803","nonce":265072227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6805","nonce":265072227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6809","nonce":265072227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6807","nonce":265072227}]},"public_addr":"172.21.15.167:6803/265072227","cluster_addr":"172.21.15.167:6805/265072227","heartbeat_back_addr":"172.21.15.167:6809/265072227","heartbeat_front_addr":"172.21.15.167:6807/265072227","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.150359+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.084358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.722368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.332809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.320709+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.395153+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.935250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.801443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.151:6834/1174862515":"2024-09-17T15:52:05.995798+0000","172.21.15.151:6832/1174862515":"2024-09-17T15:52:05.995798+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:52:10.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.396+0000 7f64ea800640 1 -- 172.21.15.151:0/2810660061 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f64c407a2c0 msgr2=0x7f64c407c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:10.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.396+0000 7f64ea800640 1 --2- 172.21.15.151:0/2810660061 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f64c407a2c0 0x7f64c407c780 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f64e4157600 tx=0x7f64d0009040 comp rx=0 tx=0).stop 2024-09-16T15:52:10.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.396+0000 7f64ea800640 1 -- 172.21.15.151:0/2810660061 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f64e4073fb0 msgr2=0x7f64e4149e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:10.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.396+0000 7f64ea800640 1 --2- 172.21.15.151:0/2810660061 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f64e4073fb0 0x7f64e4149e90 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f64d4009d00 tx=0x7f64d4033cb0 comp rx=0 tx=0).stop 2024-09-16T15:52:10.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.396+0000 7f64ea800640 1 -- 172.21.15.151:0/2810660061 shutdown_connections 2024-09-16T15:52:10.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.396+0000 7f64ea800640 1 --2- 172.21.15.151:0/2810660061 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f64c407a2c0 0x7f64c407c780 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:10.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.396+0000 7f64ea800640 1 --2- 172.21.15.151:0/2810660061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64e40ccf00 0x7f64e41f7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:10.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.396+0000 7f64ea800640 1 --2- 172.21.15.151:0/2810660061 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f64e415f5a0 0x7f64e4146a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:10.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.396+0000 7f64ea800640 1 --2- 172.21.15.151:0/2810660061 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f64e4073fb0 0x7f64e4149e90 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:10.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.396+0000 7f64ea800640 1 -- 172.21.15.151:0/2810660061 >> 172.21.15.151:0/2810660061 conn(0x7f64e40b1950 msgr2=0x7f64e40b3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:10.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.396+0000 7f64ea800640 1 -- 172.21.15.151:0/2810660061 shutdown_connections 2024-09-16T15:52:10.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.396+0000 7f64ea800640 1 -- 172.21.15.151:0/2810660061 wait complete. 2024-09-16T15:52:10.414 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T15:39:11.602252+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-16T15:52:10.414 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-16T15:52:10.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.575+0000 7fba4d600640 1 -- 172.21.15.151:0/2778207556 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fba4815bec0 msgr2=0x7fba2c003a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:10.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.575+0000 7fba4d600640 1 --2- 172.21.15.151:0/2778207556 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fba4815bec0 0x7fba2c003a00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fba38009b80 tx=0x7fba2c005830 comp rx=0 tx=0).stop 2024-09-16T15:52:10.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.575+0000 7fba4cc00640 1 -- 172.21.15.151:0/2778207556 reap_dead start 2024-09-16T15:52:10.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.575+0000 7fba4d600640 1 -- 172.21.15.151:0/2778207556 shutdown_connections 2024-09-16T15:52:10.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.575+0000 7fba4d600640 1 -- 172.21.15.151:0/2778207556 >> 172.21.15.151:0/2778207556 conn(0x7fba480cf9f0 msgr2=0x7fba480cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:10.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.575+0000 7fba4d600640 1 -- 172.21.15.151:0/2778207556 shutdown_connections 2024-09-16T15:52:10.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.575+0000 7fba4d600640 1 -- 172.21.15.151:0/2778207556 wait complete. 2024-09-16T15:52:10.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.577+0000 7fba4d600640 1 Processor -- start 2024-09-16T15:52:10.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.577+0000 7fba4d600640 1 -- start start 2024-09-16T15:52:10.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.578+0000 7fba4d600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba4815bec0 0x7fba480ad880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:10.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.578+0000 7fba4d600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba4815a8c0 0x7fba480c81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:10.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.578+0000 7fba4d600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fba480c8720 0x7fba480b30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:10.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.578+0000 7fba4d600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fba48070070 con 0x7fba4815a8c0 2024-09-16T15:52:10.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.578+0000 7fba4d600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fba4806fef0 con 0x7fba4815bec0 2024-09-16T15:52:10.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.578+0000 7fba4d600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fba480cb9a0 con 0x7fba480c8720 2024-09-16T15:52:10.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.578+0000 7fba47400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba4815a8c0 0x7fba480c81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:10.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.578+0000 7fba47400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba4815a8c0 0x7fba480c81e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41360/0 (socket says 172.21.15.151:41360) 2024-09-16T15:52:10.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.578+0000 7fba47400640 1 -- 172.21.15.151:0/4075024828 learned_addr learned my addr 172.21.15.151:0/4075024828 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:52:10.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.579+0000 7fba47e00640 1 --2- 172.21.15.151:0/4075024828 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba4815bec0 0x7fba480ad880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:10.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.579+0000 7fba4cc00640 1 --2- 172.21.15.151:0/4075024828 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fba480c8720 0x7fba480b30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:10.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.579+0000 7fba47400640 1 -- 172.21.15.151:0/4075024828 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fba480c8720 msgr2=0x7fba480b30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:10.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.579+0000 7fba47400640 1 --2- 172.21.15.151:0/4075024828 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fba480c8720 0x7fba480b30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:10.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.579+0000 7fba47400640 1 -- 172.21.15.151:0/4075024828 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba4815bec0 msgr2=0x7fba480ad880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:10.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.579+0000 7fba47400640 1 --2- 172.21.15.151:0/4075024828 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba4815bec0 0x7fba480ad880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:10.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.579+0000 7fba47400640 1 -- 172.21.15.151:0/4075024828 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba480addc0 con 0x7fba4815a8c0 2024-09-16T15:52:10.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.579+0000 7fba4cc00640 1 --2- 172.21.15.151:0/4075024828 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fba480c8720 0x7fba480b30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:52:10.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.579+0000 7fba47400640 1 --2- 172.21.15.151:0/4075024828 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba4815a8c0 0x7fba480c81e0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fba34011ae0 tx=0x7fba3400f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:10.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.579+0000 7fba3fe00640 1 -- 172.21.15.151:0/4075024828 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba3400fd20 con 0x7fba4815a8c0 2024-09-16T15:52:10.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.579+0000 7fba3fe00640 1 -- 172.21.15.151:0/4075024828 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fba34005020 con 0x7fba4815a8c0 2024-09-16T15:52:10.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.579+0000 7fba3fe00640 1 -- 172.21.15.151:0/4075024828 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba34005450 con 0x7fba4815a8c0 2024-09-16T15:52:10.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.579+0000 7fba4d600640 1 -- 172.21.15.151:0/4075024828 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba4815b370 con 0x7fba4815a8c0 2024-09-16T15:52:10.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.580+0000 7fba4d600640 1 -- 172.21.15.151:0/4075024828 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba481f81f0 con 0x7fba4815a8c0 2024-09-16T15:52:10.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.581+0000 7fba4d600640 1 -- 172.21.15.151:0/4075024828 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba48073ef0 con 0x7fba4815a8c0 2024-09-16T15:52:10.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.582+0000 7fba3fe00640 1 -- 172.21.15.151:0/4075024828 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 41) ==== 103684+0+0 (secure 0 0 0) 0x7fba340047a0 con 0x7fba4815a8c0 2024-09-16T15:52:10.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.583+0000 7fba3fe00640 1 --2- 172.21.15.151:0/4075024828 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fba2807a2c0 0x7fba2807c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:10.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.583+0000 7fba3fe00640 1 -- 172.21.15.151:0/4075024828 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5027+0+0 (secure 0 0 0) 0x7fba3409a2b0 con 0x7fba4815a8c0 2024-09-16T15:52:10.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.583+0000 7fba47e00640 1 --2- 172.21.15.151:0/4075024828 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fba2807a2c0 0x7fba2807c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:10.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.584+0000 7fba47e00640 1 --2- 172.21.15.151:0/4075024828 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fba2807a2c0 0x7fba2807c780 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fba38035000 tx=0x7fba380046c0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:10.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.587+0000 7fba3fe00640 1 -- 172.21.15.151:0/4075024828 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fba34063d30 con 0x7fba4815a8c0 2024-09-16T15:52:10.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.773+0000 7fba4d600640 1 -- 172.21.15.151:0/4075024828 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fba480ce1e0 con 0x7fba4815a8c0 2024-09-16T15:52:10.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.774+0000 7fba3fe00640 1 -- 172.21.15.151:0/4075024828 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v110) ==== 93+0+10 (secure 0 0 0) 0x7fba34068760 con 0x7fba4815a8c0 2024-09-16T15:52:10.775 INFO:teuthology.orchestra.run.smithi151.stdout:pg_num: 1 2024-09-16T15:52:10.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.776+0000 7fba4d600640 1 -- 172.21.15.151:0/4075024828 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fba2807a2c0 msgr2=0x7fba2807c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:10.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.776+0000 7fba4d600640 1 --2- 172.21.15.151:0/4075024828 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fba2807a2c0 0x7fba2807c780 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fba38035000 tx=0x7fba380046c0 comp rx=0 tx=0).stop 2024-09-16T15:52:10.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.776+0000 7fba4d600640 1 -- 172.21.15.151:0/4075024828 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba4815a8c0 msgr2=0x7fba480c81e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:10.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.776+0000 7fba4d600640 1 --2- 172.21.15.151:0/4075024828 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba4815a8c0 0x7fba480c81e0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fba34011ae0 tx=0x7fba3400f3c0 comp rx=0 tx=0).stop 2024-09-16T15:52:10.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.776+0000 7fba4d600640 1 -- 172.21.15.151:0/4075024828 shutdown_connections 2024-09-16T15:52:10.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.776+0000 7fba4d600640 1 --2- 172.21.15.151:0/4075024828 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fba2807a2c0 0x7fba2807c780 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:10.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.776+0000 7fba4d600640 1 --2- 172.21.15.151:0/4075024828 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fba480c8720 0x7fba480b30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:10.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.776+0000 7fba4d600640 1 --2- 172.21.15.151:0/4075024828 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba4815a8c0 0x7fba480c81e0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:10.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.776+0000 7fba4d600640 1 --2- 172.21.15.151:0/4075024828 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba4815bec0 0x7fba480ad880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:10.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.776+0000 7fba4d600640 1 -- 172.21.15.151:0/4075024828 >> 172.21.15.151:0/4075024828 conn(0x7fba480cf9f0 msgr2=0x7fba480cabc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:10.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.776+0000 7fba4d600640 1 -- 172.21.15.151:0/4075024828 shutdown_connections 2024-09-16T15:52:10.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.776+0000 7fba4d600640 1 -- 172.21.15.151:0/4075024828 wait complete. 2024-09-16T15:52:10.793 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_equal_to_cloner_threads' 2024-09-16T15:52:10.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.949+0000 7fd1e4000640 1 -- 172.21.15.151:0/3246391055 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd1dc073fb0 msgr2=0x7fd1dc074420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:10.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.949+0000 7fd1e4000640 1 --2- 172.21.15.151:0/3246391055 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd1dc073fb0 0x7fd1dc074420 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc075130 tx=0x7fd1d402a4a0 comp rx=0 tx=0).stop 2024-09-16T15:52:10.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.950+0000 7fd1e4000640 1 -- 172.21.15.151:0/3246391055 shutdown_connections 2024-09-16T15:52:10.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.950+0000 7fd1e4000640 1 --2- 172.21.15.151:0/3246391055 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd1dc073fb0 0x7fd1dc074420 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:10.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.950+0000 7fd1e4000640 1 --2- 172.21.15.151:0/3246391055 >> v2:172.21.15.167:3300/0 conn(0x7fd1dc15f5a0 0x7fd1dc0b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:10.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.950+0000 7fd1e4000640 1 -- 172.21.15.151:0/3246391055 >> 172.21.15.151:0/3246391055 conn(0x7fd1dc0b1950 msgr2=0x7fd1dc0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:10.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.950+0000 7fd1e4000640 1 -- 172.21.15.151:0/3246391055 shutdown_connections 2024-09-16T15:52:10.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.950+0000 7fd1e4000640 1 -- 172.21.15.151:0/3246391055 wait complete. 2024-09-16T15:52:10.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.951+0000 7fd1e4000640 1 Processor -- start 2024-09-16T15:52:10.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.951+0000 7fd1e4000640 1 -- start start 2024-09-16T15:52:10.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.953+0000 7fd1e4000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd1dc073fb0 0x7fd1dc09bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:10.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.953+0000 7fd1e4000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1dc0ccf00 0x7fd1dc1f4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:10.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.953+0000 7fd1e4000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd1dc15f5a0 0x7fd1dc1536c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:10.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.953+0000 7fd1e4000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd1dc06f2e0 con 0x7fd1dc15f5a0 2024-09-16T15:52:10.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.953+0000 7fd1e4000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd1dc06f160 con 0x7fd1dc0ccf00 2024-09-16T15:52:10.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.953+0000 7fd1e4000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fd1dc06f460 con 0x7fd1dc073fb0 2024-09-16T15:52:10.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.953+0000 7fd1e2000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd1dc15f5a0 0x7fd1dc1536c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:10.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.953+0000 7fd1e2000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd1dc15f5a0 0x7fd1dc1536c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41382/0 (socket says 172.21.15.151:41382) 2024-09-16T15:52:10.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.953+0000 7fd1e2000640 1 -- 172.21.15.151:0/2136023991 learned_addr learned my addr 172.21.15.151:0/2136023991 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:52:10.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.953+0000 7fd1e1600640 1 --2- 172.21.15.151:0/2136023991 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd1dc073fb0 0x7fd1dc09bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:10.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.953+0000 7fd1e0c00640 1 --2- 172.21.15.151:0/2136023991 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1dc0ccf00 0x7fd1dc1f4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:10.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.953+0000 7fd1e2000640 1 -- 172.21.15.151:0/2136023991 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd1dc073fb0 msgr2=0x7fd1dc09bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:10.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.953+0000 7fd1e2000640 1 --2- 172.21.15.151:0/2136023991 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd1dc073fb0 0x7fd1dc09bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:10.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.953+0000 7fd1e2000640 1 -- 172.21.15.151:0/2136023991 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1dc0ccf00 msgr2=0x7fd1dc1f4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:10.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.953+0000 7fd1e2000640 1 --2- 172.21.15.151:0/2136023991 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1dc0ccf00 0x7fd1dc1f4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:10.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.953+0000 7fd1e2000640 1 -- 172.21.15.151:0/2136023991 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1dc153c00 con 0x7fd1dc15f5a0 2024-09-16T15:52:10.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.954+0000 7fd1e2000640 1 --2- 172.21.15.151:0/2136023991 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd1dc15f5a0 0x7fd1dc1536c0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fd1d40098f0 tx=0x7fd1d4032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:10.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.954+0000 7fd1d9600640 1 -- 172.21.15.151:0/2136023991 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d4040070 con 0x7fd1dc15f5a0 2024-09-16T15:52:10.957 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.954+0000 7fd1d9600640 1 -- 172.21.15.151:0/2136023991 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd1d4005310 con 0x7fd1dc15f5a0 2024-09-16T15:52:10.957 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.954+0000 7fd1d9600640 1 -- 172.21.15.151:0/2136023991 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d4005c90 con 0x7fd1dc15f5a0 2024-09-16T15:52:10.957 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.954+0000 7fd1e4000640 1 -- 172.21.15.151:0/2136023991 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1dc09ae50 con 0x7fd1dc15f5a0 2024-09-16T15:52:10.957 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.954+0000 7fd1e4000640 1 -- 172.21.15.151:0/2136023991 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1dc09b410 con 0x7fd1dc15f5a0 2024-09-16T15:52:10.957 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.955+0000 7fd1e4000640 1 -- 172.21.15.151:0/2136023991 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1dc06f9f0 con 0x7fd1dc15f5a0 2024-09-16T15:52:10.957 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.956+0000 7fd1d9600640 1 -- 172.21.15.151:0/2136023991 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 41) ==== 103684+0+0 (secure 0 0 0) 0x7fd1d4039040 con 0x7fd1dc15f5a0 2024-09-16T15:52:10.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.957+0000 7fd1d9600640 1 --2- 172.21.15.151:0/2136023991 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd1bc07a2c0 0x7fd1bc07c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:10.959 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.957+0000 7fd1d9600640 1 -- 172.21.15.151:0/2136023991 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5027+0+0 (secure 0 0 0) 0x7fd1d40ba860 con 0x7fd1dc15f5a0 2024-09-16T15:52:10.959 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.958+0000 7fd1e1600640 1 --2- 172.21.15.151:0/2136023991 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd1bc07a2c0 0x7fd1bc07c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:10.959 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.959+0000 7fd1e1600640 1 --2- 172.21.15.151:0/2136023991 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd1bc07a2c0 0x7fd1bc07c780 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc156a00 tx=0x7fd1d00095e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:10.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:10.961+0000 7fd1d9600640 1 -- 172.21.15.151:0/2136023991 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd1d40842e0 con 0x7fd1dc15f5a0 2024-09-16T15:52:11.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:11.220+0000 7fd1e4000640 1 -- 172.21.15.151:0/2136023991 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_equal_to_cloner_threads"]} v 0) -- 0x7fd1dc09ddf0 con 0x7fd1dc15f5a0 2024-09-16T15:52:12.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.026+0000 7fd1d9600640 1 -- 172.21.15.151:0/2136023991 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_equal_to_cloner_threads"]}]=0 v586) ==== 188+0+0 (secure 0 0 0) 0x7fd1d4088d10 con 0x7fd1dc15f5a0 2024-09-16T15:52:12.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.028+0000 7fd1e4000640 1 -- 172.21.15.151:0/2136023991 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd1bc07a2c0 msgr2=0x7fd1bc07c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:12.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.028+0000 7fd1e4000640 1 --2- 172.21.15.151:0/2136023991 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd1bc07a2c0 0x7fd1bc07c780 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc156a00 tx=0x7fd1d00095e0 comp rx=0 tx=0).stop 2024-09-16T15:52:12.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.028+0000 7fd1e4000640 1 -- 172.21.15.151:0/2136023991 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd1dc15f5a0 msgr2=0x7fd1dc1536c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:12.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.028+0000 7fd1e4000640 1 --2- 172.21.15.151:0/2136023991 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd1dc15f5a0 0x7fd1dc1536c0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fd1d40098f0 tx=0x7fd1d4032c10 comp rx=0 tx=0).stop 2024-09-16T15:52:12.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.028+0000 7fd1e4000640 1 -- 172.21.15.151:0/2136023991 shutdown_connections 2024-09-16T15:52:12.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.028+0000 7fd1e4000640 1 --2- 172.21.15.151:0/2136023991 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd1bc07a2c0 0x7fd1bc07c780 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:12.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.028+0000 7fd1e4000640 1 --2- 172.21.15.151:0/2136023991 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd1dc0ccf00 0x7fd1dc1f4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:12.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.028+0000 7fd1e4000640 1 --2- 172.21.15.151:0/2136023991 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd1dc15f5a0 0x7fd1dc1536c0 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:12.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.028+0000 7fd1e4000640 1 --2- 172.21.15.151:0/2136023991 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd1dc073fb0 0x7fd1dc09bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:12.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.028+0000 7fd1e4000640 1 -- 172.21.15.151:0/2136023991 >> 172.21.15.151:0/2136023991 conn(0x7fd1dc0b1950 msgr2=0x7fd1dc0b11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:12.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.029+0000 7fd1e4000640 1 -- 172.21.15.151:0/2136023991 shutdown_connections 2024-09-16T15:52:12.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.029+0000 7fd1e4000640 1 -- 172.21.15.151:0/2136023991 wait complete. 2024-09-16T15:52:12.048 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-16T15:52:12.205 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.204+0000 7fdb20e00640 1 -- 172.21.15.151:0/540001331 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb1c0ccf00 msgr2=0x7fdb1c0b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:12.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.204+0000 7fdb20e00640 1 --2- 172.21.15.151:0/540001331 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb1c0ccf00 0x7fdb1c0b2440 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fdb08009830 tx=0x7fdb0802a480 comp rx=0 tx=0).stop 2024-09-16T15:52:12.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.204+0000 7fdb20e00640 1 -- 172.21.15.151:0/540001331 shutdown_connections 2024-09-16T15:52:12.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.204+0000 7fdb20e00640 1 --2- 172.21.15.151:0/540001331 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb1c073fb0 0x7fdb1c074440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:12.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.204+0000 7fdb20e00640 1 --2- 172.21.15.151:0/540001331 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb1c0ccf00 0x7fdb1c0b2440 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:12.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.204+0000 7fdb20e00640 1 -- 172.21.15.151:0/540001331 >> 172.21.15.151:0/540001331 conn(0x7fdb1c0b1950 msgr2=0x7fdb1c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:12.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.204+0000 7fdb20e00640 1 -- 172.21.15.151:0/540001331 shutdown_connections 2024-09-16T15:52:12.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.204+0000 7fdb20e00640 1 -- 172.21.15.151:0/540001331 wait complete. 2024-09-16T15:52:12.207 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.206+0000 7fdb20e00640 1 Processor -- start 2024-09-16T15:52:12.207 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.206+0000 7fdb20e00640 1 -- start start 2024-09-16T15:52:12.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.207+0000 7fdb20e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb1c073fb0 0x7fdb1c1f4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:12.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.207+0000 7fdb20e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb1c0ccf00 0x7fdb1c15dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:12.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.207+0000 7fdb20e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb1c15f5a0 0x7fdb1c156090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:12.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.207+0000 7fdb20e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1c06f2e0 con 0x7fdb1c0ccf00 2024-09-16T15:52:12.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.207+0000 7fdb20e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1c06f160 con 0x7fdb1c15f5a0 2024-09-16T15:52:12.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.207+0000 7fdb20e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fdb1c06f460 con 0x7fdb1c073fb0 2024-09-16T15:52:12.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.207+0000 7fdb19400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb1c0ccf00 0x7fdb1c15dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:12.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.207+0000 7fdb19400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb1c0ccf00 0x7fdb1c15dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41402/0 (socket says 172.21.15.151:41402) 2024-09-16T15:52:12.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.207+0000 7fdb19400640 1 -- 172.21.15.151:0/2496170254 learned_addr learned my addr 172.21.15.151:0/2496170254 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:52:12.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.208+0000 7fdb19e00640 1 --2- 172.21.15.151:0/2496170254 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb1c073fb0 0x7fdb1c1f4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:12.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.208+0000 7fdb1a800640 1 --2- 172.21.15.151:0/2496170254 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb1c15f5a0 0x7fdb1c156090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:12.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.208+0000 7fdb19400640 1 -- 172.21.15.151:0/2496170254 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb1c073fb0 msgr2=0x7fdb1c1f4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:12.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.208+0000 7fdb19400640 1 --2- 172.21.15.151:0/2496170254 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb1c073fb0 0x7fdb1c1f4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:12.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.208+0000 7fdb19400640 1 -- 172.21.15.151:0/2496170254 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb1c15f5a0 msgr2=0x7fdb1c156090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:12.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.208+0000 7fdb19400640 1 --2- 172.21.15.151:0/2496170254 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb1c15f5a0 0x7fdb1c156090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:12.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.208+0000 7fdb19400640 1 -- 172.21.15.151:0/2496170254 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb1c1f4f70 con 0x7fdb1c0ccf00 2024-09-16T15:52:12.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.208+0000 7fdb19400640 1 --2- 172.21.15.151:0/2496170254 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb1c0ccf00 0x7fdb1c15dfe0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fdb08009950 tx=0x7fdb08032c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:12.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.208+0000 7fdb12000640 1 -- 172.21.15.151:0/2496170254 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb08040070 con 0x7fdb1c0ccf00 2024-09-16T15:52:12.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.208+0000 7fdb12000640 1 -- 172.21.15.151:0/2496170254 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb08032e70 con 0x7fdb1c0ccf00 2024-09-16T15:52:12.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.208+0000 7fdb20e00640 1 -- 172.21.15.151:0/2496170254 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb1c1f5250 con 0x7fdb1c0ccf00 2024-09-16T15:52:12.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.208+0000 7fdb12000640 1 -- 172.21.15.151:0/2496170254 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb080055b0 con 0x7fdb1c0ccf00 2024-09-16T15:52:12.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.208+0000 7fdb20e00640 1 -- 172.21.15.151:0/2496170254 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb1c1f5810 con 0x7fdb1c0ccf00 2024-09-16T15:52:12.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.209+0000 7fdb20e00640 1 -- 172.21.15.151:0/2496170254 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb1c06f980 con 0x7fdb1c0ccf00 2024-09-16T15:52:12.211 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.211+0000 7fdb12000640 1 -- 172.21.15.151:0/2496170254 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 41) ==== 103684+0+0 (secure 0 0 0) 0x7fdb08039040 con 0x7fdb1c0ccf00 2024-09-16T15:52:12.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.211+0000 7fdb12000640 1 --2- 172.21.15.151:0/2496170254 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdafc07a2c0 0x7fdafc07c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:12.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.212+0000 7fdb12000640 1 -- 172.21.15.151:0/2496170254 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5027+0+0 (secure 0 0 0) 0x7fdb080ba9a0 con 0x7fdb1c0ccf00 2024-09-16T15:52:12.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.212+0000 7fdb19e00640 1 --2- 172.21.15.151:0/2496170254 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdafc07a2c0 0x7fdafc07c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:12.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.213+0000 7fdb19e00640 1 --2- 172.21.15.151:0/2496170254 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdafc07a2c0 0x7fdafc07c780 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c0015e0 tx=0x7fdb0c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:12.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.216+0000 7fdb12000640 1 -- 172.21.15.151:0/2496170254 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fdb08084a70 con 0x7fdb1c0ccf00 2024-09-16T15:52:12.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.402+0000 7fdb20e00640 1 -- 172.21.15.151:0/2496170254 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fdb1c09b660 con 0x7fdb1c0ccf00 2024-09-16T15:52:12.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.402+0000 7fdb12000640 1 -- 172.21.15.151:0/2496170254 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v47) ==== 59+0+1998 (secure 0 0 0) 0x7fdb08084370 con 0x7fdb1c0ccf00 2024-09-16T15:52:12.404 INFO:teuthology.orchestra.run.smithi151.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-16T15:52:12.404 INFO:teuthology.orchestra.run.smithi151.stdout:mon advanced mon_op_complaint_time 120 2024-09-16T15:52:12.404 INFO:teuthology.orchestra.run.smithi151.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-16T15:52:12.404 INFO:teuthology.orchestra.run.smithi151.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-16T15:52:12.404 INFO:teuthology.orchestra.run.smithi151.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3795.338492 2024-09-16T15:52:12.404 INFO:teuthology.orchestra.run.smithi151.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 4030.355715 2024-09-16T15:52:12.404 INFO:teuthology.orchestra.run.smithi151.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3817.533640 2024-09-16T15:52:12.404 INFO:teuthology.orchestra.run.smithi151.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3950.656136 2024-09-16T15:52:12.404 INFO:teuthology.orchestra.run.smithi151.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4449.408052 2024-09-16T15:52:12.404 INFO:teuthology.orchestra.run.smithi151.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4439.932296 2024-09-16T15:52:12.404 INFO:teuthology.orchestra.run.smithi151.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4566.875579 2024-09-16T15:52:12.404 INFO:teuthology.orchestra.run.smithi151.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 6320.073632 2024-09-16T15:52:12.405 INFO:teuthology.orchestra.run.smithi151.stdout:mds advanced debug_mds 20/20 2024-09-16T15:52:12.405 INFO:teuthology.orchestra.run.smithi151.stdout:mds advanced debug_mds_balancer 20/20 2024-09-16T15:52:12.405 INFO:teuthology.orchestra.run.smithi151.stdout:mds advanced debug_ms 2/2 2024-09-16T15:52:12.405 INFO:teuthology.orchestra.run.smithi151.stdout:mds dev mds_debug_frag true 2024-09-16T15:52:12.405 INFO:teuthology.orchestra.run.smithi151.stdout:mds dev mds_debug_scatterstat true 2024-09-16T15:52:12.405 INFO:teuthology.orchestra.run.smithi151.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-16T15:52:12.405 INFO:teuthology.orchestra.run.smithi151.stdout:mds dev mds_verify_scatter true 2024-09-16T15:52:12.405 INFO:teuthology.orchestra.run.smithi151.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-16T15:52:12.405 INFO:teuthology.orchestra.run.smithi151.stdout:mds advanced rados_mon_op_timeout 900 2024-09-16T15:52:12.405 INFO:teuthology.orchestra.run.smithi151.stdout:mds advanced rados_osd_op_timeout 900 2024-09-16T15:52:12.405 INFO:teuthology.orchestra.run.smithi151.stdout:client advanced client_mount_timeout 600 2024-09-16T15:52:12.405 INFO:teuthology.orchestra.run.smithi151.stdout:client advanced debug_client 20/20 2024-09-16T15:52:12.405 INFO:teuthology.orchestra.run.smithi151.stdout:client advanced debug_ms 1/1 2024-09-16T15:52:12.405 INFO:teuthology.orchestra.run.smithi151.stdout:client advanced rados_mon_op_timeout 900 2024-09-16T15:52:12.405 INFO:teuthology.orchestra.run.smithi151.stdout:client advanced rados_osd_op_timeout 900 2024-09-16T15:52:12.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.405+0000 7fdb20e00640 1 -- 172.21.15.151:0/2496170254 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdafc07a2c0 msgr2=0x7fdafc07c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:12.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.405+0000 7fdb20e00640 1 --2- 172.21.15.151:0/2496170254 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdafc07a2c0 0x7fdafc07c780 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c0015e0 tx=0x7fdb0c009040 comp rx=0 tx=0).stop 2024-09-16T15:52:12.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.405+0000 7fdb20e00640 1 -- 172.21.15.151:0/2496170254 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb1c0ccf00 msgr2=0x7fdb1c15dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:12.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.405+0000 7fdb20e00640 1 --2- 172.21.15.151:0/2496170254 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb1c0ccf00 0x7fdb1c15dfe0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fdb08009950 tx=0x7fdb08032c80 comp rx=0 tx=0).stop 2024-09-16T15:52:12.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.405+0000 7fdb20e00640 1 -- 172.21.15.151:0/2496170254 shutdown_connections 2024-09-16T15:52:12.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.405+0000 7fdb20e00640 1 --2- 172.21.15.151:0/2496170254 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdafc07a2c0 0x7fdafc07c780 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:12.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.405+0000 7fdb20e00640 1 --2- 172.21.15.151:0/2496170254 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb1c15f5a0 0x7fdb1c156090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:12.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.405+0000 7fdb20e00640 1 --2- 172.21.15.151:0/2496170254 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb1c0ccf00 0x7fdb1c15dfe0 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:12.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.405+0000 7fdb20e00640 1 --2- 172.21.15.151:0/2496170254 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb1c073fb0 0x7fdb1c1f4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:12.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.405+0000 7fdb20e00640 1 -- 172.21.15.151:0/2496170254 >> 172.21.15.151:0/2496170254 conn(0x7fdb1c0b1950 msgr2=0x7fdb1c09aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:12.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.405+0000 7fdb20e00640 1 -- 172.21.15.151:0/2496170254 shutdown_connections 2024-09-16T15:52:12.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.405+0000 7fdb20e00640 1 -- 172.21.15.151:0/2496170254 wait complete. 2024-09-16T15:52:12.421 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-16T15:52:12.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.580+0000 7f1faac00640 1 -- 172.21.15.151:0/3348687515 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1fa415e0c0 msgr2=0x7f1fa40ae290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:12.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.580+0000 7f1faac00640 1 --2- 172.21.15.151:0/3348687515 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1fa415e0c0 0x7f1fa40ae290 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f1f98009900 tx=0x7f1f9802a610 comp rx=0 tx=0).stop 2024-09-16T15:52:12.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.580+0000 7f1faac00640 1 -- 172.21.15.151:0/3348687515 shutdown_connections 2024-09-16T15:52:12.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.580+0000 7f1faac00640 1 --2- 172.21.15.151:0/3348687515 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1fa406ff60 0x7f1fa40cd890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:12.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.580+0000 7f1faac00640 1 --2- 172.21.15.151:0/3348687515 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1fa415e0c0 0x7f1fa40ae290 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:12.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.580+0000 7f1faac00640 1 -- 172.21.15.151:0/3348687515 >> 172.21.15.151:0/3348687515 conn(0x7f1fa40cf9f0 msgr2=0x7f1fa40cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:12.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.580+0000 7f1faac00640 1 -- 172.21.15.151:0/3348687515 shutdown_connections 2024-09-16T15:52:12.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.580+0000 7f1faac00640 1 -- 172.21.15.151:0/3348687515 wait complete. 2024-09-16T15:52:12.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.582+0000 7f1faac00640 1 Processor -- start 2024-09-16T15:52:12.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.582+0000 7f1faac00640 1 -- start start 2024-09-16T15:52:12.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.583+0000 7f1faac00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1fa406ff60 0x7f1fa41f5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:12.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.583+0000 7f1faac00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1fa40cddd0 0x7f1fa4149ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:12.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.583+0000 7f1faac00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1fa415e0c0 0x7f1fa40c59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:12.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.583+0000 7f1faac00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa406f180 con 0x7f1fa406ff60 2024-09-16T15:52:12.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.583+0000 7f1faac00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa40cb9e0 con 0x7f1fa40cddd0 2024-09-16T15:52:12.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.583+0000 7f1faac00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f1fa406f300 con 0x7f1fa415e0c0 2024-09-16T15:52:12.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.583+0000 7f1fa9800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1fa406ff60 0x7f1fa41f5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:12.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.583+0000 7f1fa9800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1fa406ff60 0x7f1fa41f5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41424/0 (socket says 172.21.15.151:41424) 2024-09-16T15:52:12.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.583+0000 7f1fa9800640 1 -- 172.21.15.151:0/886530445 learned_addr learned my addr 172.21.15.151:0/886530445 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:52:12.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.583+0000 7f1fa8e00640 1 --2- 172.21.15.151:0/886530445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1fa40cddd0 0x7f1fa4149ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:12.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.583+0000 7f1faa200640 1 --2- 172.21.15.151:0/886530445 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1fa415e0c0 0x7f1fa40c59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:12.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.584+0000 7f1fa9800640 1 -- 172.21.15.151:0/886530445 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1fa415e0c0 msgr2=0x7f1fa40c59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:12.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.584+0000 7f1fa9800640 1 --2- 172.21.15.151:0/886530445 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1fa415e0c0 0x7f1fa40c59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:12.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.584+0000 7f1fa9800640 1 -- 172.21.15.151:0/886530445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1fa40cddd0 msgr2=0x7f1fa4149ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:12.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.584+0000 7f1fa9800640 1 --2- 172.21.15.151:0/886530445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1fa40cddd0 0x7f1fa4149ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:12.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.584+0000 7f1fa9800640 1 -- 172.21.15.151:0/886530445 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fa40b5390 con 0x7f1fa406ff60 2024-09-16T15:52:12.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.584+0000 7f1faa200640 1 --2- 172.21.15.151:0/886530445 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1fa415e0c0 0x7f1fa40c59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:52:12.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.584+0000 7f1fa9800640 1 --2- 172.21.15.151:0/886530445 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1fa406ff60 0x7f1fa41f5680 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f1f98006010 tx=0x7f1f980035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:12.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.584+0000 7f1fa1600640 1 -- 172.21.15.151:0/886530445 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f9803e030 con 0x7f1fa406ff60 2024-09-16T15:52:12.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.584+0000 7f1fa1600640 1 -- 172.21.15.151:0/886530445 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1f98003ae0 con 0x7f1fa406ff60 2024-09-16T15:52:12.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.584+0000 7f1fa1600640 1 -- 172.21.15.151:0/886530445 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f98035040 con 0x7f1fa406ff60 2024-09-16T15:52:12.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.584+0000 7f1faac00640 1 -- 172.21.15.151:0/886530445 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fa40b50b0 con 0x7f1fa406ff60 2024-09-16T15:52:12.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.584+0000 7f1faac00640 1 -- 172.21.15.151:0/886530445 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fa41f4e80 con 0x7f1fa406ff60 2024-09-16T15:52:12.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.585+0000 7f1faac00640 1 -- 172.21.15.151:0/886530445 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fa41f49b0 con 0x7f1fa406ff60 2024-09-16T15:52:12.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.587+0000 7f1fa1600640 1 -- 172.21.15.151:0/886530445 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 41) ==== 103684+0+0 (secure 0 0 0) 0x7f1f98041030 con 0x7f1fa406ff60 2024-09-16T15:52:12.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.587+0000 7f1fa1600640 1 --2- 172.21.15.151:0/886530445 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1f8407a2c0 0x7f1f8407c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:12.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.588+0000 7f1fa1600640 1 -- 172.21.15.151:0/886530445 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5027+0+0 (secure 0 0 0) 0x7f1f980b9ef0 con 0x7f1fa406ff60 2024-09-16T15:52:12.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.588+0000 7f1fa8e00640 1 --2- 172.21.15.151:0/886530445 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1f8407a2c0 0x7f1f8407c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:12.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.589+0000 7f1fa8e00640 1 --2- 172.21.15.151:0/886530445 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1f8407a2c0 0x7f1f8407c780 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f1fa406f130 tx=0x7f1f90005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:12.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.592+0000 7f1fa1600640 1 -- 172.21.15.151:0/886530445 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1fa40ce1e0 con 0x7f1fa406ff60 2024-09-16T15:52:12.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.777+0000 7f1faac00640 1 -- 172.21.15.151:0/886530445 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f1fa406f820 con 0x7f1fa406ff60 2024-09-16T15:52:12.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.781+0000 7f1fa1600640 1 -- 172.21.15.151:0/886530445 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v48)=0 v48) ==== 122+0+0 (secure 0 0 0) 0x7f1f98083970 con 0x7f1fa406ff60 2024-09-16T15:52:12.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.783+0000 7f1faac00640 1 -- 172.21.15.151:0/886530445 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1f8407a2c0 msgr2=0x7f1f8407c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:12.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.783+0000 7f1faac00640 1 --2- 172.21.15.151:0/886530445 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1f8407a2c0 0x7f1f8407c780 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f1fa406f130 tx=0x7f1f90005f90 comp rx=0 tx=0).stop 2024-09-16T15:52:12.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.783+0000 7f1faac00640 1 -- 172.21.15.151:0/886530445 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1fa406ff60 msgr2=0x7f1fa41f5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:12.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.783+0000 7f1faac00640 1 --2- 172.21.15.151:0/886530445 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1fa406ff60 0x7f1fa41f5680 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f1f98006010 tx=0x7f1f980035f0 comp rx=0 tx=0).stop 2024-09-16T15:52:12.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.783+0000 7f1faac00640 1 -- 172.21.15.151:0/886530445 shutdown_connections 2024-09-16T15:52:12.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.783+0000 7f1faac00640 1 --2- 172.21.15.151:0/886530445 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1f8407a2c0 0x7f1f8407c780 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:12.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.783+0000 7f1faac00640 1 --2- 172.21.15.151:0/886530445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1fa40cddd0 0x7f1fa4149ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:12.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.783+0000 7f1faac00640 1 --2- 172.21.15.151:0/886530445 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1fa415e0c0 0x7f1fa40c59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:12.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.783+0000 7f1faac00640 1 --2- 172.21.15.151:0/886530445 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1fa406ff60 0x7f1fa41f5680 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:12.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.783+0000 7f1faac00640 1 -- 172.21.15.151:0/886530445 >> 172.21.15.151:0/886530445 conn(0x7f1fa40cf9f0 msgr2=0x7f1fa40cbb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:12.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.783+0000 7f1faac00640 1 -- 172.21.15.151:0/886530445 shutdown_connections 2024-09-16T15:52:12.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:12.783+0000 7f1faac00640 1 -- 172.21.15.151:0/886530445 wait complete. 2024-09-16T15:52:12.800 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-16T15:52:12.800 DEBUG:teuthology.orchestra.run.smithi151:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:52:12.887 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:52:12.983 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-16T15:52:12.983 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:52:12.983 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:52:13.019 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T15:52:13.019 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-16T15:52:13.019 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:52:13.057 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:52:13.057 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:52:13.057 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:52:13.094 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T15:52:13.094 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-16T15:52:13.094 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:52:13.134 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:52:13.134 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-16T15:52:13.134 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:52:13.294 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.293+0000 7f8e73600640 1 -- 172.21.15.151:0/3137452563 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e6c15f5a0 msgr2=0x7f8e6c0b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:13.294 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.293+0000 7f8e73600640 1 --2- 172.21.15.151:0/3137452563 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e6c15f5a0 0x7f8e6c0b2440 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f8e60005fd0 tx=0x7f8e6002a4a0 comp rx=0 tx=0).stop 2024-09-16T15:52:13.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.293+0000 7f8e73600640 1 -- 172.21.15.151:0/3137452563 shutdown_connections 2024-09-16T15:52:13.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.293+0000 7f8e73600640 1 --2- 172.21.15.151:0/3137452563 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8e6c073fb0 0x7f8e6c074440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:13.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.293+0000 7f8e73600640 1 --2- 172.21.15.151:0/3137452563 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e6c15f5a0 0x7f8e6c0b2440 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:13.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.293+0000 7f8e73600640 1 -- 172.21.15.151:0/3137452563 >> 172.21.15.151:0/3137452563 conn(0x7f8e6c0b1950 msgr2=0x7f8e6c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:13.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.293+0000 7f8e73600640 1 -- 172.21.15.151:0/3137452563 shutdown_connections 2024-09-16T15:52:13.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.293+0000 7f8e73600640 1 -- 172.21.15.151:0/3137452563 wait complete. 2024-09-16T15:52:13.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.295+0000 7f8e73600640 1 Processor -- start 2024-09-16T15:52:13.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.295+0000 7f8e73600640 1 -- start start 2024-09-16T15:52:13.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.296+0000 7f8e73600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e6c073fb0 0x7f8e6c0cc750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:13.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.296+0000 7f8e73600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e6c0ccf00 0x7f8e6c15e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:13.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.296+0000 7f8e73600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8e6c15f5a0 0x7f8e6c1513a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:13.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.296+0000 7f8e73600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6c06f2e0 con 0x7f8e6c0ccf00 2024-09-16T15:52:13.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.296+0000 7f8e73600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6c06f160 con 0x7f8e6c073fb0 2024-09-16T15:52:13.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.296+0000 7f8e73600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f8e6c06f460 con 0x7f8e6c15f5a0 2024-09-16T15:52:13.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.296+0000 7f8e6be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e6c0ccf00 0x7f8e6c15e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:13.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.296+0000 7f8e70c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e6c073fb0 0x7f8e6c0cc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:13.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.296+0000 7f8e6be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e6c0ccf00 0x7f8e6c15e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49720/0 (socket says 172.21.15.151:49720) 2024-09-16T15:52:13.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.296+0000 7f8e6be00640 1 -- 172.21.15.151:0/913945941 learned_addr learned my addr 172.21.15.151:0/913945941 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:52:13.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.296+0000 7f8e71600640 1 --2- 172.21.15.151:0/913945941 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8e6c15f5a0 0x7f8e6c1513a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:13.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.296+0000 7f8e6be00640 1 -- 172.21.15.151:0/913945941 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8e6c15f5a0 msgr2=0x7f8e6c1513a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:13.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.296+0000 7f8e6be00640 1 --2- 172.21.15.151:0/913945941 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8e6c15f5a0 0x7f8e6c1513a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:13.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.296+0000 7f8e6be00640 1 -- 172.21.15.151:0/913945941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e6c073fb0 msgr2=0x7f8e6c0cc750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:13.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.296+0000 7f8e6be00640 1 --2- 172.21.15.151:0/913945941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e6c073fb0 0x7f8e6c0cc750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:13.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.296+0000 7f8e6be00640 1 -- 172.21.15.151:0/913945941 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e6c1f5050 con 0x7f8e6c0ccf00 2024-09-16T15:52:13.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.297+0000 7f8e6be00640 1 --2- 172.21.15.151:0/913945941 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e6c0ccf00 0x7f8e6c15e660 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f8e5800c430 tx=0x7f8e5800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:13.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.297+0000 7f8e68c00640 1 -- 172.21.15.151:0/913945941 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e58011790 con 0x7f8e6c0ccf00 2024-09-16T15:52:13.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.297+0000 7f8e68c00640 1 -- 172.21.15.151:0/913945941 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8e58017070 con 0x7f8e6c0ccf00 2024-09-16T15:52:13.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.297+0000 7f8e68c00640 1 -- 172.21.15.151:0/913945941 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e58011d00 con 0x7f8e6c0ccf00 2024-09-16T15:52:13.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.297+0000 7f8e73600640 1 -- 172.21.15.151:0/913945941 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e6c1f5330 con 0x7f8e6c0ccf00 2024-09-16T15:52:13.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.297+0000 7f8e73600640 1 -- 172.21.15.151:0/913945941 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e6c1f58f0 con 0x7f8e6c0ccf00 2024-09-16T15:52:13.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.298+0000 7f8e73600640 1 -- 172.21.15.151:0/913945941 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e6c06f980 con 0x7f8e6c0ccf00 2024-09-16T15:52:13.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.299+0000 7f8e68c00640 1 -- 172.21.15.151:0/913945941 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 41) ==== 103684+0+0 (secure 0 0 0) 0x7f8e58026430 con 0x7f8e6c0ccf00 2024-09-16T15:52:13.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.300+0000 7f8e68c00640 1 --2- 172.21.15.151:0/913945941 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8e4c07a2c0 0x7f8e4c07c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:13.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.300+0000 7f8e68c00640 1 -- 172.21.15.151:0/913945941 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5027+0+0 (secure 0 0 0) 0x7f8e58099fb0 con 0x7f8e6c0ccf00 2024-09-16T15:52:13.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.301+0000 7f8e70c00640 1 --2- 172.21.15.151:0/913945941 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8e4c07a2c0 0x7f8e4c07c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:13.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.301+0000 7f8e70c00640 1 --2- 172.21.15.151:0/913945941 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8e4c07a2c0 0x7f8e4c07c780 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f8e60033000 tx=0x7f8e60004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:13.304 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.303+0000 7f8e68c00640 1 -- 172.21.15.151:0/913945941 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8e58063ab0 con 0x7f8e6c0ccf00 2024-09-16T15:52:13.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.524+0000 7f8e73600640 1 -- 172.21.15.151:0/913945941 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8e6c0b2dd0 con 0x7f8e6c0ccf00 2024-09-16T15:52:13.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.525+0000 7f8e68c00640 1 -- 172.21.15.151:0/913945941 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) ==== 94+0+3920 (secure 0 0 0) 0x7f8e580684e0 con 0x7f8e6c0ccf00 2024-09-16T15:52:13.526 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:52:13.526 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":69,"btime":"2024-09-16T15:52:06:997977+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6680,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2638998637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":2638998637},{"type":"v1","addr":"172.21.15.151:6835","nonce":2638998637}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":7330,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3105097249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3105097249},{"type":"v1","addr":"172.21.15.167:6836","nonce":3105097249}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":7333,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/613324294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":613324294},{"type":"v1","addr":"172.21.15.167:6837","nonce":613324294}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":8361,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/3752294407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3752294407},{"type":"v1","addr":"172.21.15.151:6834","nonce":3752294407}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69}],"filesystems":[]} 2024-09-16T15:52:13.527 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 69 2024-09-16T15:52:13.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.527+0000 7f8e73600640 1 -- 172.21.15.151:0/913945941 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8e4c07a2c0 msgr2=0x7f8e4c07c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:13.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.527+0000 7f8e73600640 1 --2- 172.21.15.151:0/913945941 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8e4c07a2c0 0x7f8e4c07c780 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f8e60033000 tx=0x7f8e60004730 comp rx=0 tx=0).stop 2024-09-16T15:52:13.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.527+0000 7f8e73600640 1 -- 172.21.15.151:0/913945941 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e6c0ccf00 msgr2=0x7f8e6c15e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:13.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.527+0000 7f8e73600640 1 --2- 172.21.15.151:0/913945941 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e6c0ccf00 0x7f8e6c15e660 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f8e5800c430 tx=0x7f8e5800c8f0 comp rx=0 tx=0).stop 2024-09-16T15:52:13.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.527+0000 7f8e73600640 1 -- 172.21.15.151:0/913945941 shutdown_connections 2024-09-16T15:52:13.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.527+0000 7f8e73600640 1 --2- 172.21.15.151:0/913945941 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8e4c07a2c0 0x7f8e4c07c780 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:13.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.527+0000 7f8e73600640 1 --2- 172.21.15.151:0/913945941 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8e6c15f5a0 0x7f8e6c1513a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:13.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.527+0000 7f8e73600640 1 --2- 172.21.15.151:0/913945941 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e6c0ccf00 0x7f8e6c15e660 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:13.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.527+0000 7f8e73600640 1 --2- 172.21.15.151:0/913945941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e6c073fb0 0x7f8e6c0cc750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:13.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.527+0000 7f8e73600640 1 -- 172.21.15.151:0/913945941 >> 172.21.15.151:0/913945941 conn(0x7f8e6c0b1950 msgr2=0x7f8e6c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:13.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.527+0000 7f8e73600640 1 -- 172.21.15.151:0/913945941 shutdown_connections 2024-09-16T15:52:13.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.527+0000 7f8e73600640 1 -- 172.21.15.151:0/913945941 wait complete. 2024-09-16T15:52:13.545 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-16T15:52:13.545 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-16T15:52:13.546 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-16T15:52:13.546 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:52:13.546 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-16T15:52:13.546 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-16T15:52:13.546 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-16T15:52:13.546 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:52:13.547 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-16T15:52:13.547 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-16T15:52:13.547 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-16T15:52:13.547 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:52:13.547 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-16T15:52:13.547 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-16T15:52:13.548 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-16T15:52:13.548 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:52:13.560 INFO:tasks.ceph.mds.a:Stopped 2024-09-16T15:52:13.560 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-16T15:52:13.562 INFO:tasks.ceph.mds.c:Stopped 2024-09-16T15:52:13.562 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-16T15:52:13.563 INFO:tasks.ceph.mds.b:Stopped 2024-09-16T15:52:13.564 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-16T15:52:13.564 INFO:tasks.ceph.mds.d:Stopped 2024-09-16T15:52:13.565 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-16T15:52:13.606 INFO:tasks.ceph.mds.a:Started 2024-09-16T15:52:13.607 DEBUG:teuthology.parallel:result is None 2024-09-16T15:52:13.608 INFO:tasks.ceph.mds.b:Started 2024-09-16T15:52:13.608 DEBUG:teuthology.parallel:result is None 2024-09-16T15:52:13.609 INFO:tasks.ceph.mds.c:Started 2024-09-16T15:52:13.610 DEBUG:teuthology.parallel:result is None 2024-09-16T15:52:13.610 INFO:tasks.ceph.mds.d:Started 2024-09-16T15:52:13.610 DEBUG:teuthology.parallel:result is None 2024-09-16T15:52:13.611 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-16T15:52:13.736 INFO:tasks.ceph.mds.a.smithi151.stdout:starting mds.a at 2024-09-16T15:52:13.736 INFO:tasks.ceph.mds.c.smithi151.stdout:starting mds.c at 2024-09-16T15:52:13.738 INFO:tasks.ceph.mds.b.smithi167.stdout:starting mds.b at 2024-09-16T15:52:13.738 INFO:tasks.ceph.mds.d.smithi167.stdout:starting mds.d at 2024-09-16T15:52:13.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.775+0000 7f253fa00640 1 -- 172.21.15.151:0/3545995340 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f25380af3d0 msgr2=0x7f253806f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:13.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.775+0000 7f253fa00640 1 --2- 172.21.15.151:0/3545995340 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f25380af3d0 0x7f253806f490 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f253806d9e0 tx=0x7f252002a4d0 comp rx=0 tx=0).stop 2024-09-16T15:52:13.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.775+0000 7f253fa00640 1 -- 172.21.15.151:0/3545995340 shutdown_connections 2024-09-16T15:52:13.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.775+0000 7f253fa00640 1 --2- 172.21.15.151:0/3545995340 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f253806f9d0 0x7f2538161c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:13.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.775+0000 7f253fa00640 1 --2- 172.21.15.151:0/3545995340 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f25380af3d0 0x7f253806f490 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:13.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.775+0000 7f253fa00640 1 --2- 172.21.15.151:0/3545995340 >> v2:172.21.15.167:3300/0 conn(0x7f2538161550 0x7f253806ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:13.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.775+0000 7f253fa00640 1 -- 172.21.15.151:0/3545995340 >> 172.21.15.151:0/3545995340 conn(0x7f25380b1950 msgr2=0x7f25380b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:13.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.775+0000 7f253fa00640 1 -- 172.21.15.151:0/3545995340 shutdown_connections 2024-09-16T15:52:13.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.775+0000 7f253fa00640 1 -- 172.21.15.151:0/3545995340 wait complete. 2024-09-16T15:52:13.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.776+0000 7f253fa00640 1 Processor -- start 2024-09-16T15:52:13.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.776+0000 7f253fa00640 1 -- start start 2024-09-16T15:52:13.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.777+0000 7f253fa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f253806f9d0 0x7f25381599c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:13.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.777+0000 7f253fa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25380af3d0 0x7f253814e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:13.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.777+0000 7f253fa00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2538161550 0x7f25380ced20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:13.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.777+0000 7f253fa00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2538162560 con 0x7f253806f9d0 2024-09-16T15:52:13.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.777+0000 7f253fa00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f25381623e0 con 0x7f25380af3d0 2024-09-16T15:52:13.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.777+0000 7f253fa00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f25381626e0 con 0x7f2538161550 2024-09-16T15:52:13.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.778+0000 7f253d000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f253806f9d0 0x7f25381599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:13.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.778+0000 7f253d000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f253806f9d0 0x7f25381599c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49752/0 (socket says 172.21.15.151:49752) 2024-09-16T15:52:13.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.778+0000 7f253d000640 1 -- 172.21.15.151:0/2252080206 learned_addr learned my addr 172.21.15.151:0/2252080206 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:52:13.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.778+0000 7f252fe00640 1 --2- 172.21.15.151:0/2252080206 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25380af3d0 0x7f253814e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:13.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.778+0000 7f253da00640 1 --2- 172.21.15.151:0/2252080206 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2538161550 0x7f25380ced20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:13.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.778+0000 7f253d000640 1 -- 172.21.15.151:0/2252080206 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2538161550 msgr2=0x7f25380ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:13.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.778+0000 7f253d000640 1 --2- 172.21.15.151:0/2252080206 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2538161550 0x7f25380ced20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:13.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.778+0000 7f253d000640 1 -- 172.21.15.151:0/2252080206 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25380af3d0 msgr2=0x7f253814e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:13.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.778+0000 7f253d000640 1 --2- 172.21.15.151:0/2252080206 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25380af3d0 0x7f253814e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:13.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.778+0000 7f253d000640 1 -- 172.21.15.151:0/2252080206 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25380cdc80 con 0x7f253806f9d0 2024-09-16T15:52:13.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.778+0000 7f253da00640 1 --2- 172.21.15.151:0/2252080206 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2538161550 0x7f25380ced20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:52:13.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.778+0000 7f252fe00640 1 --2- 172.21.15.151:0/2252080206 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25380af3d0 0x7f253814e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:52:13.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.778+0000 7f253d000640 1 --2- 172.21.15.151:0/2252080206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f253806f9d0 0x7f25381599c0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f252800e790 tx=0x7f252800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:13.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.778+0000 7f2535600640 1 -- 172.21.15.151:0/2252080206 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f252800eea0 con 0x7f253806f9d0 2024-09-16T15:52:13.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.779+0000 7f2535600640 1 -- 172.21.15.151:0/2252080206 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f252800cdb0 con 0x7f253806f9d0 2024-09-16T15:52:13.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.779+0000 7f2535600640 1 -- 172.21.15.151:0/2252080206 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2528010690 con 0x7f253806f9d0 2024-09-16T15:52:13.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.779+0000 7f253fa00640 1 -- 172.21.15.151:0/2252080206 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25380cdf60 con 0x7f253806f9d0 2024-09-16T15:52:13.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.779+0000 7f253fa00640 1 -- 172.21.15.151:0/2252080206 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25380ce520 con 0x7f253806f9d0 2024-09-16T15:52:13.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.780+0000 7f253fa00640 1 -- 172.21.15.151:0/2252080206 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25380b3310 con 0x7f253806f9d0 2024-09-16T15:52:13.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.781+0000 7f2535600640 1 -- 172.21.15.151:0/2252080206 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 41) ==== 103684+0+0 (secure 0 0 0) 0x7f2528010830 con 0x7f253806f9d0 2024-09-16T15:52:13.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.782+0000 7f2535600640 1 --2- 172.21.15.151:0/2252080206 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f251407a2c0 0x7f251407c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:13.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.782+0000 7f2535600640 1 -- 172.21.15.151:0/2252080206 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5027+0+0 (secure 0 0 0) 0x7f252809a210 con 0x7f253806f9d0 2024-09-16T15:52:13.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.782+0000 7f252fe00640 1 --2- 172.21.15.151:0/2252080206 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f251407a2c0 0x7f251407c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:13.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.783+0000 7f252fe00640 1 --2- 172.21.15.151:0/2252080206 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f251407a2c0 0x7f251407c780 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f2520033000 tx=0x7f2520004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:13.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.786+0000 7f2535600640 1 -- 172.21.15.151:0/2252080206 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f25380b3310 con 0x7f253806f9d0 2024-09-16T15:52:13.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:13.982+0000 7f253fa00640 1 -- 172.21.15.151:0/2252080206 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f2538161c90 con 0x7f253806f9d0 2024-09-16T15:52:14.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.046+0000 7f2535600640 1 -- 172.21.15.151:0/2252080206 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v112) ==== 97+0+0 (secure 0 0 0) 0x7f25280642e0 con 0x7f253806f9d0 2024-09-16T15:52:14.048 INFO:teuthology.orchestra.run.smithi151.stderr: removed all blocklist entries 2024-09-16T15:52:14.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.049+0000 7f253fa00640 1 -- 172.21.15.151:0/2252080206 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f251407a2c0 msgr2=0x7f251407c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:14.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.049+0000 7f253fa00640 1 --2- 172.21.15.151:0/2252080206 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f251407a2c0 0x7f251407c780 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f2520033000 tx=0x7f2520004730 comp rx=0 tx=0).stop 2024-09-16T15:52:14.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.049+0000 7f253fa00640 1 -- 172.21.15.151:0/2252080206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f253806f9d0 msgr2=0x7f25381599c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:14.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.049+0000 7f253fa00640 1 --2- 172.21.15.151:0/2252080206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f253806f9d0 0x7f25381599c0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f252800e790 tx=0x7f252800ec50 comp rx=0 tx=0).stop 2024-09-16T15:52:14.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.049+0000 7f253fa00640 1 -- 172.21.15.151:0/2252080206 shutdown_connections 2024-09-16T15:52:14.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.049+0000 7f253fa00640 1 --2- 172.21.15.151:0/2252080206 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f251407a2c0 0x7f251407c780 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:14.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.049+0000 7f253fa00640 1 --2- 172.21.15.151:0/2252080206 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25380af3d0 0x7f253814e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:14.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.049+0000 7f253fa00640 1 --2- 172.21.15.151:0/2252080206 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2538161550 0x7f25380ced20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:14.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.049+0000 7f253fa00640 1 --2- 172.21.15.151:0/2252080206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f253806f9d0 0x7f25381599c0 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:14.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.049+0000 7f253fa00640 1 -- 172.21.15.151:0/2252080206 >> 172.21.15.151:0/2252080206 conn(0x7f25380b1950 msgr2=0x7f253806f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:14.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.050+0000 7f253fa00640 1 -- 172.21.15.151:0/2252080206 shutdown_connections 2024-09-16T15:52:14.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.050+0000 7f253fa00640 1 -- 172.21.15.151:0/2252080206 wait complete. 2024-09-16T15:52:14.069 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-16T15:52:14.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.226+0000 7fc89f800640 1 -- 172.21.15.151:0/3492964350 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8980b4ca0 msgr2=0x7fc8980cf130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:14.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.226+0000 7fc89f800640 1 --2- 172.21.15.151:0/3492964350 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8980b4ca0 0x7fc8980cf130 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fc890009ca0 tx=0x7fc89002aac0 comp rx=0 tx=0).stop 2024-09-16T15:52:14.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.226+0000 7fc89f800640 1 -- 172.21.15.151:0/3492964350 shutdown_connections 2024-09-16T15:52:14.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.226+0000 7fc89f800640 1 --2- 172.21.15.151:0/3492964350 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8980b4ca0 0x7fc8980cf130 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:14.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.226+0000 7fc89f800640 1 --2- 172.21.15.151:0/3492964350 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc8981445d0 0x7fc8980cebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:14.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.226+0000 7fc89f800640 1 -- 172.21.15.151:0/3492964350 >> 172.21.15.151:0/3492964350 conn(0x7fc8980b1950 msgr2=0x7fc8980b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:14.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.226+0000 7fc89f800640 1 -- 172.21.15.151:0/3492964350 shutdown_connections 2024-09-16T15:52:14.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.226+0000 7fc89f800640 1 -- 172.21.15.151:0/3492964350 wait complete. 2024-09-16T15:52:14.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.228+0000 7fc89f800640 1 Processor -- start 2024-09-16T15:52:14.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.228+0000 7fc89f800640 1 -- start start 2024-09-16T15:52:14.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.229+0000 7fc89f800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8980b4ca0 0x7fc8981f8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:14.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.229+0000 7fc89f800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc8980ccfd0 0x7fc8980adf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:14.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.229+0000 7fc89f800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8981445d0 0x7fc89813cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:14.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.229+0000 7fc89f800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc89815f080 con 0x7fc8981445d0 2024-09-16T15:52:14.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.229+0000 7fc89f800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc898073bc0 con 0x7fc8980b4ca0 2024-09-16T15:52:14.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.229+0000 7fc89f800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fc89815f200 con 0x7fc8980ccfd0 2024-09-16T15:52:14.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.229+0000 7fc89d800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8981445d0 0x7fc89813cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:14.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.229+0000 7fc89d800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8981445d0 0x7fc89813cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49764/0 (socket says 172.21.15.151:49764) 2024-09-16T15:52:14.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.229+0000 7fc89d800640 1 -- 172.21.15.151:0/113112767 learned_addr learned my addr 172.21.15.151:0/113112767 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:52:14.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.229+0000 7fc89ce00640 1 --2- 172.21.15.151:0/113112767 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8980b4ca0 0x7fc8981f8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:14.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.229+0000 7fc897e00640 1 --2- 172.21.15.151:0/113112767 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc8980ccfd0 0x7fc8980adf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:14.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.230+0000 7fc89d800640 1 -- 172.21.15.151:0/113112767 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc8980ccfd0 msgr2=0x7fc8980adf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:14.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.230+0000 7fc89d800640 1 --2- 172.21.15.151:0/113112767 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc8980ccfd0 0x7fc8980adf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:14.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.230+0000 7fc89d800640 1 -- 172.21.15.151:0/113112767 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8980b4ca0 msgr2=0x7fc8981f8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:14.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.230+0000 7fc89d800640 1 --2- 172.21.15.151:0/113112767 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8980b4ca0 0x7fc8981f8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:14.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.230+0000 7fc89d800640 1 -- 172.21.15.151:0/113112767 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8981f9180 con 0x7fc8981445d0 2024-09-16T15:52:14.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.230+0000 7fc89ce00640 1 --2- 172.21.15.151:0/113112767 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8980b4ca0 0x7fc8981f8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:52:14.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.230+0000 7fc89d800640 1 --2- 172.21.15.151:0/113112767 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8981445d0 0x7fc89813cd20 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fc890006010 tx=0x7fc890003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:14.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.230+0000 7fc894c00640 1 -- 172.21.15.151:0/113112767 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc890046070 con 0x7fc8981445d0 2024-09-16T15:52:14.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.230+0000 7fc894c00640 1 -- 172.21.15.151:0/113112767 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc890004040 con 0x7fc8981445d0 2024-09-16T15:52:14.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.230+0000 7fc89f800640 1 -- 172.21.15.151:0/113112767 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc89806e790 con 0x7fc8981445d0 2024-09-16T15:52:14.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.230+0000 7fc894c00640 1 -- 172.21.15.151:0/113112767 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc890004430 con 0x7fc8981445d0 2024-09-16T15:52:14.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.230+0000 7fc89f800640 1 -- 172.21.15.151:0/113112767 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc89806ed50 con 0x7fc8981445d0 2024-09-16T15:52:14.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.232+0000 7fc89f800640 1 -- 172.21.15.151:0/113112767 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc89815f6b0 con 0x7fc8981445d0 2024-09-16T15:52:14.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.232+0000 7fc894c00640 1 -- 172.21.15.151:0/113112767 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 41) ==== 103684+0+0 (secure 0 0 0) 0x7fc8900045d0 con 0x7fc8981445d0 2024-09-16T15:52:14.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.233+0000 7fc894c00640 1 --2- 172.21.15.151:0/113112767 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc87807a2c0 0x7fc87807c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:14.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.234+0000 7fc894c00640 1 -- 172.21.15.151:0/113112767 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 4941+0+0 (secure 0 0 0) 0x7fc8900ba460 con 0x7fc8981445d0 2024-09-16T15:52:14.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.234+0000 7fc89ce00640 1 --2- 172.21.15.151:0/113112767 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc87807a2c0 0x7fc87807c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:14.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.234+0000 7fc89ce00640 1 --2- 172.21.15.151:0/113112767 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc87807a2c0 0x7fc87807c780 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fc8880095f0 tx=0x7fc888005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:14.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.238+0000 7fc894c00640 1 -- 172.21.15.151:0/113112767 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc8981f9180 con 0x7fc8981445d0 2024-09-16T15:52:14.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.490+0000 7fc89f800640 1 -- 172.21.15.151:0/113112767 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fc898070a20 con 0x7fc8981445d0 2024-09-16T15:52:14.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.490+0000 7fc894c00640 1 -- 172.21.15.151:0/113112767 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) ==== 80+0+5304 (secure 0 0 0) 0x7fc890032030 con 0x7fc8981445d0 2024-09-16T15:52:14.491 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:52:14.492 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-16T15:52:14.492 INFO:teuthology.orchestra.run.smithi151.stdout: "auth_dump": [ 2024-09-16T15:52:14.492 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:52:14.492 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "mds.a", 2024-09-16T15:52:14.492 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDsUOhmJVK6AhAANfmTPS6kKn41LSD/MuyW9g==", 2024-09-16T15:52:14.492 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:52:14.492 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:52:14.492 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow *", 2024-09-16T15:52:14.492 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow *", 2024-09-16T15:52:14.492 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:52:14.492 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:52:14.492 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:52:14.492 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:52:14.493 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "mds.b", 2024-09-16T15:52:14.493 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDsUOhmX1R3FhAAUQgUsE/6wUyokXcZuBuIWg==", 2024-09-16T15:52:14.493 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:52:14.493 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:52:14.493 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow *", 2024-09-16T15:52:14.493 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow *", 2024-09-16T15:52:14.493 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:52:14.493 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:52:14.493 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:52:14.493 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:52:14.493 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "mds.c", 2024-09-16T15:52:14.493 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDsUOhmYGCKDBAAY2EFNmiFNf1x2so9wxp4YQ==", 2024-09-16T15:52:14.493 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:52:14.493 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:52:14.494 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow *", 2024-09-16T15:52:14.494 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow *", 2024-09-16T15:52:14.494 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:52:14.494 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:52:14.494 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:52:14.494 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:52:14.494 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "mds.d", 2024-09-16T15:52:14.494 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDsUOhmfCk4IBAAx12Gw2QudQ2nrPb8VlE0SA==", 2024-09-16T15:52:14.494 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:52:14.494 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:52:14.494 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow *", 2024-09-16T15:52:14.494 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow *", 2024-09-16T15:52:14.494 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:52:14.494 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:52:14.495 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:52:14.495 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:52:14.495 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.0", 2024-09-16T15:52:14.495 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDvUOhmKji5NxAAn/E3YQR7jeMnBTLBXc1G6A==", 2024-09-16T15:52:14.495 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:52:14.495 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:52:14.495 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:52:14.495 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:52:14.495 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:52:14.495 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:52:14.495 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:52:14.495 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.1", 2024-09-16T15:52:14.495 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDzUOhmmBrNBhAA0Uwh4Fam2bmKKvz5jAQBWw==", 2024-09-16T15:52:14.496 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:52:14.496 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:52:14.496 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:52:14.496 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:52:14.496 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:52:14.496 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:52:14.496 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:52:14.496 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.2", 2024-09-16T15:52:14.496 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQD2UOhmOV6GERAAjD+PQLTZrCFR1669UEEIAQ==", 2024-09-16T15:52:14.496 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:52:14.496 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:52:14.496 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:52:14.496 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:52:14.497 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:52:14.497 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:52:14.497 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:52:14.497 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.3", 2024-09-16T15:52:14.497 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQD5UOhmCRrdGRAAQC0bqWX6N1jniVGEQ2aAvg==", 2024-09-16T15:52:14.497 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:52:14.497 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:52:14.497 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:52:14.497 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:52:14.497 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:52:14.497 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:52:14.497 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:52:14.498 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.4", 2024-09-16T15:52:14.498 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQD/UOhmXI/nGRAA7t6XveKvrWgqy6rHuMQSZQ==", 2024-09-16T15:52:14.498 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:52:14.498 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:52:14.498 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:52:14.498 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:52:14.498 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:52:14.498 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:52:14.498 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:52:14.498 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.5", 2024-09-16T15:52:14.498 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQACUehmKM//IxAAsoFpiiTXNs//v3YXeUf/Iw==", 2024-09-16T15:52:14.498 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:52:14.499 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:52:14.499 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:52:14.499 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:52:14.499 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:52:14.499 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:52:14.499 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:52:14.499 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.6", 2024-09-16T15:52:14.499 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQAFUehmlo/FLRAAp5mvUSWCaBbm85QyTZY0eA==", 2024-09-16T15:52:14.499 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:52:14.499 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:52:14.499 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:52:14.499 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:52:14.499 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:52:14.499 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:52:14.500 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:52:14.500 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.7", 2024-09-16T15:52:14.500 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQAIUehmR32JNxAA27antOIKot1nScQbLZDf5g==", 2024-09-16T15:52:14.500 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:52:14.500 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:52:14.500 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:52:14.500 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:52:14.500 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:52:14.500 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:52:14.500 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:52:14.500 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "client.0", 2024-09-16T15:52:14.500 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDsUOhmf5PBKBAAKE4Y4zbA4h07IBNjPkyptw==", 2024-09-16T15:52:14.500 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:52:14.501 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:52:14.501 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow r", 2024-09-16T15:52:14.501 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-16T15:52:14.501 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:52:14.501 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:52:14.501 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:52:14.501 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "client.1", 2024-09-16T15:52:14.501 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDsUOhm15emNRAAwhK7QmA2c9IYGn04ERvY0g==", 2024-09-16T15:52:14.501 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:52:14.501 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:52:14.501 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow r", 2024-09-16T15:52:14.501 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow rw", 2024-09-16T15:52:14.501 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow rwx" 2024-09-16T15:52:14.501 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:52:14.502 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:52:14.502 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:52:14.502 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "client.2", 2024-09-16T15:52:14.502 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDsUOhm6zUyLxAAkzeiIQTuNZkMM+Oit9rC5A==", 2024-09-16T15:52:14.502 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:52:14.502 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:52:14.502 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow r", 2024-09-16T15:52:14.502 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow rw", 2024-09-16T15:52:14.502 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow rwx" 2024-09-16T15:52:14.502 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:52:14.502 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:52:14.502 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:52:14.502 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "client.3", 2024-09-16T15:52:14.502 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDtUOhmD82EABAAeDf82/hPxpPfAUk8oj5h/Q==", 2024-09-16T15:52:14.502 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:52:14.503 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:52:14.503 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow r", 2024-09-16T15:52:14.503 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow rw", 2024-09-16T15:52:14.503 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow rwx" 2024-09-16T15:52:14.503 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:52:14.503 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:52:14.503 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:52:14.503 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "client.admin", 2024-09-16T15:52:14.503 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDrUOhmQFmVDRAA8NKysbAe475xvg4CPAcFyw==", 2024-09-16T15:52:14.503 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:52:14.503 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow *", 2024-09-16T15:52:14.503 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow *", 2024-09-16T15:52:14.503 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow *", 2024-09-16T15:52:14.503 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:52:14.504 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:52:14.504 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:52:14.504 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:52:14.504 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "mgr.x", 2024-09-16T15:52:14.504 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDrUOhmloW3NxAA8UoLUfrw3wf1nzKbmtK/jw==", 2024-09-16T15:52:14.504 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:52:14.504 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow *", 2024-09-16T15:52:14.504 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile mgr", 2024-09-16T15:52:14.504 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:52:14.504 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:52:14.504 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:52:14.504 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:52:14.504 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "mgr.y", 2024-09-16T15:52:14.504 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDrUOhm1uZOLxAATgjA79vYf0psqQVo03sdDA==", 2024-09-16T15:52:14.505 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:52:14.505 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow *", 2024-09-16T15:52:14.505 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile mgr", 2024-09-16T15:52:14.505 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:52:14.505 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:52:14.505 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:52:14.505 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-09-16T15:52:14.505 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-16T15:52:14.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.492+0000 7fc89f800640 1 -- 172.21.15.151:0/113112767 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc87807a2c0 msgr2=0x7fc87807c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:14.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.492+0000 7fc89f800640 1 --2- 172.21.15.151:0/113112767 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc87807a2c0 0x7fc87807c780 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fc8880095f0 tx=0x7fc888005f90 comp rx=0 tx=0).stop 2024-09-16T15:52:14.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.492+0000 7fc89f800640 1 -- 172.21.15.151:0/113112767 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8981445d0 msgr2=0x7fc89813cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:14.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.492+0000 7fc89f800640 1 --2- 172.21.15.151:0/113112767 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8981445d0 0x7fc89813cd20 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fc890006010 tx=0x7fc890003ce0 comp rx=0 tx=0).stop 2024-09-16T15:52:14.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.492+0000 7fc89f800640 1 -- 172.21.15.151:0/113112767 shutdown_connections 2024-09-16T15:52:14.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.492+0000 7fc89f800640 1 --2- 172.21.15.151:0/113112767 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc87807a2c0 0x7fc87807c780 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:14.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.492+0000 7fc89f800640 1 --2- 172.21.15.151:0/113112767 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8981445d0 0x7fc89813cd20 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:14.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.492+0000 7fc89f800640 1 --2- 172.21.15.151:0/113112767 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc8980ccfd0 0x7fc8980adf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:14.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.492+0000 7fc89f800640 1 --2- 172.21.15.151:0/113112767 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8980b4ca0 0x7fc8981f8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:14.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.493+0000 7fc89f800640 1 -- 172.21.15.151:0/113112767 >> 172.21.15.151:0/113112767 conn(0x7fc8980b1950 msgr2=0x7fc8980acd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:14.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.493+0000 7fc89f800640 1 -- 172.21.15.151:0/113112767 shutdown_connections 2024-09-16T15:52:14.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.493+0000 7fc89f800640 1 -- 172.21.15.151:0/113112767 wait complete. 2024-09-16T15:52:14.510 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-16T15:52:14.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.666+0000 7fc734400640 1 -- 172.21.15.151:0/4150172439 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc72c073fb0 msgr2=0x7fc72c074440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:14.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.666+0000 7fc734400640 1 --2- 172.21.15.151:0/4150172439 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc72c073fb0 0x7fc72c074440 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc724005fd0 tx=0x7fc72402a820 comp rx=0 tx=0).stop 2024-09-16T15:52:14.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.666+0000 7fc734400640 1 -- 172.21.15.151:0/4150172439 shutdown_connections 2024-09-16T15:52:14.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.666+0000 7fc734400640 1 --2- 172.21.15.151:0/4150172439 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc72c073fb0 0x7fc72c074440 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:14.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.666+0000 7fc734400640 1 --2- 172.21.15.151:0/4150172439 >> v2:172.21.15.167:3300/0 conn(0x7fc72c15f5a0 0x7fc72c0b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:14.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.666+0000 7fc734400640 1 -- 172.21.15.151:0/4150172439 >> 172.21.15.151:0/4150172439 conn(0x7fc72c0b1950 msgr2=0x7fc72c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:14.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.667+0000 7fc734400640 1 -- 172.21.15.151:0/4150172439 shutdown_connections 2024-09-16T15:52:14.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.667+0000 7fc734400640 1 -- 172.21.15.151:0/4150172439 wait complete. 2024-09-16T15:52:14.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.668+0000 7fc734400640 1 Processor -- start 2024-09-16T15:52:14.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.668+0000 7fc734400640 1 -- start start 2024-09-16T15:52:14.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.669+0000 7fc734400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc72c073fb0 0x7fc72c1f7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:14.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.669+0000 7fc734400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc72c0ccf00 0x7fc72c153eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:14.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.669+0000 7fc734400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc72c15f5a0 0x7fc72c1f50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:14.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.669+0000 7fc734400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc72c06f2e0 con 0x7fc72c15f5a0 2024-09-16T15:52:14.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.669+0000 7fc734400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc72c06f160 con 0x7fc72c073fb0 2024-09-16T15:52:14.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.669+0000 7fc734400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fc72c06f460 con 0x7fc72c0ccf00 2024-09-16T15:52:14.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.670+0000 7fc732400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc72c15f5a0 0x7fc72c1f50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:14.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.670+0000 7fc732400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc72c15f5a0 0x7fc72c1f50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49774/0 (socket says 172.21.15.151:49774) 2024-09-16T15:52:14.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.670+0000 7fc732400640 1 -- 172.21.15.151:0/4060245845 learned_addr learned my addr 172.21.15.151:0/4060245845 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:52:14.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.670+0000 7fc731a00640 1 --2- 172.21.15.151:0/4060245845 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc72c073fb0 0x7fc72c1f7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:14.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.670+0000 7fc731000640 1 --2- 172.21.15.151:0/4060245845 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc72c0ccf00 0x7fc72c153eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:14.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.670+0000 7fc732400640 1 -- 172.21.15.151:0/4060245845 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc72c0ccf00 msgr2=0x7fc72c153eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:14.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.670+0000 7fc732400640 1 --2- 172.21.15.151:0/4060245845 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc72c0ccf00 0x7fc72c153eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:14.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.670+0000 7fc732400640 1 -- 172.21.15.151:0/4060245845 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc72c073fb0 msgr2=0x7fc72c1f7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:14.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.670+0000 7fc732400640 1 --2- 172.21.15.151:0/4060245845 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc72c073fb0 0x7fc72c1f7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:14.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.670+0000 7fc732400640 1 -- 172.21.15.151:0/4060245845 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc72c1f86b0 con 0x7fc72c15f5a0 2024-09-16T15:52:14.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.670+0000 7fc732400640 1 --2- 172.21.15.151:0/4060245845 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc72c15f5a0 0x7fc72c1f50a0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fc724035040 tx=0x7fc724032c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:14.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.670+0000 7fc729600640 1 -- 172.21.15.151:0/4060245845 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc724037040 con 0x7fc72c15f5a0 2024-09-16T15:52:14.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.671+0000 7fc734400640 1 -- 172.21.15.151:0/4060245845 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc72c1f8990 con 0x7fc72c15f5a0 2024-09-16T15:52:14.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.671+0000 7fc729600640 1 -- 172.21.15.151:0/4060245845 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc724042070 con 0x7fc72c15f5a0 2024-09-16T15:52:14.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.671+0000 7fc729600640 1 -- 172.21.15.151:0/4060245845 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc724003e60 con 0x7fc72c15f5a0 2024-09-16T15:52:14.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.671+0000 7fc734400640 1 -- 172.21.15.151:0/4060245845 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc72c1f4b80 con 0x7fc72c15f5a0 2024-09-16T15:52:14.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.672+0000 7fc734400640 1 -- 172.21.15.151:0/4060245845 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc72c0cf5e0 con 0x7fc72c15f5a0 2024-09-16T15:52:14.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.673+0000 7fc729600640 1 -- 172.21.15.151:0/4060245845 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 41) ==== 103684+0+0 (secure 0 0 0) 0x7fc724004260 con 0x7fc72c15f5a0 2024-09-16T15:52:14.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.674+0000 7fc729600640 1 --2- 172.21.15.151:0/4060245845 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc70c07a2c0 0x7fc70c07c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:14.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.674+0000 7fc729600640 1 -- 172.21.15.151:0/4060245845 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 4941+0+0 (secure 0 0 0) 0x7fc7240ba280 con 0x7fc72c15f5a0 2024-09-16T15:52:14.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.674+0000 7fc731a00640 1 --2- 172.21.15.151:0/4060245845 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc70c07a2c0 0x7fc70c07c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:14.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.675+0000 7fc731a00640 1 --2- 172.21.15.151:0/4060245845 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc70c07a2c0 0x7fc70c07c780 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fc72c1550c0 tx=0x7fc720005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:14.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.678+0000 7fc729600640 1 -- 172.21.15.151:0/4060245845 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc72c0cf5e0 con 0x7fc72c15f5a0 2024-09-16T15:52:14.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.863+0000 7fc734400640 1 -- 172.21.15.151:0/4060245845 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fc72c06f9f0 con 0x7fc72c15f5a0 2024-09-16T15:52:14.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.864+0000 7fc729600640 1 -- 172.21.15.151:0/4060245845 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v114) ==== 84+0+65 (secure 0 0 0) 0x7fc724083d00 con 0x7fc72c15f5a0 2024-09-16T15:52:14.866 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:52:14.866 INFO:teuthology.orchestra.run.smithi151.stdout:[ 2024-09-16T15:52:14.866 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:52:14.866 INFO:teuthology.orchestra.run.smithi151.stdout: "poolnum": 1, 2024-09-16T15:52:14.866 INFO:teuthology.orchestra.run.smithi151.stdout: "poolname": ".mgr" 2024-09-16T15:52:14.867 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:52:14.867 INFO:teuthology.orchestra.run.smithi151.stdout:] 2024-09-16T15:52:14.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.866+0000 7fc734400640 1 -- 172.21.15.151:0/4060245845 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc70c07a2c0 msgr2=0x7fc70c07c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:14.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.866+0000 7fc734400640 1 --2- 172.21.15.151:0/4060245845 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc70c07a2c0 0x7fc70c07c780 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fc72c1550c0 tx=0x7fc720005f90 comp rx=0 tx=0).stop 2024-09-16T15:52:14.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.866+0000 7fc734400640 1 -- 172.21.15.151:0/4060245845 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc72c15f5a0 msgr2=0x7fc72c1f50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:14.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.866+0000 7fc734400640 1 --2- 172.21.15.151:0/4060245845 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc72c15f5a0 0x7fc72c1f50a0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fc724035040 tx=0x7fc724032c60 comp rx=0 tx=0).stop 2024-09-16T15:52:14.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.866+0000 7fc734400640 1 -- 172.21.15.151:0/4060245845 shutdown_connections 2024-09-16T15:52:14.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.866+0000 7fc734400640 1 --2- 172.21.15.151:0/4060245845 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc70c07a2c0 0x7fc70c07c780 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:14.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.866+0000 7fc734400640 1 --2- 172.21.15.151:0/4060245845 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc72c15f5a0 0x7fc72c1f50a0 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:14.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.866+0000 7fc734400640 1 --2- 172.21.15.151:0/4060245845 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc72c0ccf00 0x7fc72c153eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:14.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.866+0000 7fc734400640 1 --2- 172.21.15.151:0/4060245845 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc72c073fb0 0x7fc72c1f7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:14.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.866+0000 7fc734400640 1 -- 172.21.15.151:0/4060245845 >> 172.21.15.151:0/4060245845 conn(0x7fc72c0b1950 msgr2=0x7fc72c0b0d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:14.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.866+0000 7fc734400640 1 -- 172.21.15.151:0/4060245845 shutdown_connections 2024-09-16T15:52:14.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:14.866+0000 7fc734400640 1 -- 172.21.15.151:0/4060245845 wait complete. 2024-09-16T15:52:14.885 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-16T15:52:14.885 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-16T15:52:15.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:15.045+0000 7f9b71600640 1 -- 172.21.15.151:0/1966243591 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b6c074460 msgr2=0x7f9b6c0cc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:15.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:15.045+0000 7f9b71600640 1 --2- 172.21.15.151:0/1966243591 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b6c074460 0x7f9b6c0cc5b0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f9b64005fd0 tx=0x7f9b6402aaa0 comp rx=0 tx=0).stop 2024-09-16T15:52:15.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:15.045+0000 7f9b71600640 1 -- 172.21.15.151:0/1966243591 shutdown_connections 2024-09-16T15:52:15.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:15.045+0000 7f9b71600640 1 --2- 172.21.15.151:0/1966243591 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b6c074460 0x7f9b6c0cc5b0 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:15.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:15.045+0000 7f9b71600640 1 --2- 172.21.15.151:0/1966243591 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9b6c15bd60 0x7f9b6c0cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:15.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:15.045+0000 7f9b71600640 1 -- 172.21.15.151:0/1966243591 >> 172.21.15.151:0/1966243591 conn(0x7f9b6c0b18e0 msgr2=0x7f9b6c0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:15.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:15.045+0000 7f9b71600640 1 -- 172.21.15.151:0/1966243591 shutdown_connections 2024-09-16T15:52:15.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:15.045+0000 7f9b71600640 1 -- 172.21.15.151:0/1966243591 wait complete. 2024-09-16T15:52:15.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:15.048+0000 7f9b71600640 1 Processor -- start 2024-09-16T15:52:15.050 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:15.048+0000 7f9b71600640 1 -- start start 2024-09-16T15:52:15.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:15.049+0000 7f9b71600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b6c06e910 0x7f9b6c155140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:15.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:15.049+0000 7f9b71600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b6c074460 0x7f9b6c155860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:15.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:15.049+0000 7f9b71600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9b6c15bd60 0x7f9b6c155da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:15.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:15.049+0000 7f9b71600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9b6c0ccc70 con 0x7f9b6c074460 2024-09-16T15:52:15.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:15.049+0000 7f9b71600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9b6c0ccaf0 con 0x7f9b6c06e910 2024-09-16T15:52:15.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:15.049+0000 7f9b71600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f9b6c0ccdf0 con 0x7f9b6c15bd60 2024-09-16T15:52:15.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:15.049+0000 7f9b6a000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b6c074460 0x7f9b6c155860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:15.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:15.049+0000 7f9b6a000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b6c074460 0x7f9b6c155860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49788/0 (socket says 172.21.15.151:49788) 2024-09-16T15:52:15.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:15.049+0000 7f9b6a000640 1 -- 172.21.15.151:0/1197957020 learned_addr learned my addr 172.21.15.151:0/1197957020 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:52:15.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:15.049+0000 7f9b6aa00640 1 --2- 172.21.15.151:0/1197957020 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b6c06e910 0x7f9b6c155140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:15.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:15.049+0000 7f9b6b400640 1 --2- 172.21.15.151:0/1197957020 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9b6c15bd60 0x7f9b6c155da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:15.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:15.050+0000 7f9b6a000640 1 -- 172.21.15.151:0/1197957020 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9b6c15bd60 msgr2=0x7f9b6c155da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:15.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:15.050+0000 7f9b6a000640 1 --2- 172.21.15.151:0/1197957020 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9b6c15bd60 0x7f9b6c155da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:15.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:15.050+0000 7f9b6a000640 1 -- 172.21.15.151:0/1197957020 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b6c06e910 msgr2=0x7f9b6c155140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:15.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:15.050+0000 7f9b6a000640 1 --2- 172.21.15.151:0/1197957020 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b6c06e910 0x7f9b6c155140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:15.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:15.050+0000 7f9b6a000640 1 -- 172.21.15.151:0/1197957020 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b6c0c9bb0 con 0x7f9b6c074460 2024-09-16T15:52:15.054 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:15.050+0000 7f9b6a000640 1 --2- 172.21.15.151:0/1197957020 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b6c074460 0x7f9b6c155860 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f9b5800c450 tx=0x7f9b5800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:15.054 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:15.051+0000 7f9b62a00640 1 -- 172.21.15.151:0/1197957020 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b580052d0 con 0x7f9b6c074460 2024-09-16T15:52:15.054 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:15.051+0000 7f9b62a00640 1 -- 172.21.15.151:0/1197957020 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9b58005470 con 0x7f9b6c074460 2024-09-16T15:52:15.054 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:15.051+0000 7f9b71600640 1 -- 172.21.15.151:0/1197957020 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b6c0c9e90 con 0x7f9b6c074460 2024-09-16T15:52:15.054 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:15.051+0000 7f9b62a00640 1 -- 172.21.15.151:0/1197957020 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b58012870 con 0x7f9b6c074460 2024-09-16T15:52:15.054 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:15.052+0000 7f9b71600640 1 -- 172.21.15.151:0/1197957020 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b6c0ca450 con 0x7f9b6c074460 2024-09-16T15:52:15.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:15.053+0000 7f9b71600640 1 -- 172.21.15.151:0/1197957020 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b6c069360 con 0x7f9b6c074460 2024-09-16T15:52:15.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:15.054+0000 7f9b62a00640 1 -- 172.21.15.151:0/1197957020 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 41) ==== 103684+0+0 (secure 0 0 0) 0x7f9b58012a10 con 0x7f9b6c074460 2024-09-16T15:52:15.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:15.055+0000 7f9b62a00640 1 --2- 172.21.15.151:0/1197957020 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9b4c07a2f0 0x7f9b4c07c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:15.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:15.055+0000 7f9b62a00640 1 -- 172.21.15.151:0/1197957020 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 4941+0+0 (secure 0 0 0) 0x7f9b580991d0 con 0x7f9b6c074460 2024-09-16T15:52:15.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:15.055+0000 7f9b6aa00640 1 --2- 172.21.15.151:0/1197957020 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9b4c07a2f0 0x7f9b4c07c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:15.058 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:15.056+0000 7f9b6aa00640 1 --2- 172.21.15.151:0/1197957020 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9b4c07a2f0 0x7f9b4c07c7b0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c0cc6f0 tx=0x7f9b5c005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:15.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:15.059+0000 7f9b62a00640 1 -- 172.21.15.151:0/1197957020 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9b6c15c140 con 0x7f9b6c074460 2024-09-16T15:52:15.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:15.244+0000 7f9b71600640 1 -- 172.21.15.151:0/1197957020 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f9b6c0cd2f0 con 0x7f9b6c074460 2024-09-16T15:52:16.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:16.057+0000 7f9b62a00640 1 -- 172.21.15.151:0/1197957020 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v115) ==== 138+0+0 (secure 0 0 0) 0x7f9b58062c50 con 0x7f9b6c074460 2024-09-16T15:52:16.059 INFO:teuthology.orchestra.run.smithi151.stderr:pool 'cephfs_metadata' created 2024-09-16T15:52:16.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:16.059+0000 7f9b71600640 1 -- 172.21.15.151:0/1197957020 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9b4c07a2f0 msgr2=0x7f9b4c07c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:16.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:16.059+0000 7f9b71600640 1 --2- 172.21.15.151:0/1197957020 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9b4c07a2f0 0x7f9b4c07c7b0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c0cc6f0 tx=0x7f9b5c005f90 comp rx=0 tx=0).stop 2024-09-16T15:52:16.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:16.059+0000 7f9b71600640 1 -- 172.21.15.151:0/1197957020 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b6c074460 msgr2=0x7f9b6c155860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:16.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:16.059+0000 7f9b71600640 1 --2- 172.21.15.151:0/1197957020 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b6c074460 0x7f9b6c155860 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f9b5800c450 tx=0x7f9b5800c910 comp rx=0 tx=0).stop 2024-09-16T15:52:16.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:16.059+0000 7f9b71600640 1 -- 172.21.15.151:0/1197957020 shutdown_connections 2024-09-16T15:52:16.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:16.059+0000 7f9b71600640 1 --2- 172.21.15.151:0/1197957020 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9b4c07a2f0 0x7f9b4c07c7b0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:16.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:16.059+0000 7f9b71600640 1 --2- 172.21.15.151:0/1197957020 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9b6c15bd60 0x7f9b6c155da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:16.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:16.059+0000 7f9b71600640 1 --2- 172.21.15.151:0/1197957020 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9b6c074460 0x7f9b6c155860 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:16.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:16.059+0000 7f9b71600640 1 --2- 172.21.15.151:0/1197957020 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9b6c06e910 0x7f9b6c155140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:16.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:16.059+0000 7f9b71600640 1 -- 172.21.15.151:0/1197957020 >> 172.21.15.151:0/1197957020 conn(0x7f9b6c0b18e0 msgr2=0x7f9b6c073d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:16.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:16.060+0000 7f9b71600640 1 -- 172.21.15.151:0/1197957020 shutdown_connections 2024-09-16T15:52:16.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:16.060+0000 7f9b71600640 1 -- 172.21.15.151:0/1197957020 wait complete. 2024-09-16T15:52:16.081 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-16T15:52:16.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:16.246+0000 7fcc45e00640 1 -- 172.21.15.151:0/2901524574 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcc400cccd0 msgr2=0x7fcc400cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:16.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:16.246+0000 7fcc45e00640 1 --2- 172.21.15.151:0/2901524574 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcc400cccd0 0x7fcc400cdbb0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fcc400d0cd0 tx=0x7fcc3802a4d0 comp rx=0 tx=0).stop 2024-09-16T15:52:16.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:16.246+0000 7fcc45e00640 1 -- 172.21.15.151:0/2901524574 shutdown_connections 2024-09-16T15:52:16.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:16.246+0000 7fcc45e00640 1 --2- 172.21.15.151:0/2901524574 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcc400cccd0 0x7fcc400cdbb0 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:16.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:16.246+0000 7fcc45e00640 1 --2- 172.21.15.151:0/2901524574 >> v2:172.21.15.167:3300/0 conn(0x7fcc40145750 0x7fcc40073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:16.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:16.246+0000 7fcc45e00640 1 -- 172.21.15.151:0/2901524574 >> 172.21.15.151:0/2901524574 conn(0x7fcc400b1840 msgr2=0x7fcc400b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:16.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:16.246+0000 7fcc45e00640 1 -- 172.21.15.151:0/2901524574 shutdown_connections 2024-09-16T15:52:16.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:16.246+0000 7fcc45e00640 1 -- 172.21.15.151:0/2901524574 wait complete. 2024-09-16T15:52:16.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:16.248+0000 7fcc45e00640 1 Processor -- start 2024-09-16T15:52:16.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:16.248+0000 7fcc45e00640 1 -- start start 2024-09-16T15:52:16.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:16.249+0000 7fcc45e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc4006efa0 0x7fcc400c61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:16.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:16.249+0000 7fcc45e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcc400cccd0 0x7fcc4015e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:16.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:16.249+0000 7fcc45e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcc40145750 0x7fcc401433c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:16.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:16.249+0000 7fcc45e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcc400ce480 con 0x7fcc400cccd0 2024-09-16T15:52:16.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:16.249+0000 7fcc45e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcc400ce300 con 0x7fcc4006efa0 2024-09-16T15:52:16.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:16.249+0000 7fcc45e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fcc400ce600 con 0x7fcc40145750 2024-09-16T15:52:16.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:16.249+0000 7fcc3e000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcc400cccd0 0x7fcc4015e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:16.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:16.249+0000 7fcc3e000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcc400cccd0 0x7fcc4015e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49814/0 (socket says 172.21.15.151:49814) 2024-09-16T15:52:16.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:16.249+0000 7fcc3e000640 1 -- 172.21.15.151:0/129803788 learned_addr learned my addr 172.21.15.151:0/129803788 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:52:16.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:16.249+0000 7fcc3ea00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc4006efa0 0x7fcc400c61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:16.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:16.250+0000 7fcc3f400640 1 --2- 172.21.15.151:0/129803788 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcc40145750 0x7fcc401433c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:16.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:16.250+0000 7fcc3e000640 1 -- 172.21.15.151:0/129803788 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcc40145750 msgr2=0x7fcc401433c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:16.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:16.250+0000 7fcc3e000640 1 --2- 172.21.15.151:0/129803788 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcc40145750 0x7fcc401433c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:16.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:16.250+0000 7fcc3e000640 1 -- 172.21.15.151:0/129803788 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc4006efa0 msgr2=0x7fcc400c61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:16.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:16.250+0000 7fcc3e000640 1 --2- 172.21.15.151:0/129803788 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc4006efa0 0x7fcc400c61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:16.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:16.250+0000 7fcc3e000640 1 -- 172.21.15.151:0/129803788 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc4013f800 con 0x7fcc400cccd0 2024-09-16T15:52:16.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:16.250+0000 7fcc3e000640 1 --2- 172.21.15.151:0/129803788 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcc400cccd0 0x7fcc4015e750 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c00c410 tx=0x7fcc2c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:16.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:16.250+0000 7fcc37400640 1 -- 172.21.15.151:0/129803788 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2c017070 con 0x7fcc400cccd0 2024-09-16T15:52:16.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:16.250+0000 7fcc37400640 1 -- 172.21.15.151:0/129803788 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcc2c0035f0 con 0x7fcc400cccd0 2024-09-16T15:52:16.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:16.250+0000 7fcc37400640 1 -- 172.21.15.151:0/129803788 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2c003a80 con 0x7fcc400cccd0 2024-09-16T15:52:16.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:16.250+0000 7fcc45e00640 1 -- 172.21.15.151:0/129803788 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc400c5430 con 0x7fcc400cccd0 2024-09-16T15:52:16.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:16.251+0000 7fcc45e00640 1 -- 172.21.15.151:0/129803788 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc400c59f0 con 0x7fcc400cccd0 2024-09-16T15:52:16.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:16.252+0000 7fcc45e00640 1 -- 172.21.15.151:0/129803788 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc400cdbb0 con 0x7fcc400cccd0 2024-09-16T15:52:16.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:16.253+0000 7fcc37400640 1 -- 172.21.15.151:0/129803788 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 41) ==== 103684+0+0 (secure 0 0 0) 0x7fcc2c003c20 con 0x7fcc400cccd0 2024-09-16T15:52:16.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:16.254+0000 7fcc37400640 1 --2- 172.21.15.151:0/129803788 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcc2007a2c0 0x7fcc2007c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:16.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:16.254+0000 7fcc37400640 1 -- 172.21.15.151:0/129803788 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5311+0+0 (secure 0 0 0) 0x7fcc2c098d40 con 0x7fcc400cccd0 2024-09-16T15:52:16.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:16.254+0000 7fcc3ea00640 1 --2- 172.21.15.151:0/129803788 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcc2007a2c0 0x7fcc2007c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:16.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:16.255+0000 7fcc3ea00640 1 --2- 172.21.15.151:0/129803788 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcc2007a2c0 0x7fcc2007c780 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fcc3000ae50 tx=0x7fcc3000a540 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:16.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:16.257+0000 7fcc37400640 1 -- 172.21.15.151:0/129803788 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fcc2c062840 con 0x7fcc400cccd0 2024-09-16T15:52:16.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:16.445+0000 7fcc45e00640 1 -- 172.21.15.151:0/129803788 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fcc4015ec90 con 0x7fcc400cccd0 2024-09-16T15:52:17.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:17.059+0000 7fcc37400640 1 -- 172.21.15.151:0/129803788 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v116) ==== 170+0+0 (secure 0 0 0) 0x7fcc2c067270 con 0x7fcc400cccd0 2024-09-16T15:52:17.062 INFO:teuthology.orchestra.run.smithi151.stderr:pool 'cephfs_data' created 2024-09-16T15:52:17.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:17.062+0000 7fcc45e00640 1 -- 172.21.15.151:0/129803788 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcc2007a2c0 msgr2=0x7fcc2007c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:17.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:17.062+0000 7fcc45e00640 1 --2- 172.21.15.151:0/129803788 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcc2007a2c0 0x7fcc2007c780 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fcc3000ae50 tx=0x7fcc3000a540 comp rx=0 tx=0).stop 2024-09-16T15:52:17.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:17.062+0000 7fcc45e00640 1 -- 172.21.15.151:0/129803788 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcc400cccd0 msgr2=0x7fcc4015e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:17.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:17.062+0000 7fcc45e00640 1 --2- 172.21.15.151:0/129803788 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcc400cccd0 0x7fcc4015e750 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c00c410 tx=0x7fcc2c00c8d0 comp rx=0 tx=0).stop 2024-09-16T15:52:17.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:17.062+0000 7fcc45e00640 1 -- 172.21.15.151:0/129803788 shutdown_connections 2024-09-16T15:52:17.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:17.062+0000 7fcc45e00640 1 --2- 172.21.15.151:0/129803788 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcc2007a2c0 0x7fcc2007c780 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:17.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:17.062+0000 7fcc45e00640 1 --2- 172.21.15.151:0/129803788 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcc40145750 0x7fcc401433c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:17.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:17.062+0000 7fcc45e00640 1 --2- 172.21.15.151:0/129803788 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcc400cccd0 0x7fcc4015e750 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:17.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:17.062+0000 7fcc45e00640 1 --2- 172.21.15.151:0/129803788 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc4006efa0 0x7fcc400c61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:17.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:17.062+0000 7fcc45e00640 1 -- 172.21.15.151:0/129803788 >> 172.21.15.151:0/129803788 conn(0x7fcc400b1840 msgr2=0x7fcc400cbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:17.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:17.062+0000 7fcc45e00640 1 -- 172.21.15.151:0/129803788 shutdown_connections 2024-09-16T15:52:17.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:17.062+0000 7fcc45e00640 1 -- 172.21.15.151:0/129803788 wait complete. 2024-09-16T15:52:17.094 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-16T15:52:17.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:17.255+0000 7f4af9e00640 1 -- 172.21.15.151:0/1331675482 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4af406efa0 msgr2=0x7f4af40cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:17.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:17.255+0000 7f4af9e00640 1 --2- 172.21.15.151:0/1331675482 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4af406efa0 0x7f4af40cdbb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4aec005fd0 tx=0x7f4aec02aa10 comp rx=0 tx=0).stop 2024-09-16T15:52:17.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:17.255+0000 7f4af9e00640 1 -- 172.21.15.151:0/1331675482 shutdown_connections 2024-09-16T15:52:17.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:17.255+0000 7f4af9e00640 1 --2- 172.21.15.151:0/1331675482 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4af406efa0 0x7f4af40cdbb0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:17.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:17.255+0000 7f4af9e00640 1 --2- 172.21.15.151:0/1331675482 >> v2:172.21.15.167:3300/0 conn(0x7f4af40cccd0 0x7f4af4073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:17.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:17.255+0000 7f4af9e00640 1 -- 172.21.15.151:0/1331675482 >> 172.21.15.151:0/1331675482 conn(0x7f4af40b1840 msgr2=0x7f4af40b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:17.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:17.255+0000 7f4af9e00640 1 -- 172.21.15.151:0/1331675482 shutdown_connections 2024-09-16T15:52:17.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:17.255+0000 7f4af9e00640 1 -- 172.21.15.151:0/1331675482 wait complete. 2024-09-16T15:52:17.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:17.257+0000 7f4af9e00640 1 Processor -- start 2024-09-16T15:52:17.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:17.257+0000 7f4af9e00640 1 -- start start 2024-09-16T15:52:17.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:17.258+0000 7f4af9e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4af406efa0 0x7f4af4142aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:17.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:17.258+0000 7f4af9e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4af40cccd0 0x7f4af41587f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:17.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:17.258+0000 7f4af9e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4af4145750 0x7f4af41410b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:17.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:17.258+0000 7f4af9e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4af40ce480 con 0x7f4af40cccd0 2024-09-16T15:52:17.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:17.258+0000 7f4af9e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4af40ce300 con 0x7f4af406efa0 2024-09-16T15:52:17.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:17.258+0000 7f4af9e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f4af40ce600 con 0x7f4af4145750 2024-09-16T15:52:17.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:17.258+0000 7f4af2000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4af40cccd0 0x7f4af41587f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:17.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:17.258+0000 7f4af2000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4af40cccd0 0x7f4af41587f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49824/0 (socket says 172.21.15.151:49824) 2024-09-16T15:52:17.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:17.258+0000 7f4af2000640 1 -- 172.21.15.151:0/3455568349 learned_addr learned my addr 172.21.15.151:0/3455568349 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:52:17.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:17.258+0000 7f4af3400640 1 --2- 172.21.15.151:0/3455568349 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4af4145750 0x7f4af41410b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:17.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:17.258+0000 7f4af2a00640 1 --2- 172.21.15.151:0/3455568349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4af406efa0 0x7f4af4142aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:17.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:17.258+0000 7f4af2000640 1 -- 172.21.15.151:0/3455568349 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4af4145750 msgr2=0x7f4af41410b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:17.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:17.258+0000 7f4af2000640 1 --2- 172.21.15.151:0/3455568349 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4af4145750 0x7f4af41410b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:17.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:17.258+0000 7f4af2000640 1 -- 172.21.15.151:0/3455568349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4af406efa0 msgr2=0x7f4af4142aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:17.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:17.258+0000 7f4af2000640 1 --2- 172.21.15.151:0/3455568349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4af406efa0 0x7f4af4142aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:17.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:17.258+0000 7f4af2000640 1 -- 172.21.15.151:0/3455568349 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4af4141a00 con 0x7f4af40cccd0 2024-09-16T15:52:17.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:17.259+0000 7f4af2000640 1 --2- 172.21.15.151:0/3455568349 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4af40cccd0 0x7f4af41587f0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f4ae000e9f0 tx=0x7f4ae000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:17.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:17.259+0000 7f4aeb400640 1 -- 172.21.15.151:0/3455568349 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae000cd80 con 0x7f4af40cccd0 2024-09-16T15:52:17.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:17.259+0000 7f4aeb400640 1 -- 172.21.15.151:0/3455568349 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4ae0010430 con 0x7f4af40cccd0 2024-09-16T15:52:17.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:17.259+0000 7f4aeb400640 1 -- 172.21.15.151:0/3455568349 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae0010840 con 0x7f4af40cccd0 2024-09-16T15:52:17.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:17.259+0000 7f4af9e00640 1 -- 172.21.15.151:0/3455568349 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4af4141ce0 con 0x7f4af40cccd0 2024-09-16T15:52:17.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:17.259+0000 7f4af9e00640 1 -- 172.21.15.151:0/3455568349 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4af41422a0 con 0x7f4af40cccd0 2024-09-16T15:52:17.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:17.260+0000 7f4af9e00640 1 -- 172.21.15.151:0/3455568349 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4af40cdbb0 con 0x7f4af40cccd0 2024-09-16T15:52:17.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:17.262+0000 7f4aeb400640 1 -- 172.21.15.151:0/3455568349 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 41) ==== 103684+0+0 (secure 0 0 0) 0x7f4ae00109e0 con 0x7f4af40cccd0 2024-09-16T15:52:17.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:17.262+0000 7f4aeb400640 1 --2- 172.21.15.151:0/3455568349 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4ad407a2c0 0x7f4ad407c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:17.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:17.263+0000 7f4aeb400640 1 -- 172.21.15.151:0/3455568349 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5693+0+0 (secure 0 0 0) 0x7f4ae009a300 con 0x7f4af40cccd0 2024-09-16T15:52:17.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:17.263+0000 7f4af2a00640 1 --2- 172.21.15.151:0/3455568349 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4ad407a2c0 0x7f4ad407c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:17.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:17.263+0000 7f4af2a00640 1 --2- 172.21.15.151:0/3455568349 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4ad407a2c0 0x7f4ad407c780 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f4ae4001720 tx=0x7f4ae4009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:17.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:17.267+0000 7f4aeb400640 1 -- 172.21.15.151:0/3455568349 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4ae0063e40 con 0x7f4af40cccd0 2024-09-16T15:52:17.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:17.487+0000 7f4af9e00640 1 -- 172.21.15.151:0/3455568349 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f4af40a9460 con 0x7f4af40cccd0 2024-09-16T15:52:18.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.085+0000 7f4aeb400640 1 -- 172.21.15.151:0/3455568349 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v72) ==== 174+0+0 (secure 0 0 0) 0x7f4ae0068870 con 0x7f4af40cccd0 2024-09-16T15:52:18.094 INFO:teuthology.orchestra.run.smithi151.stderr:new fs with metadata pool 16 and data pool 17 2024-09-16T15:52:18.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.097+0000 7f4af9e00640 1 -- 172.21.15.151:0/3455568349 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4ad407a2c0 msgr2=0x7f4ad407c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:18.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.097+0000 7f4af9e00640 1 --2- 172.21.15.151:0/3455568349 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4ad407a2c0 0x7f4ad407c780 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f4ae4001720 tx=0x7f4ae4009040 comp rx=0 tx=0).stop 2024-09-16T15:52:18.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.097+0000 7f4af9e00640 1 -- 172.21.15.151:0/3455568349 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4af40cccd0 msgr2=0x7f4af41587f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:18.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.097+0000 7f4af9e00640 1 --2- 172.21.15.151:0/3455568349 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4af40cccd0 0x7f4af41587f0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f4ae000e9f0 tx=0x7f4ae000eeb0 comp rx=0 tx=0).stop 2024-09-16T15:52:18.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.097+0000 7f4af9e00640 1 -- 172.21.15.151:0/3455568349 shutdown_connections 2024-09-16T15:52:18.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.097+0000 7f4af9e00640 1 --2- 172.21.15.151:0/3455568349 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4ad407a2c0 0x7f4ad407c780 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:18.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.097+0000 7f4af9e00640 1 --2- 172.21.15.151:0/3455568349 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4af4145750 0x7f4af41410b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:18.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.097+0000 7f4af9e00640 1 --2- 172.21.15.151:0/3455568349 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4af40cccd0 0x7f4af41587f0 secure :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f4ae000e9f0 tx=0x7f4ae000eeb0 comp rx=0 tx=0).stop 2024-09-16T15:52:18.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.097+0000 7f4af9e00640 1 --2- 172.21.15.151:0/3455568349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4af406efa0 0x7f4af4142aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:18.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.097+0000 7f4af9e00640 1 -- 172.21.15.151:0/3455568349 >> 172.21.15.151:0/3455568349 conn(0x7f4af40b1840 msgr2=0x7f4af40cea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:18.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.099+0000 7f4af9e00640 1 -- 172.21.15.151:0/3455568349 shutdown_connections 2024-09-16T15:52:18.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.099+0000 7f4af9e00640 1 -- 172.21.15.151:0/3455568349 wait complete. 2024-09-16T15:52:18.125 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:52:18.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.282+0000 7fc026400640 1 -- 172.21.15.151:0/2859387324 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc0200ccf00 msgr2=0x7fc0200acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:18.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.282+0000 7fc026400640 1 --2- 172.21.15.151:0/2859387324 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc0200ccf00 0x7fc0200acd30 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fc020075130 tx=0x7fc00c02a4d0 comp rx=0 tx=0).stop 2024-09-16T15:52:18.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.282+0000 7fc026400640 1 -- 172.21.15.151:0/2859387324 shutdown_connections 2024-09-16T15:52:18.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.282+0000 7fc026400640 1 --2- 172.21.15.151:0/2859387324 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0200ad270 0x7fc020073fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:18.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.282+0000 7fc026400640 1 --2- 172.21.15.151:0/2859387324 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc0200ccf00 0x7fc0200acd30 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:18.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.282+0000 7fc026400640 1 --2- 172.21.15.151:0/2859387324 >> v2:172.21.15.167:3300/0 conn(0x7fc02015f5a0 0x7fc0200b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:18.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.282+0000 7fc026400640 1 -- 172.21.15.151:0/2859387324 >> 172.21.15.151:0/2859387324 conn(0x7fc0200b1950 msgr2=0x7fc0200b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:18.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.282+0000 7fc026400640 1 -- 172.21.15.151:0/2859387324 shutdown_connections 2024-09-16T15:52:18.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.282+0000 7fc026400640 1 -- 172.21.15.151:0/2859387324 wait complete. 2024-09-16T15:52:18.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.284+0000 7fc026400640 1 Processor -- start 2024-09-16T15:52:18.286 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.284+0000 7fc026400640 1 -- start start 2024-09-16T15:52:18.287 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.285+0000 7fc026400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc0200ad270 0x7fc02015b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:18.287 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.285+0000 7fc026400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0200ccf00 0x7fc020140d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:18.287 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.285+0000 7fc026400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc02015f5a0 0x7fc020159610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:18.287 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.285+0000 7fc026400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc020074880 con 0x7fc0200ad270 2024-09-16T15:52:18.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.285+0000 7fc026400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc020074700 con 0x7fc0200ccf00 2024-09-16T15:52:18.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.285+0000 7fc026400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fc02006f160 con 0x7fc02015f5a0 2024-09-16T15:52:18.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.285+0000 7fc01f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc0200ad270 0x7fc02015b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:18.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.285+0000 7fc01f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc0200ad270 0x7fc02015b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49836/0 (socket says 172.21.15.151:49836) 2024-09-16T15:52:18.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.285+0000 7fc01f400640 1 -- 172.21.15.151:0/1498475299 learned_addr learned my addr 172.21.15.151:0/1498475299 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:52:18.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.285+0000 7fc01fe00640 1 --2- 172.21.15.151:0/1498475299 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc02015f5a0 0x7fc020159610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:18.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.285+0000 7fc01ea00640 1 --2- 172.21.15.151:0/1498475299 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0200ccf00 0x7fc020140d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:18.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.285+0000 7fc01f400640 1 -- 172.21.15.151:0/1498475299 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc02015f5a0 msgr2=0x7fc020159610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:18.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.285+0000 7fc01f400640 1 --2- 172.21.15.151:0/1498475299 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc02015f5a0 0x7fc020159610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:18.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.285+0000 7fc01f400640 1 -- 172.21.15.151:0/1498475299 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0200ccf00 msgr2=0x7fc020140d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:18.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.285+0000 7fc01f400640 1 --2- 172.21.15.151:0/1498475299 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0200ccf00 0x7fc020140d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:18.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.285+0000 7fc01f400640 1 -- 172.21.15.151:0/1498475299 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc020141260 con 0x7fc0200ad270 2024-09-16T15:52:18.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.286+0000 7fc01f400640 1 --2- 172.21.15.151:0/1498475299 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc0200ad270 0x7fc02015b480 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fc01000d740 tx=0x7fc01000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:18.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.286+0000 7fc017400640 1 -- 172.21.15.151:0/1498475299 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc010017070 con 0x7fc0200ad270 2024-09-16T15:52:18.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.286+0000 7fc017400640 1 -- 172.21.15.151:0/1498475299 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc010009d20 con 0x7fc0200ad270 2024-09-16T15:52:18.290 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.286+0000 7fc017400640 1 -- 172.21.15.151:0/1498475299 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0100106c0 con 0x7fc0200ad270 2024-09-16T15:52:18.290 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.286+0000 7fc026400640 1 -- 172.21.15.151:0/1498475299 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc02015bb00 con 0x7fc0200ad270 2024-09-16T15:52:18.290 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.286+0000 7fc026400640 1 -- 172.21.15.151:0/1498475299 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc02015c0c0 con 0x7fc0200ad270 2024-09-16T15:52:18.290 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.287+0000 7fc026400640 1 -- 172.21.15.151:0/1498475299 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc020073fb0 con 0x7fc0200ad270 2024-09-16T15:52:18.290 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.288+0000 7fc017400640 1 -- 172.21.15.151:0/1498475299 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 41) ==== 103684+0+0 (secure 0 0 0) 0x7fc01001a070 con 0x7fc0200ad270 2024-09-16T15:52:18.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.289+0000 7fc017400640 1 --2- 172.21.15.151:0/1498475299 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc00007a2c0 0x7fc00007c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:18.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.289+0000 7fc017400640 1 -- 172.21.15.151:0/1498475299 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5793+0+0 (secure 0 0 0) 0x7fc010099940 con 0x7fc0200ad270 2024-09-16T15:52:18.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.290+0000 7fc01ea00640 1 --2- 172.21.15.151:0/1498475299 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc00007a2c0 0x7fc00007c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:18.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.290+0000 7fc01ea00640 1 --2- 172.21.15.151:0/1498475299 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc00007a2c0 0x7fc00007c780 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fc00c033000 tx=0x7fc00c004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:18.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.294+0000 7fc017400640 1 -- 172.21.15.151:0/1498475299 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc010063340 con 0x7fc0200ad270 2024-09-16T15:52:18.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.478+0000 7fc026400640 1 -- 172.21.15.151:0/1498475299 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc0200ae4e0 con 0x7fc0200ad270 2024-09-16T15:52:18.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.479+0000 7fc017400640 1 -- 172.21.15.151:0/1498475299 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) ==== 74+0+16948 (secure 0 0 0) 0x7fc010067d70 con 0x7fc0200ad270 2024-09-16T15:52:18.482 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:52:18.482 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":117,"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","created":"2024-09-16T15:38:56.348535+0000","modified":"2024-09-16T15:52:17.490049+0000","last_up_change":"2024-09-16T15:39:10.681384+0000","last_in_change":"2024-09-16T15:39:00.544477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:39:11.602252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:52:15.256011+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-16T15:52:16.456620+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5524076-a903-4d0a-af6a-caf474c4765d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6809","nonce":3111342741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6811","nonce":3111342741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6815","nonce":3111342741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6813","nonce":3111342741}]},"public_addr":"172.21.15.151:6809/3111342741","cluster_addr":"172.21.15.151:6811/3111342741","heartbeat_back_addr":"172.21.15.151:6815/3111342741","heartbeat_front_addr":"172.21.15.151:6813/3111342741","state":["exists","up"]},{"osd":1,"uuid":"dd25925f-0ae1-494e-8529-4dc205759db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6801","nonce":3368304087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6803","nonce":3368304087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6807","nonce":3368304087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6805","nonce":3368304087}]},"public_addr":"172.21.15.151:6801/3368304087","cluster_addr":"172.21.15.151:6803/3368304087","heartbeat_back_addr":"172.21.15.151:6807/3368304087","heartbeat_front_addr":"172.21.15.151:6805/3368304087","state":["exists","up"]},{"osd":2,"uuid":"c2f18115-013d-4da6-a0fe-cb978f4494b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6823","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6825","nonce":3079373271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6827","nonce":3079373271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6831","nonce":3079373271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6829","nonce":3079373271}]},"public_addr":"172.21.15.151:6825/3079373271","cluster_addr":"172.21.15.151:6827/3079373271","heartbeat_back_addr":"172.21.15.151:6831/3079373271","heartbeat_front_addr":"172.21.15.151:6829/3079373271","state":["exists","up"]},{"osd":3,"uuid":"36edb718-2fcb-4d88-8ddc-c321eed4279d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6817","nonce":1142509041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6819","nonce":1142509041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6824","nonce":1142509041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6821","nonce":1142509041}]},"public_addr":"172.21.15.151:6817/1142509041","cluster_addr":"172.21.15.151:6819/1142509041","heartbeat_back_addr":"172.21.15.151:6824/1142509041","heartbeat_front_addr":"172.21.15.151:6821/1142509041","state":["exists","up"]},{"osd":4,"uuid":"64d33708-cd0a-4236-abe3-1b646e82ab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6811","nonce":2735495271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6813","nonce":2735495271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6817","nonce":2735495271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6815","nonce":2735495271}]},"public_addr":"172.21.15.167:6811/2735495271","cluster_addr":"172.21.15.167:6813/2735495271","heartbeat_back_addr":"172.21.15.167:6817/2735495271","heartbeat_front_addr":"172.21.15.167:6815/2735495271","state":["exists","up"]},{"osd":5,"uuid":"f14b44a9-5910-458c-b518-09a8d0de94de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6827","nonce":1494916572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6829","nonce":1494916572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6833","nonce":1494916572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6831","nonce":1494916572}]},"public_addr":"172.21.15.167:6827/1494916572","cluster_addr":"172.21.15.167:6829/1494916572","heartbeat_back_addr":"172.21.15.167:6833/1494916572","heartbeat_front_addr":"172.21.15.167:6831/1494916572","state":["exists","up"]},{"osd":6,"uuid":"39335545-2654-4710-84b0-a9aa47cfcbba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6819","nonce":2538273273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6821","nonce":2538273273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6825","nonce":2538273273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6823","nonce":2538273273}]},"public_addr":"172.21.15.167:6819/2538273273","cluster_addr":"172.21.15.167:6821/2538273273","heartbeat_back_addr":"172.21.15.167:6825/2538273273","heartbeat_front_addr":"172.21.15.167:6823/2538273273","state":["exists","up"]},{"osd":7,"uuid":"4f8bfc81-53dd-4339-ac22-1a4007d9f963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6803","nonce":265072227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6805","nonce":265072227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6809","nonce":265072227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6807","nonce":265072227}]},"public_addr":"172.21.15.167:6803/265072227","cluster_addr":"172.21.15.167:6805/265072227","heartbeat_back_addr":"172.21.15.167:6809/265072227","heartbeat_front_addr":"172.21.15.167:6807/265072227","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.150359+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.084358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.722368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.332809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.320709+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.395153+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.935250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.801443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:52:18.484 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.482+0000 7fc026400640 1 -- 172.21.15.151:0/1498475299 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc00007a2c0 msgr2=0x7fc00007c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:18.484 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.482+0000 7fc026400640 1 --2- 172.21.15.151:0/1498475299 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc00007a2c0 0x7fc00007c780 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fc00c033000 tx=0x7fc00c004730 comp rx=0 tx=0).stop 2024-09-16T15:52:18.484 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.482+0000 7fc026400640 1 -- 172.21.15.151:0/1498475299 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc0200ad270 msgr2=0x7fc02015b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:18.484 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.482+0000 7fc026400640 1 --2- 172.21.15.151:0/1498475299 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc0200ad270 0x7fc02015b480 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fc01000d740 tx=0x7fc01000dc00 comp rx=0 tx=0).stop 2024-09-16T15:52:18.484 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.482+0000 7fc026400640 1 -- 172.21.15.151:0/1498475299 shutdown_connections 2024-09-16T15:52:18.484 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.482+0000 7fc026400640 1 --2- 172.21.15.151:0/1498475299 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc00007a2c0 0x7fc00007c780 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:18.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.482+0000 7fc026400640 1 --2- 172.21.15.151:0/1498475299 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0200ccf00 0x7fc020140d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:18.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.482+0000 7fc026400640 1 --2- 172.21.15.151:0/1498475299 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc02015f5a0 0x7fc020159610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:18.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.482+0000 7fc026400640 1 --2- 172.21.15.151:0/1498475299 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc0200ad270 0x7fc02015b480 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:18.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.482+0000 7fc026400640 1 -- 172.21.15.151:0/1498475299 >> 172.21.15.151:0/1498475299 conn(0x7fc0200b1950 msgr2=0x7fc0200af5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:18.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.482+0000 7fc026400640 1 -- 172.21.15.151:0/1498475299 shutdown_connections 2024-09-16T15:52:18.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.482+0000 7fc026400640 1 -- 172.21.15.151:0/1498475299 wait complete. 2024-09-16T15:52:18.501 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:52:18.660 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.658+0000 7fb39b400640 1 -- 172.21.15.151:0/3117828733 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb394073fb0 msgr2=0x7fb394074420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:18.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.658+0000 7fb39b400640 1 --2- 172.21.15.151:0/3117828733 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb394073fb0 0x7fb394074420 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fb394075130 tx=0x7fb38c02aa30 comp rx=0 tx=0).stop 2024-09-16T15:52:18.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.658+0000 7fb39b400640 1 -- 172.21.15.151:0/3117828733 shutdown_connections 2024-09-16T15:52:18.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.658+0000 7fb39b400640 1 --2- 172.21.15.151:0/3117828733 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb394073fb0 0x7fb394074420 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:18.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.658+0000 7fb39b400640 1 --2- 172.21.15.151:0/3117828733 >> v2:172.21.15.167:3300/0 conn(0x7fb39415f5a0 0x7fb3940b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:18.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.658+0000 7fb39b400640 1 -- 172.21.15.151:0/3117828733 >> 172.21.15.151:0/3117828733 conn(0x7fb3940b1950 msgr2=0x7fb3940b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:18.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.658+0000 7fb39b400640 1 -- 172.21.15.151:0/3117828733 shutdown_connections 2024-09-16T15:52:18.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.658+0000 7fb39b400640 1 -- 172.21.15.151:0/3117828733 wait complete. 2024-09-16T15:52:18.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.660+0000 7fb39b400640 1 Processor -- start 2024-09-16T15:52:18.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.660+0000 7fb39b400640 1 -- start start 2024-09-16T15:52:18.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.661+0000 7fb39b400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb394073fb0 0x7fb39409cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:18.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.661+0000 7fb39b400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3940ccf00 0x7fb3940b3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:18.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.661+0000 7fb39b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb39415f5a0 0x7fb3940ad680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:18.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.661+0000 7fb39b400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb39406f2e0 con 0x7fb39415f5a0 2024-09-16T15:52:18.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.661+0000 7fb39b400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb39406f160 con 0x7fb394073fb0 2024-09-16T15:52:18.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.661+0000 7fb39b400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fb39406f460 con 0x7fb3940ccf00 2024-09-16T15:52:18.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.661+0000 7fb399400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb39415f5a0 0x7fb3940ad680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:18.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.661+0000 7fb399400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb39415f5a0 0x7fb3940ad680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49866/0 (socket says 172.21.15.151:49866) 2024-09-16T15:52:18.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.661+0000 7fb399400640 1 -- 172.21.15.151:0/124897587 learned_addr learned my addr 172.21.15.151:0/124897587 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:52:18.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.661+0000 7fb398a00640 1 --2- 172.21.15.151:0/124897587 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb394073fb0 0x7fb39409cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:18.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.661+0000 7fb393e00640 1 --2- 172.21.15.151:0/124897587 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3940ccf00 0x7fb3940b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:18.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.661+0000 7fb399400640 1 -- 172.21.15.151:0/124897587 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3940ccf00 msgr2=0x7fb3940b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:18.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.661+0000 7fb399400640 1 --2- 172.21.15.151:0/124897587 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3940ccf00 0x7fb3940b3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:18.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.661+0000 7fb399400640 1 -- 172.21.15.151:0/124897587 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb394073fb0 msgr2=0x7fb39409cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:18.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.661+0000 7fb399400640 1 --2- 172.21.15.151:0/124897587 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb394073fb0 0x7fb39409cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:18.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.661+0000 7fb399400640 1 -- 172.21.15.151:0/124897587 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb39409d110 con 0x7fb39415f5a0 2024-09-16T15:52:18.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.661+0000 7fb398a00640 1 --2- 172.21.15.151:0/124897587 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb394073fb0 0x7fb39409cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:52:18.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.662+0000 7fb399400640 1 --2- 172.21.15.151:0/124897587 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb39415f5a0 0x7fb3940ad680 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fb38c005fd0 tx=0x7fb38c003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:18.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.662+0000 7fb390c00640 1 -- 172.21.15.151:0/124897587 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb38c046070 con 0x7fb39415f5a0 2024-09-16T15:52:18.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.662+0000 7fb390c00640 1 -- 172.21.15.151:0/124897587 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb38c02f690 con 0x7fb39415f5a0 2024-09-16T15:52:18.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.662+0000 7fb390c00640 1 -- 172.21.15.151:0/124897587 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb38c033050 con 0x7fb39415f5a0 2024-09-16T15:52:18.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.662+0000 7fb39b400640 1 -- 172.21.15.151:0/124897587 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3940b3e90 con 0x7fb39415f5a0 2024-09-16T15:52:18.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.662+0000 7fb39b400640 1 -- 172.21.15.151:0/124897587 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3941f5860 con 0x7fb39415f5a0 2024-09-16T15:52:18.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.663+0000 7fb39b400640 1 -- 172.21.15.151:0/124897587 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3940cece0 con 0x7fb39415f5a0 2024-09-16T15:52:18.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.664+0000 7fb390c00640 1 -- 172.21.15.151:0/124897587 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 41) ==== 103684+0+0 (secure 0 0 0) 0x7fb38c049070 con 0x7fb39415f5a0 2024-09-16T15:52:18.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.665+0000 7fb390c00640 1 --2- 172.21.15.151:0/124897587 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb37407a2f0 0x7fb37407c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:18.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.665+0000 7fb390c00640 1 -- 172.21.15.151:0/124897587 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5793+0+0 (secure 0 0 0) 0x7fb38c0ba640 con 0x7fb39415f5a0 2024-09-16T15:52:18.668 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.666+0000 7fb398a00640 1 --2- 172.21.15.151:0/124897587 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb37407a2f0 0x7fb37407c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:18.668 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.666+0000 7fb398a00640 1 --2- 172.21.15.151:0/124897587 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb37407a2f0 0x7fb37407c7b0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fb388009f20 tx=0x7fb388005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:18.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.669+0000 7fb390c00640 1 -- 172.21.15.151:0/124897587 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb3940cece0 con 0x7fb39415f5a0 2024-09-16T15:52:18.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.854+0000 7fb39b400640 1 -- 172.21.15.151:0/124897587 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb39406f9f0 con 0x7fb39415f5a0 2024-09-16T15:52:18.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.855+0000 7fb390c00640 1 -- 172.21.15.151:0/124897587 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) ==== 74+0+16948 (secure 0 0 0) 0x7fb38c0840c0 con 0x7fb39415f5a0 2024-09-16T15:52:18.858 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:52:18.858 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":117,"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","created":"2024-09-16T15:38:56.348535+0000","modified":"2024-09-16T15:52:17.490049+0000","last_up_change":"2024-09-16T15:39:10.681384+0000","last_in_change":"2024-09-16T15:39:00.544477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:39:11.602252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:52:15.256011+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-16T15:52:16.456620+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5524076-a903-4d0a-af6a-caf474c4765d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6809","nonce":3111342741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6811","nonce":3111342741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6815","nonce":3111342741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6813","nonce":3111342741}]},"public_addr":"172.21.15.151:6809/3111342741","cluster_addr":"172.21.15.151:6811/3111342741","heartbeat_back_addr":"172.21.15.151:6815/3111342741","heartbeat_front_addr":"172.21.15.151:6813/3111342741","state":["exists","up"]},{"osd":1,"uuid":"dd25925f-0ae1-494e-8529-4dc205759db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6801","nonce":3368304087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6803","nonce":3368304087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6807","nonce":3368304087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6805","nonce":3368304087}]},"public_addr":"172.21.15.151:6801/3368304087","cluster_addr":"172.21.15.151:6803/3368304087","heartbeat_back_addr":"172.21.15.151:6807/3368304087","heartbeat_front_addr":"172.21.15.151:6805/3368304087","state":["exists","up"]},{"osd":2,"uuid":"c2f18115-013d-4da6-a0fe-cb978f4494b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6823","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6825","nonce":3079373271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6827","nonce":3079373271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6831","nonce":3079373271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6829","nonce":3079373271}]},"public_addr":"172.21.15.151:6825/3079373271","cluster_addr":"172.21.15.151:6827/3079373271","heartbeat_back_addr":"172.21.15.151:6831/3079373271","heartbeat_front_addr":"172.21.15.151:6829/3079373271","state":["exists","up"]},{"osd":3,"uuid":"36edb718-2fcb-4d88-8ddc-c321eed4279d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6817","nonce":1142509041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6819","nonce":1142509041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6824","nonce":1142509041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6821","nonce":1142509041}]},"public_addr":"172.21.15.151:6817/1142509041","cluster_addr":"172.21.15.151:6819/1142509041","heartbeat_back_addr":"172.21.15.151:6824/1142509041","heartbeat_front_addr":"172.21.15.151:6821/1142509041","state":["exists","up"]},{"osd":4,"uuid":"64d33708-cd0a-4236-abe3-1b646e82ab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6811","nonce":2735495271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6813","nonce":2735495271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6817","nonce":2735495271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6815","nonce":2735495271}]},"public_addr":"172.21.15.167:6811/2735495271","cluster_addr":"172.21.15.167:6813/2735495271","heartbeat_back_addr":"172.21.15.167:6817/2735495271","heartbeat_front_addr":"172.21.15.167:6815/2735495271","state":["exists","up"]},{"osd":5,"uuid":"f14b44a9-5910-458c-b518-09a8d0de94de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6827","nonce":1494916572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6829","nonce":1494916572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6833","nonce":1494916572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6831","nonce":1494916572}]},"public_addr":"172.21.15.167:6827/1494916572","cluster_addr":"172.21.15.167:6829/1494916572","heartbeat_back_addr":"172.21.15.167:6833/1494916572","heartbeat_front_addr":"172.21.15.167:6831/1494916572","state":["exists","up"]},{"osd":6,"uuid":"39335545-2654-4710-84b0-a9aa47cfcbba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6819","nonce":2538273273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6821","nonce":2538273273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6825","nonce":2538273273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6823","nonce":2538273273}]},"public_addr":"172.21.15.167:6819/2538273273","cluster_addr":"172.21.15.167:6821/2538273273","heartbeat_back_addr":"172.21.15.167:6825/2538273273","heartbeat_front_addr":"172.21.15.167:6823/2538273273","state":["exists","up"]},{"osd":7,"uuid":"4f8bfc81-53dd-4339-ac22-1a4007d9f963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6803","nonce":265072227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6805","nonce":265072227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6809","nonce":265072227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6807","nonce":265072227}]},"public_addr":"172.21.15.167:6803/265072227","cluster_addr":"172.21.15.167:6805/265072227","heartbeat_back_addr":"172.21.15.167:6809/265072227","heartbeat_front_addr":"172.21.15.167:6807/265072227","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.150359+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.084358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.722368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.332809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.320709+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.395153+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.935250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.801443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:52:18.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.857+0000 7fb39b400640 1 -- 172.21.15.151:0/124897587 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb37407a2f0 msgr2=0x7fb37407c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:18.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.857+0000 7fb39b400640 1 --2- 172.21.15.151:0/124897587 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb37407a2f0 0x7fb37407c7b0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fb388009f20 tx=0x7fb388005f90 comp rx=0 tx=0).stop 2024-09-16T15:52:18.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.857+0000 7fb39b400640 1 -- 172.21.15.151:0/124897587 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb39415f5a0 msgr2=0x7fb3940ad680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:18.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.857+0000 7fb39b400640 1 --2- 172.21.15.151:0/124897587 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb39415f5a0 0x7fb3940ad680 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fb38c005fd0 tx=0x7fb38c003f10 comp rx=0 tx=0).stop 2024-09-16T15:52:18.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.858+0000 7fb39b400640 1 -- 172.21.15.151:0/124897587 shutdown_connections 2024-09-16T15:52:18.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.858+0000 7fb39b400640 1 --2- 172.21.15.151:0/124897587 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb37407a2f0 0x7fb37407c7b0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:18.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.858+0000 7fb39b400640 1 --2- 172.21.15.151:0/124897587 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb39415f5a0 0x7fb3940ad680 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:18.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.858+0000 7fb39b400640 1 --2- 172.21.15.151:0/124897587 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3940ccf00 0x7fb3940b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:18.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.858+0000 7fb39b400640 1 --2- 172.21.15.151:0/124897587 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb394073fb0 0x7fb39409cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:18.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.858+0000 7fb39b400640 1 -- 172.21.15.151:0/124897587 >> 172.21.15.151:0/124897587 conn(0x7fb3940b1950 msgr2=0x7fb3941549d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:18.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.858+0000 7fb39b400640 1 -- 172.21.15.151:0/124897587 shutdown_connections 2024-09-16T15:52:18.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:18.858+0000 7fb39b400640 1 -- 172.21.15.151:0/124897587 wait complete. 2024-09-16T15:52:18.877 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-16T15:52:19.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:19.035+0000 7f72b5000640 1 -- 172.21.15.151:0/2202305307 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72b015bec0 msgr2=0x7f72b006fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:19.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:19.035+0000 7f72b5000640 1 --2- 172.21.15.151:0/2202305307 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72b015bec0 0x7f72b006fce0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f72b00b5600 tx=0x7f72a002a5f0 comp rx=0 tx=0).stop 2024-09-16T15:52:19.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:19.035+0000 7f72b5000640 1 -- 172.21.15.151:0/2202305307 shutdown_connections 2024-09-16T15:52:19.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:19.035+0000 7f72b5000640 1 --2- 172.21.15.151:0/2202305307 >> v2:172.21.15.167:3300/0 conn(0x7f72b00cbc00 0x7f72b0073ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:19.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:19.035+0000 7f72b5000640 1 --2- 172.21.15.151:0/2202305307 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72b015bec0 0x7f72b006fce0 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:19.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:19.035+0000 7f72b5000640 1 -- 172.21.15.151:0/2202305307 >> 172.21.15.151:0/2202305307 conn(0x7f72b00cf9f0 msgr2=0x7f72b00cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:19.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:19.035+0000 7f72b5000640 1 -- 172.21.15.151:0/2202305307 shutdown_connections 2024-09-16T15:52:19.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:19.035+0000 7f72b5000640 1 -- 172.21.15.151:0/2202305307 wait complete. 2024-09-16T15:52:19.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:19.036+0000 7f72b5000640 1 Processor -- start 2024-09-16T15:52:19.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:19.037+0000 7f72b5000640 1 -- start start 2024-09-16T15:52:19.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:19.037+0000 7f72b5000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72b0072560 0x7f72b0154ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:19.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:19.037+0000 7f72b5000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72b00cbc00 0x7f72b014cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:19.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:19.037+0000 7f72b5000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f72b015bec0 0x7f72b01580a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:19.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:19.037+0000 7f72b5000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f72b00ae8d0 con 0x7f72b00cbc00 2024-09-16T15:52:19.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:19.037+0000 7f72b5000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f72b00ae750 con 0x7f72b0072560 2024-09-16T15:52:19.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:19.037+0000 7f72b5000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f72b00aea50 con 0x7f72b015bec0 2024-09-16T15:52:19.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:19.038+0000 7f72a7e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72b00cbc00 0x7f72b014cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:19.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:19.038+0000 7f72a7e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72b00cbc00 0x7f72b014cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49888/0 (socket says 172.21.15.151:49888) 2024-09-16T15:52:19.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:19.038+0000 7f72a7e00640 1 -- 172.21.15.151:0/3477159481 learned_addr learned my addr 172.21.15.151:0/3477159481 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:52:19.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:19.038+0000 7f72af400640 1 --2- 172.21.15.151:0/3477159481 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72b0072560 0x7f72b0154ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:19.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:19.038+0000 7f72afe00640 1 --2- 172.21.15.151:0/3477159481 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f72b015bec0 0x7f72b01580a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:19.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:19.038+0000 7f72a7e00640 1 -- 172.21.15.151:0/3477159481 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f72b015bec0 msgr2=0x7f72b01580a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:19.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:19.038+0000 7f72a7e00640 1 --2- 172.21.15.151:0/3477159481 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f72b015bec0 0x7f72b01580a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:19.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:19.038+0000 7f72a7e00640 1 -- 172.21.15.151:0/3477159481 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72b0072560 msgr2=0x7f72b0154ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:19.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:19.038+0000 7f72a7e00640 1 --2- 172.21.15.151:0/3477159481 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72b0072560 0x7f72b0154ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:19.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:19.038+0000 7f72a7e00640 1 -- 172.21.15.151:0/3477159481 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72b00ca8b0 con 0x7f72b00cbc00 2024-09-16T15:52:19.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:19.039+0000 7f72a7e00640 1 --2- 172.21.15.151:0/3477159481 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72b00cbc00 0x7f72b014cf40 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f729800c450 tx=0x7f729800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:19.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:19.039+0000 7f72a7400640 1 -- 172.21.15.151:0/3477159481 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72980052d0 con 0x7f72b00cbc00 2024-09-16T15:52:19.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:19.039+0000 7f72a7400640 1 -- 172.21.15.151:0/3477159481 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7298005470 con 0x7f72b00cbc00 2024-09-16T15:52:19.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:19.039+0000 7f72a7400640 1 -- 172.21.15.151:0/3477159481 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7298012870 con 0x7f72b00cbc00 2024-09-16T15:52:19.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:19.039+0000 7f72b5000640 1 -- 172.21.15.151:0/3477159481 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72b00c6d10 con 0x7f72b00cbc00 2024-09-16T15:52:19.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:19.039+0000 7f72b5000640 1 -- 172.21.15.151:0/3477159481 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72b00c72d0 con 0x7f72b00cbc00 2024-09-16T15:52:19.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:19.040+0000 7f72b5000640 1 -- 172.21.15.151:0/3477159481 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72b00aef40 con 0x7f72b00cbc00 2024-09-16T15:52:19.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:19.041+0000 7f72a7400640 1 -- 172.21.15.151:0/3477159481 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 41) ==== 103684+0+0 (secure 0 0 0) 0x7f72980035f0 con 0x7f72b00cbc00 2024-09-16T15:52:19.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:19.042+0000 7f72a7400640 1 --2- 172.21.15.151:0/3477159481 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f728c07a2c0 0x7f728c07c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:19.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:19.042+0000 7f72a7400640 1 -- 172.21.15.151:0/3477159481 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5793+0+0 (secure 0 0 0) 0x7f7298099230 con 0x7f72b00cbc00 2024-09-16T15:52:19.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:19.042+0000 7f72af400640 1 --2- 172.21.15.151:0/3477159481 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f728c07a2c0 0x7f728c07c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:19.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:19.043+0000 7f72af400640 1 --2- 172.21.15.151:0/3477159481 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f728c07a2c0 0x7f728c07c780 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f72a0033000 tx=0x7f72a0004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:19.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:19.046+0000 7f72a7400640 1 -- 172.21.15.151:0/3477159481 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7298062cb0 con 0x7f72b00cbc00 2024-09-16T15:52:19.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:19.273+0000 7f72b5000640 1 -- 172.21.15.151:0/3477159481 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f72b015f410 con 0x7f72b00cbc00 2024-09-16T15:52:20.050 INFO:tasks.ceph.mds.a.smithi151.stderr:2024-09-16T15:52:20.049+0000 7f1f6ee00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-16T15:52:20.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:20.109+0000 7f72a7400640 1 -- 172.21.15.151:0/3477159481 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v75) ==== 118+0+0 (secure 0 0 0) 0x7f72980676e0 con 0x7f72b00cbc00 2024-09-16T15:52:20.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:20.112+0000 7f72b5000640 1 -- 172.21.15.151:0/3477159481 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f728c07a2c0 msgr2=0x7f728c07c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:20.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:20.112+0000 7f72b5000640 1 --2- 172.21.15.151:0/3477159481 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f728c07a2c0 0x7f728c07c780 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f72a0033000 tx=0x7f72a0004730 comp rx=0 tx=0).stop 2024-09-16T15:52:20.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:20.112+0000 7f72b5000640 1 -- 172.21.15.151:0/3477159481 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72b00cbc00 msgr2=0x7f72b014cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:20.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:20.112+0000 7f72b5000640 1 --2- 172.21.15.151:0/3477159481 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72b00cbc00 0x7f72b014cf40 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f729800c450 tx=0x7f729800c910 comp rx=0 tx=0).stop 2024-09-16T15:52:20.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:20.112+0000 7f72b5000640 1 -- 172.21.15.151:0/3477159481 shutdown_connections 2024-09-16T15:52:20.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:20.112+0000 7f72b5000640 1 --2- 172.21.15.151:0/3477159481 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f728c07a2c0 0x7f728c07c780 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:20.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:20.112+0000 7f72b5000640 1 --2- 172.21.15.151:0/3477159481 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f72b015bec0 0x7f72b01580a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:20.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:20.112+0000 7f72b5000640 1 --2- 172.21.15.151:0/3477159481 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f72b00cbc00 0x7f72b014cf40 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:20.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:20.112+0000 7f72b5000640 1 --2- 172.21.15.151:0/3477159481 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72b0072560 0x7f72b0154ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:20.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:20.112+0000 7f72b5000640 1 -- 172.21.15.151:0/3477159481 >> 172.21.15.151:0/3477159481 conn(0x7f72b00cf9f0 msgr2=0x7f72b006e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:20.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:20.112+0000 7f72b5000640 1 -- 172.21.15.151:0/3477159481 shutdown_connections 2024-09-16T15:52:20.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:20.112+0000 7f72b5000640 1 -- 172.21.15.151:0/3477159481 wait complete. 2024-09-16T15:52:20.131 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-16T15:52:20.131 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-16T15:52:20.294 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:20.292+0000 7fdea1a00640 1 -- 172.21.15.151:0/3595040216 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde9c145750 msgr2=0x7fde9c073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:20.294 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:20.292+0000 7fdea1a00640 1 --2- 172.21.15.151:0/3595040216 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde9c145750 0x7fde9c073440 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fde8c009830 tx=0x7fde8c02a480 comp rx=0 tx=0).stop 2024-09-16T15:52:20.294 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:20.292+0000 7fdea1a00640 1 -- 172.21.15.151:0/3595040216 shutdown_connections 2024-09-16T15:52:20.294 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:20.292+0000 7fdea1a00640 1 --2- 172.21.15.151:0/3595040216 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde9c06efa0 0x7fde9c0cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:20.294 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:20.292+0000 7fdea1a00640 1 --2- 172.21.15.151:0/3595040216 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde9c145750 0x7fde9c073440 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:20.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:20.292+0000 7fdea1a00640 1 -- 172.21.15.151:0/3595040216 >> 172.21.15.151:0/3595040216 conn(0x7fde9c0b1840 msgr2=0x7fde9c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:20.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:20.292+0000 7fdea1a00640 1 -- 172.21.15.151:0/3595040216 shutdown_connections 2024-09-16T15:52:20.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:20.292+0000 7fdea1a00640 1 -- 172.21.15.151:0/3595040216 wait complete. 2024-09-16T15:52:20.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:20.293+0000 7fdea1a00640 1 Processor -- start 2024-09-16T15:52:20.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:20.293+0000 7fdea1a00640 1 -- start start 2024-09-16T15:52:20.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:20.294+0000 7fdea1a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde9c06efa0 0x7fde9c0b0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:20.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:20.294+0000 7fdea1a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde9c0ccb00 0x7fde9c0cb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:20.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:20.294+0000 7fdea1a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde9c145750 0x7fde9c0cb910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:20.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:20.294+0000 7fdea1a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c0b2650 con 0x7fde9c06efa0 2024-09-16T15:52:20.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:20.294+0000 7fdea1a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c0cedd0 con 0x7fde9c0ccb00 2024-09-16T15:52:20.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:20.294+0000 7fdea1a00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fde9c160100 con 0x7fde9c145750 2024-09-16T15:52:20.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:20.295+0000 7fde9aa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde9c06efa0 0x7fde9c0b0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:20.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:20.295+0000 7fde9aa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde9c06efa0 0x7fde9c0b0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49898/0 (socket says 172.21.15.151:49898) 2024-09-16T15:52:20.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:20.295+0000 7fde9aa00640 1 -- 172.21.15.151:0/3061333710 learned_addr learned my addr 172.21.15.151:0/3061333710 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:52:20.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:20.295+0000 7fde93e00640 1 --2- 172.21.15.151:0/3061333710 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde9c0ccb00 0x7fde9c0cb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:20.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:20.295+0000 7fde9b400640 1 --2- 172.21.15.151:0/3061333710 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde9c145750 0x7fde9c0cb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:20.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:20.295+0000 7fde9aa00640 1 -- 172.21.15.151:0/3061333710 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde9c145750 msgr2=0x7fde9c0cb910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:20.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:20.295+0000 7fde9aa00640 1 --2- 172.21.15.151:0/3061333710 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde9c145750 0x7fde9c0cb910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:20.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:20.295+0000 7fde9aa00640 1 -- 172.21.15.151:0/3061333710 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde9c0ccb00 msgr2=0x7fde9c0cb3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:20.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:20.295+0000 7fde9aa00640 1 --2- 172.21.15.151:0/3061333710 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde9c0ccb00 0x7fde9c0cb3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:20.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:20.295+0000 7fde9aa00640 1 -- 172.21.15.151:0/3061333710 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde9c1533c0 con 0x7fde9c06efa0 2024-09-16T15:52:20.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:20.295+0000 7fde93e00640 1 --2- 172.21.15.151:0/3061333710 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde9c0ccb00 0x7fde9c0cb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:52:20.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:20.295+0000 7fde9b400640 1 --2- 172.21.15.151:0/3061333710 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde9c145750 0x7fde9c0cb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:52:20.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:20.295+0000 7fde9aa00640 1 --2- 172.21.15.151:0/3061333710 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde9c06efa0 0x7fde9c0b0bd0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fde8c000c00 tx=0x7fde8c033c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:20.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:20.296+0000 7fde92a00640 1 -- 172.21.15.151:0/3061333710 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8c005c90 con 0x7fde9c06efa0 2024-09-16T15:52:20.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:20.296+0000 7fde92a00640 1 -- 172.21.15.151:0/3061333710 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fde8c031e50 con 0x7fde9c06efa0 2024-09-16T15:52:20.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:20.296+0000 7fde92a00640 1 -- 172.21.15.151:0/3061333710 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8c0332f0 con 0x7fde9c06efa0 2024-09-16T15:52:20.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:20.296+0000 7fdea1a00640 1 -- 172.21.15.151:0/3061333710 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde9c1536a0 con 0x7fde9c06efa0 2024-09-16T15:52:20.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:20.296+0000 7fdea1a00640 1 -- 172.21.15.151:0/3061333710 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde9c153c60 con 0x7fde9c06efa0 2024-09-16T15:52:20.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:20.297+0000 7fdea1a00640 1 -- 172.21.15.151:0/3061333710 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde9c1605b0 con 0x7fde9c06efa0 2024-09-16T15:52:20.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:20.298+0000 7fde92a00640 1 -- 172.21.15.151:0/3061333710 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 41) ==== 103684+0+0 (secure 0 0 0) 0x7fde8c033490 con 0x7fde9c06efa0 2024-09-16T15:52:20.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:20.299+0000 7fde92a00640 1 --2- 172.21.15.151:0/3061333710 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fde7807a2c0 0x7fde7807c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:20.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:20.299+0000 7fde92a00640 1 -- 172.21.15.151:0/3061333710 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5793+0+0 (secure 0 0 0) 0x7fde8c0bac00 con 0x7fde9c06efa0 2024-09-16T15:52:20.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:20.300+0000 7fde93e00640 1 --2- 172.21.15.151:0/3061333710 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fde7807a2c0 0x7fde7807c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:20.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:20.300+0000 7fde93e00640 1 --2- 172.21.15.151:0/3061333710 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fde7807a2c0 0x7fde7807c780 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fde9c0af1f0 tx=0x7fde84005e70 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:20.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:20.303+0000 7fde92a00640 1 -- 172.21.15.151:0/3061333710 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fde8c03f030 con 0x7fde9c06efa0 2024-09-16T15:52:20.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:20.523+0000 7fdea1a00640 1 -- 172.21.15.151:0/3061333710 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fde9c142490 con 0x7fde9c06efa0 2024-09-16T15:52:21.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.110+0000 7fde92a00640 1 -- 172.21.15.151:0/3061333710 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v76) ==== 122+0+0 (secure 0 0 0) 0x7fde8c084cd0 con 0x7fde9c06efa0 2024-09-16T15:52:21.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.113+0000 7fdea1a00640 1 -- 172.21.15.151:0/3061333710 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fde7807a2c0 msgr2=0x7fde7807c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:21.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.113+0000 7fdea1a00640 1 --2- 172.21.15.151:0/3061333710 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fde7807a2c0 0x7fde7807c780 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fde9c0af1f0 tx=0x7fde84005e70 comp rx=0 tx=0).stop 2024-09-16T15:52:21.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.113+0000 7fdea1a00640 1 -- 172.21.15.151:0/3061333710 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde9c06efa0 msgr2=0x7fde9c0b0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:21.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.113+0000 7fdea1a00640 1 --2- 172.21.15.151:0/3061333710 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde9c06efa0 0x7fde9c0b0bd0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fde8c000c00 tx=0x7fde8c033c90 comp rx=0 tx=0).stop 2024-09-16T15:52:21.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.113+0000 7fdea1a00640 1 -- 172.21.15.151:0/3061333710 shutdown_connections 2024-09-16T15:52:21.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.113+0000 7fdea1a00640 1 --2- 172.21.15.151:0/3061333710 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fde7807a2c0 0x7fde7807c780 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:21.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.113+0000 7fdea1a00640 1 --2- 172.21.15.151:0/3061333710 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde9c0ccb00 0x7fde9c0cb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:21.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.113+0000 7fdea1a00640 1 --2- 172.21.15.151:0/3061333710 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde9c145750 0x7fde9c0cb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:21.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.113+0000 7fdea1a00640 1 --2- 172.21.15.151:0/3061333710 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde9c06efa0 0x7fde9c0b0bd0 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:21.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.113+0000 7fdea1a00640 1 -- 172.21.15.151:0/3061333710 >> 172.21.15.151:0/3061333710 conn(0x7fde9c0b1840 msgr2=0x7fde9c0cef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:21.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.113+0000 7fdea1a00640 1 -- 172.21.15.151:0/3061333710 shutdown_connections 2024-09-16T15:52:21.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.113+0000 7fdea1a00640 1 -- 172.21.15.151:0/3061333710 wait complete. 2024-09-16T15:52:21.132 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-16T15:52:21.132 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:52:21.151 DEBUG:teuthology.orchestra.run.smithi151:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:52:21.182 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:52:21.293 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.291+0000 7f1a17600640 1 -- 172.21.15.151:0/3040102590 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a100ccf00 msgr2=0x7f1a100b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:21.294 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.291+0000 7f1a17600640 1 --2- 172.21.15.151:0/3040102590 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a100ccf00 0x7f1a100b2440 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f19f8005fd0 tx=0x7f19f802a540 comp rx=0 tx=0).stop 2024-09-16T15:52:21.294 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.291+0000 7f1a17600640 1 -- 172.21.15.151:0/3040102590 shutdown_connections 2024-09-16T15:52:21.294 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.291+0000 7f1a17600640 1 --2- 172.21.15.151:0/3040102590 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1a10073fb0 0x7f1a10074440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:21.294 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.291+0000 7f1a17600640 1 --2- 172.21.15.151:0/3040102590 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a100ccf00 0x7f1a100b2440 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:21.294 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.291+0000 7f1a17600640 1 -- 172.21.15.151:0/3040102590 >> 172.21.15.151:0/3040102590 conn(0x7f1a100b1950 msgr2=0x7f1a100b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:21.294 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.291+0000 7f1a17600640 1 -- 172.21.15.151:0/3040102590 shutdown_connections 2024-09-16T15:52:21.294 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.291+0000 7f1a17600640 1 -- 172.21.15.151:0/3040102590 wait complete. 2024-09-16T15:52:21.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.293+0000 7f1a17600640 1 Processor -- start 2024-09-16T15:52:21.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.293+0000 7f1a17600640 1 -- start start 2024-09-16T15:52:21.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.294+0000 7f1a17600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1a10073fb0 0x7f1a10150c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:21.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.294+0000 7f1a17600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a100ccf00 0x7f1a101f6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:21.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.294+0000 7f1a17600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a1015f5a0 0x7f1a101484c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:21.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.294+0000 7f1a17600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1a1006f2e0 con 0x7f1a1015f5a0 2024-09-16T15:52:21.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.294+0000 7f1a17600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1a1006f160 con 0x7f1a100ccf00 2024-09-16T15:52:21.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.294+0000 7f1a17600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f1a1006f460 con 0x7f1a10073fb0 2024-09-16T15:52:21.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.294+0000 7f1a15600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a1015f5a0 0x7f1a101484c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:21.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.294+0000 7f1a15600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a1015f5a0 0x7f1a101484c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49926/0 (socket says 172.21.15.151:49926) 2024-09-16T15:52:21.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.294+0000 7f1a15600640 1 -- 172.21.15.151:0/175502004 learned_addr learned my addr 172.21.15.151:0/175502004 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:52:21.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.294+0000 7f1a14c00640 1 --2- 172.21.15.151:0/175502004 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1a10073fb0 0x7f1a10150c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:21.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.294+0000 7f1a07e00640 1 --2- 172.21.15.151:0/175502004 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a100ccf00 0x7f1a101f6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:21.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.295+0000 7f1a15600640 1 -- 172.21.15.151:0/175502004 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1a10073fb0 msgr2=0x7f1a10150c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:21.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.295+0000 7f1a15600640 1 --2- 172.21.15.151:0/175502004 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1a10073fb0 0x7f1a10150c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:21.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.295+0000 7f1a15600640 1 -- 172.21.15.151:0/175502004 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a100ccf00 msgr2=0x7f1a101f6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:21.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.295+0000 7f1a15600640 1 --2- 172.21.15.151:0/175502004 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a100ccf00 0x7f1a101f6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:21.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.295+0000 7f1a15600640 1 -- 172.21.15.151:0/175502004 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a101563a0 con 0x7f1a1015f5a0 2024-09-16T15:52:21.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.295+0000 7f1a07e00640 1 --2- 172.21.15.151:0/175502004 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a100ccf00 0x7f1a101f6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:52:21.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.295+0000 7f1a15600640 1 --2- 172.21.15.151:0/175502004 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a1015f5a0 0x7f1a101484c0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f1a0800da70 tx=0x7f1a0800df30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:21.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.295+0000 7f1a0d600640 1 -- 172.21.15.151:0/175502004 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a08017070 con 0x7f1a1015f5a0 2024-09-16T15:52:21.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.295+0000 7f1a0d600640 1 -- 172.21.15.151:0/175502004 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1a08003970 con 0x7f1a1015f5a0 2024-09-16T15:52:21.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.295+0000 7f1a17600640 1 -- 172.21.15.151:0/175502004 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a10156680 con 0x7f1a1015f5a0 2024-09-16T15:52:21.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.295+0000 7f1a0d600640 1 -- 172.21.15.151:0/175502004 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a08003dc0 con 0x7f1a1015f5a0 2024-09-16T15:52:21.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.295+0000 7f1a17600640 1 -- 172.21.15.151:0/175502004 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a101f6520 con 0x7f1a1015f5a0 2024-09-16T15:52:21.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.296+0000 7f1a17600640 1 -- 172.21.15.151:0/175502004 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a1006f980 con 0x7f1a1015f5a0 2024-09-16T15:52:21.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.297+0000 7f1a0d600640 1 -- 172.21.15.151:0/175502004 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 41) ==== 103684+0+0 (secure 0 0 0) 0x7f1a08003f60 con 0x7f1a1015f5a0 2024-09-16T15:52:21.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.298+0000 7f1a0d600640 1 --2- 172.21.15.151:0/175502004 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f19ec07a2c0 0x7f19ec07c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:21.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.298+0000 7f1a0d600640 1 -- 172.21.15.151:0/175502004 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5793+0+0 (secure 0 0 0) 0x7f1a08099c50 con 0x7f1a1015f5a0 2024-09-16T15:52:21.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.299+0000 7f1a14c00640 1 --2- 172.21.15.151:0/175502004 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f19ec07a2c0 0x7f19ec07c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:21.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.300+0000 7f1a14c00640 1 --2- 172.21.15.151:0/175502004 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f19ec07a2c0 0x7f19ec07c780 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f1a00000f60 tx=0x7f1a00009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:21.304 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.302+0000 7f1a0d600640 1 -- 172.21.15.151:0/175502004 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1a080636d0 con 0x7f1a1015f5a0 2024-09-16T15:52:21.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.525+0000 7f1a17600640 1 -- 172.21.15.151:0/175502004 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1a10074440 con 0x7f1a1015f5a0 2024-09-16T15:52:21.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.525+0000 7f1a0d600640 1 -- 172.21.15.151:0/175502004 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+5237 (secure 0 0 0) 0x7f1a08068100 con 0x7f1a1015f5a0 2024-09-16T15:52:21.528 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:52:21.528 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":76,"btime":"2024-09-16T15:52:21:110724+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7034,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1802374613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1802374613},{"type":"v1","addr":"172.21.15.167:6837","nonce":1802374613}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":7753,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3585191088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3585191088},{"type":"v1","addr":"172.21.15.167:6836","nonce":3585191088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":8457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/1321222227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1321222227},{"type":"v1","addr":"172.21.15.151:6834","nonce":1321222227}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:52:17.489987+0000","modified":"2024-09-16T15:52:20.525094+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8460":{"gid":8460,"name":"a","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.151:6835/2504927912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":2504927912},{"type":"v1","addr":"172.21.15.151:6835","nonce":2504927912}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8460,"qdb_cluster":[8460]},"id":8}]} 2024-09-16T15:52:21.528 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 76 2024-09-16T15:52:21.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.528+0000 7f1a17600640 1 -- 172.21.15.151:0/175502004 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f19ec07a2c0 msgr2=0x7f19ec07c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:21.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.528+0000 7f1a17600640 1 --2- 172.21.15.151:0/175502004 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f19ec07a2c0 0x7f19ec07c780 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f1a00000f60 tx=0x7f1a00009040 comp rx=0 tx=0).stop 2024-09-16T15:52:21.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.528+0000 7f1a17600640 1 -- 172.21.15.151:0/175502004 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a1015f5a0 msgr2=0x7f1a101484c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:21.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.528+0000 7f1a17600640 1 --2- 172.21.15.151:0/175502004 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a1015f5a0 0x7f1a101484c0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f1a0800da70 tx=0x7f1a0800df30 comp rx=0 tx=0).stop 2024-09-16T15:52:21.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.528+0000 7f1a17600640 1 -- 172.21.15.151:0/175502004 shutdown_connections 2024-09-16T15:52:21.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.528+0000 7f1a17600640 1 --2- 172.21.15.151:0/175502004 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f19ec07a2c0 0x7f19ec07c780 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:21.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.528+0000 7f1a17600640 1 --2- 172.21.15.151:0/175502004 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a100ccf00 0x7f1a101f6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:21.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.528+0000 7f1a17600640 1 --2- 172.21.15.151:0/175502004 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a1015f5a0 0x7f1a101484c0 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:21.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.528+0000 7f1a17600640 1 --2- 172.21.15.151:0/175502004 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1a10073fb0 0x7f1a10150c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:21.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.528+0000 7f1a17600640 1 -- 172.21.15.151:0/175502004 >> 172.21.15.151:0/175502004 conn(0x7f1a100b1950 msgr2=0x7f1a100b1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:21.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.528+0000 7f1a17600640 1 -- 172.21.15.151:0/175502004 shutdown_connections 2024-09-16T15:52:21.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.528+0000 7f1a17600640 1 -- 172.21.15.151:0/175502004 wait complete. 2024-09-16T15:52:21.546 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:52:21.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.742+0000 7fa051e00640 1 -- 172.21.15.151:0/1052449339 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa04c0af3d0 msgr2=0x7fa04c06f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:21.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.742+0000 7fa051e00640 1 --2- 172.21.15.151:0/1052449339 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa04c0af3d0 0x7fa04c06f490 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fa04c0cd5d0 tx=0x7fa03802a4d0 comp rx=0 tx=0).stop 2024-09-16T15:52:21.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.743+0000 7fa051e00640 1 -- 172.21.15.151:0/1052449339 shutdown_connections 2024-09-16T15:52:21.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.743+0000 7fa051e00640 1 --2- 172.21.15.151:0/1052449339 >> v2:172.21.15.167:3300/0 conn(0x7fa04c06f9d0 0x7fa04c161c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:21.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.743+0000 7fa051e00640 1 --2- 172.21.15.151:0/1052449339 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa04c0af3d0 0x7fa04c06f490 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:21.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.743+0000 7fa051e00640 1 --2- 172.21.15.151:0/1052449339 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa04c161550 0x7fa04c06ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:21.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.743+0000 7fa051e00640 1 -- 172.21.15.151:0/1052449339 >> 172.21.15.151:0/1052449339 conn(0x7fa04c0b1950 msgr2=0x7fa04c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:21.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.743+0000 7fa051e00640 1 -- 172.21.15.151:0/1052449339 shutdown_connections 2024-09-16T15:52:21.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.743+0000 7fa051e00640 1 -- 172.21.15.151:0/1052449339 wait complete. 2024-09-16T15:52:21.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.744+0000 7fa051e00640 1 Processor -- start 2024-09-16T15:52:21.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.744+0000 7fa051e00640 1 -- start start 2024-09-16T15:52:21.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.745+0000 7fa051e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa04c06f9d0 0x7fa04c145d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:21.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.745+0000 7fa051e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa04c0af3d0 0x7fa04c1f8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:21.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.745+0000 7fa051e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa04c161550 0x7fa04c09ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:21.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.745+0000 7fa051e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa04c162560 con 0x7fa04c06f9d0 2024-09-16T15:52:21.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.745+0000 7fa051e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa04c1623e0 con 0x7fa04c161550 2024-09-16T15:52:21.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.745+0000 7fa051e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fa04c1626e0 con 0x7fa04c0af3d0 2024-09-16T15:52:21.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.746+0000 7fa04aa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa04c06f9d0 0x7fa04c145d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:21.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.746+0000 7fa04aa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa04c06f9d0 0x7fa04c145d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49940/0 (socket says 172.21.15.151:49940) 2024-09-16T15:52:21.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.746+0000 7fa04aa00640 1 -- 172.21.15.151:0/3435560434 learned_addr learned my addr 172.21.15.151:0/3435560434 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:52:21.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.746+0000 7fa04a000640 1 --2- 172.21.15.151:0/3435560434 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa04c0af3d0 0x7fa04c1f8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:21.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.746+0000 7fa04b400640 1 --2- 172.21.15.151:0/3435560434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa04c161550 0x7fa04c09ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:21.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.746+0000 7fa04aa00640 1 -- 172.21.15.151:0/3435560434 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa04c0af3d0 msgr2=0x7fa04c1f8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:21.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.746+0000 7fa04aa00640 1 --2- 172.21.15.151:0/3435560434 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa04c0af3d0 0x7fa04c1f8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:21.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.746+0000 7fa04aa00640 1 -- 172.21.15.151:0/3435560434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa04c161550 msgr2=0x7fa04c09ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:21.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.746+0000 7fa04aa00640 1 --2- 172.21.15.151:0/3435560434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa04c161550 0x7fa04c09ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:21.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.746+0000 7fa04aa00640 1 -- 172.21.15.151:0/3435560434 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa04c146290 con 0x7fa04c06f9d0 2024-09-16T15:52:21.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.746+0000 7fa04b400640 1 --2- 172.21.15.151:0/3435560434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa04c161550 0x7fa04c09ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:52:21.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.746+0000 7fa04aa00640 1 --2- 172.21.15.151:0/3435560434 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa04c06f9d0 0x7fa04c145d50 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fa03c00d6d0 tx=0x7fa03c00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:21.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.747+0000 7fa043400640 1 -- 172.21.15.151:0/3435560434 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03c017070 con 0x7fa04c06f9d0 2024-09-16T15:52:21.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.747+0000 7fa043400640 1 -- 172.21.15.151:0/3435560434 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa03c005330 con 0x7fa04c06f9d0 2024-09-16T15:52:21.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.747+0000 7fa051e00640 1 -- 172.21.15.151:0/3435560434 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa04c1f8e70 con 0x7fa04c06f9d0 2024-09-16T15:52:21.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.747+0000 7fa043400640 1 -- 172.21.15.151:0/3435560434 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03c009820 con 0x7fa04c06f9d0 2024-09-16T15:52:21.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.747+0000 7fa051e00640 1 -- 172.21.15.151:0/3435560434 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa04c0cfe50 con 0x7fa04c06f9d0 2024-09-16T15:52:21.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.748+0000 7fa051e00640 1 -- 172.21.15.151:0/3435560434 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa04c161c90 con 0x7fa04c06f9d0 2024-09-16T15:52:21.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.749+0000 7fa043400640 1 -- 172.21.15.151:0/3435560434 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 41) ==== 103684+0+0 (secure 0 0 0) 0x7fa03c009bd0 con 0x7fa04c06f9d0 2024-09-16T15:52:21.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.750+0000 7fa043400640 1 --2- 172.21.15.151:0/3435560434 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa02c07a2f0 0x7fa02c07c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:21.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.750+0000 7fa043400640 1 -- 172.21.15.151:0/3435560434 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5793+0+0 (secure 0 0 0) 0x7fa03c099b40 con 0x7fa04c06f9d0 2024-09-16T15:52:21.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.751+0000 7fa04a000640 1 --2- 172.21.15.151:0/3435560434 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa02c07a2f0 0x7fa02c07c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:21.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.751+0000 7fa04a000640 1 --2- 172.21.15.151:0/3435560434 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa02c07a2f0 0x7fa02c07c7b0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fa038033000 tx=0x7fa038004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:21.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.754+0000 7fa043400640 1 -- 172.21.15.151:0/3435560434 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa04c146290 con 0x7fa04c06f9d0 2024-09-16T15:52:21.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.940+0000 7fa051e00640 1 -- 172.21.15.151:0/3435560434 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa04c073aa0 con 0x7fa04c06f9d0 2024-09-16T15:52:21.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.941+0000 7fa043400640 1 -- 172.21.15.151:0/3435560434 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) ==== 74+0+16935 (secure 0 0 0) 0x7fa03c063530 con 0x7fa04c06f9d0 2024-09-16T15:52:21.944 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:52:21.944 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":118,"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","created":"2024-09-16T15:38:56.348535+0000","modified":"2024-09-16T15:52:19.086144+0000","last_up_change":"2024-09-16T15:39:10.681384+0000","last_in_change":"2024-09-16T15:39:00.544477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:39:11.602252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:52:15.256011+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-16T15:52:16.456620+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5524076-a903-4d0a-af6a-caf474c4765d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6809","nonce":3111342741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6811","nonce":3111342741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6815","nonce":3111342741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6813","nonce":3111342741}]},"public_addr":"172.21.15.151:6809/3111342741","cluster_addr":"172.21.15.151:6811/3111342741","heartbeat_back_addr":"172.21.15.151:6815/3111342741","heartbeat_front_addr":"172.21.15.151:6813/3111342741","state":["exists","up"]},{"osd":1,"uuid":"dd25925f-0ae1-494e-8529-4dc205759db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6801","nonce":3368304087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6803","nonce":3368304087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6807","nonce":3368304087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6805","nonce":3368304087}]},"public_addr":"172.21.15.151:6801/3368304087","cluster_addr":"172.21.15.151:6803/3368304087","heartbeat_back_addr":"172.21.15.151:6807/3368304087","heartbeat_front_addr":"172.21.15.151:6805/3368304087","state":["exists","up"]},{"osd":2,"uuid":"c2f18115-013d-4da6-a0fe-cb978f4494b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6823","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6825","nonce":3079373271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6827","nonce":3079373271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6831","nonce":3079373271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6829","nonce":3079373271}]},"public_addr":"172.21.15.151:6825/3079373271","cluster_addr":"172.21.15.151:6827/3079373271","heartbeat_back_addr":"172.21.15.151:6831/3079373271","heartbeat_front_addr":"172.21.15.151:6829/3079373271","state":["exists","up"]},{"osd":3,"uuid":"36edb718-2fcb-4d88-8ddc-c321eed4279d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6817","nonce":1142509041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6819","nonce":1142509041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6824","nonce":1142509041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6821","nonce":1142509041}]},"public_addr":"172.21.15.151:6817/1142509041","cluster_addr":"172.21.15.151:6819/1142509041","heartbeat_back_addr":"172.21.15.151:6824/1142509041","heartbeat_front_addr":"172.21.15.151:6821/1142509041","state":["exists","up"]},{"osd":4,"uuid":"64d33708-cd0a-4236-abe3-1b646e82ab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6811","nonce":2735495271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6813","nonce":2735495271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6817","nonce":2735495271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6815","nonce":2735495271}]},"public_addr":"172.21.15.167:6811/2735495271","cluster_addr":"172.21.15.167:6813/2735495271","heartbeat_back_addr":"172.21.15.167:6817/2735495271","heartbeat_front_addr":"172.21.15.167:6815/2735495271","state":["exists","up"]},{"osd":5,"uuid":"f14b44a9-5910-458c-b518-09a8d0de94de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6827","nonce":1494916572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6829","nonce":1494916572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6833","nonce":1494916572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6831","nonce":1494916572}]},"public_addr":"172.21.15.167:6827/1494916572","cluster_addr":"172.21.15.167:6829/1494916572","heartbeat_back_addr":"172.21.15.167:6833/1494916572","heartbeat_front_addr":"172.21.15.167:6831/1494916572","state":["exists","up"]},{"osd":6,"uuid":"39335545-2654-4710-84b0-a9aa47cfcbba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6819","nonce":2538273273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6821","nonce":2538273273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6825","nonce":2538273273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6823","nonce":2538273273}]},"public_addr":"172.21.15.167:6819/2538273273","cluster_addr":"172.21.15.167:6821/2538273273","heartbeat_back_addr":"172.21.15.167:6825/2538273273","heartbeat_front_addr":"172.21.15.167:6823/2538273273","state":["exists","up"]},{"osd":7,"uuid":"4f8bfc81-53dd-4339-ac22-1a4007d9f963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6803","nonce":265072227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6805","nonce":265072227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6809","nonce":265072227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6807","nonce":265072227}]},"public_addr":"172.21.15.167:6803/265072227","cluster_addr":"172.21.15.167:6805/265072227","heartbeat_back_addr":"172.21.15.167:6809/265072227","heartbeat_front_addr":"172.21.15.167:6807/265072227","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.150359+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.084358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.722368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.332809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.320709+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.395153+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.935250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.801443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:52:21.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.944+0000 7fa051e00640 1 -- 172.21.15.151:0/3435560434 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa02c07a2f0 msgr2=0x7fa02c07c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:21.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.944+0000 7fa051e00640 1 --2- 172.21.15.151:0/3435560434 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa02c07a2f0 0x7fa02c07c7b0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fa038033000 tx=0x7fa038004730 comp rx=0 tx=0).stop 2024-09-16T15:52:21.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.944+0000 7fa051e00640 1 -- 172.21.15.151:0/3435560434 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa04c06f9d0 msgr2=0x7fa04c145d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:21.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.944+0000 7fa051e00640 1 --2- 172.21.15.151:0/3435560434 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa04c06f9d0 0x7fa04c145d50 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fa03c00d6d0 tx=0x7fa03c00db90 comp rx=0 tx=0).stop 2024-09-16T15:52:21.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.944+0000 7fa051e00640 1 -- 172.21.15.151:0/3435560434 shutdown_connections 2024-09-16T15:52:21.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.944+0000 7fa051e00640 1 --2- 172.21.15.151:0/3435560434 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa02c07a2f0 0x7fa02c07c7b0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:21.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.944+0000 7fa051e00640 1 --2- 172.21.15.151:0/3435560434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa04c161550 0x7fa04c09ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:21.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.944+0000 7fa051e00640 1 --2- 172.21.15.151:0/3435560434 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa04c0af3d0 0x7fa04c1f8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:21.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.944+0000 7fa051e00640 1 --2- 172.21.15.151:0/3435560434 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa04c06f9d0 0x7fa04c145d50 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:21.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.944+0000 7fa051e00640 1 -- 172.21.15.151:0/3435560434 >> 172.21.15.151:0/3435560434 conn(0x7fa04c0b1950 msgr2=0x7fa04c15ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:21.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.944+0000 7fa051e00640 1 -- 172.21.15.151:0/3435560434 shutdown_connections 2024-09-16T15:52:21.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:21.944+0000 7fa051e00640 1 -- 172.21.15.151:0/3435560434 wait complete. 2024-09-16T15:52:21.962 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T15:52:21.962 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:52:22.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.121+0000 7efec2400640 1 -- 172.21.15.151:0/1501716663 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efebc0725d0 msgr2=0x7efebc073d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:22.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.121+0000 7efec2400640 1 --2- 172.21.15.151:0/1501716663 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efebc0725d0 0x7efebc073d40 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7efebc0cd570 tx=0x7efea802a5f0 comp rx=0 tx=0).stop 2024-09-16T15:52:22.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.122+0000 7efec2400640 1 -- 172.21.15.151:0/1501716663 shutdown_connections 2024-09-16T15:52:22.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.122+0000 7efec2400640 1 --2- 172.21.15.151:0/1501716663 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efebc0725d0 0x7efebc073d40 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:22.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.122+0000 7efec2400640 1 --2- 172.21.15.151:0/1501716663 >> v2:172.21.15.167:3300/0 conn(0x7efebc15bc30 0x7efebc0b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:22.125 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.122+0000 7efec2400640 1 -- 172.21.15.151:0/1501716663 >> 172.21.15.151:0/1501716663 conn(0x7efebc0cfa20 msgr2=0x7efebc0cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:22.125 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.122+0000 7efec2400640 1 -- 172.21.15.151:0/1501716663 shutdown_connections 2024-09-16T15:52:22.125 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.122+0000 7efec2400640 1 -- 172.21.15.151:0/1501716663 wait complete. 2024-09-16T15:52:22.125 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.123+0000 7efec2400640 1 Processor -- start 2024-09-16T15:52:22.125 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.123+0000 7efec2400640 1 -- start start 2024-09-16T15:52:22.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.124+0000 7efec2400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efebc0d0540 0x7efebc153bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:22.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.124+0000 7efec2400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efebc15bc30 0x7efebc148e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:22.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.124+0000 7efec2400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efebc1f67a0 0x7efebc1f5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:22.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.124+0000 7efec2400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efebc074610 con 0x7efebc1f67a0 2024-09-16T15:52:22.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.124+0000 7efec2400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efebc074490 con 0x7efebc0d0540 2024-09-16T15:52:22.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.124+0000 7efec2400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7efebc074790 con 0x7efebc15bc30 2024-09-16T15:52:22.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.125+0000 7efec1a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efebc1f67a0 0x7efebc1f5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:22.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.125+0000 7efec1a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efebc1f67a0 0x7efebc1f5b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49954/0 (socket says 172.21.15.151:49954) 2024-09-16T15:52:22.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.125+0000 7efec1a00640 1 -- 172.21.15.151:0/3035018506 learned_addr learned my addr 172.21.15.151:0/3035018506 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:52:22.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.125+0000 7efec1000640 1 --2- 172.21.15.151:0/3035018506 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efebc0d0540 0x7efebc153bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:22.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.125+0000 7efebbe00640 1 --2- 172.21.15.151:0/3035018506 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efebc15bc30 0x7efebc148e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:22.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.125+0000 7efec1a00640 1 -- 172.21.15.151:0/3035018506 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efebc15bc30 msgr2=0x7efebc148e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:22.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.125+0000 7efec1a00640 1 --2- 172.21.15.151:0/3035018506 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efebc15bc30 0x7efebc148e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:22.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.125+0000 7efec1a00640 1 -- 172.21.15.151:0/3035018506 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efebc0d0540 msgr2=0x7efebc153bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:22.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.125+0000 7efec1a00640 1 --2- 172.21.15.151:0/3035018506 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efebc0d0540 0x7efebc153bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:22.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.125+0000 7efec1a00640 1 -- 172.21.15.151:0/3035018506 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efebc1543f0 con 0x7efebc1f67a0 2024-09-16T15:52:22.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.125+0000 7efec1a00640 1 --2- 172.21.15.151:0/3035018506 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efebc1f67a0 0x7efebc1f5b20 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7efeb400c7a0 tx=0x7efeb400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:22.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.126+0000 7efeb8c00640 1 -- 172.21.15.151:0/3035018506 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb4003b70 con 0x7efebc1f67a0 2024-09-16T15:52:22.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.126+0000 7efeb8c00640 1 -- 172.21.15.151:0/3035018506 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efeb4003d30 con 0x7efebc1f67a0 2024-09-16T15:52:22.130 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.126+0000 7efec2400640 1 -- 172.21.15.151:0/3035018506 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efebc1546d0 con 0x7efebc1f67a0 2024-09-16T15:52:22.130 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.126+0000 7efeb8c00640 1 -- 172.21.15.151:0/3035018506 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb400aab0 con 0x7efebc1f67a0 2024-09-16T15:52:22.130 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.126+0000 7efec2400640 1 -- 172.21.15.151:0/3035018506 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efebc154c90 con 0x7efebc1f67a0 2024-09-16T15:52:22.130 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.127+0000 7efec2400640 1 -- 172.21.15.151:0/3035018506 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efebc0cab70 con 0x7efebc1f67a0 2024-09-16T15:52:22.130 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.128+0000 7efeb8c00640 1 -- 172.21.15.151:0/3035018506 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 41) ==== 103684+0+0 (secure 0 0 0) 0x7efeb400ac50 con 0x7efebc1f67a0 2024-09-16T15:52:22.131 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.129+0000 7efeb8c00640 1 --2- 172.21.15.151:0/3035018506 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efe9c07a2f0 0x7efe9c07c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:22.131 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.129+0000 7efeb8c00640 1 -- 172.21.15.151:0/3035018506 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5793+0+0 (secure 0 0 0) 0x7efeb4099570 con 0x7efebc1f67a0 2024-09-16T15:52:22.131 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.129+0000 7efec1000640 1 --2- 172.21.15.151:0/3035018506 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efe9c07a2f0 0x7efe9c07c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:22.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.130+0000 7efec1000640 1 --2- 172.21.15.151:0/3035018506 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efe9c07a2f0 0x7efe9c07c7b0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7efeb000bb50 tx=0x7efeb000b2e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:22.135 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.133+0000 7efeb8c00640 1 -- 172.21.15.151:0/3035018506 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7efeb4063640 con 0x7efebc1f67a0 2024-09-16T15:52:22.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.312+0000 7efec2400640 1 -- 172.21.15.151:0/3035018506 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efebc073d00 con 0x7efe9c07a2f0 2024-09-16T15:52:22.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.317+0000 7efeb8c00640 1 -- 172.21.15.151:0/3035018506 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251751 (secure 0 0 0) 0x7efebc073d00 con 0x7efe9c07a2f0 2024-09-16T15:52:22.323 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:52:22.325 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-09-16T15:52:22.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.321+0000 7efec2400640 1 -- 172.21.15.151:0/3035018506 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efe9c07a2f0 msgr2=0x7efe9c07c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:22.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.321+0000 7efec2400640 1 --2- 172.21.15.151:0/3035018506 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efe9c07a2f0 0x7efe9c07c7b0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7efeb000bb50 tx=0x7efeb000b2e0 comp rx=0 tx=0).stop 2024-09-16T15:52:22.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.321+0000 7efec2400640 1 -- 172.21.15.151:0/3035018506 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efebc1f67a0 msgr2=0x7efebc1f5b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:22.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.321+0000 7efec2400640 1 --2- 172.21.15.151:0/3035018506 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efebc1f67a0 0x7efebc1f5b20 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7efeb400c7a0 tx=0x7efeb400cc60 comp rx=0 tx=0).stop 2024-09-16T15:52:22.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.321+0000 7efec2400640 1 -- 172.21.15.151:0/3035018506 shutdown_connections 2024-09-16T15:52:22.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.321+0000 7efec2400640 1 --2- 172.21.15.151:0/3035018506 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efe9c07a2f0 0x7efe9c07c7b0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:22.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.321+0000 7efec2400640 1 --2- 172.21.15.151:0/3035018506 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efebc1f67a0 0x7efebc1f5b20 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:22.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.321+0000 7efec2400640 1 --2- 172.21.15.151:0/3035018506 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efebc15bc30 0x7efebc148e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:22.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.321+0000 7efec2400640 1 --2- 172.21.15.151:0/3035018506 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efebc0d0540 0x7efebc153bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:22.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.321+0000 7efec2400640 1 -- 172.21.15.151:0/3035018506 >> 172.21.15.151:0/3035018506 conn(0x7efebc0cfa20 msgr2=0x7efebc06e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:22.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.321+0000 7efec2400640 1 -- 172.21.15.151:0/3035018506 shutdown_connections 2024-09-16T15:52:22.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.321+0000 7efec2400640 1 -- 172.21.15.151:0/3035018506 wait complete. 2024-09-16T15:52:22.340 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":507,"stamp":"2024-09-16T15:52:21.901931+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":134,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":102,"ondisk_log_size":102,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":165,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1079288,"kb_used_data":12592,"kb_used_omap":12,"kb_used_meta":1066483,"kb_avail":753895432,"statfs":{"total":773094113280,"available":771988922368,"internally_reserved":0,"allocated":12894208,"data_stored":6299465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12738,"internal_metadata":1092079166},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.837592"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-16T15:52:17.079031+0000","last_change":"2024-09-16T15:52:17.079031+0000","last_active":"2024-09-16T15:52:17.058582+0000","last_peered":"2024-09-16T15:52:17.058582+0000","last_clean":"2024-09-16T15:52:17.058582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:52:17.079031+0000","last_undegraded":"2024-09-16T15:52:17.079031+0000","last_fullsized":"2024-09-16T15:52:17.079031+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.324514+0000","last_change":"2024-09-16T15:52:18.095601+0000","last_active":"2024-09-16T15:52:19.324514+0000","last_peered":"2024-09-16T15:52:19.324514+0000","last_clean":"2024-09-16T15:52:19.324514+0000","last_became_active":"2024-09-16T15:52:18.095305+0000","last_became_peered":"2024-09-16T15:52:18.095305+0000","last_unstale":"2024-09-16T15:52:19.324514+0000","last_undegraded":"2024-09-16T15:52:19.324514+0000","last_fullsized":"2024-09-16T15:52:19.324514+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:46:48.807361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.095580+0000","last_change":"2024-09-16T15:52:18.108749+0000","last_active":"2024-09-16T15:52:19.095580+0000","last_peered":"2024-09-16T15:52:19.095580+0000","last_clean":"2024-09-16T15:52:19.095580+0000","last_became_active":"2024-09-16T15:52:18.104818+0000","last_became_peered":"2024-09-16T15:52:18.104818+0000","last_unstale":"2024-09-16T15:52:19.095580+0000","last_undegraded":"2024-09-16T15:52:19.095580+0000","last_fullsized":"2024-09-16T15:52:19.095580+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:32:30.003302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336245+0000","last_change":"2024-09-16T15:52:18.090782+0000","last_active":"2024-09-16T15:52:19.336245+0000","last_peered":"2024-09-16T15:52:19.336245+0000","last_clean":"2024-09-16T15:52:19.336245+0000","last_became_active":"2024-09-16T15:52:18.090432+0000","last_became_peered":"2024-09-16T15:52:18.090432+0000","last_unstale":"2024-09-16T15:52:19.336245+0000","last_undegraded":"2024-09-16T15:52:19.336245+0000","last_fullsized":"2024-09-16T15:52:19.336245+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:48:32.423354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335861+0000","last_change":"2024-09-16T15:52:18.109873+0000","last_active":"2024-09-16T15:52:19.335861+0000","last_peered":"2024-09-16T15:52:19.335861+0000","last_clean":"2024-09-16T15:52:19.335861+0000","last_became_active":"2024-09-16T15:52:18.101073+0000","last_became_peered":"2024-09-16T15:52:18.101073+0000","last_unstale":"2024-09-16T15:52:19.335861+0000","last_undegraded":"2024-09-16T15:52:19.335861+0000","last_fullsized":"2024-09-16T15:52:19.335861+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:56:57.514683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.097692+0000","last_change":"2024-09-16T15:52:18.096166+0000","last_active":"2024-09-16T15:52:19.097692+0000","last_peered":"2024-09-16T15:52:19.097692+0000","last_clean":"2024-09-16T15:52:19.097692+0000","last_became_active":"2024-09-16T15:52:18.095913+0000","last_became_peered":"2024-09-16T15:52:18.095913+0000","last_unstale":"2024-09-16T15:52:19.097692+0000","last_undegraded":"2024-09-16T15:52:19.097692+0000","last_fullsized":"2024-09-16T15:52:19.097692+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:51:43.908577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098468+0000","last_change":"2024-09-16T15:52:18.111297+0000","last_active":"2024-09-16T15:52:19.098468+0000","last_peered":"2024-09-16T15:52:19.098468+0000","last_clean":"2024-09-16T15:52:19.098468+0000","last_became_active":"2024-09-16T15:52:18.111039+0000","last_became_peered":"2024-09-16T15:52:18.111039+0000","last_unstale":"2024-09-16T15:52:19.098468+0000","last_undegraded":"2024-09-16T15:52:19.098468+0000","last_fullsized":"2024-09-16T15:52:19.098468+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:08:05.616283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098459+0000","last_change":"2024-09-16T15:52:18.094809+0000","last_active":"2024-09-16T15:52:19.098459+0000","last_peered":"2024-09-16T15:52:19.098459+0000","last_clean":"2024-09-16T15:52:19.098459+0000","last_became_active":"2024-09-16T15:52:18.094538+0000","last_became_peered":"2024-09-16T15:52:18.094538+0000","last_unstale":"2024-09-16T15:52:19.098459+0000","last_undegraded":"2024-09-16T15:52:19.098459+0000","last_fullsized":"2024-09-16T15:52:19.098459+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:02:37.475979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-16T15:52:17.076806+0000","last_change":"2024-09-16T15:52:17.076806+0000","last_active":"2024-09-16T15:52:17.058582+0000","last_peered":"2024-09-16T15:52:17.058582+0000","last_clean":"2024-09-16T15:52:17.058582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:52:17.076806+0000","last_undegraded":"2024-09-16T15:52:17.076806+0000","last_fullsized":"2024-09-16T15:52:17.076806+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336431+0000","last_change":"2024-09-16T15:52:18.101787+0000","last_active":"2024-09-16T15:52:19.336431+0000","last_peered":"2024-09-16T15:52:19.336431+0000","last_clean":"2024-09-16T15:52:19.336431+0000","last_became_active":"2024-09-16T15:52:18.100961+0000","last_became_peered":"2024-09-16T15:52:18.100961+0000","last_unstale":"2024-09-16T15:52:19.336431+0000","last_undegraded":"2024-09-16T15:52:19.336431+0000","last_fullsized":"2024-09-16T15:52:19.336431+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:31:15.594998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098664+0000","last_change":"2024-09-16T15:52:18.094110+0000","last_active":"2024-09-16T15:52:19.098664+0000","last_peered":"2024-09-16T15:52:19.098664+0000","last_clean":"2024-09-16T15:52:19.098664+0000","last_became_active":"2024-09-16T15:52:18.093858+0000","last_became_peered":"2024-09-16T15:52:18.093858+0000","last_unstale":"2024-09-16T15:52:19.098664+0000","last_undegraded":"2024-09-16T15:52:19.098664+0000","last_fullsized":"2024-09-16T15:52:19.098664+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:32:24.437229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335879+0000","last_change":"2024-09-16T15:52:18.102228+0000","last_active":"2024-09-16T15:52:19.335879+0000","last_peered":"2024-09-16T15:52:19.335879+0000","last_clean":"2024-09-16T15:52:19.335879+0000","last_became_active":"2024-09-16T15:52:18.101917+0000","last_became_peered":"2024-09-16T15:52:18.101917+0000","last_unstale":"2024-09-16T15:52:19.335879+0000","last_undegraded":"2024-09-16T15:52:19.335879+0000","last_fullsized":"2024-09-16T15:52:19.335879+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:49:20.323885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.324528+0000","last_change":"2024-09-16T15:52:18.093067+0000","last_active":"2024-09-16T15:52:19.324528+0000","last_peered":"2024-09-16T15:52:19.324528+0000","last_clean":"2024-09-16T15:52:19.324528+0000","last_became_active":"2024-09-16T15:52:18.092805+0000","last_became_peered":"2024-09-16T15:52:18.092805+0000","last_unstale":"2024-09-16T15:52:19.324528+0000","last_undegraded":"2024-09-16T15:52:19.324528+0000","last_fullsized":"2024-09-16T15:52:19.324528+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:06:51.783453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.334985+0000","last_change":"2024-09-16T15:52:18.098148+0000","last_active":"2024-09-16T15:52:19.334985+0000","last_peered":"2024-09-16T15:52:19.334985+0000","last_clean":"2024-09-16T15:52:19.334985+0000","last_became_active":"2024-09-16T15:52:18.096388+0000","last_became_peered":"2024-09-16T15:52:18.096388+0000","last_unstale":"2024-09-16T15:52:19.334985+0000","last_undegraded":"2024-09-16T15:52:19.334985+0000","last_fullsized":"2024-09-16T15:52:19.334985+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:59:34.319160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-16T15:52:17.076793+0000","last_change":"2024-09-16T15:52:17.076793+0000","last_active":"2024-09-16T15:52:17.058582+0000","last_peered":"2024-09-16T15:52:17.058582+0000","last_clean":"2024-09-16T15:52:17.058582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:52:17.076793+0000","last_undegraded":"2024-09-16T15:52:17.076793+0000","last_fullsized":"2024-09-16T15:52:17.076793+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335328+0000","last_change":"2024-09-16T15:52:18.096798+0000","last_active":"2024-09-16T15:52:19.335328+0000","last_peered":"2024-09-16T15:52:19.335328+0000","last_clean":"2024-09-16T15:52:19.335328+0000","last_became_active":"2024-09-16T15:52:18.096396+0000","last_became_peered":"2024-09-16T15:52:18.096396+0000","last_unstale":"2024-09-16T15:52:19.335328+0000","last_undegraded":"2024-09-16T15:52:19.335328+0000","last_fullsized":"2024-09-16T15:52:19.335328+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:17:25.344931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.095682+0000","last_change":"2024-09-16T15:52:18.113939+0000","last_active":"2024-09-16T15:52:19.095682+0000","last_peered":"2024-09-16T15:52:19.095682+0000","last_clean":"2024-09-16T15:52:19.095682+0000","last_became_active":"2024-09-16T15:52:18.113634+0000","last_became_peered":"2024-09-16T15:52:18.113634+0000","last_unstale":"2024-09-16T15:52:19.095682+0000","last_undegraded":"2024-09-16T15:52:19.095682+0000","last_fullsized":"2024-09-16T15:52:19.095682+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:04:19.441890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098743+0000","last_change":"2024-09-16T15:52:18.106010+0000","last_active":"2024-09-16T15:52:19.098743+0000","last_peered":"2024-09-16T15:52:19.098743+0000","last_clean":"2024-09-16T15:52:19.098743+0000","last_became_active":"2024-09-16T15:52:18.105722+0000","last_became_peered":"2024-09-16T15:52:18.105722+0000","last_unstale":"2024-09-16T15:52:19.098743+0000","last_undegraded":"2024-09-16T15:52:19.098743+0000","last_fullsized":"2024-09-16T15:52:19.098743+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:29:30.985879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.095786+0000","last_change":"2024-09-16T15:52:18.108828+0000","last_active":"2024-09-16T15:52:19.095786+0000","last_peered":"2024-09-16T15:52:19.095786+0000","last_clean":"2024-09-16T15:52:19.095786+0000","last_became_active":"2024-09-16T15:52:18.104728+0000","last_became_peered":"2024-09-16T15:52:18.104728+0000","last_unstale":"2024-09-16T15:52:19.095786+0000","last_undegraded":"2024-09-16T15:52:19.095786+0000","last_fullsized":"2024-09-16T15:52:19.095786+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:08:15.649535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.095454+0000","last_change":"2024-09-16T15:52:18.104490+0000","last_active":"2024-09-16T15:52:19.095454+0000","last_peered":"2024-09-16T15:52:19.095454+0000","last_clean":"2024-09-16T15:52:19.095454+0000","last_became_active":"2024-09-16T15:52:18.104176+0000","last_became_peered":"2024-09-16T15:52:18.104176+0000","last_unstale":"2024-09-16T15:52:19.095454+0000","last_undegraded":"2024-09-16T15:52:19.095454+0000","last_fullsized":"2024-09-16T15:52:19.095454+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:38:00.758145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098816+0000","last_change":"2024-09-16T15:52:18.100178+0000","last_active":"2024-09-16T15:52:19.098816+0000","last_peered":"2024-09-16T15:52:19.098816+0000","last_clean":"2024-09-16T15:52:19.098816+0000","last_became_active":"2024-09-16T15:52:18.099483+0000","last_became_peered":"2024-09-16T15:52:18.099483+0000","last_unstale":"2024-09-16T15:52:19.098816+0000","last_undegraded":"2024-09-16T15:52:19.098816+0000","last_fullsized":"2024-09-16T15:52:19.098816+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:16:33.962636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.097266+0000","last_change":"2024-09-16T15:52:18.095116+0000","last_active":"2024-09-16T15:52:19.097266+0000","last_peered":"2024-09-16T15:52:19.097266+0000","last_clean":"2024-09-16T15:52:19.097266+0000","last_became_active":"2024-09-16T15:52:18.090617+0000","last_became_peered":"2024-09-16T15:52:18.090617+0000","last_unstale":"2024-09-16T15:52:19.097266+0000","last_undegraded":"2024-09-16T15:52:19.097266+0000","last_fullsized":"2024-09-16T15:52:19.097266+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:23:03.363317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335071+0000","last_change":"2024-09-16T15:52:18.102959+0000","last_active":"2024-09-16T15:52:19.335071+0000","last_peered":"2024-09-16T15:52:19.335071+0000","last_clean":"2024-09-16T15:52:19.335071+0000","last_became_active":"2024-09-16T15:52:18.102684+0000","last_became_peered":"2024-09-16T15:52:18.102684+0000","last_unstale":"2024-09-16T15:52:19.335071+0000","last_undegraded":"2024-09-16T15:52:19.335071+0000","last_fullsized":"2024-09-16T15:52:19.335071+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:26:35.565582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336817+0000","last_change":"2024-09-16T15:52:18.102292+0000","last_active":"2024-09-16T15:52:19.336817+0000","last_peered":"2024-09-16T15:52:19.336817+0000","last_clean":"2024-09-16T15:52:19.336817+0000","last_became_active":"2024-09-16T15:52:18.102017+0000","last_became_peered":"2024-09-16T15:52:18.102017+0000","last_unstale":"2024-09-16T15:52:19.336817+0000","last_undegraded":"2024-09-16T15:52:19.336817+0000","last_fullsized":"2024-09-16T15:52:19.336817+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:34:23.254431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098723+0000","last_change":"2024-09-16T15:52:18.089811+0000","last_active":"2024-09-16T15:52:19.098723+0000","last_peered":"2024-09-16T15:52:19.098723+0000","last_clean":"2024-09-16T15:52:19.098723+0000","last_became_active":"2024-09-16T15:52:18.089302+0000","last_became_peered":"2024-09-16T15:52:18.089302+0000","last_unstale":"2024-09-16T15:52:19.098723+0000","last_undegraded":"2024-09-16T15:52:19.098723+0000","last_fullsized":"2024-09-16T15:52:19.098723+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:44:21.343778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.097694+0000","last_change":"2024-09-16T15:52:18.107014+0000","last_active":"2024-09-16T15:52:19.097694+0000","last_peered":"2024-09-16T15:52:19.097694+0000","last_clean":"2024-09-16T15:52:19.097694+0000","last_became_active":"2024-09-16T15:52:18.106706+0000","last_became_peered":"2024-09-16T15:52:18.106706+0000","last_unstale":"2024-09-16T15:52:19.097694+0000","last_undegraded":"2024-09-16T15:52:19.097694+0000","last_fullsized":"2024-09-16T15:52:19.097694+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:41:14.841086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336117+0000","last_change":"2024-09-16T15:52:18.102898+0000","last_active":"2024-09-16T15:52:19.336117+0000","last_peered":"2024-09-16T15:52:19.336117+0000","last_clean":"2024-09-16T15:52:19.336117+0000","last_became_active":"2024-09-16T15:52:18.102229+0000","last_became_peered":"2024-09-16T15:52:18.102229+0000","last_unstale":"2024-09-16T15:52:19.336117+0000","last_undegraded":"2024-09-16T15:52:19.336117+0000","last_fullsized":"2024-09-16T15:52:19.336117+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:21:53.345211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.097217+0000","last_change":"2024-09-16T15:52:18.099345+0000","last_active":"2024-09-16T15:52:19.097217+0000","last_peered":"2024-09-16T15:52:19.097217+0000","last_clean":"2024-09-16T15:52:19.097217+0000","last_became_active":"2024-09-16T15:52:18.098646+0000","last_became_peered":"2024-09-16T15:52:18.098646+0000","last_unstale":"2024-09-16T15:52:19.097217+0000","last_undegraded":"2024-09-16T15:52:19.097217+0000","last_fullsized":"2024-09-16T15:52:19.097217+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:28:37.287849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335126+0000","last_change":"2024-09-16T15:52:18.106525+0000","last_active":"2024-09-16T15:52:19.335126+0000","last_peered":"2024-09-16T15:52:19.335126+0000","last_clean":"2024-09-16T15:52:19.335126+0000","last_became_active":"2024-09-16T15:52:18.103778+0000","last_became_peered":"2024-09-16T15:52:18.103778+0000","last_unstale":"2024-09-16T15:52:19.335126+0000","last_undegraded":"2024-09-16T15:52:19.335126+0000","last_fullsized":"2024-09-16T15:52:19.335126+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:46:53.909612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098944+0000","last_change":"2024-09-16T15:52:18.087005+0000","last_active":"2024-09-16T15:52:19.098944+0000","last_peered":"2024-09-16T15:52:19.098944+0000","last_clean":"2024-09-16T15:52:19.098944+0000","last_became_active":"2024-09-16T15:52:18.086463+0000","last_became_peered":"2024-09-16T15:52:18.086463+0000","last_unstale":"2024-09-16T15:52:19.098944+0000","last_undegraded":"2024-09-16T15:52:19.098944+0000","last_fullsized":"2024-09-16T15:52:19.098944+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:23:31.782022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-16T15:52:17.076332+0000","last_change":"2024-09-16T15:52:17.076332+0000","last_active":"2024-09-16T15:52:17.058582+0000","last_peered":"2024-09-16T15:52:17.058582+0000","last_clean":"2024-09-16T15:52:17.058582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:52:17.076332+0000","last_undegraded":"2024-09-16T15:52:17.076332+0000","last_fullsized":"2024-09-16T15:52:17.076332+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335551+0000","last_change":"2024-09-16T15:52:18.106237+0000","last_active":"2024-09-16T15:52:19.335551+0000","last_peered":"2024-09-16T15:52:19.335551+0000","last_clean":"2024-09-16T15:52:19.335551+0000","last_became_active":"2024-09-16T15:52:18.103396+0000","last_became_peered":"2024-09-16T15:52:18.103396+0000","last_unstale":"2024-09-16T15:52:19.335551+0000","last_undegraded":"2024-09-16T15:52:19.335551+0000","last_fullsized":"2024-09-16T15:52:19.335551+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:35:46.682990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-16T15:52:17.081705+0000","last_change":"2024-09-16T15:52:17.081705+0000","last_active":"2024-09-16T15:52:17.058582+0000","last_peered":"2024-09-16T15:52:17.058582+0000","last_clean":"2024-09-16T15:52:17.058582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:52:17.081705+0000","last_undegraded":"2024-09-16T15:52:17.081705+0000","last_fullsized":"2024-09-16T15:52:17.081705+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"117'4","reported_seq":19,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335332+0000","last_change":"2024-09-16T15:52:17.086334+0000","last_active":"2024-09-16T15:52:19.335332+0000","last_peered":"2024-09-16T15:52:19.335332+0000","last_clean":"2024-09-16T15:52:19.335332+0000","last_became_active":"2024-09-16T15:52:17.086082+0000","last_became_peered":"2024-09-16T15:52:17.086082+0000","last_unstale":"2024-09-16T15:52:19.335332+0000","last_undegraded":"2024-09-16T15:52:19.335332+0000","last_fullsized":"2024-09-16T15:52:19.335332+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:25:34.552766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335430+0000","last_change":"2024-09-16T15:52:18.102484+0000","last_active":"2024-09-16T15:52:19.335430+0000","last_peered":"2024-09-16T15:52:19.335430+0000","last_clean":"2024-09-16T15:52:19.335430+0000","last_became_active":"2024-09-16T15:52:18.102159+0000","last_became_peered":"2024-09-16T15:52:18.102159+0000","last_unstale":"2024-09-16T15:52:19.335430+0000","last_undegraded":"2024-09-16T15:52:19.335430+0000","last_fullsized":"2024-09-16T15:52:19.335430+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:18:36.837288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1e","version":"117'1","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.324560+0000","last_change":"2024-09-16T15:52:17.082085+0000","last_active":"2024-09-16T15:52:19.324560+0000","last_peered":"2024-09-16T15:52:19.324560+0000","last_clean":"2024-09-16T15:52:19.324560+0000","last_became_active":"2024-09-16T15:52:17.081840+0000","last_became_peered":"2024-09-16T15:52:17.081840+0000","last_unstale":"2024-09-16T15:52:19.324560+0000","last_undegraded":"2024-09-16T15:52:19.324560+0000","last_fullsized":"2024-09-16T15:52:19.324560+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:46:52.901601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.097360+0000","last_change":"2024-09-16T15:52:18.104067+0000","last_active":"2024-09-16T15:52:19.097360+0000","last_peered":"2024-09-16T15:52:19.097360+0000","last_clean":"2024-09-16T15:52:19.097360+0000","last_became_active":"2024-09-16T15:52:18.103764+0000","last_became_peered":"2024-09-16T15:52:18.103764+0000","last_unstale":"2024-09-16T15:52:19.097360+0000","last_undegraded":"2024-09-16T15:52:19.097360+0000","last_fullsized":"2024-09-16T15:52:19.097360+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:51:28.451444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1d","version":"117'2","reported_seq":17,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336225+0000","last_change":"2024-09-16T15:52:17.083135+0000","last_active":"2024-09-16T15:52:19.336225+0000","last_peered":"2024-09-16T15:52:19.336225+0000","last_clean":"2024-09-16T15:52:19.336225+0000","last_became_active":"2024-09-16T15:52:17.082799+0000","last_became_peered":"2024-09-16T15:52:17.082799+0000","last_unstale":"2024-09-16T15:52:19.336225+0000","last_undegraded":"2024-09-16T15:52:19.336225+0000","last_fullsized":"2024-09-16T15:52:19.336225+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:43:05.707621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.095847+0000","last_change":"2024-09-16T15:52:18.109036+0000","last_active":"2024-09-16T15:52:19.095847+0000","last_peered":"2024-09-16T15:52:19.095847+0000","last_clean":"2024-09-16T15:52:19.095847+0000","last_became_active":"2024-09-16T15:52:18.106979+0000","last_became_peered":"2024-09-16T15:52:18.106979+0000","last_unstale":"2024-09-16T15:52:19.095847+0000","last_undegraded":"2024-09-16T15:52:19.095847+0000","last_fullsized":"2024-09-16T15:52:19.095847+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:38:25.952399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"117'2","reported_seq":17,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336324+0000","last_change":"2024-09-16T15:52:17.087728+0000","last_active":"2024-09-16T15:52:19.336324+0000","last_peered":"2024-09-16T15:52:19.336324+0000","last_clean":"2024-09-16T15:52:19.336324+0000","last_became_active":"2024-09-16T15:52:17.087063+0000","last_became_peered":"2024-09-16T15:52:17.087063+0000","last_unstale":"2024-09-16T15:52:19.336324+0000","last_undegraded":"2024-09-16T15:52:19.336324+0000","last_fullsized":"2024-09-16T15:52:19.336324+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:57:32.844856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336678+0000","last_change":"2024-09-16T15:52:18.111393+0000","last_active":"2024-09-16T15:52:19.336678+0000","last_peered":"2024-09-16T15:52:19.336678+0000","last_clean":"2024-09-16T15:52:19.336678+0000","last_became_active":"2024-09-16T15:52:18.111116+0000","last_became_peered":"2024-09-16T15:52:18.111116+0000","last_unstale":"2024-09-16T15:52:19.336678+0000","last_undegraded":"2024-09-16T15:52:19.336678+0000","last_fullsized":"2024-09-16T15:52:19.336678+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:59:34.319160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1b","version":"117'1","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.095273+0000","last_change":"2024-09-16T15:52:17.083411+0000","last_active":"2024-09-16T15:52:19.095273+0000","last_peered":"2024-09-16T15:52:19.095273+0000","last_clean":"2024-09-16T15:52:19.095273+0000","last_became_active":"2024-09-16T15:52:17.083098+0000","last_became_peered":"2024-09-16T15:52:17.083098+0000","last_unstale":"2024-09-16T15:52:19.095273+0000","last_undegraded":"2024-09-16T15:52:19.095273+0000","last_fullsized":"2024-09-16T15:52:19.095273+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:48:17.710440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098564+0000","last_change":"2024-09-16T15:52:18.099617+0000","last_active":"2024-09-16T15:52:19.098564+0000","last_peered":"2024-09-16T15:52:19.098564+0000","last_clean":"2024-09-16T15:52:19.098564+0000","last_became_active":"2024-09-16T15:52:18.099149+0000","last_became_peered":"2024-09-16T15:52:18.099149+0000","last_unstale":"2024-09-16T15:52:19.098564+0000","last_undegraded":"2024-09-16T15:52:19.098564+0000","last_fullsized":"2024-09-16T15:52:19.098564+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:41:19.687707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-16T15:52:17.092540+0000","last_change":"2024-09-16T15:52:17.086053+0000","last_active":"2024-09-16T15:52:17.092540+0000","last_peered":"2024-09-16T15:52:17.092540+0000","last_clean":"2024-09-16T15:52:17.092540+0000","last_became_active":"2024-09-16T15:52:17.084186+0000","last_became_peered":"2024-09-16T15:52:17.084186+0000","last_unstale":"2024-09-16T15:52:17.092540+0000","last_undegraded":"2024-09-16T15:52:17.092540+0000","last_fullsized":"2024-09-16T15:52:17.092540+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:32:52.491531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336424+0000","last_change":"2024-09-16T15:52:18.101280+0000","last_active":"2024-09-16T15:52:19.336424+0000","last_peered":"2024-09-16T15:52:19.336424+0000","last_clean":"2024-09-16T15:52:19.336424+0000","last_became_active":"2024-09-16T15:52:18.100938+0000","last_became_peered":"2024-09-16T15:52:18.100938+0000","last_unstale":"2024-09-16T15:52:19.336424+0000","last_undegraded":"2024-09-16T15:52:19.336424+0000","last_fullsized":"2024-09-16T15:52:19.336424+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:18:58.903310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098529+0000","last_change":"2024-09-16T15:52:17.090751+0000","last_active":"2024-09-16T15:52:19.098529+0000","last_peered":"2024-09-16T15:52:19.098529+0000","last_clean":"2024-09-16T15:52:19.098529+0000","last_became_active":"2024-09-16T15:52:17.090390+0000","last_became_peered":"2024-09-16T15:52:17.090390+0000","last_unstale":"2024-09-16T15:52:19.098529+0000","last_undegraded":"2024-09-16T15:52:19.098529+0000","last_fullsized":"2024-09-16T15:52:19.098529+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:58:11.585044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098903+0000","last_change":"2024-09-16T15:52:18.090433+0000","last_active":"2024-09-16T15:52:19.098903+0000","last_peered":"2024-09-16T15:52:19.098903+0000","last_clean":"2024-09-16T15:52:19.098903+0000","last_became_active":"2024-09-16T15:52:18.090169+0000","last_became_peered":"2024-09-16T15:52:18.090169+0000","last_unstale":"2024-09-16T15:52:19.098903+0000","last_undegraded":"2024-09-16T15:52:19.098903+0000","last_fullsized":"2024-09-16T15:52:19.098903+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:20:57.801785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336267+0000","last_change":"2024-09-16T15:52:17.093517+0000","last_active":"2024-09-16T15:52:19.336267+0000","last_peered":"2024-09-16T15:52:19.336267+0000","last_clean":"2024-09-16T15:52:19.336267+0000","last_became_active":"2024-09-16T15:52:17.093229+0000","last_became_peered":"2024-09-16T15:52:17.093229+0000","last_unstale":"2024-09-16T15:52:19.336267+0000","last_undegraded":"2024-09-16T15:52:19.336267+0000","last_fullsized":"2024-09-16T15:52:19.336267+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:22:18.051932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.324584+0000","last_change":"2024-09-16T15:52:18.099124+0000","last_active":"2024-09-16T15:52:19.324584+0000","last_peered":"2024-09-16T15:52:19.324584+0000","last_clean":"2024-09-16T15:52:19.324584+0000","last_became_active":"2024-09-16T15:52:18.098576+0000","last_became_peered":"2024-09-16T15:52:18.098576+0000","last_unstale":"2024-09-16T15:52:19.324584+0000","last_undegraded":"2024-09-16T15:52:19.324584+0000","last_fullsized":"2024-09-16T15:52:19.324584+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:50:42.139677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336848+0000","last_change":"2024-09-16T15:52:17.087919+0000","last_active":"2024-09-16T15:52:19.336848+0000","last_peered":"2024-09-16T15:52:19.336848+0000","last_clean":"2024-09-16T15:52:19.336848+0000","last_became_active":"2024-09-16T15:52:17.086912+0000","last_became_peered":"2024-09-16T15:52:17.086912+0000","last_unstale":"2024-09-16T15:52:19.336848+0000","last_undegraded":"2024-09-16T15:52:19.336848+0000","last_fullsized":"2024-09-16T15:52:19.336848+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:22:20.426416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.324724+0000","last_change":"2024-09-16T15:52:18.093980+0000","last_active":"2024-09-16T15:52:19.324724+0000","last_peered":"2024-09-16T15:52:19.324724+0000","last_clean":"2024-09-16T15:52:19.324724+0000","last_became_active":"2024-09-16T15:52:18.093733+0000","last_became_peered":"2024-09-16T15:52:18.093733+0000","last_unstale":"2024-09-16T15:52:19.324724+0000","last_undegraded":"2024-09-16T15:52:19.324724+0000","last_fullsized":"2024-09-16T15:52:19.324724+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:37:17.979172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336543+0000","last_change":"2024-09-16T15:52:17.088021+0000","last_active":"2024-09-16T15:52:19.336543+0000","last_peered":"2024-09-16T15:52:19.336543+0000","last_clean":"2024-09-16T15:52:19.336543+0000","last_became_active":"2024-09-16T15:52:17.087740+0000","last_became_peered":"2024-09-16T15:52:17.087740+0000","last_unstale":"2024-09-16T15:52:19.336543+0000","last_undegraded":"2024-09-16T15:52:19.336543+0000","last_fullsized":"2024-09-16T15:52:19.336543+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:20:18.506788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.097462+0000","last_change":"2024-09-16T15:52:18.088933+0000","last_active":"2024-09-16T15:52:19.097462+0000","last_peered":"2024-09-16T15:52:19.097462+0000","last_clean":"2024-09-16T15:52:19.097462+0000","last_became_active":"2024-09-16T15:52:18.088644+0000","last_became_peered":"2024-09-16T15:52:18.088644+0000","last_unstale":"2024-09-16T15:52:19.097462+0000","last_undegraded":"2024-09-16T15:52:19.097462+0000","last_fullsized":"2024-09-16T15:52:19.097462+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:50:44.707951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"117'3","reported_seq":18,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335658+0000","last_change":"2024-09-16T15:52:17.090647+0000","last_active":"2024-09-16T15:52:19.335658+0000","last_peered":"2024-09-16T15:52:19.335658+0000","last_clean":"2024-09-16T15:52:19.335658+0000","last_became_active":"2024-09-16T15:52:17.089382+0000","last_became_peered":"2024-09-16T15:52:17.089382+0000","last_unstale":"2024-09-16T15:52:19.335658+0000","last_undegraded":"2024-09-16T15:52:19.335658+0000","last_fullsized":"2024-09-16T15:52:19.335658+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:22:34.607260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098802+0000","last_change":"2024-09-16T15:52:18.107440+0000","last_active":"2024-09-16T15:52:19.098802+0000","last_peered":"2024-09-16T15:52:19.098802+0000","last_clean":"2024-09-16T15:52:19.098802+0000","last_became_active":"2024-09-16T15:52:18.107190+0000","last_became_peered":"2024-09-16T15:52:18.107190+0000","last_unstale":"2024-09-16T15:52:19.098802+0000","last_undegraded":"2024-09-16T15:52:19.098802+0000","last_fullsized":"2024-09-16T15:52:19.098802+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:05:50.369655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.14","version":"117'6","reported_seq":21,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336439+0000","last_change":"2024-09-16T15:52:17.088066+0000","last_active":"2024-09-16T15:52:19.336439+0000","last_peered":"2024-09-16T15:52:19.336439+0000","last_clean":"2024-09-16T15:52:19.336439+0000","last_became_active":"2024-09-16T15:52:17.087451+0000","last_became_peered":"2024-09-16T15:52:17.087451+0000","last_unstale":"2024-09-16T15:52:19.336439+0000","last_undegraded":"2024-09-16T15:52:19.336439+0000","last_fullsized":"2024-09-16T15:52:19.336439+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:41:10.165278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335355+0000","last_change":"2024-09-16T15:52:18.098410+0000","last_active":"2024-09-16T15:52:19.335355+0000","last_peered":"2024-09-16T15:52:19.335355+0000","last_clean":"2024-09-16T15:52:19.335355+0000","last_became_active":"2024-09-16T15:52:18.098149+0000","last_became_peered":"2024-09-16T15:52:18.098149+0000","last_unstale":"2024-09-16T15:52:19.335355+0000","last_undegraded":"2024-09-16T15:52:19.335355+0000","last_fullsized":"2024-09-16T15:52:19.335355+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:25:09.258265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"117'2","reported_seq":17,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335984+0000","last_change":"2024-09-16T15:52:17.086637+0000","last_active":"2024-09-16T15:52:19.335984+0000","last_peered":"2024-09-16T15:52:19.335984+0000","last_clean":"2024-09-16T15:52:19.335984+0000","last_became_active":"2024-09-16T15:52:17.086116+0000","last_became_peered":"2024-09-16T15:52:17.086116+0000","last_unstale":"2024-09-16T15:52:19.335984+0000","last_undegraded":"2024-09-16T15:52:19.335984+0000","last_fullsized":"2024-09-16T15:52:19.335984+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:33:42.514181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335966+0000","last_change":"2024-09-16T15:52:18.108762+0000","last_active":"2024-09-16T15:52:19.335966+0000","last_peered":"2024-09-16T15:52:19.335966+0000","last_clean":"2024-09-16T15:52:19.335966+0000","last_became_active":"2024-09-16T15:52:18.106039+0000","last_became_peered":"2024-09-16T15:52:18.106039+0000","last_unstale":"2024-09-16T15:52:19.335966+0000","last_undegraded":"2024-09-16T15:52:19.335966+0000","last_fullsized":"2024-09-16T15:52:19.335966+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:20:44.949446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.095450+0000","last_change":"2024-09-16T15:52:17.091004+0000","last_active":"2024-09-16T15:52:19.095450+0000","last_peered":"2024-09-16T15:52:19.095450+0000","last_clean":"2024-09-16T15:52:19.095450+0000","last_became_active":"2024-09-16T15:52:17.090693+0000","last_became_peered":"2024-09-16T15:52:17.090693+0000","last_unstale":"2024-09-16T15:52:19.095450+0000","last_undegraded":"2024-09-16T15:52:19.095450+0000","last_fullsized":"2024-09-16T15:52:19.095450+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:36:52.865477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336681+0000","last_change":"2024-09-16T15:52:18.096055+0000","last_active":"2024-09-16T15:52:19.336681+0000","last_peered":"2024-09-16T15:52:19.336681+0000","last_clean":"2024-09-16T15:52:19.336681+0000","last_became_active":"2024-09-16T15:52:18.095784+0000","last_became_peered":"2024-09-16T15:52:18.095784+0000","last_unstale":"2024-09-16T15:52:19.336681+0000","last_undegraded":"2024-09-16T15:52:19.336681+0000","last_fullsized":"2024-09-16T15:52:19.336681+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:01:04.752236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-16T15:52:17.089606+0000","last_change":"2024-09-16T15:52:17.084493+0000","last_active":"2024-09-16T15:52:17.089606+0000","last_peered":"2024-09-16T15:52:17.089606+0000","last_clean":"2024-09-16T15:52:17.089606+0000","last_became_active":"2024-09-16T15:52:17.083507+0000","last_became_peered":"2024-09-16T15:52:17.083507+0000","last_unstale":"2024-09-16T15:52:17.089606+0000","last_undegraded":"2024-09-16T15:52:17.089606+0000","last_fullsized":"2024-09-16T15:52:17.089606+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:22:31.431419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-16T15:52:17.076690+0000","last_change":"2024-09-16T15:52:17.076690+0000","last_active":"2024-09-16T15:52:17.058582+0000","last_peered":"2024-09-16T15:52:17.058582+0000","last_clean":"2024-09-16T15:52:17.058582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:52:17.076690+0000","last_undegraded":"2024-09-16T15:52:17.076690+0000","last_fullsized":"2024-09-16T15:52:17.076690+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.f","version":"117'1","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.097553+0000","last_change":"2024-09-16T15:52:17.079920+0000","last_active":"2024-09-16T15:52:19.097553+0000","last_peered":"2024-09-16T15:52:19.097553+0000","last_clean":"2024-09-16T15:52:19.097553+0000","last_became_active":"2024-09-16T15:52:17.079394+0000","last_became_peered":"2024-09-16T15:52:17.079394+0000","last_unstale":"2024-09-16T15:52:19.097553+0000","last_undegraded":"2024-09-16T15:52:19.097553+0000","last_fullsized":"2024-09-16T15:52:19.097553+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:27:33.808174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-16T15:52:17.075988+0000","last_change":"2024-09-16T15:52:17.075988+0000","last_active":"2024-09-16T15:52:17.058582+0000","last_peered":"2024-09-16T15:52:17.058582+0000","last_clean":"2024-09-16T15:52:17.058582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:52:17.075988+0000","last_undegraded":"2024-09-16T15:52:17.075988+0000","last_fullsized":"2024-09-16T15:52:17.075988+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"117'2","reported_seq":17,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336861+0000","last_change":"2024-09-16T15:52:17.087201+0000","last_active":"2024-09-16T15:52:19.336861+0000","last_peered":"2024-09-16T15:52:19.336861+0000","last_clean":"2024-09-16T15:52:19.336861+0000","last_became_active":"2024-09-16T15:52:17.085562+0000","last_became_peered":"2024-09-16T15:52:17.085562+0000","last_unstale":"2024-09-16T15:52:19.336861+0000","last_undegraded":"2024-09-16T15:52:19.336861+0000","last_fullsized":"2024-09-16T15:52:19.336861+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:20:46.960384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098813+0000","last_change":"2024-09-16T15:52:18.097949+0000","last_active":"2024-09-16T15:52:19.098813+0000","last_peered":"2024-09-16T15:52:19.098813+0000","last_clean":"2024-09-16T15:52:19.098813+0000","last_became_active":"2024-09-16T15:52:18.097693+0000","last_became_peered":"2024-09-16T15:52:18.097693+0000","last_unstale":"2024-09-16T15:52:19.098813+0000","last_undegraded":"2024-09-16T15:52:19.098813+0000","last_fullsized":"2024-09-16T15:52:19.098813+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:24:52.156928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.d","version":"117'1","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335394+0000","last_change":"2024-09-16T15:52:17.087628+0000","last_active":"2024-09-16T15:52:19.335394+0000","last_peered":"2024-09-16T15:52:19.335394+0000","last_clean":"2024-09-16T15:52:19.335394+0000","last_became_active":"2024-09-16T15:52:17.087384+0000","last_became_peered":"2024-09-16T15:52:17.087384+0000","last_unstale":"2024-09-16T15:52:19.335394+0000","last_undegraded":"2024-09-16T15:52:19.335394+0000","last_fullsized":"2024-09-16T15:52:19.335394+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:42:11.616130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335328+0000","last_change":"2024-09-16T15:52:18.114650+0000","last_active":"2024-09-16T15:52:19.335328+0000","last_peered":"2024-09-16T15:52:19.335328+0000","last_clean":"2024-09-16T15:52:19.335328+0000","last_became_active":"2024-09-16T15:52:18.111886+0000","last_became_peered":"2024-09-16T15:52:18.111886+0000","last_unstale":"2024-09-16T15:52:19.335328+0000","last_undegraded":"2024-09-16T15:52:19.335328+0000","last_fullsized":"2024-09-16T15:52:19.335328+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:27:33.482617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335048+0000","last_change":"2024-09-16T15:52:17.090919+0000","last_active":"2024-09-16T15:52:19.335048+0000","last_peered":"2024-09-16T15:52:19.335048+0000","last_clean":"2024-09-16T15:52:19.335048+0000","last_became_active":"2024-09-16T15:52:17.090639+0000","last_became_peered":"2024-09-16T15:52:17.090639+0000","last_unstale":"2024-09-16T15:52:19.335048+0000","last_undegraded":"2024-09-16T15:52:19.335048+0000","last_fullsized":"2024-09-16T15:52:19.335048+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:02:59.981560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.095970+0000","last_change":"2024-09-16T15:52:18.099682+0000","last_active":"2024-09-16T15:52:19.095970+0000","last_peered":"2024-09-16T15:52:19.095970+0000","last_clean":"2024-09-16T15:52:19.095970+0000","last_became_active":"2024-09-16T15:52:18.093247+0000","last_became_peered":"2024-09-16T15:52:18.093247+0000","last_unstale":"2024-09-16T15:52:19.095970+0000","last_undegraded":"2024-09-16T15:52:19.095970+0000","last_fullsized":"2024-09-16T15:52:19.095970+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:39:08.375481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.b","version":"117'2","reported_seq":17,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336094+0000","last_change":"2024-09-16T15:52:17.086366+0000","last_active":"2024-09-16T15:52:19.336094+0000","last_peered":"2024-09-16T15:52:19.336094+0000","last_clean":"2024-09-16T15:52:19.336094+0000","last_became_active":"2024-09-16T15:52:17.085765+0000","last_became_peered":"2024-09-16T15:52:17.085765+0000","last_unstale":"2024-09-16T15:52:19.336094+0000","last_undegraded":"2024-09-16T15:52:19.336094+0000","last_fullsized":"2024-09-16T15:52:19.336094+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:32:36.741556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-16T15:52:17.076139+0000","last_change":"2024-09-16T15:52:17.076139+0000","last_active":"2024-09-16T15:52:17.058582+0000","last_peered":"2024-09-16T15:52:17.058582+0000","last_clean":"2024-09-16T15:52:17.058582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:52:17.076139+0000","last_undegraded":"2024-09-16T15:52:17.076139+0000","last_fullsized":"2024-09-16T15:52:17.076139+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-16T15:52:17.094304+0000","last_change":"2024-09-16T15:52:17.090433+0000","last_active":"2024-09-16T15:52:17.094304+0000","last_peered":"2024-09-16T15:52:17.094304+0000","last_clean":"2024-09-16T15:52:17.094304+0000","last_became_active":"2024-09-16T15:52:17.089911+0000","last_became_peered":"2024-09-16T15:52:17.089911+0000","last_unstale":"2024-09-16T15:52:17.094304+0000","last_undegraded":"2024-09-16T15:52:17.094304+0000","last_fullsized":"2024-09-16T15:52:17.094304+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:29:31.775552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.334976+0000","last_change":"2024-09-16T15:52:18.102029+0000","last_active":"2024-09-16T15:52:19.334976+0000","last_peered":"2024-09-16T15:52:19.334976+0000","last_clean":"2024-09-16T15:52:19.334976+0000","last_became_active":"2024-09-16T15:52:18.101193+0000","last_became_peered":"2024-09-16T15:52:18.101193+0000","last_unstale":"2024-09-16T15:52:19.334976+0000","last_undegraded":"2024-09-16T15:52:19.334976+0000","last_fullsized":"2024-09-16T15:52:19.334976+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:14:39.539437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336623+0000","last_change":"2024-09-16T15:52:17.089371+0000","last_active":"2024-09-16T15:52:19.336623+0000","last_peered":"2024-09-16T15:52:19.336623+0000","last_clean":"2024-09-16T15:52:19.336623+0000","last_became_active":"2024-09-16T15:52:17.089099+0000","last_became_peered":"2024-09-16T15:52:17.089099+0000","last_unstale":"2024-09-16T15:52:19.336623+0000","last_undegraded":"2024-09-16T15:52:19.336623+0000","last_fullsized":"2024-09-16T15:52:19.336623+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:04:46.705752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335539+0000","last_change":"2024-09-16T15:52:18.106035+0000","last_active":"2024-09-16T15:52:19.335539+0000","last_peered":"2024-09-16T15:52:19.335539+0000","last_clean":"2024-09-16T15:52:19.335539+0000","last_became_active":"2024-09-16T15:52:18.105771+0000","last_became_peered":"2024-09-16T15:52:18.105771+0000","last_unstale":"2024-09-16T15:52:19.335539+0000","last_undegraded":"2024-09-16T15:52:19.335539+0000","last_fullsized":"2024-09-16T15:52:19.335539+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:07:04.881864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336440+0000","last_change":"2024-09-16T15:52:17.078927+0000","last_active":"2024-09-16T15:52:19.336440+0000","last_peered":"2024-09-16T15:52:19.336440+0000","last_clean":"2024-09-16T15:52:19.336440+0000","last_became_active":"2024-09-16T15:52:17.078623+0000","last_became_peered":"2024-09-16T15:52:17.078623+0000","last_unstale":"2024-09-16T15:52:19.336440+0000","last_undegraded":"2024-09-16T15:52:19.336440+0000","last_fullsized":"2024-09-16T15:52:19.336440+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:32:34.228931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335362+0000","last_change":"2024-09-16T15:52:18.099029+0000","last_active":"2024-09-16T15:52:19.335362+0000","last_peered":"2024-09-16T15:52:19.335362+0000","last_clean":"2024-09-16T15:52:19.335362+0000","last_became_active":"2024-09-16T15:52:18.098767+0000","last_became_peered":"2024-09-16T15:52:18.098767+0000","last_unstale":"2024-09-16T15:52:19.335362+0000","last_undegraded":"2024-09-16T15:52:19.335362+0000","last_fullsized":"2024-09-16T15:52:19.335362+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:22:20.108931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.7","version":"117'2","reported_seq":17,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336898+0000","last_change":"2024-09-16T15:52:17.083225+0000","last_active":"2024-09-16T15:52:19.336898+0000","last_peered":"2024-09-16T15:52:19.336898+0000","last_clean":"2024-09-16T15:52:19.336898+0000","last_became_active":"2024-09-16T15:52:17.082946+0000","last_became_peered":"2024-09-16T15:52:17.082946+0000","last_unstale":"2024-09-16T15:52:19.336898+0000","last_undegraded":"2024-09-16T15:52:19.336898+0000","last_fullsized":"2024-09-16T15:52:19.336898+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:53:28.145727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.097472+0000","last_change":"2024-09-16T15:52:18.096158+0000","last_active":"2024-09-16T15:52:19.097472+0000","last_peered":"2024-09-16T15:52:19.097472+0000","last_clean":"2024-09-16T15:52:19.097472+0000","last_became_active":"2024-09-16T15:52:18.095854+0000","last_became_peered":"2024-09-16T15:52:18.095854+0000","last_unstale":"2024-09-16T15:52:19.097472+0000","last_undegraded":"2024-09-16T15:52:19.097472+0000","last_fullsized":"2024-09-16T15:52:19.097472+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:19:03.220760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"118'1","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098686+0000","last_change":"2024-09-16T15:52:17.091331+0000","last_active":"2024-09-16T15:52:19.098686+0000","last_peered":"2024-09-16T15:52:19.098686+0000","last_clean":"2024-09-16T15:52:19.098686+0000","last_became_active":"2024-09-16T15:52:17.091071+0000","last_became_peered":"2024-09-16T15:52:17.091071+0000","last_unstale":"2024-09-16T15:52:19.098686+0000","last_undegraded":"2024-09-16T15:52:19.098686+0000","last_fullsized":"2024-09-16T15:52:19.098686+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:18:38.561424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-16T15:52:17.077848+0000","last_change":"2024-09-16T15:52:17.077848+0000","last_active":"2024-09-16T15:52:17.058582+0000","last_peered":"2024-09-16T15:52:17.058582+0000","last_clean":"2024-09-16T15:52:17.058582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:52:17.077848+0000","last_undegraded":"2024-09-16T15:52:17.077848+0000","last_fullsized":"2024-09-16T15:52:17.077848+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336336+0000","last_change":"2024-09-16T15:52:17.080632+0000","last_active":"2024-09-16T15:52:19.336336+0000","last_peered":"2024-09-16T15:52:19.336336+0000","last_clean":"2024-09-16T15:52:19.336336+0000","last_became_active":"2024-09-16T15:52:17.079770+0000","last_became_peered":"2024-09-16T15:52:17.079770+0000","last_unstale":"2024-09-16T15:52:19.336336+0000","last_undegraded":"2024-09-16T15:52:19.336336+0000","last_fullsized":"2024-09-16T15:52:19.336336+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:18:23.166654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335328+0000","last_change":"2024-09-16T15:52:18.099378+0000","last_active":"2024-09-16T15:52:19.335328+0000","last_peered":"2024-09-16T15:52:19.335328+0000","last_clean":"2024-09-16T15:52:19.335328+0000","last_became_active":"2024-09-16T15:52:18.096154+0000","last_became_peered":"2024-09-16T15:52:18.096154+0000","last_unstale":"2024-09-16T15:52:19.335328+0000","last_undegraded":"2024-09-16T15:52:19.335328+0000","last_fullsized":"2024-09-16T15:52:19.335328+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:42:01.895815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336463+0000","last_change":"2024-09-16T15:52:17.091377+0000","last_active":"2024-09-16T15:52:19.336463+0000","last_peered":"2024-09-16T15:52:19.336463+0000","last_clean":"2024-09-16T15:52:19.336463+0000","last_became_active":"2024-09-16T15:52:17.090957+0000","last_became_peered":"2024-09-16T15:52:17.090957+0000","last_unstale":"2024-09-16T15:52:19.336463+0000","last_undegraded":"2024-09-16T15:52:19.336463+0000","last_fullsized":"2024-09-16T15:52:19.336463+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:21:10.990720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.095396+0000","last_change":"2024-09-16T15:52:18.107499+0000","last_active":"2024-09-16T15:52:19.095396+0000","last_peered":"2024-09-16T15:52:19.095396+0000","last_clean":"2024-09-16T15:52:19.095396+0000","last_became_active":"2024-09-16T15:52:18.105395+0000","last_became_peered":"2024-09-16T15:52:18.105395+0000","last_unstale":"2024-09-16T15:52:19.095396+0000","last_undegraded":"2024-09-16T15:52:19.095396+0000","last_fullsized":"2024-09-16T15:52:19.095396+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:15:42.167943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098838+0000","last_change":"2024-09-16T15:52:17.088515+0000","last_active":"2024-09-16T15:52:19.098838+0000","last_peered":"2024-09-16T15:52:19.098838+0000","last_clean":"2024-09-16T15:52:19.098838+0000","last_became_active":"2024-09-16T15:52:17.088093+0000","last_became_peered":"2024-09-16T15:52:17.088093+0000","last_unstale":"2024-09-16T15:52:19.098838+0000","last_undegraded":"2024-09-16T15:52:19.098838+0000","last_fullsized":"2024-09-16T15:52:19.098838+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:41:51.353892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335174+0000","last_change":"2024-09-16T15:52:18.104181+0000","last_active":"2024-09-16T15:52:19.335174+0000","last_peered":"2024-09-16T15:52:19.335174+0000","last_clean":"2024-09-16T15:52:19.335174+0000","last_became_active":"2024-09-16T15:52:18.096537+0000","last_became_peered":"2024-09-16T15:52:18.096537+0000","last_unstale":"2024-09-16T15:52:19.335174+0000","last_undegraded":"2024-09-16T15:52:19.335174+0000","last_fullsized":"2024-09-16T15:52:19.335174+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:53:28.863711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.334882+0000","last_change":"2024-09-16T15:52:17.087423+0000","last_active":"2024-09-16T15:52:19.334882+0000","last_peered":"2024-09-16T15:52:19.334882+0000","last_clean":"2024-09-16T15:52:19.334882+0000","last_became_active":"2024-09-16T15:52:17.087145+0000","last_became_peered":"2024-09-16T15:52:17.087145+0000","last_unstale":"2024-09-16T15:52:19.334882+0000","last_undegraded":"2024-09-16T15:52:19.334882+0000","last_fullsized":"2024-09-16T15:52:19.334882+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:09:02.776672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335017+0000","last_change":"2024-09-16T15:52:18.095897+0000","last_active":"2024-09-16T15:52:19.335017+0000","last_peered":"2024-09-16T15:52:19.335017+0000","last_clean":"2024-09-16T15:52:19.335017+0000","last_became_active":"2024-09-16T15:52:18.095588+0000","last_became_peered":"2024-09-16T15:52:18.095588+0000","last_unstale":"2024-09-16T15:52:19.335017+0000","last_undegraded":"2024-09-16T15:52:19.335017+0000","last_fullsized":"2024-09-16T15:52:19.335017+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:28:29.259290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-16T15:52:17.089987+0000","last_change":"2024-09-16T15:52:17.085524+0000","last_active":"2024-09-16T15:52:17.089987+0000","last_peered":"2024-09-16T15:52:17.089987+0000","last_clean":"2024-09-16T15:52:17.089987+0000","last_became_active":"2024-09-16T15:52:17.084921+0000","last_became_peered":"2024-09-16T15:52:17.084921+0000","last_unstale":"2024-09-16T15:52:17.089987+0000","last_undegraded":"2024-09-16T15:52:17.089987+0000","last_fullsized":"2024-09-16T15:52:17.089987+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:11:21.294316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.096777+0000","last_change":"2024-09-16T15:52:18.106049+0000","last_active":"2024-09-16T15:52:19.096777+0000","last_peered":"2024-09-16T15:52:19.096777+0000","last_clean":"2024-09-16T15:52:19.096777+0000","last_became_active":"2024-09-16T15:52:18.105352+0000","last_became_peered":"2024-09-16T15:52:18.105352+0000","last_unstale":"2024-09-16T15:52:19.096777+0000","last_undegraded":"2024-09-16T15:52:19.096777+0000","last_fullsized":"2024-09-16T15:52:19.096777+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:29:18.998165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.096570+0000","last_change":"2024-09-16T15:52:17.079490+0000","last_active":"2024-09-16T15:52:19.096570+0000","last_peered":"2024-09-16T15:52:19.096570+0000","last_clean":"2024-09-16T15:52:19.096570+0000","last_became_active":"2024-09-16T15:52:17.079148+0000","last_became_peered":"2024-09-16T15:52:17.079148+0000","last_unstale":"2024-09-16T15:52:19.096570+0000","last_undegraded":"2024-09-16T15:52:19.096570+0000","last_fullsized":"2024-09-16T15:52:19.096570+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:17:11.750051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336381+0000","last_change":"2024-09-16T15:52:18.087311+0000","last_active":"2024-09-16T15:52:19.336381+0000","last_peered":"2024-09-16T15:52:19.336381+0000","last_clean":"2024-09-16T15:52:19.336381+0000","last_became_active":"2024-09-16T15:52:18.086640+0000","last_became_peered":"2024-09-16T15:52:18.086640+0000","last_unstale":"2024-09-16T15:52:19.336381+0000","last_undegraded":"2024-09-16T15:52:19.336381+0000","last_fullsized":"2024-09-16T15:52:19.336381+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:12:10.637529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.324319+0000","last_change":"2024-09-16T15:52:17.072908+0000","last_active":"2024-09-16T15:52:19.324319+0000","last_peered":"2024-09-16T15:52:19.324319+0000","last_clean":"2024-09-16T15:52:19.324319+0000","last_became_active":"2024-09-16T15:52:17.072645+0000","last_became_peered":"2024-09-16T15:52:17.072645+0000","last_unstale":"2024-09-16T15:52:19.324319+0000","last_undegraded":"2024-09-16T15:52:19.324319+0000","last_fullsized":"2024-09-16T15:52:19.324319+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:26:34.557572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":293,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.095214+0000","last_change":"2024-09-16T15:39:12.698813+0000","last_active":"2024-09-16T15:52:19.095214+0000","last_peered":"2024-09-16T15:52:19.095214+0000","last_clean":"2024-09-16T15:52:19.095214+0000","last_became_active":"2024-09-16T15:39:12.697695+0000","last_became_peered":"2024-09-16T15:39:12.697695+0000","last_unstale":"2024-09-16T15:52:19.095214+0000","last_undegraded":"2024-09-16T15:52:19.095214+0000","last_fullsized":"2024-09-16T15:52:19.095214+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_clean_scrub_stamp":"2024-09-16T15:39:11.684967+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:02.061081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":30,"ondisk_log_size":30,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509602,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":136652,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94235188,"statfs":{"total":96636764160,"available":96496832512,"internally_reserved":0,"allocated":1433600,"data_stored":627651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":138475976},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509602,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":135864,"kb_used_data":2012,"kb_used_omap":1,"kb_used_meta":133822,"kb_avail":94235976,"statfs":{"total":96636764160,"available":96497639424,"internally_reserved":0,"allocated":2060288,"data_stored":1231354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":137034183},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509601,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149172,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222668,"statfs":{"total":96636764160,"available":96484012032,"internally_reserved":0,"allocated":1474560,"data_stored":642596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509601,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131112,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240728,"statfs":{"total":96636764160,"available":96502505472,"internally_reserved":0,"allocated":1458176,"data_stored":640894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":132774345},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509601,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126968,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244872,"statfs":{"total":96636764160,"available":96506748928,"internally_reserved":0,"allocated":1470464,"data_stored":641532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":128514503},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509602,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127536,"kb_used_data":2004,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244304,"statfs":{"total":96636764160,"available":96506167296,"internally_reserved":0,"allocated":2052096,"data_stored":1231262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":128514503},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509600,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145012,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226828,"statfs":{"total":96636764160,"available":96488271872,"internally_reserved":0,"allocated":1466368,"data_stored":641546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":146995655},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542307,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126972,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244868,"statfs":{"total":96636764160,"available":96506744832,"internally_reserved":0,"allocated":1478656,"data_stored":642630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":128514508},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:52:22.345 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:52:22.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.536+0000 7fbe57200640 1 -- 172.21.15.151:0/1087785271 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbe580725d0 msgr2=0x7fbe580b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:22.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.536+0000 7fbe57200640 1 --2- 172.21.15.151:0/1087785271 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbe580725d0 0x7fbe580b28f0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fbe58074a80 tx=0x7fbe4402a560 comp rx=0 tx=0).stop 2024-09-16T15:52:22.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.536+0000 7fbe57200640 1 -- 172.21.15.151:0/1087785271 shutdown_connections 2024-09-16T15:52:22.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.536+0000 7fbe57200640 1 --2- 172.21.15.151:0/1087785271 >> v2:172.21.15.167:3300/0 conn(0x7fbe580d0540 0x7fbe5806f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:22.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.536+0000 7fbe57200640 1 --2- 172.21.15.151:0/1087785271 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbe580725d0 0x7fbe580b28f0 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:22.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.536+0000 7fbe57200640 1 -- 172.21.15.151:0/1087785271 >> 172.21.15.151:0/1087785271 conn(0x7fbe580cfa20 msgr2=0x7fbe580cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:22.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.536+0000 7fbe57200640 1 -- 172.21.15.151:0/1087785271 shutdown_connections 2024-09-16T15:52:22.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.536+0000 7fbe57200640 1 -- 172.21.15.151:0/1087785271 wait complete. 2024-09-16T15:52:22.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.538+0000 7fbe57200640 1 Processor -- start 2024-09-16T15:52:22.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.538+0000 7fbe57200640 1 -- start start 2024-09-16T15:52:22.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.539+0000 7fbe57200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbe580d0540 0x7fbe58154620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:22.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.539+0000 7fbe57200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe5815bc30 0x7fbe58155180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:22.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.539+0000 7fbe57200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbe581f3190 0x7fbe581501c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:22.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.539+0000 7fbe57200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbe58074660 con 0x7fbe580d0540 2024-09-16T15:52:22.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.539+0000 7fbe57200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbe5806f860 con 0x7fbe5815bc30 2024-09-16T15:52:22.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.539+0000 7fbe57200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fbe58155860 con 0x7fbe581f3190 2024-09-16T15:52:22.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.539+0000 7fbe55e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbe580d0540 0x7fbe58154620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:22.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.539+0000 7fbe55e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbe580d0540 0x7fbe58154620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:49982/0 (socket says 172.21.15.151:49982) 2024-09-16T15:52:22.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.539+0000 7fbe55e00640 1 -- 172.21.15.151:0/4093427183 learned_addr learned my addr 172.21.15.151:0/4093427183 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:52:22.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.539+0000 7fbe55400640 1 --2- 172.21.15.151:0/4093427183 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe5815bc30 0x7fbe58155180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:22.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.539+0000 7fbe56800640 1 --2- 172.21.15.151:0/4093427183 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbe581f3190 0x7fbe581501c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:22.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.540+0000 7fbe55e00640 1 -- 172.21.15.151:0/4093427183 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbe581f3190 msgr2=0x7fbe581501c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:22.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.540+0000 7fbe55e00640 1 --2- 172.21.15.151:0/4093427183 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbe581f3190 0x7fbe581501c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:22.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.540+0000 7fbe55e00640 1 -- 172.21.15.151:0/4093427183 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe5815bc30 msgr2=0x7fbe58155180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:22.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.540+0000 7fbe55e00640 1 --2- 172.21.15.151:0/4093427183 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe5815bc30 0x7fbe58155180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:22.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.540+0000 7fbe55e00640 1 -- 172.21.15.151:0/4093427183 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe58153860 con 0x7fbe580d0540 2024-09-16T15:52:22.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.540+0000 7fbe56800640 1 --2- 172.21.15.151:0/4093427183 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbe581f3190 0x7fbe581501c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:52:22.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.540+0000 7fbe55e00640 1 --2- 172.21.15.151:0/4093427183 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbe580d0540 0x7fbe58154620 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fbe4800b420 tx=0x7fbe4800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:22.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.540+0000 7fbe4e000640 1 -- 172.21.15.151:0/4093427183 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe48011830 con 0x7fbe580d0540 2024-09-16T15:52:22.546 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.540+0000 7fbe4e000640 1 -- 172.21.15.151:0/4093427183 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbe48016070 con 0x7fbe580d0540 2024-09-16T15:52:22.546 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.540+0000 7fbe4e000640 1 -- 172.21.15.151:0/4093427183 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe48011d60 con 0x7fbe580d0540 2024-09-16T15:52:22.546 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.540+0000 7fbe57200640 1 -- 172.21.15.151:0/4093427183 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe58153b40 con 0x7fbe580d0540 2024-09-16T15:52:22.546 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.540+0000 7fbe57200640 1 -- 172.21.15.151:0/4093427183 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe58154100 con 0x7fbe580d0540 2024-09-16T15:52:22.546 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.541+0000 7fbe57200640 1 -- 172.21.15.151:0/4093427183 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe5806fdf0 con 0x7fbe580d0540 2024-09-16T15:52:22.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.542+0000 7fbe4e000640 1 -- 172.21.15.151:0/4093427183 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 41) ==== 103684+0+0 (secure 0 0 0) 0x7fbe4801b460 con 0x7fbe580d0540 2024-09-16T15:52:22.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.543+0000 7fbe4e000640 1 --2- 172.21.15.151:0/4093427183 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbe3807a2c0 0x7fbe3807c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:22.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.543+0000 7fbe4e000640 1 -- 172.21.15.151:0/4093427183 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5793+0+0 (secure 0 0 0) 0x7fbe48099290 con 0x7fbe580d0540 2024-09-16T15:52:22.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.544+0000 7fbe55400640 1 --2- 172.21.15.151:0/4093427183 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbe3807a2c0 0x7fbe3807c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:22.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.545+0000 7fbe55400640 1 --2- 172.21.15.151:0/4093427183 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbe3807a2c0 0x7fbe3807c780 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fbe44033000 tx=0x7fbe44004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:22.549 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.547+0000 7fbe4e000640 1 -- 172.21.15.151:0/4093427183 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fbe480a3020 con 0x7fbe580d0540 2024-09-16T15:52:22.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.731+0000 7fbe57200640 1 -- 172.21.15.151:0/4093427183 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbe581542d0 con 0x7fbe3807a2c0 2024-09-16T15:52:22.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.735+0000 7fbe4e000640 1 -- 172.21.15.151:0/4093427183 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251751 (secure 0 0 0) 0x7fbe581542d0 con 0x7fbe3807a2c0 2024-09-16T15:52:22.741 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:52:22.743 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-09-16T15:52:22.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.739+0000 7fbe57200640 1 -- 172.21.15.151:0/4093427183 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbe3807a2c0 msgr2=0x7fbe3807c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:22.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.739+0000 7fbe57200640 1 --2- 172.21.15.151:0/4093427183 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbe3807a2c0 0x7fbe3807c780 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fbe44033000 tx=0x7fbe44004730 comp rx=0 tx=0).stop 2024-09-16T15:52:22.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.739+0000 7fbe57200640 1 -- 172.21.15.151:0/4093427183 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbe580d0540 msgr2=0x7fbe58154620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:22.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.739+0000 7fbe57200640 1 --2- 172.21.15.151:0/4093427183 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbe580d0540 0x7fbe58154620 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fbe4800b420 tx=0x7fbe4800b8e0 comp rx=0 tx=0).stop 2024-09-16T15:52:22.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.739+0000 7fbe57200640 1 -- 172.21.15.151:0/4093427183 shutdown_connections 2024-09-16T15:52:22.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.739+0000 7fbe57200640 1 --2- 172.21.15.151:0/4093427183 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbe3807a2c0 0x7fbe3807c780 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:22.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.739+0000 7fbe57200640 1 --2- 172.21.15.151:0/4093427183 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe5815bc30 0x7fbe58155180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:22.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.739+0000 7fbe57200640 1 --2- 172.21.15.151:0/4093427183 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbe581f3190 0x7fbe581501c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:22.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.739+0000 7fbe57200640 1 --2- 172.21.15.151:0/4093427183 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbe580d0540 0x7fbe58154620 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:22.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.739+0000 7fbe57200640 1 -- 172.21.15.151:0/4093427183 >> 172.21.15.151:0/4093427183 conn(0x7fbe580cfa20 msgr2=0x7fbe581572f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:22.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.739+0000 7fbe57200640 1 -- 172.21.15.151:0/4093427183 shutdown_connections 2024-09-16T15:52:22.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.739+0000 7fbe57200640 1 -- 172.21.15.151:0/4093427183 wait complete. 2024-09-16T15:52:22.758 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":507,"stamp":"2024-09-16T15:52:21.901931+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":134,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":102,"ondisk_log_size":102,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":165,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1079288,"kb_used_data":12592,"kb_used_omap":12,"kb_used_meta":1066483,"kb_avail":753895432,"statfs":{"total":773094113280,"available":771988922368,"internally_reserved":0,"allocated":12894208,"data_stored":6299465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12738,"internal_metadata":1092079166},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.837592"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-16T15:52:17.079031+0000","last_change":"2024-09-16T15:52:17.079031+0000","last_active":"2024-09-16T15:52:17.058582+0000","last_peered":"2024-09-16T15:52:17.058582+0000","last_clean":"2024-09-16T15:52:17.058582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:52:17.079031+0000","last_undegraded":"2024-09-16T15:52:17.079031+0000","last_fullsized":"2024-09-16T15:52:17.079031+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.324514+0000","last_change":"2024-09-16T15:52:18.095601+0000","last_active":"2024-09-16T15:52:19.324514+0000","last_peered":"2024-09-16T15:52:19.324514+0000","last_clean":"2024-09-16T15:52:19.324514+0000","last_became_active":"2024-09-16T15:52:18.095305+0000","last_became_peered":"2024-09-16T15:52:18.095305+0000","last_unstale":"2024-09-16T15:52:19.324514+0000","last_undegraded":"2024-09-16T15:52:19.324514+0000","last_fullsized":"2024-09-16T15:52:19.324514+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:46:48.807361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.095580+0000","last_change":"2024-09-16T15:52:18.108749+0000","last_active":"2024-09-16T15:52:19.095580+0000","last_peered":"2024-09-16T15:52:19.095580+0000","last_clean":"2024-09-16T15:52:19.095580+0000","last_became_active":"2024-09-16T15:52:18.104818+0000","last_became_peered":"2024-09-16T15:52:18.104818+0000","last_unstale":"2024-09-16T15:52:19.095580+0000","last_undegraded":"2024-09-16T15:52:19.095580+0000","last_fullsized":"2024-09-16T15:52:19.095580+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:32:30.003302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336245+0000","last_change":"2024-09-16T15:52:18.090782+0000","last_active":"2024-09-16T15:52:19.336245+0000","last_peered":"2024-09-16T15:52:19.336245+0000","last_clean":"2024-09-16T15:52:19.336245+0000","last_became_active":"2024-09-16T15:52:18.090432+0000","last_became_peered":"2024-09-16T15:52:18.090432+0000","last_unstale":"2024-09-16T15:52:19.336245+0000","last_undegraded":"2024-09-16T15:52:19.336245+0000","last_fullsized":"2024-09-16T15:52:19.336245+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:48:32.423354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335861+0000","last_change":"2024-09-16T15:52:18.109873+0000","last_active":"2024-09-16T15:52:19.335861+0000","last_peered":"2024-09-16T15:52:19.335861+0000","last_clean":"2024-09-16T15:52:19.335861+0000","last_became_active":"2024-09-16T15:52:18.101073+0000","last_became_peered":"2024-09-16T15:52:18.101073+0000","last_unstale":"2024-09-16T15:52:19.335861+0000","last_undegraded":"2024-09-16T15:52:19.335861+0000","last_fullsized":"2024-09-16T15:52:19.335861+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:56:57.514683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.097692+0000","last_change":"2024-09-16T15:52:18.096166+0000","last_active":"2024-09-16T15:52:19.097692+0000","last_peered":"2024-09-16T15:52:19.097692+0000","last_clean":"2024-09-16T15:52:19.097692+0000","last_became_active":"2024-09-16T15:52:18.095913+0000","last_became_peered":"2024-09-16T15:52:18.095913+0000","last_unstale":"2024-09-16T15:52:19.097692+0000","last_undegraded":"2024-09-16T15:52:19.097692+0000","last_fullsized":"2024-09-16T15:52:19.097692+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:51:43.908577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098468+0000","last_change":"2024-09-16T15:52:18.111297+0000","last_active":"2024-09-16T15:52:19.098468+0000","last_peered":"2024-09-16T15:52:19.098468+0000","last_clean":"2024-09-16T15:52:19.098468+0000","last_became_active":"2024-09-16T15:52:18.111039+0000","last_became_peered":"2024-09-16T15:52:18.111039+0000","last_unstale":"2024-09-16T15:52:19.098468+0000","last_undegraded":"2024-09-16T15:52:19.098468+0000","last_fullsized":"2024-09-16T15:52:19.098468+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:08:05.616283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098459+0000","last_change":"2024-09-16T15:52:18.094809+0000","last_active":"2024-09-16T15:52:19.098459+0000","last_peered":"2024-09-16T15:52:19.098459+0000","last_clean":"2024-09-16T15:52:19.098459+0000","last_became_active":"2024-09-16T15:52:18.094538+0000","last_became_peered":"2024-09-16T15:52:18.094538+0000","last_unstale":"2024-09-16T15:52:19.098459+0000","last_undegraded":"2024-09-16T15:52:19.098459+0000","last_fullsized":"2024-09-16T15:52:19.098459+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:02:37.475979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-16T15:52:17.076806+0000","last_change":"2024-09-16T15:52:17.076806+0000","last_active":"2024-09-16T15:52:17.058582+0000","last_peered":"2024-09-16T15:52:17.058582+0000","last_clean":"2024-09-16T15:52:17.058582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:52:17.076806+0000","last_undegraded":"2024-09-16T15:52:17.076806+0000","last_fullsized":"2024-09-16T15:52:17.076806+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336431+0000","last_change":"2024-09-16T15:52:18.101787+0000","last_active":"2024-09-16T15:52:19.336431+0000","last_peered":"2024-09-16T15:52:19.336431+0000","last_clean":"2024-09-16T15:52:19.336431+0000","last_became_active":"2024-09-16T15:52:18.100961+0000","last_became_peered":"2024-09-16T15:52:18.100961+0000","last_unstale":"2024-09-16T15:52:19.336431+0000","last_undegraded":"2024-09-16T15:52:19.336431+0000","last_fullsized":"2024-09-16T15:52:19.336431+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:31:15.594998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098664+0000","last_change":"2024-09-16T15:52:18.094110+0000","last_active":"2024-09-16T15:52:19.098664+0000","last_peered":"2024-09-16T15:52:19.098664+0000","last_clean":"2024-09-16T15:52:19.098664+0000","last_became_active":"2024-09-16T15:52:18.093858+0000","last_became_peered":"2024-09-16T15:52:18.093858+0000","last_unstale":"2024-09-16T15:52:19.098664+0000","last_undegraded":"2024-09-16T15:52:19.098664+0000","last_fullsized":"2024-09-16T15:52:19.098664+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:32:24.437229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335879+0000","last_change":"2024-09-16T15:52:18.102228+0000","last_active":"2024-09-16T15:52:19.335879+0000","last_peered":"2024-09-16T15:52:19.335879+0000","last_clean":"2024-09-16T15:52:19.335879+0000","last_became_active":"2024-09-16T15:52:18.101917+0000","last_became_peered":"2024-09-16T15:52:18.101917+0000","last_unstale":"2024-09-16T15:52:19.335879+0000","last_undegraded":"2024-09-16T15:52:19.335879+0000","last_fullsized":"2024-09-16T15:52:19.335879+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:49:20.323885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.324528+0000","last_change":"2024-09-16T15:52:18.093067+0000","last_active":"2024-09-16T15:52:19.324528+0000","last_peered":"2024-09-16T15:52:19.324528+0000","last_clean":"2024-09-16T15:52:19.324528+0000","last_became_active":"2024-09-16T15:52:18.092805+0000","last_became_peered":"2024-09-16T15:52:18.092805+0000","last_unstale":"2024-09-16T15:52:19.324528+0000","last_undegraded":"2024-09-16T15:52:19.324528+0000","last_fullsized":"2024-09-16T15:52:19.324528+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:06:51.783453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.334985+0000","last_change":"2024-09-16T15:52:18.098148+0000","last_active":"2024-09-16T15:52:19.334985+0000","last_peered":"2024-09-16T15:52:19.334985+0000","last_clean":"2024-09-16T15:52:19.334985+0000","last_became_active":"2024-09-16T15:52:18.096388+0000","last_became_peered":"2024-09-16T15:52:18.096388+0000","last_unstale":"2024-09-16T15:52:19.334985+0000","last_undegraded":"2024-09-16T15:52:19.334985+0000","last_fullsized":"2024-09-16T15:52:19.334985+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:59:34.319160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-16T15:52:17.076793+0000","last_change":"2024-09-16T15:52:17.076793+0000","last_active":"2024-09-16T15:52:17.058582+0000","last_peered":"2024-09-16T15:52:17.058582+0000","last_clean":"2024-09-16T15:52:17.058582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:52:17.076793+0000","last_undegraded":"2024-09-16T15:52:17.076793+0000","last_fullsized":"2024-09-16T15:52:17.076793+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335328+0000","last_change":"2024-09-16T15:52:18.096798+0000","last_active":"2024-09-16T15:52:19.335328+0000","last_peered":"2024-09-16T15:52:19.335328+0000","last_clean":"2024-09-16T15:52:19.335328+0000","last_became_active":"2024-09-16T15:52:18.096396+0000","last_became_peered":"2024-09-16T15:52:18.096396+0000","last_unstale":"2024-09-16T15:52:19.335328+0000","last_undegraded":"2024-09-16T15:52:19.335328+0000","last_fullsized":"2024-09-16T15:52:19.335328+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:17:25.344931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.095682+0000","last_change":"2024-09-16T15:52:18.113939+0000","last_active":"2024-09-16T15:52:19.095682+0000","last_peered":"2024-09-16T15:52:19.095682+0000","last_clean":"2024-09-16T15:52:19.095682+0000","last_became_active":"2024-09-16T15:52:18.113634+0000","last_became_peered":"2024-09-16T15:52:18.113634+0000","last_unstale":"2024-09-16T15:52:19.095682+0000","last_undegraded":"2024-09-16T15:52:19.095682+0000","last_fullsized":"2024-09-16T15:52:19.095682+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:04:19.441890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098743+0000","last_change":"2024-09-16T15:52:18.106010+0000","last_active":"2024-09-16T15:52:19.098743+0000","last_peered":"2024-09-16T15:52:19.098743+0000","last_clean":"2024-09-16T15:52:19.098743+0000","last_became_active":"2024-09-16T15:52:18.105722+0000","last_became_peered":"2024-09-16T15:52:18.105722+0000","last_unstale":"2024-09-16T15:52:19.098743+0000","last_undegraded":"2024-09-16T15:52:19.098743+0000","last_fullsized":"2024-09-16T15:52:19.098743+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:29:30.985879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.095786+0000","last_change":"2024-09-16T15:52:18.108828+0000","last_active":"2024-09-16T15:52:19.095786+0000","last_peered":"2024-09-16T15:52:19.095786+0000","last_clean":"2024-09-16T15:52:19.095786+0000","last_became_active":"2024-09-16T15:52:18.104728+0000","last_became_peered":"2024-09-16T15:52:18.104728+0000","last_unstale":"2024-09-16T15:52:19.095786+0000","last_undegraded":"2024-09-16T15:52:19.095786+0000","last_fullsized":"2024-09-16T15:52:19.095786+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:08:15.649535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.095454+0000","last_change":"2024-09-16T15:52:18.104490+0000","last_active":"2024-09-16T15:52:19.095454+0000","last_peered":"2024-09-16T15:52:19.095454+0000","last_clean":"2024-09-16T15:52:19.095454+0000","last_became_active":"2024-09-16T15:52:18.104176+0000","last_became_peered":"2024-09-16T15:52:18.104176+0000","last_unstale":"2024-09-16T15:52:19.095454+0000","last_undegraded":"2024-09-16T15:52:19.095454+0000","last_fullsized":"2024-09-16T15:52:19.095454+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:38:00.758145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098816+0000","last_change":"2024-09-16T15:52:18.100178+0000","last_active":"2024-09-16T15:52:19.098816+0000","last_peered":"2024-09-16T15:52:19.098816+0000","last_clean":"2024-09-16T15:52:19.098816+0000","last_became_active":"2024-09-16T15:52:18.099483+0000","last_became_peered":"2024-09-16T15:52:18.099483+0000","last_unstale":"2024-09-16T15:52:19.098816+0000","last_undegraded":"2024-09-16T15:52:19.098816+0000","last_fullsized":"2024-09-16T15:52:19.098816+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:16:33.962636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.097266+0000","last_change":"2024-09-16T15:52:18.095116+0000","last_active":"2024-09-16T15:52:19.097266+0000","last_peered":"2024-09-16T15:52:19.097266+0000","last_clean":"2024-09-16T15:52:19.097266+0000","last_became_active":"2024-09-16T15:52:18.090617+0000","last_became_peered":"2024-09-16T15:52:18.090617+0000","last_unstale":"2024-09-16T15:52:19.097266+0000","last_undegraded":"2024-09-16T15:52:19.097266+0000","last_fullsized":"2024-09-16T15:52:19.097266+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:23:03.363317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335071+0000","last_change":"2024-09-16T15:52:18.102959+0000","last_active":"2024-09-16T15:52:19.335071+0000","last_peered":"2024-09-16T15:52:19.335071+0000","last_clean":"2024-09-16T15:52:19.335071+0000","last_became_active":"2024-09-16T15:52:18.102684+0000","last_became_peered":"2024-09-16T15:52:18.102684+0000","last_unstale":"2024-09-16T15:52:19.335071+0000","last_undegraded":"2024-09-16T15:52:19.335071+0000","last_fullsized":"2024-09-16T15:52:19.335071+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:26:35.565582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336817+0000","last_change":"2024-09-16T15:52:18.102292+0000","last_active":"2024-09-16T15:52:19.336817+0000","last_peered":"2024-09-16T15:52:19.336817+0000","last_clean":"2024-09-16T15:52:19.336817+0000","last_became_active":"2024-09-16T15:52:18.102017+0000","last_became_peered":"2024-09-16T15:52:18.102017+0000","last_unstale":"2024-09-16T15:52:19.336817+0000","last_undegraded":"2024-09-16T15:52:19.336817+0000","last_fullsized":"2024-09-16T15:52:19.336817+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:34:23.254431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098723+0000","last_change":"2024-09-16T15:52:18.089811+0000","last_active":"2024-09-16T15:52:19.098723+0000","last_peered":"2024-09-16T15:52:19.098723+0000","last_clean":"2024-09-16T15:52:19.098723+0000","last_became_active":"2024-09-16T15:52:18.089302+0000","last_became_peered":"2024-09-16T15:52:18.089302+0000","last_unstale":"2024-09-16T15:52:19.098723+0000","last_undegraded":"2024-09-16T15:52:19.098723+0000","last_fullsized":"2024-09-16T15:52:19.098723+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:44:21.343778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.097694+0000","last_change":"2024-09-16T15:52:18.107014+0000","last_active":"2024-09-16T15:52:19.097694+0000","last_peered":"2024-09-16T15:52:19.097694+0000","last_clean":"2024-09-16T15:52:19.097694+0000","last_became_active":"2024-09-16T15:52:18.106706+0000","last_became_peered":"2024-09-16T15:52:18.106706+0000","last_unstale":"2024-09-16T15:52:19.097694+0000","last_undegraded":"2024-09-16T15:52:19.097694+0000","last_fullsized":"2024-09-16T15:52:19.097694+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:41:14.841086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336117+0000","last_change":"2024-09-16T15:52:18.102898+0000","last_active":"2024-09-16T15:52:19.336117+0000","last_peered":"2024-09-16T15:52:19.336117+0000","last_clean":"2024-09-16T15:52:19.336117+0000","last_became_active":"2024-09-16T15:52:18.102229+0000","last_became_peered":"2024-09-16T15:52:18.102229+0000","last_unstale":"2024-09-16T15:52:19.336117+0000","last_undegraded":"2024-09-16T15:52:19.336117+0000","last_fullsized":"2024-09-16T15:52:19.336117+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:21:53.345211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.097217+0000","last_change":"2024-09-16T15:52:18.099345+0000","last_active":"2024-09-16T15:52:19.097217+0000","last_peered":"2024-09-16T15:52:19.097217+0000","last_clean":"2024-09-16T15:52:19.097217+0000","last_became_active":"2024-09-16T15:52:18.098646+0000","last_became_peered":"2024-09-16T15:52:18.098646+0000","last_unstale":"2024-09-16T15:52:19.097217+0000","last_undegraded":"2024-09-16T15:52:19.097217+0000","last_fullsized":"2024-09-16T15:52:19.097217+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:28:37.287849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335126+0000","last_change":"2024-09-16T15:52:18.106525+0000","last_active":"2024-09-16T15:52:19.335126+0000","last_peered":"2024-09-16T15:52:19.335126+0000","last_clean":"2024-09-16T15:52:19.335126+0000","last_became_active":"2024-09-16T15:52:18.103778+0000","last_became_peered":"2024-09-16T15:52:18.103778+0000","last_unstale":"2024-09-16T15:52:19.335126+0000","last_undegraded":"2024-09-16T15:52:19.335126+0000","last_fullsized":"2024-09-16T15:52:19.335126+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:46:53.909612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098944+0000","last_change":"2024-09-16T15:52:18.087005+0000","last_active":"2024-09-16T15:52:19.098944+0000","last_peered":"2024-09-16T15:52:19.098944+0000","last_clean":"2024-09-16T15:52:19.098944+0000","last_became_active":"2024-09-16T15:52:18.086463+0000","last_became_peered":"2024-09-16T15:52:18.086463+0000","last_unstale":"2024-09-16T15:52:19.098944+0000","last_undegraded":"2024-09-16T15:52:19.098944+0000","last_fullsized":"2024-09-16T15:52:19.098944+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:23:31.782022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-16T15:52:17.076332+0000","last_change":"2024-09-16T15:52:17.076332+0000","last_active":"2024-09-16T15:52:17.058582+0000","last_peered":"2024-09-16T15:52:17.058582+0000","last_clean":"2024-09-16T15:52:17.058582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:52:17.076332+0000","last_undegraded":"2024-09-16T15:52:17.076332+0000","last_fullsized":"2024-09-16T15:52:17.076332+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335551+0000","last_change":"2024-09-16T15:52:18.106237+0000","last_active":"2024-09-16T15:52:19.335551+0000","last_peered":"2024-09-16T15:52:19.335551+0000","last_clean":"2024-09-16T15:52:19.335551+0000","last_became_active":"2024-09-16T15:52:18.103396+0000","last_became_peered":"2024-09-16T15:52:18.103396+0000","last_unstale":"2024-09-16T15:52:19.335551+0000","last_undegraded":"2024-09-16T15:52:19.335551+0000","last_fullsized":"2024-09-16T15:52:19.335551+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:35:46.682990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-16T15:52:17.081705+0000","last_change":"2024-09-16T15:52:17.081705+0000","last_active":"2024-09-16T15:52:17.058582+0000","last_peered":"2024-09-16T15:52:17.058582+0000","last_clean":"2024-09-16T15:52:17.058582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:52:17.081705+0000","last_undegraded":"2024-09-16T15:52:17.081705+0000","last_fullsized":"2024-09-16T15:52:17.081705+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"117'4","reported_seq":19,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335332+0000","last_change":"2024-09-16T15:52:17.086334+0000","last_active":"2024-09-16T15:52:19.335332+0000","last_peered":"2024-09-16T15:52:19.335332+0000","last_clean":"2024-09-16T15:52:19.335332+0000","last_became_active":"2024-09-16T15:52:17.086082+0000","last_became_peered":"2024-09-16T15:52:17.086082+0000","last_unstale":"2024-09-16T15:52:19.335332+0000","last_undegraded":"2024-09-16T15:52:19.335332+0000","last_fullsized":"2024-09-16T15:52:19.335332+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:25:34.552766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335430+0000","last_change":"2024-09-16T15:52:18.102484+0000","last_active":"2024-09-16T15:52:19.335430+0000","last_peered":"2024-09-16T15:52:19.335430+0000","last_clean":"2024-09-16T15:52:19.335430+0000","last_became_active":"2024-09-16T15:52:18.102159+0000","last_became_peered":"2024-09-16T15:52:18.102159+0000","last_unstale":"2024-09-16T15:52:19.335430+0000","last_undegraded":"2024-09-16T15:52:19.335430+0000","last_fullsized":"2024-09-16T15:52:19.335430+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:18:36.837288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1e","version":"117'1","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.324560+0000","last_change":"2024-09-16T15:52:17.082085+0000","last_active":"2024-09-16T15:52:19.324560+0000","last_peered":"2024-09-16T15:52:19.324560+0000","last_clean":"2024-09-16T15:52:19.324560+0000","last_became_active":"2024-09-16T15:52:17.081840+0000","last_became_peered":"2024-09-16T15:52:17.081840+0000","last_unstale":"2024-09-16T15:52:19.324560+0000","last_undegraded":"2024-09-16T15:52:19.324560+0000","last_fullsized":"2024-09-16T15:52:19.324560+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:46:52.901601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.097360+0000","last_change":"2024-09-16T15:52:18.104067+0000","last_active":"2024-09-16T15:52:19.097360+0000","last_peered":"2024-09-16T15:52:19.097360+0000","last_clean":"2024-09-16T15:52:19.097360+0000","last_became_active":"2024-09-16T15:52:18.103764+0000","last_became_peered":"2024-09-16T15:52:18.103764+0000","last_unstale":"2024-09-16T15:52:19.097360+0000","last_undegraded":"2024-09-16T15:52:19.097360+0000","last_fullsized":"2024-09-16T15:52:19.097360+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:51:28.451444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1d","version":"117'2","reported_seq":17,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336225+0000","last_change":"2024-09-16T15:52:17.083135+0000","last_active":"2024-09-16T15:52:19.336225+0000","last_peered":"2024-09-16T15:52:19.336225+0000","last_clean":"2024-09-16T15:52:19.336225+0000","last_became_active":"2024-09-16T15:52:17.082799+0000","last_became_peered":"2024-09-16T15:52:17.082799+0000","last_unstale":"2024-09-16T15:52:19.336225+0000","last_undegraded":"2024-09-16T15:52:19.336225+0000","last_fullsized":"2024-09-16T15:52:19.336225+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:43:05.707621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.095847+0000","last_change":"2024-09-16T15:52:18.109036+0000","last_active":"2024-09-16T15:52:19.095847+0000","last_peered":"2024-09-16T15:52:19.095847+0000","last_clean":"2024-09-16T15:52:19.095847+0000","last_became_active":"2024-09-16T15:52:18.106979+0000","last_became_peered":"2024-09-16T15:52:18.106979+0000","last_unstale":"2024-09-16T15:52:19.095847+0000","last_undegraded":"2024-09-16T15:52:19.095847+0000","last_fullsized":"2024-09-16T15:52:19.095847+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:38:25.952399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"117'2","reported_seq":17,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336324+0000","last_change":"2024-09-16T15:52:17.087728+0000","last_active":"2024-09-16T15:52:19.336324+0000","last_peered":"2024-09-16T15:52:19.336324+0000","last_clean":"2024-09-16T15:52:19.336324+0000","last_became_active":"2024-09-16T15:52:17.087063+0000","last_became_peered":"2024-09-16T15:52:17.087063+0000","last_unstale":"2024-09-16T15:52:19.336324+0000","last_undegraded":"2024-09-16T15:52:19.336324+0000","last_fullsized":"2024-09-16T15:52:19.336324+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:57:32.844856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336678+0000","last_change":"2024-09-16T15:52:18.111393+0000","last_active":"2024-09-16T15:52:19.336678+0000","last_peered":"2024-09-16T15:52:19.336678+0000","last_clean":"2024-09-16T15:52:19.336678+0000","last_became_active":"2024-09-16T15:52:18.111116+0000","last_became_peered":"2024-09-16T15:52:18.111116+0000","last_unstale":"2024-09-16T15:52:19.336678+0000","last_undegraded":"2024-09-16T15:52:19.336678+0000","last_fullsized":"2024-09-16T15:52:19.336678+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:59:34.319160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1b","version":"117'1","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.095273+0000","last_change":"2024-09-16T15:52:17.083411+0000","last_active":"2024-09-16T15:52:19.095273+0000","last_peered":"2024-09-16T15:52:19.095273+0000","last_clean":"2024-09-16T15:52:19.095273+0000","last_became_active":"2024-09-16T15:52:17.083098+0000","last_became_peered":"2024-09-16T15:52:17.083098+0000","last_unstale":"2024-09-16T15:52:19.095273+0000","last_undegraded":"2024-09-16T15:52:19.095273+0000","last_fullsized":"2024-09-16T15:52:19.095273+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:48:17.710440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098564+0000","last_change":"2024-09-16T15:52:18.099617+0000","last_active":"2024-09-16T15:52:19.098564+0000","last_peered":"2024-09-16T15:52:19.098564+0000","last_clean":"2024-09-16T15:52:19.098564+0000","last_became_active":"2024-09-16T15:52:18.099149+0000","last_became_peered":"2024-09-16T15:52:18.099149+0000","last_unstale":"2024-09-16T15:52:19.098564+0000","last_undegraded":"2024-09-16T15:52:19.098564+0000","last_fullsized":"2024-09-16T15:52:19.098564+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:41:19.687707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-16T15:52:17.092540+0000","last_change":"2024-09-16T15:52:17.086053+0000","last_active":"2024-09-16T15:52:17.092540+0000","last_peered":"2024-09-16T15:52:17.092540+0000","last_clean":"2024-09-16T15:52:17.092540+0000","last_became_active":"2024-09-16T15:52:17.084186+0000","last_became_peered":"2024-09-16T15:52:17.084186+0000","last_unstale":"2024-09-16T15:52:17.092540+0000","last_undegraded":"2024-09-16T15:52:17.092540+0000","last_fullsized":"2024-09-16T15:52:17.092540+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:32:52.491531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336424+0000","last_change":"2024-09-16T15:52:18.101280+0000","last_active":"2024-09-16T15:52:19.336424+0000","last_peered":"2024-09-16T15:52:19.336424+0000","last_clean":"2024-09-16T15:52:19.336424+0000","last_became_active":"2024-09-16T15:52:18.100938+0000","last_became_peered":"2024-09-16T15:52:18.100938+0000","last_unstale":"2024-09-16T15:52:19.336424+0000","last_undegraded":"2024-09-16T15:52:19.336424+0000","last_fullsized":"2024-09-16T15:52:19.336424+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:18:58.903310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098529+0000","last_change":"2024-09-16T15:52:17.090751+0000","last_active":"2024-09-16T15:52:19.098529+0000","last_peered":"2024-09-16T15:52:19.098529+0000","last_clean":"2024-09-16T15:52:19.098529+0000","last_became_active":"2024-09-16T15:52:17.090390+0000","last_became_peered":"2024-09-16T15:52:17.090390+0000","last_unstale":"2024-09-16T15:52:19.098529+0000","last_undegraded":"2024-09-16T15:52:19.098529+0000","last_fullsized":"2024-09-16T15:52:19.098529+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:58:11.585044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098903+0000","last_change":"2024-09-16T15:52:18.090433+0000","last_active":"2024-09-16T15:52:19.098903+0000","last_peered":"2024-09-16T15:52:19.098903+0000","last_clean":"2024-09-16T15:52:19.098903+0000","last_became_active":"2024-09-16T15:52:18.090169+0000","last_became_peered":"2024-09-16T15:52:18.090169+0000","last_unstale":"2024-09-16T15:52:19.098903+0000","last_undegraded":"2024-09-16T15:52:19.098903+0000","last_fullsized":"2024-09-16T15:52:19.098903+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:20:57.801785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336267+0000","last_change":"2024-09-16T15:52:17.093517+0000","last_active":"2024-09-16T15:52:19.336267+0000","last_peered":"2024-09-16T15:52:19.336267+0000","last_clean":"2024-09-16T15:52:19.336267+0000","last_became_active":"2024-09-16T15:52:17.093229+0000","last_became_peered":"2024-09-16T15:52:17.093229+0000","last_unstale":"2024-09-16T15:52:19.336267+0000","last_undegraded":"2024-09-16T15:52:19.336267+0000","last_fullsized":"2024-09-16T15:52:19.336267+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:22:18.051932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.324584+0000","last_change":"2024-09-16T15:52:18.099124+0000","last_active":"2024-09-16T15:52:19.324584+0000","last_peered":"2024-09-16T15:52:19.324584+0000","last_clean":"2024-09-16T15:52:19.324584+0000","last_became_active":"2024-09-16T15:52:18.098576+0000","last_became_peered":"2024-09-16T15:52:18.098576+0000","last_unstale":"2024-09-16T15:52:19.324584+0000","last_undegraded":"2024-09-16T15:52:19.324584+0000","last_fullsized":"2024-09-16T15:52:19.324584+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:50:42.139677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336848+0000","last_change":"2024-09-16T15:52:17.087919+0000","last_active":"2024-09-16T15:52:19.336848+0000","last_peered":"2024-09-16T15:52:19.336848+0000","last_clean":"2024-09-16T15:52:19.336848+0000","last_became_active":"2024-09-16T15:52:17.086912+0000","last_became_peered":"2024-09-16T15:52:17.086912+0000","last_unstale":"2024-09-16T15:52:19.336848+0000","last_undegraded":"2024-09-16T15:52:19.336848+0000","last_fullsized":"2024-09-16T15:52:19.336848+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:22:20.426416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.324724+0000","last_change":"2024-09-16T15:52:18.093980+0000","last_active":"2024-09-16T15:52:19.324724+0000","last_peered":"2024-09-16T15:52:19.324724+0000","last_clean":"2024-09-16T15:52:19.324724+0000","last_became_active":"2024-09-16T15:52:18.093733+0000","last_became_peered":"2024-09-16T15:52:18.093733+0000","last_unstale":"2024-09-16T15:52:19.324724+0000","last_undegraded":"2024-09-16T15:52:19.324724+0000","last_fullsized":"2024-09-16T15:52:19.324724+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:37:17.979172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336543+0000","last_change":"2024-09-16T15:52:17.088021+0000","last_active":"2024-09-16T15:52:19.336543+0000","last_peered":"2024-09-16T15:52:19.336543+0000","last_clean":"2024-09-16T15:52:19.336543+0000","last_became_active":"2024-09-16T15:52:17.087740+0000","last_became_peered":"2024-09-16T15:52:17.087740+0000","last_unstale":"2024-09-16T15:52:19.336543+0000","last_undegraded":"2024-09-16T15:52:19.336543+0000","last_fullsized":"2024-09-16T15:52:19.336543+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:20:18.506788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.097462+0000","last_change":"2024-09-16T15:52:18.088933+0000","last_active":"2024-09-16T15:52:19.097462+0000","last_peered":"2024-09-16T15:52:19.097462+0000","last_clean":"2024-09-16T15:52:19.097462+0000","last_became_active":"2024-09-16T15:52:18.088644+0000","last_became_peered":"2024-09-16T15:52:18.088644+0000","last_unstale":"2024-09-16T15:52:19.097462+0000","last_undegraded":"2024-09-16T15:52:19.097462+0000","last_fullsized":"2024-09-16T15:52:19.097462+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:50:44.707951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"117'3","reported_seq":18,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335658+0000","last_change":"2024-09-16T15:52:17.090647+0000","last_active":"2024-09-16T15:52:19.335658+0000","last_peered":"2024-09-16T15:52:19.335658+0000","last_clean":"2024-09-16T15:52:19.335658+0000","last_became_active":"2024-09-16T15:52:17.089382+0000","last_became_peered":"2024-09-16T15:52:17.089382+0000","last_unstale":"2024-09-16T15:52:19.335658+0000","last_undegraded":"2024-09-16T15:52:19.335658+0000","last_fullsized":"2024-09-16T15:52:19.335658+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:22:34.607260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098802+0000","last_change":"2024-09-16T15:52:18.107440+0000","last_active":"2024-09-16T15:52:19.098802+0000","last_peered":"2024-09-16T15:52:19.098802+0000","last_clean":"2024-09-16T15:52:19.098802+0000","last_became_active":"2024-09-16T15:52:18.107190+0000","last_became_peered":"2024-09-16T15:52:18.107190+0000","last_unstale":"2024-09-16T15:52:19.098802+0000","last_undegraded":"2024-09-16T15:52:19.098802+0000","last_fullsized":"2024-09-16T15:52:19.098802+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:05:50.369655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.14","version":"117'6","reported_seq":21,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336439+0000","last_change":"2024-09-16T15:52:17.088066+0000","last_active":"2024-09-16T15:52:19.336439+0000","last_peered":"2024-09-16T15:52:19.336439+0000","last_clean":"2024-09-16T15:52:19.336439+0000","last_became_active":"2024-09-16T15:52:17.087451+0000","last_became_peered":"2024-09-16T15:52:17.087451+0000","last_unstale":"2024-09-16T15:52:19.336439+0000","last_undegraded":"2024-09-16T15:52:19.336439+0000","last_fullsized":"2024-09-16T15:52:19.336439+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:41:10.165278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335355+0000","last_change":"2024-09-16T15:52:18.098410+0000","last_active":"2024-09-16T15:52:19.335355+0000","last_peered":"2024-09-16T15:52:19.335355+0000","last_clean":"2024-09-16T15:52:19.335355+0000","last_became_active":"2024-09-16T15:52:18.098149+0000","last_became_peered":"2024-09-16T15:52:18.098149+0000","last_unstale":"2024-09-16T15:52:19.335355+0000","last_undegraded":"2024-09-16T15:52:19.335355+0000","last_fullsized":"2024-09-16T15:52:19.335355+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:25:09.258265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"117'2","reported_seq":17,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335984+0000","last_change":"2024-09-16T15:52:17.086637+0000","last_active":"2024-09-16T15:52:19.335984+0000","last_peered":"2024-09-16T15:52:19.335984+0000","last_clean":"2024-09-16T15:52:19.335984+0000","last_became_active":"2024-09-16T15:52:17.086116+0000","last_became_peered":"2024-09-16T15:52:17.086116+0000","last_unstale":"2024-09-16T15:52:19.335984+0000","last_undegraded":"2024-09-16T15:52:19.335984+0000","last_fullsized":"2024-09-16T15:52:19.335984+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:33:42.514181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335966+0000","last_change":"2024-09-16T15:52:18.108762+0000","last_active":"2024-09-16T15:52:19.335966+0000","last_peered":"2024-09-16T15:52:19.335966+0000","last_clean":"2024-09-16T15:52:19.335966+0000","last_became_active":"2024-09-16T15:52:18.106039+0000","last_became_peered":"2024-09-16T15:52:18.106039+0000","last_unstale":"2024-09-16T15:52:19.335966+0000","last_undegraded":"2024-09-16T15:52:19.335966+0000","last_fullsized":"2024-09-16T15:52:19.335966+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:20:44.949446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.095450+0000","last_change":"2024-09-16T15:52:17.091004+0000","last_active":"2024-09-16T15:52:19.095450+0000","last_peered":"2024-09-16T15:52:19.095450+0000","last_clean":"2024-09-16T15:52:19.095450+0000","last_became_active":"2024-09-16T15:52:17.090693+0000","last_became_peered":"2024-09-16T15:52:17.090693+0000","last_unstale":"2024-09-16T15:52:19.095450+0000","last_undegraded":"2024-09-16T15:52:19.095450+0000","last_fullsized":"2024-09-16T15:52:19.095450+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:36:52.865477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336681+0000","last_change":"2024-09-16T15:52:18.096055+0000","last_active":"2024-09-16T15:52:19.336681+0000","last_peered":"2024-09-16T15:52:19.336681+0000","last_clean":"2024-09-16T15:52:19.336681+0000","last_became_active":"2024-09-16T15:52:18.095784+0000","last_became_peered":"2024-09-16T15:52:18.095784+0000","last_unstale":"2024-09-16T15:52:19.336681+0000","last_undegraded":"2024-09-16T15:52:19.336681+0000","last_fullsized":"2024-09-16T15:52:19.336681+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:01:04.752236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-16T15:52:17.089606+0000","last_change":"2024-09-16T15:52:17.084493+0000","last_active":"2024-09-16T15:52:17.089606+0000","last_peered":"2024-09-16T15:52:17.089606+0000","last_clean":"2024-09-16T15:52:17.089606+0000","last_became_active":"2024-09-16T15:52:17.083507+0000","last_became_peered":"2024-09-16T15:52:17.083507+0000","last_unstale":"2024-09-16T15:52:17.089606+0000","last_undegraded":"2024-09-16T15:52:17.089606+0000","last_fullsized":"2024-09-16T15:52:17.089606+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:22:31.431419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-16T15:52:17.076690+0000","last_change":"2024-09-16T15:52:17.076690+0000","last_active":"2024-09-16T15:52:17.058582+0000","last_peered":"2024-09-16T15:52:17.058582+0000","last_clean":"2024-09-16T15:52:17.058582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:52:17.076690+0000","last_undegraded":"2024-09-16T15:52:17.076690+0000","last_fullsized":"2024-09-16T15:52:17.076690+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.f","version":"117'1","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.097553+0000","last_change":"2024-09-16T15:52:17.079920+0000","last_active":"2024-09-16T15:52:19.097553+0000","last_peered":"2024-09-16T15:52:19.097553+0000","last_clean":"2024-09-16T15:52:19.097553+0000","last_became_active":"2024-09-16T15:52:17.079394+0000","last_became_peered":"2024-09-16T15:52:17.079394+0000","last_unstale":"2024-09-16T15:52:19.097553+0000","last_undegraded":"2024-09-16T15:52:19.097553+0000","last_fullsized":"2024-09-16T15:52:19.097553+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:27:33.808174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-16T15:52:17.075988+0000","last_change":"2024-09-16T15:52:17.075988+0000","last_active":"2024-09-16T15:52:17.058582+0000","last_peered":"2024-09-16T15:52:17.058582+0000","last_clean":"2024-09-16T15:52:17.058582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:52:17.075988+0000","last_undegraded":"2024-09-16T15:52:17.075988+0000","last_fullsized":"2024-09-16T15:52:17.075988+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"117'2","reported_seq":17,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336861+0000","last_change":"2024-09-16T15:52:17.087201+0000","last_active":"2024-09-16T15:52:19.336861+0000","last_peered":"2024-09-16T15:52:19.336861+0000","last_clean":"2024-09-16T15:52:19.336861+0000","last_became_active":"2024-09-16T15:52:17.085562+0000","last_became_peered":"2024-09-16T15:52:17.085562+0000","last_unstale":"2024-09-16T15:52:19.336861+0000","last_undegraded":"2024-09-16T15:52:19.336861+0000","last_fullsized":"2024-09-16T15:52:19.336861+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:20:46.960384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098813+0000","last_change":"2024-09-16T15:52:18.097949+0000","last_active":"2024-09-16T15:52:19.098813+0000","last_peered":"2024-09-16T15:52:19.098813+0000","last_clean":"2024-09-16T15:52:19.098813+0000","last_became_active":"2024-09-16T15:52:18.097693+0000","last_became_peered":"2024-09-16T15:52:18.097693+0000","last_unstale":"2024-09-16T15:52:19.098813+0000","last_undegraded":"2024-09-16T15:52:19.098813+0000","last_fullsized":"2024-09-16T15:52:19.098813+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:24:52.156928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.d","version":"117'1","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335394+0000","last_change":"2024-09-16T15:52:17.087628+0000","last_active":"2024-09-16T15:52:19.335394+0000","last_peered":"2024-09-16T15:52:19.335394+0000","last_clean":"2024-09-16T15:52:19.335394+0000","last_became_active":"2024-09-16T15:52:17.087384+0000","last_became_peered":"2024-09-16T15:52:17.087384+0000","last_unstale":"2024-09-16T15:52:19.335394+0000","last_undegraded":"2024-09-16T15:52:19.335394+0000","last_fullsized":"2024-09-16T15:52:19.335394+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:42:11.616130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335328+0000","last_change":"2024-09-16T15:52:18.114650+0000","last_active":"2024-09-16T15:52:19.335328+0000","last_peered":"2024-09-16T15:52:19.335328+0000","last_clean":"2024-09-16T15:52:19.335328+0000","last_became_active":"2024-09-16T15:52:18.111886+0000","last_became_peered":"2024-09-16T15:52:18.111886+0000","last_unstale":"2024-09-16T15:52:19.335328+0000","last_undegraded":"2024-09-16T15:52:19.335328+0000","last_fullsized":"2024-09-16T15:52:19.335328+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:27:33.482617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335048+0000","last_change":"2024-09-16T15:52:17.090919+0000","last_active":"2024-09-16T15:52:19.335048+0000","last_peered":"2024-09-16T15:52:19.335048+0000","last_clean":"2024-09-16T15:52:19.335048+0000","last_became_active":"2024-09-16T15:52:17.090639+0000","last_became_peered":"2024-09-16T15:52:17.090639+0000","last_unstale":"2024-09-16T15:52:19.335048+0000","last_undegraded":"2024-09-16T15:52:19.335048+0000","last_fullsized":"2024-09-16T15:52:19.335048+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:02:59.981560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.095970+0000","last_change":"2024-09-16T15:52:18.099682+0000","last_active":"2024-09-16T15:52:19.095970+0000","last_peered":"2024-09-16T15:52:19.095970+0000","last_clean":"2024-09-16T15:52:19.095970+0000","last_became_active":"2024-09-16T15:52:18.093247+0000","last_became_peered":"2024-09-16T15:52:18.093247+0000","last_unstale":"2024-09-16T15:52:19.095970+0000","last_undegraded":"2024-09-16T15:52:19.095970+0000","last_fullsized":"2024-09-16T15:52:19.095970+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:39:08.375481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.b","version":"117'2","reported_seq":17,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336094+0000","last_change":"2024-09-16T15:52:17.086366+0000","last_active":"2024-09-16T15:52:19.336094+0000","last_peered":"2024-09-16T15:52:19.336094+0000","last_clean":"2024-09-16T15:52:19.336094+0000","last_became_active":"2024-09-16T15:52:17.085765+0000","last_became_peered":"2024-09-16T15:52:17.085765+0000","last_unstale":"2024-09-16T15:52:19.336094+0000","last_undegraded":"2024-09-16T15:52:19.336094+0000","last_fullsized":"2024-09-16T15:52:19.336094+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:32:36.741556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-16T15:52:17.076139+0000","last_change":"2024-09-16T15:52:17.076139+0000","last_active":"2024-09-16T15:52:17.058582+0000","last_peered":"2024-09-16T15:52:17.058582+0000","last_clean":"2024-09-16T15:52:17.058582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:52:17.076139+0000","last_undegraded":"2024-09-16T15:52:17.076139+0000","last_fullsized":"2024-09-16T15:52:17.076139+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-16T15:52:17.094304+0000","last_change":"2024-09-16T15:52:17.090433+0000","last_active":"2024-09-16T15:52:17.094304+0000","last_peered":"2024-09-16T15:52:17.094304+0000","last_clean":"2024-09-16T15:52:17.094304+0000","last_became_active":"2024-09-16T15:52:17.089911+0000","last_became_peered":"2024-09-16T15:52:17.089911+0000","last_unstale":"2024-09-16T15:52:17.094304+0000","last_undegraded":"2024-09-16T15:52:17.094304+0000","last_fullsized":"2024-09-16T15:52:17.094304+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:29:31.775552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.334976+0000","last_change":"2024-09-16T15:52:18.102029+0000","last_active":"2024-09-16T15:52:19.334976+0000","last_peered":"2024-09-16T15:52:19.334976+0000","last_clean":"2024-09-16T15:52:19.334976+0000","last_became_active":"2024-09-16T15:52:18.101193+0000","last_became_peered":"2024-09-16T15:52:18.101193+0000","last_unstale":"2024-09-16T15:52:19.334976+0000","last_undegraded":"2024-09-16T15:52:19.334976+0000","last_fullsized":"2024-09-16T15:52:19.334976+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:14:39.539437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336623+0000","last_change":"2024-09-16T15:52:17.089371+0000","last_active":"2024-09-16T15:52:19.336623+0000","last_peered":"2024-09-16T15:52:19.336623+0000","last_clean":"2024-09-16T15:52:19.336623+0000","last_became_active":"2024-09-16T15:52:17.089099+0000","last_became_peered":"2024-09-16T15:52:17.089099+0000","last_unstale":"2024-09-16T15:52:19.336623+0000","last_undegraded":"2024-09-16T15:52:19.336623+0000","last_fullsized":"2024-09-16T15:52:19.336623+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:04:46.705752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335539+0000","last_change":"2024-09-16T15:52:18.106035+0000","last_active":"2024-09-16T15:52:19.335539+0000","last_peered":"2024-09-16T15:52:19.335539+0000","last_clean":"2024-09-16T15:52:19.335539+0000","last_became_active":"2024-09-16T15:52:18.105771+0000","last_became_peered":"2024-09-16T15:52:18.105771+0000","last_unstale":"2024-09-16T15:52:19.335539+0000","last_undegraded":"2024-09-16T15:52:19.335539+0000","last_fullsized":"2024-09-16T15:52:19.335539+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:07:04.881864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336440+0000","last_change":"2024-09-16T15:52:17.078927+0000","last_active":"2024-09-16T15:52:19.336440+0000","last_peered":"2024-09-16T15:52:19.336440+0000","last_clean":"2024-09-16T15:52:19.336440+0000","last_became_active":"2024-09-16T15:52:17.078623+0000","last_became_peered":"2024-09-16T15:52:17.078623+0000","last_unstale":"2024-09-16T15:52:19.336440+0000","last_undegraded":"2024-09-16T15:52:19.336440+0000","last_fullsized":"2024-09-16T15:52:19.336440+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:32:34.228931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335362+0000","last_change":"2024-09-16T15:52:18.099029+0000","last_active":"2024-09-16T15:52:19.335362+0000","last_peered":"2024-09-16T15:52:19.335362+0000","last_clean":"2024-09-16T15:52:19.335362+0000","last_became_active":"2024-09-16T15:52:18.098767+0000","last_became_peered":"2024-09-16T15:52:18.098767+0000","last_unstale":"2024-09-16T15:52:19.335362+0000","last_undegraded":"2024-09-16T15:52:19.335362+0000","last_fullsized":"2024-09-16T15:52:19.335362+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:22:20.108931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.7","version":"117'2","reported_seq":17,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336898+0000","last_change":"2024-09-16T15:52:17.083225+0000","last_active":"2024-09-16T15:52:19.336898+0000","last_peered":"2024-09-16T15:52:19.336898+0000","last_clean":"2024-09-16T15:52:19.336898+0000","last_became_active":"2024-09-16T15:52:17.082946+0000","last_became_peered":"2024-09-16T15:52:17.082946+0000","last_unstale":"2024-09-16T15:52:19.336898+0000","last_undegraded":"2024-09-16T15:52:19.336898+0000","last_fullsized":"2024-09-16T15:52:19.336898+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:53:28.145727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.097472+0000","last_change":"2024-09-16T15:52:18.096158+0000","last_active":"2024-09-16T15:52:19.097472+0000","last_peered":"2024-09-16T15:52:19.097472+0000","last_clean":"2024-09-16T15:52:19.097472+0000","last_became_active":"2024-09-16T15:52:18.095854+0000","last_became_peered":"2024-09-16T15:52:18.095854+0000","last_unstale":"2024-09-16T15:52:19.097472+0000","last_undegraded":"2024-09-16T15:52:19.097472+0000","last_fullsized":"2024-09-16T15:52:19.097472+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:19:03.220760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"118'1","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098686+0000","last_change":"2024-09-16T15:52:17.091331+0000","last_active":"2024-09-16T15:52:19.098686+0000","last_peered":"2024-09-16T15:52:19.098686+0000","last_clean":"2024-09-16T15:52:19.098686+0000","last_became_active":"2024-09-16T15:52:17.091071+0000","last_became_peered":"2024-09-16T15:52:17.091071+0000","last_unstale":"2024-09-16T15:52:19.098686+0000","last_undegraded":"2024-09-16T15:52:19.098686+0000","last_fullsized":"2024-09-16T15:52:19.098686+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:18:38.561424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-16T15:52:17.077848+0000","last_change":"2024-09-16T15:52:17.077848+0000","last_active":"2024-09-16T15:52:17.058582+0000","last_peered":"2024-09-16T15:52:17.058582+0000","last_clean":"2024-09-16T15:52:17.058582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:52:17.077848+0000","last_undegraded":"2024-09-16T15:52:17.077848+0000","last_fullsized":"2024-09-16T15:52:17.077848+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336336+0000","last_change":"2024-09-16T15:52:17.080632+0000","last_active":"2024-09-16T15:52:19.336336+0000","last_peered":"2024-09-16T15:52:19.336336+0000","last_clean":"2024-09-16T15:52:19.336336+0000","last_became_active":"2024-09-16T15:52:17.079770+0000","last_became_peered":"2024-09-16T15:52:17.079770+0000","last_unstale":"2024-09-16T15:52:19.336336+0000","last_undegraded":"2024-09-16T15:52:19.336336+0000","last_fullsized":"2024-09-16T15:52:19.336336+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:18:23.166654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335328+0000","last_change":"2024-09-16T15:52:18.099378+0000","last_active":"2024-09-16T15:52:19.335328+0000","last_peered":"2024-09-16T15:52:19.335328+0000","last_clean":"2024-09-16T15:52:19.335328+0000","last_became_active":"2024-09-16T15:52:18.096154+0000","last_became_peered":"2024-09-16T15:52:18.096154+0000","last_unstale":"2024-09-16T15:52:19.335328+0000","last_undegraded":"2024-09-16T15:52:19.335328+0000","last_fullsized":"2024-09-16T15:52:19.335328+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:42:01.895815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336463+0000","last_change":"2024-09-16T15:52:17.091377+0000","last_active":"2024-09-16T15:52:19.336463+0000","last_peered":"2024-09-16T15:52:19.336463+0000","last_clean":"2024-09-16T15:52:19.336463+0000","last_became_active":"2024-09-16T15:52:17.090957+0000","last_became_peered":"2024-09-16T15:52:17.090957+0000","last_unstale":"2024-09-16T15:52:19.336463+0000","last_undegraded":"2024-09-16T15:52:19.336463+0000","last_fullsized":"2024-09-16T15:52:19.336463+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:21:10.990720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.095396+0000","last_change":"2024-09-16T15:52:18.107499+0000","last_active":"2024-09-16T15:52:19.095396+0000","last_peered":"2024-09-16T15:52:19.095396+0000","last_clean":"2024-09-16T15:52:19.095396+0000","last_became_active":"2024-09-16T15:52:18.105395+0000","last_became_peered":"2024-09-16T15:52:18.105395+0000","last_unstale":"2024-09-16T15:52:19.095396+0000","last_undegraded":"2024-09-16T15:52:19.095396+0000","last_fullsized":"2024-09-16T15:52:19.095396+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:15:42.167943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098838+0000","last_change":"2024-09-16T15:52:17.088515+0000","last_active":"2024-09-16T15:52:19.098838+0000","last_peered":"2024-09-16T15:52:19.098838+0000","last_clean":"2024-09-16T15:52:19.098838+0000","last_became_active":"2024-09-16T15:52:17.088093+0000","last_became_peered":"2024-09-16T15:52:17.088093+0000","last_unstale":"2024-09-16T15:52:19.098838+0000","last_undegraded":"2024-09-16T15:52:19.098838+0000","last_fullsized":"2024-09-16T15:52:19.098838+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:41:51.353892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335174+0000","last_change":"2024-09-16T15:52:18.104181+0000","last_active":"2024-09-16T15:52:19.335174+0000","last_peered":"2024-09-16T15:52:19.335174+0000","last_clean":"2024-09-16T15:52:19.335174+0000","last_became_active":"2024-09-16T15:52:18.096537+0000","last_became_peered":"2024-09-16T15:52:18.096537+0000","last_unstale":"2024-09-16T15:52:19.335174+0000","last_undegraded":"2024-09-16T15:52:19.335174+0000","last_fullsized":"2024-09-16T15:52:19.335174+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:53:28.863711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.334882+0000","last_change":"2024-09-16T15:52:17.087423+0000","last_active":"2024-09-16T15:52:19.334882+0000","last_peered":"2024-09-16T15:52:19.334882+0000","last_clean":"2024-09-16T15:52:19.334882+0000","last_became_active":"2024-09-16T15:52:17.087145+0000","last_became_peered":"2024-09-16T15:52:17.087145+0000","last_unstale":"2024-09-16T15:52:19.334882+0000","last_undegraded":"2024-09-16T15:52:19.334882+0000","last_fullsized":"2024-09-16T15:52:19.334882+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:09:02.776672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335017+0000","last_change":"2024-09-16T15:52:18.095897+0000","last_active":"2024-09-16T15:52:19.335017+0000","last_peered":"2024-09-16T15:52:19.335017+0000","last_clean":"2024-09-16T15:52:19.335017+0000","last_became_active":"2024-09-16T15:52:18.095588+0000","last_became_peered":"2024-09-16T15:52:18.095588+0000","last_unstale":"2024-09-16T15:52:19.335017+0000","last_undegraded":"2024-09-16T15:52:19.335017+0000","last_fullsized":"2024-09-16T15:52:19.335017+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:28:29.259290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-16T15:52:17.089987+0000","last_change":"2024-09-16T15:52:17.085524+0000","last_active":"2024-09-16T15:52:17.089987+0000","last_peered":"2024-09-16T15:52:17.089987+0000","last_clean":"2024-09-16T15:52:17.089987+0000","last_became_active":"2024-09-16T15:52:17.084921+0000","last_became_peered":"2024-09-16T15:52:17.084921+0000","last_unstale":"2024-09-16T15:52:17.089987+0000","last_undegraded":"2024-09-16T15:52:17.089987+0000","last_fullsized":"2024-09-16T15:52:17.089987+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:11:21.294316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.096777+0000","last_change":"2024-09-16T15:52:18.106049+0000","last_active":"2024-09-16T15:52:19.096777+0000","last_peered":"2024-09-16T15:52:19.096777+0000","last_clean":"2024-09-16T15:52:19.096777+0000","last_became_active":"2024-09-16T15:52:18.105352+0000","last_became_peered":"2024-09-16T15:52:18.105352+0000","last_unstale":"2024-09-16T15:52:19.096777+0000","last_undegraded":"2024-09-16T15:52:19.096777+0000","last_fullsized":"2024-09-16T15:52:19.096777+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:29:18.998165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.096570+0000","last_change":"2024-09-16T15:52:17.079490+0000","last_active":"2024-09-16T15:52:19.096570+0000","last_peered":"2024-09-16T15:52:19.096570+0000","last_clean":"2024-09-16T15:52:19.096570+0000","last_became_active":"2024-09-16T15:52:17.079148+0000","last_became_peered":"2024-09-16T15:52:17.079148+0000","last_unstale":"2024-09-16T15:52:19.096570+0000","last_undegraded":"2024-09-16T15:52:19.096570+0000","last_fullsized":"2024-09-16T15:52:19.096570+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:17:11.750051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336381+0000","last_change":"2024-09-16T15:52:18.087311+0000","last_active":"2024-09-16T15:52:19.336381+0000","last_peered":"2024-09-16T15:52:19.336381+0000","last_clean":"2024-09-16T15:52:19.336381+0000","last_became_active":"2024-09-16T15:52:18.086640+0000","last_became_peered":"2024-09-16T15:52:18.086640+0000","last_unstale":"2024-09-16T15:52:19.336381+0000","last_undegraded":"2024-09-16T15:52:19.336381+0000","last_fullsized":"2024-09-16T15:52:19.336381+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:12:10.637529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.324319+0000","last_change":"2024-09-16T15:52:17.072908+0000","last_active":"2024-09-16T15:52:19.324319+0000","last_peered":"2024-09-16T15:52:19.324319+0000","last_clean":"2024-09-16T15:52:19.324319+0000","last_became_active":"2024-09-16T15:52:17.072645+0000","last_became_peered":"2024-09-16T15:52:17.072645+0000","last_unstale":"2024-09-16T15:52:19.324319+0000","last_undegraded":"2024-09-16T15:52:19.324319+0000","last_fullsized":"2024-09-16T15:52:19.324319+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:26:34.557572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":293,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.095214+0000","last_change":"2024-09-16T15:39:12.698813+0000","last_active":"2024-09-16T15:52:19.095214+0000","last_peered":"2024-09-16T15:52:19.095214+0000","last_clean":"2024-09-16T15:52:19.095214+0000","last_became_active":"2024-09-16T15:39:12.697695+0000","last_became_peered":"2024-09-16T15:39:12.697695+0000","last_unstale":"2024-09-16T15:52:19.095214+0000","last_undegraded":"2024-09-16T15:52:19.095214+0000","last_fullsized":"2024-09-16T15:52:19.095214+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_clean_scrub_stamp":"2024-09-16T15:39:11.684967+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:02.061081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":30,"ondisk_log_size":30,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509602,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":136652,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94235188,"statfs":{"total":96636764160,"available":96496832512,"internally_reserved":0,"allocated":1433600,"data_stored":627651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":138475976},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509602,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":135864,"kb_used_data":2012,"kb_used_omap":1,"kb_used_meta":133822,"kb_avail":94235976,"statfs":{"total":96636764160,"available":96497639424,"internally_reserved":0,"allocated":2060288,"data_stored":1231354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":137034183},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509601,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149172,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222668,"statfs":{"total":96636764160,"available":96484012032,"internally_reserved":0,"allocated":1474560,"data_stored":642596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509601,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131112,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240728,"statfs":{"total":96636764160,"available":96502505472,"internally_reserved":0,"allocated":1458176,"data_stored":640894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":132774345},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509601,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126968,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244872,"statfs":{"total":96636764160,"available":96506748928,"internally_reserved":0,"allocated":1470464,"data_stored":641532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":128514503},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509602,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127536,"kb_used_data":2004,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244304,"statfs":{"total":96636764160,"available":96506167296,"internally_reserved":0,"allocated":2052096,"data_stored":1231262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":128514503},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509600,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145012,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226828,"statfs":{"total":96636764160,"available":96488271872,"internally_reserved":0,"allocated":1466368,"data_stored":641546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":146995655},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542307,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126972,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244868,"statfs":{"total":96636764160,"available":96506744832,"internally_reserved":0,"allocated":1478656,"data_stored":642630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":128514508},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:52:22.762 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:52:22.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.955+0000 7f551b200640 1 -- 172.21.15.151:0/2099002193 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f54f8004d40 msgr2=0x7f54f80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:22.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.955+0000 7f551b200640 1 --2- 172.21.15.151:0/2099002193 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f54f8004d40 0x7f54f80051c0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f550c009ed0 tx=0x7f54f8005830 comp rx=0 tx=0).stop 2024-09-16T15:52:22.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.955+0000 7f5519200640 1 -- 172.21.15.151:0/2099002193 reap_dead start 2024-09-16T15:52:22.959 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.956+0000 7f551b200640 1 -- 172.21.15.151:0/2099002193 shutdown_connections 2024-09-16T15:52:22.959 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.956+0000 7f551b200640 1 -- 172.21.15.151:0/2099002193 >> 172.21.15.151:0/2099002193 conn(0x7f55140b1950 msgr2=0x7f55140b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:22.959 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.956+0000 7f551b200640 1 -- 172.21.15.151:0/2099002193 shutdown_connections 2024-09-16T15:52:22.959 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.956+0000 7f551b200640 1 -- 172.21.15.151:0/2099002193 wait complete. 2024-09-16T15:52:22.959 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.957+0000 7f551b200640 1 Processor -- start 2024-09-16T15:52:22.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.957+0000 7f551b200640 1 -- start start 2024-09-16T15:52:22.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.958+0000 7f551b200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f54f8004d40 0x7f5514146860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:22.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.958+0000 7f551b200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5514154680 0x7f5514154b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:22.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.958+0000 7f551b200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f55141f4970 0x7f55141f4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:22.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.958+0000 7f551b200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5514074840 con 0x7f54f8004d40 2024-09-16T15:52:22.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.958+0000 7f551b200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f55140746c0 con 0x7f5514154680 2024-09-16T15:52:22.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.958+0000 7f551b200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f551406f160 con 0x7f55141f4970 2024-09-16T15:52:22.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.958+0000 7f5513e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f54f8004d40 0x7f5514146860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:22.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.959+0000 7f5513e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f54f8004d40 0x7f5514146860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41342/0 (socket says 172.21.15.151:41342) 2024-09-16T15:52:22.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.959+0000 7f5513e00640 1 -- 172.21.15.151:0/3199946947 learned_addr learned my addr 172.21.15.151:0/3199946947 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:52:22.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.959+0000 7f5519200640 1 --2- 172.21.15.151:0/3199946947 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f55141f4970 0x7f55141f4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:22.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.959+0000 7f5513400640 1 --2- 172.21.15.151:0/3199946947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5514154680 0x7f5514154b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:22.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.959+0000 7f5513e00640 1 -- 172.21.15.151:0/3199946947 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f55141f4970 msgr2=0x7f55141f4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:22.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.959+0000 7f5513e00640 1 --2- 172.21.15.151:0/3199946947 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f55141f4970 0x7f55141f4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:22.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.959+0000 7f5513e00640 1 -- 172.21.15.151:0/3199946947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5514154680 msgr2=0x7f5514154b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:22.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.959+0000 7f5513e00640 1 --2- 172.21.15.151:0/3199946947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5514154680 0x7f5514154b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:22.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.959+0000 7f5513e00640 1 -- 172.21.15.151:0/3199946947 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5514146da0 con 0x7f54f8004d40 2024-09-16T15:52:22.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.959+0000 7f5519200640 1 --2- 172.21.15.151:0/3199946947 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f55141f4970 0x7f55141f4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:52:22.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.959+0000 7f5513400640 1 --2- 172.21.15.151:0/3199946947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5514154680 0x7f5514154b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:52:22.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.959+0000 7f5513e00640 1 --2- 172.21.15.151:0/3199946947 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f54f8004d40 0x7f5514146860 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f5504006680 tx=0x7f55040066b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:22.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.959+0000 7f550be00640 1 -- 172.21.15.151:0/3199946947 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5504015070 con 0x7f54f8004d40 2024-09-16T15:52:22.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.959+0000 7f550be00640 1 -- 172.21.15.151:0/3199946947 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5504011900 con 0x7f54f8004d40 2024-09-16T15:52:22.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.959+0000 7f550be00640 1 -- 172.21.15.151:0/3199946947 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5504011d50 con 0x7f54f8004d40 2024-09-16T15:52:22.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.959+0000 7f551b200640 1 -- 172.21.15.151:0/3199946947 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55141f56c0 con 0x7f54f8004d40 2024-09-16T15:52:22.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.960+0000 7f551b200640 1 -- 172.21.15.151:0/3199946947 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55140ceeb0 con 0x7f54f8004d40 2024-09-16T15:52:22.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.961+0000 7f551b200640 1 -- 172.21.15.151:0/3199946947 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f551406f6c0 con 0x7f54f8004d40 2024-09-16T15:52:22.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.962+0000 7f550be00640 1 -- 172.21.15.151:0/3199946947 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 41) ==== 103684+0+0 (secure 0 0 0) 0x7f5504018070 con 0x7f54f8004d40 2024-09-16T15:52:22.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.963+0000 7f550be00640 1 --2- 172.21.15.151:0/3199946947 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f54f407a2f0 0x7f54f407c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:22.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.963+0000 7f550be00640 1 -- 172.21.15.151:0/3199946947 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5793+0+0 (secure 0 0 0) 0x7f550409a240 con 0x7f54f8004d40 2024-09-16T15:52:22.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.963+0000 7f5513400640 1 --2- 172.21.15.151:0/3199946947 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f54f407a2f0 0x7f54f407c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:22.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.964+0000 7f5513400640 1 --2- 172.21.15.151:0/3199946947 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f54f407a2f0 0x7f54f407c7b0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f55141f5920 tx=0x7f5500006b10 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:22.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:22.967+0000 7f550be00640 1 -- 172.21.15.151:0/3199946947 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f5504064310 con 0x7f54f8004d40 2024-09-16T15:52:23.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.150+0000 7f551b200640 1 -- 172.21.15.151:0/3199946947 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f55140ae680 con 0x7f54f407a2f0 2024-09-16T15:52:23.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.155+0000 7f550be00640 1 -- 172.21.15.151:0/3199946947 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251751 (secure 0 0 0) 0x7f55140ae680 con 0x7f54f407a2f0 2024-09-16T15:52:23.161 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:52:23.162 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-09-16T15:52:23.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.159+0000 7f551b200640 1 -- 172.21.15.151:0/3199946947 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f54f407a2f0 msgr2=0x7f54f407c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:23.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.159+0000 7f551b200640 1 --2- 172.21.15.151:0/3199946947 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f54f407a2f0 0x7f54f407c7b0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f55141f5920 tx=0x7f5500006b10 comp rx=0 tx=0).stop 2024-09-16T15:52:23.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.159+0000 7f551b200640 1 -- 172.21.15.151:0/3199946947 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f54f8004d40 msgr2=0x7f5514146860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:23.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.159+0000 7f551b200640 1 --2- 172.21.15.151:0/3199946947 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f54f8004d40 0x7f5514146860 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f5504006680 tx=0x7f55040066b0 comp rx=0 tx=0).stop 2024-09-16T15:52:23.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.159+0000 7f551b200640 1 -- 172.21.15.151:0/3199946947 shutdown_connections 2024-09-16T15:52:23.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.159+0000 7f551b200640 1 --2- 172.21.15.151:0/3199946947 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f54f407a2f0 0x7f54f407c7b0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:23.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.159+0000 7f551b200640 1 --2- 172.21.15.151:0/3199946947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5514154680 0x7f5514154b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:23.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.160+0000 7f551b200640 1 --2- 172.21.15.151:0/3199946947 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f55141f4970 0x7f55141f4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:23.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.160+0000 7f551b200640 1 --2- 172.21.15.151:0/3199946947 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f54f8004d40 0x7f5514146860 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:23.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.160+0000 7f551b200640 1 -- 172.21.15.151:0/3199946947 >> 172.21.15.151:0/3199946947 conn(0x7f55140b1950 msgr2=0x7f5514153f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:23.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.160+0000 7f551b200640 1 -- 172.21.15.151:0/3199946947 shutdown_connections 2024-09-16T15:52:23.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.160+0000 7f551b200640 1 -- 172.21.15.151:0/3199946947 wait complete. 2024-09-16T15:52:23.178 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":507,"stamp":"2024-09-16T15:52:21.901931+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":134,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":102,"ondisk_log_size":102,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":165,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1079288,"kb_used_data":12592,"kb_used_omap":12,"kb_used_meta":1066483,"kb_avail":753895432,"statfs":{"total":773094113280,"available":771988922368,"internally_reserved":0,"allocated":12894208,"data_stored":6299465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12738,"internal_metadata":1092079166},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.837592"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-16T15:52:17.079031+0000","last_change":"2024-09-16T15:52:17.079031+0000","last_active":"2024-09-16T15:52:17.058582+0000","last_peered":"2024-09-16T15:52:17.058582+0000","last_clean":"2024-09-16T15:52:17.058582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:52:17.079031+0000","last_undegraded":"2024-09-16T15:52:17.079031+0000","last_fullsized":"2024-09-16T15:52:17.079031+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.324514+0000","last_change":"2024-09-16T15:52:18.095601+0000","last_active":"2024-09-16T15:52:19.324514+0000","last_peered":"2024-09-16T15:52:19.324514+0000","last_clean":"2024-09-16T15:52:19.324514+0000","last_became_active":"2024-09-16T15:52:18.095305+0000","last_became_peered":"2024-09-16T15:52:18.095305+0000","last_unstale":"2024-09-16T15:52:19.324514+0000","last_undegraded":"2024-09-16T15:52:19.324514+0000","last_fullsized":"2024-09-16T15:52:19.324514+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:46:48.807361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.095580+0000","last_change":"2024-09-16T15:52:18.108749+0000","last_active":"2024-09-16T15:52:19.095580+0000","last_peered":"2024-09-16T15:52:19.095580+0000","last_clean":"2024-09-16T15:52:19.095580+0000","last_became_active":"2024-09-16T15:52:18.104818+0000","last_became_peered":"2024-09-16T15:52:18.104818+0000","last_unstale":"2024-09-16T15:52:19.095580+0000","last_undegraded":"2024-09-16T15:52:19.095580+0000","last_fullsized":"2024-09-16T15:52:19.095580+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:32:30.003302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336245+0000","last_change":"2024-09-16T15:52:18.090782+0000","last_active":"2024-09-16T15:52:19.336245+0000","last_peered":"2024-09-16T15:52:19.336245+0000","last_clean":"2024-09-16T15:52:19.336245+0000","last_became_active":"2024-09-16T15:52:18.090432+0000","last_became_peered":"2024-09-16T15:52:18.090432+0000","last_unstale":"2024-09-16T15:52:19.336245+0000","last_undegraded":"2024-09-16T15:52:19.336245+0000","last_fullsized":"2024-09-16T15:52:19.336245+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:48:32.423354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335861+0000","last_change":"2024-09-16T15:52:18.109873+0000","last_active":"2024-09-16T15:52:19.335861+0000","last_peered":"2024-09-16T15:52:19.335861+0000","last_clean":"2024-09-16T15:52:19.335861+0000","last_became_active":"2024-09-16T15:52:18.101073+0000","last_became_peered":"2024-09-16T15:52:18.101073+0000","last_unstale":"2024-09-16T15:52:19.335861+0000","last_undegraded":"2024-09-16T15:52:19.335861+0000","last_fullsized":"2024-09-16T15:52:19.335861+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:56:57.514683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.097692+0000","last_change":"2024-09-16T15:52:18.096166+0000","last_active":"2024-09-16T15:52:19.097692+0000","last_peered":"2024-09-16T15:52:19.097692+0000","last_clean":"2024-09-16T15:52:19.097692+0000","last_became_active":"2024-09-16T15:52:18.095913+0000","last_became_peered":"2024-09-16T15:52:18.095913+0000","last_unstale":"2024-09-16T15:52:19.097692+0000","last_undegraded":"2024-09-16T15:52:19.097692+0000","last_fullsized":"2024-09-16T15:52:19.097692+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:51:43.908577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098468+0000","last_change":"2024-09-16T15:52:18.111297+0000","last_active":"2024-09-16T15:52:19.098468+0000","last_peered":"2024-09-16T15:52:19.098468+0000","last_clean":"2024-09-16T15:52:19.098468+0000","last_became_active":"2024-09-16T15:52:18.111039+0000","last_became_peered":"2024-09-16T15:52:18.111039+0000","last_unstale":"2024-09-16T15:52:19.098468+0000","last_undegraded":"2024-09-16T15:52:19.098468+0000","last_fullsized":"2024-09-16T15:52:19.098468+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:08:05.616283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098459+0000","last_change":"2024-09-16T15:52:18.094809+0000","last_active":"2024-09-16T15:52:19.098459+0000","last_peered":"2024-09-16T15:52:19.098459+0000","last_clean":"2024-09-16T15:52:19.098459+0000","last_became_active":"2024-09-16T15:52:18.094538+0000","last_became_peered":"2024-09-16T15:52:18.094538+0000","last_unstale":"2024-09-16T15:52:19.098459+0000","last_undegraded":"2024-09-16T15:52:19.098459+0000","last_fullsized":"2024-09-16T15:52:19.098459+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:02:37.475979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-16T15:52:17.076806+0000","last_change":"2024-09-16T15:52:17.076806+0000","last_active":"2024-09-16T15:52:17.058582+0000","last_peered":"2024-09-16T15:52:17.058582+0000","last_clean":"2024-09-16T15:52:17.058582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:52:17.076806+0000","last_undegraded":"2024-09-16T15:52:17.076806+0000","last_fullsized":"2024-09-16T15:52:17.076806+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336431+0000","last_change":"2024-09-16T15:52:18.101787+0000","last_active":"2024-09-16T15:52:19.336431+0000","last_peered":"2024-09-16T15:52:19.336431+0000","last_clean":"2024-09-16T15:52:19.336431+0000","last_became_active":"2024-09-16T15:52:18.100961+0000","last_became_peered":"2024-09-16T15:52:18.100961+0000","last_unstale":"2024-09-16T15:52:19.336431+0000","last_undegraded":"2024-09-16T15:52:19.336431+0000","last_fullsized":"2024-09-16T15:52:19.336431+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:31:15.594998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098664+0000","last_change":"2024-09-16T15:52:18.094110+0000","last_active":"2024-09-16T15:52:19.098664+0000","last_peered":"2024-09-16T15:52:19.098664+0000","last_clean":"2024-09-16T15:52:19.098664+0000","last_became_active":"2024-09-16T15:52:18.093858+0000","last_became_peered":"2024-09-16T15:52:18.093858+0000","last_unstale":"2024-09-16T15:52:19.098664+0000","last_undegraded":"2024-09-16T15:52:19.098664+0000","last_fullsized":"2024-09-16T15:52:19.098664+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:32:24.437229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335879+0000","last_change":"2024-09-16T15:52:18.102228+0000","last_active":"2024-09-16T15:52:19.335879+0000","last_peered":"2024-09-16T15:52:19.335879+0000","last_clean":"2024-09-16T15:52:19.335879+0000","last_became_active":"2024-09-16T15:52:18.101917+0000","last_became_peered":"2024-09-16T15:52:18.101917+0000","last_unstale":"2024-09-16T15:52:19.335879+0000","last_undegraded":"2024-09-16T15:52:19.335879+0000","last_fullsized":"2024-09-16T15:52:19.335879+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:49:20.323885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.324528+0000","last_change":"2024-09-16T15:52:18.093067+0000","last_active":"2024-09-16T15:52:19.324528+0000","last_peered":"2024-09-16T15:52:19.324528+0000","last_clean":"2024-09-16T15:52:19.324528+0000","last_became_active":"2024-09-16T15:52:18.092805+0000","last_became_peered":"2024-09-16T15:52:18.092805+0000","last_unstale":"2024-09-16T15:52:19.324528+0000","last_undegraded":"2024-09-16T15:52:19.324528+0000","last_fullsized":"2024-09-16T15:52:19.324528+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:06:51.783453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.334985+0000","last_change":"2024-09-16T15:52:18.098148+0000","last_active":"2024-09-16T15:52:19.334985+0000","last_peered":"2024-09-16T15:52:19.334985+0000","last_clean":"2024-09-16T15:52:19.334985+0000","last_became_active":"2024-09-16T15:52:18.096388+0000","last_became_peered":"2024-09-16T15:52:18.096388+0000","last_unstale":"2024-09-16T15:52:19.334985+0000","last_undegraded":"2024-09-16T15:52:19.334985+0000","last_fullsized":"2024-09-16T15:52:19.334985+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:59:34.319160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-16T15:52:17.076793+0000","last_change":"2024-09-16T15:52:17.076793+0000","last_active":"2024-09-16T15:52:17.058582+0000","last_peered":"2024-09-16T15:52:17.058582+0000","last_clean":"2024-09-16T15:52:17.058582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:52:17.076793+0000","last_undegraded":"2024-09-16T15:52:17.076793+0000","last_fullsized":"2024-09-16T15:52:17.076793+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335328+0000","last_change":"2024-09-16T15:52:18.096798+0000","last_active":"2024-09-16T15:52:19.335328+0000","last_peered":"2024-09-16T15:52:19.335328+0000","last_clean":"2024-09-16T15:52:19.335328+0000","last_became_active":"2024-09-16T15:52:18.096396+0000","last_became_peered":"2024-09-16T15:52:18.096396+0000","last_unstale":"2024-09-16T15:52:19.335328+0000","last_undegraded":"2024-09-16T15:52:19.335328+0000","last_fullsized":"2024-09-16T15:52:19.335328+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:17:25.344931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.095682+0000","last_change":"2024-09-16T15:52:18.113939+0000","last_active":"2024-09-16T15:52:19.095682+0000","last_peered":"2024-09-16T15:52:19.095682+0000","last_clean":"2024-09-16T15:52:19.095682+0000","last_became_active":"2024-09-16T15:52:18.113634+0000","last_became_peered":"2024-09-16T15:52:18.113634+0000","last_unstale":"2024-09-16T15:52:19.095682+0000","last_undegraded":"2024-09-16T15:52:19.095682+0000","last_fullsized":"2024-09-16T15:52:19.095682+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:04:19.441890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098743+0000","last_change":"2024-09-16T15:52:18.106010+0000","last_active":"2024-09-16T15:52:19.098743+0000","last_peered":"2024-09-16T15:52:19.098743+0000","last_clean":"2024-09-16T15:52:19.098743+0000","last_became_active":"2024-09-16T15:52:18.105722+0000","last_became_peered":"2024-09-16T15:52:18.105722+0000","last_unstale":"2024-09-16T15:52:19.098743+0000","last_undegraded":"2024-09-16T15:52:19.098743+0000","last_fullsized":"2024-09-16T15:52:19.098743+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:29:30.985879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.095786+0000","last_change":"2024-09-16T15:52:18.108828+0000","last_active":"2024-09-16T15:52:19.095786+0000","last_peered":"2024-09-16T15:52:19.095786+0000","last_clean":"2024-09-16T15:52:19.095786+0000","last_became_active":"2024-09-16T15:52:18.104728+0000","last_became_peered":"2024-09-16T15:52:18.104728+0000","last_unstale":"2024-09-16T15:52:19.095786+0000","last_undegraded":"2024-09-16T15:52:19.095786+0000","last_fullsized":"2024-09-16T15:52:19.095786+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:08:15.649535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.095454+0000","last_change":"2024-09-16T15:52:18.104490+0000","last_active":"2024-09-16T15:52:19.095454+0000","last_peered":"2024-09-16T15:52:19.095454+0000","last_clean":"2024-09-16T15:52:19.095454+0000","last_became_active":"2024-09-16T15:52:18.104176+0000","last_became_peered":"2024-09-16T15:52:18.104176+0000","last_unstale":"2024-09-16T15:52:19.095454+0000","last_undegraded":"2024-09-16T15:52:19.095454+0000","last_fullsized":"2024-09-16T15:52:19.095454+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:38:00.758145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098816+0000","last_change":"2024-09-16T15:52:18.100178+0000","last_active":"2024-09-16T15:52:19.098816+0000","last_peered":"2024-09-16T15:52:19.098816+0000","last_clean":"2024-09-16T15:52:19.098816+0000","last_became_active":"2024-09-16T15:52:18.099483+0000","last_became_peered":"2024-09-16T15:52:18.099483+0000","last_unstale":"2024-09-16T15:52:19.098816+0000","last_undegraded":"2024-09-16T15:52:19.098816+0000","last_fullsized":"2024-09-16T15:52:19.098816+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:16:33.962636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.097266+0000","last_change":"2024-09-16T15:52:18.095116+0000","last_active":"2024-09-16T15:52:19.097266+0000","last_peered":"2024-09-16T15:52:19.097266+0000","last_clean":"2024-09-16T15:52:19.097266+0000","last_became_active":"2024-09-16T15:52:18.090617+0000","last_became_peered":"2024-09-16T15:52:18.090617+0000","last_unstale":"2024-09-16T15:52:19.097266+0000","last_undegraded":"2024-09-16T15:52:19.097266+0000","last_fullsized":"2024-09-16T15:52:19.097266+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:23:03.363317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335071+0000","last_change":"2024-09-16T15:52:18.102959+0000","last_active":"2024-09-16T15:52:19.335071+0000","last_peered":"2024-09-16T15:52:19.335071+0000","last_clean":"2024-09-16T15:52:19.335071+0000","last_became_active":"2024-09-16T15:52:18.102684+0000","last_became_peered":"2024-09-16T15:52:18.102684+0000","last_unstale":"2024-09-16T15:52:19.335071+0000","last_undegraded":"2024-09-16T15:52:19.335071+0000","last_fullsized":"2024-09-16T15:52:19.335071+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:26:35.565582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336817+0000","last_change":"2024-09-16T15:52:18.102292+0000","last_active":"2024-09-16T15:52:19.336817+0000","last_peered":"2024-09-16T15:52:19.336817+0000","last_clean":"2024-09-16T15:52:19.336817+0000","last_became_active":"2024-09-16T15:52:18.102017+0000","last_became_peered":"2024-09-16T15:52:18.102017+0000","last_unstale":"2024-09-16T15:52:19.336817+0000","last_undegraded":"2024-09-16T15:52:19.336817+0000","last_fullsized":"2024-09-16T15:52:19.336817+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:34:23.254431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098723+0000","last_change":"2024-09-16T15:52:18.089811+0000","last_active":"2024-09-16T15:52:19.098723+0000","last_peered":"2024-09-16T15:52:19.098723+0000","last_clean":"2024-09-16T15:52:19.098723+0000","last_became_active":"2024-09-16T15:52:18.089302+0000","last_became_peered":"2024-09-16T15:52:18.089302+0000","last_unstale":"2024-09-16T15:52:19.098723+0000","last_undegraded":"2024-09-16T15:52:19.098723+0000","last_fullsized":"2024-09-16T15:52:19.098723+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:44:21.343778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.097694+0000","last_change":"2024-09-16T15:52:18.107014+0000","last_active":"2024-09-16T15:52:19.097694+0000","last_peered":"2024-09-16T15:52:19.097694+0000","last_clean":"2024-09-16T15:52:19.097694+0000","last_became_active":"2024-09-16T15:52:18.106706+0000","last_became_peered":"2024-09-16T15:52:18.106706+0000","last_unstale":"2024-09-16T15:52:19.097694+0000","last_undegraded":"2024-09-16T15:52:19.097694+0000","last_fullsized":"2024-09-16T15:52:19.097694+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:41:14.841086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336117+0000","last_change":"2024-09-16T15:52:18.102898+0000","last_active":"2024-09-16T15:52:19.336117+0000","last_peered":"2024-09-16T15:52:19.336117+0000","last_clean":"2024-09-16T15:52:19.336117+0000","last_became_active":"2024-09-16T15:52:18.102229+0000","last_became_peered":"2024-09-16T15:52:18.102229+0000","last_unstale":"2024-09-16T15:52:19.336117+0000","last_undegraded":"2024-09-16T15:52:19.336117+0000","last_fullsized":"2024-09-16T15:52:19.336117+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:21:53.345211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.097217+0000","last_change":"2024-09-16T15:52:18.099345+0000","last_active":"2024-09-16T15:52:19.097217+0000","last_peered":"2024-09-16T15:52:19.097217+0000","last_clean":"2024-09-16T15:52:19.097217+0000","last_became_active":"2024-09-16T15:52:18.098646+0000","last_became_peered":"2024-09-16T15:52:18.098646+0000","last_unstale":"2024-09-16T15:52:19.097217+0000","last_undegraded":"2024-09-16T15:52:19.097217+0000","last_fullsized":"2024-09-16T15:52:19.097217+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:28:37.287849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335126+0000","last_change":"2024-09-16T15:52:18.106525+0000","last_active":"2024-09-16T15:52:19.335126+0000","last_peered":"2024-09-16T15:52:19.335126+0000","last_clean":"2024-09-16T15:52:19.335126+0000","last_became_active":"2024-09-16T15:52:18.103778+0000","last_became_peered":"2024-09-16T15:52:18.103778+0000","last_unstale":"2024-09-16T15:52:19.335126+0000","last_undegraded":"2024-09-16T15:52:19.335126+0000","last_fullsized":"2024-09-16T15:52:19.335126+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:46:53.909612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098944+0000","last_change":"2024-09-16T15:52:18.087005+0000","last_active":"2024-09-16T15:52:19.098944+0000","last_peered":"2024-09-16T15:52:19.098944+0000","last_clean":"2024-09-16T15:52:19.098944+0000","last_became_active":"2024-09-16T15:52:18.086463+0000","last_became_peered":"2024-09-16T15:52:18.086463+0000","last_unstale":"2024-09-16T15:52:19.098944+0000","last_undegraded":"2024-09-16T15:52:19.098944+0000","last_fullsized":"2024-09-16T15:52:19.098944+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:23:31.782022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-16T15:52:17.076332+0000","last_change":"2024-09-16T15:52:17.076332+0000","last_active":"2024-09-16T15:52:17.058582+0000","last_peered":"2024-09-16T15:52:17.058582+0000","last_clean":"2024-09-16T15:52:17.058582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:52:17.076332+0000","last_undegraded":"2024-09-16T15:52:17.076332+0000","last_fullsized":"2024-09-16T15:52:17.076332+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335551+0000","last_change":"2024-09-16T15:52:18.106237+0000","last_active":"2024-09-16T15:52:19.335551+0000","last_peered":"2024-09-16T15:52:19.335551+0000","last_clean":"2024-09-16T15:52:19.335551+0000","last_became_active":"2024-09-16T15:52:18.103396+0000","last_became_peered":"2024-09-16T15:52:18.103396+0000","last_unstale":"2024-09-16T15:52:19.335551+0000","last_undegraded":"2024-09-16T15:52:19.335551+0000","last_fullsized":"2024-09-16T15:52:19.335551+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:35:46.682990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-16T15:52:17.081705+0000","last_change":"2024-09-16T15:52:17.081705+0000","last_active":"2024-09-16T15:52:17.058582+0000","last_peered":"2024-09-16T15:52:17.058582+0000","last_clean":"2024-09-16T15:52:17.058582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:52:17.081705+0000","last_undegraded":"2024-09-16T15:52:17.081705+0000","last_fullsized":"2024-09-16T15:52:17.081705+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"117'4","reported_seq":19,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335332+0000","last_change":"2024-09-16T15:52:17.086334+0000","last_active":"2024-09-16T15:52:19.335332+0000","last_peered":"2024-09-16T15:52:19.335332+0000","last_clean":"2024-09-16T15:52:19.335332+0000","last_became_active":"2024-09-16T15:52:17.086082+0000","last_became_peered":"2024-09-16T15:52:17.086082+0000","last_unstale":"2024-09-16T15:52:19.335332+0000","last_undegraded":"2024-09-16T15:52:19.335332+0000","last_fullsized":"2024-09-16T15:52:19.335332+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:25:34.552766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335430+0000","last_change":"2024-09-16T15:52:18.102484+0000","last_active":"2024-09-16T15:52:19.335430+0000","last_peered":"2024-09-16T15:52:19.335430+0000","last_clean":"2024-09-16T15:52:19.335430+0000","last_became_active":"2024-09-16T15:52:18.102159+0000","last_became_peered":"2024-09-16T15:52:18.102159+0000","last_unstale":"2024-09-16T15:52:19.335430+0000","last_undegraded":"2024-09-16T15:52:19.335430+0000","last_fullsized":"2024-09-16T15:52:19.335430+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:18:36.837288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1e","version":"117'1","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.324560+0000","last_change":"2024-09-16T15:52:17.082085+0000","last_active":"2024-09-16T15:52:19.324560+0000","last_peered":"2024-09-16T15:52:19.324560+0000","last_clean":"2024-09-16T15:52:19.324560+0000","last_became_active":"2024-09-16T15:52:17.081840+0000","last_became_peered":"2024-09-16T15:52:17.081840+0000","last_unstale":"2024-09-16T15:52:19.324560+0000","last_undegraded":"2024-09-16T15:52:19.324560+0000","last_fullsized":"2024-09-16T15:52:19.324560+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:46:52.901601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.097360+0000","last_change":"2024-09-16T15:52:18.104067+0000","last_active":"2024-09-16T15:52:19.097360+0000","last_peered":"2024-09-16T15:52:19.097360+0000","last_clean":"2024-09-16T15:52:19.097360+0000","last_became_active":"2024-09-16T15:52:18.103764+0000","last_became_peered":"2024-09-16T15:52:18.103764+0000","last_unstale":"2024-09-16T15:52:19.097360+0000","last_undegraded":"2024-09-16T15:52:19.097360+0000","last_fullsized":"2024-09-16T15:52:19.097360+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:51:28.451444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1d","version":"117'2","reported_seq":17,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336225+0000","last_change":"2024-09-16T15:52:17.083135+0000","last_active":"2024-09-16T15:52:19.336225+0000","last_peered":"2024-09-16T15:52:19.336225+0000","last_clean":"2024-09-16T15:52:19.336225+0000","last_became_active":"2024-09-16T15:52:17.082799+0000","last_became_peered":"2024-09-16T15:52:17.082799+0000","last_unstale":"2024-09-16T15:52:19.336225+0000","last_undegraded":"2024-09-16T15:52:19.336225+0000","last_fullsized":"2024-09-16T15:52:19.336225+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:43:05.707621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.095847+0000","last_change":"2024-09-16T15:52:18.109036+0000","last_active":"2024-09-16T15:52:19.095847+0000","last_peered":"2024-09-16T15:52:19.095847+0000","last_clean":"2024-09-16T15:52:19.095847+0000","last_became_active":"2024-09-16T15:52:18.106979+0000","last_became_peered":"2024-09-16T15:52:18.106979+0000","last_unstale":"2024-09-16T15:52:19.095847+0000","last_undegraded":"2024-09-16T15:52:19.095847+0000","last_fullsized":"2024-09-16T15:52:19.095847+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:38:25.952399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"117'2","reported_seq":17,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336324+0000","last_change":"2024-09-16T15:52:17.087728+0000","last_active":"2024-09-16T15:52:19.336324+0000","last_peered":"2024-09-16T15:52:19.336324+0000","last_clean":"2024-09-16T15:52:19.336324+0000","last_became_active":"2024-09-16T15:52:17.087063+0000","last_became_peered":"2024-09-16T15:52:17.087063+0000","last_unstale":"2024-09-16T15:52:19.336324+0000","last_undegraded":"2024-09-16T15:52:19.336324+0000","last_fullsized":"2024-09-16T15:52:19.336324+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:57:32.844856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336678+0000","last_change":"2024-09-16T15:52:18.111393+0000","last_active":"2024-09-16T15:52:19.336678+0000","last_peered":"2024-09-16T15:52:19.336678+0000","last_clean":"2024-09-16T15:52:19.336678+0000","last_became_active":"2024-09-16T15:52:18.111116+0000","last_became_peered":"2024-09-16T15:52:18.111116+0000","last_unstale":"2024-09-16T15:52:19.336678+0000","last_undegraded":"2024-09-16T15:52:19.336678+0000","last_fullsized":"2024-09-16T15:52:19.336678+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:59:34.319160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1b","version":"117'1","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.095273+0000","last_change":"2024-09-16T15:52:17.083411+0000","last_active":"2024-09-16T15:52:19.095273+0000","last_peered":"2024-09-16T15:52:19.095273+0000","last_clean":"2024-09-16T15:52:19.095273+0000","last_became_active":"2024-09-16T15:52:17.083098+0000","last_became_peered":"2024-09-16T15:52:17.083098+0000","last_unstale":"2024-09-16T15:52:19.095273+0000","last_undegraded":"2024-09-16T15:52:19.095273+0000","last_fullsized":"2024-09-16T15:52:19.095273+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:48:17.710440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098564+0000","last_change":"2024-09-16T15:52:18.099617+0000","last_active":"2024-09-16T15:52:19.098564+0000","last_peered":"2024-09-16T15:52:19.098564+0000","last_clean":"2024-09-16T15:52:19.098564+0000","last_became_active":"2024-09-16T15:52:18.099149+0000","last_became_peered":"2024-09-16T15:52:18.099149+0000","last_unstale":"2024-09-16T15:52:19.098564+0000","last_undegraded":"2024-09-16T15:52:19.098564+0000","last_fullsized":"2024-09-16T15:52:19.098564+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:41:19.687707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-16T15:52:17.092540+0000","last_change":"2024-09-16T15:52:17.086053+0000","last_active":"2024-09-16T15:52:17.092540+0000","last_peered":"2024-09-16T15:52:17.092540+0000","last_clean":"2024-09-16T15:52:17.092540+0000","last_became_active":"2024-09-16T15:52:17.084186+0000","last_became_peered":"2024-09-16T15:52:17.084186+0000","last_unstale":"2024-09-16T15:52:17.092540+0000","last_undegraded":"2024-09-16T15:52:17.092540+0000","last_fullsized":"2024-09-16T15:52:17.092540+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:32:52.491531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336424+0000","last_change":"2024-09-16T15:52:18.101280+0000","last_active":"2024-09-16T15:52:19.336424+0000","last_peered":"2024-09-16T15:52:19.336424+0000","last_clean":"2024-09-16T15:52:19.336424+0000","last_became_active":"2024-09-16T15:52:18.100938+0000","last_became_peered":"2024-09-16T15:52:18.100938+0000","last_unstale":"2024-09-16T15:52:19.336424+0000","last_undegraded":"2024-09-16T15:52:19.336424+0000","last_fullsized":"2024-09-16T15:52:19.336424+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:18:58.903310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098529+0000","last_change":"2024-09-16T15:52:17.090751+0000","last_active":"2024-09-16T15:52:19.098529+0000","last_peered":"2024-09-16T15:52:19.098529+0000","last_clean":"2024-09-16T15:52:19.098529+0000","last_became_active":"2024-09-16T15:52:17.090390+0000","last_became_peered":"2024-09-16T15:52:17.090390+0000","last_unstale":"2024-09-16T15:52:19.098529+0000","last_undegraded":"2024-09-16T15:52:19.098529+0000","last_fullsized":"2024-09-16T15:52:19.098529+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:58:11.585044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098903+0000","last_change":"2024-09-16T15:52:18.090433+0000","last_active":"2024-09-16T15:52:19.098903+0000","last_peered":"2024-09-16T15:52:19.098903+0000","last_clean":"2024-09-16T15:52:19.098903+0000","last_became_active":"2024-09-16T15:52:18.090169+0000","last_became_peered":"2024-09-16T15:52:18.090169+0000","last_unstale":"2024-09-16T15:52:19.098903+0000","last_undegraded":"2024-09-16T15:52:19.098903+0000","last_fullsized":"2024-09-16T15:52:19.098903+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:20:57.801785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336267+0000","last_change":"2024-09-16T15:52:17.093517+0000","last_active":"2024-09-16T15:52:19.336267+0000","last_peered":"2024-09-16T15:52:19.336267+0000","last_clean":"2024-09-16T15:52:19.336267+0000","last_became_active":"2024-09-16T15:52:17.093229+0000","last_became_peered":"2024-09-16T15:52:17.093229+0000","last_unstale":"2024-09-16T15:52:19.336267+0000","last_undegraded":"2024-09-16T15:52:19.336267+0000","last_fullsized":"2024-09-16T15:52:19.336267+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:22:18.051932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.324584+0000","last_change":"2024-09-16T15:52:18.099124+0000","last_active":"2024-09-16T15:52:19.324584+0000","last_peered":"2024-09-16T15:52:19.324584+0000","last_clean":"2024-09-16T15:52:19.324584+0000","last_became_active":"2024-09-16T15:52:18.098576+0000","last_became_peered":"2024-09-16T15:52:18.098576+0000","last_unstale":"2024-09-16T15:52:19.324584+0000","last_undegraded":"2024-09-16T15:52:19.324584+0000","last_fullsized":"2024-09-16T15:52:19.324584+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:50:42.139677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336848+0000","last_change":"2024-09-16T15:52:17.087919+0000","last_active":"2024-09-16T15:52:19.336848+0000","last_peered":"2024-09-16T15:52:19.336848+0000","last_clean":"2024-09-16T15:52:19.336848+0000","last_became_active":"2024-09-16T15:52:17.086912+0000","last_became_peered":"2024-09-16T15:52:17.086912+0000","last_unstale":"2024-09-16T15:52:19.336848+0000","last_undegraded":"2024-09-16T15:52:19.336848+0000","last_fullsized":"2024-09-16T15:52:19.336848+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:22:20.426416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.324724+0000","last_change":"2024-09-16T15:52:18.093980+0000","last_active":"2024-09-16T15:52:19.324724+0000","last_peered":"2024-09-16T15:52:19.324724+0000","last_clean":"2024-09-16T15:52:19.324724+0000","last_became_active":"2024-09-16T15:52:18.093733+0000","last_became_peered":"2024-09-16T15:52:18.093733+0000","last_unstale":"2024-09-16T15:52:19.324724+0000","last_undegraded":"2024-09-16T15:52:19.324724+0000","last_fullsized":"2024-09-16T15:52:19.324724+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:37:17.979172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336543+0000","last_change":"2024-09-16T15:52:17.088021+0000","last_active":"2024-09-16T15:52:19.336543+0000","last_peered":"2024-09-16T15:52:19.336543+0000","last_clean":"2024-09-16T15:52:19.336543+0000","last_became_active":"2024-09-16T15:52:17.087740+0000","last_became_peered":"2024-09-16T15:52:17.087740+0000","last_unstale":"2024-09-16T15:52:19.336543+0000","last_undegraded":"2024-09-16T15:52:19.336543+0000","last_fullsized":"2024-09-16T15:52:19.336543+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:20:18.506788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.097462+0000","last_change":"2024-09-16T15:52:18.088933+0000","last_active":"2024-09-16T15:52:19.097462+0000","last_peered":"2024-09-16T15:52:19.097462+0000","last_clean":"2024-09-16T15:52:19.097462+0000","last_became_active":"2024-09-16T15:52:18.088644+0000","last_became_peered":"2024-09-16T15:52:18.088644+0000","last_unstale":"2024-09-16T15:52:19.097462+0000","last_undegraded":"2024-09-16T15:52:19.097462+0000","last_fullsized":"2024-09-16T15:52:19.097462+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:50:44.707951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"117'3","reported_seq":18,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335658+0000","last_change":"2024-09-16T15:52:17.090647+0000","last_active":"2024-09-16T15:52:19.335658+0000","last_peered":"2024-09-16T15:52:19.335658+0000","last_clean":"2024-09-16T15:52:19.335658+0000","last_became_active":"2024-09-16T15:52:17.089382+0000","last_became_peered":"2024-09-16T15:52:17.089382+0000","last_unstale":"2024-09-16T15:52:19.335658+0000","last_undegraded":"2024-09-16T15:52:19.335658+0000","last_fullsized":"2024-09-16T15:52:19.335658+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:22:34.607260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098802+0000","last_change":"2024-09-16T15:52:18.107440+0000","last_active":"2024-09-16T15:52:19.098802+0000","last_peered":"2024-09-16T15:52:19.098802+0000","last_clean":"2024-09-16T15:52:19.098802+0000","last_became_active":"2024-09-16T15:52:18.107190+0000","last_became_peered":"2024-09-16T15:52:18.107190+0000","last_unstale":"2024-09-16T15:52:19.098802+0000","last_undegraded":"2024-09-16T15:52:19.098802+0000","last_fullsized":"2024-09-16T15:52:19.098802+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:05:50.369655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.14","version":"117'6","reported_seq":21,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336439+0000","last_change":"2024-09-16T15:52:17.088066+0000","last_active":"2024-09-16T15:52:19.336439+0000","last_peered":"2024-09-16T15:52:19.336439+0000","last_clean":"2024-09-16T15:52:19.336439+0000","last_became_active":"2024-09-16T15:52:17.087451+0000","last_became_peered":"2024-09-16T15:52:17.087451+0000","last_unstale":"2024-09-16T15:52:19.336439+0000","last_undegraded":"2024-09-16T15:52:19.336439+0000","last_fullsized":"2024-09-16T15:52:19.336439+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:41:10.165278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335355+0000","last_change":"2024-09-16T15:52:18.098410+0000","last_active":"2024-09-16T15:52:19.335355+0000","last_peered":"2024-09-16T15:52:19.335355+0000","last_clean":"2024-09-16T15:52:19.335355+0000","last_became_active":"2024-09-16T15:52:18.098149+0000","last_became_peered":"2024-09-16T15:52:18.098149+0000","last_unstale":"2024-09-16T15:52:19.335355+0000","last_undegraded":"2024-09-16T15:52:19.335355+0000","last_fullsized":"2024-09-16T15:52:19.335355+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:25:09.258265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"117'2","reported_seq":17,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335984+0000","last_change":"2024-09-16T15:52:17.086637+0000","last_active":"2024-09-16T15:52:19.335984+0000","last_peered":"2024-09-16T15:52:19.335984+0000","last_clean":"2024-09-16T15:52:19.335984+0000","last_became_active":"2024-09-16T15:52:17.086116+0000","last_became_peered":"2024-09-16T15:52:17.086116+0000","last_unstale":"2024-09-16T15:52:19.335984+0000","last_undegraded":"2024-09-16T15:52:19.335984+0000","last_fullsized":"2024-09-16T15:52:19.335984+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:33:42.514181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335966+0000","last_change":"2024-09-16T15:52:18.108762+0000","last_active":"2024-09-16T15:52:19.335966+0000","last_peered":"2024-09-16T15:52:19.335966+0000","last_clean":"2024-09-16T15:52:19.335966+0000","last_became_active":"2024-09-16T15:52:18.106039+0000","last_became_peered":"2024-09-16T15:52:18.106039+0000","last_unstale":"2024-09-16T15:52:19.335966+0000","last_undegraded":"2024-09-16T15:52:19.335966+0000","last_fullsized":"2024-09-16T15:52:19.335966+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:20:44.949446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.095450+0000","last_change":"2024-09-16T15:52:17.091004+0000","last_active":"2024-09-16T15:52:19.095450+0000","last_peered":"2024-09-16T15:52:19.095450+0000","last_clean":"2024-09-16T15:52:19.095450+0000","last_became_active":"2024-09-16T15:52:17.090693+0000","last_became_peered":"2024-09-16T15:52:17.090693+0000","last_unstale":"2024-09-16T15:52:19.095450+0000","last_undegraded":"2024-09-16T15:52:19.095450+0000","last_fullsized":"2024-09-16T15:52:19.095450+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:36:52.865477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336681+0000","last_change":"2024-09-16T15:52:18.096055+0000","last_active":"2024-09-16T15:52:19.336681+0000","last_peered":"2024-09-16T15:52:19.336681+0000","last_clean":"2024-09-16T15:52:19.336681+0000","last_became_active":"2024-09-16T15:52:18.095784+0000","last_became_peered":"2024-09-16T15:52:18.095784+0000","last_unstale":"2024-09-16T15:52:19.336681+0000","last_undegraded":"2024-09-16T15:52:19.336681+0000","last_fullsized":"2024-09-16T15:52:19.336681+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:01:04.752236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-16T15:52:17.089606+0000","last_change":"2024-09-16T15:52:17.084493+0000","last_active":"2024-09-16T15:52:17.089606+0000","last_peered":"2024-09-16T15:52:17.089606+0000","last_clean":"2024-09-16T15:52:17.089606+0000","last_became_active":"2024-09-16T15:52:17.083507+0000","last_became_peered":"2024-09-16T15:52:17.083507+0000","last_unstale":"2024-09-16T15:52:17.089606+0000","last_undegraded":"2024-09-16T15:52:17.089606+0000","last_fullsized":"2024-09-16T15:52:17.089606+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:22:31.431419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-16T15:52:17.076690+0000","last_change":"2024-09-16T15:52:17.076690+0000","last_active":"2024-09-16T15:52:17.058582+0000","last_peered":"2024-09-16T15:52:17.058582+0000","last_clean":"2024-09-16T15:52:17.058582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:52:17.076690+0000","last_undegraded":"2024-09-16T15:52:17.076690+0000","last_fullsized":"2024-09-16T15:52:17.076690+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.f","version":"117'1","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.097553+0000","last_change":"2024-09-16T15:52:17.079920+0000","last_active":"2024-09-16T15:52:19.097553+0000","last_peered":"2024-09-16T15:52:19.097553+0000","last_clean":"2024-09-16T15:52:19.097553+0000","last_became_active":"2024-09-16T15:52:17.079394+0000","last_became_peered":"2024-09-16T15:52:17.079394+0000","last_unstale":"2024-09-16T15:52:19.097553+0000","last_undegraded":"2024-09-16T15:52:19.097553+0000","last_fullsized":"2024-09-16T15:52:19.097553+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:27:33.808174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-16T15:52:17.075988+0000","last_change":"2024-09-16T15:52:17.075988+0000","last_active":"2024-09-16T15:52:17.058582+0000","last_peered":"2024-09-16T15:52:17.058582+0000","last_clean":"2024-09-16T15:52:17.058582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:52:17.075988+0000","last_undegraded":"2024-09-16T15:52:17.075988+0000","last_fullsized":"2024-09-16T15:52:17.075988+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"117'2","reported_seq":17,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336861+0000","last_change":"2024-09-16T15:52:17.087201+0000","last_active":"2024-09-16T15:52:19.336861+0000","last_peered":"2024-09-16T15:52:19.336861+0000","last_clean":"2024-09-16T15:52:19.336861+0000","last_became_active":"2024-09-16T15:52:17.085562+0000","last_became_peered":"2024-09-16T15:52:17.085562+0000","last_unstale":"2024-09-16T15:52:19.336861+0000","last_undegraded":"2024-09-16T15:52:19.336861+0000","last_fullsized":"2024-09-16T15:52:19.336861+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:20:46.960384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098813+0000","last_change":"2024-09-16T15:52:18.097949+0000","last_active":"2024-09-16T15:52:19.098813+0000","last_peered":"2024-09-16T15:52:19.098813+0000","last_clean":"2024-09-16T15:52:19.098813+0000","last_became_active":"2024-09-16T15:52:18.097693+0000","last_became_peered":"2024-09-16T15:52:18.097693+0000","last_unstale":"2024-09-16T15:52:19.098813+0000","last_undegraded":"2024-09-16T15:52:19.098813+0000","last_fullsized":"2024-09-16T15:52:19.098813+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:24:52.156928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.d","version":"117'1","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335394+0000","last_change":"2024-09-16T15:52:17.087628+0000","last_active":"2024-09-16T15:52:19.335394+0000","last_peered":"2024-09-16T15:52:19.335394+0000","last_clean":"2024-09-16T15:52:19.335394+0000","last_became_active":"2024-09-16T15:52:17.087384+0000","last_became_peered":"2024-09-16T15:52:17.087384+0000","last_unstale":"2024-09-16T15:52:19.335394+0000","last_undegraded":"2024-09-16T15:52:19.335394+0000","last_fullsized":"2024-09-16T15:52:19.335394+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:42:11.616130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335328+0000","last_change":"2024-09-16T15:52:18.114650+0000","last_active":"2024-09-16T15:52:19.335328+0000","last_peered":"2024-09-16T15:52:19.335328+0000","last_clean":"2024-09-16T15:52:19.335328+0000","last_became_active":"2024-09-16T15:52:18.111886+0000","last_became_peered":"2024-09-16T15:52:18.111886+0000","last_unstale":"2024-09-16T15:52:19.335328+0000","last_undegraded":"2024-09-16T15:52:19.335328+0000","last_fullsized":"2024-09-16T15:52:19.335328+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:27:33.482617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335048+0000","last_change":"2024-09-16T15:52:17.090919+0000","last_active":"2024-09-16T15:52:19.335048+0000","last_peered":"2024-09-16T15:52:19.335048+0000","last_clean":"2024-09-16T15:52:19.335048+0000","last_became_active":"2024-09-16T15:52:17.090639+0000","last_became_peered":"2024-09-16T15:52:17.090639+0000","last_unstale":"2024-09-16T15:52:19.335048+0000","last_undegraded":"2024-09-16T15:52:19.335048+0000","last_fullsized":"2024-09-16T15:52:19.335048+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:02:59.981560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.095970+0000","last_change":"2024-09-16T15:52:18.099682+0000","last_active":"2024-09-16T15:52:19.095970+0000","last_peered":"2024-09-16T15:52:19.095970+0000","last_clean":"2024-09-16T15:52:19.095970+0000","last_became_active":"2024-09-16T15:52:18.093247+0000","last_became_peered":"2024-09-16T15:52:18.093247+0000","last_unstale":"2024-09-16T15:52:19.095970+0000","last_undegraded":"2024-09-16T15:52:19.095970+0000","last_fullsized":"2024-09-16T15:52:19.095970+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:39:08.375481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.b","version":"117'2","reported_seq":17,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336094+0000","last_change":"2024-09-16T15:52:17.086366+0000","last_active":"2024-09-16T15:52:19.336094+0000","last_peered":"2024-09-16T15:52:19.336094+0000","last_clean":"2024-09-16T15:52:19.336094+0000","last_became_active":"2024-09-16T15:52:17.085765+0000","last_became_peered":"2024-09-16T15:52:17.085765+0000","last_unstale":"2024-09-16T15:52:19.336094+0000","last_undegraded":"2024-09-16T15:52:19.336094+0000","last_fullsized":"2024-09-16T15:52:19.336094+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:32:36.741556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-16T15:52:17.076139+0000","last_change":"2024-09-16T15:52:17.076139+0000","last_active":"2024-09-16T15:52:17.058582+0000","last_peered":"2024-09-16T15:52:17.058582+0000","last_clean":"2024-09-16T15:52:17.058582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:52:17.076139+0000","last_undegraded":"2024-09-16T15:52:17.076139+0000","last_fullsized":"2024-09-16T15:52:17.076139+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-16T15:52:17.094304+0000","last_change":"2024-09-16T15:52:17.090433+0000","last_active":"2024-09-16T15:52:17.094304+0000","last_peered":"2024-09-16T15:52:17.094304+0000","last_clean":"2024-09-16T15:52:17.094304+0000","last_became_active":"2024-09-16T15:52:17.089911+0000","last_became_peered":"2024-09-16T15:52:17.089911+0000","last_unstale":"2024-09-16T15:52:17.094304+0000","last_undegraded":"2024-09-16T15:52:17.094304+0000","last_fullsized":"2024-09-16T15:52:17.094304+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:29:31.775552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.334976+0000","last_change":"2024-09-16T15:52:18.102029+0000","last_active":"2024-09-16T15:52:19.334976+0000","last_peered":"2024-09-16T15:52:19.334976+0000","last_clean":"2024-09-16T15:52:19.334976+0000","last_became_active":"2024-09-16T15:52:18.101193+0000","last_became_peered":"2024-09-16T15:52:18.101193+0000","last_unstale":"2024-09-16T15:52:19.334976+0000","last_undegraded":"2024-09-16T15:52:19.334976+0000","last_fullsized":"2024-09-16T15:52:19.334976+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:14:39.539437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336623+0000","last_change":"2024-09-16T15:52:17.089371+0000","last_active":"2024-09-16T15:52:19.336623+0000","last_peered":"2024-09-16T15:52:19.336623+0000","last_clean":"2024-09-16T15:52:19.336623+0000","last_became_active":"2024-09-16T15:52:17.089099+0000","last_became_peered":"2024-09-16T15:52:17.089099+0000","last_unstale":"2024-09-16T15:52:19.336623+0000","last_undegraded":"2024-09-16T15:52:19.336623+0000","last_fullsized":"2024-09-16T15:52:19.336623+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:04:46.705752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335539+0000","last_change":"2024-09-16T15:52:18.106035+0000","last_active":"2024-09-16T15:52:19.335539+0000","last_peered":"2024-09-16T15:52:19.335539+0000","last_clean":"2024-09-16T15:52:19.335539+0000","last_became_active":"2024-09-16T15:52:18.105771+0000","last_became_peered":"2024-09-16T15:52:18.105771+0000","last_unstale":"2024-09-16T15:52:19.335539+0000","last_undegraded":"2024-09-16T15:52:19.335539+0000","last_fullsized":"2024-09-16T15:52:19.335539+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:07:04.881864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336440+0000","last_change":"2024-09-16T15:52:17.078927+0000","last_active":"2024-09-16T15:52:19.336440+0000","last_peered":"2024-09-16T15:52:19.336440+0000","last_clean":"2024-09-16T15:52:19.336440+0000","last_became_active":"2024-09-16T15:52:17.078623+0000","last_became_peered":"2024-09-16T15:52:17.078623+0000","last_unstale":"2024-09-16T15:52:19.336440+0000","last_undegraded":"2024-09-16T15:52:19.336440+0000","last_fullsized":"2024-09-16T15:52:19.336440+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:32:34.228931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335362+0000","last_change":"2024-09-16T15:52:18.099029+0000","last_active":"2024-09-16T15:52:19.335362+0000","last_peered":"2024-09-16T15:52:19.335362+0000","last_clean":"2024-09-16T15:52:19.335362+0000","last_became_active":"2024-09-16T15:52:18.098767+0000","last_became_peered":"2024-09-16T15:52:18.098767+0000","last_unstale":"2024-09-16T15:52:19.335362+0000","last_undegraded":"2024-09-16T15:52:19.335362+0000","last_fullsized":"2024-09-16T15:52:19.335362+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:22:20.108931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.7","version":"117'2","reported_seq":17,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336898+0000","last_change":"2024-09-16T15:52:17.083225+0000","last_active":"2024-09-16T15:52:19.336898+0000","last_peered":"2024-09-16T15:52:19.336898+0000","last_clean":"2024-09-16T15:52:19.336898+0000","last_became_active":"2024-09-16T15:52:17.082946+0000","last_became_peered":"2024-09-16T15:52:17.082946+0000","last_unstale":"2024-09-16T15:52:19.336898+0000","last_undegraded":"2024-09-16T15:52:19.336898+0000","last_fullsized":"2024-09-16T15:52:19.336898+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:53:28.145727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.097472+0000","last_change":"2024-09-16T15:52:18.096158+0000","last_active":"2024-09-16T15:52:19.097472+0000","last_peered":"2024-09-16T15:52:19.097472+0000","last_clean":"2024-09-16T15:52:19.097472+0000","last_became_active":"2024-09-16T15:52:18.095854+0000","last_became_peered":"2024-09-16T15:52:18.095854+0000","last_unstale":"2024-09-16T15:52:19.097472+0000","last_undegraded":"2024-09-16T15:52:19.097472+0000","last_fullsized":"2024-09-16T15:52:19.097472+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:19:03.220760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"118'1","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098686+0000","last_change":"2024-09-16T15:52:17.091331+0000","last_active":"2024-09-16T15:52:19.098686+0000","last_peered":"2024-09-16T15:52:19.098686+0000","last_clean":"2024-09-16T15:52:19.098686+0000","last_became_active":"2024-09-16T15:52:17.091071+0000","last_became_peered":"2024-09-16T15:52:17.091071+0000","last_unstale":"2024-09-16T15:52:19.098686+0000","last_undegraded":"2024-09-16T15:52:19.098686+0000","last_fullsized":"2024-09-16T15:52:19.098686+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:18:38.561424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":1,"reported_epoch":116,"state":"creating+peering","last_fresh":"2024-09-16T15:52:17.077848+0000","last_change":"2024-09-16T15:52:17.077848+0000","last_active":"2024-09-16T15:52:17.058582+0000","last_peered":"2024-09-16T15:52:17.058582+0000","last_clean":"2024-09-16T15:52:17.058582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:52:17.077848+0000","last_undegraded":"2024-09-16T15:52:17.077848+0000","last_fullsized":"2024-09-16T15:52:17.077848+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336336+0000","last_change":"2024-09-16T15:52:17.080632+0000","last_active":"2024-09-16T15:52:19.336336+0000","last_peered":"2024-09-16T15:52:19.336336+0000","last_clean":"2024-09-16T15:52:19.336336+0000","last_became_active":"2024-09-16T15:52:17.079770+0000","last_became_peered":"2024-09-16T15:52:17.079770+0000","last_unstale":"2024-09-16T15:52:19.336336+0000","last_undegraded":"2024-09-16T15:52:19.336336+0000","last_fullsized":"2024-09-16T15:52:19.336336+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:18:23.166654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335328+0000","last_change":"2024-09-16T15:52:18.099378+0000","last_active":"2024-09-16T15:52:19.335328+0000","last_peered":"2024-09-16T15:52:19.335328+0000","last_clean":"2024-09-16T15:52:19.335328+0000","last_became_active":"2024-09-16T15:52:18.096154+0000","last_became_peered":"2024-09-16T15:52:18.096154+0000","last_unstale":"2024-09-16T15:52:19.335328+0000","last_undegraded":"2024-09-16T15:52:19.335328+0000","last_fullsized":"2024-09-16T15:52:19.335328+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:42:01.895815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336463+0000","last_change":"2024-09-16T15:52:17.091377+0000","last_active":"2024-09-16T15:52:19.336463+0000","last_peered":"2024-09-16T15:52:19.336463+0000","last_clean":"2024-09-16T15:52:19.336463+0000","last_became_active":"2024-09-16T15:52:17.090957+0000","last_became_peered":"2024-09-16T15:52:17.090957+0000","last_unstale":"2024-09-16T15:52:19.336463+0000","last_undegraded":"2024-09-16T15:52:19.336463+0000","last_fullsized":"2024-09-16T15:52:19.336463+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:21:10.990720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.095396+0000","last_change":"2024-09-16T15:52:18.107499+0000","last_active":"2024-09-16T15:52:19.095396+0000","last_peered":"2024-09-16T15:52:19.095396+0000","last_clean":"2024-09-16T15:52:19.095396+0000","last_became_active":"2024-09-16T15:52:18.105395+0000","last_became_peered":"2024-09-16T15:52:18.105395+0000","last_unstale":"2024-09-16T15:52:19.095396+0000","last_undegraded":"2024-09-16T15:52:19.095396+0000","last_fullsized":"2024-09-16T15:52:19.095396+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:15:42.167943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098838+0000","last_change":"2024-09-16T15:52:17.088515+0000","last_active":"2024-09-16T15:52:19.098838+0000","last_peered":"2024-09-16T15:52:19.098838+0000","last_clean":"2024-09-16T15:52:19.098838+0000","last_became_active":"2024-09-16T15:52:17.088093+0000","last_became_peered":"2024-09-16T15:52:17.088093+0000","last_unstale":"2024-09-16T15:52:19.098838+0000","last_undegraded":"2024-09-16T15:52:19.098838+0000","last_fullsized":"2024-09-16T15:52:19.098838+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:41:51.353892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335174+0000","last_change":"2024-09-16T15:52:18.104181+0000","last_active":"2024-09-16T15:52:19.335174+0000","last_peered":"2024-09-16T15:52:19.335174+0000","last_clean":"2024-09-16T15:52:19.335174+0000","last_became_active":"2024-09-16T15:52:18.096537+0000","last_became_peered":"2024-09-16T15:52:18.096537+0000","last_unstale":"2024-09-16T15:52:19.335174+0000","last_undegraded":"2024-09-16T15:52:19.335174+0000","last_fullsized":"2024-09-16T15:52:19.335174+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:53:28.863711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.334882+0000","last_change":"2024-09-16T15:52:17.087423+0000","last_active":"2024-09-16T15:52:19.334882+0000","last_peered":"2024-09-16T15:52:19.334882+0000","last_clean":"2024-09-16T15:52:19.334882+0000","last_became_active":"2024-09-16T15:52:17.087145+0000","last_became_peered":"2024-09-16T15:52:17.087145+0000","last_unstale":"2024-09-16T15:52:19.334882+0000","last_undegraded":"2024-09-16T15:52:19.334882+0000","last_fullsized":"2024-09-16T15:52:19.334882+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:09:02.776672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335017+0000","last_change":"2024-09-16T15:52:18.095897+0000","last_active":"2024-09-16T15:52:19.335017+0000","last_peered":"2024-09-16T15:52:19.335017+0000","last_clean":"2024-09-16T15:52:19.335017+0000","last_became_active":"2024-09-16T15:52:18.095588+0000","last_became_peered":"2024-09-16T15:52:18.095588+0000","last_unstale":"2024-09-16T15:52:19.335017+0000","last_undegraded":"2024-09-16T15:52:19.335017+0000","last_fullsized":"2024-09-16T15:52:19.335017+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:28:29.259290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-16T15:52:17.089987+0000","last_change":"2024-09-16T15:52:17.085524+0000","last_active":"2024-09-16T15:52:17.089987+0000","last_peered":"2024-09-16T15:52:17.089987+0000","last_clean":"2024-09-16T15:52:17.089987+0000","last_became_active":"2024-09-16T15:52:17.084921+0000","last_became_peered":"2024-09-16T15:52:17.084921+0000","last_unstale":"2024-09-16T15:52:17.089987+0000","last_undegraded":"2024-09-16T15:52:17.089987+0000","last_fullsized":"2024-09-16T15:52:17.089987+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:11:21.294316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.096777+0000","last_change":"2024-09-16T15:52:18.106049+0000","last_active":"2024-09-16T15:52:19.096777+0000","last_peered":"2024-09-16T15:52:19.096777+0000","last_clean":"2024-09-16T15:52:19.096777+0000","last_became_active":"2024-09-16T15:52:18.105352+0000","last_became_peered":"2024-09-16T15:52:18.105352+0000","last_unstale":"2024-09-16T15:52:19.096777+0000","last_undegraded":"2024-09-16T15:52:19.096777+0000","last_fullsized":"2024-09-16T15:52:19.096777+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:29:18.998165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.096570+0000","last_change":"2024-09-16T15:52:17.079490+0000","last_active":"2024-09-16T15:52:19.096570+0000","last_peered":"2024-09-16T15:52:19.096570+0000","last_clean":"2024-09-16T15:52:19.096570+0000","last_became_active":"2024-09-16T15:52:17.079148+0000","last_became_peered":"2024-09-16T15:52:17.079148+0000","last_unstale":"2024-09-16T15:52:19.096570+0000","last_undegraded":"2024-09-16T15:52:19.096570+0000","last_fullsized":"2024-09-16T15:52:19.096570+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:17:11.750051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336381+0000","last_change":"2024-09-16T15:52:18.087311+0000","last_active":"2024-09-16T15:52:19.336381+0000","last_peered":"2024-09-16T15:52:19.336381+0000","last_clean":"2024-09-16T15:52:19.336381+0000","last_became_active":"2024-09-16T15:52:18.086640+0000","last_became_peered":"2024-09-16T15:52:18.086640+0000","last_unstale":"2024-09-16T15:52:19.336381+0000","last_undegraded":"2024-09-16T15:52:19.336381+0000","last_fullsized":"2024-09-16T15:52:19.336381+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:12:10.637529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.324319+0000","last_change":"2024-09-16T15:52:17.072908+0000","last_active":"2024-09-16T15:52:19.324319+0000","last_peered":"2024-09-16T15:52:19.324319+0000","last_clean":"2024-09-16T15:52:19.324319+0000","last_became_active":"2024-09-16T15:52:17.072645+0000","last_became_peered":"2024-09-16T15:52:17.072645+0000","last_unstale":"2024-09-16T15:52:19.324319+0000","last_undegraded":"2024-09-16T15:52:19.324319+0000","last_fullsized":"2024-09-16T15:52:19.324319+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:26:34.557572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":293,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.095214+0000","last_change":"2024-09-16T15:39:12.698813+0000","last_active":"2024-09-16T15:52:19.095214+0000","last_peered":"2024-09-16T15:52:19.095214+0000","last_clean":"2024-09-16T15:52:19.095214+0000","last_became_active":"2024-09-16T15:39:12.697695+0000","last_became_peered":"2024-09-16T15:39:12.697695+0000","last_unstale":"2024-09-16T15:52:19.095214+0000","last_undegraded":"2024-09-16T15:52:19.095214+0000","last_fullsized":"2024-09-16T15:52:19.095214+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_clean_scrub_stamp":"2024-09-16T15:39:11.684967+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:02.061081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":30,"ondisk_log_size":30,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509602,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":136652,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94235188,"statfs":{"total":96636764160,"available":96496832512,"internally_reserved":0,"allocated":1433600,"data_stored":627651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":138475976},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509602,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":135864,"kb_used_data":2012,"kb_used_omap":1,"kb_used_meta":133822,"kb_avail":94235976,"statfs":{"total":96636764160,"available":96497639424,"internally_reserved":0,"allocated":2060288,"data_stored":1231354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":137034183},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509601,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149172,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222668,"statfs":{"total":96636764160,"available":96484012032,"internally_reserved":0,"allocated":1474560,"data_stored":642596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509601,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131112,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240728,"statfs":{"total":96636764160,"available":96502505472,"internally_reserved":0,"allocated":1458176,"data_stored":640894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":132774345},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509601,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126968,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244872,"statfs":{"total":96636764160,"available":96506748928,"internally_reserved":0,"allocated":1470464,"data_stored":641532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":128514503},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509602,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127536,"kb_used_data":2004,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244304,"statfs":{"total":96636764160,"available":96506167296,"internally_reserved":0,"allocated":2052096,"data_stored":1231262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":128514503},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509600,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145012,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226828,"statfs":{"total":96636764160,"available":96488271872,"internally_reserved":0,"allocated":1466368,"data_stored":641546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":146995655},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542307,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126972,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244868,"statfs":{"total":96636764160,"available":96506744832,"internally_reserved":0,"allocated":1478656,"data_stored":642630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":128514508},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:52:23.181 INFO:tasks.ceph.ceph_manager.ceph:PG 17.3f is not active+clean 2024-09-16T15:52:23.182 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:52:17.079031+0000', 'last_change': '2024-09-16T15:52:17.079031+0000', 'last_active': '2024-09-16T15:52:17.058582+0000', 'last_peered': '2024-09-16T15:52:17.058582+0000', 'last_clean': '2024-09-16T15:52:17.058582+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:52:17.079031+0000', 'last_undegraded': '2024-09-16T15:52:17.079031+0000', 'last_fullsized': '2024-09-16T15:52:17.079031+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:52:17.058582+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:52:17.058582+0000', 'last_clean_scrub_stamp': '2024-09-16T15:52:17.058582+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-16T15:52:23.182 INFO:tasks.ceph.ceph_manager.ceph:PG 17.37 is not active+clean 2024-09-16T15:52:23.182 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:52:17.076806+0000', 'last_change': '2024-09-16T15:52:17.076806+0000', 'last_active': '2024-09-16T15:52:17.058582+0000', 'last_peered': '2024-09-16T15:52:17.058582+0000', 'last_clean': '2024-09-16T15:52:17.058582+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:52:17.076806+0000', 'last_undegraded': '2024-09-16T15:52:17.076806+0000', 'last_fullsized': '2024-09-16T15:52:17.076806+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:52:17.058582+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:52:17.058582+0000', 'last_clean_scrub_stamp': '2024-09-16T15:52:17.058582+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-16T15:52:23.182 INFO:tasks.ceph.ceph_manager.ceph:PG 17.31 is not active+clean 2024-09-16T15:52:23.182 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:52:17.076793+0000', 'last_change': '2024-09-16T15:52:17.076793+0000', 'last_active': '2024-09-16T15:52:17.058582+0000', 'last_peered': '2024-09-16T15:52:17.058582+0000', 'last_clean': '2024-09-16T15:52:17.058582+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:52:17.076793+0000', 'last_undegraded': '2024-09-16T15:52:17.076793+0000', 'last_fullsized': '2024-09-16T15:52:17.076793+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:52:17.058582+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:52:17.058582+0000', 'last_clean_scrub_stamp': '2024-09-16T15:52:17.058582+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-16T15:52:23.182 INFO:tasks.ceph.ceph_manager.ceph:PG 17.21 is not active+clean 2024-09-16T15:52:23.182 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:52:17.076332+0000', 'last_change': '2024-09-16T15:52:17.076332+0000', 'last_active': '2024-09-16T15:52:17.058582+0000', 'last_peered': '2024-09-16T15:52:17.058582+0000', 'last_clean': '2024-09-16T15:52:17.058582+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:52:17.076332+0000', 'last_undegraded': '2024-09-16T15:52:17.076332+0000', 'last_fullsized': '2024-09-16T15:52:17.076332+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:52:17.058582+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:52:17.058582+0000', 'last_clean_scrub_stamp': '2024-09-16T15:52:17.058582+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-16T15:52:23.182 INFO:tasks.ceph.ceph_manager.ceph:PG 17.1e is not active+clean 2024-09-16T15:52:23.183 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:52:17.081705+0000', 'last_change': '2024-09-16T15:52:17.081705+0000', 'last_active': '2024-09-16T15:52:17.058582+0000', 'last_peered': '2024-09-16T15:52:17.058582+0000', 'last_clean': '2024-09-16T15:52:17.058582+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:52:17.081705+0000', 'last_undegraded': '2024-09-16T15:52:17.081705+0000', 'last_fullsized': '2024-09-16T15:52:17.081705+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:52:17.058582+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:52:17.058582+0000', 'last_clean_scrub_stamp': '2024-09-16T15:52:17.058582+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-16T15:52:23.183 INFO:tasks.ceph.ceph_manager.ceph:PG 17.e is not active+clean 2024-09-16T15:52:23.183 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:52:17.076690+0000', 'last_change': '2024-09-16T15:52:17.076690+0000', 'last_active': '2024-09-16T15:52:17.058582+0000', 'last_peered': '2024-09-16T15:52:17.058582+0000', 'last_clean': '2024-09-16T15:52:17.058582+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:52:17.076690+0000', 'last_undegraded': '2024-09-16T15:52:17.076690+0000', 'last_fullsized': '2024-09-16T15:52:17.076690+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:52:17.058582+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:52:17.058582+0000', 'last_clean_scrub_stamp': '2024-09-16T15:52:17.058582+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-16T15:52:23.183 INFO:tasks.ceph.ceph_manager.ceph:PG 17.f is not active+clean 2024-09-16T15:52:23.183 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:52:17.075988+0000', 'last_change': '2024-09-16T15:52:17.075988+0000', 'last_active': '2024-09-16T15:52:17.058582+0000', 'last_peered': '2024-09-16T15:52:17.058582+0000', 'last_clean': '2024-09-16T15:52:17.058582+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:52:17.075988+0000', 'last_undegraded': '2024-09-16T15:52:17.075988+0000', 'last_fullsized': '2024-09-16T15:52:17.075988+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:52:17.058582+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:52:17.058582+0000', 'last_clean_scrub_stamp': '2024-09-16T15:52:17.058582+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-16T15:52:23.183 INFO:tasks.ceph.ceph_manager.ceph:PG 17.b is not active+clean 2024-09-16T15:52:23.183 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:52:17.076139+0000', 'last_change': '2024-09-16T15:52:17.076139+0000', 'last_active': '2024-09-16T15:52:17.058582+0000', 'last_peered': '2024-09-16T15:52:17.058582+0000', 'last_clean': '2024-09-16T15:52:17.058582+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:52:17.076139+0000', 'last_undegraded': '2024-09-16T15:52:17.076139+0000', 'last_fullsized': '2024-09-16T15:52:17.076139+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:52:17.058582+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:52:17.058582+0000', 'last_clean_scrub_stamp': '2024-09-16T15:52:17.058582+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-16T15:52:23.183 INFO:tasks.ceph.ceph_manager.ceph:PG 17.4 is not active+clean 2024-09-16T15:52:23.183 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 116, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:52:17.077848+0000', 'last_change': '2024-09-16T15:52:17.077848+0000', 'last_active': '2024-09-16T15:52:17.058582+0000', 'last_peered': '2024-09-16T15:52:17.058582+0000', 'last_clean': '2024-09-16T15:52:17.058582+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:52:17.077848+0000', 'last_undegraded': '2024-09-16T15:52:17.077848+0000', 'last_fullsized': '2024-09-16T15:52:17.077848+0000', 'mapping_epoch': 116, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 116, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:52:17.058582+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:52:17.058582+0000', 'last_clean_scrub_stamp': '2024-09-16T15:52:17.058582+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-16T15:52:23.184 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T15:52:23.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.373+0000 7f448d400640 1 -- 172.21.15.151:0/2329104717 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f448806f220 msgr2=0x7f448806f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:23.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.373+0000 7f448d400640 1 --2- 172.21.15.151:0/2329104717 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f448806f220 0x7f448806f6b0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f4480005fd0 tx=0x7f448002aaa0 comp rx=0 tx=0).stop 2024-09-16T15:52:23.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.373+0000 7f448d400640 1 -- 172.21.15.151:0/2329104717 shutdown_connections 2024-09-16T15:52:23.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.373+0000 7f448d400640 1 --2- 172.21.15.151:0/2329104717 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f448806f220 0x7f448806f6b0 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:23.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.373+0000 7f448d400640 1 -- 172.21.15.151:0/2329104717 >> 172.21.15.151:0/2329104717 conn(0x7f44880b18e0 msgr2=0x7f44880b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:23.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.373+0000 7f448d400640 1 -- 172.21.15.151:0/2329104717 shutdown_connections 2024-09-16T15:52:23.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.373+0000 7f448d400640 1 -- 172.21.15.151:0/2329104717 wait complete. 2024-09-16T15:52:23.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.375+0000 7f448d400640 1 Processor -- start 2024-09-16T15:52:23.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.375+0000 7f448d400640 1 -- start start 2024-09-16T15:52:23.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.376+0000 7f448d400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f448806f220 0x7f448809bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:23.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.376+0000 7f448d400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4488159aa0 0x7f44880ceec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:23.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.376+0000 7f448d400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f448815cff0 0x7f44881f3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:23.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.376+0000 7f448d400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f44880cc790 con 0x7f4488159aa0 2024-09-16T15:52:23.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.376+0000 7f448d400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f44880cc610 con 0x7f448806f220 2024-09-16T15:52:23.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.376+0000 7f448d400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f44880cc910 con 0x7f448815cff0 2024-09-16T15:52:23.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.376+0000 7f4485e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4488159aa0 0x7f44880ceec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:23.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.376+0000 7f4485e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4488159aa0 0x7f44880ceec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41366/0 (socket says 172.21.15.151:41366) 2024-09-16T15:52:23.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.376+0000 7f4485e00640 1 -- 172.21.15.151:0/841963557 learned_addr learned my addr 172.21.15.151:0/841963557 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:52:23.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.376+0000 7f4486800640 1 --2- 172.21.15.151:0/841963557 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f448806f220 0x7f448809bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:23.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.376+0000 7f4487200640 1 --2- 172.21.15.151:0/841963557 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f448815cff0 0x7f44881f3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:23.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.376+0000 7f4485e00640 1 -- 172.21.15.151:0/841963557 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f448815cff0 msgr2=0x7f44881f3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:23.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.376+0000 7f4485e00640 1 --2- 172.21.15.151:0/841963557 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f448815cff0 0x7f44881f3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:23.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.376+0000 7f4485e00640 1 -- 172.21.15.151:0/841963557 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f448806f220 msgr2=0x7f448809bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:23.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.376+0000 7f4485e00640 1 --2- 172.21.15.151:0/841963557 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f448806f220 0x7f448809bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:23.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.376+0000 7f4485e00640 1 -- 172.21.15.151:0/841963557 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44880cf400 con 0x7f4488159aa0 2024-09-16T15:52:23.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.376+0000 7f4487200640 1 --2- 172.21.15.151:0/841963557 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f448815cff0 0x7f44881f3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:52:23.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.377+0000 7f4485e00640 1 --2- 172.21.15.151:0/841963557 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4488159aa0 0x7f44880ceec0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f447400b3f0 tx=0x7f447400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:23.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.377+0000 7f447ea00640 1 -- 172.21.15.151:0/841963557 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4474015020 con 0x7f4488159aa0 2024-09-16T15:52:23.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.377+0000 7f447ea00640 1 -- 172.21.15.151:0/841963557 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4474011920 con 0x7f4488159aa0 2024-09-16T15:52:23.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.377+0000 7f447ea00640 1 -- 172.21.15.151:0/841963557 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4474011d70 con 0x7f4488159aa0 2024-09-16T15:52:23.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.377+0000 7f448d400640 1 -- 172.21.15.151:0/841963557 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f448809ac50 con 0x7f4488159aa0 2024-09-16T15:52:23.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.377+0000 7f448d400640 1 -- 172.21.15.151:0/841963557 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f448809b210 con 0x7f4488159aa0 2024-09-16T15:52:23.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.378+0000 7f448d400640 1 -- 172.21.15.151:0/841963557 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4488069360 con 0x7f4488159aa0 2024-09-16T15:52:23.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.379+0000 7f447ea00640 1 -- 172.21.15.151:0/841963557 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 41) ==== 103684+0+0 (secure 0 0 0) 0x7f4474018020 con 0x7f4488159aa0 2024-09-16T15:52:23.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.381+0000 7f447ea00640 1 --2- 172.21.15.151:0/841963557 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f446807a2c0 0x7f446807c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:23.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.381+0000 7f447ea00640 1 -- 172.21.15.151:0/841963557 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5793+0+0 (secure 0 0 0) 0x7f44740993c0 con 0x7f4488159aa0 2024-09-16T15:52:23.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.381+0000 7f4486800640 1 --2- 172.21.15.151:0/841963557 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f446807a2c0 0x7f446807c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:23.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.382+0000 7f4486800640 1 --2- 172.21.15.151:0/841963557 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f446807a2c0 0x7f446807c780 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f44780017c0 tx=0x7f4478009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:23.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.385+0000 7f447ea00640 1 -- 172.21.15.151:0/841963557 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f447401f030 con 0x7f4488159aa0 2024-09-16T15:52:23.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.641+0000 7f448d400640 1 -- 172.21.15.151:0/841963557 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f4488070980 con 0x7f4488159aa0 2024-09-16T15:52:23.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.641+0000 7f447ea00640 1 -- 172.21.15.151:0/841963557 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1305 (secure 0 0 0) 0x7f4474063490 con 0x7f4488159aa0 2024-09-16T15:52:23.644 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:52:23.644 INFO:teuthology.orchestra.run.smithi151.stdout:{"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":807,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":118,"num_osds":8,"num_up_osds":8,"osd_up_since":1726501150,"num_in_osds":8,"osd_in_since":1726501140,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":88},{"state_name":"creating+peering","count":9}],"num_pgs":97,"num_pools":3,"num_objects":20,"data_bytes":592778,"bytes_used":1105190912,"bytes_avail":771988922368,"bytes_total":773094113280,"inactive_pgs_ratio":0.092783503234386444,"write_bytes_sec":2455,"read_op_per_sec":0,"write_op_per_sec":5},"fsmap":{"epoch":76,"btime":"2024-09-16T15:52:21:110724+0000","id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"a","status":"up:active","gid":8460}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":16,"modified":"2024-09-16T15:51:59.896138+0000","services":{"mds":{"daemons":{"summary":"","c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-16T15:52:23.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.643+0000 7f448d400640 1 -- 172.21.15.151:0/841963557 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f446807a2c0 msgr2=0x7f446807c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:23.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.643+0000 7f448d400640 1 --2- 172.21.15.151:0/841963557 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f446807a2c0 0x7f446807c780 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f44780017c0 tx=0x7f4478009040 comp rx=0 tx=0).stop 2024-09-16T15:52:23.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.644+0000 7f448d400640 1 -- 172.21.15.151:0/841963557 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4488159aa0 msgr2=0x7f44880ceec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:23.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.644+0000 7f448d400640 1 --2- 172.21.15.151:0/841963557 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4488159aa0 0x7f44880ceec0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f447400b3f0 tx=0x7f447400b8b0 comp rx=0 tx=0).stop 2024-09-16T15:52:23.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.644+0000 7f448d400640 1 -- 172.21.15.151:0/841963557 shutdown_connections 2024-09-16T15:52:23.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.644+0000 7f448d400640 1 --2- 172.21.15.151:0/841963557 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f446807a2c0 0x7f446807c780 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:23.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.644+0000 7f448d400640 1 --2- 172.21.15.151:0/841963557 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f448815cff0 0x7f44881f3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:23.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.644+0000 7f448d400640 1 --2- 172.21.15.151:0/841963557 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4488159aa0 0x7f44880ceec0 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:23.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.644+0000 7f448d400640 1 --2- 172.21.15.151:0/841963557 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f448806f220 0x7f448809bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:23.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.644+0000 7f448d400640 1 -- 172.21.15.151:0/841963557 >> 172.21.15.151:0/841963557 conn(0x7f44880b18e0 msgr2=0x7f44880af980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:23.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.644+0000 7f448d400640 1 -- 172.21.15.151:0/841963557 shutdown_connections 2024-09-16T15:52:23.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.644+0000 7f448d400640 1 -- 172.21.15.151:0/841963557 wait complete. 2024-09-16T15:52:23.661 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-16T15:52:23.661 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:52:23.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.819+0000 7f6680000640 1 -- 172.21.15.151:0/29852218 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6660003a00 msgr2=0x7f6660003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:23.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.819+0000 7f6680000640 1 --2- 172.21.15.151:0/29852218 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6660003a00 0x7f6660003e20 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f666c009fd0 tx=0x7f6660005830 comp rx=0 tx=0).stop 2024-09-16T15:52:23.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.819+0000 7f667e000640 1 -- 172.21.15.151:0/29852218 reap_dead start 2024-09-16T15:52:23.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.819+0000 7f6680000640 1 -- 172.21.15.151:0/29852218 shutdown_connections 2024-09-16T15:52:23.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.819+0000 7f6680000640 1 -- 172.21.15.151:0/29852218 >> 172.21.15.151:0/29852218 conn(0x7f66780b1950 msgr2=0x7f66780b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:23.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.819+0000 7f6680000640 1 -- 172.21.15.151:0/29852218 shutdown_connections 2024-09-16T15:52:23.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.819+0000 7f6680000640 1 -- 172.21.15.151:0/29852218 wait complete. 2024-09-16T15:52:23.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.820+0000 7f6680000640 1 Processor -- start 2024-09-16T15:52:23.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.821+0000 7f6680000640 1 -- start start 2024-09-16T15:52:23.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.821+0000 7f6680000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6660003a00 0x7f66781521e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:23.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.821+0000 7f6680000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f66781f3440 0x7f6678155630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:23.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.821+0000 7f6680000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6678155b70 0x7f66781f5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:23.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.822+0000 7f6680000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f66780b25c0 con 0x7f66781f3440 2024-09-16T15:52:23.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.822+0000 7f6680000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f66780b2440 con 0x7f6660003a00 2024-09-16T15:52:23.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.822+0000 7f6680000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f66780b2740 con 0x7f6678155b70 2024-09-16T15:52:23.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.822+0000 7f667cc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f66781f3440 0x7f6678155630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:23.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.822+0000 7f667cc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f66781f3440 0x7f6678155630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41384/0 (socket says 172.21.15.151:41384) 2024-09-16T15:52:23.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.822+0000 7f667cc00640 1 -- 172.21.15.151:0/95608236 learned_addr learned my addr 172.21.15.151:0/95608236 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:52:23.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.822+0000 7f667e000640 1 --2- 172.21.15.151:0/95608236 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6678155b70 0x7f66781f5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:23.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.822+0000 7f667d600640 1 --2- 172.21.15.151:0/95608236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6660003a00 0x7f66781521e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:23.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.822+0000 7f667cc00640 1 -- 172.21.15.151:0/95608236 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6678155b70 msgr2=0x7f66781f5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:23.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.822+0000 7f667cc00640 1 --2- 172.21.15.151:0/95608236 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6678155b70 0x7f66781f5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:23.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.822+0000 7f667cc00640 1 -- 172.21.15.151:0/95608236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6660003a00 msgr2=0x7f66781521e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:23.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.822+0000 7f667cc00640 1 --2- 172.21.15.151:0/95608236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6660003a00 0x7f66781521e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:23.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.822+0000 7f667cc00640 1 -- 172.21.15.151:0/95608236 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66781f6990 con 0x7f66781f3440 2024-09-16T15:52:23.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.822+0000 7f667e000640 1 --2- 172.21.15.151:0/95608236 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6678155b70 0x7f66781f5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:52:23.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.822+0000 7f667d600640 1 --2- 172.21.15.151:0/95608236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6660003a00 0x7f66781521e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:52:23.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.822+0000 7f667cc00640 1 --2- 172.21.15.151:0/95608236 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f66781f3440 0x7f6678155630 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f6664011b50 tx=0x7f666400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:23.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.823+0000 7f6675600640 1 -- 172.21.15.151:0/95608236 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6664004220 con 0x7f66781f3440 2024-09-16T15:52:23.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.823+0000 7f6675600640 1 -- 172.21.15.151:0/95608236 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f66640043c0 con 0x7f66781f3440 2024-09-16T15:52:23.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.823+0000 7f6680000640 1 -- 172.21.15.151:0/95608236 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66781f2960 con 0x7f66781f3440 2024-09-16T15:52:23.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.823+0000 7f6675600640 1 -- 172.21.15.151:0/95608236 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6664005280 con 0x7f66781f3440 2024-09-16T15:52:23.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.823+0000 7f6680000640 1 -- 172.21.15.151:0/95608236 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6678152720 con 0x7f66781f3440 2024-09-16T15:52:23.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.824+0000 7f6680000640 1 -- 172.21.15.151:0/95608236 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66781f2750 con 0x7f66781f3440 2024-09-16T15:52:23.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.825+0000 7f6675600640 1 -- 172.21.15.151:0/95608236 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 41) ==== 103684+0+0 (secure 0 0 0) 0x7f6664020020 con 0x7f66781f3440 2024-09-16T15:52:23.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.826+0000 7f6675600640 1 --2- 172.21.15.151:0/95608236 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f665807a2c0 0x7f665807c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:23.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.827+0000 7f6675600640 1 -- 172.21.15.151:0/95608236 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5793+0+0 (secure 0 0 0) 0x7f666409a780 con 0x7f66781f3440 2024-09-16T15:52:23.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.827+0000 7f667d600640 1 --2- 172.21.15.151:0/95608236 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f665807a2c0 0x7f665807c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:23.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.828+0000 7f667d600640 1 --2- 172.21.15.151:0/95608236 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f665807a2c0 0x7f665807c780 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f666c034000 tx=0x7f666c004a40 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:23.833 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:23.831+0000 7f6675600640 1 -- 172.21.15.151:0/95608236 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6664064850 con 0x7f66781f3440 2024-09-16T15:52:24.017 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:24.015+0000 7f6680000640 1 -- 172.21.15.151:0/95608236 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f66780acd30 con 0x7f665807a2c0 2024-09-16T15:52:24.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:24.020+0000 7f6675600640 1 -- 172.21.15.151:0/95608236 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252495 (secure 0 0 0) 0x7f66780acd30 con 0x7f665807a2c0 2024-09-16T15:52:24.025 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:52:24.026 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-09-16T15:52:24.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:24.024+0000 7f6680000640 1 -- 172.21.15.151:0/95608236 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f665807a2c0 msgr2=0x7f665807c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:24.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:24.024+0000 7f6680000640 1 --2- 172.21.15.151:0/95608236 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f665807a2c0 0x7f665807c780 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f666c034000 tx=0x7f666c004a40 comp rx=0 tx=0).stop 2024-09-16T15:52:24.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:24.024+0000 7f6680000640 1 -- 172.21.15.151:0/95608236 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f66781f3440 msgr2=0x7f6678155630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:24.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:24.024+0000 7f6680000640 1 --2- 172.21.15.151:0/95608236 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f66781f3440 0x7f6678155630 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f6664011b50 tx=0x7f666400f790 comp rx=0 tx=0).stop 2024-09-16T15:52:24.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:24.024+0000 7f6680000640 1 -- 172.21.15.151:0/95608236 shutdown_connections 2024-09-16T15:52:24.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:24.024+0000 7f6680000640 1 --2- 172.21.15.151:0/95608236 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f665807a2c0 0x7f665807c780 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:24.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:24.024+0000 7f6680000640 1 --2- 172.21.15.151:0/95608236 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6678155b70 0x7f66781f5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:24.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:24.024+0000 7f6680000640 1 --2- 172.21.15.151:0/95608236 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f66781f3440 0x7f6678155630 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:24.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:24.024+0000 7f6680000640 1 --2- 172.21.15.151:0/95608236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6660003a00 0x7f66781521e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:24.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:24.024+0000 7f6680000640 1 -- 172.21.15.151:0/95608236 >> 172.21.15.151:0/95608236 conn(0x7f66780b1950 msgr2=0x7f667806f320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:24.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:24.024+0000 7f6680000640 1 -- 172.21.15.151:0/95608236 shutdown_connections 2024-09-16T15:52:24.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:24.024+0000 7f6680000640 1 -- 172.21.15.151:0/95608236 wait complete. 2024-09-16T15:52:24.043 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":508,"stamp":"2024-09-16T15:52:23.902665+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":187,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1079320,"kb_used_data":12616,"kb_used_omap":12,"kb_used_meta":1066483,"kb_avail":753895400,"statfs":{"total":773094113280,"available":771988889600,"internally_reserved":0,"allocated":12918784,"data_stored":6312708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12738,"internal_metadata":1092079166},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.836172"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098680+0000","last_change":"2024-09-16T15:52:18.113147+0000","last_active":"2024-09-16T15:52:19.098680+0000","last_peered":"2024-09-16T15:52:19.098680+0000","last_clean":"2024-09-16T15:52:19.098680+0000","last_became_active":"2024-09-16T15:52:18.112860+0000","last_became_peered":"2024-09-16T15:52:18.112860+0000","last_unstale":"2024-09-16T15:52:19.098680+0000","last_undegraded":"2024-09-16T15:52:19.098680+0000","last_fullsized":"2024-09-16T15:52:19.098680+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:50:42.139677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.324514+0000","last_change":"2024-09-16T15:52:18.095601+0000","last_active":"2024-09-16T15:52:19.324514+0000","last_peered":"2024-09-16T15:52:19.324514+0000","last_clean":"2024-09-16T15:52:19.324514+0000","last_became_active":"2024-09-16T15:52:18.095305+0000","last_became_peered":"2024-09-16T15:52:18.095305+0000","last_unstale":"2024-09-16T15:52:19.324514+0000","last_undegraded":"2024-09-16T15:52:19.324514+0000","last_fullsized":"2024-09-16T15:52:19.324514+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:46:48.807361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.095580+0000","last_change":"2024-09-16T15:52:18.108749+0000","last_active":"2024-09-16T15:52:19.095580+0000","last_peered":"2024-09-16T15:52:19.095580+0000","last_clean":"2024-09-16T15:52:19.095580+0000","last_became_active":"2024-09-16T15:52:18.104818+0000","last_became_peered":"2024-09-16T15:52:18.104818+0000","last_unstale":"2024-09-16T15:52:19.095580+0000","last_undegraded":"2024-09-16T15:52:19.095580+0000","last_fullsized":"2024-09-16T15:52:19.095580+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:32:30.003302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336245+0000","last_change":"2024-09-16T15:52:18.090782+0000","last_active":"2024-09-16T15:52:19.336245+0000","last_peered":"2024-09-16T15:52:19.336245+0000","last_clean":"2024-09-16T15:52:19.336245+0000","last_became_active":"2024-09-16T15:52:18.090432+0000","last_became_peered":"2024-09-16T15:52:18.090432+0000","last_unstale":"2024-09-16T15:52:19.336245+0000","last_undegraded":"2024-09-16T15:52:19.336245+0000","last_fullsized":"2024-09-16T15:52:19.336245+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:48:32.423354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335861+0000","last_change":"2024-09-16T15:52:18.109873+0000","last_active":"2024-09-16T15:52:19.335861+0000","last_peered":"2024-09-16T15:52:19.335861+0000","last_clean":"2024-09-16T15:52:19.335861+0000","last_became_active":"2024-09-16T15:52:18.101073+0000","last_became_peered":"2024-09-16T15:52:18.101073+0000","last_unstale":"2024-09-16T15:52:19.335861+0000","last_undegraded":"2024-09-16T15:52:19.335861+0000","last_fullsized":"2024-09-16T15:52:19.335861+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:56:57.514683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.097692+0000","last_change":"2024-09-16T15:52:18.096166+0000","last_active":"2024-09-16T15:52:19.097692+0000","last_peered":"2024-09-16T15:52:19.097692+0000","last_clean":"2024-09-16T15:52:19.097692+0000","last_became_active":"2024-09-16T15:52:18.095913+0000","last_became_peered":"2024-09-16T15:52:18.095913+0000","last_unstale":"2024-09-16T15:52:19.097692+0000","last_undegraded":"2024-09-16T15:52:19.097692+0000","last_fullsized":"2024-09-16T15:52:19.097692+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:51:43.908577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098468+0000","last_change":"2024-09-16T15:52:18.111297+0000","last_active":"2024-09-16T15:52:19.098468+0000","last_peered":"2024-09-16T15:52:19.098468+0000","last_clean":"2024-09-16T15:52:19.098468+0000","last_became_active":"2024-09-16T15:52:18.111039+0000","last_became_peered":"2024-09-16T15:52:18.111039+0000","last_unstale":"2024-09-16T15:52:19.098468+0000","last_undegraded":"2024-09-16T15:52:19.098468+0000","last_fullsized":"2024-09-16T15:52:19.098468+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:08:05.616283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098459+0000","last_change":"2024-09-16T15:52:18.094809+0000","last_active":"2024-09-16T15:52:19.098459+0000","last_peered":"2024-09-16T15:52:19.098459+0000","last_clean":"2024-09-16T15:52:19.098459+0000","last_became_active":"2024-09-16T15:52:18.094538+0000","last_became_peered":"2024-09-16T15:52:18.094538+0000","last_unstale":"2024-09-16T15:52:19.098459+0000","last_undegraded":"2024-09-16T15:52:19.098459+0000","last_fullsized":"2024-09-16T15:52:19.098459+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:02:37.475979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098684+0000","last_change":"2024-09-16T15:52:18.112140+0000","last_active":"2024-09-16T15:52:19.098684+0000","last_peered":"2024-09-16T15:52:19.098684+0000","last_clean":"2024-09-16T15:52:19.098684+0000","last_became_active":"2024-09-16T15:52:18.111020+0000","last_became_peered":"2024-09-16T15:52:18.111020+0000","last_unstale":"2024-09-16T15:52:19.098684+0000","last_undegraded":"2024-09-16T15:52:19.098684+0000","last_fullsized":"2024-09-16T15:52:19.098684+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:37:17.979172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336431+0000","last_change":"2024-09-16T15:52:18.101787+0000","last_active":"2024-09-16T15:52:19.336431+0000","last_peered":"2024-09-16T15:52:19.336431+0000","last_clean":"2024-09-16T15:52:19.336431+0000","last_became_active":"2024-09-16T15:52:18.100961+0000","last_became_peered":"2024-09-16T15:52:18.100961+0000","last_unstale":"2024-09-16T15:52:19.336431+0000","last_undegraded":"2024-09-16T15:52:19.336431+0000","last_fullsized":"2024-09-16T15:52:19.336431+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:31:15.594998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098664+0000","last_change":"2024-09-16T15:52:18.094110+0000","last_active":"2024-09-16T15:52:19.098664+0000","last_peered":"2024-09-16T15:52:19.098664+0000","last_clean":"2024-09-16T15:52:19.098664+0000","last_became_active":"2024-09-16T15:52:18.093858+0000","last_became_peered":"2024-09-16T15:52:18.093858+0000","last_unstale":"2024-09-16T15:52:19.098664+0000","last_undegraded":"2024-09-16T15:52:19.098664+0000","last_fullsized":"2024-09-16T15:52:19.098664+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:32:24.437229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335879+0000","last_change":"2024-09-16T15:52:18.102228+0000","last_active":"2024-09-16T15:52:19.335879+0000","last_peered":"2024-09-16T15:52:19.335879+0000","last_clean":"2024-09-16T15:52:19.335879+0000","last_became_active":"2024-09-16T15:52:18.101917+0000","last_became_peered":"2024-09-16T15:52:18.101917+0000","last_unstale":"2024-09-16T15:52:19.335879+0000","last_undegraded":"2024-09-16T15:52:19.335879+0000","last_fullsized":"2024-09-16T15:52:19.335879+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:49:20.323885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.324528+0000","last_change":"2024-09-16T15:52:18.093067+0000","last_active":"2024-09-16T15:52:19.324528+0000","last_peered":"2024-09-16T15:52:19.324528+0000","last_clean":"2024-09-16T15:52:19.324528+0000","last_became_active":"2024-09-16T15:52:18.092805+0000","last_became_peered":"2024-09-16T15:52:18.092805+0000","last_unstale":"2024-09-16T15:52:19.324528+0000","last_undegraded":"2024-09-16T15:52:19.324528+0000","last_fullsized":"2024-09-16T15:52:19.324528+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:06:51.783453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.334985+0000","last_change":"2024-09-16T15:52:18.098148+0000","last_active":"2024-09-16T15:52:19.334985+0000","last_peered":"2024-09-16T15:52:19.334985+0000","last_clean":"2024-09-16T15:52:19.334985+0000","last_became_active":"2024-09-16T15:52:18.096388+0000","last_became_peered":"2024-09-16T15:52:18.096388+0000","last_unstale":"2024-09-16T15:52:19.334985+0000","last_undegraded":"2024-09-16T15:52:19.334985+0000","last_fullsized":"2024-09-16T15:52:19.334985+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:59:34.319160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098156+0000","last_change":"2024-09-16T15:52:18.115196+0000","last_active":"2024-09-16T15:52:19.098156+0000","last_peered":"2024-09-16T15:52:19.098156+0000","last_clean":"2024-09-16T15:52:19.098156+0000","last_became_active":"2024-09-16T15:52:18.114878+0000","last_became_peered":"2024-09-16T15:52:18.114878+0000","last_unstale":"2024-09-16T15:52:19.098156+0000","last_undegraded":"2024-09-16T15:52:19.098156+0000","last_fullsized":"2024-09-16T15:52:19.098156+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:55:52.511395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335328+0000","last_change":"2024-09-16T15:52:18.096798+0000","last_active":"2024-09-16T15:52:19.335328+0000","last_peered":"2024-09-16T15:52:19.335328+0000","last_clean":"2024-09-16T15:52:19.335328+0000","last_became_active":"2024-09-16T15:52:18.096396+0000","last_became_peered":"2024-09-16T15:52:18.096396+0000","last_unstale":"2024-09-16T15:52:19.335328+0000","last_undegraded":"2024-09-16T15:52:19.335328+0000","last_fullsized":"2024-09-16T15:52:19.335328+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:17:25.344931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.095682+0000","last_change":"2024-09-16T15:52:18.113939+0000","last_active":"2024-09-16T15:52:19.095682+0000","last_peered":"2024-09-16T15:52:19.095682+0000","last_clean":"2024-09-16T15:52:19.095682+0000","last_became_active":"2024-09-16T15:52:18.113634+0000","last_became_peered":"2024-09-16T15:52:18.113634+0000","last_unstale":"2024-09-16T15:52:19.095682+0000","last_undegraded":"2024-09-16T15:52:19.095682+0000","last_fullsized":"2024-09-16T15:52:19.095682+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:04:19.441890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098743+0000","last_change":"2024-09-16T15:52:18.106010+0000","last_active":"2024-09-16T15:52:19.098743+0000","last_peered":"2024-09-16T15:52:19.098743+0000","last_clean":"2024-09-16T15:52:19.098743+0000","last_became_active":"2024-09-16T15:52:18.105722+0000","last_became_peered":"2024-09-16T15:52:18.105722+0000","last_unstale":"2024-09-16T15:52:19.098743+0000","last_undegraded":"2024-09-16T15:52:19.098743+0000","last_fullsized":"2024-09-16T15:52:19.098743+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:29:30.985879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.095786+0000","last_change":"2024-09-16T15:52:18.108828+0000","last_active":"2024-09-16T15:52:19.095786+0000","last_peered":"2024-09-16T15:52:19.095786+0000","last_clean":"2024-09-16T15:52:19.095786+0000","last_became_active":"2024-09-16T15:52:18.104728+0000","last_became_peered":"2024-09-16T15:52:18.104728+0000","last_unstale":"2024-09-16T15:52:19.095786+0000","last_undegraded":"2024-09-16T15:52:19.095786+0000","last_fullsized":"2024-09-16T15:52:19.095786+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:08:15.649535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.095454+0000","last_change":"2024-09-16T15:52:18.104490+0000","last_active":"2024-09-16T15:52:19.095454+0000","last_peered":"2024-09-16T15:52:19.095454+0000","last_clean":"2024-09-16T15:52:19.095454+0000","last_became_active":"2024-09-16T15:52:18.104176+0000","last_became_peered":"2024-09-16T15:52:18.104176+0000","last_unstale":"2024-09-16T15:52:19.095454+0000","last_undegraded":"2024-09-16T15:52:19.095454+0000","last_fullsized":"2024-09-16T15:52:19.095454+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:38:00.758145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098816+0000","last_change":"2024-09-16T15:52:18.100178+0000","last_active":"2024-09-16T15:52:19.098816+0000","last_peered":"2024-09-16T15:52:19.098816+0000","last_clean":"2024-09-16T15:52:19.098816+0000","last_became_active":"2024-09-16T15:52:18.099483+0000","last_became_peered":"2024-09-16T15:52:18.099483+0000","last_unstale":"2024-09-16T15:52:19.098816+0000","last_undegraded":"2024-09-16T15:52:19.098816+0000","last_fullsized":"2024-09-16T15:52:19.098816+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:16:33.962636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.097266+0000","last_change":"2024-09-16T15:52:18.095116+0000","last_active":"2024-09-16T15:52:19.097266+0000","last_peered":"2024-09-16T15:52:19.097266+0000","last_clean":"2024-09-16T15:52:19.097266+0000","last_became_active":"2024-09-16T15:52:18.090617+0000","last_became_peered":"2024-09-16T15:52:18.090617+0000","last_unstale":"2024-09-16T15:52:19.097266+0000","last_undegraded":"2024-09-16T15:52:19.097266+0000","last_fullsized":"2024-09-16T15:52:19.097266+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:23:03.363317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335071+0000","last_change":"2024-09-16T15:52:18.102959+0000","last_active":"2024-09-16T15:52:19.335071+0000","last_peered":"2024-09-16T15:52:19.335071+0000","last_clean":"2024-09-16T15:52:19.335071+0000","last_became_active":"2024-09-16T15:52:18.102684+0000","last_became_peered":"2024-09-16T15:52:18.102684+0000","last_unstale":"2024-09-16T15:52:19.335071+0000","last_undegraded":"2024-09-16T15:52:19.335071+0000","last_fullsized":"2024-09-16T15:52:19.335071+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:26:35.565582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336817+0000","last_change":"2024-09-16T15:52:18.102292+0000","last_active":"2024-09-16T15:52:19.336817+0000","last_peered":"2024-09-16T15:52:19.336817+0000","last_clean":"2024-09-16T15:52:19.336817+0000","last_became_active":"2024-09-16T15:52:18.102017+0000","last_became_peered":"2024-09-16T15:52:18.102017+0000","last_unstale":"2024-09-16T15:52:19.336817+0000","last_undegraded":"2024-09-16T15:52:19.336817+0000","last_fullsized":"2024-09-16T15:52:19.336817+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:34:23.254431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098723+0000","last_change":"2024-09-16T15:52:18.089811+0000","last_active":"2024-09-16T15:52:19.098723+0000","last_peered":"2024-09-16T15:52:19.098723+0000","last_clean":"2024-09-16T15:52:19.098723+0000","last_became_active":"2024-09-16T15:52:18.089302+0000","last_became_peered":"2024-09-16T15:52:18.089302+0000","last_unstale":"2024-09-16T15:52:19.098723+0000","last_undegraded":"2024-09-16T15:52:19.098723+0000","last_fullsized":"2024-09-16T15:52:19.098723+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:44:21.343778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.097694+0000","last_change":"2024-09-16T15:52:18.107014+0000","last_active":"2024-09-16T15:52:19.097694+0000","last_peered":"2024-09-16T15:52:19.097694+0000","last_clean":"2024-09-16T15:52:19.097694+0000","last_became_active":"2024-09-16T15:52:18.106706+0000","last_became_peered":"2024-09-16T15:52:18.106706+0000","last_unstale":"2024-09-16T15:52:19.097694+0000","last_undegraded":"2024-09-16T15:52:19.097694+0000","last_fullsized":"2024-09-16T15:52:19.097694+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:41:14.841086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336117+0000","last_change":"2024-09-16T15:52:18.102898+0000","last_active":"2024-09-16T15:52:19.336117+0000","last_peered":"2024-09-16T15:52:19.336117+0000","last_clean":"2024-09-16T15:52:19.336117+0000","last_became_active":"2024-09-16T15:52:18.102229+0000","last_became_peered":"2024-09-16T15:52:18.102229+0000","last_unstale":"2024-09-16T15:52:19.336117+0000","last_undegraded":"2024-09-16T15:52:19.336117+0000","last_fullsized":"2024-09-16T15:52:19.336117+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:21:53.345211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.097217+0000","last_change":"2024-09-16T15:52:18.099345+0000","last_active":"2024-09-16T15:52:19.097217+0000","last_peered":"2024-09-16T15:52:19.097217+0000","last_clean":"2024-09-16T15:52:19.097217+0000","last_became_active":"2024-09-16T15:52:18.098646+0000","last_became_peered":"2024-09-16T15:52:18.098646+0000","last_unstale":"2024-09-16T15:52:19.097217+0000","last_undegraded":"2024-09-16T15:52:19.097217+0000","last_fullsized":"2024-09-16T15:52:19.097217+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:28:37.287849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335126+0000","last_change":"2024-09-16T15:52:18.106525+0000","last_active":"2024-09-16T15:52:19.335126+0000","last_peered":"2024-09-16T15:52:19.335126+0000","last_clean":"2024-09-16T15:52:19.335126+0000","last_became_active":"2024-09-16T15:52:18.103778+0000","last_became_peered":"2024-09-16T15:52:18.103778+0000","last_unstale":"2024-09-16T15:52:19.335126+0000","last_undegraded":"2024-09-16T15:52:19.335126+0000","last_fullsized":"2024-09-16T15:52:19.335126+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:46:53.909612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098944+0000","last_change":"2024-09-16T15:52:18.087005+0000","last_active":"2024-09-16T15:52:19.098944+0000","last_peered":"2024-09-16T15:52:19.098944+0000","last_clean":"2024-09-16T15:52:19.098944+0000","last_became_active":"2024-09-16T15:52:18.086463+0000","last_became_peered":"2024-09-16T15:52:18.086463+0000","last_unstale":"2024-09-16T15:52:19.098944+0000","last_undegraded":"2024-09-16T15:52:19.098944+0000","last_fullsized":"2024-09-16T15:52:19.098944+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:23:31.782022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098218+0000","last_change":"2024-09-16T15:52:18.111474+0000","last_active":"2024-09-16T15:52:19.098218+0000","last_peered":"2024-09-16T15:52:19.098218+0000","last_clean":"2024-09-16T15:52:19.098218+0000","last_became_active":"2024-09-16T15:52:18.110658+0000","last_became_peered":"2024-09-16T15:52:18.110658+0000","last_unstale":"2024-09-16T15:52:19.098218+0000","last_undegraded":"2024-09-16T15:52:19.098218+0000","last_fullsized":"2024-09-16T15:52:19.098218+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:06:51.783453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335551+0000","last_change":"2024-09-16T15:52:18.106237+0000","last_active":"2024-09-16T15:52:19.335551+0000","last_peered":"2024-09-16T15:52:19.335551+0000","last_clean":"2024-09-16T15:52:19.335551+0000","last_became_active":"2024-09-16T15:52:18.103396+0000","last_became_peered":"2024-09-16T15:52:18.103396+0000","last_unstale":"2024-09-16T15:52:19.335551+0000","last_undegraded":"2024-09-16T15:52:19.335551+0000","last_fullsized":"2024-09-16T15:52:19.335551+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:35:46.682990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098906+0000","last_change":"2024-09-16T15:52:18.104576+0000","last_active":"2024-09-16T15:52:19.098906+0000","last_peered":"2024-09-16T15:52:19.098906+0000","last_clean":"2024-09-16T15:52:19.098906+0000","last_became_active":"2024-09-16T15:52:18.104278+0000","last_became_peered":"2024-09-16T15:52:18.104278+0000","last_unstale":"2024-09-16T15:52:19.098906+0000","last_undegraded":"2024-09-16T15:52:19.098906+0000","last_fullsized":"2024-09-16T15:52:19.098906+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:55:55.824799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"117'4","reported_seq":19,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335332+0000","last_change":"2024-09-16T15:52:17.086334+0000","last_active":"2024-09-16T15:52:19.335332+0000","last_peered":"2024-09-16T15:52:19.335332+0000","last_clean":"2024-09-16T15:52:19.335332+0000","last_became_active":"2024-09-16T15:52:17.086082+0000","last_became_peered":"2024-09-16T15:52:17.086082+0000","last_unstale":"2024-09-16T15:52:19.335332+0000","last_undegraded":"2024-09-16T15:52:19.335332+0000","last_fullsized":"2024-09-16T15:52:19.335332+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:25:34.552766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335430+0000","last_change":"2024-09-16T15:52:18.102484+0000","last_active":"2024-09-16T15:52:19.335430+0000","last_peered":"2024-09-16T15:52:19.335430+0000","last_clean":"2024-09-16T15:52:19.335430+0000","last_became_active":"2024-09-16T15:52:18.102159+0000","last_became_peered":"2024-09-16T15:52:18.102159+0000","last_unstale":"2024-09-16T15:52:19.335430+0000","last_undegraded":"2024-09-16T15:52:19.335430+0000","last_fullsized":"2024-09-16T15:52:19.335430+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:18:36.837288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1e","version":"117'1","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.324560+0000","last_change":"2024-09-16T15:52:17.082085+0000","last_active":"2024-09-16T15:52:19.324560+0000","last_peered":"2024-09-16T15:52:19.324560+0000","last_clean":"2024-09-16T15:52:19.324560+0000","last_became_active":"2024-09-16T15:52:17.081840+0000","last_became_peered":"2024-09-16T15:52:17.081840+0000","last_unstale":"2024-09-16T15:52:19.324560+0000","last_undegraded":"2024-09-16T15:52:19.324560+0000","last_fullsized":"2024-09-16T15:52:19.324560+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:46:52.901601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.097360+0000","last_change":"2024-09-16T15:52:18.104067+0000","last_active":"2024-09-16T15:52:19.097360+0000","last_peered":"2024-09-16T15:52:19.097360+0000","last_clean":"2024-09-16T15:52:19.097360+0000","last_became_active":"2024-09-16T15:52:18.103764+0000","last_became_peered":"2024-09-16T15:52:18.103764+0000","last_unstale":"2024-09-16T15:52:19.097360+0000","last_undegraded":"2024-09-16T15:52:19.097360+0000","last_fullsized":"2024-09-16T15:52:19.097360+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:51:28.451444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1d","version":"117'2","reported_seq":17,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336225+0000","last_change":"2024-09-16T15:52:17.083135+0000","last_active":"2024-09-16T15:52:19.336225+0000","last_peered":"2024-09-16T15:52:19.336225+0000","last_clean":"2024-09-16T15:52:19.336225+0000","last_became_active":"2024-09-16T15:52:17.082799+0000","last_became_peered":"2024-09-16T15:52:17.082799+0000","last_unstale":"2024-09-16T15:52:19.336225+0000","last_undegraded":"2024-09-16T15:52:19.336225+0000","last_fullsized":"2024-09-16T15:52:19.336225+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:43:05.707621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.095847+0000","last_change":"2024-09-16T15:52:18.109036+0000","last_active":"2024-09-16T15:52:19.095847+0000","last_peered":"2024-09-16T15:52:19.095847+0000","last_clean":"2024-09-16T15:52:19.095847+0000","last_became_active":"2024-09-16T15:52:18.106979+0000","last_became_peered":"2024-09-16T15:52:18.106979+0000","last_unstale":"2024-09-16T15:52:19.095847+0000","last_undegraded":"2024-09-16T15:52:19.095847+0000","last_fullsized":"2024-09-16T15:52:19.095847+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:38:25.952399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"117'2","reported_seq":17,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336324+0000","last_change":"2024-09-16T15:52:17.087728+0000","last_active":"2024-09-16T15:52:19.336324+0000","last_peered":"2024-09-16T15:52:19.336324+0000","last_clean":"2024-09-16T15:52:19.336324+0000","last_became_active":"2024-09-16T15:52:17.087063+0000","last_became_peered":"2024-09-16T15:52:17.087063+0000","last_unstale":"2024-09-16T15:52:19.336324+0000","last_undegraded":"2024-09-16T15:52:19.336324+0000","last_fullsized":"2024-09-16T15:52:19.336324+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:57:32.844856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336678+0000","last_change":"2024-09-16T15:52:18.111393+0000","last_active":"2024-09-16T15:52:19.336678+0000","last_peered":"2024-09-16T15:52:19.336678+0000","last_clean":"2024-09-16T15:52:19.336678+0000","last_became_active":"2024-09-16T15:52:18.111116+0000","last_became_peered":"2024-09-16T15:52:18.111116+0000","last_unstale":"2024-09-16T15:52:19.336678+0000","last_undegraded":"2024-09-16T15:52:19.336678+0000","last_fullsized":"2024-09-16T15:52:19.336678+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:59:34.319160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1b","version":"117'1","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.095273+0000","last_change":"2024-09-16T15:52:17.083411+0000","last_active":"2024-09-16T15:52:19.095273+0000","last_peered":"2024-09-16T15:52:19.095273+0000","last_clean":"2024-09-16T15:52:19.095273+0000","last_became_active":"2024-09-16T15:52:17.083098+0000","last_became_peered":"2024-09-16T15:52:17.083098+0000","last_unstale":"2024-09-16T15:52:19.095273+0000","last_undegraded":"2024-09-16T15:52:19.095273+0000","last_fullsized":"2024-09-16T15:52:19.095273+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:48:17.710440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098564+0000","last_change":"2024-09-16T15:52:18.099617+0000","last_active":"2024-09-16T15:52:19.098564+0000","last_peered":"2024-09-16T15:52:19.098564+0000","last_clean":"2024-09-16T15:52:19.098564+0000","last_became_active":"2024-09-16T15:52:18.099149+0000","last_became_peered":"2024-09-16T15:52:18.099149+0000","last_unstale":"2024-09-16T15:52:19.098564+0000","last_undegraded":"2024-09-16T15:52:19.098564+0000","last_fullsized":"2024-09-16T15:52:19.098564+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:41:19.687707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1a","version":"117'1","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098345+0000","last_change":"2024-09-16T15:52:17.086053+0000","last_active":"2024-09-16T15:52:19.098345+0000","last_peered":"2024-09-16T15:52:19.098345+0000","last_clean":"2024-09-16T15:52:19.098345+0000","last_became_active":"2024-09-16T15:52:17.084186+0000","last_became_peered":"2024-09-16T15:52:17.084186+0000","last_unstale":"2024-09-16T15:52:19.098345+0000","last_undegraded":"2024-09-16T15:52:19.098345+0000","last_fullsized":"2024-09-16T15:52:19.098345+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:21:52.337200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336424+0000","last_change":"2024-09-16T15:52:18.101280+0000","last_active":"2024-09-16T15:52:19.336424+0000","last_peered":"2024-09-16T15:52:19.336424+0000","last_clean":"2024-09-16T15:52:19.336424+0000","last_became_active":"2024-09-16T15:52:18.100938+0000","last_became_peered":"2024-09-16T15:52:18.100938+0000","last_unstale":"2024-09-16T15:52:19.336424+0000","last_undegraded":"2024-09-16T15:52:19.336424+0000","last_fullsized":"2024-09-16T15:52:19.336424+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:18:58.903310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098529+0000","last_change":"2024-09-16T15:52:17.090751+0000","last_active":"2024-09-16T15:52:19.098529+0000","last_peered":"2024-09-16T15:52:19.098529+0000","last_clean":"2024-09-16T15:52:19.098529+0000","last_became_active":"2024-09-16T15:52:17.090390+0000","last_became_peered":"2024-09-16T15:52:17.090390+0000","last_unstale":"2024-09-16T15:52:19.098529+0000","last_undegraded":"2024-09-16T15:52:19.098529+0000","last_fullsized":"2024-09-16T15:52:19.098529+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:58:11.585044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098903+0000","last_change":"2024-09-16T15:52:18.090433+0000","last_active":"2024-09-16T15:52:19.098903+0000","last_peered":"2024-09-16T15:52:19.098903+0000","last_clean":"2024-09-16T15:52:19.098903+0000","last_became_active":"2024-09-16T15:52:18.090169+0000","last_became_peered":"2024-09-16T15:52:18.090169+0000","last_unstale":"2024-09-16T15:52:19.098903+0000","last_undegraded":"2024-09-16T15:52:19.098903+0000","last_fullsized":"2024-09-16T15:52:19.098903+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:20:57.801785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336267+0000","last_change":"2024-09-16T15:52:17.093517+0000","last_active":"2024-09-16T15:52:19.336267+0000","last_peered":"2024-09-16T15:52:19.336267+0000","last_clean":"2024-09-16T15:52:19.336267+0000","last_became_active":"2024-09-16T15:52:17.093229+0000","last_became_peered":"2024-09-16T15:52:17.093229+0000","last_unstale":"2024-09-16T15:52:19.336267+0000","last_undegraded":"2024-09-16T15:52:19.336267+0000","last_fullsized":"2024-09-16T15:52:19.336267+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:22:18.051932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.324584+0000","last_change":"2024-09-16T15:52:18.099124+0000","last_active":"2024-09-16T15:52:19.324584+0000","last_peered":"2024-09-16T15:52:19.324584+0000","last_clean":"2024-09-16T15:52:19.324584+0000","last_became_active":"2024-09-16T15:52:18.098576+0000","last_became_peered":"2024-09-16T15:52:18.098576+0000","last_unstale":"2024-09-16T15:52:19.324584+0000","last_undegraded":"2024-09-16T15:52:19.324584+0000","last_fullsized":"2024-09-16T15:52:19.324584+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:50:42.139677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336848+0000","last_change":"2024-09-16T15:52:17.087919+0000","last_active":"2024-09-16T15:52:19.336848+0000","last_peered":"2024-09-16T15:52:19.336848+0000","last_clean":"2024-09-16T15:52:19.336848+0000","last_became_active":"2024-09-16T15:52:17.086912+0000","last_became_peered":"2024-09-16T15:52:17.086912+0000","last_unstale":"2024-09-16T15:52:19.336848+0000","last_undegraded":"2024-09-16T15:52:19.336848+0000","last_fullsized":"2024-09-16T15:52:19.336848+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:22:20.426416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.324724+0000","last_change":"2024-09-16T15:52:18.093980+0000","last_active":"2024-09-16T15:52:19.324724+0000","last_peered":"2024-09-16T15:52:19.324724+0000","last_clean":"2024-09-16T15:52:19.324724+0000","last_became_active":"2024-09-16T15:52:18.093733+0000","last_became_peered":"2024-09-16T15:52:18.093733+0000","last_unstale":"2024-09-16T15:52:19.324724+0000","last_undegraded":"2024-09-16T15:52:19.324724+0000","last_fullsized":"2024-09-16T15:52:19.324724+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:37:17.979172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336543+0000","last_change":"2024-09-16T15:52:17.088021+0000","last_active":"2024-09-16T15:52:19.336543+0000","last_peered":"2024-09-16T15:52:19.336543+0000","last_clean":"2024-09-16T15:52:19.336543+0000","last_became_active":"2024-09-16T15:52:17.087740+0000","last_became_peered":"2024-09-16T15:52:17.087740+0000","last_unstale":"2024-09-16T15:52:19.336543+0000","last_undegraded":"2024-09-16T15:52:19.336543+0000","last_fullsized":"2024-09-16T15:52:19.336543+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:20:18.506788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.097462+0000","last_change":"2024-09-16T15:52:18.088933+0000","last_active":"2024-09-16T15:52:19.097462+0000","last_peered":"2024-09-16T15:52:19.097462+0000","last_clean":"2024-09-16T15:52:19.097462+0000","last_became_active":"2024-09-16T15:52:18.088644+0000","last_became_peered":"2024-09-16T15:52:18.088644+0000","last_unstale":"2024-09-16T15:52:19.097462+0000","last_undegraded":"2024-09-16T15:52:19.097462+0000","last_fullsized":"2024-09-16T15:52:19.097462+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:50:44.707951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"117'3","reported_seq":18,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335658+0000","last_change":"2024-09-16T15:52:17.090647+0000","last_active":"2024-09-16T15:52:19.335658+0000","last_peered":"2024-09-16T15:52:19.335658+0000","last_clean":"2024-09-16T15:52:19.335658+0000","last_became_active":"2024-09-16T15:52:17.089382+0000","last_became_peered":"2024-09-16T15:52:17.089382+0000","last_unstale":"2024-09-16T15:52:19.335658+0000","last_undegraded":"2024-09-16T15:52:19.335658+0000","last_fullsized":"2024-09-16T15:52:19.335658+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:22:34.607260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098802+0000","last_change":"2024-09-16T15:52:18.107440+0000","last_active":"2024-09-16T15:52:19.098802+0000","last_peered":"2024-09-16T15:52:19.098802+0000","last_clean":"2024-09-16T15:52:19.098802+0000","last_became_active":"2024-09-16T15:52:18.107190+0000","last_became_peered":"2024-09-16T15:52:18.107190+0000","last_unstale":"2024-09-16T15:52:19.098802+0000","last_undegraded":"2024-09-16T15:52:19.098802+0000","last_fullsized":"2024-09-16T15:52:19.098802+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:05:50.369655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.14","version":"117'6","reported_seq":21,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336439+0000","last_change":"2024-09-16T15:52:17.088066+0000","last_active":"2024-09-16T15:52:19.336439+0000","last_peered":"2024-09-16T15:52:19.336439+0000","last_clean":"2024-09-16T15:52:19.336439+0000","last_became_active":"2024-09-16T15:52:17.087451+0000","last_became_peered":"2024-09-16T15:52:17.087451+0000","last_unstale":"2024-09-16T15:52:19.336439+0000","last_undegraded":"2024-09-16T15:52:19.336439+0000","last_fullsized":"2024-09-16T15:52:19.336439+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:41:10.165278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335355+0000","last_change":"2024-09-16T15:52:18.098410+0000","last_active":"2024-09-16T15:52:19.335355+0000","last_peered":"2024-09-16T15:52:19.335355+0000","last_clean":"2024-09-16T15:52:19.335355+0000","last_became_active":"2024-09-16T15:52:18.098149+0000","last_became_peered":"2024-09-16T15:52:18.098149+0000","last_unstale":"2024-09-16T15:52:19.335355+0000","last_undegraded":"2024-09-16T15:52:19.335355+0000","last_fullsized":"2024-09-16T15:52:19.335355+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:25:09.258265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"117'2","reported_seq":17,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335984+0000","last_change":"2024-09-16T15:52:17.086637+0000","last_active":"2024-09-16T15:52:19.335984+0000","last_peered":"2024-09-16T15:52:19.335984+0000","last_clean":"2024-09-16T15:52:19.335984+0000","last_became_active":"2024-09-16T15:52:17.086116+0000","last_became_peered":"2024-09-16T15:52:17.086116+0000","last_unstale":"2024-09-16T15:52:19.335984+0000","last_undegraded":"2024-09-16T15:52:19.335984+0000","last_fullsized":"2024-09-16T15:52:19.335984+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:33:42.514181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335966+0000","last_change":"2024-09-16T15:52:18.108762+0000","last_active":"2024-09-16T15:52:19.335966+0000","last_peered":"2024-09-16T15:52:19.335966+0000","last_clean":"2024-09-16T15:52:19.335966+0000","last_became_active":"2024-09-16T15:52:18.106039+0000","last_became_peered":"2024-09-16T15:52:18.106039+0000","last_unstale":"2024-09-16T15:52:19.335966+0000","last_undegraded":"2024-09-16T15:52:19.335966+0000","last_fullsized":"2024-09-16T15:52:19.335966+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:20:44.949446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.095450+0000","last_change":"2024-09-16T15:52:17.091004+0000","last_active":"2024-09-16T15:52:19.095450+0000","last_peered":"2024-09-16T15:52:19.095450+0000","last_clean":"2024-09-16T15:52:19.095450+0000","last_became_active":"2024-09-16T15:52:17.090693+0000","last_became_peered":"2024-09-16T15:52:17.090693+0000","last_unstale":"2024-09-16T15:52:19.095450+0000","last_undegraded":"2024-09-16T15:52:19.095450+0000","last_fullsized":"2024-09-16T15:52:19.095450+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:36:52.865477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336681+0000","last_change":"2024-09-16T15:52:18.096055+0000","last_active":"2024-09-16T15:52:19.336681+0000","last_peered":"2024-09-16T15:52:19.336681+0000","last_clean":"2024-09-16T15:52:19.336681+0000","last_became_active":"2024-09-16T15:52:18.095784+0000","last_became_peered":"2024-09-16T15:52:18.095784+0000","last_unstale":"2024-09-16T15:52:19.336681+0000","last_undegraded":"2024-09-16T15:52:19.336681+0000","last_fullsized":"2024-09-16T15:52:19.336681+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:01:04.752236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"117'8","reported_seq":23,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.097984+0000","last_change":"2024-09-16T15:52:17.084493+0000","last_active":"2024-09-16T15:52:19.097984+0000","last_peered":"2024-09-16T15:52:19.097984+0000","last_clean":"2024-09-16T15:52:19.097984+0000","last_became_active":"2024-09-16T15:52:17.083507+0000","last_became_peered":"2024-09-16T15:52:17.083507+0000","last_unstale":"2024-09-16T15:52:19.097984+0000","last_undegraded":"2024-09-16T15:52:19.097984+0000","last_fullsized":"2024-09-16T15:52:19.097984+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:02:28.448564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098947+0000","last_change":"2024-09-16T15:52:18.108841+0000","last_active":"2024-09-16T15:52:19.098947+0000","last_peered":"2024-09-16T15:52:19.098947+0000","last_clean":"2024-09-16T15:52:19.098947+0000","last_became_active":"2024-09-16T15:52:18.108511+0000","last_became_peered":"2024-09-16T15:52:18.108511+0000","last_unstale":"2024-09-16T15:52:19.098947+0000","last_undegraded":"2024-09-16T15:52:19.098947+0000","last_fullsized":"2024-09-16T15:52:19.098947+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:37:45.632048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.f","version":"117'1","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.097553+0000","last_change":"2024-09-16T15:52:17.079920+0000","last_active":"2024-09-16T15:52:19.097553+0000","last_peered":"2024-09-16T15:52:19.097553+0000","last_clean":"2024-09-16T15:52:19.097553+0000","last_became_active":"2024-09-16T15:52:17.079394+0000","last_became_peered":"2024-09-16T15:52:17.079394+0000","last_unstale":"2024-09-16T15:52:19.097553+0000","last_undegraded":"2024-09-16T15:52:19.097553+0000","last_fullsized":"2024-09-16T15:52:19.097553+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:27:33.808174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098908+0000","last_change":"2024-09-16T15:52:18.117263+0000","last_active":"2024-09-16T15:52:19.098908+0000","last_peered":"2024-09-16T15:52:19.098908+0000","last_clean":"2024-09-16T15:52:19.098908+0000","last_became_active":"2024-09-16T15:52:18.116999+0000","last_became_peered":"2024-09-16T15:52:18.116999+0000","last_unstale":"2024-09-16T15:52:19.098908+0000","last_undegraded":"2024-09-16T15:52:19.098908+0000","last_fullsized":"2024-09-16T15:52:19.098908+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:40:44.369986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"117'2","reported_seq":17,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336861+0000","last_change":"2024-09-16T15:52:17.087201+0000","last_active":"2024-09-16T15:52:19.336861+0000","last_peered":"2024-09-16T15:52:19.336861+0000","last_clean":"2024-09-16T15:52:19.336861+0000","last_became_active":"2024-09-16T15:52:17.085562+0000","last_became_peered":"2024-09-16T15:52:17.085562+0000","last_unstale":"2024-09-16T15:52:19.336861+0000","last_undegraded":"2024-09-16T15:52:19.336861+0000","last_fullsized":"2024-09-16T15:52:19.336861+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:20:46.960384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098813+0000","last_change":"2024-09-16T15:52:18.097949+0000","last_active":"2024-09-16T15:52:19.098813+0000","last_peered":"2024-09-16T15:52:19.098813+0000","last_clean":"2024-09-16T15:52:19.098813+0000","last_became_active":"2024-09-16T15:52:18.097693+0000","last_became_peered":"2024-09-16T15:52:18.097693+0000","last_unstale":"2024-09-16T15:52:19.098813+0000","last_undegraded":"2024-09-16T15:52:19.098813+0000","last_fullsized":"2024-09-16T15:52:19.098813+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:24:52.156928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.d","version":"117'1","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335394+0000","last_change":"2024-09-16T15:52:17.087628+0000","last_active":"2024-09-16T15:52:19.335394+0000","last_peered":"2024-09-16T15:52:19.335394+0000","last_clean":"2024-09-16T15:52:19.335394+0000","last_became_active":"2024-09-16T15:52:17.087384+0000","last_became_peered":"2024-09-16T15:52:17.087384+0000","last_unstale":"2024-09-16T15:52:19.335394+0000","last_undegraded":"2024-09-16T15:52:19.335394+0000","last_fullsized":"2024-09-16T15:52:19.335394+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:42:11.616130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335328+0000","last_change":"2024-09-16T15:52:18.114650+0000","last_active":"2024-09-16T15:52:19.335328+0000","last_peered":"2024-09-16T15:52:19.335328+0000","last_clean":"2024-09-16T15:52:19.335328+0000","last_became_active":"2024-09-16T15:52:18.111886+0000","last_became_peered":"2024-09-16T15:52:18.111886+0000","last_unstale":"2024-09-16T15:52:19.335328+0000","last_undegraded":"2024-09-16T15:52:19.335328+0000","last_fullsized":"2024-09-16T15:52:19.335328+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:27:33.482617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335048+0000","last_change":"2024-09-16T15:52:17.090919+0000","last_active":"2024-09-16T15:52:19.335048+0000","last_peered":"2024-09-16T15:52:19.335048+0000","last_clean":"2024-09-16T15:52:19.335048+0000","last_became_active":"2024-09-16T15:52:17.090639+0000","last_became_peered":"2024-09-16T15:52:17.090639+0000","last_unstale":"2024-09-16T15:52:19.335048+0000","last_undegraded":"2024-09-16T15:52:19.335048+0000","last_fullsized":"2024-09-16T15:52:19.335048+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:02:59.981560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.095970+0000","last_change":"2024-09-16T15:52:18.099682+0000","last_active":"2024-09-16T15:52:19.095970+0000","last_peered":"2024-09-16T15:52:19.095970+0000","last_clean":"2024-09-16T15:52:19.095970+0000","last_became_active":"2024-09-16T15:52:18.093247+0000","last_became_peered":"2024-09-16T15:52:18.093247+0000","last_unstale":"2024-09-16T15:52:19.095970+0000","last_undegraded":"2024-09-16T15:52:19.095970+0000","last_fullsized":"2024-09-16T15:52:19.095970+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:39:08.375481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.b","version":"117'2","reported_seq":17,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336094+0000","last_change":"2024-09-16T15:52:17.086366+0000","last_active":"2024-09-16T15:52:19.336094+0000","last_peered":"2024-09-16T15:52:19.336094+0000","last_clean":"2024-09-16T15:52:19.336094+0000","last_became_active":"2024-09-16T15:52:17.085765+0000","last_became_peered":"2024-09-16T15:52:17.085765+0000","last_unstale":"2024-09-16T15:52:19.336094+0000","last_undegraded":"2024-09-16T15:52:19.336094+0000","last_fullsized":"2024-09-16T15:52:19.336094+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:32:36.741556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098325+0000","last_change":"2024-09-16T15:52:18.103582+0000","last_active":"2024-09-16T15:52:19.098325+0000","last_peered":"2024-09-16T15:52:19.098325+0000","last_clean":"2024-09-16T15:52:19.098325+0000","last_became_active":"2024-09-16T15:52:18.103268+0000","last_became_peered":"2024-09-16T15:52:18.103268+0000","last_unstale":"2024-09-16T15:52:19.098325+0000","last_undegraded":"2024-09-16T15:52:19.098325+0000","last_fullsized":"2024-09-16T15:52:19.098325+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:46:48.807361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098358+0000","last_change":"2024-09-16T15:52:17.090433+0000","last_active":"2024-09-16T15:52:19.098358+0000","last_peered":"2024-09-16T15:52:19.098358+0000","last_clean":"2024-09-16T15:52:19.098358+0000","last_became_active":"2024-09-16T15:52:17.089911+0000","last_became_peered":"2024-09-16T15:52:17.089911+0000","last_unstale":"2024-09-16T15:52:19.098358+0000","last_undegraded":"2024-09-16T15:52:19.098358+0000","last_fullsized":"2024-09-16T15:52:19.098358+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:48:31.415343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.334976+0000","last_change":"2024-09-16T15:52:18.102029+0000","last_active":"2024-09-16T15:52:19.334976+0000","last_peered":"2024-09-16T15:52:19.334976+0000","last_clean":"2024-09-16T15:52:19.334976+0000","last_became_active":"2024-09-16T15:52:18.101193+0000","last_became_peered":"2024-09-16T15:52:18.101193+0000","last_unstale":"2024-09-16T15:52:19.334976+0000","last_undegraded":"2024-09-16T15:52:19.334976+0000","last_fullsized":"2024-09-16T15:52:19.334976+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:14:39.539437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336623+0000","last_change":"2024-09-16T15:52:17.089371+0000","last_active":"2024-09-16T15:52:19.336623+0000","last_peered":"2024-09-16T15:52:19.336623+0000","last_clean":"2024-09-16T15:52:19.336623+0000","last_became_active":"2024-09-16T15:52:17.089099+0000","last_became_peered":"2024-09-16T15:52:17.089099+0000","last_unstale":"2024-09-16T15:52:19.336623+0000","last_undegraded":"2024-09-16T15:52:19.336623+0000","last_fullsized":"2024-09-16T15:52:19.336623+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:04:46.705752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335539+0000","last_change":"2024-09-16T15:52:18.106035+0000","last_active":"2024-09-16T15:52:19.335539+0000","last_peered":"2024-09-16T15:52:19.335539+0000","last_clean":"2024-09-16T15:52:19.335539+0000","last_became_active":"2024-09-16T15:52:18.105771+0000","last_became_peered":"2024-09-16T15:52:18.105771+0000","last_unstale":"2024-09-16T15:52:19.335539+0000","last_undegraded":"2024-09-16T15:52:19.335539+0000","last_fullsized":"2024-09-16T15:52:19.335539+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:07:04.881864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336440+0000","last_change":"2024-09-16T15:52:17.078927+0000","last_active":"2024-09-16T15:52:19.336440+0000","last_peered":"2024-09-16T15:52:19.336440+0000","last_clean":"2024-09-16T15:52:19.336440+0000","last_became_active":"2024-09-16T15:52:17.078623+0000","last_became_peered":"2024-09-16T15:52:17.078623+0000","last_unstale":"2024-09-16T15:52:19.336440+0000","last_undegraded":"2024-09-16T15:52:19.336440+0000","last_fullsized":"2024-09-16T15:52:19.336440+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:32:34.228931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335362+0000","last_change":"2024-09-16T15:52:18.099029+0000","last_active":"2024-09-16T15:52:19.335362+0000","last_peered":"2024-09-16T15:52:19.335362+0000","last_clean":"2024-09-16T15:52:19.335362+0000","last_became_active":"2024-09-16T15:52:18.098767+0000","last_became_peered":"2024-09-16T15:52:18.098767+0000","last_unstale":"2024-09-16T15:52:19.335362+0000","last_undegraded":"2024-09-16T15:52:19.335362+0000","last_fullsized":"2024-09-16T15:52:19.335362+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:22:20.108931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.7","version":"117'2","reported_seq":17,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336898+0000","last_change":"2024-09-16T15:52:17.083225+0000","last_active":"2024-09-16T15:52:19.336898+0000","last_peered":"2024-09-16T15:52:19.336898+0000","last_clean":"2024-09-16T15:52:19.336898+0000","last_became_active":"2024-09-16T15:52:17.082946+0000","last_became_peered":"2024-09-16T15:52:17.082946+0000","last_unstale":"2024-09-16T15:52:19.336898+0000","last_undegraded":"2024-09-16T15:52:19.336898+0000","last_fullsized":"2024-09-16T15:52:19.336898+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:53:28.145727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.097472+0000","last_change":"2024-09-16T15:52:18.096158+0000","last_active":"2024-09-16T15:52:19.097472+0000","last_peered":"2024-09-16T15:52:19.097472+0000","last_clean":"2024-09-16T15:52:19.097472+0000","last_became_active":"2024-09-16T15:52:18.095854+0000","last_became_peered":"2024-09-16T15:52:18.095854+0000","last_unstale":"2024-09-16T15:52:19.097472+0000","last_undegraded":"2024-09-16T15:52:19.097472+0000","last_fullsized":"2024-09-16T15:52:19.097472+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:19:03.220760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"118'1","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098686+0000","last_change":"2024-09-16T15:52:17.091331+0000","last_active":"2024-09-16T15:52:19.098686+0000","last_peered":"2024-09-16T15:52:19.098686+0000","last_clean":"2024-09-16T15:52:19.098686+0000","last_became_active":"2024-09-16T15:52:17.091071+0000","last_became_peered":"2024-09-16T15:52:17.091071+0000","last_unstale":"2024-09-16T15:52:19.098686+0000","last_undegraded":"2024-09-16T15:52:19.098686+0000","last_fullsized":"2024-09-16T15:52:19.098686+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:18:38.561424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098802+0000","last_change":"2024-09-16T15:52:18.103429+0000","last_active":"2024-09-16T15:52:19.098802+0000","last_peered":"2024-09-16T15:52:19.098802+0000","last_clean":"2024-09-16T15:52:19.098802+0000","last_became_active":"2024-09-16T15:52:18.102940+0000","last_became_peered":"2024-09-16T15:52:18.102940+0000","last_unstale":"2024-09-16T15:52:19.098802+0000","last_undegraded":"2024-09-16T15:52:19.098802+0000","last_fullsized":"2024-09-16T15:52:19.098802+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:09:04.359467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336336+0000","last_change":"2024-09-16T15:52:17.080632+0000","last_active":"2024-09-16T15:52:19.336336+0000","last_peered":"2024-09-16T15:52:19.336336+0000","last_clean":"2024-09-16T15:52:19.336336+0000","last_became_active":"2024-09-16T15:52:17.079770+0000","last_became_peered":"2024-09-16T15:52:17.079770+0000","last_unstale":"2024-09-16T15:52:19.336336+0000","last_undegraded":"2024-09-16T15:52:19.336336+0000","last_fullsized":"2024-09-16T15:52:19.336336+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:18:23.166654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335328+0000","last_change":"2024-09-16T15:52:18.099378+0000","last_active":"2024-09-16T15:52:19.335328+0000","last_peered":"2024-09-16T15:52:19.335328+0000","last_clean":"2024-09-16T15:52:19.335328+0000","last_became_active":"2024-09-16T15:52:18.096154+0000","last_became_peered":"2024-09-16T15:52:18.096154+0000","last_unstale":"2024-09-16T15:52:19.335328+0000","last_undegraded":"2024-09-16T15:52:19.335328+0000","last_fullsized":"2024-09-16T15:52:19.335328+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:42:01.895815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336463+0000","last_change":"2024-09-16T15:52:17.091377+0000","last_active":"2024-09-16T15:52:19.336463+0000","last_peered":"2024-09-16T15:52:19.336463+0000","last_clean":"2024-09-16T15:52:19.336463+0000","last_became_active":"2024-09-16T15:52:17.090957+0000","last_became_peered":"2024-09-16T15:52:17.090957+0000","last_unstale":"2024-09-16T15:52:19.336463+0000","last_undegraded":"2024-09-16T15:52:19.336463+0000","last_fullsized":"2024-09-16T15:52:19.336463+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:21:10.990720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.095396+0000","last_change":"2024-09-16T15:52:18.107499+0000","last_active":"2024-09-16T15:52:19.095396+0000","last_peered":"2024-09-16T15:52:19.095396+0000","last_clean":"2024-09-16T15:52:19.095396+0000","last_became_active":"2024-09-16T15:52:18.105395+0000","last_became_peered":"2024-09-16T15:52:18.105395+0000","last_unstale":"2024-09-16T15:52:19.095396+0000","last_undegraded":"2024-09-16T15:52:19.095396+0000","last_fullsized":"2024-09-16T15:52:19.095396+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:15:42.167943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098838+0000","last_change":"2024-09-16T15:52:17.088515+0000","last_active":"2024-09-16T15:52:19.098838+0000","last_peered":"2024-09-16T15:52:19.098838+0000","last_clean":"2024-09-16T15:52:19.098838+0000","last_became_active":"2024-09-16T15:52:17.088093+0000","last_became_peered":"2024-09-16T15:52:17.088093+0000","last_unstale":"2024-09-16T15:52:19.098838+0000","last_undegraded":"2024-09-16T15:52:19.098838+0000","last_fullsized":"2024-09-16T15:52:19.098838+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:41:51.353892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335174+0000","last_change":"2024-09-16T15:52:18.104181+0000","last_active":"2024-09-16T15:52:19.335174+0000","last_peered":"2024-09-16T15:52:19.335174+0000","last_clean":"2024-09-16T15:52:19.335174+0000","last_became_active":"2024-09-16T15:52:18.096537+0000","last_became_peered":"2024-09-16T15:52:18.096537+0000","last_unstale":"2024-09-16T15:52:19.335174+0000","last_undegraded":"2024-09-16T15:52:19.335174+0000","last_fullsized":"2024-09-16T15:52:19.335174+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:53:28.863711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.334882+0000","last_change":"2024-09-16T15:52:17.087423+0000","last_active":"2024-09-16T15:52:19.334882+0000","last_peered":"2024-09-16T15:52:19.334882+0000","last_clean":"2024-09-16T15:52:19.334882+0000","last_became_active":"2024-09-16T15:52:17.087145+0000","last_became_peered":"2024-09-16T15:52:17.087145+0000","last_unstale":"2024-09-16T15:52:19.334882+0000","last_undegraded":"2024-09-16T15:52:19.334882+0000","last_fullsized":"2024-09-16T15:52:19.334882+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:09:02.776672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335017+0000","last_change":"2024-09-16T15:52:18.095897+0000","last_active":"2024-09-16T15:52:19.335017+0000","last_peered":"2024-09-16T15:52:19.335017+0000","last_clean":"2024-09-16T15:52:19.335017+0000","last_became_active":"2024-09-16T15:52:18.095588+0000","last_became_peered":"2024-09-16T15:52:18.095588+0000","last_unstale":"2024-09-16T15:52:19.335017+0000","last_undegraded":"2024-09-16T15:52:19.335017+0000","last_fullsized":"2024-09-16T15:52:19.335017+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:28:29.259290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098479+0000","last_change":"2024-09-16T15:52:17.085524+0000","last_active":"2024-09-16T15:52:19.098479+0000","last_peered":"2024-09-16T15:52:19.098479+0000","last_clean":"2024-09-16T15:52:19.098479+0000","last_became_active":"2024-09-16T15:52:17.084921+0000","last_became_peered":"2024-09-16T15:52:17.084921+0000","last_unstale":"2024-09-16T15:52:19.098479+0000","last_undegraded":"2024-09-16T15:52:19.098479+0000","last_fullsized":"2024-09-16T15:52:19.098479+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:20:43.941435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.096777+0000","last_change":"2024-09-16T15:52:18.106049+0000","last_active":"2024-09-16T15:52:19.096777+0000","last_peered":"2024-09-16T15:52:19.096777+0000","last_clean":"2024-09-16T15:52:19.096777+0000","last_became_active":"2024-09-16T15:52:18.105352+0000","last_became_peered":"2024-09-16T15:52:18.105352+0000","last_unstale":"2024-09-16T15:52:19.096777+0000","last_undegraded":"2024-09-16T15:52:19.096777+0000","last_fullsized":"2024-09-16T15:52:19.096777+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:29:18.998165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.096570+0000","last_change":"2024-09-16T15:52:17.079490+0000","last_active":"2024-09-16T15:52:19.096570+0000","last_peered":"2024-09-16T15:52:19.096570+0000","last_clean":"2024-09-16T15:52:19.096570+0000","last_became_active":"2024-09-16T15:52:17.079148+0000","last_became_peered":"2024-09-16T15:52:17.079148+0000","last_unstale":"2024-09-16T15:52:19.096570+0000","last_undegraded":"2024-09-16T15:52:19.096570+0000","last_fullsized":"2024-09-16T15:52:19.096570+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:17:11.750051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336381+0000","last_change":"2024-09-16T15:52:18.087311+0000","last_active":"2024-09-16T15:52:19.336381+0000","last_peered":"2024-09-16T15:52:19.336381+0000","last_clean":"2024-09-16T15:52:19.336381+0000","last_became_active":"2024-09-16T15:52:18.086640+0000","last_became_peered":"2024-09-16T15:52:18.086640+0000","last_unstale":"2024-09-16T15:52:19.336381+0000","last_undegraded":"2024-09-16T15:52:19.336381+0000","last_fullsized":"2024-09-16T15:52:19.336381+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:12:10.637529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.324319+0000","last_change":"2024-09-16T15:52:17.072908+0000","last_active":"2024-09-16T15:52:19.324319+0000","last_peered":"2024-09-16T15:52:19.324319+0000","last_clean":"2024-09-16T15:52:19.324319+0000","last_became_active":"2024-09-16T15:52:17.072645+0000","last_became_peered":"2024-09-16T15:52:17.072645+0000","last_unstale":"2024-09-16T15:52:19.324319+0000","last_undegraded":"2024-09-16T15:52:19.324319+0000","last_fullsized":"2024-09-16T15:52:19.324319+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:26:34.557572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":293,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.095214+0000","last_change":"2024-09-16T15:39:12.698813+0000","last_active":"2024-09-16T15:52:19.095214+0000","last_peered":"2024-09-16T15:52:19.095214+0000","last_clean":"2024-09-16T15:52:19.095214+0000","last_became_active":"2024-09-16T15:39:12.697695+0000","last_became_peered":"2024-09-16T15:39:12.697695+0000","last_unstale":"2024-09-16T15:52:19.095214+0000","last_undegraded":"2024-09-16T15:52:19.095214+0000","last_fullsized":"2024-09-16T15:52:19.095214+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_clean_scrub_stamp":"2024-09-16T15:39:11.684967+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:02.061081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509603,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":136684,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94235156,"statfs":{"total":96636764160,"available":96496799744,"internally_reserved":0,"allocated":1458176,"data_stored":640894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":138475976},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509602,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":135864,"kb_used_data":2012,"kb_used_omap":1,"kb_used_meta":133822,"kb_avail":94235976,"statfs":{"total":96636764160,"available":96497639424,"internally_reserved":0,"allocated":2060288,"data_stored":1231354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":137034183},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509601,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149172,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222668,"statfs":{"total":96636764160,"available":96484012032,"internally_reserved":0,"allocated":1474560,"data_stored":642596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509601,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131112,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240728,"statfs":{"total":96636764160,"available":96502505472,"internally_reserved":0,"allocated":1458176,"data_stored":640894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":132774345},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509601,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126968,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244872,"statfs":{"total":96636764160,"available":96506748928,"internally_reserved":0,"allocated":1470464,"data_stored":641532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":128514503},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509602,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127536,"kb_used_data":2004,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244304,"statfs":{"total":96636764160,"available":96506167296,"internally_reserved":0,"allocated":2052096,"data_stored":1231262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":128514503},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509600,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145012,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226828,"statfs":{"total":96636764160,"available":96488271872,"internally_reserved":0,"allocated":1466368,"data_stored":641546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":146995655},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542307,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126972,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244868,"statfs":{"total":96636764160,"available":96506744832,"internally_reserved":0,"allocated":1478656,"data_stored":642630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":128514508},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:52:27.047 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:52:27.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.214+0000 7fa99c800640 1 -- 172.21.15.151:0/292162094 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa994072df0 msgr2=0x7fa9940732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:27.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.214+0000 7fa99c800640 1 --2- 172.21.15.151:0/292162094 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa994072df0 0x7fa9940732a0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fa990005fd0 tx=0x7fa99002aa10 comp rx=0 tx=0).stop 2024-09-16T15:52:27.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.214+0000 7fa99c800640 1 -- 172.21.15.151:0/292162094 shutdown_connections 2024-09-16T15:52:27.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.214+0000 7fa99c800640 1 --2- 172.21.15.151:0/292162094 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa994072df0 0x7fa9940732a0 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:27.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.214+0000 7fa99c800640 1 --2- 172.21.15.151:0/292162094 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9940af3d0 0x7fa99406f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:27.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.214+0000 7fa99c800640 1 -- 172.21.15.151:0/292162094 >> 172.21.15.151:0/292162094 conn(0x7fa9940b1950 msgr2=0x7fa9940b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:27.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.214+0000 7fa99c800640 1 -- 172.21.15.151:0/292162094 shutdown_connections 2024-09-16T15:52:27.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.214+0000 7fa99c800640 1 -- 172.21.15.151:0/292162094 wait complete. 2024-09-16T15:52:27.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.216+0000 7fa99c800640 1 Processor -- start 2024-09-16T15:52:27.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.216+0000 7fa99c800640 1 -- start start 2024-09-16T15:52:27.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.217+0000 7fa99c800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa994072df0 0x7fa99409aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:27.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.217+0000 7fa99c800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9940af3d0 0x7fa9941f43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:27.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.217+0000 7fa99c800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa994161550 0x7fa9941f8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:27.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.217+0000 7fa99c800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa9940737e0 con 0x7fa994161550 2024-09-16T15:52:27.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.217+0000 7fa99c800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa99406fdb0 con 0x7fa9940af3d0 2024-09-16T15:52:27.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.217+0000 7fa99c800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fa994073960 con 0x7fa994072df0 2024-09-16T15:52:27.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.217+0000 7fa99a800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa994161550 0x7fa9941f8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:27.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.217+0000 7fa99a800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa994161550 0x7fa9941f8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41396/0 (socket says 172.21.15.151:41396) 2024-09-16T15:52:27.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.217+0000 7fa99a800640 1 -- 172.21.15.151:0/1270389219 learned_addr learned my addr 172.21.15.151:0/1270389219 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:52:27.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.217+0000 7fa999400640 1 --2- 172.21.15.151:0/1270389219 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9940af3d0 0x7fa9941f43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:27.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.217+0000 7fa999e00640 1 --2- 172.21.15.151:0/1270389219 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa994072df0 0x7fa99409aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:27.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.218+0000 7fa99a800640 1 -- 172.21.15.151:0/1270389219 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa994072df0 msgr2=0x7fa99409aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:27.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.218+0000 7fa99a800640 1 --2- 172.21.15.151:0/1270389219 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa994072df0 0x7fa99409aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:27.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.218+0000 7fa99a800640 1 -- 172.21.15.151:0/1270389219 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9940af3d0 msgr2=0x7fa9941f43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:27.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.218+0000 7fa99a800640 1 --2- 172.21.15.151:0/1270389219 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9940af3d0 0x7fa9941f43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:27.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.218+0000 7fa99a800640 1 -- 172.21.15.151:0/1270389219 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa99409cf60 con 0x7fa994161550 2024-09-16T15:52:27.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.218+0000 7fa99a800640 1 --2- 172.21.15.151:0/1270389219 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa994161550 0x7fa9941f8000 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fa990006010 tx=0x7fa990003f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:27.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.218+0000 7fa98e000640 1 -- 172.21.15.151:0/1270389219 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa990047070 con 0x7fa994161550 2024-09-16T15:52:27.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.218+0000 7fa98e000640 1 -- 172.21.15.151:0/1270389219 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa990004250 con 0x7fa994161550 2024-09-16T15:52:27.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.218+0000 7fa98e000640 1 -- 172.21.15.151:0/1270389219 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa990033050 con 0x7fa994161550 2024-09-16T15:52:27.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.218+0000 7fa99c800640 1 -- 172.21.15.151:0/1270389219 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa99409d240 con 0x7fa994161550 2024-09-16T15:52:27.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.218+0000 7fa99c800640 1 -- 172.21.15.151:0/1270389219 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa99409d800 con 0x7fa994161550 2024-09-16T15:52:27.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.219+0000 7fa99c800640 1 -- 172.21.15.151:0/1270389219 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9940aeb20 con 0x7fa994161550 2024-09-16T15:52:27.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.221+0000 7fa98e000640 1 -- 172.21.15.151:0/1270389219 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 41) ==== 103684+0+0 (secure 0 0 0) 0x7fa99004a070 con 0x7fa994161550 2024-09-16T15:52:27.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.222+0000 7fa98e000640 1 --2- 172.21.15.151:0/1270389219 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa97807a2c0 0x7fa97807c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:27.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.222+0000 7fa98e000640 1 -- 172.21.15.151:0/1270389219 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5793+0+0 (secure 0 0 0) 0x7fa9900baa10 con 0x7fa994161550 2024-09-16T15:52:27.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.222+0000 7fa999e00640 1 --2- 172.21.15.151:0/1270389219 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa97807a2c0 0x7fa97807c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:27.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.223+0000 7fa999e00640 1 --2- 172.21.15.151:0/1270389219 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa97807a2c0 0x7fa97807c780 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fa988000f60 tx=0x7fa988009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:27.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.226+0000 7fa98e000640 1 -- 172.21.15.151:0/1270389219 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa9940aeb20 con 0x7fa994161550 2024-09-16T15:52:27.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.405+0000 7fa99c800640 1 -- 172.21.15.151:0/1270389219 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa994073ec0 con 0x7fa97807a2c0 2024-09-16T15:52:27.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.411+0000 7fa98e000640 1 -- 172.21.15.151:0/1270389219 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252483 (secure 0 0 0) 0x7fa994073ec0 con 0x7fa97807a2c0 2024-09-16T15:52:27.415 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:52:27.417 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-09-16T15:52:27.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.415+0000 7fa99c800640 1 -- 172.21.15.151:0/1270389219 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa97807a2c0 msgr2=0x7fa97807c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:27.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.415+0000 7fa99c800640 1 --2- 172.21.15.151:0/1270389219 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa97807a2c0 0x7fa97807c780 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fa988000f60 tx=0x7fa988009040 comp rx=0 tx=0).stop 2024-09-16T15:52:27.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.415+0000 7fa99c800640 1 -- 172.21.15.151:0/1270389219 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa994161550 msgr2=0x7fa9941f8000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:27.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.415+0000 7fa99c800640 1 --2- 172.21.15.151:0/1270389219 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa994161550 0x7fa9941f8000 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fa990006010 tx=0x7fa990003f30 comp rx=0 tx=0).stop 2024-09-16T15:52:27.419 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.415+0000 7fa99c800640 1 -- 172.21.15.151:0/1270389219 shutdown_connections 2024-09-16T15:52:27.419 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.415+0000 7fa99c800640 1 --2- 172.21.15.151:0/1270389219 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa97807a2c0 0x7fa97807c780 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:27.419 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.415+0000 7fa99c800640 1 --2- 172.21.15.151:0/1270389219 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9940af3d0 0x7fa9941f43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:27.419 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.415+0000 7fa99c800640 1 --2- 172.21.15.151:0/1270389219 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa994161550 0x7fa9941f8000 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:27.419 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.415+0000 7fa99c800640 1 --2- 172.21.15.151:0/1270389219 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa994072df0 0x7fa99409aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:27.419 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.415+0000 7fa99c800640 1 -- 172.21.15.151:0/1270389219 >> 172.21.15.151:0/1270389219 conn(0x7fa9940b1950 msgr2=0x7fa9940adf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:27.419 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.415+0000 7fa99c800640 1 -- 172.21.15.151:0/1270389219 shutdown_connections 2024-09-16T15:52:27.419 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.415+0000 7fa99c800640 1 -- 172.21.15.151:0/1270389219 wait complete. 2024-09-16T15:52:27.433 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":509,"stamp":"2024-09-16T15:52:25.903225+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":187,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1079320,"kb_used_data":12616,"kb_used_omap":12,"kb_used_meta":1066483,"kb_avail":753895400,"statfs":{"total":773094113280,"available":771988889600,"internally_reserved":0,"allocated":12918784,"data_stored":6312708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12738,"internal_metadata":1092079166},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002349"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098680+0000","last_change":"2024-09-16T15:52:18.113147+0000","last_active":"2024-09-16T15:52:19.098680+0000","last_peered":"2024-09-16T15:52:19.098680+0000","last_clean":"2024-09-16T15:52:19.098680+0000","last_became_active":"2024-09-16T15:52:18.112860+0000","last_became_peered":"2024-09-16T15:52:18.112860+0000","last_unstale":"2024-09-16T15:52:19.098680+0000","last_undegraded":"2024-09-16T15:52:19.098680+0000","last_fullsized":"2024-09-16T15:52:19.098680+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:50:42.139677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.324514+0000","last_change":"2024-09-16T15:52:18.095601+0000","last_active":"2024-09-16T15:52:19.324514+0000","last_peered":"2024-09-16T15:52:19.324514+0000","last_clean":"2024-09-16T15:52:19.324514+0000","last_became_active":"2024-09-16T15:52:18.095305+0000","last_became_peered":"2024-09-16T15:52:18.095305+0000","last_unstale":"2024-09-16T15:52:19.324514+0000","last_undegraded":"2024-09-16T15:52:19.324514+0000","last_fullsized":"2024-09-16T15:52:19.324514+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:46:48.807361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.095580+0000","last_change":"2024-09-16T15:52:18.108749+0000","last_active":"2024-09-16T15:52:19.095580+0000","last_peered":"2024-09-16T15:52:19.095580+0000","last_clean":"2024-09-16T15:52:19.095580+0000","last_became_active":"2024-09-16T15:52:18.104818+0000","last_became_peered":"2024-09-16T15:52:18.104818+0000","last_unstale":"2024-09-16T15:52:19.095580+0000","last_undegraded":"2024-09-16T15:52:19.095580+0000","last_fullsized":"2024-09-16T15:52:19.095580+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:32:30.003302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336245+0000","last_change":"2024-09-16T15:52:18.090782+0000","last_active":"2024-09-16T15:52:19.336245+0000","last_peered":"2024-09-16T15:52:19.336245+0000","last_clean":"2024-09-16T15:52:19.336245+0000","last_became_active":"2024-09-16T15:52:18.090432+0000","last_became_peered":"2024-09-16T15:52:18.090432+0000","last_unstale":"2024-09-16T15:52:19.336245+0000","last_undegraded":"2024-09-16T15:52:19.336245+0000","last_fullsized":"2024-09-16T15:52:19.336245+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:48:32.423354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335861+0000","last_change":"2024-09-16T15:52:18.109873+0000","last_active":"2024-09-16T15:52:19.335861+0000","last_peered":"2024-09-16T15:52:19.335861+0000","last_clean":"2024-09-16T15:52:19.335861+0000","last_became_active":"2024-09-16T15:52:18.101073+0000","last_became_peered":"2024-09-16T15:52:18.101073+0000","last_unstale":"2024-09-16T15:52:19.335861+0000","last_undegraded":"2024-09-16T15:52:19.335861+0000","last_fullsized":"2024-09-16T15:52:19.335861+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:56:57.514683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.097692+0000","last_change":"2024-09-16T15:52:18.096166+0000","last_active":"2024-09-16T15:52:19.097692+0000","last_peered":"2024-09-16T15:52:19.097692+0000","last_clean":"2024-09-16T15:52:19.097692+0000","last_became_active":"2024-09-16T15:52:18.095913+0000","last_became_peered":"2024-09-16T15:52:18.095913+0000","last_unstale":"2024-09-16T15:52:19.097692+0000","last_undegraded":"2024-09-16T15:52:19.097692+0000","last_fullsized":"2024-09-16T15:52:19.097692+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:51:43.908577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098468+0000","last_change":"2024-09-16T15:52:18.111297+0000","last_active":"2024-09-16T15:52:19.098468+0000","last_peered":"2024-09-16T15:52:19.098468+0000","last_clean":"2024-09-16T15:52:19.098468+0000","last_became_active":"2024-09-16T15:52:18.111039+0000","last_became_peered":"2024-09-16T15:52:18.111039+0000","last_unstale":"2024-09-16T15:52:19.098468+0000","last_undegraded":"2024-09-16T15:52:19.098468+0000","last_fullsized":"2024-09-16T15:52:19.098468+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:08:05.616283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098459+0000","last_change":"2024-09-16T15:52:18.094809+0000","last_active":"2024-09-16T15:52:19.098459+0000","last_peered":"2024-09-16T15:52:19.098459+0000","last_clean":"2024-09-16T15:52:19.098459+0000","last_became_active":"2024-09-16T15:52:18.094538+0000","last_became_peered":"2024-09-16T15:52:18.094538+0000","last_unstale":"2024-09-16T15:52:19.098459+0000","last_undegraded":"2024-09-16T15:52:19.098459+0000","last_fullsized":"2024-09-16T15:52:19.098459+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:02:37.475979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098684+0000","last_change":"2024-09-16T15:52:18.112140+0000","last_active":"2024-09-16T15:52:19.098684+0000","last_peered":"2024-09-16T15:52:19.098684+0000","last_clean":"2024-09-16T15:52:19.098684+0000","last_became_active":"2024-09-16T15:52:18.111020+0000","last_became_peered":"2024-09-16T15:52:18.111020+0000","last_unstale":"2024-09-16T15:52:19.098684+0000","last_undegraded":"2024-09-16T15:52:19.098684+0000","last_fullsized":"2024-09-16T15:52:19.098684+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:37:17.979172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336431+0000","last_change":"2024-09-16T15:52:18.101787+0000","last_active":"2024-09-16T15:52:19.336431+0000","last_peered":"2024-09-16T15:52:19.336431+0000","last_clean":"2024-09-16T15:52:19.336431+0000","last_became_active":"2024-09-16T15:52:18.100961+0000","last_became_peered":"2024-09-16T15:52:18.100961+0000","last_unstale":"2024-09-16T15:52:19.336431+0000","last_undegraded":"2024-09-16T15:52:19.336431+0000","last_fullsized":"2024-09-16T15:52:19.336431+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:31:15.594998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098664+0000","last_change":"2024-09-16T15:52:18.094110+0000","last_active":"2024-09-16T15:52:19.098664+0000","last_peered":"2024-09-16T15:52:19.098664+0000","last_clean":"2024-09-16T15:52:19.098664+0000","last_became_active":"2024-09-16T15:52:18.093858+0000","last_became_peered":"2024-09-16T15:52:18.093858+0000","last_unstale":"2024-09-16T15:52:19.098664+0000","last_undegraded":"2024-09-16T15:52:19.098664+0000","last_fullsized":"2024-09-16T15:52:19.098664+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:32:24.437229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335879+0000","last_change":"2024-09-16T15:52:18.102228+0000","last_active":"2024-09-16T15:52:19.335879+0000","last_peered":"2024-09-16T15:52:19.335879+0000","last_clean":"2024-09-16T15:52:19.335879+0000","last_became_active":"2024-09-16T15:52:18.101917+0000","last_became_peered":"2024-09-16T15:52:18.101917+0000","last_unstale":"2024-09-16T15:52:19.335879+0000","last_undegraded":"2024-09-16T15:52:19.335879+0000","last_fullsized":"2024-09-16T15:52:19.335879+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:49:20.323885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.324528+0000","last_change":"2024-09-16T15:52:18.093067+0000","last_active":"2024-09-16T15:52:19.324528+0000","last_peered":"2024-09-16T15:52:19.324528+0000","last_clean":"2024-09-16T15:52:19.324528+0000","last_became_active":"2024-09-16T15:52:18.092805+0000","last_became_peered":"2024-09-16T15:52:18.092805+0000","last_unstale":"2024-09-16T15:52:19.324528+0000","last_undegraded":"2024-09-16T15:52:19.324528+0000","last_fullsized":"2024-09-16T15:52:19.324528+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:06:51.783453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.334985+0000","last_change":"2024-09-16T15:52:18.098148+0000","last_active":"2024-09-16T15:52:19.334985+0000","last_peered":"2024-09-16T15:52:19.334985+0000","last_clean":"2024-09-16T15:52:19.334985+0000","last_became_active":"2024-09-16T15:52:18.096388+0000","last_became_peered":"2024-09-16T15:52:18.096388+0000","last_unstale":"2024-09-16T15:52:19.334985+0000","last_undegraded":"2024-09-16T15:52:19.334985+0000","last_fullsized":"2024-09-16T15:52:19.334985+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:59:34.319160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098156+0000","last_change":"2024-09-16T15:52:18.115196+0000","last_active":"2024-09-16T15:52:19.098156+0000","last_peered":"2024-09-16T15:52:19.098156+0000","last_clean":"2024-09-16T15:52:19.098156+0000","last_became_active":"2024-09-16T15:52:18.114878+0000","last_became_peered":"2024-09-16T15:52:18.114878+0000","last_unstale":"2024-09-16T15:52:19.098156+0000","last_undegraded":"2024-09-16T15:52:19.098156+0000","last_fullsized":"2024-09-16T15:52:19.098156+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:55:52.511395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335328+0000","last_change":"2024-09-16T15:52:18.096798+0000","last_active":"2024-09-16T15:52:19.335328+0000","last_peered":"2024-09-16T15:52:19.335328+0000","last_clean":"2024-09-16T15:52:19.335328+0000","last_became_active":"2024-09-16T15:52:18.096396+0000","last_became_peered":"2024-09-16T15:52:18.096396+0000","last_unstale":"2024-09-16T15:52:19.335328+0000","last_undegraded":"2024-09-16T15:52:19.335328+0000","last_fullsized":"2024-09-16T15:52:19.335328+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:17:25.344931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.095682+0000","last_change":"2024-09-16T15:52:18.113939+0000","last_active":"2024-09-16T15:52:19.095682+0000","last_peered":"2024-09-16T15:52:19.095682+0000","last_clean":"2024-09-16T15:52:19.095682+0000","last_became_active":"2024-09-16T15:52:18.113634+0000","last_became_peered":"2024-09-16T15:52:18.113634+0000","last_unstale":"2024-09-16T15:52:19.095682+0000","last_undegraded":"2024-09-16T15:52:19.095682+0000","last_fullsized":"2024-09-16T15:52:19.095682+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:04:19.441890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098743+0000","last_change":"2024-09-16T15:52:18.106010+0000","last_active":"2024-09-16T15:52:19.098743+0000","last_peered":"2024-09-16T15:52:19.098743+0000","last_clean":"2024-09-16T15:52:19.098743+0000","last_became_active":"2024-09-16T15:52:18.105722+0000","last_became_peered":"2024-09-16T15:52:18.105722+0000","last_unstale":"2024-09-16T15:52:19.098743+0000","last_undegraded":"2024-09-16T15:52:19.098743+0000","last_fullsized":"2024-09-16T15:52:19.098743+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:29:30.985879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.095786+0000","last_change":"2024-09-16T15:52:18.108828+0000","last_active":"2024-09-16T15:52:19.095786+0000","last_peered":"2024-09-16T15:52:19.095786+0000","last_clean":"2024-09-16T15:52:19.095786+0000","last_became_active":"2024-09-16T15:52:18.104728+0000","last_became_peered":"2024-09-16T15:52:18.104728+0000","last_unstale":"2024-09-16T15:52:19.095786+0000","last_undegraded":"2024-09-16T15:52:19.095786+0000","last_fullsized":"2024-09-16T15:52:19.095786+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:08:15.649535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.095454+0000","last_change":"2024-09-16T15:52:18.104490+0000","last_active":"2024-09-16T15:52:19.095454+0000","last_peered":"2024-09-16T15:52:19.095454+0000","last_clean":"2024-09-16T15:52:19.095454+0000","last_became_active":"2024-09-16T15:52:18.104176+0000","last_became_peered":"2024-09-16T15:52:18.104176+0000","last_unstale":"2024-09-16T15:52:19.095454+0000","last_undegraded":"2024-09-16T15:52:19.095454+0000","last_fullsized":"2024-09-16T15:52:19.095454+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:38:00.758145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098816+0000","last_change":"2024-09-16T15:52:18.100178+0000","last_active":"2024-09-16T15:52:19.098816+0000","last_peered":"2024-09-16T15:52:19.098816+0000","last_clean":"2024-09-16T15:52:19.098816+0000","last_became_active":"2024-09-16T15:52:18.099483+0000","last_became_peered":"2024-09-16T15:52:18.099483+0000","last_unstale":"2024-09-16T15:52:19.098816+0000","last_undegraded":"2024-09-16T15:52:19.098816+0000","last_fullsized":"2024-09-16T15:52:19.098816+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:16:33.962636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.097266+0000","last_change":"2024-09-16T15:52:18.095116+0000","last_active":"2024-09-16T15:52:19.097266+0000","last_peered":"2024-09-16T15:52:19.097266+0000","last_clean":"2024-09-16T15:52:19.097266+0000","last_became_active":"2024-09-16T15:52:18.090617+0000","last_became_peered":"2024-09-16T15:52:18.090617+0000","last_unstale":"2024-09-16T15:52:19.097266+0000","last_undegraded":"2024-09-16T15:52:19.097266+0000","last_fullsized":"2024-09-16T15:52:19.097266+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:23:03.363317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335071+0000","last_change":"2024-09-16T15:52:18.102959+0000","last_active":"2024-09-16T15:52:19.335071+0000","last_peered":"2024-09-16T15:52:19.335071+0000","last_clean":"2024-09-16T15:52:19.335071+0000","last_became_active":"2024-09-16T15:52:18.102684+0000","last_became_peered":"2024-09-16T15:52:18.102684+0000","last_unstale":"2024-09-16T15:52:19.335071+0000","last_undegraded":"2024-09-16T15:52:19.335071+0000","last_fullsized":"2024-09-16T15:52:19.335071+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:26:35.565582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336817+0000","last_change":"2024-09-16T15:52:18.102292+0000","last_active":"2024-09-16T15:52:19.336817+0000","last_peered":"2024-09-16T15:52:19.336817+0000","last_clean":"2024-09-16T15:52:19.336817+0000","last_became_active":"2024-09-16T15:52:18.102017+0000","last_became_peered":"2024-09-16T15:52:18.102017+0000","last_unstale":"2024-09-16T15:52:19.336817+0000","last_undegraded":"2024-09-16T15:52:19.336817+0000","last_fullsized":"2024-09-16T15:52:19.336817+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:34:23.254431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098723+0000","last_change":"2024-09-16T15:52:18.089811+0000","last_active":"2024-09-16T15:52:19.098723+0000","last_peered":"2024-09-16T15:52:19.098723+0000","last_clean":"2024-09-16T15:52:19.098723+0000","last_became_active":"2024-09-16T15:52:18.089302+0000","last_became_peered":"2024-09-16T15:52:18.089302+0000","last_unstale":"2024-09-16T15:52:19.098723+0000","last_undegraded":"2024-09-16T15:52:19.098723+0000","last_fullsized":"2024-09-16T15:52:19.098723+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:44:21.343778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.097694+0000","last_change":"2024-09-16T15:52:18.107014+0000","last_active":"2024-09-16T15:52:19.097694+0000","last_peered":"2024-09-16T15:52:19.097694+0000","last_clean":"2024-09-16T15:52:19.097694+0000","last_became_active":"2024-09-16T15:52:18.106706+0000","last_became_peered":"2024-09-16T15:52:18.106706+0000","last_unstale":"2024-09-16T15:52:19.097694+0000","last_undegraded":"2024-09-16T15:52:19.097694+0000","last_fullsized":"2024-09-16T15:52:19.097694+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:41:14.841086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336117+0000","last_change":"2024-09-16T15:52:18.102898+0000","last_active":"2024-09-16T15:52:19.336117+0000","last_peered":"2024-09-16T15:52:19.336117+0000","last_clean":"2024-09-16T15:52:19.336117+0000","last_became_active":"2024-09-16T15:52:18.102229+0000","last_became_peered":"2024-09-16T15:52:18.102229+0000","last_unstale":"2024-09-16T15:52:19.336117+0000","last_undegraded":"2024-09-16T15:52:19.336117+0000","last_fullsized":"2024-09-16T15:52:19.336117+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:21:53.345211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.097217+0000","last_change":"2024-09-16T15:52:18.099345+0000","last_active":"2024-09-16T15:52:19.097217+0000","last_peered":"2024-09-16T15:52:19.097217+0000","last_clean":"2024-09-16T15:52:19.097217+0000","last_became_active":"2024-09-16T15:52:18.098646+0000","last_became_peered":"2024-09-16T15:52:18.098646+0000","last_unstale":"2024-09-16T15:52:19.097217+0000","last_undegraded":"2024-09-16T15:52:19.097217+0000","last_fullsized":"2024-09-16T15:52:19.097217+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:28:37.287849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335126+0000","last_change":"2024-09-16T15:52:18.106525+0000","last_active":"2024-09-16T15:52:19.335126+0000","last_peered":"2024-09-16T15:52:19.335126+0000","last_clean":"2024-09-16T15:52:19.335126+0000","last_became_active":"2024-09-16T15:52:18.103778+0000","last_became_peered":"2024-09-16T15:52:18.103778+0000","last_unstale":"2024-09-16T15:52:19.335126+0000","last_undegraded":"2024-09-16T15:52:19.335126+0000","last_fullsized":"2024-09-16T15:52:19.335126+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:46:53.909612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098944+0000","last_change":"2024-09-16T15:52:18.087005+0000","last_active":"2024-09-16T15:52:19.098944+0000","last_peered":"2024-09-16T15:52:19.098944+0000","last_clean":"2024-09-16T15:52:19.098944+0000","last_became_active":"2024-09-16T15:52:18.086463+0000","last_became_peered":"2024-09-16T15:52:18.086463+0000","last_unstale":"2024-09-16T15:52:19.098944+0000","last_undegraded":"2024-09-16T15:52:19.098944+0000","last_fullsized":"2024-09-16T15:52:19.098944+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:23:31.782022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098218+0000","last_change":"2024-09-16T15:52:18.111474+0000","last_active":"2024-09-16T15:52:19.098218+0000","last_peered":"2024-09-16T15:52:19.098218+0000","last_clean":"2024-09-16T15:52:19.098218+0000","last_became_active":"2024-09-16T15:52:18.110658+0000","last_became_peered":"2024-09-16T15:52:18.110658+0000","last_unstale":"2024-09-16T15:52:19.098218+0000","last_undegraded":"2024-09-16T15:52:19.098218+0000","last_fullsized":"2024-09-16T15:52:19.098218+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:06:51.783453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335551+0000","last_change":"2024-09-16T15:52:18.106237+0000","last_active":"2024-09-16T15:52:19.335551+0000","last_peered":"2024-09-16T15:52:19.335551+0000","last_clean":"2024-09-16T15:52:19.335551+0000","last_became_active":"2024-09-16T15:52:18.103396+0000","last_became_peered":"2024-09-16T15:52:18.103396+0000","last_unstale":"2024-09-16T15:52:19.335551+0000","last_undegraded":"2024-09-16T15:52:19.335551+0000","last_fullsized":"2024-09-16T15:52:19.335551+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:35:46.682990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098906+0000","last_change":"2024-09-16T15:52:18.104576+0000","last_active":"2024-09-16T15:52:19.098906+0000","last_peered":"2024-09-16T15:52:19.098906+0000","last_clean":"2024-09-16T15:52:19.098906+0000","last_became_active":"2024-09-16T15:52:18.104278+0000","last_became_peered":"2024-09-16T15:52:18.104278+0000","last_unstale":"2024-09-16T15:52:19.098906+0000","last_undegraded":"2024-09-16T15:52:19.098906+0000","last_fullsized":"2024-09-16T15:52:19.098906+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:55:55.824799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"117'4","reported_seq":19,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335332+0000","last_change":"2024-09-16T15:52:17.086334+0000","last_active":"2024-09-16T15:52:19.335332+0000","last_peered":"2024-09-16T15:52:19.335332+0000","last_clean":"2024-09-16T15:52:19.335332+0000","last_became_active":"2024-09-16T15:52:17.086082+0000","last_became_peered":"2024-09-16T15:52:17.086082+0000","last_unstale":"2024-09-16T15:52:19.335332+0000","last_undegraded":"2024-09-16T15:52:19.335332+0000","last_fullsized":"2024-09-16T15:52:19.335332+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:25:34.552766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335430+0000","last_change":"2024-09-16T15:52:18.102484+0000","last_active":"2024-09-16T15:52:19.335430+0000","last_peered":"2024-09-16T15:52:19.335430+0000","last_clean":"2024-09-16T15:52:19.335430+0000","last_became_active":"2024-09-16T15:52:18.102159+0000","last_became_peered":"2024-09-16T15:52:18.102159+0000","last_unstale":"2024-09-16T15:52:19.335430+0000","last_undegraded":"2024-09-16T15:52:19.335430+0000","last_fullsized":"2024-09-16T15:52:19.335430+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:18:36.837288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1e","version":"117'1","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.324560+0000","last_change":"2024-09-16T15:52:17.082085+0000","last_active":"2024-09-16T15:52:19.324560+0000","last_peered":"2024-09-16T15:52:19.324560+0000","last_clean":"2024-09-16T15:52:19.324560+0000","last_became_active":"2024-09-16T15:52:17.081840+0000","last_became_peered":"2024-09-16T15:52:17.081840+0000","last_unstale":"2024-09-16T15:52:19.324560+0000","last_undegraded":"2024-09-16T15:52:19.324560+0000","last_fullsized":"2024-09-16T15:52:19.324560+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:46:52.901601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.097360+0000","last_change":"2024-09-16T15:52:18.104067+0000","last_active":"2024-09-16T15:52:19.097360+0000","last_peered":"2024-09-16T15:52:19.097360+0000","last_clean":"2024-09-16T15:52:19.097360+0000","last_became_active":"2024-09-16T15:52:18.103764+0000","last_became_peered":"2024-09-16T15:52:18.103764+0000","last_unstale":"2024-09-16T15:52:19.097360+0000","last_undegraded":"2024-09-16T15:52:19.097360+0000","last_fullsized":"2024-09-16T15:52:19.097360+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:51:28.451444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1d","version":"117'2","reported_seq":17,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336225+0000","last_change":"2024-09-16T15:52:17.083135+0000","last_active":"2024-09-16T15:52:19.336225+0000","last_peered":"2024-09-16T15:52:19.336225+0000","last_clean":"2024-09-16T15:52:19.336225+0000","last_became_active":"2024-09-16T15:52:17.082799+0000","last_became_peered":"2024-09-16T15:52:17.082799+0000","last_unstale":"2024-09-16T15:52:19.336225+0000","last_undegraded":"2024-09-16T15:52:19.336225+0000","last_fullsized":"2024-09-16T15:52:19.336225+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:43:05.707621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.095847+0000","last_change":"2024-09-16T15:52:18.109036+0000","last_active":"2024-09-16T15:52:19.095847+0000","last_peered":"2024-09-16T15:52:19.095847+0000","last_clean":"2024-09-16T15:52:19.095847+0000","last_became_active":"2024-09-16T15:52:18.106979+0000","last_became_peered":"2024-09-16T15:52:18.106979+0000","last_unstale":"2024-09-16T15:52:19.095847+0000","last_undegraded":"2024-09-16T15:52:19.095847+0000","last_fullsized":"2024-09-16T15:52:19.095847+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:38:25.952399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"117'2","reported_seq":17,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336324+0000","last_change":"2024-09-16T15:52:17.087728+0000","last_active":"2024-09-16T15:52:19.336324+0000","last_peered":"2024-09-16T15:52:19.336324+0000","last_clean":"2024-09-16T15:52:19.336324+0000","last_became_active":"2024-09-16T15:52:17.087063+0000","last_became_peered":"2024-09-16T15:52:17.087063+0000","last_unstale":"2024-09-16T15:52:19.336324+0000","last_undegraded":"2024-09-16T15:52:19.336324+0000","last_fullsized":"2024-09-16T15:52:19.336324+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:57:32.844856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336678+0000","last_change":"2024-09-16T15:52:18.111393+0000","last_active":"2024-09-16T15:52:19.336678+0000","last_peered":"2024-09-16T15:52:19.336678+0000","last_clean":"2024-09-16T15:52:19.336678+0000","last_became_active":"2024-09-16T15:52:18.111116+0000","last_became_peered":"2024-09-16T15:52:18.111116+0000","last_unstale":"2024-09-16T15:52:19.336678+0000","last_undegraded":"2024-09-16T15:52:19.336678+0000","last_fullsized":"2024-09-16T15:52:19.336678+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:59:34.319160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1b","version":"117'1","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.095273+0000","last_change":"2024-09-16T15:52:17.083411+0000","last_active":"2024-09-16T15:52:19.095273+0000","last_peered":"2024-09-16T15:52:19.095273+0000","last_clean":"2024-09-16T15:52:19.095273+0000","last_became_active":"2024-09-16T15:52:17.083098+0000","last_became_peered":"2024-09-16T15:52:17.083098+0000","last_unstale":"2024-09-16T15:52:19.095273+0000","last_undegraded":"2024-09-16T15:52:19.095273+0000","last_fullsized":"2024-09-16T15:52:19.095273+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:48:17.710440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098564+0000","last_change":"2024-09-16T15:52:18.099617+0000","last_active":"2024-09-16T15:52:19.098564+0000","last_peered":"2024-09-16T15:52:19.098564+0000","last_clean":"2024-09-16T15:52:19.098564+0000","last_became_active":"2024-09-16T15:52:18.099149+0000","last_became_peered":"2024-09-16T15:52:18.099149+0000","last_unstale":"2024-09-16T15:52:19.098564+0000","last_undegraded":"2024-09-16T15:52:19.098564+0000","last_fullsized":"2024-09-16T15:52:19.098564+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:41:19.687707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1a","version":"117'1","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098345+0000","last_change":"2024-09-16T15:52:17.086053+0000","last_active":"2024-09-16T15:52:19.098345+0000","last_peered":"2024-09-16T15:52:19.098345+0000","last_clean":"2024-09-16T15:52:19.098345+0000","last_became_active":"2024-09-16T15:52:17.084186+0000","last_became_peered":"2024-09-16T15:52:17.084186+0000","last_unstale":"2024-09-16T15:52:19.098345+0000","last_undegraded":"2024-09-16T15:52:19.098345+0000","last_fullsized":"2024-09-16T15:52:19.098345+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:21:52.337200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336424+0000","last_change":"2024-09-16T15:52:18.101280+0000","last_active":"2024-09-16T15:52:19.336424+0000","last_peered":"2024-09-16T15:52:19.336424+0000","last_clean":"2024-09-16T15:52:19.336424+0000","last_became_active":"2024-09-16T15:52:18.100938+0000","last_became_peered":"2024-09-16T15:52:18.100938+0000","last_unstale":"2024-09-16T15:52:19.336424+0000","last_undegraded":"2024-09-16T15:52:19.336424+0000","last_fullsized":"2024-09-16T15:52:19.336424+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:18:58.903310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098529+0000","last_change":"2024-09-16T15:52:17.090751+0000","last_active":"2024-09-16T15:52:19.098529+0000","last_peered":"2024-09-16T15:52:19.098529+0000","last_clean":"2024-09-16T15:52:19.098529+0000","last_became_active":"2024-09-16T15:52:17.090390+0000","last_became_peered":"2024-09-16T15:52:17.090390+0000","last_unstale":"2024-09-16T15:52:19.098529+0000","last_undegraded":"2024-09-16T15:52:19.098529+0000","last_fullsized":"2024-09-16T15:52:19.098529+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:58:11.585044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098903+0000","last_change":"2024-09-16T15:52:18.090433+0000","last_active":"2024-09-16T15:52:19.098903+0000","last_peered":"2024-09-16T15:52:19.098903+0000","last_clean":"2024-09-16T15:52:19.098903+0000","last_became_active":"2024-09-16T15:52:18.090169+0000","last_became_peered":"2024-09-16T15:52:18.090169+0000","last_unstale":"2024-09-16T15:52:19.098903+0000","last_undegraded":"2024-09-16T15:52:19.098903+0000","last_fullsized":"2024-09-16T15:52:19.098903+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:20:57.801785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336267+0000","last_change":"2024-09-16T15:52:17.093517+0000","last_active":"2024-09-16T15:52:19.336267+0000","last_peered":"2024-09-16T15:52:19.336267+0000","last_clean":"2024-09-16T15:52:19.336267+0000","last_became_active":"2024-09-16T15:52:17.093229+0000","last_became_peered":"2024-09-16T15:52:17.093229+0000","last_unstale":"2024-09-16T15:52:19.336267+0000","last_undegraded":"2024-09-16T15:52:19.336267+0000","last_fullsized":"2024-09-16T15:52:19.336267+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:22:18.051932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.324584+0000","last_change":"2024-09-16T15:52:18.099124+0000","last_active":"2024-09-16T15:52:19.324584+0000","last_peered":"2024-09-16T15:52:19.324584+0000","last_clean":"2024-09-16T15:52:19.324584+0000","last_became_active":"2024-09-16T15:52:18.098576+0000","last_became_peered":"2024-09-16T15:52:18.098576+0000","last_unstale":"2024-09-16T15:52:19.324584+0000","last_undegraded":"2024-09-16T15:52:19.324584+0000","last_fullsized":"2024-09-16T15:52:19.324584+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:50:42.139677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336848+0000","last_change":"2024-09-16T15:52:17.087919+0000","last_active":"2024-09-16T15:52:19.336848+0000","last_peered":"2024-09-16T15:52:19.336848+0000","last_clean":"2024-09-16T15:52:19.336848+0000","last_became_active":"2024-09-16T15:52:17.086912+0000","last_became_peered":"2024-09-16T15:52:17.086912+0000","last_unstale":"2024-09-16T15:52:19.336848+0000","last_undegraded":"2024-09-16T15:52:19.336848+0000","last_fullsized":"2024-09-16T15:52:19.336848+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:22:20.426416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.324724+0000","last_change":"2024-09-16T15:52:18.093980+0000","last_active":"2024-09-16T15:52:19.324724+0000","last_peered":"2024-09-16T15:52:19.324724+0000","last_clean":"2024-09-16T15:52:19.324724+0000","last_became_active":"2024-09-16T15:52:18.093733+0000","last_became_peered":"2024-09-16T15:52:18.093733+0000","last_unstale":"2024-09-16T15:52:19.324724+0000","last_undegraded":"2024-09-16T15:52:19.324724+0000","last_fullsized":"2024-09-16T15:52:19.324724+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:37:17.979172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336543+0000","last_change":"2024-09-16T15:52:17.088021+0000","last_active":"2024-09-16T15:52:19.336543+0000","last_peered":"2024-09-16T15:52:19.336543+0000","last_clean":"2024-09-16T15:52:19.336543+0000","last_became_active":"2024-09-16T15:52:17.087740+0000","last_became_peered":"2024-09-16T15:52:17.087740+0000","last_unstale":"2024-09-16T15:52:19.336543+0000","last_undegraded":"2024-09-16T15:52:19.336543+0000","last_fullsized":"2024-09-16T15:52:19.336543+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:20:18.506788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.097462+0000","last_change":"2024-09-16T15:52:18.088933+0000","last_active":"2024-09-16T15:52:19.097462+0000","last_peered":"2024-09-16T15:52:19.097462+0000","last_clean":"2024-09-16T15:52:19.097462+0000","last_became_active":"2024-09-16T15:52:18.088644+0000","last_became_peered":"2024-09-16T15:52:18.088644+0000","last_unstale":"2024-09-16T15:52:19.097462+0000","last_undegraded":"2024-09-16T15:52:19.097462+0000","last_fullsized":"2024-09-16T15:52:19.097462+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:50:44.707951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"117'3","reported_seq":18,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335658+0000","last_change":"2024-09-16T15:52:17.090647+0000","last_active":"2024-09-16T15:52:19.335658+0000","last_peered":"2024-09-16T15:52:19.335658+0000","last_clean":"2024-09-16T15:52:19.335658+0000","last_became_active":"2024-09-16T15:52:17.089382+0000","last_became_peered":"2024-09-16T15:52:17.089382+0000","last_unstale":"2024-09-16T15:52:19.335658+0000","last_undegraded":"2024-09-16T15:52:19.335658+0000","last_fullsized":"2024-09-16T15:52:19.335658+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:22:34.607260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098802+0000","last_change":"2024-09-16T15:52:18.107440+0000","last_active":"2024-09-16T15:52:19.098802+0000","last_peered":"2024-09-16T15:52:19.098802+0000","last_clean":"2024-09-16T15:52:19.098802+0000","last_became_active":"2024-09-16T15:52:18.107190+0000","last_became_peered":"2024-09-16T15:52:18.107190+0000","last_unstale":"2024-09-16T15:52:19.098802+0000","last_undegraded":"2024-09-16T15:52:19.098802+0000","last_fullsized":"2024-09-16T15:52:19.098802+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:05:50.369655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.14","version":"117'6","reported_seq":21,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336439+0000","last_change":"2024-09-16T15:52:17.088066+0000","last_active":"2024-09-16T15:52:19.336439+0000","last_peered":"2024-09-16T15:52:19.336439+0000","last_clean":"2024-09-16T15:52:19.336439+0000","last_became_active":"2024-09-16T15:52:17.087451+0000","last_became_peered":"2024-09-16T15:52:17.087451+0000","last_unstale":"2024-09-16T15:52:19.336439+0000","last_undegraded":"2024-09-16T15:52:19.336439+0000","last_fullsized":"2024-09-16T15:52:19.336439+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:41:10.165278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335355+0000","last_change":"2024-09-16T15:52:18.098410+0000","last_active":"2024-09-16T15:52:19.335355+0000","last_peered":"2024-09-16T15:52:19.335355+0000","last_clean":"2024-09-16T15:52:19.335355+0000","last_became_active":"2024-09-16T15:52:18.098149+0000","last_became_peered":"2024-09-16T15:52:18.098149+0000","last_unstale":"2024-09-16T15:52:19.335355+0000","last_undegraded":"2024-09-16T15:52:19.335355+0000","last_fullsized":"2024-09-16T15:52:19.335355+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:25:09.258265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"117'2","reported_seq":17,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335984+0000","last_change":"2024-09-16T15:52:17.086637+0000","last_active":"2024-09-16T15:52:19.335984+0000","last_peered":"2024-09-16T15:52:19.335984+0000","last_clean":"2024-09-16T15:52:19.335984+0000","last_became_active":"2024-09-16T15:52:17.086116+0000","last_became_peered":"2024-09-16T15:52:17.086116+0000","last_unstale":"2024-09-16T15:52:19.335984+0000","last_undegraded":"2024-09-16T15:52:19.335984+0000","last_fullsized":"2024-09-16T15:52:19.335984+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:33:42.514181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335966+0000","last_change":"2024-09-16T15:52:18.108762+0000","last_active":"2024-09-16T15:52:19.335966+0000","last_peered":"2024-09-16T15:52:19.335966+0000","last_clean":"2024-09-16T15:52:19.335966+0000","last_became_active":"2024-09-16T15:52:18.106039+0000","last_became_peered":"2024-09-16T15:52:18.106039+0000","last_unstale":"2024-09-16T15:52:19.335966+0000","last_undegraded":"2024-09-16T15:52:19.335966+0000","last_fullsized":"2024-09-16T15:52:19.335966+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:20:44.949446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.095450+0000","last_change":"2024-09-16T15:52:17.091004+0000","last_active":"2024-09-16T15:52:19.095450+0000","last_peered":"2024-09-16T15:52:19.095450+0000","last_clean":"2024-09-16T15:52:19.095450+0000","last_became_active":"2024-09-16T15:52:17.090693+0000","last_became_peered":"2024-09-16T15:52:17.090693+0000","last_unstale":"2024-09-16T15:52:19.095450+0000","last_undegraded":"2024-09-16T15:52:19.095450+0000","last_fullsized":"2024-09-16T15:52:19.095450+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:36:52.865477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336681+0000","last_change":"2024-09-16T15:52:18.096055+0000","last_active":"2024-09-16T15:52:19.336681+0000","last_peered":"2024-09-16T15:52:19.336681+0000","last_clean":"2024-09-16T15:52:19.336681+0000","last_became_active":"2024-09-16T15:52:18.095784+0000","last_became_peered":"2024-09-16T15:52:18.095784+0000","last_unstale":"2024-09-16T15:52:19.336681+0000","last_undegraded":"2024-09-16T15:52:19.336681+0000","last_fullsized":"2024-09-16T15:52:19.336681+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:01:04.752236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"117'8","reported_seq":23,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.097984+0000","last_change":"2024-09-16T15:52:17.084493+0000","last_active":"2024-09-16T15:52:19.097984+0000","last_peered":"2024-09-16T15:52:19.097984+0000","last_clean":"2024-09-16T15:52:19.097984+0000","last_became_active":"2024-09-16T15:52:17.083507+0000","last_became_peered":"2024-09-16T15:52:17.083507+0000","last_unstale":"2024-09-16T15:52:19.097984+0000","last_undegraded":"2024-09-16T15:52:19.097984+0000","last_fullsized":"2024-09-16T15:52:19.097984+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:02:28.448564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098947+0000","last_change":"2024-09-16T15:52:18.108841+0000","last_active":"2024-09-16T15:52:19.098947+0000","last_peered":"2024-09-16T15:52:19.098947+0000","last_clean":"2024-09-16T15:52:19.098947+0000","last_became_active":"2024-09-16T15:52:18.108511+0000","last_became_peered":"2024-09-16T15:52:18.108511+0000","last_unstale":"2024-09-16T15:52:19.098947+0000","last_undegraded":"2024-09-16T15:52:19.098947+0000","last_fullsized":"2024-09-16T15:52:19.098947+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:37:45.632048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.f","version":"117'1","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.097553+0000","last_change":"2024-09-16T15:52:17.079920+0000","last_active":"2024-09-16T15:52:19.097553+0000","last_peered":"2024-09-16T15:52:19.097553+0000","last_clean":"2024-09-16T15:52:19.097553+0000","last_became_active":"2024-09-16T15:52:17.079394+0000","last_became_peered":"2024-09-16T15:52:17.079394+0000","last_unstale":"2024-09-16T15:52:19.097553+0000","last_undegraded":"2024-09-16T15:52:19.097553+0000","last_fullsized":"2024-09-16T15:52:19.097553+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:27:33.808174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098908+0000","last_change":"2024-09-16T15:52:18.117263+0000","last_active":"2024-09-16T15:52:19.098908+0000","last_peered":"2024-09-16T15:52:19.098908+0000","last_clean":"2024-09-16T15:52:19.098908+0000","last_became_active":"2024-09-16T15:52:18.116999+0000","last_became_peered":"2024-09-16T15:52:18.116999+0000","last_unstale":"2024-09-16T15:52:19.098908+0000","last_undegraded":"2024-09-16T15:52:19.098908+0000","last_fullsized":"2024-09-16T15:52:19.098908+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:40:44.369986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"117'2","reported_seq":17,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336861+0000","last_change":"2024-09-16T15:52:17.087201+0000","last_active":"2024-09-16T15:52:19.336861+0000","last_peered":"2024-09-16T15:52:19.336861+0000","last_clean":"2024-09-16T15:52:19.336861+0000","last_became_active":"2024-09-16T15:52:17.085562+0000","last_became_peered":"2024-09-16T15:52:17.085562+0000","last_unstale":"2024-09-16T15:52:19.336861+0000","last_undegraded":"2024-09-16T15:52:19.336861+0000","last_fullsized":"2024-09-16T15:52:19.336861+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:20:46.960384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098813+0000","last_change":"2024-09-16T15:52:18.097949+0000","last_active":"2024-09-16T15:52:19.098813+0000","last_peered":"2024-09-16T15:52:19.098813+0000","last_clean":"2024-09-16T15:52:19.098813+0000","last_became_active":"2024-09-16T15:52:18.097693+0000","last_became_peered":"2024-09-16T15:52:18.097693+0000","last_unstale":"2024-09-16T15:52:19.098813+0000","last_undegraded":"2024-09-16T15:52:19.098813+0000","last_fullsized":"2024-09-16T15:52:19.098813+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:24:52.156928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.d","version":"117'1","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335394+0000","last_change":"2024-09-16T15:52:17.087628+0000","last_active":"2024-09-16T15:52:19.335394+0000","last_peered":"2024-09-16T15:52:19.335394+0000","last_clean":"2024-09-16T15:52:19.335394+0000","last_became_active":"2024-09-16T15:52:17.087384+0000","last_became_peered":"2024-09-16T15:52:17.087384+0000","last_unstale":"2024-09-16T15:52:19.335394+0000","last_undegraded":"2024-09-16T15:52:19.335394+0000","last_fullsized":"2024-09-16T15:52:19.335394+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:42:11.616130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335328+0000","last_change":"2024-09-16T15:52:18.114650+0000","last_active":"2024-09-16T15:52:19.335328+0000","last_peered":"2024-09-16T15:52:19.335328+0000","last_clean":"2024-09-16T15:52:19.335328+0000","last_became_active":"2024-09-16T15:52:18.111886+0000","last_became_peered":"2024-09-16T15:52:18.111886+0000","last_unstale":"2024-09-16T15:52:19.335328+0000","last_undegraded":"2024-09-16T15:52:19.335328+0000","last_fullsized":"2024-09-16T15:52:19.335328+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:27:33.482617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335048+0000","last_change":"2024-09-16T15:52:17.090919+0000","last_active":"2024-09-16T15:52:19.335048+0000","last_peered":"2024-09-16T15:52:19.335048+0000","last_clean":"2024-09-16T15:52:19.335048+0000","last_became_active":"2024-09-16T15:52:17.090639+0000","last_became_peered":"2024-09-16T15:52:17.090639+0000","last_unstale":"2024-09-16T15:52:19.335048+0000","last_undegraded":"2024-09-16T15:52:19.335048+0000","last_fullsized":"2024-09-16T15:52:19.335048+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:02:59.981560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.095970+0000","last_change":"2024-09-16T15:52:18.099682+0000","last_active":"2024-09-16T15:52:19.095970+0000","last_peered":"2024-09-16T15:52:19.095970+0000","last_clean":"2024-09-16T15:52:19.095970+0000","last_became_active":"2024-09-16T15:52:18.093247+0000","last_became_peered":"2024-09-16T15:52:18.093247+0000","last_unstale":"2024-09-16T15:52:19.095970+0000","last_undegraded":"2024-09-16T15:52:19.095970+0000","last_fullsized":"2024-09-16T15:52:19.095970+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:39:08.375481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.b","version":"117'2","reported_seq":17,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336094+0000","last_change":"2024-09-16T15:52:17.086366+0000","last_active":"2024-09-16T15:52:19.336094+0000","last_peered":"2024-09-16T15:52:19.336094+0000","last_clean":"2024-09-16T15:52:19.336094+0000","last_became_active":"2024-09-16T15:52:17.085765+0000","last_became_peered":"2024-09-16T15:52:17.085765+0000","last_unstale":"2024-09-16T15:52:19.336094+0000","last_undegraded":"2024-09-16T15:52:19.336094+0000","last_fullsized":"2024-09-16T15:52:19.336094+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:32:36.741556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098325+0000","last_change":"2024-09-16T15:52:18.103582+0000","last_active":"2024-09-16T15:52:19.098325+0000","last_peered":"2024-09-16T15:52:19.098325+0000","last_clean":"2024-09-16T15:52:19.098325+0000","last_became_active":"2024-09-16T15:52:18.103268+0000","last_became_peered":"2024-09-16T15:52:18.103268+0000","last_unstale":"2024-09-16T15:52:19.098325+0000","last_undegraded":"2024-09-16T15:52:19.098325+0000","last_fullsized":"2024-09-16T15:52:19.098325+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:46:48.807361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098358+0000","last_change":"2024-09-16T15:52:17.090433+0000","last_active":"2024-09-16T15:52:19.098358+0000","last_peered":"2024-09-16T15:52:19.098358+0000","last_clean":"2024-09-16T15:52:19.098358+0000","last_became_active":"2024-09-16T15:52:17.089911+0000","last_became_peered":"2024-09-16T15:52:17.089911+0000","last_unstale":"2024-09-16T15:52:19.098358+0000","last_undegraded":"2024-09-16T15:52:19.098358+0000","last_fullsized":"2024-09-16T15:52:19.098358+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:48:31.415343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.334976+0000","last_change":"2024-09-16T15:52:18.102029+0000","last_active":"2024-09-16T15:52:19.334976+0000","last_peered":"2024-09-16T15:52:19.334976+0000","last_clean":"2024-09-16T15:52:19.334976+0000","last_became_active":"2024-09-16T15:52:18.101193+0000","last_became_peered":"2024-09-16T15:52:18.101193+0000","last_unstale":"2024-09-16T15:52:19.334976+0000","last_undegraded":"2024-09-16T15:52:19.334976+0000","last_fullsized":"2024-09-16T15:52:19.334976+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:14:39.539437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336623+0000","last_change":"2024-09-16T15:52:17.089371+0000","last_active":"2024-09-16T15:52:19.336623+0000","last_peered":"2024-09-16T15:52:19.336623+0000","last_clean":"2024-09-16T15:52:19.336623+0000","last_became_active":"2024-09-16T15:52:17.089099+0000","last_became_peered":"2024-09-16T15:52:17.089099+0000","last_unstale":"2024-09-16T15:52:19.336623+0000","last_undegraded":"2024-09-16T15:52:19.336623+0000","last_fullsized":"2024-09-16T15:52:19.336623+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:04:46.705752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335539+0000","last_change":"2024-09-16T15:52:18.106035+0000","last_active":"2024-09-16T15:52:19.335539+0000","last_peered":"2024-09-16T15:52:19.335539+0000","last_clean":"2024-09-16T15:52:19.335539+0000","last_became_active":"2024-09-16T15:52:18.105771+0000","last_became_peered":"2024-09-16T15:52:18.105771+0000","last_unstale":"2024-09-16T15:52:19.335539+0000","last_undegraded":"2024-09-16T15:52:19.335539+0000","last_fullsized":"2024-09-16T15:52:19.335539+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:07:04.881864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336440+0000","last_change":"2024-09-16T15:52:17.078927+0000","last_active":"2024-09-16T15:52:19.336440+0000","last_peered":"2024-09-16T15:52:19.336440+0000","last_clean":"2024-09-16T15:52:19.336440+0000","last_became_active":"2024-09-16T15:52:17.078623+0000","last_became_peered":"2024-09-16T15:52:17.078623+0000","last_unstale":"2024-09-16T15:52:19.336440+0000","last_undegraded":"2024-09-16T15:52:19.336440+0000","last_fullsized":"2024-09-16T15:52:19.336440+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:32:34.228931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335362+0000","last_change":"2024-09-16T15:52:18.099029+0000","last_active":"2024-09-16T15:52:19.335362+0000","last_peered":"2024-09-16T15:52:19.335362+0000","last_clean":"2024-09-16T15:52:19.335362+0000","last_became_active":"2024-09-16T15:52:18.098767+0000","last_became_peered":"2024-09-16T15:52:18.098767+0000","last_unstale":"2024-09-16T15:52:19.335362+0000","last_undegraded":"2024-09-16T15:52:19.335362+0000","last_fullsized":"2024-09-16T15:52:19.335362+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:22:20.108931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.7","version":"117'2","reported_seq":17,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336898+0000","last_change":"2024-09-16T15:52:17.083225+0000","last_active":"2024-09-16T15:52:19.336898+0000","last_peered":"2024-09-16T15:52:19.336898+0000","last_clean":"2024-09-16T15:52:19.336898+0000","last_became_active":"2024-09-16T15:52:17.082946+0000","last_became_peered":"2024-09-16T15:52:17.082946+0000","last_unstale":"2024-09-16T15:52:19.336898+0000","last_undegraded":"2024-09-16T15:52:19.336898+0000","last_fullsized":"2024-09-16T15:52:19.336898+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:53:28.145727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.097472+0000","last_change":"2024-09-16T15:52:18.096158+0000","last_active":"2024-09-16T15:52:19.097472+0000","last_peered":"2024-09-16T15:52:19.097472+0000","last_clean":"2024-09-16T15:52:19.097472+0000","last_became_active":"2024-09-16T15:52:18.095854+0000","last_became_peered":"2024-09-16T15:52:18.095854+0000","last_unstale":"2024-09-16T15:52:19.097472+0000","last_undegraded":"2024-09-16T15:52:19.097472+0000","last_fullsized":"2024-09-16T15:52:19.097472+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:19:03.220760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"118'1","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098686+0000","last_change":"2024-09-16T15:52:17.091331+0000","last_active":"2024-09-16T15:52:19.098686+0000","last_peered":"2024-09-16T15:52:19.098686+0000","last_clean":"2024-09-16T15:52:19.098686+0000","last_became_active":"2024-09-16T15:52:17.091071+0000","last_became_peered":"2024-09-16T15:52:17.091071+0000","last_unstale":"2024-09-16T15:52:19.098686+0000","last_undegraded":"2024-09-16T15:52:19.098686+0000","last_fullsized":"2024-09-16T15:52:19.098686+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:18:38.561424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098802+0000","last_change":"2024-09-16T15:52:18.103429+0000","last_active":"2024-09-16T15:52:19.098802+0000","last_peered":"2024-09-16T15:52:19.098802+0000","last_clean":"2024-09-16T15:52:19.098802+0000","last_became_active":"2024-09-16T15:52:18.102940+0000","last_became_peered":"2024-09-16T15:52:18.102940+0000","last_unstale":"2024-09-16T15:52:19.098802+0000","last_undegraded":"2024-09-16T15:52:19.098802+0000","last_fullsized":"2024-09-16T15:52:19.098802+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:09:04.359467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336336+0000","last_change":"2024-09-16T15:52:17.080632+0000","last_active":"2024-09-16T15:52:19.336336+0000","last_peered":"2024-09-16T15:52:19.336336+0000","last_clean":"2024-09-16T15:52:19.336336+0000","last_became_active":"2024-09-16T15:52:17.079770+0000","last_became_peered":"2024-09-16T15:52:17.079770+0000","last_unstale":"2024-09-16T15:52:19.336336+0000","last_undegraded":"2024-09-16T15:52:19.336336+0000","last_fullsized":"2024-09-16T15:52:19.336336+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:18:23.166654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335328+0000","last_change":"2024-09-16T15:52:18.099378+0000","last_active":"2024-09-16T15:52:19.335328+0000","last_peered":"2024-09-16T15:52:19.335328+0000","last_clean":"2024-09-16T15:52:19.335328+0000","last_became_active":"2024-09-16T15:52:18.096154+0000","last_became_peered":"2024-09-16T15:52:18.096154+0000","last_unstale":"2024-09-16T15:52:19.335328+0000","last_undegraded":"2024-09-16T15:52:19.335328+0000","last_fullsized":"2024-09-16T15:52:19.335328+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:42:01.895815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336463+0000","last_change":"2024-09-16T15:52:17.091377+0000","last_active":"2024-09-16T15:52:19.336463+0000","last_peered":"2024-09-16T15:52:19.336463+0000","last_clean":"2024-09-16T15:52:19.336463+0000","last_became_active":"2024-09-16T15:52:17.090957+0000","last_became_peered":"2024-09-16T15:52:17.090957+0000","last_unstale":"2024-09-16T15:52:19.336463+0000","last_undegraded":"2024-09-16T15:52:19.336463+0000","last_fullsized":"2024-09-16T15:52:19.336463+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:21:10.990720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.095396+0000","last_change":"2024-09-16T15:52:18.107499+0000","last_active":"2024-09-16T15:52:19.095396+0000","last_peered":"2024-09-16T15:52:19.095396+0000","last_clean":"2024-09-16T15:52:19.095396+0000","last_became_active":"2024-09-16T15:52:18.105395+0000","last_became_peered":"2024-09-16T15:52:18.105395+0000","last_unstale":"2024-09-16T15:52:19.095396+0000","last_undegraded":"2024-09-16T15:52:19.095396+0000","last_fullsized":"2024-09-16T15:52:19.095396+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:15:42.167943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098838+0000","last_change":"2024-09-16T15:52:17.088515+0000","last_active":"2024-09-16T15:52:19.098838+0000","last_peered":"2024-09-16T15:52:19.098838+0000","last_clean":"2024-09-16T15:52:19.098838+0000","last_became_active":"2024-09-16T15:52:17.088093+0000","last_became_peered":"2024-09-16T15:52:17.088093+0000","last_unstale":"2024-09-16T15:52:19.098838+0000","last_undegraded":"2024-09-16T15:52:19.098838+0000","last_fullsized":"2024-09-16T15:52:19.098838+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:41:51.353892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335174+0000","last_change":"2024-09-16T15:52:18.104181+0000","last_active":"2024-09-16T15:52:19.335174+0000","last_peered":"2024-09-16T15:52:19.335174+0000","last_clean":"2024-09-16T15:52:19.335174+0000","last_became_active":"2024-09-16T15:52:18.096537+0000","last_became_peered":"2024-09-16T15:52:18.096537+0000","last_unstale":"2024-09-16T15:52:19.335174+0000","last_undegraded":"2024-09-16T15:52:19.335174+0000","last_fullsized":"2024-09-16T15:52:19.335174+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:53:28.863711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.334882+0000","last_change":"2024-09-16T15:52:17.087423+0000","last_active":"2024-09-16T15:52:19.334882+0000","last_peered":"2024-09-16T15:52:19.334882+0000","last_clean":"2024-09-16T15:52:19.334882+0000","last_became_active":"2024-09-16T15:52:17.087145+0000","last_became_peered":"2024-09-16T15:52:17.087145+0000","last_unstale":"2024-09-16T15:52:19.334882+0000","last_undegraded":"2024-09-16T15:52:19.334882+0000","last_fullsized":"2024-09-16T15:52:19.334882+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:09:02.776672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.335017+0000","last_change":"2024-09-16T15:52:18.095897+0000","last_active":"2024-09-16T15:52:19.335017+0000","last_peered":"2024-09-16T15:52:19.335017+0000","last_clean":"2024-09-16T15:52:19.335017+0000","last_became_active":"2024-09-16T15:52:18.095588+0000","last_became_peered":"2024-09-16T15:52:18.095588+0000","last_unstale":"2024-09-16T15:52:19.335017+0000","last_undegraded":"2024-09-16T15:52:19.335017+0000","last_fullsized":"2024-09-16T15:52:19.335017+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:28:29.259290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.098479+0000","last_change":"2024-09-16T15:52:17.085524+0000","last_active":"2024-09-16T15:52:19.098479+0000","last_peered":"2024-09-16T15:52:19.098479+0000","last_clean":"2024-09-16T15:52:19.098479+0000","last_became_active":"2024-09-16T15:52:17.084921+0000","last_became_peered":"2024-09-16T15:52:17.084921+0000","last_unstale":"2024-09-16T15:52:19.098479+0000","last_undegraded":"2024-09-16T15:52:19.098479+0000","last_fullsized":"2024-09-16T15:52:19.098479+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:20:43.941435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.096777+0000","last_change":"2024-09-16T15:52:18.106049+0000","last_active":"2024-09-16T15:52:19.096777+0000","last_peered":"2024-09-16T15:52:19.096777+0000","last_clean":"2024-09-16T15:52:19.096777+0000","last_became_active":"2024-09-16T15:52:18.105352+0000","last_became_peered":"2024-09-16T15:52:18.105352+0000","last_unstale":"2024-09-16T15:52:19.096777+0000","last_undegraded":"2024-09-16T15:52:19.096777+0000","last_fullsized":"2024-09-16T15:52:19.096777+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:29:18.998165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.096570+0000","last_change":"2024-09-16T15:52:17.079490+0000","last_active":"2024-09-16T15:52:19.096570+0000","last_peered":"2024-09-16T15:52:19.096570+0000","last_clean":"2024-09-16T15:52:19.096570+0000","last_became_active":"2024-09-16T15:52:17.079148+0000","last_became_peered":"2024-09-16T15:52:17.079148+0000","last_unstale":"2024-09-16T15:52:19.096570+0000","last_undegraded":"2024-09-16T15:52:19.096570+0000","last_fullsized":"2024-09-16T15:52:19.096570+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:17:11.750051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.336381+0000","last_change":"2024-09-16T15:52:18.087311+0000","last_active":"2024-09-16T15:52:19.336381+0000","last_peered":"2024-09-16T15:52:19.336381+0000","last_clean":"2024-09-16T15:52:19.336381+0000","last_became_active":"2024-09-16T15:52:18.086640+0000","last_became_peered":"2024-09-16T15:52:18.086640+0000","last_unstale":"2024-09-16T15:52:19.336381+0000","last_undegraded":"2024-09-16T15:52:19.336381+0000","last_fullsized":"2024-09-16T15:52:19.336381+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:17.058582+0000","last_clean_scrub_stamp":"2024-09-16T15:52:17.058582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:12:10.637529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.324319+0000","last_change":"2024-09-16T15:52:17.072908+0000","last_active":"2024-09-16T15:52:19.324319+0000","last_peered":"2024-09-16T15:52:19.324319+0000","last_clean":"2024-09-16T15:52:19.324319+0000","last_became_active":"2024-09-16T15:52:17.072645+0000","last_became_peered":"2024-09-16T15:52:17.072645+0000","last_unstale":"2024-09-16T15:52:19.324319+0000","last_undegraded":"2024-09-16T15:52:19.324319+0000","last_fullsized":"2024-09-16T15:52:19.324319+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:52:16.050571+0000","last_clean_scrub_stamp":"2024-09-16T15:52:16.050571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:26:34.557572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":293,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T15:52:19.095214+0000","last_change":"2024-09-16T15:39:12.698813+0000","last_active":"2024-09-16T15:52:19.095214+0000","last_peered":"2024-09-16T15:52:19.095214+0000","last_clean":"2024-09-16T15:52:19.095214+0000","last_became_active":"2024-09-16T15:39:12.697695+0000","last_became_peered":"2024-09-16T15:39:12.697695+0000","last_unstale":"2024-09-16T15:52:19.095214+0000","last_undegraded":"2024-09-16T15:52:19.095214+0000","last_fullsized":"2024-09-16T15:52:19.095214+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_clean_scrub_stamp":"2024-09-16T15:39:11.684967+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:02.061081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509603,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":136684,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94235156,"statfs":{"total":96636764160,"available":96496799744,"internally_reserved":0,"allocated":1458176,"data_stored":640894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":138475976},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509602,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":135864,"kb_used_data":2012,"kb_used_omap":1,"kb_used_meta":133822,"kb_avail":94235976,"statfs":{"total":96636764160,"available":96497639424,"internally_reserved":0,"allocated":2060288,"data_stored":1231354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":137034183},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509601,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149172,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222668,"statfs":{"total":96636764160,"available":96484012032,"internally_reserved":0,"allocated":1474560,"data_stored":642596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151255493},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509601,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131112,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240728,"statfs":{"total":96636764160,"available":96502505472,"internally_reserved":0,"allocated":1458176,"data_stored":640894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":132774345},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509601,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126968,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244872,"statfs":{"total":96636764160,"available":96506748928,"internally_reserved":0,"allocated":1470464,"data_stored":641532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":128514503},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509602,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":127536,"kb_used_data":2004,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244304,"statfs":{"total":96636764160,"available":96506167296,"internally_reserved":0,"allocated":2052096,"data_stored":1231262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":128514503},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509600,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145012,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226828,"statfs":{"total":96636764160,"available":96488271872,"internally_reserved":0,"allocated":1466368,"data_stored":641546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":146995655},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542308,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126972,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94244868,"statfs":{"total":96636764160,"available":96506744832,"internally_reserved":0,"allocated":1478656,"data_stored":642630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":128514508},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:52:27.437 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T15:52:27.437 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-16T15:52:27.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.636+0000 7f3d48200640 1 -- 172.21.15.151:0/2600765013 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3d4015c080 msgr2=0x7f3d400cf120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:27.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.636+0000 7f3d48200640 1 --2- 172.21.15.151:0/2600765013 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3d4015c080 0x7f3d400cf120 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f3d4015b4d0 tx=0x7f3d2c02a540 comp rx=0 tx=0).stop 2024-09-16T15:52:27.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.636+0000 7f3d48200640 1 -- 172.21.15.151:0/2600765013 shutdown_connections 2024-09-16T15:52:27.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.636+0000 7f3d48200640 1 --2- 172.21.15.151:0/2600765013 >> v2:172.21.15.167:3300/0 conn(0x7f3d4015e980 0x7f3d4015ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:27.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.636+0000 7f3d48200640 1 --2- 172.21.15.151:0/2600765013 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3d4015c080 0x7f3d400cf120 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:27.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.636+0000 7f3d48200640 1 --2- 172.21.15.151:0/2600765013 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3d400b3370 0x7f3d400cebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:27.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.636+0000 7f3d48200640 1 -- 172.21.15.151:0/2600765013 >> 172.21.15.151:0/2600765013 conn(0x7f3d400b1840 msgr2=0x7f3d400b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:27.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.637+0000 7f3d48200640 1 -- 172.21.15.151:0/2600765013 shutdown_connections 2024-09-16T15:52:27.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.637+0000 7f3d48200640 1 -- 172.21.15.151:0/2600765013 wait complete. 2024-09-16T15:52:27.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.638+0000 7f3d48200640 1 Processor -- start 2024-09-16T15:52:27.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.638+0000 7f3d48200640 1 -- start start 2024-09-16T15:52:27.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.639+0000 7f3d48200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3d400b3370 0x7f3d40135cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:27.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.639+0000 7f3d48200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d4015c080 0x7f3d40151d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:27.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.639+0000 7f3d48200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3d4015e980 0x7f3d4006f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:27.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.639+0000 7f3d48200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3d40140770 con 0x7f3d4015e980 2024-09-16T15:52:27.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.639+0000 7f3d48200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3d401405f0 con 0x7f3d4015c080 2024-09-16T15:52:27.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.639+0000 7f3d48200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f3d401408f0 con 0x7f3d400b3370 2024-09-16T15:52:27.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.639+0000 7f3d46200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3d4015e980 0x7f3d4006f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:27.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.639+0000 7f3d46200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3d4015e980 0x7f3d4006f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41416/0 (socket says 172.21.15.151:41416) 2024-09-16T15:52:27.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.639+0000 7f3d46200640 1 -- 172.21.15.151:0/587969779 learned_addr learned my addr 172.21.15.151:0/587969779 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:52:27.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.640+0000 7f3d44e00640 1 --2- 172.21.15.151:0/587969779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d4015c080 0x7f3d40151d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:27.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.640+0000 7f3d45800640 1 --2- 172.21.15.151:0/587969779 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3d400b3370 0x7f3d40135cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:27.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.640+0000 7f3d46200640 1 -- 172.21.15.151:0/587969779 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3d400b3370 msgr2=0x7f3d40135cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:27.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.640+0000 7f3d46200640 1 --2- 172.21.15.151:0/587969779 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3d400b3370 0x7f3d40135cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:27.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.640+0000 7f3d46200640 1 -- 172.21.15.151:0/587969779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d4015c080 msgr2=0x7f3d40151d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:27.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.640+0000 7f3d46200640 1 --2- 172.21.15.151:0/587969779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d4015c080 0x7f3d40151d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:27.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.640+0000 7f3d46200640 1 -- 172.21.15.151:0/587969779 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d4006f720 con 0x7f3d4015e980 2024-09-16T15:52:27.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.640+0000 7f3d46200640 1 --2- 172.21.15.151:0/587969779 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3d4015e980 0x7f3d4006f1e0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f3d3800dac0 tx=0x7f3d3800df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:27.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.640+0000 7f3d3d600640 1 -- 172.21.15.151:0/587969779 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d38017070 con 0x7f3d4015e980 2024-09-16T15:52:27.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.640+0000 7f3d3d600640 1 -- 172.21.15.151:0/587969779 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3d38003970 con 0x7f3d4015e980 2024-09-16T15:52:27.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.640+0000 7f3d3d600640 1 -- 172.21.15.151:0/587969779 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d38003dc0 con 0x7f3d4015e980 2024-09-16T15:52:27.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.640+0000 7f3d48200640 1 -- 172.21.15.151:0/587969779 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d4006ff60 con 0x7f3d4015e980 2024-09-16T15:52:27.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.641+0000 7f3d48200640 1 -- 172.21.15.151:0/587969779 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d4006fc80 con 0x7f3d4015e980 2024-09-16T15:52:27.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.642+0000 7f3d48200640 1 -- 172.21.15.151:0/587969779 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d4015afb0 con 0x7f3d4015e980 2024-09-16T15:52:27.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.643+0000 7f3d3d600640 1 -- 172.21.15.151:0/587969779 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 41) ==== 103684+0+0 (secure 0 0 0) 0x7f3d38003f60 con 0x7f3d4015e980 2024-09-16T15:52:27.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.644+0000 7f3d3d600640 1 --2- 172.21.15.151:0/587969779 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3d2007a2c0 0x7f3d2007c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:27.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.644+0000 7f3d3d600640 1 -- 172.21.15.151:0/587969779 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5793+0+0 (secure 0 0 0) 0x7f3d38099c80 con 0x7f3d4015e980 2024-09-16T15:52:27.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.644+0000 7f3d45800640 1 --2- 172.21.15.151:0/587969779 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3d2007a2c0 0x7f3d2007c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:27.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.645+0000 7f3d45800640 1 --2- 172.21.15.151:0/587969779 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3d2007a2c0 0x7f3d2007c780 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f3d34005e00 tx=0x7f3d34005c70 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:27.650 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.648+0000 7f3d3d600640 1 -- 172.21.15.151:0/587969779 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f3d4015afb0 con 0x7f3d4015e980 2024-09-16T15:52:27.903 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.901+0000 7f3d48200640 1 -- 172.21.15.151:0/587969779 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f3d400cd5b0 con 0x7f3d4015e980 2024-09-16T15:52:27.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.904+0000 7f3d3d600640 1 -- 172.21.15.151:0/587969779 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) ==== 190+0+147 (secure 0 0 0) 0x7f3d400cd5b0 con 0x7f3d4015e980 2024-09-16T15:52:27.906 INFO:teuthology.orchestra.run.smithi151.stdout:[client.0] 2024-09-16T15:52:27.906 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQDsUOhmf5PBKBAAKE4Y4zbA4h07IBNjPkyptw== 2024-09-16T15:52:27.906 INFO:teuthology.orchestra.run.smithi151.stdout: caps mds = "allow" 2024-09-16T15:52:27.906 INFO:teuthology.orchestra.run.smithi151.stdout: caps mon = "allow r" 2024-09-16T15:52:27.906 INFO:teuthology.orchestra.run.smithi151.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-16T15:52:27.906 INFO:teuthology.orchestra.run.smithi151.stderr:updated caps for client.0 2024-09-16T15:52:27.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.906+0000 7f3d48200640 1 -- 172.21.15.151:0/587969779 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3d2007a2c0 msgr2=0x7f3d2007c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:27.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.906+0000 7f3d48200640 1 --2- 172.21.15.151:0/587969779 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3d2007a2c0 0x7f3d2007c780 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f3d34005e00 tx=0x7f3d34005c70 comp rx=0 tx=0).stop 2024-09-16T15:52:27.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.906+0000 7f3d48200640 1 -- 172.21.15.151:0/587969779 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3d4015e980 msgr2=0x7f3d4006f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:27.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.906+0000 7f3d48200640 1 --2- 172.21.15.151:0/587969779 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3d4015e980 0x7f3d4006f1e0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f3d3800dac0 tx=0x7f3d3800df80 comp rx=0 tx=0).stop 2024-09-16T15:52:27.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.906+0000 7f3d48200640 1 -- 172.21.15.151:0/587969779 shutdown_connections 2024-09-16T15:52:27.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.906+0000 7f3d48200640 1 --2- 172.21.15.151:0/587969779 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3d2007a2c0 0x7f3d2007c780 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:27.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.906+0000 7f3d48200640 1 --2- 172.21.15.151:0/587969779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d4015c080 0x7f3d40151d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:27.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.906+0000 7f3d48200640 1 --2- 172.21.15.151:0/587969779 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3d4015e980 0x7f3d4006f1e0 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:27.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.906+0000 7f3d48200640 1 --2- 172.21.15.151:0/587969779 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3d400b3370 0x7f3d40135cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:27.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.906+0000 7f3d48200640 1 -- 172.21.15.151:0/587969779 >> 172.21.15.151:0/587969779 conn(0x7f3d400b1840 msgr2=0x7f3d400ce000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:27.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.906+0000 7f3d48200640 1 -- 172.21.15.151:0/587969779 shutdown_connections 2024-09-16T15:52:27.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:27.906+0000 7f3d48200640 1 -- 172.21.15.151:0/587969779 wait complete. 2024-09-16T15:52:27.924 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:52:28.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.080+0000 7fc8b3e00640 1 -- 172.21.15.151:0/2811875313 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc894004800 msgr2=0x7fc894004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:28.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.080+0000 7fc8b3e00640 1 --2- 172.21.15.151:0/2811875313 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc894004800 0x7fc894004c80 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac0752e0 tx=0x7fc894005830 comp rx=0 tx=0).stop 2024-09-16T15:52:28.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.080+0000 7fc8b1e00640 1 -- 172.21.15.151:0/2811875313 reap_dead start 2024-09-16T15:52:28.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.080+0000 7fc8b3e00640 1 -- 172.21.15.151:0/2811875313 shutdown_connections 2024-09-16T15:52:28.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.080+0000 7fc8b3e00640 1 -- 172.21.15.151:0/2811875313 >> 172.21.15.151:0/2811875313 conn(0x7fc8ac0b1950 msgr2=0x7fc8ac0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:28.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.080+0000 7fc8b3e00640 1 -- 172.21.15.151:0/2811875313 shutdown_connections 2024-09-16T15:52:28.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.080+0000 7fc8b3e00640 1 -- 172.21.15.151:0/2811875313 wait complete. 2024-09-16T15:52:28.083 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.082+0000 7fc8b3e00640 1 Processor -- start 2024-09-16T15:52:28.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.082+0000 7fc8b3e00640 1 -- start start 2024-09-16T15:52:28.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.083+0000 7fc8b3e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc894004800 0x7fc8ac1f46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:28.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.083+0000 7fc8b3e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc8ac1536c0 0x7fc8ac1f7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:28.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.083+0000 7fc8b3e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8ac09aa00 0x7fc8ac09cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:28.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.083+0000 7fc8b3e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ac06ef50 con 0x7fc8ac09aa00 2024-09-16T15:52:28.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.083+0000 7fc8b3e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ac06fdb0 con 0x7fc894004800 2024-09-16T15:52:28.086 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.083+0000 7fc8b3e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fc8ac06f0d0 con 0x7fc8ac1536c0 2024-09-16T15:52:28.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.083+0000 7fc8b1e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8ac09aa00 0x7fc8ac09cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:28.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.083+0000 7fc8b1e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8ac09aa00 0x7fc8ac09cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41434/0 (socket says 172.21.15.151:41434) 2024-09-16T15:52:28.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.083+0000 7fc8b1e00640 1 -- 172.21.15.151:0/3355201938 learned_addr learned my addr 172.21.15.151:0/3355201938 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:52:28.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.083+0000 7fc8b0a00640 1 --2- 172.21.15.151:0/3355201938 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc8ac1536c0 0x7fc8ac1f7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:28.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.083+0000 7fc8b1400640 1 --2- 172.21.15.151:0/3355201938 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc894004800 0x7fc8ac1f46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:28.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.083+0000 7fc8b1e00640 1 -- 172.21.15.151:0/3355201938 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc8ac1536c0 msgr2=0x7fc8ac1f7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:28.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.083+0000 7fc8b1e00640 1 --2- 172.21.15.151:0/3355201938 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc8ac1536c0 0x7fc8ac1f7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:28.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.083+0000 7fc8b1e00640 1 -- 172.21.15.151:0/3355201938 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc894004800 msgr2=0x7fc8ac1f46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:28.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.083+0000 7fc8b1e00640 1 --2- 172.21.15.151:0/3355201938 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc894004800 0x7fc8ac1f46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:28.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.083+0000 7fc8b1e00640 1 -- 172.21.15.151:0/3355201938 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8ac153b00 con 0x7fc8ac09aa00 2024-09-16T15:52:28.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.084+0000 7fc8b1e00640 1 --2- 172.21.15.151:0/3355201938 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8ac09aa00 0x7fc8ac09cb50 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4005fd0 tx=0x7fc8a4011f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:28.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.084+0000 7fc8a9600640 1 -- 172.21.15.151:0/3355201938 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a4004b20 con 0x7fc8ac09aa00 2024-09-16T15:52:28.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.084+0000 7fc8a9600640 1 -- 172.21.15.151:0/3355201938 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc8a4004cc0 con 0x7fc8ac09aa00 2024-09-16T15:52:28.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.084+0000 7fc8b3e00640 1 -- 172.21.15.151:0/3355201938 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8ac1f8410 con 0x7fc8ac09aa00 2024-09-16T15:52:28.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.084+0000 7fc8a9600640 1 -- 172.21.15.151:0/3355201938 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a4003bd0 con 0x7fc8ac09aa00 2024-09-16T15:52:28.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.084+0000 7fc8b3e00640 1 -- 172.21.15.151:0/3355201938 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8ac1f89d0 con 0x7fc8ac09aa00 2024-09-16T15:52:28.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.085+0000 7fc8b3e00640 1 -- 172.21.15.151:0/3355201938 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8ac0732a0 con 0x7fc8ac09aa00 2024-09-16T15:52:28.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.086+0000 7fc8a9600640 1 -- 172.21.15.151:0/3355201938 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 41) ==== 103684+0+0 (secure 0 0 0) 0x7fc8a4004030 con 0x7fc8ac09aa00 2024-09-16T15:52:28.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.087+0000 7fc8a9600640 1 --2- 172.21.15.151:0/3355201938 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc88c07a2f0 0x7fc88c07c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:28.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.087+0000 7fc8a9600640 1 -- 172.21.15.151:0/3355201938 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5793+0+0 (secure 0 0 0) 0x7fc8a409a640 con 0x7fc8ac09aa00 2024-09-16T15:52:28.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.088+0000 7fc8b1400640 1 --2- 172.21.15.151:0/3355201938 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc88c07a2f0 0x7fc88c07c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:28.090 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.088+0000 7fc8b1400640 1 --2- 172.21.15.151:0/3355201938 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc88c07a2f0 0x7fc88c07c7b0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fc8a0009af0 tx=0x7fc8a0005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:28.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.091+0000 7fc8a9600640 1 -- 172.21.15.151:0/3355201938 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc8a401f060 con 0x7fc8ac09aa00 2024-09-16T15:52:28.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.312+0000 7fc8b3e00640 1 -- 172.21.15.151:0/3355201938 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc8ac0b4000 con 0x7fc8ac09aa00 2024-09-16T15:52:28.315 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.313+0000 7fc8a9600640 1 -- 172.21.15.151:0/3355201938 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+5237 (secure 0 0 0) 0x7fc8a4064710 con 0x7fc8ac09aa00 2024-09-16T15:52:28.315 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:52:28.315 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":76,"btime":"2024-09-16T15:52:21:110724+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7034,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1802374613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1802374613},{"type":"v1","addr":"172.21.15.167:6837","nonce":1802374613}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":7753,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3585191088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3585191088},{"type":"v1","addr":"172.21.15.167:6836","nonce":3585191088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":8457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/1321222227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1321222227},{"type":"v1","addr":"172.21.15.151:6834","nonce":1321222227}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:52:17.489987+0000","modified":"2024-09-16T15:52:20.525094+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8460":{"gid":8460,"name":"a","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.151:6835/2504927912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":2504927912},{"type":"v1","addr":"172.21.15.151:6835","nonce":2504927912}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8460,"qdb_cluster":[8460]},"id":8}]} 2024-09-16T15:52:28.315 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 76 2024-09-16T15:52:28.318 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.315+0000 7fc8b3e00640 1 -- 172.21.15.151:0/3355201938 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc88c07a2f0 msgr2=0x7fc88c07c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:28.318 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.315+0000 7fc8b3e00640 1 --2- 172.21.15.151:0/3355201938 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc88c07a2f0 0x7fc88c07c7b0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fc8a0009af0 tx=0x7fc8a0005f90 comp rx=0 tx=0).stop 2024-09-16T15:52:28.318 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.315+0000 7fc8b3e00640 1 -- 172.21.15.151:0/3355201938 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8ac09aa00 msgr2=0x7fc8ac09cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:28.318 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.315+0000 7fc8b3e00640 1 --2- 172.21.15.151:0/3355201938 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8ac09aa00 0x7fc8ac09cb50 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4005fd0 tx=0x7fc8a4011f80 comp rx=0 tx=0).stop 2024-09-16T15:52:28.318 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.315+0000 7fc8b3e00640 1 -- 172.21.15.151:0/3355201938 shutdown_connections 2024-09-16T15:52:28.318 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.315+0000 7fc8b3e00640 1 --2- 172.21.15.151:0/3355201938 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc88c07a2f0 0x7fc88c07c7b0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:28.318 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.315+0000 7fc8b3e00640 1 --2- 172.21.15.151:0/3355201938 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8ac09aa00 0x7fc8ac09cb50 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:28.318 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.315+0000 7fc8b3e00640 1 --2- 172.21.15.151:0/3355201938 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc8ac1536c0 0x7fc8ac1f7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:28.318 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.315+0000 7fc8b3e00640 1 --2- 172.21.15.151:0/3355201938 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc894004800 0x7fc8ac1f46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:28.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.315+0000 7fc8b3e00640 1 -- 172.21.15.151:0/3355201938 >> 172.21.15.151:0/3355201938 conn(0x7fc8ac0b1950 msgr2=0x7fc8ac15ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:28.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.316+0000 7fc8b3e00640 1 -- 172.21.15.151:0/3355201938 shutdown_connections 2024-09-16T15:52:28.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.316+0000 7fc8b3e00640 1 -- 172.21.15.151:0/3355201938 wait complete. 2024-09-16T15:52:28.737 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 76, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-16T15:52:17.489987+0000', 'modified': '2024-09-16T15:52:20.525094+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8460}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8460': {'gid': 8460, 'name': 'a', 'rank': 0, 'incarnation': 72, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.151:6835/2504927912', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6833', 'nonce': 2504927912}, {'type': 'v1', 'addr': '172.21.15.151:6835', 'nonce': 2504927912}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8460, 'qdb_cluster': [8460]} 2024-09-16T15:52:28.816 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-16T15:52:28.816 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-16T15:52:28.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.973+0000 7fb183400640 1 -- 172.21.15.151:0/2577224797 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb17c0ccf00 msgr2=0x7fb17c0b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:28.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.973+0000 7fb183400640 1 --2- 172.21.15.151:0/2577224797 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb17c0ccf00 0x7fb17c0b2440 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fb168009830 tx=0x7fb16802a480 comp rx=0 tx=0).stop 2024-09-16T15:52:28.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.973+0000 7fb183400640 1 -- 172.21.15.151:0/2577224797 shutdown_connections 2024-09-16T15:52:28.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.973+0000 7fb183400640 1 --2- 172.21.15.151:0/2577224797 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb17c073fb0 0x7fb17c074440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:28.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.973+0000 7fb183400640 1 --2- 172.21.15.151:0/2577224797 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb17c0ccf00 0x7fb17c0b2440 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:28.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.973+0000 7fb183400640 1 -- 172.21.15.151:0/2577224797 >> 172.21.15.151:0/2577224797 conn(0x7fb17c0b1950 msgr2=0x7fb17c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:28.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.973+0000 7fb183400640 1 -- 172.21.15.151:0/2577224797 shutdown_connections 2024-09-16T15:52:28.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.973+0000 7fb183400640 1 -- 172.21.15.151:0/2577224797 wait complete. 2024-09-16T15:52:28.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.975+0000 7fb183400640 1 Processor -- start 2024-09-16T15:52:28.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.975+0000 7fb183400640 1 -- start start 2024-09-16T15:52:28.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.976+0000 7fb183400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb17c073fb0 0x7fb17c0b46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:28.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.976+0000 7fb183400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb17c0ccf00 0x7fb17c0cf890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:28.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.976+0000 7fb183400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb17c15f5a0 0x7fb17c1f4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:28.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.976+0000 7fb183400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb17c06f2e0 con 0x7fb17c073fb0 2024-09-16T15:52:28.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.976+0000 7fb183400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb17c06f160 con 0x7fb17c0ccf00 2024-09-16T15:52:28.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.976+0000 7fb183400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fb17c06f460 con 0x7fb17c15f5a0 2024-09-16T15:52:28.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.976+0000 7fb180a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb17c073fb0 0x7fb17c0b46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:28.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.976+0000 7fb180a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb17c073fb0 0x7fb17c0b46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41450/0 (socket says 172.21.15.151:41450) 2024-09-16T15:52:28.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.976+0000 7fb180a00640 1 -- 172.21.15.151:0/636342493 learned_addr learned my addr 172.21.15.151:0/636342493 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:52:28.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.977+0000 7fb17be00640 1 --2- 172.21.15.151:0/636342493 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb17c0ccf00 0x7fb17c0cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:28.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.977+0000 7fb180a00640 1 -- 172.21.15.151:0/636342493 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb17c15f5a0 msgr2=0x7fb17c1f4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:28.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.977+0000 7fb181400640 1 --2- 172.21.15.151:0/636342493 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb17c15f5a0 0x7fb17c1f4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:28.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.977+0000 7fb180a00640 1 --2- 172.21.15.151:0/636342493 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb17c15f5a0 0x7fb17c1f4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:28.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.977+0000 7fb180a00640 1 -- 172.21.15.151:0/636342493 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb17c0ccf00 msgr2=0x7fb17c0cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:28.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.977+0000 7fb180a00640 1 --2- 172.21.15.151:0/636342493 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb17c0ccf00 0x7fb17c0cf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:28.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.977+0000 7fb180a00640 1 -- 172.21.15.151:0/636342493 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb17c1f7b50 con 0x7fb17c073fb0 2024-09-16T15:52:28.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.977+0000 7fb17be00640 1 --2- 172.21.15.151:0/636342493 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb17c0ccf00 0x7fb17c0cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:52:28.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.977+0000 7fb181400640 1 --2- 172.21.15.151:0/636342493 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb17c15f5a0 0x7fb17c1f4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:52:28.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.977+0000 7fb180a00640 1 --2- 172.21.15.151:0/636342493 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb17c073fb0 0x7fb17c0b46e0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fb16c00c3c0 tx=0x7fb16c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:28.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.977+0000 7fb178c00640 1 -- 172.21.15.151:0/636342493 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16c017070 con 0x7fb17c073fb0 2024-09-16T15:52:28.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.977+0000 7fb178c00640 1 -- 172.21.15.151:0/636342493 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb16c0035f0 con 0x7fb17c073fb0 2024-09-16T15:52:28.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.977+0000 7fb178c00640 1 -- 172.21.15.151:0/636342493 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16c003a80 con 0x7fb17c073fb0 2024-09-16T15:52:28.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.977+0000 7fb183400640 1 -- 172.21.15.151:0/636342493 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb17c0b31c0 con 0x7fb17c073fb0 2024-09-16T15:52:28.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.977+0000 7fb183400640 1 -- 172.21.15.151:0/636342493 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb17c0b3a60 con 0x7fb17c073fb0 2024-09-16T15:52:28.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.980+0000 7fb178c00640 1 -- 172.21.15.151:0/636342493 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 41) ==== 103684+0+0 (secure 0 0 0) 0x7fb16c01a070 con 0x7fb17c073fb0 2024-09-16T15:52:28.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.981+0000 7fb178c00640 1 --2- 172.21.15.151:0/636342493 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb15c07a2c0 0x7fb15c07c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:28.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.981+0000 7fb178c00640 1 -- 172.21.15.151:0/636342493 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5793+0+0 (secure 0 0 0) 0x7fb16c098ae0 con 0x7fb17c073fb0 2024-09-16T15:52:28.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.981+0000 7fb1862b5740 1 Processor -- start 2024-09-16T15:52:28.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.981+0000 7fb1862b5740 1 -- start start 2024-09-16T15:52:28.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.981+0000 7fb17be00640 1 --2- 172.21.15.151:0/636342493 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb15c07a2c0 0x7fb15c07c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:28.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.982+0000 7fb1862b5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567e97d28b0 0x5567e97d2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:28.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.982+0000 7fb17be00640 1 --2- 172.21.15.151:0/636342493 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb15c07a2c0 0x7fb15c07c780 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fb168034000 tx=0x7fb168004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:28.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.982+0000 7fb1862b5740 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5567e978da20 0x5567e97d1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:28.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.982+0000 7fb1862b5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567e978dfa0 0x5567e97d18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:28.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.982+0000 7fb1862b5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5567e9710430 con 0x5567e97d28b0 2024-09-16T15:52:28.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.982+0000 7fb1862b5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5567e97005d0 con 0x5567e978dfa0 2024-09-16T15:52:28.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.982+0000 7fb1862b5740 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x5567e96efa00 con 0x5567e978da20 2024-09-16T15:52:28.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.982+0000 7fb180a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567e97d28b0 0x5567e97d2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:28.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.982+0000 7fb180a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567e97d28b0 0x5567e97d2c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41462/0 (socket says 172.21.15.151:41462) 2024-09-16T15:52:28.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.982+0000 7fb180a00640 1 -- 172.21.15.151:0/3436513059 learned_addr learned my addr 172.21.15.151:0/3436513059 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:52:28.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.982+0000 7fb181400640 1 --2- 172.21.15.151:0/3436513059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567e978dfa0 0x5567e97d18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:28.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.982+0000 7fb17be00640 1 --2- 172.21.15.151:0/3436513059 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5567e978da20 0x5567e97d1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:28.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.983+0000 7fb180a00640 1 -- 172.21.15.151:0/3436513059 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5567e978da20 msgr2=0x5567e97d1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:52:28.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.983+0000 7fb180a00640 1 --2- 172.21.15.151:0/3436513059 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5567e978da20 0x5567e97d1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:28.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.983+0000 7fb180a00640 1 -- 172.21.15.151:0/3436513059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567e978dfa0 msgr2=0x5567e97d18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:52:28.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.983+0000 7fb180a00640 1 --2- 172.21.15.151:0/3436513059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567e978dfa0 0x5567e97d18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:28.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.983+0000 7fb180a00640 1 -- 172.21.15.151:0/3436513059 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567e96b8520 con 0x5567e97d28b0 2024-09-16T15:52:28.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.983+0000 7fb180a00640 1 --2- 172.21.15.151:0/3436513059 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567e97d28b0 0x5567e97d2c90 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fb16c00c390 tx=0x7fb16c062080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3385dea865de07e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:28.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.983+0000 7fb171200640 1 -- 172.21.15.151:0/3436513059 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16c06bbb0 con 0x5567e97d28b0 2024-09-16T15:52:28.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.983+0000 7fb171200640 1 -- 172.21.15.151:0/3436513059 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb16c066f90 con 0x5567e97d28b0 2024-09-16T15:52:28.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.983+0000 7fb171200640 1 -- 172.21.15.151:0/3436513059 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16c061970 con 0x5567e97d28b0 2024-09-16T15:52:28.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.984+0000 7fb1862b5740 1 -- 172.21.15.151:0/3436513059 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567e97d28b0 msgr2=0x5567e97d2c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:28.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.984+0000 7fb1862b5740 1 --2- 172.21.15.151:0/3436513059 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567e97d28b0 0x5567e97d2c90 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fb16c00c390 tx=0x7fb16c062080 comp rx=0 tx=0).stop 2024-09-16T15:52:28.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.984+0000 7fb1862b5740 1 -- 172.21.15.151:0/3436513059 shutdown_connections 2024-09-16T15:52:28.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.984+0000 7fb1862b5740 1 --2- 172.21.15.151:0/3436513059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567e978dfa0 0x5567e97d18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:28.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.984+0000 7fb1862b5740 1 --2- 172.21.15.151:0/3436513059 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5567e978da20 0x5567e97d1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:28.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.984+0000 7fb1862b5740 1 --2- 172.21.15.151:0/3436513059 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567e97d28b0 0x5567e97d2c90 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:28.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.984+0000 7fb1862b5740 1 -- 172.21.15.151:0/3436513059 >> 172.21.15.151:0/3436513059 conn(0x5567e9788ff0 msgr2=0x5567e978ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:28.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.984+0000 7fb1862b5740 1 -- 172.21.15.151:0/3436513059 shutdown_connections 2024-09-16T15:52:28.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.984+0000 7fb1862b5740 1 -- 172.21.15.151:0/3436513059 wait complete. 2024-09-16T15:52:28.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.985+0000 7fb1862b5740 1 -- start start 2024-09-16T15:52:28.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.985+0000 7fb1862b5740 1 Processor -- start 2024-09-16T15:52:28.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.986+0000 7fb1862b5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567e978dfa0 0x5567e97e54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:28.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.986+0000 7fb1862b5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567e97d28b0 0x5567e97a1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:28.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.986+0000 7fb1862b5740 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5567e97a1b40 0x5567e97a7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:28.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.986+0000 7fb1862b5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5567e9696330 con 0x5567e97d28b0 2024-09-16T15:52:28.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.986+0000 7fb1862b5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5567e96f3810 con 0x5567e978dfa0 2024-09-16T15:52:28.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.986+0000 7fb1862b5740 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x5567e9460620 con 0x5567e97a1b40 2024-09-16T15:52:28.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.986+0000 7fb1862b5740 10 client.0 _ll_register_callbacks cb 0x5567e969b420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T15:52:28.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.986+0000 7fb17be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567e97d28b0 0x5567e97a1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:28.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.986+0000 7fb17be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567e97d28b0 0x5567e97a1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41464/0 (socket says 172.21.15.151:41464) 2024-09-16T15:52:28.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.986+0000 7fb17be00640 1 -- 172.21.15.151:0/818333111 learned_addr learned my addr 172.21.15.151:0/818333111 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:52:28.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.986+0000 7fb180a00640 1 --2- 172.21.15.151:0/818333111 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567e978dfa0 0x5567e97e54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:28.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.986+0000 7fb181400640 1 --2- 172.21.15.151:0/818333111 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5567e97a1b40 0x5567e97a7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:28.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.986+0000 7fb17be00640 1 -- 172.21.15.151:0/818333111 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5567e97a1b40 msgr2=0x5567e97a7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:52:28.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.987+0000 7fb17be00640 1 --2- 172.21.15.151:0/818333111 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5567e97a1b40 0x5567e97a7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:28.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.987+0000 7fb17be00640 1 -- 172.21.15.151:0/818333111 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567e978dfa0 msgr2=0x5567e97e54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:52:28.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.987+0000 7fb17be00640 1 --2- 172.21.15.151:0/818333111 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567e978dfa0 0x5567e97e54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:28.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.987+0000 7fb17be00640 1 -- 172.21.15.151:0/818333111 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567e9451850 con 0x5567e97d28b0 2024-09-16T15:52:28.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.987+0000 7fb181400640 1 --2- 172.21.15.151:0/818333111 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5567e97a1b40 0x5567e97a7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:52:28.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.987+0000 7fb180a00640 1 --2- 172.21.15.151:0/818333111 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567e978dfa0 0x5567e97e54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:52:28.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.987+0000 7fb17be00640 1 --2- 172.21.15.151:0/818333111 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567e97d28b0 0x5567e97a1600 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fb16802aa70 tx=0x7fb168005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e20bdd0d69b2ebf server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:28.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.987+0000 7fb165600640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-16T15:52:28.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.987+0000 7fb165600640 1 -- 172.21.15.151:0/818333111 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb168049840 con 0x5567e97d28b0 2024-09-16T15:52:28.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.987+0000 7fb1862b5740 1 -- 172.21.15.151:0/818333111 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb16c066f90 con 0x5567e97d28b0 2024-09-16T15:52:28.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.987+0000 7fb165600640 1 -- 172.21.15.151:0/818333111 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 5793+0+0 (secure 0 0 0) 0x7fb1680499e0 con 0x5567e97d28b0 2024-09-16T15:52:28.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.987+0000 7fb165600640 1 -- 172.21.15.151:0/818333111 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fb168059050 con 0x5567e97d28b0 2024-09-16T15:52:28.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.987+0000 7fb1862b5740 10 client.8610 fetch_fsmap learned FSMap version 76 2024-09-16T15:52:28.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.987+0000 7fb1862b5740 1 -- 172.21.15.151:0/818333111 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) -- 0x5567e94c86b0 con 0x5567e97d28b0 2024-09-16T15:52:28.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.988+0000 7fb165600640 1 -- 172.21.15.151:0/818333111 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb16c066f90 con 0x5567e97d28b0 2024-09-16T15:52:28.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.988+0000 7fb165600640 1 -- 172.21.15.151:0/818333111 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 76) ==== 3502+0+0 (secure 0 0 0) 0x7fb168049cd0 con 0x5567e97d28b0 2024-09-16T15:52:28.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.988+0000 7fb1862b5740 10 client.8610 fetch_fsmap finished waiting for FSMap version 76 2024-09-16T15:52:28.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.988+0000 7fb1862b5740 10 client.8610 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-16T15:52:28.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.988+0000 7fb1862b5740 20 client.8610 populate_metadata read hostname 'smithi151' 2024-09-16T15:52:28.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.988+0000 7fb1862b5740 1 --2- 172.21.15.151:0/818333111 >> [v2:172.21.15.151:6833/2504927912,v1:172.21.15.151:6835/2504927912] conn(0x5567e97d17b0 0x5567e97d1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:28.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.988+0000 7fb1862b5740 4 client.8610 mds_command: new command op to 8460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:52:28.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.988+0000 7fb1862b5740 1 -- 172.21.15.151:0/818333111 --> [v2:172.21.15.151:6833/2504927912,v1:172.21.15.151:6835/2504927912] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567e9702520 con 0x5567e97d17b0 2024-09-16T15:52:28.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.988+0000 7fb180a00640 1 --2- 172.21.15.151:0/818333111 >> [v2:172.21.15.151:6833/2504927912,v1:172.21.15.151:6835/2504927912] conn(0x5567e97d17b0 0x5567e97d1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:28.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.988+0000 7fb180a00640 1 --2- 172.21.15.151:0/818333111 >> [v2:172.21.15.151:6833/2504927912,v1:172.21.15.151:6835/2504927912] conn(0x5567e97d17b0 0x5567e97d1b90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c25344e0c865d40 server_cookie=254b01ab2989a5fb in_seq=0 out_seq=0 2024-09-16T15:52:28.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.988+0000 7fb165600640 10 client.8610 ms_handle_connect on v2:172.21.15.151:6833/2504927912 2024-09-16T15:52:28.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.989+0000 7fb165600640 1 -- 172.21.15.151:0/818333111 <== mds.0 v2:172.21.15.151:6833/2504927912 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5567e9702520 con 0x5567e97d17b0 2024-09-16T15:52:28.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.989+0000 7fb165600640 10 client.8610 handle_command_reply: tid=0 2024-09-16T15:52:28.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.989+0000 7fb1862b5740 1 client.8610 shutdown 2024-09-16T15:52:28.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.989+0000 7fb1862b5740 1 -- 172.21.15.151:0/818333111 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567e97d28b0 msgr2=0x5567e97a1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:28.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.989+0000 7fb1862b5740 1 --2- 172.21.15.151:0/818333111 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567e97d28b0 0x5567e97a1600 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fb16802aa70 tx=0x7fb168005f90 comp rx=0 tx=0).stop 2024-09-16T15:52:28.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.990+0000 7fb1862b5740 1 -- 172.21.15.151:0/818333111 shutdown_connections 2024-09-16T15:52:28.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.990+0000 7fb1862b5740 1 --2- 172.21.15.151:0/818333111 >> [v2:172.21.15.151:6833/2504927912,v1:172.21.15.151:6835/2504927912] conn(0x5567e97d17b0 0x5567e97d1b90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:28.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.990+0000 7fb1862b5740 1 --2- 172.21.15.151:0/818333111 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5567e97a1b40 0x5567e97a7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:28.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.990+0000 7fb1862b5740 1 --2- 172.21.15.151:0/818333111 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567e97d28b0 0x5567e97a1600 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:28.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.990+0000 7fb165600640 0 client.8610 ms_handle_reset on v2:172.21.15.151:6833/2504927912 2024-09-16T15:52:28.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.990+0000 7fb1862b5740 1 --2- 172.21.15.151:0/818333111 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567e978dfa0 0x5567e97e54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:28.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.990+0000 7fb1862b5740 1 -- 172.21.15.151:0/818333111 >> 172.21.15.151:0/818333111 conn(0x5567e978da20 msgr2=0x5567e97cb420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:28.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.990+0000 7fb1862b5740 1 -- 172.21.15.151:0/818333111 shutdown_connections 2024-09-16T15:52:28.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.990+0000 7fb1862b5740 1 -- 172.21.15.151:0/818333111 wait complete. 2024-09-16T15:52:28.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:28.990+0000 7fb1862b5740 20 client.8610 trim_cache size 0 max 16384 2024-09-16T15:52:29.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.009+0000 7fb1862b5740 1 Processor -- start 2024-09-16T15:52:29.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.009+0000 7fb1862b5740 1 -- start start 2024-09-16T15:52:29.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.010+0000 7fb1862b5740 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5567e978dfa0 0x5567e978c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:29.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.010+0000 7fb1862b5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567e97a1b40 0x5567e97c7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:29.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.010+0000 7fb1862b5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567e97d28b0 0x5567e97b4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:29.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.010+0000 7fb1862b5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb17c0cde90 con 0x5567e97a1b40 2024-09-16T15:52:29.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.010+0000 7fb1862b5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb17c14ebb0 con 0x5567e97d28b0 2024-09-16T15:52:29.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.010+0000 7fb1862b5740 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fb17c1f8940 con 0x5567e978dfa0 2024-09-16T15:52:29.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.010+0000 7fb17be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567e97a1b40 0x5567e97c7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:29.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.010+0000 7fb17be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567e97a1b40 0x5567e97c7940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41472/0 (socket says 172.21.15.151:41472) 2024-09-16T15:52:29.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.010+0000 7fb17be00640 1 -- 172.21.15.151:0/3948937817 learned_addr learned my addr 172.21.15.151:0/3948937817 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:52:29.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.010+0000 7fb180a00640 1 --2- 172.21.15.151:0/3948937817 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5567e978dfa0 0x5567e978c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:29.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.010+0000 7fb181400640 1 --2- 172.21.15.151:0/3948937817 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567e97d28b0 0x5567e97b4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:29.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.010+0000 7fb17be00640 1 -- 172.21.15.151:0/3948937817 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5567e978dfa0 msgr2=0x5567e978c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:52:29.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.010+0000 7fb17be00640 1 --2- 172.21.15.151:0/3948937817 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5567e978dfa0 0x5567e978c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:29.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.010+0000 7fb17be00640 1 -- 172.21.15.151:0/3948937817 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567e97d28b0 msgr2=0x5567e97b4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:52:29.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.010+0000 7fb17be00640 1 --2- 172.21.15.151:0/3948937817 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567e97d28b0 0x5567e97b4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:29.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.010+0000 7fb17be00640 1 -- 172.21.15.151:0/3948937817 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567e97d4320 con 0x5567e97a1b40 2024-09-16T15:52:29.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.011+0000 7fb17be00640 1 --2- 172.21.15.151:0/3948937817 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567e97a1b40 0x5567e97c7940 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fb16803b970 tx=0x7fb16803b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb5508f9aa305a35 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:29.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.011+0000 7fb167e00640 1 -- 172.21.15.151:0/3948937817 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16803bdc0 con 0x5567e97a1b40 2024-09-16T15:52:29.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.011+0000 7fb167e00640 1 -- 172.21.15.151:0/3948937817 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb168049640 con 0x5567e97a1b40 2024-09-16T15:52:29.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.011+0000 7fb167e00640 1 -- 172.21.15.151:0/3948937817 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb168049a70 con 0x5567e97a1b40 2024-09-16T15:52:29.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.011+0000 7fb1862b5740 1 -- 172.21.15.151:0/3948937817 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567e97a1b40 msgr2=0x5567e97c7940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:29.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.011+0000 7fb1862b5740 1 --2- 172.21.15.151:0/3948937817 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567e97a1b40 0x5567e97c7940 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fb16803b970 tx=0x7fb16803b9a0 comp rx=0 tx=0).stop 2024-09-16T15:52:29.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.011+0000 7fb1862b5740 1 -- 172.21.15.151:0/3948937817 shutdown_connections 2024-09-16T15:52:29.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.011+0000 7fb1862b5740 1 --2- 172.21.15.151:0/3948937817 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567e97d28b0 0x5567e97b4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:29.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.012+0000 7fb1862b5740 1 --2- 172.21.15.151:0/3948937817 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567e97a1b40 0x5567e97c7940 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:29.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.012+0000 7fb1862b5740 1 --2- 172.21.15.151:0/3948937817 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5567e978dfa0 0x5567e978c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:29.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.012+0000 7fb1862b5740 1 -- 172.21.15.151:0/3948937817 >> 172.21.15.151:0/3948937817 conn(0x5567e978da20 msgr2=0x5567e97d1d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:29.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.012+0000 7fb1862b5740 1 -- 172.21.15.151:0/3948937817 shutdown_connections 2024-09-16T15:52:29.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.012+0000 7fb1862b5740 1 -- 172.21.15.151:0/3948937817 wait complete. 2024-09-16T15:52:29.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.012+0000 7fb1862b5740 1 -- start start 2024-09-16T15:52:29.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.012+0000 7fb1862b5740 1 Processor -- start 2024-09-16T15:52:29.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.013+0000 7fb1862b5740 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5567e978dfa0 0x5567e9851ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:29.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.013+0000 7fb1862b5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567e97a1b40 0x5567e97e3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:29.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.013+0000 7fb1862b5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567e97d28b0 0x5567e97e3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:29.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.013+0000 7fb1862b5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5567e97b5450 con 0x5567e97d28b0 2024-09-16T15:52:29.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.013+0000 7fb1862b5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5567e97a4fd0 con 0x5567e97a1b40 2024-09-16T15:52:29.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.013+0000 7fb1862b5740 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x5567e97e5a80 con 0x5567e978dfa0 2024-09-16T15:52:29.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.013+0000 7fb1862b5740 10 client.0 _ll_register_callbacks cb 0x7fb168003a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T15:52:29.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.014+0000 7fb181400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567e97d28b0 0x5567e97e3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:29.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.014+0000 7fb181400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567e97d28b0 0x5567e97e3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41478/0 (socket says 172.21.15.151:41478) 2024-09-16T15:52:29.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.014+0000 7fb181400640 1 -- 172.21.15.151:0/1116723435 learned_addr learned my addr 172.21.15.151:0/1116723435 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:52:29.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.014+0000 7fb180a00640 1 --2- 172.21.15.151:0/1116723435 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5567e978dfa0 0x5567e9851ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:29.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.014+0000 7fb17be00640 1 --2- 172.21.15.151:0/1116723435 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567e97a1b40 0x5567e97e3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:29.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.014+0000 7fb181400640 1 -- 172.21.15.151:0/1116723435 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5567e978dfa0 msgr2=0x5567e9851ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:52:29.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.014+0000 7fb181400640 1 --2- 172.21.15.151:0/1116723435 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5567e978dfa0 0x5567e9851ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:29.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.014+0000 7fb181400640 1 -- 172.21.15.151:0/1116723435 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567e97a1b40 msgr2=0x5567e97e3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:52:29.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.014+0000 7fb181400640 1 --2- 172.21.15.151:0/1116723435 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567e97a1b40 0x5567e97e3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:29.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.014+0000 7fb181400640 1 -- 172.21.15.151:0/1116723435 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567e97ce700 con 0x5567e97d28b0 2024-09-16T15:52:29.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.014+0000 7fb181400640 1 --2- 172.21.15.151:0/1116723435 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567e97d28b0 0x5567e97e3d40 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fb174013850 tx=0x7fb174013d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11370a86244bf00b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:29.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.014+0000 7fb165600640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-16T15:52:29.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.015+0000 7fb165600640 1 -- 172.21.15.151:0/1116723435 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb174021060 con 0x5567e97d28b0 2024-09-16T15:52:29.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.015+0000 7fb165600640 1 -- 172.21.15.151:0/1116723435 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 5793+0+0 (secure 0 0 0) 0x7fb174011b30 con 0x5567e97d28b0 2024-09-16T15:52:29.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.015+0000 7fb1862b5740 1 -- 172.21.15.151:0/1116723435 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb168049640 con 0x5567e97d28b0 2024-09-16T15:52:29.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.015+0000 7fb165600640 1 -- 172.21.15.151:0/1116723435 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fb174027050 con 0x5567e97d28b0 2024-09-16T15:52:29.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.015+0000 7fb1862b5740 10 client.8616 fetch_fsmap learned FSMap version 76 2024-09-16T15:52:29.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.015+0000 7fb1862b5740 1 -- 172.21.15.151:0/1116723435 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) -- 0x5567e9786180 con 0x5567e97d28b0 2024-09-16T15:52:29.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.015+0000 7fb165600640 1 -- 172.21.15.151:0/1116723435 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb168049640 con 0x5567e97d28b0 2024-09-16T15:52:29.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.015+0000 7fb165600640 1 -- 172.21.15.151:0/1116723435 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 76) ==== 3502+0+0 (secure 0 0 0) 0x7fb17402a020 con 0x5567e97d28b0 2024-09-16T15:52:29.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.015+0000 7fb1862b5740 10 client.8616 fetch_fsmap finished waiting for FSMap version 76 2024-09-16T15:52:29.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.015+0000 7fb1862b5740 10 client.8616 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-16T15:52:29.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.015+0000 7fb1862b5740 20 client.8616 populate_metadata read hostname 'smithi151' 2024-09-16T15:52:29.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.015+0000 7fb1862b5740 1 --2- 172.21.15.151:0/1116723435 >> [v2:172.21.15.151:6833/2504927912,v1:172.21.15.151:6835/2504927912] conn(0x5567e97b62b0 0x5567e97ca5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:29.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.015+0000 7fb1862b5740 4 client.8616 mds_command: new command op to 8460 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T15:52:29.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.015+0000 7fb1862b5740 1 -- 172.21.15.151:0/1116723435 --> [v2:172.21.15.151:6833/2504927912,v1:172.21.15.151:6835/2504927912] -- command(tid 0: {"prefix": "status"}) -- 0x5567e96fdfd0 con 0x5567e97b62b0 2024-09-16T15:52:29.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.016+0000 7fb180a00640 1 --2- 172.21.15.151:0/1116723435 >> [v2:172.21.15.151:6833/2504927912,v1:172.21.15.151:6835/2504927912] conn(0x5567e97b62b0 0x5567e97ca5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:29.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.016+0000 7fb180a00640 1 --2- 172.21.15.151:0/1116723435 >> [v2:172.21.15.151:6833/2504927912,v1:172.21.15.151:6835/2504927912] conn(0x5567e97b62b0 0x5567e97ca5e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a968430b5274bd23 server_cookie=fd5bfa4f4db2697 in_seq=0 out_seq=0 2024-09-16T15:52:29.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.016+0000 7fb165600640 10 client.8616 ms_handle_connect on v2:172.21.15.151:6833/2504927912 2024-09-16T15:52:29.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.016+0000 7fb165600640 1 -- 172.21.15.151:0/1116723435 <== mds.0 v2:172.21.15.151:6833/2504927912 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5567e96fdfd0 con 0x5567e97b62b0 2024-09-16T15:52:29.023 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.016+0000 7fb165600640 10 client.8616 handle_command_reply: tid=0 2024-09-16T15:52:29.024 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.016+0000 7fb1862b5740 1 client.8616 shutdown 2024-09-16T15:52:29.024 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.016+0000 7fb1862b5740 1 -- 172.21.15.151:0/1116723435 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567e97d28b0 msgr2=0x5567e97e3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:29.024 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.016+0000 7fb1862b5740 1 --2- 172.21.15.151:0/1116723435 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567e97d28b0 0x5567e97e3d40 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fb174013850 tx=0x7fb174013d10 comp rx=0 tx=0).stop 2024-09-16T15:52:29.024 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.017+0000 7fb1862b5740 1 -- 172.21.15.151:0/1116723435 shutdown_connections 2024-09-16T15:52:29.024 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.017+0000 7fb1862b5740 1 --2- 172.21.15.151:0/1116723435 >> [v2:172.21.15.151:6833/2504927912,v1:172.21.15.151:6835/2504927912] conn(0x5567e97b62b0 0x5567e97ca5e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:29.024 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.017+0000 7fb1862b5740 1 --2- 172.21.15.151:0/1116723435 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5567e97a1b40 0x5567e97e3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:29.024 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.017+0000 7fb1862b5740 1 --2- 172.21.15.151:0/1116723435 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5567e97d28b0 0x5567e97e3d40 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:29.024 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.017+0000 7fb1862b5740 1 --2- 172.21.15.151:0/1116723435 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x5567e978dfa0 0x5567e9851ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:29.024 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.017+0000 7fb165600640 0 client.8616 ms_handle_reset on v2:172.21.15.151:6833/2504927912 2024-09-16T15:52:29.024 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.017+0000 7fb1862b5740 1 -- 172.21.15.151:0/1116723435 >> 172.21.15.151:0/1116723435 conn(0x5567e978da20 msgr2=0x5567e978c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:29.024 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.017+0000 7fb1862b5740 1 -- 172.21.15.151:0/1116723435 shutdown_connections 2024-09-16T15:52:29.024 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.017+0000 7fb1862b5740 1 -- 172.21.15.151:0/1116723435 wait complete. 2024-09-16T15:52:29.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.017+0000 7fb1862b5740 20 client.8616 trim_cache size 0 max 16384 2024-09-16T15:52:29.025 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-16T15:52:29.025 INFO:teuthology.orchestra.run.smithi151.stdout: "cluster_fsid": "1757be18-e41a-4446-bb4f-ee4ac81c39df", 2024-09-16T15:52:29.025 INFO:teuthology.orchestra.run.smithi151.stdout: "whoami": 0, 2024-09-16T15:52:29.025 INFO:teuthology.orchestra.run.smithi151.stdout: "id": 8460, 2024-09-16T15:52:29.025 INFO:teuthology.orchestra.run.smithi151.stdout: "want_state": "up:active", 2024-09-16T15:52:29.025 INFO:teuthology.orchestra.run.smithi151.stdout: "state": "up:active", 2024-09-16T15:52:29.025 INFO:teuthology.orchestra.run.smithi151.stdout: "fs_name": "cephfs", 2024-09-16T15:52:29.025 INFO:teuthology.orchestra.run.smithi151.stdout: "rank_uptime": 10.930519657, 2024-09-16T15:52:29.026 INFO:teuthology.orchestra.run.smithi151.stdout: "mdsmap_epoch": 76, 2024-09-16T15:52:29.026 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_epoch": 118, 2024-09-16T15:52:29.026 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_epoch_barrier": 118, 2024-09-16T15:52:29.026 INFO:teuthology.orchestra.run.smithi151.stdout: "uptime": 15.276553137000001 2024-09-16T15:52:29.026 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-16T15:52:29.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.017+0000 7fb183400640 1 -- 172.21.15.151:0/636342493 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb15c07a2c0 msgr2=0x7fb15c07c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:29.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.017+0000 7fb183400640 1 --2- 172.21.15.151:0/636342493 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb15c07a2c0 0x7fb15c07c780 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fb168034000 tx=0x7fb168004730 comp rx=0 tx=0).stop 2024-09-16T15:52:29.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.017+0000 7fb183400640 1 -- 172.21.15.151:0/636342493 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb17c073fb0 msgr2=0x7fb17c0b46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:29.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.017+0000 7fb183400640 1 --2- 172.21.15.151:0/636342493 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb17c073fb0 0x7fb17c0b46e0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fb16c00c3c0 tx=0x7fb16c00c880 comp rx=0 tx=0).stop 2024-09-16T15:52:29.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.017+0000 7fb183400640 1 -- 172.21.15.151:0/636342493 shutdown_connections 2024-09-16T15:52:29.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.017+0000 7fb183400640 1 --2- 172.21.15.151:0/636342493 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb15c07a2c0 0x7fb15c07c780 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:29.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.017+0000 7fb183400640 1 --2- 172.21.15.151:0/636342493 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb17c0ccf00 0x7fb17c0cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:29.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.017+0000 7fb183400640 1 --2- 172.21.15.151:0/636342493 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb17c15f5a0 0x7fb17c1f4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:29.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.017+0000 7fb183400640 1 --2- 172.21.15.151:0/636342493 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb17c073fb0 0x7fb17c0b46e0 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:29.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.018+0000 7fb183400640 1 -- 172.21.15.151:0/636342493 >> 172.21.15.151:0/636342493 conn(0x7fb17c0b1950 msgr2=0x7fb17c15dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:29.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.018+0000 7fb183400640 1 -- 172.21.15.151:0/636342493 shutdown_connections 2024-09-16T15:52:29.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.018+0000 7fb183400640 1 -- 172.21.15.151:0/636342493 wait complete. 2024-09-16T15:52:29.035 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-16T15:52:29.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.192+0000 7fc712a00640 1 -- 172.21.15.151:0/1435544194 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc70c15f5a0 msgr2=0x7fc70c0b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:29.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.192+0000 7fc712a00640 1 --2- 172.21.15.151:0/1435544194 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc70c15f5a0 0x7fc70c0b2440 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc009860 tx=0x7fc6fc02a4a0 comp rx=0 tx=0).stop 2024-09-16T15:52:29.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.192+0000 7fc712a00640 1 -- 172.21.15.151:0/1435544194 shutdown_connections 2024-09-16T15:52:29.194 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.192+0000 7fc712a00640 1 --2- 172.21.15.151:0/1435544194 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc70c15f5a0 0x7fc70c0b2440 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:29.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.192+0000 7fc712a00640 1 -- 172.21.15.151:0/1435544194 >> 172.21.15.151:0/1435544194 conn(0x7fc70c0b1950 msgr2=0x7fc70c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:29.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.192+0000 7fc712a00640 1 -- 172.21.15.151:0/1435544194 shutdown_connections 2024-09-16T15:52:29.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.192+0000 7fc712a00640 1 -- 172.21.15.151:0/1435544194 wait complete. 2024-09-16T15:52:29.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.194+0000 7fc712a00640 1 Processor -- start 2024-09-16T15:52:29.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.194+0000 7fc712a00640 1 -- start start 2024-09-16T15:52:29.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.195+0000 7fc712a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc70c073fb0 0x7fc70c154100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:29.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.195+0000 7fc712a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc70c0ccf00 0x7fc70c14c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:29.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.195+0000 7fc712a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc70c15f5a0 0x7fc70c14c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:29.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.195+0000 7fc712a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c06ef30 con 0x7fc70c15f5a0 2024-09-16T15:52:29.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.195+0000 7fc712a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c06fa80 con 0x7fc70c0ccf00 2024-09-16T15:52:29.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.195+0000 7fc712a00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fc70c074460 con 0x7fc70c073fb0 2024-09-16T15:52:29.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.195+0000 7fc710a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc70c15f5a0 0x7fc70c14c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:29.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.195+0000 7fc710a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc70c15f5a0 0x7fc70c14c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41492/0 (socket says 172.21.15.151:41492) 2024-09-16T15:52:29.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.195+0000 7fc710a00640 1 -- 172.21.15.151:0/3648461658 learned_addr learned my addr 172.21.15.151:0/3648461658 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:52:29.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.195+0000 7fc70be00640 1 --2- 172.21.15.151:0/3648461658 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc70c073fb0 0x7fc70c154100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:29.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.195+0000 7fc70b400640 1 --2- 172.21.15.151:0/3648461658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc70c0ccf00 0x7fc70c14c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:29.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.195+0000 7fc710a00640 1 -- 172.21.15.151:0/3648461658 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc70c073fb0 msgr2=0x7fc70c154100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:29.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.195+0000 7fc710a00640 1 --2- 172.21.15.151:0/3648461658 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc70c073fb0 0x7fc70c154100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:29.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.195+0000 7fc710a00640 1 -- 172.21.15.151:0/3648461658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc70c0ccf00 msgr2=0x7fc70c14c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:29.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.195+0000 7fc710a00640 1 --2- 172.21.15.151:0/3648461658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc70c0ccf00 0x7fc70c14c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:29.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.195+0000 7fc710a00640 1 -- 172.21.15.151:0/3648461658 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc70c1f8380 con 0x7fc70c15f5a0 2024-09-16T15:52:29.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.196+0000 7fc710a00640 1 --2- 172.21.15.151:0/3648461658 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc70c15f5a0 0x7fc70c14c7d0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fc70400cc00 tx=0x7fc70400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:29.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.196+0000 7fc703e00640 1 -- 172.21.15.151:0/3648461658 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc704016020 con 0x7fc70c15f5a0 2024-09-16T15:52:29.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.196+0000 7fc703e00640 1 -- 172.21.15.151:0/3648461658 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc704012420 con 0x7fc70c15f5a0 2024-09-16T15:52:29.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.196+0000 7fc703e00640 1 -- 172.21.15.151:0/3648461658 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc704012890 con 0x7fc70c15f5a0 2024-09-16T15:52:29.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.196+0000 7fc712a00640 1 -- 172.21.15.151:0/3648461658 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc70c1f8620 con 0x7fc70c15f5a0 2024-09-16T15:52:29.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.196+0000 7fc712a00640 1 -- 172.21.15.151:0/3648461658 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc70c1f8be0 con 0x7fc70c15f5a0 2024-09-16T15:52:29.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.197+0000 7fc712a00640 1 -- 172.21.15.151:0/3648461658 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc70c0aeb60 con 0x7fc70c15f5a0 2024-09-16T15:52:29.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.198+0000 7fc703e00640 1 -- 172.21.15.151:0/3648461658 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 41) ==== 103684+0+0 (secure 0 0 0) 0x7fc704019020 con 0x7fc70c15f5a0 2024-09-16T15:52:29.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.199+0000 7fc703e00640 1 --2- 172.21.15.151:0/3648461658 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc6ec07a2c0 0x7fc6ec07c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:29.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.199+0000 7fc703e00640 1 -- 172.21.15.151:0/3648461658 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5793+0+0 (secure 0 0 0) 0x7fc704099b60 con 0x7fc70c15f5a0 2024-09-16T15:52:29.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.200+0000 7fc70be00640 1 --2- 172.21.15.151:0/3648461658 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc6ec07a2c0 0x7fc6ec07c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:29.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.201+0000 7fc70be00640 1 --2- 172.21.15.151:0/3648461658 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc6ec07a2c0 0x7fc6ec07c780 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc034000 tx=0x7fc6fc004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:29.205 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.204+0000 7fc703e00640 1 -- 172.21.15.151:0/3648461658 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc704020030 con 0x7fc70c15f5a0 2024-09-16T15:52:29.425 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.423+0000 7fc712a00640 1 -- 172.21.15.151:0/3648461658 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc70c154e80 con 0x7fc70c15f5a0 2024-09-16T15:52:29.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.424+0000 7fc703e00640 1 -- 172.21.15.151:0/3648461658 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v76) ==== 53+0+73 (secure 0 0 0) 0x7fc7040635e0 con 0x7fc70c15f5a0 2024-09-16T15:52:29.426 INFO:teuthology.orchestra.run.smithi151.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-16T15:52:29.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.426+0000 7fc712a00640 1 -- 172.21.15.151:0/3648461658 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc6ec07a2c0 msgr2=0x7fc6ec07c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:29.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.426+0000 7fc712a00640 1 --2- 172.21.15.151:0/3648461658 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc6ec07a2c0 0x7fc6ec07c780 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc034000 tx=0x7fc6fc004730 comp rx=0 tx=0).stop 2024-09-16T15:52:29.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.426+0000 7fc712a00640 1 -- 172.21.15.151:0/3648461658 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc70c15f5a0 msgr2=0x7fc70c14c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:29.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.426+0000 7fc712a00640 1 --2- 172.21.15.151:0/3648461658 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc70c15f5a0 0x7fc70c14c7d0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fc70400cc00 tx=0x7fc70400a430 comp rx=0 tx=0).stop 2024-09-16T15:52:29.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.426+0000 7fc712a00640 1 -- 172.21.15.151:0/3648461658 shutdown_connections 2024-09-16T15:52:29.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.426+0000 7fc712a00640 1 --2- 172.21.15.151:0/3648461658 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc6ec07a2c0 0x7fc6ec07c780 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:29.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.426+0000 7fc712a00640 1 --2- 172.21.15.151:0/3648461658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc70c0ccf00 0x7fc70c14c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:29.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.426+0000 7fc712a00640 1 --2- 172.21.15.151:0/3648461658 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc70c15f5a0 0x7fc70c14c7d0 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:29.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.426+0000 7fc712a00640 1 --2- 172.21.15.151:0/3648461658 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc70c073fb0 0x7fc70c154100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:29.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.426+0000 7fc712a00640 1 -- 172.21.15.151:0/3648461658 >> 172.21.15.151:0/3648461658 conn(0x7fc70c0b1950 msgr2=0x7fc70c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:29.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.426+0000 7fc712a00640 1 -- 172.21.15.151:0/3648461658 shutdown_connections 2024-09-16T15:52:29.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29.426+0000 7fc712a00640 1 -- 172.21.15.151:0/3648461658 wait complete. 2024-09-16T15:52:29.444 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-16T15:52:29.444 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T15:52:29.444 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi151.front.sepia.ceph.com 2024-09-16T15:52:29.444 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-16T15:52:29.444 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-16T15:52:29.444 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-16T15:52:29.444 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-16T15:52:29.444 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T15:52:29.444 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T15:52:29.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:52:29.445 DEBUG:teuthology.orchestra.run.smithi151:> ip addr 2024-09-16T15:52:29.481 INFO:teuthology.orchestra.run.smithi151.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T15:52:29.481 INFO:teuthology.orchestra.run.smithi151.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T15:52:29.481 INFO:teuthology.orchestra.run.smithi151.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T15:52:29.481 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:52:29.481 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 ::1/128 scope host 2024-09-16T15:52:29.481 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:52:29.481 INFO:teuthology.orchestra.run.smithi151.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:52:29.481 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:da:6e:0c brd ff:ff:ff:ff:ff:ff 2024-09-16T15:52:29.481 INFO:teuthology.orchestra.run.smithi151.stdout: altname enp7s0f0 2024-09-16T15:52:29.481 INFO:teuthology.orchestra.run.smithi151.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:52:29.481 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:da:6e:0d brd ff:ff:ff:ff:ff:ff 2024-09-16T15:52:29.481 INFO:teuthology.orchestra.run.smithi151.stdout: altname enp7s0f1 2024-09-16T15:52:29.481 INFO:teuthology.orchestra.run.smithi151.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:52:29.482 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:88:6f:62 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:52:29.482 INFO:teuthology.orchestra.run.smithi151.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T15:52:29.482 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:88:6f:63 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:52:29.482 INFO:teuthology.orchestra.run.smithi151.stdout: inet 172.21.15.151/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-16T15:52:29.482 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft 41940sec preferred_lft 41940sec 2024-09-16T15:52:29.482 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 fe80::ec4:7aff:fe88:6f63/64 scope link noprefixroute 2024-09-16T15:52:29.482 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:52:29.482 INFO:teuthology.orchestra.run.smithi151.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-16T15:52:29.482 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 62:3f:3b:48:ad:f8 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:52:29.482 INFO:teuthology.orchestra.run.smithi151.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-16T15:52:29.482 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:52:29.482 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 fe80::40fa:7fff:fedf:2a00/64 scope link 2024-09-16T15:52:29.482 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:52:29.482 INFO:teuthology.orchestra.run.smithi151.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T15:52:29.482 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 62:3f:3b:48:ad:f8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T15:52:29.483 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 fe80::603f:3bff:fe48:adf8/64 scope link 2024-09-16T15:52:29.483 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:52:29.483 INFO:teuthology.orchestra.run.smithi151.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T15:52:29.483 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether b6:ac:fe:34:a7:66 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-16T15:52:29.483 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 fe80::b4ac:feff:fe34:a766/64 scope link 2024-09-16T15:52:29.483 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:52:29.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:52:29.483 DEBUG:teuthology.orchestra.run.smithi151:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T15:52:29.574 INFO:teuthology.orchestra.run.smithi151.stdout:1 2024-09-16T15:52:29.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:52:29.578 DEBUG:teuthology.orchestra.run.smithi151:> route 2024-09-16T15:52:29.614 INFO:teuthology.orchestra.run.smithi151.stdout:Kernel IP routing table 2024-09-16T15:52:29.614 INFO:teuthology.orchestra.run.smithi151.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T15:52:29.615 INFO:teuthology.orchestra.run.smithi151.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-16T15:52:29.615 INFO:teuthology.orchestra.run.smithi151.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-16T15:52:29.615 INFO:teuthology.orchestra.run.smithi151.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T15:52:29.615 DEBUG:teuthology.orchestra.run.smithi151:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T15:52:29.615 DEBUG:teuthology.orchestra.run.smithi151:> set -e 2024-09-16T15:52:29.615 DEBUG:teuthology.orchestra.run.smithi151:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-16T15:52:29.615 DEBUG:teuthology.orchestra.run.smithi151:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-16T15:52:29.615 DEBUG:teuthology.orchestra.run.smithi151:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-16T15:52:29.615 DEBUG:teuthology.orchestra.run.smithi151:> ') 2024-09-16T15:52:29.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:52:29.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:52:29.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:52:29.813 DEBUG:teuthology.orchestra.run.smithi151:> ip netns list 2024-09-16T15:52:29.848 INFO:teuthology.orchestra.run.smithi151.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-16T15:52:29.849 INFO:teuthology.orchestra.run.smithi151.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-16T15:52:29.849 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T15:52:29.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:52:29.849 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip link set brx.0 up 2024-09-16T15:52:29.945 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:52:29.945 DEBUG:teuthology.orchestra.run.smithi151:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-16T15:52:29.980 INFO:teuthology.orchestra.run.smithi151.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-16T15:52:29.981 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:52:29.981 DEBUG:teuthology.orchestra.run.smithi151:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-16T15:52:30.056 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-16T15:52:30.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:52:30.057 DEBUG:teuthology.orchestra.run.smithi151:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-16T15:52:30.186 INFO:teuthology.orchestra.run.smithi151.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-16T15:52:30.186 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-16T15:52:30.186 INFO:teuthology.orchestra.run.smithi151.stdout:invalid new device string format 2024-09-16T15:52:30.202 INFO:teuthology.orchestra.run.smithi151.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-16T15:52:30.202 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-16T15:52:30.202 INFO:teuthology.orchestra.run.smithi151.stdout:invalid new device string format 2024-09-16T15:52:30.202 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: resolved to: "172.21.15.151:3300,172.21.15.167:3300,172.21.15.167:3301" 2024-09-16T15:52:30.202 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-16T15:52:30.224 INFO:teuthology.orchestra.run.smithi151.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-16T15:52:30.225 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-16T15:52:30.225 INFO:teuthology.orchestra.run.smithi151.stdout:invalid new device string format 2024-09-16T15:52:30.225 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: resolved to: "172.21.15.151:3300,172.21.15.167:3300,172.21.15.167:3301" 2024-09-16T15:52:30.225 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-16T15:52:30.225 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: resolved to: "172.21.15.151:6789,172.21.15.167:6789,172.21.15.167:6790" 2024-09-16T15:52:30.225 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.151:6789,172.21.15.167:6789,172.21.15.167:6790:/ 2024-09-16T15:52:30.225 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=1757be18-e41a-4446-bb4f-ee4ac81c39df" will pass to kernel 2024-09-16T15:52:30.227 INFO:teuthology.orchestra.run.smithi151.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-16T15:52:30.227 INFO:teuthology.orchestra.run.smithi151.stdout: You just mounted a file system that supports labels which does not 2024-09-16T15:52:30.227 INFO:teuthology.orchestra.run.smithi151.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-16T15:52:30.228 INFO:teuthology.orchestra.run.smithi151.stdout: applications will generate AVC messages and not be allowed access to 2024-09-16T15:52:30.228 INFO:teuthology.orchestra.run.smithi151.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-16T15:52:30.230 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-16T15:52:30.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:52:30.230 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-16T15:52:30.287 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T15:52:30.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:30 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:52:30.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:52:30.382 INFO:teuthology.orchestra.run.smithi151.stdout:client8625 2024-09-16T15:52:30.383 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:52:30.383 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:52:30.418 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-16T15:52:30.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:30.619+0000 7f9c17000640 1 -- 172.21.15.151:0/238222396 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9c100733e0 msgr2=0x7f9c100cce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:30.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:30.619+0000 7f9c17000640 1 --2- 172.21.15.151:0/238222396 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9c100733e0 0x7f9c100cce50 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f9c10069890 tx=0x7f9bfc02a5f0 comp rx=0 tx=0).stop 2024-09-16T15:52:30.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:30.619+0000 7f9c17000640 1 -- 172.21.15.151:0/238222396 shutdown_connections 2024-09-16T15:52:30.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:30.619+0000 7f9c17000640 1 --2- 172.21.15.151:0/238222396 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9c1015c350 0x7f9c1015c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:30.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:30.619+0000 7f9c17000640 1 --2- 172.21.15.151:0/238222396 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9c100733e0 0x7f9c100cce50 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:30.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:30.619+0000 7f9c17000640 1 --2- 172.21.15.151:0/238222396 >> v2:172.21.15.167:3300/0 conn(0x7f9c1015f910 0x7f9c1015fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:30.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:30.619+0000 7f9c17000640 1 -- 172.21.15.151:0/238222396 >> 172.21.15.151:0/238222396 conn(0x7f9c100b18e0 msgr2=0x7f9c100b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:30.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:30.619+0000 7f9c17000640 1 -- 172.21.15.151:0/238222396 shutdown_connections 2024-09-16T15:52:30.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:30.619+0000 7f9c17000640 1 -- 172.21.15.151:0/238222396 wait complete. 2024-09-16T15:52:30.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:30.620+0000 7f9c17000640 1 Processor -- start 2024-09-16T15:52:30.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:30.621+0000 7f9c17000640 1 -- start start 2024-09-16T15:52:30.624 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:30.621+0000 7f9c17000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c1015c350 0x7f9c10156a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:30.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:30.621+0000 7f9c17000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9c1015f910 0x7f9c10145510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:30.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:30.621+0000 7f9c17000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9c10156100 0x7f9c100add80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:30.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:30.621+0000 7f9c17000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9c10074a50 con 0x7f9c10156100 2024-09-16T15:52:30.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:30.621+0000 7f9c17000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9c100cd5a0 con 0x7f9c1015c350 2024-09-16T15:52:30.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:30.621+0000 7f9c17000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f9c1015cd40 con 0x7f9c1015f910 2024-09-16T15:52:30.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:30.622+0000 7f9c15000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9c10156100 0x7f9c100add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:30.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:30.622+0000 7f9c15000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9c10156100 0x7f9c100add80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41506/0 (socket says 172.21.15.151:41506) 2024-09-16T15:52:30.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:30.622+0000 7f9c15000640 1 -- 172.21.15.151:0/2462027126 learned_addr learned my addr 172.21.15.151:0/2462027126 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:52:30.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:30.622+0000 7f9c0f400640 1 --2- 172.21.15.151:0/2462027126 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9c1015f910 0x7f9c10145510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:30.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:30.622+0000 7f9c0fe00640 1 --2- 172.21.15.151:0/2462027126 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c1015c350 0x7f9c10156a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:30.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:30.622+0000 7f9c15000640 1 -- 172.21.15.151:0/2462027126 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9c1015f910 msgr2=0x7f9c10145510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:30.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:30.622+0000 7f9c15000640 1 --2- 172.21.15.151:0/2462027126 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9c1015f910 0x7f9c10145510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:30.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:30.622+0000 7f9c15000640 1 -- 172.21.15.151:0/2462027126 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c1015c350 msgr2=0x7f9c10156a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:30.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:30.622+0000 7f9c15000640 1 --2- 172.21.15.151:0/2462027126 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c1015c350 0x7f9c10156a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:30.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:30.622+0000 7f9c15000640 1 -- 172.21.15.151:0/2462027126 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c10159450 con 0x7f9c10156100 2024-09-16T15:52:30.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:30.622+0000 7f9c0fe00640 1 --2- 172.21.15.151:0/2462027126 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c1015c350 0x7f9c10156a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:52:30.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:30.622+0000 7f9c0f400640 1 --2- 172.21.15.151:0/2462027126 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9c1015f910 0x7f9c10145510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:52:30.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:30.622+0000 7f9c15000640 1 --2- 172.21.15.151:0/2462027126 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9c10156100 0x7f9c100add80 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f9c0800eb40 tx=0x7f9c0800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:30.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:30.622+0000 7f9c07e00640 1 -- 172.21.15.151:0/2462027126 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c08004b20 con 0x7f9c10156100 2024-09-16T15:52:30.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:30.623+0000 7f9c07e00640 1 -- 172.21.15.151:0/2462027126 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9c08004ce0 con 0x7f9c10156100 2024-09-16T15:52:30.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:30.623+0000 7f9c07e00640 1 -- 172.21.15.151:0/2462027126 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c08005280 con 0x7f9c10156100 2024-09-16T15:52:30.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:30.623+0000 7f9c17000640 1 -- 172.21.15.151:0/2462027126 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c101f2fc0 con 0x7f9c10156100 2024-09-16T15:52:30.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:30.623+0000 7f9c17000640 1 -- 172.21.15.151:0/2462027126 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c101f3580 con 0x7f9c10156100 2024-09-16T15:52:30.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:30.624+0000 7f9c17000640 1 -- 172.21.15.151:0/2462027126 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c10069360 con 0x7f9c10156100 2024-09-16T15:52:30.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:30.625+0000 7f9c07e00640 1 -- 172.21.15.151:0/2462027126 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 41) ==== 103684+0+0 (secure 0 0 0) 0x7f9c08005420 con 0x7f9c10156100 2024-09-16T15:52:30.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:30.626+0000 7f9c07e00640 1 --2- 172.21.15.151:0/2462027126 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9bf007a2c0 0x7f9bf007c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:30.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:30.626+0000 7f9c07e00640 1 -- 172.21.15.151:0/2462027126 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5793+0+0 (secure 0 0 0) 0x7f9c0809a4c0 con 0x7f9c10156100 2024-09-16T15:52:30.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:30.627+0000 7f9c0fe00640 1 --2- 172.21.15.151:0/2462027126 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9bf007a2c0 0x7f9bf007c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:30.630 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:30.628+0000 7f9c0fe00640 1 --2- 172.21.15.151:0/2462027126 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9bf007a2c0 0x7f9bf007c780 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f9c00009a30 tx=0x7f9c00005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:30.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:30.630+0000 7f9c07e00640 1 -- 172.21.15.151:0/2462027126 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9c08064590 con 0x7f9c10156100 2024-09-16T15:52:30.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:30.850+0000 7f9c17000640 1 -- 172.21.15.151:0/2462027126 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f9c1015c800 con 0x7f9c10156100 2024-09-16T15:52:31.150 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.148+0000 7f9c07e00640 1 -- 172.21.15.151:0/2462027126 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v77) ==== 136+0+0 (secure 0 0 0) 0x7f9c08063e90 con 0x7f9c10156100 2024-09-16T15:52:31.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.151+0000 7f9c17000640 1 -- 172.21.15.151:0/2462027126 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9bf007a2c0 msgr2=0x7f9bf007c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:31.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.151+0000 7f9c17000640 1 --2- 172.21.15.151:0/2462027126 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9bf007a2c0 0x7f9bf007c780 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f9c00009a30 tx=0x7f9c00005f90 comp rx=0 tx=0).stop 2024-09-16T15:52:31.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.151+0000 7f9c17000640 1 -- 172.21.15.151:0/2462027126 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9c10156100 msgr2=0x7f9c100add80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:31.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.151+0000 7f9c17000640 1 --2- 172.21.15.151:0/2462027126 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9c10156100 0x7f9c100add80 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f9c0800eb40 tx=0x7f9c0800c2d0 comp rx=0 tx=0).stop 2024-09-16T15:52:31.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.151+0000 7f9c17000640 1 -- 172.21.15.151:0/2462027126 shutdown_connections 2024-09-16T15:52:31.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.151+0000 7f9c17000640 1 --2- 172.21.15.151:0/2462027126 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9bf007a2c0 0x7f9bf007c780 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:31.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.151+0000 7f9c17000640 1 --2- 172.21.15.151:0/2462027126 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9c10156100 0x7f9c100add80 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:31.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.151+0000 7f9c17000640 1 --2- 172.21.15.151:0/2462027126 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9c1015f910 0x7f9c10145510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:31.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.151+0000 7f9c17000640 1 --2- 172.21.15.151:0/2462027126 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c1015c350 0x7f9c10156a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:31.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.151+0000 7f9c17000640 1 -- 172.21.15.151:0/2462027126 >> 172.21.15.151:0/2462027126 conn(0x7f9c100b18e0 msgr2=0x7f9c1006f3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:31.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.151+0000 7f9c17000640 1 -- 172.21.15.151:0/2462027126 shutdown_connections 2024-09-16T15:52:31.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.151+0000 7f9c17000640 1 -- 172.21.15.151:0/2462027126 wait complete. 2024-09-16T15:52:31.171 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-16T15:52:31.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.328+0000 7fa42ca00640 1 -- 172.21.15.151:0/3549526182 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4280ccf00 msgr2=0x7fa4280b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:31.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.328+0000 7fa42ca00640 1 --2- 172.21.15.151:0/3549526182 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4280ccf00 0x7fa4280b2440 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fa414009830 tx=0x7fa41402a530 comp rx=0 tx=0).stop 2024-09-16T15:52:31.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.328+0000 7fa42ca00640 1 -- 172.21.15.151:0/3549526182 shutdown_connections 2024-09-16T15:52:31.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.328+0000 7fa42ca00640 1 --2- 172.21.15.151:0/3549526182 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4280ccf00 0x7fa4280b2440 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:31.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.328+0000 7fa42ca00640 1 -- 172.21.15.151:0/3549526182 >> 172.21.15.151:0/3549526182 conn(0x7fa4280b1950 msgr2=0x7fa4280b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:31.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.328+0000 7fa42ca00640 1 -- 172.21.15.151:0/3549526182 shutdown_connections 2024-09-16T15:52:31.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.328+0000 7fa42ca00640 1 -- 172.21.15.151:0/3549526182 wait complete. 2024-09-16T15:52:31.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.329+0000 7fa42ca00640 1 Processor -- start 2024-09-16T15:52:31.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.329+0000 7fa42ca00640 1 -- start start 2024-09-16T15:52:31.333 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.330+0000 7fa42ca00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa428073fb0 0x7fa42815bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:31.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.330+0000 7fa42ca00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa4280ccf00 0x7fa4280b0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:31.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.330+0000 7fa42ca00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa42815f5a0 0x7fa4281f5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:31.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.330+0000 7fa42ca00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa42806f9f0 con 0x7fa428073fb0 2024-09-16T15:52:31.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.330+0000 7fa42ca00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa42806f870 con 0x7fa42815f5a0 2024-09-16T15:52:31.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.330+0000 7fa42ca00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fa428074420 con 0x7fa4280ccf00 2024-09-16T15:52:31.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.330+0000 7fa425e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa428073fb0 0x7fa42815bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:31.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.331+0000 7fa425e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa428073fb0 0x7fa42815bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41524/0 (socket says 172.21.15.151:41524) 2024-09-16T15:52:31.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.331+0000 7fa425e00640 1 -- 172.21.15.151:0/2094302558 learned_addr learned my addr 172.21.15.151:0/2094302558 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:52:31.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.331+0000 7fa425400640 1 --2- 172.21.15.151:0/2094302558 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa4280ccf00 0x7fa4280b0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:31.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.331+0000 7fa426800640 1 --2- 172.21.15.151:0/2094302558 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa42815f5a0 0x7fa4281f5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:31.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.331+0000 7fa425e00640 1 -- 172.21.15.151:0/2094302558 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa4280ccf00 msgr2=0x7fa4280b0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:31.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.331+0000 7fa425e00640 1 --2- 172.21.15.151:0/2094302558 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa4280ccf00 0x7fa4280b0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:31.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.331+0000 7fa425e00640 1 -- 172.21.15.151:0/2094302558 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa42815f5a0 msgr2=0x7fa4281f5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:31.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.331+0000 7fa425e00640 1 --2- 172.21.15.151:0/2094302558 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa42815f5a0 0x7fa4281f5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:31.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.331+0000 7fa425e00640 1 -- 172.21.15.151:0/2094302558 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4281568f0 con 0x7fa428073fb0 2024-09-16T15:52:31.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.331+0000 7fa425e00640 1 --2- 172.21.15.151:0/2094302558 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa428073fb0 0x7fa42815bbe0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fa41800c760 tx=0x7fa41800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:31.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.331+0000 7fa41e000640 1 -- 172.21.15.151:0/2094302558 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa418003da0 con 0x7fa428073fb0 2024-09-16T15:52:31.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.331+0000 7fa41e000640 1 -- 172.21.15.151:0/2094302558 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa418005070 con 0x7fa428073fb0 2024-09-16T15:52:31.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.331+0000 7fa41e000640 1 -- 172.21.15.151:0/2094302558 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa418005480 con 0x7fa428073fb0 2024-09-16T15:52:31.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.331+0000 7fa42ca00640 1 -- 172.21.15.151:0/2094302558 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4281f65b0 con 0x7fa428073fb0 2024-09-16T15:52:31.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.332+0000 7fa42ca00640 1 -- 172.21.15.151:0/2094302558 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa42815b3e0 con 0x7fa428073fb0 2024-09-16T15:52:31.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.333+0000 7fa42ca00640 1 -- 172.21.15.151:0/2094302558 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4280b2440 con 0x7fa428073fb0 2024-09-16T15:52:31.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.334+0000 7fa41e000640 1 -- 172.21.15.151:0/2094302558 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 41) ==== 103684+0+0 (secure 0 0 0) 0x7fa418005620 con 0x7fa428073fb0 2024-09-16T15:52:31.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.335+0000 7fa41e000640 1 --2- 172.21.15.151:0/2094302558 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa40807a2c0 0x7fa40807c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:31.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.335+0000 7fa41e000640 1 -- 172.21.15.151:0/2094302558 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5793+0+0 (secure 0 0 0) 0x7fa418099600 con 0x7fa428073fb0 2024-09-16T15:52:31.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.335+0000 7fa425400640 1 --2- 172.21.15.151:0/2094302558 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa40807a2c0 0x7fa40807c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:31.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.336+0000 7fa425400640 1 --2- 172.21.15.151:0/2094302558 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa40807a2c0 0x7fa40807c780 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fa414034000 tx=0x7fa414004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:31.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.339+0000 7fa41e000640 1 -- 172.21.15.151:0/2094302558 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa4180636d0 con 0x7fa428073fb0 2024-09-16T15:52:31.548 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.545+0000 7fa42ca00640 1 -- 172.21.15.151:0/2094302558 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fa42809dd80 con 0x7fa40807a2c0 2024-09-16T15:52:31.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.548+0000 7fa41e000640 1 -- 172.21.15.151:0/2094302558 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fa42809dd80 con 0x7fa40807a2c0 2024-09-16T15:52:31.550 INFO:teuthology.orchestra.run.smithi151.stdout:[ 2024-09-16T15:52:31.550 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:52:31.550 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "cephfs" 2024-09-16T15:52:31.550 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:52:31.550 INFO:teuthology.orchestra.run.smithi151.stdout:] 2024-09-16T15:52:31.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.550+0000 7fa42ca00640 1 -- 172.21.15.151:0/2094302558 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa40807a2c0 msgr2=0x7fa40807c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:31.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.550+0000 7fa42ca00640 1 --2- 172.21.15.151:0/2094302558 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa40807a2c0 0x7fa40807c780 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fa414034000 tx=0x7fa414004730 comp rx=0 tx=0).stop 2024-09-16T15:52:31.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.550+0000 7fa42ca00640 1 -- 172.21.15.151:0/2094302558 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa428073fb0 msgr2=0x7fa42815bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:31.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.550+0000 7fa42ca00640 1 --2- 172.21.15.151:0/2094302558 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa428073fb0 0x7fa42815bbe0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fa41800c760 tx=0x7fa41800cc20 comp rx=0 tx=0).stop 2024-09-16T15:52:31.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.550+0000 7fa42ca00640 1 -- 172.21.15.151:0/2094302558 shutdown_connections 2024-09-16T15:52:31.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.550+0000 7fa42ca00640 1 --2- 172.21.15.151:0/2094302558 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa40807a2c0 0x7fa40807c780 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:31.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.550+0000 7fa42ca00640 1 --2- 172.21.15.151:0/2094302558 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa42815f5a0 0x7fa4281f5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:31.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.550+0000 7fa42ca00640 1 --2- 172.21.15.151:0/2094302558 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa4280ccf00 0x7fa4280b0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:31.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.550+0000 7fa42ca00640 1 --2- 172.21.15.151:0/2094302558 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa428073fb0 0x7fa42815bbe0 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:31.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.550+0000 7fa42ca00640 1 -- 172.21.15.151:0/2094302558 >> 172.21.15.151:0/2094302558 conn(0x7fa4280b1950 msgr2=0x7fa42815e130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:31.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.550+0000 7fa42ca00640 1 -- 172.21.15.151:0/2094302558 shutdown_connections 2024-09-16T15:52:31.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.550+0000 7fa42ca00640 1 -- 172.21.15.151:0/2094302558 wait complete. 2024-09-16T15:52:31.569 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-16T15:52:31.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.727+0000 7f593ca00640 1 -- 172.21.15.151:0/1676985260 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f59380add40 msgr2=0x7f59380ae1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:31.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.727+0000 7f593ca00640 1 --2- 172.21.15.151:0/1676985260 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f59380add40 0x7f59380ae1b0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f59380ce8c0 tx=0x7f593002a9a0 comp rx=0 tx=0).stop 2024-09-16T15:52:31.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.727+0000 7f593ca00640 1 -- 172.21.15.151:0/1676985260 shutdown_connections 2024-09-16T15:52:31.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.727+0000 7f593ca00640 1 --2- 172.21.15.151:0/1676985260 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f59380add40 0x7f59380ae1b0 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:31.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.727+0000 7f593ca00640 1 --2- 172.21.15.151:0/1676985260 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f59380afd30 0x7f59380b0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:31.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.727+0000 7f593ca00640 1 --2- 172.21.15.151:0/1676985260 >> v2:172.21.15.167:3300/0 conn(0x7f593815f130 0x7f59380cc490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:31.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.727+0000 7f593ca00640 1 -- 172.21.15.151:0/1676985260 >> 172.21.15.151:0/1676985260 conn(0x7f59380d0d40 msgr2=0x7f59380d1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:31.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.727+0000 7f593ca00640 1 -- 172.21.15.151:0/1676985260 shutdown_connections 2024-09-16T15:52:31.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.727+0000 7f593ca00640 1 -- 172.21.15.151:0/1676985260 wait complete. 2024-09-16T15:52:31.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.728+0000 7f593ca00640 1 Processor -- start 2024-09-16T15:52:31.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.729+0000 7f593ca00640 1 -- start start 2024-09-16T15:52:31.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.729+0000 7f593ca00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f59380add40 0x7f5938147ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:31.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.729+0000 7f593ca00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59380afd30 0x7f59381f28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:31.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.730+0000 7f593ca00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f593815f130 0x7f59381f2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:31.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.730+0000 7f593ca00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f593806f7f0 con 0x7f59380add40 2024-09-16T15:52:31.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.730+0000 7f593ca00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f593806f670 con 0x7f59380afd30 2024-09-16T15:52:31.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.730+0000 7f593ca00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f593806f970 con 0x7f593815f130 2024-09-16T15:52:31.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.730+0000 7f5935e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f59380add40 0x7f5938147ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:31.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.730+0000 7f5935e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f59380add40 0x7f5938147ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41546/0 (socket says 172.21.15.151:41546) 2024-09-16T15:52:31.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.730+0000 7f5935e00640 1 -- 172.21.15.151:0/2549945545 learned_addr learned my addr 172.21.15.151:0/2549945545 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:52:31.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.730+0000 7f592d400640 1 --2- 172.21.15.151:0/2549945545 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59380afd30 0x7f59381f28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:31.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.730+0000 7f5936800640 1 --2- 172.21.15.151:0/2549945545 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f593815f130 0x7f59381f2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:31.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.730+0000 7f5935e00640 1 -- 172.21.15.151:0/2549945545 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f593815f130 msgr2=0x7f59381f2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:31.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.730+0000 7f5935e00640 1 --2- 172.21.15.151:0/2549945545 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f593815f130 0x7f59381f2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:31.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.730+0000 7f5935e00640 1 -- 172.21.15.151:0/2549945545 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59380afd30 msgr2=0x7f59381f28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:31.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.730+0000 7f5935e00640 1 --2- 172.21.15.151:0/2549945545 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59380afd30 0x7f59381f28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:31.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.730+0000 7f5935e00640 1 -- 172.21.15.151:0/2549945545 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59381f6de0 con 0x7f59380add40 2024-09-16T15:52:31.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.730+0000 7f592d400640 1 --2- 172.21.15.151:0/2549945545 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59380afd30 0x7f59381f28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:52:31.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.730+0000 7f5936800640 1 --2- 172.21.15.151:0/2549945545 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f593815f130 0x7f59381f2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:52:31.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.730+0000 7f5935e00640 1 --2- 172.21.15.151:0/2549945545 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f59380add40 0x7f5938147ee0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f592800e790 tx=0x7f592800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:31.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.731+0000 7f592ea00640 1 -- 172.21.15.151:0/2549945545 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592800eea0 con 0x7f59380add40 2024-09-16T15:52:31.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.731+0000 7f592ea00640 1 -- 172.21.15.151:0/2549945545 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f592800cdb0 con 0x7f59380add40 2024-09-16T15:52:31.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.731+0000 7f593ca00640 1 -- 172.21.15.151:0/2549945545 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59381f33f0 con 0x7f59380add40 2024-09-16T15:52:31.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.731+0000 7f592ea00640 1 -- 172.21.15.151:0/2549945545 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5928010690 con 0x7f59380add40 2024-09-16T15:52:31.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.731+0000 7f593ca00640 1 -- 172.21.15.151:0/2549945545 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59380b4f50 con 0x7f59380add40 2024-09-16T15:52:31.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.732+0000 7f593ca00640 1 -- 172.21.15.151:0/2549945545 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f593806fec0 con 0x7f59380add40 2024-09-16T15:52:31.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.733+0000 7f592ea00640 1 -- 172.21.15.151:0/2549945545 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 41) ==== 103684+0+0 (secure 0 0 0) 0x7f5928010830 con 0x7f59380add40 2024-09-16T15:52:31.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.734+0000 7f592ea00640 1 --2- 172.21.15.151:0/2549945545 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f591407a2c0 0x7f591407c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:31.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.734+0000 7f592ea00640 1 -- 172.21.15.151:0/2549945545 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5793+0+0 (secure 0 0 0) 0x7f592809a190 con 0x7f59380add40 2024-09-16T15:52:31.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.734+0000 7f592d400640 1 --2- 172.21.15.151:0/2549945545 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f591407a2c0 0x7f591407c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:31.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.735+0000 7f592d400640 1 --2- 172.21.15.151:0/2549945545 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f591407a2c0 0x7f591407c780 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f5938071290 tx=0x7f5920005e50 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:31.740 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.738+0000 7f592ea00640 1 -- 172.21.15.151:0/2549945545 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f5928064260 con 0x7f59380add40 2024-09-16T15:52:31.925 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.923+0000 7f593ca00640 1 -- 172.21.15.151:0/2549945545 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f59380b4d40 con 0x7f59380add40 2024-09-16T15:52:31.926 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.924+0000 7f592ea00640 1 -- 172.21.15.151:0/2549945545 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v48)=0 v48) ==== 122+0+0 (secure 0 0 0) 0x7f5928063b60 con 0x7f59380add40 2024-09-16T15:52:31.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.926+0000 7f593ca00640 1 -- 172.21.15.151:0/2549945545 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f591407a2c0 msgr2=0x7f591407c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:31.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.926+0000 7f593ca00640 1 --2- 172.21.15.151:0/2549945545 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f591407a2c0 0x7f591407c780 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f5938071290 tx=0x7f5920005e50 comp rx=0 tx=0).stop 2024-09-16T15:52:31.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.926+0000 7f593ca00640 1 -- 172.21.15.151:0/2549945545 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f59380add40 msgr2=0x7f5938147ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:31.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.926+0000 7f593ca00640 1 --2- 172.21.15.151:0/2549945545 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f59380add40 0x7f5938147ee0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f592800e790 tx=0x7f592800ec50 comp rx=0 tx=0).stop 2024-09-16T15:52:31.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.926+0000 7f593ca00640 1 -- 172.21.15.151:0/2549945545 shutdown_connections 2024-09-16T15:52:31.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.926+0000 7f593ca00640 1 --2- 172.21.15.151:0/2549945545 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f591407a2c0 0x7f591407c780 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:31.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.926+0000 7f593ca00640 1 --2- 172.21.15.151:0/2549945545 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59380afd30 0x7f59381f28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:31.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.926+0000 7f593ca00640 1 --2- 172.21.15.151:0/2549945545 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f593815f130 0x7f59381f2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:31.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.926+0000 7f593ca00640 1 --2- 172.21.15.151:0/2549945545 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f59380add40 0x7f5938147ee0 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:31.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.926+0000 7f593ca00640 1 -- 172.21.15.151:0/2549945545 >> 172.21.15.151:0/2549945545 conn(0x7f59380d0d40 msgr2=0x7f5938073ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:31.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.926+0000 7f593ca00640 1 -- 172.21.15.151:0/2549945545 shutdown_connections 2024-09-16T15:52:31.929 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:31.926+0000 7f593ca00640 1 -- 172.21.15.151:0/2549945545 wait complete. 2024-09-16T15:52:31.946 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mgr mgr/volumes/max_concurrent_clones 2024-09-16T15:52:32.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.103+0000 7f9eb3200640 1 -- 172.21.15.151:0/3903057765 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9eac06e880 msgr2=0x7f9eac15ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:32.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.103+0000 7f9eb3200640 1 --2- 172.21.15.151:0/3903057765 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9eac06e880 0x7f9eac15ec50 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c005fd0 tx=0x7f9e9c02a530 comp rx=0 tx=0).stop 2024-09-16T15:52:32.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.103+0000 7f9eb3200640 1 -- 172.21.15.151:0/3903057765 shutdown_connections 2024-09-16T15:52:32.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.103+0000 7f9eb3200640 1 --2- 172.21.15.151:0/3903057765 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9eac06e880 0x7f9eac15ec50 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:32.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.103+0000 7f9eb3200640 1 --2- 172.21.15.151:0/3903057765 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9eac15e870 0x7f9eac0ae5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:32.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.103+0000 7f9eb3200640 1 -- 172.21.15.151:0/3903057765 >> 172.21.15.151:0/3903057765 conn(0x7f9eac0cf9f0 msgr2=0x7f9eac0cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:32.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.103+0000 7f9eb3200640 1 -- 172.21.15.151:0/3903057765 shutdown_connections 2024-09-16T15:52:32.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.103+0000 7f9eb3200640 1 -- 172.21.15.151:0/3903057765 wait complete. 2024-09-16T15:52:32.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.105+0000 7f9eb3200640 1 Processor -- start 2024-09-16T15:52:32.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.105+0000 7f9eb3200640 1 -- start start 2024-09-16T15:52:32.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.106+0000 7f9eb3200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9eac06e880 0x7f9eac0c4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:32.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.106+0000 7f9eb3200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9eac15e870 0x7f9eac151f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:32.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.106+0000 7f9eb3200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9eac15f190 0x7f9eac0ca930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:32.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.106+0000 7f9eb3200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac074230 con 0x7f9eac15f190 2024-09-16T15:52:32.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.106+0000 7f9eb3200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac0740b0 con 0x7f9eac15e870 2024-09-16T15:52:32.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.106+0000 7f9eb3200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f9eac0743b0 con 0x7f9eac06e880 2024-09-16T15:52:32.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.106+0000 7f9eb2800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9eac15f190 0x7f9eac0ca930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:32.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.106+0000 7f9eb2800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9eac15f190 0x7f9eac0ca930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41556/0 (socket says 172.21.15.151:41556) 2024-09-16T15:52:32.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.106+0000 7f9eb2800640 1 -- 172.21.15.151:0/2979457 learned_addr learned my addr 172.21.15.151:0/2979457 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:52:32.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.107+0000 7f9eb1e00640 1 --2- 172.21.15.151:0/2979457 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9eac06e880 0x7f9eac0c4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:32.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.107+0000 7f9eb1400640 1 --2- 172.21.15.151:0/2979457 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9eac15e870 0x7f9eac151f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:32.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.107+0000 7f9eb2800640 1 -- 172.21.15.151:0/2979457 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9eac06e880 msgr2=0x7f9eac0c4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:32.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.107+0000 7f9eb2800640 1 --2- 172.21.15.151:0/2979457 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9eac06e880 0x7f9eac0c4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:32.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.107+0000 7f9eb2800640 1 -- 172.21.15.151:0/2979457 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9eac15e870 msgr2=0x7f9eac151f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:32.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.107+0000 7f9eb2800640 1 --2- 172.21.15.151:0/2979457 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9eac15e870 0x7f9eac151f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:32.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.107+0000 7f9eb2800640 1 -- 172.21.15.151:0/2979457 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eac1f6180 con 0x7f9eac15f190 2024-09-16T15:52:32.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.107+0000 7f9eb1400640 1 --2- 172.21.15.151:0/2979457 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9eac15e870 0x7f9eac151f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:52:32.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.107+0000 7f9eb2800640 1 --2- 172.21.15.151:0/2979457 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9eac15f190 0x7f9eac0ca930 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f9ea800b970 tx=0x7f9ea800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:32.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.107+0000 7f9ea6000640 1 -- 172.21.15.151:0/2979457 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea8019070 con 0x7f9eac15f190 2024-09-16T15:52:32.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.107+0000 7f9ea6000640 1 -- 172.21.15.151:0/2979457 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ea800cc30 con 0x7f9eac15f190 2024-09-16T15:52:32.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.107+0000 7f9ea6000640 1 -- 172.21.15.151:0/2979457 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea80146a0 con 0x7f9eac15f190 2024-09-16T15:52:32.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.107+0000 7f9eb3200640 1 -- 172.21.15.151:0/2979457 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9eac0c3e70 con 0x7f9eac15f190 2024-09-16T15:52:32.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.108+0000 7f9eb3200640 1 -- 172.21.15.151:0/2979457 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9eac0c4430 con 0x7f9eac15f190 2024-09-16T15:52:32.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.109+0000 7f9eb3200640 1 -- 172.21.15.151:0/2979457 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9eac15ec50 con 0x7f9eac15f190 2024-09-16T15:52:32.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.110+0000 7f9ea6000640 1 -- 172.21.15.151:0/2979457 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 41) ==== 103684+0+0 (secure 0 0 0) 0x7f9ea801c070 con 0x7f9eac15f190 2024-09-16T15:52:32.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.111+0000 7f9ea6000640 1 --2- 172.21.15.151:0/2979457 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9e9007a2c0 0x7f9e9007c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:32.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.111+0000 7f9ea6000640 1 -- 172.21.15.151:0/2979457 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5793+0+0 (secure 0 0 0) 0x7f9ea8099460 con 0x7f9eac15f190 2024-09-16T15:52:32.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.111+0000 7f9eb1e00640 1 --2- 172.21.15.151:0/2979457 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9e9007a2c0 0x7f9e9007c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:32.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.112+0000 7f9eb1e00640 1 --2- 172.21.15.151:0/2979457 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9e9007a2c0 0x7f9e9007c780 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0005fd0 tx=0x7f9ea0005850 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:32.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.115+0000 7f9ea6000640 1 -- 172.21.15.151:0/2979457 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9ea8063530 con 0x7f9eac15f190 2024-09-16T15:52:32.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.300+0000 7f9eb3200640 1 -- 172.21.15.151:0/2979457 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"} v 0) -- 0x7f9eac0c3300 con 0x7f9eac15f190 2024-09-16T15:52:32.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.301+0000 7f9ea6000640 1 -- 172.21.15.151:0/2979457 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"}]=0 v48) ==== 116+0+2 (secure 0 0 0) 0x7f9ea8062e30 con 0x7f9eac15f190 2024-09-16T15:52:32.303 INFO:teuthology.orchestra.run.smithi151.stdout:4 2024-09-16T15:52:32.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.303+0000 7f9eb3200640 1 -- 172.21.15.151:0/2979457 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9e9007a2c0 msgr2=0x7f9e9007c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:32.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.303+0000 7f9eb3200640 1 --2- 172.21.15.151:0/2979457 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9e9007a2c0 0x7f9e9007c780 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0005fd0 tx=0x7f9ea0005850 comp rx=0 tx=0).stop 2024-09-16T15:52:32.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.303+0000 7f9eb3200640 1 -- 172.21.15.151:0/2979457 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9eac15f190 msgr2=0x7f9eac0ca930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:32.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.303+0000 7f9eb3200640 1 --2- 172.21.15.151:0/2979457 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9eac15f190 0x7f9eac0ca930 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f9ea800b970 tx=0x7f9ea800be30 comp rx=0 tx=0).stop 2024-09-16T15:52:32.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.303+0000 7f9eb3200640 1 -- 172.21.15.151:0/2979457 shutdown_connections 2024-09-16T15:52:32.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.303+0000 7f9eb3200640 1 --2- 172.21.15.151:0/2979457 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9e9007a2c0 0x7f9e9007c780 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:32.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.303+0000 7f9eb3200640 1 --2- 172.21.15.151:0/2979457 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9eac15e870 0x7f9eac151f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:32.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.303+0000 7f9eb3200640 1 --2- 172.21.15.151:0/2979457 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9eac15f190 0x7f9eac0ca930 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:32.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.303+0000 7f9eb3200640 1 --2- 172.21.15.151:0/2979457 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9eac06e880 0x7f9eac0c4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:32.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.303+0000 7f9eb3200640 1 -- 172.21.15.151:0/2979457 >> 172.21.15.151:0/2979457 conn(0x7f9eac0cf9f0 msgr2=0x7f9eac159380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:32.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.304+0000 7f9eb3200640 1 -- 172.21.15.151:0/2979457 shutdown_connections 2024-09-16T15:52:32.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.304+0000 7f9eb3200640 1 -- 172.21.15.151:0/2979457 wait complete. 2024-09-16T15:52:32.321 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-16T15:52:32.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.479+0000 7fddbbc00640 1 -- 172.21.15.151:0/2016155200 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fddb40cccd0 msgr2=0x7fddb40cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:32.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.479+0000 7fddbbc00640 1 --2- 172.21.15.151:0/2016155200 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fddb40cccd0 0x7fddb40cdbb0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fddac005fd0 tx=0x7fddac02a540 comp rx=0 tx=0).stop 2024-09-16T15:52:32.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.479+0000 7fddbbc00640 1 -- 172.21.15.151:0/2016155200 shutdown_connections 2024-09-16T15:52:32.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.479+0000 7fddbbc00640 1 --2- 172.21.15.151:0/2016155200 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fddb40cccd0 0x7fddb40cdbb0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:32.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.479+0000 7fddbbc00640 1 --2- 172.21.15.151:0/2016155200 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fddb4145750 0x7fddb4073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:32.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.479+0000 7fddbbc00640 1 -- 172.21.15.151:0/2016155200 >> 172.21.15.151:0/2016155200 conn(0x7fddb40b1840 msgr2=0x7fddb40b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:32.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.479+0000 7fddbbc00640 1 -- 172.21.15.151:0/2016155200 shutdown_connections 2024-09-16T15:52:32.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.479+0000 7fddbbc00640 1 -- 172.21.15.151:0/2016155200 wait complete. 2024-09-16T15:52:32.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.480+0000 7fddbbc00640 1 Processor -- start 2024-09-16T15:52:32.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.481+0000 7fddbbc00640 1 -- start start 2024-09-16T15:52:32.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.481+0000 7fddbbc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fddb406efa0 0x7fddb41f3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:32.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.481+0000 7fddbbc00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fddb40cccd0 0x7fddb41e5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:32.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.481+0000 7fddbbc00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fddb4145750 0x7fddb413edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:32.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.481+0000 7fddbbc00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fddb40ce480 con 0x7fddb406efa0 2024-09-16T15:52:32.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.481+0000 7fddbbc00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fddb40ce300 con 0x7fddb40cccd0 2024-09-16T15:52:32.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.481+0000 7fddbbc00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fddb40ce600 con 0x7fddb4145750 2024-09-16T15:52:32.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.482+0000 7fddb9200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fddb406efa0 0x7fddb41f3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:32.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.482+0000 7fddb9200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fddb406efa0 0x7fddb41f3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41580/0 (socket says 172.21.15.151:41580) 2024-09-16T15:52:32.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.482+0000 7fddb9200640 1 -- 172.21.15.151:0/2104493205 learned_addr learned my addr 172.21.15.151:0/2104493205 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:52:32.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.482+0000 7fddb3e00640 1 --2- 172.21.15.151:0/2104493205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fddb40cccd0 0x7fddb41e5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:32.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.482+0000 7fddb9c00640 1 --2- 172.21.15.151:0/2104493205 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fddb4145750 0x7fddb413edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:32.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.482+0000 7fddb9200640 1 -- 172.21.15.151:0/2104493205 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fddb4145750 msgr2=0x7fddb413edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:32.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.482+0000 7fddb9200640 1 --2- 172.21.15.151:0/2104493205 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fddb4145750 0x7fddb413edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:32.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.482+0000 7fddb9200640 1 -- 172.21.15.151:0/2104493205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fddb40cccd0 msgr2=0x7fddb41e5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:32.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.482+0000 7fddb9200640 1 --2- 172.21.15.151:0/2104493205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fddb40cccd0 0x7fddb41e5300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:32.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.482+0000 7fddb9200640 1 -- 172.21.15.151:0/2104493205 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddb41e5840 con 0x7fddb406efa0 2024-09-16T15:52:32.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.482+0000 7fddb9200640 1 --2- 172.21.15.151:0/2104493205 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fddb406efa0 0x7fddb41f3a60 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fdda800e9f0 tx=0x7fdda800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:32.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.483+0000 7fddb0c00640 1 -- 172.21.15.151:0/2104493205 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda800cd80 con 0x7fddb406efa0 2024-09-16T15:52:32.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.483+0000 7fddb0c00640 1 -- 172.21.15.151:0/2104493205 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdda8004c10 con 0x7fddb406efa0 2024-09-16T15:52:32.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.483+0000 7fddbbc00640 1 -- 172.21.15.151:0/2104493205 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddb413f300 con 0x7fddb406efa0 2024-09-16T15:52:32.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.483+0000 7fddb0c00640 1 -- 172.21.15.151:0/2104493205 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda8005020 con 0x7fddb406efa0 2024-09-16T15:52:32.487 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.483+0000 7fddbbc00640 1 -- 172.21.15.151:0/2104493205 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddb41e4540 con 0x7fddb406efa0 2024-09-16T15:52:32.487 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.484+0000 7fddbbc00640 1 -- 172.21.15.151:0/2104493205 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddb40cf520 con 0x7fddb406efa0 2024-09-16T15:52:32.487 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.485+0000 7fddb0c00640 1 -- 172.21.15.151:0/2104493205 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 41) ==== 103684+0+0 (secure 0 0 0) 0x7fdda80051c0 con 0x7fddb406efa0 2024-09-16T15:52:32.488 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.486+0000 7fddb0c00640 1 --2- 172.21.15.151:0/2104493205 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdd9407a2f0 0x7fdd9407c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:32.488 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.486+0000 7fddb0c00640 1 -- 172.21.15.151:0/2104493205 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5793+0+0 (secure 0 0 0) 0x7fdda809a0f0 con 0x7fddb406efa0 2024-09-16T15:52:32.488 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.487+0000 7fddb3e00640 1 --2- 172.21.15.151:0/2104493205 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdd9407a2f0 0x7fdd9407c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:32.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.487+0000 7fddb3e00640 1 --2- 172.21.15.151:0/2104493205 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdd9407a2f0 0x7fdd9407c7b0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fddb406aca0 tx=0x7fdda0009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:32.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.490+0000 7fddb0c00640 1 -- 172.21.15.151:0/2104493205 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fdda8016030 con 0x7fddb406efa0 2024-09-16T15:52:32.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.677+0000 7fddbbc00640 1 -- 172.21.15.151:0/2104493205 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7fddb40cf240 con 0x7fddb406efa0 2024-09-16T15:52:32.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.681+0000 7fddb0c00640 1 -- 172.21.15.151:0/2104493205 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v49)=0 v49) ==== 131+0+0 (secure 0 0 0) 0x7fdda8063bf0 con 0x7fddb406efa0 2024-09-16T15:52:32.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.683+0000 7fddbbc00640 1 -- 172.21.15.151:0/2104493205 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdd9407a2f0 msgr2=0x7fdd9407c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:32.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.683+0000 7fddbbc00640 1 --2- 172.21.15.151:0/2104493205 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdd9407a2f0 0x7fdd9407c7b0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fddb406aca0 tx=0x7fdda0009040 comp rx=0 tx=0).stop 2024-09-16T15:52:32.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.683+0000 7fddbbc00640 1 -- 172.21.15.151:0/2104493205 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fddb406efa0 msgr2=0x7fddb41f3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:32.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.683+0000 7fddbbc00640 1 --2- 172.21.15.151:0/2104493205 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fddb406efa0 0x7fddb41f3a60 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fdda800e9f0 tx=0x7fdda800eeb0 comp rx=0 tx=0).stop 2024-09-16T15:52:32.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.683+0000 7fddbbc00640 1 -- 172.21.15.151:0/2104493205 shutdown_connections 2024-09-16T15:52:32.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.683+0000 7fddbbc00640 1 --2- 172.21.15.151:0/2104493205 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdd9407a2f0 0x7fdd9407c7b0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:32.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.683+0000 7fddbbc00640 1 --2- 172.21.15.151:0/2104493205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fddb40cccd0 0x7fddb41e5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:32.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.683+0000 7fddbbc00640 1 --2- 172.21.15.151:0/2104493205 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fddb4145750 0x7fddb413edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:32.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.683+0000 7fddbbc00640 1 --2- 172.21.15.151:0/2104493205 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fddb406efa0 0x7fddb41f3a60 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:32.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.683+0000 7fddbbc00640 1 -- 172.21.15.151:0/2104493205 >> 172.21.15.151:0/2104493205 conn(0x7fddb40b1840 msgr2=0x7fddb4144050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:32.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.684+0000 7fddbbc00640 1 -- 172.21.15.151:0/2104493205 shutdown_connections 2024-09-16T15:52:32.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.684+0000 7fddbbc00640 1 -- 172.21.15.151:0/2104493205 wait complete. 2024-09-16T15:52:32.701 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-09-16T15:52:32.706 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:52:32.704+0000 7f48f2600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:52:32.706 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:52:32.704+0000 7f48f2600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:52:32.706 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:52:32.704+0000 7f48f2600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:52:32.706 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:52:32.704+0000 7f48f2600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:52:32.706 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:52:32.704+0000 7f48f2600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:52:32.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.863+0000 7f770d400640 1 -- 172.21.15.151:0/1589251720 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f770806e590 msgr2=0x7f770806fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:32.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.863+0000 7f770d400640 1 --2- 172.21.15.151:0/1589251720 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f770806e590 0x7f770806fe40 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f76f4009830 tx=0x7f76f402a510 comp rx=0 tx=0).stop 2024-09-16T15:52:32.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.863+0000 7f770d400640 1 -- 172.21.15.151:0/1589251720 shutdown_connections 2024-09-16T15:52:32.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.863+0000 7f770d400640 1 --2- 172.21.15.151:0/1589251720 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f770806e590 0x7f770806fe40 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:32.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.863+0000 7f770d400640 1 -- 172.21.15.151:0/1589251720 >> 172.21.15.151:0/1589251720 conn(0x7f77080cfa70 msgr2=0x7f77080cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:32.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.863+0000 7f770d400640 1 -- 172.21.15.151:0/1589251720 shutdown_connections 2024-09-16T15:52:32.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.863+0000 7f770d400640 1 -- 172.21.15.151:0/1589251720 wait complete. 2024-09-16T15:52:32.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.865+0000 7f770d400640 1 Processor -- start 2024-09-16T15:52:32.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.865+0000 7f770d400640 1 -- start start 2024-09-16T15:52:32.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.866+0000 7f770d400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f770806e590 0x7f7708152230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:32.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.866+0000 7f770d400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7708070380 0x7f77080c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:32.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.866+0000 7f770d400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f770815bda0 0x7f77080c92f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:32.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.866+0000 7f770d400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f77080cbc60 con 0x7f7708070380 2024-09-16T15:52:32.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.866+0000 7f770d400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f77080cbae0 con 0x7f770815bda0 2024-09-16T15:52:32.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.866+0000 7f770d400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f77080cbde0 con 0x7f770806e590 2024-09-16T15:52:32.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.866+0000 7f7707400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7708070380 0x7f77080c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:32.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.866+0000 7f7707400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7708070380 0x7f77080c8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:45254/0 (socket says 172.21.15.151:45254) 2024-09-16T15:52:32.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.866+0000 7f7707400640 1 -- 172.21.15.151:0/191697094 learned_addr learned my addr 172.21.15.151:0/191697094 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:52:32.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.866+0000 7f7707e00640 1 --2- 172.21.15.151:0/191697094 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f770806e590 0x7f7708152230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:32.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.866+0000 7f770ca00640 1 --2- 172.21.15.151:0/191697094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f770815bda0 0x7f77080c92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:32.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.866+0000 7f7707400640 1 -- 172.21.15.151:0/191697094 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f770806e590 msgr2=0x7f7708152230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:32.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.866+0000 7f7707400640 1 --2- 172.21.15.151:0/191697094 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f770806e590 0x7f7708152230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:32.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.866+0000 7f7707400640 1 -- 172.21.15.151:0/191697094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f770815bda0 msgr2=0x7f77080c92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:32.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.866+0000 7f7707400640 1 --2- 172.21.15.151:0/191697094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f770815bda0 0x7f77080c92f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:32.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.867+0000 7f7707400640 1 -- 172.21.15.151:0/191697094 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77080c35a0 con 0x7f7708070380 2024-09-16T15:52:32.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.867+0000 7f7707400640 1 --2- 172.21.15.151:0/191697094 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7708070380 0x7f77080c8db0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f76f402ab00 tx=0x7f76f4032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:32.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.867+0000 7f76ffe00640 1 -- 172.21.15.151:0/191697094 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f4040070 con 0x7f7708070380 2024-09-16T15:52:32.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.867+0000 7f76ffe00640 1 -- 172.21.15.151:0/191697094 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f76f4005310 con 0x7f7708070380 2024-09-16T15:52:32.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.867+0000 7f76ffe00640 1 -- 172.21.15.151:0/191697094 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f4005c90 con 0x7f7708070380 2024-09-16T15:52:32.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.867+0000 7f770d400640 1 -- 172.21.15.151:0/191697094 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77081f2330 con 0x7f7708070380 2024-09-16T15:52:32.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.867+0000 7f770d400640 1 -- 172.21.15.151:0/191697094 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77081f28f0 con 0x7f7708070380 2024-09-16T15:52:32.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.868+0000 7f770d400640 1 -- 172.21.15.151:0/191697094 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f770806fe40 con 0x7f7708070380 2024-09-16T15:52:32.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.870+0000 7f76ffe00640 1 -- 172.21.15.151:0/191697094 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 41) ==== 103684+0+0 (secure 0 0 0) 0x7f76f4039040 con 0x7f7708070380 2024-09-16T15:52:32.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.871+0000 7f76ffe00640 1 --2- 172.21.15.151:0/191697094 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f76e807a2c0 0x7f76e807c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:32.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.871+0000 7f76ffe00640 1 -- 172.21.15.151:0/191697094 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5793+0+0 (secure 0 0 0) 0x7f76f40baa20 con 0x7f7708070380 2024-09-16T15:52:32.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.871+0000 7f7707e00640 1 --2- 172.21.15.151:0/191697094 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f76e807a2c0 0x7f76e807c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:32.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.872+0000 7f7707e00640 1 --2- 172.21.15.151:0/191697094 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f76e807a2c0 0x7f76e807c780 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f76f8007fd0 tx=0x7f76f8001390 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:32.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:32.875+0000 7f76ffe00640 1 -- 172.21.15.151:0/191697094 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f76f40844a0 con 0x7f7708070380 2024-09-16T15:52:33.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.086+0000 7f770d400640 1 -- 172.21.15.151:0/191697094 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7f7708069890 con 0x7f76e807a2c0 2024-09-16T15:52:33.099 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:52:33.098+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:52:33.099 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:52:33.098+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:52:33.099 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:52:33.098+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:52:33.099 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:52:33.098+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:52:33.099 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:52:33.098+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:52:33.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.135+0000 7f76ffe00640 1 -- 172.21.15.151:0/191697094 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7708069890 con 0x7f76e807a2c0 2024-09-16T15:52:33.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.138+0000 7f770d400640 1 -- 172.21.15.151:0/191697094 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f76e807a2c0 msgr2=0x7f76e807c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:33.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.138+0000 7f770d400640 1 --2- 172.21.15.151:0/191697094 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f76e807a2c0 0x7f76e807c780 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f76f8007fd0 tx=0x7f76f8001390 comp rx=0 tx=0).stop 2024-09-16T15:52:33.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.138+0000 7f770d400640 1 -- 172.21.15.151:0/191697094 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7708070380 msgr2=0x7f77080c8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:33.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.138+0000 7f770d400640 1 --2- 172.21.15.151:0/191697094 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7708070380 0x7f77080c8db0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f76f402ab00 tx=0x7f76f4032c10 comp rx=0 tx=0).stop 2024-09-16T15:52:33.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.138+0000 7f770d400640 1 -- 172.21.15.151:0/191697094 shutdown_connections 2024-09-16T15:52:33.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.138+0000 7f770d400640 1 --2- 172.21.15.151:0/191697094 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f76e807a2c0 0x7f76e807c780 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:33.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.138+0000 7f770d400640 1 --2- 172.21.15.151:0/191697094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f770815bda0 0x7f77080c92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:33.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.138+0000 7f770d400640 1 --2- 172.21.15.151:0/191697094 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7708070380 0x7f77080c8db0 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:33.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.138+0000 7f770d400640 1 --2- 172.21.15.151:0/191697094 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f770806e590 0x7f7708152230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:33.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.138+0000 7f770d400640 1 -- 172.21.15.151:0/191697094 >> 172.21.15.151:0/191697094 conn(0x7f77080cfa70 msgr2=0x7f77080b2a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:33.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.138+0000 7f770d400640 1 -- 172.21.15.151:0/191697094 shutdown_connections 2024-09-16T15:52:33.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.138+0000 7f770d400640 1 -- 172.21.15.151:0/191697094 wait complete. 2024-09-16T15:52:33.156 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-16T15:52:33.318 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.315+0000 7f79cf400640 1 -- 172.21.15.151:0/1455603604 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79c8161760 msgr2=0x7f79c80ce4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:33.318 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.315+0000 7f79cf400640 1 --2- 172.21.15.151:0/1455603604 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79c8161760 0x7f79c80ce4a0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f79b8009830 tx=0x7f79b802a5c0 comp rx=0 tx=0).stop 2024-09-16T15:52:33.318 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.315+0000 7f79cf400640 1 -- 172.21.15.151:0/1455603604 shutdown_connections 2024-09-16T15:52:33.318 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.315+0000 7f79cf400640 1 --2- 172.21.15.151:0/1455603604 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79c8161760 0x7f79c80ce4a0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:33.318 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.315+0000 7f79cf400640 1 -- 172.21.15.151:0/1455603604 >> 172.21.15.151:0/1455603604 conn(0x7f79c80b1840 msgr2=0x7f79c80b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:33.318 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.315+0000 7f79cf400640 1 -- 172.21.15.151:0/1455603604 shutdown_connections 2024-09-16T15:52:33.318 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.316+0000 7f79cf400640 1 -- 172.21.15.151:0/1455603604 wait complete. 2024-09-16T15:52:33.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.317+0000 7f79cf400640 1 Processor -- start 2024-09-16T15:52:33.319 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.317+0000 7f79cf400640 1 -- start start 2024-09-16T15:52:33.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.318+0000 7f79cf400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79c80af190 0x7f79c81f8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:33.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.318+0000 7f79cf400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79c80cec60 0x7f79c815fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:33.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.318+0000 7f79cf400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79c8161760 0x7f79c8160170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:33.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.318+0000 7f79cf400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f79c81623a0 con 0x7f79c80cec60 2024-09-16T15:52:33.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.318+0000 7f79cf400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f79c8070530 con 0x7f79c80af190 2024-09-16T15:52:33.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.318+0000 7f79cf400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f79c8162520 con 0x7f79c8161760 2024-09-16T15:52:33.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.318+0000 7f79c7e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79c80cec60 0x7f79c815fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:33.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.318+0000 7f79c7e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79c80cec60 0x7f79c815fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:45266/0 (socket says 172.21.15.151:45266) 2024-09-16T15:52:33.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.318+0000 7f79c7e00640 1 -- 172.21.15.151:0/1835221757 learned_addr learned my addr 172.21.15.151:0/1835221757 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:52:33.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.319+0000 7f79cca00640 1 --2- 172.21.15.151:0/1835221757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79c80af190 0x7f79c81f8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:33.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.319+0000 7f79cd400640 1 --2- 172.21.15.151:0/1835221757 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79c8161760 0x7f79c8160170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:33.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.319+0000 7f79c7e00640 1 -- 172.21.15.151:0/1835221757 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79c8161760 msgr2=0x7f79c8160170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:33.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.319+0000 7f79c7e00640 1 --2- 172.21.15.151:0/1835221757 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79c8161760 0x7f79c8160170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:33.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.319+0000 7f79c7e00640 1 -- 172.21.15.151:0/1835221757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79c80af190 msgr2=0x7f79c81f8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:33.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.319+0000 7f79c7e00640 1 --2- 172.21.15.151:0/1835221757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79c80af190 0x7f79c81f8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:33.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.319+0000 7f79c7e00640 1 -- 172.21.15.151:0/1835221757 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79c81467e0 con 0x7f79c80cec60 2024-09-16T15:52:33.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.319+0000 7f79cca00640 1 --2- 172.21.15.151:0/1835221757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79c80af190 0x7f79c81f8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:52:33.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.319+0000 7f79cd400640 1 --2- 172.21.15.151:0/1835221757 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79c8161760 0x7f79c8160170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:52:33.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.319+0000 7f79c7e00640 1 --2- 172.21.15.151:0/1835221757 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79c80cec60 0x7f79c815fc30 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f79b400c530 tx=0x7f79b400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:33.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.319+0000 7f79c4c00640 1 -- 172.21.15.151:0/1835221757 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b40052d0 con 0x7f79c80cec60 2024-09-16T15:52:33.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.319+0000 7f79c4c00640 1 -- 172.21.15.151:0/1835221757 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f79b4005470 con 0x7f79c80cec60 2024-09-16T15:52:33.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.319+0000 7f79c4c00640 1 -- 172.21.15.151:0/1835221757 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b4012870 con 0x7f79c80cec60 2024-09-16T15:52:33.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.319+0000 7f79cf400640 1 -- 172.21.15.151:0/1835221757 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79c8146ac0 con 0x7f79c80cec60 2024-09-16T15:52:33.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.320+0000 7f79cf400640 1 -- 172.21.15.151:0/1835221757 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79c8147080 con 0x7f79c80cec60 2024-09-16T15:52:33.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.321+0000 7f79cf400640 1 -- 172.21.15.151:0/1835221757 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79c81629d0 con 0x7f79c80cec60 2024-09-16T15:52:33.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.322+0000 7f79c4c00640 1 -- 172.21.15.151:0/1835221757 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 41) ==== 103684+0+0 (secure 0 0 0) 0x7f79b40035f0 con 0x7f79c80cec60 2024-09-16T15:52:33.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.323+0000 7f79c4c00640 1 --2- 172.21.15.151:0/1835221757 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f79a807a2c0 0x7f79a807c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:33.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.323+0000 7f79c4c00640 1 -- 172.21.15.151:0/1835221757 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5793+0+0 (secure 0 0 0) 0x7f79b40993f0 con 0x7f79c80cec60 2024-09-16T15:52:33.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.323+0000 7f79cca00640 1 --2- 172.21.15.151:0/1835221757 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f79a807a2c0 0x7f79a807c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:33.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.324+0000 7f79cca00640 1 --2- 172.21.15.151:0/1835221757 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f79a807a2c0 0x7f79a807c780 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f79b8034000 tx=0x7f79b8004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:33.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.327+0000 7f79c4c00640 1 -- 172.21.15.151:0/1835221757 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f79b40634c0 con 0x7f79c80cec60 2024-09-16T15:52:33.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.540+0000 7f79cf400640 1 -- 172.21.15.151:0/1835221757 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f79c80ae7b0 con 0x7f79a807a2c0 2024-09-16T15:52:33.549 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.547+0000 7f79c4c00640 1 -- 172.21.15.151:0/1835221757 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f79c80ae7b0 con 0x7f79a807a2c0 2024-09-16T15:52:33.549 INFO:teuthology.orchestra.run.smithi151.stdout:/volumes/_nogroup/sv1/0926f44c-1384-450e-9d35-6c9cabee85aa 2024-09-16T15:52:33.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.550+0000 7f79cf400640 1 -- 172.21.15.151:0/1835221757 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f79a807a2c0 msgr2=0x7f79a807c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:33.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.550+0000 7f79cf400640 1 --2- 172.21.15.151:0/1835221757 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f79a807a2c0 0x7f79a807c780 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f79b8034000 tx=0x7f79b8004730 comp rx=0 tx=0).stop 2024-09-16T15:52:33.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.550+0000 7f79cf400640 1 -- 172.21.15.151:0/1835221757 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79c80cec60 msgr2=0x7f79c815fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:33.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.550+0000 7f79cf400640 1 --2- 172.21.15.151:0/1835221757 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79c80cec60 0x7f79c815fc30 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f79b400c530 tx=0x7f79b400c9f0 comp rx=0 tx=0).stop 2024-09-16T15:52:33.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.550+0000 7f79cf400640 1 -- 172.21.15.151:0/1835221757 shutdown_connections 2024-09-16T15:52:33.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.550+0000 7f79cf400640 1 --2- 172.21.15.151:0/1835221757 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f79a807a2c0 0x7f79a807c780 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:33.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.550+0000 7f79cf400640 1 --2- 172.21.15.151:0/1835221757 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79c8161760 0x7f79c8160170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:33.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.550+0000 7f79cf400640 1 --2- 172.21.15.151:0/1835221757 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79c80cec60 0x7f79c815fc30 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:33.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.550+0000 7f79cf400640 1 --2- 172.21.15.151:0/1835221757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79c80af190 0x7f79c81f8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:33.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.550+0000 7f79cf400640 1 -- 172.21.15.151:0/1835221757 >> 172.21.15.151:0/1835221757 conn(0x7f79c80b1840 msgr2=0x7f79c8073c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:33.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.550+0000 7f79cf400640 1 -- 172.21.15.151:0/1835221757 shutdown_connections 2024-09-16T15:52:33.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.550+0000 7f79cf400640 1 -- 172.21.15.151:0/1835221757 wait complete. 2024-09-16T15:52:33.568 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-16T15:52:33.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.728+0000 7feba3000640 1 -- 172.21.15.151:0/4271815692 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feb9c0cccd0 msgr2=0x7feb9c0cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:33.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.728+0000 7feba3000640 1 --2- 172.21.15.151:0/4271815692 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feb9c0cccd0 0x7feb9c0cdbb0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7feb9c0d0cd0 tx=0x7feb9402a560 comp rx=0 tx=0).stop 2024-09-16T15:52:33.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.728+0000 7feba3000640 1 -- 172.21.15.151:0/4271815692 shutdown_connections 2024-09-16T15:52:33.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.728+0000 7feba3000640 1 --2- 172.21.15.151:0/4271815692 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feb9c0cccd0 0x7feb9c0cdbb0 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:33.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.728+0000 7feba3000640 1 --2- 172.21.15.151:0/4271815692 >> v2:172.21.15.167:3300/0 conn(0x7feb9c145750 0x7feb9c073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:33.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.728+0000 7feba3000640 1 -- 172.21.15.151:0/4271815692 >> 172.21.15.151:0/4271815692 conn(0x7feb9c0b1840 msgr2=0x7feb9c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:33.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.728+0000 7feba3000640 1 -- 172.21.15.151:0/4271815692 shutdown_connections 2024-09-16T15:52:33.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.728+0000 7feba3000640 1 -- 172.21.15.151:0/4271815692 wait complete. 2024-09-16T15:52:33.732 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.730+0000 7feba3000640 1 Processor -- start 2024-09-16T15:52:33.732 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.730+0000 7feba3000640 1 -- start start 2024-09-16T15:52:33.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.731+0000 7feba3000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb9c06efa0 0x7feb9c13e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:33.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.731+0000 7feba3000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feb9c0cccd0 0x7feb9c1f84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:33.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.731+0000 7feba3000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb9c145750 0x7feb9c1f8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:33.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.731+0000 7feba3000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7feb9c0ce480 con 0x7feb9c0cccd0 2024-09-16T15:52:33.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.731+0000 7feba3000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7feb9c0ce300 con 0x7feb9c145750 2024-09-16T15:52:33.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.731+0000 7feba3000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7feb9c0ce600 con 0x7feb9c06efa0 2024-09-16T15:52:33.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.731+0000 7feb9b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feb9c0cccd0 0x7feb9c1f84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:33.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.731+0000 7feb9b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feb9c0cccd0 0x7feb9c1f84f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:45294/0 (socket says 172.21.15.151:45294) 2024-09-16T15:52:33.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.731+0000 7feb9b400640 1 -- 172.21.15.151:0/1781820880 learned_addr learned my addr 172.21.15.151:0/1781820880 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:52:33.735 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.731+0000 7feb9be00640 1 --2- 172.21.15.151:0/1781820880 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb9c06efa0 0x7feb9c13e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:33.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.731+0000 7feba1000640 1 --2- 172.21.15.151:0/1781820880 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb9c145750 0x7feb9c1f8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:33.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.732+0000 7feb9b400640 1 -- 172.21.15.151:0/1781820880 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb9c06efa0 msgr2=0x7feb9c13e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:33.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.732+0000 7feb9b400640 1 --2- 172.21.15.151:0/1781820880 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb9c06efa0 0x7feb9c13e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:33.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.732+0000 7feb9b400640 1 -- 172.21.15.151:0/1781820880 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb9c145750 msgr2=0x7feb9c1f8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:33.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.732+0000 7feb9b400640 1 --2- 172.21.15.151:0/1781820880 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb9c145750 0x7feb9c1f8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:33.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.732+0000 7feb9b400640 1 -- 172.21.15.151:0/1781820880 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb9c0c5e30 con 0x7feb9c0cccd0 2024-09-16T15:52:33.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.732+0000 7feb9b400640 1 --2- 172.21.15.151:0/1781820880 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feb9c0cccd0 0x7feb9c1f84f0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7feb8800c450 tx=0x7feb8800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:33.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.732+0000 7feb93e00640 1 -- 172.21.15.151:0/1781820880 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb880052d0 con 0x7feb9c0cccd0 2024-09-16T15:52:33.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.732+0000 7feb93e00640 1 -- 172.21.15.151:0/1781820880 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feb88005470 con 0x7feb9c0cccd0 2024-09-16T15:52:33.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.732+0000 7feb93e00640 1 -- 172.21.15.151:0/1781820880 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb88012870 con 0x7feb9c0cccd0 2024-09-16T15:52:33.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.732+0000 7feba3000640 1 -- 172.21.15.151:0/1781820880 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb9c0c65e0 con 0x7feb9c0cccd0 2024-09-16T15:52:33.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.732+0000 7feba3000640 1 -- 172.21.15.151:0/1781820880 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb9c0c6ba0 con 0x7feb9c0cccd0 2024-09-16T15:52:33.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.733+0000 7feba3000640 1 -- 172.21.15.151:0/1781820880 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb9c0cdbb0 con 0x7feb9c0cccd0 2024-09-16T15:52:33.737 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.735+0000 7feb93e00640 1 -- 172.21.15.151:0/1781820880 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 41) ==== 103684+0+0 (secure 0 0 0) 0x7feb880035f0 con 0x7feb9c0cccd0 2024-09-16T15:52:33.738 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.736+0000 7feb93e00640 1 --2- 172.21.15.151:0/1781820880 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7feb7c07a2c0 0x7feb7c07c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:33.738 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.736+0000 7feb93e00640 1 -- 172.21.15.151:0/1781820880 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5793+0+0 (secure 0 0 0) 0x7feb88099230 con 0x7feb9c0cccd0 2024-09-16T15:52:33.738 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.736+0000 7feb9be00640 1 --2- 172.21.15.151:0/1781820880 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7feb7c07a2c0 0x7feb7c07c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:33.739 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.737+0000 7feb9be00640 1 --2- 172.21.15.151:0/1781820880 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7feb7c07a2c0 0x7feb7c07c780 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7feb8c005fd0 tx=0x7feb8c009500 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:33.742 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.740+0000 7feb93e00640 1 -- 172.21.15.151:0/1781820880 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7feb88062cb0 con 0x7feb9c0cccd0 2024-09-16T15:52:33.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.951+0000 7feba3000640 1 -- 172.21.15.151:0/1781820880 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7feb9c0c63f0 con 0x7feb7c07a2c0 2024-09-16T15:52:33.959 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.958+0000 7feb93e00640 1 -- 172.21.15.151:0/1781820880 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7feb9c0c63f0 con 0x7feb7c07a2c0 2024-09-16T15:52:33.959 INFO:teuthology.orchestra.run.smithi151.stdout:/volumes/_nogroup/sv1/0926f44c-1384-450e-9d35-6c9cabee85aa 2024-09-16T15:52:33.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.960+0000 7feba3000640 1 -- 172.21.15.151:0/1781820880 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7feb7c07a2c0 msgr2=0x7feb7c07c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:33.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.960+0000 7feba3000640 1 --2- 172.21.15.151:0/1781820880 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7feb7c07a2c0 0x7feb7c07c780 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7feb8c005fd0 tx=0x7feb8c009500 comp rx=0 tx=0).stop 2024-09-16T15:52:33.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.960+0000 7feba3000640 1 -- 172.21.15.151:0/1781820880 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feb9c0cccd0 msgr2=0x7feb9c1f84f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:33.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.960+0000 7feba3000640 1 --2- 172.21.15.151:0/1781820880 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feb9c0cccd0 0x7feb9c1f84f0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7feb8800c450 tx=0x7feb8800c910 comp rx=0 tx=0).stop 2024-09-16T15:52:33.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.960+0000 7feba3000640 1 -- 172.21.15.151:0/1781820880 shutdown_connections 2024-09-16T15:52:33.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.960+0000 7feba3000640 1 --2- 172.21.15.151:0/1781820880 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7feb7c07a2c0 0x7feb7c07c780 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:33.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.960+0000 7feba3000640 1 --2- 172.21.15.151:0/1781820880 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb9c145750 0x7feb9c1f8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:33.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.960+0000 7feba3000640 1 --2- 172.21.15.151:0/1781820880 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feb9c0cccd0 0x7feb9c1f84f0 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:33.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.960+0000 7feba3000640 1 --2- 172.21.15.151:0/1781820880 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7feb9c06efa0 0x7feb9c13e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:33.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.960+0000 7feba3000640 1 -- 172.21.15.151:0/1781820880 >> 172.21.15.151:0/1781820880 conn(0x7feb9c0b1840 msgr2=0x7feb9c0b0640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:33.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.960+0000 7feba3000640 1 -- 172.21.15.151:0/1781820880 shutdown_connections 2024-09-16T15:52:33.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:33.960+0000 7feba3000640 1 -- 172.21.15.151:0/1781820880 wait complete. 2024-09-16T15:52:33.978 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/0926f44c-1384-450e-9d35-6c9cabee85aa 2024-09-16T15:52:33.979 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:52:33.979 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:52:34.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:52:34.014 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/0926f44c-1384-450e-9d35-6c9cabee85aa/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T15:52:42.028 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records in 2024-09-16T15:52:42.028 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records out 2024-09-16T15:52:42.028 INFO:teuthology.orchestra.run.smithi151.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.93171 s, 135 MB/s 2024-09-16T15:52:42.029 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:52:42.029 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:52:42.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:52:42.065 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/0926f44c-1384-450e-9d35-6c9cabee85aa/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T15:52:50.211 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records in 2024-09-16T15:52:50.211 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records out 2024-09-16T15:52:50.211 INFO:teuthology.orchestra.run.smithi151.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.06858 s, 133 MB/s 2024-09-16T15:52:50.212 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:52:50.213 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:52:50.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:52:50.249 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/0926f44c-1384-450e-9d35-6c9cabee85aa/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T15:52:51.274 DEBUG:teuthology.orchestra.run.smithi151:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:52:51.278 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:52:58.460 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records in 2024-09-16T15:52:58.460 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records out 2024-09-16T15:52:58.461 INFO:teuthology.orchestra.run.smithi151.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.1349 s, 132 MB/s 2024-09-16T15:52:58.462 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-09-16T15:52:58.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:58.624+0000 7f37bd800640 1 -- 172.21.15.151:0/3911657707 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37b815f9a0 msgr2=0x7f37b815fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:58.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:58.625+0000 7f37bd800640 1 --2- 172.21.15.151:0/3911657707 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37b815f9a0 0x7f37b815fd80 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f37a8005fd0 tx=0x7f37a802a5a0 comp rx=0 tx=0).stop 2024-09-16T15:52:58.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:58.625+0000 7f37bd800640 1 -- 172.21.15.151:0/3911657707 shutdown_connections 2024-09-16T15:52:58.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:58.625+0000 7f37bd800640 1 --2- 172.21.15.151:0/3911657707 >> v2:172.21.15.167:3300/0 conn(0x7f37b806e6f0 0x7f37b806eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:58.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:58.625+0000 7f37bd800640 1 --2- 172.21.15.151:0/3911657707 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37b815f9a0 0x7f37b815fd80 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:58.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:58.625+0000 7f37bd800640 1 -- 172.21.15.151:0/3911657707 >> 172.21.15.151:0/3911657707 conn(0x7f37b80b18e0 msgr2=0x7f37b80b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:58.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:58.625+0000 7f37bd800640 1 -- 172.21.15.151:0/3911657707 shutdown_connections 2024-09-16T15:52:58.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:58.625+0000 7f37bd800640 1 -- 172.21.15.151:0/3911657707 wait complete. 2024-09-16T15:52:58.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:58.626+0000 7f37bd800640 1 Processor -- start 2024-09-16T15:52:58.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:58.627+0000 7f37bd800640 1 -- start start 2024-09-16T15:52:58.630 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:58.627+0000 7f37bd800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f37b806e6f0 0x7f37b809cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:58.630 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:58.628+0000 7f37bd800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37b8073de0 0x7f37b81f8230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:58.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:58.628+0000 7f37bd800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37b815f9a0 0x7f37b81526a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:58.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:58.628+0000 7f37bd800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f37b80cd550 con 0x7f37b8073de0 2024-09-16T15:52:58.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:58.628+0000 7f37bd800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f37b80cd3d0 con 0x7f37b815f9a0 2024-09-16T15:52:58.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:58.628+0000 7f37bd800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f37b806f0e0 con 0x7f37b806e6f0 2024-09-16T15:52:58.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:58.628+0000 7f37b6000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37b8073de0 0x7f37b81f8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:58.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:58.628+0000 7f37b7400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37b815f9a0 0x7f37b81526a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:58.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:58.628+0000 7f37b6a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f37b806e6f0 0x7f37b809cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:58.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:58.628+0000 7f37b6000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37b8073de0 0x7f37b81f8230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47318/0 (socket says 172.21.15.151:47318) 2024-09-16T15:52:58.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:58.628+0000 7f37b7400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37b815f9a0 0x7f37b81526a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.151:36064/0 (socket says 172.21.15.151:36064) 2024-09-16T15:52:58.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:58.628+0000 7f37b6000640 1 -- 172.21.15.151:0/1557317830 learned_addr learned my addr 172.21.15.151:0/1557317830 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:52:58.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:58.628+0000 7f37b7400640 1 -- 172.21.15.151:0/1557317830 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f37b806e6f0 msgr2=0x7f37b809cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:58.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:58.628+0000 7f37b7400640 1 --2- 172.21.15.151:0/1557317830 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f37b806e6f0 0x7f37b809cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:58.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:58.628+0000 7f37b7400640 1 -- 172.21.15.151:0/1557317830 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37b8073de0 msgr2=0x7f37b81f8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:58.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:58.628+0000 7f37b7400640 1 --2- 172.21.15.151:0/1557317830 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37b8073de0 0x7f37b81f8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:58.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:58.628+0000 7f37b7400640 1 -- 172.21.15.151:0/1557317830 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37b81f8770 con 0x7f37b815f9a0 2024-09-16T15:52:58.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:58.629+0000 7f37b7400640 1 --2- 172.21.15.151:0/1557317830 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37b815f9a0 0x7f37b81526a0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f37b0009fd0 tx=0x7f37b000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:58.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:58.629+0000 7f37aea00640 1 -- 172.21.15.151:0/1557317830 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b000ce20 con 0x7f37b815f9a0 2024-09-16T15:52:58.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:58.629+0000 7f37aea00640 1 -- 172.21.15.151:0/1557317830 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f37b0004c10 con 0x7f37b815f9a0 2024-09-16T15:52:58.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:58.629+0000 7f37bd800640 1 -- 172.21.15.151:0/1557317830 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37b81f89f0 con 0x7f37b815f9a0 2024-09-16T15:52:58.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:58.629+0000 7f37aea00640 1 -- 172.21.15.151:0/1557317830 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b0005020 con 0x7f37b815f9a0 2024-09-16T15:52:58.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:58.629+0000 7f37bd800640 1 -- 172.21.15.151:0/1557317830 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37b81f8fb0 con 0x7f37b815f9a0 2024-09-16T15:52:58.634 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:58.630+0000 7f37bd800640 1 -- 172.21.15.151:0/1557317830 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37b806f560 con 0x7f37b815f9a0 2024-09-16T15:52:58.634 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:58.632+0000 7f37aea00640 1 -- 172.21.15.151:0/1557317830 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 42) ==== 103786+0+0 (secure 0 0 0) 0x7f37b00051c0 con 0x7f37b815f9a0 2024-09-16T15:52:58.634 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:58.632+0000 7f37aea00640 1 --2- 172.21.15.151:0/1557317830 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f379807a3e0 0x7f379807c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:58.634 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:58.633+0000 7f37aea00640 1 -- 172.21.15.151:0/1557317830 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5793+0+0 (secure 0 0 0) 0x7f37b009a580 con 0x7f37b815f9a0 2024-09-16T15:52:58.634 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:58.633+0000 7f37b6a00640 1 --2- 172.21.15.151:0/1557317830 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f379807a3e0 0x7f379807c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:58.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:58.633+0000 7f37b6a00640 1 --2- 172.21.15.151:0/1557317830 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f379807a3e0 0x7f379807c8a0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f37a8033000 tx=0x7f37a8004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:58.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:58.637+0000 7f37aea00640 1 -- 172.21.15.151:0/1557317830 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f37b81f8770 con 0x7f37b815f9a0 2024-09-16T15:52:58.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:58.854+0000 7f37bd800640 1 -- 172.21.15.151:0/1557317830 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7f37b80cb520 con 0x7f379807a3e0 2024-09-16T15:52:58.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:58.868+0000 7f37aea00640 1 -- 172.21.15.151:0/1557317830 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f37b80cb520 con 0x7f379807a3e0 2024-09-16T15:52:58.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:58.871+0000 7f37bd800640 1 -- 172.21.15.151:0/1557317830 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f379807a3e0 msgr2=0x7f379807c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:58.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:58.871+0000 7f37bd800640 1 --2- 172.21.15.151:0/1557317830 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f379807a3e0 0x7f379807c8a0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f37a8033000 tx=0x7f37a8004730 comp rx=0 tx=0).stop 2024-09-16T15:52:58.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:58.871+0000 7f37bd800640 1 -- 172.21.15.151:0/1557317830 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37b815f9a0 msgr2=0x7f37b81526a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:58.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:58.871+0000 7f37bd800640 1 --2- 172.21.15.151:0/1557317830 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37b815f9a0 0x7f37b81526a0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f37b0009fd0 tx=0x7f37b000ef10 comp rx=0 tx=0).stop 2024-09-16T15:52:58.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:58.871+0000 7f37bd800640 1 -- 172.21.15.151:0/1557317830 shutdown_connections 2024-09-16T15:52:58.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:58.871+0000 7f37bd800640 1 --2- 172.21.15.151:0/1557317830 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f379807a3e0 0x7f379807c8a0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:58.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:58.871+0000 7f37bd800640 1 --2- 172.21.15.151:0/1557317830 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37b815f9a0 0x7f37b81526a0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:58.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:58.871+0000 7f37bd800640 1 --2- 172.21.15.151:0/1557317830 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f37b8073de0 0x7f37b81f8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:58.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:58.871+0000 7f37bd800640 1 --2- 172.21.15.151:0/1557317830 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f37b806e6f0 0x7f37b809cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:58.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:58.871+0000 7f37bd800640 1 -- 172.21.15.151:0/1557317830 >> 172.21.15.151:0/1557317830 conn(0x7f37b80b18e0 msgr2=0x7f37b80aed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:58.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:58.871+0000 7f37bd800640 1 -- 172.21.15.151:0/1557317830 shutdown_connections 2024-09-16T15:52:58.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:58.871+0000 7f37bd800640 1 -- 172.21.15.151:0/1557317830 wait complete. 2024-09-16T15:52:58.888 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-16T15:52:59.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:59.048+0000 7f0e49400640 1 -- 172.21.15.151:0/834108872 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0e440b3790 msgr2=0x7f0e440ce920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:59.050 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:59.048+0000 7f0e49400640 1 --2- 172.21.15.151:0/834108872 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0e440b3790 0x7f0e440ce920 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0e34005fd0 tx=0x7f0e3402a5d0 comp rx=0 tx=0).stop 2024-09-16T15:52:59.050 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:59.048+0000 7f0e49400640 1 -- 172.21.15.151:0/834108872 shutdown_connections 2024-09-16T15:52:59.050 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:59.048+0000 7f0e49400640 1 --2- 172.21.15.151:0/834108872 >> v2:172.21.15.167:3300/0 conn(0x7f0e440b4d10 0x7f0e440d1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:59.050 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:59.048+0000 7f0e49400640 1 --2- 172.21.15.151:0/834108872 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0e440b3790 0x7f0e440ce920 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:59.050 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:59.048+0000 7f0e49400640 1 -- 172.21.15.151:0/834108872 >> 172.21.15.151:0/834108872 conn(0x7f0e4409d230 msgr2=0x7f0e440d3f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:59.050 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:59.048+0000 7f0e49400640 1 -- 172.21.15.151:0/834108872 shutdown_connections 2024-09-16T15:52:59.050 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:59.048+0000 7f0e49400640 1 -- 172.21.15.151:0/834108872 wait complete. 2024-09-16T15:52:59.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:59.050+0000 7f0e49400640 1 Processor -- start 2024-09-16T15:52:59.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:59.050+0000 7f0e49400640 1 -- start start 2024-09-16T15:52:59.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:59.051+0000 7f0e49400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0e440b3790 0x7f0e441f8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:59.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:59.051+0000 7f0e49400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e440b3b70 0x7f0e440ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:59.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:59.051+0000 7f0e49400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0e440b4d10 0x7f0e440c92a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:59.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:59.051+0000 7f0e49400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0e440b1340 con 0x7f0e440b4d10 2024-09-16T15:52:59.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:59.051+0000 7f0e49400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0e440b11c0 con 0x7f0e440b3b70 2024-09-16T15:52:59.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:59.051+0000 7f0e49400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f0e440b14c0 con 0x7f0e440b3790 2024-09-16T15:52:59.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:59.051+0000 7f0e43e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0e440b3790 0x7f0e441f8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:59.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:59.051+0000 7f0e48a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0e440b4d10 0x7f0e440c92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:59.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:59.051+0000 7f0e3be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e440b3b70 0x7f0e440ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:59.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:59.051+0000 7f0e43e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0e440b3790 0x7f0e441f8c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.151:57432/0 (socket says 172.21.15.151:57432) 2024-09-16T15:52:59.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:59.051+0000 7f0e43e00640 1 -- 172.21.15.151:0/371187900 learned_addr learned my addr 172.21.15.151:0/371187900 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:52:59.054 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:59.051+0000 7f0e43e00640 1 -- 172.21.15.151:0/371187900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e440b3b70 msgr2=0x7f0e440ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:59.054 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:59.051+0000 7f0e43e00640 1 --2- 172.21.15.151:0/371187900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e440b3b70 0x7f0e440ccdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:59.054 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:59.051+0000 7f0e43e00640 1 -- 172.21.15.151:0/371187900 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0e440b4d10 msgr2=0x7f0e440c92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:59.054 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:59.051+0000 7f0e43e00640 1 --2- 172.21.15.151:0/371187900 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0e440b4d10 0x7f0e440c92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:59.054 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:59.051+0000 7f0e43e00640 1 -- 172.21.15.151:0/371187900 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e441f91c0 con 0x7f0e440b3790 2024-09-16T15:52:59.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:59.051+0000 7f0e48a00640 1 --2- 172.21.15.151:0/371187900 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0e440b4d10 0x7f0e440c92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:52:59.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:59.052+0000 7f0e43e00640 1 --2- 172.21.15.151:0/371187900 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0e440b3790 0x7f0e441f8c80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0e34009850 tx=0x7f0e34005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:59.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:59.052+0000 7f0e40c00640 1 -- 172.21.15.151:0/371187900 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e3403f070 con 0x7f0e440b3790 2024-09-16T15:52:59.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:59.052+0000 7f0e40c00640 1 -- 172.21.15.151:0/371187900 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e34003bf0 con 0x7f0e440b3790 2024-09-16T15:52:59.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:59.052+0000 7f0e40c00640 1 -- 172.21.15.151:0/371187900 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e34035030 con 0x7f0e440b3790 2024-09-16T15:52:59.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:59.052+0000 7f0e49400640 1 -- 172.21.15.151:0/371187900 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e441f4f70 con 0x7f0e440b3790 2024-09-16T15:52:59.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:59.052+0000 7f0e49400640 1 -- 172.21.15.151:0/371187900 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e441f5210 con 0x7f0e440b3790 2024-09-16T15:52:59.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:59.053+0000 7f0e49400640 1 -- 172.21.15.151:0/371187900 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e440d1d60 con 0x7f0e440b3790 2024-09-16T15:52:59.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:59.054+0000 7f0e40c00640 1 -- 172.21.15.151:0/371187900 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 42) ==== 103786+0+0 (secure 0 0 0) 0x7f0e34042070 con 0x7f0e440b3790 2024-09-16T15:52:59.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:59.055+0000 7f0e40c00640 1 --2- 172.21.15.151:0/371187900 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0e2007a410 0x7f0e2007c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:52:59.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:59.055+0000 7f0e40c00640 1 -- 172.21.15.151:0/371187900 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(118..118 src has 1..118) ==== 5793+0+0 (secure 0 0 0) 0x7f0e340ba510 con 0x7f0e440b3790 2024-09-16T15:52:59.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:59.056+0000 7f0e3be00640 1 --2- 172.21.15.151:0/371187900 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0e2007a410 0x7f0e2007c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:52:59.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:59.056+0000 7f0e3be00640 1 --2- 172.21.15.151:0/371187900 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0e2007a410 0x7f0e2007c8d0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c0095f0 tx=0x7f0e2c005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:52:59.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:59.059+0000 7f0e40c00640 1 -- 172.21.15.151:0/371187900 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0e34084010 con 0x7f0e440b3790 2024-09-16T15:52:59.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:59.270+0000 7f0e49400640 1 -- 172.21.15.151:0/371187900 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f0e440d17e0 con 0x7f0e2007a410 2024-09-16T15:52:59.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:59.275+0000 7f0e40c00640 1 -- 172.21.15.151:0/371187900 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f0e440d17e0 con 0x7f0e2007a410 2024-09-16T15:52:59.276 INFO:teuthology.orchestra.run.smithi151.stdout:/volumes/_nogroup/sv1/0926f44c-1384-450e-9d35-6c9cabee85aa 2024-09-16T15:52:59.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:59.277+0000 7f0e49400640 1 -- 172.21.15.151:0/371187900 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0e2007a410 msgr2=0x7f0e2007c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:59.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:59.277+0000 7f0e49400640 1 --2- 172.21.15.151:0/371187900 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0e2007a410 0x7f0e2007c8d0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c0095f0 tx=0x7f0e2c005f90 comp rx=0 tx=0).stop 2024-09-16T15:52:59.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:59.277+0000 7f0e49400640 1 -- 172.21.15.151:0/371187900 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0e440b3790 msgr2=0x7f0e441f8c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:52:59.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:59.277+0000 7f0e49400640 1 --2- 172.21.15.151:0/371187900 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0e440b3790 0x7f0e441f8c80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0e34009850 tx=0x7f0e34005e80 comp rx=0 tx=0).stop 2024-09-16T15:52:59.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:59.277+0000 7f0e49400640 1 -- 172.21.15.151:0/371187900 shutdown_connections 2024-09-16T15:52:59.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:59.277+0000 7f0e49400640 1 --2- 172.21.15.151:0/371187900 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0e2007a410 0x7f0e2007c8d0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:59.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:59.277+0000 7f0e49400640 1 --2- 172.21.15.151:0/371187900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e440b3b70 0x7f0e440ccdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:59.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:59.277+0000 7f0e49400640 1 --2- 172.21.15.151:0/371187900 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0e440b4d10 0x7f0e440c92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:59.279 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:59.277+0000 7f0e49400640 1 --2- 172.21.15.151:0/371187900 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0e440b3790 0x7f0e441f8c80 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:52:59.280 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:59.277+0000 7f0e49400640 1 -- 172.21.15.151:0/371187900 >> 172.21.15.151:0/371187900 conn(0x7f0e4409d230 msgr2=0x7f0e440d1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:52:59.280 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:59.277+0000 7f0e49400640 1 -- 172.21.15.151:0/371187900 shutdown_connections 2024-09-16T15:52:59.280 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:52:59.278+0000 7f0e49400640 1 -- 172.21.15.151:0/371187900 wait complete. 2024-09-16T15:52:59.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:52:59.295 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/0926f44c-1384-450e-9d35-6c9cabee85aa 2024-09-16T15:52:59.295 DEBUG:teuthology.orchestra.run.smithi151:> ) 2024-09-16T15:52:59.330 INFO:teuthology.orchestra.run.smithi151.stdout:# file: volumes/_nogroup/sv1/0926f44c-1384-450e-9d35-6c9cabee85aa 2024-09-16T15:52:59.330 INFO:teuthology.orchestra.run.smithi151.stdout:ceph.dir.rbytes="2972172288" 2024-09-16T15:52:59.330 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:52:59.331 INFO:tasks.cephfs.test_volumes:r_size = 2972172288 exp_size = 3221225472 2024-09-16T15:53:01.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:53:01.332 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/0926f44c-1384-450e-9d35-6c9cabee85aa 2024-09-16T15:53:01.332 DEBUG:teuthology.orchestra.run.smithi151:> ) 2024-09-16T15:53:01.372 INFO:teuthology.orchestra.run.smithi151.stdout:# file: volumes/_nogroup/sv1/0926f44c-1384-450e-9d35-6c9cabee85aa 2024-09-16T15:53:01.372 INFO:teuthology.orchestra.run.smithi151.stdout:ceph.dir.rbytes="2972172288" 2024-09-16T15:53:01.372 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:53:01.373 INFO:tasks.cephfs.test_volumes:r_size = 2972172288 exp_size = 3221225472 2024-09-16T15:53:03.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:53:03.374 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/0926f44c-1384-450e-9d35-6c9cabee85aa 2024-09-16T15:53:03.374 DEBUG:teuthology.orchestra.run.smithi151:> ) 2024-09-16T15:53:03.414 INFO:teuthology.orchestra.run.smithi151.stdout:# file: volumes/_nogroup/sv1/0926f44c-1384-450e-9d35-6c9cabee85aa 2024-09-16T15:53:03.414 INFO:teuthology.orchestra.run.smithi151.stdout:ceph.dir.rbytes="2972172288" 2024-09-16T15:53:03.415 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:53:03.415 INFO:tasks.cephfs.test_volumes:r_size = 2972172288 exp_size = 3221225472 2024-09-16T15:53:05.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:53:05.417 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/0926f44c-1384-450e-9d35-6c9cabee85aa 2024-09-16T15:53:05.417 DEBUG:teuthology.orchestra.run.smithi151:> ) 2024-09-16T15:53:05.458 INFO:teuthology.orchestra.run.smithi151.stdout:# file: volumes/_nogroup/sv1/0926f44c-1384-450e-9d35-6c9cabee85aa 2024-09-16T15:53:05.459 INFO:teuthology.orchestra.run.smithi151.stdout:ceph.dir.rbytes="3221225472" 2024-09-16T15:53:05.459 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:53:05.459 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 exp_size = 3221225472 2024-09-16T15:53:05.460 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone52540 2024-09-16T15:53:05.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:05.667+0000 7f4334a00640 1 -- 172.21.15.151:0/104057084 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f43300b3370 msgr2=0x7f43300cebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:05.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:05.667+0000 7f4334a00640 1 --2- 172.21.15.151:0/104057084 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f43300b3370 0x7f43300cebe0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f4320005fd0 tx=0x7f432002a5d0 comp rx=0 tx=0).stop 2024-09-16T15:53:05.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:05.667+0000 7f4334a00640 1 -- 172.21.15.151:0/104057084 shutdown_connections 2024-09-16T15:53:05.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:05.667+0000 7f4334a00640 1 --2- 172.21.15.151:0/104057084 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f433015f090 0x7f43300cf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:05.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:05.668+0000 7f4334a00640 1 --2- 172.21.15.151:0/104057084 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f43300b3370 0x7f43300cebe0 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:05.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:05.668+0000 7f4334a00640 1 -- 172.21.15.151:0/104057084 >> 172.21.15.151:0/104057084 conn(0x7f43300b1840 msgr2=0x7f43300b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:05.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:05.668+0000 7f4334a00640 1 -- 172.21.15.151:0/104057084 shutdown_connections 2024-09-16T15:53:05.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:05.668+0000 7f4334a00640 1 -- 172.21.15.151:0/104057084 wait complete. 2024-09-16T15:53:05.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:05.669+0000 7f4334a00640 1 Processor -- start 2024-09-16T15:53:05.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:05.669+0000 7f4334a00640 1 -- start start 2024-09-16T15:53:05.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:05.670+0000 7f4334a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43300b3370 0x7f4330151650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:05.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:05.670+0000 7f4334a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f433015c080 0x7f433015dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:05.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:05.670+0000 7f4334a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f433015f090 0x7f433013a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:05.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:05.670+0000 7f4334a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4330164e50 con 0x7f433015c080 2024-09-16T15:53:05.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:05.670+0000 7f4334a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4330164cd0 con 0x7f43300b3370 2024-09-16T15:53:05.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:05.670+0000 7f4334a00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f4330164fd0 con 0x7f433015f090 2024-09-16T15:53:05.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:05.670+0000 7f432d400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f433015c080 0x7f433015dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:05.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:05.671+0000 7f432d400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f433015c080 0x7f433015dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53664/0 (socket says 172.21.15.151:53664) 2024-09-16T15:53:05.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:05.671+0000 7f432d400640 1 -- 172.21.15.151:0/2085818653 learned_addr learned my addr 172.21.15.151:0/2085818653 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:53:05.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:05.671+0000 7f432de00640 1 --2- 172.21.15.151:0/2085818653 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43300b3370 0x7f4330151650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:05.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:05.671+0000 7f432e800640 1 --2- 172.21.15.151:0/2085818653 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f433015f090 0x7f433013a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:05.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:05.671+0000 7f432d400640 1 -- 172.21.15.151:0/2085818653 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f433015f090 msgr2=0x7f433013a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:05.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:05.671+0000 7f432d400640 1 --2- 172.21.15.151:0/2085818653 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f433015f090 0x7f433013a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:05.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:05.671+0000 7f432d400640 1 -- 172.21.15.151:0/2085818653 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43300b3370 msgr2=0x7f4330151650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:05.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:05.671+0000 7f432d400640 1 --2- 172.21.15.151:0/2085818653 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43300b3370 0x7f4330151650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:05.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:05.671+0000 7f432d400640 1 -- 172.21.15.151:0/2085818653 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4330139960 con 0x7f433015c080 2024-09-16T15:53:05.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:05.671+0000 7f432d400640 1 --2- 172.21.15.151:0/2085818653 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f433015c080 0x7f433015dac0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f431c00c400 tx=0x7f431c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:05.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:05.671+0000 7f4326000640 1 -- 172.21.15.151:0/2085818653 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431c017070 con 0x7f433015c080 2024-09-16T15:53:05.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:05.671+0000 7f4326000640 1 -- 172.21.15.151:0/2085818653 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f431c0053e0 con 0x7f433015c080 2024-09-16T15:53:05.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:05.671+0000 7f4334a00640 1 -- 172.21.15.151:0/2085818653 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4330139c40 con 0x7f433015c080 2024-09-16T15:53:05.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:05.671+0000 7f4326000640 1 -- 172.21.15.151:0/2085818653 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431c003870 con 0x7f433015c080 2024-09-16T15:53:05.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:05.672+0000 7f4334a00640 1 -- 172.21.15.151:0/2085818653 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f433015e000 con 0x7f433015c080 2024-09-16T15:53:05.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:05.673+0000 7f4334a00640 1 -- 172.21.15.151:0/2085818653 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f433015e980 con 0x7f433015c080 2024-09-16T15:53:05.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:05.674+0000 7f4326000640 1 -- 172.21.15.151:0/2085818653 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 42) ==== 103786+0+0 (secure 0 0 0) 0x7f431c01a070 con 0x7f433015c080 2024-09-16T15:53:05.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:05.675+0000 7f4326000640 1 --2- 172.21.15.151:0/2085818653 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f431007a440 0x7f431007c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:05.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:05.675+0000 7f4326000640 1 -- 172.21.15.151:0/2085818653 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5851+0+0 (secure 0 0 0) 0x7f431c098b70 con 0x7f433015c080 2024-09-16T15:53:05.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:05.675+0000 7f432de00640 1 --2- 172.21.15.151:0/2085818653 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f431007a440 0x7f431007c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:05.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:05.676+0000 7f432de00640 1 --2- 172.21.15.151:0/2085818653 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f431007a440 0x7f431007c900 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f4320033000 tx=0x7f4320004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:05.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:05.679+0000 7f4326000640 1 -- 172.21.15.151:0/2085818653 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f433015b320 con 0x7f433015c080 2024-09-16T15:53:05.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:05.899+0000 7f4334a00640 1 -- 172.21.15.151:0/2085818653 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone52540", "target": ["mon-mgr", ""]}) -- 0x7f4330138860 con 0x7f431007a440 2024-09-16T15:53:05.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:05.969+0000 7f4326000640 1 -- 172.21.15.151:0/2085818653 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4330138860 con 0x7f431007a440 2024-09-16T15:53:05.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:05.972+0000 7f4334a00640 1 -- 172.21.15.151:0/2085818653 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f431007a440 msgr2=0x7f431007c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:05.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:05.972+0000 7f4334a00640 1 --2- 172.21.15.151:0/2085818653 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f431007a440 0x7f431007c900 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f4320033000 tx=0x7f4320004730 comp rx=0 tx=0).stop 2024-09-16T15:53:05.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:05.972+0000 7f4334a00640 1 -- 172.21.15.151:0/2085818653 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f433015c080 msgr2=0x7f433015dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:05.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:05.972+0000 7f4334a00640 1 --2- 172.21.15.151:0/2085818653 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f433015c080 0x7f433015dac0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f431c00c400 tx=0x7f431c00c8c0 comp rx=0 tx=0).stop 2024-09-16T15:53:05.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:05.972+0000 7f4334a00640 1 -- 172.21.15.151:0/2085818653 shutdown_connections 2024-09-16T15:53:05.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:05.972+0000 7f4334a00640 1 --2- 172.21.15.151:0/2085818653 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f431007a440 0x7f431007c900 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:05.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:05.972+0000 7f4334a00640 1 --2- 172.21.15.151:0/2085818653 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f433015f090 0x7f433013a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:05.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:05.972+0000 7f4334a00640 1 --2- 172.21.15.151:0/2085818653 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f433015c080 0x7f433015dac0 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:05.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:05.972+0000 7f4334a00640 1 --2- 172.21.15.151:0/2085818653 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43300b3370 0x7f4330151650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:05.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:05.972+0000 7f4334a00640 1 -- 172.21.15.151:0/2085818653 >> 172.21.15.151:0/2085818653 conn(0x7f43300b1840 msgr2=0x7f433013ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:05.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:05.972+0000 7f4334a00640 1 -- 172.21.15.151:0/2085818653 shutdown_connections 2024-09-16T15:53:05.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:05.972+0000 7f4334a00640 1 -- 172.21.15.151:0/2085818653 wait complete. 2024-09-16T15:53:05.975 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:53:05.973+0000 7f48f7600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:53:05.975 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:53:05.973+0000 7f48f7600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:53:05.975 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:53:05.973+0000 7f48f7600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:53:05.975 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:53:05.973+0000 7f48f7600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:53:05.975 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:53:05.973+0000 7f48f7600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:53:05.990 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone15851 2024-09-16T15:53:05.991 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:53:05.990+0000 7f48f5800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:53:05.992 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:53:05.990+0000 7f48f5800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:53:05.992 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:53:05.990+0000 7f48f5800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:53:05.992 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:53:05.990+0000 7f48f5800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:53:05.992 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:53:05.990+0000 7f48f5800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:53:06.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:06.160+0000 7f4410a00640 1 -- 172.21.15.151:0/2997367687 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f440c145750 msgr2=0x7f440c073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:06.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:06.160+0000 7f4410a00640 1 --2- 172.21.15.151:0/2997367687 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f440c145750 0x7f440c073440 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f440c0cfb70 tx=0x7f43fc02a560 comp rx=0 tx=0).stop 2024-09-16T15:53:06.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:06.161+0000 7f4410a00640 1 -- 172.21.15.151:0/2997367687 shutdown_connections 2024-09-16T15:53:06.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:06.161+0000 7f4410a00640 1 --2- 172.21.15.151:0/2997367687 >> v2:172.21.15.167:3300/0 conn(0x7f440c0cccd0 0x7f440c0cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:06.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:06.161+0000 7f4410a00640 1 --2- 172.21.15.151:0/2997367687 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f440c145750 0x7f440c073440 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:06.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:06.161+0000 7f4410a00640 1 -- 172.21.15.151:0/2997367687 >> 172.21.15.151:0/2997367687 conn(0x7f440c0b1840 msgr2=0x7f440c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:06.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:06.161+0000 7f4410a00640 1 -- 172.21.15.151:0/2997367687 shutdown_connections 2024-09-16T15:53:06.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:06.161+0000 7f4410a00640 1 -- 172.21.15.151:0/2997367687 wait complete. 2024-09-16T15:53:06.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:06.162+0000 7f4410a00640 1 Processor -- start 2024-09-16T15:53:06.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:06.162+0000 7f4410a00640 1 -- start start 2024-09-16T15:53:06.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:06.163+0000 7f4410a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f440c06efa0 0x7f440c1f4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:06.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:06.163+0000 7f4410a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f440c0cccd0 0x7f440c0b0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:06.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:06.163+0000 7f4410a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f440c145750 0x7f440c0c5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:06.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:06.163+0000 7f4410a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f440c0ce480 con 0x7f440c145750 2024-09-16T15:53:06.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:06.163+0000 7f4410a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f440c0ce300 con 0x7f440c06efa0 2024-09-16T15:53:06.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:06.163+0000 7f4410a00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f440c0ce600 con 0x7f440c0cccd0 2024-09-16T15:53:06.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:06.163+0000 7f440a800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f440c145750 0x7f440c0c5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:06.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:06.164+0000 7f440a800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f440c145750 0x7f440c0c5f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53692/0 (socket says 172.21.15.151:53692) 2024-09-16T15:53:06.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:06.164+0000 7f440a800640 1 -- 172.21.15.151:0/1204648054 learned_addr learned my addr 172.21.15.151:0/1204648054 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:53:06.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:06.164+0000 7f4409e00640 1 --2- 172.21.15.151:0/1204648054 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f440c06efa0 0x7f440c1f4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:06.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:06.164+0000 7f4409400640 1 --2- 172.21.15.151:0/1204648054 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f440c0cccd0 0x7f440c0b0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:06.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:06.164+0000 7f440a800640 1 -- 172.21.15.151:0/1204648054 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f440c0cccd0 msgr2=0x7f440c0b0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:06.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:06.164+0000 7f440a800640 1 --2- 172.21.15.151:0/1204648054 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f440c0cccd0 0x7f440c0b0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:06.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:06.164+0000 7f440a800640 1 -- 172.21.15.151:0/1204648054 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f440c06efa0 msgr2=0x7f440c1f4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:06.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:06.164+0000 7f440a800640 1 --2- 172.21.15.151:0/1204648054 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f440c06efa0 0x7f440c1f4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:06.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:06.164+0000 7f440a800640 1 -- 172.21.15.151:0/1204648054 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f440c13fdc0 con 0x7f440c145750 2024-09-16T15:53:06.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:06.164+0000 7f440a800640 1 --2- 172.21.15.151:0/1204648054 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f440c145750 0x7f440c0c5f50 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f440400eaa0 tx=0x7f440400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:06.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:06.164+0000 7f4402000640 1 -- 172.21.15.151:0/1204648054 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440400cdb0 con 0x7f440c145750 2024-09-16T15:53:06.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:06.164+0000 7f4402000640 1 -- 172.21.15.151:0/1204648054 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4404004c10 con 0x7f440c145750 2024-09-16T15:53:06.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:06.164+0000 7f4402000640 1 -- 172.21.15.151:0/1204648054 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4404005020 con 0x7f440c145750 2024-09-16T15:53:06.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:06.164+0000 7f4410a00640 1 -- 172.21.15.151:0/1204648054 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f440c1400a0 con 0x7f440c145750 2024-09-16T15:53:06.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:06.165+0000 7f4410a00640 1 -- 172.21.15.151:0/1204648054 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f440c140660 con 0x7f440c145750 2024-09-16T15:53:06.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:06.166+0000 7f4410a00640 1 -- 172.21.15.151:0/1204648054 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f440c0cdbb0 con 0x7f440c145750 2024-09-16T15:53:06.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:06.167+0000 7f4402000640 1 -- 172.21.15.151:0/1204648054 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 42) ==== 103786+0+0 (secure 0 0 0) 0x7f44040051c0 con 0x7f440c145750 2024-09-16T15:53:06.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:06.168+0000 7f4402000640 1 --2- 172.21.15.151:0/1204648054 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f43ec07a410 0x7f43ec07c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:06.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:06.168+0000 7f4402000640 1 -- 172.21.15.151:0/1204648054 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5851+0+0 (secure 0 0 0) 0x7f440409a0f0 con 0x7f440c145750 2024-09-16T15:53:06.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:06.168+0000 7f4409e00640 1 --2- 172.21.15.151:0/1204648054 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f43ec07a410 0x7f43ec07c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:06.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:06.169+0000 7f4409e00640 1 --2- 172.21.15.151:0/1204648054 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f43ec07a410 0x7f43ec07c8d0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f43fc033000 tx=0x7f43fc004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:06.173 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:06.172+0000 7f4402000640 1 -- 172.21.15.151:0/1204648054 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4404016030 con 0x7f440c145750 2024-09-16T15:53:06.281 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:06.280+0000 7f4402000640 1 -- 172.21.15.151:0/1204648054 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mgrmap(e 43) ==== 103837+0+0 (secure 0 0 0) 0x7f440405fe80 con 0x7f440c145750 2024-09-16T15:53:06.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:06.405+0000 7f4410a00640 1 -- 172.21.15.151:0/1204648054 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone15851", "target": ["mon-mgr", ""]}) -- 0x7f440c0af0f0 con 0x7f43ec07a410 2024-09-16T15:53:06.974 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:53:06.973+0000 7f489ea00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:53:06.975 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:53:06.973+0000 7f489ea00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:53:06.975 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:53:06.973+0000 7f489ea00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:53:06.975 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:53:06.973+0000 7f489ea00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:53:06.975 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:53:06.973+0000 7f489ea00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:53:07.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:07.045+0000 7f4402000640 1 -- 172.21.15.151:0/1204648054 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f440c0af0f0 con 0x7f43ec07a410 2024-09-16T15:53:07.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:07.048+0000 7f4410a00640 1 -- 172.21.15.151:0/1204648054 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f43ec07a410 msgr2=0x7f43ec07c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:07.050 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:07.048+0000 7f4410a00640 1 --2- 172.21.15.151:0/1204648054 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f43ec07a410 0x7f43ec07c8d0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f43fc033000 tx=0x7f43fc004730 comp rx=0 tx=0).stop 2024-09-16T15:53:07.050 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:07.048+0000 7f4410a00640 1 -- 172.21.15.151:0/1204648054 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f440c145750 msgr2=0x7f440c0c5f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:07.050 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:07.048+0000 7f4410a00640 1 --2- 172.21.15.151:0/1204648054 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f440c145750 0x7f440c0c5f50 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f440400eaa0 tx=0x7f440400ef60 comp rx=0 tx=0).stop 2024-09-16T15:53:07.050 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:07.048+0000 7f4410a00640 1 -- 172.21.15.151:0/1204648054 shutdown_connections 2024-09-16T15:53:07.050 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:07.048+0000 7f4410a00640 1 --2- 172.21.15.151:0/1204648054 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f43ec07a410 0x7f43ec07c8d0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:07.050 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:07.048+0000 7f4410a00640 1 --2- 172.21.15.151:0/1204648054 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f440c145750 0x7f440c0c5f50 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:07.050 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:07.048+0000 7f4410a00640 1 --2- 172.21.15.151:0/1204648054 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f440c0cccd0 0x7f440c0b0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:07.050 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:07.048+0000 7f4410a00640 1 --2- 172.21.15.151:0/1204648054 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f440c06efa0 0x7f440c1f4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:07.050 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:07.048+0000 7f4410a00640 1 -- 172.21.15.151:0/1204648054 >> 172.21.15.151:0/1204648054 conn(0x7f440c0b1840 msgr2=0x7f440c0b4670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:07.050 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:07.048+0000 7f4410a00640 1 -- 172.21.15.151:0/1204648054 shutdown_connections 2024-09-16T15:53:07.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:07.048+0000 7f4410a00640 1 -- 172.21.15.151:0/1204648054 wait complete. 2024-09-16T15:53:07.069 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone76352 2024-09-16T15:53:07.317 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:07.315+0000 7f0b6e600640 1 -- 172.21.15.151:0/1471199135 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0b68161590 msgr2=0x7f0b680b6e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:07.317 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:07.315+0000 7f0b6e600640 1 --2- 172.21.15.151:0/1471199135 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0b68161590 0x7f0b680b6e00 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f0b58005fd0 tx=0x7f0b5802a8b0 comp rx=0 tx=0).stop 2024-09-16T15:53:07.317 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:07.315+0000 7f0b6e600640 1 -- 172.21.15.151:0/1471199135 shutdown_connections 2024-09-16T15:53:07.317 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:07.315+0000 7f0b6e600640 1 --2- 172.21.15.151:0/1471199135 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0b680b5630 0x7f0b680cf5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:07.317 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:07.315+0000 7f0b6e600640 1 --2- 172.21.15.151:0/1471199135 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0b68161590 0x7f0b680b6e00 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:07.317 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:07.315+0000 7f0b6e600640 1 -- 172.21.15.151:0/1471199135 >> 172.21.15.151:0/1471199135 conn(0x7f0b680d3fb0 msgr2=0x7f0b680d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:07.318 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:07.315+0000 7f0b6e600640 1 -- 172.21.15.151:0/1471199135 shutdown_connections 2024-09-16T15:53:07.318 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:07.315+0000 7f0b6e600640 1 -- 172.21.15.151:0/1471199135 wait complete. 2024-09-16T15:53:07.318 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:07.317+0000 7f0b6e600640 1 Processor -- start 2024-09-16T15:53:07.318 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:07.317+0000 7f0b6e600640 1 -- start start 2024-09-16T15:53:07.320 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:07.318+0000 7f0b6e600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0b680b5630 0x7f0b6815a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:07.320 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:07.318+0000 7f0b6e600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0b680b7340 0x7f0b680c9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:07.320 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:07.318+0000 7f0b6e600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0b68161590 0x7f0b68157370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:07.320 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:07.318+0000 7f0b6e600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0b680cfe80 con 0x7f0b68161590 2024-09-16T15:53:07.320 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:07.318+0000 7f0b6e600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0b680cfd00 con 0x7f0b680b5630 2024-09-16T15:53:07.320 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:07.318+0000 7f0b6e600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f0b68162430 con 0x7f0b680b7340 2024-09-16T15:53:07.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:07.319+0000 7f0b67e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0b68161590 0x7f0b68157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:07.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:07.319+0000 7f0b67e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0b68161590 0x7f0b68157370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53696/0 (socket says 172.21.15.151:53696) 2024-09-16T15:53:07.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:07.319+0000 7f0b67e00640 1 -- 172.21.15.151:0/1843992595 learned_addr learned my addr 172.21.15.151:0/1843992595 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:53:07.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:07.319+0000 7f0b67400640 1 --2- 172.21.15.151:0/1843992595 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0b680b5630 0x7f0b6815a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:07.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:07.319+0000 7f0b5ea00640 1 --2- 172.21.15.151:0/1843992595 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0b680b7340 0x7f0b680c9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:07.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:07.319+0000 7f0b67e00640 1 -- 172.21.15.151:0/1843992595 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0b680b7340 msgr2=0x7f0b680c9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:07.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:07.319+0000 7f0b67e00640 1 --2- 172.21.15.151:0/1843992595 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0b680b7340 0x7f0b680c9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:07.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:07.319+0000 7f0b67e00640 1 -- 172.21.15.151:0/1843992595 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0b680b5630 msgr2=0x7f0b6815a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:07.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:07.319+0000 7f0b67e00640 1 --2- 172.21.15.151:0/1843992595 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0b680b5630 0x7f0b6815a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:07.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:07.319+0000 7f0b67e00640 1 -- 172.21.15.151:0/1843992595 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b6815a6e0 con 0x7f0b68161590 2024-09-16T15:53:07.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:07.320+0000 7f0b67e00640 1 --2- 172.21.15.151:0/1843992595 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0b68161590 0x7f0b68157370 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f0b60007fd0 tx=0x7f0b6000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:07.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:07.320+0000 7f0b5fe00640 1 -- 172.21.15.151:0/1843992595 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b60011790 con 0x7f0b68161590 2024-09-16T15:53:07.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:07.320+0000 7f0b5fe00640 1 -- 172.21.15.151:0/1843992595 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0b60017070 con 0x7f0b68161590 2024-09-16T15:53:07.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:07.320+0000 7f0b5fe00640 1 -- 172.21.15.151:0/1843992595 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b60011ce0 con 0x7f0b68161590 2024-09-16T15:53:07.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:07.320+0000 7f0b6e600640 1 -- 172.21.15.151:0/1843992595 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b681596e0 con 0x7f0b68161590 2024-09-16T15:53:07.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:07.321+0000 7f0b6e600640 1 -- 172.21.15.151:0/1843992595 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b681599a0 con 0x7f0b68161590 2024-09-16T15:53:07.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:07.322+0000 7f0b6e600640 1 -- 172.21.15.151:0/1843992595 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b6806f300 con 0x7f0b68161590 2024-09-16T15:53:07.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:07.324+0000 7f0b5fe00640 1 -- 172.21.15.151:0/1843992595 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 43) ==== 103837+0+0 (secure 0 0 0) 0x7f0b6001c460 con 0x7f0b68161590 2024-09-16T15:53:07.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:07.325+0000 7f0b5fe00640 1 --2- 172.21.15.151:0/1843992595 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0b4407a470 0x7f0b4407c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:07.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:07.325+0000 7f0b5fe00640 1 -- 172.21.15.151:0/1843992595 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5851+0+0 (secure 0 0 0) 0x7f0b60099690 con 0x7f0b68161590 2024-09-16T15:53:07.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:07.329+0000 7f0b67400640 1 --2- 172.21.15.151:0/1843992595 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0b4407a470 0x7f0b4407c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:07.333 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:07.331+0000 7f0b5fe00640 1 -- 172.21.15.151:0/1843992595 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0b6806f300 con 0x7f0b68161590 2024-09-16T15:53:07.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:07.337+0000 7f0b67400640 1 --2- 172.21.15.151:0/1843992595 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0b4407a470 0x7f0b4407c930 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f0b58034000 tx=0x7f0b58004ab0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:07.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:07.754+0000 7f0b6e600640 1 -- 172.21.15.151:0/1843992595 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone76352", "target": ["mon-mgr", ""]}) -- 0x7f0b680b9f30 con 0x7f0b4407a470 2024-09-16T15:53:08.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:08.376+0000 7f0b5fe00640 1 -- 172.21.15.151:0/1843992595 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mgrmap(e 44) ==== 103939+0+0 (secure 0 0 0) 0x7f0b6005f350 con 0x7f0b68161590 2024-09-16T15:53:08.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:08.561+0000 7f0b5fe00640 1 -- 172.21.15.151:0/1843992595 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0b680b9f30 con 0x7f0b4407a470 2024-09-16T15:53:08.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:08.564+0000 7f0b6e600640 1 -- 172.21.15.151:0/1843992595 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0b4407a470 msgr2=0x7f0b4407c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:08.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:08.564+0000 7f0b6e600640 1 --2- 172.21.15.151:0/1843992595 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0b4407a470 0x7f0b4407c930 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f0b58034000 tx=0x7f0b58004ab0 comp rx=0 tx=0).stop 2024-09-16T15:53:08.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:08.564+0000 7f0b6e600640 1 -- 172.21.15.151:0/1843992595 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0b68161590 msgr2=0x7f0b68157370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:08.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:08.564+0000 7f0b6e600640 1 --2- 172.21.15.151:0/1843992595 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0b68161590 0x7f0b68157370 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f0b60007fd0 tx=0x7f0b6000c890 comp rx=0 tx=0).stop 2024-09-16T15:53:08.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:08.564+0000 7f0b6e600640 1 -- 172.21.15.151:0/1843992595 shutdown_connections 2024-09-16T15:53:08.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:08.564+0000 7f0b6e600640 1 --2- 172.21.15.151:0/1843992595 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0b4407a470 0x7f0b4407c930 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:08.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:08.564+0000 7f0b6e600640 1 --2- 172.21.15.151:0/1843992595 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0b68161590 0x7f0b68157370 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:08.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:08.564+0000 7f0b6e600640 1 --2- 172.21.15.151:0/1843992595 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0b680b7340 0x7f0b680c9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:08.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:08.564+0000 7f0b6e600640 1 --2- 172.21.15.151:0/1843992595 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0b680b5630 0x7f0b6815a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:08.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:08.564+0000 7f0b6e600640 1 -- 172.21.15.151:0/1843992595 >> 172.21.15.151:0/1843992595 conn(0x7f0b680d3fb0 msgr2=0x7f0b6806d490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:08.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:08.565+0000 7f0b6e600640 1 -- 172.21.15.151:0/1843992595 shutdown_connections 2024-09-16T15:53:08.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:08.565+0000 7f0b6e600640 1 -- 172.21.15.151:0/1843992595 wait complete. 2024-09-16T15:53:08.673 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:53:08.668+0000 7f48f6200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:53:08.674 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:53:08.668+0000 7f48f6200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:53:08.674 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:53:08.668+0000 7f48f6200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:53:08.674 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:53:08.668+0000 7f48f6200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:53:08.674 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:53:08.668+0000 7f48f6200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:53:09.607 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone52540 2024-09-16T15:53:09.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:09.906+0000 7f108be00640 1 -- 172.21.15.151:0/3906188275 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f108c161460 msgr2=0x7f108c0d22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:09.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:09.906+0000 7f108be00640 1 --2- 172.21.15.151:0/3906188275 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f108c161460 0x7f108c0d22f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f107c005fd0 tx=0x7f107c02a4d0 comp rx=0 tx=0).stop 2024-09-16T15:53:09.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:09.907+0000 7f108be00640 1 -- 172.21.15.151:0/3906188275 shutdown_connections 2024-09-16T15:53:09.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:09.907+0000 7f108be00640 1 --2- 172.21.15.151:0/3906188275 >> v2:172.21.15.167:3300/0 conn(0x7f108c0b5480 0x7f108c0d2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:09.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:09.907+0000 7f108be00640 1 --2- 172.21.15.151:0/3906188275 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f108c161460 0x7f108c0d22f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:09.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:09.907+0000 7f108be00640 1 -- 172.21.15.151:0/3906188275 >> 172.21.15.151:0/3906188275 conn(0x7f108c09d0b0 msgr2=0x7f108c0d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:09.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:09.907+0000 7f108be00640 1 -- 172.21.15.151:0/3906188275 shutdown_connections 2024-09-16T15:53:09.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:09.908+0000 7f108be00640 1 -- 172.21.15.151:0/3906188275 wait complete. 2024-09-16T15:53:09.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:09.912+0000 7f108be00640 1 Processor -- start 2024-09-16T15:53:09.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:09.912+0000 7f108be00640 1 -- start start 2024-09-16T15:53:09.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:09.915+0000 7f108be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f108c0b4f40 0x7f108c15df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:09.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:09.915+0000 7f108be00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f108c0b5480 0x7f108c15e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:09.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:09.915+0000 7f108be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f108c161460 0x7f108c1f8f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:09.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:09.915+0000 7f108be00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f108c0cf0b0 con 0x7f108c0b4f40 2024-09-16T15:53:09.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:09.915+0000 7f108be00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f108c0cef30 con 0x7f108c161460 2024-09-16T15:53:09.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:09.915+0000 7f108be00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f108c0cf230 con 0x7f108c0b5480 2024-09-16T15:53:09.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:09.915+0000 7f108aa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f108c0b4f40 0x7f108c15df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:09.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:09.915+0000 7f108aa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f108c0b4f40 0x7f108c15df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53706/0 (socket says 172.21.15.151:53706) 2024-09-16T15:53:09.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:09.915+0000 7f108aa00640 1 -- 172.21.15.151:0/1758832257 learned_addr learned my addr 172.21.15.151:0/1758832257 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:53:09.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:09.916+0000 7f108aa00640 1 -- 172.21.15.151:0/1758832257 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f108c0b5480 msgr2=0x7f108c15e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:09.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:09.916+0000 7f108aa00640 1 --2- 172.21.15.151:0/1758832257 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f108c0b5480 0x7f108c15e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:09.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:09.916+0000 7f108aa00640 1 -- 172.21.15.151:0/1758832257 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f108c161460 msgr2=0x7f108c1f8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:09.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:09.916+0000 7f108aa00640 1 --2- 172.21.15.151:0/1758832257 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f108c161460 0x7f108c1f8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:09.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:09.916+0000 7f108aa00640 1 -- 172.21.15.151:0/1758832257 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f108c1f9bc0 con 0x7f108c0b4f40 2024-09-16T15:53:09.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:09.917+0000 7f108aa00640 1 --2- 172.21.15.151:0/1758832257 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f108c0b4f40 0x7f108c15df70 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f107c034040 tx=0x7f107c0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:09.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:09.917+0000 7f1083400640 1 -- 172.21.15.151:0/1758832257 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f107c03e030 con 0x7f108c0b4f40 2024-09-16T15:53:09.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:09.917+0000 7f1083400640 1 -- 172.21.15.151:0/1758832257 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f107c0047a0 con 0x7f108c0b4f40 2024-09-16T15:53:09.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:09.917+0000 7f1083400640 1 -- 172.21.15.151:0/1758832257 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f107c035040 con 0x7f108c0b4f40 2024-09-16T15:53:09.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:09.917+0000 7f108be00640 1 -- 172.21.15.151:0/1758832257 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f108c0b0c20 con 0x7f108c0b4f40 2024-09-16T15:53:09.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:09.917+0000 7f108be00640 1 -- 172.21.15.151:0/1758832257 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f108c0b11e0 con 0x7f108c0b4f40 2024-09-16T15:53:09.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:09.920+0000 7f108be00640 1 -- 172.21.15.151:0/1758832257 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1064005290 con 0x7f108c0b4f40 2024-09-16T15:53:09.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:09.922+0000 7f1083400640 1 -- 172.21.15.151:0/1758832257 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 44) ==== 103939+0+0 (secure 0 0 0) 0x7f107c041030 con 0x7f108c0b4f40 2024-09-16T15:53:09.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:09.925+0000 7f1083400640 1 --2- 172.21.15.151:0/1758832257 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f106c07a660 0x7f106c07cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:09.928 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:09.926+0000 7f1083400640 1 -- 172.21.15.151:0/1758832257 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5851+0+0 (secure 0 0 0) 0x7f107c0b9ef0 con 0x7f108c0b4f40 2024-09-16T15:53:09.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:09.932+0000 7f1083400640 1 -- 172.21.15.151:0/1758832257 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f107c084040 con 0x7f108c0b4f40 2024-09-16T15:53:09.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:09.936+0000 7f108a000640 1 --2- 172.21.15.151:0/1758832257 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f106c07a660 0x7f106c07cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:09.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:09.950+0000 7f108a000640 1 --2- 172.21.15.151:0/1758832257 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f106c07a660 0x7f106c07cb20 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f108c15df00 tx=0x7f1078006c90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:10.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:10.308+0000 7f108be00640 1 -- 172.21.15.151:0/1758832257 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone52540", "target": ["mon-mgr", ""]}) -- 0x7f1064000bc0 con 0x7f106c07a660 2024-09-16T15:53:10.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:10.431+0000 7f1083400640 1 -- 172.21.15.151:0/1758832257 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mgrmap(e 45) ==== 103990+0+0 (secure 0 0 0) 0x7f107c07fbb0 con 0x7f108c0b4f40 2024-09-16T15:53:10.701 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:53:10.699+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:53:10.701 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:53:10.699+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:53:10.701 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:53:10.699+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:53:10.701 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:53:10.699+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:53:10.702 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:53:10.699+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:53:11.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:11.555+0000 7f1083400640 1 -- 172.21.15.151:0/1758832257 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1064000bc0 con 0x7f106c07a660 2024-09-16T15:53:11.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:11.563+0000 7f108be00640 1 -- 172.21.15.151:0/1758832257 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f106c07a660 msgr2=0x7f106c07cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:11.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:11.563+0000 7f108be00640 1 --2- 172.21.15.151:0/1758832257 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f106c07a660 0x7f106c07cb20 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f108c15df00 tx=0x7f1078006c90 comp rx=0 tx=0).stop 2024-09-16T15:53:11.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:11.563+0000 7f108be00640 1 -- 172.21.15.151:0/1758832257 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f108c0b4f40 msgr2=0x7f108c15df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:11.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:11.563+0000 7f108be00640 1 --2- 172.21.15.151:0/1758832257 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f108c0b4f40 0x7f108c15df70 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f107c034040 tx=0x7f107c0039e0 comp rx=0 tx=0).stop 2024-09-16T15:53:11.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:11.563+0000 7f108be00640 1 -- 172.21.15.151:0/1758832257 shutdown_connections 2024-09-16T15:53:11.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:11.563+0000 7f108be00640 1 --2- 172.21.15.151:0/1758832257 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f106c07a660 0x7f106c07cb20 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:11.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:11.563+0000 7f108be00640 1 --2- 172.21.15.151:0/1758832257 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f108c161460 0x7f108c1f8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:11.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:11.563+0000 7f108be00640 1 --2- 172.21.15.151:0/1758832257 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f108c0b5480 0x7f108c15e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:11.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:11.563+0000 7f108be00640 1 --2- 172.21.15.151:0/1758832257 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f108c0b4f40 0x7f108c15df70 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:11.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:11.563+0000 7f108be00640 1 -- 172.21.15.151:0/1758832257 >> 172.21.15.151:0/1758832257 conn(0x7f108c09d0b0 msgr2=0x7f108c0cd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:11.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:11.564+0000 7f108be00640 1 -- 172.21.15.151:0/1758832257 shutdown_connections 2024-09-16T15:53:11.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:11.564+0000 7f108be00640 1 -- 172.21.15.151:0/1758832257 wait complete. 2024-09-16T15:53:11.590 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone15851 2024-09-16T15:53:11.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:11.845+0000 7f6664e00640 1 -- 172.21.15.151:0/1957637727 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6660161760 msgr2=0x7f66600ce4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:11.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:11.845+0000 7f6664e00640 1 --2- 172.21.15.151:0/1957637727 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6660161760 0x7f66600ce4a0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f6650005fd0 tx=0x7f665002a8b0 comp rx=0 tx=0).stop 2024-09-16T15:53:11.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:11.846+0000 7f6664e00640 1 -- 172.21.15.151:0/1957637727 shutdown_connections 2024-09-16T15:53:11.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:11.846+0000 7f6664e00640 1 --2- 172.21.15.151:0/1957637727 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6660161760 0x7f66600ce4a0 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:11.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:11.846+0000 7f6664e00640 1 -- 172.21.15.151:0/1957637727 >> 172.21.15.151:0/1957637727 conn(0x7f66600b1840 msgr2=0x7f66600b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:11.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:11.846+0000 7f6664e00640 1 -- 172.21.15.151:0/1957637727 shutdown_connections 2024-09-16T15:53:11.848 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:11.846+0000 7f6664e00640 1 -- 172.21.15.151:0/1957637727 wait complete. 2024-09-16T15:53:11.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:11.849+0000 7f6664e00640 1 Processor -- start 2024-09-16T15:53:11.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:11.849+0000 7f6664e00640 1 -- start start 2024-09-16T15:53:11.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:11.851+0000 7f6664e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f66600af190 0x7f6660146130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:11.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:11.851+0000 7f6664e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66600cec60 0x7f66601602f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:11.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:11.851+0000 7f6664e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6660161760 0x7f6660160830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:11.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:11.851+0000 7f6664e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6660162520 con 0x7f66600af190 2024-09-16T15:53:11.852 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:11.851+0000 7f6664e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f66601623a0 con 0x7f6660161760 2024-09-16T15:53:11.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:11.851+0000 7f6664e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f66601626a0 con 0x7f66600cec60 2024-09-16T15:53:11.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:11.852+0000 7f665de00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f66600af190 0x7f6660146130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:11.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:11.852+0000 7f665de00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f66600af190 0x7f6660146130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:53730/0 (socket says 172.21.15.151:53730) 2024-09-16T15:53:11.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:11.852+0000 7f665de00640 1 -- 172.21.15.151:0/4165639087 learned_addr learned my addr 172.21.15.151:0/4165639087 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:53:11.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:11.852+0000 7f665e800640 1 --2- 172.21.15.151:0/4165639087 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6660161760 0x7f6660160830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:11.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:11.853+0000 7f6655400640 1 --2- 172.21.15.151:0/4165639087 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66600cec60 0x7f66601602f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:11.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:11.853+0000 7f665de00640 1 -- 172.21.15.151:0/4165639087 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66600cec60 msgr2=0x7f66601602f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:11.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:11.853+0000 7f665de00640 1 --2- 172.21.15.151:0/4165639087 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66600cec60 0x7f66601602f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:11.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:11.853+0000 7f665de00640 1 -- 172.21.15.151:0/4165639087 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6660161760 msgr2=0x7f6660160830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:11.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:11.853+0000 7f665de00640 1 --2- 172.21.15.151:0/4165639087 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6660161760 0x7f6660160830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:11.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:11.853+0000 7f665de00640 1 -- 172.21.15.151:0/4165639087 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66601f4d80 con 0x7f66600af190 2024-09-16T15:53:11.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:11.853+0000 7f665de00640 1 --2- 172.21.15.151:0/4165639087 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f66600af190 0x7f6660146130 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f6650006010 tx=0x7f6650003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:11.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:11.854+0000 7f6656a00640 1 -- 172.21.15.151:0/4165639087 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6650031da0 con 0x7f66600af190 2024-09-16T15:53:11.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:11.854+0000 7f6664e00640 1 -- 172.21.15.151:0/4165639087 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66601f5060 con 0x7f66600af190 2024-09-16T15:53:11.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:11.854+0000 7f6664e00640 1 -- 172.21.15.151:0/4165639087 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66601f5620 con 0x7f66600af190 2024-09-16T15:53:11.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:11.854+0000 7f6656a00640 1 -- 172.21.15.151:0/4165639087 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6650003e10 con 0x7f66600af190 2024-09-16T15:53:11.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:11.854+0000 7f6656a00640 1 -- 172.21.15.151:0/4165639087 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66500332f0 con 0x7f66600af190 2024-09-16T15:53:11.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:11.855+0000 7f6664e00640 1 -- 172.21.15.151:0/4165639087 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6638005290 con 0x7f66600af190 2024-09-16T15:53:11.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:11.857+0000 7f6656a00640 1 -- 172.21.15.151:0/4165639087 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 45) ==== 103990+0+0 (secure 0 0 0) 0x7f6650033490 con 0x7f66600af190 2024-09-16T15:53:11.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:11.859+0000 7f6656a00640 1 --2- 172.21.15.151:0/4165639087 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f663c07a650 0x7f663c07cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:11.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:11.859+0000 7f6656a00640 1 -- 172.21.15.151:0/4165639087 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5851+0+0 (secure 0 0 0) 0x7f66500bad50 con 0x7f66600af190 2024-09-16T15:53:11.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:11.863+0000 7f6656a00640 1 -- 172.21.15.151:0/4165639087 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f66500c5020 con 0x7f66600af190 2024-09-16T15:53:11.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:11.869+0000 7f6655400640 1 --2- 172.21.15.151:0/4165639087 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f663c07a650 0x7f663c07cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:11.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:11.892+0000 7f6655400640 1 --2- 172.21.15.151:0/4165639087 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f663c07a650 0x7f663c07cb10 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f6648007fd0 tx=0x7f6648009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:12.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:12.208+0000 7f6664e00640 1 -- 172.21.15.151:0/4165639087 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone15851", "target": ["mon-mgr", ""]}) -- 0x7f6638000bc0 con 0x7f663c07a650 2024-09-16T15:53:12.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:12.468+0000 7f6656a00640 1 -- 172.21.15.151:0/4165639087 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mgrmap(e 46) ==== 104041+0+0 (secure 0 0 0) 0x7f6650080990 con 0x7f66600af190 2024-09-16T15:53:12.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:12.837+0000 7f6656a00640 1 -- 172.21.15.151:0/4165639087 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6638000bc0 con 0x7f663c07a650 2024-09-16T15:53:12.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:12.840+0000 7f6664e00640 1 -- 172.21.15.151:0/4165639087 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f663c07a650 msgr2=0x7f663c07cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:12.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:12.840+0000 7f6664e00640 1 --2- 172.21.15.151:0/4165639087 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f663c07a650 0x7f663c07cb10 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f6648007fd0 tx=0x7f6648009040 comp rx=0 tx=0).stop 2024-09-16T15:53:12.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:12.840+0000 7f6664e00640 1 -- 172.21.15.151:0/4165639087 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f66600af190 msgr2=0x7f6660146130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:12.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:12.840+0000 7f6664e00640 1 --2- 172.21.15.151:0/4165639087 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f66600af190 0x7f6660146130 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f6650006010 tx=0x7f6650003c20 comp rx=0 tx=0).stop 2024-09-16T15:53:12.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:12.840+0000 7f6664e00640 1 -- 172.21.15.151:0/4165639087 shutdown_connections 2024-09-16T15:53:12.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:12.840+0000 7f6664e00640 1 --2- 172.21.15.151:0/4165639087 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f663c07a650 0x7f663c07cb10 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:12.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:12.840+0000 7f6664e00640 1 --2- 172.21.15.151:0/4165639087 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6660161760 0x7f6660160830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:12.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:12.840+0000 7f6664e00640 1 --2- 172.21.15.151:0/4165639087 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f66600cec60 0x7f66601602f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:12.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:12.840+0000 7f6664e00640 1 --2- 172.21.15.151:0/4165639087 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f66600af190 0x7f6660146130 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:12.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:12.840+0000 7f6664e00640 1 -- 172.21.15.151:0/4165639087 >> 172.21.15.151:0/4165639087 conn(0x7f66600b1840 msgr2=0x7f66600b33d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:12.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:12.840+0000 7f6664e00640 1 -- 172.21.15.151:0/4165639087 shutdown_connections 2024-09-16T15:53:12.843 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:12.841+0000 7f6664e00640 1 -- 172.21.15.151:0/4165639087 wait complete. 2024-09-16T15:53:12.870 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone76352 2024-09-16T15:53:13.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.157+0000 7f8f0ac00640 1 -- 172.21.15.151:0/1225955907 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f0415e980 msgr2=0x7f8f0415ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:13.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.157+0000 7f8f0ac00640 1 --2- 172.21.15.151:0/1225955907 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f0415e980 0x7f8f0415ee10 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f8f0415b4d0 tx=0x7f8efc02aa00 comp rx=0 tx=0).stop 2024-09-16T15:53:13.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.158+0000 7f8f0ac00640 1 -- 172.21.15.151:0/1225955907 shutdown_connections 2024-09-16T15:53:13.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.158+0000 7f8f0ac00640 1 --2- 172.21.15.151:0/1225955907 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f0415e980 0x7f8f0415ee10 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:13.160 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.158+0000 7f8f0ac00640 1 --2- 172.21.15.151:0/1225955907 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8f0415c080 0x7f8f040cf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:13.160 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.158+0000 7f8f0ac00640 1 --2- 172.21.15.151:0/1225955907 >> v2:172.21.15.167:3300/0 conn(0x7f8f040b3370 0x7f8f040cebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:13.160 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.158+0000 7f8f0ac00640 1 -- 172.21.15.151:0/1225955907 >> 172.21.15.151:0/1225955907 conn(0x7f8f040b1840 msgr2=0x7f8f040b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:13.160 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.158+0000 7f8f0ac00640 1 -- 172.21.15.151:0/1225955907 shutdown_connections 2024-09-16T15:53:13.160 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.158+0000 7f8f0ac00640 1 -- 172.21.15.151:0/1225955907 wait complete. 2024-09-16T15:53:13.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.162+0000 7f8f0ac00640 1 Processor -- start 2024-09-16T15:53:13.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.163+0000 7f8f0ac00640 1 -- start start 2024-09-16T15:53:13.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.165+0000 7f8f0ac00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f040b3370 0x7f8f0415c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:13.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.165+0000 7f8f0ac00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8f0415c080 0x7f8f041fc260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:13.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.165+0000 7f8f0ac00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f0415e980 0x7f8f0413a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:13.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.165+0000 7f8f0ac00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8f04140560 con 0x7f8f040b3370 2024-09-16T15:53:13.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.165+0000 7f8f0ac00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8f041403e0 con 0x7f8f0415e980 2024-09-16T15:53:13.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.165+0000 7f8f0ac00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f8f041406e0 con 0x7f8f0415c080 2024-09-16T15:53:13.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.166+0000 7f8f03e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f040b3370 0x7f8f0415c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:13.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.166+0000 7f8f03e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f040b3370 0x7f8f0415c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33572/0 (socket says 172.21.15.151:33572) 2024-09-16T15:53:13.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.166+0000 7f8f03e00640 1 -- 172.21.15.151:0/1228376237 learned_addr learned my addr 172.21.15.151:0/1228376237 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:53:13.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.167+0000 7f8f08c00640 1 --2- 172.21.15.151:0/1228376237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f0415e980 0x7f8f0413a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:13.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.167+0000 7f8f03400640 1 --2- 172.21.15.151:0/1228376237 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8f0415c080 0x7f8f041fc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:13.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.167+0000 7f8f08c00640 1 -- 172.21.15.151:0/1228376237 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8f0415c080 msgr2=0x7f8f041fc260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:13.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.167+0000 7f8f08c00640 1 --2- 172.21.15.151:0/1228376237 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8f0415c080 0x7f8f041fc260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:13.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.167+0000 7f8f08c00640 1 -- 172.21.15.151:0/1228376237 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f040b3370 msgr2=0x7f8f0415c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:13.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.168+0000 7f8f08c00640 1 --2- 172.21.15.151:0/1228376237 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f040b3370 0x7f8f0415c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:13.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.168+0000 7f8f08c00640 1 -- 172.21.15.151:0/1228376237 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f0413aed0 con 0x7f8f0415e980 2024-09-16T15:53:13.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.168+0000 7f8f03e00640 1 --2- 172.21.15.151:0/1228376237 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f040b3370 0x7f8f0415c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:53:13.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.169+0000 7f8f08c00640 1 --2- 172.21.15.151:0/1228376237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f0415e980 0x7f8f0413a0c0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f8efc032000 tx=0x7f8efc003f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:13.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.170+0000 7f8efbe00640 1 -- 172.21.15.151:0/1228376237 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8efc033050 con 0x7f8f0415e980 2024-09-16T15:53:13.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.170+0000 7f8efbe00640 1 -- 172.21.15.151:0/1228376237 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8efc0044d0 con 0x7f8f0415e980 2024-09-16T15:53:13.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.170+0000 7f8efbe00640 1 -- 172.21.15.151:0/1228376237 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8efc038450 con 0x7f8f0415e980 2024-09-16T15:53:13.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.170+0000 7f8f0ac00640 1 -- 172.21.15.151:0/1228376237 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f041fcce0 con 0x7f8f0415e980 2024-09-16T15:53:13.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.171+0000 7f8f0ac00640 1 -- 172.21.15.151:0/1228376237 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f0413b450 con 0x7f8f0415e980 2024-09-16T15:53:13.174 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.173+0000 7f8f0ac00640 1 -- 172.21.15.151:0/1228376237 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f0415ee10 con 0x7f8f0415e980 2024-09-16T15:53:13.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.175+0000 7f8efbe00640 1 -- 172.21.15.151:0/1228376237 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 46) ==== 104041+0+0 (secure 0 0 0) 0x7f8efc036070 con 0x7f8f0415e980 2024-09-16T15:53:13.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.176+0000 7f8efbe00640 1 --2- 172.21.15.151:0/1228376237 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8ee407a6b0 0x7f8ee407cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:13.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.176+0000 7f8efbe00640 1 -- 172.21.15.151:0/1228376237 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5851+0+0 (secure 0 0 0) 0x7f8efc0bac10 con 0x7f8f0415e980 2024-09-16T15:53:13.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.177+0000 7f8f03e00640 1 --2- 172.21.15.151:0/1228376237 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8ee407a6b0 0x7f8ee407cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:13.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.178+0000 7f8f03e00640 1 --2- 172.21.15.151:0/1228376237 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8ee407a6b0 0x7f8ee407cb70 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4009e10 tx=0x7f8ef40091c0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:13.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.182+0000 7f8efbe00640 1 -- 172.21.15.151:0/1228376237 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8efc084de0 con 0x7f8f0415e980 2024-09-16T15:53:13.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.507+0000 7f8f0ac00640 1 -- 172.21.15.151:0/1228376237 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone76352", "target": ["mon-mgr", ""]}) -- 0x7f8f041fd0e0 con 0x7f8ee407a6b0 2024-09-16T15:53:13.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.676+0000 7f8efbe00640 1 -- 172.21.15.151:0/1228376237 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8ef4009a60 con 0x7f8ee407a6b0 2024-09-16T15:53:13.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.679+0000 7f8f0ac00640 1 -- 172.21.15.151:0/1228376237 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8ee407a6b0 msgr2=0x7f8ee407cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:13.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.679+0000 7f8f0ac00640 1 --2- 172.21.15.151:0/1228376237 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8ee407a6b0 0x7f8ee407cb70 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4009e10 tx=0x7f8ef40091c0 comp rx=0 tx=0).stop 2024-09-16T15:53:13.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.679+0000 7f8f0ac00640 1 -- 172.21.15.151:0/1228376237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f0415e980 msgr2=0x7f8f0413a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:13.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.679+0000 7f8f0ac00640 1 --2- 172.21.15.151:0/1228376237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f0415e980 0x7f8f0413a0c0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f8efc032000 tx=0x7f8efc003f10 comp rx=0 tx=0).stop 2024-09-16T15:53:13.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.679+0000 7f8f0ac00640 1 -- 172.21.15.151:0/1228376237 shutdown_connections 2024-09-16T15:53:13.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.679+0000 7f8f0ac00640 1 --2- 172.21.15.151:0/1228376237 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8ee407a6b0 0x7f8ee407cb70 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:13.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.679+0000 7f8f0ac00640 1 --2- 172.21.15.151:0/1228376237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f0415e980 0x7f8f0413a0c0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:13.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.679+0000 7f8f0ac00640 1 --2- 172.21.15.151:0/1228376237 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8f0415c080 0x7f8f041fc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:13.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.679+0000 7f8f0ac00640 1 --2- 172.21.15.151:0/1228376237 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f040b3370 0x7f8f0415c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:13.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.679+0000 7f8f0ac00640 1 -- 172.21.15.151:0/1228376237 >> 172.21.15.151:0/1228376237 conn(0x7f8f040b1840 msgr2=0x7f8f040af7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:13.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.679+0000 7f8f0ac00640 1 -- 172.21.15.151:0/1228376237 shutdown_connections 2024-09-16T15:53:13.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.679+0000 7f8f0ac00640 1 -- 172.21.15.151:0/1228376237 wait complete. 2024-09-16T15:53:13.696 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone52540 2024-09-16T15:53:13.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.858+0000 7f28d1000640 1 -- 172.21.15.151:0/3772272238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28cc06efa0 msgr2=0x7f28cc073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:13.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.858+0000 7f28d1000640 1 --2- 172.21.15.151:0/3772272238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28cc06efa0 0x7f28cc073440 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f28c4005fd0 tx=0x7f28c402a840 comp rx=0 tx=0).stop 2024-09-16T15:53:13.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.858+0000 7f28d1000640 1 -- 172.21.15.151:0/3772272238 shutdown_connections 2024-09-16T15:53:13.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.858+0000 7f28d1000640 1 --2- 172.21.15.151:0/3772272238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28cc06efa0 0x7f28cc073440 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:13.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.858+0000 7f28d1000640 1 -- 172.21.15.151:0/3772272238 >> 172.21.15.151:0/3772272238 conn(0x7f28cc0b1840 msgr2=0x7f28cc0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:13.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.858+0000 7f28d1000640 1 -- 172.21.15.151:0/3772272238 shutdown_connections 2024-09-16T15:53:13.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.859+0000 7f28d1000640 1 -- 172.21.15.151:0/3772272238 wait complete. 2024-09-16T15:53:13.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.860+0000 7f28d1000640 1 Processor -- start 2024-09-16T15:53:13.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.860+0000 7f28d1000640 1 -- start start 2024-09-16T15:53:13.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.861+0000 7f28d1000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28cc06efa0 0x7f28cc143500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:13.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.861+0000 7f28d1000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28cc0cccd0 0x7f28cc0c9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:13.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.861+0000 7f28d1000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f28cc145750 0x7f28cc1466a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:13.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.861+0000 7f28d1000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f28cc0ce6a0 con 0x7f28cc06efa0 2024-09-16T15:53:13.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.861+0000 7f28d1000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f28cc0ce520 con 0x7f28cc0cccd0 2024-09-16T15:53:13.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.861+0000 7f28d1000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f28cc0ce820 con 0x7f28cc145750 2024-09-16T15:53:13.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.862+0000 7f28c9e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28cc06efa0 0x7f28cc143500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:13.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.862+0000 7f28c9e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28cc06efa0 0x7f28cc143500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33582/0 (socket says 172.21.15.151:33582) 2024-09-16T15:53:13.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.862+0000 7f28c9e00640 1 -- 172.21.15.151:0/623110464 learned_addr learned my addr 172.21.15.151:0/623110464 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:53:13.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.862+0000 7f28ca800640 1 --2- 172.21.15.151:0/623110464 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f28cc145750 0x7f28cc1466a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:13.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.862+0000 7f28c9400640 1 --2- 172.21.15.151:0/623110464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28cc0cccd0 0x7f28cc0c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:13.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.862+0000 7f28c9e00640 1 -- 172.21.15.151:0/623110464 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f28cc145750 msgr2=0x7f28cc1466a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:13.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.862+0000 7f28c9e00640 1 --2- 172.21.15.151:0/623110464 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f28cc145750 0x7f28cc1466a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:13.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.862+0000 7f28c9e00640 1 -- 172.21.15.151:0/623110464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28cc0cccd0 msgr2=0x7f28cc0c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:13.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.862+0000 7f28c9e00640 1 --2- 172.21.15.151:0/623110464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28cc0cccd0 0x7f28cc0c9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:13.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.862+0000 7f28c9e00640 1 -- 172.21.15.151:0/623110464 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28cc146200 con 0x7f28cc06efa0 2024-09-16T15:53:13.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.862+0000 7f28c9e00640 1 --2- 172.21.15.151:0/623110464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28cc06efa0 0x7f28cc143500 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f28bc00c660 tx=0x7f28bc00cb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:13.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.863+0000 7f28c2000640 1 -- 172.21.15.151:0/623110464 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28bc016020 con 0x7f28cc06efa0 2024-09-16T15:53:13.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.863+0000 7f28c2000640 1 -- 172.21.15.151:0/623110464 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f28bc00a930 con 0x7f28cc06efa0 2024-09-16T15:53:13.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.863+0000 7f28c2000640 1 -- 172.21.15.151:0/623110464 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28bc00ce60 con 0x7f28cc06efa0 2024-09-16T15:53:13.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.863+0000 7f28d1000640 1 -- 172.21.15.151:0/623110464 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28cc0c8c60 con 0x7f28cc06efa0 2024-09-16T15:53:13.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.863+0000 7f28d1000640 1 -- 172.21.15.151:0/623110464 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28cc0c9220 con 0x7f28cc06efa0 2024-09-16T15:53:13.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.864+0000 7f28d1000640 1 -- 172.21.15.151:0/623110464 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28cc0cecd0 con 0x7f28cc06efa0 2024-09-16T15:53:13.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.865+0000 7f28c2000640 1 -- 172.21.15.151:0/623110464 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 46) ==== 104041+0+0 (secure 0 0 0) 0x7f28bc019020 con 0x7f28cc06efa0 2024-09-16T15:53:13.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.866+0000 7f28c2000640 1 --2- 172.21.15.151:0/623110464 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f28ac07a6b0 0x7f28ac07cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:13.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.866+0000 7f28c2000640 1 -- 172.21.15.151:0/623110464 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5851+0+0 (secure 0 0 0) 0x7f28bc099830 con 0x7f28cc06efa0 2024-09-16T15:53:13.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.867+0000 7f28c9400640 1 --2- 172.21.15.151:0/623110464 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f28ac07a6b0 0x7f28ac07cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:13.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.867+0000 7f28c9400640 1 --2- 172.21.15.151:0/623110464 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f28ac07a6b0 0x7f28ac07cb70 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f28cc06aea0 tx=0x7f28b8009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:13.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:13.871+0000 7f28c2000640 1 -- 172.21.15.151:0/623110464 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f28bc020020 con 0x7f28cc06efa0 2024-09-16T15:53:14.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.094+0000 7f28d1000640 1 -- 172.21.15.151:0/623110464 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone52540", "target": ["mon-mgr", ""]}) -- 0x7f28cc0b2fc0 con 0x7f28ac07a6b0 2024-09-16T15:53:14.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.103+0000 7f28c2000640 1 -- 172.21.15.151:0/623110464 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7f28cc0b2fc0 con 0x7f28ac07a6b0 2024-09-16T15:53:14.104 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-16T15:53:14.104 INFO:teuthology.orchestra.run.smithi151.stdout: "status": { 2024-09-16T15:53:14.105 INFO:teuthology.orchestra.run.smithi151.stdout: "state": "canceled", 2024-09-16T15:53:14.105 INFO:teuthology.orchestra.run.smithi151.stdout: "source": { 2024-09-16T15:53:14.105 INFO:teuthology.orchestra.run.smithi151.stdout: "volume": "cephfs", 2024-09-16T15:53:14.105 INFO:teuthology.orchestra.run.smithi151.stdout: "subvolume": "sv1", 2024-09-16T15:53:14.105 INFO:teuthology.orchestra.run.smithi151.stdout: "snapshot": "ss1" 2024-09-16T15:53:14.105 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:53:14.105 INFO:teuthology.orchestra.run.smithi151.stdout: "failure": { 2024-09-16T15:53:14.105 INFO:teuthology.orchestra.run.smithi151.stdout: "errno": "4", 2024-09-16T15:53:14.105 INFO:teuthology.orchestra.run.smithi151.stdout: "error_msg": "user interrupted clone operation" 2024-09-16T15:53:14.105 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:53:14.105 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:53:14.105 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-16T15:53:14.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.106+0000 7f28d1000640 1 -- 172.21.15.151:0/623110464 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f28ac07a6b0 msgr2=0x7f28ac07cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:14.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.106+0000 7f28d1000640 1 --2- 172.21.15.151:0/623110464 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f28ac07a6b0 0x7f28ac07cb70 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f28cc06aea0 tx=0x7f28b8009040 comp rx=0 tx=0).stop 2024-09-16T15:53:14.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.106+0000 7f28d1000640 1 -- 172.21.15.151:0/623110464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28cc06efa0 msgr2=0x7f28cc143500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:14.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.106+0000 7f28d1000640 1 --2- 172.21.15.151:0/623110464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28cc06efa0 0x7f28cc143500 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f28bc00c660 tx=0x7f28bc00cb20 comp rx=0 tx=0).stop 2024-09-16T15:53:14.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.106+0000 7f28d1000640 1 -- 172.21.15.151:0/623110464 shutdown_connections 2024-09-16T15:53:14.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.106+0000 7f28d1000640 1 --2- 172.21.15.151:0/623110464 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f28ac07a6b0 0x7f28ac07cb70 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:14.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.106+0000 7f28d1000640 1 --2- 172.21.15.151:0/623110464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28cc0cccd0 0x7f28cc0c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:14.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.106+0000 7f28d1000640 1 --2- 172.21.15.151:0/623110464 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f28cc145750 0x7f28cc1466a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:14.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.106+0000 7f28d1000640 1 --2- 172.21.15.151:0/623110464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28cc06efa0 0x7f28cc143500 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:14.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.106+0000 7f28d1000640 1 -- 172.21.15.151:0/623110464 >> 172.21.15.151:0/623110464 conn(0x7f28cc0b1840 msgr2=0x7f28cc06f410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:14.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.106+0000 7f28d1000640 1 -- 172.21.15.151:0/623110464 shutdown_connections 2024-09-16T15:53:14.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.106+0000 7f28d1000640 1 -- 172.21.15.151:0/623110464 wait complete. 2024-09-16T15:53:14.123 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-16T15:53:14.123 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone15851 2024-09-16T15:53:14.286 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.284+0000 7ff4d7c00640 1 -- 172.21.15.151:0/4106097059 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff4d00cec60 msgr2=0x7ff4d00cf0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:14.286 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.284+0000 7ff4d7c00640 1 --2- 172.21.15.151:0/4106097059 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff4d00cec60 0x7ff4d00cf0b0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7ff4d00cfba0 tx=0x7ff4c802aa10 comp rx=0 tx=0).stop 2024-09-16T15:53:14.286 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.284+0000 7ff4d7c00640 1 -- 172.21.15.151:0/4106097059 shutdown_connections 2024-09-16T15:53:14.286 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.284+0000 7ff4d7c00640 1 --2- 172.21.15.151:0/4106097059 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff4d00cec60 0x7ff4d00cf0b0 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:14.286 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.284+0000 7ff4d7c00640 1 --2- 172.21.15.151:0/4106097059 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4d00af190 0x7ff4d00b2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:14.286 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.284+0000 7ff4d7c00640 1 --2- 172.21.15.151:0/4106097059 >> v2:172.21.15.167:3300/0 conn(0x7ff4d0161760 0x7ff4d00ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:14.286 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.284+0000 7ff4d7c00640 1 -- 172.21.15.151:0/4106097059 >> 172.21.15.151:0/4106097059 conn(0x7ff4d00b1840 msgr2=0x7ff4d00b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:14.286 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.284+0000 7ff4d7c00640 1 -- 172.21.15.151:0/4106097059 shutdown_connections 2024-09-16T15:53:14.286 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.285+0000 7ff4d7c00640 1 -- 172.21.15.151:0/4106097059 wait complete. 2024-09-16T15:53:14.287 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.286+0000 7ff4d7c00640 1 Processor -- start 2024-09-16T15:53:14.287 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.286+0000 7ff4d7c00640 1 -- start start 2024-09-16T15:53:14.290 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.287+0000 7ff4d7c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff4d00af190 0x7ff4d015ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:14.290 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.287+0000 7ff4d7c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4d00cec60 0x7ff4d0148610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:14.290 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.287+0000 7ff4d7c00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4d0161760 0x7ff4d0158a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:14.290 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.287+0000 7ff4d7c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d00b3200 con 0x7ff4d00af190 2024-09-16T15:53:14.290 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.287+0000 7ff4d7c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d00b3080 con 0x7ff4d00cec60 2024-09-16T15:53:14.290 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.287+0000 7ff4d7c00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7ff4d0162190 con 0x7ff4d0161760 2024-09-16T15:53:14.290 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.287+0000 7ff4d5200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff4d00af190 0x7ff4d015ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:14.290 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.287+0000 7ff4d5200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff4d00af190 0x7ff4d015ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33596/0 (socket says 172.21.15.151:33596) 2024-09-16T15:53:14.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.287+0000 7ff4d5200640 1 -- 172.21.15.151:0/300837139 learned_addr learned my addr 172.21.15.151:0/300837139 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:53:14.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.288+0000 7ff4d5c00640 1 --2- 172.21.15.151:0/300837139 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4d0161760 0x7ff4d0158a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:14.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.288+0000 7ff4cfe00640 1 --2- 172.21.15.151:0/300837139 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4d00cec60 0x7ff4d0148610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:14.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.288+0000 7ff4d5200640 1 -- 172.21.15.151:0/300837139 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4d0161760 msgr2=0x7ff4d0158a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:14.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.288+0000 7ff4d5200640 1 --2- 172.21.15.151:0/300837139 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4d0161760 0x7ff4d0158a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:14.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.288+0000 7ff4d5200640 1 -- 172.21.15.151:0/300837139 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4d00cec60 msgr2=0x7ff4d0148610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:14.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.288+0000 7ff4d5200640 1 --2- 172.21.15.151:0/300837139 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4d00cec60 0x7ff4d0148610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:14.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.288+0000 7ff4d5200640 1 -- 172.21.15.151:0/300837139 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4d015fae0 con 0x7ff4d00af190 2024-09-16T15:53:14.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.288+0000 7ff4cfe00640 1 --2- 172.21.15.151:0/300837139 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4d00cec60 0x7ff4d0148610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:53:14.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.288+0000 7ff4d5200640 1 --2- 172.21.15.151:0/300837139 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff4d00af190 0x7ff4d015ada0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7ff4c400e790 tx=0x7ff4c400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:14.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.288+0000 7ff4ccc00640 1 -- 172.21.15.151:0/300837139 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c400eea0 con 0x7ff4d00af190 2024-09-16T15:53:14.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.289+0000 7ff4ccc00640 1 -- 172.21.15.151:0/300837139 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff4c400cdb0 con 0x7ff4d00af190 2024-09-16T15:53:14.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.289+0000 7ff4d7c00640 1 -- 172.21.15.151:0/300837139 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4d015fdc0 con 0x7ff4d00af190 2024-09-16T15:53:14.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.289+0000 7ff4ccc00640 1 -- 172.21.15.151:0/300837139 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c4010690 con 0x7ff4d00af190 2024-09-16T15:53:14.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.289+0000 7ff4d7c00640 1 -- 172.21.15.151:0/300837139 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4d0160380 con 0x7ff4d00af190 2024-09-16T15:53:14.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.290+0000 7ff4d7c00640 1 -- 172.21.15.151:0/300837139 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4d0071030 con 0x7ff4d00af190 2024-09-16T15:53:14.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.291+0000 7ff4ccc00640 1 -- 172.21.15.151:0/300837139 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 46) ==== 104041+0+0 (secure 0 0 0) 0x7ff4c4010830 con 0x7ff4d00af190 2024-09-16T15:53:14.293 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.292+0000 7ff4ccc00640 1 --2- 172.21.15.151:0/300837139 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff4b007a6b0 0x7ff4b007cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:14.293 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.292+0000 7ff4ccc00640 1 -- 172.21.15.151:0/300837139 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5851+0+0 (secure 0 0 0) 0x7ff4c409a2c0 con 0x7ff4d00af190 2024-09-16T15:53:14.293 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.292+0000 7ff4cfe00640 1 --2- 172.21.15.151:0/300837139 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff4b007a6b0 0x7ff4b007cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:14.294 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.293+0000 7ff4cfe00640 1 --2- 172.21.15.151:0/300837139 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff4b007a6b0 0x7ff4b007cb70 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc009ba0 tx=0x7ff4bc005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:14.297 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.296+0000 7ff4ccc00640 1 -- 172.21.15.151:0/300837139 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff4c4063d40 con 0x7ff4d00af190 2024-09-16T15:53:14.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.519+0000 7ff4d7c00640 1 -- 172.21.15.151:0/300837139 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone15851", "target": ["mon-mgr", ""]}) -- 0x7ff4d0148b50 con 0x7ff4b007a6b0 2024-09-16T15:53:14.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.528+0000 7ff4ccc00640 1 -- 172.21.15.151:0/300837139 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7ff4d0148b50 con 0x7ff4b007a6b0 2024-09-16T15:53:14.529 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-16T15:53:14.529 INFO:teuthology.orchestra.run.smithi151.stdout: "status": { 2024-09-16T15:53:14.529 INFO:teuthology.orchestra.run.smithi151.stdout: "state": "canceled", 2024-09-16T15:53:14.529 INFO:teuthology.orchestra.run.smithi151.stdout: "source": { 2024-09-16T15:53:14.529 INFO:teuthology.orchestra.run.smithi151.stdout: "volume": "cephfs", 2024-09-16T15:53:14.530 INFO:teuthology.orchestra.run.smithi151.stdout: "subvolume": "sv1", 2024-09-16T15:53:14.530 INFO:teuthology.orchestra.run.smithi151.stdout: "snapshot": "ss1" 2024-09-16T15:53:14.530 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:53:14.530 INFO:teuthology.orchestra.run.smithi151.stdout: "failure": { 2024-09-16T15:53:14.530 INFO:teuthology.orchestra.run.smithi151.stdout: "errno": "4", 2024-09-16T15:53:14.530 INFO:teuthology.orchestra.run.smithi151.stdout: "error_msg": "user interrupted clone operation" 2024-09-16T15:53:14.530 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:53:14.530 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:53:14.530 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-16T15:53:14.532 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.530+0000 7ff4d7c00640 1 -- 172.21.15.151:0/300837139 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff4b007a6b0 msgr2=0x7ff4b007cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:14.532 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.530+0000 7ff4d7c00640 1 --2- 172.21.15.151:0/300837139 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff4b007a6b0 0x7ff4b007cb70 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc009ba0 tx=0x7ff4bc005f90 comp rx=0 tx=0).stop 2024-09-16T15:53:14.532 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.530+0000 7ff4d7c00640 1 -- 172.21.15.151:0/300837139 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff4d00af190 msgr2=0x7ff4d015ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:14.532 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.530+0000 7ff4d7c00640 1 --2- 172.21.15.151:0/300837139 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff4d00af190 0x7ff4d015ada0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7ff4c400e790 tx=0x7ff4c400ec50 comp rx=0 tx=0).stop 2024-09-16T15:53:14.532 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.530+0000 7ff4d7c00640 1 -- 172.21.15.151:0/300837139 shutdown_connections 2024-09-16T15:53:14.532 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.530+0000 7ff4d7c00640 1 --2- 172.21.15.151:0/300837139 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff4b007a6b0 0x7ff4b007cb70 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:14.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.530+0000 7ff4d7c00640 1 --2- 172.21.15.151:0/300837139 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4d00cec60 0x7ff4d0148610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:14.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.531+0000 7ff4d7c00640 1 --2- 172.21.15.151:0/300837139 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff4d0161760 0x7ff4d0158a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:14.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.531+0000 7ff4d7c00640 1 --2- 172.21.15.151:0/300837139 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff4d00af190 0x7ff4d015ada0 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:14.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.531+0000 7ff4d7c00640 1 -- 172.21.15.151:0/300837139 >> 172.21.15.151:0/300837139 conn(0x7ff4d00b1840 msgr2=0x7ff4d00b0cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:14.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.531+0000 7ff4d7c00640 1 -- 172.21.15.151:0/300837139 shutdown_connections 2024-09-16T15:53:14.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.531+0000 7ff4d7c00640 1 -- 172.21.15.151:0/300837139 wait complete. 2024-09-16T15:53:14.548 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-16T15:53:14.549 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone76352 2024-09-16T15:53:14.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.710+0000 7f6b23e00640 1 -- 172.21.15.151:0/3396302258 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b240b43c0 msgr2=0x7f6b080051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:14.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.710+0000 7f6b23e00640 1 --2- 172.21.15.151:0/3396302258 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b240b43c0 0x7f6b080051c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6b2408d200 tx=0x7f6b08005830 comp rx=0 tx=0).stop 2024-09-16T15:53:14.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.710+0000 7f6b23400640 1 -- 172.21.15.151:0/3396302258 reap_dead start 2024-09-16T15:53:14.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.710+0000 7f6b23e00640 1 -- 172.21.15.151:0/3396302258 shutdown_connections 2024-09-16T15:53:14.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.710+0000 7f6b23e00640 1 -- 172.21.15.151:0/3396302258 >> 172.21.15.151:0/3396302258 conn(0x7f6b2409d0b0 msgr2=0x7f6b240d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:14.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.710+0000 7f6b23e00640 1 -- 172.21.15.151:0/3396302258 shutdown_connections 2024-09-16T15:53:14.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.710+0000 7f6b23e00640 1 -- 172.21.15.151:0/3396302258 wait complete. 2024-09-16T15:53:14.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.712+0000 7f6b23e00640 1 Processor -- start 2024-09-16T15:53:14.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.712+0000 7f6b23e00640 1 -- start start 2024-09-16T15:53:14.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.713+0000 7f6b23e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6b240b43c0 0x7f6b241f8110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:14.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.713+0000 7f6b23e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b24159510 0x7f6b240b1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:14.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.713+0000 7f6b23e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b240b80f0 0x7f6b241f50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:14.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.713+0000 7f6b23e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6b240d2830 con 0x7f6b240b43c0 2024-09-16T15:53:14.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.713+0000 7f6b23e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6b240b5200 con 0x7f6b240b80f0 2024-09-16T15:53:14.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.713+0000 7f6b23e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f6b240d29b0 con 0x7f6b24159510 2024-09-16T15:53:14.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.713+0000 7f6b22a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6b240b43c0 0x7f6b241f8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:14.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.713+0000 7f6b22a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6b240b43c0 0x7f6b241f8110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33618/0 (socket says 172.21.15.151:33618) 2024-09-16T15:53:14.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.713+0000 7f6b22a00640 1 -- 172.21.15.151:0/3947340617 learned_addr learned my addr 172.21.15.151:0/3947340617 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:53:14.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.713+0000 7f6b22000640 1 --2- 172.21.15.151:0/3947340617 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b24159510 0x7f6b240b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:14.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.713+0000 7f6b23400640 1 --2- 172.21.15.151:0/3947340617 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b240b80f0 0x7f6b241f50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:14.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.714+0000 7f6b22a00640 1 -- 172.21.15.151:0/3947340617 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b24159510 msgr2=0x7f6b240b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:14.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.714+0000 7f6b22a00640 1 --2- 172.21.15.151:0/3947340617 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b24159510 0x7f6b240b1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:14.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.714+0000 7f6b22a00640 1 -- 172.21.15.151:0/3947340617 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b240b80f0 msgr2=0x7f6b241f50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:14.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.714+0000 7f6b22a00640 1 --2- 172.21.15.151:0/3947340617 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b240b80f0 0x7f6b241f50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:14.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.714+0000 7f6b22a00640 1 -- 172.21.15.151:0/3947340617 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b241f4c30 con 0x7f6b240b43c0 2024-09-16T15:53:14.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.714+0000 7f6b22a00640 1 --2- 172.21.15.151:0/3947340617 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6b240b43c0 0x7f6b241f8110 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f6b14011f50 tx=0x7f6b1400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:14.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.714+0000 7f6b1aa00640 1 -- 172.21.15.151:0/3947340617 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b1401e070 con 0x7f6b240b43c0 2024-09-16T15:53:14.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.714+0000 7f6b1aa00640 1 -- 172.21.15.151:0/3947340617 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6b14003d80 con 0x7f6b240b43c0 2024-09-16T15:53:14.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.714+0000 7f6b23e00640 1 -- 172.21.15.151:0/3947340617 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b241f86f0 con 0x7f6b240b43c0 2024-09-16T15:53:14.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.714+0000 7f6b1aa00640 1 -- 172.21.15.151:0/3947340617 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b140042b0 con 0x7f6b240b43c0 2024-09-16T15:53:14.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.714+0000 7f6b23e00640 1 -- 172.21.15.151:0/3947340617 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b2415b0f0 con 0x7f6b240b43c0 2024-09-16T15:53:14.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.716+0000 7f6b23e00640 1 -- 172.21.15.151:0/3947340617 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b240d22f0 con 0x7f6b240b43c0 2024-09-16T15:53:14.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.717+0000 7f6b1aa00640 1 -- 172.21.15.151:0/3947340617 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 46) ==== 104041+0+0 (secure 0 0 0) 0x7f6b14021070 con 0x7f6b240b43c0 2024-09-16T15:53:14.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.718+0000 7f6b1aa00640 1 --2- 172.21.15.151:0/3947340617 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6b0407a6b0 0x7f6b0407cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:14.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.718+0000 7f6b1aa00640 1 -- 172.21.15.151:0/3947340617 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5851+0+0 (secure 0 0 0) 0x7f6b1409a970 con 0x7f6b240b43c0 2024-09-16T15:53:14.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.718+0000 7f6b22000640 1 --2- 172.21.15.151:0/3947340617 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6b0407a6b0 0x7f6b0407cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:14.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.719+0000 7f6b22000640 1 --2- 172.21.15.151:0/3947340617 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6b0407a6b0 0x7f6b0407cb70 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f6b240b8080 tx=0x7f6b100067a0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:14.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.722+0000 7f6b1aa00640 1 -- 172.21.15.151:0/3947340617 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6b14064370 con 0x7f6b240b43c0 2024-09-16T15:53:14.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:14.944+0000 7f6b23e00640 1 -- 172.21.15.151:0/3947340617 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone76352", "target": ["mon-mgr", ""]}) -- 0x7f6b24069d80 con 0x7f6b0407a6b0 2024-09-16T15:53:18.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:18.758+0000 7f6b1aa00640 1 -- 172.21.15.151:0/3947340617 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7f6b10006c90 con 0x7f6b0407a6b0 2024-09-16T15:53:18.759 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-16T15:53:18.759 INFO:teuthology.orchestra.run.smithi151.stdout: "status": { 2024-09-16T15:53:18.760 INFO:teuthology.orchestra.run.smithi151.stdout: "state": "canceled", 2024-09-16T15:53:18.760 INFO:teuthology.orchestra.run.smithi151.stdout: "source": { 2024-09-16T15:53:18.760 INFO:teuthology.orchestra.run.smithi151.stdout: "volume": "cephfs", 2024-09-16T15:53:18.760 INFO:teuthology.orchestra.run.smithi151.stdout: "subvolume": "sv1", 2024-09-16T15:53:18.760 INFO:teuthology.orchestra.run.smithi151.stdout: "snapshot": "ss1" 2024-09-16T15:53:18.760 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:53:18.760 INFO:teuthology.orchestra.run.smithi151.stdout: "failure": { 2024-09-16T15:53:18.760 INFO:teuthology.orchestra.run.smithi151.stdout: "errno": "4", 2024-09-16T15:53:18.760 INFO:teuthology.orchestra.run.smithi151.stdout: "error_msg": "user interrupted clone operation" 2024-09-16T15:53:18.760 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:53:18.760 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:53:18.760 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-16T15:53:18.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:18.761+0000 7f6b23e00640 1 -- 172.21.15.151:0/3947340617 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6b0407a6b0 msgr2=0x7f6b0407cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:18.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:18.761+0000 7f6b23e00640 1 --2- 172.21.15.151:0/3947340617 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6b0407a6b0 0x7f6b0407cb70 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f6b240b8080 tx=0x7f6b100067a0 comp rx=0 tx=0).stop 2024-09-16T15:53:18.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:18.761+0000 7f6b23e00640 1 -- 172.21.15.151:0/3947340617 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6b240b43c0 msgr2=0x7f6b241f8110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:18.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:18.761+0000 7f6b23e00640 1 --2- 172.21.15.151:0/3947340617 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6b240b43c0 0x7f6b241f8110 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f6b14011f50 tx=0x7f6b1400f790 comp rx=0 tx=0).stop 2024-09-16T15:53:18.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:18.761+0000 7f6b23e00640 1 -- 172.21.15.151:0/3947340617 shutdown_connections 2024-09-16T15:53:18.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:18.761+0000 7f6b23e00640 1 --2- 172.21.15.151:0/3947340617 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6b0407a6b0 0x7f6b0407cb70 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:18.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:18.761+0000 7f6b23e00640 1 --2- 172.21.15.151:0/3947340617 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6b240b80f0 0x7f6b241f50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:18.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:18.761+0000 7f6b23e00640 1 --2- 172.21.15.151:0/3947340617 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6b24159510 0x7f6b240b1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:18.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:18.761+0000 7f6b23e00640 1 --2- 172.21.15.151:0/3947340617 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6b240b43c0 0x7f6b241f8110 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:18.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:18.761+0000 7f6b23e00640 1 -- 172.21.15.151:0/3947340617 >> 172.21.15.151:0/3947340617 conn(0x7f6b2409d0b0 msgr2=0x7f6b240ce820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:18.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:18.762+0000 7f6b23e00640 1 -- 172.21.15.151:0/3947340617 shutdown_connections 2024-09-16T15:53:18.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:18.762+0000 7f6b23e00640 1 -- 172.21.15.151:0/3947340617 wait complete. 2024-09-16T15:53:18.782 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-16T15:53:18.782 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-16T15:53:18.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:18.944+0000 7fac4d600640 1 -- 172.21.15.151:0/4276144317 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac2c004800 msgr2=0x7fac2c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:18.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:18.944+0000 7fac4d600640 1 --2- 172.21.15.151:0/4276144317 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac2c004800 0x7fac2c004c80 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fac4806f1d0 tx=0x7fac2c005830 comp rx=0 tx=0).stop 2024-09-16T15:53:18.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:18.944+0000 7fac47400640 1 -- 172.21.15.151:0/4276144317 reap_dead start 2024-09-16T15:53:18.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:18.944+0000 7fac4d600640 1 -- 172.21.15.151:0/4276144317 shutdown_connections 2024-09-16T15:53:18.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:18.944+0000 7fac4d600640 1 -- 172.21.15.151:0/4276144317 >> 172.21.15.151:0/4276144317 conn(0x7fac480b18e0 msgr2=0x7fac480b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:18.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:18.944+0000 7fac4d600640 1 -- 172.21.15.151:0/4276144317 shutdown_connections 2024-09-16T15:53:18.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:18.944+0000 7fac4d600640 1 -- 172.21.15.151:0/4276144317 wait complete. 2024-09-16T15:53:18.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:18.946+0000 7fac4d600640 1 Processor -- start 2024-09-16T15:53:18.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:18.946+0000 7fac4d600640 1 -- start start 2024-09-16T15:53:18.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:18.947+0000 7fac4d600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac2c004800 0x7fac48154e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:18.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:18.947+0000 7fac4d600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac480b3820 0x7fac480b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:18.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:18.947+0000 7fac4d600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac481f7e90 0x7fac481f8320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:18.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:18.947+0000 7fac4d600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fac480cca20 con 0x7fac480b3820 2024-09-16T15:53:18.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:18.947+0000 7fac4d600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fac480cc8a0 con 0x7fac481f7e90 2024-09-16T15:53:18.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:18.947+0000 7fac4d600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fac480ccba0 con 0x7fac2c004800 2024-09-16T15:53:18.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:18.947+0000 7fac3e000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac480b3820 0x7fac480b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:18.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:18.947+0000 7fac3e000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac480b3820 0x7fac480b3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33632/0 (socket says 172.21.15.151:33632) 2024-09-16T15:53:18.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:18.947+0000 7fac3e000640 1 -- 172.21.15.151:0/2287425429 learned_addr learned my addr 172.21.15.151:0/2287425429 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:53:18.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:18.947+0000 7fac47400640 1 --2- 172.21.15.151:0/2287425429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac481f7e90 0x7fac481f8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:18.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:18.947+0000 7fac46a00640 1 --2- 172.21.15.151:0/2287425429 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac2c004800 0x7fac48154e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:18.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:18.948+0000 7fac3e000640 1 -- 172.21.15.151:0/2287425429 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac2c004800 msgr2=0x7fac48154e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:18.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:18.948+0000 7fac3e000640 1 --2- 172.21.15.151:0/2287425429 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac2c004800 0x7fac48154e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:18.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:18.948+0000 7fac3e000640 1 -- 172.21.15.151:0/2287425429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac481f7e90 msgr2=0x7fac481f8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:18.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:18.948+0000 7fac3e000640 1 --2- 172.21.15.151:0/2287425429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac481f7e90 0x7fac481f8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:18.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:18.948+0000 7fac3e000640 1 -- 172.21.15.151:0/2287425429 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac48155390 con 0x7fac480b3820 2024-09-16T15:53:18.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:18.948+0000 7fac3e000640 1 --2- 172.21.15.151:0/2287425429 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac480b3820 0x7fac480b3ca0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fac3002ae50 tx=0x7fac300054c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:18.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:18.948+0000 7fac3f400640 1 -- 172.21.15.151:0/2287425429 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac30043020 con 0x7fac480b3820 2024-09-16T15:53:18.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:18.948+0000 7fac3f400640 1 -- 172.21.15.151:0/2287425429 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fac300035f0 con 0x7fac480b3820 2024-09-16T15:53:18.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:18.948+0000 7fac3f400640 1 -- 172.21.15.151:0/2287425429 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac30003a00 con 0x7fac480b3820 2024-09-16T15:53:18.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:18.948+0000 7fac4d600640 1 -- 172.21.15.151:0/2287425429 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac480cfa90 con 0x7fac480b3820 2024-09-16T15:53:18.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:18.949+0000 7fac4d600640 1 -- 172.21.15.151:0/2287425429 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac480d0050 con 0x7fac480b3820 2024-09-16T15:53:18.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:18.950+0000 7fac4d600640 1 -- 172.21.15.151:0/2287425429 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac48069360 con 0x7fac480b3820 2024-09-16T15:53:18.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:18.951+0000 7fac3f400640 1 -- 172.21.15.151:0/2287425429 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 46) ==== 104041+0+0 (secure 0 0 0) 0x7fac30046020 con 0x7fac480b3820 2024-09-16T15:53:18.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:18.952+0000 7fac3f400640 1 --2- 172.21.15.151:0/2287425429 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fac2407a6b0 0x7fac2407cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:18.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:18.952+0000 7fac3f400640 1 -- 172.21.15.151:0/2287425429 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5851+0+0 (secure 0 0 0) 0x7fac300b9580 con 0x7fac480b3820 2024-09-16T15:53:18.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:18.952+0000 7fac46a00640 1 --2- 172.21.15.151:0/2287425429 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fac2407a6b0 0x7fac2407cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:18.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:18.953+0000 7fac46a00640 1 --2- 172.21.15.151:0/2287425429 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fac2407a6b0 0x7fac2407cb70 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fac38006bd0 tx=0x7fac380067a0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:18.957 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:18.956+0000 7fac3f400640 1 -- 172.21.15.151:0/2287425429 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fac3002f070 con 0x7fac480b3820 2024-09-16T15:53:19.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.212+0000 7fac4d600640 1 -- 172.21.15.151:0/2287425429 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fac4806e8b0 con 0x7fac480b3820 2024-09-16T15:53:19.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.213+0000 7fac3f400640 1 -- 172.21.15.151:0/2287425429 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+2610 (secure 0 0 0) 0x7fac30083000 con 0x7fac480b3820 2024-09-16T15:53:19.214 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:53:19.214 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-16T15:53:19.214 INFO:teuthology.orchestra.run.smithi151.stdout: "fsid": "1757be18-e41a-4446-bb4f-ee4ac81c39df", 2024-09-16T15:53:19.214 INFO:teuthology.orchestra.run.smithi151.stdout: "health": { 2024-09-16T15:53:19.214 INFO:teuthology.orchestra.run.smithi151.stdout: "status": "HEALTH_OK", 2024-09-16T15:53:19.214 INFO:teuthology.orchestra.run.smithi151.stdout: "checks": {}, 2024-09-16T15:53:19.215 INFO:teuthology.orchestra.run.smithi151.stdout: "mutes": [] 2024-09-16T15:53:19.215 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:53:19.215 INFO:teuthology.orchestra.run.smithi151.stdout: "election_epoch": 4, 2024-09-16T15:53:19.215 INFO:teuthology.orchestra.run.smithi151.stdout: "quorum": [ 2024-09-16T15:53:19.215 INFO:teuthology.orchestra.run.smithi151.stdout: 0, 2024-09-16T15:53:19.215 INFO:teuthology.orchestra.run.smithi151.stdout: 1, 2024-09-16T15:53:19.215 INFO:teuthology.orchestra.run.smithi151.stdout: 2 2024-09-16T15:53:19.215 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:53:19.215 INFO:teuthology.orchestra.run.smithi151.stdout: "quorum_names": [ 2024-09-16T15:53:19.215 INFO:teuthology.orchestra.run.smithi151.stdout: "a", 2024-09-16T15:53:19.215 INFO:teuthology.orchestra.run.smithi151.stdout: "b", 2024-09-16T15:53:19.215 INFO:teuthology.orchestra.run.smithi151.stdout: "c" 2024-09-16T15:53:19.215 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:53:19.215 INFO:teuthology.orchestra.run.smithi151.stdout: "quorum_age": 862, 2024-09-16T15:53:19.216 INFO:teuthology.orchestra.run.smithi151.stdout: "monmap": { 2024-09-16T15:53:19.216 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 1, 2024-09-16T15:53:19.216 INFO:teuthology.orchestra.run.smithi151.stdout: "min_mon_release_name": "squid", 2024-09-16T15:53:19.216 INFO:teuthology.orchestra.run.smithi151.stdout: "num_mons": 3 2024-09-16T15:53:19.216 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:53:19.216 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap": { 2024-09-16T15:53:19.216 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 120, 2024-09-16T15:53:19.216 INFO:teuthology.orchestra.run.smithi151.stdout: "num_osds": 8, 2024-09-16T15:53:19.216 INFO:teuthology.orchestra.run.smithi151.stdout: "num_up_osds": 8, 2024-09-16T15:53:19.216 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_up_since": 1726501150, 2024-09-16T15:53:19.216 INFO:teuthology.orchestra.run.smithi151.stdout: "num_in_osds": 8, 2024-09-16T15:53:19.216 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_in_since": 1726501140, 2024-09-16T15:53:19.216 INFO:teuthology.orchestra.run.smithi151.stdout: "num_remapped_pgs": 0 2024-09-16T15:53:19.217 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:53:19.217 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap": { 2024-09-16T15:53:19.217 INFO:teuthology.orchestra.run.smithi151.stdout: "pgs_by_state": [ 2024-09-16T15:53:19.217 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:53:19.217 INFO:teuthology.orchestra.run.smithi151.stdout: "state_name": "active+clean", 2024-09-16T15:53:19.217 INFO:teuthology.orchestra.run.smithi151.stdout: "count": 97 2024-09-16T15:53:19.217 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:53:19.217 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:53:19.217 INFO:teuthology.orchestra.run.smithi151.stdout: "num_pgs": 97, 2024-09-16T15:53:19.217 INFO:teuthology.orchestra.run.smithi151.stdout: "num_pools": 3, 2024-09-16T15:53:19.217 INFO:teuthology.orchestra.run.smithi151.stdout: "num_objects": 1817, 2024-09-16T15:53:19.217 INFO:teuthology.orchestra.run.smithi151.stdout: "data_bytes": 7492333648, 2024-09-16T15:53:19.217 INFO:teuthology.orchestra.run.smithi151.stdout: "bytes_used": 16098824192, 2024-09-16T15:53:19.217 INFO:teuthology.orchestra.run.smithi151.stdout: "bytes_avail": 756995289088, 2024-09-16T15:53:19.217 INFO:teuthology.orchestra.run.smithi151.stdout: "bytes_total": 773094113280, 2024-09-16T15:53:19.218 INFO:teuthology.orchestra.run.smithi151.stdout: "recovering_objects_per_sec": 1, 2024-09-16T15:53:19.218 INFO:teuthology.orchestra.run.smithi151.stdout: "recovering_bytes_per_sec": 4320167, 2024-09-16T15:53:19.218 INFO:teuthology.orchestra.run.smithi151.stdout: "recovering_keys_per_sec": 0, 2024-09-16T15:53:19.218 INFO:teuthology.orchestra.run.smithi151.stdout: "num_objects_recovered": 13, 2024-09-16T15:53:19.218 INFO:teuthology.orchestra.run.smithi151.stdout: "num_bytes_recovered": 50331648, 2024-09-16T15:53:19.218 INFO:teuthology.orchestra.run.smithi151.stdout: "num_keys_recovered": 0, 2024-09-16T15:53:19.218 INFO:teuthology.orchestra.run.smithi151.stdout: "read_bytes_sec": 372977787, 2024-09-16T15:53:19.218 INFO:teuthology.orchestra.run.smithi151.stdout: "write_bytes_sec": 366547459, 2024-09-16T15:53:19.218 INFO:teuthology.orchestra.run.smithi151.stdout: "read_op_per_sec": 92, 2024-09-16T15:53:19.218 INFO:teuthology.orchestra.run.smithi151.stdout: "write_op_per_sec": 104 2024-09-16T15:53:19.218 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:53:19.218 INFO:teuthology.orchestra.run.smithi151.stdout: "fsmap": { 2024-09-16T15:53:19.218 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 77, 2024-09-16T15:53:19.218 INFO:teuthology.orchestra.run.smithi151.stdout: "btime": "2024-09-16T15:52:31:147854+0000", 2024-09-16T15:53:19.218 INFO:teuthology.orchestra.run.smithi151.stdout: "id": 8, 2024-09-16T15:53:19.218 INFO:teuthology.orchestra.run.smithi151.stdout: "up": 1, 2024-09-16T15:53:19.219 INFO:teuthology.orchestra.run.smithi151.stdout: "in": 1, 2024-09-16T15:53:19.219 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 1, 2024-09-16T15:53:19.219 INFO:teuthology.orchestra.run.smithi151.stdout: "by_rank": [ 2024-09-16T15:53:19.219 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:53:19.219 INFO:teuthology.orchestra.run.smithi151.stdout: "filesystem_id": 8, 2024-09-16T15:53:19.219 INFO:teuthology.orchestra.run.smithi151.stdout: "rank": 0, 2024-09-16T15:53:19.219 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "a", 2024-09-16T15:53:19.219 INFO:teuthology.orchestra.run.smithi151.stdout: "status": "up:active", 2024-09-16T15:53:19.219 INFO:teuthology.orchestra.run.smithi151.stdout: "gid": 8460 2024-09-16T15:53:19.219 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:53:19.219 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:53:19.219 INFO:teuthology.orchestra.run.smithi151.stdout: "up:standby": 3 2024-09-16T15:53:19.219 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:53:19.219 INFO:teuthology.orchestra.run.smithi151.stdout: "mgrmap": { 2024-09-16T15:53:19.219 INFO:teuthology.orchestra.run.smithi151.stdout: "available": true, 2024-09-16T15:53:19.220 INFO:teuthology.orchestra.run.smithi151.stdout: "num_standbys": 1, 2024-09-16T15:53:19.220 INFO:teuthology.orchestra.run.smithi151.stdout: "modules": [ 2024-09-16T15:53:19.220 INFO:teuthology.orchestra.run.smithi151.stdout: "iostat", 2024-09-16T15:53:19.220 INFO:teuthology.orchestra.run.smithi151.stdout: "nfs", 2024-09-16T15:53:19.220 INFO:teuthology.orchestra.run.smithi151.stdout: "restful" 2024-09-16T15:53:19.220 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:53:19.220 INFO:teuthology.orchestra.run.smithi151.stdout: "services": {} 2024-09-16T15:53:19.220 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:53:19.220 INFO:teuthology.orchestra.run.smithi151.stdout: "servicemap": { 2024-09-16T15:53:19.220 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 17, 2024-09-16T15:53:19.220 INFO:teuthology.orchestra.run.smithi151.stdout: "modified": "2024-09-16T15:53:15.928515+0000", 2024-09-16T15:53:19.220 INFO:teuthology.orchestra.run.smithi151.stdout: "services": { 2024-09-16T15:53:19.220 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": { 2024-09-16T15:53:19.220 INFO:teuthology.orchestra.run.smithi151.stdout: "daemons": { 2024-09-16T15:53:19.221 INFO:teuthology.orchestra.run.smithi151.stdout: "summary": "", 2024-09-16T15:53:19.221 INFO:teuthology.orchestra.run.smithi151.stdout: "a": { 2024-09-16T15:53:19.221 INFO:teuthology.orchestra.run.smithi151.stdout: "start_epoch": 0, 2024-09-16T15:53:19.221 INFO:teuthology.orchestra.run.smithi151.stdout: "start_stamp": "0.000000", 2024-09-16T15:53:19.221 INFO:teuthology.orchestra.run.smithi151.stdout: "gid": 0, 2024-09-16T15:53:19.221 INFO:teuthology.orchestra.run.smithi151.stdout: "addr": "(unrecognized address family 0)/0", 2024-09-16T15:53:19.221 INFO:teuthology.orchestra.run.smithi151.stdout: "metadata": {}, 2024-09-16T15:53:19.221 INFO:teuthology.orchestra.run.smithi151.stdout: "task_status": {} 2024-09-16T15:53:19.221 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:53:19.221 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:53:19.221 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:53:19.221 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:53:19.221 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:53:19.221 INFO:teuthology.orchestra.run.smithi151.stdout: "progress_events": {} 2024-09-16T15:53:19.222 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-16T15:53:19.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.215+0000 7fac4d600640 1 -- 172.21.15.151:0/2287425429 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fac2407a6b0 msgr2=0x7fac2407cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:19.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.215+0000 7fac4d600640 1 --2- 172.21.15.151:0/2287425429 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fac2407a6b0 0x7fac2407cb70 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fac38006bd0 tx=0x7fac380067a0 comp rx=0 tx=0).stop 2024-09-16T15:53:19.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.215+0000 7fac4d600640 1 -- 172.21.15.151:0/2287425429 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac480b3820 msgr2=0x7fac480b3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:19.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.215+0000 7fac4d600640 1 --2- 172.21.15.151:0/2287425429 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac480b3820 0x7fac480b3ca0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fac3002ae50 tx=0x7fac300054c0 comp rx=0 tx=0).stop 2024-09-16T15:53:19.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.215+0000 7fac4d600640 1 -- 172.21.15.151:0/2287425429 shutdown_connections 2024-09-16T15:53:19.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.215+0000 7fac4d600640 1 --2- 172.21.15.151:0/2287425429 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fac2407a6b0 0x7fac2407cb70 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:19.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.215+0000 7fac4d600640 1 --2- 172.21.15.151:0/2287425429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac481f7e90 0x7fac481f8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:19.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.215+0000 7fac4d600640 1 --2- 172.21.15.151:0/2287425429 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac480b3820 0x7fac480b3ca0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:19.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.215+0000 7fac4d600640 1 --2- 172.21.15.151:0/2287425429 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac2c004800 0x7fac48154e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:19.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.215+0000 7fac4d600640 1 -- 172.21.15.151:0/2287425429 >> 172.21.15.151:0/2287425429 conn(0x7fac480b18e0 msgr2=0x7fac480b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:19.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.215+0000 7fac4d600640 1 -- 172.21.15.151:0/2287425429 shutdown_connections 2024-09-16T15:53:19.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.215+0000 7fac4d600640 1 -- 172.21.15.151:0/2287425429 wait complete. 2024-09-16T15:53:19.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:53:19.232 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -lh volumes/_nogroup/subvol_clone52540/0926f44c-1384-450e-9d35-6c9cabee85aa 2024-09-16T15:53:19.232 DEBUG:teuthology.orchestra.run.smithi151:> ) 2024-09-16T15:53:19.269 INFO:teuthology.orchestra.run.smithi151.stderr:ls: cannot access 'volumes/_nogroup/subvol_clone52540/0926f44c-1384-450e-9d35-6c9cabee85aa': No such file or directory 2024-09-16T15:53:19.269 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-16T15:53:19.270 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-16T15:53:19.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.467+0000 7f546c000640 1 -- 172.21.15.151:0/578007235 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f544c003a00 msgr2=0x7f544c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:19.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.467+0000 7f546c000640 1 --2- 172.21.15.151:0/578007235 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f544c003a00 0x7f544c003e20 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f5458002230 tx=0x7f544c005830 comp rx=0 tx=0).stop 2024-09-16T15:53:19.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.467+0000 7f546a000640 1 -- 172.21.15.151:0/578007235 reap_dead start 2024-09-16T15:53:19.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.467+0000 7f546c000640 1 -- 172.21.15.151:0/578007235 shutdown_connections 2024-09-16T15:53:19.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.467+0000 7f546c000640 1 -- 172.21.15.151:0/578007235 >> 172.21.15.151:0/578007235 conn(0x7f54640b1950 msgr2=0x7f54640b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:19.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.467+0000 7f546c000640 1 -- 172.21.15.151:0/578007235 shutdown_connections 2024-09-16T15:53:19.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.467+0000 7f546c000640 1 -- 172.21.15.151:0/578007235 wait complete. 2024-09-16T15:53:19.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.469+0000 7f546c000640 1 Processor -- start 2024-09-16T15:53:19.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.469+0000 7f546c000640 1 -- start start 2024-09-16T15:53:19.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.470+0000 7f546c000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f544c003a00 0x7f54640b0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:19.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.470+0000 7f546c000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f54640c8fd0 0x7f5464153d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:19.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.470+0000 7f546c000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f54641f4fa0 0x7f546415d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:19.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.470+0000 7f546c000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f54640745c0 con 0x7f54640c8fd0 2024-09-16T15:53:19.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.470+0000 7f546c000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5464074440 con 0x7f544c003a00 2024-09-16T15:53:19.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.470+0000 7f546c000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f5464074740 con 0x7f54641f4fa0 2024-09-16T15:53:19.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.470+0000 7f5468c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f54640c8fd0 0x7f5464153d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:19.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.470+0000 7f5468c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f54640c8fd0 0x7f5464153d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33644/0 (socket says 172.21.15.151:33644) 2024-09-16T15:53:19.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.470+0000 7f5468c00640 1 -- 172.21.15.151:0/3220373684 learned_addr learned my addr 172.21.15.151:0/3220373684 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:53:19.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.471+0000 7f5468c00640 1 -- 172.21.15.151:0/3220373684 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f54641f4fa0 msgr2=0x7f546415d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:19.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.471+0000 7f5468c00640 1 --2- 172.21.15.151:0/3220373684 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f54641f4fa0 0x7f546415d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:19.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.471+0000 7f5468c00640 1 -- 172.21.15.151:0/3220373684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f544c003a00 msgr2=0x7f54640b0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:19.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.471+0000 7f5469600640 1 --2- 172.21.15.151:0/3220373684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f544c003a00 0x7f54640b0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:19.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.471+0000 7f5468c00640 1 --2- 172.21.15.151:0/3220373684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f544c003a00 0x7f54640b0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:19.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.471+0000 7f5468c00640 1 -- 172.21.15.151:0/3220373684 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54640b0cb0 con 0x7f54640c8fd0 2024-09-16T15:53:19.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.471+0000 7f5468c00640 1 --2- 172.21.15.151:0/3220373684 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f54640c8fd0 0x7f5464153d40 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f54500119d0 tx=0x7f5450011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:19.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.471+0000 7f5461600640 1 -- 172.21.15.151:0/3220373684 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f545001f070 con 0x7f54640c8fd0 2024-09-16T15:53:19.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.471+0000 7f5461600640 1 -- 172.21.15.151:0/3220373684 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f545000fe40 con 0x7f54640c8fd0 2024-09-16T15:53:19.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.471+0000 7f5461600640 1 -- 172.21.15.151:0/3220373684 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54500126b0 con 0x7f54640c8fd0 2024-09-16T15:53:19.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.471+0000 7f546c000640 1 -- 172.21.15.151:0/3220373684 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54640b0f90 con 0x7f54640c8fd0 2024-09-16T15:53:19.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.471+0000 7f546c000640 1 -- 172.21.15.151:0/3220373684 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54641f4a80 con 0x7f54640c8fd0 2024-09-16T15:53:19.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.472+0000 7f546c000640 1 -- 172.21.15.151:0/3220373684 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54641f4590 con 0x7f54640c8fd0 2024-09-16T15:53:19.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.474+0000 7f5461600640 1 -- 172.21.15.151:0/3220373684 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 46) ==== 104041+0+0 (secure 0 0 0) 0x7f5450022070 con 0x7f54640c8fd0 2024-09-16T15:53:19.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.474+0000 7f5461600640 1 --2- 172.21.15.151:0/3220373684 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f544407a6e0 0x7f544407cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:19.475 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.475+0000 7f5461600640 1 -- 172.21.15.151:0/3220373684 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5851+0+0 (secure 0 0 0) 0x7f545009a240 con 0x7f54640c8fd0 2024-09-16T15:53:19.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.475+0000 7f5469600640 1 --2- 172.21.15.151:0/3220373684 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f544407a6e0 0x7f544407cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:19.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.476+0000 7f5469600640 1 --2- 172.21.15.151:0/3220373684 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f544407a6e0 0x7f544407cba0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f5458033000 tx=0x7f5458038040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:19.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.479+0000 7f5461600640 1 -- 172.21.15.151:0/3220373684 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f5450063cc0 con 0x7f54640c8fd0 2024-09-16T15:53:19.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.687+0000 7f546c000640 1 -- 172.21.15.151:0/3220373684 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f5464154280 con 0x7f544407a6e0 2024-09-16T15:53:19.690 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.689+0000 7f5461600640 1 -- 172.21.15.151:0/3220373684 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f545802f040 con 0x7f544407a6e0 2024-09-16T15:53:19.690 INFO:teuthology.orchestra.run.smithi151.stdout:[ 2024-09-16T15:53:19.691 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:53:19.691 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "cephfs" 2024-09-16T15:53:19.691 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:53:19.691 INFO:teuthology.orchestra.run.smithi151.stdout:] 2024-09-16T15:53:19.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.692+0000 7f546c000640 1 -- 172.21.15.151:0/3220373684 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f544407a6e0 msgr2=0x7f544407cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:19.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.692+0000 7f546c000640 1 --2- 172.21.15.151:0/3220373684 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f544407a6e0 0x7f544407cba0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f5458033000 tx=0x7f5458038040 comp rx=0 tx=0).stop 2024-09-16T15:53:19.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.692+0000 7f546c000640 1 -- 172.21.15.151:0/3220373684 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f54640c8fd0 msgr2=0x7f5464153d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:19.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.692+0000 7f546c000640 1 --2- 172.21.15.151:0/3220373684 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f54640c8fd0 0x7f5464153d40 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f54500119d0 tx=0x7f5450011e90 comp rx=0 tx=0).stop 2024-09-16T15:53:19.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.692+0000 7f546c000640 1 -- 172.21.15.151:0/3220373684 shutdown_connections 2024-09-16T15:53:19.694 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.692+0000 7f546c000640 1 --2- 172.21.15.151:0/3220373684 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f544407a6e0 0x7f544407cba0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:19.694 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.692+0000 7f546c000640 1 --2- 172.21.15.151:0/3220373684 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f54641f4fa0 0x7f546415d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:19.694 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.692+0000 7f546c000640 1 --2- 172.21.15.151:0/3220373684 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f54640c8fd0 0x7f5464153d40 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:19.694 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.692+0000 7f546c000640 1 --2- 172.21.15.151:0/3220373684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f544c003a00 0x7f54640b0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:19.694 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.692+0000 7f546c000640 1 -- 172.21.15.151:0/3220373684 >> 172.21.15.151:0/3220373684 conn(0x7f54640b1950 msgr2=0x7f54640ae5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:19.694 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.692+0000 7f546c000640 1 -- 172.21.15.151:0/3220373684 shutdown_connections 2024-09-16T15:53:19.694 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.692+0000 7f546c000640 1 -- 172.21.15.151:0/3220373684 wait complete. 2024-09-16T15:53:19.709 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-09-16T15:53:19.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.868+0000 7f280ae00640 1 -- 172.21.15.151:0/2426556326 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f280406efa0 msgr2=0x7f2804073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:19.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.868+0000 7f280ae00640 1 --2- 172.21.15.151:0/2426556326 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f280406efa0 0x7f2804073440 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f27fc005fd0 tx=0x7f27fc02a8b0 comp rx=0 tx=0).stop 2024-09-16T15:53:19.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.868+0000 7f280ae00640 1 -- 172.21.15.151:0/2426556326 shutdown_connections 2024-09-16T15:53:19.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.868+0000 7f280ae00640 1 --2- 172.21.15.151:0/2426556326 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f280406efa0 0x7f2804073440 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:19.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.868+0000 7f280ae00640 1 -- 172.21.15.151:0/2426556326 >> 172.21.15.151:0/2426556326 conn(0x7f28040b1840 msgr2=0x7f28040b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:19.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.868+0000 7f280ae00640 1 -- 172.21.15.151:0/2426556326 shutdown_connections 2024-09-16T15:53:19.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.868+0000 7f280ae00640 1 -- 172.21.15.151:0/2426556326 wait complete. 2024-09-16T15:53:19.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.870+0000 7f280ae00640 1 Processor -- start 2024-09-16T15:53:19.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.870+0000 7f280ae00640 1 -- start start 2024-09-16T15:53:19.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.871+0000 7f280ae00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f280406efa0 0x7f28041f4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:19.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.871+0000 7f280ae00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f28040cccd0 0x7f280413bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:19.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.871+0000 7f280ae00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2804145750 0x7f2804141c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:19.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.871+0000 7f280ae00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f28040ce6a0 con 0x7f2804145750 2024-09-16T15:53:19.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.871+0000 7f280ae00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f28040ce520 con 0x7f280406efa0 2024-09-16T15:53:19.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.871+0000 7f280ae00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f28040ce820 con 0x7f28040cccd0 2024-09-16T15:53:19.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.871+0000 7f2808e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2804145750 0x7f2804141c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:19.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.871+0000 7f2808e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2804145750 0x7f2804141c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33664/0 (socket says 172.21.15.151:33664) 2024-09-16T15:53:19.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.871+0000 7f2808e00640 1 -- 172.21.15.151:0/4081015050 learned_addr learned my addr 172.21.15.151:0/4081015050 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:53:19.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.871+0000 7f2803e00640 1 --2- 172.21.15.151:0/4081015050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f280406efa0 0x7f28041f4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:19.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.872+0000 7f27fbe00640 1 --2- 172.21.15.151:0/4081015050 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f28040cccd0 0x7f280413bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:19.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.872+0000 7f2808e00640 1 -- 172.21.15.151:0/4081015050 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f28040cccd0 msgr2=0x7f280413bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:19.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.872+0000 7f2808e00640 1 --2- 172.21.15.151:0/4081015050 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f28040cccd0 0x7f280413bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:19.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.872+0000 7f2808e00640 1 -- 172.21.15.151:0/4081015050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f280406efa0 msgr2=0x7f28041f4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:19.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.872+0000 7f2808e00640 1 --2- 172.21.15.151:0/4081015050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f280406efa0 0x7f28041f4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:19.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.872+0000 7f2808e00640 1 -- 172.21.15.151:0/4081015050 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28041424e0 con 0x7f2804145750 2024-09-16T15:53:19.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.872+0000 7f2803e00640 1 --2- 172.21.15.151:0/4081015050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f280406efa0 0x7f28041f4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:53:19.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.872+0000 7f2808e00640 1 --2- 172.21.15.151:0/4081015050 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2804145750 0x7f2804141c20 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f27fc009cd0 tx=0x7f27fc003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:19.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.872+0000 7f2800c00640 1 -- 172.21.15.151:0/4081015050 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27fc040070 con 0x7f2804145750 2024-09-16T15:53:19.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.872+0000 7f2800c00640 1 -- 172.21.15.151:0/4081015050 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f27fc004250 con 0x7f2804145750 2024-09-16T15:53:19.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.872+0000 7f2800c00640 1 -- 172.21.15.151:0/4081015050 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27fc0046a0 con 0x7f2804145750 2024-09-16T15:53:19.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.872+0000 7f280ae00640 1 -- 172.21.15.151:0/4081015050 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28041427c0 con 0x7f2804145750 2024-09-16T15:53:19.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.872+0000 7f280ae00640 1 -- 172.21.15.151:0/4081015050 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2804142d80 con 0x7f2804145750 2024-09-16T15:53:19.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.873+0000 7f280ae00640 1 -- 172.21.15.151:0/4081015050 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28040cecd0 con 0x7f2804145750 2024-09-16T15:53:19.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.875+0000 7f2800c00640 1 -- 172.21.15.151:0/4081015050 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 46) ==== 104041+0+0 (secure 0 0 0) 0x7f27fc039040 con 0x7f2804145750 2024-09-16T15:53:19.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.876+0000 7f2800c00640 1 --2- 172.21.15.151:0/4081015050 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f27e007a6b0 0x7f27e007cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:19.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.876+0000 7f2800c00640 1 -- 172.21.15.151:0/4081015050 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5851+0+0 (secure 0 0 0) 0x7f27fc0bb180 con 0x7f2804145750 2024-09-16T15:53:19.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.876+0000 7f2803e00640 1 --2- 172.21.15.151:0/4081015050 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f27e007a6b0 0x7f27e007cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:19.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.877+0000 7f2803e00640 1 --2- 172.21.15.151:0/4081015050 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f27e007a6b0 0x7f27e007cb70 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f27f40046b0 tx=0x7f27f4009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:19.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:19.880+0000 7f2800c00640 1 -- 172.21.15.151:0/4081015050 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f27fc085250 con 0x7f2804145750 2024-09-16T15:53:20.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.091+0000 7f280ae00640 1 -- 172.21.15.151:0/4081015050 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f28040d02a0 con 0x7f27e007a6b0 2024-09-16T15:53:20.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.099+0000 7f2800c00640 1 -- 172.21.15.151:0/4081015050 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+185 (secure 0 0 0) 0x7f28040d02a0 con 0x7f27e007a6b0 2024-09-16T15:53:20.100 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:53:20.101 INFO:teuthology.orchestra.run.smithi151.stdout:[ 2024-09-16T15:53:20.101 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:53:20.101 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "subvol_clone15851" 2024-09-16T15:53:20.101 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:53:20.101 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:53:20.101 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sv1" 2024-09-16T15:53:20.101 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:53:20.101 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:53:20.101 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "subvol_clone52540" 2024-09-16T15:53:20.101 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:53:20.101 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:53:20.101 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "subvol_clone76352" 2024-09-16T15:53:20.101 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:53:20.101 INFO:teuthology.orchestra.run.smithi151.stdout:] 2024-09-16T15:53:20.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.102+0000 7f280ae00640 1 -- 172.21.15.151:0/4081015050 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f27e007a6b0 msgr2=0x7f27e007cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:20.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.102+0000 7f280ae00640 1 --2- 172.21.15.151:0/4081015050 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f27e007a6b0 0x7f27e007cb70 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f27f40046b0 tx=0x7f27f4009040 comp rx=0 tx=0).stop 2024-09-16T15:53:20.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.102+0000 7f280ae00640 1 -- 172.21.15.151:0/4081015050 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2804145750 msgr2=0x7f2804141c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:20.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.102+0000 7f280ae00640 1 --2- 172.21.15.151:0/4081015050 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2804145750 0x7f2804141c20 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f27fc009cd0 tx=0x7f27fc003c20 comp rx=0 tx=0).stop 2024-09-16T15:53:20.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.102+0000 7f280ae00640 1 -- 172.21.15.151:0/4081015050 shutdown_connections 2024-09-16T15:53:20.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.102+0000 7f280ae00640 1 --2- 172.21.15.151:0/4081015050 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f27e007a6b0 0x7f27e007cb70 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:20.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.102+0000 7f280ae00640 1 --2- 172.21.15.151:0/4081015050 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2804145750 0x7f2804141c20 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:20.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.102+0000 7f280ae00640 1 --2- 172.21.15.151:0/4081015050 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f28040cccd0 0x7f280413bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:20.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.102+0000 7f280ae00640 1 --2- 172.21.15.151:0/4081015050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f280406efa0 0x7f28041f4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:20.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.102+0000 7f280ae00640 1 -- 172.21.15.151:0/4081015050 >> 172.21.15.151:0/4081015050 conn(0x7f28040b1840 msgr2=0x7f28040aa5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:20.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.102+0000 7f280ae00640 1 -- 172.21.15.151:0/4081015050 shutdown_connections 2024-09-16T15:53:20.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.102+0000 7f280ae00640 1 -- 172.21.15.151:0/4081015050 wait complete. 2024-09-16T15:53:20.119 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone15851 2024-09-16T15:53:20.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.281+0000 7fb51f600640 1 -- 172.21.15.151:0/1304477021 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb518073de0 msgr2=0x7fb518074240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:20.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.281+0000 7fb51f600640 1 --2- 172.21.15.151:0/1304477021 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb518073de0 0x7fb518074240 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fb504005fd0 tx=0x7fb50402a4a0 comp rx=0 tx=0).stop 2024-09-16T15:53:20.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.281+0000 7fb51f600640 1 -- 172.21.15.151:0/1304477021 shutdown_connections 2024-09-16T15:53:20.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.281+0000 7fb51f600640 1 --2- 172.21.15.151:0/1304477021 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb518073de0 0x7fb518074240 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:20.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.281+0000 7fb51f600640 1 --2- 172.21.15.151:0/1304477021 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb51815f9a0 0x7fb51815fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:20.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.281+0000 7fb51f600640 1 -- 172.21.15.151:0/1304477021 >> 172.21.15.151:0/1304477021 conn(0x7fb5180b18e0 msgr2=0x7fb5180b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:20.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.281+0000 7fb51f600640 1 -- 172.21.15.151:0/1304477021 shutdown_connections 2024-09-16T15:53:20.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.281+0000 7fb51f600640 1 -- 172.21.15.151:0/1304477021 wait complete. 2024-09-16T15:53:20.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.283+0000 7fb51f600640 1 Processor -- start 2024-09-16T15:53:20.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.283+0000 7fb51f600640 1 -- start start 2024-09-16T15:53:20.286 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.284+0000 7fb51f600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb518073de0 0x7fb5180ce8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:20.286 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.284+0000 7fb51f600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb51815c3f0 0x7fb518153dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:20.286 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.284+0000 7fb51f600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb51815f9a0 0x7fb5181522e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:20.286 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.284+0000 7fb51f600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb5180cd3d0 con 0x7fb518073de0 2024-09-16T15:53:20.286 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.284+0000 7fb51f600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb5180ccbd0 con 0x7fb51815c3f0 2024-09-16T15:53:20.286 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.284+0000 7fb51f600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fb5180cd550 con 0x7fb51815f9a0 2024-09-16T15:53:20.286 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.284+0000 7fb51cc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb518073de0 0x7fb5180ce8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:20.287 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.284+0000 7fb51cc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb518073de0 0x7fb5180ce8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33682/0 (socket says 172.21.15.151:33682) 2024-09-16T15:53:20.287 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.284+0000 7fb51cc00640 1 -- 172.21.15.151:0/2513087622 learned_addr learned my addr 172.21.15.151:0/2513087622 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:53:20.287 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.284+0000 7fb517e00640 1 --2- 172.21.15.151:0/2513087622 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb51815c3f0 0x7fb518153dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:20.287 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.284+0000 7fb51d600640 1 --2- 172.21.15.151:0/2513087622 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb51815f9a0 0x7fb5181522e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:20.287 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.284+0000 7fb51cc00640 1 -- 172.21.15.151:0/2513087622 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb51815f9a0 msgr2=0x7fb5181522e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:20.287 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.284+0000 7fb51cc00640 1 --2- 172.21.15.151:0/2513087622 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb51815f9a0 0x7fb5181522e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:20.287 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.284+0000 7fb51cc00640 1 -- 172.21.15.151:0/2513087622 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb51815c3f0 msgr2=0x7fb518153dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:20.287 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.284+0000 7fb51cc00640 1 --2- 172.21.15.151:0/2513087622 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb51815c3f0 0x7fb518153dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:20.287 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.284+0000 7fb51cc00640 1 -- 172.21.15.151:0/2513087622 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5180b11d0 con 0x7fb518073de0 2024-09-16T15:53:20.287 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.285+0000 7fb51cc00640 1 --2- 172.21.15.151:0/2513087622 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb518073de0 0x7fb5180ce8e0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fb50800ea20 tx=0x7fb50800eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:20.287 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.285+0000 7fb514c00640 1 -- 172.21.15.151:0/2513087622 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb50800cde0 con 0x7fb518073de0 2024-09-16T15:53:20.287 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.285+0000 7fb514c00640 1 -- 172.21.15.151:0/2513087622 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb508004c10 con 0x7fb518073de0 2024-09-16T15:53:20.287 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.285+0000 7fb51f600640 1 -- 172.21.15.151:0/2513087622 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5180b14b0 con 0x7fb518073de0 2024-09-16T15:53:20.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.285+0000 7fb514c00640 1 -- 172.21.15.151:0/2513087622 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb508005020 con 0x7fb518073de0 2024-09-16T15:53:20.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.285+0000 7fb51f600640 1 -- 172.21.15.151:0/2513087622 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb518156a70 con 0x7fb518073de0 2024-09-16T15:53:20.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.286+0000 7fb51f600640 1 -- 172.21.15.151:0/2513087622 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb518069360 con 0x7fb518073de0 2024-09-16T15:53:20.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.287+0000 7fb514c00640 1 -- 172.21.15.151:0/2513087622 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 46) ==== 104041+0+0 (secure 0 0 0) 0x7fb5080051c0 con 0x7fb518073de0 2024-09-16T15:53:20.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.288+0000 7fb514c00640 1 --2- 172.21.15.151:0/2513087622 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb4f807a6b0 0x7fb4f807cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:20.290 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.288+0000 7fb514c00640 1 -- 172.21.15.151:0/2513087622 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5851+0+0 (secure 0 0 0) 0x7fb50809a360 con 0x7fb518073de0 2024-09-16T15:53:20.290 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.289+0000 7fb517e00640 1 --2- 172.21.15.151:0/2513087622 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb4f807a6b0 0x7fb4f807cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:20.290 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.290+0000 7fb517e00640 1 --2- 172.21.15.151:0/2513087622 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb4f807a6b0 0x7fb4f807cb70 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fb504033000 tx=0x7fb504004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:20.293 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.292+0000 7fb514c00640 1 -- 172.21.15.151:0/2513087622 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb508016030 con 0x7fb518073de0 2024-09-16T15:53:20.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.506+0000 7fb51f600640 1 -- 172.21.15.151:0/2513087622 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone15851", "target": ["mon-mgr", ""]}) -- 0x7fb5180b0f80 con 0x7fb4f807a6b0 2024-09-16T15:53:20.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.526+0000 7fb514c00640 1 -- 172.21.15.151:0/2513087622 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb5180b0f80 con 0x7fb4f807a6b0 2024-09-16T15:53:20.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.528+0000 7fb51f600640 1 -- 172.21.15.151:0/2513087622 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb4f807a6b0 msgr2=0x7fb4f807cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:20.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.528+0000 7fb51f600640 1 --2- 172.21.15.151:0/2513087622 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb4f807a6b0 0x7fb4f807cb70 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fb504033000 tx=0x7fb504004730 comp rx=0 tx=0).stop 2024-09-16T15:53:20.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.528+0000 7fb51f600640 1 -- 172.21.15.151:0/2513087622 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb518073de0 msgr2=0x7fb5180ce8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:20.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.528+0000 7fb51f600640 1 --2- 172.21.15.151:0/2513087622 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb518073de0 0x7fb5180ce8e0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fb50800ea20 tx=0x7fb50800eee0 comp rx=0 tx=0).stop 2024-09-16T15:53:20.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.528+0000 7fb51f600640 1 -- 172.21.15.151:0/2513087622 shutdown_connections 2024-09-16T15:53:20.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.528+0000 7fb51f600640 1 --2- 172.21.15.151:0/2513087622 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb4f807a6b0 0x7fb4f807cb70 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:20.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.528+0000 7fb51f600640 1 --2- 172.21.15.151:0/2513087622 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb51815c3f0 0x7fb518153dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:20.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.528+0000 7fb51f600640 1 --2- 172.21.15.151:0/2513087622 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb51815f9a0 0x7fb5181522e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:20.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.528+0000 7fb51f600640 1 --2- 172.21.15.151:0/2513087622 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb518073de0 0x7fb5180ce8e0 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:20.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.528+0000 7fb51f600640 1 -- 172.21.15.151:0/2513087622 >> 172.21.15.151:0/2513087622 conn(0x7fb5180b18e0 msgr2=0x7fb51815ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:20.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.529+0000 7fb51f600640 1 -- 172.21.15.151:0/2513087622 shutdown_connections 2024-09-16T15:53:20.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.529+0000 7fb51f600640 1 -- 172.21.15.151:0/2513087622 wait complete. 2024-09-16T15:53:20.540 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:53:20.539+0000 7f48f1c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:53:20.540 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:53:20.539+0000 7f48f1c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:53:20.540 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:53:20.539+0000 7f48f1c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:53:20.540 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:53:20.539+0000 7f48f1c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:53:20.540 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:53:20.539+0000 7f48f1c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:53:20.547 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-09-16T15:53:20.711 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.710+0000 7fa018e00640 1 -- 172.21.15.151:0/4051867022 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa01415f910 msgr2=0x7fa01415fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:20.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.710+0000 7fa018e00640 1 --2- 172.21.15.151:0/4051867022 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa01415f910 0x7fa01415fcf0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fa014069890 tx=0x7fa00402a5d0 comp rx=0 tx=0).stop 2024-09-16T15:53:20.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.710+0000 7fa018e00640 1 -- 172.21.15.151:0/4051867022 shutdown_connections 2024-09-16T15:53:20.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.710+0000 7fa018e00640 1 --2- 172.21.15.151:0/4051867022 >> v2:172.21.15.167:3300/0 conn(0x7fa01406e6f0 0x7fa01406eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:20.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.710+0000 7fa018e00640 1 --2- 172.21.15.151:0/4051867022 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa01415f910 0x7fa01415fcf0 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:20.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.710+0000 7fa018e00640 1 -- 172.21.15.151:0/4051867022 >> 172.21.15.151:0/4051867022 conn(0x7fa0140b18e0 msgr2=0x7fa0140b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:20.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.710+0000 7fa018e00640 1 -- 172.21.15.151:0/4051867022 shutdown_connections 2024-09-16T15:53:20.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.710+0000 7fa018e00640 1 -- 172.21.15.151:0/4051867022 wait complete. 2024-09-16T15:53:20.712 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.711+0000 7fa018e00640 1 Processor -- start 2024-09-16T15:53:20.713 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.712+0000 7fa018e00640 1 -- start start 2024-09-16T15:53:20.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.712+0000 7fa018e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa01406e6f0 0x7fa0140acd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:20.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.712+0000 7fa018e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa01415f910 0x7fa014155840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:20.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.712+0000 7fa018e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0140aea20 0x7fa0140b34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:20.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.712+0000 7fa018e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa01406f260 con 0x7fa0140aea20 2024-09-16T15:53:20.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.712+0000 7fa018e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa01406f0e0 con 0x7fa01415f910 2024-09-16T15:53:20.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.712+0000 7fa018e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fa01406f3e0 con 0x7fa01406e6f0 2024-09-16T15:53:20.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.713+0000 7fa012800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0140aea20 0x7fa0140b34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:20.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.713+0000 7fa012800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0140aea20 0x7fa0140b34f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33708/0 (socket says 172.21.15.151:33708) 2024-09-16T15:53:20.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.713+0000 7fa012800640 1 -- 172.21.15.151:0/2873507600 learned_addr learned my addr 172.21.15.151:0/2873507600 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:53:20.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.713+0000 7fa011400640 1 --2- 172.21.15.151:0/2873507600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa01415f910 0x7fa014155840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:20.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.713+0000 7fa011e00640 1 --2- 172.21.15.151:0/2873507600 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa01406e6f0 0x7fa0140acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:20.715 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.713+0000 7fa012800640 1 -- 172.21.15.151:0/2873507600 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa01406e6f0 msgr2=0x7fa0140acd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:20.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.713+0000 7fa012800640 1 --2- 172.21.15.151:0/2873507600 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa01406e6f0 0x7fa0140acd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:20.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.713+0000 7fa012800640 1 -- 172.21.15.151:0/2873507600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa01415f910 msgr2=0x7fa014155840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:20.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.713+0000 7fa012800640 1 --2- 172.21.15.151:0/2873507600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa01415f910 0x7fa014155840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:20.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.713+0000 7fa012800640 1 -- 172.21.15.151:0/2873507600 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0140ad3c0 con 0x7fa0140aea20 2024-09-16T15:53:20.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.714+0000 7fa012800640 1 --2- 172.21.15.151:0/2873507600 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0140aea20 0x7fa0140b34f0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fa00c00b9a0 tx=0x7fa00c00be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:20.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.714+0000 7fa00a000640 1 -- 172.21.15.151:0/2873507600 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa00c019070 con 0x7fa0140aea20 2024-09-16T15:53:20.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.714+0000 7fa00a000640 1 -- 172.21.15.151:0/2873507600 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa00c00ccb0 con 0x7fa0140aea20 2024-09-16T15:53:20.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.714+0000 7fa00a000640 1 -- 172.21.15.151:0/2873507600 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa00c0146a0 con 0x7fa0140aea20 2024-09-16T15:53:20.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.714+0000 7fa018e00640 1 -- 172.21.15.151:0/2873507600 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0140ad6a0 con 0x7fa0140aea20 2024-09-16T15:53:20.716 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.714+0000 7fa018e00640 1 -- 172.21.15.151:0/2873507600 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0140adc60 con 0x7fa0140aea20 2024-09-16T15:53:20.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.715+0000 7fa018e00640 1 -- 172.21.15.151:0/2873507600 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa014069360 con 0x7fa0140aea20 2024-09-16T15:53:20.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.716+0000 7fa00a000640 1 -- 172.21.15.151:0/2873507600 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 46) ==== 104041+0+0 (secure 0 0 0) 0x7fa00c01c070 con 0x7fa0140aea20 2024-09-16T15:53:20.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.717+0000 7fa00a000640 1 --2- 172.21.15.151:0/2873507600 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9ff407a6e0 0x7f9ff407cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:20.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.717+0000 7fa00a000640 1 -- 172.21.15.151:0/2873507600 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5851+0+0 (secure 0 0 0) 0x7fa00c099540 con 0x7fa0140aea20 2024-09-16T15:53:20.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.718+0000 7fa011e00640 1 --2- 172.21.15.151:0/2873507600 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9ff407a6e0 0x7f9ff407cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:20.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.718+0000 7fa011e00640 1 --2- 172.21.15.151:0/2873507600 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9ff407a6e0 0x7f9ff407cba0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fa004033000 tx=0x7fa004004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:20.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.721+0000 7fa00a000640 1 -- 172.21.15.151:0/2873507600 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa00c063060 con 0x7fa0140aea20 2024-09-16T15:53:20.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.936+0000 7fa018e00640 1 -- 172.21.15.151:0/2873507600 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa0140ade30 con 0x7f9ff407a6e0 2024-09-16T15:53:20.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.954+0000 7fa00a000640 1 -- 172.21.15.151:0/2873507600 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fa0040044b0 con 0x7f9ff407a6e0 2024-09-16T15:53:20.955 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:53:20.955 INFO:teuthology.orchestra.run.smithi151.stdout:[ 2024-09-16T15:53:20.956 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:53:20.956 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ss1" 2024-09-16T15:53:20.956 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:53:20.956 INFO:teuthology.orchestra.run.smithi151.stdout:] 2024-09-16T15:53:20.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.956+0000 7fa018e00640 1 -- 172.21.15.151:0/2873507600 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9ff407a6e0 msgr2=0x7f9ff407cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:20.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.956+0000 7fa018e00640 1 --2- 172.21.15.151:0/2873507600 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9ff407a6e0 0x7f9ff407cba0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fa004033000 tx=0x7fa004004730 comp rx=0 tx=0).stop 2024-09-16T15:53:20.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.957+0000 7fa018e00640 1 -- 172.21.15.151:0/2873507600 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0140aea20 msgr2=0x7fa0140b34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:20.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.957+0000 7fa018e00640 1 --2- 172.21.15.151:0/2873507600 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0140aea20 0x7fa0140b34f0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fa00c00b9a0 tx=0x7fa00c00be60 comp rx=0 tx=0).stop 2024-09-16T15:53:20.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.957+0000 7fa018e00640 1 -- 172.21.15.151:0/2873507600 shutdown_connections 2024-09-16T15:53:20.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.957+0000 7fa018e00640 1 --2- 172.21.15.151:0/2873507600 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9ff407a6e0 0x7f9ff407cba0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:20.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.957+0000 7fa018e00640 1 --2- 172.21.15.151:0/2873507600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa01415f910 0x7fa014155840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:20.959 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.957+0000 7fa018e00640 1 --2- 172.21.15.151:0/2873507600 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0140aea20 0x7fa0140b34f0 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:20.959 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.957+0000 7fa018e00640 1 --2- 172.21.15.151:0/2873507600 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa01406e6f0 0x7fa0140acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:20.959 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.957+0000 7fa018e00640 1 -- 172.21.15.151:0/2873507600 >> 172.21.15.151:0/2873507600 conn(0x7fa0140b18e0 msgr2=0x7fa0140b11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:20.959 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.957+0000 7fa018e00640 1 -- 172.21.15.151:0/2873507600 shutdown_connections 2024-09-16T15:53:20.959 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:20.957+0000 7fa018e00640 1 -- 172.21.15.151:0/2873507600 wait complete. 2024-09-16T15:53:20.975 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-09-16T15:53:21.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.142+0000 7eff7b000640 1 -- 172.21.15.151:0/4022622271 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff5c0051c0 msgr2=0x7eff5c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:21.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.142+0000 7eff7b000640 1 --2- 172.21.15.151:0/4022622271 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff5c0051c0 0x7eff5c00b9c0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7eff6c00c640 tx=0x7eff5c005830 comp rx=0 tx=0).stop 2024-09-16T15:53:21.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.142+0000 7eff7b000640 1 -- 172.21.15.151:0/4022622271 shutdown_connections 2024-09-16T15:53:21.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.142+0000 7eff7b000640 1 --2- 172.21.15.151:0/4022622271 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff5c0051c0 0x7eff5c00b9c0 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:21.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.142+0000 7eff7b000640 1 -- 172.21.15.151:0/4022622271 >> 172.21.15.151:0/4022622271 conn(0x7eff740cc490 msgr2=0x7eff740cc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:21.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.142+0000 7eff7b000640 1 -- 172.21.15.151:0/4022622271 shutdown_connections 2024-09-16T15:53:21.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.142+0000 7eff7b000640 1 -- 172.21.15.151:0/4022622271 wait complete. 2024-09-16T15:53:21.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.144+0000 7eff7b000640 1 Processor -- start 2024-09-16T15:53:21.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.144+0000 7eff7b000640 1 -- start start 2024-09-16T15:53:21.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.145+0000 7eff7b000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7eff5c0051c0 0x7eff741f89f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:21.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.145+0000 7eff7b000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff741f49f0 0x7eff7414e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:21.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.145+0000 7eff7b000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff740c7d00 0x7eff7414fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:21.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.145+0000 7eff7b000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7eff740ca4f0 con 0x7eff740c7d00 2024-09-16T15:53:21.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.145+0000 7eff7b000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7eff740ca370 con 0x7eff741f49f0 2024-09-16T15:53:21.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.145+0000 7eff7b000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7eff740c6430 con 0x7eff5c0051c0 2024-09-16T15:53:21.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.145+0000 7eff7a600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff740c7d00 0x7eff7414fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:21.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.145+0000 7eff7a600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff740c7d00 0x7eff7414fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33736/0 (socket says 172.21.15.151:33736) 2024-09-16T15:53:21.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.145+0000 7eff7a600640 1 -- 172.21.15.151:0/3809433603 learned_addr learned my addr 172.21.15.151:0/3809433603 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:53:21.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.145+0000 7eff79c00640 1 --2- 172.21.15.151:0/3809433603 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7eff5c0051c0 0x7eff741f89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:21.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.145+0000 7eff79200640 1 --2- 172.21.15.151:0/3809433603 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff741f49f0 0x7eff7414e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:21.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.146+0000 7eff7a600640 1 -- 172.21.15.151:0/3809433603 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7eff5c0051c0 msgr2=0x7eff741f89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:21.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.146+0000 7eff7a600640 1 --2- 172.21.15.151:0/3809433603 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7eff5c0051c0 0x7eff741f89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:21.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.146+0000 7eff7a600640 1 -- 172.21.15.151:0/3809433603 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff741f49f0 msgr2=0x7eff7414e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:21.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.146+0000 7eff7a600640 1 --2- 172.21.15.151:0/3809433603 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff741f49f0 0x7eff7414e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:21.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.146+0000 7eff7a600640 1 -- 172.21.15.151:0/3809433603 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff7414ead0 con 0x7eff740c7d00 2024-09-16T15:53:21.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.146+0000 7eff7a600640 1 --2- 172.21.15.151:0/3809433603 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff740c7d00 0x7eff7414fd20 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7eff6c00c760 tx=0x7eff6c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:21.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.146+0000 7eff71600640 1 -- 172.21.15.151:0/3809433603 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff6c02bbb0 con 0x7eff740c7d00 2024-09-16T15:53:21.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.146+0000 7eff71600640 1 -- 172.21.15.151:0/3809433603 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7eff6c03a070 con 0x7eff740c7d00 2024-09-16T15:53:21.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.146+0000 7eff71600640 1 -- 172.21.15.151:0/3809433603 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff6c0063d0 con 0x7eff740c7d00 2024-09-16T15:53:21.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.146+0000 7eff7b000640 1 -- 172.21.15.151:0/3809433603 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff741f4ef0 con 0x7eff740c7d00 2024-09-16T15:53:21.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.146+0000 7eff7b000640 1 -- 172.21.15.151:0/3809433603 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff741f54b0 con 0x7eff740c7d00 2024-09-16T15:53:21.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.147+0000 7eff7b000640 1 -- 172.21.15.151:0/3809433603 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff740c6820 con 0x7eff740c7d00 2024-09-16T15:53:21.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.149+0000 7eff71600640 1 -- 172.21.15.151:0/3809433603 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 46) ==== 104041+0+0 (secure 0 0 0) 0x7eff6c006570 con 0x7eff740c7d00 2024-09-16T15:53:21.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.150+0000 7eff71600640 1 --2- 172.21.15.151:0/3809433603 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7eff5407a6b0 0x7eff5407cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:21.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.150+0000 7eff71600640 1 -- 172.21.15.151:0/3809433603 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5851+0+0 (secure 0 0 0) 0x7eff6c0b9c10 con 0x7eff740c7d00 2024-09-16T15:53:21.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.150+0000 7eff79c00640 1 --2- 172.21.15.151:0/3809433603 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7eff5407a6b0 0x7eff5407cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:21.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.151+0000 7eff79c00640 1 --2- 172.21.15.151:0/3809433603 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7eff5407a6b0 0x7eff5407cb70 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7eff740c7bc0 tx=0x7eff68005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:21.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.154+0000 7eff71600640 1 -- 172.21.15.151:0/3809433603 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7eff7414ead0 con 0x7eff740c7d00 2024-09-16T15:53:21.366 DEBUG:teuthology.orchestra.run.smithi151:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:53:21.371 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:53:21.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.371+0000 7eff7b000640 1 -- 172.21.15.151:0/3809433603 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7eff740725e0 con 0x7eff5407a6b0 2024-09-16T15:53:21.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.405+0000 7eff71600640 1 -- 172.21.15.151:0/3809433603 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7eff740725e0 con 0x7eff5407a6b0 2024-09-16T15:53:21.406 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:53:21.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.408+0000 7eff7b000640 1 -- 172.21.15.151:0/3809433603 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7eff5407a6b0 msgr2=0x7eff5407cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:21.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.408+0000 7eff7b000640 1 --2- 172.21.15.151:0/3809433603 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7eff5407a6b0 0x7eff5407cb70 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7eff740c7bc0 tx=0x7eff68005f90 comp rx=0 tx=0).stop 2024-09-16T15:53:21.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.408+0000 7eff7b000640 1 -- 172.21.15.151:0/3809433603 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff740c7d00 msgr2=0x7eff7414fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:21.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.408+0000 7eff7b000640 1 --2- 172.21.15.151:0/3809433603 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff740c7d00 0x7eff7414fd20 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7eff6c00c760 tx=0x7eff6c003ba0 comp rx=0 tx=0).stop 2024-09-16T15:53:21.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.408+0000 7eff7b000640 1 -- 172.21.15.151:0/3809433603 shutdown_connections 2024-09-16T15:53:21.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.408+0000 7eff7b000640 1 --2- 172.21.15.151:0/3809433603 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7eff5407a6b0 0x7eff5407cb70 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:21.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.408+0000 7eff7b000640 1 --2- 172.21.15.151:0/3809433603 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff741f49f0 0x7eff7414e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:21.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.408+0000 7eff7b000640 1 --2- 172.21.15.151:0/3809433603 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff740c7d00 0x7eff7414fd20 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:21.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.408+0000 7eff7b000640 1 --2- 172.21.15.151:0/3809433603 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7eff5c0051c0 0x7eff741f89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:21.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.408+0000 7eff7b000640 1 -- 172.21.15.151:0/3809433603 >> 172.21.15.151:0/3809433603 conn(0x7eff740cc490 msgr2=0x7eff7415fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:21.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.408+0000 7eff7b000640 1 -- 172.21.15.151:0/3809433603 shutdown_connections 2024-09-16T15:53:21.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.408+0000 7eff7b000640 1 -- 172.21.15.151:0/3809433603 wait complete. 2024-09-16T15:53:21.425 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-09-16T15:53:21.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.656+0000 7f695cc00640 1 -- 172.21.15.151:0/2430698463 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f695806efa0 msgr2=0x7f69580cc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:21.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.656+0000 7f695cc00640 1 --2- 172.21.15.151:0/2430698463 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f695806efa0 0x7f69580cc5c0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f69580d0cd0 tx=0x7f694402a4a0 comp rx=0 tx=0).stop 2024-09-16T15:53:21.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.656+0000 7f695cc00640 1 -- 172.21.15.151:0/2430698463 shutdown_connections 2024-09-16T15:53:21.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.656+0000 7f695cc00640 1 --2- 172.21.15.151:0/2430698463 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f695806efa0 0x7f69580cc5c0 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:21.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.656+0000 7f695cc00640 1 --2- 172.21.15.151:0/2430698463 >> v2:172.21.15.167:3300/0 conn(0x7f6958145750 0x7f6958073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:21.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.656+0000 7f695cc00640 1 -- 172.21.15.151:0/2430698463 >> 172.21.15.151:0/2430698463 conn(0x7f69580b1840 msgr2=0x7f69580b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:21.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.656+0000 7f695cc00640 1 -- 172.21.15.151:0/2430698463 shutdown_connections 2024-09-16T15:53:21.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.656+0000 7f695cc00640 1 -- 172.21.15.151:0/2430698463 wait complete. 2024-09-16T15:53:21.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.658+0000 7f695cc00640 1 Processor -- start 2024-09-16T15:53:21.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.658+0000 7f695cc00640 1 -- start start 2024-09-16T15:53:21.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.659+0000 7f695cc00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f695806efa0 0x7f69580ae950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:21.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.659+0000 7f695cc00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f69580ccb00 0x7f69580c6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:21.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.659+0000 7f695cc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6958145750 0x7f69580c7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:21.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.659+0000 7f695cc00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f69580ced40 con 0x7f6958145750 2024-09-16T15:53:21.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.659+0000 7f695cc00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f69580cebc0 con 0x7f695806efa0 2024-09-16T15:53:21.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.659+0000 7f695cc00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f69580ceec0 con 0x7f69580ccb00 2024-09-16T15:53:21.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.659+0000 7f6956800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6958145750 0x7f69580c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:21.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.659+0000 7f6955e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f695806efa0 0x7f69580ae950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:21.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.659+0000 7f6955e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f695806efa0 0x7f69580ae950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.151:58624/0 (socket says 172.21.15.151:58624) 2024-09-16T15:53:21.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.659+0000 7f6955e00640 1 -- 172.21.15.151:0/3732399230 learned_addr learned my addr 172.21.15.151:0/3732399230 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:53:21.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.659+0000 7f6955400640 1 --2- 172.21.15.151:0/3732399230 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f69580ccb00 0x7f69580c6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:21.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.660+0000 7f6956800640 1 -- 172.21.15.151:0/3732399230 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f69580ccb00 msgr2=0x7f69580c6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:21.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.660+0000 7f6956800640 1 --2- 172.21.15.151:0/3732399230 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f69580ccb00 0x7f69580c6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:21.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.660+0000 7f6956800640 1 -- 172.21.15.151:0/3732399230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f695806efa0 msgr2=0x7f69580ae950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:21.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.660+0000 7f6956800640 1 --2- 172.21.15.151:0/3732399230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f695806efa0 0x7f69580ae950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:21.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.660+0000 7f6956800640 1 -- 172.21.15.151:0/3732399230 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69581f9190 con 0x7f6958145750 2024-09-16T15:53:21.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.660+0000 7f6956800640 1 --2- 172.21.15.151:0/3732399230 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6958145750 0x7f69580c7520 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f695000caa0 tx=0x7f695000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:21.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.660+0000 7f694e000640 1 -- 172.21.15.151:0/3732399230 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6950016020 con 0x7f6958145750 2024-09-16T15:53:21.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.660+0000 7f694e000640 1 -- 172.21.15.151:0/3732399230 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f695000ad10 con 0x7f6958145750 2024-09-16T15:53:21.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.660+0000 7f695cc00640 1 -- 172.21.15.151:0/3732399230 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69581f4e50 con 0x7f6958145750 2024-09-16T15:53:21.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.661+0000 7f694e000640 1 -- 172.21.15.151:0/3732399230 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6950012680 con 0x7f6958145750 2024-09-16T15:53:21.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.661+0000 7f695cc00640 1 -- 172.21.15.151:0/3732399230 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69581f5410 con 0x7f6958145750 2024-09-16T15:53:21.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.662+0000 7f695cc00640 1 -- 172.21.15.151:0/3732399230 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6958075290 con 0x7f6958145750 2024-09-16T15:53:21.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.663+0000 7f694e000640 1 -- 172.21.15.151:0/3732399230 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 46) ==== 104041+0+0 (secure 0 0 0) 0x7f6950019020 con 0x7f6958145750 2024-09-16T15:53:21.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.664+0000 7f694e000640 1 --2- 172.21.15.151:0/3732399230 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f693807a6b0 0x7f693807cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:21.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.664+0000 7f694e000640 1 -- 172.21.15.151:0/3732399230 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5851+0+0 (secure 0 0 0) 0x7f6950099dc0 con 0x7f6958145750 2024-09-16T15:53:21.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.664+0000 7f6955e00640 1 --2- 172.21.15.151:0/3732399230 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f693807a6b0 0x7f693807cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:21.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.665+0000 7f6955e00640 1 --2- 172.21.15.151:0/3732399230 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f693807a6b0 0x7f693807cb70 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f6948009bf0 tx=0x7f6948005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:21.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.668+0000 7f694e000640 1 -- 172.21.15.151:0/3732399230 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6958075290 con 0x7f6958145750 2024-09-16T15:53:21.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.884+0000 7f695cc00640 1 -- 172.21.15.151:0/3732399230 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f69580cc5c0 con 0x7f693807a6b0 2024-09-16T15:53:21.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.904+0000 7f694e000640 1 -- 172.21.15.151:0/3732399230 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f69580cc5c0 con 0x7f693807a6b0 2024-09-16T15:53:21.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.906+0000 7f695cc00640 1 -- 172.21.15.151:0/3732399230 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f693807a6b0 msgr2=0x7f693807cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:21.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.906+0000 7f695cc00640 1 --2- 172.21.15.151:0/3732399230 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f693807a6b0 0x7f693807cb70 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f6948009bf0 tx=0x7f6948005f90 comp rx=0 tx=0).stop 2024-09-16T15:53:21.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.906+0000 7f695cc00640 1 -- 172.21.15.151:0/3732399230 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6958145750 msgr2=0x7f69580c7520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:21.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.906+0000 7f695cc00640 1 --2- 172.21.15.151:0/3732399230 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6958145750 0x7f69580c7520 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f695000caa0 tx=0x7f695000cf60 comp rx=0 tx=0).stop 2024-09-16T15:53:21.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.906+0000 7f695cc00640 1 -- 172.21.15.151:0/3732399230 shutdown_connections 2024-09-16T15:53:21.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.906+0000 7f695cc00640 1 --2- 172.21.15.151:0/3732399230 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f693807a6b0 0x7f693807cb70 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:21.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.906+0000 7f695cc00640 1 --2- 172.21.15.151:0/3732399230 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6958145750 0x7f69580c7520 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:21.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.906+0000 7f695cc00640 1 --2- 172.21.15.151:0/3732399230 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f69580ccb00 0x7f69580c6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:21.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.906+0000 7f695cc00640 1 --2- 172.21.15.151:0/3732399230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f695806efa0 0x7f69580ae950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:21.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.906+0000 7f695cc00640 1 -- 172.21.15.151:0/3732399230 >> 172.21.15.151:0/3732399230 conn(0x7f69580b1840 msgr2=0x7f69580af950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:21.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.907+0000 7f695cc00640 1 -- 172.21.15.151:0/3732399230 shutdown_connections 2024-09-16T15:53:21.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:21.907+0000 7f695cc00640 1 -- 172.21.15.151:0/3732399230 wait complete. 2024-09-16T15:53:21.932 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone52540 2024-09-16T15:53:22.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.094+0000 7f3b71200640 1 -- 172.21.15.151:0/4076591836 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3b6c15f2b0 msgr2=0x7f3b6c0d0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:22.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.094+0000 7f3b71200640 1 --2- 172.21.15.151:0/4076591836 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3b6c15f2b0 0x7f3b6c0d0650 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c009830 tx=0x7f3b5c02a510 comp rx=0 tx=0).stop 2024-09-16T15:53:22.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.094+0000 7f3b71200640 1 -- 172.21.15.151:0/4076591836 shutdown_connections 2024-09-16T15:53:22.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.094+0000 7f3b71200640 1 --2- 172.21.15.151:0/4076591836 >> v2:172.21.15.167:3300/0 conn(0x7f3b6c0cc500 0x7f3b6c0cc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:22.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.094+0000 7f3b71200640 1 --2- 172.21.15.151:0/4076591836 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3b6c15f2b0 0x7f3b6c0d0650 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:22.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.094+0000 7f3b71200640 1 -- 172.21.15.151:0/4076591836 >> 172.21.15.151:0/4076591836 conn(0x7f3b6c0d1cd0 msgr2=0x7f3b6c0d20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:22.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.094+0000 7f3b71200640 1 -- 172.21.15.151:0/4076591836 shutdown_connections 2024-09-16T15:53:22.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.094+0000 7f3b71200640 1 -- 172.21.15.151:0/4076591836 wait complete. 2024-09-16T15:53:22.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.096+0000 7f3b71200640 1 Processor -- start 2024-09-16T15:53:22.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.096+0000 7f3b71200640 1 -- start start 2024-09-16T15:53:22.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.097+0000 7f3b71200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3b6c0cc500 0x7f3b6c1f4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:22.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.097+0000 7f3b71200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b6c0ce550 0x7f3b6c06f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:22.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.097+0000 7f3b71200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b6c15f2b0 0x7f3b6c0b6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:22.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.097+0000 7f3b71200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3b6c071af0 con 0x7f3b6c15f2b0 2024-09-16T15:53:22.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.097+0000 7f3b71200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3b6c071970 con 0x7f3b6c0ce550 2024-09-16T15:53:22.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.097+0000 7f3b71200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f3b6c0ccef0 con 0x7f3b6c0cc500 2024-09-16T15:53:22.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.098+0000 7f3b6be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b6c15f2b0 0x7f3b6c0b6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:22.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.098+0000 7f3b6b400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3b6c0cc500 0x7f3b6c1f4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:22.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.098+0000 7f3b6be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b6c15f2b0 0x7f3b6c0b6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33760/0 (socket says 172.21.15.151:33760) 2024-09-16T15:53:22.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.098+0000 7f3b6b400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3b6c0cc500 0x7f3b6c1f4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.151:56832/0 (socket says 172.21.15.151:56832) 2024-09-16T15:53:22.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.098+0000 7f3b6be00640 1 -- 172.21.15.151:0/782418970 learned_addr learned my addr 172.21.15.151:0/782418970 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:53:22.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.098+0000 7f3b6aa00640 1 --2- 172.21.15.151:0/782418970 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b6c0ce550 0x7f3b6c06f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:22.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.098+0000 7f3b6aa00640 1 -- 172.21.15.151:0/782418970 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3b6c0cc500 msgr2=0x7f3b6c1f4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:22.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.098+0000 7f3b6aa00640 1 --2- 172.21.15.151:0/782418970 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3b6c0cc500 0x7f3b6c1f4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:22.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.098+0000 7f3b6aa00640 1 -- 172.21.15.151:0/782418970 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b6c15f2b0 msgr2=0x7f3b6c0b6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:22.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.098+0000 7f3b6aa00640 1 --2- 172.21.15.151:0/782418970 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b6c15f2b0 0x7f3b6c0b6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:22.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.098+0000 7f3b6aa00640 1 -- 172.21.15.151:0/782418970 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b6c06dc80 con 0x7f3b6c0ce550 2024-09-16T15:53:22.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.098+0000 7f3b6b400640 1 --2- 172.21.15.151:0/782418970 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3b6c0cc500 0x7f3b6c1f4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:53:22.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.098+0000 7f3b6be00640 1 --2- 172.21.15.151:0/782418970 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b6c15f2b0 0x7f3b6c0b6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:53:22.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.099+0000 7f3b6aa00640 1 --2- 172.21.15.151:0/782418970 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b6c0ce550 0x7f3b6c06f6b0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f3b5800c390 tx=0x7f3b5800c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:22.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.099+0000 7f3b63400640 1 -- 172.21.15.151:0/782418970 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b58011790 con 0x7f3b6c0ce550 2024-09-16T15:53:22.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.099+0000 7f3b63400640 1 -- 172.21.15.151:0/782418970 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3b58017070 con 0x7f3b6c0ce550 2024-09-16T15:53:22.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.099+0000 7f3b63400640 1 -- 172.21.15.151:0/782418970 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b58011ce0 con 0x7f3b6c0ce550 2024-09-16T15:53:22.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.099+0000 7f3b71200640 1 -- 172.21.15.151:0/782418970 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b6c06df60 con 0x7f3b6c0ce550 2024-09-16T15:53:22.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.099+0000 7f3b71200640 1 -- 172.21.15.151:0/782418970 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b6c06e520 con 0x7f3b6c0ce550 2024-09-16T15:53:22.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.100+0000 7f3b71200640 1 -- 172.21.15.151:0/782418970 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b6c0cc9b0 con 0x7f3b6c0ce550 2024-09-16T15:53:22.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.102+0000 7f3b63400640 1 -- 172.21.15.151:0/782418970 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 46) ==== 104041+0+0 (secure 0 0 0) 0x7f3b5801c460 con 0x7f3b6c0ce550 2024-09-16T15:53:22.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.103+0000 7f3b63400640 1 --2- 172.21.15.151:0/782418970 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3b4c07a6b0 0x7f3b4c07cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:22.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.103+0000 7f3b63400640 1 -- 172.21.15.151:0/782418970 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5851+0+0 (secure 0 0 0) 0x7f3b580994a0 con 0x7f3b6c0ce550 2024-09-16T15:53:22.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.103+0000 7f3b6b400640 1 --2- 172.21.15.151:0/782418970 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3b4c07a6b0 0x7f3b4c07cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:22.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.104+0000 7f3b6b400640 1 --2- 172.21.15.151:0/782418970 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3b4c07a6b0 0x7f3b4c07cb70 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c034000 tx=0x7f3b5c004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:22.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.108+0000 7f3b63400640 1 -- 172.21.15.151:0/782418970 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f3b58062fa0 con 0x7f3b6c0ce550 2024-09-16T15:53:22.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.428+0000 7f3b71200640 1 -- 172.21.15.151:0/782418970 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone52540", "target": ["mon-mgr", ""]}) -- 0x7f3b6c0af6a0 con 0x7f3b4c07a6b0 2024-09-16T15:53:22.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.671+0000 7f3b63400640 1 -- 172.21.15.151:0/782418970 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3b6c0af6a0 con 0x7f3b4c07a6b0 2024-09-16T15:53:22.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.674+0000 7f3b71200640 1 -- 172.21.15.151:0/782418970 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3b4c07a6b0 msgr2=0x7f3b4c07cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:22.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.674+0000 7f3b71200640 1 --2- 172.21.15.151:0/782418970 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3b4c07a6b0 0x7f3b4c07cb70 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c034000 tx=0x7f3b5c004730 comp rx=0 tx=0).stop 2024-09-16T15:53:22.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.674+0000 7f3b71200640 1 -- 172.21.15.151:0/782418970 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b6c0ce550 msgr2=0x7f3b6c06f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:22.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.674+0000 7f3b71200640 1 --2- 172.21.15.151:0/782418970 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b6c0ce550 0x7f3b6c06f6b0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f3b5800c390 tx=0x7f3b5800c850 comp rx=0 tx=0).stop 2024-09-16T15:53:22.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.674+0000 7f3b71200640 1 -- 172.21.15.151:0/782418970 shutdown_connections 2024-09-16T15:53:22.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.674+0000 7f3b71200640 1 --2- 172.21.15.151:0/782418970 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3b4c07a6b0 0x7f3b4c07cb70 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:22.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.674+0000 7f3b71200640 1 --2- 172.21.15.151:0/782418970 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b6c0ce550 0x7f3b6c06f6b0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:22.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.674+0000 7f3b71200640 1 --2- 172.21.15.151:0/782418970 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b6c15f2b0 0x7f3b6c0b6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:22.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.674+0000 7f3b71200640 1 --2- 172.21.15.151:0/782418970 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3b6c0cc500 0x7f3b6c1f4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:22.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.674+0000 7f3b71200640 1 -- 172.21.15.151:0/782418970 >> 172.21.15.151:0/782418970 conn(0x7f3b6c0d1cd0 msgr2=0x7f3b6c0d20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:22.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.674+0000 7f3b71200640 1 -- 172.21.15.151:0/782418970 shutdown_connections 2024-09-16T15:53:22.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.674+0000 7f3b71200640 1 -- 172.21.15.151:0/782418970 wait complete. 2024-09-16T15:53:22.697 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone76352 2024-09-16T15:53:22.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.862+0000 7f820e800640 1 -- 172.21.15.151:0/3005393290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f820815bda0 msgr2=0x7f8208073010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:22.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.862+0000 7f820e800640 1 --2- 172.21.15.151:0/3005393290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f820815bda0 0x7f8208073010 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f81fc009830 tx=0x7f81fc02a480 comp rx=0 tx=0).stop 2024-09-16T15:53:22.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.862+0000 7f820e800640 1 -- 172.21.15.151:0/3005393290 shutdown_connections 2024-09-16T15:53:22.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.862+0000 7f820e800640 1 --2- 172.21.15.151:0/3005393290 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8208070550 0x7f82080cb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:22.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.862+0000 7f820e800640 1 --2- 172.21.15.151:0/3005393290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f820815bda0 0x7f8208073010 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:22.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.862+0000 7f820e800640 1 -- 172.21.15.151:0/3005393290 >> 172.21.15.151:0/3005393290 conn(0x7f82080cfa70 msgr2=0x7f82080cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:22.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.862+0000 7f820e800640 1 -- 172.21.15.151:0/3005393290 shutdown_connections 2024-09-16T15:53:22.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.862+0000 7f820e800640 1 -- 172.21.15.151:0/3005393290 wait complete. 2024-09-16T15:53:22.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.863+0000 7f820e800640 1 Processor -- start 2024-09-16T15:53:22.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.863+0000 7f820e800640 1 -- start start 2024-09-16T15:53:22.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.864+0000 7f820e800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f820806e590 0x7f82081f6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:22.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.864+0000 7f820e800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8208070550 0x7f82081f2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:22.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.864+0000 7f820e800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f820815bda0 0x7f8208151d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:22.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.864+0000 7f820e800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f82080cba90 con 0x7f820815bda0 2024-09-16T15:53:22.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.864+0000 7f820e800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f82080cb910 con 0x7f820806e590 2024-09-16T15:53:22.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.864+0000 7f820e800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f82080cbc10 con 0x7f8208070550 2024-09-16T15:53:22.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.865+0000 7f820de00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f820815bda0 0x7f8208151d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:22.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.865+0000 7f820de00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f820815bda0 0x7f8208151d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41898/0 (socket says 172.21.15.151:41898) 2024-09-16T15:53:22.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.865+0000 7f820de00640 1 -- 172.21.15.151:0/2412817661 learned_addr learned my addr 172.21.15.151:0/2412817661 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:53:22.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.865+0000 7f820d400640 1 --2- 172.21.15.151:0/2412817661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f820806e590 0x7f82081f6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:22.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.865+0000 7f820ca00640 1 --2- 172.21.15.151:0/2412817661 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8208070550 0x7f82081f2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:22.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.865+0000 7f820de00640 1 -- 172.21.15.151:0/2412817661 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8208070550 msgr2=0x7f82081f2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:22.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.865+0000 7f820de00640 1 --2- 172.21.15.151:0/2412817661 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8208070550 0x7f82081f2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:22.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.865+0000 7f820de00640 1 -- 172.21.15.151:0/2412817661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f820806e590 msgr2=0x7f82081f6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:22.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.865+0000 7f820de00640 1 --2- 172.21.15.151:0/2412817661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f820806e590 0x7f82081f6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:22.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.865+0000 7f820de00640 1 -- 172.21.15.151:0/2412817661 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82081f2860 con 0x7f820815bda0 2024-09-16T15:53:22.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.865+0000 7f820de00640 1 --2- 172.21.15.151:0/2412817661 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f820815bda0 0x7f8208151d90 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f820000eb10 tx=0x7f820000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:22.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.866+0000 7f8205600640 1 -- 172.21.15.151:0/2412817661 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8200004b20 con 0x7f820815bda0 2024-09-16T15:53:22.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.866+0000 7f820e800640 1 -- 172.21.15.151:0/2412817661 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82081f2ae0 con 0x7f820815bda0 2024-09-16T15:53:22.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.866+0000 7f8205600640 1 -- 172.21.15.151:0/2412817661 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8200004ce0 con 0x7f820815bda0 2024-09-16T15:53:22.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.866+0000 7f8205600640 1 -- 172.21.15.151:0/2412817661 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8200005280 con 0x7f820815bda0 2024-09-16T15:53:22.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.866+0000 7f820e800640 1 -- 172.21.15.151:0/2412817661 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82081f30a0 con 0x7f820815bda0 2024-09-16T15:53:22.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.867+0000 7f820e800640 1 -- 172.21.15.151:0/2412817661 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82080cd320 con 0x7f820815bda0 2024-09-16T15:53:22.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.868+0000 7f8205600640 1 -- 172.21.15.151:0/2412817661 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 47) ==== 104092+0+0 (secure 0 0 0) 0x7f8200005420 con 0x7f820815bda0 2024-09-16T15:53:22.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.869+0000 7f8205600640 1 --2- 172.21.15.151:0/2412817661 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f81e807a740 0x7f81e807cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:22.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.869+0000 7f8205600640 1 -- 172.21.15.151:0/2412817661 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5851+0+0 (secure 0 0 0) 0x7f820009a5a0 con 0x7f820815bda0 2024-09-16T15:53:22.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.869+0000 7f820d400640 1 --2- 172.21.15.151:0/2412817661 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f81e807a740 0x7f81e807cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:22.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.870+0000 7f820d400640 1 --2- 172.21.15.151:0/2412817661 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f81e807a740 0x7f81e807cc00 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f81fc034000 tx=0x7f81fc004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:22.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:22.873+0000 7f8205600640 1 -- 172.21.15.151:0/2412817661 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f82081f2860 con 0x7f820815bda0 2024-09-16T15:53:23.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:23.083+0000 7f820e800640 1 -- 172.21.15.151:0/2412817661 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone76352", "target": ["mon-mgr", ""]}) -- 0x7f82080af590 con 0x7f81e807a740 2024-09-16T15:53:23.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:23.102+0000 7f8205600640 1 -- 172.21.15.151:0/2412817661 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f82080af590 con 0x7f81e807a740 2024-09-16T15:53:23.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:23.104+0000 7f820e800640 1 -- 172.21.15.151:0/2412817661 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f81e807a740 msgr2=0x7f81e807cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:23.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:23.104+0000 7f820e800640 1 --2- 172.21.15.151:0/2412817661 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f81e807a740 0x7f81e807cc00 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f81fc034000 tx=0x7f81fc004730 comp rx=0 tx=0).stop 2024-09-16T15:53:23.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:23.104+0000 7f820e800640 1 -- 172.21.15.151:0/2412817661 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f820815bda0 msgr2=0x7f8208151d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:23.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:23.104+0000 7f820e800640 1 --2- 172.21.15.151:0/2412817661 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f820815bda0 0x7f8208151d90 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f820000eb10 tx=0x7f820000c2d0 comp rx=0 tx=0).stop 2024-09-16T15:53:23.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:23.105+0000 7f820e800640 1 -- 172.21.15.151:0/2412817661 shutdown_connections 2024-09-16T15:53:23.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:23.105+0000 7f820e800640 1 --2- 172.21.15.151:0/2412817661 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f81e807a740 0x7f81e807cc00 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:23.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:23.105+0000 7f820e800640 1 --2- 172.21.15.151:0/2412817661 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f820815bda0 0x7f8208151d90 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:23.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:23.105+0000 7f820e800640 1 --2- 172.21.15.151:0/2412817661 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8208070550 0x7f82081f2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:23.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:23.105+0000 7f820e800640 1 --2- 172.21.15.151:0/2412817661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f820806e590 0x7f82081f6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:23.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:23.105+0000 7f820e800640 1 -- 172.21.15.151:0/2412817661 >> 172.21.15.151:0/2412817661 conn(0x7f82080cfa70 msgr2=0x7f82080b2b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:23.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:23.105+0000 7f820e800640 1 -- 172.21.15.151:0/2412817661 shutdown_connections 2024-09-16T15:53:23.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:23.105+0000 7f820e800640 1 -- 172.21.15.151:0/2412817661 wait complete. 2024-09-16T15:53:23.123 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-16T15:53:23.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:23 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:53:23.209 INFO:teuthology.orchestra.run.smithi151.stdout:2 2024-09-16T15:53:23.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:53:23.218 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-16T15:53:23.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:23.378+0000 7faf78000640 1 -- 172.21.15.151:0/3110921488 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf7006efa0 msgr2=0x7faf70073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:23.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:23.378+0000 7faf78000640 1 --2- 172.21.15.151:0/3110921488 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf7006efa0 0x7faf70073440 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7faf68009830 tx=0x7faf6802a530 comp rx=0 tx=0).stop 2024-09-16T15:53:23.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:23.378+0000 7faf78000640 1 -- 172.21.15.151:0/3110921488 shutdown_connections 2024-09-16T15:53:23.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:23.378+0000 7faf78000640 1 --2- 172.21.15.151:0/3110921488 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf7006efa0 0x7faf70073440 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:23.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:23.378+0000 7faf78000640 1 -- 172.21.15.151:0/3110921488 >> 172.21.15.151:0/3110921488 conn(0x7faf700b1840 msgr2=0x7faf700b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:23.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:23.378+0000 7faf78000640 1 -- 172.21.15.151:0/3110921488 shutdown_connections 2024-09-16T15:53:23.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:23.378+0000 7faf78000640 1 -- 172.21.15.151:0/3110921488 wait complete. 2024-09-16T15:53:23.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:23.380+0000 7faf78000640 1 Processor -- start 2024-09-16T15:53:23.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:23.380+0000 7faf78000640 1 -- start start 2024-09-16T15:53:23.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:23.381+0000 7faf78000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf7006efa0 0x7faf7013f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:23.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:23.381+0000 7faf78000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf700cccd0 0x7faf7013fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:23.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:23.381+0000 7faf78000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf70145750 0x7faf7013df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:23.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:23.381+0000 7faf78000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7faf700ce6a0 con 0x7faf70145750 2024-09-16T15:53:23.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:23.381+0000 7faf78000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7faf700ce520 con 0x7faf7006efa0 2024-09-16T15:53:23.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:23.381+0000 7faf78000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7faf700ce820 con 0x7faf700cccd0 2024-09-16T15:53:23.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:23.381+0000 7faf76000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf70145750 0x7faf7013df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:23.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:23.381+0000 7faf76000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf70145750 0x7faf7013df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41918/0 (socket says 172.21.15.151:41918) 2024-09-16T15:53:23.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:23.381+0000 7faf76000640 1 -- 172.21.15.151:0/1471493774 learned_addr learned my addr 172.21.15.151:0/1471493774 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:53:23.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:23.381+0000 7faf75600640 1 --2- 172.21.15.151:0/1471493774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf7006efa0 0x7faf7013f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:23.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:23.381+0000 7faf74c00640 1 --2- 172.21.15.151:0/1471493774 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf700cccd0 0x7faf7013fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:23.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:23.382+0000 7faf76000640 1 -- 172.21.15.151:0/1471493774 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf700cccd0 msgr2=0x7faf7013fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:23.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:23.382+0000 7faf76000640 1 --2- 172.21.15.151:0/1471493774 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf700cccd0 0x7faf7013fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:23.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:23.382+0000 7faf76000640 1 -- 172.21.15.151:0/1471493774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf7006efa0 msgr2=0x7faf7013f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:23.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:23.382+0000 7faf76000640 1 --2- 172.21.15.151:0/1471493774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf7006efa0 0x7faf7013f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:23.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:23.382+0000 7faf76000640 1 -- 172.21.15.151:0/1471493774 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf700ab960 con 0x7faf70145750 2024-09-16T15:53:23.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:23.382+0000 7faf76000640 1 --2- 172.21.15.151:0/1471493774 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf70145750 0x7faf7013df30 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7faf68009950 tx=0x7faf68032cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:23.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:23.382+0000 7faf6d600640 1 -- 172.21.15.151:0/1471493774 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf68040070 con 0x7faf70145750 2024-09-16T15:53:23.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:23.382+0000 7faf6d600640 1 -- 172.21.15.151:0/1471493774 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faf68005310 con 0x7faf70145750 2024-09-16T15:53:23.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:23.382+0000 7faf78000640 1 -- 172.21.15.151:0/1471493774 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf700abc40 con 0x7faf70145750 2024-09-16T15:53:23.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:23.382+0000 7faf6d600640 1 -- 172.21.15.151:0/1471493774 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf68003800 con 0x7faf70145750 2024-09-16T15:53:23.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:23.383+0000 7faf78000640 1 -- 172.21.15.151:0/1471493774 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf700abf20 con 0x7faf70145750 2024-09-16T15:53:23.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:23.384+0000 7faf78000640 1 -- 172.21.15.151:0/1471493774 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf700cecd0 con 0x7faf70145750 2024-09-16T15:53:23.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:23.385+0000 7faf6d600640 1 -- 172.21.15.151:0/1471493774 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 47) ==== 104092+0+0 (secure 0 0 0) 0x7faf68039040 con 0x7faf70145750 2024-09-16T15:53:23.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:23.386+0000 7faf6d600640 1 --2- 172.21.15.151:0/1471493774 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7faf5007a740 0x7faf5007cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:23.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:23.386+0000 7faf6d600640 1 -- 172.21.15.151:0/1471493774 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5851+0+0 (secure 0 0 0) 0x7faf680baab0 con 0x7faf70145750 2024-09-16T15:53:23.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:23.386+0000 7faf75600640 1 --2- 172.21.15.151:0/1471493774 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7faf5007a740 0x7faf5007cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:23.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:23.387+0000 7faf75600640 1 --2- 172.21.15.151:0/1471493774 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7faf5007a740 0x7faf5007cc00 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7faf64001e20 tx=0x7faf64009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:23.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:23.390+0000 7faf6d600640 1 -- 172.21.15.151:0/1471493774 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7faf680c6080 con 0x7faf70145750 2024-09-16T15:53:23.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:23.574+0000 7faf78000640 1 -- 172.21.15.151:0/1471493774 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7faf700b0cf0 con 0x7faf70145750 2024-09-16T15:53:23.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:23.575+0000 7faf6d600640 1 -- 172.21.15.151:0/1471493774 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v49)=0 v49) ==== 131+0+0 (secure 0 0 0) 0x7faf680845d0 con 0x7faf70145750 2024-09-16T15:53:23.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:23.577+0000 7faf78000640 1 -- 172.21.15.151:0/1471493774 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7faf5007a740 msgr2=0x7faf5007cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:23.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:23.577+0000 7faf78000640 1 --2- 172.21.15.151:0/1471493774 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7faf5007a740 0x7faf5007cc00 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7faf64001e20 tx=0x7faf64009040 comp rx=0 tx=0).stop 2024-09-16T15:53:23.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:23.577+0000 7faf78000640 1 -- 172.21.15.151:0/1471493774 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf70145750 msgr2=0x7faf7013df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:23.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:23.577+0000 7faf78000640 1 --2- 172.21.15.151:0/1471493774 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf70145750 0x7faf7013df30 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7faf68009950 tx=0x7faf68032cd0 comp rx=0 tx=0).stop 2024-09-16T15:53:23.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:23.577+0000 7faf78000640 1 -- 172.21.15.151:0/1471493774 shutdown_connections 2024-09-16T15:53:23.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:23.577+0000 7faf78000640 1 --2- 172.21.15.151:0/1471493774 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7faf5007a740 0x7faf5007cc00 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:23.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:23.577+0000 7faf78000640 1 --2- 172.21.15.151:0/1471493774 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf70145750 0x7faf7013df30 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:23.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:23.577+0000 7faf78000640 1 --2- 172.21.15.151:0/1471493774 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faf700cccd0 0x7faf7013fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:23.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:23.577+0000 7faf78000640 1 --2- 172.21.15.151:0/1471493774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf7006efa0 0x7faf7013f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:23.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:23.577+0000 7faf78000640 1 -- 172.21.15.151:0/1471493774 >> 172.21.15.151:0/1471493774 conn(0x7faf700b1840 msgr2=0x7faf700cfd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:23.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:23.577+0000 7faf78000640 1 -- 172.21.15.151:0/1471493774 shutdown_connections 2024-09-16T15:53:23.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:23.577+0000 7faf78000640 1 -- 172.21.15.151:0/1471493774 wait complete. 2024-09-16T15:53:23.593 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-16T15:53:23.594 DEBUG:teuthology.orchestra.run.smithi151:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:53:23.695 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:53:23.781 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-16T15:53:23.781 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:53:23.782 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:53:23.782 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:53:23.817 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:53:23.818 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:53:23.893 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-16T15:53:24.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.092+0000 7f6267400640 1 -- 172.21.15.151:0/4039152127 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f62600ccf00 msgr2=0x7f62600b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:24.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.092+0000 7f6267400640 1 --2- 172.21.15.151:0/4039152127 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f62600ccf00 0x7f62600b2440 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f624c009830 tx=0x7f624c02a480 comp rx=0 tx=0).stop 2024-09-16T15:53:24.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.092+0000 7f6267400640 1 -- 172.21.15.151:0/4039152127 shutdown_connections 2024-09-16T15:53:24.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.092+0000 7f6267400640 1 --2- 172.21.15.151:0/4039152127 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f62600ccf00 0x7f62600b2440 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:24.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.092+0000 7f6267400640 1 -- 172.21.15.151:0/4039152127 >> 172.21.15.151:0/4039152127 conn(0x7f62600b1950 msgr2=0x7f62600b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:24.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.092+0000 7f6267400640 1 -- 172.21.15.151:0/4039152127 shutdown_connections 2024-09-16T15:53:24.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.092+0000 7f6267400640 1 -- 172.21.15.151:0/4039152127 wait complete. 2024-09-16T15:53:24.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.094+0000 7f6267400640 1 Processor -- start 2024-09-16T15:53:24.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.094+0000 7f6267400640 1 -- start start 2024-09-16T15:53:24.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.095+0000 7f6267400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6260073fb0 0x7f626015a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:24.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.095+0000 7f6267400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f62600ccf00 0x7f6260155300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:24.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.095+0000 7f6267400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f626015f5a0 0x7f62600cb030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:24.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.095+0000 7f6267400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f626006f9f0 con 0x7f62600ccf00 2024-09-16T15:53:24.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.095+0000 7f6267400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f626006f870 con 0x7f626015f5a0 2024-09-16T15:53:24.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.095+0000 7f6267400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f6260074420 con 0x7f6260073fb0 2024-09-16T15:53:24.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.095+0000 7f625fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f62600ccf00 0x7f6260155300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:24.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.095+0000 7f625fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f62600ccf00 0x7f6260155300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41934/0 (socket says 172.21.15.151:41934) 2024-09-16T15:53:24.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.095+0000 7f625fe00640 1 -- 172.21.15.151:0/2779203509 learned_addr learned my addr 172.21.15.151:0/2779203509 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:53:24.098 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.096+0000 7f6264a00640 1 --2- 172.21.15.151:0/2779203509 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6260073fb0 0x7f626015a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:24.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.096+0000 7f6265400640 1 --2- 172.21.15.151:0/2779203509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f626015f5a0 0x7f62600cb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:24.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.096+0000 7f625fe00640 1 -- 172.21.15.151:0/2779203509 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6260073fb0 msgr2=0x7f626015a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:24.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.096+0000 7f625fe00640 1 --2- 172.21.15.151:0/2779203509 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6260073fb0 0x7f626015a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:24.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.096+0000 7f625fe00640 1 -- 172.21.15.151:0/2779203509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f626015f5a0 msgr2=0x7f62600cb030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:24.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.096+0000 7f625fe00640 1 --2- 172.21.15.151:0/2779203509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f626015f5a0 0x7f62600cb030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:24.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.096+0000 7f625fe00640 1 -- 172.21.15.151:0/2779203509 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f626015abe0 con 0x7f62600ccf00 2024-09-16T15:53:24.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.096+0000 7f6264a00640 1 --2- 172.21.15.151:0/2779203509 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6260073fb0 0x7f626015a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:53:24.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.096+0000 7f6265400640 1 --2- 172.21.15.151:0/2779203509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f626015f5a0 0x7f62600cb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:53:24.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.096+0000 7f625fe00640 1 --2- 172.21.15.151:0/2779203509 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f62600ccf00 0x7f6260155300 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f624c009950 tx=0x7f624c032d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:24.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.096+0000 7f625cc00640 1 -- 172.21.15.151:0/2779203509 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f624c040070 con 0x7f62600ccf00 2024-09-16T15:53:24.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.096+0000 7f625cc00640 1 -- 172.21.15.151:0/2779203509 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f624c003af0 con 0x7f62600ccf00 2024-09-16T15:53:24.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.096+0000 7f625cc00640 1 -- 172.21.15.151:0/2779203509 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f624c045460 con 0x7f62600ccf00 2024-09-16T15:53:24.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.096+0000 7f6267400640 1 -- 172.21.15.151:0/2779203509 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f626015aec0 con 0x7f62600ccf00 2024-09-16T15:53:24.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.097+0000 7f6267400640 1 -- 172.21.15.151:0/2779203509 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f626015b480 con 0x7f62600ccf00 2024-09-16T15:53:24.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.098+0000 7f6267400640 1 -- 172.21.15.151:0/2779203509 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f626015b7b0 con 0x7f62600ccf00 2024-09-16T15:53:24.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.099+0000 7f625cc00640 1 -- 172.21.15.151:0/2779203509 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 47) ==== 104092+0+0 (secure 0 0 0) 0x7f624c039040 con 0x7f62600ccf00 2024-09-16T15:53:24.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.100+0000 7f625cc00640 1 --2- 172.21.15.151:0/2779203509 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f624007a770 0x7f624007cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:24.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.100+0000 7f625cc00640 1 -- 172.21.15.151:0/2779203509 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5907+0+0 (secure 0 0 0) 0x7f624c0bb040 con 0x7f62600ccf00 2024-09-16T15:53:24.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.100+0000 7f6264a00640 1 --2- 172.21.15.151:0/2779203509 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f624007a770 0x7f624007cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:24.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.101+0000 7f6264a00640 1 --2- 172.21.15.151:0/2779203509 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f624007a770 0x7f624007cc30 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f62500012e0 tx=0x7f6250009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:24.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.104+0000 7f625cc00640 1 -- 172.21.15.151:0/2779203509 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f624c085110 con 0x7f62600ccf00 2024-09-16T15:53:24.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.456+0000 7f6267400640 1 -- 172.21.15.151:0/2779203509 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f62600b2440 con 0x7f62600ccf00 2024-09-16T15:53:24.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.456+0000 7f625cc00640 1 -- 172.21.15.151:0/2779203509 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v121) ==== 80+0+0 (secure 0 0 0) 0x7f624c084a10 con 0x7f62600ccf00 2024-09-16T15:53:24.458 INFO:teuthology.orchestra.run.smithi151.stderr:listed 0 entries 2024-09-16T15:53:24.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.459+0000 7f6267400640 1 -- 172.21.15.151:0/2779203509 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f624007a770 msgr2=0x7f624007cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:24.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.459+0000 7f6267400640 1 --2- 172.21.15.151:0/2779203509 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f624007a770 0x7f624007cc30 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f62500012e0 tx=0x7f6250009040 comp rx=0 tx=0).stop 2024-09-16T15:53:24.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.459+0000 7f6267400640 1 -- 172.21.15.151:0/2779203509 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f62600ccf00 msgr2=0x7f6260155300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:24.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.459+0000 7f6267400640 1 --2- 172.21.15.151:0/2779203509 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f62600ccf00 0x7f6260155300 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f624c009950 tx=0x7f624c032d50 comp rx=0 tx=0).stop 2024-09-16T15:53:24.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.459+0000 7f6267400640 1 -- 172.21.15.151:0/2779203509 shutdown_connections 2024-09-16T15:53:24.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.459+0000 7f6267400640 1 --2- 172.21.15.151:0/2779203509 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f624007a770 0x7f624007cc30 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:24.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.459+0000 7f6267400640 1 --2- 172.21.15.151:0/2779203509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f626015f5a0 0x7f62600cb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:24.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.459+0000 7f6267400640 1 --2- 172.21.15.151:0/2779203509 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f62600ccf00 0x7f6260155300 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:24.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.459+0000 7f6267400640 1 --2- 172.21.15.151:0/2779203509 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6260073fb0 0x7f626015a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:24.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.459+0000 7f6267400640 1 -- 172.21.15.151:0/2779203509 >> 172.21.15.151:0/2779203509 conn(0x7f62600b1950 msgr2=0x7f62600af5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:24.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.459+0000 7f6267400640 1 -- 172.21.15.151:0/2779203509 shutdown_connections 2024-09-16T15:53:24.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.459+0000 7f6267400640 1 -- 172.21.15.151:0/2779203509 wait complete. 2024-09-16T15:53:24.486 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-16T15:53:24.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:53:24.486 DEBUG:teuthology.orchestra.run.smithi151:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-16T15:53:24.619 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi151.front.sepia.ceph.com 2024-09-16T15:53:24.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:53:24.619 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-16T15:53:24.688 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:53:24.688 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-16T15:53:24.689 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:53:24.741 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:53:24.741 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:53:24.742 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:53:24.742 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:53:24.779 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T15:53:24.779 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:53:24.780 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-16T15:53:24.780 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:53:24.816 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:53:24.816 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:53:24.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.975+0000 7f0c31200640 1 -- 172.21.15.151:0/2608320432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c2c073fb0 msgr2=0x7f0c2c074460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:24.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.975+0000 7f0c31200640 1 --2- 172.21.15.151:0/2608320432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c2c073fb0 0x7f0c2c074460 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f0c24005fd0 tx=0x7f0c2402a820 comp rx=0 tx=0).stop 2024-09-16T15:53:24.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.976+0000 7f0c31200640 1 -- 172.21.15.151:0/2608320432 shutdown_connections 2024-09-16T15:53:24.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.976+0000 7f0c31200640 1 --2- 172.21.15.151:0/2608320432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c2c073fb0 0x7f0c2c074460 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:24.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.976+0000 7f0c31200640 1 --2- 172.21.15.151:0/2608320432 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c2c15f5a0 0x7f0c2c0b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:24.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.976+0000 7f0c31200640 1 -- 172.21.15.151:0/2608320432 >> 172.21.15.151:0/2608320432 conn(0x7f0c2c0b1950 msgr2=0x7f0c2c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:24.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.976+0000 7f0c31200640 1 -- 172.21.15.151:0/2608320432 shutdown_connections 2024-09-16T15:53:24.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.976+0000 7f0c31200640 1 -- 172.21.15.151:0/2608320432 wait complete. 2024-09-16T15:53:24.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.977+0000 7f0c31200640 1 Processor -- start 2024-09-16T15:53:24.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.977+0000 7f0c31200640 1 -- start start 2024-09-16T15:53:24.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.978+0000 7f0c31200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c2c073fb0 0x7f0c2c14c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:24.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.978+0000 7f0c31200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c2c0ccf00 0x7f0c2c14fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:24.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.978+0000 7f0c31200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c2c15f5a0 0x7f0c2c0cf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:24.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.978+0000 7f0c31200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0c2c06f2e0 con 0x7f0c2c073fb0 2024-09-16T15:53:24.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.978+0000 7f0c31200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0c2c06f160 con 0x7f0c2c15f5a0 2024-09-16T15:53:24.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.978+0000 7f0c31200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f0c2c06f460 con 0x7f0c2c0ccf00 2024-09-16T15:53:24.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.978+0000 7f0c29e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c2c073fb0 0x7f0c2c14c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:24.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.979+0000 7f0c29e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c2c073fb0 0x7f0c2c14c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41936/0 (socket says 172.21.15.151:41936) 2024-09-16T15:53:24.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.979+0000 7f0c29e00640 1 -- 172.21.15.151:0/2881889534 learned_addr learned my addr 172.21.15.151:0/2881889534 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:53:24.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.979+0000 7f0c2a800640 1 --2- 172.21.15.151:0/2881889534 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c2c15f5a0 0x7f0c2c0cf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:24.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.979+0000 7f0c29400640 1 --2- 172.21.15.151:0/2881889534 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c2c0ccf00 0x7f0c2c14fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:24.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.979+0000 7f0c29e00640 1 -- 172.21.15.151:0/2881889534 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c2c0ccf00 msgr2=0x7f0c2c14fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:24.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.979+0000 7f0c29e00640 1 --2- 172.21.15.151:0/2881889534 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c2c0ccf00 0x7f0c2c14fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:24.981 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.979+0000 7f0c29e00640 1 -- 172.21.15.151:0/2881889534 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c2c15f5a0 msgr2=0x7f0c2c0cf4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:24.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.979+0000 7f0c29e00640 1 --2- 172.21.15.151:0/2881889534 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c2c15f5a0 0x7f0c2c0cf4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:24.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.979+0000 7f0c29e00640 1 -- 172.21.15.151:0/2881889534 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c2c1505e0 con 0x7f0c2c073fb0 2024-09-16T15:53:24.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.979+0000 7f0c29e00640 1 --2- 172.21.15.151:0/2881889534 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c2c073fb0 0x7f0c2c14c7a0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c00e9c0 tx=0x7f0c1c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:24.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.979+0000 7f0c22000640 1 -- 172.21.15.151:0/2881889534 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c00cd80 con 0x7f0c2c073fb0 2024-09-16T15:53:24.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.980+0000 7f0c31200640 1 -- 172.21.15.151:0/2881889534 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c2c14bcc0 con 0x7f0c2c073fb0 2024-09-16T15:53:24.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.980+0000 7f0c22000640 1 -- 172.21.15.151:0/2881889534 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0c1c004020 con 0x7f0c2c073fb0 2024-09-16T15:53:24.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.980+0000 7f0c22000640 1 -- 172.21.15.151:0/2881889534 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c004450 con 0x7f0c2c073fb0 2024-09-16T15:53:24.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.980+0000 7f0c31200640 1 -- 172.21.15.151:0/2881889534 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c2c14c280 con 0x7f0c2c073fb0 2024-09-16T15:53:24.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.981+0000 7f0c31200640 1 -- 172.21.15.151:0/2881889534 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c2c06f990 con 0x7f0c2c073fb0 2024-09-16T15:53:24.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.982+0000 7f0c22000640 1 -- 172.21.15.151:0/2881889534 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 47) ==== 104092+0+0 (secure 0 0 0) 0x7f0c1c0035f0 con 0x7f0c2c073fb0 2024-09-16T15:53:24.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.983+0000 7f0c22000640 1 --2- 172.21.15.151:0/2881889534 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0c0c07a740 0x7f0c0c07cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:24.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.983+0000 7f0c22000640 1 -- 172.21.15.151:0/2881889534 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5907+0+0 (secure 0 0 0) 0x7f0c1c09a2f0 con 0x7f0c2c073fb0 2024-09-16T15:53:24.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.983+0000 7f0c29400640 1 --2- 172.21.15.151:0/2881889534 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0c0c07a740 0x7f0c0c07cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:24.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.984+0000 7f0c29400640 1 --2- 172.21.15.151:0/2881889534 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0c0c07a740 0x7f0c0c07cc00 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c070620 tx=0x7f0c18009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:24.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:24.987+0000 7f0c22000640 1 -- 172.21.15.151:0/2881889534 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0c1c064340 con 0x7f0c2c073fb0 2024-09-16T15:53:25.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.260+0000 7f0c31200640 1 -- 172.21.15.151:0/2881889534 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0c2c06acb0 con 0x7f0c2c073fb0 2024-09-16T15:53:25.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.262+0000 7f0c22000640 1 -- 172.21.15.151:0/2881889534 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+5237 (secure 0 0 0) 0x7f0c1c063c40 con 0x7f0c2c073fb0 2024-09-16T15:53:25.263 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:53:25.263 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":77,"btime":"2024-09-16T15:52:31:147854+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7034,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1802374613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1802374613},{"type":"v1","addr":"172.21.15.167:6837","nonce":1802374613}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":7753,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3585191088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3585191088},{"type":"v1","addr":"172.21.15.167:6836","nonce":3585191088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":8457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/1321222227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1321222227},{"type":"v1","addr":"172.21.15.151:6834","nonce":1321222227}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:52:17.489987+0000","modified":"2024-09-16T15:52:20.525094+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8460":{"gid":8460,"name":"a","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.151:6835/2504927912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":2504927912},{"type":"v1","addr":"172.21.15.151:6835","nonce":2504927912}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8460,"qdb_cluster":[8460]},"id":8}]} 2024-09-16T15:53:25.263 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 77 2024-09-16T15:53:25.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.266+0000 7f0c31200640 1 -- 172.21.15.151:0/2881889534 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0c0c07a740 msgr2=0x7f0c0c07cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:25.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.266+0000 7f0c31200640 1 --2- 172.21.15.151:0/2881889534 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0c0c07a740 0x7f0c0c07cc00 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c070620 tx=0x7f0c18009040 comp rx=0 tx=0).stop 2024-09-16T15:53:25.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.266+0000 7f0c31200640 1 -- 172.21.15.151:0/2881889534 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c2c073fb0 msgr2=0x7f0c2c14c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:25.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.266+0000 7f0c31200640 1 --2- 172.21.15.151:0/2881889534 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c2c073fb0 0x7f0c2c14c7a0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c00e9c0 tx=0x7f0c1c00ee80 comp rx=0 tx=0).stop 2024-09-16T15:53:25.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.266+0000 7f0c31200640 1 -- 172.21.15.151:0/2881889534 shutdown_connections 2024-09-16T15:53:25.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.266+0000 7f0c31200640 1 --2- 172.21.15.151:0/2881889534 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0c0c07a740 0x7f0c0c07cc00 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:25.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.266+0000 7f0c31200640 1 --2- 172.21.15.151:0/2881889534 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c2c15f5a0 0x7f0c2c0cf4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:25.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.266+0000 7f0c31200640 1 --2- 172.21.15.151:0/2881889534 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c2c0ccf00 0x7f0c2c14fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:25.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.266+0000 7f0c31200640 1 --2- 172.21.15.151:0/2881889534 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c2c073fb0 0x7f0c2c14c7a0 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:25.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.266+0000 7f0c31200640 1 -- 172.21.15.151:0/2881889534 >> 172.21.15.151:0/2881889534 conn(0x7f0c2c0b1950 msgr2=0x7f0c2c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:25.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.266+0000 7f0c31200640 1 -- 172.21.15.151:0/2881889534 shutdown_connections 2024-09-16T15:53:25.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.266+0000 7f0c31200640 1 -- 172.21.15.151:0/2881889534 wait complete. 2024-09-16T15:53:25.293 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:53:25.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.451+0000 7f2d47a00640 1 -- 172.21.15.151:0/2528989238 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d4015f5a0 msgr2=0x7f2d400b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:25.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.451+0000 7f2d47a00640 1 --2- 172.21.15.151:0/2528989238 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d4015f5a0 0x7f2d400b2440 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2d34005fd0 tx=0x7f2d3402a540 comp rx=0 tx=0).stop 2024-09-16T15:53:25.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.451+0000 7f2d47a00640 1 -- 172.21.15.151:0/2528989238 shutdown_connections 2024-09-16T15:53:25.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.451+0000 7f2d47a00640 1 --2- 172.21.15.151:0/2528989238 >> v2:172.21.15.167:3300/0 conn(0x7f2d40073fb0 0x7f2d40074460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:25.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.451+0000 7f2d47a00640 1 --2- 172.21.15.151:0/2528989238 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d4015f5a0 0x7f2d400b2440 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:25.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.451+0000 7f2d47a00640 1 -- 172.21.15.151:0/2528989238 >> 172.21.15.151:0/2528989238 conn(0x7f2d400b1950 msgr2=0x7f2d400b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:25.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.451+0000 7f2d47a00640 1 -- 172.21.15.151:0/2528989238 shutdown_connections 2024-09-16T15:53:25.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.451+0000 7f2d47a00640 1 -- 172.21.15.151:0/2528989238 wait complete. 2024-09-16T15:53:25.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.453+0000 7f2d47a00640 1 Processor -- start 2024-09-16T15:53:25.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.453+0000 7f2d47a00640 1 -- start start 2024-09-16T15:53:25.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.454+0000 7f2d47a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d40073fb0 0x7f2d4009c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:25.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.454+0000 7f2d47a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d400ccf00 0x7f2d40156090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:25.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.454+0000 7f2d47a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d4015f5a0 0x7f2d401543a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:25.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.454+0000 7f2d47a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2d4006f2e0 con 0x7f2d400ccf00 2024-09-16T15:53:25.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.454+0000 7f2d47a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2d4006f160 con 0x7f2d40073fb0 2024-09-16T15:53:25.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.454+0000 7f2d47a00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f2d4006f460 con 0x7f2d4015f5a0 2024-09-16T15:53:25.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.454+0000 7f2d3fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d400ccf00 0x7f2d40156090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:25.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.454+0000 7f2d3fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d400ccf00 0x7f2d40156090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41948/0 (socket says 172.21.15.151:41948) 2024-09-16T15:53:25.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.454+0000 7f2d3fe00640 1 -- 172.21.15.151:0/1444573030 learned_addr learned my addr 172.21.15.151:0/1444573030 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:53:25.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.454+0000 7f2d45a00640 1 --2- 172.21.15.151:0/1444573030 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d4015f5a0 0x7f2d401543a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:25.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.454+0000 7f2d45000640 1 --2- 172.21.15.151:0/1444573030 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d40073fb0 0x7f2d4009c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:25.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.455+0000 7f2d3fe00640 1 -- 172.21.15.151:0/1444573030 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d4015f5a0 msgr2=0x7f2d401543a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:25.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.455+0000 7f2d3fe00640 1 --2- 172.21.15.151:0/1444573030 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d4015f5a0 0x7f2d401543a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:25.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.455+0000 7f2d3fe00640 1 -- 172.21.15.151:0/1444573030 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d40073fb0 msgr2=0x7f2d4009c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:25.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.455+0000 7f2d3fe00640 1 --2- 172.21.15.151:0/1444573030 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d40073fb0 0x7f2d4009c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:25.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.455+0000 7f2d3fe00640 1 -- 172.21.15.151:0/1444573030 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d400cb240 con 0x7f2d400ccf00 2024-09-16T15:53:25.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.455+0000 7f2d3fe00640 1 --2- 172.21.15.151:0/1444573030 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d400ccf00 0x7f2d40156090 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c00c420 tx=0x7f2d2c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:25.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.455+0000 7f2d3cc00640 1 -- 172.21.15.151:0/1444573030 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2c017070 con 0x7f2d400ccf00 2024-09-16T15:53:25.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.455+0000 7f2d3cc00640 1 -- 172.21.15.151:0/1444573030 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2d2c0053c0 con 0x7f2d400ccf00 2024-09-16T15:53:25.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.455+0000 7f2d3cc00640 1 -- 172.21.15.151:0/1444573030 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2c003850 con 0x7f2d400ccf00 2024-09-16T15:53:25.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.455+0000 7f2d47a00640 1 -- 172.21.15.151:0/1444573030 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d400cb520 con 0x7f2d400ccf00 2024-09-16T15:53:25.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.455+0000 7f2d47a00640 1 -- 172.21.15.151:0/1444573030 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d400cbae0 con 0x7f2d400ccf00 2024-09-16T15:53:25.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.456+0000 7f2d47a00640 1 -- 172.21.15.151:0/1444573030 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d4006f9f0 con 0x7f2d400ccf00 2024-09-16T15:53:25.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.458+0000 7f2d3cc00640 1 -- 172.21.15.151:0/1444573030 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 47) ==== 104092+0+0 (secure 0 0 0) 0x7f2d2c01a070 con 0x7f2d400ccf00 2024-09-16T15:53:25.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.459+0000 7f2d3cc00640 1 --2- 172.21.15.151:0/1444573030 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2d2007a770 0x7f2d2007cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:25.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.459+0000 7f2d3cc00640 1 -- 172.21.15.151:0/1444573030 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5907+0+0 (secure 0 0 0) 0x7f2d2c098bc0 con 0x7f2d400ccf00 2024-09-16T15:53:25.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.459+0000 7f2d45000640 1 --2- 172.21.15.151:0/1444573030 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2d2007a770 0x7f2d2007cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:25.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.460+0000 7f2d45000640 1 --2- 172.21.15.151:0/1444573030 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2d2007a770 0x7f2d2007cc30 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f2d34033000 tx=0x7f2d34004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:25.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.463+0000 7f2d3cc00640 1 -- 172.21.15.151:0/1444573030 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2d2c0a1030 con 0x7f2d400ccf00 2024-09-16T15:53:25.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.686+0000 7f2d47a00640 1 -- 172.21.15.151:0/1444573030 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2d40069ac0 con 0x7f2d400ccf00 2024-09-16T15:53:25.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.687+0000 7f2d3cc00640 1 -- 172.21.15.151:0/1444573030 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+5237 (secure 0 0 0) 0x7f2d2c0626e0 con 0x7f2d400ccf00 2024-09-16T15:53:25.688 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:53:25.688 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":77,"btime":"2024-09-16T15:52:31:147854+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7034,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1802374613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1802374613},{"type":"v1","addr":"172.21.15.167:6837","nonce":1802374613}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":7753,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3585191088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3585191088},{"type":"v1","addr":"172.21.15.167:6836","nonce":3585191088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":8457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/1321222227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1321222227},{"type":"v1","addr":"172.21.15.151:6834","nonce":1321222227}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:52:17.489987+0000","modified":"2024-09-16T15:52:20.525094+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8460":{"gid":8460,"name":"a","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.151:6835/2504927912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":2504927912},{"type":"v1","addr":"172.21.15.151:6835","nonce":2504927912}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8460,"qdb_cluster":[8460]},"id":8}]} 2024-09-16T15:53:25.688 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 77 2024-09-16T15:53:25.690 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.689+0000 7f2d47a00640 1 -- 172.21.15.151:0/1444573030 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2d2007a770 msgr2=0x7f2d2007cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:25.690 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.689+0000 7f2d47a00640 1 --2- 172.21.15.151:0/1444573030 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2d2007a770 0x7f2d2007cc30 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f2d34033000 tx=0x7f2d34004730 comp rx=0 tx=0).stop 2024-09-16T15:53:25.690 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.689+0000 7f2d47a00640 1 -- 172.21.15.151:0/1444573030 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d400ccf00 msgr2=0x7f2d40156090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:25.690 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.689+0000 7f2d47a00640 1 --2- 172.21.15.151:0/1444573030 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d400ccf00 0x7f2d40156090 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c00c420 tx=0x7f2d2c00c8e0 comp rx=0 tx=0).stop 2024-09-16T15:53:25.690 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.689+0000 7f2d47a00640 1 -- 172.21.15.151:0/1444573030 shutdown_connections 2024-09-16T15:53:25.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.689+0000 7f2d47a00640 1 --2- 172.21.15.151:0/1444573030 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2d2007a770 0x7f2d2007cc30 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:25.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.689+0000 7f2d47a00640 1 --2- 172.21.15.151:0/1444573030 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d4015f5a0 0x7f2d401543a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:25.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.689+0000 7f2d47a00640 1 --2- 172.21.15.151:0/1444573030 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d400ccf00 0x7f2d40156090 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:25.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.689+0000 7f2d47a00640 1 --2- 172.21.15.151:0/1444573030 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d40073fb0 0x7f2d4009c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:25.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.689+0000 7f2d47a00640 1 -- 172.21.15.151:0/1444573030 >> 172.21.15.151:0/1444573030 conn(0x7f2d400b1950 msgr2=0x7f2d400d0170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:25.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.689+0000 7f2d47a00640 1 -- 172.21.15.151:0/1444573030 shutdown_connections 2024-09-16T15:53:25.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.689+0000 7f2d47a00640 1 -- 172.21.15.151:0/1444573030 wait complete. 2024-09-16T15:53:25.706 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:53:25.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.862+0000 7efddc000640 1 -- 172.21.15.151:0/2689914894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efdd40cf300 msgr2=0x7efdd40729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:25.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.862+0000 7efddc000640 1 --2- 172.21.15.151:0/2689914894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efdd40cf300 0x7efdd40729a0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7efdc0009860 tx=0x7efdc002a5f0 comp rx=0 tx=0).stop 2024-09-16T15:53:25.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.862+0000 7efddc000640 1 -- 172.21.15.151:0/2689914894 shutdown_connections 2024-09-16T15:53:25.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.862+0000 7efddc000640 1 --2- 172.21.15.151:0/2689914894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efdd40cf300 0x7efdd40729a0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:25.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.862+0000 7efddc000640 1 -- 172.21.15.151:0/2689914894 >> 172.21.15.151:0/2689914894 conn(0x7efdd40b1950 msgr2=0x7efdd40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:25.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.862+0000 7efddc000640 1 -- 172.21.15.151:0/2689914894 shutdown_connections 2024-09-16T15:53:25.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.862+0000 7efddc000640 1 -- 172.21.15.151:0/2689914894 wait complete. 2024-09-16T15:53:25.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.864+0000 7efddc000640 1 Processor -- start 2024-09-16T15:53:25.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.864+0000 7efddc000640 1 -- start start 2024-09-16T15:53:25.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.865+0000 7efddc000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efdd40ccfd0 0x7efdd41e9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:25.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.865+0000 7efddc000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efdd40cf300 0x7efdd41401f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:25.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.865+0000 7efddc000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efdd41445d0 0x7efdd40b08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:25.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.865+0000 7efddc000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efdd40735f0 con 0x7efdd40cf300 2024-09-16T15:53:25.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.865+0000 7efddc000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efdd40b4f60 con 0x7efdd41445d0 2024-09-16T15:53:25.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.865+0000 7efddc000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7efdd4073770 con 0x7efdd40ccfd0 2024-09-16T15:53:25.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.865+0000 7efdd8c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efdd40cf300 0x7efdd41401f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:25.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.865+0000 7efdd8c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efdd40cf300 0x7efdd41401f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41954/0 (socket says 172.21.15.151:41954) 2024-09-16T15:53:25.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.865+0000 7efdd8c00640 1 -- 172.21.15.151:0/4047348543 learned_addr learned my addr 172.21.15.151:0/4047348543 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:53:25.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.865+0000 7efdda000640 1 --2- 172.21.15.151:0/4047348543 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efdd41445d0 0x7efdd40b08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:25.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.865+0000 7efdd9600640 1 --2- 172.21.15.151:0/4047348543 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efdd40ccfd0 0x7efdd41e9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:25.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.866+0000 7efdd8c00640 1 -- 172.21.15.151:0/4047348543 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efdd40ccfd0 msgr2=0x7efdd41e9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:25.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.866+0000 7efdd8c00640 1 --2- 172.21.15.151:0/4047348543 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efdd40ccfd0 0x7efdd41e9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:25.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.866+0000 7efdd8c00640 1 -- 172.21.15.151:0/4047348543 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efdd41445d0 msgr2=0x7efdd40b08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:25.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.866+0000 7efdd8c00640 1 --2- 172.21.15.151:0/4047348543 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efdd41445d0 0x7efdd40b08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:25.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.866+0000 7efdd8c00640 1 -- 172.21.15.151:0/4047348543 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdd4141fa0 con 0x7efdd40cf300 2024-09-16T15:53:25.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.866+0000 7efdd8c00640 1 --2- 172.21.15.151:0/4047348543 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efdd40cf300 0x7efdd41401f0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7efdc0009980 tx=0x7efdc0032b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:25.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.866+0000 7efdd1600640 1 -- 172.21.15.151:0/4047348543 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc0040070 con 0x7efdd40cf300 2024-09-16T15:53:25.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.866+0000 7efdd1600640 1 -- 172.21.15.151:0/4047348543 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efdc0005310 con 0x7efdd40cf300 2024-09-16T15:53:25.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.866+0000 7efdd1600640 1 -- 172.21.15.151:0/4047348543 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc00047a0 con 0x7efdd40cf300 2024-09-16T15:53:25.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.866+0000 7efddc000640 1 -- 172.21.15.151:0/4047348543 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdd4140730 con 0x7efdd40cf300 2024-09-16T15:53:25.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.867+0000 7efddc000640 1 -- 172.21.15.151:0/4047348543 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdd413be90 con 0x7efdd40cf300 2024-09-16T15:53:25.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.868+0000 7efddc000640 1 -- 172.21.15.151:0/4047348543 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdd4073c20 con 0x7efdd40cf300 2024-09-16T15:53:25.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.869+0000 7efdd1600640 1 -- 172.21.15.151:0/4047348543 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 47) ==== 104092+0+0 (secure 0 0 0) 0x7efdc0039040 con 0x7efdd40cf300 2024-09-16T15:53:25.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.870+0000 7efdd1600640 1 --2- 172.21.15.151:0/4047348543 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efdb407a740 0x7efdb407cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:25.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.870+0000 7efdd1600640 1 -- 172.21.15.151:0/4047348543 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5907+0+0 (secure 0 0 0) 0x7efdc00ba880 con 0x7efdd40cf300 2024-09-16T15:53:25.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.870+0000 7efdd9600640 1 --2- 172.21.15.151:0/4047348543 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efdb407a740 0x7efdb407cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:25.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.871+0000 7efdd9600640 1 --2- 172.21.15.151:0/4047348543 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efdb407a740 0x7efdb407cc00 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7efdc80046b0 tx=0x7efdc8007cc0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:25.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:25.874+0000 7efdd1600640 1 -- 172.21.15.151:0/4047348543 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7efdc00843a0 con 0x7efdd40cf300 2024-09-16T15:53:26.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.059+0000 7efddc000640 1 -- 172.21.15.151:0/4047348543 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efdd4143250 con 0x7efdd40cf300 2024-09-16T15:53:26.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.060+0000 7efdd1600640 1 -- 172.21.15.151:0/4047348543 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) ==== 74+0+17133 (secure 0 0 0) 0x7efdc0088dd0 con 0x7efdd40cf300 2024-09-16T15:53:26.061 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:53:26.062 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":121,"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","created":"2024-09-16T15:38:56.348535+0000","modified":"2024-09-16T15:53:23.762259+0000","last_up_change":"2024-09-16T15:39:10.681384+0000","last_in_change":"2024-09-16T15:39:00.544477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:39:11.602252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:52:15.256011+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-16T15:52:16.456620+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":121,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5524076-a903-4d0a-af6a-caf474c4765d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6809","nonce":3111342741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6811","nonce":3111342741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6815","nonce":3111342741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6813","nonce":3111342741}]},"public_addr":"172.21.15.151:6809/3111342741","cluster_addr":"172.21.15.151:6811/3111342741","heartbeat_back_addr":"172.21.15.151:6815/3111342741","heartbeat_front_addr":"172.21.15.151:6813/3111342741","state":["exists","up"]},{"osd":1,"uuid":"dd25925f-0ae1-494e-8529-4dc205759db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6801","nonce":3368304087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6803","nonce":3368304087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6807","nonce":3368304087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6805","nonce":3368304087}]},"public_addr":"172.21.15.151:6801/3368304087","cluster_addr":"172.21.15.151:6803/3368304087","heartbeat_back_addr":"172.21.15.151:6807/3368304087","heartbeat_front_addr":"172.21.15.151:6805/3368304087","state":["exists","up"]},{"osd":2,"uuid":"c2f18115-013d-4da6-a0fe-cb978f4494b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6823","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6825","nonce":3079373271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6827","nonce":3079373271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6831","nonce":3079373271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6829","nonce":3079373271}]},"public_addr":"172.21.15.151:6825/3079373271","cluster_addr":"172.21.15.151:6827/3079373271","heartbeat_back_addr":"172.21.15.151:6831/3079373271","heartbeat_front_addr":"172.21.15.151:6829/3079373271","state":["exists","up"]},{"osd":3,"uuid":"36edb718-2fcb-4d88-8ddc-c321eed4279d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6817","nonce":1142509041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6819","nonce":1142509041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6824","nonce":1142509041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6821","nonce":1142509041}]},"public_addr":"172.21.15.151:6817/1142509041","cluster_addr":"172.21.15.151:6819/1142509041","heartbeat_back_addr":"172.21.15.151:6824/1142509041","heartbeat_front_addr":"172.21.15.151:6821/1142509041","state":["exists","up"]},{"osd":4,"uuid":"64d33708-cd0a-4236-abe3-1b646e82ab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6811","nonce":2735495271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6813","nonce":2735495271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6817","nonce":2735495271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6815","nonce":2735495271}]},"public_addr":"172.21.15.167:6811/2735495271","cluster_addr":"172.21.15.167:6813/2735495271","heartbeat_back_addr":"172.21.15.167:6817/2735495271","heartbeat_front_addr":"172.21.15.167:6815/2735495271","state":["exists","up"]},{"osd":5,"uuid":"f14b44a9-5910-458c-b518-09a8d0de94de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6827","nonce":1494916572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6829","nonce":1494916572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6833","nonce":1494916572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6831","nonce":1494916572}]},"public_addr":"172.21.15.167:6827/1494916572","cluster_addr":"172.21.15.167:6829/1494916572","heartbeat_back_addr":"172.21.15.167:6833/1494916572","heartbeat_front_addr":"172.21.15.167:6831/1494916572","state":["exists","up"]},{"osd":6,"uuid":"39335545-2654-4710-84b0-a9aa47cfcbba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6819","nonce":2538273273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6821","nonce":2538273273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6825","nonce":2538273273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6823","nonce":2538273273}]},"public_addr":"172.21.15.167:6819/2538273273","cluster_addr":"172.21.15.167:6821/2538273273","heartbeat_back_addr":"172.21.15.167:6825/2538273273","heartbeat_front_addr":"172.21.15.167:6823/2538273273","state":["exists","up"]},{"osd":7,"uuid":"4f8bfc81-53dd-4339-ac22-1a4007d9f963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6803","nonce":265072227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6805","nonce":265072227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6809","nonce":265072227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6807","nonce":265072227}]},"public_addr":"172.21.15.167:6803/265072227","cluster_addr":"172.21.15.167:6805/265072227","heartbeat_back_addr":"172.21.15.167:6809/265072227","heartbeat_front_addr":"172.21.15.167:6807/265072227","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.150359+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.084358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.722368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.332809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.320709+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.395153+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.935250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.801443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.1d","mappings":[{"from":4,"to":3}]},{"pgid":"17.1c","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":17,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:53:26.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.062+0000 7efddc000640 1 -- 172.21.15.151:0/4047348543 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efdb407a740 msgr2=0x7efdb407cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:26.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.062+0000 7efddc000640 1 --2- 172.21.15.151:0/4047348543 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efdb407a740 0x7efdb407cc00 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7efdc80046b0 tx=0x7efdc8007cc0 comp rx=0 tx=0).stop 2024-09-16T15:53:26.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.062+0000 7efddc000640 1 -- 172.21.15.151:0/4047348543 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efdd40cf300 msgr2=0x7efdd41401f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:26.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.062+0000 7efddc000640 1 --2- 172.21.15.151:0/4047348543 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efdd40cf300 0x7efdd41401f0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7efdc0009980 tx=0x7efdc0032b90 comp rx=0 tx=0).stop 2024-09-16T15:53:26.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.062+0000 7efddc000640 1 -- 172.21.15.151:0/4047348543 shutdown_connections 2024-09-16T15:53:26.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.062+0000 7efddc000640 1 --2- 172.21.15.151:0/4047348543 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efdb407a740 0x7efdb407cc00 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:26.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.062+0000 7efddc000640 1 --2- 172.21.15.151:0/4047348543 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efdd41445d0 0x7efdd40b08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:26.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.062+0000 7efddc000640 1 --2- 172.21.15.151:0/4047348543 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efdd40cf300 0x7efdd41401f0 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:26.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.062+0000 7efddc000640 1 --2- 172.21.15.151:0/4047348543 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efdd40ccfd0 0x7efdd41e9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:26.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.062+0000 7efddc000640 1 -- 172.21.15.151:0/4047348543 >> 172.21.15.151:0/4047348543 conn(0x7efdd40b1950 msgr2=0x7efdd40ce980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:26.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.062+0000 7efddc000640 1 -- 172.21.15.151:0/4047348543 shutdown_connections 2024-09-16T15:53:26.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.063+0000 7efddc000640 1 -- 172.21.15.151:0/4047348543 wait complete. 2024-09-16T15:53:26.080 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-16T15:53:26.080 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:53:26.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.236+0000 7f28b5c00640 1 -- 172.21.15.151:0/147149329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28b00ccb30 msgr2=0x7f28b006f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:26.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.236+0000 7f28b5c00640 1 --2- 172.21.15.151:0/147149329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28b00ccb30 0x7f28b006f850 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f289c009830 tx=0x7f289c02a4a0 comp rx=0 tx=0).stop 2024-09-16T15:53:26.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.236+0000 7f28b5c00640 1 -- 172.21.15.151:0/147149329 shutdown_connections 2024-09-16T15:53:26.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.236+0000 7f28b5c00640 1 --2- 172.21.15.151:0/147149329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28b00ccb30 0x7f28b006f850 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:26.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.236+0000 7f28b5c00640 1 --2- 172.21.15.151:0/147149329 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f28b01438b0 0x7f28b00cc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:26.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.236+0000 7f28b5c00640 1 -- 172.21.15.151:0/147149329 >> 172.21.15.151:0/147149329 conn(0x7f28b00b1950 msgr2=0x7f28b00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:26.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.236+0000 7f28b5c00640 1 -- 172.21.15.151:0/147149329 shutdown_connections 2024-09-16T15:53:26.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.236+0000 7f28b5c00640 1 -- 172.21.15.151:0/147149329 wait complete. 2024-09-16T15:53:26.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.238+0000 7f28b5c00640 1 Processor -- start 2024-09-16T15:53:26.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.238+0000 7f28b5c00640 1 -- start start 2024-09-16T15:53:26.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.239+0000 7f28b5c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28b006fd90 0x7f28b00caea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:26.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.239+0000 7f28b5c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28b00ccb30 0x7f28b012fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:26.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.239+0000 7f28b5c00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f28b01438b0 0x7f28b00c9240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:26.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.239+0000 7f28b5c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f28b0070920 con 0x7f28b00ccb30 2024-09-16T15:53:26.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.239+0000 7f28b5c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f28b00b4f60 con 0x7f28b006fd90 2024-09-16T15:53:26.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.239+0000 7f28b5c00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f28b0070aa0 con 0x7f28b01438b0 2024-09-16T15:53:26.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.239+0000 7f28ae000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28b00ccb30 0x7f28b012fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:26.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.239+0000 7f28ae000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28b00ccb30 0x7f28b012fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41960/0 (socket says 172.21.15.151:41960) 2024-09-16T15:53:26.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.239+0000 7f28ae000640 1 -- 172.21.15.151:0/798437303 learned_addr learned my addr 172.21.15.151:0/798437303 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:53:26.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.240+0000 7f28aea00640 1 --2- 172.21.15.151:0/798437303 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28b006fd90 0x7f28b00caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:26.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.240+0000 7f28af400640 1 --2- 172.21.15.151:0/798437303 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f28b01438b0 0x7f28b00c9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:26.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.240+0000 7f28ae000640 1 -- 172.21.15.151:0/798437303 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f28b01438b0 msgr2=0x7f28b00c9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:26.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.240+0000 7f28ae000640 1 --2- 172.21.15.151:0/798437303 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f28b01438b0 0x7f28b00c9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:26.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.240+0000 7f28ae000640 1 -- 172.21.15.151:0/798437303 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28b006fd90 msgr2=0x7f28b00caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:26.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.240+0000 7f28ae000640 1 --2- 172.21.15.151:0/798437303 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28b006fd90 0x7f28b00caea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:26.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.240+0000 7f28ae000640 1 -- 172.21.15.151:0/798437303 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28b01302f0 con 0x7f28b00ccb30 2024-09-16T15:53:26.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.240+0000 7f28ae000640 1 --2- 172.21.15.151:0/798437303 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28b00ccb30 0x7f28b012fdb0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f289c009950 tx=0x7f289c030650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:26.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.240+0000 7f28a6a00640 1 -- 172.21.15.151:0/798437303 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289c040070 con 0x7f28b00ccb30 2024-09-16T15:53:26.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.240+0000 7f28a6a00640 1 -- 172.21.15.151:0/798437303 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f289c030d40 con 0x7f28b00ccb30 2024-09-16T15:53:26.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.240+0000 7f28b5c00640 1 -- 172.21.15.151:0/798437303 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28b00cb800 con 0x7f28b00ccb30 2024-09-16T15:53:26.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.240+0000 7f28a6a00640 1 -- 172.21.15.151:0/798437303 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289c004280 con 0x7f28b00ccb30 2024-09-16T15:53:26.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.241+0000 7f28b5c00640 1 -- 172.21.15.151:0/798437303 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28b00cbdc0 con 0x7f28b00ccb30 2024-09-16T15:53:26.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.242+0000 7f28b5c00640 1 -- 172.21.15.151:0/798437303 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28b006f850 con 0x7f28b00ccb30 2024-09-16T15:53:26.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.243+0000 7f28a6a00640 1 -- 172.21.15.151:0/798437303 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 47) ==== 104092+0+0 (secure 0 0 0) 0x7f289c039040 con 0x7f28b00ccb30 2024-09-16T15:53:26.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.244+0000 7f28a6a00640 1 --2- 172.21.15.151:0/798437303 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f289007a740 0x7f289007cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:26.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.244+0000 7f28a6a00640 1 -- 172.21.15.151:0/798437303 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5907+0+0 (secure 0 0 0) 0x7f289c0ba2d0 con 0x7f28b00ccb30 2024-09-16T15:53:26.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.245+0000 7f28aea00640 1 --2- 172.21.15.151:0/798437303 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f289007a740 0x7f289007cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:26.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.245+0000 7f28aea00640 1 --2- 172.21.15.151:0/798437303 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f289007a740 0x7f289007cc00 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f28b009ae80 tx=0x7f28a0005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:26.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.249+0000 7f28a6a00640 1 -- 172.21.15.151:0/798437303 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f28b01302f0 con 0x7f28b00ccb30 2024-09-16T15:53:26.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.469+0000 7f28b5c00640 1 -- 172.21.15.151:0/798437303 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f28b00cea60 con 0x7f28b00ccb30 2024-09-16T15:53:26.471 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.470+0000 7f28a6a00640 1 -- 172.21.15.151:0/798437303 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+5237 (secure 0 0 0) 0x7f289c083cd0 con 0x7f28b00ccb30 2024-09-16T15:53:26.471 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:53:26.471 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":77,"btime":"2024-09-16T15:52:31:147854+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7034,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1802374613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1802374613},{"type":"v1","addr":"172.21.15.167:6837","nonce":1802374613}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":7753,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3585191088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3585191088},{"type":"v1","addr":"172.21.15.167:6836","nonce":3585191088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":8457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/1321222227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1321222227},{"type":"v1","addr":"172.21.15.151:6834","nonce":1321222227}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:52:17.489987+0000","modified":"2024-09-16T15:52:20.525094+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8460":{"gid":8460,"name":"a","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.151:6835/2504927912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":2504927912},{"type":"v1","addr":"172.21.15.151:6835","nonce":2504927912}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8460,"qdb_cluster":[8460]},"id":8}]} 2024-09-16T15:53:26.471 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 77 2024-09-16T15:53:26.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.472+0000 7f28b5c00640 1 -- 172.21.15.151:0/798437303 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f289007a740 msgr2=0x7f289007cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:26.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.472+0000 7f28b5c00640 1 --2- 172.21.15.151:0/798437303 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f289007a740 0x7f289007cc00 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f28b009ae80 tx=0x7f28a0005f90 comp rx=0 tx=0).stop 2024-09-16T15:53:26.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.472+0000 7f28b5c00640 1 -- 172.21.15.151:0/798437303 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28b00ccb30 msgr2=0x7f28b012fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:26.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.472+0000 7f28b5c00640 1 --2- 172.21.15.151:0/798437303 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28b00ccb30 0x7f28b012fdb0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f289c009950 tx=0x7f289c030650 comp rx=0 tx=0).stop 2024-09-16T15:53:26.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.472+0000 7f28b5c00640 1 -- 172.21.15.151:0/798437303 shutdown_connections 2024-09-16T15:53:26.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.472+0000 7f28b5c00640 1 --2- 172.21.15.151:0/798437303 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f289007a740 0x7f289007cc00 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:26.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.472+0000 7f28b5c00640 1 --2- 172.21.15.151:0/798437303 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f28b01438b0 0x7f28b00c9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:26.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.472+0000 7f28b5c00640 1 --2- 172.21.15.151:0/798437303 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28b00ccb30 0x7f28b012fdb0 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:26.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.472+0000 7f28b5c00640 1 --2- 172.21.15.151:0/798437303 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f28b006fd90 0x7f28b00caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:26.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.472+0000 7f28b5c00640 1 -- 172.21.15.151:0/798437303 >> 172.21.15.151:0/798437303 conn(0x7f28b00b1950 msgr2=0x7f28b015e570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:26.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.472+0000 7f28b5c00640 1 -- 172.21.15.151:0/798437303 shutdown_connections 2024-09-16T15:53:26.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.472+0000 7f28b5c00640 1 -- 172.21.15.151:0/798437303 wait complete. 2024-09-16T15:53:26.489 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:53:26.648 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.646+0000 7f7b03e00640 1 -- 172.21.15.151:0/1753468119 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ae80051c0 msgr2=0x7f7ae800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:26.648 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.646+0000 7f7b03e00640 1 --2- 172.21.15.151:0/1753468119 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ae80051c0 0x7f7ae800b9c0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f7afc001dc0 tx=0x7f7ae8005830 comp rx=0 tx=0).stop 2024-09-16T15:53:26.648 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.646+0000 7f7b03e00640 1 -- 172.21.15.151:0/1753468119 shutdown_connections 2024-09-16T15:53:26.648 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.646+0000 7f7b03e00640 1 --2- 172.21.15.151:0/1753468119 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ae80051c0 0x7f7ae800b9c0 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:26.648 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.646+0000 7f7b03e00640 1 -- 172.21.15.151:0/1753468119 >> 172.21.15.151:0/1753468119 conn(0x7f7b040cfa20 msgr2=0x7f7b040cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:26.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.646+0000 7f7b03e00640 1 -- 172.21.15.151:0/1753468119 shutdown_connections 2024-09-16T15:53:26.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.647+0000 7f7b03e00640 1 -- 172.21.15.151:0/1753468119 wait complete. 2024-09-16T15:53:26.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.648+0000 7f7b03e00640 1 Processor -- start 2024-09-16T15:53:26.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.648+0000 7f7b03e00640 1 -- start start 2024-09-16T15:53:26.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.649+0000 7f7b03e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7ae80051c0 0x7f7b041e0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:26.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.649+0000 7f7b03e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b041e19b0 0x7f7b040ce840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:26.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.649+0000 7f7b03e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b0414da70 0x7f7b0414def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:26.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.649+0000 7f7b03e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7b040b2c80 con 0x7f7b041e19b0 2024-09-16T15:53:26.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.649+0000 7f7b03e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7b040b2b00 con 0x7f7b0414da70 2024-09-16T15:53:26.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.649+0000 7f7b03e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f7b04073d40 con 0x7f7ae80051c0 2024-09-16T15:53:26.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.649+0000 7f7b02000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b041e19b0 0x7f7b040ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:26.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.649+0000 7f7b02000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b041e19b0 0x7f7b040ce840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41968/0 (socket says 172.21.15.151:41968) 2024-09-16T15:53:26.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.650+0000 7f7b02000640 1 -- 172.21.15.151:0/4265131468 learned_addr learned my addr 172.21.15.151:0/4265131468 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:53:26.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.650+0000 7f7b02a00640 1 --2- 172.21.15.151:0/4265131468 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7ae80051c0 0x7f7b041e0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:26.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.650+0000 7f7b03400640 1 --2- 172.21.15.151:0/4265131468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b0414da70 0x7f7b0414def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:26.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.650+0000 7f7b02000640 1 -- 172.21.15.151:0/4265131468 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7ae80051c0 msgr2=0x7f7b041e0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:26.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.650+0000 7f7b02000640 1 --2- 172.21.15.151:0/4265131468 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7ae80051c0 0x7f7b041e0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:26.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.650+0000 7f7b02000640 1 -- 172.21.15.151:0/4265131468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b0414da70 msgr2=0x7f7b0414def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:26.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.650+0000 7f7b02000640 1 --2- 172.21.15.151:0/4265131468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b0414da70 0x7f7b0414def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:26.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.650+0000 7f7b02000640 1 -- 172.21.15.151:0/4265131468 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b04151550 con 0x7f7b041e19b0 2024-09-16T15:53:26.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.650+0000 7f7b02000640 1 --2- 172.21.15.151:0/4265131468 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b041e19b0 0x7f7b040ce840 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f7af00066b0 tx=0x7f7af00039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:26.653 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.650+0000 7f7afb400640 1 -- 172.21.15.151:0/4265131468 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af0015070 con 0x7f7b041e19b0 2024-09-16T15:53:26.653 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.650+0000 7f7afb400640 1 -- 172.21.15.151:0/4265131468 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7af0011920 con 0x7f7b041e19b0 2024-09-16T15:53:26.653 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.650+0000 7f7afb400640 1 -- 172.21.15.151:0/4265131468 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af0011d70 con 0x7f7b041e19b0 2024-09-16T15:53:26.653 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.650+0000 7f7b03e00640 1 -- 172.21.15.151:0/4265131468 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b041f43a0 con 0x7f7b041e19b0 2024-09-16T15:53:26.653 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.651+0000 7f7b03e00640 1 -- 172.21.15.151:0/4265131468 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b0414f410 con 0x7f7b041e19b0 2024-09-16T15:53:26.653 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.652+0000 7f7b03e00640 1 -- 172.21.15.151:0/4265131468 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b040742a0 con 0x7f7b041e19b0 2024-09-16T15:53:26.654 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.653+0000 7f7afb400640 1 -- 172.21.15.151:0/4265131468 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 47) ==== 104092+0+0 (secure 0 0 0) 0x7f7af0018070 con 0x7f7b041e19b0 2024-09-16T15:53:26.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.654+0000 7f7afb400640 1 --2- 172.21.15.151:0/4265131468 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7ae407a740 0x7f7ae407cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:26.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.654+0000 7f7afb400640 1 -- 172.21.15.151:0/4265131468 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5907+0+0 (secure 0 0 0) 0x7f7af009a930 con 0x7f7b041e19b0 2024-09-16T15:53:26.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.654+0000 7f7b02a00640 1 --2- 172.21.15.151:0/4265131468 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7ae407a740 0x7f7ae407cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:26.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.655+0000 7f7b02a00640 1 --2- 172.21.15.151:0/4265131468 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7ae407a740 0x7f7ae407cc00 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f7af4009a20 tx=0x7f7af4005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:26.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.658+0000 7f7afb400640 1 -- 172.21.15.151:0/4265131468 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7af0064980 con 0x7f7b041e19b0 2024-09-16T15:53:26.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.880+0000 7f7b03e00640 1 -- 172.21.15.151:0/4265131468 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7b040d0050 con 0x7f7b041e19b0 2024-09-16T15:53:26.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.881+0000 7f7afb400640 1 -- 172.21.15.151:0/4265131468 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+5237 (secure 0 0 0) 0x7f7af0064280 con 0x7f7b041e19b0 2024-09-16T15:53:26.882 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:53:26.882 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":77,"btime":"2024-09-16T15:52:31:147854+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7034,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1802374613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1802374613},{"type":"v1","addr":"172.21.15.167:6837","nonce":1802374613}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":7753,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3585191088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3585191088},{"type":"v1","addr":"172.21.15.167:6836","nonce":3585191088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":8457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/1321222227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1321222227},{"type":"v1","addr":"172.21.15.151:6834","nonce":1321222227}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:52:17.489987+0000","modified":"2024-09-16T15:52:20.525094+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8460},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8460":{"gid":8460,"name":"a","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.151:6835/2504927912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":2504927912},{"type":"v1","addr":"172.21.15.151:6835","nonce":2504927912}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8460,"qdb_cluster":[8460]},"id":8}]} 2024-09-16T15:53:26.882 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 77 2024-09-16T15:53:26.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.883+0000 7f7b03e00640 1 -- 172.21.15.151:0/4265131468 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7ae407a740 msgr2=0x7f7ae407cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:26.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.883+0000 7f7b03e00640 1 --2- 172.21.15.151:0/4265131468 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7ae407a740 0x7f7ae407cc00 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f7af4009a20 tx=0x7f7af4005f90 comp rx=0 tx=0).stop 2024-09-16T15:53:26.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.883+0000 7f7b03e00640 1 -- 172.21.15.151:0/4265131468 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b041e19b0 msgr2=0x7f7b040ce840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:26.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.883+0000 7f7b03e00640 1 --2- 172.21.15.151:0/4265131468 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b041e19b0 0x7f7b040ce840 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f7af00066b0 tx=0x7f7af00039c0 comp rx=0 tx=0).stop 2024-09-16T15:53:26.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.883+0000 7f7b03e00640 1 -- 172.21.15.151:0/4265131468 shutdown_connections 2024-09-16T15:53:26.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.883+0000 7f7b03e00640 1 --2- 172.21.15.151:0/4265131468 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7ae407a740 0x7f7ae407cc00 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:26.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.883+0000 7f7b03e00640 1 --2- 172.21.15.151:0/4265131468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b0414da70 0x7f7b0414def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:26.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.883+0000 7f7b03e00640 1 --2- 172.21.15.151:0/4265131468 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b041e19b0 0x7f7b040ce840 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:26.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.883+0000 7f7b03e00640 1 --2- 172.21.15.151:0/4265131468 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7ae80051c0 0x7f7b041e0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:26.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.883+0000 7f7b03e00640 1 -- 172.21.15.151:0/4265131468 >> 172.21.15.151:0/4265131468 conn(0x7f7b040cfa20 msgr2=0x7f7b0415aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:26.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.884+0000 7f7b03e00640 1 -- 172.21.15.151:0/4265131468 shutdown_connections 2024-09-16T15:53:26.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:26.884+0000 7f7b03e00640 1 -- 172.21.15.151:0/4265131468 wait complete. 2024-09-16T15:53:26.900 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:53:27.058 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.057+0000 7f2ff3200640 1 -- 172.21.15.151:0/1025970642 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ff415bc30 msgr2=0x7f2ff40b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:27.058 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.057+0000 7f2ff3200640 1 --2- 172.21.15.151:0/1025970642 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ff415bc30 0x7f2ff40b28f0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4009830 tx=0x7f2fe402a540 comp rx=0 tx=0).stop 2024-09-16T15:53:27.058 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.057+0000 7f2ff3200640 1 -- 172.21.15.151:0/1025970642 shutdown_connections 2024-09-16T15:53:27.058 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.057+0000 7f2ff3200640 1 --2- 172.21.15.151:0/1025970642 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ff415bc30 0x7f2ff40b28f0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:27.058 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.057+0000 7f2ff3200640 1 -- 172.21.15.151:0/1025970642 >> 172.21.15.151:0/1025970642 conn(0x7f2ff40cfa20 msgr2=0x7f2ff40cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:27.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.057+0000 7f2ff3200640 1 -- 172.21.15.151:0/1025970642 shutdown_connections 2024-09-16T15:53:27.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.057+0000 7f2ff3200640 1 -- 172.21.15.151:0/1025970642 wait complete. 2024-09-16T15:53:27.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.058+0000 7f2ff3200640 1 Processor -- start 2024-09-16T15:53:27.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.059+0000 7f2ff3200640 1 -- start start 2024-09-16T15:53:27.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.059+0000 7f2ff3200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2ff40d0540 0x7f2ff4145d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:27.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.059+0000 7f2ff3200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2ff415bc30 0x7f2ff41508d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:27.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.059+0000 7f2ff3200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ff40c8d00 0x7f2ff40c91b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:27.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.059+0000 7f2ff3200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff4074660 con 0x7f2ff415bc30 2024-09-16T15:53:27.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.059+0000 7f2ff3200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff406fa30 con 0x7f2ff40c8d00 2024-09-16T15:53:27.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.059+0000 7f2ff3200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f2ff414e8e0 con 0x7f2ff40d0540 2024-09-16T15:53:27.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.060+0000 7f2ff1400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2ff415bc30 0x7f2ff41508d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:27.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.060+0000 7f2ff1400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2ff415bc30 0x7f2ff41508d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41978/0 (socket says 172.21.15.151:41978) 2024-09-16T15:53:27.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.060+0000 7f2ff1400640 1 -- 172.21.15.151:0/2050850896 learned_addr learned my addr 172.21.15.151:0/2050850896 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:53:27.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.060+0000 7f2ff2800640 1 --2- 172.21.15.151:0/2050850896 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ff40c8d00 0x7f2ff40c91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:27.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.060+0000 7f2ff1e00640 1 --2- 172.21.15.151:0/2050850896 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2ff40d0540 0x7f2ff4145d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:27.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.060+0000 7f2ff1400640 1 -- 172.21.15.151:0/2050850896 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2ff40d0540 msgr2=0x7f2ff4145d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:27.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.060+0000 7f2ff1400640 1 --2- 172.21.15.151:0/2050850896 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2ff40d0540 0x7f2ff4145d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:27.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.060+0000 7f2ff1400640 1 -- 172.21.15.151:0/2050850896 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ff40c8d00 msgr2=0x7f2ff40c91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:27.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.060+0000 7f2ff1400640 1 --2- 172.21.15.151:0/2050850896 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ff40c8d00 0x7f2ff40c91b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:27.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.060+0000 7f2ff1400640 1 -- 172.21.15.151:0/2050850896 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ff414eb30 con 0x7f2ff415bc30 2024-09-16T15:53:27.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.060+0000 7f2ff2800640 1 --2- 172.21.15.151:0/2050850896 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ff40c8d00 0x7f2ff40c91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:53:27.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.060+0000 7f2ff1400640 1 --2- 172.21.15.151:0/2050850896 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2ff415bc30 0x7f2ff41508d0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f2fe000c420 tx=0x7f2fe000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:27.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.061+0000 7f2fea000640 1 -- 172.21.15.151:0/2050850896 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe0017070 con 0x7f2ff415bc30 2024-09-16T15:53:27.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.061+0000 7f2fea000640 1 -- 172.21.15.151:0/2050850896 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2fe00053c0 con 0x7f2ff415bc30 2024-09-16T15:53:27.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.061+0000 7f2fea000640 1 -- 172.21.15.151:0/2050850896 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe0003850 con 0x7f2ff415bc30 2024-09-16T15:53:27.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.061+0000 7f2ff3200640 1 -- 172.21.15.151:0/2050850896 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ff414ee10 con 0x7f2ff415bc30 2024-09-16T15:53:27.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.061+0000 7f2ff3200640 1 -- 172.21.15.151:0/2050850896 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ff414f3d0 con 0x7f2ff415bc30 2024-09-16T15:53:27.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.062+0000 7f2ff3200640 1 -- 172.21.15.151:0/2050850896 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ff40b28f0 con 0x7f2ff415bc30 2024-09-16T15:53:27.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.063+0000 7f2fea000640 1 -- 172.21.15.151:0/2050850896 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 47) ==== 104092+0+0 (secure 0 0 0) 0x7f2fe001a070 con 0x7f2ff415bc30 2024-09-16T15:53:27.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.064+0000 7f2fea000640 1 --2- 172.21.15.151:0/2050850896 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2fd407a740 0x7f2fd407cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:27.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.064+0000 7f2fea000640 1 -- 172.21.15.151:0/2050850896 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5907+0+0 (secure 0 0 0) 0x7f2fe0098cf0 con 0x7f2ff415bc30 2024-09-16T15:53:27.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.064+0000 7f2ff1e00640 1 --2- 172.21.15.151:0/2050850896 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2fd407a740 0x7f2fd407cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:27.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.065+0000 7f2ff1e00640 1 --2- 172.21.15.151:0/2050850896 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2fd407a740 0x7f2fd407cc00 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4034000 tx=0x7f2fe4033040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:27.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.068+0000 7f2fea000640 1 -- 172.21.15.151:0/2050850896 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2fe0062de0 con 0x7f2ff415bc30 2024-09-16T15:53:27.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.254+0000 7f2ff3200640 1 -- 172.21.15.151:0/2050850896 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2ff40cacc0 con 0x7f2ff415bc30 2024-09-16T15:53:27.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.255+0000 7f2fea000640 1 -- 172.21.15.151:0/2050850896 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) ==== 74+0+17133 (secure 0 0 0) 0x7f2fe00626e0 con 0x7f2ff415bc30 2024-09-16T15:53:27.257 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:53:27.257 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":121,"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","created":"2024-09-16T15:38:56.348535+0000","modified":"2024-09-16T15:53:23.762259+0000","last_up_change":"2024-09-16T15:39:10.681384+0000","last_in_change":"2024-09-16T15:39:00.544477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:39:11.602252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:52:15.256011+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-16T15:52:16.456620+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":121,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5524076-a903-4d0a-af6a-caf474c4765d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6809","nonce":3111342741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6811","nonce":3111342741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6815","nonce":3111342741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6813","nonce":3111342741}]},"public_addr":"172.21.15.151:6809/3111342741","cluster_addr":"172.21.15.151:6811/3111342741","heartbeat_back_addr":"172.21.15.151:6815/3111342741","heartbeat_front_addr":"172.21.15.151:6813/3111342741","state":["exists","up"]},{"osd":1,"uuid":"dd25925f-0ae1-494e-8529-4dc205759db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6801","nonce":3368304087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6803","nonce":3368304087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6807","nonce":3368304087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6805","nonce":3368304087}]},"public_addr":"172.21.15.151:6801/3368304087","cluster_addr":"172.21.15.151:6803/3368304087","heartbeat_back_addr":"172.21.15.151:6807/3368304087","heartbeat_front_addr":"172.21.15.151:6805/3368304087","state":["exists","up"]},{"osd":2,"uuid":"c2f18115-013d-4da6-a0fe-cb978f4494b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6823","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6825","nonce":3079373271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6827","nonce":3079373271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6831","nonce":3079373271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6829","nonce":3079373271}]},"public_addr":"172.21.15.151:6825/3079373271","cluster_addr":"172.21.15.151:6827/3079373271","heartbeat_back_addr":"172.21.15.151:6831/3079373271","heartbeat_front_addr":"172.21.15.151:6829/3079373271","state":["exists","up"]},{"osd":3,"uuid":"36edb718-2fcb-4d88-8ddc-c321eed4279d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6817","nonce":1142509041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6819","nonce":1142509041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6824","nonce":1142509041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6821","nonce":1142509041}]},"public_addr":"172.21.15.151:6817/1142509041","cluster_addr":"172.21.15.151:6819/1142509041","heartbeat_back_addr":"172.21.15.151:6824/1142509041","heartbeat_front_addr":"172.21.15.151:6821/1142509041","state":["exists","up"]},{"osd":4,"uuid":"64d33708-cd0a-4236-abe3-1b646e82ab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6811","nonce":2735495271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6813","nonce":2735495271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6817","nonce":2735495271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6815","nonce":2735495271}]},"public_addr":"172.21.15.167:6811/2735495271","cluster_addr":"172.21.15.167:6813/2735495271","heartbeat_back_addr":"172.21.15.167:6817/2735495271","heartbeat_front_addr":"172.21.15.167:6815/2735495271","state":["exists","up"]},{"osd":5,"uuid":"f14b44a9-5910-458c-b518-09a8d0de94de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6827","nonce":1494916572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6829","nonce":1494916572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6833","nonce":1494916572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6831","nonce":1494916572}]},"public_addr":"172.21.15.167:6827/1494916572","cluster_addr":"172.21.15.167:6829/1494916572","heartbeat_back_addr":"172.21.15.167:6833/1494916572","heartbeat_front_addr":"172.21.15.167:6831/1494916572","state":["exists","up"]},{"osd":6,"uuid":"39335545-2654-4710-84b0-a9aa47cfcbba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6819","nonce":2538273273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6821","nonce":2538273273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6825","nonce":2538273273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6823","nonce":2538273273}]},"public_addr":"172.21.15.167:6819/2538273273","cluster_addr":"172.21.15.167:6821/2538273273","heartbeat_back_addr":"172.21.15.167:6825/2538273273","heartbeat_front_addr":"172.21.15.167:6823/2538273273","state":["exists","up"]},{"osd":7,"uuid":"4f8bfc81-53dd-4339-ac22-1a4007d9f963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6803","nonce":265072227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6805","nonce":265072227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6809","nonce":265072227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6807","nonce":265072227}]},"public_addr":"172.21.15.167:6803/265072227","cluster_addr":"172.21.15.167:6805/265072227","heartbeat_back_addr":"172.21.15.167:6809/265072227","heartbeat_front_addr":"172.21.15.167:6807/265072227","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.150359+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.084358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.722368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.332809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.320709+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.395153+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.935250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.801443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.1d","mappings":[{"from":4,"to":3}]},{"pgid":"17.1c","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":17,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:53:27.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.258+0000 7f2ff3200640 1 -- 172.21.15.151:0/2050850896 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2fd407a740 msgr2=0x7f2fd407cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:27.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.258+0000 7f2ff3200640 1 --2- 172.21.15.151:0/2050850896 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2fd407a740 0x7f2fd407cc00 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4034000 tx=0x7f2fe4033040 comp rx=0 tx=0).stop 2024-09-16T15:53:27.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.258+0000 7f2ff3200640 1 -- 172.21.15.151:0/2050850896 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2ff415bc30 msgr2=0x7f2ff41508d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:27.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.258+0000 7f2ff3200640 1 --2- 172.21.15.151:0/2050850896 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2ff415bc30 0x7f2ff41508d0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f2fe000c420 tx=0x7f2fe000c8e0 comp rx=0 tx=0).stop 2024-09-16T15:53:27.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.258+0000 7f2ff3200640 1 -- 172.21.15.151:0/2050850896 shutdown_connections 2024-09-16T15:53:27.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.258+0000 7f2ff3200640 1 --2- 172.21.15.151:0/2050850896 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2fd407a740 0x7f2fd407cc00 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:27.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.258+0000 7f2ff3200640 1 --2- 172.21.15.151:0/2050850896 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ff40c8d00 0x7f2ff40c91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:27.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.258+0000 7f2ff3200640 1 --2- 172.21.15.151:0/2050850896 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2ff415bc30 0x7f2ff41508d0 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:27.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.258+0000 7f2ff3200640 1 --2- 172.21.15.151:0/2050850896 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2ff40d0540 0x7f2ff4145d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:27.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.258+0000 7f2ff3200640 1 -- 172.21.15.151:0/2050850896 >> 172.21.15.151:0/2050850896 conn(0x7f2ff40cfa20 msgr2=0x7f2ff4159100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:27.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.258+0000 7f2ff3200640 1 -- 172.21.15.151:0/2050850896 shutdown_connections 2024-09-16T15:53:27.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.258+0000 7f2ff3200640 1 -- 172.21.15.151:0/2050850896 wait complete. 2024-09-16T15:53:27.276 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-16T15:53:27.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.434+0000 7f0c16000640 1 -- 172.21.15.151:0/1418089659 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c100d0540 msgr2=0x7f0c100b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:27.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.434+0000 7f0c16000640 1 --2- 172.21.15.151:0/1418089659 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c100d0540 0x7f0c100b28f0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f0c08009830 tx=0x7f0c0802a5c0 comp rx=0 tx=0).stop 2024-09-16T15:53:27.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.434+0000 7f0c16000640 1 -- 172.21.15.151:0/1418089659 shutdown_connections 2024-09-16T15:53:27.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.434+0000 7f0c16000640 1 --2- 172.21.15.151:0/1418089659 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c100d0540 0x7f0c100b28f0 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:27.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.434+0000 7f0c16000640 1 -- 172.21.15.151:0/1418089659 >> 172.21.15.151:0/1418089659 conn(0x7f0c100cfa20 msgr2=0x7f0c100cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:27.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.435+0000 7f0c16000640 1 -- 172.21.15.151:0/1418089659 shutdown_connections 2024-09-16T15:53:27.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.435+0000 7f0c16000640 1 -- 172.21.15.151:0/1418089659 wait complete. 2024-09-16T15:53:27.437 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.436+0000 7f0c16000640 1 Processor -- start 2024-09-16T15:53:27.437 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.436+0000 7f0c16000640 1 -- start start 2024-09-16T15:53:27.439 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.437+0000 7f0c16000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c100d0540 0x7f0c10151f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:27.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.437+0000 7f0c16000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c1015bc30 0x7f0c100c8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:27.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.437+0000 7f0c16000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c100c8fd0 0x7f0c10154e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:27.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.437+0000 7f0c16000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0c10074660 con 0x7f0c100c8fd0 2024-09-16T15:53:27.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.437+0000 7f0c16000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0c1006fa30 con 0x7f0c100d0540 2024-09-16T15:53:27.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.437+0000 7f0c16000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f0c101524b0 con 0x7f0c1015bc30 2024-09-16T15:53:27.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.437+0000 7f0c15600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c100c8fd0 0x7f0c10154e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:27.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.437+0000 7f0c15600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c100c8fd0 0x7f0c10154e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41994/0 (socket says 172.21.15.151:41994) 2024-09-16T15:53:27.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.437+0000 7f0c15600640 1 -- 172.21.15.151:0/860931666 learned_addr learned my addr 172.21.15.151:0/860931666 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:53:27.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.438+0000 7f0c0fe00640 1 --2- 172.21.15.151:0/860931666 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c1015bc30 0x7f0c100c8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:27.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.438+0000 7f0c14c00640 1 --2- 172.21.15.151:0/860931666 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c100d0540 0x7f0c10151f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:27.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.438+0000 7f0c15600640 1 -- 172.21.15.151:0/860931666 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c1015bc30 msgr2=0x7f0c100c8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:27.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.438+0000 7f0c15600640 1 --2- 172.21.15.151:0/860931666 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c1015bc30 0x7f0c100c8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:27.441 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.438+0000 7f0c15600640 1 -- 172.21.15.151:0/860931666 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c100d0540 msgr2=0x7f0c10151f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:27.441 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.438+0000 7f0c15600640 1 --2- 172.21.15.151:0/860931666 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c100d0540 0x7f0c10151f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:27.441 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.438+0000 7f0c15600640 1 -- 172.21.15.151:0/860931666 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c101f4490 con 0x7f0c100c8fd0 2024-09-16T15:53:27.441 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.438+0000 7f0c15600640 1 --2- 172.21.15.151:0/860931666 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c100c8fd0 0x7f0c10154e10 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f0c0802abb0 tx=0x7f0c08032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:27.441 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.438+0000 7f0c0cc00640 1 -- 172.21.15.151:0/860931666 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c08040070 con 0x7f0c100c8fd0 2024-09-16T15:53:27.441 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.438+0000 7f0c0cc00640 1 -- 172.21.15.151:0/860931666 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0c08005310 con 0x7f0c100c8fd0 2024-09-16T15:53:27.441 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.438+0000 7f0c0cc00640 1 -- 172.21.15.151:0/860931666 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c080047a0 con 0x7f0c100c8fd0 2024-09-16T15:53:27.441 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.439+0000 7f0c16000640 1 -- 172.21.15.151:0/860931666 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c101f4770 con 0x7f0c100c8fd0 2024-09-16T15:53:27.441 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.439+0000 7f0c16000640 1 -- 172.21.15.151:0/860931666 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c101f4d30 con 0x7f0c100c8fd0 2024-09-16T15:53:27.442 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.440+0000 7f0c16000640 1 -- 172.21.15.151:0/860931666 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c100b28f0 con 0x7f0c100c8fd0 2024-09-16T15:53:27.442 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.441+0000 7f0c0cc00640 1 -- 172.21.15.151:0/860931666 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 47) ==== 104092+0+0 (secure 0 0 0) 0x7f0c08039040 con 0x7f0c100c8fd0 2024-09-16T15:53:27.443 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.442+0000 7f0c0cc00640 1 --2- 172.21.15.151:0/860931666 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0bf007a770 0x7f0bf007cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:27.443 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.442+0000 7f0c0cc00640 1 -- 172.21.15.151:0/860931666 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5907+0+0 (secure 0 0 0) 0x7f0c080bac30 con 0x7f0c100c8fd0 2024-09-16T15:53:27.443 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.442+0000 7f0c14c00640 1 --2- 172.21.15.151:0/860931666 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0bf007a770 0x7f0bf007cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:27.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.443+0000 7f0c14c00640 1 --2- 172.21.15.151:0/860931666 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0bf007a770 0x7f0bf007cc30 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f0c000012e0 tx=0x7f0c00009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:27.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.446+0000 7f0c0cc00640 1 -- 172.21.15.151:0/860931666 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0c08084750 con 0x7f0c100c8fd0 2024-09-16T15:53:27.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.669+0000 7f0c16000640 1 -- 172.21.15.151:0/860931666 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f0c10069850 con 0x7f0c100c8fd0 2024-09-16T15:53:27.670 INFO:tasks.ceph.mon.a.smithi151.stderr:2024-09-16T15:53:27.669+0000 7f62c3200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T15:53:27.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.672+0000 7f0c0cc00640 1 -- 172.21.15.151:0/860931666 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v78) ==== 161+0+0 (secure 0 0 0) 0x7f0c08089180 con 0x7f0c100c8fd0 2024-09-16T15:53:27.673 INFO:teuthology.orchestra.run.smithi151.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-16T15:53:27.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.675+0000 7f0c16000640 1 -- 172.21.15.151:0/860931666 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0bf007a770 msgr2=0x7f0bf007cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:27.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.675+0000 7f0c16000640 1 --2- 172.21.15.151:0/860931666 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0bf007a770 0x7f0bf007cc30 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f0c000012e0 tx=0x7f0c00009040 comp rx=0 tx=0).stop 2024-09-16T15:53:27.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.675+0000 7f0c16000640 1 -- 172.21.15.151:0/860931666 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c100c8fd0 msgr2=0x7f0c10154e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:27.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.675+0000 7f0c16000640 1 --2- 172.21.15.151:0/860931666 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c100c8fd0 0x7f0c10154e10 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f0c0802abb0 tx=0x7f0c08032c10 comp rx=0 tx=0).stop 2024-09-16T15:53:27.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.675+0000 7f0c16000640 1 -- 172.21.15.151:0/860931666 shutdown_connections 2024-09-16T15:53:27.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.675+0000 7f0c16000640 1 --2- 172.21.15.151:0/860931666 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0bf007a770 0x7f0bf007cc30 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:27.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.675+0000 7f0c16000640 1 --2- 172.21.15.151:0/860931666 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c100c8fd0 0x7f0c10154e10 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:27.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.675+0000 7f0c16000640 1 --2- 172.21.15.151:0/860931666 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c1015bc30 0x7f0c100c8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:27.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.675+0000 7f0c16000640 1 --2- 172.21.15.151:0/860931666 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c100d0540 0x7f0c10151f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:27.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.675+0000 7f0c16000640 1 -- 172.21.15.151:0/860931666 >> 172.21.15.151:0/860931666 conn(0x7f0c100cfa20 msgr2=0x7f0c100c9ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:27.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.676+0000 7f0c16000640 1 -- 172.21.15.151:0/860931666 shutdown_connections 2024-09-16T15:53:27.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.676+0000 7f0c16000640 1 -- 172.21.15.151:0/860931666 wait complete. 2024-09-16T15:53:27.694 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-16T15:53:27.745 INFO:tasks.ceph.mds.a.smithi151.stdout:starting mds.a at 2024-09-16T15:53:27.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.860+0000 7ff56e800640 1 -- 172.21.15.151:0/2208141579 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff56815bec0 msgr2=0x7ff56806fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:27.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.860+0000 7ff56e800640 1 --2- 172.21.15.151:0/2208141579 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff56815bec0 0x7ff56806fce0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff55c009860 tx=0x7ff55c02a5a0 comp rx=0 tx=0).stop 2024-09-16T15:53:27.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.860+0000 7ff56e800640 1 -- 172.21.15.151:0/2208141579 shutdown_connections 2024-09-16T15:53:27.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.860+0000 7ff56e800640 1 --2- 172.21.15.151:0/2208141579 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff56815bec0 0x7ff56806fce0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:27.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.860+0000 7ff56e800640 1 -- 172.21.15.151:0/2208141579 >> 172.21.15.151:0/2208141579 conn(0x7ff5680cf9f0 msgr2=0x7ff5680cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:27.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.860+0000 7ff56e800640 1 -- 172.21.15.151:0/2208141579 shutdown_connections 2024-09-16T15:53:27.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.860+0000 7ff56e800640 1 -- 172.21.15.151:0/2208141579 wait complete. 2024-09-16T15:53:27.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.862+0000 7ff56e800640 1 Processor -- start 2024-09-16T15:53:27.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.862+0000 7ff56e800640 1 -- start start 2024-09-16T15:53:27.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.863+0000 7ff56e800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff568072560 0x7ff5680c9280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:27.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.863+0000 7ff56e800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff5680cbc00 0x7ff5680c7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:27.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.863+0000 7ff56e800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff56815bec0 0x7ff56814b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:27.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.863+0000 7ff56e800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff5680aed50 con 0x7ff568072560 2024-09-16T15:53:27.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.863+0000 7ff56e800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff5680aebd0 con 0x7ff56815bec0 2024-09-16T15:53:27.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.863+0000 7ff56e800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7ff5680aeed0 con 0x7ff5680cbc00 2024-09-16T15:53:27.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.863+0000 7ff56d400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff568072560 0x7ff5680c9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:27.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.863+0000 7ff56d400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff568072560 0x7ff5680c9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:42020/0 (socket says 172.21.15.151:42020) 2024-09-16T15:53:27.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.863+0000 7ff56d400640 1 -- 172.21.15.151:0/1495703464 learned_addr learned my addr 172.21.15.151:0/1495703464 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:53:27.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.863+0000 7ff56ca00640 1 --2- 172.21.15.151:0/1495703464 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff5680cbc00 0x7ff5680c7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:27.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.863+0000 7ff56de00640 1 --2- 172.21.15.151:0/1495703464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff56815bec0 0x7ff56814b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:27.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.863+0000 7ff56d400640 1 -- 172.21.15.151:0/1495703464 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff5680cbc00 msgr2=0x7ff5680c7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:27.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.863+0000 7ff56d400640 1 --2- 172.21.15.151:0/1495703464 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff5680cbc00 0x7ff5680c7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:27.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.863+0000 7ff56d400640 1 -- 172.21.15.151:0/1495703464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff56815bec0 msgr2=0x7ff56814b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:27.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.863+0000 7ff56d400640 1 --2- 172.21.15.151:0/1495703464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff56815bec0 0x7ff56814b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:27.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.863+0000 7ff56d400640 1 -- 172.21.15.151:0/1495703464 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff56814cc00 con 0x7ff568072560 2024-09-16T15:53:27.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.863+0000 7ff56ca00640 1 --2- 172.21.15.151:0/1495703464 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff5680cbc00 0x7ff5680c7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:53:27.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.864+0000 7ff56d400640 1 --2- 172.21.15.151:0/1495703464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff568072560 0x7ff5680c9280 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7ff55c02ab90 tx=0x7ff55c033ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:27.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.864+0000 7ff565600640 1 -- 172.21.15.151:0/1495703464 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff55c031b50 con 0x7ff568072560 2024-09-16T15:53:27.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.864+0000 7ff565600640 1 -- 172.21.15.151:0/1495703464 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff55c031de0 con 0x7ff568072560 2024-09-16T15:53:27.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.864+0000 7ff56e800640 1 -- 172.21.15.151:0/1495703464 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5680c81e0 con 0x7ff568072560 2024-09-16T15:53:27.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.864+0000 7ff565600640 1 -- 172.21.15.151:0/1495703464 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff55c033330 con 0x7ff568072560 2024-09-16T15:53:27.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.864+0000 7ff56e800640 1 -- 172.21.15.151:0/1495703464 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5680c87a0 con 0x7ff568072560 2024-09-16T15:53:27.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.865+0000 7ff56e800640 1 -- 172.21.15.151:0/1495703464 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff56806fce0 con 0x7ff568072560 2024-09-16T15:53:27.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.866+0000 7ff565600640 1 -- 172.21.15.151:0/1495703464 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 47) ==== 104092+0+0 (secure 0 0 0) 0x7ff55c0334d0 con 0x7ff568072560 2024-09-16T15:53:27.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.867+0000 7ff565600640 1 --2- 172.21.15.151:0/1495703464 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff54807a740 0x7ff54807cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:27.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.867+0000 7ff565600640 1 -- 172.21.15.151:0/1495703464 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5965+0+0 (secure 0 0 0) 0x7ff55c0bacb0 con 0x7ff568072560 2024-09-16T15:53:27.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.868+0000 7ff56ca00640 1 --2- 172.21.15.151:0/1495703464 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff54807a740 0x7ff54807cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:27.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.869+0000 7ff56ca00640 1 --2- 172.21.15.151:0/1495703464 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff54807a740 0x7ff54807cc00 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7ff5680add50 tx=0x7ff554009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:27.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:27.872+0000 7ff565600640 1 -- 172.21.15.151:0/1495703464 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff55c03f030 con 0x7ff568072560 2024-09-16T15:53:28.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:28.095+0000 7ff56e800640 1 -- 172.21.15.151:0/1495703464 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7ff56815f0d0 con 0x7ff568072560 2024-09-16T15:53:28.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:28.674+0000 7ff565600640 1 -- 172.21.15.151:0/1495703464 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v79) ==== 104+0+0 (secure 0 0 0) 0x7ff55c084d00 con 0x7ff568072560 2024-09-16T15:53:28.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:28.678+0000 7ff56e800640 1 -- 172.21.15.151:0/1495703464 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff54807a740 msgr2=0x7ff54807cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:28.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:28.678+0000 7ff56e800640 1 --2- 172.21.15.151:0/1495703464 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff54807a740 0x7ff54807cc00 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7ff5680add50 tx=0x7ff554009040 comp rx=0 tx=0).stop 2024-09-16T15:53:28.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:28.678+0000 7ff56e800640 1 -- 172.21.15.151:0/1495703464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff568072560 msgr2=0x7ff5680c9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:28.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:28.678+0000 7ff56e800640 1 --2- 172.21.15.151:0/1495703464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff568072560 0x7ff5680c9280 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7ff55c02ab90 tx=0x7ff55c033ba0 comp rx=0 tx=0).stop 2024-09-16T15:53:28.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:28.679+0000 7ff56e800640 1 -- 172.21.15.151:0/1495703464 shutdown_connections 2024-09-16T15:53:28.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:28.679+0000 7ff56e800640 1 --2- 172.21.15.151:0/1495703464 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff54807a740 0x7ff54807cc00 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:28.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:28.679+0000 7ff56e800640 1 --2- 172.21.15.151:0/1495703464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff56815bec0 0x7ff56814b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:28.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:28.679+0000 7ff56e800640 1 --2- 172.21.15.151:0/1495703464 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff5680cbc00 0x7ff5680c7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:28.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:28.679+0000 7ff56e800640 1 --2- 172.21.15.151:0/1495703464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff568072560 0x7ff5680c9280 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:28.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:28.679+0000 7ff56e800640 1 -- 172.21.15.151:0/1495703464 >> 172.21.15.151:0/1495703464 conn(0x7ff5680cf9f0 msgr2=0x7ff5680af3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:28.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:28.679+0000 7ff56e800640 1 -- 172.21.15.151:0/1495703464 shutdown_connections 2024-09-16T15:53:28.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:28.679+0000 7ff56e800640 1 -- 172.21.15.151:0/1495703464 wait complete. 2024-09-16T15:53:28.699 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-16T15:53:28.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:28.866+0000 7f924ac00640 1 -- 172.21.15.151:0/48599883 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f922c004d90 msgr2=0x7f922c005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:28.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:28.866+0000 7f924ac00640 1 --2- 172.21.15.151:0/48599883 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f922c004d90 0x7f922c005210 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f92440caea0 tx=0x7f922c005830 comp rx=0 tx=0).stop 2024-09-16T15:53:28.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:28.866+0000 7f924a200640 1 -- 172.21.15.151:0/48599883 reap_dead start 2024-09-16T15:53:28.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:28.866+0000 7f924ac00640 1 -- 172.21.15.151:0/48599883 shutdown_connections 2024-09-16T15:53:28.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:28.866+0000 7f924ac00640 1 -- 172.21.15.151:0/48599883 >> 172.21.15.151:0/48599883 conn(0x7f92440cfa70 msgr2=0x7f92440cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:28.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:28.866+0000 7f924ac00640 1 -- 172.21.15.151:0/48599883 shutdown_connections 2024-09-16T15:53:28.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:28.866+0000 7f924ac00640 1 -- 172.21.15.151:0/48599883 wait complete. 2024-09-16T15:53:28.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:28.868+0000 7f924ac00640 1 Processor -- start 2024-09-16T15:53:28.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:28.868+0000 7f924ac00640 1 -- start start 2024-09-16T15:53:28.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:28.869+0000 7f924ac00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f922c004d90 0x7f92441f1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:28.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:28.869+0000 7f924ac00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f924414c990 0x7f92441f4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:28.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:28.869+0000 7f924ac00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f924414ce10 0x7f92441f4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:28.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:28.869+0000 7f924ac00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f924415ef80 con 0x7f924414c990 2024-09-16T15:53:28.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:28.869+0000 7f924ac00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f924415ee00 con 0x7f922c004d90 2024-09-16T15:53:28.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:28.869+0000 7f924ac00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f924415f100 con 0x7f924414ce10 2024-09-16T15:53:28.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:28.869+0000 7f9248e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f924414c990 0x7f92441f4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:28.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:28.869+0000 7f9248e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f924414c990 0x7f92441f4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:42044/0 (socket says 172.21.15.151:42044) 2024-09-16T15:53:28.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:28.869+0000 7f9248e00640 1 -- 172.21.15.151:0/4178409029 learned_addr learned my addr 172.21.15.151:0/4178409029 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:53:28.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:28.869+0000 7f924a200640 1 --2- 172.21.15.151:0/4178409029 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f924414ce10 0x7f92441f4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:28.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:28.869+0000 7f9249800640 1 --2- 172.21.15.151:0/4178409029 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f922c004d90 0x7f92441f1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:28.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:28.869+0000 7f9248e00640 1 -- 172.21.15.151:0/4178409029 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f924414ce10 msgr2=0x7f92441f4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:28.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:28.869+0000 7f9248e00640 1 --2- 172.21.15.151:0/4178409029 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f924414ce10 0x7f92441f4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:28.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:28.869+0000 7f9248e00640 1 -- 172.21.15.151:0/4178409029 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f922c004d90 msgr2=0x7f92441f1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:28.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:28.869+0000 7f9248e00640 1 --2- 172.21.15.151:0/4178409029 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f922c004d90 0x7f92441f1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:28.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:28.869+0000 7f9248e00640 1 -- 172.21.15.151:0/4178409029 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f924414d350 con 0x7f924414c990 2024-09-16T15:53:28.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:28.870+0000 7f9248e00640 1 --2- 172.21.15.151:0/4178409029 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f924414c990 0x7f92441f4840 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f92300066e0 tx=0x7f92300039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:28.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:28.870+0000 7f9241600640 1 -- 172.21.15.151:0/4178409029 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9230015070 con 0x7f924414c990 2024-09-16T15:53:28.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:28.870+0000 7f9241600640 1 -- 172.21.15.151:0/4178409029 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f923000f980 con 0x7f924414c990 2024-09-16T15:53:28.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:28.870+0000 7f9241600640 1 -- 172.21.15.151:0/4178409029 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f923000fdd0 con 0x7f924414c990 2024-09-16T15:53:28.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:28.870+0000 7f924ac00640 1 -- 172.21.15.151:0/4178409029 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f924414d630 con 0x7f924414c990 2024-09-16T15:53:28.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:28.870+0000 7f924ac00640 1 -- 172.21.15.151:0/4178409029 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92441f21f0 con 0x7f924414c990 2024-09-16T15:53:28.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:28.871+0000 7f924ac00640 1 -- 172.21.15.151:0/4178409029 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9244075160 con 0x7f924414c990 2024-09-16T15:53:28.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:28.872+0000 7f9241600640 1 -- 172.21.15.151:0/4178409029 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 47) ==== 104092+0+0 (secure 0 0 0) 0x7f9230011430 con 0x7f924414c990 2024-09-16T15:53:28.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:28.873+0000 7f9241600640 1 --2- 172.21.15.151:0/4178409029 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f922407a740 0x7f922407cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:28.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:28.873+0000 7f9241600640 1 -- 172.21.15.151:0/4178409029 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5965+0+0 (secure 0 0 0) 0x7f923009a630 con 0x7f924414c990 2024-09-16T15:53:28.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:28.874+0000 7f9249800640 1 --2- 172.21.15.151:0/4178409029 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f922407a740 0x7f922407cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:28.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:28.875+0000 7f9249800640 1 --2- 172.21.15.151:0/4178409029 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f922407a740 0x7f922407cc00 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f9238005fd0 tx=0x7f9238009530 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:28.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:28.877+0000 7f9241600640 1 -- 172.21.15.151:0/4178409029 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9244075160 con 0x7f924414c990 2024-09-16T15:53:29.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:29.063+0000 7f924ac00640 1 -- 172.21.15.151:0/4178409029 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f92440ca380 con 0x7f924414c990 2024-09-16T15:53:29.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:29.803+0000 7f9241600640 1 -- 172.21.15.151:0/4178409029 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v123) ==== 181+0+0 (secure 0 0 0) 0x7f9230064190 con 0x7f924414c990 2024-09-16T15:53:29.804 INFO:teuthology.orchestra.run.smithi151.stderr:pool 'cephfs_metadata' removed 2024-09-16T15:53:29.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:29.806+0000 7f924ac00640 1 -- 172.21.15.151:0/4178409029 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f922407a740 msgr2=0x7f922407cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:29.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:29.806+0000 7f924ac00640 1 --2- 172.21.15.151:0/4178409029 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f922407a740 0x7f922407cc00 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f9238005fd0 tx=0x7f9238009530 comp rx=0 tx=0).stop 2024-09-16T15:53:29.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:29.806+0000 7f924ac00640 1 -- 172.21.15.151:0/4178409029 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f924414c990 msgr2=0x7f92441f4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:29.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:29.806+0000 7f924ac00640 1 --2- 172.21.15.151:0/4178409029 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f924414c990 0x7f92441f4840 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f92300066e0 tx=0x7f92300039c0 comp rx=0 tx=0).stop 2024-09-16T15:53:29.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:29.806+0000 7f924ac00640 1 -- 172.21.15.151:0/4178409029 shutdown_connections 2024-09-16T15:53:29.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:29.806+0000 7f924ac00640 1 --2- 172.21.15.151:0/4178409029 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f922407a740 0x7f922407cc00 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:29.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:29.806+0000 7f924ac00640 1 --2- 172.21.15.151:0/4178409029 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f924414ce10 0x7f92441f4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:29.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:29.806+0000 7f924ac00640 1 --2- 172.21.15.151:0/4178409029 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f924414c990 0x7f92441f4840 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:29.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:29.806+0000 7f924ac00640 1 --2- 172.21.15.151:0/4178409029 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f922c004d90 0x7f92441f1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:29.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:29.806+0000 7f924ac00640 1 -- 172.21.15.151:0/4178409029 >> 172.21.15.151:0/4178409029 conn(0x7f92440cfa70 msgr2=0x7f9244160500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:29.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:29.807+0000 7f924ac00640 1 -- 172.21.15.151:0/4178409029 shutdown_connections 2024-09-16T15:53:29.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:29.807+0000 7f924ac00640 1 -- 172.21.15.151:0/4178409029 wait complete. 2024-09-16T15:53:29.826 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-16T15:53:29.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:29.989+0000 7fc31f200640 1 -- 172.21.15.151:0/1080591937 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc31806e910 msgr2=0x7fc318073d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:29.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:29.989+0000 7fc31f200640 1 --2- 172.21.15.151:0/1080591937 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc31806e910 0x7fc318073d50 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fc304009830 tx=0x7fc30402a510 comp rx=0 tx=0).stop 2024-09-16T15:53:29.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:29.989+0000 7fc31f200640 1 -- 172.21.15.151:0/1080591937 shutdown_connections 2024-09-16T15:53:29.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:29.989+0000 7fc31f200640 1 --2- 172.21.15.151:0/1080591937 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc31806e910 0x7fc318073d50 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:29.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:29.989+0000 7fc31f200640 1 --2- 172.21.15.151:0/1080591937 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc31815bd60 0x7fc3180cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:29.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:29.989+0000 7fc31f200640 1 -- 172.21.15.151:0/1080591937 >> 172.21.15.151:0/1080591937 conn(0x7fc3180b18e0 msgr2=0x7fc3180b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:29.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:29.990+0000 7fc31f200640 1 -- 172.21.15.151:0/1080591937 shutdown_connections 2024-09-16T15:53:29.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:29.990+0000 7fc31f200640 1 -- 172.21.15.151:0/1080591937 wait complete. 2024-09-16T15:53:29.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:29.991+0000 7fc31f200640 1 Processor -- start 2024-09-16T15:53:29.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:29.991+0000 7fc31f200640 1 -- start start 2024-09-16T15:53:29.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:29.992+0000 7fc31f200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc31806e910 0x7fc3181492a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:29.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:29.992+0000 7fc31f200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc318074290 0x7fc31815c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:29.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:29.992+0000 7fc31f200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc31815bd60 0x7fc3180aeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:29.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:29.992+0000 7fc31f200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc31815ff70 con 0x7fc318074290 2024-09-16T15:53:29.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:29.992+0000 7fc31f200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc318074a20 con 0x7fc31806e910 2024-09-16T15:53:29.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:29.992+0000 7fc31f200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fc3181600f0 con 0x7fc31815bd60 2024-09-16T15:53:29.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:29.993+0000 7fc317400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc318074290 0x7fc31815c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:29.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:29.993+0000 7fc317400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc318074290 0x7fc31815c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:42064/0 (socket says 172.21.15.151:42064) 2024-09-16T15:53:29.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:29.993+0000 7fc317400640 1 -- 172.21.15.151:0/3688694947 learned_addr learned my addr 172.21.15.151:0/3688694947 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:53:29.996 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:29.993+0000 7fc317e00640 1 --2- 172.21.15.151:0/3688694947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc31806e910 0x7fc3181492a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:29.996 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:29.993+0000 7fc31d200640 1 --2- 172.21.15.151:0/3688694947 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc31815bd60 0x7fc3180aeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:29.996 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:29.993+0000 7fc317400640 1 -- 172.21.15.151:0/3688694947 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc31815bd60 msgr2=0x7fc3180aeaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:29.996 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:29.993+0000 7fc317400640 1 --2- 172.21.15.151:0/3688694947 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc31815bd60 0x7fc3180aeaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:29.996 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:29.993+0000 7fc317400640 1 -- 172.21.15.151:0/3688694947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc31806e910 msgr2=0x7fc3181492a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:29.996 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:29.993+0000 7fc317400640 1 --2- 172.21.15.151:0/3688694947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc31806e910 0x7fc3181492a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:29.996 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:29.993+0000 7fc317400640 1 -- 172.21.15.151:0/3688694947 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3180c9fb0 con 0x7fc318074290 2024-09-16T15:53:29.996 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:29.993+0000 7fc317400640 1 --2- 172.21.15.151:0/3688694947 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc318074290 0x7fc31815c3b0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fc304009950 tx=0x7fc304032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:29.996 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:29.994+0000 7fc30fe00640 1 -- 172.21.15.151:0/3688694947 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc304040070 con 0x7fc318074290 2024-09-16T15:53:29.996 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:29.994+0000 7fc30fe00640 1 -- 172.21.15.151:0/3688694947 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc304005330 con 0x7fc318074290 2024-09-16T15:53:29.996 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:29.994+0000 7fc31f200640 1 -- 172.21.15.151:0/3688694947 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3181497e0 con 0x7fc318074290 2024-09-16T15:53:29.996 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:29.994+0000 7fc30fe00640 1 -- 172.21.15.151:0/3688694947 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3040047a0 con 0x7fc318074290 2024-09-16T15:53:29.996 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:29.994+0000 7fc31f200640 1 -- 172.21.15.151:0/3688694947 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3180ca530 con 0x7fc318074290 2024-09-16T15:53:29.997 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:29.995+0000 7fc31f200640 1 -- 172.21.15.151:0/3688694947 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc318069360 con 0x7fc318074290 2024-09-16T15:53:29.997 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:29.996+0000 7fc30fe00640 1 -- 172.21.15.151:0/3688694947 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 47) ==== 104092+0+0 (secure 0 0 0) 0x7fc304039040 con 0x7fc318074290 2024-09-16T15:53:29.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:29.997+0000 7fc30fe00640 1 --2- 172.21.15.151:0/3688694947 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc2f807a740 0x7fc2f807cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:29.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:29.997+0000 7fc30fe00640 1 -- 172.21.15.151:0/3688694947 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5498+0+0 (secure 0 0 0) 0x7fc3040babe0 con 0x7fc318074290 2024-09-16T15:53:29.999 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:29.998+0000 7fc317e00640 1 --2- 172.21.15.151:0/3688694947 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc2f807a740 0x7fc2f807cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:29.999 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:29.998+0000 7fc317e00640 1 --2- 172.21.15.151:0/3688694947 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc2f807a740 0x7fc2f807cc00 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fc30800bb50 tx=0x7fc30800b2e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:30.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:30.001+0000 7fc30fe00640 1 -- 172.21.15.151:0/3688694947 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc304084660 con 0x7fc318074290 2024-09-16T15:53:30.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:30.197+0000 7fc31f200640 1 -- 172.21.15.151:0/3688694947 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc3180cbcc0 con 0x7fc318074290 2024-09-16T15:53:30.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:30.810+0000 7fc30fe00640 1 -- 172.21.15.151:0/3688694947 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v124) ==== 169+0+0 (secure 0 0 0) 0x7fc304089090 con 0x7fc318074290 2024-09-16T15:53:30.811 INFO:teuthology.orchestra.run.smithi151.stderr:pool 'cephfs_data' removed 2024-09-16T15:53:30.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:30.814+0000 7fc31f200640 1 -- 172.21.15.151:0/3688694947 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc2f807a740 msgr2=0x7fc2f807cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:30.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:30.814+0000 7fc31f200640 1 --2- 172.21.15.151:0/3688694947 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc2f807a740 0x7fc2f807cc00 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fc30800bb50 tx=0x7fc30800b2e0 comp rx=0 tx=0).stop 2024-09-16T15:53:30.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:30.814+0000 7fc31f200640 1 -- 172.21.15.151:0/3688694947 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc318074290 msgr2=0x7fc31815c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:30.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:30.814+0000 7fc31f200640 1 --2- 172.21.15.151:0/3688694947 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc318074290 0x7fc31815c3b0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fc304009950 tx=0x7fc304032c10 comp rx=0 tx=0).stop 2024-09-16T15:53:30.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:30.814+0000 7fc31f200640 1 -- 172.21.15.151:0/3688694947 shutdown_connections 2024-09-16T15:53:30.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:30.814+0000 7fc31f200640 1 --2- 172.21.15.151:0/3688694947 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc2f807a740 0x7fc2f807cc00 secure :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fc30800bb50 tx=0x7fc30800b2e0 comp rx=0 tx=0).stop 2024-09-16T15:53:30.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:30.814+0000 7fc31f200640 1 --2- 172.21.15.151:0/3688694947 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc31815bd60 0x7fc3180aeaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:30.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:30.814+0000 7fc31f200640 1 --2- 172.21.15.151:0/3688694947 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc318074290 0x7fc31815c3b0 secure :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fc304009950 tx=0x7fc304032c10 comp rx=0 tx=0).stop 2024-09-16T15:53:30.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:30.814+0000 7fc31f200640 1 --2- 172.21.15.151:0/3688694947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc31806e910 0x7fc3181492a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:30.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:30.814+0000 7fc31f200640 1 -- 172.21.15.151:0/3688694947 >> 172.21.15.151:0/3688694947 conn(0x7fc3180b18e0 msgr2=0x7fc3181573c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:30.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:30.814+0000 7fc31f200640 1 -- 172.21.15.151:0/3688694947 shutdown_connections 2024-09-16T15:53:30.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:30.814+0000 7fc31f200640 1 -- 172.21.15.151:0/3688694947 wait complete. 2024-09-16T15:53:30.834 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-16T15:53:30.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:30.992+0000 7f9bda200640 1 -- 172.21.15.151:0/4001009620 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9bd40ccf00 msgr2=0x7f9bd40acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:30.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:30.992+0000 7f9bda200640 1 --2- 172.21.15.151:0/4001009620 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9bd40ccf00 0x7f9bd40acd30 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f9bc0005fd0 tx=0x7f9bc002a540 comp rx=0 tx=0).stop 2024-09-16T15:53:30.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:30.992+0000 7f9bda200640 1 -- 172.21.15.151:0/4001009620 shutdown_connections 2024-09-16T15:53:30.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:30.992+0000 7f9bda200640 1 --2- 172.21.15.151:0/4001009620 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9bd40ccf00 0x7f9bd40acd30 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:30.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:30.992+0000 7f9bda200640 1 --2- 172.21.15.151:0/4001009620 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9bd415f5a0 0x7f9bd40b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:30.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:30.992+0000 7f9bda200640 1 -- 172.21.15.151:0/4001009620 >> 172.21.15.151:0/4001009620 conn(0x7f9bd40b1950 msgr2=0x7f9bd40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:30.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:30.992+0000 7f9bda200640 1 -- 172.21.15.151:0/4001009620 shutdown_connections 2024-09-16T15:53:30.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:30.992+0000 7f9bda200640 1 -- 172.21.15.151:0/4001009620 wait complete. 2024-09-16T15:53:30.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:30.993+0000 7f9bda200640 1 Processor -- start 2024-09-16T15:53:30.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:30.994+0000 7f9bda200640 1 -- start start 2024-09-16T15:53:30.996 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:30.994+0000 7f9bda200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9bd40ad270 0x7f9bd414c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:30.996 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:30.995+0000 7f9bda200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bd40ccf00 0x7f9bd415e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:30.996 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:30.995+0000 7f9bda200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9bd415f5a0 0x7f9bd415e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:30.996 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:30.995+0000 7f9bda200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd406f2e0 con 0x7f9bd415f5a0 2024-09-16T15:53:30.997 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:30.995+0000 7f9bda200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd406f160 con 0x7f9bd40ccf00 2024-09-16T15:53:30.997 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:30.995+0000 7f9bda200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f9bd406f460 con 0x7f9bd40ad270 2024-09-16T15:53:30.997 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:30.995+0000 7f9bd3e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9bd415f5a0 0x7f9bd415e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:30.997 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:30.995+0000 7f9bd3e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9bd415f5a0 0x7f9bd415e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:42072/0 (socket says 172.21.15.151:42072) 2024-09-16T15:53:30.997 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:30.995+0000 7f9bd3400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9bd40ad270 0x7f9bd414c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:30.997 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:30.995+0000 7f9bd3e00640 1 -- 172.21.15.151:0/1533021826 learned_addr learned my addr 172.21.15.151:0/1533021826 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:53:30.997 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:30.995+0000 7f9bd2a00640 1 --2- 172.21.15.151:0/1533021826 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bd40ccf00 0x7f9bd415e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:30.997 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:30.995+0000 7f9bd3e00640 1 -- 172.21.15.151:0/1533021826 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9bd40ad270 msgr2=0x7f9bd414c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:30.997 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:30.995+0000 7f9bd3e00640 1 --2- 172.21.15.151:0/1533021826 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9bd40ad270 0x7f9bd414c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:30.997 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:30.995+0000 7f9bd3e00640 1 -- 172.21.15.151:0/1533021826 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bd40ccf00 msgr2=0x7f9bd415e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:30.997 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:30.995+0000 7f9bd3e00640 1 --2- 172.21.15.151:0/1533021826 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bd40ccf00 0x7f9bd415e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:30.997 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:30.995+0000 7f9bd3e00640 1 -- 172.21.15.151:0/1533021826 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bd40ce590 con 0x7f9bd415f5a0 2024-09-16T15:53:30.997 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:30.996+0000 7f9bd3e00640 1 --2- 172.21.15.151:0/1533021826 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9bd415f5a0 0x7f9bd415e720 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc00c740 tx=0x7f9bcc00cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:30.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:30.996+0000 7f9bcb400640 1 -- 172.21.15.151:0/1533021826 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bcc003b70 con 0x7f9bd415f5a0 2024-09-16T15:53:30.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:30.996+0000 7f9bcb400640 1 -- 172.21.15.151:0/1533021826 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9bcc00ce50 con 0x7f9bd415f5a0 2024-09-16T15:53:30.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:30.996+0000 7f9bcb400640 1 -- 172.21.15.151:0/1533021826 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bcc00a850 con 0x7f9bd415f5a0 2024-09-16T15:53:30.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:30.996+0000 7f9bda200640 1 -- 172.21.15.151:0/1533021826 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bd40ce870 con 0x7f9bd415f5a0 2024-09-16T15:53:30.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:30.996+0000 7f9bda200640 1 -- 172.21.15.151:0/1533021826 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bd40cee30 con 0x7f9bd415f5a0 2024-09-16T15:53:30.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:30.997+0000 7f9bda200640 1 -- 172.21.15.151:0/1533021826 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bd406f980 con 0x7f9bd415f5a0 2024-09-16T15:53:30.999 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:30.998+0000 7f9bcb400640 1 -- 172.21.15.151:0/1533021826 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 47) ==== 104092+0+0 (secure 0 0 0) 0x7f9bcc00a9f0 con 0x7f9bd415f5a0 2024-09-16T15:53:31.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:30.999+0000 7f9bcb400640 1 --2- 172.21.15.151:0/1533021826 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9bb407a740 0x7f9bb407cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:31.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:30.999+0000 7f9bcb400640 1 -- 172.21.15.151:0/1533021826 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5055+0+0 (secure 0 0 0) 0x7f9bcc099630 con 0x7f9bd415f5a0 2024-09-16T15:53:31.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.000+0000 7f9bd3400640 1 --2- 172.21.15.151:0/1533021826 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9bb407a740 0x7f9bb407cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:31.001 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.000+0000 7f9bd3400640 1 --2- 172.21.15.151:0/1533021826 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9bb407a740 0x7f9bb407cc00 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f9bc4004630 tx=0x7f9bc400b2e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:31.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.003+0000 7f9bcb400640 1 -- 172.21.15.151:0/1533021826 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9bcc0630b0 con 0x7f9bd415f5a0 2024-09-16T15:53:31.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.188+0000 7f9bda200640 1 -- 172.21.15.151:0/1533021826 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f9bd40ae830 con 0x7f9bd415f5a0 2024-09-16T15:53:31.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.192+0000 7f9bcb400640 1 -- 172.21.15.151:0/1533021826 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v50) ==== 71+0+0 (secure 0 0 0) 0x7f9bcc067ae0 con 0x7f9bd415f5a0 2024-09-16T15:53:31.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.195+0000 7f9bda200640 1 -- 172.21.15.151:0/1533021826 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9bb407a740 msgr2=0x7f9bb407cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:31.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.195+0000 7f9bda200640 1 --2- 172.21.15.151:0/1533021826 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9bb407a740 0x7f9bb407cc00 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f9bc4004630 tx=0x7f9bc400b2e0 comp rx=0 tx=0).stop 2024-09-16T15:53:31.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.195+0000 7f9bda200640 1 -- 172.21.15.151:0/1533021826 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9bd415f5a0 msgr2=0x7f9bd415e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:31.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.195+0000 7f9bda200640 1 --2- 172.21.15.151:0/1533021826 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9bd415f5a0 0x7f9bd415e720 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc00c740 tx=0x7f9bcc00cc00 comp rx=0 tx=0).stop 2024-09-16T15:53:31.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.195+0000 7f9bda200640 1 -- 172.21.15.151:0/1533021826 shutdown_connections 2024-09-16T15:53:31.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.195+0000 7f9bda200640 1 --2- 172.21.15.151:0/1533021826 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9bb407a740 0x7f9bb407cc00 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:31.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.195+0000 7f9bda200640 1 --2- 172.21.15.151:0/1533021826 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bd40ccf00 0x7f9bd415e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:31.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.195+0000 7f9bda200640 1 --2- 172.21.15.151:0/1533021826 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9bd415f5a0 0x7f9bd415e720 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:31.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.195+0000 7f9bda200640 1 --2- 172.21.15.151:0/1533021826 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9bd40ad270 0x7f9bd414c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:31.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.195+0000 7f9bda200640 1 -- 172.21.15.151:0/1533021826 >> 172.21.15.151:0/1533021826 conn(0x7f9bd40b1950 msgr2=0x7f9bd409b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:31.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.195+0000 7f9bda200640 1 -- 172.21.15.151:0/1533021826 shutdown_connections 2024-09-16T15:53:31.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.195+0000 7f9bda200640 1 -- 172.21.15.151:0/1533021826 wait complete. 2024-09-16T15:53:31.212 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_equal_to_cloner_threads' 2024-09-16T15:53:31.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.368+0000 7f7fdc800640 1 -- 172.21.15.151:0/1317462329 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7fd4070f50 msgr2=0x7f7fd4071360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:31.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.368+0000 7f7fdc800640 1 --2- 172.21.15.151:0/1317462329 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7fd4070f50 0x7f7fd4071360 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f7fd0005fd0 tx=0x7f7fd002a940 comp rx=0 tx=0).stop 2024-09-16T15:53:31.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.368+0000 7f7fdc800640 1 -- 172.21.15.151:0/1317462329 shutdown_connections 2024-09-16T15:53:31.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.368+0000 7f7fdc800640 1 --2- 172.21.15.151:0/1317462329 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7fd4070f50 0x7f7fd4071360 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:31.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.368+0000 7f7fdc800640 1 -- 172.21.15.151:0/1317462329 >> 172.21.15.151:0/1317462329 conn(0x7f7fd40b2b90 msgr2=0x7f7fd40b2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:31.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.368+0000 7f7fdc800640 1 -- 172.21.15.151:0/1317462329 shutdown_connections 2024-09-16T15:53:31.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.368+0000 7f7fdc800640 1 -- 172.21.15.151:0/1317462329 wait complete. 2024-09-16T15:53:31.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.369+0000 7f7fdc800640 1 Processor -- start 2024-09-16T15:53:31.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.370+0000 7f7fdc800640 1 -- start start 2024-09-16T15:53:31.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.370+0000 7f7fdc800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7fd406cb70 0x7f7fd40b3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:31.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.370+0000 7f7fdc800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7fd4070f50 0x7f7fd41403f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:31.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.371+0000 7f7fdc800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7fd40ce310 0x7f7fd4143600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:31.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.371+0000 7f7fdc800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd40cd0a0 con 0x7f7fd4070f50 2024-09-16T15:53:31.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.371+0000 7f7fdc800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd40ccf20 con 0x7f7fd40ce310 2024-09-16T15:53:31.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.371+0000 7f7fdc800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f7fd40cd220 con 0x7f7fd406cb70 2024-09-16T15:53:31.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.371+0000 7f7fd9400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7fd4070f50 0x7f7fd41403f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:31.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.371+0000 7f7fd9400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7fd4070f50 0x7f7fd41403f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:42078/0 (socket says 172.21.15.151:42078) 2024-09-16T15:53:31.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.371+0000 7f7fd9400640 1 -- 172.21.15.151:0/1634223640 learned_addr learned my addr 172.21.15.151:0/1634223640 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:53:31.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.371+0000 7f7fd9e00640 1 --2- 172.21.15.151:0/1634223640 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7fd406cb70 0x7f7fd40b3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:31.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.371+0000 7f7fda800640 1 --2- 172.21.15.151:0/1634223640 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7fd40ce310 0x7f7fd4143600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:31.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.371+0000 7f7fd9400640 1 -- 172.21.15.151:0/1634223640 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7fd406cb70 msgr2=0x7f7fd40b3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:31.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.371+0000 7f7fd9400640 1 --2- 172.21.15.151:0/1634223640 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7fd406cb70 0x7f7fd40b3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:31.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.371+0000 7f7fd9400640 1 -- 172.21.15.151:0/1634223640 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7fd40ce310 msgr2=0x7f7fd4143600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:31.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.371+0000 7f7fd9400640 1 --2- 172.21.15.151:0/1634223640 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7fd40ce310 0x7f7fd4143600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:31.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.371+0000 7f7fd9400640 1 -- 172.21.15.151:0/1634223640 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fd40b6530 con 0x7f7fd4070f50 2024-09-16T15:53:31.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.371+0000 7f7fd9e00640 1 --2- 172.21.15.151:0/1634223640 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7fd406cb70 0x7f7fd40b3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:53:31.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.371+0000 7f7fda800640 1 --2- 172.21.15.151:0/1634223640 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7fd40ce310 0x7f7fd4143600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:53:31.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.371+0000 7f7fd9400640 1 --2- 172.21.15.151:0/1634223640 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7fd4070f50 0x7f7fd41403f0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f7fc400c730 tx=0x7f7fc400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:31.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.372+0000 7f7fce000640 1 -- 172.21.15.151:0/1634223640 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc4016020 con 0x7f7fd4070f50 2024-09-16T15:53:31.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.372+0000 7f7fce000640 1 -- 172.21.15.151:0/1634223640 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7fc400a840 con 0x7f7fd4070f50 2024-09-16T15:53:31.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.372+0000 7f7fce000640 1 -- 172.21.15.151:0/1634223640 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc400acb0 con 0x7f7fd4070f50 2024-09-16T15:53:31.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.372+0000 7f7fdc800640 1 -- 172.21.15.151:0/1634223640 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fd40b67d0 con 0x7f7fd4070f50 2024-09-16T15:53:31.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.372+0000 7f7fdc800640 1 -- 172.21.15.151:0/1634223640 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fd40b6d90 con 0x7f7fd4070f50 2024-09-16T15:53:31.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.373+0000 7f7fdc800640 1 -- 172.21.15.151:0/1634223640 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fd4071360 con 0x7f7fd4070f50 2024-09-16T15:53:31.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.374+0000 7f7fce000640 1 -- 172.21.15.151:0/1634223640 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 47) ==== 104092+0+0 (secure 0 0 0) 0x7f7fc4019020 con 0x7f7fd4070f50 2024-09-16T15:53:31.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.375+0000 7f7fce000640 1 --2- 172.21.15.151:0/1634223640 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7fb807a740 0x7f7fb807cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:31.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.375+0000 7f7fce000640 1 -- 172.21.15.151:0/1634223640 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5055+0+0 (secure 0 0 0) 0x7f7fc4099690 con 0x7f7fd4070f50 2024-09-16T15:53:31.376 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.376+0000 7f7fd9e00640 1 --2- 172.21.15.151:0/1634223640 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7fb807a740 0x7f7fb807cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:31.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.376+0000 7f7fd9e00640 1 --2- 172.21.15.151:0/1634223640 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7fb807a740 0x7f7fb807cc00 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8007fd0 tx=0x7f7fc8009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:31.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.379+0000 7f7fce000640 1 -- 172.21.15.151:0/1634223640 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7fc4020030 con 0x7f7fd4070f50 2024-09-16T15:53:31.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.638+0000 7f7fdc800640 1 -- 172.21.15.151:0/1634223640 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_equal_to_cloner_threads"]} v 0) -- 0x7f7fd415f850 con 0x7f7fd4070f50 2024-09-16T15:53:31.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.811+0000 7f7fce000640 1 -- 172.21.15.151:0/1634223640 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_equal_to_cloner_threads"]}]=0 v647) ==== 185+0+0 (secure 0 0 0) 0x7f7fd40718a0 con 0x7f7fd4070f50 2024-09-16T15:53:31.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.814+0000 7f7fdc800640 1 -- 172.21.15.151:0/1634223640 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7fb807a740 msgr2=0x7f7fb807cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:31.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.814+0000 7f7fdc800640 1 --2- 172.21.15.151:0/1634223640 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7fb807a740 0x7f7fb807cc00 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8007fd0 tx=0x7f7fc8009040 comp rx=0 tx=0).stop 2024-09-16T15:53:31.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.814+0000 7f7fdc800640 1 -- 172.21.15.151:0/1634223640 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7fd4070f50 msgr2=0x7f7fd41403f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:31.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.814+0000 7f7fdc800640 1 --2- 172.21.15.151:0/1634223640 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7fd4070f50 0x7f7fd41403f0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f7fc400c730 tx=0x7f7fc400cbf0 comp rx=0 tx=0).stop 2024-09-16T15:53:31.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.814+0000 7f7fdc800640 1 -- 172.21.15.151:0/1634223640 shutdown_connections 2024-09-16T15:53:31.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.814+0000 7f7fdc800640 1 --2- 172.21.15.151:0/1634223640 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7fb807a740 0x7f7fb807cc00 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:31.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.814+0000 7f7fdc800640 1 --2- 172.21.15.151:0/1634223640 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7fd40ce310 0x7f7fd4143600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:31.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.814+0000 7f7fdc800640 1 --2- 172.21.15.151:0/1634223640 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7fd4070f50 0x7f7fd41403f0 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:31.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.814+0000 7f7fdc800640 1 --2- 172.21.15.151:0/1634223640 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7fd406cb70 0x7f7fd40b3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:31.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.814+0000 7f7fdc800640 1 -- 172.21.15.151:0/1634223640 >> 172.21.15.151:0/1634223640 conn(0x7f7fd40b2b90 msgr2=0x7f7fd40b1430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:31.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.814+0000 7f7fdc800640 1 -- 172.21.15.151:0/1634223640 shutdown_connections 2024-09-16T15:53:31.816 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.814+0000 7f7fdc800640 1 -- 172.21.15.151:0/1634223640 wait complete. 2024-09-16T15:53:31.831 INFO:tasks.cephfs_test_runner:Test that progress bars, that printed for 3 ongoing clone jobs, are ... ok 2024-09-16T15:53:31.831 INFO:tasks.cephfs_test_runner:Starting test: test_when_clones_cancelled_are_less_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that the progress bar that is printed for 1 ongoing clone job is 2024-09-16T15:53:31.831 INFO:tasks.cephfs_test_runner:test_when_clones_cancelled_are_less_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-09-16T15:53:31.832 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:53:31.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.990+0000 7f83d7600640 1 -- 172.21.15.151:0/392101412 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f83d00ccf00 msgr2=0x7f83d00acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:31.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.990+0000 7f83d7600640 1 --2- 172.21.15.151:0/392101412 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f83d00ccf00 0x7f83d00acd30 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f83bc005fd0 tx=0x7f83bc02a540 comp rx=0 tx=0).stop 2024-09-16T15:53:31.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.990+0000 7f83d7600640 1 -- 172.21.15.151:0/392101412 shutdown_connections 2024-09-16T15:53:31.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.990+0000 7f83d7600640 1 --2- 172.21.15.151:0/392101412 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f83d00ccf00 0x7f83d00acd30 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:31.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.990+0000 7f83d7600640 1 --2- 172.21.15.151:0/392101412 >> v2:172.21.15.167:3300/0 conn(0x7f83d015f5a0 0x7f83d00b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:31.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.990+0000 7f83d7600640 1 -- 172.21.15.151:0/392101412 >> 172.21.15.151:0/392101412 conn(0x7f83d00b1950 msgr2=0x7f83d00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:31.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.990+0000 7f83d7600640 1 -- 172.21.15.151:0/392101412 shutdown_connections 2024-09-16T15:53:31.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.990+0000 7f83d7600640 1 -- 172.21.15.151:0/392101412 wait complete. 2024-09-16T15:53:31.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.992+0000 7f83d7600640 1 Processor -- start 2024-09-16T15:53:31.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.992+0000 7f83d7600640 1 -- start start 2024-09-16T15:53:31.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.993+0000 7f83d7600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f83d00ad270 0x7f83d009d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:31.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.993+0000 7f83d7600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f83d00ccf00 0x7f83d01f4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:31.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.993+0000 7f83d7600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f83d015f5a0 0x7f83d0158650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:31.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.993+0000 7f83d7600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f83d006f2e0 con 0x7f83d00ccf00 2024-09-16T15:53:31.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.993+0000 7f83d7600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f83d006f160 con 0x7f83d015f5a0 2024-09-16T15:53:31.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.993+0000 7f83d7600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f83d006f460 con 0x7f83d00ad270 2024-09-16T15:53:31.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.993+0000 7f83cfe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f83d00ccf00 0x7f83d01f4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:31.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.993+0000 7f83cfe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f83d00ccf00 0x7f83d01f4de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:42086/0 (socket says 172.21.15.151:42086) 2024-09-16T15:53:31.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.993+0000 7f83cfe00640 1 -- 172.21.15.151:0/459036528 learned_addr learned my addr 172.21.15.151:0/459036528 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:53:31.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.993+0000 7f83d4c00640 1 --2- 172.21.15.151:0/459036528 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f83d00ad270 0x7f83d009d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:31.995 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.993+0000 7f83d5600640 1 --2- 172.21.15.151:0/459036528 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f83d015f5a0 0x7f83d0158650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:31.996 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.994+0000 7f83cfe00640 1 -- 172.21.15.151:0/459036528 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f83d00ad270 msgr2=0x7f83d009d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:31.996 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.994+0000 7f83cfe00640 1 --2- 172.21.15.151:0/459036528 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f83d00ad270 0x7f83d009d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:31.996 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.994+0000 7f83cfe00640 1 -- 172.21.15.151:0/459036528 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f83d015f5a0 msgr2=0x7f83d0158650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:31.996 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.994+0000 7f83cfe00640 1 --2- 172.21.15.151:0/459036528 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f83d015f5a0 0x7f83d0158650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:31.996 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.994+0000 7f83cfe00640 1 -- 172.21.15.151:0/459036528 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83d01564e0 con 0x7f83d00ccf00 2024-09-16T15:53:31.996 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.994+0000 7f83d4c00640 1 --2- 172.21.15.151:0/459036528 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f83d00ad270 0x7f83d009d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:53:31.996 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.994+0000 7f83cfe00640 1 --2- 172.21.15.151:0/459036528 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f83d00ccf00 0x7f83d01f4de0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f83bc009850 tx=0x7f83bc0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:31.996 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.994+0000 7f83ccc00640 1 -- 172.21.15.151:0/459036528 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83bc03f070 con 0x7f83d00ccf00 2024-09-16T15:53:31.996 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.994+0000 7f83ccc00640 1 -- 172.21.15.151:0/459036528 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f83bc0047a0 con 0x7f83d00ccf00 2024-09-16T15:53:31.996 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.994+0000 7f83d7600640 1 -- 172.21.15.151:0/459036528 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83d01567c0 con 0x7f83d00ccf00 2024-09-16T15:53:31.996 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.994+0000 7f83ccc00640 1 -- 172.21.15.151:0/459036528 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83bc035030 con 0x7f83d00ccf00 2024-09-16T15:53:31.996 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.994+0000 7f83d7600640 1 -- 172.21.15.151:0/459036528 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83d01f48c0 con 0x7f83d00ccf00 2024-09-16T15:53:31.997 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.995+0000 7f83d7600640 1 -- 172.21.15.151:0/459036528 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83d006f9f0 con 0x7f83d00ccf00 2024-09-16T15:53:31.997 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.996+0000 7f83ccc00640 1 -- 172.21.15.151:0/459036528 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 47) ==== 104092+0+0 (secure 0 0 0) 0x7f83bc042070 con 0x7f83d00ccf00 2024-09-16T15:53:31.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.997+0000 7f83ccc00640 1 --2- 172.21.15.151:0/459036528 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f83b007a770 0x7f83b007cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:31.998 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.998+0000 7f83ccc00640 1 -- 172.21.15.151:0/459036528 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5055+0+0 (secure 0 0 0) 0x7f83bc0ba5f0 con 0x7f83d00ccf00 2024-09-16T15:53:31.999 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.998+0000 7f83d4c00640 1 --2- 172.21.15.151:0/459036528 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f83b007a770 0x7f83b007cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:31.999 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:31.999+0000 7f83d4c00640 1 --2- 172.21.15.151:0/459036528 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f83b007a770 0x7f83b007cc30 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f83c0005fd0 tx=0x7f83c0009500 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:32.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.002+0000 7f83ccc00640 1 -- 172.21.15.151:0/459036528 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f83bc084110 con 0x7f83d00ccf00 2024-09-16T15:53:32.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.185+0000 7f83d7600640 1 -- 172.21.15.151:0/459036528 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f83d0156990 con 0x7f83d00ccf00 2024-09-16T15:53:32.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.186+0000 7f83ccc00640 1 -- 172.21.15.151:0/459036528 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) ==== 74+0+13281 (secure 0 0 0) 0x7f83bc088b40 con 0x7f83d00ccf00 2024-09-16T15:53:32.187 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:53:32.187 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":124,"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","created":"2024-09-16T15:38:56.348535+0000","modified":"2024-09-16T15:53:30.804110+0000","last_up_change":"2024-09-16T15:39:10.681384+0000","last_in_change":"2024-09-16T15:39:00.544477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:39:11.602252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5524076-a903-4d0a-af6a-caf474c4765d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6809","nonce":3111342741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6811","nonce":3111342741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6815","nonce":3111342741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6813","nonce":3111342741}]},"public_addr":"172.21.15.151:6809/3111342741","cluster_addr":"172.21.15.151:6811/3111342741","heartbeat_back_addr":"172.21.15.151:6815/3111342741","heartbeat_front_addr":"172.21.15.151:6813/3111342741","state":["exists","up"]},{"osd":1,"uuid":"dd25925f-0ae1-494e-8529-4dc205759db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6801","nonce":3368304087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6803","nonce":3368304087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6807","nonce":3368304087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6805","nonce":3368304087}]},"public_addr":"172.21.15.151:6801/3368304087","cluster_addr":"172.21.15.151:6803/3368304087","heartbeat_back_addr":"172.21.15.151:6807/3368304087","heartbeat_front_addr":"172.21.15.151:6805/3368304087","state":["exists","up"]},{"osd":2,"uuid":"c2f18115-013d-4da6-a0fe-cb978f4494b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6823","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6825","nonce":3079373271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6827","nonce":3079373271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6831","nonce":3079373271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6829","nonce":3079373271}]},"public_addr":"172.21.15.151:6825/3079373271","cluster_addr":"172.21.15.151:6827/3079373271","heartbeat_back_addr":"172.21.15.151:6831/3079373271","heartbeat_front_addr":"172.21.15.151:6829/3079373271","state":["exists","up"]},{"osd":3,"uuid":"36edb718-2fcb-4d88-8ddc-c321eed4279d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6817","nonce":1142509041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6819","nonce":1142509041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6824","nonce":1142509041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6821","nonce":1142509041}]},"public_addr":"172.21.15.151:6817/1142509041","cluster_addr":"172.21.15.151:6819/1142509041","heartbeat_back_addr":"172.21.15.151:6824/1142509041","heartbeat_front_addr":"172.21.15.151:6821/1142509041","state":["exists","up"]},{"osd":4,"uuid":"64d33708-cd0a-4236-abe3-1b646e82ab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6811","nonce":2735495271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6813","nonce":2735495271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6817","nonce":2735495271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6815","nonce":2735495271}]},"public_addr":"172.21.15.167:6811/2735495271","cluster_addr":"172.21.15.167:6813/2735495271","heartbeat_back_addr":"172.21.15.167:6817/2735495271","heartbeat_front_addr":"172.21.15.167:6815/2735495271","state":["exists","up"]},{"osd":5,"uuid":"f14b44a9-5910-458c-b518-09a8d0de94de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6827","nonce":1494916572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6829","nonce":1494916572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6833","nonce":1494916572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6831","nonce":1494916572}]},"public_addr":"172.21.15.167:6827/1494916572","cluster_addr":"172.21.15.167:6829/1494916572","heartbeat_back_addr":"172.21.15.167:6833/1494916572","heartbeat_front_addr":"172.21.15.167:6831/1494916572","state":["exists","up"]},{"osd":6,"uuid":"39335545-2654-4710-84b0-a9aa47cfcbba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6819","nonce":2538273273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6821","nonce":2538273273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6825","nonce":2538273273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6823","nonce":2538273273}]},"public_addr":"172.21.15.167:6819/2538273273","cluster_addr":"172.21.15.167:6821/2538273273","heartbeat_back_addr":"172.21.15.167:6825/2538273273","heartbeat_front_addr":"172.21.15.167:6823/2538273273","state":["exists","up"]},{"osd":7,"uuid":"4f8bfc81-53dd-4339-ac22-1a4007d9f963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6803","nonce":265072227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6805","nonce":265072227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6809","nonce":265072227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6807","nonce":265072227}]},"public_addr":"172.21.15.167:6803/265072227","cluster_addr":"172.21.15.167:6805/265072227","heartbeat_back_addr":"172.21.15.167:6809/265072227","heartbeat_front_addr":"172.21.15.167:6807/265072227","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.150359+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.084358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.722368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.332809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.320709+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.395153+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.935250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.801443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.151:6835/2504927912":"2024-09-17T15:53:27.670213+0000","172.21.15.151:6833/2504927912":"2024-09-17T15:53:27.670213+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:53:32.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.188+0000 7f83d7600640 1 -- 172.21.15.151:0/459036528 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f83b007a770 msgr2=0x7f83b007cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:32.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.188+0000 7f83d7600640 1 --2- 172.21.15.151:0/459036528 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f83b007a770 0x7f83b007cc30 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f83c0005fd0 tx=0x7f83c0009500 comp rx=0 tx=0).stop 2024-09-16T15:53:32.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.189+0000 7f83d7600640 1 -- 172.21.15.151:0/459036528 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f83d00ccf00 msgr2=0x7f83d01f4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:32.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.189+0000 7f83d7600640 1 --2- 172.21.15.151:0/459036528 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f83d00ccf00 0x7f83d01f4de0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f83bc009850 tx=0x7f83bc0038a0 comp rx=0 tx=0).stop 2024-09-16T15:53:32.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.189+0000 7f83d7600640 1 -- 172.21.15.151:0/459036528 shutdown_connections 2024-09-16T15:53:32.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.189+0000 7f83d7600640 1 --2- 172.21.15.151:0/459036528 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f83b007a770 0x7f83b007cc30 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:32.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.189+0000 7f83d7600640 1 --2- 172.21.15.151:0/459036528 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f83d015f5a0 0x7f83d0158650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:32.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.189+0000 7f83d7600640 1 --2- 172.21.15.151:0/459036528 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f83d00ccf00 0x7f83d01f4de0 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:32.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.189+0000 7f83d7600640 1 --2- 172.21.15.151:0/459036528 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f83d00ad270 0x7f83d009d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:32.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.189+0000 7f83d7600640 1 -- 172.21.15.151:0/459036528 >> 172.21.15.151:0/459036528 conn(0x7f83d00b1950 msgr2=0x7f83d00b2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:32.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.189+0000 7f83d7600640 1 -- 172.21.15.151:0/459036528 shutdown_connections 2024-09-16T15:53:32.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.189+0000 7f83d7600640 1 -- 172.21.15.151:0/459036528 wait complete. 2024-09-16T15:53:32.206 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T15:39:11.602252+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-16T15:53:32.206 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-16T15:53:32.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.372+0000 7f119ee00640 1 -- 172.21.15.151:0/642466926 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f117c003a50 msgr2=0x7f117c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:32.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.372+0000 7f119ee00640 1 --2- 172.21.15.151:0/642466926 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f117c003a50 0x7f117c003e70 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f1188001690 tx=0x7f117c005830 comp rx=0 tx=0).stop 2024-09-16T15:53:32.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.372+0000 7f119ee00640 1 -- 172.21.15.151:0/642466926 shutdown_connections 2024-09-16T15:53:32.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.372+0000 7f119ee00640 1 --2- 172.21.15.151:0/642466926 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f117c003a50 0x7f117c003e70 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:32.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.372+0000 7f119ee00640 1 -- 172.21.15.151:0/642466926 >> 172.21.15.151:0/642466926 conn(0x7f11980b1840 msgr2=0x7f11980b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:32.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.372+0000 7f119ee00640 1 -- 172.21.15.151:0/642466926 shutdown_connections 2024-09-16T15:53:32.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.372+0000 7f119ee00640 1 -- 172.21.15.151:0/642466926 wait complete. 2024-09-16T15:53:32.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.374+0000 7f119ee00640 1 Processor -- start 2024-09-16T15:53:32.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.374+0000 7f119ee00640 1 -- start start 2024-09-16T15:53:32.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.375+0000 7f119ee00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f117c003a50 0x7f11981f5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:32.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.375+0000 7f119ee00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1198144290 0x7f1198150b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:32.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.375+0000 7f119ee00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1198144710 0x7f119814eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:32.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.375+0000 7f119ee00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f11980737d0 con 0x7f1198144290 2024-09-16T15:53:32.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.375+0000 7f119ee00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1198073650 con 0x7f117c003a50 2024-09-16T15:53:32.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.375+0000 7f119ee00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f11980cc5c0 con 0x7f1198144710 2024-09-16T15:53:32.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.375+0000 7f1197400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1198144290 0x7f1198150b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:32.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.375+0000 7f1197400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1198144290 0x7f1198150b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:42106/0 (socket says 172.21.15.151:42106) 2024-09-16T15:53:32.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.375+0000 7f1197400640 1 -- 172.21.15.151:0/3955355559 learned_addr learned my addr 172.21.15.151:0/3955355559 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:53:32.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.376+0000 7f1197e00640 1 --2- 172.21.15.151:0/3955355559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f117c003a50 0x7f11981f5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:32.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.376+0000 7f119ce00640 1 --2- 172.21.15.151:0/3955355559 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1198144710 0x7f119814eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:32.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.376+0000 7f1197400640 1 -- 172.21.15.151:0/3955355559 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1198144710 msgr2=0x7f119814eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:32.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.376+0000 7f1197400640 1 --2- 172.21.15.151:0/3955355559 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1198144710 0x7f119814eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:32.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.376+0000 7f1197400640 1 -- 172.21.15.151:0/3955355559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f117c003a50 msgr2=0x7f11981f5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:32.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.376+0000 7f1197400640 1 --2- 172.21.15.151:0/3955355559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f117c003a50 0x7f11981f5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:32.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.376+0000 7f1197400640 1 -- 172.21.15.151:0/3955355559 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1198151aa0 con 0x7f1198144290 2024-09-16T15:53:32.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.376+0000 7f119ce00640 1 --2- 172.21.15.151:0/3955355559 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1198144710 0x7f119814eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:53:32.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.376+0000 7f1197400640 1 --2- 172.21.15.151:0/3955355559 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1198144290 0x7f1198150b70 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f1184006690 tx=0x7f1184003a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:32.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.376+0000 7f118fe00640 1 -- 172.21.15.151:0/3955355559 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1184006c70 con 0x7f1198144290 2024-09-16T15:53:32.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.376+0000 7f118fe00640 1 -- 172.21.15.151:0/3955355559 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1184016020 con 0x7f1198144290 2024-09-16T15:53:32.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.376+0000 7f118fe00640 1 -- 172.21.15.151:0/3955355559 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118400fa20 con 0x7f1198144290 2024-09-16T15:53:32.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.376+0000 7f119ee00640 1 -- 172.21.15.151:0/3955355559 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11981514e0 con 0x7f1198144290 2024-09-16T15:53:32.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.377+0000 7f119ee00640 1 -- 172.21.15.151:0/3955355559 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11981f5210 con 0x7f1198144290 2024-09-16T15:53:32.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.378+0000 7f119ee00640 1 -- 172.21.15.151:0/3955355559 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f119815f9f0 con 0x7f1198144290 2024-09-16T15:53:32.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.379+0000 7f118fe00640 1 -- 172.21.15.151:0/3955355559 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 47) ==== 104092+0+0 (secure 0 0 0) 0x7f118400fbc0 con 0x7f1198144290 2024-09-16T15:53:32.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.380+0000 7f118fe00640 1 --2- 172.21.15.151:0/3955355559 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f117807a740 0x7f117807cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:32.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.380+0000 7f118fe00640 1 -- 172.21.15.151:0/3955355559 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5055+0+0 (secure 0 0 0) 0x7f1184099fe0 con 0x7f1198144290 2024-09-16T15:53:32.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.381+0000 7f1197e00640 1 --2- 172.21.15.151:0/3955355559 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f117807a740 0x7f117807cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:32.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.381+0000 7f1197e00640 1 --2- 172.21.15.151:0/3955355559 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f117807a740 0x7f117807cc00 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f1188032000 tx=0x7f118802c040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:32.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.384+0000 7f118fe00640 1 -- 172.21.15.151:0/3955355559 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1184020030 con 0x7f1198144290 2024-09-16T15:53:32.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.568+0000 7f119ee00640 1 -- 172.21.15.151:0/3955355559 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f11980aa180 con 0x7f1198144290 2024-09-16T15:53:32.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.568+0000 7f118fe00640 1 -- 172.21.15.151:0/3955355559 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v124) ==== 93+0+10 (secure 0 0 0) 0x7f11840640b0 con 0x7f1198144290 2024-09-16T15:53:32.570 INFO:teuthology.orchestra.run.smithi151.stdout:pg_num: 1 2024-09-16T15:53:32.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.571+0000 7f119ee00640 1 -- 172.21.15.151:0/3955355559 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f117807a740 msgr2=0x7f117807cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:32.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.571+0000 7f119ee00640 1 --2- 172.21.15.151:0/3955355559 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f117807a740 0x7f117807cc00 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f1188032000 tx=0x7f118802c040 comp rx=0 tx=0).stop 2024-09-16T15:53:32.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.571+0000 7f119ee00640 1 -- 172.21.15.151:0/3955355559 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1198144290 msgr2=0x7f1198150b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:32.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.571+0000 7f119ee00640 1 --2- 172.21.15.151:0/3955355559 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1198144290 0x7f1198150b70 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f1184006690 tx=0x7f1184003a80 comp rx=0 tx=0).stop 2024-09-16T15:53:32.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.571+0000 7f119ee00640 1 -- 172.21.15.151:0/3955355559 shutdown_connections 2024-09-16T15:53:32.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.571+0000 7f119ee00640 1 --2- 172.21.15.151:0/3955355559 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f117807a740 0x7f117807cc00 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:32.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.571+0000 7f119ee00640 1 --2- 172.21.15.151:0/3955355559 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1198144710 0x7f119814eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:32.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.571+0000 7f119ee00640 1 --2- 172.21.15.151:0/3955355559 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1198144290 0x7f1198150b70 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:32.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.571+0000 7f119ee00640 1 --2- 172.21.15.151:0/3955355559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f117c003a50 0x7f11981f5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:32.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.571+0000 7f119ee00640 1 -- 172.21.15.151:0/3955355559 >> 172.21.15.151:0/3955355559 conn(0x7f11980b1840 msgr2=0x7f11980b0090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:32.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.571+0000 7f119ee00640 1 -- 172.21.15.151:0/3955355559 shutdown_connections 2024-09-16T15:53:32.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.571+0000 7f119ee00640 1 -- 172.21.15.151:0/3955355559 wait complete. 2024-09-16T15:53:32.589 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_less_than_cloner_threads' 2024-09-16T15:53:32.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.746+0000 7fcdd0e00640 1 -- 172.21.15.151:0/3845126179 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcdcc06fb90 msgr2=0x7fcdcc06ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:32.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.746+0000 7fcdd0e00640 1 --2- 172.21.15.151:0/3845126179 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcdcc06fb90 0x7fcdcc06ff70 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc0ce660 tx=0x7fcdbc02a4a0 comp rx=0 tx=0).stop 2024-09-16T15:53:32.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.746+0000 7fcdd0e00640 1 -- 172.21.15.151:0/3845126179 shutdown_connections 2024-09-16T15:53:32.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.746+0000 7fcdd0e00640 1 --2- 172.21.15.151:0/3845126179 >> v2:172.21.15.167:3300/0 conn(0x7fcdcc0cdd00 0x7fcdcc0adcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:32.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.746+0000 7fcdd0e00640 1 --2- 172.21.15.151:0/3845126179 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcdcc06fb90 0x7fcdcc06ff70 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:32.747 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.746+0000 7fcdd0e00640 1 -- 172.21.15.151:0/3845126179 >> 172.21.15.151:0/3845126179 conn(0x7fcdcc0cfa20 msgr2=0x7fcdcc0cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:32.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.746+0000 7fcdd0e00640 1 -- 172.21.15.151:0/3845126179 shutdown_connections 2024-09-16T15:53:32.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.746+0000 7fcdd0e00640 1 -- 172.21.15.151:0/3845126179 wait complete. 2024-09-16T15:53:32.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.747+0000 7fcdd0e00640 1 Processor -- start 2024-09-16T15:53:32.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.747+0000 7fcdd0e00640 1 -- start start 2024-09-16T15:53:32.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.748+0000 7fcdd0e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcdcc06fb90 0x7fcdcc154560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:32.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.748+0000 7fcdd0e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcdcc0ad880 0x7fcdcc0c7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:32.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.748+0000 7fcdd0e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcdcc0cdd00 0x7fcdcc0c8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:32.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.748+0000 7fcdd0e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcdcc15ed70 con 0x7fcdcc0cdd00 2024-09-16T15:53:32.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.748+0000 7fcdd0e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcdcc15ebf0 con 0x7fcdcc06fb90 2024-09-16T15:53:32.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.748+0000 7fcdd0e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fcdcc15eef0 con 0x7fcdcc0ad880 2024-09-16T15:53:32.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.749+0000 7fcdcbe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcdcc0cdd00 0x7fcdcc0c8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:32.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.749+0000 7fcdcbe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcdcc0cdd00 0x7fcdcc0c8080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:42118/0 (socket says 172.21.15.151:42118) 2024-09-16T15:53:32.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.749+0000 7fcdcbe00640 1 -- 172.21.15.151:0/685067535 learned_addr learned my addr 172.21.15.151:0/685067535 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:53:32.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.749+0000 7fcdcb400640 1 --2- 172.21.15.151:0/685067535 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcdcc06fb90 0x7fcdcc154560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:32.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.749+0000 7fcdcaa00640 1 --2- 172.21.15.151:0/685067535 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcdcc0ad880 0x7fcdcc0c7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:32.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.749+0000 7fcdcbe00640 1 -- 172.21.15.151:0/685067535 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcdcc0ad880 msgr2=0x7fcdcc0c7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:32.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.749+0000 7fcdcbe00640 1 --2- 172.21.15.151:0/685067535 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcdcc0ad880 0x7fcdcc0c7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:32.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.749+0000 7fcdcbe00640 1 -- 172.21.15.151:0/685067535 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcdcc06fb90 msgr2=0x7fcdcc154560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:32.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.749+0000 7fcdcbe00640 1 --2- 172.21.15.151:0/685067535 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcdcc06fb90 0x7fcdcc154560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:32.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.749+0000 7fcdcbe00640 1 -- 172.21.15.151:0/685067535 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdcc154aa0 con 0x7fcdcc0cdd00 2024-09-16T15:53:32.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.749+0000 7fcdcbe00640 1 --2- 172.21.15.151:0/685067535 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcdcc0cdd00 0x7fcdcc0c8080 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fcdc400ea80 tx=0x7fcdc400ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:32.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.750+0000 7fcdc3400640 1 -- 172.21.15.151:0/685067535 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc4004b20 con 0x7fcdcc0cdd00 2024-09-16T15:53:32.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.750+0000 7fcdc3400640 1 -- 172.21.15.151:0/685067535 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcdc4004ce0 con 0x7fcdcc0cdd00 2024-09-16T15:53:32.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.750+0000 7fcdc3400640 1 -- 172.21.15.151:0/685067535 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc4005280 con 0x7fcdcc0cdd00 2024-09-16T15:53:32.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.750+0000 7fcdd0e00640 1 -- 172.21.15.151:0/685067535 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdcc0b0f40 con 0x7fcdcc0cdd00 2024-09-16T15:53:32.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.750+0000 7fcdd0e00640 1 -- 172.21.15.151:0/685067535 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdcc1447e0 con 0x7fcdcc0cdd00 2024-09-16T15:53:32.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.751+0000 7fcdd0e00640 1 -- 172.21.15.151:0/685067535 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdcc06ff70 con 0x7fcdcc0cdd00 2024-09-16T15:53:32.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.752+0000 7fcdc3400640 1 -- 172.21.15.151:0/685067535 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 47) ==== 104092+0+0 (secure 0 0 0) 0x7fcdc4005420 con 0x7fcdcc0cdd00 2024-09-16T15:53:32.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.753+0000 7fcdc3400640 1 --2- 172.21.15.151:0/685067535 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcdac07a740 0x7fcdac07cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:32.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.753+0000 7fcdc3400640 1 -- 172.21.15.151:0/685067535 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5055+0+0 (secure 0 0 0) 0x7fcdc409a4f0 con 0x7fcdcc0cdd00 2024-09-16T15:53:32.754 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.753+0000 7fcdcb400640 1 --2- 172.21.15.151:0/685067535 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcdac07a740 0x7fcdac07cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:32.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.754+0000 7fcdcb400640 1 --2- 172.21.15.151:0/685067535 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcdac07a740 0x7fcdac07cc00 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc034000 tx=0x7fcdbc004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:32.758 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:32.757+0000 7fcdc3400640 1 -- 172.21.15.151:0/685067535 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fcdcc154aa0 con 0x7fcdcc0cdd00 2024-09-16T15:53:33.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:33.013+0000 7fcdd0e00640 1 -- 172.21.15.151:0/685067535 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_less_than_cloner_threads"]} v 0) -- 0x7fcdcc15d990 con 0x7fcdcc0cdd00 2024-09-16T15:53:33.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:33.819+0000 7fcdc3400640 1 -- 172.21.15.151:0/685067535 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_less_than_cloner_threads"]}]=0 v649) ==== 189+0+0 (secure 0 0 0) 0x7fcdc4063f70 con 0x7fcdcc0cdd00 2024-09-16T15:53:33.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:33.822+0000 7fcdd0e00640 1 -- 172.21.15.151:0/685067535 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcdac07a740 msgr2=0x7fcdac07cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:33.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:33.822+0000 7fcdd0e00640 1 --2- 172.21.15.151:0/685067535 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcdac07a740 0x7fcdac07cc00 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc034000 tx=0x7fcdbc004730 comp rx=0 tx=0).stop 2024-09-16T15:53:33.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:33.822+0000 7fcdd0e00640 1 -- 172.21.15.151:0/685067535 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcdcc0cdd00 msgr2=0x7fcdcc0c8080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:33.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:33.822+0000 7fcdd0e00640 1 --2- 172.21.15.151:0/685067535 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcdcc0cdd00 0x7fcdcc0c8080 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fcdc400ea80 tx=0x7fcdc400ef40 comp rx=0 tx=0).stop 2024-09-16T15:53:33.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:33.822+0000 7fcdd0e00640 1 -- 172.21.15.151:0/685067535 shutdown_connections 2024-09-16T15:53:33.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:33.822+0000 7fcdd0e00640 1 --2- 172.21.15.151:0/685067535 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcdac07a740 0x7fcdac07cc00 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:33.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:33.822+0000 7fcdd0e00640 1 --2- 172.21.15.151:0/685067535 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcdcc0cdd00 0x7fcdcc0c8080 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:33.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:33.822+0000 7fcdd0e00640 1 --2- 172.21.15.151:0/685067535 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcdcc0ad880 0x7fcdcc0c7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:33.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:33.822+0000 7fcdd0e00640 1 --2- 172.21.15.151:0/685067535 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcdcc06fb90 0x7fcdcc154560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:33.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:33.822+0000 7fcdd0e00640 1 -- 172.21.15.151:0/685067535 >> 172.21.15.151:0/685067535 conn(0x7fcdcc0cfa20 msgr2=0x7fcdcc15cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:33.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:33.822+0000 7fcdd0e00640 1 -- 172.21.15.151:0/685067535 shutdown_connections 2024-09-16T15:53:33.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:33.822+0000 7fcdd0e00640 1 -- 172.21.15.151:0/685067535 wait complete. 2024-09-16T15:53:33.842 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-16T15:53:34.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.006+0000 7f351c000640 1 -- 172.21.15.151:0/131341669 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3514073fb0 msgr2=0x7f3514074440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:34.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.006+0000 7f351c000640 1 --2- 172.21.15.151:0/131341669 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3514073fb0 0x7f3514074440 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f350c005fd0 tx=0x7f350c02a820 comp rx=0 tx=0).stop 2024-09-16T15:53:34.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.006+0000 7f351c000640 1 -- 172.21.15.151:0/131341669 shutdown_connections 2024-09-16T15:53:34.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.006+0000 7f351c000640 1 --2- 172.21.15.151:0/131341669 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3514073fb0 0x7f3514074440 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:34.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.006+0000 7f351c000640 1 --2- 172.21.15.151:0/131341669 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f35140ccf00 0x7f35140b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:34.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.006+0000 7f351c000640 1 -- 172.21.15.151:0/131341669 >> 172.21.15.151:0/131341669 conn(0x7f35140b1950 msgr2=0x7f35140b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:34.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.006+0000 7f351c000640 1 -- 172.21.15.151:0/131341669 shutdown_connections 2024-09-16T15:53:34.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.006+0000 7f351c000640 1 -- 172.21.15.151:0/131341669 wait complete. 2024-09-16T15:53:34.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.007+0000 7f351c000640 1 Processor -- start 2024-09-16T15:53:34.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.008+0000 7f351c000640 1 -- start start 2024-09-16T15:53:34.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.008+0000 7f351c000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3514073fb0 0x7f3514158f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:34.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.008+0000 7f351c000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f35140ccf00 0x7f3514147a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:34.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.008+0000 7f351c000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f351415f5a0 0x7f3514147220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:34.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.008+0000 7f351c000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f351406f2e0 con 0x7f35140ccf00 2024-09-16T15:53:34.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.008+0000 7f351c000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f351406f160 con 0x7f351415f5a0 2024-09-16T15:53:34.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.008+0000 7f351c000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f351406f460 con 0x7f3514073fb0 2024-09-16T15:53:34.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.009+0000 7f3518c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f35140ccf00 0x7f3514147a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:34.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.009+0000 7f3518c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f35140ccf00 0x7f3514147a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:56416/0 (socket says 172.21.15.151:56416) 2024-09-16T15:53:34.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.009+0000 7f3518c00640 1 -- 172.21.15.151:0/739489858 learned_addr learned my addr 172.21.15.151:0/739489858 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:53:34.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.009+0000 7f3519600640 1 --2- 172.21.15.151:0/739489858 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3514073fb0 0x7f3514158f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:34.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.009+0000 7f3518c00640 1 -- 172.21.15.151:0/739489858 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3514073fb0 msgr2=0x7f3514158f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:34.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.009+0000 7f351a000640 1 --2- 172.21.15.151:0/739489858 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f351415f5a0 0x7f3514147220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:34.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.009+0000 7f3518c00640 1 --2- 172.21.15.151:0/739489858 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3514073fb0 0x7f3514158f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:34.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.009+0000 7f3518c00640 1 -- 172.21.15.151:0/739489858 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f351415f5a0 msgr2=0x7f3514147220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:34.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.009+0000 7f3518c00640 1 --2- 172.21.15.151:0/739489858 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f351415f5a0 0x7f3514147220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:34.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.009+0000 7f3518c00640 1 -- 172.21.15.151:0/739489858 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3514147760 con 0x7f35140ccf00 2024-09-16T15:53:34.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.009+0000 7f3519600640 1 --2- 172.21.15.151:0/739489858 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3514073fb0 0x7f3514158f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:53:34.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.009+0000 7f351a000640 1 --2- 172.21.15.151:0/739489858 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f351415f5a0 0x7f3514147220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:53:34.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.009+0000 7f3518c00640 1 --2- 172.21.15.151:0/739489858 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f35140ccf00 0x7f3514147a40 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f350000d950 tx=0x7f350000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:34.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.010+0000 7f3511600640 1 -- 172.21.15.151:0/739489858 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3500017070 con 0x7f35140ccf00 2024-09-16T15:53:34.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.010+0000 7f3511600640 1 -- 172.21.15.151:0/739489858 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3500009d20 con 0x7f35140ccf00 2024-09-16T15:53:34.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.010+0000 7f351c000640 1 -- 172.21.15.151:0/739489858 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35140c4c60 con 0x7f35140ccf00 2024-09-16T15:53:34.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.010+0000 7f3511600640 1 -- 172.21.15.151:0/739489858 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35000106c0 con 0x7f35140ccf00 2024-09-16T15:53:34.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.010+0000 7f351c000640 1 -- 172.21.15.151:0/739489858 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f351414d7e0 con 0x7f35140ccf00 2024-09-16T15:53:34.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.011+0000 7f351c000640 1 -- 172.21.15.151:0/739489858 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f351406f9f0 con 0x7f35140ccf00 2024-09-16T15:53:34.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.012+0000 7f3511600640 1 -- 172.21.15.151:0/739489858 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 47) ==== 104092+0+0 (secure 0 0 0) 0x7f3500010860 con 0x7f35140ccf00 2024-09-16T15:53:34.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.013+0000 7f3511600640 1 --2- 172.21.15.151:0/739489858 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f34f407a740 0x7f34f407cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:34.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.013+0000 7f3511600640 1 -- 172.21.15.151:0/739489858 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5055+0+0 (secure 0 0 0) 0x7f3500099e10 con 0x7f35140ccf00 2024-09-16T15:53:34.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.013+0000 7f3519600640 1 --2- 172.21.15.151:0/739489858 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f34f407a740 0x7f34f407cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:34.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.014+0000 7f3519600640 1 --2- 172.21.15.151:0/739489858 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f34f407a740 0x7f34f407cc00 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f3508001220 tx=0x7f3508009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:34.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.017+0000 7f3511600640 1 -- 172.21.15.151:0/739489858 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f3514147760 con 0x7f35140ccf00 2024-09-16T15:53:34.205 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.204+0000 7f351c000640 1 -- 172.21.15.151:0/739489858 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f35140c50e0 con 0x7f35140ccf00 2024-09-16T15:53:34.206 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.205+0000 7f3511600640 1 -- 172.21.15.151:0/739489858 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v50) ==== 59+0+1998 (secure 0 0 0) 0x7f3500063ee0 con 0x7f35140ccf00 2024-09-16T15:53:34.206 INFO:teuthology.orchestra.run.smithi151.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-16T15:53:34.206 INFO:teuthology.orchestra.run.smithi151.stdout:mon advanced mon_op_complaint_time 120 2024-09-16T15:53:34.206 INFO:teuthology.orchestra.run.smithi151.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-16T15:53:34.206 INFO:teuthology.orchestra.run.smithi151.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-16T15:53:34.206 INFO:teuthology.orchestra.run.smithi151.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3795.338492 2024-09-16T15:53:34.206 INFO:teuthology.orchestra.run.smithi151.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 4030.355715 2024-09-16T15:53:34.206 INFO:teuthology.orchestra.run.smithi151.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3817.533640 2024-09-16T15:53:34.206 INFO:teuthology.orchestra.run.smithi151.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3950.656136 2024-09-16T15:53:34.206 INFO:teuthology.orchestra.run.smithi151.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4449.408052 2024-09-16T15:53:34.206 INFO:teuthology.orchestra.run.smithi151.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4439.932296 2024-09-16T15:53:34.207 INFO:teuthology.orchestra.run.smithi151.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4566.875579 2024-09-16T15:53:34.207 INFO:teuthology.orchestra.run.smithi151.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 6320.073632 2024-09-16T15:53:34.207 INFO:teuthology.orchestra.run.smithi151.stdout:mds advanced debug_mds 20/20 2024-09-16T15:53:34.207 INFO:teuthology.orchestra.run.smithi151.stdout:mds advanced debug_mds_balancer 20/20 2024-09-16T15:53:34.207 INFO:teuthology.orchestra.run.smithi151.stdout:mds advanced debug_ms 2/2 2024-09-16T15:53:34.207 INFO:teuthology.orchestra.run.smithi151.stdout:mds dev mds_debug_frag true 2024-09-16T15:53:34.207 INFO:teuthology.orchestra.run.smithi151.stdout:mds dev mds_debug_scatterstat true 2024-09-16T15:53:34.207 INFO:teuthology.orchestra.run.smithi151.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-16T15:53:34.207 INFO:teuthology.orchestra.run.smithi151.stdout:mds dev mds_verify_scatter true 2024-09-16T15:53:34.207 INFO:teuthology.orchestra.run.smithi151.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-16T15:53:34.207 INFO:teuthology.orchestra.run.smithi151.stdout:mds advanced rados_mon_op_timeout 900 2024-09-16T15:53:34.207 INFO:teuthology.orchestra.run.smithi151.stdout:mds advanced rados_osd_op_timeout 900 2024-09-16T15:53:34.207 INFO:teuthology.orchestra.run.smithi151.stdout:client advanced client_mount_timeout 600 2024-09-16T15:53:34.207 INFO:teuthology.orchestra.run.smithi151.stdout:client advanced debug_client 20/20 2024-09-16T15:53:34.207 INFO:teuthology.orchestra.run.smithi151.stdout:client advanced debug_ms 1/1 2024-09-16T15:53:34.208 INFO:teuthology.orchestra.run.smithi151.stdout:client advanced rados_mon_op_timeout 900 2024-09-16T15:53:34.208 INFO:teuthology.orchestra.run.smithi151.stdout:client advanced rados_osd_op_timeout 900 2024-09-16T15:53:34.208 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.207+0000 7f351c000640 1 -- 172.21.15.151:0/739489858 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f34f407a740 msgr2=0x7f34f407cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:34.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.207+0000 7f351c000640 1 --2- 172.21.15.151:0/739489858 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f34f407a740 0x7f34f407cc00 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f3508001220 tx=0x7f3508009040 comp rx=0 tx=0).stop 2024-09-16T15:53:34.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.207+0000 7f351c000640 1 -- 172.21.15.151:0/739489858 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f35140ccf00 msgr2=0x7f3514147a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:34.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.207+0000 7f351c000640 1 --2- 172.21.15.151:0/739489858 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f35140ccf00 0x7f3514147a40 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f350000d950 tx=0x7f350000de10 comp rx=0 tx=0).stop 2024-09-16T15:53:34.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.207+0000 7f351c000640 1 -- 172.21.15.151:0/739489858 shutdown_connections 2024-09-16T15:53:34.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.207+0000 7f351c000640 1 --2- 172.21.15.151:0/739489858 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f34f407a740 0x7f34f407cc00 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:34.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.207+0000 7f351c000640 1 --2- 172.21.15.151:0/739489858 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f351415f5a0 0x7f3514147220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:34.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.207+0000 7f351c000640 1 --2- 172.21.15.151:0/739489858 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f35140ccf00 0x7f3514147a40 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:34.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.207+0000 7f351c000640 1 --2- 172.21.15.151:0/739489858 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3514073fb0 0x7f3514158f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:34.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.207+0000 7f351c000640 1 -- 172.21.15.151:0/739489858 >> 172.21.15.151:0/739489858 conn(0x7f35140b1950 msgr2=0x7f351409c360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:34.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.207+0000 7f351c000640 1 -- 172.21.15.151:0/739489858 shutdown_connections 2024-09-16T15:53:34.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.207+0000 7f351c000640 1 -- 172.21.15.151:0/739489858 wait complete. 2024-09-16T15:53:34.225 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-16T15:53:34.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.384+0000 7f34bcc00640 1 -- 172.21.15.151:0/1438195631 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f34b80cb4f0 msgr2=0x7f34b80cb9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:34.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.384+0000 7f34bcc00640 1 --2- 172.21.15.151:0/1438195631 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f34b80cb4f0 0x7f34b80cb9a0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f34b8072e80 tx=0x7f34b002a560 comp rx=0 tx=0).stop 2024-09-16T15:53:34.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.384+0000 7f34bcc00640 1 -- 172.21.15.151:0/1438195631 shutdown_connections 2024-09-16T15:53:34.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.384+0000 7f34bcc00640 1 --2- 172.21.15.151:0/1438195631 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f34b80cb4f0 0x7f34b80cb9a0 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:34.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.384+0000 7f34bcc00640 1 --2- 172.21.15.151:0/1438195631 >> v2:172.21.15.167:3300/0 conn(0x7f34b8072560 0x7f34b8073ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:34.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.384+0000 7f34bcc00640 1 --2- 172.21.15.151:0/1438195631 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f34b815bec0 0x7f34b806fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:34.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.384+0000 7f34bcc00640 1 -- 172.21.15.151:0/1438195631 >> 172.21.15.151:0/1438195631 conn(0x7f34b80cf9f0 msgr2=0x7f34b80cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:34.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.384+0000 7f34bcc00640 1 -- 172.21.15.151:0/1438195631 shutdown_connections 2024-09-16T15:53:34.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.384+0000 7f34bcc00640 1 -- 172.21.15.151:0/1438195631 wait complete. 2024-09-16T15:53:34.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.386+0000 7f34bcc00640 1 Processor -- start 2024-09-16T15:53:34.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.386+0000 7f34bcc00640 1 -- start start 2024-09-16T15:53:34.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.387+0000 7f34bcc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f34b8072560 0x7f34b81f2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:34.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.387+0000 7f34bcc00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f34b80cb4f0 0x7f34b80adb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:34.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.387+0000 7f34bcc00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f34b815bec0 0x7f34b8159b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:34.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.387+0000 7f34bcc00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f34b80ae850 con 0x7f34b8072560 2024-09-16T15:53:34.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.387+0000 7f34bcc00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f34b80ae6d0 con 0x7f34b80cb4f0 2024-09-16T15:53:34.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.387+0000 7f34bcc00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f34b80ae9d0 con 0x7f34b815bec0 2024-09-16T15:53:34.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.387+0000 7f34b7400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f34b8072560 0x7f34b81f2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:34.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.387+0000 7f34b7400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f34b8072560 0x7f34b81f2570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:56426/0 (socket says 172.21.15.151:56426) 2024-09-16T15:53:34.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.387+0000 7f34b7400640 1 -- 172.21.15.151:0/3356022190 learned_addr learned my addr 172.21.15.151:0/3356022190 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:53:34.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.387+0000 7f34b6a00640 1 --2- 172.21.15.151:0/3356022190 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f34b80cb4f0 0x7f34b80adb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:34.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.387+0000 7f34b7e00640 1 --2- 172.21.15.151:0/3356022190 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f34b815bec0 0x7f34b8159b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:34.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.388+0000 7f34b7400640 1 -- 172.21.15.151:0/3356022190 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f34b815bec0 msgr2=0x7f34b8159b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:34.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.388+0000 7f34b7400640 1 --2- 172.21.15.151:0/3356022190 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f34b815bec0 0x7f34b8159b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:34.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.388+0000 7f34b7400640 1 -- 172.21.15.151:0/3356022190 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f34b80cb4f0 msgr2=0x7f34b80adb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:34.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.388+0000 7f34b7400640 1 --2- 172.21.15.151:0/3356022190 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f34b80cb4f0 0x7f34b80adb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:34.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.388+0000 7f34b7400640 1 -- 172.21.15.151:0/3356022190 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34b81f2d10 con 0x7f34b8072560 2024-09-16T15:53:34.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.388+0000 7f34b7e00640 1 --2- 172.21.15.151:0/3356022190 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f34b815bec0 0x7f34b8159b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:53:34.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.388+0000 7f34b7400640 1 --2- 172.21.15.151:0/3356022190 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f34b8072560 0x7f34b81f2570 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f34a800ea50 tx=0x7f34a800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:34.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.388+0000 7f34af400640 1 -- 172.21.15.151:0/3356022190 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a800cd00 con 0x7f34b8072560 2024-09-16T15:53:34.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.388+0000 7f34af400640 1 -- 172.21.15.151:0/3356022190 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f34a8011470 con 0x7f34b8072560 2024-09-16T15:53:34.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.388+0000 7f34af400640 1 -- 172.21.15.151:0/3356022190 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a80118a0 con 0x7f34b8072560 2024-09-16T15:53:34.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.388+0000 7f34bcc00640 1 -- 172.21.15.151:0/3356022190 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34b81f2ff0 con 0x7f34b8072560 2024-09-16T15:53:34.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.388+0000 7f34bcc00640 1 -- 172.21.15.151:0/3356022190 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34b80a95f0 con 0x7f34b8072560 2024-09-16T15:53:34.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.389+0000 7f34bcc00640 1 -- 172.21.15.151:0/3356022190 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34b80aee80 con 0x7f34b8072560 2024-09-16T15:53:34.391 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.390+0000 7f34af400640 1 -- 172.21.15.151:0/3356022190 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 47) ==== 104092+0+0 (secure 0 0 0) 0x7f34a8011a40 con 0x7f34b8072560 2024-09-16T15:53:34.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.391+0000 7f34af400640 1 --2- 172.21.15.151:0/3356022190 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f349807a740 0x7f349807cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:34.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.392+0000 7f34af400640 1 -- 172.21.15.151:0/3356022190 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5055+0+0 (secure 0 0 0) 0x7f34a809a540 con 0x7f34b8072560 2024-09-16T15:53:34.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.392+0000 7f34b6a00640 1 --2- 172.21.15.151:0/3356022190 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f349807a740 0x7f349807cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:34.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.393+0000 7f34b6a00640 1 --2- 172.21.15.151:0/3356022190 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f349807a740 0x7f349807cc00 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f34b80cab10 tx=0x7f34a400b310 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:34.396 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.395+0000 7f34af400640 1 -- 172.21.15.151:0/3356022190 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f34a8016030 con 0x7f34b8072560 2024-09-16T15:53:34.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.580+0000 7f34bcc00640 1 -- 172.21.15.151:0/3356022190 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f34b80b06c0 con 0x7f34b8072560 2024-09-16T15:53:34.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.584+0000 7f34af400640 1 -- 172.21.15.151:0/3356022190 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v51)=0 v51) ==== 122+0+0 (secure 0 0 0) 0x7f34a8064610 con 0x7f34b8072560 2024-09-16T15:53:34.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.587+0000 7f34bcc00640 1 -- 172.21.15.151:0/3356022190 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f349807a740 msgr2=0x7f349807cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:34.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.587+0000 7f34bcc00640 1 --2- 172.21.15.151:0/3356022190 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f349807a740 0x7f349807cc00 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f34b80cab10 tx=0x7f34a400b310 comp rx=0 tx=0).stop 2024-09-16T15:53:34.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.587+0000 7f34bcc00640 1 -- 172.21.15.151:0/3356022190 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f34b8072560 msgr2=0x7f34b81f2570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:34.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.587+0000 7f34bcc00640 1 --2- 172.21.15.151:0/3356022190 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f34b8072560 0x7f34b81f2570 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f34a800ea50 tx=0x7f34a800ee50 comp rx=0 tx=0).stop 2024-09-16T15:53:34.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.587+0000 7f34bcc00640 1 -- 172.21.15.151:0/3356022190 shutdown_connections 2024-09-16T15:53:34.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.587+0000 7f34bcc00640 1 --2- 172.21.15.151:0/3356022190 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f349807a740 0x7f349807cc00 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:34.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.587+0000 7f34bcc00640 1 --2- 172.21.15.151:0/3356022190 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f34b80cb4f0 0x7f34b80adb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:34.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.587+0000 7f34bcc00640 1 --2- 172.21.15.151:0/3356022190 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f34b815bec0 0x7f34b8159b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:34.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.587+0000 7f34bcc00640 1 --2- 172.21.15.151:0/3356022190 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f34b8072560 0x7f34b81f2570 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:34.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.587+0000 7f34bcc00640 1 -- 172.21.15.151:0/3356022190 >> 172.21.15.151:0/3356022190 conn(0x7f34b80cf9f0 msgr2=0x7f34b80cab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:34.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.587+0000 7f34bcc00640 1 -- 172.21.15.151:0/3356022190 shutdown_connections 2024-09-16T15:53:34.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:34.587+0000 7f34bcc00640 1 -- 172.21.15.151:0/3356022190 wait complete. 2024-09-16T15:53:34.603 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-16T15:53:34.604 DEBUG:teuthology.orchestra.run.smithi151:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:53:34.703 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:53:34.798 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-16T15:53:34.798 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:53:34.798 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:53:34.834 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T15:53:34.835 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-16T15:53:34.835 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:53:34.871 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:53:34.871 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:53:34.871 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:53:34.910 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T15:53:34.910 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-16T15:53:34.910 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:53:34.947 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:53:34.947 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-16T15:53:34.947 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:53:35.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:35.107+0000 7f08d4a00640 1 -- 172.21.15.151:0/344671128 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f08d006fb90 msgr2=0x7f08d006ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:35.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:35.107+0000 7f08d4a00640 1 --2- 172.21.15.151:0/344671128 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f08d006fb90 0x7f08d006ff70 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f08d0158920 tx=0x7f08c002a560 comp rx=0 tx=0).stop 2024-09-16T15:53:35.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:35.107+0000 7f08d4a00640 1 -- 172.21.15.151:0/344671128 shutdown_connections 2024-09-16T15:53:35.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:35.107+0000 7f08d4a00640 1 --2- 172.21.15.151:0/344671128 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f08d00cd5f0 0x7f08d00cda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:35.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:35.107+0000 7f08d4a00640 1 --2- 172.21.15.151:0/344671128 >> v2:172.21.15.167:3300/0 conn(0x7f08d00ad880 0x7f08d00adcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:35.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:35.107+0000 7f08d4a00640 1 --2- 172.21.15.151:0/344671128 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f08d006fb90 0x7f08d006ff70 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:35.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:35.107+0000 7f08d4a00640 1 -- 172.21.15.151:0/344671128 >> 172.21.15.151:0/344671128 conn(0x7f08d00cfa20 msgr2=0x7f08d00cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:35.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:35.107+0000 7f08d4a00640 1 -- 172.21.15.151:0/344671128 shutdown_connections 2024-09-16T15:53:35.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:35.107+0000 7f08d4a00640 1 -- 172.21.15.151:0/344671128 wait complete. 2024-09-16T15:53:35.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:35.108+0000 7f08d4a00640 1 Processor -- start 2024-09-16T15:53:35.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:35.109+0000 7f08d4a00640 1 -- start start 2024-09-16T15:53:35.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:35.109+0000 7f08d4a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08d006fb90 0x7f08d01f7980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:35.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:35.109+0000 7f08d4a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f08d00ad880 0x7f08d0146a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:35.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:35.109+0000 7f08d4a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f08d00cd5f0 0x7f08d0146f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:35.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:35.109+0000 7f08d4a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f08d015ebf0 con 0x7f08d00cd5f0 2024-09-16T15:53:35.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:35.109+0000 7f08d4a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f08d015ea70 con 0x7f08d006fb90 2024-09-16T15:53:35.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:35.109+0000 7f08d4a00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f08d015ed70 con 0x7f08d00ad880 2024-09-16T15:53:35.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:35.110+0000 7f08cf200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f08d00cd5f0 0x7f08d0146f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:35.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:35.110+0000 7f08cf200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f08d00cd5f0 0x7f08d0146f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:56442/0 (socket says 172.21.15.151:56442) 2024-09-16T15:53:35.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:35.110+0000 7f08cf200640 1 -- 172.21.15.151:0/360159462 learned_addr learned my addr 172.21.15.151:0/360159462 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:53:35.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:35.110+0000 7f08cde00640 1 --2- 172.21.15.151:0/360159462 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f08d00ad880 0x7f08d0146a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:35.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:35.110+0000 7f08ce800640 1 --2- 172.21.15.151:0/360159462 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08d006fb90 0x7f08d01f7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:35.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:35.110+0000 7f08cf200640 1 -- 172.21.15.151:0/360159462 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f08d00ad880 msgr2=0x7f08d0146a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:35.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:35.110+0000 7f08cf200640 1 --2- 172.21.15.151:0/360159462 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f08d00ad880 0x7f08d0146a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:35.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:35.110+0000 7f08cf200640 1 -- 172.21.15.151:0/360159462 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08d006fb90 msgr2=0x7f08d01f7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:35.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:35.110+0000 7f08cf200640 1 --2- 172.21.15.151:0/360159462 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08d006fb90 0x7f08d01f7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:35.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:35.110+0000 7f08cf200640 1 -- 172.21.15.151:0/360159462 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08d00c9670 con 0x7f08d00cd5f0 2024-09-16T15:53:35.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:35.111+0000 7f08cf200640 1 --2- 172.21.15.151:0/360159462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f08d00cd5f0 0x7f08d0146f60 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f08c800da10 tx=0x7f08c800ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:35.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:35.111+0000 7f08c6a00640 1 -- 172.21.15.151:0/360159462 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c8017070 con 0x7f08d00cd5f0 2024-09-16T15:53:35.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:35.111+0000 7f08c6a00640 1 -- 172.21.15.151:0/360159462 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f08c8003970 con 0x7f08d00cd5f0 2024-09-16T15:53:35.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:35.111+0000 7f08c6a00640 1 -- 172.21.15.151:0/360159462 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c800be60 con 0x7f08d00cd5f0 2024-09-16T15:53:35.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:35.111+0000 7f08d4a00640 1 -- 172.21.15.151:0/360159462 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08d00c8d50 con 0x7f08d00cd5f0 2024-09-16T15:53:35.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:35.111+0000 7f08d4a00640 1 -- 172.21.15.151:0/360159462 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08d00b5900 con 0x7f08d00cd5f0 2024-09-16T15:53:35.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:35.112+0000 7f08d4a00640 1 -- 172.21.15.151:0/360159462 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08d00adcd0 con 0x7f08d00cd5f0 2024-09-16T15:53:35.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:35.113+0000 7f08c6a00640 1 -- 172.21.15.151:0/360159462 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 47) ==== 104092+0+0 (secure 0 0 0) 0x7f08c8010460 con 0x7f08d00cd5f0 2024-09-16T15:53:35.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:35.114+0000 7f08c6a00640 1 --2- 172.21.15.151:0/360159462 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f08b007a740 0x7f08b007cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:35.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:35.114+0000 7f08c6a00640 1 -- 172.21.15.151:0/360159462 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5055+0+0 (secure 0 0 0) 0x7f08c8099dd0 con 0x7f08d00cd5f0 2024-09-16T15:53:35.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:35.114+0000 7f08ce800640 1 --2- 172.21.15.151:0/360159462 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f08b007a740 0x7f08b007cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:35.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:35.115+0000 7f08ce800640 1 --2- 172.21.15.151:0/360159462 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f08b007a740 0x7f08b007cc00 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f08c0033000 tx=0x7f08c0004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:35.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:35.118+0000 7f08c6a00640 1 -- 172.21.15.151:0/360159462 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f08c80a5080 con 0x7f08d00cd5f0 2024-09-16T15:53:35.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:35.339+0000 7f08d4a00640 1 -- 172.21.15.151:0/360159462 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f08d0161250 con 0x7f08d00cd5f0 2024-09-16T15:53:35.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:35.340+0000 7f08c6a00640 1 -- 172.21.15.151:0/360159462 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) ==== 94+0+3920 (secure 0 0 0) 0x7f08c8063f40 con 0x7f08d00cd5f0 2024-09-16T15:53:35.341 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:53:35.341 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":79,"btime":"2024-09-16T15:53:28:672753+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7034,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/1802374613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1802374613},{"type":"v1","addr":"172.21.15.167:6837","nonce":1802374613}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":7753,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3585191088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":3585191088},{"type":"v1","addr":"172.21.15.167:6836","nonce":3585191088}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":8457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/1321222227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1321222227},{"type":"v1","addr":"172.21.15.151:6834","nonce":1321222227}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":8904,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/808008231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":808008231},{"type":"v1","addr":"172.21.15.151:6835","nonce":808008231}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[]} 2024-09-16T15:53:35.341 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 79 2024-09-16T15:53:35.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:35.342+0000 7f08d4a00640 1 -- 172.21.15.151:0/360159462 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f08b007a740 msgr2=0x7f08b007cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:35.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:35.342+0000 7f08d4a00640 1 --2- 172.21.15.151:0/360159462 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f08b007a740 0x7f08b007cc00 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f08c0033000 tx=0x7f08c0004730 comp rx=0 tx=0).stop 2024-09-16T15:53:35.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:35.342+0000 7f08d4a00640 1 -- 172.21.15.151:0/360159462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f08d00cd5f0 msgr2=0x7f08d0146f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:35.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:35.342+0000 7f08d4a00640 1 --2- 172.21.15.151:0/360159462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f08d00cd5f0 0x7f08d0146f60 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f08c800da10 tx=0x7f08c800ded0 comp rx=0 tx=0).stop 2024-09-16T15:53:35.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:35.342+0000 7f08d4a00640 1 -- 172.21.15.151:0/360159462 shutdown_connections 2024-09-16T15:53:35.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:35.342+0000 7f08d4a00640 1 --2- 172.21.15.151:0/360159462 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f08b007a740 0x7f08b007cc00 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:35.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:35.342+0000 7f08d4a00640 1 --2- 172.21.15.151:0/360159462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f08d00cd5f0 0x7f08d0146f60 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:35.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:35.342+0000 7f08d4a00640 1 --2- 172.21.15.151:0/360159462 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f08d00ad880 0x7f08d0146a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:35.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:35.342+0000 7f08d4a00640 1 --2- 172.21.15.151:0/360159462 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08d006fb90 0x7f08d01f7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:35.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:35.342+0000 7f08d4a00640 1 -- 172.21.15.151:0/360159462 >> 172.21.15.151:0/360159462 conn(0x7f08d00cfa20 msgr2=0x7f08d006e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:35.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:35.342+0000 7f08d4a00640 1 -- 172.21.15.151:0/360159462 shutdown_connections 2024-09-16T15:53:35.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:35.342+0000 7f08d4a00640 1 -- 172.21.15.151:0/360159462 wait complete. 2024-09-16T15:53:35.360 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-16T15:53:35.360 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-16T15:53:35.360 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-16T15:53:35.360 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:53:35.360 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-16T15:53:35.361 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-16T15:53:35.361 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-16T15:53:35.361 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:53:35.361 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-16T15:53:35.361 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-16T15:53:35.361 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-16T15:53:35.361 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:53:35.361 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-16T15:53:35.361 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-16T15:53:35.362 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-16T15:53:35.362 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:53:35.375 INFO:tasks.ceph.mds.c:Stopped 2024-09-16T15:53:35.375 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-16T15:53:35.378 INFO:tasks.ceph.mds.b:Stopped 2024-09-16T15:53:35.378 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-16T15:53:35.379 INFO:tasks.ceph.mds.d:Stopped 2024-09-16T15:53:35.379 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-16T15:53:35.419 INFO:tasks.ceph.mds.c:Started 2024-09-16T15:53:35.419 DEBUG:teuthology.parallel:result is None 2024-09-16T15:53:35.423 INFO:tasks.ceph.mds.b:Started 2024-09-16T15:53:35.423 DEBUG:teuthology.parallel:result is None 2024-09-16T15:53:35.425 INFO:tasks.ceph.mds.d:Started 2024-09-16T15:53:35.425 DEBUG:teuthology.parallel:result is None 2024-09-16T15:53:35.545 INFO:tasks.ceph.mds.c.smithi151.stdout:starting mds.c at 2024-09-16T15:53:35.553 INFO:tasks.ceph.mds.b.smithi167.stdout:starting mds.b at 2024-09-16T15:53:35.553 INFO:tasks.ceph.mds.d.smithi167.stdout:starting mds.d at 2024-09-16T15:53:39.079 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-09-16T15:53:41.375 INFO:tasks.ceph.mds.a:Stopped 2024-09-16T15:53:41.375 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-16T15:53:41.380 INFO:tasks.ceph.mds.a:Started 2024-09-16T15:53:41.380 DEBUG:teuthology.parallel:result is None 2024-09-16T15:53:41.381 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-16T15:53:41.518 INFO:tasks.ceph.mds.a.smithi151.stdout:starting mds.a at 2024-09-16T15:53:41.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:41.550+0000 7fc255400640 1 -- 172.21.15.151:0/287537751 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc250073c20 msgr2=0x7fc250154720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:41.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:41.550+0000 7fc255400640 1 --2- 172.21.15.151:0/287537751 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc250073c20 0x7fc250154720 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc248005fd0 tx=0x7fc24802aaa0 comp rx=0 tx=0).stop 2024-09-16T15:53:41.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:41.550+0000 7fc255400640 1 -- 172.21.15.151:0/287537751 shutdown_connections 2024-09-16T15:53:41.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:41.550+0000 7fc255400640 1 --2- 172.21.15.151:0/287537751 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc250073c20 0x7fc250154720 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:41.552 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:41.550+0000 7fc255400640 1 --2- 172.21.15.151:0/287537751 >> v2:172.21.15.167:3300/0 conn(0x7fc25015bb90 0x7fc2500736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:41.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:41.550+0000 7fc255400640 1 -- 172.21.15.151:0/287537751 >> 172.21.15.151:0/287537751 conn(0x7fc2500d0b30 msgr2=0x7fc2500d0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:41.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:41.550+0000 7fc255400640 1 -- 172.21.15.151:0/287537751 shutdown_connections 2024-09-16T15:53:41.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:41.550+0000 7fc255400640 1 -- 172.21.15.151:0/287537751 wait complete. 2024-09-16T15:53:41.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:41.552+0000 7fc255400640 1 Processor -- start 2024-09-16T15:53:41.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:41.552+0000 7fc255400640 1 -- start start 2024-09-16T15:53:41.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:41.553+0000 7fc255400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc250071130 0x7fc2501e8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:41.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:41.553+0000 7fc255400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc250073c20 0x7fc2501e8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:41.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:41.553+0000 7fc255400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc25015bb90 0x7fc250150890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:41.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:41.553+0000 7fc255400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc250154de0 con 0x7fc25015bb90 2024-09-16T15:53:41.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:41.553+0000 7fc255400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc250154c60 con 0x7fc250073c20 2024-09-16T15:53:41.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:41.553+0000 7fc255400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fc250154f60 con 0x7fc250071130 2024-09-16T15:53:41.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:41.553+0000 7fc254a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc25015bb90 0x7fc250150890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:41.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:41.553+0000 7fc24fe00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc250071130 0x7fc2501e8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:41.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:41.553+0000 7fc254a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc25015bb90 0x7fc250150890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:56484/0 (socket says 172.21.15.151:56484) 2024-09-16T15:53:41.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:41.553+0000 7fc254a00640 1 -- 172.21.15.151:0/3288638128 learned_addr learned my addr 172.21.15.151:0/3288638128 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:53:41.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:41.553+0000 7fc24f400640 1 --2- 172.21.15.151:0/3288638128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc250073c20 0x7fc2501e8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:41.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:41.553+0000 7fc24fe00640 1 -- 172.21.15.151:0/3288638128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc250073c20 msgr2=0x7fc2501e8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:41.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:41.553+0000 7fc24fe00640 1 --2- 172.21.15.151:0/3288638128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc250073c20 0x7fc2501e8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:41.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:41.553+0000 7fc24fe00640 1 -- 172.21.15.151:0/3288638128 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc25015bb90 msgr2=0x7fc250150890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:41.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:41.553+0000 7fc24fe00640 1 --2- 172.21.15.151:0/3288638128 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc25015bb90 0x7fc250150890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:41.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:41.553+0000 7fc24fe00640 1 -- 172.21.15.151:0/3288638128 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2500cacc0 con 0x7fc250071130 2024-09-16T15:53:41.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:41.554+0000 7fc24fe00640 1 --2- 172.21.15.151:0/3288638128 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc250071130 0x7fc2501e8150 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc24000c3c0 tx=0x7fc24000c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:41.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:41.554+0000 7fc247e00640 1 -- 172.21.15.151:0/3288638128 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc240017070 con 0x7fc250071130 2024-09-16T15:53:41.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:41.554+0000 7fc247e00640 1 -- 172.21.15.151:0/3288638128 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc2400053c0 con 0x7fc250071130 2024-09-16T15:53:41.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:41.554+0000 7fc247e00640 1 -- 172.21.15.151:0/3288638128 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc240003850 con 0x7fc250071130 2024-09-16T15:53:41.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:41.554+0000 7fc255400640 1 -- 172.21.15.151:0/3288638128 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc250152030 con 0x7fc250071130 2024-09-16T15:53:41.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:41.554+0000 7fc255400640 1 -- 172.21.15.151:0/3288638128 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2501525f0 con 0x7fc250071130 2024-09-16T15:53:41.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:41.555+0000 7fc255400640 1 -- 172.21.15.151:0/3288638128 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc250154720 con 0x7fc250071130 2024-09-16T15:53:41.558 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:41.557+0000 7fc247e00640 1 -- 172.21.15.151:0/3288638128 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 47) ==== 104092+0+0 (secure 0 0 0) 0x7fc24001a070 con 0x7fc250071130 2024-09-16T15:53:41.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:41.557+0000 7fc247e00640 1 --2- 172.21.15.151:0/3288638128 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc23007a740 0x7fc23007cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:41.559 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:41.558+0000 7fc247e00640 1 -- 172.21.15.151:0/3288638128 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(127..127 src has 1..127) ==== 5055+0+0 (secure 0 0 0) 0x7fc240098bc0 con 0x7fc250071130 2024-09-16T15:53:41.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:41.558+0000 7fc24f400640 1 --2- 172.21.15.151:0/3288638128 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc23007a740 0x7fc23007cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:41.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:41.558+0000 7fc24f400640 1 --2- 172.21.15.151:0/3288638128 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc23007a740 0x7fc23007cc00 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fc2500b6530 tx=0x7fc23c005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:41.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:41.562+0000 7fc247e00640 1 -- 172.21.15.151:0/3288638128 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc240062640 con 0x7fc250071130 2024-09-16T15:53:41.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:41.757+0000 7fc255400640 1 -- 172.21.15.151:0/3288638128 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fc2500d1a10 con 0x7fc250071130 2024-09-16T15:53:43.001 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:42.999+0000 7fc247e00640 1 -- 172.21.15.151:0/3288638128 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v129) ==== 97+0+0 (secure 0 0 0) 0x7fc240067070 con 0x7fc250071130 2024-09-16T15:53:43.001 INFO:teuthology.orchestra.run.smithi151.stderr: removed all blocklist entries 2024-09-16T15:53:43.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.001+0000 7fc255400640 1 -- 172.21.15.151:0/3288638128 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc23007a740 msgr2=0x7fc23007cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:43.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.001+0000 7fc255400640 1 --2- 172.21.15.151:0/3288638128 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc23007a740 0x7fc23007cc00 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fc2500b6530 tx=0x7fc23c005f90 comp rx=0 tx=0).stop 2024-09-16T15:53:43.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.001+0000 7fc255400640 1 -- 172.21.15.151:0/3288638128 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc250071130 msgr2=0x7fc2501e8150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:43.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.001+0000 7fc255400640 1 --2- 172.21.15.151:0/3288638128 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc250071130 0x7fc2501e8150 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc24000c3c0 tx=0x7fc24000c880 comp rx=0 tx=0).stop 2024-09-16T15:53:43.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.001+0000 7fc255400640 1 -- 172.21.15.151:0/3288638128 shutdown_connections 2024-09-16T15:53:43.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.001+0000 7fc255400640 1 --2- 172.21.15.151:0/3288638128 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc23007a740 0x7fc23007cc00 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:43.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.001+0000 7fc255400640 1 --2- 172.21.15.151:0/3288638128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc250073c20 0x7fc2501e8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:43.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.001+0000 7fc255400640 1 --2- 172.21.15.151:0/3288638128 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc25015bb90 0x7fc250150890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:43.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.001+0000 7fc255400640 1 --2- 172.21.15.151:0/3288638128 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc250071130 0x7fc2501e8150 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:43.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.001+0000 7fc255400640 1 -- 172.21.15.151:0/3288638128 >> 172.21.15.151:0/3288638128 conn(0x7fc2500d0b30 msgr2=0x7fc2500ccd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:43.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.002+0000 7fc255400640 1 -- 172.21.15.151:0/3288638128 shutdown_connections 2024-09-16T15:53:43.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.002+0000 7fc255400640 1 -- 172.21.15.151:0/3288638128 wait complete. 2024-09-16T15:53:43.022 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-16T15:53:43.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.179+0000 7f9c20a00640 1 -- 172.21.15.151:0/853604243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c1c0af3d0 msgr2=0x7f9c1c06f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:43.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.179+0000 7f9c20a00640 1 --2- 172.21.15.151:0/853604243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c1c0af3d0 0x7f9c1c06f490 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f9c08009830 tx=0x7f9c0802a480 comp rx=0 tx=0).stop 2024-09-16T15:53:43.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.179+0000 7f9c20a00640 1 -- 172.21.15.151:0/853604243 shutdown_connections 2024-09-16T15:53:43.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.179+0000 7f9c20a00640 1 --2- 172.21.15.151:0/853604243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c1c0af3d0 0x7f9c1c06f490 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:43.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.179+0000 7f9c20a00640 1 --2- 172.21.15.151:0/853604243 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9c1c161550 0x7f9c1c06ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:43.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.179+0000 7f9c20a00640 1 -- 172.21.15.151:0/853604243 >> 172.21.15.151:0/853604243 conn(0x7f9c1c0b1950 msgr2=0x7f9c1c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:43.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.179+0000 7f9c20a00640 1 -- 172.21.15.151:0/853604243 shutdown_connections 2024-09-16T15:53:43.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.179+0000 7f9c20a00640 1 -- 172.21.15.151:0/853604243 wait complete. 2024-09-16T15:53:43.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.180+0000 7f9c20a00640 1 Processor -- start 2024-09-16T15:53:43.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.180+0000 7f9c20a00640 1 -- start start 2024-09-16T15:53:43.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.181+0000 7f9c20a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9c1c06f9d0 0x7f9c1c0cec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:43.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.181+0000 7f9c20a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9c1c0af3d0 0x7f9c1c15a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:43.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.181+0000 7f9c20a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c1c161550 0x7f9c1c1f5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:43.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.181+0000 7f9c20a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9c1c1623a0 con 0x7f9c1c06f9d0 2024-09-16T15:53:43.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.181+0000 7f9c20a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9c1c074010 con 0x7f9c1c161550 2024-09-16T15:53:43.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.181+0000 7f9c20a00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f9c1c162520 con 0x7f9c1c0af3d0 2024-09-16T15:53:43.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.182+0000 7f9c19e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9c1c06f9d0 0x7f9c1c0cec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:43.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.182+0000 7f9c19e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9c1c06f9d0 0x7f9c1c0cec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43542/0 (socket says 172.21.15.151:43542) 2024-09-16T15:53:43.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.182+0000 7f9c19e00640 1 -- 172.21.15.151:0/1176502765 learned_addr learned my addr 172.21.15.151:0/1176502765 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:53:43.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.182+0000 7f9c19400640 1 --2- 172.21.15.151:0/1176502765 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9c1c0af3d0 0x7f9c1c15a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:43.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.182+0000 7f9c1a800640 1 --2- 172.21.15.151:0/1176502765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c1c161550 0x7f9c1c1f5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:43.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.182+0000 7f9c19e00640 1 -- 172.21.15.151:0/1176502765 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9c1c0af3d0 msgr2=0x7f9c1c15a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:43.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.182+0000 7f9c19e00640 1 --2- 172.21.15.151:0/1176502765 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9c1c0af3d0 0x7f9c1c15a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:43.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.182+0000 7f9c19e00640 1 -- 172.21.15.151:0/1176502765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c1c161550 msgr2=0x7f9c1c1f5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:43.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.182+0000 7f9c19e00640 1 --2- 172.21.15.151:0/1176502765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c1c161550 0x7f9c1c1f5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:43.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.182+0000 7f9c19e00640 1 -- 172.21.15.151:0/1176502765 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c1c1f5a50 con 0x7f9c1c06f9d0 2024-09-16T15:53:43.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.182+0000 7f9c19e00640 1 --2- 172.21.15.151:0/1176502765 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9c1c06f9d0 0x7f9c1c0cec20 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c00e790 tx=0x7f9c0c00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:43.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.182+0000 7f9c12000640 1 -- 172.21.15.151:0/1176502765 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c00eea0 con 0x7f9c1c06f9d0 2024-09-16T15:53:43.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.182+0000 7f9c12000640 1 -- 172.21.15.151:0/1176502765 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9c0c00cdb0 con 0x7f9c1c06f9d0 2024-09-16T15:53:43.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.182+0000 7f9c12000640 1 -- 172.21.15.151:0/1176502765 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c010690 con 0x7f9c1c06f9d0 2024-09-16T15:53:43.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.183+0000 7f9c20a00640 1 -- 172.21.15.151:0/1176502765 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c1c0cdb80 con 0x7f9c1c06f9d0 2024-09-16T15:53:43.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.183+0000 7f9c20a00640 1 -- 172.21.15.151:0/1176502765 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c1c0ce140 con 0x7f9c1c06f9d0 2024-09-16T15:53:43.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.184+0000 7f9c20a00640 1 -- 172.21.15.151:0/1176502765 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c1c0760b0 con 0x7f9c1c06f9d0 2024-09-16T15:53:43.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.185+0000 7f9c12000640 1 -- 172.21.15.151:0/1176502765 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 47) ==== 104092+0+0 (secure 0 0 0) 0x7f9c0c010830 con 0x7f9c1c06f9d0 2024-09-16T15:53:43.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.186+0000 7f9c12000640 1 --2- 172.21.15.151:0/1176502765 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9bfc07a770 0x7f9bfc07cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:43.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.186+0000 7f9c12000640 1 -- 172.21.15.151:0/1176502765 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 4969+0+0 (secure 0 0 0) 0x7f9c0c09a330 con 0x7f9c1c06f9d0 2024-09-16T15:53:43.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.186+0000 7f9c19400640 1 --2- 172.21.15.151:0/1176502765 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9bfc07a770 0x7f9bfc07cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:43.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.187+0000 7f9c19400640 1 --2- 172.21.15.151:0/1176502765 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9bfc07a770 0x7f9bfc07cc30 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f9c08034000 tx=0x7f9c08004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:43.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.190+0000 7f9c12000640 1 -- 172.21.15.151:0/1176502765 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9c0c064400 con 0x7f9c1c06f9d0 2024-09-16T15:53:43.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.443+0000 7f9c20a00640 1 -- 172.21.15.151:0/1176502765 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f9c1c162ab0 con 0x7f9c1c06f9d0 2024-09-16T15:53:43.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.444+0000 7f9c12000640 1 -- 172.21.15.151:0/1176502765 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) ==== 80+0+5304 (secure 0 0 0) 0x7f9c0c063d00 con 0x7f9c1c06f9d0 2024-09-16T15:53:43.446 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:53:43.446 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-16T15:53:43.446 INFO:teuthology.orchestra.run.smithi151.stdout: "auth_dump": [ 2024-09-16T15:53:43.446 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:53:43.446 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "mds.a", 2024-09-16T15:53:43.446 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDsUOhmJVK6AhAANfmTPS6kKn41LSD/MuyW9g==", 2024-09-16T15:53:43.446 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:53:43.446 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:53:43.447 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow *", 2024-09-16T15:53:43.447 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow *", 2024-09-16T15:53:43.447 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:53:43.447 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:53:43.447 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:53:43.447 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:53:43.447 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "mds.b", 2024-09-16T15:53:43.447 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDsUOhmX1R3FhAAUQgUsE/6wUyokXcZuBuIWg==", 2024-09-16T15:53:43.447 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:53:43.447 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:53:43.447 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow *", 2024-09-16T15:53:43.448 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow *", 2024-09-16T15:53:43.448 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:53:43.448 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:53:43.448 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:53:43.448 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:53:43.448 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "mds.c", 2024-09-16T15:53:43.448 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDsUOhmYGCKDBAAY2EFNmiFNf1x2so9wxp4YQ==", 2024-09-16T15:53:43.448 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:53:43.448 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:53:43.448 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow *", 2024-09-16T15:53:43.448 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow *", 2024-09-16T15:53:43.448 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:53:43.448 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:53:43.448 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:53:43.448 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:53:43.449 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "mds.d", 2024-09-16T15:53:43.449 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDsUOhmfCk4IBAAx12Gw2QudQ2nrPb8VlE0SA==", 2024-09-16T15:53:43.449 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:53:43.449 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:53:43.449 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow *", 2024-09-16T15:53:43.449 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow *", 2024-09-16T15:53:43.449 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:53:43.449 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:53:43.449 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:53:43.449 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:53:43.449 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.0", 2024-09-16T15:53:43.449 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDvUOhmKji5NxAAn/E3YQR7jeMnBTLBXc1G6A==", 2024-09-16T15:53:43.449 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:53:43.450 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:53:43.450 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:53:43.450 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:53:43.450 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:53:43.450 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:53:43.450 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:53:43.450 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.1", 2024-09-16T15:53:43.450 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDzUOhmmBrNBhAA0Uwh4Fam2bmKKvz5jAQBWw==", 2024-09-16T15:53:43.450 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:53:43.450 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:53:43.450 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:53:43.450 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:53:43.450 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:53:43.450 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:53:43.450 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:53:43.451 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.2", 2024-09-16T15:53:43.451 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQD2UOhmOV6GERAAjD+PQLTZrCFR1669UEEIAQ==", 2024-09-16T15:53:43.451 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:53:43.451 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:53:43.451 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:53:43.451 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:53:43.451 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:53:43.451 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:53:43.451 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:53:43.451 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.3", 2024-09-16T15:53:43.451 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQD5UOhmCRrdGRAAQC0bqWX6N1jniVGEQ2aAvg==", 2024-09-16T15:53:43.451 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:53:43.451 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:53:43.451 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:53:43.451 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:53:43.452 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:53:43.452 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:53:43.452 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:53:43.452 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.4", 2024-09-16T15:53:43.452 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQD/UOhmXI/nGRAA7t6XveKvrWgqy6rHuMQSZQ==", 2024-09-16T15:53:43.452 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:53:43.452 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:53:43.452 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:53:43.452 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:53:43.452 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:53:43.452 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:53:43.452 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:53:43.452 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.5", 2024-09-16T15:53:43.452 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQACUehmKM//IxAAsoFpiiTXNs//v3YXeUf/Iw==", 2024-09-16T15:53:43.453 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:53:43.453 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:53:43.453 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:53:43.453 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:53:43.453 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:53:43.453 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:53:43.453 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:53:43.453 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.6", 2024-09-16T15:53:43.453 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQAFUehmlo/FLRAAp5mvUSWCaBbm85QyTZY0eA==", 2024-09-16T15:53:43.453 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:53:43.453 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:53:43.453 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:53:43.453 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:53:43.453 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:53:43.454 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:53:43.454 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:53:43.454 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.7", 2024-09-16T15:53:43.454 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQAIUehmR32JNxAA27antOIKot1nScQbLZDf5g==", 2024-09-16T15:53:43.454 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:53:43.454 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:53:43.454 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:53:43.454 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:53:43.454 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:53:43.454 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:53:43.454 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:53:43.454 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "client.0", 2024-09-16T15:53:43.454 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDsUOhmf5PBKBAAKE4Y4zbA4h07IBNjPkyptw==", 2024-09-16T15:53:43.454 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:53:43.454 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:53:43.455 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow r", 2024-09-16T15:53:43.455 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-16T15:53:43.455 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:53:43.455 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:53:43.455 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:53:43.455 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "client.1", 2024-09-16T15:53:43.455 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDsUOhm15emNRAAwhK7QmA2c9IYGn04ERvY0g==", 2024-09-16T15:53:43.455 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:53:43.455 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:53:43.455 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow r", 2024-09-16T15:53:43.455 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow rw", 2024-09-16T15:53:43.455 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow rwx" 2024-09-16T15:53:43.455 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:53:43.455 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:53:43.455 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:53:43.456 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "client.2", 2024-09-16T15:53:43.456 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDsUOhm6zUyLxAAkzeiIQTuNZkMM+Oit9rC5A==", 2024-09-16T15:53:43.456 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:53:43.456 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:53:43.456 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow r", 2024-09-16T15:53:43.456 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow rw", 2024-09-16T15:53:43.456 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow rwx" 2024-09-16T15:53:43.456 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:53:43.456 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:53:43.456 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:53:43.456 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "client.3", 2024-09-16T15:53:43.456 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDtUOhmD82EABAAeDf82/hPxpPfAUk8oj5h/Q==", 2024-09-16T15:53:43.456 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:53:43.456 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:53:43.456 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow r", 2024-09-16T15:53:43.457 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow rw", 2024-09-16T15:53:43.457 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow rwx" 2024-09-16T15:53:43.457 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:53:43.457 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:53:43.457 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:53:43.457 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "client.admin", 2024-09-16T15:53:43.457 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDrUOhmQFmVDRAA8NKysbAe475xvg4CPAcFyw==", 2024-09-16T15:53:43.457 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:53:43.457 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow *", 2024-09-16T15:53:43.457 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow *", 2024-09-16T15:53:43.457 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow *", 2024-09-16T15:53:43.457 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:53:43.457 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:53:43.457 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:53:43.457 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:53:43.457 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "mgr.x", 2024-09-16T15:53:43.458 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDrUOhmloW3NxAA8UoLUfrw3wf1nzKbmtK/jw==", 2024-09-16T15:53:43.458 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:53:43.458 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow *", 2024-09-16T15:53:43.458 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile mgr", 2024-09-16T15:53:43.458 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:53:43.458 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:53:43.458 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:53:43.458 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:53:43.458 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "mgr.y", 2024-09-16T15:53:43.458 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDrUOhm1uZOLxAATgjA79vYf0psqQVo03sdDA==", 2024-09-16T15:53:43.458 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:53:43.458 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow *", 2024-09-16T15:53:43.458 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile mgr", 2024-09-16T15:53:43.458 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:53:43.458 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:53:43.458 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:53:43.459 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-09-16T15:53:43.459 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-16T15:53:43.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.446+0000 7f9c20a00640 1 -- 172.21.15.151:0/1176502765 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9bfc07a770 msgr2=0x7f9bfc07cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:43.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.446+0000 7f9c20a00640 1 --2- 172.21.15.151:0/1176502765 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9bfc07a770 0x7f9bfc07cc30 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f9c08034000 tx=0x7f9c08004730 comp rx=0 tx=0).stop 2024-09-16T15:53:43.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.446+0000 7f9c20a00640 1 -- 172.21.15.151:0/1176502765 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9c1c06f9d0 msgr2=0x7f9c1c0cec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:43.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.446+0000 7f9c20a00640 1 --2- 172.21.15.151:0/1176502765 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9c1c06f9d0 0x7f9c1c0cec20 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c00e790 tx=0x7f9c0c00ec50 comp rx=0 tx=0).stop 2024-09-16T15:53:43.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.446+0000 7f9c20a00640 1 -- 172.21.15.151:0/1176502765 shutdown_connections 2024-09-16T15:53:43.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.446+0000 7f9c20a00640 1 --2- 172.21.15.151:0/1176502765 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9bfc07a770 0x7f9bfc07cc30 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:43.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.446+0000 7f9c20a00640 1 --2- 172.21.15.151:0/1176502765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9c1c161550 0x7f9c1c1f5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:43.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.446+0000 7f9c20a00640 1 --2- 172.21.15.151:0/1176502765 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9c1c0af3d0 0x7f9c1c15a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:43.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.446+0000 7f9c20a00640 1 --2- 172.21.15.151:0/1176502765 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9c1c06f9d0 0x7f9c1c0cec20 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:43.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.446+0000 7f9c20a00640 1 -- 172.21.15.151:0/1176502765 >> 172.21.15.151:0/1176502765 conn(0x7f9c1c0b1950 msgr2=0x7f9c1c15ef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:43.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.446+0000 7f9c20a00640 1 -- 172.21.15.151:0/1176502765 shutdown_connections 2024-09-16T15:53:43.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.446+0000 7f9c20a00640 1 -- 172.21.15.151:0/1176502765 wait complete. 2024-09-16T15:53:43.465 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-16T15:53:43.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.621+0000 7fac86a00640 1 -- 172.21.15.151:0/318417151 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac80073fb0 msgr2=0x7fac80074420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:43.624 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.621+0000 7fac86a00640 1 --2- 172.21.15.151:0/318417151 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac80073fb0 0x7fac80074420 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fac800cfbb0 tx=0x7fac7802a560 comp rx=0 tx=0).stop 2024-09-16T15:53:43.624 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.621+0000 7fac86a00640 1 -- 172.21.15.151:0/318417151 shutdown_connections 2024-09-16T15:53:43.624 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.621+0000 7fac86a00640 1 --2- 172.21.15.151:0/318417151 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac80073fb0 0x7fac80074420 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:43.624 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.621+0000 7fac86a00640 1 --2- 172.21.15.151:0/318417151 >> v2:172.21.15.167:3300/0 conn(0x7fac800ccf00 0x7fac800b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:43.624 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.621+0000 7fac86a00640 1 -- 172.21.15.151:0/318417151 >> 172.21.15.151:0/318417151 conn(0x7fac800b1950 msgr2=0x7fac800b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:43.624 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.621+0000 7fac86a00640 1 -- 172.21.15.151:0/318417151 shutdown_connections 2024-09-16T15:53:43.624 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.621+0000 7fac86a00640 1 -- 172.21.15.151:0/318417151 wait complete. 2024-09-16T15:53:43.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.623+0000 7fac86a00640 1 Processor -- start 2024-09-16T15:53:43.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.623+0000 7fac86a00640 1 -- start start 2024-09-16T15:53:43.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.624+0000 7fac86a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac80073fb0 0x7fac801528d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:43.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.624+0000 7fac86a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac800ccf00 0x7fac801f6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:43.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.624+0000 7fac86a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac8015f5a0 0x7fac801f6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:43.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.624+0000 7fac86a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fac8006f2e0 con 0x7fac8015f5a0 2024-09-16T15:53:43.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.624+0000 7fac86a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fac8006f160 con 0x7fac800ccf00 2024-09-16T15:53:43.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.624+0000 7fac86a00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fac8006f460 con 0x7fac80073fb0 2024-09-16T15:53:43.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.624+0000 7fac84a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac8015f5a0 0x7fac801f6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:43.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.624+0000 7fac84a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac8015f5a0 0x7fac801f6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43556/0 (socket says 172.21.15.151:43556) 2024-09-16T15:53:43.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.624+0000 7fac84a00640 1 -- 172.21.15.151:0/3044544101 learned_addr learned my addr 172.21.15.151:0/3044544101 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:53:43.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.624+0000 7fac7fe00640 1 --2- 172.21.15.151:0/3044544101 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac80073fb0 0x7fac801528d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:43.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.624+0000 7fac7f400640 1 --2- 172.21.15.151:0/3044544101 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac800ccf00 0x7fac801f6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:43.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.625+0000 7fac84a00640 1 -- 172.21.15.151:0/3044544101 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac80073fb0 msgr2=0x7fac801528d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:43.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.625+0000 7fac84a00640 1 --2- 172.21.15.151:0/3044544101 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac80073fb0 0x7fac801528d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:43.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.625+0000 7fac84a00640 1 -- 172.21.15.151:0/3044544101 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac800ccf00 msgr2=0x7fac801f6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:43.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.625+0000 7fac84a00640 1 --2- 172.21.15.151:0/3044544101 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac800ccf00 0x7fac801f6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:43.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.625+0000 7fac84a00640 1 -- 172.21.15.151:0/3044544101 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac80153000 con 0x7fac8015f5a0 2024-09-16T15:53:43.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.625+0000 7fac84a00640 1 --2- 172.21.15.151:0/3044544101 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac8015f5a0 0x7fac801f6660 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fac780099c0 tx=0x7fac78003960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:43.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.625+0000 7fac77e00640 1 -- 172.21.15.151:0/3044544101 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac7803f070 con 0x7fac8015f5a0 2024-09-16T15:53:43.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.625+0000 7fac77e00640 1 -- 172.21.15.151:0/3044544101 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fac780047a0 con 0x7fac8015f5a0 2024-09-16T15:53:43.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.625+0000 7fac86a00640 1 -- 172.21.15.151:0/3044544101 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac801532e0 con 0x7fac8015f5a0 2024-09-16T15:53:43.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.625+0000 7fac77e00640 1 -- 172.21.15.151:0/3044544101 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac78035030 con 0x7fac8015f5a0 2024-09-16T15:53:43.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.625+0000 7fac86a00640 1 -- 172.21.15.151:0/3044544101 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac801538a0 con 0x7fac8015f5a0 2024-09-16T15:53:43.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.626+0000 7fac86a00640 1 -- 172.21.15.151:0/3044544101 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac8006f9f0 con 0x7fac8015f5a0 2024-09-16T15:53:43.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.628+0000 7fac77e00640 1 -- 172.21.15.151:0/3044544101 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 47) ==== 104092+0+0 (secure 0 0 0) 0x7fac78042070 con 0x7fac8015f5a0 2024-09-16T15:53:43.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.628+0000 7fac77e00640 1 --2- 172.21.15.151:0/3044544101 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fac6007a740 0x7fac6007cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:43.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.629+0000 7fac77e00640 1 -- 172.21.15.151:0/3044544101 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 4969+0+0 (secure 0 0 0) 0x7fac780ba730 con 0x7fac8015f5a0 2024-09-16T15:53:43.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.629+0000 7fac7fe00640 1 --2- 172.21.15.151:0/3044544101 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fac6007a740 0x7fac6007cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:43.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.630+0000 7fac7fe00640 1 --2- 172.21.15.151:0/3044544101 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fac6007a740 0x7fac6007cc00 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fac70001e20 tx=0x7fac70009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:43.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.633+0000 7fac77e00640 1 -- 172.21.15.151:0/3044544101 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fac780841b0 con 0x7fac8015f5a0 2024-09-16T15:53:43.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.822+0000 7fac86a00640 1 -- 172.21.15.151:0/3044544101 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fac8009d8a0 con 0x7fac8015f5a0 2024-09-16T15:53:43.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.822+0000 7fac77e00640 1 -- 172.21.15.151:0/3044544101 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v129) ==== 84+0+65 (secure 0 0 0) 0x7fac78088be0 con 0x7fac8015f5a0 2024-09-16T15:53:43.824 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:53:43.824 INFO:teuthology.orchestra.run.smithi151.stdout:[ 2024-09-16T15:53:43.825 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:53:43.825 INFO:teuthology.orchestra.run.smithi151.stdout: "poolnum": 1, 2024-09-16T15:53:43.825 INFO:teuthology.orchestra.run.smithi151.stdout: "poolname": ".mgr" 2024-09-16T15:53:43.825 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:53:43.825 INFO:teuthology.orchestra.run.smithi151.stdout:] 2024-09-16T15:53:43.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.824+0000 7fac86a00640 1 -- 172.21.15.151:0/3044544101 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fac6007a740 msgr2=0x7fac6007cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:43.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.824+0000 7fac86a00640 1 --2- 172.21.15.151:0/3044544101 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fac6007a740 0x7fac6007cc00 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fac70001e20 tx=0x7fac70009040 comp rx=0 tx=0).stop 2024-09-16T15:53:43.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.824+0000 7fac86a00640 1 -- 172.21.15.151:0/3044544101 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac8015f5a0 msgr2=0x7fac801f6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:43.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.824+0000 7fac86a00640 1 --2- 172.21.15.151:0/3044544101 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac8015f5a0 0x7fac801f6660 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fac780099c0 tx=0x7fac78003960 comp rx=0 tx=0).stop 2024-09-16T15:53:43.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.824+0000 7fac86a00640 1 -- 172.21.15.151:0/3044544101 shutdown_connections 2024-09-16T15:53:43.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.824+0000 7fac86a00640 1 --2- 172.21.15.151:0/3044544101 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fac6007a740 0x7fac6007cc00 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:43.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.824+0000 7fac86a00640 1 --2- 172.21.15.151:0/3044544101 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac800ccf00 0x7fac801f6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:43.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.824+0000 7fac86a00640 1 --2- 172.21.15.151:0/3044544101 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac8015f5a0 0x7fac801f6660 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:43.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.824+0000 7fac86a00640 1 --2- 172.21.15.151:0/3044544101 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac80073fb0 0x7fac801528d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:43.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.824+0000 7fac86a00640 1 -- 172.21.15.151:0/3044544101 >> 172.21.15.151:0/3044544101 conn(0x7fac800b1950 msgr2=0x7fac800acd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:43.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.825+0000 7fac86a00640 1 -- 172.21.15.151:0/3044544101 shutdown_connections 2024-09-16T15:53:43.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:43.825+0000 7fac86a00640 1 -- 172.21.15.151:0/3044544101 wait complete. 2024-09-16T15:53:43.842 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-16T15:53:43.842 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-16T15:53:44.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:44.001+0000 7f77da400640 1 -- 172.21.15.151:0/1846023977 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f77d406f220 msgr2=0x7f77d406f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:44.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:44.001+0000 7f77da400640 1 --2- 172.21.15.151:0/1846023977 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f77d406f220 0x7f77d406f6d0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f77cc009900 tx=0x7f77cc02a610 comp rx=0 tx=0).stop 2024-09-16T15:53:44.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:44.002+0000 7f77da400640 1 -- 172.21.15.151:0/1846023977 shutdown_connections 2024-09-16T15:53:44.004 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:44.002+0000 7f77da400640 1 --2- 172.21.15.151:0/1846023977 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f77d406f220 0x7f77d406f6d0 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:44.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:44.002+0000 7f77da400640 1 --2- 172.21.15.151:0/1846023977 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f77d415cff0 0x7f77d406e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:44.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:44.002+0000 7f77da400640 1 -- 172.21.15.151:0/1846023977 >> 172.21.15.151:0/1846023977 conn(0x7f77d40b18e0 msgr2=0x7f77d40b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:44.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:44.002+0000 7f77da400640 1 -- 172.21.15.151:0/1846023977 shutdown_connections 2024-09-16T15:53:44.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:44.002+0000 7f77da400640 1 -- 172.21.15.151:0/1846023977 wait complete. 2024-09-16T15:53:44.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:44.004+0000 7f77da400640 1 Processor -- start 2024-09-16T15:53:44.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:44.004+0000 7f77da400640 1 -- start start 2024-09-16T15:53:44.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:44.006+0000 7f77da400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f77d406f220 0x7f77d41f3330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:44.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:44.006+0000 7f77da400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77d4159aa0 0x7f77d41524e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:44.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:44.006+0000 7f77da400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f77d415cff0 0x7f77d4154340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:44.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:44.006+0000 7f77da400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f77d40cc520 con 0x7f77d406f220 2024-09-16T15:53:44.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:44.006+0000 7f77da400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f77d40cc3a0 con 0x7f77d4159aa0 2024-09-16T15:53:44.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:44.006+0000 7f77da400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f77d40cc6a0 con 0x7f77d415cff0 2024-09-16T15:53:44.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:44.006+0000 7f77d3400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f77d406f220 0x7f77d41f3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:44.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:44.006+0000 7f77d3400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f77d406f220 0x7f77d41f3330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43572/0 (socket says 172.21.15.151:43572) 2024-09-16T15:53:44.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:44.006+0000 7f77d3400640 1 -- 172.21.15.151:0/2882711391 learned_addr learned my addr 172.21.15.151:0/2882711391 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:53:44.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:44.006+0000 7f77d2a00640 1 --2- 172.21.15.151:0/2882711391 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77d4159aa0 0x7f77d41524e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:44.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:44.006+0000 7f77d3400640 1 -- 172.21.15.151:0/2882711391 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f77d415cff0 msgr2=0x7f77d4154340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:44.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:44.006+0000 7f77d3400640 1 --2- 172.21.15.151:0/2882711391 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f77d415cff0 0x7f77d4154340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:44.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:44.006+0000 7f77d3400640 1 -- 172.21.15.151:0/2882711391 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77d4159aa0 msgr2=0x7f77d41524e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:44.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:44.006+0000 7f77d3400640 1 --2- 172.21.15.151:0/2882711391 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77d4159aa0 0x7f77d41524e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:44.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:44.006+0000 7f77d3400640 1 -- 172.21.15.151:0/2882711391 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77d41f3870 con 0x7f77d406f220 2024-09-16T15:53:44.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:44.007+0000 7f77d3400640 1 --2- 172.21.15.151:0/2882711391 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f77d406f220 0x7f77d41f3330 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f77c400c870 tx=0x7f77c400cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:44.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:44.007+0000 7f77cb400640 1 -- 172.21.15.151:0/2882711391 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c4003b70 con 0x7f77d406f220 2024-09-16T15:53:44.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:44.007+0000 7f77cb400640 1 -- 172.21.15.151:0/2882711391 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f77c4003d10 con 0x7f77d406f220 2024-09-16T15:53:44.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:44.007+0000 7f77cb400640 1 -- 172.21.15.151:0/2882711391 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c400aab0 con 0x7f77d406f220 2024-09-16T15:53:44.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:44.007+0000 7f77da400640 1 -- 172.21.15.151:0/2882711391 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77d41532a0 con 0x7f77d406f220 2024-09-16T15:53:44.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:44.007+0000 7f77da400640 1 -- 172.21.15.151:0/2882711391 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77d4153860 con 0x7f77d406f220 2024-09-16T15:53:44.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:44.008+0000 7f77da400640 1 -- 172.21.15.151:0/2882711391 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77d4069360 con 0x7f77d406f220 2024-09-16T15:53:44.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:44.010+0000 7f77cb400640 1 -- 172.21.15.151:0/2882711391 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 47) ==== 104092+0+0 (secure 0 0 0) 0x7f77c400ac50 con 0x7f77d406f220 2024-09-16T15:53:44.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:44.011+0000 7f77cb400640 1 --2- 172.21.15.151:0/2882711391 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f77b407a740 0x7f77b407cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:44.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:44.011+0000 7f77cb400640 1 -- 172.21.15.151:0/2882711391 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 4969+0+0 (secure 0 0 0) 0x7f77c40997b0 con 0x7f77d406f220 2024-09-16T15:53:44.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:44.011+0000 7f77d2a00640 1 --2- 172.21.15.151:0/2882711391 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f77b407a740 0x7f77b407cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:44.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:44.012+0000 7f77d2a00640 1 --2- 172.21.15.151:0/2882711391 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f77b407a740 0x7f77b407cc00 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f77c0005fd0 tx=0x7f77c0005850 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:44.017 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:44.015+0000 7f77cb400640 1 -- 172.21.15.151:0/2882711391 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f77d41f3870 con 0x7f77d406f220 2024-09-16T15:53:44.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:44.201+0000 7f77da400640 1 -- 172.21.15.151:0/2882711391 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f77d4070980 con 0x7f77d406f220 2024-09-16T15:53:45.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:45.016+0000 7f77cb400640 1 -- 172.21.15.151:0/2882711391 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v130) ==== 138+0+0 (secure 0 0 0) 0x7f77c4063920 con 0x7f77d406f220 2024-09-16T15:53:45.018 INFO:teuthology.orchestra.run.smithi151.stderr:pool 'cephfs_metadata' created 2024-09-16T15:53:45.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:45.019+0000 7f77da400640 1 -- 172.21.15.151:0/2882711391 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f77b407a740 msgr2=0x7f77b407cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:45.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:45.019+0000 7f77da400640 1 --2- 172.21.15.151:0/2882711391 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f77b407a740 0x7f77b407cc00 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f77c0005fd0 tx=0x7f77c0005850 comp rx=0 tx=0).stop 2024-09-16T15:53:45.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:45.019+0000 7f77da400640 1 -- 172.21.15.151:0/2882711391 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f77d406f220 msgr2=0x7f77d41f3330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:45.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:45.019+0000 7f77da400640 1 --2- 172.21.15.151:0/2882711391 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f77d406f220 0x7f77d41f3330 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f77c400c870 tx=0x7f77c400cd30 comp rx=0 tx=0).stop 2024-09-16T15:53:45.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:45.019+0000 7f77da400640 1 -- 172.21.15.151:0/2882711391 shutdown_connections 2024-09-16T15:53:45.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:45.019+0000 7f77da400640 1 --2- 172.21.15.151:0/2882711391 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f77b407a740 0x7f77b407cc00 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:45.021 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:45.019+0000 7f77da400640 1 --2- 172.21.15.151:0/2882711391 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77d4159aa0 0x7f77d41524e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:45.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:45.019+0000 7f77da400640 1 --2- 172.21.15.151:0/2882711391 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f77d415cff0 0x7f77d4154340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:45.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:45.019+0000 7f77da400640 1 --2- 172.21.15.151:0/2882711391 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f77d406f220 0x7f77d41f3330 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:45.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:45.019+0000 7f77da400640 1 -- 172.21.15.151:0/2882711391 >> 172.21.15.151:0/2882711391 conn(0x7f77d40b18e0 msgr2=0x7f77d415e450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:45.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:45.019+0000 7f77da400640 1 -- 172.21.15.151:0/2882711391 shutdown_connections 2024-09-16T15:53:45.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:45.019+0000 7f77da400640 1 -- 172.21.15.151:0/2882711391 wait complete. 2024-09-16T15:53:45.042 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-16T15:53:45.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:45.206+0000 7faa7c000640 1 -- 172.21.15.151:0/912598166 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faa74145750 msgr2=0x7faa74073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:45.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:45.206+0000 7faa7c000640 1 --2- 172.21.15.151:0/912598166 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faa74145750 0x7faa74073440 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7faa740cfb70 tx=0x7faa6802a980 comp rx=0 tx=0).stop 2024-09-16T15:53:45.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:45.207+0000 7faa7c000640 1 -- 172.21.15.151:0/912598166 shutdown_connections 2024-09-16T15:53:45.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:45.207+0000 7faa7c000640 1 --2- 172.21.15.151:0/912598166 >> v2:172.21.15.167:3300/0 conn(0x7faa740cccd0 0x7faa740cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:45.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:45.207+0000 7faa7c000640 1 --2- 172.21.15.151:0/912598166 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faa74145750 0x7faa74073440 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:45.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:45.207+0000 7faa7c000640 1 -- 172.21.15.151:0/912598166 >> 172.21.15.151:0/912598166 conn(0x7faa740b1840 msgr2=0x7faa740b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:45.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:45.207+0000 7faa7c000640 1 -- 172.21.15.151:0/912598166 shutdown_connections 2024-09-16T15:53:45.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:45.207+0000 7faa7c000640 1 -- 172.21.15.151:0/912598166 wait complete. 2024-09-16T15:53:45.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:45.208+0000 7faa7c000640 1 Processor -- start 2024-09-16T15:53:45.210 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:45.208+0000 7faa7c000640 1 -- start start 2024-09-16T15:53:45.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:45.209+0000 7faa7c000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faa7406efa0 0x7faa7415e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:45.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:45.209+0000 7faa7c000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faa740cccd0 0x7faa740aced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:45.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:45.209+0000 7faa7c000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa74145750 0x7faa740a94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:45.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:45.209+0000 7faa7c000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7faa740ce480 con 0x7faa740cccd0 2024-09-16T15:53:45.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:45.209+0000 7faa7c000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7faa740ce300 con 0x7faa74145750 2024-09-16T15:53:45.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:45.209+0000 7faa7c000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7faa740ce600 con 0x7faa7406efa0 2024-09-16T15:53:45.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:45.209+0000 7faa78c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faa740cccd0 0x7faa740aced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:45.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:45.210+0000 7faa78c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faa740cccd0 0x7faa740aced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43596/0 (socket says 172.21.15.151:43596) 2024-09-16T15:53:45.212 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:45.210+0000 7faa78c00640 1 -- 172.21.15.151:0/3075534232 learned_addr learned my addr 172.21.15.151:0/3075534232 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:53:45.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:45.210+0000 7faa7a000640 1 --2- 172.21.15.151:0/3075534232 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa74145750 0x7faa740a94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:45.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:45.210+0000 7faa79600640 1 --2- 172.21.15.151:0/3075534232 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faa7406efa0 0x7faa7415e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:45.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:45.210+0000 7faa78c00640 1 -- 172.21.15.151:0/3075534232 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faa7406efa0 msgr2=0x7faa7415e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:45.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:45.210+0000 7faa78c00640 1 --2- 172.21.15.151:0/3075534232 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faa7406efa0 0x7faa7415e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:45.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:45.210+0000 7faa78c00640 1 -- 172.21.15.151:0/3075534232 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa74145750 msgr2=0x7faa740a94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:45.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:45.210+0000 7faa78c00640 1 --2- 172.21.15.151:0/3075534232 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa74145750 0x7faa740a94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:45.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:45.210+0000 7faa78c00640 1 -- 172.21.15.151:0/3075534232 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa74141fa0 con 0x7faa740cccd0 2024-09-16T15:53:45.213 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:45.210+0000 7faa78c00640 1 --2- 172.21.15.151:0/3075534232 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faa740cccd0 0x7faa740aced0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7faa6000c430 tx=0x7faa6000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:45.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:45.210+0000 7faa71600640 1 -- 172.21.15.151:0/3075534232 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa60011790 con 0x7faa740cccd0 2024-09-16T15:53:45.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:45.210+0000 7faa71600640 1 -- 172.21.15.151:0/3075534232 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faa60017070 con 0x7faa740cccd0 2024-09-16T15:53:45.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:45.210+0000 7faa71600640 1 -- 172.21.15.151:0/3075534232 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa60011d00 con 0x7faa740cccd0 2024-09-16T15:53:45.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:45.210+0000 7faa7c000640 1 -- 172.21.15.151:0/3075534232 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa74140f00 con 0x7faa740cccd0 2024-09-16T15:53:45.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:45.211+0000 7faa7c000640 1 -- 172.21.15.151:0/3075534232 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa741414c0 con 0x7faa740cccd0 2024-09-16T15:53:45.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:45.212+0000 7faa7c000640 1 -- 172.21.15.151:0/3075534232 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa740cdbb0 con 0x7faa740cccd0 2024-09-16T15:53:45.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:45.214+0000 7faa71600640 1 -- 172.21.15.151:0/3075534232 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 47) ==== 104092+0+0 (secure 0 0 0) 0x7faa60026430 con 0x7faa740cccd0 2024-09-16T15:53:45.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:45.215+0000 7faa71600640 1 --2- 172.21.15.151:0/3075534232 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7faa5407a740 0x7faa5407cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:45.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:45.215+0000 7faa71600640 1 -- 172.21.15.151:0/3075534232 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5339+0+0 (secure 0 0 0) 0x7faa6009a2d0 con 0x7faa740cccd0 2024-09-16T15:53:45.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:45.216+0000 7faa79600640 1 --2- 172.21.15.151:0/3075534232 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7faa5407a740 0x7faa5407cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:45.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:45.217+0000 7faa79600640 1 --2- 172.21.15.151:0/3075534232 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7faa5407a740 0x7faa5407cc00 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7faa68035000 tx=0x7faa68004ab0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:45.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:45.219+0000 7faa71600640 1 -- 172.21.15.151:0/3075534232 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7faa60064420 con 0x7faa740cccd0 2024-09-16T15:53:45.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:45.403+0000 7faa7c000640 1 -- 172.21.15.151:0/3075534232 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7faa74069a10 con 0x7faa740cccd0 2024-09-16T15:53:45.791 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-09-16T15:53:46.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:46.025+0000 7faa71600640 1 -- 172.21.15.151:0/3075534232 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v131) ==== 170+0+0 (secure 0 0 0) 0x7faa60063d20 con 0x7faa740cccd0 2024-09-16T15:53:46.027 INFO:teuthology.orchestra.run.smithi151.stderr:pool 'cephfs_data' created 2024-09-16T15:53:46.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:46.029+0000 7faa7c000640 1 -- 172.21.15.151:0/3075534232 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7faa5407a740 msgr2=0x7faa5407cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:46.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:46.029+0000 7faa7c000640 1 --2- 172.21.15.151:0/3075534232 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7faa5407a740 0x7faa5407cc00 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7faa68035000 tx=0x7faa68004ab0 comp rx=0 tx=0).stop 2024-09-16T15:53:46.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:46.029+0000 7faa7c000640 1 -- 172.21.15.151:0/3075534232 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faa740cccd0 msgr2=0x7faa740aced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:46.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:46.029+0000 7faa7c000640 1 --2- 172.21.15.151:0/3075534232 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faa740cccd0 0x7faa740aced0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7faa6000c430 tx=0x7faa6000c8f0 comp rx=0 tx=0).stop 2024-09-16T15:53:46.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:46.029+0000 7faa7c000640 1 -- 172.21.15.151:0/3075534232 shutdown_connections 2024-09-16T15:53:46.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:46.029+0000 7faa7c000640 1 --2- 172.21.15.151:0/3075534232 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7faa5407a740 0x7faa5407cc00 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:46.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:46.029+0000 7faa7c000640 1 --2- 172.21.15.151:0/3075534232 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faa74145750 0x7faa740a94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:46.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:46.029+0000 7faa7c000640 1 --2- 172.21.15.151:0/3075534232 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faa740cccd0 0x7faa740aced0 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:46.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:46.029+0000 7faa7c000640 1 --2- 172.21.15.151:0/3075534232 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faa7406efa0 0x7faa7415e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:46.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:46.029+0000 7faa7c000640 1 -- 172.21.15.151:0/3075534232 >> 172.21.15.151:0/3075534232 conn(0x7faa740b1840 msgr2=0x7faa740b0610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:46.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:46.029+0000 7faa7c000640 1 -- 172.21.15.151:0/3075534232 shutdown_connections 2024-09-16T15:53:46.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:46.029+0000 7faa7c000640 1 -- 172.21.15.151:0/3075534232 wait complete. 2024-09-16T15:53:46.064 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-16T15:53:46.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:46.225+0000 7fe49ca00640 1 -- 172.21.15.151:0/942282434 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe498145750 msgr2=0x7fe498073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:46.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:46.225+0000 7fe49ca00640 1 --2- 172.21.15.151:0/942282434 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe498145750 0x7fe498073440 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fe488005fd0 tx=0x7fe48802a540 comp rx=0 tx=0).stop 2024-09-16T15:53:46.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:46.225+0000 7fe49ca00640 1 -- 172.21.15.151:0/942282434 shutdown_connections 2024-09-16T15:53:46.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:46.225+0000 7fe49ca00640 1 --2- 172.21.15.151:0/942282434 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe4980cccd0 0x7fe4980cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:46.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:46.225+0000 7fe49ca00640 1 --2- 172.21.15.151:0/942282434 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe498145750 0x7fe498073440 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:46.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:46.225+0000 7fe49ca00640 1 -- 172.21.15.151:0/942282434 >> 172.21.15.151:0/942282434 conn(0x7fe4980b1840 msgr2=0x7fe4980b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:46.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:46.225+0000 7fe49ca00640 1 -- 172.21.15.151:0/942282434 shutdown_connections 2024-09-16T15:53:46.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:46.225+0000 7fe49ca00640 1 -- 172.21.15.151:0/942282434 wait complete. 2024-09-16T15:53:46.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:46.227+0000 7fe49ca00640 1 Processor -- start 2024-09-16T15:53:46.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:46.227+0000 7fe49ca00640 1 -- start start 2024-09-16T15:53:46.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:46.228+0000 7fe49ca00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe49806efa0 0x7fe4981f2c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:46.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:46.228+0000 7fe49ca00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe4980cccd0 0x7fe4980aa900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:46.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:46.228+0000 7fe49ca00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe498145750 0x7fe4980ad490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:46.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:46.228+0000 7fe49ca00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe4980ce480 con 0x7fe4980cccd0 2024-09-16T15:53:46.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:46.228+0000 7fe49ca00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe4980ce300 con 0x7fe49806efa0 2024-09-16T15:53:46.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:46.228+0000 7fe49ca00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fe4980ce600 con 0x7fe498145750 2024-09-16T15:53:46.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:46.228+0000 7fe495400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe4980cccd0 0x7fe4980aa900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:46.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:46.228+0000 7fe495400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe4980cccd0 0x7fe4980aa900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43616/0 (socket says 172.21.15.151:43616) 2024-09-16T15:53:46.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:46.228+0000 7fe495400640 1 -- 172.21.15.151:0/4072750641 learned_addr learned my addr 172.21.15.151:0/4072750641 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:53:46.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:46.228+0000 7fe495e00640 1 --2- 172.21.15.151:0/4072750641 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe49806efa0 0x7fe4981f2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:46.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:46.228+0000 7fe496800640 1 --2- 172.21.15.151:0/4072750641 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe498145750 0x7fe4980ad490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:46.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:46.229+0000 7fe495400640 1 -- 172.21.15.151:0/4072750641 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe498145750 msgr2=0x7fe4980ad490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:46.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:46.229+0000 7fe495400640 1 --2- 172.21.15.151:0/4072750641 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe498145750 0x7fe4980ad490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:46.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:46.229+0000 7fe495400640 1 -- 172.21.15.151:0/4072750641 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe49806efa0 msgr2=0x7fe4981f2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:46.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:46.229+0000 7fe495400640 1 --2- 172.21.15.151:0/4072750641 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe49806efa0 0x7fe4981f2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:46.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:46.229+0000 7fe495400640 1 -- 172.21.15.151:0/4072750641 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4980ab3e0 con 0x7fe4980cccd0 2024-09-16T15:53:46.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:46.229+0000 7fe495400640 1 --2- 172.21.15.151:0/4072750641 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe4980cccd0 0x7fe4980aa900 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7fe48400c480 tx=0x7fe48400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:46.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:46.229+0000 7fe48e000640 1 -- 172.21.15.151:0/4072750641 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4840052d0 con 0x7fe4980cccd0 2024-09-16T15:53:46.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:46.229+0000 7fe48e000640 1 -- 172.21.15.151:0/4072750641 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe484005470 con 0x7fe4980cccd0 2024-09-16T15:53:46.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:46.229+0000 7fe48e000640 1 -- 172.21.15.151:0/4072750641 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe484012870 con 0x7fe4980cccd0 2024-09-16T15:53:46.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:46.229+0000 7fe49ca00640 1 -- 172.21.15.151:0/4072750641 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4980ab6c0 con 0x7fe4980cccd0 2024-09-16T15:53:46.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:46.229+0000 7fe49ca00640 1 -- 172.21.15.151:0/4072750641 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4980abc80 con 0x7fe4980cccd0 2024-09-16T15:53:46.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:46.230+0000 7fe49ca00640 1 -- 172.21.15.151:0/4072750641 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4980cdbb0 con 0x7fe4980cccd0 2024-09-16T15:53:46.233 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:46.232+0000 7fe48e000640 1 -- 172.21.15.151:0/4072750641 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 47) ==== 104092+0+0 (secure 0 0 0) 0x7fe4840035f0 con 0x7fe4980cccd0 2024-09-16T15:53:46.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:46.233+0000 7fe48e000640 1 --2- 172.21.15.151:0/4072750641 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe47807a740 0x7fe47807cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:46.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:46.233+0000 7fe48e000640 1 -- 172.21.15.151:0/4072750641 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5721+0+0 (secure 0 0 0) 0x7fe4840993b0 con 0x7fe4980cccd0 2024-09-16T15:53:46.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:46.233+0000 7fe495e00640 1 --2- 172.21.15.151:0/4072750641 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe47807a740 0x7fe47807cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:46.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:46.234+0000 7fe495e00640 1 --2- 172.21.15.151:0/4072750641 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe47807a740 0x7fe47807cc00 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fe488033000 tx=0x7fe488004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:46.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:46.237+0000 7fe48e000640 1 -- 172.21.15.151:0/4072750641 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe484062eb0 con 0x7fe4980cccd0 2024-09-16T15:53:46.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:46.459+0000 7fe49ca00640 1 -- 172.21.15.151:0/4072750641 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fe4980cca60 con 0x7fe4980cccd0 2024-09-16T15:53:47.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.027+0000 7fe48e000640 1 -- 172.21.15.151:0/4072750641 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 18 and data pool 19 v83) ==== 174+0+0 (secure 0 0 0) 0x7fe4840678e0 con 0x7fe4980cccd0 2024-09-16T15:53:47.029 INFO:teuthology.orchestra.run.smithi151.stderr:new fs with metadata pool 18 and data pool 19 2024-09-16T15:53:47.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.030+0000 7fe49ca00640 1 -- 172.21.15.151:0/4072750641 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe47807a740 msgr2=0x7fe47807cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:47.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.030+0000 7fe49ca00640 1 --2- 172.21.15.151:0/4072750641 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe47807a740 0x7fe47807cc00 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fe488033000 tx=0x7fe488004730 comp rx=0 tx=0).stop 2024-09-16T15:53:47.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.030+0000 7fe49ca00640 1 -- 172.21.15.151:0/4072750641 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe4980cccd0 msgr2=0x7fe4980aa900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:47.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.030+0000 7fe49ca00640 1 --2- 172.21.15.151:0/4072750641 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe4980cccd0 0x7fe4980aa900 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7fe48400c480 tx=0x7fe48400c940 comp rx=0 tx=0).stop 2024-09-16T15:53:47.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.031+0000 7fe49ca00640 1 -- 172.21.15.151:0/4072750641 shutdown_connections 2024-09-16T15:53:47.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.031+0000 7fe49ca00640 1 --2- 172.21.15.151:0/4072750641 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe47807a740 0x7fe47807cc00 secure :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fe488033000 tx=0x7fe488004730 comp rx=0 tx=0).stop 2024-09-16T15:53:47.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.031+0000 7fe49ca00640 1 --2- 172.21.15.151:0/4072750641 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe498145750 0x7fe4980ad490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:47.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.031+0000 7fe49ca00640 1 --2- 172.21.15.151:0/4072750641 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe4980cccd0 0x7fe4980aa900 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:47.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.031+0000 7fe49ca00640 1 --2- 172.21.15.151:0/4072750641 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe49806efa0 0x7fe4981f2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:47.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.031+0000 7fe49ca00640 1 -- 172.21.15.151:0/4072750641 >> 172.21.15.151:0/4072750641 conn(0x7fe4980b1840 msgr2=0x7fe4980cf9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:47.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.033+0000 7fe49ca00640 1 -- 172.21.15.151:0/4072750641 shutdown_connections 2024-09-16T15:53:47.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.033+0000 7fe49ca00640 1 -- 172.21.15.151:0/4072750641 wait complete. 2024-09-16T15:53:47.071 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:53:47.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.228+0000 7eff10c00640 1 -- 172.21.15.151:0/1679389850 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efef0004800 msgr2=0x7efef0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:47.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.228+0000 7eff10c00640 1 --2- 172.21.15.151:0/1679389850 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efef0004800 0x7efef0004c80 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7efef8001690 tx=0x7efef0005830 comp rx=0 tx=0).stop 2024-09-16T15:53:47.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.228+0000 7eff10c00640 1 -- 172.21.15.151:0/1679389850 shutdown_connections 2024-09-16T15:53:47.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.228+0000 7eff10c00640 1 --2- 172.21.15.151:0/1679389850 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efef0004800 0x7efef0004c80 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:47.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.228+0000 7eff10c00640 1 -- 172.21.15.151:0/1679389850 >> 172.21.15.151:0/1679389850 conn(0x7eff0c0b1950 msgr2=0x7eff0c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:47.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.228+0000 7eff10c00640 1 -- 172.21.15.151:0/1679389850 shutdown_connections 2024-09-16T15:53:47.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.228+0000 7eff10c00640 1 -- 172.21.15.151:0/1679389850 wait complete. 2024-09-16T15:53:47.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.230+0000 7eff10c00640 1 Processor -- start 2024-09-16T15:53:47.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.230+0000 7eff10c00640 1 -- start start 2024-09-16T15:53:47.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.231+0000 7eff10c00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efef0004800 0x7eff0c156cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:47.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.231+0000 7eff10c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff0c157230 0x7eff0c0b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:47.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.231+0000 7eff10c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff0c0b4310 0x7eff0c148270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:47.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.231+0000 7eff10c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7eff0c074420 con 0x7eff0c0b4310 2024-09-16T15:53:47.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.231+0000 7eff10c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7eff0c0b2700 con 0x7eff0c157230 2024-09-16T15:53:47.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.231+0000 7eff10c00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7eff0c0745a0 con 0x7efef0004800 2024-09-16T15:53:47.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.231+0000 7eff0a800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff0c0b4310 0x7eff0c148270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:47.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.231+0000 7eff0a800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff0c0b4310 0x7eff0c148270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43638/0 (socket says 172.21.15.151:43638) 2024-09-16T15:53:47.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.231+0000 7eff0a800640 1 -- 172.21.15.151:0/834640960 learned_addr learned my addr 172.21.15.151:0/834640960 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:53:47.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.231+0000 7eff09e00640 1 --2- 172.21.15.151:0/834640960 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efef0004800 0x7eff0c156cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:47.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.231+0000 7eff09400640 1 --2- 172.21.15.151:0/834640960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff0c157230 0x7eff0c0b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:47.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.232+0000 7eff0a800640 1 -- 172.21.15.151:0/834640960 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efef0004800 msgr2=0x7eff0c156cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:47.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.232+0000 7eff0a800640 1 --2- 172.21.15.151:0/834640960 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efef0004800 0x7eff0c156cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:47.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.232+0000 7eff0a800640 1 -- 172.21.15.151:0/834640960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff0c157230 msgr2=0x7eff0c0b3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:47.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.232+0000 7eff0a800640 1 --2- 172.21.15.151:0/834640960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff0c157230 0x7eff0c0b3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:47.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.232+0000 7eff0a800640 1 -- 172.21.15.151:0/834640960 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff0c0b2d50 con 0x7eff0c0b4310 2024-09-16T15:53:47.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.232+0000 7eff09400640 1 --2- 172.21.15.151:0/834640960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff0c157230 0x7eff0c0b3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:53:47.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.232+0000 7eff0a800640 1 --2- 172.21.15.151:0/834640960 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff0c0b4310 0x7eff0c148270 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7eff0c076720 tx=0x7eff0400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:47.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.232+0000 7eff02000640 1 -- 172.21.15.151:0/834640960 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff0400ca10 con 0x7eff0c0b4310 2024-09-16T15:53:47.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.232+0000 7eff02000640 1 -- 172.21.15.151:0/834640960 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7eff04017070 con 0x7eff0c0b4310 2024-09-16T15:53:47.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.232+0000 7eff10c00640 1 -- 172.21.15.151:0/834640960 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff0c0b3010 con 0x7eff0c0b4310 2024-09-16T15:53:47.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.232+0000 7eff02000640 1 -- 172.21.15.151:0/834640960 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff04011460 con 0x7eff0c0b4310 2024-09-16T15:53:47.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.232+0000 7eff10c00640 1 -- 172.21.15.151:0/834640960 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff0c0b35d0 con 0x7eff0c0b4310 2024-09-16T15:53:47.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.233+0000 7eff10c00640 1 -- 172.21.15.151:0/834640960 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff0c0acd30 con 0x7eff0c0b4310 2024-09-16T15:53:47.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.235+0000 7eff02000640 1 -- 172.21.15.151:0/834640960 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 47) ==== 104092+0+0 (secure 0 0 0) 0x7eff04011600 con 0x7eff0c0b4310 2024-09-16T15:53:47.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.235+0000 7eff02000640 1 --2- 172.21.15.151:0/834640960 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efeec07a770 0x7efeec07cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:47.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.236+0000 7eff02000640 1 -- 172.21.15.151:0/834640960 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5821+0+0 (secure 0 0 0) 0x7eff04099e30 con 0x7eff0c0b4310 2024-09-16T15:53:47.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.236+0000 7eff09e00640 1 --2- 172.21.15.151:0/834640960 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efeec07a770 0x7efeec07cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:47.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.237+0000 7eff09e00640 1 --2- 172.21.15.151:0/834640960 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efeec07a770 0x7efeec07cc30 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7eff0c052890 tx=0x7efefc005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:47.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.240+0000 7eff02000640 1 -- 172.21.15.151:0/834640960 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7eff04063f00 con 0x7eff0c0b4310 2024-09-16T15:53:47.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.426+0000 7eff10c00640 1 -- 172.21.15.151:0/834640960 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7eff0c0aff10 con 0x7eff0c0b4310 2024-09-16T15:53:47.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.427+0000 7eff02000640 1 -- 172.21.15.151:0/834640960 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v132) ==== 74+0+16937 (secure 0 0 0) 0x7eff04063800 con 0x7eff0c0b4310 2024-09-16T15:53:47.429 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:53:47.429 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":132,"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","created":"2024-09-16T15:38:56.348535+0000","modified":"2024-09-16T15:53:46.461873+0000","last_up_change":"2024-09-16T15:39:10.681384+0000","last_in_change":"2024-09-16T15:39:00.544477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:39:11.602252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:53:44.212522+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-16T15:53:45.415072+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5524076-a903-4d0a-af6a-caf474c4765d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6809","nonce":3111342741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6811","nonce":3111342741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6815","nonce":3111342741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6813","nonce":3111342741}]},"public_addr":"172.21.15.151:6809/3111342741","cluster_addr":"172.21.15.151:6811/3111342741","heartbeat_back_addr":"172.21.15.151:6815/3111342741","heartbeat_front_addr":"172.21.15.151:6813/3111342741","state":["exists","up"]},{"osd":1,"uuid":"dd25925f-0ae1-494e-8529-4dc205759db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6801","nonce":3368304087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6803","nonce":3368304087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6807","nonce":3368304087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6805","nonce":3368304087}]},"public_addr":"172.21.15.151:6801/3368304087","cluster_addr":"172.21.15.151:6803/3368304087","heartbeat_back_addr":"172.21.15.151:6807/3368304087","heartbeat_front_addr":"172.21.15.151:6805/3368304087","state":["exists","up"]},{"osd":2,"uuid":"c2f18115-013d-4da6-a0fe-cb978f4494b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6823","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6825","nonce":3079373271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6827","nonce":3079373271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6831","nonce":3079373271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6829","nonce":3079373271}]},"public_addr":"172.21.15.151:6825/3079373271","cluster_addr":"172.21.15.151:6827/3079373271","heartbeat_back_addr":"172.21.15.151:6831/3079373271","heartbeat_front_addr":"172.21.15.151:6829/3079373271","state":["exists","up"]},{"osd":3,"uuid":"36edb718-2fcb-4d88-8ddc-c321eed4279d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6817","nonce":1142509041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6819","nonce":1142509041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6824","nonce":1142509041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6821","nonce":1142509041}]},"public_addr":"172.21.15.151:6817/1142509041","cluster_addr":"172.21.15.151:6819/1142509041","heartbeat_back_addr":"172.21.15.151:6824/1142509041","heartbeat_front_addr":"172.21.15.151:6821/1142509041","state":["exists","up"]},{"osd":4,"uuid":"64d33708-cd0a-4236-abe3-1b646e82ab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6811","nonce":2735495271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6813","nonce":2735495271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6817","nonce":2735495271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6815","nonce":2735495271}]},"public_addr":"172.21.15.167:6811/2735495271","cluster_addr":"172.21.15.167:6813/2735495271","heartbeat_back_addr":"172.21.15.167:6817/2735495271","heartbeat_front_addr":"172.21.15.167:6815/2735495271","state":["exists","up"]},{"osd":5,"uuid":"f14b44a9-5910-458c-b518-09a8d0de94de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6827","nonce":1494916572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6829","nonce":1494916572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6833","nonce":1494916572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6831","nonce":1494916572}]},"public_addr":"172.21.15.167:6827/1494916572","cluster_addr":"172.21.15.167:6829/1494916572","heartbeat_back_addr":"172.21.15.167:6833/1494916572","heartbeat_front_addr":"172.21.15.167:6831/1494916572","state":["exists","up"]},{"osd":6,"uuid":"39335545-2654-4710-84b0-a9aa47cfcbba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6819","nonce":2538273273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6821","nonce":2538273273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6825","nonce":2538273273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6823","nonce":2538273273}]},"public_addr":"172.21.15.167:6819/2538273273","cluster_addr":"172.21.15.167:6821/2538273273","heartbeat_back_addr":"172.21.15.167:6825/2538273273","heartbeat_front_addr":"172.21.15.167:6823/2538273273","state":["exists","up"]},{"osd":7,"uuid":"4f8bfc81-53dd-4339-ac22-1a4007d9f963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6803","nonce":265072227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6805","nonce":265072227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6809","nonce":265072227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6807","nonce":265072227}]},"public_addr":"172.21.15.167:6803/265072227","cluster_addr":"172.21.15.167:6805/265072227","heartbeat_back_addr":"172.21.15.167:6809/265072227","heartbeat_front_addr":"172.21.15.167:6807/265072227","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.150359+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.084358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.722368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.332809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.320709+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.395153+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.935250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.801443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:53:47.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.429+0000 7eff10c00640 1 -- 172.21.15.151:0/834640960 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efeec07a770 msgr2=0x7efeec07cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:47.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.429+0000 7eff10c00640 1 --2- 172.21.15.151:0/834640960 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efeec07a770 0x7efeec07cc30 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7eff0c052890 tx=0x7efefc005f90 comp rx=0 tx=0).stop 2024-09-16T15:53:47.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.429+0000 7eff10c00640 1 -- 172.21.15.151:0/834640960 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff0c0b4310 msgr2=0x7eff0c148270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:47.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.429+0000 7eff10c00640 1 --2- 172.21.15.151:0/834640960 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff0c0b4310 0x7eff0c148270 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7eff0c076720 tx=0x7eff0400ecd0 comp rx=0 tx=0).stop 2024-09-16T15:53:47.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.429+0000 7eff10c00640 1 -- 172.21.15.151:0/834640960 shutdown_connections 2024-09-16T15:53:47.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.429+0000 7eff10c00640 1 --2- 172.21.15.151:0/834640960 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efeec07a770 0x7efeec07cc30 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:47.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.429+0000 7eff10c00640 1 --2- 172.21.15.151:0/834640960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff0c157230 0x7eff0c0b3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:47.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.429+0000 7eff10c00640 1 --2- 172.21.15.151:0/834640960 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff0c0b4310 0x7eff0c148270 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:47.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.429+0000 7eff10c00640 1 --2- 172.21.15.151:0/834640960 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efef0004800 0x7eff0c156cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:47.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.429+0000 7eff10c00640 1 -- 172.21.15.151:0/834640960 >> 172.21.15.151:0/834640960 conn(0x7eff0c0b1950 msgr2=0x7eff0c0addd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:47.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.429+0000 7eff10c00640 1 -- 172.21.15.151:0/834640960 shutdown_connections 2024-09-16T15:53:47.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.429+0000 7eff10c00640 1 -- 172.21.15.151:0/834640960 wait complete. 2024-09-16T15:53:47.448 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:53:47.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.605+0000 7f1398c00640 1 -- 172.21.15.151:0/713238523 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13940d0540 msgr2=0x7f139406f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:47.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.605+0000 7f1398c00640 1 --2- 172.21.15.151:0/713238523 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13940d0540 0x7f139406f320 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f13940cd570 tx=0x7f138c02aac0 comp rx=0 tx=0).stop 2024-09-16T15:53:47.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.605+0000 7f1398c00640 1 -- 172.21.15.151:0/713238523 shutdown_connections 2024-09-16T15:53:47.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.605+0000 7f1398c00640 1 --2- 172.21.15.151:0/713238523 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13940d0540 0x7f139406f320 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:47.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.605+0000 7f1398c00640 1 --2- 172.21.15.151:0/713238523 >> v2:172.21.15.167:3300/0 conn(0x7f139415bc30 0x7f13940b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:47.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.605+0000 7f1398c00640 1 -- 172.21.15.151:0/713238523 >> 172.21.15.151:0/713238523 conn(0x7f13940cfa20 msgr2=0x7f13940cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:47.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.605+0000 7f1398c00640 1 -- 172.21.15.151:0/713238523 shutdown_connections 2024-09-16T15:53:47.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.605+0000 7f1398c00640 1 -- 172.21.15.151:0/713238523 wait complete. 2024-09-16T15:53:47.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.606+0000 7f1398c00640 1 Processor -- start 2024-09-16T15:53:47.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.607+0000 7f1398c00640 1 -- start start 2024-09-16T15:53:47.611 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.607+0000 7f1398c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13940d0540 0x7f13941f7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:47.611 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.607+0000 7f1398c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f139415bc30 0x7f13941f8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:47.611 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.607+0000 7f1398c00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13941f86c0 0x7f1394148e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:47.611 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.607+0000 7f1398c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1394074660 con 0x7f139415bc30 2024-09-16T15:53:47.611 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.607+0000 7f1398c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f139406f860 con 0x7f13940d0540 2024-09-16T15:53:47.611 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.607+0000 7f1398c00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f13940af710 con 0x7f13941f86c0 2024-09-16T15:53:47.611 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.608+0000 7f1392a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f139415bc30 0x7f13941f8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:47.611 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.608+0000 7f1392a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f139415bc30 0x7f13941f8180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43654/0 (socket says 172.21.15.151:43654) 2024-09-16T15:53:47.611 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.608+0000 7f1392a00640 1 -- 172.21.15.151:0/3043567147 learned_addr learned my addr 172.21.15.151:0/3043567147 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:53:47.611 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.608+0000 7f1393400640 1 --2- 172.21.15.151:0/3043567147 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13940d0540 0x7f13941f7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:47.611 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.608+0000 7f1393e00640 1 --2- 172.21.15.151:0/3043567147 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13941f86c0 0x7f1394148e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:47.611 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.608+0000 7f1392a00640 1 -- 172.21.15.151:0/3043567147 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13941f86c0 msgr2=0x7f1394148e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:47.611 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.608+0000 7f1392a00640 1 --2- 172.21.15.151:0/3043567147 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13941f86c0 0x7f1394148e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:47.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.608+0000 7f1392a00640 1 -- 172.21.15.151:0/3043567147 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13940d0540 msgr2=0x7f13941f7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:47.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.608+0000 7f1392a00640 1 --2- 172.21.15.151:0/3043567147 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13940d0540 0x7f13941f7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:47.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.608+0000 7f1392a00640 1 -- 172.21.15.151:0/3043567147 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13941f4300 con 0x7f139415bc30 2024-09-16T15:53:47.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.608+0000 7f1392a00640 1 --2- 172.21.15.151:0/3043567147 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f139415bc30 0x7f13941f8180 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f138000c450 tx=0x7f138000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:47.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.609+0000 7f138b400640 1 -- 172.21.15.151:0/3043567147 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13800052d0 con 0x7f139415bc30 2024-09-16T15:53:47.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.609+0000 7f138b400640 1 -- 172.21.15.151:0/3043567147 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1380005470 con 0x7f139415bc30 2024-09-16T15:53:47.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.609+0000 7f138b400640 1 -- 172.21.15.151:0/3043567147 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1380012870 con 0x7f139415bc30 2024-09-16T15:53:47.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.609+0000 7f1398c00640 1 -- 172.21.15.151:0/3043567147 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13940ae670 con 0x7f139415bc30 2024-09-16T15:53:47.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.609+0000 7f1398c00640 1 -- 172.21.15.151:0/3043567147 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13940aec30 con 0x7f139415bc30 2024-09-16T15:53:47.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.610+0000 7f1398c00640 1 -- 172.21.15.151:0/3043567147 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f139406fdf0 con 0x7f139415bc30 2024-09-16T15:53:47.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.611+0000 7f138b400640 1 -- 172.21.15.151:0/3043567147 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 47) ==== 104092+0+0 (secure 0 0 0) 0x7f13800035f0 con 0x7f139415bc30 2024-09-16T15:53:47.614 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.613+0000 7f138b400640 1 --2- 172.21.15.151:0/3043567147 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f137407a740 0x7f137407cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:47.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.613+0000 7f138b400640 1 -- 172.21.15.151:0/3043567147 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5821+0+0 (secure 0 0 0) 0x7f13800992d0 con 0x7f139415bc30 2024-09-16T15:53:47.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.613+0000 7f1393400640 1 --2- 172.21.15.151:0/3043567147 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f137407a740 0x7f137407cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:47.616 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.614+0000 7f1393400640 1 --2- 172.21.15.151:0/3043567147 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f137407a740 0x7f137407cc00 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f1384009b30 tx=0x7f1384005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:47.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.617+0000 7f138b400640 1 -- 172.21.15.151:0/3043567147 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1380062df0 con 0x7f139415bc30 2024-09-16T15:53:47.801 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.799+0000 7f1398c00640 1 -- 172.21.15.151:0/3043567147 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f13940d01f0 con 0x7f139415bc30 2024-09-16T15:53:47.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.801+0000 7f138b400640 1 -- 172.21.15.151:0/3043567147 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v132) ==== 74+0+16937 (secure 0 0 0) 0x7f1380067820 con 0x7f139415bc30 2024-09-16T15:53:47.803 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:53:47.803 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":132,"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","created":"2024-09-16T15:38:56.348535+0000","modified":"2024-09-16T15:53:46.461873+0000","last_up_change":"2024-09-16T15:39:10.681384+0000","last_in_change":"2024-09-16T15:39:00.544477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:39:11.602252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:53:44.212522+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-16T15:53:45.415072+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5524076-a903-4d0a-af6a-caf474c4765d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6809","nonce":3111342741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6811","nonce":3111342741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6815","nonce":3111342741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6813","nonce":3111342741}]},"public_addr":"172.21.15.151:6809/3111342741","cluster_addr":"172.21.15.151:6811/3111342741","heartbeat_back_addr":"172.21.15.151:6815/3111342741","heartbeat_front_addr":"172.21.15.151:6813/3111342741","state":["exists","up"]},{"osd":1,"uuid":"dd25925f-0ae1-494e-8529-4dc205759db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6801","nonce":3368304087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6803","nonce":3368304087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6807","nonce":3368304087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6805","nonce":3368304087}]},"public_addr":"172.21.15.151:6801/3368304087","cluster_addr":"172.21.15.151:6803/3368304087","heartbeat_back_addr":"172.21.15.151:6807/3368304087","heartbeat_front_addr":"172.21.15.151:6805/3368304087","state":["exists","up"]},{"osd":2,"uuid":"c2f18115-013d-4da6-a0fe-cb978f4494b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6823","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6825","nonce":3079373271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6827","nonce":3079373271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6831","nonce":3079373271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6829","nonce":3079373271}]},"public_addr":"172.21.15.151:6825/3079373271","cluster_addr":"172.21.15.151:6827/3079373271","heartbeat_back_addr":"172.21.15.151:6831/3079373271","heartbeat_front_addr":"172.21.15.151:6829/3079373271","state":["exists","up"]},{"osd":3,"uuid":"36edb718-2fcb-4d88-8ddc-c321eed4279d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6817","nonce":1142509041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6819","nonce":1142509041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6824","nonce":1142509041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6821","nonce":1142509041}]},"public_addr":"172.21.15.151:6817/1142509041","cluster_addr":"172.21.15.151:6819/1142509041","heartbeat_back_addr":"172.21.15.151:6824/1142509041","heartbeat_front_addr":"172.21.15.151:6821/1142509041","state":["exists","up"]},{"osd":4,"uuid":"64d33708-cd0a-4236-abe3-1b646e82ab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6811","nonce":2735495271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6813","nonce":2735495271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6817","nonce":2735495271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6815","nonce":2735495271}]},"public_addr":"172.21.15.167:6811/2735495271","cluster_addr":"172.21.15.167:6813/2735495271","heartbeat_back_addr":"172.21.15.167:6817/2735495271","heartbeat_front_addr":"172.21.15.167:6815/2735495271","state":["exists","up"]},{"osd":5,"uuid":"f14b44a9-5910-458c-b518-09a8d0de94de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6827","nonce":1494916572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6829","nonce":1494916572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6833","nonce":1494916572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6831","nonce":1494916572}]},"public_addr":"172.21.15.167:6827/1494916572","cluster_addr":"172.21.15.167:6829/1494916572","heartbeat_back_addr":"172.21.15.167:6833/1494916572","heartbeat_front_addr":"172.21.15.167:6831/1494916572","state":["exists","up"]},{"osd":6,"uuid":"39335545-2654-4710-84b0-a9aa47cfcbba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6819","nonce":2538273273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6821","nonce":2538273273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6825","nonce":2538273273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6823","nonce":2538273273}]},"public_addr":"172.21.15.167:6819/2538273273","cluster_addr":"172.21.15.167:6821/2538273273","heartbeat_back_addr":"172.21.15.167:6825/2538273273","heartbeat_front_addr":"172.21.15.167:6823/2538273273","state":["exists","up"]},{"osd":7,"uuid":"4f8bfc81-53dd-4339-ac22-1a4007d9f963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6803","nonce":265072227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6805","nonce":265072227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6809","nonce":265072227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6807","nonce":265072227}]},"public_addr":"172.21.15.167:6803/265072227","cluster_addr":"172.21.15.167:6805/265072227","heartbeat_back_addr":"172.21.15.167:6809/265072227","heartbeat_front_addr":"172.21.15.167:6807/265072227","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.150359+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.084358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.722368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.332809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.320709+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.395153+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.935250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.801443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:53:47.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.803+0000 7f1398c00640 1 -- 172.21.15.151:0/3043567147 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f137407a740 msgr2=0x7f137407cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:47.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.803+0000 7f1398c00640 1 --2- 172.21.15.151:0/3043567147 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f137407a740 0x7f137407cc00 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f1384009b30 tx=0x7f1384005f90 comp rx=0 tx=0).stop 2024-09-16T15:53:47.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.803+0000 7f1398c00640 1 -- 172.21.15.151:0/3043567147 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f139415bc30 msgr2=0x7f13941f8180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:47.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.803+0000 7f1398c00640 1 --2- 172.21.15.151:0/3043567147 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f139415bc30 0x7f13941f8180 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f138000c450 tx=0x7f138000c910 comp rx=0 tx=0).stop 2024-09-16T15:53:47.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.803+0000 7f1398c00640 1 -- 172.21.15.151:0/3043567147 shutdown_connections 2024-09-16T15:53:47.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.803+0000 7f1398c00640 1 --2- 172.21.15.151:0/3043567147 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f137407a740 0x7f137407cc00 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:47.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.803+0000 7f1398c00640 1 --2- 172.21.15.151:0/3043567147 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13941f86c0 0x7f1394148e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:47.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.803+0000 7f1398c00640 1 --2- 172.21.15.151:0/3043567147 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f139415bc30 0x7f13941f8180 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:47.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.803+0000 7f1398c00640 1 --2- 172.21.15.151:0/3043567147 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f13940d0540 0x7f13941f7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:47.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.803+0000 7f1398c00640 1 -- 172.21.15.151:0/3043567147 >> 172.21.15.151:0/3043567147 conn(0x7f13940cfa20 msgr2=0x7f1394158200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:47.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.803+0000 7f1398c00640 1 -- 172.21.15.151:0/3043567147 shutdown_connections 2024-09-16T15:53:47.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.803+0000 7f1398c00640 1 -- 172.21.15.151:0/3043567147 wait complete. 2024-09-16T15:53:47.822 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-16T15:53:47.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.980+0000 7fcfd3800640 1 -- 172.21.15.151:0/3697813399 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfcc0d2960 msgr2=0x7fcfcc160510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:47.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.980+0000 7fcfd3800640 1 --2- 172.21.15.151:0/3697813399 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfcc0d2960 0x7fcfcc160510 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc0b1440 tx=0x7fcfb802a5f0 comp rx=0 tx=0).stop 2024-09-16T15:53:47.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.980+0000 7fcfd3800640 1 -- 172.21.15.151:0/3697813399 shutdown_connections 2024-09-16T15:53:47.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.980+0000 7fcfd3800640 1 --2- 172.21.15.151:0/3697813399 >> v2:172.21.15.167:3300/0 conn(0x7fcfcc0b0b30 0x7fcfcc0af320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:47.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.980+0000 7fcfd3800640 1 --2- 172.21.15.151:0/3697813399 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfcc0d2960 0x7fcfcc160510 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:47.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.980+0000 7fcfd3800640 1 -- 172.21.15.151:0/3697813399 >> 172.21.15.151:0/3697813399 conn(0x7fcfcc0d1e40 msgr2=0x7fcfcc0d2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:47.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.980+0000 7fcfd3800640 1 -- 172.21.15.151:0/3697813399 shutdown_connections 2024-09-16T15:53:47.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.980+0000 7fcfd3800640 1 -- 172.21.15.151:0/3697813399 wait complete. 2024-09-16T15:53:47.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.982+0000 7fcfd3800640 1 Processor -- start 2024-09-16T15:53:47.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.982+0000 7fcfd3800640 1 -- start start 2024-09-16T15:53:47.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.983+0000 7fcfd3800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcfcc0b0b30 0x7fcfcc152500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:47.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.983+0000 7fcfd3800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfcc0d00b0 0x7fcfcc0c5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:47.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.983+0000 7fcfd3800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcfcc0d2960 0x7fcfcc15a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:47.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.983+0000 7fcfd3800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcfcc0af860 con 0x7fcfcc0d00b0 2024-09-16T15:53:47.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.983+0000 7fcfd3800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcfcc0d06a0 con 0x7fcfcc0d2960 2024-09-16T15:53:47.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.983+0000 7fcfd3800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fcfcc0af9e0 con 0x7fcfcc0b0b30 2024-09-16T15:53:47.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.983+0000 7fcfcbe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfcc0d00b0 0x7fcfcc0c5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:47.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.983+0000 7fcfcbe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfcc0d00b0 0x7fcfcc0c5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43672/0 (socket says 172.21.15.151:43672) 2024-09-16T15:53:47.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.983+0000 7fcfcbe00640 1 -- 172.21.15.151:0/1602204637 learned_addr learned my addr 172.21.15.151:0/1602204637 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:53:47.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.983+0000 7fcfd1800640 1 --2- 172.21.15.151:0/1602204637 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcfcc0d2960 0x7fcfcc15a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:47.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.983+0000 7fcfd0e00640 1 --2- 172.21.15.151:0/1602204637 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcfcc0b0b30 0x7fcfcc152500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:47.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.984+0000 7fcfcbe00640 1 -- 172.21.15.151:0/1602204637 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcfcc0b0b30 msgr2=0x7fcfcc152500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:47.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.984+0000 7fcfcbe00640 1 --2- 172.21.15.151:0/1602204637 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcfcc0b0b30 0x7fcfcc152500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:47.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.984+0000 7fcfcbe00640 1 -- 172.21.15.151:0/1602204637 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcfcc0d2960 msgr2=0x7fcfcc15a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:47.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.984+0000 7fcfcbe00640 1 --2- 172.21.15.151:0/1602204637 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcfcc0d2960 0x7fcfcc15a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:47.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.984+0000 7fcfcbe00640 1 -- 172.21.15.151:0/1602204637 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfcc0cd260 con 0x7fcfcc0d00b0 2024-09-16T15:53:47.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.984+0000 7fcfd0e00640 1 --2- 172.21.15.151:0/1602204637 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcfcc0b0b30 0x7fcfcc152500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:53:47.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.984+0000 7fcfcbe00640 1 --2- 172.21.15.151:0/1602204637 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfcc0d00b0 0x7fcfcc0c5970 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8034040 tx=0x7fcfb8003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:47.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.984+0000 7fcfc8c00640 1 -- 172.21.15.151:0/1602204637 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb803e030 con 0x7fcfcc0d00b0 2024-09-16T15:53:47.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.984+0000 7fcfc8c00640 1 -- 172.21.15.151:0/1602204637 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcfb8003e00 con 0x7fcfcc0d00b0 2024-09-16T15:53:47.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.984+0000 7fcfc8c00640 1 -- 172.21.15.151:0/1602204637 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb8035040 con 0x7fcfcc0d00b0 2024-09-16T15:53:47.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.984+0000 7fcfd3800640 1 -- 172.21.15.151:0/1602204637 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfcc0cd540 con 0x7fcfcc0d00b0 2024-09-16T15:53:47.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.984+0000 7fcfd3800640 1 -- 172.21.15.151:0/1602204637 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfcc0cdfc0 con 0x7fcfcc0d00b0 2024-09-16T15:53:47.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.985+0000 7fcfd3800640 1 -- 172.21.15.151:0/1602204637 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfcc0af320 con 0x7fcfcc0d00b0 2024-09-16T15:53:47.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.987+0000 7fcfc8c00640 1 -- 172.21.15.151:0/1602204637 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 47) ==== 104092+0+0 (secure 0 0 0) 0x7fcfb8041030 con 0x7fcfcc0d00b0 2024-09-16T15:53:47.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.987+0000 7fcfc8c00640 1 --2- 172.21.15.151:0/1602204637 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcfac07a770 0x7fcfac07cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:47.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.988+0000 7fcfc8c00640 1 -- 172.21.15.151:0/1602204637 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5821+0+0 (secure 0 0 0) 0x7fcfb80b9f90 con 0x7fcfcc0d00b0 2024-09-16T15:53:47.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.988+0000 7fcfd0e00640 1 --2- 172.21.15.151:0/1602204637 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcfac07a770 0x7fcfac07cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:47.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.988+0000 7fcfd0e00640 1 --2- 172.21.15.151:0/1602204637 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcfac07a770 0x7fcfac07cc30 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc007fd0 tx=0x7fcfbc009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:47.994 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:47.992+0000 7fcfc8c00640 1 -- 172.21.15.151:0/1602204637 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fcfb8083a10 con 0x7fcfcc0d00b0 2024-09-16T15:53:48.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:48.215+0000 7fcfd3800640 1 -- 172.21.15.151:0/1602204637 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fcfcc0c5330 con 0x7fcfcc0d00b0 2024-09-16T15:53:49.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:49.058+0000 7fcfc8c00640 1 -- 172.21.15.151:0/1602204637 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v86) ==== 118+0+0 (secure 0 0 0) 0x7fcfb8088440 con 0x7fcfcc0d00b0 2024-09-16T15:53:49.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:49.061+0000 7fcfd3800640 1 -- 172.21.15.151:0/1602204637 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcfac07a770 msgr2=0x7fcfac07cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:49.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:49.061+0000 7fcfd3800640 1 --2- 172.21.15.151:0/1602204637 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcfac07a770 0x7fcfac07cc30 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc007fd0 tx=0x7fcfbc009040 comp rx=0 tx=0).stop 2024-09-16T15:53:49.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:49.061+0000 7fcfd3800640 1 -- 172.21.15.151:0/1602204637 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfcc0d00b0 msgr2=0x7fcfcc0c5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:49.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:49.061+0000 7fcfd3800640 1 --2- 172.21.15.151:0/1602204637 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfcc0d00b0 0x7fcfcc0c5970 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8034040 tx=0x7fcfb8003910 comp rx=0 tx=0).stop 2024-09-16T15:53:49.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:49.061+0000 7fcfd3800640 1 -- 172.21.15.151:0/1602204637 shutdown_connections 2024-09-16T15:53:49.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:49.061+0000 7fcfd3800640 1 --2- 172.21.15.151:0/1602204637 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcfac07a770 0x7fcfac07cc30 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:49.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:49.061+0000 7fcfd3800640 1 --2- 172.21.15.151:0/1602204637 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcfcc0d2960 0x7fcfcc15a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:49.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:49.061+0000 7fcfd3800640 1 --2- 172.21.15.151:0/1602204637 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfcc0d00b0 0x7fcfcc0c5970 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:49.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:49.061+0000 7fcfd3800640 1 --2- 172.21.15.151:0/1602204637 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcfcc0b0b30 0x7fcfcc152500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:49.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:49.061+0000 7fcfd3800640 1 -- 172.21.15.151:0/1602204637 >> 172.21.15.151:0/1602204637 conn(0x7fcfcc0d1e40 msgr2=0x7fcfcc0b4d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:49.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:49.061+0000 7fcfd3800640 1 -- 172.21.15.151:0/1602204637 shutdown_connections 2024-09-16T15:53:49.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:49.061+0000 7fcfd3800640 1 -- 172.21.15.151:0/1602204637 wait complete. 2024-09-16T15:53:49.081 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-16T15:53:49.081 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-16T15:53:49.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:49.241+0000 7fdd9d200640 1 -- 172.21.15.151:0/2577631094 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdd7c0052e0 msgr2=0x7fdd7c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:49.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:49.241+0000 7fdd9d200640 1 --2- 172.21.15.151:0/2577631094 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdd7c0052e0 0x7fdd7c00b9c0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fdd90005fd0 tx=0x7fdd7c005830 comp rx=0 tx=0).stop 2024-09-16T15:53:49.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:49.241+0000 7fdd9d200640 1 -- 172.21.15.151:0/2577631094 shutdown_connections 2024-09-16T15:53:49.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:49.241+0000 7fdd9d200640 1 --2- 172.21.15.151:0/2577631094 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdd7c0052e0 0x7fdd7c00b9c0 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:49.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:49.241+0000 7fdd9d200640 1 -- 172.21.15.151:0/2577631094 >> 172.21.15.151:0/2577631094 conn(0x7fdd980b1840 msgr2=0x7fdd980b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:49.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:49.242+0000 7fdd9d200640 1 -- 172.21.15.151:0/2577631094 shutdown_connections 2024-09-16T15:53:49.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:49.242+0000 7fdd9d200640 1 -- 172.21.15.151:0/2577631094 wait complete. 2024-09-16T15:53:49.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:49.243+0000 7fdd9d200640 1 Processor -- start 2024-09-16T15:53:49.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:49.243+0000 7fdd9d200640 1 -- start start 2024-09-16T15:53:49.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:49.244+0000 7fdd9d200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdd7c0052e0 0x7fdd9815eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:49.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:49.244+0000 7fdd9d200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdd981f7f30 0x7fdd981f83b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:49.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:49.244+0000 7fdd9d200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd981e7f30 0x7fdd981e83b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:49.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:49.244+0000 7fdd9d200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdd980737d0 con 0x7fdd981f7f30 2024-09-16T15:53:49.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:49.244+0000 7fdd9d200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdd98073650 con 0x7fdd981e7f30 2024-09-16T15:53:49.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:49.244+0000 7fdd9d200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fdd980cc5c0 con 0x7fdd7c0052e0 2024-09-16T15:53:49.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:49.244+0000 7fdd8fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdd981f7f30 0x7fdd981f83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:49.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:49.244+0000 7fdd8fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdd981f7f30 0x7fdd981f83b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43684/0 (socket says 172.21.15.151:43684) 2024-09-16T15:53:49.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:49.244+0000 7fdd8fe00640 1 -- 172.21.15.151:0/2035226403 learned_addr learned my addr 172.21.15.151:0/2035226403 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:53:49.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:49.245+0000 7fdd96800640 1 --2- 172.21.15.151:0/2035226403 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd981e7f30 0x7fdd981e83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:49.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:49.245+0000 7fdd95e00640 1 --2- 172.21.15.151:0/2035226403 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdd7c0052e0 0x7fdd9815eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:49.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:49.245+0000 7fdd8fe00640 1 -- 172.21.15.151:0/2035226403 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdd7c0052e0 msgr2=0x7fdd9815eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:49.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:49.245+0000 7fdd8fe00640 1 --2- 172.21.15.151:0/2035226403 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdd7c0052e0 0x7fdd9815eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:49.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:49.245+0000 7fdd8fe00640 1 -- 172.21.15.151:0/2035226403 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd981e7f30 msgr2=0x7fdd981e83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:49.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:49.245+0000 7fdd8fe00640 1 --2- 172.21.15.151:0/2035226403 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd981e7f30 0x7fdd981e83b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:49.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:49.245+0000 7fdd8fe00640 1 -- 172.21.15.151:0/2035226403 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd981e8ca0 con 0x7fdd981f7f30 2024-09-16T15:53:49.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:49.245+0000 7fdd8fe00640 1 --2- 172.21.15.151:0/2035226403 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdd981f7f30 0x7fdd981f83b0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fdd8000e5c0 tx=0x7fdd8000ea80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:49.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:49.245+0000 7fdd8e000640 1 -- 172.21.15.151:0/2035226403 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd80015070 con 0x7fdd981f7f30 2024-09-16T15:53:49.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:49.245+0000 7fdd8e000640 1 -- 172.21.15.151:0/2035226403 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdd8000ca00 con 0x7fdd981f7f30 2024-09-16T15:53:49.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:49.245+0000 7fdd9d200640 1 -- 172.21.15.151:0/2035226403 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd981e8f80 con 0x7fdd981f7f30 2024-09-16T15:53:49.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:49.245+0000 7fdd8e000640 1 -- 172.21.15.151:0/2035226403 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd8000ce50 con 0x7fdd981f7f30 2024-09-16T15:53:49.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:49.246+0000 7fdd9d200640 1 -- 172.21.15.151:0/2035226403 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd981e9540 con 0x7fdd981f7f30 2024-09-16T15:53:49.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:49.247+0000 7fdd9d200640 1 -- 172.21.15.151:0/2035226403 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd9815f9f0 con 0x7fdd981f7f30 2024-09-16T15:53:49.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:49.248+0000 7fdd8e000640 1 -- 172.21.15.151:0/2035226403 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 47) ==== 104092+0+0 (secure 0 0 0) 0x7fdd80018070 con 0x7fdd981f7f30 2024-09-16T15:53:49.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:49.249+0000 7fdd8e000640 1 --2- 172.21.15.151:0/2035226403 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdd7407a910 0x7fdd7407cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:49.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:49.249+0000 7fdd8e000640 1 -- 172.21.15.151:0/2035226403 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5821+0+0 (secure 0 0 0) 0x7fdd80099da0 con 0x7fdd981f7f30 2024-09-16T15:53:49.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:49.249+0000 7fdd95e00640 1 --2- 172.21.15.151:0/2035226403 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdd7407a910 0x7fdd7407cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:49.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:49.250+0000 7fdd95e00640 1 --2- 172.21.15.151:0/2035226403 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdd7407a910 0x7fdd7407cdd0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fdd980cfb40 tx=0x7fdd88006b70 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:49.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:49.253+0000 7fdd8e000640 1 -- 172.21.15.151:0/2035226403 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fdd80063df0 con 0x7fdd981f7f30 2024-09-16T15:53:49.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:49.475+0000 7fdd9d200640 1 -- 172.21.15.151:0/2035226403 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fdd980cc3a0 con 0x7fdd981f7f30 2024-09-16T15:53:50.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.065+0000 7fdd8e000640 1 -- 172.21.15.151:0/2035226403 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v87) ==== 122+0+0 (secure 0 0 0) 0x7fdd800636f0 con 0x7fdd981f7f30 2024-09-16T15:53:50.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.068+0000 7fdd9d200640 1 -- 172.21.15.151:0/2035226403 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdd7407a910 msgr2=0x7fdd7407cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:50.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.068+0000 7fdd9d200640 1 --2- 172.21.15.151:0/2035226403 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdd7407a910 0x7fdd7407cdd0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fdd980cfb40 tx=0x7fdd88006b70 comp rx=0 tx=0).stop 2024-09-16T15:53:50.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.068+0000 7fdd9d200640 1 -- 172.21.15.151:0/2035226403 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdd981f7f30 msgr2=0x7fdd981f83b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:50.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.068+0000 7fdd9d200640 1 --2- 172.21.15.151:0/2035226403 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdd981f7f30 0x7fdd981f83b0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fdd8000e5c0 tx=0x7fdd8000ea80 comp rx=0 tx=0).stop 2024-09-16T15:53:50.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.068+0000 7fdd9d200640 1 -- 172.21.15.151:0/2035226403 shutdown_connections 2024-09-16T15:53:50.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.068+0000 7fdd9d200640 1 --2- 172.21.15.151:0/2035226403 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdd7407a910 0x7fdd7407cdd0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:50.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.068+0000 7fdd9d200640 1 --2- 172.21.15.151:0/2035226403 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd981e7f30 0x7fdd981e83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:50.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.068+0000 7fdd9d200640 1 --2- 172.21.15.151:0/2035226403 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdd981f7f30 0x7fdd981f83b0 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:50.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.068+0000 7fdd9d200640 1 --2- 172.21.15.151:0/2035226403 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdd7c0052e0 0x7fdd9815eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:50.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.069+0000 7fdd9d200640 1 -- 172.21.15.151:0/2035226403 >> 172.21.15.151:0/2035226403 conn(0x7fdd980b1840 msgr2=0x7fdd9806f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:50.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.069+0000 7fdd9d200640 1 -- 172.21.15.151:0/2035226403 shutdown_connections 2024-09-16T15:53:50.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.069+0000 7fdd9d200640 1 -- 172.21.15.151:0/2035226403 wait complete. 2024-09-16T15:53:50.088 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-16T15:53:50.088 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:53:50.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.245+0000 7fba3be00640 1 -- 172.21.15.151:0/2315665574 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba340ccf00 msgr2=0x7fba340b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:50.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.245+0000 7fba3be00640 1 --2- 172.21.15.151:0/2315665574 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba340ccf00 0x7fba340b2440 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fba3406dac0 tx=0x7fba2002a4a0 comp rx=0 tx=0).stop 2024-09-16T15:53:50.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.245+0000 7fba3be00640 1 -- 172.21.15.151:0/2315665574 shutdown_connections 2024-09-16T15:53:50.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.245+0000 7fba3be00640 1 --2- 172.21.15.151:0/2315665574 >> v2:172.21.15.167:3300/0 conn(0x7fba34073fb0 0x7fba34074420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:50.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.245+0000 7fba3be00640 1 --2- 172.21.15.151:0/2315665574 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba340ccf00 0x7fba340b2440 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:50.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.245+0000 7fba3be00640 1 -- 172.21.15.151:0/2315665574 >> 172.21.15.151:0/2315665574 conn(0x7fba340b1950 msgr2=0x7fba340b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:50.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.245+0000 7fba3be00640 1 -- 172.21.15.151:0/2315665574 shutdown_connections 2024-09-16T15:53:50.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.245+0000 7fba3be00640 1 -- 172.21.15.151:0/2315665574 wait complete. 2024-09-16T15:53:50.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.247+0000 7fba3be00640 1 Processor -- start 2024-09-16T15:53:50.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.247+0000 7fba3be00640 1 -- start start 2024-09-16T15:53:50.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.248+0000 7fba3be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba34073fb0 0x7fba3414e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:50.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.248+0000 7fba3be00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fba340ccf00 0x7fba3414c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:50.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.248+0000 7fba3be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba3415f5a0 0x7fba3415a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:50.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.248+0000 7fba3be00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fba3406f2e0 con 0x7fba3415f5a0 2024-09-16T15:53:50.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.248+0000 7fba3be00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fba3406f160 con 0x7fba34073fb0 2024-09-16T15:53:50.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.248+0000 7fba3be00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fba3406f460 con 0x7fba340ccf00 2024-09-16T15:53:50.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.248+0000 7fba39e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba3415f5a0 0x7fba3415a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:50.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.248+0000 7fba39e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba3415f5a0 0x7fba3415a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43708/0 (socket says 172.21.15.151:43708) 2024-09-16T15:53:50.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.248+0000 7fba39e00640 1 -- 172.21.15.151:0/3349520542 learned_addr learned my addr 172.21.15.151:0/3349520542 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:53:50.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.249+0000 7fba39e00640 1 -- 172.21.15.151:0/3349520542 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fba340ccf00 msgr2=0x7fba3414c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:50.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.249+0000 7fba39e00640 1 --2- 172.21.15.151:0/3349520542 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fba340ccf00 0x7fba3414c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:50.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.249+0000 7fba39e00640 1 -- 172.21.15.151:0/3349520542 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba34073fb0 msgr2=0x7fba3414e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:50.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.249+0000 7fba39400640 1 --2- 172.21.15.151:0/3349520542 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba34073fb0 0x7fba3414e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:50.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.249+0000 7fba39e00640 1 --2- 172.21.15.151:0/3349520542 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba34073fb0 0x7fba3414e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:50.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.249+0000 7fba39e00640 1 -- 172.21.15.151:0/3349520542 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba3415b4b0 con 0x7fba3415f5a0 2024-09-16T15:53:50.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.249+0000 7fba39400640 1 --2- 172.21.15.151:0/3349520542 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba34073fb0 0x7fba3414e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:53:50.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.249+0000 7fba39e00640 1 --2- 172.21.15.151:0/3349520542 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba3415f5a0 0x7fba3415a4a0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fba2c00ead0 tx=0x7fba2c00ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:50.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.249+0000 7fba31600640 1 -- 172.21.15.151:0/3349520542 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2c00cde0 con 0x7fba3415f5a0 2024-09-16T15:53:50.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.249+0000 7fba31600640 1 -- 172.21.15.151:0/3349520542 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fba2c004c10 con 0x7fba3415f5a0 2024-09-16T15:53:50.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.249+0000 7fba31600640 1 -- 172.21.15.151:0/3349520542 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2c005020 con 0x7fba3415f5a0 2024-09-16T15:53:50.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.249+0000 7fba3be00640 1 -- 172.21.15.151:0/3349520542 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba3415aef0 con 0x7fba3415f5a0 2024-09-16T15:53:50.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.249+0000 7fba3be00640 1 -- 172.21.15.151:0/3349520542 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba340cdb80 con 0x7fba3415f5a0 2024-09-16T15:53:50.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.250+0000 7fba3be00640 1 -- 172.21.15.151:0/3349520542 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba340ce060 con 0x7fba3415f5a0 2024-09-16T15:53:50.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.252+0000 7fba31600640 1 -- 172.21.15.151:0/3349520542 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 47) ==== 104092+0+0 (secure 0 0 0) 0x7fba2c0051c0 con 0x7fba3415f5a0 2024-09-16T15:53:50.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.253+0000 7fba31600640 1 --2- 172.21.15.151:0/3349520542 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fba1407a740 0x7fba1407cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:50.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.253+0000 7fba31600640 1 -- 172.21.15.151:0/3349520542 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5821+0+0 (secure 0 0 0) 0x7fba2c09a330 con 0x7fba3415f5a0 2024-09-16T15:53:50.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.253+0000 7fba39400640 1 --2- 172.21.15.151:0/3349520542 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fba1407a740 0x7fba1407cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:50.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.254+0000 7fba39400640 1 --2- 172.21.15.151:0/3349520542 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fba1407a740 0x7fba1407cc00 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fba28007fd0 tx=0x7fba28009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:50.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.257+0000 7fba31600640 1 -- 172.21.15.151:0/3349520542 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fba340ce060 con 0x7fba3415f5a0 2024-09-16T15:53:50.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.476+0000 7fba3be00640 1 -- 172.21.15.151:0/3349520542 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fba3406f9f0 con 0x7fba3415f5a0 2024-09-16T15:53:50.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.476+0000 7fba31600640 1 -- 172.21.15.151:0/3349520542 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) ==== 94+0+5234 (secure 0 0 0) 0x7fba2c016030 con 0x7fba3415f5a0 2024-09-16T15:53:50.479 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:53:50.479 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":87,"btime":"2024-09-16T15:53:50:060393+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8994,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/2450183746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":2450183746},{"type":"v1","addr":"172.21.15.151:6833","nonce":2450183746}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8997,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1952630294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1952630294},{"type":"v1","addr":"172.21.15.167:6836","nonce":1952630294}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":9000,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/127383056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":127383056},{"type":"v1","addr":"172.21.15.167:6837","nonce":127383056}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:53:46.461837+0000","modified":"2024-09-16T15:53:49.478024+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9009},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9009":{"gid":9009,"name":"a","rank":0,"incarnation":83,"state":"up:active","state_seq":3,"addr":"172.21.15.151:6835/1977294668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":1977294668},{"type":"v1","addr":"172.21.15.151:6835","nonce":1977294668}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9009,"qdb_cluster":[9009]},"id":9}]} 2024-09-16T15:53:50.479 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 87 2024-09-16T15:53:50.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.479+0000 7fba3be00640 1 -- 172.21.15.151:0/3349520542 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fba1407a740 msgr2=0x7fba1407cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:50.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.479+0000 7fba3be00640 1 --2- 172.21.15.151:0/3349520542 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fba1407a740 0x7fba1407cc00 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fba28007fd0 tx=0x7fba28009040 comp rx=0 tx=0).stop 2024-09-16T15:53:50.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.479+0000 7fba3be00640 1 -- 172.21.15.151:0/3349520542 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba3415f5a0 msgr2=0x7fba3415a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:50.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.479+0000 7fba3be00640 1 --2- 172.21.15.151:0/3349520542 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba3415f5a0 0x7fba3415a4a0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fba2c00ead0 tx=0x7fba2c00ef90 comp rx=0 tx=0).stop 2024-09-16T15:53:50.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.479+0000 7fba3be00640 1 -- 172.21.15.151:0/3349520542 shutdown_connections 2024-09-16T15:53:50.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.479+0000 7fba3be00640 1 --2- 172.21.15.151:0/3349520542 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fba1407a740 0x7fba1407cc00 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:50.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.479+0000 7fba3be00640 1 --2- 172.21.15.151:0/3349520542 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba3415f5a0 0x7fba3415a4a0 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:50.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.479+0000 7fba3be00640 1 --2- 172.21.15.151:0/3349520542 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fba340ccf00 0x7fba3414c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:50.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.479+0000 7fba3be00640 1 --2- 172.21.15.151:0/3349520542 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fba34073fb0 0x7fba3414e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:50.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.479+0000 7fba3be00640 1 -- 172.21.15.151:0/3349520542 >> 172.21.15.151:0/3349520542 conn(0x7fba340b1950 msgr2=0x7fba34153f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:50.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.479+0000 7fba3be00640 1 -- 172.21.15.151:0/3349520542 shutdown_connections 2024-09-16T15:53:50.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.479+0000 7fba3be00640 1 -- 172.21.15.151:0/3349520542 wait complete. 2024-09-16T15:53:50.497 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:53:50.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.653+0000 7fb414c00640 1 -- 172.21.15.151:0/2981803334 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4100d0540 msgr2=0x7fb41006f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:50.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.653+0000 7fb414c00640 1 --2- 172.21.15.151:0/2981803334 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4100d0540 0x7fb41006f320 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fb408005fd0 tx=0x7fb40802aa90 comp rx=0 tx=0).stop 2024-09-16T15:53:50.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.653+0000 7fb414c00640 1 -- 172.21.15.151:0/2981803334 shutdown_connections 2024-09-16T15:53:50.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.653+0000 7fb414c00640 1 --2- 172.21.15.151:0/2981803334 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4100d0540 0x7fb41006f320 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:50.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.653+0000 7fb414c00640 1 --2- 172.21.15.151:0/2981803334 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb4100725d0 0x7fb4100b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:50.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.653+0000 7fb414c00640 1 -- 172.21.15.151:0/2981803334 >> 172.21.15.151:0/2981803334 conn(0x7fb4100cfa20 msgr2=0x7fb4100cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:50.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.653+0000 7fb414c00640 1 -- 172.21.15.151:0/2981803334 shutdown_connections 2024-09-16T15:53:50.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.653+0000 7fb414c00640 1 -- 172.21.15.151:0/2981803334 wait complete. 2024-09-16T15:53:50.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.655+0000 7fb414c00640 1 Processor -- start 2024-09-16T15:53:50.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.655+0000 7fb414c00640 1 -- start start 2024-09-16T15:53:50.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.656+0000 7fb414c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4100d0540 0x7fb4101f51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:50.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.656+0000 7fb414c00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb41015bc30 0x7fb410155f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:50.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.656+0000 7fb414c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4100c7fa0 0x7fb4100ad880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:50.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.656+0000 7fb414c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb410074660 con 0x7fb4100d0540 2024-09-16T15:53:50.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.656+0000 7fb414c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb41006f860 con 0x7fb4100c7fa0 2024-09-16T15:53:50.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.656+0000 7fb414c00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fb410154d50 con 0x7fb41015bc30 2024-09-16T15:53:50.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.656+0000 7fb40f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4100d0540 0x7fb4101f51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:50.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.656+0000 7fb40f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4100d0540 0x7fb4101f51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43724/0 (socket says 172.21.15.151:43724) 2024-09-16T15:53:50.660 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.656+0000 7fb40f400640 1 -- 172.21.15.151:0/758579398 learned_addr learned my addr 172.21.15.151:0/758579398 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:53:50.660 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.656+0000 7fb407e00640 1 --2- 172.21.15.151:0/758579398 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb41015bc30 0x7fb410155f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:50.660 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.656+0000 7fb40fe00640 1 --2- 172.21.15.151:0/758579398 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4100c7fa0 0x7fb4100ad880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:50.660 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.656+0000 7fb40f400640 1 -- 172.21.15.151:0/758579398 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb41015bc30 msgr2=0x7fb410155f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:50.660 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.656+0000 7fb40f400640 1 --2- 172.21.15.151:0/758579398 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb41015bc30 0x7fb410155f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:50.660 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.656+0000 7fb40f400640 1 -- 172.21.15.151:0/758579398 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4100c7fa0 msgr2=0x7fb4100ad880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:50.660 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.656+0000 7fb40f400640 1 --2- 172.21.15.151:0/758579398 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4100c7fa0 0x7fb4100ad880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:50.660 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.657+0000 7fb40f400640 1 -- 172.21.15.151:0/758579398 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4100c8430 con 0x7fb4100d0540 2024-09-16T15:53:50.660 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.657+0000 7fb40f400640 1 --2- 172.21.15.151:0/758579398 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4100d0540 0x7fb4101f51b0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7fb40000c3d0 tx=0x7fb40000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:50.660 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.657+0000 7fb407400640 1 -- 172.21.15.151:0/758579398 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb400017070 con 0x7fb4100d0540 2024-09-16T15:53:50.660 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.657+0000 7fb407400640 1 -- 172.21.15.151:0/758579398 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb4000053c0 con 0x7fb4100d0540 2024-09-16T15:53:50.660 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.657+0000 7fb414c00640 1 -- 172.21.15.151:0/758579398 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb410156620 con 0x7fb4100d0540 2024-09-16T15:53:50.660 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.657+0000 7fb407400640 1 -- 172.21.15.151:0/758579398 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb400003850 con 0x7fb4100d0540 2024-09-16T15:53:50.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.657+0000 7fb414c00640 1 -- 172.21.15.151:0/758579398 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4101549e0 con 0x7fb4100d0540 2024-09-16T15:53:50.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.658+0000 7fb414c00640 1 -- 172.21.15.151:0/758579398 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4100caec0 con 0x7fb4100d0540 2024-09-16T15:53:50.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.659+0000 7fb407400640 1 -- 172.21.15.151:0/758579398 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 47) ==== 104092+0+0 (secure 0 0 0) 0x7fb40001a070 con 0x7fb4100d0540 2024-09-16T15:53:50.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.660+0000 7fb407400640 1 --2- 172.21.15.151:0/758579398 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb3ec07a740 0x7fb3ec07cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:50.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.661+0000 7fb407400640 1 -- 172.21.15.151:0/758579398 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5821+0+0 (secure 0 0 0) 0x7fb400098d10 con 0x7fb4100d0540 2024-09-16T15:53:50.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.661+0000 7fb407e00640 1 --2- 172.21.15.151:0/758579398 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb3ec07a740 0x7fb3ec07cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:50.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.662+0000 7fb407e00640 1 --2- 172.21.15.151:0/758579398 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb3ec07a740 0x7fb3ec07cc00 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fb3f80095f0 tx=0x7fb3f8005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:50.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.665+0000 7fb407400640 1 -- 172.21.15.151:0/758579398 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb4100caec0 con 0x7fb4100d0540 2024-09-16T15:53:50.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.853+0000 7fb414c00640 1 -- 172.21.15.151:0/758579398 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb41006fdf0 con 0x7fb4100d0540 2024-09-16T15:53:50.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.854+0000 7fb407400640 1 -- 172.21.15.151:0/758579398 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v133) ==== 74+0+16924 (secure 0 0 0) 0x7fb400062710 con 0x7fb4100d0540 2024-09-16T15:53:50.856 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:53:50.856 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":133,"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","created":"2024-09-16T15:38:56.348535+0000","modified":"2024-09-16T15:53:48.028984+0000","last_up_change":"2024-09-16T15:39:10.681384+0000","last_in_change":"2024-09-16T15:39:00.544477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:39:11.602252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:53:44.212522+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-16T15:53:45.415072+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5524076-a903-4d0a-af6a-caf474c4765d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6809","nonce":3111342741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6811","nonce":3111342741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6815","nonce":3111342741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6813","nonce":3111342741}]},"public_addr":"172.21.15.151:6809/3111342741","cluster_addr":"172.21.15.151:6811/3111342741","heartbeat_back_addr":"172.21.15.151:6815/3111342741","heartbeat_front_addr":"172.21.15.151:6813/3111342741","state":["exists","up"]},{"osd":1,"uuid":"dd25925f-0ae1-494e-8529-4dc205759db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6801","nonce":3368304087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6803","nonce":3368304087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6807","nonce":3368304087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6805","nonce":3368304087}]},"public_addr":"172.21.15.151:6801/3368304087","cluster_addr":"172.21.15.151:6803/3368304087","heartbeat_back_addr":"172.21.15.151:6807/3368304087","heartbeat_front_addr":"172.21.15.151:6805/3368304087","state":["exists","up"]},{"osd":2,"uuid":"c2f18115-013d-4da6-a0fe-cb978f4494b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6823","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6825","nonce":3079373271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6827","nonce":3079373271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6831","nonce":3079373271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6829","nonce":3079373271}]},"public_addr":"172.21.15.151:6825/3079373271","cluster_addr":"172.21.15.151:6827/3079373271","heartbeat_back_addr":"172.21.15.151:6831/3079373271","heartbeat_front_addr":"172.21.15.151:6829/3079373271","state":["exists","up"]},{"osd":3,"uuid":"36edb718-2fcb-4d88-8ddc-c321eed4279d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6817","nonce":1142509041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6819","nonce":1142509041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6824","nonce":1142509041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6821","nonce":1142509041}]},"public_addr":"172.21.15.151:6817/1142509041","cluster_addr":"172.21.15.151:6819/1142509041","heartbeat_back_addr":"172.21.15.151:6824/1142509041","heartbeat_front_addr":"172.21.15.151:6821/1142509041","state":["exists","up"]},{"osd":4,"uuid":"64d33708-cd0a-4236-abe3-1b646e82ab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6811","nonce":2735495271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6813","nonce":2735495271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6817","nonce":2735495271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6815","nonce":2735495271}]},"public_addr":"172.21.15.167:6811/2735495271","cluster_addr":"172.21.15.167:6813/2735495271","heartbeat_back_addr":"172.21.15.167:6817/2735495271","heartbeat_front_addr":"172.21.15.167:6815/2735495271","state":["exists","up"]},{"osd":5,"uuid":"f14b44a9-5910-458c-b518-09a8d0de94de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6827","nonce":1494916572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6829","nonce":1494916572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6833","nonce":1494916572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6831","nonce":1494916572}]},"public_addr":"172.21.15.167:6827/1494916572","cluster_addr":"172.21.15.167:6829/1494916572","heartbeat_back_addr":"172.21.15.167:6833/1494916572","heartbeat_front_addr":"172.21.15.167:6831/1494916572","state":["exists","up"]},{"osd":6,"uuid":"39335545-2654-4710-84b0-a9aa47cfcbba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6819","nonce":2538273273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6821","nonce":2538273273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6825","nonce":2538273273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6823","nonce":2538273273}]},"public_addr":"172.21.15.167:6819/2538273273","cluster_addr":"172.21.15.167:6821/2538273273","heartbeat_back_addr":"172.21.15.167:6825/2538273273","heartbeat_front_addr":"172.21.15.167:6823/2538273273","state":["exists","up"]},{"osd":7,"uuid":"4f8bfc81-53dd-4339-ac22-1a4007d9f963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6803","nonce":265072227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6805","nonce":265072227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6809","nonce":265072227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6807","nonce":265072227}]},"public_addr":"172.21.15.167:6803/265072227","cluster_addr":"172.21.15.167:6805/265072227","heartbeat_back_addr":"172.21.15.167:6809/265072227","heartbeat_front_addr":"172.21.15.167:6807/265072227","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.150359+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.084358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.722368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.332809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.320709+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.395153+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.935250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.801443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:53:50.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.856+0000 7fb414c00640 1 -- 172.21.15.151:0/758579398 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb3ec07a740 msgr2=0x7fb3ec07cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:50.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.856+0000 7fb414c00640 1 --2- 172.21.15.151:0/758579398 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb3ec07a740 0x7fb3ec07cc00 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fb3f80095f0 tx=0x7fb3f8005f90 comp rx=0 tx=0).stop 2024-09-16T15:53:50.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.856+0000 7fb414c00640 1 -- 172.21.15.151:0/758579398 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4100d0540 msgr2=0x7fb4101f51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:50.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.856+0000 7fb414c00640 1 --2- 172.21.15.151:0/758579398 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4100d0540 0x7fb4101f51b0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7fb40000c3d0 tx=0x7fb40000c890 comp rx=0 tx=0).stop 2024-09-16T15:53:50.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.856+0000 7fb414c00640 1 -- 172.21.15.151:0/758579398 shutdown_connections 2024-09-16T15:53:50.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.856+0000 7fb414c00640 1 --2- 172.21.15.151:0/758579398 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb3ec07a740 0x7fb3ec07cc00 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:50.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.856+0000 7fb414c00640 1 --2- 172.21.15.151:0/758579398 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4100c7fa0 0x7fb4100ad880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:50.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.856+0000 7fb414c00640 1 --2- 172.21.15.151:0/758579398 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb41015bc30 0x7fb410155f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:50.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.857+0000 7fb414c00640 1 --2- 172.21.15.151:0/758579398 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4100d0540 0x7fb4101f51b0 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:50.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.857+0000 7fb414c00640 1 -- 172.21.15.151:0/758579398 >> 172.21.15.151:0/758579398 conn(0x7fb4100cfa20 msgr2=0x7fb4100ca7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:50.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.857+0000 7fb414c00640 1 -- 172.21.15.151:0/758579398 shutdown_connections 2024-09-16T15:53:50.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:50.857+0000 7fb414c00640 1 -- 172.21.15.151:0/758579398 wait complete. 2024-09-16T15:53:50.876 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T15:53:50.877 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:53:51.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.032+0000 7ff832600640 1 -- 172.21.15.151:0/362144300 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff82c161430 msgr2=0x7ff82c0b5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:51.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.032+0000 7ff832600640 1 --2- 172.21.15.151:0/362144300 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff82c161430 0x7ff82c0b5540 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7ff82c08e090 tx=0x7ff81c02a420 comp rx=0 tx=0).stop 2024-09-16T15:53:51.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.033+0000 7ff832600640 1 -- 172.21.15.151:0/362144300 shutdown_connections 2024-09-16T15:53:51.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.033+0000 7ff832600640 1 --2- 172.21.15.151:0/362144300 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff82c0cef40 0x7ff82c0cf390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:51.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.033+0000 7ff832600640 1 --2- 172.21.15.151:0/362144300 >> v2:172.21.15.167:3300/0 conn(0x7ff82c0b4aa0 0x7ff82c0b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:51.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.033+0000 7ff832600640 1 --2- 172.21.15.151:0/362144300 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff82c161430 0x7ff82c0b5540 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:51.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.033+0000 7ff832600640 1 -- 172.21.15.151:0/362144300 >> 172.21.15.151:0/362144300 conn(0x7ff82c0d3dc0 msgr2=0x7ff82c0d41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:51.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.033+0000 7ff832600640 1 -- 172.21.15.151:0/362144300 shutdown_connections 2024-09-16T15:53:51.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.033+0000 7ff832600640 1 -- 172.21.15.151:0/362144300 wait complete. 2024-09-16T15:53:51.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.034+0000 7ff832600640 1 Processor -- start 2024-09-16T15:53:51.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.034+0000 7ff832600640 1 -- start start 2024-09-16T15:53:51.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.035+0000 7ff832600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff82c0b4aa0 0x7ff82c15fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:51.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.035+0000 7ff832600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff82c0cef40 0x7ff82c1f79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:51.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.035+0000 7ff832600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff82c161430 0x7ff82c1f7f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:51.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.035+0000 7ff832600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff82c0cfc60 con 0x7ff82c0b4aa0 2024-09-16T15:53:51.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.035+0000 7ff832600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff82c0cfae0 con 0x7ff82c161430 2024-09-16T15:53:51.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.035+0000 7ff832600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7ff82c0cfde0 con 0x7ff82c0cef40 2024-09-16T15:53:51.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.036+0000 7ff831200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff82c0b4aa0 0x7ff82c15fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:51.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.036+0000 7ff831200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff82c0b4aa0 0x7ff82c15fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43750/0 (socket says 172.21.15.151:43750) 2024-09-16T15:53:51.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.036+0000 7ff831200640 1 -- 172.21.15.151:0/3858646965 learned_addr learned my addr 172.21.15.151:0/3858646965 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:53:51.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.036+0000 7ff82be00640 1 --2- 172.21.15.151:0/3858646965 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff82c0cef40 0x7ff82c1f79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:51.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.036+0000 7ff831c00640 1 --2- 172.21.15.151:0/3858646965 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff82c161430 0x7ff82c1f7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:51.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.036+0000 7ff831200640 1 -- 172.21.15.151:0/3858646965 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff82c0cef40 msgr2=0x7ff82c1f79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:51.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.036+0000 7ff831200640 1 --2- 172.21.15.151:0/3858646965 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff82c0cef40 0x7ff82c1f79c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:51.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.036+0000 7ff831200640 1 -- 172.21.15.151:0/3858646965 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff82c161430 msgr2=0x7ff82c1f7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:51.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.036+0000 7ff831200640 1 --2- 172.21.15.151:0/3858646965 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff82c161430 0x7ff82c1f7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:51.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.036+0000 7ff831200640 1 -- 172.21.15.151:0/3858646965 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff82c0b2860 con 0x7ff82c0b4aa0 2024-09-16T15:53:51.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.036+0000 7ff831c00640 1 --2- 172.21.15.151:0/3858646965 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff82c161430 0x7ff82c1f7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:53:51.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.036+0000 7ff831200640 1 --2- 172.21.15.151:0/3858646965 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff82c0b4aa0 0x7ff82c15fa60 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7ff81c034040 tx=0x7ff81c0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:51.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.036+0000 7ff828c00640 1 -- 172.21.15.151:0/3858646965 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81c03e030 con 0x7ff82c0b4aa0 2024-09-16T15:53:51.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.037+0000 7ff828c00640 1 -- 172.21.15.151:0/3858646965 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff81c003ca0 con 0x7ff82c0b4aa0 2024-09-16T15:53:51.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.037+0000 7ff832600640 1 -- 172.21.15.151:0/3858646965 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff82c0b2b40 con 0x7ff82c0b4aa0 2024-09-16T15:53:51.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.037+0000 7ff828c00640 1 -- 172.21.15.151:0/3858646965 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81c035040 con 0x7ff82c0b4aa0 2024-09-16T15:53:51.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.037+0000 7ff832600640 1 -- 172.21.15.151:0/3858646965 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff82c0b3100 con 0x7ff82c0b4aa0 2024-09-16T15:53:51.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.038+0000 7ff832600640 1 -- 172.21.15.151:0/3858646965 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff82c0cf390 con 0x7ff82c0b4aa0 2024-09-16T15:53:51.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.039+0000 7ff828c00640 1 -- 172.21.15.151:0/3858646965 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 47) ==== 104092+0+0 (secure 0 0 0) 0x7ff81c041030 con 0x7ff82c0b4aa0 2024-09-16T15:53:51.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.040+0000 7ff828c00640 1 --2- 172.21.15.151:0/3858646965 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff80c07a740 0x7ff80c07cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:51.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.041+0000 7ff828c00640 1 -- 172.21.15.151:0/3858646965 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5821+0+0 (secure 0 0 0) 0x7ff81c0b9ef0 con 0x7ff82c0b4aa0 2024-09-16T15:53:51.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.041+0000 7ff82be00640 1 --2- 172.21.15.151:0/3858646965 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff80c07a740 0x7ff80c07cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:51.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.042+0000 7ff82be00640 1 --2- 172.21.15.151:0/3858646965 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff80c07a740 0x7ff80c07cc00 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7ff818009b50 tx=0x7ff818005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:51.047 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.045+0000 7ff828c00640 1 -- 172.21.15.151:0/3858646965 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff81c083970 con 0x7ff82c0b4aa0 2024-09-16T15:53:51.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.228+0000 7ff832600640 1 -- 172.21.15.151:0/3858646965 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff82c0b3340 con 0x7ff80c07a740 2024-09-16T15:53:51.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.234+0000 7ff828c00640 1 -- 172.21.15.151:0/3858646965 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247003 (secure 0 0 0) 0x7ff82c0b3340 con 0x7ff80c07a740 2024-09-16T15:53:51.239 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:53:51.240 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-09-16T15:53:51.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.238+0000 7ff832600640 1 -- 172.21.15.151:0/3858646965 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff80c07a740 msgr2=0x7ff80c07cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:51.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.238+0000 7ff832600640 1 --2- 172.21.15.151:0/3858646965 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff80c07a740 0x7ff80c07cc00 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7ff818009b50 tx=0x7ff818005f90 comp rx=0 tx=0).stop 2024-09-16T15:53:51.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.238+0000 7ff832600640 1 -- 172.21.15.151:0/3858646965 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff82c0b4aa0 msgr2=0x7ff82c15fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:51.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.238+0000 7ff832600640 1 --2- 172.21.15.151:0/3858646965 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff82c0b4aa0 0x7ff82c15fa60 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7ff81c034040 tx=0x7ff81c0038a0 comp rx=0 tx=0).stop 2024-09-16T15:53:51.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.238+0000 7ff832600640 1 -- 172.21.15.151:0/3858646965 shutdown_connections 2024-09-16T15:53:51.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.238+0000 7ff832600640 1 --2- 172.21.15.151:0/3858646965 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff80c07a740 0x7ff80c07cc00 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:51.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.238+0000 7ff832600640 1 --2- 172.21.15.151:0/3858646965 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff82c161430 0x7ff82c1f7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:51.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.238+0000 7ff832600640 1 --2- 172.21.15.151:0/3858646965 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff82c0cef40 0x7ff82c1f79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:51.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.238+0000 7ff832600640 1 --2- 172.21.15.151:0/3858646965 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff82c0b4aa0 0x7ff82c15fa60 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:51.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.238+0000 7ff832600640 1 -- 172.21.15.151:0/3858646965 >> 172.21.15.151:0/3858646965 conn(0x7ff82c0d3dc0 msgr2=0x7ff82c0b9050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:51.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.238+0000 7ff832600640 1 -- 172.21.15.151:0/3858646965 shutdown_connections 2024-09-16T15:53:51.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.238+0000 7ff832600640 1 -- 172.21.15.151:0/3858646965 wait complete. 2024-09-16T15:53:51.257 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":566,"stamp":"2024-09-16T15:53:49.936763+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":182,"acting":182,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":28,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1077756,"kb_used_data":13828,"kb_used_omap":12,"kb_used_meta":1063731,"kb_avail":753896964,"statfs":{"total":773094113280,"available":771990491136,"internally_reserved":0,"allocated":14159872,"data_stored":6892291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12738,"internal_metadata":1089261118},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.916433"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.042290+0000","last_change":"2024-09-16T15:53:46.042290+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.042290+0000","last_undegraded":"2024-09-16T15:53:46.042290+0000","last_fullsized":"2024-09-16T15:53:46.042290+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.043416+0000","last_change":"2024-09-16T15:53:46.043416+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.043416+0000","last_undegraded":"2024-09-16T15:53:46.043416+0000","last_fullsized":"2024-09-16T15:53:46.043416+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.041857+0000","last_change":"2024-09-16T15:53:46.041857+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.041857+0000","last_undegraded":"2024-09-16T15:53:46.041857+0000","last_fullsized":"2024-09-16T15:53:46.041857+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.031217+0000","last_change":"2024-09-16T15:53:46.031217+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.031217+0000","last_undegraded":"2024-09-16T15:53:46.031217+0000","last_fullsized":"2024-09-16T15:53:46.031217+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.037204+0000","last_change":"2024-09-16T15:53:46.037204+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.037204+0000","last_undegraded":"2024-09-16T15:53:46.037204+0000","last_fullsized":"2024-09-16T15:53:46.037204+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.040603+0000","last_change":"2024-09-16T15:53:46.040603+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.040603+0000","last_undegraded":"2024-09-16T15:53:46.040603+0000","last_fullsized":"2024-09-16T15:53:46.040603+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.041279+0000","last_change":"2024-09-16T15:53:46.041279+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.041279+0000","last_undegraded":"2024-09-16T15:53:46.041279+0000","last_fullsized":"2024-09-16T15:53:46.041279+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.039521+0000","last_change":"2024-09-16T15:53:46.039521+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.039521+0000","last_undegraded":"2024-09-16T15:53:46.039521+0000","last_fullsized":"2024-09-16T15:53:46.039521+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.040481+0000","last_change":"2024-09-16T15:53:46.040481+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.040481+0000","last_undegraded":"2024-09-16T15:53:46.040481+0000","last_fullsized":"2024-09-16T15:53:46.040481+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.042888+0000","last_change":"2024-09-16T15:53:46.042888+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.042888+0000","last_undegraded":"2024-09-16T15:53:46.042888+0000","last_fullsized":"2024-09-16T15:53:46.042888+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.031743+0000","last_change":"2024-09-16T15:53:46.031743+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.031743+0000","last_undegraded":"2024-09-16T15:53:46.031743+0000","last_fullsized":"2024-09-16T15:53:46.031743+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.042568+0000","last_change":"2024-09-16T15:53:46.042568+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.042568+0000","last_undegraded":"2024-09-16T15:53:46.042568+0000","last_fullsized":"2024-09-16T15:53:46.042568+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.031210+0000","last_change":"2024-09-16T15:53:46.031210+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.031210+0000","last_undegraded":"2024-09-16T15:53:46.031210+0000","last_fullsized":"2024-09-16T15:53:46.031210+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":10,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-16T15:53:47.058871+0000","last_change":"2024-09-16T15:53:47.056102+0000","last_active":"2024-09-16T15:53:47.058871+0000","last_peered":"2024-09-16T15:53:47.058871+0000","last_clean":"2024-09-16T15:53:47.058871+0000","last_became_active":"2024-09-16T15:53:47.055786+0000","last_became_peered":"2024-09-16T15:53:47.055786+0000","last_unstale":"2024-09-16T15:53:47.058871+0000","last_undegraded":"2024-09-16T15:53:47.058871+0000","last_fullsized":"2024-09-16T15:53:47.058871+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:02:29.501341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:53:46.018368+0000","last_change":"2024-09-16T15:53:46.018368+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.018368+0000","last_undegraded":"2024-09-16T15:53:46.018368+0000","last_fullsized":"2024-09-16T15:53:46.018368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":10,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-16T15:53:47.063272+0000","last_change":"2024-09-16T15:53:47.060703+0000","last_active":"2024-09-16T15:53:47.063272+0000","last_peered":"2024-09-16T15:53:47.063272+0000","last_clean":"2024-09-16T15:53:47.063272+0000","last_became_active":"2024-09-16T15:53:47.060125+0000","last_became_peered":"2024-09-16T15:53:47.060125+0000","last_unstale":"2024-09-16T15:53:47.063272+0000","last_undegraded":"2024-09-16T15:53:47.063272+0000","last_fullsized":"2024-09-16T15:53:47.063272+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:29:02.714478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.039985+0000","last_change":"2024-09-16T15:53:46.039985+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.039985+0000","last_undegraded":"2024-09-16T15:53:46.039985+0000","last_fullsized":"2024-09-16T15:53:46.039985+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:53:46.018368+0000","last_change":"2024-09-16T15:53:46.018368+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.018368+0000","last_undegraded":"2024-09-16T15:53:46.018368+0000","last_fullsized":"2024-09-16T15:53:46.018368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.038804+0000","last_change":"2024-09-16T15:53:46.038804+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.038804+0000","last_undegraded":"2024-09-16T15:53:46.038804+0000","last_fullsized":"2024-09-16T15:53:46.038804+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:53:46.018368+0000","last_change":"2024-09-16T15:53:46.018368+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.018368+0000","last_undegraded":"2024-09-16T15:53:46.018368+0000","last_fullsized":"2024-09-16T15:53:46.018368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.038853+0000","last_change":"2024-09-16T15:53:46.038853+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.038853+0000","last_undegraded":"2024-09-16T15:53:46.038853+0000","last_fullsized":"2024-09-16T15:53:46.038853+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.040567+0000","last_change":"2024-09-16T15:53:46.040567+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.040567+0000","last_undegraded":"2024-09-16T15:53:46.040567+0000","last_fullsized":"2024-09-16T15:53:46.040567+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.037506+0000","last_change":"2024-09-16T15:53:46.037506+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.037506+0000","last_undegraded":"2024-09-16T15:53:46.037506+0000","last_fullsized":"2024-09-16T15:53:46.037506+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:53:46.018368+0000","last_change":"2024-09-16T15:53:46.018368+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.018368+0000","last_undegraded":"2024-09-16T15:53:46.018368+0000","last_fullsized":"2024-09-16T15:53:46.018368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.034184+0000","last_change":"2024-09-16T15:53:46.034184+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.034184+0000","last_undegraded":"2024-09-16T15:53:46.034184+0000","last_fullsized":"2024-09-16T15:53:46.034184+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.040330+0000","last_change":"2024-09-16T15:53:46.040330+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.040330+0000","last_undegraded":"2024-09-16T15:53:46.040330+0000","last_fullsized":"2024-09-16T15:53:46.040330+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":10,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-16T15:53:47.068210+0000","last_change":"2024-09-16T15:53:47.067044+0000","last_active":"2024-09-16T15:53:47.068210+0000","last_peered":"2024-09-16T15:53:47.068210+0000","last_clean":"2024-09-16T15:53:47.068210+0000","last_became_active":"2024-09-16T15:53:47.066772+0000","last_became_peered":"2024-09-16T15:53:47.066772+0000","last_unstale":"2024-09-16T15:53:47.068210+0000","last_undegraded":"2024-09-16T15:53:47.068210+0000","last_fullsized":"2024-09-16T15:53:47.068210+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:27:45.682352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.036344+0000","last_change":"2024-09-16T15:53:46.036344+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.036344+0000","last_undegraded":"2024-09-16T15:53:46.036344+0000","last_fullsized":"2024-09-16T15:53:46.036344+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.041980+0000","last_change":"2024-09-16T15:53:46.041980+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.041980+0000","last_undegraded":"2024-09-16T15:53:46.041980+0000","last_fullsized":"2024-09-16T15:53:46.041980+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.038900+0000","last_change":"2024-09-16T15:53:46.038900+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.038900+0000","last_undegraded":"2024-09-16T15:53:46.038900+0000","last_fullsized":"2024-09-16T15:53:46.038900+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.035309+0000","last_change":"2024-09-16T15:53:46.035309+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.035309+0000","last_undegraded":"2024-09-16T15:53:46.035309+0000","last_fullsized":"2024-09-16T15:53:46.035309+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.046495+0000","last_change":"2024-09-16T15:53:46.046495+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.046495+0000","last_undegraded":"2024-09-16T15:53:46.046495+0000","last_fullsized":"2024-09-16T15:53:46.046495+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":10,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-16T15:53:47.062974+0000","last_change":"2024-09-16T15:53:47.056337+0000","last_active":"2024-09-16T15:53:47.062974+0000","last_peered":"2024-09-16T15:53:47.062974+0000","last_clean":"2024-09-16T15:53:47.062974+0000","last_became_active":"2024-09-16T15:53:47.056039+0000","last_became_peered":"2024-09-16T15:53:47.056039+0000","last_unstale":"2024-09-16T15:53:47.062974+0000","last_undegraded":"2024-09-16T15:53:47.062974+0000","last_fullsized":"2024-09-16T15:53:47.062974+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:12:43.131784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T15:53:46.059217+0000","last_change":"2024-09-16T15:53:46.049579+0000","last_active":"2024-09-16T15:53:46.059217+0000","last_peered":"2024-09-16T15:53:46.059217+0000","last_clean":"2024-09-16T15:53:46.059217+0000","last_became_active":"2024-09-16T15:53:46.049306+0000","last_became_peered":"2024-09-16T15:53:46.049306+0000","last_unstale":"2024-09-16T15:53:46.059217+0000","last_undegraded":"2024-09-16T15:53:46.059217+0000","last_fullsized":"2024-09-16T15:53:46.059217+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:52:56.403013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.037825+0000","last_change":"2024-09-16T15:53:46.037825+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.037825+0000","last_undegraded":"2024-09-16T15:53:46.037825+0000","last_fullsized":"2024-09-16T15:53:46.037825+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T15:53:46.059575+0000","last_change":"2024-09-16T15:53:46.055817+0000","last_active":"2024-09-16T15:53:46.059575+0000","last_peered":"2024-09-16T15:53:46.059575+0000","last_clean":"2024-09-16T15:53:46.059575+0000","last_became_active":"2024-09-16T15:53:46.055540+0000","last_became_peered":"2024-09-16T15:53:46.055540+0000","last_unstale":"2024-09-16T15:53:46.059575+0000","last_undegraded":"2024-09-16T15:53:46.059575+0000","last_fullsized":"2024-09-16T15:53:46.059575+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:11:20.894617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.036684+0000","last_change":"2024-09-16T15:53:46.036684+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.036684+0000","last_undegraded":"2024-09-16T15:53:46.036684+0000","last_fullsized":"2024-09-16T15:53:46.036684+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T15:53:46.045935+0000","last_change":"2024-09-16T15:53:46.043053+0000","last_active":"2024-09-16T15:53:46.045935+0000","last_peered":"2024-09-16T15:53:46.045935+0000","last_clean":"2024-09-16T15:53:46.045935+0000","last_became_active":"2024-09-16T15:53:46.042758+0000","last_became_peered":"2024-09-16T15:53:46.042758+0000","last_unstale":"2024-09-16T15:53:46.045935+0000","last_undegraded":"2024-09-16T15:53:46.045935+0000","last_fullsized":"2024-09-16T15:53:46.045935+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:42:47.639276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.038140+0000","last_change":"2024-09-16T15:53:46.038140+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.038140+0000","last_undegraded":"2024-09-16T15:53:46.038140+0000","last_fullsized":"2024-09-16T15:53:46.038140+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T15:53:46.050362+0000","last_change":"2024-09-16T15:53:46.046499+0000","last_active":"2024-09-16T15:53:46.050362+0000","last_peered":"2024-09-16T15:53:46.050362+0000","last_clean":"2024-09-16T15:53:46.050362+0000","last_became_active":"2024-09-16T15:53:46.046246+0000","last_became_peered":"2024-09-16T15:53:46.046246+0000","last_unstale":"2024-09-16T15:53:46.050362+0000","last_undegraded":"2024-09-16T15:53:46.050362+0000","last_fullsized":"2024-09-16T15:53:46.050362+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:50.321365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:53:46.018368+0000","last_change":"2024-09-16T15:53:46.018368+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.018368+0000","last_undegraded":"2024-09-16T15:53:46.018368+0000","last_fullsized":"2024-09-16T15:53:46.018368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T15:53:46.052654+0000","last_change":"2024-09-16T15:53:46.048816+0000","last_active":"2024-09-16T15:53:46.052654+0000","last_peered":"2024-09-16T15:53:46.052654+0000","last_clean":"2024-09-16T15:53:46.052654+0000","last_became_active":"2024-09-16T15:53:46.048416+0000","last_became_peered":"2024-09-16T15:53:46.048416+0000","last_unstale":"2024-09-16T15:53:46.052654+0000","last_undegraded":"2024-09-16T15:53:46.052654+0000","last_fullsized":"2024-09-16T15:53:46.052654+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:50:02.529556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.038462+0000","last_change":"2024-09-16T15:53:46.038462+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.038462+0000","last_undegraded":"2024-09-16T15:53:46.038462+0000","last_fullsized":"2024-09-16T15:53:46.038462+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-16T15:53:45.025483+0000","last_change":"2024-09-16T15:53:45.025483+0000","last_active":"2024-09-16T15:53:45.010151+0000","last_peered":"2024-09-16T15:53:45.010151+0000","last_clean":"2024-09-16T15:53:45.010151+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:45.025483+0000","last_undegraded":"2024-09-16T15:53:45.025483+0000","last_fullsized":"2024-09-16T15:53:45.025483+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.039318+0000","last_change":"2024-09-16T15:53:46.039318+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.039318+0000","last_undegraded":"2024-09-16T15:53:46.039318+0000","last_fullsized":"2024-09-16T15:53:46.039318+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T15:53:46.059072+0000","last_change":"2024-09-16T15:53:46.052810+0000","last_active":"2024-09-16T15:53:46.059072+0000","last_peered":"2024-09-16T15:53:46.059072+0000","last_clean":"2024-09-16T15:53:46.059072+0000","last_became_active":"2024-09-16T15:53:46.052461+0000","last_became_peered":"2024-09-16T15:53:46.052461+0000","last_unstale":"2024-09-16T15:53:46.059072+0000","last_undegraded":"2024-09-16T15:53:46.059072+0000","last_fullsized":"2024-09-16T15:53:46.059072+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:20:19.397356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.037477+0000","last_change":"2024-09-16T15:53:46.037477+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.037477+0000","last_undegraded":"2024-09-16T15:53:46.037477+0000","last_fullsized":"2024-09-16T15:53:46.037477+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T15:53:46.052385+0000","last_change":"2024-09-16T15:53:46.050598+0000","last_active":"2024-09-16T15:53:46.052385+0000","last_peered":"2024-09-16T15:53:46.052385+0000","last_clean":"2024-09-16T15:53:46.052385+0000","last_became_active":"2024-09-16T15:53:46.050355+0000","last_became_peered":"2024-09-16T15:53:46.050355+0000","last_unstale":"2024-09-16T15:53:46.052385+0000","last_undegraded":"2024-09-16T15:53:46.052385+0000","last_fullsized":"2024-09-16T15:53:46.052385+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:28:46.376887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.039821+0000","last_change":"2024-09-16T15:53:46.039821+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.039821+0000","last_undegraded":"2024-09-16T15:53:46.039821+0000","last_fullsized":"2024-09-16T15:53:46.039821+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T15:53:46.054603+0000","last_change":"2024-09-16T15:53:46.041207+0000","last_active":"2024-09-16T15:53:46.054603+0000","last_peered":"2024-09-16T15:53:46.054603+0000","last_clean":"2024-09-16T15:53:46.054603+0000","last_became_active":"2024-09-16T15:53:46.040942+0000","last_became_peered":"2024-09-16T15:53:46.040942+0000","last_unstale":"2024-09-16T15:53:46.054603+0000","last_undegraded":"2024-09-16T15:53:46.054603+0000","last_fullsized":"2024-09-16T15:53:46.054603+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:06:50.417480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.042730+0000","last_change":"2024-09-16T15:53:46.042730+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.042730+0000","last_undegraded":"2024-09-16T15:53:46.042730+0000","last_fullsized":"2024-09-16T15:53:46.042730+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-16T15:53:45.024968+0000","last_change":"2024-09-16T15:53:45.024968+0000","last_active":"2024-09-16T15:53:45.010151+0000","last_peered":"2024-09-16T15:53:45.010151+0000","last_clean":"2024-09-16T15:53:45.010151+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:45.024968+0000","last_undegraded":"2024-09-16T15:53:45.024968+0000","last_fullsized":"2024-09-16T15:53:45.024968+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.038076+0000","last_change":"2024-09-16T15:53:46.038076+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.038076+0000","last_undegraded":"2024-09-16T15:53:46.038076+0000","last_fullsized":"2024-09-16T15:53:46.038076+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T15:53:46.059281+0000","last_change":"2024-09-16T15:53:46.051421+0000","last_active":"2024-09-16T15:53:46.059281+0000","last_peered":"2024-09-16T15:53:46.059281+0000","last_clean":"2024-09-16T15:53:46.059281+0000","last_became_active":"2024-09-16T15:53:46.049074+0000","last_became_peered":"2024-09-16T15:53:46.049074+0000","last_unstale":"2024-09-16T15:53:46.059281+0000","last_undegraded":"2024-09-16T15:53:46.059281+0000","last_fullsized":"2024-09-16T15:53:46.059281+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:17:54.102543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.038820+0000","last_change":"2024-09-16T15:53:46.038820+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.038820+0000","last_undegraded":"2024-09-16T15:53:46.038820+0000","last_fullsized":"2024-09-16T15:53:46.038820+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T15:53:46.056488+0000","last_change":"2024-09-16T15:53:46.055615+0000","last_active":"2024-09-16T15:53:46.056488+0000","last_peered":"2024-09-16T15:53:46.056488+0000","last_clean":"2024-09-16T15:53:46.056488+0000","last_became_active":"2024-09-16T15:53:46.055155+0000","last_became_peered":"2024-09-16T15:53:46.055155+0000","last_unstale":"2024-09-16T15:53:46.056488+0000","last_undegraded":"2024-09-16T15:53:46.056488+0000","last_fullsized":"2024-09-16T15:53:46.056488+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:54:56.815280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":10,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-16T15:53:47.059010+0000","last_change":"2024-09-16T15:53:47.055028+0000","last_active":"2024-09-16T15:53:47.059010+0000","last_peered":"2024-09-16T15:53:47.059010+0000","last_clean":"2024-09-16T15:53:47.059010+0000","last_became_active":"2024-09-16T15:53:47.054638+0000","last_became_peered":"2024-09-16T15:53:47.054638+0000","last_unstale":"2024-09-16T15:53:47.059010+0000","last_undegraded":"2024-09-16T15:53:47.059010+0000","last_fullsized":"2024-09-16T15:53:47.059010+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:49:07.189417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-16T15:53:47.042905+0000","last_change":"2024-09-16T15:53:46.053325+0000","last_active":"2024-09-16T15:53:47.042905+0000","last_peered":"2024-09-16T15:53:47.042905+0000","last_clean":"2024-09-16T15:53:47.042905+0000","last_became_active":"2024-09-16T15:53:46.051989+0000","last_became_peered":"2024-09-16T15:53:46.051989+0000","last_unstale":"2024-09-16T15:53:47.042905+0000","last_undegraded":"2024-09-16T15:53:47.042905+0000","last_fullsized":"2024-09-16T15:53:47.042905+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:10:52.279943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":10,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-16T15:53:47.063798+0000","last_change":"2024-09-16T15:53:47.062301+0000","last_active":"2024-09-16T15:53:47.063798+0000","last_peered":"2024-09-16T15:53:47.063798+0000","last_clean":"2024-09-16T15:53:47.063798+0000","last_became_active":"2024-09-16T15:53:47.058392+0000","last_became_peered":"2024-09-16T15:53:47.058392+0000","last_unstale":"2024-09-16T15:53:47.063798+0000","last_undegraded":"2024-09-16T15:53:47.063798+0000","last_fullsized":"2024-09-16T15:53:47.063798+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:35:38.397852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T15:53:46.058498+0000","last_change":"2024-09-16T15:53:46.055507+0000","last_active":"2024-09-16T15:53:46.058498+0000","last_peered":"2024-09-16T15:53:46.058498+0000","last_clean":"2024-09-16T15:53:46.058498+0000","last_became_active":"2024-09-16T15:53:46.055247+0000","last_became_peered":"2024-09-16T15:53:46.055247+0000","last_unstale":"2024-09-16T15:53:46.058498+0000","last_undegraded":"2024-09-16T15:53:46.058498+0000","last_fullsized":"2024-09-16T15:53:46.058498+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:07:18.321225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.037392+0000","last_change":"2024-09-16T15:53:46.037392+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.037392+0000","last_undegraded":"2024-09-16T15:53:46.037392+0000","last_fullsized":"2024-09-16T15:53:46.037392+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T15:53:46.058960+0000","last_change":"2024-09-16T15:53:46.054698+0000","last_active":"2024-09-16T15:53:46.058960+0000","last_peered":"2024-09-16T15:53:46.058960+0000","last_clean":"2024-09-16T15:53:46.058960+0000","last_became_active":"2024-09-16T15:53:46.050527+0000","last_became_peered":"2024-09-16T15:53:46.050527+0000","last_unstale":"2024-09-16T15:53:46.058960+0000","last_undegraded":"2024-09-16T15:53:46.058960+0000","last_fullsized":"2024-09-16T15:53:46.058960+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:24:23.869366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.042645+0000","last_change":"2024-09-16T15:53:46.042645+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.042645+0000","last_undegraded":"2024-09-16T15:53:46.042645+0000","last_fullsized":"2024-09-16T15:53:46.042645+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T15:53:46.054629+0000","last_change":"2024-09-16T15:53:46.041954+0000","last_active":"2024-09-16T15:53:46.054629+0000","last_peered":"2024-09-16T15:53:46.054629+0000","last_clean":"2024-09-16T15:53:46.054629+0000","last_became_active":"2024-09-16T15:53:46.041272+0000","last_became_peered":"2024-09-16T15:53:46.041272+0000","last_unstale":"2024-09-16T15:53:46.054629+0000","last_undegraded":"2024-09-16T15:53:46.054629+0000","last_fullsized":"2024-09-16T15:53:46.054629+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:20:31.172329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.041139+0000","last_change":"2024-09-16T15:53:46.041139+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.041139+0000","last_undegraded":"2024-09-16T15:53:46.041139+0000","last_fullsized":"2024-09-16T15:53:46.041139+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T15:53:46.049906+0000","last_change":"2024-09-16T15:53:46.048066+0000","last_active":"2024-09-16T15:53:46.049906+0000","last_peered":"2024-09-16T15:53:46.049906+0000","last_clean":"2024-09-16T15:53:46.049906+0000","last_became_active":"2024-09-16T15:53:46.047765+0000","last_became_peered":"2024-09-16T15:53:46.047765+0000","last_unstale":"2024-09-16T15:53:46.049906+0000","last_undegraded":"2024-09-16T15:53:46.049906+0000","last_fullsized":"2024-09-16T15:53:46.049906+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:33:52.388798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.036912+0000","last_change":"2024-09-16T15:53:46.036912+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.036912+0000","last_undegraded":"2024-09-16T15:53:46.036912+0000","last_fullsized":"2024-09-16T15:53:46.036912+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T15:53:46.054123+0000","last_change":"2024-09-16T15:53:46.050413+0000","last_active":"2024-09-16T15:53:46.054123+0000","last_peered":"2024-09-16T15:53:46.054123+0000","last_clean":"2024-09-16T15:53:46.054123+0000","last_became_active":"2024-09-16T15:53:46.050136+0000","last_became_peered":"2024-09-16T15:53:46.050136+0000","last_unstale":"2024-09-16T15:53:46.054123+0000","last_undegraded":"2024-09-16T15:53:46.054123+0000","last_fullsized":"2024-09-16T15:53:46.054123+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:55:06.155424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.038394+0000","last_change":"2024-09-16T15:53:46.038394+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.038394+0000","last_undegraded":"2024-09-16T15:53:46.038394+0000","last_fullsized":"2024-09-16T15:53:46.038394+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T15:53:46.059203+0000","last_change":"2024-09-16T15:53:46.052237+0000","last_active":"2024-09-16T15:53:46.059203+0000","last_peered":"2024-09-16T15:53:46.059203+0000","last_clean":"2024-09-16T15:53:46.059203+0000","last_became_active":"2024-09-16T15:53:46.051939+0000","last_became_peered":"2024-09-16T15:53:46.051939+0000","last_unstale":"2024-09-16T15:53:46.059203+0000","last_undegraded":"2024-09-16T15:53:46.059203+0000","last_fullsized":"2024-09-16T15:53:46.059203+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:52:12.659520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.036480+0000","last_change":"2024-09-16T15:53:46.036480+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.036480+0000","last_undegraded":"2024-09-16T15:53:46.036480+0000","last_fullsized":"2024-09-16T15:53:46.036480+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T15:53:46.051376+0000","last_change":"2024-09-16T15:53:46.049709+0000","last_active":"2024-09-16T15:53:46.051376+0000","last_peered":"2024-09-16T15:53:46.051376+0000","last_clean":"2024-09-16T15:53:46.051376+0000","last_became_active":"2024-09-16T15:53:46.049412+0000","last_became_peered":"2024-09-16T15:53:46.049412+0000","last_unstale":"2024-09-16T15:53:46.051376+0000","last_undegraded":"2024-09-16T15:53:46.051376+0000","last_fullsized":"2024-09-16T15:53:46.051376+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:54:26.788931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.037329+0000","last_change":"2024-09-16T15:53:46.037329+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.037329+0000","last_undegraded":"2024-09-16T15:53:46.037329+0000","last_fullsized":"2024-09-16T15:53:46.037329+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-16T15:53:45.025032+0000","last_change":"2024-09-16T15:53:45.025032+0000","last_active":"2024-09-16T15:53:45.010151+0000","last_peered":"2024-09-16T15:53:45.010151+0000","last_clean":"2024-09-16T15:53:45.010151+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:45.025032+0000","last_undegraded":"2024-09-16T15:53:45.025032+0000","last_fullsized":"2024-09-16T15:53:45.025032+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":10,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-16T15:53:47.063249+0000","last_change":"2024-09-16T15:53:47.058656+0000","last_active":"2024-09-16T15:53:47.063249+0000","last_peered":"2024-09-16T15:53:47.063249+0000","last_clean":"2024-09-16T15:53:47.063249+0000","last_became_active":"2024-09-16T15:53:47.057260+0000","last_became_peered":"2024-09-16T15:53:47.057260+0000","last_unstale":"2024-09-16T15:53:47.063249+0000","last_undegraded":"2024-09-16T15:53:47.063249+0000","last_fullsized":"2024-09-16T15:53:47.063249+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:18:36.519474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-16T15:53:47.041902+0000","last_change":"2024-09-16T15:53:46.038714+0000","last_active":"2024-09-16T15:53:47.041902+0000","last_peered":"2024-09-16T15:53:47.041902+0000","last_clean":"2024-09-16T15:53:47.041902+0000","last_became_active":"2024-09-16T15:53:46.038411+0000","last_became_peered":"2024-09-16T15:53:46.038411+0000","last_unstale":"2024-09-16T15:53:47.041902+0000","last_undegraded":"2024-09-16T15:53:47.041902+0000","last_fullsized":"2024-09-16T15:53:47.041902+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:43:20.313472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.035844+0000","last_change":"2024-09-16T15:53:46.035844+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.035844+0000","last_undegraded":"2024-09-16T15:53:46.035844+0000","last_fullsized":"2024-09-16T15:53:46.035844+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T15:53:46.054391+0000","last_change":"2024-09-16T15:53:46.049975+0000","last_active":"2024-09-16T15:53:46.054391+0000","last_peered":"2024-09-16T15:53:46.054391+0000","last_clean":"2024-09-16T15:53:46.054391+0000","last_became_active":"2024-09-16T15:53:46.049667+0000","last_became_peered":"2024-09-16T15:53:46.049667+0000","last_unstale":"2024-09-16T15:53:46.054391+0000","last_undegraded":"2024-09-16T15:53:46.054391+0000","last_fullsized":"2024-09-16T15:53:46.054391+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:45:49.295347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.040296+0000","last_change":"2024-09-16T15:53:46.040296+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.040296+0000","last_undegraded":"2024-09-16T15:53:46.040296+0000","last_fullsized":"2024-09-16T15:53:46.040296+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-16T15:53:45.025087+0000","last_change":"2024-09-16T15:53:45.025087+0000","last_active":"2024-09-16T15:53:45.010151+0000","last_peered":"2024-09-16T15:53:45.010151+0000","last_clean":"2024-09-16T15:53:45.010151+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:45.025087+0000","last_undegraded":"2024-09-16T15:53:45.025087+0000","last_fullsized":"2024-09-16T15:53:45.025087+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.040819+0000","last_change":"2024-09-16T15:53:46.040819+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.040819+0000","last_undegraded":"2024-09-16T15:53:46.040819+0000","last_fullsized":"2024-09-16T15:53:46.040819+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T15:53:46.059389+0000","last_change":"2024-09-16T15:53:46.055909+0000","last_active":"2024-09-16T15:53:46.059389+0000","last_peered":"2024-09-16T15:53:46.059389+0000","last_clean":"2024-09-16T15:53:46.059389+0000","last_became_active":"2024-09-16T15:53:46.055567+0000","last_became_peered":"2024-09-16T15:53:46.055567+0000","last_unstale":"2024-09-16T15:53:46.059389+0000","last_undegraded":"2024-09-16T15:53:46.059389+0000","last_fullsized":"2024-09-16T15:53:46.059389+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:42:42.792655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":10,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-16T15:53:47.058902+0000","last_change":"2024-09-16T15:53:47.055081+0000","last_active":"2024-09-16T15:53:47.058902+0000","last_peered":"2024-09-16T15:53:47.058902+0000","last_clean":"2024-09-16T15:53:47.058902+0000","last_became_active":"2024-09-16T15:53:47.054772+0000","last_became_peered":"2024-09-16T15:53:47.054772+0000","last_unstale":"2024-09-16T15:53:47.058902+0000","last_undegraded":"2024-09-16T15:53:47.058902+0000","last_fullsized":"2024-09-16T15:53:47.058902+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:36:50.843181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-16T15:53:45.024985+0000","last_change":"2024-09-16T15:53:45.024985+0000","last_active":"2024-09-16T15:53:45.010151+0000","last_peered":"2024-09-16T15:53:45.010151+0000","last_clean":"2024-09-16T15:53:45.010151+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:45.024985+0000","last_undegraded":"2024-09-16T15:53:45.024985+0000","last_fullsized":"2024-09-16T15:53:45.024985+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.036181+0000","last_change":"2024-09-16T15:53:46.036181+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.036181+0000","last_undegraded":"2024-09-16T15:53:46.036181+0000","last_fullsized":"2024-09-16T15:53:46.036181+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-16T15:53:47.042871+0000","last_change":"2024-09-16T15:53:46.057982+0000","last_active":"2024-09-16T15:53:47.042871+0000","last_peered":"2024-09-16T15:53:47.042871+0000","last_clean":"2024-09-16T15:53:47.042871+0000","last_became_active":"2024-09-16T15:53:46.057707+0000","last_became_peered":"2024-09-16T15:53:46.057707+0000","last_unstale":"2024-09-16T15:53:47.042871+0000","last_undegraded":"2024-09-16T15:53:47.042871+0000","last_fullsized":"2024-09-16T15:53:47.042871+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:59:40.544624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.034301+0000","last_change":"2024-09-16T15:53:46.034301+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.034301+0000","last_undegraded":"2024-09-16T15:53:46.034301+0000","last_fullsized":"2024-09-16T15:53:46.034301+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T15:53:46.055777+0000","last_change":"2024-09-16T15:53:46.052804+0000","last_active":"2024-09-16T15:53:46.055777+0000","last_peered":"2024-09-16T15:53:46.055777+0000","last_clean":"2024-09-16T15:53:46.055777+0000","last_became_active":"2024-09-16T15:53:46.052460+0000","last_became_peered":"2024-09-16T15:53:46.052460+0000","last_unstale":"2024-09-16T15:53:46.055777+0000","last_undegraded":"2024-09-16T15:53:46.055777+0000","last_fullsized":"2024-09-16T15:53:46.055777+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:55:25.120515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.035212+0000","last_change":"2024-09-16T15:53:46.035212+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.035212+0000","last_undegraded":"2024-09-16T15:53:46.035212+0000","last_fullsized":"2024-09-16T15:53:46.035212+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T15:53:46.054865+0000","last_change":"2024-09-16T15:53:46.044167+0000","last_active":"2024-09-16T15:53:46.054865+0000","last_peered":"2024-09-16T15:53:46.054865+0000","last_clean":"2024-09-16T15:53:46.054865+0000","last_became_active":"2024-09-16T15:53:46.043914+0000","last_became_peered":"2024-09-16T15:53:46.043914+0000","last_unstale":"2024-09-16T15:53:46.054865+0000","last_undegraded":"2024-09-16T15:53:46.054865+0000","last_fullsized":"2024-09-16T15:53:46.054865+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:17:08.359001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:53:46.018368+0000","last_change":"2024-09-16T15:53:46.018368+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.018368+0000","last_undegraded":"2024-09-16T15:53:46.018368+0000","last_fullsized":"2024-09-16T15:53:46.018368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T15:53:46.058473+0000","last_change":"2024-09-16T15:53:46.055377+0000","last_active":"2024-09-16T15:53:46.058473+0000","last_peered":"2024-09-16T15:53:46.058473+0000","last_clean":"2024-09-16T15:53:46.058473+0000","last_became_active":"2024-09-16T15:53:46.055135+0000","last_became_peered":"2024-09-16T15:53:46.055135+0000","last_unstale":"2024-09-16T15:53:46.058473+0000","last_undegraded":"2024-09-16T15:53:46.058473+0000","last_fullsized":"2024-09-16T15:53:46.058473+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:30:58.937449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":10,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-16T15:53:47.062177+0000","last_change":"2024-09-16T15:53:47.057817+0000","last_active":"2024-09-16T15:53:47.062177+0000","last_peered":"2024-09-16T15:53:47.062177+0000","last_clean":"2024-09-16T15:53:47.062177+0000","last_became_active":"2024-09-16T15:53:47.057440+0000","last_became_peered":"2024-09-16T15:53:47.057440+0000","last_unstale":"2024-09-16T15:53:47.062177+0000","last_undegraded":"2024-09-16T15:53:47.062177+0000","last_fullsized":"2024-09-16T15:53:47.062177+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:56:27.484880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-16T15:53:47.043225+0000","last_change":"2024-09-16T15:53:46.047710+0000","last_active":"2024-09-16T15:53:47.043225+0000","last_peered":"2024-09-16T15:53:47.043225+0000","last_clean":"2024-09-16T15:53:47.043225+0000","last_became_active":"2024-09-16T15:53:46.047320+0000","last_became_peered":"2024-09-16T15:53:46.047320+0000","last_unstale":"2024-09-16T15:53:47.043225+0000","last_undegraded":"2024-09-16T15:53:47.043225+0000","last_fullsized":"2024-09-16T15:53:47.043225+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:40:37.494885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.036166+0000","last_change":"2024-09-16T15:53:46.036166+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.036166+0000","last_undegraded":"2024-09-16T15:53:46.036166+0000","last_fullsized":"2024-09-16T15:53:46.036166+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T15:53:46.057121+0000","last_change":"2024-09-16T15:53:46.048215+0000","last_active":"2024-09-16T15:53:46.057121+0000","last_peered":"2024-09-16T15:53:46.057121+0000","last_clean":"2024-09-16T15:53:46.057121+0000","last_became_active":"2024-09-16T15:53:46.045488+0000","last_became_peered":"2024-09-16T15:53:46.045488+0000","last_unstale":"2024-09-16T15:53:46.057121+0000","last_undegraded":"2024-09-16T15:53:46.057121+0000","last_fullsized":"2024-09-16T15:53:46.057121+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:53:11.860146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":319,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T15:53:46.040564+0000","last_change":"2024-09-16T15:39:12.698813+0000","last_active":"2024-09-16T15:53:46.040564+0000","last_peered":"2024-09-16T15:53:46.040564+0000","last_clean":"2024-09-16T15:53:46.040564+0000","last_became_active":"2024-09-16T15:39:12.697695+0000","last_became_peered":"2024-09-16T15:39:12.697695+0000","last_unstale":"2024-09-16T15:53:46.040564+0000","last_undegraded":"2024-09-16T15:53:46.040564+0000","last_fullsized":"2024-09-16T15:53:46.040564+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_clean_scrub_stamp":"2024-09-16T15:39:11.684967+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:02.061081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":116,"acting":116,"num_store_stats":7},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509620,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":159068,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":157438,"kb_avail":94212772,"statfs":{"total":96636764160,"available":96473878528,"internally_reserved":0,"allocated":1638400,"data_stored":723164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":161216968},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509619,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":119376,"kb_used_data":2168,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94252464,"statfs":{"total":96636764160,"available":96514523136,"internally_reserved":0,"allocated":2220032,"data_stored":1306645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":119994823},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509618,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":136828,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94235012,"statfs":{"total":96636764160,"available":96496652288,"internally_reserved":0,"allocated":1613824,"data_stored":709890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":138475973},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509618,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":118780,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94253060,"statfs":{"total":96636764160,"available":96515133440,"internally_reserved":0,"allocated":1613824,"data_stored":709890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":119994825},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509618,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131276,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240564,"statfs":{"total":96636764160,"available":96502337536,"internally_reserved":0,"allocated":1626112,"data_stored":716277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509619,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131856,"kb_used_data":2168,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94239984,"statfs":{"total":96636764160,"available":96501743616,"internally_reserved":0,"allocated":2220032,"data_stored":1306645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509617,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149308,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222532,"statfs":{"total":96636764160,"available":96483872768,"internally_reserved":0,"allocated":1613824,"data_stored":709890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":151255495},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542324,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131264,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240576,"statfs":{"total":96636764160,"available":96502349824,"internally_reserved":0,"allocated":1613824,"data_stored":709890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":132774348},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:53:51.261 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:53:51.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.453+0000 7f12fd800640 1 -- 172.21.15.151:0/866331053 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12dc0052e0 msgr2=0x7f12dc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:51.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.453+0000 7f12fd800640 1 --2- 172.21.15.151:0/866331053 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12dc0052e0 0x7f12dc00b9c0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f12f0005fd0 tx=0x7f12dc005830 comp rx=0 tx=0).stop 2024-09-16T15:53:51.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.454+0000 7f12fd800640 1 -- 172.21.15.151:0/866331053 shutdown_connections 2024-09-16T15:53:51.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.454+0000 7f12fd800640 1 --2- 172.21.15.151:0/866331053 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12dc0052e0 0x7f12dc00b9c0 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:51.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.454+0000 7f12fd800640 1 -- 172.21.15.151:0/866331053 >> 172.21.15.151:0/866331053 conn(0x7f12f80b1950 msgr2=0x7f12f80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:51.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.454+0000 7f12fd800640 1 -- 172.21.15.151:0/866331053 shutdown_connections 2024-09-16T15:53:51.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.454+0000 7f12fd800640 1 -- 172.21.15.151:0/866331053 wait complete. 2024-09-16T15:53:51.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.455+0000 7f12fd800640 1 Processor -- start 2024-09-16T15:53:51.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.455+0000 7f12fd800640 1 -- start start 2024-09-16T15:53:51.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.456+0000 7f12fd800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12dc0052e0 0x7f12f80d0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:51.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.456+0000 7f12fd800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f12f81557e0 0x7f12f81f8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:51.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.456+0000 7f12fd800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12f81f61d0 0x7f12f81f6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:51.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.456+0000 7f12fd800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f12f80ad0c0 con 0x7f12f81f61d0 2024-09-16T15:53:51.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.456+0000 7f12fd800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f12f80acf40 con 0x7f12dc0052e0 2024-09-16T15:53:51.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.456+0000 7f12fd800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f12f80ad240 con 0x7f12f81557e0 2024-09-16T15:53:51.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.457+0000 7f12f7400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12f81f61d0 0x7f12f81f6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:51.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.457+0000 7f12f7400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12f81f61d0 0x7f12f81f6650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43776/0 (socket says 172.21.15.151:43776) 2024-09-16T15:53:51.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.457+0000 7f12f7400640 1 -- 172.21.15.151:0/301485565 learned_addr learned my addr 172.21.15.151:0/301485565 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:53:51.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.457+0000 7f12f6a00640 1 --2- 172.21.15.151:0/301485565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12dc0052e0 0x7f12f80d0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:51.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.457+0000 7f12f6000640 1 --2- 172.21.15.151:0/301485565 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f12f81557e0 0x7f12f81f8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:51.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.457+0000 7f12f7400640 1 -- 172.21.15.151:0/301485565 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f12f81557e0 msgr2=0x7f12f81f8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:51.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.457+0000 7f12f7400640 1 --2- 172.21.15.151:0/301485565 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f12f81557e0 0x7f12f81f8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:51.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.457+0000 7f12f7400640 1 -- 172.21.15.151:0/301485565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12dc0052e0 msgr2=0x7f12f80d0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:51.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.457+0000 7f12f7400640 1 --2- 172.21.15.151:0/301485565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12dc0052e0 0x7f12f80d0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:51.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.457+0000 7f12f7400640 1 -- 172.21.15.151:0/301485565 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12f8155c60 con 0x7f12f81f61d0 2024-09-16T15:53:51.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.457+0000 7f12f6000640 1 --2- 172.21.15.151:0/301485565 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f12f81557e0 0x7f12f81f8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:53:51.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.457+0000 7f12f7400640 1 --2- 172.21.15.151:0/301485565 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12f81f61d0 0x7f12f81f6650 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f12f0009e20 tx=0x7f12f00042d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:51.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.458+0000 7f12eea00640 1 -- 172.21.15.151:0/301485565 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f0009200 con 0x7f12f81f61d0 2024-09-16T15:53:51.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.458+0000 7f12eea00640 1 -- 172.21.15.151:0/301485565 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f12f0043070 con 0x7f12f81f61d0 2024-09-16T15:53:51.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.458+0000 7f12eea00640 1 -- 172.21.15.151:0/301485565 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f0009ad0 con 0x7f12f81f61d0 2024-09-16T15:53:51.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.458+0000 7f12fd800640 1 -- 172.21.15.151:0/301485565 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12f81f5200 con 0x7f12f81f61d0 2024-09-16T15:53:51.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.458+0000 7f12fd800640 1 -- 172.21.15.151:0/301485565 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12f81f9050 con 0x7f12f81f61d0 2024-09-16T15:53:51.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.459+0000 7f12fd800640 1 -- 172.21.15.151:0/301485565 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12f81f53d0 con 0x7f12f81f61d0 2024-09-16T15:53:51.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.460+0000 7f12eea00640 1 -- 172.21.15.151:0/301485565 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 47) ==== 104092+0+0 (secure 0 0 0) 0x7f12f0048460 con 0x7f12f81f61d0 2024-09-16T15:53:51.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.461+0000 7f12eea00640 1 --2- 172.21.15.151:0/301485565 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f12d807a770 0x7f12d807cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:51.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.461+0000 7f12eea00640 1 -- 172.21.15.151:0/301485565 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5821+0+0 (secure 0 0 0) 0x7f12f00ba9d0 con 0x7f12f81f61d0 2024-09-16T15:53:51.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.461+0000 7f12f6a00640 1 --2- 172.21.15.151:0/301485565 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f12d807a770 0x7f12d807cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:51.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.462+0000 7f12f6a00640 1 --2- 172.21.15.151:0/301485565 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f12d807a770 0x7f12d807cc30 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f12f80cf920 tx=0x7f12e8006040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:51.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.465+0000 7f12eea00640 1 -- 172.21.15.151:0/301485565 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f12f0084aa0 con 0x7f12f81f61d0 2024-09-16T15:53:51.486 DEBUG:teuthology.orchestra.run.smithi151:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:53:51.490 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:53:51.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.654+0000 7f12fd800640 1 -- 172.21.15.151:0/301485565 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f12f8073fb0 con 0x7f12d807a770 2024-09-16T15:53:51.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.659+0000 7f12eea00640 1 -- 172.21.15.151:0/301485565 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247003 (secure 0 0 0) 0x7f12f8073fb0 con 0x7f12d807a770 2024-09-16T15:53:51.664 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:53:51.665 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-09-16T15:53:51.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.663+0000 7f12fd800640 1 -- 172.21.15.151:0/301485565 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f12d807a770 msgr2=0x7f12d807cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:51.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.663+0000 7f12fd800640 1 --2- 172.21.15.151:0/301485565 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f12d807a770 0x7f12d807cc30 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f12f80cf920 tx=0x7f12e8006040 comp rx=0 tx=0).stop 2024-09-16T15:53:51.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.663+0000 7f12fd800640 1 -- 172.21.15.151:0/301485565 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12f81f61d0 msgr2=0x7f12f81f6650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:51.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.663+0000 7f12fd800640 1 --2- 172.21.15.151:0/301485565 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12f81f61d0 0x7f12f81f6650 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f12f0009e20 tx=0x7f12f00042d0 comp rx=0 tx=0).stop 2024-09-16T15:53:51.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.663+0000 7f12fd800640 1 -- 172.21.15.151:0/301485565 shutdown_connections 2024-09-16T15:53:51.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.663+0000 7f12fd800640 1 --2- 172.21.15.151:0/301485565 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f12d807a770 0x7f12d807cc30 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:51.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.663+0000 7f12fd800640 1 --2- 172.21.15.151:0/301485565 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12f81f61d0 0x7f12f81f6650 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:51.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.663+0000 7f12fd800640 1 --2- 172.21.15.151:0/301485565 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f12f81557e0 0x7f12f81f8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:51.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.663+0000 7f12fd800640 1 --2- 172.21.15.151:0/301485565 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12dc0052e0 0x7f12f80d0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:51.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.663+0000 7f12fd800640 1 -- 172.21.15.151:0/301485565 >> 172.21.15.151:0/301485565 conn(0x7f12f80b1950 msgr2=0x7f12f80b11e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:51.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.664+0000 7f12fd800640 1 -- 172.21.15.151:0/301485565 shutdown_connections 2024-09-16T15:53:51.668 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.664+0000 7f12fd800640 1 -- 172.21.15.151:0/301485565 wait complete. 2024-09-16T15:53:51.681 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":566,"stamp":"2024-09-16T15:53:49.936763+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":182,"acting":182,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":28,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1077756,"kb_used_data":13828,"kb_used_omap":12,"kb_used_meta":1063731,"kb_avail":753896964,"statfs":{"total":773094113280,"available":771990491136,"internally_reserved":0,"allocated":14159872,"data_stored":6892291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12738,"internal_metadata":1089261118},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.916433"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.042290+0000","last_change":"2024-09-16T15:53:46.042290+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.042290+0000","last_undegraded":"2024-09-16T15:53:46.042290+0000","last_fullsized":"2024-09-16T15:53:46.042290+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.043416+0000","last_change":"2024-09-16T15:53:46.043416+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.043416+0000","last_undegraded":"2024-09-16T15:53:46.043416+0000","last_fullsized":"2024-09-16T15:53:46.043416+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.041857+0000","last_change":"2024-09-16T15:53:46.041857+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.041857+0000","last_undegraded":"2024-09-16T15:53:46.041857+0000","last_fullsized":"2024-09-16T15:53:46.041857+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.031217+0000","last_change":"2024-09-16T15:53:46.031217+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.031217+0000","last_undegraded":"2024-09-16T15:53:46.031217+0000","last_fullsized":"2024-09-16T15:53:46.031217+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.037204+0000","last_change":"2024-09-16T15:53:46.037204+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.037204+0000","last_undegraded":"2024-09-16T15:53:46.037204+0000","last_fullsized":"2024-09-16T15:53:46.037204+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.040603+0000","last_change":"2024-09-16T15:53:46.040603+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.040603+0000","last_undegraded":"2024-09-16T15:53:46.040603+0000","last_fullsized":"2024-09-16T15:53:46.040603+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.041279+0000","last_change":"2024-09-16T15:53:46.041279+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.041279+0000","last_undegraded":"2024-09-16T15:53:46.041279+0000","last_fullsized":"2024-09-16T15:53:46.041279+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.039521+0000","last_change":"2024-09-16T15:53:46.039521+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.039521+0000","last_undegraded":"2024-09-16T15:53:46.039521+0000","last_fullsized":"2024-09-16T15:53:46.039521+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.040481+0000","last_change":"2024-09-16T15:53:46.040481+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.040481+0000","last_undegraded":"2024-09-16T15:53:46.040481+0000","last_fullsized":"2024-09-16T15:53:46.040481+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.042888+0000","last_change":"2024-09-16T15:53:46.042888+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.042888+0000","last_undegraded":"2024-09-16T15:53:46.042888+0000","last_fullsized":"2024-09-16T15:53:46.042888+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.031743+0000","last_change":"2024-09-16T15:53:46.031743+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.031743+0000","last_undegraded":"2024-09-16T15:53:46.031743+0000","last_fullsized":"2024-09-16T15:53:46.031743+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.042568+0000","last_change":"2024-09-16T15:53:46.042568+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.042568+0000","last_undegraded":"2024-09-16T15:53:46.042568+0000","last_fullsized":"2024-09-16T15:53:46.042568+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.031210+0000","last_change":"2024-09-16T15:53:46.031210+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.031210+0000","last_undegraded":"2024-09-16T15:53:46.031210+0000","last_fullsized":"2024-09-16T15:53:46.031210+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":10,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-16T15:53:47.058871+0000","last_change":"2024-09-16T15:53:47.056102+0000","last_active":"2024-09-16T15:53:47.058871+0000","last_peered":"2024-09-16T15:53:47.058871+0000","last_clean":"2024-09-16T15:53:47.058871+0000","last_became_active":"2024-09-16T15:53:47.055786+0000","last_became_peered":"2024-09-16T15:53:47.055786+0000","last_unstale":"2024-09-16T15:53:47.058871+0000","last_undegraded":"2024-09-16T15:53:47.058871+0000","last_fullsized":"2024-09-16T15:53:47.058871+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:02:29.501341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:53:46.018368+0000","last_change":"2024-09-16T15:53:46.018368+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.018368+0000","last_undegraded":"2024-09-16T15:53:46.018368+0000","last_fullsized":"2024-09-16T15:53:46.018368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":10,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-16T15:53:47.063272+0000","last_change":"2024-09-16T15:53:47.060703+0000","last_active":"2024-09-16T15:53:47.063272+0000","last_peered":"2024-09-16T15:53:47.063272+0000","last_clean":"2024-09-16T15:53:47.063272+0000","last_became_active":"2024-09-16T15:53:47.060125+0000","last_became_peered":"2024-09-16T15:53:47.060125+0000","last_unstale":"2024-09-16T15:53:47.063272+0000","last_undegraded":"2024-09-16T15:53:47.063272+0000","last_fullsized":"2024-09-16T15:53:47.063272+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:29:02.714478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.039985+0000","last_change":"2024-09-16T15:53:46.039985+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.039985+0000","last_undegraded":"2024-09-16T15:53:46.039985+0000","last_fullsized":"2024-09-16T15:53:46.039985+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:53:46.018368+0000","last_change":"2024-09-16T15:53:46.018368+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.018368+0000","last_undegraded":"2024-09-16T15:53:46.018368+0000","last_fullsized":"2024-09-16T15:53:46.018368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.038804+0000","last_change":"2024-09-16T15:53:46.038804+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.038804+0000","last_undegraded":"2024-09-16T15:53:46.038804+0000","last_fullsized":"2024-09-16T15:53:46.038804+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:53:46.018368+0000","last_change":"2024-09-16T15:53:46.018368+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.018368+0000","last_undegraded":"2024-09-16T15:53:46.018368+0000","last_fullsized":"2024-09-16T15:53:46.018368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.038853+0000","last_change":"2024-09-16T15:53:46.038853+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.038853+0000","last_undegraded":"2024-09-16T15:53:46.038853+0000","last_fullsized":"2024-09-16T15:53:46.038853+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.040567+0000","last_change":"2024-09-16T15:53:46.040567+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.040567+0000","last_undegraded":"2024-09-16T15:53:46.040567+0000","last_fullsized":"2024-09-16T15:53:46.040567+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.037506+0000","last_change":"2024-09-16T15:53:46.037506+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.037506+0000","last_undegraded":"2024-09-16T15:53:46.037506+0000","last_fullsized":"2024-09-16T15:53:46.037506+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:53:46.018368+0000","last_change":"2024-09-16T15:53:46.018368+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.018368+0000","last_undegraded":"2024-09-16T15:53:46.018368+0000","last_fullsized":"2024-09-16T15:53:46.018368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.034184+0000","last_change":"2024-09-16T15:53:46.034184+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.034184+0000","last_undegraded":"2024-09-16T15:53:46.034184+0000","last_fullsized":"2024-09-16T15:53:46.034184+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.040330+0000","last_change":"2024-09-16T15:53:46.040330+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.040330+0000","last_undegraded":"2024-09-16T15:53:46.040330+0000","last_fullsized":"2024-09-16T15:53:46.040330+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":10,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-16T15:53:47.068210+0000","last_change":"2024-09-16T15:53:47.067044+0000","last_active":"2024-09-16T15:53:47.068210+0000","last_peered":"2024-09-16T15:53:47.068210+0000","last_clean":"2024-09-16T15:53:47.068210+0000","last_became_active":"2024-09-16T15:53:47.066772+0000","last_became_peered":"2024-09-16T15:53:47.066772+0000","last_unstale":"2024-09-16T15:53:47.068210+0000","last_undegraded":"2024-09-16T15:53:47.068210+0000","last_fullsized":"2024-09-16T15:53:47.068210+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:27:45.682352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.036344+0000","last_change":"2024-09-16T15:53:46.036344+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.036344+0000","last_undegraded":"2024-09-16T15:53:46.036344+0000","last_fullsized":"2024-09-16T15:53:46.036344+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.041980+0000","last_change":"2024-09-16T15:53:46.041980+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.041980+0000","last_undegraded":"2024-09-16T15:53:46.041980+0000","last_fullsized":"2024-09-16T15:53:46.041980+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.038900+0000","last_change":"2024-09-16T15:53:46.038900+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.038900+0000","last_undegraded":"2024-09-16T15:53:46.038900+0000","last_fullsized":"2024-09-16T15:53:46.038900+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.035309+0000","last_change":"2024-09-16T15:53:46.035309+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.035309+0000","last_undegraded":"2024-09-16T15:53:46.035309+0000","last_fullsized":"2024-09-16T15:53:46.035309+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.046495+0000","last_change":"2024-09-16T15:53:46.046495+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.046495+0000","last_undegraded":"2024-09-16T15:53:46.046495+0000","last_fullsized":"2024-09-16T15:53:46.046495+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":10,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-16T15:53:47.062974+0000","last_change":"2024-09-16T15:53:47.056337+0000","last_active":"2024-09-16T15:53:47.062974+0000","last_peered":"2024-09-16T15:53:47.062974+0000","last_clean":"2024-09-16T15:53:47.062974+0000","last_became_active":"2024-09-16T15:53:47.056039+0000","last_became_peered":"2024-09-16T15:53:47.056039+0000","last_unstale":"2024-09-16T15:53:47.062974+0000","last_undegraded":"2024-09-16T15:53:47.062974+0000","last_fullsized":"2024-09-16T15:53:47.062974+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:12:43.131784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T15:53:46.059217+0000","last_change":"2024-09-16T15:53:46.049579+0000","last_active":"2024-09-16T15:53:46.059217+0000","last_peered":"2024-09-16T15:53:46.059217+0000","last_clean":"2024-09-16T15:53:46.059217+0000","last_became_active":"2024-09-16T15:53:46.049306+0000","last_became_peered":"2024-09-16T15:53:46.049306+0000","last_unstale":"2024-09-16T15:53:46.059217+0000","last_undegraded":"2024-09-16T15:53:46.059217+0000","last_fullsized":"2024-09-16T15:53:46.059217+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:52:56.403013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.037825+0000","last_change":"2024-09-16T15:53:46.037825+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.037825+0000","last_undegraded":"2024-09-16T15:53:46.037825+0000","last_fullsized":"2024-09-16T15:53:46.037825+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T15:53:46.059575+0000","last_change":"2024-09-16T15:53:46.055817+0000","last_active":"2024-09-16T15:53:46.059575+0000","last_peered":"2024-09-16T15:53:46.059575+0000","last_clean":"2024-09-16T15:53:46.059575+0000","last_became_active":"2024-09-16T15:53:46.055540+0000","last_became_peered":"2024-09-16T15:53:46.055540+0000","last_unstale":"2024-09-16T15:53:46.059575+0000","last_undegraded":"2024-09-16T15:53:46.059575+0000","last_fullsized":"2024-09-16T15:53:46.059575+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:11:20.894617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.036684+0000","last_change":"2024-09-16T15:53:46.036684+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.036684+0000","last_undegraded":"2024-09-16T15:53:46.036684+0000","last_fullsized":"2024-09-16T15:53:46.036684+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T15:53:46.045935+0000","last_change":"2024-09-16T15:53:46.043053+0000","last_active":"2024-09-16T15:53:46.045935+0000","last_peered":"2024-09-16T15:53:46.045935+0000","last_clean":"2024-09-16T15:53:46.045935+0000","last_became_active":"2024-09-16T15:53:46.042758+0000","last_became_peered":"2024-09-16T15:53:46.042758+0000","last_unstale":"2024-09-16T15:53:46.045935+0000","last_undegraded":"2024-09-16T15:53:46.045935+0000","last_fullsized":"2024-09-16T15:53:46.045935+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:42:47.639276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.038140+0000","last_change":"2024-09-16T15:53:46.038140+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.038140+0000","last_undegraded":"2024-09-16T15:53:46.038140+0000","last_fullsized":"2024-09-16T15:53:46.038140+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T15:53:46.050362+0000","last_change":"2024-09-16T15:53:46.046499+0000","last_active":"2024-09-16T15:53:46.050362+0000","last_peered":"2024-09-16T15:53:46.050362+0000","last_clean":"2024-09-16T15:53:46.050362+0000","last_became_active":"2024-09-16T15:53:46.046246+0000","last_became_peered":"2024-09-16T15:53:46.046246+0000","last_unstale":"2024-09-16T15:53:46.050362+0000","last_undegraded":"2024-09-16T15:53:46.050362+0000","last_fullsized":"2024-09-16T15:53:46.050362+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:30:50.321365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:53:46.018368+0000","last_change":"2024-09-16T15:53:46.018368+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.018368+0000","last_undegraded":"2024-09-16T15:53:46.018368+0000","last_fullsized":"2024-09-16T15:53:46.018368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T15:53:46.052654+0000","last_change":"2024-09-16T15:53:46.048816+0000","last_active":"2024-09-16T15:53:46.052654+0000","last_peered":"2024-09-16T15:53:46.052654+0000","last_clean":"2024-09-16T15:53:46.052654+0000","last_became_active":"2024-09-16T15:53:46.048416+0000","last_became_peered":"2024-09-16T15:53:46.048416+0000","last_unstale":"2024-09-16T15:53:46.052654+0000","last_undegraded":"2024-09-16T15:53:46.052654+0000","last_fullsized":"2024-09-16T15:53:46.052654+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:50:02.529556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.038462+0000","last_change":"2024-09-16T15:53:46.038462+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.038462+0000","last_undegraded":"2024-09-16T15:53:46.038462+0000","last_fullsized":"2024-09-16T15:53:46.038462+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-16T15:53:45.025483+0000","last_change":"2024-09-16T15:53:45.025483+0000","last_active":"2024-09-16T15:53:45.010151+0000","last_peered":"2024-09-16T15:53:45.010151+0000","last_clean":"2024-09-16T15:53:45.010151+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:45.025483+0000","last_undegraded":"2024-09-16T15:53:45.025483+0000","last_fullsized":"2024-09-16T15:53:45.025483+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.039318+0000","last_change":"2024-09-16T15:53:46.039318+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.039318+0000","last_undegraded":"2024-09-16T15:53:46.039318+0000","last_fullsized":"2024-09-16T15:53:46.039318+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T15:53:46.059072+0000","last_change":"2024-09-16T15:53:46.052810+0000","last_active":"2024-09-16T15:53:46.059072+0000","last_peered":"2024-09-16T15:53:46.059072+0000","last_clean":"2024-09-16T15:53:46.059072+0000","last_became_active":"2024-09-16T15:53:46.052461+0000","last_became_peered":"2024-09-16T15:53:46.052461+0000","last_unstale":"2024-09-16T15:53:46.059072+0000","last_undegraded":"2024-09-16T15:53:46.059072+0000","last_fullsized":"2024-09-16T15:53:46.059072+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:20:19.397356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.037477+0000","last_change":"2024-09-16T15:53:46.037477+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.037477+0000","last_undegraded":"2024-09-16T15:53:46.037477+0000","last_fullsized":"2024-09-16T15:53:46.037477+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T15:53:46.052385+0000","last_change":"2024-09-16T15:53:46.050598+0000","last_active":"2024-09-16T15:53:46.052385+0000","last_peered":"2024-09-16T15:53:46.052385+0000","last_clean":"2024-09-16T15:53:46.052385+0000","last_became_active":"2024-09-16T15:53:46.050355+0000","last_became_peered":"2024-09-16T15:53:46.050355+0000","last_unstale":"2024-09-16T15:53:46.052385+0000","last_undegraded":"2024-09-16T15:53:46.052385+0000","last_fullsized":"2024-09-16T15:53:46.052385+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:28:46.376887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.039821+0000","last_change":"2024-09-16T15:53:46.039821+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.039821+0000","last_undegraded":"2024-09-16T15:53:46.039821+0000","last_fullsized":"2024-09-16T15:53:46.039821+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T15:53:46.054603+0000","last_change":"2024-09-16T15:53:46.041207+0000","last_active":"2024-09-16T15:53:46.054603+0000","last_peered":"2024-09-16T15:53:46.054603+0000","last_clean":"2024-09-16T15:53:46.054603+0000","last_became_active":"2024-09-16T15:53:46.040942+0000","last_became_peered":"2024-09-16T15:53:46.040942+0000","last_unstale":"2024-09-16T15:53:46.054603+0000","last_undegraded":"2024-09-16T15:53:46.054603+0000","last_fullsized":"2024-09-16T15:53:46.054603+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:06:50.417480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.042730+0000","last_change":"2024-09-16T15:53:46.042730+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.042730+0000","last_undegraded":"2024-09-16T15:53:46.042730+0000","last_fullsized":"2024-09-16T15:53:46.042730+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-16T15:53:45.024968+0000","last_change":"2024-09-16T15:53:45.024968+0000","last_active":"2024-09-16T15:53:45.010151+0000","last_peered":"2024-09-16T15:53:45.010151+0000","last_clean":"2024-09-16T15:53:45.010151+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:45.024968+0000","last_undegraded":"2024-09-16T15:53:45.024968+0000","last_fullsized":"2024-09-16T15:53:45.024968+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.038076+0000","last_change":"2024-09-16T15:53:46.038076+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.038076+0000","last_undegraded":"2024-09-16T15:53:46.038076+0000","last_fullsized":"2024-09-16T15:53:46.038076+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T15:53:46.059281+0000","last_change":"2024-09-16T15:53:46.051421+0000","last_active":"2024-09-16T15:53:46.059281+0000","last_peered":"2024-09-16T15:53:46.059281+0000","last_clean":"2024-09-16T15:53:46.059281+0000","last_became_active":"2024-09-16T15:53:46.049074+0000","last_became_peered":"2024-09-16T15:53:46.049074+0000","last_unstale":"2024-09-16T15:53:46.059281+0000","last_undegraded":"2024-09-16T15:53:46.059281+0000","last_fullsized":"2024-09-16T15:53:46.059281+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:17:54.102543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.038820+0000","last_change":"2024-09-16T15:53:46.038820+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.038820+0000","last_undegraded":"2024-09-16T15:53:46.038820+0000","last_fullsized":"2024-09-16T15:53:46.038820+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T15:53:46.056488+0000","last_change":"2024-09-16T15:53:46.055615+0000","last_active":"2024-09-16T15:53:46.056488+0000","last_peered":"2024-09-16T15:53:46.056488+0000","last_clean":"2024-09-16T15:53:46.056488+0000","last_became_active":"2024-09-16T15:53:46.055155+0000","last_became_peered":"2024-09-16T15:53:46.055155+0000","last_unstale":"2024-09-16T15:53:46.056488+0000","last_undegraded":"2024-09-16T15:53:46.056488+0000","last_fullsized":"2024-09-16T15:53:46.056488+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:54:56.815280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":10,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-16T15:53:47.059010+0000","last_change":"2024-09-16T15:53:47.055028+0000","last_active":"2024-09-16T15:53:47.059010+0000","last_peered":"2024-09-16T15:53:47.059010+0000","last_clean":"2024-09-16T15:53:47.059010+0000","last_became_active":"2024-09-16T15:53:47.054638+0000","last_became_peered":"2024-09-16T15:53:47.054638+0000","last_unstale":"2024-09-16T15:53:47.059010+0000","last_undegraded":"2024-09-16T15:53:47.059010+0000","last_fullsized":"2024-09-16T15:53:47.059010+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:49:07.189417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-16T15:53:47.042905+0000","last_change":"2024-09-16T15:53:46.053325+0000","last_active":"2024-09-16T15:53:47.042905+0000","last_peered":"2024-09-16T15:53:47.042905+0000","last_clean":"2024-09-16T15:53:47.042905+0000","last_became_active":"2024-09-16T15:53:46.051989+0000","last_became_peered":"2024-09-16T15:53:46.051989+0000","last_unstale":"2024-09-16T15:53:47.042905+0000","last_undegraded":"2024-09-16T15:53:47.042905+0000","last_fullsized":"2024-09-16T15:53:47.042905+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:10:52.279943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":10,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-16T15:53:47.063798+0000","last_change":"2024-09-16T15:53:47.062301+0000","last_active":"2024-09-16T15:53:47.063798+0000","last_peered":"2024-09-16T15:53:47.063798+0000","last_clean":"2024-09-16T15:53:47.063798+0000","last_became_active":"2024-09-16T15:53:47.058392+0000","last_became_peered":"2024-09-16T15:53:47.058392+0000","last_unstale":"2024-09-16T15:53:47.063798+0000","last_undegraded":"2024-09-16T15:53:47.063798+0000","last_fullsized":"2024-09-16T15:53:47.063798+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:35:38.397852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T15:53:46.058498+0000","last_change":"2024-09-16T15:53:46.055507+0000","last_active":"2024-09-16T15:53:46.058498+0000","last_peered":"2024-09-16T15:53:46.058498+0000","last_clean":"2024-09-16T15:53:46.058498+0000","last_became_active":"2024-09-16T15:53:46.055247+0000","last_became_peered":"2024-09-16T15:53:46.055247+0000","last_unstale":"2024-09-16T15:53:46.058498+0000","last_undegraded":"2024-09-16T15:53:46.058498+0000","last_fullsized":"2024-09-16T15:53:46.058498+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:07:18.321225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.037392+0000","last_change":"2024-09-16T15:53:46.037392+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.037392+0000","last_undegraded":"2024-09-16T15:53:46.037392+0000","last_fullsized":"2024-09-16T15:53:46.037392+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T15:53:46.058960+0000","last_change":"2024-09-16T15:53:46.054698+0000","last_active":"2024-09-16T15:53:46.058960+0000","last_peered":"2024-09-16T15:53:46.058960+0000","last_clean":"2024-09-16T15:53:46.058960+0000","last_became_active":"2024-09-16T15:53:46.050527+0000","last_became_peered":"2024-09-16T15:53:46.050527+0000","last_unstale":"2024-09-16T15:53:46.058960+0000","last_undegraded":"2024-09-16T15:53:46.058960+0000","last_fullsized":"2024-09-16T15:53:46.058960+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:24:23.869366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.042645+0000","last_change":"2024-09-16T15:53:46.042645+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.042645+0000","last_undegraded":"2024-09-16T15:53:46.042645+0000","last_fullsized":"2024-09-16T15:53:46.042645+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T15:53:46.054629+0000","last_change":"2024-09-16T15:53:46.041954+0000","last_active":"2024-09-16T15:53:46.054629+0000","last_peered":"2024-09-16T15:53:46.054629+0000","last_clean":"2024-09-16T15:53:46.054629+0000","last_became_active":"2024-09-16T15:53:46.041272+0000","last_became_peered":"2024-09-16T15:53:46.041272+0000","last_unstale":"2024-09-16T15:53:46.054629+0000","last_undegraded":"2024-09-16T15:53:46.054629+0000","last_fullsized":"2024-09-16T15:53:46.054629+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:20:31.172329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.041139+0000","last_change":"2024-09-16T15:53:46.041139+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.041139+0000","last_undegraded":"2024-09-16T15:53:46.041139+0000","last_fullsized":"2024-09-16T15:53:46.041139+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T15:53:46.049906+0000","last_change":"2024-09-16T15:53:46.048066+0000","last_active":"2024-09-16T15:53:46.049906+0000","last_peered":"2024-09-16T15:53:46.049906+0000","last_clean":"2024-09-16T15:53:46.049906+0000","last_became_active":"2024-09-16T15:53:46.047765+0000","last_became_peered":"2024-09-16T15:53:46.047765+0000","last_unstale":"2024-09-16T15:53:46.049906+0000","last_undegraded":"2024-09-16T15:53:46.049906+0000","last_fullsized":"2024-09-16T15:53:46.049906+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:33:52.388798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.036912+0000","last_change":"2024-09-16T15:53:46.036912+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.036912+0000","last_undegraded":"2024-09-16T15:53:46.036912+0000","last_fullsized":"2024-09-16T15:53:46.036912+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T15:53:46.054123+0000","last_change":"2024-09-16T15:53:46.050413+0000","last_active":"2024-09-16T15:53:46.054123+0000","last_peered":"2024-09-16T15:53:46.054123+0000","last_clean":"2024-09-16T15:53:46.054123+0000","last_became_active":"2024-09-16T15:53:46.050136+0000","last_became_peered":"2024-09-16T15:53:46.050136+0000","last_unstale":"2024-09-16T15:53:46.054123+0000","last_undegraded":"2024-09-16T15:53:46.054123+0000","last_fullsized":"2024-09-16T15:53:46.054123+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:55:06.155424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.038394+0000","last_change":"2024-09-16T15:53:46.038394+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.038394+0000","last_undegraded":"2024-09-16T15:53:46.038394+0000","last_fullsized":"2024-09-16T15:53:46.038394+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T15:53:46.059203+0000","last_change":"2024-09-16T15:53:46.052237+0000","last_active":"2024-09-16T15:53:46.059203+0000","last_peered":"2024-09-16T15:53:46.059203+0000","last_clean":"2024-09-16T15:53:46.059203+0000","last_became_active":"2024-09-16T15:53:46.051939+0000","last_became_peered":"2024-09-16T15:53:46.051939+0000","last_unstale":"2024-09-16T15:53:46.059203+0000","last_undegraded":"2024-09-16T15:53:46.059203+0000","last_fullsized":"2024-09-16T15:53:46.059203+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:52:12.659520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.036480+0000","last_change":"2024-09-16T15:53:46.036480+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.036480+0000","last_undegraded":"2024-09-16T15:53:46.036480+0000","last_fullsized":"2024-09-16T15:53:46.036480+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T15:53:46.051376+0000","last_change":"2024-09-16T15:53:46.049709+0000","last_active":"2024-09-16T15:53:46.051376+0000","last_peered":"2024-09-16T15:53:46.051376+0000","last_clean":"2024-09-16T15:53:46.051376+0000","last_became_active":"2024-09-16T15:53:46.049412+0000","last_became_peered":"2024-09-16T15:53:46.049412+0000","last_unstale":"2024-09-16T15:53:46.051376+0000","last_undegraded":"2024-09-16T15:53:46.051376+0000","last_fullsized":"2024-09-16T15:53:46.051376+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:54:26.788931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.037329+0000","last_change":"2024-09-16T15:53:46.037329+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.037329+0000","last_undegraded":"2024-09-16T15:53:46.037329+0000","last_fullsized":"2024-09-16T15:53:46.037329+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-16T15:53:45.025032+0000","last_change":"2024-09-16T15:53:45.025032+0000","last_active":"2024-09-16T15:53:45.010151+0000","last_peered":"2024-09-16T15:53:45.010151+0000","last_clean":"2024-09-16T15:53:45.010151+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:45.025032+0000","last_undegraded":"2024-09-16T15:53:45.025032+0000","last_fullsized":"2024-09-16T15:53:45.025032+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":10,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-16T15:53:47.063249+0000","last_change":"2024-09-16T15:53:47.058656+0000","last_active":"2024-09-16T15:53:47.063249+0000","last_peered":"2024-09-16T15:53:47.063249+0000","last_clean":"2024-09-16T15:53:47.063249+0000","last_became_active":"2024-09-16T15:53:47.057260+0000","last_became_peered":"2024-09-16T15:53:47.057260+0000","last_unstale":"2024-09-16T15:53:47.063249+0000","last_undegraded":"2024-09-16T15:53:47.063249+0000","last_fullsized":"2024-09-16T15:53:47.063249+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:18:36.519474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-16T15:53:47.041902+0000","last_change":"2024-09-16T15:53:46.038714+0000","last_active":"2024-09-16T15:53:47.041902+0000","last_peered":"2024-09-16T15:53:47.041902+0000","last_clean":"2024-09-16T15:53:47.041902+0000","last_became_active":"2024-09-16T15:53:46.038411+0000","last_became_peered":"2024-09-16T15:53:46.038411+0000","last_unstale":"2024-09-16T15:53:47.041902+0000","last_undegraded":"2024-09-16T15:53:47.041902+0000","last_fullsized":"2024-09-16T15:53:47.041902+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:43:20.313472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.035844+0000","last_change":"2024-09-16T15:53:46.035844+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.035844+0000","last_undegraded":"2024-09-16T15:53:46.035844+0000","last_fullsized":"2024-09-16T15:53:46.035844+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T15:53:46.054391+0000","last_change":"2024-09-16T15:53:46.049975+0000","last_active":"2024-09-16T15:53:46.054391+0000","last_peered":"2024-09-16T15:53:46.054391+0000","last_clean":"2024-09-16T15:53:46.054391+0000","last_became_active":"2024-09-16T15:53:46.049667+0000","last_became_peered":"2024-09-16T15:53:46.049667+0000","last_unstale":"2024-09-16T15:53:46.054391+0000","last_undegraded":"2024-09-16T15:53:46.054391+0000","last_fullsized":"2024-09-16T15:53:46.054391+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:45:49.295347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.040296+0000","last_change":"2024-09-16T15:53:46.040296+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.040296+0000","last_undegraded":"2024-09-16T15:53:46.040296+0000","last_fullsized":"2024-09-16T15:53:46.040296+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-16T15:53:45.025087+0000","last_change":"2024-09-16T15:53:45.025087+0000","last_active":"2024-09-16T15:53:45.010151+0000","last_peered":"2024-09-16T15:53:45.010151+0000","last_clean":"2024-09-16T15:53:45.010151+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:45.025087+0000","last_undegraded":"2024-09-16T15:53:45.025087+0000","last_fullsized":"2024-09-16T15:53:45.025087+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.040819+0000","last_change":"2024-09-16T15:53:46.040819+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.040819+0000","last_undegraded":"2024-09-16T15:53:46.040819+0000","last_fullsized":"2024-09-16T15:53:46.040819+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T15:53:46.059389+0000","last_change":"2024-09-16T15:53:46.055909+0000","last_active":"2024-09-16T15:53:46.059389+0000","last_peered":"2024-09-16T15:53:46.059389+0000","last_clean":"2024-09-16T15:53:46.059389+0000","last_became_active":"2024-09-16T15:53:46.055567+0000","last_became_peered":"2024-09-16T15:53:46.055567+0000","last_unstale":"2024-09-16T15:53:46.059389+0000","last_undegraded":"2024-09-16T15:53:46.059389+0000","last_fullsized":"2024-09-16T15:53:46.059389+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:42:42.792655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":10,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-16T15:53:47.058902+0000","last_change":"2024-09-16T15:53:47.055081+0000","last_active":"2024-09-16T15:53:47.058902+0000","last_peered":"2024-09-16T15:53:47.058902+0000","last_clean":"2024-09-16T15:53:47.058902+0000","last_became_active":"2024-09-16T15:53:47.054772+0000","last_became_peered":"2024-09-16T15:53:47.054772+0000","last_unstale":"2024-09-16T15:53:47.058902+0000","last_undegraded":"2024-09-16T15:53:47.058902+0000","last_fullsized":"2024-09-16T15:53:47.058902+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:36:50.843181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":1,"reported_epoch":130,"state":"creating+peering","last_fresh":"2024-09-16T15:53:45.024985+0000","last_change":"2024-09-16T15:53:45.024985+0000","last_active":"2024-09-16T15:53:45.010151+0000","last_peered":"2024-09-16T15:53:45.010151+0000","last_clean":"2024-09-16T15:53:45.010151+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:45.024985+0000","last_undegraded":"2024-09-16T15:53:45.024985+0000","last_fullsized":"2024-09-16T15:53:45.024985+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.036181+0000","last_change":"2024-09-16T15:53:46.036181+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.036181+0000","last_undegraded":"2024-09-16T15:53:46.036181+0000","last_fullsized":"2024-09-16T15:53:46.036181+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-16T15:53:47.042871+0000","last_change":"2024-09-16T15:53:46.057982+0000","last_active":"2024-09-16T15:53:47.042871+0000","last_peered":"2024-09-16T15:53:47.042871+0000","last_clean":"2024-09-16T15:53:47.042871+0000","last_became_active":"2024-09-16T15:53:46.057707+0000","last_became_peered":"2024-09-16T15:53:46.057707+0000","last_unstale":"2024-09-16T15:53:47.042871+0000","last_undegraded":"2024-09-16T15:53:47.042871+0000","last_fullsized":"2024-09-16T15:53:47.042871+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:59:40.544624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.034301+0000","last_change":"2024-09-16T15:53:46.034301+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.034301+0000","last_undegraded":"2024-09-16T15:53:46.034301+0000","last_fullsized":"2024-09-16T15:53:46.034301+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T15:53:46.055777+0000","last_change":"2024-09-16T15:53:46.052804+0000","last_active":"2024-09-16T15:53:46.055777+0000","last_peered":"2024-09-16T15:53:46.055777+0000","last_clean":"2024-09-16T15:53:46.055777+0000","last_became_active":"2024-09-16T15:53:46.052460+0000","last_became_peered":"2024-09-16T15:53:46.052460+0000","last_unstale":"2024-09-16T15:53:46.055777+0000","last_undegraded":"2024-09-16T15:53:46.055777+0000","last_fullsized":"2024-09-16T15:53:46.055777+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:55:25.120515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.035212+0000","last_change":"2024-09-16T15:53:46.035212+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.035212+0000","last_undegraded":"2024-09-16T15:53:46.035212+0000","last_fullsized":"2024-09-16T15:53:46.035212+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T15:53:46.054865+0000","last_change":"2024-09-16T15:53:46.044167+0000","last_active":"2024-09-16T15:53:46.054865+0000","last_peered":"2024-09-16T15:53:46.054865+0000","last_clean":"2024-09-16T15:53:46.054865+0000","last_became_active":"2024-09-16T15:53:46.043914+0000","last_became_peered":"2024-09-16T15:53:46.043914+0000","last_unstale":"2024-09-16T15:53:46.054865+0000","last_undegraded":"2024-09-16T15:53:46.054865+0000","last_fullsized":"2024-09-16T15:53:46.054865+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:17:08.359001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T15:53:46.018368+0000","last_change":"2024-09-16T15:53:46.018368+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.018368+0000","last_undegraded":"2024-09-16T15:53:46.018368+0000","last_fullsized":"2024-09-16T15:53:46.018368+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T15:53:46.058473+0000","last_change":"2024-09-16T15:53:46.055377+0000","last_active":"2024-09-16T15:53:46.058473+0000","last_peered":"2024-09-16T15:53:46.058473+0000","last_clean":"2024-09-16T15:53:46.058473+0000","last_became_active":"2024-09-16T15:53:46.055135+0000","last_became_peered":"2024-09-16T15:53:46.055135+0000","last_unstale":"2024-09-16T15:53:46.058473+0000","last_undegraded":"2024-09-16T15:53:46.058473+0000","last_fullsized":"2024-09-16T15:53:46.058473+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:30:58.937449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":10,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-16T15:53:47.062177+0000","last_change":"2024-09-16T15:53:47.057817+0000","last_active":"2024-09-16T15:53:47.062177+0000","last_peered":"2024-09-16T15:53:47.062177+0000","last_clean":"2024-09-16T15:53:47.062177+0000","last_became_active":"2024-09-16T15:53:47.057440+0000","last_became_peered":"2024-09-16T15:53:47.057440+0000","last_unstale":"2024-09-16T15:53:47.062177+0000","last_undegraded":"2024-09-16T15:53:47.062177+0000","last_fullsized":"2024-09-16T15:53:47.062177+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:56:27.484880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-16T15:53:47.043225+0000","last_change":"2024-09-16T15:53:46.047710+0000","last_active":"2024-09-16T15:53:47.043225+0000","last_peered":"2024-09-16T15:53:47.043225+0000","last_clean":"2024-09-16T15:53:47.043225+0000","last_became_active":"2024-09-16T15:53:46.047320+0000","last_became_peered":"2024-09-16T15:53:46.047320+0000","last_unstale":"2024-09-16T15:53:47.043225+0000","last_undegraded":"2024-09-16T15:53:47.043225+0000","last_fullsized":"2024-09-16T15:53:47.043225+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:40:37.494885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":1,"reported_epoch":131,"state":"creating+peering","last_fresh":"2024-09-16T15:53:46.036166+0000","last_change":"2024-09-16T15:53:46.036166+0000","last_active":"2024-09-16T15:53:46.018368+0000","last_peered":"2024-09-16T15:53:46.018368+0000","last_clean":"2024-09-16T15:53:46.018368+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:53:46.036166+0000","last_undegraded":"2024-09-16T15:53:46.036166+0000","last_fullsized":"2024-09-16T15:53:46.036166+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":10,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T15:53:46.057121+0000","last_change":"2024-09-16T15:53:46.048215+0000","last_active":"2024-09-16T15:53:46.057121+0000","last_peered":"2024-09-16T15:53:46.057121+0000","last_clean":"2024-09-16T15:53:46.057121+0000","last_became_active":"2024-09-16T15:53:46.045488+0000","last_became_peered":"2024-09-16T15:53:46.045488+0000","last_unstale":"2024-09-16T15:53:46.057121+0000","last_undegraded":"2024-09-16T15:53:46.057121+0000","last_fullsized":"2024-09-16T15:53:46.057121+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:53:11.860146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":319,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-16T15:53:46.040564+0000","last_change":"2024-09-16T15:39:12.698813+0000","last_active":"2024-09-16T15:53:46.040564+0000","last_peered":"2024-09-16T15:53:46.040564+0000","last_clean":"2024-09-16T15:53:46.040564+0000","last_became_active":"2024-09-16T15:39:12.697695+0000","last_became_peered":"2024-09-16T15:39:12.697695+0000","last_unstale":"2024-09-16T15:53:46.040564+0000","last_undegraded":"2024-09-16T15:53:46.040564+0000","last_fullsized":"2024-09-16T15:53:46.040564+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_clean_scrub_stamp":"2024-09-16T15:39:11.684967+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:02.061081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":116,"acting":116,"num_store_stats":7},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509620,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":159068,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":157438,"kb_avail":94212772,"statfs":{"total":96636764160,"available":96473878528,"internally_reserved":0,"allocated":1638400,"data_stored":723164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":161216968},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509619,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":119376,"kb_used_data":2168,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94252464,"statfs":{"total":96636764160,"available":96514523136,"internally_reserved":0,"allocated":2220032,"data_stored":1306645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":119994823},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509618,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":136828,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94235012,"statfs":{"total":96636764160,"available":96496652288,"internally_reserved":0,"allocated":1613824,"data_stored":709890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":138475973},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509618,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":118780,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94253060,"statfs":{"total":96636764160,"available":96515133440,"internally_reserved":0,"allocated":1613824,"data_stored":709890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":119994825},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509618,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131276,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240564,"statfs":{"total":96636764160,"available":96502337536,"internally_reserved":0,"allocated":1626112,"data_stored":716277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509619,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131856,"kb_used_data":2168,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94239984,"statfs":{"total":96636764160,"available":96501743616,"internally_reserved":0,"allocated":2220032,"data_stored":1306645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509617,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149308,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222532,"statfs":{"total":96636764160,"available":96483872768,"internally_reserved":0,"allocated":1613824,"data_stored":709890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":151255495},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542324,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131264,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240576,"statfs":{"total":96636764160,"available":96502349824,"internally_reserved":0,"allocated":1613824,"data_stored":709890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":132774348},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:53:51.685 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:53:51.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.877+0000 7fe3b8a00640 1 -- 172.21.15.151:0/2149816120 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3b40725d0 msgr2=0x7fe3b40b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:51.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.877+0000 7fe3b8a00640 1 --2- 172.21.15.151:0/2149816120 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3b40725d0 0x7fe3b40b28f0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fe39c009830 tx=0x7fe39c02a5c0 comp rx=0 tx=0).stop 2024-09-16T15:53:51.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.877+0000 7fe3b8a00640 1 -- 172.21.15.151:0/2149816120 shutdown_connections 2024-09-16T15:53:51.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.877+0000 7fe3b8a00640 1 --2- 172.21.15.151:0/2149816120 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3b40725d0 0x7fe3b40b28f0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:51.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.877+0000 7fe3b8a00640 1 -- 172.21.15.151:0/2149816120 >> 172.21.15.151:0/2149816120 conn(0x7fe3b40cfa20 msgr2=0x7fe3b40cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:51.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.878+0000 7fe3b8a00640 1 -- 172.21.15.151:0/2149816120 shutdown_connections 2024-09-16T15:53:51.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.878+0000 7fe3b8a00640 1 -- 172.21.15.151:0/2149816120 wait complete. 2024-09-16T15:53:51.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.879+0000 7fe3b8a00640 1 Processor -- start 2024-09-16T15:53:51.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.879+0000 7fe3b8a00640 1 -- start start 2024-09-16T15:53:51.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.880+0000 7fe3b8a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3b40d0540 0x7fe3b4155ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:51.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.880+0000 7fe3b8a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe3b415bc30 0x7fe3b41f2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:51.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.880+0000 7fe3b8a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe3b4154780 0x7fe3b41ef6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:51.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.880+0000 7fe3b8a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b4074660 con 0x7fe3b4154780 2024-09-16T15:53:51.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.880+0000 7fe3b8a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b406fa30 con 0x7fe3b40d0540 2024-09-16T15:53:51.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.880+0000 7fe3b8a00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fe3b41e1370 con 0x7fe3b415bc30 2024-09-16T15:53:51.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.880+0000 7fe3b3e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe3b4154780 0x7fe3b41ef6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:51.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.880+0000 7fe3b3e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe3b4154780 0x7fe3b41ef6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43780/0 (socket says 172.21.15.151:43780) 2024-09-16T15:53:51.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.881+0000 7fe3b3e00640 1 -- 172.21.15.151:0/3749206337 learned_addr learned my addr 172.21.15.151:0/3749206337 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:53:51.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.881+0000 7fe3abe00640 1 --2- 172.21.15.151:0/3749206337 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe3b415bc30 0x7fe3b41f2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:51.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.881+0000 7fe3b3400640 1 --2- 172.21.15.151:0/3749206337 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3b40d0540 0x7fe3b4155ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:51.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.881+0000 7fe3b3e00640 1 -- 172.21.15.151:0/3749206337 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe3b415bc30 msgr2=0x7fe3b41f2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:51.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.881+0000 7fe3b3e00640 1 --2- 172.21.15.151:0/3749206337 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe3b415bc30 0x7fe3b41f2c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:51.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.881+0000 7fe3b3e00640 1 -- 172.21.15.151:0/3749206337 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3b40d0540 msgr2=0x7fe3b4155ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:51.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.881+0000 7fe3b3e00640 1 --2- 172.21.15.151:0/3749206337 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3b40d0540 0x7fe3b4155ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:51.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.881+0000 7fe3b3e00640 1 -- 172.21.15.151:0/3749206337 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3b41f31d0 con 0x7fe3b4154780 2024-09-16T15:53:51.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.881+0000 7fe3b3e00640 1 --2- 172.21.15.151:0/3749206337 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe3b4154780 0x7fe3b41ef6f0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac00cbb0 tx=0x7fe3ac00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:51.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.881+0000 7fe3ab400640 1 -- 172.21.15.151:0/3749206337 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac016020 con 0x7fe3b4154780 2024-09-16T15:53:51.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.881+0000 7fe3ab400640 1 -- 172.21.15.151:0/3749206337 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe3ac00ae30 con 0x7fe3b4154780 2024-09-16T15:53:51.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.881+0000 7fe3b8a00640 1 -- 172.21.15.151:0/3749206337 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3b41efc30 con 0x7fe3b4154780 2024-09-16T15:53:51.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.882+0000 7fe3ab400640 1 -- 172.21.15.151:0/3749206337 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac012660 con 0x7fe3b4154780 2024-09-16T15:53:51.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.882+0000 7fe3b8a00640 1 -- 172.21.15.151:0/3749206337 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3b4154f20 con 0x7fe3b4154780 2024-09-16T15:53:51.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.883+0000 7fe3b8a00640 1 -- 172.21.15.151:0/3749206337 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3b4050ee0 con 0x7fe3b4154780 2024-09-16T15:53:51.885 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.884+0000 7fe3ab400640 1 -- 172.21.15.151:0/3749206337 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 47) ==== 104092+0+0 (secure 0 0 0) 0x7fe3ac019020 con 0x7fe3b4154780 2024-09-16T15:53:51.886 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.885+0000 7fe3ab400640 1 --2- 172.21.15.151:0/3749206337 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe39007a740 0x7fe39007cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:51.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.885+0000 7fe3ab400640 1 -- 172.21.15.151:0/3749206337 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5821+0+0 (secure 0 0 0) 0x7fe3ac099bf0 con 0x7fe3b4154780 2024-09-16T15:53:51.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.885+0000 7fe3b3400640 1 --2- 172.21.15.151:0/3749206337 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe39007a740 0x7fe39007cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:51.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.886+0000 7fe3b3400640 1 --2- 172.21.15.151:0/3749206337 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe39007a740 0x7fe39007cc00 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fe3a4000f60 tx=0x7fe3a4009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:51.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:51.889+0000 7fe3ab400640 1 -- 172.21.15.151:0/3749206337 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe3b4050ee0 con 0x7fe3b4154780 2024-09-16T15:53:52.007 INFO:tasks.ceph.mds.a.smithi151.stderr:2024-09-16T15:53:52.005+0000 7fd32f000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-16T15:53:52.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.077+0000 7fe3b8a00640 1 -- 172.21.15.151:0/3749206337 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe3b406a180 con 0x7fe39007a740 2024-09-16T15:53:52.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.082+0000 7fe3ab400640 1 -- 172.21.15.151:0/3749206337 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252498 (secure 0 0 0) 0x7fe3b406a180 con 0x7fe39007a740 2024-09-16T15:53:52.087 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:53:52.088 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-09-16T15:53:52.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.086+0000 7fe3b8a00640 1 -- 172.21.15.151:0/3749206337 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe39007a740 msgr2=0x7fe39007cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:52.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.086+0000 7fe3b8a00640 1 --2- 172.21.15.151:0/3749206337 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe39007a740 0x7fe39007cc00 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fe3a4000f60 tx=0x7fe3a4009040 comp rx=0 tx=0).stop 2024-09-16T15:53:52.090 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.086+0000 7fe3b8a00640 1 -- 172.21.15.151:0/3749206337 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe3b4154780 msgr2=0x7fe3b41ef6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:52.090 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.086+0000 7fe3b8a00640 1 --2- 172.21.15.151:0/3749206337 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe3b4154780 0x7fe3b41ef6f0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac00cbb0 tx=0x7fe3ac00a430 comp rx=0 tx=0).stop 2024-09-16T15:53:52.090 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.086+0000 7fe3b8a00640 1 -- 172.21.15.151:0/3749206337 shutdown_connections 2024-09-16T15:53:52.090 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.086+0000 7fe3b8a00640 1 --2- 172.21.15.151:0/3749206337 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe39007a740 0x7fe39007cc00 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:52.090 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.086+0000 7fe3b8a00640 1 --2- 172.21.15.151:0/3749206337 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe3b4154780 0x7fe3b41ef6f0 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:52.090 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.086+0000 7fe3b8a00640 1 --2- 172.21.15.151:0/3749206337 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe3b415bc30 0x7fe3b41f2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:52.090 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.086+0000 7fe3b8a00640 1 --2- 172.21.15.151:0/3749206337 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3b40d0540 0x7fe3b4155ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:52.090 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.086+0000 7fe3b8a00640 1 -- 172.21.15.151:0/3749206337 >> 172.21.15.151:0/3749206337 conn(0x7fe3b40cfa20 msgr2=0x7fe3b40cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:52.090 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.086+0000 7fe3b8a00640 1 -- 172.21.15.151:0/3749206337 shutdown_connections 2024-09-16T15:53:52.090 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.087+0000 7fe3b8a00640 1 -- 172.21.15.151:0/3749206337 wait complete. 2024-09-16T15:53:52.104 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":567,"stamp":"2024-09-16T15:53:51.937187+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":164,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1078028,"kb_used_data":14108,"kb_used_omap":12,"kb_used_meta":1063731,"kb_avail":753896692,"statfs":{"total":773094113280,"available":771990212608,"internally_reserved":0,"allocated":14446592,"data_stored":7015752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12738,"internal_metadata":1089261118},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001532"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.274843+0000","last_change":"2024-09-16T15:53:47.064079+0000","last_active":"2024-09-16T15:53:49.274843+0000","last_peered":"2024-09-16T15:53:49.274843+0000","last_clean":"2024-09-16T15:53:49.274843+0000","last_became_active":"2024-09-16T15:53:47.063709+0000","last_became_peered":"2024-09-16T15:53:47.063709+0000","last_unstale":"2024-09-16T15:53:49.274843+0000","last_undegraded":"2024-09-16T15:53:49.274843+0000","last_fullsized":"2024-09-16T15:53:49.274843+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:52:00.354801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.274872+0000","last_change":"2024-09-16T15:53:47.054602+0000","last_active":"2024-09-16T15:53:49.274872+0000","last_peered":"2024-09-16T15:53:49.274872+0000","last_clean":"2024-09-16T15:53:49.274872+0000","last_became_active":"2024-09-16T15:53:47.054286+0000","last_became_peered":"2024-09-16T15:53:47.054286+0000","last_unstale":"2024-09-16T15:53:49.274872+0000","last_undegraded":"2024-09-16T15:53:49.274872+0000","last_fullsized":"2024-09-16T15:53:49.274872+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:18:36.519474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.604743+0000","last_change":"2024-09-16T15:53:47.049667+0000","last_active":"2024-09-16T15:53:48.604743+0000","last_peered":"2024-09-16T15:53:48.604743+0000","last_clean":"2024-09-16T15:53:48.604743+0000","last_became_active":"2024-09-16T15:53:47.048431+0000","last_became_peered":"2024-09-16T15:53:47.048431+0000","last_unstale":"2024-09-16T15:53:48.604743+0000","last_undegraded":"2024-09-16T15:53:48.604743+0000","last_fullsized":"2024-09-16T15:53:48.604743+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:25:27.052540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.036138+0000","last_change":"2024-09-16T15:53:47.066311+0000","last_active":"2024-09-16T15:53:48.036138+0000","last_peered":"2024-09-16T15:53:48.036138+0000","last_clean":"2024-09-16T15:53:48.036138+0000","last_became_active":"2024-09-16T15:53:47.066005+0000","last_became_peered":"2024-09-16T15:53:47.066005+0000","last_unstale":"2024-09-16T15:53:48.036138+0000","last_undegraded":"2024-09-16T15:53:48.036138+0000","last_fullsized":"2024-09-16T15:53:48.036138+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:27:17.121069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.038493+0000","last_change":"2024-09-16T15:53:47.067568+0000","last_active":"2024-09-16T15:53:49.038493+0000","last_peered":"2024-09-16T15:53:49.038493+0000","last_clean":"2024-09-16T15:53:49.038493+0000","last_became_active":"2024-09-16T15:53:47.067220+0000","last_became_peered":"2024-09-16T15:53:47.067220+0000","last_unstale":"2024-09-16T15:53:49.038493+0000","last_undegraded":"2024-09-16T15:53:49.038493+0000","last_fullsized":"2024-09-16T15:53:49.038493+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:39:06.152472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.035632+0000","last_change":"2024-09-16T15:53:47.066404+0000","last_active":"2024-09-16T15:53:48.035632+0000","last_peered":"2024-09-16T15:53:48.035632+0000","last_clean":"2024-09-16T15:53:48.035632+0000","last_became_active":"2024-09-16T15:53:47.064379+0000","last_became_peered":"2024-09-16T15:53:47.064379+0000","last_unstale":"2024-09-16T15:53:48.035632+0000","last_undegraded":"2024-09-16T15:53:48.035632+0000","last_fullsized":"2024-09-16T15:53:48.035632+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:57:58.563630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.038351+0000","last_change":"2024-09-16T15:53:47.049576+0000","last_active":"2024-09-16T15:53:49.038351+0000","last_peered":"2024-09-16T15:53:49.038351+0000","last_clean":"2024-09-16T15:53:49.038351+0000","last_became_active":"2024-09-16T15:53:47.049291+0000","last_became_peered":"2024-09-16T15:53:47.049291+0000","last_unstale":"2024-09-16T15:53:49.038351+0000","last_undegraded":"2024-09-16T15:53:49.038351+0000","last_fullsized":"2024-09-16T15:53:49.038351+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:10:04.681002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.038227+0000","last_change":"2024-09-16T15:53:47.054562+0000","last_active":"2024-09-16T15:53:49.038227+0000","last_peered":"2024-09-16T15:53:49.038227+0000","last_clean":"2024-09-16T15:53:49.038227+0000","last_became_active":"2024-09-16T15:53:47.054310+0000","last_became_peered":"2024-09-16T15:53:47.054310+0000","last_unstale":"2024-09-16T15:53:49.038227+0000","last_undegraded":"2024-09-16T15:53:49.038227+0000","last_fullsized":"2024-09-16T15:53:49.038227+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:36:01.782108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.604647+0000","last_change":"2024-09-16T15:53:47.045614+0000","last_active":"2024-09-16T15:53:48.604647+0000","last_peered":"2024-09-16T15:53:48.604647+0000","last_clean":"2024-09-16T15:53:48.604647+0000","last_became_active":"2024-09-16T15:53:47.045131+0000","last_became_peered":"2024-09-16T15:53:47.045131+0000","last_unstale":"2024-09-16T15:53:48.604647+0000","last_undegraded":"2024-09-16T15:53:48.604647+0000","last_fullsized":"2024-09-16T15:53:48.604647+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:25:39.938644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.274853+0000","last_change":"2024-09-16T15:53:47.063974+0000","last_active":"2024-09-16T15:53:49.274853+0000","last_peered":"2024-09-16T15:53:49.274853+0000","last_clean":"2024-09-16T15:53:49.274853+0000","last_became_active":"2024-09-16T15:53:47.063634+0000","last_became_peered":"2024-09-16T15:53:47.063634+0000","last_unstale":"2024-09-16T15:53:49.274853+0000","last_undegraded":"2024-09-16T15:53:49.274853+0000","last_fullsized":"2024-09-16T15:53:49.274853+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:22:44.163439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.036512+0000","last_change":"2024-09-16T15:53:47.045716+0000","last_active":"2024-09-16T15:53:48.036512+0000","last_peered":"2024-09-16T15:53:48.036512+0000","last_clean":"2024-09-16T15:53:48.036512+0000","last_became_active":"2024-09-16T15:53:47.045439+0000","last_became_peered":"2024-09-16T15:53:47.045439+0000","last_unstale":"2024-09-16T15:53:48.036512+0000","last_undegraded":"2024-09-16T15:53:48.036512+0000","last_fullsized":"2024-09-16T15:53:48.036512+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:40:26.604250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.036410+0000","last_change":"2024-09-16T15:53:47.070120+0000","last_active":"2024-09-16T15:53:48.036410+0000","last_peered":"2024-09-16T15:53:48.036410+0000","last_clean":"2024-09-16T15:53:48.036410+0000","last_became_active":"2024-09-16T15:53:47.068816+0000","last_became_peered":"2024-09-16T15:53:47.068816+0000","last_unstale":"2024-09-16T15:53:48.036410+0000","last_undegraded":"2024-09-16T15:53:48.036410+0000","last_fullsized":"2024-09-16T15:53:48.036410+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:32:42.788318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.036127+0000","last_change":"2024-09-16T15:53:47.052483+0000","last_active":"2024-09-16T15:53:48.036127+0000","last_peered":"2024-09-16T15:53:48.036127+0000","last_clean":"2024-09-16T15:53:48.036127+0000","last_became_active":"2024-09-16T15:53:47.052188+0000","last_became_peered":"2024-09-16T15:53:47.052188+0000","last_unstale":"2024-09-16T15:53:48.036127+0000","last_undegraded":"2024-09-16T15:53:48.036127+0000","last_fullsized":"2024-09-16T15:53:48.036127+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:28:45.284824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":10,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-16T15:53:47.058871+0000","last_change":"2024-09-16T15:53:47.056102+0000","last_active":"2024-09-16T15:53:47.058871+0000","last_peered":"2024-09-16T15:53:47.058871+0000","last_clean":"2024-09-16T15:53:47.058871+0000","last_became_active":"2024-09-16T15:53:47.055786+0000","last_became_peered":"2024-09-16T15:53:47.055786+0000","last_unstale":"2024-09-16T15:53:47.058871+0000","last_undegraded":"2024-09-16T15:53:47.058871+0000","last_fullsized":"2024-09-16T15:53:47.058871+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:02:29.501341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.481280+0000","last_change":"2024-09-16T15:53:47.053898+0000","last_active":"2024-09-16T15:53:48.481280+0000","last_peered":"2024-09-16T15:53:48.481280+0000","last_clean":"2024-09-16T15:53:48.481280+0000","last_became_active":"2024-09-16T15:53:47.053585+0000","last_became_peered":"2024-09-16T15:53:47.053585+0000","last_unstale":"2024-09-16T15:53:48.481280+0000","last_undegraded":"2024-09-16T15:53:48.481280+0000","last_fullsized":"2024-09-16T15:53:48.481280+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:00:53.660785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":10,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-16T15:53:47.063272+0000","last_change":"2024-09-16T15:53:47.060703+0000","last_active":"2024-09-16T15:53:47.063272+0000","last_peered":"2024-09-16T15:53:47.063272+0000","last_clean":"2024-09-16T15:53:47.063272+0000","last_became_active":"2024-09-16T15:53:47.060125+0000","last_became_peered":"2024-09-16T15:53:47.060125+0000","last_unstale":"2024-09-16T15:53:47.063272+0000","last_undegraded":"2024-09-16T15:53:47.063272+0000","last_fullsized":"2024-09-16T15:53:47.063272+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:29:02.714478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.604792+0000","last_change":"2024-09-16T15:53:47.047406+0000","last_active":"2024-09-16T15:53:48.604792+0000","last_peered":"2024-09-16T15:53:48.604792+0000","last_clean":"2024-09-16T15:53:48.604792+0000","last_became_active":"2024-09-16T15:53:47.045413+0000","last_became_peered":"2024-09-16T15:53:47.045413+0000","last_unstale":"2024-09-16T15:53:48.604792+0000","last_undegraded":"2024-09-16T15:53:48.604792+0000","last_fullsized":"2024-09-16T15:53:48.604792+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:24:19.371541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.481128+0000","last_change":"2024-09-16T15:53:47.049501+0000","last_active":"2024-09-16T15:53:48.481128+0000","last_peered":"2024-09-16T15:53:48.481128+0000","last_clean":"2024-09-16T15:53:48.481128+0000","last_became_active":"2024-09-16T15:53:47.046606+0000","last_became_peered":"2024-09-16T15:53:47.046606+0000","last_unstale":"2024-09-16T15:53:48.481128+0000","last_undegraded":"2024-09-16T15:53:48.481128+0000","last_fullsized":"2024-09-16T15:53:48.481128+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:00:13.468850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.274676+0000","last_change":"2024-09-16T15:53:47.055350+0000","last_active":"2024-09-16T15:53:49.274676+0000","last_peered":"2024-09-16T15:53:49.274676+0000","last_clean":"2024-09-16T15:53:49.274676+0000","last_became_active":"2024-09-16T15:53:47.051177+0000","last_became_peered":"2024-09-16T15:53:47.051177+0000","last_unstale":"2024-09-16T15:53:49.274676+0000","last_undegraded":"2024-09-16T15:53:49.274676+0000","last_fullsized":"2024-09-16T15:53:49.274676+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:13:16.281809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.481139+0000","last_change":"2024-09-16T15:53:47.065030+0000","last_active":"2024-09-16T15:53:48.481139+0000","last_peered":"2024-09-16T15:53:48.481139+0000","last_clean":"2024-09-16T15:53:48.481139+0000","last_became_active":"2024-09-16T15:53:47.064738+0000","last_became_peered":"2024-09-16T15:53:47.064738+0000","last_unstale":"2024-09-16T15:53:48.481139+0000","last_undegraded":"2024-09-16T15:53:48.481139+0000","last_fullsized":"2024-09-16T15:53:48.481139+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:07:01.165101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.035907+0000","last_change":"2024-09-16T15:53:47.063760+0000","last_active":"2024-09-16T15:53:48.035907+0000","last_peered":"2024-09-16T15:53:48.035907+0000","last_clean":"2024-09-16T15:53:48.035907+0000","last_became_active":"2024-09-16T15:53:47.063404+0000","last_became_peered":"2024-09-16T15:53:47.063404+0000","last_unstale":"2024-09-16T15:53:48.035907+0000","last_undegraded":"2024-09-16T15:53:48.035907+0000","last_fullsized":"2024-09-16T15:53:48.035907+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:30:28.946598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.274350+0000","last_change":"2024-09-16T15:53:47.044151+0000","last_active":"2024-09-16T15:53:49.274350+0000","last_peered":"2024-09-16T15:53:49.274350+0000","last_clean":"2024-09-16T15:53:49.274350+0000","last_became_active":"2024-09-16T15:53:47.043246+0000","last_became_peered":"2024-09-16T15:53:47.043246+0000","last_unstale":"2024-09-16T15:53:49.274350+0000","last_undegraded":"2024-09-16T15:53:49.274350+0000","last_fullsized":"2024-09-16T15:53:49.274350+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:14:07.626424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.273638+0000","last_change":"2024-09-16T15:53:47.051983+0000","last_active":"2024-09-16T15:53:49.273638+0000","last_peered":"2024-09-16T15:53:49.273638+0000","last_clean":"2024-09-16T15:53:49.273638+0000","last_became_active":"2024-09-16T15:53:47.050704+0000","last_became_peered":"2024-09-16T15:53:47.050704+0000","last_unstale":"2024-09-16T15:53:49.273638+0000","last_undegraded":"2024-09-16T15:53:49.273638+0000","last_fullsized":"2024-09-16T15:53:49.273638+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:35:02.303710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.480980+0000","last_change":"2024-09-16T15:53:47.069714+0000","last_active":"2024-09-16T15:53:48.480980+0000","last_peered":"2024-09-16T15:53:48.480980+0000","last_clean":"2024-09-16T15:53:48.480980+0000","last_became_active":"2024-09-16T15:53:47.069413+0000","last_became_peered":"2024-09-16T15:53:47.069413+0000","last_unstale":"2024-09-16T15:53:48.480980+0000","last_undegraded":"2024-09-16T15:53:48.480980+0000","last_fullsized":"2024-09-16T15:53:48.480980+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:27:29.156081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.036803+0000","last_change":"2024-09-16T15:53:47.053885+0000","last_active":"2024-09-16T15:53:48.036803+0000","last_peered":"2024-09-16T15:53:48.036803+0000","last_clean":"2024-09-16T15:53:48.036803+0000","last_became_active":"2024-09-16T15:53:47.052585+0000","last_became_peered":"2024-09-16T15:53:47.052585+0000","last_unstale":"2024-09-16T15:53:48.036803+0000","last_undegraded":"2024-09-16T15:53:48.036803+0000","last_fullsized":"2024-09-16T15:53:48.036803+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:52:32.247179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.275135+0000","last_change":"2024-09-16T15:53:47.047903+0000","last_active":"2024-09-16T15:53:49.275135+0000","last_peered":"2024-09-16T15:53:49.275135+0000","last_clean":"2024-09-16T15:53:49.275135+0000","last_became_active":"2024-09-16T15:53:47.047524+0000","last_became_peered":"2024-09-16T15:53:47.047524+0000","last_unstale":"2024-09-16T15:53:49.275135+0000","last_undegraded":"2024-09-16T15:53:49.275135+0000","last_fullsized":"2024-09-16T15:53:49.275135+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:28:37.082216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":10,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-16T15:53:47.068210+0000","last_change":"2024-09-16T15:53:47.067044+0000","last_active":"2024-09-16T15:53:47.068210+0000","last_peered":"2024-09-16T15:53:47.068210+0000","last_clean":"2024-09-16T15:53:47.068210+0000","last_became_active":"2024-09-16T15:53:47.066772+0000","last_became_peered":"2024-09-16T15:53:47.066772+0000","last_unstale":"2024-09-16T15:53:47.068210+0000","last_undegraded":"2024-09-16T15:53:47.068210+0000","last_fullsized":"2024-09-16T15:53:47.068210+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:27:45.682352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.273899+0000","last_change":"2024-09-16T15:53:47.047353+0000","last_active":"2024-09-16T15:53:49.273899+0000","last_peered":"2024-09-16T15:53:49.273899+0000","last_clean":"2024-09-16T15:53:49.273899+0000","last_became_active":"2024-09-16T15:53:47.047102+0000","last_became_peered":"2024-09-16T15:53:47.047102+0000","last_unstale":"2024-09-16T15:53:49.273899+0000","last_undegraded":"2024-09-16T15:53:49.273899+0000","last_fullsized":"2024-09-16T15:53:49.273899+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:21:20.787684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.274512+0000","last_change":"2024-09-16T15:53:47.046370+0000","last_active":"2024-09-16T15:53:49.274512+0000","last_peered":"2024-09-16T15:53:49.274512+0000","last_clean":"2024-09-16T15:53:49.274512+0000","last_became_active":"2024-09-16T15:53:47.046059+0000","last_became_peered":"2024-09-16T15:53:47.046059+0000","last_unstale":"2024-09-16T15:53:49.274512+0000","last_undegraded":"2024-09-16T15:53:49.274512+0000","last_fullsized":"2024-09-16T15:53:49.274512+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:10:04.681002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.274463+0000","last_change":"2024-09-16T15:53:47.049350+0000","last_active":"2024-09-16T15:53:49.274463+0000","last_peered":"2024-09-16T15:53:49.274463+0000","last_clean":"2024-09-16T15:53:49.274463+0000","last_became_active":"2024-09-16T15:53:47.048759+0000","last_became_peered":"2024-09-16T15:53:47.048759+0000","last_unstale":"2024-09-16T15:53:49.274463+0000","last_undegraded":"2024-09-16T15:53:49.274463+0000","last_fullsized":"2024-09-16T15:53:49.274463+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:36:01.782108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.273712+0000","last_change":"2024-09-16T15:53:47.050603+0000","last_active":"2024-09-16T15:53:49.273712+0000","last_peered":"2024-09-16T15:53:49.273712+0000","last_clean":"2024-09-16T15:53:49.273712+0000","last_became_active":"2024-09-16T15:53:47.050186+0000","last_became_peered":"2024-09-16T15:53:47.050186+0000","last_unstale":"2024-09-16T15:53:49.273712+0000","last_undegraded":"2024-09-16T15:53:49.273712+0000","last_fullsized":"2024-09-16T15:53:49.273712+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:49:30.590331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.604485+0000","last_change":"2024-09-16T15:53:47.049125+0000","last_active":"2024-09-16T15:53:48.604485+0000","last_peered":"2024-09-16T15:53:48.604485+0000","last_clean":"2024-09-16T15:53:48.604485+0000","last_became_active":"2024-09-16T15:53:47.048889+0000","last_became_peered":"2024-09-16T15:53:47.048889+0000","last_unstale":"2024-09-16T15:53:48.604485+0000","last_undegraded":"2024-09-16T15:53:48.604485+0000","last_fullsized":"2024-09-16T15:53:48.604485+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:06:42.055204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":10,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-16T15:53:47.062974+0000","last_change":"2024-09-16T15:53:47.056337+0000","last_active":"2024-09-16T15:53:47.062974+0000","last_peered":"2024-09-16T15:53:47.062974+0000","last_clean":"2024-09-16T15:53:47.062974+0000","last_became_active":"2024-09-16T15:53:47.056039+0000","last_became_peered":"2024-09-16T15:53:47.056039+0000","last_unstale":"2024-09-16T15:53:47.062974+0000","last_undegraded":"2024-09-16T15:53:47.062974+0000","last_fullsized":"2024-09-16T15:53:47.062974+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:12:43.131784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1f","version":"132'4","reported_seq":19,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.038798+0000","last_change":"2024-09-16T15:53:46.049579+0000","last_active":"2024-09-16T15:53:49.038798+0000","last_peered":"2024-09-16T15:53:49.038798+0000","last_clean":"2024-09-16T15:53:49.038798+0000","last_became_active":"2024-09-16T15:53:46.049306+0000","last_became_peered":"2024-09-16T15:53:46.049306+0000","last_unstale":"2024-09-16T15:53:49.038798+0000","last_undegraded":"2024-09-16T15:53:49.038798+0000","last_fullsized":"2024-09-16T15:53:49.038798+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:55:49.783269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.604895+0000","last_change":"2024-09-16T15:53:47.039878+0000","last_active":"2024-09-16T15:53:48.604895+0000","last_peered":"2024-09-16T15:53:48.604895+0000","last_clean":"2024-09-16T15:53:48.604895+0000","last_became_active":"2024-09-16T15:53:47.038999+0000","last_became_peered":"2024-09-16T15:53:47.038999+0000","last_unstale":"2024-09-16T15:53:48.604895+0000","last_undegraded":"2024-09-16T15:53:48.604895+0000","last_fullsized":"2024-09-16T15:53:48.604895+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:34:03.640512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1e","version":"132'1","reported_seq":16,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.604710+0000","last_change":"2024-09-16T15:53:46.055817+0000","last_active":"2024-09-16T15:53:48.604710+0000","last_peered":"2024-09-16T15:53:48.604710+0000","last_clean":"2024-09-16T15:53:48.604710+0000","last_became_active":"2024-09-16T15:53:46.055540+0000","last_became_peered":"2024-09-16T15:53:46.055540+0000","last_unstale":"2024-09-16T15:53:48.604710+0000","last_undegraded":"2024-09-16T15:53:48.604710+0000","last_fullsized":"2024-09-16T15:53:48.604710+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:23:12.275241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.604508+0000","last_change":"2024-09-16T15:53:47.055252+0000","last_active":"2024-09-16T15:53:48.604508+0000","last_peered":"2024-09-16T15:53:48.604508+0000","last_clean":"2024-09-16T15:53:48.604508+0000","last_became_active":"2024-09-16T15:53:47.054974+0000","last_became_peered":"2024-09-16T15:53:47.054974+0000","last_unstale":"2024-09-16T15:53:48.604508+0000","last_undegraded":"2024-09-16T15:53:48.604508+0000","last_fullsized":"2024-09-16T15:53:48.604508+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:54.366435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1d","version":"132'2","reported_seq":17,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.036479+0000","last_change":"2024-09-16T15:53:46.043053+0000","last_active":"2024-09-16T15:53:48.036479+0000","last_peered":"2024-09-16T15:53:48.036479+0000","last_clean":"2024-09-16T15:53:48.036479+0000","last_became_active":"2024-09-16T15:53:46.042758+0000","last_became_peered":"2024-09-16T15:53:46.042758+0000","last_unstale":"2024-09-16T15:53:48.036479+0000","last_undegraded":"2024-09-16T15:53:48.036479+0000","last_fullsized":"2024-09-16T15:53:48.036479+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:49:50.234698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.036593+0000","last_change":"2024-09-16T15:53:47.069273+0000","last_active":"2024-09-16T15:53:48.036593+0000","last_peered":"2024-09-16T15:53:48.036593+0000","last_clean":"2024-09-16T15:53:48.036593+0000","last_became_active":"2024-09-16T15:53:47.067006+0000","last_became_peered":"2024-09-16T15:53:47.067006+0000","last_unstale":"2024-09-16T15:53:48.036593+0000","last_undegraded":"2024-09-16T15:53:48.036593+0000","last_fullsized":"2024-09-16T15:53:48.036593+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:07:19.329442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1c","version":"132'2","reported_seq":17,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.274667+0000","last_change":"2024-09-16T15:53:46.046499+0000","last_active":"2024-09-16T15:53:49.274667+0000","last_peered":"2024-09-16T15:53:49.274667+0000","last_clean":"2024-09-16T15:53:49.274667+0000","last_became_active":"2024-09-16T15:53:46.046246+0000","last_became_peered":"2024-09-16T15:53:46.046246+0000","last_unstale":"2024-09-16T15:53:49.274667+0000","last_undegraded":"2024-09-16T15:53:49.274667+0000","last_fullsized":"2024-09-16T15:53:49.274667+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:07:27.829527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.480951+0000","last_change":"2024-09-16T15:53:47.060948+0000","last_active":"2024-09-16T15:53:48.480951+0000","last_peered":"2024-09-16T15:53:48.480951+0000","last_clean":"2024-09-16T15:53:48.480951+0000","last_became_active":"2024-09-16T15:53:47.060538+0000","last_became_peered":"2024-09-16T15:53:47.060538+0000","last_unstale":"2024-09-16T15:53:48.480951+0000","last_undegraded":"2024-09-16T15:53:48.480951+0000","last_fullsized":"2024-09-16T15:53:48.480951+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:56:18.849300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1b","version":"132'1","reported_seq":16,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.274448+0000","last_change":"2024-09-16T15:53:46.048816+0000","last_active":"2024-09-16T15:53:49.274448+0000","last_peered":"2024-09-16T15:53:49.274448+0000","last_clean":"2024-09-16T15:53:49.274448+0000","last_became_active":"2024-09-16T15:53:46.048416+0000","last_became_peered":"2024-09-16T15:53:46.048416+0000","last_unstale":"2024-09-16T15:53:49.274448+0000","last_undegraded":"2024-09-16T15:53:49.274448+0000","last_fullsized":"2024-09-16T15:53:49.274448+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:13:59.648869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.035988+0000","last_change":"2024-09-16T15:53:47.068600+0000","last_active":"2024-09-16T15:53:48.035988+0000","last_peered":"2024-09-16T15:53:48.035988+0000","last_clean":"2024-09-16T15:53:48.035988+0000","last_became_active":"2024-09-16T15:53:47.066780+0000","last_became_peered":"2024-09-16T15:53:47.066780+0000","last_unstale":"2024-09-16T15:53:48.035988+0000","last_undegraded":"2024-09-16T15:53:48.035988+0000","last_fullsized":"2024-09-16T15:53:48.035988+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:25:28.695527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"132'1","reported_seq":16,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.480731+0000","last_change":"2024-09-16T15:53:46.051615+0000","last_active":"2024-09-16T15:53:48.480731+0000","last_peered":"2024-09-16T15:53:48.480731+0000","last_clean":"2024-09-16T15:53:48.480731+0000","last_became_active":"2024-09-16T15:53:46.051253+0000","last_became_peered":"2024-09-16T15:53:46.051253+0000","last_unstale":"2024-09-16T15:53:48.480731+0000","last_undegraded":"2024-09-16T15:53:48.480731+0000","last_fullsized":"2024-09-16T15:53:48.480731+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:11:54.504207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.035617+0000","last_change":"2024-09-16T15:53:47.063618+0000","last_active":"2024-09-16T15:53:48.035617+0000","last_peered":"2024-09-16T15:53:48.035617+0000","last_clean":"2024-09-16T15:53:48.035617+0000","last_became_active":"2024-09-16T15:53:47.063347+0000","last_became_peered":"2024-09-16T15:53:47.063347+0000","last_unstale":"2024-09-16T15:53:48.035617+0000","last_undegraded":"2024-09-16T15:53:48.035617+0000","last_fullsized":"2024-09-16T15:53:48.035617+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:58:32.257998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.604391+0000","last_change":"2024-09-16T15:53:46.052810+0000","last_active":"2024-09-16T15:53:48.604391+0000","last_peered":"2024-09-16T15:53:48.604391+0000","last_clean":"2024-09-16T15:53:48.604391+0000","last_became_active":"2024-09-16T15:53:46.052461+0000","last_became_peered":"2024-09-16T15:53:46.052461+0000","last_unstale":"2024-09-16T15:53:48.604391+0000","last_undegraded":"2024-09-16T15:53:48.604391+0000","last_fullsized":"2024-09-16T15:53:48.604391+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:36:21.458493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.274292+0000","last_change":"2024-09-16T15:53:47.041379+0000","last_active":"2024-09-16T15:53:49.274292+0000","last_peered":"2024-09-16T15:53:49.274292+0000","last_clean":"2024-09-16T15:53:49.274292+0000","last_became_active":"2024-09-16T15:53:47.041100+0000","last_became_peered":"2024-09-16T15:53:47.041100+0000","last_unstale":"2024-09-16T15:53:49.274292+0000","last_undegraded":"2024-09-16T15:53:49.274292+0000","last_fullsized":"2024-09-16T15:53:49.274292+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:57:10.468784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.274281+0000","last_change":"2024-09-16T15:53:46.050598+0000","last_active":"2024-09-16T15:53:49.274281+0000","last_peered":"2024-09-16T15:53:49.274281+0000","last_clean":"2024-09-16T15:53:49.274281+0000","last_became_active":"2024-09-16T15:53:46.050355+0000","last_became_peered":"2024-09-16T15:53:46.050355+0000","last_unstale":"2024-09-16T15:53:49.274281+0000","last_undegraded":"2024-09-16T15:53:49.274281+0000","last_fullsized":"2024-09-16T15:53:49.274281+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:00:09.267267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.275145+0000","last_change":"2024-09-16T15:53:47.052882+0000","last_active":"2024-09-16T15:53:49.275145+0000","last_peered":"2024-09-16T15:53:49.275145+0000","last_clean":"2024-09-16T15:53:49.275145+0000","last_became_active":"2024-09-16T15:53:47.052537+0000","last_became_peered":"2024-09-16T15:53:47.052537+0000","last_unstale":"2024-09-16T15:53:49.275145+0000","last_undegraded":"2024-09-16T15:53:49.275145+0000","last_fullsized":"2024-09-16T15:53:49.275145+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:04:06.435765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.036663+0000","last_change":"2024-09-16T15:53:46.041207+0000","last_active":"2024-09-16T15:53:48.036663+0000","last_peered":"2024-09-16T15:53:48.036663+0000","last_clean":"2024-09-16T15:53:48.036663+0000","last_became_active":"2024-09-16T15:53:46.040942+0000","last_became_peered":"2024-09-16T15:53:46.040942+0000","last_unstale":"2024-09-16T15:53:48.036663+0000","last_undegraded":"2024-09-16T15:53:48.036663+0000","last_fullsized":"2024-09-16T15:53:48.036663+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:00:03.561191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.038694+0000","last_change":"2024-09-16T15:53:47.054422+0000","last_active":"2024-09-16T15:53:49.038694+0000","last_peered":"2024-09-16T15:53:49.038694+0000","last_clean":"2024-09-16T15:53:49.038694+0000","last_became_active":"2024-09-16T15:53:47.054120+0000","last_became_peered":"2024-09-16T15:53:47.054120+0000","last_unstale":"2024-09-16T15:53:49.038694+0000","last_undegraded":"2024-09-16T15:53:49.038694+0000","last_fullsized":"2024-09-16T15:53:49.038694+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:25:48.625685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.480939+0000","last_change":"2024-09-16T15:53:46.049887+0000","last_active":"2024-09-16T15:53:48.480939+0000","last_peered":"2024-09-16T15:53:48.480939+0000","last_clean":"2024-09-16T15:53:48.480939+0000","last_became_active":"2024-09-16T15:53:46.049554+0000","last_became_peered":"2024-09-16T15:53:46.049554+0000","last_unstale":"2024-09-16T15:53:48.480939+0000","last_undegraded":"2024-09-16T15:53:48.480939+0000","last_fullsized":"2024-09-16T15:53:48.480939+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:39:03.893863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.036446+0000","last_change":"2024-09-16T15:53:47.069807+0000","last_active":"2024-09-16T15:53:48.036446+0000","last_peered":"2024-09-16T15:53:48.036446+0000","last_clean":"2024-09-16T15:53:48.036446+0000","last_became_active":"2024-09-16T15:53:47.068406+0000","last_became_peered":"2024-09-16T15:53:47.068406+0000","last_unstale":"2024-09-16T15:53:48.036446+0000","last_undegraded":"2024-09-16T15:53:48.036446+0000","last_fullsized":"2024-09-16T15:53:48.036446+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:51:00.666807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.15","version":"132'3","reported_seq":18,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.038592+0000","last_change":"2024-09-16T15:53:46.051421+0000","last_active":"2024-09-16T15:53:49.038592+0000","last_peered":"2024-09-16T15:53:49.038592+0000","last_clean":"2024-09-16T15:53:49.038592+0000","last_became_active":"2024-09-16T15:53:46.049074+0000","last_became_peered":"2024-09-16T15:53:46.049074+0000","last_unstale":"2024-09-16T15:53:49.038592+0000","last_undegraded":"2024-09-16T15:53:49.038592+0000","last_fullsized":"2024-09-16T15:53:49.038592+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:03:46.041805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.273880+0000","last_change":"2024-09-16T15:53:47.068386+0000","last_active":"2024-09-16T15:53:49.273880+0000","last_peered":"2024-09-16T15:53:49.273880+0000","last_clean":"2024-09-16T15:53:49.273880+0000","last_became_active":"2024-09-16T15:53:47.067650+0000","last_became_peered":"2024-09-16T15:53:47.067650+0000","last_unstale":"2024-09-16T15:53:49.273880+0000","last_undegraded":"2024-09-16T15:53:49.273880+0000","last_fullsized":"2024-09-16T15:53:49.273880+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:30:31.196430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.14","version":"132'6","reported_seq":21,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.273912+0000","last_change":"2024-09-16T15:53:46.055615+0000","last_active":"2024-09-16T15:53:49.273912+0000","last_peered":"2024-09-16T15:53:49.273912+0000","last_clean":"2024-09-16T15:53:49.273912+0000","last_became_active":"2024-09-16T15:53:46.055155+0000","last_became_peered":"2024-09-16T15:53:46.055155+0000","last_unstale":"2024-09-16T15:53:49.273912+0000","last_undegraded":"2024-09-16T15:53:49.273912+0000","last_fullsized":"2024-09-16T15:53:49.273912+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:43:29.847384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":10,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-16T15:53:47.059010+0000","last_change":"2024-09-16T15:53:47.055028+0000","last_active":"2024-09-16T15:53:47.059010+0000","last_peered":"2024-09-16T15:53:47.059010+0000","last_clean":"2024-09-16T15:53:47.059010+0000","last_became_active":"2024-09-16T15:53:47.054638+0000","last_became_peered":"2024-09-16T15:53:47.054638+0000","last_unstale":"2024-09-16T15:53:47.059010+0000","last_undegraded":"2024-09-16T15:53:47.059010+0000","last_fullsized":"2024-09-16T15:53:47.059010+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:49:07.189417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-16T15:53:47.042905+0000","last_change":"2024-09-16T15:53:46.053325+0000","last_active":"2024-09-16T15:53:47.042905+0000","last_peered":"2024-09-16T15:53:47.042905+0000","last_clean":"2024-09-16T15:53:47.042905+0000","last_became_active":"2024-09-16T15:53:46.051989+0000","last_became_peered":"2024-09-16T15:53:46.051989+0000","last_unstale":"2024-09-16T15:53:47.042905+0000","last_undegraded":"2024-09-16T15:53:47.042905+0000","last_fullsized":"2024-09-16T15:53:47.042905+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:10:52.279943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":10,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-16T15:53:47.063798+0000","last_change":"2024-09-16T15:53:47.062301+0000","last_active":"2024-09-16T15:53:47.063798+0000","last_peered":"2024-09-16T15:53:47.063798+0000","last_clean":"2024-09-16T15:53:47.063798+0000","last_became_active":"2024-09-16T15:53:47.058392+0000","last_became_peered":"2024-09-16T15:53:47.058392+0000","last_unstale":"2024-09-16T15:53:47.063798+0000","last_undegraded":"2024-09-16T15:53:47.063798+0000","last_fullsized":"2024-09-16T15:53:47.063798+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:35:38.397852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.035810+0000","last_change":"2024-09-16T15:53:46.055507+0000","last_active":"2024-09-16T15:53:48.035810+0000","last_peered":"2024-09-16T15:53:48.035810+0000","last_clean":"2024-09-16T15:53:48.035810+0000","last_became_active":"2024-09-16T15:53:46.055247+0000","last_became_peered":"2024-09-16T15:53:46.055247+0000","last_unstale":"2024-09-16T15:53:48.035810+0000","last_undegraded":"2024-09-16T15:53:48.035810+0000","last_fullsized":"2024-09-16T15:53:48.035810+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:26:45.397159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.035599+0000","last_change":"2024-09-16T15:53:47.065045+0000","last_active":"2024-09-16T15:53:48.035599+0000","last_peered":"2024-09-16T15:53:48.035599+0000","last_clean":"2024-09-16T15:53:48.035599+0000","last_became_active":"2024-09-16T15:53:47.064756+0000","last_became_peered":"2024-09-16T15:53:47.064756+0000","last_unstale":"2024-09-16T15:53:48.035599+0000","last_undegraded":"2024-09-16T15:53:48.035599+0000","last_fullsized":"2024-09-16T15:53:48.035599+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:29:02.714478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.10","version":"132'8","reported_seq":23,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.035597+0000","last_change":"2024-09-16T15:53:46.054698+0000","last_active":"2024-09-16T15:53:48.035597+0000","last_peered":"2024-09-16T15:53:48.035597+0000","last_clean":"2024-09-16T15:53:48.035597+0000","last_became_active":"2024-09-16T15:53:46.050527+0000","last_became_peered":"2024-09-16T15:53:46.050527+0000","last_unstale":"2024-09-16T15:53:48.035597+0000","last_undegraded":"2024-09-16T15:53:48.035597+0000","last_fullsized":"2024-09-16T15:53:48.035597+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:29:09.646752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.038576+0000","last_change":"2024-09-16T15:53:47.049220+0000","last_active":"2024-09-16T15:53:49.038576+0000","last_peered":"2024-09-16T15:53:49.038576+0000","last_clean":"2024-09-16T15:53:49.038576+0000","last_became_active":"2024-09-16T15:53:47.048911+0000","last_became_peered":"2024-09-16T15:53:47.048911+0000","last_unstale":"2024-09-16T15:53:49.038576+0000","last_undegraded":"2024-09-16T15:53:49.038576+0000","last_fullsized":"2024-09-16T15:53:49.038576+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:13:16.281809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.f","version":"132'1","reported_seq":16,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.036692+0000","last_change":"2024-09-16T15:53:46.041954+0000","last_active":"2024-09-16T15:53:48.036692+0000","last_peered":"2024-09-16T15:53:48.036692+0000","last_clean":"2024-09-16T15:53:48.036692+0000","last_became_active":"2024-09-16T15:53:46.041272+0000","last_became_peered":"2024-09-16T15:53:46.041272+0000","last_unstale":"2024-09-16T15:53:48.036692+0000","last_undegraded":"2024-09-16T15:53:48.036692+0000","last_fullsized":"2024-09-16T15:53:48.036692+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:30:51.150005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.038733+0000","last_change":"2024-09-16T15:53:47.052264+0000","last_active":"2024-09-16T15:53:49.038733+0000","last_peered":"2024-09-16T15:53:49.038733+0000","last_clean":"2024-09-16T15:53:49.038733+0000","last_became_active":"2024-09-16T15:53:47.051995+0000","last_became_peered":"2024-09-16T15:53:47.051995+0000","last_unstale":"2024-09-16T15:53:49.038733+0000","last_undegraded":"2024-09-16T15:53:49.038733+0000","last_fullsized":"2024-09-16T15:53:49.038733+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:52:00.354801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"132'2","reported_seq":17,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.036582+0000","last_change":"2024-09-16T15:53:46.048066+0000","last_active":"2024-09-16T15:53:48.036582+0000","last_peered":"2024-09-16T15:53:48.036582+0000","last_clean":"2024-09-16T15:53:48.036582+0000","last_became_active":"2024-09-16T15:53:46.047765+0000","last_became_peered":"2024-09-16T15:53:46.047765+0000","last_unstale":"2024-09-16T15:53:48.036582+0000","last_undegraded":"2024-09-16T15:53:48.036582+0000","last_fullsized":"2024-09-16T15:53:48.036582+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:25:07.219846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.274731+0000","last_change":"2024-09-16T15:53:47.054306+0000","last_active":"2024-09-16T15:53:49.274731+0000","last_peered":"2024-09-16T15:53:49.274731+0000","last_clean":"2024-09-16T15:53:49.274731+0000","last_became_active":"2024-09-16T15:53:47.054002+0000","last_became_peered":"2024-09-16T15:53:47.054002+0000","last_unstale":"2024-09-16T15:53:49.274731+0000","last_undegraded":"2024-09-16T15:53:49.274731+0000","last_fullsized":"2024-09-16T15:53:49.274731+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:25:48.625685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.d","version":"132'1","reported_seq":16,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.604918+0000","last_change":"2024-09-16T15:53:46.050413+0000","last_active":"2024-09-16T15:53:48.604918+0000","last_peered":"2024-09-16T15:53:48.604918+0000","last_clean":"2024-09-16T15:53:48.604918+0000","last_became_active":"2024-09-16T15:53:46.050136+0000","last_became_peered":"2024-09-16T15:53:46.050136+0000","last_unstale":"2024-09-16T15:53:48.604918+0000","last_undegraded":"2024-09-16T15:53:48.604918+0000","last_fullsized":"2024-09-16T15:53:48.604918+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:35:42.185986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.274929+0000","last_change":"2024-09-16T15:53:47.055311+0000","last_active":"2024-09-16T15:53:49.274929+0000","last_peered":"2024-09-16T15:53:49.274929+0000","last_clean":"2024-09-16T15:53:49.274929+0000","last_became_active":"2024-09-16T15:53:47.053613+0000","last_became_peered":"2024-09-16T15:53:47.053613+0000","last_unstale":"2024-09-16T15:53:49.274929+0000","last_undegraded":"2024-09-16T15:53:49.274929+0000","last_fullsized":"2024-09-16T15:53:49.274929+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:52:21.045751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.038449+0000","last_change":"2024-09-16T15:53:46.052237+0000","last_active":"2024-09-16T15:53:49.038449+0000","last_peered":"2024-09-16T15:53:49.038449+0000","last_clean":"2024-09-16T15:53:49.038449+0000","last_became_active":"2024-09-16T15:53:46.051939+0000","last_became_peered":"2024-09-16T15:53:46.051939+0000","last_unstale":"2024-09-16T15:53:49.038449+0000","last_undegraded":"2024-09-16T15:53:49.038449+0000","last_fullsized":"2024-09-16T15:53:49.038449+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:43:15.863610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.274616+0000","last_change":"2024-09-16T15:53:47.055016+0000","last_active":"2024-09-16T15:53:49.274616+0000","last_peered":"2024-09-16T15:53:49.274616+0000","last_clean":"2024-09-16T15:53:49.274616+0000","last_became_active":"2024-09-16T15:53:47.054029+0000","last_became_peered":"2024-09-16T15:53:47.054029+0000","last_unstale":"2024-09-16T15:53:49.274616+0000","last_undegraded":"2024-09-16T15:53:49.274616+0000","last_fullsized":"2024-09-16T15:53:49.274616+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:39:06.152472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.b","version":"132'2","reported_seq":17,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.604585+0000","last_change":"2024-09-16T15:53:46.049709+0000","last_active":"2024-09-16T15:53:48.604585+0000","last_peered":"2024-09-16T15:53:48.604585+0000","last_clean":"2024-09-16T15:53:48.604585+0000","last_became_active":"2024-09-16T15:53:46.049412+0000","last_became_peered":"2024-09-16T15:53:46.049412+0000","last_unstale":"2024-09-16T15:53:48.604585+0000","last_undegraded":"2024-09-16T15:53:48.604585+0000","last_fullsized":"2024-09-16T15:53:48.604585+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:35:48.612183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.273696+0000","last_change":"2024-09-16T15:53:47.052068+0000","last_active":"2024-09-16T15:53:49.273696+0000","last_peered":"2024-09-16T15:53:49.273696+0000","last_clean":"2024-09-16T15:53:49.273696+0000","last_became_active":"2024-09-16T15:53:47.051755+0000","last_became_peered":"2024-09-16T15:53:47.051755+0000","last_unstale":"2024-09-16T15:53:49.273696+0000","last_undegraded":"2024-09-16T15:53:49.273696+0000","last_fullsized":"2024-09-16T15:53:49.273696+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:30:26.898180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.481002+0000","last_change":"2024-09-16T15:53:46.051522+0000","last_active":"2024-09-16T15:53:48.481002+0000","last_peered":"2024-09-16T15:53:48.481002+0000","last_clean":"2024-09-16T15:53:48.481002+0000","last_became_active":"2024-09-16T15:53:46.051005+0000","last_became_peered":"2024-09-16T15:53:46.051005+0000","last_unstale":"2024-09-16T15:53:48.481002+0000","last_undegraded":"2024-09-16T15:53:48.481002+0000","last_fullsized":"2024-09-16T15:53:48.481002+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:10:36.900762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":10,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-16T15:53:47.063249+0000","last_change":"2024-09-16T15:53:47.058656+0000","last_active":"2024-09-16T15:53:47.063249+0000","last_peered":"2024-09-16T15:53:47.063249+0000","last_clean":"2024-09-16T15:53:47.063249+0000","last_became_active":"2024-09-16T15:53:47.057260+0000","last_became_peered":"2024-09-16T15:53:47.057260+0000","last_unstale":"2024-09-16T15:53:47.063249+0000","last_undegraded":"2024-09-16T15:53:47.063249+0000","last_fullsized":"2024-09-16T15:53:47.063249+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:18:36.519474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-16T15:53:47.041902+0000","last_change":"2024-09-16T15:53:46.038714+0000","last_active":"2024-09-16T15:53:47.041902+0000","last_peered":"2024-09-16T15:53:47.041902+0000","last_clean":"2024-09-16T15:53:47.041902+0000","last_became_active":"2024-09-16T15:53:46.038411+0000","last_became_peered":"2024-09-16T15:53:46.038411+0000","last_unstale":"2024-09-16T15:53:47.041902+0000","last_undegraded":"2024-09-16T15:53:47.041902+0000","last_fullsized":"2024-09-16T15:53:47.041902+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:43:20.313472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.035857+0000","last_change":"2024-09-16T15:53:47.065949+0000","last_active":"2024-09-16T15:53:48.035857+0000","last_peered":"2024-09-16T15:53:48.035857+0000","last_clean":"2024-09-16T15:53:48.035857+0000","last_became_active":"2024-09-16T15:53:47.064802+0000","last_became_peered":"2024-09-16T15:53:47.064802+0000","last_unstale":"2024-09-16T15:53:48.035857+0000","last_undegraded":"2024-09-16T15:53:48.035857+0000","last_fullsized":"2024-09-16T15:53:48.035857+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:09:34.576069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.035763+0000","last_change":"2024-09-16T15:53:46.049975+0000","last_active":"2024-09-16T15:53:48.035763+0000","last_peered":"2024-09-16T15:53:48.035763+0000","last_clean":"2024-09-16T15:53:48.035763+0000","last_became_active":"2024-09-16T15:53:46.049667+0000","last_became_peered":"2024-09-16T15:53:46.049667+0000","last_unstale":"2024-09-16T15:53:48.035763+0000","last_undegraded":"2024-09-16T15:53:48.035763+0000","last_fullsized":"2024-09-16T15:53:48.035763+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:50:23.146038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.273923+0000","last_change":"2024-09-16T15:53:47.066513+0000","last_active":"2024-09-16T15:53:49.273923+0000","last_peered":"2024-09-16T15:53:49.273923+0000","last_clean":"2024-09-16T15:53:49.273923+0000","last_became_active":"2024-09-16T15:53:47.066252+0000","last_became_peered":"2024-09-16T15:53:47.066252+0000","last_unstale":"2024-09-16T15:53:49.273923+0000","last_undegraded":"2024-09-16T15:53:49.273923+0000","last_fullsized":"2024-09-16T15:53:49.273923+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:18:41.717848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.7","version":"132'2","reported_seq":17,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.480957+0000","last_change":"2024-09-16T15:53:46.051198+0000","last_active":"2024-09-16T15:53:48.480957+0000","last_peered":"2024-09-16T15:53:48.480957+0000","last_clean":"2024-09-16T15:53:48.480957+0000","last_became_active":"2024-09-16T15:53:46.050837+0000","last_became_peered":"2024-09-16T15:53:46.050837+0000","last_unstale":"2024-09-16T15:53:48.480957+0000","last_undegraded":"2024-09-16T15:53:48.480957+0000","last_fullsized":"2024-09-16T15:53:48.480957+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:38:17.885868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.274984+0000","last_change":"2024-09-16T15:53:47.060499+0000","last_active":"2024-09-16T15:53:49.274984+0000","last_peered":"2024-09-16T15:53:49.274984+0000","last_clean":"2024-09-16T15:53:49.274984+0000","last_became_active":"2024-09-16T15:53:47.059699+0000","last_became_peered":"2024-09-16T15:53:47.059699+0000","last_unstale":"2024-09-16T15:53:49.274984+0000","last_undegraded":"2024-09-16T15:53:49.274984+0000","last_fullsized":"2024-09-16T15:53:49.274984+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:35:38.397852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.6","version":"133'1","reported_seq":16,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.038798+0000","last_change":"2024-09-16T15:53:46.055909+0000","last_active":"2024-09-16T15:53:49.038798+0000","last_peered":"2024-09-16T15:53:49.038798+0000","last_clean":"2024-09-16T15:53:49.038798+0000","last_became_active":"2024-09-16T15:53:46.055567+0000","last_became_peered":"2024-09-16T15:53:46.055567+0000","last_unstale":"2024-09-16T15:53:49.038798+0000","last_undegraded":"2024-09-16T15:53:49.038798+0000","last_fullsized":"2024-09-16T15:53:49.038798+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:27:29.536408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":10,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-16T15:53:47.058902+0000","last_change":"2024-09-16T15:53:47.055081+0000","last_active":"2024-09-16T15:53:47.058902+0000","last_peered":"2024-09-16T15:53:47.058902+0000","last_clean":"2024-09-16T15:53:47.058902+0000","last_became_active":"2024-09-16T15:53:47.054772+0000","last_became_peered":"2024-09-16T15:53:47.054772+0000","last_unstale":"2024-09-16T15:53:47.058902+0000","last_undegraded":"2024-09-16T15:53:47.058902+0000","last_fullsized":"2024-09-16T15:53:47.058902+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:36:50.843181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.481141+0000","last_change":"2024-09-16T15:53:46.056629+0000","last_active":"2024-09-16T15:53:48.481141+0000","last_peered":"2024-09-16T15:53:48.481141+0000","last_clean":"2024-09-16T15:53:48.481141+0000","last_became_active":"2024-09-16T15:53:46.056360+0000","last_became_peered":"2024-09-16T15:53:46.056360+0000","last_unstale":"2024-09-16T15:53:48.481141+0000","last_undegraded":"2024-09-16T15:53:48.481141+0000","last_fullsized":"2024-09-16T15:53:48.481141+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:57:33.513974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.604813+0000","last_change":"2024-09-16T15:53:47.048289+0000","last_active":"2024-09-16T15:53:48.604813+0000","last_peered":"2024-09-16T15:53:48.604813+0000","last_clean":"2024-09-16T15:53:48.604813+0000","last_became_active":"2024-09-16T15:53:47.047818+0000","last_became_peered":"2024-09-16T15:53:47.047818+0000","last_unstale":"2024-09-16T15:53:48.604813+0000","last_undegraded":"2024-09-16T15:53:48.604813+0000","last_fullsized":"2024-09-16T15:53:48.604813+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:52:33.800592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-16T15:53:47.042871+0000","last_change":"2024-09-16T15:53:46.057982+0000","last_active":"2024-09-16T15:53:47.042871+0000","last_peered":"2024-09-16T15:53:47.042871+0000","last_clean":"2024-09-16T15:53:47.042871+0000","last_became_active":"2024-09-16T15:53:46.057707+0000","last_became_peered":"2024-09-16T15:53:46.057707+0000","last_unstale":"2024-09-16T15:53:47.042871+0000","last_undegraded":"2024-09-16T15:53:47.042871+0000","last_fullsized":"2024-09-16T15:53:47.042871+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:59:40.544624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.035623+0000","last_change":"2024-09-16T15:53:47.062803+0000","last_active":"2024-09-16T15:53:48.035623+0000","last_peered":"2024-09-16T15:53:48.035623+0000","last_clean":"2024-09-16T15:53:48.035623+0000","last_became_active":"2024-09-16T15:53:47.060237+0000","last_became_peered":"2024-09-16T15:53:47.060237+0000","last_unstale":"2024-09-16T15:53:48.035623+0000","last_undegraded":"2024-09-16T15:53:48.035623+0000","last_fullsized":"2024-09-16T15:53:48.035623+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:52:21.045751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.274595+0000","last_change":"2024-09-16T15:53:46.052804+0000","last_active":"2024-09-16T15:53:49.274595+0000","last_peered":"2024-09-16T15:53:49.274595+0000","last_clean":"2024-09-16T15:53:49.274595+0000","last_became_active":"2024-09-16T15:53:46.052460+0000","last_became_peered":"2024-09-16T15:53:46.052460+0000","last_unstale":"2024-09-16T15:53:49.274595+0000","last_undegraded":"2024-09-16T15:53:49.274595+0000","last_fullsized":"2024-09-16T15:53:49.274595+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:36:04.003482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.036048+0000","last_change":"2024-09-16T15:53:47.065967+0000","last_active":"2024-09-16T15:53:48.036048+0000","last_peered":"2024-09-16T15:53:48.036048+0000","last_clean":"2024-09-16T15:53:48.036048+0000","last_became_active":"2024-09-16T15:53:47.065715+0000","last_became_peered":"2024-09-16T15:53:47.065715+0000","last_unstale":"2024-09-16T15:53:48.036048+0000","last_undegraded":"2024-09-16T15:53:48.036048+0000","last_fullsized":"2024-09-16T15:53:48.036048+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:48:44.181946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.035831+0000","last_change":"2024-09-16T15:53:46.044167+0000","last_active":"2024-09-16T15:53:48.035831+0000","last_peered":"2024-09-16T15:53:48.035831+0000","last_clean":"2024-09-16T15:53:48.035831+0000","last_became_active":"2024-09-16T15:53:46.043914+0000","last_became_peered":"2024-09-16T15:53:46.043914+0000","last_unstale":"2024-09-16T15:53:48.035831+0000","last_undegraded":"2024-09-16T15:53:48.035831+0000","last_fullsized":"2024-09-16T15:53:48.035831+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:20:21.174844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.481211+0000","last_change":"2024-09-16T15:53:47.069960+0000","last_active":"2024-09-16T15:53:48.481211+0000","last_peered":"2024-09-16T15:53:48.481211+0000","last_clean":"2024-09-16T15:53:48.481211+0000","last_became_active":"2024-09-16T15:53:47.069614+0000","last_became_peered":"2024-09-16T15:53:47.069614+0000","last_unstale":"2024-09-16T15:53:48.481211+0000","last_undegraded":"2024-09-16T15:53:48.481211+0000","last_fullsized":"2024-09-16T15:53:48.481211+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:25:21.431101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.035927+0000","last_change":"2024-09-16T15:53:46.055377+0000","last_active":"2024-09-16T15:53:48.035927+0000","last_peered":"2024-09-16T15:53:48.035927+0000","last_clean":"2024-09-16T15:53:48.035927+0000","last_became_active":"2024-09-16T15:53:46.055135+0000","last_became_peered":"2024-09-16T15:53:46.055135+0000","last_unstale":"2024-09-16T15:53:48.035927+0000","last_undegraded":"2024-09-16T15:53:48.035927+0000","last_fullsized":"2024-09-16T15:53:48.035927+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:04:47.485315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":10,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-16T15:53:47.062177+0000","last_change":"2024-09-16T15:53:47.057817+0000","last_active":"2024-09-16T15:53:47.062177+0000","last_peered":"2024-09-16T15:53:47.062177+0000","last_clean":"2024-09-16T15:53:47.062177+0000","last_became_active":"2024-09-16T15:53:47.057440+0000","last_became_peered":"2024-09-16T15:53:47.057440+0000","last_unstale":"2024-09-16T15:53:47.062177+0000","last_undegraded":"2024-09-16T15:53:47.062177+0000","last_fullsized":"2024-09-16T15:53:47.062177+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:56:27.484880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-16T15:53:47.043225+0000","last_change":"2024-09-16T15:53:46.047710+0000","last_active":"2024-09-16T15:53:47.043225+0000","last_peered":"2024-09-16T15:53:47.043225+0000","last_clean":"2024-09-16T15:53:47.043225+0000","last_became_active":"2024-09-16T15:53:46.047320+0000","last_became_peered":"2024-09-16T15:53:46.047320+0000","last_unstale":"2024-09-16T15:53:47.043225+0000","last_undegraded":"2024-09-16T15:53:47.043225+0000","last_fullsized":"2024-09-16T15:53:47.043225+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:40:37.494885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.604526+0000","last_change":"2024-09-16T15:53:47.039385+0000","last_active":"2024-09-16T15:53:48.604526+0000","last_peered":"2024-09-16T15:53:48.604526+0000","last_clean":"2024-09-16T15:53:48.604526+0000","last_became_active":"2024-09-16T15:53:47.039077+0000","last_became_peered":"2024-09-16T15:53:47.039077+0000","last_unstale":"2024-09-16T15:53:48.604526+0000","last_undegraded":"2024-09-16T15:53:48.604526+0000","last_fullsized":"2024-09-16T15:53:48.604526+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:30:30.250921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.13","version":"132'2","reported_seq":17,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.036176+0000","last_change":"2024-09-16T15:53:46.048215+0000","last_active":"2024-09-16T15:53:48.036176+0000","last_peered":"2024-09-16T15:53:48.036176+0000","last_clean":"2024-09-16T15:53:48.036176+0000","last_became_active":"2024-09-16T15:53:46.045488+0000","last_became_peered":"2024-09-16T15:53:46.045488+0000","last_unstale":"2024-09-16T15:53:48.036176+0000","last_undegraded":"2024-09-16T15:53:48.036176+0000","last_fullsized":"2024-09-16T15:53:48.036176+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:59:52.882286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":323,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.273491+0000","last_change":"2024-09-16T15:39:12.698813+0000","last_active":"2024-09-16T15:53:49.273491+0000","last_peered":"2024-09-16T15:53:49.273491+0000","last_clean":"2024-09-16T15:53:49.273491+0000","last_became_active":"2024-09-16T15:39:12.697695+0000","last_became_peered":"2024-09-16T15:39:12.697695+0000","last_unstale":"2024-09-16T15:53:49.273491+0000","last_undegraded":"2024-09-16T15:53:49.273491+0000","last_fullsized":"2024-09-16T15:53:49.273491+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_clean_scrub_stamp":"2024-09-16T15:39:11.684967+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:02.061081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509620,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":159068,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":157438,"kb_avail":94212772,"statfs":{"total":96636764160,"available":96473878528,"internally_reserved":0,"allocated":1638400,"data_stored":723164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":161216968},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509620,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":119408,"kb_used_data":2204,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94252432,"statfs":{"total":96636764160,"available":96514490368,"internally_reserved":0,"allocated":2256896,"data_stored":1320686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":119994823},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509619,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":136864,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94234976,"statfs":{"total":96636764160,"available":96496615424,"internally_reserved":0,"allocated":1650688,"data_stored":729576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":138475973},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509619,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":118820,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94253020,"statfs":{"total":96636764160,"available":96515092480,"internally_reserved":0,"allocated":1654784,"data_stored":729646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":119994825},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509619,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131308,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240532,"statfs":{"total":96636764160,"available":96502304768,"internally_reserved":0,"allocated":1658880,"data_stored":730248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509620,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131892,"kb_used_data":2208,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94239948,"statfs":{"total":96636764160,"available":96501706752,"internally_reserved":0,"allocated":2260992,"data_stored":1321578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509618,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149364,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222476,"statfs":{"total":96636764160,"available":96483815424,"internally_reserved":0,"allocated":1671168,"data_stored":731244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":151255495},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542325,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131304,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240536,"statfs":{"total":96636764160,"available":96502308864,"internally_reserved":0,"allocated":1654784,"data_stored":729610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":132774348},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:53:52.108 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T15:53:52.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.299+0000 7f1403200640 1 -- 172.21.15.151:0/1127891967 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14040d1260 msgr2=0x7f14040cf9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:52.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.300+0000 7f1403200640 1 --2- 172.21.15.151:0/1127891967 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14040d1260 0x7f14040cf9b0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f13fc005fd0 tx=0x7f13fc02a940 comp rx=0 tx=0).stop 2024-09-16T15:53:52.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.300+0000 7f1403200640 1 -- 172.21.15.151:0/1127891967 shutdown_connections 2024-09-16T15:53:52.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.300+0000 7f1403200640 1 --2- 172.21.15.151:0/1127891967 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14040d1260 0x7f14040cf9b0 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:52.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.300+0000 7f1403200640 1 -- 172.21.15.151:0/1127891967 >> 172.21.15.151:0/1127891967 conn(0x7f14040d3e40 msgr2=0x7f14040d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:52.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.300+0000 7f1403200640 1 -- 172.21.15.151:0/1127891967 shutdown_connections 2024-09-16T15:53:52.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.300+0000 7f1403200640 1 -- 172.21.15.151:0/1127891967 wait complete. 2024-09-16T15:53:52.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.301+0000 7f1403200640 1 Processor -- start 2024-09-16T15:53:52.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.301+0000 7f1403200640 1 -- start start 2024-09-16T15:53:52.304 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.302+0000 7f1403200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14040d1260 0x7f140415c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:52.304 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.302+0000 7f1403200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14040d25d0 0x7f14041f4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:52.304 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.302+0000 7f1403200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f140415e050 0x7f14040ca050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:52.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.302+0000 7f1403200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f14040b7380 con 0x7f14040d1260 2024-09-16T15:53:52.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.302+0000 7f1403200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f14040d4d10 con 0x7f140415e050 2024-09-16T15:53:52.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.302+0000 7f1403200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f14040b7500 con 0x7f14040d25d0 2024-09-16T15:53:52.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.302+0000 7f1401e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14040d1260 0x7f140415c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:52.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.303+0000 7f1401e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14040d1260 0x7f140415c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43790/0 (socket says 172.21.15.151:43790) 2024-09-16T15:53:52.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.303+0000 7f1401e00640 1 -- 172.21.15.151:0/1224434108 learned_addr learned my addr 172.21.15.151:0/1224434108 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:53:52.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.303+0000 7f1402800640 1 --2- 172.21.15.151:0/1224434108 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f140415e050 0x7f14040ca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:52.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.303+0000 7f1401400640 1 --2- 172.21.15.151:0/1224434108 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14040d25d0 0x7f14041f4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:52.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.303+0000 7f1401e00640 1 -- 172.21.15.151:0/1224434108 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14040d25d0 msgr2=0x7f14041f4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:52.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.303+0000 7f1401e00640 1 --2- 172.21.15.151:0/1224434108 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14040d25d0 0x7f14041f4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:52.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.303+0000 7f1401e00640 1 -- 172.21.15.151:0/1224434108 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f140415e050 msgr2=0x7f14040ca050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:52.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.303+0000 7f1401e00640 1 --2- 172.21.15.151:0/1224434108 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f140415e050 0x7f14040ca050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:52.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.303+0000 7f1401e00640 1 -- 172.21.15.151:0/1224434108 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14041f4f60 con 0x7f14040d1260 2024-09-16T15:53:52.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.303+0000 7f1402800640 1 --2- 172.21.15.151:0/1224434108 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f140415e050 0x7f14040ca050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:53:52.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.303+0000 7f1401400640 1 --2- 172.21.15.151:0/1224434108 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14040d25d0 0x7f14041f4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:53:52.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.303+0000 7f1401e00640 1 --2- 172.21.15.151:0/1224434108 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14040d1260 0x7f140415c820 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f13f400c400 tx=0x7f13f400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:52.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.303+0000 7f13fa000640 1 -- 172.21.15.151:0/1224434108 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f4017070 con 0x7f14040d1260 2024-09-16T15:53:52.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.304+0000 7f13fa000640 1 -- 172.21.15.151:0/1224434108 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f13f40053c0 con 0x7f14040d1260 2024-09-16T15:53:52.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.304+0000 7f13fa000640 1 -- 172.21.15.151:0/1224434108 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f4003850 con 0x7f14040d1260 2024-09-16T15:53:52.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.304+0000 7f1403200640 1 -- 172.21.15.151:0/1224434108 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14041f5240 con 0x7f14040d1260 2024-09-16T15:53:52.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.304+0000 7f1403200640 1 -- 172.21.15.151:0/1224434108 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f140415cd60 con 0x7f14040d1260 2024-09-16T15:53:52.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.305+0000 7f1403200640 1 -- 172.21.15.151:0/1224434108 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14041f5480 con 0x7f14040d1260 2024-09-16T15:53:52.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.306+0000 7f13fa000640 1 -- 172.21.15.151:0/1224434108 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 47) ==== 104092+0+0 (secure 0 0 0) 0x7f13f401a070 con 0x7f14040d1260 2024-09-16T15:53:52.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.308+0000 7f13fa000640 1 --2- 172.21.15.151:0/1224434108 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f13e407a740 0x7f13e407cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:52.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.308+0000 7f13fa000640 1 -- 172.21.15.151:0/1224434108 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5821+0+0 (secure 0 0 0) 0x7f13f4098cd0 con 0x7f14040d1260 2024-09-16T15:53:52.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.308+0000 7f1401400640 1 --2- 172.21.15.151:0/1224434108 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f13e407a740 0x7f13e407cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:52.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.309+0000 7f1401400640 1 --2- 172.21.15.151:0/1224434108 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f13e407a740 0x7f13e407cc00 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f14040b92a0 tx=0x7f13f0009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:52.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.312+0000 7f13fa000640 1 -- 172.21.15.151:0/1224434108 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f14041f5480 con 0x7f14040d1260 2024-09-16T15:53:52.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.567+0000 7f1403200640 1 -- 172.21.15.151:0/1224434108 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f14040b79b0 con 0x7f14040d1260 2024-09-16T15:53:52.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.567+0000 7f13fa000640 1 -- 172.21.15.151:0/1224434108 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1058 (secure 0 0 0) 0x7f13f4062da0 con 0x7f14040d1260 2024-09-16T15:53:52.569 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:53:52.569 INFO:teuthology.orchestra.run.smithi151.stdout:{"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":896,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":133,"num_osds":8,"num_up_osds":8,"osd_up_since":1726501150,"num_in_osds":8,"osd_in_since":1726501140,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":1103900672,"bytes_avail":771990212608,"bytes_total":773094113280,"write_bytes_sec":2388,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":87,"btime":"2024-09-16T15:53:50:060393+0000","id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"a","status":"up:active","gid":9009}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":18,"modified":"2024-09-16T15:53:25.931213+0000","services":{}},"progress_events":{}} 2024-09-16T15:53:52.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.569+0000 7f1403200640 1 -- 172.21.15.151:0/1224434108 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f13e407a740 msgr2=0x7f13e407cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:52.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.569+0000 7f1403200640 1 --2- 172.21.15.151:0/1224434108 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f13e407a740 0x7f13e407cc00 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f14040b92a0 tx=0x7f13f0009040 comp rx=0 tx=0).stop 2024-09-16T15:53:52.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.569+0000 7f1403200640 1 -- 172.21.15.151:0/1224434108 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14040d1260 msgr2=0x7f140415c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:52.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.569+0000 7f1403200640 1 --2- 172.21.15.151:0/1224434108 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14040d1260 0x7f140415c820 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f13f400c400 tx=0x7f13f400c8c0 comp rx=0 tx=0).stop 2024-09-16T15:53:52.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.569+0000 7f1403200640 1 -- 172.21.15.151:0/1224434108 shutdown_connections 2024-09-16T15:53:52.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.569+0000 7f1403200640 1 --2- 172.21.15.151:0/1224434108 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f13e407a740 0x7f13e407cc00 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:52.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.569+0000 7f1403200640 1 --2- 172.21.15.151:0/1224434108 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f140415e050 0x7f14040ca050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:52.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.569+0000 7f1403200640 1 --2- 172.21.15.151:0/1224434108 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14040d25d0 0x7f14041f4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:52.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.569+0000 7f1403200640 1 --2- 172.21.15.151:0/1224434108 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14040d1260 0x7f140415c820 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:52.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.569+0000 7f1403200640 1 -- 172.21.15.151:0/1224434108 >> 172.21.15.151:0/1224434108 conn(0x7f14040d3e40 msgr2=0x7f14040b6c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:52.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.570+0000 7f1403200640 1 -- 172.21.15.151:0/1224434108 shutdown_connections 2024-09-16T15:53:52.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.570+0000 7f1403200640 1 -- 172.21.15.151:0/1224434108 wait complete. 2024-09-16T15:53:52.588 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-16T15:53:52.588 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:53:52.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.743+0000 7efefb200640 1 -- 172.21.15.151:0/3701307638 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efef4072df0 msgr2=0x7efef4073240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:52.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.743+0000 7efefb200640 1 --2- 172.21.15.151:0/3701307638 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efef4072df0 0x7efef4073240 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7efef40b4890 tx=0x7efeec02a4b0 comp rx=0 tx=0).stop 2024-09-16T15:53:52.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.743+0000 7efefb200640 1 -- 172.21.15.151:0/3701307638 shutdown_connections 2024-09-16T15:53:52.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.743+0000 7efefb200640 1 --2- 172.21.15.151:0/3701307638 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efef4072df0 0x7efef4073240 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:52.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.743+0000 7efefb200640 1 --2- 172.21.15.151:0/3701307638 >> v2:172.21.15.167:3300/0 conn(0x7efef40af3d0 0x7efef406f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:52.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.743+0000 7efefb200640 1 -- 172.21.15.151:0/3701307638 >> 172.21.15.151:0/3701307638 conn(0x7efef40b1950 msgr2=0x7efef40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:52.745 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.743+0000 7efefb200640 1 -- 172.21.15.151:0/3701307638 shutdown_connections 2024-09-16T15:53:52.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.743+0000 7efefb200640 1 -- 172.21.15.151:0/3701307638 wait complete. 2024-09-16T15:53:52.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.744+0000 7efefb200640 1 Processor -- start 2024-09-16T15:53:52.746 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.745+0000 7efefb200640 1 -- start start 2024-09-16T15:53:52.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.745+0000 7efefb200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efef4072df0 0x7efef41540c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:52.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.745+0000 7efefb200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efef40af3d0 0x7efef409d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:52.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.745+0000 7efefb200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efef4161550 0x7efef409d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:52.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.745+0000 7efefb200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efef4073780 con 0x7efef4072df0 2024-09-16T15:53:52.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.745+0000 7efefb200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efef406fdb0 con 0x7efef40af3d0 2024-09-16T15:53:52.748 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.745+0000 7efefb200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7efef4073900 con 0x7efef4161550 2024-09-16T15:53:52.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.746+0000 7efef3e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efef4072df0 0x7efef41540c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:52.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.746+0000 7efef3e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efef4072df0 0x7efef41540c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43810/0 (socket says 172.21.15.151:43810) 2024-09-16T15:53:52.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.746+0000 7efef3e00640 1 -- 172.21.15.151:0/1761571131 learned_addr learned my addr 172.21.15.151:0/1761571131 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:53:52.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.746+0000 7efef3400640 1 --2- 172.21.15.151:0/1761571131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efef40af3d0 0x7efef409d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:52.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.746+0000 7efef9200640 1 --2- 172.21.15.151:0/1761571131 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efef4161550 0x7efef409d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:52.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.746+0000 7efef3e00640 1 -- 172.21.15.151:0/1761571131 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efef4161550 msgr2=0x7efef409d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:52.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.746+0000 7efef3e00640 1 --2- 172.21.15.151:0/1761571131 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efef4161550 0x7efef409d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:52.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.746+0000 7efef3e00640 1 -- 172.21.15.151:0/1761571131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efef40af3d0 msgr2=0x7efef409d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:52.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.746+0000 7efef3e00640 1 --2- 172.21.15.151:0/1761571131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efef40af3d0 0x7efef409d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:52.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.746+0000 7efef3e00640 1 -- 172.21.15.151:0/1761571131 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efef4154600 con 0x7efef4072df0 2024-09-16T15:53:52.749 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.746+0000 7efef3e00640 1 --2- 172.21.15.151:0/1761571131 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efef4072df0 0x7efef41540c0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7efee400ba50 tx=0x7efee400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:52.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.747+0000 7efeebe00640 1 -- 172.21.15.151:0/1761571131 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee4019070 con 0x7efef4072df0 2024-09-16T15:53:52.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.747+0000 7efeebe00640 1 -- 172.21.15.151:0/1761571131 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efee400cd10 con 0x7efef4072df0 2024-09-16T15:53:52.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.747+0000 7efeebe00640 1 -- 172.21.15.151:0/1761571131 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee40146a0 con 0x7efef4072df0 2024-09-16T15:53:52.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.747+0000 7efefb200640 1 -- 172.21.15.151:0/1761571131 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efef41f4b10 con 0x7efef4072df0 2024-09-16T15:53:52.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.747+0000 7efefb200640 1 -- 172.21.15.151:0/1761571131 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efef41f5070 con 0x7efef4072df0 2024-09-16T15:53:52.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.748+0000 7efefb200640 1 -- 172.21.15.151:0/1761571131 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efef4073df0 con 0x7efef4072df0 2024-09-16T15:53:52.751 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.749+0000 7efeebe00640 1 -- 172.21.15.151:0/1761571131 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 47) ==== 104092+0+0 (secure 0 0 0) 0x7efee401c070 con 0x7efef4072df0 2024-09-16T15:53:52.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.750+0000 7efeebe00640 1 --2- 172.21.15.151:0/1761571131 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efed407a740 0x7efed407cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:52.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.750+0000 7efeebe00640 1 -- 172.21.15.151:0/1761571131 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5821+0+0 (secure 0 0 0) 0x7efee40995f0 con 0x7efef4072df0 2024-09-16T15:53:52.752 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.750+0000 7efef3400640 1 --2- 172.21.15.151:0/1761571131 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efed407a740 0x7efed407cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:52.753 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.751+0000 7efef3400640 1 --2- 172.21.15.151:0/1761571131 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efed407a740 0x7efed407cc00 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7efee0005fd0 tx=0x7efee0005850 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:52.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.754+0000 7efeebe00640 1 -- 172.21.15.151:0/1761571131 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7efef41f4b10 con 0x7efef4072df0 2024-09-16T15:53:52.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.936+0000 7efefb200640 1 -- 172.21.15.151:0/1761571131 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efef4154e40 con 0x7efed407a740 2024-09-16T15:53:52.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.942+0000 7efeebe00640 1 -- 172.21.15.151:0/1761571131 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252498 (secure 0 0 0) 0x7efef4154e40 con 0x7efed407a740 2024-09-16T15:53:52.946 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:53:52.948 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-09-16T15:53:52.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.946+0000 7efefb200640 1 -- 172.21.15.151:0/1761571131 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efed407a740 msgr2=0x7efed407cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:52.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.946+0000 7efefb200640 1 --2- 172.21.15.151:0/1761571131 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efed407a740 0x7efed407cc00 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7efee0005fd0 tx=0x7efee0005850 comp rx=0 tx=0).stop 2024-09-16T15:53:52.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.946+0000 7efefb200640 1 -- 172.21.15.151:0/1761571131 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efef4072df0 msgr2=0x7efef41540c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:52.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.946+0000 7efefb200640 1 --2- 172.21.15.151:0/1761571131 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efef4072df0 0x7efef41540c0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7efee400ba50 tx=0x7efee400bf10 comp rx=0 tx=0).stop 2024-09-16T15:53:52.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.946+0000 7efefb200640 1 -- 172.21.15.151:0/1761571131 shutdown_connections 2024-09-16T15:53:52.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.946+0000 7efefb200640 1 --2- 172.21.15.151:0/1761571131 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efed407a740 0x7efed407cc00 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:52.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.946+0000 7efefb200640 1 --2- 172.21.15.151:0/1761571131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efef40af3d0 0x7efef409d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:52.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.946+0000 7efefb200640 1 --2- 172.21.15.151:0/1761571131 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efef4161550 0x7efef409d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:52.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.946+0000 7efefb200640 1 --2- 172.21.15.151:0/1761571131 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efef4072df0 0x7efef41540c0 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:52.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.946+0000 7efefb200640 1 -- 172.21.15.151:0/1761571131 >> 172.21.15.151:0/1761571131 conn(0x7efef40b1950 msgr2=0x7efef415ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:52.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.946+0000 7efefb200640 1 -- 172.21.15.151:0/1761571131 shutdown_connections 2024-09-16T15:53:52.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:52.946+0000 7efefb200640 1 -- 172.21.15.151:0/1761571131 wait complete. 2024-09-16T15:53:52.963 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":567,"stamp":"2024-09-16T15:53:51.937187+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":164,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1078028,"kb_used_data":14108,"kb_used_omap":12,"kb_used_meta":1063731,"kb_avail":753896692,"statfs":{"total":773094113280,"available":771990212608,"internally_reserved":0,"allocated":14446592,"data_stored":7015752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12738,"internal_metadata":1089261118},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001532"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.274843+0000","last_change":"2024-09-16T15:53:47.064079+0000","last_active":"2024-09-16T15:53:49.274843+0000","last_peered":"2024-09-16T15:53:49.274843+0000","last_clean":"2024-09-16T15:53:49.274843+0000","last_became_active":"2024-09-16T15:53:47.063709+0000","last_became_peered":"2024-09-16T15:53:47.063709+0000","last_unstale":"2024-09-16T15:53:49.274843+0000","last_undegraded":"2024-09-16T15:53:49.274843+0000","last_fullsized":"2024-09-16T15:53:49.274843+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:52:00.354801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.274872+0000","last_change":"2024-09-16T15:53:47.054602+0000","last_active":"2024-09-16T15:53:49.274872+0000","last_peered":"2024-09-16T15:53:49.274872+0000","last_clean":"2024-09-16T15:53:49.274872+0000","last_became_active":"2024-09-16T15:53:47.054286+0000","last_became_peered":"2024-09-16T15:53:47.054286+0000","last_unstale":"2024-09-16T15:53:49.274872+0000","last_undegraded":"2024-09-16T15:53:49.274872+0000","last_fullsized":"2024-09-16T15:53:49.274872+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:18:36.519474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.604743+0000","last_change":"2024-09-16T15:53:47.049667+0000","last_active":"2024-09-16T15:53:48.604743+0000","last_peered":"2024-09-16T15:53:48.604743+0000","last_clean":"2024-09-16T15:53:48.604743+0000","last_became_active":"2024-09-16T15:53:47.048431+0000","last_became_peered":"2024-09-16T15:53:47.048431+0000","last_unstale":"2024-09-16T15:53:48.604743+0000","last_undegraded":"2024-09-16T15:53:48.604743+0000","last_fullsized":"2024-09-16T15:53:48.604743+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:25:27.052540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.036138+0000","last_change":"2024-09-16T15:53:47.066311+0000","last_active":"2024-09-16T15:53:48.036138+0000","last_peered":"2024-09-16T15:53:48.036138+0000","last_clean":"2024-09-16T15:53:48.036138+0000","last_became_active":"2024-09-16T15:53:47.066005+0000","last_became_peered":"2024-09-16T15:53:47.066005+0000","last_unstale":"2024-09-16T15:53:48.036138+0000","last_undegraded":"2024-09-16T15:53:48.036138+0000","last_fullsized":"2024-09-16T15:53:48.036138+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:27:17.121069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.038493+0000","last_change":"2024-09-16T15:53:47.067568+0000","last_active":"2024-09-16T15:53:49.038493+0000","last_peered":"2024-09-16T15:53:49.038493+0000","last_clean":"2024-09-16T15:53:49.038493+0000","last_became_active":"2024-09-16T15:53:47.067220+0000","last_became_peered":"2024-09-16T15:53:47.067220+0000","last_unstale":"2024-09-16T15:53:49.038493+0000","last_undegraded":"2024-09-16T15:53:49.038493+0000","last_fullsized":"2024-09-16T15:53:49.038493+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:39:06.152472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.035632+0000","last_change":"2024-09-16T15:53:47.066404+0000","last_active":"2024-09-16T15:53:48.035632+0000","last_peered":"2024-09-16T15:53:48.035632+0000","last_clean":"2024-09-16T15:53:48.035632+0000","last_became_active":"2024-09-16T15:53:47.064379+0000","last_became_peered":"2024-09-16T15:53:47.064379+0000","last_unstale":"2024-09-16T15:53:48.035632+0000","last_undegraded":"2024-09-16T15:53:48.035632+0000","last_fullsized":"2024-09-16T15:53:48.035632+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:57:58.563630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.038351+0000","last_change":"2024-09-16T15:53:47.049576+0000","last_active":"2024-09-16T15:53:49.038351+0000","last_peered":"2024-09-16T15:53:49.038351+0000","last_clean":"2024-09-16T15:53:49.038351+0000","last_became_active":"2024-09-16T15:53:47.049291+0000","last_became_peered":"2024-09-16T15:53:47.049291+0000","last_unstale":"2024-09-16T15:53:49.038351+0000","last_undegraded":"2024-09-16T15:53:49.038351+0000","last_fullsized":"2024-09-16T15:53:49.038351+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:10:04.681002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.038227+0000","last_change":"2024-09-16T15:53:47.054562+0000","last_active":"2024-09-16T15:53:49.038227+0000","last_peered":"2024-09-16T15:53:49.038227+0000","last_clean":"2024-09-16T15:53:49.038227+0000","last_became_active":"2024-09-16T15:53:47.054310+0000","last_became_peered":"2024-09-16T15:53:47.054310+0000","last_unstale":"2024-09-16T15:53:49.038227+0000","last_undegraded":"2024-09-16T15:53:49.038227+0000","last_fullsized":"2024-09-16T15:53:49.038227+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:36:01.782108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.604647+0000","last_change":"2024-09-16T15:53:47.045614+0000","last_active":"2024-09-16T15:53:48.604647+0000","last_peered":"2024-09-16T15:53:48.604647+0000","last_clean":"2024-09-16T15:53:48.604647+0000","last_became_active":"2024-09-16T15:53:47.045131+0000","last_became_peered":"2024-09-16T15:53:47.045131+0000","last_unstale":"2024-09-16T15:53:48.604647+0000","last_undegraded":"2024-09-16T15:53:48.604647+0000","last_fullsized":"2024-09-16T15:53:48.604647+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:25:39.938644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.274853+0000","last_change":"2024-09-16T15:53:47.063974+0000","last_active":"2024-09-16T15:53:49.274853+0000","last_peered":"2024-09-16T15:53:49.274853+0000","last_clean":"2024-09-16T15:53:49.274853+0000","last_became_active":"2024-09-16T15:53:47.063634+0000","last_became_peered":"2024-09-16T15:53:47.063634+0000","last_unstale":"2024-09-16T15:53:49.274853+0000","last_undegraded":"2024-09-16T15:53:49.274853+0000","last_fullsized":"2024-09-16T15:53:49.274853+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:22:44.163439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.036512+0000","last_change":"2024-09-16T15:53:47.045716+0000","last_active":"2024-09-16T15:53:48.036512+0000","last_peered":"2024-09-16T15:53:48.036512+0000","last_clean":"2024-09-16T15:53:48.036512+0000","last_became_active":"2024-09-16T15:53:47.045439+0000","last_became_peered":"2024-09-16T15:53:47.045439+0000","last_unstale":"2024-09-16T15:53:48.036512+0000","last_undegraded":"2024-09-16T15:53:48.036512+0000","last_fullsized":"2024-09-16T15:53:48.036512+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:40:26.604250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.036410+0000","last_change":"2024-09-16T15:53:47.070120+0000","last_active":"2024-09-16T15:53:48.036410+0000","last_peered":"2024-09-16T15:53:48.036410+0000","last_clean":"2024-09-16T15:53:48.036410+0000","last_became_active":"2024-09-16T15:53:47.068816+0000","last_became_peered":"2024-09-16T15:53:47.068816+0000","last_unstale":"2024-09-16T15:53:48.036410+0000","last_undegraded":"2024-09-16T15:53:48.036410+0000","last_fullsized":"2024-09-16T15:53:48.036410+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:32:42.788318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.036127+0000","last_change":"2024-09-16T15:53:47.052483+0000","last_active":"2024-09-16T15:53:48.036127+0000","last_peered":"2024-09-16T15:53:48.036127+0000","last_clean":"2024-09-16T15:53:48.036127+0000","last_became_active":"2024-09-16T15:53:47.052188+0000","last_became_peered":"2024-09-16T15:53:47.052188+0000","last_unstale":"2024-09-16T15:53:48.036127+0000","last_undegraded":"2024-09-16T15:53:48.036127+0000","last_fullsized":"2024-09-16T15:53:48.036127+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:28:45.284824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":10,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-16T15:53:47.058871+0000","last_change":"2024-09-16T15:53:47.056102+0000","last_active":"2024-09-16T15:53:47.058871+0000","last_peered":"2024-09-16T15:53:47.058871+0000","last_clean":"2024-09-16T15:53:47.058871+0000","last_became_active":"2024-09-16T15:53:47.055786+0000","last_became_peered":"2024-09-16T15:53:47.055786+0000","last_unstale":"2024-09-16T15:53:47.058871+0000","last_undegraded":"2024-09-16T15:53:47.058871+0000","last_fullsized":"2024-09-16T15:53:47.058871+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:02:29.501341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.481280+0000","last_change":"2024-09-16T15:53:47.053898+0000","last_active":"2024-09-16T15:53:48.481280+0000","last_peered":"2024-09-16T15:53:48.481280+0000","last_clean":"2024-09-16T15:53:48.481280+0000","last_became_active":"2024-09-16T15:53:47.053585+0000","last_became_peered":"2024-09-16T15:53:47.053585+0000","last_unstale":"2024-09-16T15:53:48.481280+0000","last_undegraded":"2024-09-16T15:53:48.481280+0000","last_fullsized":"2024-09-16T15:53:48.481280+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:00:53.660785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":10,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-16T15:53:47.063272+0000","last_change":"2024-09-16T15:53:47.060703+0000","last_active":"2024-09-16T15:53:47.063272+0000","last_peered":"2024-09-16T15:53:47.063272+0000","last_clean":"2024-09-16T15:53:47.063272+0000","last_became_active":"2024-09-16T15:53:47.060125+0000","last_became_peered":"2024-09-16T15:53:47.060125+0000","last_unstale":"2024-09-16T15:53:47.063272+0000","last_undegraded":"2024-09-16T15:53:47.063272+0000","last_fullsized":"2024-09-16T15:53:47.063272+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:29:02.714478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.604792+0000","last_change":"2024-09-16T15:53:47.047406+0000","last_active":"2024-09-16T15:53:48.604792+0000","last_peered":"2024-09-16T15:53:48.604792+0000","last_clean":"2024-09-16T15:53:48.604792+0000","last_became_active":"2024-09-16T15:53:47.045413+0000","last_became_peered":"2024-09-16T15:53:47.045413+0000","last_unstale":"2024-09-16T15:53:48.604792+0000","last_undegraded":"2024-09-16T15:53:48.604792+0000","last_fullsized":"2024-09-16T15:53:48.604792+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:24:19.371541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.481128+0000","last_change":"2024-09-16T15:53:47.049501+0000","last_active":"2024-09-16T15:53:48.481128+0000","last_peered":"2024-09-16T15:53:48.481128+0000","last_clean":"2024-09-16T15:53:48.481128+0000","last_became_active":"2024-09-16T15:53:47.046606+0000","last_became_peered":"2024-09-16T15:53:47.046606+0000","last_unstale":"2024-09-16T15:53:48.481128+0000","last_undegraded":"2024-09-16T15:53:48.481128+0000","last_fullsized":"2024-09-16T15:53:48.481128+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:00:13.468850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.274676+0000","last_change":"2024-09-16T15:53:47.055350+0000","last_active":"2024-09-16T15:53:49.274676+0000","last_peered":"2024-09-16T15:53:49.274676+0000","last_clean":"2024-09-16T15:53:49.274676+0000","last_became_active":"2024-09-16T15:53:47.051177+0000","last_became_peered":"2024-09-16T15:53:47.051177+0000","last_unstale":"2024-09-16T15:53:49.274676+0000","last_undegraded":"2024-09-16T15:53:49.274676+0000","last_fullsized":"2024-09-16T15:53:49.274676+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:13:16.281809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.481139+0000","last_change":"2024-09-16T15:53:47.065030+0000","last_active":"2024-09-16T15:53:48.481139+0000","last_peered":"2024-09-16T15:53:48.481139+0000","last_clean":"2024-09-16T15:53:48.481139+0000","last_became_active":"2024-09-16T15:53:47.064738+0000","last_became_peered":"2024-09-16T15:53:47.064738+0000","last_unstale":"2024-09-16T15:53:48.481139+0000","last_undegraded":"2024-09-16T15:53:48.481139+0000","last_fullsized":"2024-09-16T15:53:48.481139+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:07:01.165101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.035907+0000","last_change":"2024-09-16T15:53:47.063760+0000","last_active":"2024-09-16T15:53:48.035907+0000","last_peered":"2024-09-16T15:53:48.035907+0000","last_clean":"2024-09-16T15:53:48.035907+0000","last_became_active":"2024-09-16T15:53:47.063404+0000","last_became_peered":"2024-09-16T15:53:47.063404+0000","last_unstale":"2024-09-16T15:53:48.035907+0000","last_undegraded":"2024-09-16T15:53:48.035907+0000","last_fullsized":"2024-09-16T15:53:48.035907+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:30:28.946598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.274350+0000","last_change":"2024-09-16T15:53:47.044151+0000","last_active":"2024-09-16T15:53:49.274350+0000","last_peered":"2024-09-16T15:53:49.274350+0000","last_clean":"2024-09-16T15:53:49.274350+0000","last_became_active":"2024-09-16T15:53:47.043246+0000","last_became_peered":"2024-09-16T15:53:47.043246+0000","last_unstale":"2024-09-16T15:53:49.274350+0000","last_undegraded":"2024-09-16T15:53:49.274350+0000","last_fullsized":"2024-09-16T15:53:49.274350+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:14:07.626424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.273638+0000","last_change":"2024-09-16T15:53:47.051983+0000","last_active":"2024-09-16T15:53:49.273638+0000","last_peered":"2024-09-16T15:53:49.273638+0000","last_clean":"2024-09-16T15:53:49.273638+0000","last_became_active":"2024-09-16T15:53:47.050704+0000","last_became_peered":"2024-09-16T15:53:47.050704+0000","last_unstale":"2024-09-16T15:53:49.273638+0000","last_undegraded":"2024-09-16T15:53:49.273638+0000","last_fullsized":"2024-09-16T15:53:49.273638+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:35:02.303710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.480980+0000","last_change":"2024-09-16T15:53:47.069714+0000","last_active":"2024-09-16T15:53:48.480980+0000","last_peered":"2024-09-16T15:53:48.480980+0000","last_clean":"2024-09-16T15:53:48.480980+0000","last_became_active":"2024-09-16T15:53:47.069413+0000","last_became_peered":"2024-09-16T15:53:47.069413+0000","last_unstale":"2024-09-16T15:53:48.480980+0000","last_undegraded":"2024-09-16T15:53:48.480980+0000","last_fullsized":"2024-09-16T15:53:48.480980+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:27:29.156081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.036803+0000","last_change":"2024-09-16T15:53:47.053885+0000","last_active":"2024-09-16T15:53:48.036803+0000","last_peered":"2024-09-16T15:53:48.036803+0000","last_clean":"2024-09-16T15:53:48.036803+0000","last_became_active":"2024-09-16T15:53:47.052585+0000","last_became_peered":"2024-09-16T15:53:47.052585+0000","last_unstale":"2024-09-16T15:53:48.036803+0000","last_undegraded":"2024-09-16T15:53:48.036803+0000","last_fullsized":"2024-09-16T15:53:48.036803+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:52:32.247179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.275135+0000","last_change":"2024-09-16T15:53:47.047903+0000","last_active":"2024-09-16T15:53:49.275135+0000","last_peered":"2024-09-16T15:53:49.275135+0000","last_clean":"2024-09-16T15:53:49.275135+0000","last_became_active":"2024-09-16T15:53:47.047524+0000","last_became_peered":"2024-09-16T15:53:47.047524+0000","last_unstale":"2024-09-16T15:53:49.275135+0000","last_undegraded":"2024-09-16T15:53:49.275135+0000","last_fullsized":"2024-09-16T15:53:49.275135+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:28:37.082216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":10,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-16T15:53:47.068210+0000","last_change":"2024-09-16T15:53:47.067044+0000","last_active":"2024-09-16T15:53:47.068210+0000","last_peered":"2024-09-16T15:53:47.068210+0000","last_clean":"2024-09-16T15:53:47.068210+0000","last_became_active":"2024-09-16T15:53:47.066772+0000","last_became_peered":"2024-09-16T15:53:47.066772+0000","last_unstale":"2024-09-16T15:53:47.068210+0000","last_undegraded":"2024-09-16T15:53:47.068210+0000","last_fullsized":"2024-09-16T15:53:47.068210+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:27:45.682352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.273899+0000","last_change":"2024-09-16T15:53:47.047353+0000","last_active":"2024-09-16T15:53:49.273899+0000","last_peered":"2024-09-16T15:53:49.273899+0000","last_clean":"2024-09-16T15:53:49.273899+0000","last_became_active":"2024-09-16T15:53:47.047102+0000","last_became_peered":"2024-09-16T15:53:47.047102+0000","last_unstale":"2024-09-16T15:53:49.273899+0000","last_undegraded":"2024-09-16T15:53:49.273899+0000","last_fullsized":"2024-09-16T15:53:49.273899+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:21:20.787684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.274512+0000","last_change":"2024-09-16T15:53:47.046370+0000","last_active":"2024-09-16T15:53:49.274512+0000","last_peered":"2024-09-16T15:53:49.274512+0000","last_clean":"2024-09-16T15:53:49.274512+0000","last_became_active":"2024-09-16T15:53:47.046059+0000","last_became_peered":"2024-09-16T15:53:47.046059+0000","last_unstale":"2024-09-16T15:53:49.274512+0000","last_undegraded":"2024-09-16T15:53:49.274512+0000","last_fullsized":"2024-09-16T15:53:49.274512+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:10:04.681002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.274463+0000","last_change":"2024-09-16T15:53:47.049350+0000","last_active":"2024-09-16T15:53:49.274463+0000","last_peered":"2024-09-16T15:53:49.274463+0000","last_clean":"2024-09-16T15:53:49.274463+0000","last_became_active":"2024-09-16T15:53:47.048759+0000","last_became_peered":"2024-09-16T15:53:47.048759+0000","last_unstale":"2024-09-16T15:53:49.274463+0000","last_undegraded":"2024-09-16T15:53:49.274463+0000","last_fullsized":"2024-09-16T15:53:49.274463+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:36:01.782108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.273712+0000","last_change":"2024-09-16T15:53:47.050603+0000","last_active":"2024-09-16T15:53:49.273712+0000","last_peered":"2024-09-16T15:53:49.273712+0000","last_clean":"2024-09-16T15:53:49.273712+0000","last_became_active":"2024-09-16T15:53:47.050186+0000","last_became_peered":"2024-09-16T15:53:47.050186+0000","last_unstale":"2024-09-16T15:53:49.273712+0000","last_undegraded":"2024-09-16T15:53:49.273712+0000","last_fullsized":"2024-09-16T15:53:49.273712+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:49:30.590331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.604485+0000","last_change":"2024-09-16T15:53:47.049125+0000","last_active":"2024-09-16T15:53:48.604485+0000","last_peered":"2024-09-16T15:53:48.604485+0000","last_clean":"2024-09-16T15:53:48.604485+0000","last_became_active":"2024-09-16T15:53:47.048889+0000","last_became_peered":"2024-09-16T15:53:47.048889+0000","last_unstale":"2024-09-16T15:53:48.604485+0000","last_undegraded":"2024-09-16T15:53:48.604485+0000","last_fullsized":"2024-09-16T15:53:48.604485+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:06:42.055204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":10,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-16T15:53:47.062974+0000","last_change":"2024-09-16T15:53:47.056337+0000","last_active":"2024-09-16T15:53:47.062974+0000","last_peered":"2024-09-16T15:53:47.062974+0000","last_clean":"2024-09-16T15:53:47.062974+0000","last_became_active":"2024-09-16T15:53:47.056039+0000","last_became_peered":"2024-09-16T15:53:47.056039+0000","last_unstale":"2024-09-16T15:53:47.062974+0000","last_undegraded":"2024-09-16T15:53:47.062974+0000","last_fullsized":"2024-09-16T15:53:47.062974+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:12:43.131784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1f","version":"132'4","reported_seq":19,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.038798+0000","last_change":"2024-09-16T15:53:46.049579+0000","last_active":"2024-09-16T15:53:49.038798+0000","last_peered":"2024-09-16T15:53:49.038798+0000","last_clean":"2024-09-16T15:53:49.038798+0000","last_became_active":"2024-09-16T15:53:46.049306+0000","last_became_peered":"2024-09-16T15:53:46.049306+0000","last_unstale":"2024-09-16T15:53:49.038798+0000","last_undegraded":"2024-09-16T15:53:49.038798+0000","last_fullsized":"2024-09-16T15:53:49.038798+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:55:49.783269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.604895+0000","last_change":"2024-09-16T15:53:47.039878+0000","last_active":"2024-09-16T15:53:48.604895+0000","last_peered":"2024-09-16T15:53:48.604895+0000","last_clean":"2024-09-16T15:53:48.604895+0000","last_became_active":"2024-09-16T15:53:47.038999+0000","last_became_peered":"2024-09-16T15:53:47.038999+0000","last_unstale":"2024-09-16T15:53:48.604895+0000","last_undegraded":"2024-09-16T15:53:48.604895+0000","last_fullsized":"2024-09-16T15:53:48.604895+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:34:03.640512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1e","version":"132'1","reported_seq":16,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.604710+0000","last_change":"2024-09-16T15:53:46.055817+0000","last_active":"2024-09-16T15:53:48.604710+0000","last_peered":"2024-09-16T15:53:48.604710+0000","last_clean":"2024-09-16T15:53:48.604710+0000","last_became_active":"2024-09-16T15:53:46.055540+0000","last_became_peered":"2024-09-16T15:53:46.055540+0000","last_unstale":"2024-09-16T15:53:48.604710+0000","last_undegraded":"2024-09-16T15:53:48.604710+0000","last_fullsized":"2024-09-16T15:53:48.604710+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:23:12.275241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.604508+0000","last_change":"2024-09-16T15:53:47.055252+0000","last_active":"2024-09-16T15:53:48.604508+0000","last_peered":"2024-09-16T15:53:48.604508+0000","last_clean":"2024-09-16T15:53:48.604508+0000","last_became_active":"2024-09-16T15:53:47.054974+0000","last_became_peered":"2024-09-16T15:53:47.054974+0000","last_unstale":"2024-09-16T15:53:48.604508+0000","last_undegraded":"2024-09-16T15:53:48.604508+0000","last_fullsized":"2024-09-16T15:53:48.604508+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:54.366435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1d","version":"132'2","reported_seq":17,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.036479+0000","last_change":"2024-09-16T15:53:46.043053+0000","last_active":"2024-09-16T15:53:48.036479+0000","last_peered":"2024-09-16T15:53:48.036479+0000","last_clean":"2024-09-16T15:53:48.036479+0000","last_became_active":"2024-09-16T15:53:46.042758+0000","last_became_peered":"2024-09-16T15:53:46.042758+0000","last_unstale":"2024-09-16T15:53:48.036479+0000","last_undegraded":"2024-09-16T15:53:48.036479+0000","last_fullsized":"2024-09-16T15:53:48.036479+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:49:50.234698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.036593+0000","last_change":"2024-09-16T15:53:47.069273+0000","last_active":"2024-09-16T15:53:48.036593+0000","last_peered":"2024-09-16T15:53:48.036593+0000","last_clean":"2024-09-16T15:53:48.036593+0000","last_became_active":"2024-09-16T15:53:47.067006+0000","last_became_peered":"2024-09-16T15:53:47.067006+0000","last_unstale":"2024-09-16T15:53:48.036593+0000","last_undegraded":"2024-09-16T15:53:48.036593+0000","last_fullsized":"2024-09-16T15:53:48.036593+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:07:19.329442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1c","version":"132'2","reported_seq":17,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.274667+0000","last_change":"2024-09-16T15:53:46.046499+0000","last_active":"2024-09-16T15:53:49.274667+0000","last_peered":"2024-09-16T15:53:49.274667+0000","last_clean":"2024-09-16T15:53:49.274667+0000","last_became_active":"2024-09-16T15:53:46.046246+0000","last_became_peered":"2024-09-16T15:53:46.046246+0000","last_unstale":"2024-09-16T15:53:49.274667+0000","last_undegraded":"2024-09-16T15:53:49.274667+0000","last_fullsized":"2024-09-16T15:53:49.274667+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:07:27.829527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.480951+0000","last_change":"2024-09-16T15:53:47.060948+0000","last_active":"2024-09-16T15:53:48.480951+0000","last_peered":"2024-09-16T15:53:48.480951+0000","last_clean":"2024-09-16T15:53:48.480951+0000","last_became_active":"2024-09-16T15:53:47.060538+0000","last_became_peered":"2024-09-16T15:53:47.060538+0000","last_unstale":"2024-09-16T15:53:48.480951+0000","last_undegraded":"2024-09-16T15:53:48.480951+0000","last_fullsized":"2024-09-16T15:53:48.480951+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:56:18.849300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1b","version":"132'1","reported_seq":16,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.274448+0000","last_change":"2024-09-16T15:53:46.048816+0000","last_active":"2024-09-16T15:53:49.274448+0000","last_peered":"2024-09-16T15:53:49.274448+0000","last_clean":"2024-09-16T15:53:49.274448+0000","last_became_active":"2024-09-16T15:53:46.048416+0000","last_became_peered":"2024-09-16T15:53:46.048416+0000","last_unstale":"2024-09-16T15:53:49.274448+0000","last_undegraded":"2024-09-16T15:53:49.274448+0000","last_fullsized":"2024-09-16T15:53:49.274448+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:13:59.648869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.035988+0000","last_change":"2024-09-16T15:53:47.068600+0000","last_active":"2024-09-16T15:53:48.035988+0000","last_peered":"2024-09-16T15:53:48.035988+0000","last_clean":"2024-09-16T15:53:48.035988+0000","last_became_active":"2024-09-16T15:53:47.066780+0000","last_became_peered":"2024-09-16T15:53:47.066780+0000","last_unstale":"2024-09-16T15:53:48.035988+0000","last_undegraded":"2024-09-16T15:53:48.035988+0000","last_fullsized":"2024-09-16T15:53:48.035988+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:25:28.695527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"132'1","reported_seq":16,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.480731+0000","last_change":"2024-09-16T15:53:46.051615+0000","last_active":"2024-09-16T15:53:48.480731+0000","last_peered":"2024-09-16T15:53:48.480731+0000","last_clean":"2024-09-16T15:53:48.480731+0000","last_became_active":"2024-09-16T15:53:46.051253+0000","last_became_peered":"2024-09-16T15:53:46.051253+0000","last_unstale":"2024-09-16T15:53:48.480731+0000","last_undegraded":"2024-09-16T15:53:48.480731+0000","last_fullsized":"2024-09-16T15:53:48.480731+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:11:54.504207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.035617+0000","last_change":"2024-09-16T15:53:47.063618+0000","last_active":"2024-09-16T15:53:48.035617+0000","last_peered":"2024-09-16T15:53:48.035617+0000","last_clean":"2024-09-16T15:53:48.035617+0000","last_became_active":"2024-09-16T15:53:47.063347+0000","last_became_peered":"2024-09-16T15:53:47.063347+0000","last_unstale":"2024-09-16T15:53:48.035617+0000","last_undegraded":"2024-09-16T15:53:48.035617+0000","last_fullsized":"2024-09-16T15:53:48.035617+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:58:32.257998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.604391+0000","last_change":"2024-09-16T15:53:46.052810+0000","last_active":"2024-09-16T15:53:48.604391+0000","last_peered":"2024-09-16T15:53:48.604391+0000","last_clean":"2024-09-16T15:53:48.604391+0000","last_became_active":"2024-09-16T15:53:46.052461+0000","last_became_peered":"2024-09-16T15:53:46.052461+0000","last_unstale":"2024-09-16T15:53:48.604391+0000","last_undegraded":"2024-09-16T15:53:48.604391+0000","last_fullsized":"2024-09-16T15:53:48.604391+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:36:21.458493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.274292+0000","last_change":"2024-09-16T15:53:47.041379+0000","last_active":"2024-09-16T15:53:49.274292+0000","last_peered":"2024-09-16T15:53:49.274292+0000","last_clean":"2024-09-16T15:53:49.274292+0000","last_became_active":"2024-09-16T15:53:47.041100+0000","last_became_peered":"2024-09-16T15:53:47.041100+0000","last_unstale":"2024-09-16T15:53:49.274292+0000","last_undegraded":"2024-09-16T15:53:49.274292+0000","last_fullsized":"2024-09-16T15:53:49.274292+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:57:10.468784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.274281+0000","last_change":"2024-09-16T15:53:46.050598+0000","last_active":"2024-09-16T15:53:49.274281+0000","last_peered":"2024-09-16T15:53:49.274281+0000","last_clean":"2024-09-16T15:53:49.274281+0000","last_became_active":"2024-09-16T15:53:46.050355+0000","last_became_peered":"2024-09-16T15:53:46.050355+0000","last_unstale":"2024-09-16T15:53:49.274281+0000","last_undegraded":"2024-09-16T15:53:49.274281+0000","last_fullsized":"2024-09-16T15:53:49.274281+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:00:09.267267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.275145+0000","last_change":"2024-09-16T15:53:47.052882+0000","last_active":"2024-09-16T15:53:49.275145+0000","last_peered":"2024-09-16T15:53:49.275145+0000","last_clean":"2024-09-16T15:53:49.275145+0000","last_became_active":"2024-09-16T15:53:47.052537+0000","last_became_peered":"2024-09-16T15:53:47.052537+0000","last_unstale":"2024-09-16T15:53:49.275145+0000","last_undegraded":"2024-09-16T15:53:49.275145+0000","last_fullsized":"2024-09-16T15:53:49.275145+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:04:06.435765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.036663+0000","last_change":"2024-09-16T15:53:46.041207+0000","last_active":"2024-09-16T15:53:48.036663+0000","last_peered":"2024-09-16T15:53:48.036663+0000","last_clean":"2024-09-16T15:53:48.036663+0000","last_became_active":"2024-09-16T15:53:46.040942+0000","last_became_peered":"2024-09-16T15:53:46.040942+0000","last_unstale":"2024-09-16T15:53:48.036663+0000","last_undegraded":"2024-09-16T15:53:48.036663+0000","last_fullsized":"2024-09-16T15:53:48.036663+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:00:03.561191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.038694+0000","last_change":"2024-09-16T15:53:47.054422+0000","last_active":"2024-09-16T15:53:49.038694+0000","last_peered":"2024-09-16T15:53:49.038694+0000","last_clean":"2024-09-16T15:53:49.038694+0000","last_became_active":"2024-09-16T15:53:47.054120+0000","last_became_peered":"2024-09-16T15:53:47.054120+0000","last_unstale":"2024-09-16T15:53:49.038694+0000","last_undegraded":"2024-09-16T15:53:49.038694+0000","last_fullsized":"2024-09-16T15:53:49.038694+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:25:48.625685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.480939+0000","last_change":"2024-09-16T15:53:46.049887+0000","last_active":"2024-09-16T15:53:48.480939+0000","last_peered":"2024-09-16T15:53:48.480939+0000","last_clean":"2024-09-16T15:53:48.480939+0000","last_became_active":"2024-09-16T15:53:46.049554+0000","last_became_peered":"2024-09-16T15:53:46.049554+0000","last_unstale":"2024-09-16T15:53:48.480939+0000","last_undegraded":"2024-09-16T15:53:48.480939+0000","last_fullsized":"2024-09-16T15:53:48.480939+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:39:03.893863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.036446+0000","last_change":"2024-09-16T15:53:47.069807+0000","last_active":"2024-09-16T15:53:48.036446+0000","last_peered":"2024-09-16T15:53:48.036446+0000","last_clean":"2024-09-16T15:53:48.036446+0000","last_became_active":"2024-09-16T15:53:47.068406+0000","last_became_peered":"2024-09-16T15:53:47.068406+0000","last_unstale":"2024-09-16T15:53:48.036446+0000","last_undegraded":"2024-09-16T15:53:48.036446+0000","last_fullsized":"2024-09-16T15:53:48.036446+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:51:00.666807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.15","version":"132'3","reported_seq":18,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.038592+0000","last_change":"2024-09-16T15:53:46.051421+0000","last_active":"2024-09-16T15:53:49.038592+0000","last_peered":"2024-09-16T15:53:49.038592+0000","last_clean":"2024-09-16T15:53:49.038592+0000","last_became_active":"2024-09-16T15:53:46.049074+0000","last_became_peered":"2024-09-16T15:53:46.049074+0000","last_unstale":"2024-09-16T15:53:49.038592+0000","last_undegraded":"2024-09-16T15:53:49.038592+0000","last_fullsized":"2024-09-16T15:53:49.038592+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:03:46.041805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.273880+0000","last_change":"2024-09-16T15:53:47.068386+0000","last_active":"2024-09-16T15:53:49.273880+0000","last_peered":"2024-09-16T15:53:49.273880+0000","last_clean":"2024-09-16T15:53:49.273880+0000","last_became_active":"2024-09-16T15:53:47.067650+0000","last_became_peered":"2024-09-16T15:53:47.067650+0000","last_unstale":"2024-09-16T15:53:49.273880+0000","last_undegraded":"2024-09-16T15:53:49.273880+0000","last_fullsized":"2024-09-16T15:53:49.273880+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:30:31.196430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.14","version":"132'6","reported_seq":21,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.273912+0000","last_change":"2024-09-16T15:53:46.055615+0000","last_active":"2024-09-16T15:53:49.273912+0000","last_peered":"2024-09-16T15:53:49.273912+0000","last_clean":"2024-09-16T15:53:49.273912+0000","last_became_active":"2024-09-16T15:53:46.055155+0000","last_became_peered":"2024-09-16T15:53:46.055155+0000","last_unstale":"2024-09-16T15:53:49.273912+0000","last_undegraded":"2024-09-16T15:53:49.273912+0000","last_fullsized":"2024-09-16T15:53:49.273912+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:43:29.847384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":10,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-16T15:53:47.059010+0000","last_change":"2024-09-16T15:53:47.055028+0000","last_active":"2024-09-16T15:53:47.059010+0000","last_peered":"2024-09-16T15:53:47.059010+0000","last_clean":"2024-09-16T15:53:47.059010+0000","last_became_active":"2024-09-16T15:53:47.054638+0000","last_became_peered":"2024-09-16T15:53:47.054638+0000","last_unstale":"2024-09-16T15:53:47.059010+0000","last_undegraded":"2024-09-16T15:53:47.059010+0000","last_fullsized":"2024-09-16T15:53:47.059010+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:49:07.189417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-16T15:53:47.042905+0000","last_change":"2024-09-16T15:53:46.053325+0000","last_active":"2024-09-16T15:53:47.042905+0000","last_peered":"2024-09-16T15:53:47.042905+0000","last_clean":"2024-09-16T15:53:47.042905+0000","last_became_active":"2024-09-16T15:53:46.051989+0000","last_became_peered":"2024-09-16T15:53:46.051989+0000","last_unstale":"2024-09-16T15:53:47.042905+0000","last_undegraded":"2024-09-16T15:53:47.042905+0000","last_fullsized":"2024-09-16T15:53:47.042905+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:10:52.279943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":10,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-16T15:53:47.063798+0000","last_change":"2024-09-16T15:53:47.062301+0000","last_active":"2024-09-16T15:53:47.063798+0000","last_peered":"2024-09-16T15:53:47.063798+0000","last_clean":"2024-09-16T15:53:47.063798+0000","last_became_active":"2024-09-16T15:53:47.058392+0000","last_became_peered":"2024-09-16T15:53:47.058392+0000","last_unstale":"2024-09-16T15:53:47.063798+0000","last_undegraded":"2024-09-16T15:53:47.063798+0000","last_fullsized":"2024-09-16T15:53:47.063798+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:35:38.397852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.035810+0000","last_change":"2024-09-16T15:53:46.055507+0000","last_active":"2024-09-16T15:53:48.035810+0000","last_peered":"2024-09-16T15:53:48.035810+0000","last_clean":"2024-09-16T15:53:48.035810+0000","last_became_active":"2024-09-16T15:53:46.055247+0000","last_became_peered":"2024-09-16T15:53:46.055247+0000","last_unstale":"2024-09-16T15:53:48.035810+0000","last_undegraded":"2024-09-16T15:53:48.035810+0000","last_fullsized":"2024-09-16T15:53:48.035810+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:26:45.397159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.035599+0000","last_change":"2024-09-16T15:53:47.065045+0000","last_active":"2024-09-16T15:53:48.035599+0000","last_peered":"2024-09-16T15:53:48.035599+0000","last_clean":"2024-09-16T15:53:48.035599+0000","last_became_active":"2024-09-16T15:53:47.064756+0000","last_became_peered":"2024-09-16T15:53:47.064756+0000","last_unstale":"2024-09-16T15:53:48.035599+0000","last_undegraded":"2024-09-16T15:53:48.035599+0000","last_fullsized":"2024-09-16T15:53:48.035599+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:29:02.714478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.10","version":"132'8","reported_seq":23,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.035597+0000","last_change":"2024-09-16T15:53:46.054698+0000","last_active":"2024-09-16T15:53:48.035597+0000","last_peered":"2024-09-16T15:53:48.035597+0000","last_clean":"2024-09-16T15:53:48.035597+0000","last_became_active":"2024-09-16T15:53:46.050527+0000","last_became_peered":"2024-09-16T15:53:46.050527+0000","last_unstale":"2024-09-16T15:53:48.035597+0000","last_undegraded":"2024-09-16T15:53:48.035597+0000","last_fullsized":"2024-09-16T15:53:48.035597+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:29:09.646752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.038576+0000","last_change":"2024-09-16T15:53:47.049220+0000","last_active":"2024-09-16T15:53:49.038576+0000","last_peered":"2024-09-16T15:53:49.038576+0000","last_clean":"2024-09-16T15:53:49.038576+0000","last_became_active":"2024-09-16T15:53:47.048911+0000","last_became_peered":"2024-09-16T15:53:47.048911+0000","last_unstale":"2024-09-16T15:53:49.038576+0000","last_undegraded":"2024-09-16T15:53:49.038576+0000","last_fullsized":"2024-09-16T15:53:49.038576+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:13:16.281809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.f","version":"132'1","reported_seq":16,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.036692+0000","last_change":"2024-09-16T15:53:46.041954+0000","last_active":"2024-09-16T15:53:48.036692+0000","last_peered":"2024-09-16T15:53:48.036692+0000","last_clean":"2024-09-16T15:53:48.036692+0000","last_became_active":"2024-09-16T15:53:46.041272+0000","last_became_peered":"2024-09-16T15:53:46.041272+0000","last_unstale":"2024-09-16T15:53:48.036692+0000","last_undegraded":"2024-09-16T15:53:48.036692+0000","last_fullsized":"2024-09-16T15:53:48.036692+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:30:51.150005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.038733+0000","last_change":"2024-09-16T15:53:47.052264+0000","last_active":"2024-09-16T15:53:49.038733+0000","last_peered":"2024-09-16T15:53:49.038733+0000","last_clean":"2024-09-16T15:53:49.038733+0000","last_became_active":"2024-09-16T15:53:47.051995+0000","last_became_peered":"2024-09-16T15:53:47.051995+0000","last_unstale":"2024-09-16T15:53:49.038733+0000","last_undegraded":"2024-09-16T15:53:49.038733+0000","last_fullsized":"2024-09-16T15:53:49.038733+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:52:00.354801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"132'2","reported_seq":17,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.036582+0000","last_change":"2024-09-16T15:53:46.048066+0000","last_active":"2024-09-16T15:53:48.036582+0000","last_peered":"2024-09-16T15:53:48.036582+0000","last_clean":"2024-09-16T15:53:48.036582+0000","last_became_active":"2024-09-16T15:53:46.047765+0000","last_became_peered":"2024-09-16T15:53:46.047765+0000","last_unstale":"2024-09-16T15:53:48.036582+0000","last_undegraded":"2024-09-16T15:53:48.036582+0000","last_fullsized":"2024-09-16T15:53:48.036582+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:25:07.219846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.274731+0000","last_change":"2024-09-16T15:53:47.054306+0000","last_active":"2024-09-16T15:53:49.274731+0000","last_peered":"2024-09-16T15:53:49.274731+0000","last_clean":"2024-09-16T15:53:49.274731+0000","last_became_active":"2024-09-16T15:53:47.054002+0000","last_became_peered":"2024-09-16T15:53:47.054002+0000","last_unstale":"2024-09-16T15:53:49.274731+0000","last_undegraded":"2024-09-16T15:53:49.274731+0000","last_fullsized":"2024-09-16T15:53:49.274731+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:25:48.625685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.d","version":"132'1","reported_seq":16,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.604918+0000","last_change":"2024-09-16T15:53:46.050413+0000","last_active":"2024-09-16T15:53:48.604918+0000","last_peered":"2024-09-16T15:53:48.604918+0000","last_clean":"2024-09-16T15:53:48.604918+0000","last_became_active":"2024-09-16T15:53:46.050136+0000","last_became_peered":"2024-09-16T15:53:46.050136+0000","last_unstale":"2024-09-16T15:53:48.604918+0000","last_undegraded":"2024-09-16T15:53:48.604918+0000","last_fullsized":"2024-09-16T15:53:48.604918+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:35:42.185986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.274929+0000","last_change":"2024-09-16T15:53:47.055311+0000","last_active":"2024-09-16T15:53:49.274929+0000","last_peered":"2024-09-16T15:53:49.274929+0000","last_clean":"2024-09-16T15:53:49.274929+0000","last_became_active":"2024-09-16T15:53:47.053613+0000","last_became_peered":"2024-09-16T15:53:47.053613+0000","last_unstale":"2024-09-16T15:53:49.274929+0000","last_undegraded":"2024-09-16T15:53:49.274929+0000","last_fullsized":"2024-09-16T15:53:49.274929+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:52:21.045751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.038449+0000","last_change":"2024-09-16T15:53:46.052237+0000","last_active":"2024-09-16T15:53:49.038449+0000","last_peered":"2024-09-16T15:53:49.038449+0000","last_clean":"2024-09-16T15:53:49.038449+0000","last_became_active":"2024-09-16T15:53:46.051939+0000","last_became_peered":"2024-09-16T15:53:46.051939+0000","last_unstale":"2024-09-16T15:53:49.038449+0000","last_undegraded":"2024-09-16T15:53:49.038449+0000","last_fullsized":"2024-09-16T15:53:49.038449+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:43:15.863610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.274616+0000","last_change":"2024-09-16T15:53:47.055016+0000","last_active":"2024-09-16T15:53:49.274616+0000","last_peered":"2024-09-16T15:53:49.274616+0000","last_clean":"2024-09-16T15:53:49.274616+0000","last_became_active":"2024-09-16T15:53:47.054029+0000","last_became_peered":"2024-09-16T15:53:47.054029+0000","last_unstale":"2024-09-16T15:53:49.274616+0000","last_undegraded":"2024-09-16T15:53:49.274616+0000","last_fullsized":"2024-09-16T15:53:49.274616+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:39:06.152472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.b","version":"132'2","reported_seq":17,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.604585+0000","last_change":"2024-09-16T15:53:46.049709+0000","last_active":"2024-09-16T15:53:48.604585+0000","last_peered":"2024-09-16T15:53:48.604585+0000","last_clean":"2024-09-16T15:53:48.604585+0000","last_became_active":"2024-09-16T15:53:46.049412+0000","last_became_peered":"2024-09-16T15:53:46.049412+0000","last_unstale":"2024-09-16T15:53:48.604585+0000","last_undegraded":"2024-09-16T15:53:48.604585+0000","last_fullsized":"2024-09-16T15:53:48.604585+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:35:48.612183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.273696+0000","last_change":"2024-09-16T15:53:47.052068+0000","last_active":"2024-09-16T15:53:49.273696+0000","last_peered":"2024-09-16T15:53:49.273696+0000","last_clean":"2024-09-16T15:53:49.273696+0000","last_became_active":"2024-09-16T15:53:47.051755+0000","last_became_peered":"2024-09-16T15:53:47.051755+0000","last_unstale":"2024-09-16T15:53:49.273696+0000","last_undegraded":"2024-09-16T15:53:49.273696+0000","last_fullsized":"2024-09-16T15:53:49.273696+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:30:26.898180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.481002+0000","last_change":"2024-09-16T15:53:46.051522+0000","last_active":"2024-09-16T15:53:48.481002+0000","last_peered":"2024-09-16T15:53:48.481002+0000","last_clean":"2024-09-16T15:53:48.481002+0000","last_became_active":"2024-09-16T15:53:46.051005+0000","last_became_peered":"2024-09-16T15:53:46.051005+0000","last_unstale":"2024-09-16T15:53:48.481002+0000","last_undegraded":"2024-09-16T15:53:48.481002+0000","last_fullsized":"2024-09-16T15:53:48.481002+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:10:36.900762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":10,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-16T15:53:47.063249+0000","last_change":"2024-09-16T15:53:47.058656+0000","last_active":"2024-09-16T15:53:47.063249+0000","last_peered":"2024-09-16T15:53:47.063249+0000","last_clean":"2024-09-16T15:53:47.063249+0000","last_became_active":"2024-09-16T15:53:47.057260+0000","last_became_peered":"2024-09-16T15:53:47.057260+0000","last_unstale":"2024-09-16T15:53:47.063249+0000","last_undegraded":"2024-09-16T15:53:47.063249+0000","last_fullsized":"2024-09-16T15:53:47.063249+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:18:36.519474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-16T15:53:47.041902+0000","last_change":"2024-09-16T15:53:46.038714+0000","last_active":"2024-09-16T15:53:47.041902+0000","last_peered":"2024-09-16T15:53:47.041902+0000","last_clean":"2024-09-16T15:53:47.041902+0000","last_became_active":"2024-09-16T15:53:46.038411+0000","last_became_peered":"2024-09-16T15:53:46.038411+0000","last_unstale":"2024-09-16T15:53:47.041902+0000","last_undegraded":"2024-09-16T15:53:47.041902+0000","last_fullsized":"2024-09-16T15:53:47.041902+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:43:20.313472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.035857+0000","last_change":"2024-09-16T15:53:47.065949+0000","last_active":"2024-09-16T15:53:48.035857+0000","last_peered":"2024-09-16T15:53:48.035857+0000","last_clean":"2024-09-16T15:53:48.035857+0000","last_became_active":"2024-09-16T15:53:47.064802+0000","last_became_peered":"2024-09-16T15:53:47.064802+0000","last_unstale":"2024-09-16T15:53:48.035857+0000","last_undegraded":"2024-09-16T15:53:48.035857+0000","last_fullsized":"2024-09-16T15:53:48.035857+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:09:34.576069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.035763+0000","last_change":"2024-09-16T15:53:46.049975+0000","last_active":"2024-09-16T15:53:48.035763+0000","last_peered":"2024-09-16T15:53:48.035763+0000","last_clean":"2024-09-16T15:53:48.035763+0000","last_became_active":"2024-09-16T15:53:46.049667+0000","last_became_peered":"2024-09-16T15:53:46.049667+0000","last_unstale":"2024-09-16T15:53:48.035763+0000","last_undegraded":"2024-09-16T15:53:48.035763+0000","last_fullsized":"2024-09-16T15:53:48.035763+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:50:23.146038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.273923+0000","last_change":"2024-09-16T15:53:47.066513+0000","last_active":"2024-09-16T15:53:49.273923+0000","last_peered":"2024-09-16T15:53:49.273923+0000","last_clean":"2024-09-16T15:53:49.273923+0000","last_became_active":"2024-09-16T15:53:47.066252+0000","last_became_peered":"2024-09-16T15:53:47.066252+0000","last_unstale":"2024-09-16T15:53:49.273923+0000","last_undegraded":"2024-09-16T15:53:49.273923+0000","last_fullsized":"2024-09-16T15:53:49.273923+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:18:41.717848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.7","version":"132'2","reported_seq":17,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.480957+0000","last_change":"2024-09-16T15:53:46.051198+0000","last_active":"2024-09-16T15:53:48.480957+0000","last_peered":"2024-09-16T15:53:48.480957+0000","last_clean":"2024-09-16T15:53:48.480957+0000","last_became_active":"2024-09-16T15:53:46.050837+0000","last_became_peered":"2024-09-16T15:53:46.050837+0000","last_unstale":"2024-09-16T15:53:48.480957+0000","last_undegraded":"2024-09-16T15:53:48.480957+0000","last_fullsized":"2024-09-16T15:53:48.480957+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:38:17.885868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.274984+0000","last_change":"2024-09-16T15:53:47.060499+0000","last_active":"2024-09-16T15:53:49.274984+0000","last_peered":"2024-09-16T15:53:49.274984+0000","last_clean":"2024-09-16T15:53:49.274984+0000","last_became_active":"2024-09-16T15:53:47.059699+0000","last_became_peered":"2024-09-16T15:53:47.059699+0000","last_unstale":"2024-09-16T15:53:49.274984+0000","last_undegraded":"2024-09-16T15:53:49.274984+0000","last_fullsized":"2024-09-16T15:53:49.274984+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:35:38.397852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.6","version":"133'1","reported_seq":16,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.038798+0000","last_change":"2024-09-16T15:53:46.055909+0000","last_active":"2024-09-16T15:53:49.038798+0000","last_peered":"2024-09-16T15:53:49.038798+0000","last_clean":"2024-09-16T15:53:49.038798+0000","last_became_active":"2024-09-16T15:53:46.055567+0000","last_became_peered":"2024-09-16T15:53:46.055567+0000","last_unstale":"2024-09-16T15:53:49.038798+0000","last_undegraded":"2024-09-16T15:53:49.038798+0000","last_fullsized":"2024-09-16T15:53:49.038798+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:27:29.536408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":10,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-16T15:53:47.058902+0000","last_change":"2024-09-16T15:53:47.055081+0000","last_active":"2024-09-16T15:53:47.058902+0000","last_peered":"2024-09-16T15:53:47.058902+0000","last_clean":"2024-09-16T15:53:47.058902+0000","last_became_active":"2024-09-16T15:53:47.054772+0000","last_became_peered":"2024-09-16T15:53:47.054772+0000","last_unstale":"2024-09-16T15:53:47.058902+0000","last_undegraded":"2024-09-16T15:53:47.058902+0000","last_fullsized":"2024-09-16T15:53:47.058902+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:36:50.843181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.481141+0000","last_change":"2024-09-16T15:53:46.056629+0000","last_active":"2024-09-16T15:53:48.481141+0000","last_peered":"2024-09-16T15:53:48.481141+0000","last_clean":"2024-09-16T15:53:48.481141+0000","last_became_active":"2024-09-16T15:53:46.056360+0000","last_became_peered":"2024-09-16T15:53:46.056360+0000","last_unstale":"2024-09-16T15:53:48.481141+0000","last_undegraded":"2024-09-16T15:53:48.481141+0000","last_fullsized":"2024-09-16T15:53:48.481141+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:57:33.513974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.604813+0000","last_change":"2024-09-16T15:53:47.048289+0000","last_active":"2024-09-16T15:53:48.604813+0000","last_peered":"2024-09-16T15:53:48.604813+0000","last_clean":"2024-09-16T15:53:48.604813+0000","last_became_active":"2024-09-16T15:53:47.047818+0000","last_became_peered":"2024-09-16T15:53:47.047818+0000","last_unstale":"2024-09-16T15:53:48.604813+0000","last_undegraded":"2024-09-16T15:53:48.604813+0000","last_fullsized":"2024-09-16T15:53:48.604813+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:52:33.800592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-16T15:53:47.042871+0000","last_change":"2024-09-16T15:53:46.057982+0000","last_active":"2024-09-16T15:53:47.042871+0000","last_peered":"2024-09-16T15:53:47.042871+0000","last_clean":"2024-09-16T15:53:47.042871+0000","last_became_active":"2024-09-16T15:53:46.057707+0000","last_became_peered":"2024-09-16T15:53:46.057707+0000","last_unstale":"2024-09-16T15:53:47.042871+0000","last_undegraded":"2024-09-16T15:53:47.042871+0000","last_fullsized":"2024-09-16T15:53:47.042871+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:59:40.544624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.035623+0000","last_change":"2024-09-16T15:53:47.062803+0000","last_active":"2024-09-16T15:53:48.035623+0000","last_peered":"2024-09-16T15:53:48.035623+0000","last_clean":"2024-09-16T15:53:48.035623+0000","last_became_active":"2024-09-16T15:53:47.060237+0000","last_became_peered":"2024-09-16T15:53:47.060237+0000","last_unstale":"2024-09-16T15:53:48.035623+0000","last_undegraded":"2024-09-16T15:53:48.035623+0000","last_fullsized":"2024-09-16T15:53:48.035623+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:52:21.045751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.274595+0000","last_change":"2024-09-16T15:53:46.052804+0000","last_active":"2024-09-16T15:53:49.274595+0000","last_peered":"2024-09-16T15:53:49.274595+0000","last_clean":"2024-09-16T15:53:49.274595+0000","last_became_active":"2024-09-16T15:53:46.052460+0000","last_became_peered":"2024-09-16T15:53:46.052460+0000","last_unstale":"2024-09-16T15:53:49.274595+0000","last_undegraded":"2024-09-16T15:53:49.274595+0000","last_fullsized":"2024-09-16T15:53:49.274595+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:36:04.003482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.036048+0000","last_change":"2024-09-16T15:53:47.065967+0000","last_active":"2024-09-16T15:53:48.036048+0000","last_peered":"2024-09-16T15:53:48.036048+0000","last_clean":"2024-09-16T15:53:48.036048+0000","last_became_active":"2024-09-16T15:53:47.065715+0000","last_became_peered":"2024-09-16T15:53:47.065715+0000","last_unstale":"2024-09-16T15:53:48.036048+0000","last_undegraded":"2024-09-16T15:53:48.036048+0000","last_fullsized":"2024-09-16T15:53:48.036048+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:48:44.181946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.035831+0000","last_change":"2024-09-16T15:53:46.044167+0000","last_active":"2024-09-16T15:53:48.035831+0000","last_peered":"2024-09-16T15:53:48.035831+0000","last_clean":"2024-09-16T15:53:48.035831+0000","last_became_active":"2024-09-16T15:53:46.043914+0000","last_became_peered":"2024-09-16T15:53:46.043914+0000","last_unstale":"2024-09-16T15:53:48.035831+0000","last_undegraded":"2024-09-16T15:53:48.035831+0000","last_fullsized":"2024-09-16T15:53:48.035831+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:20:21.174844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.481211+0000","last_change":"2024-09-16T15:53:47.069960+0000","last_active":"2024-09-16T15:53:48.481211+0000","last_peered":"2024-09-16T15:53:48.481211+0000","last_clean":"2024-09-16T15:53:48.481211+0000","last_became_active":"2024-09-16T15:53:47.069614+0000","last_became_peered":"2024-09-16T15:53:47.069614+0000","last_unstale":"2024-09-16T15:53:48.481211+0000","last_undegraded":"2024-09-16T15:53:48.481211+0000","last_fullsized":"2024-09-16T15:53:48.481211+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:25:21.431101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.035927+0000","last_change":"2024-09-16T15:53:46.055377+0000","last_active":"2024-09-16T15:53:48.035927+0000","last_peered":"2024-09-16T15:53:48.035927+0000","last_clean":"2024-09-16T15:53:48.035927+0000","last_became_active":"2024-09-16T15:53:46.055135+0000","last_became_peered":"2024-09-16T15:53:46.055135+0000","last_unstale":"2024-09-16T15:53:48.035927+0000","last_undegraded":"2024-09-16T15:53:48.035927+0000","last_fullsized":"2024-09-16T15:53:48.035927+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:04:47.485315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":10,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-16T15:53:47.062177+0000","last_change":"2024-09-16T15:53:47.057817+0000","last_active":"2024-09-16T15:53:47.062177+0000","last_peered":"2024-09-16T15:53:47.062177+0000","last_clean":"2024-09-16T15:53:47.062177+0000","last_became_active":"2024-09-16T15:53:47.057440+0000","last_became_peered":"2024-09-16T15:53:47.057440+0000","last_unstale":"2024-09-16T15:53:47.062177+0000","last_undegraded":"2024-09-16T15:53:47.062177+0000","last_fullsized":"2024-09-16T15:53:47.062177+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:56:27.484880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-16T15:53:47.043225+0000","last_change":"2024-09-16T15:53:46.047710+0000","last_active":"2024-09-16T15:53:47.043225+0000","last_peered":"2024-09-16T15:53:47.043225+0000","last_clean":"2024-09-16T15:53:47.043225+0000","last_became_active":"2024-09-16T15:53:46.047320+0000","last_became_peered":"2024-09-16T15:53:46.047320+0000","last_unstale":"2024-09-16T15:53:47.043225+0000","last_undegraded":"2024-09-16T15:53:47.043225+0000","last_fullsized":"2024-09-16T15:53:47.043225+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:40:37.494885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.604526+0000","last_change":"2024-09-16T15:53:47.039385+0000","last_active":"2024-09-16T15:53:48.604526+0000","last_peered":"2024-09-16T15:53:48.604526+0000","last_clean":"2024-09-16T15:53:48.604526+0000","last_became_active":"2024-09-16T15:53:47.039077+0000","last_became_peered":"2024-09-16T15:53:47.039077+0000","last_unstale":"2024-09-16T15:53:48.604526+0000","last_undegraded":"2024-09-16T15:53:48.604526+0000","last_fullsized":"2024-09-16T15:53:48.604526+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:30:30.250921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.13","version":"132'2","reported_seq":17,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.036176+0000","last_change":"2024-09-16T15:53:46.048215+0000","last_active":"2024-09-16T15:53:48.036176+0000","last_peered":"2024-09-16T15:53:48.036176+0000","last_clean":"2024-09-16T15:53:48.036176+0000","last_became_active":"2024-09-16T15:53:46.045488+0000","last_became_peered":"2024-09-16T15:53:46.045488+0000","last_unstale":"2024-09-16T15:53:48.036176+0000","last_undegraded":"2024-09-16T15:53:48.036176+0000","last_fullsized":"2024-09-16T15:53:48.036176+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:59:52.882286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":323,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.273491+0000","last_change":"2024-09-16T15:39:12.698813+0000","last_active":"2024-09-16T15:53:49.273491+0000","last_peered":"2024-09-16T15:53:49.273491+0000","last_clean":"2024-09-16T15:53:49.273491+0000","last_became_active":"2024-09-16T15:39:12.697695+0000","last_became_peered":"2024-09-16T15:39:12.697695+0000","last_unstale":"2024-09-16T15:53:49.273491+0000","last_undegraded":"2024-09-16T15:53:49.273491+0000","last_fullsized":"2024-09-16T15:53:49.273491+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_clean_scrub_stamp":"2024-09-16T15:39:11.684967+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:02.061081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509620,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":159068,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":157438,"kb_avail":94212772,"statfs":{"total":96636764160,"available":96473878528,"internally_reserved":0,"allocated":1638400,"data_stored":723164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":161216968},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509620,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":119408,"kb_used_data":2204,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94252432,"statfs":{"total":96636764160,"available":96514490368,"internally_reserved":0,"allocated":2256896,"data_stored":1320686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":119994823},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509619,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":136864,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94234976,"statfs":{"total":96636764160,"available":96496615424,"internally_reserved":0,"allocated":1650688,"data_stored":729576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":138475973},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509619,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":118820,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94253020,"statfs":{"total":96636764160,"available":96515092480,"internally_reserved":0,"allocated":1654784,"data_stored":729646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":119994825},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509619,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131308,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240532,"statfs":{"total":96636764160,"available":96502304768,"internally_reserved":0,"allocated":1658880,"data_stored":730248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509620,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131892,"kb_used_data":2208,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94239948,"statfs":{"total":96636764160,"available":96501706752,"internally_reserved":0,"allocated":2260992,"data_stored":1321578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509618,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149364,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222476,"statfs":{"total":96636764160,"available":96483815424,"internally_reserved":0,"allocated":1671168,"data_stored":731244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":151255495},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542325,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131304,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240536,"statfs":{"total":96636764160,"available":96502308864,"internally_reserved":0,"allocated":1654784,"data_stored":729610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":132774348},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:53:55.968 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:53:56.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.137+0000 7f9343200640 1 -- 172.21.15.151:0/342904851 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f93440725d0 msgr2=0x7f9344073d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:56.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.137+0000 7f9343200640 1 --2- 172.21.15.151:0/342904851 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f93440725d0 0x7f9344073d40 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f9330005fd0 tx=0x7f933002a5d0 comp rx=0 tx=0).stop 2024-09-16T15:53:56.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.137+0000 7f9343200640 1 -- 172.21.15.151:0/342904851 shutdown_connections 2024-09-16T15:53:56.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.137+0000 7f9343200640 1 --2- 172.21.15.151:0/342904851 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f93440725d0 0x7f9344073d40 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:56.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.137+0000 7f9343200640 1 --2- 172.21.15.151:0/342904851 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f934415bc30 0x7f93440b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:56.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.137+0000 7f9343200640 1 -- 172.21.15.151:0/342904851 >> 172.21.15.151:0/342904851 conn(0x7f93440cfa20 msgr2=0x7f93440cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:56.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.137+0000 7f9343200640 1 -- 172.21.15.151:0/342904851 shutdown_connections 2024-09-16T15:53:56.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.137+0000 7f9343200640 1 -- 172.21.15.151:0/342904851 wait complete. 2024-09-16T15:53:56.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.139+0000 7f9343200640 1 Processor -- start 2024-09-16T15:53:56.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.139+0000 7f9343200640 1 -- start start 2024-09-16T15:53:56.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.140+0000 7f9343200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f93440d0540 0x7f93441f48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:56.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.140+0000 7f9343200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f934415bc30 0x7f93440ca030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:56.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.140+0000 7f9343200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f93440b4df0 0x7f93440b52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:56.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.140+0000 7f9343200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9344074610 con 0x7f93440d0540 2024-09-16T15:53:56.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.140+0000 7f9343200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9344074490 con 0x7f93440b4df0 2024-09-16T15:53:56.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.140+0000 7f9343200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f9344074790 con 0x7f934415bc30 2024-09-16T15:53:56.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.140+0000 7f9341e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f93440d0540 0x7f93441f48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:56.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.140+0000 7f9341e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f93440d0540 0x7f93441f48c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:54768/0 (socket says 172.21.15.151:54768) 2024-09-16T15:53:56.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.140+0000 7f9341e00640 1 -- 172.21.15.151:0/1859810266 learned_addr learned my addr 172.21.15.151:0/1859810266 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:53:56.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.140+0000 7f9341400640 1 --2- 172.21.15.151:0/1859810266 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f934415bc30 0x7f93440ca030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:56.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.140+0000 7f9342800640 1 --2- 172.21.15.151:0/1859810266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f93440b4df0 0x7f93440b52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:56.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.141+0000 7f9341e00640 1 -- 172.21.15.151:0/1859810266 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f934415bc30 msgr2=0x7f93440ca030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:56.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.141+0000 7f9341e00640 1 --2- 172.21.15.151:0/1859810266 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f934415bc30 0x7f93440ca030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:56.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.141+0000 7f9341e00640 1 -- 172.21.15.151:0/1859810266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f93440b4df0 msgr2=0x7f93440b52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:56.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.141+0000 7f9341e00640 1 --2- 172.21.15.151:0/1859810266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f93440b4df0 0x7f93440b52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:56.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.141+0000 7f9341e00640 1 -- 172.21.15.151:0/1859810266 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f934415e610 con 0x7f93440d0540 2024-09-16T15:53:56.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.141+0000 7f9342800640 1 --2- 172.21.15.151:0/1859810266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f93440b4df0 0x7f93440b52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:53:56.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.141+0000 7f9341e00640 1 --2- 172.21.15.151:0/1859810266 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f93440d0540 0x7f93441f48c0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f933400e790 tx=0x7f933400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:56.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.141+0000 7f933a000640 1 -- 172.21.15.151:0/1859810266 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f933400eea0 con 0x7f93440d0540 2024-09-16T15:53:56.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.141+0000 7f933a000640 1 -- 172.21.15.151:0/1859810266 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f933400cdb0 con 0x7f93440d0540 2024-09-16T15:53:56.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.141+0000 7f933a000640 1 -- 172.21.15.151:0/1859810266 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9334010690 con 0x7f93440d0540 2024-09-16T15:53:56.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.141+0000 7f9343200640 1 -- 172.21.15.151:0/1859810266 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f934415e8f0 con 0x7f93440d0540 2024-09-16T15:53:56.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.141+0000 7f9343200640 1 -- 172.21.15.151:0/1859810266 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f934415eeb0 con 0x7f93440d0540 2024-09-16T15:53:56.146 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.142+0000 7f9343200640 1 -- 172.21.15.151:0/1859810266 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f934406fc60 con 0x7f93440d0540 2024-09-16T15:53:56.146 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.144+0000 7f933a000640 1 -- 172.21.15.151:0/1859810266 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 47) ==== 104092+0+0 (secure 0 0 0) 0x7f9334010830 con 0x7f93440d0540 2024-09-16T15:53:56.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.145+0000 7f933a000640 1 --2- 172.21.15.151:0/1859810266 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f932407a740 0x7f932407cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:56.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.145+0000 7f933a000640 1 -- 172.21.15.151:0/1859810266 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5821+0+0 (secure 0 0 0) 0x7f933409a2f0 con 0x7f93440d0540 2024-09-16T15:53:56.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.145+0000 7f9341400640 1 --2- 172.21.15.151:0/1859810266 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f932407a740 0x7f932407cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:56.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.146+0000 7f9341400640 1 --2- 172.21.15.151:0/1859810266 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f932407a740 0x7f932407cc00 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f9330033000 tx=0x7f9330004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:56.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.149+0000 7f933a000640 1 -- 172.21.15.151:0/1859810266 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9334063d70 con 0x7f93440d0540 2024-09-16T15:53:56.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.330+0000 7f9343200640 1 -- 172.21.15.151:0/1859810266 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f93441532d0 con 0x7f932407a740 2024-09-16T15:53:56.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.335+0000 7f933a000640 1 -- 172.21.15.151:0/1859810266 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252487 (secure 0 0 0) 0x7f93441532d0 con 0x7f932407a740 2024-09-16T15:53:56.340 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:53:56.341 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-09-16T15:53:56.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.339+0000 7f9343200640 1 -- 172.21.15.151:0/1859810266 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f932407a740 msgr2=0x7f932407cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:56.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.339+0000 7f9343200640 1 --2- 172.21.15.151:0/1859810266 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f932407a740 0x7f932407cc00 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f9330033000 tx=0x7f9330004730 comp rx=0 tx=0).stop 2024-09-16T15:53:56.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.339+0000 7f9343200640 1 -- 172.21.15.151:0/1859810266 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f93440d0540 msgr2=0x7f93441f48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:56.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.339+0000 7f9343200640 1 --2- 172.21.15.151:0/1859810266 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f93440d0540 0x7f93441f48c0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f933400e790 tx=0x7f933400ec50 comp rx=0 tx=0).stop 2024-09-16T15:53:56.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.339+0000 7f9343200640 1 -- 172.21.15.151:0/1859810266 shutdown_connections 2024-09-16T15:53:56.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.339+0000 7f9343200640 1 --2- 172.21.15.151:0/1859810266 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f932407a740 0x7f932407cc00 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:56.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.339+0000 7f9343200640 1 --2- 172.21.15.151:0/1859810266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f93440b4df0 0x7f93440b52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:56.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.339+0000 7f9343200640 1 --2- 172.21.15.151:0/1859810266 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f934415bc30 0x7f93440ca030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:56.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.339+0000 7f9343200640 1 --2- 172.21.15.151:0/1859810266 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f93440d0540 0x7f93441f48c0 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:56.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.339+0000 7f9343200640 1 -- 172.21.15.151:0/1859810266 >> 172.21.15.151:0/1859810266 conn(0x7f93440cfa20 msgr2=0x7f93440ca9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:56.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.339+0000 7f9343200640 1 -- 172.21.15.151:0/1859810266 shutdown_connections 2024-09-16T15:53:56.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.339+0000 7f9343200640 1 -- 172.21.15.151:0/1859810266 wait complete. 2024-09-16T15:53:56.357 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":569,"stamp":"2024-09-16T15:53:55.938231+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":183,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1078032,"kb_used_data":14120,"kb_used_omap":12,"kb_used_meta":1063731,"kb_avail":753896688,"statfs":{"total":773094113280,"available":771990208512,"internally_reserved":0,"allocated":14458880,"data_stored":7022164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12738,"internal_metadata":1089261118},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.906835"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.274843+0000","last_change":"2024-09-16T15:53:47.064079+0000","last_active":"2024-09-16T15:53:49.274843+0000","last_peered":"2024-09-16T15:53:49.274843+0000","last_clean":"2024-09-16T15:53:49.274843+0000","last_became_active":"2024-09-16T15:53:47.063709+0000","last_became_peered":"2024-09-16T15:53:47.063709+0000","last_unstale":"2024-09-16T15:53:49.274843+0000","last_undegraded":"2024-09-16T15:53:49.274843+0000","last_fullsized":"2024-09-16T15:53:49.274843+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:52:00.354801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.274872+0000","last_change":"2024-09-16T15:53:47.054602+0000","last_active":"2024-09-16T15:53:49.274872+0000","last_peered":"2024-09-16T15:53:49.274872+0000","last_clean":"2024-09-16T15:53:49.274872+0000","last_became_active":"2024-09-16T15:53:47.054286+0000","last_became_peered":"2024-09-16T15:53:47.054286+0000","last_unstale":"2024-09-16T15:53:49.274872+0000","last_undegraded":"2024-09-16T15:53:49.274872+0000","last_fullsized":"2024-09-16T15:53:49.274872+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:18:36.519474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.604743+0000","last_change":"2024-09-16T15:53:47.049667+0000","last_active":"2024-09-16T15:53:48.604743+0000","last_peered":"2024-09-16T15:53:48.604743+0000","last_clean":"2024-09-16T15:53:48.604743+0000","last_became_active":"2024-09-16T15:53:47.048431+0000","last_became_peered":"2024-09-16T15:53:47.048431+0000","last_unstale":"2024-09-16T15:53:48.604743+0000","last_undegraded":"2024-09-16T15:53:48.604743+0000","last_fullsized":"2024-09-16T15:53:48.604743+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:25:27.052540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.036138+0000","last_change":"2024-09-16T15:53:47.066311+0000","last_active":"2024-09-16T15:53:48.036138+0000","last_peered":"2024-09-16T15:53:48.036138+0000","last_clean":"2024-09-16T15:53:48.036138+0000","last_became_active":"2024-09-16T15:53:47.066005+0000","last_became_peered":"2024-09-16T15:53:47.066005+0000","last_unstale":"2024-09-16T15:53:48.036138+0000","last_undegraded":"2024-09-16T15:53:48.036138+0000","last_fullsized":"2024-09-16T15:53:48.036138+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:27:17.121069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.038493+0000","last_change":"2024-09-16T15:53:47.067568+0000","last_active":"2024-09-16T15:53:49.038493+0000","last_peered":"2024-09-16T15:53:49.038493+0000","last_clean":"2024-09-16T15:53:49.038493+0000","last_became_active":"2024-09-16T15:53:47.067220+0000","last_became_peered":"2024-09-16T15:53:47.067220+0000","last_unstale":"2024-09-16T15:53:49.038493+0000","last_undegraded":"2024-09-16T15:53:49.038493+0000","last_fullsized":"2024-09-16T15:53:49.038493+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:39:06.152472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.035632+0000","last_change":"2024-09-16T15:53:47.066404+0000","last_active":"2024-09-16T15:53:48.035632+0000","last_peered":"2024-09-16T15:53:48.035632+0000","last_clean":"2024-09-16T15:53:48.035632+0000","last_became_active":"2024-09-16T15:53:47.064379+0000","last_became_peered":"2024-09-16T15:53:47.064379+0000","last_unstale":"2024-09-16T15:53:48.035632+0000","last_undegraded":"2024-09-16T15:53:48.035632+0000","last_fullsized":"2024-09-16T15:53:48.035632+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:57:58.563630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.038351+0000","last_change":"2024-09-16T15:53:47.049576+0000","last_active":"2024-09-16T15:53:49.038351+0000","last_peered":"2024-09-16T15:53:49.038351+0000","last_clean":"2024-09-16T15:53:49.038351+0000","last_became_active":"2024-09-16T15:53:47.049291+0000","last_became_peered":"2024-09-16T15:53:47.049291+0000","last_unstale":"2024-09-16T15:53:49.038351+0000","last_undegraded":"2024-09-16T15:53:49.038351+0000","last_fullsized":"2024-09-16T15:53:49.038351+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:10:04.681002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.038227+0000","last_change":"2024-09-16T15:53:47.054562+0000","last_active":"2024-09-16T15:53:49.038227+0000","last_peered":"2024-09-16T15:53:49.038227+0000","last_clean":"2024-09-16T15:53:49.038227+0000","last_became_active":"2024-09-16T15:53:47.054310+0000","last_became_peered":"2024-09-16T15:53:47.054310+0000","last_unstale":"2024-09-16T15:53:49.038227+0000","last_undegraded":"2024-09-16T15:53:49.038227+0000","last_fullsized":"2024-09-16T15:53:49.038227+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:36:01.782108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.604647+0000","last_change":"2024-09-16T15:53:47.045614+0000","last_active":"2024-09-16T15:53:48.604647+0000","last_peered":"2024-09-16T15:53:48.604647+0000","last_clean":"2024-09-16T15:53:48.604647+0000","last_became_active":"2024-09-16T15:53:47.045131+0000","last_became_peered":"2024-09-16T15:53:47.045131+0000","last_unstale":"2024-09-16T15:53:48.604647+0000","last_undegraded":"2024-09-16T15:53:48.604647+0000","last_fullsized":"2024-09-16T15:53:48.604647+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:25:39.938644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.274853+0000","last_change":"2024-09-16T15:53:47.063974+0000","last_active":"2024-09-16T15:53:49.274853+0000","last_peered":"2024-09-16T15:53:49.274853+0000","last_clean":"2024-09-16T15:53:49.274853+0000","last_became_active":"2024-09-16T15:53:47.063634+0000","last_became_peered":"2024-09-16T15:53:47.063634+0000","last_unstale":"2024-09-16T15:53:49.274853+0000","last_undegraded":"2024-09-16T15:53:49.274853+0000","last_fullsized":"2024-09-16T15:53:49.274853+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:22:44.163439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.036512+0000","last_change":"2024-09-16T15:53:47.045716+0000","last_active":"2024-09-16T15:53:48.036512+0000","last_peered":"2024-09-16T15:53:48.036512+0000","last_clean":"2024-09-16T15:53:48.036512+0000","last_became_active":"2024-09-16T15:53:47.045439+0000","last_became_peered":"2024-09-16T15:53:47.045439+0000","last_unstale":"2024-09-16T15:53:48.036512+0000","last_undegraded":"2024-09-16T15:53:48.036512+0000","last_fullsized":"2024-09-16T15:53:48.036512+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:40:26.604250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.036410+0000","last_change":"2024-09-16T15:53:47.070120+0000","last_active":"2024-09-16T15:53:48.036410+0000","last_peered":"2024-09-16T15:53:48.036410+0000","last_clean":"2024-09-16T15:53:48.036410+0000","last_became_active":"2024-09-16T15:53:47.068816+0000","last_became_peered":"2024-09-16T15:53:47.068816+0000","last_unstale":"2024-09-16T15:53:48.036410+0000","last_undegraded":"2024-09-16T15:53:48.036410+0000","last_fullsized":"2024-09-16T15:53:48.036410+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:32:42.788318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.036127+0000","last_change":"2024-09-16T15:53:47.052483+0000","last_active":"2024-09-16T15:53:48.036127+0000","last_peered":"2024-09-16T15:53:48.036127+0000","last_clean":"2024-09-16T15:53:48.036127+0000","last_became_active":"2024-09-16T15:53:47.052188+0000","last_became_peered":"2024-09-16T15:53:47.052188+0000","last_unstale":"2024-09-16T15:53:48.036127+0000","last_undegraded":"2024-09-16T15:53:48.036127+0000","last_fullsized":"2024-09-16T15:53:48.036127+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:28:45.284824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.034522+0000","last_change":"2024-09-16T15:53:47.056102+0000","last_active":"2024-09-16T15:53:48.034522+0000","last_peered":"2024-09-16T15:53:48.034522+0000","last_clean":"2024-09-16T15:53:48.034522+0000","last_became_active":"2024-09-16T15:53:47.055786+0000","last_became_peered":"2024-09-16T15:53:47.055786+0000","last_unstale":"2024-09-16T15:53:48.034522+0000","last_undegraded":"2024-09-16T15:53:48.034522+0000","last_fullsized":"2024-09-16T15:53:48.034522+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:30:28.946598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.481280+0000","last_change":"2024-09-16T15:53:47.053898+0000","last_active":"2024-09-16T15:53:48.481280+0000","last_peered":"2024-09-16T15:53:48.481280+0000","last_clean":"2024-09-16T15:53:48.481280+0000","last_became_active":"2024-09-16T15:53:47.053585+0000","last_became_peered":"2024-09-16T15:53:47.053585+0000","last_unstale":"2024-09-16T15:53:48.481280+0000","last_undegraded":"2024-09-16T15:53:48.481280+0000","last_fullsized":"2024-09-16T15:53:48.481280+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:00:53.660785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.034479+0000","last_change":"2024-09-16T15:53:47.060703+0000","last_active":"2024-09-16T15:53:48.034479+0000","last_peered":"2024-09-16T15:53:48.034479+0000","last_clean":"2024-09-16T15:53:48.034479+0000","last_became_active":"2024-09-16T15:53:47.060125+0000","last_became_peered":"2024-09-16T15:53:47.060125+0000","last_unstale":"2024-09-16T15:53:48.034479+0000","last_undegraded":"2024-09-16T15:53:48.034479+0000","last_fullsized":"2024-09-16T15:53:48.034479+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:04:06.435765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.604792+0000","last_change":"2024-09-16T15:53:47.047406+0000","last_active":"2024-09-16T15:53:48.604792+0000","last_peered":"2024-09-16T15:53:48.604792+0000","last_clean":"2024-09-16T15:53:48.604792+0000","last_became_active":"2024-09-16T15:53:47.045413+0000","last_became_peered":"2024-09-16T15:53:47.045413+0000","last_unstale":"2024-09-16T15:53:48.604792+0000","last_undegraded":"2024-09-16T15:53:48.604792+0000","last_fullsized":"2024-09-16T15:53:48.604792+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:24:19.371541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.481128+0000","last_change":"2024-09-16T15:53:47.049501+0000","last_active":"2024-09-16T15:53:48.481128+0000","last_peered":"2024-09-16T15:53:48.481128+0000","last_clean":"2024-09-16T15:53:48.481128+0000","last_became_active":"2024-09-16T15:53:47.046606+0000","last_became_peered":"2024-09-16T15:53:47.046606+0000","last_unstale":"2024-09-16T15:53:48.481128+0000","last_undegraded":"2024-09-16T15:53:48.481128+0000","last_fullsized":"2024-09-16T15:53:48.481128+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:00:13.468850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.274676+0000","last_change":"2024-09-16T15:53:47.055350+0000","last_active":"2024-09-16T15:53:49.274676+0000","last_peered":"2024-09-16T15:53:49.274676+0000","last_clean":"2024-09-16T15:53:49.274676+0000","last_became_active":"2024-09-16T15:53:47.051177+0000","last_became_peered":"2024-09-16T15:53:47.051177+0000","last_unstale":"2024-09-16T15:53:49.274676+0000","last_undegraded":"2024-09-16T15:53:49.274676+0000","last_fullsized":"2024-09-16T15:53:49.274676+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:13:16.281809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.481139+0000","last_change":"2024-09-16T15:53:47.065030+0000","last_active":"2024-09-16T15:53:48.481139+0000","last_peered":"2024-09-16T15:53:48.481139+0000","last_clean":"2024-09-16T15:53:48.481139+0000","last_became_active":"2024-09-16T15:53:47.064738+0000","last_became_peered":"2024-09-16T15:53:47.064738+0000","last_unstale":"2024-09-16T15:53:48.481139+0000","last_undegraded":"2024-09-16T15:53:48.481139+0000","last_fullsized":"2024-09-16T15:53:48.481139+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:07:01.165101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.035907+0000","last_change":"2024-09-16T15:53:47.063760+0000","last_active":"2024-09-16T15:53:48.035907+0000","last_peered":"2024-09-16T15:53:48.035907+0000","last_clean":"2024-09-16T15:53:48.035907+0000","last_became_active":"2024-09-16T15:53:47.063404+0000","last_became_peered":"2024-09-16T15:53:47.063404+0000","last_unstale":"2024-09-16T15:53:48.035907+0000","last_undegraded":"2024-09-16T15:53:48.035907+0000","last_fullsized":"2024-09-16T15:53:48.035907+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:30:28.946598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.274350+0000","last_change":"2024-09-16T15:53:47.044151+0000","last_active":"2024-09-16T15:53:49.274350+0000","last_peered":"2024-09-16T15:53:49.274350+0000","last_clean":"2024-09-16T15:53:49.274350+0000","last_became_active":"2024-09-16T15:53:47.043246+0000","last_became_peered":"2024-09-16T15:53:47.043246+0000","last_unstale":"2024-09-16T15:53:49.274350+0000","last_undegraded":"2024-09-16T15:53:49.274350+0000","last_fullsized":"2024-09-16T15:53:49.274350+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:14:07.626424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.273638+0000","last_change":"2024-09-16T15:53:47.051983+0000","last_active":"2024-09-16T15:53:49.273638+0000","last_peered":"2024-09-16T15:53:49.273638+0000","last_clean":"2024-09-16T15:53:49.273638+0000","last_became_active":"2024-09-16T15:53:47.050704+0000","last_became_peered":"2024-09-16T15:53:47.050704+0000","last_unstale":"2024-09-16T15:53:49.273638+0000","last_undegraded":"2024-09-16T15:53:49.273638+0000","last_fullsized":"2024-09-16T15:53:49.273638+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:35:02.303710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.480980+0000","last_change":"2024-09-16T15:53:47.069714+0000","last_active":"2024-09-16T15:53:48.480980+0000","last_peered":"2024-09-16T15:53:48.480980+0000","last_clean":"2024-09-16T15:53:48.480980+0000","last_became_active":"2024-09-16T15:53:47.069413+0000","last_became_peered":"2024-09-16T15:53:47.069413+0000","last_unstale":"2024-09-16T15:53:48.480980+0000","last_undegraded":"2024-09-16T15:53:48.480980+0000","last_fullsized":"2024-09-16T15:53:48.480980+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:27:29.156081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.036803+0000","last_change":"2024-09-16T15:53:47.053885+0000","last_active":"2024-09-16T15:53:48.036803+0000","last_peered":"2024-09-16T15:53:48.036803+0000","last_clean":"2024-09-16T15:53:48.036803+0000","last_became_active":"2024-09-16T15:53:47.052585+0000","last_became_peered":"2024-09-16T15:53:47.052585+0000","last_unstale":"2024-09-16T15:53:48.036803+0000","last_undegraded":"2024-09-16T15:53:48.036803+0000","last_fullsized":"2024-09-16T15:53:48.036803+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:52:32.247179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.275135+0000","last_change":"2024-09-16T15:53:47.047903+0000","last_active":"2024-09-16T15:53:49.275135+0000","last_peered":"2024-09-16T15:53:49.275135+0000","last_clean":"2024-09-16T15:53:49.275135+0000","last_became_active":"2024-09-16T15:53:47.047524+0000","last_became_peered":"2024-09-16T15:53:47.047524+0000","last_unstale":"2024-09-16T15:53:49.275135+0000","last_undegraded":"2024-09-16T15:53:49.275135+0000","last_fullsized":"2024-09-16T15:53:49.275135+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:28:37.082216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.035316+0000","last_change":"2024-09-16T15:53:47.067044+0000","last_active":"2024-09-16T15:53:48.035316+0000","last_peered":"2024-09-16T15:53:48.035316+0000","last_clean":"2024-09-16T15:53:48.035316+0000","last_became_active":"2024-09-16T15:53:47.066772+0000","last_became_peered":"2024-09-16T15:53:47.066772+0000","last_unstale":"2024-09-16T15:53:48.035316+0000","last_undegraded":"2024-09-16T15:53:48.035316+0000","last_fullsized":"2024-09-16T15:53:48.035316+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:50:24.154254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.273899+0000","last_change":"2024-09-16T15:53:47.047353+0000","last_active":"2024-09-16T15:53:49.273899+0000","last_peered":"2024-09-16T15:53:49.273899+0000","last_clean":"2024-09-16T15:53:49.273899+0000","last_became_active":"2024-09-16T15:53:47.047102+0000","last_became_peered":"2024-09-16T15:53:47.047102+0000","last_unstale":"2024-09-16T15:53:49.273899+0000","last_undegraded":"2024-09-16T15:53:49.273899+0000","last_fullsized":"2024-09-16T15:53:49.273899+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:21:20.787684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.274512+0000","last_change":"2024-09-16T15:53:47.046370+0000","last_active":"2024-09-16T15:53:49.274512+0000","last_peered":"2024-09-16T15:53:49.274512+0000","last_clean":"2024-09-16T15:53:49.274512+0000","last_became_active":"2024-09-16T15:53:47.046059+0000","last_became_peered":"2024-09-16T15:53:47.046059+0000","last_unstale":"2024-09-16T15:53:49.274512+0000","last_undegraded":"2024-09-16T15:53:49.274512+0000","last_fullsized":"2024-09-16T15:53:49.274512+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:10:04.681002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.274463+0000","last_change":"2024-09-16T15:53:47.049350+0000","last_active":"2024-09-16T15:53:49.274463+0000","last_peered":"2024-09-16T15:53:49.274463+0000","last_clean":"2024-09-16T15:53:49.274463+0000","last_became_active":"2024-09-16T15:53:47.048759+0000","last_became_peered":"2024-09-16T15:53:47.048759+0000","last_unstale":"2024-09-16T15:53:49.274463+0000","last_undegraded":"2024-09-16T15:53:49.274463+0000","last_fullsized":"2024-09-16T15:53:49.274463+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:36:01.782108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.273712+0000","last_change":"2024-09-16T15:53:47.050603+0000","last_active":"2024-09-16T15:53:49.273712+0000","last_peered":"2024-09-16T15:53:49.273712+0000","last_clean":"2024-09-16T15:53:49.273712+0000","last_became_active":"2024-09-16T15:53:47.050186+0000","last_became_peered":"2024-09-16T15:53:47.050186+0000","last_unstale":"2024-09-16T15:53:49.273712+0000","last_undegraded":"2024-09-16T15:53:49.273712+0000","last_fullsized":"2024-09-16T15:53:49.273712+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:49:30.590331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.604485+0000","last_change":"2024-09-16T15:53:47.049125+0000","last_active":"2024-09-16T15:53:48.604485+0000","last_peered":"2024-09-16T15:53:48.604485+0000","last_clean":"2024-09-16T15:53:48.604485+0000","last_became_active":"2024-09-16T15:53:47.048889+0000","last_became_peered":"2024-09-16T15:53:47.048889+0000","last_unstale":"2024-09-16T15:53:48.604485+0000","last_undegraded":"2024-09-16T15:53:48.604485+0000","last_fullsized":"2024-09-16T15:53:48.604485+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:06:42.055204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.035168+0000","last_change":"2024-09-16T15:53:47.056337+0000","last_active":"2024-09-16T15:53:48.035168+0000","last_peered":"2024-09-16T15:53:48.035168+0000","last_clean":"2024-09-16T15:53:48.035168+0000","last_became_active":"2024-09-16T15:53:47.056039+0000","last_became_peered":"2024-09-16T15:53:47.056039+0000","last_unstale":"2024-09-16T15:53:48.035168+0000","last_undegraded":"2024-09-16T15:53:48.035168+0000","last_fullsized":"2024-09-16T15:53:48.035168+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:26:21.116714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1f","version":"132'4","reported_seq":19,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.038798+0000","last_change":"2024-09-16T15:53:46.049579+0000","last_active":"2024-09-16T15:53:49.038798+0000","last_peered":"2024-09-16T15:53:49.038798+0000","last_clean":"2024-09-16T15:53:49.038798+0000","last_became_active":"2024-09-16T15:53:46.049306+0000","last_became_peered":"2024-09-16T15:53:46.049306+0000","last_unstale":"2024-09-16T15:53:49.038798+0000","last_undegraded":"2024-09-16T15:53:49.038798+0000","last_fullsized":"2024-09-16T15:53:49.038798+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:55:49.783269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.604895+0000","last_change":"2024-09-16T15:53:47.039878+0000","last_active":"2024-09-16T15:53:48.604895+0000","last_peered":"2024-09-16T15:53:48.604895+0000","last_clean":"2024-09-16T15:53:48.604895+0000","last_became_active":"2024-09-16T15:53:47.038999+0000","last_became_peered":"2024-09-16T15:53:47.038999+0000","last_unstale":"2024-09-16T15:53:48.604895+0000","last_undegraded":"2024-09-16T15:53:48.604895+0000","last_fullsized":"2024-09-16T15:53:48.604895+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:34:03.640512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1e","version":"132'1","reported_seq":16,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.604710+0000","last_change":"2024-09-16T15:53:46.055817+0000","last_active":"2024-09-16T15:53:48.604710+0000","last_peered":"2024-09-16T15:53:48.604710+0000","last_clean":"2024-09-16T15:53:48.604710+0000","last_became_active":"2024-09-16T15:53:46.055540+0000","last_became_peered":"2024-09-16T15:53:46.055540+0000","last_unstale":"2024-09-16T15:53:48.604710+0000","last_undegraded":"2024-09-16T15:53:48.604710+0000","last_fullsized":"2024-09-16T15:53:48.604710+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:23:12.275241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.604508+0000","last_change":"2024-09-16T15:53:47.055252+0000","last_active":"2024-09-16T15:53:48.604508+0000","last_peered":"2024-09-16T15:53:48.604508+0000","last_clean":"2024-09-16T15:53:48.604508+0000","last_became_active":"2024-09-16T15:53:47.054974+0000","last_became_peered":"2024-09-16T15:53:47.054974+0000","last_unstale":"2024-09-16T15:53:48.604508+0000","last_undegraded":"2024-09-16T15:53:48.604508+0000","last_fullsized":"2024-09-16T15:53:48.604508+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:12:54.366435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1d","version":"132'2","reported_seq":17,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.036479+0000","last_change":"2024-09-16T15:53:46.043053+0000","last_active":"2024-09-16T15:53:48.036479+0000","last_peered":"2024-09-16T15:53:48.036479+0000","last_clean":"2024-09-16T15:53:48.036479+0000","last_became_active":"2024-09-16T15:53:46.042758+0000","last_became_peered":"2024-09-16T15:53:46.042758+0000","last_unstale":"2024-09-16T15:53:48.036479+0000","last_undegraded":"2024-09-16T15:53:48.036479+0000","last_fullsized":"2024-09-16T15:53:48.036479+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:49:50.234698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.036593+0000","last_change":"2024-09-16T15:53:47.069273+0000","last_active":"2024-09-16T15:53:48.036593+0000","last_peered":"2024-09-16T15:53:48.036593+0000","last_clean":"2024-09-16T15:53:48.036593+0000","last_became_active":"2024-09-16T15:53:47.067006+0000","last_became_peered":"2024-09-16T15:53:47.067006+0000","last_unstale":"2024-09-16T15:53:48.036593+0000","last_undegraded":"2024-09-16T15:53:48.036593+0000","last_fullsized":"2024-09-16T15:53:48.036593+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:07:19.329442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1c","version":"132'2","reported_seq":17,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.274667+0000","last_change":"2024-09-16T15:53:46.046499+0000","last_active":"2024-09-16T15:53:49.274667+0000","last_peered":"2024-09-16T15:53:49.274667+0000","last_clean":"2024-09-16T15:53:49.274667+0000","last_became_active":"2024-09-16T15:53:46.046246+0000","last_became_peered":"2024-09-16T15:53:46.046246+0000","last_unstale":"2024-09-16T15:53:49.274667+0000","last_undegraded":"2024-09-16T15:53:49.274667+0000","last_fullsized":"2024-09-16T15:53:49.274667+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:07:27.829527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.480951+0000","last_change":"2024-09-16T15:53:47.060948+0000","last_active":"2024-09-16T15:53:48.480951+0000","last_peered":"2024-09-16T15:53:48.480951+0000","last_clean":"2024-09-16T15:53:48.480951+0000","last_became_active":"2024-09-16T15:53:47.060538+0000","last_became_peered":"2024-09-16T15:53:47.060538+0000","last_unstale":"2024-09-16T15:53:48.480951+0000","last_undegraded":"2024-09-16T15:53:48.480951+0000","last_fullsized":"2024-09-16T15:53:48.480951+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:56:18.849300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1b","version":"132'1","reported_seq":16,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.274448+0000","last_change":"2024-09-16T15:53:46.048816+0000","last_active":"2024-09-16T15:53:49.274448+0000","last_peered":"2024-09-16T15:53:49.274448+0000","last_clean":"2024-09-16T15:53:49.274448+0000","last_became_active":"2024-09-16T15:53:46.048416+0000","last_became_peered":"2024-09-16T15:53:46.048416+0000","last_unstale":"2024-09-16T15:53:49.274448+0000","last_undegraded":"2024-09-16T15:53:49.274448+0000","last_fullsized":"2024-09-16T15:53:49.274448+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:13:59.648869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.035988+0000","last_change":"2024-09-16T15:53:47.068600+0000","last_active":"2024-09-16T15:53:48.035988+0000","last_peered":"2024-09-16T15:53:48.035988+0000","last_clean":"2024-09-16T15:53:48.035988+0000","last_became_active":"2024-09-16T15:53:47.066780+0000","last_became_peered":"2024-09-16T15:53:47.066780+0000","last_unstale":"2024-09-16T15:53:48.035988+0000","last_undegraded":"2024-09-16T15:53:48.035988+0000","last_fullsized":"2024-09-16T15:53:48.035988+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:25:28.695527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"132'1","reported_seq":16,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.480731+0000","last_change":"2024-09-16T15:53:46.051615+0000","last_active":"2024-09-16T15:53:48.480731+0000","last_peered":"2024-09-16T15:53:48.480731+0000","last_clean":"2024-09-16T15:53:48.480731+0000","last_became_active":"2024-09-16T15:53:46.051253+0000","last_became_peered":"2024-09-16T15:53:46.051253+0000","last_unstale":"2024-09-16T15:53:48.480731+0000","last_undegraded":"2024-09-16T15:53:48.480731+0000","last_fullsized":"2024-09-16T15:53:48.480731+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:11:54.504207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.035617+0000","last_change":"2024-09-16T15:53:47.063618+0000","last_active":"2024-09-16T15:53:48.035617+0000","last_peered":"2024-09-16T15:53:48.035617+0000","last_clean":"2024-09-16T15:53:48.035617+0000","last_became_active":"2024-09-16T15:53:47.063347+0000","last_became_peered":"2024-09-16T15:53:47.063347+0000","last_unstale":"2024-09-16T15:53:48.035617+0000","last_undegraded":"2024-09-16T15:53:48.035617+0000","last_fullsized":"2024-09-16T15:53:48.035617+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:58:32.257998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.604391+0000","last_change":"2024-09-16T15:53:46.052810+0000","last_active":"2024-09-16T15:53:48.604391+0000","last_peered":"2024-09-16T15:53:48.604391+0000","last_clean":"2024-09-16T15:53:48.604391+0000","last_became_active":"2024-09-16T15:53:46.052461+0000","last_became_peered":"2024-09-16T15:53:46.052461+0000","last_unstale":"2024-09-16T15:53:48.604391+0000","last_undegraded":"2024-09-16T15:53:48.604391+0000","last_fullsized":"2024-09-16T15:53:48.604391+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:36:21.458493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.274292+0000","last_change":"2024-09-16T15:53:47.041379+0000","last_active":"2024-09-16T15:53:49.274292+0000","last_peered":"2024-09-16T15:53:49.274292+0000","last_clean":"2024-09-16T15:53:49.274292+0000","last_became_active":"2024-09-16T15:53:47.041100+0000","last_became_peered":"2024-09-16T15:53:47.041100+0000","last_unstale":"2024-09-16T15:53:49.274292+0000","last_undegraded":"2024-09-16T15:53:49.274292+0000","last_fullsized":"2024-09-16T15:53:49.274292+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:57:10.468784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.274281+0000","last_change":"2024-09-16T15:53:46.050598+0000","last_active":"2024-09-16T15:53:49.274281+0000","last_peered":"2024-09-16T15:53:49.274281+0000","last_clean":"2024-09-16T15:53:49.274281+0000","last_became_active":"2024-09-16T15:53:46.050355+0000","last_became_peered":"2024-09-16T15:53:46.050355+0000","last_unstale":"2024-09-16T15:53:49.274281+0000","last_undegraded":"2024-09-16T15:53:49.274281+0000","last_fullsized":"2024-09-16T15:53:49.274281+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:00:09.267267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.275145+0000","last_change":"2024-09-16T15:53:47.052882+0000","last_active":"2024-09-16T15:53:49.275145+0000","last_peered":"2024-09-16T15:53:49.275145+0000","last_clean":"2024-09-16T15:53:49.275145+0000","last_became_active":"2024-09-16T15:53:47.052537+0000","last_became_peered":"2024-09-16T15:53:47.052537+0000","last_unstale":"2024-09-16T15:53:49.275145+0000","last_undegraded":"2024-09-16T15:53:49.275145+0000","last_fullsized":"2024-09-16T15:53:49.275145+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:04:06.435765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.036663+0000","last_change":"2024-09-16T15:53:46.041207+0000","last_active":"2024-09-16T15:53:48.036663+0000","last_peered":"2024-09-16T15:53:48.036663+0000","last_clean":"2024-09-16T15:53:48.036663+0000","last_became_active":"2024-09-16T15:53:46.040942+0000","last_became_peered":"2024-09-16T15:53:46.040942+0000","last_unstale":"2024-09-16T15:53:48.036663+0000","last_undegraded":"2024-09-16T15:53:48.036663+0000","last_fullsized":"2024-09-16T15:53:48.036663+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:00:03.561191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.038694+0000","last_change":"2024-09-16T15:53:47.054422+0000","last_active":"2024-09-16T15:53:49.038694+0000","last_peered":"2024-09-16T15:53:49.038694+0000","last_clean":"2024-09-16T15:53:49.038694+0000","last_became_active":"2024-09-16T15:53:47.054120+0000","last_became_peered":"2024-09-16T15:53:47.054120+0000","last_unstale":"2024-09-16T15:53:49.038694+0000","last_undegraded":"2024-09-16T15:53:49.038694+0000","last_fullsized":"2024-09-16T15:53:49.038694+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:25:48.625685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.480939+0000","last_change":"2024-09-16T15:53:46.049887+0000","last_active":"2024-09-16T15:53:48.480939+0000","last_peered":"2024-09-16T15:53:48.480939+0000","last_clean":"2024-09-16T15:53:48.480939+0000","last_became_active":"2024-09-16T15:53:46.049554+0000","last_became_peered":"2024-09-16T15:53:46.049554+0000","last_unstale":"2024-09-16T15:53:48.480939+0000","last_undegraded":"2024-09-16T15:53:48.480939+0000","last_fullsized":"2024-09-16T15:53:48.480939+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:39:03.893863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.036446+0000","last_change":"2024-09-16T15:53:47.069807+0000","last_active":"2024-09-16T15:53:48.036446+0000","last_peered":"2024-09-16T15:53:48.036446+0000","last_clean":"2024-09-16T15:53:48.036446+0000","last_became_active":"2024-09-16T15:53:47.068406+0000","last_became_peered":"2024-09-16T15:53:47.068406+0000","last_unstale":"2024-09-16T15:53:48.036446+0000","last_undegraded":"2024-09-16T15:53:48.036446+0000","last_fullsized":"2024-09-16T15:53:48.036446+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:51:00.666807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.15","version":"132'3","reported_seq":18,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.038592+0000","last_change":"2024-09-16T15:53:46.051421+0000","last_active":"2024-09-16T15:53:49.038592+0000","last_peered":"2024-09-16T15:53:49.038592+0000","last_clean":"2024-09-16T15:53:49.038592+0000","last_became_active":"2024-09-16T15:53:46.049074+0000","last_became_peered":"2024-09-16T15:53:46.049074+0000","last_unstale":"2024-09-16T15:53:49.038592+0000","last_undegraded":"2024-09-16T15:53:49.038592+0000","last_fullsized":"2024-09-16T15:53:49.038592+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:03:46.041805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.273880+0000","last_change":"2024-09-16T15:53:47.068386+0000","last_active":"2024-09-16T15:53:49.273880+0000","last_peered":"2024-09-16T15:53:49.273880+0000","last_clean":"2024-09-16T15:53:49.273880+0000","last_became_active":"2024-09-16T15:53:47.067650+0000","last_became_peered":"2024-09-16T15:53:47.067650+0000","last_unstale":"2024-09-16T15:53:49.273880+0000","last_undegraded":"2024-09-16T15:53:49.273880+0000","last_fullsized":"2024-09-16T15:53:49.273880+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:30:31.196430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.14","version":"132'6","reported_seq":21,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.273912+0000","last_change":"2024-09-16T15:53:46.055615+0000","last_active":"2024-09-16T15:53:49.273912+0000","last_peered":"2024-09-16T15:53:49.273912+0000","last_clean":"2024-09-16T15:53:49.273912+0000","last_became_active":"2024-09-16T15:53:46.055155+0000","last_became_peered":"2024-09-16T15:53:46.055155+0000","last_unstale":"2024-09-16T15:53:49.273912+0000","last_undegraded":"2024-09-16T15:53:49.273912+0000","last_fullsized":"2024-09-16T15:53:49.273912+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:43:29.847384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.035122+0000","last_change":"2024-09-16T15:53:47.055028+0000","last_active":"2024-09-16T15:53:48.035122+0000","last_peered":"2024-09-16T15:53:48.035122+0000","last_clean":"2024-09-16T15:53:48.035122+0000","last_became_active":"2024-09-16T15:53:47.054638+0000","last_became_peered":"2024-09-16T15:53:47.054638+0000","last_unstale":"2024-09-16T15:53:48.035122+0000","last_undegraded":"2024-09-16T15:53:48.035122+0000","last_fullsized":"2024-09-16T15:53:48.035122+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:30:59.945665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.034753+0000","last_change":"2024-09-16T15:53:46.053325+0000","last_active":"2024-09-16T15:53:48.034753+0000","last_peered":"2024-09-16T15:53:48.034753+0000","last_clean":"2024-09-16T15:53:48.034753+0000","last_became_active":"2024-09-16T15:53:46.051989+0000","last_became_peered":"2024-09-16T15:53:46.051989+0000","last_unstale":"2024-09-16T15:53:48.034753+0000","last_undegraded":"2024-09-16T15:53:48.034753+0000","last_fullsized":"2024-09-16T15:53:48.034753+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:10:52.279943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.034761+0000","last_change":"2024-09-16T15:53:47.062301+0000","last_active":"2024-09-16T15:53:48.034761+0000","last_peered":"2024-09-16T15:53:48.034761+0000","last_clean":"2024-09-16T15:53:48.034761+0000","last_became_active":"2024-09-16T15:53:47.058392+0000","last_became_peered":"2024-09-16T15:53:47.058392+0000","last_unstale":"2024-09-16T15:53:48.034761+0000","last_undegraded":"2024-09-16T15:53:48.034761+0000","last_fullsized":"2024-09-16T15:53:48.034761+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:45:50.303564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.035810+0000","last_change":"2024-09-16T15:53:46.055507+0000","last_active":"2024-09-16T15:53:48.035810+0000","last_peered":"2024-09-16T15:53:48.035810+0000","last_clean":"2024-09-16T15:53:48.035810+0000","last_became_active":"2024-09-16T15:53:46.055247+0000","last_became_peered":"2024-09-16T15:53:46.055247+0000","last_unstale":"2024-09-16T15:53:48.035810+0000","last_undegraded":"2024-09-16T15:53:48.035810+0000","last_fullsized":"2024-09-16T15:53:48.035810+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:26:45.397159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.035599+0000","last_change":"2024-09-16T15:53:47.065045+0000","last_active":"2024-09-16T15:53:48.035599+0000","last_peered":"2024-09-16T15:53:48.035599+0000","last_clean":"2024-09-16T15:53:48.035599+0000","last_became_active":"2024-09-16T15:53:47.064756+0000","last_became_peered":"2024-09-16T15:53:47.064756+0000","last_unstale":"2024-09-16T15:53:48.035599+0000","last_undegraded":"2024-09-16T15:53:48.035599+0000","last_fullsized":"2024-09-16T15:53:48.035599+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:29:02.714478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.10","version":"132'8","reported_seq":23,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.035597+0000","last_change":"2024-09-16T15:53:46.054698+0000","last_active":"2024-09-16T15:53:48.035597+0000","last_peered":"2024-09-16T15:53:48.035597+0000","last_clean":"2024-09-16T15:53:48.035597+0000","last_became_active":"2024-09-16T15:53:46.050527+0000","last_became_peered":"2024-09-16T15:53:46.050527+0000","last_unstale":"2024-09-16T15:53:48.035597+0000","last_undegraded":"2024-09-16T15:53:48.035597+0000","last_fullsized":"2024-09-16T15:53:48.035597+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:29:09.646752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.038576+0000","last_change":"2024-09-16T15:53:47.049220+0000","last_active":"2024-09-16T15:53:49.038576+0000","last_peered":"2024-09-16T15:53:49.038576+0000","last_clean":"2024-09-16T15:53:49.038576+0000","last_became_active":"2024-09-16T15:53:47.048911+0000","last_became_peered":"2024-09-16T15:53:47.048911+0000","last_unstale":"2024-09-16T15:53:49.038576+0000","last_undegraded":"2024-09-16T15:53:49.038576+0000","last_fullsized":"2024-09-16T15:53:49.038576+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:13:16.281809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.f","version":"132'1","reported_seq":16,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.036692+0000","last_change":"2024-09-16T15:53:46.041954+0000","last_active":"2024-09-16T15:53:48.036692+0000","last_peered":"2024-09-16T15:53:48.036692+0000","last_clean":"2024-09-16T15:53:48.036692+0000","last_became_active":"2024-09-16T15:53:46.041272+0000","last_became_peered":"2024-09-16T15:53:46.041272+0000","last_unstale":"2024-09-16T15:53:48.036692+0000","last_undegraded":"2024-09-16T15:53:48.036692+0000","last_fullsized":"2024-09-16T15:53:48.036692+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:30:51.150005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.038733+0000","last_change":"2024-09-16T15:53:47.052264+0000","last_active":"2024-09-16T15:53:49.038733+0000","last_peered":"2024-09-16T15:53:49.038733+0000","last_clean":"2024-09-16T15:53:49.038733+0000","last_became_active":"2024-09-16T15:53:47.051995+0000","last_became_peered":"2024-09-16T15:53:47.051995+0000","last_unstale":"2024-09-16T15:53:49.038733+0000","last_undegraded":"2024-09-16T15:53:49.038733+0000","last_fullsized":"2024-09-16T15:53:49.038733+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:52:00.354801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"132'2","reported_seq":17,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.036582+0000","last_change":"2024-09-16T15:53:46.048066+0000","last_active":"2024-09-16T15:53:48.036582+0000","last_peered":"2024-09-16T15:53:48.036582+0000","last_clean":"2024-09-16T15:53:48.036582+0000","last_became_active":"2024-09-16T15:53:46.047765+0000","last_became_peered":"2024-09-16T15:53:46.047765+0000","last_unstale":"2024-09-16T15:53:48.036582+0000","last_undegraded":"2024-09-16T15:53:48.036582+0000","last_fullsized":"2024-09-16T15:53:48.036582+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:25:07.219846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.274731+0000","last_change":"2024-09-16T15:53:47.054306+0000","last_active":"2024-09-16T15:53:49.274731+0000","last_peered":"2024-09-16T15:53:49.274731+0000","last_clean":"2024-09-16T15:53:49.274731+0000","last_became_active":"2024-09-16T15:53:47.054002+0000","last_became_peered":"2024-09-16T15:53:47.054002+0000","last_unstale":"2024-09-16T15:53:49.274731+0000","last_undegraded":"2024-09-16T15:53:49.274731+0000","last_fullsized":"2024-09-16T15:53:49.274731+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:25:48.625685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.d","version":"132'1","reported_seq":16,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.604918+0000","last_change":"2024-09-16T15:53:46.050413+0000","last_active":"2024-09-16T15:53:48.604918+0000","last_peered":"2024-09-16T15:53:48.604918+0000","last_clean":"2024-09-16T15:53:48.604918+0000","last_became_active":"2024-09-16T15:53:46.050136+0000","last_became_peered":"2024-09-16T15:53:46.050136+0000","last_unstale":"2024-09-16T15:53:48.604918+0000","last_undegraded":"2024-09-16T15:53:48.604918+0000","last_fullsized":"2024-09-16T15:53:48.604918+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:35:42.185986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.274929+0000","last_change":"2024-09-16T15:53:47.055311+0000","last_active":"2024-09-16T15:53:49.274929+0000","last_peered":"2024-09-16T15:53:49.274929+0000","last_clean":"2024-09-16T15:53:49.274929+0000","last_became_active":"2024-09-16T15:53:47.053613+0000","last_became_peered":"2024-09-16T15:53:47.053613+0000","last_unstale":"2024-09-16T15:53:49.274929+0000","last_undegraded":"2024-09-16T15:53:49.274929+0000","last_fullsized":"2024-09-16T15:53:49.274929+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:52:21.045751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.038449+0000","last_change":"2024-09-16T15:53:46.052237+0000","last_active":"2024-09-16T15:53:49.038449+0000","last_peered":"2024-09-16T15:53:49.038449+0000","last_clean":"2024-09-16T15:53:49.038449+0000","last_became_active":"2024-09-16T15:53:46.051939+0000","last_became_peered":"2024-09-16T15:53:46.051939+0000","last_unstale":"2024-09-16T15:53:49.038449+0000","last_undegraded":"2024-09-16T15:53:49.038449+0000","last_fullsized":"2024-09-16T15:53:49.038449+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:43:15.863610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.274616+0000","last_change":"2024-09-16T15:53:47.055016+0000","last_active":"2024-09-16T15:53:49.274616+0000","last_peered":"2024-09-16T15:53:49.274616+0000","last_clean":"2024-09-16T15:53:49.274616+0000","last_became_active":"2024-09-16T15:53:47.054029+0000","last_became_peered":"2024-09-16T15:53:47.054029+0000","last_unstale":"2024-09-16T15:53:49.274616+0000","last_undegraded":"2024-09-16T15:53:49.274616+0000","last_fullsized":"2024-09-16T15:53:49.274616+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:39:06.152472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.b","version":"132'2","reported_seq":17,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.604585+0000","last_change":"2024-09-16T15:53:46.049709+0000","last_active":"2024-09-16T15:53:48.604585+0000","last_peered":"2024-09-16T15:53:48.604585+0000","last_clean":"2024-09-16T15:53:48.604585+0000","last_became_active":"2024-09-16T15:53:46.049412+0000","last_became_peered":"2024-09-16T15:53:46.049412+0000","last_unstale":"2024-09-16T15:53:48.604585+0000","last_undegraded":"2024-09-16T15:53:48.604585+0000","last_fullsized":"2024-09-16T15:53:48.604585+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:35:48.612183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.273696+0000","last_change":"2024-09-16T15:53:47.052068+0000","last_active":"2024-09-16T15:53:49.273696+0000","last_peered":"2024-09-16T15:53:49.273696+0000","last_clean":"2024-09-16T15:53:49.273696+0000","last_became_active":"2024-09-16T15:53:47.051755+0000","last_became_peered":"2024-09-16T15:53:47.051755+0000","last_unstale":"2024-09-16T15:53:49.273696+0000","last_undegraded":"2024-09-16T15:53:49.273696+0000","last_fullsized":"2024-09-16T15:53:49.273696+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:30:26.898180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.481002+0000","last_change":"2024-09-16T15:53:46.051522+0000","last_active":"2024-09-16T15:53:48.481002+0000","last_peered":"2024-09-16T15:53:48.481002+0000","last_clean":"2024-09-16T15:53:48.481002+0000","last_became_active":"2024-09-16T15:53:46.051005+0000","last_became_peered":"2024-09-16T15:53:46.051005+0000","last_unstale":"2024-09-16T15:53:48.481002+0000","last_undegraded":"2024-09-16T15:53:48.481002+0000","last_fullsized":"2024-09-16T15:53:48.481002+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:10:36.900762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.034741+0000","last_change":"2024-09-16T15:53:47.058656+0000","last_active":"2024-09-16T15:53:48.034741+0000","last_peered":"2024-09-16T15:53:48.034741+0000","last_clean":"2024-09-16T15:53:48.034741+0000","last_became_active":"2024-09-16T15:53:47.057260+0000","last_became_peered":"2024-09-16T15:53:47.057260+0000","last_unstale":"2024-09-16T15:53:48.034741+0000","last_undegraded":"2024-09-16T15:53:48.034741+0000","last_fullsized":"2024-09-16T15:53:48.034741+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:33:53.397015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.034453+0000","last_change":"2024-09-16T15:53:46.038714+0000","last_active":"2024-09-16T15:53:48.034453+0000","last_peered":"2024-09-16T15:53:48.034453+0000","last_clean":"2024-09-16T15:53:48.034453+0000","last_became_active":"2024-09-16T15:53:46.038411+0000","last_became_peered":"2024-09-16T15:53:46.038411+0000","last_unstale":"2024-09-16T15:53:48.034453+0000","last_undegraded":"2024-09-16T15:53:48.034453+0000","last_fullsized":"2024-09-16T15:53:48.034453+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:43:20.313472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.035857+0000","last_change":"2024-09-16T15:53:47.065949+0000","last_active":"2024-09-16T15:53:48.035857+0000","last_peered":"2024-09-16T15:53:48.035857+0000","last_clean":"2024-09-16T15:53:48.035857+0000","last_became_active":"2024-09-16T15:53:47.064802+0000","last_became_peered":"2024-09-16T15:53:47.064802+0000","last_unstale":"2024-09-16T15:53:48.035857+0000","last_undegraded":"2024-09-16T15:53:48.035857+0000","last_fullsized":"2024-09-16T15:53:48.035857+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:09:34.576069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.035763+0000","last_change":"2024-09-16T15:53:46.049975+0000","last_active":"2024-09-16T15:53:48.035763+0000","last_peered":"2024-09-16T15:53:48.035763+0000","last_clean":"2024-09-16T15:53:48.035763+0000","last_became_active":"2024-09-16T15:53:46.049667+0000","last_became_peered":"2024-09-16T15:53:46.049667+0000","last_unstale":"2024-09-16T15:53:48.035763+0000","last_undegraded":"2024-09-16T15:53:48.035763+0000","last_fullsized":"2024-09-16T15:53:48.035763+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:50:23.146038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.273923+0000","last_change":"2024-09-16T15:53:47.066513+0000","last_active":"2024-09-16T15:53:49.273923+0000","last_peered":"2024-09-16T15:53:49.273923+0000","last_clean":"2024-09-16T15:53:49.273923+0000","last_became_active":"2024-09-16T15:53:47.066252+0000","last_became_peered":"2024-09-16T15:53:47.066252+0000","last_unstale":"2024-09-16T15:53:49.273923+0000","last_undegraded":"2024-09-16T15:53:49.273923+0000","last_fullsized":"2024-09-16T15:53:49.273923+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:18:41.717848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.7","version":"132'2","reported_seq":17,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.480957+0000","last_change":"2024-09-16T15:53:46.051198+0000","last_active":"2024-09-16T15:53:48.480957+0000","last_peered":"2024-09-16T15:53:48.480957+0000","last_clean":"2024-09-16T15:53:48.480957+0000","last_became_active":"2024-09-16T15:53:46.050837+0000","last_became_peered":"2024-09-16T15:53:46.050837+0000","last_unstale":"2024-09-16T15:53:48.480957+0000","last_undegraded":"2024-09-16T15:53:48.480957+0000","last_fullsized":"2024-09-16T15:53:48.480957+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:38:17.885868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.274984+0000","last_change":"2024-09-16T15:53:47.060499+0000","last_active":"2024-09-16T15:53:49.274984+0000","last_peered":"2024-09-16T15:53:49.274984+0000","last_clean":"2024-09-16T15:53:49.274984+0000","last_became_active":"2024-09-16T15:53:47.059699+0000","last_became_peered":"2024-09-16T15:53:47.059699+0000","last_unstale":"2024-09-16T15:53:49.274984+0000","last_undegraded":"2024-09-16T15:53:49.274984+0000","last_fullsized":"2024-09-16T15:53:49.274984+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:35:38.397852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.6","version":"133'1","reported_seq":16,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.038798+0000","last_change":"2024-09-16T15:53:46.055909+0000","last_active":"2024-09-16T15:53:49.038798+0000","last_peered":"2024-09-16T15:53:49.038798+0000","last_clean":"2024-09-16T15:53:49.038798+0000","last_became_active":"2024-09-16T15:53:46.055567+0000","last_became_peered":"2024-09-16T15:53:46.055567+0000","last_unstale":"2024-09-16T15:53:49.038798+0000","last_undegraded":"2024-09-16T15:53:49.038798+0000","last_fullsized":"2024-09-16T15:53:49.038798+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:27:29.536408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.035153+0000","last_change":"2024-09-16T15:53:47.055081+0000","last_active":"2024-09-16T15:53:48.035153+0000","last_peered":"2024-09-16T15:53:48.035153+0000","last_clean":"2024-09-16T15:53:48.035153+0000","last_became_active":"2024-09-16T15:53:47.054772+0000","last_became_peered":"2024-09-16T15:53:47.054772+0000","last_unstale":"2024-09-16T15:53:48.035153+0000","last_undegraded":"2024-09-16T15:53:48.035153+0000","last_fullsized":"2024-09-16T15:53:48.035153+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:32:42.788318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.481141+0000","last_change":"2024-09-16T15:53:46.056629+0000","last_active":"2024-09-16T15:53:48.481141+0000","last_peered":"2024-09-16T15:53:48.481141+0000","last_clean":"2024-09-16T15:53:48.481141+0000","last_became_active":"2024-09-16T15:53:46.056360+0000","last_became_peered":"2024-09-16T15:53:46.056360+0000","last_unstale":"2024-09-16T15:53:48.481141+0000","last_undegraded":"2024-09-16T15:53:48.481141+0000","last_fullsized":"2024-09-16T15:53:48.481141+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:57:33.513974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.604813+0000","last_change":"2024-09-16T15:53:47.048289+0000","last_active":"2024-09-16T15:53:48.604813+0000","last_peered":"2024-09-16T15:53:48.604813+0000","last_clean":"2024-09-16T15:53:48.604813+0000","last_became_active":"2024-09-16T15:53:47.047818+0000","last_became_peered":"2024-09-16T15:53:47.047818+0000","last_unstale":"2024-09-16T15:53:48.604813+0000","last_undegraded":"2024-09-16T15:53:48.604813+0000","last_fullsized":"2024-09-16T15:53:48.604813+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:52:33.800592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.035325+0000","last_change":"2024-09-16T15:53:46.057982+0000","last_active":"2024-09-16T15:53:48.035325+0000","last_peered":"2024-09-16T15:53:48.035325+0000","last_clean":"2024-09-16T15:53:48.035325+0000","last_became_active":"2024-09-16T15:53:46.057707+0000","last_became_peered":"2024-09-16T15:53:46.057707+0000","last_unstale":"2024-09-16T15:53:48.035325+0000","last_undegraded":"2024-09-16T15:53:48.035325+0000","last_fullsized":"2024-09-16T15:53:48.035325+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:59:40.544624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.035623+0000","last_change":"2024-09-16T15:53:47.062803+0000","last_active":"2024-09-16T15:53:48.035623+0000","last_peered":"2024-09-16T15:53:48.035623+0000","last_clean":"2024-09-16T15:53:48.035623+0000","last_became_active":"2024-09-16T15:53:47.060237+0000","last_became_peered":"2024-09-16T15:53:47.060237+0000","last_unstale":"2024-09-16T15:53:48.035623+0000","last_undegraded":"2024-09-16T15:53:48.035623+0000","last_fullsized":"2024-09-16T15:53:48.035623+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:52:21.045751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.274595+0000","last_change":"2024-09-16T15:53:46.052804+0000","last_active":"2024-09-16T15:53:49.274595+0000","last_peered":"2024-09-16T15:53:49.274595+0000","last_clean":"2024-09-16T15:53:49.274595+0000","last_became_active":"2024-09-16T15:53:46.052460+0000","last_became_peered":"2024-09-16T15:53:46.052460+0000","last_unstale":"2024-09-16T15:53:49.274595+0000","last_undegraded":"2024-09-16T15:53:49.274595+0000","last_fullsized":"2024-09-16T15:53:49.274595+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:36:04.003482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.036048+0000","last_change":"2024-09-16T15:53:47.065967+0000","last_active":"2024-09-16T15:53:48.036048+0000","last_peered":"2024-09-16T15:53:48.036048+0000","last_clean":"2024-09-16T15:53:48.036048+0000","last_became_active":"2024-09-16T15:53:47.065715+0000","last_became_peered":"2024-09-16T15:53:47.065715+0000","last_unstale":"2024-09-16T15:53:48.036048+0000","last_undegraded":"2024-09-16T15:53:48.036048+0000","last_fullsized":"2024-09-16T15:53:48.036048+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:48:44.181946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.035831+0000","last_change":"2024-09-16T15:53:46.044167+0000","last_active":"2024-09-16T15:53:48.035831+0000","last_peered":"2024-09-16T15:53:48.035831+0000","last_clean":"2024-09-16T15:53:48.035831+0000","last_became_active":"2024-09-16T15:53:46.043914+0000","last_became_peered":"2024-09-16T15:53:46.043914+0000","last_unstale":"2024-09-16T15:53:48.035831+0000","last_undegraded":"2024-09-16T15:53:48.035831+0000","last_fullsized":"2024-09-16T15:53:48.035831+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:20:21.174844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.481211+0000","last_change":"2024-09-16T15:53:47.069960+0000","last_active":"2024-09-16T15:53:48.481211+0000","last_peered":"2024-09-16T15:53:48.481211+0000","last_clean":"2024-09-16T15:53:48.481211+0000","last_became_active":"2024-09-16T15:53:47.069614+0000","last_became_peered":"2024-09-16T15:53:47.069614+0000","last_unstale":"2024-09-16T15:53:48.481211+0000","last_undegraded":"2024-09-16T15:53:48.481211+0000","last_fullsized":"2024-09-16T15:53:48.481211+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:25:21.431101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.035927+0000","last_change":"2024-09-16T15:53:46.055377+0000","last_active":"2024-09-16T15:53:48.035927+0000","last_peered":"2024-09-16T15:53:48.035927+0000","last_clean":"2024-09-16T15:53:48.035927+0000","last_became_active":"2024-09-16T15:53:46.055135+0000","last_became_peered":"2024-09-16T15:53:46.055135+0000","last_unstale":"2024-09-16T15:53:48.035927+0000","last_undegraded":"2024-09-16T15:53:48.035927+0000","last_fullsized":"2024-09-16T15:53:48.035927+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:04:47.485315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.034482+0000","last_change":"2024-09-16T15:53:47.057817+0000","last_active":"2024-09-16T15:53:48.034482+0000","last_peered":"2024-09-16T15:53:48.034482+0000","last_clean":"2024-09-16T15:53:48.034482+0000","last_became_active":"2024-09-16T15:53:47.057440+0000","last_became_peered":"2024-09-16T15:53:47.057440+0000","last_unstale":"2024-09-16T15:53:48.034482+0000","last_undegraded":"2024-09-16T15:53:48.034482+0000","last_fullsized":"2024-09-16T15:53:48.034482+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:35:53.392206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":15,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.034905+0000","last_change":"2024-09-16T15:53:46.047710+0000","last_active":"2024-09-16T15:53:48.034905+0000","last_peered":"2024-09-16T15:53:48.034905+0000","last_clean":"2024-09-16T15:53:48.034905+0000","last_became_active":"2024-09-16T15:53:46.047320+0000","last_became_peered":"2024-09-16T15:53:46.047320+0000","last_unstale":"2024-09-16T15:53:48.034905+0000","last_undegraded":"2024-09-16T15:53:48.034905+0000","last_fullsized":"2024-09-16T15:53:48.034905+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:40:37.494885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":13,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.604526+0000","last_change":"2024-09-16T15:53:47.039385+0000","last_active":"2024-09-16T15:53:48.604526+0000","last_peered":"2024-09-16T15:53:48.604526+0000","last_clean":"2024-09-16T15:53:48.604526+0000","last_became_active":"2024-09-16T15:53:47.039077+0000","last_became_peered":"2024-09-16T15:53:47.039077+0000","last_unstale":"2024-09-16T15:53:48.604526+0000","last_undegraded":"2024-09-16T15:53:48.604526+0000","last_fullsized":"2024-09-16T15:53:48.604526+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:46.018368+0000","last_clean_scrub_stamp":"2024-09-16T15:53:46.018368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:30:30.250921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.13","version":"132'2","reported_seq":17,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:48.036176+0000","last_change":"2024-09-16T15:53:46.048215+0000","last_active":"2024-09-16T15:53:48.036176+0000","last_peered":"2024-09-16T15:53:48.036176+0000","last_clean":"2024-09-16T15:53:48.036176+0000","last_became_active":"2024-09-16T15:53:46.045488+0000","last_became_peered":"2024-09-16T15:53:46.045488+0000","last_unstale":"2024-09-16T15:53:48.036176+0000","last_undegraded":"2024-09-16T15:53:48.036176+0000","last_fullsized":"2024-09-16T15:53:48.036176+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:53:45.010151+0000","last_clean_scrub_stamp":"2024-09-16T15:53:45.010151+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:59:52.882286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":323,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-09-16T15:53:49.273491+0000","last_change":"2024-09-16T15:39:12.698813+0000","last_active":"2024-09-16T15:53:49.273491+0000","last_peered":"2024-09-16T15:53:49.273491+0000","last_clean":"2024-09-16T15:53:49.273491+0000","last_became_active":"2024-09-16T15:39:12.697695+0000","last_became_peered":"2024-09-16T15:39:12.697695+0000","last_unstale":"2024-09-16T15:53:49.273491+0000","last_undegraded":"2024-09-16T15:53:49.273491+0000","last_fullsized":"2024-09-16T15:53:49.273491+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_clean_scrub_stamp":"2024-09-16T15:39:11.684967+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:02.061081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509621,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":159072,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":157438,"kb_avail":94212768,"statfs":{"total":96636764160,"available":96473874432,"internally_reserved":0,"allocated":1650688,"data_stored":729576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":161216968},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509620,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":119408,"kb_used_data":2204,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94252432,"statfs":{"total":96636764160,"available":96514490368,"internally_reserved":0,"allocated":2256896,"data_stored":1320686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":119994823},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509619,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":136864,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94234976,"statfs":{"total":96636764160,"available":96496615424,"internally_reserved":0,"allocated":1650688,"data_stored":729576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":138475973},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509619,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":118820,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94253020,"statfs":{"total":96636764160,"available":96515092480,"internally_reserved":0,"allocated":1654784,"data_stored":729646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":119994825},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509619,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131308,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240532,"statfs":{"total":96636764160,"available":96502304768,"internally_reserved":0,"allocated":1658880,"data_stored":730248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509620,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131892,"kb_used_data":2208,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94239948,"statfs":{"total":96636764160,"available":96501706752,"internally_reserved":0,"allocated":2260992,"data_stored":1321578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509618,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149364,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222476,"statfs":{"total":96636764160,"available":96483815424,"internally_reserved":0,"allocated":1671168,"data_stored":731244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":151255495},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542326,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131304,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240536,"statfs":{"total":96636764160,"available":96502308864,"internally_reserved":0,"allocated":1654784,"data_stored":729610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":132774348},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:53:56.361 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T15:53:56.361 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-16T15:53:56.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.561+0000 7f6339600640 1 -- 172.21.15.151:0/2032940620 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f633415f670 msgr2=0x7f63340c6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:56.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.561+0000 7f6339600640 1 --2- 172.21.15.151:0/2032940620 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f633415f670 0x7f63340c6430 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f6324005fd0 tx=0x7f632402a520 comp rx=0 tx=0).stop 2024-09-16T15:53:56.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.561+0000 7f6339600640 1 -- 172.21.15.151:0/2032940620 shutdown_connections 2024-09-16T15:53:56.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.561+0000 7f6339600640 1 --2- 172.21.15.151:0/2032940620 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f63340c5730 0x7f633415fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:56.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.561+0000 7f6339600640 1 --2- 172.21.15.151:0/2032940620 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f633415f670 0x7f63340c6430 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:56.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.561+0000 7f6339600640 1 -- 172.21.15.151:0/2032940620 >> 172.21.15.151:0/2032940620 conn(0x7f63340cc490 msgr2=0x7f63340cc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:56.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.561+0000 7f6339600640 1 -- 172.21.15.151:0/2032940620 shutdown_connections 2024-09-16T15:53:56.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.561+0000 7f6339600640 1 -- 172.21.15.151:0/2032940620 wait complete. 2024-09-16T15:53:56.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.563+0000 7f6339600640 1 Processor -- start 2024-09-16T15:53:56.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.563+0000 7f6339600640 1 -- start start 2024-09-16T15:53:56.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.564+0000 7f6339600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63340c5730 0x7f63341f7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:56.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.564+0000 7f6339600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f633415c0f0 0x7f63341f8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:56.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.564+0000 7f6339600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f633415f670 0x7f63341f4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:56.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.564+0000 7f6339600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f63340ca370 con 0x7f63340c5730 2024-09-16T15:53:56.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.564+0000 7f6339600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f63340700d0 con 0x7f633415f670 2024-09-16T15:53:56.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.564+0000 7f6339600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f63340ca4f0 con 0x7f633415c0f0 2024-09-16T15:53:56.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.564+0000 7f6333e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63340c5730 0x7f63341f7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:56.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.564+0000 7f6333e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63340c5730 0x7f63341f7f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:54784/0 (socket says 172.21.15.151:54784) 2024-09-16T15:53:56.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.564+0000 7f6333e00640 1 -- 172.21.15.151:0/1432929866 learned_addr learned my addr 172.21.15.151:0/1432929866 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:53:56.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.564+0000 7f6333400640 1 --2- 172.21.15.151:0/1432929866 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f633415c0f0 0x7f63341f8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:56.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.564+0000 7f6338c00640 1 --2- 172.21.15.151:0/1432929866 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f633415f670 0x7f63341f4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:56.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.564+0000 7f6333e00640 1 -- 172.21.15.151:0/1432929866 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f633415c0f0 msgr2=0x7f63341f8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:56.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.564+0000 7f6333e00640 1 --2- 172.21.15.151:0/1432929866 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f633415c0f0 0x7f63341f8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:56.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.564+0000 7f6333e00640 1 -- 172.21.15.151:0/1432929866 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f633415f670 msgr2=0x7f63341f4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:56.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.564+0000 7f6333e00640 1 --2- 172.21.15.151:0/1432929866 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f633415f670 0x7f63341f4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:56.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.564+0000 7f6333e00640 1 -- 172.21.15.151:0/1432929866 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63340c8b10 con 0x7f63340c5730 2024-09-16T15:53:56.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.564+0000 7f6333400640 1 --2- 172.21.15.151:0/1432929866 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f633415c0f0 0x7f63341f8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:53:56.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.565+0000 7f6333e00640 1 --2- 172.21.15.151:0/1432929866 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63340c5730 0x7f63341f7f20 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f6324034040 tx=0x7f63240038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:56.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.565+0000 7f632be00640 1 -- 172.21.15.151:0/1432929866 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f632403e030 con 0x7f63340c5730 2024-09-16T15:53:56.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.565+0000 7f632be00640 1 -- 172.21.15.151:0/1432929866 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6324003d90 con 0x7f63340c5730 2024-09-16T15:53:56.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.565+0000 7f632be00640 1 -- 172.21.15.151:0/1432929866 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6324035040 con 0x7f63340c5730 2024-09-16T15:53:56.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.565+0000 7f6339600640 1 -- 172.21.15.151:0/1432929866 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63340c8550 con 0x7f63340c5730 2024-09-16T15:53:56.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.565+0000 7f6339600640 1 -- 172.21.15.151:0/1432929866 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63341f5be0 con 0x7f63340c5730 2024-09-16T15:53:56.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.566+0000 7f6339600640 1 -- 172.21.15.151:0/1432929866 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63340ca710 con 0x7f63340c5730 2024-09-16T15:53:56.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.567+0000 7f632be00640 1 -- 172.21.15.151:0/1432929866 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 47) ==== 104092+0+0 (secure 0 0 0) 0x7f6324041030 con 0x7f63340c5730 2024-09-16T15:53:56.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.568+0000 7f632be00640 1 --2- 172.21.15.151:0/1432929866 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f631407a740 0x7f631407cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:56.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.568+0000 7f632be00640 1 -- 172.21.15.151:0/1432929866 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5821+0+0 (secure 0 0 0) 0x7f63240ba490 con 0x7f63340c5730 2024-09-16T15:53:56.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.569+0000 7f6333400640 1 --2- 172.21.15.151:0/1432929866 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f631407a740 0x7f631407cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:56.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.570+0000 7f6333400640 1 --2- 172.21.15.151:0/1432929866 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f631407a740 0x7f631407cc00 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f63200058c0 tx=0x7f6320005850 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:56.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.573+0000 7f632be00640 1 -- 172.21.15.151:0/1432929866 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f63340ca710 con 0x7f63340c5730 2024-09-16T15:53:56.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.824+0000 7f6339600640 1 -- 172.21.15.151:0/1432929866 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f633415ce90 con 0x7f63340c5730 2024-09-16T15:53:56.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.826+0000 7f632be00640 1 -- 172.21.15.151:0/1432929866 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) ==== 190+0+147 (secure 0 0 0) 0x7f6324084560 con 0x7f63340c5730 2024-09-16T15:53:56.828 INFO:teuthology.orchestra.run.smithi151.stdout:[client.0] 2024-09-16T15:53:56.829 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQDsUOhmf5PBKBAAKE4Y4zbA4h07IBNjPkyptw== 2024-09-16T15:53:56.829 INFO:teuthology.orchestra.run.smithi151.stdout: caps mds = "allow" 2024-09-16T15:53:56.829 INFO:teuthology.orchestra.run.smithi151.stdout: caps mon = "allow r" 2024-09-16T15:53:56.829 INFO:teuthology.orchestra.run.smithi151.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-16T15:53:56.829 INFO:teuthology.orchestra.run.smithi151.stderr:updated caps for client.0 2024-09-16T15:53:56.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.829+0000 7f6339600640 1 -- 172.21.15.151:0/1432929866 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f631407a740 msgr2=0x7f631407cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:56.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.829+0000 7f6339600640 1 --2- 172.21.15.151:0/1432929866 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f631407a740 0x7f631407cc00 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f63200058c0 tx=0x7f6320005850 comp rx=0 tx=0).stop 2024-09-16T15:53:56.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.829+0000 7f6339600640 1 -- 172.21.15.151:0/1432929866 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63340c5730 msgr2=0x7f63341f7f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:56.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.829+0000 7f6339600640 1 --2- 172.21.15.151:0/1432929866 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63340c5730 0x7f63341f7f20 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f6324034040 tx=0x7f63240038a0 comp rx=0 tx=0).stop 2024-09-16T15:53:56.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.829+0000 7f6339600640 1 -- 172.21.15.151:0/1432929866 shutdown_connections 2024-09-16T15:53:56.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.829+0000 7f6339600640 1 --2- 172.21.15.151:0/1432929866 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f631407a740 0x7f631407cc00 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:56.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.829+0000 7f6339600640 1 --2- 172.21.15.151:0/1432929866 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f633415f670 0x7f63341f4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:56.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.829+0000 7f6339600640 1 --2- 172.21.15.151:0/1432929866 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f633415c0f0 0x7f63341f8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:56.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.829+0000 7f6339600640 1 --2- 172.21.15.151:0/1432929866 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63340c5730 0x7f63341f7f20 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:56.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.829+0000 7f6339600640 1 -- 172.21.15.151:0/1432929866 >> 172.21.15.151:0/1432929866 conn(0x7f63340cc490 msgr2=0x7f63341520a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:56.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.829+0000 7f6339600640 1 -- 172.21.15.151:0/1432929866 shutdown_connections 2024-09-16T15:53:56.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:56.829+0000 7f6339600640 1 -- 172.21.15.151:0/1432929866 wait complete. 2024-09-16T15:53:56.847 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:53:57.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.003+0000 7f360a000640 1 -- 172.21.15.151:0/2980889593 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f36040cf1f0 msgr2=0x7f360406eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:57.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.003+0000 7f360a000640 1 --2- 172.21.15.151:0/2980889593 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f36040cf1f0 0x7f360406eec0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f36040b2530 tx=0x7f35fc02aac0 comp rx=0 tx=0).stop 2024-09-16T15:53:57.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.003+0000 7f360a000640 1 -- 172.21.15.151:0/2980889593 shutdown_connections 2024-09-16T15:53:57.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.003+0000 7f360a000640 1 --2- 172.21.15.151:0/2980889593 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f36040cf1f0 0x7f360406eec0 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:57.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.003+0000 7f360a000640 1 --2- 172.21.15.151:0/2980889593 >> v2:172.21.15.167:3300/0 conn(0x7f360415a280 0x7f36040ca070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:57.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.003+0000 7f360a000640 1 -- 172.21.15.151:0/2980889593 >> 172.21.15.151:0/2980889593 conn(0x7f36040cfa20 msgr2=0x7f36040cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:57.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.003+0000 7f360a000640 1 -- 172.21.15.151:0/2980889593 shutdown_connections 2024-09-16T15:53:57.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.003+0000 7f360a000640 1 -- 172.21.15.151:0/2980889593 wait complete. 2024-09-16T15:53:57.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.004+0000 7f360a000640 1 Processor -- start 2024-09-16T15:53:57.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.004+0000 7f360a000640 1 -- start start 2024-09-16T15:53:57.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.005+0000 7f360a000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f36040725d0 0x7f360415a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:57.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.005+0000 7f360a000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36040cf1f0 0x7f36040c5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:57.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.005+0000 7f360a000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f360415a280 0x7f36041f5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:57.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.005+0000 7f360a000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3604072a40 con 0x7f360415a280 2024-09-16T15:53:57.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.005+0000 7f360a000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f36040d0970 con 0x7f36040cf1f0 2024-09-16T15:53:57.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.005+0000 7f360a000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f3604072bc0 con 0x7f36040725d0 2024-09-16T15:53:57.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.006+0000 7f3609600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f360415a280 0x7f36041f5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:57.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.006+0000 7f3609600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f360415a280 0x7f36041f5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:54808/0 (socket says 172.21.15.151:54808) 2024-09-16T15:53:57.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.006+0000 7f3609600640 1 -- 172.21.15.151:0/3796991511 learned_addr learned my addr 172.21.15.151:0/3796991511 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:53:57.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.006+0000 7f3608c00640 1 --2- 172.21.15.151:0/3796991511 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f36040725d0 0x7f360415a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:57.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.006+0000 7f3603e00640 1 --2- 172.21.15.151:0/3796991511 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36040cf1f0 0x7f36040c5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:57.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.006+0000 7f3609600640 1 -- 172.21.15.151:0/3796991511 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f36040725d0 msgr2=0x7f360415a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:57.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.006+0000 7f3609600640 1 --2- 172.21.15.151:0/3796991511 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f36040725d0 0x7f360415a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:57.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.006+0000 7f3609600640 1 -- 172.21.15.151:0/3796991511 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36040cf1f0 msgr2=0x7f36040c5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:57.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.006+0000 7f3609600640 1 --2- 172.21.15.151:0/3796991511 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36040cf1f0 0x7f36040c5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:57.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.006+0000 7f3609600640 1 -- 172.21.15.151:0/3796991511 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36040c8830 con 0x7f360415a280 2024-09-16T15:53:57.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.006+0000 7f3608c00640 1 --2- 172.21.15.151:0/3796991511 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f36040725d0 0x7f360415a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:53:57.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.006+0000 7f3609600640 1 --2- 172.21.15.151:0/3796991511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f360415a280 0x7f36041f5a60 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f35fc02b0b0 tx=0x7f35fc003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:57.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.007+0000 7f3600c00640 1 -- 172.21.15.151:0/3796991511 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35fc044070 con 0x7f360415a280 2024-09-16T15:53:57.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.007+0000 7f3600c00640 1 -- 172.21.15.151:0/3796991511 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f35fc003e50 con 0x7f360415a280 2024-09-16T15:53:57.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.007+0000 7f3600c00640 1 -- 172.21.15.151:0/3796991511 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35fc0044b0 con 0x7f360415a280 2024-09-16T15:53:57.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.007+0000 7f360a000640 1 -- 172.21.15.151:0/3796991511 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36040c8b10 con 0x7f360415a280 2024-09-16T15:53:57.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.007+0000 7f360a000640 1 -- 172.21.15.151:0/3796991511 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36040c90d0 con 0x7f360415a280 2024-09-16T15:53:57.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.008+0000 7f360a000640 1 -- 172.21.15.151:0/3796991511 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36040ca070 con 0x7f360415a280 2024-09-16T15:53:57.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.009+0000 7f3600c00640 1 -- 172.21.15.151:0/3796991511 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 47) ==== 104092+0+0 (secure 0 0 0) 0x7f35fc004650 con 0x7f360415a280 2024-09-16T15:53:57.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.010+0000 7f3600c00640 1 --2- 172.21.15.151:0/3796991511 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f35e407a740 0x7f35e407cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:57.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.010+0000 7f3600c00640 1 -- 172.21.15.151:0/3796991511 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5821+0+0 (secure 0 0 0) 0x7f35fc0b9f70 con 0x7f360415a280 2024-09-16T15:53:57.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.010+0000 7f3608c00640 1 --2- 172.21.15.151:0/3796991511 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f35e407a740 0x7f35e407cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:57.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.011+0000 7f3608c00640 1 --2- 172.21.15.151:0/3796991511 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f35e407a740 0x7f35e407cc00 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f35f8009b80 tx=0x7f35f8005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:57.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.014+0000 7f3600c00640 1 -- 172.21.15.151:0/3796991511 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f35fc032030 con 0x7f360415a280 2024-09-16T15:53:57.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.233+0000 7f360a000640 1 -- 172.21.15.151:0/3796991511 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f360415ecc0 con 0x7f360415a280 2024-09-16T15:53:57.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.233+0000 7f3600c00640 1 -- 172.21.15.151:0/3796991511 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) ==== 94+0+5234 (secure 0 0 0) 0x7f35fc084040 con 0x7f360415a280 2024-09-16T15:53:57.235 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:53:57.235 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":87,"btime":"2024-09-16T15:53:50:060393+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8994,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/2450183746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":2450183746},{"type":"v1","addr":"172.21.15.151:6833","nonce":2450183746}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8997,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1952630294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1952630294},{"type":"v1","addr":"172.21.15.167:6836","nonce":1952630294}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":9000,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/127383056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":127383056},{"type":"v1","addr":"172.21.15.167:6837","nonce":127383056}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:53:46.461837+0000","modified":"2024-09-16T15:53:49.478024+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9009},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9009":{"gid":9009,"name":"a","rank":0,"incarnation":83,"state":"up:active","state_seq":3,"addr":"172.21.15.151:6835/1977294668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":1977294668},{"type":"v1","addr":"172.21.15.151:6835","nonce":1977294668}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9009,"qdb_cluster":[9009]},"id":9}]} 2024-09-16T15:53:57.236 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 87 2024-09-16T15:53:57.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.236+0000 7f360a000640 1 -- 172.21.15.151:0/3796991511 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f35e407a740 msgr2=0x7f35e407cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:57.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.236+0000 7f360a000640 1 --2- 172.21.15.151:0/3796991511 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f35e407a740 0x7f35e407cc00 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f35f8009b80 tx=0x7f35f8005f90 comp rx=0 tx=0).stop 2024-09-16T15:53:57.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.236+0000 7f360a000640 1 -- 172.21.15.151:0/3796991511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f360415a280 msgr2=0x7f36041f5a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:57.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.236+0000 7f360a000640 1 --2- 172.21.15.151:0/3796991511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f360415a280 0x7f36041f5a60 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f35fc02b0b0 tx=0x7f35fc003ce0 comp rx=0 tx=0).stop 2024-09-16T15:53:57.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.236+0000 7f360a000640 1 -- 172.21.15.151:0/3796991511 shutdown_connections 2024-09-16T15:53:57.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.236+0000 7f360a000640 1 --2- 172.21.15.151:0/3796991511 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f35e407a740 0x7f35e407cc00 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:57.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.236+0000 7f360a000640 1 --2- 172.21.15.151:0/3796991511 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f36040cf1f0 0x7f36040c5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:57.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.236+0000 7f360a000640 1 --2- 172.21.15.151:0/3796991511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f360415a280 0x7f36041f5a60 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:57.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.236+0000 7f360a000640 1 --2- 172.21.15.151:0/3796991511 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f36040725d0 0x7f360415a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:57.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.236+0000 7f360a000640 1 -- 172.21.15.151:0/3796991511 >> 172.21.15.151:0/3796991511 conn(0x7f36040cfa20 msgr2=0x7f3604155f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:57.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.236+0000 7f360a000640 1 -- 172.21.15.151:0/3796991511 shutdown_connections 2024-09-16T15:53:57.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.236+0000 7f360a000640 1 -- 172.21.15.151:0/3796991511 wait complete. 2024-09-16T15:53:57.656 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 87, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-16T15:53:46.461837+0000', 'modified': '2024-09-16T15:53:49.478024+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9009}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9009': {'gid': 9009, 'name': 'a', 'rank': 0, 'incarnation': 83, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.151:6835/1977294668', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6834', 'nonce': 1977294668}, {'type': 'v1', 'addr': '172.21.15.151:6835', 'nonce': 1977294668}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [19], 'metadata_pool': 18, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9009, 'qdb_cluster': [9009]} 2024-09-16T15:53:57.657 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-16T15:53:57.657 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-16T15:53:57.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.814+0000 7f0bfc400640 1 -- 172.21.15.151:0/1952243566 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0bf4073e00 msgr2=0x7f0bf4168e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:57.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.814+0000 7f0bfc400640 1 --2- 172.21.15.151:0/1952243566 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0bf4073e00 0x7f0bf4168e10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0bec009be0 tx=0x7f0bec02a800 comp rx=0 tx=0).stop 2024-09-16T15:53:57.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.815+0000 7f0bfc400640 1 -- 172.21.15.151:0/1952243566 shutdown_connections 2024-09-16T15:53:57.817 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.815+0000 7f0bfc400640 1 --2- 172.21.15.151:0/1952243566 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0bf4073e00 0x7f0bf4168e10 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:57.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.815+0000 7f0bfc400640 1 -- 172.21.15.151:0/1952243566 >> 172.21.15.151:0/1952243566 conn(0x7f0bf40b1950 msgr2=0x7f0bf40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:57.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.815+0000 7f0bfc400640 1 -- 172.21.15.151:0/1952243566 shutdown_connections 2024-09-16T15:53:57.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.815+0000 7f0bfc400640 1 -- 172.21.15.151:0/1952243566 wait complete. 2024-09-16T15:53:57.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.816+0000 7f0bfc400640 1 Processor -- start 2024-09-16T15:53:57.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.816+0000 7f0bfc400640 1 -- start start 2024-09-16T15:53:57.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.817+0000 7f0bfc400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0bf4073e00 0x7f0bf414f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:57.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.817+0000 7f0bfc400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0bf40af3d0 0x7f0bf4148000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:57.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.817+0000 7f0bfc400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0bf4161550 0x7f0bf41f8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:57.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.817+0000 7f0bfc400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf41694d0 con 0x7f0bf4161550 2024-09-16T15:53:57.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.817+0000 7f0bfc400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf4169350 con 0x7f0bf4073e00 2024-09-16T15:53:57.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.817+0000 7f0bfc400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f0bf4169650 con 0x7f0bf40af3d0 2024-09-16T15:53:57.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.818+0000 7f0bfa400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0bf4161550 0x7f0bf41f8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:57.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.818+0000 7f0bfa400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0bf4161550 0x7f0bf41f8240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:54822/0 (socket says 172.21.15.151:54822) 2024-09-16T15:53:57.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.818+0000 7f0bfa400640 1 -- 172.21.15.151:0/385258505 learned_addr learned my addr 172.21.15.151:0/385258505 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:53:57.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.818+0000 7f0bf9000640 1 --2- 172.21.15.151:0/385258505 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0bf40af3d0 0x7f0bf4148000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:57.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.818+0000 7f0bf9a00640 1 --2- 172.21.15.151:0/385258505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0bf4073e00 0x7f0bf414f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:57.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.818+0000 7f0bfa400640 1 -- 172.21.15.151:0/385258505 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0bf40af3d0 msgr2=0x7f0bf4148000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:57.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.818+0000 7f0bfa400640 1 --2- 172.21.15.151:0/385258505 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0bf40af3d0 0x7f0bf4148000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:57.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.818+0000 7f0bfa400640 1 -- 172.21.15.151:0/385258505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0bf4073e00 msgr2=0x7f0bf414f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:57.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.818+0000 7f0bfa400640 1 --2- 172.21.15.151:0/385258505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0bf4073e00 0x7f0bf414f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:57.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.818+0000 7f0bfa400640 1 -- 172.21.15.151:0/385258505 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bf414f750 con 0x7f0bf4161550 2024-09-16T15:53:57.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.818+0000 7f0bfa400640 1 --2- 172.21.15.151:0/385258505 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0bf4161550 0x7f0bf41f8240 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f0bec034040 tx=0x7f0bec032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:57.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.819+0000 7f0bf1600640 1 -- 172.21.15.151:0/385258505 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bec041020 con 0x7f0bf4161550 2024-09-16T15:53:57.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.819+0000 7f0bf1600640 1 -- 172.21.15.151:0/385258505 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0bec004330 con 0x7f0bf4161550 2024-09-16T15:53:57.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.819+0000 7f0bfc400640 1 -- 172.21.15.151:0/385258505 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bf414f9f0 con 0x7f0bf4161550 2024-09-16T15:53:57.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.819+0000 7f0bf1600640 1 -- 172.21.15.151:0/385258505 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bec036040 con 0x7f0bf4161550 2024-09-16T15:53:57.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.819+0000 7f0bfc400640 1 -- 172.21.15.151:0/385258505 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bf4151fd0 con 0x7f0bf4161550 2024-09-16T15:53:57.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.821+0000 7f0bf1600640 1 -- 172.21.15.151:0/385258505 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 47) ==== 104092+0+0 (secure 0 0 0) 0x7f0bec044020 con 0x7f0bf4161550 2024-09-16T15:53:57.824 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.822+0000 7f0bf1600640 1 --2- 172.21.15.151:0/385258505 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0bd407a770 0x7f0bd407cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:57.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.822+0000 7f0bf1600640 1 -- 172.21.15.151:0/385258505 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5821+0+0 (secure 0 0 0) 0x7f0bec0ba8a0 con 0x7f0bf4161550 2024-09-16T15:53:57.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.822+0000 7f0bff481740 1 Processor -- start 2024-09-16T15:53:57.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.822+0000 7f0bff481740 1 -- start start 2024-09-16T15:53:57.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.823+0000 7f0bf9a00640 1 --2- 172.21.15.151:0/385258505 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0bd407a770 0x7f0bd407cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:57.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.823+0000 7f0bf9a00640 1 --2- 172.21.15.151:0/385258505 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0bd407a770 0x7f0bd407cc30 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f0be8001860 tx=0x7f0be8009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:57.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.824+0000 7f0bff481740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a053348b0 0x556a05334c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:57.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.824+0000 7f0bff481740 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556a052efa20 0x556a05333390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:57.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.824+0000 7f0bff481740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556a052effa0 0x556a053338d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:57.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.824+0000 7f0bff481740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x556a05272430 con 0x556a052effa0 2024-09-16T15:53:57.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.824+0000 7f0bff481740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556a052625d0 con 0x556a053348b0 2024-09-16T15:53:57.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.824+0000 7f0bff481740 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x556a05251a00 con 0x556a052efa20 2024-09-16T15:53:57.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.824+0000 7f0bfa400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556a052effa0 0x556a053338d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:57.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.824+0000 7f0bf9a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a053348b0 0x556a05334c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:57.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.824+0000 7f0bf9a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a053348b0 0x556a05334c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.151:44432/0 (socket says 172.21.15.151:44432) 2024-09-16T15:53:57.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.824+0000 7f0bf9a00640 1 -- 172.21.15.151:0/1915925814 learned_addr learned my addr 172.21.15.151:0/1915925814 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:53:57.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.824+0000 7f0bf9000640 1 --2- 172.21.15.151:0/1915925814 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556a052efa20 0x556a05333390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:57.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.824+0000 7f0bfa400640 1 -- 172.21.15.151:0/1915925814 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556a052efa20 msgr2=0x556a05333390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:53:57.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.824+0000 7f0bfa400640 1 --2- 172.21.15.151:0/1915925814 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556a052efa20 0x556a05333390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:57.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.824+0000 7f0bfa400640 1 -- 172.21.15.151:0/1915925814 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a053348b0 msgr2=0x556a05334c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:53:57.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.824+0000 7f0bfa400640 1 --2- 172.21.15.151:0/1915925814 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a053348b0 0x556a05334c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:57.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.824+0000 7f0bfa400640 1 -- 172.21.15.151:0/1915925814 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a0521a520 con 0x556a052effa0 2024-09-16T15:53:57.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.825+0000 7f0bfa400640 1 --2- 172.21.15.151:0/1915925814 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556a052effa0 0x556a053338d0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f0bec0bad50 tx=0x7f0bec083cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d1f73ef036b2f9c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:57.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.825+0000 7f0be6000640 1 -- 172.21.15.151:0/1915925814 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bec089e60 con 0x556a052effa0 2024-09-16T15:53:57.828 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.825+0000 7f0be6000640 1 -- 172.21.15.151:0/1915925814 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0bec083db0 con 0x556a052effa0 2024-09-16T15:53:57.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.825+0000 7f0be6000640 1 -- 172.21.15.151:0/1915925814 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bec0842a0 con 0x556a052effa0 2024-09-16T15:53:57.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.825+0000 7f0bff481740 1 -- 172.21.15.151:0/1915925814 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556a052effa0 msgr2=0x556a053338d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:57.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.825+0000 7f0bff481740 1 --2- 172.21.15.151:0/1915925814 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556a052effa0 0x556a053338d0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f0bec0bad50 tx=0x7f0bec083cb0 comp rx=0 tx=0).stop 2024-09-16T15:53:57.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.826+0000 7f0bff481740 1 -- 172.21.15.151:0/1915925814 shutdown_connections 2024-09-16T15:53:57.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.826+0000 7f0bff481740 1 --2- 172.21.15.151:0/1915925814 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556a052effa0 0x556a053338d0 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:57.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.826+0000 7f0bff481740 1 --2- 172.21.15.151:0/1915925814 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556a052efa20 0x556a05333390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:57.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.826+0000 7f0bff481740 1 --2- 172.21.15.151:0/1915925814 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a053348b0 0x556a05334c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:57.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.826+0000 7f0bff481740 1 -- 172.21.15.151:0/1915925814 >> 172.21.15.151:0/1915925814 conn(0x556a052eaff0 msgr2=0x556a052eea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:57.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.826+0000 7f0bff481740 1 -- 172.21.15.151:0/1915925814 shutdown_connections 2024-09-16T15:53:57.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.826+0000 7f0bff481740 1 -- 172.21.15.151:0/1915925814 wait complete. 2024-09-16T15:53:57.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.826+0000 7f0bff481740 1 -- start start 2024-09-16T15:53:57.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.826+0000 7f0bff481740 1 Processor -- start 2024-09-16T15:53:57.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.827+0000 7f0bff481740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a052effa0 0x556a05329ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:57.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.827+0000 7f0bff481740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556a053348b0 0x556a05312b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:57.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.827+0000 7f0bff481740 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556a0532b910 0x556a05313870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:57.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.827+0000 7f0bff481740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x556a051f8330 con 0x556a053348b0 2024-09-16T15:53:57.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.827+0000 7f0bff481740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556a05255810 con 0x556a052effa0 2024-09-16T15:53:57.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.827+0000 7f0bff481740 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x556a04fc2620 con 0x556a0532b910 2024-09-16T15:53:57.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.828+0000 7f0bff481740 10 client.0 _ll_register_callbacks cb 0x556a051fd420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T15:53:57.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.828+0000 7f0bf9000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556a053348b0 0x556a05312b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:57.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.828+0000 7f0bf9000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556a053348b0 0x556a05312b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:54846/0 (socket says 172.21.15.151:54846) 2024-09-16T15:53:57.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.828+0000 7f0bf9000640 1 -- 172.21.15.151:0/3602603018 learned_addr learned my addr 172.21.15.151:0/3602603018 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:53:57.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.828+0000 7f0bf9a00640 1 --2- 172.21.15.151:0/3602603018 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a052effa0 0x556a05329ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:57.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.828+0000 7f0bfa400640 1 --2- 172.21.15.151:0/3602603018 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556a0532b910 0x556a05313870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:57.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.828+0000 7f0bf9000640 1 -- 172.21.15.151:0/3602603018 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556a0532b910 msgr2=0x556a05313870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:53:57.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.828+0000 7f0bf9000640 1 --2- 172.21.15.151:0/3602603018 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556a0532b910 0x556a05313870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:57.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.828+0000 7f0bf9000640 1 -- 172.21.15.151:0/3602603018 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a052effa0 msgr2=0x556a05329ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:53:57.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.828+0000 7f0bf9000640 1 --2- 172.21.15.151:0/3602603018 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a052effa0 0x556a05329ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:57.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.828+0000 7f0bf9000640 1 -- 172.21.15.151:0/3602603018 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a04fb3850 con 0x556a053348b0 2024-09-16T15:53:57.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.828+0000 7f0bf9000640 1 --2- 172.21.15.151:0/3602603018 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556a053348b0 0x556a05312b10 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f0be0012370 tx=0x7f0be0012770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=239155bd26e3276a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:57.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.828+0000 7f0bdaa00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-16T15:53:57.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.829+0000 7f0bdaa00640 1 -- 172.21.15.151:0/3602603018 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be0019020 con 0x556a053348b0 2024-09-16T15:53:57.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.829+0000 7f0bdaa00640 1 -- 172.21.15.151:0/3602603018 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(133..133 src has 1..133) ==== 5821+0+0 (secure 0 0 0) 0x7f0be00052e0 con 0x556a053348b0 2024-09-16T15:53:57.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.829+0000 7f0bff481740 1 -- 172.21.15.151:0/3602603018 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bec083db0 con 0x556a053348b0 2024-09-16T15:53:57.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.829+0000 7f0bdaa00640 1 -- 172.21.15.151:0/3602603018 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f0be0023050 con 0x556a053348b0 2024-09-16T15:53:57.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.829+0000 7f0bff481740 10 client.9141 fetch_fsmap learned FSMap version 87 2024-09-16T15:53:57.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.829+0000 7f0bff481740 1 -- 172.21.15.151:0/3602603018 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) -- 0x556a0502a6b0 con 0x556a053348b0 2024-09-16T15:53:57.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.829+0000 7f0bdaa00640 1 -- 172.21.15.151:0/3602603018 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0bec083db0 con 0x556a053348b0 2024-09-16T15:53:57.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.829+0000 7f0bdaa00640 1 -- 172.21.15.151:0/3602603018 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 87) ==== 3502+0+0 (secure 0 0 0) 0x7f0be0026020 con 0x556a053348b0 2024-09-16T15:53:57.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.829+0000 7f0bff481740 10 client.9141 fetch_fsmap finished waiting for FSMap version 87 2024-09-16T15:53:57.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.829+0000 7f0bff481740 10 client.9141 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-16T15:53:57.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.829+0000 7f0bff481740 20 client.9141 populate_metadata read hostname 'smithi151' 2024-09-16T15:53:57.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.829+0000 7f0bff481740 1 --2- 172.21.15.151:0/3602603018 >> [v2:172.21.15.151:6834/1977294668,v1:172.21.15.151:6835/1977294668] conn(0x556a053470d0 0x556a053474b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:57.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.829+0000 7f0bff481740 4 client.9141 mds_command: new command op to 9009 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:53:57.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.829+0000 7f0bff481740 1 -- 172.21.15.151:0/3602603018 --> [v2:172.21.15.151:6834/1977294668,v1:172.21.15.151:6835/1977294668] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a05264520 con 0x556a053470d0 2024-09-16T15:53:57.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.830+0000 7f0bf9a00640 1 --2- 172.21.15.151:0/3602603018 >> [v2:172.21.15.151:6834/1977294668,v1:172.21.15.151:6835/1977294668] conn(0x556a053470d0 0x556a053474b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:57.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.830+0000 7f0bf9a00640 1 --2- 172.21.15.151:0/3602603018 >> [v2:172.21.15.151:6834/1977294668,v1:172.21.15.151:6835/1977294668] conn(0x556a053470d0 0x556a053474b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c5b52452d812065 server_cookie=ad513e4d2c60283a in_seq=0 out_seq=0 2024-09-16T15:53:57.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.830+0000 7f0bdaa00640 10 client.9141 ms_handle_connect on v2:172.21.15.151:6834/1977294668 2024-09-16T15:53:57.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.831+0000 7f0bdaa00640 1 -- 172.21.15.151:0/3602603018 <== mds.0 v2:172.21.15.151:6834/1977294668 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556a05264520 con 0x556a053470d0 2024-09-16T15:53:57.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.831+0000 7f0bdaa00640 10 client.9141 handle_command_reply: tid=0 2024-09-16T15:53:57.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.831+0000 7f0bff481740 1 client.9141 shutdown 2024-09-16T15:53:57.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.831+0000 7f0bff481740 1 -- 172.21.15.151:0/3602603018 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556a053348b0 msgr2=0x556a05312b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:57.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.831+0000 7f0bff481740 1 --2- 172.21.15.151:0/3602603018 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556a053348b0 0x556a05312b10 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f0be0012370 tx=0x7f0be0012770 comp rx=0 tx=0).stop 2024-09-16T15:53:57.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.832+0000 7f0bff481740 1 -- 172.21.15.151:0/3602603018 shutdown_connections 2024-09-16T15:53:57.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.832+0000 7f0bff481740 1 --2- 172.21.15.151:0/3602603018 >> [v2:172.21.15.151:6834/1977294668,v1:172.21.15.151:6835/1977294668] conn(0x556a053470d0 0x556a053474b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:57.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.832+0000 7f0bff481740 1 --2- 172.21.15.151:0/3602603018 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556a0532b910 0x556a05313870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:57.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.832+0000 7f0bff481740 1 --2- 172.21.15.151:0/3602603018 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556a053348b0 0x556a05312b10 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:57.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.832+0000 7f0bff481740 1 --2- 172.21.15.151:0/3602603018 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a052effa0 0x556a05329ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:57.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.832+0000 7f0bdaa00640 0 client.9141 ms_handle_reset on v2:172.21.15.151:6834/1977294668 2024-09-16T15:53:57.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.832+0000 7f0bff481740 1 -- 172.21.15.151:0/3602603018 >> 172.21.15.151:0/3602603018 conn(0x556a052efa20 msgr2=0x556a052e8660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:57.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.832+0000 7f0bff481740 1 -- 172.21.15.151:0/3602603018 shutdown_connections 2024-09-16T15:53:57.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.832+0000 7f0bff481740 1 -- 172.21.15.151:0/3602603018 wait complete. 2024-09-16T15:53:57.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.832+0000 7f0bff481740 20 client.9141 trim_cache size 0 max 16384 2024-09-16T15:53:57.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.851+0000 7f0bff481740 1 Processor -- start 2024-09-16T15:53:57.853 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.851+0000 7f0bff481740 1 -- start start 2024-09-16T15:53:57.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.852+0000 7f0bff481740 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556a052effa0 0x556a0532c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:57.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.852+0000 7f0bff481740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556a0532b910 0x556a05336880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:57.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.852+0000 7f0bff481740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a053348b0 0x556a053336a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:57.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.852+0000 7f0bff481740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf4200220 con 0x556a0532b910 2024-09-16T15:53:57.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.852+0000 7f0bff481740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf41ff5b0 con 0x556a053348b0 2024-09-16T15:53:57.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.852+0000 7f0bff481740 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f0bf41ffc00 con 0x556a052effa0 2024-09-16T15:53:57.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.852+0000 7f0bf9000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556a0532b910 0x556a05336880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:57.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.852+0000 7f0bf9000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556a0532b910 0x556a05336880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:54856/0 (socket says 172.21.15.151:54856) 2024-09-16T15:53:57.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.852+0000 7f0bf9000640 1 -- 172.21.15.151:0/1910987985 learned_addr learned my addr 172.21.15.151:0/1910987985 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:53:57.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.852+0000 7f0bf9a00640 1 --2- 172.21.15.151:0/1910987985 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556a052effa0 0x556a0532c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:57.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.852+0000 7f0bfa400640 1 --2- 172.21.15.151:0/1910987985 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a053348b0 0x556a053336a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:57.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.852+0000 7f0bf9000640 1 -- 172.21.15.151:0/1910987985 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556a052effa0 msgr2=0x556a0532c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:53:57.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.852+0000 7f0bf9000640 1 --2- 172.21.15.151:0/1910987985 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556a052effa0 0x556a0532c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:57.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.852+0000 7f0bf9000640 1 -- 172.21.15.151:0/1910987985 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a053348b0 msgr2=0x556a053336a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:53:57.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.852+0000 7f0bf9000640 1 --2- 172.21.15.151:0/1910987985 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a053348b0 0x556a053336a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:57.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.852+0000 7f0bf9000640 1 -- 172.21.15.151:0/1910987985 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a05313630 con 0x556a0532b910 2024-09-16T15:53:57.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.852+0000 7f0bf9000640 1 --2- 172.21.15.151:0/1910987985 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556a0532b910 0x556a05336880 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f0be0007a80 tx=0x7f0be0007ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5520ae9854ceb1b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:57.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.853+0000 7f0be5600640 1 -- 172.21.15.151:0/1910987985 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be002c070 con 0x556a0532b910 2024-09-16T15:53:57.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.853+0000 7f0be5600640 1 -- 172.21.15.151:0/1910987985 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0be0003840 con 0x556a0532b910 2024-09-16T15:53:57.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.853+0000 7f0be5600640 1 -- 172.21.15.151:0/1910987985 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be0019070 con 0x556a0532b910 2024-09-16T15:53:57.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.853+0000 7f0bff481740 1 -- 172.21.15.151:0/1910987985 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556a0532b910 msgr2=0x556a05336880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:57.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.853+0000 7f0bff481740 1 --2- 172.21.15.151:0/1910987985 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556a0532b910 0x556a05336880 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f0be0007a80 tx=0x7f0be0007ab0 comp rx=0 tx=0).stop 2024-09-16T15:53:57.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.853+0000 7f0bff481740 1 -- 172.21.15.151:0/1910987985 shutdown_connections 2024-09-16T15:53:57.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.853+0000 7f0bff481740 1 --2- 172.21.15.151:0/1910987985 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a053348b0 0x556a053336a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:57.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.853+0000 7f0bff481740 1 --2- 172.21.15.151:0/1910987985 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556a0532b910 0x556a05336880 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:57.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.853+0000 7f0bff481740 1 --2- 172.21.15.151:0/1910987985 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556a052effa0 0x556a0532c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:57.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.853+0000 7f0bff481740 1 -- 172.21.15.151:0/1910987985 >> 172.21.15.151:0/1910987985 conn(0x556a052efa20 msgr2=0x556a0532ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:57.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.853+0000 7f0bff481740 1 -- 172.21.15.151:0/1910987985 shutdown_connections 2024-09-16T15:53:57.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.853+0000 7f0bff481740 1 -- 172.21.15.151:0/1910987985 wait complete. 2024-09-16T15:53:57.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.854+0000 7f0bff481740 1 -- start start 2024-09-16T15:53:57.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.854+0000 7f0bff481740 1 Processor -- start 2024-09-16T15:53:57.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.855+0000 7f0bff481740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556a052effa0 0x556a053b4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:57.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.855+0000 7f0bff481740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a0532b910 0x556a053336a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:57.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.855+0000 7f0bff481740 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556a053348b0 0x556a05333be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:57.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.855+0000 7f0bff481740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x556a05313140 con 0x556a052effa0 2024-09-16T15:53:57.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.855+0000 7f0bff481740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556a0532bd80 con 0x556a0532b910 2024-09-16T15:53:57.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.855+0000 7f0bff481740 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x556a05343ee0 con 0x556a053348b0 2024-09-16T15:53:57.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.855+0000 7f0bff481740 10 client.0 _ll_register_callbacks cb 0x7f0be0001b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T15:53:57.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.855+0000 7f0bf9a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556a052effa0 0x556a053b4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:57.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.855+0000 7f0bf9a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556a052effa0 0x556a053b4020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:54864/0 (socket says 172.21.15.151:54864) 2024-09-16T15:53:57.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.855+0000 7f0bf9a00640 1 -- 172.21.15.151:0/3924580201 learned_addr learned my addr 172.21.15.151:0/3924580201 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:53:57.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.856+0000 7f0bf9000640 1 --2- 172.21.15.151:0/3924580201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a0532b910 0x556a053336a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:57.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.856+0000 7f0bfa400640 1 --2- 172.21.15.151:0/3924580201 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556a053348b0 0x556a05333be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:57.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.856+0000 7f0bf9a00640 1 -- 172.21.15.151:0/3924580201 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556a053348b0 msgr2=0x556a05333be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:53:57.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.856+0000 7f0bf9a00640 1 --2- 172.21.15.151:0/3924580201 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556a053348b0 0x556a05333be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:57.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.856+0000 7f0bf9a00640 1 -- 172.21.15.151:0/3924580201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a0532b910 msgr2=0x556a053336a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:53:57.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.856+0000 7f0bf9a00640 1 --2- 172.21.15.151:0/3924580201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a0532b910 0x556a053336a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:57.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.856+0000 7f0bf9a00640 1 -- 172.21.15.151:0/3924580201 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a0534d0d0 con 0x556a052effa0 2024-09-16T15:53:57.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.856+0000 7f0bf9a00640 1 --2- 172.21.15.151:0/3924580201 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556a052effa0 0x556a053b4020 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f0be8001bd0 tx=0x7f0be8001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54543b12fa952997 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:57.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.856+0000 7f0bdaa00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-16T15:53:57.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.856+0000 7f0bdaa00640 1 -- 172.21.15.151:0/3924580201 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be8030060 con 0x556a052effa0 2024-09-16T15:53:57.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.856+0000 7f0bdaa00640 1 -- 172.21.15.151:0/3924580201 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(133..133 src has 1..133) ==== 5821+0+0 (secure 0 0 0) 0x7f0be8023720 con 0x556a052effa0 2024-09-16T15:53:57.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.856+0000 7f0bff481740 1 -- 172.21.15.151:0/3924580201 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0be0003840 con 0x556a052effa0 2024-09-16T15:53:57.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.857+0000 7f0bdaa00640 1 -- 172.21.15.151:0/3924580201 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f0be8036050 con 0x556a052effa0 2024-09-16T15:53:57.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.857+0000 7f0bff481740 10 client.9147 fetch_fsmap learned FSMap version 87 2024-09-16T15:53:57.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.857+0000 7f0bff481740 1 -- 172.21.15.151:0/3924580201 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) -- 0x556a053436e0 con 0x556a052effa0 2024-09-16T15:53:57.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.857+0000 7f0bdaa00640 1 -- 172.21.15.151:0/3924580201 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0be0003840 con 0x556a052effa0 2024-09-16T15:53:57.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.857+0000 7f0bdaa00640 1 -- 172.21.15.151:0/3924580201 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 87) ==== 3502+0+0 (secure 0 0 0) 0x7f0be8039020 con 0x556a052effa0 2024-09-16T15:53:57.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.857+0000 7f0bff481740 10 client.9147 fetch_fsmap finished waiting for FSMap version 87 2024-09-16T15:53:57.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.857+0000 7f0bff481740 10 client.9147 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-16T15:53:57.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.857+0000 7f0bff481740 20 client.9147 populate_metadata read hostname 'smithi151' 2024-09-16T15:53:57.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.857+0000 7f0bff481740 1 --2- 172.21.15.151:0/3924580201 >> [v2:172.21.15.151:6834/1977294668,v1:172.21.15.151:6835/1977294668] conn(0x556a052e93c0 0x556a053bfe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:57.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.857+0000 7f0bff481740 4 client.9147 mds_command: new command op to 9009 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T15:53:57.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.857+0000 7f0bff481740 1 -- 172.21.15.151:0/3924580201 --> [v2:172.21.15.151:6834/1977294668,v1:172.21.15.151:6835/1977294668] -- command(tid 0: {"prefix": "status"}) -- 0x556a05331340 con 0x556a052e93c0 2024-09-16T15:53:57.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.857+0000 7f0bf9000640 1 --2- 172.21.15.151:0/3924580201 >> [v2:172.21.15.151:6834/1977294668,v1:172.21.15.151:6835/1977294668] conn(0x556a052e93c0 0x556a053bfe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:57.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.858+0000 7f0bf9000640 1 --2- 172.21.15.151:0/3924580201 >> [v2:172.21.15.151:6834/1977294668,v1:172.21.15.151:6835/1977294668] conn(0x556a052e93c0 0x556a053bfe80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a00a061ee40a11aa server_cookie=6e98e195b3a26f67 in_seq=0 out_seq=0 2024-09-16T15:53:57.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.858+0000 7f0bdaa00640 10 client.9147 ms_handle_connect on v2:172.21.15.151:6834/1977294668 2024-09-16T15:53:57.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.858+0000 7f0bdaa00640 1 -- 172.21.15.151:0/3924580201 <== mds.0 v2:172.21.15.151:6834/1977294668 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x556a05331340 con 0x556a052e93c0 2024-09-16T15:53:57.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.858+0000 7f0bdaa00640 10 client.9147 handle_command_reply: tid=0 2024-09-16T15:53:57.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.858+0000 7f0bff481740 1 client.9147 shutdown 2024-09-16T15:53:57.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.858+0000 7f0bff481740 1 -- 172.21.15.151:0/3924580201 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556a052effa0 msgr2=0x556a053b4020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:57.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.858+0000 7f0bff481740 1 --2- 172.21.15.151:0/3924580201 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556a052effa0 0x556a053b4020 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f0be8001bd0 tx=0x7f0be8001a40 comp rx=0 tx=0).stop 2024-09-16T15:53:57.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.858+0000 7f0bff481740 1 -- 172.21.15.151:0/3924580201 shutdown_connections 2024-09-16T15:53:57.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.858+0000 7f0bff481740 1 --2- 172.21.15.151:0/3924580201 >> [v2:172.21.15.151:6834/1977294668,v1:172.21.15.151:6835/1977294668] conn(0x556a052e93c0 0x556a053bfe80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:57.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.858+0000 7f0bff481740 1 --2- 172.21.15.151:0/3924580201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a0532b910 0x556a053336a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:57.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.858+0000 7f0bff481740 1 --2- 172.21.15.151:0/3924580201 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x556a053348b0 0x556a05333be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:57.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.858+0000 7f0bff481740 1 --2- 172.21.15.151:0/3924580201 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556a052effa0 0x556a053b4020 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:57.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.858+0000 7f0bdaa00640 0 client.9147 ms_handle_reset on v2:172.21.15.151:6834/1977294668 2024-09-16T15:53:57.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.858+0000 7f0bff481740 1 -- 172.21.15.151:0/3924580201 >> 172.21.15.151:0/3924580201 conn(0x556a052efa20 msgr2=0x556a05336880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:57.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.858+0000 7f0bff481740 1 -- 172.21.15.151:0/3924580201 shutdown_connections 2024-09-16T15:53:57.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.858+0000 7f0bff481740 1 -- 172.21.15.151:0/3924580201 wait complete. 2024-09-16T15:53:57.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.859+0000 7f0bff481740 20 client.9147 trim_cache size 0 max 16384 2024-09-16T15:53:57.866 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-16T15:53:57.866 INFO:teuthology.orchestra.run.smithi151.stdout: "cluster_fsid": "1757be18-e41a-4446-bb4f-ee4ac81c39df", 2024-09-16T15:53:57.866 INFO:teuthology.orchestra.run.smithi151.stdout: "whoami": 0, 2024-09-16T15:53:57.866 INFO:teuthology.orchestra.run.smithi151.stdout: "id": 9009, 2024-09-16T15:53:57.866 INFO:teuthology.orchestra.run.smithi151.stdout: "want_state": "up:active", 2024-09-16T15:53:57.866 INFO:teuthology.orchestra.run.smithi151.stdout: "state": "up:active", 2024-09-16T15:53:57.866 INFO:teuthology.orchestra.run.smithi151.stdout: "fs_name": "cephfs", 2024-09-16T15:53:57.866 INFO:teuthology.orchestra.run.smithi151.stdout: "rank_uptime": 10.830119822, 2024-09-16T15:53:57.866 INFO:teuthology.orchestra.run.smithi151.stdout: "mdsmap_epoch": 87, 2024-09-16T15:53:57.867 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_epoch": 133, 2024-09-16T15:53:57.867 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_epoch_barrier": 133, 2024-09-16T15:53:57.867 INFO:teuthology.orchestra.run.smithi151.stdout: "uptime": 16.338080560000002 2024-09-16T15:53:57.867 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-16T15:53:57.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.859+0000 7f0bfc400640 1 -- 172.21.15.151:0/385258505 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0bd407a770 msgr2=0x7f0bd407cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:57.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.859+0000 7f0bfc400640 1 --2- 172.21.15.151:0/385258505 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0bd407a770 0x7f0bd407cc30 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f0be8001860 tx=0x7f0be8009040 comp rx=0 tx=0).stop 2024-09-16T15:53:57.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.859+0000 7f0bfc400640 1 -- 172.21.15.151:0/385258505 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0bf4161550 msgr2=0x7f0bf41f8240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:57.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.859+0000 7f0bfc400640 1 --2- 172.21.15.151:0/385258505 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0bf4161550 0x7f0bf41f8240 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f0bec034040 tx=0x7f0bec032d10 comp rx=0 tx=0).stop 2024-09-16T15:53:57.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.859+0000 7f0bfc400640 1 -- 172.21.15.151:0/385258505 shutdown_connections 2024-09-16T15:53:57.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.859+0000 7f0bfc400640 1 --2- 172.21.15.151:0/385258505 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0bd407a770 0x7f0bd407cc30 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:57.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.859+0000 7f0bfc400640 1 --2- 172.21.15.151:0/385258505 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0bf4161550 0x7f0bf41f8240 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:57.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.859+0000 7f0bfc400640 1 --2- 172.21.15.151:0/385258505 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0bf40af3d0 0x7f0bf4148000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:57.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.859+0000 7f0bfc400640 1 --2- 172.21.15.151:0/385258505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0bf4073e00 0x7f0bf414f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:57.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.859+0000 7f0bfc400640 1 -- 172.21.15.151:0/385258505 >> 172.21.15.151:0/385258505 conn(0x7f0bf40b1950 msgr2=0x7f0bf40adef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:57.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.859+0000 7f0bfc400640 1 -- 172.21.15.151:0/385258505 shutdown_connections 2024-09-16T15:53:57.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:57.859+0000 7f0bfc400640 1 -- 172.21.15.151:0/385258505 wait complete. 2024-09-16T15:53:57.877 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-16T15:53:58.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:58.032+0000 7f46ba400640 1 -- 172.21.15.151:0/2201930952 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f46b40aecc0 msgr2=0x7f46b415dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:58.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:58.032+0000 7f46ba400640 1 --2- 172.21.15.151:0/2201930952 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f46b40aecc0 0x7f46b415dc30 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f46a0009900 tx=0x7f46a002a600 comp rx=0 tx=0).stop 2024-09-16T15:53:58.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:58.032+0000 7f46ba400640 1 -- 172.21.15.151:0/2201930952 shutdown_connections 2024-09-16T15:53:58.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:58.032+0000 7f46ba400640 1 --2- 172.21.15.151:0/2201930952 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f46b40aecc0 0x7f46b415dc30 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:58.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:58.032+0000 7f46ba400640 1 --2- 172.21.15.151:0/2201930952 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f46b40caa50 0x7f46b40cae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:58.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:58.032+0000 7f46ba400640 1 -- 172.21.15.151:0/2201930952 >> 172.21.15.151:0/2201930952 conn(0x7f46b40cfa20 msgr2=0x7f46b40cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:58.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:58.032+0000 7f46ba400640 1 -- 172.21.15.151:0/2201930952 shutdown_connections 2024-09-16T15:53:58.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:58.032+0000 7f46ba400640 1 -- 172.21.15.151:0/2201930952 wait complete. 2024-09-16T15:53:58.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:58.034+0000 7f46ba400640 1 Processor -- start 2024-09-16T15:53:58.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:58.034+0000 7f46ba400640 1 -- start start 2024-09-16T15:53:58.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:58.035+0000 7f46ba400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f46b40aecc0 0x7f46b4157f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:58.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:58.035+0000 7f46ba400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f46b40caa50 0x7f46b41f50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:58.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:58.035+0000 7f46ba400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46b40cb370 0x7f46b40afaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:58.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:58.035+0000 7f46ba400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f46b415e500 con 0x7f46b40caa50 2024-09-16T15:53:58.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:58.035+0000 7f46ba400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f46b415e380 con 0x7f46b40cb370 2024-09-16T15:53:58.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:58.035+0000 7f46ba400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f46b415e680 con 0x7f46b40aecc0 2024-09-16T15:53:58.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:58.035+0000 7f46b3e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f46b40caa50 0x7f46b41f50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:58.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:58.035+0000 7f46b3e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f46b40caa50 0x7f46b41f50c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:54888/0 (socket says 172.21.15.151:54888) 2024-09-16T15:53:58.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:58.035+0000 7f46b3e00640 1 -- 172.21.15.151:0/1844047939 learned_addr learned my addr 172.21.15.151:0/1844047939 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:53:58.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:58.035+0000 7f46b9000640 1 --2- 172.21.15.151:0/1844047939 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f46b40aecc0 0x7f46b4157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:58.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:58.035+0000 7f46b9a00640 1 --2- 172.21.15.151:0/1844047939 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46b40cb370 0x7f46b40afaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:58.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:58.036+0000 7f46b3e00640 1 -- 172.21.15.151:0/1844047939 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f46b40aecc0 msgr2=0x7f46b4157f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:58.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:58.036+0000 7f46b3e00640 1 --2- 172.21.15.151:0/1844047939 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f46b40aecc0 0x7f46b4157f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:58.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:58.036+0000 7f46b3e00640 1 -- 172.21.15.151:0/1844047939 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46b40cb370 msgr2=0x7f46b40afaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:58.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:58.036+0000 7f46b3e00640 1 --2- 172.21.15.151:0/1844047939 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46b40cb370 0x7f46b40afaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:58.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:58.036+0000 7f46b3e00640 1 -- 172.21.15.151:0/1844047939 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46b40b0030 con 0x7f46b40caa50 2024-09-16T15:53:58.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:58.036+0000 7f46b3e00640 1 --2- 172.21.15.151:0/1844047939 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f46b40caa50 0x7f46b41f50c0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f46a0009a20 tx=0x7f46a00038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:58.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:58.036+0000 7f46b0c00640 1 -- 172.21.15.151:0/1844047939 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a003f070 con 0x7f46b40caa50 2024-09-16T15:53:58.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:58.036+0000 7f46b0c00640 1 -- 172.21.15.151:0/1844047939 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f46a00047a0 con 0x7f46b40caa50 2024-09-16T15:53:58.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:58.036+0000 7f46b0c00640 1 -- 172.21.15.151:0/1844047939 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a0035030 con 0x7f46b40caa50 2024-09-16T15:53:58.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:58.036+0000 7f46ba400640 1 -- 172.21.15.151:0/1844047939 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46b4158610 con 0x7f46b40caa50 2024-09-16T15:53:58.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:58.036+0000 7f46ba400640 1 -- 172.21.15.151:0/1844047939 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46b40af2f0 con 0x7f46b40caa50 2024-09-16T15:53:58.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:58.037+0000 7f46ba400640 1 -- 172.21.15.151:0/1844047939 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46b415dc30 con 0x7f46b40caa50 2024-09-16T15:53:58.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:58.039+0000 7f46b0c00640 1 -- 172.21.15.151:0/1844047939 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 47) ==== 104092+0+0 (secure 0 0 0) 0x7f46a0042070 con 0x7f46b40caa50 2024-09-16T15:53:58.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:58.040+0000 7f46b0c00640 1 --2- 172.21.15.151:0/1844047939 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f469407a740 0x7f469407cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:58.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:58.040+0000 7f46b0c00640 1 -- 172.21.15.151:0/1844047939 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5821+0+0 (secure 0 0 0) 0x7f46a00ba9f0 con 0x7f46b40caa50 2024-09-16T15:53:58.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:58.040+0000 7f46b9000640 1 --2- 172.21.15.151:0/1844047939 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f469407a740 0x7f469407cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:58.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:58.041+0000 7f46b9000640 1 --2- 172.21.15.151:0/1844047939 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f469407a740 0x7f469407cc00 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f46a8009bd0 tx=0x7f46a8005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:58.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:58.044+0000 7f46b0c00640 1 -- 172.21.15.151:0/1844047939 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f46a0084470 con 0x7f46b40caa50 2024-09-16T15:53:58.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:58.265+0000 7f46ba400640 1 -- 172.21.15.151:0/1844047939 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f46b40ce980 con 0x7f46b40caa50 2024-09-16T15:53:58.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:58.265+0000 7f46b0c00640 1 -- 172.21.15.151:0/1844047939 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v87) ==== 53+0+73 (secure 0 0 0) 0x7f46a0088ea0 con 0x7f46b40caa50 2024-09-16T15:53:58.267 INFO:teuthology.orchestra.run.smithi151.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-16T15:53:58.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:58.267+0000 7f46ba400640 1 -- 172.21.15.151:0/1844047939 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f469407a740 msgr2=0x7f469407cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:58.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:58.267+0000 7f46ba400640 1 --2- 172.21.15.151:0/1844047939 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f469407a740 0x7f469407cc00 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f46a8009bd0 tx=0x7f46a8005f90 comp rx=0 tx=0).stop 2024-09-16T15:53:58.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:58.268+0000 7f46ba400640 1 -- 172.21.15.151:0/1844047939 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f46b40caa50 msgr2=0x7f46b41f50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:58.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:58.268+0000 7f46ba400640 1 --2- 172.21.15.151:0/1844047939 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f46b40caa50 0x7f46b41f50c0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f46a0009a20 tx=0x7f46a00038a0 comp rx=0 tx=0).stop 2024-09-16T15:53:58.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:58.268+0000 7f46ba400640 1 -- 172.21.15.151:0/1844047939 shutdown_connections 2024-09-16T15:53:58.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:58.268+0000 7f46ba400640 1 --2- 172.21.15.151:0/1844047939 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f469407a740 0x7f469407cc00 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:58.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:58.268+0000 7f46ba400640 1 --2- 172.21.15.151:0/1844047939 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f46b40cb370 0x7f46b40afaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:58.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:58.268+0000 7f46ba400640 1 --2- 172.21.15.151:0/1844047939 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f46b40caa50 0x7f46b41f50c0 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:58.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:58.268+0000 7f46ba400640 1 --2- 172.21.15.151:0/1844047939 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f46b40aecc0 0x7f46b4157f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:58.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:58.268+0000 7f46ba400640 1 -- 172.21.15.151:0/1844047939 >> 172.21.15.151:0/1844047939 conn(0x7f46b40cfa20 msgr2=0x7f46b4074390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:58.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:58.268+0000 7f46ba400640 1 -- 172.21.15.151:0/1844047939 shutdown_connections 2024-09-16T15:53:58.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:58.268+0000 7f46ba400640 1 -- 172.21.15.151:0/1844047939 wait complete. 2024-09-16T15:53:58.286 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-16T15:53:58.286 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T15:53:58.286 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi151.front.sepia.ceph.com 2024-09-16T15:53:58.286 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-16T15:53:58.286 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-16T15:53:58.286 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-16T15:53:58.286 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-16T15:53:58.286 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T15:53:58.286 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T15:53:58.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:53:58.287 DEBUG:teuthology.orchestra.run.smithi151:> ip addr 2024-09-16T15:53:58.322 INFO:teuthology.orchestra.run.smithi151.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T15:53:58.322 INFO:teuthology.orchestra.run.smithi151.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T15:53:58.323 INFO:teuthology.orchestra.run.smithi151.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T15:53:58.323 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:53:58.323 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 ::1/128 scope host 2024-09-16T15:53:58.323 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:53:58.323 INFO:teuthology.orchestra.run.smithi151.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:53:58.323 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:da:6e:0c brd ff:ff:ff:ff:ff:ff 2024-09-16T15:53:58.323 INFO:teuthology.orchestra.run.smithi151.stdout: altname enp7s0f0 2024-09-16T15:53:58.323 INFO:teuthology.orchestra.run.smithi151.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:53:58.323 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:da:6e:0d brd ff:ff:ff:ff:ff:ff 2024-09-16T15:53:58.323 INFO:teuthology.orchestra.run.smithi151.stdout: altname enp7s0f1 2024-09-16T15:53:58.323 INFO:teuthology.orchestra.run.smithi151.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:53:58.323 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:88:6f:62 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:53:58.323 INFO:teuthology.orchestra.run.smithi151.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T15:53:58.323 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:88:6f:63 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:53:58.323 INFO:teuthology.orchestra.run.smithi151.stdout: inet 172.21.15.151/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-16T15:53:58.324 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft 41851sec preferred_lft 41851sec 2024-09-16T15:53:58.324 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 fe80::ec4:7aff:fe88:6f63/64 scope link noprefixroute 2024-09-16T15:53:58.324 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:53:58.324 INFO:teuthology.orchestra.run.smithi151.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-16T15:53:58.324 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 62:3f:3b:48:ad:f8 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:53:58.324 INFO:teuthology.orchestra.run.smithi151.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-16T15:53:58.324 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:53:58.324 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 fe80::40fa:7fff:fedf:2a00/64 scope link 2024-09-16T15:53:58.324 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:53:58.324 INFO:teuthology.orchestra.run.smithi151.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T15:53:58.324 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 62:3f:3b:48:ad:f8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T15:53:58.324 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 fe80::603f:3bff:fe48:adf8/64 scope link 2024-09-16T15:53:58.324 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:53:58.324 INFO:teuthology.orchestra.run.smithi151.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T15:53:58.324 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether b6:ac:fe:34:a7:66 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-16T15:53:58.325 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 fe80::b4ac:feff:fe34:a766/64 scope link 2024-09-16T15:53:58.325 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:53:58.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:53:58.326 DEBUG:teuthology.orchestra.run.smithi151:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T15:53:58.419 INFO:teuthology.orchestra.run.smithi151.stdout:1 2024-09-16T15:53:58.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:53:58.422 DEBUG:teuthology.orchestra.run.smithi151:> route 2024-09-16T15:53:58.458 INFO:teuthology.orchestra.run.smithi151.stdout:Kernel IP routing table 2024-09-16T15:53:58.458 INFO:teuthology.orchestra.run.smithi151.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T15:53:58.458 INFO:teuthology.orchestra.run.smithi151.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-16T15:53:58.458 INFO:teuthology.orchestra.run.smithi151.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-16T15:53:58.458 INFO:teuthology.orchestra.run.smithi151.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T15:53:58.458 DEBUG:teuthology.orchestra.run.smithi151:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T15:53:58.458 DEBUG:teuthology.orchestra.run.smithi151:> set -e 2024-09-16T15:53:58.458 DEBUG:teuthology.orchestra.run.smithi151:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-16T15:53:58.458 DEBUG:teuthology.orchestra.run.smithi151:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-16T15:53:58.458 DEBUG:teuthology.orchestra.run.smithi151:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-16T15:53:58.458 DEBUG:teuthology.orchestra.run.smithi151:> ') 2024-09-16T15:53:58.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:53:58.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:53:58.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:53:58.655 DEBUG:teuthology.orchestra.run.smithi151:> ip netns list 2024-09-16T15:53:58.691 INFO:teuthology.orchestra.run.smithi151.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-16T15:53:58.691 INFO:teuthology.orchestra.run.smithi151.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-16T15:53:58.691 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T15:53:58.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:53:58.691 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip link set brx.0 up 2024-09-16T15:53:58.787 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:53:58.787 DEBUG:teuthology.orchestra.run.smithi151:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-16T15:53:58.822 INFO:teuthology.orchestra.run.smithi151.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-16T15:53:58.822 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:53:58.822 DEBUG:teuthology.orchestra.run.smithi151:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-16T15:53:58.896 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-16T15:53:58.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:53:58.896 DEBUG:teuthology.orchestra.run.smithi151:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-16T15:53:59.025 INFO:teuthology.orchestra.run.smithi151.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-16T15:53:59.025 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-16T15:53:59.025 INFO:teuthology.orchestra.run.smithi151.stdout:invalid new device string format 2024-09-16T15:53:59.040 INFO:teuthology.orchestra.run.smithi151.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-16T15:53:59.040 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-16T15:53:59.040 INFO:teuthology.orchestra.run.smithi151.stdout:invalid new device string format 2024-09-16T15:53:59.040 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: resolved to: "172.21.15.151:3300,172.21.15.167:3300,172.21.15.167:3301" 2024-09-16T15:53:59.041 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-16T15:53:59.061 INFO:teuthology.orchestra.run.smithi151.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-16T15:53:59.061 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-16T15:53:59.061 INFO:teuthology.orchestra.run.smithi151.stdout:invalid new device string format 2024-09-16T15:53:59.062 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: resolved to: "172.21.15.151:3300,172.21.15.167:3300,172.21.15.167:3301" 2024-09-16T15:53:59.062 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-16T15:53:59.062 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: resolved to: "172.21.15.151:6789,172.21.15.167:6789,172.21.15.167:6790" 2024-09-16T15:53:59.062 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.151:6789,172.21.15.167:6789,172.21.15.167:6790:/ 2024-09-16T15:53:59.062 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=1757be18-e41a-4446-bb4f-ee4ac81c39df" will pass to kernel 2024-09-16T15:53:59.063 INFO:teuthology.orchestra.run.smithi151.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-16T15:53:59.063 INFO:teuthology.orchestra.run.smithi151.stdout: You just mounted a file system that supports labels which does not 2024-09-16T15:53:59.063 INFO:teuthology.orchestra.run.smithi151.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-16T15:53:59.064 INFO:teuthology.orchestra.run.smithi151.stdout: applications will generate AVC messages and not be allowed access to 2024-09-16T15:53:59.064 INFO:teuthology.orchestra.run.smithi151.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-16T15:53:59.066 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-16T15:53:59.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:53:59.067 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-16T15:53:59.122 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T15:53:59.202 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:59 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:53:59.209 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:53:59.217 INFO:teuthology.orchestra.run.smithi151.stdout:client7511 2024-09-16T15:53:59.217 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:53:59.217 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:53:59.253 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-16T15:53:59.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:59.455+0000 7fbf50600640 1 -- 172.21.15.151:0/2836972238 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbf48074630 msgr2=0x7fbf48070320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:59.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:59.455+0000 7fbf50600640 1 --2- 172.21.15.151:0/2836972238 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbf48074630 0x7fbf48070320 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbf34009860 tx=0x7fbf3402a480 comp rx=0 tx=0).stop 2024-09-16T15:53:59.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:59.455+0000 7fbf50600640 1 -- 172.21.15.151:0/2836972238 shutdown_connections 2024-09-16T15:53:59.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:59.455+0000 7fbf50600640 1 --2- 172.21.15.151:0/2836972238 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbf48074630 0x7fbf48070320 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:59.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:59.455+0000 7fbf50600640 1 -- 172.21.15.151:0/2836972238 >> 172.21.15.151:0/2836972238 conn(0x7fbf480b18e0 msgr2=0x7fbf480b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:53:59.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:59.455+0000 7fbf50600640 1 -- 172.21.15.151:0/2836972238 shutdown_connections 2024-09-16T15:53:59.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:59.455+0000 7fbf50600640 1 -- 172.21.15.151:0/2836972238 wait complete. 2024-09-16T15:53:59.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:59.456+0000 7fbf50600640 1 Processor -- start 2024-09-16T15:53:59.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:59.457+0000 7fbf50600640 1 -- start start 2024-09-16T15:53:59.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:59.457+0000 7fbf50600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbf48074630 0x7fbf48158b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:59.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:59.457+0000 7fbf50600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbf480b4880 0x7fbf4815ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:59.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:59.458+0000 7fbf50600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf4815a3b0 0x7fbf480add80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:59.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:59.458+0000 7fbf50600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbf480ce6e0 con 0x7fbf48074630 2024-09-16T15:53:59.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:59.458+0000 7fbf50600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbf480ce560 con 0x7fbf4815a3b0 2024-09-16T15:53:59.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:59.458+0000 7fbf50600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fbf4815eb00 con 0x7fbf480b4880 2024-09-16T15:53:59.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:59.458+0000 7fbf4dc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbf48074630 0x7fbf48158b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:59.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:59.458+0000 7fbf4dc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbf48074630 0x7fbf48158b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:54898/0 (socket says 172.21.15.151:54898) 2024-09-16T15:53:59.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:59.458+0000 7fbf4dc00640 1 -- 172.21.15.151:0/115475699 learned_addr learned my addr 172.21.15.151:0/115475699 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:53:59.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:59.458+0000 7fbf4dc00640 1 -- 172.21.15.151:0/115475699 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbf480b4880 msgr2=0x7fbf4815ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:59.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:59.458+0000 7fbf4dc00640 1 --2- 172.21.15.151:0/115475699 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbf480b4880 0x7fbf4815ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:59.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:59.458+0000 7fbf4dc00640 1 -- 172.21.15.151:0/115475699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf4815a3b0 msgr2=0x7fbf480add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:53:59.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:59.458+0000 7fbf4dc00640 1 --2- 172.21.15.151:0/115475699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf4815a3b0 0x7fbf480add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:53:59.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:59.458+0000 7fbf4dc00640 1 -- 172.21.15.151:0/115475699 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf4809bf10 con 0x7fbf48074630 2024-09-16T15:53:59.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:59.458+0000 7fbf4dc00640 1 --2- 172.21.15.151:0/115475699 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbf48074630 0x7fbf48158b90 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c00c770 tx=0x7fbf3c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:59.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:59.459+0000 7fbf45600640 1 -- 172.21.15.151:0/115475699 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c003b70 con 0x7fbf48074630 2024-09-16T15:53:59.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:59.459+0000 7fbf45600640 1 -- 172.21.15.151:0/115475699 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbf3c003d10 con 0x7fbf48074630 2024-09-16T15:53:59.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:59.459+0000 7fbf45600640 1 -- 172.21.15.151:0/115475699 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c00aab0 con 0x7fbf48074630 2024-09-16T15:53:59.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:59.459+0000 7fbf50600640 1 -- 172.21.15.151:0/115475699 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf4809c1f0 con 0x7fbf48074630 2024-09-16T15:53:59.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:59.459+0000 7fbf50600640 1 -- 172.21.15.151:0/115475699 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf4809c7b0 con 0x7fbf48074630 2024-09-16T15:53:59.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:59.460+0000 7fbf50600640 1 -- 172.21.15.151:0/115475699 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf48069360 con 0x7fbf48074630 2024-09-16T15:53:59.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:59.461+0000 7fbf45600640 1 -- 172.21.15.151:0/115475699 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 47) ==== 104092+0+0 (secure 0 0 0) 0x7fbf3c00ac50 con 0x7fbf48074630 2024-09-16T15:53:59.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:59.462+0000 7fbf45600640 1 --2- 172.21.15.151:0/115475699 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbf2807a740 0x7fbf2807cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:53:59.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:59.462+0000 7fbf45600640 1 -- 172.21.15.151:0/115475699 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5821+0+0 (secure 0 0 0) 0x7fbf3c099840 con 0x7fbf48074630 2024-09-16T15:53:59.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:59.463+0000 7fbf4d200640 1 --2- 172.21.15.151:0/115475699 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbf2807a740 0x7fbf2807cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:53:59.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:59.463+0000 7fbf4d200640 1 --2- 172.21.15.151:0/115475699 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbf2807a740 0x7fbf2807cc00 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fbf34034000 tx=0x7fbf340046c0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:53:59.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:59.466+0000 7fbf45600640 1 -- 172.21.15.151:0/115475699 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fbf3c063910 con 0x7fbf48074630 2024-09-16T15:53:59.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:53:59.691+0000 7fbf50600640 1 -- 172.21.15.151:0/115475699 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fbf480cc910 con 0x7fbf48074630 2024-09-16T15:54:00.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:00.536+0000 7fbf45600640 1 -- 172.21.15.151:0/115475699 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v88) ==== 136+0+0 (secure 0 0 0) 0x7fbf3c063210 con 0x7fbf48074630 2024-09-16T15:54:00.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:00.538+0000 7fbf50600640 1 -- 172.21.15.151:0/115475699 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbf2807a740 msgr2=0x7fbf2807cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:00.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:00.538+0000 7fbf50600640 1 --2- 172.21.15.151:0/115475699 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbf2807a740 0x7fbf2807cc00 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fbf34034000 tx=0x7fbf340046c0 comp rx=0 tx=0).stop 2024-09-16T15:54:00.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:00.538+0000 7fbf50600640 1 -- 172.21.15.151:0/115475699 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbf48074630 msgr2=0x7fbf48158b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:00.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:00.539+0000 7fbf50600640 1 --2- 172.21.15.151:0/115475699 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbf48074630 0x7fbf48158b90 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c00c770 tx=0x7fbf3c00cc30 comp rx=0 tx=0).stop 2024-09-16T15:54:00.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:00.539+0000 7fbf50600640 1 -- 172.21.15.151:0/115475699 shutdown_connections 2024-09-16T15:54:00.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:00.539+0000 7fbf50600640 1 --2- 172.21.15.151:0/115475699 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbf2807a740 0x7fbf2807cc00 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:00.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:00.539+0000 7fbf50600640 1 --2- 172.21.15.151:0/115475699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf4815a3b0 0x7fbf480add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:00.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:00.539+0000 7fbf50600640 1 --2- 172.21.15.151:0/115475699 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbf480b4880 0x7fbf4815ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:00.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:00.539+0000 7fbf50600640 1 --2- 172.21.15.151:0/115475699 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbf48074630 0x7fbf48158b90 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:00.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:00.539+0000 7fbf50600640 1 -- 172.21.15.151:0/115475699 >> 172.21.15.151:0/115475699 conn(0x7fbf480b18e0 msgr2=0x7fbf480b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:00.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:00.539+0000 7fbf50600640 1 -- 172.21.15.151:0/115475699 shutdown_connections 2024-09-16T15:54:00.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:00.539+0000 7fbf50600640 1 -- 172.21.15.151:0/115475699 wait complete. 2024-09-16T15:54:00.559 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-16T15:54:00.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:00.719+0000 7f1a27000640 1 -- 172.21.15.151:0/2142675441 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1a200b5370 msgr2=0x7f1a200b3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:00.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:00.719+0000 7f1a27000640 1 --2- 172.21.15.151:0/2142675441 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1a200b5370 0x7f1a200b3bc0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c009860 tx=0x7f1a0c02a4a0 comp rx=0 tx=0).stop 2024-09-16T15:54:00.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:00.719+0000 7f1a27000640 1 -- 172.21.15.151:0/2142675441 shutdown_connections 2024-09-16T15:54:00.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:00.719+0000 7f1a27000640 1 --2- 172.21.15.151:0/2142675441 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1a200b5370 0x7f1a200b3bc0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:00.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:00.719+0000 7f1a27000640 1 -- 172.21.15.151:0/2142675441 >> 172.21.15.151:0/2142675441 conn(0x7f1a200d3df0 msgr2=0x7f1a200d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:00.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:00.719+0000 7f1a27000640 1 -- 172.21.15.151:0/2142675441 shutdown_connections 2024-09-16T15:54:00.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:00.719+0000 7f1a27000640 1 -- 172.21.15.151:0/2142675441 wait complete. 2024-09-16T15:54:00.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:00.721+0000 7f1a27000640 1 Processor -- start 2024-09-16T15:54:00.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:00.721+0000 7f1a27000640 1 -- start start 2024-09-16T15:54:00.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:00.722+0000 7f1a27000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a200b4100 0x7f1a200c7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:00.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:00.722+0000 7f1a27000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1a200b5370 0x7f1a201f4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:00.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:00.722+0000 7f1a27000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a20161440 0x7f1a20155ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:00.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:00.722+0000 7f1a27000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1a200b2210 con 0x7f1a200b4100 2024-09-16T15:54:00.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:00.722+0000 7f1a27000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1a200b2090 con 0x7f1a20161440 2024-09-16T15:54:00.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:00.722+0000 7f1a27000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f1a200b2390 con 0x7f1a200b5370 2024-09-16T15:54:00.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:00.722+0000 7f1a25c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a200b4100 0x7f1a200c7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:00.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:00.722+0000 7f1a25c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a200b4100 0x7f1a200c7920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:54900/0 (socket says 172.21.15.151:54900) 2024-09-16T15:54:00.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:00.722+0000 7f1a25c00640 1 -- 172.21.15.151:0/3627902928 learned_addr learned my addr 172.21.15.151:0/3627902928 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:54:00.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:00.723+0000 7f1a26600640 1 --2- 172.21.15.151:0/3627902928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a20161440 0x7f1a20155ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:00.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:00.723+0000 7f1a25200640 1 --2- 172.21.15.151:0/3627902928 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1a200b5370 0x7f1a201f4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:00.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:00.723+0000 7f1a25c00640 1 -- 172.21.15.151:0/3627902928 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1a200b5370 msgr2=0x7f1a201f4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:00.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:00.723+0000 7f1a25c00640 1 --2- 172.21.15.151:0/3627902928 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1a200b5370 0x7f1a201f4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:00.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:00.723+0000 7f1a25c00640 1 -- 172.21.15.151:0/3627902928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a20161440 msgr2=0x7f1a20155ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:00.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:00.723+0000 7f1a25c00640 1 --2- 172.21.15.151:0/3627902928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a20161440 0x7f1a20155ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:00.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:00.723+0000 7f1a25c00640 1 -- 172.21.15.151:0/3627902928 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a200cdae0 con 0x7f1a200b4100 2024-09-16T15:54:00.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:00.723+0000 7f1a25c00640 1 --2- 172.21.15.151:0/3627902928 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a200b4100 0x7f1a200c7920 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f1a1400c390 tx=0x7f1a1400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:00.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:00.723+0000 7f1a1d600640 1 -- 172.21.15.151:0/3627902928 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a14017070 con 0x7f1a200b4100 2024-09-16T15:54:00.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:00.723+0000 7f1a1d600640 1 -- 172.21.15.151:0/3627902928 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1a140053c0 con 0x7f1a200b4100 2024-09-16T15:54:00.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:00.723+0000 7f1a1d600640 1 -- 172.21.15.151:0/3627902928 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a14003850 con 0x7f1a200b4100 2024-09-16T15:54:00.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:00.723+0000 7f1a27000640 1 -- 172.21.15.151:0/3627902928 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a200cddc0 con 0x7f1a200b4100 2024-09-16T15:54:00.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:00.723+0000 7f1a27000640 1 -- 172.21.15.151:0/3627902928 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a200b8080 con 0x7f1a200b4100 2024-09-16T15:54:00.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:00.725+0000 7f1a27000640 1 -- 172.21.15.151:0/3627902928 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a200b27b0 con 0x7f1a200b4100 2024-09-16T15:54:00.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:00.726+0000 7f1a1d600640 1 -- 172.21.15.151:0/3627902928 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 47) ==== 104092+0+0 (secure 0 0 0) 0x7f1a1401a070 con 0x7f1a200b4100 2024-09-16T15:54:00.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:00.727+0000 7f1a1d600640 1 --2- 172.21.15.151:0/3627902928 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1a0007a740 0x7f1a0007cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:00.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:00.727+0000 7f1a1d600640 1 -- 172.21.15.151:0/3627902928 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5821+0+0 (secure 0 0 0) 0x7f1a14098de0 con 0x7f1a200b4100 2024-09-16T15:54:00.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:00.728+0000 7f1a25200640 1 --2- 172.21.15.151:0/3627902928 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1a0007a740 0x7f1a0007cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:00.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:00.728+0000 7f1a25200640 1 --2- 172.21.15.151:0/3627902928 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1a0007a740 0x7f1a0007cc00 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c034000 tx=0x7f1a0c004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:00.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:00.732+0000 7f1a1d600640 1 -- 172.21.15.151:0/3627902928 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1a14062eb0 con 0x7f1a200b4100 2024-09-16T15:54:00.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:00.938+0000 7f1a27000640 1 -- 172.21.15.151:0/3627902928 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f1a200cd890 con 0x7f1a0007a740 2024-09-16T15:54:00.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:00.940+0000 7f1a1d600640 1 -- 172.21.15.151:0/3627902928 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f1a200cd890 con 0x7f1a0007a740 2024-09-16T15:54:00.942 INFO:teuthology.orchestra.run.smithi151.stdout:[ 2024-09-16T15:54:00.942 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:54:00.942 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "cephfs" 2024-09-16T15:54:00.942 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:54:00.942 INFO:teuthology.orchestra.run.smithi151.stdout:] 2024-09-16T15:54:00.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:00.942+0000 7f1a27000640 1 -- 172.21.15.151:0/3627902928 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1a0007a740 msgr2=0x7f1a0007cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:00.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:00.942+0000 7f1a27000640 1 --2- 172.21.15.151:0/3627902928 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1a0007a740 0x7f1a0007cc00 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c034000 tx=0x7f1a0c004730 comp rx=0 tx=0).stop 2024-09-16T15:54:00.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:00.942+0000 7f1a27000640 1 -- 172.21.15.151:0/3627902928 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a200b4100 msgr2=0x7f1a200c7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:00.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:00.942+0000 7f1a27000640 1 --2- 172.21.15.151:0/3627902928 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a200b4100 0x7f1a200c7920 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f1a1400c390 tx=0x7f1a1400c850 comp rx=0 tx=0).stop 2024-09-16T15:54:00.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:00.942+0000 7f1a27000640 1 -- 172.21.15.151:0/3627902928 shutdown_connections 2024-09-16T15:54:00.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:00.942+0000 7f1a27000640 1 --2- 172.21.15.151:0/3627902928 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1a0007a740 0x7f1a0007cc00 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:00.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:00.942+0000 7f1a27000640 1 --2- 172.21.15.151:0/3627902928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a20161440 0x7f1a20155ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:00.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:00.942+0000 7f1a27000640 1 --2- 172.21.15.151:0/3627902928 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1a200b5370 0x7f1a201f4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:00.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:00.942+0000 7f1a27000640 1 --2- 172.21.15.151:0/3627902928 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a200b4100 0x7f1a200c7920 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:00.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:00.942+0000 7f1a27000640 1 -- 172.21.15.151:0/3627902928 >> 172.21.15.151:0/3627902928 conn(0x7f1a200d3df0 msgr2=0x7f1a2006d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:00.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:00.942+0000 7f1a27000640 1 -- 172.21.15.151:0/3627902928 shutdown_connections 2024-09-16T15:54:00.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:00.942+0000 7f1a27000640 1 -- 172.21.15.151:0/3627902928 wait complete. 2024-09-16T15:54:00.960 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-16T15:54:01.122 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.120+0000 7f630da00640 1 -- 172.21.15.151:0/2019575842 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63080739d0 msgr2=0x7f630815e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:01.122 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.120+0000 7f630da00640 1 --2- 172.21.15.151:0/2019575842 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63080739d0 0x7f630815e0f0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f63080b23e0 tx=0x7f630002a8b0 comp rx=0 tx=0).stop 2024-09-16T15:54:01.122 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.120+0000 7f630da00640 1 -- 172.21.15.151:0/2019575842 shutdown_connections 2024-09-16T15:54:01.122 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.120+0000 7f630da00640 1 --2- 172.21.15.151:0/2019575842 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63080739d0 0x7f630815e0f0 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:01.122 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.120+0000 7f630da00640 1 --2- 172.21.15.151:0/2019575842 >> v2:172.21.15.167:3300/0 conn(0x7f63080cd490 0x7f63080cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:01.122 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.120+0000 7f630da00640 1 -- 172.21.15.151:0/2019575842 >> 172.21.15.151:0/2019575842 conn(0x7f63080b1840 msgr2=0x7f63080b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:01.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.120+0000 7f630da00640 1 -- 172.21.15.151:0/2019575842 shutdown_connections 2024-09-16T15:54:01.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.120+0000 7f630da00640 1 -- 172.21.15.151:0/2019575842 wait complete. 2024-09-16T15:54:01.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.121+0000 7f630da00640 1 Processor -- start 2024-09-16T15:54:01.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.121+0000 7f630da00640 1 -- start start 2024-09-16T15:54:01.125 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.122+0000 7f630da00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f63080739d0 0x7f6308139f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:01.125 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.122+0000 7f630da00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63080cd490 0x7f63080b0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:01.125 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.122+0000 7f630da00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63081517a0 0x7f6308151c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:01.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.122+0000 7f630da00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f63080cc7d0 con 0x7f63081517a0 2024-09-16T15:54:01.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.122+0000 7f630da00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f63080cc650 con 0x7f63080cd490 2024-09-16T15:54:01.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.122+0000 7f630da00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f63080cc950 con 0x7f63080739d0 2024-09-16T15:54:01.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.123+0000 7f6307400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63081517a0 0x7f6308151c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:01.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.123+0000 7f6307400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63081517a0 0x7f6308151c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:54906/0 (socket says 172.21.15.151:54906) 2024-09-16T15:54:01.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.123+0000 7f6307400640 1 -- 172.21.15.151:0/2605415229 learned_addr learned my addr 172.21.15.151:0/2605415229 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:54:01.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.123+0000 7f6306a00640 1 --2- 172.21.15.151:0/2605415229 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f63080739d0 0x7f6308139f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:01.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.123+0000 7f62ffe00640 1 --2- 172.21.15.151:0/2605415229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63080cd490 0x7f63080b0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:01.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.123+0000 7f6307400640 1 -- 172.21.15.151:0/2605415229 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f63080739d0 msgr2=0x7f6308139f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:01.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.123+0000 7f6307400640 1 --2- 172.21.15.151:0/2605415229 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f63080739d0 0x7f6308139f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:01.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.123+0000 7f6307400640 1 -- 172.21.15.151:0/2605415229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63080cd490 msgr2=0x7f63080b0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:01.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.123+0000 7f6307400640 1 --2- 172.21.15.151:0/2605415229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63080cd490 0x7f63080b0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:01.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.123+0000 7f6307400640 1 -- 172.21.15.151:0/2605415229 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f630813a460 con 0x7f63081517a0 2024-09-16T15:54:01.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.123+0000 7f6307400640 1 --2- 172.21.15.151:0/2605415229 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63081517a0 0x7f6308151c20 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f6300005fd0 tx=0x7f6300005c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:01.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.124+0000 7f62fea00640 1 -- 172.21.15.151:0/2605415229 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6300040070 con 0x7f63081517a0 2024-09-16T15:54:01.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.124+0000 7f62fea00640 1 -- 172.21.15.151:0/2605415229 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6300004040 con 0x7f63081517a0 2024-09-16T15:54:01.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.124+0000 7f630da00640 1 -- 172.21.15.151:0/2605415229 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f630814db10 con 0x7f63081517a0 2024-09-16T15:54:01.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.124+0000 7f62fea00640 1 -- 172.21.15.151:0/2605415229 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6300004470 con 0x7f63081517a0 2024-09-16T15:54:01.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.124+0000 7f630da00640 1 -- 172.21.15.151:0/2605415229 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6308154280 con 0x7f63081517a0 2024-09-16T15:54:01.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.125+0000 7f630da00640 1 -- 172.21.15.151:0/2605415229 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63080cdad0 con 0x7f63081517a0 2024-09-16T15:54:01.128 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.126+0000 7f62fea00640 1 -- 172.21.15.151:0/2605415229 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 47) ==== 104092+0+0 (secure 0 0 0) 0x7f6300039040 con 0x7f63081517a0 2024-09-16T15:54:01.129 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.127+0000 7f62fea00640 1 --2- 172.21.15.151:0/2605415229 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f62e407a740 0x7f62e407cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:01.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.127+0000 7f62fea00640 1 -- 172.21.15.151:0/2605415229 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5821+0+0 (secure 0 0 0) 0x7f63000baea0 con 0x7f63081517a0 2024-09-16T15:54:01.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.128+0000 7f6306a00640 1 --2- 172.21.15.151:0/2605415229 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f62e407a740 0x7f62e407cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:01.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.128+0000 7f6306a00640 1 --2- 172.21.15.151:0/2605415229 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f62e407a740 0x7f62e407cc00 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f62f8007fd0 tx=0x7f62f8001370 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:01.133 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.131+0000 7f62fea00640 1 -- 172.21.15.151:0/2605415229 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f63000848a0 con 0x7f63081517a0 2024-09-16T15:54:01.321 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.319+0000 7f630da00640 1 -- 172.21.15.151:0/2605415229 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f63080b2430 con 0x7f63081517a0 2024-09-16T15:54:01.321 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.320+0000 7f62fea00640 1 -- 172.21.15.151:0/2605415229 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v51)=0 v51) ==== 122+0+0 (secure 0 0 0) 0x7f63000892d0 con 0x7f63081517a0 2024-09-16T15:54:01.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.322+0000 7f630da00640 1 -- 172.21.15.151:0/2605415229 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f62e407a740 msgr2=0x7f62e407cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:01.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.322+0000 7f630da00640 1 --2- 172.21.15.151:0/2605415229 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f62e407a740 0x7f62e407cc00 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f62f8007fd0 tx=0x7f62f8001370 comp rx=0 tx=0).stop 2024-09-16T15:54:01.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.322+0000 7f630da00640 1 -- 172.21.15.151:0/2605415229 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63081517a0 msgr2=0x7f6308151c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:01.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.322+0000 7f630da00640 1 --2- 172.21.15.151:0/2605415229 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63081517a0 0x7f6308151c20 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f6300005fd0 tx=0x7f6300005c80 comp rx=0 tx=0).stop 2024-09-16T15:54:01.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.322+0000 7f630da00640 1 -- 172.21.15.151:0/2605415229 shutdown_connections 2024-09-16T15:54:01.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.322+0000 7f630da00640 1 --2- 172.21.15.151:0/2605415229 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f62e407a740 0x7f62e407cc00 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:01.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.322+0000 7f630da00640 1 --2- 172.21.15.151:0/2605415229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63080cd490 0x7f63080b0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:01.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.322+0000 7f630da00640 1 --2- 172.21.15.151:0/2605415229 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63081517a0 0x7f6308151c20 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:01.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.322+0000 7f630da00640 1 --2- 172.21.15.151:0/2605415229 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f63080739d0 0x7f6308139f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:01.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.322+0000 7f630da00640 1 -- 172.21.15.151:0/2605415229 >> 172.21.15.151:0/2605415229 conn(0x7f63080b1840 msgr2=0x7f63080d0080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:01.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.322+0000 7f630da00640 1 -- 172.21.15.151:0/2605415229 shutdown_connections 2024-09-16T15:54:01.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.322+0000 7f630da00640 1 -- 172.21.15.151:0/2605415229 wait complete. 2024-09-16T15:54:01.341 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mgr mgr/volumes/max_concurrent_clones 2024-09-16T15:54:01.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.500+0000 7f6446800640 1 -- 172.21.15.151:0/2671704396 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f64400cbc00 msgr2=0x7f644006fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:01.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.500+0000 7f6446800640 1 --2- 172.21.15.151:0/2671704396 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f64400cbc00 0x7f644006fce0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f642c009830 tx=0x7f642c02a510 comp rx=0 tx=0).stop 2024-09-16T15:54:01.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.500+0000 7f6446800640 1 -- 172.21.15.151:0/2671704396 shutdown_connections 2024-09-16T15:54:01.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.500+0000 7f6446800640 1 --2- 172.21.15.151:0/2671704396 >> v2:172.21.15.167:3300/0 conn(0x7f6440072560 0x7f6440073ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:01.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.500+0000 7f6446800640 1 --2- 172.21.15.151:0/2671704396 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f64400cbc00 0x7f644006fce0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:01.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.500+0000 7f6446800640 1 -- 172.21.15.151:0/2671704396 >> 172.21.15.151:0/2671704396 conn(0x7f64400cf9f0 msgr2=0x7f64400cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:01.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.500+0000 7f6446800640 1 -- 172.21.15.151:0/2671704396 shutdown_connections 2024-09-16T15:54:01.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.500+0000 7f6446800640 1 -- 172.21.15.151:0/2671704396 wait complete. 2024-09-16T15:54:01.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.502+0000 7f6446800640 1 Processor -- start 2024-09-16T15:54:01.504 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.502+0000 7f6446800640 1 -- start start 2024-09-16T15:54:01.506 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.503+0000 7f6446800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6440072560 0x7f64400ceea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:01.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.503+0000 7f6446800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f64400cbc00 0x7f644015a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:01.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.503+0000 7f6446800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f644015bec0 0x7f644015a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:01.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.503+0000 7f6446800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f64400ae990 con 0x7f644015bec0 2024-09-16T15:54:01.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.503+0000 7f6446800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f64400ae810 con 0x7f6440072560 2024-09-16T15:54:01.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.503+0000 7f6446800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f64400aeb10 con 0x7f64400cbc00 2024-09-16T15:54:01.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.503+0000 7f6445e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f644015bec0 0x7f644015a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:01.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.503+0000 7f6445400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6440072560 0x7f64400ceea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:01.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.503+0000 7f6445e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f644015bec0 0x7f644015a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:54922/0 (socket says 172.21.15.151:54922) 2024-09-16T15:54:01.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.503+0000 7f6445e00640 1 -- 172.21.15.151:0/892998244 learned_addr learned my addr 172.21.15.151:0/892998244 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:54:01.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.503+0000 7f6444a00640 1 --2- 172.21.15.151:0/892998244 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f64400cbc00 0x7f644015a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:01.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.504+0000 7f6445e00640 1 -- 172.21.15.151:0/892998244 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f64400cbc00 msgr2=0x7f644015a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:01.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.504+0000 7f6445e00640 1 --2- 172.21.15.151:0/892998244 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f64400cbc00 0x7f644015a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:01.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.504+0000 7f6445e00640 1 -- 172.21.15.151:0/892998244 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6440072560 msgr2=0x7f64400ceea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:01.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.504+0000 7f6445e00640 1 --2- 172.21.15.151:0/892998244 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6440072560 0x7f64400ceea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:01.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.504+0000 7f6445e00640 1 -- 172.21.15.151:0/892998244 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64401f80a0 con 0x7f644015bec0 2024-09-16T15:54:01.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.504+0000 7f6445400640 1 --2- 172.21.15.151:0/892998244 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6440072560 0x7f64400ceea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:54:01.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.504+0000 7f6445e00640 1 --2- 172.21.15.151:0/892998244 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f644015bec0 0x7f644015a6e0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f6438009fa0 tx=0x7f643800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:01.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.504+0000 7f643d600640 1 -- 172.21.15.151:0/892998244 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6438004b20 con 0x7f644015bec0 2024-09-16T15:54:01.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.504+0000 7f643d600640 1 -- 172.21.15.151:0/892998244 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6438004db0 con 0x7f644015bec0 2024-09-16T15:54:01.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.504+0000 7f643d600640 1 -- 172.21.15.151:0/892998244 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6438003bd0 con 0x7f644015bec0 2024-09-16T15:54:01.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.504+0000 7f6446800640 1 -- 172.21.15.151:0/892998244 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64401f8380 con 0x7f644015bec0 2024-09-16T15:54:01.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.504+0000 7f6446800640 1 -- 172.21.15.151:0/892998244 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64401f8940 con 0x7f644015bec0 2024-09-16T15:54:01.509 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.506+0000 7f6446800640 1 -- 172.21.15.151:0/892998244 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64400af030 con 0x7f644015bec0 2024-09-16T15:54:01.509 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.507+0000 7f643d600640 1 -- 172.21.15.151:0/892998244 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 47) ==== 104092+0+0 (secure 0 0 0) 0x7f643801d020 con 0x7f644015bec0 2024-09-16T15:54:01.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.508+0000 7f643d600640 1 --2- 172.21.15.151:0/892998244 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f642007a740 0x7f642007cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:01.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.508+0000 7f643d600640 1 -- 172.21.15.151:0/892998244 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5821+0+0 (secure 0 0 0) 0x7f6438099f00 con 0x7f644015bec0 2024-09-16T15:54:01.510 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.508+0000 7f6445400640 1 --2- 172.21.15.151:0/892998244 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f642007a740 0x7f642007cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:01.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.509+0000 7f6445400640 1 --2- 172.21.15.151:0/892998244 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f642007a740 0x7f642007cc00 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f6434001b10 tx=0x7f6434009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:01.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.512+0000 7f643d600640 1 -- 172.21.15.151:0/892998244 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6438063a20 con 0x7f644015bec0 2024-09-16T15:54:01.702 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.700+0000 7f6446800640 1 -- 172.21.15.151:0/892998244 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"} v 0) -- 0x7f64400b4e80 con 0x7f644015bec0 2024-09-16T15:54:01.703 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.701+0000 7f643d600640 1 -- 172.21.15.151:0/892998244 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"}]=0 v51) ==== 116+0+2 (secure 0 0 0) 0x7f6438068450 con 0x7f644015bec0 2024-09-16T15:54:01.703 INFO:teuthology.orchestra.run.smithi151.stdout:4 2024-09-16T15:54:01.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.703+0000 7f6446800640 1 -- 172.21.15.151:0/892998244 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f642007a740 msgr2=0x7f642007cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:01.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.703+0000 7f6446800640 1 --2- 172.21.15.151:0/892998244 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f642007a740 0x7f642007cc00 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f6434001b10 tx=0x7f6434009040 comp rx=0 tx=0).stop 2024-09-16T15:54:01.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.703+0000 7f6446800640 1 -- 172.21.15.151:0/892998244 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f644015bec0 msgr2=0x7f644015a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:01.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.703+0000 7f6446800640 1 --2- 172.21.15.151:0/892998244 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f644015bec0 0x7f644015a6e0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f6438009fa0 tx=0x7f643800c800 comp rx=0 tx=0).stop 2024-09-16T15:54:01.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.704+0000 7f6446800640 1 -- 172.21.15.151:0/892998244 shutdown_connections 2024-09-16T15:54:01.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.704+0000 7f6446800640 1 --2- 172.21.15.151:0/892998244 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f642007a740 0x7f642007cc00 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:01.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.704+0000 7f6446800640 1 --2- 172.21.15.151:0/892998244 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f644015bec0 0x7f644015a6e0 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:01.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.704+0000 7f6446800640 1 --2- 172.21.15.151:0/892998244 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f64400cbc00 0x7f644015a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:01.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.704+0000 7f6446800640 1 --2- 172.21.15.151:0/892998244 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6440072560 0x7f64400ceea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:01.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.704+0000 7f6446800640 1 -- 172.21.15.151:0/892998244 >> 172.21.15.151:0/892998244 conn(0x7f64400cf9f0 msgr2=0x7f644015e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:01.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.704+0000 7f6446800640 1 -- 172.21.15.151:0/892998244 shutdown_connections 2024-09-16T15:54:01.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.704+0000 7f6446800640 1 -- 172.21.15.151:0/892998244 wait complete. 2024-09-16T15:54:01.722 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-16T15:54:01.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.881+0000 7f1dffe00640 1 -- 172.21.15.151:0/3077646806 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1df8145750 msgr2=0x7f1df8073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:01.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.881+0000 7f1dffe00640 1 --2- 172.21.15.151:0/3077646806 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1df8145750 0x7f1df8073440 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1dec009860 tx=0x7f1dec02a510 comp rx=0 tx=0).stop 2024-09-16T15:54:01.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.881+0000 7f1dffe00640 1 -- 172.21.15.151:0/3077646806 shutdown_connections 2024-09-16T15:54:01.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.881+0000 7f1dffe00640 1 --2- 172.21.15.151:0/3077646806 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1df8145750 0x7f1df8073440 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:01.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.881+0000 7f1dffe00640 1 -- 172.21.15.151:0/3077646806 >> 172.21.15.151:0/3077646806 conn(0x7f1df80b1840 msgr2=0x7f1df80b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:01.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.881+0000 7f1dffe00640 1 -- 172.21.15.151:0/3077646806 shutdown_connections 2024-09-16T15:54:01.883 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.881+0000 7f1dffe00640 1 -- 172.21.15.151:0/3077646806 wait complete. 2024-09-16T15:54:01.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.882+0000 7f1dffe00640 1 Processor -- start 2024-09-16T15:54:01.884 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.882+0000 7f1dffe00640 1 -- start start 2024-09-16T15:54:01.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.883+0000 7f1dffe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1df806efa0 0x7f1df81411c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:01.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.883+0000 7f1dffe00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1df80cccd0 0x7f1df81f5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:01.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.883+0000 7f1dffe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1df8145750 0x7f1df81f8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:01.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.883+0000 7f1dffe00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1df80ce6d0 con 0x7f1df806efa0 2024-09-16T15:54:01.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.883+0000 7f1dffe00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1df80ce550 con 0x7f1df8145750 2024-09-16T15:54:01.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.883+0000 7f1dffe00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f1df80ce850 con 0x7f1df80cccd0 2024-09-16T15:54:01.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.883+0000 7f1dfd400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1df806efa0 0x7f1df81411c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:01.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.884+0000 7f1dfd400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1df806efa0 0x7f1df81411c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:54930/0 (socket says 172.21.15.151:54930) 2024-09-16T15:54:01.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.884+0000 7f1dfd400640 1 -- 172.21.15.151:0/4163829654 learned_addr learned my addr 172.21.15.151:0/4163829654 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:54:01.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.884+0000 7f1dfca00640 1 --2- 172.21.15.151:0/4163829654 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1df80cccd0 0x7f1df81f5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:01.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.884+0000 7f1dfde00640 1 --2- 172.21.15.151:0/4163829654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1df8145750 0x7f1df81f8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:01.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.884+0000 7f1dfd400640 1 -- 172.21.15.151:0/4163829654 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1df80cccd0 msgr2=0x7f1df81f5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:01.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.884+0000 7f1dfd400640 1 --2- 172.21.15.151:0/4163829654 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1df80cccd0 0x7f1df81f5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:01.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.884+0000 7f1dfd400640 1 -- 172.21.15.151:0/4163829654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1df8145750 msgr2=0x7f1df81f8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:01.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.884+0000 7f1dfd400640 1 --2- 172.21.15.151:0/4163829654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1df8145750 0x7f1df81f8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:01.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.884+0000 7f1dfd400640 1 -- 172.21.15.151:0/4163829654 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1df81e8820 con 0x7f1df806efa0 2024-09-16T15:54:01.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.884+0000 7f1dfde00640 1 --2- 172.21.15.151:0/4163829654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1df8145750 0x7f1df81f8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:54:01.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.884+0000 7f1dfd400640 1 --2- 172.21.15.151:0/4163829654 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1df806efa0 0x7f1df81411c0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f1dec009980 tx=0x7f1dec033ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:01.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.885+0000 7f1df5600640 1 -- 172.21.15.151:0/4163829654 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dec031b50 con 0x7f1df806efa0 2024-09-16T15:54:01.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.885+0000 7f1df5600640 1 -- 172.21.15.151:0/4163829654 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1dec031de0 con 0x7f1df806efa0 2024-09-16T15:54:01.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.885+0000 7f1dffe00640 1 -- 172.21.15.151:0/4163829654 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1df81f8b50 con 0x7f1df806efa0 2024-09-16T15:54:01.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.885+0000 7f1df5600640 1 -- 172.21.15.151:0/4163829654 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dec033330 con 0x7f1df806efa0 2024-09-16T15:54:01.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.885+0000 7f1dffe00640 1 -- 172.21.15.151:0/4163829654 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1df81f4d00 con 0x7f1df806efa0 2024-09-16T15:54:01.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.886+0000 7f1dffe00640 1 -- 172.21.15.151:0/4163829654 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1df81f4b10 con 0x7f1df806efa0 2024-09-16T15:54:01.889 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.887+0000 7f1df5600640 1 -- 172.21.15.151:0/4163829654 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 47) ==== 104092+0+0 (secure 0 0 0) 0x7f1dec0334d0 con 0x7f1df806efa0 2024-09-16T15:54:01.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.888+0000 7f1df5600640 1 --2- 172.21.15.151:0/4163829654 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1dd807a740 0x7f1dd807cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:01.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.888+0000 7f1df5600640 1 -- 172.21.15.151:0/4163829654 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5821+0+0 (secure 0 0 0) 0x7f1dec0baaf0 con 0x7f1df806efa0 2024-09-16T15:54:01.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.889+0000 7f1dfca00640 1 --2- 172.21.15.151:0/4163829654 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1dd807a740 0x7f1dd807cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:01.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.889+0000 7f1dfca00640 1 --2- 172.21.15.151:0/4163829654 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1dd807a740 0x7f1dd807cc00 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f1df8141990 tx=0x7f1de4009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:01.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:01.893+0000 7f1df5600640 1 -- 172.21.15.151:0/4163829654 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1dec03f030 con 0x7f1df806efa0 2024-09-16T15:54:02.081 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.079+0000 7f1dffe00640 1 -- 172.21.15.151:0/4163829654 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f1df80ced00 con 0x7f1df806efa0 2024-09-16T15:54:02.084 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.083+0000 7f1df5600640 1 -- 172.21.15.151:0/4163829654 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v52)=0 v52) ==== 131+0+0 (secure 0 0 0) 0x7f1dec084b40 con 0x7f1df806efa0 2024-09-16T15:54:02.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.085+0000 7f1dffe00640 1 -- 172.21.15.151:0/4163829654 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1dd807a740 msgr2=0x7f1dd807cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:02.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.085+0000 7f1dffe00640 1 --2- 172.21.15.151:0/4163829654 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1dd807a740 0x7f1dd807cc00 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f1df8141990 tx=0x7f1de4009040 comp rx=0 tx=0).stop 2024-09-16T15:54:02.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.085+0000 7f1dffe00640 1 -- 172.21.15.151:0/4163829654 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1df806efa0 msgr2=0x7f1df81411c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:02.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.085+0000 7f1dffe00640 1 --2- 172.21.15.151:0/4163829654 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1df806efa0 0x7f1df81411c0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f1dec009980 tx=0x7f1dec033ba0 comp rx=0 tx=0).stop 2024-09-16T15:54:02.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.085+0000 7f1dffe00640 1 -- 172.21.15.151:0/4163829654 shutdown_connections 2024-09-16T15:54:02.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.085+0000 7f1dffe00640 1 --2- 172.21.15.151:0/4163829654 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1dd807a740 0x7f1dd807cc00 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:02.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.085+0000 7f1dffe00640 1 --2- 172.21.15.151:0/4163829654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1df8145750 0x7f1df81f8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:02.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.085+0000 7f1dffe00640 1 --2- 172.21.15.151:0/4163829654 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1df80cccd0 0x7f1df81f5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:02.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.085+0000 7f1dffe00640 1 --2- 172.21.15.151:0/4163829654 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1df806efa0 0x7f1df81411c0 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:02.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.085+0000 7f1dffe00640 1 -- 172.21.15.151:0/4163829654 >> 172.21.15.151:0/4163829654 conn(0x7f1df80b1840 msgr2=0x7f1df80b3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:02.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.085+0000 7f1dffe00640 1 -- 172.21.15.151:0/4163829654 shutdown_connections 2024-09-16T15:54:02.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.085+0000 7f1dffe00640 1 -- 172.21.15.151:0/4163829654 wait complete. 2024-09-16T15:54:02.104 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-09-16T15:54:02.109 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:54:02.108+0000 7f48f3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:54:02.109 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:54:02.108+0000 7f48f3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:54:02.109 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:54:02.108+0000 7f48f3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:54:02.109 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:54:02.108+0000 7f48f3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:54:02.109 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:54:02.108+0000 7f48f3000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:54:02.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.264+0000 7f6c36800640 1 -- 172.21.15.151:0/2885883937 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c3006f220 msgr2=0x7f6c3006f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:02.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.264+0000 7f6c36800640 1 --2- 172.21.15.151:0/2885883937 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c3006f220 0x7f6c3006f6d0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f6c28005fd0 tx=0x7f6c2802aa50 comp rx=0 tx=0).stop 2024-09-16T15:54:02.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.264+0000 7f6c36800640 1 -- 172.21.15.151:0/2885883937 shutdown_connections 2024-09-16T15:54:02.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.264+0000 7f6c36800640 1 --2- 172.21.15.151:0/2885883937 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c3006f220 0x7f6c3006f6d0 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:02.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.264+0000 7f6c36800640 1 --2- 172.21.15.151:0/2885883937 >> v2:172.21.15.167:3300/0 conn(0x7f6c3015cff0 0x7f6c3006e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:02.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.264+0000 7f6c36800640 1 --2- 172.21.15.151:0/2885883937 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c30159aa0 0x7f6c300741a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:02.267 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.264+0000 7f6c36800640 1 -- 172.21.15.151:0/2885883937 >> 172.21.15.151:0/2885883937 conn(0x7f6c300b18e0 msgr2=0x7f6c300b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:02.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.264+0000 7f6c36800640 1 -- 172.21.15.151:0/2885883937 shutdown_connections 2024-09-16T15:54:02.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.264+0000 7f6c36800640 1 -- 172.21.15.151:0/2885883937 wait complete. 2024-09-16T15:54:02.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.266+0000 7f6c36800640 1 Processor -- start 2024-09-16T15:54:02.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.266+0000 7f6c36800640 1 -- start start 2024-09-16T15:54:02.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.267+0000 7f6c36800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c3006f220 0x7f6c30153da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:02.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.267+0000 7f6c36800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c30159aa0 0x7f6c300add80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:02.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.267+0000 7f6c36800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c3015cff0 0x7f6c300ae2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:02.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.267+0000 7f6c36800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6c300cc310 con 0x7f6c30159aa0 2024-09-16T15:54:02.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.267+0000 7f6c36800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6c300cc190 con 0x7f6c3015cff0 2024-09-16T15:54:02.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.267+0000 7f6c36800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f6c300cc490 con 0x7f6c3006f220 2024-09-16T15:54:02.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.267+0000 7f6c2ea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c30159aa0 0x7f6c300add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:02.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.267+0000 7f6c2ea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c30159aa0 0x7f6c300add80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:54938/0 (socket says 172.21.15.151:54938) 2024-09-16T15:54:02.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.267+0000 7f6c2ea00640 1 -- 172.21.15.151:0/3030187857 learned_addr learned my addr 172.21.15.151:0/3030187857 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:54:02.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.268+0000 7f6c2fe00640 1 --2- 172.21.15.151:0/3030187857 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c3015cff0 0x7f6c300ae2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:02.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.268+0000 7f6c2f400640 1 --2- 172.21.15.151:0/3030187857 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c3006f220 0x7f6c30153da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:02.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.268+0000 7f6c2ea00640 1 -- 172.21.15.151:0/3030187857 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c3006f220 msgr2=0x7f6c30153da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:02.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.268+0000 7f6c2ea00640 1 --2- 172.21.15.151:0/3030187857 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c3006f220 0x7f6c30153da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:02.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.268+0000 7f6c2ea00640 1 -- 172.21.15.151:0/3030187857 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c3015cff0 msgr2=0x7f6c300ae2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:02.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.268+0000 7f6c2ea00640 1 --2- 172.21.15.151:0/3030187857 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c3015cff0 0x7f6c300ae2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:02.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.268+0000 7f6c2ea00640 1 -- 172.21.15.151:0/3030187857 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c301f5a10 con 0x7f6c30159aa0 2024-09-16T15:54:02.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.268+0000 7f6c2fe00640 1 --2- 172.21.15.151:0/3030187857 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c3015cff0 0x7f6c300ae2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:54:02.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.268+0000 7f6c2ea00640 1 --2- 172.21.15.151:0/3030187857 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c30159aa0 0x7f6c300add80 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c00d6d0 tx=0x7f6c1c00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:02.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.268+0000 7f6c27400640 1 -- 172.21.15.151:0/3030187857 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c017070 con 0x7f6c30159aa0 2024-09-16T15:54:02.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.268+0000 7f6c27400640 1 -- 172.21.15.151:0/3030187857 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6c1c004310 con 0x7f6c30159aa0 2024-09-16T15:54:02.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.268+0000 7f6c27400640 1 -- 172.21.15.151:0/3030187857 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c009a80 con 0x7f6c30159aa0 2024-09-16T15:54:02.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.268+0000 7f6c36800640 1 -- 172.21.15.151:0/3030187857 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c301f4e90 con 0x7f6c30159aa0 2024-09-16T15:54:02.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.268+0000 7f6c36800640 1 -- 172.21.15.151:0/3030187857 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c301f5450 con 0x7f6c30159aa0 2024-09-16T15:54:02.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.270+0000 7f6c36800640 1 -- 172.21.15.151:0/3030187857 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c30069360 con 0x7f6c30159aa0 2024-09-16T15:54:02.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.271+0000 7f6c27400640 1 -- 172.21.15.151:0/3030187857 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 47) ==== 104092+0+0 (secure 0 0 0) 0x7f6c1c0035f0 con 0x7f6c30159aa0 2024-09-16T15:54:02.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.272+0000 7f6c27400640 1 --2- 172.21.15.151:0/3030187857 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6c1007a740 0x7f6c1007cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:02.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.272+0000 7f6c27400640 1 -- 172.21.15.151:0/3030187857 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5821+0+0 (secure 0 0 0) 0x7f6c1c0998e0 con 0x7f6c30159aa0 2024-09-16T15:54:02.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.272+0000 7f6c2f400640 1 --2- 172.21.15.151:0/3030187857 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6c1007a740 0x7f6c1007cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:02.275 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.273+0000 7f6c2f400640 1 --2- 172.21.15.151:0/3030187857 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6c1007a740 0x7f6c1007cc00 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f6c20005d80 tx=0x7f6c20005c90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:02.278 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.276+0000 7f6c27400640 1 -- 172.21.15.151:0/3030187857 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f6c1c0639b0 con 0x7f6c30159aa0 2024-09-16T15:54:02.487 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.485+0000 7f6c36800640 1 -- 172.21.15.151:0/3030187857 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7f6c30070980 con 0x7f6c1007a740 2024-09-16T15:54:02.498 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:54:02.497+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:54:02.498 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:54:02.497+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:54:02.498 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:54:02.497+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:54:02.498 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:54:02.497+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:54:02.498 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:54:02.497+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:54:02.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.533+0000 7f6c27400640 1 -- 172.21.15.151:0/3030187857 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6c30070980 con 0x7f6c1007a740 2024-09-16T15:54:02.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.535+0000 7f6c36800640 1 -- 172.21.15.151:0/3030187857 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6c1007a740 msgr2=0x7f6c1007cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:02.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.535+0000 7f6c36800640 1 --2- 172.21.15.151:0/3030187857 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6c1007a740 0x7f6c1007cc00 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f6c20005d80 tx=0x7f6c20005c90 comp rx=0 tx=0).stop 2024-09-16T15:54:02.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.535+0000 7f6c36800640 1 -- 172.21.15.151:0/3030187857 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c30159aa0 msgr2=0x7f6c300add80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:02.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.535+0000 7f6c36800640 1 --2- 172.21.15.151:0/3030187857 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c30159aa0 0x7f6c300add80 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c00d6d0 tx=0x7f6c1c00db90 comp rx=0 tx=0).stop 2024-09-16T15:54:02.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.535+0000 7f6c36800640 1 -- 172.21.15.151:0/3030187857 shutdown_connections 2024-09-16T15:54:02.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.535+0000 7f6c36800640 1 --2- 172.21.15.151:0/3030187857 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f6c1007a740 0x7f6c1007cc00 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:02.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.535+0000 7f6c36800640 1 --2- 172.21.15.151:0/3030187857 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6c3015cff0 0x7f6c300ae2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:02.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.535+0000 7f6c36800640 1 --2- 172.21.15.151:0/3030187857 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6c30159aa0 0x7f6c300add80 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:02.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.535+0000 7f6c36800640 1 --2- 172.21.15.151:0/3030187857 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6c3006f220 0x7f6c30153da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:02.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.535+0000 7f6c36800640 1 -- 172.21.15.151:0/3030187857 >> 172.21.15.151:0/3030187857 conn(0x7f6c300b18e0 msgr2=0x7f6c3015f880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:02.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.536+0000 7f6c36800640 1 -- 172.21.15.151:0/3030187857 shutdown_connections 2024-09-16T15:54:02.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.536+0000 7f6c36800640 1 -- 172.21.15.151:0/3030187857 wait complete. 2024-09-16T15:54:02.557 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-16T15:54:02.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.717+0000 7fc16ca00640 1 -- 172.21.15.151:0/4255644359 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc14c0052e0 msgr2=0x7fc14c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:02.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.717+0000 7fc16ca00640 1 --2- 172.21.15.151:0/4255644359 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc14c0052e0 0x7fc14c00b9c0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fc160001690 tx=0x7fc14c005830 comp rx=0 tx=0).stop 2024-09-16T15:54:02.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.717+0000 7fc16ca00640 1 -- 172.21.15.151:0/4255644359 shutdown_connections 2024-09-16T15:54:02.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.717+0000 7fc16ca00640 1 --2- 172.21.15.151:0/4255644359 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc14c0052e0 0x7fc14c00b9c0 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:02.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.717+0000 7fc16ca00640 1 -- 172.21.15.151:0/4255644359 >> 172.21.15.151:0/4255644359 conn(0x7fc1680d3d50 msgr2=0x7fc1680d4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:02.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.717+0000 7fc16ca00640 1 -- 172.21.15.151:0/4255644359 shutdown_connections 2024-09-16T15:54:02.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.717+0000 7fc16ca00640 1 -- 172.21.15.151:0/4255644359 wait complete. 2024-09-16T15:54:02.720 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.718+0000 7fc16ca00640 1 Processor -- start 2024-09-16T15:54:02.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.719+0000 7fc16ca00640 1 -- start start 2024-09-16T15:54:02.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.719+0000 7fc16ca00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc14c0052e0 0x7fc1680b8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:02.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.720+0000 7fc16ca00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1680c7880 0x7fc1681f8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:02.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.720+0000 7fc16ca00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1681f3b20 0x7fc1680c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:02.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.720+0000 7fc16ca00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc1680b3a10 con 0x7fc1680c7880 2024-09-16T15:54:02.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.720+0000 7fc16ca00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc1680b3890 con 0x7fc1681f3b20 2024-09-16T15:54:02.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.720+0000 7fc16ca00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fc1680b3b90 con 0x7fc14c0052e0 2024-09-16T15:54:02.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.720+0000 7fc165e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1680c7880 0x7fc1681f8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:02.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.720+0000 7fc165e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1680c7880 0x7fc1681f8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:54954/0 (socket says 172.21.15.151:54954) 2024-09-16T15:54:02.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.720+0000 7fc165e00640 1 -- 172.21.15.151:0/3737552259 learned_addr learned my addr 172.21.15.151:0/3737552259 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:54:02.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.720+0000 7fc167200640 1 --2- 172.21.15.151:0/3737552259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1681f3b20 0x7fc1680c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:02.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.720+0000 7fc166800640 1 --2- 172.21.15.151:0/3737552259 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc14c0052e0 0x7fc1680b8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:02.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.720+0000 7fc165e00640 1 -- 172.21.15.151:0/3737552259 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc14c0052e0 msgr2=0x7fc1680b8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:02.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.720+0000 7fc165e00640 1 --2- 172.21.15.151:0/3737552259 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc14c0052e0 0x7fc1680b8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:02.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.720+0000 7fc165e00640 1 -- 172.21.15.151:0/3737552259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1681f3b20 msgr2=0x7fc1680c9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:02.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.720+0000 7fc165e00640 1 --2- 172.21.15.151:0/3737552259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1681f3b20 0x7fc1680c9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:02.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.720+0000 7fc165e00640 1 -- 172.21.15.151:0/3737552259 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1680b8860 con 0x7fc1680c7880 2024-09-16T15:54:02.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.720+0000 7fc167200640 1 --2- 172.21.15.151:0/3737552259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1681f3b20 0x7fc1680c9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:54:02.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.720+0000 7fc165e00640 1 --2- 172.21.15.151:0/3737552259 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1680c7880 0x7fc1681f8350 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fc150011ab0 tx=0x7fc150011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:02.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.721+0000 7fc15ea00640 1 -- 172.21.15.151:0/3737552259 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc15001e070 con 0x7fc1680c7880 2024-09-16T15:54:02.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.721+0000 7fc15ea00640 1 -- 172.21.15.151:0/3737552259 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc15000fe70 con 0x7fc1680c7880 2024-09-16T15:54:02.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.721+0000 7fc15ea00640 1 -- 172.21.15.151:0/3737552259 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1500042b0 con 0x7fc1680c7880 2024-09-16T15:54:02.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.721+0000 7fc16ca00640 1 -- 172.21.15.151:0/3737552259 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1680b8b40 con 0x7fc1680c7880 2024-09-16T15:54:02.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.721+0000 7fc16ca00640 1 -- 172.21.15.151:0/3737552259 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1680b9100 con 0x7fc1680c7880 2024-09-16T15:54:02.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.722+0000 7fc16ca00640 1 -- 172.21.15.151:0/3737552259 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1680cfb10 con 0x7fc1680c7880 2024-09-16T15:54:02.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.723+0000 7fc15ea00640 1 -- 172.21.15.151:0/3737552259 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 47) ==== 104092+0+0 (secure 0 0 0) 0x7fc150021070 con 0x7fc1680c7880 2024-09-16T15:54:02.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.724+0000 7fc15ea00640 1 --2- 172.21.15.151:0/3737552259 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc14407a740 0x7fc14407cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:02.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.724+0000 7fc15ea00640 1 -- 172.21.15.151:0/3737552259 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5821+0+0 (secure 0 0 0) 0x7fc15009a380 con 0x7fc1680c7880 2024-09-16T15:54:02.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.725+0000 7fc166800640 1 --2- 172.21.15.151:0/3737552259 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc14407a740 0x7fc14407cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:02.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.725+0000 7fc166800640 1 --2- 172.21.15.151:0/3737552259 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc14407a740 0x7fc14407cc00 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fc158001be0 tx=0x7fc158001f30 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:02.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.729+0000 7fc15ea00640 1 -- 172.21.15.151:0/3737552259 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc150063e00 con 0x7fc1680c7880 2024-09-16T15:54:02.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.941+0000 7fc16ca00640 1 -- 172.21.15.151:0/3737552259 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fc1680b5410 con 0x7fc14407a740 2024-09-16T15:54:02.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.948+0000 7fc15ea00640 1 -- 172.21.15.151:0/3737552259 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7fc1680b5410 con 0x7fc14407a740 2024-09-16T15:54:02.949 INFO:teuthology.orchestra.run.smithi151.stdout:/volumes/_nogroup/sv1/4ecd5764-29e8-466f-92ae-20e498fe9453 2024-09-16T15:54:02.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.949+0000 7fc16ca00640 1 -- 172.21.15.151:0/3737552259 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc14407a740 msgr2=0x7fc14407cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:02.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.949+0000 7fc16ca00640 1 --2- 172.21.15.151:0/3737552259 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc14407a740 0x7fc14407cc00 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fc158001be0 tx=0x7fc158001f30 comp rx=0 tx=0).stop 2024-09-16T15:54:02.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.949+0000 7fc16ca00640 1 -- 172.21.15.151:0/3737552259 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1680c7880 msgr2=0x7fc1681f8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:02.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.949+0000 7fc16ca00640 1 --2- 172.21.15.151:0/3737552259 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1680c7880 0x7fc1681f8350 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fc150011ab0 tx=0x7fc150011f70 comp rx=0 tx=0).stop 2024-09-16T15:54:02.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.950+0000 7fc16ca00640 1 -- 172.21.15.151:0/3737552259 shutdown_connections 2024-09-16T15:54:02.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.950+0000 7fc16ca00640 1 --2- 172.21.15.151:0/3737552259 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc14407a740 0x7fc14407cc00 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:02.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.950+0000 7fc16ca00640 1 --2- 172.21.15.151:0/3737552259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc1681f3b20 0x7fc1680c9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:02.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.950+0000 7fc16ca00640 1 --2- 172.21.15.151:0/3737552259 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1680c7880 0x7fc1681f8350 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:02.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.950+0000 7fc16ca00640 1 --2- 172.21.15.151:0/3737552259 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc14c0052e0 0x7fc1680b8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:02.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.950+0000 7fc16ca00640 1 -- 172.21.15.151:0/3737552259 >> 172.21.15.151:0/3737552259 conn(0x7fc1680d3d50 msgr2=0x7fc1680d2310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:02.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.950+0000 7fc16ca00640 1 -- 172.21.15.151:0/3737552259 shutdown_connections 2024-09-16T15:54:02.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:02.950+0000 7fc16ca00640 1 -- 172.21.15.151:0/3737552259 wait complete. 2024-09-16T15:54:02.969 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-16T15:54:03.133 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:03.131+0000 7efe2f200640 1 -- 172.21.15.151:0/969324275 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe28145750 msgr2=0x7efe28073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:03.134 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:03.131+0000 7efe2f200640 1 --2- 172.21.15.151:0/969324275 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe28145750 0x7efe28073440 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7efe280af750 tx=0x7efe1802a4d0 comp rx=0 tx=0).stop 2024-09-16T15:54:03.134 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:03.131+0000 7efe2f200640 1 -- 172.21.15.151:0/969324275 shutdown_connections 2024-09-16T15:54:03.134 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:03.131+0000 7efe2f200640 1 --2- 172.21.15.151:0/969324275 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efe280ccb00 0x7efe2815f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:03.134 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:03.131+0000 7efe2f200640 1 --2- 172.21.15.151:0/969324275 >> v2:172.21.15.167:3300/0 conn(0x7efe2806efa0 0x7efe280cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:03.134 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:03.131+0000 7efe2f200640 1 --2- 172.21.15.151:0/969324275 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe28145750 0x7efe28073440 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:03.134 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:03.131+0000 7efe2f200640 1 -- 172.21.15.151:0/969324275 >> 172.21.15.151:0/969324275 conn(0x7efe280b1840 msgr2=0x7efe280b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:03.134 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:03.131+0000 7efe2f200640 1 -- 172.21.15.151:0/969324275 shutdown_connections 2024-09-16T15:54:03.134 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:03.131+0000 7efe2f200640 1 -- 172.21.15.151:0/969324275 wait complete. 2024-09-16T15:54:03.135 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:03.133+0000 7efe2f200640 1 Processor -- start 2024-09-16T15:54:03.135 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:03.133+0000 7efe2f200640 1 -- start start 2024-09-16T15:54:03.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:03.134+0000 7efe2f200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe2806efa0 0x7efe280c4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:03.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:03.134+0000 7efe2f200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe280ccb00 0x7efe28157700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:03.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:03.134+0000 7efe2f200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efe28145750 0x7efe28142990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:03.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:03.134+0000 7efe2f200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efe281602c0 con 0x7efe280ccb00 2024-09-16T15:54:03.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:03.134+0000 7efe2f200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efe28160140 con 0x7efe2806efa0 2024-09-16T15:54:03.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:03.134+0000 7efe2f200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7efe28160440 con 0x7efe28145750 2024-09-16T15:54:03.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:03.134+0000 7efe27400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe280ccb00 0x7efe28157700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:03.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:03.134+0000 7efe27400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe280ccb00 0x7efe28157700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:56060/0 (socket says 172.21.15.151:56060) 2024-09-16T15:54:03.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:03.134+0000 7efe27400640 1 -- 172.21.15.151:0/3460951975 learned_addr learned my addr 172.21.15.151:0/3460951975 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:54:03.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:03.134+0000 7efe27e00640 1 --2- 172.21.15.151:0/3460951975 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe2806efa0 0x7efe280c4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:03.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:03.134+0000 7efe2d200640 1 --2- 172.21.15.151:0/3460951975 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efe28145750 0x7efe28142990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:03.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:03.134+0000 7efe27400640 1 -- 172.21.15.151:0/3460951975 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efe28145750 msgr2=0x7efe28142990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:03.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:03.134+0000 7efe27400640 1 --2- 172.21.15.151:0/3460951975 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efe28145750 0x7efe28142990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:03.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:03.134+0000 7efe27400640 1 -- 172.21.15.151:0/3460951975 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe2806efa0 msgr2=0x7efe280c4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:03.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:03.134+0000 7efe27400640 1 --2- 172.21.15.151:0/3460951975 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe2806efa0 0x7efe280c4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:03.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:03.134+0000 7efe27400640 1 -- 172.21.15.151:0/3460951975 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe28142ed0 con 0x7efe280ccb00 2024-09-16T15:54:03.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:03.135+0000 7efe27400640 1 --2- 172.21.15.151:0/3460951975 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe280ccb00 0x7efe28157700 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7efe1400d770 tx=0x7efe1400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:03.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:03.135+0000 7efe1fe00640 1 -- 172.21.15.151:0/3460951975 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe14017070 con 0x7efe280ccb00 2024-09-16T15:54:03.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:03.135+0000 7efe1fe00640 1 -- 172.21.15.151:0/3460951975 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efe14009d20 con 0x7efe280ccb00 2024-09-16T15:54:03.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:03.135+0000 7efe2f200640 1 -- 172.21.15.151:0/3460951975 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe28143e00 con 0x7efe280ccb00 2024-09-16T15:54:03.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:03.135+0000 7efe1fe00640 1 -- 172.21.15.151:0/3460951975 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe140106c0 con 0x7efe280ccb00 2024-09-16T15:54:03.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:03.135+0000 7efe2f200640 1 -- 172.21.15.151:0/3460951975 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe28142190 con 0x7efe280ccb00 2024-09-16T15:54:03.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:03.136+0000 7efe2f200640 1 -- 172.21.15.151:0/3460951975 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe2815f9f0 con 0x7efe280ccb00 2024-09-16T15:54:03.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:03.137+0000 7efe1fe00640 1 -- 172.21.15.151:0/3460951975 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 47) ==== 104092+0+0 (secure 0 0 0) 0x7efe1401a070 con 0x7efe280ccb00 2024-09-16T15:54:03.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:03.138+0000 7efe1fe00640 1 --2- 172.21.15.151:0/3460951975 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efe0807a740 0x7efe0807cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:03.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:03.138+0000 7efe1fe00640 1 -- 172.21.15.151:0/3460951975 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5821+0+0 (secure 0 0 0) 0x7efe14099b10 con 0x7efe280ccb00 2024-09-16T15:54:03.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:03.139+0000 7efe27e00640 1 --2- 172.21.15.151:0/3460951975 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efe0807a740 0x7efe0807cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:03.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:03.140+0000 7efe27e00640 1 --2- 172.21.15.151:0/3460951975 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efe0807a740 0x7efe0807cc00 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7efe18033000 tx=0x7efe18004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:03.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:03.142+0000 7efe1fe00640 1 -- 172.21.15.151:0/3460951975 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7efe14063510 con 0x7efe280ccb00 2024-09-16T15:54:03.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:03.360+0000 7efe2f200640 1 -- 172.21.15.151:0/3460951975 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7efe28157c40 con 0x7efe0807a740 2024-09-16T15:54:03.368 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:03.366+0000 7efe1fe00640 1 -- 172.21.15.151:0/3460951975 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7efe18004210 con 0x7efe0807a740 2024-09-16T15:54:03.368 INFO:teuthology.orchestra.run.smithi151.stdout:/volumes/_nogroup/sv1/4ecd5764-29e8-466f-92ae-20e498fe9453 2024-09-16T15:54:03.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:03.368+0000 7efe2f200640 1 -- 172.21.15.151:0/3460951975 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efe0807a740 msgr2=0x7efe0807cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:03.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:03.368+0000 7efe2f200640 1 --2- 172.21.15.151:0/3460951975 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efe0807a740 0x7efe0807cc00 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7efe18033000 tx=0x7efe18004730 comp rx=0 tx=0).stop 2024-09-16T15:54:03.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:03.368+0000 7efe2f200640 1 -- 172.21.15.151:0/3460951975 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe280ccb00 msgr2=0x7efe28157700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:03.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:03.368+0000 7efe2f200640 1 --2- 172.21.15.151:0/3460951975 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe280ccb00 0x7efe28157700 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7efe1400d770 tx=0x7efe1400dc30 comp rx=0 tx=0).stop 2024-09-16T15:54:03.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:03.368+0000 7efe2f200640 1 -- 172.21.15.151:0/3460951975 shutdown_connections 2024-09-16T15:54:03.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:03.368+0000 7efe2f200640 1 --2- 172.21.15.151:0/3460951975 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efe0807a740 0x7efe0807cc00 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:03.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:03.368+0000 7efe2f200640 1 --2- 172.21.15.151:0/3460951975 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efe28145750 0x7efe28142990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:03.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:03.368+0000 7efe2f200640 1 --2- 172.21.15.151:0/3460951975 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe280ccb00 0x7efe28157700 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:03.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:03.368+0000 7efe2f200640 1 --2- 172.21.15.151:0/3460951975 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe2806efa0 0x7efe280c4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:03.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:03.369+0000 7efe2f200640 1 -- 172.21.15.151:0/3460951975 >> 172.21.15.151:0/3460951975 conn(0x7efe280b1840 msgr2=0x7efe280b2c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:03.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:03.369+0000 7efe2f200640 1 -- 172.21.15.151:0/3460951975 shutdown_connections 2024-09-16T15:54:03.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:03.369+0000 7efe2f200640 1 -- 172.21.15.151:0/3460951975 wait complete. 2024-09-16T15:54:03.387 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/4ecd5764-29e8-466f-92ae-20e498fe9453 2024-09-16T15:54:03.387 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:54:03.387 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:54:03.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:54:03.423 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/4ecd5764-29e8-466f-92ae-20e498fe9453/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T15:54:12.012 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records in 2024-09-16T15:54:12.012 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records out 2024-09-16T15:54:12.012 INFO:teuthology.orchestra.run.smithi151.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.50824 s, 126 MB/s 2024-09-16T15:54:12.013 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:54:12.013 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:54:12.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:54:12.049 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/4ecd5764-29e8-466f-92ae-20e498fe9453/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T15:54:20.417 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records in 2024-09-16T15:54:20.417 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records out 2024-09-16T15:54:20.417 INFO:teuthology.orchestra.run.smithi151.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.29167 s, 129 MB/s 2024-09-16T15:54:20.418 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:54:20.418 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:54:20.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:54:20.454 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/4ecd5764-29e8-466f-92ae-20e498fe9453/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T15:54:21.585 DEBUG:teuthology.orchestra.run.smithi151:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:54:21.589 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:54:28.921 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records in 2024-09-16T15:54:28.921 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records out 2024-09-16T15:54:28.922 INFO:teuthology.orchestra.run.smithi151.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.3902 s, 128 MB/s 2024-09-16T15:54:28.923 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-09-16T15:54:29.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.086+0000 7f103f200640 1 -- 172.21.15.151:0/944650699 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1024004d90 msgr2=0x7f1024005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:29.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.086+0000 7f103f200640 1 --2- 172.21.15.151:0/944650699 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1024004d90 0x7f1024005210 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f103800c560 tx=0x7f1024005830 comp rx=0 tx=0).stop 2024-09-16T15:54:29.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.086+0000 7f103e800640 1 -- 172.21.15.151:0/944650699 reap_dead start 2024-09-16T15:54:29.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.086+0000 7f103f200640 1 -- 172.21.15.151:0/944650699 shutdown_connections 2024-09-16T15:54:29.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.086+0000 7f103f200640 1 -- 172.21.15.151:0/944650699 >> 172.21.15.151:0/944650699 conn(0x7f10400cfa70 msgr2=0x7f10400cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:29.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.086+0000 7f103f200640 1 -- 172.21.15.151:0/944650699 shutdown_connections 2024-09-16T15:54:29.087 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.086+0000 7f103f200640 1 -- 172.21.15.151:0/944650699 wait complete. 2024-09-16T15:54:29.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.087+0000 7f103f200640 1 Processor -- start 2024-09-16T15:54:29.088 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.088+0000 7f103f200640 1 -- start start 2024-09-16T15:54:29.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.088+0000 7f103f200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1024004d90 0x7f1040151b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:29.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.088+0000 7f103f200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f10400ce880 0x7f10400ced00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:29.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.088+0000 7f103f200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10400ad880 0x7f10400add00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:29.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.089+0000 7f103f200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f10400701d0 con 0x7f10400ad880 2024-09-16T15:54:29.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.089+0000 7f103f200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1040070050 con 0x7f1024004d90 2024-09-16T15:54:29.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.089+0000 7f103f200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f10400cb8d0 con 0x7f10400ce880 2024-09-16T15:54:29.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.089+0000 7f103e800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10400ad880 0x7f10400add00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:29.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.089+0000 7f103d400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f10400ce880 0x7f10400ced00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:29.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.089+0000 7f103de00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1024004d90 0x7f1040151b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:29.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.089+0000 7f103e800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10400ad880 0x7f10400add00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:57330/0 (socket says 172.21.15.151:57330) 2024-09-16T15:54:29.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.089+0000 7f103e800640 1 -- 172.21.15.151:0/2491446394 learned_addr learned my addr 172.21.15.151:0/2491446394 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:54:29.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.089+0000 7f103d400640 1 -- 172.21.15.151:0/2491446394 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1024004d90 msgr2=0x7f1040151b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:29.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.089+0000 7f103d400640 1 --2- 172.21.15.151:0/2491446394 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1024004d90 0x7f1040151b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:29.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.089+0000 7f103d400640 1 -- 172.21.15.151:0/2491446394 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10400ad880 msgr2=0x7f10400add00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:29.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.089+0000 7f103d400640 1 --2- 172.21.15.151:0/2491446394 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10400ad880 0x7f10400add00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:29.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.089+0000 7f103d400640 1 -- 172.21.15.151:0/2491446394 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10401f5090 con 0x7f10400ce880 2024-09-16T15:54:29.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.090+0000 7f103d400640 1 --2- 172.21.15.151:0/2491446394 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f10400ce880 0x7f10400ced00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f102c011830 tx=0x7f102c011cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:29.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.090+0000 7f1036000640 1 -- 172.21.15.151:0/2491446394 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f102c004460 con 0x7f10400ce880 2024-09-16T15:54:29.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.090+0000 7f1036000640 1 -- 172.21.15.151:0/2491446394 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f102c00f9a0 con 0x7f10400ce880 2024-09-16T15:54:29.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.090+0000 7f1036000640 1 -- 172.21.15.151:0/2491446394 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f102c00fdf0 con 0x7f10400ce880 2024-09-16T15:54:29.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.090+0000 7f103f200640 1 -- 172.21.15.151:0/2491446394 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10401f5350 con 0x7f10400ce880 2024-09-16T15:54:29.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.090+0000 7f103f200640 1 -- 172.21.15.151:0/2491446394 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f10401f5910 con 0x7f10400ce880 2024-09-16T15:54:29.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.091+0000 7f103f200640 1 -- 172.21.15.151:0/2491446394 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10400cbe30 con 0x7f10400ce880 2024-09-16T15:54:29.093 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.092+0000 7f1036000640 1 -- 172.21.15.151:0/2491446394 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 48) ==== 104194+0+0 (secure 0 0 0) 0x7f102c005020 con 0x7f10400ce880 2024-09-16T15:54:29.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.093+0000 7f1036000640 1 --2- 172.21.15.151:0/2491446394 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f102007a890 0x7f102007cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:29.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.093+0000 7f1036000640 1 -- 172.21.15.151:0/2491446394 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(133..133 src has 1..133) ==== 5821+0+0 (secure 0 0 0) 0x7f102c09a480 con 0x7f10400ce880 2024-09-16T15:54:29.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.093+0000 7f103de00640 1 --2- 172.21.15.151:0/2491446394 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f102007a890 0x7f102007cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:29.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.094+0000 7f103de00640 1 --2- 172.21.15.151:0/2491446394 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f102007a890 0x7f102007cd50 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f1030006fd0 tx=0x7f1030006b10 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:29.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.098+0000 7f1036000640 1 -- 172.21.15.151:0/2491446394 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f102c063f00 con 0x7f10400ce880 2024-09-16T15:54:29.313 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.312+0000 7f103f200640 1 -- 172.21.15.151:0/2491446394 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7f10400b3630 con 0x7f102007a890 2024-09-16T15:54:29.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.325+0000 7f1036000640 1 -- 172.21.15.151:0/2491446394 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f10400b3630 con 0x7f102007a890 2024-09-16T15:54:29.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.328+0000 7f103f200640 1 -- 172.21.15.151:0/2491446394 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f102007a890 msgr2=0x7f102007cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:29.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.328+0000 7f103f200640 1 --2- 172.21.15.151:0/2491446394 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f102007a890 0x7f102007cd50 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f1030006fd0 tx=0x7f1030006b10 comp rx=0 tx=0).stop 2024-09-16T15:54:29.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.328+0000 7f103f200640 1 -- 172.21.15.151:0/2491446394 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f10400ce880 msgr2=0x7f10400ced00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:29.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.328+0000 7f103f200640 1 --2- 172.21.15.151:0/2491446394 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f10400ce880 0x7f10400ced00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f102c011830 tx=0x7f102c011cf0 comp rx=0 tx=0).stop 2024-09-16T15:54:29.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.328+0000 7f103f200640 1 -- 172.21.15.151:0/2491446394 shutdown_connections 2024-09-16T15:54:29.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.328+0000 7f103f200640 1 --2- 172.21.15.151:0/2491446394 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f102007a890 0x7f102007cd50 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:29.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.328+0000 7f103f200640 1 --2- 172.21.15.151:0/2491446394 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10400ad880 0x7f10400add00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:29.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.328+0000 7f103f200640 1 --2- 172.21.15.151:0/2491446394 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f10400ce880 0x7f10400ced00 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:29.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.328+0000 7f103f200640 1 --2- 172.21.15.151:0/2491446394 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1024004d90 0x7f1040151b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:29.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.328+0000 7f103f200640 1 -- 172.21.15.151:0/2491446394 >> 172.21.15.151:0/2491446394 conn(0x7f10400cfa70 msgr2=0x7f10400cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:29.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.328+0000 7f103f200640 1 -- 172.21.15.151:0/2491446394 shutdown_connections 2024-09-16T15:54:29.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.328+0000 7f103f200640 1 -- 172.21.15.151:0/2491446394 wait complete. 2024-09-16T15:54:29.345 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-16T15:54:29.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.506+0000 7f1570000640 1 -- 172.21.15.151:0/1896554478 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1568145750 msgr2=0x7f1568073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:29.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.506+0000 7f1570000640 1 --2- 172.21.15.151:0/1896554478 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1568145750 0x7f1568073440 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f155c009860 tx=0x7f155c02a510 comp rx=0 tx=0).stop 2024-09-16T15:54:29.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.506+0000 7f1570000640 1 -- 172.21.15.151:0/1896554478 shutdown_connections 2024-09-16T15:54:29.507 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.506+0000 7f1570000640 1 --2- 172.21.15.151:0/1896554478 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1568145750 0x7f1568073440 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:29.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.506+0000 7f1570000640 1 -- 172.21.15.151:0/1896554478 >> 172.21.15.151:0/1896554478 conn(0x7f15680b1840 msgr2=0x7f15680b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:29.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.506+0000 7f1570000640 1 -- 172.21.15.151:0/1896554478 shutdown_connections 2024-09-16T15:54:29.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.506+0000 7f1570000640 1 -- 172.21.15.151:0/1896554478 wait complete. 2024-09-16T15:54:29.508 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.507+0000 7f1570000640 1 Processor -- start 2024-09-16T15:54:29.509 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.508+0000 7f1570000640 1 -- start start 2024-09-16T15:54:29.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.508+0000 7f1570000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f156806efa0 0x7f156815b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:29.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.509+0000 7f1570000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15680cccd0 0x7f15681f31d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:29.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.509+0000 7f1570000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1568145750 0x7f156815ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:29.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.509+0000 7f1570000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f15680ce6d0 con 0x7f15680cccd0 2024-09-16T15:54:29.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.509+0000 7f1570000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f15680ce550 con 0x7f1568145750 2024-09-16T15:54:29.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.509+0000 7f1570000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f15680ce850 con 0x7f156806efa0 2024-09-16T15:54:29.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.509+0000 7f156cc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15680cccd0 0x7f15681f31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:29.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.509+0000 7f156cc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15680cccd0 0x7f15681f31d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:57344/0 (socket says 172.21.15.151:57344) 2024-09-16T15:54:29.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.509+0000 7f156cc00640 1 -- 172.21.15.151:0/1833027267 learned_addr learned my addr 172.21.15.151:0/1833027267 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:54:29.511 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.509+0000 7f156d600640 1 --2- 172.21.15.151:0/1833027267 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f156806efa0 0x7f156815b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:29.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.509+0000 7f156e000640 1 --2- 172.21.15.151:0/1833027267 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1568145750 0x7f156815ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:29.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.509+0000 7f156cc00640 1 -- 172.21.15.151:0/1833027267 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f156806efa0 msgr2=0x7f156815b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:29.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.509+0000 7f156cc00640 1 --2- 172.21.15.151:0/1833027267 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f156806efa0 0x7f156815b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:29.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.509+0000 7f156cc00640 1 -- 172.21.15.151:0/1833027267 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1568145750 msgr2=0x7f156815ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:29.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.509+0000 7f156cc00640 1 --2- 172.21.15.151:0/1833027267 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1568145750 0x7f156815ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:29.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.509+0000 7f156cc00640 1 -- 172.21.15.151:0/1833027267 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15681f3710 con 0x7f15680cccd0 2024-09-16T15:54:29.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.509+0000 7f156e000640 1 --2- 172.21.15.151:0/1833027267 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1568145750 0x7f156815ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:54:29.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.510+0000 7f156cc00640 1 --2- 172.21.15.151:0/1833027267 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15680cccd0 0x7f15681f31d0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f155400c4c0 tx=0x7f155400c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:29.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.510+0000 7f1565600640 1 -- 172.21.15.151:0/1833027267 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15540052d0 con 0x7f15680cccd0 2024-09-16T15:54:29.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.510+0000 7f1565600640 1 -- 172.21.15.151:0/1833027267 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1554005490 con 0x7f15680cccd0 2024-09-16T15:54:29.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.510+0000 7f1565600640 1 -- 172.21.15.151:0/1833027267 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1554012890 con 0x7f15680cccd0 2024-09-16T15:54:29.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.510+0000 7f1570000640 1 -- 172.21.15.151:0/1833027267 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15680aece0 con 0x7f15680cccd0 2024-09-16T15:54:29.512 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.510+0000 7f1570000640 1 -- 172.21.15.151:0/1833027267 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15680aad50 con 0x7f15680cccd0 2024-09-16T15:54:29.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.511+0000 7f1570000640 1 -- 172.21.15.151:0/1833027267 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15680ced00 con 0x7f15680cccd0 2024-09-16T15:54:29.513 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.512+0000 7f1565600640 1 -- 172.21.15.151:0/1833027267 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 48) ==== 104194+0+0 (secure 0 0 0) 0x7f1554012a30 con 0x7f15680cccd0 2024-09-16T15:54:29.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.513+0000 7f1565600640 1 --2- 172.21.15.151:0/1833027267 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f154807a860 0x7f154807cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:29.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.513+0000 7f1565600640 1 -- 172.21.15.151:0/1833027267 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5821+0+0 (secure 0 0 0) 0x7f1554099570 con 0x7f15680cccd0 2024-09-16T15:54:29.514 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.514+0000 7f156d600640 1 --2- 172.21.15.151:0/1833027267 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f154807a860 0x7f154807cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:29.515 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.514+0000 7f156d600640 1 --2- 172.21.15.151:0/1833027267 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f154807a860 0x7f154807cd20 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f155c034000 tx=0x7f155c004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:29.518 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.517+0000 7f1565600640 1 -- 172.21.15.151:0/1833027267 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f15681f3710 con 0x7f15680cccd0 2024-09-16T15:54:29.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.730+0000 7f1570000640 1 -- 172.21.15.151:0/1833027267 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f15680d04c0 con 0x7f154807a860 2024-09-16T15:54:29.736 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.735+0000 7f1565600640 1 -- 172.21.15.151:0/1833027267 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f15680d04c0 con 0x7f154807a860 2024-09-16T15:54:29.736 INFO:teuthology.orchestra.run.smithi151.stdout:/volumes/_nogroup/sv1/4ecd5764-29e8-466f-92ae-20e498fe9453 2024-09-16T15:54:29.738 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.737+0000 7f1570000640 1 -- 172.21.15.151:0/1833027267 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f154807a860 msgr2=0x7f154807cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:29.739 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.737+0000 7f1570000640 1 --2- 172.21.15.151:0/1833027267 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f154807a860 0x7f154807cd20 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f155c034000 tx=0x7f155c004730 comp rx=0 tx=0).stop 2024-09-16T15:54:29.739 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.737+0000 7f1570000640 1 -- 172.21.15.151:0/1833027267 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15680cccd0 msgr2=0x7f15681f31d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:29.739 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.737+0000 7f1570000640 1 --2- 172.21.15.151:0/1833027267 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15680cccd0 0x7f15681f31d0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f155400c4c0 tx=0x7f155400c980 comp rx=0 tx=0).stop 2024-09-16T15:54:29.739 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.737+0000 7f1570000640 1 -- 172.21.15.151:0/1833027267 shutdown_connections 2024-09-16T15:54:29.739 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.737+0000 7f1570000640 1 --2- 172.21.15.151:0/1833027267 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f154807a860 0x7f154807cd20 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:29.739 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.737+0000 7f1570000640 1 --2- 172.21.15.151:0/1833027267 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1568145750 0x7f156815ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:29.739 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.737+0000 7f1570000640 1 --2- 172.21.15.151:0/1833027267 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15680cccd0 0x7f15681f31d0 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:29.739 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.737+0000 7f1570000640 1 --2- 172.21.15.151:0/1833027267 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f156806efa0 0x7f156815b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:29.739 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.737+0000 7f1570000640 1 -- 172.21.15.151:0/1833027267 >> 172.21.15.151:0/1833027267 conn(0x7f15680b1840 msgr2=0x7f15680af630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:29.739 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.737+0000 7f1570000640 1 -- 172.21.15.151:0/1833027267 shutdown_connections 2024-09-16T15:54:29.739 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:29.737+0000 7f1570000640 1 -- 172.21.15.151:0/1833027267 wait complete. 2024-09-16T15:54:29.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:54:29.755 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/4ecd5764-29e8-466f-92ae-20e498fe9453 2024-09-16T15:54:29.755 DEBUG:teuthology.orchestra.run.smithi151:> ) 2024-09-16T15:54:29.790 INFO:teuthology.orchestra.run.smithi151.stdout:# file: volumes/_nogroup/sv1/4ecd5764-29e8-466f-92ae-20e498fe9453 2024-09-16T15:54:29.790 INFO:teuthology.orchestra.run.smithi151.stdout:ceph.dir.rbytes="2972172288" 2024-09-16T15:54:29.790 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:54:29.791 INFO:tasks.cephfs.test_volumes:r_size = 2972172288 exp_size = 3221225472 2024-09-16T15:54:31.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:54:31.792 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/4ecd5764-29e8-466f-92ae-20e498fe9453 2024-09-16T15:54:31.793 DEBUG:teuthology.orchestra.run.smithi151:> ) 2024-09-16T15:54:31.833 INFO:teuthology.orchestra.run.smithi151.stdout:# file: volumes/_nogroup/sv1/4ecd5764-29e8-466f-92ae-20e498fe9453 2024-09-16T15:54:31.833 INFO:teuthology.orchestra.run.smithi151.stdout:ceph.dir.rbytes="3221225472" 2024-09-16T15:54:31.833 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:54:31.834 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 exp_size = 3221225472 2024-09-16T15:54:31.834 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 ss1clone1 2024-09-16T15:54:32.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:32.046+0000 7f4c28c00640 1 -- 172.21.15.151:0/347306124 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c240cda90 msgr2=0x7f4c240ce320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:32.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:32.046+0000 7f4c28c00640 1 --2- 172.21.15.151:0/347306124 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c240cda90 0x7f4c240ce320 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4c10009830 tx=0x7f4c1002a410 comp rx=0 tx=0).stop 2024-09-16T15:54:32.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:32.046+0000 7f4c28c00640 1 -- 172.21.15.151:0/347306124 shutdown_connections 2024-09-16T15:54:32.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:32.046+0000 7f4c28c00640 1 --2- 172.21.15.151:0/347306124 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c240cda90 0x7f4c240ce320 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:32.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:32.046+0000 7f4c28c00640 1 --2- 172.21.15.151:0/347306124 >> v2:172.21.15.167:3300/0 conn(0x7f4c2415aa30 0x7f4c240cd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:32.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:32.046+0000 7f4c28c00640 1 -- 172.21.15.151:0/347306124 >> 172.21.15.151:0/347306124 conn(0x7f4c240cc490 msgr2=0x7f4c240cc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:32.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:32.046+0000 7f4c28c00640 1 -- 172.21.15.151:0/347306124 shutdown_connections 2024-09-16T15:54:32.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:32.046+0000 7f4c28c00640 1 -- 172.21.15.151:0/347306124 wait complete. 2024-09-16T15:54:32.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:32.048+0000 7f4c28c00640 1 Processor -- start 2024-09-16T15:54:32.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:32.048+0000 7f4c28c00640 1 -- start start 2024-09-16T15:54:32.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:32.049+0000 7f4c28c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4c240cda90 0x7f4c240c92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:32.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:32.049+0000 7f4c28c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c240ce860 0x7f4c24073680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:32.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:32.049+0000 7f4c28c00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c2415aa30 0x7f4c24073bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:32.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:32.049+0000 7f4c28c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4c2415ed40 con 0x7f4c240cda90 2024-09-16T15:54:32.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:32.049+0000 7f4c28c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4c2415ebc0 con 0x7f4c240ce860 2024-09-16T15:54:32.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:32.049+0000 7f4c28c00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f4c2415eec0 con 0x7f4c2415aa30 2024-09-16T15:54:32.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:32.049+0000 7f4c23400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4c240cda90 0x7f4c240c92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:32.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:32.049+0000 7f4c23400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4c240cda90 0x7f4c240c92b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:57358/0 (socket says 172.21.15.151:57358) 2024-09-16T15:54:32.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:32.049+0000 7f4c23400640 1 -- 172.21.15.151:0/2559148175 learned_addr learned my addr 172.21.15.151:0/2559148175 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:54:32.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:32.050+0000 7f4c23400640 1 -- 172.21.15.151:0/2559148175 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c2415aa30 msgr2=0x7f4c24073bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:32.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:32.050+0000 7f4c23400640 1 --2- 172.21.15.151:0/2559148175 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c2415aa30 0x7f4c24073bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:32.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:32.050+0000 7f4c23400640 1 -- 172.21.15.151:0/2559148175 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c240ce860 msgr2=0x7f4c24073680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:32.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:32.050+0000 7f4c23400640 1 --2- 172.21.15.151:0/2559148175 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c240ce860 0x7f4c24073680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:32.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:32.050+0000 7f4c23400640 1 -- 172.21.15.151:0/2559148175 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c240c6b80 con 0x7f4c240cda90 2024-09-16T15:54:32.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:32.050+0000 7f4c23400640 1 --2- 172.21.15.151:0/2559148175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4c240cda90 0x7f4c240c92b0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f4c1400eb30 tx=0x7f4c1400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:32.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:32.050+0000 7f4c1b400640 1 -- 172.21.15.151:0/2559148175 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c14010430 con 0x7f4c240cda90 2024-09-16T15:54:32.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:32.050+0000 7f4c1b400640 1 -- 172.21.15.151:0/2559148175 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4c140105d0 con 0x7f4c240cda90 2024-09-16T15:54:32.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:32.050+0000 7f4c1b400640 1 -- 172.21.15.151:0/2559148175 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c14010a20 con 0x7f4c240cda90 2024-09-16T15:54:32.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:32.050+0000 7f4c28c00640 1 -- 172.21.15.151:0/2559148175 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c240c6e20 con 0x7f4c240cda90 2024-09-16T15:54:32.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:32.050+0000 7f4c28c00640 1 -- 172.21.15.151:0/2559148175 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c240c73e0 con 0x7f4c240cda90 2024-09-16T15:54:32.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:32.051+0000 7f4c28c00640 1 -- 172.21.15.151:0/2559148175 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c240adda0 con 0x7f4c240cda90 2024-09-16T15:54:32.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:32.053+0000 7f4c1b400640 1 -- 172.21.15.151:0/2559148175 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 48) ==== 104194+0+0 (secure 0 0 0) 0x7f4c14010bc0 con 0x7f4c240cda90 2024-09-16T15:54:32.054 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:32.053+0000 7f4c1b400640 1 --2- 172.21.15.151:0/2559148175 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4c0407a860 0x7f4c0407cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:32.054 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:32.054+0000 7f4c1b400640 1 -- 172.21.15.151:0/2559148175 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5821+0+0 (secure 0 0 0) 0x7f4c1409a840 con 0x7f4c240cda90 2024-09-16T15:54:32.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:32.054+0000 7f4c22a00640 1 --2- 172.21.15.151:0/2559148175 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4c0407a860 0x7f4c0407cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:32.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:32.055+0000 7f4c22a00640 1 --2- 172.21.15.151:0/2559148175 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4c0407a860 0x7f4c0407cd20 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f4c10034000 tx=0x7f4c100046c0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:32.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:32.058+0000 7f4c1b400640 1 -- 172.21.15.151:0/2559148175 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4c14064910 con 0x7f4c240cda90 2024-09-16T15:54:32.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:32.273+0000 7f4c28c00640 1 -- 172.21.15.151:0/2559148175 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f4c240adac0 con 0x7f4c0407a860 2024-09-16T15:54:32.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:32.336+0000 7f4c1b400640 1 -- 172.21.15.151:0/2559148175 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4c240adac0 con 0x7f4c0407a860 2024-09-16T15:54:32.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:32.339+0000 7f4c28c00640 1 -- 172.21.15.151:0/2559148175 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4c0407a860 msgr2=0x7f4c0407cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:32.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:32.339+0000 7f4c28c00640 1 --2- 172.21.15.151:0/2559148175 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4c0407a860 0x7f4c0407cd20 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f4c10034000 tx=0x7f4c100046c0 comp rx=0 tx=0).stop 2024-09-16T15:54:32.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:32.339+0000 7f4c28c00640 1 -- 172.21.15.151:0/2559148175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4c240cda90 msgr2=0x7f4c240c92b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:32.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:32.339+0000 7f4c28c00640 1 --2- 172.21.15.151:0/2559148175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4c240cda90 0x7f4c240c92b0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f4c1400eb30 tx=0x7f4c1400c2d0 comp rx=0 tx=0).stop 2024-09-16T15:54:32.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:32.339+0000 7f4c28c00640 1 -- 172.21.15.151:0/2559148175 shutdown_connections 2024-09-16T15:54:32.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:32.339+0000 7f4c28c00640 1 --2- 172.21.15.151:0/2559148175 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4c0407a860 0x7f4c0407cd20 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:32.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:32.339+0000 7f4c28c00640 1 --2- 172.21.15.151:0/2559148175 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c240ce860 0x7f4c24073680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:32.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:32.339+0000 7f4c28c00640 1 --2- 172.21.15.151:0/2559148175 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4c2415aa30 0x7f4c24073bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:32.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:32.339+0000 7f4c28c00640 1 --2- 172.21.15.151:0/2559148175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4c240cda90 0x7f4c240c92b0 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:32.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:32.339+0000 7f4c28c00640 1 -- 172.21.15.151:0/2559148175 >> 172.21.15.151:0/2559148175 conn(0x7f4c240cc490 msgr2=0x7f4c24157280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:32.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:32.339+0000 7f4c28c00640 1 -- 172.21.15.151:0/2559148175 shutdown_connections 2024-09-16T15:54:32.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:32.339+0000 7f4c28c00640 1 -- 172.21.15.151:0/2559148175 wait complete. 2024-09-16T15:54:32.342 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:54:32.341+0000 7f48f6c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:54:32.342 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:54:32.341+0000 7f48f6c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:54:32.342 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:54:32.341+0000 7f48f6c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:54:32.342 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:54:32.341+0000 7f48f6c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:54:32.342 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:54:32.341+0000 7f48f6c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:54:32.361 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:54:32.360+0000 7f48f6200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:54:32.361 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:54:32.360+0000 7f48f6200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:54:32.361 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:54:32.360+0000 7f48f6200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:54:32.361 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:54:32.360+0000 7f48f6200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:54:32.361 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:54:32.360+0000 7f48f6200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:54:33.348 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:54:33.347+0000 7f485b200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:54:33.348 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:54:33.347+0000 7f485b200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:54:33.348 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:54:33.347+0000 7f485b200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:54:33.348 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:54:33.347+0000 7f485b200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:54:33.348 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:54:33.347+0000 7f485b200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:54:33.359 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs ss1clone1 2024-09-16T15:54:33.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:33.592+0000 7f126d200640 1 -- 172.21.15.151:0/1667119416 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f126815bb90 msgr2=0x7f12680b2810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:33.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:33.592+0000 7f126d200640 1 --2- 172.21.15.151:0/1667119416 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f126815bb90 0x7f12680b2810 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f126806ff40 tx=0x7f125802aa10 comp rx=0 tx=0).stop 2024-09-16T15:54:33.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:33.593+0000 7f126d200640 1 -- 172.21.15.151:0/1667119416 shutdown_connections 2024-09-16T15:54:33.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:33.593+0000 7f126d200640 1 --2- 172.21.15.151:0/1667119416 >> v2:172.21.15.167:3300/0 conn(0x7f12680cf3e0 0x7f1268074280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:33.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:33.593+0000 7f126d200640 1 --2- 172.21.15.151:0/1667119416 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f12680cefc0 0x7f1268073d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:33.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:33.593+0000 7f126d200640 1 --2- 172.21.15.151:0/1667119416 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f126815bb90 0x7f12680b2810 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:33.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:33.593+0000 7f126d200640 1 -- 172.21.15.151:0/1667119416 >> 172.21.15.151:0/1667119416 conn(0x7f12680b1510 msgr2=0x7f12680b18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:33.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:33.593+0000 7f126d200640 1 -- 172.21.15.151:0/1667119416 shutdown_connections 2024-09-16T15:54:33.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:33.593+0000 7f126d200640 1 -- 172.21.15.151:0/1667119416 wait complete. 2024-09-16T15:54:33.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:33.595+0000 7f126d200640 1 Processor -- start 2024-09-16T15:54:33.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:33.595+0000 7f126d200640 1 -- start start 2024-09-16T15:54:33.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:33.597+0000 7f126d200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12680cefc0 0x7f12680caa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:33.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:33.597+0000 7f126d200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f12680cf3e0 0x7f12680cb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:33.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:33.597+0000 7f126d200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f126815bb90 0x7f12680afd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:33.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:33.597+0000 7f126d200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f126806eee0 con 0x7f12680cefc0 2024-09-16T15:54:33.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:33.597+0000 7f126d200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f126806ed60 con 0x7f126815bb90 2024-09-16T15:54:33.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:33.597+0000 7f126d200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f126806f060 con 0x7f12680cf3e0 2024-09-16T15:54:33.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:33.597+0000 7f1267400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12680cefc0 0x7f12680caa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:33.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:33.597+0000 7f1267400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12680cefc0 0x7f12680caa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41462/0 (socket says 172.21.15.151:41462) 2024-09-16T15:54:33.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:33.597+0000 7f1267400640 1 -- 172.21.15.151:0/3573042854 learned_addr learned my addr 172.21.15.151:0/3573042854 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:54:33.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:33.597+0000 7f1266a00640 1 --2- 172.21.15.151:0/3573042854 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f12680cf3e0 0x7f12680cb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:33.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:33.597+0000 7f1267400640 1 -- 172.21.15.151:0/3573042854 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f12680cf3e0 msgr2=0x7f12680cb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:33.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:33.597+0000 7f1267e00640 1 --2- 172.21.15.151:0/3573042854 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f126815bb90 0x7f12680afd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:33.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:33.598+0000 7f1267400640 1 --2- 172.21.15.151:0/3573042854 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f12680cf3e0 0x7f12680cb3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:33.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:33.598+0000 7f1267400640 1 -- 172.21.15.151:0/3573042854 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f126815bb90 msgr2=0x7f12680afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:33.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:33.598+0000 7f1267400640 1 --2- 172.21.15.151:0/3573042854 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f126815bb90 0x7f12680afd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:33.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:33.598+0000 7f1267400640 1 -- 172.21.15.151:0/3573042854 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12680c9fb0 con 0x7f12680cefc0 2024-09-16T15:54:33.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:33.598+0000 7f1266a00640 1 --2- 172.21.15.151:0/3573042854 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f12680cf3e0 0x7f12680cb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:54:33.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:33.598+0000 7f1267e00640 1 --2- 172.21.15.151:0/3573042854 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f126815bb90 0x7f12680afd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:54:33.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:33.598+0000 7f1267400640 1 --2- 172.21.15.151:0/3573042854 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12680cefc0 0x7f12680caa70 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f125802af30 tx=0x7f1258003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:33.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:33.598+0000 7f125f400640 1 -- 172.21.15.151:0/3573042854 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1258033050 con 0x7f12680cefc0 2024-09-16T15:54:33.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:33.598+0000 7f125f400640 1 -- 172.21.15.151:0/3573042854 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f12580043d0 con 0x7f12680cefc0 2024-09-16T15:54:33.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:33.598+0000 7f126d200640 1 -- 172.21.15.151:0/3573042854 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12680cbbe0 con 0x7f12680cefc0 2024-09-16T15:54:33.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:33.598+0000 7f125f400640 1 -- 172.21.15.151:0/3573042854 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1258004820 con 0x7f12680cefc0 2024-09-16T15:54:33.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:33.599+0000 7f126d200640 1 -- 172.21.15.151:0/3573042854 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12680ca270 con 0x7f12680cefc0 2024-09-16T15:54:33.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:33.600+0000 7f126d200640 1 -- 172.21.15.151:0/3573042854 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12680b2810 con 0x7f12680cefc0 2024-09-16T15:54:33.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:33.601+0000 7f125f400640 1 -- 172.21.15.151:0/3573042854 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 48) ==== 104194+0+0 (secure 0 0 0) 0x7f1258036070 con 0x7f12680cefc0 2024-09-16T15:54:33.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:33.603+0000 7f125f400640 1 --2- 172.21.15.151:0/3573042854 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f124807a860 0x7f124807cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:33.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:33.603+0000 7f125f400640 1 -- 172.21.15.151:0/3573042854 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5821+0+0 (secure 0 0 0) 0x7f12580bab70 con 0x7f12680cefc0 2024-09-16T15:54:33.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:33.604+0000 7f1266a00640 1 --2- 172.21.15.151:0/3573042854 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f124807a860 0x7f124807cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:33.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:33.606+0000 7f1266a00640 1 --2- 172.21.15.151:0/3573042854 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f124807a860 0x7f124807cd20 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f1254006fd0 tx=0x7f1254008040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:33.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:33.607+0000 7f125f400640 1 -- 172.21.15.151:0/3573042854 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1258084ce0 con 0x7f12680cefc0 2024-09-16T15:54:33.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:33.926+0000 7f126d200640 1 -- 172.21.15.151:0/3573042854 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7f12680ca8c0 con 0x7f124807a860 2024-09-16T15:54:34.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:34.059+0000 7f125f400640 1 -- 172.21.15.151:0/3573042854 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f12680ca8c0 con 0x7f124807a860 2024-09-16T15:54:34.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:34.061+0000 7f126d200640 1 -- 172.21.15.151:0/3573042854 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f124807a860 msgr2=0x7f124807cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:34.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:34.061+0000 7f126d200640 1 --2- 172.21.15.151:0/3573042854 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f124807a860 0x7f124807cd20 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f1254006fd0 tx=0x7f1254008040 comp rx=0 tx=0).stop 2024-09-16T15:54:34.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:34.061+0000 7f126d200640 1 -- 172.21.15.151:0/3573042854 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12680cefc0 msgr2=0x7f12680caa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:34.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:34.061+0000 7f126d200640 1 --2- 172.21.15.151:0/3573042854 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12680cefc0 0x7f12680caa70 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f125802af30 tx=0x7f1258003f80 comp rx=0 tx=0).stop 2024-09-16T15:54:34.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:34.061+0000 7f126d200640 1 -- 172.21.15.151:0/3573042854 shutdown_connections 2024-09-16T15:54:34.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:34.061+0000 7f126d200640 1 --2- 172.21.15.151:0/3573042854 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f124807a860 0x7f124807cd20 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:34.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:34.061+0000 7f126d200640 1 --2- 172.21.15.151:0/3573042854 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f126815bb90 0x7f12680afd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:34.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:34.061+0000 7f126d200640 1 --2- 172.21.15.151:0/3573042854 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f12680cf3e0 0x7f12680cb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:34.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:34.061+0000 7f126d200640 1 --2- 172.21.15.151:0/3573042854 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f12680cefc0 0x7f12680caa70 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:34.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:34.061+0000 7f126d200640 1 -- 172.21.15.151:0/3573042854 >> 172.21.15.151:0/3573042854 conn(0x7f12680b1510 msgr2=0x7f12680b4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:34.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:34.061+0000 7f126d200640 1 -- 172.21.15.151:0/3573042854 shutdown_connections 2024-09-16T15:54:34.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:34.061+0000 7f126d200640 1 -- 172.21.15.151:0/3573042854 wait complete. 2024-09-16T15:54:34.080 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-09-16T15:54:34.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:34.239+0000 7fa966000640 1 -- 172.21.15.151:0/2665521272 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9600ccb00 msgr2=0x7fa96015f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:34.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:34.239+0000 7fa966000640 1 --2- 172.21.15.151:0/2665521272 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9600ccb00 0x7fa96015f9f0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fa960074ff0 tx=0x7fa95802a4d0 comp rx=0 tx=0).stop 2024-09-16T15:54:34.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:34.239+0000 7fa966000640 1 -- 172.21.15.151:0/2665521272 shutdown_connections 2024-09-16T15:54:34.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:34.239+0000 7fa966000640 1 --2- 172.21.15.151:0/2665521272 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9600ccb00 0x7fa96015f9f0 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:34.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:34.239+0000 7fa966000640 1 --2- 172.21.15.151:0/2665521272 >> v2:172.21.15.167:3300/0 conn(0x7fa96006efa0 0x7fa9600cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:34.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:34.239+0000 7fa966000640 1 --2- 172.21.15.151:0/2665521272 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa960145750 0x7fa960073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:34.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:34.239+0000 7fa966000640 1 -- 172.21.15.151:0/2665521272 >> 172.21.15.151:0/2665521272 conn(0x7fa9600b1840 msgr2=0x7fa9600b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:34.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:34.239+0000 7fa966000640 1 -- 172.21.15.151:0/2665521272 shutdown_connections 2024-09-16T15:54:34.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:34.239+0000 7fa966000640 1 -- 172.21.15.151:0/2665521272 wait complete. 2024-09-16T15:54:34.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:34.241+0000 7fa966000640 1 Processor -- start 2024-09-16T15:54:34.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:34.241+0000 7fa966000640 1 -- start start 2024-09-16T15:54:34.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:34.242+0000 7fa966000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa96006efa0 0x7fa960139a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:34.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:34.242+0000 7fa966000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9600ccb00 0x7fa9600b3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:34.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:34.242+0000 7fa966000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa960145750 0x7fa9601f3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:34.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:34.242+0000 7fa966000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa9601600b0 con 0x7fa96006efa0 2024-09-16T15:54:34.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:34.242+0000 7fa966000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa96015ff30 con 0x7fa960145750 2024-09-16T15:54:34.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:34.242+0000 7fa966000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fa960160230 con 0x7fa9600ccb00 2024-09-16T15:54:34.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:34.242+0000 7fa95f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa96006efa0 0x7fa960139a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:34.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:34.242+0000 7fa95f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa96006efa0 0x7fa960139a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41476/0 (socket says 172.21.15.151:41476) 2024-09-16T15:54:34.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:34.242+0000 7fa95f400640 1 -- 172.21.15.151:0/2653465718 learned_addr learned my addr 172.21.15.151:0/2653465718 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:54:34.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:34.242+0000 7fa95fe00640 1 --2- 172.21.15.151:0/2653465718 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa960145750 0x7fa9601f3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:34.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:34.242+0000 7fa95ea00640 1 --2- 172.21.15.151:0/2653465718 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9600ccb00 0x7fa9600b3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:34.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:34.242+0000 7fa95f400640 1 -- 172.21.15.151:0/2653465718 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9600ccb00 msgr2=0x7fa9600b3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:34.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:34.242+0000 7fa95f400640 1 --2- 172.21.15.151:0/2653465718 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9600ccb00 0x7fa9600b3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:34.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:34.242+0000 7fa95f400640 1 -- 172.21.15.151:0/2653465718 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa960145750 msgr2=0x7fa9601f3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:34.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:34.242+0000 7fa95f400640 1 --2- 172.21.15.151:0/2653465718 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa960145750 0x7fa9601f3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:34.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:34.242+0000 7fa95f400640 1 -- 172.21.15.151:0/2653465718 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9600b3d50 con 0x7fa96006efa0 2024-09-16T15:54:34.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:34.242+0000 7fa95fe00640 1 --2- 172.21.15.151:0/2653465718 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa960145750 0x7fa9601f3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:54:34.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:34.242+0000 7fa95ea00640 1 --2- 172.21.15.151:0/2653465718 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9600ccb00 0x7fa9600b3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:54:34.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:34.243+0000 7fa95f400640 1 --2- 172.21.15.151:0/2653465718 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa96006efa0 0x7fa960139a60 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fa95000e9f0 tx=0x7fa95000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:34.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:34.243+0000 7fa957400640 1 -- 172.21.15.151:0/2653465718 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa95000cd80 con 0x7fa96006efa0 2024-09-16T15:54:34.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:34.243+0000 7fa957400640 1 -- 172.21.15.151:0/2653465718 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa950004c10 con 0x7fa96006efa0 2024-09-16T15:54:34.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:34.243+0000 7fa966000640 1 -- 172.21.15.151:0/2653465718 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9600b4030 con 0x7fa96006efa0 2024-09-16T15:54:34.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:34.243+0000 7fa957400640 1 -- 172.21.15.151:0/2653465718 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa950005020 con 0x7fa96006efa0 2024-09-16T15:54:34.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:34.243+0000 7fa966000640 1 -- 172.21.15.151:0/2653465718 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9600b45f0 con 0x7fa96006efa0 2024-09-16T15:54:34.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:34.244+0000 7fa966000640 1 -- 172.21.15.151:0/2653465718 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa960160710 con 0x7fa96006efa0 2024-09-16T15:54:34.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:34.245+0000 7fa957400640 1 -- 172.21.15.151:0/2653465718 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 48) ==== 104194+0+0 (secure 0 0 0) 0x7fa9500051c0 con 0x7fa96006efa0 2024-09-16T15:54:34.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:34.247+0000 7fa957400640 1 --2- 172.21.15.151:0/2653465718 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa94007a890 0x7fa94007cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:34.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:34.247+0000 7fa957400640 1 -- 172.21.15.151:0/2653465718 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5821+0+0 (secure 0 0 0) 0x7fa95009a2f0 con 0x7fa96006efa0 2024-09-16T15:54:34.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:34.247+0000 7fa95ea00640 1 --2- 172.21.15.151:0/2653465718 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa94007a890 0x7fa94007cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:34.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:34.248+0000 7fa95ea00640 1 --2- 172.21.15.151:0/2653465718 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa94007a890 0x7fa94007cd50 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fa94c005fd0 tx=0x7fa94c0094d0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:34.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:34.251+0000 7fa957400640 1 -- 172.21.15.151:0/2653465718 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa950016030 con 0x7fa96006efa0 2024-09-16T15:54:34.477 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:34.476+0000 7fa966000640 1 -- 172.21.15.151:0/2653465718 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7fa9600b0270 con 0x7fa94007a890 2024-09-16T15:54:34.650 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:34.649+0000 7fa957400640 1 -- 172.21.15.151:0/2653465718 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mgrmap(e 49) ==== 104347+0+0 (secure 0 0 0) 0x7fa95005ffd0 con 0x7fa96006efa0 2024-09-16T15:54:36.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:36.537+0000 7fa957400640 1 -- 172.21.15.151:0/2653465718 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7fa9600b0270 con 0x7fa94007a890 2024-09-16T15:54:36.539 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-16T15:54:36.539 INFO:teuthology.orchestra.run.smithi151.stdout: "status": { 2024-09-16T15:54:36.539 INFO:teuthology.orchestra.run.smithi151.stdout: "state": "canceled", 2024-09-16T15:54:36.539 INFO:teuthology.orchestra.run.smithi151.stdout: "source": { 2024-09-16T15:54:36.539 INFO:teuthology.orchestra.run.smithi151.stdout: "volume": "cephfs", 2024-09-16T15:54:36.539 INFO:teuthology.orchestra.run.smithi151.stdout: "subvolume": "sv1", 2024-09-16T15:54:36.539 INFO:teuthology.orchestra.run.smithi151.stdout: "snapshot": "ss1" 2024-09-16T15:54:36.539 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:54:36.539 INFO:teuthology.orchestra.run.smithi151.stdout: "failure": { 2024-09-16T15:54:36.540 INFO:teuthology.orchestra.run.smithi151.stdout: "errno": "4", 2024-09-16T15:54:36.540 INFO:teuthology.orchestra.run.smithi151.stdout: "error_msg": "user interrupted clone operation" 2024-09-16T15:54:36.540 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:54:36.540 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:54:36.540 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-16T15:54:36.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:36.540+0000 7fa966000640 1 -- 172.21.15.151:0/2653465718 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa94007a890 msgr2=0x7fa94007cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:36.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:36.540+0000 7fa966000640 1 --2- 172.21.15.151:0/2653465718 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa94007a890 0x7fa94007cd50 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fa94c005fd0 tx=0x7fa94c0094d0 comp rx=0 tx=0).stop 2024-09-16T15:54:36.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:36.541+0000 7fa966000640 1 -- 172.21.15.151:0/2653465718 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa96006efa0 msgr2=0x7fa960139a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:36.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:36.541+0000 7fa966000640 1 --2- 172.21.15.151:0/2653465718 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa96006efa0 0x7fa960139a60 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fa95000e9f0 tx=0x7fa95000eeb0 comp rx=0 tx=0).stop 2024-09-16T15:54:36.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:36.541+0000 7fa966000640 1 -- 172.21.15.151:0/2653465718 shutdown_connections 2024-09-16T15:54:36.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:36.541+0000 7fa966000640 1 --2- 172.21.15.151:0/2653465718 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa94007a890 0x7fa94007cd50 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:36.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:36.541+0000 7fa966000640 1 --2- 172.21.15.151:0/2653465718 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa960145750 0x7fa9601f3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:36.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:36.541+0000 7fa966000640 1 --2- 172.21.15.151:0/2653465718 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa9600ccb00 0x7fa9600b3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:36.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:36.541+0000 7fa966000640 1 --2- 172.21.15.151:0/2653465718 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa96006efa0 0x7fa960139a60 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:36.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:36.541+0000 7fa966000640 1 -- 172.21.15.151:0/2653465718 >> 172.21.15.151:0/2653465718 conn(0x7fa9600b1840 msgr2=0x7fa9600b0d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:36.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:36.541+0000 7fa966000640 1 -- 172.21.15.151:0/2653465718 shutdown_connections 2024-09-16T15:54:36.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:36.541+0000 7fa966000640 1 -- 172.21.15.151:0/2653465718 wait complete. 2024-09-16T15:54:36.560 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-16T15:54:36.561 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-16T15:54:36.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:36.721+0000 7f6116800640 1 -- 172.21.15.151:0/3548371243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61101563a0 msgr2=0x7f611015cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:36.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:36.721+0000 7f6116800640 1 --2- 172.21.15.151:0/3548371243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61101563a0 0x7f611015cad0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f6104005fd0 tx=0x7f610402a5d0 comp rx=0 tx=0).stop 2024-09-16T15:54:36.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:36.721+0000 7f6116800640 1 -- 172.21.15.151:0/3548371243 shutdown_connections 2024-09-16T15:54:36.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:36.721+0000 7f6116800640 1 --2- 172.21.15.151:0/3548371243 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f61100ccd70 0x7f6110073280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:36.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:36.721+0000 7f6116800640 1 --2- 172.21.15.151:0/3548371243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61101563a0 0x7f611015cad0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:36.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:36.721+0000 7f6116800640 1 -- 172.21.15.151:0/3548371243 >> 172.21.15.151:0/3548371243 conn(0x7f61100b18e0 msgr2=0x7f61100b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:36.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:36.721+0000 7f6116800640 1 -- 172.21.15.151:0/3548371243 shutdown_connections 2024-09-16T15:54:36.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:36.721+0000 7f6116800640 1 -- 172.21.15.151:0/3548371243 wait complete. 2024-09-16T15:54:36.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:36.723+0000 7f6116800640 1 Processor -- start 2024-09-16T15:54:36.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:36.723+0000 7f6116800640 1 -- start start 2024-09-16T15:54:36.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:36.724+0000 7f6116800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f611006e9b0 0x7f61101f8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:36.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:36.724+0000 7f6116800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f61100ccd70 0x7f6110154ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:36.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:36.724+0000 7f6116800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f61101563a0 0x7f61100cec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:36.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:36.724+0000 7f6116800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6110073c00 con 0x7f61101563a0 2024-09-16T15:54:36.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:36.724+0000 7f6116800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6110073a80 con 0x7f611006e9b0 2024-09-16T15:54:36.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:36.724+0000 7f6116800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f6110073d80 con 0x7f61100ccd70 2024-09-16T15:54:36.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:36.724+0000 7f6115e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f61101563a0 0x7f61100cec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:36.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:36.724+0000 7f6115e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f61101563a0 0x7f61100cec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41482/0 (socket says 172.21.15.151:41482) 2024-09-16T15:54:36.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:36.724+0000 7f6115e00640 1 -- 172.21.15.151:0/2673446176 learned_addr learned my addr 172.21.15.151:0/2673446176 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:54:36.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:36.724+0000 7f6115400640 1 --2- 172.21.15.151:0/2673446176 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f611006e9b0 0x7f61101f8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:36.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:36.724+0000 7f6114a00640 1 --2- 172.21.15.151:0/2673446176 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f61100ccd70 0x7f6110154ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:36.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:36.724+0000 7f6115e00640 1 -- 172.21.15.151:0/2673446176 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f61100ccd70 msgr2=0x7f6110154ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:36.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:36.724+0000 7f6115e00640 1 --2- 172.21.15.151:0/2673446176 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f61100ccd70 0x7f6110154ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:36.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:36.724+0000 7f6115e00640 1 -- 172.21.15.151:0/2673446176 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f611006e9b0 msgr2=0x7f61101f8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:36.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:36.724+0000 7f6115e00640 1 --2- 172.21.15.151:0/2673446176 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f611006e9b0 0x7f61101f8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:36.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:36.724+0000 7f6115e00640 1 -- 172.21.15.151:0/2673446176 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61101f8e60 con 0x7f61101563a0 2024-09-16T15:54:36.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:36.725+0000 7f6115e00640 1 --2- 172.21.15.151:0/2673446176 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f61101563a0 0x7f61100cec20 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f610800ec50 tx=0x7f610800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:36.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:36.725+0000 7f610d600640 1 -- 172.21.15.151:0/2673446176 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6108010630 con 0x7f61101563a0 2024-09-16T15:54:36.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:36.725+0000 7f610d600640 1 -- 172.21.15.151:0/2673446176 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f61080107d0 con 0x7f61101563a0 2024-09-16T15:54:36.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:36.725+0000 7f610d600640 1 -- 172.21.15.151:0/2673446176 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6108010c20 con 0x7f61101563a0 2024-09-16T15:54:36.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:36.725+0000 7f6116800640 1 -- 172.21.15.151:0/2673446176 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6110155010 con 0x7f61101563a0 2024-09-16T15:54:36.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:36.725+0000 7f6116800640 1 -- 172.21.15.151:0/2673446176 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f611015afb0 con 0x7f61101563a0 2024-09-16T15:54:36.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:36.726+0000 7f6116800640 1 -- 172.21.15.151:0/2673446176 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61100742a0 con 0x7f61101563a0 2024-09-16T15:54:36.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:36.727+0000 7f610d600640 1 -- 172.21.15.151:0/2673446176 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 49) ==== 104347+0+0 (secure 0 0 0) 0x7f6108021430 con 0x7f61101563a0 2024-09-16T15:54:36.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:36.728+0000 7f610d600640 1 --2- 172.21.15.151:0/2673446176 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f60f007aa10 0x7f60f007ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:36.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:36.728+0000 7f610d600640 1 -- 172.21.15.151:0/2673446176 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5821+0+0 (secure 0 0 0) 0x7f610809a950 con 0x7f61101563a0 2024-09-16T15:54:36.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:36.729+0000 7f6115400640 1 --2- 172.21.15.151:0/2673446176 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f60f007aa10 0x7f60f007ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:36.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:36.730+0000 7f6115400640 1 --2- 172.21.15.151:0/2673446176 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f60f007aa10 0x7f60f007ced0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f6104033000 tx=0x7f6104004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:36.733 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:36.732+0000 7f610d600640 1 -- 172.21.15.151:0/2673446176 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f61101f8e60 con 0x7f61101563a0 2024-09-16T15:54:36.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:36.988+0000 7f6116800640 1 -- 172.21.15.151:0/2673446176 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f6110070ae0 con 0x7f61101563a0 2024-09-16T15:54:36.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:36.988+0000 7f610d600640 1 -- 172.21.15.151:0/2673446176 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+6657 (secure 0 0 0) 0x7f61080643d0 con 0x7f61101563a0 2024-09-16T15:54:36.990 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:54:36.990 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-16T15:54:36.990 INFO:teuthology.orchestra.run.smithi151.stdout: "fsid": "1757be18-e41a-4446-bb4f-ee4ac81c39df", 2024-09-16T15:54:36.990 INFO:teuthology.orchestra.run.smithi151.stdout: "health": { 2024-09-16T15:54:36.990 INFO:teuthology.orchestra.run.smithi151.stdout: "status": "HEALTH_OK", 2024-09-16T15:54:36.990 INFO:teuthology.orchestra.run.smithi151.stdout: "checks": {}, 2024-09-16T15:54:36.990 INFO:teuthology.orchestra.run.smithi151.stdout: "mutes": [] 2024-09-16T15:54:36.990 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:54:36.990 INFO:teuthology.orchestra.run.smithi151.stdout: "election_epoch": 4, 2024-09-16T15:54:36.990 INFO:teuthology.orchestra.run.smithi151.stdout: "quorum": [ 2024-09-16T15:54:36.990 INFO:teuthology.orchestra.run.smithi151.stdout: 0, 2024-09-16T15:54:36.990 INFO:teuthology.orchestra.run.smithi151.stdout: 1, 2024-09-16T15:54:36.991 INFO:teuthology.orchestra.run.smithi151.stdout: 2 2024-09-16T15:54:36.991 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:54:36.991 INFO:teuthology.orchestra.run.smithi151.stdout: "quorum_names": [ 2024-09-16T15:54:36.991 INFO:teuthology.orchestra.run.smithi151.stdout: "a", 2024-09-16T15:54:36.991 INFO:teuthology.orchestra.run.smithi151.stdout: "b", 2024-09-16T15:54:36.991 INFO:teuthology.orchestra.run.smithi151.stdout: "c" 2024-09-16T15:54:36.991 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:54:36.991 INFO:teuthology.orchestra.run.smithi151.stdout: "quorum_age": 940, 2024-09-16T15:54:36.991 INFO:teuthology.orchestra.run.smithi151.stdout: "monmap": { 2024-09-16T15:54:36.991 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 1, 2024-09-16T15:54:36.991 INFO:teuthology.orchestra.run.smithi151.stdout: "min_mon_release_name": "squid", 2024-09-16T15:54:36.991 INFO:teuthology.orchestra.run.smithi151.stdout: "num_mons": 3 2024-09-16T15:54:36.992 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:54:36.992 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap": { 2024-09-16T15:54:36.992 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 133, 2024-09-16T15:54:36.992 INFO:teuthology.orchestra.run.smithi151.stdout: "num_osds": 8, 2024-09-16T15:54:36.992 INFO:teuthology.orchestra.run.smithi151.stdout: "num_up_osds": 8, 2024-09-16T15:54:36.992 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_up_since": 1726501150, 2024-09-16T15:54:36.992 INFO:teuthology.orchestra.run.smithi151.stdout: "num_in_osds": 8, 2024-09-16T15:54:36.992 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_in_since": 1726501140, 2024-09-16T15:54:36.992 INFO:teuthology.orchestra.run.smithi151.stdout: "num_remapped_pgs": 0 2024-09-16T15:54:36.992 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:54:36.992 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap": { 2024-09-16T15:54:36.992 INFO:teuthology.orchestra.run.smithi151.stdout: "pgs_by_state": [ 2024-09-16T15:54:36.992 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:54:36.992 INFO:teuthology.orchestra.run.smithi151.stdout: "state_name": "active+clean", 2024-09-16T15:54:36.993 INFO:teuthology.orchestra.run.smithi151.stdout: "count": 97 2024-09-16T15:54:36.993 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:54:36.993 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:54:36.993 INFO:teuthology.orchestra.run.smithi151.stdout: "num_pgs": 97, 2024-09-16T15:54:36.993 INFO:teuthology.orchestra.run.smithi151.stdout: "num_pools": 3, 2024-09-16T15:54:36.993 INFO:teuthology.orchestra.run.smithi151.stdout: "num_objects": 860, 2024-09-16T15:54:36.993 INFO:teuthology.orchestra.run.smithi151.stdout: "data_bytes": 3477852501, 2024-09-16T15:54:36.993 INFO:teuthology.orchestra.run.smithi151.stdout: "bytes_used": 8041713664, 2024-09-16T15:54:36.993 INFO:teuthology.orchestra.run.smithi151.stdout: "bytes_avail": 765052399616, 2024-09-16T15:54:36.993 INFO:teuthology.orchestra.run.smithi151.stdout: "bytes_total": 773094113280, 2024-09-16T15:54:36.993 INFO:teuthology.orchestra.run.smithi151.stdout: "read_bytes_sec": 19565094, 2024-09-16T15:54:36.993 INFO:teuthology.orchestra.run.smithi151.stdout: "write_bytes_sec": 111452258, 2024-09-16T15:54:36.993 INFO:teuthology.orchestra.run.smithi151.stdout: "read_op_per_sec": 5, 2024-09-16T15:54:36.994 INFO:teuthology.orchestra.run.smithi151.stdout: "write_op_per_sec": 29 2024-09-16T15:54:36.994 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:54:36.994 INFO:teuthology.orchestra.run.smithi151.stdout: "fsmap": { 2024-09-16T15:54:36.994 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 88, 2024-09-16T15:54:36.994 INFO:teuthology.orchestra.run.smithi151.stdout: "btime": "2024-09-16T15:54:00:531362+0000", 2024-09-16T15:54:36.994 INFO:teuthology.orchestra.run.smithi151.stdout: "id": 9, 2024-09-16T15:54:36.994 INFO:teuthology.orchestra.run.smithi151.stdout: "up": 1, 2024-09-16T15:54:36.994 INFO:teuthology.orchestra.run.smithi151.stdout: "in": 1, 2024-09-16T15:54:36.994 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 1, 2024-09-16T15:54:36.994 INFO:teuthology.orchestra.run.smithi151.stdout: "by_rank": [ 2024-09-16T15:54:36.994 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:54:36.994 INFO:teuthology.orchestra.run.smithi151.stdout: "filesystem_id": 9, 2024-09-16T15:54:36.994 INFO:teuthology.orchestra.run.smithi151.stdout: "rank": 0, 2024-09-16T15:54:36.994 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "a", 2024-09-16T15:54:36.995 INFO:teuthology.orchestra.run.smithi151.stdout: "status": "up:active", 2024-09-16T15:54:36.995 INFO:teuthology.orchestra.run.smithi151.stdout: "gid": 9009 2024-09-16T15:54:36.995 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:54:36.995 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:54:36.995 INFO:teuthology.orchestra.run.smithi151.stdout: "up:standby": 3 2024-09-16T15:54:36.995 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:54:36.995 INFO:teuthology.orchestra.run.smithi151.stdout: "mgrmap": { 2024-09-16T15:54:36.995 INFO:teuthology.orchestra.run.smithi151.stdout: "available": true, 2024-09-16T15:54:36.995 INFO:teuthology.orchestra.run.smithi151.stdout: "num_standbys": 1, 2024-09-16T15:54:36.995 INFO:teuthology.orchestra.run.smithi151.stdout: "modules": [ 2024-09-16T15:54:36.995 INFO:teuthology.orchestra.run.smithi151.stdout: "iostat", 2024-09-16T15:54:36.995 INFO:teuthology.orchestra.run.smithi151.stdout: "nfs", 2024-09-16T15:54:36.995 INFO:teuthology.orchestra.run.smithi151.stdout: "restful" 2024-09-16T15:54:36.996 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:54:36.996 INFO:teuthology.orchestra.run.smithi151.stdout: "services": {} 2024-09-16T15:54:36.996 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:54:36.996 INFO:teuthology.orchestra.run.smithi151.stdout: "servicemap": { 2024-09-16T15:54:36.996 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 19, 2024-09-16T15:54:36.996 INFO:teuthology.orchestra.run.smithi151.stdout: "modified": "2024-09-16T15:54:35.952767+0000", 2024-09-16T15:54:36.996 INFO:teuthology.orchestra.run.smithi151.stdout: "services": { 2024-09-16T15:54:36.996 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": { 2024-09-16T15:54:36.996 INFO:teuthology.orchestra.run.smithi151.stdout: "daemons": { 2024-09-16T15:54:36.996 INFO:teuthology.orchestra.run.smithi151.stdout: "summary": "", 2024-09-16T15:54:36.996 INFO:teuthology.orchestra.run.smithi151.stdout: "a": { 2024-09-16T15:54:36.996 INFO:teuthology.orchestra.run.smithi151.stdout: "start_epoch": 0, 2024-09-16T15:54:36.996 INFO:teuthology.orchestra.run.smithi151.stdout: "start_stamp": "0.000000", 2024-09-16T15:54:36.997 INFO:teuthology.orchestra.run.smithi151.stdout: "gid": 0, 2024-09-16T15:54:36.997 INFO:teuthology.orchestra.run.smithi151.stdout: "addr": "(unrecognized address family 0)/0", 2024-09-16T15:54:36.997 INFO:teuthology.orchestra.run.smithi151.stdout: "metadata": {}, 2024-09-16T15:54:36.997 INFO:teuthology.orchestra.run.smithi151.stdout: "task_status": {} 2024-09-16T15:54:36.997 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:54:36.997 INFO:teuthology.orchestra.run.smithi151.stdout: "b": { 2024-09-16T15:54:36.997 INFO:teuthology.orchestra.run.smithi151.stdout: "start_epoch": 0, 2024-09-16T15:54:36.997 INFO:teuthology.orchestra.run.smithi151.stdout: "start_stamp": "0.000000", 2024-09-16T15:54:36.997 INFO:teuthology.orchestra.run.smithi151.stdout: "gid": 0, 2024-09-16T15:54:36.997 INFO:teuthology.orchestra.run.smithi151.stdout: "addr": "(unrecognized address family 0)/0", 2024-09-16T15:54:36.997 INFO:teuthology.orchestra.run.smithi151.stdout: "metadata": {}, 2024-09-16T15:54:36.997 INFO:teuthology.orchestra.run.smithi151.stdout: "task_status": {} 2024-09-16T15:54:36.997 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:54:36.997 INFO:teuthology.orchestra.run.smithi151.stdout: "d": { 2024-09-16T15:54:36.998 INFO:teuthology.orchestra.run.smithi151.stdout: "start_epoch": 0, 2024-09-16T15:54:36.998 INFO:teuthology.orchestra.run.smithi151.stdout: "start_stamp": "0.000000", 2024-09-16T15:54:36.998 INFO:teuthology.orchestra.run.smithi151.stdout: "gid": 0, 2024-09-16T15:54:36.998 INFO:teuthology.orchestra.run.smithi151.stdout: "addr": "(unrecognized address family 0)/0", 2024-09-16T15:54:36.998 INFO:teuthology.orchestra.run.smithi151.stdout: "metadata": {}, 2024-09-16T15:54:36.998 INFO:teuthology.orchestra.run.smithi151.stdout: "task_status": {} 2024-09-16T15:54:36.998 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:54:36.998 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:54:36.998 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:54:36.998 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": { 2024-09-16T15:54:36.998 INFO:teuthology.orchestra.run.smithi151.stdout: "daemons": { 2024-09-16T15:54:36.998 INFO:teuthology.orchestra.run.smithi151.stdout: "summary": "", 2024-09-16T15:54:36.998 INFO:teuthology.orchestra.run.smithi151.stdout: "x": { 2024-09-16T15:54:36.999 INFO:teuthology.orchestra.run.smithi151.stdout: "start_epoch": 0, 2024-09-16T15:54:36.999 INFO:teuthology.orchestra.run.smithi151.stdout: "start_stamp": "0.000000", 2024-09-16T15:54:36.999 INFO:teuthology.orchestra.run.smithi151.stdout: "gid": 0, 2024-09-16T15:54:36.999 INFO:teuthology.orchestra.run.smithi151.stdout: "addr": "(unrecognized address family 0)/0", 2024-09-16T15:54:36.999 INFO:teuthology.orchestra.run.smithi151.stdout: "metadata": {}, 2024-09-16T15:54:36.999 INFO:teuthology.orchestra.run.smithi151.stdout: "task_status": {} 2024-09-16T15:54:36.999 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:54:36.999 INFO:teuthology.orchestra.run.smithi151.stdout: "y": { 2024-09-16T15:54:36.999 INFO:teuthology.orchestra.run.smithi151.stdout: "start_epoch": 0, 2024-09-16T15:54:36.999 INFO:teuthology.orchestra.run.smithi151.stdout: "start_stamp": "0.000000", 2024-09-16T15:54:36.999 INFO:teuthology.orchestra.run.smithi151.stdout: "gid": 0, 2024-09-16T15:54:36.999 INFO:teuthology.orchestra.run.smithi151.stdout: "addr": "(unrecognized address family 0)/0", 2024-09-16T15:54:36.999 INFO:teuthology.orchestra.run.smithi151.stdout: "metadata": {}, 2024-09-16T15:54:36.999 INFO:teuthology.orchestra.run.smithi151.stdout: "task_status": {} 2024-09-16T15:54:37.000 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:54:37.000 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:54:37.000 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:54:37.000 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": { 2024-09-16T15:54:37.000 INFO:teuthology.orchestra.run.smithi151.stdout: "daemons": { 2024-09-16T15:54:37.000 INFO:teuthology.orchestra.run.smithi151.stdout: "summary": "", 2024-09-16T15:54:37.000 INFO:teuthology.orchestra.run.smithi151.stdout: "a": { 2024-09-16T15:54:37.000 INFO:teuthology.orchestra.run.smithi151.stdout: "start_epoch": 0, 2024-09-16T15:54:37.000 INFO:teuthology.orchestra.run.smithi151.stdout: "start_stamp": "0.000000", 2024-09-16T15:54:37.000 INFO:teuthology.orchestra.run.smithi151.stdout: "gid": 0, 2024-09-16T15:54:37.000 INFO:teuthology.orchestra.run.smithi151.stdout: "addr": "(unrecognized address family 0)/0", 2024-09-16T15:54:37.000 INFO:teuthology.orchestra.run.smithi151.stdout: "metadata": {}, 2024-09-16T15:54:37.000 INFO:teuthology.orchestra.run.smithi151.stdout: "task_status": {} 2024-09-16T15:54:37.001 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:54:37.001 INFO:teuthology.orchestra.run.smithi151.stdout: "c": { 2024-09-16T15:54:37.001 INFO:teuthology.orchestra.run.smithi151.stdout: "start_epoch": 0, 2024-09-16T15:54:37.001 INFO:teuthology.orchestra.run.smithi151.stdout: "start_stamp": "0.000000", 2024-09-16T15:54:37.001 INFO:teuthology.orchestra.run.smithi151.stdout: "gid": 0, 2024-09-16T15:54:37.001 INFO:teuthology.orchestra.run.smithi151.stdout: "addr": "(unrecognized address family 0)/0", 2024-09-16T15:54:37.001 INFO:teuthology.orchestra.run.smithi151.stdout: "metadata": {}, 2024-09-16T15:54:37.001 INFO:teuthology.orchestra.run.smithi151.stdout: "task_status": {} 2024-09-16T15:54:37.001 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:54:37.001 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:54:37.001 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:54:37.001 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": { 2024-09-16T15:54:37.001 INFO:teuthology.orchestra.run.smithi151.stdout: "daemons": { 2024-09-16T15:54:37.001 INFO:teuthology.orchestra.run.smithi151.stdout: "summary": "", 2024-09-16T15:54:37.002 INFO:teuthology.orchestra.run.smithi151.stdout: "2": { 2024-09-16T15:54:37.002 INFO:teuthology.orchestra.run.smithi151.stdout: "start_epoch": 0, 2024-09-16T15:54:37.002 INFO:teuthology.orchestra.run.smithi151.stdout: "start_stamp": "0.000000", 2024-09-16T15:54:37.002 INFO:teuthology.orchestra.run.smithi151.stdout: "gid": 0, 2024-09-16T15:54:37.002 INFO:teuthology.orchestra.run.smithi151.stdout: "addr": "(unrecognized address family 0)/0", 2024-09-16T15:54:37.002 INFO:teuthology.orchestra.run.smithi151.stdout: "metadata": {}, 2024-09-16T15:54:37.002 INFO:teuthology.orchestra.run.smithi151.stdout: "task_status": {} 2024-09-16T15:54:37.002 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:54:37.002 INFO:teuthology.orchestra.run.smithi151.stdout: "3": { 2024-09-16T15:54:37.002 INFO:teuthology.orchestra.run.smithi151.stdout: "start_epoch": 0, 2024-09-16T15:54:37.002 INFO:teuthology.orchestra.run.smithi151.stdout: "start_stamp": "0.000000", 2024-09-16T15:54:37.002 INFO:teuthology.orchestra.run.smithi151.stdout: "gid": 0, 2024-09-16T15:54:37.002 INFO:teuthology.orchestra.run.smithi151.stdout: "addr": "(unrecognized address family 0)/0", 2024-09-16T15:54:37.002 INFO:teuthology.orchestra.run.smithi151.stdout: "metadata": {}, 2024-09-16T15:54:37.003 INFO:teuthology.orchestra.run.smithi151.stdout: "task_status": {} 2024-09-16T15:54:37.003 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:54:37.003 INFO:teuthology.orchestra.run.smithi151.stdout: "4": { 2024-09-16T15:54:37.003 INFO:teuthology.orchestra.run.smithi151.stdout: "start_epoch": 0, 2024-09-16T15:54:37.003 INFO:teuthology.orchestra.run.smithi151.stdout: "start_stamp": "0.000000", 2024-09-16T15:54:37.003 INFO:teuthology.orchestra.run.smithi151.stdout: "gid": 0, 2024-09-16T15:54:37.003 INFO:teuthology.orchestra.run.smithi151.stdout: "addr": "(unrecognized address family 0)/0", 2024-09-16T15:54:37.003 INFO:teuthology.orchestra.run.smithi151.stdout: "metadata": {}, 2024-09-16T15:54:37.003 INFO:teuthology.orchestra.run.smithi151.stdout: "task_status": {} 2024-09-16T15:54:37.003 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:54:37.003 INFO:teuthology.orchestra.run.smithi151.stdout: "5": { 2024-09-16T15:54:37.003 INFO:teuthology.orchestra.run.smithi151.stdout: "start_epoch": 0, 2024-09-16T15:54:37.003 INFO:teuthology.orchestra.run.smithi151.stdout: "start_stamp": "0.000000", 2024-09-16T15:54:37.003 INFO:teuthology.orchestra.run.smithi151.stdout: "gid": 0, 2024-09-16T15:54:37.003 INFO:teuthology.orchestra.run.smithi151.stdout: "addr": "(unrecognized address family 0)/0", 2024-09-16T15:54:37.004 INFO:teuthology.orchestra.run.smithi151.stdout: "metadata": {}, 2024-09-16T15:54:37.004 INFO:teuthology.orchestra.run.smithi151.stdout: "task_status": {} 2024-09-16T15:54:37.004 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:54:37.004 INFO:teuthology.orchestra.run.smithi151.stdout: "6": { 2024-09-16T15:54:37.004 INFO:teuthology.orchestra.run.smithi151.stdout: "start_epoch": 0, 2024-09-16T15:54:37.004 INFO:teuthology.orchestra.run.smithi151.stdout: "start_stamp": "0.000000", 2024-09-16T15:54:37.004 INFO:teuthology.orchestra.run.smithi151.stdout: "gid": 0, 2024-09-16T15:54:37.004 INFO:teuthology.orchestra.run.smithi151.stdout: "addr": "(unrecognized address family 0)/0", 2024-09-16T15:54:37.004 INFO:teuthology.orchestra.run.smithi151.stdout: "metadata": {}, 2024-09-16T15:54:37.004 INFO:teuthology.orchestra.run.smithi151.stdout: "task_status": {} 2024-09-16T15:54:37.004 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:54:37.004 INFO:teuthology.orchestra.run.smithi151.stdout: "7": { 2024-09-16T15:54:37.004 INFO:teuthology.orchestra.run.smithi151.stdout: "start_epoch": 0, 2024-09-16T15:54:37.004 INFO:teuthology.orchestra.run.smithi151.stdout: "start_stamp": "0.000000", 2024-09-16T15:54:37.005 INFO:teuthology.orchestra.run.smithi151.stdout: "gid": 0, 2024-09-16T15:54:37.005 INFO:teuthology.orchestra.run.smithi151.stdout: "addr": "(unrecognized address family 0)/0", 2024-09-16T15:54:37.005 INFO:teuthology.orchestra.run.smithi151.stdout: "metadata": {}, 2024-09-16T15:54:37.005 INFO:teuthology.orchestra.run.smithi151.stdout: "task_status": {} 2024-09-16T15:54:37.005 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:54:37.005 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:54:37.005 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:54:37.005 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:54:37.005 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:54:37.005 INFO:teuthology.orchestra.run.smithi151.stdout: "progress_events": {} 2024-09-16T15:54:37.005 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-16T15:54:37.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:36.990+0000 7f6116800640 1 -- 172.21.15.151:0/2673446176 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f60f007aa10 msgr2=0x7f60f007ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:37.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:36.990+0000 7f6116800640 1 --2- 172.21.15.151:0/2673446176 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f60f007aa10 0x7f60f007ced0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f6104033000 tx=0x7f6104004730 comp rx=0 tx=0).stop 2024-09-16T15:54:37.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:36.991+0000 7f6116800640 1 -- 172.21.15.151:0/2673446176 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f61101563a0 msgr2=0x7f61100cec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:37.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:36.991+0000 7f6116800640 1 --2- 172.21.15.151:0/2673446176 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f61101563a0 0x7f61100cec20 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f610800ec50 tx=0x7f610800c6a0 comp rx=0 tx=0).stop 2024-09-16T15:54:37.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:36.991+0000 7f6116800640 1 -- 172.21.15.151:0/2673446176 shutdown_connections 2024-09-16T15:54:37.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:36.991+0000 7f6116800640 1 --2- 172.21.15.151:0/2673446176 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f60f007aa10 0x7f60f007ced0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:37.006 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:36.991+0000 7f6116800640 1 --2- 172.21.15.151:0/2673446176 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f61101563a0 0x7f61100cec20 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:37.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:36.991+0000 7f6116800640 1 --2- 172.21.15.151:0/2673446176 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f61100ccd70 0x7f6110154ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:37.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:36.991+0000 7f6116800640 1 --2- 172.21.15.151:0/2673446176 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f611006e9b0 0x7f61101f8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:37.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:36.991+0000 7f6116800640 1 -- 172.21.15.151:0/2673446176 >> 172.21.15.151:0/2673446176 conn(0x7f61100b18e0 msgr2=0x7f6110075890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:37.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:36.991+0000 7f6116800640 1 -- 172.21.15.151:0/2673446176 shutdown_connections 2024-09-16T15:54:37.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:36.991+0000 7f6116800640 1 -- 172.21.15.151:0/2673446176 wait complete. 2024-09-16T15:54:37.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:54:37.008 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -lh volumes/_nogroup/ss1clone1/4ecd5764-29e8-466f-92ae-20e498fe9453 2024-09-16T15:54:37.008 DEBUG:teuthology.orchestra.run.smithi151:> ) 2024-09-16T15:54:37.045 INFO:teuthology.orchestra.run.smithi151.stderr:ls: cannot access 'volumes/_nogroup/ss1clone1/4ecd5764-29e8-466f-92ae-20e498fe9453': No such file or directory 2024-09-16T15:54:37.045 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-16T15:54:37.046 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-16T15:54:37.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.245+0000 7fc5e3000640 1 -- 172.21.15.151:0/405636053 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc5c0004800 msgr2=0x7fc5c0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:37.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.245+0000 7fc5e3000640 1 --2- 172.21.15.151:0/405636053 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc5c0004800 0x7fc5c0004c80 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7fc5c800bbb0 tx=0x7fc5c0005830 comp rx=0 tx=0).stop 2024-09-16T15:54:37.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.245+0000 7fc5e1000640 1 -- 172.21.15.151:0/405636053 reap_dead start 2024-09-16T15:54:37.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.245+0000 7fc5e3000640 1 -- 172.21.15.151:0/405636053 shutdown_connections 2024-09-16T15:54:37.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.245+0000 7fc5e3000640 1 -- 172.21.15.151:0/405636053 >> 172.21.15.151:0/405636053 conn(0x7fc5dc0b1950 msgr2=0x7fc5dc0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:37.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.245+0000 7fc5e3000640 1 -- 172.21.15.151:0/405636053 shutdown_connections 2024-09-16T15:54:37.247 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.245+0000 7fc5e3000640 1 -- 172.21.15.151:0/405636053 wait complete. 2024-09-16T15:54:37.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.247+0000 7fc5e3000640 1 Processor -- start 2024-09-16T15:54:37.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.247+0000 7fc5e3000640 1 -- start start 2024-09-16T15:54:37.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.248+0000 7fc5e3000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc5c0004800 0x7fc5dc1f4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:37.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.248+0000 7fc5e3000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5dc1e83c0 0x7fc5dc153370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:37.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.248+0000 7fc5e3000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5dc15a4a0 0x7fc5dc1544d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:37.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.248+0000 7fc5e3000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc5dc0745c0 con 0x7fc5c0004800 2024-09-16T15:54:37.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.248+0000 7fc5e3000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc5dc074440 con 0x7fc5dc15a4a0 2024-09-16T15:54:37.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.248+0000 7fc5e3000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fc5dc074740 con 0x7fc5dc1e83c0 2024-09-16T15:54:37.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.248+0000 7fc5dbe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc5c0004800 0x7fc5dc1f4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:37.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.248+0000 7fc5dbe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc5c0004800 0x7fc5dc1f4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41496/0 (socket says 172.21.15.151:41496) 2024-09-16T15:54:37.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.248+0000 7fc5dbe00640 1 -- 172.21.15.151:0/4219662246 learned_addr learned my addr 172.21.15.151:0/4219662246 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:54:37.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.248+0000 7fc5db400640 1 --2- 172.21.15.151:0/4219662246 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5dc1e83c0 0x7fc5dc153370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:37.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.248+0000 7fc5e1000640 1 --2- 172.21.15.151:0/4219662246 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5dc15a4a0 0x7fc5dc1544d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:37.250 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.249+0000 7fc5dbe00640 1 -- 172.21.15.151:0/4219662246 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5dc1e83c0 msgr2=0x7fc5dc153370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:37.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.249+0000 7fc5dbe00640 1 --2- 172.21.15.151:0/4219662246 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5dc1e83c0 0x7fc5dc153370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:37.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.249+0000 7fc5dbe00640 1 -- 172.21.15.151:0/4219662246 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5dc15a4a0 msgr2=0x7fc5dc1544d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:37.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.249+0000 7fc5dbe00640 1 --2- 172.21.15.151:0/4219662246 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5dc15a4a0 0x7fc5dc1544d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:37.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.249+0000 7fc5dbe00640 1 -- 172.21.15.151:0/4219662246 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5dc15b4c0 con 0x7fc5c0004800 2024-09-16T15:54:37.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.249+0000 7fc5dbe00640 1 --2- 172.21.15.151:0/4219662246 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc5c0004800 0x7fc5dc1f4c90 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc006290 tx=0x7fc5cc006690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:37.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.249+0000 7fc5d3e00640 1 -- 172.21.15.151:0/4219662246 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc015070 con 0x7fc5c0004800 2024-09-16T15:54:37.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.249+0000 7fc5d3e00640 1 -- 172.21.15.151:0/4219662246 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc5cc00fb50 con 0x7fc5c0004800 2024-09-16T15:54:37.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.249+0000 7fc5e3000640 1 -- 172.21.15.151:0/4219662246 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5dc15b7a0 con 0x7fc5c0004800 2024-09-16T15:54:37.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.249+0000 7fc5d3e00640 1 -- 172.21.15.151:0/4219662246 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc011430 con 0x7fc5c0004800 2024-09-16T15:54:37.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.249+0000 7fc5e3000640 1 -- 172.21.15.151:0/4219662246 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5dc15a920 con 0x7fc5c0004800 2024-09-16T15:54:37.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.250+0000 7fc5e3000640 1 -- 172.21.15.151:0/4219662246 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5dc15b2e0 con 0x7fc5c0004800 2024-09-16T15:54:37.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.251+0000 7fc5d3e00640 1 -- 172.21.15.151:0/4219662246 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 49) ==== 104347+0+0 (secure 0 0 0) 0x7fc5cc0115d0 con 0x7fc5c0004800 2024-09-16T15:54:37.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.252+0000 7fc5d3e00640 1 --2- 172.21.15.151:0/4219662246 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc5bc07aa40 0x7fc5bc07cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:37.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.252+0000 7fc5d3e00640 1 -- 172.21.15.151:0/4219662246 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5821+0+0 (secure 0 0 0) 0x7fc5cc09a950 con 0x7fc5c0004800 2024-09-16T15:54:37.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.253+0000 7fc5db400640 1 --2- 172.21.15.151:0/4219662246 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc5bc07aa40 0x7fc5bc07cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:37.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.254+0000 7fc5db400640 1 --2- 172.21.15.151:0/4219662246 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc5bc07aa40 0x7fc5bc07cf00 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fc5c8036000 tx=0x7fc5c80046c0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:37.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.257+0000 7fc5d3e00640 1 -- 172.21.15.151:0/4219662246 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc5dc15b2e0 con 0x7fc5c0004800 2024-09-16T15:54:37.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.462+0000 7fc5e3000640 1 -- 172.21.15.151:0/4219662246 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fc5dc1f8260 con 0x7fc5bc07aa40 2024-09-16T15:54:37.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.464+0000 7fc5d3e00640 1 -- 172.21.15.151:0/4219662246 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fc5dc1f8260 con 0x7fc5bc07aa40 2024-09-16T15:54:37.465 INFO:teuthology.orchestra.run.smithi151.stdout:[ 2024-09-16T15:54:37.465 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:54:37.466 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "cephfs" 2024-09-16T15:54:37.466 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:54:37.466 INFO:teuthology.orchestra.run.smithi151.stdout:] 2024-09-16T15:54:37.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.466+0000 7fc5e3000640 1 -- 172.21.15.151:0/4219662246 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc5bc07aa40 msgr2=0x7fc5bc07cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:37.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.466+0000 7fc5e3000640 1 --2- 172.21.15.151:0/4219662246 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc5bc07aa40 0x7fc5bc07cf00 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fc5c8036000 tx=0x7fc5c80046c0 comp rx=0 tx=0).stop 2024-09-16T15:54:37.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.466+0000 7fc5e3000640 1 -- 172.21.15.151:0/4219662246 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc5c0004800 msgr2=0x7fc5dc1f4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:37.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.466+0000 7fc5e3000640 1 --2- 172.21.15.151:0/4219662246 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc5c0004800 0x7fc5dc1f4c90 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc006290 tx=0x7fc5cc006690 comp rx=0 tx=0).stop 2024-09-16T15:54:37.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.467+0000 7fc5e3000640 1 -- 172.21.15.151:0/4219662246 shutdown_connections 2024-09-16T15:54:37.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.467+0000 7fc5e3000640 1 --2- 172.21.15.151:0/4219662246 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc5bc07aa40 0x7fc5bc07cf00 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:37.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.467+0000 7fc5e3000640 1 --2- 172.21.15.151:0/4219662246 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5dc15a4a0 0x7fc5dc1544d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:37.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.467+0000 7fc5e3000640 1 --2- 172.21.15.151:0/4219662246 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc5dc1e83c0 0x7fc5dc153370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:37.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.467+0000 7fc5e3000640 1 --2- 172.21.15.151:0/4219662246 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc5c0004800 0x7fc5dc1f4c90 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:37.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.467+0000 7fc5e3000640 1 -- 172.21.15.151:0/4219662246 >> 172.21.15.151:0/4219662246 conn(0x7fc5dc0b1950 msgr2=0x7fc5dc0acd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:37.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.467+0000 7fc5e3000640 1 -- 172.21.15.151:0/4219662246 shutdown_connections 2024-09-16T15:54:37.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.467+0000 7fc5e3000640 1 -- 172.21.15.151:0/4219662246 wait complete. 2024-09-16T15:54:37.483 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-09-16T15:54:37.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.643+0000 7f25e3e00640 1 -- 172.21.15.151:0/2079681505 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f25dc0ccb00 msgr2=0x7f25dc15f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:37.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.643+0000 7f25e3e00640 1 --2- 172.21.15.151:0/2079681505 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f25dc0ccb00 0x7f25dc15f9f0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f25dc074ff0 tx=0x7f25d402a980 comp rx=0 tx=0).stop 2024-09-16T15:54:37.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.643+0000 7f25e3e00640 1 -- 172.21.15.151:0/2079681505 shutdown_connections 2024-09-16T15:54:37.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.643+0000 7f25e3e00640 1 --2- 172.21.15.151:0/2079681505 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f25dc0ccb00 0x7f25dc15f9f0 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:37.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.643+0000 7f25e3e00640 1 --2- 172.21.15.151:0/2079681505 >> v2:172.21.15.167:3300/0 conn(0x7f25dc06efa0 0x7f25dc0cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:37.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.643+0000 7f25e3e00640 1 --2- 172.21.15.151:0/2079681505 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25dc145750 0x7f25dc073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:37.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.643+0000 7f25e3e00640 1 -- 172.21.15.151:0/2079681505 >> 172.21.15.151:0/2079681505 conn(0x7f25dc0b1840 msgr2=0x7f25dc0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:37.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.644+0000 7f25e3e00640 1 -- 172.21.15.151:0/2079681505 shutdown_connections 2024-09-16T15:54:37.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.644+0000 7f25e3e00640 1 -- 172.21.15.151:0/2079681505 wait complete. 2024-09-16T15:54:37.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.645+0000 7f25e3e00640 1 Processor -- start 2024-09-16T15:54:37.646 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.645+0000 7f25e3e00640 1 -- start start 2024-09-16T15:54:37.648 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.646+0000 7f25e3e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25dc06efa0 0x7f25dc15f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:37.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.646+0000 7f25e3e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f25dc0ccb00 0x7f25dc0ade40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:37.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.646+0000 7f25e3e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25dc145750 0x7f25dc0ae380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:37.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.646+0000 7f25e3e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f25dc1600b0 con 0x7f25dc0ccb00 2024-09-16T15:54:37.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.646+0000 7f25e3e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f25dc15ff30 con 0x7f25dc145750 2024-09-16T15:54:37.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.646+0000 7f25e3e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f25dc160230 con 0x7f25dc06efa0 2024-09-16T15:54:37.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.646+0000 7f25e0a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f25dc0ccb00 0x7f25dc0ade40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:37.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.646+0000 7f25e0a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f25dc0ccb00 0x7f25dc0ade40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41516/0 (socket says 172.21.15.151:41516) 2024-09-16T15:54:37.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.646+0000 7f25e0a00640 1 -- 172.21.15.151:0/2967308390 learned_addr learned my addr 172.21.15.151:0/2967308390 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:54:37.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.647+0000 7f25e1400640 1 --2- 172.21.15.151:0/2967308390 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25dc06efa0 0x7f25dc15f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:37.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.647+0000 7f25e0a00640 1 -- 172.21.15.151:0/2967308390 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25dc06efa0 msgr2=0x7f25dc15f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:37.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.647+0000 7f25e1e00640 1 --2- 172.21.15.151:0/2967308390 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25dc145750 0x7f25dc0ae380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:37.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.647+0000 7f25e0a00640 1 --2- 172.21.15.151:0/2967308390 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25dc06efa0 0x7f25dc15f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:37.650 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.647+0000 7f25e0a00640 1 -- 172.21.15.151:0/2967308390 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25dc145750 msgr2=0x7f25dc0ae380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:37.650 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.647+0000 7f25e0a00640 1 --2- 172.21.15.151:0/2967308390 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25dc145750 0x7f25dc0ae380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:37.650 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.647+0000 7f25e0a00640 1 -- 172.21.15.151:0/2967308390 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25dc1f55d0 con 0x7f25dc0ccb00 2024-09-16T15:54:37.650 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.647+0000 7f25e1e00640 1 --2- 172.21.15.151:0/2967308390 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25dc145750 0x7f25dc0ae380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:54:37.650 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.647+0000 7f25e0a00640 1 --2- 172.21.15.151:0/2967308390 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f25dc0ccb00 0x7f25dc0ade40 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f25c800d720 tx=0x7f25c800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:37.650 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.647+0000 7f25d9600640 1 -- 172.21.15.151:0/2967308390 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c8017070 con 0x7f25dc0ccb00 2024-09-16T15:54:37.650 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.647+0000 7f25d9600640 1 -- 172.21.15.151:0/2967308390 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f25c8009d20 con 0x7f25dc0ccb00 2024-09-16T15:54:37.650 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.647+0000 7f25e3e00640 1 -- 172.21.15.151:0/2967308390 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25dc0ca0b0 con 0x7f25dc0ccb00 2024-09-16T15:54:37.650 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.647+0000 7f25d9600640 1 -- 172.21.15.151:0/2967308390 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c8004280 con 0x7f25dc0ccb00 2024-09-16T15:54:37.650 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.648+0000 7f25e3e00640 1 -- 172.21.15.151:0/2967308390 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25dc15fc20 con 0x7f25dc0ccb00 2024-09-16T15:54:37.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.649+0000 7f25e3e00640 1 -- 172.21.15.151:0/2967308390 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25dc160710 con 0x7f25dc0ccb00 2024-09-16T15:54:37.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.650+0000 7f25d9600640 1 -- 172.21.15.151:0/2967308390 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 49) ==== 104347+0+0 (secure 0 0 0) 0x7f25c8005020 con 0x7f25dc0ccb00 2024-09-16T15:54:37.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.651+0000 7f25d9600640 1 --2- 172.21.15.151:0/2967308390 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f25bc07aa40 0x7f25bc07cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:37.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.651+0000 7f25d9600640 1 -- 172.21.15.151:0/2967308390 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5821+0+0 (secure 0 0 0) 0x7f25c8099ea0 con 0x7f25dc0ccb00 2024-09-16T15:54:37.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.651+0000 7f25e1400640 1 --2- 172.21.15.151:0/2967308390 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f25bc07aa40 0x7f25bc07cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:37.653 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.652+0000 7f25e1400640 1 --2- 172.21.15.151:0/2967308390 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f25bc07aa40 0x7f25bc07cf00 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f25dc135e70 tx=0x7f25d0005c70 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:37.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.655+0000 7f25d9600640 1 -- 172.21.15.151:0/2967308390 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f25c8020020 con 0x7f25dc0ccb00 2024-09-16T15:54:37.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.866+0000 7f25e3e00640 1 -- 172.21.15.151:0/2967308390 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f25dc144c20 con 0x7f25bc07aa40 2024-09-16T15:54:37.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.873+0000 7f25d9600640 1 -- 172.21.15.151:0/2967308390 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+79 (secure 0 0 0) 0x7f25dc144c20 con 0x7f25bc07aa40 2024-09-16T15:54:37.874 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:54:37.874 INFO:teuthology.orchestra.run.smithi151.stdout:[ 2024-09-16T15:54:37.874 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:54:37.874 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sv1" 2024-09-16T15:54:37.874 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:54:37.874 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:54:37.874 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ss1clone1" 2024-09-16T15:54:37.874 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:54:37.874 INFO:teuthology.orchestra.run.smithi151.stdout:] 2024-09-16T15:54:37.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.875+0000 7f25e3e00640 1 -- 172.21.15.151:0/2967308390 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f25bc07aa40 msgr2=0x7f25bc07cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:37.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.875+0000 7f25e3e00640 1 --2- 172.21.15.151:0/2967308390 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f25bc07aa40 0x7f25bc07cf00 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f25dc135e70 tx=0x7f25d0005c70 comp rx=0 tx=0).stop 2024-09-16T15:54:37.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.875+0000 7f25e3e00640 1 -- 172.21.15.151:0/2967308390 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f25dc0ccb00 msgr2=0x7f25dc0ade40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:37.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.875+0000 7f25e3e00640 1 --2- 172.21.15.151:0/2967308390 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f25dc0ccb00 0x7f25dc0ade40 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f25c800d720 tx=0x7f25c800dbe0 comp rx=0 tx=0).stop 2024-09-16T15:54:37.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.875+0000 7f25e3e00640 1 -- 172.21.15.151:0/2967308390 shutdown_connections 2024-09-16T15:54:37.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.875+0000 7f25e3e00640 1 --2- 172.21.15.151:0/2967308390 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f25bc07aa40 0x7f25bc07cf00 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:37.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.875+0000 7f25e3e00640 1 --2- 172.21.15.151:0/2967308390 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f25dc145750 0x7f25dc0ae380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:37.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.875+0000 7f25e3e00640 1 --2- 172.21.15.151:0/2967308390 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f25dc0ccb00 0x7f25dc0ade40 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:37.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.875+0000 7f25e3e00640 1 --2- 172.21.15.151:0/2967308390 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f25dc06efa0 0x7f25dc15f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:37.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.875+0000 7f25e3e00640 1 -- 172.21.15.151:0/2967308390 >> 172.21.15.151:0/2967308390 conn(0x7f25dc0b1840 msgr2=0x7f25dc06f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:37.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.875+0000 7f25e3e00640 1 -- 172.21.15.151:0/2967308390 shutdown_connections 2024-09-16T15:54:37.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:37.875+0000 7f25e3e00640 1 -- 172.21.15.151:0/2967308390 wait complete. 2024-09-16T15:54:37.892 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-09-16T15:54:38.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.054+0000 7f76a5400640 1 -- 172.21.15.151:0/3457508021 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76a0073de0 msgr2=0x7f76a0074240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:38.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.054+0000 7f76a5400640 1 --2- 172.21.15.151:0/3457508021 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76a0073de0 0x7f76a0074240 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f7688005fd0 tx=0x7f768802a540 comp rx=0 tx=0).stop 2024-09-16T15:54:38.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.054+0000 7f76a5400640 1 -- 172.21.15.151:0/3457508021 shutdown_connections 2024-09-16T15:54:38.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.054+0000 7f76a5400640 1 --2- 172.21.15.151:0/3457508021 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f76a006e6f0 0x7f76a006eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:38.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.054+0000 7f76a5400640 1 --2- 172.21.15.151:0/3457508021 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76a0073de0 0x7f76a0074240 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:38.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.054+0000 7f76a5400640 1 -- 172.21.15.151:0/3457508021 >> 172.21.15.151:0/3457508021 conn(0x7f76a00b18e0 msgr2=0x7f76a00b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:38.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.054+0000 7f76a5400640 1 -- 172.21.15.151:0/3457508021 shutdown_connections 2024-09-16T15:54:38.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.054+0000 7f76a5400640 1 -- 172.21.15.151:0/3457508021 wait complete. 2024-09-16T15:54:38.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.056+0000 7f76a5400640 1 Processor -- start 2024-09-16T15:54:38.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.056+0000 7f76a5400640 1 -- start start 2024-09-16T15:54:38.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.057+0000 7f76a5400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f76a006e6f0 0x7f76a00cf910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:38.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.057+0000 7f76a5400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f76a0073de0 0x7f76a0152080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:38.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.057+0000 7f76a5400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76a015f9a0 0x7f76a0144320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:38.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.057+0000 7f76a5400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f76a006f0e0 con 0x7f76a006e6f0 2024-09-16T15:54:38.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.057+0000 7f76a5400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f76a00cd5e0 con 0x7f76a015f9a0 2024-09-16T15:54:38.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.057+0000 7f76a5400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f76a006f260 con 0x7f76a0073de0 2024-09-16T15:54:38.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.057+0000 7f769e800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f76a006e6f0 0x7f76a00cf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:38.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.057+0000 7f769e800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f76a006e6f0 0x7f76a00cf910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41518/0 (socket says 172.21.15.151:41518) 2024-09-16T15:54:38.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.057+0000 7f769e800640 1 -- 172.21.15.151:0/1906875654 learned_addr learned my addr 172.21.15.151:0/1906875654 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:54:38.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.057+0000 7f769f200640 1 --2- 172.21.15.151:0/1906875654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76a015f9a0 0x7f76a0144320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:38.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.057+0000 7f7697e00640 1 --2- 172.21.15.151:0/1906875654 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f76a0073de0 0x7f76a0152080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:38.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.058+0000 7f769e800640 1 -- 172.21.15.151:0/1906875654 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f76a0073de0 msgr2=0x7f76a0152080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:38.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.058+0000 7f769e800640 1 --2- 172.21.15.151:0/1906875654 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f76a0073de0 0x7f76a0152080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:38.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.058+0000 7f769e800640 1 -- 172.21.15.151:0/1906875654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76a015f9a0 msgr2=0x7f76a0144320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:38.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.058+0000 7f769e800640 1 --2- 172.21.15.151:0/1906875654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76a015f9a0 0x7f76a0144320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:38.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.058+0000 7f769e800640 1 -- 172.21.15.151:0/1906875654 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76a009adc0 con 0x7f76a006e6f0 2024-09-16T15:54:38.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.058+0000 7f769e800640 1 --2- 172.21.15.151:0/1906875654 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f76a006e6f0 0x7f76a00cf910 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f769000c3a0 tx=0x7f769000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:38.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.058+0000 7f7696a00640 1 -- 172.21.15.151:0/1906875654 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7690017070 con 0x7f76a006e6f0 2024-09-16T15:54:38.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.058+0000 7f7696a00640 1 -- 172.21.15.151:0/1906875654 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f76900053c0 con 0x7f76a006e6f0 2024-09-16T15:54:38.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.058+0000 7f76a5400640 1 -- 172.21.15.151:0/1906875654 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76a00cee30 con 0x7f76a006e6f0 2024-09-16T15:54:38.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.058+0000 7f7696a00640 1 -- 172.21.15.151:0/1906875654 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7690003850 con 0x7f76a006e6f0 2024-09-16T15:54:38.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.058+0000 7f76a5400640 1 -- 172.21.15.151:0/1906875654 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76a00cf3f0 con 0x7f76a006e6f0 2024-09-16T15:54:38.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.059+0000 7f76a5400640 1 -- 172.21.15.151:0/1906875654 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76a0069360 con 0x7f76a006e6f0 2024-09-16T15:54:38.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.060+0000 7f7696a00640 1 -- 172.21.15.151:0/1906875654 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 49) ==== 104347+0+0 (secure 0 0 0) 0x7f769001a070 con 0x7f76a006e6f0 2024-09-16T15:54:38.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.062+0000 7f7696a00640 1 --2- 172.21.15.151:0/1906875654 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f767c07aa10 0x7f767c07ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:38.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.062+0000 7f7696a00640 1 -- 172.21.15.151:0/1906875654 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5821+0+0 (secure 0 0 0) 0x7f7690098d80 con 0x7f76a006e6f0 2024-09-16T15:54:38.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.062+0000 7f7697e00640 1 --2- 172.21.15.151:0/1906875654 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f767c07aa10 0x7f767c07ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:38.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.063+0000 7f7697e00640 1 --2- 172.21.15.151:0/1906875654 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f767c07aa10 0x7f767c07ced0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f7688033000 tx=0x7f7688004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:38.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.066+0000 7f7696a00640 1 -- 172.21.15.151:0/1906875654 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7690062800 con 0x7f76a006e6f0 2024-09-16T15:54:38.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.284+0000 7f76a5400640 1 -- 172.21.15.151:0/1906875654 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f76a006eba0 con 0x7f767c07aa10 2024-09-16T15:54:38.304 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.303+0000 7f7696a00640 1 -- 172.21.15.151:0/1906875654 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f76a006eba0 con 0x7f767c07aa10 2024-09-16T15:54:38.304 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:54:38.304 INFO:teuthology.orchestra.run.smithi151.stdout:[ 2024-09-16T15:54:38.304 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:54:38.304 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ss1" 2024-09-16T15:54:38.305 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:54:38.305 INFO:teuthology.orchestra.run.smithi151.stdout:] 2024-09-16T15:54:38.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.306+0000 7f76a5400640 1 -- 172.21.15.151:0/1906875654 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f767c07aa10 msgr2=0x7f767c07ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:38.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.306+0000 7f76a5400640 1 --2- 172.21.15.151:0/1906875654 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f767c07aa10 0x7f767c07ced0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f7688033000 tx=0x7f7688004730 comp rx=0 tx=0).stop 2024-09-16T15:54:38.307 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.306+0000 7f76a5400640 1 -- 172.21.15.151:0/1906875654 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f76a006e6f0 msgr2=0x7f76a00cf910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:38.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.306+0000 7f76a5400640 1 --2- 172.21.15.151:0/1906875654 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f76a006e6f0 0x7f76a00cf910 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f769000c3a0 tx=0x7f769000c860 comp rx=0 tx=0).stop 2024-09-16T15:54:38.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.306+0000 7f76a5400640 1 -- 172.21.15.151:0/1906875654 shutdown_connections 2024-09-16T15:54:38.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.306+0000 7f76a5400640 1 --2- 172.21.15.151:0/1906875654 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f767c07aa10 0x7f767c07ced0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:38.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.306+0000 7f76a5400640 1 --2- 172.21.15.151:0/1906875654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76a015f9a0 0x7f76a0144320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:38.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.306+0000 7f76a5400640 1 --2- 172.21.15.151:0/1906875654 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f76a0073de0 0x7f76a0152080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:38.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.306+0000 7f76a5400640 1 --2- 172.21.15.151:0/1906875654 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f76a006e6f0 0x7f76a00cf910 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:38.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.306+0000 7f76a5400640 1 -- 172.21.15.151:0/1906875654 >> 172.21.15.151:0/1906875654 conn(0x7f76a00b18e0 msgr2=0x7f76a00ace10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:38.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.306+0000 7f76a5400640 1 -- 172.21.15.151:0/1906875654 shutdown_connections 2024-09-16T15:54:38.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.306+0000 7f76a5400640 1 -- 172.21.15.151:0/1906875654 wait complete. 2024-09-16T15:54:38.324 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-09-16T15:54:38.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.489+0000 7fa98c400640 1 -- 172.21.15.151:0/772952655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa96c003a50 msgr2=0x7fa96c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:38.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.489+0000 7fa98c400640 1 --2- 172.21.15.151:0/772952655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa96c003a50 0x7fa96c003e70 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fa97800c260 tx=0x7fa96c005830 comp rx=0 tx=0).stop 2024-09-16T15:54:38.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.489+0000 7fa98a400640 1 -- 172.21.15.151:0/772952655 reap_dead start 2024-09-16T15:54:38.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.489+0000 7fa98c400640 1 -- 172.21.15.151:0/772952655 shutdown_connections 2024-09-16T15:54:38.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.489+0000 7fa98c400640 1 -- 172.21.15.151:0/772952655 >> 172.21.15.151:0/772952655 conn(0x7fa9840b1840 msgr2=0x7fa9840b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:38.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.489+0000 7fa98c400640 1 -- 172.21.15.151:0/772952655 shutdown_connections 2024-09-16T15:54:38.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.489+0000 7fa98c400640 1 -- 172.21.15.151:0/772952655 wait complete. 2024-09-16T15:54:38.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.490+0000 7fa98c400640 1 Processor -- start 2024-09-16T15:54:38.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.490+0000 7fa98c400640 1 -- start start 2024-09-16T15:54:38.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.491+0000 7fa98c400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa96c003a50 0x7fa984143b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:38.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.491+0000 7fa98c400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa984144350 0x7fa984158b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:38.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.491+0000 7fa98c400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9841590a0 0x7fa9841f4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:38.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.491+0000 7fa98c400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa9840b2ed0 con 0x7fa9841590a0 2024-09-16T15:54:38.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.491+0000 7fa98c400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa9840b2d50 con 0x7fa984144350 2024-09-16T15:54:38.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.491+0000 7fa98c400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fa9840b3050 con 0x7fa96c003a50 2024-09-16T15:54:38.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.492+0000 7fa98a400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9841590a0 0x7fa9841f4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:38.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.492+0000 7fa98a400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9841590a0 0x7fa9841f4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41546/0 (socket says 172.21.15.151:41546) 2024-09-16T15:54:38.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.492+0000 7fa98a400640 1 -- 172.21.15.151:0/2935026343 learned_addr learned my addr 172.21.15.151:0/2935026343 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:54:38.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.492+0000 7fa989000640 1 --2- 172.21.15.151:0/2935026343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa984144350 0x7fa984158b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:38.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.492+0000 7fa989a00640 1 --2- 172.21.15.151:0/2935026343 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa96c003a50 0x7fa984143b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:38.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.492+0000 7fa98a400640 1 -- 172.21.15.151:0/2935026343 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa96c003a50 msgr2=0x7fa984143b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:38.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.492+0000 7fa98a400640 1 --2- 172.21.15.151:0/2935026343 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa96c003a50 0x7fa984143b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:38.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.492+0000 7fa98a400640 1 -- 172.21.15.151:0/2935026343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa984144350 msgr2=0x7fa984158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:38.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.492+0000 7fa98a400640 1 --2- 172.21.15.151:0/2935026343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa984144350 0x7fa984158b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:38.494 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.492+0000 7fa98a400640 1 -- 172.21.15.151:0/2935026343 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa984159520 con 0x7fa9841590a0 2024-09-16T15:54:38.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.492+0000 7fa98a400640 1 --2- 172.21.15.151:0/2935026343 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9841590a0 0x7fa9841f4b00 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fa97c005fd0 tx=0x7fa97c00fef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:38.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.493+0000 7fa981600640 1 -- 172.21.15.151:0/2935026343 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa97c00ddd0 con 0x7fa9841590a0 2024-09-16T15:54:38.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.493+0000 7fa981600640 1 -- 172.21.15.151:0/2935026343 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa97c017070 con 0x7fa9841590a0 2024-09-16T15:54:38.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.493+0000 7fa981600640 1 -- 172.21.15.151:0/2935026343 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa97c003970 con 0x7fa9841590a0 2024-09-16T15:54:38.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.493+0000 7fa98c400640 1 -- 172.21.15.151:0/2935026343 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa98415e2d0 con 0x7fa9841590a0 2024-09-16T15:54:38.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.493+0000 7fa98c400640 1 -- 172.21.15.151:0/2935026343 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa98415e890 con 0x7fa9841590a0 2024-09-16T15:54:38.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.494+0000 7fa98c400640 1 -- 172.21.15.151:0/2935026343 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9840cf0b0 con 0x7fa9841590a0 2024-09-16T15:54:38.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.495+0000 7fa981600640 1 -- 172.21.15.151:0/2935026343 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 49) ==== 104347+0+0 (secure 0 0 0) 0x7fa97c003b10 con 0x7fa9841590a0 2024-09-16T15:54:38.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.496+0000 7fa981600640 1 --2- 172.21.15.151:0/2935026343 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa96407aa70 0x7fa96407cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:38.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.496+0000 7fa981600640 1 -- 172.21.15.151:0/2935026343 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5821+0+0 (secure 0 0 0) 0x7fa97c099fd0 con 0x7fa9841590a0 2024-09-16T15:54:38.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.496+0000 7fa989a00640 1 --2- 172.21.15.151:0/2935026343 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa96407aa70 0x7fa96407cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:38.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.497+0000 7fa989a00640 1 --2- 172.21.15.151:0/2935026343 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa96407aa70 0x7fa96407cf30 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fa978033000 tx=0x7fa978038040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:38.501 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.500+0000 7fa981600640 1 -- 172.21.15.151:0/2935026343 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa97c063a50 con 0x7fa9841590a0 2024-09-16T15:54:38.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.720+0000 7fa98c400640 1 -- 172.21.15.151:0/2935026343 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa9841f58a0 con 0x7fa96407aa70 2024-09-16T15:54:38.756 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.755+0000 7fa981600640 1 -- 172.21.15.151:0/2935026343 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa9841f58a0 con 0x7fa96407aa70 2024-09-16T15:54:38.756 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:54:38.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.758+0000 7fa98c400640 1 -- 172.21.15.151:0/2935026343 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa96407aa70 msgr2=0x7fa96407cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:38.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.758+0000 7fa98c400640 1 --2- 172.21.15.151:0/2935026343 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa96407aa70 0x7fa96407cf30 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fa978033000 tx=0x7fa978038040 comp rx=0 tx=0).stop 2024-09-16T15:54:38.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.758+0000 7fa98c400640 1 -- 172.21.15.151:0/2935026343 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9841590a0 msgr2=0x7fa9841f4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:38.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.758+0000 7fa98c400640 1 --2- 172.21.15.151:0/2935026343 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9841590a0 0x7fa9841f4b00 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fa97c005fd0 tx=0x7fa97c00fef0 comp rx=0 tx=0).stop 2024-09-16T15:54:38.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.758+0000 7fa98c400640 1 -- 172.21.15.151:0/2935026343 shutdown_connections 2024-09-16T15:54:38.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.758+0000 7fa98c400640 1 --2- 172.21.15.151:0/2935026343 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa96407aa70 0x7fa96407cf30 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:38.759 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.758+0000 7fa98c400640 1 --2- 172.21.15.151:0/2935026343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa984144350 0x7fa984158b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:38.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.758+0000 7fa98c400640 1 --2- 172.21.15.151:0/2935026343 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9841590a0 0x7fa9841f4b00 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:38.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.758+0000 7fa98c400640 1 --2- 172.21.15.151:0/2935026343 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa96c003a50 0x7fa984143b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:38.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.758+0000 7fa98c400640 1 -- 172.21.15.151:0/2935026343 >> 172.21.15.151:0/2935026343 conn(0x7fa9840b1840 msgr2=0x7fa9840cdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:38.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.758+0000 7fa98c400640 1 -- 172.21.15.151:0/2935026343 shutdown_connections 2024-09-16T15:54:38.760 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.758+0000 7fa98c400640 1 -- 172.21.15.151:0/2935026343 wait complete. 2024-09-16T15:54:38.775 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-09-16T15:54:38.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.934+0000 7f9033e00640 1 -- 172.21.15.151:0/2883341329 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f902c145750 msgr2=0x7f902c073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:38.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.934+0000 7f9033e00640 1 --2- 172.21.15.151:0/2883341329 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f902c145750 0x7f902c073440 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f902c0cfb70 tx=0x7f902002a4a0 comp rx=0 tx=0).stop 2024-09-16T15:54:38.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.934+0000 7f9033e00640 1 -- 172.21.15.151:0/2883341329 shutdown_connections 2024-09-16T15:54:38.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.934+0000 7f9033e00640 1 --2- 172.21.15.151:0/2883341329 >> v2:172.21.15.167:3300/0 conn(0x7f902c0cccd0 0x7f902c0cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:38.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.934+0000 7f9033e00640 1 --2- 172.21.15.151:0/2883341329 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f902c145750 0x7f902c073440 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:38.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.934+0000 7f9033e00640 1 -- 172.21.15.151:0/2883341329 >> 172.21.15.151:0/2883341329 conn(0x7f902c0b1840 msgr2=0x7f902c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:38.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.934+0000 7f9033e00640 1 -- 172.21.15.151:0/2883341329 shutdown_connections 2024-09-16T15:54:38.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.934+0000 7f9033e00640 1 -- 172.21.15.151:0/2883341329 wait complete. 2024-09-16T15:54:38.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.936+0000 7f9033e00640 1 Processor -- start 2024-09-16T15:54:38.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.936+0000 7f9033e00640 1 -- start start 2024-09-16T15:54:38.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.937+0000 7f9033e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f902c06efa0 0x7f902c0cb490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:38.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.937+0000 7f9033e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f902c0cccd0 0x7f902c1f2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:38.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.937+0000 7f9033e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f902c145750 0x7f902c0ac550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:38.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.937+0000 7f9033e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f902c0ce480 con 0x7f902c145750 2024-09-16T15:54:38.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.937+0000 7f9033e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f902c0ce300 con 0x7f902c0cccd0 2024-09-16T15:54:38.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.937+0000 7f9033e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f902c0ce600 con 0x7f902c06efa0 2024-09-16T15:54:38.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.937+0000 7f9031e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f902c145750 0x7f902c0ac550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:38.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.937+0000 7f9031e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f902c145750 0x7f902c0ac550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41566/0 (socket says 172.21.15.151:41566) 2024-09-16T15:54:38.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.937+0000 7f9031e00640 1 -- 172.21.15.151:0/1734340328 learned_addr learned my addr 172.21.15.151:0/1734340328 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:54:38.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.937+0000 7f9031400640 1 --2- 172.21.15.151:0/1734340328 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f902c06efa0 0x7f902c0cb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:38.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.937+0000 7f9030a00640 1 --2- 172.21.15.151:0/1734340328 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f902c0cccd0 0x7f902c1f2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:38.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.937+0000 7f9031e00640 1 -- 172.21.15.151:0/1734340328 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f902c06efa0 msgr2=0x7f902c0cb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:38.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.937+0000 7f9031e00640 1 --2- 172.21.15.151:0/1734340328 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f902c06efa0 0x7f902c0cb490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:38.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.937+0000 7f9031e00640 1 -- 172.21.15.151:0/1734340328 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f902c0cccd0 msgr2=0x7f902c1f2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:38.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.937+0000 7f9031e00640 1 --2- 172.21.15.151:0/1734340328 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f902c0cccd0 0x7f902c1f2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:38.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.937+0000 7f9031e00640 1 -- 172.21.15.151:0/1734340328 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f902c140a10 con 0x7f902c145750 2024-09-16T15:54:38.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.938+0000 7f9031e00640 1 --2- 172.21.15.151:0/1734340328 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f902c145750 0x7f902c0ac550 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f902400eaa0 tx=0x7f902400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:38.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.938+0000 7f9029600640 1 -- 172.21.15.151:0/1734340328 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f902400cdb0 con 0x7f902c145750 2024-09-16T15:54:38.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.938+0000 7f9029600640 1 -- 172.21.15.151:0/1734340328 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9024004c10 con 0x7f902c145750 2024-09-16T15:54:38.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.938+0000 7f9029600640 1 -- 172.21.15.151:0/1734340328 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9024005020 con 0x7f902c145750 2024-09-16T15:54:38.940 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.938+0000 7f9033e00640 1 -- 172.21.15.151:0/1734340328 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f902c140cf0 con 0x7f902c145750 2024-09-16T15:54:38.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.938+0000 7f9033e00640 1 -- 172.21.15.151:0/1734340328 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f902c1412b0 con 0x7f902c145750 2024-09-16T15:54:38.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.939+0000 7f9033e00640 1 -- 172.21.15.151:0/1734340328 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f902c0cdbb0 con 0x7f902c145750 2024-09-16T15:54:38.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.941+0000 7f9029600640 1 -- 172.21.15.151:0/1734340328 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 49) ==== 104347+0+0 (secure 0 0 0) 0x7f90240051c0 con 0x7f902c145750 2024-09-16T15:54:38.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.941+0000 7f9029600640 1 --2- 172.21.15.151:0/1734340328 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f900c07aa10 0x7f900c07ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:38.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.942+0000 7f9029600640 1 -- 172.21.15.151:0/1734340328 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5821+0+0 (secure 0 0 0) 0x7f902409a420 con 0x7f902c145750 2024-09-16T15:54:38.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.942+0000 7f9031400640 1 --2- 172.21.15.151:0/1734340328 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f900c07aa10 0x7f900c07ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:38.944 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.943+0000 7f9031400640 1 --2- 172.21.15.151:0/1734340328 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f900c07aa10 0x7f900c07ced0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f9020034000 tx=0x7f9020004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:38.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:38.946+0000 7f9029600640 1 -- 172.21.15.151:0/1734340328 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9024016030 con 0x7f902c145750 2024-09-16T15:54:39.156 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:39.155+0000 7f9033e00640 1 -- 172.21.15.151:0/1734340328 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f902c144e00 con 0x7f900c07aa10 2024-09-16T15:54:39.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:39.181+0000 7f9029600640 1 -- 172.21.15.151:0/1734340328 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f902c144e00 con 0x7f900c07aa10 2024-09-16T15:54:39.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:39.183+0000 7f9033e00640 1 -- 172.21.15.151:0/1734340328 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f900c07aa10 msgr2=0x7f900c07ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:39.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:39.183+0000 7f9033e00640 1 --2- 172.21.15.151:0/1734340328 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f900c07aa10 0x7f900c07ced0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f9020034000 tx=0x7f9020004730 comp rx=0 tx=0).stop 2024-09-16T15:54:39.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:39.183+0000 7f9033e00640 1 -- 172.21.15.151:0/1734340328 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f902c145750 msgr2=0x7f902c0ac550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:39.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:39.183+0000 7f9033e00640 1 --2- 172.21.15.151:0/1734340328 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f902c145750 0x7f902c0ac550 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f902400eaa0 tx=0x7f902400ef60 comp rx=0 tx=0).stop 2024-09-16T15:54:39.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:39.183+0000 7f9033e00640 1 -- 172.21.15.151:0/1734340328 shutdown_connections 2024-09-16T15:54:39.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:39.183+0000 7f9033e00640 1 --2- 172.21.15.151:0/1734340328 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f900c07aa10 0x7f900c07ced0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:39.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:39.183+0000 7f9033e00640 1 --2- 172.21.15.151:0/1734340328 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f902c0cccd0 0x7f902c1f2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:39.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:39.183+0000 7f9033e00640 1 --2- 172.21.15.151:0/1734340328 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f902c145750 0x7f902c0ac550 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:39.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:39.183+0000 7f9033e00640 1 --2- 172.21.15.151:0/1734340328 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f902c06efa0 0x7f902c0cb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:39.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:39.183+0000 7f9033e00640 1 -- 172.21.15.151:0/1734340328 >> 172.21.15.151:0/1734340328 conn(0x7f902c0b1840 msgr2=0x7f902c1431d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:39.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:39.184+0000 7f9033e00640 1 -- 172.21.15.151:0/1734340328 shutdown_connections 2024-09-16T15:54:39.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:39.184+0000 7f9033e00640 1 -- 172.21.15.151:0/1734340328 wait complete. 2024-09-16T15:54:39.196 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:54:39.195+0000 7f48f1c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:54:39.196 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:54:39.195+0000 7f48f1c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:54:39.196 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:54:39.195+0000 7f48f1c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:54:39.196 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:54:39.195+0000 7f48f1c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:54:39.196 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:54:39.195+0000 7f48f1c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:54:39.201 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs ss1clone1 2024-09-16T15:54:39.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:39.364+0000 7fb733200640 1 -- 172.21.15.151:0/1874422220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7341611a0 msgr2=0x7fb7340d48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:39.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:39.364+0000 7fb733200640 1 --2- 172.21.15.151:0/1874422220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7341611a0 0x7fb7340d48f0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fb724009830 tx=0x7fb72402a5c0 comp rx=0 tx=0).stop 2024-09-16T15:54:39.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:39.364+0000 7fb733200640 1 -- 172.21.15.151:0/1874422220 shutdown_connections 2024-09-16T15:54:39.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:39.364+0000 7fb733200640 1 --2- 172.21.15.151:0/1874422220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7341611a0 0x7fb7340d48f0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:39.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:39.364+0000 7fb733200640 1 -- 172.21.15.151:0/1874422220 >> 172.21.15.151:0/1874422220 conn(0x7fb7340d3dd0 msgr2=0x7fb7340d41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:39.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:39.365+0000 7fb733200640 1 -- 172.21.15.151:0/1874422220 shutdown_connections 2024-09-16T15:54:39.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:39.365+0000 7fb733200640 1 -- 172.21.15.151:0/1874422220 wait complete. 2024-09-16T15:54:39.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:39.366+0000 7fb733200640 1 Processor -- start 2024-09-16T15:54:39.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:39.366+0000 7fb733200640 1 -- start start 2024-09-16T15:54:39.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:39.367+0000 7fb733200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb7340cfa90 0x7fb7340c8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:39.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:39.367+0000 7fb733200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7341611a0 0x7fb73415d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:39.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:39.367+0000 7fb733200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb734161580 0x7fb7340c9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:39.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:39.367+0000 7fb733200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb734169140 con 0x7fb7340cfa90 2024-09-16T15:54:39.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:39.367+0000 7fb733200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb734168fc0 con 0x7fb7341611a0 2024-09-16T15:54:39.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:39.367+0000 7fb733200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fb7341692c0 con 0x7fb734161580 2024-09-16T15:54:39.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:39.367+0000 7fb731e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb7340cfa90 0x7fb7340c8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:39.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:39.367+0000 7fb731e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb7340cfa90 0x7fb7340c8640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41576/0 (socket says 172.21.15.151:41576) 2024-09-16T15:54:39.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:39.367+0000 7fb731e00640 1 -- 172.21.15.151:0/1968332474 learned_addr learned my addr 172.21.15.151:0/1968332474 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:54:39.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:39.368+0000 7fb731400640 1 --2- 172.21.15.151:0/1968332474 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7341611a0 0x7fb73415d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:39.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:39.368+0000 7fb732800640 1 --2- 172.21.15.151:0/1968332474 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb734161580 0x7fb7340c9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:39.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:39.368+0000 7fb731e00640 1 -- 172.21.15.151:0/1968332474 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb734161580 msgr2=0x7fb7340c9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:39.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:39.368+0000 7fb731e00640 1 --2- 172.21.15.151:0/1968332474 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb734161580 0x7fb7340c9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:39.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:39.368+0000 7fb731e00640 1 -- 172.21.15.151:0/1968332474 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7341611a0 msgr2=0x7fb73415d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:39.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:39.368+0000 7fb731e00640 1 --2- 172.21.15.151:0/1968332474 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7341611a0 0x7fb73415d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:39.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:39.368+0000 7fb731e00640 1 -- 172.21.15.151:0/1968332474 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7340b8b80 con 0x7fb7340cfa90 2024-09-16T15:54:39.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:39.368+0000 7fb731e00640 1 --2- 172.21.15.151:0/1968332474 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb7340cfa90 0x7fb7340c8640 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fb724009780 tx=0x7fb724032b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:39.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:39.368+0000 7fb72a000640 1 -- 172.21.15.151:0/1968332474 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb724040070 con 0x7fb7340cfa90 2024-09-16T15:54:39.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:39.368+0000 7fb72a000640 1 -- 172.21.15.151:0/1968332474 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb724005310 con 0x7fb7340cfa90 2024-09-16T15:54:39.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:39.368+0000 7fb733200640 1 -- 172.21.15.151:0/1968332474 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb73415ee60 con 0x7fb7340cfa90 2024-09-16T15:54:39.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:39.369+0000 7fb72a000640 1 -- 172.21.15.151:0/1968332474 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7240047a0 con 0x7fb7340cfa90 2024-09-16T15:54:39.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:39.369+0000 7fb733200640 1 -- 172.21.15.151:0/1968332474 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7340b8850 con 0x7fb7340cfa90 2024-09-16T15:54:39.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:39.370+0000 7fb733200640 1 -- 172.21.15.151:0/1968332474 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7340d48f0 con 0x7fb7340cfa90 2024-09-16T15:54:39.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:39.372+0000 7fb72a000640 1 -- 172.21.15.151:0/1968332474 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 49) ==== 104347+0+0 (secure 0 0 0) 0x7fb724039040 con 0x7fb7340cfa90 2024-09-16T15:54:39.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:39.373+0000 7fb72a000640 1 --2- 172.21.15.151:0/1968332474 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb71407aa10 0x7fb71407ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:39.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:39.373+0000 7fb72a000640 1 -- 172.21.15.151:0/1968332474 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5821+0+0 (secure 0 0 0) 0x7fb7240baed0 con 0x7fb7340cfa90 2024-09-16T15:54:39.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:39.373+0000 7fb731400640 1 --2- 172.21.15.151:0/1968332474 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb71407aa10 0x7fb71407ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:39.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:39.374+0000 7fb731400640 1 --2- 172.21.15.151:0/1968332474 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb71407aa10 0x7fb71407ced0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fb720007fd0 tx=0x7fb720009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:39.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:39.376+0000 7fb72a000640 1 -- 172.21.15.151:0/1968332474 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb724085120 con 0x7fb7340cfa90 2024-09-16T15:54:39.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:39.600+0000 7fb733200640 1 -- 172.21.15.151:0/1968332474 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) -- 0x7fb7340b2580 con 0x7fb71407aa10 2024-09-16T15:54:39.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:39.900+0000 7fb72a000640 1 -- 172.21.15.151:0/1968332474 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb7340b2580 con 0x7fb71407aa10 2024-09-16T15:54:39.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:39.903+0000 7fb733200640 1 -- 172.21.15.151:0/1968332474 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb71407aa10 msgr2=0x7fb71407ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:39.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:39.903+0000 7fb733200640 1 --2- 172.21.15.151:0/1968332474 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb71407aa10 0x7fb71407ced0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fb720007fd0 tx=0x7fb720009040 comp rx=0 tx=0).stop 2024-09-16T15:54:39.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:39.903+0000 7fb733200640 1 -- 172.21.15.151:0/1968332474 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb7340cfa90 msgr2=0x7fb7340c8640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:39.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:39.903+0000 7fb733200640 1 --2- 172.21.15.151:0/1968332474 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb7340cfa90 0x7fb7340c8640 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fb724009780 tx=0x7fb724032b90 comp rx=0 tx=0).stop 2024-09-16T15:54:39.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:39.903+0000 7fb733200640 1 -- 172.21.15.151:0/1968332474 shutdown_connections 2024-09-16T15:54:39.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:39.903+0000 7fb733200640 1 --2- 172.21.15.151:0/1968332474 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb71407aa10 0x7fb71407ced0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:39.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:39.903+0000 7fb733200640 1 --2- 172.21.15.151:0/1968332474 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7341611a0 0x7fb73415d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:39.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:39.903+0000 7fb733200640 1 --2- 172.21.15.151:0/1968332474 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb734161580 0x7fb7340c9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:39.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:39.903+0000 7fb733200640 1 --2- 172.21.15.151:0/1968332474 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb7340cfa90 0x7fb7340c8640 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:39.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:39.903+0000 7fb733200640 1 -- 172.21.15.151:0/1968332474 >> 172.21.15.151:0/1968332474 conn(0x7fb7340d3dd0 msgr2=0x7fb7340d23b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:39.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:39.903+0000 7fb733200640 1 -- 172.21.15.151:0/1968332474 shutdown_connections 2024-09-16T15:54:39.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:39.903+0000 7fb733200640 1 -- 172.21.15.151:0/1968332474 wait complete. 2024-09-16T15:54:39.925 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-16T15:54:40.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:54:40.011 INFO:teuthology.orchestra.run.smithi151.stdout:2 2024-09-16T15:54:40.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:54:40.020 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-16T15:54:40.182 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.181+0000 7f77abe00640 1 -- 172.21.15.151:0/357741650 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f77a4145750 msgr2=0x7f77a4073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:40.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.181+0000 7f77abe00640 1 --2- 172.21.15.151:0/357741650 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f77a4145750 0x7f77a4073440 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f77a40af750 tx=0x7f779802a4d0 comp rx=0 tx=0).stop 2024-09-16T15:54:40.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.181+0000 7f77abe00640 1 -- 172.21.15.151:0/357741650 shutdown_connections 2024-09-16T15:54:40.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.181+0000 7f77abe00640 1 --2- 172.21.15.151:0/357741650 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f77a40ccb00 0x7f77a415f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:40.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.181+0000 7f77abe00640 1 --2- 172.21.15.151:0/357741650 >> v2:172.21.15.167:3300/0 conn(0x7f77a406efa0 0x7f77a40cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:40.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.181+0000 7f77abe00640 1 --2- 172.21.15.151:0/357741650 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f77a4145750 0x7f77a4073440 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:40.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.181+0000 7f77abe00640 1 -- 172.21.15.151:0/357741650 >> 172.21.15.151:0/357741650 conn(0x7f77a40b1840 msgr2=0x7f77a40b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:40.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.181+0000 7f77abe00640 1 -- 172.21.15.151:0/357741650 shutdown_connections 2024-09-16T15:54:40.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.181+0000 7f77abe00640 1 -- 172.21.15.151:0/357741650 wait complete. 2024-09-16T15:54:40.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.183+0000 7f77abe00640 1 Processor -- start 2024-09-16T15:54:40.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.183+0000 7f77abe00640 1 -- start start 2024-09-16T15:54:40.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.184+0000 7f77abe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f77a406efa0 0x7f77a40b4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:40.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.184+0000 7f77abe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77a40ccb00 0x7f77a41f52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:40.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.184+0000 7f77abe00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f77a4145750 0x7f77a41f7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:40.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.184+0000 7f77abe00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f77a41602c0 con 0x7f77a406efa0 2024-09-16T15:54:40.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.184+0000 7f77abe00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f77a4160140 con 0x7f77a40ccb00 2024-09-16T15:54:40.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.184+0000 7f77abe00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f77a4160440 con 0x7f77a4145750 2024-09-16T15:54:40.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.184+0000 7f77a9400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f77a406efa0 0x7f77a40b4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:40.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.184+0000 7f77a9400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f77a406efa0 0x7f77a40b4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41598/0 (socket says 172.21.15.151:41598) 2024-09-16T15:54:40.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.184+0000 7f77a9400640 1 -- 172.21.15.151:0/2684566683 learned_addr learned my addr 172.21.15.151:0/2684566683 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:54:40.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.184+0000 7f77a8a00640 1 --2- 172.21.15.151:0/2684566683 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77a40ccb00 0x7f77a41f52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:40.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.184+0000 7f77a9e00640 1 --2- 172.21.15.151:0/2684566683 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f77a4145750 0x7f77a41f7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:40.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.184+0000 7f77a9400640 1 -- 172.21.15.151:0/2684566683 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f77a4145750 msgr2=0x7f77a41f7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:40.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.184+0000 7f77a9400640 1 --2- 172.21.15.151:0/2684566683 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f77a4145750 0x7f77a41f7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:40.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.184+0000 7f77a9400640 1 -- 172.21.15.151:0/2684566683 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77a40ccb00 msgr2=0x7f77a41f52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:40.187 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.184+0000 7f77a9400640 1 --2- 172.21.15.151:0/2684566683 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77a40ccb00 0x7f77a41f52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:40.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.184+0000 7f77a9400640 1 -- 172.21.15.151:0/2684566683 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77a40b4c60 con 0x7f77a406efa0 2024-09-16T15:54:40.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.185+0000 7f77a9400640 1 --2- 172.21.15.151:0/2684566683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f77a406efa0 0x7f77a40b4720 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f7798034040 tx=0x7f7798003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:40.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.185+0000 7f77a1600640 1 -- 172.21.15.151:0/2684566683 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779803e030 con 0x7f77a406efa0 2024-09-16T15:54:40.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.185+0000 7f77a1600640 1 -- 172.21.15.151:0/2684566683 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7798003e00 con 0x7f77a406efa0 2024-09-16T15:54:40.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.185+0000 7f77a1600640 1 -- 172.21.15.151:0/2684566683 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7798035040 con 0x7f77a406efa0 2024-09-16T15:54:40.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.185+0000 7f77abe00640 1 -- 172.21.15.151:0/2684566683 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77a41f8890 con 0x7f77a406efa0 2024-09-16T15:54:40.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.185+0000 7f77abe00640 1 -- 172.21.15.151:0/2684566683 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77a41f4ad0 con 0x7f77a406efa0 2024-09-16T15:54:40.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.186+0000 7f77abe00640 1 -- 172.21.15.151:0/2684566683 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77a415f9f0 con 0x7f77a406efa0 2024-09-16T15:54:40.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.187+0000 7f77a1600640 1 -- 172.21.15.151:0/2684566683 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 49) ==== 104347+0+0 (secure 0 0 0) 0x7f7798041030 con 0x7f77a406efa0 2024-09-16T15:54:40.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.188+0000 7f77a1600640 1 --2- 172.21.15.151:0/2684566683 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f778407aa10 0x7f778407ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:40.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.188+0000 7f77a1600640 1 -- 172.21.15.151:0/2684566683 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5821+0+0 (secure 0 0 0) 0x7f77980ba0b0 con 0x7f77a406efa0 2024-09-16T15:54:40.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.189+0000 7f77a8a00640 1 --2- 172.21.15.151:0/2684566683 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f778407aa10 0x7f778407ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:40.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.190+0000 7f77a8a00640 1 --2- 172.21.15.151:0/2684566683 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f778407aa10 0x7f778407ced0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f77a40cc250 tx=0x7f779000b2e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:40.193 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.193+0000 7f77a1600640 1 -- 172.21.15.151:0/2684566683 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7798083b30 con 0x7f77a406efa0 2024-09-16T15:54:40.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.380+0000 7f77abe00640 1 -- 172.21.15.151:0/2684566683 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f77a406ed30 con 0x7f77a406efa0 2024-09-16T15:54:40.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.380+0000 7f77a1600640 1 -- 172.21.15.151:0/2684566683 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v52)=0 v52) ==== 131+0+0 (secure 0 0 0) 0x7f7798088560 con 0x7f77a406efa0 2024-09-16T15:54:40.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.383+0000 7f77abe00640 1 -- 172.21.15.151:0/2684566683 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f778407aa10 msgr2=0x7f778407ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:40.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.383+0000 7f77abe00640 1 --2- 172.21.15.151:0/2684566683 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f778407aa10 0x7f778407ced0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f77a40cc250 tx=0x7f779000b2e0 comp rx=0 tx=0).stop 2024-09-16T15:54:40.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.383+0000 7f77abe00640 1 -- 172.21.15.151:0/2684566683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f77a406efa0 msgr2=0x7f77a40b4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:40.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.383+0000 7f77abe00640 1 --2- 172.21.15.151:0/2684566683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f77a406efa0 0x7f77a40b4720 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f7798034040 tx=0x7f7798003910 comp rx=0 tx=0).stop 2024-09-16T15:54:40.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.383+0000 7f77abe00640 1 -- 172.21.15.151:0/2684566683 shutdown_connections 2024-09-16T15:54:40.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.383+0000 7f77abe00640 1 --2- 172.21.15.151:0/2684566683 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f778407aa10 0x7f778407ced0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:40.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.383+0000 7f77abe00640 1 --2- 172.21.15.151:0/2684566683 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77a40ccb00 0x7f77a41f52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:40.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.383+0000 7f77abe00640 1 --2- 172.21.15.151:0/2684566683 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f77a4145750 0x7f77a41f7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:40.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.383+0000 7f77abe00640 1 --2- 172.21.15.151:0/2684566683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f77a406efa0 0x7f77a40b4720 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:40.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.383+0000 7f77abe00640 1 -- 172.21.15.151:0/2684566683 >> 172.21.15.151:0/2684566683 conn(0x7f77a40b1840 msgr2=0x7f77a406f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:40.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.383+0000 7f77abe00640 1 -- 172.21.15.151:0/2684566683 shutdown_connections 2024-09-16T15:54:40.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.383+0000 7f77abe00640 1 -- 172.21.15.151:0/2684566683 wait complete. 2024-09-16T15:54:40.400 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-16T15:54:40.400 DEBUG:teuthology.orchestra.run.smithi151:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:54:40.494 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:54:40.586 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-16T15:54:40.586 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:54:40.586 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:54:40.586 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:54:40.622 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:54:40.623 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:54:40.699 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-16T15:54:40.904 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.903+0000 7f74ba600640 1 -- 172.21.15.151:0/1991174485 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7498004800 msgr2=0x7f7498004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:40.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.903+0000 7f74ba600640 1 --2- 172.21.15.151:0/1991174485 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7498004800 0x7f7498004c80 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f74a0006f60 tx=0x7f7498005830 comp rx=0 tx=0).stop 2024-09-16T15:54:40.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.903+0000 7f74b3e00640 1 -- 172.21.15.151:0/1991174485 reap_dead start 2024-09-16T15:54:40.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.903+0000 7f74ba600640 1 -- 172.21.15.151:0/1991174485 shutdown_connections 2024-09-16T15:54:40.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.903+0000 7f74ba600640 1 -- 172.21.15.151:0/1991174485 >> 172.21.15.151:0/1991174485 conn(0x7f74b40b1950 msgr2=0x7f74b40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:40.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.903+0000 7f74ba600640 1 -- 172.21.15.151:0/1991174485 shutdown_connections 2024-09-16T15:54:40.905 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.903+0000 7f74ba600640 1 -- 172.21.15.151:0/1991174485 wait complete. 2024-09-16T15:54:40.908 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.904+0000 7f74ba600640 1 Processor -- start 2024-09-16T15:54:40.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.905+0000 7f74ba600640 1 -- start start 2024-09-16T15:54:40.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.905+0000 7f74ba600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7498004800 0x7f74b41f1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:40.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.906+0000 7f74ba600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74b414c8f0 0x7f74b4156570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:40.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.906+0000 7f74ba600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74b415b980 0x7f74b415be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:40.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.906+0000 7f74ba600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f74b406f9f0 con 0x7f74b414c8f0 2024-09-16T15:54:40.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.906+0000 7f74ba600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f74b406f870 con 0x7f74b415b980 2024-09-16T15:54:40.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.906+0000 7f74ba600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f74b40b2440 con 0x7f7498004800 2024-09-16T15:54:40.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.906+0000 7f74b2a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74b414c8f0 0x7f74b4156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:40.909 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.906+0000 7f74b2a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74b414c8f0 0x7f74b4156570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41616/0 (socket says 172.21.15.151:41616) 2024-09-16T15:54:40.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.906+0000 7f74b2a00640 1 -- 172.21.15.151:0/3760936278 learned_addr learned my addr 172.21.15.151:0/3760936278 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:54:40.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.906+0000 7f74b3e00640 1 --2- 172.21.15.151:0/3760936278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74b415b980 0x7f74b415be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:40.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.906+0000 7f74b3400640 1 --2- 172.21.15.151:0/3760936278 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7498004800 0x7f74b41f1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:40.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.906+0000 7f74b2a00640 1 -- 172.21.15.151:0/3760936278 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7498004800 msgr2=0x7f74b41f1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:40.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.906+0000 7f74b2a00640 1 --2- 172.21.15.151:0/3760936278 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7498004800 0x7f74b41f1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:40.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.906+0000 7f74b2a00640 1 -- 172.21.15.151:0/3760936278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74b415b980 msgr2=0x7f74b415be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:40.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.906+0000 7f74b2a00640 1 --2- 172.21.15.151:0/3760936278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74b415b980 0x7f74b415be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:40.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.906+0000 7f74b2a00640 1 -- 172.21.15.151:0/3760936278 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74b4156ec0 con 0x7f74b414c8f0 2024-09-16T15:54:40.910 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.907+0000 7f74b2a00640 1 --2- 172.21.15.151:0/3760936278 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74b414c8f0 0x7f74b4156570 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f74a0006f60 tx=0x7f74a0001080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:40.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.907+0000 7f74ab400640 1 -- 172.21.15.151:0/3760936278 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a0032030 con 0x7f74b414c8f0 2024-09-16T15:54:40.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.907+0000 7f74ab400640 1 -- 172.21.15.151:0/3760936278 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f74a00038e0 con 0x7f74b414c8f0 2024-09-16T15:54:40.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.907+0000 7f74ab400640 1 -- 172.21.15.151:0/3760936278 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a0003d70 con 0x7f74b414c8f0 2024-09-16T15:54:40.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.907+0000 7f74ba600640 1 -- 172.21.15.151:0/3760936278 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74b41571a0 con 0x7f74b414c8f0 2024-09-16T15:54:40.911 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.907+0000 7f74ba600640 1 -- 172.21.15.151:0/3760936278 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74b4157760 con 0x7f74b414c8f0 2024-09-16T15:54:40.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.908+0000 7f74ba600640 1 -- 172.21.15.151:0/3760936278 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74b4157f60 con 0x7f74b414c8f0 2024-09-16T15:54:40.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.909+0000 7f74ab400640 1 -- 172.21.15.151:0/3760936278 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 50) ==== 104398+0+0 (secure 0 0 0) 0x7f74a0035020 con 0x7f74b414c8f0 2024-09-16T15:54:40.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.910+0000 7f74ab400640 1 --2- 172.21.15.151:0/3760936278 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f749407aaa0 0x7f749407cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:40.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.910+0000 7f74ab400640 1 -- 172.21.15.151:0/3760936278 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5821+0+0 (secure 0 0 0) 0x7f74a00ba640 con 0x7f74b414c8f0 2024-09-16T15:54:40.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.911+0000 7f74b3400640 1 --2- 172.21.15.151:0/3760936278 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f749407aaa0 0x7f749407cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:40.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.911+0000 7f74b3400640 1 --2- 172.21.15.151:0/3760936278 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f749407aaa0 0x7f749407cf60 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f74b4075130 tx=0x7f74a4009500 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:40.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:40.914+0000 7f74ab400640 1 -- 172.21.15.151:0/3760936278 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f74b4157f60 con 0x7f74b414c8f0 2024-09-16T15:54:41.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.113+0000 7f74ba600640 1 -- 172.21.15.151:0/3760936278 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f74b4074460 con 0x7f74b414c8f0 2024-09-16T15:54:41.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.113+0000 7f74ab400640 1 -- 172.21.15.151:0/3760936278 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v133) ==== 80+0+0 (secure 0 0 0) 0x7f74a0047060 con 0x7f74b414c8f0 2024-09-16T15:54:41.115 INFO:teuthology.orchestra.run.smithi151.stderr:listed 0 entries 2024-09-16T15:54:41.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.116+0000 7f74ba600640 1 -- 172.21.15.151:0/3760936278 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f749407aaa0 msgr2=0x7f749407cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:41.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.116+0000 7f74ba600640 1 --2- 172.21.15.151:0/3760936278 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f749407aaa0 0x7f749407cf60 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f74b4075130 tx=0x7f74a4009500 comp rx=0 tx=0).stop 2024-09-16T15:54:41.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.116+0000 7f74ba600640 1 -- 172.21.15.151:0/3760936278 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74b414c8f0 msgr2=0x7f74b4156570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:41.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.116+0000 7f74ba600640 1 --2- 172.21.15.151:0/3760936278 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74b414c8f0 0x7f74b4156570 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f74a0006f60 tx=0x7f74a0001080 comp rx=0 tx=0).stop 2024-09-16T15:54:41.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.116+0000 7f74ba600640 1 -- 172.21.15.151:0/3760936278 shutdown_connections 2024-09-16T15:54:41.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.116+0000 7f74ba600640 1 --2- 172.21.15.151:0/3760936278 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f749407aaa0 0x7f749407cf60 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:41.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.116+0000 7f74ba600640 1 --2- 172.21.15.151:0/3760936278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74b415b980 0x7f74b415be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:41.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.116+0000 7f74ba600640 1 --2- 172.21.15.151:0/3760936278 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74b414c8f0 0x7f74b4156570 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:41.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.116+0000 7f74ba600640 1 --2- 172.21.15.151:0/3760936278 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7498004800 0x7f74b41f1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:41.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.116+0000 7f74ba600640 1 -- 172.21.15.151:0/3760936278 >> 172.21.15.151:0/3760936278 conn(0x7f74b40b1950 msgr2=0x7f74b40cdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:41.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.116+0000 7f74ba600640 1 -- 172.21.15.151:0/3760936278 shutdown_connections 2024-09-16T15:54:41.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.116+0000 7f74ba600640 1 -- 172.21.15.151:0/3760936278 wait complete. 2024-09-16T15:54:41.133 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-16T15:54:41.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:54:41.134 DEBUG:teuthology.orchestra.run.smithi151:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-16T15:54:41.222 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi151.front.sepia.ceph.com 2024-09-16T15:54:41.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:54:41.222 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-16T15:54:41.270 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:54:41.271 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-16T15:54:41.271 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:54:41.308 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:54:41.309 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:54:41.309 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:54:41.309 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:54:41.346 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T15:54:41.347 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:54:41.347 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-16T15:54:41.347 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:54:41.385 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:54:41.385 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:54:41.549 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.547+0000 7ff9e0a00640 1 -- 172.21.15.151:0/2282905443 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff9c0004d40 msgr2=0x7ff9c00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:41.549 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.547+0000 7ff9e0a00640 1 --2- 172.21.15.151:0/2282905443 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff9c0004d40 0x7ff9c00051c0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4009b50 tx=0x7ff9c0005830 comp rx=0 tx=0).stop 2024-09-16T15:54:41.549 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.547+0000 7ff9db200640 1 -- 172.21.15.151:0/2282905443 reap_dead start 2024-09-16T15:54:41.549 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.547+0000 7ff9e0a00640 1 -- 172.21.15.151:0/2282905443 shutdown_connections 2024-09-16T15:54:41.549 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.547+0000 7ff9e0a00640 1 -- 172.21.15.151:0/2282905443 >> 172.21.15.151:0/2282905443 conn(0x7ff9dc0d3de0 msgr2=0x7ff9dc0d41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:41.549 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.547+0000 7ff9e0a00640 1 -- 172.21.15.151:0/2282905443 shutdown_connections 2024-09-16T15:54:41.549 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.547+0000 7ff9e0a00640 1 -- 172.21.15.151:0/2282905443 wait complete. 2024-09-16T15:54:41.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.549+0000 7ff9e0a00640 1 Processor -- start 2024-09-16T15:54:41.550 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.549+0000 7ff9e0a00640 1 -- start start 2024-09-16T15:54:41.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.550+0000 7ff9e0a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff9c0004d40 0x7ff9dc154c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:41.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.550+0000 7ff9e0a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff9dc1553b0 0x7ff9dc0b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:41.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.550+0000 7ff9e0a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff9dc1f8110 0x7ff9dc1f44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:41.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.550+0000 7ff9e0a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff9dc162c80 con 0x7ff9c0004d40 2024-09-16T15:54:41.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.550+0000 7ff9e0a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff9dc162b00 con 0x7ff9dc1f8110 2024-09-16T15:54:41.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.550+0000 7ff9e0a00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7ff9dc162e00 con 0x7ff9dc1553b0 2024-09-16T15:54:41.553 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.550+0000 7ff9da800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff9c0004d40 0x7ff9dc154c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:41.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.550+0000 7ff9da800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff9c0004d40 0x7ff9dc154c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41642/0 (socket says 172.21.15.151:41642) 2024-09-16T15:54:41.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.550+0000 7ff9da800640 1 -- 172.21.15.151:0/473167003 learned_addr learned my addr 172.21.15.151:0/473167003 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:54:41.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.551+0000 7ff9db200640 1 --2- 172.21.15.151:0/473167003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff9dc1f8110 0x7ff9dc1f44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:41.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.551+0000 7ff9d9e00640 1 --2- 172.21.15.151:0/473167003 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff9dc1553b0 0x7ff9dc0b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:41.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.551+0000 7ff9da800640 1 -- 172.21.15.151:0/473167003 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff9dc1553b0 msgr2=0x7ff9dc0b6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:41.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.551+0000 7ff9da800640 1 --2- 172.21.15.151:0/473167003 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff9dc1553b0 0x7ff9dc0b6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:41.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.551+0000 7ff9da800640 1 -- 172.21.15.151:0/473167003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff9dc1f8110 msgr2=0x7ff9dc1f44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:41.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.551+0000 7ff9da800640 1 --2- 172.21.15.151:0/473167003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff9dc1f8110 0x7ff9dc1f44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:41.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.551+0000 7ff9da800640 1 -- 172.21.15.151:0/473167003 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9dc155150 con 0x7ff9c0004d40 2024-09-16T15:54:41.554 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.551+0000 7ff9d9e00640 1 --2- 172.21.15.151:0/473167003 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff9dc1553b0 0x7ff9dc0b6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:54:41.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.551+0000 7ff9da800640 1 --2- 172.21.15.151:0/473167003 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff9c0004d40 0x7ff9dc154c10 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc006380 tx=0x7ff9cc006840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:41.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.551+0000 7ff9d2a00640 1 -- 172.21.15.151:0/473167003 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc00fc00 con 0x7ff9c0004d40 2024-09-16T15:54:41.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.551+0000 7ff9d2a00640 1 -- 172.21.15.151:0/473167003 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff9cc016020 con 0x7ff9c0004d40 2024-09-16T15:54:41.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.551+0000 7ff9e0a00640 1 -- 172.21.15.151:0/473167003 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9dc0b97c0 con 0x7ff9c0004d40 2024-09-16T15:54:41.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.551+0000 7ff9d2a00640 1 -- 172.21.15.151:0/473167003 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc011430 con 0x7ff9c0004d40 2024-09-16T15:54:41.555 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.551+0000 7ff9e0a00640 1 -- 172.21.15.151:0/473167003 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9dc0b9490 con 0x7ff9c0004d40 2024-09-16T15:54:41.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.552+0000 7ff9e0a00640 1 -- 172.21.15.151:0/473167003 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9dc0b9ce0 con 0x7ff9c0004d40 2024-09-16T15:54:41.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.554+0000 7ff9d2a00640 1 -- 172.21.15.151:0/473167003 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 50) ==== 104398+0+0 (secure 0 0 0) 0x7ff9cc0115d0 con 0x7ff9c0004d40 2024-09-16T15:54:41.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.555+0000 7ff9d2a00640 1 --2- 172.21.15.151:0/473167003 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff9bc07aad0 0x7ff9bc07cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:41.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.555+0000 7ff9d2a00640 1 -- 172.21.15.151:0/473167003 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5821+0+0 (secure 0 0 0) 0x7ff9cc09a9d0 con 0x7ff9c0004d40 2024-09-16T15:54:41.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.555+0000 7ff9d9e00640 1 --2- 172.21.15.151:0/473167003 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff9bc07aad0 0x7ff9bc07cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:41.557 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.556+0000 7ff9d9e00640 1 --2- 172.21.15.151:0/473167003 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff9bc07aad0 0x7ff9bc07cf90 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc0b4500 tx=0x7ff9c8005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:41.560 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.559+0000 7ff9d2a00640 1 -- 172.21.15.151:0/473167003 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7ff9dc0b9ce0 con 0x7ff9c0004d40 2024-09-16T15:54:41.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.780+0000 7ff9e0a00640 1 -- 172.21.15.151:0/473167003 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff9dc06eb50 con 0x7ff9c0004d40 2024-09-16T15:54:41.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.780+0000 7ff9d2a00640 1 -- 172.21.15.151:0/473167003 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) ==== 94+0+5234 (secure 0 0 0) 0x7ff9dc06eb50 con 0x7ff9c0004d40 2024-09-16T15:54:41.782 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:54:41.782 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":88,"btime":"2024-09-16T15:54:00:531362+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8994,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/2450183746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":2450183746},{"type":"v1","addr":"172.21.15.151:6833","nonce":2450183746}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8997,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1952630294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1952630294},{"type":"v1","addr":"172.21.15.167:6836","nonce":1952630294}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":9000,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/127383056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":127383056},{"type":"v1","addr":"172.21.15.167:6837","nonce":127383056}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:53:46.461837+0000","modified":"2024-09-16T15:53:49.478024+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9009},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9009":{"gid":9009,"name":"a","rank":0,"incarnation":83,"state":"up:active","state_seq":3,"addr":"172.21.15.151:6835/1977294668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":1977294668},{"type":"v1","addr":"172.21.15.151:6835","nonce":1977294668}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9009,"qdb_cluster":[9009]},"id":9}]} 2024-09-16T15:54:41.782 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 88 2024-09-16T15:54:41.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.783+0000 7ff9e0a00640 1 -- 172.21.15.151:0/473167003 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff9bc07aad0 msgr2=0x7ff9bc07cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:41.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.783+0000 7ff9e0a00640 1 --2- 172.21.15.151:0/473167003 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff9bc07aad0 0x7ff9bc07cf90 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc0b4500 tx=0x7ff9c8005f90 comp rx=0 tx=0).stop 2024-09-16T15:54:41.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.783+0000 7ff9e0a00640 1 -- 172.21.15.151:0/473167003 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff9c0004d40 msgr2=0x7ff9dc154c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:41.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.783+0000 7ff9e0a00640 1 --2- 172.21.15.151:0/473167003 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff9c0004d40 0x7ff9dc154c10 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc006380 tx=0x7ff9cc006840 comp rx=0 tx=0).stop 2024-09-16T15:54:41.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.783+0000 7ff9e0a00640 1 -- 172.21.15.151:0/473167003 shutdown_connections 2024-09-16T15:54:41.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.783+0000 7ff9e0a00640 1 --2- 172.21.15.151:0/473167003 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7ff9bc07aad0 0x7ff9bc07cf90 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:41.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.783+0000 7ff9e0a00640 1 --2- 172.21.15.151:0/473167003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff9dc1f8110 0x7ff9dc1f44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:41.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.783+0000 7ff9e0a00640 1 --2- 172.21.15.151:0/473167003 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff9dc1553b0 0x7ff9dc0b6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:41.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.783+0000 7ff9e0a00640 1 --2- 172.21.15.151:0/473167003 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff9c0004d40 0x7ff9dc154c10 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:41.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.783+0000 7ff9e0a00640 1 -- 172.21.15.151:0/473167003 >> 172.21.15.151:0/473167003 conn(0x7ff9dc0d3de0 msgr2=0x7ff9dc06e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:41.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.783+0000 7ff9e0a00640 1 -- 172.21.15.151:0/473167003 shutdown_connections 2024-09-16T15:54:41.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.783+0000 7ff9e0a00640 1 -- 172.21.15.151:0/473167003 wait complete. 2024-09-16T15:54:41.804 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:54:41.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.962+0000 7f024ec00640 1 -- 172.21.15.151:0/725265345 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02480725d0 msgr2=0x7f02480b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:41.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.962+0000 7f024ec00640 1 --2- 172.21.15.151:0/725265345 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02480725d0 0x7f02480b28f0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f0234009830 tx=0x7f023402a5c0 comp rx=0 tx=0).stop 2024-09-16T15:54:41.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.962+0000 7f024ec00640 1 -- 172.21.15.151:0/725265345 shutdown_connections 2024-09-16T15:54:41.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.962+0000 7f024ec00640 1 --2- 172.21.15.151:0/725265345 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f02480725d0 0x7f02480b28f0 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:41.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.962+0000 7f024ec00640 1 -- 172.21.15.151:0/725265345 >> 172.21.15.151:0/725265345 conn(0x7f02480cfa20 msgr2=0x7f02480cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:41.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.962+0000 7f024ec00640 1 -- 172.21.15.151:0/725265345 shutdown_connections 2024-09-16T15:54:41.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.962+0000 7f024ec00640 1 -- 172.21.15.151:0/725265345 wait complete. 2024-09-16T15:54:41.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.964+0000 7f024ec00640 1 Processor -- start 2024-09-16T15:54:41.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.964+0000 7f024ec00640 1 -- start start 2024-09-16T15:54:41.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.965+0000 7f024ec00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f02480d0540 0x7f02480b0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:41.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.965+0000 7f024ec00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f024815bc30 0x7f02480cce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:41.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.965+0000 7f024ec00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f02480cd350 0x7f02481f7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:41.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.965+0000 7f024ec00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0248074660 con 0x7f024815bc30 2024-09-16T15:54:41.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.965+0000 7f024ec00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f024806fa30 con 0x7f02480d0540 2024-09-16T15:54:41.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.965+0000 7f024ec00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f02480b1470 con 0x7f02480cd350 2024-09-16T15:54:41.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.965+0000 7f024ce00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f024815bc30 0x7f02480cce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:41.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.965+0000 7f024ce00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f024815bc30 0x7f02480cce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41658/0 (socket says 172.21.15.151:41658) 2024-09-16T15:54:41.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.965+0000 7f024ce00640 1 -- 172.21.15.151:0/4096519021 learned_addr learned my addr 172.21.15.151:0/4096519021 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:54:41.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.965+0000 7f024d800640 1 --2- 172.21.15.151:0/4096519021 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f02480d0540 0x7f02480b0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:41.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.966+0000 7f024e200640 1 --2- 172.21.15.151:0/4096519021 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f02480cd350 0x7f02481f7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:41.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.966+0000 7f024ce00640 1 -- 172.21.15.151:0/4096519021 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f02480cd350 msgr2=0x7f02481f7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:41.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.966+0000 7f024ce00640 1 --2- 172.21.15.151:0/4096519021 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f02480cd350 0x7f02481f7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:41.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.966+0000 7f024ce00640 1 -- 172.21.15.151:0/4096519021 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f02480d0540 msgr2=0x7f02480b0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:41.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.966+0000 7f024ce00640 1 --2- 172.21.15.151:0/4096519021 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f02480d0540 0x7f02480b0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:41.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.966+0000 7f024ce00640 1 -- 172.21.15.151:0/4096519021 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02480cd890 con 0x7f024815bc30 2024-09-16T15:54:41.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.966+0000 7f024ce00640 1 --2- 172.21.15.151:0/4096519021 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f024815bc30 0x7f02480cce10 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f0234009950 tx=0x7f0234032b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:41.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.966+0000 7f0245600640 1 -- 172.21.15.151:0/4096519021 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0234040070 con 0x7f024815bc30 2024-09-16T15:54:41.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.966+0000 7f0245600640 1 -- 172.21.15.151:0/4096519021 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0234005310 con 0x7f024815bc30 2024-09-16T15:54:41.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.966+0000 7f0245600640 1 -- 172.21.15.151:0/4096519021 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02340047a0 con 0x7f024815bc30 2024-09-16T15:54:41.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.966+0000 7f024ec00640 1 -- 172.21.15.151:0/4096519021 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02480cdb70 con 0x7f024815bc30 2024-09-16T15:54:41.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.966+0000 7f024ec00640 1 -- 172.21.15.151:0/4096519021 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02481f8460 con 0x7f024815bc30 2024-09-16T15:54:41.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.967+0000 7f024ec00640 1 -- 172.21.15.151:0/4096519021 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02480b28f0 con 0x7f024815bc30 2024-09-16T15:54:41.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.969+0000 7f0245600640 1 -- 172.21.15.151:0/4096519021 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 50) ==== 104398+0+0 (secure 0 0 0) 0x7f0234039040 con 0x7f024815bc30 2024-09-16T15:54:41.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.970+0000 7f0245600640 1 --2- 172.21.15.151:0/4096519021 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f022807aaa0 0x7f022807cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:41.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.970+0000 7f0245600640 1 -- 172.21.15.151:0/4096519021 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5877+0+0 (secure 0 0 0) 0x7f02340babf0 con 0x7f024815bc30 2024-09-16T15:54:41.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.970+0000 7f024d800640 1 --2- 172.21.15.151:0/4096519021 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f022807aaa0 0x7f022807cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:41.972 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.971+0000 7f024d800640 1 --2- 172.21.15.151:0/4096519021 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f022807aaa0 0x7f022807cf60 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f023c000f60 tx=0x7f023c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:41.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:41.973+0000 7f0245600640 1 -- 172.21.15.151:0/4096519021 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0234084d40 con 0x7f024815bc30 2024-09-16T15:54:42.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.321+0000 7f024ec00640 1 -- 172.21.15.151:0/4096519021 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f02480cdd40 con 0x7f024815bc30 2024-09-16T15:54:42.323 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.322+0000 7f0245600640 1 -- 172.21.15.151:0/4096519021 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) ==== 94+0+5234 (secure 0 0 0) 0x7f0234084640 con 0x7f024815bc30 2024-09-16T15:54:42.325 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:54:42.325 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":88,"btime":"2024-09-16T15:54:00:531362+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8994,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/2450183746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":2450183746},{"type":"v1","addr":"172.21.15.151:6833","nonce":2450183746}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8997,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1952630294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1952630294},{"type":"v1","addr":"172.21.15.167:6836","nonce":1952630294}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":9000,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/127383056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":127383056},{"type":"v1","addr":"172.21.15.167:6837","nonce":127383056}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:53:46.461837+0000","modified":"2024-09-16T15:53:49.478024+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9009},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9009":{"gid":9009,"name":"a","rank":0,"incarnation":83,"state":"up:active","state_seq":3,"addr":"172.21.15.151:6835/1977294668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":1977294668},{"type":"v1","addr":"172.21.15.151:6835","nonce":1977294668}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9009,"qdb_cluster":[9009]},"id":9}]} 2024-09-16T15:54:42.325 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 88 2024-09-16T15:54:42.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.326+0000 7f024ec00640 1 -- 172.21.15.151:0/4096519021 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f022807aaa0 msgr2=0x7f022807cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:42.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.326+0000 7f024ec00640 1 --2- 172.21.15.151:0/4096519021 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f022807aaa0 0x7f022807cf60 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f023c000f60 tx=0x7f023c009040 comp rx=0 tx=0).stop 2024-09-16T15:54:42.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.326+0000 7f024ec00640 1 -- 172.21.15.151:0/4096519021 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f024815bc30 msgr2=0x7f02480cce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:42.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.326+0000 7f024ec00640 1 --2- 172.21.15.151:0/4096519021 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f024815bc30 0x7f02480cce10 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f0234009950 tx=0x7f0234032b90 comp rx=0 tx=0).stop 2024-09-16T15:54:42.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.327+0000 7f024ec00640 1 -- 172.21.15.151:0/4096519021 shutdown_connections 2024-09-16T15:54:42.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.327+0000 7f024ec00640 1 --2- 172.21.15.151:0/4096519021 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f022807aaa0 0x7f022807cf60 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:42.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.327+0000 7f024ec00640 1 --2- 172.21.15.151:0/4096519021 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f02480cd350 0x7f02481f7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:42.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.327+0000 7f024ec00640 1 --2- 172.21.15.151:0/4096519021 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f024815bc30 0x7f02480cce10 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:42.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.327+0000 7f024ec00640 1 --2- 172.21.15.151:0/4096519021 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f02480d0540 0x7f02480b0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:42.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.327+0000 7f024ec00640 1 -- 172.21.15.151:0/4096519021 >> 172.21.15.151:0/4096519021 conn(0x7f02480cfa20 msgr2=0x7f024806e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:42.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.328+0000 7f024ec00640 1 -- 172.21.15.151:0/4096519021 shutdown_connections 2024-09-16T15:54:42.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.328+0000 7f024ec00640 1 -- 172.21.15.151:0/4096519021 wait complete. 2024-09-16T15:54:42.355 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:54:42.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.563+0000 7fb970a00640 1 -- 172.21.15.151:0/2828204763 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb96c0725d0 msgr2=0x7fb96c073d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:42.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.563+0000 7fb970a00640 1 --2- 172.21.15.151:0/2828204763 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb96c0725d0 0x7fb96c073d40 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fb96c0cd570 tx=0x7fb95802a5f0 comp rx=0 tx=0).stop 2024-09-16T15:54:42.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.564+0000 7fb970a00640 1 -- 172.21.15.151:0/2828204763 shutdown_connections 2024-09-16T15:54:42.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.564+0000 7fb970a00640 1 --2- 172.21.15.151:0/2828204763 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb96c0725d0 0x7fb96c073d40 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:42.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.564+0000 7fb970a00640 1 --2- 172.21.15.151:0/2828204763 >> v2:172.21.15.167:3300/0 conn(0x7fb96c15bc30 0x7fb96c0b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:42.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.564+0000 7fb970a00640 1 -- 172.21.15.151:0/2828204763 >> 172.21.15.151:0/2828204763 conn(0x7fb96c0cfa20 msgr2=0x7fb96c0cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:42.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.564+0000 7fb970a00640 1 -- 172.21.15.151:0/2828204763 shutdown_connections 2024-09-16T15:54:42.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.564+0000 7fb970a00640 1 -- 172.21.15.151:0/2828204763 wait complete. 2024-09-16T15:54:42.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.565+0000 7fb970a00640 1 Processor -- start 2024-09-16T15:54:42.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.565+0000 7fb970a00640 1 -- start start 2024-09-16T15:54:42.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.566+0000 7fb970a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb96c0d0540 0x7fb96c153730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:42.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.566+0000 7fb970a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb96c15bc30 0x7fb96c1f23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:42.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.566+0000 7fb970a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb96c150f50 0x7fb96c151710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:42.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.566+0000 7fb970a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb96c074610 con 0x7fb96c150f50 2024-09-16T15:54:42.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.566+0000 7fb970a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb96c074490 con 0x7fb96c15bc30 2024-09-16T15:54:42.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.566+0000 7fb970a00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fb96c074790 con 0x7fb96c0d0540 2024-09-16T15:54:42.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.567+0000 7fb96be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb96c150f50 0x7fb96c151710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:42.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.567+0000 7fb96aa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb96c15bc30 0x7fb96c1f23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:42.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.567+0000 7fb96b400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb96c0d0540 0x7fb96c153730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:42.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.567+0000 7fb96be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb96c150f50 0x7fb96c151710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:41676/0 (socket says 172.21.15.151:41676) 2024-09-16T15:54:42.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.567+0000 7fb96be00640 1 -- 172.21.15.151:0/3344120505 learned_addr learned my addr 172.21.15.151:0/3344120505 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:54:42.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.567+0000 7fb96be00640 1 -- 172.21.15.151:0/3344120505 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb96c0d0540 msgr2=0x7fb96c153730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:42.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.567+0000 7fb96be00640 1 --2- 172.21.15.151:0/3344120505 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb96c0d0540 0x7fb96c153730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:42.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.567+0000 7fb96be00640 1 -- 172.21.15.151:0/3344120505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb96c15bc30 msgr2=0x7fb96c1f23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:42.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.567+0000 7fb96be00640 1 --2- 172.21.15.151:0/3344120505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb96c15bc30 0x7fb96c1f23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:42.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.567+0000 7fb96be00640 1 -- 172.21.15.151:0/3344120505 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb96c153c70 con 0x7fb96c150f50 2024-09-16T15:54:42.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.567+0000 7fb96be00640 1 --2- 172.21.15.151:0/3344120505 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb96c150f50 0x7fb96c151710 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7fb96400c7a0 tx=0x7fb96400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:42.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.568+0000 7fb963400640 1 -- 172.21.15.151:0/3344120505 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb964003b70 con 0x7fb96c150f50 2024-09-16T15:54:42.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.568+0000 7fb963400640 1 -- 172.21.15.151:0/3344120505 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb964003d10 con 0x7fb96c150f50 2024-09-16T15:54:42.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.568+0000 7fb963400640 1 -- 172.21.15.151:0/3344120505 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96400aab0 con 0x7fb96c150f50 2024-09-16T15:54:42.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.568+0000 7fb970a00640 1 -- 172.21.15.151:0/3344120505 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb96c154790 con 0x7fb96c150f50 2024-09-16T15:54:42.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.568+0000 7fb970a00640 1 -- 172.21.15.151:0/3344120505 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb96c153ef0 con 0x7fb96c150f50 2024-09-16T15:54:42.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.569+0000 7fb970a00640 1 -- 172.21.15.151:0/3344120505 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb96c06fd20 con 0x7fb96c150f50 2024-09-16T15:54:42.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.570+0000 7fb963400640 1 -- 172.21.15.151:0/3344120505 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 50) ==== 104398+0+0 (secure 0 0 0) 0x7fb96400ac50 con 0x7fb96c150f50 2024-09-16T15:54:42.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.571+0000 7fb963400640 1 --2- 172.21.15.151:0/3344120505 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb94c07aaa0 0x7fb94c07cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:42.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.571+0000 7fb963400640 1 -- 172.21.15.151:0/3344120505 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5877+0+0 (secure 0 0 0) 0x7fb964099860 con 0x7fb96c150f50 2024-09-16T15:54:42.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.571+0000 7fb96b400640 1 --2- 172.21.15.151:0/3344120505 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb94c07aaa0 0x7fb94c07cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:42.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.572+0000 7fb96b400640 1 --2- 172.21.15.151:0/3344120505 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb94c07aaa0 0x7fb94c07cf60 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fb95c004630 tx=0x7fb95c00b2e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:42.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.575+0000 7fb963400640 1 -- 172.21.15.151:0/3344120505 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb96c153c70 con 0x7fb96c150f50 2024-09-16T15:54:42.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.762+0000 7fb970a00640 1 -- 172.21.15.151:0/3344120505 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb96c157bb0 con 0x7fb96c150f50 2024-09-16T15:54:42.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.763+0000 7fb963400640 1 -- 172.21.15.151:0/3344120505 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) ==== 74+0+17036 (secure 0 0 0) 0x7fb9640632e0 con 0x7fb96c150f50 2024-09-16T15:54:42.765 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:54:42.765 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":134,"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","created":"2024-09-16T15:38:56.348535+0000","modified":"2024-09-16T15:54:41.780024+0000","last_up_change":"2024-09-16T15:39:10.681384+0000","last_in_change":"2024-09-16T15:39:00.544477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:39:11.602252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:53:44.212522+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-16T15:53:45.415072+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":134,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5524076-a903-4d0a-af6a-caf474c4765d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6809","nonce":3111342741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6811","nonce":3111342741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6815","nonce":3111342741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6813","nonce":3111342741}]},"public_addr":"172.21.15.151:6809/3111342741","cluster_addr":"172.21.15.151:6811/3111342741","heartbeat_back_addr":"172.21.15.151:6815/3111342741","heartbeat_front_addr":"172.21.15.151:6813/3111342741","state":["exists","up"]},{"osd":1,"uuid":"dd25925f-0ae1-494e-8529-4dc205759db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6801","nonce":3368304087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6803","nonce":3368304087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6807","nonce":3368304087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6805","nonce":3368304087}]},"public_addr":"172.21.15.151:6801/3368304087","cluster_addr":"172.21.15.151:6803/3368304087","heartbeat_back_addr":"172.21.15.151:6807/3368304087","heartbeat_front_addr":"172.21.15.151:6805/3368304087","state":["exists","up"]},{"osd":2,"uuid":"c2f18115-013d-4da6-a0fe-cb978f4494b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6823","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6825","nonce":3079373271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6827","nonce":3079373271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6831","nonce":3079373271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6829","nonce":3079373271}]},"public_addr":"172.21.15.151:6825/3079373271","cluster_addr":"172.21.15.151:6827/3079373271","heartbeat_back_addr":"172.21.15.151:6831/3079373271","heartbeat_front_addr":"172.21.15.151:6829/3079373271","state":["exists","up"]},{"osd":3,"uuid":"36edb718-2fcb-4d88-8ddc-c321eed4279d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6817","nonce":1142509041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6819","nonce":1142509041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6824","nonce":1142509041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6821","nonce":1142509041}]},"public_addr":"172.21.15.151:6817/1142509041","cluster_addr":"172.21.15.151:6819/1142509041","heartbeat_back_addr":"172.21.15.151:6824/1142509041","heartbeat_front_addr":"172.21.15.151:6821/1142509041","state":["exists","up"]},{"osd":4,"uuid":"64d33708-cd0a-4236-abe3-1b646e82ab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6811","nonce":2735495271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6813","nonce":2735495271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6817","nonce":2735495271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6815","nonce":2735495271}]},"public_addr":"172.21.15.167:6811/2735495271","cluster_addr":"172.21.15.167:6813/2735495271","heartbeat_back_addr":"172.21.15.167:6817/2735495271","heartbeat_front_addr":"172.21.15.167:6815/2735495271","state":["exists","up"]},{"osd":5,"uuid":"f14b44a9-5910-458c-b518-09a8d0de94de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6827","nonce":1494916572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6829","nonce":1494916572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6833","nonce":1494916572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6831","nonce":1494916572}]},"public_addr":"172.21.15.167:6827/1494916572","cluster_addr":"172.21.15.167:6829/1494916572","heartbeat_back_addr":"172.21.15.167:6833/1494916572","heartbeat_front_addr":"172.21.15.167:6831/1494916572","state":["exists","up"]},{"osd":6,"uuid":"39335545-2654-4710-84b0-a9aa47cfcbba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6819","nonce":2538273273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6821","nonce":2538273273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6825","nonce":2538273273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6823","nonce":2538273273}]},"public_addr":"172.21.15.167:6819/2538273273","cluster_addr":"172.21.15.167:6821/2538273273","heartbeat_back_addr":"172.21.15.167:6825/2538273273","heartbeat_front_addr":"172.21.15.167:6823/2538273273","state":["exists","up"]},{"osd":7,"uuid":"4f8bfc81-53dd-4339-ac22-1a4007d9f963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6803","nonce":265072227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6805","nonce":265072227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6809","nonce":265072227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6807","nonce":265072227}]},"public_addr":"172.21.15.167:6803/265072227","cluster_addr":"172.21.15.167:6805/265072227","heartbeat_back_addr":"172.21.15.167:6809/265072227","heartbeat_front_addr":"172.21.15.167:6807/265072227","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.150359+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.084358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.722368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.332809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.320709+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.395153+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.935250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.801443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":19,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:54:42.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.765+0000 7fb970a00640 1 -- 172.21.15.151:0/3344120505 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb94c07aaa0 msgr2=0x7fb94c07cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:42.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.765+0000 7fb970a00640 1 --2- 172.21.15.151:0/3344120505 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb94c07aaa0 0x7fb94c07cf60 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fb95c004630 tx=0x7fb95c00b2e0 comp rx=0 tx=0).stop 2024-09-16T15:54:42.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.765+0000 7fb970a00640 1 -- 172.21.15.151:0/3344120505 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb96c150f50 msgr2=0x7fb96c151710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:42.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.765+0000 7fb970a00640 1 --2- 172.21.15.151:0/3344120505 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb96c150f50 0x7fb96c151710 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7fb96400c7a0 tx=0x7fb96400cc60 comp rx=0 tx=0).stop 2024-09-16T15:54:42.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.766+0000 7fb970a00640 1 -- 172.21.15.151:0/3344120505 shutdown_connections 2024-09-16T15:54:42.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.766+0000 7fb970a00640 1 --2- 172.21.15.151:0/3344120505 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb94c07aaa0 0x7fb94c07cf60 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:42.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.766+0000 7fb970a00640 1 --2- 172.21.15.151:0/3344120505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb96c15bc30 0x7fb96c1f23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:42.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.766+0000 7fb970a00640 1 --2- 172.21.15.151:0/3344120505 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb96c150f50 0x7fb96c151710 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:42.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.766+0000 7fb970a00640 1 --2- 172.21.15.151:0/3344120505 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb96c0d0540 0x7fb96c153730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:42.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.766+0000 7fb970a00640 1 -- 172.21.15.151:0/3344120505 >> 172.21.15.151:0/3344120505 conn(0x7fb96c0cfa20 msgr2=0x7fb96c0cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:42.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.766+0000 7fb970a00640 1 -- 172.21.15.151:0/3344120505 shutdown_connections 2024-09-16T15:54:42.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.766+0000 7fb970a00640 1 -- 172.21.15.151:0/3344120505 wait complete. 2024-09-16T15:54:42.783 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-16T15:54:42.784 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:54:42.941 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.940+0000 7fac6d800640 1 -- 172.21.15.151:0/1673298604 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac681612d0 msgr2=0x7fac680b4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:42.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.940+0000 7fac6d800640 1 --2- 172.21.15.151:0/1673298604 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac681612d0 0x7fac680b4e00 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fac58005fd0 tx=0x7fac5802a820 comp rx=0 tx=0).stop 2024-09-16T15:54:42.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.940+0000 7fac6d800640 1 -- 172.21.15.151:0/1673298604 shutdown_connections 2024-09-16T15:54:42.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.940+0000 7fac6d800640 1 --2- 172.21.15.151:0/1673298604 >> v2:172.21.15.167:3300/0 conn(0x7fac681616b0 0x7fac680d2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:42.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.940+0000 7fac6d800640 1 --2- 172.21.15.151:0/1673298604 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac681612d0 0x7fac680b4e00 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:42.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.940+0000 7fac6d800640 1 -- 172.21.15.151:0/1673298604 >> 172.21.15.151:0/1673298604 conn(0x7fac680d3df0 msgr2=0x7fac680d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:42.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.940+0000 7fac6d800640 1 -- 172.21.15.151:0/1673298604 shutdown_connections 2024-09-16T15:54:42.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.940+0000 7fac6d800640 1 -- 172.21.15.151:0/1673298604 wait complete. 2024-09-16T15:54:42.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.941+0000 7fac6d800640 1 Processor -- start 2024-09-16T15:54:42.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.942+0000 7fac6d800640 1 -- start start 2024-09-16T15:54:42.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.942+0000 7fac6d800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac680d4980 0x7fac680cd150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:42.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.942+0000 7fac6d800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac681612d0 0x7fac6815de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:42.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.943+0000 7fac6d800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac681616b0 0x7fac6815e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:42.945 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.943+0000 7fac6d800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fac680b1c50 con 0x7fac681612d0 2024-09-16T15:54:42.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.943+0000 7fac6d800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fac680b1ad0 con 0x7fac680d4980 2024-09-16T15:54:42.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.943+0000 7fac6d800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fac680b1dd0 con 0x7fac681616b0 2024-09-16T15:54:42.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.943+0000 7fac67400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac681612d0 0x7fac6815de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:42.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.943+0000 7fac67400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac681612d0 0x7fac6815de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44174/0 (socket says 172.21.15.151:44174) 2024-09-16T15:54:42.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.943+0000 7fac67400640 1 -- 172.21.15.151:0/2631622687 learned_addr learned my addr 172.21.15.151:0/2631622687 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:54:42.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.943+0000 7fac67e00640 1 --2- 172.21.15.151:0/2631622687 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac680d4980 0x7fac680cd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:42.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.943+0000 7fac67400640 1 -- 172.21.15.151:0/2631622687 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac681616b0 msgr2=0x7fac6815e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:42.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.943+0000 7fac6ce00640 1 --2- 172.21.15.151:0/2631622687 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac681616b0 0x7fac6815e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:42.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.943+0000 7fac67400640 1 --2- 172.21.15.151:0/2631622687 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac681616b0 0x7fac6815e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:42.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.943+0000 7fac67400640 1 -- 172.21.15.151:0/2631622687 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac680d4980 msgr2=0x7fac680cd150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:42.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.943+0000 7fac67400640 1 --2- 172.21.15.151:0/2631622687 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac680d4980 0x7fac680cd150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:42.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.943+0000 7fac67400640 1 -- 172.21.15.151:0/2631622687 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac680cc100 con 0x7fac681612d0 2024-09-16T15:54:42.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.943+0000 7fac6ce00640 1 --2- 172.21.15.151:0/2631622687 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac681616b0 0x7fac6815e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:54:42.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.944+0000 7fac67400640 1 --2- 172.21.15.151:0/2631622687 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac681612d0 0x7fac6815de50 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7fac5400c730 tx=0x7fac5400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:42.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.944+0000 7fac5fe00640 1 -- 172.21.15.151:0/2631622687 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac54006a90 con 0x7fac681612d0 2024-09-16T15:54:42.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.944+0000 7fac5fe00640 1 -- 172.21.15.151:0/2631622687 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fac54006c30 con 0x7fac681612d0 2024-09-16T15:54:42.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.944+0000 7fac6d800640 1 -- 172.21.15.151:0/2631622687 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac680cc3e0 con 0x7fac681612d0 2024-09-16T15:54:42.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.944+0000 7fac5fe00640 1 -- 172.21.15.151:0/2631622687 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac54006e60 con 0x7fac681612d0 2024-09-16T15:54:42.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.944+0000 7fac6d800640 1 -- 172.21.15.151:0/2631622687 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac681580e0 con 0x7fac681612d0 2024-09-16T15:54:42.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.945+0000 7fac6d800640 1 -- 172.21.15.151:0/2631622687 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac680b2360 con 0x7fac681612d0 2024-09-16T15:54:42.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.946+0000 7fac5fe00640 1 -- 172.21.15.151:0/2631622687 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 50) ==== 104398+0+0 (secure 0 0 0) 0x7fac54007290 con 0x7fac681612d0 2024-09-16T15:54:42.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.947+0000 7fac5fe00640 1 --2- 172.21.15.151:0/2631622687 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fac4807aad0 0x7fac4807cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:42.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.947+0000 7fac5fe00640 1 -- 172.21.15.151:0/2631622687 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5877+0+0 (secure 0 0 0) 0x7fac540996f0 con 0x7fac681612d0 2024-09-16T15:54:42.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.948+0000 7fac67e00640 1 --2- 172.21.15.151:0/2631622687 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fac4807aad0 0x7fac4807cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:42.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.948+0000 7fac67e00640 1 --2- 172.21.15.151:0/2631622687 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fac4807aad0 0x7fac4807cf90 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fac58034000 tx=0x7fac58004ab0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:42.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:42.951+0000 7fac5fe00640 1 -- 172.21.15.151:0/2631622687 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fac680d2cc0 con 0x7fac681612d0 2024-09-16T15:54:43.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.176+0000 7fac6d800640 1 -- 172.21.15.151:0/2631622687 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fac6815e9a0 con 0x7fac681612d0 2024-09-16T15:54:43.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.176+0000 7fac5fe00640 1 -- 172.21.15.151:0/2631622687 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) ==== 94+0+5234 (secure 0 0 0) 0x7fac54063740 con 0x7fac681612d0 2024-09-16T15:54:43.178 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:54:43.178 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":88,"btime":"2024-09-16T15:54:00:531362+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8994,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/2450183746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":2450183746},{"type":"v1","addr":"172.21.15.151:6833","nonce":2450183746}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8997,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1952630294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1952630294},{"type":"v1","addr":"172.21.15.167:6836","nonce":1952630294}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":9000,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/127383056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":127383056},{"type":"v1","addr":"172.21.15.167:6837","nonce":127383056}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:53:46.461837+0000","modified":"2024-09-16T15:53:49.478024+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9009},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9009":{"gid":9009,"name":"a","rank":0,"incarnation":83,"state":"up:active","state_seq":3,"addr":"172.21.15.151:6835/1977294668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":1977294668},{"type":"v1","addr":"172.21.15.151:6835","nonce":1977294668}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9009,"qdb_cluster":[9009]},"id":9}]} 2024-09-16T15:54:43.178 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 88 2024-09-16T15:54:43.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.179+0000 7fac6d800640 1 -- 172.21.15.151:0/2631622687 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fac4807aad0 msgr2=0x7fac4807cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:43.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.179+0000 7fac6d800640 1 --2- 172.21.15.151:0/2631622687 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fac4807aad0 0x7fac4807cf90 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fac58034000 tx=0x7fac58004ab0 comp rx=0 tx=0).stop 2024-09-16T15:54:43.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.179+0000 7fac6d800640 1 -- 172.21.15.151:0/2631622687 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac681612d0 msgr2=0x7fac6815de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:43.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.179+0000 7fac6d800640 1 --2- 172.21.15.151:0/2631622687 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac681612d0 0x7fac6815de50 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7fac5400c730 tx=0x7fac5400cbf0 comp rx=0 tx=0).stop 2024-09-16T15:54:43.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.179+0000 7fac6d800640 1 -- 172.21.15.151:0/2631622687 shutdown_connections 2024-09-16T15:54:43.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.179+0000 7fac6d800640 1 --2- 172.21.15.151:0/2631622687 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fac4807aad0 0x7fac4807cf90 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:43.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.179+0000 7fac6d800640 1 --2- 172.21.15.151:0/2631622687 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fac681616b0 0x7fac6815e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:43.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.179+0000 7fac6d800640 1 --2- 172.21.15.151:0/2631622687 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fac681612d0 0x7fac6815de50 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:43.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.179+0000 7fac6d800640 1 --2- 172.21.15.151:0/2631622687 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac680d4980 0x7fac680cd150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:43.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.179+0000 7fac6d800640 1 -- 172.21.15.151:0/2631622687 >> 172.21.15.151:0/2631622687 conn(0x7fac680d3df0 msgr2=0x7fac6806d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:43.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.179+0000 7fac6d800640 1 -- 172.21.15.151:0/2631622687 shutdown_connections 2024-09-16T15:54:43.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.179+0000 7fac6d800640 1 -- 172.21.15.151:0/2631622687 wait complete. 2024-09-16T15:54:43.196 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:54:43.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.353+0000 7f78eb400640 1 -- 172.21.15.151:0/1981289010 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f78e40b4240 msgr2=0x7f78e40730c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:43.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.353+0000 7f78eb400640 1 --2- 172.21.15.151:0/1981289010 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f78e40b4240 0x7f78e40730c0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f78d0009830 tx=0x7f78d002a510 comp rx=0 tx=0).stop 2024-09-16T15:54:43.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.353+0000 7f78eb400640 1 -- 172.21.15.151:0/1981289010 shutdown_connections 2024-09-16T15:54:43.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.353+0000 7f78eb400640 1 --2- 172.21.15.151:0/1981289010 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f78e40b4240 0x7f78e40730c0 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:43.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.353+0000 7f78eb400640 1 --2- 172.21.15.151:0/1981289010 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f78e415bdc0 0x7f78e40b3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:43.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.353+0000 7f78eb400640 1 -- 172.21.15.151:0/1981289010 >> 172.21.15.151:0/1981289010 conn(0x7f78e40b2230 msgr2=0x7f78e40b2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:43.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.353+0000 7f78eb400640 1 -- 172.21.15.151:0/1981289010 shutdown_connections 2024-09-16T15:54:43.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.353+0000 7f78eb400640 1 -- 172.21.15.151:0/1981289010 wait complete. 2024-09-16T15:54:43.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.355+0000 7f78eb400640 1 Processor -- start 2024-09-16T15:54:43.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.355+0000 7f78eb400640 1 -- start start 2024-09-16T15:54:43.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.356+0000 7f78eb400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f78e4073600 0x7f78e4153760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:43.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.356+0000 7f78eb400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78e40b4240 0x7f78e41f51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:43.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.356+0000 7f78eb400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f78e415bdc0 0x7f78e40ae2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:43.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.356+0000 7f78eb400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f78e4151e60 con 0x7f78e415bdc0 2024-09-16T15:54:43.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.356+0000 7f78eb400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f78e415c3b0 con 0x7f78e40b4240 2024-09-16T15:54:43.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.356+0000 7f78eb400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f78e4151fe0 con 0x7f78e4073600 2024-09-16T15:54:43.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.356+0000 7f78e9400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f78e415bdc0 0x7f78e40ae2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:43.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.356+0000 7f78e8a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f78e4073600 0x7f78e4153760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:43.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.356+0000 7f78e9400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f78e415bdc0 0x7f78e40ae2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44196/0 (socket says 172.21.15.151:44196) 2024-09-16T15:54:43.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.356+0000 7f78e9400640 1 -- 172.21.15.151:0/3247751419 learned_addr learned my addr 172.21.15.151:0/3247751419 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:54:43.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.356+0000 7f78e3e00640 1 --2- 172.21.15.151:0/3247751419 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78e40b4240 0x7f78e41f51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:43.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.357+0000 7f78e9400640 1 -- 172.21.15.151:0/3247751419 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f78e4073600 msgr2=0x7f78e4153760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:43.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.357+0000 7f78e9400640 1 --2- 172.21.15.151:0/3247751419 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f78e4073600 0x7f78e4153760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:43.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.357+0000 7f78e9400640 1 -- 172.21.15.151:0/3247751419 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78e40b4240 msgr2=0x7f78e41f51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:43.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.357+0000 7f78e9400640 1 --2- 172.21.15.151:0/3247751419 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78e40b4240 0x7f78e41f51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:43.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.357+0000 7f78e9400640 1 -- 172.21.15.151:0/3247751419 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78e4153dd0 con 0x7f78e415bdc0 2024-09-16T15:54:43.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.357+0000 7f78e9400640 1 --2- 172.21.15.151:0/3247751419 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f78e415bdc0 0x7f78e40ae2f0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f78dc00c430 tx=0x7f78dc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:43.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.357+0000 7f78e0c00640 1 -- 172.21.15.151:0/3247751419 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78dc011790 con 0x7f78e415bdc0 2024-09-16T15:54:43.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.357+0000 7f78e0c00640 1 -- 172.21.15.151:0/3247751419 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f78dc017070 con 0x7f78e415bdc0 2024-09-16T15:54:43.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.357+0000 7f78e0c00640 1 -- 172.21.15.151:0/3247751419 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78dc011ce0 con 0x7f78e415bdc0 2024-09-16T15:54:43.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.357+0000 7f78eb400640 1 -- 172.21.15.151:0/3247751419 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78e41540b0 con 0x7f78e415bdc0 2024-09-16T15:54:43.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.358+0000 7f78eb400640 1 -- 172.21.15.151:0/3247751419 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78e4154670 con 0x7f78e415bdc0 2024-09-16T15:54:43.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.359+0000 7f78eb400640 1 -- 172.21.15.151:0/3247751419 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78e40730c0 con 0x7f78e415bdc0 2024-09-16T15:54:43.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.360+0000 7f78e0c00640 1 -- 172.21.15.151:0/3247751419 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 50) ==== 104398+0+0 (secure 0 0 0) 0x7f78dc01c460 con 0x7f78e415bdc0 2024-09-16T15:54:43.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.361+0000 7f78e0c00640 1 --2- 172.21.15.151:0/3247751419 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f78c407aaa0 0x7f78c407cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:43.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.361+0000 7f78e0c00640 1 -- 172.21.15.151:0/3247751419 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5877+0+0 (secure 0 0 0) 0x7f78dc099640 con 0x7f78e415bdc0 2024-09-16T15:54:43.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.362+0000 7f78e8a00640 1 --2- 172.21.15.151:0/3247751419 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f78c407aaa0 0x7f78c407cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:43.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.363+0000 7f78e8a00640 1 --2- 172.21.15.151:0/3247751419 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f78c407aaa0 0x7f78c407cf60 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f78d4005fd0 tx=0x7f78d4009500 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:43.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.366+0000 7f78e0c00640 1 -- 172.21.15.151:0/3247751419 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f78dc063160 con 0x7f78e415bdc0 2024-09-16T15:54:43.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.587+0000 7f78eb400640 1 -- 172.21.15.151:0/3247751419 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f78e41554b0 con 0x7f78e415bdc0 2024-09-16T15:54:43.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.587+0000 7f78e0c00640 1 -- 172.21.15.151:0/3247751419 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) ==== 94+0+5234 (secure 0 0 0) 0x7f78dc067b90 con 0x7f78e415bdc0 2024-09-16T15:54:43.589 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:54:43.589 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":88,"btime":"2024-09-16T15:54:00:531362+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8994,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/2450183746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":2450183746},{"type":"v1","addr":"172.21.15.151:6833","nonce":2450183746}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8997,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1952630294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1952630294},{"type":"v1","addr":"172.21.15.167:6836","nonce":1952630294}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":9000,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/127383056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":127383056},{"type":"v1","addr":"172.21.15.167:6837","nonce":127383056}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:53:46.461837+0000","modified":"2024-09-16T15:53:49.478024+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9009},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9009":{"gid":9009,"name":"a","rank":0,"incarnation":83,"state":"up:active","state_seq":3,"addr":"172.21.15.151:6835/1977294668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":1977294668},{"type":"v1","addr":"172.21.15.151:6835","nonce":1977294668}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9009,"qdb_cluster":[9009]},"id":9}]} 2024-09-16T15:54:43.589 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 88 2024-09-16T15:54:43.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.589+0000 7f78eb400640 1 -- 172.21.15.151:0/3247751419 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f78c407aaa0 msgr2=0x7f78c407cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:43.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.589+0000 7f78eb400640 1 --2- 172.21.15.151:0/3247751419 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f78c407aaa0 0x7f78c407cf60 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f78d4005fd0 tx=0x7f78d4009500 comp rx=0 tx=0).stop 2024-09-16T15:54:43.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.589+0000 7f78eb400640 1 -- 172.21.15.151:0/3247751419 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f78e415bdc0 msgr2=0x7f78e40ae2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:43.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.589+0000 7f78eb400640 1 --2- 172.21.15.151:0/3247751419 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f78e415bdc0 0x7f78e40ae2f0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f78dc00c430 tx=0x7f78dc00c8f0 comp rx=0 tx=0).stop 2024-09-16T15:54:43.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.590+0000 7f78eb400640 1 -- 172.21.15.151:0/3247751419 shutdown_connections 2024-09-16T15:54:43.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.590+0000 7f78eb400640 1 --2- 172.21.15.151:0/3247751419 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f78c407aaa0 0x7f78c407cf60 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:43.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.590+0000 7f78eb400640 1 --2- 172.21.15.151:0/3247751419 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f78e40b4240 0x7f78e41f51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:43.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.590+0000 7f78eb400640 1 --2- 172.21.15.151:0/3247751419 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f78e415bdc0 0x7f78e40ae2f0 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:43.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.590+0000 7f78eb400640 1 --2- 172.21.15.151:0/3247751419 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f78e4073600 0x7f78e4153760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:43.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.590+0000 7f78eb400640 1 -- 172.21.15.151:0/3247751419 >> 172.21.15.151:0/3247751419 conn(0x7f78e40b2230 msgr2=0x7f78e406f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:43.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.590+0000 7f78eb400640 1 -- 172.21.15.151:0/3247751419 shutdown_connections 2024-09-16T15:54:43.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.590+0000 7f78eb400640 1 -- 172.21.15.151:0/3247751419 wait complete. 2024-09-16T15:54:43.607 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:54:43.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.765+0000 7fbb9e000640 1 -- 172.21.15.151:0/1737107721 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb7c003a30 msgr2=0x7fbb7c003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:43.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.765+0000 7fbb9e000640 1 --2- 172.21.15.151:0/1737107721 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb7c003a30 0x7fbb7c003e50 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fbb8800f610 tx=0x7fbb7c005830 comp rx=0 tx=0).stop 2024-09-16T15:54:43.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.765+0000 7fbb97e00640 1 -- 172.21.15.151:0/1737107721 reap_dead start 2024-09-16T15:54:43.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.765+0000 7fbb9e000640 1 -- 172.21.15.151:0/1737107721 shutdown_connections 2024-09-16T15:54:43.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.765+0000 7fbb9e000640 1 -- 172.21.15.151:0/1737107721 >> 172.21.15.151:0/1737107721 conn(0x7fbb980b1950 msgr2=0x7fbb980b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:43.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.765+0000 7fbb9e000640 1 -- 172.21.15.151:0/1737107721 shutdown_connections 2024-09-16T15:54:43.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.765+0000 7fbb9e000640 1 -- 172.21.15.151:0/1737107721 wait complete. 2024-09-16T15:54:43.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.767+0000 7fbb9e000640 1 Processor -- start 2024-09-16T15:54:43.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.767+0000 7fbb9e000640 1 -- start start 2024-09-16T15:54:43.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.768+0000 7fbb9e000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb7c003a30 0x7fbb981497e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:43.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.768+0000 7fbb9e000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbb98146210 0x7fbb9815c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:43.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.768+0000 7fbb9e000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb98146670 0x7fbb9815cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:43.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.768+0000 7fbb9e000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbb980746a0 con 0x7fbb7c003a30 2024-09-16T15:54:43.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.768+0000 7fbb9e000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9815dd20 con 0x7fbb98146670 2024-09-16T15:54:43.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.768+0000 7fbb9e000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fbb98074820 con 0x7fbb98146210 2024-09-16T15:54:43.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.768+0000 7fbb97400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb7c003a30 0x7fbb981497e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:43.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.768+0000 7fbb97400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb7c003a30 0x7fbb981497e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44210/0 (socket says 172.21.15.151:44210) 2024-09-16T15:54:43.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.768+0000 7fbb97400640 1 -- 172.21.15.151:0/2033385529 learned_addr learned my addr 172.21.15.151:0/2033385529 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:54:43.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.768+0000 7fbb97400640 1 -- 172.21.15.151:0/2033385529 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbb98146210 msgr2=0x7fbb9815c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:43.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.768+0000 7fbb97400640 1 --2- 172.21.15.151:0/2033385529 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbb98146210 0x7fbb9815c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:43.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.768+0000 7fbb97400640 1 -- 172.21.15.151:0/2033385529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb98146670 msgr2=0x7fbb9815cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:43.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.768+0000 7fbb97400640 1 --2- 172.21.15.151:0/2033385529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb98146670 0x7fbb9815cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:43.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.768+0000 7fbb97400640 1 -- 172.21.15.151:0/2033385529 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb9815d590 con 0x7fbb7c003a30 2024-09-16T15:54:43.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.769+0000 7fbb97400640 1 --2- 172.21.15.151:0/2033385529 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb7c003a30 0x7fbb981497e0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fbb8802cb70 tx=0x7fbb88033de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:43.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.769+0000 7fbb8f400640 1 -- 172.21.15.151:0/2033385529 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb88041070 con 0x7fbb7c003a30 2024-09-16T15:54:43.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.769+0000 7fbb8f400640 1 -- 172.21.15.151:0/2033385529 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbb88033e50 con 0x7fbb7c003a30 2024-09-16T15:54:43.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.769+0000 7fbb9e000640 1 -- 172.21.15.151:0/2033385529 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb9815d2b0 con 0x7fbb7c003a30 2024-09-16T15:54:43.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.769+0000 7fbb8f400640 1 -- 172.21.15.151:0/2033385529 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb88037030 con 0x7fbb7c003a30 2024-09-16T15:54:43.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.769+0000 7fbb9e000640 1 -- 172.21.15.151:0/2033385529 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb98148fe0 con 0x7fbb7c003a30 2024-09-16T15:54:43.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.770+0000 7fbb9e000640 1 -- 172.21.15.151:0/2033385529 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb980b4900 con 0x7fbb7c003a30 2024-09-16T15:54:43.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.771+0000 7fbb8f400640 1 -- 172.21.15.151:0/2033385529 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 50) ==== 104398+0+0 (secure 0 0 0) 0x7fbb8803d670 con 0x7fbb7c003a30 2024-09-16T15:54:43.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.772+0000 7fbb8f400640 1 --2- 172.21.15.151:0/2033385529 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbb7807aaa0 0x7fbb7807cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:43.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.772+0000 7fbb8f400640 1 -- 172.21.15.151:0/2033385529 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5877+0+0 (secure 0 0 0) 0x7fbb880bb6e0 con 0x7fbb7c003a30 2024-09-16T15:54:43.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.773+0000 7fbb96a00640 1 --2- 172.21.15.151:0/2033385529 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbb7807aaa0 0x7fbb7807cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:43.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.773+0000 7fbb96a00640 1 --2- 172.21.15.151:0/2033385529 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbb7807aaa0 0x7fbb7807cf60 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fbb84000fb0 tx=0x7fbb84001d90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:43.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.776+0000 7fbb8f400640 1 -- 172.21.15.151:0/2033385529 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fbb88085850 con 0x7fbb7c003a30 2024-09-16T15:54:43.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.963+0000 7fbb9e000640 1 -- 172.21.15.151:0/2033385529 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbb981eac90 con 0x7fbb7c003a30 2024-09-16T15:54:43.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.964+0000 7fbb8f400640 1 -- 172.21.15.151:0/2033385529 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) ==== 74+0+17036 (secure 0 0 0) 0x7fbb88085150 con 0x7fbb7c003a30 2024-09-16T15:54:43.966 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:54:43.966 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":134,"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","created":"2024-09-16T15:38:56.348535+0000","modified":"2024-09-16T15:54:41.780024+0000","last_up_change":"2024-09-16T15:39:10.681384+0000","last_in_change":"2024-09-16T15:39:00.544477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:39:11.602252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:53:44.212522+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-16T15:53:45.415072+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":134,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5524076-a903-4d0a-af6a-caf474c4765d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6809","nonce":3111342741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6811","nonce":3111342741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6815","nonce":3111342741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6813","nonce":3111342741}]},"public_addr":"172.21.15.151:6809/3111342741","cluster_addr":"172.21.15.151:6811/3111342741","heartbeat_back_addr":"172.21.15.151:6815/3111342741","heartbeat_front_addr":"172.21.15.151:6813/3111342741","state":["exists","up"]},{"osd":1,"uuid":"dd25925f-0ae1-494e-8529-4dc205759db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6801","nonce":3368304087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6803","nonce":3368304087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6807","nonce":3368304087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6805","nonce":3368304087}]},"public_addr":"172.21.15.151:6801/3368304087","cluster_addr":"172.21.15.151:6803/3368304087","heartbeat_back_addr":"172.21.15.151:6807/3368304087","heartbeat_front_addr":"172.21.15.151:6805/3368304087","state":["exists","up"]},{"osd":2,"uuid":"c2f18115-013d-4da6-a0fe-cb978f4494b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6823","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6825","nonce":3079373271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6827","nonce":3079373271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6831","nonce":3079373271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6829","nonce":3079373271}]},"public_addr":"172.21.15.151:6825/3079373271","cluster_addr":"172.21.15.151:6827/3079373271","heartbeat_back_addr":"172.21.15.151:6831/3079373271","heartbeat_front_addr":"172.21.15.151:6829/3079373271","state":["exists","up"]},{"osd":3,"uuid":"36edb718-2fcb-4d88-8ddc-c321eed4279d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6817","nonce":1142509041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6819","nonce":1142509041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6824","nonce":1142509041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6821","nonce":1142509041}]},"public_addr":"172.21.15.151:6817/1142509041","cluster_addr":"172.21.15.151:6819/1142509041","heartbeat_back_addr":"172.21.15.151:6824/1142509041","heartbeat_front_addr":"172.21.15.151:6821/1142509041","state":["exists","up"]},{"osd":4,"uuid":"64d33708-cd0a-4236-abe3-1b646e82ab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6811","nonce":2735495271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6813","nonce":2735495271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6817","nonce":2735495271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6815","nonce":2735495271}]},"public_addr":"172.21.15.167:6811/2735495271","cluster_addr":"172.21.15.167:6813/2735495271","heartbeat_back_addr":"172.21.15.167:6817/2735495271","heartbeat_front_addr":"172.21.15.167:6815/2735495271","state":["exists","up"]},{"osd":5,"uuid":"f14b44a9-5910-458c-b518-09a8d0de94de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6827","nonce":1494916572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6829","nonce":1494916572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6833","nonce":1494916572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6831","nonce":1494916572}]},"public_addr":"172.21.15.167:6827/1494916572","cluster_addr":"172.21.15.167:6829/1494916572","heartbeat_back_addr":"172.21.15.167:6833/1494916572","heartbeat_front_addr":"172.21.15.167:6831/1494916572","state":["exists","up"]},{"osd":6,"uuid":"39335545-2654-4710-84b0-a9aa47cfcbba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6819","nonce":2538273273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6821","nonce":2538273273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6825","nonce":2538273273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6823","nonce":2538273273}]},"public_addr":"172.21.15.167:6819/2538273273","cluster_addr":"172.21.15.167:6821/2538273273","heartbeat_back_addr":"172.21.15.167:6825/2538273273","heartbeat_front_addr":"172.21.15.167:6823/2538273273","state":["exists","up"]},{"osd":7,"uuid":"4f8bfc81-53dd-4339-ac22-1a4007d9f963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6803","nonce":265072227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6805","nonce":265072227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6809","nonce":265072227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6807","nonce":265072227}]},"public_addr":"172.21.15.167:6803/265072227","cluster_addr":"172.21.15.167:6805/265072227","heartbeat_back_addr":"172.21.15.167:6809/265072227","heartbeat_front_addr":"172.21.15.167:6807/265072227","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.150359+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.084358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.722368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.332809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.320709+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.395153+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.935250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.801443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":19,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:54:43.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.967+0000 7fbb9e000640 1 -- 172.21.15.151:0/2033385529 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbb7807aaa0 msgr2=0x7fbb7807cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:43.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.967+0000 7fbb9e000640 1 --2- 172.21.15.151:0/2033385529 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbb7807aaa0 0x7fbb7807cf60 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fbb84000fb0 tx=0x7fbb84001d90 comp rx=0 tx=0).stop 2024-09-16T15:54:43.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.967+0000 7fbb9e000640 1 -- 172.21.15.151:0/2033385529 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb7c003a30 msgr2=0x7fbb981497e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:43.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.967+0000 7fbb9e000640 1 --2- 172.21.15.151:0/2033385529 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb7c003a30 0x7fbb981497e0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fbb8802cb70 tx=0x7fbb88033de0 comp rx=0 tx=0).stop 2024-09-16T15:54:43.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.967+0000 7fbb9e000640 1 -- 172.21.15.151:0/2033385529 shutdown_connections 2024-09-16T15:54:43.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.967+0000 7fbb9e000640 1 --2- 172.21.15.151:0/2033385529 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbb7807aaa0 0x7fbb7807cf60 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:43.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.967+0000 7fbb9e000640 1 --2- 172.21.15.151:0/2033385529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbb98146670 0x7fbb9815cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:43.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.967+0000 7fbb9e000640 1 --2- 172.21.15.151:0/2033385529 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbb98146210 0x7fbb9815c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:43.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.967+0000 7fbb9e000640 1 --2- 172.21.15.151:0/2033385529 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb7c003a30 0x7fbb981497e0 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:43.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.967+0000 7fbb9e000640 1 -- 172.21.15.151:0/2033385529 >> 172.21.15.151:0/2033385529 conn(0x7fbb980b1950 msgr2=0x7fbb9815a400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:43.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.967+0000 7fbb9e000640 1 -- 172.21.15.151:0/2033385529 shutdown_connections 2024-09-16T15:54:43.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:43.967+0000 7fbb9e000640 1 -- 172.21.15.151:0/2033385529 wait complete. 2024-09-16T15:54:43.986 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-16T15:54:44.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.142+0000 7f23ef400640 1 -- 172.21.15.151:0/1712677074 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f23e8074150 msgr2=0x7f23e806e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:44.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.142+0000 7f23ef400640 1 --2- 172.21.15.151:0/1712677074 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f23e8074150 0x7f23e806e760 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f23e8075310 tx=0x7f23d402a410 comp rx=0 tx=0).stop 2024-09-16T15:54:44.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.142+0000 7f23ef400640 1 -- 172.21.15.151:0/1712677074 shutdown_connections 2024-09-16T15:54:44.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.142+0000 7f23ef400640 1 --2- 172.21.15.151:0/1712677074 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f23e8074150 0x7f23e806e760 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:44.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.142+0000 7f23ef400640 1 --2- 172.21.15.151:0/1712677074 >> v2:172.21.15.167:3300/0 conn(0x7f23e80b43f0 0x7f23e80b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:44.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.142+0000 7f23ef400640 1 -- 172.21.15.151:0/1712677074 >> 172.21.15.151:0/1712677074 conn(0x7f23e80b1950 msgr2=0x7f23e80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:44.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.142+0000 7f23ef400640 1 -- 172.21.15.151:0/1712677074 shutdown_connections 2024-09-16T15:54:44.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.142+0000 7f23ef400640 1 -- 172.21.15.151:0/1712677074 wait complete. 2024-09-16T15:54:44.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.144+0000 7f23ef400640 1 Processor -- start 2024-09-16T15:54:44.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.144+0000 7f23ef400640 1 -- start start 2024-09-16T15:54:44.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.145+0000 7f23ef400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f23e8074150 0x7f23e809ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:44.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.145+0000 7f23ef400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23e80746a0 0x7f23e815d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:44.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.145+0000 7f23ef400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f23e80b43f0 0x7f23e81578c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:44.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.145+0000 7f23ef400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f23e806f3b0 con 0x7f23e80b43f0 2024-09-16T15:54:44.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.145+0000 7f23ef400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f23e815dd20 con 0x7f23e80746a0 2024-09-16T15:54:44.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.145+0000 7f23ef400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f23e806f530 con 0x7f23e8074150 2024-09-16T15:54:44.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.145+0000 7f23ed400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f23e80b43f0 0x7f23e81578c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:44.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.145+0000 7f23ed400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f23e80b43f0 0x7f23e81578c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44216/0 (socket says 172.21.15.151:44216) 2024-09-16T15:54:44.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.145+0000 7f23ed400640 1 -- 172.21.15.151:0/3779063445 learned_addr learned my addr 172.21.15.151:0/3779063445 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:54:44.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.145+0000 7f23e7e00640 1 --2- 172.21.15.151:0/3779063445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23e80746a0 0x7f23e815d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:44.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.145+0000 7f23eca00640 1 --2- 172.21.15.151:0/3779063445 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f23e8074150 0x7f23e809ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:44.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.145+0000 7f23ed400640 1 -- 172.21.15.151:0/3779063445 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f23e8074150 msgr2=0x7f23e809ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:44.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.145+0000 7f23ed400640 1 --2- 172.21.15.151:0/3779063445 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f23e8074150 0x7f23e809ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:44.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.145+0000 7f23ed400640 1 -- 172.21.15.151:0/3779063445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23e80746a0 msgr2=0x7f23e815d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:44.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.145+0000 7f23ed400640 1 --2- 172.21.15.151:0/3779063445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23e80746a0 0x7f23e815d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:44.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.145+0000 7f23ed400640 1 -- 172.21.15.151:0/3779063445 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23e81f64f0 con 0x7f23e80b43f0 2024-09-16T15:54:44.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.146+0000 7f23ed400640 1 --2- 172.21.15.151:0/3779063445 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f23e80b43f0 0x7f23e81578c0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f23e000c7e0 tx=0x7f23e000cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:44.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.146+0000 7f23e4c00640 1 -- 172.21.15.151:0/3779063445 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e0017070 con 0x7f23e80b43f0 2024-09-16T15:54:44.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.146+0000 7f23e4c00640 1 -- 172.21.15.151:0/3779063445 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f23e0005070 con 0x7f23e80b43f0 2024-09-16T15:54:44.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.146+0000 7f23ef400640 1 -- 172.21.15.151:0/3779063445 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23e81f67d0 con 0x7f23e80b43f0 2024-09-16T15:54:44.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.146+0000 7f23e4c00640 1 -- 172.21.15.151:0/3779063445 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e00054c0 con 0x7f23e80b43f0 2024-09-16T15:54:44.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.146+0000 7f23ef400640 1 -- 172.21.15.151:0/3779063445 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23e81f6d90 con 0x7f23e80b43f0 2024-09-16T15:54:44.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.147+0000 7f23ef400640 1 -- 172.21.15.151:0/3779063445 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23e8073d70 con 0x7f23e80b43f0 2024-09-16T15:54:44.150 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.148+0000 7f23e4c00640 1 -- 172.21.15.151:0/3779063445 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 50) ==== 104398+0+0 (secure 0 0 0) 0x7f23e001a070 con 0x7f23e80b43f0 2024-09-16T15:54:44.150 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.149+0000 7f23e4c00640 1 --2- 172.21.15.151:0/3779063445 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f23c807aaa0 0x7f23c807cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:44.150 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.149+0000 7f23e4c00640 1 -- 172.21.15.151:0/3779063445 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5877+0+0 (secure 0 0 0) 0x7f23e0099180 con 0x7f23e80b43f0 2024-09-16T15:54:44.150 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.150+0000 7f23eca00640 1 --2- 172.21.15.151:0/3779063445 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f23c807aaa0 0x7f23c807cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:44.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.150+0000 7f23eca00640 1 --2- 172.21.15.151:0/3779063445 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f23c807aaa0 0x7f23c807cf60 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f23dc00bc50 tx=0x7f23dc00b3e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:44.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.153+0000 7f23e4c00640 1 -- 172.21.15.151:0/3779063445 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f23e80cf9d0 con 0x7f23e80b43f0 2024-09-16T15:54:44.375 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.374+0000 7f23ef400640 1 -- 172.21.15.151:0/3779063445 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f23e806f030 con 0x7f23e80b43f0 2024-09-16T15:54:44.375 INFO:tasks.ceph.mon.a.smithi151.stderr:2024-09-16T15:54:44.375+0000 7f62c3200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T15:54:44.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.377+0000 7f23e4c00640 1 -- 172.21.15.151:0/3779063445 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v89) ==== 161+0+0 (secure 0 0 0) 0x7f23e0062b80 con 0x7f23e80b43f0 2024-09-16T15:54:44.378 INFO:teuthology.orchestra.run.smithi151.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-16T15:54:44.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.380+0000 7f23ef400640 1 -- 172.21.15.151:0/3779063445 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f23c807aaa0 msgr2=0x7f23c807cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:44.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.380+0000 7f23ef400640 1 --2- 172.21.15.151:0/3779063445 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f23c807aaa0 0x7f23c807cf60 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f23dc00bc50 tx=0x7f23dc00b3e0 comp rx=0 tx=0).stop 2024-09-16T15:54:44.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.380+0000 7f23ef400640 1 -- 172.21.15.151:0/3779063445 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f23e80b43f0 msgr2=0x7f23e81578c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:44.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.380+0000 7f23ef400640 1 --2- 172.21.15.151:0/3779063445 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f23e80b43f0 0x7f23e81578c0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f23e000c7e0 tx=0x7f23e000cca0 comp rx=0 tx=0).stop 2024-09-16T15:54:44.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.380+0000 7f23ef400640 1 -- 172.21.15.151:0/3779063445 shutdown_connections 2024-09-16T15:54:44.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.380+0000 7f23ef400640 1 --2- 172.21.15.151:0/3779063445 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f23c807aaa0 0x7f23c807cf60 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:44.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.380+0000 7f23ef400640 1 --2- 172.21.15.151:0/3779063445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f23e80746a0 0x7f23e815d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:44.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.380+0000 7f23ef400640 1 --2- 172.21.15.151:0/3779063445 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f23e80b43f0 0x7f23e81578c0 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:44.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.380+0000 7f23ef400640 1 --2- 172.21.15.151:0/3779063445 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f23e8074150 0x7f23e809ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:44.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.380+0000 7f23ef400640 1 -- 172.21.15.151:0/3779063445 >> 172.21.15.151:0/3779063445 conn(0x7f23e80b1950 msgr2=0x7f23e80af970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:44.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.381+0000 7f23ef400640 1 -- 172.21.15.151:0/3779063445 shutdown_connections 2024-09-16T15:54:44.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.381+0000 7f23ef400640 1 -- 172.21.15.151:0/3779063445 wait complete. 2024-09-16T15:54:44.398 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-16T15:54:44.448 INFO:tasks.ceph.mds.a.smithi151.stdout:starting mds.a at 2024-09-16T15:54:44.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.562+0000 7fd0eae00640 1 -- 172.21.15.151:0/1824413150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0e415bec0 msgr2=0x7fd0e406fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:44.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.562+0000 7fd0eae00640 1 --2- 172.21.15.151:0/1824413150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0e415bec0 0x7fd0e406fce0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fd0d8005fd0 tx=0x7fd0d802aa90 comp rx=0 tx=0).stop 2024-09-16T15:54:44.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.562+0000 7fd0eae00640 1 -- 172.21.15.151:0/1824413150 shutdown_connections 2024-09-16T15:54:44.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.562+0000 7fd0eae00640 1 --2- 172.21.15.151:0/1824413150 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0e4072560 0x7fd0e4073ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:44.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.562+0000 7fd0eae00640 1 --2- 172.21.15.151:0/1824413150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0e415bec0 0x7fd0e406fce0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:44.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.562+0000 7fd0eae00640 1 -- 172.21.15.151:0/1824413150 >> 172.21.15.151:0/1824413150 conn(0x7fd0e40cf9f0 msgr2=0x7fd0e40cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:44.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.562+0000 7fd0eae00640 1 -- 172.21.15.151:0/1824413150 shutdown_connections 2024-09-16T15:54:44.564 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.562+0000 7fd0eae00640 1 -- 172.21.15.151:0/1824413150 wait complete. 2024-09-16T15:54:44.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.564+0000 7fd0eae00640 1 Processor -- start 2024-09-16T15:54:44.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.564+0000 7fd0eae00640 1 -- start start 2024-09-16T15:54:44.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.565+0000 7fd0eae00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0e4072560 0x7fd0e41f2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:44.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.565+0000 7fd0eae00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0e40cb4f0 0x7fd0e41f3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:44.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.565+0000 7fd0eae00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0e415bec0 0x7fd0e41f4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:44.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.565+0000 7fd0eae00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e40cbe70 con 0x7fd0e40cb4f0 2024-09-16T15:54:44.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.565+0000 7fd0eae00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e40cbcf0 con 0x7fd0e4072560 2024-09-16T15:54:44.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.565+0000 7fd0eae00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fd0e40aebd0 con 0x7fd0e415bec0 2024-09-16T15:54:44.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.565+0000 7fd0e9000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0e40cb4f0 0x7fd0e41f3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:44.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.565+0000 7fd0e9000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0e40cb4f0 0x7fd0e41f3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44236/0 (socket says 172.21.15.151:44236) 2024-09-16T15:54:44.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.565+0000 7fd0e9000640 1 -- 172.21.15.151:0/2069087372 learned_addr learned my addr 172.21.15.151:0/2069087372 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:54:44.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.566+0000 7fd0e9a00640 1 --2- 172.21.15.151:0/2069087372 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0e4072560 0x7fd0e41f2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:44.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.566+0000 7fd0ea400640 1 --2- 172.21.15.151:0/2069087372 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0e415bec0 0x7fd0e41f4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:44.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.566+0000 7fd0e9000640 1 -- 172.21.15.151:0/2069087372 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0e415bec0 msgr2=0x7fd0e41f4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:44.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.566+0000 7fd0e9000640 1 --2- 172.21.15.151:0/2069087372 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0e415bec0 0x7fd0e41f4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:44.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.566+0000 7fd0e9000640 1 -- 172.21.15.151:0/2069087372 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0e4072560 msgr2=0x7fd0e41f2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:44.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.566+0000 7fd0e9000640 1 --2- 172.21.15.151:0/2069087372 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0e4072560 0x7fd0e41f2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:44.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.566+0000 7fd0e9000640 1 -- 172.21.15.151:0/2069087372 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0e41f3300 con 0x7fd0e40cb4f0 2024-09-16T15:54:44.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.566+0000 7fd0e9000640 1 --2- 172.21.15.151:0/2069087372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0e40cb4f0 0x7fd0e41f3590 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fd0d000e9c0 tx=0x7fd0d000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:44.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.566+0000 7fd0e1600640 1 -- 172.21.15.151:0/2069087372 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d000cd80 con 0x7fd0e40cb4f0 2024-09-16T15:54:44.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.566+0000 7fd0e1600640 1 -- 172.21.15.151:0/2069087372 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd0d0004020 con 0x7fd0e40cb4f0 2024-09-16T15:54:44.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.566+0000 7fd0eae00640 1 -- 172.21.15.151:0/2069087372 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0e41f3e90 con 0x7fd0e40cb4f0 2024-09-16T15:54:44.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.566+0000 7fd0e1600640 1 -- 172.21.15.151:0/2069087372 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d0004450 con 0x7fd0e40cb4f0 2024-09-16T15:54:44.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.566+0000 7fd0eae00640 1 -- 172.21.15.151:0/2069087372 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0e41f4450 con 0x7fd0e40cb4f0 2024-09-16T15:54:44.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.567+0000 7fd0eae00640 1 -- 172.21.15.151:0/2069087372 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0e40af050 con 0x7fd0e40cb4f0 2024-09-16T15:54:44.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.569+0000 7fd0e1600640 1 -- 172.21.15.151:0/2069087372 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 50) ==== 104398+0+0 (secure 0 0 0) 0x7fd0d00035f0 con 0x7fd0e40cb4f0 2024-09-16T15:54:44.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.570+0000 7fd0e1600640 1 --2- 172.21.15.151:0/2069087372 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd0c407aaa0 0x7fd0c407cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:44.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.570+0000 7fd0e1600640 1 -- 172.21.15.151:0/2069087372 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5935+0+0 (secure 0 0 0) 0x7fd0d009a5b0 con 0x7fd0e40cb4f0 2024-09-16T15:54:44.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.570+0000 7fd0e9a00640 1 --2- 172.21.15.151:0/2069087372 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd0c407aaa0 0x7fd0c407cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:44.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.571+0000 7fd0e9a00640 1 --2- 172.21.15.151:0/2069087372 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd0c407aaa0 0x7fd0c407cf60 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fd0d8035000 tx=0x7fd0d8038040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:44.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.574+0000 7fd0e1600640 1 -- 172.21.15.151:0/2069087372 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd0d0064680 con 0x7fd0e40cb4f0 2024-09-16T15:54:44.798 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:44.797+0000 7fd0eae00640 1 -- 172.21.15.151:0/2069087372 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fd0e40705c0 con 0x7fd0e40cb4f0 2024-09-16T15:54:45.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:45.379+0000 7fd0e1600640 1 -- 172.21.15.151:0/2069087372 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v90) ==== 104+0+0 (secure 0 0 0) 0x7fd0d0063f80 con 0x7fd0e40cb4f0 2024-09-16T15:54:45.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:45.381+0000 7fd0eae00640 1 -- 172.21.15.151:0/2069087372 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd0c407aaa0 msgr2=0x7fd0c407cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:45.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:45.381+0000 7fd0eae00640 1 --2- 172.21.15.151:0/2069087372 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd0c407aaa0 0x7fd0c407cf60 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fd0d8035000 tx=0x7fd0d8038040 comp rx=0 tx=0).stop 2024-09-16T15:54:45.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:45.381+0000 7fd0eae00640 1 -- 172.21.15.151:0/2069087372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0e40cb4f0 msgr2=0x7fd0e41f3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:45.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:45.381+0000 7fd0eae00640 1 --2- 172.21.15.151:0/2069087372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0e40cb4f0 0x7fd0e41f3590 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fd0d000e9c0 tx=0x7fd0d000ee80 comp rx=0 tx=0).stop 2024-09-16T15:54:45.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:45.382+0000 7fd0eae00640 1 -- 172.21.15.151:0/2069087372 shutdown_connections 2024-09-16T15:54:45.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:45.382+0000 7fd0eae00640 1 --2- 172.21.15.151:0/2069087372 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd0c407aaa0 0x7fd0c407cf60 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:45.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:45.382+0000 7fd0eae00640 1 --2- 172.21.15.151:0/2069087372 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0e415bec0 0x7fd0e41f4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:45.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:45.382+0000 7fd0eae00640 1 --2- 172.21.15.151:0/2069087372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0e40cb4f0 0x7fd0e41f3590 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:45.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:45.382+0000 7fd0eae00640 1 --2- 172.21.15.151:0/2069087372 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0e4072560 0x7fd0e41f2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:45.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:45.382+0000 7fd0eae00640 1 -- 172.21.15.151:0/2069087372 >> 172.21.15.151:0/2069087372 conn(0x7fd0e40cf9f0 msgr2=0x7fd0e40b3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:45.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:45.382+0000 7fd0eae00640 1 -- 172.21.15.151:0/2069087372 shutdown_connections 2024-09-16T15:54:45.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:45.382+0000 7fd0eae00640 1 -- 172.21.15.151:0/2069087372 wait complete. 2024-09-16T15:54:45.403 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-16T15:54:45.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:45.568+0000 7fb643e00640 1 -- 172.21.15.151:0/314303510 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb628004d40 msgr2=0x7fb6280051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:45.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:45.568+0000 7fb643e00640 1 --2- 172.21.15.151:0/314303510 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb628004d40 0x7fb6280051c0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7fb63c009ed0 tx=0x7fb628005830 comp rx=0 tx=0).stop 2024-09-16T15:54:45.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:45.568+0000 7fb643400640 1 -- 172.21.15.151:0/314303510 reap_dead start 2024-09-16T15:54:45.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:45.568+0000 7fb643e00640 1 -- 172.21.15.151:0/314303510 shutdown_connections 2024-09-16T15:54:45.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:45.568+0000 7fb643e00640 1 -- 172.21.15.151:0/314303510 >> 172.21.15.151:0/314303510 conn(0x7fb6440cfa70 msgr2=0x7fb6440cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:45.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:45.568+0000 7fb643e00640 1 -- 172.21.15.151:0/314303510 shutdown_connections 2024-09-16T15:54:45.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:45.568+0000 7fb643e00640 1 -- 172.21.15.151:0/314303510 wait complete. 2024-09-16T15:54:45.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:45.570+0000 7fb643e00640 1 Processor -- start 2024-09-16T15:54:45.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:45.570+0000 7fb643e00640 1 -- start start 2024-09-16T15:54:45.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:45.571+0000 7fb643e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb628004d40 0x7fb6440ce900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:45.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:45.571+0000 7fb643e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb644155100 0x7fb644155580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:45.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:45.571+0000 7fb643e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb644151620 0x7fb644151aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:45.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:45.571+0000 7fb643e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb6441649b0 con 0x7fb644151620 2024-09-16T15:54:45.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:45.571+0000 7fb643e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb644164830 con 0x7fb628004d40 2024-09-16T15:54:45.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:45.571+0000 7fb643e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fb644164b30 con 0x7fb644155100 2024-09-16T15:54:45.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:45.571+0000 7fb643400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb644151620 0x7fb644151aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:45.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:45.571+0000 7fb643400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb644151620 0x7fb644151aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44264/0 (socket says 172.21.15.151:44264) 2024-09-16T15:54:45.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:45.571+0000 7fb643400640 1 -- 172.21.15.151:0/2544713419 learned_addr learned my addr 172.21.15.151:0/2544713419 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:54:45.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:45.571+0000 7fb642a00640 1 --2- 172.21.15.151:0/2544713419 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb628004d40 0x7fb6440ce900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:45.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:45.571+0000 7fb642000640 1 --2- 172.21.15.151:0/2544713419 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb644155100 0x7fb644155580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:45.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:45.571+0000 7fb643400640 1 -- 172.21.15.151:0/2544713419 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb644155100 msgr2=0x7fb644155580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:45.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:45.571+0000 7fb643400640 1 --2- 172.21.15.151:0/2544713419 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb644155100 0x7fb644155580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:45.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:45.571+0000 7fb643400640 1 -- 172.21.15.151:0/2544713419 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb628004d40 msgr2=0x7fb6440ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:45.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:45.571+0000 7fb643400640 1 --2- 172.21.15.151:0/2544713419 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb628004d40 0x7fb6440ce900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:45.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:45.571+0000 7fb643400640 1 -- 172.21.15.151:0/2544713419 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb644153c20 con 0x7fb644151620 2024-09-16T15:54:45.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:45.572+0000 7fb643400640 1 --2- 172.21.15.151:0/2544713419 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb644151620 0x7fb644151aa0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7fb63c02aec0 tx=0x7fb63c033bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:45.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:45.572+0000 7fb63aa00640 1 -- 172.21.15.151:0/2544713419 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63c042070 con 0x7fb644151620 2024-09-16T15:54:45.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:45.572+0000 7fb63aa00640 1 -- 172.21.15.151:0/2544713419 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb63c003970 con 0x7fb644151620 2024-09-16T15:54:45.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:45.572+0000 7fb63aa00640 1 -- 172.21.15.151:0/2544713419 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63c038040 con 0x7fb644151620 2024-09-16T15:54:45.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:45.572+0000 7fb643e00640 1 -- 172.21.15.151:0/2544713419 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb644153ec0 con 0x7fb644151620 2024-09-16T15:54:45.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:45.572+0000 7fb643e00640 1 -- 172.21.15.151:0/2544713419 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb644154480 con 0x7fb644151620 2024-09-16T15:54:45.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:45.573+0000 7fb643e00640 1 -- 172.21.15.151:0/2544713419 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb64406f140 con 0x7fb644151620 2024-09-16T15:54:45.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:45.574+0000 7fb63aa00640 1 -- 172.21.15.151:0/2544713419 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 50) ==== 104398+0+0 (secure 0 0 0) 0x7fb63c004030 con 0x7fb644151620 2024-09-16T15:54:45.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:45.575+0000 7fb63aa00640 1 --2- 172.21.15.151:0/2544713419 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb62407aaa0 0x7fb62407cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:45.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:45.575+0000 7fb63aa00640 1 -- 172.21.15.151:0/2544713419 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5935+0+0 (secure 0 0 0) 0x7fb63c0b9c80 con 0x7fb644151620 2024-09-16T15:54:45.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:45.576+0000 7fb642a00640 1 --2- 172.21.15.151:0/2544713419 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb62407aaa0 0x7fb62407cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:45.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:45.577+0000 7fb642a00640 1 --2- 172.21.15.151:0/2544713419 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb62407aaa0 0x7fb62407cf60 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fb634001eb0 tx=0x7fb634006040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:45.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:45.580+0000 7fb63aa00640 1 -- 172.21.15.151:0/2544713419 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb64406f140 con 0x7fb644151620 2024-09-16T15:54:45.763 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:45.762+0000 7fb643e00640 1 -- 172.21.15.151:0/2544713419 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb64406fe40 con 0x7fb644151620 2024-09-16T15:54:46.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:46.380+0000 7fb63aa00640 1 -- 172.21.15.151:0/2544713419 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v136) ==== 181+0+0 (secure 0 0 0) 0x7fb63c083700 con 0x7fb644151620 2024-09-16T15:54:46.382 INFO:teuthology.orchestra.run.smithi151.stderr:pool 'cephfs_metadata' removed 2024-09-16T15:54:46.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:46.383+0000 7fb643e00640 1 -- 172.21.15.151:0/2544713419 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb62407aaa0 msgr2=0x7fb62407cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:46.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:46.383+0000 7fb643e00640 1 --2- 172.21.15.151:0/2544713419 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb62407aaa0 0x7fb62407cf60 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fb634001eb0 tx=0x7fb634006040 comp rx=0 tx=0).stop 2024-09-16T15:54:46.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:46.384+0000 7fb643e00640 1 -- 172.21.15.151:0/2544713419 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb644151620 msgr2=0x7fb644151aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:46.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:46.384+0000 7fb643e00640 1 --2- 172.21.15.151:0/2544713419 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb644151620 0x7fb644151aa0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7fb63c02aec0 tx=0x7fb63c033bf0 comp rx=0 tx=0).stop 2024-09-16T15:54:46.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:46.384+0000 7fb643e00640 1 -- 172.21.15.151:0/2544713419 shutdown_connections 2024-09-16T15:54:46.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:46.384+0000 7fb643e00640 1 --2- 172.21.15.151:0/2544713419 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb62407aaa0 0x7fb62407cf60 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:46.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:46.384+0000 7fb643e00640 1 --2- 172.21.15.151:0/2544713419 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb644151620 0x7fb644151aa0 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:46.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:46.384+0000 7fb643e00640 1 --2- 172.21.15.151:0/2544713419 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb644155100 0x7fb644155580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:46.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:46.384+0000 7fb643e00640 1 --2- 172.21.15.151:0/2544713419 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb628004d40 0x7fb6440ce900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:46.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:46.384+0000 7fb643e00640 1 -- 172.21.15.151:0/2544713419 >> 172.21.15.151:0/2544713419 conn(0x7fb6440cfa70 msgr2=0x7fb6440ad880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:46.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:46.384+0000 7fb643e00640 1 -- 172.21.15.151:0/2544713419 shutdown_connections 2024-09-16T15:54:46.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:46.384+0000 7fb643e00640 1 -- 172.21.15.151:0/2544713419 wait complete. 2024-09-16T15:54:46.403 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-16T15:54:46.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:46.564+0000 7f4874600640 1 -- 172.21.15.151:0/3575919088 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f486c073fd0 msgr2=0x7f486c073070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:46.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:46.564+0000 7f4874600640 1 --2- 172.21.15.151:0/3575919088 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f486c073fd0 0x7f486c073070 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f486c0b4310 tx=0x7f485802a580 comp rx=0 tx=0).stop 2024-09-16T15:54:46.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:46.564+0000 7f4874600640 1 -- 172.21.15.151:0/3575919088 shutdown_connections 2024-09-16T15:54:46.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:46.564+0000 7f4874600640 1 --2- 172.21.15.151:0/3575919088 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f486c0735b0 0x7f486c15e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:46.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:46.564+0000 7f4874600640 1 --2- 172.21.15.151:0/3575919088 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f486c073fd0 0x7f486c073070 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:46.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:46.564+0000 7f4874600640 1 --2- 172.21.15.151:0/3575919088 >> v2:172.21.15.167:3300/0 conn(0x7f486c15aba0 0x7f486c15af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:46.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:46.564+0000 7f4874600640 1 -- 172.21.15.151:0/3575919088 >> 172.21.15.151:0/3575919088 conn(0x7f486c06cb70 msgr2=0x7f486c06cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:46.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:46.564+0000 7f4874600640 1 -- 172.21.15.151:0/3575919088 shutdown_connections 2024-09-16T15:54:46.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:46.564+0000 7f4874600640 1 -- 172.21.15.151:0/3575919088 wait complete. 2024-09-16T15:54:46.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:46.565+0000 7f4874600640 1 Processor -- start 2024-09-16T15:54:46.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:46.566+0000 7f4874600640 1 -- start start 2024-09-16T15:54:46.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:46.566+0000 7f4874600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f486c0735b0 0x7f486c0cf100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:46.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:46.566+0000 7f4874600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f486c073fd0 0x7f486c1f7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:46.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:46.566+0000 7f4874600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f486c15aba0 0x7f486c1561e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:46.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:46.566+0000 7f4874600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f486c15eec0 con 0x7f486c15aba0 2024-09-16T15:54:46.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:46.566+0000 7f4874600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f486c15ed40 con 0x7f486c073fd0 2024-09-16T15:54:46.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:46.566+0000 7f4874600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f486c15f040 con 0x7f486c0735b0 2024-09-16T15:54:46.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:46.567+0000 7f4872600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f486c15aba0 0x7f486c1561e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:46.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:46.567+0000 7f4872600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f486c15aba0 0x7f486c1561e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44292/0 (socket says 172.21.15.151:44292) 2024-09-16T15:54:46.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:46.567+0000 7f4872600640 1 -- 172.21.15.151:0/783387883 learned_addr learned my addr 172.21.15.151:0/783387883 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:54:46.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:46.567+0000 7f4871c00640 1 --2- 172.21.15.151:0/783387883 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f486c0735b0 0x7f486c0cf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:46.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:46.567+0000 7f4871200640 1 --2- 172.21.15.151:0/783387883 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f486c073fd0 0x7f486c1f7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:46.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:46.567+0000 7f4872600640 1 -- 172.21.15.151:0/783387883 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f486c0735b0 msgr2=0x7f486c0cf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:46.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:46.567+0000 7f4872600640 1 --2- 172.21.15.151:0/783387883 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f486c0735b0 0x7f486c0cf100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:46.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:46.567+0000 7f4872600640 1 -- 172.21.15.151:0/783387883 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f486c073fd0 msgr2=0x7f486c1f7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:46.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:46.567+0000 7f4872600640 1 --2- 172.21.15.151:0/783387883 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f486c073fd0 0x7f486c1f7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:46.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:46.567+0000 7f4872600640 1 -- 172.21.15.151:0/783387883 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f486c155f00 con 0x7f486c15aba0 2024-09-16T15:54:46.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:46.567+0000 7f4872600640 1 --2- 172.21.15.151:0/783387883 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f486c15aba0 0x7f486c1561e0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f486400eaa0 tx=0x7f486400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:46.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:46.568+0000 7f4869600640 1 -- 172.21.15.151:0/783387883 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4864004b20 con 0x7f486c15aba0 2024-09-16T15:54:46.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:46.568+0000 7f4869600640 1 -- 172.21.15.151:0/783387883 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4864004db0 con 0x7f486c15aba0 2024-09-16T15:54:46.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:46.568+0000 7f4869600640 1 -- 172.21.15.151:0/783387883 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4864005280 con 0x7f486c15aba0 2024-09-16T15:54:46.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:46.568+0000 7f4874600640 1 -- 172.21.15.151:0/783387883 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f486c0ce620 con 0x7f486c15aba0 2024-09-16T15:54:46.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:46.568+0000 7f4874600640 1 -- 172.21.15.151:0/783387883 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f486c0cebe0 con 0x7f486c15aba0 2024-09-16T15:54:46.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:46.569+0000 7f4874600640 1 -- 172.21.15.151:0/783387883 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f486c15f540 con 0x7f486c15aba0 2024-09-16T15:54:46.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:46.570+0000 7f4869600640 1 -- 172.21.15.151:0/783387883 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 50) ==== 104398+0+0 (secure 0 0 0) 0x7f4864005420 con 0x7f486c15aba0 2024-09-16T15:54:46.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:46.571+0000 7f4869600640 1 --2- 172.21.15.151:0/783387883 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f484c07aaa0 0x7f484c07cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:46.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:46.572+0000 7f4869600640 1 -- 172.21.15.151:0/783387883 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 5497+0+0 (secure 0 0 0) 0x7f486409a770 con 0x7f486c15aba0 2024-09-16T15:54:46.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:46.572+0000 7f4871c00640 1 --2- 172.21.15.151:0/783387883 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f484c07aaa0 0x7f484c07cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:46.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:46.573+0000 7f4871c00640 1 --2- 172.21.15.151:0/783387883 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f484c07aaa0 0x7f484c07cf60 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f486c157e60 tx=0x7f4860005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:46.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:46.576+0000 7f4869600640 1 -- 172.21.15.151:0/783387883 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4864064270 con 0x7f486c15aba0 2024-09-16T15:54:46.762 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:46.761+0000 7f4874600640 1 -- 172.21.15.151:0/783387883 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f486c0cc890 con 0x7f486c15aba0 2024-09-16T15:54:47.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.389+0000 7f4869600640 1 -- 172.21.15.151:0/783387883 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v137) ==== 169+0+0 (secure 0 0 0) 0x7f4864068ca0 con 0x7f486c15aba0 2024-09-16T15:54:47.390 INFO:teuthology.orchestra.run.smithi151.stderr:pool 'cephfs_data' removed 2024-09-16T15:54:47.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.391+0000 7f4874600640 1 -- 172.21.15.151:0/783387883 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f484c07aaa0 msgr2=0x7f484c07cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:47.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.391+0000 7f4874600640 1 --2- 172.21.15.151:0/783387883 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f484c07aaa0 0x7f484c07cf60 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f486c157e60 tx=0x7f4860005f90 comp rx=0 tx=0).stop 2024-09-16T15:54:47.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.391+0000 7f4874600640 1 -- 172.21.15.151:0/783387883 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f486c15aba0 msgr2=0x7f486c1561e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:47.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.391+0000 7f4874600640 1 --2- 172.21.15.151:0/783387883 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f486c15aba0 0x7f486c1561e0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f486400eaa0 tx=0x7f486400ef60 comp rx=0 tx=0).stop 2024-09-16T15:54:47.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.391+0000 7f4874600640 1 -- 172.21.15.151:0/783387883 shutdown_connections 2024-09-16T15:54:47.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.391+0000 7f4874600640 1 --2- 172.21.15.151:0/783387883 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f484c07aaa0 0x7f484c07cf60 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:47.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.391+0000 7f4874600640 1 --2- 172.21.15.151:0/783387883 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f486c073fd0 0x7f486c1f7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:47.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.391+0000 7f4874600640 1 --2- 172.21.15.151:0/783387883 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f486c15aba0 0x7f486c1561e0 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:47.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.391+0000 7f4874600640 1 --2- 172.21.15.151:0/783387883 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f486c0735b0 0x7f486c0cf100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:47.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.391+0000 7f4874600640 1 -- 172.21.15.151:0/783387883 >> 172.21.15.151:0/783387883 conn(0x7f486c06cb70 msgr2=0x7f486c0d0420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:47.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.392+0000 7f4874600640 1 -- 172.21.15.151:0/783387883 shutdown_connections 2024-09-16T15:54:47.394 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.392+0000 7f4874600640 1 -- 172.21.15.151:0/783387883 wait complete. 2024-09-16T15:54:47.412 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-16T15:54:47.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.574+0000 7f2da3a00640 1 -- 172.21.15.151:0/3883355866 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d80004800 msgr2=0x7f2d80004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:47.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.574+0000 7f2da3a00640 1 --2- 172.21.15.151:0/3883355866 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d80004800 0x7f2d80004c80 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f2d88001690 tx=0x7f2d80005830 comp rx=0 tx=0).stop 2024-09-16T15:54:47.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.574+0000 7f2da3a00640 1 -- 172.21.15.151:0/3883355866 shutdown_connections 2024-09-16T15:54:47.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.574+0000 7f2da3a00640 1 --2- 172.21.15.151:0/3883355866 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d80004800 0x7f2d80004c80 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:47.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.574+0000 7f2da3a00640 1 -- 172.21.15.151:0/3883355866 >> 172.21.15.151:0/3883355866 conn(0x7f2d9c0b1950 msgr2=0x7f2d9c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:47.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.574+0000 7f2da3a00640 1 -- 172.21.15.151:0/3883355866 shutdown_connections 2024-09-16T15:54:47.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.574+0000 7f2da3a00640 1 -- 172.21.15.151:0/3883355866 wait complete. 2024-09-16T15:54:47.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.576+0000 7f2da3a00640 1 Processor -- start 2024-09-16T15:54:47.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.576+0000 7f2da3a00640 1 -- start start 2024-09-16T15:54:47.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.577+0000 7f2da3a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d80004800 0x7f2d9c159150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:47.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.577+0000 7f2da3a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d9c159b40 0x7f2d9c1f1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:47.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.577+0000 7f2da3a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d9c1f7fd0 0x7f2d9c1f8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:47.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.577+0000 7f2da3a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2d9c074420 con 0x7f2d9c1f7fd0 2024-09-16T15:54:47.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.577+0000 7f2da3a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2d9c0b2700 con 0x7f2d80004800 2024-09-16T15:54:47.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.577+0000 7f2da3a00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f2d9c0745a0 con 0x7f2d9c159b40 2024-09-16T15:54:47.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.577+0000 7f2da1a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d9c1f7fd0 0x7f2d9c1f8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:47.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.577+0000 7f2da1a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d9c1f7fd0 0x7f2d9c1f8480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44322/0 (socket says 172.21.15.151:44322) 2024-09-16T15:54:47.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.577+0000 7f2da1a00640 1 -- 172.21.15.151:0/32735673 learned_addr learned my addr 172.21.15.151:0/32735673 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:54:47.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.577+0000 7f2d9be00640 1 --2- 172.21.15.151:0/32735673 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d9c159b40 0x7f2d9c1f1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:47.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.577+0000 7f2da1000640 1 --2- 172.21.15.151:0/32735673 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d80004800 0x7f2d9c159150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:47.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.578+0000 7f2da1a00640 1 -- 172.21.15.151:0/32735673 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d9c159b40 msgr2=0x7f2d9c1f1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:47.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.578+0000 7f2da1a00640 1 --2- 172.21.15.151:0/32735673 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d9c159b40 0x7f2d9c1f1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:47.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.578+0000 7f2da1a00640 1 -- 172.21.15.151:0/32735673 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d80004800 msgr2=0x7f2d9c159150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:47.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.578+0000 7f2da1a00640 1 --2- 172.21.15.151:0/32735673 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d80004800 0x7f2d9c159150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:47.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.578+0000 7f2da1a00640 1 -- 172.21.15.151:0/32735673 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d9c0b3760 con 0x7f2d9c1f7fd0 2024-09-16T15:54:47.580 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.578+0000 7f2da1a00640 1 --2- 172.21.15.151:0/32735673 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d9c1f7fd0 0x7f2d9c1f8480 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f2d94006fd0 tx=0x7f2d94006660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:47.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.578+0000 7f2d98c00640 1 -- 172.21.15.151:0/32735673 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d9400fa80 con 0x7f2d9c1f7fd0 2024-09-16T15:54:47.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.578+0000 7f2d98c00640 1 -- 172.21.15.151:0/32735673 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2d9400fc20 con 0x7f2d9c1f7fd0 2024-09-16T15:54:47.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.578+0000 7f2d98c00640 1 -- 172.21.15.151:0/32735673 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d94011430 con 0x7f2d9c1f7fd0 2024-09-16T15:54:47.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.578+0000 7f2da3a00640 1 -- 172.21.15.151:0/32735673 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d9c0b3a40 con 0x7f2d9c1f7fd0 2024-09-16T15:54:47.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.578+0000 7f2da3a00640 1 -- 172.21.15.151:0/32735673 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d9c0b4000 con 0x7f2d9c1f7fd0 2024-09-16T15:54:47.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.579+0000 7f2da3a00640 1 -- 172.21.15.151:0/32735673 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d9c15b1c0 con 0x7f2d9c1f7fd0 2024-09-16T15:54:47.581 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.581+0000 7f2d98c00640 1 -- 172.21.15.151:0/32735673 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 50) ==== 104398+0+0 (secure 0 0 0) 0x7f2d940115d0 con 0x7f2d9c1f7fd0 2024-09-16T15:54:47.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.581+0000 7f2d98c00640 1 --2- 172.21.15.151:0/32735673 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2d7c07aad0 0x7f2d7c07cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:47.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.582+0000 7f2d98c00640 1 -- 172.21.15.151:0/32735673 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5083+0+0 (secure 0 0 0) 0x7f2d9409a370 con 0x7f2d9c1f7fd0 2024-09-16T15:54:47.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.582+0000 7f2da1000640 1 --2- 172.21.15.151:0/32735673 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2d7c07aad0 0x7f2d7c07cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:47.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.583+0000 7f2da1000640 1 --2- 172.21.15.151:0/32735673 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2d7c07aad0 0x7f2d7c07cf90 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c052890 tx=0x7f2d8c005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:47.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.586+0000 7f2d98c00640 1 -- 172.21.15.151:0/32735673 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2d9c15b1c0 con 0x7f2d9c1f7fd0 2024-09-16T15:54:47.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.781+0000 7f2da3a00640 1 -- 172.21.15.151:0/32735673 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f2d9c0acd30 con 0x7f2d9c1f7fd0 2024-09-16T15:54:47.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.786+0000 7f2d98c00640 1 -- 172.21.15.151:0/32735673 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v53) ==== 71+0+0 (secure 0 0 0) 0x7f2d94064440 con 0x7f2d9c1f7fd0 2024-09-16T15:54:47.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.788+0000 7f2da3a00640 1 -- 172.21.15.151:0/32735673 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2d7c07aad0 msgr2=0x7f2d7c07cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:47.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.788+0000 7f2da3a00640 1 --2- 172.21.15.151:0/32735673 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2d7c07aad0 0x7f2d7c07cf90 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c052890 tx=0x7f2d8c005f90 comp rx=0 tx=0).stop 2024-09-16T15:54:47.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.788+0000 7f2da3a00640 1 -- 172.21.15.151:0/32735673 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d9c1f7fd0 msgr2=0x7f2d9c1f8480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:47.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.788+0000 7f2da3a00640 1 --2- 172.21.15.151:0/32735673 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d9c1f7fd0 0x7f2d9c1f8480 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f2d94006fd0 tx=0x7f2d94006660 comp rx=0 tx=0).stop 2024-09-16T15:54:47.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.788+0000 7f2da3a00640 1 -- 172.21.15.151:0/32735673 shutdown_connections 2024-09-16T15:54:47.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.788+0000 7f2da3a00640 1 --2- 172.21.15.151:0/32735673 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2d7c07aad0 0x7f2d7c07cf90 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:47.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.788+0000 7f2da3a00640 1 --2- 172.21.15.151:0/32735673 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d9c1f7fd0 0x7f2d9c1f8480 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:47.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.788+0000 7f2da3a00640 1 --2- 172.21.15.151:0/32735673 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d9c159b40 0x7f2d9c1f1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:47.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.788+0000 7f2da3a00640 1 --2- 172.21.15.151:0/32735673 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d80004800 0x7f2d9c159150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:47.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.788+0000 7f2da3a00640 1 -- 172.21.15.151:0/32735673 >> 172.21.15.151:0/32735673 conn(0x7f2d9c0b1950 msgr2=0x7f2d9c0adf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:47.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.788+0000 7f2da3a00640 1 -- 172.21.15.151:0/32735673 shutdown_connections 2024-09-16T15:54:47.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.788+0000 7f2da3a00640 1 -- 172.21.15.151:0/32735673 wait complete. 2024-09-16T15:54:47.805 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_less_than_cloner_threads' 2024-09-16T15:54:47.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.962+0000 7f2db4c00640 1 -- 172.21.15.151:0/2536109520 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2db0073fb0 msgr2=0x7f2db0074420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:47.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.962+0000 7f2db4c00640 1 --2- 172.21.15.151:0/2536109520 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2db0073fb0 0x7f2db0074420 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f2db0075130 tx=0x7f2da802a4a0 comp rx=0 tx=0).stop 2024-09-16T15:54:47.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.962+0000 7f2db4c00640 1 -- 172.21.15.151:0/2536109520 shutdown_connections 2024-09-16T15:54:47.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.962+0000 7f2db4c00640 1 --2- 172.21.15.151:0/2536109520 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2db0073fb0 0x7f2db0074420 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:47.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.962+0000 7f2db4c00640 1 --2- 172.21.15.151:0/2536109520 >> v2:172.21.15.167:3300/0 conn(0x7f2db015f5a0 0x7f2db00b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:47.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.962+0000 7f2db4c00640 1 -- 172.21.15.151:0/2536109520 >> 172.21.15.151:0/2536109520 conn(0x7f2db00b1950 msgr2=0x7f2db00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:47.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.962+0000 7f2db4c00640 1 -- 172.21.15.151:0/2536109520 shutdown_connections 2024-09-16T15:54:47.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.962+0000 7f2db4c00640 1 -- 172.21.15.151:0/2536109520 wait complete. 2024-09-16T15:54:47.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.964+0000 7f2db4c00640 1 Processor -- start 2024-09-16T15:54:47.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.964+0000 7f2db4c00640 1 -- start start 2024-09-16T15:54:47.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.965+0000 7f2db4c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2db0073fb0 0x7f2db009aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:47.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.965+0000 7f2db4c00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2db00ccf00 0x7f2db00b4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:47.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.965+0000 7f2db4c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2db015f5a0 0x7f2db015c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:47.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.965+0000 7f2db4c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2db006f2e0 con 0x7f2db015f5a0 2024-09-16T15:54:47.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.965+0000 7f2db4c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2db006f160 con 0x7f2db0073fb0 2024-09-16T15:54:47.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.965+0000 7f2db4c00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f2db006f460 con 0x7f2db00ccf00 2024-09-16T15:54:47.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.965+0000 7f2dae800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2db015f5a0 0x7f2db015c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:47.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.965+0000 7f2dae800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2db015f5a0 0x7f2db015c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44354/0 (socket says 172.21.15.151:44354) 2024-09-16T15:54:47.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.965+0000 7f2dae800640 1 -- 172.21.15.151:0/913620819 learned_addr learned my addr 172.21.15.151:0/913620819 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:54:47.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.965+0000 7f2dade00640 1 --2- 172.21.15.151:0/913620819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2db0073fb0 0x7f2db009aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:47.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.965+0000 7f2dad400640 1 --2- 172.21.15.151:0/913620819 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2db00ccf00 0x7f2db00b4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:47.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.965+0000 7f2dae800640 1 -- 172.21.15.151:0/913620819 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2db00ccf00 msgr2=0x7f2db00b4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:47.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.965+0000 7f2dae800640 1 --2- 172.21.15.151:0/913620819 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2db00ccf00 0x7f2db00b4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:47.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.965+0000 7f2dae800640 1 -- 172.21.15.151:0/913620819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2db0073fb0 msgr2=0x7f2db009aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:47.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.965+0000 7f2dae800640 1 --2- 172.21.15.151:0/913620819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2db0073fb0 0x7f2db009aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:47.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.966+0000 7f2dae800640 1 -- 172.21.15.151:0/913620819 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2db009af40 con 0x7f2db015f5a0 2024-09-16T15:54:47.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.966+0000 7f2dad400640 1 --2- 172.21.15.151:0/913620819 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2db00ccf00 0x7f2db00b4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:54:47.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.966+0000 7f2dae800640 1 --2- 172.21.15.151:0/913620819 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2db015f5a0 0x7f2db015c080 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f2da802aa90 tx=0x7f2da8032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:47.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.966+0000 7f2da6000640 1 -- 172.21.15.151:0/913620819 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da8040070 con 0x7f2db015f5a0 2024-09-16T15:54:47.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.966+0000 7f2da6000640 1 -- 172.21.15.151:0/913620819 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2da8005310 con 0x7f2db015f5a0 2024-09-16T15:54:47.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.966+0000 7f2da6000640 1 -- 172.21.15.151:0/913620819 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da8005c90 con 0x7f2db015f5a0 2024-09-16T15:54:47.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.966+0000 7f2db4c00640 1 -- 172.21.15.151:0/913620819 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2db01f7cc0 con 0x7f2db015f5a0 2024-09-16T15:54:47.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.966+0000 7f2db4c00640 1 -- 172.21.15.151:0/913620819 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2db01f8280 con 0x7f2db015f5a0 2024-09-16T15:54:47.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.967+0000 7f2db4c00640 1 -- 172.21.15.151:0/913620819 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2db006f9f0 con 0x7f2db015f5a0 2024-09-16T15:54:47.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.968+0000 7f2da6000640 1 -- 172.21.15.151:0/913620819 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 50) ==== 104398+0+0 (secure 0 0 0) 0x7f2da8039040 con 0x7f2db015f5a0 2024-09-16T15:54:47.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.969+0000 7f2da6000640 1 --2- 172.21.15.151:0/913620819 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2d9007aaa0 0x7f2d9007cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:47.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.969+0000 7f2da6000640 1 -- 172.21.15.151:0/913620819 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5083+0+0 (secure 0 0 0) 0x7f2da80bac90 con 0x7f2db015f5a0 2024-09-16T15:54:47.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.970+0000 7f2dade00640 1 --2- 172.21.15.151:0/913620819 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2d9007aaa0 0x7f2d9007cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:47.972 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.971+0000 7f2dade00640 1 --2- 172.21.15.151:0/913620819 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2d9007aaa0 0x7f2d9007cf60 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f2db0150760 tx=0x7f2da0009500 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:47.974 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:47.973+0000 7f2da6000640 1 -- 172.21.15.151:0/913620819 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2da8084e00 con 0x7f2db015f5a0 2024-09-16T15:54:48.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.225+0000 7f2db4c00640 1 -- 172.21.15.151:0/913620819 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_less_than_cloner_threads"]} v 0) -- 0x7f2db00ad200 con 0x7f2db015f5a0 2024-09-16T15:54:48.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.391+0000 7f2da6000640 1 -- 172.21.15.151:0/913620819 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_less_than_cloner_threads"]}]=0 v705) ==== 186+0+0 (secure 0 0 0) 0x7f2da8084700 con 0x7f2db015f5a0 2024-09-16T15:54:48.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.393+0000 7f2db4c00640 1 -- 172.21.15.151:0/913620819 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2d9007aaa0 msgr2=0x7f2d9007cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:48.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.393+0000 7f2db4c00640 1 --2- 172.21.15.151:0/913620819 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2d9007aaa0 0x7f2d9007cf60 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f2db0150760 tx=0x7f2da0009500 comp rx=0 tx=0).stop 2024-09-16T15:54:48.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.393+0000 7f2db4c00640 1 -- 172.21.15.151:0/913620819 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2db015f5a0 msgr2=0x7f2db015c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:48.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.393+0000 7f2db4c00640 1 --2- 172.21.15.151:0/913620819 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2db015f5a0 0x7f2db015c080 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f2da802aa90 tx=0x7f2da8032c10 comp rx=0 tx=0).stop 2024-09-16T15:54:48.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.393+0000 7f2db4c00640 1 -- 172.21.15.151:0/913620819 shutdown_connections 2024-09-16T15:54:48.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.393+0000 7f2db4c00640 1 --2- 172.21.15.151:0/913620819 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2d9007aaa0 0x7f2d9007cf60 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:48.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.393+0000 7f2db4c00640 1 --2- 172.21.15.151:0/913620819 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2db015f5a0 0x7f2db015c080 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:48.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.393+0000 7f2db4c00640 1 --2- 172.21.15.151:0/913620819 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2db00ccf00 0x7f2db00b4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:48.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.393+0000 7f2db4c00640 1 --2- 172.21.15.151:0/913620819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2db0073fb0 0x7f2db009aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:48.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.393+0000 7f2db4c00640 1 -- 172.21.15.151:0/913620819 >> 172.21.15.151:0/913620819 conn(0x7f2db00b1950 msgr2=0x7f2db00ce2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:48.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.393+0000 7f2db4c00640 1 -- 172.21.15.151:0/913620819 shutdown_connections 2024-09-16T15:54:48.395 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.393+0000 7f2db4c00640 1 -- 172.21.15.151:0/913620819 wait complete. 2024-09-16T15:54:48.411 INFO:tasks.cephfs_test_runner:Test that the progress bar that is printed for 1 ongoing clone job is ... ok 2024-09-16T15:54:48.411 INFO:tasks.cephfs_test_runner:Starting test: test_when_clones_cancelled_are_more_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) Test that both the progress bars, that are printed for all 7 clone 2024-09-16T15:54:48.411 INFO:tasks.cephfs_test_runner:test_when_clones_cancelled_are_more_than_cloner_threads (tasks.cephfs.test_volumes.TestCloneProgressReporter) 2024-09-16T15:54:48.412 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:54:48.571 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.570+0000 7f243a200640 1 -- 172.21.15.151:0/760736463 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2418003c10 msgr2=0x7f2418004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:48.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.570+0000 7f243a200640 1 --2- 172.21.15.151:0/760736463 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2418003c10 0x7f2418004010 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f2424001830 tx=0x7f2418005830 comp rx=0 tx=0).stop 2024-09-16T15:54:48.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.570+0000 7f2433e00640 1 -- 172.21.15.151:0/760736463 reap_dead start 2024-09-16T15:54:48.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.570+0000 7f243a200640 1 -- 172.21.15.151:0/760736463 shutdown_connections 2024-09-16T15:54:48.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.570+0000 7f243a200640 1 -- 172.21.15.151:0/760736463 >> 172.21.15.151:0/760736463 conn(0x7f24340b1950 msgr2=0x7f24340b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:48.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.570+0000 7f243a200640 1 -- 172.21.15.151:0/760736463 shutdown_connections 2024-09-16T15:54:48.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.570+0000 7f243a200640 1 -- 172.21.15.151:0/760736463 wait complete. 2024-09-16T15:54:48.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.572+0000 7f243a200640 1 Processor -- start 2024-09-16T15:54:48.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.572+0000 7f243a200640 1 -- start start 2024-09-16T15:54:48.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.573+0000 7f243a200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2418003c10 0x7f24341f5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:48.575 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.573+0000 7f243a200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f243415a570 0x7f24341f88d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:48.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.573+0000 7f243a200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f243409beb0 0x7f2434155390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:48.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.573+0000 7f243a200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2434074840 con 0x7f243409beb0 2024-09-16T15:54:48.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.573+0000 7f243a200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f24340746c0 con 0x7f243415a570 2024-09-16T15:54:48.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.573+0000 7f243a200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f243406f160 con 0x7f2418003c10 2024-09-16T15:54:48.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.573+0000 7f2433e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f243409beb0 0x7f2434155390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:48.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.573+0000 7f2433e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f243409beb0 0x7f2434155390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44372/0 (socket says 172.21.15.151:44372) 2024-09-16T15:54:48.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.573+0000 7f2433e00640 1 -- 172.21.15.151:0/3071281893 learned_addr learned my addr 172.21.15.151:0/3071281893 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:54:48.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.573+0000 7f2432a00640 1 --2- 172.21.15.151:0/3071281893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f243415a570 0x7f24341f88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:48.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.573+0000 7f2433400640 1 --2- 172.21.15.151:0/3071281893 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2418003c10 0x7f24341f5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:48.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.574+0000 7f2433e00640 1 -- 172.21.15.151:0/3071281893 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2418003c10 msgr2=0x7f24341f5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:48.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.574+0000 7f2433e00640 1 --2- 172.21.15.151:0/3071281893 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2418003c10 0x7f24341f5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:48.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.574+0000 7f2433e00640 1 -- 172.21.15.151:0/3071281893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f243415a570 msgr2=0x7f24341f88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:48.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.574+0000 7f2433e00640 1 --2- 172.21.15.151:0/3071281893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f243415a570 0x7f24341f88d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:48.576 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.574+0000 7f2433e00640 1 -- 172.21.15.151:0/3071281893 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2434159d00 con 0x7f243409beb0 2024-09-16T15:54:48.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.574+0000 7f2433e00640 1 --2- 172.21.15.151:0/3071281893 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f243409beb0 0x7f2434155390 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f243406e130 tx=0x7f242c010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:48.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.574+0000 7f242b400640 1 -- 172.21.15.151:0/3071281893 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f242c003970 con 0x7f243409beb0 2024-09-16T15:54:48.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.574+0000 7f242b400640 1 -- 172.21.15.151:0/3071281893 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f242c015070 con 0x7f243409beb0 2024-09-16T15:54:48.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.574+0000 7f242b400640 1 -- 172.21.15.151:0/3071281893 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f242c003db0 con 0x7f243409beb0 2024-09-16T15:54:48.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.574+0000 7f243a200640 1 -- 172.21.15.151:0/3071281893 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24341f8110 con 0x7f243409beb0 2024-09-16T15:54:48.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.575+0000 7f243a200640 1 -- 172.21.15.151:0/3071281893 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f243415a280 con 0x7f243409beb0 2024-09-16T15:54:48.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.576+0000 7f243a200640 1 -- 172.21.15.151:0/3071281893 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f243406f650 con 0x7f243409beb0 2024-09-16T15:54:48.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.577+0000 7f242b400640 1 -- 172.21.15.151:0/3071281893 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 50) ==== 104398+0+0 (secure 0 0 0) 0x7f242c004030 con 0x7f243409beb0 2024-09-16T15:54:48.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.577+0000 7f242b400640 1 --2- 172.21.15.151:0/3071281893 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f241407aad0 0x7f241407cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:48.578 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.578+0000 7f242b400640 1 -- 172.21.15.151:0/3071281893 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5083+0+0 (secure 0 0 0) 0x7f242c09a4d0 con 0x7f243409beb0 2024-09-16T15:54:48.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.578+0000 7f2433400640 1 --2- 172.21.15.151:0/3071281893 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f241407aad0 0x7f241407cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:48.579 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.579+0000 7f2433400640 1 --2- 172.21.15.151:0/3071281893 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f241407aad0 0x7f241407cf90 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f2424032000 tx=0x7f2424001360 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:48.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.581+0000 7f242b400640 1 -- 172.21.15.151:0/3071281893 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f242c063f50 con 0x7f243409beb0 2024-09-16T15:54:48.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.767+0000 7f243a200640 1 -- 172.21.15.151:0/3071281893 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f243415e100 con 0x7f243409beb0 2024-09-16T15:54:48.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.768+0000 7f242b400640 1 -- 172.21.15.151:0/3071281893 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) ==== 74+0+13326 (secure 0 0 0) 0x7f242c068980 con 0x7f243409beb0 2024-09-16T15:54:48.769 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:54:48.770 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":137,"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","created":"2024-09-16T15:38:56.348535+0000","modified":"2024-09-16T15:54:47.381491+0000","last_up_change":"2024-09-16T15:39:10.681384+0000","last_in_change":"2024-09-16T15:39:00.544477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:39:11.602252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5524076-a903-4d0a-af6a-caf474c4765d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6809","nonce":3111342741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6811","nonce":3111342741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6815","nonce":3111342741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6813","nonce":3111342741}]},"public_addr":"172.21.15.151:6809/3111342741","cluster_addr":"172.21.15.151:6811/3111342741","heartbeat_back_addr":"172.21.15.151:6815/3111342741","heartbeat_front_addr":"172.21.15.151:6813/3111342741","state":["exists","up"]},{"osd":1,"uuid":"dd25925f-0ae1-494e-8529-4dc205759db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6801","nonce":3368304087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6803","nonce":3368304087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6807","nonce":3368304087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6805","nonce":3368304087}]},"public_addr":"172.21.15.151:6801/3368304087","cluster_addr":"172.21.15.151:6803/3368304087","heartbeat_back_addr":"172.21.15.151:6807/3368304087","heartbeat_front_addr":"172.21.15.151:6805/3368304087","state":["exists","up"]},{"osd":2,"uuid":"c2f18115-013d-4da6-a0fe-cb978f4494b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6823","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6825","nonce":3079373271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6827","nonce":3079373271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6831","nonce":3079373271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6829","nonce":3079373271}]},"public_addr":"172.21.15.151:6825/3079373271","cluster_addr":"172.21.15.151:6827/3079373271","heartbeat_back_addr":"172.21.15.151:6831/3079373271","heartbeat_front_addr":"172.21.15.151:6829/3079373271","state":["exists","up"]},{"osd":3,"uuid":"36edb718-2fcb-4d88-8ddc-c321eed4279d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6817","nonce":1142509041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6819","nonce":1142509041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6824","nonce":1142509041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6821","nonce":1142509041}]},"public_addr":"172.21.15.151:6817/1142509041","cluster_addr":"172.21.15.151:6819/1142509041","heartbeat_back_addr":"172.21.15.151:6824/1142509041","heartbeat_front_addr":"172.21.15.151:6821/1142509041","state":["exists","up"]},{"osd":4,"uuid":"64d33708-cd0a-4236-abe3-1b646e82ab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6811","nonce":2735495271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6813","nonce":2735495271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6817","nonce":2735495271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6815","nonce":2735495271}]},"public_addr":"172.21.15.167:6811/2735495271","cluster_addr":"172.21.15.167:6813/2735495271","heartbeat_back_addr":"172.21.15.167:6817/2735495271","heartbeat_front_addr":"172.21.15.167:6815/2735495271","state":["exists","up"]},{"osd":5,"uuid":"f14b44a9-5910-458c-b518-09a8d0de94de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6827","nonce":1494916572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6829","nonce":1494916572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6833","nonce":1494916572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6831","nonce":1494916572}]},"public_addr":"172.21.15.167:6827/1494916572","cluster_addr":"172.21.15.167:6829/1494916572","heartbeat_back_addr":"172.21.15.167:6833/1494916572","heartbeat_front_addr":"172.21.15.167:6831/1494916572","state":["exists","up"]},{"osd":6,"uuid":"39335545-2654-4710-84b0-a9aa47cfcbba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6819","nonce":2538273273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6821","nonce":2538273273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6825","nonce":2538273273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6823","nonce":2538273273}]},"public_addr":"172.21.15.167:6819/2538273273","cluster_addr":"172.21.15.167:6821/2538273273","heartbeat_back_addr":"172.21.15.167:6825/2538273273","heartbeat_front_addr":"172.21.15.167:6823/2538273273","state":["exists","up"]},{"osd":7,"uuid":"4f8bfc81-53dd-4339-ac22-1a4007d9f963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6803","nonce":265072227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6805","nonce":265072227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6809","nonce":265072227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6807","nonce":265072227}]},"public_addr":"172.21.15.167:6803/265072227","cluster_addr":"172.21.15.167:6805/265072227","heartbeat_back_addr":"172.21.15.167:6809/265072227","heartbeat_front_addr":"172.21.15.167:6807/265072227","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.150359+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.084358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.722368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.332809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.320709+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.395153+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.935250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.801443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.151:6835/1977294668":"2024-09-17T15:54:44.375515+0000","172.21.15.151:6834/1977294668":"2024-09-17T15:54:44.375515+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:54:48.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.770+0000 7f243a200640 1 -- 172.21.15.151:0/3071281893 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f241407aad0 msgr2=0x7f241407cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:48.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.770+0000 7f243a200640 1 --2- 172.21.15.151:0/3071281893 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f241407aad0 0x7f241407cf90 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f2424032000 tx=0x7f2424001360 comp rx=0 tx=0).stop 2024-09-16T15:54:48.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.770+0000 7f243a200640 1 -- 172.21.15.151:0/3071281893 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f243409beb0 msgr2=0x7f2434155390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:48.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.770+0000 7f243a200640 1 --2- 172.21.15.151:0/3071281893 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f243409beb0 0x7f2434155390 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f243406e130 tx=0x7f242c010f40 comp rx=0 tx=0).stop 2024-09-16T15:54:48.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.771+0000 7f243a200640 1 -- 172.21.15.151:0/3071281893 shutdown_connections 2024-09-16T15:54:48.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.771+0000 7f243a200640 1 --2- 172.21.15.151:0/3071281893 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f241407aad0 0x7f241407cf90 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:48.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.771+0000 7f243a200640 1 --2- 172.21.15.151:0/3071281893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f243415a570 0x7f24341f88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:48.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.771+0000 7f243a200640 1 --2- 172.21.15.151:0/3071281893 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f243409beb0 0x7f2434155390 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:48.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.771+0000 7f243a200640 1 --2- 172.21.15.151:0/3071281893 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2418003c10 0x7f24341f5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:48.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.771+0000 7f243a200640 1 -- 172.21.15.151:0/3071281893 >> 172.21.15.151:0/3071281893 conn(0x7f24340b1950 msgr2=0x7f24340cfaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:48.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.771+0000 7f243a200640 1 -- 172.21.15.151:0/3071281893 shutdown_connections 2024-09-16T15:54:48.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.771+0000 7f243a200640 1 -- 172.21.15.151:0/3071281893 wait complete. 2024-09-16T15:54:48.788 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T15:39:11.602252+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-16T15:54:48.788 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-16T15:54:48.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.949+0000 7f4dee200640 1 -- 172.21.15.151:0/4053089657 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4dcc004800 msgr2=0x7f4dcc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:48.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.949+0000 7f4dee200640 1 --2- 172.21.15.151:0/4053089657 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4dcc004800 0x7f4dcc004c80 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f4dd4009b50 tx=0x7f4dcc005830 comp rx=0 tx=0).stop 2024-09-16T15:54:48.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.949+0000 7f4de7e00640 1 -- 172.21.15.151:0/4053089657 reap_dead start 2024-09-16T15:54:48.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.949+0000 7f4dee200640 1 -- 172.21.15.151:0/4053089657 shutdown_connections 2024-09-16T15:54:48.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.949+0000 7f4dee200640 1 -- 172.21.15.151:0/4053089657 >> 172.21.15.151:0/4053089657 conn(0x7f4de80b1840 msgr2=0x7f4de80b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:48.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.949+0000 7f4dee200640 1 -- 172.21.15.151:0/4053089657 shutdown_connections 2024-09-16T15:54:48.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.949+0000 7f4dee200640 1 -- 172.21.15.151:0/4053089657 wait complete. 2024-09-16T15:54:48.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.951+0000 7f4dee200640 1 Processor -- start 2024-09-16T15:54:48.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.951+0000 7f4dee200640 1 -- start start 2024-09-16T15:54:48.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.952+0000 7f4dee200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4dcc004800 0x7f4de813f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:48.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.952+0000 7f4dee200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4de815e0b0 0x7f4de81f7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:48.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.952+0000 7f4dee200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4de81f8310 0x7f4de8141b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:48.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.952+0000 7f4dee200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4de80737d0 con 0x7f4de815e0b0 2024-09-16T15:54:48.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.952+0000 7f4dee200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4de8073650 con 0x7f4dcc004800 2024-09-16T15:54:48.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.952+0000 7f4dee200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f4de80ce0f0 con 0x7f4de81f8310 2024-09-16T15:54:48.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.952+0000 7f4de6a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4de815e0b0 0x7f4de81f7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:48.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.952+0000 7f4de6a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4de815e0b0 0x7f4de81f7dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44390/0 (socket says 172.21.15.151:44390) 2024-09-16T15:54:48.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.952+0000 7f4de6a00640 1 -- 172.21.15.151:0/3975385579 learned_addr learned my addr 172.21.15.151:0/3975385579 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:54:48.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.952+0000 7f4de7400640 1 --2- 172.21.15.151:0/3975385579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4dcc004800 0x7f4de813f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:48.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.952+0000 7f4de7e00640 1 --2- 172.21.15.151:0/3975385579 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4de81f8310 0x7f4de8141b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:48.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.953+0000 7f4de6a00640 1 -- 172.21.15.151:0/3975385579 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4de81f8310 msgr2=0x7f4de8141b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:48.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.953+0000 7f4de6a00640 1 --2- 172.21.15.151:0/3975385579 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4de81f8310 0x7f4de8141b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:48.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.953+0000 7f4de6a00640 1 -- 172.21.15.151:0/3975385579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4dcc004800 msgr2=0x7f4de813f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:48.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.953+0000 7f4de6a00640 1 --2- 172.21.15.151:0/3975385579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4dcc004800 0x7f4de813f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:48.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.953+0000 7f4de6a00640 1 -- 172.21.15.151:0/3975385579 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4de815eb70 con 0x7f4de815e0b0 2024-09-16T15:54:48.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.953+0000 7f4de6a00640 1 --2- 172.21.15.151:0/3975385579 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4de815e0b0 0x7f4de81f7dd0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f4dd4009a70 tx=0x7f4dd4033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:48.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.953+0000 7f4ddf400640 1 -- 172.21.15.151:0/3975385579 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd4043070 con 0x7f4de815e0b0 2024-09-16T15:54:48.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.953+0000 7f4ddf400640 1 -- 172.21.15.151:0/3975385579 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4dd4004320 con 0x7f4de815e0b0 2024-09-16T15:54:48.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.953+0000 7f4dee200640 1 -- 172.21.15.151:0/3975385579 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4de81f89b0 con 0x7f4de815e0b0 2024-09-16T15:54:48.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.953+0000 7f4ddf400640 1 -- 172.21.15.151:0/3975385579 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd4038040 con 0x7f4de815e0b0 2024-09-16T15:54:48.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.953+0000 7f4dee200640 1 -- 172.21.15.151:0/3975385579 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4de813e840 con 0x7f4de815e0b0 2024-09-16T15:54:48.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.954+0000 7f4dee200640 1 -- 172.21.15.151:0/3975385579 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4de81f8720 con 0x7f4de815e0b0 2024-09-16T15:54:48.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.955+0000 7f4ddf400640 1 -- 172.21.15.151:0/3975385579 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 50) ==== 104398+0+0 (secure 0 0 0) 0x7f4dd4046070 con 0x7f4de815e0b0 2024-09-16T15:54:48.957 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.956+0000 7f4ddf400640 1 --2- 172.21.15.151:0/3975385579 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4dc807aad0 0x7f4dc807cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:48.957 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.956+0000 7f4ddf400640 1 -- 172.21.15.151:0/3975385579 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5083+0+0 (secure 0 0 0) 0x7f4dd40b9d40 con 0x7f4de815e0b0 2024-09-16T15:54:48.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.957+0000 7f4de7400640 1 --2- 172.21.15.151:0/3975385579 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4dc807aad0 0x7f4dc807cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:48.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.957+0000 7f4de7400640 1 --2- 172.21.15.151:0/3975385579 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4dc807aad0 0x7f4dc807cf90 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f4dd8001b10 tx=0x7f4dd8006b20 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:48.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:48.960+0000 7f4ddf400640 1 -- 172.21.15.151:0/3975385579 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4de815eb70 con 0x7f4de815e0b0 2024-09-16T15:54:49.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:49.143+0000 7f4dee200640 1 -- 172.21.15.151:0/3975385579 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4de8143d30 con 0x7f4de815e0b0 2024-09-16T15:54:49.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:49.144+0000 7f4ddf400640 1 -- 172.21.15.151:0/3975385579 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v137) ==== 93+0+10 (secure 0 0 0) 0x7f4dd4083860 con 0x7f4de815e0b0 2024-09-16T15:54:49.145 INFO:teuthology.orchestra.run.smithi151.stdout:pg_num: 1 2024-09-16T15:54:49.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:49.146+0000 7f4dee200640 1 -- 172.21.15.151:0/3975385579 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4dc807aad0 msgr2=0x7f4dc807cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:49.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:49.146+0000 7f4dee200640 1 --2- 172.21.15.151:0/3975385579 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4dc807aad0 0x7f4dc807cf90 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f4dd8001b10 tx=0x7f4dd8006b20 comp rx=0 tx=0).stop 2024-09-16T15:54:49.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:49.146+0000 7f4dee200640 1 -- 172.21.15.151:0/3975385579 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4de815e0b0 msgr2=0x7f4de81f7dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:49.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:49.146+0000 7f4dee200640 1 --2- 172.21.15.151:0/3975385579 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4de815e0b0 0x7f4de81f7dd0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f4dd4009a70 tx=0x7f4dd4033670 comp rx=0 tx=0).stop 2024-09-16T15:54:49.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:49.146+0000 7f4dee200640 1 -- 172.21.15.151:0/3975385579 shutdown_connections 2024-09-16T15:54:49.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:49.146+0000 7f4dee200640 1 --2- 172.21.15.151:0/3975385579 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4dc807aad0 0x7f4dc807cf90 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:49.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:49.146+0000 7f4dee200640 1 --2- 172.21.15.151:0/3975385579 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4de81f8310 0x7f4de8141b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:49.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:49.146+0000 7f4dee200640 1 --2- 172.21.15.151:0/3975385579 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4de815e0b0 0x7f4de81f7dd0 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:49.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:49.146+0000 7f4dee200640 1 --2- 172.21.15.151:0/3975385579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4dcc004800 0x7f4de813f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:49.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:49.146+0000 7f4dee200640 1 -- 172.21.15.151:0/3975385579 >> 172.21.15.151:0/3975385579 conn(0x7f4de80b1840 msgr2=0x7f4de80aa410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:49.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:49.146+0000 7f4dee200640 1 -- 172.21.15.151:0/3975385579 shutdown_connections 2024-09-16T15:54:49.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:49.146+0000 7f4dee200640 1 -- 172.21.15.151:0/3975385579 wait complete. 2024-09-16T15:54:49.164 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_more_than_cloner_threads' 2024-09-16T15:54:49.321 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:49.319+0000 7fb2f9c00640 1 -- 172.21.15.151:0/2542189551 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb2f40725d0 msgr2=0x7fb2f4073d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:49.321 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:49.319+0000 7fb2f9c00640 1 --2- 172.21.15.151:0/2542189551 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb2f40725d0 0x7fb2f4073d40 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7fb2f40cd570 tx=0x7fb2dc02a540 comp rx=0 tx=0).stop 2024-09-16T15:54:49.321 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:49.319+0000 7fb2f9c00640 1 -- 172.21.15.151:0/2542189551 shutdown_connections 2024-09-16T15:54:49.321 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:49.319+0000 7fb2f9c00640 1 --2- 172.21.15.151:0/2542189551 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb2f40725d0 0x7fb2f4073d40 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:49.321 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:49.319+0000 7fb2f9c00640 1 --2- 172.21.15.151:0/2542189551 >> v2:172.21.15.167:3300/0 conn(0x7fb2f415bc30 0x7fb2f40b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:49.321 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:49.319+0000 7fb2f9c00640 1 -- 172.21.15.151:0/2542189551 >> 172.21.15.151:0/2542189551 conn(0x7fb2f40cfa20 msgr2=0x7fb2f40cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:49.321 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:49.319+0000 7fb2f9c00640 1 -- 172.21.15.151:0/2542189551 shutdown_connections 2024-09-16T15:54:49.321 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:49.319+0000 7fb2f9c00640 1 -- 172.21.15.151:0/2542189551 wait complete. 2024-09-16T15:54:49.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:49.321+0000 7fb2f9c00640 1 Processor -- start 2024-09-16T15:54:49.322 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:49.321+0000 7fb2f9c00640 1 -- start start 2024-09-16T15:54:49.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:49.322+0000 7fb2f9c00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb2f40d0540 0x7fb2f4156f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:49.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:49.322+0000 7fb2f9c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2f415bc30 0x7fb2f40c5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:49.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:49.322+0000 7fb2f9c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb2f4154e00 0x7fb2f40c5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:49.324 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:49.322+0000 7fb2f9c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f4074610 con 0x7fb2f4154e00 2024-09-16T15:54:49.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:49.322+0000 7fb2f9c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f4074490 con 0x7fb2f415bc30 2024-09-16T15:54:49.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:49.322+0000 7fb2f9c00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fb2f4074790 con 0x7fb2f40d0540 2024-09-16T15:54:49.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:49.322+0000 7fb2f9200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb2f4154e00 0x7fb2f40c5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:49.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:49.322+0000 7fb2f9200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb2f4154e00 0x7fb2f40c5f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44394/0 (socket says 172.21.15.151:44394) 2024-09-16T15:54:49.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:49.322+0000 7fb2f9200640 1 -- 172.21.15.151:0/2303915617 learned_addr learned my addr 172.21.15.151:0/2303915617 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:54:49.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:49.322+0000 7fb2f3e00640 1 --2- 172.21.15.151:0/2303915617 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb2f40d0540 0x7fb2f4156f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:49.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:49.322+0000 7fb2ebe00640 1 --2- 172.21.15.151:0/2303915617 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2f415bc30 0x7fb2f40c5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:49.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:49.322+0000 7fb2f9200640 1 -- 172.21.15.151:0/2303915617 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb2f40d0540 msgr2=0x7fb2f4156f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:49.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:49.322+0000 7fb2f9200640 1 --2- 172.21.15.151:0/2303915617 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb2f40d0540 0x7fb2f4156f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:49.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:49.322+0000 7fb2f9200640 1 -- 172.21.15.151:0/2303915617 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2f415bc30 msgr2=0x7fb2f40c5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:49.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:49.322+0000 7fb2f9200640 1 --2- 172.21.15.151:0/2303915617 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2f415bc30 0x7fb2f40c5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:49.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:49.322+0000 7fb2f9200640 1 -- 172.21.15.151:0/2303915617 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2f40a96c0 con 0x7fb2f4154e00 2024-09-16T15:54:49.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:49.322+0000 7fb2ebe00640 1 --2- 172.21.15.151:0/2303915617 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2f415bc30 0x7fb2f40c5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:54:49.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:49.323+0000 7fb2f9200640 1 --2- 172.21.15.151:0/2303915617 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb2f4154e00 0x7fb2f40c5f80 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec00cbc0 tx=0x7fb2ec00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:49.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:49.323+0000 7fb2f0c00640 1 -- 172.21.15.151:0/2303915617 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec016020 con 0x7fb2f4154e00 2024-09-16T15:54:49.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:49.323+0000 7fb2f0c00640 1 -- 172.21.15.151:0/2303915617 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb2ec00ae30 con 0x7fb2f4154e00 2024-09-16T15:54:49.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:49.323+0000 7fb2f0c00640 1 -- 172.21.15.151:0/2303915617 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec012660 con 0x7fb2f4154e00 2024-09-16T15:54:49.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:49.323+0000 7fb2f9c00640 1 -- 172.21.15.151:0/2303915617 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2f40a99a0 con 0x7fb2f4154e00 2024-09-16T15:54:49.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:49.323+0000 7fb2f9c00640 1 -- 172.21.15.151:0/2303915617 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2f40a9f60 con 0x7fb2f4154e00 2024-09-16T15:54:49.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:49.324+0000 7fb2f9c00640 1 -- 172.21.15.151:0/2303915617 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2f406fd20 con 0x7fb2f4154e00 2024-09-16T15:54:49.326 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:49.325+0000 7fb2f0c00640 1 -- 172.21.15.151:0/2303915617 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 50) ==== 104398+0+0 (secure 0 0 0) 0x7fb2ec019020 con 0x7fb2f4154e00 2024-09-16T15:54:49.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:49.326+0000 7fb2f0c00640 1 --2- 172.21.15.151:0/2303915617 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb2d007aad0 0x7fb2d007cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:49.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:49.326+0000 7fb2f0c00640 1 -- 172.21.15.151:0/2303915617 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5083+0+0 (secure 0 0 0) 0x7fb2ec09a160 con 0x7fb2f4154e00 2024-09-16T15:54:49.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:49.327+0000 7fb2f3e00640 1 --2- 172.21.15.151:0/2303915617 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb2d007aad0 0x7fb2d007cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:49.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:49.327+0000 7fb2f3e00640 1 --2- 172.21.15.151:0/2303915617 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb2d007aad0 0x7fb2d007cf90 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4005fd0 tx=0x7fb2e4005850 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:49.331 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:49.330+0000 7fb2f0c00640 1 -- 172.21.15.151:0/2303915617 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb2ec020020 con 0x7fb2f4154e00 2024-09-16T15:54:49.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:49.583+0000 7fb2f9c00640 1 -- 172.21.15.151:0/2303915617 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_more_than_cloner_threads"]} v 0) -- 0x7fb2f40b2c10 con 0x7fb2f4154e00 2024-09-16T15:54:50.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.401+0000 7fb2f0c00640 1 -- 172.21.15.151:0/2303915617 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_more_than_cloner_threads"]}]=0 v707) ==== 189+0+0 (secure 0 0 0) 0x7fb2ec064230 con 0x7fb2f4154e00 2024-09-16T15:54:50.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.404+0000 7fb2f9c00640 1 -- 172.21.15.151:0/2303915617 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb2d007aad0 msgr2=0x7fb2d007cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:50.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.404+0000 7fb2f9c00640 1 --2- 172.21.15.151:0/2303915617 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb2d007aad0 0x7fb2d007cf90 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4005fd0 tx=0x7fb2e4005850 comp rx=0 tx=0).stop 2024-09-16T15:54:50.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.404+0000 7fb2f9c00640 1 -- 172.21.15.151:0/2303915617 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb2f4154e00 msgr2=0x7fb2f40c5f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:50.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.404+0000 7fb2f9c00640 1 --2- 172.21.15.151:0/2303915617 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb2f4154e00 0x7fb2f40c5f80 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec00cbc0 tx=0x7fb2ec00a430 comp rx=0 tx=0).stop 2024-09-16T15:54:50.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.404+0000 7fb2f9c00640 1 -- 172.21.15.151:0/2303915617 shutdown_connections 2024-09-16T15:54:50.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.404+0000 7fb2f9c00640 1 --2- 172.21.15.151:0/2303915617 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb2d007aad0 0x7fb2d007cf90 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:50.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.404+0000 7fb2f9c00640 1 --2- 172.21.15.151:0/2303915617 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2f415bc30 0x7fb2f40c5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:50.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.404+0000 7fb2f9c00640 1 --2- 172.21.15.151:0/2303915617 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb2f4154e00 0x7fb2f40c5f80 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:50.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.404+0000 7fb2f9c00640 1 --2- 172.21.15.151:0/2303915617 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb2f40d0540 0x7fb2f4156f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:50.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.404+0000 7fb2f9c00640 1 -- 172.21.15.151:0/2303915617 >> 172.21.15.151:0/2303915617 conn(0x7fb2f40cfa20 msgr2=0x7fb2f40cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:50.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.404+0000 7fb2f9c00640 1 -- 172.21.15.151:0/2303915617 shutdown_connections 2024-09-16T15:54:50.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.404+0000 7fb2f9c00640 1 -- 172.21.15.151:0/2303915617 wait complete. 2024-09-16T15:54:50.423 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-16T15:54:50.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.581+0000 7f98c6c00640 1 -- 172.21.15.151:0/1571055907 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f98c015bc30 msgr2=0x7f98c00b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:50.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.581+0000 7f98c6c00640 1 --2- 172.21.15.151:0/1571055907 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f98c015bc30 0x7f98c00b28f0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f98c0073b00 tx=0x7f98b402a8b0 comp rx=0 tx=0).stop 2024-09-16T15:54:50.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.581+0000 7f98c6c00640 1 -- 172.21.15.151:0/1571055907 shutdown_connections 2024-09-16T15:54:50.582 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.581+0000 7f98c6c00640 1 --2- 172.21.15.151:0/1571055907 >> v2:172.21.15.167:3300/0 conn(0x7f98c00725d0 0x7f98c0073d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:50.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.581+0000 7f98c6c00640 1 --2- 172.21.15.151:0/1571055907 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f98c015bc30 0x7f98c00b28f0 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:50.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.581+0000 7f98c6c00640 1 -- 172.21.15.151:0/1571055907 >> 172.21.15.151:0/1571055907 conn(0x7f98c00cfa20 msgr2=0x7f98c00cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:50.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.581+0000 7f98c6c00640 1 -- 172.21.15.151:0/1571055907 shutdown_connections 2024-09-16T15:54:50.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.581+0000 7f98c6c00640 1 -- 172.21.15.151:0/1571055907 wait complete. 2024-09-16T15:54:50.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.583+0000 7f98c6c00640 1 Processor -- start 2024-09-16T15:54:50.584 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.583+0000 7f98c6c00640 1 -- start start 2024-09-16T15:54:50.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.584+0000 7f98c6c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f98c00d0540 0x7f98c0151e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:50.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.584+0000 7f98c6c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98c015bc30 0x7f98c01f4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:50.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.584+0000 7f98c6c00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f98c01f8370 0x7f98c00ab5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:50.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.584+0000 7f98c6c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f98c0074610 con 0x7f98c00d0540 2024-09-16T15:54:50.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.584+0000 7f98c6c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f98c0074490 con 0x7f98c015bc30 2024-09-16T15:54:50.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.584+0000 7f98c6c00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f98c0074790 con 0x7f98c01f8370 2024-09-16T15:54:50.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.584+0000 7f98c5800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f98c00d0540 0x7f98c0151e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:50.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.584+0000 7f98c5800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f98c00d0540 0x7f98c0151e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44422/0 (socket says 172.21.15.151:44422) 2024-09-16T15:54:50.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.584+0000 7f98c5800640 1 -- 172.21.15.151:0/3185763249 learned_addr learned my addr 172.21.15.151:0/3185763249 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:54:50.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.584+0000 7f98c4e00640 1 --2- 172.21.15.151:0/3185763249 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98c015bc30 0x7f98c01f4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:50.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.584+0000 7f98c6200640 1 --2- 172.21.15.151:0/3185763249 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f98c01f8370 0x7f98c00ab5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:50.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.584+0000 7f98c5800640 1 -- 172.21.15.151:0/3185763249 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f98c01f8370 msgr2=0x7f98c00ab5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:50.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.584+0000 7f98c5800640 1 --2- 172.21.15.151:0/3185763249 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f98c01f8370 0x7f98c00ab5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:50.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.584+0000 7f98c5800640 1 -- 172.21.15.151:0/3185763249 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98c015bc30 msgr2=0x7f98c01f4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:50.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.584+0000 7f98c5800640 1 --2- 172.21.15.151:0/3185763249 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98c015bc30 0x7f98c01f4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:50.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.585+0000 7f98c5800640 1 -- 172.21.15.151:0/3185763249 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98c00aae30 con 0x7f98c00d0540 2024-09-16T15:54:50.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.585+0000 7f98c4e00640 1 --2- 172.21.15.151:0/3185763249 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98c015bc30 0x7f98c01f4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:54:50.587 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.585+0000 7f98c6200640 1 --2- 172.21.15.151:0/3185763249 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f98c01f8370 0x7f98c00ab5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:54:50.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.585+0000 7f98c5800640 1 --2- 172.21.15.151:0/3185763249 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f98c00d0540 0x7f98c0151e70 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f98b4005fd0 tx=0x7f98b40316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:50.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.585+0000 7f98bd600640 1 -- 172.21.15.151:0/3185763249 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b4031990 con 0x7f98c00d0540 2024-09-16T15:54:50.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.585+0000 7f98bd600640 1 -- 172.21.15.151:0/3185763249 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f98b4031c20 con 0x7f98c00d0540 2024-09-16T15:54:50.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.585+0000 7f98bd600640 1 -- 172.21.15.151:0/3185763249 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b4033070 con 0x7f98c00d0540 2024-09-16T15:54:50.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.585+0000 7f98c6c00640 1 -- 172.21.15.151:0/3185763249 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98c01f4a30 con 0x7f98c00d0540 2024-09-16T15:54:50.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.585+0000 7f98c6c00640 1 -- 172.21.15.151:0/3185763249 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98c01f4ff0 con 0x7f98c00d0540 2024-09-16T15:54:50.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.586+0000 7f98c6c00640 1 -- 172.21.15.151:0/3185763249 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98c006fd20 con 0x7f98c00d0540 2024-09-16T15:54:50.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.588+0000 7f98bd600640 1 -- 172.21.15.151:0/3185763249 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 50) ==== 104398+0+0 (secure 0 0 0) 0x7f98b4033210 con 0x7f98c00d0540 2024-09-16T15:54:50.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.588+0000 7f98bd600640 1 --2- 172.21.15.151:0/3185763249 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f98a007aaa0 0x7f98a007cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:50.589 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.589+0000 7f98bd600640 1 -- 172.21.15.151:0/3185763249 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5083+0+0 (secure 0 0 0) 0x7f98b40baed0 con 0x7f98c00d0540 2024-09-16T15:54:50.590 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.589+0000 7f98c4e00640 1 --2- 172.21.15.151:0/3185763249 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f98a007aaa0 0x7f98a007cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:50.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.590+0000 7f98c4e00640 1 --2- 172.21.15.151:0/3185763249 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f98a007aaa0 0x7f98a007cf60 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f98ac0058c0 tx=0x7f98ac005850 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:50.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.593+0000 7f98bd600640 1 -- 172.21.15.151:0/3185763249 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f98b403f030 con 0x7f98c00d0540 2024-09-16T15:54:50.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.779+0000 7f98c6c00640 1 -- 172.21.15.151:0/3185763249 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f98c00ab1d0 con 0x7f98c00d0540 2024-09-16T15:54:50.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.780+0000 7f98bd600640 1 -- 172.21.15.151:0/3185763249 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v53) ==== 59+0+1998 (secure 0 0 0) 0x7f98b4084fa0 con 0x7f98c00d0540 2024-09-16T15:54:50.781 INFO:teuthology.orchestra.run.smithi151.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-16T15:54:50.781 INFO:teuthology.orchestra.run.smithi151.stdout:mon advanced mon_op_complaint_time 120 2024-09-16T15:54:50.781 INFO:teuthology.orchestra.run.smithi151.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-16T15:54:50.781 INFO:teuthology.orchestra.run.smithi151.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-16T15:54:50.781 INFO:teuthology.orchestra.run.smithi151.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3795.338492 2024-09-16T15:54:50.781 INFO:teuthology.orchestra.run.smithi151.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 4030.355715 2024-09-16T15:54:50.781 INFO:teuthology.orchestra.run.smithi151.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3817.533640 2024-09-16T15:54:50.781 INFO:teuthology.orchestra.run.smithi151.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3950.656136 2024-09-16T15:54:50.782 INFO:teuthology.orchestra.run.smithi151.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4449.408052 2024-09-16T15:54:50.782 INFO:teuthology.orchestra.run.smithi151.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4439.932296 2024-09-16T15:54:50.782 INFO:teuthology.orchestra.run.smithi151.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4566.875579 2024-09-16T15:54:50.782 INFO:teuthology.orchestra.run.smithi151.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 6320.073632 2024-09-16T15:54:50.782 INFO:teuthology.orchestra.run.smithi151.stdout:mds advanced debug_mds 20/20 2024-09-16T15:54:50.782 INFO:teuthology.orchestra.run.smithi151.stdout:mds advanced debug_mds_balancer 20/20 2024-09-16T15:54:50.782 INFO:teuthology.orchestra.run.smithi151.stdout:mds advanced debug_ms 2/2 2024-09-16T15:54:50.782 INFO:teuthology.orchestra.run.smithi151.stdout:mds dev mds_debug_frag true 2024-09-16T15:54:50.782 INFO:teuthology.orchestra.run.smithi151.stdout:mds dev mds_debug_scatterstat true 2024-09-16T15:54:50.782 INFO:teuthology.orchestra.run.smithi151.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-16T15:54:50.782 INFO:teuthology.orchestra.run.smithi151.stdout:mds dev mds_verify_scatter true 2024-09-16T15:54:50.782 INFO:teuthology.orchestra.run.smithi151.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-16T15:54:50.782 INFO:teuthology.orchestra.run.smithi151.stdout:mds advanced rados_mon_op_timeout 900 2024-09-16T15:54:50.782 INFO:teuthology.orchestra.run.smithi151.stdout:mds advanced rados_osd_op_timeout 900 2024-09-16T15:54:50.783 INFO:teuthology.orchestra.run.smithi151.stdout:client advanced client_mount_timeout 600 2024-09-16T15:54:50.783 INFO:teuthology.orchestra.run.smithi151.stdout:client advanced debug_client 20/20 2024-09-16T15:54:50.783 INFO:teuthology.orchestra.run.smithi151.stdout:client advanced debug_ms 1/1 2024-09-16T15:54:50.783 INFO:teuthology.orchestra.run.smithi151.stdout:client advanced rados_mon_op_timeout 900 2024-09-16T15:54:50.783 INFO:teuthology.orchestra.run.smithi151.stdout:client advanced rados_osd_op_timeout 900 2024-09-16T15:54:50.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.782+0000 7f98c6c00640 1 -- 172.21.15.151:0/3185763249 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f98a007aaa0 msgr2=0x7f98a007cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:50.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.782+0000 7f98c6c00640 1 --2- 172.21.15.151:0/3185763249 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f98a007aaa0 0x7f98a007cf60 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f98ac0058c0 tx=0x7f98ac005850 comp rx=0 tx=0).stop 2024-09-16T15:54:50.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.782+0000 7f98c6c00640 1 -- 172.21.15.151:0/3185763249 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f98c00d0540 msgr2=0x7f98c0151e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:50.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.782+0000 7f98c6c00640 1 --2- 172.21.15.151:0/3185763249 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f98c00d0540 0x7f98c0151e70 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f98b4005fd0 tx=0x7f98b40316f0 comp rx=0 tx=0).stop 2024-09-16T15:54:50.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.782+0000 7f98c6c00640 1 -- 172.21.15.151:0/3185763249 shutdown_connections 2024-09-16T15:54:50.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.782+0000 7f98c6c00640 1 --2- 172.21.15.151:0/3185763249 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f98a007aaa0 0x7f98a007cf60 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:50.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.782+0000 7f98c6c00640 1 --2- 172.21.15.151:0/3185763249 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f98c015bc30 0x7f98c01f4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:50.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.782+0000 7f98c6c00640 1 --2- 172.21.15.151:0/3185763249 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f98c01f8370 0x7f98c00ab5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:50.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.782+0000 7f98c6c00640 1 --2- 172.21.15.151:0/3185763249 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f98c00d0540 0x7f98c0151e70 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:50.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.782+0000 7f98c6c00640 1 -- 172.21.15.151:0/3185763249 >> 172.21.15.151:0/3185763249 conn(0x7f98c00cfa20 msgr2=0x7f98c00b35e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:50.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.782+0000 7f98c6c00640 1 -- 172.21.15.151:0/3185763249 shutdown_connections 2024-09-16T15:54:50.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.782+0000 7f98c6c00640 1 -- 172.21.15.151:0/3185763249 wait complete. 2024-09-16T15:54:50.799 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-16T15:54:50.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.959+0000 7fd0e6800640 1 -- 172.21.15.151:0/4240653947 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0e00cccd0 msgr2=0x7fd0e0073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:50.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.959+0000 7fd0e6800640 1 --2- 172.21.15.151:0/4240653947 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0e00cccd0 0x7fd0e0073440 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc009830 tx=0x7fd0cc02a480 comp rx=0 tx=0).stop 2024-09-16T15:54:50.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.959+0000 7fd0e6800640 1 -- 172.21.15.151:0/4240653947 shutdown_connections 2024-09-16T15:54:50.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.959+0000 7fd0e6800640 1 --2- 172.21.15.151:0/4240653947 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0e006efa0 0x7fd0e00cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:50.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.959+0000 7fd0e6800640 1 --2- 172.21.15.151:0/4240653947 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0e00cccd0 0x7fd0e0073440 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:50.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.959+0000 7fd0e6800640 1 -- 172.21.15.151:0/4240653947 >> 172.21.15.151:0/4240653947 conn(0x7fd0e00b1840 msgr2=0x7fd0e00b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:50.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.960+0000 7fd0e6800640 1 -- 172.21.15.151:0/4240653947 shutdown_connections 2024-09-16T15:54:50.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.960+0000 7fd0e6800640 1 -- 172.21.15.151:0/4240653947 wait complete. 2024-09-16T15:54:50.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.961+0000 7fd0e6800640 1 Processor -- start 2024-09-16T15:54:50.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.961+0000 7fd0e6800640 1 -- start start 2024-09-16T15:54:50.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.962+0000 7fd0e6800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0e006efa0 0x7fd0e0152440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:50.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.962+0000 7fd0e6800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0e00cccd0 0x7fd0e00cbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:50.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.962+0000 7fd0e6800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0e0145750 0x7fd0e00c4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:50.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.962+0000 7fd0e6800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e00ce480 con 0x7fd0e006efa0 2024-09-16T15:54:50.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.962+0000 7fd0e6800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e00ce300 con 0x7fd0e00cccd0 2024-09-16T15:54:50.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.962+0000 7fd0e6800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fd0e00ce600 con 0x7fd0e0145750 2024-09-16T15:54:50.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.962+0000 7fd0df400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0e006efa0 0x7fd0e0152440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:50.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.962+0000 7fd0df400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0e006efa0 0x7fd0e0152440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44434/0 (socket says 172.21.15.151:44434) 2024-09-16T15:54:50.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.962+0000 7fd0df400640 1 -- 172.21.15.151:0/370855207 learned_addr learned my addr 172.21.15.151:0/370855207 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:54:50.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.963+0000 7fd0dea00640 1 --2- 172.21.15.151:0/370855207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0e00cccd0 0x7fd0e00cbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:50.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.963+0000 7fd0dfe00640 1 --2- 172.21.15.151:0/370855207 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0e0145750 0x7fd0e00c4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:50.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.963+0000 7fd0df400640 1 -- 172.21.15.151:0/370855207 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0e0145750 msgr2=0x7fd0e00c4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:50.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.963+0000 7fd0df400640 1 --2- 172.21.15.151:0/370855207 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0e0145750 0x7fd0e00c4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:50.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.963+0000 7fd0df400640 1 -- 172.21.15.151:0/370855207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0e00cccd0 msgr2=0x7fd0e00cbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:50.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.963+0000 7fd0df400640 1 --2- 172.21.15.151:0/370855207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0e00cccd0 0x7fd0e00cbcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:50.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.963+0000 7fd0df400640 1 -- 172.21.15.151:0/370855207 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0e01f4e20 con 0x7fd0e006efa0 2024-09-16T15:54:50.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.963+0000 7fd0df400640 1 --2- 172.21.15.151:0/370855207 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0e006efa0 0x7fd0e0152440 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7fd0d000c3d0 tx=0x7fd0d000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:50.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.963+0000 7fd0d7400640 1 -- 172.21.15.151:0/370855207 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d0011790 con 0x7fd0e006efa0 2024-09-16T15:54:50.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.963+0000 7fd0d7400640 1 -- 172.21.15.151:0/370855207 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd0d0017070 con 0x7fd0e006efa0 2024-09-16T15:54:50.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.963+0000 7fd0d7400640 1 -- 172.21.15.151:0/370855207 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d0011cc0 con 0x7fd0e006efa0 2024-09-16T15:54:50.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.963+0000 7fd0e6800640 1 -- 172.21.15.151:0/370855207 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0e01f5100 con 0x7fd0e006efa0 2024-09-16T15:54:50.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.964+0000 7fd0e6800640 1 -- 172.21.15.151:0/370855207 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0e01f56c0 con 0x7fd0e006efa0 2024-09-16T15:54:50.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.965+0000 7fd0e6800640 1 -- 172.21.15.151:0/370855207 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0e00cdbb0 con 0x7fd0e006efa0 2024-09-16T15:54:50.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.966+0000 7fd0d7400640 1 -- 172.21.15.151:0/370855207 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 50) ==== 104398+0+0 (secure 0 0 0) 0x7fd0d001c460 con 0x7fd0e006efa0 2024-09-16T15:54:50.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.967+0000 7fd0d7400640 1 --2- 172.21.15.151:0/370855207 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd0c007aaa0 0x7fd0c007cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:50.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.967+0000 7fd0d7400640 1 -- 172.21.15.151:0/370855207 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5083+0+0 (secure 0 0 0) 0x7fd0d0099690 con 0x7fd0e006efa0 2024-09-16T15:54:50.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.967+0000 7fd0dea00640 1 --2- 172.21.15.151:0/370855207 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd0c007aaa0 0x7fd0c007cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:50.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.968+0000 7fd0dea00640 1 --2- 172.21.15.151:0/370855207 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd0c007aaa0 0x7fd0c007cf60 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc034000 tx=0x7fd0cc004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:50.972 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:50.971+0000 7fd0d7400640 1 -- 172.21.15.151:0/370855207 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd0d00631b0 con 0x7fd0e006efa0 2024-09-16T15:54:51.161 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:51.160+0000 7fd0e6800640 1 -- 172.21.15.151:0/370855207 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fd0e0152980 con 0x7fd0e006efa0 2024-09-16T15:54:51.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:51.164+0000 7fd0d7400640 1 -- 172.21.15.151:0/370855207 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v54)=0 v54) ==== 122+0+0 (secure 0 0 0) 0x7fd0d0067be0 con 0x7fd0e006efa0 2024-09-16T15:54:51.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:51.167+0000 7fd0e6800640 1 -- 172.21.15.151:0/370855207 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd0c007aaa0 msgr2=0x7fd0c007cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:51.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:51.167+0000 7fd0e6800640 1 --2- 172.21.15.151:0/370855207 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd0c007aaa0 0x7fd0c007cf60 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc034000 tx=0x7fd0cc004730 comp rx=0 tx=0).stop 2024-09-16T15:54:51.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:51.167+0000 7fd0e6800640 1 -- 172.21.15.151:0/370855207 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0e006efa0 msgr2=0x7fd0e0152440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:51.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:51.167+0000 7fd0e6800640 1 --2- 172.21.15.151:0/370855207 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0e006efa0 0x7fd0e0152440 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7fd0d000c3d0 tx=0x7fd0d000c890 comp rx=0 tx=0).stop 2024-09-16T15:54:51.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:51.167+0000 7fd0e6800640 1 -- 172.21.15.151:0/370855207 shutdown_connections 2024-09-16T15:54:51.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:51.167+0000 7fd0e6800640 1 --2- 172.21.15.151:0/370855207 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd0c007aaa0 0x7fd0c007cf60 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:51.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:51.167+0000 7fd0e6800640 1 --2- 172.21.15.151:0/370855207 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0e00cccd0 0x7fd0e00cbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:51.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:51.167+0000 7fd0e6800640 1 --2- 172.21.15.151:0/370855207 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0e0145750 0x7fd0e00c4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:51.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:51.167+0000 7fd0e6800640 1 --2- 172.21.15.151:0/370855207 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0e006efa0 0x7fd0e0152440 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:51.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:51.167+0000 7fd0e6800640 1 -- 172.21.15.151:0/370855207 >> 172.21.15.151:0/370855207 conn(0x7fd0e00b1840 msgr2=0x7fd0e015f200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:51.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:51.167+0000 7fd0e6800640 1 -- 172.21.15.151:0/370855207 shutdown_connections 2024-09-16T15:54:51.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:51.167+0000 7fd0e6800640 1 -- 172.21.15.151:0/370855207 wait complete. 2024-09-16T15:54:51.184 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-16T15:54:51.184 DEBUG:teuthology.orchestra.run.smithi151:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:54:51.273 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:54:51.373 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-16T15:54:51.373 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:54:51.374 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:54:51.410 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T15:54:51.410 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-16T15:54:51.410 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:54:51.447 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:54:51.447 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:54:51.447 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:54:51.485 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T15:54:51.485 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-16T15:54:51.485 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:54:51.524 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:54:51.524 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-16T15:54:51.524 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:54:51.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:51.683+0000 7f59b7e00640 1 -- 172.21.15.151:0/1916961940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59b80b28d0 msgr2=0x7f59b80d22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:51.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:51.683+0000 7f59b7e00640 1 --2- 172.21.15.151:0/1916961940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59b80b28d0 0x7f59b80d22a0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f59a4009830 tx=0x7f59a402a5c0 comp rx=0 tx=0).stop 2024-09-16T15:54:51.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:51.683+0000 7f59b7e00640 1 -- 172.21.15.151:0/1916961940 shutdown_connections 2024-09-16T15:54:51.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:51.683+0000 7f59b7e00640 1 --2- 172.21.15.151:0/1916961940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59b80b28d0 0x7f59b80d22a0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:51.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:51.683+0000 7f59b7e00640 1 -- 172.21.15.151:0/1916961940 >> 172.21.15.151:0/1916961940 conn(0x7f59b80d3de0 msgr2=0x7f59b80d41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:51.685 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:51.683+0000 7f59b7e00640 1 -- 172.21.15.151:0/1916961940 shutdown_connections 2024-09-16T15:54:51.685 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:51.683+0000 7f59b7e00640 1 -- 172.21.15.151:0/1916961940 wait complete. 2024-09-16T15:54:51.685 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:51.684+0000 7f59b7e00640 1 Processor -- start 2024-09-16T15:54:51.685 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:51.685+0000 7f59b7e00640 1 -- start start 2024-09-16T15:54:51.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:51.685+0000 7f59b7e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f59b80b28d0 0x7f59b814ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:51.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:51.686+0000 7f59b7e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59b80b4d10 0x7f59b8160f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:51.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:51.686+0000 7f59b7e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f59b80d1ec0 0x7f59b80cd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:51.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:51.686+0000 7f59b7e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f59b81692e0 con 0x7f59b80b28d0 2024-09-16T15:54:51.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:51.686+0000 7f59b7e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f59b80b4a00 con 0x7f59b80b4d10 2024-09-16T15:54:51.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:51.686+0000 7f59b7e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f59b8169460 con 0x7f59b80d1ec0 2024-09-16T15:54:51.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:51.686+0000 7f59b6a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f59b80b28d0 0x7f59b814ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:51.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:51.686+0000 7f59b6a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f59b80b28d0 0x7f59b814ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44444/0 (socket says 172.21.15.151:44444) 2024-09-16T15:54:51.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:51.686+0000 7f59b6a00640 1 -- 172.21.15.151:0/4062611636 learned_addr learned my addr 172.21.15.151:0/4062611636 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:54:51.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:51.686+0000 7f59b6000640 1 --2- 172.21.15.151:0/4062611636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59b80b4d10 0x7f59b8160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:51.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:51.686+0000 7f59b7400640 1 --2- 172.21.15.151:0/4062611636 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f59b80d1ec0 0x7f59b80cd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:51.689 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:51.686+0000 7f59b6a00640 1 -- 172.21.15.151:0/4062611636 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f59b80d1ec0 msgr2=0x7f59b80cd9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:51.689 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:51.686+0000 7f59b6a00640 1 --2- 172.21.15.151:0/4062611636 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f59b80d1ec0 0x7f59b80cd9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:51.689 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:51.686+0000 7f59b6a00640 1 -- 172.21.15.151:0/4062611636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59b80b4d10 msgr2=0x7f59b8160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:51.689 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:51.686+0000 7f59b6a00640 1 --2- 172.21.15.151:0/4062611636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59b80b4d10 0x7f59b8160f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:51.689 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:51.686+0000 7f59b6a00640 1 -- 172.21.15.151:0/4062611636 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59b815fc20 con 0x7f59b80b28d0 2024-09-16T15:54:51.689 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:51.686+0000 7f59b6a00640 1 --2- 172.21.15.151:0/4062611636 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f59b80b28d0 0x7f59b814ded0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f59a800c3a0 tx=0x7f59a800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:51.689 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:51.687+0000 7f59aea00640 1 -- 172.21.15.151:0/4062611636 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a8017070 con 0x7f59b80b28d0 2024-09-16T15:54:51.689 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:51.687+0000 7f59aea00640 1 -- 172.21.15.151:0/4062611636 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f59a80053c0 con 0x7f59b80b28d0 2024-09-16T15:54:51.689 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:51.687+0000 7f59aea00640 1 -- 172.21.15.151:0/4062611636 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a8003850 con 0x7f59b80b28d0 2024-09-16T15:54:51.689 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:51.687+0000 7f59b7e00640 1 -- 172.21.15.151:0/4062611636 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59b815fec0 con 0x7f59b80b28d0 2024-09-16T15:54:51.689 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:51.687+0000 7f59b7e00640 1 -- 172.21.15.151:0/4062611636 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59b8160480 con 0x7f59b80b28d0 2024-09-16T15:54:51.690 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:51.688+0000 7f59b7e00640 1 -- 172.21.15.151:0/4062611636 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59b80d22a0 con 0x7f59b80b28d0 2024-09-16T15:54:51.690 DEBUG:teuthology.orchestra.run.smithi151:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:54:51.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:51.689+0000 7f59aea00640 1 -- 172.21.15.151:0/4062611636 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 50) ==== 104398+0+0 (secure 0 0 0) 0x7f59a801a070 con 0x7f59b80b28d0 2024-09-16T15:54:51.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:51.690+0000 7f59aea00640 1 --2- 172.21.15.151:0/4062611636 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f599807aaa0 0x7f599807cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:51.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:51.690+0000 7f59aea00640 1 -- 172.21.15.151:0/4062611636 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5083+0+0 (secure 0 0 0) 0x7f59a8098d00 con 0x7f59b80b28d0 2024-09-16T15:54:51.694 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:54:51.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:51.691+0000 7f59b6000640 1 --2- 172.21.15.151:0/4062611636 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f599807aaa0 0x7f599807cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:51.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:51.691+0000 7f59b6000640 1 --2- 172.21.15.151:0/4062611636 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f599807aaa0 0x7f599807cf60 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f59a4034000 tx=0x7f59a4004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:51.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:51.694+0000 7f59aea00640 1 -- 172.21.15.151:0/4062611636 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f59a8062820 con 0x7f59b80b28d0 2024-09-16T15:54:51.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:51.919+0000 7f59b7e00640 1 -- 172.21.15.151:0/4062611636 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f59b80b4020 con 0x7f59b80b28d0 2024-09-16T15:54:51.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:51.919+0000 7f59aea00640 1 -- 172.21.15.151:0/4062611636 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) ==== 94+0+3920 (secure 0 0 0) 0x7f59a8067250 con 0x7f59b80b28d0 2024-09-16T15:54:51.921 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:54:51.921 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":90,"btime":"2024-09-16T15:54:45:377608+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8994,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/2450183746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":2450183746},{"type":"v1","addr":"172.21.15.151:6833","nonce":2450183746}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":8997,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/1952630294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":1952630294},{"type":"v1","addr":"172.21.15.167:6836","nonce":1952630294}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":9000,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/127383056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":127383056},{"type":"v1","addr":"172.21.15.167:6837","nonce":127383056}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":9372,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2899304083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2899304083},{"type":"v1","addr":"172.21.15.151:6835","nonce":2899304083}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90}],"filesystems":[]} 2024-09-16T15:54:51.921 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 90 2024-09-16T15:54:51.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:51.922+0000 7f59b7e00640 1 -- 172.21.15.151:0/4062611636 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f599807aaa0 msgr2=0x7f599807cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:51.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:51.922+0000 7f59b7e00640 1 --2- 172.21.15.151:0/4062611636 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f599807aaa0 0x7f599807cf60 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f59a4034000 tx=0x7f59a4004730 comp rx=0 tx=0).stop 2024-09-16T15:54:51.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:51.922+0000 7f59b7e00640 1 -- 172.21.15.151:0/4062611636 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f59b80b28d0 msgr2=0x7f59b814ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:51.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:51.922+0000 7f59b7e00640 1 --2- 172.21.15.151:0/4062611636 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f59b80b28d0 0x7f59b814ded0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f59a800c3a0 tx=0x7f59a800c860 comp rx=0 tx=0).stop 2024-09-16T15:54:51.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:51.922+0000 7f59b7e00640 1 -- 172.21.15.151:0/4062611636 shutdown_connections 2024-09-16T15:54:51.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:51.922+0000 7f59b7e00640 1 --2- 172.21.15.151:0/4062611636 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f599807aaa0 0x7f599807cf60 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:51.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:51.922+0000 7f59b7e00640 1 --2- 172.21.15.151:0/4062611636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59b80b4d10 0x7f59b8160f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:51.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:51.922+0000 7f59b7e00640 1 --2- 172.21.15.151:0/4062611636 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f59b80d1ec0 0x7f59b80cd9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:51.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:51.922+0000 7f59b7e00640 1 --2- 172.21.15.151:0/4062611636 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f59b80b28d0 0x7f59b814ded0 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:51.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:51.922+0000 7f59b7e00640 1 -- 172.21.15.151:0/4062611636 >> 172.21.15.151:0/4062611636 conn(0x7f59b80d3de0 msgr2=0x7f59b806e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:51.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:51.922+0000 7f59b7e00640 1 -- 172.21.15.151:0/4062611636 shutdown_connections 2024-09-16T15:54:51.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:51.922+0000 7f59b7e00640 1 -- 172.21.15.151:0/4062611636 wait complete. 2024-09-16T15:54:51.940 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-16T15:54:51.940 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-16T15:54:51.940 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-16T15:54:51.940 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:54:51.940 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-16T15:54:51.940 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-16T15:54:51.941 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-16T15:54:51.941 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:54:51.941 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-16T15:54:51.941 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-16T15:54:51.941 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-16T15:54:51.941 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:54:51.941 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-16T15:54:51.942 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-16T15:54:51.942 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-16T15:54:51.942 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:54:51.955 INFO:tasks.ceph.mds.a:Stopped 2024-09-16T15:54:51.956 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-16T15:54:51.956 INFO:tasks.ceph.mds.c:Stopped 2024-09-16T15:54:51.957 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-16T15:54:51.958 INFO:tasks.ceph.mds.b:Stopped 2024-09-16T15:54:51.958 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-16T15:54:51.959 INFO:tasks.ceph.mds.d:Stopped 2024-09-16T15:54:51.959 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-16T15:54:52.001 INFO:tasks.ceph.mds.a:Started 2024-09-16T15:54:52.001 DEBUG:teuthology.parallel:result is None 2024-09-16T15:54:52.003 INFO:tasks.ceph.mds.b:Started 2024-09-16T15:54:52.003 DEBUG:teuthology.parallel:result is None 2024-09-16T15:54:52.004 INFO:tasks.ceph.mds.c:Started 2024-09-16T15:54:52.004 DEBUG:teuthology.parallel:result is None 2024-09-16T15:54:52.005 INFO:tasks.ceph.mds.d:Started 2024-09-16T15:54:52.005 DEBUG:teuthology.parallel:result is None 2024-09-16T15:54:52.006 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-16T15:54:52.132 INFO:tasks.ceph.mds.a.smithi151.stdout:starting mds.a at 2024-09-16T15:54:52.132 INFO:tasks.ceph.mds.b.smithi167.stdout:starting mds.b at 2024-09-16T15:54:52.133 INFO:tasks.ceph.mds.c.smithi151.stdout:starting mds.c at 2024-09-16T15:54:52.133 INFO:tasks.ceph.mds.d.smithi167.stdout:starting mds.d at 2024-09-16T15:54:52.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.171+0000 7f941fe00640 1 -- 172.21.15.151:0/3716107037 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f94200b5370 msgr2=0x7f94200b3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:52.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.171+0000 7f941fe00640 1 --2- 172.21.15.151:0/3716107037 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f94200b5370 0x7f94200b3bc0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f94200d4ea0 tx=0x7f940c02a510 comp rx=0 tx=0).stop 2024-09-16T15:54:52.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.171+0000 7f941fe00640 1 -- 172.21.15.151:0/3716107037 shutdown_connections 2024-09-16T15:54:52.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.171+0000 7f941fe00640 1 --2- 172.21.15.151:0/3716107037 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f94200b5370 0x7f94200b3bc0 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:52.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.171+0000 7f941fe00640 1 -- 172.21.15.151:0/3716107037 >> 172.21.15.151:0/3716107037 conn(0x7f94200d3df0 msgr2=0x7f94200d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:52.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.171+0000 7f941fe00640 1 -- 172.21.15.151:0/3716107037 shutdown_connections 2024-09-16T15:54:52.172 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.171+0000 7f941fe00640 1 -- 172.21.15.151:0/3716107037 wait complete. 2024-09-16T15:54:52.174 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.172+0000 7f941fe00640 1 Processor -- start 2024-09-16T15:54:52.174 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.173+0000 7f941fe00640 1 -- start start 2024-09-16T15:54:52.175 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.173+0000 7f941fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f94200b4100 0x7f942014f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:52.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.173+0000 7f941fe00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f94200b5370 0x7f94201f8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:52.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.174+0000 7f941fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9420161440 0x7f9420150ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:52.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.174+0000 7f941fe00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f94200b2210 con 0x7f94200b4100 2024-09-16T15:54:52.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.174+0000 7f941fe00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f94200b2090 con 0x7f9420161440 2024-09-16T15:54:52.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.174+0000 7f941fe00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f94200b2390 con 0x7f94200b5370 2024-09-16T15:54:52.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.174+0000 7f941ea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f94200b4100 0x7f942014f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:52.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.174+0000 7f941ea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f94200b4100 0x7f942014f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44500/0 (socket says 172.21.15.151:44500) 2024-09-16T15:54:52.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.174+0000 7f941ea00640 1 -- 172.21.15.151:0/3633449390 learned_addr learned my addr 172.21.15.151:0/3633449390 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:54:52.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.174+0000 7f941f400640 1 --2- 172.21.15.151:0/3633449390 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9420161440 0x7f9420150ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:52.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.174+0000 7f941e000640 1 --2- 172.21.15.151:0/3633449390 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f94200b5370 0x7f94201f8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:52.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.174+0000 7f941ea00640 1 -- 172.21.15.151:0/3633449390 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f94200b5370 msgr2=0x7f94201f8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:52.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.174+0000 7f941ea00640 1 --2- 172.21.15.151:0/3633449390 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f94200b5370 0x7f94201f8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:52.176 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.174+0000 7f941ea00640 1 -- 172.21.15.151:0/3633449390 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9420161440 msgr2=0x7f9420150ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:52.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.174+0000 7f941ea00640 1 --2- 172.21.15.151:0/3633449390 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9420161440 0x7f9420150ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:52.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.174+0000 7f941ea00640 1 -- 172.21.15.151:0/3633449390 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9420151400 con 0x7f94200b4100 2024-09-16T15:54:52.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.174+0000 7f941ea00640 1 --2- 172.21.15.151:0/3633449390 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f94200b4100 0x7f942014f460 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f941000c430 tx=0x7f941000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:52.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.175+0000 7f9416a00640 1 -- 172.21.15.151:0/3633449390 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9410017070 con 0x7f94200b4100 2024-09-16T15:54:52.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.175+0000 7f9416a00640 1 -- 172.21.15.151:0/3633449390 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f94100053e0 con 0x7f94200b4100 2024-09-16T15:54:52.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.175+0000 7f9416a00640 1 -- 172.21.15.151:0/3633449390 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9410003850 con 0x7f94200b4100 2024-09-16T15:54:52.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.175+0000 7f941fe00640 1 -- 172.21.15.151:0/3633449390 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94200b8a90 con 0x7f94200b4100 2024-09-16T15:54:52.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.175+0000 7f941fe00640 1 -- 172.21.15.151:0/3633449390 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94200b9050 con 0x7f94200b4100 2024-09-16T15:54:52.177 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.176+0000 7f941fe00640 1 -- 172.21.15.151:0/3633449390 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f942015f7a0 con 0x7f94200b4100 2024-09-16T15:54:52.178 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.177+0000 7f9416a00640 1 -- 172.21.15.151:0/3633449390 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 50) ==== 104398+0+0 (secure 0 0 0) 0x7f941001a070 con 0x7f94200b4100 2024-09-16T15:54:52.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.178+0000 7f9416a00640 1 --2- 172.21.15.151:0/3633449390 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f940007aad0 0x7f940007cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:52.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.178+0000 7f9416a00640 1 -- 172.21.15.151:0/3633449390 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5083+0+0 (secure 0 0 0) 0x7f9410098d20 con 0x7f94200b4100 2024-09-16T15:54:52.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.178+0000 7f941e000640 1 --2- 172.21.15.151:0/3633449390 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f940007aad0 0x7f940007cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:52.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.179+0000 7f941e000640 1 --2- 172.21.15.151:0/3633449390 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f940007aad0 0x7f940007cf90 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f940c034000 tx=0x7f940c004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:52.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.182+0000 7f9416a00640 1 -- 172.21.15.151:0/3633449390 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f942015f7a0 con 0x7f94200b4100 2024-09-16T15:54:52.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.382+0000 7f941fe00640 1 -- 172.21.15.151:0/3633449390 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f94200b2840 con 0x7f94200b4100 2024-09-16T15:54:52.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.423+0000 7f9416a00640 1 -- 172.21.15.151:0/3633449390 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v139) ==== 97+0+0 (secure 0 0 0) 0x7f94100627a0 con 0x7f94200b4100 2024-09-16T15:54:52.425 INFO:teuthology.orchestra.run.smithi151.stderr: removed all blocklist entries 2024-09-16T15:54:52.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.426+0000 7f941fe00640 1 -- 172.21.15.151:0/3633449390 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f940007aad0 msgr2=0x7f940007cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:52.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.426+0000 7f941fe00640 1 --2- 172.21.15.151:0/3633449390 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f940007aad0 0x7f940007cf90 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f940c034000 tx=0x7f940c004730 comp rx=0 tx=0).stop 2024-09-16T15:54:52.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.426+0000 7f941fe00640 1 -- 172.21.15.151:0/3633449390 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f94200b4100 msgr2=0x7f942014f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:52.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.426+0000 7f941fe00640 1 --2- 172.21.15.151:0/3633449390 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f94200b4100 0x7f942014f460 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f941000c430 tx=0x7f941000c8f0 comp rx=0 tx=0).stop 2024-09-16T15:54:52.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.426+0000 7f941fe00640 1 -- 172.21.15.151:0/3633449390 shutdown_connections 2024-09-16T15:54:52.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.426+0000 7f941fe00640 1 --2- 172.21.15.151:0/3633449390 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f940007aad0 0x7f940007cf90 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:52.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.426+0000 7f941fe00640 1 --2- 172.21.15.151:0/3633449390 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9420161440 0x7f9420150ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:52.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.426+0000 7f941fe00640 1 --2- 172.21.15.151:0/3633449390 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f94200b5370 0x7f94201f8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:52.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.426+0000 7f941fe00640 1 --2- 172.21.15.151:0/3633449390 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f94200b4100 0x7f942014f460 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:52.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.426+0000 7f941fe00640 1 -- 172.21.15.151:0/3633449390 >> 172.21.15.151:0/3633449390 conn(0x7f94200d3df0 msgr2=0x7f94200d41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:52.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.426+0000 7f941fe00640 1 -- 172.21.15.151:0/3633449390 shutdown_connections 2024-09-16T15:54:52.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.426+0000 7f941fe00640 1 -- 172.21.15.151:0/3633449390 wait complete. 2024-09-16T15:54:52.445 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-16T15:54:52.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.603+0000 7fb34a600640 1 -- 172.21.15.151:0/988910321 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb344073fb0 msgr2=0x7fb344074440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:52.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.603+0000 7fb34a600640 1 --2- 172.21.15.151:0/988910321 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb344073fb0 0x7fb344074440 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7fb33c005fd0 tx=0x7fb33c02aa10 comp rx=0 tx=0).stop 2024-09-16T15:54:52.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.603+0000 7fb34a600640 1 -- 172.21.15.151:0/988910321 shutdown_connections 2024-09-16T15:54:52.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.603+0000 7fb34a600640 1 --2- 172.21.15.151:0/988910321 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb344073fb0 0x7fb344074440 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:52.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.603+0000 7fb34a600640 1 --2- 172.21.15.151:0/988910321 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb3440ccf00 0x7fb3440b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:52.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.603+0000 7fb34a600640 1 -- 172.21.15.151:0/988910321 >> 172.21.15.151:0/988910321 conn(0x7fb3440b1950 msgr2=0x7fb3440b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:52.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.603+0000 7fb34a600640 1 -- 172.21.15.151:0/988910321 shutdown_connections 2024-09-16T15:54:52.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.603+0000 7fb34a600640 1 -- 172.21.15.151:0/988910321 wait complete. 2024-09-16T15:54:52.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.604+0000 7fb34a600640 1 Processor -- start 2024-09-16T15:54:52.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.604+0000 7fb34a600640 1 -- start start 2024-09-16T15:54:52.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.605+0000 7fb34a600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb344073fb0 0x7fb34414e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:52.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.605+0000 7fb34a600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3440ccf00 0x7fb34409b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:52.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.605+0000 7fb34a600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb34415f5a0 0x7fb3441f7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:52.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.605+0000 7fb34a600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb34406f2e0 con 0x7fb34415f5a0 2024-09-16T15:54:52.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.605+0000 7fb34a600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb34406f160 con 0x7fb3440ccf00 2024-09-16T15:54:52.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.605+0000 7fb34a600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fb34406f460 con 0x7fb344073fb0 2024-09-16T15:54:52.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.606+0000 7fb343e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb34415f5a0 0x7fb3441f7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:52.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.606+0000 7fb343e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb34415f5a0 0x7fb3441f7e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44508/0 (socket says 172.21.15.151:44508) 2024-09-16T15:54:52.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.606+0000 7fb343e00640 1 -- 172.21.15.151:0/354647294 learned_addr learned my addr 172.21.15.151:0/354647294 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:54:52.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.606+0000 7fb343400640 1 --2- 172.21.15.151:0/354647294 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb344073fb0 0x7fb34414e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:52.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.606+0000 7fb33be00640 1 --2- 172.21.15.151:0/354647294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3440ccf00 0x7fb34409b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:52.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.606+0000 7fb343e00640 1 -- 172.21.15.151:0/354647294 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb344073fb0 msgr2=0x7fb34414e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:52.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.606+0000 7fb343e00640 1 --2- 172.21.15.151:0/354647294 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb344073fb0 0x7fb34414e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:52.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.606+0000 7fb343e00640 1 -- 172.21.15.151:0/354647294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3440ccf00 msgr2=0x7fb34409b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:52.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.606+0000 7fb343e00640 1 --2- 172.21.15.151:0/354647294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3440ccf00 0x7fb34409b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:52.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.606+0000 7fb343e00640 1 -- 172.21.15.151:0/354647294 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb34414ec20 con 0x7fb34415f5a0 2024-09-16T15:54:52.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.606+0000 7fb343e00640 1 --2- 172.21.15.151:0/354647294 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb34415f5a0 0x7fb3441f7e70 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7fb33c006010 tx=0x7fb33c003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:52.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.607+0000 7fb33b400640 1 -- 172.21.15.151:0/354647294 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33c047070 con 0x7fb34415f5a0 2024-09-16T15:54:52.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.607+0000 7fb33b400640 1 -- 172.21.15.151:0/354647294 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb33c004250 con 0x7fb34415f5a0 2024-09-16T15:54:52.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.607+0000 7fb34a600640 1 -- 172.21.15.151:0/354647294 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb34409c010 con 0x7fb34415f5a0 2024-09-16T15:54:52.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.607+0000 7fb33b400640 1 -- 172.21.15.151:0/354647294 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33c033050 con 0x7fb34415f5a0 2024-09-16T15:54:52.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.607+0000 7fb34a600640 1 -- 172.21.15.151:0/354647294 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb34409c5d0 con 0x7fb34415f5a0 2024-09-16T15:54:52.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.608+0000 7fb34a600640 1 -- 172.21.15.151:0/354647294 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb34406f9f0 con 0x7fb34415f5a0 2024-09-16T15:54:52.610 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.609+0000 7fb33b400640 1 -- 172.21.15.151:0/354647294 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 50) ==== 104398+0+0 (secure 0 0 0) 0x7fb33c04a070 con 0x7fb34415f5a0 2024-09-16T15:54:52.611 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.610+0000 7fb33b400640 1 --2- 172.21.15.151:0/354647294 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb32007aaa0 0x7fb32007cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:52.611 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.610+0000 7fb33b400640 1 -- 172.21.15.151:0/354647294 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 4997+0+0 (secure 0 0 0) 0x7fb33c0bad30 con 0x7fb34415f5a0 2024-09-16T15:54:52.611 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.611+0000 7fb343400640 1 --2- 172.21.15.151:0/354647294 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb32007aaa0 0x7fb32007cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:52.612 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.611+0000 7fb343400640 1 --2- 172.21.15.151:0/354647294 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb32007aaa0 0x7fb32007cf60 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fb334001720 tx=0x7fb334009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:52.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.614+0000 7fb33b400640 1 -- 172.21.15.151:0/354647294 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb34414ec20 con 0x7fb34415f5a0 2024-09-16T15:54:52.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.864+0000 7fb34a600640 1 -- 172.21.15.151:0/354647294 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fb3440acf80 con 0x7fb34415f5a0 2024-09-16T15:54:52.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.864+0000 7fb33b400640 1 -- 172.21.15.151:0/354647294 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) ==== 80+0+5304 (secure 0 0 0) 0x7fb33c0847b0 con 0x7fb34415f5a0 2024-09-16T15:54:52.865 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:54:52.865 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-16T15:54:52.865 INFO:teuthology.orchestra.run.smithi151.stdout: "auth_dump": [ 2024-09-16T15:54:52.866 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:54:52.866 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "mds.a", 2024-09-16T15:54:52.866 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDsUOhmJVK6AhAANfmTPS6kKn41LSD/MuyW9g==", 2024-09-16T15:54:52.866 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:54:52.866 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:54:52.866 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow *", 2024-09-16T15:54:52.866 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow *", 2024-09-16T15:54:52.866 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:54:52.866 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:54:52.866 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:54:52.866 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:54:52.866 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "mds.b", 2024-09-16T15:54:52.866 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDsUOhmX1R3FhAAUQgUsE/6wUyokXcZuBuIWg==", 2024-09-16T15:54:52.866 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:54:52.867 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:54:52.867 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow *", 2024-09-16T15:54:52.867 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow *", 2024-09-16T15:54:52.867 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:54:52.867 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:54:52.867 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:54:52.867 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:54:52.867 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "mds.c", 2024-09-16T15:54:52.867 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDsUOhmYGCKDBAAY2EFNmiFNf1x2so9wxp4YQ==", 2024-09-16T15:54:52.867 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:54:52.867 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:54:52.867 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow *", 2024-09-16T15:54:52.867 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow *", 2024-09-16T15:54:52.867 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:54:52.868 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:54:52.868 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:54:52.868 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:54:52.868 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "mds.d", 2024-09-16T15:54:52.868 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDsUOhmfCk4IBAAx12Gw2QudQ2nrPb8VlE0SA==", 2024-09-16T15:54:52.868 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:54:52.868 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:54:52.868 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow *", 2024-09-16T15:54:52.868 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow *", 2024-09-16T15:54:52.868 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:54:52.868 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:54:52.868 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:54:52.868 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:54:52.868 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.0", 2024-09-16T15:54:52.868 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDvUOhmKji5NxAAn/E3YQR7jeMnBTLBXc1G6A==", 2024-09-16T15:54:52.869 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:54:52.869 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:54:52.869 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:54:52.869 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:54:52.869 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:54:52.869 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:54:52.869 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:54:52.869 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.1", 2024-09-16T15:54:52.869 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDzUOhmmBrNBhAA0Uwh4Fam2bmKKvz5jAQBWw==", 2024-09-16T15:54:52.869 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:54:52.869 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:54:52.869 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:54:52.869 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:54:52.869 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:54:52.869 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:54:52.870 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:54:52.870 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.2", 2024-09-16T15:54:52.870 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQD2UOhmOV6GERAAjD+PQLTZrCFR1669UEEIAQ==", 2024-09-16T15:54:52.870 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:54:52.870 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:54:52.870 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:54:52.870 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:54:52.870 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:54:52.870 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:54:52.870 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:54:52.870 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.3", 2024-09-16T15:54:52.870 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQD5UOhmCRrdGRAAQC0bqWX6N1jniVGEQ2aAvg==", 2024-09-16T15:54:52.870 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:54:52.870 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:54:52.870 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:54:52.871 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:54:52.871 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:54:52.871 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:54:52.871 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:54:52.871 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.4", 2024-09-16T15:54:52.871 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQD/UOhmXI/nGRAA7t6XveKvrWgqy6rHuMQSZQ==", 2024-09-16T15:54:52.871 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:54:52.871 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:54:52.871 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:54:52.871 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:54:52.871 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:54:52.871 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:54:52.871 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:54:52.871 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.5", 2024-09-16T15:54:52.872 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQACUehmKM//IxAAsoFpiiTXNs//v3YXeUf/Iw==", 2024-09-16T15:54:52.872 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:54:52.872 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:54:52.872 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:54:52.872 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:54:52.872 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:54:52.872 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:54:52.872 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:54:52.872 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.6", 2024-09-16T15:54:52.872 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQAFUehmlo/FLRAAp5mvUSWCaBbm85QyTZY0eA==", 2024-09-16T15:54:52.872 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:54:52.872 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:54:52.872 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:54:52.872 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:54:52.872 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:54:52.873 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:54:52.873 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:54:52.873 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "osd.7", 2024-09-16T15:54:52.873 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQAIUehmR32JNxAA27antOIKot1nScQbLZDf5g==", 2024-09-16T15:54:52.873 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:54:52.873 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow profile osd", 2024-09-16T15:54:52.873 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile osd", 2024-09-16T15:54:52.873 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:54:52.873 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:54:52.873 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:54:52.873 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:54:52.873 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "client.0", 2024-09-16T15:54:52.873 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDsUOhmf5PBKBAAKE4Y4zbA4h07IBNjPkyptw==", 2024-09-16T15:54:52.873 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:54:52.873 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:54:52.873 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow r", 2024-09-16T15:54:52.874 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-16T15:54:52.874 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:54:52.874 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:54:52.874 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:54:52.874 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "client.1", 2024-09-16T15:54:52.874 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDsUOhm15emNRAAwhK7QmA2c9IYGn04ERvY0g==", 2024-09-16T15:54:52.874 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:54:52.874 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:54:52.874 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow r", 2024-09-16T15:54:52.874 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow rw", 2024-09-16T15:54:52.874 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow rwx" 2024-09-16T15:54:52.874 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:54:52.874 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:54:52.874 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:54:52.874 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "client.2", 2024-09-16T15:54:52.875 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDsUOhm6zUyLxAAkzeiIQTuNZkMM+Oit9rC5A==", 2024-09-16T15:54:52.875 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:54:52.875 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:54:52.875 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow r", 2024-09-16T15:54:52.875 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow rw", 2024-09-16T15:54:52.875 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow rwx" 2024-09-16T15:54:52.875 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:54:52.875 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:54:52.875 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:54:52.875 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "client.3", 2024-09-16T15:54:52.875 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDtUOhmD82EABAAeDf82/hPxpPfAUk8oj5h/Q==", 2024-09-16T15:54:52.875 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:54:52.876 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow", 2024-09-16T15:54:52.876 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow r", 2024-09-16T15:54:52.876 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow rw", 2024-09-16T15:54:52.876 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow rwx" 2024-09-16T15:54:52.876 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:54:52.876 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:54:52.876 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:54:52.876 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "client.admin", 2024-09-16T15:54:52.876 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDrUOhmQFmVDRAA8NKysbAe475xvg4CPAcFyw==", 2024-09-16T15:54:52.876 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:54:52.876 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow *", 2024-09-16T15:54:52.876 INFO:teuthology.orchestra.run.smithi151.stdout: "mgr": "allow *", 2024-09-16T15:54:52.876 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow *", 2024-09-16T15:54:52.876 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:54:52.876 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:54:52.877 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:54:52.877 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:54:52.877 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "mgr.x", 2024-09-16T15:54:52.877 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDrUOhmloW3NxAA8UoLUfrw3wf1nzKbmtK/jw==", 2024-09-16T15:54:52.877 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:54:52.877 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow *", 2024-09-16T15:54:52.877 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile mgr", 2024-09-16T15:54:52.877 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:54:52.877 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:54:52.877 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:54:52.877 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:54:52.877 INFO:teuthology.orchestra.run.smithi151.stdout: "entity": "mgr.y", 2024-09-16T15:54:52.877 INFO:teuthology.orchestra.run.smithi151.stdout: "key": "AQDrUOhm1uZOLxAATgjA79vYf0psqQVo03sdDA==", 2024-09-16T15:54:52.877 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": { 2024-09-16T15:54:52.877 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": "allow *", 2024-09-16T15:54:52.877 INFO:teuthology.orchestra.run.smithi151.stdout: "mon": "allow profile mgr", 2024-09-16T15:54:52.878 INFO:teuthology.orchestra.run.smithi151.stdout: "osd": "allow *" 2024-09-16T15:54:52.878 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:54:52.878 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:54:52.878 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-09-16T15:54:52.878 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-16T15:54:52.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.866+0000 7fb34a600640 1 -- 172.21.15.151:0/354647294 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb32007aaa0 msgr2=0x7fb32007cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:52.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.866+0000 7fb34a600640 1 --2- 172.21.15.151:0/354647294 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb32007aaa0 0x7fb32007cf60 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fb334001720 tx=0x7fb334009040 comp rx=0 tx=0).stop 2024-09-16T15:54:52.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.866+0000 7fb34a600640 1 -- 172.21.15.151:0/354647294 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb34415f5a0 msgr2=0x7fb3441f7e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:52.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.866+0000 7fb34a600640 1 --2- 172.21.15.151:0/354647294 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb34415f5a0 0x7fb3441f7e70 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7fb33c006010 tx=0x7fb33c003f10 comp rx=0 tx=0).stop 2024-09-16T15:54:52.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.866+0000 7fb34a600640 1 -- 172.21.15.151:0/354647294 shutdown_connections 2024-09-16T15:54:52.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.866+0000 7fb34a600640 1 --2- 172.21.15.151:0/354647294 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb32007aaa0 0x7fb32007cf60 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:52.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.866+0000 7fb34a600640 1 --2- 172.21.15.151:0/354647294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3440ccf00 0x7fb34409b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:52.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.866+0000 7fb34a600640 1 --2- 172.21.15.151:0/354647294 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb34415f5a0 0x7fb3441f7e70 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:52.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.867+0000 7fb34a600640 1 --2- 172.21.15.151:0/354647294 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb344073fb0 0x7fb34414e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:52.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.867+0000 7fb34a600640 1 -- 172.21.15.151:0/354647294 >> 172.21.15.151:0/354647294 conn(0x7fb3440b1950 msgr2=0x7fb3440b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:52.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.867+0000 7fb34a600640 1 -- 172.21.15.151:0/354647294 shutdown_connections 2024-09-16T15:54:52.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:52.867+0000 7fb34a600640 1 -- 172.21.15.151:0/354647294 wait complete. 2024-09-16T15:54:52.883 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-16T15:54:53.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.040+0000 7f1438000640 1 -- 172.21.15.151:0/2872657938 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f143015f5a0 msgr2=0x7f14300b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:53.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.040+0000 7f1438000640 1 --2- 172.21.15.151:0/2872657938 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f143015f5a0 0x7f14300b2440 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f1430076720 tx=0x7f142402a4d0 comp rx=0 tx=0).stop 2024-09-16T15:54:53.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.040+0000 7f1438000640 1 -- 172.21.15.151:0/2872657938 shutdown_connections 2024-09-16T15:54:53.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.040+0000 7f1438000640 1 --2- 172.21.15.151:0/2872657938 >> v2:172.21.15.167:3300/0 conn(0x7f14300ad270 0x7f1430073fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:53.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.040+0000 7f1438000640 1 --2- 172.21.15.151:0/2872657938 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f14300ccf00 0x7f14300acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:53.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.040+0000 7f1438000640 1 --2- 172.21.15.151:0/2872657938 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f143015f5a0 0x7f14300b2440 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:53.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.040+0000 7f1438000640 1 -- 172.21.15.151:0/2872657938 >> 172.21.15.151:0/2872657938 conn(0x7f14300b1950 msgr2=0x7f14300b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:53.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.040+0000 7f1438000640 1 -- 172.21.15.151:0/2872657938 shutdown_connections 2024-09-16T15:54:53.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.040+0000 7f1438000640 1 -- 172.21.15.151:0/2872657938 wait complete. 2024-09-16T15:54:53.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.042+0000 7f1438000640 1 Processor -- start 2024-09-16T15:54:53.043 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.042+0000 7f1438000640 1 -- start start 2024-09-16T15:54:53.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.043+0000 7f1438000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14300ad270 0x7f14301445b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:53.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.043+0000 7f1438000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14300ccf00 0x7f1430152a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:53.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.043+0000 7f1438000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f143015f5a0 0x7f1430152fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:53.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.043+0000 7f1438000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f143006f160 con 0x7f14300ad270 2024-09-16T15:54:53.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.043+0000 7f1438000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1430074780 con 0x7f14300ccf00 2024-09-16T15:54:53.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.043+0000 7f1438000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f143006f2e0 con 0x7f143015f5a0 2024-09-16T15:54:53.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.043+0000 7f1435600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14300ad270 0x7f14301445b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:53.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.043+0000 7f1435600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14300ad270 0x7f14301445b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52438/0 (socket says 172.21.15.151:52438) 2024-09-16T15:54:53.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.043+0000 7f1435600640 1 -- 172.21.15.151:0/2620220550 learned_addr learned my addr 172.21.15.151:0/2620220550 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:54:53.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.043+0000 7f1434c00640 1 --2- 172.21.15.151:0/2620220550 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14300ccf00 0x7f1430152a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:53.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.043+0000 7f1436000640 1 --2- 172.21.15.151:0/2620220550 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f143015f5a0 0x7f1430152fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:53.046 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.044+0000 7f1435600640 1 -- 172.21.15.151:0/2620220550 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f143015f5a0 msgr2=0x7f1430152fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:53.047 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.044+0000 7f1435600640 1 --2- 172.21.15.151:0/2620220550 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f143015f5a0 0x7f1430152fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:53.047 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.044+0000 7f1435600640 1 -- 172.21.15.151:0/2620220550 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14300ccf00 msgr2=0x7f1430152a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:53.047 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.044+0000 7f1435600640 1 --2- 172.21.15.151:0/2620220550 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14300ccf00 0x7f1430152a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:53.047 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.044+0000 7f1435600640 1 -- 172.21.15.151:0/2620220550 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1430151790 con 0x7f14300ad270 2024-09-16T15:54:53.047 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.044+0000 7f1434c00640 1 --2- 172.21.15.151:0/2620220550 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14300ccf00 0x7f1430152a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:54:53.047 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.044+0000 7f1435600640 1 --2- 172.21.15.151:0/2620220550 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14300ad270 0x7f14301445b0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f1424000c00 tx=0x7f14240038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:53.047 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.044+0000 7f142d600640 1 -- 172.21.15.151:0/2620220550 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f142403e030 con 0x7f14300ad270 2024-09-16T15:54:53.047 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.044+0000 7f142d600640 1 -- 172.21.15.151:0/2620220550 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1424003ca0 con 0x7f14300ad270 2024-09-16T15:54:53.047 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.044+0000 7f142d600640 1 -- 172.21.15.151:0/2620220550 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1424035040 con 0x7f14300ad270 2024-09-16T15:54:53.047 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.044+0000 7f1438000640 1 -- 172.21.15.151:0/2620220550 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1430151a70 con 0x7f14300ad270 2024-09-16T15:54:53.047 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.044+0000 7f1438000640 1 -- 172.21.15.151:0/2620220550 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1430152030 con 0x7f14300ad270 2024-09-16T15:54:53.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.045+0000 7f1438000640 1 -- 172.21.15.151:0/2620220550 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1430152270 con 0x7f14300ad270 2024-09-16T15:54:53.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.047+0000 7f142d600640 1 -- 172.21.15.151:0/2620220550 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 50) ==== 104398+0+0 (secure 0 0 0) 0x7f1424041030 con 0x7f14300ad270 2024-09-16T15:54:53.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.047+0000 7f142d600640 1 --2- 172.21.15.151:0/2620220550 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f141007aaa0 0x7f141007cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:53.048 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.048+0000 7f142d600640 1 -- 172.21.15.151:0/2620220550 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 4997+0+0 (secure 0 0 0) 0x7f14240ba000 con 0x7f14300ad270 2024-09-16T15:54:53.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.048+0000 7f1434c00640 1 --2- 172.21.15.151:0/2620220550 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f141007aaa0 0x7f141007cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:53.050 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.049+0000 7f1434c00640 1 --2- 172.21.15.151:0/2620220550 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f141007aaa0 0x7f141007cf60 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f143014fee0 tx=0x7f141c005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:53.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.051+0000 7f142d600640 1 -- 172.21.15.151:0/2620220550 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1430152270 con 0x7f14300ad270 2024-09-16T15:54:53.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.236+0000 7f1438000640 1 -- 172.21.15.151:0/2620220550 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f143009bc30 con 0x7f14300ad270 2024-09-16T15:54:53.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.237+0000 7f142d600640 1 -- 172.21.15.151:0/2620220550 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v141) ==== 84+0+65 (secure 0 0 0) 0x7f14240840d0 con 0x7f14300ad270 2024-09-16T15:54:53.238 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:54:53.238 INFO:teuthology.orchestra.run.smithi151.stdout:[ 2024-09-16T15:54:53.238 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:54:53.238 INFO:teuthology.orchestra.run.smithi151.stdout: "poolnum": 1, 2024-09-16T15:54:53.238 INFO:teuthology.orchestra.run.smithi151.stdout: "poolname": ".mgr" 2024-09-16T15:54:53.238 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:54:53.239 INFO:teuthology.orchestra.run.smithi151.stdout:] 2024-09-16T15:54:53.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.239+0000 7f1438000640 1 -- 172.21.15.151:0/2620220550 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f141007aaa0 msgr2=0x7f141007cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:53.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.239+0000 7f1438000640 1 --2- 172.21.15.151:0/2620220550 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f141007aaa0 0x7f141007cf60 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f143014fee0 tx=0x7f141c005f90 comp rx=0 tx=0).stop 2024-09-16T15:54:53.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.239+0000 7f1438000640 1 -- 172.21.15.151:0/2620220550 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14300ad270 msgr2=0x7f14301445b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:53.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.239+0000 7f1438000640 1 --2- 172.21.15.151:0/2620220550 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14300ad270 0x7f14301445b0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f1424000c00 tx=0x7f14240038a0 comp rx=0 tx=0).stop 2024-09-16T15:54:53.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.240+0000 7f1438000640 1 -- 172.21.15.151:0/2620220550 shutdown_connections 2024-09-16T15:54:53.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.240+0000 7f1438000640 1 --2- 172.21.15.151:0/2620220550 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f141007aaa0 0x7f141007cf60 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:53.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.240+0000 7f1438000640 1 --2- 172.21.15.151:0/2620220550 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f14300ccf00 0x7f1430152a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:53.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.240+0000 7f1438000640 1 --2- 172.21.15.151:0/2620220550 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f143015f5a0 0x7f1430152fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:53.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.240+0000 7f1438000640 1 --2- 172.21.15.151:0/2620220550 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f14300ad270 0x7f14301445b0 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:53.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.240+0000 7f1438000640 1 -- 172.21.15.151:0/2620220550 >> 172.21.15.151:0/2620220550 conn(0x7f14300b1950 msgr2=0x7f143009c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:53.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.240+0000 7f1438000640 1 -- 172.21.15.151:0/2620220550 shutdown_connections 2024-09-16T15:54:53.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.240+0000 7f1438000640 1 -- 172.21.15.151:0/2620220550 wait complete. 2024-09-16T15:54:53.257 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-16T15:54:53.258 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-16T15:54:53.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.421+0000 7fd83ae00640 1 -- 172.21.15.151:0/2039169121 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd834159aa0 msgr2=0x7fd8340741a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:53.422 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.421+0000 7fd83ae00640 1 --2- 172.21.15.151:0/2039169121 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd834159aa0 0x7fd8340741a0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fd834075110 tx=0x7fd82402aa30 comp rx=0 tx=0).stop 2024-09-16T15:54:53.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.421+0000 7fd83ae00640 1 -- 172.21.15.151:0/2039169121 shutdown_connections 2024-09-16T15:54:53.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.421+0000 7fd83ae00640 1 --2- 172.21.15.151:0/2039169121 >> v2:172.21.15.167:3300/0 conn(0x7fd83406f220 0x7fd83406f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:53.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.421+0000 7fd83ae00640 1 --2- 172.21.15.151:0/2039169121 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd834159aa0 0x7fd8340741a0 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:53.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.421+0000 7fd83ae00640 1 -- 172.21.15.151:0/2039169121 >> 172.21.15.151:0/2039169121 conn(0x7fd8340b18e0 msgr2=0x7fd8340b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:53.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.421+0000 7fd83ae00640 1 -- 172.21.15.151:0/2039169121 shutdown_connections 2024-09-16T15:54:53.423 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.421+0000 7fd83ae00640 1 -- 172.21.15.151:0/2039169121 wait complete. 2024-09-16T15:54:53.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.423+0000 7fd83ae00640 1 Processor -- start 2024-09-16T15:54:53.424 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.423+0000 7fd83ae00640 1 -- start start 2024-09-16T15:54:53.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.424+0000 7fd83ae00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd83406f220 0x7fd834151080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:53.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.424+0000 7fd83ae00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd834159aa0 0x7fd8340cb390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:53.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.424+0000 7fd83ae00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd83415cff0 0x7fd8340ce4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:53.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.424+0000 7fd83ae00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd8340cc520 con 0x7fd83406f220 2024-09-16T15:54:53.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.424+0000 7fd83ae00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd8340cc3a0 con 0x7fd83415cff0 2024-09-16T15:54:53.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.424+0000 7fd83ae00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fd8340cc6a0 con 0x7fd834159aa0 2024-09-16T15:54:53.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.424+0000 7fd833e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd83406f220 0x7fd834151080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:53.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.424+0000 7fd833e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd83406f220 0x7fd834151080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52454/0 (socket says 172.21.15.151:52454) 2024-09-16T15:54:53.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.424+0000 7fd833e00640 1 -- 172.21.15.151:0/2372740055 learned_addr learned my addr 172.21.15.151:0/2372740055 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:54:53.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.424+0000 7fd833400640 1 --2- 172.21.15.151:0/2372740055 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd834159aa0 0x7fd8340cb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:53.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.424+0000 7fd838e00640 1 --2- 172.21.15.151:0/2372740055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd83415cff0 0x7fd8340ce4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:53.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.425+0000 7fd833e00640 1 -- 172.21.15.151:0/2372740055 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd834159aa0 msgr2=0x7fd8340cb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:53.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.425+0000 7fd833e00640 1 --2- 172.21.15.151:0/2372740055 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd834159aa0 0x7fd8340cb390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:53.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.425+0000 7fd833e00640 1 -- 172.21.15.151:0/2372740055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd83415cff0 msgr2=0x7fd8340ce4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:53.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.425+0000 7fd833e00640 1 --2- 172.21.15.151:0/2372740055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd83415cff0 0x7fd8340ce4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:53.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.425+0000 7fd833e00640 1 -- 172.21.15.151:0/2372740055 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8341515c0 con 0x7fd83406f220 2024-09-16T15:54:53.428 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.425+0000 7fd833e00640 1 --2- 172.21.15.151:0/2372740055 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd83406f220 0x7fd834151080 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fd824009bf0 tx=0x7fd824003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:53.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.426+0000 7fd82be00640 1 -- 172.21.15.151:0/2372740055 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd824047070 con 0x7fd83406f220 2024-09-16T15:54:53.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.426+0000 7fd82be00640 1 -- 172.21.15.151:0/2372740055 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd824004250 con 0x7fd83406f220 2024-09-16T15:54:53.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.426+0000 7fd83ae00640 1 -- 172.21.15.151:0/2372740055 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8340ca2f0 con 0x7fd83406f220 2024-09-16T15:54:53.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.426+0000 7fd82be00640 1 -- 172.21.15.151:0/2372740055 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd824033050 con 0x7fd83406f220 2024-09-16T15:54:53.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.426+0000 7fd83ae00640 1 -- 172.21.15.151:0/2372740055 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8340ca8b0 con 0x7fd83406f220 2024-09-16T15:54:53.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.427+0000 7fd83ae00640 1 -- 172.21.15.151:0/2372740055 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8340ccba0 con 0x7fd83406f220 2024-09-16T15:54:53.430 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.429+0000 7fd82be00640 1 -- 172.21.15.151:0/2372740055 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 50) ==== 104398+0+0 (secure 0 0 0) 0x7fd82404a070 con 0x7fd83406f220 2024-09-16T15:54:53.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.430+0000 7fd82be00640 1 --2- 172.21.15.151:0/2372740055 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd81407aaa0 0x7fd81407cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:53.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.430+0000 7fd82be00640 1 -- 172.21.15.151:0/2372740055 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 4997+0+0 (secure 0 0 0) 0x7fd8240bac60 con 0x7fd83406f220 2024-09-16T15:54:53.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.430+0000 7fd833400640 1 --2- 172.21.15.151:0/2372740055 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd81407aaa0 0x7fd81407cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:53.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.431+0000 7fd833400640 1 --2- 172.21.15.151:0/2372740055 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd81407aaa0 0x7fd81407cf60 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fd8340cd8b0 tx=0x7fd820009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:53.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.435+0000 7fd82be00640 1 -- 172.21.15.151:0/2372740055 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd8240846e0 con 0x7fd83406f220 2024-09-16T15:54:53.619 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:53.618+0000 7fd83ae00640 1 -- 172.21.15.151:0/2372740055 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fd8340b2790 con 0x7fd83406f220 2024-09-16T15:54:54.433 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:54.432+0000 7fd82be00640 1 -- 172.21.15.151:0/2372740055 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v142) ==== 138+0+0 (secure 0 0 0) 0x7fd824089110 con 0x7fd83406f220 2024-09-16T15:54:54.434 INFO:teuthology.orchestra.run.smithi151.stderr:pool 'cephfs_metadata' created 2024-09-16T15:54:54.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:54.434+0000 7fd83ae00640 1 -- 172.21.15.151:0/2372740055 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd81407aaa0 msgr2=0x7fd81407cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:54.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:54.435+0000 7fd83ae00640 1 --2- 172.21.15.151:0/2372740055 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd81407aaa0 0x7fd81407cf60 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fd8340cd8b0 tx=0x7fd820009040 comp rx=0 tx=0).stop 2024-09-16T15:54:54.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:54.435+0000 7fd83ae00640 1 -- 172.21.15.151:0/2372740055 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd83406f220 msgr2=0x7fd834151080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:54.437 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:54.435+0000 7fd83ae00640 1 --2- 172.21.15.151:0/2372740055 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd83406f220 0x7fd834151080 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fd824009bf0 tx=0x7fd824003f10 comp rx=0 tx=0).stop 2024-09-16T15:54:54.437 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:54.435+0000 7fd83ae00640 1 -- 172.21.15.151:0/2372740055 shutdown_connections 2024-09-16T15:54:54.437 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:54.435+0000 7fd83ae00640 1 --2- 172.21.15.151:0/2372740055 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd81407aaa0 0x7fd81407cf60 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:54.437 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:54.435+0000 7fd83ae00640 1 --2- 172.21.15.151:0/2372740055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd83415cff0 0x7fd8340ce4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:54.437 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:54.435+0000 7fd83ae00640 1 --2- 172.21.15.151:0/2372740055 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd834159aa0 0x7fd8340cb390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:54.437 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:54.435+0000 7fd83ae00640 1 --2- 172.21.15.151:0/2372740055 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd83406f220 0x7fd834151080 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:54.437 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:54.435+0000 7fd83ae00640 1 -- 172.21.15.151:0/2372740055 >> 172.21.15.151:0/2372740055 conn(0x7fd8340b18e0 msgr2=0x7fd8340aeca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:54.437 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:54.435+0000 7fd83ae00640 1 -- 172.21.15.151:0/2372740055 shutdown_connections 2024-09-16T15:54:54.437 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:54.435+0000 7fd83ae00640 1 -- 172.21.15.151:0/2372740055 wait complete. 2024-09-16T15:54:54.458 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-16T15:54:54.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:54.626+0000 7fc74c000640 1 -- 172.21.15.151:0/2175702024 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7440cccd0 msgr2=0x7fc744073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:54.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:54.626+0000 7fc74c000640 1 --2- 172.21.15.151:0/2175702024 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7440cccd0 0x7fc744073440 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fc730009830 tx=0x7fc73002a480 comp rx=0 tx=0).stop 2024-09-16T15:54:54.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:54.626+0000 7fc74c000640 1 -- 172.21.15.151:0/2175702024 shutdown_connections 2024-09-16T15:54:54.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:54.626+0000 7fc74c000640 1 --2- 172.21.15.151:0/2175702024 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7440cccd0 0x7fc744073440 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:54.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:54.626+0000 7fc74c000640 1 -- 172.21.15.151:0/2175702024 >> 172.21.15.151:0/2175702024 conn(0x7fc7440b1840 msgr2=0x7fc7440b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:54.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:54.626+0000 7fc74c000640 1 -- 172.21.15.151:0/2175702024 shutdown_connections 2024-09-16T15:54:54.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:54.626+0000 7fc74c000640 1 -- 172.21.15.151:0/2175702024 wait complete. 2024-09-16T15:54:54.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:54.628+0000 7fc74c000640 1 Processor -- start 2024-09-16T15:54:54.629 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:54.628+0000 7fc74c000640 1 -- start start 2024-09-16T15:54:54.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:54.629+0000 7fc74c000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc74406efa0 0x7fc74415ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:54.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:54.629+0000 7fc74c000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7440cccd0 0x7fc7441f6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:54.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:54.629+0000 7fc74c000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc744145750 0x7fc7440c9de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:54.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:54.629+0000 7fc74c000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc7440ce6a0 con 0x7fc7440cccd0 2024-09-16T15:54:54.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:54.629+0000 7fc74c000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc7440ce520 con 0x7fc74406efa0 2024-09-16T15:54:54.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:54.629+0000 7fc74c000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fc7440ce820 con 0x7fc744145750 2024-09-16T15:54:54.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:54.629+0000 7fc748c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7440cccd0 0x7fc7441f6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:54.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:54.629+0000 7fc748c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7440cccd0 0x7fc7441f6ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52478/0 (socket says 172.21.15.151:52478) 2024-09-16T15:54:54.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:54.629+0000 7fc748c00640 1 -- 172.21.15.151:0/716605150 learned_addr learned my addr 172.21.15.151:0/716605150 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:54:54.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:54.629+0000 7fc749600640 1 --2- 172.21.15.151:0/716605150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc74406efa0 0x7fc74415ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:54.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:54.629+0000 7fc74a000640 1 --2- 172.21.15.151:0/716605150 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc744145750 0x7fc7440c9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:54.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:54.629+0000 7fc748c00640 1 -- 172.21.15.151:0/716605150 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc744145750 msgr2=0x7fc7440c9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:54.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:54.629+0000 7fc748c00640 1 --2- 172.21.15.151:0/716605150 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc744145750 0x7fc7440c9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:54.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:54.629+0000 7fc748c00640 1 -- 172.21.15.151:0/716605150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc74406efa0 msgr2=0x7fc74415ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:54.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:54.629+0000 7fc748c00640 1 --2- 172.21.15.151:0/716605150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc74406efa0 0x7fc74415ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:54.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:54.629+0000 7fc748c00640 1 -- 172.21.15.151:0/716605150 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7441f6ff0 con 0x7fc7440cccd0 2024-09-16T15:54:54.634 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:54.630+0000 7fc748c00640 1 --2- 172.21.15.151:0/716605150 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7440cccd0 0x7fc7441f6ab0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fc730000c00 tx=0x7fc730032d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:54.634 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:54.630+0000 7fc741600640 1 -- 172.21.15.151:0/716605150 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc730040070 con 0x7fc7440cccd0 2024-09-16T15:54:54.634 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:54.630+0000 7fc741600640 1 -- 172.21.15.151:0/716605150 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc730003af0 con 0x7fc7440cccd0 2024-09-16T15:54:54.634 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:54.630+0000 7fc74c000640 1 -- 172.21.15.151:0/716605150 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7441379a0 con 0x7fc7440cccd0 2024-09-16T15:54:54.634 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:54.630+0000 7fc741600640 1 -- 172.21.15.151:0/716605150 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc730045460 con 0x7fc7440cccd0 2024-09-16T15:54:54.634 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:54.630+0000 7fc74c000640 1 -- 172.21.15.151:0/716605150 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc744137f60 con 0x7fc7440cccd0 2024-09-16T15:54:54.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:54.631+0000 7fc74c000640 1 -- 172.21.15.151:0/716605150 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7440cecd0 con 0x7fc7440cccd0 2024-09-16T15:54:54.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:54.633+0000 7fc741600640 1 -- 172.21.15.151:0/716605150 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 50) ==== 104398+0+0 (secure 0 0 0) 0x7fc730039040 con 0x7fc7440cccd0 2024-09-16T15:54:54.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:54.634+0000 7fc741600640 1 --2- 172.21.15.151:0/716605150 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc72407aaa0 0x7fc72407cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:54.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:54.634+0000 7fc741600640 1 -- 172.21.15.151:0/716605150 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5367+0+0 (secure 0 0 0) 0x7fc7300bb3c0 con 0x7fc7440cccd0 2024-09-16T15:54:54.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:54.634+0000 7fc749600640 1 --2- 172.21.15.151:0/716605150 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc72407aaa0 0x7fc72407cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:54.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:54.635+0000 7fc749600640 1 --2- 172.21.15.151:0/716605150 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc72407aaa0 0x7fc72407cf60 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fc7440aaad0 tx=0x7fc73800d040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:54.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:54.637+0000 7fc741600640 1 -- 172.21.15.151:0/716605150 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc7441f6ff0 con 0x7fc7440cccd0 2024-09-16T15:54:54.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:54.822+0000 7fc74c000640 1 -- 172.21.15.151:0/716605150 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fc7440736f0 con 0x7fc7440cccd0 2024-09-16T15:54:55.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:55.435+0000 7fc741600640 1 -- 172.21.15.151:0/716605150 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v143) ==== 170+0+0 (secure 0 0 0) 0x7fc730084ec0 con 0x7fc7440cccd0 2024-09-16T15:54:55.436 INFO:teuthology.orchestra.run.smithi151.stderr:pool 'cephfs_data' created 2024-09-16T15:54:55.439 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:55.438+0000 7fc74c000640 1 -- 172.21.15.151:0/716605150 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc72407aaa0 msgr2=0x7fc72407cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:55.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:55.438+0000 7fc74c000640 1 --2- 172.21.15.151:0/716605150 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc72407aaa0 0x7fc72407cf60 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fc7440aaad0 tx=0x7fc73800d040 comp rx=0 tx=0).stop 2024-09-16T15:54:55.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:55.438+0000 7fc74c000640 1 -- 172.21.15.151:0/716605150 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7440cccd0 msgr2=0x7fc7441f6ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:55.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:55.438+0000 7fc74c000640 1 --2- 172.21.15.151:0/716605150 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7440cccd0 0x7fc7441f6ab0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fc730000c00 tx=0x7fc730032d50 comp rx=0 tx=0).stop 2024-09-16T15:54:55.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:55.438+0000 7fc74c000640 1 -- 172.21.15.151:0/716605150 shutdown_connections 2024-09-16T15:54:55.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:55.438+0000 7fc74c000640 1 --2- 172.21.15.151:0/716605150 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc72407aaa0 0x7fc72407cf60 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:55.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:55.438+0000 7fc74c000640 1 --2- 172.21.15.151:0/716605150 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc744145750 0x7fc7440c9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:55.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:55.438+0000 7fc74c000640 1 --2- 172.21.15.151:0/716605150 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7440cccd0 0x7fc7441f6ab0 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:55.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:55.438+0000 7fc74c000640 1 --2- 172.21.15.151:0/716605150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc74406efa0 0x7fc74415ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:55.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:55.438+0000 7fc74c000640 1 -- 172.21.15.151:0/716605150 >> 172.21.15.151:0/716605150 conn(0x7fc7440b1840 msgr2=0x7fc7440cbd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:55.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:55.438+0000 7fc74c000640 1 -- 172.21.15.151:0/716605150 shutdown_connections 2024-09-16T15:54:55.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:55.439+0000 7fc74c000640 1 -- 172.21.15.151:0/716605150 wait complete. 2024-09-16T15:54:55.468 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-16T15:54:55.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:55.629+0000 7f2d42000640 1 -- 172.21.15.151:0/874533156 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d20003a50 msgr2=0x7f2d20003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:55.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:55.629+0000 7f2d42000640 1 --2- 172.21.15.151:0/874533156 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d20003a50 0x7f2d20003e70 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c001dc0 tx=0x7f2d20005830 comp rx=0 tx=0).stop 2024-09-16T15:54:55.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:55.629+0000 7f2d42000640 1 -- 172.21.15.151:0/874533156 shutdown_connections 2024-09-16T15:54:55.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:55.630+0000 7f2d42000640 1 --2- 172.21.15.151:0/874533156 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d20003a50 0x7f2d20003e70 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:55.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:55.630+0000 7f2d42000640 1 -- 172.21.15.151:0/874533156 >> 172.21.15.151:0/874533156 conn(0x7f2d3c0cf9f0 msgr2=0x7f2d3c0cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:55.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:55.630+0000 7f2d42000640 1 -- 172.21.15.151:0/874533156 shutdown_connections 2024-09-16T15:54:55.631 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:55.630+0000 7f2d42000640 1 -- 172.21.15.151:0/874533156 wait complete. 2024-09-16T15:54:55.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:55.631+0000 7f2d42000640 1 Processor -- start 2024-09-16T15:54:55.632 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:55.631+0000 7f2d42000640 1 -- start start 2024-09-16T15:54:55.634 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:55.632+0000 7f2d42000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d20003a50 0x7f2d3c0c6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:55.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:55.632+0000 7f2d42000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d3c153d30 0x7f2d3c0a9470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:55.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:55.632+0000 7f2d42000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d3c1f3cd0 0x7f2d3c0ce720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:55.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:55.632+0000 7f2d42000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2d3c074070 con 0x7f2d20003a50 2024-09-16T15:54:55.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:55.632+0000 7f2d42000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2d3c073ef0 con 0x7f2d3c153d30 2024-09-16T15:54:55.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:55.632+0000 7f2d42000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f2d3c0741f0 con 0x7f2d3c1f3cd0 2024-09-16T15:54:55.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:55.632+0000 7f2d40c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d20003a50 0x7f2d3c0c6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:55.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:55.632+0000 7f2d40c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d20003a50 0x7f2d3c0c6500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52498/0 (socket says 172.21.15.151:52498) 2024-09-16T15:54:55.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:55.632+0000 7f2d40c00640 1 -- 172.21.15.151:0/3812941890 learned_addr learned my addr 172.21.15.151:0/3812941890 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:54:55.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:55.633+0000 7f2d41600640 1 --2- 172.21.15.151:0/3812941890 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d3c1f3cd0 0x7f2d3c0ce720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:55.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:55.633+0000 7f2d3be00640 1 --2- 172.21.15.151:0/3812941890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d3c153d30 0x7f2d3c0a9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:55.635 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:55.633+0000 7f2d40c00640 1 -- 172.21.15.151:0/3812941890 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d3c1f3cd0 msgr2=0x7f2d3c0ce720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:55.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:55.633+0000 7f2d40c00640 1 --2- 172.21.15.151:0/3812941890 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d3c1f3cd0 0x7f2d3c0ce720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:55.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:55.633+0000 7f2d40c00640 1 -- 172.21.15.151:0/3812941890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d3c153d30 msgr2=0x7f2d3c0a9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:55.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:55.633+0000 7f2d40c00640 1 --2- 172.21.15.151:0/3812941890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d3c153d30 0x7f2d3c0a9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:55.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:55.633+0000 7f2d40c00640 1 -- 172.21.15.151:0/3812941890 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d3c1f8ae0 con 0x7f2d20003a50 2024-09-16T15:54:55.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:55.633+0000 7f2d3be00640 1 --2- 172.21.15.151:0/3812941890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d3c153d30 0x7f2d3c0a9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:54:55.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:55.633+0000 7f2d40c00640 1 --2- 172.21.15.151:0/3812941890 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d20003a50 0x7f2d3c0c6500 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c001ee0 tx=0x7f2d2c003aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:55.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:55.633+0000 7f2d38c00640 1 -- 172.21.15.151:0/3812941890 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2c006460 con 0x7f2d20003a50 2024-09-16T15:54:55.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:55.633+0000 7f2d38c00640 1 -- 172.21.15.151:0/3812941890 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2d2c0066f0 con 0x7f2d20003a50 2024-09-16T15:54:55.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:55.633+0000 7f2d38c00640 1 -- 172.21.15.151:0/3812941890 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2c0068c0 con 0x7f2d20003a50 2024-09-16T15:54:55.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:55.633+0000 7f2d42000640 1 -- 172.21.15.151:0/3812941890 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d3c1f7f60 con 0x7f2d20003a50 2024-09-16T15:54:55.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:55.634+0000 7f2d42000640 1 -- 172.21.15.151:0/3812941890 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d3c1f8520 con 0x7f2d20003a50 2024-09-16T15:54:55.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:55.635+0000 7f2d42000640 1 -- 172.21.15.151:0/3812941890 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d3c0cb660 con 0x7f2d20003a50 2024-09-16T15:54:55.637 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:55.636+0000 7f2d38c00640 1 -- 172.21.15.151:0/3812941890 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 50) ==== 104398+0+0 (secure 0 0 0) 0x7f2d2c046020 con 0x7f2d20003a50 2024-09-16T15:54:55.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:55.637+0000 7f2d38c00640 1 --2- 172.21.15.151:0/3812941890 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2d1c07aad0 0x7f2d1c07cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:55.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:55.637+0000 7f2d38c00640 1 -- 172.21.15.151:0/3812941890 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5749+0+0 (secure 0 0 0) 0x7f2d2c0ba530 con 0x7f2d20003a50 2024-09-16T15:54:55.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:55.638+0000 7f2d3be00640 1 --2- 172.21.15.151:0/3812941890 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2d1c07aad0 0x7f2d1c07cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:55.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:55.638+0000 7f2d3be00640 1 --2- 172.21.15.151:0/3812941890 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2d1c07aad0 0x7f2d1c07cf90 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c1572d0 tx=0x7f2d28005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:55.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:55.641+0000 7f2d38c00640 1 -- 172.21.15.151:0/3812941890 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2d2c04d070 con 0x7f2d20003a50 2024-09-16T15:54:55.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:55.863+0000 7f2d42000640 1 -- 172.21.15.151:0/3812941890 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f2d3c0b3fe0 con 0x7f2d20003a50 2024-09-16T15:54:56.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:56.461+0000 7f2d38c00640 1 -- 172.21.15.151:0/3812941890 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 20 and data pool 21 v93) ==== 174+0+0 (secure 0 0 0) 0x7f2d2c084680 con 0x7f2d20003a50 2024-09-16T15:54:56.463 INFO:teuthology.orchestra.run.smithi151.stderr:new fs with metadata pool 20 and data pool 21 2024-09-16T15:54:56.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:56.466+0000 7f2d42000640 1 -- 172.21.15.151:0/3812941890 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2d1c07aad0 msgr2=0x7f2d1c07cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:56.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:56.466+0000 7f2d42000640 1 --2- 172.21.15.151:0/3812941890 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2d1c07aad0 0x7f2d1c07cf90 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c1572d0 tx=0x7f2d28005f90 comp rx=0 tx=0).stop 2024-09-16T15:54:56.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:56.466+0000 7f2d42000640 1 -- 172.21.15.151:0/3812941890 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d20003a50 msgr2=0x7f2d3c0c6500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:56.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:56.466+0000 7f2d42000640 1 --2- 172.21.15.151:0/3812941890 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d20003a50 0x7f2d3c0c6500 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c001ee0 tx=0x7f2d2c003aa0 comp rx=0 tx=0).stop 2024-09-16T15:54:56.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:56.469+0000 7f2d42000640 1 -- 172.21.15.151:0/3812941890 shutdown_connections 2024-09-16T15:54:56.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:56.469+0000 7f2d42000640 1 --2- 172.21.15.151:0/3812941890 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2d1c07aad0 0x7f2d1c07cf90 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:56.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:56.469+0000 7f2d42000640 1 --2- 172.21.15.151:0/3812941890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d3c153d30 0x7f2d3c0a9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:56.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:56.469+0000 7f2d42000640 1 --2- 172.21.15.151:0/3812941890 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2d3c1f3cd0 0x7f2d3c0ce720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:56.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:56.469+0000 7f2d42000640 1 --2- 172.21.15.151:0/3812941890 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d20003a50 0x7f2d3c0c6500 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:56.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:56.469+0000 7f2d42000640 1 -- 172.21.15.151:0/3812941890 >> 172.21.15.151:0/3812941890 conn(0x7f2d3c0cf9f0 msgr2=0x7f2d3c0b30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:56.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:56.470+0000 7f2d42000640 1 -- 172.21.15.151:0/3812941890 shutdown_connections 2024-09-16T15:54:56.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:56.472+0000 7f2d42000640 1 -- 172.21.15.151:0/3812941890 wait complete. 2024-09-16T15:54:56.498 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:54:56.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:56.656+0000 7faaea200640 1 -- 172.21.15.151:0/2022513577 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faae40b2440 msgr2=0x7faae406f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:56.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:56.656+0000 7faaea200640 1 --2- 172.21.15.151:0/2022513577 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faae40b2440 0x7faae406f160 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7faadc009830 tx=0x7faadc02a520 comp rx=0 tx=0).stop 2024-09-16T15:54:56.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:56.657+0000 7faaea200640 1 -- 172.21.15.151:0/2022513577 shutdown_connections 2024-09-16T15:54:56.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:56.657+0000 7faaea200640 1 --2- 172.21.15.151:0/2022513577 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faae40b2440 0x7faae406f160 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:56.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:56.657+0000 7faaea200640 1 -- 172.21.15.151:0/2022513577 >> 172.21.15.151:0/2022513577 conn(0x7faae40b1950 msgr2=0x7faae40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:56.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:56.657+0000 7faaea200640 1 -- 172.21.15.151:0/2022513577 shutdown_connections 2024-09-16T15:54:56.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:56.657+0000 7faaea200640 1 -- 172.21.15.151:0/2022513577 wait complete. 2024-09-16T15:54:56.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:56.658+0000 7faaea200640 1 Processor -- start 2024-09-16T15:54:56.659 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:56.659+0000 7faaea200640 1 -- start start 2024-09-16T15:54:56.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:56.659+0000 7faaea200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faae40b2440 0x7faae4154a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:56.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:56.659+0000 7faaea200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faae40ccf00 0x7faae4148680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:56.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:56.659+0000 7faaea200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faae415f5a0 0x7faae415b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:56.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:56.660+0000 7faaea200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7faae4074840 con 0x7faae40b2440 2024-09-16T15:54:56.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:56.660+0000 7faaea200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7faae40746c0 con 0x7faae40ccf00 2024-09-16T15:54:56.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:56.660+0000 7faaea200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7faae406f6a0 con 0x7faae415f5a0 2024-09-16T15:54:56.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:56.660+0000 7faae3400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faae40b2440 0x7faae4154a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:56.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:56.660+0000 7faae3400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faae40b2440 0x7faae4154a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52510/0 (socket says 172.21.15.151:52510) 2024-09-16T15:54:56.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:56.660+0000 7faae3400640 1 -- 172.21.15.151:0/3133784695 learned_addr learned my addr 172.21.15.151:0/3133784695 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:54:56.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:56.660+0000 7faae2a00640 1 --2- 172.21.15.151:0/3133784695 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faae40ccf00 0x7faae4148680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:56.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:56.660+0000 7faae3e00640 1 --2- 172.21.15.151:0/3133784695 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faae415f5a0 0x7faae415b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:56.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:56.660+0000 7faae3400640 1 -- 172.21.15.151:0/3133784695 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faae415f5a0 msgr2=0x7faae415b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:56.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:56.660+0000 7faae3400640 1 --2- 172.21.15.151:0/3133784695 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faae415f5a0 0x7faae415b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:56.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:56.660+0000 7faae3400640 1 -- 172.21.15.151:0/3133784695 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faae40ccf00 msgr2=0x7faae4148680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:56.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:56.660+0000 7faae3400640 1 --2- 172.21.15.151:0/3133784695 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faae40ccf00 0x7faae4148680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:56.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:56.660+0000 7faae3400640 1 -- 172.21.15.151:0/3133784695 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faae4148bc0 con 0x7faae40b2440 2024-09-16T15:54:56.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:56.660+0000 7faae3e00640 1 --2- 172.21.15.151:0/3133784695 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faae415f5a0 0x7faae415b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:54:56.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:56.661+0000 7faae3400640 1 --2- 172.21.15.151:0/3133784695 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faae40b2440 0x7faae4154a70 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7faad400c760 tx=0x7faad400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:56.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:56.661+0000 7faadb400640 1 -- 172.21.15.151:0/3133784695 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faad4003b70 con 0x7faae40b2440 2024-09-16T15:54:56.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:56.661+0000 7faadb400640 1 -- 172.21.15.151:0/3133784695 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faad4003d30 con 0x7faae40b2440 2024-09-16T15:54:56.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:56.661+0000 7faadb400640 1 -- 172.21.15.151:0/3133784695 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faad400aab0 con 0x7faae40b2440 2024-09-16T15:54:56.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:56.661+0000 7faaea200640 1 -- 172.21.15.151:0/3133784695 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faae4156df0 con 0x7faae40b2440 2024-09-16T15:54:56.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:56.661+0000 7faaea200640 1 -- 172.21.15.151:0/3133784695 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faae41573b0 con 0x7faae40b2440 2024-09-16T15:54:56.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:56.662+0000 7faaea200640 1 -- 172.21.15.151:0/3133784695 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faae406f160 con 0x7faae40b2440 2024-09-16T15:54:56.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:56.663+0000 7faadb400640 1 -- 172.21.15.151:0/3133784695 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 50) ==== 104398+0+0 (secure 0 0 0) 0x7faad400ac50 con 0x7faae40b2440 2024-09-16T15:54:56.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:56.664+0000 7faadb400640 1 --2- 172.21.15.151:0/3133784695 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7faac407aaa0 0x7faac407cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:56.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:56.664+0000 7faadb400640 1 -- 172.21.15.151:0/3133784695 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5849+0+0 (secure 0 0 0) 0x7faad4099850 con 0x7faae40b2440 2024-09-16T15:54:56.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:56.665+0000 7faae2a00640 1 --2- 172.21.15.151:0/3133784695 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7faac407aaa0 0x7faac407cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:56.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:56.665+0000 7faae2a00640 1 --2- 172.21.15.151:0/3133784695 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7faac407aaa0 0x7faac407cf60 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7faae4155c30 tx=0x7faad0007e90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:56.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:56.668+0000 7faadb400640 1 -- 172.21.15.151:0/3133784695 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7faae4148bc0 con 0x7faae40b2440 2024-09-16T15:54:56.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:56.853+0000 7faaea200640 1 -- 172.21.15.151:0/3133784695 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faae40b1f80 con 0x7faae40b2440 2024-09-16T15:54:56.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:56.854+0000 7faadb400640 1 -- 172.21.15.151:0/3133784695 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v144) ==== 74+0+17034 (secure 0 0 0) 0x7faad4063920 con 0x7faae40b2440 2024-09-16T15:54:56.856 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:54:56.856 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":144,"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","created":"2024-09-16T15:38:56.348535+0000","modified":"2024-09-16T15:54:55.864289+0000","last_up_change":"2024-09-16T15:39:10.681384+0000","last_in_change":"2024-09-16T15:39:00.544477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:39:11.602252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:54:53.628710+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-16T15:54:54.832847+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5524076-a903-4d0a-af6a-caf474c4765d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6809","nonce":3111342741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6811","nonce":3111342741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6815","nonce":3111342741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6813","nonce":3111342741}]},"public_addr":"172.21.15.151:6809/3111342741","cluster_addr":"172.21.15.151:6811/3111342741","heartbeat_back_addr":"172.21.15.151:6815/3111342741","heartbeat_front_addr":"172.21.15.151:6813/3111342741","state":["exists","up"]},{"osd":1,"uuid":"dd25925f-0ae1-494e-8529-4dc205759db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6801","nonce":3368304087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6803","nonce":3368304087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6807","nonce":3368304087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6805","nonce":3368304087}]},"public_addr":"172.21.15.151:6801/3368304087","cluster_addr":"172.21.15.151:6803/3368304087","heartbeat_back_addr":"172.21.15.151:6807/3368304087","heartbeat_front_addr":"172.21.15.151:6805/3368304087","state":["exists","up"]},{"osd":2,"uuid":"c2f18115-013d-4da6-a0fe-cb978f4494b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6823","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6825","nonce":3079373271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6827","nonce":3079373271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6831","nonce":3079373271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6829","nonce":3079373271}]},"public_addr":"172.21.15.151:6825/3079373271","cluster_addr":"172.21.15.151:6827/3079373271","heartbeat_back_addr":"172.21.15.151:6831/3079373271","heartbeat_front_addr":"172.21.15.151:6829/3079373271","state":["exists","up"]},{"osd":3,"uuid":"36edb718-2fcb-4d88-8ddc-c321eed4279d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6817","nonce":1142509041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6819","nonce":1142509041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6824","nonce":1142509041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6821","nonce":1142509041}]},"public_addr":"172.21.15.151:6817/1142509041","cluster_addr":"172.21.15.151:6819/1142509041","heartbeat_back_addr":"172.21.15.151:6824/1142509041","heartbeat_front_addr":"172.21.15.151:6821/1142509041","state":["exists","up"]},{"osd":4,"uuid":"64d33708-cd0a-4236-abe3-1b646e82ab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6811","nonce":2735495271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6813","nonce":2735495271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6817","nonce":2735495271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6815","nonce":2735495271}]},"public_addr":"172.21.15.167:6811/2735495271","cluster_addr":"172.21.15.167:6813/2735495271","heartbeat_back_addr":"172.21.15.167:6817/2735495271","heartbeat_front_addr":"172.21.15.167:6815/2735495271","state":["exists","up"]},{"osd":5,"uuid":"f14b44a9-5910-458c-b518-09a8d0de94de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6827","nonce":1494916572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6829","nonce":1494916572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6833","nonce":1494916572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6831","nonce":1494916572}]},"public_addr":"172.21.15.167:6827/1494916572","cluster_addr":"172.21.15.167:6829/1494916572","heartbeat_back_addr":"172.21.15.167:6833/1494916572","heartbeat_front_addr":"172.21.15.167:6831/1494916572","state":["exists","up"]},{"osd":6,"uuid":"39335545-2654-4710-84b0-a9aa47cfcbba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6819","nonce":2538273273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6821","nonce":2538273273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6825","nonce":2538273273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6823","nonce":2538273273}]},"public_addr":"172.21.15.167:6819/2538273273","cluster_addr":"172.21.15.167:6821/2538273273","heartbeat_back_addr":"172.21.15.167:6825/2538273273","heartbeat_front_addr":"172.21.15.167:6823/2538273273","state":["exists","up"]},{"osd":7,"uuid":"4f8bfc81-53dd-4339-ac22-1a4007d9f963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6803","nonce":265072227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6805","nonce":265072227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6809","nonce":265072227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6807","nonce":265072227}]},"public_addr":"172.21.15.167:6803/265072227","cluster_addr":"172.21.15.167:6805/265072227","heartbeat_back_addr":"172.21.15.167:6809/265072227","heartbeat_front_addr":"172.21.15.167:6807/265072227","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.150359+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.084358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.722368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.332809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.320709+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.395153+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.935250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.801443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:54:56.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:56.857+0000 7faaea200640 1 -- 172.21.15.151:0/3133784695 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7faac407aaa0 msgr2=0x7faac407cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:56.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:56.857+0000 7faaea200640 1 --2- 172.21.15.151:0/3133784695 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7faac407aaa0 0x7faac407cf60 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7faae4155c30 tx=0x7faad0007e90 comp rx=0 tx=0).stop 2024-09-16T15:54:56.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:56.857+0000 7faaea200640 1 -- 172.21.15.151:0/3133784695 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faae40b2440 msgr2=0x7faae4154a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:56.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:56.857+0000 7faaea200640 1 --2- 172.21.15.151:0/3133784695 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faae40b2440 0x7faae4154a70 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7faad400c760 tx=0x7faad400cc20 comp rx=0 tx=0).stop 2024-09-16T15:54:56.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:56.857+0000 7faaea200640 1 -- 172.21.15.151:0/3133784695 shutdown_connections 2024-09-16T15:54:56.858 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:56.857+0000 7faaea200640 1 --2- 172.21.15.151:0/3133784695 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7faac407aaa0 0x7faac407cf60 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:56.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:56.857+0000 7faaea200640 1 --2- 172.21.15.151:0/3133784695 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faae40ccf00 0x7faae4148680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:56.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:56.857+0000 7faaea200640 1 --2- 172.21.15.151:0/3133784695 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7faae415f5a0 0x7faae415b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:56.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:56.857+0000 7faaea200640 1 --2- 172.21.15.151:0/3133784695 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faae40b2440 0x7faae4154a70 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:56.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:56.857+0000 7faaea200640 1 -- 172.21.15.151:0/3133784695 >> 172.21.15.151:0/3133784695 conn(0x7faae40b1950 msgr2=0x7faae40cdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:56.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:56.857+0000 7faaea200640 1 -- 172.21.15.151:0/3133784695 shutdown_connections 2024-09-16T15:54:56.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:56.857+0000 7faaea200640 1 -- 172.21.15.151:0/3133784695 wait complete. 2024-09-16T15:54:56.875 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:54:57.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.033+0000 7f1055600640 1 -- 172.21.15.151:0/966432782 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1034004d90 msgr2=0x7f1034005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:57.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.033+0000 7f1055600640 1 --2- 172.21.15.151:0/966432782 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1034004d90 0x7f1034005210 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f1048009ed0 tx=0x7f1034005830 comp rx=0 tx=0).stop 2024-09-16T15:54:57.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.033+0000 7f104f400640 1 -- 172.21.15.151:0/966432782 reap_dead start 2024-09-16T15:54:57.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.033+0000 7f1055600640 1 -- 172.21.15.151:0/966432782 shutdown_connections 2024-09-16T15:54:57.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.033+0000 7f1055600640 1 -- 172.21.15.151:0/966432782 >> 172.21.15.151:0/966432782 conn(0x7f10500b1950 msgr2=0x7f10500b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:57.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.033+0000 7f1055600640 1 -- 172.21.15.151:0/966432782 shutdown_connections 2024-09-16T15:54:57.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.033+0000 7f1055600640 1 -- 172.21.15.151:0/966432782 wait complete. 2024-09-16T15:54:57.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.035+0000 7f1055600640 1 Processor -- start 2024-09-16T15:54:57.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.035+0000 7f1055600640 1 -- start start 2024-09-16T15:54:57.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.036+0000 7f1055600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1034004d90 0x7f10501f44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:57.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.036+0000 7f1055600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10500d03c0 0x7f105014e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:57.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.036+0000 7f1055600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10500cf360 0x7f10500b4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:57.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.036+0000 7f1055600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1050074840 con 0x7f10500d03c0 2024-09-16T15:54:57.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.036+0000 7f1055600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f10500746c0 con 0x7f10500cf360 2024-09-16T15:54:57.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.036+0000 7f1055600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f105006f160 con 0x7f1034004d90 2024-09-16T15:54:57.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.036+0000 7f104e000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10500d03c0 0x7f105014e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:57.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.036+0000 7f104e000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10500d03c0 0x7f105014e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52536/0 (socket says 172.21.15.151:52536) 2024-09-16T15:54:57.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.036+0000 7f104e000640 1 -- 172.21.15.151:0/3961947772 learned_addr learned my addr 172.21.15.151:0/3961947772 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:54:57.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.036+0000 7f104ea00640 1 --2- 172.21.15.151:0/3961947772 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1034004d90 0x7f10501f44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:57.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.036+0000 7f104f400640 1 --2- 172.21.15.151:0/3961947772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10500cf360 0x7f10500b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:57.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.036+0000 7f104e000640 1 -- 172.21.15.151:0/3961947772 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1034004d90 msgr2=0x7f10501f44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:57.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.036+0000 7f104e000640 1 --2- 172.21.15.151:0/3961947772 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1034004d90 0x7f10501f44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:57.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.036+0000 7f104e000640 1 -- 172.21.15.151:0/3961947772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10500cf360 msgr2=0x7f10500b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:57.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.036+0000 7f104e000640 1 --2- 172.21.15.151:0/3961947772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10500cf360 0x7f10500b4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:57.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.036+0000 7f104e000640 1 -- 172.21.15.151:0/3961947772 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10500cebd0 con 0x7f10500d03c0 2024-09-16T15:54:57.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.036+0000 7f104f400640 1 --2- 172.21.15.151:0/3961947772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10500cf360 0x7f10500b4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:54:57.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.036+0000 7f104e000640 1 --2- 172.21.15.151:0/3961947772 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10500d03c0 0x7f105014e6e0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f103c006600 tx=0x7f103c0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:57.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.037+0000 7f1046a00640 1 -- 172.21.15.151:0/3961947772 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f103c015070 con 0x7f10500d03c0 2024-09-16T15:54:57.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.037+0000 7f1046a00640 1 -- 172.21.15.151:0/3961947772 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f103c011860 con 0x7f10500d03c0 2024-09-16T15:54:57.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.037+0000 7f1046a00640 1 -- 172.21.15.151:0/3961947772 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f103c011cb0 con 0x7f10500d03c0 2024-09-16T15:54:57.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.037+0000 7f1055600640 1 -- 172.21.15.151:0/3961947772 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10501f4d80 con 0x7f10500d03c0 2024-09-16T15:54:57.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.037+0000 7f1055600640 1 -- 172.21.15.151:0/3961947772 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10501f4a20 con 0x7f10500d03c0 2024-09-16T15:54:57.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.038+0000 7f1055600640 1 -- 172.21.15.151:0/3961947772 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f105006f6c0 con 0x7f10500d03c0 2024-09-16T15:54:57.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.039+0000 7f1046a00640 1 -- 172.21.15.151:0/3961947772 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 50) ==== 104398+0+0 (secure 0 0 0) 0x7f103c018070 con 0x7f10500d03c0 2024-09-16T15:54:57.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.040+0000 7f1046a00640 1 --2- 172.21.15.151:0/3961947772 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f103007aad0 0x7f103007cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:57.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.040+0000 7f1046a00640 1 -- 172.21.15.151:0/3961947772 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5849+0+0 (secure 0 0 0) 0x7f103c09a730 con 0x7f10500d03c0 2024-09-16T15:54:57.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.041+0000 7f104ea00640 1 --2- 172.21.15.151:0/3961947772 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f103007aad0 0x7f103007cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:57.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.041+0000 7f104ea00640 1 --2- 172.21.15.151:0/3961947772 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f103007aad0 0x7f103007cf90 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f1040000c00 tx=0x7f1040006b10 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:57.045 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.044+0000 7f1046a00640 1 -- 172.21.15.151:0/3961947772 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f103c0641b0 con 0x7f10500d03c0 2024-09-16T15:54:57.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.234+0000 7f1055600640 1 -- 172.21.15.151:0/3961947772 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1050069ac0 con 0x7f10500d03c0 2024-09-16T15:54:57.280 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.235+0000 7f1046a00640 1 -- 172.21.15.151:0/3961947772 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v144) ==== 74+0+17034 (secure 0 0 0) 0x7f103c068be0 con 0x7f10500d03c0 2024-09-16T15:54:57.280 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:54:57.280 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":144,"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","created":"2024-09-16T15:38:56.348535+0000","modified":"2024-09-16T15:54:55.864289+0000","last_up_change":"2024-09-16T15:39:10.681384+0000","last_in_change":"2024-09-16T15:39:00.544477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:39:11.602252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:54:53.628710+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-16T15:54:54.832847+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5524076-a903-4d0a-af6a-caf474c4765d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6809","nonce":3111342741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6811","nonce":3111342741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6815","nonce":3111342741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6813","nonce":3111342741}]},"public_addr":"172.21.15.151:6809/3111342741","cluster_addr":"172.21.15.151:6811/3111342741","heartbeat_back_addr":"172.21.15.151:6815/3111342741","heartbeat_front_addr":"172.21.15.151:6813/3111342741","state":["exists","up"]},{"osd":1,"uuid":"dd25925f-0ae1-494e-8529-4dc205759db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6801","nonce":3368304087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6803","nonce":3368304087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6807","nonce":3368304087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6805","nonce":3368304087}]},"public_addr":"172.21.15.151:6801/3368304087","cluster_addr":"172.21.15.151:6803/3368304087","heartbeat_back_addr":"172.21.15.151:6807/3368304087","heartbeat_front_addr":"172.21.15.151:6805/3368304087","state":["exists","up"]},{"osd":2,"uuid":"c2f18115-013d-4da6-a0fe-cb978f4494b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6823","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6825","nonce":3079373271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6827","nonce":3079373271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6831","nonce":3079373271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6829","nonce":3079373271}]},"public_addr":"172.21.15.151:6825/3079373271","cluster_addr":"172.21.15.151:6827/3079373271","heartbeat_back_addr":"172.21.15.151:6831/3079373271","heartbeat_front_addr":"172.21.15.151:6829/3079373271","state":["exists","up"]},{"osd":3,"uuid":"36edb718-2fcb-4d88-8ddc-c321eed4279d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6817","nonce":1142509041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6819","nonce":1142509041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6824","nonce":1142509041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6821","nonce":1142509041}]},"public_addr":"172.21.15.151:6817/1142509041","cluster_addr":"172.21.15.151:6819/1142509041","heartbeat_back_addr":"172.21.15.151:6824/1142509041","heartbeat_front_addr":"172.21.15.151:6821/1142509041","state":["exists","up"]},{"osd":4,"uuid":"64d33708-cd0a-4236-abe3-1b646e82ab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6811","nonce":2735495271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6813","nonce":2735495271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6817","nonce":2735495271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6815","nonce":2735495271}]},"public_addr":"172.21.15.167:6811/2735495271","cluster_addr":"172.21.15.167:6813/2735495271","heartbeat_back_addr":"172.21.15.167:6817/2735495271","heartbeat_front_addr":"172.21.15.167:6815/2735495271","state":["exists","up"]},{"osd":5,"uuid":"f14b44a9-5910-458c-b518-09a8d0de94de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6827","nonce":1494916572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6829","nonce":1494916572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6833","nonce":1494916572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6831","nonce":1494916572}]},"public_addr":"172.21.15.167:6827/1494916572","cluster_addr":"172.21.15.167:6829/1494916572","heartbeat_back_addr":"172.21.15.167:6833/1494916572","heartbeat_front_addr":"172.21.15.167:6831/1494916572","state":["exists","up"]},{"osd":6,"uuid":"39335545-2654-4710-84b0-a9aa47cfcbba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6819","nonce":2538273273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6821","nonce":2538273273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6825","nonce":2538273273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6823","nonce":2538273273}]},"public_addr":"172.21.15.167:6819/2538273273","cluster_addr":"172.21.15.167:6821/2538273273","heartbeat_back_addr":"172.21.15.167:6825/2538273273","heartbeat_front_addr":"172.21.15.167:6823/2538273273","state":["exists","up"]},{"osd":7,"uuid":"4f8bfc81-53dd-4339-ac22-1a4007d9f963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6803","nonce":265072227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6805","nonce":265072227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6809","nonce":265072227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6807","nonce":265072227}]},"public_addr":"172.21.15.167:6803/265072227","cluster_addr":"172.21.15.167:6805/265072227","heartbeat_back_addr":"172.21.15.167:6809/265072227","heartbeat_front_addr":"172.21.15.167:6807/265072227","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.150359+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.084358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.722368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.332809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.320709+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.395153+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.935250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.801443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:54:57.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.238+0000 7f1055600640 1 -- 172.21.15.151:0/3961947772 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f103007aad0 msgr2=0x7f103007cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:57.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.238+0000 7f1055600640 1 --2- 172.21.15.151:0/3961947772 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f103007aad0 0x7f103007cf90 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f1040000c00 tx=0x7f1040006b10 comp rx=0 tx=0).stop 2024-09-16T15:54:57.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.238+0000 7f1055600640 1 -- 172.21.15.151:0/3961947772 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10500d03c0 msgr2=0x7f105014e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:57.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.238+0000 7f1055600640 1 --2- 172.21.15.151:0/3961947772 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10500d03c0 0x7f105014e6e0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f103c006600 tx=0x7f103c0039c0 comp rx=0 tx=0).stop 2024-09-16T15:54:57.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.238+0000 7f1055600640 1 -- 172.21.15.151:0/3961947772 shutdown_connections 2024-09-16T15:54:57.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.238+0000 7f1055600640 1 --2- 172.21.15.151:0/3961947772 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f103007aad0 0x7f103007cf90 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:57.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.238+0000 7f1055600640 1 --2- 172.21.15.151:0/3961947772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f10500cf360 0x7f10500b4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:57.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.238+0000 7f1055600640 1 --2- 172.21.15.151:0/3961947772 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10500d03c0 0x7f105014e6e0 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:57.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.238+0000 7f1055600640 1 --2- 172.21.15.151:0/3961947772 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1034004d90 0x7f10501f44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:57.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.238+0000 7f1055600640 1 -- 172.21.15.151:0/3961947772 >> 172.21.15.151:0/3961947772 conn(0x7f10500b1950 msgr2=0x7f105009cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:57.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.238+0000 7f1055600640 1 -- 172.21.15.151:0/3961947772 shutdown_connections 2024-09-16T15:54:57.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.238+0000 7f1055600640 1 -- 172.21.15.151:0/3961947772 wait complete. 2024-09-16T15:54:57.283 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-16T15:54:57.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.443+0000 7fce77200640 1 -- 172.21.15.151:0/3542514489 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce780b3450 msgr2=0x7fce780b3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:57.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.443+0000 7fce77200640 1 --2- 172.21.15.151:0/3542514489 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce780b3450 0x7fce780b3890 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fce64009830 tx=0x7fce6402a480 comp rx=0 tx=0).stop 2024-09-16T15:54:57.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.443+0000 7fce77200640 1 -- 172.21.15.151:0/3542514489 shutdown_connections 2024-09-16T15:54:57.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.443+0000 7fce77200640 1 --2- 172.21.15.151:0/3542514489 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce780b3450 0x7fce780b3890 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:57.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.443+0000 7fce77200640 1 --2- 172.21.15.151:0/3542514489 >> v2:172.21.15.167:3300/0 conn(0x7fce7815df60 0x7fce780d0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:57.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.443+0000 7fce77200640 1 -- 172.21.15.151:0/3542514489 >> 172.21.15.151:0/3542514489 conn(0x7fce780d3d50 msgr2=0x7fce780d4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:57.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.443+0000 7fce77200640 1 -- 172.21.15.151:0/3542514489 shutdown_connections 2024-09-16T15:54:57.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.443+0000 7fce77200640 1 -- 172.21.15.151:0/3542514489 wait complete. 2024-09-16T15:54:57.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.445+0000 7fce77200640 1 Processor -- start 2024-09-16T15:54:57.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.445+0000 7fce77200640 1 -- start start 2024-09-16T15:54:57.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.446+0000 7fce77200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce780b3450 0x7fce7814f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:57.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.446+0000 7fce77200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce780cf4b0 0x7fce780b6b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:57.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.446+0000 7fce77200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce7815df60 0x7fce780b5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:57.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.446+0000 7fce77200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fce780d2a40 con 0x7fce780cf4b0 2024-09-16T15:54:57.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.446+0000 7fce77200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fce780d2bc0 con 0x7fce7815df60 2024-09-16T15:54:57.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.446+0000 7fce77200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fce780cf960 con 0x7fce780b3450 2024-09-16T15:54:57.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.446+0000 7fce75400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce780cf4b0 0x7fce780b6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:57.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.446+0000 7fce75400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce780cf4b0 0x7fce780b6b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52552/0 (socket says 172.21.15.151:52552) 2024-09-16T15:54:57.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.446+0000 7fce75400640 1 -- 172.21.15.151:0/3285134474 learned_addr learned my addr 172.21.15.151:0/3285134474 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:54:57.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.446+0000 7fce75e00640 1 --2- 172.21.15.151:0/3285134474 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce780b3450 0x7fce7814f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:57.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.446+0000 7fce76800640 1 --2- 172.21.15.151:0/3285134474 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce7815df60 0x7fce780b5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:57.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.446+0000 7fce75400640 1 -- 172.21.15.151:0/3285134474 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce780b3450 msgr2=0x7fce7814f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:57.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.446+0000 7fce75400640 1 --2- 172.21.15.151:0/3285134474 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce780b3450 0x7fce7814f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:57.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.446+0000 7fce75400640 1 -- 172.21.15.151:0/3285134474 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce7815df60 msgr2=0x7fce780b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:57.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.446+0000 7fce75400640 1 --2- 172.21.15.151:0/3285134474 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce7815df60 0x7fce780b5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:57.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.446+0000 7fce75400640 1 -- 172.21.15.151:0/3285134474 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce7814f6e0 con 0x7fce780cf4b0 2024-09-16T15:54:57.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.447+0000 7fce75400640 1 --2- 172.21.15.151:0/3285134474 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce780cf4b0 0x7fce780b6b80 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7fce64009950 tx=0x7fce64032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:57.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.447+0000 7fce6e000640 1 -- 172.21.15.151:0/3285134474 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce64040070 con 0x7fce780cf4b0 2024-09-16T15:54:57.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.447+0000 7fce6e000640 1 -- 172.21.15.151:0/3285134474 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fce64005310 con 0x7fce780cf4b0 2024-09-16T15:54:57.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.447+0000 7fce6e000640 1 -- 172.21.15.151:0/3285134474 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce64005c90 con 0x7fce780cf4b0 2024-09-16T15:54:57.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.447+0000 7fce77200640 1 -- 172.21.15.151:0/3285134474 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce780cc4e0 con 0x7fce780cf4b0 2024-09-16T15:54:57.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.447+0000 7fce77200640 1 -- 172.21.15.151:0/3285134474 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce780ccaa0 con 0x7fce780cf4b0 2024-09-16T15:54:57.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.448+0000 7fce77200640 1 -- 172.21.15.151:0/3285134474 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce780b3890 con 0x7fce780cf4b0 2024-09-16T15:54:57.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.449+0000 7fce6e000640 1 -- 172.21.15.151:0/3285134474 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 50) ==== 104398+0+0 (secure 0 0 0) 0x7fce64039040 con 0x7fce780cf4b0 2024-09-16T15:54:57.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.450+0000 7fce6e000640 1 --2- 172.21.15.151:0/3285134474 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fce5807aaa0 0x7fce5807cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:57.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.451+0000 7fce6e000640 1 -- 172.21.15.151:0/3285134474 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5849+0+0 (secure 0 0 0) 0x7fce640babc0 con 0x7fce780cf4b0 2024-09-16T15:54:57.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.451+0000 7fce75e00640 1 --2- 172.21.15.151:0/3285134474 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fce5807aaa0 0x7fce5807cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:57.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.452+0000 7fce75e00640 1 --2- 172.21.15.151:0/3285134474 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fce5807aaa0 0x7fce5807cf60 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fce680095f0 tx=0x7fce68005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:57.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.455+0000 7fce6e000640 1 -- 172.21.15.151:0/3285134474 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fce7814f6e0 con 0x7fce780cf4b0 2024-09-16T15:54:57.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:57.679+0000 7fce77200640 1 -- 172.21.15.151:0/3285134474 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fce780d16f0 con 0x7fce780cf4b0 2024-09-16T15:54:58.432 INFO:tasks.ceph.mds.b.smithi167.stderr:2024-09-16T15:54:58.432+0000 7f2179600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-16T15:54:58.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:58.477+0000 7fce6e000640 1 -- 172.21.15.151:0/3285134474 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v95) ==== 118+0+0 (secure 0 0 0) 0x7fce640845b0 con 0x7fce780cf4b0 2024-09-16T15:54:58.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:58.480+0000 7fce77200640 1 -- 172.21.15.151:0/3285134474 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fce5807aaa0 msgr2=0x7fce5807cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:58.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:58.480+0000 7fce77200640 1 --2- 172.21.15.151:0/3285134474 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fce5807aaa0 0x7fce5807cf60 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fce680095f0 tx=0x7fce68005f90 comp rx=0 tx=0).stop 2024-09-16T15:54:58.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:58.480+0000 7fce77200640 1 -- 172.21.15.151:0/3285134474 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce780cf4b0 msgr2=0x7fce780b6b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:58.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:58.480+0000 7fce77200640 1 --2- 172.21.15.151:0/3285134474 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce780cf4b0 0x7fce780b6b80 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7fce64009950 tx=0x7fce64032c10 comp rx=0 tx=0).stop 2024-09-16T15:54:58.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:58.480+0000 7fce77200640 1 -- 172.21.15.151:0/3285134474 shutdown_connections 2024-09-16T15:54:58.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:58.480+0000 7fce77200640 1 --2- 172.21.15.151:0/3285134474 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fce5807aaa0 0x7fce5807cf60 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:58.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:58.480+0000 7fce77200640 1 --2- 172.21.15.151:0/3285134474 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fce7815df60 0x7fce780b5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:58.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:58.480+0000 7fce77200640 1 --2- 172.21.15.151:0/3285134474 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce780cf4b0 0x7fce780b6b80 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:58.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:58.480+0000 7fce77200640 1 --2- 172.21.15.151:0/3285134474 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fce780b3450 0x7fce7814f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:58.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:58.480+0000 7fce77200640 1 -- 172.21.15.151:0/3285134474 >> 172.21.15.151:0/3285134474 conn(0x7fce780d3d50 msgr2=0x7fce7806e240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:58.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:58.480+0000 7fce77200640 1 -- 172.21.15.151:0/3285134474 shutdown_connections 2024-09-16T15:54:58.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:58.480+0000 7fce77200640 1 -- 172.21.15.151:0/3285134474 wait complete. 2024-09-16T15:54:58.499 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-16T15:54:58.499 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-16T15:54:58.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:58.664+0000 7fe897200640 1 -- 172.21.15.151:0/2958594186 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe87c004d40 msgr2=0x7fe87c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:58.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:58.664+0000 7fe897200640 1 --2- 172.21.15.151:0/2958594186 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe87c004d40 0x7fe87c0051c0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7fe890009f00 tx=0x7fe87c005830 comp rx=0 tx=0).stop 2024-09-16T15:54:58.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:58.664+0000 7fe896800640 1 -- 172.21.15.151:0/2958594186 reap_dead start 2024-09-16T15:54:58.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:58.664+0000 7fe897200640 1 -- 172.21.15.151:0/2958594186 shutdown_connections 2024-09-16T15:54:58.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:58.664+0000 7fe897200640 1 -- 172.21.15.151:0/2958594186 >> 172.21.15.151:0/2958594186 conn(0x7fe8980cf9f0 msgr2=0x7fe8980cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:58.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:58.664+0000 7fe897200640 1 -- 172.21.15.151:0/2958594186 shutdown_connections 2024-09-16T15:54:58.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:58.664+0000 7fe897200640 1 -- 172.21.15.151:0/2958594186 wait complete. 2024-09-16T15:54:58.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:58.666+0000 7fe897200640 1 Processor -- start 2024-09-16T15:54:58.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:58.666+0000 7fe897200640 1 -- start start 2024-09-16T15:54:58.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:58.667+0000 7fe897200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe87c004d40 0x7fe898150f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:58.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:58.667+0000 7fe897200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe898159480 0x7fe8980c6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:58.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:58.667+0000 7fe897200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe8980c8570 0x7fe8980c89f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:58.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:58.667+0000 7fe897200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe8980ae760 con 0x7fe8980c8570 2024-09-16T15:54:58.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:58.667+0000 7fe897200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe8980ae5e0 con 0x7fe87c004d40 2024-09-16T15:54:58.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:58.667+0000 7fe897200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fe8980ae8e0 con 0x7fe898159480 2024-09-16T15:54:58.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:58.667+0000 7fe896800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe8980c8570 0x7fe8980c89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:58.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:58.667+0000 7fe896800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe8980c8570 0x7fe8980c89f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52566/0 (socket says 172.21.15.151:52566) 2024-09-16T15:54:58.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:58.667+0000 7fe896800640 1 -- 172.21.15.151:0/1472030622 learned_addr learned my addr 172.21.15.151:0/1472030622 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:54:58.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:58.667+0000 7fe895400640 1 --2- 172.21.15.151:0/1472030622 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe898159480 0x7fe8980c6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:58.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:58.667+0000 7fe895e00640 1 --2- 172.21.15.151:0/1472030622 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe87c004d40 0x7fe898150f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:58.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:58.668+0000 7fe896800640 1 -- 172.21.15.151:0/1472030622 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe898159480 msgr2=0x7fe8980c6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:58.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:58.668+0000 7fe896800640 1 --2- 172.21.15.151:0/1472030622 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe898159480 0x7fe8980c6ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:58.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:58.668+0000 7fe896800640 1 -- 172.21.15.151:0/1472030622 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe87c004d40 msgr2=0x7fe898150f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:58.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:58.668+0000 7fe896800640 1 --2- 172.21.15.151:0/1472030622 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe87c004d40 0x7fe898150f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:58.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:58.668+0000 7fe896800640 1 -- 172.21.15.151:0/1472030622 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8980c72f0 con 0x7fe8980c8570 2024-09-16T15:54:58.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:58.668+0000 7fe895e00640 1 --2- 172.21.15.151:0/1472030622 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe87c004d40 0x7fe898150f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:54:58.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:58.668+0000 7fe896800640 1 --2- 172.21.15.151:0/1472030622 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe8980c8570 0x7fe8980c89f0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7fe89000e380 tx=0x7fe890033d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:58.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:58.668+0000 7fe88e000640 1 -- 172.21.15.151:0/1472030622 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe890041070 con 0x7fe8980c8570 2024-09-16T15:54:58.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:58.668+0000 7fe88e000640 1 -- 172.21.15.151:0/1472030622 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe890003c60 con 0x7fe8980c8570 2024-09-16T15:54:58.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:58.668+0000 7fe897200640 1 -- 172.21.15.151:0/1472030622 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8980c75d0 con 0x7fe8980c8570 2024-09-16T15:54:58.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:58.668+0000 7fe88e000640 1 -- 172.21.15.151:0/1472030622 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe890038040 con 0x7fe8980c8570 2024-09-16T15:54:58.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:58.668+0000 7fe897200640 1 -- 172.21.15.151:0/1472030622 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8980c7b90 con 0x7fe8980c8570 2024-09-16T15:54:58.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:58.669+0000 7fe897200640 1 -- 172.21.15.151:0/1472030622 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8980739a0 con 0x7fe8980c8570 2024-09-16T15:54:58.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:58.671+0000 7fe88e000640 1 -- 172.21.15.151:0/1472030622 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 50) ==== 104398+0+0 (secure 0 0 0) 0x7fe890004290 con 0x7fe8980c8570 2024-09-16T15:54:58.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:58.672+0000 7fe88e000640 1 --2- 172.21.15.151:0/1472030622 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe87807aad0 0x7fe87807cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:58.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:58.672+0000 7fe88e000640 1 -- 172.21.15.151:0/1472030622 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5849+0+0 (secure 0 0 0) 0x7fe8900b9e80 con 0x7fe8980c8570 2024-09-16T15:54:58.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:58.673+0000 7fe895e00640 1 --2- 172.21.15.151:0/1472030622 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe87807aad0 0x7fe87807cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:58.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:58.673+0000 7fe895e00640 1 --2- 172.21.15.151:0/1472030622 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe87807aad0 0x7fe87807cf90 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fe888001860 tx=0x7fe888001150 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:58.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:58.676+0000 7fe88e000640 1 -- 172.21.15.151:0/1472030622 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe890083900 con 0x7fe8980c8570 2024-09-16T15:54:58.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:58.897+0000 7fe897200640 1 -- 172.21.15.151:0/1472030622 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fe8980d0710 con 0x7fe8980c8570 2024-09-16T15:54:59.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:59.478+0000 7fe88e000640 1 -- 172.21.15.151:0/1472030622 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v96) ==== 122+0+0 (secure 0 0 0) 0x7fe890088330 con 0x7fe8980c8570 2024-09-16T15:54:59.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:59.482+0000 7fe897200640 1 -- 172.21.15.151:0/1472030622 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe87807aad0 msgr2=0x7fe87807cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:59.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:59.482+0000 7fe897200640 1 --2- 172.21.15.151:0/1472030622 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe87807aad0 0x7fe87807cf90 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fe888001860 tx=0x7fe888001150 comp rx=0 tx=0).stop 2024-09-16T15:54:59.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:59.482+0000 7fe897200640 1 -- 172.21.15.151:0/1472030622 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe8980c8570 msgr2=0x7fe8980c89f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:59.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:59.482+0000 7fe897200640 1 --2- 172.21.15.151:0/1472030622 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe8980c8570 0x7fe8980c89f0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7fe89000e380 tx=0x7fe890033d30 comp rx=0 tx=0).stop 2024-09-16T15:54:59.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:59.482+0000 7fe897200640 1 -- 172.21.15.151:0/1472030622 shutdown_connections 2024-09-16T15:54:59.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:59.482+0000 7fe897200640 1 --2- 172.21.15.151:0/1472030622 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe87807aad0 0x7fe87807cf90 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:59.484 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:59.482+0000 7fe897200640 1 --2- 172.21.15.151:0/1472030622 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe8980c8570 0x7fe8980c89f0 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:59.484 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:59.482+0000 7fe897200640 1 --2- 172.21.15.151:0/1472030622 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe898159480 0x7fe8980c6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:59.484 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:59.482+0000 7fe897200640 1 --2- 172.21.15.151:0/1472030622 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe87c004d40 0x7fe898150f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:59.484 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:59.482+0000 7fe897200640 1 -- 172.21.15.151:0/1472030622 >> 172.21.15.151:0/1472030622 conn(0x7fe8980cf9f0 msgr2=0x7fe8980ca840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:59.484 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:59.482+0000 7fe897200640 1 -- 172.21.15.151:0/1472030622 shutdown_connections 2024-09-16T15:54:59.484 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:59.482+0000 7fe897200640 1 -- 172.21.15.151:0/1472030622 wait complete. 2024-09-16T15:54:59.501 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-16T15:54:59.501 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:54:59.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:59.659+0000 7fcb82e00640 1 -- 172.21.15.151:0/2336383818 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb60004000 msgr2=0x7fcb60004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:59.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:59.659+0000 7fcb82e00640 1 --2- 172.21.15.151:0/2336383818 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb60004000 0x7fcb60004800 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fcb68009b50 tx=0x7fcb60005830 comp rx=0 tx=0).stop 2024-09-16T15:54:59.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:59.660+0000 7fcb80e00640 1 -- 172.21.15.151:0/2336383818 reap_dead start 2024-09-16T15:54:59.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:59.660+0000 7fcb82e00640 1 -- 172.21.15.151:0/2336383818 shutdown_connections 2024-09-16T15:54:59.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:59.660+0000 7fcb82e00640 1 -- 172.21.15.151:0/2336383818 >> 172.21.15.151:0/2336383818 conn(0x7fcb7c0b2230 msgr2=0x7fcb7c0b2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:59.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:59.660+0000 7fcb82e00640 1 -- 172.21.15.151:0/2336383818 shutdown_connections 2024-09-16T15:54:59.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:59.660+0000 7fcb82e00640 1 -- 172.21.15.151:0/2336383818 wait complete. 2024-09-16T15:54:59.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:59.661+0000 7fcb82e00640 1 Processor -- start 2024-09-16T15:54:59.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:59.662+0000 7fcb82e00640 1 -- start start 2024-09-16T15:54:59.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:59.662+0000 7fcb82e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb60004000 0x7fcb7c155170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:59.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:59.662+0000 7fcb82e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb7c1f54b0 0x7fcb7c1f80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:59.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:59.662+0000 7fcb82e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb7c1f8610 0x7fcb7c06f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:59.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:59.662+0000 7fcb82e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcb7c070e40 con 0x7fcb7c1f54b0 2024-09-16T15:54:59.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:59.662+0000 7fcb82e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcb7c070cc0 con 0x7fcb7c1f8610 2024-09-16T15:54:59.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:59.662+0000 7fcb82e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fcb7c162860 con 0x7fcb60004000 2024-09-16T15:54:59.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:59.663+0000 7fcb7b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb7c1f54b0 0x7fcb7c1f80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:59.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:59.663+0000 7fcb7b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb7c1f54b0 0x7fcb7c1f80d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52576/0 (socket says 172.21.15.151:52576) 2024-09-16T15:54:59.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:59.663+0000 7fcb7b400640 1 -- 172.21.15.151:0/128722813 learned_addr learned my addr 172.21.15.151:0/128722813 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:54:59.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:59.663+0000 7fcb7be00640 1 --2- 172.21.15.151:0/128722813 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb60004000 0x7fcb7c155170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:59.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:59.663+0000 7fcb80e00640 1 --2- 172.21.15.151:0/128722813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb7c1f8610 0x7fcb7c06f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:59.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:59.663+0000 7fcb7b400640 1 -- 172.21.15.151:0/128722813 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb60004000 msgr2=0x7fcb7c155170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:59.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:59.663+0000 7fcb7b400640 1 --2- 172.21.15.151:0/128722813 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb60004000 0x7fcb7c155170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:59.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:59.663+0000 7fcb7b400640 1 -- 172.21.15.151:0/128722813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb7c1f8610 msgr2=0x7fcb7c06f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:59.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:59.663+0000 7fcb7b400640 1 --2- 172.21.15.151:0/128722813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb7c1f8610 0x7fcb7c06f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:59.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:59.663+0000 7fcb7b400640 1 -- 172.21.15.151:0/128722813 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb7c1556b0 con 0x7fcb7c1f54b0 2024-09-16T15:54:59.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:59.663+0000 7fcb7b400640 1 --2- 172.21.15.151:0/128722813 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb7c1f54b0 0x7fcb7c1f80d0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7fcb68036040 tx=0x7fcb68033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:59.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:59.664+0000 7fcb73e00640 1 -- 172.21.15.151:0/128722813 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb68038040 con 0x7fcb7c1f54b0 2024-09-16T15:54:59.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:59.664+0000 7fcb73e00640 1 -- 172.21.15.151:0/128722813 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcb68041070 con 0x7fcb7c1f54b0 2024-09-16T15:54:59.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:59.664+0000 7fcb73e00640 1 -- 172.21.15.151:0/128722813 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb68033df0 con 0x7fcb7c1f54b0 2024-09-16T15:54:59.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:59.664+0000 7fcb82e00640 1 -- 172.21.15.151:0/128722813 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb7c1f4cd0 con 0x7fcb7c1f54b0 2024-09-16T15:54:59.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:59.664+0000 7fcb82e00640 1 -- 172.21.15.151:0/128722813 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb7c1f8e00 con 0x7fcb7c1f54b0 2024-09-16T15:54:59.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:59.665+0000 7fcb82e00640 1 -- 172.21.15.151:0/128722813 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb7c1f4ae0 con 0x7fcb7c1f54b0 2024-09-16T15:54:59.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:59.666+0000 7fcb73e00640 1 -- 172.21.15.151:0/128722813 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 50) ==== 104398+0+0 (secure 0 0 0) 0x7fcb68046460 con 0x7fcb7c1f54b0 2024-09-16T15:54:59.668 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:59.667+0000 7fcb73e00640 1 --2- 172.21.15.151:0/128722813 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcb5c07aad0 0x7fcb5c07cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:54:59.668 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:59.667+0000 7fcb73e00640 1 -- 172.21.15.151:0/128722813 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5849+0+0 (secure 0 0 0) 0x7fcb680ba410 con 0x7fcb7c1f54b0 2024-09-16T15:54:59.668 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:59.667+0000 7fcb7be00640 1 --2- 172.21.15.151:0/128722813 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcb5c07aad0 0x7fcb5c07cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:54:59.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:59.668+0000 7fcb7be00640 1 --2- 172.21.15.151:0/128722813 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcb5c07aad0 0x7fcb5c07cf90 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c161700 tx=0x7fcb6c000ed0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:54:59.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:59.671+0000 7fcb73e00640 1 -- 172.21.15.151:0/128722813 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fcb7c1556b0 con 0x7fcb7c1f54b0 2024-09-16T15:54:59.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:59.893+0000 7fcb82e00640 1 -- 172.21.15.151:0/128722813 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcb7c06cb70 con 0x7fcb7c1f54b0 2024-09-16T15:54:59.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:59.893+0000 7fcb73e00640 1 -- 172.21.15.151:0/128722813 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) ==== 94+0+5236 (secure 0 0 0) 0x7fcb680844e0 con 0x7fcb7c1f54b0 2024-09-16T15:54:59.894 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:54:59.895 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":96,"btime":"2024-09-16T15:54:59:477590+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7739,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2795044069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2795044069},{"type":"v1","addr":"172.21.15.167:6837","nonce":2795044069}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":8644,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/675530853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":675530853},{"type":"v1","addr":"172.21.15.151:6835","nonce":675530853}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":9468,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/1930858450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1930858450},{"type":"v1","addr":"172.21.15.151:6834","nonce":1930858450}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:54:55.864255+0000","modified":"2024-09-16T15:54:58.898875+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9474},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9474":{"gid":9474,"name":"b","rank":0,"incarnation":93,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6836/1231498178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1231498178},{"type":"v1","addr":"172.21.15.167:6836","nonce":1231498178}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9474,"qdb_cluster":[9474]},"id":10}]} 2024-09-16T15:54:59.895 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 96 2024-09-16T15:54:59.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:59.896+0000 7fcb82e00640 1 -- 172.21.15.151:0/128722813 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcb5c07aad0 msgr2=0x7fcb5c07cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:59.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:59.896+0000 7fcb82e00640 1 --2- 172.21.15.151:0/128722813 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcb5c07aad0 0x7fcb5c07cf90 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c161700 tx=0x7fcb6c000ed0 comp rx=0 tx=0).stop 2024-09-16T15:54:59.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:59.896+0000 7fcb82e00640 1 -- 172.21.15.151:0/128722813 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb7c1f54b0 msgr2=0x7fcb7c1f80d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:54:59.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:59.896+0000 7fcb82e00640 1 --2- 172.21.15.151:0/128722813 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb7c1f54b0 0x7fcb7c1f80d0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7fcb68036040 tx=0x7fcb68033670 comp rx=0 tx=0).stop 2024-09-16T15:54:59.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:59.896+0000 7fcb82e00640 1 -- 172.21.15.151:0/128722813 shutdown_connections 2024-09-16T15:54:59.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:59.896+0000 7fcb82e00640 1 --2- 172.21.15.151:0/128722813 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcb5c07aad0 0x7fcb5c07cf90 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:59.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:59.896+0000 7fcb82e00640 1 --2- 172.21.15.151:0/128722813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb7c1f8610 0x7fcb7c06f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:59.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:59.896+0000 7fcb82e00640 1 --2- 172.21.15.151:0/128722813 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb7c1f54b0 0x7fcb7c1f80d0 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:59.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:59.896+0000 7fcb82e00640 1 --2- 172.21.15.151:0/128722813 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb60004000 0x7fcb7c155170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:54:59.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:59.896+0000 7fcb82e00640 1 -- 172.21.15.151:0/128722813 >> 172.21.15.151:0/128722813 conn(0x7fcb7c0b2230 msgr2=0x7fcb7c0af820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:54:59.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:59.896+0000 7fcb82e00640 1 -- 172.21.15.151:0/128722813 shutdown_connections 2024-09-16T15:54:59.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:54:59.896+0000 7fcb82e00640 1 -- 172.21.15.151:0/128722813 wait complete. 2024-09-16T15:54:59.913 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:55:00.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.069+0000 7f1dfb600640 1 -- 172.21.15.151:0/1294155231 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1df40af3d0 msgr2=0x7f1df406f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:00.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.069+0000 7f1dfb600640 1 --2- 172.21.15.151:0/1294155231 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1df40af3d0 0x7f1df406f660 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f1de0009830 tx=0x7f1de002a520 comp rx=0 tx=0).stop 2024-09-16T15:55:00.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.069+0000 7f1dfb600640 1 -- 172.21.15.151:0/1294155231 shutdown_connections 2024-09-16T15:55:00.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.069+0000 7f1dfb600640 1 --2- 172.21.15.151:0/1294155231 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1df40af3d0 0x7f1df406f660 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:00.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.069+0000 7f1dfb600640 1 -- 172.21.15.151:0/1294155231 >> 172.21.15.151:0/1294155231 conn(0x7f1df40b1950 msgr2=0x7f1df40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:55:00.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.069+0000 7f1dfb600640 1 -- 172.21.15.151:0/1294155231 shutdown_connections 2024-09-16T15:55:00.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.069+0000 7f1dfb600640 1 -- 172.21.15.151:0/1294155231 wait complete. 2024-09-16T15:55:00.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.071+0000 7f1dfb600640 1 Processor -- start 2024-09-16T15:55:00.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.071+0000 7f1dfb600640 1 -- start start 2024-09-16T15:55:00.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.072+0000 7f1dfb600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1df4072df0 0x7f1df4156630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:00.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.072+0000 7f1dfb600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1df40af3d0 0x7f1df41f7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:00.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.072+0000 7f1dfb600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1df4161550 0x7f1df41f4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:00.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.072+0000 7f1dfb600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1df4073950 con 0x7f1df4072df0 2024-09-16T15:55:00.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.072+0000 7f1dfb600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1df406fdb0 con 0x7f1df4161550 2024-09-16T15:55:00.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.072+0000 7f1dfb600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f1df4073ad0 con 0x7f1df40af3d0 2024-09-16T15:55:00.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.072+0000 7f1df8c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1df4072df0 0x7f1df4156630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:00.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.072+0000 7f1df8c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1df4072df0 0x7f1df4156630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52580/0 (socket says 172.21.15.151:52580) 2024-09-16T15:55:00.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.072+0000 7f1df8c00640 1 -- 172.21.15.151:0/2197371393 learned_addr learned my addr 172.21.15.151:0/2197371393 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:55:00.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.072+0000 7f1df9600640 1 --2- 172.21.15.151:0/2197371393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1df4161550 0x7f1df41f4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:00.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.072+0000 7f1df3e00640 1 --2- 172.21.15.151:0/2197371393 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1df40af3d0 0x7f1df41f7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:00.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.072+0000 7f1df8c00640 1 -- 172.21.15.151:0/2197371393 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1df40af3d0 msgr2=0x7f1df41f7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:00.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.072+0000 7f1df8c00640 1 --2- 172.21.15.151:0/2197371393 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1df40af3d0 0x7f1df41f7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:00.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.072+0000 7f1df8c00640 1 -- 172.21.15.151:0/2197371393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1df4161550 msgr2=0x7f1df41f4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:00.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.072+0000 7f1df8c00640 1 --2- 172.21.15.151:0/2197371393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1df4161550 0x7f1df41f4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:00.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.072+0000 7f1df8c00640 1 -- 172.21.15.151:0/2197371393 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1df4156b70 con 0x7f1df4072df0 2024-09-16T15:55:00.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.072+0000 7f1df3e00640 1 --2- 172.21.15.151:0/2197371393 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1df40af3d0 0x7f1df41f7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:55:00.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.073+0000 7f1df8c00640 1 --2- 172.21.15.151:0/2197371393 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1df4072df0 0x7f1df4156630 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f1de800c3f0 tx=0x7f1de800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:55:00.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.073+0000 7f1df0c00640 1 -- 172.21.15.151:0/2197371393 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de8017070 con 0x7f1df4072df0 2024-09-16T15:55:00.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.073+0000 7f1df0c00640 1 -- 172.21.15.151:0/2197371393 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1de80053e0 con 0x7f1df4072df0 2024-09-16T15:55:00.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.073+0000 7f1dfb600640 1 -- 172.21.15.151:0/2197371393 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1df41f85d0 con 0x7f1df4072df0 2024-09-16T15:55:00.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.073+0000 7f1df0c00640 1 -- 172.21.15.151:0/2197371393 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de8003870 con 0x7f1df4072df0 2024-09-16T15:55:00.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.073+0000 7f1dfb600640 1 -- 172.21.15.151:0/2197371393 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1df41f8b90 con 0x7f1df4072df0 2024-09-16T15:55:00.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.074+0000 7f1dfb600640 1 -- 172.21.15.151:0/2197371393 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1df4073fc0 con 0x7f1df4072df0 2024-09-16T15:55:00.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.075+0000 7f1df0c00640 1 -- 172.21.15.151:0/2197371393 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 50) ==== 104398+0+0 (secure 0 0 0) 0x7f1de801a070 con 0x7f1df4072df0 2024-09-16T15:55:00.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.077+0000 7f1df0c00640 1 --2- 172.21.15.151:0/2197371393 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1dd407aaa0 0x7f1dd407cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:00.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.077+0000 7f1df0c00640 1 -- 172.21.15.151:0/2197371393 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5849+0+0 (secure 0 0 0) 0x7f1de8098dd0 con 0x7f1df4072df0 2024-09-16T15:55:00.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.077+0000 7f1df3e00640 1 --2- 172.21.15.151:0/2197371393 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1dd407aaa0 0x7f1dd407cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:00.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.078+0000 7f1df3e00640 1 --2- 172.21.15.151:0/2197371393 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1dd407aaa0 0x7f1dd407cf60 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f1de0034000 tx=0x7f1de0004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:55:00.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.081+0000 7f1df0c00640 1 -- 172.21.15.151:0/2197371393 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1de80627d0 con 0x7f1df4072df0 2024-09-16T15:55:00.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.267+0000 7f1dfb600640 1 -- 172.21.15.151:0/2197371393 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1df41f82c0 con 0x7f1df4072df0 2024-09-16T15:55:00.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.268+0000 7f1df0c00640 1 -- 172.21.15.151:0/2197371393 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) ==== 74+0+17021 (secure 0 0 0) 0x7f1de8067200 con 0x7f1df4072df0 2024-09-16T15:55:00.269 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:55:00.269 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":145,"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","created":"2024-09-16T15:38:56.348535+0000","modified":"2024-09-16T15:54:57.461534+0000","last_up_change":"2024-09-16T15:39:10.681384+0000","last_in_change":"2024-09-16T15:39:00.544477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:39:11.602252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:54:53.628710+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-16T15:54:54.832847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5524076-a903-4d0a-af6a-caf474c4765d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6809","nonce":3111342741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6811","nonce":3111342741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6815","nonce":3111342741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6813","nonce":3111342741}]},"public_addr":"172.21.15.151:6809/3111342741","cluster_addr":"172.21.15.151:6811/3111342741","heartbeat_back_addr":"172.21.15.151:6815/3111342741","heartbeat_front_addr":"172.21.15.151:6813/3111342741","state":["exists","up"]},{"osd":1,"uuid":"dd25925f-0ae1-494e-8529-4dc205759db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6801","nonce":3368304087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6803","nonce":3368304087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6807","nonce":3368304087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6805","nonce":3368304087}]},"public_addr":"172.21.15.151:6801/3368304087","cluster_addr":"172.21.15.151:6803/3368304087","heartbeat_back_addr":"172.21.15.151:6807/3368304087","heartbeat_front_addr":"172.21.15.151:6805/3368304087","state":["exists","up"]},{"osd":2,"uuid":"c2f18115-013d-4da6-a0fe-cb978f4494b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6823","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6825","nonce":3079373271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6827","nonce":3079373271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6831","nonce":3079373271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6829","nonce":3079373271}]},"public_addr":"172.21.15.151:6825/3079373271","cluster_addr":"172.21.15.151:6827/3079373271","heartbeat_back_addr":"172.21.15.151:6831/3079373271","heartbeat_front_addr":"172.21.15.151:6829/3079373271","state":["exists","up"]},{"osd":3,"uuid":"36edb718-2fcb-4d88-8ddc-c321eed4279d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6817","nonce":1142509041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6819","nonce":1142509041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6824","nonce":1142509041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6821","nonce":1142509041}]},"public_addr":"172.21.15.151:6817/1142509041","cluster_addr":"172.21.15.151:6819/1142509041","heartbeat_back_addr":"172.21.15.151:6824/1142509041","heartbeat_front_addr":"172.21.15.151:6821/1142509041","state":["exists","up"]},{"osd":4,"uuid":"64d33708-cd0a-4236-abe3-1b646e82ab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6811","nonce":2735495271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6813","nonce":2735495271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6817","nonce":2735495271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6815","nonce":2735495271}]},"public_addr":"172.21.15.167:6811/2735495271","cluster_addr":"172.21.15.167:6813/2735495271","heartbeat_back_addr":"172.21.15.167:6817/2735495271","heartbeat_front_addr":"172.21.15.167:6815/2735495271","state":["exists","up"]},{"osd":5,"uuid":"f14b44a9-5910-458c-b518-09a8d0de94de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6827","nonce":1494916572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6829","nonce":1494916572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6833","nonce":1494916572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6831","nonce":1494916572}]},"public_addr":"172.21.15.167:6827/1494916572","cluster_addr":"172.21.15.167:6829/1494916572","heartbeat_back_addr":"172.21.15.167:6833/1494916572","heartbeat_front_addr":"172.21.15.167:6831/1494916572","state":["exists","up"]},{"osd":6,"uuid":"39335545-2654-4710-84b0-a9aa47cfcbba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6819","nonce":2538273273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6821","nonce":2538273273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6825","nonce":2538273273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6823","nonce":2538273273}]},"public_addr":"172.21.15.167:6819/2538273273","cluster_addr":"172.21.15.167:6821/2538273273","heartbeat_back_addr":"172.21.15.167:6825/2538273273","heartbeat_front_addr":"172.21.15.167:6823/2538273273","state":["exists","up"]},{"osd":7,"uuid":"4f8bfc81-53dd-4339-ac22-1a4007d9f963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6803","nonce":265072227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6805","nonce":265072227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6809","nonce":265072227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6807","nonce":265072227}]},"public_addr":"172.21.15.167:6803/265072227","cluster_addr":"172.21.15.167:6805/265072227","heartbeat_back_addr":"172.21.15.167:6809/265072227","heartbeat_front_addr":"172.21.15.167:6807/265072227","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.150359+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.084358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.722368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.332809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.320709+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.395153+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.935250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.801443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:55:00.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.270+0000 7f1dfb600640 1 -- 172.21.15.151:0/2197371393 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1dd407aaa0 msgr2=0x7f1dd407cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:00.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.270+0000 7f1dfb600640 1 --2- 172.21.15.151:0/2197371393 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1dd407aaa0 0x7f1dd407cf60 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f1de0034000 tx=0x7f1de0004730 comp rx=0 tx=0).stop 2024-09-16T15:55:00.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.270+0000 7f1dfb600640 1 -- 172.21.15.151:0/2197371393 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1df4072df0 msgr2=0x7f1df4156630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:00.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.270+0000 7f1dfb600640 1 --2- 172.21.15.151:0/2197371393 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1df4072df0 0x7f1df4156630 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f1de800c3f0 tx=0x7f1de800c8b0 comp rx=0 tx=0).stop 2024-09-16T15:55:00.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.271+0000 7f1dfb600640 1 -- 172.21.15.151:0/2197371393 shutdown_connections 2024-09-16T15:55:00.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.271+0000 7f1dfb600640 1 --2- 172.21.15.151:0/2197371393 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1dd407aaa0 0x7f1dd407cf60 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:00.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.271+0000 7f1dfb600640 1 --2- 172.21.15.151:0/2197371393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1df4161550 0x7f1df41f4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:00.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.271+0000 7f1dfb600640 1 --2- 172.21.15.151:0/2197371393 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1df40af3d0 0x7f1df41f7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:00.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.271+0000 7f1dfb600640 1 --2- 172.21.15.151:0/2197371393 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1df4072df0 0x7f1df4156630 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:00.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.271+0000 7f1dfb600640 1 -- 172.21.15.151:0/2197371393 >> 172.21.15.151:0/2197371393 conn(0x7f1df40b1950 msgr2=0x7f1df415e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:55:00.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.271+0000 7f1dfb600640 1 -- 172.21.15.151:0/2197371393 shutdown_connections 2024-09-16T15:55:00.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.271+0000 7f1dfb600640 1 -- 172.21.15.151:0/2197371393 wait complete. 2024-09-16T15:55:00.287 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T15:55:00.288 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:55:00.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.444+0000 7f9bbb200640 1 -- 172.21.15.151:0/3498040601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bb4072df0 msgr2=0x7f9bb40732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:00.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.444+0000 7f9bbb200640 1 --2- 172.21.15.151:0/3498040601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bb4072df0 0x7f9bb40732a0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f9bac005fd0 tx=0x7f9bac02aa10 comp rx=0 tx=0).stop 2024-09-16T15:55:00.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.444+0000 7f9bbb200640 1 -- 172.21.15.151:0/3498040601 shutdown_connections 2024-09-16T15:55:00.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.444+0000 7f9bbb200640 1 --2- 172.21.15.151:0/3498040601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bb4072df0 0x7f9bb40732a0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:00.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.444+0000 7f9bbb200640 1 --2- 172.21.15.151:0/3498040601 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9bb40af3d0 0x7f9bb406f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:00.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.444+0000 7f9bbb200640 1 -- 172.21.15.151:0/3498040601 >> 172.21.15.151:0/3498040601 conn(0x7f9bb40b1950 msgr2=0x7f9bb40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:55:00.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.444+0000 7f9bbb200640 1 -- 172.21.15.151:0/3498040601 shutdown_connections 2024-09-16T15:55:00.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.444+0000 7f9bbb200640 1 -- 172.21.15.151:0/3498040601 wait complete. 2024-09-16T15:55:00.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.446+0000 7f9bbb200640 1 Processor -- start 2024-09-16T15:55:00.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.446+0000 7f9bbb200640 1 -- start start 2024-09-16T15:55:00.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.447+0000 7f9bbb200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9bb4072df0 0x7f9bb40cfe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:00.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.447+0000 7f9bbb200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9bb40af3d0 0x7f9bb40cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:00.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.447+0000 7f9bbb200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bb4161550 0x7f9bb4154310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:00.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.447+0000 7f9bbb200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb40737e0 con 0x7f9bb40af3d0 2024-09-16T15:55:00.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.447+0000 7f9bbb200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb406fdb0 con 0x7f9bb4161550 2024-09-16T15:55:00.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.447+0000 7f9bbb200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f9bb4073960 con 0x7f9bb4072df0 2024-09-16T15:55:00.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.447+0000 7f9bb3400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9bb40af3d0 0x7f9bb40cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:00.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.447+0000 7f9bb3400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9bb40af3d0 0x7f9bb40cdd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52584/0 (socket says 172.21.15.151:52584) 2024-09-16T15:55:00.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.447+0000 7f9bb3400640 1 -- 172.21.15.151:0/125099794 learned_addr learned my addr 172.21.15.151:0/125099794 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:55:00.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.447+0000 7f9bb3e00640 1 --2- 172.21.15.151:0/125099794 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9bb4072df0 0x7f9bb40cfe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:00.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.447+0000 7f9bb9200640 1 --2- 172.21.15.151:0/125099794 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bb4161550 0x7f9bb4154310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:00.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.447+0000 7f9bb3400640 1 -- 172.21.15.151:0/125099794 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9bb4072df0 msgr2=0x7f9bb40cfe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:00.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.447+0000 7f9bb3400640 1 --2- 172.21.15.151:0/125099794 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9bb4072df0 0x7f9bb40cfe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:00.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.447+0000 7f9bb3400640 1 -- 172.21.15.151:0/125099794 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bb4161550 msgr2=0x7f9bb4154310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:00.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.447+0000 7f9bb3400640 1 --2- 172.21.15.151:0/125099794 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bb4161550 0x7f9bb4154310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:00.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.447+0000 7f9bb3400640 1 -- 172.21.15.151:0/125099794 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bb40cf0e0 con 0x7f9bb40af3d0 2024-09-16T15:55:00.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.448+0000 7f9bb3400640 1 --2- 172.21.15.151:0/125099794 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9bb40af3d0 0x7f9bb40cdd50 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f9ba000e9c0 tx=0x7f9ba000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:55:00.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.448+0000 7f9babe00640 1 -- 172.21.15.151:0/125099794 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba000cd80 con 0x7f9bb40af3d0 2024-09-16T15:55:00.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.448+0000 7f9babe00640 1 -- 172.21.15.151:0/125099794 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ba0004020 con 0x7f9bb40af3d0 2024-09-16T15:55:00.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.448+0000 7f9bbb200640 1 -- 172.21.15.151:0/125099794 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bb41f5890 con 0x7f9bb40af3d0 2024-09-16T15:55:00.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.448+0000 7f9babe00640 1 -- 172.21.15.151:0/125099794 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba0004450 con 0x7f9bb40af3d0 2024-09-16T15:55:00.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.448+0000 7f9bbb200640 1 -- 172.21.15.151:0/125099794 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bb40cf960 con 0x7f9bb40af3d0 2024-09-16T15:55:00.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.449+0000 7f9bbb200640 1 -- 172.21.15.151:0/125099794 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bb4073ec0 con 0x7f9bb40af3d0 2024-09-16T15:55:00.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.450+0000 7f9babe00640 1 -- 172.21.15.151:0/125099794 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 50) ==== 104398+0+0 (secure 0 0 0) 0x7f9ba00035f0 con 0x7f9bb40af3d0 2024-09-16T15:55:00.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.451+0000 7f9babe00640 1 --2- 172.21.15.151:0/125099794 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9b9407aad0 0x7f9b9407cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:00.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.451+0000 7f9babe00640 1 -- 172.21.15.151:0/125099794 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5849+0+0 (secure 0 0 0) 0x7f9ba009a3e0 con 0x7f9bb40af3d0 2024-09-16T15:55:00.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.452+0000 7f9bb3e00640 1 --2- 172.21.15.151:0/125099794 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9b9407aad0 0x7f9b9407cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:00.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.452+0000 7f9bb3e00640 1 --2- 172.21.15.151:0/125099794 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9b9407aad0 0x7f9b9407cf90 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4001720 tx=0x7f9ba4009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:55:00.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.456+0000 7f9babe00640 1 -- 172.21.15.151:0/125099794 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9bb41f5890 con 0x7f9bb40af3d0 2024-09-16T15:55:00.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.639+0000 7f9bbb200640 1 -- 172.21.15.151:0/125099794 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9bb40aec30 con 0x7f9b9407aad0 2024-09-16T15:55:00.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.644+0000 7f9babe00640 1 -- 172.21.15.151:0/125099794 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+248706 (secure 0 0 0) 0x7f9bb40aec30 con 0x7f9b9407aad0 2024-09-16T15:55:00.648 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:55:00.649 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-09-16T15:55:00.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.648+0000 7f9bbb200640 1 -- 172.21.15.151:0/125099794 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9b9407aad0 msgr2=0x7f9b9407cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:00.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.648+0000 7f9bbb200640 1 --2- 172.21.15.151:0/125099794 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9b9407aad0 0x7f9b9407cf90 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4001720 tx=0x7f9ba4009040 comp rx=0 tx=0).stop 2024-09-16T15:55:00.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.648+0000 7f9bbb200640 1 -- 172.21.15.151:0/125099794 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9bb40af3d0 msgr2=0x7f9bb40cdd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:00.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.648+0000 7f9bbb200640 1 --2- 172.21.15.151:0/125099794 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9bb40af3d0 0x7f9bb40cdd50 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f9ba000e9c0 tx=0x7f9ba000ee80 comp rx=0 tx=0).stop 2024-09-16T15:55:00.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.648+0000 7f9bbb200640 1 -- 172.21.15.151:0/125099794 shutdown_connections 2024-09-16T15:55:00.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.648+0000 7f9bbb200640 1 --2- 172.21.15.151:0/125099794 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9b9407aad0 0x7f9b9407cf90 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:00.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.648+0000 7f9bbb200640 1 --2- 172.21.15.151:0/125099794 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9bb4161550 0x7f9bb4154310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:00.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.648+0000 7f9bbb200640 1 --2- 172.21.15.151:0/125099794 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9bb40af3d0 0x7f9bb40cdd50 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:00.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.648+0000 7f9bbb200640 1 --2- 172.21.15.151:0/125099794 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9bb4072df0 0x7f9bb40cfe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:00.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.648+0000 7f9bbb200640 1 -- 172.21.15.151:0/125099794 >> 172.21.15.151:0/125099794 conn(0x7f9bb40b1950 msgr2=0x7f9bb406ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:55:00.651 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.648+0000 7f9bbb200640 1 -- 172.21.15.151:0/125099794 shutdown_connections 2024-09-16T15:55:00.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.648+0000 7f9bbb200640 1 -- 172.21.15.151:0/125099794 wait complete. 2024-09-16T15:55:00.665 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":613,"stamp":"2024-09-16T15:54:59.958152+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":102,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1092900,"kb_used_data":15080,"kb_used_omap":12,"kb_used_meta":1077619,"kb_avail":753881820,"statfs":{"total":773094113280,"available":771974983680,"internally_reserved":0,"allocated":15441920,"data_stored":7480917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12738,"internal_metadata":1103482430},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.523823"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.445126+0000","last_change":"2024-09-16T15:54:55.445126+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.445126+0000","last_undegraded":"2024-09-16T15:54:55.445126+0000","last_fullsized":"2024-09-16T15:54:55.445126+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482752+0000","last_change":"2024-09-16T15:54:56.489100+0000","last_active":"2024-09-16T15:54:57.482752+0000","last_peered":"2024-09-16T15:54:57.482752+0000","last_clean":"2024-09-16T15:54:57.482752+0000","last_became_active":"2024-09-16T15:54:56.488821+0000","last_became_peered":"2024-09-16T15:54:56.488821+0000","last_unstale":"2024-09-16T15:54:57.482752+0000","last_undegraded":"2024-09-16T15:54:57.482752+0000","last_fullsized":"2024-09-16T15:54:57.482752+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:43:37.723000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.446549+0000","last_change":"2024-09-16T15:54:55.446549+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.446549+0000","last_undegraded":"2024-09-16T15:54:55.446549+0000","last_fullsized":"2024-09-16T15:54:55.446549+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.449901+0000","last_change":"2024-09-16T15:54:55.449901+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.449901+0000","last_undegraded":"2024-09-16T15:54:55.449901+0000","last_fullsized":"2024-09-16T15:54:55.449901+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-16T15:54:56.479967+0000","last_change":"2024-09-16T15:54:56.471793+0000","last_active":"2024-09-16T15:54:56.479967+0000","last_peered":"2024-09-16T15:54:56.479967+0000","last_clean":"2024-09-16T15:54:56.479967+0000","last_became_active":"2024-09-16T15:54:56.471048+0000","last_became_peered":"2024-09-16T15:54:56.471048+0000","last_unstale":"2024-09-16T15:54:56.479967+0000","last_undegraded":"2024-09-16T15:54:56.479967+0000","last_fullsized":"2024-09-16T15:54:56.479967+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:15:17.040666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.449521+0000","last_change":"2024-09-16T15:54:55.449521+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.449521+0000","last_undegraded":"2024-09-16T15:54:55.449521+0000","last_fullsized":"2024-09-16T15:54:55.449521+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.449819+0000","last_change":"2024-09-16T15:54:55.449819+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.449819+0000","last_undegraded":"2024-09-16T15:54:55.449819+0000","last_fullsized":"2024-09-16T15:54:55.449819+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.443482+0000","last_change":"2024-09-16T15:54:55.443482+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.443482+0000","last_undegraded":"2024-09-16T15:54:55.443482+0000","last_fullsized":"2024-09-16T15:54:55.443482+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.450377+0000","last_change":"2024-09-16T15:54:55.450377+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.450377+0000","last_undegraded":"2024-09-16T15:54:55.450377+0000","last_fullsized":"2024-09-16T15:54:55.450377+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.445356+0000","last_change":"2024-09-16T15:54:55.445356+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.445356+0000","last_undegraded":"2024-09-16T15:54:55.445356+0000","last_fullsized":"2024-09-16T15:54:55.445356+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.448658+0000","last_change":"2024-09-16T15:54:55.448658+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.448658+0000","last_undegraded":"2024-09-16T15:54:55.448658+0000","last_fullsized":"2024-09-16T15:54:55.448658+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.445187+0000","last_change":"2024-09-16T15:54:55.445187+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.445187+0000","last_undegraded":"2024-09-16T15:54:55.445187+0000","last_fullsized":"2024-09-16T15:54:55.445187+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-16T15:54:56.479929+0000","last_change":"2024-09-16T15:54:56.469650+0000","last_active":"2024-09-16T15:54:56.479929+0000","last_peered":"2024-09-16T15:54:56.479929+0000","last_clean":"2024-09-16T15:54:56.479929+0000","last_became_active":"2024-09-16T15:54:56.469407+0000","last_became_peered":"2024-09-16T15:54:56.469407+0000","last_unstale":"2024-09-16T15:54:56.479929+0000","last_undegraded":"2024-09-16T15:54:56.479929+0000","last_fullsized":"2024-09-16T15:54:56.479929+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:58:19.883027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.447273+0000","last_change":"2024-09-16T15:54:55.447273+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.447273+0000","last_undegraded":"2024-09-16T15:54:55.447273+0000","last_fullsized":"2024-09-16T15:54:55.447273+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-16T15:54:56.479726+0000","last_change":"2024-09-16T15:54:56.469340+0000","last_active":"2024-09-16T15:54:56.479726+0000","last_peered":"2024-09-16T15:54:56.479726+0000","last_clean":"2024-09-16T15:54:56.479726+0000","last_became_active":"2024-09-16T15:54:56.469088+0000","last_became_peered":"2024-09-16T15:54:56.469088+0000","last_unstale":"2024-09-16T15:54:56.479726+0000","last_undegraded":"2024-09-16T15:54:56.479726+0000","last_fullsized":"2024-09-16T15:54:56.479726+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:28:09.504034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.447718+0000","last_change":"2024-09-16T15:54:55.447718+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.447718+0000","last_undegraded":"2024-09-16T15:54:55.447718+0000","last_fullsized":"2024-09-16T15:54:55.447718+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.443888+0000","last_change":"2024-09-16T15:54:55.443888+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.443888+0000","last_undegraded":"2024-09-16T15:54:55.443888+0000","last_fullsized":"2024-09-16T15:54:55.443888+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-16T15:54:56.481270+0000","last_change":"2024-09-16T15:54:56.476168+0000","last_active":"2024-09-16T15:54:56.481270+0000","last_peered":"2024-09-16T15:54:56.481270+0000","last_clean":"2024-09-16T15:54:56.481270+0000","last_became_active":"2024-09-16T15:54:56.475925+0000","last_became_peered":"2024-09-16T15:54:56.475925+0000","last_unstale":"2024-09-16T15:54:56.481270+0000","last_undegraded":"2024-09-16T15:54:56.481270+0000","last_fullsized":"2024-09-16T15:54:56.481270+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:37:11.196350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.445058+0000","last_change":"2024-09-16T15:54:55.445058+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.445058+0000","last_undegraded":"2024-09-16T15:54:55.445058+0000","last_fullsized":"2024-09-16T15:54:55.445058+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.445531+0000","last_change":"2024-09-16T15:54:55.445531+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.445531+0000","last_undegraded":"2024-09-16T15:54:55.445531+0000","last_fullsized":"2024-09-16T15:54:55.445531+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.442779+0000","last_change":"2024-09-16T15:54:55.442779+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.442779+0000","last_undegraded":"2024-09-16T15:54:55.442779+0000","last_fullsized":"2024-09-16T15:54:55.442779+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.446794+0000","last_change":"2024-09-16T15:54:55.446794+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.446794+0000","last_undegraded":"2024-09-16T15:54:55.446794+0000","last_fullsized":"2024-09-16T15:54:55.446794+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.449202+0000","last_change":"2024-09-16T15:54:55.449202+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.449202+0000","last_undegraded":"2024-09-16T15:54:55.449202+0000","last_fullsized":"2024-09-16T15:54:55.449202+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-16T15:54:56.466175+0000","last_change":"2024-09-16T15:54:56.462956+0000","last_active":"2024-09-16T15:54:56.466175+0000","last_peered":"2024-09-16T15:54:56.466175+0000","last_clean":"2024-09-16T15:54:56.466175+0000","last_became_active":"2024-09-16T15:54:56.462662+0000","last_became_peered":"2024-09-16T15:54:56.462662+0000","last_unstale":"2024-09-16T15:54:56.466175+0000","last_undegraded":"2024-09-16T15:54:56.466175+0000","last_fullsized":"2024-09-16T15:54:56.466175+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:30:20.069211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482638+0000","last_change":"2024-09-16T15:54:56.481953+0000","last_active":"2024-09-16T15:54:57.482638+0000","last_peered":"2024-09-16T15:54:57.482638+0000","last_clean":"2024-09-16T15:54:57.482638+0000","last_became_active":"2024-09-16T15:54:56.481478+0000","last_became_peered":"2024-09-16T15:54:56.481478+0000","last_unstale":"2024-09-16T15:54:57.482638+0000","last_undegraded":"2024-09-16T15:54:57.482638+0000","last_fullsized":"2024-09-16T15:54:57.482638+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:05:20.399047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.445204+0000","last_change":"2024-09-16T15:54:55.445204+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.445204+0000","last_undegraded":"2024-09-16T15:54:55.445204+0000","last_fullsized":"2024-09-16T15:54:55.445204+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482766+0000","last_change":"2024-09-16T15:54:56.481838+0000","last_active":"2024-09-16T15:54:57.482766+0000","last_peered":"2024-09-16T15:54:57.482766+0000","last_clean":"2024-09-16T15:54:57.482766+0000","last_became_active":"2024-09-16T15:54:56.481562+0000","last_became_peered":"2024-09-16T15:54:56.481562+0000","last_unstale":"2024-09-16T15:54:57.482766+0000","last_undegraded":"2024-09-16T15:54:57.482766+0000","last_fullsized":"2024-09-16T15:54:57.482766+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:45:37.323453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482964+0000","last_change":"2024-09-16T15:54:56.490124+0000","last_active":"2024-09-16T15:54:57.482964+0000","last_peered":"2024-09-16T15:54:57.482964+0000","last_clean":"2024-09-16T15:54:57.482964+0000","last_became_active":"2024-09-16T15:54:56.489754+0000","last_became_peered":"2024-09-16T15:54:56.489754+0000","last_unstale":"2024-09-16T15:54:57.482964+0000","last_undegraded":"2024-09-16T15:54:57.482964+0000","last_fullsized":"2024-09-16T15:54:57.482964+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:27:59.188389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.444909+0000","last_change":"2024-09-16T15:54:55.444909+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.444909+0000","last_undegraded":"2024-09-16T15:54:55.444909+0000","last_fullsized":"2024-09-16T15:54:55.444909+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.445300+0000","last_change":"2024-09-16T15:54:55.445300+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.445300+0000","last_undegraded":"2024-09-16T15:54:55.445300+0000","last_fullsized":"2024-09-16T15:54:55.445300+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482388+0000","last_change":"2024-09-16T15:54:56.487219+0000","last_active":"2024-09-16T15:54:57.482388+0000","last_peered":"2024-09-16T15:54:57.482388+0000","last_clean":"2024-09-16T15:54:57.482388+0000","last_became_active":"2024-09-16T15:54:56.483904+0000","last_became_peered":"2024-09-16T15:54:56.483904+0000","last_unstale":"2024-09-16T15:54:57.482388+0000","last_undegraded":"2024-09-16T15:54:57.482388+0000","last_fullsized":"2024-09-16T15:54:57.482388+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:29:17.110559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482516+0000","last_change":"2024-09-16T15:54:56.477414+0000","last_active":"2024-09-16T15:54:57.482516+0000","last_peered":"2024-09-16T15:54:57.482516+0000","last_clean":"2024-09-16T15:54:57.482516+0000","last_became_active":"2024-09-16T15:54:56.476813+0000","last_became_peered":"2024-09-16T15:54:56.476813+0000","last_unstale":"2024-09-16T15:54:57.482516+0000","last_undegraded":"2024-09-16T15:54:57.482516+0000","last_fullsized":"2024-09-16T15:54:57.482516+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:07:02.927763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-16T15:54:56.481294+0000","last_change":"2024-09-16T15:54:56.478959+0000","last_active":"2024-09-16T15:54:56.481294+0000","last_peered":"2024-09-16T15:54:56.481294+0000","last_clean":"2024-09-16T15:54:56.481294+0000","last_became_active":"2024-09-16T15:54:56.478054+0000","last_became_peered":"2024-09-16T15:54:56.478054+0000","last_unstale":"2024-09-16T15:54:56.481294+0000","last_undegraded":"2024-09-16T15:54:56.481294+0000","last_fullsized":"2024-09-16T15:54:56.481294+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:11:14.095245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.459521+0000","last_change":"2024-09-16T15:54:55.458310+0000","last_active":"2024-09-16T15:54:55.459521+0000","last_peered":"2024-09-16T15:54:55.459521+0000","last_clean":"2024-09-16T15:54:55.459521+0000","last_became_active":"2024-09-16T15:54:55.457467+0000","last_became_peered":"2024-09-16T15:54:55.457467+0000","last_unstale":"2024-09-16T15:54:55.459521+0000","last_undegraded":"2024-09-16T15:54:55.459521+0000","last_fullsized":"2024-09-16T15:54:55.459521+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:52:32.558241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.447849+0000","last_change":"2024-09-16T15:54:55.447849+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.447849+0000","last_undegraded":"2024-09-16T15:54:55.447849+0000","last_fullsized":"2024-09-16T15:54:55.447849+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.459132+0000","last_change":"2024-09-16T15:54:55.458458+0000","last_active":"2024-09-16T15:54:55.459132+0000","last_peered":"2024-09-16T15:54:55.459132+0000","last_clean":"2024-09-16T15:54:55.459132+0000","last_became_active":"2024-09-16T15:54:55.457678+0000","last_became_peered":"2024-09-16T15:54:55.457678+0000","last_unstale":"2024-09-16T15:54:55.459132+0000","last_undegraded":"2024-09-16T15:54:55.459132+0000","last_fullsized":"2024-09-16T15:54:55.459132+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:20:58.630650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.450883+0000","last_change":"2024-09-16T15:54:55.450883+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.450883+0000","last_undegraded":"2024-09-16T15:54:55.450883+0000","last_fullsized":"2024-09-16T15:54:55.450883+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.465013+0000","last_change":"2024-09-16T15:54:55.464149+0000","last_active":"2024-09-16T15:54:55.465013+0000","last_peered":"2024-09-16T15:54:55.465013+0000","last_clean":"2024-09-16T15:54:55.465013+0000","last_became_active":"2024-09-16T15:54:55.463890+0000","last_became_peered":"2024-09-16T15:54:55.463890+0000","last_unstale":"2024-09-16T15:54:55.465013+0000","last_undegraded":"2024-09-16T15:54:55.465013+0000","last_fullsized":"2024-09-16T15:54:55.465013+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:03:49.222319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.449829+0000","last_change":"2024-09-16T15:54:55.449829+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.449829+0000","last_undegraded":"2024-09-16T15:54:55.449829+0000","last_fullsized":"2024-09-16T15:54:55.449829+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.460351+0000","last_change":"2024-09-16T15:54:55.458344+0000","last_active":"2024-09-16T15:54:55.460351+0000","last_peered":"2024-09-16T15:54:55.460351+0000","last_clean":"2024-09-16T15:54:55.460351+0000","last_became_active":"2024-09-16T15:54:55.455322+0000","last_became_peered":"2024-09-16T15:54:55.455322+0000","last_unstale":"2024-09-16T15:54:55.460351+0000","last_undegraded":"2024-09-16T15:54:55.460351+0000","last_fullsized":"2024-09-16T15:54:55.460351+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:14:46.177213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482428+0000","last_change":"2024-09-16T15:54:56.489708+0000","last_active":"2024-09-16T15:54:57.482428+0000","last_peered":"2024-09-16T15:54:57.482428+0000","last_clean":"2024-09-16T15:54:57.482428+0000","last_became_active":"2024-09-16T15:54:56.489545+0000","last_became_peered":"2024-09-16T15:54:56.489545+0000","last_unstale":"2024-09-16T15:54:57.482428+0000","last_undegraded":"2024-09-16T15:54:57.482428+0000","last_fullsized":"2024-09-16T15:54:57.482428+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:59:43.257767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.456546+0000","last_change":"2024-09-16T15:54:55.449973+0000","last_active":"2024-09-16T15:54:55.456546+0000","last_peered":"2024-09-16T15:54:55.456546+0000","last_clean":"2024-09-16T15:54:55.456546+0000","last_became_active":"2024-09-16T15:54:55.449676+0000","last_became_peered":"2024-09-16T15:54:55.449676+0000","last_unstale":"2024-09-16T15:54:55.456546+0000","last_undegraded":"2024-09-16T15:54:55.456546+0000","last_fullsized":"2024-09-16T15:54:55.456546+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:01:09.437888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.450288+0000","last_change":"2024-09-16T15:54:55.450288+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.450288+0000","last_undegraded":"2024-09-16T15:54:55.450288+0000","last_fullsized":"2024-09-16T15:54:55.450288+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1a","version":"144'1","reported_seq":14,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-16T15:54:56.486191+0000","last_change":"2024-09-16T15:54:55.455815+0000","last_active":"2024-09-16T15:54:56.486191+0000","last_peered":"2024-09-16T15:54:56.486191+0000","last_clean":"2024-09-16T15:54:56.486191+0000","last_became_active":"2024-09-16T15:54:55.455533+0000","last_became_peered":"2024-09-16T15:54:55.455533+0000","last_unstale":"2024-09-16T15:54:56.486191+0000","last_undegraded":"2024-09-16T15:54:56.486191+0000","last_fullsized":"2024-09-16T15:54:56.486191+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:01:12.975683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482536+0000","last_change":"2024-09-16T15:54:56.477672+0000","last_active":"2024-09-16T15:54:57.482536+0000","last_peered":"2024-09-16T15:54:57.482536+0000","last_clean":"2024-09-16T15:54:57.482536+0000","last_became_active":"2024-09-16T15:54:56.477153+0000","last_became_peered":"2024-09-16T15:54:56.477153+0000","last_unstale":"2024-09-16T15:54:57.482536+0000","last_undegraded":"2024-09-16T15:54:57.482536+0000","last_fullsized":"2024-09-16T15:54:57.482536+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:41:22.581512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.457395+0000","last_change":"2024-09-16T15:54:55.455189+0000","last_active":"2024-09-16T15:54:55.457395+0000","last_peered":"2024-09-16T15:54:55.457395+0000","last_clean":"2024-09-16T15:54:55.457395+0000","last_became_active":"2024-09-16T15:54:55.454896+0000","last_became_peered":"2024-09-16T15:54:55.454896+0000","last_unstale":"2024-09-16T15:54:55.457395+0000","last_undegraded":"2024-09-16T15:54:55.457395+0000","last_fullsized":"2024-09-16T15:54:55.457395+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:20:34.830239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.448700+0000","last_change":"2024-09-16T15:54:55.448700+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.448700+0000","last_undegraded":"2024-09-16T15:54:55.448700+0000","last_fullsized":"2024-09-16T15:54:55.448700+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.461802+0000","last_change":"2024-09-16T15:54:55.460804+0000","last_active":"2024-09-16T15:54:55.461802+0000","last_peered":"2024-09-16T15:54:55.461802+0000","last_clean":"2024-09-16T15:54:55.461802+0000","last_became_active":"2024-09-16T15:54:55.460502+0000","last_became_peered":"2024-09-16T15:54:55.460502+0000","last_unstale":"2024-09-16T15:54:55.461802+0000","last_undegraded":"2024-09-16T15:54:55.461802+0000","last_fullsized":"2024-09-16T15:54:55.461802+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:02:30.256738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.454247+0000","last_change":"2024-09-16T15:54:55.454247+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.454247+0000","last_undegraded":"2024-09-16T15:54:55.454247+0000","last_fullsized":"2024-09-16T15:54:55.454247+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.463235+0000","last_change":"2024-09-16T15:54:55.460989+0000","last_active":"2024-09-16T15:54:55.463235+0000","last_peered":"2024-09-16T15:54:55.463235+0000","last_clean":"2024-09-16T15:54:55.463235+0000","last_became_active":"2024-09-16T15:54:55.459752+0000","last_became_peered":"2024-09-16T15:54:55.459752+0000","last_unstale":"2024-09-16T15:54:55.463235+0000","last_undegraded":"2024-09-16T15:54:55.463235+0000","last_fullsized":"2024-09-16T15:54:55.463235+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:14:12.423767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.443768+0000","last_change":"2024-09-16T15:54:55.443768+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.443768+0000","last_undegraded":"2024-09-16T15:54:55.443768+0000","last_fullsized":"2024-09-16T15:54:55.443768+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-16T15:54:56.459082+0000","last_change":"2024-09-16T15:54:55.462309+0000","last_active":"2024-09-16T15:54:56.459082+0000","last_peered":"2024-09-16T15:54:56.459082+0000","last_clean":"2024-09-16T15:54:56.459082+0000","last_became_active":"2024-09-16T15:54:55.462039+0000","last_became_peered":"2024-09-16T15:54:55.462039+0000","last_unstale":"2024-09-16T15:54:56.459082+0000","last_undegraded":"2024-09-16T15:54:56.459082+0000","last_fullsized":"2024-09-16T15:54:56.459082+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:32:00.564497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.452496+0000","last_change":"2024-09-16T15:54:55.452496+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.452496+0000","last_undegraded":"2024-09-16T15:54:55.452496+0000","last_fullsized":"2024-09-16T15:54:55.452496+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.465057+0000","last_change":"2024-09-16T15:54:55.461505+0000","last_active":"2024-09-16T15:54:55.465057+0000","last_peered":"2024-09-16T15:54:55.465057+0000","last_clean":"2024-09-16T15:54:55.465057+0000","last_became_active":"2024-09-16T15:54:55.461174+0000","last_became_peered":"2024-09-16T15:54:55.461174+0000","last_unstale":"2024-09-16T15:54:55.465057+0000","last_undegraded":"2024-09-16T15:54:55.465057+0000","last_fullsized":"2024-09-16T15:54:55.465057+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:58:07.658068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.450155+0000","last_change":"2024-09-16T15:54:55.450155+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.450155+0000","last_undegraded":"2024-09-16T15:54:55.450155+0000","last_fullsized":"2024-09-16T15:54:55.450155+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.464678+0000","last_change":"2024-09-16T15:54:55.462728+0000","last_active":"2024-09-16T15:54:55.464678+0000","last_peered":"2024-09-16T15:54:55.464678+0000","last_clean":"2024-09-16T15:54:55.464678+0000","last_became_active":"2024-09-16T15:54:55.462424+0000","last_became_peered":"2024-09-16T15:54:55.462424+0000","last_unstale":"2024-09-16T15:54:55.464678+0000","last_undegraded":"2024-09-16T15:54:55.464678+0000","last_fullsized":"2024-09-16T15:54:55.464678+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:30:10.712547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.445578+0000","last_change":"2024-09-16T15:54:55.445578+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.445578+0000","last_undegraded":"2024-09-16T15:54:55.445578+0000","last_fullsized":"2024-09-16T15:54:55.445578+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.458040+0000","last_change":"2024-09-16T15:54:55.455831+0000","last_active":"2024-09-16T15:54:55.458040+0000","last_peered":"2024-09-16T15:54:55.458040+0000","last_clean":"2024-09-16T15:54:55.458040+0000","last_became_active":"2024-09-16T15:54:55.455574+0000","last_became_peered":"2024-09-16T15:54:55.455574+0000","last_unstale":"2024-09-16T15:54:55.458040+0000","last_undegraded":"2024-09-16T15:54:55.458040+0000","last_fullsized":"2024-09-16T15:54:55.458040+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:43:13.336878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.453165+0000","last_change":"2024-09-16T15:54:55.453165+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.453165+0000","last_undegraded":"2024-09-16T15:54:55.453165+0000","last_fullsized":"2024-09-16T15:54:55.453165+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482490+0000","last_change":"2024-09-16T15:54:55.457472+0000","last_active":"2024-09-16T15:54:57.482490+0000","last_peered":"2024-09-16T15:54:57.482490+0000","last_clean":"2024-09-16T15:54:57.482490+0000","last_became_active":"2024-09-16T15:54:55.456471+0000","last_became_peered":"2024-09-16T15:54:55.456471+0000","last_unstale":"2024-09-16T15:54:57.482490+0000","last_undegraded":"2024-09-16T15:54:57.482490+0000","last_fullsized":"2024-09-16T15:54:57.482490+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:14:12.568813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.448288+0000","last_change":"2024-09-16T15:54:55.448288+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.448288+0000","last_undegraded":"2024-09-16T15:54:55.448288+0000","last_fullsized":"2024-09-16T15:54:55.448288+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.462749+0000","last_change":"2024-09-16T15:54:55.461170+0000","last_active":"2024-09-16T15:54:55.462749+0000","last_peered":"2024-09-16T15:54:55.462749+0000","last_clean":"2024-09-16T15:54:55.462749+0000","last_became_active":"2024-09-16T15:54:55.460915+0000","last_became_peered":"2024-09-16T15:54:55.460915+0000","last_unstale":"2024-09-16T15:54:55.462749+0000","last_undegraded":"2024-09-16T15:54:55.462749+0000","last_fullsized":"2024-09-16T15:54:55.462749+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:25:12.508089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-16T15:54:56.467209+0000","last_change":"2024-09-16T15:54:56.464019+0000","last_active":"2024-09-16T15:54:56.467209+0000","last_peered":"2024-09-16T15:54:56.467209+0000","last_clean":"2024-09-16T15:54:56.467209+0000","last_became_active":"2024-09-16T15:54:56.463741+0000","last_became_peered":"2024-09-16T15:54:56.463741+0000","last_unstale":"2024-09-16T15:54:56.467209+0000","last_undegraded":"2024-09-16T15:54:56.467209+0000","last_fullsized":"2024-09-16T15:54:56.467209+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:01:03.304745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.464322+0000","last_change":"2024-09-16T15:54:55.461113+0000","last_active":"2024-09-16T15:54:55.464322+0000","last_peered":"2024-09-16T15:54:55.464322+0000","last_clean":"2024-09-16T15:54:55.464322+0000","last_became_active":"2024-09-16T15:54:55.460866+0000","last_became_peered":"2024-09-16T15:54:55.460866+0000","last_unstale":"2024-09-16T15:54:55.464322+0000","last_undegraded":"2024-09-16T15:54:55.464322+0000","last_fullsized":"2024-09-16T15:54:55.464322+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:14:24.286251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482797+0000","last_change":"2024-09-16T15:54:56.491736+0000","last_active":"2024-09-16T15:54:57.482797+0000","last_peered":"2024-09-16T15:54:57.482797+0000","last_clean":"2024-09-16T15:54:57.482797+0000","last_became_active":"2024-09-16T15:54:56.491464+0000","last_became_peered":"2024-09-16T15:54:56.491464+0000","last_unstale":"2024-09-16T15:54:57.482797+0000","last_undegraded":"2024-09-16T15:54:57.482797+0000","last_fullsized":"2024-09-16T15:54:57.482797+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:35:45.264168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.460583+0000","last_change":"2024-09-16T15:54:55.458328+0000","last_active":"2024-09-16T15:54:55.460583+0000","last_peered":"2024-09-16T15:54:55.460583+0000","last_clean":"2024-09-16T15:54:55.460583+0000","last_became_active":"2024-09-16T15:54:55.457453+0000","last_became_peered":"2024-09-16T15:54:55.457453+0000","last_unstale":"2024-09-16T15:54:55.460583+0000","last_undegraded":"2024-09-16T15:54:55.460583+0000","last_fullsized":"2024-09-16T15:54:55.460583+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:16:21.173713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-16T15:54:56.467102+0000","last_change":"2024-09-16T15:54:56.464797+0000","last_active":"2024-09-16T15:54:56.467102+0000","last_peered":"2024-09-16T15:54:56.467102+0000","last_clean":"2024-09-16T15:54:56.467102+0000","last_became_active":"2024-09-16T15:54:56.463873+0000","last_became_peered":"2024-09-16T15:54:56.463873+0000","last_unstale":"2024-09-16T15:54:56.467102+0000","last_undegraded":"2024-09-16T15:54:56.467102+0000","last_fullsized":"2024-09-16T15:54:56.467102+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:02:11.742600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.467261+0000","last_change":"2024-09-16T15:54:55.463365+0000","last_active":"2024-09-16T15:54:55.467261+0000","last_peered":"2024-09-16T15:54:55.467261+0000","last_clean":"2024-09-16T15:54:55.467261+0000","last_became_active":"2024-09-16T15:54:55.462246+0000","last_became_peered":"2024-09-16T15:54:55.462246+0000","last_unstale":"2024-09-16T15:54:55.467261+0000","last_undegraded":"2024-09-16T15:54:55.467261+0000","last_fullsized":"2024-09-16T15:54:55.467261+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:54:15.667120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.448139+0000","last_change":"2024-09-16T15:54:55.448139+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.448139+0000","last_undegraded":"2024-09-16T15:54:55.448139+0000","last_fullsized":"2024-09-16T15:54:55.448139+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482928+0000","last_change":"2024-09-16T15:54:55.463637+0000","last_active":"2024-09-16T15:54:57.482928+0000","last_peered":"2024-09-16T15:54:57.482928+0000","last_clean":"2024-09-16T15:54:57.482928+0000","last_became_active":"2024-09-16T15:54:55.463289+0000","last_became_peered":"2024-09-16T15:54:55.463289+0000","last_unstale":"2024-09-16T15:54:57.482928+0000","last_undegraded":"2024-09-16T15:54:57.482928+0000","last_fullsized":"2024-09-16T15:54:57.482928+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:16:31.781727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.444870+0000","last_change":"2024-09-16T15:54:55.444870+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.444870+0000","last_undegraded":"2024-09-16T15:54:55.444870+0000","last_fullsized":"2024-09-16T15:54:55.444870+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.463186+0000","last_change":"2024-09-16T15:54:55.459884+0000","last_active":"2024-09-16T15:54:55.463186+0000","last_peered":"2024-09-16T15:54:55.463186+0000","last_clean":"2024-09-16T15:54:55.463186+0000","last_became_active":"2024-09-16T15:54:55.459556+0000","last_became_peered":"2024-09-16T15:54:55.459556+0000","last_unstale":"2024-09-16T15:54:55.463186+0000","last_undegraded":"2024-09-16T15:54:55.463186+0000","last_fullsized":"2024-09-16T15:54:55.463186+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:38:34.406517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482617+0000","last_change":"2024-09-16T15:54:56.485052+0000","last_active":"2024-09-16T15:54:57.482617+0000","last_peered":"2024-09-16T15:54:57.482617+0000","last_clean":"2024-09-16T15:54:57.482617+0000","last_became_active":"2024-09-16T15:54:56.483119+0000","last_became_peered":"2024-09-16T15:54:56.483119+0000","last_unstale":"2024-09-16T15:54:57.482617+0000","last_undegraded":"2024-09-16T15:54:57.482617+0000","last_fullsized":"2024-09-16T15:54:57.482617+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:50:40.992458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.467038+0000","last_change":"2024-09-16T15:54:55.464323+0000","last_active":"2024-09-16T15:54:55.467038+0000","last_peered":"2024-09-16T15:54:55.467038+0000","last_clean":"2024-09-16T15:54:55.467038+0000","last_became_active":"2024-09-16T15:54:55.463480+0000","last_became_peered":"2024-09-16T15:54:55.463480+0000","last_unstale":"2024-09-16T15:54:55.467038+0000","last_undegraded":"2024-09-16T15:54:55.467038+0000","last_fullsized":"2024-09-16T15:54:55.467038+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:39:22.898239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.443764+0000","last_change":"2024-09-16T15:54:55.443764+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.443764+0000","last_undegraded":"2024-09-16T15:54:55.443764+0000","last_fullsized":"2024-09-16T15:54:55.443764+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.463551+0000","last_change":"2024-09-16T15:54:55.459040+0000","last_active":"2024-09-16T15:54:55.463551+0000","last_peered":"2024-09-16T15:54:55.463551+0000","last_clean":"2024-09-16T15:54:55.463551+0000","last_became_active":"2024-09-16T15:54:55.458758+0000","last_became_peered":"2024-09-16T15:54:55.458758+0000","last_unstale":"2024-09-16T15:54:55.463551+0000","last_undegraded":"2024-09-16T15:54:55.463551+0000","last_fullsized":"2024-09-16T15:54:55.463551+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:31:02.873968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482549+0000","last_change":"2024-09-16T15:54:56.485351+0000","last_active":"2024-09-16T15:54:57.482549+0000","last_peered":"2024-09-16T15:54:57.482549+0000","last_clean":"2024-09-16T15:54:57.482549+0000","last_became_active":"2024-09-16T15:54:56.484236+0000","last_became_peered":"2024-09-16T15:54:56.484236+0000","last_unstale":"2024-09-16T15:54:57.482549+0000","last_undegraded":"2024-09-16T15:54:57.482549+0000","last_fullsized":"2024-09-16T15:54:57.482549+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:48:52.418050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.467028+0000","last_change":"2024-09-16T15:54:55.463632+0000","last_active":"2024-09-16T15:54:55.467028+0000","last_peered":"2024-09-16T15:54:55.467028+0000","last_clean":"2024-09-16T15:54:55.467028+0000","last_became_active":"2024-09-16T15:54:55.463347+0000","last_became_peered":"2024-09-16T15:54:55.463347+0000","last_unstale":"2024-09-16T15:54:55.467028+0000","last_undegraded":"2024-09-16T15:54:55.467028+0000","last_fullsized":"2024-09-16T15:54:55.467028+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:47:46.317280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.451273+0000","last_change":"2024-09-16T15:54:55.451273+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.451273+0000","last_undegraded":"2024-09-16T15:54:55.451273+0000","last_fullsized":"2024-09-16T15:54:55.451273+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.464299+0000","last_change":"2024-09-16T15:54:55.459856+0000","last_active":"2024-09-16T15:54:55.464299+0000","last_peered":"2024-09-16T15:54:55.464299+0000","last_clean":"2024-09-16T15:54:55.464299+0000","last_became_active":"2024-09-16T15:54:55.458670+0000","last_became_peered":"2024-09-16T15:54:55.458670+0000","last_unstale":"2024-09-16T15:54:55.464299+0000","last_undegraded":"2024-09-16T15:54:55.464299+0000","last_fullsized":"2024-09-16T15:54:55.464299+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:49:34.676553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.453193+0000","last_change":"2024-09-16T15:54:55.453193+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.453193+0000","last_undegraded":"2024-09-16T15:54:55.453193+0000","last_fullsized":"2024-09-16T15:54:55.453193+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.6","version":"145'1","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482760+0000","last_change":"2024-09-16T15:54:55.464427+0000","last_active":"2024-09-16T15:54:57.482760+0000","last_peered":"2024-09-16T15:54:57.482760+0000","last_clean":"2024-09-16T15:54:57.482760+0000","last_became_active":"2024-09-16T15:54:55.464043+0000","last_became_peered":"2024-09-16T15:54:55.464043+0000","last_unstale":"2024-09-16T15:54:57.482760+0000","last_undegraded":"2024-09-16T15:54:57.482760+0000","last_fullsized":"2024-09-16T15:54:57.482760+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:06:18.925531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.444371+0000","last_change":"2024-09-16T15:54:55.444371+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.444371+0000","last_undegraded":"2024-09-16T15:54:55.444371+0000","last_fullsized":"2024-09-16T15:54:55.444371+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.463400+0000","last_change":"2024-09-16T15:54:55.460656+0000","last_active":"2024-09-16T15:54:55.463400+0000","last_peered":"2024-09-16T15:54:55.463400+0000","last_clean":"2024-09-16T15:54:55.463400+0000","last_became_active":"2024-09-16T15:54:55.460149+0000","last_became_peered":"2024-09-16T15:54:55.460149+0000","last_unstale":"2024-09-16T15:54:55.463400+0000","last_undegraded":"2024-09-16T15:54:55.463400+0000","last_fullsized":"2024-09-16T15:54:55.463400+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:17:46.439566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.445555+0000","last_change":"2024-09-16T15:54:55.445555+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.445555+0000","last_undegraded":"2024-09-16T15:54:55.445555+0000","last_fullsized":"2024-09-16T15:54:55.445555+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.465100+0000","last_change":"2024-09-16T15:54:55.461333+0000","last_active":"2024-09-16T15:54:55.465100+0000","last_peered":"2024-09-16T15:54:55.465100+0000","last_clean":"2024-09-16T15:54:55.465100+0000","last_became_active":"2024-09-16T15:54:55.460016+0000","last_became_peered":"2024-09-16T15:54:55.460016+0000","last_unstale":"2024-09-16T15:54:55.465100+0000","last_undegraded":"2024-09-16T15:54:55.465100+0000","last_fullsized":"2024-09-16T15:54:55.465100+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:07:03.672777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.447708+0000","last_change":"2024-09-16T15:54:55.447708+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.447708+0000","last_undegraded":"2024-09-16T15:54:55.447708+0000","last_fullsized":"2024-09-16T15:54:55.447708+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482633+0000","last_change":"2024-09-16T15:54:55.455582+0000","last_active":"2024-09-16T15:54:57.482633+0000","last_peered":"2024-09-16T15:54:57.482633+0000","last_clean":"2024-09-16T15:54:57.482633+0000","last_became_active":"2024-09-16T15:54:55.453745+0000","last_became_peered":"2024-09-16T15:54:55.453745+0000","last_unstale":"2024-09-16T15:54:57.482633+0000","last_undegraded":"2024-09-16T15:54:57.482633+0000","last_fullsized":"2024-09-16T15:54:57.482633+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:42:00.578523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.448094+0000","last_change":"2024-09-16T15:54:55.448094+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.448094+0000","last_undegraded":"2024-09-16T15:54:55.448094+0000","last_fullsized":"2024-09-16T15:54:55.448094+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.455725+0000","last_change":"2024-09-16T15:54:55.449566+0000","last_active":"2024-09-16T15:54:55.455725+0000","last_peered":"2024-09-16T15:54:55.455725+0000","last_clean":"2024-09-16T15:54:55.455725+0000","last_became_active":"2024-09-16T15:54:55.449318+0000","last_became_peered":"2024-09-16T15:54:55.449318+0000","last_unstale":"2024-09-16T15:54:55.455725+0000","last_undegraded":"2024-09-16T15:54:55.455725+0000","last_fullsized":"2024-09-16T15:54:55.455725+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:38:14.471185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.451007+0000","last_change":"2024-09-16T15:54:55.451007+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.451007+0000","last_undegraded":"2024-09-16T15:54:55.451007+0000","last_fullsized":"2024-09-16T15:54:55.451007+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.462530+0000","last_change":"2024-09-16T15:54:55.461228+0000","last_active":"2024-09-16T15:54:55.462530+0000","last_peered":"2024-09-16T15:54:55.462530+0000","last_clean":"2024-09-16T15:54:55.462530+0000","last_became_active":"2024-09-16T15:54:55.460939+0000","last_became_peered":"2024-09-16T15:54:55.460939+0000","last_unstale":"2024-09-16T15:54:55.462530+0000","last_undegraded":"2024-09-16T15:54:55.462530+0000","last_fullsized":"2024-09-16T15:54:55.462530+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:20:34.830239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.442822+0000","last_change":"2024-09-16T15:54:55.442822+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.442822+0000","last_undegraded":"2024-09-16T15:54:55.442822+0000","last_fullsized":"2024-09-16T15:54:55.442822+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.463143+0000","last_change":"2024-09-16T15:54:55.459516+0000","last_active":"2024-09-16T15:54:55.463143+0000","last_peered":"2024-09-16T15:54:55.463143+0000","last_clean":"2024-09-16T15:54:55.463143+0000","last_became_active":"2024-09-16T15:54:55.459041+0000","last_became_peered":"2024-09-16T15:54:55.459041+0000","last_unstale":"2024-09-16T15:54:55.463143+0000","last_undegraded":"2024-09-16T15:54:55.463143+0000","last_fullsized":"2024-09-16T15:54:55.463143+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:11:04.495951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.449438+0000","last_change":"2024-09-16T15:54:55.449438+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.449438+0000","last_undegraded":"2024-09-16T15:54:55.449438+0000","last_fullsized":"2024-09-16T15:54:55.449438+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.463059+0000","last_change":"2024-09-16T15:54:55.461430+0000","last_active":"2024-09-16T15:54:55.463059+0000","last_peered":"2024-09-16T15:54:55.463059+0000","last_clean":"2024-09-16T15:54:55.463059+0000","last_became_active":"2024-09-16T15:54:55.461166+0000","last_became_peered":"2024-09-16T15:54:55.461166+0000","last_unstale":"2024-09-16T15:54:55.463059+0000","last_undegraded":"2024-09-16T15:54:55.463059+0000","last_fullsized":"2024-09-16T15:54:55.463059+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:20:58.630650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":343,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-16T15:54:56.457213+0000","last_change":"2024-09-16T15:39:12.698813+0000","last_active":"2024-09-16T15:54:56.457213+0000","last_peered":"2024-09-16T15:54:56.457213+0000","last_clean":"2024-09-16T15:54:56.457213+0000","last_became_active":"2024-09-16T15:39:12.697695+0000","last_became_peered":"2024-09-16T15:39:12.697695+0000","last_unstale":"2024-09-16T15:54:56.457213+0000","last_undegraded":"2024-09-16T15:54:56.457213+0000","last_fullsized":"2024-09-16T15:54:56.457213+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_clean_scrub_stamp":"2024-09-16T15:39:11.684967+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:02.061081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509634,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":163348,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":161598,"kb_avail":94208492,"statfs":{"total":96636764160,"available":96469495808,"internally_reserved":0,"allocated":1769472,"data_stored":781042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":165476808},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509633,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123672,"kb_used_data":2308,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248168,"statfs":{"total":96636764160,"available":96510124032,"internally_reserved":0,"allocated":2363392,"data_stored":1371410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":124254663},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509632,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":136996,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94234844,"statfs":{"total":96636764160,"available":96496480256,"internally_reserved":0,"allocated":1781760,"data_stored":787457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":138475973},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509632,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":137000,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94234840,"statfs":{"total":96636764160,"available":96496476160,"internally_reserved":0,"allocated":1781760,"data_stored":787457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":138475977},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509632,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":118952,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94252888,"statfs":{"total":96636764160,"available":96514957312,"internally_reserved":0,"allocated":1781760,"data_stored":787457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":119994823},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509633,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":132004,"kb_used_data":2320,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94239836,"statfs":{"total":96636764160,"available":96501592064,"internally_reserved":0,"allocated":2375680,"data_stored":1377825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509631,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149476,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222364,"statfs":{"total":96636764160,"available":96483700736,"internally_reserved":0,"allocated":1781760,"data_stored":787457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":151255495},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542339,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131452,"kb_used_data":1764,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240388,"statfs":{"total":96636764160,"available":96502157312,"internally_reserved":0,"allocated":1806336,"data_stored":800812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":132774348},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:55:00.669 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:55:00.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.862+0000 7fc0fde00640 1 -- 172.21.15.151:0/3549581695 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc0dc004000 msgr2=0x7fc0dc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:00.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.862+0000 7fc0fde00640 1 --2- 172.21.15.151:0/3549581695 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc0dc004000 0x7fc0dc004800 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7fc0e40016c0 tx=0x7fc0dc005830 comp rx=0 tx=0).stop 2024-09-16T15:55:00.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.862+0000 7fc0f7400640 1 -- 172.21.15.151:0/3549581695 reap_dead start 2024-09-16T15:55:00.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.862+0000 7fc0fde00640 1 -- 172.21.15.151:0/3549581695 shutdown_connections 2024-09-16T15:55:00.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.862+0000 7fc0fde00640 1 -- 172.21.15.151:0/3549581695 >> 172.21.15.151:0/3549581695 conn(0x7fc0f80b1950 msgr2=0x7fc0f80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:55:00.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.862+0000 7fc0fde00640 1 -- 172.21.15.151:0/3549581695 shutdown_connections 2024-09-16T15:55:00.864 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.862+0000 7fc0fde00640 1 -- 172.21.15.151:0/3549581695 wait complete. 2024-09-16T15:55:00.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.864+0000 7fc0fde00640 1 Processor -- start 2024-09-16T15:55:00.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.864+0000 7fc0fde00640 1 -- start start 2024-09-16T15:55:00.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.865+0000 7fc0fde00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0dc004000 0x7fc0f8143530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:00.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.865+0000 7fc0fde00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc0f8152880 0x7fc0f81f5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:00.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.865+0000 7fc0fde00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0f8150a30 0x7fc0f80cec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:00.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.865+0000 7fc0fde00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f8166f80 con 0x7fc0f8152880 2024-09-16T15:55:00.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.865+0000 7fc0fde00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f8166e00 con 0x7fc0f8150a30 2024-09-16T15:55:00.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.865+0000 7fc0fde00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fc0f8167100 con 0x7fc0dc004000 2024-09-16T15:55:00.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.865+0000 7fc0f6000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc0f8152880 0x7fc0f81f5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:00.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.865+0000 7fc0f6000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc0f8152880 0x7fc0f81f5220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52606/0 (socket says 172.21.15.151:52606) 2024-09-16T15:55:00.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.865+0000 7fc0f6000640 1 -- 172.21.15.151:0/3762421292 learned_addr learned my addr 172.21.15.151:0/3762421292 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:55:00.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.866+0000 7fc0f6a00640 1 --2- 172.21.15.151:0/3762421292 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0dc004000 0x7fc0f8143530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:00.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.866+0000 7fc0f7400640 1 --2- 172.21.15.151:0/3762421292 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0f8150a30 0x7fc0f80cec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:00.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.866+0000 7fc0f6000640 1 -- 172.21.15.151:0/3762421292 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0dc004000 msgr2=0x7fc0f8143530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:00.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.866+0000 7fc0f6000640 1 --2- 172.21.15.151:0/3762421292 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0dc004000 0x7fc0f8143530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:00.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.866+0000 7fc0f6000640 1 -- 172.21.15.151:0/3762421292 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0f8150a30 msgr2=0x7fc0f80cec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:00.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.866+0000 7fc0f6000640 1 --2- 172.21.15.151:0/3762421292 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0f8150a30 0x7fc0f80cec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:00.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.866+0000 7fc0f6000640 1 -- 172.21.15.151:0/3762421292 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0f80af9a0 con 0x7fc0f8152880 2024-09-16T15:55:00.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.866+0000 7fc0f6000640 1 --2- 172.21.15.151:0/3762421292 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc0f8152880 0x7fc0f81f5220 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7fc0e40017b0 tx=0x7fc0e4006360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:55:00.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.866+0000 7fc0ef400640 1 -- 172.21.15.151:0/3762421292 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e4038070 con 0x7fc0f8152880 2024-09-16T15:55:00.868 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.866+0000 7fc0ef400640 1 -- 172.21.15.151:0/3762421292 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc0e4003af0 con 0x7fc0f8152880 2024-09-16T15:55:00.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.866+0000 7fc0fde00640 1 -- 172.21.15.151:0/3762421292 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0f80afc80 con 0x7fc0f8152880 2024-09-16T15:55:00.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.866+0000 7fc0ef400640 1 -- 172.21.15.151:0/3762421292 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e4033420 con 0x7fc0f8152880 2024-09-16T15:55:00.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.866+0000 7fc0fde00640 1 -- 172.21.15.151:0/3762421292 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0f81504e0 con 0x7fc0f8152880 2024-09-16T15:55:00.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.868+0000 7fc0fde00640 1 -- 172.21.15.151:0/3762421292 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0f806ef50 con 0x7fc0f8152880 2024-09-16T15:55:00.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.869+0000 7fc0ef400640 1 -- 172.21.15.151:0/3762421292 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 50) ==== 104398+0+0 (secure 0 0 0) 0x7fc0e4043020 con 0x7fc0f8152880 2024-09-16T15:55:00.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.870+0000 7fc0ef400640 1 --2- 172.21.15.151:0/3762421292 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc0d807aaa0 0x7fc0d807cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:00.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.870+0000 7fc0ef400640 1 -- 172.21.15.151:0/3762421292 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5849+0+0 (secure 0 0 0) 0x7fc0e40b9160 con 0x7fc0f8152880 2024-09-16T15:55:00.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.871+0000 7fc0f6a00640 1 --2- 172.21.15.151:0/3762421292 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc0d807aaa0 0x7fc0d807cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:00.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.871+0000 7fc0f6a00640 1 --2- 172.21.15.151:0/3762421292 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc0d807aaa0 0x7fc0d807cf60 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8000fb0 tx=0x7fc0e80067a0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:55:00.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:00.874+0000 7fc0ef400640 1 -- 172.21.15.151:0/3762421292 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc0e4083230 con 0x7fc0f8152880 2024-09-16T15:55:01.054 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.053+0000 7fc0fde00640 1 -- 172.21.15.151:0/3762421292 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc0f815ef10 con 0x7fc0d807aaa0 2024-09-16T15:55:01.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.058+0000 7fc0ef400640 1 -- 172.21.15.151:0/3762421292 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+248706 (secure 0 0 0) 0x7fc0f815ef10 con 0x7fc0d807aaa0 2024-09-16T15:55:01.062 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:55:01.063 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-09-16T15:55:01.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.062+0000 7fc0fde00640 1 -- 172.21.15.151:0/3762421292 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc0d807aaa0 msgr2=0x7fc0d807cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:01.064 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.062+0000 7fc0fde00640 1 --2- 172.21.15.151:0/3762421292 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc0d807aaa0 0x7fc0d807cf60 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8000fb0 tx=0x7fc0e80067a0 comp rx=0 tx=0).stop 2024-09-16T15:55:01.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.062+0000 7fc0fde00640 1 -- 172.21.15.151:0/3762421292 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc0f8152880 msgr2=0x7fc0f81f5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:01.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.062+0000 7fc0fde00640 1 --2- 172.21.15.151:0/3762421292 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc0f8152880 0x7fc0f81f5220 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7fc0e40017b0 tx=0x7fc0e4006360 comp rx=0 tx=0).stop 2024-09-16T15:55:01.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.062+0000 7fc0fde00640 1 -- 172.21.15.151:0/3762421292 shutdown_connections 2024-09-16T15:55:01.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.062+0000 7fc0fde00640 1 --2- 172.21.15.151:0/3762421292 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc0d807aaa0 0x7fc0d807cf60 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:01.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.062+0000 7fc0fde00640 1 --2- 172.21.15.151:0/3762421292 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0f8150a30 0x7fc0f80cec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:01.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.062+0000 7fc0fde00640 1 --2- 172.21.15.151:0/3762421292 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc0f8152880 0x7fc0f81f5220 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:01.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.062+0000 7fc0fde00640 1 --2- 172.21.15.151:0/3762421292 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc0dc004000 0x7fc0f8143530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:01.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.062+0000 7fc0fde00640 1 -- 172.21.15.151:0/3762421292 >> 172.21.15.151:0/3762421292 conn(0x7fc0f80b1950 msgr2=0x7fc0f806f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:55:01.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.062+0000 7fc0fde00640 1 -- 172.21.15.151:0/3762421292 shutdown_connections 2024-09-16T15:55:01.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.062+0000 7fc0fde00640 1 -- 172.21.15.151:0/3762421292 wait complete. 2024-09-16T15:55:01.080 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":613,"stamp":"2024-09-16T15:54:59.958152+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":102,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1092900,"kb_used_data":15080,"kb_used_omap":12,"kb_used_meta":1077619,"kb_avail":753881820,"statfs":{"total":773094113280,"available":771974983680,"internally_reserved":0,"allocated":15441920,"data_stored":7480917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12738,"internal_metadata":1103482430},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.523823"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.445126+0000","last_change":"2024-09-16T15:54:55.445126+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.445126+0000","last_undegraded":"2024-09-16T15:54:55.445126+0000","last_fullsized":"2024-09-16T15:54:55.445126+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482752+0000","last_change":"2024-09-16T15:54:56.489100+0000","last_active":"2024-09-16T15:54:57.482752+0000","last_peered":"2024-09-16T15:54:57.482752+0000","last_clean":"2024-09-16T15:54:57.482752+0000","last_became_active":"2024-09-16T15:54:56.488821+0000","last_became_peered":"2024-09-16T15:54:56.488821+0000","last_unstale":"2024-09-16T15:54:57.482752+0000","last_undegraded":"2024-09-16T15:54:57.482752+0000","last_fullsized":"2024-09-16T15:54:57.482752+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:43:37.723000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.446549+0000","last_change":"2024-09-16T15:54:55.446549+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.446549+0000","last_undegraded":"2024-09-16T15:54:55.446549+0000","last_fullsized":"2024-09-16T15:54:55.446549+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.449901+0000","last_change":"2024-09-16T15:54:55.449901+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.449901+0000","last_undegraded":"2024-09-16T15:54:55.449901+0000","last_fullsized":"2024-09-16T15:54:55.449901+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-16T15:54:56.479967+0000","last_change":"2024-09-16T15:54:56.471793+0000","last_active":"2024-09-16T15:54:56.479967+0000","last_peered":"2024-09-16T15:54:56.479967+0000","last_clean":"2024-09-16T15:54:56.479967+0000","last_became_active":"2024-09-16T15:54:56.471048+0000","last_became_peered":"2024-09-16T15:54:56.471048+0000","last_unstale":"2024-09-16T15:54:56.479967+0000","last_undegraded":"2024-09-16T15:54:56.479967+0000","last_fullsized":"2024-09-16T15:54:56.479967+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:15:17.040666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.449521+0000","last_change":"2024-09-16T15:54:55.449521+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.449521+0000","last_undegraded":"2024-09-16T15:54:55.449521+0000","last_fullsized":"2024-09-16T15:54:55.449521+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.449819+0000","last_change":"2024-09-16T15:54:55.449819+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.449819+0000","last_undegraded":"2024-09-16T15:54:55.449819+0000","last_fullsized":"2024-09-16T15:54:55.449819+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.443482+0000","last_change":"2024-09-16T15:54:55.443482+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.443482+0000","last_undegraded":"2024-09-16T15:54:55.443482+0000","last_fullsized":"2024-09-16T15:54:55.443482+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.450377+0000","last_change":"2024-09-16T15:54:55.450377+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.450377+0000","last_undegraded":"2024-09-16T15:54:55.450377+0000","last_fullsized":"2024-09-16T15:54:55.450377+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.445356+0000","last_change":"2024-09-16T15:54:55.445356+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.445356+0000","last_undegraded":"2024-09-16T15:54:55.445356+0000","last_fullsized":"2024-09-16T15:54:55.445356+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.448658+0000","last_change":"2024-09-16T15:54:55.448658+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.448658+0000","last_undegraded":"2024-09-16T15:54:55.448658+0000","last_fullsized":"2024-09-16T15:54:55.448658+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.445187+0000","last_change":"2024-09-16T15:54:55.445187+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.445187+0000","last_undegraded":"2024-09-16T15:54:55.445187+0000","last_fullsized":"2024-09-16T15:54:55.445187+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-16T15:54:56.479929+0000","last_change":"2024-09-16T15:54:56.469650+0000","last_active":"2024-09-16T15:54:56.479929+0000","last_peered":"2024-09-16T15:54:56.479929+0000","last_clean":"2024-09-16T15:54:56.479929+0000","last_became_active":"2024-09-16T15:54:56.469407+0000","last_became_peered":"2024-09-16T15:54:56.469407+0000","last_unstale":"2024-09-16T15:54:56.479929+0000","last_undegraded":"2024-09-16T15:54:56.479929+0000","last_fullsized":"2024-09-16T15:54:56.479929+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:58:19.883027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.447273+0000","last_change":"2024-09-16T15:54:55.447273+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.447273+0000","last_undegraded":"2024-09-16T15:54:55.447273+0000","last_fullsized":"2024-09-16T15:54:55.447273+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-16T15:54:56.479726+0000","last_change":"2024-09-16T15:54:56.469340+0000","last_active":"2024-09-16T15:54:56.479726+0000","last_peered":"2024-09-16T15:54:56.479726+0000","last_clean":"2024-09-16T15:54:56.479726+0000","last_became_active":"2024-09-16T15:54:56.469088+0000","last_became_peered":"2024-09-16T15:54:56.469088+0000","last_unstale":"2024-09-16T15:54:56.479726+0000","last_undegraded":"2024-09-16T15:54:56.479726+0000","last_fullsized":"2024-09-16T15:54:56.479726+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:28:09.504034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.447718+0000","last_change":"2024-09-16T15:54:55.447718+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.447718+0000","last_undegraded":"2024-09-16T15:54:55.447718+0000","last_fullsized":"2024-09-16T15:54:55.447718+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.443888+0000","last_change":"2024-09-16T15:54:55.443888+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.443888+0000","last_undegraded":"2024-09-16T15:54:55.443888+0000","last_fullsized":"2024-09-16T15:54:55.443888+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-16T15:54:56.481270+0000","last_change":"2024-09-16T15:54:56.476168+0000","last_active":"2024-09-16T15:54:56.481270+0000","last_peered":"2024-09-16T15:54:56.481270+0000","last_clean":"2024-09-16T15:54:56.481270+0000","last_became_active":"2024-09-16T15:54:56.475925+0000","last_became_peered":"2024-09-16T15:54:56.475925+0000","last_unstale":"2024-09-16T15:54:56.481270+0000","last_undegraded":"2024-09-16T15:54:56.481270+0000","last_fullsized":"2024-09-16T15:54:56.481270+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:37:11.196350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.445058+0000","last_change":"2024-09-16T15:54:55.445058+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.445058+0000","last_undegraded":"2024-09-16T15:54:55.445058+0000","last_fullsized":"2024-09-16T15:54:55.445058+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.445531+0000","last_change":"2024-09-16T15:54:55.445531+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.445531+0000","last_undegraded":"2024-09-16T15:54:55.445531+0000","last_fullsized":"2024-09-16T15:54:55.445531+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.442779+0000","last_change":"2024-09-16T15:54:55.442779+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.442779+0000","last_undegraded":"2024-09-16T15:54:55.442779+0000","last_fullsized":"2024-09-16T15:54:55.442779+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.446794+0000","last_change":"2024-09-16T15:54:55.446794+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.446794+0000","last_undegraded":"2024-09-16T15:54:55.446794+0000","last_fullsized":"2024-09-16T15:54:55.446794+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.449202+0000","last_change":"2024-09-16T15:54:55.449202+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.449202+0000","last_undegraded":"2024-09-16T15:54:55.449202+0000","last_fullsized":"2024-09-16T15:54:55.449202+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-16T15:54:56.466175+0000","last_change":"2024-09-16T15:54:56.462956+0000","last_active":"2024-09-16T15:54:56.466175+0000","last_peered":"2024-09-16T15:54:56.466175+0000","last_clean":"2024-09-16T15:54:56.466175+0000","last_became_active":"2024-09-16T15:54:56.462662+0000","last_became_peered":"2024-09-16T15:54:56.462662+0000","last_unstale":"2024-09-16T15:54:56.466175+0000","last_undegraded":"2024-09-16T15:54:56.466175+0000","last_fullsized":"2024-09-16T15:54:56.466175+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:30:20.069211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482638+0000","last_change":"2024-09-16T15:54:56.481953+0000","last_active":"2024-09-16T15:54:57.482638+0000","last_peered":"2024-09-16T15:54:57.482638+0000","last_clean":"2024-09-16T15:54:57.482638+0000","last_became_active":"2024-09-16T15:54:56.481478+0000","last_became_peered":"2024-09-16T15:54:56.481478+0000","last_unstale":"2024-09-16T15:54:57.482638+0000","last_undegraded":"2024-09-16T15:54:57.482638+0000","last_fullsized":"2024-09-16T15:54:57.482638+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:05:20.399047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.445204+0000","last_change":"2024-09-16T15:54:55.445204+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.445204+0000","last_undegraded":"2024-09-16T15:54:55.445204+0000","last_fullsized":"2024-09-16T15:54:55.445204+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482766+0000","last_change":"2024-09-16T15:54:56.481838+0000","last_active":"2024-09-16T15:54:57.482766+0000","last_peered":"2024-09-16T15:54:57.482766+0000","last_clean":"2024-09-16T15:54:57.482766+0000","last_became_active":"2024-09-16T15:54:56.481562+0000","last_became_peered":"2024-09-16T15:54:56.481562+0000","last_unstale":"2024-09-16T15:54:57.482766+0000","last_undegraded":"2024-09-16T15:54:57.482766+0000","last_fullsized":"2024-09-16T15:54:57.482766+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:45:37.323453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482964+0000","last_change":"2024-09-16T15:54:56.490124+0000","last_active":"2024-09-16T15:54:57.482964+0000","last_peered":"2024-09-16T15:54:57.482964+0000","last_clean":"2024-09-16T15:54:57.482964+0000","last_became_active":"2024-09-16T15:54:56.489754+0000","last_became_peered":"2024-09-16T15:54:56.489754+0000","last_unstale":"2024-09-16T15:54:57.482964+0000","last_undegraded":"2024-09-16T15:54:57.482964+0000","last_fullsized":"2024-09-16T15:54:57.482964+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:27:59.188389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.444909+0000","last_change":"2024-09-16T15:54:55.444909+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.444909+0000","last_undegraded":"2024-09-16T15:54:55.444909+0000","last_fullsized":"2024-09-16T15:54:55.444909+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.445300+0000","last_change":"2024-09-16T15:54:55.445300+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.445300+0000","last_undegraded":"2024-09-16T15:54:55.445300+0000","last_fullsized":"2024-09-16T15:54:55.445300+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482388+0000","last_change":"2024-09-16T15:54:56.487219+0000","last_active":"2024-09-16T15:54:57.482388+0000","last_peered":"2024-09-16T15:54:57.482388+0000","last_clean":"2024-09-16T15:54:57.482388+0000","last_became_active":"2024-09-16T15:54:56.483904+0000","last_became_peered":"2024-09-16T15:54:56.483904+0000","last_unstale":"2024-09-16T15:54:57.482388+0000","last_undegraded":"2024-09-16T15:54:57.482388+0000","last_fullsized":"2024-09-16T15:54:57.482388+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:29:17.110559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482516+0000","last_change":"2024-09-16T15:54:56.477414+0000","last_active":"2024-09-16T15:54:57.482516+0000","last_peered":"2024-09-16T15:54:57.482516+0000","last_clean":"2024-09-16T15:54:57.482516+0000","last_became_active":"2024-09-16T15:54:56.476813+0000","last_became_peered":"2024-09-16T15:54:56.476813+0000","last_unstale":"2024-09-16T15:54:57.482516+0000","last_undegraded":"2024-09-16T15:54:57.482516+0000","last_fullsized":"2024-09-16T15:54:57.482516+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:07:02.927763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-16T15:54:56.481294+0000","last_change":"2024-09-16T15:54:56.478959+0000","last_active":"2024-09-16T15:54:56.481294+0000","last_peered":"2024-09-16T15:54:56.481294+0000","last_clean":"2024-09-16T15:54:56.481294+0000","last_became_active":"2024-09-16T15:54:56.478054+0000","last_became_peered":"2024-09-16T15:54:56.478054+0000","last_unstale":"2024-09-16T15:54:56.481294+0000","last_undegraded":"2024-09-16T15:54:56.481294+0000","last_fullsized":"2024-09-16T15:54:56.481294+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:11:14.095245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.459521+0000","last_change":"2024-09-16T15:54:55.458310+0000","last_active":"2024-09-16T15:54:55.459521+0000","last_peered":"2024-09-16T15:54:55.459521+0000","last_clean":"2024-09-16T15:54:55.459521+0000","last_became_active":"2024-09-16T15:54:55.457467+0000","last_became_peered":"2024-09-16T15:54:55.457467+0000","last_unstale":"2024-09-16T15:54:55.459521+0000","last_undegraded":"2024-09-16T15:54:55.459521+0000","last_fullsized":"2024-09-16T15:54:55.459521+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:52:32.558241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.447849+0000","last_change":"2024-09-16T15:54:55.447849+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.447849+0000","last_undegraded":"2024-09-16T15:54:55.447849+0000","last_fullsized":"2024-09-16T15:54:55.447849+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.459132+0000","last_change":"2024-09-16T15:54:55.458458+0000","last_active":"2024-09-16T15:54:55.459132+0000","last_peered":"2024-09-16T15:54:55.459132+0000","last_clean":"2024-09-16T15:54:55.459132+0000","last_became_active":"2024-09-16T15:54:55.457678+0000","last_became_peered":"2024-09-16T15:54:55.457678+0000","last_unstale":"2024-09-16T15:54:55.459132+0000","last_undegraded":"2024-09-16T15:54:55.459132+0000","last_fullsized":"2024-09-16T15:54:55.459132+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:20:58.630650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.450883+0000","last_change":"2024-09-16T15:54:55.450883+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.450883+0000","last_undegraded":"2024-09-16T15:54:55.450883+0000","last_fullsized":"2024-09-16T15:54:55.450883+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.465013+0000","last_change":"2024-09-16T15:54:55.464149+0000","last_active":"2024-09-16T15:54:55.465013+0000","last_peered":"2024-09-16T15:54:55.465013+0000","last_clean":"2024-09-16T15:54:55.465013+0000","last_became_active":"2024-09-16T15:54:55.463890+0000","last_became_peered":"2024-09-16T15:54:55.463890+0000","last_unstale":"2024-09-16T15:54:55.465013+0000","last_undegraded":"2024-09-16T15:54:55.465013+0000","last_fullsized":"2024-09-16T15:54:55.465013+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:03:49.222319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.449829+0000","last_change":"2024-09-16T15:54:55.449829+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.449829+0000","last_undegraded":"2024-09-16T15:54:55.449829+0000","last_fullsized":"2024-09-16T15:54:55.449829+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.460351+0000","last_change":"2024-09-16T15:54:55.458344+0000","last_active":"2024-09-16T15:54:55.460351+0000","last_peered":"2024-09-16T15:54:55.460351+0000","last_clean":"2024-09-16T15:54:55.460351+0000","last_became_active":"2024-09-16T15:54:55.455322+0000","last_became_peered":"2024-09-16T15:54:55.455322+0000","last_unstale":"2024-09-16T15:54:55.460351+0000","last_undegraded":"2024-09-16T15:54:55.460351+0000","last_fullsized":"2024-09-16T15:54:55.460351+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:14:46.177213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482428+0000","last_change":"2024-09-16T15:54:56.489708+0000","last_active":"2024-09-16T15:54:57.482428+0000","last_peered":"2024-09-16T15:54:57.482428+0000","last_clean":"2024-09-16T15:54:57.482428+0000","last_became_active":"2024-09-16T15:54:56.489545+0000","last_became_peered":"2024-09-16T15:54:56.489545+0000","last_unstale":"2024-09-16T15:54:57.482428+0000","last_undegraded":"2024-09-16T15:54:57.482428+0000","last_fullsized":"2024-09-16T15:54:57.482428+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:59:43.257767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.456546+0000","last_change":"2024-09-16T15:54:55.449973+0000","last_active":"2024-09-16T15:54:55.456546+0000","last_peered":"2024-09-16T15:54:55.456546+0000","last_clean":"2024-09-16T15:54:55.456546+0000","last_became_active":"2024-09-16T15:54:55.449676+0000","last_became_peered":"2024-09-16T15:54:55.449676+0000","last_unstale":"2024-09-16T15:54:55.456546+0000","last_undegraded":"2024-09-16T15:54:55.456546+0000","last_fullsized":"2024-09-16T15:54:55.456546+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:01:09.437888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.450288+0000","last_change":"2024-09-16T15:54:55.450288+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.450288+0000","last_undegraded":"2024-09-16T15:54:55.450288+0000","last_fullsized":"2024-09-16T15:54:55.450288+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1a","version":"144'1","reported_seq":14,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-16T15:54:56.486191+0000","last_change":"2024-09-16T15:54:55.455815+0000","last_active":"2024-09-16T15:54:56.486191+0000","last_peered":"2024-09-16T15:54:56.486191+0000","last_clean":"2024-09-16T15:54:56.486191+0000","last_became_active":"2024-09-16T15:54:55.455533+0000","last_became_peered":"2024-09-16T15:54:55.455533+0000","last_unstale":"2024-09-16T15:54:56.486191+0000","last_undegraded":"2024-09-16T15:54:56.486191+0000","last_fullsized":"2024-09-16T15:54:56.486191+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:01:12.975683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482536+0000","last_change":"2024-09-16T15:54:56.477672+0000","last_active":"2024-09-16T15:54:57.482536+0000","last_peered":"2024-09-16T15:54:57.482536+0000","last_clean":"2024-09-16T15:54:57.482536+0000","last_became_active":"2024-09-16T15:54:56.477153+0000","last_became_peered":"2024-09-16T15:54:56.477153+0000","last_unstale":"2024-09-16T15:54:57.482536+0000","last_undegraded":"2024-09-16T15:54:57.482536+0000","last_fullsized":"2024-09-16T15:54:57.482536+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:41:22.581512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.457395+0000","last_change":"2024-09-16T15:54:55.455189+0000","last_active":"2024-09-16T15:54:55.457395+0000","last_peered":"2024-09-16T15:54:55.457395+0000","last_clean":"2024-09-16T15:54:55.457395+0000","last_became_active":"2024-09-16T15:54:55.454896+0000","last_became_peered":"2024-09-16T15:54:55.454896+0000","last_unstale":"2024-09-16T15:54:55.457395+0000","last_undegraded":"2024-09-16T15:54:55.457395+0000","last_fullsized":"2024-09-16T15:54:55.457395+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:20:34.830239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.448700+0000","last_change":"2024-09-16T15:54:55.448700+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.448700+0000","last_undegraded":"2024-09-16T15:54:55.448700+0000","last_fullsized":"2024-09-16T15:54:55.448700+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.461802+0000","last_change":"2024-09-16T15:54:55.460804+0000","last_active":"2024-09-16T15:54:55.461802+0000","last_peered":"2024-09-16T15:54:55.461802+0000","last_clean":"2024-09-16T15:54:55.461802+0000","last_became_active":"2024-09-16T15:54:55.460502+0000","last_became_peered":"2024-09-16T15:54:55.460502+0000","last_unstale":"2024-09-16T15:54:55.461802+0000","last_undegraded":"2024-09-16T15:54:55.461802+0000","last_fullsized":"2024-09-16T15:54:55.461802+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:02:30.256738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.454247+0000","last_change":"2024-09-16T15:54:55.454247+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.454247+0000","last_undegraded":"2024-09-16T15:54:55.454247+0000","last_fullsized":"2024-09-16T15:54:55.454247+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.463235+0000","last_change":"2024-09-16T15:54:55.460989+0000","last_active":"2024-09-16T15:54:55.463235+0000","last_peered":"2024-09-16T15:54:55.463235+0000","last_clean":"2024-09-16T15:54:55.463235+0000","last_became_active":"2024-09-16T15:54:55.459752+0000","last_became_peered":"2024-09-16T15:54:55.459752+0000","last_unstale":"2024-09-16T15:54:55.463235+0000","last_undegraded":"2024-09-16T15:54:55.463235+0000","last_fullsized":"2024-09-16T15:54:55.463235+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:14:12.423767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.443768+0000","last_change":"2024-09-16T15:54:55.443768+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.443768+0000","last_undegraded":"2024-09-16T15:54:55.443768+0000","last_fullsized":"2024-09-16T15:54:55.443768+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-16T15:54:56.459082+0000","last_change":"2024-09-16T15:54:55.462309+0000","last_active":"2024-09-16T15:54:56.459082+0000","last_peered":"2024-09-16T15:54:56.459082+0000","last_clean":"2024-09-16T15:54:56.459082+0000","last_became_active":"2024-09-16T15:54:55.462039+0000","last_became_peered":"2024-09-16T15:54:55.462039+0000","last_unstale":"2024-09-16T15:54:56.459082+0000","last_undegraded":"2024-09-16T15:54:56.459082+0000","last_fullsized":"2024-09-16T15:54:56.459082+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:32:00.564497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.452496+0000","last_change":"2024-09-16T15:54:55.452496+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.452496+0000","last_undegraded":"2024-09-16T15:54:55.452496+0000","last_fullsized":"2024-09-16T15:54:55.452496+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.465057+0000","last_change":"2024-09-16T15:54:55.461505+0000","last_active":"2024-09-16T15:54:55.465057+0000","last_peered":"2024-09-16T15:54:55.465057+0000","last_clean":"2024-09-16T15:54:55.465057+0000","last_became_active":"2024-09-16T15:54:55.461174+0000","last_became_peered":"2024-09-16T15:54:55.461174+0000","last_unstale":"2024-09-16T15:54:55.465057+0000","last_undegraded":"2024-09-16T15:54:55.465057+0000","last_fullsized":"2024-09-16T15:54:55.465057+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:58:07.658068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.450155+0000","last_change":"2024-09-16T15:54:55.450155+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.450155+0000","last_undegraded":"2024-09-16T15:54:55.450155+0000","last_fullsized":"2024-09-16T15:54:55.450155+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.464678+0000","last_change":"2024-09-16T15:54:55.462728+0000","last_active":"2024-09-16T15:54:55.464678+0000","last_peered":"2024-09-16T15:54:55.464678+0000","last_clean":"2024-09-16T15:54:55.464678+0000","last_became_active":"2024-09-16T15:54:55.462424+0000","last_became_peered":"2024-09-16T15:54:55.462424+0000","last_unstale":"2024-09-16T15:54:55.464678+0000","last_undegraded":"2024-09-16T15:54:55.464678+0000","last_fullsized":"2024-09-16T15:54:55.464678+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:30:10.712547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.445578+0000","last_change":"2024-09-16T15:54:55.445578+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.445578+0000","last_undegraded":"2024-09-16T15:54:55.445578+0000","last_fullsized":"2024-09-16T15:54:55.445578+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.458040+0000","last_change":"2024-09-16T15:54:55.455831+0000","last_active":"2024-09-16T15:54:55.458040+0000","last_peered":"2024-09-16T15:54:55.458040+0000","last_clean":"2024-09-16T15:54:55.458040+0000","last_became_active":"2024-09-16T15:54:55.455574+0000","last_became_peered":"2024-09-16T15:54:55.455574+0000","last_unstale":"2024-09-16T15:54:55.458040+0000","last_undegraded":"2024-09-16T15:54:55.458040+0000","last_fullsized":"2024-09-16T15:54:55.458040+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:43:13.336878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.453165+0000","last_change":"2024-09-16T15:54:55.453165+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.453165+0000","last_undegraded":"2024-09-16T15:54:55.453165+0000","last_fullsized":"2024-09-16T15:54:55.453165+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482490+0000","last_change":"2024-09-16T15:54:55.457472+0000","last_active":"2024-09-16T15:54:57.482490+0000","last_peered":"2024-09-16T15:54:57.482490+0000","last_clean":"2024-09-16T15:54:57.482490+0000","last_became_active":"2024-09-16T15:54:55.456471+0000","last_became_peered":"2024-09-16T15:54:55.456471+0000","last_unstale":"2024-09-16T15:54:57.482490+0000","last_undegraded":"2024-09-16T15:54:57.482490+0000","last_fullsized":"2024-09-16T15:54:57.482490+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:14:12.568813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.448288+0000","last_change":"2024-09-16T15:54:55.448288+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.448288+0000","last_undegraded":"2024-09-16T15:54:55.448288+0000","last_fullsized":"2024-09-16T15:54:55.448288+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.462749+0000","last_change":"2024-09-16T15:54:55.461170+0000","last_active":"2024-09-16T15:54:55.462749+0000","last_peered":"2024-09-16T15:54:55.462749+0000","last_clean":"2024-09-16T15:54:55.462749+0000","last_became_active":"2024-09-16T15:54:55.460915+0000","last_became_peered":"2024-09-16T15:54:55.460915+0000","last_unstale":"2024-09-16T15:54:55.462749+0000","last_undegraded":"2024-09-16T15:54:55.462749+0000","last_fullsized":"2024-09-16T15:54:55.462749+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:25:12.508089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-16T15:54:56.467209+0000","last_change":"2024-09-16T15:54:56.464019+0000","last_active":"2024-09-16T15:54:56.467209+0000","last_peered":"2024-09-16T15:54:56.467209+0000","last_clean":"2024-09-16T15:54:56.467209+0000","last_became_active":"2024-09-16T15:54:56.463741+0000","last_became_peered":"2024-09-16T15:54:56.463741+0000","last_unstale":"2024-09-16T15:54:56.467209+0000","last_undegraded":"2024-09-16T15:54:56.467209+0000","last_fullsized":"2024-09-16T15:54:56.467209+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:01:03.304745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.464322+0000","last_change":"2024-09-16T15:54:55.461113+0000","last_active":"2024-09-16T15:54:55.464322+0000","last_peered":"2024-09-16T15:54:55.464322+0000","last_clean":"2024-09-16T15:54:55.464322+0000","last_became_active":"2024-09-16T15:54:55.460866+0000","last_became_peered":"2024-09-16T15:54:55.460866+0000","last_unstale":"2024-09-16T15:54:55.464322+0000","last_undegraded":"2024-09-16T15:54:55.464322+0000","last_fullsized":"2024-09-16T15:54:55.464322+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:14:24.286251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482797+0000","last_change":"2024-09-16T15:54:56.491736+0000","last_active":"2024-09-16T15:54:57.482797+0000","last_peered":"2024-09-16T15:54:57.482797+0000","last_clean":"2024-09-16T15:54:57.482797+0000","last_became_active":"2024-09-16T15:54:56.491464+0000","last_became_peered":"2024-09-16T15:54:56.491464+0000","last_unstale":"2024-09-16T15:54:57.482797+0000","last_undegraded":"2024-09-16T15:54:57.482797+0000","last_fullsized":"2024-09-16T15:54:57.482797+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:35:45.264168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.460583+0000","last_change":"2024-09-16T15:54:55.458328+0000","last_active":"2024-09-16T15:54:55.460583+0000","last_peered":"2024-09-16T15:54:55.460583+0000","last_clean":"2024-09-16T15:54:55.460583+0000","last_became_active":"2024-09-16T15:54:55.457453+0000","last_became_peered":"2024-09-16T15:54:55.457453+0000","last_unstale":"2024-09-16T15:54:55.460583+0000","last_undegraded":"2024-09-16T15:54:55.460583+0000","last_fullsized":"2024-09-16T15:54:55.460583+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:16:21.173713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-16T15:54:56.467102+0000","last_change":"2024-09-16T15:54:56.464797+0000","last_active":"2024-09-16T15:54:56.467102+0000","last_peered":"2024-09-16T15:54:56.467102+0000","last_clean":"2024-09-16T15:54:56.467102+0000","last_became_active":"2024-09-16T15:54:56.463873+0000","last_became_peered":"2024-09-16T15:54:56.463873+0000","last_unstale":"2024-09-16T15:54:56.467102+0000","last_undegraded":"2024-09-16T15:54:56.467102+0000","last_fullsized":"2024-09-16T15:54:56.467102+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:02:11.742600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.467261+0000","last_change":"2024-09-16T15:54:55.463365+0000","last_active":"2024-09-16T15:54:55.467261+0000","last_peered":"2024-09-16T15:54:55.467261+0000","last_clean":"2024-09-16T15:54:55.467261+0000","last_became_active":"2024-09-16T15:54:55.462246+0000","last_became_peered":"2024-09-16T15:54:55.462246+0000","last_unstale":"2024-09-16T15:54:55.467261+0000","last_undegraded":"2024-09-16T15:54:55.467261+0000","last_fullsized":"2024-09-16T15:54:55.467261+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:54:15.667120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.448139+0000","last_change":"2024-09-16T15:54:55.448139+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.448139+0000","last_undegraded":"2024-09-16T15:54:55.448139+0000","last_fullsized":"2024-09-16T15:54:55.448139+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482928+0000","last_change":"2024-09-16T15:54:55.463637+0000","last_active":"2024-09-16T15:54:57.482928+0000","last_peered":"2024-09-16T15:54:57.482928+0000","last_clean":"2024-09-16T15:54:57.482928+0000","last_became_active":"2024-09-16T15:54:55.463289+0000","last_became_peered":"2024-09-16T15:54:55.463289+0000","last_unstale":"2024-09-16T15:54:57.482928+0000","last_undegraded":"2024-09-16T15:54:57.482928+0000","last_fullsized":"2024-09-16T15:54:57.482928+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:16:31.781727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.444870+0000","last_change":"2024-09-16T15:54:55.444870+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.444870+0000","last_undegraded":"2024-09-16T15:54:55.444870+0000","last_fullsized":"2024-09-16T15:54:55.444870+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.463186+0000","last_change":"2024-09-16T15:54:55.459884+0000","last_active":"2024-09-16T15:54:55.463186+0000","last_peered":"2024-09-16T15:54:55.463186+0000","last_clean":"2024-09-16T15:54:55.463186+0000","last_became_active":"2024-09-16T15:54:55.459556+0000","last_became_peered":"2024-09-16T15:54:55.459556+0000","last_unstale":"2024-09-16T15:54:55.463186+0000","last_undegraded":"2024-09-16T15:54:55.463186+0000","last_fullsized":"2024-09-16T15:54:55.463186+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:38:34.406517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482617+0000","last_change":"2024-09-16T15:54:56.485052+0000","last_active":"2024-09-16T15:54:57.482617+0000","last_peered":"2024-09-16T15:54:57.482617+0000","last_clean":"2024-09-16T15:54:57.482617+0000","last_became_active":"2024-09-16T15:54:56.483119+0000","last_became_peered":"2024-09-16T15:54:56.483119+0000","last_unstale":"2024-09-16T15:54:57.482617+0000","last_undegraded":"2024-09-16T15:54:57.482617+0000","last_fullsized":"2024-09-16T15:54:57.482617+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:50:40.992458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.467038+0000","last_change":"2024-09-16T15:54:55.464323+0000","last_active":"2024-09-16T15:54:55.467038+0000","last_peered":"2024-09-16T15:54:55.467038+0000","last_clean":"2024-09-16T15:54:55.467038+0000","last_became_active":"2024-09-16T15:54:55.463480+0000","last_became_peered":"2024-09-16T15:54:55.463480+0000","last_unstale":"2024-09-16T15:54:55.467038+0000","last_undegraded":"2024-09-16T15:54:55.467038+0000","last_fullsized":"2024-09-16T15:54:55.467038+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:39:22.898239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.443764+0000","last_change":"2024-09-16T15:54:55.443764+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.443764+0000","last_undegraded":"2024-09-16T15:54:55.443764+0000","last_fullsized":"2024-09-16T15:54:55.443764+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.463551+0000","last_change":"2024-09-16T15:54:55.459040+0000","last_active":"2024-09-16T15:54:55.463551+0000","last_peered":"2024-09-16T15:54:55.463551+0000","last_clean":"2024-09-16T15:54:55.463551+0000","last_became_active":"2024-09-16T15:54:55.458758+0000","last_became_peered":"2024-09-16T15:54:55.458758+0000","last_unstale":"2024-09-16T15:54:55.463551+0000","last_undegraded":"2024-09-16T15:54:55.463551+0000","last_fullsized":"2024-09-16T15:54:55.463551+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:31:02.873968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482549+0000","last_change":"2024-09-16T15:54:56.485351+0000","last_active":"2024-09-16T15:54:57.482549+0000","last_peered":"2024-09-16T15:54:57.482549+0000","last_clean":"2024-09-16T15:54:57.482549+0000","last_became_active":"2024-09-16T15:54:56.484236+0000","last_became_peered":"2024-09-16T15:54:56.484236+0000","last_unstale":"2024-09-16T15:54:57.482549+0000","last_undegraded":"2024-09-16T15:54:57.482549+0000","last_fullsized":"2024-09-16T15:54:57.482549+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:48:52.418050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.467028+0000","last_change":"2024-09-16T15:54:55.463632+0000","last_active":"2024-09-16T15:54:55.467028+0000","last_peered":"2024-09-16T15:54:55.467028+0000","last_clean":"2024-09-16T15:54:55.467028+0000","last_became_active":"2024-09-16T15:54:55.463347+0000","last_became_peered":"2024-09-16T15:54:55.463347+0000","last_unstale":"2024-09-16T15:54:55.467028+0000","last_undegraded":"2024-09-16T15:54:55.467028+0000","last_fullsized":"2024-09-16T15:54:55.467028+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:47:46.317280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.451273+0000","last_change":"2024-09-16T15:54:55.451273+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.451273+0000","last_undegraded":"2024-09-16T15:54:55.451273+0000","last_fullsized":"2024-09-16T15:54:55.451273+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.464299+0000","last_change":"2024-09-16T15:54:55.459856+0000","last_active":"2024-09-16T15:54:55.464299+0000","last_peered":"2024-09-16T15:54:55.464299+0000","last_clean":"2024-09-16T15:54:55.464299+0000","last_became_active":"2024-09-16T15:54:55.458670+0000","last_became_peered":"2024-09-16T15:54:55.458670+0000","last_unstale":"2024-09-16T15:54:55.464299+0000","last_undegraded":"2024-09-16T15:54:55.464299+0000","last_fullsized":"2024-09-16T15:54:55.464299+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:49:34.676553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.453193+0000","last_change":"2024-09-16T15:54:55.453193+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.453193+0000","last_undegraded":"2024-09-16T15:54:55.453193+0000","last_fullsized":"2024-09-16T15:54:55.453193+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.6","version":"145'1","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482760+0000","last_change":"2024-09-16T15:54:55.464427+0000","last_active":"2024-09-16T15:54:57.482760+0000","last_peered":"2024-09-16T15:54:57.482760+0000","last_clean":"2024-09-16T15:54:57.482760+0000","last_became_active":"2024-09-16T15:54:55.464043+0000","last_became_peered":"2024-09-16T15:54:55.464043+0000","last_unstale":"2024-09-16T15:54:57.482760+0000","last_undegraded":"2024-09-16T15:54:57.482760+0000","last_fullsized":"2024-09-16T15:54:57.482760+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:06:18.925531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.444371+0000","last_change":"2024-09-16T15:54:55.444371+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.444371+0000","last_undegraded":"2024-09-16T15:54:55.444371+0000","last_fullsized":"2024-09-16T15:54:55.444371+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.463400+0000","last_change":"2024-09-16T15:54:55.460656+0000","last_active":"2024-09-16T15:54:55.463400+0000","last_peered":"2024-09-16T15:54:55.463400+0000","last_clean":"2024-09-16T15:54:55.463400+0000","last_became_active":"2024-09-16T15:54:55.460149+0000","last_became_peered":"2024-09-16T15:54:55.460149+0000","last_unstale":"2024-09-16T15:54:55.463400+0000","last_undegraded":"2024-09-16T15:54:55.463400+0000","last_fullsized":"2024-09-16T15:54:55.463400+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:17:46.439566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.445555+0000","last_change":"2024-09-16T15:54:55.445555+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.445555+0000","last_undegraded":"2024-09-16T15:54:55.445555+0000","last_fullsized":"2024-09-16T15:54:55.445555+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.465100+0000","last_change":"2024-09-16T15:54:55.461333+0000","last_active":"2024-09-16T15:54:55.465100+0000","last_peered":"2024-09-16T15:54:55.465100+0000","last_clean":"2024-09-16T15:54:55.465100+0000","last_became_active":"2024-09-16T15:54:55.460016+0000","last_became_peered":"2024-09-16T15:54:55.460016+0000","last_unstale":"2024-09-16T15:54:55.465100+0000","last_undegraded":"2024-09-16T15:54:55.465100+0000","last_fullsized":"2024-09-16T15:54:55.465100+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:07:03.672777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.447708+0000","last_change":"2024-09-16T15:54:55.447708+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.447708+0000","last_undegraded":"2024-09-16T15:54:55.447708+0000","last_fullsized":"2024-09-16T15:54:55.447708+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482633+0000","last_change":"2024-09-16T15:54:55.455582+0000","last_active":"2024-09-16T15:54:57.482633+0000","last_peered":"2024-09-16T15:54:57.482633+0000","last_clean":"2024-09-16T15:54:57.482633+0000","last_became_active":"2024-09-16T15:54:55.453745+0000","last_became_peered":"2024-09-16T15:54:55.453745+0000","last_unstale":"2024-09-16T15:54:57.482633+0000","last_undegraded":"2024-09-16T15:54:57.482633+0000","last_fullsized":"2024-09-16T15:54:57.482633+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:42:00.578523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.448094+0000","last_change":"2024-09-16T15:54:55.448094+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.448094+0000","last_undegraded":"2024-09-16T15:54:55.448094+0000","last_fullsized":"2024-09-16T15:54:55.448094+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.455725+0000","last_change":"2024-09-16T15:54:55.449566+0000","last_active":"2024-09-16T15:54:55.455725+0000","last_peered":"2024-09-16T15:54:55.455725+0000","last_clean":"2024-09-16T15:54:55.455725+0000","last_became_active":"2024-09-16T15:54:55.449318+0000","last_became_peered":"2024-09-16T15:54:55.449318+0000","last_unstale":"2024-09-16T15:54:55.455725+0000","last_undegraded":"2024-09-16T15:54:55.455725+0000","last_fullsized":"2024-09-16T15:54:55.455725+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:38:14.471185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.451007+0000","last_change":"2024-09-16T15:54:55.451007+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.451007+0000","last_undegraded":"2024-09-16T15:54:55.451007+0000","last_fullsized":"2024-09-16T15:54:55.451007+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.462530+0000","last_change":"2024-09-16T15:54:55.461228+0000","last_active":"2024-09-16T15:54:55.462530+0000","last_peered":"2024-09-16T15:54:55.462530+0000","last_clean":"2024-09-16T15:54:55.462530+0000","last_became_active":"2024-09-16T15:54:55.460939+0000","last_became_peered":"2024-09-16T15:54:55.460939+0000","last_unstale":"2024-09-16T15:54:55.462530+0000","last_undegraded":"2024-09-16T15:54:55.462530+0000","last_fullsized":"2024-09-16T15:54:55.462530+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:20:34.830239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.442822+0000","last_change":"2024-09-16T15:54:55.442822+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.442822+0000","last_undegraded":"2024-09-16T15:54:55.442822+0000","last_fullsized":"2024-09-16T15:54:55.442822+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.463143+0000","last_change":"2024-09-16T15:54:55.459516+0000","last_active":"2024-09-16T15:54:55.463143+0000","last_peered":"2024-09-16T15:54:55.463143+0000","last_clean":"2024-09-16T15:54:55.463143+0000","last_became_active":"2024-09-16T15:54:55.459041+0000","last_became_peered":"2024-09-16T15:54:55.459041+0000","last_unstale":"2024-09-16T15:54:55.463143+0000","last_undegraded":"2024-09-16T15:54:55.463143+0000","last_fullsized":"2024-09-16T15:54:55.463143+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:11:04.495951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.449438+0000","last_change":"2024-09-16T15:54:55.449438+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.449438+0000","last_undegraded":"2024-09-16T15:54:55.449438+0000","last_fullsized":"2024-09-16T15:54:55.449438+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.463059+0000","last_change":"2024-09-16T15:54:55.461430+0000","last_active":"2024-09-16T15:54:55.463059+0000","last_peered":"2024-09-16T15:54:55.463059+0000","last_clean":"2024-09-16T15:54:55.463059+0000","last_became_active":"2024-09-16T15:54:55.461166+0000","last_became_peered":"2024-09-16T15:54:55.461166+0000","last_unstale":"2024-09-16T15:54:55.463059+0000","last_undegraded":"2024-09-16T15:54:55.463059+0000","last_fullsized":"2024-09-16T15:54:55.463059+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:20:58.630650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":343,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-16T15:54:56.457213+0000","last_change":"2024-09-16T15:39:12.698813+0000","last_active":"2024-09-16T15:54:56.457213+0000","last_peered":"2024-09-16T15:54:56.457213+0000","last_clean":"2024-09-16T15:54:56.457213+0000","last_became_active":"2024-09-16T15:39:12.697695+0000","last_became_peered":"2024-09-16T15:39:12.697695+0000","last_unstale":"2024-09-16T15:54:56.457213+0000","last_undegraded":"2024-09-16T15:54:56.457213+0000","last_fullsized":"2024-09-16T15:54:56.457213+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_clean_scrub_stamp":"2024-09-16T15:39:11.684967+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:02.061081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509634,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":163348,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":161598,"kb_avail":94208492,"statfs":{"total":96636764160,"available":96469495808,"internally_reserved":0,"allocated":1769472,"data_stored":781042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":165476808},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509633,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123672,"kb_used_data":2308,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248168,"statfs":{"total":96636764160,"available":96510124032,"internally_reserved":0,"allocated":2363392,"data_stored":1371410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":124254663},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509632,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":136996,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94234844,"statfs":{"total":96636764160,"available":96496480256,"internally_reserved":0,"allocated":1781760,"data_stored":787457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":138475973},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509632,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":137000,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94234840,"statfs":{"total":96636764160,"available":96496476160,"internally_reserved":0,"allocated":1781760,"data_stored":787457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":138475977},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509632,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":118952,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94252888,"statfs":{"total":96636764160,"available":96514957312,"internally_reserved":0,"allocated":1781760,"data_stored":787457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":119994823},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509633,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":132004,"kb_used_data":2320,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94239836,"statfs":{"total":96636764160,"available":96501592064,"internally_reserved":0,"allocated":2375680,"data_stored":1377825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509631,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149476,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222364,"statfs":{"total":96636764160,"available":96483700736,"internally_reserved":0,"allocated":1781760,"data_stored":787457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":151255495},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542339,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131452,"kb_used_data":1764,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240388,"statfs":{"total":96636764160,"available":96502157312,"internally_reserved":0,"allocated":1806336,"data_stored":800812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":132774348},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:55:01.084 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:55:01.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.277+0000 7fd904600640 1 -- 172.21.15.151:0/243226382 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8fc0ccb30 msgr2=0x7fd8fc06f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:01.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.277+0000 7fd904600640 1 --2- 172.21.15.151:0/243226382 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8fc0ccb30 0x7fd8fc06f850 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7fd8e8005fd0 tx=0x7fd8e802a5d0 comp rx=0 tx=0).stop 2024-09-16T15:55:01.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.277+0000 7fd904600640 1 -- 172.21.15.151:0/243226382 shutdown_connections 2024-09-16T15:55:01.282 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.277+0000 7fd904600640 1 --2- 172.21.15.151:0/243226382 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8fc0ccb30 0x7fd8fc06f850 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:01.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.277+0000 7fd904600640 1 --2- 172.21.15.151:0/243226382 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8fc1438b0 0x7fd8fc0cc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:01.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.277+0000 7fd904600640 1 -- 172.21.15.151:0/243226382 >> 172.21.15.151:0/243226382 conn(0x7fd8fc0b1950 msgr2=0x7fd8fc0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:55:01.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.277+0000 7fd904600640 1 -- 172.21.15.151:0/243226382 shutdown_connections 2024-09-16T15:55:01.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.277+0000 7fd904600640 1 -- 172.21.15.151:0/243226382 wait complete. 2024-09-16T15:55:01.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.279+0000 7fd904600640 1 Processor -- start 2024-09-16T15:55:01.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.279+0000 7fd904600640 1 -- start start 2024-09-16T15:55:01.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.280+0000 7fd904600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8fc06fd90 0x7fd8fc15ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:01.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.280+0000 7fd904600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8fc0ccb30 0x7fd8fc13fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:01.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.280+0000 7fd904600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8fc1438b0 0x7fd8fc140100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:01.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.280+0000 7fd904600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd8fc070aa0 con 0x7fd8fc1438b0 2024-09-16T15:55:01.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.280+0000 7fd904600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd8fc070920 con 0x7fd8fc0ccb30 2024-09-16T15:55:01.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.280+0000 7fd904600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fd8fc15e300 con 0x7fd8fc06fd90 2024-09-16T15:55:01.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.280+0000 7fd902600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8fc1438b0 0x7fd8fc140100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:01.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.280+0000 7fd902600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8fc1438b0 0x7fd8fc140100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52632/0 (socket says 172.21.15.151:52632) 2024-09-16T15:55:01.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.280+0000 7fd902600640 1 -- 172.21.15.151:0/4255962354 learned_addr learned my addr 172.21.15.151:0/4255962354 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:55:01.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.280+0000 7fd901c00640 1 --2- 172.21.15.151:0/4255962354 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8fc06fd90 0x7fd8fc15ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:01.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.280+0000 7fd901200640 1 --2- 172.21.15.151:0/4255962354 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8fc0ccb30 0x7fd8fc13fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:01.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.280+0000 7fd902600640 1 -- 172.21.15.151:0/4255962354 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8fc06fd90 msgr2=0x7fd8fc15ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:01.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.280+0000 7fd902600640 1 --2- 172.21.15.151:0/4255962354 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8fc06fd90 0x7fd8fc15ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:01.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.280+0000 7fd902600640 1 -- 172.21.15.151:0/4255962354 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8fc0ccb30 msgr2=0x7fd8fc13fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:01.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.280+0000 7fd902600640 1 --2- 172.21.15.151:0/4255962354 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8fc0ccb30 0x7fd8fc13fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:01.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.280+0000 7fd902600640 1 -- 172.21.15.151:0/4255962354 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8fc15b2c0 con 0x7fd8fc1438b0 2024-09-16T15:55:01.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.280+0000 7fd901200640 1 --2- 172.21.15.151:0/4255962354 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8fc0ccb30 0x7fd8fc13fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:55:01.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.281+0000 7fd902600640 1 --2- 172.21.15.151:0/4255962354 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8fc1438b0 0x7fd8fc140100 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7fd8f400c7a0 tx=0x7fd8f400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:55:01.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.281+0000 7fd8f9600640 1 -- 172.21.15.151:0/4255962354 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f4003b70 con 0x7fd8fc1438b0 2024-09-16T15:55:01.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.281+0000 7fd8f9600640 1 -- 172.21.15.151:0/4255962354 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd8f4003d10 con 0x7fd8fc1438b0 2024-09-16T15:55:01.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.281+0000 7fd8f9600640 1 -- 172.21.15.151:0/4255962354 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f400aab0 con 0x7fd8fc1438b0 2024-09-16T15:55:01.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.281+0000 7fd904600640 1 -- 172.21.15.151:0/4255962354 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8fc0c90e0 con 0x7fd8fc1438b0 2024-09-16T15:55:01.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.281+0000 7fd904600640 1 -- 172.21.15.151:0/4255962354 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8fc0c9380 con 0x7fd8fc1438b0 2024-09-16T15:55:01.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.282+0000 7fd904600640 1 -- 172.21.15.151:0/4255962354 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8fc15e780 con 0x7fd8fc1438b0 2024-09-16T15:55:01.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.283+0000 7fd8f9600640 1 -- 172.21.15.151:0/4255962354 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 50) ==== 104398+0+0 (secure 0 0 0) 0x7fd8f400ac50 con 0x7fd8fc1438b0 2024-09-16T15:55:01.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.284+0000 7fd8f9600640 1 --2- 172.21.15.151:0/4255962354 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd8dc07aaa0 0x7fd8dc07cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:01.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.284+0000 7fd8f9600640 1 -- 172.21.15.151:0/4255962354 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5849+0+0 (secure 0 0 0) 0x7fd8f40999a0 con 0x7fd8fc1438b0 2024-09-16T15:55:01.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.284+0000 7fd901c00640 1 --2- 172.21.15.151:0/4255962354 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd8dc07aaa0 0x7fd8dc07cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:01.286 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.285+0000 7fd901c00640 1 --2- 172.21.15.151:0/4255962354 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd8dc07aaa0 0x7fd8dc07cf60 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0005fd0 tx=0x7fd8f0005850 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:55:01.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.288+0000 7fd8f9600640 1 -- 172.21.15.151:0/4255962354 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd8fc0cdae0 con 0x7fd8fc1438b0 2024-09-16T15:55:01.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.469+0000 7fd904600640 1 -- 172.21.15.151:0/4255962354 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd8fc15bab0 con 0x7fd8dc07aaa0 2024-09-16T15:55:01.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.473+0000 7fd8f9600640 1 -- 172.21.15.151:0/4255962354 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+248706 (secure 0 0 0) 0x7fd8fc15bab0 con 0x7fd8dc07aaa0 2024-09-16T15:55:01.477 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:55:01.479 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-09-16T15:55:01.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.477+0000 7fd904600640 1 -- 172.21.15.151:0/4255962354 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd8dc07aaa0 msgr2=0x7fd8dc07cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:01.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.477+0000 7fd904600640 1 --2- 172.21.15.151:0/4255962354 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd8dc07aaa0 0x7fd8dc07cf60 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0005fd0 tx=0x7fd8f0005850 comp rx=0 tx=0).stop 2024-09-16T15:55:01.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.477+0000 7fd904600640 1 -- 172.21.15.151:0/4255962354 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8fc1438b0 msgr2=0x7fd8fc140100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:01.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.477+0000 7fd904600640 1 --2- 172.21.15.151:0/4255962354 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8fc1438b0 0x7fd8fc140100 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7fd8f400c7a0 tx=0x7fd8f400cc60 comp rx=0 tx=0).stop 2024-09-16T15:55:01.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.477+0000 7fd904600640 1 -- 172.21.15.151:0/4255962354 shutdown_connections 2024-09-16T15:55:01.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.477+0000 7fd904600640 1 --2- 172.21.15.151:0/4255962354 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd8dc07aaa0 0x7fd8dc07cf60 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:01.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.477+0000 7fd904600640 1 --2- 172.21.15.151:0/4255962354 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8fc0ccb30 0x7fd8fc13fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:01.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.477+0000 7fd904600640 1 --2- 172.21.15.151:0/4255962354 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd8fc1438b0 0x7fd8fc140100 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:01.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.477+0000 7fd904600640 1 --2- 172.21.15.151:0/4255962354 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd8fc06fd90 0x7fd8fc15ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:01.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.477+0000 7fd904600640 1 -- 172.21.15.151:0/4255962354 >> 172.21.15.151:0/4255962354 conn(0x7fd8fc0b1950 msgr2=0x7fd8fc15c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:55:01.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.477+0000 7fd904600640 1 -- 172.21.15.151:0/4255962354 shutdown_connections 2024-09-16T15:55:01.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.478+0000 7fd904600640 1 -- 172.21.15.151:0/4255962354 wait complete. 2024-09-16T15:55:01.495 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":613,"stamp":"2024-09-16T15:54:59.958152+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":102,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1092900,"kb_used_data":15080,"kb_used_omap":12,"kb_used_meta":1077619,"kb_avail":753881820,"statfs":{"total":773094113280,"available":771974983680,"internally_reserved":0,"allocated":15441920,"data_stored":7480917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12738,"internal_metadata":1103482430},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.523823"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.445126+0000","last_change":"2024-09-16T15:54:55.445126+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.445126+0000","last_undegraded":"2024-09-16T15:54:55.445126+0000","last_fullsized":"2024-09-16T15:54:55.445126+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482752+0000","last_change":"2024-09-16T15:54:56.489100+0000","last_active":"2024-09-16T15:54:57.482752+0000","last_peered":"2024-09-16T15:54:57.482752+0000","last_clean":"2024-09-16T15:54:57.482752+0000","last_became_active":"2024-09-16T15:54:56.488821+0000","last_became_peered":"2024-09-16T15:54:56.488821+0000","last_unstale":"2024-09-16T15:54:57.482752+0000","last_undegraded":"2024-09-16T15:54:57.482752+0000","last_fullsized":"2024-09-16T15:54:57.482752+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:43:37.723000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.446549+0000","last_change":"2024-09-16T15:54:55.446549+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.446549+0000","last_undegraded":"2024-09-16T15:54:55.446549+0000","last_fullsized":"2024-09-16T15:54:55.446549+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.449901+0000","last_change":"2024-09-16T15:54:55.449901+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.449901+0000","last_undegraded":"2024-09-16T15:54:55.449901+0000","last_fullsized":"2024-09-16T15:54:55.449901+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-16T15:54:56.479967+0000","last_change":"2024-09-16T15:54:56.471793+0000","last_active":"2024-09-16T15:54:56.479967+0000","last_peered":"2024-09-16T15:54:56.479967+0000","last_clean":"2024-09-16T15:54:56.479967+0000","last_became_active":"2024-09-16T15:54:56.471048+0000","last_became_peered":"2024-09-16T15:54:56.471048+0000","last_unstale":"2024-09-16T15:54:56.479967+0000","last_undegraded":"2024-09-16T15:54:56.479967+0000","last_fullsized":"2024-09-16T15:54:56.479967+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:15:17.040666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.449521+0000","last_change":"2024-09-16T15:54:55.449521+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.449521+0000","last_undegraded":"2024-09-16T15:54:55.449521+0000","last_fullsized":"2024-09-16T15:54:55.449521+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.449819+0000","last_change":"2024-09-16T15:54:55.449819+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.449819+0000","last_undegraded":"2024-09-16T15:54:55.449819+0000","last_fullsized":"2024-09-16T15:54:55.449819+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.443482+0000","last_change":"2024-09-16T15:54:55.443482+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.443482+0000","last_undegraded":"2024-09-16T15:54:55.443482+0000","last_fullsized":"2024-09-16T15:54:55.443482+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.450377+0000","last_change":"2024-09-16T15:54:55.450377+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.450377+0000","last_undegraded":"2024-09-16T15:54:55.450377+0000","last_fullsized":"2024-09-16T15:54:55.450377+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.445356+0000","last_change":"2024-09-16T15:54:55.445356+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.445356+0000","last_undegraded":"2024-09-16T15:54:55.445356+0000","last_fullsized":"2024-09-16T15:54:55.445356+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.448658+0000","last_change":"2024-09-16T15:54:55.448658+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.448658+0000","last_undegraded":"2024-09-16T15:54:55.448658+0000","last_fullsized":"2024-09-16T15:54:55.448658+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.445187+0000","last_change":"2024-09-16T15:54:55.445187+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.445187+0000","last_undegraded":"2024-09-16T15:54:55.445187+0000","last_fullsized":"2024-09-16T15:54:55.445187+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-16T15:54:56.479929+0000","last_change":"2024-09-16T15:54:56.469650+0000","last_active":"2024-09-16T15:54:56.479929+0000","last_peered":"2024-09-16T15:54:56.479929+0000","last_clean":"2024-09-16T15:54:56.479929+0000","last_became_active":"2024-09-16T15:54:56.469407+0000","last_became_peered":"2024-09-16T15:54:56.469407+0000","last_unstale":"2024-09-16T15:54:56.479929+0000","last_undegraded":"2024-09-16T15:54:56.479929+0000","last_fullsized":"2024-09-16T15:54:56.479929+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:58:19.883027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.447273+0000","last_change":"2024-09-16T15:54:55.447273+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.447273+0000","last_undegraded":"2024-09-16T15:54:55.447273+0000","last_fullsized":"2024-09-16T15:54:55.447273+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-16T15:54:56.479726+0000","last_change":"2024-09-16T15:54:56.469340+0000","last_active":"2024-09-16T15:54:56.479726+0000","last_peered":"2024-09-16T15:54:56.479726+0000","last_clean":"2024-09-16T15:54:56.479726+0000","last_became_active":"2024-09-16T15:54:56.469088+0000","last_became_peered":"2024-09-16T15:54:56.469088+0000","last_unstale":"2024-09-16T15:54:56.479726+0000","last_undegraded":"2024-09-16T15:54:56.479726+0000","last_fullsized":"2024-09-16T15:54:56.479726+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:28:09.504034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.447718+0000","last_change":"2024-09-16T15:54:55.447718+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.447718+0000","last_undegraded":"2024-09-16T15:54:55.447718+0000","last_fullsized":"2024-09-16T15:54:55.447718+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.443888+0000","last_change":"2024-09-16T15:54:55.443888+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.443888+0000","last_undegraded":"2024-09-16T15:54:55.443888+0000","last_fullsized":"2024-09-16T15:54:55.443888+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-16T15:54:56.481270+0000","last_change":"2024-09-16T15:54:56.476168+0000","last_active":"2024-09-16T15:54:56.481270+0000","last_peered":"2024-09-16T15:54:56.481270+0000","last_clean":"2024-09-16T15:54:56.481270+0000","last_became_active":"2024-09-16T15:54:56.475925+0000","last_became_peered":"2024-09-16T15:54:56.475925+0000","last_unstale":"2024-09-16T15:54:56.481270+0000","last_undegraded":"2024-09-16T15:54:56.481270+0000","last_fullsized":"2024-09-16T15:54:56.481270+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:37:11.196350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.445058+0000","last_change":"2024-09-16T15:54:55.445058+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.445058+0000","last_undegraded":"2024-09-16T15:54:55.445058+0000","last_fullsized":"2024-09-16T15:54:55.445058+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.445531+0000","last_change":"2024-09-16T15:54:55.445531+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.445531+0000","last_undegraded":"2024-09-16T15:54:55.445531+0000","last_fullsized":"2024-09-16T15:54:55.445531+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.442779+0000","last_change":"2024-09-16T15:54:55.442779+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.442779+0000","last_undegraded":"2024-09-16T15:54:55.442779+0000","last_fullsized":"2024-09-16T15:54:55.442779+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.446794+0000","last_change":"2024-09-16T15:54:55.446794+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.446794+0000","last_undegraded":"2024-09-16T15:54:55.446794+0000","last_fullsized":"2024-09-16T15:54:55.446794+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.449202+0000","last_change":"2024-09-16T15:54:55.449202+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.449202+0000","last_undegraded":"2024-09-16T15:54:55.449202+0000","last_fullsized":"2024-09-16T15:54:55.449202+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-16T15:54:56.466175+0000","last_change":"2024-09-16T15:54:56.462956+0000","last_active":"2024-09-16T15:54:56.466175+0000","last_peered":"2024-09-16T15:54:56.466175+0000","last_clean":"2024-09-16T15:54:56.466175+0000","last_became_active":"2024-09-16T15:54:56.462662+0000","last_became_peered":"2024-09-16T15:54:56.462662+0000","last_unstale":"2024-09-16T15:54:56.466175+0000","last_undegraded":"2024-09-16T15:54:56.466175+0000","last_fullsized":"2024-09-16T15:54:56.466175+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:30:20.069211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482638+0000","last_change":"2024-09-16T15:54:56.481953+0000","last_active":"2024-09-16T15:54:57.482638+0000","last_peered":"2024-09-16T15:54:57.482638+0000","last_clean":"2024-09-16T15:54:57.482638+0000","last_became_active":"2024-09-16T15:54:56.481478+0000","last_became_peered":"2024-09-16T15:54:56.481478+0000","last_unstale":"2024-09-16T15:54:57.482638+0000","last_undegraded":"2024-09-16T15:54:57.482638+0000","last_fullsized":"2024-09-16T15:54:57.482638+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:05:20.399047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.445204+0000","last_change":"2024-09-16T15:54:55.445204+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.445204+0000","last_undegraded":"2024-09-16T15:54:55.445204+0000","last_fullsized":"2024-09-16T15:54:55.445204+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482766+0000","last_change":"2024-09-16T15:54:56.481838+0000","last_active":"2024-09-16T15:54:57.482766+0000","last_peered":"2024-09-16T15:54:57.482766+0000","last_clean":"2024-09-16T15:54:57.482766+0000","last_became_active":"2024-09-16T15:54:56.481562+0000","last_became_peered":"2024-09-16T15:54:56.481562+0000","last_unstale":"2024-09-16T15:54:57.482766+0000","last_undegraded":"2024-09-16T15:54:57.482766+0000","last_fullsized":"2024-09-16T15:54:57.482766+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:45:37.323453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482964+0000","last_change":"2024-09-16T15:54:56.490124+0000","last_active":"2024-09-16T15:54:57.482964+0000","last_peered":"2024-09-16T15:54:57.482964+0000","last_clean":"2024-09-16T15:54:57.482964+0000","last_became_active":"2024-09-16T15:54:56.489754+0000","last_became_peered":"2024-09-16T15:54:56.489754+0000","last_unstale":"2024-09-16T15:54:57.482964+0000","last_undegraded":"2024-09-16T15:54:57.482964+0000","last_fullsized":"2024-09-16T15:54:57.482964+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:27:59.188389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.444909+0000","last_change":"2024-09-16T15:54:55.444909+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.444909+0000","last_undegraded":"2024-09-16T15:54:55.444909+0000","last_fullsized":"2024-09-16T15:54:55.444909+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.445300+0000","last_change":"2024-09-16T15:54:55.445300+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.445300+0000","last_undegraded":"2024-09-16T15:54:55.445300+0000","last_fullsized":"2024-09-16T15:54:55.445300+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482388+0000","last_change":"2024-09-16T15:54:56.487219+0000","last_active":"2024-09-16T15:54:57.482388+0000","last_peered":"2024-09-16T15:54:57.482388+0000","last_clean":"2024-09-16T15:54:57.482388+0000","last_became_active":"2024-09-16T15:54:56.483904+0000","last_became_peered":"2024-09-16T15:54:56.483904+0000","last_unstale":"2024-09-16T15:54:57.482388+0000","last_undegraded":"2024-09-16T15:54:57.482388+0000","last_fullsized":"2024-09-16T15:54:57.482388+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:29:17.110559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482516+0000","last_change":"2024-09-16T15:54:56.477414+0000","last_active":"2024-09-16T15:54:57.482516+0000","last_peered":"2024-09-16T15:54:57.482516+0000","last_clean":"2024-09-16T15:54:57.482516+0000","last_became_active":"2024-09-16T15:54:56.476813+0000","last_became_peered":"2024-09-16T15:54:56.476813+0000","last_unstale":"2024-09-16T15:54:57.482516+0000","last_undegraded":"2024-09-16T15:54:57.482516+0000","last_fullsized":"2024-09-16T15:54:57.482516+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:07:02.927763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-16T15:54:56.481294+0000","last_change":"2024-09-16T15:54:56.478959+0000","last_active":"2024-09-16T15:54:56.481294+0000","last_peered":"2024-09-16T15:54:56.481294+0000","last_clean":"2024-09-16T15:54:56.481294+0000","last_became_active":"2024-09-16T15:54:56.478054+0000","last_became_peered":"2024-09-16T15:54:56.478054+0000","last_unstale":"2024-09-16T15:54:56.481294+0000","last_undegraded":"2024-09-16T15:54:56.481294+0000","last_fullsized":"2024-09-16T15:54:56.481294+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:11:14.095245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.459521+0000","last_change":"2024-09-16T15:54:55.458310+0000","last_active":"2024-09-16T15:54:55.459521+0000","last_peered":"2024-09-16T15:54:55.459521+0000","last_clean":"2024-09-16T15:54:55.459521+0000","last_became_active":"2024-09-16T15:54:55.457467+0000","last_became_peered":"2024-09-16T15:54:55.457467+0000","last_unstale":"2024-09-16T15:54:55.459521+0000","last_undegraded":"2024-09-16T15:54:55.459521+0000","last_fullsized":"2024-09-16T15:54:55.459521+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:52:32.558241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.447849+0000","last_change":"2024-09-16T15:54:55.447849+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.447849+0000","last_undegraded":"2024-09-16T15:54:55.447849+0000","last_fullsized":"2024-09-16T15:54:55.447849+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.459132+0000","last_change":"2024-09-16T15:54:55.458458+0000","last_active":"2024-09-16T15:54:55.459132+0000","last_peered":"2024-09-16T15:54:55.459132+0000","last_clean":"2024-09-16T15:54:55.459132+0000","last_became_active":"2024-09-16T15:54:55.457678+0000","last_became_peered":"2024-09-16T15:54:55.457678+0000","last_unstale":"2024-09-16T15:54:55.459132+0000","last_undegraded":"2024-09-16T15:54:55.459132+0000","last_fullsized":"2024-09-16T15:54:55.459132+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:20:58.630650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.450883+0000","last_change":"2024-09-16T15:54:55.450883+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.450883+0000","last_undegraded":"2024-09-16T15:54:55.450883+0000","last_fullsized":"2024-09-16T15:54:55.450883+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.465013+0000","last_change":"2024-09-16T15:54:55.464149+0000","last_active":"2024-09-16T15:54:55.465013+0000","last_peered":"2024-09-16T15:54:55.465013+0000","last_clean":"2024-09-16T15:54:55.465013+0000","last_became_active":"2024-09-16T15:54:55.463890+0000","last_became_peered":"2024-09-16T15:54:55.463890+0000","last_unstale":"2024-09-16T15:54:55.465013+0000","last_undegraded":"2024-09-16T15:54:55.465013+0000","last_fullsized":"2024-09-16T15:54:55.465013+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:03:49.222319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.449829+0000","last_change":"2024-09-16T15:54:55.449829+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.449829+0000","last_undegraded":"2024-09-16T15:54:55.449829+0000","last_fullsized":"2024-09-16T15:54:55.449829+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.460351+0000","last_change":"2024-09-16T15:54:55.458344+0000","last_active":"2024-09-16T15:54:55.460351+0000","last_peered":"2024-09-16T15:54:55.460351+0000","last_clean":"2024-09-16T15:54:55.460351+0000","last_became_active":"2024-09-16T15:54:55.455322+0000","last_became_peered":"2024-09-16T15:54:55.455322+0000","last_unstale":"2024-09-16T15:54:55.460351+0000","last_undegraded":"2024-09-16T15:54:55.460351+0000","last_fullsized":"2024-09-16T15:54:55.460351+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:14:46.177213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482428+0000","last_change":"2024-09-16T15:54:56.489708+0000","last_active":"2024-09-16T15:54:57.482428+0000","last_peered":"2024-09-16T15:54:57.482428+0000","last_clean":"2024-09-16T15:54:57.482428+0000","last_became_active":"2024-09-16T15:54:56.489545+0000","last_became_peered":"2024-09-16T15:54:56.489545+0000","last_unstale":"2024-09-16T15:54:57.482428+0000","last_undegraded":"2024-09-16T15:54:57.482428+0000","last_fullsized":"2024-09-16T15:54:57.482428+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:59:43.257767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.456546+0000","last_change":"2024-09-16T15:54:55.449973+0000","last_active":"2024-09-16T15:54:55.456546+0000","last_peered":"2024-09-16T15:54:55.456546+0000","last_clean":"2024-09-16T15:54:55.456546+0000","last_became_active":"2024-09-16T15:54:55.449676+0000","last_became_peered":"2024-09-16T15:54:55.449676+0000","last_unstale":"2024-09-16T15:54:55.456546+0000","last_undegraded":"2024-09-16T15:54:55.456546+0000","last_fullsized":"2024-09-16T15:54:55.456546+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:01:09.437888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.450288+0000","last_change":"2024-09-16T15:54:55.450288+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.450288+0000","last_undegraded":"2024-09-16T15:54:55.450288+0000","last_fullsized":"2024-09-16T15:54:55.450288+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1a","version":"144'1","reported_seq":14,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-16T15:54:56.486191+0000","last_change":"2024-09-16T15:54:55.455815+0000","last_active":"2024-09-16T15:54:56.486191+0000","last_peered":"2024-09-16T15:54:56.486191+0000","last_clean":"2024-09-16T15:54:56.486191+0000","last_became_active":"2024-09-16T15:54:55.455533+0000","last_became_peered":"2024-09-16T15:54:55.455533+0000","last_unstale":"2024-09-16T15:54:56.486191+0000","last_undegraded":"2024-09-16T15:54:56.486191+0000","last_fullsized":"2024-09-16T15:54:56.486191+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:01:12.975683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482536+0000","last_change":"2024-09-16T15:54:56.477672+0000","last_active":"2024-09-16T15:54:57.482536+0000","last_peered":"2024-09-16T15:54:57.482536+0000","last_clean":"2024-09-16T15:54:57.482536+0000","last_became_active":"2024-09-16T15:54:56.477153+0000","last_became_peered":"2024-09-16T15:54:56.477153+0000","last_unstale":"2024-09-16T15:54:57.482536+0000","last_undegraded":"2024-09-16T15:54:57.482536+0000","last_fullsized":"2024-09-16T15:54:57.482536+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:41:22.581512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.457395+0000","last_change":"2024-09-16T15:54:55.455189+0000","last_active":"2024-09-16T15:54:55.457395+0000","last_peered":"2024-09-16T15:54:55.457395+0000","last_clean":"2024-09-16T15:54:55.457395+0000","last_became_active":"2024-09-16T15:54:55.454896+0000","last_became_peered":"2024-09-16T15:54:55.454896+0000","last_unstale":"2024-09-16T15:54:55.457395+0000","last_undegraded":"2024-09-16T15:54:55.457395+0000","last_fullsized":"2024-09-16T15:54:55.457395+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:20:34.830239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.448700+0000","last_change":"2024-09-16T15:54:55.448700+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.448700+0000","last_undegraded":"2024-09-16T15:54:55.448700+0000","last_fullsized":"2024-09-16T15:54:55.448700+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.461802+0000","last_change":"2024-09-16T15:54:55.460804+0000","last_active":"2024-09-16T15:54:55.461802+0000","last_peered":"2024-09-16T15:54:55.461802+0000","last_clean":"2024-09-16T15:54:55.461802+0000","last_became_active":"2024-09-16T15:54:55.460502+0000","last_became_peered":"2024-09-16T15:54:55.460502+0000","last_unstale":"2024-09-16T15:54:55.461802+0000","last_undegraded":"2024-09-16T15:54:55.461802+0000","last_fullsized":"2024-09-16T15:54:55.461802+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:02:30.256738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.454247+0000","last_change":"2024-09-16T15:54:55.454247+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.454247+0000","last_undegraded":"2024-09-16T15:54:55.454247+0000","last_fullsized":"2024-09-16T15:54:55.454247+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.463235+0000","last_change":"2024-09-16T15:54:55.460989+0000","last_active":"2024-09-16T15:54:55.463235+0000","last_peered":"2024-09-16T15:54:55.463235+0000","last_clean":"2024-09-16T15:54:55.463235+0000","last_became_active":"2024-09-16T15:54:55.459752+0000","last_became_peered":"2024-09-16T15:54:55.459752+0000","last_unstale":"2024-09-16T15:54:55.463235+0000","last_undegraded":"2024-09-16T15:54:55.463235+0000","last_fullsized":"2024-09-16T15:54:55.463235+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:14:12.423767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.443768+0000","last_change":"2024-09-16T15:54:55.443768+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.443768+0000","last_undegraded":"2024-09-16T15:54:55.443768+0000","last_fullsized":"2024-09-16T15:54:55.443768+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":13,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-16T15:54:56.459082+0000","last_change":"2024-09-16T15:54:55.462309+0000","last_active":"2024-09-16T15:54:56.459082+0000","last_peered":"2024-09-16T15:54:56.459082+0000","last_clean":"2024-09-16T15:54:56.459082+0000","last_became_active":"2024-09-16T15:54:55.462039+0000","last_became_peered":"2024-09-16T15:54:55.462039+0000","last_unstale":"2024-09-16T15:54:56.459082+0000","last_undegraded":"2024-09-16T15:54:56.459082+0000","last_fullsized":"2024-09-16T15:54:56.459082+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:32:00.564497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.452496+0000","last_change":"2024-09-16T15:54:55.452496+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.452496+0000","last_undegraded":"2024-09-16T15:54:55.452496+0000","last_fullsized":"2024-09-16T15:54:55.452496+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.465057+0000","last_change":"2024-09-16T15:54:55.461505+0000","last_active":"2024-09-16T15:54:55.465057+0000","last_peered":"2024-09-16T15:54:55.465057+0000","last_clean":"2024-09-16T15:54:55.465057+0000","last_became_active":"2024-09-16T15:54:55.461174+0000","last_became_peered":"2024-09-16T15:54:55.461174+0000","last_unstale":"2024-09-16T15:54:55.465057+0000","last_undegraded":"2024-09-16T15:54:55.465057+0000","last_fullsized":"2024-09-16T15:54:55.465057+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:58:07.658068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.450155+0000","last_change":"2024-09-16T15:54:55.450155+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.450155+0000","last_undegraded":"2024-09-16T15:54:55.450155+0000","last_fullsized":"2024-09-16T15:54:55.450155+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.464678+0000","last_change":"2024-09-16T15:54:55.462728+0000","last_active":"2024-09-16T15:54:55.464678+0000","last_peered":"2024-09-16T15:54:55.464678+0000","last_clean":"2024-09-16T15:54:55.464678+0000","last_became_active":"2024-09-16T15:54:55.462424+0000","last_became_peered":"2024-09-16T15:54:55.462424+0000","last_unstale":"2024-09-16T15:54:55.464678+0000","last_undegraded":"2024-09-16T15:54:55.464678+0000","last_fullsized":"2024-09-16T15:54:55.464678+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:30:10.712547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.445578+0000","last_change":"2024-09-16T15:54:55.445578+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.445578+0000","last_undegraded":"2024-09-16T15:54:55.445578+0000","last_fullsized":"2024-09-16T15:54:55.445578+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.458040+0000","last_change":"2024-09-16T15:54:55.455831+0000","last_active":"2024-09-16T15:54:55.458040+0000","last_peered":"2024-09-16T15:54:55.458040+0000","last_clean":"2024-09-16T15:54:55.458040+0000","last_became_active":"2024-09-16T15:54:55.455574+0000","last_became_peered":"2024-09-16T15:54:55.455574+0000","last_unstale":"2024-09-16T15:54:55.458040+0000","last_undegraded":"2024-09-16T15:54:55.458040+0000","last_fullsized":"2024-09-16T15:54:55.458040+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:43:13.336878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.453165+0000","last_change":"2024-09-16T15:54:55.453165+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.453165+0000","last_undegraded":"2024-09-16T15:54:55.453165+0000","last_fullsized":"2024-09-16T15:54:55.453165+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482490+0000","last_change":"2024-09-16T15:54:55.457472+0000","last_active":"2024-09-16T15:54:57.482490+0000","last_peered":"2024-09-16T15:54:57.482490+0000","last_clean":"2024-09-16T15:54:57.482490+0000","last_became_active":"2024-09-16T15:54:55.456471+0000","last_became_peered":"2024-09-16T15:54:55.456471+0000","last_unstale":"2024-09-16T15:54:57.482490+0000","last_undegraded":"2024-09-16T15:54:57.482490+0000","last_fullsized":"2024-09-16T15:54:57.482490+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:14:12.568813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.448288+0000","last_change":"2024-09-16T15:54:55.448288+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.448288+0000","last_undegraded":"2024-09-16T15:54:55.448288+0000","last_fullsized":"2024-09-16T15:54:55.448288+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.462749+0000","last_change":"2024-09-16T15:54:55.461170+0000","last_active":"2024-09-16T15:54:55.462749+0000","last_peered":"2024-09-16T15:54:55.462749+0000","last_clean":"2024-09-16T15:54:55.462749+0000","last_became_active":"2024-09-16T15:54:55.460915+0000","last_became_peered":"2024-09-16T15:54:55.460915+0000","last_unstale":"2024-09-16T15:54:55.462749+0000","last_undegraded":"2024-09-16T15:54:55.462749+0000","last_fullsized":"2024-09-16T15:54:55.462749+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:25:12.508089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-16T15:54:56.467209+0000","last_change":"2024-09-16T15:54:56.464019+0000","last_active":"2024-09-16T15:54:56.467209+0000","last_peered":"2024-09-16T15:54:56.467209+0000","last_clean":"2024-09-16T15:54:56.467209+0000","last_became_active":"2024-09-16T15:54:56.463741+0000","last_became_peered":"2024-09-16T15:54:56.463741+0000","last_unstale":"2024-09-16T15:54:56.467209+0000","last_undegraded":"2024-09-16T15:54:56.467209+0000","last_fullsized":"2024-09-16T15:54:56.467209+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:01:03.304745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.464322+0000","last_change":"2024-09-16T15:54:55.461113+0000","last_active":"2024-09-16T15:54:55.464322+0000","last_peered":"2024-09-16T15:54:55.464322+0000","last_clean":"2024-09-16T15:54:55.464322+0000","last_became_active":"2024-09-16T15:54:55.460866+0000","last_became_peered":"2024-09-16T15:54:55.460866+0000","last_unstale":"2024-09-16T15:54:55.464322+0000","last_undegraded":"2024-09-16T15:54:55.464322+0000","last_fullsized":"2024-09-16T15:54:55.464322+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:14:24.286251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482797+0000","last_change":"2024-09-16T15:54:56.491736+0000","last_active":"2024-09-16T15:54:57.482797+0000","last_peered":"2024-09-16T15:54:57.482797+0000","last_clean":"2024-09-16T15:54:57.482797+0000","last_became_active":"2024-09-16T15:54:56.491464+0000","last_became_peered":"2024-09-16T15:54:56.491464+0000","last_unstale":"2024-09-16T15:54:57.482797+0000","last_undegraded":"2024-09-16T15:54:57.482797+0000","last_fullsized":"2024-09-16T15:54:57.482797+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:35:45.264168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.460583+0000","last_change":"2024-09-16T15:54:55.458328+0000","last_active":"2024-09-16T15:54:55.460583+0000","last_peered":"2024-09-16T15:54:55.460583+0000","last_clean":"2024-09-16T15:54:55.460583+0000","last_became_active":"2024-09-16T15:54:55.457453+0000","last_became_peered":"2024-09-16T15:54:55.457453+0000","last_unstale":"2024-09-16T15:54:55.460583+0000","last_undegraded":"2024-09-16T15:54:55.460583+0000","last_fullsized":"2024-09-16T15:54:55.460583+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:16:21.173713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":10,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-16T15:54:56.467102+0000","last_change":"2024-09-16T15:54:56.464797+0000","last_active":"2024-09-16T15:54:56.467102+0000","last_peered":"2024-09-16T15:54:56.467102+0000","last_clean":"2024-09-16T15:54:56.467102+0000","last_became_active":"2024-09-16T15:54:56.463873+0000","last_became_peered":"2024-09-16T15:54:56.463873+0000","last_unstale":"2024-09-16T15:54:56.467102+0000","last_undegraded":"2024-09-16T15:54:56.467102+0000","last_fullsized":"2024-09-16T15:54:56.467102+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:02:11.742600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.467261+0000","last_change":"2024-09-16T15:54:55.463365+0000","last_active":"2024-09-16T15:54:55.467261+0000","last_peered":"2024-09-16T15:54:55.467261+0000","last_clean":"2024-09-16T15:54:55.467261+0000","last_became_active":"2024-09-16T15:54:55.462246+0000","last_became_peered":"2024-09-16T15:54:55.462246+0000","last_unstale":"2024-09-16T15:54:55.467261+0000","last_undegraded":"2024-09-16T15:54:55.467261+0000","last_fullsized":"2024-09-16T15:54:55.467261+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:54:15.667120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.448139+0000","last_change":"2024-09-16T15:54:55.448139+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.448139+0000","last_undegraded":"2024-09-16T15:54:55.448139+0000","last_fullsized":"2024-09-16T15:54:55.448139+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482928+0000","last_change":"2024-09-16T15:54:55.463637+0000","last_active":"2024-09-16T15:54:57.482928+0000","last_peered":"2024-09-16T15:54:57.482928+0000","last_clean":"2024-09-16T15:54:57.482928+0000","last_became_active":"2024-09-16T15:54:55.463289+0000","last_became_peered":"2024-09-16T15:54:55.463289+0000","last_unstale":"2024-09-16T15:54:57.482928+0000","last_undegraded":"2024-09-16T15:54:57.482928+0000","last_fullsized":"2024-09-16T15:54:57.482928+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:16:31.781727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.444870+0000","last_change":"2024-09-16T15:54:55.444870+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.444870+0000","last_undegraded":"2024-09-16T15:54:55.444870+0000","last_fullsized":"2024-09-16T15:54:55.444870+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.463186+0000","last_change":"2024-09-16T15:54:55.459884+0000","last_active":"2024-09-16T15:54:55.463186+0000","last_peered":"2024-09-16T15:54:55.463186+0000","last_clean":"2024-09-16T15:54:55.463186+0000","last_became_active":"2024-09-16T15:54:55.459556+0000","last_became_peered":"2024-09-16T15:54:55.459556+0000","last_unstale":"2024-09-16T15:54:55.463186+0000","last_undegraded":"2024-09-16T15:54:55.463186+0000","last_fullsized":"2024-09-16T15:54:55.463186+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:38:34.406517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482617+0000","last_change":"2024-09-16T15:54:56.485052+0000","last_active":"2024-09-16T15:54:57.482617+0000","last_peered":"2024-09-16T15:54:57.482617+0000","last_clean":"2024-09-16T15:54:57.482617+0000","last_became_active":"2024-09-16T15:54:56.483119+0000","last_became_peered":"2024-09-16T15:54:56.483119+0000","last_unstale":"2024-09-16T15:54:57.482617+0000","last_undegraded":"2024-09-16T15:54:57.482617+0000","last_fullsized":"2024-09-16T15:54:57.482617+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:50:40.992458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.467038+0000","last_change":"2024-09-16T15:54:55.464323+0000","last_active":"2024-09-16T15:54:55.467038+0000","last_peered":"2024-09-16T15:54:55.467038+0000","last_clean":"2024-09-16T15:54:55.467038+0000","last_became_active":"2024-09-16T15:54:55.463480+0000","last_became_peered":"2024-09-16T15:54:55.463480+0000","last_unstale":"2024-09-16T15:54:55.467038+0000","last_undegraded":"2024-09-16T15:54:55.467038+0000","last_fullsized":"2024-09-16T15:54:55.467038+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:39:22.898239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.443764+0000","last_change":"2024-09-16T15:54:55.443764+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.443764+0000","last_undegraded":"2024-09-16T15:54:55.443764+0000","last_fullsized":"2024-09-16T15:54:55.443764+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.463551+0000","last_change":"2024-09-16T15:54:55.459040+0000","last_active":"2024-09-16T15:54:55.463551+0000","last_peered":"2024-09-16T15:54:55.463551+0000","last_clean":"2024-09-16T15:54:55.463551+0000","last_became_active":"2024-09-16T15:54:55.458758+0000","last_became_peered":"2024-09-16T15:54:55.458758+0000","last_unstale":"2024-09-16T15:54:55.463551+0000","last_undegraded":"2024-09-16T15:54:55.463551+0000","last_fullsized":"2024-09-16T15:54:55.463551+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:31:02.873968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482549+0000","last_change":"2024-09-16T15:54:56.485351+0000","last_active":"2024-09-16T15:54:57.482549+0000","last_peered":"2024-09-16T15:54:57.482549+0000","last_clean":"2024-09-16T15:54:57.482549+0000","last_became_active":"2024-09-16T15:54:56.484236+0000","last_became_peered":"2024-09-16T15:54:56.484236+0000","last_unstale":"2024-09-16T15:54:57.482549+0000","last_undegraded":"2024-09-16T15:54:57.482549+0000","last_fullsized":"2024-09-16T15:54:57.482549+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:48:52.418050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.467028+0000","last_change":"2024-09-16T15:54:55.463632+0000","last_active":"2024-09-16T15:54:55.467028+0000","last_peered":"2024-09-16T15:54:55.467028+0000","last_clean":"2024-09-16T15:54:55.467028+0000","last_became_active":"2024-09-16T15:54:55.463347+0000","last_became_peered":"2024-09-16T15:54:55.463347+0000","last_unstale":"2024-09-16T15:54:55.467028+0000","last_undegraded":"2024-09-16T15:54:55.467028+0000","last_fullsized":"2024-09-16T15:54:55.467028+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:47:46.317280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.451273+0000","last_change":"2024-09-16T15:54:55.451273+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.451273+0000","last_undegraded":"2024-09-16T15:54:55.451273+0000","last_fullsized":"2024-09-16T15:54:55.451273+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.464299+0000","last_change":"2024-09-16T15:54:55.459856+0000","last_active":"2024-09-16T15:54:55.464299+0000","last_peered":"2024-09-16T15:54:55.464299+0000","last_clean":"2024-09-16T15:54:55.464299+0000","last_became_active":"2024-09-16T15:54:55.458670+0000","last_became_peered":"2024-09-16T15:54:55.458670+0000","last_unstale":"2024-09-16T15:54:55.464299+0000","last_undegraded":"2024-09-16T15:54:55.464299+0000","last_fullsized":"2024-09-16T15:54:55.464299+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:49:34.676553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.453193+0000","last_change":"2024-09-16T15:54:55.453193+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.453193+0000","last_undegraded":"2024-09-16T15:54:55.453193+0000","last_fullsized":"2024-09-16T15:54:55.453193+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.6","version":"145'1","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482760+0000","last_change":"2024-09-16T15:54:55.464427+0000","last_active":"2024-09-16T15:54:57.482760+0000","last_peered":"2024-09-16T15:54:57.482760+0000","last_clean":"2024-09-16T15:54:57.482760+0000","last_became_active":"2024-09-16T15:54:55.464043+0000","last_became_peered":"2024-09-16T15:54:55.464043+0000","last_unstale":"2024-09-16T15:54:57.482760+0000","last_undegraded":"2024-09-16T15:54:57.482760+0000","last_fullsized":"2024-09-16T15:54:57.482760+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:06:18.925531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.444371+0000","last_change":"2024-09-16T15:54:55.444371+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.444371+0000","last_undegraded":"2024-09-16T15:54:55.444371+0000","last_fullsized":"2024-09-16T15:54:55.444371+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.463400+0000","last_change":"2024-09-16T15:54:55.460656+0000","last_active":"2024-09-16T15:54:55.463400+0000","last_peered":"2024-09-16T15:54:55.463400+0000","last_clean":"2024-09-16T15:54:55.463400+0000","last_became_active":"2024-09-16T15:54:55.460149+0000","last_became_peered":"2024-09-16T15:54:55.460149+0000","last_unstale":"2024-09-16T15:54:55.463400+0000","last_undegraded":"2024-09-16T15:54:55.463400+0000","last_fullsized":"2024-09-16T15:54:55.463400+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:17:46.439566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.445555+0000","last_change":"2024-09-16T15:54:55.445555+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.445555+0000","last_undegraded":"2024-09-16T15:54:55.445555+0000","last_fullsized":"2024-09-16T15:54:55.445555+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.465100+0000","last_change":"2024-09-16T15:54:55.461333+0000","last_active":"2024-09-16T15:54:55.465100+0000","last_peered":"2024-09-16T15:54:55.465100+0000","last_clean":"2024-09-16T15:54:55.465100+0000","last_became_active":"2024-09-16T15:54:55.460016+0000","last_became_peered":"2024-09-16T15:54:55.460016+0000","last_unstale":"2024-09-16T15:54:55.465100+0000","last_undegraded":"2024-09-16T15:54:55.465100+0000","last_fullsized":"2024-09-16T15:54:55.465100+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:07:03.672777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.447708+0000","last_change":"2024-09-16T15:54:55.447708+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.447708+0000","last_undegraded":"2024-09-16T15:54:55.447708+0000","last_fullsized":"2024-09-16T15:54:55.447708+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482633+0000","last_change":"2024-09-16T15:54:55.455582+0000","last_active":"2024-09-16T15:54:57.482633+0000","last_peered":"2024-09-16T15:54:57.482633+0000","last_clean":"2024-09-16T15:54:57.482633+0000","last_became_active":"2024-09-16T15:54:55.453745+0000","last_became_peered":"2024-09-16T15:54:55.453745+0000","last_unstale":"2024-09-16T15:54:57.482633+0000","last_undegraded":"2024-09-16T15:54:57.482633+0000","last_fullsized":"2024-09-16T15:54:57.482633+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:42:00.578523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.448094+0000","last_change":"2024-09-16T15:54:55.448094+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.448094+0000","last_undegraded":"2024-09-16T15:54:55.448094+0000","last_fullsized":"2024-09-16T15:54:55.448094+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.455725+0000","last_change":"2024-09-16T15:54:55.449566+0000","last_active":"2024-09-16T15:54:55.455725+0000","last_peered":"2024-09-16T15:54:55.455725+0000","last_clean":"2024-09-16T15:54:55.455725+0000","last_became_active":"2024-09-16T15:54:55.449318+0000","last_became_peered":"2024-09-16T15:54:55.449318+0000","last_unstale":"2024-09-16T15:54:55.455725+0000","last_undegraded":"2024-09-16T15:54:55.455725+0000","last_fullsized":"2024-09-16T15:54:55.455725+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:38:14.471185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.451007+0000","last_change":"2024-09-16T15:54:55.451007+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.451007+0000","last_undegraded":"2024-09-16T15:54:55.451007+0000","last_fullsized":"2024-09-16T15:54:55.451007+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.462530+0000","last_change":"2024-09-16T15:54:55.461228+0000","last_active":"2024-09-16T15:54:55.462530+0000","last_peered":"2024-09-16T15:54:55.462530+0000","last_clean":"2024-09-16T15:54:55.462530+0000","last_became_active":"2024-09-16T15:54:55.460939+0000","last_became_peered":"2024-09-16T15:54:55.460939+0000","last_unstale":"2024-09-16T15:54:55.462530+0000","last_undegraded":"2024-09-16T15:54:55.462530+0000","last_fullsized":"2024-09-16T15:54:55.462530+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:20:34.830239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.442822+0000","last_change":"2024-09-16T15:54:55.442822+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.442822+0000","last_undegraded":"2024-09-16T15:54:55.442822+0000","last_fullsized":"2024-09-16T15:54:55.442822+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.463143+0000","last_change":"2024-09-16T15:54:55.459516+0000","last_active":"2024-09-16T15:54:55.463143+0000","last_peered":"2024-09-16T15:54:55.463143+0000","last_clean":"2024-09-16T15:54:55.463143+0000","last_became_active":"2024-09-16T15:54:55.459041+0000","last_became_peered":"2024-09-16T15:54:55.459041+0000","last_unstale":"2024-09-16T15:54:55.463143+0000","last_undegraded":"2024-09-16T15:54:55.463143+0000","last_fullsized":"2024-09-16T15:54:55.463143+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:11:04.495951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":1,"reported_epoch":143,"state":"creating+peering","last_fresh":"2024-09-16T15:54:55.449438+0000","last_change":"2024-09-16T15:54:55.449438+0000","last_active":"2024-09-16T15:54:55.432610+0000","last_peered":"2024-09-16T15:54:55.432610+0000","last_clean":"2024-09-16T15:54:55.432610+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T15:54:55.449438+0000","last_undegraded":"2024-09-16T15:54:55.449438+0000","last_fullsized":"2024-09-16T15:54:55.449438+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":10,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-09-16T15:54:55.463059+0000","last_change":"2024-09-16T15:54:55.461430+0000","last_active":"2024-09-16T15:54:55.463059+0000","last_peered":"2024-09-16T15:54:55.463059+0000","last_clean":"2024-09-16T15:54:55.463059+0000","last_became_active":"2024-09-16T15:54:55.461166+0000","last_became_peered":"2024-09-16T15:54:55.461166+0000","last_unstale":"2024-09-16T15:54:55.463059+0000","last_undegraded":"2024-09-16T15:54:55.463059+0000","last_fullsized":"2024-09-16T15:54:55.463059+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:20:58.630650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":343,"reported_epoch":144,"state":"active+clean","last_fresh":"2024-09-16T15:54:56.457213+0000","last_change":"2024-09-16T15:39:12.698813+0000","last_active":"2024-09-16T15:54:56.457213+0000","last_peered":"2024-09-16T15:54:56.457213+0000","last_clean":"2024-09-16T15:54:56.457213+0000","last_became_active":"2024-09-16T15:39:12.697695+0000","last_became_peered":"2024-09-16T15:39:12.697695+0000","last_unstale":"2024-09-16T15:54:56.457213+0000","last_undegraded":"2024-09-16T15:54:56.457213+0000","last_fullsized":"2024-09-16T15:54:56.457213+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_clean_scrub_stamp":"2024-09-16T15:39:11.684967+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:02.061081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509634,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":163348,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":161598,"kb_avail":94208492,"statfs":{"total":96636764160,"available":96469495808,"internally_reserved":0,"allocated":1769472,"data_stored":781042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":165476808},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509633,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123672,"kb_used_data":2308,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248168,"statfs":{"total":96636764160,"available":96510124032,"internally_reserved":0,"allocated":2363392,"data_stored":1371410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":124254663},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509632,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":136996,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94234844,"statfs":{"total":96636764160,"available":96496480256,"internally_reserved":0,"allocated":1781760,"data_stored":787457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":138475973},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509632,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":137000,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94234840,"statfs":{"total":96636764160,"available":96496476160,"internally_reserved":0,"allocated":1781760,"data_stored":787457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":138475977},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509632,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":118952,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94252888,"statfs":{"total":96636764160,"available":96514957312,"internally_reserved":0,"allocated":1781760,"data_stored":787457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":119994823},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509633,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":132004,"kb_used_data":2320,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94239836,"statfs":{"total":96636764160,"available":96501592064,"internally_reserved":0,"allocated":2375680,"data_stored":1377825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509631,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149476,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222364,"statfs":{"total":96636764160,"available":96483700736,"internally_reserved":0,"allocated":1781760,"data_stored":787457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":151255495},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542339,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131452,"kb_used_data":1764,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240388,"statfs":{"total":96636764160,"available":96502157312,"internally_reserved":0,"allocated":1806336,"data_stored":800812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":132774348},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:55:01.498 INFO:tasks.ceph.ceph_manager.ceph:PG 21.3f is not active+clean 2024-09-16T15:55:01.499 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:54:55.445126+0000', 'last_change': '2024-09-16T15:54:55.445126+0000', 'last_active': '2024-09-16T15:54:55.432610+0000', 'last_peered': '2024-09-16T15:54:55.432610+0000', 'last_clean': '2024-09-16T15:54:55.432610+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:54:55.445126+0000', 'last_undegraded': '2024-09-16T15:54:55.445126+0000', 'last_fullsized': '2024-09-16T15:54:55.445126+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_clean_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-16T15:55:01.499 INFO:tasks.ceph.ceph_manager.ceph:PG 21.3d is not active+clean 2024-09-16T15:55:01.499 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:54:55.446549+0000', 'last_change': '2024-09-16T15:54:55.446549+0000', 'last_active': '2024-09-16T15:54:55.432610+0000', 'last_peered': '2024-09-16T15:54:55.432610+0000', 'last_clean': '2024-09-16T15:54:55.432610+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:54:55.446549+0000', 'last_undegraded': '2024-09-16T15:54:55.446549+0000', 'last_fullsized': '2024-09-16T15:54:55.446549+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_clean_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T15:55:01.499 INFO:tasks.ceph.ceph_manager.ceph:PG 21.3c is not active+clean 2024-09-16T15:55:01.499 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:54:55.449901+0000', 'last_change': '2024-09-16T15:54:55.449901+0000', 'last_active': '2024-09-16T15:54:55.432610+0000', 'last_peered': '2024-09-16T15:54:55.432610+0000', 'last_clean': '2024-09-16T15:54:55.432610+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:54:55.449901+0000', 'last_undegraded': '2024-09-16T15:54:55.449901+0000', 'last_fullsized': '2024-09-16T15:54:55.449901+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_clean_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T15:55:01.499 INFO:tasks.ceph.ceph_manager.ceph:PG 21.3a is not active+clean 2024-09-16T15:55:01.499 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:54:55.449521+0000', 'last_change': '2024-09-16T15:54:55.449521+0000', 'last_active': '2024-09-16T15:54:55.432610+0000', 'last_peered': '2024-09-16T15:54:55.432610+0000', 'last_clean': '2024-09-16T15:54:55.432610+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:54:55.449521+0000', 'last_undegraded': '2024-09-16T15:54:55.449521+0000', 'last_fullsized': '2024-09-16T15:54:55.449521+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_clean_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-16T15:55:01.499 INFO:tasks.ceph.ceph_manager.ceph:PG 21.39 is not active+clean 2024-09-16T15:55:01.499 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:54:55.449819+0000', 'last_change': '2024-09-16T15:54:55.449819+0000', 'last_active': '2024-09-16T15:54:55.432610+0000', 'last_peered': '2024-09-16T15:54:55.432610+0000', 'last_clean': '2024-09-16T15:54:55.432610+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:54:55.449819+0000', 'last_undegraded': '2024-09-16T15:54:55.449819+0000', 'last_fullsized': '2024-09-16T15:54:55.449819+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_clean_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-16T15:55:01.499 INFO:tasks.ceph.ceph_manager.ceph:PG 21.38 is not active+clean 2024-09-16T15:55:01.499 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:54:55.443482+0000', 'last_change': '2024-09-16T15:54:55.443482+0000', 'last_active': '2024-09-16T15:54:55.432610+0000', 'last_peered': '2024-09-16T15:54:55.432610+0000', 'last_clean': '2024-09-16T15:54:55.432610+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:54:55.443482+0000', 'last_undegraded': '2024-09-16T15:54:55.443482+0000', 'last_fullsized': '2024-09-16T15:54:55.443482+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_clean_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-16T15:55:01.500 INFO:tasks.ceph.ceph_manager.ceph:PG 21.37 is not active+clean 2024-09-16T15:55:01.500 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:54:55.450377+0000', 'last_change': '2024-09-16T15:54:55.450377+0000', 'last_active': '2024-09-16T15:54:55.432610+0000', 'last_peered': '2024-09-16T15:54:55.432610+0000', 'last_clean': '2024-09-16T15:54:55.432610+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:54:55.450377+0000', 'last_undegraded': '2024-09-16T15:54:55.450377+0000', 'last_fullsized': '2024-09-16T15:54:55.450377+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_clean_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-16T15:55:01.500 INFO:tasks.ceph.ceph_manager.ceph:PG 21.36 is not active+clean 2024-09-16T15:55:01.500 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:54:55.445356+0000', 'last_change': '2024-09-16T15:54:55.445356+0000', 'last_active': '2024-09-16T15:54:55.432610+0000', 'last_peered': '2024-09-16T15:54:55.432610+0000', 'last_clean': '2024-09-16T15:54:55.432610+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:54:55.445356+0000', 'last_undegraded': '2024-09-16T15:54:55.445356+0000', 'last_fullsized': '2024-09-16T15:54:55.445356+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_clean_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T15:55:01.500 INFO:tasks.ceph.ceph_manager.ceph:PG 21.35 is not active+clean 2024-09-16T15:55:01.500 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:54:55.448658+0000', 'last_change': '2024-09-16T15:54:55.448658+0000', 'last_active': '2024-09-16T15:54:55.432610+0000', 'last_peered': '2024-09-16T15:54:55.432610+0000', 'last_clean': '2024-09-16T15:54:55.432610+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:54:55.448658+0000', 'last_undegraded': '2024-09-16T15:54:55.448658+0000', 'last_fullsized': '2024-09-16T15:54:55.448658+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_clean_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-16T15:55:01.500 INFO:tasks.ceph.ceph_manager.ceph:PG 21.34 is not active+clean 2024-09-16T15:55:01.500 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:54:55.445187+0000', 'last_change': '2024-09-16T15:54:55.445187+0000', 'last_active': '2024-09-16T15:54:55.432610+0000', 'last_peered': '2024-09-16T15:54:55.432610+0000', 'last_clean': '2024-09-16T15:54:55.432610+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:54:55.445187+0000', 'last_undegraded': '2024-09-16T15:54:55.445187+0000', 'last_fullsized': '2024-09-16T15:54:55.445187+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_clean_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T15:55:01.500 INFO:tasks.ceph.ceph_manager.ceph:PG 21.32 is not active+clean 2024-09-16T15:55:01.500 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:54:55.447273+0000', 'last_change': '2024-09-16T15:54:55.447273+0000', 'last_active': '2024-09-16T15:54:55.432610+0000', 'last_peered': '2024-09-16T15:54:55.432610+0000', 'last_clean': '2024-09-16T15:54:55.432610+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:54:55.447273+0000', 'last_undegraded': '2024-09-16T15:54:55.447273+0000', 'last_fullsized': '2024-09-16T15:54:55.447273+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_clean_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T15:55:01.500 INFO:tasks.ceph.ceph_manager.ceph:PG 21.30 is not active+clean 2024-09-16T15:55:01.500 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:54:55.447718+0000', 'last_change': '2024-09-16T15:54:55.447718+0000', 'last_active': '2024-09-16T15:54:55.432610+0000', 'last_peered': '2024-09-16T15:54:55.432610+0000', 'last_clean': '2024-09-16T15:54:55.432610+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:54:55.447718+0000', 'last_undegraded': '2024-09-16T15:54:55.447718+0000', 'last_fullsized': '2024-09-16T15:54:55.447718+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_clean_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-16T15:55:01.501 INFO:tasks.ceph.ceph_manager.ceph:PG 21.2f is not active+clean 2024-09-16T15:55:01.501 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:54:55.443888+0000', 'last_change': '2024-09-16T15:54:55.443888+0000', 'last_active': '2024-09-16T15:54:55.432610+0000', 'last_peered': '2024-09-16T15:54:55.432610+0000', 'last_clean': '2024-09-16T15:54:55.432610+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:54:55.443888+0000', 'last_undegraded': '2024-09-16T15:54:55.443888+0000', 'last_fullsized': '2024-09-16T15:54:55.443888+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_clean_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T15:55:01.501 INFO:tasks.ceph.ceph_manager.ceph:PG 21.2d is not active+clean 2024-09-16T15:55:01.501 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:54:55.445058+0000', 'last_change': '2024-09-16T15:54:55.445058+0000', 'last_active': '2024-09-16T15:54:55.432610+0000', 'last_peered': '2024-09-16T15:54:55.432610+0000', 'last_clean': '2024-09-16T15:54:55.432610+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:54:55.445058+0000', 'last_undegraded': '2024-09-16T15:54:55.445058+0000', 'last_fullsized': '2024-09-16T15:54:55.445058+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_clean_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-16T15:55:01.501 INFO:tasks.ceph.ceph_manager.ceph:PG 21.2c is not active+clean 2024-09-16T15:55:01.501 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:54:55.445531+0000', 'last_change': '2024-09-16T15:54:55.445531+0000', 'last_active': '2024-09-16T15:54:55.432610+0000', 'last_peered': '2024-09-16T15:54:55.432610+0000', 'last_clean': '2024-09-16T15:54:55.432610+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:54:55.445531+0000', 'last_undegraded': '2024-09-16T15:54:55.445531+0000', 'last_fullsized': '2024-09-16T15:54:55.445531+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_clean_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-16T15:55:01.501 INFO:tasks.ceph.ceph_manager.ceph:PG 21.2b is not active+clean 2024-09-16T15:55:01.501 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:54:55.442779+0000', 'last_change': '2024-09-16T15:54:55.442779+0000', 'last_active': '2024-09-16T15:54:55.432610+0000', 'last_peered': '2024-09-16T15:54:55.432610+0000', 'last_clean': '2024-09-16T15:54:55.432610+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:54:55.442779+0000', 'last_undegraded': '2024-09-16T15:54:55.442779+0000', 'last_fullsized': '2024-09-16T15:54:55.442779+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_clean_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T15:55:01.501 INFO:tasks.ceph.ceph_manager.ceph:PG 21.2a is not active+clean 2024-09-16T15:55:01.501 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:54:55.446794+0000', 'last_change': '2024-09-16T15:54:55.446794+0000', 'last_active': '2024-09-16T15:54:55.432610+0000', 'last_peered': '2024-09-16T15:54:55.432610+0000', 'last_clean': '2024-09-16T15:54:55.432610+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:54:55.446794+0000', 'last_undegraded': '2024-09-16T15:54:55.446794+0000', 'last_fullsized': '2024-09-16T15:54:55.446794+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_clean_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T15:55:01.501 INFO:tasks.ceph.ceph_manager.ceph:PG 21.29 is not active+clean 2024-09-16T15:55:01.502 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:54:55.449202+0000', 'last_change': '2024-09-16T15:54:55.449202+0000', 'last_active': '2024-09-16T15:54:55.432610+0000', 'last_peered': '2024-09-16T15:54:55.432610+0000', 'last_clean': '2024-09-16T15:54:55.432610+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:54:55.449202+0000', 'last_undegraded': '2024-09-16T15:54:55.449202+0000', 'last_fullsized': '2024-09-16T15:54:55.449202+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_clean_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T15:55:01.502 INFO:tasks.ceph.ceph_manager.ceph:PG 21.26 is not active+clean 2024-09-16T15:55:01.502 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:54:55.445204+0000', 'last_change': '2024-09-16T15:54:55.445204+0000', 'last_active': '2024-09-16T15:54:55.432610+0000', 'last_peered': '2024-09-16T15:54:55.432610+0000', 'last_clean': '2024-09-16T15:54:55.432610+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:54:55.445204+0000', 'last_undegraded': '2024-09-16T15:54:55.445204+0000', 'last_fullsized': '2024-09-16T15:54:55.445204+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_clean_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-16T15:55:01.502 INFO:tasks.ceph.ceph_manager.ceph:PG 21.23 is not active+clean 2024-09-16T15:55:01.502 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:54:55.444909+0000', 'last_change': '2024-09-16T15:54:55.444909+0000', 'last_active': '2024-09-16T15:54:55.432610+0000', 'last_peered': '2024-09-16T15:54:55.432610+0000', 'last_clean': '2024-09-16T15:54:55.432610+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:54:55.444909+0000', 'last_undegraded': '2024-09-16T15:54:55.444909+0000', 'last_fullsized': '2024-09-16T15:54:55.444909+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_clean_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-16T15:55:01.502 INFO:tasks.ceph.ceph_manager.ceph:PG 21.22 is not active+clean 2024-09-16T15:55:01.502 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:54:55.445300+0000', 'last_change': '2024-09-16T15:54:55.445300+0000', 'last_active': '2024-09-16T15:54:55.432610+0000', 'last_peered': '2024-09-16T15:54:55.432610+0000', 'last_clean': '2024-09-16T15:54:55.432610+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:54:55.445300+0000', 'last_undegraded': '2024-09-16T15:54:55.445300+0000', 'last_fullsized': '2024-09-16T15:54:55.445300+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_clean_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T15:55:01.502 INFO:tasks.ceph.ceph_manager.ceph:PG 21.1f is not active+clean 2024-09-16T15:55:01.502 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:54:55.447849+0000', 'last_change': '2024-09-16T15:54:55.447849+0000', 'last_active': '2024-09-16T15:54:55.432610+0000', 'last_peered': '2024-09-16T15:54:55.432610+0000', 'last_clean': '2024-09-16T15:54:55.432610+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:54:55.447849+0000', 'last_undegraded': '2024-09-16T15:54:55.447849+0000', 'last_fullsized': '2024-09-16T15:54:55.447849+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_clean_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-16T15:55:01.502 INFO:tasks.ceph.ceph_manager.ceph:PG 21.1c is not active+clean 2024-09-16T15:55:01.502 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:54:55.450883+0000', 'last_change': '2024-09-16T15:54:55.450883+0000', 'last_active': '2024-09-16T15:54:55.432610+0000', 'last_peered': '2024-09-16T15:54:55.432610+0000', 'last_clean': '2024-09-16T15:54:55.432610+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:54:55.450883+0000', 'last_undegraded': '2024-09-16T15:54:55.450883+0000', 'last_fullsized': '2024-09-16T15:54:55.450883+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_clean_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-16T15:55:01.502 INFO:tasks.ceph.ceph_manager.ceph:PG 21.1d is not active+clean 2024-09-16T15:55:01.503 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:54:55.449829+0000', 'last_change': '2024-09-16T15:54:55.449829+0000', 'last_active': '2024-09-16T15:54:55.432610+0000', 'last_peered': '2024-09-16T15:54:55.432610+0000', 'last_clean': '2024-09-16T15:54:55.432610+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:54:55.449829+0000', 'last_undegraded': '2024-09-16T15:54:55.449829+0000', 'last_fullsized': '2024-09-16T15:54:55.449829+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_clean_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T15:55:01.503 INFO:tasks.ceph.ceph_manager.ceph:PG 21.1b is not active+clean 2024-09-16T15:55:01.503 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:54:55.450288+0000', 'last_change': '2024-09-16T15:54:55.450288+0000', 'last_active': '2024-09-16T15:54:55.432610+0000', 'last_peered': '2024-09-16T15:54:55.432610+0000', 'last_clean': '2024-09-16T15:54:55.432610+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:54:55.450288+0000', 'last_undegraded': '2024-09-16T15:54:55.450288+0000', 'last_fullsized': '2024-09-16T15:54:55.450288+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_clean_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T15:55:01.503 INFO:tasks.ceph.ceph_manager.ceph:PG 21.19 is not active+clean 2024-09-16T15:55:01.503 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:54:55.448700+0000', 'last_change': '2024-09-16T15:54:55.448700+0000', 'last_active': '2024-09-16T15:54:55.432610+0000', 'last_peered': '2024-09-16T15:54:55.432610+0000', 'last_clean': '2024-09-16T15:54:55.432610+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:54:55.448700+0000', 'last_undegraded': '2024-09-16T15:54:55.448700+0000', 'last_fullsized': '2024-09-16T15:54:55.448700+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_clean_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T15:55:01.503 INFO:tasks.ceph.ceph_manager.ceph:PG 21.16 is not active+clean 2024-09-16T15:55:01.503 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:54:55.454247+0000', 'last_change': '2024-09-16T15:54:55.454247+0000', 'last_active': '2024-09-16T15:54:55.432610+0000', 'last_peered': '2024-09-16T15:54:55.432610+0000', 'last_clean': '2024-09-16T15:54:55.432610+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:54:55.454247+0000', 'last_undegraded': '2024-09-16T15:54:55.454247+0000', 'last_fullsized': '2024-09-16T15:54:55.454247+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_clean_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T15:55:01.503 INFO:tasks.ceph.ceph_manager.ceph:PG 21.17 is not active+clean 2024-09-16T15:55:01.503 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:54:55.443768+0000', 'last_change': '2024-09-16T15:54:55.443768+0000', 'last_active': '2024-09-16T15:54:55.432610+0000', 'last_peered': '2024-09-16T15:54:55.432610+0000', 'last_clean': '2024-09-16T15:54:55.432610+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:54:55.443768+0000', 'last_undegraded': '2024-09-16T15:54:55.443768+0000', 'last_fullsized': '2024-09-16T15:54:55.443768+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_clean_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T15:55:01.503 INFO:tasks.ceph.ceph_manager.ceph:PG 21.15 is not active+clean 2024-09-16T15:55:01.503 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:54:55.452496+0000', 'last_change': '2024-09-16T15:54:55.452496+0000', 'last_active': '2024-09-16T15:54:55.432610+0000', 'last_peered': '2024-09-16T15:54:55.432610+0000', 'last_clean': '2024-09-16T15:54:55.432610+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:54:55.452496+0000', 'last_undegraded': '2024-09-16T15:54:55.452496+0000', 'last_fullsized': '2024-09-16T15:54:55.452496+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_clean_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-16T15:55:01.503 INFO:tasks.ceph.ceph_manager.ceph:PG 21.12 is not active+clean 2024-09-16T15:55:01.503 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:54:55.450155+0000', 'last_change': '2024-09-16T15:54:55.450155+0000', 'last_active': '2024-09-16T15:54:55.432610+0000', 'last_peered': '2024-09-16T15:54:55.432610+0000', 'last_clean': '2024-09-16T15:54:55.432610+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:54:55.450155+0000', 'last_undegraded': '2024-09-16T15:54:55.450155+0000', 'last_fullsized': '2024-09-16T15:54:55.450155+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_clean_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T15:55:01.504 INFO:tasks.ceph.ceph_manager.ceph:PG 21.13 is not active+clean 2024-09-16T15:55:01.504 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:54:55.445578+0000', 'last_change': '2024-09-16T15:54:55.445578+0000', 'last_active': '2024-09-16T15:54:55.432610+0000', 'last_peered': '2024-09-16T15:54:55.432610+0000', 'last_clean': '2024-09-16T15:54:55.432610+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:54:55.445578+0000', 'last_undegraded': '2024-09-16T15:54:55.445578+0000', 'last_fullsized': '2024-09-16T15:54:55.445578+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_clean_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-16T15:55:01.504 INFO:tasks.ceph.ceph_manager.ceph:PG 21.10 is not active+clean 2024-09-16T15:55:01.504 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:54:55.453165+0000', 'last_change': '2024-09-16T15:54:55.453165+0000', 'last_active': '2024-09-16T15:54:55.432610+0000', 'last_peered': '2024-09-16T15:54:55.432610+0000', 'last_clean': '2024-09-16T15:54:55.432610+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:54:55.453165+0000', 'last_undegraded': '2024-09-16T15:54:55.453165+0000', 'last_fullsized': '2024-09-16T15:54:55.453165+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_clean_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T15:55:01.504 INFO:tasks.ceph.ceph_manager.ceph:PG 21.11 is not active+clean 2024-09-16T15:55:01.504 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:54:55.448288+0000', 'last_change': '2024-09-16T15:54:55.448288+0000', 'last_active': '2024-09-16T15:54:55.432610+0000', 'last_peered': '2024-09-16T15:54:55.432610+0000', 'last_clean': '2024-09-16T15:54:55.432610+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:54:55.448288+0000', 'last_undegraded': '2024-09-16T15:54:55.448288+0000', 'last_fullsized': '2024-09-16T15:54:55.448288+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_clean_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T15:55:01.504 INFO:tasks.ceph.ceph_manager.ceph:PG 21.d is not active+clean 2024-09-16T15:55:01.504 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:54:55.448139+0000', 'last_change': '2024-09-16T15:54:55.448139+0000', 'last_active': '2024-09-16T15:54:55.432610+0000', 'last_peered': '2024-09-16T15:54:55.432610+0000', 'last_clean': '2024-09-16T15:54:55.432610+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:54:55.448139+0000', 'last_undegraded': '2024-09-16T15:54:55.448139+0000', 'last_fullsized': '2024-09-16T15:54:55.448139+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_clean_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-16T15:55:01.504 INFO:tasks.ceph.ceph_manager.ceph:PG 21.a is not active+clean 2024-09-16T15:55:01.504 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:54:55.444870+0000', 'last_change': '2024-09-16T15:54:55.444870+0000', 'last_active': '2024-09-16T15:54:55.432610+0000', 'last_peered': '2024-09-16T15:54:55.432610+0000', 'last_clean': '2024-09-16T15:54:55.432610+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:54:55.444870+0000', 'last_undegraded': '2024-09-16T15:54:55.444870+0000', 'last_fullsized': '2024-09-16T15:54:55.444870+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_clean_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T15:55:01.505 INFO:tasks.ceph.ceph_manager.ceph:PG 21.8 is not active+clean 2024-09-16T15:55:01.505 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:54:55.443764+0000', 'last_change': '2024-09-16T15:54:55.443764+0000', 'last_active': '2024-09-16T15:54:55.432610+0000', 'last_peered': '2024-09-16T15:54:55.432610+0000', 'last_clean': '2024-09-16T15:54:55.432610+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:54:55.443764+0000', 'last_undegraded': '2024-09-16T15:54:55.443764+0000', 'last_fullsized': '2024-09-16T15:54:55.443764+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_clean_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-16T15:55:01.505 INFO:tasks.ceph.ceph_manager.ceph:PG 21.6 is not active+clean 2024-09-16T15:55:01.505 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:54:55.451273+0000', 'last_change': '2024-09-16T15:54:55.451273+0000', 'last_active': '2024-09-16T15:54:55.432610+0000', 'last_peered': '2024-09-16T15:54:55.432610+0000', 'last_clean': '2024-09-16T15:54:55.432610+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:54:55.451273+0000', 'last_undegraded': '2024-09-16T15:54:55.451273+0000', 'last_fullsized': '2024-09-16T15:54:55.451273+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_clean_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T15:55:01.505 INFO:tasks.ceph.ceph_manager.ceph:PG 21.7 is not active+clean 2024-09-16T15:55:01.505 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:54:55.453193+0000', 'last_change': '2024-09-16T15:54:55.453193+0000', 'last_active': '2024-09-16T15:54:55.432610+0000', 'last_peered': '2024-09-16T15:54:55.432610+0000', 'last_clean': '2024-09-16T15:54:55.432610+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:54:55.453193+0000', 'last_undegraded': '2024-09-16T15:54:55.453193+0000', 'last_fullsized': '2024-09-16T15:54:55.453193+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_clean_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T15:55:01.505 INFO:tasks.ceph.ceph_manager.ceph:PG 21.4 is not active+clean 2024-09-16T15:55:01.505 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:54:55.444371+0000', 'last_change': '2024-09-16T15:54:55.444371+0000', 'last_active': '2024-09-16T15:54:55.432610+0000', 'last_peered': '2024-09-16T15:54:55.432610+0000', 'last_clean': '2024-09-16T15:54:55.432610+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:54:55.444371+0000', 'last_undegraded': '2024-09-16T15:54:55.444371+0000', 'last_fullsized': '2024-09-16T15:54:55.444371+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_clean_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-16T15:55:01.505 INFO:tasks.ceph.ceph_manager.ceph:PG 21.5 is not active+clean 2024-09-16T15:55:01.505 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:54:55.445555+0000', 'last_change': '2024-09-16T15:54:55.445555+0000', 'last_active': '2024-09-16T15:54:55.432610+0000', 'last_peered': '2024-09-16T15:54:55.432610+0000', 'last_clean': '2024-09-16T15:54:55.432610+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:54:55.445555+0000', 'last_undegraded': '2024-09-16T15:54:55.445555+0000', 'last_fullsized': '2024-09-16T15:54:55.445555+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_clean_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-16T15:55:01.505 INFO:tasks.ceph.ceph_manager.ceph:PG 21.2 is not active+clean 2024-09-16T15:55:01.505 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:54:55.447708+0000', 'last_change': '2024-09-16T15:54:55.447708+0000', 'last_active': '2024-09-16T15:54:55.432610+0000', 'last_peered': '2024-09-16T15:54:55.432610+0000', 'last_clean': '2024-09-16T15:54:55.432610+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:54:55.447708+0000', 'last_undegraded': '2024-09-16T15:54:55.447708+0000', 'last_fullsized': '2024-09-16T15:54:55.447708+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_clean_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-16T15:55:01.506 INFO:tasks.ceph.ceph_manager.ceph:PG 21.3 is not active+clean 2024-09-16T15:55:01.506 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:54:55.448094+0000', 'last_change': '2024-09-16T15:54:55.448094+0000', 'last_active': '2024-09-16T15:54:55.432610+0000', 'last_peered': '2024-09-16T15:54:55.432610+0000', 'last_clean': '2024-09-16T15:54:55.432610+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:54:55.448094+0000', 'last_undegraded': '2024-09-16T15:54:55.448094+0000', 'last_fullsized': '2024-09-16T15:54:55.448094+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_clean_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T15:55:01.506 INFO:tasks.ceph.ceph_manager.ceph:PG 21.0 is not active+clean 2024-09-16T15:55:01.506 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:54:55.451007+0000', 'last_change': '2024-09-16T15:54:55.451007+0000', 'last_active': '2024-09-16T15:54:55.432610+0000', 'last_peered': '2024-09-16T15:54:55.432610+0000', 'last_clean': '2024-09-16T15:54:55.432610+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:54:55.451007+0000', 'last_undegraded': '2024-09-16T15:54:55.451007+0000', 'last_fullsized': '2024-09-16T15:54:55.451007+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_clean_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-16T15:55:01.506 INFO:tasks.ceph.ceph_manager.ceph:PG 21.1 is not active+clean 2024-09-16T15:55:01.506 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:54:55.442822+0000', 'last_change': '2024-09-16T15:54:55.442822+0000', 'last_active': '2024-09-16T15:54:55.432610+0000', 'last_peered': '2024-09-16T15:54:55.432610+0000', 'last_clean': '2024-09-16T15:54:55.432610+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:54:55.442822+0000', 'last_undegraded': '2024-09-16T15:54:55.442822+0000', 'last_fullsized': '2024-09-16T15:54:55.442822+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_clean_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-16T15:55:01.506 INFO:tasks.ceph.ceph_manager.ceph:PG 21.14 is not active+clean 2024-09-16T15:55:01.506 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '21.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 143, 'state': 'creating+peering', 'last_fresh': '2024-09-16T15:54:55.449438+0000', 'last_change': '2024-09-16T15:54:55.449438+0000', 'last_active': '2024-09-16T15:54:55.432610+0000', 'last_peered': '2024-09-16T15:54:55.432610+0000', 'last_clean': '2024-09-16T15:54:55.432610+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-16T15:54:55.449438+0000', 'last_undegraded': '2024-09-16T15:54:55.449438+0000', 'last_fullsized': '2024-09-16T15:54:55.449438+0000', 'mapping_epoch': 143, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 143, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'last_clean_scrub_stamp': '2024-09-16T15:54:55.432610+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-16T15:55:01.507 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T15:55:01.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.692+0000 7f4ccaa00640 1 -- 172.21.15.151:0/343613106 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4cc40d25d0 msgr2=0x7f4cc40d4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:01.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.692+0000 7f4ccaa00640 1 --2- 172.21.15.151:0/343613106 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4cc40d25d0 0x7f4cc40d4960 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc005fd0 tx=0x7f4cbc02aae0 comp rx=0 tx=0).stop 2024-09-16T15:55:01.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.692+0000 7f4ccaa00640 1 -- 172.21.15.151:0/343613106 shutdown_connections 2024-09-16T15:55:01.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.692+0000 7f4ccaa00640 1 --2- 172.21.15.151:0/343613106 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4cc40d25d0 0x7f4cc40d4960 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:01.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.692+0000 7f4ccaa00640 1 --2- 172.21.15.151:0/343613106 >> v2:172.21.15.167:3300/0 conn(0x7f4cc415e050 0x7f4cc40cf9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:01.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.692+0000 7f4ccaa00640 1 -- 172.21.15.151:0/343613106 >> 172.21.15.151:0/343613106 conn(0x7f4cc40d3e40 msgr2=0x7f4cc40d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:55:01.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.692+0000 7f4ccaa00640 1 -- 172.21.15.151:0/343613106 shutdown_connections 2024-09-16T15:55:01.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.692+0000 7f4ccaa00640 1 -- 172.21.15.151:0/343613106 wait complete. 2024-09-16T15:55:01.694 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.693+0000 7f4ccaa00640 1 Processor -- start 2024-09-16T15:55:01.694 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.693+0000 7f4ccaa00640 1 -- start start 2024-09-16T15:55:01.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.694+0000 7f4ccaa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4cc40d1260 0x7f4cc40cde10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:01.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.694+0000 7f4ccaa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4cc40d25d0 0x7f4cc41f7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:01.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.694+0000 7f4ccaa00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4cc415e050 0x7f4cc41f8230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:01.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.694+0000 7f4ccaa00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc40b7110 con 0x7f4cc40d25d0 2024-09-16T15:55:01.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.694+0000 7f4ccaa00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc40b6f90 con 0x7f4cc40d1260 2024-09-16T15:55:01.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.694+0000 7f4ccaa00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f4cc40b7290 con 0x7f4cc415e050 2024-09-16T15:55:01.697 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.695+0000 7f4cc8c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4cc40d25d0 0x7f4cc41f7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:01.697 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.695+0000 7f4cc8c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4cc40d25d0 0x7f4cc41f7cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52646/0 (socket says 172.21.15.151:52646) 2024-09-16T15:55:01.697 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.695+0000 7f4cc8c00640 1 -- 172.21.15.151:0/291916412 learned_addr learned my addr 172.21.15.151:0/291916412 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:55:01.697 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.695+0000 7f4cca000640 1 --2- 172.21.15.151:0/291916412 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4cc415e050 0x7f4cc41f8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:01.697 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.695+0000 7f4cc9600640 1 --2- 172.21.15.151:0/291916412 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4cc40d1260 0x7f4cc40cde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:01.697 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.695+0000 7f4cc8c00640 1 -- 172.21.15.151:0/291916412 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4cc415e050 msgr2=0x7f4cc41f8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:01.697 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.695+0000 7f4cc8c00640 1 --2- 172.21.15.151:0/291916412 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4cc415e050 0x7f4cc41f8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:01.697 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.695+0000 7f4cc8c00640 1 -- 172.21.15.151:0/291916412 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4cc40d1260 msgr2=0x7f4cc40cde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:01.697 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.695+0000 7f4cc8c00640 1 --2- 172.21.15.151:0/291916412 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4cc40d1260 0x7f4cc40cde10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:01.697 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.695+0000 7f4cc8c00640 1 -- 172.21.15.151:0/291916412 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cc40cd9d0 con 0x7f4cc40d25d0 2024-09-16T15:55:01.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.696+0000 7f4cc8c00640 1 --2- 172.21.15.151:0/291916412 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4cc40d25d0 0x7f4cc41f7cf0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f4cb000c500 tx=0x7f4cb000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:55:01.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.696+0000 7f4cc1600640 1 -- 172.21.15.151:0/291916412 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb0017070 con 0x7f4cc40d25d0 2024-09-16T15:55:01.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.696+0000 7f4cc1600640 1 -- 172.21.15.151:0/291916412 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4cb00035f0 con 0x7f4cc40d25d0 2024-09-16T15:55:01.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.696+0000 7f4ccaa00640 1 -- 172.21.15.151:0/291916412 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cc40cd130 con 0x7f4cc40d25d0 2024-09-16T15:55:01.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.696+0000 7f4cc1600640 1 -- 172.21.15.151:0/291916412 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb0003a40 con 0x7f4cc40d25d0 2024-09-16T15:55:01.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.696+0000 7f4ccaa00640 1 -- 172.21.15.151:0/291916412 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cc40cd6f0 con 0x7f4cc40d25d0 2024-09-16T15:55:01.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.697+0000 7f4ccaa00640 1 -- 172.21.15.151:0/291916412 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cc41f5ba0 con 0x7f4cc40d25d0 2024-09-16T15:55:01.699 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.698+0000 7f4cc1600640 1 -- 172.21.15.151:0/291916412 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 50) ==== 104398+0+0 (secure 0 0 0) 0x7f4cb001a070 con 0x7f4cc40d25d0 2024-09-16T15:55:01.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.699+0000 7f4cc1600640 1 --2- 172.21.15.151:0/291916412 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4ca407aaa0 0x7f4ca407cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:01.700 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.699+0000 7f4cc1600640 1 -- 172.21.15.151:0/291916412 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5849+0+0 (secure 0 0 0) 0x7f4cb0098f20 con 0x7f4cc40d25d0 2024-09-16T15:55:01.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.700+0000 7f4cc9600640 1 --2- 172.21.15.151:0/291916412 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4ca407aaa0 0x7f4ca407cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:01.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.700+0000 7f4cc9600640 1 --2- 172.21.15.151:0/291916412 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4ca407aaa0 0x7f4ca407cf60 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f4cb8009b30 tx=0x7f4cb8005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:55:01.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.703+0000 7f4cc1600640 1 -- 172.21.15.151:0/291916412 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4cb00629a0 con 0x7f4cc40d25d0 2024-09-16T15:55:01.957 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.956+0000 7f4ccaa00640 1 -- 172.21.15.151:0/291916412 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f4cc40b77b0 con 0x7f4cc40d25d0 2024-09-16T15:55:01.957 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.956+0000 7f4cc1600640 1 -- 172.21.15.151:0/291916412 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+2692 (secure 0 0 0) 0x7f4cb00673d0 con 0x7f4cc40d25d0 2024-09-16T15:55:01.957 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:55:01.958 INFO:teuthology.orchestra.run.smithi151.stdout:{"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":965,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":145,"num_osds":8,"num_up_osds":8,"osd_up_since":1726501150,"num_in_osds":8,"osd_in_since":1726501140,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":52},{"state_name":"creating+peering","count":45}],"num_pgs":97,"num_pools":3,"num_objects":3,"data_bytes":590368,"bytes_used":1119129600,"bytes_avail":771974983680,"bytes_total":773094113280,"inactive_pgs_ratio":0.46391752362251282,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":96,"btime":"2024-09-16T15:54:59:477590+0000","id":10,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":10,"rank":0,"name":"b","status":"up:active","gid":9474}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":20,"modified":"2024-09-16T15:54:37.953302+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"mgr":{"daemons":{"summary":"","x":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"y":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"mon":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"osd":{"daemons":{"summary":"","2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"3":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"4":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"5":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"6":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"7":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-16T15:55:01.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.959+0000 7f4ccaa00640 1 -- 172.21.15.151:0/291916412 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4ca407aaa0 msgr2=0x7f4ca407cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:01.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.959+0000 7f4ccaa00640 1 --2- 172.21.15.151:0/291916412 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4ca407aaa0 0x7f4ca407cf60 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f4cb8009b30 tx=0x7f4cb8005f90 comp rx=0 tx=0).stop 2024-09-16T15:55:01.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.959+0000 7f4ccaa00640 1 -- 172.21.15.151:0/291916412 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4cc40d25d0 msgr2=0x7f4cc41f7cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:01.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.959+0000 7f4ccaa00640 1 --2- 172.21.15.151:0/291916412 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4cc40d25d0 0x7f4cc41f7cf0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f4cb000c500 tx=0x7f4cb000c9c0 comp rx=0 tx=0).stop 2024-09-16T15:55:01.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.959+0000 7f4ccaa00640 1 -- 172.21.15.151:0/291916412 shutdown_connections 2024-09-16T15:55:01.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.959+0000 7f4ccaa00640 1 --2- 172.21.15.151:0/291916412 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f4ca407aaa0 0x7f4ca407cf60 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:01.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.959+0000 7f4ccaa00640 1 --2- 172.21.15.151:0/291916412 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4cc415e050 0x7f4cc41f8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:01.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.959+0000 7f4ccaa00640 1 --2- 172.21.15.151:0/291916412 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4cc40d25d0 0x7f4cc41f7cf0 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:01.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.959+0000 7f4ccaa00640 1 --2- 172.21.15.151:0/291916412 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4cc40d1260 0x7f4cc40cde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:01.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.959+0000 7f4ccaa00640 1 -- 172.21.15.151:0/291916412 >> 172.21.15.151:0/291916412 conn(0x7f4cc40d3e40 msgr2=0x7f4cc40b8c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:55:01.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.959+0000 7f4ccaa00640 1 -- 172.21.15.151:0/291916412 shutdown_connections 2024-09-16T15:55:01.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:01.959+0000 7f4ccaa00640 1 -- 172.21.15.151:0/291916412 wait complete. 2024-09-16T15:55:01.976 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-16T15:55:01.976 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:55:02.135 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:02.133+0000 7f930fe00640 1 -- 172.21.15.151:0/2996540471 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f92f0004800 msgr2=0x7f92f0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:02.135 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:02.133+0000 7f930fe00640 1 --2- 172.21.15.151:0/2996540471 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f92f0004800 0x7f92f0004c80 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f92f400c260 tx=0x7f92f0005830 comp rx=0 tx=0).stop 2024-09-16T15:55:02.135 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:02.133+0000 7f930de00640 1 -- 172.21.15.151:0/2996540471 reap_dead start 2024-09-16T15:55:02.135 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:02.133+0000 7f930fe00640 1 -- 172.21.15.151:0/2996540471 shutdown_connections 2024-09-16T15:55:02.135 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:02.133+0000 7f930fe00640 1 -- 172.21.15.151:0/2996540471 >> 172.21.15.151:0/2996540471 conn(0x7f93080b1950 msgr2=0x7f93080b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:55:02.135 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:02.133+0000 7f930fe00640 1 -- 172.21.15.151:0/2996540471 shutdown_connections 2024-09-16T15:55:02.135 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:02.133+0000 7f930fe00640 1 -- 172.21.15.151:0/2996540471 wait complete. 2024-09-16T15:55:02.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:02.135+0000 7f930fe00640 1 Processor -- start 2024-09-16T15:55:02.136 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:02.135+0000 7f930fe00640 1 -- start start 2024-09-16T15:55:02.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:02.136+0000 7f930fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92f0004800 0x7f93081529a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:02.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:02.136+0000 7f930fe00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f93081f4930 0x7f93081f88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:02.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:02.136+0000 7f930fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f930815a970 0x7f93080afb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:02.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:02.136+0000 7f930fe00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9308074840 con 0x7f930815a970 2024-09-16T15:55:02.137 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:02.136+0000 7f930fe00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f93080746c0 con 0x7f92f0004800 2024-09-16T15:55:02.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:02.136+0000 7f930fe00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f930806f160 con 0x7f93081f4930 2024-09-16T15:55:02.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:02.136+0000 7f930de00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f930815a970 0x7f93080afb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:02.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:02.136+0000 7f930de00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f930815a970 0x7f93080afb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52662/0 (socket says 172.21.15.151:52662) 2024-09-16T15:55:02.138 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:02.136+0000 7f930de00640 1 -- 172.21.15.151:0/1179400117 learned_addr learned my addr 172.21.15.151:0/1179400117 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:55:02.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:02.137+0000 7f930ca00640 1 --2- 172.21.15.151:0/1179400117 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f93081f4930 0x7f93081f88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:02.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:02.137+0000 7f930d400640 1 --2- 172.21.15.151:0/1179400117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92f0004800 0x7f93081529a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:02.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:02.137+0000 7f930de00640 1 -- 172.21.15.151:0/1179400117 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f93081f4930 msgr2=0x7f93081f88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:02.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:02.137+0000 7f930de00640 1 --2- 172.21.15.151:0/1179400117 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f93081f4930 0x7f93081f88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:02.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:02.137+0000 7f930de00640 1 -- 172.21.15.151:0/1179400117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92f0004800 msgr2=0x7f93081529a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:02.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:02.137+0000 7f930de00640 1 --2- 172.21.15.151:0/1179400117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92f0004800 0x7f93081529a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:02.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:02.137+0000 7f930de00640 1 -- 172.21.15.151:0/1179400117 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93081f8e20 con 0x7f930815a970 2024-09-16T15:55:02.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:02.137+0000 7f930d400640 1 --2- 172.21.15.151:0/1179400117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92f0004800 0x7f93081529a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:55:02.139 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:02.137+0000 7f930de00640 1 --2- 172.21.15.151:0/1179400117 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f930815a970 0x7f93080afb70 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f9300005fd0 tx=0x7f930000fa90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:55:02.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:02.137+0000 7f9305600640 1 -- 172.21.15.151:0/1179400117 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9300004230 con 0x7f930815a970 2024-09-16T15:55:02.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:02.137+0000 7f9305600640 1 -- 172.21.15.151:0/1179400117 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9300005020 con 0x7f930815a970 2024-09-16T15:55:02.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:02.137+0000 7f930fe00640 1 -- 172.21.15.151:0/1179400117 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93081f4e60 con 0x7f930815a970 2024-09-16T15:55:02.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:02.137+0000 7f9305600640 1 -- 172.21.15.151:0/1179400117 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9300005470 con 0x7f930815a970 2024-09-16T15:55:02.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:02.137+0000 7f930fe00640 1 -- 172.21.15.151:0/1179400117 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93081f5420 con 0x7f930815a970 2024-09-16T15:55:02.140 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:02.139+0000 7f930fe00640 1 -- 172.21.15.151:0/1179400117 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f930806f6c0 con 0x7f930815a970 2024-09-16T15:55:02.141 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:02.140+0000 7f9305600640 1 -- 172.21.15.151:0/1179400117 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 50) ==== 104398+0+0 (secure 0 0 0) 0x7f9300019070 con 0x7f930815a970 2024-09-16T15:55:02.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:02.141+0000 7f9305600640 1 --2- 172.21.15.151:0/1179400117 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f92e807aaa0 0x7f92e807cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:02.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:02.141+0000 7f9305600640 1 -- 172.21.15.151:0/1179400117 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5849+0+0 (secure 0 0 0) 0x7f9300099c10 con 0x7f930815a970 2024-09-16T15:55:02.142 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:02.141+0000 7f930d400640 1 --2- 172.21.15.151:0/1179400117 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f92e807aaa0 0x7f92e807cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:02.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:02.142+0000 7f930d400640 1 --2- 172.21.15.151:0/1179400117 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f92e807aaa0 0x7f92e807cf60 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f92fc006fd0 tx=0x7f92fc006e40 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:55:02.146 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:02.145+0000 7f9305600640 1 -- 172.21.15.151:0/1179400117 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f9300063690 con 0x7f930815a970 2024-09-16T15:55:02.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:02.325+0000 7f930fe00640 1 -- 172.21.15.151:0/1179400117 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f93080ad120 con 0x7f92e807aaa0 2024-09-16T15:55:02.332 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:02.330+0000 7f9305600640 1 -- 172.21.15.151:0/1179400117 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252485 (secure 0 0 0) 0x7f93080ad120 con 0x7f92e807aaa0 2024-09-16T15:55:02.335 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:55:02.336 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-09-16T15:55:02.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:02.334+0000 7f930fe00640 1 -- 172.21.15.151:0/1179400117 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f92e807aaa0 msgr2=0x7f92e807cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:02.337 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:02.334+0000 7f930fe00640 1 --2- 172.21.15.151:0/1179400117 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f92e807aaa0 0x7f92e807cf60 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f92fc006fd0 tx=0x7f92fc006e40 comp rx=0 tx=0).stop 2024-09-16T15:55:02.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:02.334+0000 7f930fe00640 1 -- 172.21.15.151:0/1179400117 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f930815a970 msgr2=0x7f93080afb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:02.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:02.334+0000 7f930fe00640 1 --2- 172.21.15.151:0/1179400117 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f930815a970 0x7f93080afb70 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f9300005fd0 tx=0x7f930000fa90 comp rx=0 tx=0).stop 2024-09-16T15:55:02.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:02.334+0000 7f930fe00640 1 -- 172.21.15.151:0/1179400117 shutdown_connections 2024-09-16T15:55:02.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:02.334+0000 7f930fe00640 1 --2- 172.21.15.151:0/1179400117 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f92e807aaa0 0x7f92e807cf60 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:02.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:02.334+0000 7f930fe00640 1 --2- 172.21.15.151:0/1179400117 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f930815a970 0x7f93080afb70 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:02.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:02.334+0000 7f930fe00640 1 --2- 172.21.15.151:0/1179400117 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f93081f4930 0x7f93081f88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:02.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:02.334+0000 7f930fe00640 1 --2- 172.21.15.151:0/1179400117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92f0004800 0x7f93081529a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:02.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:02.334+0000 7f930fe00640 1 -- 172.21.15.151:0/1179400117 >> 172.21.15.151:0/1179400117 conn(0x7f93080b1950 msgr2=0x7f93080b0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:55:02.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:02.335+0000 7f930fe00640 1 -- 172.21.15.151:0/1179400117 shutdown_connections 2024-09-16T15:55:02.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:02.335+0000 7f930fe00640 1 -- 172.21.15.151:0/1179400117 wait complete. 2024-09-16T15:55:02.352 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":614,"stamp":"2024-09-16T15:55:01.958616+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1093144,"kb_used_data":15336,"kb_used_omap":12,"kb_used_meta":1077619,"kb_avail":753881576,"statfs":{"total":773094113280,"available":771974733824,"internally_reserved":0,"allocated":15704064,"data_stored":7592052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12738,"internal_metadata":1103482430},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.518422"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:58.637821+0000","last_change":"2024-09-16T15:54:56.472578+0000","last_active":"2024-09-16T15:54:58.637821+0000","last_peered":"2024-09-16T15:54:58.637821+0000","last_clean":"2024-09-16T15:54:58.637821+0000","last_became_active":"2024-09-16T15:54:56.472252+0000","last_became_peered":"2024-09-16T15:54:56.472252+0000","last_unstale":"2024-09-16T15:54:58.637821+0000","last_undegraded":"2024-09-16T15:54:58.637821+0000","last_fullsized":"2024-09-16T15:54:58.637821+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:56:21.875089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482752+0000","last_change":"2024-09-16T15:54:56.489100+0000","last_active":"2024-09-16T15:54:57.482752+0000","last_peered":"2024-09-16T15:54:57.482752+0000","last_clean":"2024-09-16T15:54:57.482752+0000","last_became_active":"2024-09-16T15:54:56.488821+0000","last_became_peered":"2024-09-16T15:54:56.488821+0000","last_unstale":"2024-09-16T15:54:57.482752+0000","last_undegraded":"2024-09-16T15:54:57.482752+0000","last_fullsized":"2024-09-16T15:54:57.482752+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:43:37.723000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:58.636536+0000","last_change":"2024-09-16T15:54:56.476405+0000","last_active":"2024-09-16T15:54:58.636536+0000","last_peered":"2024-09-16T15:54:58.636536+0000","last_clean":"2024-09-16T15:54:58.636536+0000","last_became_active":"2024-09-16T15:54:56.476131+0000","last_became_peered":"2024-09-16T15:54:56.476131+0000","last_unstale":"2024-09-16T15:54:58.636536+0000","last_undegraded":"2024-09-16T15:54:58.636536+0000","last_fullsized":"2024-09-16T15:54:58.636536+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:19:00.583126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.485346+0000","last_change":"2024-09-16T15:54:56.471661+0000","last_active":"2024-09-16T15:54:57.485346+0000","last_peered":"2024-09-16T15:54:57.485346+0000","last_clean":"2024-09-16T15:54:57.485346+0000","last_became_active":"2024-09-16T15:54:56.470781+0000","last_became_peered":"2024-09-16T15:54:56.470781+0000","last_unstale":"2024-09-16T15:54:57.485346+0000","last_undegraded":"2024-09-16T15:54:57.485346+0000","last_fullsized":"2024-09-16T15:54:57.485346+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:24:22.697700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.465954+0000","last_change":"2024-09-16T15:54:56.471793+0000","last_active":"2024-09-16T15:54:57.465954+0000","last_peered":"2024-09-16T15:54:57.465954+0000","last_clean":"2024-09-16T15:54:57.465954+0000","last_became_active":"2024-09-16T15:54:56.471048+0000","last_became_peered":"2024-09-16T15:54:56.471048+0000","last_unstale":"2024-09-16T15:54:57.465954+0000","last_undegraded":"2024-09-16T15:54:57.465954+0000","last_fullsized":"2024-09-16T15:54:57.465954+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:26:58.039927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.467346+0000","last_change":"2024-09-16T15:54:56.471354+0000","last_active":"2024-09-16T15:54:57.467346+0000","last_peered":"2024-09-16T15:54:57.467346+0000","last_clean":"2024-09-16T15:54:57.467346+0000","last_became_active":"2024-09-16T15:54:56.471046+0000","last_became_peered":"2024-09-16T15:54:56.471046+0000","last_unstale":"2024-09-16T15:54:57.467346+0000","last_undegraded":"2024-09-16T15:54:57.467346+0000","last_fullsized":"2024-09-16T15:54:57.467346+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:57:20.690077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.467953+0000","last_change":"2024-09-16T15:54:56.476190+0000","last_active":"2024-09-16T15:54:57.467953+0000","last_peered":"2024-09-16T15:54:57.467953+0000","last_clean":"2024-09-16T15:54:57.467953+0000","last_became_active":"2024-09-16T15:54:56.474458+0000","last_became_peered":"2024-09-16T15:54:56.474458+0000","last_unstale":"2024-09-16T15:54:57.467953+0000","last_undegraded":"2024-09-16T15:54:57.467953+0000","last_fullsized":"2024-09-16T15:54:57.467953+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:26:23.564893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:58.637090+0000","last_change":"2024-09-16T15:54:56.467982+0000","last_active":"2024-09-16T15:54:58.637090+0000","last_peered":"2024-09-16T15:54:58.637090+0000","last_clean":"2024-09-16T15:54:58.637090+0000","last_became_active":"2024-09-16T15:54:56.467673+0000","last_became_peered":"2024-09-16T15:54:56.467673+0000","last_unstale":"2024-09-16T15:54:58.637090+0000","last_undegraded":"2024-09-16T15:54:58.637090+0000","last_fullsized":"2024-09-16T15:54:58.637090+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:02:55.948651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.468351+0000","last_change":"2024-09-16T15:54:56.472125+0000","last_active":"2024-09-16T15:54:57.468351+0000","last_peered":"2024-09-16T15:54:57.468351+0000","last_clean":"2024-09-16T15:54:57.468351+0000","last_became_active":"2024-09-16T15:54:56.471838+0000","last_became_peered":"2024-09-16T15:54:56.471838+0000","last_unstale":"2024-09-16T15:54:57.468351+0000","last_undegraded":"2024-09-16T15:54:57.468351+0000","last_fullsized":"2024-09-16T15:54:57.468351+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:56:02.309346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:58.636387+0000","last_change":"2024-09-16T15:54:56.465955+0000","last_active":"2024-09-16T15:54:58.636387+0000","last_peered":"2024-09-16T15:54:58.636387+0000","last_clean":"2024-09-16T15:54:58.636387+0000","last_became_active":"2024-09-16T15:54:56.465650+0000","last_became_peered":"2024-09-16T15:54:56.465650+0000","last_unstale":"2024-09-16T15:54:58.636387+0000","last_undegraded":"2024-09-16T15:54:58.636387+0000","last_fullsized":"2024-09-16T15:54:58.636387+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:49:24.687125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.467936+0000","last_change":"2024-09-16T15:54:56.471442+0000","last_active":"2024-09-16T15:54:57.467936+0000","last_peered":"2024-09-16T15:54:57.467936+0000","last_clean":"2024-09-16T15:54:57.467936+0000","last_became_active":"2024-09-16T15:54:56.471066+0000","last_became_peered":"2024-09-16T15:54:56.471066+0000","last_unstale":"2024-09-16T15:54:57.467936+0000","last_undegraded":"2024-09-16T15:54:57.467936+0000","last_fullsized":"2024-09-16T15:54:57.467936+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:45:30.837961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:58.637091+0000","last_change":"2024-09-16T15:54:56.464965+0000","last_active":"2024-09-16T15:54:58.637091+0000","last_peered":"2024-09-16T15:54:58.637091+0000","last_clean":"2024-09-16T15:54:58.637091+0000","last_became_active":"2024-09-16T15:54:56.464696+0000","last_became_peered":"2024-09-16T15:54:56.464696+0000","last_unstale":"2024-09-16T15:54:58.637091+0000","last_undegraded":"2024-09-16T15:54:58.637091+0000","last_fullsized":"2024-09-16T15:54:58.637091+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:22:38.823057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.465983+0000","last_change":"2024-09-16T15:54:56.469650+0000","last_active":"2024-09-16T15:54:57.465983+0000","last_peered":"2024-09-16T15:54:57.465983+0000","last_clean":"2024-09-16T15:54:57.465983+0000","last_became_active":"2024-09-16T15:54:56.469407+0000","last_became_peered":"2024-09-16T15:54:56.469407+0000","last_unstale":"2024-09-16T15:54:57.465983+0000","last_undegraded":"2024-09-16T15:54:57.465983+0000","last_fullsized":"2024-09-16T15:54:57.465983+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:53:09.769043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:58.636725+0000","last_change":"2024-09-16T15:54:56.470160+0000","last_active":"2024-09-16T15:54:58.636725+0000","last_peered":"2024-09-16T15:54:58.636725+0000","last_clean":"2024-09-16T15:54:58.636725+0000","last_became_active":"2024-09-16T15:54:56.469853+0000","last_became_peered":"2024-09-16T15:54:56.469853+0000","last_unstale":"2024-09-16T15:54:58.636725+0000","last_undegraded":"2024-09-16T15:54:58.636725+0000","last_fullsized":"2024-09-16T15:54:58.636725+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:01:06.763095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.465820+0000","last_change":"2024-09-16T15:54:56.469340+0000","last_active":"2024-09-16T15:54:57.465820+0000","last_peered":"2024-09-16T15:54:57.465820+0000","last_clean":"2024-09-16T15:54:57.465820+0000","last_became_active":"2024-09-16T15:54:56.469088+0000","last_became_peered":"2024-09-16T15:54:56.469088+0000","last_unstale":"2024-09-16T15:54:57.465820+0000","last_undegraded":"2024-09-16T15:54:57.465820+0000","last_fullsized":"2024-09-16T15:54:57.465820+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:14:25.696051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.467072+0000","last_change":"2024-09-16T15:54:56.468502+0000","last_active":"2024-09-16T15:54:57.467072+0000","last_peered":"2024-09-16T15:54:57.467072+0000","last_clean":"2024-09-16T15:54:57.467072+0000","last_became_active":"2024-09-16T15:54:56.468114+0000","last_became_peered":"2024-09-16T15:54:56.468114+0000","last_unstale":"2024-09-16T15:54:57.467072+0000","last_undegraded":"2024-09-16T15:54:57.467072+0000","last_fullsized":"2024-09-16T15:54:57.467072+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:56:21.875089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:58.636608+0000","last_change":"2024-09-16T15:54:56.472177+0000","last_active":"2024-09-16T15:54:58.636608+0000","last_peered":"2024-09-16T15:54:58.636608+0000","last_clean":"2024-09-16T15:54:58.636608+0000","last_became_active":"2024-09-16T15:54:56.471919+0000","last_became_peered":"2024-09-16T15:54:56.471919+0000","last_unstale":"2024-09-16T15:54:58.636608+0000","last_undegraded":"2024-09-16T15:54:58.636608+0000","last_fullsized":"2024-09-16T15:54:58.636608+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:33:53.169474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.466607+0000","last_change":"2024-09-16T15:54:56.476168+0000","last_active":"2024-09-16T15:54:57.466607+0000","last_peered":"2024-09-16T15:54:57.466607+0000","last_clean":"2024-09-16T15:54:57.466607+0000","last_became_active":"2024-09-16T15:54:56.475925+0000","last_became_peered":"2024-09-16T15:54:56.475925+0000","last_unstale":"2024-09-16T15:54:57.466607+0000","last_undegraded":"2024-09-16T15:54:57.466607+0000","last_fullsized":"2024-09-16T15:54:57.466607+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:59:07.977873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:58.637607+0000","last_change":"2024-09-16T15:54:56.478473+0000","last_active":"2024-09-16T15:54:58.637607+0000","last_peered":"2024-09-16T15:54:58.637607+0000","last_clean":"2024-09-16T15:54:58.637607+0000","last_became_active":"2024-09-16T15:54:56.478180+0000","last_became_peered":"2024-09-16T15:54:56.478180+0000","last_unstale":"2024-09-16T15:54:58.637607+0000","last_undegraded":"2024-09-16T15:54:58.637607+0000","last_fullsized":"2024-09-16T15:54:58.637607+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:02:33.014042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:58.637600+0000","last_change":"2024-09-16T15:54:56.473192+0000","last_active":"2024-09-16T15:54:58.637600+0000","last_peered":"2024-09-16T15:54:58.637600+0000","last_clean":"2024-09-16T15:54:58.637600+0000","last_became_active":"2024-09-16T15:54:56.472904+0000","last_became_peered":"2024-09-16T15:54:56.472904+0000","last_unstale":"2024-09-16T15:54:58.637600+0000","last_undegraded":"2024-09-16T15:54:58.637600+0000","last_fullsized":"2024-09-16T15:54:58.637600+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:25:17.946877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:58.636238+0000","last_change":"2024-09-16T15:54:56.474951+0000","last_active":"2024-09-16T15:54:58.636238+0000","last_peered":"2024-09-16T15:54:58.636238+0000","last_clean":"2024-09-16T15:54:58.636238+0000","last_became_active":"2024-09-16T15:54:56.474696+0000","last_became_peered":"2024-09-16T15:54:56.474696+0000","last_unstale":"2024-09-16T15:54:58.636238+0000","last_undegraded":"2024-09-16T15:54:58.636238+0000","last_fullsized":"2024-09-16T15:54:58.636238+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:04:42.650883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:58.636814+0000","last_change":"2024-09-16T15:54:56.464036+0000","last_active":"2024-09-16T15:54:58.636814+0000","last_peered":"2024-09-16T15:54:58.636814+0000","last_clean":"2024-09-16T15:54:58.636814+0000","last_became_active":"2024-09-16T15:54:56.463740+0000","last_became_peered":"2024-09-16T15:54:56.463740+0000","last_unstale":"2024-09-16T15:54:58.636814+0000","last_undegraded":"2024-09-16T15:54:58.636814+0000","last_fullsized":"2024-09-16T15:54:58.636814+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:25:17.946877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.484907+0000","last_change":"2024-09-16T15:54:56.475214+0000","last_active":"2024-09-16T15:54:57.484907+0000","last_peered":"2024-09-16T15:54:57.484907+0000","last_clean":"2024-09-16T15:54:57.484907+0000","last_became_active":"2024-09-16T15:54:56.474695+0000","last_became_peered":"2024-09-16T15:54:56.474695+0000","last_unstale":"2024-09-16T15:54:57.484907+0000","last_undegraded":"2024-09-16T15:54:57.484907+0000","last_fullsized":"2024-09-16T15:54:57.484907+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:56:16.577884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.465716+0000","last_change":"2024-09-16T15:54:56.462956+0000","last_active":"2024-09-16T15:54:57.465716+0000","last_peered":"2024-09-16T15:54:57.465716+0000","last_clean":"2024-09-16T15:54:57.465716+0000","last_became_active":"2024-09-16T15:54:56.462662+0000","last_became_peered":"2024-09-16T15:54:56.462662+0000","last_unstale":"2024-09-16T15:54:57.465716+0000","last_undegraded":"2024-09-16T15:54:57.465716+0000","last_fullsized":"2024-09-16T15:54:57.465716+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:40:15.566714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482638+0000","last_change":"2024-09-16T15:54:56.481953+0000","last_active":"2024-09-16T15:54:57.482638+0000","last_peered":"2024-09-16T15:54:57.482638+0000","last_clean":"2024-09-16T15:54:57.482638+0000","last_became_active":"2024-09-16T15:54:56.481478+0000","last_became_peered":"2024-09-16T15:54:56.481478+0000","last_unstale":"2024-09-16T15:54:57.482638+0000","last_undegraded":"2024-09-16T15:54:57.482638+0000","last_fullsized":"2024-09-16T15:54:57.482638+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:05:20.399047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:58.637746+0000","last_change":"2024-09-16T15:54:56.473412+0000","last_active":"2024-09-16T15:54:58.637746+0000","last_peered":"2024-09-16T15:54:58.637746+0000","last_clean":"2024-09-16T15:54:58.637746+0000","last_became_active":"2024-09-16T15:54:56.472172+0000","last_became_peered":"2024-09-16T15:54:56.472172+0000","last_unstale":"2024-09-16T15:54:58.637746+0000","last_undegraded":"2024-09-16T15:54:58.637746+0000","last_fullsized":"2024-09-16T15:54:58.637746+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:23:24.896104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482766+0000","last_change":"2024-09-16T15:54:56.481838+0000","last_active":"2024-09-16T15:54:57.482766+0000","last_peered":"2024-09-16T15:54:57.482766+0000","last_clean":"2024-09-16T15:54:57.482766+0000","last_became_active":"2024-09-16T15:54:56.481562+0000","last_became_peered":"2024-09-16T15:54:56.481562+0000","last_unstale":"2024-09-16T15:54:57.482766+0000","last_undegraded":"2024-09-16T15:54:57.482766+0000","last_fullsized":"2024-09-16T15:54:57.482766+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:45:37.323453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482964+0000","last_change":"2024-09-16T15:54:56.490124+0000","last_active":"2024-09-16T15:54:57.482964+0000","last_peered":"2024-09-16T15:54:57.482964+0000","last_clean":"2024-09-16T15:54:57.482964+0000","last_became_active":"2024-09-16T15:54:56.489754+0000","last_became_peered":"2024-09-16T15:54:56.489754+0000","last_unstale":"2024-09-16T15:54:57.482964+0000","last_undegraded":"2024-09-16T15:54:57.482964+0000","last_fullsized":"2024-09-16T15:54:57.482964+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:27:59.188389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:58.637378+0000","last_change":"2024-09-16T15:54:56.467142+0000","last_active":"2024-09-16T15:54:58.637378+0000","last_peered":"2024-09-16T15:54:58.637378+0000","last_clean":"2024-09-16T15:54:58.637378+0000","last_became_active":"2024-09-16T15:54:56.466606+0000","last_became_peered":"2024-09-16T15:54:56.466606+0000","last_unstale":"2024-09-16T15:54:58.637378+0000","last_undegraded":"2024-09-16T15:54:58.637378+0000","last_fullsized":"2024-09-16T15:54:58.637378+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:39:52.118744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:58.637019+0000","last_change":"2024-09-16T15:54:56.462947+0000","last_active":"2024-09-16T15:54:58.637019+0000","last_peered":"2024-09-16T15:54:58.637019+0000","last_clean":"2024-09-16T15:54:58.637019+0000","last_became_active":"2024-09-16T15:54:56.462275+0000","last_became_peered":"2024-09-16T15:54:56.462275+0000","last_unstale":"2024-09-16T15:54:58.637019+0000","last_undegraded":"2024-09-16T15:54:58.637019+0000","last_fullsized":"2024-09-16T15:54:58.637019+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:02:33.014042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482388+0000","last_change":"2024-09-16T15:54:56.487219+0000","last_active":"2024-09-16T15:54:57.482388+0000","last_peered":"2024-09-16T15:54:57.482388+0000","last_clean":"2024-09-16T15:54:57.482388+0000","last_became_active":"2024-09-16T15:54:56.483904+0000","last_became_peered":"2024-09-16T15:54:56.483904+0000","last_unstale":"2024-09-16T15:54:57.482388+0000","last_undegraded":"2024-09-16T15:54:57.482388+0000","last_fullsized":"2024-09-16T15:54:57.482388+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:29:17.110559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482516+0000","last_change":"2024-09-16T15:54:56.477414+0000","last_active":"2024-09-16T15:54:57.482516+0000","last_peered":"2024-09-16T15:54:57.482516+0000","last_clean":"2024-09-16T15:54:57.482516+0000","last_became_active":"2024-09-16T15:54:56.476813+0000","last_became_peered":"2024-09-16T15:54:56.476813+0000","last_unstale":"2024-09-16T15:54:57.482516+0000","last_undegraded":"2024-09-16T15:54:57.482516+0000","last_fullsized":"2024-09-16T15:54:57.482516+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:07:02.927763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.466482+0000","last_change":"2024-09-16T15:54:56.478959+0000","last_active":"2024-09-16T15:54:57.466482+0000","last_peered":"2024-09-16T15:54:57.466482+0000","last_clean":"2024-09-16T15:54:57.466482+0000","last_became_active":"2024-09-16T15:54:56.478054+0000","last_became_peered":"2024-09-16T15:54:56.478054+0000","last_unstale":"2024-09-16T15:54:57.466482+0000","last_undegraded":"2024-09-16T15:54:57.466482+0000","last_fullsized":"2024-09-16T15:54:57.466482+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:59:41.672240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"144'4","reported_seq":19,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:58.636555+0000","last_change":"2024-09-16T15:54:55.458310+0000","last_active":"2024-09-16T15:54:58.636555+0000","last_peered":"2024-09-16T15:54:58.636555+0000","last_clean":"2024-09-16T15:54:58.636555+0000","last_became_active":"2024-09-16T15:54:55.457467+0000","last_became_peered":"2024-09-16T15:54:55.457467+0000","last_unstale":"2024-09-16T15:54:58.636555+0000","last_undegraded":"2024-09-16T15:54:58.636555+0000","last_fullsized":"2024-09-16T15:54:58.636555+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:02:01.796826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.468152+0000","last_change":"2024-09-16T15:54:56.471200+0000","last_active":"2024-09-16T15:54:57.468152+0000","last_peered":"2024-09-16T15:54:57.468152+0000","last_clean":"2024-09-16T15:54:57.468152+0000","last_became_active":"2024-09-16T15:54:56.470909+0000","last_became_peered":"2024-09-16T15:54:56.470909+0000","last_unstale":"2024-09-16T15:54:57.468152+0000","last_undegraded":"2024-09-16T15:54:57.468152+0000","last_fullsized":"2024-09-16T15:54:57.468152+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:19:15.320656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.1e","version":"144'1","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.468281+0000","last_change":"2024-09-16T15:54:55.458458+0000","last_active":"2024-09-16T15:54:57.468281+0000","last_peered":"2024-09-16T15:54:57.468281+0000","last_clean":"2024-09-16T15:54:57.468281+0000","last_became_active":"2024-09-16T15:54:55.457678+0000","last_became_peered":"2024-09-16T15:54:55.457678+0000","last_unstale":"2024-09-16T15:54:57.468281+0000","last_undegraded":"2024-09-16T15:54:57.468281+0000","last_fullsized":"2024-09-16T15:54:57.468281+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:56:15.569916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.467865+0000","last_change":"2024-09-16T15:54:56.472344+0000","last_active":"2024-09-16T15:54:57.467865+0000","last_peered":"2024-09-16T15:54:57.467865+0000","last_clean":"2024-09-16T15:54:57.467865+0000","last_became_active":"2024-09-16T15:54:56.472039+0000","last_became_peered":"2024-09-16T15:54:56.472039+0000","last_unstale":"2024-09-16T15:54:57.467865+0000","last_undegraded":"2024-09-16T15:54:57.467865+0000","last_fullsized":"2024-09-16T15:54:57.467865+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:06:28.751729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.1d","version":"144'2","reported_seq":17,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:58.636477+0000","last_change":"2024-09-16T15:54:55.464149+0000","last_active":"2024-09-16T15:54:58.636477+0000","last_peered":"2024-09-16T15:54:58.636477+0000","last_clean":"2024-09-16T15:54:58.636477+0000","last_became_active":"2024-09-16T15:54:55.463890+0000","last_became_peered":"2024-09-16T15:54:55.463890+0000","last_unstale":"2024-09-16T15:54:58.636477+0000","last_undegraded":"2024-09-16T15:54:58.636477+0000","last_fullsized":"2024-09-16T15:54:58.636477+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:48:51.814287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.485642+0000","last_change":"2024-09-16T15:54:56.473972+0000","last_active":"2024-09-16T15:54:57.485642+0000","last_peered":"2024-09-16T15:54:57.485642+0000","last_clean":"2024-09-16T15:54:57.485642+0000","last_became_active":"2024-09-16T15:54:56.471515+0000","last_became_peered":"2024-09-16T15:54:56.471515+0000","last_unstale":"2024-09-16T15:54:57.485642+0000","last_undegraded":"2024-09-16T15:54:57.485642+0000","last_fullsized":"2024-09-16T15:54:57.485642+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:56:02.309346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1c","version":"144'2","reported_seq":17,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:58.636343+0000","last_change":"2024-09-16T15:54:55.458344+0000","last_active":"2024-09-16T15:54:58.636343+0000","last_peered":"2024-09-16T15:54:58.636343+0000","last_clean":"2024-09-16T15:54:58.636343+0000","last_became_active":"2024-09-16T15:54:55.455322+0000","last_became_peered":"2024-09-16T15:54:55.455322+0000","last_unstale":"2024-09-16T15:54:58.636343+0000","last_undegraded":"2024-09-16T15:54:58.636343+0000","last_fullsized":"2024-09-16T15:54:58.636343+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:15:00.858963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482428+0000","last_change":"2024-09-16T15:54:56.489708+0000","last_active":"2024-09-16T15:54:57.482428+0000","last_peered":"2024-09-16T15:54:57.482428+0000","last_clean":"2024-09-16T15:54:57.482428+0000","last_became_active":"2024-09-16T15:54:56.489545+0000","last_became_peered":"2024-09-16T15:54:56.489545+0000","last_unstale":"2024-09-16T15:54:57.482428+0000","last_undegraded":"2024-09-16T15:54:57.482428+0000","last_fullsized":"2024-09-16T15:54:57.482428+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:59:43.257767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"144'1","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:58.637343+0000","last_change":"2024-09-16T15:54:55.449973+0000","last_active":"2024-09-16T15:54:58.637343+0000","last_peered":"2024-09-16T15:54:58.637343+0000","last_clean":"2024-09-16T15:54:58.637343+0000","last_became_active":"2024-09-16T15:54:55.449676+0000","last_became_peered":"2024-09-16T15:54:55.449676+0000","last_unstale":"2024-09-16T15:54:58.637343+0000","last_undegraded":"2024-09-16T15:54:58.637343+0000","last_fullsized":"2024-09-16T15:54:58.637343+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:51:54.914345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.485452+0000","last_change":"2024-09-16T15:54:56.474604+0000","last_active":"2024-09-16T15:54:57.485452+0000","last_peered":"2024-09-16T15:54:57.485452+0000","last_clean":"2024-09-16T15:54:57.485452+0000","last_became_active":"2024-09-16T15:54:56.474050+0000","last_became_peered":"2024-09-16T15:54:56.474050+0000","last_unstale":"2024-09-16T15:54:57.485452+0000","last_undegraded":"2024-09-16T15:54:57.485452+0000","last_fullsized":"2024-09-16T15:54:57.485452+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:01:06.763095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1a","version":"144'1","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.465848+0000","last_change":"2024-09-16T15:54:55.455815+0000","last_active":"2024-09-16T15:54:57.465848+0000","last_peered":"2024-09-16T15:54:57.465848+0000","last_clean":"2024-09-16T15:54:57.465848+0000","last_became_active":"2024-09-16T15:54:55.455533+0000","last_became_peered":"2024-09-16T15:54:55.455533+0000","last_unstale":"2024-09-16T15:54:57.465848+0000","last_undegraded":"2024-09-16T15:54:57.465848+0000","last_fullsized":"2024-09-16T15:54:57.465848+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:01:12.975683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482536+0000","last_change":"2024-09-16T15:54:56.477672+0000","last_active":"2024-09-16T15:54:57.482536+0000","last_peered":"2024-09-16T15:54:57.482536+0000","last_clean":"2024-09-16T15:54:57.482536+0000","last_became_active":"2024-09-16T15:54:56.477153+0000","last_became_peered":"2024-09-16T15:54:56.477153+0000","last_unstale":"2024-09-16T15:54:57.482536+0000","last_undegraded":"2024-09-16T15:54:57.482536+0000","last_fullsized":"2024-09-16T15:54:57.482536+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:41:22.581512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.467690+0000","last_change":"2024-09-16T15:54:55.455189+0000","last_active":"2024-09-16T15:54:57.467690+0000","last_peered":"2024-09-16T15:54:57.467690+0000","last_clean":"2024-09-16T15:54:57.467690+0000","last_became_active":"2024-09-16T15:54:55.454896+0000","last_became_peered":"2024-09-16T15:54:55.454896+0000","last_unstale":"2024-09-16T15:54:57.467690+0000","last_undegraded":"2024-09-16T15:54:57.467690+0000","last_fullsized":"2024-09-16T15:54:57.467690+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:55:36.203423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.484976+0000","last_change":"2024-09-16T15:54:56.471430+0000","last_active":"2024-09-16T15:54:57.484976+0000","last_peered":"2024-09-16T15:54:57.484976+0000","last_clean":"2024-09-16T15:54:57.484976+0000","last_became_active":"2024-09-16T15:54:56.471186+0000","last_became_peered":"2024-09-16T15:54:56.471186+0000","last_unstale":"2024-09-16T15:54:57.484976+0000","last_undegraded":"2024-09-16T15:54:57.484976+0000","last_fullsized":"2024-09-16T15:54:57.484976+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:12:31.317077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:58.635886+0000","last_change":"2024-09-16T15:54:55.460804+0000","last_active":"2024-09-16T15:54:58.635886+0000","last_peered":"2024-09-16T15:54:58.635886+0000","last_clean":"2024-09-16T15:54:58.635886+0000","last_became_active":"2024-09-16T15:54:55.460502+0000","last_became_peered":"2024-09-16T15:54:55.460502+0000","last_unstale":"2024-09-16T15:54:58.635886+0000","last_undegraded":"2024-09-16T15:54:58.635886+0000","last_fullsized":"2024-09-16T15:54:58.635886+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:34:33.303999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.485470+0000","last_change":"2024-09-16T15:54:56.474594+0000","last_active":"2024-09-16T15:54:57.485470+0000","last_peered":"2024-09-16T15:54:57.485470+0000","last_clean":"2024-09-16T15:54:57.485470+0000","last_became_active":"2024-09-16T15:54:56.474339+0000","last_became_peered":"2024-09-16T15:54:56.474339+0000","last_unstale":"2024-09-16T15:54:57.485470+0000","last_undegraded":"2024-09-16T15:54:57.485470+0000","last_fullsized":"2024-09-16T15:54:57.485470+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:36:59.034643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.468369+0000","last_change":"2024-09-16T15:54:55.460989+0000","last_active":"2024-09-16T15:54:57.468369+0000","last_peered":"2024-09-16T15:54:57.468369+0000","last_clean":"2024-09-16T15:54:57.468369+0000","last_became_active":"2024-09-16T15:54:55.459752+0000","last_became_peered":"2024-09-16T15:54:55.459752+0000","last_unstale":"2024-09-16T15:54:57.468369+0000","last_undegraded":"2024-09-16T15:54:57.468369+0000","last_fullsized":"2024-09-16T15:54:57.468369+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:21:28.811848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:58.637049+0000","last_change":"2024-09-16T15:54:56.469352+0000","last_active":"2024-09-16T15:54:58.637049+0000","last_peered":"2024-09-16T15:54:58.637049+0000","last_clean":"2024-09-16T15:54:58.637049+0000","last_became_active":"2024-09-16T15:54:56.469088+0000","last_became_peered":"2024-09-16T15:54:56.469088+0000","last_unstale":"2024-09-16T15:54:58.637049+0000","last_undegraded":"2024-09-16T15:54:58.637049+0000","last_fullsized":"2024-09-16T15:54:58.637049+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:23:24.896104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.466726+0000","last_change":"2024-09-16T15:54:55.462309+0000","last_active":"2024-09-16T15:54:57.466726+0000","last_peered":"2024-09-16T15:54:57.466726+0000","last_clean":"2024-09-16T15:54:57.466726+0000","last_became_active":"2024-09-16T15:54:55.462039+0000","last_became_peered":"2024-09-16T15:54:55.462039+0000","last_unstale":"2024-09-16T15:54:57.466726+0000","last_undegraded":"2024-09-16T15:54:57.466726+0000","last_fullsized":"2024-09-16T15:54:57.466726+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:32:00.564497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.467536+0000","last_change":"2024-09-16T15:54:56.470284+0000","last_active":"2024-09-16T15:54:57.467536+0000","last_peered":"2024-09-16T15:54:57.467536+0000","last_clean":"2024-09-16T15:54:57.467536+0000","last_became_active":"2024-09-16T15:54:56.470015+0000","last_became_peered":"2024-09-16T15:54:56.470015+0000","last_unstale":"2024-09-16T15:54:57.467536+0000","last_undegraded":"2024-09-16T15:54:57.467536+0000","last_fullsized":"2024-09-16T15:54:57.467536+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:15:20.308450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.14","version":"144'6","reported_seq":21,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.468028+0000","last_change":"2024-09-16T15:54:55.461505+0000","last_active":"2024-09-16T15:54:57.468028+0000","last_peered":"2024-09-16T15:54:57.468028+0000","last_clean":"2024-09-16T15:54:57.468028+0000","last_became_active":"2024-09-16T15:54:55.461174+0000","last_became_peered":"2024-09-16T15:54:55.461174+0000","last_unstale":"2024-09-16T15:54:57.468028+0000","last_undegraded":"2024-09-16T15:54:57.468028+0000","last_fullsized":"2024-09-16T15:54:57.468028+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:02:54.940683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.484971+0000","last_change":"2024-09-16T15:54:56.467821+0000","last_active":"2024-09-16T15:54:57.484971+0000","last_peered":"2024-09-16T15:54:57.484971+0000","last_clean":"2024-09-16T15:54:57.484971+0000","last_became_active":"2024-09-16T15:54:56.467543+0000","last_became_peered":"2024-09-16T15:54:56.467543+0000","last_unstale":"2024-09-16T15:54:57.484971+0000","last_undegraded":"2024-09-16T15:54:57.484971+0000","last_fullsized":"2024-09-16T15:54:57.484971+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:21:29.819815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.13","version":"144'2","reported_seq":17,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:58.636388+0000","last_change":"2024-09-16T15:54:55.462728+0000","last_active":"2024-09-16T15:54:58.636388+0000","last_peered":"2024-09-16T15:54:58.636388+0000","last_clean":"2024-09-16T15:54:58.636388+0000","last_became_active":"2024-09-16T15:54:55.462424+0000","last_became_peered":"2024-09-16T15:54:55.462424+0000","last_unstale":"2024-09-16T15:54:58.636388+0000","last_undegraded":"2024-09-16T15:54:58.636388+0000","last_fullsized":"2024-09-16T15:54:58.636388+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:11:11.408510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:58.637160+0000","last_change":"2024-09-16T15:54:56.466637+0000","last_active":"2024-09-16T15:54:58.637160+0000","last_peered":"2024-09-16T15:54:58.637160+0000","last_clean":"2024-09-16T15:54:58.637160+0000","last_became_active":"2024-09-16T15:54:56.466352+0000","last_became_peered":"2024-09-16T15:54:56.466352+0000","last_unstale":"2024-09-16T15:54:58.637160+0000","last_undegraded":"2024-09-16T15:54:58.637160+0000","last_fullsized":"2024-09-16T15:54:58.637160+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:06:40.951217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:58.637246+0000","last_change":"2024-09-16T15:54:55.455831+0000","last_active":"2024-09-16T15:54:58.637246+0000","last_peered":"2024-09-16T15:54:58.637246+0000","last_clean":"2024-09-16T15:54:58.637246+0000","last_became_active":"2024-09-16T15:54:55.455574+0000","last_became_peered":"2024-09-16T15:54:55.455574+0000","last_unstale":"2024-09-16T15:54:58.637246+0000","last_undegraded":"2024-09-16T15:54:58.637246+0000","last_fullsized":"2024-09-16T15:54:58.637246+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:38:34.406517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.484867+0000","last_change":"2024-09-16T15:54:56.470963+0000","last_active":"2024-09-16T15:54:57.484867+0000","last_peered":"2024-09-16T15:54:57.484867+0000","last_clean":"2024-09-16T15:54:57.484867+0000","last_became_active":"2024-09-16T15:54:56.470705+0000","last_became_peered":"2024-09-16T15:54:56.470705+0000","last_unstale":"2024-09-16T15:54:57.484867+0000","last_undegraded":"2024-09-16T15:54:57.484867+0000","last_fullsized":"2024-09-16T15:54:57.484867+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:54:16.675087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482490+0000","last_change":"2024-09-16T15:54:55.457472+0000","last_active":"2024-09-16T15:54:57.482490+0000","last_peered":"2024-09-16T15:54:57.482490+0000","last_clean":"2024-09-16T15:54:57.482490+0000","last_became_active":"2024-09-16T15:54:55.456471+0000","last_became_peered":"2024-09-16T15:54:55.456471+0000","last_unstale":"2024-09-16T15:54:57.482490+0000","last_undegraded":"2024-09-16T15:54:57.482490+0000","last_fullsized":"2024-09-16T15:54:57.482490+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:14:12.568813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.485149+0000","last_change":"2024-09-16T15:54:56.474937+0000","last_active":"2024-09-16T15:54:57.485149+0000","last_peered":"2024-09-16T15:54:57.485149+0000","last_clean":"2024-09-16T15:54:57.485149+0000","last_became_active":"2024-09-16T15:54:56.474373+0000","last_became_peered":"2024-09-16T15:54:56.474373+0000","last_unstale":"2024-09-16T15:54:57.485149+0000","last_undegraded":"2024-09-16T15:54:57.485149+0000","last_fullsized":"2024-09-16T15:54:57.485149+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:37:31.880953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.10","version":"144'8","reported_seq":23,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:58.637061+0000","last_change":"2024-09-16T15:54:55.461170+0000","last_active":"2024-09-16T15:54:58.637061+0000","last_peered":"2024-09-16T15:54:58.637061+0000","last_clean":"2024-09-16T15:54:58.637061+0000","last_became_active":"2024-09-16T15:54:55.460915+0000","last_became_peered":"2024-09-16T15:54:55.460915+0000","last_unstale":"2024-09-16T15:54:58.637061+0000","last_undegraded":"2024-09-16T15:54:58.637061+0000","last_fullsized":"2024-09-16T15:54:58.637061+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:51:08.351645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.466631+0000","last_change":"2024-09-16T15:54:56.464019+0000","last_active":"2024-09-16T15:54:57.466631+0000","last_peered":"2024-09-16T15:54:57.466631+0000","last_clean":"2024-09-16T15:54:57.466631+0000","last_became_active":"2024-09-16T15:54:56.463741+0000","last_became_peered":"2024-09-16T15:54:56.463741+0000","last_unstale":"2024-09-16T15:54:57.466631+0000","last_undegraded":"2024-09-16T15:54:57.466631+0000","last_fullsized":"2024-09-16T15:54:57.466631+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:53:30.459994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.f","version":"144'1","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.485636+0000","last_change":"2024-09-16T15:54:55.461113+0000","last_active":"2024-09-16T15:54:57.485636+0000","last_peered":"2024-09-16T15:54:57.485636+0000","last_clean":"2024-09-16T15:54:57.485636+0000","last_became_active":"2024-09-16T15:54:55.460866+0000","last_became_peered":"2024-09-16T15:54:55.460866+0000","last_unstale":"2024-09-16T15:54:57.485636+0000","last_undegraded":"2024-09-16T15:54:57.485636+0000","last_fullsized":"2024-09-16T15:54:57.485636+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:28:25.527345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482797+0000","last_change":"2024-09-16T15:54:56.491736+0000","last_active":"2024-09-16T15:54:57.482797+0000","last_peered":"2024-09-16T15:54:57.482797+0000","last_clean":"2024-09-16T15:54:57.482797+0000","last_became_active":"2024-09-16T15:54:56.491464+0000","last_became_peered":"2024-09-16T15:54:56.491464+0000","last_unstale":"2024-09-16T15:54:57.482797+0000","last_undegraded":"2024-09-16T15:54:57.482797+0000","last_fullsized":"2024-09-16T15:54:57.482797+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:35:45.264168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.e","version":"144'2","reported_seq":17,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:58.637646+0000","last_change":"2024-09-16T15:54:55.458328+0000","last_active":"2024-09-16T15:54:58.637646+0000","last_peered":"2024-09-16T15:54:58.637646+0000","last_clean":"2024-09-16T15:54:58.637646+0000","last_became_active":"2024-09-16T15:54:55.457453+0000","last_became_peered":"2024-09-16T15:54:55.457453+0000","last_unstale":"2024-09-16T15:54:58.637646+0000","last_undegraded":"2024-09-16T15:54:58.637646+0000","last_fullsized":"2024-09-16T15:54:58.637646+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:36:05.229515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.466008+0000","last_change":"2024-09-16T15:54:56.464797+0000","last_active":"2024-09-16T15:54:57.466008+0000","last_peered":"2024-09-16T15:54:57.466008+0000","last_clean":"2024-09-16T15:54:57.466008+0000","last_became_active":"2024-09-16T15:54:56.463873+0000","last_became_peered":"2024-09-16T15:54:56.463873+0000","last_unstale":"2024-09-16T15:54:57.466008+0000","last_undegraded":"2024-09-16T15:54:57.466008+0000","last_fullsized":"2024-09-16T15:54:57.466008+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:23:53.577681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.d","version":"144'1","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.468345+0000","last_change":"2024-09-16T15:54:55.463365+0000","last_active":"2024-09-16T15:54:57.468345+0000","last_peered":"2024-09-16T15:54:57.468345+0000","last_clean":"2024-09-16T15:54:57.468345+0000","last_became_active":"2024-09-16T15:54:55.462246+0000","last_became_peered":"2024-09-16T15:54:55.462246+0000","last_unstale":"2024-09-16T15:54:57.468345+0000","last_undegraded":"2024-09-16T15:54:57.468345+0000","last_fullsized":"2024-09-16T15:54:57.468345+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:49:42.884684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.468115+0000","last_change":"2024-09-16T15:54:56.474299+0000","last_active":"2024-09-16T15:54:57.468115+0000","last_peered":"2024-09-16T15:54:57.468115+0000","last_clean":"2024-09-16T15:54:57.468115+0000","last_became_active":"2024-09-16T15:54:56.473995+0000","last_became_peered":"2024-09-16T15:54:56.473995+0000","last_unstale":"2024-09-16T15:54:57.468115+0000","last_undegraded":"2024-09-16T15:54:57.468115+0000","last_fullsized":"2024-09-16T15:54:57.468115+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:12:41.893235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482928+0000","last_change":"2024-09-16T15:54:55.463637+0000","last_active":"2024-09-16T15:54:57.482928+0000","last_peered":"2024-09-16T15:54:57.482928+0000","last_clean":"2024-09-16T15:54:57.482928+0000","last_became_active":"2024-09-16T15:54:55.463289+0000","last_became_peered":"2024-09-16T15:54:55.463289+0000","last_unstale":"2024-09-16T15:54:57.482928+0000","last_undegraded":"2024-09-16T15:54:57.482928+0000","last_fullsized":"2024-09-16T15:54:57.482928+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:16:31.781727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:58.637148+0000","last_change":"2024-09-16T15:54:56.463211+0000","last_active":"2024-09-16T15:54:58.637148+0000","last_peered":"2024-09-16T15:54:58.637148+0000","last_clean":"2024-09-16T15:54:58.637148+0000","last_became_active":"2024-09-16T15:54:56.462693+0000","last_became_peered":"2024-09-16T15:54:56.462693+0000","last_unstale":"2024-09-16T15:54:58.637148+0000","last_undegraded":"2024-09-16T15:54:58.637148+0000","last_fullsized":"2024-09-16T15:54:58.637148+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:56:21.875089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.b","version":"144'2","reported_seq":17,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.467334+0000","last_change":"2024-09-16T15:54:55.459884+0000","last_active":"2024-09-16T15:54:57.467334+0000","last_peered":"2024-09-16T15:54:57.467334+0000","last_clean":"2024-09-16T15:54:57.467334+0000","last_became_active":"2024-09-16T15:54:55.459556+0000","last_became_peered":"2024-09-16T15:54:55.459556+0000","last_unstale":"2024-09-16T15:54:57.467334+0000","last_undegraded":"2024-09-16T15:54:57.467334+0000","last_fullsized":"2024-09-16T15:54:57.467334+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:12:46.866334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482617+0000","last_change":"2024-09-16T15:54:56.485052+0000","last_active":"2024-09-16T15:54:57.482617+0000","last_peered":"2024-09-16T15:54:57.482617+0000","last_clean":"2024-09-16T15:54:57.482617+0000","last_became_active":"2024-09-16T15:54:56.483119+0000","last_became_peered":"2024-09-16T15:54:56.483119+0000","last_unstale":"2024-09-16T15:54:57.482617+0000","last_undegraded":"2024-09-16T15:54:57.482617+0000","last_fullsized":"2024-09-16T15:54:57.482617+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:50:40.992458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.485142+0000","last_change":"2024-09-16T15:54:55.464323+0000","last_active":"2024-09-16T15:54:57.485142+0000","last_peered":"2024-09-16T15:54:57.485142+0000","last_clean":"2024-09-16T15:54:57.485142+0000","last_became_active":"2024-09-16T15:54:55.463480+0000","last_became_peered":"2024-09-16T15:54:55.463480+0000","last_unstale":"2024-09-16T15:54:57.485142+0000","last_undegraded":"2024-09-16T15:54:57.485142+0000","last_fullsized":"2024-09-16T15:54:57.485142+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:52:22.970018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:58.636155+0000","last_change":"2024-09-16T15:54:56.472643+0000","last_active":"2024-09-16T15:54:58.636155+0000","last_peered":"2024-09-16T15:54:58.636155+0000","last_clean":"2024-09-16T15:54:58.636155+0000","last_became_active":"2024-09-16T15:54:56.472222+0000","last_became_peered":"2024-09-16T15:54:56.472222+0000","last_unstale":"2024-09-16T15:54:58.636155+0000","last_undegraded":"2024-09-16T15:54:58.636155+0000","last_fullsized":"2024-09-16T15:54:58.636155+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:39:52.118744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.485127+0000","last_change":"2024-09-16T15:54:55.459040+0000","last_active":"2024-09-16T15:54:57.485127+0000","last_peered":"2024-09-16T15:54:57.485127+0000","last_clean":"2024-09-16T15:54:57.485127+0000","last_became_active":"2024-09-16T15:54:55.458758+0000","last_became_peered":"2024-09-16T15:54:55.458758+0000","last_unstale":"2024-09-16T15:54:57.485127+0000","last_undegraded":"2024-09-16T15:54:57.485127+0000","last_fullsized":"2024-09-16T15:54:57.485127+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:53:40.653455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482549+0000","last_change":"2024-09-16T15:54:56.485351+0000","last_active":"2024-09-16T15:54:57.482549+0000","last_peered":"2024-09-16T15:54:57.482549+0000","last_clean":"2024-09-16T15:54:57.482549+0000","last_became_active":"2024-09-16T15:54:56.484236+0000","last_became_peered":"2024-09-16T15:54:56.484236+0000","last_unstale":"2024-09-16T15:54:57.482549+0000","last_undegraded":"2024-09-16T15:54:57.482549+0000","last_fullsized":"2024-09-16T15:54:57.482549+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:48:52.418050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.485149+0000","last_change":"2024-09-16T15:54:55.463632+0000","last_active":"2024-09-16T15:54:57.485149+0000","last_peered":"2024-09-16T15:54:57.485149+0000","last_clean":"2024-09-16T15:54:57.485149+0000","last_became_active":"2024-09-16T15:54:55.463347+0000","last_became_peered":"2024-09-16T15:54:55.463347+0000","last_unstale":"2024-09-16T15:54:57.485149+0000","last_undegraded":"2024-09-16T15:54:57.485149+0000","last_fullsized":"2024-09-16T15:54:57.485149+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:00:37.618200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.485545+0000","last_change":"2024-09-16T15:54:56.470743+0000","last_active":"2024-09-16T15:54:57.485545+0000","last_peered":"2024-09-16T15:54:57.485545+0000","last_clean":"2024-09-16T15:54:57.485545+0000","last_became_active":"2024-09-16T15:54:56.470476+0000","last_became_peered":"2024-09-16T15:54:56.470476+0000","last_unstale":"2024-09-16T15:54:57.485545+0000","last_undegraded":"2024-09-16T15:54:57.485545+0000","last_fullsized":"2024-09-16T15:54:57.485545+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:12:41.893235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.7","version":"144'2","reported_seq":17,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.485730+0000","last_change":"2024-09-16T15:54:55.459856+0000","last_active":"2024-09-16T15:54:57.485730+0000","last_peered":"2024-09-16T15:54:57.485730+0000","last_clean":"2024-09-16T15:54:57.485730+0000","last_became_active":"2024-09-16T15:54:55.458670+0000","last_became_peered":"2024-09-16T15:54:55.458670+0000","last_unstale":"2024-09-16T15:54:57.485730+0000","last_undegraded":"2024-09-16T15:54:57.485730+0000","last_fullsized":"2024-09-16T15:54:57.485730+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:41:35.010525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.485465+0000","last_change":"2024-09-16T15:54:56.472974+0000","last_active":"2024-09-16T15:54:57.485465+0000","last_peered":"2024-09-16T15:54:57.485465+0000","last_clean":"2024-09-16T15:54:57.485465+0000","last_became_active":"2024-09-16T15:54:56.472712+0000","last_became_peered":"2024-09-16T15:54:56.472712+0000","last_unstale":"2024-09-16T15:54:57.485465+0000","last_undegraded":"2024-09-16T15:54:57.485465+0000","last_fullsized":"2024-09-16T15:54:57.485465+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:57:20.690077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.6","version":"145'1","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482760+0000","last_change":"2024-09-16T15:54:55.464427+0000","last_active":"2024-09-16T15:54:57.482760+0000","last_peered":"2024-09-16T15:54:57.482760+0000","last_clean":"2024-09-16T15:54:57.482760+0000","last_became_active":"2024-09-16T15:54:55.464043+0000","last_became_peered":"2024-09-16T15:54:55.464043+0000","last_unstale":"2024-09-16T15:54:57.482760+0000","last_undegraded":"2024-09-16T15:54:57.482760+0000","last_fullsized":"2024-09-16T15:54:57.482760+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:06:18.925531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:58.637740+0000","last_change":"2024-09-16T15:54:56.467486+0000","last_active":"2024-09-16T15:54:58.637740+0000","last_peered":"2024-09-16T15:54:58.637740+0000","last_clean":"2024-09-16T15:54:58.637740+0000","last_became_active":"2024-09-16T15:54:56.467207+0000","last_became_peered":"2024-09-16T15:54:56.467207+0000","last_unstale":"2024-09-16T15:54:58.637740+0000","last_undegraded":"2024-09-16T15:54:58.637740+0000","last_fullsized":"2024-09-16T15:54:58.637740+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:22:38.823057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.468451+0000","last_change":"2024-09-16T15:54:55.460656+0000","last_active":"2024-09-16T15:54:57.468451+0000","last_peered":"2024-09-16T15:54:57.468451+0000","last_clean":"2024-09-16T15:54:57.468451+0000","last_became_active":"2024-09-16T15:54:55.460149+0000","last_became_peered":"2024-09-16T15:54:55.460149+0000","last_unstale":"2024-09-16T15:54:57.468451+0000","last_undegraded":"2024-09-16T15:54:57.468451+0000","last_fullsized":"2024-09-16T15:54:57.468451+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:31:15.859741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:58.637480+0000","last_change":"2024-09-16T15:54:56.476395+0000","last_active":"2024-09-16T15:54:58.637480+0000","last_peered":"2024-09-16T15:54:58.637480+0000","last_clean":"2024-09-16T15:54:58.637480+0000","last_became_active":"2024-09-16T15:54:56.475195+0000","last_became_peered":"2024-09-16T15:54:56.475195+0000","last_unstale":"2024-09-16T15:54:58.637480+0000","last_undegraded":"2024-09-16T15:54:58.637480+0000","last_fullsized":"2024-09-16T15:54:58.637480+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:01:06.763095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.468176+0000","last_change":"2024-09-16T15:54:55.461333+0000","last_active":"2024-09-16T15:54:57.468176+0000","last_peered":"2024-09-16T15:54:57.468176+0000","last_clean":"2024-09-16T15:54:57.468176+0000","last_became_active":"2024-09-16T15:54:55.460016+0000","last_became_peered":"2024-09-16T15:54:55.460016+0000","last_unstale":"2024-09-16T15:54:57.468176+0000","last_undegraded":"2024-09-16T15:54:57.468176+0000","last_fullsized":"2024-09-16T15:54:57.468176+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:06:39.943250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.467593+0000","last_change":"2024-09-16T15:54:56.463581+0000","last_active":"2024-09-16T15:54:57.467593+0000","last_peered":"2024-09-16T15:54:57.467593+0000","last_clean":"2024-09-16T15:54:57.467593+0000","last_became_active":"2024-09-16T15:54:56.463191+0000","last_became_peered":"2024-09-16T15:54:56.463191+0000","last_unstale":"2024-09-16T15:54:57.467593+0000","last_undegraded":"2024-09-16T15:54:57.467593+0000","last_fullsized":"2024-09-16T15:54:57.467593+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:08:31.123222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482633+0000","last_change":"2024-09-16T15:54:55.455582+0000","last_active":"2024-09-16T15:54:57.482633+0000","last_peered":"2024-09-16T15:54:57.482633+0000","last_clean":"2024-09-16T15:54:57.482633+0000","last_became_active":"2024-09-16T15:54:55.453745+0000","last_became_peered":"2024-09-16T15:54:55.453745+0000","last_unstale":"2024-09-16T15:54:57.482633+0000","last_undegraded":"2024-09-16T15:54:57.482633+0000","last_fullsized":"2024-09-16T15:54:57.482633+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:42:00.578523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.485464+0000","last_change":"2024-09-16T15:54:56.468239+0000","last_active":"2024-09-16T15:54:57.485464+0000","last_peered":"2024-09-16T15:54:57.485464+0000","last_clean":"2024-09-16T15:54:57.485464+0000","last_became_active":"2024-09-16T15:54:56.467883+0000","last_became_peered":"2024-09-16T15:54:56.467883+0000","last_unstale":"2024-09-16T15:54:57.485464+0000","last_undegraded":"2024-09-16T15:54:57.485464+0000","last_fullsized":"2024-09-16T15:54:57.485464+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:02:33.014042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:58.636347+0000","last_change":"2024-09-16T15:54:55.449566+0000","last_active":"2024-09-16T15:54:58.636347+0000","last_peered":"2024-09-16T15:54:58.636347+0000","last_clean":"2024-09-16T15:54:58.636347+0000","last_became_active":"2024-09-16T15:54:55.449318+0000","last_became_peered":"2024-09-16T15:54:55.449318+0000","last_unstale":"2024-09-16T15:54:58.636347+0000","last_undegraded":"2024-09-16T15:54:58.636347+0000","last_fullsized":"2024-09-16T15:54:58.636347+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:58:12.275694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.484867+0000","last_change":"2024-09-16T15:54:56.471555+0000","last_active":"2024-09-16T15:54:57.484867+0000","last_peered":"2024-09-16T15:54:57.484867+0000","last_clean":"2024-09-16T15:54:57.484867+0000","last_became_active":"2024-09-16T15:54:56.471313+0000","last_became_peered":"2024-09-16T15:54:56.471313+0000","last_unstale":"2024-09-16T15:54:57.484867+0000","last_undegraded":"2024-09-16T15:54:57.484867+0000","last_fullsized":"2024-09-16T15:54:57.484867+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:12:47.874301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:58.636058+0000","last_change":"2024-09-16T15:54:55.461228+0000","last_active":"2024-09-16T15:54:58.636058+0000","last_peered":"2024-09-16T15:54:58.636058+0000","last_clean":"2024-09-16T15:54:58.636058+0000","last_became_active":"2024-09-16T15:54:55.460939+0000","last_became_peered":"2024-09-16T15:54:55.460939+0000","last_unstale":"2024-09-16T15:54:58.636058+0000","last_undegraded":"2024-09-16T15:54:58.636058+0000","last_fullsized":"2024-09-16T15:54:58.636058+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:11:04.495951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:58.635913+0000","last_change":"2024-09-16T15:54:56.464762+0000","last_active":"2024-09-16T15:54:58.635913+0000","last_peered":"2024-09-16T15:54:58.635913+0000","last_clean":"2024-09-16T15:54:58.635913+0000","last_became_active":"2024-09-16T15:54:56.464478+0000","last_became_peered":"2024-09-16T15:54:56.464478+0000","last_unstale":"2024-09-16T15:54:58.635913+0000","last_undegraded":"2024-09-16T15:54:58.635913+0000","last_fullsized":"2024-09-16T15:54:58.635913+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:39:31.943265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.467453+0000","last_change":"2024-09-16T15:54:55.459516+0000","last_active":"2024-09-16T15:54:57.467453+0000","last_peered":"2024-09-16T15:54:57.467453+0000","last_clean":"2024-09-16T15:54:57.467453+0000","last_became_active":"2024-09-16T15:54:55.459041+0000","last_became_peered":"2024-09-16T15:54:55.459041+0000","last_unstale":"2024-09-16T15:54:57.467453+0000","last_undegraded":"2024-09-16T15:54:57.467453+0000","last_fullsized":"2024-09-16T15:54:57.467453+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:16:32.668378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.468567+0000","last_change":"2024-09-16T15:54:56.472953+0000","last_active":"2024-09-16T15:54:57.468567+0000","last_peered":"2024-09-16T15:54:57.468567+0000","last_clean":"2024-09-16T15:54:57.468567+0000","last_became_active":"2024-09-16T15:54:56.472646+0000","last_became_peered":"2024-09-16T15:54:56.472646+0000","last_unstale":"2024-09-16T15:54:57.468567+0000","last_undegraded":"2024-09-16T15:54:57.468567+0000","last_fullsized":"2024-09-16T15:54:57.468567+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:07:51.469446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.15","version":"144'3","reported_seq":18,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:58.637066+0000","last_change":"2024-09-16T15:54:55.461430+0000","last_active":"2024-09-16T15:54:58.637066+0000","last_peered":"2024-09-16T15:54:58.637066+0000","last_clean":"2024-09-16T15:54:58.637066+0000","last_became_active":"2024-09-16T15:54:55.461166+0000","last_became_peered":"2024-09-16T15:54:55.461166+0000","last_unstale":"2024-09-16T15:54:58.637066+0000","last_undegraded":"2024-09-16T15:54:58.637066+0000","last_fullsized":"2024-09-16T15:54:58.637066+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:14:12.423767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":345,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.465675+0000","last_change":"2024-09-16T15:39:12.698813+0000","last_active":"2024-09-16T15:54:57.465675+0000","last_peered":"2024-09-16T15:54:57.465675+0000","last_clean":"2024-09-16T15:54:57.465675+0000","last_became_active":"2024-09-16T15:39:12.697695+0000","last_became_peered":"2024-09-16T15:39:12.697695+0000","last_unstale":"2024-09-16T15:54:57.465675+0000","last_undegraded":"2024-09-16T15:54:57.465675+0000","last_fullsized":"2024-09-16T15:54:57.465675+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_clean_scrub_stamp":"2024-09-16T15:39:11.684967+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:02.061081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509635,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":163384,"kb_used_data":1764,"kb_used_omap":1,"kb_used_meta":161598,"kb_avail":94208456,"statfs":{"total":96636764160,"available":96469458944,"internally_reserved":0,"allocated":1806336,"data_stored":800812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":165476808},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509634,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123732,"kb_used_data":2368,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248108,"statfs":{"total":96636764160,"available":96510062592,"internally_reserved":0,"allocated":2424832,"data_stored":1392986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":124254663},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509633,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":137036,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94234804,"statfs":{"total":96636764160,"available":96496439296,"internally_reserved":0,"allocated":1826816,"data_stored":803028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":138475973},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509633,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":137024,"kb_used_data":1772,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94234816,"statfs":{"total":96636764160,"available":96496451584,"internally_reserved":0,"allocated":1814528,"data_stored":801428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":138475977},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509633,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":118976,"kb_used_data":1768,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94252864,"statfs":{"total":96636764160,"available":96514932736,"internally_reserved":0,"allocated":1810432,"data_stored":800834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":119994823},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509634,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":132040,"kb_used_data":2348,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94239800,"statfs":{"total":96636764160,"available":96501555200,"internally_reserved":0,"allocated":2404352,"data_stored":1391250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509632,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149500,"kb_used_data":1768,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222340,"statfs":{"total":96636764160,"available":96483676160,"internally_reserved":0,"allocated":1810432,"data_stored":800902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":151255495},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542339,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131452,"kb_used_data":1764,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240388,"statfs":{"total":96636764160,"available":96502157312,"internally_reserved":0,"allocated":1806336,"data_stored":800812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":132774348},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:55:05.356 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:55:05.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.524+0000 7fd114c00640 1 -- 172.21.15.151:0/1527762576 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd11015f5a0 msgr2=0x7fd1100b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:05.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.524+0000 7fd114c00640 1 --2- 172.21.15.151:0/1527762576 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd11015f5a0 0x7fd1100b2440 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd100005fd0 tx=0x7fd10002a540 comp rx=0 tx=0).stop 2024-09-16T15:55:05.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.525+0000 7fd114c00640 1 -- 172.21.15.151:0/1527762576 shutdown_connections 2024-09-16T15:55:05.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.525+0000 7fd114c00640 1 --2- 172.21.15.151:0/1527762576 >> v2:172.21.15.167:3300/0 conn(0x7fd110073fb0 0x7fd110074460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:05.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.525+0000 7fd114c00640 1 --2- 172.21.15.151:0/1527762576 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd11015f5a0 0x7fd1100b2440 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:05.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.525+0000 7fd114c00640 1 -- 172.21.15.151:0/1527762576 >> 172.21.15.151:0/1527762576 conn(0x7fd1100b1950 msgr2=0x7fd1100b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:55:05.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.525+0000 7fd114c00640 1 -- 172.21.15.151:0/1527762576 shutdown_connections 2024-09-16T15:55:05.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.525+0000 7fd114c00640 1 -- 172.21.15.151:0/1527762576 wait complete. 2024-09-16T15:55:05.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.526+0000 7fd114c00640 1 Processor -- start 2024-09-16T15:55:05.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.526+0000 7fd114c00640 1 -- start start 2024-09-16T15:55:05.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.527+0000 7fd114c00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd110073fb0 0x7fd110150280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:05.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.527+0000 7fd114c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd1100ccf00 0x7fd1100cc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:05.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.527+0000 7fd114c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd11015f5a0 0x7fd1101f5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:05.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.527+0000 7fd114c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd11006f2e0 con 0x7fd1100ccf00 2024-09-16T15:55:05.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.527+0000 7fd114c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd11006f160 con 0x7fd11015f5a0 2024-09-16T15:55:05.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.527+0000 7fd114c00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fd11006f460 con 0x7fd110073fb0 2024-09-16T15:55:05.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.528+0000 7fd10d400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd1100ccf00 0x7fd1100cc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:05.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.528+0000 7fd10de00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd110073fb0 0x7fd110150280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:05.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.528+0000 7fd10de00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd110073fb0 0x7fd110150280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.151:42666/0 (socket says 172.21.15.151:42666) 2024-09-16T15:55:05.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.528+0000 7fd10de00640 1 -- 172.21.15.151:0/2500391220 learned_addr learned my addr 172.21.15.151:0/2500391220 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:55:05.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.528+0000 7fd10e800640 1 --2- 172.21.15.151:0/2500391220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd11015f5a0 0x7fd1101f5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:05.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.528+0000 7fd10de00640 1 -- 172.21.15.151:0/2500391220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd11015f5a0 msgr2=0x7fd1101f5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:05.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.528+0000 7fd10de00640 1 --2- 172.21.15.151:0/2500391220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd11015f5a0 0x7fd1101f5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:05.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.528+0000 7fd10de00640 1 -- 172.21.15.151:0/2500391220 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd1100ccf00 msgr2=0x7fd1100cc5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:05.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.528+0000 7fd10de00640 1 --2- 172.21.15.151:0/2500391220 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd1100ccf00 0x7fd1100cc5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:05.531 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.528+0000 7fd10de00640 1 -- 172.21.15.151:0/2500391220 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd11014e400 con 0x7fd110073fb0 2024-09-16T15:55:05.532 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.529+0000 7fd10de00640 1 --2- 172.21.15.151:0/2500391220 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd110073fb0 0x7fd110150280 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd100034040 tx=0x7fd1000038a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:55:05.532 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.529+0000 7fd106000640 1 -- 172.21.15.151:0/2500391220 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd10003e030 con 0x7fd110073fb0 2024-09-16T15:55:05.532 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.529+0000 7fd106000640 1 -- 172.21.15.151:0/2500391220 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd100003d90 con 0x7fd110073fb0 2024-09-16T15:55:05.532 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.529+0000 7fd106000640 1 -- 172.21.15.151:0/2500391220 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd100035040 con 0x7fd110073fb0 2024-09-16T15:55:05.532 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.529+0000 7fd114c00640 1 -- 172.21.15.151:0/2500391220 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd11014e6e0 con 0x7fd110073fb0 2024-09-16T15:55:05.532 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.529+0000 7fd114c00640 1 -- 172.21.15.151:0/2500391220 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd11014eca0 con 0x7fd110073fb0 2024-09-16T15:55:05.532 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.530+0000 7fd114c00640 1 -- 172.21.15.151:0/2500391220 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd11006f9f0 con 0x7fd110073fb0 2024-09-16T15:55:05.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.531+0000 7fd106000640 1 -- 172.21.15.151:0/2500391220 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 50) ==== 104398+0+0 (secure 0 0 0) 0x7fd100041030 con 0x7fd110073fb0 2024-09-16T15:55:05.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.532+0000 7fd106000640 1 --2- 172.21.15.151:0/2500391220 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd0f007aaa0 0x7fd0f007cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:05.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.532+0000 7fd106000640 1 -- 172.21.15.151:0/2500391220 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(146..146 src has 1..146) ==== 5907+0+0 (secure 0 0 0) 0x7fd1000ba3f0 con 0x7fd110073fb0 2024-09-16T15:55:05.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.533+0000 7fd10d400640 1 --2- 172.21.15.151:0/2500391220 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd0f007aaa0 0x7fd0f007cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:05.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.533+0000 7fd10d400640 1 --2- 172.21.15.151:0/2500391220 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd0f007aaa0 0x7fd0f007cf60 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fd11014fb40 tx=0x7fd0fc009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:55:05.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.537+0000 7fd106000640 1 -- 172.21.15.151:0/2500391220 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd100083e70 con 0x7fd110073fb0 2024-09-16T15:55:05.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.721+0000 7fd114c00640 1 -- 172.21.15.151:0/2500391220 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd1100afc30 con 0x7fd0f007aaa0 2024-09-16T15:55:05.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.726+0000 7fd106000640 1 -- 172.21.15.151:0/2500391220 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252473 (secure 0 0 0) 0x7fd1100afc30 con 0x7fd0f007aaa0 2024-09-16T15:55:05.731 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:55:05.732 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-09-16T15:55:05.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.730+0000 7fd114c00640 1 -- 172.21.15.151:0/2500391220 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd0f007aaa0 msgr2=0x7fd0f007cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:05.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.730+0000 7fd114c00640 1 --2- 172.21.15.151:0/2500391220 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd0f007aaa0 0x7fd0f007cf60 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fd11014fb40 tx=0x7fd0fc009040 comp rx=0 tx=0).stop 2024-09-16T15:55:05.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.730+0000 7fd114c00640 1 -- 172.21.15.151:0/2500391220 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd110073fb0 msgr2=0x7fd110150280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:05.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.730+0000 7fd114c00640 1 --2- 172.21.15.151:0/2500391220 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd110073fb0 0x7fd110150280 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd100034040 tx=0x7fd1000038a0 comp rx=0 tx=0).stop 2024-09-16T15:55:05.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.730+0000 7fd114c00640 1 -- 172.21.15.151:0/2500391220 shutdown_connections 2024-09-16T15:55:05.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.730+0000 7fd114c00640 1 --2- 172.21.15.151:0/2500391220 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd0f007aaa0 0x7fd0f007cf60 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:05.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.730+0000 7fd114c00640 1 --2- 172.21.15.151:0/2500391220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd11015f5a0 0x7fd1101f5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:05.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.730+0000 7fd114c00640 1 --2- 172.21.15.151:0/2500391220 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd1100ccf00 0x7fd1100cc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:05.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.730+0000 7fd114c00640 1 --2- 172.21.15.151:0/2500391220 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd110073fb0 0x7fd110150280 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:05.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.730+0000 7fd114c00640 1 -- 172.21.15.151:0/2500391220 >> 172.21.15.151:0/2500391220 conn(0x7fd1100b1950 msgr2=0x7fd11009c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:55:05.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.730+0000 7fd114c00640 1 -- 172.21.15.151:0/2500391220 shutdown_connections 2024-09-16T15:55:05.734 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.730+0000 7fd114c00640 1 -- 172.21.15.151:0/2500391220 wait complete. 2024-09-16T15:55:05.748 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":616,"stamp":"2024-09-16T15:55:05.507334+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1093144,"kb_used_data":15336,"kb_used_omap":12,"kb_used_meta":1077619,"kb_avail":753881576,"statfs":{"total":773094113280,"available":771974733824,"internally_reserved":0,"allocated":15704064,"data_stored":7592052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12738,"internal_metadata":1103482430},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.053278"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:58.637821+0000","last_change":"2024-09-16T15:54:56.472578+0000","last_active":"2024-09-16T15:54:58.637821+0000","last_peered":"2024-09-16T15:54:58.637821+0000","last_clean":"2024-09-16T15:54:58.637821+0000","last_became_active":"2024-09-16T15:54:56.472252+0000","last_became_peered":"2024-09-16T15:54:56.472252+0000","last_unstale":"2024-09-16T15:54:58.637821+0000","last_undegraded":"2024-09-16T15:54:58.637821+0000","last_fullsized":"2024-09-16T15:54:58.637821+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:56:21.875089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482752+0000","last_change":"2024-09-16T15:54:56.489100+0000","last_active":"2024-09-16T15:54:57.482752+0000","last_peered":"2024-09-16T15:54:57.482752+0000","last_clean":"2024-09-16T15:54:57.482752+0000","last_became_active":"2024-09-16T15:54:56.488821+0000","last_became_peered":"2024-09-16T15:54:56.488821+0000","last_unstale":"2024-09-16T15:54:57.482752+0000","last_undegraded":"2024-09-16T15:54:57.482752+0000","last_fullsized":"2024-09-16T15:54:57.482752+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:43:37.723000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:58.636536+0000","last_change":"2024-09-16T15:54:56.476405+0000","last_active":"2024-09-16T15:54:58.636536+0000","last_peered":"2024-09-16T15:54:58.636536+0000","last_clean":"2024-09-16T15:54:58.636536+0000","last_became_active":"2024-09-16T15:54:56.476131+0000","last_became_peered":"2024-09-16T15:54:56.476131+0000","last_unstale":"2024-09-16T15:54:58.636536+0000","last_undegraded":"2024-09-16T15:54:58.636536+0000","last_fullsized":"2024-09-16T15:54:58.636536+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:19:00.583126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.485346+0000","last_change":"2024-09-16T15:54:56.471661+0000","last_active":"2024-09-16T15:54:57.485346+0000","last_peered":"2024-09-16T15:54:57.485346+0000","last_clean":"2024-09-16T15:54:57.485346+0000","last_became_active":"2024-09-16T15:54:56.470781+0000","last_became_peered":"2024-09-16T15:54:56.470781+0000","last_unstale":"2024-09-16T15:54:57.485346+0000","last_undegraded":"2024-09-16T15:54:57.485346+0000","last_fullsized":"2024-09-16T15:54:57.485346+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:24:22.697700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.465954+0000","last_change":"2024-09-16T15:54:56.471793+0000","last_active":"2024-09-16T15:54:57.465954+0000","last_peered":"2024-09-16T15:54:57.465954+0000","last_clean":"2024-09-16T15:54:57.465954+0000","last_became_active":"2024-09-16T15:54:56.471048+0000","last_became_peered":"2024-09-16T15:54:56.471048+0000","last_unstale":"2024-09-16T15:54:57.465954+0000","last_undegraded":"2024-09-16T15:54:57.465954+0000","last_fullsized":"2024-09-16T15:54:57.465954+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:26:58.039927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.467346+0000","last_change":"2024-09-16T15:54:56.471354+0000","last_active":"2024-09-16T15:54:57.467346+0000","last_peered":"2024-09-16T15:54:57.467346+0000","last_clean":"2024-09-16T15:54:57.467346+0000","last_became_active":"2024-09-16T15:54:56.471046+0000","last_became_peered":"2024-09-16T15:54:56.471046+0000","last_unstale":"2024-09-16T15:54:57.467346+0000","last_undegraded":"2024-09-16T15:54:57.467346+0000","last_fullsized":"2024-09-16T15:54:57.467346+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:57:20.690077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.467953+0000","last_change":"2024-09-16T15:54:56.476190+0000","last_active":"2024-09-16T15:54:57.467953+0000","last_peered":"2024-09-16T15:54:57.467953+0000","last_clean":"2024-09-16T15:54:57.467953+0000","last_became_active":"2024-09-16T15:54:56.474458+0000","last_became_peered":"2024-09-16T15:54:56.474458+0000","last_unstale":"2024-09-16T15:54:57.467953+0000","last_undegraded":"2024-09-16T15:54:57.467953+0000","last_fullsized":"2024-09-16T15:54:57.467953+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:26:23.564893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:58.637090+0000","last_change":"2024-09-16T15:54:56.467982+0000","last_active":"2024-09-16T15:54:58.637090+0000","last_peered":"2024-09-16T15:54:58.637090+0000","last_clean":"2024-09-16T15:54:58.637090+0000","last_became_active":"2024-09-16T15:54:56.467673+0000","last_became_peered":"2024-09-16T15:54:56.467673+0000","last_unstale":"2024-09-16T15:54:58.637090+0000","last_undegraded":"2024-09-16T15:54:58.637090+0000","last_fullsized":"2024-09-16T15:54:58.637090+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:02:55.948651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.468351+0000","last_change":"2024-09-16T15:54:56.472125+0000","last_active":"2024-09-16T15:54:57.468351+0000","last_peered":"2024-09-16T15:54:57.468351+0000","last_clean":"2024-09-16T15:54:57.468351+0000","last_became_active":"2024-09-16T15:54:56.471838+0000","last_became_peered":"2024-09-16T15:54:56.471838+0000","last_unstale":"2024-09-16T15:54:57.468351+0000","last_undegraded":"2024-09-16T15:54:57.468351+0000","last_fullsized":"2024-09-16T15:54:57.468351+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:56:02.309346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:58.636387+0000","last_change":"2024-09-16T15:54:56.465955+0000","last_active":"2024-09-16T15:54:58.636387+0000","last_peered":"2024-09-16T15:54:58.636387+0000","last_clean":"2024-09-16T15:54:58.636387+0000","last_became_active":"2024-09-16T15:54:56.465650+0000","last_became_peered":"2024-09-16T15:54:56.465650+0000","last_unstale":"2024-09-16T15:54:58.636387+0000","last_undegraded":"2024-09-16T15:54:58.636387+0000","last_fullsized":"2024-09-16T15:54:58.636387+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:49:24.687125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.467936+0000","last_change":"2024-09-16T15:54:56.471442+0000","last_active":"2024-09-16T15:54:57.467936+0000","last_peered":"2024-09-16T15:54:57.467936+0000","last_clean":"2024-09-16T15:54:57.467936+0000","last_became_active":"2024-09-16T15:54:56.471066+0000","last_became_peered":"2024-09-16T15:54:56.471066+0000","last_unstale":"2024-09-16T15:54:57.467936+0000","last_undegraded":"2024-09-16T15:54:57.467936+0000","last_fullsized":"2024-09-16T15:54:57.467936+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:45:30.837961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:58.637091+0000","last_change":"2024-09-16T15:54:56.464965+0000","last_active":"2024-09-16T15:54:58.637091+0000","last_peered":"2024-09-16T15:54:58.637091+0000","last_clean":"2024-09-16T15:54:58.637091+0000","last_became_active":"2024-09-16T15:54:56.464696+0000","last_became_peered":"2024-09-16T15:54:56.464696+0000","last_unstale":"2024-09-16T15:54:58.637091+0000","last_undegraded":"2024-09-16T15:54:58.637091+0000","last_fullsized":"2024-09-16T15:54:58.637091+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:22:38.823057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.465983+0000","last_change":"2024-09-16T15:54:56.469650+0000","last_active":"2024-09-16T15:54:57.465983+0000","last_peered":"2024-09-16T15:54:57.465983+0000","last_clean":"2024-09-16T15:54:57.465983+0000","last_became_active":"2024-09-16T15:54:56.469407+0000","last_became_peered":"2024-09-16T15:54:56.469407+0000","last_unstale":"2024-09-16T15:54:57.465983+0000","last_undegraded":"2024-09-16T15:54:57.465983+0000","last_fullsized":"2024-09-16T15:54:57.465983+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:53:09.769043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:58.636725+0000","last_change":"2024-09-16T15:54:56.470160+0000","last_active":"2024-09-16T15:54:58.636725+0000","last_peered":"2024-09-16T15:54:58.636725+0000","last_clean":"2024-09-16T15:54:58.636725+0000","last_became_active":"2024-09-16T15:54:56.469853+0000","last_became_peered":"2024-09-16T15:54:56.469853+0000","last_unstale":"2024-09-16T15:54:58.636725+0000","last_undegraded":"2024-09-16T15:54:58.636725+0000","last_fullsized":"2024-09-16T15:54:58.636725+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:01:06.763095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.465820+0000","last_change":"2024-09-16T15:54:56.469340+0000","last_active":"2024-09-16T15:54:57.465820+0000","last_peered":"2024-09-16T15:54:57.465820+0000","last_clean":"2024-09-16T15:54:57.465820+0000","last_became_active":"2024-09-16T15:54:56.469088+0000","last_became_peered":"2024-09-16T15:54:56.469088+0000","last_unstale":"2024-09-16T15:54:57.465820+0000","last_undegraded":"2024-09-16T15:54:57.465820+0000","last_fullsized":"2024-09-16T15:54:57.465820+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:14:25.696051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.467072+0000","last_change":"2024-09-16T15:54:56.468502+0000","last_active":"2024-09-16T15:54:57.467072+0000","last_peered":"2024-09-16T15:54:57.467072+0000","last_clean":"2024-09-16T15:54:57.467072+0000","last_became_active":"2024-09-16T15:54:56.468114+0000","last_became_peered":"2024-09-16T15:54:56.468114+0000","last_unstale":"2024-09-16T15:54:57.467072+0000","last_undegraded":"2024-09-16T15:54:57.467072+0000","last_fullsized":"2024-09-16T15:54:57.467072+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:56:21.875089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:58.636608+0000","last_change":"2024-09-16T15:54:56.472177+0000","last_active":"2024-09-16T15:54:58.636608+0000","last_peered":"2024-09-16T15:54:58.636608+0000","last_clean":"2024-09-16T15:54:58.636608+0000","last_became_active":"2024-09-16T15:54:56.471919+0000","last_became_peered":"2024-09-16T15:54:56.471919+0000","last_unstale":"2024-09-16T15:54:58.636608+0000","last_undegraded":"2024-09-16T15:54:58.636608+0000","last_fullsized":"2024-09-16T15:54:58.636608+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:33:53.169474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.466607+0000","last_change":"2024-09-16T15:54:56.476168+0000","last_active":"2024-09-16T15:54:57.466607+0000","last_peered":"2024-09-16T15:54:57.466607+0000","last_clean":"2024-09-16T15:54:57.466607+0000","last_became_active":"2024-09-16T15:54:56.475925+0000","last_became_peered":"2024-09-16T15:54:56.475925+0000","last_unstale":"2024-09-16T15:54:57.466607+0000","last_undegraded":"2024-09-16T15:54:57.466607+0000","last_fullsized":"2024-09-16T15:54:57.466607+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:59:07.977873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:58.637607+0000","last_change":"2024-09-16T15:54:56.478473+0000","last_active":"2024-09-16T15:54:58.637607+0000","last_peered":"2024-09-16T15:54:58.637607+0000","last_clean":"2024-09-16T15:54:58.637607+0000","last_became_active":"2024-09-16T15:54:56.478180+0000","last_became_peered":"2024-09-16T15:54:56.478180+0000","last_unstale":"2024-09-16T15:54:58.637607+0000","last_undegraded":"2024-09-16T15:54:58.637607+0000","last_fullsized":"2024-09-16T15:54:58.637607+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:02:33.014042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:58.637600+0000","last_change":"2024-09-16T15:54:56.473192+0000","last_active":"2024-09-16T15:54:58.637600+0000","last_peered":"2024-09-16T15:54:58.637600+0000","last_clean":"2024-09-16T15:54:58.637600+0000","last_became_active":"2024-09-16T15:54:56.472904+0000","last_became_peered":"2024-09-16T15:54:56.472904+0000","last_unstale":"2024-09-16T15:54:58.637600+0000","last_undegraded":"2024-09-16T15:54:58.637600+0000","last_fullsized":"2024-09-16T15:54:58.637600+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:25:17.946877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:58.636238+0000","last_change":"2024-09-16T15:54:56.474951+0000","last_active":"2024-09-16T15:54:58.636238+0000","last_peered":"2024-09-16T15:54:58.636238+0000","last_clean":"2024-09-16T15:54:58.636238+0000","last_became_active":"2024-09-16T15:54:56.474696+0000","last_became_peered":"2024-09-16T15:54:56.474696+0000","last_unstale":"2024-09-16T15:54:58.636238+0000","last_undegraded":"2024-09-16T15:54:58.636238+0000","last_fullsized":"2024-09-16T15:54:58.636238+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:04:42.650883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:58.636814+0000","last_change":"2024-09-16T15:54:56.464036+0000","last_active":"2024-09-16T15:54:58.636814+0000","last_peered":"2024-09-16T15:54:58.636814+0000","last_clean":"2024-09-16T15:54:58.636814+0000","last_became_active":"2024-09-16T15:54:56.463740+0000","last_became_peered":"2024-09-16T15:54:56.463740+0000","last_unstale":"2024-09-16T15:54:58.636814+0000","last_undegraded":"2024-09-16T15:54:58.636814+0000","last_fullsized":"2024-09-16T15:54:58.636814+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:25:17.946877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.484907+0000","last_change":"2024-09-16T15:54:56.475214+0000","last_active":"2024-09-16T15:54:57.484907+0000","last_peered":"2024-09-16T15:54:57.484907+0000","last_clean":"2024-09-16T15:54:57.484907+0000","last_became_active":"2024-09-16T15:54:56.474695+0000","last_became_peered":"2024-09-16T15:54:56.474695+0000","last_unstale":"2024-09-16T15:54:57.484907+0000","last_undegraded":"2024-09-16T15:54:57.484907+0000","last_fullsized":"2024-09-16T15:54:57.484907+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:56:16.577884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.465716+0000","last_change":"2024-09-16T15:54:56.462956+0000","last_active":"2024-09-16T15:54:57.465716+0000","last_peered":"2024-09-16T15:54:57.465716+0000","last_clean":"2024-09-16T15:54:57.465716+0000","last_became_active":"2024-09-16T15:54:56.462662+0000","last_became_peered":"2024-09-16T15:54:56.462662+0000","last_unstale":"2024-09-16T15:54:57.465716+0000","last_undegraded":"2024-09-16T15:54:57.465716+0000","last_fullsized":"2024-09-16T15:54:57.465716+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:40:15.566714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482638+0000","last_change":"2024-09-16T15:54:56.481953+0000","last_active":"2024-09-16T15:54:57.482638+0000","last_peered":"2024-09-16T15:54:57.482638+0000","last_clean":"2024-09-16T15:54:57.482638+0000","last_became_active":"2024-09-16T15:54:56.481478+0000","last_became_peered":"2024-09-16T15:54:56.481478+0000","last_unstale":"2024-09-16T15:54:57.482638+0000","last_undegraded":"2024-09-16T15:54:57.482638+0000","last_fullsized":"2024-09-16T15:54:57.482638+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:05:20.399047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:58.637746+0000","last_change":"2024-09-16T15:54:56.473412+0000","last_active":"2024-09-16T15:54:58.637746+0000","last_peered":"2024-09-16T15:54:58.637746+0000","last_clean":"2024-09-16T15:54:58.637746+0000","last_became_active":"2024-09-16T15:54:56.472172+0000","last_became_peered":"2024-09-16T15:54:56.472172+0000","last_unstale":"2024-09-16T15:54:58.637746+0000","last_undegraded":"2024-09-16T15:54:58.637746+0000","last_fullsized":"2024-09-16T15:54:58.637746+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:23:24.896104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482766+0000","last_change":"2024-09-16T15:54:56.481838+0000","last_active":"2024-09-16T15:54:57.482766+0000","last_peered":"2024-09-16T15:54:57.482766+0000","last_clean":"2024-09-16T15:54:57.482766+0000","last_became_active":"2024-09-16T15:54:56.481562+0000","last_became_peered":"2024-09-16T15:54:56.481562+0000","last_unstale":"2024-09-16T15:54:57.482766+0000","last_undegraded":"2024-09-16T15:54:57.482766+0000","last_fullsized":"2024-09-16T15:54:57.482766+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:45:37.323453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482964+0000","last_change":"2024-09-16T15:54:56.490124+0000","last_active":"2024-09-16T15:54:57.482964+0000","last_peered":"2024-09-16T15:54:57.482964+0000","last_clean":"2024-09-16T15:54:57.482964+0000","last_became_active":"2024-09-16T15:54:56.489754+0000","last_became_peered":"2024-09-16T15:54:56.489754+0000","last_unstale":"2024-09-16T15:54:57.482964+0000","last_undegraded":"2024-09-16T15:54:57.482964+0000","last_fullsized":"2024-09-16T15:54:57.482964+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:27:59.188389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:58.637378+0000","last_change":"2024-09-16T15:54:56.467142+0000","last_active":"2024-09-16T15:54:58.637378+0000","last_peered":"2024-09-16T15:54:58.637378+0000","last_clean":"2024-09-16T15:54:58.637378+0000","last_became_active":"2024-09-16T15:54:56.466606+0000","last_became_peered":"2024-09-16T15:54:56.466606+0000","last_unstale":"2024-09-16T15:54:58.637378+0000","last_undegraded":"2024-09-16T15:54:58.637378+0000","last_fullsized":"2024-09-16T15:54:58.637378+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:39:52.118744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:58.637019+0000","last_change":"2024-09-16T15:54:56.462947+0000","last_active":"2024-09-16T15:54:58.637019+0000","last_peered":"2024-09-16T15:54:58.637019+0000","last_clean":"2024-09-16T15:54:58.637019+0000","last_became_active":"2024-09-16T15:54:56.462275+0000","last_became_peered":"2024-09-16T15:54:56.462275+0000","last_unstale":"2024-09-16T15:54:58.637019+0000","last_undegraded":"2024-09-16T15:54:58.637019+0000","last_fullsized":"2024-09-16T15:54:58.637019+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:02:33.014042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482388+0000","last_change":"2024-09-16T15:54:56.487219+0000","last_active":"2024-09-16T15:54:57.482388+0000","last_peered":"2024-09-16T15:54:57.482388+0000","last_clean":"2024-09-16T15:54:57.482388+0000","last_became_active":"2024-09-16T15:54:56.483904+0000","last_became_peered":"2024-09-16T15:54:56.483904+0000","last_unstale":"2024-09-16T15:54:57.482388+0000","last_undegraded":"2024-09-16T15:54:57.482388+0000","last_fullsized":"2024-09-16T15:54:57.482388+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:29:17.110559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482516+0000","last_change":"2024-09-16T15:54:56.477414+0000","last_active":"2024-09-16T15:54:57.482516+0000","last_peered":"2024-09-16T15:54:57.482516+0000","last_clean":"2024-09-16T15:54:57.482516+0000","last_became_active":"2024-09-16T15:54:56.476813+0000","last_became_peered":"2024-09-16T15:54:56.476813+0000","last_unstale":"2024-09-16T15:54:57.482516+0000","last_undegraded":"2024-09-16T15:54:57.482516+0000","last_fullsized":"2024-09-16T15:54:57.482516+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:07:02.927763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.466482+0000","last_change":"2024-09-16T15:54:56.478959+0000","last_active":"2024-09-16T15:54:57.466482+0000","last_peered":"2024-09-16T15:54:57.466482+0000","last_clean":"2024-09-16T15:54:57.466482+0000","last_became_active":"2024-09-16T15:54:56.478054+0000","last_became_peered":"2024-09-16T15:54:56.478054+0000","last_unstale":"2024-09-16T15:54:57.466482+0000","last_undegraded":"2024-09-16T15:54:57.466482+0000","last_fullsized":"2024-09-16T15:54:57.466482+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:59:41.672240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"144'4","reported_seq":19,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:58.636555+0000","last_change":"2024-09-16T15:54:55.458310+0000","last_active":"2024-09-16T15:54:58.636555+0000","last_peered":"2024-09-16T15:54:58.636555+0000","last_clean":"2024-09-16T15:54:58.636555+0000","last_became_active":"2024-09-16T15:54:55.457467+0000","last_became_peered":"2024-09-16T15:54:55.457467+0000","last_unstale":"2024-09-16T15:54:58.636555+0000","last_undegraded":"2024-09-16T15:54:58.636555+0000","last_fullsized":"2024-09-16T15:54:58.636555+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:02:01.796826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.468152+0000","last_change":"2024-09-16T15:54:56.471200+0000","last_active":"2024-09-16T15:54:57.468152+0000","last_peered":"2024-09-16T15:54:57.468152+0000","last_clean":"2024-09-16T15:54:57.468152+0000","last_became_active":"2024-09-16T15:54:56.470909+0000","last_became_peered":"2024-09-16T15:54:56.470909+0000","last_unstale":"2024-09-16T15:54:57.468152+0000","last_undegraded":"2024-09-16T15:54:57.468152+0000","last_fullsized":"2024-09-16T15:54:57.468152+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:19:15.320656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.1e","version":"144'1","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.468281+0000","last_change":"2024-09-16T15:54:55.458458+0000","last_active":"2024-09-16T15:54:57.468281+0000","last_peered":"2024-09-16T15:54:57.468281+0000","last_clean":"2024-09-16T15:54:57.468281+0000","last_became_active":"2024-09-16T15:54:55.457678+0000","last_became_peered":"2024-09-16T15:54:55.457678+0000","last_unstale":"2024-09-16T15:54:57.468281+0000","last_undegraded":"2024-09-16T15:54:57.468281+0000","last_fullsized":"2024-09-16T15:54:57.468281+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:56:15.569916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.467865+0000","last_change":"2024-09-16T15:54:56.472344+0000","last_active":"2024-09-16T15:54:57.467865+0000","last_peered":"2024-09-16T15:54:57.467865+0000","last_clean":"2024-09-16T15:54:57.467865+0000","last_became_active":"2024-09-16T15:54:56.472039+0000","last_became_peered":"2024-09-16T15:54:56.472039+0000","last_unstale":"2024-09-16T15:54:57.467865+0000","last_undegraded":"2024-09-16T15:54:57.467865+0000","last_fullsized":"2024-09-16T15:54:57.467865+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:06:28.751729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.1d","version":"144'2","reported_seq":17,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:58.636477+0000","last_change":"2024-09-16T15:54:55.464149+0000","last_active":"2024-09-16T15:54:58.636477+0000","last_peered":"2024-09-16T15:54:58.636477+0000","last_clean":"2024-09-16T15:54:58.636477+0000","last_became_active":"2024-09-16T15:54:55.463890+0000","last_became_peered":"2024-09-16T15:54:55.463890+0000","last_unstale":"2024-09-16T15:54:58.636477+0000","last_undegraded":"2024-09-16T15:54:58.636477+0000","last_fullsized":"2024-09-16T15:54:58.636477+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:48:51.814287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.485642+0000","last_change":"2024-09-16T15:54:56.473972+0000","last_active":"2024-09-16T15:54:57.485642+0000","last_peered":"2024-09-16T15:54:57.485642+0000","last_clean":"2024-09-16T15:54:57.485642+0000","last_became_active":"2024-09-16T15:54:56.471515+0000","last_became_peered":"2024-09-16T15:54:56.471515+0000","last_unstale":"2024-09-16T15:54:57.485642+0000","last_undegraded":"2024-09-16T15:54:57.485642+0000","last_fullsized":"2024-09-16T15:54:57.485642+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:56:02.309346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1c","version":"144'2","reported_seq":17,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:58.636343+0000","last_change":"2024-09-16T15:54:55.458344+0000","last_active":"2024-09-16T15:54:58.636343+0000","last_peered":"2024-09-16T15:54:58.636343+0000","last_clean":"2024-09-16T15:54:58.636343+0000","last_became_active":"2024-09-16T15:54:55.455322+0000","last_became_peered":"2024-09-16T15:54:55.455322+0000","last_unstale":"2024-09-16T15:54:58.636343+0000","last_undegraded":"2024-09-16T15:54:58.636343+0000","last_fullsized":"2024-09-16T15:54:58.636343+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:15:00.858963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482428+0000","last_change":"2024-09-16T15:54:56.489708+0000","last_active":"2024-09-16T15:54:57.482428+0000","last_peered":"2024-09-16T15:54:57.482428+0000","last_clean":"2024-09-16T15:54:57.482428+0000","last_became_active":"2024-09-16T15:54:56.489545+0000","last_became_peered":"2024-09-16T15:54:56.489545+0000","last_unstale":"2024-09-16T15:54:57.482428+0000","last_undegraded":"2024-09-16T15:54:57.482428+0000","last_fullsized":"2024-09-16T15:54:57.482428+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:59:43.257767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"144'1","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:58.637343+0000","last_change":"2024-09-16T15:54:55.449973+0000","last_active":"2024-09-16T15:54:58.637343+0000","last_peered":"2024-09-16T15:54:58.637343+0000","last_clean":"2024-09-16T15:54:58.637343+0000","last_became_active":"2024-09-16T15:54:55.449676+0000","last_became_peered":"2024-09-16T15:54:55.449676+0000","last_unstale":"2024-09-16T15:54:58.637343+0000","last_undegraded":"2024-09-16T15:54:58.637343+0000","last_fullsized":"2024-09-16T15:54:58.637343+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:51:54.914345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.485452+0000","last_change":"2024-09-16T15:54:56.474604+0000","last_active":"2024-09-16T15:54:57.485452+0000","last_peered":"2024-09-16T15:54:57.485452+0000","last_clean":"2024-09-16T15:54:57.485452+0000","last_became_active":"2024-09-16T15:54:56.474050+0000","last_became_peered":"2024-09-16T15:54:56.474050+0000","last_unstale":"2024-09-16T15:54:57.485452+0000","last_undegraded":"2024-09-16T15:54:57.485452+0000","last_fullsized":"2024-09-16T15:54:57.485452+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:01:06.763095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1a","version":"144'1","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.465848+0000","last_change":"2024-09-16T15:54:55.455815+0000","last_active":"2024-09-16T15:54:57.465848+0000","last_peered":"2024-09-16T15:54:57.465848+0000","last_clean":"2024-09-16T15:54:57.465848+0000","last_became_active":"2024-09-16T15:54:55.455533+0000","last_became_peered":"2024-09-16T15:54:55.455533+0000","last_unstale":"2024-09-16T15:54:57.465848+0000","last_undegraded":"2024-09-16T15:54:57.465848+0000","last_fullsized":"2024-09-16T15:54:57.465848+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:01:12.975683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482536+0000","last_change":"2024-09-16T15:54:56.477672+0000","last_active":"2024-09-16T15:54:57.482536+0000","last_peered":"2024-09-16T15:54:57.482536+0000","last_clean":"2024-09-16T15:54:57.482536+0000","last_became_active":"2024-09-16T15:54:56.477153+0000","last_became_peered":"2024-09-16T15:54:56.477153+0000","last_unstale":"2024-09-16T15:54:57.482536+0000","last_undegraded":"2024-09-16T15:54:57.482536+0000","last_fullsized":"2024-09-16T15:54:57.482536+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:41:22.581512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.467690+0000","last_change":"2024-09-16T15:54:55.455189+0000","last_active":"2024-09-16T15:54:57.467690+0000","last_peered":"2024-09-16T15:54:57.467690+0000","last_clean":"2024-09-16T15:54:57.467690+0000","last_became_active":"2024-09-16T15:54:55.454896+0000","last_became_peered":"2024-09-16T15:54:55.454896+0000","last_unstale":"2024-09-16T15:54:57.467690+0000","last_undegraded":"2024-09-16T15:54:57.467690+0000","last_fullsized":"2024-09-16T15:54:57.467690+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:55:36.203423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.484976+0000","last_change":"2024-09-16T15:54:56.471430+0000","last_active":"2024-09-16T15:54:57.484976+0000","last_peered":"2024-09-16T15:54:57.484976+0000","last_clean":"2024-09-16T15:54:57.484976+0000","last_became_active":"2024-09-16T15:54:56.471186+0000","last_became_peered":"2024-09-16T15:54:56.471186+0000","last_unstale":"2024-09-16T15:54:57.484976+0000","last_undegraded":"2024-09-16T15:54:57.484976+0000","last_fullsized":"2024-09-16T15:54:57.484976+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:12:31.317077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:58.635886+0000","last_change":"2024-09-16T15:54:55.460804+0000","last_active":"2024-09-16T15:54:58.635886+0000","last_peered":"2024-09-16T15:54:58.635886+0000","last_clean":"2024-09-16T15:54:58.635886+0000","last_became_active":"2024-09-16T15:54:55.460502+0000","last_became_peered":"2024-09-16T15:54:55.460502+0000","last_unstale":"2024-09-16T15:54:58.635886+0000","last_undegraded":"2024-09-16T15:54:58.635886+0000","last_fullsized":"2024-09-16T15:54:58.635886+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:34:33.303999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.485470+0000","last_change":"2024-09-16T15:54:56.474594+0000","last_active":"2024-09-16T15:54:57.485470+0000","last_peered":"2024-09-16T15:54:57.485470+0000","last_clean":"2024-09-16T15:54:57.485470+0000","last_became_active":"2024-09-16T15:54:56.474339+0000","last_became_peered":"2024-09-16T15:54:56.474339+0000","last_unstale":"2024-09-16T15:54:57.485470+0000","last_undegraded":"2024-09-16T15:54:57.485470+0000","last_fullsized":"2024-09-16T15:54:57.485470+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:36:59.034643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.468369+0000","last_change":"2024-09-16T15:54:55.460989+0000","last_active":"2024-09-16T15:54:57.468369+0000","last_peered":"2024-09-16T15:54:57.468369+0000","last_clean":"2024-09-16T15:54:57.468369+0000","last_became_active":"2024-09-16T15:54:55.459752+0000","last_became_peered":"2024-09-16T15:54:55.459752+0000","last_unstale":"2024-09-16T15:54:57.468369+0000","last_undegraded":"2024-09-16T15:54:57.468369+0000","last_fullsized":"2024-09-16T15:54:57.468369+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:21:28.811848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:58.637049+0000","last_change":"2024-09-16T15:54:56.469352+0000","last_active":"2024-09-16T15:54:58.637049+0000","last_peered":"2024-09-16T15:54:58.637049+0000","last_clean":"2024-09-16T15:54:58.637049+0000","last_became_active":"2024-09-16T15:54:56.469088+0000","last_became_peered":"2024-09-16T15:54:56.469088+0000","last_unstale":"2024-09-16T15:54:58.637049+0000","last_undegraded":"2024-09-16T15:54:58.637049+0000","last_fullsized":"2024-09-16T15:54:58.637049+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:23:24.896104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.466726+0000","last_change":"2024-09-16T15:54:55.462309+0000","last_active":"2024-09-16T15:54:57.466726+0000","last_peered":"2024-09-16T15:54:57.466726+0000","last_clean":"2024-09-16T15:54:57.466726+0000","last_became_active":"2024-09-16T15:54:55.462039+0000","last_became_peered":"2024-09-16T15:54:55.462039+0000","last_unstale":"2024-09-16T15:54:57.466726+0000","last_undegraded":"2024-09-16T15:54:57.466726+0000","last_fullsized":"2024-09-16T15:54:57.466726+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:32:00.564497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.467536+0000","last_change":"2024-09-16T15:54:56.470284+0000","last_active":"2024-09-16T15:54:57.467536+0000","last_peered":"2024-09-16T15:54:57.467536+0000","last_clean":"2024-09-16T15:54:57.467536+0000","last_became_active":"2024-09-16T15:54:56.470015+0000","last_became_peered":"2024-09-16T15:54:56.470015+0000","last_unstale":"2024-09-16T15:54:57.467536+0000","last_undegraded":"2024-09-16T15:54:57.467536+0000","last_fullsized":"2024-09-16T15:54:57.467536+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:15:20.308450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.14","version":"144'6","reported_seq":21,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.468028+0000","last_change":"2024-09-16T15:54:55.461505+0000","last_active":"2024-09-16T15:54:57.468028+0000","last_peered":"2024-09-16T15:54:57.468028+0000","last_clean":"2024-09-16T15:54:57.468028+0000","last_became_active":"2024-09-16T15:54:55.461174+0000","last_became_peered":"2024-09-16T15:54:55.461174+0000","last_unstale":"2024-09-16T15:54:57.468028+0000","last_undegraded":"2024-09-16T15:54:57.468028+0000","last_fullsized":"2024-09-16T15:54:57.468028+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:02:54.940683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.484971+0000","last_change":"2024-09-16T15:54:56.467821+0000","last_active":"2024-09-16T15:54:57.484971+0000","last_peered":"2024-09-16T15:54:57.484971+0000","last_clean":"2024-09-16T15:54:57.484971+0000","last_became_active":"2024-09-16T15:54:56.467543+0000","last_became_peered":"2024-09-16T15:54:56.467543+0000","last_unstale":"2024-09-16T15:54:57.484971+0000","last_undegraded":"2024-09-16T15:54:57.484971+0000","last_fullsized":"2024-09-16T15:54:57.484971+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:21:29.819815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.13","version":"144'2","reported_seq":17,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:58.636388+0000","last_change":"2024-09-16T15:54:55.462728+0000","last_active":"2024-09-16T15:54:58.636388+0000","last_peered":"2024-09-16T15:54:58.636388+0000","last_clean":"2024-09-16T15:54:58.636388+0000","last_became_active":"2024-09-16T15:54:55.462424+0000","last_became_peered":"2024-09-16T15:54:55.462424+0000","last_unstale":"2024-09-16T15:54:58.636388+0000","last_undegraded":"2024-09-16T15:54:58.636388+0000","last_fullsized":"2024-09-16T15:54:58.636388+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:11:11.408510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:58.637160+0000","last_change":"2024-09-16T15:54:56.466637+0000","last_active":"2024-09-16T15:54:58.637160+0000","last_peered":"2024-09-16T15:54:58.637160+0000","last_clean":"2024-09-16T15:54:58.637160+0000","last_became_active":"2024-09-16T15:54:56.466352+0000","last_became_peered":"2024-09-16T15:54:56.466352+0000","last_unstale":"2024-09-16T15:54:58.637160+0000","last_undegraded":"2024-09-16T15:54:58.637160+0000","last_fullsized":"2024-09-16T15:54:58.637160+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:06:40.951217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:58.637246+0000","last_change":"2024-09-16T15:54:55.455831+0000","last_active":"2024-09-16T15:54:58.637246+0000","last_peered":"2024-09-16T15:54:58.637246+0000","last_clean":"2024-09-16T15:54:58.637246+0000","last_became_active":"2024-09-16T15:54:55.455574+0000","last_became_peered":"2024-09-16T15:54:55.455574+0000","last_unstale":"2024-09-16T15:54:58.637246+0000","last_undegraded":"2024-09-16T15:54:58.637246+0000","last_fullsized":"2024-09-16T15:54:58.637246+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:38:34.406517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.484867+0000","last_change":"2024-09-16T15:54:56.470963+0000","last_active":"2024-09-16T15:54:57.484867+0000","last_peered":"2024-09-16T15:54:57.484867+0000","last_clean":"2024-09-16T15:54:57.484867+0000","last_became_active":"2024-09-16T15:54:56.470705+0000","last_became_peered":"2024-09-16T15:54:56.470705+0000","last_unstale":"2024-09-16T15:54:57.484867+0000","last_undegraded":"2024-09-16T15:54:57.484867+0000","last_fullsized":"2024-09-16T15:54:57.484867+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:54:16.675087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482490+0000","last_change":"2024-09-16T15:54:55.457472+0000","last_active":"2024-09-16T15:54:57.482490+0000","last_peered":"2024-09-16T15:54:57.482490+0000","last_clean":"2024-09-16T15:54:57.482490+0000","last_became_active":"2024-09-16T15:54:55.456471+0000","last_became_peered":"2024-09-16T15:54:55.456471+0000","last_unstale":"2024-09-16T15:54:57.482490+0000","last_undegraded":"2024-09-16T15:54:57.482490+0000","last_fullsized":"2024-09-16T15:54:57.482490+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:14:12.568813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.485149+0000","last_change":"2024-09-16T15:54:56.474937+0000","last_active":"2024-09-16T15:54:57.485149+0000","last_peered":"2024-09-16T15:54:57.485149+0000","last_clean":"2024-09-16T15:54:57.485149+0000","last_became_active":"2024-09-16T15:54:56.474373+0000","last_became_peered":"2024-09-16T15:54:56.474373+0000","last_unstale":"2024-09-16T15:54:57.485149+0000","last_undegraded":"2024-09-16T15:54:57.485149+0000","last_fullsized":"2024-09-16T15:54:57.485149+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:37:31.880953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.10","version":"144'8","reported_seq":23,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:58.637061+0000","last_change":"2024-09-16T15:54:55.461170+0000","last_active":"2024-09-16T15:54:58.637061+0000","last_peered":"2024-09-16T15:54:58.637061+0000","last_clean":"2024-09-16T15:54:58.637061+0000","last_became_active":"2024-09-16T15:54:55.460915+0000","last_became_peered":"2024-09-16T15:54:55.460915+0000","last_unstale":"2024-09-16T15:54:58.637061+0000","last_undegraded":"2024-09-16T15:54:58.637061+0000","last_fullsized":"2024-09-16T15:54:58.637061+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:51:08.351645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.466631+0000","last_change":"2024-09-16T15:54:56.464019+0000","last_active":"2024-09-16T15:54:57.466631+0000","last_peered":"2024-09-16T15:54:57.466631+0000","last_clean":"2024-09-16T15:54:57.466631+0000","last_became_active":"2024-09-16T15:54:56.463741+0000","last_became_peered":"2024-09-16T15:54:56.463741+0000","last_unstale":"2024-09-16T15:54:57.466631+0000","last_undegraded":"2024-09-16T15:54:57.466631+0000","last_fullsized":"2024-09-16T15:54:57.466631+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:53:30.459994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.f","version":"144'1","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.485636+0000","last_change":"2024-09-16T15:54:55.461113+0000","last_active":"2024-09-16T15:54:57.485636+0000","last_peered":"2024-09-16T15:54:57.485636+0000","last_clean":"2024-09-16T15:54:57.485636+0000","last_became_active":"2024-09-16T15:54:55.460866+0000","last_became_peered":"2024-09-16T15:54:55.460866+0000","last_unstale":"2024-09-16T15:54:57.485636+0000","last_undegraded":"2024-09-16T15:54:57.485636+0000","last_fullsized":"2024-09-16T15:54:57.485636+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:28:25.527345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482797+0000","last_change":"2024-09-16T15:54:56.491736+0000","last_active":"2024-09-16T15:54:57.482797+0000","last_peered":"2024-09-16T15:54:57.482797+0000","last_clean":"2024-09-16T15:54:57.482797+0000","last_became_active":"2024-09-16T15:54:56.491464+0000","last_became_peered":"2024-09-16T15:54:56.491464+0000","last_unstale":"2024-09-16T15:54:57.482797+0000","last_undegraded":"2024-09-16T15:54:57.482797+0000","last_fullsized":"2024-09-16T15:54:57.482797+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:35:45.264168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.e","version":"144'2","reported_seq":17,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:58.637646+0000","last_change":"2024-09-16T15:54:55.458328+0000","last_active":"2024-09-16T15:54:58.637646+0000","last_peered":"2024-09-16T15:54:58.637646+0000","last_clean":"2024-09-16T15:54:58.637646+0000","last_became_active":"2024-09-16T15:54:55.457453+0000","last_became_peered":"2024-09-16T15:54:55.457453+0000","last_unstale":"2024-09-16T15:54:58.637646+0000","last_undegraded":"2024-09-16T15:54:58.637646+0000","last_fullsized":"2024-09-16T15:54:58.637646+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:36:05.229515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.466008+0000","last_change":"2024-09-16T15:54:56.464797+0000","last_active":"2024-09-16T15:54:57.466008+0000","last_peered":"2024-09-16T15:54:57.466008+0000","last_clean":"2024-09-16T15:54:57.466008+0000","last_became_active":"2024-09-16T15:54:56.463873+0000","last_became_peered":"2024-09-16T15:54:56.463873+0000","last_unstale":"2024-09-16T15:54:57.466008+0000","last_undegraded":"2024-09-16T15:54:57.466008+0000","last_fullsized":"2024-09-16T15:54:57.466008+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:23:53.577681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.d","version":"144'1","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.468345+0000","last_change":"2024-09-16T15:54:55.463365+0000","last_active":"2024-09-16T15:54:57.468345+0000","last_peered":"2024-09-16T15:54:57.468345+0000","last_clean":"2024-09-16T15:54:57.468345+0000","last_became_active":"2024-09-16T15:54:55.462246+0000","last_became_peered":"2024-09-16T15:54:55.462246+0000","last_unstale":"2024-09-16T15:54:57.468345+0000","last_undegraded":"2024-09-16T15:54:57.468345+0000","last_fullsized":"2024-09-16T15:54:57.468345+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:49:42.884684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.468115+0000","last_change":"2024-09-16T15:54:56.474299+0000","last_active":"2024-09-16T15:54:57.468115+0000","last_peered":"2024-09-16T15:54:57.468115+0000","last_clean":"2024-09-16T15:54:57.468115+0000","last_became_active":"2024-09-16T15:54:56.473995+0000","last_became_peered":"2024-09-16T15:54:56.473995+0000","last_unstale":"2024-09-16T15:54:57.468115+0000","last_undegraded":"2024-09-16T15:54:57.468115+0000","last_fullsized":"2024-09-16T15:54:57.468115+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:12:41.893235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482928+0000","last_change":"2024-09-16T15:54:55.463637+0000","last_active":"2024-09-16T15:54:57.482928+0000","last_peered":"2024-09-16T15:54:57.482928+0000","last_clean":"2024-09-16T15:54:57.482928+0000","last_became_active":"2024-09-16T15:54:55.463289+0000","last_became_peered":"2024-09-16T15:54:55.463289+0000","last_unstale":"2024-09-16T15:54:57.482928+0000","last_undegraded":"2024-09-16T15:54:57.482928+0000","last_fullsized":"2024-09-16T15:54:57.482928+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:16:31.781727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:58.637148+0000","last_change":"2024-09-16T15:54:56.463211+0000","last_active":"2024-09-16T15:54:58.637148+0000","last_peered":"2024-09-16T15:54:58.637148+0000","last_clean":"2024-09-16T15:54:58.637148+0000","last_became_active":"2024-09-16T15:54:56.462693+0000","last_became_peered":"2024-09-16T15:54:56.462693+0000","last_unstale":"2024-09-16T15:54:58.637148+0000","last_undegraded":"2024-09-16T15:54:58.637148+0000","last_fullsized":"2024-09-16T15:54:58.637148+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:56:21.875089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.b","version":"144'2","reported_seq":17,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.467334+0000","last_change":"2024-09-16T15:54:55.459884+0000","last_active":"2024-09-16T15:54:57.467334+0000","last_peered":"2024-09-16T15:54:57.467334+0000","last_clean":"2024-09-16T15:54:57.467334+0000","last_became_active":"2024-09-16T15:54:55.459556+0000","last_became_peered":"2024-09-16T15:54:55.459556+0000","last_unstale":"2024-09-16T15:54:57.467334+0000","last_undegraded":"2024-09-16T15:54:57.467334+0000","last_fullsized":"2024-09-16T15:54:57.467334+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:12:46.866334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482617+0000","last_change":"2024-09-16T15:54:56.485052+0000","last_active":"2024-09-16T15:54:57.482617+0000","last_peered":"2024-09-16T15:54:57.482617+0000","last_clean":"2024-09-16T15:54:57.482617+0000","last_became_active":"2024-09-16T15:54:56.483119+0000","last_became_peered":"2024-09-16T15:54:56.483119+0000","last_unstale":"2024-09-16T15:54:57.482617+0000","last_undegraded":"2024-09-16T15:54:57.482617+0000","last_fullsized":"2024-09-16T15:54:57.482617+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:50:40.992458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.485142+0000","last_change":"2024-09-16T15:54:55.464323+0000","last_active":"2024-09-16T15:54:57.485142+0000","last_peered":"2024-09-16T15:54:57.485142+0000","last_clean":"2024-09-16T15:54:57.485142+0000","last_became_active":"2024-09-16T15:54:55.463480+0000","last_became_peered":"2024-09-16T15:54:55.463480+0000","last_unstale":"2024-09-16T15:54:57.485142+0000","last_undegraded":"2024-09-16T15:54:57.485142+0000","last_fullsized":"2024-09-16T15:54:57.485142+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:52:22.970018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:58.636155+0000","last_change":"2024-09-16T15:54:56.472643+0000","last_active":"2024-09-16T15:54:58.636155+0000","last_peered":"2024-09-16T15:54:58.636155+0000","last_clean":"2024-09-16T15:54:58.636155+0000","last_became_active":"2024-09-16T15:54:56.472222+0000","last_became_peered":"2024-09-16T15:54:56.472222+0000","last_unstale":"2024-09-16T15:54:58.636155+0000","last_undegraded":"2024-09-16T15:54:58.636155+0000","last_fullsized":"2024-09-16T15:54:58.636155+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T01:39:52.118744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.485127+0000","last_change":"2024-09-16T15:54:55.459040+0000","last_active":"2024-09-16T15:54:57.485127+0000","last_peered":"2024-09-16T15:54:57.485127+0000","last_clean":"2024-09-16T15:54:57.485127+0000","last_became_active":"2024-09-16T15:54:55.458758+0000","last_became_peered":"2024-09-16T15:54:55.458758+0000","last_unstale":"2024-09-16T15:54:57.485127+0000","last_undegraded":"2024-09-16T15:54:57.485127+0000","last_fullsized":"2024-09-16T15:54:57.485127+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:53:40.653455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482549+0000","last_change":"2024-09-16T15:54:56.485351+0000","last_active":"2024-09-16T15:54:57.482549+0000","last_peered":"2024-09-16T15:54:57.482549+0000","last_clean":"2024-09-16T15:54:57.482549+0000","last_became_active":"2024-09-16T15:54:56.484236+0000","last_became_peered":"2024-09-16T15:54:56.484236+0000","last_unstale":"2024-09-16T15:54:57.482549+0000","last_undegraded":"2024-09-16T15:54:57.482549+0000","last_fullsized":"2024-09-16T15:54:57.482549+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T23:48:52.418050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.485149+0000","last_change":"2024-09-16T15:54:55.463632+0000","last_active":"2024-09-16T15:54:57.485149+0000","last_peered":"2024-09-16T15:54:57.485149+0000","last_clean":"2024-09-16T15:54:57.485149+0000","last_became_active":"2024-09-16T15:54:55.463347+0000","last_became_peered":"2024-09-16T15:54:55.463347+0000","last_unstale":"2024-09-16T15:54:57.485149+0000","last_undegraded":"2024-09-16T15:54:57.485149+0000","last_fullsized":"2024-09-16T15:54:57.485149+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:00:37.618200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.485545+0000","last_change":"2024-09-16T15:54:56.470743+0000","last_active":"2024-09-16T15:54:57.485545+0000","last_peered":"2024-09-16T15:54:57.485545+0000","last_clean":"2024-09-16T15:54:57.485545+0000","last_became_active":"2024-09-16T15:54:56.470476+0000","last_became_peered":"2024-09-16T15:54:56.470476+0000","last_unstale":"2024-09-16T15:54:57.485545+0000","last_undegraded":"2024-09-16T15:54:57.485545+0000","last_fullsized":"2024-09-16T15:54:57.485545+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:12:41.893235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.7","version":"144'2","reported_seq":17,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.485730+0000","last_change":"2024-09-16T15:54:55.459856+0000","last_active":"2024-09-16T15:54:57.485730+0000","last_peered":"2024-09-16T15:54:57.485730+0000","last_clean":"2024-09-16T15:54:57.485730+0000","last_became_active":"2024-09-16T15:54:55.458670+0000","last_became_peered":"2024-09-16T15:54:55.458670+0000","last_unstale":"2024-09-16T15:54:57.485730+0000","last_undegraded":"2024-09-16T15:54:57.485730+0000","last_fullsized":"2024-09-16T15:54:57.485730+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:41:35.010525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.485465+0000","last_change":"2024-09-16T15:54:56.472974+0000","last_active":"2024-09-16T15:54:57.485465+0000","last_peered":"2024-09-16T15:54:57.485465+0000","last_clean":"2024-09-16T15:54:57.485465+0000","last_became_active":"2024-09-16T15:54:56.472712+0000","last_became_peered":"2024-09-16T15:54:56.472712+0000","last_unstale":"2024-09-16T15:54:57.485465+0000","last_undegraded":"2024-09-16T15:54:57.485465+0000","last_fullsized":"2024-09-16T15:54:57.485465+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:57:20.690077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.6","version":"145'1","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482760+0000","last_change":"2024-09-16T15:54:55.464427+0000","last_active":"2024-09-16T15:54:57.482760+0000","last_peered":"2024-09-16T15:54:57.482760+0000","last_clean":"2024-09-16T15:54:57.482760+0000","last_became_active":"2024-09-16T15:54:55.464043+0000","last_became_peered":"2024-09-16T15:54:55.464043+0000","last_unstale":"2024-09-16T15:54:57.482760+0000","last_undegraded":"2024-09-16T15:54:57.482760+0000","last_fullsized":"2024-09-16T15:54:57.482760+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:06:18.925531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:58.637740+0000","last_change":"2024-09-16T15:54:56.467486+0000","last_active":"2024-09-16T15:54:58.637740+0000","last_peered":"2024-09-16T15:54:58.637740+0000","last_clean":"2024-09-16T15:54:58.637740+0000","last_became_active":"2024-09-16T15:54:56.467207+0000","last_became_peered":"2024-09-16T15:54:56.467207+0000","last_unstale":"2024-09-16T15:54:58.637740+0000","last_undegraded":"2024-09-16T15:54:58.637740+0000","last_fullsized":"2024-09-16T15:54:58.637740+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:22:38.823057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.468451+0000","last_change":"2024-09-16T15:54:55.460656+0000","last_active":"2024-09-16T15:54:57.468451+0000","last_peered":"2024-09-16T15:54:57.468451+0000","last_clean":"2024-09-16T15:54:57.468451+0000","last_became_active":"2024-09-16T15:54:55.460149+0000","last_became_peered":"2024-09-16T15:54:55.460149+0000","last_unstale":"2024-09-16T15:54:57.468451+0000","last_undegraded":"2024-09-16T15:54:57.468451+0000","last_fullsized":"2024-09-16T15:54:57.468451+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:31:15.859741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:58.637480+0000","last_change":"2024-09-16T15:54:56.476395+0000","last_active":"2024-09-16T15:54:58.637480+0000","last_peered":"2024-09-16T15:54:58.637480+0000","last_clean":"2024-09-16T15:54:58.637480+0000","last_became_active":"2024-09-16T15:54:56.475195+0000","last_became_peered":"2024-09-16T15:54:56.475195+0000","last_unstale":"2024-09-16T15:54:58.637480+0000","last_undegraded":"2024-09-16T15:54:58.637480+0000","last_fullsized":"2024-09-16T15:54:58.637480+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:01:06.763095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.468176+0000","last_change":"2024-09-16T15:54:55.461333+0000","last_active":"2024-09-16T15:54:57.468176+0000","last_peered":"2024-09-16T15:54:57.468176+0000","last_clean":"2024-09-16T15:54:57.468176+0000","last_became_active":"2024-09-16T15:54:55.460016+0000","last_became_peered":"2024-09-16T15:54:55.460016+0000","last_unstale":"2024-09-16T15:54:57.468176+0000","last_undegraded":"2024-09-16T15:54:57.468176+0000","last_fullsized":"2024-09-16T15:54:57.468176+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:06:39.943250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.467593+0000","last_change":"2024-09-16T15:54:56.463581+0000","last_active":"2024-09-16T15:54:57.467593+0000","last_peered":"2024-09-16T15:54:57.467593+0000","last_clean":"2024-09-16T15:54:57.467593+0000","last_became_active":"2024-09-16T15:54:56.463191+0000","last_became_peered":"2024-09-16T15:54:56.463191+0000","last_unstale":"2024-09-16T15:54:57.467593+0000","last_undegraded":"2024-09-16T15:54:57.467593+0000","last_fullsized":"2024-09-16T15:54:57.467593+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:08:31.123222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.482633+0000","last_change":"2024-09-16T15:54:55.455582+0000","last_active":"2024-09-16T15:54:57.482633+0000","last_peered":"2024-09-16T15:54:57.482633+0000","last_clean":"2024-09-16T15:54:57.482633+0000","last_became_active":"2024-09-16T15:54:55.453745+0000","last_became_peered":"2024-09-16T15:54:55.453745+0000","last_unstale":"2024-09-16T15:54:57.482633+0000","last_undegraded":"2024-09-16T15:54:57.482633+0000","last_fullsized":"2024-09-16T15:54:57.482633+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:42:00.578523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.485464+0000","last_change":"2024-09-16T15:54:56.468239+0000","last_active":"2024-09-16T15:54:57.485464+0000","last_peered":"2024-09-16T15:54:57.485464+0000","last_clean":"2024-09-16T15:54:57.485464+0000","last_became_active":"2024-09-16T15:54:56.467883+0000","last_became_peered":"2024-09-16T15:54:56.467883+0000","last_unstale":"2024-09-16T15:54:57.485464+0000","last_undegraded":"2024-09-16T15:54:57.485464+0000","last_fullsized":"2024-09-16T15:54:57.485464+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:02:33.014042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:58.636347+0000","last_change":"2024-09-16T15:54:55.449566+0000","last_active":"2024-09-16T15:54:58.636347+0000","last_peered":"2024-09-16T15:54:58.636347+0000","last_clean":"2024-09-16T15:54:58.636347+0000","last_became_active":"2024-09-16T15:54:55.449318+0000","last_became_peered":"2024-09-16T15:54:55.449318+0000","last_unstale":"2024-09-16T15:54:58.636347+0000","last_undegraded":"2024-09-16T15:54:58.636347+0000","last_fullsized":"2024-09-16T15:54:58.636347+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:58:12.275694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.484867+0000","last_change":"2024-09-16T15:54:56.471555+0000","last_active":"2024-09-16T15:54:57.484867+0000","last_peered":"2024-09-16T15:54:57.484867+0000","last_clean":"2024-09-16T15:54:57.484867+0000","last_became_active":"2024-09-16T15:54:56.471313+0000","last_became_peered":"2024-09-16T15:54:56.471313+0000","last_unstale":"2024-09-16T15:54:57.484867+0000","last_undegraded":"2024-09-16T15:54:57.484867+0000","last_fullsized":"2024-09-16T15:54:57.484867+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:12:47.874301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:58.636058+0000","last_change":"2024-09-16T15:54:55.461228+0000","last_active":"2024-09-16T15:54:58.636058+0000","last_peered":"2024-09-16T15:54:58.636058+0000","last_clean":"2024-09-16T15:54:58.636058+0000","last_became_active":"2024-09-16T15:54:55.460939+0000","last_became_peered":"2024-09-16T15:54:55.460939+0000","last_unstale":"2024-09-16T15:54:58.636058+0000","last_undegraded":"2024-09-16T15:54:58.636058+0000","last_fullsized":"2024-09-16T15:54:58.636058+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:11:04.495951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:58.635913+0000","last_change":"2024-09-16T15:54:56.464762+0000","last_active":"2024-09-16T15:54:58.635913+0000","last_peered":"2024-09-16T15:54:58.635913+0000","last_clean":"2024-09-16T15:54:58.635913+0000","last_became_active":"2024-09-16T15:54:56.464478+0000","last_became_peered":"2024-09-16T15:54:56.464478+0000","last_unstale":"2024-09-16T15:54:58.635913+0000","last_undegraded":"2024-09-16T15:54:58.635913+0000","last_fullsized":"2024-09-16T15:54:58.635913+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:39:31.943265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.467453+0000","last_change":"2024-09-16T15:54:55.459516+0000","last_active":"2024-09-16T15:54:57.467453+0000","last_peered":"2024-09-16T15:54:57.467453+0000","last_clean":"2024-09-16T15:54:57.467453+0000","last_became_active":"2024-09-16T15:54:55.459041+0000","last_became_peered":"2024-09-16T15:54:55.459041+0000","last_unstale":"2024-09-16T15:54:57.467453+0000","last_undegraded":"2024-09-16T15:54:57.467453+0000","last_fullsized":"2024-09-16T15:54:57.467453+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:16:32.668378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.468567+0000","last_change":"2024-09-16T15:54:56.472953+0000","last_active":"2024-09-16T15:54:57.468567+0000","last_peered":"2024-09-16T15:54:57.468567+0000","last_clean":"2024-09-16T15:54:57.468567+0000","last_became_active":"2024-09-16T15:54:56.472646+0000","last_became_peered":"2024-09-16T15:54:56.472646+0000","last_unstale":"2024-09-16T15:54:57.468567+0000","last_undegraded":"2024-09-16T15:54:57.468567+0000","last_fullsized":"2024-09-16T15:54:57.468567+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:55.432610+0000","last_clean_scrub_stamp":"2024-09-16T15:54:55.432610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T00:07:51.469446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.15","version":"144'3","reported_seq":18,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:58.637066+0000","last_change":"2024-09-16T15:54:55.461430+0000","last_active":"2024-09-16T15:54:58.637066+0000","last_peered":"2024-09-16T15:54:58.637066+0000","last_clean":"2024-09-16T15:54:58.637066+0000","last_became_active":"2024-09-16T15:54:55.461166+0000","last_became_peered":"2024-09-16T15:54:55.461166+0000","last_unstale":"2024-09-16T15:54:58.637066+0000","last_undegraded":"2024-09-16T15:54:58.637066+0000","last_fullsized":"2024-09-16T15:54:58.637066+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:54:54.424643+0000","last_clean_scrub_stamp":"2024-09-16T15:54:54.424643+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:14:12.423767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":345,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T15:54:57.465675+0000","last_change":"2024-09-16T15:39:12.698813+0000","last_active":"2024-09-16T15:54:57.465675+0000","last_peered":"2024-09-16T15:54:57.465675+0000","last_clean":"2024-09-16T15:54:57.465675+0000","last_became_active":"2024-09-16T15:39:12.697695+0000","last_became_peered":"2024-09-16T15:39:12.697695+0000","last_unstale":"2024-09-16T15:54:57.465675+0000","last_undegraded":"2024-09-16T15:54:57.465675+0000","last_fullsized":"2024-09-16T15:54:57.465675+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_clean_scrub_stamp":"2024-09-16T15:39:11.684967+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:02.061081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509635,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":163384,"kb_used_data":1764,"kb_used_omap":1,"kb_used_meta":161598,"kb_avail":94208456,"statfs":{"total":96636764160,"available":96469458944,"internally_reserved":0,"allocated":1806336,"data_stored":800812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":165476808},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509634,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":123732,"kb_used_data":2368,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94248108,"statfs":{"total":96636764160,"available":96510062592,"internally_reserved":0,"allocated":2424832,"data_stored":1392986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":124254663},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509633,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":137036,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94234804,"statfs":{"total":96636764160,"available":96496439296,"internally_reserved":0,"allocated":1826816,"data_stored":803028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":138475973},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509633,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":137024,"kb_used_data":1772,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94234816,"statfs":{"total":96636764160,"available":96496451584,"internally_reserved":0,"allocated":1814528,"data_stored":801428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":138475977},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509633,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":118976,"kb_used_data":1768,"kb_used_omap":1,"kb_used_meta":117182,"kb_avail":94252864,"statfs":{"total":96636764160,"available":96514932736,"internally_reserved":0,"allocated":1810432,"data_stored":800834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":119994823},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509634,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":132040,"kb_used_data":2348,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94239800,"statfs":{"total":96636764160,"available":96501555200,"internally_reserved":0,"allocated":2404352,"data_stored":1391250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":132774343},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509632,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149500,"kb_used_data":1768,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222340,"statfs":{"total":96636764160,"available":96483676160,"internally_reserved":0,"allocated":1810432,"data_stored":800902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":151255495},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542340,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131452,"kb_used_data":1764,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94240388,"statfs":{"total":96636764160,"available":96502157312,"internally_reserved":0,"allocated":1806336,"data_stored":800812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":132774348},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:55:05.752 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T15:55:05.752 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-16T15:55:05.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.954+0000 7f2619c00640 1 -- 172.21.15.151:0/2126581582 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f261406e660 msgr2=0x7f261406eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:05.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.954+0000 7f2619c00640 1 --2- 172.21.15.151:0/2126581582 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f261406e660 0x7f261406eac0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f2614159c50 tx=0x7f260c02aac0 comp rx=0 tx=0).stop 2024-09-16T15:55:05.957 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.954+0000 7f2619c00640 1 -- 172.21.15.151:0/2126581582 shutdown_connections 2024-09-16T15:55:05.957 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.954+0000 7f2619c00640 1 --2- 172.21.15.151:0/2126581582 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f261406e660 0x7f261406eac0 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:05.957 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.954+0000 7f2619c00640 1 --2- 172.21.15.151:0/2126581582 >> v2:172.21.15.167:3300/0 conn(0x7f26140c5730 0x7f26140c6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:05.957 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.954+0000 7f2619c00640 1 -- 172.21.15.151:0/2126581582 >> 172.21.15.151:0/2126581582 conn(0x7f26140cc490 msgr2=0x7f26140cc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:55:05.957 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.954+0000 7f2619c00640 1 -- 172.21.15.151:0/2126581582 shutdown_connections 2024-09-16T15:55:05.957 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.954+0000 7f2619c00640 1 -- 172.21.15.151:0/2126581582 wait complete. 2024-09-16T15:55:05.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.956+0000 7f2619c00640 1 Processor -- start 2024-09-16T15:55:05.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.956+0000 7f2619c00640 1 -- start start 2024-09-16T15:55:05.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.957+0000 7f2619c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f261406e660 0x7f26141f6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:05.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.957+0000 7f2619c00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26140c5730 0x7f2614155cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:05.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.957+0000 7f2619c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f261415f670 0x7f26140c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:05.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.957+0000 7f2619c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f26140ca4f0 con 0x7f261415f670 2024-09-16T15:55:05.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.957+0000 7f2619c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f26140ca370 con 0x7f261406e660 2024-09-16T15:55:05.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.957+0000 7f2619c00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f261406f000 con 0x7f26140c5730 2024-09-16T15:55:05.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.957+0000 7f2619200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f261415f670 0x7f26140c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:05.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.957+0000 7f2619200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f261415f670 0x7f26140c8fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44728/0 (socket says 172.21.15.151:44728) 2024-09-16T15:55:05.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.957+0000 7f2619200640 1 -- 172.21.15.151:0/1435196647 learned_addr learned my addr 172.21.15.151:0/1435196647 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:55:05.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.957+0000 7f2613e00640 1 --2- 172.21.15.151:0/1435196647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f261406e660 0x7f26141f6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:05.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.957+0000 7f2613400640 1 --2- 172.21.15.151:0/1435196647 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26140c5730 0x7f2614155cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:05.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.958+0000 7f2619200640 1 -- 172.21.15.151:0/1435196647 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26140c5730 msgr2=0x7f2614155cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:05.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.958+0000 7f2619200640 1 --2- 172.21.15.151:0/1435196647 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26140c5730 0x7f2614155cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:05.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.958+0000 7f2619200640 1 -- 172.21.15.151:0/1435196647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f261406e660 msgr2=0x7f26141f6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:05.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.958+0000 7f2619200640 1 --2- 172.21.15.151:0/1435196647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f261406e660 0x7f26141f6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:05.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.958+0000 7f2619200640 1 -- 172.21.15.151:0/1435196647 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26141f6c50 con 0x7f261415f670 2024-09-16T15:55:05.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.958+0000 7f2619200640 1 --2- 172.21.15.151:0/1435196647 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f261415f670 0x7f26140c8fa0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f260c02b0b0 tx=0x7f260c003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:55:05.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.958+0000 7f260be00640 1 -- 172.21.15.151:0/1435196647 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260c044070 con 0x7f261415f670 2024-09-16T15:55:05.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.958+0000 7f260be00640 1 -- 172.21.15.151:0/1435196647 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f260c004250 con 0x7f261415f670 2024-09-16T15:55:05.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.958+0000 7f2619c00640 1 -- 172.21.15.151:0/1435196647 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26140c7f00 con 0x7f261415f670 2024-09-16T15:55:05.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.958+0000 7f260be00640 1 -- 172.21.15.151:0/1435196647 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260c004680 con 0x7f261415f670 2024-09-16T15:55:05.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.958+0000 7f2619c00640 1 -- 172.21.15.151:0/1435196647 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26140c87a0 con 0x7f261415f670 2024-09-16T15:55:05.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.959+0000 7f2619c00640 1 -- 172.21.15.151:0/1435196647 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f261406eac0 con 0x7f261415f670 2024-09-16T15:55:05.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.961+0000 7f260be00640 1 -- 172.21.15.151:0/1435196647 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 50) ==== 104398+0+0 (secure 0 0 0) 0x7f260c02c840 con 0x7f261415f670 2024-09-16T15:55:05.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.962+0000 7f260be00640 1 --2- 172.21.15.151:0/1435196647 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f25f407aaa0 0x7f25f407cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:05.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.962+0000 7f260be00640 1 -- 172.21.15.151:0/1435196647 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5907+0+0 (secure 0 0 0) 0x7f260c0ba120 con 0x7f261415f670 2024-09-16T15:55:05.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.962+0000 7f2613e00640 1 --2- 172.21.15.151:0/1435196647 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f25f407aaa0 0x7f25f407cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:05.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.963+0000 7f2613e00640 1 --2- 172.21.15.151:0/1435196647 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f25f407aaa0 0x7f25f407cf60 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f2604007fd0 tx=0x7f2604009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:55:05.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:05.966+0000 7f260be00640 1 -- 172.21.15.151:0/1435196647 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f26141f6c50 con 0x7f261415f670 2024-09-16T15:55:06.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:06.221+0000 7f2619c00640 1 -- 172.21.15.151:0/1435196647 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f26140b1930 con 0x7f261415f670 2024-09-16T15:55:06.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:06.224+0000 7f260be00640 1 -- 172.21.15.151:0/1435196647 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v21) ==== 190+0+147 (secure 0 0 0) 0x7f260c032030 con 0x7f261415f670 2024-09-16T15:55:06.226 INFO:teuthology.orchestra.run.smithi151.stdout:[client.0] 2024-09-16T15:55:06.226 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQDsUOhmf5PBKBAAKE4Y4zbA4h07IBNjPkyptw== 2024-09-16T15:55:06.226 INFO:teuthology.orchestra.run.smithi151.stdout: caps mds = "allow" 2024-09-16T15:55:06.226 INFO:teuthology.orchestra.run.smithi151.stdout: caps mon = "allow r" 2024-09-16T15:55:06.226 INFO:teuthology.orchestra.run.smithi151.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-16T15:55:06.226 INFO:teuthology.orchestra.run.smithi151.stderr:updated caps for client.0 2024-09-16T15:55:06.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:06.226+0000 7f2619c00640 1 -- 172.21.15.151:0/1435196647 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f25f407aaa0 msgr2=0x7f25f407cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:06.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:06.226+0000 7f2619c00640 1 --2- 172.21.15.151:0/1435196647 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f25f407aaa0 0x7f25f407cf60 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f2604007fd0 tx=0x7f2604009040 comp rx=0 tx=0).stop 2024-09-16T15:55:06.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:06.226+0000 7f2619c00640 1 -- 172.21.15.151:0/1435196647 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f261415f670 msgr2=0x7f26140c8fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:06.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:06.226+0000 7f2619c00640 1 --2- 172.21.15.151:0/1435196647 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f261415f670 0x7f26140c8fa0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f260c02b0b0 tx=0x7f260c003f10 comp rx=0 tx=0).stop 2024-09-16T15:55:06.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:06.226+0000 7f2619c00640 1 -- 172.21.15.151:0/1435196647 shutdown_connections 2024-09-16T15:55:06.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:06.226+0000 7f2619c00640 1 --2- 172.21.15.151:0/1435196647 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f25f407aaa0 0x7f25f407cf60 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:06.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:06.226+0000 7f2619c00640 1 --2- 172.21.15.151:0/1435196647 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f261415f670 0x7f26140c8fa0 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:06.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:06.226+0000 7f2619c00640 1 --2- 172.21.15.151:0/1435196647 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26140c5730 0x7f2614155cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:06.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:06.226+0000 7f2619c00640 1 --2- 172.21.15.151:0/1435196647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f261406e660 0x7f26141f6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:06.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:06.226+0000 7f2619c00640 1 -- 172.21.15.151:0/1435196647 >> 172.21.15.151:0/1435196647 conn(0x7f26140cc490 msgr2=0x7f26140b4970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:55:06.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:06.226+0000 7f2619c00640 1 -- 172.21.15.151:0/1435196647 shutdown_connections 2024-09-16T15:55:06.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:06.227+0000 7f2619c00640 1 -- 172.21.15.151:0/1435196647 wait complete. 2024-09-16T15:55:06.246 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:55:06.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:06.401+0000 7f2699e00640 1 -- 172.21.15.151:0/271452229 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f26940cf300 msgr2=0x7f26940729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:06.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:06.401+0000 7f2699e00640 1 --2- 172.21.15.151:0/271452229 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f26940cf300 0x7f26940729a0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f2680005fd0 tx=0x7f268002a660 comp rx=0 tx=0).stop 2024-09-16T15:55:06.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:06.401+0000 7f2699e00640 1 -- 172.21.15.151:0/271452229 shutdown_connections 2024-09-16T15:55:06.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:06.401+0000 7f2699e00640 1 --2- 172.21.15.151:0/271452229 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26940ccfd0 0x7f2694072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:06.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:06.401+0000 7f2699e00640 1 --2- 172.21.15.151:0/271452229 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f26940cf300 0x7f26940729a0 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:06.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:06.401+0000 7f2699e00640 1 -- 172.21.15.151:0/271452229 >> 172.21.15.151:0/271452229 conn(0x7f26940b1950 msgr2=0x7f26940b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:55:06.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:06.401+0000 7f2699e00640 1 -- 172.21.15.151:0/271452229 shutdown_connections 2024-09-16T15:55:06.404 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:06.401+0000 7f2699e00640 1 -- 172.21.15.151:0/271452229 wait complete. 2024-09-16T15:55:06.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:06.403+0000 7f2699e00640 1 Processor -- start 2024-09-16T15:55:06.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:06.403+0000 7f2699e00640 1 -- start start 2024-09-16T15:55:06.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:06.404+0000 7f2699e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26940ccfd0 0x7f269413f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:06.406 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:06.404+0000 7f2699e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26940cf300 0x7f269414eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:06.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:06.404+0000 7f2699e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f26941445d0 0x7f269409ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:06.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:06.404+0000 7f2699e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2694073420 con 0x7f26941445d0 2024-09-16T15:55:06.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:06.404+0000 7f2699e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f26940b4eb0 con 0x7f26940ccfd0 2024-09-16T15:55:06.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:06.404+0000 7f2699e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f26940735a0 con 0x7f26940cf300 2024-09-16T15:55:06.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:06.404+0000 7f2693400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f26941445d0 0x7f269409ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:06.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:06.404+0000 7f2693400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f26941445d0 0x7f269409ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44750/0 (socket says 172.21.15.151:44750) 2024-09-16T15:55:06.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:06.404+0000 7f2693400640 1 -- 172.21.15.151:0/1427792496 learned_addr learned my addr 172.21.15.151:0/1427792496 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:55:06.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:06.404+0000 7f2692000640 1 --2- 172.21.15.151:0/1427792496 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26940cf300 0x7f269414eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:06.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:06.405+0000 7f2692a00640 1 --2- 172.21.15.151:0/1427792496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26940ccfd0 0x7f269413f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:06.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:06.405+0000 7f2693400640 1 -- 172.21.15.151:0/1427792496 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26940cf300 msgr2=0x7f269414eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:06.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:06.405+0000 7f2693400640 1 --2- 172.21.15.151:0/1427792496 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26940cf300 0x7f269414eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:06.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:06.405+0000 7f2693400640 1 -- 172.21.15.151:0/1427792496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26940ccfd0 msgr2=0x7f269413f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:06.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:06.405+0000 7f2693400640 1 --2- 172.21.15.151:0/1427792496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26940ccfd0 0x7f269413f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:06.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:06.405+0000 7f2693400640 1 -- 172.21.15.151:0/1427792496 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f269414f960 con 0x7f26941445d0 2024-09-16T15:55:06.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:06.405+0000 7f2692a00640 1 --2- 172.21.15.151:0/1427792496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26940ccfd0 0x7f269413f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:55:06.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:06.405+0000 7f2693400640 1 --2- 172.21.15.151:0/1427792496 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f26941445d0 0x7f269409ca10 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f268c00d750 tx=0x7f268c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:55:06.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:06.405+0000 7f268b400640 1 -- 172.21.15.151:0/1427792496 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f268c017070 con 0x7f26941445d0 2024-09-16T15:55:06.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:06.405+0000 7f268b400640 1 -- 172.21.15.151:0/1427792496 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f268c009cc0 con 0x7f26941445d0 2024-09-16T15:55:06.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:06.405+0000 7f268b400640 1 -- 172.21.15.151:0/1427792496 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f268c00de60 con 0x7f26941445d0 2024-09-16T15:55:06.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:06.405+0000 7f2699e00640 1 -- 172.21.15.151:0/1427792496 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f269414fc40 con 0x7f26941445d0 2024-09-16T15:55:06.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:06.405+0000 7f2699e00640 1 -- 172.21.15.151:0/1427792496 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2694150200 con 0x7f26941445d0 2024-09-16T15:55:06.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:06.407+0000 7f2699e00640 1 -- 172.21.15.151:0/1427792496 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2694073a80 con 0x7f26941445d0 2024-09-16T15:55:06.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:06.408+0000 7f268b400640 1 -- 172.21.15.151:0/1427792496 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 50) ==== 104398+0+0 (secure 0 0 0) 0x7f268c005280 con 0x7f26941445d0 2024-09-16T15:55:06.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:06.409+0000 7f268b400640 1 --2- 172.21.15.151:0/1427792496 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f267407aaa0 0x7f267407cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:06.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:06.409+0000 7f268b400640 1 -- 172.21.15.151:0/1427792496 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5907+0+0 (secure 0 0 0) 0x7f268c099b70 con 0x7f26941445d0 2024-09-16T15:55:06.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:06.409+0000 7f2692a00640 1 --2- 172.21.15.151:0/1427792496 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f267407aaa0 0x7f267407cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:06.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:06.410+0000 7f2692a00640 1 --2- 172.21.15.151:0/1427792496 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f267407aaa0 0x7f267407cf60 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f2684001d10 tx=0x7f2684009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:55:06.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:06.413+0000 7f268b400640 1 -- 172.21.15.151:0/1427792496 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f268c063ce0 con 0x7f26941445d0 2024-09-16T15:55:06.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:06.637+0000 7f2699e00640 1 -- 172.21.15.151:0/1427792496 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2694073d00 con 0x7f26941445d0 2024-09-16T15:55:06.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:06.638+0000 7f268b400640 1 -- 172.21.15.151:0/1427792496 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) ==== 94+0+5236 (secure 0 0 0) 0x7f268c0635e0 con 0x7f26941445d0 2024-09-16T15:55:06.640 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:55:06.640 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":96,"btime":"2024-09-16T15:54:59:477590+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7739,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2795044069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2795044069},{"type":"v1","addr":"172.21.15.167:6837","nonce":2795044069}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":8644,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/675530853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":675530853},{"type":"v1","addr":"172.21.15.151:6835","nonce":675530853}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":9468,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/1930858450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1930858450},{"type":"v1","addr":"172.21.15.151:6834","nonce":1930858450}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:54:55.864255+0000","modified":"2024-09-16T15:54:58.898875+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9474},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9474":{"gid":9474,"name":"b","rank":0,"incarnation":93,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6836/1231498178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1231498178},{"type":"v1","addr":"172.21.15.167:6836","nonce":1231498178}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9474,"qdb_cluster":[9474]},"id":10}]} 2024-09-16T15:55:06.640 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 96 2024-09-16T15:55:06.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:06.640+0000 7f2699e00640 1 -- 172.21.15.151:0/1427792496 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f267407aaa0 msgr2=0x7f267407cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:06.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:06.640+0000 7f2699e00640 1 --2- 172.21.15.151:0/1427792496 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f267407aaa0 0x7f267407cf60 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f2684001d10 tx=0x7f2684009040 comp rx=0 tx=0).stop 2024-09-16T15:55:06.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:06.641+0000 7f2699e00640 1 -- 172.21.15.151:0/1427792496 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f26941445d0 msgr2=0x7f269409ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:06.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:06.641+0000 7f2699e00640 1 --2- 172.21.15.151:0/1427792496 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f26941445d0 0x7f269409ca10 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f268c00d750 tx=0x7f268c00dc10 comp rx=0 tx=0).stop 2024-09-16T15:55:06.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:06.641+0000 7f2699e00640 1 -- 172.21.15.151:0/1427792496 shutdown_connections 2024-09-16T15:55:06.643 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:06.641+0000 7f2699e00640 1 --2- 172.21.15.151:0/1427792496 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f267407aaa0 0x7f267407cf60 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:06.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:06.641+0000 7f2699e00640 1 --2- 172.21.15.151:0/1427792496 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f26941445d0 0x7f269409ca10 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:06.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:06.641+0000 7f2699e00640 1 --2- 172.21.15.151:0/1427792496 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26940cf300 0x7f269414eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:06.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:06.641+0000 7f2699e00640 1 --2- 172.21.15.151:0/1427792496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26940ccfd0 0x7f269413f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:06.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:06.641+0000 7f2699e00640 1 -- 172.21.15.151:0/1427792496 >> 172.21.15.151:0/1427792496 conn(0x7f26940b1950 msgr2=0x7f26940afb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:55:06.644 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:06.641+0000 7f2699e00640 1 -- 172.21.15.151:0/1427792496 shutdown_connections 2024-09-16T15:55:06.645 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:06.641+0000 7f2699e00640 1 -- 172.21.15.151:0/1427792496 wait complete. 2024-09-16T15:55:07.062 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 96, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-16T15:54:55.864255+0000', 'modified': '2024-09-16T15:54:58.898875+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9474}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9474': {'gid': 9474, 'name': 'b', 'rank': 0, 'incarnation': 93, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.167:6836/1231498178', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6834', 'nonce': 1231498178}, {'type': 'v1', 'addr': '172.21.15.167:6836', 'nonce': 1231498178}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9474, 'qdb_cluster': [9474]} 2024-09-16T15:55:07.062 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-16T15:55:07.062 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-16T15:55:07.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.221+0000 7f0e7be00640 1 -- 172.21.15.151:0/3853391837 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0e600052e0 msgr2=0x7f0e6000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:07.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.221+0000 7f0e7be00640 1 --2- 172.21.15.151:0/3853391837 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0e600052e0 0x7f0e6000b9c0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f0e74005fd0 tx=0x7f0e60005830 comp rx=0 tx=0).stop 2024-09-16T15:55:07.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.221+0000 7f0e7be00640 1 -- 172.21.15.151:0/3853391837 shutdown_connections 2024-09-16T15:55:07.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.221+0000 7f0e7be00640 1 --2- 172.21.15.151:0/3853391837 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0e600052e0 0x7f0e6000b9c0 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:07.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.221+0000 7f0e7be00640 1 -- 172.21.15.151:0/3853391837 >> 172.21.15.151:0/3853391837 conn(0x7f0e7c0d1c40 msgr2=0x7f0e7c0d2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:55:07.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.221+0000 7f0e7be00640 1 -- 172.21.15.151:0/3853391837 shutdown_connections 2024-09-16T15:55:07.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.221+0000 7f0e7be00640 1 -- 172.21.15.151:0/3853391837 wait complete. 2024-09-16T15:55:07.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.222+0000 7f0e7be00640 1 Processor -- start 2024-09-16T15:55:07.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.223+0000 7f0e7be00640 1 -- start start 2024-09-16T15:55:07.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.223+0000 7f0e7be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0e600052e0 0x7f0e7c0b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:07.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.223+0000 7f0e7be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e7c0c72e0 0x7f0e7c157fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:07.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.223+0000 7f0e7be00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0e7c0c7760 0x7f0e7c158510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:07.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.223+0000 7f0e7be00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7c0d00e0 con 0x7f0e600052e0 2024-09-16T15:55:07.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.223+0000 7f0e7be00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7c0cff60 con 0x7f0e7c0c72e0 2024-09-16T15:55:07.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.224+0000 7f0e7be00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f0e7c0d0260 con 0x7f0e7c0c7760 2024-09-16T15:55:07.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.224+0000 7f0e7aa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0e600052e0 0x7f0e7c0b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:07.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.224+0000 7f0e7aa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0e600052e0 0x7f0e7c0b26d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44756/0 (socket says 172.21.15.151:44756) 2024-09-16T15:55:07.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.224+0000 7f0e7aa00640 1 -- 172.21.15.151:0/2589899920 learned_addr learned my addr 172.21.15.151:0/2589899920 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:55:07.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.224+0000 7f0e7aa00640 1 -- 172.21.15.151:0/2589899920 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0e7c0c7760 msgr2=0x7f0e7c158510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:07.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.224+0000 7f0e7aa00640 1 --2- 172.21.15.151:0/2589899920 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0e7c0c7760 0x7f0e7c158510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:07.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.224+0000 7f0e7aa00640 1 -- 172.21.15.151:0/2589899920 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e7c0c72e0 msgr2=0x7f0e7c157fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:07.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.224+0000 7f0e7a000640 1 --2- 172.21.15.151:0/2589899920 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e7c0c72e0 0x7f0e7c157fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:07.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.224+0000 7f0e7aa00640 1 --2- 172.21.15.151:0/2589899920 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e7c0c72e0 0x7f0e7c157fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:07.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.224+0000 7f0e7aa00640 1 -- 172.21.15.151:0/2589899920 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e7c1f8800 con 0x7f0e600052e0 2024-09-16T15:55:07.227 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.224+0000 7f0e7aa00640 1 --2- 172.21.15.151:0/2589899920 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0e600052e0 0x7f0e7c0b26d0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c0066b0 tx=0x7f0e6c0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:55:07.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.225+0000 7f0e73400640 1 -- 172.21.15.151:0/2589899920 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c015070 con 0x7f0e600052e0 2024-09-16T15:55:07.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.225+0000 7f0e73400640 1 -- 172.21.15.151:0/2589899920 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e6c011920 con 0x7f0e600052e0 2024-09-16T15:55:07.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.225+0000 7f0e73400640 1 -- 172.21.15.151:0/2589899920 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c011d70 con 0x7f0e600052e0 2024-09-16T15:55:07.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.225+0000 7f0e7be00640 1 -- 172.21.15.151:0/2589899920 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e7c1f4700 con 0x7f0e600052e0 2024-09-16T15:55:07.228 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.225+0000 7f0e7be00640 1 -- 172.21.15.151:0/2589899920 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e7c1f4cc0 con 0x7f0e600052e0 2024-09-16T15:55:07.230 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.228+0000 7f0e73400640 1 -- 172.21.15.151:0/2589899920 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 50) ==== 104398+0+0 (secure 0 0 0) 0x7f0e6c018070 con 0x7f0e600052e0 2024-09-16T15:55:07.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.229+0000 7f0e73400640 1 --2- 172.21.15.151:0/2589899920 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0e5c07aaa0 0x7f0e5c07cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:07.231 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.229+0000 7f0e84c98740 1 Processor -- start 2024-09-16T15:55:07.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.230+0000 7f0e73400640 1 -- 172.21.15.151:0/2589899920 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5907+0+0 (secure 0 0 0) 0x7f0e6c09a930 con 0x7f0e600052e0 2024-09-16T15:55:07.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.230+0000 7f0e84c98740 1 -- start start 2024-09-16T15:55:07.232 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.230+0000 7f0e7a000640 1 --2- 172.21.15.151:0/2589899920 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0e5c07aaa0 0x7f0e5c07cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:07.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.231+0000 7f0e84c98740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557487348aa0 0x557487348e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:07.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.231+0000 7f0e84c98740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557487303c10 0x5574873477b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:07.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.231+0000 7f0e84c98740 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557487304190 0x557487347cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:07.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.231+0000 7f0e84c98740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557487286430 con 0x557487348aa0 2024-09-16T15:55:07.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.231+0000 7f0e84c98740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5574872765d0 con 0x557487303c10 2024-09-16T15:55:07.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.231+0000 7f0e84c98740 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x557487265a00 con 0x557487304190 2024-09-16T15:55:07.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.231+0000 7f0e7a000640 1 --2- 172.21.15.151:0/2589899920 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0e5c07aaa0 0x7f0e5c07cf60 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c0c91e0 tx=0x7f0e68006b70 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:55:07.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.231+0000 7f0e7aa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557487348aa0 0x557487348e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:07.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.231+0000 7f0e7aa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557487348aa0 0x557487348e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44762/0 (socket says 172.21.15.151:44762) 2024-09-16T15:55:07.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.231+0000 7f0e7aa00640 1 -- 172.21.15.151:0/3062914328 learned_addr learned my addr 172.21.15.151:0/3062914328 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:55:07.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.232+0000 7f0e7a000640 1 --2- 172.21.15.151:0/3062914328 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557487303c10 0x5574873477b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:07.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.232+0000 7f0e7b400640 1 --2- 172.21.15.151:0/3062914328 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557487304190 0x557487347cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:07.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.232+0000 7f0e7aa00640 1 -- 172.21.15.151:0/3062914328 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557487304190 msgr2=0x557487347cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:55:07.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.232+0000 7f0e7aa00640 1 --2- 172.21.15.151:0/3062914328 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557487304190 0x557487347cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:07.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.232+0000 7f0e7aa00640 1 -- 172.21.15.151:0/3062914328 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557487303c10 msgr2=0x5574873477b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:55:07.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.232+0000 7f0e7aa00640 1 --2- 172.21.15.151:0/3062914328 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557487303c10 0x5574873477b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:07.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.232+0000 7f0e7aa00640 1 -- 172.21.15.151:0/3062914328 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55748722e520 con 0x557487348aa0 2024-09-16T15:55:07.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.232+0000 7f0e7aa00640 1 --2- 172.21.15.151:0/3062914328 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557487348aa0 0x557487348e80 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c064210 tx=0x7f0e6c063d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5777c5319288b520 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:55:07.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.233+0000 7f0e67400640 1 -- 172.21.15.151:0/3062914328 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c01f040 con 0x557487348aa0 2024-09-16T15:55:07.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.233+0000 7f0e67400640 1 -- 172.21.15.151:0/3062914328 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e6c068d60 con 0x557487348aa0 2024-09-16T15:55:07.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.233+0000 7f0e67400640 1 -- 172.21.15.151:0/3062914328 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c06d9e0 con 0x557487348aa0 2024-09-16T15:55:07.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.234+0000 7f0e84c98740 1 -- 172.21.15.151:0/3062914328 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557487348aa0 msgr2=0x557487348e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:07.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.234+0000 7f0e84c98740 1 --2- 172.21.15.151:0/3062914328 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557487348aa0 0x557487348e80 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c064210 tx=0x7f0e6c063d90 comp rx=0 tx=0).stop 2024-09-16T15:55:07.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.234+0000 7f0e84c98740 1 -- 172.21.15.151:0/3062914328 shutdown_connections 2024-09-16T15:55:07.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.234+0000 7f0e84c98740 1 --2- 172.21.15.151:0/3062914328 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557487303c10 0x5574873477b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:07.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.234+0000 7f0e84c98740 1 --2- 172.21.15.151:0/3062914328 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557487304190 0x557487347cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:07.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.234+0000 7f0e84c98740 1 --2- 172.21.15.151:0/3062914328 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557487348aa0 0x557487348e80 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:07.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.234+0000 7f0e84c98740 1 -- 172.21.15.151:0/3062914328 >> 172.21.15.151:0/3062914328 conn(0x5574872ff1e0 msgr2=0x557487302c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:55:07.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.234+0000 7f0e84c98740 1 -- 172.21.15.151:0/3062914328 shutdown_connections 2024-09-16T15:55:07.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.234+0000 7f0e84c98740 1 -- 172.21.15.151:0/3062914328 wait complete. 2024-09-16T15:55:07.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.235+0000 7f0e84c98740 1 -- start start 2024-09-16T15:55:07.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.235+0000 7f0e84c98740 1 Processor -- start 2024-09-16T15:55:07.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.236+0000 7f0e84c98740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557487304190 0x557487327ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:07.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.236+0000 7f0e84c98740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557487348aa0 0x557487320880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:07.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.236+0000 7f0e84c98740 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55748732fe80 0x557487323220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:07.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.236+0000 7f0e84c98740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55748720c330 con 0x557487304190 2024-09-16T15:55:07.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.236+0000 7f0e84c98740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x557487269810 con 0x557487348aa0 2024-09-16T15:55:07.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.236+0000 7f0e84c98740 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x557486fd6620 con 0x55748732fe80 2024-09-16T15:55:07.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.236+0000 7f0e84c98740 10 client.0 _ll_register_callbacks cb 0x557487211420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T15:55:07.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.236+0000 7f0e7aa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557487304190 0x557487327ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:07.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.236+0000 7f0e7a000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557487348aa0 0x557487320880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:07.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.236+0000 7f0e7b400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55748732fe80 0x557487323220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:07.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.236+0000 7f0e7aa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557487304190 0x557487327ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44772/0 (socket says 172.21.15.151:44772) 2024-09-16T15:55:07.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.236+0000 7f0e7aa00640 1 -- 172.21.15.151:0/3477594443 learned_addr learned my addr 172.21.15.151:0/3477594443 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:55:07.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.237+0000 7f0e7aa00640 1 -- 172.21.15.151:0/3477594443 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55748732fe80 msgr2=0x557487323220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:55:07.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.237+0000 7f0e7aa00640 1 --2- 172.21.15.151:0/3477594443 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55748732fe80 0x557487323220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:07.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.237+0000 7f0e7aa00640 1 -- 172.21.15.151:0/3477594443 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557487348aa0 msgr2=0x557487320880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:55:07.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.237+0000 7f0e7aa00640 1 --2- 172.21.15.151:0/3477594443 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557487348aa0 0x557487320880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:07.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.237+0000 7f0e7aa00640 1 -- 172.21.15.151:0/3477594443 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557486fc7850 con 0x557487304190 2024-09-16T15:55:07.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.237+0000 7f0e7b400640 1 --2- 172.21.15.151:0/3477594443 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55748732fe80 0x557487323220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:55:07.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.237+0000 7f0e7a000640 1 --2- 172.21.15.151:0/3477594443 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557487348aa0 0x557487320880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:55:07.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.237+0000 7f0e7aa00640 1 --2- 172.21.15.151:0/3477594443 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557487304190 0x557487327ae0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c09ade0 tx=0x7f0e6c06db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7dbcd77ffb0860f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:55:07.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.237+0000 7f0e5be00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-16T15:55:07.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.237+0000 7f0e5be00640 1 -- 172.21.15.151:0/3477594443 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c01f040 con 0x557487304190 2024-09-16T15:55:07.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.237+0000 7f0e5be00640 1 -- 172.21.15.151:0/3477594443 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(147..147 src has 1..147) ==== 5907+0+0 (secure 0 0 0) 0x7f0e6c0634e0 con 0x557487304190 2024-09-16T15:55:07.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.237+0000 7f0e84c98740 1 -- 172.21.15.151:0/3477594443 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e6c068d60 con 0x557487304190 2024-09-16T15:55:07.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.237+0000 7f0e5be00640 1 -- 172.21.15.151:0/3477594443 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) ==== 24+0+0 (secure 0 0 0) 0x7f0e6c07e390 con 0x557487304190 2024-09-16T15:55:07.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.237+0000 7f0e84c98740 10 client.9627 fetch_fsmap learned FSMap version 96 2024-09-16T15:55:07.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.237+0000 7f0e84c98740 1 -- 172.21.15.151:0/3477594443 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=2+}) -- 0x55748703e6b0 con 0x557487304190 2024-09-16T15:55:07.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.238+0000 7f0e5be00640 1 -- 172.21.15.151:0/3477594443 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e6c068d60 con 0x557487304190 2024-09-16T15:55:07.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.238+0000 7f0e5be00640 1 -- 172.21.15.151:0/3477594443 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 96) ==== 3502+0+0 (secure 0 0 0) 0x7f0e6c05fbd0 con 0x557487304190 2024-09-16T15:55:07.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.238+0000 7f0e84c98740 10 client.9627 fetch_fsmap finished waiting for FSMap version 96 2024-09-16T15:55:07.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.238+0000 7f0e84c98740 10 client.9627 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-16T15:55:07.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.238+0000 7f0e84c98740 20 client.9627 populate_metadata read hostname 'smithi151' 2024-09-16T15:55:07.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.238+0000 7f0e84c98740 1 --2- 172.21.15.151:0/3477594443 >> [v2:172.21.15.167:6834/1231498178,v1:172.21.15.167:6836/1231498178] conn(0x557487363c70 0x557487364050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:07.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.238+0000 7f0e84c98740 4 client.9627 mds_command: new command op to 9474 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T15:55:07.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.238+0000 7f0e84c98740 1 -- 172.21.15.151:0/3477594443 --> [v2:172.21.15.167:6834/1231498178,v1:172.21.15.167:6836/1231498178] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557487278520 con 0x557487363c70 2024-09-16T15:55:07.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.238+0000 7f0e7a000640 1 --2- 172.21.15.151:0/3477594443 >> [v2:172.21.15.167:6834/1231498178,v1:172.21.15.167:6836/1231498178] conn(0x557487363c70 0x557487364050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:07.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.239+0000 7f0e7a000640 1 --2- 172.21.15.151:0/3477594443 >> [v2:172.21.15.167:6834/1231498178,v1:172.21.15.167:6836/1231498178] conn(0x557487363c70 0x557487364050 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=661d2291ed1e6531 server_cookie=995579130fac2151 in_seq=0 out_seq=0 2024-09-16T15:55:07.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.239+0000 7f0e5be00640 10 client.9627 ms_handle_connect on v2:172.21.15.167:6834/1231498178 2024-09-16T15:55:07.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.240+0000 7f0e5be00640 1 -- 172.21.15.151:0/3477594443 <== mds.0 v2:172.21.15.167:6834/1231498178 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557487278520 con 0x557487363c70 2024-09-16T15:55:07.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.240+0000 7f0e5be00640 10 client.9627 handle_command_reply: tid=0 2024-09-16T15:55:07.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.240+0000 7f0e84c98740 1 client.9627 shutdown 2024-09-16T15:55:07.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.240+0000 7f0e84c98740 1 -- 172.21.15.151:0/3477594443 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557487304190 msgr2=0x557487327ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:07.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.240+0000 7f0e84c98740 1 --2- 172.21.15.151:0/3477594443 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557487304190 0x557487327ae0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c09ade0 tx=0x7f0e6c06db40 comp rx=0 tx=0).stop 2024-09-16T15:55:07.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.240+0000 7f0e84c98740 1 -- 172.21.15.151:0/3477594443 shutdown_connections 2024-09-16T15:55:07.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.240+0000 7f0e84c98740 1 --2- 172.21.15.151:0/3477594443 >> [v2:172.21.15.167:6834/1231498178,v1:172.21.15.167:6836/1231498178] conn(0x557487363c70 0x557487364050 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:07.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.240+0000 7f0e84c98740 1 --2- 172.21.15.151:0/3477594443 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557487348aa0 0x557487320880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:07.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.240+0000 7f0e84c98740 1 --2- 172.21.15.151:0/3477594443 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55748732fe80 0x557487323220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:07.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.240+0000 7f0e5be00640 0 client.9627 ms_handle_reset on v2:172.21.15.167:6834/1231498178 2024-09-16T15:55:07.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.240+0000 7f0e84c98740 1 --2- 172.21.15.151:0/3477594443 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557487304190 0x557487327ae0 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:07.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.240+0000 7f0e84c98740 1 -- 172.21.15.151:0/3477594443 >> 172.21.15.151:0/3477594443 conn(0x557487303c10 msgr2=0x55748732ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:55:07.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.240+0000 7f0e84c98740 1 -- 172.21.15.151:0/3477594443 shutdown_connections 2024-09-16T15:55:07.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.241+0000 7f0e84c98740 1 -- 172.21.15.151:0/3477594443 wait complete. 2024-09-16T15:55:07.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.241+0000 7f0e84c98740 20 client.9627 trim_cache size 0 max 16384 2024-09-16T15:55:07.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.259+0000 7f0e84c98740 1 Processor -- start 2024-09-16T15:55:07.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.259+0000 7f0e84c98740 1 -- start start 2024-09-16T15:55:07.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.260+0000 7f0e84c98740 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557487304190 0x557487340b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:07.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.260+0000 7f0e84c98740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55748732fe80 0x5574873029c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:07.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.260+0000 7f0e84c98740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557487348aa0 0x557487302f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:07.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.260+0000 7f0e84c98740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7c1e2940 con 0x55748732fe80 2024-09-16T15:55:07.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.260+0000 7f0e84c98740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7c06f020 con 0x557487348aa0 2024-09-16T15:55:07.264 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.260+0000 7f0e84c98740 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f0e7c1e2270 con 0x557487304190 2024-09-16T15:55:07.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.261+0000 7f0e7a000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55748732fe80 0x5574873029c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:07.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.261+0000 7f0e7a000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55748732fe80 0x5574873029c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44788/0 (socket says 172.21.15.151:44788) 2024-09-16T15:55:07.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.261+0000 7f0e7a000640 1 -- 172.21.15.151:0/2791419825 learned_addr learned my addr 172.21.15.151:0/2791419825 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:55:07.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.261+0000 7f0e7b400640 1 --2- 172.21.15.151:0/2791419825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557487348aa0 0x557487302f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:07.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.261+0000 7f0e7aa00640 1 --2- 172.21.15.151:0/2791419825 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557487304190 0x557487340b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:07.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.261+0000 7f0e7a000640 1 -- 172.21.15.151:0/2791419825 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557487304190 msgr2=0x557487340b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:55:07.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.261+0000 7f0e7a000640 1 --2- 172.21.15.151:0/2791419825 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557487304190 0x557487340b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:07.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.261+0000 7f0e7a000640 1 -- 172.21.15.151:0/2791419825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557487348aa0 msgr2=0x557487302f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:55:07.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.261+0000 7f0e7a000640 1 --2- 172.21.15.151:0/2791419825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557487348aa0 0x557487302f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:07.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.261+0000 7f0e7a000640 1 -- 172.21.15.151:0/2791419825 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55748732f6c0 con 0x55748732fe80 2024-09-16T15:55:07.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.261+0000 7f0e7b400640 1 --2- 172.21.15.151:0/2791419825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557487348aa0 0x557487302f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:55:07.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.261+0000 7f0e7a000640 1 --2- 172.21.15.151:0/2791419825 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55748732fe80 0x5574873029c0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f0e6800fe20 tx=0x7f0e68012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7961a7908efb2605 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:55:07.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.261+0000 7f0e66a00640 1 -- 172.21.15.151:0/2791419825 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e68031450 con 0x55748732fe80 2024-09-16T15:55:07.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.261+0000 7f0e66a00640 1 -- 172.21.15.151:0/2791419825 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e680315f0 con 0x55748732fe80 2024-09-16T15:55:07.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.261+0000 7f0e66a00640 1 -- 172.21.15.151:0/2791419825 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e680317a0 con 0x55748732fe80 2024-09-16T15:55:07.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.262+0000 7f0e84c98740 1 -- 172.21.15.151:0/2791419825 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55748732fe80 msgr2=0x5574873029c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:07.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.262+0000 7f0e84c98740 1 --2- 172.21.15.151:0/2791419825 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55748732fe80 0x5574873029c0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f0e6800fe20 tx=0x7f0e68012000 comp rx=0 tx=0).stop 2024-09-16T15:55:07.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.262+0000 7f0e84c98740 1 -- 172.21.15.151:0/2791419825 shutdown_connections 2024-09-16T15:55:07.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.262+0000 7f0e84c98740 1 --2- 172.21.15.151:0/2791419825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557487348aa0 0x557487302f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:07.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.262+0000 7f0e84c98740 1 --2- 172.21.15.151:0/2791419825 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55748732fe80 0x5574873029c0 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:07.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.262+0000 7f0e84c98740 1 --2- 172.21.15.151:0/2791419825 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x557487304190 0x557487340b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:07.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.262+0000 7f0e84c98740 1 -- 172.21.15.151:0/2791419825 >> 172.21.15.151:0/2791419825 conn(0x557487303c10 msgr2=0x55748732ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:55:07.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.262+0000 7f0e84c98740 1 -- 172.21.15.151:0/2791419825 shutdown_connections 2024-09-16T15:55:07.266 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.262+0000 7f0e84c98740 1 -- 172.21.15.151:0/2791419825 wait complete. 2024-09-16T15:55:07.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.263+0000 7f0e84c98740 1 -- start start 2024-09-16T15:55:07.269 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.263+0000 7f0e84c98740 1 Processor -- start 2024-09-16T15:55:07.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.264+0000 7f0e84c98740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557487304190 0x557487373010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:07.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.264+0000 7f0e84c98740 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55748732fe80 0x5574873d2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:07.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.264+0000 7f0e84c98740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557487348aa0 0x5574872fe3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:07.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.264+0000 7f0e84c98740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5574873282a0 con 0x557487304190 2024-09-16T15:55:07.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.264+0000 7f0e84c98740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55748720c330 con 0x557487348aa0 2024-09-16T15:55:07.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.264+0000 7f0e84c98740 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x5574873272a0 con 0x55748732fe80 2024-09-16T15:55:07.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.264+0000 7f0e84c98740 10 client.0 _ll_register_callbacks cb 0x7f0e600036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T15:55:07.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.264+0000 7f0e7aa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557487304190 0x557487373010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:07.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.264+0000 7f0e7aa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557487304190 0x557487373010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44798/0 (socket says 172.21.15.151:44798) 2024-09-16T15:55:07.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.264+0000 7f0e7aa00640 1 -- 172.21.15.151:0/4212269875 learned_addr learned my addr 172.21.15.151:0/4212269875 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:55:07.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.265+0000 7f0e7aa00640 1 -- 172.21.15.151:0/4212269875 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55748732fe80 msgr2=0x5574873d2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:55:07.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.265+0000 7f0e7aa00640 1 --2- 172.21.15.151:0/4212269875 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55748732fe80 0x5574873d2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:07.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.265+0000 7f0e7aa00640 1 -- 172.21.15.151:0/4212269875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557487348aa0 msgr2=0x5574872fe3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T15:55:07.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.265+0000 7f0e7aa00640 1 --2- 172.21.15.151:0/4212269875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557487348aa0 0x5574872fe3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:07.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.265+0000 7f0e7aa00640 1 -- 172.21.15.151:0/4212269875 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55748736aae0 con 0x557487304190 2024-09-16T15:55:07.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.265+0000 7f0e7aa00640 1 --2- 172.21.15.151:0/4212269875 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557487304190 0x557487373010 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c09ade0 tx=0x7f0e6c060700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=302f476f1f3927cf server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:55:07.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.265+0000 7f0e5be00640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-16T15:55:07.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.265+0000 7f0e5be00640 1 -- 172.21.15.151:0/4212269875 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c06de60 con 0x557487304190 2024-09-16T15:55:07.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.265+0000 7f0e5be00640 1 -- 172.21.15.151:0/4212269875 <== mon.0 v2:172.21.15.151:3300/0 2 ==== osd_map(147..147 src has 1..147) ==== 5907+0+0 (secure 0 0 0) 0x7f0e6c0634e0 con 0x557487304190 2024-09-16T15:55:07.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.265+0000 7f0e84c98740 1 -- 172.21.15.151:0/4212269875 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e680315f0 con 0x557487304190 2024-09-16T15:55:07.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.265+0000 7f0e5be00640 1 -- 172.21.15.151:0/4212269875 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) ==== 24+0+0 (secure 0 0 0) 0x7f0e6c075050 con 0x557487304190 2024-09-16T15:55:07.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.265+0000 7f0e84c98740 10 client.9633 fetch_fsmap learned FSMap version 96 2024-09-16T15:55:07.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.265+0000 7f0e84c98740 1 -- 172.21.15.151:0/4212269875 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=2+}) -- 0x55748736aae0 con 0x557487304190 2024-09-16T15:55:07.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.265+0000 7f0e5be00640 1 -- 172.21.15.151:0/4212269875 <== mon.0 v2:172.21.15.151:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e680315f0 con 0x557487304190 2024-09-16T15:55:07.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.265+0000 7f0e5be00640 1 -- 172.21.15.151:0/4212269875 <== mon.0 v2:172.21.15.151:3300/0 5 ==== fsmap(e 96) ==== 3502+0+0 (secure 0 0 0) 0x7f0e6c078020 con 0x557487304190 2024-09-16T15:55:07.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.266+0000 7f0e84c98740 10 client.9633 fetch_fsmap finished waiting for FSMap version 96 2024-09-16T15:55:07.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.266+0000 7f0e84c98740 10 client.9633 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-16T15:55:07.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.266+0000 7f0e84c98740 20 client.9633 populate_metadata read hostname 'smithi151' 2024-09-16T15:55:07.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.266+0000 7f0e84c98740 1 --2- 172.21.15.151:0/4212269875 >> [v2:172.21.15.167:6834/1231498178,v1:172.21.15.167:6836/1231498178] conn(0x557487340d50 0x557487341130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:07.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.266+0000 7f0e84c98740 4 client.9633 mds_command: new command op to 9474 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T15:55:07.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.266+0000 7f0e84c98740 1 -- 172.21.15.151:0/4212269875 --> [v2:172.21.15.167:6834/1231498178,v1:172.21.15.167:6836/1231498178] -- command(tid 0: {"prefix": "status"}) -- 0x557487273fd0 con 0x557487340d50 2024-09-16T15:55:07.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.266+0000 7f0e7a000640 1 --2- 172.21.15.151:0/4212269875 >> [v2:172.21.15.167:6834/1231498178,v1:172.21.15.167:6836/1231498178] conn(0x557487340d50 0x557487341130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:07.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.266+0000 7f0e7a000640 1 --2- 172.21.15.151:0/4212269875 >> [v2:172.21.15.167:6834/1231498178,v1:172.21.15.167:6836/1231498178] conn(0x557487340d50 0x557487341130 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52edc72625a55ea5 server_cookie=d9c0641f64dc7f74 in_seq=0 out_seq=0 2024-09-16T15:55:07.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.266+0000 7f0e5be00640 10 client.9633 ms_handle_connect on v2:172.21.15.167:6834/1231498178 2024-09-16T15:55:07.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.267+0000 7f0e5be00640 1 -- 172.21.15.151:0/4212269875 <== mds.0 v2:172.21.15.167:6834/1231498178 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x7f0e68037160 con 0x557487340d50 2024-09-16T15:55:07.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.267+0000 7f0e5be00640 10 client.9633 handle_command_reply: tid=0 2024-09-16T15:55:07.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.267+0000 7f0e84c98740 1 client.9633 shutdown 2024-09-16T15:55:07.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.267+0000 7f0e84c98740 1 -- 172.21.15.151:0/4212269875 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557487304190 msgr2=0x557487373010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:07.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.267+0000 7f0e84c98740 1 --2- 172.21.15.151:0/4212269875 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557487304190 0x557487373010 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c09ade0 tx=0x7f0e6c060700 comp rx=0 tx=0).stop 2024-09-16T15:55:07.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.267+0000 7f0e84c98740 1 -- 172.21.15.151:0/4212269875 shutdown_connections 2024-09-16T15:55:07.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.267+0000 7f0e84c98740 1 --2- 172.21.15.151:0/4212269875 >> [v2:172.21.15.167:6834/1231498178,v1:172.21.15.167:6836/1231498178] conn(0x557487340d50 0x557487341130 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:07.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.267+0000 7f0e84c98740 1 --2- 172.21.15.151:0/4212269875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557487348aa0 0x5574872fe3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:07.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.267+0000 7f0e84c98740 1 --2- 172.21.15.151:0/4212269875 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x55748732fe80 0x5574873d2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:07.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.267+0000 7f0e5be00640 0 client.9633 ms_handle_reset on v2:172.21.15.167:6834/1231498178 2024-09-16T15:55:07.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.267+0000 7f0e84c98740 1 --2- 172.21.15.151:0/4212269875 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557487304190 0x557487373010 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:07.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.267+0000 7f0e84c98740 1 -- 172.21.15.151:0/4212269875 >> 172.21.15.151:0/4212269875 conn(0x557487303c10 msgr2=0x55748734b150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:55:07.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.267+0000 7f0e84c98740 1 -- 172.21.15.151:0/4212269875 shutdown_connections 2024-09-16T15:55:07.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.267+0000 7f0e84c98740 1 -- 172.21.15.151:0/4212269875 wait complete. 2024-09-16T15:55:07.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.267+0000 7f0e84c98740 20 client.9633 trim_cache size 0 max 16384 2024-09-16T15:55:07.274 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-16T15:55:07.274 INFO:teuthology.orchestra.run.smithi151.stdout: "cluster_fsid": "1757be18-e41a-4446-bb4f-ee4ac81c39df", 2024-09-16T15:55:07.274 INFO:teuthology.orchestra.run.smithi151.stdout: "whoami": 0, 2024-09-16T15:55:07.274 INFO:teuthology.orchestra.run.smithi151.stdout: "id": 9474, 2024-09-16T15:55:07.275 INFO:teuthology.orchestra.run.smithi151.stdout: "want_state": "up:active", 2024-09-16T15:55:07.275 INFO:teuthology.orchestra.run.smithi151.stdout: "state": "up:active", 2024-09-16T15:55:07.275 INFO:teuthology.orchestra.run.smithi151.stdout: "fs_name": "cephfs", 2024-09-16T15:55:07.275 INFO:teuthology.orchestra.run.smithi151.stdout: "rank_uptime": 10.805086812000001, 2024-09-16T15:55:07.275 INFO:teuthology.orchestra.run.smithi151.stdout: "mdsmap_epoch": 96, 2024-09-16T15:55:07.275 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_epoch": 147, 2024-09-16T15:55:07.275 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_epoch_barrier": 145, 2024-09-16T15:55:07.275 INFO:teuthology.orchestra.run.smithi151.stdout: "uptime": 15.131661149999999 2024-09-16T15:55:07.275 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-16T15:55:07.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.268+0000 7f0e7be00640 1 -- 172.21.15.151:0/2589899920 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0e5c07aaa0 msgr2=0x7f0e5c07cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:07.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.268+0000 7f0e7be00640 1 --2- 172.21.15.151:0/2589899920 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0e5c07aaa0 0x7f0e5c07cf60 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c0c91e0 tx=0x7f0e68006b70 comp rx=0 tx=0).stop 2024-09-16T15:55:07.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.268+0000 7f0e7be00640 1 -- 172.21.15.151:0/2589899920 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0e600052e0 msgr2=0x7f0e7c0b26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:07.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.268+0000 7f0e7be00640 1 --2- 172.21.15.151:0/2589899920 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0e600052e0 0x7f0e7c0b26d0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c0066b0 tx=0x7f0e6c0039c0 comp rx=0 tx=0).stop 2024-09-16T15:55:07.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.268+0000 7f0e7be00640 1 -- 172.21.15.151:0/2589899920 shutdown_connections 2024-09-16T15:55:07.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.268+0000 7f0e7be00640 1 --2- 172.21.15.151:0/2589899920 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0e5c07aaa0 0x7f0e5c07cf60 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:07.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.268+0000 7f0e7be00640 1 --2- 172.21.15.151:0/2589899920 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0e7c0c72e0 0x7f0e7c157fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:07.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.268+0000 7f0e7be00640 1 --2- 172.21.15.151:0/2589899920 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0e7c0c7760 0x7f0e7c158510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:07.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.268+0000 7f0e7be00640 1 --2- 172.21.15.151:0/2589899920 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0e600052e0 0x7f0e7c0b26d0 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:07.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.268+0000 7f0e7be00640 1 -- 172.21.15.151:0/2589899920 >> 172.21.15.151:0/2589899920 conn(0x7f0e7c0d1c40 msgr2=0x7f0e7c159be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:55:07.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.268+0000 7f0e7be00640 1 -- 172.21.15.151:0/2589899920 shutdown_connections 2024-09-16T15:55:07.277 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.268+0000 7f0e7be00640 1 -- 172.21.15.151:0/2589899920 wait complete. 2024-09-16T15:55:07.286 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-16T15:55:07.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.442+0000 7fcde5000640 1 -- 172.21.15.151:0/516029325 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcde00ccf00 msgr2=0x7fcde00b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:07.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.442+0000 7fcde5000640 1 --2- 172.21.15.151:0/516029325 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcde00ccf00 0x7fcde00b2440 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc009860 tx=0x7fcdcc02a510 comp rx=0 tx=0).stop 2024-09-16T15:55:07.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.442+0000 7fcde5000640 1 -- 172.21.15.151:0/516029325 shutdown_connections 2024-09-16T15:55:07.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.442+0000 7fcde5000640 1 --2- 172.21.15.151:0/516029325 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcde00ccf00 0x7fcde00b2440 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:07.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.442+0000 7fcde5000640 1 -- 172.21.15.151:0/516029325 >> 172.21.15.151:0/516029325 conn(0x7fcde00b1950 msgr2=0x7fcde00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:55:07.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.443+0000 7fcde5000640 1 -- 172.21.15.151:0/516029325 shutdown_connections 2024-09-16T15:55:07.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.443+0000 7fcde5000640 1 -- 172.21.15.151:0/516029325 wait complete. 2024-09-16T15:55:07.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.444+0000 7fcde5000640 1 Processor -- start 2024-09-16T15:55:07.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.444+0000 7fcde5000640 1 -- start start 2024-09-16T15:55:07.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.445+0000 7fcde5000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcde0073fb0 0x7fcde0158f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:07.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.445+0000 7fcde5000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcde00ccf00 0x7fcde015de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:07.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.445+0000 7fcde5000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcde015f5a0 0x7fcde01f2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:07.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.445+0000 7fcde5000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcde006ef30 con 0x7fcde015f5a0 2024-09-16T15:55:07.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.445+0000 7fcde5000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcde006fa80 con 0x7fcde00ccf00 2024-09-16T15:55:07.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.445+0000 7fcde5000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fcde0074440 con 0x7fcde0073fb0 2024-09-16T15:55:07.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.445+0000 7fcdde800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcde015f5a0 0x7fcde01f2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:07.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.445+0000 7fcdde800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcde015f5a0 0x7fcde01f2610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44808/0 (socket says 172.21.15.151:44808) 2024-09-16T15:55:07.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.445+0000 7fcdde800640 1 -- 172.21.15.151:0/3485591870 learned_addr learned my addr 172.21.15.151:0/3485591870 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:55:07.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.446+0000 7fcddde00640 1 --2- 172.21.15.151:0/3485591870 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcde0073fb0 0x7fcde0158f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:07.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.446+0000 7fcddd400640 1 --2- 172.21.15.151:0/3485591870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcde00ccf00 0x7fcde015de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:07.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.446+0000 7fcdde800640 1 -- 172.21.15.151:0/3485591870 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcde0073fb0 msgr2=0x7fcde0158f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:07.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.446+0000 7fcdde800640 1 --2- 172.21.15.151:0/3485591870 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcde0073fb0 0x7fcde0158f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:07.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.446+0000 7fcdde800640 1 -- 172.21.15.151:0/3485591870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcde00ccf00 msgr2=0x7fcde015de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:07.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.446+0000 7fcdde800640 1 --2- 172.21.15.151:0/3485591870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcde00ccf00 0x7fcde015de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:07.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.446+0000 7fcdde800640 1 -- 172.21.15.151:0/3485591870 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcde0156130 con 0x7fcde015f5a0 2024-09-16T15:55:07.449 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.446+0000 7fcdde800640 1 --2- 172.21.15.151:0/3485591870 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcde015f5a0 0x7fcde01f2610 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7fcdd800c400 tx=0x7fcdd800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:55:07.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.446+0000 7fcdd6000640 1 -- 172.21.15.151:0/3485591870 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd8011790 con 0x7fcde015f5a0 2024-09-16T15:55:07.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.446+0000 7fcdd6000640 1 -- 172.21.15.151:0/3485591870 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcdd8017070 con 0x7fcde015f5a0 2024-09-16T15:55:07.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.446+0000 7fcde5000640 1 -- 172.21.15.151:0/3485591870 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcde0156430 con 0x7fcde015f5a0 2024-09-16T15:55:07.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.446+0000 7fcdd6000640 1 -- 172.21.15.151:0/3485591870 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd8011ce0 con 0x7fcde015f5a0 2024-09-16T15:55:07.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.447+0000 7fcde5000640 1 -- 172.21.15.151:0/3485591870 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcde015c7d0 con 0x7fcde015f5a0 2024-09-16T15:55:07.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.448+0000 7fcde5000640 1 -- 172.21.15.151:0/3485591870 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcde00b2440 con 0x7fcde015f5a0 2024-09-16T15:55:07.450 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.449+0000 7fcdd6000640 1 -- 172.21.15.151:0/3485591870 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 50) ==== 104398+0+0 (secure 0 0 0) 0x7fcdd801c460 con 0x7fcde015f5a0 2024-09-16T15:55:07.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.450+0000 7fcdd6000640 1 --2- 172.21.15.151:0/3485591870 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcdc007aaa0 0x7fcdc007cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:07.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.450+0000 7fcdd6000640 1 -- 172.21.15.151:0/3485591870 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5907+0+0 (secure 0 0 0) 0x7fcdd8099a00 con 0x7fcde015f5a0 2024-09-16T15:55:07.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.451+0000 7fcddde00640 1 --2- 172.21.15.151:0/3485591870 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcdc007aaa0 0x7fcdc007cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:07.453 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.452+0000 7fcddde00640 1 --2- 172.21.15.151:0/3485591870 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcdc007aaa0 0x7fcdc007cf60 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fcdd0007fd0 tx=0x7fcdd0009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:55:07.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.455+0000 7fcdd6000640 1 -- 172.21.15.151:0/3485591870 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fcde0156430 con 0x7fcde015f5a0 2024-09-16T15:55:07.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.681+0000 7fcde5000640 1 -- 172.21.15.151:0/3485591870 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fcde0155ce0 con 0x7fcde015f5a0 2024-09-16T15:55:07.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.681+0000 7fcdd6000640 1 -- 172.21.15.151:0/3485591870 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v96) ==== 53+0+73 (secure 0 0 0) 0x7fcdd8063a50 con 0x7fcde015f5a0 2024-09-16T15:55:07.683 INFO:teuthology.orchestra.run.smithi151.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-16T15:55:07.685 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.683+0000 7fcde5000640 1 -- 172.21.15.151:0/3485591870 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcdc007aaa0 msgr2=0x7fcdc007cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:07.685 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.683+0000 7fcde5000640 1 --2- 172.21.15.151:0/3485591870 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcdc007aaa0 0x7fcdc007cf60 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fcdd0007fd0 tx=0x7fcdd0009040 comp rx=0 tx=0).stop 2024-09-16T15:55:07.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.683+0000 7fcde5000640 1 -- 172.21.15.151:0/3485591870 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcde015f5a0 msgr2=0x7fcde01f2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:07.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.683+0000 7fcde5000640 1 --2- 172.21.15.151:0/3485591870 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcde015f5a0 0x7fcde01f2610 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7fcdd800c400 tx=0x7fcdd800c8c0 comp rx=0 tx=0).stop 2024-09-16T15:55:07.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.683+0000 7fcde5000640 1 -- 172.21.15.151:0/3485591870 shutdown_connections 2024-09-16T15:55:07.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.683+0000 7fcde5000640 1 --2- 172.21.15.151:0/3485591870 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcdc007aaa0 0x7fcdc007cf60 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:07.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.683+0000 7fcde5000640 1 --2- 172.21.15.151:0/3485591870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcde00ccf00 0x7fcde015de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:07.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.683+0000 7fcde5000640 1 --2- 172.21.15.151:0/3485591870 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcde015f5a0 0x7fcde01f2610 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:07.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.683+0000 7fcde5000640 1 --2- 172.21.15.151:0/3485591870 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcde0073fb0 0x7fcde0158f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:07.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.683+0000 7fcde5000640 1 -- 172.21.15.151:0/3485591870 >> 172.21.15.151:0/3485591870 conn(0x7fcde00b1950 msgr2=0x7fcde00af580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:55:07.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.683+0000 7fcde5000640 1 -- 172.21.15.151:0/3485591870 shutdown_connections 2024-09-16T15:55:07.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07.683+0000 7fcde5000640 1 -- 172.21.15.151:0/3485591870 wait complete. 2024-09-16T15:55:07.702 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-16T15:55:07.702 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T15:55:07.702 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi151.front.sepia.ceph.com 2024-09-16T15:55:07.702 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-16T15:55:07.702 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-16T15:55:07.702 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-16T15:55:07.702 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-16T15:55:07.702 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T15:55:07.702 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T15:55:07.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:55:07.702 DEBUG:teuthology.orchestra.run.smithi151:> ip addr 2024-09-16T15:55:07.738 INFO:teuthology.orchestra.run.smithi151.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T15:55:07.738 INFO:teuthology.orchestra.run.smithi151.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T15:55:07.738 INFO:teuthology.orchestra.run.smithi151.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T15:55:07.738 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:55:07.738 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 ::1/128 scope host 2024-09-16T15:55:07.739 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:55:07.739 INFO:teuthology.orchestra.run.smithi151.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:55:07.739 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:da:6e:0c brd ff:ff:ff:ff:ff:ff 2024-09-16T15:55:07.739 INFO:teuthology.orchestra.run.smithi151.stdout: altname enp7s0f0 2024-09-16T15:55:07.739 INFO:teuthology.orchestra.run.smithi151.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:55:07.739 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:da:6e:0d brd ff:ff:ff:ff:ff:ff 2024-09-16T15:55:07.739 INFO:teuthology.orchestra.run.smithi151.stdout: altname enp7s0f1 2024-09-16T15:55:07.739 INFO:teuthology.orchestra.run.smithi151.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T15:55:07.739 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:88:6f:62 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:55:07.739 INFO:teuthology.orchestra.run.smithi151.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T15:55:07.739 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 0c:c4:7a:88:6f:63 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:55:07.739 INFO:teuthology.orchestra.run.smithi151.stdout: inet 172.21.15.151/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-16T15:55:07.739 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft 41782sec preferred_lft 41782sec 2024-09-16T15:55:07.739 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 fe80::ec4:7aff:fe88:6f63/64 scope link noprefixroute 2024-09-16T15:55:07.740 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:55:07.740 INFO:teuthology.orchestra.run.smithi151.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-16T15:55:07.740 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 62:3f:3b:48:ad:f8 brd ff:ff:ff:ff:ff:ff 2024-09-16T15:55:07.740 INFO:teuthology.orchestra.run.smithi151.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-16T15:55:07.740 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:55:07.740 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 fe80::40fa:7fff:fedf:2a00/64 scope link 2024-09-16T15:55:07.740 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:55:07.740 INFO:teuthology.orchestra.run.smithi151.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T15:55:07.740 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether 62:3f:3b:48:ad:f8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T15:55:07.740 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 fe80::603f:3bff:fe48:adf8/64 scope link 2024-09-16T15:55:07.740 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:55:07.740 INFO:teuthology.orchestra.run.smithi151.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-16T15:55:07.740 INFO:teuthology.orchestra.run.smithi151.stdout: link/ether b6:ac:fe:34:a7:66 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-16T15:55:07.741 INFO:teuthology.orchestra.run.smithi151.stdout: inet6 fe80::b4ac:feff:fe34:a766/64 scope link 2024-09-16T15:55:07.741 INFO:teuthology.orchestra.run.smithi151.stdout: valid_lft forever preferred_lft forever 2024-09-16T15:55:07.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:55:07.742 DEBUG:teuthology.orchestra.run.smithi151:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T15:55:07.835 INFO:teuthology.orchestra.run.smithi151.stdout:1 2024-09-16T15:55:07.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:55:07.838 DEBUG:teuthology.orchestra.run.smithi151:> route 2024-09-16T15:55:07.874 INFO:teuthology.orchestra.run.smithi151.stdout:Kernel IP routing table 2024-09-16T15:55:07.874 INFO:teuthology.orchestra.run.smithi151.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T15:55:07.874 INFO:teuthology.orchestra.run.smithi151.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-16T15:55:07.874 INFO:teuthology.orchestra.run.smithi151.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-16T15:55:07.874 INFO:teuthology.orchestra.run.smithi151.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T15:55:07.875 DEBUG:teuthology.orchestra.run.smithi151:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T15:55:07.875 DEBUG:teuthology.orchestra.run.smithi151:> set -e 2024-09-16T15:55:07.875 DEBUG:teuthology.orchestra.run.smithi151:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-16T15:55:07.875 DEBUG:teuthology.orchestra.run.smithi151:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-16T15:55:07.875 DEBUG:teuthology.orchestra.run.smithi151:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-16T15:55:07.875 DEBUG:teuthology.orchestra.run.smithi151:> ') 2024-09-16T15:55:07.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T15:55:08.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:55:08.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:55:08.071 DEBUG:teuthology.orchestra.run.smithi151:> ip netns list 2024-09-16T15:55:08.106 INFO:teuthology.orchestra.run.smithi151.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-16T15:55:08.106 INFO:teuthology.orchestra.run.smithi151.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-16T15:55:08.107 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T15:55:08.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:55:08.107 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip link set brx.0 up 2024-09-16T15:55:08.203 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:55:08.204 DEBUG:teuthology.orchestra.run.smithi151:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-16T15:55:08.239 INFO:teuthology.orchestra.run.smithi151.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-16T15:55:08.239 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T15:55:08.239 DEBUG:teuthology.orchestra.run.smithi151:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-16T15:55:08.313 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-16T15:55:08.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:55:08.314 DEBUG:teuthology.orchestra.run.smithi151:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-16T15:55:08.442 INFO:teuthology.orchestra.run.smithi151.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-16T15:55:08.442 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-16T15:55:08.442 INFO:teuthology.orchestra.run.smithi151.stdout:invalid new device string format 2024-09-16T15:55:08.458 INFO:teuthology.orchestra.run.smithi151.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-16T15:55:08.458 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-16T15:55:08.458 INFO:teuthology.orchestra.run.smithi151.stdout:invalid new device string format 2024-09-16T15:55:08.458 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: resolved to: "172.21.15.151:3300,172.21.15.167:3300,172.21.15.167:3301" 2024-09-16T15:55:08.458 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-16T15:55:08.483 INFO:teuthology.orchestra.run.smithi151.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-16T15:55:08.483 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-16T15:55:08.483 INFO:teuthology.orchestra.run.smithi151.stdout:invalid new device string format 2024-09-16T15:55:08.483 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: resolved to: "172.21.15.151:3300,172.21.15.167:3300,172.21.15.167:3301" 2024-09-16T15:55:08.483 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-16T15:55:08.483 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: resolved to: "172.21.15.151:6789,172.21.15.167:6789,172.21.15.167:6790" 2024-09-16T15:55:08.483 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.151:6789,172.21.15.167:6789,172.21.15.167:6790:/ 2024-09-16T15:55:08.483 INFO:teuthology.orchestra.run.smithi151.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=1757be18-e41a-4446-bb4f-ee4ac81c39df" will pass to kernel 2024-09-16T15:55:08.485 INFO:teuthology.orchestra.run.smithi151.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-16T15:55:08.485 INFO:teuthology.orchestra.run.smithi151.stdout: You just mounted a file system that supports labels which does not 2024-09-16T15:55:08.485 INFO:teuthology.orchestra.run.smithi151.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-16T15:55:08.485 INFO:teuthology.orchestra.run.smithi151.stdout: applications will generate AVC messages and not be allowed access to 2024-09-16T15:55:08.485 INFO:teuthology.orchestra.run.smithi151.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-16T15:55:08.488 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-16T15:55:08.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:55:08.488 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-16T15:55:08.544 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T15:55:08.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:08 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:55:08.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:55:08.647 INFO:teuthology.orchestra.run.smithi151.stdout:client8782 2024-09-16T15:55:08.647 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:55:08.647 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:55:08.682 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-16T15:55:08.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:08.885+0000 7f141a600640 1 -- 172.21.15.151:0/1602365614 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f141415cff0 msgr2=0x7f141406e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:08.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:08.885+0000 7f141a600640 1 --2- 172.21.15.151:0/1602365614 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f141415cff0 0x7f141406e6f0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f13fc009830 tx=0x7f13fc02a510 comp rx=0 tx=0).stop 2024-09-16T15:55:08.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:08.885+0000 7f141a600640 1 -- 172.21.15.151:0/1602365614 shutdown_connections 2024-09-16T15:55:08.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:08.885+0000 7f141a600640 1 --2- 172.21.15.151:0/1602365614 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f141406f220 0x7f141406f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:08.887 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:08.885+0000 7f141a600640 1 --2- 172.21.15.151:0/1602365614 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f141415cff0 0x7f141406e6f0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:08.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:08.885+0000 7f141a600640 1 -- 172.21.15.151:0/1602365614 >> 172.21.15.151:0/1602365614 conn(0x7f14140b18e0 msgr2=0x7f14140b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:55:08.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:08.885+0000 7f141a600640 1 -- 172.21.15.151:0/1602365614 shutdown_connections 2024-09-16T15:55:08.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:08.885+0000 7f141a600640 1 -- 172.21.15.151:0/1602365614 wait complete. 2024-09-16T15:55:08.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:08.886+0000 7f141a600640 1 Processor -- start 2024-09-16T15:55:08.888 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:08.887+0000 7f141a600640 1 -- start start 2024-09-16T15:55:08.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:08.887+0000 7f141a600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f141406f220 0x7f141414d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:08.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:08.888+0000 7f141a600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1414159aa0 0x7f14140ce550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:08.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:08.888+0000 7f141a600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f141415cff0 0x7f14140cea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:08.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:08.888+0000 7f141a600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f14140cc520 con 0x7f141406f220 2024-09-16T15:55:08.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:08.888+0000 7f141a600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f14140cc3a0 con 0x7f1414159aa0 2024-09-16T15:55:08.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:08.888+0000 7f141a600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f14140cc6a0 con 0x7f141415cff0 2024-09-16T15:55:08.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:08.888+0000 7f1413400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f141406f220 0x7f141414d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:08.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:08.888+0000 7f1413400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f141406f220 0x7f141414d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44820/0 (socket says 172.21.15.151:44820) 2024-09-16T15:55:08.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:08.888+0000 7f1413400640 1 -- 172.21.15.151:0/3140131629 learned_addr learned my addr 172.21.15.151:0/3140131629 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:55:08.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:08.888+0000 7f1413e00640 1 --2- 172.21.15.151:0/3140131629 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f141415cff0 0x7f14140cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:08.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:08.888+0000 7f140be00640 1 --2- 172.21.15.151:0/3140131629 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1414159aa0 0x7f14140ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:08.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:08.888+0000 7f1413400640 1 -- 172.21.15.151:0/3140131629 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f141415cff0 msgr2=0x7f14140cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:08.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:08.888+0000 7f1413400640 1 --2- 172.21.15.151:0/3140131629 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f141415cff0 0x7f14140cea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:08.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:08.888+0000 7f1413400640 1 -- 172.21.15.151:0/3140131629 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1414159aa0 msgr2=0x7f14140ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:08.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:08.888+0000 7f1413400640 1 --2- 172.21.15.151:0/3140131629 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1414159aa0 0x7f14140ce550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:08.891 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:08.888+0000 7f1413400640 1 -- 172.21.15.151:0/3140131629 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14140ad970 con 0x7f141406f220 2024-09-16T15:55:08.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:08.889+0000 7f1413400640 1 --2- 172.21.15.151:0/3140131629 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f141406f220 0x7f141414d650 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f140400c480 tx=0x7f140400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:55:08.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:08.889+0000 7f140b400640 1 -- 172.21.15.151:0/3140131629 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1404016020 con 0x7f141406f220 2024-09-16T15:55:08.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:08.889+0000 7f140b400640 1 -- 172.21.15.151:0/3140131629 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1404012920 con 0x7f141406f220 2024-09-16T15:55:08.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:08.889+0000 7f140b400640 1 -- 172.21.15.151:0/3140131629 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1404012d70 con 0x7f141406f220 2024-09-16T15:55:08.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:08.889+0000 7f141a600640 1 -- 172.21.15.151:0/3140131629 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14140adc50 con 0x7f141406f220 2024-09-16T15:55:08.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:08.889+0000 7f141a600640 1 -- 172.21.15.151:0/3140131629 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14140af230 con 0x7f141406f220 2024-09-16T15:55:08.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:08.890+0000 7f141a600640 1 -- 172.21.15.151:0/3140131629 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1414069360 con 0x7f141406f220 2024-09-16T15:55:08.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:08.891+0000 7f140b400640 1 -- 172.21.15.151:0/3140131629 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 50) ==== 104398+0+0 (secure 0 0 0) 0x7f1404019020 con 0x7f141406f220 2024-09-16T15:55:08.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:08.893+0000 7f140b400640 1 --2- 172.21.15.151:0/3140131629 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f13f007aaa0 0x7f13f007cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:08.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:08.893+0000 7f140b400640 1 -- 172.21.15.151:0/3140131629 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5907+0+0 (secure 0 0 0) 0x7f1404099a30 con 0x7f141406f220 2024-09-16T15:55:08.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:08.893+0000 7f140be00640 1 --2- 172.21.15.151:0/3140131629 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f13f007aaa0 0x7f13f007cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:08.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:08.894+0000 7f140be00640 1 --2- 172.21.15.151:0/3140131629 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f13f007aaa0 0x7f13f007cf60 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f13fc034000 tx=0x7f13fc004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:55:08.899 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:08.897+0000 7f140b400640 1 -- 172.21.15.151:0/3140131629 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1404020020 con 0x7f141406f220 2024-09-16T15:55:09.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:09.119+0000 7f141a600640 1 -- 172.21.15.151:0/3140131629 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f14140aef50 con 0x7f141406f220 2024-09-16T15:55:09.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:09.524+0000 7f140b400640 1 -- 172.21.15.151:0/3140131629 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v97) ==== 136+0+0 (secure 0 0 0) 0x7f14140aef50 con 0x7f141406f220 2024-09-16T15:55:09.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:09.526+0000 7f141a600640 1 -- 172.21.15.151:0/3140131629 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f13f007aaa0 msgr2=0x7f13f007cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:09.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:09.526+0000 7f141a600640 1 --2- 172.21.15.151:0/3140131629 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f13f007aaa0 0x7f13f007cf60 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f13fc034000 tx=0x7f13fc004730 comp rx=0 tx=0).stop 2024-09-16T15:55:09.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:09.526+0000 7f141a600640 1 -- 172.21.15.151:0/3140131629 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f141406f220 msgr2=0x7f141414d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:09.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:09.526+0000 7f141a600640 1 --2- 172.21.15.151:0/3140131629 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f141406f220 0x7f141414d650 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f140400c480 tx=0x7f140400c940 comp rx=0 tx=0).stop 2024-09-16T15:55:09.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:09.526+0000 7f141a600640 1 -- 172.21.15.151:0/3140131629 shutdown_connections 2024-09-16T15:55:09.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:09.526+0000 7f141a600640 1 --2- 172.21.15.151:0/3140131629 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f13f007aaa0 0x7f13f007cf60 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:09.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:09.526+0000 7f141a600640 1 --2- 172.21.15.151:0/3140131629 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1414159aa0 0x7f14140ce550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:09.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:09.526+0000 7f141a600640 1 --2- 172.21.15.151:0/3140131629 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f141415cff0 0x7f14140cea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:09.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:09.526+0000 7f141a600640 1 --2- 172.21.15.151:0/3140131629 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f141406f220 0x7f141414d650 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:09.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:09.526+0000 7f141a600640 1 -- 172.21.15.151:0/3140131629 >> 172.21.15.151:0/3140131629 conn(0x7f14140b18e0 msgr2=0x7f141415dad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:55:09.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:09.526+0000 7f141a600640 1 -- 172.21.15.151:0/3140131629 shutdown_connections 2024-09-16T15:55:09.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:09.526+0000 7f141a600640 1 -- 172.21.15.151:0/3140131629 wait complete. 2024-09-16T15:55:09.545 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-16T15:55:09.703 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:09.701+0000 7f0c13400640 1 -- 172.21.15.151:0/2640772 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c0c15f5a0 msgr2=0x7f0c0c0b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:09.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:09.701+0000 7f0c13400640 1 --2- 172.21.15.151:0/2640772 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c0c15f5a0 0x7f0c0c0b2440 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c06dac0 tx=0x7f0bfc02a4b0 comp rx=0 tx=0).stop 2024-09-16T15:55:09.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:09.701+0000 7f0c13400640 1 -- 172.21.15.151:0/2640772 shutdown_connections 2024-09-16T15:55:09.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:09.701+0000 7f0c13400640 1 --2- 172.21.15.151:0/2640772 >> v2:172.21.15.167:3300/0 conn(0x7f0c0c073fb0 0x7f0c0c074420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:09.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:09.701+0000 7f0c13400640 1 --2- 172.21.15.151:0/2640772 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c0c15f5a0 0x7f0c0c0b2440 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:09.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:09.701+0000 7f0c13400640 1 -- 172.21.15.151:0/2640772 >> 172.21.15.151:0/2640772 conn(0x7f0c0c0b1950 msgr2=0x7f0c0c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:55:09.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:09.701+0000 7f0c13400640 1 -- 172.21.15.151:0/2640772 shutdown_connections 2024-09-16T15:55:09.704 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:09.701+0000 7f0c13400640 1 -- 172.21.15.151:0/2640772 wait complete. 2024-09-16T15:55:09.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:09.703+0000 7f0c13400640 1 Processor -- start 2024-09-16T15:55:09.705 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:09.703+0000 7f0c13400640 1 -- start start 2024-09-16T15:55:09.706 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:09.704+0000 7f0c13400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c0c073fb0 0x7f0c0c152970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:09.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:09.704+0000 7f0c13400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c0c0ccf00 0x7f0c0c152eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:09.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:09.704+0000 7f0c13400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c0c15f5a0 0x7f0c0c1f2b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:09.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:09.704+0000 7f0c13400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0c0c06f2e0 con 0x7f0c0c073fb0 2024-09-16T15:55:09.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:09.704+0000 7f0c13400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0c0c06f160 con 0x7f0c0c15f5a0 2024-09-16T15:55:09.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:09.704+0000 7f0c13400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f0c0c06f460 con 0x7f0c0c0ccf00 2024-09-16T15:55:09.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:09.704+0000 7f0c10a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c0c073fb0 0x7f0c0c152970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:09.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:09.704+0000 7f0c10a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c0c073fb0 0x7f0c0c152970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44838/0 (socket says 172.21.15.151:44838) 2024-09-16T15:55:09.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:09.704+0000 7f0c10a00640 1 -- 172.21.15.151:0/3960543300 learned_addr learned my addr 172.21.15.151:0/3960543300 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:55:09.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:09.704+0000 7f0c03e00640 1 --2- 172.21.15.151:0/3960543300 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c0c0ccf00 0x7f0c0c152eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:09.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:09.704+0000 7f0c11400640 1 --2- 172.21.15.151:0/3960543300 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c0c15f5a0 0x7f0c0c1f2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:09.707 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:09.704+0000 7f0c10a00640 1 -- 172.21.15.151:0/3960543300 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c0c0ccf00 msgr2=0x7f0c0c152eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:09.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:09.704+0000 7f0c10a00640 1 --2- 172.21.15.151:0/3960543300 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c0c0ccf00 0x7f0c0c152eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:09.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:09.704+0000 7f0c10a00640 1 -- 172.21.15.151:0/3960543300 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c0c15f5a0 msgr2=0x7f0c0c1f2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:09.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:09.704+0000 7f0c10a00640 1 --2- 172.21.15.151:0/3960543300 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c0c15f5a0 0x7f0c0c1f2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:09.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:09.704+0000 7f0c10a00640 1 -- 172.21.15.151:0/3960543300 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c0c150ea0 con 0x7f0c0c073fb0 2024-09-16T15:55:09.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:09.704+0000 7f0c11400640 1 --2- 172.21.15.151:0/3960543300 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c0c15f5a0 0x7f0c0c1f2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:55:09.708 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:09.705+0000 7f0c10a00640 1 --2- 172.21.15.151:0/3960543300 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c0c073fb0 0x7f0c0c152970 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc009920 tx=0x7f0bfc033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:55:09.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:09.705+0000 7f0c09600640 1 -- 172.21.15.151:0/3960543300 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc031d60 con 0x7f0c0c073fb0 2024-09-16T15:55:09.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:09.705+0000 7f0c09600640 1 -- 172.21.15.151:0/3960543300 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0bfc033e10 con 0x7f0c0c073fb0 2024-09-16T15:55:09.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:09.705+0000 7f0c13400640 1 -- 172.21.15.151:0/3960543300 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c0c151180 con 0x7f0c0c073fb0 2024-09-16T15:55:09.709 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:09.705+0000 7f0c09600640 1 -- 172.21.15.151:0/3960543300 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc005260 con 0x7f0c0c073fb0 2024-09-16T15:55:09.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:09.705+0000 7f0c13400640 1 -- 172.21.15.151:0/3960543300 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c0c151740 con 0x7f0c0c073fb0 2024-09-16T15:55:09.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:09.706+0000 7f0c13400640 1 -- 172.21.15.151:0/3960543300 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c0c06f9f0 con 0x7f0c0c073fb0 2024-09-16T15:55:09.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:09.707+0000 7f0c09600640 1 -- 172.21.15.151:0/3960543300 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 50) ==== 104398+0+0 (secure 0 0 0) 0x7f0bfc005400 con 0x7f0c0c073fb0 2024-09-16T15:55:09.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:09.708+0000 7f0c09600640 1 --2- 172.21.15.151:0/3960543300 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0be807aaa0 0x7f0be807cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:09.710 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:09.709+0000 7f0c09600640 1 -- 172.21.15.151:0/3960543300 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5907+0+0 (secure 0 0 0) 0x7f0bfc0bab40 con 0x7f0c0c073fb0 2024-09-16T15:55:09.711 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:09.709+0000 7f0c03e00640 1 --2- 172.21.15.151:0/3960543300 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0be807aaa0 0x7f0be807cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:09.711 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:09.710+0000 7f0c03e00640 1 --2- 172.21.15.151:0/3960543300 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0be807aaa0 0x7f0be807cf60 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c14ffd0 tx=0x7f0bf4007e90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:55:09.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:09.712+0000 7f0c09600640 1 -- 172.21.15.151:0/3960543300 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0bfc0c1e20 con 0x7f0c0c073fb0 2024-09-16T15:55:09.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:09.919+0000 7f0c13400640 1 -- 172.21.15.151:0/3960543300 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f0c0c071190 con 0x7f0be807aaa0 2024-09-16T15:55:09.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:09.922+0000 7f0c09600640 1 -- 172.21.15.151:0/3960543300 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f0c0c071190 con 0x7f0be807aaa0 2024-09-16T15:55:09.923 INFO:teuthology.orchestra.run.smithi151.stdout:[ 2024-09-16T15:55:09.924 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:55:09.924 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "cephfs" 2024-09-16T15:55:09.924 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:55:09.924 INFO:teuthology.orchestra.run.smithi151.stdout:] 2024-09-16T15:55:09.926 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:09.923+0000 7f0c13400640 1 -- 172.21.15.151:0/3960543300 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0be807aaa0 msgr2=0x7f0be807cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:09.926 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:09.923+0000 7f0c13400640 1 --2- 172.21.15.151:0/3960543300 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0be807aaa0 0x7f0be807cf60 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c14ffd0 tx=0x7f0bf4007e90 comp rx=0 tx=0).stop 2024-09-16T15:55:09.926 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:09.924+0000 7f0c13400640 1 -- 172.21.15.151:0/3960543300 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c0c073fb0 msgr2=0x7f0c0c152970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:09.926 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:09.924+0000 7f0c13400640 1 --2- 172.21.15.151:0/3960543300 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c0c073fb0 0x7f0c0c152970 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc009920 tx=0x7f0bfc033c20 comp rx=0 tx=0).stop 2024-09-16T15:55:09.926 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:09.924+0000 7f0c13400640 1 -- 172.21.15.151:0/3960543300 shutdown_connections 2024-09-16T15:55:09.926 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:09.924+0000 7f0c13400640 1 --2- 172.21.15.151:0/3960543300 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0be807aaa0 0x7f0be807cf60 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:09.926 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:09.924+0000 7f0c13400640 1 --2- 172.21.15.151:0/3960543300 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c0c15f5a0 0x7f0c0c1f2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:09.926 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:09.924+0000 7f0c13400640 1 --2- 172.21.15.151:0/3960543300 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0c0c0ccf00 0x7f0c0c152eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:09.926 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:09.924+0000 7f0c13400640 1 --2- 172.21.15.151:0/3960543300 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c0c073fb0 0x7f0c0c152970 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:09.926 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:09.924+0000 7f0c13400640 1 -- 172.21.15.151:0/3960543300 >> 172.21.15.151:0/3960543300 conn(0x7f0c0c0b1950 msgr2=0x7f0c0c0b0c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:55:09.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:09.924+0000 7f0c13400640 1 -- 172.21.15.151:0/3960543300 shutdown_connections 2024-09-16T15:55:09.927 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:09.924+0000 7f0c13400640 1 -- 172.21.15.151:0/3960543300 wait complete. 2024-09-16T15:55:09.942 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-16T15:55:10.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.101+0000 7fcffb200640 1 -- 172.21.15.151:0/2907569486 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfd8005280 msgr2=0x7fcfd800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:10.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.101+0000 7fcffb200640 1 --2- 172.21.15.151:0/2907569486 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfd8005280 0x7fcfd800b9c0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7fcfec006f90 tx=0x7fcfd8005830 comp rx=0 tx=0).stop 2024-09-16T15:55:10.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.102+0000 7fcffb200640 1 -- 172.21.15.151:0/2907569486 shutdown_connections 2024-09-16T15:55:10.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.102+0000 7fcffb200640 1 --2- 172.21.15.151:0/2907569486 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfd8005280 0x7fcfd800b9c0 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:10.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.102+0000 7fcffb200640 1 -- 172.21.15.151:0/2907569486 >> 172.21.15.151:0/2907569486 conn(0x7fcff40b1840 msgr2=0x7fcff40b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:55:10.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.102+0000 7fcffb200640 1 -- 172.21.15.151:0/2907569486 shutdown_connections 2024-09-16T15:55:10.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.102+0000 7fcffb200640 1 -- 172.21.15.151:0/2907569486 wait complete. 2024-09-16T15:55:10.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.103+0000 7fcffb200640 1 Processor -- start 2024-09-16T15:55:10.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.103+0000 7fcffb200640 1 -- start start 2024-09-16T15:55:10.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.104+0000 7fcffb200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfd8005280 0x7fcff4141ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:10.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.104+0000 7fcffb200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcff41f80e0 0x7fcff41f8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:10.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.104+0000 7fcffb200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcff41f8a80 0x7fcff41f4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:10.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.104+0000 7fcffb200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcff40737d0 con 0x7fcfd8005280 2024-09-16T15:55:10.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.104+0000 7fcffb200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcff4073650 con 0x7fcff41f80e0 2024-09-16T15:55:10.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.104+0000 7fcffb200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fcff40cc5c0 con 0x7fcff41f8a80 2024-09-16T15:55:10.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.104+0000 7fcff3e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfd8005280 0x7fcff4141ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:10.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.105+0000 7fcff3e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfd8005280 0x7fcff4141ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44842/0 (socket says 172.21.15.151:44842) 2024-09-16T15:55:10.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.105+0000 7fcff3e00640 1 -- 172.21.15.151:0/3688139820 learned_addr learned my addr 172.21.15.151:0/3688139820 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:55:10.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.105+0000 7fcff9200640 1 --2- 172.21.15.151:0/3688139820 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcff41f8a80 0x7fcff41f4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:10.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.105+0000 7fcff3400640 1 --2- 172.21.15.151:0/3688139820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcff41f80e0 0x7fcff41f8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:10.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.105+0000 7fcff3e00640 1 -- 172.21.15.151:0/3688139820 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcff41f8a80 msgr2=0x7fcff41f4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:10.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.105+0000 7fcff3e00640 1 --2- 172.21.15.151:0/3688139820 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcff41f8a80 0x7fcff41f4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:10.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.105+0000 7fcff3e00640 1 -- 172.21.15.151:0/3688139820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcff41f80e0 msgr2=0x7fcff41f8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:10.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.105+0000 7fcff3e00640 1 --2- 172.21.15.151:0/3688139820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcff41f80e0 0x7fcff41f8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:10.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.105+0000 7fcff3e00640 1 -- 172.21.15.151:0/3688139820 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcff41e7da0 con 0x7fcfd8005280 2024-09-16T15:55:10.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.105+0000 7fcff3400640 1 --2- 172.21.15.151:0/3688139820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcff41f80e0 0x7fcff41f8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:55:10.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.105+0000 7fcff3e00640 1 --2- 172.21.15.151:0/3688139820 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfd8005280 0x7fcff4141ac0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4011c40 tx=0x7fcfe400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:55:10.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.105+0000 7fcfebe00640 1 -- 172.21.15.151:0/3688139820 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe4003e40 con 0x7fcfd8005280 2024-09-16T15:55:10.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.106+0000 7fcfebe00640 1 -- 172.21.15.151:0/3688139820 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcfe4004b20 con 0x7fcfd8005280 2024-09-16T15:55:10.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.106+0000 7fcffb200640 1 -- 172.21.15.151:0/3688139820 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcff41f8f10 con 0x7fcfd8005280 2024-09-16T15:55:10.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.106+0000 7fcfebe00640 1 -- 172.21.15.151:0/3688139820 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe4004050 con 0x7fcfd8005280 2024-09-16T15:55:10.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.106+0000 7fcffb200640 1 -- 172.21.15.151:0/3688139820 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcff41447b0 con 0x7fcfd8005280 2024-09-16T15:55:10.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.107+0000 7fcffb200640 1 -- 172.21.15.151:0/3688139820 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcff415f9f0 con 0x7fcfd8005280 2024-09-16T15:55:10.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.108+0000 7fcfebe00640 1 -- 172.21.15.151:0/3688139820 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 50) ==== 104398+0+0 (secure 0 0 0) 0x7fcfe40041f0 con 0x7fcfd8005280 2024-09-16T15:55:10.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.109+0000 7fcfebe00640 1 --2- 172.21.15.151:0/3688139820 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcfd407aaa0 0x7fcfd407cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:10.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.109+0000 7fcfebe00640 1 -- 172.21.15.151:0/3688139820 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5907+0+0 (secure 0 0 0) 0x7fcfe409af00 con 0x7fcfd8005280 2024-09-16T15:55:10.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.110+0000 7fcff3400640 1 --2- 172.21.15.151:0/3688139820 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcfd407aaa0 0x7fcfd407cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:10.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.110+0000 7fcff3400640 1 --2- 172.21.15.151:0/3688139820 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcfd407aaa0 0x7fcfd407cf60 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fcff415eef0 tx=0x7fcfe0006130 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:55:10.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.113+0000 7fcfebe00640 1 -- 172.21.15.151:0/3688139820 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fcff41f8f10 con 0x7fcfd8005280 2024-09-16T15:55:10.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.296+0000 7fcffb200640 1 -- 172.21.15.151:0/3688139820 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fcff41e4f70 con 0x7fcfd8005280 2024-09-16T15:55:10.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.297+0000 7fcfebe00640 1 -- 172.21.15.151:0/3688139820 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v54)=0 v54) ==== 122+0+0 (secure 0 0 0) 0x7fcfe4064f50 con 0x7fcfd8005280 2024-09-16T15:55:10.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.299+0000 7fcffb200640 1 -- 172.21.15.151:0/3688139820 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcfd407aaa0 msgr2=0x7fcfd407cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:10.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.299+0000 7fcffb200640 1 --2- 172.21.15.151:0/3688139820 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcfd407aaa0 0x7fcfd407cf60 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fcff415eef0 tx=0x7fcfe0006130 comp rx=0 tx=0).stop 2024-09-16T15:55:10.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.299+0000 7fcffb200640 1 -- 172.21.15.151:0/3688139820 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfd8005280 msgr2=0x7fcff4141ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:10.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.299+0000 7fcffb200640 1 --2- 172.21.15.151:0/3688139820 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfd8005280 0x7fcff4141ac0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4011c40 tx=0x7fcfe400f790 comp rx=0 tx=0).stop 2024-09-16T15:55:10.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.299+0000 7fcffb200640 1 -- 172.21.15.151:0/3688139820 shutdown_connections 2024-09-16T15:55:10.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.299+0000 7fcffb200640 1 --2- 172.21.15.151:0/3688139820 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcfd407aaa0 0x7fcfd407cf60 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:10.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.299+0000 7fcffb200640 1 --2- 172.21.15.151:0/3688139820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcff41f80e0 0x7fcff41f8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:10.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.299+0000 7fcffb200640 1 --2- 172.21.15.151:0/3688139820 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcff41f8a80 0x7fcff41f4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:10.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.299+0000 7fcffb200640 1 --2- 172.21.15.151:0/3688139820 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcfd8005280 0x7fcff4141ac0 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:10.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.299+0000 7fcffb200640 1 -- 172.21.15.151:0/3688139820 >> 172.21.15.151:0/3688139820 conn(0x7fcff40b1840 msgr2=0x7fcff40aeb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:55:10.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.299+0000 7fcffb200640 1 -- 172.21.15.151:0/3688139820 shutdown_connections 2024-09-16T15:55:10.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.300+0000 7fcffb200640 1 -- 172.21.15.151:0/3688139820 wait complete. 2024-09-16T15:55:10.318 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config get mgr mgr/volumes/max_concurrent_clones 2024-09-16T15:55:10.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.475+0000 7fe2a8800640 1 -- 172.21.15.151:0/3588198494 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe2a00af190 msgr2=0x7fe2a00b2b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:10.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.475+0000 7fe2a8800640 1 --2- 172.21.15.151:0/3588198494 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe2a00af190 0x7fe2a00b2b40 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fe290009830 tx=0x7fe29002a500 comp rx=0 tx=0).stop 2024-09-16T15:55:10.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.476+0000 7fe2a8800640 1 -- 172.21.15.151:0/3588198494 shutdown_connections 2024-09-16T15:55:10.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.476+0000 7fe2a8800640 1 --2- 172.21.15.151:0/3588198494 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe2a00cec60 0x7fe2a00cf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:10.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.476+0000 7fe2a8800640 1 --2- 172.21.15.151:0/3588198494 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe2a00af190 0x7fe2a00b2b40 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:10.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.476+0000 7fe2a8800640 1 -- 172.21.15.151:0/3588198494 >> 172.21.15.151:0/3588198494 conn(0x7fe2a00b1840 msgr2=0x7fe2a00b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:55:10.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.476+0000 7fe2a8800640 1 -- 172.21.15.151:0/3588198494 shutdown_connections 2024-09-16T15:55:10.478 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.476+0000 7fe2a8800640 1 -- 172.21.15.151:0/3588198494 wait complete. 2024-09-16T15:55:10.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.477+0000 7fe2a8800640 1 Processor -- start 2024-09-16T15:55:10.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.477+0000 7fe2a8800640 1 -- start start 2024-09-16T15:55:10.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.478+0000 7fe2a8800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2a00af190 0x7fe2a00c4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:10.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.478+0000 7fe2a8800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe2a00cec60 0x7fe2a01f4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:10.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.478+0000 7fe2a8800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe2a0161760 0x7fe2a01f4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:10.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.478+0000 7fe2a8800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a0162310 con 0x7fe2a00cec60 2024-09-16T15:55:10.481 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.478+0000 7fe2a8800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a0162190 con 0x7fe2a00af190 2024-09-16T15:55:10.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.478+0000 7fe2a8800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fe2a0162490 con 0x7fe2a0161760 2024-09-16T15:55:10.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.478+0000 7fe2a5400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe2a00cec60 0x7fe2a01f4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:10.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.479+0000 7fe2a5400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe2a00cec60 0x7fe2a01f4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44858/0 (socket says 172.21.15.151:44858) 2024-09-16T15:55:10.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.479+0000 7fe2a5400640 1 -- 172.21.15.151:0/3193505130 learned_addr learned my addr 172.21.15.151:0/3193505130 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:55:10.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.479+0000 7fe2a5e00640 1 --2- 172.21.15.151:0/3193505130 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2a00af190 0x7fe2a00c4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:10.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.479+0000 7fe2a6800640 1 --2- 172.21.15.151:0/3193505130 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe2a0161760 0x7fe2a01f4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:10.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.479+0000 7fe2a5400640 1 -- 172.21.15.151:0/3193505130 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe2a0161760 msgr2=0x7fe2a01f4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:10.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.479+0000 7fe2a5400640 1 --2- 172.21.15.151:0/3193505130 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe2a0161760 0x7fe2a01f4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:10.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.479+0000 7fe2a5400640 1 -- 172.21.15.151:0/3193505130 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2a00af190 msgr2=0x7fe2a00c4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:10.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.479+0000 7fe2a5400640 1 --2- 172.21.15.151:0/3193505130 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2a00af190 0x7fe2a00c4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:10.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.479+0000 7fe2a5400640 1 -- 172.21.15.151:0/3193505130 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2a01f5390 con 0x7fe2a00cec60 2024-09-16T15:55:10.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.479+0000 7fe2a5400640 1 --2- 172.21.15.151:0/3193505130 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe2a00cec60 0x7fe2a01f4740 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7fe290000c00 tx=0x7fe290032c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:55:10.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.480+0000 7fe29a000640 1 -- 172.21.15.151:0/3193505130 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe290040070 con 0x7fe2a00cec60 2024-09-16T15:55:10.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.480+0000 7fe29a000640 1 -- 172.21.15.151:0/3193505130 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe290032e70 con 0x7fe2a00cec60 2024-09-16T15:55:10.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.480+0000 7fe29a000640 1 -- 172.21.15.151:0/3193505130 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe290003d90 con 0x7fe2a00cec60 2024-09-16T15:55:10.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.480+0000 7fe2a8800640 1 -- 172.21.15.151:0/3193505130 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2a01f5670 con 0x7fe2a00cec60 2024-09-16T15:55:10.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.480+0000 7fe2a8800640 1 -- 172.21.15.151:0/3193505130 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2a01585d0 con 0x7fe2a00cec60 2024-09-16T15:55:10.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.481+0000 7fe2a8800640 1 -- 172.21.15.151:0/3193505130 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2a0162920 con 0x7fe2a00cec60 2024-09-16T15:55:10.484 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.482+0000 7fe29a000640 1 -- 172.21.15.151:0/3193505130 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 50) ==== 104398+0+0 (secure 0 0 0) 0x7fe290039040 con 0x7fe2a00cec60 2024-09-16T15:55:10.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.483+0000 7fe29a000640 1 --2- 172.21.15.151:0/3193505130 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe28407aaa0 0x7fe28407cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:10.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.483+0000 7fe29a000640 1 -- 172.21.15.151:0/3193505130 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5907+0+0 (secure 0 0 0) 0x7fe2900baef0 con 0x7fe2a00cec60 2024-09-16T15:55:10.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.483+0000 7fe2a5e00640 1 --2- 172.21.15.151:0/3193505130 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe28407aaa0 0x7fe28407cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:10.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.484+0000 7fe2a5e00640 1 --2- 172.21.15.151:0/3193505130 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe28407aaa0 0x7fe28407cf60 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fe294001720 tx=0x7fe294007e90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:55:10.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.487+0000 7fe29a000640 1 -- 172.21.15.151:0/3193505130 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe290084970 con 0x7fe2a00cec60 2024-09-16T15:55:10.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.673+0000 7fe2a8800640 1 -- 172.21.15.151:0/3193505130 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"} v 0) -- 0x7fe2a0069a10 con 0x7fe2a00cec60 2024-09-16T15:55:10.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.673+0000 7fe29a000640 1 -- 172.21.15.151:0/3193505130 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/volumes/max_concurrent_clones"}]=0 v54) ==== 116+0+2 (secure 0 0 0) 0x7fe2900893a0 con 0x7fe2a00cec60 2024-09-16T15:55:10.675 INFO:teuthology.orchestra.run.smithi151.stdout:4 2024-09-16T15:55:10.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.675+0000 7fe2a8800640 1 -- 172.21.15.151:0/3193505130 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe28407aaa0 msgr2=0x7fe28407cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:10.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.675+0000 7fe2a8800640 1 --2- 172.21.15.151:0/3193505130 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe28407aaa0 0x7fe28407cf60 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fe294001720 tx=0x7fe294007e90 comp rx=0 tx=0).stop 2024-09-16T15:55:10.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.675+0000 7fe2a8800640 1 -- 172.21.15.151:0/3193505130 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe2a00cec60 msgr2=0x7fe2a01f4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:10.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.675+0000 7fe2a8800640 1 --2- 172.21.15.151:0/3193505130 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe2a00cec60 0x7fe2a01f4740 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7fe290000c00 tx=0x7fe290032c80 comp rx=0 tx=0).stop 2024-09-16T15:55:10.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.676+0000 7fe2a8800640 1 -- 172.21.15.151:0/3193505130 shutdown_connections 2024-09-16T15:55:10.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.676+0000 7fe2a8800640 1 --2- 172.21.15.151:0/3193505130 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe28407aaa0 0x7fe28407cf60 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:10.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.676+0000 7fe2a8800640 1 --2- 172.21.15.151:0/3193505130 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe2a0161760 0x7fe2a01f4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:10.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.676+0000 7fe2a8800640 1 --2- 172.21.15.151:0/3193505130 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe2a00cec60 0x7fe2a01f4740 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:10.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.676+0000 7fe2a8800640 1 --2- 172.21.15.151:0/3193505130 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2a00af190 0x7fe2a00c4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:10.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.676+0000 7fe2a8800640 1 -- 172.21.15.151:0/3193505130 >> 172.21.15.151:0/3193505130 conn(0x7fe2a00b1840 msgr2=0x7fe2a00727f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:55:10.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.676+0000 7fe2a8800640 1 -- 172.21.15.151:0/3193505130 shutdown_connections 2024-09-16T15:55:10.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.676+0000 7fe2a8800640 1 -- 172.21.15.151:0/3193505130 wait complete. 2024-09-16T15:55:10.694 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-16T15:55:10.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.853+0000 7fc49a600640 1 -- 172.21.15.151:0/2479200905 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc49406efa0 msgr2=0x7fc494073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:10.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.853+0000 7fc49a600640 1 --2- 172.21.15.151:0/2479200905 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc49406efa0 0x7fc494073440 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fc48c009830 tx=0x7fc48c02a530 comp rx=0 tx=0).stop 2024-09-16T15:55:10.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.853+0000 7fc49a600640 1 -- 172.21.15.151:0/2479200905 shutdown_connections 2024-09-16T15:55:10.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.853+0000 7fc49a600640 1 --2- 172.21.15.151:0/2479200905 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc49406efa0 0x7fc494073440 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:10.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.853+0000 7fc49a600640 1 -- 172.21.15.151:0/2479200905 >> 172.21.15.151:0/2479200905 conn(0x7fc4940b1840 msgr2=0x7fc4940b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:55:10.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.854+0000 7fc49a600640 1 -- 172.21.15.151:0/2479200905 shutdown_connections 2024-09-16T15:55:10.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.854+0000 7fc49a600640 1 -- 172.21.15.151:0/2479200905 wait complete. 2024-09-16T15:55:10.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.855+0000 7fc49a600640 1 Processor -- start 2024-09-16T15:55:10.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.855+0000 7fc49a600640 1 -- start start 2024-09-16T15:55:10.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.856+0000 7fc49a600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc49406efa0 0x7fc494155190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:10.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.856+0000 7fc49a600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4940cccd0 0x7fc4940cc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:10.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.856+0000 7fc49a600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc494145750 0x7fc494140530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:10.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.856+0000 7fc49a600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc4940ce6a0 con 0x7fc49406efa0 2024-09-16T15:55:10.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.856+0000 7fc49a600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc4940ce520 con 0x7fc4940cccd0 2024-09-16T15:55:10.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.856+0000 7fc49a600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fc4940ce820 con 0x7fc494145750 2024-09-16T15:55:10.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.856+0000 7fc493400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc49406efa0 0x7fc494155190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:10.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.856+0000 7fc493400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc49406efa0 0x7fc494155190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44874/0 (socket says 172.21.15.151:44874) 2024-09-16T15:55:10.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.856+0000 7fc493400640 1 -- 172.21.15.151:0/1177461166 learned_addr learned my addr 172.21.15.151:0/1177461166 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:55:10.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.857+0000 7fc492a00640 1 --2- 172.21.15.151:0/1177461166 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4940cccd0 0x7fc4940cc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:10.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.857+0000 7fc493e00640 1 --2- 172.21.15.151:0/1177461166 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc494145750 0x7fc494140530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:10.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.857+0000 7fc493400640 1 -- 172.21.15.151:0/1177461166 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc494145750 msgr2=0x7fc494140530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:10.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.857+0000 7fc493400640 1 --2- 172.21.15.151:0/1177461166 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc494145750 0x7fc494140530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:10.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.857+0000 7fc493400640 1 -- 172.21.15.151:0/1177461166 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4940cccd0 msgr2=0x7fc4940cc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:10.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.857+0000 7fc493400640 1 --2- 172.21.15.151:0/1177461166 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4940cccd0 0x7fc4940cc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:10.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.857+0000 7fc493400640 1 -- 172.21.15.151:0/1177461166 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc494140e30 con 0x7fc49406efa0 2024-09-16T15:55:10.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.857+0000 7fc493400640 1 --2- 172.21.15.151:0/1177461166 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc49406efa0 0x7fc494155190 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fc48400cbb0 tx=0x7fc48400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:55:10.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.857+0000 7fc48b400640 1 -- 172.21.15.151:0/1177461166 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc484016020 con 0x7fc49406efa0 2024-09-16T15:55:10.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.857+0000 7fc48b400640 1 -- 172.21.15.151:0/1177461166 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc48400ad90 con 0x7fc49406efa0 2024-09-16T15:55:10.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.857+0000 7fc48b400640 1 -- 172.21.15.151:0/1177461166 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc484012680 con 0x7fc49406efa0 2024-09-16T15:55:10.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.857+0000 7fc49a600640 1 -- 172.21.15.151:0/1177461166 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc494141110 con 0x7fc49406efa0 2024-09-16T15:55:10.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.858+0000 7fc49a600640 1 -- 172.21.15.151:0/1177461166 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4941416d0 con 0x7fc49406efa0 2024-09-16T15:55:10.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.859+0000 7fc49a600640 1 -- 172.21.15.151:0/1177461166 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4940cecd0 con 0x7fc49406efa0 2024-09-16T15:55:10.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.860+0000 7fc48b400640 1 -- 172.21.15.151:0/1177461166 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 50) ==== 104398+0+0 (secure 0 0 0) 0x7fc484019020 con 0x7fc49406efa0 2024-09-16T15:55:10.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.861+0000 7fc48b400640 1 --2- 172.21.15.151:0/1177461166 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc47407aaa0 0x7fc47407cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:10.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.861+0000 7fc48b400640 1 -- 172.21.15.151:0/1177461166 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5907+0+0 (secure 0 0 0) 0x7fc484099da0 con 0x7fc49406efa0 2024-09-16T15:55:10.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.861+0000 7fc492a00640 1 --2- 172.21.15.151:0/1177461166 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc47407aaa0 0x7fc47407cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:10.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.862+0000 7fc492a00640 1 --2- 172.21.15.151:0/1177461166 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc47407aaa0 0x7fc47407cf60 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fc480000c00 tx=0x7fc480009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:55:10.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:10.865+0000 7fc48b400640 1 -- 172.21.15.151:0/1177461166 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc484020030 con 0x7fc49406efa0 2024-09-16T15:55:11.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.050+0000 7fc49a600640 1 -- 172.21.15.151:0/1177461166 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7fc4940cc830 con 0x7fc49406efa0 2024-09-16T15:55:11.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.054+0000 7fc48b400640 1 -- 172.21.15.151:0/1177461166 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v55)=0 v55) ==== 131+0+0 (secure 0 0 0) 0x7fc484063ef0 con 0x7fc49406efa0 2024-09-16T15:55:11.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.057+0000 7fc49a600640 1 -- 172.21.15.151:0/1177461166 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc47407aaa0 msgr2=0x7fc47407cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:11.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.057+0000 7fc49a600640 1 --2- 172.21.15.151:0/1177461166 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc47407aaa0 0x7fc47407cf60 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fc480000c00 tx=0x7fc480009040 comp rx=0 tx=0).stop 2024-09-16T15:55:11.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.057+0000 7fc49a600640 1 -- 172.21.15.151:0/1177461166 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc49406efa0 msgr2=0x7fc494155190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:11.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.057+0000 7fc49a600640 1 --2- 172.21.15.151:0/1177461166 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc49406efa0 0x7fc494155190 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fc48400cbb0 tx=0x7fc48400a060 comp rx=0 tx=0).stop 2024-09-16T15:55:11.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.057+0000 7fc49a600640 1 -- 172.21.15.151:0/1177461166 shutdown_connections 2024-09-16T15:55:11.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.057+0000 7fc49a600640 1 --2- 172.21.15.151:0/1177461166 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc47407aaa0 0x7fc47407cf60 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:11.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.057+0000 7fc49a600640 1 --2- 172.21.15.151:0/1177461166 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc4940cccd0 0x7fc4940cc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:11.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.057+0000 7fc49a600640 1 --2- 172.21.15.151:0/1177461166 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc494145750 0x7fc494140530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:11.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.057+0000 7fc49a600640 1 --2- 172.21.15.151:0/1177461166 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc49406efa0 0x7fc494155190 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:11.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.057+0000 7fc49a600640 1 -- 172.21.15.151:0/1177461166 >> 172.21.15.151:0/1177461166 conn(0x7fc4940b1840 msgr2=0x7fc4940b1060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:55:11.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.057+0000 7fc49a600640 1 -- 172.21.15.151:0/1177461166 shutdown_connections 2024-09-16T15:55:11.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.057+0000 7fc49a600640 1 -- 172.21.15.151:0/1177461166 wait complete. 2024-09-16T15:55:11.075 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/snapshot_clone_no_wait false 2024-09-16T15:55:11.080 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:55:11.079+0000 7f48f2600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:55:11.080 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:55:11.079+0000 7f48f2600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:55:11.080 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:55:11.079+0000 7f48f2600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:55:11.080 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:55:11.079+0000 7f48f2600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:55:11.080 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:55:11.079+0000 7f48f2600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:55:11.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.233+0000 7f7d37200640 1 -- 172.21.15.151:0/2624941196 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d300cccd0 msgr2=0x7f7d300cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:11.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.233+0000 7f7d37200640 1 --2- 172.21.15.151:0/2624941196 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d300cccd0 0x7f7d300cdbb0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f7d28005fd0 tx=0x7f7d2802a810 comp rx=0 tx=0).stop 2024-09-16T15:55:11.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.234+0000 7f7d37200640 1 -- 172.21.15.151:0/2624941196 shutdown_connections 2024-09-16T15:55:11.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.234+0000 7f7d37200640 1 --2- 172.21.15.151:0/2624941196 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d300cccd0 0x7f7d300cdbb0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:11.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.234+0000 7f7d37200640 1 --2- 172.21.15.151:0/2624941196 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7d30145750 0x7f7d30073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:11.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.234+0000 7f7d37200640 1 -- 172.21.15.151:0/2624941196 >> 172.21.15.151:0/2624941196 conn(0x7f7d300b1840 msgr2=0x7f7d300b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:55:11.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.234+0000 7f7d37200640 1 -- 172.21.15.151:0/2624941196 shutdown_connections 2024-09-16T15:55:11.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.234+0000 7f7d37200640 1 -- 172.21.15.151:0/2624941196 wait complete. 2024-09-16T15:55:11.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.235+0000 7f7d37200640 1 Processor -- start 2024-09-16T15:55:11.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.235+0000 7f7d37200640 1 -- start start 2024-09-16T15:55:11.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.236+0000 7f7d37200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d3006efa0 0x7f7d3013c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:11.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.236+0000 7f7d37200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7d300cccd0 0x7f7d3013d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:11.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.236+0000 7f7d37200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d30145750 0x7f7d30142350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:11.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.236+0000 7f7d37200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7d300ce480 con 0x7f7d3006efa0 2024-09-16T15:55:11.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.236+0000 7f7d37200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7d300ce300 con 0x7f7d30145750 2024-09-16T15:55:11.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.236+0000 7f7d37200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f7d300ce600 con 0x7f7d300cccd0 2024-09-16T15:55:11.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.237+0000 7f7d2fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d3006efa0 0x7f7d3013c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:11.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.237+0000 7f7d2fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d3006efa0 0x7f7d3013c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44884/0 (socket says 172.21.15.151:44884) 2024-09-16T15:55:11.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.237+0000 7f7d2fe00640 1 -- 172.21.15.151:0/122214819 learned_addr learned my addr 172.21.15.151:0/122214819 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:55:11.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.237+0000 7f7d27400640 1 --2- 172.21.15.151:0/122214819 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7d300cccd0 0x7f7d3013d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:11.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.237+0000 7f7d35200640 1 --2- 172.21.15.151:0/122214819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d30145750 0x7f7d30142350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:11.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.237+0000 7f7d2fe00640 1 -- 172.21.15.151:0/122214819 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7d300cccd0 msgr2=0x7f7d3013d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:11.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.237+0000 7f7d2fe00640 1 --2- 172.21.15.151:0/122214819 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7d300cccd0 0x7f7d3013d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:11.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.237+0000 7f7d2fe00640 1 -- 172.21.15.151:0/122214819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d30145750 msgr2=0x7f7d30142350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:11.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.237+0000 7f7d2fe00640 1 --2- 172.21.15.151:0/122214819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d30145750 0x7f7d30142350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:11.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.237+0000 7f7d2fe00640 1 -- 172.21.15.151:0/122214819 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d3013e700 con 0x7f7d3006efa0 2024-09-16T15:55:11.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.237+0000 7f7d2fe00640 1 --2- 172.21.15.151:0/122214819 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d3006efa0 0x7f7d3013c170 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f7d2000e7c0 tx=0x7f7d2000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:55:11.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.238+0000 7f7d2cc00640 1 -- 172.21.15.151:0/122214819 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2000ccc0 con 0x7f7d3006efa0 2024-09-16T15:55:11.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.238+0000 7f7d2cc00640 1 -- 172.21.15.151:0/122214819 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7d20010430 con 0x7f7d3006efa0 2024-09-16T15:55:11.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.238+0000 7f7d37200640 1 -- 172.21.15.151:0/122214819 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d3013de60 con 0x7f7d3006efa0 2024-09-16T15:55:11.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.238+0000 7f7d2cc00640 1 -- 172.21.15.151:0/122214819 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d20010860 con 0x7f7d3006efa0 2024-09-16T15:55:11.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.238+0000 7f7d37200640 1 -- 172.21.15.151:0/122214819 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d3013e420 con 0x7f7d3006efa0 2024-09-16T15:55:11.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.239+0000 7f7d37200640 1 -- 172.21.15.151:0/122214819 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d300cdbb0 con 0x7f7d3006efa0 2024-09-16T15:55:11.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.240+0000 7f7d2cc00640 1 -- 172.21.15.151:0/122214819 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 50) ==== 104398+0+0 (secure 0 0 0) 0x7f7d20010a00 con 0x7f7d3006efa0 2024-09-16T15:55:11.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.242+0000 7f7d2cc00640 1 --2- 172.21.15.151:0/122214819 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7d0c07aaa0 0x7f7d0c07cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:11.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.242+0000 7f7d2cc00640 1 -- 172.21.15.151:0/122214819 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5907+0+0 (secure 0 0 0) 0x7f7d2009a3f0 con 0x7f7d3006efa0 2024-09-16T15:55:11.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.242+0000 7f7d27400640 1 --2- 172.21.15.151:0/122214819 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7d0c07aaa0 0x7f7d0c07cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:11.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.243+0000 7f7d27400640 1 --2- 172.21.15.151:0/122214819 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7d0c07aaa0 0x7f7d0c07cf60 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f7d18007fd0 tx=0x7f7d18001410 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:55:11.248 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.246+0000 7f7d2cc00640 1 -- 172.21.15.151:0/122214819 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7d20063e70 con 0x7f7d3006efa0 2024-09-16T15:55:11.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.429+0000 7f7d37200640 1 -- 172.21.15.151:0/122214819 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/snapshot_clone_no_wait}] v 0) -- 0x7f7d3015ef70 con 0x7f7d3006efa0 2024-09-16T15:55:11.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.434+0000 7f7d2cc00640 1 -- 172.21.15.151:0/122214819 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/snapshot_clone_no_wait}]=0 v56)=0 v56) ==== 136+0+0 (secure 0 0 0) 0x7f7d200688a0 con 0x7f7d3006efa0 2024-09-16T15:55:11.439 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.436+0000 7f7d37200640 1 -- 172.21.15.151:0/122214819 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7d0c07aaa0 msgr2=0x7f7d0c07cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:11.439 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.436+0000 7f7d37200640 1 --2- 172.21.15.151:0/122214819 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7d0c07aaa0 0x7f7d0c07cf60 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f7d18007fd0 tx=0x7f7d18001410 comp rx=0 tx=0).stop 2024-09-16T15:55:11.439 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.436+0000 7f7d37200640 1 -- 172.21.15.151:0/122214819 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d3006efa0 msgr2=0x7f7d3013c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:11.439 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.436+0000 7f7d37200640 1 --2- 172.21.15.151:0/122214819 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d3006efa0 0x7f7d3013c170 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f7d2000e7c0 tx=0x7f7d2000ec80 comp rx=0 tx=0).stop 2024-09-16T15:55:11.439 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.436+0000 7f7d37200640 1 -- 172.21.15.151:0/122214819 shutdown_connections 2024-09-16T15:55:11.439 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.436+0000 7f7d37200640 1 --2- 172.21.15.151:0/122214819 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7d0c07aaa0 0x7f7d0c07cf60 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:11.439 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.436+0000 7f7d37200640 1 --2- 172.21.15.151:0/122214819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d30145750 0x7f7d30142350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:11.439 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.436+0000 7f7d37200640 1 --2- 172.21.15.151:0/122214819 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7d300cccd0 0x7f7d3013d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:11.439 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.436+0000 7f7d37200640 1 --2- 172.21.15.151:0/122214819 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d3006efa0 0x7f7d3013c170 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:11.439 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.436+0000 7f7d37200640 1 -- 172.21.15.151:0/122214819 >> 172.21.15.151:0/122214819 conn(0x7f7d300b1840 msgr2=0x7f7d300ae360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:55:11.439 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.436+0000 7f7d37200640 1 -- 172.21.15.151:0/122214819 shutdown_connections 2024-09-16T15:55:11.439 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.436+0000 7f7d37200640 1 -- 172.21.15.151:0/122214819 wait complete. 2024-09-16T15:55:11.454 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-09-16T15:55:11.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.615+0000 7f099be00640 1 -- 172.21.15.151:0/3689266940 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f099c070380 msgr2=0x7f099c0cb1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:11.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.615+0000 7f099be00640 1 --2- 172.21.15.151:0/3689266940 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f099c070380 0x7f099c0cb1c0 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f0994005fd0 tx=0x7f099402aa00 comp rx=0 tx=0).stop 2024-09-16T15:55:11.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.615+0000 7f099be00640 1 -- 172.21.15.151:0/3689266940 shutdown_connections 2024-09-16T15:55:11.617 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.615+0000 7f099be00640 1 --2- 172.21.15.151:0/3689266940 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f099c070380 0x7f099c0cb1c0 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:11.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.615+0000 7f099be00640 1 --2- 172.21.15.151:0/3689266940 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f099c06e590 0x7f099c06fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:11.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.615+0000 7f099be00640 1 -- 172.21.15.151:0/3689266940 >> 172.21.15.151:0/3689266940 conn(0x7f099c0cfa70 msgr2=0x7f099c0cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:55:11.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.615+0000 7f099be00640 1 -- 172.21.15.151:0/3689266940 shutdown_connections 2024-09-16T15:55:11.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.615+0000 7f099be00640 1 -- 172.21.15.151:0/3689266940 wait complete. 2024-09-16T15:55:11.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.616+0000 7f099be00640 1 Processor -- start 2024-09-16T15:55:11.618 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.617+0000 7f099be00640 1 -- start start 2024-09-16T15:55:11.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.617+0000 7f099be00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f099c06e590 0x7f099c1f3040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:11.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.617+0000 7f099be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f099c070380 0x7f099c1556a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:11.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.617+0000 7f099be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f099c15bda0 0x7f099c151f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:11.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.617+0000 7f099be00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f099c0cb910 con 0x7f099c15bda0 2024-09-16T15:55:11.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.617+0000 7f099be00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f099c0708f0 con 0x7f099c070380 2024-09-16T15:55:11.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.617+0000 7f099be00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f099c0cba90 con 0x7f099c06e590 2024-09-16T15:55:11.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.618+0000 7f099b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f099c15bda0 0x7f099c151f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:11.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.618+0000 7f099b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f099c15bda0 0x7f099c151f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44898/0 (socket says 172.21.15.151:44898) 2024-09-16T15:55:11.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.618+0000 7f099b400640 1 -- 172.21.15.151:0/3902711679 learned_addr learned my addr 172.21.15.151:0/3902711679 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:55:11.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.618+0000 7f099aa00640 1 --2- 172.21.15.151:0/3902711679 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f099c06e590 0x7f099c1f3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:11.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.618+0000 7f099a000640 1 --2- 172.21.15.151:0/3902711679 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f099c070380 0x7f099c1556a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:11.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.618+0000 7f099b400640 1 -- 172.21.15.151:0/3902711679 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f099c06e590 msgr2=0x7f099c1f3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:11.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.618+0000 7f099b400640 1 --2- 172.21.15.151:0/3902711679 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f099c06e590 0x7f099c1f3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:11.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.618+0000 7f099b400640 1 -- 172.21.15.151:0/3902711679 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f099c070380 msgr2=0x7f099c1556a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:11.621 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.618+0000 7f099b400640 1 --2- 172.21.15.151:0/3902711679 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f099c070380 0x7f099c1556a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:11.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.618+0000 7f099b400640 1 -- 172.21.15.151:0/3902711679 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f099c155ed0 con 0x7f099c15bda0 2024-09-16T15:55:11.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.619+0000 7f099b400640 1 --2- 172.21.15.151:0/3902711679 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f099c15bda0 0x7f099c151f30 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f0994006010 tx=0x7f0994003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:55:11.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.619+0000 7f0992a00640 1 -- 172.21.15.151:0/3902711679 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0994047070 con 0x7f099c15bda0 2024-09-16T15:55:11.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.619+0000 7f0992a00640 1 -- 172.21.15.151:0/3902711679 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0994004250 con 0x7f099c15bda0 2024-09-16T15:55:11.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.619+0000 7f099be00640 1 -- 172.21.15.151:0/3902711679 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f099c152f10 con 0x7f099c15bda0 2024-09-16T15:55:11.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.619+0000 7f0992a00640 1 -- 172.21.15.151:0/3902711679 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0994033050 con 0x7f099c15bda0 2024-09-16T15:55:11.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.619+0000 7f099be00640 1 -- 172.21.15.151:0/3902711679 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f099c155be0 con 0x7f099c15bda0 2024-09-16T15:55:11.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.620+0000 7f099be00640 1 -- 172.21.15.151:0/3902711679 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f099c0cc020 con 0x7f099c15bda0 2024-09-16T15:55:11.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.621+0000 7f0992a00640 1 -- 172.21.15.151:0/3902711679 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 50) ==== 104398+0+0 (secure 0 0 0) 0x7f099404a070 con 0x7f099c15bda0 2024-09-16T15:55:11.624 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.622+0000 7f0992a00640 1 --2- 172.21.15.151:0/3902711679 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f097c07aaa0 0x7f097c07cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:11.624 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.622+0000 7f0992a00640 1 -- 172.21.15.151:0/3902711679 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5907+0+0 (secure 0 0 0) 0x7f09940bac00 con 0x7f099c15bda0 2024-09-16T15:55:11.624 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.623+0000 7f099aa00640 1 --2- 172.21.15.151:0/3902711679 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f097c07aaa0 0x7f097c07cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:11.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.623+0000 7f099aa00640 1 --2- 172.21.15.151:0/3902711679 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f097c07aaa0 0x7f097c07cf60 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f099c159420 tx=0x7f098c009060 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:55:11.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.627+0000 7f0992a00640 1 -- 172.21.15.151:0/3902711679 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0994084680 con 0x7f099c15bda0 2024-09-16T15:55:11.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.836+0000 7f099be00640 1 -- 172.21.15.151:0/3902711679 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) -- 0x7f099c074670 con 0x7f097c07aaa0 2024-09-16T15:55:11.849 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:55:11.848+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:55:11.849 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:55:11.848+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:55:11.849 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:55:11.848+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:55:11.849 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:55:11.848+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:55:11.849 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:55:11.848+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:55:11.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.888+0000 7f0992a00640 1 -- 172.21.15.151:0/3902711679 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f099c074670 con 0x7f097c07aaa0 2024-09-16T15:55:11.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.891+0000 7f099be00640 1 -- 172.21.15.151:0/3902711679 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f097c07aaa0 msgr2=0x7f097c07cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:11.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.891+0000 7f099be00640 1 --2- 172.21.15.151:0/3902711679 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f097c07aaa0 0x7f097c07cf60 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f099c159420 tx=0x7f098c009060 comp rx=0 tx=0).stop 2024-09-16T15:55:11.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.891+0000 7f099be00640 1 -- 172.21.15.151:0/3902711679 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f099c15bda0 msgr2=0x7f099c151f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:11.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.891+0000 7f099be00640 1 --2- 172.21.15.151:0/3902711679 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f099c15bda0 0x7f099c151f30 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f0994006010 tx=0x7f0994003f10 comp rx=0 tx=0).stop 2024-09-16T15:55:11.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.891+0000 7f099be00640 1 -- 172.21.15.151:0/3902711679 shutdown_connections 2024-09-16T15:55:11.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.891+0000 7f099be00640 1 --2- 172.21.15.151:0/3902711679 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f097c07aaa0 0x7f097c07cf60 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:11.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.891+0000 7f099be00640 1 --2- 172.21.15.151:0/3902711679 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f099c070380 0x7f099c1556a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:11.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.891+0000 7f099be00640 1 --2- 172.21.15.151:0/3902711679 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f099c15bda0 0x7f099c151f30 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:11.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.891+0000 7f099be00640 1 --2- 172.21.15.151:0/3902711679 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f099c06e590 0x7f099c1f3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:11.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.891+0000 7f099be00640 1 -- 172.21.15.151:0/3902711679 >> 172.21.15.151:0/3902711679 conn(0x7f099c0cfa70 msgr2=0x7f099c0737c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:55:11.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.891+0000 7f099be00640 1 -- 172.21.15.151:0/3902711679 shutdown_connections 2024-09-16T15:55:11.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:11.891+0000 7f099be00640 1 -- 172.21.15.151:0/3902711679 wait complete. 2024-09-16T15:55:11.909 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-16T15:55:12.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.068+0000 7fc89e400640 1 -- 172.21.15.151:0/769252039 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8980ad880 msgr2=0x7fc8980adcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:12.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.068+0000 7fc89e400640 1 --2- 172.21.15.151:0/769252039 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8980ad880 0x7fc8980adcf0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7fc890005fd0 tx=0x7fc89002a970 comp rx=0 tx=0).stop 2024-09-16T15:55:12.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.068+0000 7fc89e400640 1 -- 172.21.15.151:0/769252039 shutdown_connections 2024-09-16T15:55:12.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.068+0000 7fc89e400640 1 --2- 172.21.15.151:0/769252039 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8980ad880 0x7fc8980adcf0 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:12.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.068+0000 7fc89e400640 1 --2- 172.21.15.151:0/769252039 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc89806e580 0x7fc89806e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:12.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.068+0000 7fc89e400640 1 -- 172.21.15.151:0/769252039 >> 172.21.15.151:0/769252039 conn(0x7fc8980b1510 msgr2=0x7fc8980b18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:55:12.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.068+0000 7fc89e400640 1 -- 172.21.15.151:0/769252039 shutdown_connections 2024-09-16T15:55:12.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.068+0000 7fc89e400640 1 -- 172.21.15.151:0/769252039 wait complete. 2024-09-16T15:55:12.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.070+0000 7fc89e400640 1 Processor -- start 2024-09-16T15:55:12.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.070+0000 7fc89e400640 1 -- start start 2024-09-16T15:55:12.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.071+0000 7fc89e400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc89806e580 0x7fc8980c6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:12.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.071+0000 7fc89e400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8980ad880 0x7fc8981f8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:12.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.071+0000 7fc89e400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc898154c90 0x7fc8981f4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:12.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.071+0000 7fc89e400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc8980ca340 con 0x7fc8980ad880 2024-09-16T15:55:12.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.071+0000 7fc89e400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc8980ca1c0 con 0x7fc898154c90 2024-09-16T15:55:12.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.071+0000 7fc89e400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fc8980ca4c0 con 0x7fc89806e580 2024-09-16T15:55:12.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.071+0000 7fc897e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8980ad880 0x7fc8981f8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:12.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.071+0000 7fc897e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8980ad880 0x7fc8981f8690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44910/0 (socket says 172.21.15.151:44910) 2024-09-16T15:55:12.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.071+0000 7fc897e00640 1 -- 172.21.15.151:0/524306705 learned_addr learned my addr 172.21.15.151:0/524306705 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:55:12.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.072+0000 7fc89d000640 1 --2- 172.21.15.151:0/524306705 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc89806e580 0x7fc8980c6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:12.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.072+0000 7fc89da00640 1 --2- 172.21.15.151:0/524306705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc898154c90 0x7fc8981f4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:12.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.072+0000 7fc897e00640 1 -- 172.21.15.151:0/524306705 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc89806e580 msgr2=0x7fc8980c6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:12.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.072+0000 7fc897e00640 1 --2- 172.21.15.151:0/524306705 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc89806e580 0x7fc8980c6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:12.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.072+0000 7fc897e00640 1 -- 172.21.15.151:0/524306705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc898154c90 msgr2=0x7fc8981f4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:12.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.072+0000 7fc897e00640 1 --2- 172.21.15.151:0/524306705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc898154c90 0x7fc8981f4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:12.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.072+0000 7fc897e00640 1 -- 172.21.15.151:0/524306705 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8981f5100 con 0x7fc8980ad880 2024-09-16T15:55:12.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.072+0000 7fc89d000640 1 --2- 172.21.15.151:0/524306705 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc89806e580 0x7fc8980c6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:55:12.075 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.072+0000 7fc897e00640 1 --2- 172.21.15.151:0/524306705 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8980ad880 0x7fc8981f8690 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7fc88400c8c0 tx=0x7fc88400cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:55:12.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.072+0000 7fc894c00640 1 -- 172.21.15.151:0/524306705 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc884004220 con 0x7fc8980ad880 2024-09-16T15:55:12.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.072+0000 7fc894c00640 1 -- 172.21.15.151:0/524306705 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc8840079b0 con 0x7fc8980ad880 2024-09-16T15:55:12.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.072+0000 7fc894c00640 1 -- 172.21.15.151:0/524306705 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc884007de0 con 0x7fc8980ad880 2024-09-16T15:55:12.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.072+0000 7fc89e400640 1 -- 172.21.15.151:0/524306705 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8981f53e0 con 0x7fc8980ad880 2024-09-16T15:55:12.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.073+0000 7fc89e400640 1 -- 172.21.15.151:0/524306705 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8980737a0 con 0x7fc8980ad880 2024-09-16T15:55:12.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.074+0000 7fc89e400640 1 -- 172.21.15.151:0/524306705 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8980ca9e0 con 0x7fc8980ad880 2024-09-16T15:55:12.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.075+0000 7fc894c00640 1 -- 172.21.15.151:0/524306705 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 50) ==== 104398+0+0 (secure 0 0 0) 0x7fc884005020 con 0x7fc8980ad880 2024-09-16T15:55:12.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.076+0000 7fc894c00640 1 --2- 172.21.15.151:0/524306705 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc87807aaa0 0x7fc87807cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:12.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.076+0000 7fc894c00640 1 -- 172.21.15.151:0/524306705 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5907+0+0 (secure 0 0 0) 0x7fc884099f20 con 0x7fc8980ad880 2024-09-16T15:55:12.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.076+0000 7fc89d000640 1 --2- 172.21.15.151:0/524306705 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc87807aaa0 0x7fc87807cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:12.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.077+0000 7fc89d000640 1 --2- 172.21.15.151:0/524306705 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc87807aaa0 0x7fc87807cf60 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fc888006fd0 tx=0x7fc888008040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:55:12.082 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.080+0000 7fc894c00640 1 -- 172.21.15.151:0/524306705 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc884014030 con 0x7fc8980ad880 2024-09-16T15:55:12.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.293+0000 7fc89e400640 1 -- 172.21.15.151:0/524306705 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fc898069d20 con 0x7fc87807aaa0 2024-09-16T15:55:12.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.300+0000 7fc894c00640 1 -- 172.21.15.151:0/524306705 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7fc898069d20 con 0x7fc87807aaa0 2024-09-16T15:55:12.302 INFO:teuthology.orchestra.run.smithi151.stdout:/volumes/_nogroup/sv1/61d89b29-c3ef-41d1-a443-b6dc24b686d3 2024-09-16T15:55:12.304 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.302+0000 7fc89e400640 1 -- 172.21.15.151:0/524306705 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc87807aaa0 msgr2=0x7fc87807cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:12.304 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.302+0000 7fc89e400640 1 --2- 172.21.15.151:0/524306705 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc87807aaa0 0x7fc87807cf60 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fc888006fd0 tx=0x7fc888008040 comp rx=0 tx=0).stop 2024-09-16T15:55:12.304 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.302+0000 7fc89e400640 1 -- 172.21.15.151:0/524306705 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8980ad880 msgr2=0x7fc8981f8690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:12.304 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.302+0000 7fc89e400640 1 --2- 172.21.15.151:0/524306705 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8980ad880 0x7fc8981f8690 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7fc88400c8c0 tx=0x7fc88400cd80 comp rx=0 tx=0).stop 2024-09-16T15:55:12.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.302+0000 7fc89e400640 1 -- 172.21.15.151:0/524306705 shutdown_connections 2024-09-16T15:55:12.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.302+0000 7fc89e400640 1 --2- 172.21.15.151:0/524306705 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc87807aaa0 0x7fc87807cf60 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:12.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.302+0000 7fc89e400640 1 --2- 172.21.15.151:0/524306705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc898154c90 0x7fc8981f4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:12.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.302+0000 7fc89e400640 1 --2- 172.21.15.151:0/524306705 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8980ad880 0x7fc8981f8690 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:12.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.302+0000 7fc89e400640 1 --2- 172.21.15.151:0/524306705 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc89806e580 0x7fc8980c6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:12.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.302+0000 7fc89e400640 1 -- 172.21.15.151:0/524306705 >> 172.21.15.151:0/524306705 conn(0x7fc8980b1510 msgr2=0x7fc8980cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:55:12.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.302+0000 7fc89e400640 1 -- 172.21.15.151:0/524306705 shutdown_connections 2024-09-16T15:55:12.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.302+0000 7fc89e400640 1 -- 172.21.15.151:0/524306705 wait complete. 2024-09-16T15:55:12.321 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-16T15:55:12.487 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.484+0000 7f5747200640 1 -- 172.21.15.151:0/896897532 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57480d2000 msgr2=0x7f57480d23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:12.487 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.484+0000 7f5747200640 1 --2- 172.21.15.151:0/896897532 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57480d2000 0x7f57480d23e0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f5738009860 tx=0x7f573802a610 comp rx=0 tx=0).stop 2024-09-16T15:55:12.487 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.484+0000 7f5747200640 1 -- 172.21.15.151:0/896897532 shutdown_connections 2024-09-16T15:55:12.487 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.484+0000 7f5747200640 1 --2- 172.21.15.151:0/896897532 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57480d2000 0x7f57480d23e0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:12.487 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.484+0000 7f5747200640 1 -- 172.21.15.151:0/896897532 >> 172.21.15.151:0/896897532 conn(0x7f57480d3e20 msgr2=0x7f57480d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:55:12.487 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.485+0000 7f5747200640 1 -- 172.21.15.151:0/896897532 shutdown_connections 2024-09-16T15:55:12.487 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.485+0000 7f5747200640 1 -- 172.21.15.151:0/896897532 wait complete. 2024-09-16T15:55:12.488 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.486+0000 7f5747200640 1 Processor -- start 2024-09-16T15:55:12.488 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.486+0000 7f5747200640 1 -- start start 2024-09-16T15:55:12.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.487+0000 7f5747200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f57480b4e70 0x7f574814e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:12.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.487+0000 7f5747200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f57480d2000 0x7f5748159730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:12.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.487+0000 7f5747200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57480d2920 0x7f5748159c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:12.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.487+0000 7f5747200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f57481692e0 con 0x7f57480d2000 2024-09-16T15:55:12.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.487+0000 7f5747200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5748169160 con 0x7f57480d2920 2024-09-16T15:55:12.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.487+0000 7f5747200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f5748169460 con 0x7f57480b4e70 2024-09-16T15:55:12.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.487+0000 7f5745400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f57480d2000 0x7f5748159730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:12.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.487+0000 7f5745400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f57480d2000 0x7f5748159730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44920/0 (socket says 172.21.15.151:44920) 2024-09-16T15:55:12.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.487+0000 7f5745400640 1 -- 172.21.15.151:0/1808916158 learned_addr learned my addr 172.21.15.151:0/1808916158 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:55:12.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.488+0000 7f5745e00640 1 --2- 172.21.15.151:0/1808916158 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f57480b4e70 0x7f574814e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:12.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.488+0000 7f5746800640 1 --2- 172.21.15.151:0/1808916158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57480d2920 0x7f5748159c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:12.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.488+0000 7f5745400640 1 -- 172.21.15.151:0/1808916158 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f57480b4e70 msgr2=0x7f574814e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:12.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.488+0000 7f5745400640 1 --2- 172.21.15.151:0/1808916158 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f57480b4e70 0x7f574814e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:12.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.488+0000 7f5745400640 1 -- 172.21.15.151:0/1808916158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57480d2920 msgr2=0x7f5748159c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:12.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.488+0000 7f5745400640 1 --2- 172.21.15.151:0/1808916158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57480d2920 0x7f5748159c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:12.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.488+0000 7f5745400640 1 -- 172.21.15.151:0/1808916158 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57480b9990 con 0x7f57480d2000 2024-09-16T15:55:12.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.488+0000 7f5745400640 1 --2- 172.21.15.151:0/1808916158 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f57480d2000 0x7f5748159730 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f5734007fd0 tx=0x7f573400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:55:12.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.488+0000 7f573e000640 1 -- 172.21.15.151:0/1808916158 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5734011790 con 0x7f57480d2000 2024-09-16T15:55:12.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.488+0000 7f573e000640 1 -- 172.21.15.151:0/1808916158 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5734017070 con 0x7f57480d2000 2024-09-16T15:55:12.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.489+0000 7f5747200640 1 -- 172.21.15.151:0/1808916158 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5748156d40 con 0x7f57480d2000 2024-09-16T15:55:12.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.489+0000 7f573e000640 1 -- 172.21.15.151:0/1808916158 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5734011d20 con 0x7f57480d2000 2024-09-16T15:55:12.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.489+0000 7f5747200640 1 -- 172.21.15.151:0/1808916158 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5748157300 con 0x7f57480d2000 2024-09-16T15:55:12.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.490+0000 7f5747200640 1 -- 172.21.15.151:0/1808916158 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5748151580 con 0x7f57480d2000 2024-09-16T15:55:12.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.491+0000 7f573e000640 1 -- 172.21.15.151:0/1808916158 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 50) ==== 104398+0+0 (secure 0 0 0) 0x7f573401c460 con 0x7f57480d2000 2024-09-16T15:55:12.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.493+0000 7f573e000640 1 --2- 172.21.15.151:0/1808916158 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f572807aaa0 0x7f572807cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:12.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.493+0000 7f573e000640 1 -- 172.21.15.151:0/1808916158 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5907+0+0 (secure 0 0 0) 0x7f57340998f0 con 0x7f57480d2000 2024-09-16T15:55:12.495 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.494+0000 7f5745e00640 1 --2- 172.21.15.151:0/1808916158 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f572807aaa0 0x7f572807cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:12.496 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.494+0000 7f5745e00640 1 --2- 172.21.15.151:0/1808916158 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f572807aaa0 0x7f572807cf60 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f5738034000 tx=0x7f5738004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:55:12.499 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.497+0000 7f573e000640 1 -- 172.21.15.151:0/1808916158 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f57340632f0 con 0x7f57480d2000 2024-09-16T15:55:12.556 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.554+0000 7f573e000640 1 -- 172.21.15.151:0/1808916158 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mgrmap(e 51) ==== 104500+0+0 (secure 0 0 0) 0x7f573405f560 con 0x7f57480d2000 2024-09-16T15:55:12.717 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.715+0000 7f5747200640 1 -- 172.21.15.151:0/1808916158 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f57480d23e0 con 0x7f572807aaa0 2024-09-16T15:55:12.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.720+0000 7f573e000640 1 -- 172.21.15.151:0/1808916158 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f57480d23e0 con 0x7f572807aaa0 2024-09-16T15:55:12.723 INFO:teuthology.orchestra.run.smithi151.stdout:/volumes/_nogroup/sv1/61d89b29-c3ef-41d1-a443-b6dc24b686d3 2024-09-16T15:55:12.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.723+0000 7f5747200640 1 -- 172.21.15.151:0/1808916158 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f572807aaa0 msgr2=0x7f572807cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:12.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.723+0000 7f5747200640 1 --2- 172.21.15.151:0/1808916158 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f572807aaa0 0x7f572807cf60 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f5738034000 tx=0x7f5738004730 comp rx=0 tx=0).stop 2024-09-16T15:55:12.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.723+0000 7f5747200640 1 -- 172.21.15.151:0/1808916158 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f57480d2000 msgr2=0x7f5748159730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:12.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.723+0000 7f5747200640 1 --2- 172.21.15.151:0/1808916158 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f57480d2000 0x7f5748159730 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f5734007fd0 tx=0x7f573400c8e0 comp rx=0 tx=0).stop 2024-09-16T15:55:12.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.723+0000 7f5747200640 1 -- 172.21.15.151:0/1808916158 shutdown_connections 2024-09-16T15:55:12.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.723+0000 7f5747200640 1 --2- 172.21.15.151:0/1808916158 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f572807aaa0 0x7f572807cf60 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:12.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.723+0000 7f5747200640 1 --2- 172.21.15.151:0/1808916158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f57480d2920 0x7f5748159c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:12.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.723+0000 7f5747200640 1 --2- 172.21.15.151:0/1808916158 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f57480d2000 0x7f5748159730 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:12.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.723+0000 7f5747200640 1 --2- 172.21.15.151:0/1808916158 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f57480b4e70 0x7f574814e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:12.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.723+0000 7f5747200640 1 -- 172.21.15.151:0/1808916158 >> 172.21.15.151:0/1808916158 conn(0x7f57480d3e20 msgr2=0x7f574806e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:55:12.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.723+0000 7f5747200640 1 -- 172.21.15.151:0/1808916158 shutdown_connections 2024-09-16T15:55:12.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:12.723+0000 7f5747200640 1 -- 172.21.15.151:0/1808916158 wait complete. 2024-09-16T15:55:12.741 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/61d89b29-c3ef-41d1-a443-b6dc24b686d3 2024-09-16T15:55:12.741 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:55:12.742 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:55:12.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:55:12.778 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/61d89b29-c3ef-41d1-a443-b6dc24b686d3/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T15:55:20.931 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records in 2024-09-16T15:55:20.931 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records out 2024-09-16T15:55:20.931 INFO:teuthology.orchestra.run.smithi151.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.06922 s, 133 MB/s 2024-09-16T15:55:20.932 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:55:20.933 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:55:20.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:55:20.968 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/61d89b29-c3ef-41d1-a443-b6dc24b686d3/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T15:55:21.791 DEBUG:teuthology.orchestra.run.smithi151:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:55:21.797 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:55:29.030 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records in 2024-09-16T15:55:29.031 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records out 2024-09-16T15:55:29.031 INFO:teuthology.orchestra.run.smithi151.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.98633 s, 134 MB/s 2024-09-16T15:55:29.032 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:55:29.032 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:55:29.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:55:29.067 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/61d89b29-c3ef-41d1-a443-b6dc24b686d3/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-09-16T15:55:37.086 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records in 2024-09-16T15:55:37.086 INFO:teuthology.orchestra.run.smithi151.stderr:1024+0 records out 2024-09-16T15:55:37.086 INFO:teuthology.orchestra.run.smithi151.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.94221 s, 135 MB/s 2024-09-16T15:55:37.087 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-09-16T15:55:37.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.250+0000 7f4594800640 1 -- 172.21.15.151:0/4283644178 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f458c0cd110 msgr2=0x7f458c0cebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:37.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.250+0000 7f4594800640 1 --2- 172.21.15.151:0/4283644178 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f458c0cd110 0x7f458c0cebe0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f4588005fd0 tx=0x7f458802aa90 comp rx=0 tx=0).stop 2024-09-16T15:55:37.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.250+0000 7f4594800640 1 -- 172.21.15.151:0/4283644178 shutdown_connections 2024-09-16T15:55:37.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.250+0000 7f4594800640 1 --2- 172.21.15.151:0/4283644178 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f458c0cd110 0x7f458c0cebe0 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:37.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.250+0000 7f4594800640 1 -- 172.21.15.151:0/4283644178 >> 172.21.15.151:0/4283644178 conn(0x7f458c0b18e0 msgr2=0x7f458c0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:55:37.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.250+0000 7f4594800640 1 -- 172.21.15.151:0/4283644178 shutdown_connections 2024-09-16T15:55:37.251 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.250+0000 7f4594800640 1 -- 172.21.15.151:0/4283644178 wait complete. 2024-09-16T15:55:37.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.251+0000 7f4594800640 1 Processor -- start 2024-09-16T15:55:37.252 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.252+0000 7f4594800640 1 -- start start 2024-09-16T15:55:37.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.252+0000 7f4594800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f458c0ccca0 0x7f458c14b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:37.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.252+0000 7f4594800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f458c0cd110 0x7f458c06f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:37.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.252+0000 7f4594800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f458c158900 0x7f458c1f65d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:37.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.252+0000 7f4594800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f458c06ef80 con 0x7f458c158900 2024-09-16T15:55:37.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.253+0000 7f4594800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f458c06ee00 con 0x7f458c0ccca0 2024-09-16T15:55:37.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.253+0000 7f4594800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f458c0cf120 con 0x7f458c0cd110 2024-09-16T15:55:37.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.253+0000 7f4592800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f458c158900 0x7f458c1f65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:37.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.253+0000 7f4592800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f458c158900 0x7f458c1f65d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47746/0 (socket says 172.21.15.151:47746) 2024-09-16T15:55:37.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.253+0000 7f4592800640 1 -- 172.21.15.151:0/2570286919 learned_addr learned my addr 172.21.15.151:0/2570286919 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:55:37.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.253+0000 7f4591e00640 1 --2- 172.21.15.151:0/2570286919 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f458c0ccca0 0x7f458c14b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:37.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.253+0000 7f4591400640 1 --2- 172.21.15.151:0/2570286919 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f458c0cd110 0x7f458c06f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:37.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.253+0000 7f4592800640 1 -- 172.21.15.151:0/2570286919 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f458c0cd110 msgr2=0x7f458c06f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:37.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.253+0000 7f4592800640 1 --2- 172.21.15.151:0/2570286919 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f458c0cd110 0x7f458c06f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:37.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.253+0000 7f4592800640 1 -- 172.21.15.151:0/2570286919 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f458c0ccca0 msgr2=0x7f458c14b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:37.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.253+0000 7f4592800640 1 --2- 172.21.15.151:0/2570286919 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f458c0ccca0 0x7f458c14b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:37.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.253+0000 7f4592800640 1 -- 172.21.15.151:0/2570286919 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f458c06f640 con 0x7f458c158900 2024-09-16T15:55:37.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.254+0000 7f4592800640 1 --2- 172.21.15.151:0/2570286919 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f458c158900 0x7f458c1f65d0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f4588006010 tx=0x7f4588003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:55:37.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.254+0000 7f4586000640 1 -- 172.21.15.151:0/2570286919 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4588046070 con 0x7f458c158900 2024-09-16T15:55:37.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.254+0000 7f4586000640 1 -- 172.21.15.151:0/2570286919 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4588004250 con 0x7f458c158900 2024-09-16T15:55:37.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.254+0000 7f4586000640 1 -- 172.21.15.151:0/2570286919 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4588004680 con 0x7f458c158900 2024-09-16T15:55:37.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.254+0000 7f4594800640 1 -- 172.21.15.151:0/2570286919 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f458c06f910 con 0x7f458c158900 2024-09-16T15:55:37.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.254+0000 7f4594800640 1 -- 172.21.15.151:0/2570286919 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f458c1f5dd0 con 0x7f458c158900 2024-09-16T15:55:37.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.255+0000 7f4594800640 1 -- 172.21.15.151:0/2570286919 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f458c069360 con 0x7f458c158900 2024-09-16T15:55:37.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.256+0000 7f4586000640 1 -- 172.21.15.151:0/2570286919 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 51) ==== 104500+0+0 (secure 0 0 0) 0x7f4588049070 con 0x7f458c158900 2024-09-16T15:55:37.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.257+0000 7f4586000640 1 --2- 172.21.15.151:0/2570286919 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f457007abf0 0x7f457007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:37.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.257+0000 7f4586000640 1 -- 172.21.15.151:0/2570286919 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5907+0+0 (secure 0 0 0) 0x7f45880baa60 con 0x7f458c158900 2024-09-16T15:55:37.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.258+0000 7f4591e00640 1 --2- 172.21.15.151:0/2570286919 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f457007abf0 0x7f457007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:37.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.258+0000 7f4591e00640 1 --2- 172.21.15.151:0/2570286919 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f457007abf0 0x7f457007d0b0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f4580001720 tx=0x7f4580009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:55:37.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.262+0000 7f4586000640 1 -- 172.21.15.151:0/2570286919 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f4588032030 con 0x7f458c158900 2024-09-16T15:55:37.476 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.475+0000 7f4594800640 1 -- 172.21.15.151:0/2570286919 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) -- 0x7f458c0cebe0 con 0x7f457007abf0 2024-09-16T15:55:37.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.488+0000 7f4586000640 1 -- 172.21.15.151:0/2570286919 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f458c0cebe0 con 0x7f457007abf0 2024-09-16T15:55:37.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.491+0000 7f4594800640 1 -- 172.21.15.151:0/2570286919 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f457007abf0 msgr2=0x7f457007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:37.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.491+0000 7f4594800640 1 --2- 172.21.15.151:0/2570286919 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f457007abf0 0x7f457007d0b0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f4580001720 tx=0x7f4580009040 comp rx=0 tx=0).stop 2024-09-16T15:55:37.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.491+0000 7f4594800640 1 -- 172.21.15.151:0/2570286919 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f458c158900 msgr2=0x7f458c1f65d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:37.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.491+0000 7f4594800640 1 --2- 172.21.15.151:0/2570286919 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f458c158900 0x7f458c1f65d0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f4588006010 tx=0x7f4588003f10 comp rx=0 tx=0).stop 2024-09-16T15:55:37.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.491+0000 7f4594800640 1 -- 172.21.15.151:0/2570286919 shutdown_connections 2024-09-16T15:55:37.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.491+0000 7f4594800640 1 --2- 172.21.15.151:0/2570286919 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f457007abf0 0x7f457007d0b0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:37.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.491+0000 7f4594800640 1 --2- 172.21.15.151:0/2570286919 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f458c158900 0x7f458c1f65d0 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:37.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.491+0000 7f4594800640 1 --2- 172.21.15.151:0/2570286919 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f458c0cd110 0x7f458c06f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:37.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.491+0000 7f4594800640 1 --2- 172.21.15.151:0/2570286919 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f458c0ccca0 0x7f458c14b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:37.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.491+0000 7f4594800640 1 -- 172.21.15.151:0/2570286919 >> 172.21.15.151:0/2570286919 conn(0x7f458c0b18e0 msgr2=0x7f458c15b170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:55:37.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.491+0000 7f4594800640 1 -- 172.21.15.151:0/2570286919 shutdown_connections 2024-09-16T15:55:37.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.491+0000 7f4594800640 1 -- 172.21.15.151:0/2570286919 wait complete. 2024-09-16T15:55:37.507 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-09-16T15:55:37.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.669+0000 7f357f000640 1 -- 172.21.15.151:0/3772521032 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f35780cccd0 msgr2=0x7f35780cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:37.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.669+0000 7f357f000640 1 --2- 172.21.15.151:0/3772521032 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f35780cccd0 0x7f35780cdbb0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f3570005fd0 tx=0x7f357002aa10 comp rx=0 tx=0).stop 2024-09-16T15:55:37.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.669+0000 7f357f000640 1 -- 172.21.15.151:0/3772521032 shutdown_connections 2024-09-16T15:55:37.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.669+0000 7f357f000640 1 --2- 172.21.15.151:0/3772521032 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f35780cccd0 0x7f35780cdbb0 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:37.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.669+0000 7f357f000640 1 --2- 172.21.15.151:0/3772521032 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3578145750 0x7f3578073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:37.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.669+0000 7f357f000640 1 -- 172.21.15.151:0/3772521032 >> 172.21.15.151:0/3772521032 conn(0x7f35780b1840 msgr2=0x7f35780b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:55:37.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.669+0000 7f357f000640 1 -- 172.21.15.151:0/3772521032 shutdown_connections 2024-09-16T15:55:37.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.669+0000 7f357f000640 1 -- 172.21.15.151:0/3772521032 wait complete. 2024-09-16T15:55:37.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.671+0000 7f357f000640 1 Processor -- start 2024-09-16T15:55:37.672 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.671+0000 7f357f000640 1 -- start start 2024-09-16T15:55:37.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.672+0000 7f357f000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f357806efa0 0x7f35781f8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:37.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.672+0000 7f357f000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f35780cccd0 0x7f357815e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:37.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.672+0000 7f357f000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3578145750 0x7f35781e8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:37.673 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.672+0000 7f357f000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f35780ce480 con 0x7f3578145750 2024-09-16T15:55:37.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.672+0000 7f357f000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f35780ce300 con 0x7f357806efa0 2024-09-16T15:55:37.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.672+0000 7f357f000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f35780ce600 con 0x7f35780cccd0 2024-09-16T15:55:37.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.672+0000 7f357d000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3578145750 0x7f35781e8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:37.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.672+0000 7f357d000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3578145750 0x7f35781e8600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47772/0 (socket says 172.21.15.151:47772) 2024-09-16T15:55:37.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.672+0000 7f357d000640 1 -- 172.21.15.151:0/678274193 learned_addr learned my addr 172.21.15.151:0/678274193 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:55:37.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.672+0000 7f3577e00640 1 --2- 172.21.15.151:0/678274193 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f357806efa0 0x7f35781f8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:37.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.672+0000 7f3577400640 1 --2- 172.21.15.151:0/678274193 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f35780cccd0 0x7f357815e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:37.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.673+0000 7f357d000640 1 -- 172.21.15.151:0/678274193 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f35780cccd0 msgr2=0x7f357815e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:37.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.673+0000 7f357d000640 1 --2- 172.21.15.151:0/678274193 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f35780cccd0 0x7f357815e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:37.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.673+0000 7f357d000640 1 -- 172.21.15.151:0/678274193 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f357806efa0 msgr2=0x7f35781f8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:37.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.673+0000 7f357d000640 1 --2- 172.21.15.151:0/678274193 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f357806efa0 0x7f35781f8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:37.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.673+0000 7f357d000640 1 -- 172.21.15.151:0/678274193 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f357813c310 con 0x7f3578145750 2024-09-16T15:55:37.675 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.673+0000 7f3577400640 1 --2- 172.21.15.151:0/678274193 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f35780cccd0 0x7f357815e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:55:37.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.673+0000 7f357d000640 1 --2- 172.21.15.151:0/678274193 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3578145750 0x7f35781e8600 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f3570006010 tx=0x7f3570003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:55:37.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.673+0000 7f356fe00640 1 -- 172.21.15.151:0/678274193 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3570047070 con 0x7f3578145750 2024-09-16T15:55:37.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.673+0000 7f356fe00640 1 -- 172.21.15.151:0/678274193 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3570004250 con 0x7f3578145750 2024-09-16T15:55:37.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.673+0000 7f356fe00640 1 -- 172.21.15.151:0/678274193 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3570033050 con 0x7f3578145750 2024-09-16T15:55:37.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.673+0000 7f357f000640 1 -- 172.21.15.151:0/678274193 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f357813c5d0 con 0x7f3578145750 2024-09-16T15:55:37.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.673+0000 7f357f000640 1 -- 172.21.15.151:0/678274193 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f357815e9d0 con 0x7f3578145750 2024-09-16T15:55:37.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.674+0000 7f357f000640 1 -- 172.21.15.151:0/678274193 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35780cdbb0 con 0x7f3578145750 2024-09-16T15:55:37.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.676+0000 7f356fe00640 1 -- 172.21.15.151:0/678274193 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 51) ==== 104500+0+0 (secure 0 0 0) 0x7f357004a070 con 0x7f3578145750 2024-09-16T15:55:37.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.676+0000 7f356fe00640 1 --2- 172.21.15.151:0/678274193 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f355807abc0 0x7f355807d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:37.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.677+0000 7f356fe00640 1 -- 172.21.15.151:0/678274193 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5907+0+0 (secure 0 0 0) 0x7f35700bad20 con 0x7f3578145750 2024-09-16T15:55:37.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.677+0000 7f3577e00640 1 --2- 172.21.15.151:0/678274193 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f355807abc0 0x7f355807d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:37.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.678+0000 7f3577e00640 1 --2- 172.21.15.151:0/678274193 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f355807abc0 0x7f355807d080 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f35781432f0 tx=0x7f356800b2e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:55:37.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.680+0000 7f356fe00640 1 -- 172.21.15.151:0/678274193 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f35700847a0 con 0x7f3578145750 2024-09-16T15:55:37.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.891+0000 7f357f000640 1 -- 172.21.15.151:0/678274193 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7f357813c7a0 con 0x7f355807abc0 2024-09-16T15:55:37.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.897+0000 7f356fe00640 1 -- 172.21.15.151:0/678274193 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+59 (secure 0 0 0) 0x7f357813c7a0 con 0x7f355807abc0 2024-09-16T15:55:37.898 INFO:teuthology.orchestra.run.smithi151.stdout:/volumes/_nogroup/sv1/61d89b29-c3ef-41d1-a443-b6dc24b686d3 2024-09-16T15:55:37.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.899+0000 7f357f000640 1 -- 172.21.15.151:0/678274193 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f355807abc0 msgr2=0x7f355807d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:37.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.899+0000 7f357f000640 1 --2- 172.21.15.151:0/678274193 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f355807abc0 0x7f355807d080 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f35781432f0 tx=0x7f356800b2e0 comp rx=0 tx=0).stop 2024-09-16T15:55:37.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.899+0000 7f357f000640 1 -- 172.21.15.151:0/678274193 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3578145750 msgr2=0x7f35781e8600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:37.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.899+0000 7f357f000640 1 --2- 172.21.15.151:0/678274193 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3578145750 0x7f35781e8600 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f3570006010 tx=0x7f3570003f10 comp rx=0 tx=0).stop 2024-09-16T15:55:37.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.899+0000 7f357f000640 1 -- 172.21.15.151:0/678274193 shutdown_connections 2024-09-16T15:55:37.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.899+0000 7f357f000640 1 --2- 172.21.15.151:0/678274193 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f355807abc0 0x7f355807d080 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:37.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.899+0000 7f357f000640 1 --2- 172.21.15.151:0/678274193 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3578145750 0x7f35781e8600 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:37.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.899+0000 7f357f000640 1 --2- 172.21.15.151:0/678274193 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f35780cccd0 0x7f357815e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:37.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.899+0000 7f357f000640 1 --2- 172.21.15.151:0/678274193 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f357806efa0 0x7f35781f8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:37.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.899+0000 7f357f000640 1 -- 172.21.15.151:0/678274193 >> 172.21.15.151:0/678274193 conn(0x7f35780b1840 msgr2=0x7f35781440c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:55:37.901 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.899+0000 7f357f000640 1 -- 172.21.15.151:0/678274193 shutdown_connections 2024-09-16T15:55:37.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:37.899+0000 7f357f000640 1 -- 172.21.15.151:0/678274193 wait complete. 2024-09-16T15:55:37.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:55:37.916 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/61d89b29-c3ef-41d1-a443-b6dc24b686d3 2024-09-16T15:55:37.916 DEBUG:teuthology.orchestra.run.smithi151:> ) 2024-09-16T15:55:37.952 INFO:teuthology.orchestra.run.smithi151.stdout:# file: volumes/_nogroup/sv1/61d89b29-c3ef-41d1-a443-b6dc24b686d3 2024-09-16T15:55:37.952 INFO:teuthology.orchestra.run.smithi151.stdout:ceph.dir.rbytes="2972172288" 2024-09-16T15:55:37.952 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:55:37.953 INFO:tasks.cephfs.test_volumes:r_size = 2972172288 exp_size = 3221225472 2024-09-16T15:55:39.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:55:39.954 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/61d89b29-c3ef-41d1-a443-b6dc24b686d3 2024-09-16T15:55:39.954 DEBUG:teuthology.orchestra.run.smithi151:> ) 2024-09-16T15:55:39.996 INFO:teuthology.orchestra.run.smithi151.stdout:# file: volumes/_nogroup/sv1/61d89b29-c3ef-41d1-a443-b6dc24b686d3 2024-09-16T15:55:39.996 INFO:teuthology.orchestra.run.smithi151.stdout:ceph.dir.rbytes="2972172288" 2024-09-16T15:55:39.996 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:55:39.997 INFO:tasks.cephfs.test_volumes:r_size = 2972172288 exp_size = 3221225472 2024-09-16T15:55:41.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:55:41.998 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/61d89b29-c3ef-41d1-a443-b6dc24b686d3 2024-09-16T15:55:41.998 DEBUG:teuthology.orchestra.run.smithi151:> ) 2024-09-16T15:55:42.039 INFO:teuthology.orchestra.run.smithi151.stdout:# file: volumes/_nogroup/sv1/61d89b29-c3ef-41d1-a443-b6dc24b686d3 2024-09-16T15:55:42.040 INFO:teuthology.orchestra.run.smithi151.stdout:ceph.dir.rbytes="2972172288" 2024-09-16T15:55:42.040 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:55:42.041 INFO:tasks.cephfs.test_volumes:r_size = 2972172288 exp_size = 3221225472 2024-09-16T15:55:44.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:55:44.042 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/61d89b29-c3ef-41d1-a443-b6dc24b686d3 2024-09-16T15:55:44.042 DEBUG:teuthology.orchestra.run.smithi151:> ) 2024-09-16T15:55:44.083 INFO:teuthology.orchestra.run.smithi151.stdout:# file: volumes/_nogroup/sv1/61d89b29-c3ef-41d1-a443-b6dc24b686d3 2024-09-16T15:55:44.083 INFO:teuthology.orchestra.run.smithi151.stdout:ceph.dir.rbytes="3221225472" 2024-09-16T15:55:44.083 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:55:44.084 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 exp_size = 3221225472 2024-09-16T15:55:44.084 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone62040 2024-09-16T15:55:44.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.293+0000 7f0d8fe00640 1 -- 172.21.15.151:0/1442977923 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d9006e660 msgr2=0x7f0d9006ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:44.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.293+0000 7f0d8fe00640 1 --2- 172.21.15.151:0/1442977923 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d9006e660 0x7f0d9006ead0 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f0d90159c50 tx=0x7f0d8802a9d0 comp rx=0 tx=0).stop 2024-09-16T15:55:44.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.294+0000 7f0d8fe00640 1 -- 172.21.15.151:0/1442977923 shutdown_connections 2024-09-16T15:55:44.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.294+0000 7f0d8fe00640 1 --2- 172.21.15.151:0/1442977923 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d9006e660 0x7f0d9006ead0 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:44.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.294+0000 7f0d8fe00640 1 --2- 172.21.15.151:0/1442977923 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d9015f670 0x7f0d900c6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:44.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.294+0000 7f0d8fe00640 1 -- 172.21.15.151:0/1442977923 >> 172.21.15.151:0/1442977923 conn(0x7f0d900cc490 msgr2=0x7f0d900cc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:55:44.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.294+0000 7f0d8fe00640 1 -- 172.21.15.151:0/1442977923 shutdown_connections 2024-09-16T15:55:44.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.294+0000 7f0d8fe00640 1 -- 172.21.15.151:0/1442977923 wait complete. 2024-09-16T15:55:44.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.295+0000 7f0d8fe00640 1 Processor -- start 2024-09-16T15:55:44.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.295+0000 7f0d8fe00640 1 -- start start 2024-09-16T15:55:44.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.296+0000 7f0d8fe00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d9006e660 0x7f0d900c9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:44.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.296+0000 7f0d8fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d900c5730 0x7f0d900b3420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:44.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.296+0000 7f0d8fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d9015f670 0x7f0d901f4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:44.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.296+0000 7f0d8fe00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0d900ca4f0 con 0x7f0d9015f670 2024-09-16T15:55:44.298 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.296+0000 7f0d8fe00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0d900ca370 con 0x7f0d900c5730 2024-09-16T15:55:44.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.296+0000 7f0d8fe00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f0d9006f010 con 0x7f0d9006e660 2024-09-16T15:55:44.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.297+0000 7f0d8f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d9015f670 0x7f0d901f4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:44.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.297+0000 7f0d8f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d9015f670 0x7f0d901f4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:59242/0 (socket says 172.21.15.151:59242) 2024-09-16T15:55:44.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.297+0000 7f0d8f400640 1 -- 172.21.15.151:0/2447052454 learned_addr learned my addr 172.21.15.151:0/2447052454 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:55:44.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.297+0000 7f0d8ea00640 1 --2- 172.21.15.151:0/2447052454 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d9006e660 0x7f0d900c9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:44.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.297+0000 7f0d86000640 1 --2- 172.21.15.151:0/2447052454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d900c5730 0x7f0d900b3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:44.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.297+0000 7f0d8f400640 1 -- 172.21.15.151:0/2447052454 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d9006e660 msgr2=0x7f0d900c9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:44.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.297+0000 7f0d8f400640 1 --2- 172.21.15.151:0/2447052454 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d9006e660 0x7f0d900c9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:44.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.297+0000 7f0d8f400640 1 -- 172.21.15.151:0/2447052454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d900c5730 msgr2=0x7f0d900b3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:44.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.297+0000 7f0d8f400640 1 --2- 172.21.15.151:0/2447052454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d900c5730 0x7f0d900b3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:44.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.297+0000 7f0d8f400640 1 -- 172.21.15.151:0/2447052454 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d900b2960 con 0x7f0d9015f670 2024-09-16T15:55:44.299 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.297+0000 7f0d8f400640 1 --2- 172.21.15.151:0/2447052454 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d9015f670 0x7f0d901f4660 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f0d88009d60 tx=0x7f0d88003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:55:44.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.298+0000 7f0d87400640 1 -- 172.21.15.151:0/2447052454 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d88047070 con 0x7f0d9015f670 2024-09-16T15:55:44.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.298+0000 7f0d87400640 1 -- 172.21.15.151:0/2447052454 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0d88003cf0 con 0x7f0d9015f670 2024-09-16T15:55:44.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.298+0000 7f0d8fe00640 1 -- 172.21.15.151:0/2447052454 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d900c9650 con 0x7f0d9015f670 2024-09-16T15:55:44.300 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.298+0000 7f0d87400640 1 -- 172.21.15.151:0/2447052454 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d88033050 con 0x7f0d9015f670 2024-09-16T15:55:44.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.298+0000 7f0d8fe00640 1 -- 172.21.15.151:0/2447052454 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d900b2c20 con 0x7f0d9015f670 2024-09-16T15:55:44.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.299+0000 7f0d8fe00640 1 -- 172.21.15.151:0/2447052454 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d9006ead0 con 0x7f0d9015f670 2024-09-16T15:55:44.301 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.300+0000 7f0d87400640 1 -- 172.21.15.151:0/2447052454 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 51) ==== 104500+0+0 (secure 0 0 0) 0x7f0d8804a070 con 0x7f0d9015f670 2024-09-16T15:55:44.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.301+0000 7f0d87400640 1 --2- 172.21.15.151:0/2447052454 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0d6c07abc0 0x7f0d6c07d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:44.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.301+0000 7f0d87400640 1 -- 172.21.15.151:0/2447052454 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5907+0+0 (secure 0 0 0) 0x7f0d880ba880 con 0x7f0d9015f670 2024-09-16T15:55:44.302 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.301+0000 7f0d8ea00640 1 --2- 172.21.15.151:0/2447052454 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0d6c07abc0 0x7f0d6c07d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:44.303 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.302+0000 7f0d8ea00640 1 --2- 172.21.15.151:0/2447052454 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0d6c07abc0 0x7f0d6c07d080 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f0d8000bb50 tx=0x7f0d8000b2e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:55:44.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.305+0000 7f0d87400640 1 -- 172.21.15.151:0/2447052454 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0d88084300 con 0x7f0d9015f670 2024-09-16T15:55:44.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.528+0000 7f0d8fe00640 1 -- 172.21.15.151:0/2447052454 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone62040", "target": ["mon-mgr", ""]}) -- 0x7f0d900aead0 con 0x7f0d6c07abc0 2024-09-16T15:55:44.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.594+0000 7f0d87400640 1 -- 172.21.15.151:0/2447052454 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0d900aead0 con 0x7f0d6c07abc0 2024-09-16T15:55:44.598 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.597+0000 7f0d8fe00640 1 -- 172.21.15.151:0/2447052454 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0d6c07abc0 msgr2=0x7f0d6c07d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:44.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.597+0000 7f0d8fe00640 1 --2- 172.21.15.151:0/2447052454 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0d6c07abc0 0x7f0d6c07d080 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f0d8000bb50 tx=0x7f0d8000b2e0 comp rx=0 tx=0).stop 2024-09-16T15:55:44.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.597+0000 7f0d8fe00640 1 -- 172.21.15.151:0/2447052454 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d9015f670 msgr2=0x7f0d901f4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:44.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.597+0000 7f0d8fe00640 1 --2- 172.21.15.151:0/2447052454 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d9015f670 0x7f0d901f4660 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f0d88009d60 tx=0x7f0d88003970 comp rx=0 tx=0).stop 2024-09-16T15:55:44.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.597+0000 7f0d8fe00640 1 -- 172.21.15.151:0/2447052454 shutdown_connections 2024-09-16T15:55:44.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.597+0000 7f0d8fe00640 1 --2- 172.21.15.151:0/2447052454 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0d6c07abc0 0x7f0d6c07d080 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:44.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.597+0000 7f0d8fe00640 1 --2- 172.21.15.151:0/2447052454 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d900c5730 0x7f0d900b3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:44.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.597+0000 7f0d8fe00640 1 --2- 172.21.15.151:0/2447052454 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d9015f670 0x7f0d901f4660 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:44.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.597+0000 7f0d8fe00640 1 --2- 172.21.15.151:0/2447052454 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d9006e660 0x7f0d900c9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:44.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.597+0000 7f0d8fe00640 1 -- 172.21.15.151:0/2447052454 >> 172.21.15.151:0/2447052454 conn(0x7f0d900cc490 msgr2=0x7f0d900739f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:55:44.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.597+0000 7f0d8fe00640 1 -- 172.21.15.151:0/2447052454 shutdown_connections 2024-09-16T15:55:44.599 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.597+0000 7f0d8fe00640 1 -- 172.21.15.151:0/2447052454 wait complete. 2024-09-16T15:55:44.601 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:55:44.600+0000 7f48f6200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:55:44.601 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:55:44.600+0000 7f48f6200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:55:44.601 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:55:44.600+0000 7f48f6200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:55:44.601 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:55:44.600+0000 7f48f6200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:55:44.601 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:55:44.600+0000 7f48f6200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:55:44.615 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone21631 2024-09-16T15:55:44.618 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:55:44.617+0000 7f48f6c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:55:44.618 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:55:44.617+0000 7f48f6c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:55:44.618 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:55:44.617+0000 7f48f6c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:55:44.618 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:55:44.617+0000 7f48f6c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:55:44.618 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:55:44.617+0000 7f48f6c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:55:44.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.778+0000 7f0a5cc00640 1 -- 172.21.15.151:0/4185679755 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a580ccb00 msgr2=0x7f0a5815f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:44.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.778+0000 7f0a5cc00640 1 --2- 172.21.15.151:0/4185679755 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a580ccb00 0x7f0a5815f9f0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f0a58074ff0 tx=0x7f0a5002a980 comp rx=0 tx=0).stop 2024-09-16T15:55:44.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.779+0000 7f0a5cc00640 1 -- 172.21.15.151:0/4185679755 shutdown_connections 2024-09-16T15:55:44.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.779+0000 7f0a5cc00640 1 --2- 172.21.15.151:0/4185679755 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a580ccb00 0x7f0a5815f9f0 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:44.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.779+0000 7f0a5cc00640 1 --2- 172.21.15.151:0/4185679755 >> v2:172.21.15.167:3300/0 conn(0x7f0a5806efa0 0x7f0a580cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:44.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.779+0000 7f0a5cc00640 1 --2- 172.21.15.151:0/4185679755 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a58145750 0x7f0a58073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:44.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.779+0000 7f0a5cc00640 1 -- 172.21.15.151:0/4185679755 >> 172.21.15.151:0/4185679755 conn(0x7f0a580b1840 msgr2=0x7f0a580b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:55:44.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.779+0000 7f0a5cc00640 1 -- 172.21.15.151:0/4185679755 shutdown_connections 2024-09-16T15:55:44.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.779+0000 7f0a5cc00640 1 -- 172.21.15.151:0/4185679755 wait complete. 2024-09-16T15:55:44.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.780+0000 7f0a5cc00640 1 Processor -- start 2024-09-16T15:55:44.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.780+0000 7f0a5cc00640 1 -- start start 2024-09-16T15:55:44.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.781+0000 7f0a5cc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a5806efa0 0x7f0a580ac310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:44.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.781+0000 7f0a5cc00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a580ccb00 0x7f0a581efda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:44.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.781+0000 7f0a5cc00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a58145750 0x7f0a58137c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:44.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.781+0000 7f0a5cc00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0a581600b0 con 0x7f0a5806efa0 2024-09-16T15:55:44.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.781+0000 7f0a5cc00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0a5815ff30 con 0x7f0a58145750 2024-09-16T15:55:44.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.781+0000 7f0a5cc00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f0a58160230 con 0x7f0a580ccb00 2024-09-16T15:55:44.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.782+0000 7f0a55e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a5806efa0 0x7f0a580ac310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:44.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.782+0000 7f0a55e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a5806efa0 0x7f0a580ac310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:59252/0 (socket says 172.21.15.151:59252) 2024-09-16T15:55:44.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.782+0000 7f0a55e00640 1 -- 172.21.15.151:0/3945271411 learned_addr learned my addr 172.21.15.151:0/3945271411 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:55:44.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.782+0000 7f0a55400640 1 --2- 172.21.15.151:0/3945271411 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a580ccb00 0x7f0a581efda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:44.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.782+0000 7f0a56800640 1 --2- 172.21.15.151:0/3945271411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a58145750 0x7f0a58137c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:44.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.782+0000 7f0a55e00640 1 -- 172.21.15.151:0/3945271411 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a580ccb00 msgr2=0x7f0a581efda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:44.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.782+0000 7f0a55e00640 1 --2- 172.21.15.151:0/3945271411 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a580ccb00 0x7f0a581efda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:44.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.782+0000 7f0a55e00640 1 -- 172.21.15.151:0/3945271411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a58145750 msgr2=0x7f0a58137c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:44.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.782+0000 7f0a55e00640 1 --2- 172.21.15.151:0/3945271411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a58145750 0x7f0a58137c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:44.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.782+0000 7f0a55e00640 1 -- 172.21.15.151:0/3945271411 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a581f02e0 con 0x7f0a5806efa0 2024-09-16T15:55:44.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.782+0000 7f0a55400640 1 --2- 172.21.15.151:0/3945271411 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a580ccb00 0x7f0a581efda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:55:44.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.782+0000 7f0a56800640 1 --2- 172.21.15.151:0/3945271411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a58145750 0x7f0a58137c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:55:44.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.782+0000 7f0a55e00640 1 --2- 172.21.15.151:0/3945271411 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a5806efa0 0x7f0a580ac310 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f0a4800d740 tx=0x7f0a4800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:55:44.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.782+0000 7f0a4e000640 1 -- 172.21.15.151:0/3945271411 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a48017070 con 0x7f0a5806efa0 2024-09-16T15:55:44.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.783+0000 7f0a4e000640 1 -- 172.21.15.151:0/3945271411 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0a48009ab0 con 0x7f0a5806efa0 2024-09-16T15:55:44.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.783+0000 7f0a4e000640 1 -- 172.21.15.151:0/3945271411 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a48010460 con 0x7f0a5806efa0 2024-09-16T15:55:44.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.783+0000 7f0a5cc00640 1 -- 172.21.15.151:0/3945271411 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a580ac960 con 0x7f0a5806efa0 2024-09-16T15:55:44.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.783+0000 7f0a5cc00640 1 -- 172.21.15.151:0/3945271411 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a580acf20 con 0x7f0a5806efa0 2024-09-16T15:55:44.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.784+0000 7f0a5cc00640 1 -- 172.21.15.151:0/3945271411 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a58160710 con 0x7f0a5806efa0 2024-09-16T15:55:44.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.785+0000 7f0a4e000640 1 -- 172.21.15.151:0/3945271411 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 51) ==== 104500+0+0 (secure 0 0 0) 0x7f0a4801a070 con 0x7f0a5806efa0 2024-09-16T15:55:44.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.786+0000 7f0a4e000640 1 --2- 172.21.15.151:0/3945271411 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0a3807abf0 0x7f0a3807d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:44.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.786+0000 7f0a4e000640 1 -- 172.21.15.151:0/3945271411 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5907+0+0 (secure 0 0 0) 0x7f0a48099c40 con 0x7f0a5806efa0 2024-09-16T15:55:44.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.786+0000 7f0a55400640 1 --2- 172.21.15.151:0/3945271411 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0a3807abf0 0x7f0a3807d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:44.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.787+0000 7f0a55400640 1 --2- 172.21.15.151:0/3945271411 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0a3807abf0 0x7f0a3807d0b0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f0a580ad5d0 tx=0x7f0a440094b0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:55:44.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:44.790+0000 7f0a4e000640 1 -- 172.21.15.151:0/3945271411 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0a48063d10 con 0x7f0a5806efa0 2024-09-16T15:55:45.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:45.009+0000 7f0a5cc00640 1 -- 172.21.15.151:0/3945271411 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone21631", "target": ["mon-mgr", ""]}) -- 0x7f0a580b0460 con 0x7f0a3807abf0 2024-09-16T15:55:45.600 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:55:45.599+0000 7f4851c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:55:45.601 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:55:45.599+0000 7f4851c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:55:45.601 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:55:45.599+0000 7f4851c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:55:45.601 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:55:45.599+0000 7f4851c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:55:45.601 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:55:45.599+0000 7f4851c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:55:45.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:45.690+0000 7f0a4e000640 1 -- 172.21.15.151:0/3945271411 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0a580b0460 con 0x7f0a3807abf0 2024-09-16T15:55:45.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:45.695+0000 7f0a5cc00640 1 -- 172.21.15.151:0/3945271411 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0a3807abf0 msgr2=0x7f0a3807d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:45.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:45.695+0000 7f0a5cc00640 1 --2- 172.21.15.151:0/3945271411 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0a3807abf0 0x7f0a3807d0b0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f0a580ad5d0 tx=0x7f0a440094b0 comp rx=0 tx=0).stop 2024-09-16T15:55:45.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:45.695+0000 7f0a5cc00640 1 -- 172.21.15.151:0/3945271411 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a5806efa0 msgr2=0x7f0a580ac310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:45.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:45.695+0000 7f0a5cc00640 1 --2- 172.21.15.151:0/3945271411 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a5806efa0 0x7f0a580ac310 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f0a4800d740 tx=0x7f0a4800dc00 comp rx=0 tx=0).stop 2024-09-16T15:55:45.697 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:45.695+0000 7f0a5cc00640 1 -- 172.21.15.151:0/3945271411 shutdown_connections 2024-09-16T15:55:45.697 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:45.695+0000 7f0a5cc00640 1 --2- 172.21.15.151:0/3945271411 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0a3807abf0 0x7f0a3807d0b0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:45.697 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:45.695+0000 7f0a5cc00640 1 --2- 172.21.15.151:0/3945271411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a58145750 0x7f0a58137c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:45.697 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:45.695+0000 7f0a5cc00640 1 --2- 172.21.15.151:0/3945271411 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a580ccb00 0x7f0a581efda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:45.697 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:45.695+0000 7f0a5cc00640 1 --2- 172.21.15.151:0/3945271411 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a5806efa0 0x7f0a580ac310 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:45.697 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:45.695+0000 7f0a5cc00640 1 -- 172.21.15.151:0/3945271411 >> 172.21.15.151:0/3945271411 conn(0x7f0a580b1840 msgr2=0x7f0a581420f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:55:45.697 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:45.695+0000 7f0a5cc00640 1 -- 172.21.15.151:0/3945271411 shutdown_connections 2024-09-16T15:55:45.697 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:45.695+0000 7f0a5cc00640 1 -- 172.21.15.151:0/3945271411 wait complete. 2024-09-16T15:55:45.719 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone54262 2024-09-16T15:55:45.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:45.962+0000 7f2202800640 1 -- 172.21.15.151:0/3732579669 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f21e4003a50 msgr2=0x7f21e4003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:45.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:45.962+0000 7f2202800640 1 --2- 172.21.15.151:0/3732579669 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f21e4003a50 0x7f21e4003e70 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f21f00099d0 tx=0x7f21e4005830 comp rx=0 tx=0).stop 2024-09-16T15:55:45.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:45.962+0000 7f2201e00640 1 -- 172.21.15.151:0/3732579669 reap_dead start 2024-09-16T15:55:45.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:45.962+0000 7f2202800640 1 -- 172.21.15.151:0/3732579669 shutdown_connections 2024-09-16T15:55:45.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:45.962+0000 7f2202800640 1 -- 172.21.15.151:0/3732579669 >> 172.21.15.151:0/3732579669 conn(0x7f21fc0cc490 msgr2=0x7f21fc0cc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:55:45.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:45.962+0000 7f2202800640 1 -- 172.21.15.151:0/3732579669 shutdown_connections 2024-09-16T15:55:45.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:45.962+0000 7f2202800640 1 -- 172.21.15.151:0/3732579669 wait complete. 2024-09-16T15:55:45.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:45.964+0000 7f2202800640 1 Processor -- start 2024-09-16T15:55:45.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:45.964+0000 7f2202800640 1 -- start start 2024-09-16T15:55:45.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:45.965+0000 7f2202800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f21e4003a50 0x7f21fc0b4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:45.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:45.965+0000 7f2202800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f21fc143230 0x7f21fc0b3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:45.966 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:45.965+0000 7f2202800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f21fc0b3d60 0x7f21fc0b4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:45.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:45.965+0000 7f2202800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f21fc0addb0 con 0x7f21fc143230 2024-09-16T15:55:45.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:45.965+0000 7f2202800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f21fc0cdd80 con 0x7f21e4003a50 2024-09-16T15:55:45.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:45.965+0000 7f2202800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f21fc0adf30 con 0x7f21fc0b3d60 2024-09-16T15:55:45.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:45.965+0000 7f2200a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f21fc143230 0x7f21fc0b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:45.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:45.965+0000 7f2200a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f21fc143230 0x7f21fc0b3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:59284/0 (socket says 172.21.15.151:59284) 2024-09-16T15:55:45.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:45.965+0000 7f2200a00640 1 -- 172.21.15.151:0/2987051374 learned_addr learned my addr 172.21.15.151:0/2987051374 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:55:45.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:45.966+0000 7f2201e00640 1 --2- 172.21.15.151:0/2987051374 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f21fc0b3d60 0x7f21fc0b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:45.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:45.966+0000 7f2200a00640 1 -- 172.21.15.151:0/2987051374 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f21fc0b3d60 msgr2=0x7f21fc0b4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:45.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:45.966+0000 7f2200a00640 1 --2- 172.21.15.151:0/2987051374 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f21fc0b3d60 0x7f21fc0b4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:45.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:45.966+0000 7f2200a00640 1 -- 172.21.15.151:0/2987051374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f21e4003a50 msgr2=0x7f21fc0b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:45.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:45.966+0000 7f2200a00640 1 --2- 172.21.15.151:0/2987051374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f21e4003a50 0x7f21fc0b4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:45.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:45.966+0000 7f2200a00640 1 -- 172.21.15.151:0/2987051374 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21fc134210 con 0x7f21fc143230 2024-09-16T15:55:45.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:45.966+0000 7f2200a00640 1 --2- 172.21.15.151:0/2987051374 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f21fc143230 0x7f21fc0b3820 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f21e800e5f0 tx=0x7f21e800eab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:55:45.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:45.966+0000 7f21f9600640 1 -- 172.21.15.151:0/2987051374 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e8015070 con 0x7f21fc143230 2024-09-16T15:55:45.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:45.966+0000 7f21f9600640 1 -- 172.21.15.151:0/2987051374 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f21e80109b0 con 0x7f21fc143230 2024-09-16T15:55:45.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:45.966+0000 7f2202800640 1 -- 172.21.15.151:0/2987051374 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21fc0c6a50 con 0x7f21fc143230 2024-09-16T15:55:45.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:45.967+0000 7f21f9600640 1 -- 172.21.15.151:0/2987051374 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e8010e00 con 0x7f21fc143230 2024-09-16T15:55:45.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:45.967+0000 7f2202800640 1 -- 172.21.15.151:0/2987051374 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21fc1344b0 con 0x7f21fc143230 2024-09-16T15:55:45.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:45.968+0000 7f2202800640 1 -- 172.21.15.151:0/2987051374 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21fc0ad870 con 0x7f21fc143230 2024-09-16T15:55:45.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:45.972+0000 7f21f9600640 1 -- 172.21.15.151:0/2987051374 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 51) ==== 104500+0+0 (secure 0 0 0) 0x7f21e8018070 con 0x7f21fc143230 2024-09-16T15:55:45.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:45.974+0000 7f21f9600640 1 --2- 172.21.15.151:0/2987051374 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f21dc07abf0 0x7f21dc07d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:45.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:45.974+0000 7f21f9600640 1 -- 172.21.15.151:0/2987051374 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5907+0+0 (secure 0 0 0) 0x7f21e8099db0 con 0x7f21fc143230 2024-09-16T15:55:45.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:45.983+0000 7f21f9600640 1 -- 172.21.15.151:0/2987051374 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f21e8063f20 con 0x7f21fc143230 2024-09-16T15:55:45.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:45.985+0000 7f2201400640 1 --2- 172.21.15.151:0/2987051374 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f21dc07abf0 0x7f21dc07d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:45.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:45.987+0000 7f2201400640 1 --2- 172.21.15.151:0/2987051374 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f21dc07abf0 0x7f21dc07d0b0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f21f0034000 tx=0x7f21f00047d0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:55:46.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:46.342+0000 7f2202800640 1 -- 172.21.15.151:0/2987051374 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone54262", "target": ["mon-mgr", ""]}) -- 0x7f21fc074de0 con 0x7f21dc07abf0 2024-09-16T15:55:46.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:46.670+0000 7f21f9600640 1 -- 172.21.15.151:0/2987051374 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mgrmap(e 52) ==== 104653+0+0 (secure 0 0 0) 0x7f21e805fa90 con 0x7f21fc143230 2024-09-16T15:55:47.085 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:47.084+0000 7f21f9600640 1 -- 172.21.15.151:0/2987051374 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f21fc074de0 con 0x7f21dc07abf0 2024-09-16T15:55:47.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:47.087+0000 7f2202800640 1 -- 172.21.15.151:0/2987051374 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f21dc07abf0 msgr2=0x7f21dc07d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:47.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:47.087+0000 7f2202800640 1 --2- 172.21.15.151:0/2987051374 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f21dc07abf0 0x7f21dc07d0b0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f21f0034000 tx=0x7f21f00047d0 comp rx=0 tx=0).stop 2024-09-16T15:55:47.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:47.087+0000 7f2202800640 1 -- 172.21.15.151:0/2987051374 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f21fc143230 msgr2=0x7f21fc0b3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:47.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:47.087+0000 7f2202800640 1 --2- 172.21.15.151:0/2987051374 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f21fc143230 0x7f21fc0b3820 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f21e800e5f0 tx=0x7f21e800eab0 comp rx=0 tx=0).stop 2024-09-16T15:55:47.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:47.088+0000 7f2202800640 1 -- 172.21.15.151:0/2987051374 shutdown_connections 2024-09-16T15:55:47.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:47.088+0000 7f2202800640 1 --2- 172.21.15.151:0/2987051374 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f21dc07abf0 0x7f21dc07d0b0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:47.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:47.088+0000 7f2202800640 1 --2- 172.21.15.151:0/2987051374 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f21fc0b3d60 0x7f21fc0b4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:47.089 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:47.088+0000 7f2202800640 1 --2- 172.21.15.151:0/2987051374 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f21fc143230 0x7f21fc0b3820 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:47.090 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:47.088+0000 7f2202800640 1 --2- 172.21.15.151:0/2987051374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f21e4003a50 0x7f21fc0b4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:47.090 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:47.088+0000 7f2202800640 1 -- 172.21.15.151:0/2987051374 >> 172.21.15.151:0/2987051374 conn(0x7f21fc0cc490 msgr2=0x7f21fc06f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:55:47.090 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:47.088+0000 7f2202800640 1 -- 172.21.15.151:0/2987051374 shutdown_connections 2024-09-16T15:55:47.090 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:47.088+0000 7f2202800640 1 -- 172.21.15.151:0/2987051374 wait complete. 2024-09-16T15:55:47.122 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone83343 2024-09-16T15:55:47.130 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:55:47.126+0000 7f48f6c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:55:47.131 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:55:47.126+0000 7f48f6c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:55:47.131 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:55:47.126+0000 7f48f6c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:55:47.131 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:55:47.126+0000 7f48f6c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:55:47.131 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:55:47.126+0000 7f48f6c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:55:47.310 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:55:47.309+0000 7f48f6c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:55:47.310 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:55:47.309+0000 7f48f6c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:55:47.310 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:55:47.309+0000 7f48f6c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:55:47.310 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:55:47.309+0000 7f48f6c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:55:47.310 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:55:47.309+0000 7f48f6c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:55:47.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:47.370+0000 7f3b7ae00640 1 -- 172.21.15.151:0/1413403776 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b7406efa0 msgr2=0x7f3b740cc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:47.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:47.370+0000 7f3b7ae00640 1 --2- 172.21.15.151:0/1413403776 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b7406efa0 0x7f3b740cc5c0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f3b740d0cd0 tx=0x7f3b5c02a4d0 comp rx=0 tx=0).stop 2024-09-16T15:55:47.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:47.370+0000 7f3b7ae00640 1 -- 172.21.15.151:0/1413403776 shutdown_connections 2024-09-16T15:55:47.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:47.370+0000 7f3b7ae00640 1 --2- 172.21.15.151:0/1413403776 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3b740ccb00 0x7f3b7415f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:47.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:47.370+0000 7f3b7ae00640 1 --2- 172.21.15.151:0/1413403776 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b7406efa0 0x7f3b740cc5c0 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:47.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:47.370+0000 7f3b7ae00640 1 --2- 172.21.15.151:0/1413403776 >> v2:172.21.15.167:3300/0 conn(0x7f3b74145750 0x7f3b74073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:47.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:47.370+0000 7f3b7ae00640 1 -- 172.21.15.151:0/1413403776 >> 172.21.15.151:0/1413403776 conn(0x7f3b740b1840 msgr2=0x7f3b740b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:55:47.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:47.370+0000 7f3b7ae00640 1 -- 172.21.15.151:0/1413403776 shutdown_connections 2024-09-16T15:55:47.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:47.370+0000 7f3b7ae00640 1 -- 172.21.15.151:0/1413403776 wait complete. 2024-09-16T15:55:47.373 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:47.373+0000 7f3b7ae00640 1 Processor -- start 2024-09-16T15:55:47.374 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:47.373+0000 7f3b7ae00640 1 -- start start 2024-09-16T15:55:47.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:47.376+0000 7f3b7ae00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3b7406efa0 0x7f3b7414f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:47.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:47.376+0000 7f3b7ae00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b740ccb00 0x7f3b741f3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:47.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:47.376+0000 7f3b7ae00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b74145750 0x7f3b740c7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:47.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:47.376+0000 7f3b7ae00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3b741602c0 con 0x7f3b740ccb00 2024-09-16T15:55:47.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:47.376+0000 7f3b7ae00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3b74160140 con 0x7f3b74145750 2024-09-16T15:55:47.377 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:47.376+0000 7f3b7ae00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f3b74160440 con 0x7f3b7406efa0 2024-09-16T15:55:47.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:47.376+0000 7f3b6be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b740ccb00 0x7f3b741f3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:47.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:47.376+0000 7f3b6be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b740ccb00 0x7f3b741f3170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:59312/0 (socket says 172.21.15.151:59312) 2024-09-16T15:55:47.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:47.376+0000 7f3b6be00640 1 -- 172.21.15.151:0/3598067633 learned_addr learned my addr 172.21.15.151:0/3598067633 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:55:47.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:47.377+0000 7f3b73e00640 1 --2- 172.21.15.151:0/3598067633 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3b7406efa0 0x7f3b7414f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:47.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:47.377+0000 7f3b6be00640 1 -- 172.21.15.151:0/3598067633 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3b7406efa0 msgr2=0x7f3b7414f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:47.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:47.377+0000 7f3b6be00640 1 --2- 172.21.15.151:0/3598067633 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3b7406efa0 0x7f3b7414f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:47.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:47.377+0000 7f3b6be00640 1 -- 172.21.15.151:0/3598067633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b74145750 msgr2=0x7f3b740c7350 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T15:55:47.378 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:47.377+0000 7f3b6be00640 1 --2- 172.21.15.151:0/3598067633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b74145750 0x7f3b740c7350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:47.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:47.377+0000 7f3b6be00640 1 -- 172.21.15.151:0/3598067633 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b74152100 con 0x7f3b740ccb00 2024-09-16T15:55:47.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:47.377+0000 7f3b6be00640 1 --2- 172.21.15.151:0/3598067633 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b740ccb00 0x7f3b741f3170 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c009850 tx=0x7f3b5c003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:55:47.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:47.378+0000 7f3b70c00640 1 -- 172.21.15.151:0/3598067633 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b5c03f070 con 0x7f3b740ccb00 2024-09-16T15:55:47.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:47.378+0000 7f3b70c00640 1 -- 172.21.15.151:0/3598067633 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3b5c0047a0 con 0x7f3b740ccb00 2024-09-16T15:55:47.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:47.378+0000 7f3b70c00640 1 -- 172.21.15.151:0/3598067633 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b5c035030 con 0x7f3b740ccb00 2024-09-16T15:55:47.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:47.378+0000 7f3b7ae00640 1 -- 172.21.15.151:0/3598067633 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b740ac080 con 0x7f3b740ccb00 2024-09-16T15:55:47.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:47.378+0000 7f3b7ae00640 1 -- 172.21.15.151:0/3598067633 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b740ac640 con 0x7f3b740ccb00 2024-09-16T15:55:47.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:47.383+0000 7f3b70c00640 1 -- 172.21.15.151:0/3598067633 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 52) ==== 104653+0+0 (secure 0 0 0) 0x7f3b5c042070 con 0x7f3b740ccb00 2024-09-16T15:55:47.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:47.383+0000 7f3b7ae00640 1 -- 172.21.15.151:0/3598067633 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b7415f9f0 con 0x7f3b740ccb00 2024-09-16T15:55:47.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:47.384+0000 7f3b70c00640 1 --2- 172.21.15.151:0/3598067633 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3b5007ad70 0x7f3b5007d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:47.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:47.385+0000 7f3b70c00640 1 -- 172.21.15.151:0/3598067633 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5907+0+0 (secure 0 0 0) 0x7f3b5c0baa60 con 0x7f3b740ccb00 2024-09-16T15:55:47.392 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:47.391+0000 7f3b70c00640 1 -- 172.21.15.151:0/3598067633 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f3b5c084b30 con 0x7f3b740ccb00 2024-09-16T15:55:47.398 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:47.397+0000 7f3b73e00640 1 --2- 172.21.15.151:0/3598067633 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3b5007ad70 0x7f3b5007d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:47.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:47.401+0000 7f3b73e00640 1 --2- 172.21.15.151:0/3598067633 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3b5007ad70 0x7f3b5007d230 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f3b640058c0 tx=0x7f3b64005850 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:55:47.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:47.729+0000 7f3b7ae00640 1 -- 172.21.15.151:0/3598067633 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone83343", "target": ["mon-mgr", ""]}) -- 0x7f3b741f36b0 con 0x7f3b5007ad70 2024-09-16T15:55:48.741 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:48.740+0000 7f3b70c00640 1 -- 172.21.15.151:0/3598067633 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mgrmap(e 53) ==== 104755+0+0 (secure 0 0 0) 0x7f3b5c0806a0 con 0x7f3b740ccb00 2024-09-16T15:55:49.047 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:49.046+0000 7f3b70c00640 1 -- 172.21.15.151:0/3598067633 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3b741f36b0 con 0x7f3b5007ad70 2024-09-16T15:55:49.050 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:49.048+0000 7f3b7ae00640 1 -- 172.21.15.151:0/3598067633 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3b5007ad70 msgr2=0x7f3b5007d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:49.050 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:49.048+0000 7f3b7ae00640 1 --2- 172.21.15.151:0/3598067633 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3b5007ad70 0x7f3b5007d230 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f3b640058c0 tx=0x7f3b64005850 comp rx=0 tx=0).stop 2024-09-16T15:55:49.050 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:49.048+0000 7f3b7ae00640 1 -- 172.21.15.151:0/3598067633 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b740ccb00 msgr2=0x7f3b741f3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:49.050 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:49.048+0000 7f3b7ae00640 1 --2- 172.21.15.151:0/3598067633 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b740ccb00 0x7f3b741f3170 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c009850 tx=0x7f3b5c003910 comp rx=0 tx=0).stop 2024-09-16T15:55:49.050 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:49.049+0000 7f3b7ae00640 1 -- 172.21.15.151:0/3598067633 shutdown_connections 2024-09-16T15:55:49.050 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:49.049+0000 7f3b7ae00640 1 --2- 172.21.15.151:0/3598067633 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3b5007ad70 0x7f3b5007d230 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:49.050 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:49.049+0000 7f3b7ae00640 1 --2- 172.21.15.151:0/3598067633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3b74145750 0x7f3b740c7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:49.050 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:49.049+0000 7f3b7ae00640 1 --2- 172.21.15.151:0/3598067633 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b740ccb00 0x7f3b741f3170 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:49.050 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:49.049+0000 7f3b7ae00640 1 --2- 172.21.15.151:0/3598067633 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3b7406efa0 0x7f3b7414f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:49.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:49.049+0000 7f3b7ae00640 1 -- 172.21.15.151:0/3598067633 >> 172.21.15.151:0/3598067633 conn(0x7f3b740b1840 msgr2=0x7f3b740b10b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:55:49.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:49.049+0000 7f3b7ae00640 1 -- 172.21.15.151:0/3598067633 shutdown_connections 2024-09-16T15:55:49.051 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:49.049+0000 7f3b7ae00640 1 -- 172.21.15.151:0/3598067633 wait complete. 2024-09-16T15:55:49.085 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone48134 2024-09-16T15:55:49.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:49.337+0000 7f81e8c00640 1 -- 172.21.15.151:0/4194150495 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f81e40b14f0 msgr2=0x7f81e40cc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:49.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:49.337+0000 7f81e8c00640 1 --2- 172.21.15.151:0/4194150495 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f81e40b14f0 0x7f81e40cc760 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f81e4075370 tx=0x7f81cc02a620 comp rx=0 tx=0).stop 2024-09-16T15:55:49.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:49.338+0000 7f81e8c00640 1 -- 172.21.15.151:0/4194150495 shutdown_connections 2024-09-16T15:55:49.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:49.338+0000 7f81e8c00640 1 --2- 172.21.15.151:0/4194150495 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f81e40b14f0 0x7f81e40cc760 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:49.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:49.338+0000 7f81e8c00640 1 --2- 172.21.15.151:0/4194150495 >> v2:172.21.15.167:3300/0 conn(0x7f81e415df00 0x7f81e40b0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:49.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:49.338+0000 7f81e8c00640 1 -- 172.21.15.151:0/4194150495 >> 172.21.15.151:0/4194150495 conn(0x7f81e40d0330 msgr2=0x7f81e40d0740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:55:49.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:49.339+0000 7f81e8c00640 1 -- 172.21.15.151:0/4194150495 shutdown_connections 2024-09-16T15:55:49.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:49.340+0000 7f81e8c00640 1 -- 172.21.15.151:0/4194150495 wait complete. 2024-09-16T15:55:49.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:49.342+0000 7f81e8c00640 1 Processor -- start 2024-09-16T15:55:49.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:49.342+0000 7f81e8c00640 1 -- start start 2024-09-16T15:55:49.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:49.343+0000 7f81e8c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f81e40ccca0 0x7f81e40c95d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:49.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:49.343+0000 7f81e8c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81e415df00 0x7f81e40c48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:49.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:49.343+0000 7f81e8c00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f81e40b4b40 0x7f81e40c4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:49.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:49.343+0000 7f81e8c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f81e40b0b80 con 0x7f81e40ccca0 2024-09-16T15:55:49.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:49.343+0000 7f81e8c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f81e40b0a00 con 0x7f81e415df00 2024-09-16T15:55:49.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:49.343+0000 7f81e8c00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f81e40749e0 con 0x7f81e40b4b40 2024-09-16T15:55:49.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:49.343+0000 7f81e3400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f81e40ccca0 0x7f81e40c95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:49.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:49.343+0000 7f81e3400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f81e40ccca0 0x7f81e40c95d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:59336/0 (socket says 172.21.15.151:59336) 2024-09-16T15:55:49.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:49.343+0000 7f81e3400640 1 -- 172.21.15.151:0/1712134069 learned_addr learned my addr 172.21.15.151:0/1712134069 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:55:49.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:49.344+0000 7f81e2a00640 1 --2- 172.21.15.151:0/1712134069 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81e415df00 0x7f81e40c48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:49.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:49.344+0000 7f81e3e00640 1 --2- 172.21.15.151:0/1712134069 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f81e40b4b40 0x7f81e40c4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:49.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:49.344+0000 7f81e3400640 1 -- 172.21.15.151:0/1712134069 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f81e40b4b40 msgr2=0x7f81e40c4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:49.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:49.344+0000 7f81e3400640 1 --2- 172.21.15.151:0/1712134069 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f81e40b4b40 0x7f81e40c4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:49.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:49.344+0000 7f81e3400640 1 -- 172.21.15.151:0/1712134069 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81e415df00 msgr2=0x7f81e40c48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:49.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:49.344+0000 7f81e3400640 1 --2- 172.21.15.151:0/1712134069 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81e415df00 0x7f81e40c48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:49.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:49.344+0000 7f81e3400640 1 -- 172.21.15.151:0/1712134069 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81e41f8d70 con 0x7f81e40ccca0 2024-09-16T15:55:49.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:49.344+0000 7f81e3e00640 1 --2- 172.21.15.151:0/1712134069 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f81e40b4b40 0x7f81e40c4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:55:49.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:49.344+0000 7f81e2a00640 1 --2- 172.21.15.151:0/1712134069 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81e415df00 0x7f81e40c48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:55:49.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:49.344+0000 7f81e3400640 1 --2- 172.21.15.151:0/1712134069 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f81e40ccca0 0x7f81e40c95d0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f81d400d770 tx=0x7f81d400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:55:49.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:49.345+0000 7f81db400640 1 -- 172.21.15.151:0/1712134069 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d4017070 con 0x7f81e40ccca0 2024-09-16T15:55:49.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:49.345+0000 7f81e8c00640 1 -- 172.21.15.151:0/1712134069 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81e40b4ff0 con 0x7f81e40ccca0 2024-09-16T15:55:49.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:49.345+0000 7f81db400640 1 -- 172.21.15.151:0/1712134069 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f81d4009d20 con 0x7f81e40ccca0 2024-09-16T15:55:49.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:49.345+0000 7f81db400640 1 -- 172.21.15.151:0/1712134069 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d40106c0 con 0x7f81e40ccca0 2024-09-16T15:55:49.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:49.345+0000 7f81e8c00640 1 -- 172.21.15.151:0/1712134069 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81e40c9f40 con 0x7f81e40ccca0 2024-09-16T15:55:49.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:49.347+0000 7f81e8c00640 1 -- 172.21.15.151:0/1712134069 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81e4074e90 con 0x7f81e40ccca0 2024-09-16T15:55:49.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:49.350+0000 7f81db400640 1 -- 172.21.15.151:0/1712134069 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 53) ==== 104755+0+0 (secure 0 0 0) 0x7f81d401a070 con 0x7f81e40ccca0 2024-09-16T15:55:49.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:49.351+0000 7f81db400640 1 --2- 172.21.15.151:0/1712134069 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f81c007ae90 0x7f81c007d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:49.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:49.351+0000 7f81db400640 1 -- 172.21.15.151:0/1712134069 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5907+0+0 (secure 0 0 0) 0x7f81d4099de0 con 0x7f81e40ccca0 2024-09-16T15:55:49.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:49.361+0000 7f81e2a00640 1 --2- 172.21.15.151:0/1712134069 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f81c007ae90 0x7f81c007d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:49.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:49.364+0000 7f81db400640 1 -- 172.21.15.151:0/1712134069 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f81d4063ed0 con 0x7f81e40ccca0 2024-09-16T15:55:49.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:49.366+0000 7f81e2a00640 1 --2- 172.21.15.151:0/1712134069 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f81c007ae90 0x7f81c007d350 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f81cc033000 tx=0x7f81cc004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:55:49.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:49.809+0000 7f81e8c00640 1 -- 172.21.15.151:0/1712134069 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone48134", "target": ["mon-mgr", ""]}) -- 0x7f81e40747a0 con 0x7f81c007ae90 2024-09-16T15:55:51.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:51.357+0000 7f81db400640 1 -- 172.21.15.151:0/1712134069 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f81e40747a0 con 0x7f81c007ae90 2024-09-16T15:55:51.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:51.363+0000 7f81e8c00640 1 -- 172.21.15.151:0/1712134069 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f81c007ae90 msgr2=0x7f81c007d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:51.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:51.363+0000 7f81e8c00640 1 --2- 172.21.15.151:0/1712134069 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f81c007ae90 0x7f81c007d350 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f81cc033000 tx=0x7f81cc004730 comp rx=0 tx=0).stop 2024-09-16T15:55:51.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:51.363+0000 7f81e8c00640 1 -- 172.21.15.151:0/1712134069 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f81e40ccca0 msgr2=0x7f81e40c95d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:51.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:51.363+0000 7f81e8c00640 1 --2- 172.21.15.151:0/1712134069 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f81e40ccca0 0x7f81e40c95d0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f81d400d770 tx=0x7f81d400dc30 comp rx=0 tx=0).stop 2024-09-16T15:55:51.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:51.363+0000 7f81e8c00640 1 -- 172.21.15.151:0/1712134069 shutdown_connections 2024-09-16T15:55:51.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:51.363+0000 7f81e8c00640 1 --2- 172.21.15.151:0/1712134069 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f81c007ae90 0x7f81c007d350 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:51.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:51.363+0000 7f81e8c00640 1 --2- 172.21.15.151:0/1712134069 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81e415df00 0x7f81e40c48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:51.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:51.363+0000 7f81e8c00640 1 --2- 172.21.15.151:0/1712134069 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f81e40b4b40 0x7f81e40c4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:51.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:51.363+0000 7f81e8c00640 1 --2- 172.21.15.151:0/1712134069 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f81e40ccca0 0x7f81e40c95d0 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:51.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:51.363+0000 7f81e8c00640 1 -- 172.21.15.151:0/1712134069 >> 172.21.15.151:0/1712134069 conn(0x7f81e40d0330 msgr2=0x7f81e406f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:55:51.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:51.364+0000 7f81e8c00640 1 -- 172.21.15.151:0/1712134069 shutdown_connections 2024-09-16T15:55:51.368 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:51.366+0000 7f81e8c00640 1 -- 172.21.15.151:0/1712134069 wait complete. 2024-09-16T15:55:51.406 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone58175 2024-09-16T15:55:51.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:51.686+0000 7fc7e0a00640 1 -- 172.21.15.151:0/3342175522 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7dc0b8a60 msgr2=0x7fc7dc0d1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:51.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:51.686+0000 7fc7e0a00640 1 --2- 172.21.15.151:0/3342175522 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7dc0b8a60 0x7fc7dc0d1190 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fc7d4009ca0 tx=0x7fc7d402aa10 comp rx=0 tx=0).stop 2024-09-16T15:55:51.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:51.686+0000 7fc7e0a00640 1 -- 172.21.15.151:0/3342175522 shutdown_connections 2024-09-16T15:55:51.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:51.686+0000 7fc7e0a00640 1 --2- 172.21.15.151:0/3342175522 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7dc0b8a60 0x7fc7dc0d1190 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:51.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:51.686+0000 7fc7e0a00640 1 --2- 172.21.15.151:0/3342175522 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7dc0d0870 0x7fc7dc0d0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:51.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:51.686+0000 7fc7e0a00640 1 -- 172.21.15.151:0/3342175522 >> 172.21.15.151:0/3342175522 conn(0x7fc7dc0d3e60 msgr2=0x7fc7dc0d4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:55:51.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:51.686+0000 7fc7e0a00640 1 -- 172.21.15.151:0/3342175522 shutdown_connections 2024-09-16T15:55:51.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:51.686+0000 7fc7e0a00640 1 -- 172.21.15.151:0/3342175522 wait complete. 2024-09-16T15:55:51.689 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:51.688+0000 7fc7e0a00640 1 Processor -- start 2024-09-16T15:55:51.689 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:51.688+0000 7fc7e0a00640 1 -- start start 2024-09-16T15:55:51.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:51.690+0000 7fc7e0a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7dc0b8a60 0x7fc7dc09ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:51.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:51.690+0000 7fc7e0a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7dc0d0870 0x7fc7dc14de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:51.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:51.690+0000 7fc7e0a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7dc0d1f00 0x7fc7dc14e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:51.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:51.690+0000 7fc7e0a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc7dc0cf190 con 0x7fc7dc0b8a60 2024-09-16T15:55:51.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:51.690+0000 7fc7e0a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc7dc0cf010 con 0x7fc7dc0d1f00 2024-09-16T15:55:51.691 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:51.690+0000 7fc7e0a00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fc7dc0cf310 con 0x7fc7dc0d0870 2024-09-16T15:55:51.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:51.690+0000 7fc7db400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7dc0b8a60 0x7fc7dc09ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:51.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:51.690+0000 7fc7db400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7dc0b8a60 0x7fc7dc09ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:59354/0 (socket says 172.21.15.151:59354) 2024-09-16T15:55:51.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:51.690+0000 7fc7db400640 1 -- 172.21.15.151:0/1301184335 learned_addr learned my addr 172.21.15.151:0/1301184335 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:55:51.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:51.691+0000 7fc7db400640 1 -- 172.21.15.151:0/1301184335 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7dc0d0870 msgr2=0x7fc7dc14de00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T15:55:51.692 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:51.691+0000 7fc7db400640 1 --2- 172.21.15.151:0/1301184335 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7dc0d0870 0x7fc7dc14de00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:51.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:51.691+0000 7fc7db400640 1 -- 172.21.15.151:0/1301184335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7dc0d1f00 msgr2=0x7fc7dc14e340 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T15:55:51.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:51.692+0000 7fc7db400640 1 --2- 172.21.15.151:0/1301184335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7dc0d1f00 0x7fc7dc14e340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:51.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:51.692+0000 7fc7db400640 1 -- 172.21.15.151:0/1301184335 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7dc1f4da0 con 0x7fc7dc0b8a60 2024-09-16T15:55:51.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:51.692+0000 7fc7db400640 1 --2- 172.21.15.151:0/1301184335 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7dc0b8a60 0x7fc7dc09ddc0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc00b4d0 tx=0x7fc7cc00b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:55:51.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:51.692+0000 7fc7d3400640 1 -- 172.21.15.151:0/1301184335 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc014070 con 0x7fc7dc0b8a60 2024-09-16T15:55:51.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:51.692+0000 7fc7e0a00640 1 -- 172.21.15.151:0/1301184335 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7dc1f5060 con 0x7fc7dc0b8a60 2024-09-16T15:55:51.694 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:51.693+0000 7fc7d3400640 1 -- 172.21.15.151:0/1301184335 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc7cc010960 con 0x7fc7dc0b8a60 2024-09-16T15:55:51.694 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:51.693+0000 7fc7d3400640 1 -- 172.21.15.151:0/1301184335 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc010db0 con 0x7fc7dc0b8a60 2024-09-16T15:55:51.694 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:51.693+0000 7fc7e0a00640 1 -- 172.21.15.151:0/1301184335 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7dc1f5620 con 0x7fc7dc0b8a60 2024-09-16T15:55:51.695 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:51.694+0000 7fc7e0a00640 1 -- 172.21.15.151:0/1301184335 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7dc0ce900 con 0x7fc7dc0b8a60 2024-09-16T15:55:51.696 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:51.695+0000 7fc7d3400640 1 -- 172.21.15.151:0/1301184335 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 53) ==== 104755+0+0 (secure 0 0 0) 0x7fc7cc017070 con 0x7fc7dc0b8a60 2024-09-16T15:55:51.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:51.696+0000 7fc7d3400640 1 --2- 172.21.15.151:0/1301184335 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc7b807ae90 0x7fc7b807d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:51.698 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:51.697+0000 7fc7d3400640 1 -- 172.21.15.151:0/1301184335 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5907+0+0 (secure 0 0 0) 0x7fc7cc099aa0 con 0x7fc7dc0b8a60 2024-09-16T15:55:51.703 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:51.702+0000 7fc7d3400640 1 -- 172.21.15.151:0/1301184335 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc7cc063af0 con 0x7fc7dc0b8a60 2024-09-16T15:55:51.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:51.716+0000 7fc7d3e00640 1 --2- 172.21.15.151:0/1301184335 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc7b807ae90 0x7fc7b807d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:51.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:51.810+0000 7fc7d3e00640 1 --2- 172.21.15.151:0/1301184335 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc7b807ae90 0x7fc7b807d350 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc161790 tx=0x7fc7c4006b30 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:55:51.894 DEBUG:teuthology.orchestra.run.smithi151:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:55:51.899 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:55:52.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:52.075+0000 7fc7e0a00640 1 -- 172.21.15.151:0/1301184335 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone58175", "target": ["mon-mgr", ""]}) -- 0x7fc7dc0cfc00 con 0x7fc7b807ae90 2024-09-16T15:55:53.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:53.756+0000 7fc7d3400640 1 -- 172.21.15.151:0/1301184335 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc7dc0cfc00 con 0x7fc7b807ae90 2024-09-16T15:55:53.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:53.763+0000 7fc7e0a00640 1 -- 172.21.15.151:0/1301184335 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc7b807ae90 msgr2=0x7fc7b807d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:53.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:53.763+0000 7fc7e0a00640 1 --2- 172.21.15.151:0/1301184335 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc7b807ae90 0x7fc7b807d350 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc161790 tx=0x7fc7c4006b30 comp rx=0 tx=0).stop 2024-09-16T15:55:53.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:53.763+0000 7fc7e0a00640 1 -- 172.21.15.151:0/1301184335 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7dc0b8a60 msgr2=0x7fc7dc09ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:53.764 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:53.763+0000 7fc7e0a00640 1 --2- 172.21.15.151:0/1301184335 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7dc0b8a60 0x7fc7dc09ddc0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc00b4d0 tx=0x7fc7cc00b990 comp rx=0 tx=0).stop 2024-09-16T15:55:53.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:53.763+0000 7fc7e0a00640 1 -- 172.21.15.151:0/1301184335 shutdown_connections 2024-09-16T15:55:53.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:53.763+0000 7fc7e0a00640 1 --2- 172.21.15.151:0/1301184335 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc7b807ae90 0x7fc7b807d350 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:53.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:53.763+0000 7fc7e0a00640 1 --2- 172.21.15.151:0/1301184335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7dc0d1f00 0x7fc7dc14e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:53.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:53.763+0000 7fc7e0a00640 1 --2- 172.21.15.151:0/1301184335 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7dc0d0870 0x7fc7dc14de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:53.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:53.763+0000 7fc7e0a00640 1 --2- 172.21.15.151:0/1301184335 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7dc0b8a60 0x7fc7dc09ddc0 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:53.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:53.763+0000 7fc7e0a00640 1 -- 172.21.15.151:0/1301184335 >> 172.21.15.151:0/1301184335 conn(0x7fc7dc0d3e60 msgr2=0x7fc7dc15d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:55:53.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:53.764+0000 7fc7e0a00640 1 -- 172.21.15.151:0/1301184335 shutdown_connections 2024-09-16T15:55:53.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:53.765+0000 7fc7e0a00640 1 -- 172.21.15.151:0/1301184335 wait complete. 2024-09-16T15:55:53.805 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone67026 2024-09-16T15:55:54.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:54.111+0000 7fda59400640 1 -- 172.21.15.151:0/235087340 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda38007940 msgr2=0x7fda38009dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:54.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:54.111+0000 7fda59400640 1 --2- 172.21.15.151:0/235087340 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda38007940 0x7fda38009dd0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7fda4000e900 tx=0x7fda38003810 comp rx=0 tx=0).stop 2024-09-16T15:55:54.113 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:54.112+0000 7fda59400640 1 -- 172.21.15.151:0/235087340 shutdown_connections 2024-09-16T15:55:54.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:54.112+0000 7fda59400640 1 --2- 172.21.15.151:0/235087340 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda38007940 0x7fda38009dd0 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:54.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:54.112+0000 7fda59400640 1 -- 172.21.15.151:0/235087340 >> 172.21.15.151:0/235087340 conn(0x7fda540b1840 msgr2=0x7fda540b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:55:54.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:54.112+0000 7fda59400640 1 -- 172.21.15.151:0/235087340 shutdown_connections 2024-09-16T15:55:54.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:54.112+0000 7fda59400640 1 -- 172.21.15.151:0/235087340 wait complete. 2024-09-16T15:55:54.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:54.114+0000 7fda59400640 1 Processor -- start 2024-09-16T15:55:54.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:54.114+0000 7fda59400640 1 -- start start 2024-09-16T15:55:54.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:54.116+0000 7fda59400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fda38007940 0x7fda540ae430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:54.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:54.116+0000 7fda59400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fda540ac010 0x7fda541fc870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:54.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:54.116+0000 7fda59400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda54153de0 0x7fda54154290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:54.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:54.116+0000 7fda59400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fda540b4250 con 0x7fda54153de0 2024-09-16T15:55:54.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:54.116+0000 7fda59400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fda540b40d0 con 0x7fda38007940 2024-09-16T15:55:54.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:54.116+0000 7fda59400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fda540b43d0 con 0x7fda540ac010 2024-09-16T15:55:54.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:54.117+0000 7fda53200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda54153de0 0x7fda54154290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:54.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:54.117+0000 7fda53200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda54153de0 0x7fda54154290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:45308/0 (socket says 172.21.15.151:45308) 2024-09-16T15:55:54.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:54.117+0000 7fda53200640 1 -- 172.21.15.151:0/2627734059 learned_addr learned my addr 172.21.15.151:0/2627734059 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:55:54.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:54.117+0000 7fda53200640 1 -- 172.21.15.151:0/2627734059 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fda540ac010 msgr2=0x7fda541fc870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:54.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:54.117+0000 7fda53200640 1 --2- 172.21.15.151:0/2627734059 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fda540ac010 0x7fda541fc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:54.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:54.117+0000 7fda53200640 1 -- 172.21.15.151:0/2627734059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fda38007940 msgr2=0x7fda540ae430 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T15:55:54.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:54.117+0000 7fda53200640 1 --2- 172.21.15.151:0/2627734059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fda38007940 0x7fda540ae430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:54.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:54.117+0000 7fda53200640 1 -- 172.21.15.151:0/2627734059 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda540ac580 con 0x7fda54153de0 2024-09-16T15:55:54.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:54.118+0000 7fda53200640 1 --2- 172.21.15.151:0/2627734059 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda54153de0 0x7fda54154290 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7fda540b28c0 tx=0x7fda4c00ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:55:54.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:54.118+0000 7fda4aa00640 1 -- 172.21.15.151:0/2627734059 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda4c017070 con 0x7fda54153de0 2024-09-16T15:55:54.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:54.118+0000 7fda4aa00640 1 -- 172.21.15.151:0/2627734059 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fda4c005070 con 0x7fda54153de0 2024-09-16T15:55:54.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:54.118+0000 7fda4aa00640 1 -- 172.21.15.151:0/2627734059 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda4c0054c0 con 0x7fda54153de0 2024-09-16T15:55:54.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:54.118+0000 7fda59400640 1 -- 172.21.15.151:0/2627734059 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda540ac860 con 0x7fda54153de0 2024-09-16T15:55:54.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:54.119+0000 7fda59400640 1 -- 172.21.15.151:0/2627734059 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda540ace20 con 0x7fda54153de0 2024-09-16T15:55:54.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:54.120+0000 7fda59400640 1 -- 172.21.15.151:0/2627734059 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda540ad020 con 0x7fda54153de0 2024-09-16T15:55:54.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:54.122+0000 7fda4aa00640 1 -- 172.21.15.151:0/2627734059 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 53) ==== 104755+0+0 (secure 0 0 0) 0x7fda4c01a070 con 0x7fda54153de0 2024-09-16T15:55:54.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:54.125+0000 7fda4aa00640 1 --2- 172.21.15.151:0/2627734059 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fda3407aec0 0x7fda3407d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:54.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:54.125+0000 7fda4aa00640 1 -- 172.21.15.151:0/2627734059 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5907+0+0 (secure 0 0 0) 0x7fda4c099060 con 0x7fda54153de0 2024-09-16T15:55:54.127 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:54.126+0000 7fda52800640 1 --2- 172.21.15.151:0/2627734059 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fda3407aec0 0x7fda3407d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:54.132 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:54.131+0000 7fda52800640 1 --2- 172.21.15.151:0/2627734059 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fda3407aec0 0x7fda3407d380 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fda540ae200 tx=0x7fda44006c20 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:55:54.133 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:54.132+0000 7fda4aa00640 1 -- 172.21.15.151:0/2627734059 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fda540ad020 con 0x7fda54153de0 2024-09-16T15:55:54.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:54.500+0000 7fda59400640 1 -- 172.21.15.151:0/2627734059 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone67026", "target": ["mon-mgr", ""]}) -- 0x7fda540cd570 con 0x7fda3407aec0 2024-09-16T15:55:57.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:57.074+0000 7fda4aa00640 1 -- 172.21.15.151:0/2627734059 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fda540cd570 con 0x7fda3407aec0 2024-09-16T15:55:57.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:57.077+0000 7fda59400640 1 -- 172.21.15.151:0/2627734059 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fda3407aec0 msgr2=0x7fda3407d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:57.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:57.077+0000 7fda59400640 1 --2- 172.21.15.151:0/2627734059 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fda3407aec0 0x7fda3407d380 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fda540ae200 tx=0x7fda44006c20 comp rx=0 tx=0).stop 2024-09-16T15:55:57.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:57.077+0000 7fda59400640 1 -- 172.21.15.151:0/2627734059 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda54153de0 msgr2=0x7fda54154290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:57.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:57.077+0000 7fda59400640 1 --2- 172.21.15.151:0/2627734059 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda54153de0 0x7fda54154290 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7fda540b28c0 tx=0x7fda4c00ccd0 comp rx=0 tx=0).stop 2024-09-16T15:55:57.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:57.077+0000 7fda59400640 1 -- 172.21.15.151:0/2627734059 shutdown_connections 2024-09-16T15:55:57.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:57.077+0000 7fda59400640 1 --2- 172.21.15.151:0/2627734059 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fda3407aec0 0x7fda3407d380 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:57.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:57.077+0000 7fda59400640 1 --2- 172.21.15.151:0/2627734059 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda54153de0 0x7fda54154290 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:57.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:57.077+0000 7fda59400640 1 --2- 172.21.15.151:0/2627734059 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fda540ac010 0x7fda541fc870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:57.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:57.077+0000 7fda59400640 1 --2- 172.21.15.151:0/2627734059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fda38007940 0x7fda540ae430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:57.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:57.077+0000 7fda59400640 1 -- 172.21.15.151:0/2627734059 >> 172.21.15.151:0/2627734059 conn(0x7fda540b1840 msgr2=0x7fda5406fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:55:57.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:57.078+0000 7fda59400640 1 -- 172.21.15.151:0/2627734059 shutdown_connections 2024-09-16T15:55:57.080 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:57.078+0000 7fda59400640 1 -- 172.21.15.151:0/2627734059 wait complete. 2024-09-16T15:55:58.111 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone62040 2024-09-16T15:55:58.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:58.424+0000 7f276fa00640 1 -- 172.21.15.151:0/2088150574 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f274c004db0 msgr2=0x7f274c005230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:58.426 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:58.424+0000 7f276fa00640 1 --2- 172.21.15.151:0/2088150574 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f274c004db0 0x7f274c005230 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f276000c640 tx=0x7f274c005830 comp rx=0 tx=0).stop 2024-09-16T15:55:58.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:58.424+0000 7f276da00640 1 -- 172.21.15.151:0/2088150574 reap_dead start 2024-09-16T15:55:58.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:58.425+0000 7f276fa00640 1 -- 172.21.15.151:0/2088150574 shutdown_connections 2024-09-16T15:55:58.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:58.425+0000 7f276fa00640 1 -- 172.21.15.151:0/2088150574 >> 172.21.15.151:0/2088150574 conn(0x7f27680b1840 msgr2=0x7f27680b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:55:58.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:58.425+0000 7f276fa00640 1 -- 172.21.15.151:0/2088150574 shutdown_connections 2024-09-16T15:55:58.427 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:58.425+0000 7f276fa00640 1 -- 172.21.15.151:0/2088150574 wait complete. 2024-09-16T15:55:58.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:58.428+0000 7f276fa00640 1 Processor -- start 2024-09-16T15:55:58.429 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:58.428+0000 7f276fa00640 1 -- start start 2024-09-16T15:55:58.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:58.430+0000 7f276fa00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f274c004db0 0x7f27681f56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:58.431 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:58.430+0000 7f276fa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f276815d0d0 0x7f276815d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:58.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:58.430+0000 7f276fa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27681581e0 0x7f2768158640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:58.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:58.430+0000 7f276fa00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2768161bb0 con 0x7f276815d0d0 2024-09-16T15:55:58.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:58.430+0000 7f276fa00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f27680701c0 con 0x7f27681581e0 2024-09-16T15:55:58.432 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:58.430+0000 7f276fa00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f2768161d30 con 0x7f274c004db0 2024-09-16T15:55:58.434 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:58.431+0000 7f2767e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f276815d0d0 0x7f276815d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:58.434 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:58.431+0000 7f2767e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f276815d0d0 0x7f276815d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:45328/0 (socket says 172.21.15.151:45328) 2024-09-16T15:55:58.434 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:58.431+0000 7f2767e00640 1 -- 172.21.15.151:0/1509595870 learned_addr learned my addr 172.21.15.151:0/1509595870 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:55:58.434 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:58.431+0000 7f276d000640 1 --2- 172.21.15.151:0/1509595870 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f274c004db0 0x7f27681f56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:58.434 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:58.431+0000 7f2767e00640 1 -- 172.21.15.151:0/1509595870 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f274c004db0 msgr2=0x7f27681f56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:58.434 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:58.432+0000 7f2767e00640 1 --2- 172.21.15.151:0/1509595870 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f274c004db0 0x7f27681f56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:58.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:58.432+0000 7f2767e00640 1 -- 172.21.15.151:0/1509595870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27681581e0 msgr2=0x7f2768158640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:55:58.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:58.432+0000 7f2767e00640 1 --2- 172.21.15.151:0/1509595870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27681581e0 0x7f2768158640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:55:58.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:58.432+0000 7f2767e00640 1 -- 172.21.15.151:0/1509595870 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27681f5c10 con 0x7f276815d0d0 2024-09-16T15:55:58.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:58.432+0000 7f2767e00640 1 --2- 172.21.15.151:0/1509595870 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f276815d0d0 0x7f276815d960 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f275400e9c0 tx=0x7f275400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:55:58.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:58.432+0000 7f2764c00640 1 -- 172.21.15.151:0/1509595870 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f275400ad40 con 0x7f276815d0d0 2024-09-16T15:55:58.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:58.432+0000 7f2764c00640 1 -- 172.21.15.151:0/1509595870 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2754004020 con 0x7f276815d0d0 2024-09-16T15:55:58.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:58.432+0000 7f2764c00640 1 -- 172.21.15.151:0/1509595870 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2754004450 con 0x7f276815d0d0 2024-09-16T15:55:58.435 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:58.432+0000 7f276fa00640 1 -- 172.21.15.151:0/1509595870 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f276815b450 con 0x7f276815d0d0 2024-09-16T15:55:58.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:58.433+0000 7f276fa00640 1 -- 172.21.15.151:0/1509595870 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f276815ba10 con 0x7f276815d0d0 2024-09-16T15:55:58.436 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:58.435+0000 7f276fa00640 1 -- 172.21.15.151:0/1509595870 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27680b3fa0 con 0x7f276815d0d0 2024-09-16T15:55:58.439 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:58.437+0000 7f2764c00640 1 -- 172.21.15.151:0/1509595870 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 53) ==== 104755+0+0 (secure 0 0 0) 0x7f2754005020 con 0x7f276815d0d0 2024-09-16T15:55:58.439 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:58.438+0000 7f2764c00640 1 --2- 172.21.15.151:0/1509595870 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f274807ae90 0x7f274807d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:55:58.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:58.439+0000 7f2764c00640 1 -- 172.21.15.151:0/1509595870 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5907+0+0 (secure 0 0 0) 0x7f275409a6a0 con 0x7f276815d0d0 2024-09-16T15:55:58.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:58.445+0000 7f2764c00640 1 -- 172.21.15.151:0/1509595870 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f27540641a0 con 0x7f276815d0d0 2024-09-16T15:55:58.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:58.454+0000 7f276d000640 1 --2- 172.21.15.151:0/1509595870 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f274807ae90 0x7f274807d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:55:58.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:58.457+0000 7f276d000640 1 --2- 172.21.15.151:0/1509595870 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f274807ae90 0x7f274807d350 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f2758003640 tx=0x7f275800b260 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:55:58.854 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:55:58.850+0000 7f276fa00640 1 -- 172.21.15.151:0/1509595870 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone62040", "target": ["mon-mgr", ""]}) -- 0x7f276806ad20 con 0x7f274807ae90 2024-09-16T15:55:59.215 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:55:59.213+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:55:59.218 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:55:59.213+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:55:59.218 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:55:59.213+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:55:59.218 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:55:59.213+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:55:59.218 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:55:59.213+0000 7f48efe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:56:00.703 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:00.702+0000 7f2764c00640 1 -- 172.21.15.151:0/1509595870 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mgrmap(e 54) ==== 104806+0+0 (secure 0 0 0) 0x7f2754060410 con 0x7f276815d0d0 2024-09-16T15:56:01.620 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:01.619+0000 7f2764c00640 1 -- 172.21.15.151:0/1509595870 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f276806ad20 con 0x7f274807ae90 2024-09-16T15:56:01.624 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:01.622+0000 7f276fa00640 1 -- 172.21.15.151:0/1509595870 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f274807ae90 msgr2=0x7f274807d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:01.624 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:01.622+0000 7f276fa00640 1 --2- 172.21.15.151:0/1509595870 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f274807ae90 0x7f274807d350 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f2758003640 tx=0x7f275800b260 comp rx=0 tx=0).stop 2024-09-16T15:56:01.624 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:01.622+0000 7f276fa00640 1 -- 172.21.15.151:0/1509595870 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f276815d0d0 msgr2=0x7f276815d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:01.624 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:01.622+0000 7f276fa00640 1 --2- 172.21.15.151:0/1509595870 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f276815d0d0 0x7f276815d960 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f275400e9c0 tx=0x7f275400ee80 comp rx=0 tx=0).stop 2024-09-16T15:56:01.624 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:01.622+0000 7f276fa00640 1 -- 172.21.15.151:0/1509595870 shutdown_connections 2024-09-16T15:56:01.624 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:01.622+0000 7f276fa00640 1 --2- 172.21.15.151:0/1509595870 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f274807ae90 0x7f274807d350 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:01.624 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:01.622+0000 7f276fa00640 1 --2- 172.21.15.151:0/1509595870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f27681581e0 0x7f2768158640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:01.624 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:01.622+0000 7f276fa00640 1 --2- 172.21.15.151:0/1509595870 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f276815d0d0 0x7f276815d960 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:01.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:01.622+0000 7f276fa00640 1 --2- 172.21.15.151:0/1509595870 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f274c004db0 0x7f27681f56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:01.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:01.622+0000 7f276fa00640 1 -- 172.21.15.151:0/1509595870 >> 172.21.15.151:0/1509595870 conn(0x7f27680b1840 msgr2=0x7f276815a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:01.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:01.622+0000 7f276fa00640 1 -- 172.21.15.151:0/1509595870 shutdown_connections 2024-09-16T15:56:01.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:01.623+0000 7f276fa00640 1 -- 172.21.15.151:0/1509595870 wait complete. 2024-09-16T15:56:01.642 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone21631 2024-09-16T15:56:01.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:01.888+0000 7f1c76a00640 1 -- 172.21.15.151:0/3848999026 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c58004800 msgr2=0x7f1c58004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:01.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:01.888+0000 7f1c76a00640 1 --2- 172.21.15.151:0/3848999026 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c58004800 0x7f1c58004c80 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c00c6e0 tx=0x7f1c58005830 comp rx=0 tx=0).stop 2024-09-16T15:56:01.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:01.889+0000 7f1c76a00640 1 -- 172.21.15.151:0/3848999026 shutdown_connections 2024-09-16T15:56:01.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:01.889+0000 7f1c76a00640 1 --2- 172.21.15.151:0/3848999026 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c58004800 0x7f1c58004c80 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:01.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:01.889+0000 7f1c76a00640 1 -- 172.21.15.151:0/3848999026 >> 172.21.15.151:0/3848999026 conn(0x7f1c7009d0b0 msgr2=0x7f1c700d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:01.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:01.889+0000 7f1c76a00640 1 -- 172.21.15.151:0/3848999026 shutdown_connections 2024-09-16T15:56:01.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:01.889+0000 7f1c76a00640 1 -- 172.21.15.151:0/3848999026 wait complete. 2024-09-16T15:56:01.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:01.891+0000 7f1c76a00640 1 Processor -- start 2024-09-16T15:56:01.892 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:01.891+0000 7f1c76a00640 1 -- start start 2024-09-16T15:56:01.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:01.892+0000 7f1c76a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1c58004800 0x7f1c700b98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:01.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:01.892+0000 7f1c76a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c701f4f80 0x7f1c701562b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:01.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:01.892+0000 7f1c76a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c700b1ad0 0x7f1c700c8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:01.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:01.893+0000 7f1c76a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1c700d22f0 con 0x7f1c701f4f80 2024-09-16T15:56:01.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:01.893+0000 7f1c76a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1c700b51b0 con 0x7f1c700b1ad0 2024-09-16T15:56:01.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:01.893+0000 7f1c76a00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f1c700d2470 con 0x7f1c58004800 2024-09-16T15:56:01.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:01.893+0000 7f1c74c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c701f4f80 0x7f1c701562b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:01.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:01.893+0000 7f1c74c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c701f4f80 0x7f1c701562b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:45350/0 (socket says 172.21.15.151:45350) 2024-09-16T15:56:01.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:01.893+0000 7f1c74c00640 1 -- 172.21.15.151:0/1889488881 learned_addr learned my addr 172.21.15.151:0/1889488881 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:56:01.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:01.894+0000 7f1c74c00640 1 -- 172.21.15.151:0/1889488881 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1c58004800 msgr2=0x7f1c700b98b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T15:56:01.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:01.894+0000 7f1c74c00640 1 --2- 172.21.15.151:0/1889488881 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f1c58004800 0x7f1c700b98b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:01.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:01.894+0000 7f1c74c00640 1 -- 172.21.15.151:0/1889488881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c700b1ad0 msgr2=0x7f1c700c8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:01.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:01.894+0000 7f1c74c00640 1 --2- 172.21.15.151:0/1889488881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c700b1ad0 0x7f1c700c8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:01.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:01.894+0000 7f1c74c00640 1 -- 172.21.15.151:0/1889488881 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c700cc190 con 0x7f1c701f4f80 2024-09-16T15:56:01.895 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:01.894+0000 7f1c74c00640 1 --2- 172.21.15.151:0/1889488881 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c701f4f80 0x7f1c701562b0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c00c800 tx=0x7f1c5c006380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:01.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:01.894+0000 7f1c6d600640 1 -- 172.21.15.151:0/1889488881 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c004830 con 0x7f1c701f4f80 2024-09-16T15:56:01.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:01.894+0000 7f1c6d600640 1 -- 172.21.15.151:0/1889488881 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1c5c039070 con 0x7f1c701f4f80 2024-09-16T15:56:01.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:01.895+0000 7f1c76a00640 1 -- 172.21.15.151:0/1889488881 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c701f5400 con 0x7f1c701f4f80 2024-09-16T15:56:01.896 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:01.895+0000 7f1c6d600640 1 -- 172.21.15.151:0/1889488881 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c004d80 con 0x7f1c701f4f80 2024-09-16T15:56:01.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:01.895+0000 7f1c76a00640 1 -- 172.21.15.151:0/1889488881 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c700c79f0 con 0x7f1c701f4f80 2024-09-16T15:56:01.897 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:01.896+0000 7f1c76a00640 1 -- 172.21.15.151:0/1889488881 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c700d29d0 con 0x7f1c701f4f80 2024-09-16T15:56:01.898 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:01.898+0000 7f1c6d600640 1 -- 172.21.15.151:0/1889488881 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 54) ==== 104806+0+0 (secure 0 0 0) 0x7f1c5c043070 con 0x7f1c701f4f80 2024-09-16T15:56:01.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:01.899+0000 7f1c6d600640 1 --2- 172.21.15.151:0/1889488881 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1c5007af70 0x7f1c5007d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:01.900 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:01.899+0000 7f1c6d600640 1 -- 172.21.15.151:0/1889488881 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5907+0+0 (secure 0 0 0) 0x7f1c5c0b93f0 con 0x7f1c701f4f80 2024-09-16T15:56:01.906 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:01.905+0000 7f1c6d600640 1 -- 172.21.15.151:0/1889488881 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f1c5c0835c0 con 0x7f1c701f4f80 2024-09-16T15:56:01.912 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:01.911+0000 7f1c75600640 1 --2- 172.21.15.151:0/1889488881 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1c5007af70 0x7f1c5007d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:01.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:01.912+0000 7f1c75600640 1 --2- 172.21.15.151:0/1889488881 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f1c5007af70 0x7f1c5007d430 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f1c64009710 tx=0x7f1c64006d00 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:02.294 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:02.293+0000 7f1c76a00640 1 -- 172.21.15.151:0/1889488881 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone21631", "target": ["mon-mgr", ""]}) -- 0x7f1c701567f0 con 0x7f1c5007af70 2024-09-16T15:56:02.360 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:56:02.358+0000 7f48efe00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-16T15:56:02.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:02.360+0000 7f1c6d600640 1 -- 172.21.15.151:0/1889488881 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f1c701567f0 con 0x7f1c5007af70 2024-09-16T15:56:02.361 INFO:teuthology.orchestra.run.smithi151.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-16T15:56:02.378 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T15:56:02.379 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone54262 2024-09-16T15:56:02.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:02.676+0000 7f70d2800640 1 -- 172.21.15.151:0/1938524808 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f70b4004d40 msgr2=0x7f70b40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:02.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:02.676+0000 7f70d2800640 1 --2- 172.21.15.151:0/1938524808 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f70b4004d40 0x7f70b40051c0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f70cc08d200 tx=0x7f70b4005830 comp rx=0 tx=0).stop 2024-09-16T15:56:02.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:02.676+0000 7f70d1e00640 1 -- 172.21.15.151:0/1938524808 reap_dead start 2024-09-16T15:56:02.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:02.676+0000 7f70d2800640 1 -- 172.21.15.151:0/1938524808 shutdown_connections 2024-09-16T15:56:02.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:02.676+0000 7f70d2800640 1 -- 172.21.15.151:0/1938524808 >> 172.21.15.151:0/1938524808 conn(0x7f70cc09d0b0 msgr2=0x7f70cc0d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:02.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:02.676+0000 7f70d2800640 1 -- 172.21.15.151:0/1938524808 shutdown_connections 2024-09-16T15:56:02.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:02.676+0000 7f70d2800640 1 -- 172.21.15.151:0/1938524808 wait complete. 2024-09-16T15:56:02.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:02.679+0000 7f70d2800640 1 Processor -- start 2024-09-16T15:56:02.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:02.679+0000 7f70d2800640 1 -- start start 2024-09-16T15:56:02.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:02.681+0000 7f70d2800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70cc09ead0 0x7f70cc0cb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:02.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:02.681+0000 7f70d2800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f70cc0cc060 0x7f70cc1f4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:02.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:02.681+0000 7f70d2800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f70cc0b17a0 0x7f70cc0b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:02.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:02.681+0000 7f70d2800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f70cc0d2830 con 0x7f70cc0b17a0 2024-09-16T15:56:02.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:02.681+0000 7f70d2800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f70cc0b5230 con 0x7f70cc09ead0 2024-09-16T15:56:02.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:02.681+0000 7f70d2800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f70cc0d29b0 con 0x7f70cc0cc060 2024-09-16T15:56:02.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:02.681+0000 7f70d1e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f70cc0b17a0 0x7f70cc0b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:02.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:02.682+0000 7f70d1e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f70cc0b17a0 0x7f70cc0b1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:45378/0 (socket says 172.21.15.151:45378) 2024-09-16T15:56:02.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:02.682+0000 7f70d1e00640 1 -- 172.21.15.151:0/1188482771 learned_addr learned my addr 172.21.15.151:0/1188482771 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:56:02.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:02.682+0000 7f70d1e00640 1 -- 172.21.15.151:0/1188482771 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f70cc0cc060 msgr2=0x7f70cc1f4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:02.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:02.682+0000 7f70d1e00640 1 --2- 172.21.15.151:0/1188482771 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f70cc0cc060 0x7f70cc1f4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:02.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:02.682+0000 7f70d1e00640 1 -- 172.21.15.151:0/1188482771 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70cc09ead0 msgr2=0x7f70cc0cb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:02.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:02.682+0000 7f70d1e00640 1 --2- 172.21.15.151:0/1188482771 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70cc09ead0 0x7f70cc0cb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:02.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:02.682+0000 7f70d1e00640 1 -- 172.21.15.151:0/1188482771 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70cc1f50d0 con 0x7f70cc0b17a0 2024-09-16T15:56:02.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:02.683+0000 7f70d1e00640 1 --2- 172.21.15.151:0/1188482771 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f70cc0b17a0 0x7f70cc0b1c20 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f70c4002230 tx=0x7f70c40047f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:02.685 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:02.683+0000 7f70c9600640 1 -- 172.21.15.151:0/1188482771 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c4041020 con 0x7f70cc0b17a0 2024-09-16T15:56:02.685 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:02.683+0000 7f70d2800640 1 -- 172.21.15.151:0/1188482771 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70cc1f53b0 con 0x7f70cc0b17a0 2024-09-16T15:56:02.685 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:02.683+0000 7f70c9600640 1 -- 172.21.15.151:0/1188482771 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f70c402b5e0 con 0x7f70cc0b17a0 2024-09-16T15:56:02.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:02.683+0000 7f70c9600640 1 -- 172.21.15.151:0/1188482771 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c4009a00 con 0x7f70cc0b17a0 2024-09-16T15:56:02.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:02.684+0000 7f70d2800640 1 -- 172.21.15.151:0/1188482771 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70cc0cb2d0 con 0x7f70cc0b17a0 2024-09-16T15:56:02.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:02.686+0000 7f70d2800640 1 -- 172.21.15.151:0/1188482771 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70a8005290 con 0x7f70cc0b17a0 2024-09-16T15:56:02.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:02.689+0000 7f70c9600640 1 -- 172.21.15.151:0/1188482771 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 54) ==== 104806+0+0 (secure 0 0 0) 0x7f70c4044020 con 0x7f70cc0b17a0 2024-09-16T15:56:02.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:02.691+0000 7f70c9600640 1 --2- 172.21.15.151:0/1188482771 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f70ac07af20 0x7f70ac07d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:02.693 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:02.691+0000 7f70c9600640 1 -- 172.21.15.151:0/1188482771 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5907+0+0 (secure 0 0 0) 0x7f70c40ba9a0 con 0x7f70cc0b17a0 2024-09-16T15:56:02.701 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:02.700+0000 7f70c9600640 1 -- 172.21.15.151:0/1188482771 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f70c404b020 con 0x7f70cc0b17a0 2024-09-16T15:56:02.714 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:02.713+0000 7f70d1400640 1 --2- 172.21.15.151:0/1188482771 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f70ac07af20 0x7f70ac07d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:02.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:02.723+0000 7f70d1400640 1 --2- 172.21.15.151:0/1188482771 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f70ac07af20 0x7f70ac07d3e0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f70c00058c0 tx=0x7f70c0005850 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:03.150 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:03.147+0000 7f70d2800640 1 -- 172.21.15.151:0/1188482771 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone54262", "target": ["mon-mgr", ""]}) -- 0x7f70a8000bc0 con 0x7f70ac07af20 2024-09-16T15:56:05.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:05.019+0000 7f70c9600640 1 -- 172.21.15.151:0/1188482771 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f70a8000bc0 con 0x7f70ac07af20 2024-09-16T15:56:05.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:05.023+0000 7f70d2800640 1 -- 172.21.15.151:0/1188482771 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f70ac07af20 msgr2=0x7f70ac07d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:05.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:05.023+0000 7f70d2800640 1 --2- 172.21.15.151:0/1188482771 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f70ac07af20 0x7f70ac07d3e0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f70c00058c0 tx=0x7f70c0005850 comp rx=0 tx=0).stop 2024-09-16T15:56:05.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:05.023+0000 7f70d2800640 1 -- 172.21.15.151:0/1188482771 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f70cc0b17a0 msgr2=0x7f70cc0b1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:05.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:05.023+0000 7f70d2800640 1 --2- 172.21.15.151:0/1188482771 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f70cc0b17a0 0x7f70cc0b1c20 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f70c4002230 tx=0x7f70c40047f0 comp rx=0 tx=0).stop 2024-09-16T15:56:05.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:05.024+0000 7f70d2800640 1 -- 172.21.15.151:0/1188482771 shutdown_connections 2024-09-16T15:56:05.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:05.024+0000 7f70d2800640 1 --2- 172.21.15.151:0/1188482771 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f70ac07af20 0x7f70ac07d3e0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:05.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:05.024+0000 7f70d2800640 1 --2- 172.21.15.151:0/1188482771 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f70cc0b17a0 0x7f70cc0b1c20 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:05.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:05.024+0000 7f70d2800640 1 --2- 172.21.15.151:0/1188482771 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f70cc0cc060 0x7f70cc1f4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:05.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:05.024+0000 7f70d2800640 1 --2- 172.21.15.151:0/1188482771 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f70cc09ead0 0x7f70cc0cb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:05.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:05.024+0000 7f70d2800640 1 -- 172.21.15.151:0/1188482771 >> 172.21.15.151:0/1188482771 conn(0x7f70cc09d0b0 msgr2=0x7f70cc0b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:05.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:05.024+0000 7f70d2800640 1 -- 172.21.15.151:0/1188482771 shutdown_connections 2024-09-16T15:56:05.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:05.024+0000 7f70d2800640 1 -- 172.21.15.151:0/1188482771 wait complete. 2024-09-16T15:56:05.056 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone83343 2024-09-16T15:56:05.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:05.334+0000 7f3c4d600640 1 -- 172.21.15.151:0/665255269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c48161460 msgr2=0x7f3c480d22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:05.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:05.334+0000 7f3c4d600640 1 --2- 172.21.15.151:0/665255269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c48161460 0x7f3c480d22f0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f3c38009830 tx=0x7f3c3802a480 comp rx=0 tx=0).stop 2024-09-16T15:56:05.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:05.334+0000 7f3c4d600640 1 -- 172.21.15.151:0/665255269 shutdown_connections 2024-09-16T15:56:05.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:05.334+0000 7f3c4d600640 1 --2- 172.21.15.151:0/665255269 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c480b5480 0x7f3c480d2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:05.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:05.334+0000 7f3c4d600640 1 --2- 172.21.15.151:0/665255269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c48161460 0x7f3c480d22f0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:05.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:05.334+0000 7f3c4d600640 1 -- 172.21.15.151:0/665255269 >> 172.21.15.151:0/665255269 conn(0x7f3c4809d0b0 msgr2=0x7f3c480d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:05.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:05.334+0000 7f3c4d600640 1 -- 172.21.15.151:0/665255269 shutdown_connections 2024-09-16T15:56:05.336 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:05.334+0000 7f3c4d600640 1 -- 172.21.15.151:0/665255269 wait complete. 2024-09-16T15:56:05.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:05.337+0000 7f3c4d600640 1 Processor -- start 2024-09-16T15:56:05.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:05.340+0000 7f3c4d600640 1 -- start start 2024-09-16T15:56:05.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:05.341+0000 7f3c4d600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c480b4f40 0x7f3c481fbd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:05.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:05.341+0000 7f3c4d600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c480b5480 0x7f3c481f9f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:05.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:05.341+0000 7f3c4d600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3c48161460 0x7f3c480caec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:05.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:05.341+0000 7f3c4d600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3c480cf0b0 con 0x7f3c48161460 2024-09-16T15:56:05.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:05.341+0000 7f3c4d600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3c480cef30 con 0x7f3c480b5480 2024-09-16T15:56:05.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:05.341+0000 7f3c4d600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f3c480cf230 con 0x7f3c480b4f40 2024-09-16T15:56:05.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:05.342+0000 7f3c4cc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3c48161460 0x7f3c480caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:05.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:05.342+0000 7f3c4cc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3c48161460 0x7f3c480caec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33760/0 (socket says 172.21.15.151:33760) 2024-09-16T15:56:05.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:05.342+0000 7f3c4cc00640 1 -- 172.21.15.151:0/2928046404 learned_addr learned my addr 172.21.15.151:0/2928046404 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:56:05.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:05.343+0000 7f3c3f400640 1 --2- 172.21.15.151:0/2928046404 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c480b5480 0x7f3c481f9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:05.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:05.343+0000 7f3c47e00640 1 --2- 172.21.15.151:0/2928046404 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c480b4f40 0x7f3c481fbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:05.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:05.344+0000 7f3c4cc00640 1 -- 172.21.15.151:0/2928046404 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c480b4f40 msgr2=0x7f3c481fbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:05.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:05.344+0000 7f3c4cc00640 1 --2- 172.21.15.151:0/2928046404 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3c480b4f40 0x7f3c481fbd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:05.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:05.344+0000 7f3c4cc00640 1 -- 172.21.15.151:0/2928046404 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c480b5480 msgr2=0x7f3c481f9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:05.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:05.344+0000 7f3c4cc00640 1 --2- 172.21.15.151:0/2928046404 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3c480b5480 0x7f3c481f9f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:05.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:05.344+0000 7f3c4cc00640 1 -- 172.21.15.151:0/2928046404 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c481f9a20 con 0x7f3c48161460 2024-09-16T15:56:05.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:05.345+0000 7f3c4cc00640 1 --2- 172.21.15.151:0/2928046404 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3c48161460 0x7f3c480caec0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f3c4000cb30 tx=0x7f3c4000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:05.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:05.345+0000 7f3c44c00640 1 -- 172.21.15.151:0/2928046404 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c40016020 con 0x7f3c48161460 2024-09-16T15:56:05.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:05.345+0000 7f3c44c00640 1 -- 172.21.15.151:0/2928046404 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3c4000ad90 con 0x7f3c48161460 2024-09-16T15:56:05.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:05.345+0000 7f3c44c00640 1 -- 172.21.15.151:0/2928046404 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c40012680 con 0x7f3c48161460 2024-09-16T15:56:05.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:05.345+0000 7f3c4d600640 1 -- 172.21.15.151:0/2928046404 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c481f9d00 con 0x7f3c48161460 2024-09-16T15:56:05.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:05.346+0000 7f3c4d600640 1 -- 172.21.15.151:0/2928046404 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c480b2550 con 0x7f3c48161460 2024-09-16T15:56:05.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:05.347+0000 7f3c4d600640 1 -- 172.21.15.151:0/2928046404 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c480d1d10 con 0x7f3c48161460 2024-09-16T15:56:05.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:05.349+0000 7f3c44c00640 1 -- 172.21.15.151:0/2928046404 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 54) ==== 104806+0+0 (secure 0 0 0) 0x7f3c40019020 con 0x7f3c48161460 2024-09-16T15:56:05.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:05.350+0000 7f3c44c00640 1 --2- 172.21.15.151:0/2928046404 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3c2407af20 0x7f3c2407d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:05.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:05.350+0000 7f3c44c00640 1 -- 172.21.15.151:0/2928046404 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5907+0+0 (secure 0 0 0) 0x7f3c4009a210 con 0x7f3c48161460 2024-09-16T15:56:05.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:05.354+0000 7f3c47e00640 1 --2- 172.21.15.151:0/2928046404 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3c2407af20 0x7f3c2407d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:05.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:05.357+0000 7f3c44c00640 1 -- 172.21.15.151:0/2928046404 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f3c480d1d10 con 0x7f3c48161460 2024-09-16T15:56:05.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:05.379+0000 7f3c47e00640 1 --2- 172.21.15.151:0/2928046404 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3c2407af20 0x7f3c2407d3e0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f3c38034000 tx=0x7f3c38004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:05.750 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:05.748+0000 7f3c4d600640 1 -- 172.21.15.151:0/2928046404 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone83343", "target": ["mon-mgr", ""]}) -- 0x7f3c48069d80 con 0x7f3c2407af20 2024-09-16T15:56:06.555 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:56:06.553+0000 7f48efe00640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-09-16T15:56:06.574 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:06.573+0000 7f3c44c00640 1 -- 172.21.15.151:0/2928046404 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) ==== 60+0+0 (secure 0 0 0) 0x7f3c380042a0 con 0x7f3c2407af20 2024-09-16T15:56:06.576 INFO:teuthology.orchestra.run.smithi151.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-09-16T15:56:06.591 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T15:56:06.591 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone48134 2024-09-16T15:56:06.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:06.912+0000 7f5187e00640 1 -- 172.21.15.151:0/3363164178 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f51880b43c0 msgr2=0x7f51880d2830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:06.913 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:06.912+0000 7f5187e00640 1 --2- 172.21.15.151:0/3363164178 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f51880b43c0 0x7f51880d2830 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f518806cdc0 tx=0x7f518002aa30 comp rx=0 tx=0).stop 2024-09-16T15:56:06.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:06.913+0000 7f5187e00640 1 -- 172.21.15.151:0/3363164178 shutdown_connections 2024-09-16T15:56:06.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:06.913+0000 7f5187e00640 1 --2- 172.21.15.151:0/3363164178 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f51880b43c0 0x7f51880d2830 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:06.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:06.913+0000 7f5187e00640 1 --2- 172.21.15.151:0/3363164178 >> v2:172.21.15.167:3300/0 conn(0x7f5188161460 0x7f51880d22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:06.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:06.913+0000 7f5187e00640 1 -- 172.21.15.151:0/3363164178 >> 172.21.15.151:0/3363164178 conn(0x7f518809d0b0 msgr2=0x7f51880d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:06.914 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:06.913+0000 7f5187e00640 1 -- 172.21.15.151:0/3363164178 shutdown_connections 2024-09-16T15:56:06.915 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:06.913+0000 7f5187e00640 1 -- 172.21.15.151:0/3363164178 wait complete. 2024-09-16T15:56:06.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:06.916+0000 7f5187e00640 1 Processor -- start 2024-09-16T15:56:06.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:06.916+0000 7f5187e00640 1 -- start start 2024-09-16T15:56:06.918 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:06.917+0000 7f5187e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f51880b43c0 0x7f51880cc720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:06.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:06.917+0000 7f5187e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51880b5480 0x7f51880caec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:06.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:06.917+0000 7f5187e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5188161460 0x7f51880cb400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:06.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:06.917+0000 7f5187e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f51880cef30 con 0x7f51880b43c0 2024-09-16T15:56:06.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:06.917+0000 7f5187e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f51880b5150 con 0x7f5188161460 2024-09-16T15:56:06.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:06.917+0000 7f5187e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f51880cf0b0 con 0x7f51880b5480 2024-09-16T15:56:06.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:06.918+0000 7f5186a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f51880b43c0 0x7f51880cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:06.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:06.918+0000 7f5186a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f51880b43c0 0x7f51880cc720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33774/0 (socket says 172.21.15.151:33774) 2024-09-16T15:56:06.919 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:06.918+0000 7f5186a00640 1 -- 172.21.15.151:0/4248590781 learned_addr learned my addr 172.21.15.151:0/4248590781 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:56:06.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:06.918+0000 7f5186000640 1 --2- 172.21.15.151:0/4248590781 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51880b5480 0x7f51880caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:06.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:06.918+0000 7f5186a00640 1 -- 172.21.15.151:0/4248590781 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51880b5480 msgr2=0x7f51880caec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:06.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:06.918+0000 7f5186a00640 1 --2- 172.21.15.151:0/4248590781 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51880b5480 0x7f51880caec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:06.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:06.918+0000 7f5186a00640 1 -- 172.21.15.151:0/4248590781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5188161460 msgr2=0x7f51880cb400 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T15:56:06.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:06.918+0000 7f5186a00640 1 --2- 172.21.15.151:0/4248590781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5188161460 0x7f51880cb400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:06.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:06.918+0000 7f5186a00640 1 -- 172.21.15.151:0/4248590781 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51880a1760 con 0x7f51880b43c0 2024-09-16T15:56:06.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:06.918+0000 7f5186000640 1 --2- 172.21.15.151:0/4248590781 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51880b5480 0x7f51880caec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:56:06.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:06.919+0000 7f5186a00640 1 --2- 172.21.15.151:0/4248590781 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f51880b43c0 0x7f51880cc720 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f517800e7c0 tx=0x7f517800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:06.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:06.919+0000 7f517ea00640 1 -- 172.21.15.151:0/4248590781 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517800ccc0 con 0x7f51880b43c0 2024-09-16T15:56:06.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:06.919+0000 7f5187e00640 1 -- 172.21.15.151:0/4248590781 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51880cd270 con 0x7f51880b43c0 2024-09-16T15:56:06.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:06.919+0000 7f517ea00640 1 -- 172.21.15.151:0/4248590781 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f517800ce80 con 0x7f51880b43c0 2024-09-16T15:56:06.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:06.919+0000 7f517ea00640 1 -- 172.21.15.151:0/4248590781 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5178005280 con 0x7f51880b43c0 2024-09-16T15:56:06.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:06.919+0000 7f5187e00640 1 -- 172.21.15.151:0/4248590781 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51880ccc60 con 0x7f51880b43c0 2024-09-16T15:56:06.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:06.921+0000 7f5187e00640 1 -- 172.21.15.151:0/4248590781 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51880cf590 con 0x7f51880b43c0 2024-09-16T15:56:06.923 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:06.922+0000 7f517ea00640 1 -- 172.21.15.151:0/4248590781 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 54) ==== 104806+0+0 (secure 0 0 0) 0x7f5178005420 con 0x7f51880b43c0 2024-09-16T15:56:06.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:06.923+0000 7f517ea00640 1 --2- 172.21.15.151:0/4248590781 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f516807af50 0x7f516807d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:06.924 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:06.923+0000 7f517ea00640 1 -- 172.21.15.151:0/4248590781 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5907+0+0 (secure 0 0 0) 0x7f517809a580 con 0x7f51880b43c0 2024-09-16T15:56:06.930 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:06.929+0000 7f517ea00640 1 -- 172.21.15.151:0/4248590781 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f51780646f0 con 0x7f51880b43c0 2024-09-16T15:56:06.932 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:06.931+0000 7f5186000640 1 --2- 172.21.15.151:0/4248590781 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f516807af50 0x7f516807d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:06.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:06.936+0000 7f5186000640 1 --2- 172.21.15.151:0/4248590781 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f516807af50 0x7f516807d410 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f51880b9e20 tx=0x7f5174009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:07.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:07.343+0000 7f5187e00640 1 -- 172.21.15.151:0/4248590781 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone48134", "target": ["mon-mgr", ""]}) -- 0x7f5188069d80 con 0x7f516807af50 2024-09-16T15:56:08.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:08.661+0000 7f517ea00640 1 -- 172.21.15.151:0/4248590781 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5174001cf0 con 0x7f516807af50 2024-09-16T15:56:08.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:08.664+0000 7f5187e00640 1 -- 172.21.15.151:0/4248590781 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f516807af50 msgr2=0x7f516807d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:08.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:08.664+0000 7f5187e00640 1 --2- 172.21.15.151:0/4248590781 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f516807af50 0x7f516807d410 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f51880b9e20 tx=0x7f5174009040 comp rx=0 tx=0).stop 2024-09-16T15:56:08.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:08.664+0000 7f5187e00640 1 -- 172.21.15.151:0/4248590781 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f51880b43c0 msgr2=0x7f51880cc720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:08.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:08.664+0000 7f5187e00640 1 --2- 172.21.15.151:0/4248590781 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f51880b43c0 0x7f51880cc720 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f517800e7c0 tx=0x7f517800ec80 comp rx=0 tx=0).stop 2024-09-16T15:56:08.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:08.665+0000 7f5187e00640 1 -- 172.21.15.151:0/4248590781 shutdown_connections 2024-09-16T15:56:08.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:08.665+0000 7f5187e00640 1 --2- 172.21.15.151:0/4248590781 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f516807af50 0x7f516807d410 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:08.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:08.665+0000 7f5187e00640 1 --2- 172.21.15.151:0/4248590781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5188161460 0x7f51880cb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:08.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:08.665+0000 7f5187e00640 1 --2- 172.21.15.151:0/4248590781 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f51880b5480 0x7f51880caec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:08.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:08.665+0000 7f5187e00640 1 --2- 172.21.15.151:0/4248590781 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f51880b43c0 0x7f51880cc720 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:08.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:08.665+0000 7f5187e00640 1 -- 172.21.15.151:0/4248590781 >> 172.21.15.151:0/4248590781 conn(0x7f518809d0b0 msgr2=0x7f51880b3180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:08.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:08.665+0000 7f5187e00640 1 -- 172.21.15.151:0/4248590781 shutdown_connections 2024-09-16T15:56:08.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:08.665+0000 7f5187e00640 1 -- 172.21.15.151:0/4248590781 wait complete. 2024-09-16T15:56:08.703 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone58175 2024-09-16T15:56:09.007 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:09.006+0000 7fcb86000640 1 -- 172.21.15.151:0/2348181924 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb78031bf0 con 0x7fcb88161460 2024-09-16T15:56:09.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:09.007+0000 7fcb8cc00640 1 -- 172.21.15.151:0/2348181924 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb88161460 msgr2=0x7fcb880d22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:09.008 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:09.007+0000 7fcb8cc00640 1 --2- 172.21.15.151:0/2348181924 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb88161460 0x7fcb880d22f0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7fcb880d1ce0 tx=0x7fcb7802a4b0 comp rx=0 tx=0).stop 2024-09-16T15:56:09.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:09.007+0000 7fcb8cc00640 1 -- 172.21.15.151:0/2348181924 shutdown_connections 2024-09-16T15:56:09.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:09.007+0000 7fcb8cc00640 1 --2- 172.21.15.151:0/2348181924 >> v2:172.21.15.167:3300/0 conn(0x7fcb880b5480 0x7fcb880d2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:09.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:09.007+0000 7fcb8cc00640 1 --2- 172.21.15.151:0/2348181924 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb88161460 0x7fcb880d22f0 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:09.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:09.007+0000 7fcb8cc00640 1 -- 172.21.15.151:0/2348181924 >> 172.21.15.151:0/2348181924 conn(0x7fcb8809d0b0 msgr2=0x7fcb880d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:09.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:09.007+0000 7fcb8cc00640 1 -- 172.21.15.151:0/2348181924 shutdown_connections 2024-09-16T15:56:09.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:09.007+0000 7fcb8cc00640 1 -- 172.21.15.151:0/2348181924 wait complete. 2024-09-16T15:56:09.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:09.010+0000 7fcb8cc00640 1 Processor -- start 2024-09-16T15:56:09.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:09.011+0000 7fcb8cc00640 1 -- start start 2024-09-16T15:56:09.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:09.012+0000 7fcb8cc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb880b4f40 0x7fcb881f9360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:09.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:09.012+0000 7fcb8cc00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb880b5480 0x7fcb881f81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:09.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:09.012+0000 7fcb8cc00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb88161460 0x7fcb881f9970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:09.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:09.012+0000 7fcb8cc00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcb880cef30 con 0x7fcb880b4f40 2024-09-16T15:56:09.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:09.012+0000 7fcb8cc00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcb880b4610 con 0x7fcb880b5480 2024-09-16T15:56:09.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:09.012+0000 7fcb8cc00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fcb880cf0b0 con 0x7fcb88161460 2024-09-16T15:56:09.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:09.012+0000 7fcb87400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb880b4f40 0x7fcb881f9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:09.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:09.012+0000 7fcb87400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb880b4f40 0x7fcb881f9360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33804/0 (socket says 172.21.15.151:33804) 2024-09-16T15:56:09.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:09.012+0000 7fcb87400640 1 -- 172.21.15.151:0/1504354835 learned_addr learned my addr 172.21.15.151:0/1504354835 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:56:09.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:09.013+0000 7fcb87400640 1 -- 172.21.15.151:0/1504354835 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb88161460 msgr2=0x7fcb881f9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:09.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:09.013+0000 7fcb87400640 1 --2- 172.21.15.151:0/1504354835 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb88161460 0x7fcb881f9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:09.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:09.013+0000 7fcb87400640 1 -- 172.21.15.151:0/1504354835 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb880b5480 msgr2=0x7fcb881f81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:09.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:09.013+0000 7fcb87400640 1 --2- 172.21.15.151:0/1504354835 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb880b5480 0x7fcb881f81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:09.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:09.013+0000 7fcb87400640 1 -- 172.21.15.151:0/1504354835 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb880a0000 con 0x7fcb880b4f40 2024-09-16T15:56:09.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:09.013+0000 7fcb87400640 1 --2- 172.21.15.151:0/1504354835 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb880b4f40 0x7fcb881f9360 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7fcb78009920 tx=0x7fcb78033c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:09.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:09.014+0000 7fcb7f400640 1 -- 172.21.15.151:0/1504354835 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb78031d60 con 0x7fcb880b4f40 2024-09-16T15:56:09.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:09.014+0000 7fcb7f400640 1 -- 172.21.15.151:0/1504354835 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcb78005020 con 0x7fcb880b4f40 2024-09-16T15:56:09.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:09.014+0000 7fcb7f400640 1 -- 172.21.15.151:0/1504354835 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb78005470 con 0x7fcb880b4f40 2024-09-16T15:56:09.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:09.014+0000 7fcb8cc00640 1 -- 172.21.15.151:0/1504354835 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb880a02e0 con 0x7fcb880b4f40 2024-09-16T15:56:09.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:09.014+0000 7fcb8cc00640 1 -- 172.21.15.151:0/1504354835 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb880a08a0 con 0x7fcb880b4f40 2024-09-16T15:56:09.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:09.015+0000 7fcb8cc00640 1 -- 172.21.15.151:0/1504354835 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb880a0aa0 con 0x7fcb880b4f40 2024-09-16T15:56:09.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:09.017+0000 7fcb7f400640 1 -- 172.21.15.151:0/1504354835 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 54) ==== 104806+0+0 (secure 0 0 0) 0x7fcb780047a0 con 0x7fcb880b4f40 2024-09-16T15:56:09.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:09.018+0000 7fcb7f400640 1 --2- 172.21.15.151:0/1504354835 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcb6807af20 0x7fcb6807d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:09.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:09.018+0000 7fcb7f400640 1 -- 172.21.15.151:0/1504354835 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5907+0+0 (secure 0 0 0) 0x7fcb780bad00 con 0x7fcb880b4f40 2024-09-16T15:56:09.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:09.019+0000 7fcb86a00640 1 --2- 172.21.15.151:0/1504354835 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcb6807af20 0x7fcb6807d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:09.022 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:09.021+0000 7fcb86a00640 1 --2- 172.21.15.151:0/1504354835 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcb6807af20 0x7fcb6807d3e0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fcb74005fd0 tx=0x7fcb74005850 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:09.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:09.025+0000 7fcb7f400640 1 -- 172.21.15.151:0/1504354835 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fcb880a0aa0 con 0x7fcb880b4f40 2024-09-16T15:56:09.434 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:09.432+0000 7fcb8cc00640 1 -- 172.21.15.151:0/1504354835 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone58175", "target": ["mon-mgr", ""]}) -- 0x7fcb88069d80 con 0x7fcb6807af20 2024-09-16T15:56:09.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:09.984+0000 7fcb7f400640 1 -- 172.21.15.151:0/1504354835 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcb7400d9f0 con 0x7fcb6807af20 2024-09-16T15:56:09.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:09.989+0000 7fcb8cc00640 1 -- 172.21.15.151:0/1504354835 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcb6807af20 msgr2=0x7fcb6807d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:09.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:09.989+0000 7fcb8cc00640 1 --2- 172.21.15.151:0/1504354835 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcb6807af20 0x7fcb6807d3e0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fcb74005fd0 tx=0x7fcb74005850 comp rx=0 tx=0).stop 2024-09-16T15:56:09.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:09.989+0000 7fcb8cc00640 1 -- 172.21.15.151:0/1504354835 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb880b4f40 msgr2=0x7fcb881f9360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:09.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:09.989+0000 7fcb8cc00640 1 --2- 172.21.15.151:0/1504354835 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb880b4f40 0x7fcb881f9360 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7fcb78009920 tx=0x7fcb78033c90 comp rx=0 tx=0).stop 2024-09-16T15:56:09.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:09.989+0000 7fcb8cc00640 1 -- 172.21.15.151:0/1504354835 shutdown_connections 2024-09-16T15:56:09.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:09.989+0000 7fcb8cc00640 1 --2- 172.21.15.151:0/1504354835 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcb6807af20 0x7fcb6807d3e0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:09.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:09.989+0000 7fcb8cc00640 1 --2- 172.21.15.151:0/1504354835 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb880b5480 0x7fcb881f81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:09.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:09.989+0000 7fcb8cc00640 1 --2- 172.21.15.151:0/1504354835 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb88161460 0x7fcb881f9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:09.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:09.989+0000 7fcb8cc00640 1 --2- 172.21.15.151:0/1504354835 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb880b4f40 0x7fcb881f9360 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:09.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:09.989+0000 7fcb8cc00640 1 -- 172.21.15.151:0/1504354835 >> 172.21.15.151:0/1504354835 conn(0x7fcb8809d0b0 msgr2=0x7fcb880cf4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:09.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:09.989+0000 7fcb8cc00640 1 -- 172.21.15.151:0/1504354835 shutdown_connections 2024-09-16T15:56:09.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:09.989+0000 7fcb8cc00640 1 -- 172.21.15.151:0/1504354835 wait complete. 2024-09-16T15:56:10.028 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone67026 2024-09-16T15:56:10.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.254+0000 7fa144000640 1 -- 172.21.15.151:0/65788472 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa13c0cccd0 msgr2=0x7fa13c0cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:10.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.254+0000 7fa144000640 1 --2- 172.21.15.151:0/65788472 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa13c0cccd0 0x7fa13c0cdbb0 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7fa13c0d0cd0 tx=0x7fa13402aa30 comp rx=0 tx=0).stop 2024-09-16T15:56:10.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.254+0000 7fa144000640 1 -- 172.21.15.151:0/65788472 shutdown_connections 2024-09-16T15:56:10.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.254+0000 7fa144000640 1 --2- 172.21.15.151:0/65788472 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa13c0cccd0 0x7fa13c0cdbb0 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:10.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.254+0000 7fa144000640 1 --2- 172.21.15.151:0/65788472 >> v2:172.21.15.167:3300/0 conn(0x7fa13c145750 0x7fa13c073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:10.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.254+0000 7fa144000640 1 -- 172.21.15.151:0/65788472 >> 172.21.15.151:0/65788472 conn(0x7fa13c0b1840 msgr2=0x7fa13c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:10.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.254+0000 7fa144000640 1 -- 172.21.15.151:0/65788472 shutdown_connections 2024-09-16T15:56:10.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.255+0000 7fa144000640 1 -- 172.21.15.151:0/65788472 wait complete. 2024-09-16T15:56:10.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.256+0000 7fa144000640 1 Processor -- start 2024-09-16T15:56:10.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.257+0000 7fa144000640 1 -- start start 2024-09-16T15:56:10.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.258+0000 7fa144000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa13c06efa0 0x7fa13c0b46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:10.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.258+0000 7fa144000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa13c0cccd0 0x7fa13c0cfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:10.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.258+0000 7fa144000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa13c145750 0x7fa13c0d02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:10.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.258+0000 7fa144000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa13c0ce480 con 0x7fa13c0cccd0 2024-09-16T15:56:10.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.258+0000 7fa144000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa13c0ce300 con 0x7fa13c06efa0 2024-09-16T15:56:10.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.258+0000 7fa144000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fa13c0ce600 con 0x7fa13c145750 2024-09-16T15:56:10.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.259+0000 7fa140c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa13c0cccd0 0x7fa13c0cfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:10.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.259+0000 7fa140c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa13c0cccd0 0x7fa13c0cfdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33816/0 (socket says 172.21.15.151:33816) 2024-09-16T15:56:10.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.260+0000 7fa140c00640 1 -- 172.21.15.151:0/2031801506 learned_addr learned my addr 172.21.15.151:0/2031801506 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:56:10.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.260+0000 7fa141600640 1 --2- 172.21.15.151:0/2031801506 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa13c06efa0 0x7fa13c0b46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:10.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.260+0000 7fa140c00640 1 -- 172.21.15.151:0/2031801506 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa13c145750 msgr2=0x7fa13c0d02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:10.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.260+0000 7fa140c00640 1 --2- 172.21.15.151:0/2031801506 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa13c145750 0x7fa13c0d02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:10.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.260+0000 7fa140c00640 1 -- 172.21.15.151:0/2031801506 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa13c06efa0 msgr2=0x7fa13c0b46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:10.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.260+0000 7fa140c00640 1 --2- 172.21.15.151:0/2031801506 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa13c06efa0 0x7fa13c0b46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:10.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.260+0000 7fa140c00640 1 -- 172.21.15.151:0/2031801506 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa13c0b3970 con 0x7fa13c0cccd0 2024-09-16T15:56:10.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.261+0000 7fa140c00640 1 --2- 172.21.15.151:0/2031801506 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa13c0cccd0 0x7fa13c0cfdb0 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7fa12800c450 tx=0x7fa12800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:10.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.261+0000 7fa139600640 1 -- 172.21.15.151:0/2031801506 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1280052d0 con 0x7fa13c0cccd0 2024-09-16T15:56:10.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.261+0000 7fa139600640 1 -- 172.21.15.151:0/2031801506 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa128005470 con 0x7fa13c0cccd0 2024-09-16T15:56:10.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.261+0000 7fa139600640 1 -- 172.21.15.151:0/2031801506 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa128012870 con 0x7fa13c0cccd0 2024-09-16T15:56:10.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.261+0000 7fa144000640 1 -- 172.21.15.151:0/2031801506 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa13c1f83c0 con 0x7fa13c0cccd0 2024-09-16T15:56:10.263 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.262+0000 7fa144000640 1 -- 172.21.15.151:0/2031801506 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa13c1f8980 con 0x7fa13c0cccd0 2024-09-16T15:56:10.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.263+0000 7fa144000640 1 -- 172.21.15.151:0/2031801506 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa13c0736e0 con 0x7fa13c0cccd0 2024-09-16T15:56:10.268 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.267+0000 7fa139600640 1 -- 172.21.15.151:0/2031801506 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 54) ==== 104806+0+0 (secure 0 0 0) 0x7fa1280035f0 con 0x7fa13c0cccd0 2024-09-16T15:56:10.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.269+0000 7fa139600640 1 --2- 172.21.15.151:0/2031801506 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa11c07af20 0x7fa11c07d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:10.270 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.269+0000 7fa139600640 1 -- 172.21.15.151:0/2031801506 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5907+0+0 (secure 0 0 0) 0x7fa12809a120 con 0x7fa13c0cccd0 2024-09-16T15:56:10.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.271+0000 7fa141600640 1 --2- 172.21.15.151:0/2031801506 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa11c07af20 0x7fa11c07d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:10.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.275+0000 7fa139600640 1 -- 172.21.15.151:0/2031801506 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa13c0736e0 con 0x7fa13c0cccd0 2024-09-16T15:56:10.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.275+0000 7fa141600640 1 --2- 172.21.15.151:0/2031801506 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa11c07af20 0x7fa11c07d3e0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fa130005fd0 tx=0x7fa130005850 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:10.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.637+0000 7fa144000640 1 -- 172.21.15.151:0/2031801506 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone67026", "target": ["mon-mgr", ""]}) -- 0x7fa13c0cdbb0 con 0x7fa11c07af20 2024-09-16T15:56:10.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.775+0000 7fa139600640 1 -- 172.21.15.151:0/2031801506 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa13c0cdbb0 con 0x7fa11c07af20 2024-09-16T15:56:10.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.777+0000 7fa144000640 1 -- 172.21.15.151:0/2031801506 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa11c07af20 msgr2=0x7fa11c07d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:10.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.777+0000 7fa144000640 1 --2- 172.21.15.151:0/2031801506 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa11c07af20 0x7fa11c07d3e0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fa130005fd0 tx=0x7fa130005850 comp rx=0 tx=0).stop 2024-09-16T15:56:10.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.777+0000 7fa144000640 1 -- 172.21.15.151:0/2031801506 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa13c0cccd0 msgr2=0x7fa13c0cfdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:10.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.777+0000 7fa144000640 1 --2- 172.21.15.151:0/2031801506 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa13c0cccd0 0x7fa13c0cfdb0 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7fa12800c450 tx=0x7fa12800c910 comp rx=0 tx=0).stop 2024-09-16T15:56:10.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.777+0000 7fa144000640 1 -- 172.21.15.151:0/2031801506 shutdown_connections 2024-09-16T15:56:10.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.777+0000 7fa144000640 1 --2- 172.21.15.151:0/2031801506 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa11c07af20 0x7fa11c07d3e0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:10.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.777+0000 7fa144000640 1 --2- 172.21.15.151:0/2031801506 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa13c145750 0x7fa13c0d02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:10.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.777+0000 7fa144000640 1 --2- 172.21.15.151:0/2031801506 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa13c0cccd0 0x7fa13c0cfdb0 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:10.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.777+0000 7fa144000640 1 --2- 172.21.15.151:0/2031801506 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa13c06efa0 0x7fa13c0b46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:10.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.777+0000 7fa144000640 1 -- 172.21.15.151:0/2031801506 >> 172.21.15.151:0/2031801506 conn(0x7fa13c0b1840 msgr2=0x7fa13c0a9700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:10.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.777+0000 7fa144000640 1 -- 172.21.15.151:0/2031801506 shutdown_connections 2024-09-16T15:56:10.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.777+0000 7fa144000640 1 -- 172.21.15.151:0/2031801506 wait complete. 2024-09-16T15:56:10.794 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone62040 2024-09-16T15:56:10.957 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.956+0000 7f3e3b200640 1 -- 172.21.15.151:0/651396162 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e3c0cdfa0 msgr2=0x7f3e3c0ae290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:10.957 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.956+0000 7f3e3b200640 1 --2- 172.21.15.151:0/651396162 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e3c0cdfa0 0x7f3e3c0ae290 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f3e28009860 tx=0x7f3e2802a540 comp rx=0 tx=0).stop 2024-09-16T15:56:10.957 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.956+0000 7f3e3b200640 1 -- 172.21.15.151:0/651396162 shutdown_connections 2024-09-16T15:56:10.957 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.956+0000 7f3e3b200640 1 --2- 172.21.15.151:0/651396162 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3e3c06ff60 0x7f3e3c0ca7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:10.957 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.956+0000 7f3e3b200640 1 --2- 172.21.15.151:0/651396162 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e3c0cdfa0 0x7f3e3c0ae290 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:10.957 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.956+0000 7f3e3b200640 1 -- 172.21.15.151:0/651396162 >> 172.21.15.151:0/651396162 conn(0x7f3e3c0cf9f0 msgr2=0x7f3e3c0cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:10.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.956+0000 7f3e3b200640 1 -- 172.21.15.151:0/651396162 shutdown_connections 2024-09-16T15:56:10.958 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.956+0000 7f3e3b200640 1 -- 172.21.15.151:0/651396162 wait complete. 2024-09-16T15:56:10.959 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.958+0000 7f3e3b200640 1 Processor -- start 2024-09-16T15:56:10.959 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.958+0000 7f3e3b200640 1 -- start start 2024-09-16T15:56:10.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.959+0000 7f3e3b200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3e3c06ff60 0x7f3e3c150fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:10.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.959+0000 7f3e3b200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e3c0cdfa0 0x7f3e3c0b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:10.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.959+0000 7f3e3b200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e3c15e0c0 0x7f3e3c14d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:10.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.959+0000 7f3e3b200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3c0cb1d0 con 0x7f3e3c15e0c0 2024-09-16T15:56:10.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.959+0000 7f3e3b200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3c0cb050 con 0x7f3e3c0cdfa0 2024-09-16T15:56:10.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.959+0000 7f3e3b200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f3e3c0cb350 con 0x7f3e3c06ff60 2024-09-16T15:56:10.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.959+0000 7f3e39400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e3c0cdfa0 0x7f3e3c0b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:10.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.959+0000 7f3e39e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3e3c06ff60 0x7f3e3c150fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:10.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.959+0000 7f3e3a800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e3c15e0c0 0x7f3e3c14d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:10.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.959+0000 7f3e39400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e3c0cdfa0 0x7f3e3c0b4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.151:47052/0 (socket says 172.21.15.151:47052) 2024-09-16T15:56:10.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.959+0000 7f3e39400640 1 -- 172.21.15.151:0/485076918 learned_addr learned my addr 172.21.15.151:0/485076918 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:56:10.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.959+0000 7f3e39e00640 1 -- 172.21.15.151:0/485076918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e3c0cdfa0 msgr2=0x7f3e3c0b4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:10.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.959+0000 7f3e39e00640 1 --2- 172.21.15.151:0/485076918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e3c0cdfa0 0x7f3e3c0b4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:10.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.959+0000 7f3e39e00640 1 -- 172.21.15.151:0/485076918 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e3c15e0c0 msgr2=0x7f3e3c14d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:10.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.959+0000 7f3e39e00640 1 --2- 172.21.15.151:0/485076918 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e3c15e0c0 0x7f3e3c14d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:10.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.959+0000 7f3e39e00640 1 -- 172.21.15.151:0/485076918 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e3c1e3b80 con 0x7f3e3c06ff60 2024-09-16T15:56:10.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.959+0000 7f3e3a800640 1 --2- 172.21.15.151:0/485076918 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e3c15e0c0 0x7f3e3c14d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:56:10.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.960+0000 7f3e39e00640 1 --2- 172.21.15.151:0/485076918 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3e3c06ff60 0x7f3e3c150fa0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c00c3c0 tx=0x7f3e2c00c7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:10.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.960+0000 7f3e32000640 1 -- 172.21.15.151:0/485076918 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e2c017070 con 0x7f3e3c06ff60 2024-09-16T15:56:10.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.960+0000 7f3e32000640 1 -- 172.21.15.151:0/485076918 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3e2c0035f0 con 0x7f3e3c06ff60 2024-09-16T15:56:10.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.960+0000 7f3e3b200640 1 -- 172.21.15.151:0/485076918 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e3c1514e0 con 0x7f3e3c06ff60 2024-09-16T15:56:10.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.960+0000 7f3e32000640 1 -- 172.21.15.151:0/485076918 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e2c003a40 con 0x7f3e3c06ff60 2024-09-16T15:56:10.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.961+0000 7f3e3b200640 1 -- 172.21.15.151:0/485076918 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e3c1e38a0 con 0x7f3e3c06ff60 2024-09-16T15:56:10.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.962+0000 7f3e3b200640 1 -- 172.21.15.151:0/485076918 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e3c0cb8e0 con 0x7f3e3c06ff60 2024-09-16T15:56:10.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.963+0000 7f3e32000640 1 -- 172.21.15.151:0/485076918 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 54) ==== 104806+0+0 (secure 0 0 0) 0x7f3e2c01a070 con 0x7f3e3c06ff60 2024-09-16T15:56:10.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.964+0000 7f3e32000640 1 --2- 172.21.15.151:0/485076918 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3e1c07af20 0x7f3e1c07d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:10.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.964+0000 7f3e32000640 1 -- 172.21.15.151:0/485076918 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(147..147 src has 1..147) ==== 5907+0+0 (secure 0 0 0) 0x7f3e2c098f40 con 0x7f3e3c06ff60 2024-09-16T15:56:10.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.964+0000 7f3e39400640 1 --2- 172.21.15.151:0/485076918 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3e1c07af20 0x7f3e1c07d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:10.965 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.965+0000 7f3e39400640 1 --2- 172.21.15.151:0/485076918 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3e1c07af20 0x7f3e1c07d3e0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c0b0720 tx=0x7f3e28004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:10.970 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:10.969+0000 7f3e32000640 1 -- 172.21.15.151:0/485076918 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f3e3c1514e0 con 0x7f3e3c06ff60 2024-09-16T15:56:11.192 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.191+0000 7f3e3b200640 1 -- 172.21.15.151:0/485076918 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone62040", "target": ["mon-mgr", ""]}) -- 0x7f3e3c06fb60 con 0x7f3e1c07af20 2024-09-16T15:56:11.201 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.200+0000 7f3e32000640 1 -- 172.21.15.151:0/485076918 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7f3e3c06fb60 con 0x7f3e1c07af20 2024-09-16T15:56:11.201 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-16T15:56:11.201 INFO:teuthology.orchestra.run.smithi151.stdout: "status": { 2024-09-16T15:56:11.202 INFO:teuthology.orchestra.run.smithi151.stdout: "state": "canceled", 2024-09-16T15:56:11.202 INFO:teuthology.orchestra.run.smithi151.stdout: "source": { 2024-09-16T15:56:11.202 INFO:teuthology.orchestra.run.smithi151.stdout: "volume": "cephfs", 2024-09-16T15:56:11.202 INFO:teuthology.orchestra.run.smithi151.stdout: "subvolume": "sv1", 2024-09-16T15:56:11.202 INFO:teuthology.orchestra.run.smithi151.stdout: "snapshot": "ss1" 2024-09-16T15:56:11.202 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:56:11.202 INFO:teuthology.orchestra.run.smithi151.stdout: "failure": { 2024-09-16T15:56:11.202 INFO:teuthology.orchestra.run.smithi151.stdout: "errno": "4", 2024-09-16T15:56:11.202 INFO:teuthology.orchestra.run.smithi151.stdout: "error_msg": "user interrupted clone operation" 2024-09-16T15:56:11.202 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:56:11.202 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:56:11.202 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-16T15:56:11.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.203+0000 7f3e3b200640 1 -- 172.21.15.151:0/485076918 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3e1c07af20 msgr2=0x7f3e1c07d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:11.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.203+0000 7f3e3b200640 1 --2- 172.21.15.151:0/485076918 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3e1c07af20 0x7f3e1c07d3e0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c0b0720 tx=0x7f3e28004730 comp rx=0 tx=0).stop 2024-09-16T15:56:11.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.203+0000 7f3e3b200640 1 -- 172.21.15.151:0/485076918 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3e3c06ff60 msgr2=0x7f3e3c150fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:11.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.203+0000 7f3e3b200640 1 --2- 172.21.15.151:0/485076918 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3e3c06ff60 0x7f3e3c150fa0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c00c3c0 tx=0x7f3e2c00c7c0 comp rx=0 tx=0).stop 2024-09-16T15:56:11.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.203+0000 7f3e3b200640 1 -- 172.21.15.151:0/485076918 shutdown_connections 2024-09-16T15:56:11.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.203+0000 7f3e3b200640 1 --2- 172.21.15.151:0/485076918 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3e1c07af20 0x7f3e1c07d3e0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:11.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.203+0000 7f3e3b200640 1 --2- 172.21.15.151:0/485076918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e3c0cdfa0 0x7f3e3c0b4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:11.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.203+0000 7f3e3b200640 1 --2- 172.21.15.151:0/485076918 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e3c15e0c0 0x7f3e3c14d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:11.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.203+0000 7f3e3b200640 1 --2- 172.21.15.151:0/485076918 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3e3c06ff60 0x7f3e3c150fa0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:11.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.203+0000 7f3e3b200640 1 -- 172.21.15.151:0/485076918 >> 172.21.15.151:0/485076918 conn(0x7f3e3c0cf9f0 msgr2=0x7f3e3c073f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:11.204 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.203+0000 7f3e3b200640 1 -- 172.21.15.151:0/485076918 shutdown_connections 2024-09-16T15:56:11.205 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.203+0000 7f3e3b200640 1 -- 172.21.15.151:0/485076918 wait complete. 2024-09-16T15:56:11.221 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-16T15:56:11.221 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone21631 2024-09-16T15:56:11.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.380+0000 7f17b4600640 1 -- 172.21.15.151:0/1570919220 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17ac06efa0 msgr2=0x7f17ac0cc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:11.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.380+0000 7f17b4600640 1 --2- 172.21.15.151:0/1570919220 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17ac06efa0 0x7f17ac0cc5c0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f17ac0d0cd0 tx=0x7f179802a4d0 comp rx=0 tx=0).stop 2024-09-16T15:56:11.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.381+0000 7f17b4600640 1 -- 172.21.15.151:0/1570919220 shutdown_connections 2024-09-16T15:56:11.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.381+0000 7f17b4600640 1 --2- 172.21.15.151:0/1570919220 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f17ac0ccb00 0x7f17ac15f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:11.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.381+0000 7f17b4600640 1 --2- 172.21.15.151:0/1570919220 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17ac06efa0 0x7f17ac0cc5c0 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:11.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.381+0000 7f17b4600640 1 --2- 172.21.15.151:0/1570919220 >> v2:172.21.15.167:3300/0 conn(0x7f17ac145750 0x7f17ac073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:11.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.381+0000 7f17b4600640 1 -- 172.21.15.151:0/1570919220 >> 172.21.15.151:0/1570919220 conn(0x7f17ac0b1840 msgr2=0x7f17ac0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:11.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.381+0000 7f17b4600640 1 -- 172.21.15.151:0/1570919220 shutdown_connections 2024-09-16T15:56:11.382 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.381+0000 7f17b4600640 1 -- 172.21.15.151:0/1570919220 wait complete. 2024-09-16T15:56:11.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.382+0000 7f17b4600640 1 Processor -- start 2024-09-16T15:56:11.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.382+0000 7f17b4600640 1 -- start start 2024-09-16T15:56:11.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.383+0000 7f17b4600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17ac06efa0 0x7f17ac0c4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:11.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.383+0000 7f17b4600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f17ac0ccb00 0x7f17ac156ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:11.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.383+0000 7f17b4600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f17ac145750 0x7f17ac0cb900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:11.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.383+0000 7f17b4600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f17ac1602c0 con 0x7f17ac06efa0 2024-09-16T15:56:11.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.383+0000 7f17b4600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f17ac160140 con 0x7f17ac145750 2024-09-16T15:56:11.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.383+0000 7f17b4600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f17ac160440 con 0x7f17ac0ccb00 2024-09-16T15:56:11.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.384+0000 7f17b1c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17ac06efa0 0x7f17ac0c4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:11.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.384+0000 7f17b1c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17ac06efa0 0x7f17ac0c4ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33840/0 (socket says 172.21.15.151:33840) 2024-09-16T15:56:11.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.384+0000 7f17b1c00640 1 -- 172.21.15.151:0/222595956 learned_addr learned my addr 172.21.15.151:0/222595956 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:56:11.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.384+0000 7f17b2600640 1 --2- 172.21.15.151:0/222595956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f17ac145750 0x7f17ac0cb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:11.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.384+0000 7f17b1200640 1 --2- 172.21.15.151:0/222595956 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f17ac0ccb00 0x7f17ac156ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:11.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.384+0000 7f17b1c00640 1 -- 172.21.15.151:0/222595956 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f17ac0ccb00 msgr2=0x7f17ac156ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:11.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.384+0000 7f17b1c00640 1 --2- 172.21.15.151:0/222595956 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f17ac0ccb00 0x7f17ac156ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:11.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.384+0000 7f17b1c00640 1 -- 172.21.15.151:0/222595956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f17ac145750 msgr2=0x7f17ac0cb900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:11.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.384+0000 7f17b1c00640 1 --2- 172.21.15.151:0/222595956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f17ac145750 0x7f17ac0cb900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:11.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.384+0000 7f17b1c00640 1 -- 172.21.15.151:0/222595956 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17ac1f8290 con 0x7f17ac06efa0 2024-09-16T15:56:11.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.384+0000 7f17b1200640 1 --2- 172.21.15.151:0/222595956 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f17ac0ccb00 0x7f17ac156ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:56:11.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.384+0000 7f17b1c00640 1 --2- 172.21.15.151:0/222595956 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17ac06efa0 0x7f17ac0c4ab0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f17a000dac0 tx=0x7f17a000df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:11.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.384+0000 7f17a9600640 1 -- 172.21.15.151:0/222595956 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a0017070 con 0x7f17ac06efa0 2024-09-16T15:56:11.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.384+0000 7f17a9600640 1 -- 172.21.15.151:0/222595956 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f17a0003970 con 0x7f17ac06efa0 2024-09-16T15:56:11.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.384+0000 7f17a9600640 1 -- 172.21.15.151:0/222595956 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a0003dc0 con 0x7f17ac06efa0 2024-09-16T15:56:11.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.385+0000 7f17b4600640 1 -- 172.21.15.151:0/222595956 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17ac0c4ff0 con 0x7f17ac06efa0 2024-09-16T15:56:11.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.385+0000 7f17b4600640 1 -- 172.21.15.151:0/222595956 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17ac141fb0 con 0x7f17ac06efa0 2024-09-16T15:56:11.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.386+0000 7f17b4600640 1 -- 172.21.15.151:0/222595956 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17ac15f9f0 con 0x7f17ac06efa0 2024-09-16T15:56:11.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.387+0000 7f17a9600640 1 -- 172.21.15.151:0/222595956 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 54) ==== 104806+0+0 (secure 0 0 0) 0x7f17a0004020 con 0x7f17ac06efa0 2024-09-16T15:56:11.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.388+0000 7f17a9600640 1 --2- 172.21.15.151:0/222595956 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f178c07af20 0x7f178c07d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:11.389 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.388+0000 7f17a9600640 1 -- 172.21.15.151:0/222595956 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5907+0+0 (secure 0 0 0) 0x7f17a009a310 con 0x7f17ac06efa0 2024-09-16T15:56:11.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.389+0000 7f17b1200640 1 --2- 172.21.15.151:0/222595956 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f178c07af20 0x7f178c07d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:11.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.389+0000 7f17b1200640 1 --2- 172.21.15.151:0/222595956 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f178c07af20 0x7f178c07d3e0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f1798033000 tx=0x7f1798004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:11.393 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.393+0000 7f17a9600640 1 -- 172.21.15.151:0/222595956 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f17a0020030 con 0x7f17ac06efa0 2024-09-16T15:56:11.615 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.614+0000 7f17b4600640 1 -- 172.21.15.151:0/222595956 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone21631", "target": ["mon-mgr", ""]}) -- 0x7f17ac1f87e0 con 0x7f178c07af20 2024-09-16T15:56:11.624 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.624+0000 7f17a9600640 1 -- 172.21.15.151:0/222595956 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+46 (secure 0 0 0) 0x7f17ac1f87e0 con 0x7f178c07af20 2024-09-16T15:56:11.625 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-16T15:56:11.625 INFO:teuthology.orchestra.run.smithi151.stdout: "status": { 2024-09-16T15:56:11.625 INFO:teuthology.orchestra.run.smithi151.stdout: "state": "complete" 2024-09-16T15:56:11.625 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:56:11.625 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-16T15:56:11.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.626+0000 7f17b4600640 1 -- 172.21.15.151:0/222595956 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f178c07af20 msgr2=0x7f178c07d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:11.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.626+0000 7f17b4600640 1 --2- 172.21.15.151:0/222595956 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f178c07af20 0x7f178c07d3e0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f1798033000 tx=0x7f1798004730 comp rx=0 tx=0).stop 2024-09-16T15:56:11.627 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.626+0000 7f17b4600640 1 -- 172.21.15.151:0/222595956 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17ac06efa0 msgr2=0x7f17ac0c4ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:11.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.626+0000 7f17b4600640 1 --2- 172.21.15.151:0/222595956 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17ac06efa0 0x7f17ac0c4ab0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f17a000dac0 tx=0x7f17a000df80 comp rx=0 tx=0).stop 2024-09-16T15:56:11.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.626+0000 7f17b4600640 1 -- 172.21.15.151:0/222595956 shutdown_connections 2024-09-16T15:56:11.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.626+0000 7f17b4600640 1 --2- 172.21.15.151:0/222595956 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f178c07af20 0x7f178c07d3e0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:11.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.626+0000 7f17b4600640 1 --2- 172.21.15.151:0/222595956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f17ac145750 0x7f17ac0cb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:11.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.626+0000 7f17b4600640 1 --2- 172.21.15.151:0/222595956 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f17ac0ccb00 0x7f17ac156ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:11.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.626+0000 7f17b4600640 1 --2- 172.21.15.151:0/222595956 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17ac06efa0 0x7f17ac0c4ab0 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:11.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.626+0000 7f17b4600640 1 -- 172.21.15.151:0/222595956 >> 172.21.15.151:0/222595956 conn(0x7f17ac0b1840 msgr2=0x7f17ac0b0610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:11.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.626+0000 7f17b4600640 1 -- 172.21.15.151:0/222595956 shutdown_connections 2024-09-16T15:56:11.628 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.626+0000 7f17b4600640 1 -- 172.21.15.151:0/222595956 wait complete. 2024-09-16T15:56:11.643 DEBUG:tasks.cephfs.test_volumes:current clone state = complete 2024-09-16T15:56:11.643 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone54262 2024-09-16T15:56:11.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.805+0000 7f81e6600640 1 -- 172.21.15.151:0/3206898663 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f81e00cec60 msgr2=0x7f81e00cf070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:11.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.805+0000 7f81e6600640 1 --2- 172.21.15.151:0/3206898663 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f81e00cec60 0x7f81e00cf070 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f81d8005fd0 tx=0x7f81d802a8b0 comp rx=0 tx=0).stop 2024-09-16T15:56:11.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.806+0000 7f81e6600640 1 -- 172.21.15.151:0/3206898663 shutdown_connections 2024-09-16T15:56:11.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.806+0000 7f81e6600640 1 --2- 172.21.15.151:0/3206898663 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f81e00cec60 0x7f81e00cf070 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:11.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.806+0000 7f81e6600640 1 --2- 172.21.15.151:0/3206898663 >> v2:172.21.15.167:3300/0 conn(0x7f81e0161760 0x7f81e00ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:11.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.806+0000 7f81e6600640 1 -- 172.21.15.151:0/3206898663 >> 172.21.15.151:0/3206898663 conn(0x7f81e00b1840 msgr2=0x7f81e00b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:11.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.806+0000 7f81e6600640 1 -- 172.21.15.151:0/3206898663 shutdown_connections 2024-09-16T15:56:11.807 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.806+0000 7f81e6600640 1 -- 172.21.15.151:0/3206898663 wait complete. 2024-09-16T15:56:11.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.807+0000 7f81e6600640 1 Processor -- start 2024-09-16T15:56:11.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.807+0000 7f81e6600640 1 -- start start 2024-09-16T15:56:11.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.808+0000 7f81e6600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81e00af190 0x7f81e01f7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:11.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.808+0000 7f81e6600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f81e00cec60 0x7f81e01f8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:11.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.808+0000 7f81e6600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f81e0161760 0x7f81e0148da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:11.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.808+0000 7f81e6600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f81e0162310 con 0x7f81e00cec60 2024-09-16T15:56:11.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.808+0000 7f81e6600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f81e0162190 con 0x7f81e00af190 2024-09-16T15:56:11.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.808+0000 7f81e6600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f81e0162490 con 0x7f81e0161760 2024-09-16T15:56:11.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.808+0000 7f81dea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f81e00cec60 0x7f81e01f8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:11.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.809+0000 7f81dea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f81e00cec60 0x7f81e01f8530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33856/0 (socket says 172.21.15.151:33856) 2024-09-16T15:56:11.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.809+0000 7f81dea00640 1 -- 172.21.15.151:0/764032795 learned_addr learned my addr 172.21.15.151:0/764032795 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:56:11.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.809+0000 7f81dfe00640 1 --2- 172.21.15.151:0/764032795 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f81e0161760 0x7f81e0148da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:11.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.809+0000 7f81df400640 1 --2- 172.21.15.151:0/764032795 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81e00af190 0x7f81e01f7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:11.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.809+0000 7f81dea00640 1 -- 172.21.15.151:0/764032795 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f81e0161760 msgr2=0x7f81e0148da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:11.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.809+0000 7f81dea00640 1 --2- 172.21.15.151:0/764032795 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f81e0161760 0x7f81e0148da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:11.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.809+0000 7f81dea00640 1 -- 172.21.15.151:0/764032795 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81e00af190 msgr2=0x7f81e01f7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:11.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.809+0000 7f81dea00640 1 --2- 172.21.15.151:0/764032795 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81e00af190 0x7f81e01f7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:11.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.809+0000 7f81dea00640 1 -- 172.21.15.151:0/764032795 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81e01585c0 con 0x7f81e00cec60 2024-09-16T15:56:11.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.809+0000 7f81dfe00640 1 --2- 172.21.15.151:0/764032795 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f81e0161760 0x7f81e0148da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:56:11.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.809+0000 7f81dea00640 1 --2- 172.21.15.151:0/764032795 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f81e00cec60 0x7f81e01f8530 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f81cc00c730 tx=0x7f81cc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:11.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.809+0000 7f81d7400640 1 -- 172.21.15.151:0/764032795 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81cc016020 con 0x7f81e00cec60 2024-09-16T15:56:11.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.809+0000 7f81d7400640 1 -- 172.21.15.151:0/764032795 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f81cc00a930 con 0x7f81e00cec60 2024-09-16T15:56:11.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.810+0000 7f81d7400640 1 -- 172.21.15.151:0/764032795 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81cc00ad80 con 0x7f81e00cec60 2024-09-16T15:56:11.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.810+0000 7f81e6600640 1 -- 172.21.15.151:0/764032795 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81e01588a0 con 0x7f81e00cec60 2024-09-16T15:56:11.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.810+0000 7f81e6600640 1 -- 172.21.15.151:0/764032795 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81e0158e60 con 0x7f81e00cec60 2024-09-16T15:56:11.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.811+0000 7f81e6600640 1 -- 172.21.15.151:0/764032795 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81e01629a0 con 0x7f81e00cec60 2024-09-16T15:56:11.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.812+0000 7f81d7400640 1 -- 172.21.15.151:0/764032795 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 54) ==== 104806+0+0 (secure 0 0 0) 0x7f81cc019020 con 0x7f81e00cec60 2024-09-16T15:56:11.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.813+0000 7f81d7400640 1 --2- 172.21.15.151:0/764032795 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f81c007af50 0x7f81c007d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:11.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.813+0000 7f81d7400640 1 -- 172.21.15.151:0/764032795 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5907+0+0 (secure 0 0 0) 0x7f81cc0994c0 con 0x7f81e00cec60 2024-09-16T15:56:11.814 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.813+0000 7f81df400640 1 --2- 172.21.15.151:0/764032795 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f81c007af50 0x7f81c007d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:11.815 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.814+0000 7f81df400640 1 --2- 172.21.15.151:0/764032795 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f81c007af50 0x7f81c007d410 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f81d00095f0 tx=0x7f81d0005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:11.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:11.817+0000 7f81d7400640 1 -- 172.21.15.151:0/764032795 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f81cc020020 con 0x7f81e00cec60 2024-09-16T15:56:12.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.038+0000 7f81e6600640 1 -- 172.21.15.151:0/764032795 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone54262", "target": ["mon-mgr", ""]}) -- 0x7f81e00b4160 con 0x7f81c007af50 2024-09-16T15:56:12.160 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.159+0000 7f81d7400640 1 -- 172.21.15.151:0/764032795 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+46 (secure 0 0 0) 0x7f81e00b4160 con 0x7f81c007af50 2024-09-16T15:56:12.160 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-16T15:56:12.160 INFO:teuthology.orchestra.run.smithi151.stdout: "status": { 2024-09-16T15:56:12.160 INFO:teuthology.orchestra.run.smithi151.stdout: "state": "complete" 2024-09-16T15:56:12.160 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:56:12.160 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-16T15:56:12.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.161+0000 7f81e6600640 1 -- 172.21.15.151:0/764032795 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f81c007af50 msgr2=0x7f81c007d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:12.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.161+0000 7f81e6600640 1 --2- 172.21.15.151:0/764032795 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f81c007af50 0x7f81c007d410 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f81d00095f0 tx=0x7f81d0005f90 comp rx=0 tx=0).stop 2024-09-16T15:56:12.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.161+0000 7f81e6600640 1 -- 172.21.15.151:0/764032795 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f81e00cec60 msgr2=0x7f81e01f8530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:12.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.161+0000 7f81e6600640 1 --2- 172.21.15.151:0/764032795 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f81e00cec60 0x7f81e01f8530 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f81cc00c730 tx=0x7f81cc00cbf0 comp rx=0 tx=0).stop 2024-09-16T15:56:12.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.162+0000 7f81e6600640 1 -- 172.21.15.151:0/764032795 shutdown_connections 2024-09-16T15:56:12.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.162+0000 7f81e6600640 1 --2- 172.21.15.151:0/764032795 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f81c007af50 0x7f81c007d410 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:12.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.162+0000 7f81e6600640 1 --2- 172.21.15.151:0/764032795 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f81e0161760 0x7f81e0148da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:12.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.162+0000 7f81e6600640 1 --2- 172.21.15.151:0/764032795 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f81e00cec60 0x7f81e01f8530 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:12.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.162+0000 7f81e6600640 1 --2- 172.21.15.151:0/764032795 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81e00af190 0x7f81e01f7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:12.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.162+0000 7f81e6600640 1 -- 172.21.15.151:0/764032795 >> 172.21.15.151:0/764032795 conn(0x7f81e00b1840 msgr2=0x7f81e0072780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:12.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.162+0000 7f81e6600640 1 -- 172.21.15.151:0/764032795 shutdown_connections 2024-09-16T15:56:12.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.162+0000 7f81e6600640 1 -- 172.21.15.151:0/764032795 wait complete. 2024-09-16T15:56:12.178 DEBUG:tasks.cephfs.test_volumes:current clone state = complete 2024-09-16T15:56:12.178 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone83343 2024-09-16T15:56:12.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.338+0000 7fc7a7200640 1 -- 172.21.15.151:0/1003874037 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7a015bec0 msgr2=0x7fc7a006fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:12.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.338+0000 7fc7a7200640 1 --2- 172.21.15.151:0/1003874037 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7a015bec0 0x7fc7a006fce0 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7fc794009830 tx=0x7fc79402a510 comp rx=0 tx=0).stop 2024-09-16T15:56:12.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.338+0000 7fc7a7200640 1 -- 172.21.15.151:0/1003874037 shutdown_connections 2024-09-16T15:56:12.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.338+0000 7fc7a7200640 1 --2- 172.21.15.151:0/1003874037 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7a00cbc00 0x7fc7a0073ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:12.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.338+0000 7fc7a7200640 1 --2- 172.21.15.151:0/1003874037 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7a015bec0 0x7fc7a006fce0 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:12.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.338+0000 7fc7a7200640 1 -- 172.21.15.151:0/1003874037 >> 172.21.15.151:0/1003874037 conn(0x7fc7a00cf9f0 msgr2=0x7fc7a00cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:12.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.338+0000 7fc7a7200640 1 -- 172.21.15.151:0/1003874037 shutdown_connections 2024-09-16T15:56:12.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.338+0000 7fc7a7200640 1 -- 172.21.15.151:0/1003874037 wait complete. 2024-09-16T15:56:12.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.340+0000 7fc7a7200640 1 Processor -- start 2024-09-16T15:56:12.340 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.340+0000 7fc7a7200640 1 -- start start 2024-09-16T15:56:12.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.340+0000 7fc7a7200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7a0072560 0x7fc7a00c9830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:12.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.341+0000 7fc7a7200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7a00cbc00 0x7fc7a01f3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:12.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.341+0000 7fc7a7200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7a015bec0 0x7fc7a015d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:12.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.341+0000 7fc7a7200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a00ae8d0 con 0x7fc7a015bec0 2024-09-16T15:56:12.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.341+0000 7fc7a7200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a00ae750 con 0x7fc7a00cbc00 2024-09-16T15:56:12.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.341+0000 7fc7a7200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fc7a00aea50 con 0x7fc7a0072560 2024-09-16T15:56:12.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.341+0000 7fc7a6800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7a015bec0 0x7fc7a015d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:12.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.341+0000 7fc7a6800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7a015bec0 0x7fc7a015d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33868/0 (socket says 172.21.15.151:33868) 2024-09-16T15:56:12.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.341+0000 7fc7a6800640 1 -- 172.21.15.151:0/1367907954 learned_addr learned my addr 172.21.15.151:0/1367907954 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:56:12.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.341+0000 7fc7a5e00640 1 --2- 172.21.15.151:0/1367907954 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7a0072560 0x7fc7a00c9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:12.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.341+0000 7fc7a5400640 1 --2- 172.21.15.151:0/1367907954 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7a00cbc00 0x7fc7a01f3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:12.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.341+0000 7fc7a6800640 1 -- 172.21.15.151:0/1367907954 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7a0072560 msgr2=0x7fc7a00c9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:12.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.341+0000 7fc7a6800640 1 --2- 172.21.15.151:0/1367907954 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7a0072560 0x7fc7a00c9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:12.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.341+0000 7fc7a6800640 1 -- 172.21.15.151:0/1367907954 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7a00cbc00 msgr2=0x7fc7a01f3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:12.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.341+0000 7fc7a6800640 1 --2- 172.21.15.151:0/1367907954 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7a00cbc00 0x7fc7a01f3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:12.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.341+0000 7fc7a6800640 1 -- 172.21.15.151:0/1367907954 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7a00c9d70 con 0x7fc7a015bec0 2024-09-16T15:56:12.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.341+0000 7fc7a5400640 1 --2- 172.21.15.151:0/1367907954 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7a00cbc00 0x7fc7a01f3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:56:12.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.342+0000 7fc7a6800640 1 --2- 172.21.15.151:0/1367907954 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7a015bec0 0x7fc7a015d9e0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7fc79c00cce0 tx=0x7fc79c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:12.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.342+0000 7fc79a000640 1 -- 172.21.15.151:0/1367907954 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc79c00f450 con 0x7fc7a015bec0 2024-09-16T15:56:12.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.342+0000 7fc79a000640 1 -- 172.21.15.151:0/1367907954 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc79c00f700 con 0x7fc7a015bec0 2024-09-16T15:56:12.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.342+0000 7fc7a7200640 1 -- 172.21.15.151:0/1367907954 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7a00c6e80 con 0x7fc7a015bec0 2024-09-16T15:56:12.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.342+0000 7fc79a000640 1 -- 172.21.15.151:0/1367907954 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc79c00fb50 con 0x7fc7a015bec0 2024-09-16T15:56:12.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.342+0000 7fc7a7200640 1 -- 172.21.15.151:0/1367907954 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7a00c7440 con 0x7fc7a015bec0 2024-09-16T15:56:12.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.343+0000 7fc7a7200640 1 -- 172.21.15.151:0/1367907954 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7a00aef70 con 0x7fc7a015bec0 2024-09-16T15:56:12.345 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.344+0000 7fc79a000640 1 -- 172.21.15.151:0/1367907954 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 54) ==== 104806+0+0 (secure 0 0 0) 0x7fc79c00fcf0 con 0x7fc7a015bec0 2024-09-16T15:56:12.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.345+0000 7fc79a000640 1 --2- 172.21.15.151:0/1367907954 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc78407af20 0x7fc78407d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:12.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.346+0000 7fc79a000640 1 -- 172.21.15.151:0/1367907954 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5907+0+0 (secure 0 0 0) 0x7fc79c09a780 con 0x7fc7a015bec0 2024-09-16T15:56:12.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.346+0000 7fc7a5e00640 1 --2- 172.21.15.151:0/1367907954 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc78407af20 0x7fc78407d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:12.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.346+0000 7fc7a5e00640 1 --2- 172.21.15.151:0/1367907954 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc78407af20 0x7fc78407d3e0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fc794034000 tx=0x7fc794004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:12.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.349+0000 7fc79a000640 1 -- 172.21.15.151:0/1367907954 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc79c014030 con 0x7fc7a015bec0 2024-09-16T15:56:12.572 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.572+0000 7fc7a7200640 1 -- 172.21.15.151:0/1367907954 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone83343", "target": ["mon-mgr", ""]}) -- 0x7fc7a00c7bb0 con 0x7fc78407af20 2024-09-16T15:56:12.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.584+0000 7fc79a000640 1 -- 172.21.15.151:0/1367907954 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+46 (secure 0 0 0) 0x7fc7a00c7bb0 con 0x7fc78407af20 2024-09-16T15:56:12.585 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-16T15:56:12.585 INFO:teuthology.orchestra.run.smithi151.stdout: "status": { 2024-09-16T15:56:12.585 INFO:teuthology.orchestra.run.smithi151.stdout: "state": "complete" 2024-09-16T15:56:12.585 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:56:12.585 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-16T15:56:12.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.586+0000 7fc7a7200640 1 -- 172.21.15.151:0/1367907954 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc78407af20 msgr2=0x7fc78407d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:12.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.586+0000 7fc7a7200640 1 --2- 172.21.15.151:0/1367907954 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc78407af20 0x7fc78407d3e0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fc794034000 tx=0x7fc794004730 comp rx=0 tx=0).stop 2024-09-16T15:56:12.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.586+0000 7fc7a7200640 1 -- 172.21.15.151:0/1367907954 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7a015bec0 msgr2=0x7fc7a015d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:12.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.586+0000 7fc7a7200640 1 --2- 172.21.15.151:0/1367907954 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7a015bec0 0x7fc7a015d9e0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7fc79c00cce0 tx=0x7fc79c00a620 comp rx=0 tx=0).stop 2024-09-16T15:56:12.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.586+0000 7fc7a7200640 1 -- 172.21.15.151:0/1367907954 shutdown_connections 2024-09-16T15:56:12.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.586+0000 7fc7a7200640 1 --2- 172.21.15.151:0/1367907954 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc78407af20 0x7fc78407d3e0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:12.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.586+0000 7fc7a7200640 1 --2- 172.21.15.151:0/1367907954 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7a00cbc00 0x7fc7a01f3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:12.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.586+0000 7fc7a7200640 1 --2- 172.21.15.151:0/1367907954 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7a015bec0 0x7fc7a015d9e0 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:12.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.586+0000 7fc7a7200640 1 --2- 172.21.15.151:0/1367907954 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7a0072560 0x7fc7a00c9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:12.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.586+0000 7fc7a7200640 1 -- 172.21.15.151:0/1367907954 >> 172.21.15.151:0/1367907954 conn(0x7fc7a00cf9f0 msgr2=0x7fc7a006e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:12.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.586+0000 7fc7a7200640 1 -- 172.21.15.151:0/1367907954 shutdown_connections 2024-09-16T15:56:12.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.587+0000 7fc7a7200640 1 -- 172.21.15.151:0/1367907954 wait complete. 2024-09-16T15:56:12.604 DEBUG:tasks.cephfs.test_volumes:current clone state = complete 2024-09-16T15:56:12.604 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone48134 2024-09-16T15:56:12.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.764+0000 7fecf2c00640 1 -- 172.21.15.151:0/4205379765 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fecec0cbc00 msgr2=0x7fecec06fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:12.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.764+0000 7fecf2c00640 1 --2- 172.21.15.151:0/4205379765 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fecec0cbc00 0x7fecec06fce0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fecec06f130 tx=0x7fecd802a580 comp rx=0 tx=0).stop 2024-09-16T15:56:12.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.764+0000 7fecf2c00640 1 -- 172.21.15.151:0/4205379765 shutdown_connections 2024-09-16T15:56:12.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.764+0000 7fecf2c00640 1 --2- 172.21.15.151:0/4205379765 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fecec0cbc00 0x7fecec06fce0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:12.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.764+0000 7fecf2c00640 1 -- 172.21.15.151:0/4205379765 >> 172.21.15.151:0/4205379765 conn(0x7fecec0cf9f0 msgr2=0x7fecec0cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:12.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.764+0000 7fecf2c00640 1 -- 172.21.15.151:0/4205379765 shutdown_connections 2024-09-16T15:56:12.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.764+0000 7fecf2c00640 1 -- 172.21.15.151:0/4205379765 wait complete. 2024-09-16T15:56:12.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.766+0000 7fecf2c00640 1 Processor -- start 2024-09-16T15:56:12.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.766+0000 7fecf2c00640 1 -- start start 2024-09-16T15:56:12.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.767+0000 7fecf2c00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fecec072560 0x7fecec15da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:12.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.767+0000 7fecf2c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fecec0cbc00 0x7fecec15efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:12.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.767+0000 7fecf2c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fecec15bec0 0x7fecec14acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:12.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.767+0000 7fecf2c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fecec0aed50 con 0x7fecec0cbc00 2024-09-16T15:56:12.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.767+0000 7fecf2c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fecec0aebd0 con 0x7fecec15bec0 2024-09-16T15:56:12.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.767+0000 7fecf2c00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fecec0aeed0 con 0x7fecec072560 2024-09-16T15:56:12.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.767+0000 7fecf0e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fecec0cbc00 0x7fecec15efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:12.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.767+0000 7fecf0e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fecec0cbc00 0x7fecec15efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33874/0 (socket says 172.21.15.151:33874) 2024-09-16T15:56:12.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.767+0000 7fecf0e00640 1 -- 172.21.15.151:0/4190209361 learned_addr learned my addr 172.21.15.151:0/4190209361 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:56:12.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.768+0000 7fecf1800640 1 --2- 172.21.15.151:0/4190209361 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fecec072560 0x7fecec15da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:12.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.768+0000 7fecf2200640 1 --2- 172.21.15.151:0/4190209361 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fecec15bec0 0x7fecec14acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:12.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.768+0000 7fecf0e00640 1 -- 172.21.15.151:0/4190209361 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fecec072560 msgr2=0x7fecec15da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:12.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.768+0000 7fecf0e00640 1 --2- 172.21.15.151:0/4190209361 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fecec072560 0x7fecec15da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:12.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.768+0000 7fecf0e00640 1 -- 172.21.15.151:0/4190209361 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fecec15bec0 msgr2=0x7fecec14acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:12.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.768+0000 7fecf0e00640 1 --2- 172.21.15.151:0/4190209361 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fecec15bec0 0x7fecec14acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:12.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.768+0000 7fecf0e00640 1 -- 172.21.15.151:0/4190209361 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecec15c9b0 con 0x7fecec0cbc00 2024-09-16T15:56:12.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.768+0000 7fecf0e00640 1 --2- 172.21.15.151:0/4190209361 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fecec0cbc00 0x7fecec15efd0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fecd802ab70 tx=0x7fecd8032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:12.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.768+0000 7fece9600640 1 -- 172.21.15.151:0/4190209361 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd8040070 con 0x7fecec0cbc00 2024-09-16T15:56:12.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.768+0000 7fece9600640 1 -- 172.21.15.151:0/4190209361 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fecd8005310 con 0x7fecec0cbc00 2024-09-16T15:56:12.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.768+0000 7fece9600640 1 -- 172.21.15.151:0/4190209361 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd8005c90 con 0x7fecec0cbc00 2024-09-16T15:56:12.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.768+0000 7fecf2c00640 1 -- 172.21.15.151:0/4190209361 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecec15cc90 con 0x7fecec0cbc00 2024-09-16T15:56:12.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.769+0000 7fecf2c00640 1 -- 172.21.15.151:0/4190209361 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecec15d250 con 0x7fecec0cbc00 2024-09-16T15:56:12.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.769+0000 7fecf2c00640 1 -- 172.21.15.151:0/4190209361 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecec06fce0 con 0x7fecec0cbc00 2024-09-16T15:56:12.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.771+0000 7fece9600640 1 -- 172.21.15.151:0/4190209361 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 54) ==== 104806+0+0 (secure 0 0 0) 0x7fecd8039040 con 0x7fecec0cbc00 2024-09-16T15:56:12.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.771+0000 7fece9600640 1 --2- 172.21.15.151:0/4190209361 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7feccc07af20 0x7feccc07d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:12.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.772+0000 7fece9600640 1 -- 172.21.15.151:0/4190209361 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5907+0+0 (secure 0 0 0) 0x7fecd80baf40 con 0x7fecec0cbc00 2024-09-16T15:56:12.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.772+0000 7fecf1800640 1 --2- 172.21.15.151:0/4190209361 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7feccc07af20 0x7feccc07d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:12.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.773+0000 7fecf1800640 1 --2- 172.21.15.151:0/4190209361 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7feccc07af20 0x7feccc07d3e0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fece00020f0 tx=0x7fece0009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:12.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.776+0000 7fece9600640 1 -- 172.21.15.151:0/4190209361 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fecd8084f90 con 0x7fecec0cbc00 2024-09-16T15:56:12.997 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:12.996+0000 7fecf2c00640 1 -- 172.21.15.151:0/4190209361 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone48134", "target": ["mon-mgr", ""]}) -- 0x7fecec0b0400 con 0x7feccc07af20 2024-09-16T15:56:17.159 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.158+0000 7fece9600640 1 -- 172.21.15.151:0/4190209361 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7fecec0b0400 con 0x7feccc07af20 2024-09-16T15:56:17.159 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-16T15:56:17.159 INFO:teuthology.orchestra.run.smithi151.stdout: "status": { 2024-09-16T15:56:17.159 INFO:teuthology.orchestra.run.smithi151.stdout: "state": "canceled", 2024-09-16T15:56:17.160 INFO:teuthology.orchestra.run.smithi151.stdout: "source": { 2024-09-16T15:56:17.160 INFO:teuthology.orchestra.run.smithi151.stdout: "volume": "cephfs", 2024-09-16T15:56:17.160 INFO:teuthology.orchestra.run.smithi151.stdout: "subvolume": "sv1", 2024-09-16T15:56:17.160 INFO:teuthology.orchestra.run.smithi151.stdout: "snapshot": "ss1" 2024-09-16T15:56:17.160 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:56:17.160 INFO:teuthology.orchestra.run.smithi151.stdout: "failure": { 2024-09-16T15:56:17.160 INFO:teuthology.orchestra.run.smithi151.stdout: "errno": "4", 2024-09-16T15:56:17.160 INFO:teuthology.orchestra.run.smithi151.stdout: "error_msg": "user interrupted clone operation" 2024-09-16T15:56:17.160 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:56:17.160 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:56:17.160 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-16T15:56:17.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.161+0000 7fecf2c00640 1 -- 172.21.15.151:0/4190209361 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7feccc07af20 msgr2=0x7feccc07d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:17.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.161+0000 7fecf2c00640 1 --2- 172.21.15.151:0/4190209361 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7feccc07af20 0x7feccc07d3e0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fece00020f0 tx=0x7fece0009040 comp rx=0 tx=0).stop 2024-09-16T15:56:17.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.161+0000 7fecf2c00640 1 -- 172.21.15.151:0/4190209361 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fecec0cbc00 msgr2=0x7fecec15efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:17.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.161+0000 7fecf2c00640 1 --2- 172.21.15.151:0/4190209361 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fecec0cbc00 0x7fecec15efd0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fecd802ab70 tx=0x7fecd8032c10 comp rx=0 tx=0).stop 2024-09-16T15:56:17.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.161+0000 7fecf2c00640 1 -- 172.21.15.151:0/4190209361 shutdown_connections 2024-09-16T15:56:17.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.161+0000 7fecf2c00640 1 --2- 172.21.15.151:0/4190209361 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7feccc07af20 0x7feccc07d3e0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:17.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.161+0000 7fecf2c00640 1 --2- 172.21.15.151:0/4190209361 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fecec15bec0 0x7fecec14acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:17.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.161+0000 7fecf2c00640 1 --2- 172.21.15.151:0/4190209361 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fecec0cbc00 0x7fecec15efd0 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:17.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.161+0000 7fecf2c00640 1 --2- 172.21.15.151:0/4190209361 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fecec072560 0x7fecec15da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:17.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.161+0000 7fecf2c00640 1 -- 172.21.15.151:0/4190209361 >> 172.21.15.151:0/4190209361 conn(0x7fecec0cf9f0 msgr2=0x7fecec0cfdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:17.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.162+0000 7fecf2c00640 1 -- 172.21.15.151:0/4190209361 shutdown_connections 2024-09-16T15:56:17.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.162+0000 7fecf2c00640 1 -- 172.21.15.151:0/4190209361 wait complete. 2024-09-16T15:56:17.182 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-16T15:56:17.182 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone58175 2024-09-16T15:56:17.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.346+0000 7fcb3ec00640 1 -- 172.21.15.151:0/2093757012 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb380b3890 msgr2=0x7fcb380d2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:17.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.346+0000 7fcb3ec00640 1 --2- 172.21.15.151:0/2093757012 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb380b3890 0x7fcb380d2330 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7fcb30005fd0 tx=0x7fcb3002aa10 comp rx=0 tx=0).stop 2024-09-16T15:56:17.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.346+0000 7fcb3ec00640 1 -- 172.21.15.151:0/2093757012 shutdown_connections 2024-09-16T15:56:17.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.346+0000 7fcb3ec00640 1 --2- 172.21.15.151:0/2093757012 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb380b3890 0x7fcb380d2330 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:17.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.346+0000 7fcb3ec00640 1 -- 172.21.15.151:0/2093757012 >> 172.21.15.151:0/2093757012 conn(0x7fcb380d3d50 msgr2=0x7fcb380d4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:17.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.346+0000 7fcb3ec00640 1 -- 172.21.15.151:0/2093757012 shutdown_connections 2024-09-16T15:56:17.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.346+0000 7fcb3ec00640 1 -- 172.21.15.151:0/2093757012 wait complete. 2024-09-16T15:56:17.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.347+0000 7fcb3ec00640 1 Processor -- start 2024-09-16T15:56:17.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.347+0000 7fcb3ec00640 1 -- start start 2024-09-16T15:56:17.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.348+0000 7fcb3ec00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb380b3450 0x7fcb381f5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:17.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.348+0000 7fcb3ec00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb380b3890 0x7fcb3815aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:17.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.348+0000 7fcb3ec00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb3815df60 0x7fcb381f7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:17.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.348+0000 7fcb3ec00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcb380cfd40 con 0x7fcb380b3450 2024-09-16T15:56:17.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.348+0000 7fcb3ec00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcb380cfbc0 con 0x7fcb3815df60 2024-09-16T15:56:17.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.348+0000 7fcb3ec00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fcb380d2870 con 0x7fcb380b3890 2024-09-16T15:56:17.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.349+0000 7fcb3d800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb380b3450 0x7fcb381f5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:17.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.349+0000 7fcb3d800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb380b3450 0x7fcb381f5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:59660/0 (socket says 172.21.15.151:59660) 2024-09-16T15:56:17.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.349+0000 7fcb3d800640 1 -- 172.21.15.151:0/1300264983 learned_addr learned my addr 172.21.15.151:0/1300264983 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:56:17.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.349+0000 7fcb3e200640 1 --2- 172.21.15.151:0/1300264983 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb3815df60 0x7fcb381f7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:17.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.349+0000 7fcb3ce00640 1 --2- 172.21.15.151:0/1300264983 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb380b3890 0x7fcb3815aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:17.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.349+0000 7fcb3d800640 1 -- 172.21.15.151:0/1300264983 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb380b3890 msgr2=0x7fcb3815aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:17.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.349+0000 7fcb3d800640 1 --2- 172.21.15.151:0/1300264983 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb380b3890 0x7fcb3815aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:17.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.349+0000 7fcb3d800640 1 -- 172.21.15.151:0/1300264983 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb3815df60 msgr2=0x7fcb381f7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:17.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.349+0000 7fcb3d800640 1 --2- 172.21.15.151:0/1300264983 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb3815df60 0x7fcb381f7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:17.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.349+0000 7fcb3d800640 1 -- 172.21.15.151:0/1300264983 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb381f8c70 con 0x7fcb380b3450 2024-09-16T15:56:17.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.349+0000 7fcb3d800640 1 --2- 172.21.15.151:0/1300264983 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb380b3450 0x7fcb381f5130 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c00b3f0 tx=0x7fcb2c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:17.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.349+0000 7fcb35600640 1 -- 172.21.15.151:0/1300264983 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb2c015020 con 0x7fcb380b3450 2024-09-16T15:56:17.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.350+0000 7fcb35600640 1 -- 172.21.15.151:0/1300264983 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcb2c011920 con 0x7fcb380b3450 2024-09-16T15:56:17.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.350+0000 7fcb35600640 1 -- 172.21.15.151:0/1300264983 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb2c011d70 con 0x7fcb380b3450 2024-09-16T15:56:17.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.350+0000 7fcb3ec00640 1 -- 172.21.15.151:0/1300264983 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb381f5670 con 0x7fcb380b3450 2024-09-16T15:56:17.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.350+0000 7fcb3ec00640 1 -- 172.21.15.151:0/1300264983 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb381f8990 con 0x7fcb380b3450 2024-09-16T15:56:17.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.351+0000 7fcb3ec00640 1 -- 172.21.15.151:0/1300264983 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb381f4d60 con 0x7fcb380b3450 2024-09-16T15:56:17.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.352+0000 7fcb35600640 1 -- 172.21.15.151:0/1300264983 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 54) ==== 104806+0+0 (secure 0 0 0) 0x7fcb2c018020 con 0x7fcb380b3450 2024-09-16T15:56:17.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.353+0000 7fcb35600640 1 --2- 172.21.15.151:0/1300264983 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcb1807af50 0x7fcb1807d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:17.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.353+0000 7fcb35600640 1 -- 172.21.15.151:0/1300264983 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5907+0+0 (secure 0 0 0) 0x7fcb2c0998f0 con 0x7fcb380b3450 2024-09-16T15:56:17.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.353+0000 7fcb3ce00640 1 --2- 172.21.15.151:0/1300264983 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcb1807af50 0x7fcb1807d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:17.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.354+0000 7fcb3ce00640 1 --2- 172.21.15.151:0/1300264983 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcb1807af50 0x7fcb1807d410 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fcb381f7d00 tx=0x7fcb24009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:17.358 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.357+0000 7fcb35600640 1 -- 172.21.15.151:0/1300264983 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fcb2c01f030 con 0x7fcb380b3450 2024-09-16T15:56:17.577 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.576+0000 7fcb3ec00640 1 -- 172.21.15.151:0/1300264983 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone58175", "target": ["mon-mgr", ""]}) -- 0x7fcb380d2330 con 0x7fcb1807af50 2024-09-16T15:56:17.588 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.587+0000 7fcb35600640 1 -- 172.21.15.151:0/1300264983 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7fcb380d2330 con 0x7fcb1807af50 2024-09-16T15:56:17.588 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-16T15:56:17.588 INFO:teuthology.orchestra.run.smithi151.stdout: "status": { 2024-09-16T15:56:17.588 INFO:teuthology.orchestra.run.smithi151.stdout: "state": "canceled", 2024-09-16T15:56:17.588 INFO:teuthology.orchestra.run.smithi151.stdout: "source": { 2024-09-16T15:56:17.588 INFO:teuthology.orchestra.run.smithi151.stdout: "volume": "cephfs", 2024-09-16T15:56:17.588 INFO:teuthology.orchestra.run.smithi151.stdout: "subvolume": "sv1", 2024-09-16T15:56:17.588 INFO:teuthology.orchestra.run.smithi151.stdout: "snapshot": "ss1" 2024-09-16T15:56:17.588 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:56:17.589 INFO:teuthology.orchestra.run.smithi151.stdout: "failure": { 2024-09-16T15:56:17.589 INFO:teuthology.orchestra.run.smithi151.stdout: "errno": "4", 2024-09-16T15:56:17.589 INFO:teuthology.orchestra.run.smithi151.stdout: "error_msg": "user interrupted clone operation" 2024-09-16T15:56:17.589 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:56:17.589 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:56:17.589 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-16T15:56:17.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.589+0000 7fcb3ec00640 1 -- 172.21.15.151:0/1300264983 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcb1807af50 msgr2=0x7fcb1807d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:17.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.589+0000 7fcb3ec00640 1 --2- 172.21.15.151:0/1300264983 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcb1807af50 0x7fcb1807d410 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fcb381f7d00 tx=0x7fcb24009040 comp rx=0 tx=0).stop 2024-09-16T15:56:17.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.589+0000 7fcb3ec00640 1 -- 172.21.15.151:0/1300264983 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb380b3450 msgr2=0x7fcb381f5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:17.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.589+0000 7fcb3ec00640 1 --2- 172.21.15.151:0/1300264983 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb380b3450 0x7fcb381f5130 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c00b3f0 tx=0x7fcb2c00b8b0 comp rx=0 tx=0).stop 2024-09-16T15:56:17.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.589+0000 7fcb3ec00640 1 -- 172.21.15.151:0/1300264983 shutdown_connections 2024-09-16T15:56:17.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.589+0000 7fcb3ec00640 1 --2- 172.21.15.151:0/1300264983 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fcb1807af50 0x7fcb1807d410 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:17.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.589+0000 7fcb3ec00640 1 --2- 172.21.15.151:0/1300264983 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb3815df60 0x7fcb381f7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:17.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.589+0000 7fcb3ec00640 1 --2- 172.21.15.151:0/1300264983 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fcb380b3890 0x7fcb3815aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:17.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.589+0000 7fcb3ec00640 1 --2- 172.21.15.151:0/1300264983 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb380b3450 0x7fcb381f5130 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:17.591 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.589+0000 7fcb3ec00640 1 -- 172.21.15.151:0/1300264983 >> 172.21.15.151:0/1300264983 conn(0x7fcb380d3d50 msgr2=0x7fcb380b8b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:17.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.589+0000 7fcb3ec00640 1 -- 172.21.15.151:0/1300264983 shutdown_connections 2024-09-16T15:56:17.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.589+0000 7fcb3ec00640 1 -- 172.21.15.151:0/1300264983 wait complete. 2024-09-16T15:56:17.606 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-16T15:56:17.606 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone67026 2024-09-16T15:56:17.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.766+0000 7f2431e00640 1 -- 172.21.15.151:0/2448194591 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f242c15df60 msgr2=0x7f242c0d0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:17.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.766+0000 7f2431e00640 1 --2- 172.21.15.151:0/2448194591 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f242c15df60 0x7f242c0d0fa0 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f242c0cf1a0 tx=0x7f242002a560 comp rx=0 tx=0).stop 2024-09-16T15:56:17.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.766+0000 7f2431e00640 1 -- 172.21.15.151:0/2448194591 shutdown_connections 2024-09-16T15:56:17.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.766+0000 7f2431e00640 1 --2- 172.21.15.151:0/2448194591 >> v2:172.21.15.167:3300/0 conn(0x7f242c0b3450 0x7f242c0b3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:17.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.766+0000 7f2431e00640 1 --2- 172.21.15.151:0/2448194591 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f242c15df60 0x7f242c0d0fa0 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:17.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.766+0000 7f2431e00640 1 -- 172.21.15.151:0/2448194591 >> 172.21.15.151:0/2448194591 conn(0x7f242c0d3d50 msgr2=0x7f242c0d4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:17.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.766+0000 7f2431e00640 1 -- 172.21.15.151:0/2448194591 shutdown_connections 2024-09-16T15:56:17.768 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.766+0000 7f2431e00640 1 -- 172.21.15.151:0/2448194591 wait complete. 2024-09-16T15:56:17.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.767+0000 7f2431e00640 1 Processor -- start 2024-09-16T15:56:17.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.768+0000 7f2431e00640 1 -- start start 2024-09-16T15:56:17.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.768+0000 7f2431e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f242c0b3450 0x7f242c1f7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:17.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.768+0000 7f2431e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f242c0cf4b0 0x7f242c1f8220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:17.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.768+0000 7f2431e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f242c15df60 0x7f242c1f46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:17.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.768+0000 7f2431e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f242c0d1a30 con 0x7f242c0cf4b0 2024-09-16T15:56:17.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.768+0000 7f2431e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f242c0d2a40 con 0x7f242c15df60 2024-09-16T15:56:17.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.768+0000 7f2431e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f242c0d2bc0 con 0x7f242c0b3450 2024-09-16T15:56:17.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.769+0000 7f242be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f242c0cf4b0 0x7f242c1f8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:17.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.769+0000 7f242be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f242c0cf4b0 0x7f242c1f8220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:59678/0 (socket says 172.21.15.151:59678) 2024-09-16T15:56:17.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.769+0000 7f242be00640 1 -- 172.21.15.151:0/3646174058 learned_addr learned my addr 172.21.15.151:0/3646174058 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:56:17.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.769+0000 7f2431400640 1 --2- 172.21.15.151:0/3646174058 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f242c15df60 0x7f242c1f46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:17.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.769+0000 7f2430a00640 1 --2- 172.21.15.151:0/3646174058 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f242c0b3450 0x7f242c1f7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:17.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.769+0000 7f242be00640 1 -- 172.21.15.151:0/3646174058 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f242c0b3450 msgr2=0x7f242c1f7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:17.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.769+0000 7f242be00640 1 --2- 172.21.15.151:0/3646174058 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f242c0b3450 0x7f242c1f7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:17.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.769+0000 7f242be00640 1 -- 172.21.15.151:0/3646174058 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f242c15df60 msgr2=0x7f242c1f46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:17.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.769+0000 7f242be00640 1 --2- 172.21.15.151:0/3646174058 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f242c15df60 0x7f242c1f46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:17.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.769+0000 7f242be00640 1 -- 172.21.15.151:0/3646174058 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f242c0b81c0 con 0x7f242c0cf4b0 2024-09-16T15:56:17.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.769+0000 7f2431400640 1 --2- 172.21.15.151:0/3646174058 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f242c15df60 0x7f242c1f46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:56:17.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.769+0000 7f242be00640 1 --2- 172.21.15.151:0/3646174058 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f242c0cf4b0 0x7f242c1f8220 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f241800d950 tx=0x7f241800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:17.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.770+0000 7f2428c00640 1 -- 172.21.15.151:0/3646174058 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2418017070 con 0x7f242c0cf4b0 2024-09-16T15:56:17.774 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.770+0000 7f2428c00640 1 -- 172.21.15.151:0/3646174058 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2418009d20 con 0x7f242c0cf4b0 2024-09-16T15:56:17.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.770+0000 7f2428c00640 1 -- 172.21.15.151:0/3646174058 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2418004280 con 0x7f242c0cf4b0 2024-09-16T15:56:17.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.770+0000 7f2431e00640 1 -- 172.21.15.151:0/3646174058 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f242c0b84a0 con 0x7f242c0cf4b0 2024-09-16T15:56:17.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.770+0000 7f2431e00640 1 -- 172.21.15.151:0/3646174058 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f242c1f5950 con 0x7f242c0cf4b0 2024-09-16T15:56:17.775 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.771+0000 7f2431e00640 1 -- 172.21.15.151:0/3646174058 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f242c0cfb10 con 0x7f242c0cf4b0 2024-09-16T15:56:17.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.772+0000 7f2428c00640 1 -- 172.21.15.151:0/3646174058 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 54) ==== 104806+0+0 (secure 0 0 0) 0x7f24180035f0 con 0x7f242c0cf4b0 2024-09-16T15:56:17.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.773+0000 7f2428c00640 1 --2- 172.21.15.151:0/3646174058 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f240c07af20 0x7f240c07d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:17.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.773+0000 7f2428c00640 1 -- 172.21.15.151:0/3646174058 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5907+0+0 (secure 0 0 0) 0x7f2418099d40 con 0x7f242c0cf4b0 2024-09-16T15:56:17.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.774+0000 7f2430a00640 1 --2- 172.21.15.151:0/3646174058 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f240c07af20 0x7f240c07d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:17.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.775+0000 7f2430a00640 1 --2- 172.21.15.151:0/3646174058 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f240c07af20 0x7f240c07d3e0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f2420033000 tx=0x7f2420004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:17.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.777+0000 7f2428c00640 1 -- 172.21.15.151:0/3646174058 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2418063eb0 con 0x7f242c0cf4b0 2024-09-16T15:56:17.999 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:17.998+0000 7f2431e00640 1 -- 172.21.15.151:0/3646174058 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone67026", "target": ["mon-mgr", ""]}) -- 0x7f242c0d16f0 con 0x7f240c07af20 2024-09-16T15:56:18.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.010+0000 7f2428c00640 1 -- 172.21.15.151:0/3646174058 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+243 (secure 0 0 0) 0x7f242c0d16f0 con 0x7f240c07af20 2024-09-16T15:56:18.011 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-16T15:56:18.011 INFO:teuthology.orchestra.run.smithi151.stdout: "status": { 2024-09-16T15:56:18.011 INFO:teuthology.orchestra.run.smithi151.stdout: "state": "canceled", 2024-09-16T15:56:18.011 INFO:teuthology.orchestra.run.smithi151.stdout: "source": { 2024-09-16T15:56:18.011 INFO:teuthology.orchestra.run.smithi151.stdout: "volume": "cephfs", 2024-09-16T15:56:18.012 INFO:teuthology.orchestra.run.smithi151.stdout: "subvolume": "sv1", 2024-09-16T15:56:18.012 INFO:teuthology.orchestra.run.smithi151.stdout: "snapshot": "ss1" 2024-09-16T15:56:18.012 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:56:18.012 INFO:teuthology.orchestra.run.smithi151.stdout: "failure": { 2024-09-16T15:56:18.012 INFO:teuthology.orchestra.run.smithi151.stdout: "errno": "4", 2024-09-16T15:56:18.012 INFO:teuthology.orchestra.run.smithi151.stdout: "error_msg": "user interrupted clone operation" 2024-09-16T15:56:18.012 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:56:18.012 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:56:18.013 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-16T15:56:18.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.012+0000 7f2431e00640 1 -- 172.21.15.151:0/3646174058 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f240c07af20 msgr2=0x7f240c07d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:18.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.012+0000 7f2431e00640 1 --2- 172.21.15.151:0/3646174058 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f240c07af20 0x7f240c07d3e0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f2420033000 tx=0x7f2420004730 comp rx=0 tx=0).stop 2024-09-16T15:56:18.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.012+0000 7f2431e00640 1 -- 172.21.15.151:0/3646174058 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f242c0cf4b0 msgr2=0x7f242c1f8220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:18.014 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.012+0000 7f2431e00640 1 --2- 172.21.15.151:0/3646174058 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f242c0cf4b0 0x7f242c1f8220 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f241800d950 tx=0x7f241800de10 comp rx=0 tx=0).stop 2024-09-16T15:56:18.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.012+0000 7f2431e00640 1 -- 172.21.15.151:0/3646174058 shutdown_connections 2024-09-16T15:56:18.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.012+0000 7f2431e00640 1 --2- 172.21.15.151:0/3646174058 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f240c07af20 0x7f240c07d3e0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:18.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.012+0000 7f2431e00640 1 --2- 172.21.15.151:0/3646174058 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f242c15df60 0x7f242c1f46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:18.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.012+0000 7f2431e00640 1 --2- 172.21.15.151:0/3646174058 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f242c0cf4b0 0x7f242c1f8220 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:18.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.012+0000 7f2431e00640 1 --2- 172.21.15.151:0/3646174058 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f242c0b3450 0x7f242c1f7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:18.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.012+0000 7f2431e00640 1 -- 172.21.15.151:0/3646174058 >> 172.21.15.151:0/3646174058 conn(0x7f242c0d3d50 msgr2=0x7f242c0b4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:18.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.012+0000 7f2431e00640 1 -- 172.21.15.151:0/3646174058 shutdown_connections 2024-09-16T15:56:18.015 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.012+0000 7f2431e00640 1 -- 172.21.15.151:0/3646174058 wait complete. 2024-09-16T15:56:18.030 DEBUG:tasks.cephfs.test_volumes:current clone state = canceled 2024-09-16T15:56:18.030 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-09-16T15:56:18.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.188+0000 7fa852a00640 1 -- 172.21.15.151:0/622243754 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa84c06f220 msgr2=0x7fa84c06f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:18.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.188+0000 7fa852a00640 1 --2- 172.21.15.151:0/622243754 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa84c06f220 0x7fa84c06f6b0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7fa84c070850 tx=0x7fa84402aa30 comp rx=0 tx=0).stop 2024-09-16T15:56:18.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.188+0000 7fa852a00640 1 -- 172.21.15.151:0/622243754 shutdown_connections 2024-09-16T15:56:18.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.188+0000 7fa852a00640 1 --2- 172.21.15.151:0/622243754 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa84c06f220 0x7fa84c06f6b0 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:18.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.188+0000 7fa852a00640 1 --2- 172.21.15.151:0/622243754 >> v2:172.21.15.167:3300/0 conn(0x7fa84c15cff0 0x7fa84c06e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:18.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.188+0000 7fa852a00640 1 -- 172.21.15.151:0/622243754 >> 172.21.15.151:0/622243754 conn(0x7fa84c0b18e0 msgr2=0x7fa84c0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:18.190 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.188+0000 7fa852a00640 1 -- 172.21.15.151:0/622243754 shutdown_connections 2024-09-16T15:56:18.191 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.188+0000 7fa852a00640 1 -- 172.21.15.151:0/622243754 wait complete. 2024-09-16T15:56:18.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.190+0000 7fa852a00640 1 Processor -- start 2024-09-16T15:56:18.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.190+0000 7fa852a00640 1 -- start start 2024-09-16T15:56:18.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.191+0000 7fa852a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa84c06f220 0x7fa84c1f6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:18.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.191+0000 7fa852a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa84c159aa0 0x7fa84c0b0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:18.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.191+0000 7fa852a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa84c15cff0 0x7fa84c1491e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:18.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.191+0000 7fa852a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa84c0cc520 con 0x7fa84c159aa0 2024-09-16T15:56:18.195 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.191+0000 7fa852a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa84c0cc3a0 con 0x7fa84c15cff0 2024-09-16T15:56:18.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.191+0000 7fa852a00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fa84c0cc6a0 con 0x7fa84c06f220 2024-09-16T15:56:18.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.191+0000 7fa843e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa84c159aa0 0x7fa84c0b0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:18.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.191+0000 7fa843e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa84c159aa0 0x7fa84c0b0490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:59684/0 (socket says 172.21.15.151:59684) 2024-09-16T15:56:18.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.191+0000 7fa843e00640 1 -- 172.21.15.151:0/2913740384 learned_addr learned my addr 172.21.15.151:0/2913740384 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:56:18.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.191+0000 7fa850a00640 1 --2- 172.21.15.151:0/2913740384 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa84c15cff0 0x7fa84c1491e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:18.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.191+0000 7fa84be00640 1 --2- 172.21.15.151:0/2913740384 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa84c06f220 0x7fa84c1f6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:18.196 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.191+0000 7fa843e00640 1 -- 172.21.15.151:0/2913740384 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa84c06f220 msgr2=0x7fa84c1f6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:18.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.191+0000 7fa843e00640 1 --2- 172.21.15.151:0/2913740384 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa84c06f220 0x7fa84c1f6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:18.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.191+0000 7fa843e00640 1 -- 172.21.15.151:0/2913740384 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa84c15cff0 msgr2=0x7fa84c1491e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:18.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.191+0000 7fa843e00640 1 --2- 172.21.15.151:0/2913740384 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa84c15cff0 0x7fa84c1491e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:18.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.191+0000 7fa843e00640 1 -- 172.21.15.151:0/2913740384 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa84c1f7070 con 0x7fa84c159aa0 2024-09-16T15:56:18.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.192+0000 7fa843e00640 1 --2- 172.21.15.151:0/2913740384 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa84c159aa0 0x7fa84c0b0490 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7fa83400c9a0 tx=0x7fa83400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:18.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.192+0000 7fa848c00640 1 -- 172.21.15.151:0/2913740384 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa834007bb0 con 0x7fa84c159aa0 2024-09-16T15:56:18.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.192+0000 7fa848c00640 1 -- 172.21.15.151:0/2913740384 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa834007e40 con 0x7fa84c159aa0 2024-09-16T15:56:18.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.192+0000 7fa848c00640 1 -- 172.21.15.151:0/2913740384 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83400f690 con 0x7fa84c159aa0 2024-09-16T15:56:18.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.192+0000 7fa852a00640 1 -- 172.21.15.151:0/2913740384 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa84c157870 con 0x7fa84c159aa0 2024-09-16T15:56:18.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.192+0000 7fa852a00640 1 -- 172.21.15.151:0/2913740384 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa84c1f6ab0 con 0x7fa84c159aa0 2024-09-16T15:56:18.198 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.193+0000 7fa852a00640 1 -- 172.21.15.151:0/2913740384 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa84c157a40 con 0x7fa84c159aa0 2024-09-16T15:56:18.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.194+0000 7fa848c00640 1 -- 172.21.15.151:0/2913740384 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 54) ==== 104806+0+0 (secure 0 0 0) 0x7fa83400f830 con 0x7fa84c159aa0 2024-09-16T15:56:18.199 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.195+0000 7fa848c00640 1 --2- 172.21.15.151:0/2913740384 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa82807af20 0x7fa82807d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:18.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.195+0000 7fa848c00640 1 -- 172.21.15.151:0/2913740384 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5907+0+0 (secure 0 0 0) 0x7fa83409a2b0 con 0x7fa84c159aa0 2024-09-16T15:56:18.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.196+0000 7fa84be00640 1 --2- 172.21.15.151:0/2913740384 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa82807af20 0x7fa82807d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:18.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.196+0000 7fa84be00640 1 --2- 172.21.15.151:0/2913740384 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa82807af20 0x7fa82807d3e0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fa83c004f60 tx=0x7fa83c007e30 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:18.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.199+0000 7fa848c00640 1 -- 172.21.15.151:0/2913740384 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa834014030 con 0x7fa84c159aa0 2024-09-16T15:56:18.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.456+0000 7fa852a00640 1 -- 172.21.15.151:0/2913740384 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fa84c069360 con 0x7fa84c159aa0 2024-09-16T15:56:18.458 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.457+0000 7fa848c00640 1 -- 172.21.15.151:0/2913740384 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1927 (secure 0 0 0) 0x7fa834063dd0 con 0x7fa84c159aa0 2024-09-16T15:56:18.458 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:56:18.459 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-16T15:56:18.459 INFO:teuthology.orchestra.run.smithi151.stdout: "fsid": "1757be18-e41a-4446-bb4f-ee4ac81c39df", 2024-09-16T15:56:18.459 INFO:teuthology.orchestra.run.smithi151.stdout: "health": { 2024-09-16T15:56:18.459 INFO:teuthology.orchestra.run.smithi151.stdout: "status": "HEALTH_OK", 2024-09-16T15:56:18.459 INFO:teuthology.orchestra.run.smithi151.stdout: "checks": {}, 2024-09-16T15:56:18.459 INFO:teuthology.orchestra.run.smithi151.stdout: "mutes": [] 2024-09-16T15:56:18.459 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:56:18.459 INFO:teuthology.orchestra.run.smithi151.stdout: "election_epoch": 4, 2024-09-16T15:56:18.459 INFO:teuthology.orchestra.run.smithi151.stdout: "quorum": [ 2024-09-16T15:56:18.460 INFO:teuthology.orchestra.run.smithi151.stdout: 0, 2024-09-16T15:56:18.460 INFO:teuthology.orchestra.run.smithi151.stdout: 1, 2024-09-16T15:56:18.460 INFO:teuthology.orchestra.run.smithi151.stdout: 2 2024-09-16T15:56:18.460 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:56:18.460 INFO:teuthology.orchestra.run.smithi151.stdout: "quorum_names": [ 2024-09-16T15:56:18.460 INFO:teuthology.orchestra.run.smithi151.stdout: "a", 2024-09-16T15:56:18.460 INFO:teuthology.orchestra.run.smithi151.stdout: "b", 2024-09-16T15:56:18.460 INFO:teuthology.orchestra.run.smithi151.stdout: "c" 2024-09-16T15:56:18.461 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:56:18.461 INFO:teuthology.orchestra.run.smithi151.stdout: "quorum_age": 1042, 2024-09-16T15:56:18.461 INFO:teuthology.orchestra.run.smithi151.stdout: "monmap": { 2024-09-16T15:56:18.461 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 1, 2024-09-16T15:56:18.461 INFO:teuthology.orchestra.run.smithi151.stdout: "min_mon_release_name": "squid", 2024-09-16T15:56:18.461 INFO:teuthology.orchestra.run.smithi151.stdout: "num_mons": 3 2024-09-16T15:56:18.461 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:56:18.461 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap": { 2024-09-16T15:56:18.462 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 147, 2024-09-16T15:56:18.462 INFO:teuthology.orchestra.run.smithi151.stdout: "num_osds": 8, 2024-09-16T15:56:18.462 INFO:teuthology.orchestra.run.smithi151.stdout: "num_up_osds": 8, 2024-09-16T15:56:18.462 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_up_since": 1726501150, 2024-09-16T15:56:18.462 INFO:teuthology.orchestra.run.smithi151.stdout: "num_in_osds": 8, 2024-09-16T15:56:18.462 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_in_since": 1726501140, 2024-09-16T15:56:18.462 INFO:teuthology.orchestra.run.smithi151.stdout: "num_remapped_pgs": 0 2024-09-16T15:56:18.462 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:56:18.462 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap": { 2024-09-16T15:56:18.462 INFO:teuthology.orchestra.run.smithi151.stdout: "pgs_by_state": [ 2024-09-16T15:56:18.463 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:56:18.463 INFO:teuthology.orchestra.run.smithi151.stdout: "state_name": "active+clean", 2024-09-16T15:56:18.463 INFO:teuthology.orchestra.run.smithi151.stdout: "count": 97 2024-09-16T15:56:18.463 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:56:18.463 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:56:18.463 INFO:teuthology.orchestra.run.smithi151.stdout: "num_pgs": 97, 2024-09-16T15:56:18.463 INFO:teuthology.orchestra.run.smithi151.stdout: "num_pools": 3, 2024-09-16T15:56:18.463 INFO:teuthology.orchestra.run.smithi151.stdout: "num_objects": 4949, 2024-09-16T15:56:18.463 INFO:teuthology.orchestra.run.smithi151.stdout: "data_bytes": 20613039654, 2024-09-16T15:56:18.464 INFO:teuthology.orchestra.run.smithi151.stdout: "bytes_used": 42371436544, 2024-09-16T15:56:18.464 INFO:teuthology.orchestra.run.smithi151.stdout: "bytes_avail": 730722676736, 2024-09-16T15:56:18.464 INFO:teuthology.orchestra.run.smithi151.stdout: "bytes_total": 773094113280, 2024-09-16T15:56:18.464 INFO:teuthology.orchestra.run.smithi151.stdout: "read_bytes_sec": 564233978, 2024-09-16T15:56:18.464 INFO:teuthology.orchestra.run.smithi151.stdout: "write_bytes_sec": 562545563, 2024-09-16T15:56:18.464 INFO:teuthology.orchestra.run.smithi151.stdout: "read_op_per_sec": 138, 2024-09-16T15:56:18.464 INFO:teuthology.orchestra.run.smithi151.stdout: "write_op_per_sec": 154 2024-09-16T15:56:18.464 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:56:18.464 INFO:teuthology.orchestra.run.smithi151.stdout: "fsmap": { 2024-09-16T15:56:18.465 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 97, 2024-09-16T15:56:18.465 INFO:teuthology.orchestra.run.smithi151.stdout: "btime": "2024-09-16T15:55:09:523485+0000", 2024-09-16T15:56:18.465 INFO:teuthology.orchestra.run.smithi151.stdout: "id": 10, 2024-09-16T15:56:18.465 INFO:teuthology.orchestra.run.smithi151.stdout: "up": 1, 2024-09-16T15:56:18.465 INFO:teuthology.orchestra.run.smithi151.stdout: "in": 1, 2024-09-16T15:56:18.465 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 1, 2024-09-16T15:56:18.465 INFO:teuthology.orchestra.run.smithi151.stdout: "by_rank": [ 2024-09-16T15:56:18.465 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:56:18.465 INFO:teuthology.orchestra.run.smithi151.stdout: "filesystem_id": 10, 2024-09-16T15:56:18.466 INFO:teuthology.orchestra.run.smithi151.stdout: "rank": 0, 2024-09-16T15:56:18.466 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "b", 2024-09-16T15:56:18.466 INFO:teuthology.orchestra.run.smithi151.stdout: "status": "up:active", 2024-09-16T15:56:18.466 INFO:teuthology.orchestra.run.smithi151.stdout: "gid": 9474 2024-09-16T15:56:18.466 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:56:18.466 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:56:18.466 INFO:teuthology.orchestra.run.smithi151.stdout: "up:standby": 3 2024-09-16T15:56:18.466 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:56:18.466 INFO:teuthology.orchestra.run.smithi151.stdout: "mgrmap": { 2024-09-16T15:56:18.466 INFO:teuthology.orchestra.run.smithi151.stdout: "available": true, 2024-09-16T15:56:18.467 INFO:teuthology.orchestra.run.smithi151.stdout: "num_standbys": 1, 2024-09-16T15:56:18.467 INFO:teuthology.orchestra.run.smithi151.stdout: "modules": [ 2024-09-16T15:56:18.467 INFO:teuthology.orchestra.run.smithi151.stdout: "iostat", 2024-09-16T15:56:18.467 INFO:teuthology.orchestra.run.smithi151.stdout: "nfs", 2024-09-16T15:56:18.467 INFO:teuthology.orchestra.run.smithi151.stdout: "restful" 2024-09-16T15:56:18.467 INFO:teuthology.orchestra.run.smithi151.stdout: ], 2024-09-16T15:56:18.467 INFO:teuthology.orchestra.run.smithi151.stdout: "services": {} 2024-09-16T15:56:18.467 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:56:18.467 INFO:teuthology.orchestra.run.smithi151.stdout: "servicemap": { 2024-09-16T15:56:18.468 INFO:teuthology.orchestra.run.smithi151.stdout: "epoch": 25, 2024-09-16T15:56:18.468 INFO:teuthology.orchestra.run.smithi151.stdout: "modified": "2024-09-16T15:56:03.995726+0000", 2024-09-16T15:56:18.468 INFO:teuthology.orchestra.run.smithi151.stdout: "services": {} 2024-09-16T15:56:18.468 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:56:18.468 INFO:teuthology.orchestra.run.smithi151.stdout: "progress_events": {} 2024-09-16T15:56:18.468 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-16T15:56:18.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.459+0000 7fa852a00640 1 -- 172.21.15.151:0/2913740384 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa82807af20 msgr2=0x7fa82807d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:18.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.459+0000 7fa852a00640 1 --2- 172.21.15.151:0/2913740384 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa82807af20 0x7fa82807d3e0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fa83c004f60 tx=0x7fa83c007e30 comp rx=0 tx=0).stop 2024-09-16T15:56:18.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.459+0000 7fa852a00640 1 -- 172.21.15.151:0/2913740384 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa84c159aa0 msgr2=0x7fa84c0b0490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:18.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.459+0000 7fa852a00640 1 --2- 172.21.15.151:0/2913740384 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa84c159aa0 0x7fa84c0b0490 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7fa83400c9a0 tx=0x7fa83400ce60 comp rx=0 tx=0).stop 2024-09-16T15:56:18.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.459+0000 7fa852a00640 1 -- 172.21.15.151:0/2913740384 shutdown_connections 2024-09-16T15:56:18.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.460+0000 7fa852a00640 1 --2- 172.21.15.151:0/2913740384 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa82807af20 0x7fa82807d3e0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:18.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.460+0000 7fa852a00640 1 --2- 172.21.15.151:0/2913740384 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa84c15cff0 0x7fa84c1491e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:18.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.460+0000 7fa852a00640 1 --2- 172.21.15.151:0/2913740384 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa84c159aa0 0x7fa84c0b0490 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:18.471 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.460+0000 7fa852a00640 1 --2- 172.21.15.151:0/2913740384 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa84c06f220 0x7fa84c1f6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:18.471 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.460+0000 7fa852a00640 1 -- 172.21.15.151:0/2913740384 >> 172.21.15.151:0/2913740384 conn(0x7fa84c0b18e0 msgr2=0x7fa84c0ccd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:18.471 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.460+0000 7fa852a00640 1 -- 172.21.15.151:0/2913740384 shutdown_connections 2024-09-16T15:56:18.471 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.460+0000 7fa852a00640 1 -- 172.21.15.151:0/2913740384 wait complete. 2024-09-16T15:56:18.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:56:18.479 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -lh volumes/_nogroup/subvol_clone62040/61d89b29-c3ef-41d1-a443-b6dc24b686d3 2024-09-16T15:56:18.479 DEBUG:teuthology.orchestra.run.smithi151:> ) 2024-09-16T15:56:18.517 INFO:teuthology.orchestra.run.smithi151.stderr:ls: cannot access 'volumes/_nogroup/subvol_clone62040/61d89b29-c3ef-41d1-a443-b6dc24b686d3': No such file or directory 2024-09-16T15:56:18.518 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-16T15:56:18.519 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-16T15:56:18.718 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.717+0000 7fb26d400640 1 -- 172.21.15.151:0/1596285201 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb24c0039e0 msgr2=0x7fb24c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:18.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.717+0000 7fb26d400640 1 --2- 172.21.15.151:0/1596285201 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb24c0039e0 0x7fb24c003dc0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7fb258009b50 tx=0x7fb24c005830 comp rx=0 tx=0).stop 2024-09-16T15:56:18.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.717+0000 7fb267200640 1 -- 172.21.15.151:0/1596285201 reap_dead start 2024-09-16T15:56:18.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.717+0000 7fb26d400640 1 -- 172.21.15.151:0/1596285201 shutdown_connections 2024-09-16T15:56:18.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.717+0000 7fb26d400640 1 -- 172.21.15.151:0/1596285201 >> 172.21.15.151:0/1596285201 conn(0x7fb2680b2230 msgr2=0x7fb2680b2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:18.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.717+0000 7fb26d400640 1 -- 172.21.15.151:0/1596285201 shutdown_connections 2024-09-16T15:56:18.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.717+0000 7fb26d400640 1 -- 172.21.15.151:0/1596285201 wait complete. 2024-09-16T15:56:18.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.718+0000 7fb26d400640 1 Processor -- start 2024-09-16T15:56:18.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.719+0000 7fb26d400640 1 -- start start 2024-09-16T15:56:18.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.719+0000 7fb26d400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb24c0039e0 0x7fb268145100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:18.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.719+0000 7fb26d400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb26814e230 0x7fb26814ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:18.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.719+0000 7fb26d400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb2680aed40 0x7fb2680af1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:18.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.719+0000 7fb26d400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb268162860 con 0x7fb24c0039e0 2024-09-16T15:56:18.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.719+0000 7fb26d400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb26815ed40 con 0x7fb26814e230 2024-09-16T15:56:18.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.719+0000 7fb26d400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fb2681629e0 con 0x7fb2680aed40 2024-09-16T15:56:18.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.720+0000 7fb266800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb24c0039e0 0x7fb268145100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:18.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.720+0000 7fb266800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb24c0039e0 0x7fb268145100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:59700/0 (socket says 172.21.15.151:59700) 2024-09-16T15:56:18.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.720+0000 7fb266800640 1 -- 172.21.15.151:0/2326017810 learned_addr learned my addr 172.21.15.151:0/2326017810 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:56:18.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.720+0000 7fb25fe00640 1 --2- 172.21.15.151:0/2326017810 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb26814e230 0x7fb26814ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:18.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.720+0000 7fb267200640 1 --2- 172.21.15.151:0/2326017810 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb2680aed40 0x7fb2680af1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:18.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.720+0000 7fb266800640 1 -- 172.21.15.151:0/2326017810 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb2680aed40 msgr2=0x7fb2680af1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:18.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.720+0000 7fb266800640 1 --2- 172.21.15.151:0/2326017810 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb2680aed40 0x7fb2680af1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:18.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.720+0000 7fb266800640 1 -- 172.21.15.151:0/2326017810 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb26814e230 msgr2=0x7fb26814ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:18.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.720+0000 7fb266800640 1 --2- 172.21.15.151:0/2326017810 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb26814e230 0x7fb26814ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:18.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.720+0000 7fb266800640 1 -- 172.21.15.151:0/2326017810 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb268145640 con 0x7fb24c0039e0 2024-09-16T15:56:18.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.720+0000 7fb25fe00640 1 --2- 172.21.15.151:0/2326017810 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb26814e230 0x7fb26814ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:56:18.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.720+0000 7fb266800640 1 --2- 172.21.15.151:0/2326017810 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb24c0039e0 0x7fb268145100 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7fb258036040 tx=0x7fb258033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:18.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.721+0000 7fb25ea00640 1 -- 172.21.15.151:0/2326017810 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb258043070 con 0x7fb24c0039e0 2024-09-16T15:56:18.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.721+0000 7fb25ea00640 1 -- 172.21.15.151:0/2326017810 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb258004320 con 0x7fb24c0039e0 2024-09-16T15:56:18.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.721+0000 7fb25ea00640 1 -- 172.21.15.151:0/2326017810 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb258038040 con 0x7fb24c0039e0 2024-09-16T15:56:18.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.721+0000 7fb26d400640 1 -- 172.21.15.151:0/2326017810 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2681508a0 con 0x7fb24c0039e0 2024-09-16T15:56:18.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.721+0000 7fb26d400640 1 -- 172.21.15.151:0/2326017810 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb268150e60 con 0x7fb24c0039e0 2024-09-16T15:56:18.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.722+0000 7fb26d400640 1 -- 172.21.15.151:0/2326017810 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb268068990 con 0x7fb24c0039e0 2024-09-16T15:56:18.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.723+0000 7fb25ea00640 1 -- 172.21.15.151:0/2326017810 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 54) ==== 104806+0+0 (secure 0 0 0) 0x7fb258046070 con 0x7fb24c0039e0 2024-09-16T15:56:18.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.724+0000 7fb25ea00640 1 --2- 172.21.15.151:0/2326017810 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb24407af50 0x7fb24407d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:18.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.724+0000 7fb25ea00640 1 -- 172.21.15.151:0/2326017810 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5907+0+0 (secure 0 0 0) 0x7fb2580ba070 con 0x7fb24c0039e0 2024-09-16T15:56:18.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.725+0000 7fb25fe00640 1 --2- 172.21.15.151:0/2326017810 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb24407af50 0x7fb24407d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:18.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.725+0000 7fb25fe00640 1 --2- 172.21.15.151:0/2326017810 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb24407af50 0x7fb24407d410 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fb268161730 tx=0x7fb250006b10 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:18.730 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.728+0000 7fb25ea00640 1 -- 172.21.15.151:0/2326017810 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb258083af0 con 0x7fb24c0039e0 2024-09-16T15:56:18.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.936+0000 7fb26d400640 1 -- 172.21.15.151:0/2326017810 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fb2680d0270 con 0x7fb24407af50 2024-09-16T15:56:18.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.938+0000 7fb25ea00640 1 -- 172.21.15.151:0/2326017810 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fb2680d0270 con 0x7fb24407af50 2024-09-16T15:56:18.939 INFO:teuthology.orchestra.run.smithi151.stdout:[ 2024-09-16T15:56:18.940 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:56:18.940 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "cephfs" 2024-09-16T15:56:18.940 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:56:18.940 INFO:teuthology.orchestra.run.smithi151.stdout:] 2024-09-16T15:56:18.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.940+0000 7fb26d400640 1 -- 172.21.15.151:0/2326017810 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb24407af50 msgr2=0x7fb24407d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:18.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.940+0000 7fb26d400640 1 --2- 172.21.15.151:0/2326017810 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb24407af50 0x7fb24407d410 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fb268161730 tx=0x7fb250006b10 comp rx=0 tx=0).stop 2024-09-16T15:56:18.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.940+0000 7fb26d400640 1 -- 172.21.15.151:0/2326017810 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb24c0039e0 msgr2=0x7fb268145100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:18.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.940+0000 7fb26d400640 1 --2- 172.21.15.151:0/2326017810 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb24c0039e0 0x7fb268145100 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7fb258036040 tx=0x7fb258033670 comp rx=0 tx=0).stop 2024-09-16T15:56:18.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.940+0000 7fb26d400640 1 -- 172.21.15.151:0/2326017810 shutdown_connections 2024-09-16T15:56:18.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.940+0000 7fb26d400640 1 --2- 172.21.15.151:0/2326017810 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb24407af50 0x7fb24407d410 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:18.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.940+0000 7fb26d400640 1 --2- 172.21.15.151:0/2326017810 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb26814e230 0x7fb26814ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:18.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.940+0000 7fb26d400640 1 --2- 172.21.15.151:0/2326017810 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb2680aed40 0x7fb2680af1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:18.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.940+0000 7fb26d400640 1 --2- 172.21.15.151:0/2326017810 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb24c0039e0 0x7fb268145100 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:18.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.940+0000 7fb26d400640 1 -- 172.21.15.151:0/2326017810 >> 172.21.15.151:0/2326017810 conn(0x7fb2680b2230 msgr2=0x7fb2680b0290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:18.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.940+0000 7fb26d400640 1 -- 172.21.15.151:0/2326017810 shutdown_connections 2024-09-16T15:56:18.943 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:18.941+0000 7fb26d400640 1 -- 172.21.15.151:0/2326017810 wait complete. 2024-09-16T15:56:18.958 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-09-16T15:56:19.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.117+0000 7fd5aca00640 1 -- 172.21.15.151:0/3476708776 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5a806efa0 msgr2=0x7fd5a80cc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:19.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.117+0000 7fd5aca00640 1 --2- 172.21.15.151:0/3476708776 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5a806efa0 0x7fd5a80cc5c0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7fd594009830 tx=0x7fd59402a480 comp rx=0 tx=0).stop 2024-09-16T15:56:19.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.117+0000 7fd5aca00640 1 -- 172.21.15.151:0/3476708776 shutdown_connections 2024-09-16T15:56:19.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.117+0000 7fd5aca00640 1 --2- 172.21.15.151:0/3476708776 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5a806efa0 0x7fd5a80cc5c0 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:19.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.117+0000 7fd5aca00640 1 --2- 172.21.15.151:0/3476708776 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5a8145750 0x7fd5a8073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:19.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.117+0000 7fd5aca00640 1 -- 172.21.15.151:0/3476708776 >> 172.21.15.151:0/3476708776 conn(0x7fd5a80b1840 msgr2=0x7fd5a80b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:19.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.117+0000 7fd5aca00640 1 -- 172.21.15.151:0/3476708776 shutdown_connections 2024-09-16T15:56:19.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.117+0000 7fd5aca00640 1 -- 172.21.15.151:0/3476708776 wait complete. 2024-09-16T15:56:19.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.119+0000 7fd5aca00640 1 Processor -- start 2024-09-16T15:56:19.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.119+0000 7fd5aca00640 1 -- start start 2024-09-16T15:56:19.122 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.120+0000 7fd5aca00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5a806efa0 0x7fd5a81f1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:19.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.120+0000 7fd5aca00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5a80ccb00 0x7fd5a81f5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:19.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.120+0000 7fd5aca00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5a8145750 0x7fd5a81f5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:19.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.120+0000 7fd5aca00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a80b26b0 con 0x7fd5a806efa0 2024-09-16T15:56:19.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.120+0000 7fd5aca00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a80cedd0 con 0x7fd5a8145750 2024-09-16T15:56:19.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.120+0000 7fd5aca00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fd5a8160100 con 0x7fd5a80ccb00 2024-09-16T15:56:19.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.120+0000 7fd5a5e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5a806efa0 0x7fd5a81f1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:19.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.120+0000 7fd5a5e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5a806efa0 0x7fd5a81f1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:59722/0 (socket says 172.21.15.151:59722) 2024-09-16T15:56:19.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.120+0000 7fd5a5e00640 1 -- 172.21.15.151:0/1194260889 learned_addr learned my addr 172.21.15.151:0/1194260889 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:56:19.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.120+0000 7fd5a5400640 1 --2- 172.21.15.151:0/1194260889 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5a80ccb00 0x7fd5a81f5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:19.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.120+0000 7fd5a6800640 1 --2- 172.21.15.151:0/1194260889 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5a8145750 0x7fd5a81f5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:19.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.121+0000 7fd5a5e00640 1 -- 172.21.15.151:0/1194260889 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5a80ccb00 msgr2=0x7fd5a81f5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:19.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.121+0000 7fd5a5e00640 1 --2- 172.21.15.151:0/1194260889 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5a80ccb00 0x7fd5a81f5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:19.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.121+0000 7fd5a5e00640 1 -- 172.21.15.151:0/1194260889 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5a8145750 msgr2=0x7fd5a81f5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:19.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.121+0000 7fd5a5e00640 1 --2- 172.21.15.151:0/1194260889 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5a8145750 0x7fd5a81f5f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:19.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.121+0000 7fd5a5e00640 1 -- 172.21.15.151:0/1194260889 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5a80aeab0 con 0x7fd5a806efa0 2024-09-16T15:56:19.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.121+0000 7fd5a5e00640 1 --2- 172.21.15.151:0/1194260889 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5a806efa0 0x7fd5a81f1ce0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7fd59800e790 tx=0x7fd59800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:19.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.121+0000 7fd59e000640 1 -- 172.21.15.151:0/1194260889 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59800eea0 con 0x7fd5a806efa0 2024-09-16T15:56:19.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.121+0000 7fd59e000640 1 -- 172.21.15.151:0/1194260889 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd59800cdb0 con 0x7fd5a806efa0 2024-09-16T15:56:19.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.121+0000 7fd59e000640 1 -- 172.21.15.151:0/1194260889 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd598010690 con 0x7fd5a806efa0 2024-09-16T15:56:19.124 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.121+0000 7fd5aca00640 1 -- 172.21.15.151:0/1194260889 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5a80aed90 con 0x7fd5a806efa0 2024-09-16T15:56:19.125 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.121+0000 7fd5aca00640 1 -- 172.21.15.151:0/1194260889 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5a80af350 con 0x7fd5a806efa0 2024-09-16T15:56:19.125 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.122+0000 7fd5aca00640 1 -- 172.21.15.151:0/1194260889 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5a8145290 con 0x7fd5a806efa0 2024-09-16T15:56:19.125 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.124+0000 7fd59e000640 1 -- 172.21.15.151:0/1194260889 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 54) ==== 104806+0+0 (secure 0 0 0) 0x7fd598010830 con 0x7fd5a806efa0 2024-09-16T15:56:19.125 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.125+0000 7fd59e000640 1 --2- 172.21.15.151:0/1194260889 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd58807af20 0x7fd58807d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:19.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.125+0000 7fd59e000640 1 -- 172.21.15.151:0/1194260889 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5907+0+0 (secure 0 0 0) 0x7fd59809a600 con 0x7fd5a806efa0 2024-09-16T15:56:19.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.125+0000 7fd5a5400640 1 --2- 172.21.15.151:0/1194260889 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd58807af20 0x7fd58807d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:19.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.126+0000 7fd5a5400640 1 --2- 172.21.15.151:0/1194260889 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd58807af20 0x7fd58807d3e0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fd594034000 tx=0x7fd594004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:19.130 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.129+0000 7fd59e000640 1 -- 172.21.15.151:0/1194260889 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd5980646d0 con 0x7fd5a806efa0 2024-09-16T15:56:19.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.340+0000 7fd5aca00640 1 -- 172.21.15.151:0/1194260889 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd5a81605b0 con 0x7fd58807af20 2024-09-16T15:56:19.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.352+0000 7fd59e000640 1 -- 172.21.15.151:0/1194260889 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+381 (secure 0 0 0) 0x7fd5a81605b0 con 0x7fd58807af20 2024-09-16T15:56:19.352 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:56:19.352 INFO:teuthology.orchestra.run.smithi151.stdout:[ 2024-09-16T15:56:19.352 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:56:19.353 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "subvol_clone48134" 2024-09-16T15:56:19.353 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:56:19.353 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:56:19.353 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "subvol_clone54262" 2024-09-16T15:56:19.353 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:56:19.353 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:56:19.353 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "subvol_clone62040" 2024-09-16T15:56:19.353 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:56:19.353 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:56:19.353 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "sv1" 2024-09-16T15:56:19.353 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:56:19.353 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:56:19.353 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "subvol_clone58175" 2024-09-16T15:56:19.354 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:56:19.354 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:56:19.354 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "subvol_clone67026" 2024-09-16T15:56:19.354 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:56:19.354 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:56:19.354 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "subvol_clone83343" 2024-09-16T15:56:19.354 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-16T15:56:19.354 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:56:19.354 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "subvol_clone21631" 2024-09-16T15:56:19.354 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:56:19.354 INFO:teuthology.orchestra.run.smithi151.stdout:] 2024-09-16T15:56:19.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.353+0000 7fd5aca00640 1 -- 172.21.15.151:0/1194260889 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd58807af20 msgr2=0x7fd58807d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:19.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.353+0000 7fd5aca00640 1 --2- 172.21.15.151:0/1194260889 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd58807af20 0x7fd58807d3e0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fd594034000 tx=0x7fd594004730 comp rx=0 tx=0).stop 2024-09-16T15:56:19.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.354+0000 7fd5aca00640 1 -- 172.21.15.151:0/1194260889 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5a806efa0 msgr2=0x7fd5a81f1ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:19.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.354+0000 7fd5aca00640 1 --2- 172.21.15.151:0/1194260889 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5a806efa0 0x7fd5a81f1ce0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7fd59800e790 tx=0x7fd59800ec50 comp rx=0 tx=0).stop 2024-09-16T15:56:19.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.354+0000 7fd5aca00640 1 -- 172.21.15.151:0/1194260889 shutdown_connections 2024-09-16T15:56:19.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.354+0000 7fd5aca00640 1 --2- 172.21.15.151:0/1194260889 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd58807af20 0x7fd58807d3e0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:19.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.354+0000 7fd5aca00640 1 --2- 172.21.15.151:0/1194260889 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5a8145750 0x7fd5a81f5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:19.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.354+0000 7fd5aca00640 1 --2- 172.21.15.151:0/1194260889 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd5a80ccb00 0x7fd5a81f5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:19.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.354+0000 7fd5aca00640 1 --2- 172.21.15.151:0/1194260889 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd5a806efa0 0x7fd5a81f1ce0 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:19.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.354+0000 7fd5aca00640 1 -- 172.21.15.151:0/1194260889 >> 172.21.15.151:0/1194260889 conn(0x7fd5a80b1840 msgr2=0x7fd5a81449b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:19.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.354+0000 7fd5aca00640 1 -- 172.21.15.151:0/1194260889 shutdown_connections 2024-09-16T15:56:19.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.354+0000 7fd5aca00640 1 -- 172.21.15.151:0/1194260889 wait complete. 2024-09-16T15:56:19.371 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone48134 2024-09-16T15:56:19.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.533+0000 7fe4b9e00640 1 -- 172.21.15.151:0/648786379 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe4b415cff0 msgr2=0x7fe4b406e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:19.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.533+0000 7fe4b9e00640 1 --2- 172.21.15.151:0/648786379 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe4b415cff0 0x7fe4b406e6f0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7fe4a0005fd0 tx=0x7fe4a002a5d0 comp rx=0 tx=0).stop 2024-09-16T15:56:19.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.533+0000 7fe4b9e00640 1 -- 172.21.15.151:0/648786379 shutdown_connections 2024-09-16T15:56:19.534 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.533+0000 7fe4b9e00640 1 --2- 172.21.15.151:0/648786379 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe4b406f220 0x7fe4b406f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:19.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.533+0000 7fe4b9e00640 1 --2- 172.21.15.151:0/648786379 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe4b415cff0 0x7fe4b406e6f0 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:19.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.533+0000 7fe4b9e00640 1 -- 172.21.15.151:0/648786379 >> 172.21.15.151:0/648786379 conn(0x7fe4b40b18e0 msgr2=0x7fe4b40b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:19.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.533+0000 7fe4b9e00640 1 -- 172.21.15.151:0/648786379 shutdown_connections 2024-09-16T15:56:19.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.533+0000 7fe4b9e00640 1 -- 172.21.15.151:0/648786379 wait complete. 2024-09-16T15:56:19.535 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.535+0000 7fe4b9e00640 1 Processor -- start 2024-09-16T15:56:19.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.535+0000 7fe4b9e00640 1 -- start start 2024-09-16T15:56:19.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.535+0000 7fe4b9e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe4b406f220 0x7fe4b40ae2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:19.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.536+0000 7fe4b9e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe4b4159aa0 0x7fe4b41f2820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:19.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.536+0000 7fe4b9e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4b415cff0 0x7fe4b41529a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:19.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.536+0000 7fe4b9e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b40cc520 con 0x7fe4b406f220 2024-09-16T15:56:19.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.536+0000 7fe4b9e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b40cc3a0 con 0x7fe4b415cff0 2024-09-16T15:56:19.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.536+0000 7fe4b9e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fe4b40cc6a0 con 0x7fe4b4159aa0 2024-09-16T15:56:19.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.536+0000 7fe4b2a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe4b406f220 0x7fe4b40ae2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:19.538 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.536+0000 7fe4b2a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe4b406f220 0x7fe4b40ae2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:59738/0 (socket says 172.21.15.151:59738) 2024-09-16T15:56:19.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.536+0000 7fe4b2a00640 1 -- 172.21.15.151:0/3747230385 learned_addr learned my addr 172.21.15.151:0/3747230385 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:56:19.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.536+0000 7fe4b2000640 1 --2- 172.21.15.151:0/3747230385 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe4b4159aa0 0x7fe4b41f2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:19.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.536+0000 7fe4b3400640 1 --2- 172.21.15.151:0/3747230385 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4b415cff0 0x7fe4b41529a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:19.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.536+0000 7fe4b2a00640 1 -- 172.21.15.151:0/3747230385 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe4b4159aa0 msgr2=0x7fe4b41f2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:19.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.536+0000 7fe4b2a00640 1 --2- 172.21.15.151:0/3747230385 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe4b4159aa0 0x7fe4b41f2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:19.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.536+0000 7fe4b2a00640 1 -- 172.21.15.151:0/3747230385 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4b415cff0 msgr2=0x7fe4b41529a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:19.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.536+0000 7fe4b2a00640 1 --2- 172.21.15.151:0/3747230385 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4b415cff0 0x7fe4b41529a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:19.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.536+0000 7fe4b2a00640 1 -- 172.21.15.151:0/3747230385 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4b40ad230 con 0x7fe4b406f220 2024-09-16T15:56:19.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.537+0000 7fe4b2a00640 1 --2- 172.21.15.151:0/3747230385 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe4b406f220 0x7fe4b40ae2d0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7fe4a400c480 tx=0x7fe4a400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:19.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.537+0000 7fe4ab400640 1 -- 172.21.15.151:0/3747230385 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a40052d0 con 0x7fe4b406f220 2024-09-16T15:56:19.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.537+0000 7fe4ab400640 1 -- 172.21.15.151:0/3747230385 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe4a4005470 con 0x7fe4b406f220 2024-09-16T15:56:19.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.537+0000 7fe4ab400640 1 -- 172.21.15.151:0/3747230385 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a4012870 con 0x7fe4b406f220 2024-09-16T15:56:19.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.537+0000 7fe4b9e00640 1 -- 172.21.15.151:0/3747230385 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4b40ad510 con 0x7fe4b406f220 2024-09-16T15:56:19.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.537+0000 7fe4b9e00640 1 -- 172.21.15.151:0/3747230385 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4b40adad0 con 0x7fe4b406f220 2024-09-16T15:56:19.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.538+0000 7fe4b9e00640 1 -- 172.21.15.151:0/3747230385 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4b40ccbc0 con 0x7fe4b406f220 2024-09-16T15:56:19.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.539+0000 7fe4ab400640 1 -- 172.21.15.151:0/3747230385 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 54) ==== 104806+0+0 (secure 0 0 0) 0x7fe4a40035f0 con 0x7fe4b406f220 2024-09-16T15:56:19.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.540+0000 7fe4ab400640 1 --2- 172.21.15.151:0/3747230385 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe49407af20 0x7fe49407d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:19.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.540+0000 7fe4ab400640 1 -- 172.21.15.151:0/3747230385 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5907+0+0 (secure 0 0 0) 0x7fe4a4099700 con 0x7fe4b406f220 2024-09-16T15:56:19.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.541+0000 7fe4b2000640 1 --2- 172.21.15.151:0/3747230385 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe49407af20 0x7fe49407d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:19.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.542+0000 7fe4b2000640 1 --2- 172.21.15.151:0/3747230385 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe49407af20 0x7fe49407d3e0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fe4a0033000 tx=0x7fe4a0004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:19.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.544+0000 7fe4ab400640 1 -- 172.21.15.151:0/3747230385 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe4a4063180 con 0x7fe4b406f220 2024-09-16T15:56:19.755 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.754+0000 7fe4b9e00640 1 -- 172.21.15.151:0/3747230385 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone48134", "target": ["mon-mgr", ""]}) -- 0x7fe4b40af6f0 con 0x7fe49407af20 2024-09-16T15:56:19.776 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.775+0000 7fe4ab400640 1 -- 172.21.15.151:0/3747230385 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe4b40af6f0 con 0x7fe49407af20 2024-09-16T15:56:19.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.777+0000 7fe4b9e00640 1 -- 172.21.15.151:0/3747230385 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe49407af20 msgr2=0x7fe49407d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:19.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.777+0000 7fe4b9e00640 1 --2- 172.21.15.151:0/3747230385 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe49407af20 0x7fe49407d3e0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fe4a0033000 tx=0x7fe4a0004730 comp rx=0 tx=0).stop 2024-09-16T15:56:19.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.777+0000 7fe4b9e00640 1 -- 172.21.15.151:0/3747230385 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe4b406f220 msgr2=0x7fe4b40ae2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:19.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.777+0000 7fe4b9e00640 1 --2- 172.21.15.151:0/3747230385 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe4b406f220 0x7fe4b40ae2d0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7fe4a400c480 tx=0x7fe4a400c940 comp rx=0 tx=0).stop 2024-09-16T15:56:19.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.777+0000 7fe4b9e00640 1 -- 172.21.15.151:0/3747230385 shutdown_connections 2024-09-16T15:56:19.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.777+0000 7fe4b9e00640 1 --2- 172.21.15.151:0/3747230385 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe49407af20 0x7fe49407d3e0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:19.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.777+0000 7fe4b9e00640 1 --2- 172.21.15.151:0/3747230385 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe4b415cff0 0x7fe4b41529a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:19.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.777+0000 7fe4b9e00640 1 --2- 172.21.15.151:0/3747230385 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe4b4159aa0 0x7fe4b41f2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:19.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.777+0000 7fe4b9e00640 1 --2- 172.21.15.151:0/3747230385 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe4b406f220 0x7fe4b40ae2d0 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:19.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.777+0000 7fe4b9e00640 1 -- 172.21.15.151:0/3747230385 >> 172.21.15.151:0/3747230385 conn(0x7fe4b40b18e0 msgr2=0x7fe4b415d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:19.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.777+0000 7fe4b9e00640 1 -- 172.21.15.151:0/3747230385 shutdown_connections 2024-09-16T15:56:19.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.777+0000 7fe4b9e00640 1 -- 172.21.15.151:0/3747230385 wait complete. 2024-09-16T15:56:19.790 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:56:19.789+0000 7f48f3a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:56:19.790 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:56:19.789+0000 7f48f3a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:56:19.790 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:56:19.789+0000 7f48f3a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:56:19.790 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:56:19.789+0000 7f48f3a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:56:19.790 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-16T15:56:19.789+0000 7f48f3a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T15:56:19.794 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone54262 2024-09-16T15:56:19.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.955+0000 7f39c4000640 1 -- 172.21.15.151:0/90693044 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39bc073de0 msgr2=0x7f39bc074240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:19.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.955+0000 7f39c4000640 1 --2- 172.21.15.151:0/90693044 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39bc073de0 0x7f39bc074240 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f39a8005fd0 tx=0x7f39a802a4a0 comp rx=0 tx=0).stop 2024-09-16T15:56:19.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.955+0000 7f39c4000640 1 -- 172.21.15.151:0/90693044 shutdown_connections 2024-09-16T15:56:19.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.955+0000 7f39c4000640 1 --2- 172.21.15.151:0/90693044 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f39bc06e6f0 0x7f39bc06eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:19.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.955+0000 7f39c4000640 1 --2- 172.21.15.151:0/90693044 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39bc073de0 0x7f39bc074240 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:19.956 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.955+0000 7f39c4000640 1 -- 172.21.15.151:0/90693044 >> 172.21.15.151:0/90693044 conn(0x7f39bc0b18e0 msgr2=0x7f39bc0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:19.957 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.955+0000 7f39c4000640 1 -- 172.21.15.151:0/90693044 shutdown_connections 2024-09-16T15:56:19.957 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.955+0000 7f39c4000640 1 -- 172.21.15.151:0/90693044 wait complete. 2024-09-16T15:56:19.957 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.956+0000 7f39c4000640 1 Processor -- start 2024-09-16T15:56:19.957 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.957+0000 7f39c4000640 1 -- start start 2024-09-16T15:56:19.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.957+0000 7f39c4000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39bc06e6f0 0x7f39bc144400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:19.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.957+0000 7f39c4000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39bc073de0 0x7f39bc0d01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:19.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.957+0000 7f39c4000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f39bc15f9a0 0x7f39bc156a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:19.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.957+0000 7f39c4000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f39bc06f0e0 con 0x7f39bc073de0 2024-09-16T15:56:19.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.957+0000 7f39c4000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f39bc0cd5e0 con 0x7f39bc06e6f0 2024-09-16T15:56:19.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.957+0000 7f39c4000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f39bc06f260 con 0x7f39bc15f9a0 2024-09-16T15:56:19.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.958+0000 7f39c0c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39bc073de0 0x7f39bc0d01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:19.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.958+0000 7f39c0c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39bc073de0 0x7f39bc0d01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:59748/0 (socket says 172.21.15.151:59748) 2024-09-16T15:56:19.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.958+0000 7f39c0c00640 1 -- 172.21.15.151:0/1953349277 learned_addr learned my addr 172.21.15.151:0/1953349277 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:56:19.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.958+0000 7f39c1600640 1 --2- 172.21.15.151:0/1953349277 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39bc06e6f0 0x7f39bc144400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:19.960 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.958+0000 7f39c2000640 1 --2- 172.21.15.151:0/1953349277 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f39bc15f9a0 0x7f39bc156a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:19.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.958+0000 7f39c0c00640 1 -- 172.21.15.151:0/1953349277 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f39bc15f9a0 msgr2=0x7f39bc156a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:19.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.958+0000 7f39c0c00640 1 --2- 172.21.15.151:0/1953349277 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f39bc15f9a0 0x7f39bc156a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:19.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.958+0000 7f39c0c00640 1 -- 172.21.15.151:0/1953349277 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39bc06e6f0 msgr2=0x7f39bc144400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:19.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.958+0000 7f39c0c00640 1 --2- 172.21.15.151:0/1953349277 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39bc06e6f0 0x7f39bc144400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:19.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.958+0000 7f39c0c00640 1 -- 172.21.15.151:0/1953349277 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39bc0cf150 con 0x7f39bc073de0 2024-09-16T15:56:19.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.958+0000 7f39c2000640 1 --2- 172.21.15.151:0/1953349277 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f39bc15f9a0 0x7f39bc156a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:56:19.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.958+0000 7f39c0c00640 1 --2- 172.21.15.151:0/1953349277 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39bc073de0 0x7f39bc0d01f0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f39a8009850 tx=0x7f39a80038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:19.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.959+0000 7f39b9600640 1 -- 172.21.15.151:0/1953349277 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a803f070 con 0x7f39bc073de0 2024-09-16T15:56:19.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.959+0000 7f39b9600640 1 -- 172.21.15.151:0/1953349277 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f39a80047a0 con 0x7f39bc073de0 2024-09-16T15:56:19.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.959+0000 7f39b9600640 1 -- 172.21.15.151:0/1953349277 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a8035030 con 0x7f39bc073de0 2024-09-16T15:56:19.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.959+0000 7f39c4000640 1 -- 172.21.15.151:0/1953349277 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39bc0cf430 con 0x7f39bc073de0 2024-09-16T15:56:19.961 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.959+0000 7f39c4000640 1 -- 172.21.15.151:0/1953349277 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39bc0cf9f0 con 0x7f39bc073de0 2024-09-16T15:56:19.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.960+0000 7f39c4000640 1 -- 172.21.15.151:0/1953349277 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39bc06eba0 con 0x7f39bc073de0 2024-09-16T15:56:19.962 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.961+0000 7f39b9600640 1 -- 172.21.15.151:0/1953349277 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 54) ==== 104806+0+0 (secure 0 0 0) 0x7f39a8042070 con 0x7f39bc073de0 2024-09-16T15:56:19.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.962+0000 7f39b9600640 1 --2- 172.21.15.151:0/1953349277 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f399c07af20 0x7f399c07d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:19.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.962+0000 7f39b9600640 1 -- 172.21.15.151:0/1953349277 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5907+0+0 (secure 0 0 0) 0x7f39a80baa20 con 0x7f39bc073de0 2024-09-16T15:56:19.963 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.962+0000 7f39c1600640 1 --2- 172.21.15.151:0/1953349277 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f399c07af20 0x7f399c07d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:19.964 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.963+0000 7f39c1600640 1 --2- 172.21.15.151:0/1953349277 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f399c07af20 0x7f399c07d3e0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f39b000b450 tx=0x7f39b000d040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:19.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:19.966+0000 7f39b9600640 1 -- 172.21.15.151:0/1953349277 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f39a80844a0 con 0x7f39bc073de0 2024-09-16T15:56:20.179 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.178+0000 7f39c4000640 1 -- 172.21.15.151:0/1953349277 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone54262", "target": ["mon-mgr", ""]}) -- 0x7f39bc0cd7b0 con 0x7f399c07af20 2024-09-16T15:56:20.200 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.199+0000 7f39b9600640 1 -- 172.21.15.151:0/1953349277 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f39bc0cd7b0 con 0x7f399c07af20 2024-09-16T15:56:20.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.201+0000 7f39c4000640 1 -- 172.21.15.151:0/1953349277 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f399c07af20 msgr2=0x7f399c07d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:20.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.201+0000 7f39c4000640 1 --2- 172.21.15.151:0/1953349277 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f399c07af20 0x7f399c07d3e0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f39b000b450 tx=0x7f39b000d040 comp rx=0 tx=0).stop 2024-09-16T15:56:20.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.201+0000 7f39c4000640 1 -- 172.21.15.151:0/1953349277 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39bc073de0 msgr2=0x7f39bc0d01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:20.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.201+0000 7f39c4000640 1 --2- 172.21.15.151:0/1953349277 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39bc073de0 0x7f39bc0d01f0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f39a8009850 tx=0x7f39a80038a0 comp rx=0 tx=0).stop 2024-09-16T15:56:20.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.201+0000 7f39c4000640 1 -- 172.21.15.151:0/1953349277 shutdown_connections 2024-09-16T15:56:20.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.201+0000 7f39c4000640 1 --2- 172.21.15.151:0/1953349277 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f399c07af20 0x7f399c07d3e0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:20.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.201+0000 7f39c4000640 1 --2- 172.21.15.151:0/1953349277 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f39bc15f9a0 0x7f39bc156a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:20.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.201+0000 7f39c4000640 1 --2- 172.21.15.151:0/1953349277 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39bc073de0 0x7f39bc0d01f0 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:20.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.201+0000 7f39c4000640 1 --2- 172.21.15.151:0/1953349277 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39bc06e6f0 0x7f39bc144400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:20.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.201+0000 7f39c4000640 1 -- 172.21.15.151:0/1953349277 >> 172.21.15.151:0/1953349277 conn(0x7f39bc0b18e0 msgr2=0x7f39bc0aed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:20.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.201+0000 7f39c4000640 1 -- 172.21.15.151:0/1953349277 shutdown_connections 2024-09-16T15:56:20.203 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.201+0000 7f39c4000640 1 -- 172.21.15.151:0/1953349277 wait complete. 2024-09-16T15:56:20.218 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone62040 2024-09-16T15:56:20.379 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.378+0000 7f358d800640 1 -- 172.21.15.151:0/3165320982 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f358815f910 msgr2=0x7f358815fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:20.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.378+0000 7f358d800640 1 --2- 172.21.15.151:0/3165320982 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f358815f910 0x7f358815fcf0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f3578005fd0 tx=0x7f357802a670 comp rx=0 tx=0).stop 2024-09-16T15:56:20.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.378+0000 7f358d800640 1 -- 172.21.15.151:0/3165320982 shutdown_connections 2024-09-16T15:56:20.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.378+0000 7f358d800640 1 --2- 172.21.15.151:0/3165320982 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f358815f910 0x7f358815fcf0 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:20.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.378+0000 7f358d800640 1 -- 172.21.15.151:0/3165320982 >> 172.21.15.151:0/3165320982 conn(0x7f35880b18e0 msgr2=0x7f35880b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:20.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.378+0000 7f358d800640 1 -- 172.21.15.151:0/3165320982 shutdown_connections 2024-09-16T15:56:20.380 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.378+0000 7f358d800640 1 -- 172.21.15.151:0/3165320982 wait complete. 2024-09-16T15:56:20.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.380+0000 7f358d800640 1 Processor -- start 2024-09-16T15:56:20.381 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.380+0000 7f358d800640 1 -- start start 2024-09-16T15:56:20.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.381+0000 7f358d800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f358806e6f0 0x7f3588159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:20.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.381+0000 7f358d800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f358815f910 0x7f35881f9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:20.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.381+0000 7f358d800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3588166030 0x7f358814bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:20.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.381+0000 7f358d800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f358806f430 con 0x7f3588166030 2024-09-16T15:56:20.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.381+0000 7f358d800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f358806f2b0 con 0x7f358806e6f0 2024-09-16T15:56:20.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.381+0000 7f358d800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f358806f5b0 con 0x7f358815f910 2024-09-16T15:56:20.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.381+0000 7f3587400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3588166030 0x7f358814bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:20.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.381+0000 7f3587400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3588166030 0x7f358814bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:59762/0 (socket says 172.21.15.151:59762) 2024-09-16T15:56:20.383 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.381+0000 7f3587400640 1 -- 172.21.15.151:0/4125696285 learned_addr learned my addr 172.21.15.151:0/4125696285 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:56:20.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.381+0000 7f3586a00640 1 --2- 172.21.15.151:0/4125696285 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f358806e6f0 0x7f3588159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:20.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.381+0000 7f3586000640 1 --2- 172.21.15.151:0/4125696285 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f358815f910 0x7f35881f9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:20.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.381+0000 7f3587400640 1 -- 172.21.15.151:0/4125696285 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f358815f910 msgr2=0x7f35881f9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:20.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.381+0000 7f3587400640 1 --2- 172.21.15.151:0/4125696285 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f358815f910 0x7f35881f9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:20.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.381+0000 7f3587400640 1 -- 172.21.15.151:0/4125696285 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f358806e6f0 msgr2=0x7f3588159ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:20.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.381+0000 7f3587400640 1 --2- 172.21.15.151:0/4125696285 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f358806e6f0 0x7f3588159ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:20.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.382+0000 7f3587400640 1 -- 172.21.15.151:0/4125696285 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35881665d0 con 0x7f3588166030 2024-09-16T15:56:20.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.382+0000 7f3586000640 1 --2- 172.21.15.151:0/4125696285 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f358815f910 0x7f35881f9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:56:20.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.382+0000 7f3587400640 1 --2- 172.21.15.151:0/4125696285 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3588166030 0x7f358814bdc0 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f358000c740 tx=0x7f358000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:20.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.382+0000 7f357ea00640 1 -- 172.21.15.151:0/4125696285 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3580016020 con 0x7f3588166030 2024-09-16T15:56:20.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.382+0000 7f357ea00640 1 -- 172.21.15.151:0/4125696285 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f358000ce50 con 0x7f3588166030 2024-09-16T15:56:20.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.382+0000 7f358d800640 1 -- 172.21.15.151:0/4125696285 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35881668b0 con 0x7f3588166030 2024-09-16T15:56:20.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.382+0000 7f357ea00640 1 -- 172.21.15.151:0/4125696285 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3580003bf0 con 0x7f3588166030 2024-09-16T15:56:20.384 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.382+0000 7f358d800640 1 -- 172.21.15.151:0/4125696285 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35881f9980 con 0x7f3588166030 2024-09-16T15:56:20.385 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.383+0000 7f358d800640 1 -- 172.21.15.151:0/4125696285 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3588069360 con 0x7f3588166030 2024-09-16T15:56:20.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.384+0000 7f357ea00640 1 -- 172.21.15.151:0/4125696285 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 54) ==== 104806+0+0 (secure 0 0 0) 0x7f358000a850 con 0x7f3588166030 2024-09-16T15:56:20.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.385+0000 7f357ea00640 1 --2- 172.21.15.151:0/4125696285 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f356807af20 0x7f356807d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:20.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.386+0000 7f357ea00640 1 -- 172.21.15.151:0/4125696285 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5907+0+0 (secure 0 0 0) 0x7f3580099810 con 0x7f3588166030 2024-09-16T15:56:20.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.386+0000 7f3586a00640 1 --2- 172.21.15.151:0/4125696285 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f356807af20 0x7f356807d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:20.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.386+0000 7f3586a00640 1 --2- 172.21.15.151:0/4125696285 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f356807af20 0x7f356807d3e0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f3578033000 tx=0x7f3578004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:20.390 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.390+0000 7f357ea00640 1 -- 172.21.15.151:0/4125696285 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f3580020020 con 0x7f3588166030 2024-09-16T15:56:20.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.601+0000 7f358d800640 1 -- 172.21.15.151:0/4125696285 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone62040", "target": ["mon-mgr", ""]}) -- 0x7f3588072ee0 con 0x7f356807af20 2024-09-16T15:56:20.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.622+0000 7f357ea00640 1 -- 172.21.15.151:0/4125696285 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3588072ee0 con 0x7f356807af20 2024-09-16T15:56:20.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.624+0000 7f358d800640 1 -- 172.21.15.151:0/4125696285 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f356807af20 msgr2=0x7f356807d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:20.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.624+0000 7f358d800640 1 --2- 172.21.15.151:0/4125696285 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f356807af20 0x7f356807d3e0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f3578033000 tx=0x7f3578004730 comp rx=0 tx=0).stop 2024-09-16T15:56:20.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.624+0000 7f358d800640 1 -- 172.21.15.151:0/4125696285 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3588166030 msgr2=0x7f358814bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:20.625 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.624+0000 7f358d800640 1 --2- 172.21.15.151:0/4125696285 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3588166030 0x7f358814bdc0 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f358000c740 tx=0x7f358000cc00 comp rx=0 tx=0).stop 2024-09-16T15:56:20.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.624+0000 7f358d800640 1 -- 172.21.15.151:0/4125696285 shutdown_connections 2024-09-16T15:56:20.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.624+0000 7f358d800640 1 --2- 172.21.15.151:0/4125696285 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f356807af20 0x7f356807d3e0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:20.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.624+0000 7f358d800640 1 --2- 172.21.15.151:0/4125696285 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3588166030 0x7f358814bdc0 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:20.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.624+0000 7f358d800640 1 --2- 172.21.15.151:0/4125696285 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f358815f910 0x7f35881f9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:20.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.624+0000 7f358d800640 1 --2- 172.21.15.151:0/4125696285 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f358806e6f0 0x7f3588159ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:20.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.624+0000 7f358d800640 1 -- 172.21.15.151:0/4125696285 >> 172.21.15.151:0/4125696285 conn(0x7f35880b18e0 msgr2=0x7f35880b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:20.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.624+0000 7f358d800640 1 -- 172.21.15.151:0/4125696285 shutdown_connections 2024-09-16T15:56:20.626 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.624+0000 7f358d800640 1 -- 172.21.15.151:0/4125696285 wait complete. 2024-09-16T15:56:20.642 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-09-16T15:56:20.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.807+0000 7f5538000640 1 -- 172.21.15.151:0/717749301 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5530164d20 msgr2=0x7f5530167110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:20.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.807+0000 7f5538000640 1 --2- 172.21.15.151:0/717749301 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5530164d20 0x7f5530167110 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f5528005fd0 tx=0x7f552802a8b0 comp rx=0 tx=0).stop 2024-09-16T15:56:20.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.807+0000 7f5538000640 1 -- 172.21.15.151:0/717749301 shutdown_connections 2024-09-16T15:56:20.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.807+0000 7f5538000640 1 --2- 172.21.15.151:0/717749301 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5530164d20 0x7f5530167110 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:20.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.807+0000 7f5538000640 1 -- 172.21.15.151:0/717749301 >> 172.21.15.151:0/717749301 conn(0x7f55300b18e0 msgr2=0x7f55300b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:20.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.807+0000 7f5538000640 1 -- 172.21.15.151:0/717749301 shutdown_connections 2024-09-16T15:56:20.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.807+0000 7f5538000640 1 -- 172.21.15.151:0/717749301 wait complete. 2024-09-16T15:56:20.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.809+0000 7f5538000640 1 Processor -- start 2024-09-16T15:56:20.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.809+0000 7f5538000640 1 -- start start 2024-09-16T15:56:20.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.810+0000 7f5538000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5530073de0 0x7f5530144fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:20.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.810+0000 7f5538000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f553015f9a0 0x7f5530147710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:20.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.810+0000 7f5538000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5530164d20 0x7f5530147c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:20.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.810+0000 7f5538000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5530167650 con 0x7f5530164d20 2024-09-16T15:56:20.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.810+0000 7f5538000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f55300cd5e0 con 0x7f553015f9a0 2024-09-16T15:56:20.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.810+0000 7f5538000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f55301677d0 con 0x7f5530073de0 2024-09-16T15:56:20.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.810+0000 7f5536000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5530164d20 0x7f5530147c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:20.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.810+0000 7f5536000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5530164d20 0x7f5530147c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:59778/0 (socket says 172.21.15.151:59778) 2024-09-16T15:56:20.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.810+0000 7f5536000640 1 -- 172.21.15.151:0/1128945778 learned_addr learned my addr 172.21.15.151:0/1128945778 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:56:20.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.810+0000 7f5534c00640 1 --2- 172.21.15.151:0/1128945778 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f553015f9a0 0x7f5530147710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:20.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.810+0000 7f5535600640 1 --2- 172.21.15.151:0/1128945778 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5530073de0 0x7f5530144fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:20.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.811+0000 7f5536000640 1 -- 172.21.15.151:0/1128945778 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5530073de0 msgr2=0x7f5530144fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:20.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.811+0000 7f5536000640 1 --2- 172.21.15.151:0/1128945778 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5530073de0 0x7f5530144fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:20.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.811+0000 7f5536000640 1 -- 172.21.15.151:0/1128945778 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f553015f9a0 msgr2=0x7f5530147710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:20.812 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.811+0000 7f5536000640 1 --2- 172.21.15.151:0/1128945778 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f553015f9a0 0x7f5530147710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:20.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.811+0000 7f5536000640 1 -- 172.21.15.151:0/1128945778 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55300adc90 con 0x7f5530164d20 2024-09-16T15:56:20.813 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.811+0000 7f5536000640 1 --2- 172.21.15.151:0/1128945778 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5530164d20 0x7f5530147c50 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f5528009cd0 tx=0x7f5528003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:20.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.817+0000 7f552d600640 1 -- 172.21.15.151:0/1128945778 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5528003e10 con 0x7f5530164d20 2024-09-16T15:56:20.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.817+0000 7f552d600640 1 -- 172.21.15.151:0/1128945778 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5528031c90 con 0x7f5530164d20 2024-09-16T15:56:20.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.817+0000 7f5538000640 1 -- 172.21.15.151:0/1128945778 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55300adf70 con 0x7f5530164d20 2024-09-16T15:56:20.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.817+0000 7f552d600640 1 -- 172.21.15.151:0/1128945778 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5528031e60 con 0x7f5530164d20 2024-09-16T15:56:20.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.817+0000 7f5538000640 1 -- 172.21.15.151:0/1128945778 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55300ae530 con 0x7f5530164d20 2024-09-16T15:56:20.819 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.819+0000 7f5538000640 1 -- 172.21.15.151:0/1128945778 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5530069360 con 0x7f5530164d20 2024-09-16T15:56:20.820 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.820+0000 7f552d600640 1 -- 172.21.15.151:0/1128945778 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 55) ==== 104857+0+0 (secure 0 0 0) 0x7f5528033330 con 0x7f5530164d20 2024-09-16T15:56:20.821 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.821+0000 7f552d600640 1 --2- 172.21.15.151:0/1128945778 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f551007afe0 0x7f551007d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:20.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.821+0000 7f552d600640 1 -- 172.21.15.151:0/1128945778 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5907+0+0 (secure 0 0 0) 0x7f55280bb2b0 con 0x7f5530164d20 2024-09-16T15:56:20.822 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.821+0000 7f5535600640 1 --2- 172.21.15.151:0/1128945778 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f551007afe0 0x7f551007d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:20.823 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.822+0000 7f5535600640 1 --2- 172.21.15.151:0/1128945778 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f551007afe0 0x7f551007d4a0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f55240046b0 tx=0x7f5524009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:20.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:20.825+0000 7f552d600640 1 -- 172.21.15.151:0/1128945778 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f552803f030 con 0x7f5530164d20 2024-09-16T15:56:21.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.038+0000 7f5538000640 1 -- 172.21.15.151:0/1128945778 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f55300cd7b0 con 0x7f551007afe0 2024-09-16T15:56:21.054 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.053+0000 7f552d600640 1 -- 172.21.15.151:0/1128945778 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f55300cd7b0 con 0x7f551007afe0 2024-09-16T15:56:21.054 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:56:21.054 INFO:teuthology.orchestra.run.smithi151.stdout:[ 2024-09-16T15:56:21.054 INFO:teuthology.orchestra.run.smithi151.stdout: { 2024-09-16T15:56:21.054 INFO:teuthology.orchestra.run.smithi151.stdout: "name": "ss1" 2024-09-16T15:56:21.055 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-16T15:56:21.055 INFO:teuthology.orchestra.run.smithi151.stdout:] 2024-09-16T15:56:21.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.056+0000 7f5538000640 1 -- 172.21.15.151:0/1128945778 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f551007afe0 msgr2=0x7f551007d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:21.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.056+0000 7f5538000640 1 --2- 172.21.15.151:0/1128945778 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f551007afe0 0x7f551007d4a0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f55240046b0 tx=0x7f5524009040 comp rx=0 tx=0).stop 2024-09-16T15:56:21.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.056+0000 7f5538000640 1 -- 172.21.15.151:0/1128945778 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5530164d20 msgr2=0x7f5530147c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:21.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.056+0000 7f5538000640 1 --2- 172.21.15.151:0/1128945778 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5530164d20 0x7f5530147c50 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f5528009cd0 tx=0x7f5528003c20 comp rx=0 tx=0).stop 2024-09-16T15:56:21.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.056+0000 7f5538000640 1 -- 172.21.15.151:0/1128945778 shutdown_connections 2024-09-16T15:56:21.058 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.056+0000 7f5538000640 1 --2- 172.21.15.151:0/1128945778 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f551007afe0 0x7f551007d4a0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:21.058 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.056+0000 7f5538000640 1 --2- 172.21.15.151:0/1128945778 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f553015f9a0 0x7f5530147710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:21.058 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.056+0000 7f5538000640 1 --2- 172.21.15.151:0/1128945778 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5530164d20 0x7f5530147c50 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:21.058 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.056+0000 7f5538000640 1 --2- 172.21.15.151:0/1128945778 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5530073de0 0x7f5530144fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:21.058 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.056+0000 7f5538000640 1 -- 172.21.15.151:0/1128945778 >> 172.21.15.151:0/1128945778 conn(0x7f55300b18e0 msgr2=0x7f55301666c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:21.058 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.056+0000 7f5538000640 1 -- 172.21.15.151:0/1128945778 shutdown_connections 2024-09-16T15:56:21.058 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.056+0000 7f5538000640 1 -- 172.21.15.151:0/1128945778 wait complete. 2024-09-16T15:56:21.073 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-09-16T15:56:21.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.236+0000 7f7e8ae00640 1 -- 172.21.15.151:0/3712290870 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7e84145750 msgr2=0x7f7e84073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:21.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.236+0000 7f7e8ae00640 1 --2- 172.21.15.151:0/3712290870 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7e84145750 0x7f7e84073440 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7e74005fd0 tx=0x7f7e7402a540 comp rx=0 tx=0).stop 2024-09-16T15:56:21.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.237+0000 7f7e8ae00640 1 -- 172.21.15.151:0/3712290870 shutdown_connections 2024-09-16T15:56:21.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.237+0000 7f7e8ae00640 1 --2- 172.21.15.151:0/3712290870 >> v2:172.21.15.167:3300/0 conn(0x7f7e8406efa0 0x7f7e840cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:21.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.237+0000 7f7e8ae00640 1 --2- 172.21.15.151:0/3712290870 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7e84145750 0x7f7e84073440 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:21.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.237+0000 7f7e8ae00640 1 -- 172.21.15.151:0/3712290870 >> 172.21.15.151:0/3712290870 conn(0x7f7e840b1840 msgr2=0x7f7e840b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:21.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.237+0000 7f7e8ae00640 1 -- 172.21.15.151:0/3712290870 shutdown_connections 2024-09-16T15:56:21.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.237+0000 7f7e8ae00640 1 -- 172.21.15.151:0/3712290870 wait complete. 2024-09-16T15:56:21.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.238+0000 7f7e8ae00640 1 Processor -- start 2024-09-16T15:56:21.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.238+0000 7f7e8ae00640 1 -- start start 2024-09-16T15:56:21.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.239+0000 7f7e8ae00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7e8406efa0 0x7f7e841f49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:21.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.239+0000 7f7e8ae00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7e840ccb00 0x7f7e841435a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:21.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.239+0000 7f7e8ae00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e84145750 0x7f7e840cada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:21.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.239+0000 7f7e8ae00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7e840b26b0 con 0x7f7e84145750 2024-09-16T15:56:21.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.239+0000 7f7e8ae00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7e840cedd0 con 0x7f7e8406efa0 2024-09-16T15:56:21.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.239+0000 7f7e8ae00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f7e84160100 con 0x7f7e840ccb00 2024-09-16T15:56:21.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.240+0000 7f7e88e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e84145750 0x7f7e840cada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:21.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.240+0000 7f7e88e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e84145750 0x7f7e840cada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:59794/0 (socket says 172.21.15.151:59794) 2024-09-16T15:56:21.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.240+0000 7f7e88e00640 1 -- 172.21.15.151:0/39041965 learned_addr learned my addr 172.21.15.151:0/39041965 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:56:21.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.240+0000 7f7e7b400640 1 --2- 172.21.15.151:0/39041965 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7e840ccb00 0x7f7e841435a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:21.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.240+0000 7f7e83e00640 1 --2- 172.21.15.151:0/39041965 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7e8406efa0 0x7f7e841f49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:21.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.240+0000 7f7e88e00640 1 -- 172.21.15.151:0/39041965 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7e840ccb00 msgr2=0x7f7e841435a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:21.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.240+0000 7f7e88e00640 1 --2- 172.21.15.151:0/39041965 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7e840ccb00 0x7f7e841435a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:21.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.240+0000 7f7e88e00640 1 -- 172.21.15.151:0/39041965 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7e8406efa0 msgr2=0x7f7e841f49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:21.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.240+0000 7f7e88e00640 1 --2- 172.21.15.151:0/39041965 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7e8406efa0 0x7f7e841f49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:21.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.240+0000 7f7e88e00640 1 -- 172.21.15.151:0/39041965 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e840c9da0 con 0x7f7e84145750 2024-09-16T15:56:21.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.241+0000 7f7e88e00640 1 --2- 172.21.15.151:0/39041965 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e84145750 0x7f7e840cada0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c00c7a0 tx=0x7f7e7c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:21.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.241+0000 7f7e80c00640 1 -- 172.21.15.151:0/39041965 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7c003b70 con 0x7f7e84145750 2024-09-16T15:56:21.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.241+0000 7f7e80c00640 1 -- 172.21.15.151:0/39041965 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7e7c003d10 con 0x7f7e84145750 2024-09-16T15:56:21.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.241+0000 7f7e80c00640 1 -- 172.21.15.151:0/39041965 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7c00aab0 con 0x7f7e84145750 2024-09-16T15:56:21.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.241+0000 7f7e8ae00640 1 -- 172.21.15.151:0/39041965 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e840b0d20 con 0x7f7e84145750 2024-09-16T15:56:21.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.241+0000 7f7e8ae00640 1 -- 172.21.15.151:0/39041965 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e840b2e90 con 0x7f7e84145750 2024-09-16T15:56:21.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.242+0000 7f7e8ae00640 1 -- 172.21.15.151:0/39041965 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e841605b0 con 0x7f7e84145750 2024-09-16T15:56:21.244 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.243+0000 7f7e80c00640 1 -- 172.21.15.151:0/39041965 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 55) ==== 104857+0+0 (secure 0 0 0) 0x7f7e7c00ac50 con 0x7f7e84145750 2024-09-16T15:56:21.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.244+0000 7f7e80c00640 1 --2- 172.21.15.151:0/39041965 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7e6007afe0 0x7f7e6007d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:21.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.244+0000 7f7e80c00640 1 -- 172.21.15.151:0/39041965 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5907+0+0 (secure 0 0 0) 0x7f7e7c099a30 con 0x7f7e84145750 2024-09-16T15:56:21.245 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.245+0000 7f7e83e00640 1 --2- 172.21.15.151:0/39041965 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7e6007afe0 0x7f7e6007d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:21.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.245+0000 7f7e83e00640 1 --2- 172.21.15.151:0/39041965 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7e6007afe0 0x7f7e6007d4a0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f7e74033000 tx=0x7f7e74004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:21.249 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.248+0000 7f7e80c00640 1 -- 172.21.15.151:0/39041965 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7e7c0634b0 con 0x7f7e84145750 2024-09-16T15:56:21.467 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.466+0000 7f7e8ae00640 1 -- 172.21.15.151:0/39041965 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7e8415fec0 con 0x7f7e6007afe0 2024-09-16T15:56:21.498 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.498+0000 7f7e80c00640 1 -- 172.21.15.151:0/39041965 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7e8415fec0 con 0x7f7e6007afe0 2024-09-16T15:56:21.498 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:56:21.501 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.500+0000 7f7e8ae00640 1 -- 172.21.15.151:0/39041965 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7e6007afe0 msgr2=0x7f7e6007d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:21.501 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.500+0000 7f7e8ae00640 1 --2- 172.21.15.151:0/39041965 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7e6007afe0 0x7f7e6007d4a0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f7e74033000 tx=0x7f7e74004730 comp rx=0 tx=0).stop 2024-09-16T15:56:21.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.500+0000 7f7e8ae00640 1 -- 172.21.15.151:0/39041965 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e84145750 msgr2=0x7f7e840cada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:21.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.500+0000 7f7e8ae00640 1 --2- 172.21.15.151:0/39041965 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e84145750 0x7f7e840cada0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c00c7a0 tx=0x7f7e7c00cc60 comp rx=0 tx=0).stop 2024-09-16T15:56:21.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.500+0000 7f7e8ae00640 1 -- 172.21.15.151:0/39041965 shutdown_connections 2024-09-16T15:56:21.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.500+0000 7f7e8ae00640 1 --2- 172.21.15.151:0/39041965 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7e6007afe0 0x7f7e6007d4a0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:21.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.500+0000 7f7e8ae00640 1 --2- 172.21.15.151:0/39041965 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7e84145750 0x7f7e840cada0 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:21.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.500+0000 7f7e8ae00640 1 --2- 172.21.15.151:0/39041965 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7e840ccb00 0x7f7e841435a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:21.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.500+0000 7f7e8ae00640 1 --2- 172.21.15.151:0/39041965 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7e8406efa0 0x7f7e841f49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:21.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.500+0000 7f7e8ae00640 1 -- 172.21.15.151:0/39041965 >> 172.21.15.151:0/39041965 conn(0x7f7e840b1840 msgr2=0x7f7e840cfbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:21.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.500+0000 7f7e8ae00640 1 -- 172.21.15.151:0/39041965 shutdown_connections 2024-09-16T15:56:21.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.500+0000 7f7e8ae00640 1 -- 172.21.15.151:0/39041965 wait complete. 2024-09-16T15:56:21.517 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-09-16T15:56:21.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.678+0000 7fdb1ac00640 1 -- 172.21.15.151:0/2372922481 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb14161460 msgr2=0x7fdb140d22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:21.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.678+0000 7fdb1ac00640 1 --2- 172.21.15.151:0/2372922481 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb14161460 0x7fdb140d22f0 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7fdb140ce5f0 tx=0x7fdb0802a4a0 comp rx=0 tx=0).stop 2024-09-16T15:56:21.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.678+0000 7fdb1ac00640 1 -- 172.21.15.151:0/2372922481 shutdown_connections 2024-09-16T15:56:21.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.678+0000 7fdb1ac00640 1 --2- 172.21.15.151:0/2372922481 >> v2:172.21.15.167:3300/0 conn(0x7fdb140b43c0 0x7fdb140d2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:21.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.678+0000 7fdb1ac00640 1 --2- 172.21.15.151:0/2372922481 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb14161460 0x7fdb140d22f0 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:21.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.678+0000 7fdb1ac00640 1 -- 172.21.15.151:0/2372922481 >> 172.21.15.151:0/2372922481 conn(0x7fdb1409d0b0 msgr2=0x7fdb140d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:21.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.678+0000 7fdb1ac00640 1 -- 172.21.15.151:0/2372922481 shutdown_connections 2024-09-16T15:56:21.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.678+0000 7fdb1ac00640 1 -- 172.21.15.151:0/2372922481 wait complete. 2024-09-16T15:56:21.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.680+0000 7fdb1ac00640 1 Processor -- start 2024-09-16T15:56:21.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.680+0000 7fdb1ac00640 1 -- start start 2024-09-16T15:56:21.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.681+0000 7fdb1ac00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb140b43c0 0x7fdb140b7650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:21.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.681+0000 7fdb1ac00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb140b5480 0x7fdb14157000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:21.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.681+0000 7fdb1ac00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb14161460 0x7fdb141e81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:21.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.681+0000 7fdb1ac00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdb140cef30 con 0x7fdb140b5480 2024-09-16T15:56:21.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.681+0000 7fdb1ac00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdb140b5150 con 0x7fdb140b43c0 2024-09-16T15:56:21.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.681+0000 7fdb1ac00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fdb140cf0b0 con 0x7fdb14161460 2024-09-16T15:56:21.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.681+0000 7fdb18e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb140b5480 0x7fdb14157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:21.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.681+0000 7fdb18e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb140b5480 0x7fdb14157000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:59810/0 (socket says 172.21.15.151:59810) 2024-09-16T15:56:21.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.681+0000 7fdb18e00640 1 -- 172.21.15.151:0/2719166732 learned_addr learned my addr 172.21.15.151:0/2719166732 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:56:21.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.681+0000 7fdb19800640 1 --2- 172.21.15.151:0/2719166732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb140b43c0 0x7fdb140b7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:21.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.681+0000 7fdb1a200640 1 --2- 172.21.15.151:0/2719166732 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb14161460 0x7fdb141e81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:21.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.681+0000 7fdb18e00640 1 -- 172.21.15.151:0/2719166732 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb14161460 msgr2=0x7fdb141e81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:21.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.681+0000 7fdb18e00640 1 --2- 172.21.15.151:0/2719166732 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb14161460 0x7fdb141e81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:21.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.681+0000 7fdb18e00640 1 -- 172.21.15.151:0/2719166732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb140b43c0 msgr2=0x7fdb140b7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:21.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.681+0000 7fdb18e00640 1 --2- 172.21.15.151:0/2719166732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb140b43c0 0x7fdb140b7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:21.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.681+0000 7fdb18e00640 1 -- 172.21.15.151:0/2719166732 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb1415af70 con 0x7fdb140b5480 2024-09-16T15:56:21.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.681+0000 7fdb19800640 1 --2- 172.21.15.151:0/2719166732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb140b43c0 0x7fdb140b7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:56:21.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.682+0000 7fdb18e00640 1 --2- 172.21.15.151:0/2719166732 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb140b5480 0x7fdb14157000 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7fdb0000c460 tx=0x7fdb0000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:21.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.682+0000 7fdb11600640 1 -- 172.21.15.151:0/2719166732 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb00016020 con 0x7fdb140b5480 2024-09-16T15:56:21.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.682+0000 7fdb11600640 1 -- 172.21.15.151:0/2719166732 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb00012920 con 0x7fdb140b5480 2024-09-16T15:56:21.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.682+0000 7fdb11600640 1 -- 172.21.15.151:0/2719166732 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb00012d50 con 0x7fdb140b5480 2024-09-16T15:56:21.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.682+0000 7fdb1ac00640 1 -- 172.21.15.151:0/2719166732 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb140b7b90 con 0x7fdb140b5480 2024-09-16T15:56:21.685 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.682+0000 7fdb1ac00640 1 -- 172.21.15.151:0/2719166732 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb14158ab0 con 0x7fdb140b5480 2024-09-16T15:56:21.685 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.683+0000 7fdb1ac00640 1 -- 172.21.15.151:0/2719166732 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb140cf590 con 0x7fdb140b5480 2024-09-16T15:56:21.685 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.684+0000 7fdb11600640 1 -- 172.21.15.151:0/2719166732 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 55) ==== 104857+0+0 (secure 0 0 0) 0x7fdb00019020 con 0x7fdb140b5480 2024-09-16T15:56:21.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.685+0000 7fdb11600640 1 --2- 172.21.15.151:0/2719166732 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdaf407afb0 0x7fdaf407d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:21.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.685+0000 7fdb11600640 1 -- 172.21.15.151:0/2719166732 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5907+0+0 (secure 0 0 0) 0x7fdb00099910 con 0x7fdb140b5480 2024-09-16T15:56:21.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.686+0000 7fdb19800640 1 --2- 172.21.15.151:0/2719166732 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdaf407afb0 0x7fdaf407d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:21.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.687+0000 7fdb19800640 1 --2- 172.21.15.151:0/2719166732 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdaf407afb0 0x7fdaf407d470 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fdb08034000 tx=0x7fdb08004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:21.690 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.689+0000 7fdb11600640 1 -- 172.21.15.151:0/2719166732 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fdb00020030 con 0x7fdb140b5480 2024-09-16T15:56:21.902 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.901+0000 7fdb1ac00640 1 -- 172.21.15.151:0/2719166732 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) -- 0x7fdb14159590 con 0x7fdaf407afb0 2024-09-16T15:56:21.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.933+0000 7fdb11600640 1 -- 172.21.15.151:0/2719166732 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fdb08004280 con 0x7fdaf407afb0 2024-09-16T15:56:21.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.935+0000 7fdb1ac00640 1 -- 172.21.15.151:0/2719166732 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdaf407afb0 msgr2=0x7fdaf407d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:21.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.935+0000 7fdb1ac00640 1 --2- 172.21.15.151:0/2719166732 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdaf407afb0 0x7fdaf407d470 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fdb08034000 tx=0x7fdb08004730 comp rx=0 tx=0).stop 2024-09-16T15:56:21.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.935+0000 7fdb1ac00640 1 -- 172.21.15.151:0/2719166732 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb140b5480 msgr2=0x7fdb14157000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:21.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.935+0000 7fdb1ac00640 1 --2- 172.21.15.151:0/2719166732 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb140b5480 0x7fdb14157000 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7fdb0000c460 tx=0x7fdb0000c920 comp rx=0 tx=0).stop 2024-09-16T15:56:21.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.936+0000 7fdb1ac00640 1 -- 172.21.15.151:0/2719166732 shutdown_connections 2024-09-16T15:56:21.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.936+0000 7fdb1ac00640 1 --2- 172.21.15.151:0/2719166732 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdaf407afb0 0x7fdaf407d470 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:21.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.936+0000 7fdb1ac00640 1 --2- 172.21.15.151:0/2719166732 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb14161460 0x7fdb141e81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:21.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.936+0000 7fdb1ac00640 1 --2- 172.21.15.151:0/2719166732 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb140b5480 0x7fdb14157000 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:21.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.936+0000 7fdb1ac00640 1 --2- 172.21.15.151:0/2719166732 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb140b43c0 0x7fdb140b7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:21.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.936+0000 7fdb1ac00640 1 -- 172.21.15.151:0/2719166732 >> 172.21.15.151:0/2719166732 conn(0x7fdb1409d0b0 msgr2=0x7fdb140ca650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:21.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.936+0000 7fdb1ac00640 1 -- 172.21.15.151:0/2719166732 shutdown_connections 2024-09-16T15:56:21.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:21.936+0000 7fdb1ac00640 1 -- 172.21.15.151:0/2719166732 wait complete. 2024-09-16T15:56:21.953 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone58175 2024-09-16T15:56:22.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.114+0000 7facb2e00640 1 -- 172.21.15.151:0/2460032537 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7facac06f220 msgr2=0x7facac06f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:22.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.114+0000 7facb2e00640 1 --2- 172.21.15.151:0/2460032537 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7facac06f220 0x7facac06f6d0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7faca4005fd0 tx=0x7faca402a580 comp rx=0 tx=0).stop 2024-09-16T15:56:22.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.114+0000 7facb2e00640 1 -- 172.21.15.151:0/2460032537 shutdown_connections 2024-09-16T15:56:22.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.114+0000 7facb2e00640 1 --2- 172.21.15.151:0/2460032537 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7facac06f220 0x7facac06f6d0 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:22.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.114+0000 7facb2e00640 1 --2- 172.21.15.151:0/2460032537 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7facac15cff0 0x7facac06e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:22.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.114+0000 7facb2e00640 1 --2- 172.21.15.151:0/2460032537 >> v2:172.21.15.167:3300/0 conn(0x7facac159aa0 0x7facac0741a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:22.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.114+0000 7facb2e00640 1 -- 172.21.15.151:0/2460032537 >> 172.21.15.151:0/2460032537 conn(0x7facac0b18e0 msgr2=0x7facac0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:22.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.114+0000 7facb2e00640 1 -- 172.21.15.151:0/2460032537 shutdown_connections 2024-09-16T15:56:22.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.114+0000 7facb2e00640 1 -- 172.21.15.151:0/2460032537 wait complete. 2024-09-16T15:56:22.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.116+0000 7facb2e00640 1 Processor -- start 2024-09-16T15:56:22.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.116+0000 7facb2e00640 1 -- start start 2024-09-16T15:56:22.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.117+0000 7facb2e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7facac06f220 0x7facac14a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:22.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.117+0000 7facb2e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7facac159aa0 0x7facac09b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:22.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.117+0000 7facb2e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7facac15cff0 0x7facac09b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:22.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.117+0000 7facb2e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7facac0cc310 con 0x7facac15cff0 2024-09-16T15:56:22.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.117+0000 7facb2e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7facac0cc190 con 0x7facac06f220 2024-09-16T15:56:22.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.117+0000 7facb2e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7facac0cc490 con 0x7facac159aa0 2024-09-16T15:56:22.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.117+0000 7facb0e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7facac15cff0 0x7facac09b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:22.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.117+0000 7facb0e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7facac15cff0 0x7facac09b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:59838/0 (socket says 172.21.15.151:59838) 2024-09-16T15:56:22.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.117+0000 7facb0e00640 1 -- 172.21.15.151:0/3142485912 learned_addr learned my addr 172.21.15.151:0/3142485912 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:56:22.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.117+0000 7facabe00640 1 --2- 172.21.15.151:0/3142485912 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7facac06f220 0x7facac14a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:22.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.117+0000 7facab400640 1 --2- 172.21.15.151:0/3142485912 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7facac159aa0 0x7facac09b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:22.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.118+0000 7facb0e00640 1 -- 172.21.15.151:0/3142485912 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7facac159aa0 msgr2=0x7facac09b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:22.119 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.118+0000 7facb0e00640 1 --2- 172.21.15.151:0/3142485912 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7facac159aa0 0x7facac09b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:22.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.118+0000 7facb0e00640 1 -- 172.21.15.151:0/3142485912 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7facac06f220 msgr2=0x7facac14a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:22.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.118+0000 7facb0e00640 1 --2- 172.21.15.151:0/3142485912 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7facac06f220 0x7facac14a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:22.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.118+0000 7facb0e00640 1 -- 172.21.15.151:0/3142485912 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facac157bd0 con 0x7facac15cff0 2024-09-16T15:56:22.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.118+0000 7facb0e00640 1 --2- 172.21.15.151:0/3142485912 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7facac15cff0 0x7facac09b9e0 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7faca4006010 tx=0x7faca40039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:22.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.118+0000 7faca3e00640 1 -- 172.21.15.151:0/3142485912 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca403f070 con 0x7facac15cff0 2024-09-16T15:56:22.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.118+0000 7faca3e00640 1 -- 172.21.15.151:0/3142485912 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faca4004890 con 0x7facac15cff0 2024-09-16T15:56:22.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.118+0000 7faca3e00640 1 -- 172.21.15.151:0/3142485912 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca4035030 con 0x7facac15cff0 2024-09-16T15:56:22.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.118+0000 7facb2e00640 1 -- 172.21.15.151:0/3142485912 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facac157eb0 con 0x7facac15cff0 2024-09-16T15:56:22.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.118+0000 7facb2e00640 1 -- 172.21.15.151:0/3142485912 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facac14ab40 con 0x7facac15cff0 2024-09-16T15:56:22.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.119+0000 7facb2e00640 1 -- 172.21.15.151:0/3142485912 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facac069360 con 0x7facac15cff0 2024-09-16T15:56:22.121 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.120+0000 7faca3e00640 1 -- 172.21.15.151:0/3142485912 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 55) ==== 104857+0+0 (secure 0 0 0) 0x7faca4042070 con 0x7facac15cff0 2024-09-16T15:56:22.122 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.121+0000 7faca3e00640 1 --2- 172.21.15.151:0/3142485912 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fac8c07afb0 0x7fac8c07d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:22.122 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.121+0000 7faca3e00640 1 -- 172.21.15.151:0/3142485912 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5907+0+0 (secure 0 0 0) 0x7faca40baba0 con 0x7facac15cff0 2024-09-16T15:56:22.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.122+0000 7facabe00640 1 --2- 172.21.15.151:0/3142485912 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fac8c07afb0 0x7fac8c07d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:22.123 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.122+0000 7facabe00640 1 --2- 172.21.15.151:0/3142485912 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fac8c07afb0 0x7fac8c07d470 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fac9c009f20 tx=0x7fac9c0094f0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:22.126 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.125+0000 7faca3e00640 1 -- 172.21.15.151:0/3142485912 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7faca4084c70 con 0x7facac15cff0 2024-09-16T15:56:22.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.334+0000 7facb2e00640 1 -- 172.21.15.151:0/3142485912 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone58175", "target": ["mon-mgr", ""]}) -- 0x7facac070980 con 0x7fac8c07afb0 2024-09-16T15:56:22.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.349+0000 7faca3e00640 1 -- 172.21.15.151:0/3142485912 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7facac070980 con 0x7fac8c07afb0 2024-09-16T15:56:22.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.352+0000 7facb2e00640 1 -- 172.21.15.151:0/3142485912 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fac8c07afb0 msgr2=0x7fac8c07d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:22.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.352+0000 7facb2e00640 1 --2- 172.21.15.151:0/3142485912 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fac8c07afb0 0x7fac8c07d470 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fac9c009f20 tx=0x7fac9c0094f0 comp rx=0 tx=0).stop 2024-09-16T15:56:22.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.352+0000 7facb2e00640 1 -- 172.21.15.151:0/3142485912 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7facac15cff0 msgr2=0x7facac09b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:22.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.352+0000 7facb2e00640 1 --2- 172.21.15.151:0/3142485912 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7facac15cff0 0x7facac09b9e0 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7faca4006010 tx=0x7faca40039e0 comp rx=0 tx=0).stop 2024-09-16T15:56:22.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.352+0000 7facb2e00640 1 -- 172.21.15.151:0/3142485912 shutdown_connections 2024-09-16T15:56:22.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.352+0000 7facb2e00640 1 --2- 172.21.15.151:0/3142485912 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fac8c07afb0 0x7fac8c07d470 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:22.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.352+0000 7facb2e00640 1 --2- 172.21.15.151:0/3142485912 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7facac15cff0 0x7facac09b9e0 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:22.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.352+0000 7facb2e00640 1 --2- 172.21.15.151:0/3142485912 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7facac159aa0 0x7facac09b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:22.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.352+0000 7facb2e00640 1 --2- 172.21.15.151:0/3142485912 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7facac06f220 0x7facac14a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:22.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.352+0000 7facb2e00640 1 -- 172.21.15.151:0/3142485912 >> 172.21.15.151:0/3142485912 conn(0x7facac0b18e0 msgr2=0x7facac15dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:22.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.352+0000 7facb2e00640 1 -- 172.21.15.151:0/3142485912 shutdown_connections 2024-09-16T15:56:22.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.352+0000 7facb2e00640 1 -- 172.21.15.151:0/3142485912 wait complete. 2024-09-16T15:56:22.372 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone67026 2024-09-16T15:56:22.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.534+0000 7f63db000640 1 -- 172.21.15.151:0/4170415182 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63d406e6f0 msgr2=0x7f63d406eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:22.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.534+0000 7f63db000640 1 --2- 172.21.15.151:0/4170415182 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63d406e6f0 0x7f63d406eba0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f63cc005fd0 tx=0x7f63cc02a660 comp rx=0 tx=0).stop 2024-09-16T15:56:22.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.534+0000 7f63db000640 1 -- 172.21.15.151:0/4170415182 shutdown_connections 2024-09-16T15:56:22.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.534+0000 7f63db000640 1 --2- 172.21.15.151:0/4170415182 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63d406e6f0 0x7f63d406eba0 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:22.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.534+0000 7f63db000640 1 --2- 172.21.15.151:0/4170415182 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f63d415f910 0x7f63d415fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:22.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.534+0000 7f63db000640 1 -- 172.21.15.151:0/4170415182 >> 172.21.15.151:0/4170415182 conn(0x7f63d40b18e0 msgr2=0x7f63d40b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:22.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.534+0000 7f63db000640 1 -- 172.21.15.151:0/4170415182 shutdown_connections 2024-09-16T15:56:22.536 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.535+0000 7f63db000640 1 -- 172.21.15.151:0/4170415182 wait complete. 2024-09-16T15:56:22.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.536+0000 7f63db000640 1 Processor -- start 2024-09-16T15:56:22.537 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.536+0000 7f63db000640 1 -- start start 2024-09-16T15:56:22.539 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.537+0000 7f63db000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f63d406e6f0 0x7f63d4158030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:22.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.537+0000 7f63db000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63d415f910 0x7f63d40b46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:22.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.537+0000 7f63db000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63d4155650 0x7f63d40ae310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:22.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.537+0000 7f63db000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f63d406f260 con 0x7f63d4155650 2024-09-16T15:56:22.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.537+0000 7f63db000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f63d406f0e0 con 0x7f63d415f910 2024-09-16T15:56:22.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.537+0000 7f63db000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f63d406f3e0 con 0x7f63d406e6f0 2024-09-16T15:56:22.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.537+0000 7f63d9000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63d4155650 0x7f63d40ae310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:22.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.538+0000 7f63d9000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63d4155650 0x7f63d40ae310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:59846/0 (socket says 172.21.15.151:59846) 2024-09-16T15:56:22.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.538+0000 7f63d9000640 1 -- 172.21.15.151:0/4279786478 learned_addr learned my addr 172.21.15.151:0/4279786478 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:56:22.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.538+0000 7f63d3e00640 1 --2- 172.21.15.151:0/4279786478 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f63d406e6f0 0x7f63d4158030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:22.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.538+0000 7f63d3400640 1 --2- 172.21.15.151:0/4279786478 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63d415f910 0x7f63d40b46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:22.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.538+0000 7f63d9000640 1 -- 172.21.15.151:0/4279786478 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f63d406e6f0 msgr2=0x7f63d4158030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:22.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.538+0000 7f63d9000640 1 --2- 172.21.15.151:0/4279786478 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f63d406e6f0 0x7f63d4158030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:22.540 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.538+0000 7f63d9000640 1 -- 172.21.15.151:0/4279786478 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63d415f910 msgr2=0x7f63d40b46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:22.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.538+0000 7f63d9000640 1 --2- 172.21.15.151:0/4279786478 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63d415f910 0x7f63d40b46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:22.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.538+0000 7f63d9000640 1 -- 172.21.15.151:0/4279786478 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63d41f5330 con 0x7f63d4155650 2024-09-16T15:56:22.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.538+0000 7f63d3e00640 1 --2- 172.21.15.151:0/4279786478 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f63d406e6f0 0x7f63d4158030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:56:22.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.538+0000 7f63d9000640 1 --2- 172.21.15.151:0/4279786478 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63d4155650 0x7f63d40ae310 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f63cc006010 tx=0x7f63cc0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:22.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.538+0000 7f63cbe00640 1 -- 172.21.15.151:0/4279786478 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63cc03f070 con 0x7f63d4155650 2024-09-16T15:56:22.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.539+0000 7f63cbe00640 1 -- 172.21.15.151:0/4279786478 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f63cc0047a0 con 0x7f63d4155650 2024-09-16T15:56:22.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.539+0000 7f63db000640 1 -- 172.21.15.151:0/4279786478 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63d41f5610 con 0x7f63d4155650 2024-09-16T15:56:22.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.539+0000 7f63cbe00640 1 -- 172.21.15.151:0/4279786478 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63cc035030 con 0x7f63d4155650 2024-09-16T15:56:22.541 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.539+0000 7f63db000640 1 -- 172.21.15.151:0/4279786478 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63d4158570 con 0x7f63d4155650 2024-09-16T15:56:22.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.540+0000 7f63db000640 1 -- 172.21.15.151:0/4279786478 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63d4069360 con 0x7f63d4155650 2024-09-16T15:56:22.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.541+0000 7f63cbe00640 1 -- 172.21.15.151:0/4279786478 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 55) ==== 104857+0+0 (secure 0 0 0) 0x7f63cc042070 con 0x7f63d4155650 2024-09-16T15:56:22.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.542+0000 7f63cbe00640 1 --2- 172.21.15.151:0/4279786478 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f63b407afb0 0x7f63b407d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:22.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.542+0000 7f63cbe00640 1 -- 172.21.15.151:0/4279786478 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5907+0+0 (secure 0 0 0) 0x7f63cc0baa60 con 0x7f63d4155650 2024-09-16T15:56:22.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.542+0000 7f63d3e00640 1 --2- 172.21.15.151:0/4279786478 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f63b407afb0 0x7f63b407d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:22.544 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.543+0000 7f63d3e00640 1 --2- 172.21.15.151:0/4279786478 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f63b407afb0 0x7f63b407d470 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f63c4005fd0 tx=0x7f63c4009480 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:22.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.546+0000 7f63cbe00640 1 -- 172.21.15.151:0/4279786478 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f63cc084bb0 con 0x7f63d4155650 2024-09-16T15:56:22.621 DEBUG:teuthology.orchestra.run.smithi151:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:56:22.630 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:56:22.959 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:22.958+0000 7f63db000640 1 -- 172.21.15.151:0/4279786478 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone67026", "target": ["mon-mgr", ""]}) -- 0x7f63d406eba0 con 0x7f63b407afb0 2024-09-16T15:56:27.153 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.151+0000 7f63cbe00640 1 -- 172.21.15.151:0/4279786478 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f63d406eba0 con 0x7f63b407afb0 2024-09-16T15:56:27.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.154+0000 7f63db000640 1 -- 172.21.15.151:0/4279786478 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f63b407afb0 msgr2=0x7f63b407d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:27.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.154+0000 7f63db000640 1 --2- 172.21.15.151:0/4279786478 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f63b407afb0 0x7f63b407d470 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f63c4005fd0 tx=0x7f63c4009480 comp rx=0 tx=0).stop 2024-09-16T15:56:27.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.154+0000 7f63db000640 1 -- 172.21.15.151:0/4279786478 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63d4155650 msgr2=0x7f63d40ae310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:27.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.154+0000 7f63db000640 1 --2- 172.21.15.151:0/4279786478 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63d4155650 0x7f63d40ae310 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f63cc006010 tx=0x7f63cc0038a0 comp rx=0 tx=0).stop 2024-09-16T15:56:27.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.154+0000 7f63db000640 1 -- 172.21.15.151:0/4279786478 shutdown_connections 2024-09-16T15:56:27.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.154+0000 7f63db000640 1 --2- 172.21.15.151:0/4279786478 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f63b407afb0 0x7f63b407d470 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:27.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.154+0000 7f63db000640 1 --2- 172.21.15.151:0/4279786478 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f63d415f910 0x7f63d40b46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:27.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.154+0000 7f63db000640 1 --2- 172.21.15.151:0/4279786478 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63d4155650 0x7f63d40ae310 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:27.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.154+0000 7f63db000640 1 --2- 172.21.15.151:0/4279786478 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f63d406e6f0 0x7f63d4158030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:27.157 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.154+0000 7f63db000640 1 -- 172.21.15.151:0/4279786478 >> 172.21.15.151:0/4279786478 conn(0x7f63d40b18e0 msgr2=0x7f63d4156920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:27.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.155+0000 7f63db000640 1 -- 172.21.15.151:0/4279786478 shutdown_connections 2024-09-16T15:56:27.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.155+0000 7f63db000640 1 -- 172.21.15.151:0/4279786478 wait complete. 2024-09-16T15:56:27.177 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone83343 2024-09-16T15:56:27.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.341+0000 7fc758400640 1 -- 172.21.15.151:0/3156247766 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc75015dfc0 msgr2=0x7fc75006f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:27.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.341+0000 7fc758400640 1 --2- 172.21.15.151:0/3156247766 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc75015dfc0 0x7fc75006f300 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fc744009830 tx=0x7fc74402a530 comp rx=0 tx=0).stop 2024-09-16T15:56:27.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.341+0000 7fc758400640 1 -- 172.21.15.151:0/3156247766 shutdown_connections 2024-09-16T15:56:27.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.341+0000 7fc758400640 1 --2- 172.21.15.151:0/3156247766 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc7500b4770 0x7fc75006e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:27.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.341+0000 7fc758400640 1 --2- 172.21.15.151:0/3156247766 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc75015dfc0 0x7fc75006f300 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:27.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.341+0000 7fc758400640 1 -- 172.21.15.151:0/3156247766 >> 172.21.15.151:0/3156247766 conn(0x7fc7500b18e0 msgr2=0x7fc7500b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:27.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.341+0000 7fc758400640 1 -- 172.21.15.151:0/3156247766 shutdown_connections 2024-09-16T15:56:27.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.341+0000 7fc758400640 1 -- 172.21.15.151:0/3156247766 wait complete. 2024-09-16T15:56:27.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.342+0000 7fc758400640 1 Processor -- start 2024-09-16T15:56:27.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.342+0000 7fc758400640 1 -- start start 2024-09-16T15:56:27.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.343+0000 7fc758400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc75006ec30 0x7fc7501f4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:27.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.343+0000 7fc758400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7500b4770 0x7fc75014a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:27.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.343+0000 7fc758400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc75015dfc0 0x7fc75015b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:27.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.343+0000 7fc758400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc7501622c0 con 0x7fc7500b4770 2024-09-16T15:56:27.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.343+0000 7fc758400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc7500ae8f0 con 0x7fc75006ec30 2024-09-16T15:56:27.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.343+0000 7fc758400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fc750162440 con 0x7fc75015dfc0 2024-09-16T15:56:27.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.344+0000 7fc755000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7500b4770 0x7fc75014a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:27.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.344+0000 7fc755000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7500b4770 0x7fc75014a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43532/0 (socket says 172.21.15.151:43532) 2024-09-16T15:56:27.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.344+0000 7fc755000640 1 -- 172.21.15.151:0/1764595928 learned_addr learned my addr 172.21.15.151:0/1764595928 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:56:27.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.344+0000 7fc756400640 1 --2- 172.21.15.151:0/1764595928 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc75015dfc0 0x7fc75015b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:27.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.344+0000 7fc755a00640 1 --2- 172.21.15.151:0/1764595928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc75006ec30 0x7fc7501f4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:27.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.344+0000 7fc755000640 1 -- 172.21.15.151:0/1764595928 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc75015dfc0 msgr2=0x7fc75015b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:27.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.344+0000 7fc755000640 1 --2- 172.21.15.151:0/1764595928 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc75015dfc0 0x7fc75015b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:27.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.344+0000 7fc755000640 1 -- 172.21.15.151:0/1764595928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc75006ec30 msgr2=0x7fc7501f4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:27.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.344+0000 7fc755000640 1 --2- 172.21.15.151:0/1764595928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc75006ec30 0x7fc7501f4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:27.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.344+0000 7fc755000640 1 -- 172.21.15.151:0/1764595928 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7501f5600 con 0x7fc7500b4770 2024-09-16T15:56:27.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.344+0000 7fc755000640 1 --2- 172.21.15.151:0/1764595928 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7500b4770 0x7fc75014a540 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7fc73c00e9a0 tx=0x7fc73c00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:27.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.345+0000 7fc74d600640 1 -- 172.21.15.151:0/1764595928 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc73c00cd80 con 0x7fc7500b4770 2024-09-16T15:56:27.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.345+0000 7fc74d600640 1 -- 172.21.15.151:0/1764595928 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc73c010430 con 0x7fc7500b4770 2024-09-16T15:56:27.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.345+0000 7fc74d600640 1 -- 172.21.15.151:0/1764595928 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc73c010880 con 0x7fc7500b4770 2024-09-16T15:56:27.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.345+0000 7fc758400640 1 -- 172.21.15.151:0/1764595928 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7501f58e0 con 0x7fc7500b4770 2024-09-16T15:56:27.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.345+0000 7fc758400640 1 -- 172.21.15.151:0/1764595928 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc75014aa80 con 0x7fc7500b4770 2024-09-16T15:56:27.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.346+0000 7fc758400640 1 -- 172.21.15.151:0/1764595928 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc750069360 con 0x7fc7500b4770 2024-09-16T15:56:27.349 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.347+0000 7fc74d600640 1 -- 172.21.15.151:0/1764595928 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 55) ==== 104857+0+0 (secure 0 0 0) 0x7fc73c010a20 con 0x7fc7500b4770 2024-09-16T15:56:27.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.348+0000 7fc74d600640 1 --2- 172.21.15.151:0/1764595928 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc73007afb0 0x7fc73007d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:27.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.348+0000 7fc74d600640 1 -- 172.21.15.151:0/1764595928 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5963+0+0 (secure 0 0 0) 0x7fc73c09a660 con 0x7fc7500b4770 2024-09-16T15:56:27.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.349+0000 7fc755a00640 1 --2- 172.21.15.151:0/1764595928 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc73007afb0 0x7fc73007d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:27.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.349+0000 7fc755a00640 1 --2- 172.21.15.151:0/1764595928 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc73007afb0 0x7fc73007d470 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fc744034000 tx=0x7fc744004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:27.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.352+0000 7fc74d600640 1 -- 172.21.15.151:0/1764595928 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc73c0640e0 con 0x7fc7500b4770 2024-09-16T15:56:27.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.560+0000 7fc758400640 1 -- 172.21.15.151:0/1764595928 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone83343", "target": ["mon-mgr", ""]}) -- 0x7fc7501f53a0 con 0x7fc73007afb0 2024-09-16T15:56:27.583 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.581+0000 7fc74d600640 1 -- 172.21.15.151:0/1764595928 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc744004380 con 0x7fc73007afb0 2024-09-16T15:56:27.585 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.583+0000 7fc758400640 1 -- 172.21.15.151:0/1764595928 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc73007afb0 msgr2=0x7fc73007d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:27.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.583+0000 7fc758400640 1 --2- 172.21.15.151:0/1764595928 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc73007afb0 0x7fc73007d470 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fc744034000 tx=0x7fc744004730 comp rx=0 tx=0).stop 2024-09-16T15:56:27.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.583+0000 7fc758400640 1 -- 172.21.15.151:0/1764595928 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7500b4770 msgr2=0x7fc75014a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:27.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.583+0000 7fc758400640 1 --2- 172.21.15.151:0/1764595928 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7500b4770 0x7fc75014a540 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7fc73c00e9a0 tx=0x7fc73c00ee60 comp rx=0 tx=0).stop 2024-09-16T15:56:27.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.583+0000 7fc758400640 1 -- 172.21.15.151:0/1764595928 shutdown_connections 2024-09-16T15:56:27.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.583+0000 7fc758400640 1 --2- 172.21.15.151:0/1764595928 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc73007afb0 0x7fc73007d470 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:27.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.583+0000 7fc758400640 1 --2- 172.21.15.151:0/1764595928 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc75015dfc0 0x7fc75015b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:27.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.583+0000 7fc758400640 1 --2- 172.21.15.151:0/1764595928 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7500b4770 0x7fc75014a540 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:27.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.583+0000 7fc758400640 1 --2- 172.21.15.151:0/1764595928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc75006ec30 0x7fc7501f4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:27.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.583+0000 7fc758400640 1 -- 172.21.15.151:0/1764595928 >> 172.21.15.151:0/1764595928 conn(0x7fc7500b18e0 msgr2=0x7fc7500af980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:27.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.583+0000 7fc758400640 1 -- 172.21.15.151:0/1764595928 shutdown_connections 2024-09-16T15:56:27.586 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.583+0000 7fc758400640 1 -- 172.21.15.151:0/1764595928 wait complete. 2024-09-16T15:56:27.602 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone21631 2024-09-16T15:56:27.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.763+0000 7fdbbae00640 1 -- 172.21.15.151:0/2626114561 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb9c0039e0 msgr2=0x7fdb9c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:27.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.763+0000 7fdbbae00640 1 --2- 172.21.15.151:0/2626114561 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdb9c0039e0 0x7fdb9c003dc0 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7fdba8009b20 tx=0x7fdb9c005830 comp rx=0 tx=0).stop 2024-09-16T15:56:27.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.763+0000 7fdbba400640 1 -- 172.21.15.151:0/2626114561 reap_dead start 2024-09-16T15:56:27.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.763+0000 7fdbbae00640 1 -- 172.21.15.151:0/2626114561 shutdown_connections 2024-09-16T15:56:27.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.763+0000 7fdbbae00640 1 -- 172.21.15.151:0/2626114561 >> 172.21.15.151:0/2626114561 conn(0x7fdbb40cfa70 msgr2=0x7fdbb40cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:27.765 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.763+0000 7fdbbae00640 1 -- 172.21.15.151:0/2626114561 shutdown_connections 2024-09-16T15:56:27.766 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.763+0000 7fdbbae00640 1 -- 172.21.15.151:0/2626114561 wait complete. 2024-09-16T15:56:27.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.765+0000 7fdbbae00640 1 Processor -- start 2024-09-16T15:56:27.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.765+0000 7fdbbae00640 1 -- start start 2024-09-16T15:56:27.769 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.766+0000 7fdbbae00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb9c0039e0 0x7fdbb41f5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:27.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.766+0000 7fdbbae00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbb40c9950 0x7fdbb41ef9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:27.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.766+0000 7fdbbae00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdbb40ad230 0x7fdbb40ad6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:27.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.766+0000 7fdbbae00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb4073120 con 0x7fdbb40ad230 2024-09-16T15:56:27.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.766+0000 7fdbbae00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb4072fa0 con 0x7fdbb40c9950 2024-09-16T15:56:27.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.766+0000 7fdbbae00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fdbb40cd640 con 0x7fdb9c0039e0 2024-09-16T15:56:27.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.766+0000 7fdbba400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdbb40ad230 0x7fdbb40ad6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:27.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.766+0000 7fdbb9a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb9c0039e0 0x7fdbb41f5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:27.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.766+0000 7fdbb9000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbb40c9950 0x7fdbb41ef9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:27.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.766+0000 7fdbba400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdbb40ad230 0x7fdbb40ad6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43556/0 (socket says 172.21.15.151:43556) 2024-09-16T15:56:27.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.766+0000 7fdbba400640 1 -- 172.21.15.151:0/3642639050 learned_addr learned my addr 172.21.15.151:0/3642639050 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:56:27.770 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.766+0000 7fdbb9a00640 1 -- 172.21.15.151:0/3642639050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbb40c9950 msgr2=0x7fdbb41ef9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:27.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.766+0000 7fdbb9a00640 1 --2- 172.21.15.151:0/3642639050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbb40c9950 0x7fdbb41ef9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:27.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.767+0000 7fdbb9a00640 1 -- 172.21.15.151:0/3642639050 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdbb40ad230 msgr2=0x7fdbb40ad6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:27.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.767+0000 7fdbb9a00640 1 --2- 172.21.15.151:0/3642639050 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdbb40ad230 0x7fdbb40ad6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:27.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.767+0000 7fdbb9a00640 1 -- 172.21.15.151:0/3642639050 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbb41f2730 con 0x7fdb9c0039e0 2024-09-16T15:56:27.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.767+0000 7fdbba400640 1 --2- 172.21.15.151:0/3642639050 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdbb40ad230 0x7fdbb40ad6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:56:27.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.767+0000 7fdbb9a00640 1 --2- 172.21.15.151:0/3642639050 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb9c0039e0 0x7fdbb41f5c10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdba8032000 tx=0x7fdba8033670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:27.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.767+0000 7fdbb1600640 1 -- 172.21.15.151:0/3642639050 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba8043070 con 0x7fdb9c0039e0 2024-09-16T15:56:27.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.767+0000 7fdbb1600640 1 -- 172.21.15.151:0/3642639050 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdba8004140 con 0x7fdb9c0039e0 2024-09-16T15:56:27.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.767+0000 7fdbbae00640 1 -- 172.21.15.151:0/3642639050 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbb41f6b20 con 0x7fdb9c0039e0 2024-09-16T15:56:27.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.767+0000 7fdbb1600640 1 -- 172.21.15.151:0/3642639050 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba8038040 con 0x7fdb9c0039e0 2024-09-16T15:56:27.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.767+0000 7fdbbae00640 1 -- 172.21.15.151:0/3642639050 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbb41f6560 con 0x7fdb9c0039e0 2024-09-16T15:56:27.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.768+0000 7fdbbae00640 1 -- 172.21.15.151:0/3642639050 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbb406ff30 con 0x7fdb9c0039e0 2024-09-16T15:56:27.772 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.770+0000 7fdbb1600640 1 -- 172.21.15.151:0/3642639050 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 55) ==== 104857+0+0 (secure 0 0 0) 0x7fdba8046070 con 0x7fdb9c0039e0 2024-09-16T15:56:27.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.771+0000 7fdbb1600640 1 --2- 172.21.15.151:0/3642639050 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdb9407afb0 0x7fdb9407d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:27.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.771+0000 7fdbb1600640 1 -- 172.21.15.151:0/3642639050 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(148..148 src has 1..148) ==== 5963+0+0 (secure 0 0 0) 0x7fdba80b9f80 con 0x7fdb9c0039e0 2024-09-16T15:56:27.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.771+0000 7fdbb9000640 1 --2- 172.21.15.151:0/3642639050 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdb9407afb0 0x7fdb9407d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:27.773 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.772+0000 7fdbb9000640 1 --2- 172.21.15.151:0/3642639050 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdb9407afb0 0x7fdb9407d470 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fdba0006930 tx=0x7fdba00068c0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:27.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.775+0000 7fdbb1600640 1 -- 172.21.15.151:0/3642639050 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fdba8083b80 con 0x7fdb9c0039e0 2024-09-16T15:56:27.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:27.987+0000 7fdbbae00640 1 -- 172.21.15.151:0/3642639050 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone21631", "target": ["mon-mgr", ""]}) -- 0x7fdbb40b59f0 con 0x7fdb9407afb0 2024-09-16T15:56:28.009 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.008+0000 7fdbb1600640 1 -- 172.21.15.151:0/3642639050 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fdbb40b59f0 con 0x7fdb9407afb0 2024-09-16T15:56:28.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.010+0000 7fdbbae00640 1 -- 172.21.15.151:0/3642639050 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdb9407afb0 msgr2=0x7fdb9407d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:28.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.010+0000 7fdbbae00640 1 --2- 172.21.15.151:0/3642639050 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdb9407afb0 0x7fdb9407d470 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fdba0006930 tx=0x7fdba00068c0 comp rx=0 tx=0).stop 2024-09-16T15:56:28.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.010+0000 7fdbbae00640 1 -- 172.21.15.151:0/3642639050 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb9c0039e0 msgr2=0x7fdbb41f5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:28.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.010+0000 7fdbbae00640 1 --2- 172.21.15.151:0/3642639050 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb9c0039e0 0x7fdbb41f5c10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdba8032000 tx=0x7fdba8033670 comp rx=0 tx=0).stop 2024-09-16T15:56:28.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.010+0000 7fdbbae00640 1 -- 172.21.15.151:0/3642639050 shutdown_connections 2024-09-16T15:56:28.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.010+0000 7fdbbae00640 1 --2- 172.21.15.151:0/3642639050 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fdb9407afb0 0x7fdb9407d470 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:28.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.010+0000 7fdbbae00640 1 --2- 172.21.15.151:0/3642639050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbb40c9950 0x7fdbb41ef9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:28.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.010+0000 7fdbbae00640 1 --2- 172.21.15.151:0/3642639050 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdbb40ad230 0x7fdbb40ad6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:28.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.010+0000 7fdbbae00640 1 --2- 172.21.15.151:0/3642639050 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdb9c0039e0 0x7fdbb41f5c10 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:28.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.010+0000 7fdbbae00640 1 -- 172.21.15.151:0/3642639050 >> 172.21.15.151:0/3642639050 conn(0x7fdbb40cfa70 msgr2=0x7fdbb40b4ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:28.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.010+0000 7fdbbae00640 1 -- 172.21.15.151:0/3642639050 shutdown_connections 2024-09-16T15:56:28.013 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.010+0000 7fdbbae00640 1 -- 172.21.15.151:0/3642639050 wait complete. 2024-09-16T15:56:28.028 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-16T15:56:28.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T15:56:28.114 INFO:teuthology.orchestra.run.smithi151.stdout:2 2024-09-16T15:56:28.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T15:56:28.122 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/max_concurrent_clones 4 2024-09-16T15:56:28.283 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.281+0000 7f7bf6800640 1 -- 172.21.15.151:0/2453174058 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7bf0145750 msgr2=0x7f7bf0073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:28.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.281+0000 7f7bf6800640 1 --2- 172.21.15.151:0/2453174058 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7bf0145750 0x7f7bf0073440 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f7bf00af750 tx=0x7f7be002a4d0 comp rx=0 tx=0).stop 2024-09-16T15:56:28.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.281+0000 7f7bf6800640 1 -- 172.21.15.151:0/2453174058 shutdown_connections 2024-09-16T15:56:28.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.281+0000 7f7bf6800640 1 --2- 172.21.15.151:0/2453174058 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7bf00ccb00 0x7f7bf015f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:28.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.281+0000 7f7bf6800640 1 --2- 172.21.15.151:0/2453174058 >> v2:172.21.15.167:3300/0 conn(0x7f7bf006efa0 0x7f7bf00cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:28.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.281+0000 7f7bf6800640 1 --2- 172.21.15.151:0/2453174058 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7bf0145750 0x7f7bf0073440 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:28.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.281+0000 7f7bf6800640 1 -- 172.21.15.151:0/2453174058 >> 172.21.15.151:0/2453174058 conn(0x7f7bf00b1840 msgr2=0x7f7bf00b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:28.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.281+0000 7f7bf6800640 1 -- 172.21.15.151:0/2453174058 shutdown_connections 2024-09-16T15:56:28.284 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.281+0000 7f7bf6800640 1 -- 172.21.15.151:0/2453174058 wait complete. 2024-09-16T15:56:28.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.283+0000 7f7bf6800640 1 Processor -- start 2024-09-16T15:56:28.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.283+0000 7f7bf6800640 1 -- start start 2024-09-16T15:56:28.287 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.284+0000 7f7bf6800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7bf006efa0 0x7f7bf01f8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:28.287 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.284+0000 7f7bf6800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7bf00ccb00 0x7f7bf013f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:28.287 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.284+0000 7f7bf6800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7bf0145750 0x7f7bf00adfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:28.287 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.284+0000 7f7bf6800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf01602c0 con 0x7f7bf00ccb00 2024-09-16T15:56:28.287 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.284+0000 7f7bf6800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf0160140 con 0x7f7bf006efa0 2024-09-16T15:56:28.287 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.284+0000 7f7bf6800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f7bf0160440 con 0x7f7bf0145750 2024-09-16T15:56:28.287 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.284+0000 7f7beea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7bf00ccb00 0x7f7bf013f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:28.287 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.284+0000 7f7beea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7bf00ccb00 0x7f7bf013f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43568/0 (socket says 172.21.15.151:43568) 2024-09-16T15:56:28.287 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.284+0000 7f7beea00640 1 -- 172.21.15.151:0/2638765990 learned_addr learned my addr 172.21.15.151:0/2638765990 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:56:28.287 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.284+0000 7f7bef400640 1 --2- 172.21.15.151:0/2638765990 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7bf006efa0 0x7f7bf01f8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:28.287 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.284+0000 7f7befe00640 1 --2- 172.21.15.151:0/2638765990 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7bf0145750 0x7f7bf00adfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:28.287 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.285+0000 7f7beea00640 1 -- 172.21.15.151:0/2638765990 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7bf0145750 msgr2=0x7f7bf00adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:28.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.285+0000 7f7beea00640 1 --2- 172.21.15.151:0/2638765990 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7bf0145750 0x7f7bf00adfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:28.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.285+0000 7f7beea00640 1 -- 172.21.15.151:0/2638765990 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7bf006efa0 msgr2=0x7f7bf01f8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:28.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.285+0000 7f7beea00640 1 --2- 172.21.15.151:0/2638765990 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7bf006efa0 0x7f7bf01f8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:28.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.285+0000 7f7beea00640 1 -- 172.21.15.151:0/2638765990 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bf013f9d0 con 0x7f7bf00ccb00 2024-09-16T15:56:28.288 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.285+0000 7f7beea00640 1 --2- 172.21.15.151:0/2638765990 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7bf00ccb00 0x7f7bf013f080 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc00d720 tx=0x7f7bdc00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:28.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.285+0000 7f7be7400640 1 -- 172.21.15.151:0/2638765990 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bdc017070 con 0x7f7bf00ccb00 2024-09-16T15:56:28.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.285+0000 7f7be7400640 1 -- 172.21.15.151:0/2638765990 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7bdc009d20 con 0x7f7bf00ccb00 2024-09-16T15:56:28.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.285+0000 7f7be7400640 1 -- 172.21.15.151:0/2638765990 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bdc004280 con 0x7f7bf00ccb00 2024-09-16T15:56:28.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.285+0000 7f7bf6800640 1 -- 172.21.15.151:0/2638765990 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bf013fcb0 con 0x7f7bf00ccb00 2024-09-16T15:56:28.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.285+0000 7f7bf6800640 1 -- 172.21.15.151:0/2638765990 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bf01f7d80 con 0x7f7bf00ccb00 2024-09-16T15:56:28.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.286+0000 7f7bf6800640 1 -- 172.21.15.151:0/2638765990 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bf015f9f0 con 0x7f7bf00ccb00 2024-09-16T15:56:28.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.288+0000 7f7be7400640 1 -- 172.21.15.151:0/2638765990 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 55) ==== 104857+0+0 (secure 0 0 0) 0x7f7bdc005020 con 0x7f7bf00ccb00 2024-09-16T15:56:28.290 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.288+0000 7f7be7400640 1 --2- 172.21.15.151:0/2638765990 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7bd007b010 0x7f7bd007d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:28.290 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.289+0000 7f7be7400640 1 -- 172.21.15.151:0/2638765990 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5963+0+0 (secure 0 0 0) 0x7f7bdc099fb0 con 0x7f7bf00ccb00 2024-09-16T15:56:28.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.289+0000 7f7bef400640 1 --2- 172.21.15.151:0/2638765990 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7bd007b010 0x7f7bd007d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:28.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.290+0000 7f7bef400640 1 --2- 172.21.15.151:0/2638765990 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7bd007b010 0x7f7bd007d4d0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f7be0033000 tx=0x7f7be0004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:28.294 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.293+0000 7f7be7400640 1 -- 172.21.15.151:0/2638765990 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7bdc020020 con 0x7f7bf00ccb00 2024-09-16T15:56:28.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.477+0000 7f7bf6800640 1 -- 172.21.15.151:0/2638765990 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/max_concurrent_clones}] v 0) -- 0x7f7bf0069a10 con 0x7f7bf00ccb00 2024-09-16T15:56:28.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.477+0000 7f7be7400640 1 -- 172.21.15.151:0/2638765990 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/max_concurrent_clones}]=0 v56)=0 v56) ==== 131+0+0 (secure 0 0 0) 0x7f7bdc063a30 con 0x7f7bf00ccb00 2024-09-16T15:56:28.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.479+0000 7f7bf6800640 1 -- 172.21.15.151:0/2638765990 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7bd007b010 msgr2=0x7f7bd007d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:28.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.479+0000 7f7bf6800640 1 --2- 172.21.15.151:0/2638765990 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7bd007b010 0x7f7bd007d4d0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f7be0033000 tx=0x7f7be0004730 comp rx=0 tx=0).stop 2024-09-16T15:56:28.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.479+0000 7f7bf6800640 1 -- 172.21.15.151:0/2638765990 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7bf00ccb00 msgr2=0x7f7bf013f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:28.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.479+0000 7f7bf6800640 1 --2- 172.21.15.151:0/2638765990 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7bf00ccb00 0x7f7bf013f080 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc00d720 tx=0x7f7bdc00dbe0 comp rx=0 tx=0).stop 2024-09-16T15:56:28.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.479+0000 7f7bf6800640 1 -- 172.21.15.151:0/2638765990 shutdown_connections 2024-09-16T15:56:28.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.479+0000 7f7bf6800640 1 --2- 172.21.15.151:0/2638765990 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7bd007b010 0x7f7bd007d4d0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:28.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.479+0000 7f7bf6800640 1 --2- 172.21.15.151:0/2638765990 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7bf0145750 0x7f7bf00adfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:28.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.479+0000 7f7bf6800640 1 --2- 172.21.15.151:0/2638765990 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7bf00ccb00 0x7f7bf013f080 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:28.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.479+0000 7f7bf6800640 1 --2- 172.21.15.151:0/2638765990 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7bf006efa0 0x7f7bf01f8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:28.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.479+0000 7f7bf6800640 1 -- 172.21.15.151:0/2638765990 >> 172.21.15.151:0/2638765990 conn(0x7f7bf00b1840 msgr2=0x7f7bf00cfd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:28.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.480+0000 7f7bf6800640 1 -- 172.21.15.151:0/2638765990 shutdown_connections 2024-09-16T15:56:28.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.480+0000 7f7bf6800640 1 -- 172.21.15.151:0/2638765990 wait complete. 2024-09-16T15:56:28.497 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-16T15:56:28.497 DEBUG:teuthology.orchestra.run.smithi151:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:56:28.584 DEBUG:teuthology.orchestra.run.smithi167:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-16T15:56:28.667 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-16T15:56:28.667 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:56:28.667 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:56:28.667 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:56:28.703 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:56:28.703 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:56:28.778 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-16T15:56:28.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.976+0000 7f09e2800640 1 -- 172.21.15.151:0/2242868734 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f09dc0af3d0 msgr2=0x7f09dc06f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:28.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.976+0000 7f09e2800640 1 --2- 172.21.15.151:0/2242868734 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f09dc0af3d0 0x7f09dc06f660 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f09dc0752e0 tx=0x7f09c802a460 comp rx=0 tx=0).stop 2024-09-16T15:56:28.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.977+0000 7f09e2800640 1 -- 172.21.15.151:0/2242868734 shutdown_connections 2024-09-16T15:56:28.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.977+0000 7f09e2800640 1 --2- 172.21.15.151:0/2242868734 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f09dc0af3d0 0x7f09dc06f660 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:28.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.977+0000 7f09e2800640 1 -- 172.21.15.151:0/2242868734 >> 172.21.15.151:0/2242868734 conn(0x7f09dc0b1950 msgr2=0x7f09dc0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:28.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.977+0000 7f09e2800640 1 -- 172.21.15.151:0/2242868734 shutdown_connections 2024-09-16T15:56:28.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.977+0000 7f09e2800640 1 -- 172.21.15.151:0/2242868734 wait complete. 2024-09-16T15:56:28.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.978+0000 7f09e2800640 1 Processor -- start 2024-09-16T15:56:28.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.978+0000 7f09e2800640 1 -- start start 2024-09-16T15:56:28.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.979+0000 7f09e2800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f09dc072df0 0x7f09dc14ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:28.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.979+0000 7f09e2800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f09dc0af3d0 0x7f09dc09ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:28.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.979+0000 7f09e2800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09dc161550 0x7f09dc09b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:28.982 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.979+0000 7f09e2800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f09dc073950 con 0x7f09dc072df0 2024-09-16T15:56:28.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.979+0000 7f09e2800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f09dc06fdb0 con 0x7f09dc161550 2024-09-16T15:56:28.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.979+0000 7f09e2800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f09dc073ad0 con 0x7f09dc0af3d0 2024-09-16T15:56:28.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.980+0000 7f09db400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f09dc072df0 0x7f09dc14ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:28.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.980+0000 7f09db400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f09dc072df0 0x7f09dc14ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43584/0 (socket says 172.21.15.151:43584) 2024-09-16T15:56:28.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.980+0000 7f09db400640 1 -- 172.21.15.151:0/1850281245 learned_addr learned my addr 172.21.15.151:0/1850281245 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:56:28.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.980+0000 7f09daa00640 1 --2- 172.21.15.151:0/1850281245 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f09dc0af3d0 0x7f09dc09ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:28.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.980+0000 7f09dbe00640 1 --2- 172.21.15.151:0/1850281245 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09dc161550 0x7f09dc09b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:28.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.980+0000 7f09db400640 1 -- 172.21.15.151:0/1850281245 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f09dc0af3d0 msgr2=0x7f09dc09ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:28.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.980+0000 7f09db400640 1 --2- 172.21.15.151:0/1850281245 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f09dc0af3d0 0x7f09dc09ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:28.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.980+0000 7f09db400640 1 -- 172.21.15.151:0/1850281245 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09dc161550 msgr2=0x7f09dc09b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:28.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.980+0000 7f09db400640 1 --2- 172.21.15.151:0/1850281245 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09dc161550 0x7f09dc09b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:28.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.980+0000 7f09db400640 1 -- 172.21.15.151:0/1850281245 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09dc1562d0 con 0x7f09dc072df0 2024-09-16T15:56:28.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.980+0000 7f09dbe00640 1 --2- 172.21.15.151:0/1850281245 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09dc161550 0x7f09dc09b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:56:28.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.980+0000 7f09db400640 1 --2- 172.21.15.151:0/1850281245 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f09dc072df0 0x7f09dc14ed30 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f09cc00cbc0 tx=0x7f09cc00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:28.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.981+0000 7f09d3400640 1 -- 172.21.15.151:0/1850281245 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09cc016020 con 0x7f09dc072df0 2024-09-16T15:56:28.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.981+0000 7f09d3400640 1 -- 172.21.15.151:0/1850281245 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f09cc00ad90 con 0x7f09dc072df0 2024-09-16T15:56:28.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.981+0000 7f09d3400640 1 -- 172.21.15.151:0/1850281245 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09cc012680 con 0x7f09dc072df0 2024-09-16T15:56:28.984 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.981+0000 7f09e2800640 1 -- 172.21.15.151:0/1850281245 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09dc1565b0 con 0x7f09dc072df0 2024-09-16T15:56:28.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.981+0000 7f09e2800640 1 -- 172.21.15.151:0/1850281245 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09dc0ce7d0 con 0x7f09dc072df0 2024-09-16T15:56:28.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.982+0000 7f09e2800640 1 -- 172.21.15.151:0/1850281245 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09dc073fc0 con 0x7f09dc072df0 2024-09-16T15:56:28.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.983+0000 7f09d3400640 1 -- 172.21.15.151:0/1850281245 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 55) ==== 104857+0+0 (secure 0 0 0) 0x7f09cc019020 con 0x7f09dc072df0 2024-09-16T15:56:28.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.984+0000 7f09d3400640 1 --2- 172.21.15.151:0/1850281245 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f09bc07afb0 0x7f09bc07d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:28.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.984+0000 7f09d3400640 1 -- 172.21.15.151:0/1850281245 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5963+0+0 (secure 0 0 0) 0x7f09cc09a2a0 con 0x7f09dc072df0 2024-09-16T15:56:28.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.985+0000 7f09daa00640 1 --2- 172.21.15.151:0/1850281245 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f09bc07afb0 0x7f09bc07d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:28.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.985+0000 7f09daa00640 1 --2- 172.21.15.151:0/1850281245 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f09bc07afb0 0x7f09bc07d470 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f09c8034000 tx=0x7f09c8004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:28.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:28.988+0000 7f09d3400640 1 -- 172.21.15.151:0/1850281245 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f09cc020020 con 0x7f09dc072df0 2024-09-16T15:56:29.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:29.183+0000 7f09e2800640 1 -- 172.21.15.151:0/1850281245 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f09dc06e9f0 con 0x7f09dc072df0 2024-09-16T15:56:29.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:29.184+0000 7f09d3400640 1 -- 172.21.15.151:0/1850281245 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v148) ==== 80+0+0 (secure 0 0 0) 0x7f09cc064370 con 0x7f09dc072df0 2024-09-16T15:56:29.186 INFO:teuthology.orchestra.run.smithi151.stderr:listed 0 entries 2024-09-16T15:56:29.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:29.186+0000 7f09e2800640 1 -- 172.21.15.151:0/1850281245 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f09bc07afb0 msgr2=0x7f09bc07d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:29.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:29.186+0000 7f09e2800640 1 --2- 172.21.15.151:0/1850281245 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f09bc07afb0 0x7f09bc07d470 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f09c8034000 tx=0x7f09c8004730 comp rx=0 tx=0).stop 2024-09-16T15:56:29.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:29.186+0000 7f09e2800640 1 -- 172.21.15.151:0/1850281245 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f09dc072df0 msgr2=0x7f09dc14ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:29.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:29.186+0000 7f09e2800640 1 --2- 172.21.15.151:0/1850281245 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f09dc072df0 0x7f09dc14ed30 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f09cc00cbc0 tx=0x7f09cc00a060 comp rx=0 tx=0).stop 2024-09-16T15:56:29.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:29.186+0000 7f09e2800640 1 -- 172.21.15.151:0/1850281245 shutdown_connections 2024-09-16T15:56:29.188 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:29.186+0000 7f09e2800640 1 --2- 172.21.15.151:0/1850281245 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f09bc07afb0 0x7f09bc07d470 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:29.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:29.186+0000 7f09e2800640 1 --2- 172.21.15.151:0/1850281245 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09dc161550 0x7f09dc09b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:29.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:29.186+0000 7f09e2800640 1 --2- 172.21.15.151:0/1850281245 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f09dc0af3d0 0x7f09dc09ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:29.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:29.186+0000 7f09e2800640 1 --2- 172.21.15.151:0/1850281245 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f09dc072df0 0x7f09dc14ed30 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:29.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:29.186+0000 7f09e2800640 1 -- 172.21.15.151:0/1850281245 >> 172.21.15.151:0/1850281245 conn(0x7f09dc0b1950 msgr2=0x7f09dc15ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:29.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:29.186+0000 7f09e2800640 1 -- 172.21.15.151:0/1850281245 shutdown_connections 2024-09-16T15:56:29.189 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:29.186+0000 7f09e2800640 1 -- 172.21.15.151:0/1850281245 wait complete. 2024-09-16T15:56:29.205 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-16T15:56:29.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:56:29.205 DEBUG:teuthology.orchestra.run.smithi151:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-16T15:56:29.301 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi151.front.sepia.ceph.com 2024-09-16T15:56:29.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:56:29.301 DEBUG:teuthology.orchestra.run.smithi151:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-16T15:56:29.345 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:56:29.345 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-16T15:56:29.345 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:56:29.383 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:56:29.383 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:56:29.383 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:56:29.383 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:56:29.421 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T15:56:29.421 INFO:tasks.cephfs.mount:Terminating background process 2024-09-16T15:56:29.421 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-16T15:56:29.421 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:56:29.459 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:56:29.459 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:56:29.654 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:29.652+0000 7f0a59600640 1 -- 172.21.15.151:0/1158103837 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a540b4100 msgr2=0x7f0a540b1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:29.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:29.652+0000 7f0a59600640 1 --2- 172.21.15.151:0/1158103837 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a540b4100 0x7f0a540b1770 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f0a540d4ea0 tx=0x7f0a4c02a7f0 comp rx=0 tx=0).stop 2024-09-16T15:56:29.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:29.653+0000 7f0a59600640 1 -- 172.21.15.151:0/1158103837 shutdown_connections 2024-09-16T15:56:29.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:29.653+0000 7f0a59600640 1 --2- 172.21.15.151:0/1158103837 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a540b4100 0x7f0a540b1770 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:29.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:29.653+0000 7f0a59600640 1 --2- 172.21.15.151:0/1158103837 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a540b5370 0x7f0a540b3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:29.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:29.653+0000 7f0a59600640 1 -- 172.21.15.151:0/1158103837 >> 172.21.15.151:0/1158103837 conn(0x7f0a540d3df0 msgr2=0x7f0a540d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:29.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:29.653+0000 7f0a59600640 1 -- 172.21.15.151:0/1158103837 shutdown_connections 2024-09-16T15:56:29.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:29.653+0000 7f0a59600640 1 -- 172.21.15.151:0/1158103837 wait complete. 2024-09-16T15:56:29.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:29.656+0000 7f0a59600640 1 Processor -- start 2024-09-16T15:56:29.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:29.658+0000 7f0a59600640 1 -- start start 2024-09-16T15:56:29.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:29.659+0000 7f0a59600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a540b4100 0x7f0a5414d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:29.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:29.659+0000 7f0a59600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a540b5370 0x7f0a540cfbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:29.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:29.659+0000 7f0a59600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a54161440 0x7f0a5414ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:29.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:29.659+0000 7f0a59600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0a540b2040 con 0x7f0a540b5370 2024-09-16T15:56:29.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:29.659+0000 7f0a59600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0a540b1ec0 con 0x7f0a540b4100 2024-09-16T15:56:29.661 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:29.659+0000 7f0a59600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f0a540b21c0 con 0x7f0a54161440 2024-09-16T15:56:29.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:29.660+0000 7f0a53e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a540b4100 0x7f0a5414d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:29.662 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:29.660+0000 7f0a53e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a540b4100 0x7f0a5414d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.151:35076/0 (socket says 172.21.15.151:35076) 2024-09-16T15:56:29.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:29.660+0000 7f0a53e00640 1 -- 172.21.15.151:0/2741523618 learned_addr learned my addr 172.21.15.151:0/2741523618 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:56:29.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:29.661+0000 7f0a58c00640 1 --2- 172.21.15.151:0/2741523618 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a54161440 0x7f0a5414ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:29.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:29.661+0000 7f0a53400640 1 --2- 172.21.15.151:0/2741523618 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a540b5370 0x7f0a540cfbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:29.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:29.661+0000 7f0a53e00640 1 -- 172.21.15.151:0/2741523618 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a54161440 msgr2=0x7f0a5414ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:29.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:29.661+0000 7f0a53e00640 1 --2- 172.21.15.151:0/2741523618 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a54161440 0x7f0a5414ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:29.663 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:29.661+0000 7f0a53e00640 1 -- 172.21.15.151:0/2741523618 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a540b5370 msgr2=0x7f0a540cfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:29.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:29.662+0000 7f0a53e00640 1 --2- 172.21.15.151:0/2741523618 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a540b5370 0x7f0a540cfbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:29.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:29.662+0000 7f0a53e00640 1 -- 172.21.15.151:0/2741523618 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a5415dfc0 con 0x7f0a540b4100 2024-09-16T15:56:29.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:29.662+0000 7f0a53e00640 1 --2- 172.21.15.151:0/2741523618 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a540b4100 0x7f0a5414d430 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f0a4400c3a0 tx=0x7f0a4400c860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:29.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:29.663+0000 7f0a4be00640 1 -- 172.21.15.151:0/2741523618 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a44011790 con 0x7f0a540b4100 2024-09-16T15:56:29.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:29.663+0000 7f0a4be00640 1 -- 172.21.15.151:0/2741523618 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0a44017070 con 0x7f0a540b4100 2024-09-16T15:56:29.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:29.663+0000 7f0a4be00640 1 -- 172.21.15.151:0/2741523618 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a44011cc0 con 0x7f0a540b4100 2024-09-16T15:56:29.665 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:29.663+0000 7f0a59600640 1 -- 172.21.15.151:0/2741523618 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a540cf440 con 0x7f0a540b4100 2024-09-16T15:56:29.666 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:29.663+0000 7f0a59600640 1 -- 172.21.15.151:0/2741523618 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a5415dce0 con 0x7f0a540b4100 2024-09-16T15:56:29.667 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:29.665+0000 7f0a59600640 1 -- 172.21.15.151:0/2741523618 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a540b1770 con 0x7f0a540b4100 2024-09-16T15:56:29.669 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:29.667+0000 7f0a4be00640 1 -- 172.21.15.151:0/2741523618 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 55) ==== 104857+0+0 (secure 0 0 0) 0x7f0a44004030 con 0x7f0a540b4100 2024-09-16T15:56:29.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:29.668+0000 7f0a4be00640 1 --2- 172.21.15.151:0/2741523618 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0a3407afb0 0x7f0a3407d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:29.670 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:29.669+0000 7f0a4be00640 1 -- 172.21.15.151:0/2741523618 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5963+0+0 (secure 0 0 0) 0x7f0a44099df0 con 0x7f0a540b4100 2024-09-16T15:56:29.671 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:29.669+0000 7f0a53400640 1 --2- 172.21.15.151:0/2741523618 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0a3407afb0 0x7f0a3407d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:29.674 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:29.672+0000 7f0a53400640 1 --2- 172.21.15.151:0/2741523618 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0a3407afb0 0x7f0a3407d470 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f0a540ce1f0 tx=0x7f0a40005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:29.676 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:29.674+0000 7f0a4be00640 1 -- 172.21.15.151:0/2741523618 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f0a44064040 con 0x7f0a540b4100 2024-09-16T15:56:30.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.105+0000 7f0a59600640 1 -- 172.21.15.151:0/2741523618 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0a5414dc80 con 0x7f0a540b4100 2024-09-16T15:56:30.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.106+0000 7f0a4be00640 1 -- 172.21.15.151:0/2741523618 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) ==== 94+0+5236 (secure 0 0 0) 0x7f0a44063940 con 0x7f0a540b4100 2024-09-16T15:56:30.108 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:56:30.108 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":97,"btime":"2024-09-16T15:55:09:523485+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7739,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2795044069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2795044069},{"type":"v1","addr":"172.21.15.167:6837","nonce":2795044069}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":8644,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/675530853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":675530853},{"type":"v1","addr":"172.21.15.151:6835","nonce":675530853}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":9468,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/1930858450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1930858450},{"type":"v1","addr":"172.21.15.151:6834","nonce":1930858450}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:54:55.864255+0000","modified":"2024-09-16T15:54:58.898875+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9474},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9474":{"gid":9474,"name":"b","rank":0,"incarnation":93,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6836/1231498178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1231498178},{"type":"v1","addr":"172.21.15.167:6836","nonce":1231498178}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9474,"qdb_cluster":[9474]},"id":10}]} 2024-09-16T15:56:30.108 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 97 2024-09-16T15:56:30.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.108+0000 7f0a59600640 1 -- 172.21.15.151:0/2741523618 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0a3407afb0 msgr2=0x7f0a3407d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:30.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.108+0000 7f0a59600640 1 --2- 172.21.15.151:0/2741523618 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0a3407afb0 0x7f0a3407d470 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f0a540ce1f0 tx=0x7f0a40005f90 comp rx=0 tx=0).stop 2024-09-16T15:56:30.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.108+0000 7f0a59600640 1 -- 172.21.15.151:0/2741523618 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a540b4100 msgr2=0x7f0a5414d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:30.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.108+0000 7f0a59600640 1 --2- 172.21.15.151:0/2741523618 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a540b4100 0x7f0a5414d430 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f0a4400c3a0 tx=0x7f0a4400c860 comp rx=0 tx=0).stop 2024-09-16T15:56:30.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.108+0000 7f0a59600640 1 -- 172.21.15.151:0/2741523618 shutdown_connections 2024-09-16T15:56:30.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.108+0000 7f0a59600640 1 --2- 172.21.15.151:0/2741523618 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f0a3407afb0 0x7f0a3407d470 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:30.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.108+0000 7f0a59600640 1 --2- 172.21.15.151:0/2741523618 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0a54161440 0x7f0a5414ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:30.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.108+0000 7f0a59600640 1 --2- 172.21.15.151:0/2741523618 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0a540b5370 0x7f0a540cfbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:30.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.108+0000 7f0a59600640 1 --2- 172.21.15.151:0/2741523618 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a540b4100 0x7f0a5414d430 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:30.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.108+0000 7f0a59600640 1 -- 172.21.15.151:0/2741523618 >> 172.21.15.151:0/2741523618 conn(0x7f0a540d3df0 msgr2=0x7f0a54151380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:30.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.108+0000 7f0a59600640 1 -- 172.21.15.151:0/2741523618 shutdown_connections 2024-09-16T15:56:30.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.109+0000 7f0a59600640 1 -- 172.21.15.151:0/2741523618 wait complete. 2024-09-16T15:56:30.127 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:56:30.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.282+0000 7f39ab000640 1 -- 172.21.15.151:0/4201805322 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39a415f5a0 msgr2=0x7f39a40b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:30.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.282+0000 7f39ab000640 1 --2- 172.21.15.151:0/4201805322 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39a415f5a0 0x7f39a40b2440 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f39a40cfbb0 tx=0x7f399402a560 comp rx=0 tx=0).stop 2024-09-16T15:56:30.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.282+0000 7f39ab000640 1 -- 172.21.15.151:0/4201805322 shutdown_connections 2024-09-16T15:56:30.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.282+0000 7f39ab000640 1 --2- 172.21.15.151:0/4201805322 >> v2:172.21.15.167:3300/0 conn(0x7f39a40ccf00 0x7f39a40acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:30.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.282+0000 7f39ab000640 1 --2- 172.21.15.151:0/4201805322 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39a415f5a0 0x7f39a40b2440 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:30.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.282+0000 7f39ab000640 1 -- 172.21.15.151:0/4201805322 >> 172.21.15.151:0/4201805322 conn(0x7f39a40b1950 msgr2=0x7f39a40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:30.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.282+0000 7f39ab000640 1 -- 172.21.15.151:0/4201805322 shutdown_connections 2024-09-16T15:56:30.285 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.283+0000 7f39ab000640 1 -- 172.21.15.151:0/4201805322 wait complete. 2024-09-16T15:56:30.286 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.284+0000 7f39ab000640 1 Processor -- start 2024-09-16T15:56:30.286 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.284+0000 7f39ab000640 1 -- start start 2024-09-16T15:56:30.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.285+0000 7f39ab000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39a40ad270 0x7f39a40cc700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:30.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.285+0000 7f39ab000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39a40ccf00 0x7f39a41f1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:30.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.285+0000 7f39ab000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f39a415f5a0 0x7f39a41451b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:30.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.285+0000 7f39ab000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f39a406f160 con 0x7f39a40ccf00 2024-09-16T15:56:30.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.285+0000 7f39ab000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f39a406f2e0 con 0x7f39a40ad270 2024-09-16T15:56:30.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.285+0000 7f39ab000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f39a406f460 con 0x7f39a415f5a0 2024-09-16T15:56:30.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.285+0000 7f39a3400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39a40ccf00 0x7f39a41f1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:30.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.285+0000 7f39a3400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39a40ccf00 0x7f39a41f1ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43620/0 (socket says 172.21.15.151:43620) 2024-09-16T15:56:30.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.285+0000 7f39a3400640 1 -- 172.21.15.151:0/2283529903 learned_addr learned my addr 172.21.15.151:0/2283529903 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:56:30.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.286+0000 7f39a3e00640 1 --2- 172.21.15.151:0/2283529903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39a40ad270 0x7f39a40cc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:30.290 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.286+0000 7f39a3400640 1 -- 172.21.15.151:0/2283529903 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f39a415f5a0 msgr2=0x7f39a41451b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:30.290 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.286+0000 7f39a3400640 1 --2- 172.21.15.151:0/2283529903 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f39a415f5a0 0x7f39a41451b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:30.290 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.286+0000 7f39a3400640 1 -- 172.21.15.151:0/2283529903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39a40ad270 msgr2=0x7f39a40cc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:30.290 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.286+0000 7f39a3400640 1 --2- 172.21.15.151:0/2283529903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39a40ad270 0x7f39a40cc700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:30.290 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.286+0000 7f39a3400640 1 -- 172.21.15.151:0/2283529903 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39a40c9f40 con 0x7f39a40ccf00 2024-09-16T15:56:30.290 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.286+0000 7f39a3e00640 1 --2- 172.21.15.151:0/2283529903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39a40ad270 0x7f39a40cc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:56:30.290 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.286+0000 7f39a3400640 1 --2- 172.21.15.151:0/2283529903 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39a40ccf00 0x7f39a41f1ed0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f399000ea20 tx=0x7f399000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:30.290 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.286+0000 7f399be00640 1 -- 172.21.15.151:0/2283529903 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f399000cd20 con 0x7f39a40ccf00 2024-09-16T15:56:30.290 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.286+0000 7f399be00640 1 -- 172.21.15.151:0/2283529903 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3990005020 con 0x7f39a40ccf00 2024-09-16T15:56:30.290 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.286+0000 7f399be00640 1 -- 172.21.15.151:0/2283529903 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3990005450 con 0x7f39a40ccf00 2024-09-16T15:56:30.290 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.286+0000 7f39ab000640 1 -- 172.21.15.151:0/2283529903 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39a40c61d0 con 0x7f39a40ccf00 2024-09-16T15:56:30.290 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.286+0000 7f39ab000640 1 -- 172.21.15.151:0/2283529903 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39a40c6790 con 0x7f39a40ccf00 2024-09-16T15:56:30.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.287+0000 7f39ab000640 1 -- 172.21.15.151:0/2283529903 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39a406f9f0 con 0x7f39a40ccf00 2024-09-16T15:56:30.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.289+0000 7f399be00640 1 -- 172.21.15.151:0/2283529903 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 55) ==== 104857+0+0 (secure 0 0 0) 0x7f39900047a0 con 0x7f39a40ccf00 2024-09-16T15:56:30.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.290+0000 7f399be00640 1 --2- 172.21.15.151:0/2283529903 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f398407afb0 0x7f398407d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:30.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.290+0000 7f399be00640 1 -- 172.21.15.151:0/2283529903 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5963+0+0 (secure 0 0 0) 0x7f399009a7a0 con 0x7f39a40ccf00 2024-09-16T15:56:30.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.290+0000 7f39a3e00640 1 --2- 172.21.15.151:0/2283529903 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f398407afb0 0x7f398407d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:30.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.291+0000 7f39a3e00640 1 --2- 172.21.15.151:0/2283529903 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f398407afb0 0x7f398407d470 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f3994033000 tx=0x7f3994004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:30.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.294+0000 7f399be00640 1 -- 172.21.15.151:0/2283529903 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f39900647f0 con 0x7f39a40ccf00 2024-09-16T15:56:30.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.514+0000 7f39ab000640 1 -- 172.21.15.151:0/2283529903 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f39a409b260 con 0x7f39a40ccf00 2024-09-16T15:56:30.516 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.514+0000 7f399be00640 1 -- 172.21.15.151:0/2283529903 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) ==== 94+0+5236 (secure 0 0 0) 0x7f39900640f0 con 0x7f39a40ccf00 2024-09-16T15:56:30.516 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:56:30.517 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":97,"btime":"2024-09-16T15:55:09:523485+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7739,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2795044069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2795044069},{"type":"v1","addr":"172.21.15.167:6837","nonce":2795044069}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":8644,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/675530853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":675530853},{"type":"v1","addr":"172.21.15.151:6835","nonce":675530853}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":9468,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/1930858450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1930858450},{"type":"v1","addr":"172.21.15.151:6834","nonce":1930858450}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:54:55.864255+0000","modified":"2024-09-16T15:54:58.898875+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9474},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9474":{"gid":9474,"name":"b","rank":0,"incarnation":93,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6836/1231498178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1231498178},{"type":"v1","addr":"172.21.15.167:6836","nonce":1231498178}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9474,"qdb_cluster":[9474]},"id":10}]} 2024-09-16T15:56:30.517 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 97 2024-09-16T15:56:30.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.517+0000 7f39ab000640 1 -- 172.21.15.151:0/2283529903 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f398407afb0 msgr2=0x7f398407d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:30.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.517+0000 7f39ab000640 1 --2- 172.21.15.151:0/2283529903 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f398407afb0 0x7f398407d470 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f3994033000 tx=0x7f3994004730 comp rx=0 tx=0).stop 2024-09-16T15:56:30.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.517+0000 7f39ab000640 1 -- 172.21.15.151:0/2283529903 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39a40ccf00 msgr2=0x7f39a41f1ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:30.519 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.517+0000 7f39ab000640 1 --2- 172.21.15.151:0/2283529903 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39a40ccf00 0x7f39a41f1ed0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f399000ea20 tx=0x7f399000ee20 comp rx=0 tx=0).stop 2024-09-16T15:56:30.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.517+0000 7f39ab000640 1 -- 172.21.15.151:0/2283529903 shutdown_connections 2024-09-16T15:56:30.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.517+0000 7f39ab000640 1 --2- 172.21.15.151:0/2283529903 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f398407afb0 0x7f398407d470 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:30.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.517+0000 7f39ab000640 1 --2- 172.21.15.151:0/2283529903 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f39a415f5a0 0x7f39a41451b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:30.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.517+0000 7f39ab000640 1 --2- 172.21.15.151:0/2283529903 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f39a40ccf00 0x7f39a41f1ed0 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:30.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.517+0000 7f39ab000640 1 --2- 172.21.15.151:0/2283529903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f39a40ad270 0x7f39a40cc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:30.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.517+0000 7f39ab000640 1 -- 172.21.15.151:0/2283529903 >> 172.21.15.151:0/2283529903 conn(0x7f39a40b1950 msgr2=0x7f39a415e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:30.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.517+0000 7f39ab000640 1 -- 172.21.15.151:0/2283529903 shutdown_connections 2024-09-16T15:56:30.520 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.517+0000 7f39ab000640 1 -- 172.21.15.151:0/2283529903 wait complete. 2024-09-16T15:56:30.535 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:56:30.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.774+0000 7fd0a6600640 1 -- 172.21.15.151:0/2466174597 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0a00ca4d0 msgr2=0x7fd0a015e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:30.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.774+0000 7fd0a6600640 1 --2- 172.21.15.151:0/2466174597 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0a00ca4d0 0x7fd0a015e390 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7fd090009830 tx=0x7fd09002a4a0 comp rx=0 tx=0).stop 2024-09-16T15:56:30.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.775+0000 7fd0a6600640 1 -- 172.21.15.151:0/2466174597 shutdown_connections 2024-09-16T15:56:30.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.775+0000 7fd0a6600640 1 --2- 172.21.15.151:0/2466174597 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0a00ca4d0 0x7fd0a015e390 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:30.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.775+0000 7fd0a6600640 1 --2- 172.21.15.151:0/2466174597 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0a00709e0 0x7fd0a0073af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:30.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.775+0000 7fd0a6600640 1 -- 172.21.15.151:0/2466174597 >> 172.21.15.151:0/2466174597 conn(0x7fd0a00b2b90 msgr2=0x7fd0a00b2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:30.777 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.775+0000 7fd0a6600640 1 -- 172.21.15.151:0/2466174597 shutdown_connections 2024-09-16T15:56:30.778 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.775+0000 7fd0a6600640 1 -- 172.21.15.151:0/2466174597 wait complete. 2024-09-16T15:56:30.779 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.778+0000 7fd0a6600640 1 Processor -- start 2024-09-16T15:56:30.780 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.778+0000 7fd0a6600640 1 -- start start 2024-09-16T15:56:30.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.779+0000 7fd0a6600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0a006cb70 0x7fd0a0154810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:30.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.779+0000 7fd0a6600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0a00709e0 0x7fd0a0159b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:30.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.779+0000 7fd0a6600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0a00ca4d0 0x7fd0a015a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:30.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.779+0000 7fd0a6600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a00d0460 con 0x7fd0a00ca4d0 2024-09-16T15:56:30.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.779+0000 7fd0a6600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a00d02e0 con 0x7fd0a006cb70 2024-09-16T15:56:30.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.779+0000 7fd0a6600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fd0a00cf220 con 0x7fd0a00709e0 2024-09-16T15:56:30.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.780+0000 7fd09fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0a00ca4d0 0x7fd0a015a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:30.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.780+0000 7fd09f400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0a006cb70 0x7fd0a0154810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:30.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.780+0000 7fd09f400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0a006cb70 0x7fd0a0154810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.151:35114/0 (socket says 172.21.15.151:35114) 2024-09-16T15:56:30.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.780+0000 7fd09f400640 1 -- 172.21.15.151:0/1907238985 learned_addr learned my addr 172.21.15.151:0/1907238985 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:56:30.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.780+0000 7fd09ea00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0a00709e0 0x7fd0a0159b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:30.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.780+0000 7fd09f400640 1 -- 172.21.15.151:0/1907238985 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0a00709e0 msgr2=0x7fd0a0159b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:30.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.780+0000 7fd09f400640 1 --2- 172.21.15.151:0/1907238985 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0a00709e0 0x7fd0a0159b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:30.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.780+0000 7fd09f400640 1 -- 172.21.15.151:0/1907238985 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0a00ca4d0 msgr2=0x7fd0a015a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:30.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.781+0000 7fd09f400640 1 --2- 172.21.15.151:0/1907238985 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0a00ca4d0 0x7fd0a015a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:30.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.781+0000 7fd09f400640 1 -- 172.21.15.151:0/1907238985 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0a00b5a30 con 0x7fd0a006cb70 2024-09-16T15:56:30.783 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.781+0000 7fd09fe00640 1 --2- 172.21.15.151:0/1907238985 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0a00ca4d0 0x7fd0a015a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T15:56:30.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.781+0000 7fd09f400640 1 --2- 172.21.15.151:0/1907238985 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0a006cb70 0x7fd0a0154810 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fd08800d950 tx=0x7fd08800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:30.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.782+0000 7fd097400640 1 -- 172.21.15.151:0/1907238985 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd088017070 con 0x7fd0a006cb70 2024-09-16T15:56:30.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.782+0000 7fd097400640 1 -- 172.21.15.151:0/1907238985 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd088009d20 con 0x7fd0a006cb70 2024-09-16T15:56:30.784 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.782+0000 7fd097400640 1 -- 172.21.15.151:0/1907238985 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0880106c0 con 0x7fd0a006cb70 2024-09-16T15:56:30.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.782+0000 7fd0a6600640 1 -- 172.21.15.151:0/1907238985 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0a00b5d10 con 0x7fd0a006cb70 2024-09-16T15:56:30.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.782+0000 7fd0a6600640 1 -- 172.21.15.151:0/1907238985 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0a00b62d0 con 0x7fd0a006cb70 2024-09-16T15:56:30.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.784+0000 7fd0a6600640 1 -- 172.21.15.151:0/1907238985 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0a006d350 con 0x7fd0a006cb70 2024-09-16T15:56:30.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.787+0000 7fd097400640 1 -- 172.21.15.151:0/1907238985 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 55) ==== 104857+0+0 (secure 0 0 0) 0x7fd088010860 con 0x7fd0a006cb70 2024-09-16T15:56:30.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.789+0000 7fd097400640 1 --2- 172.21.15.151:0/1907238985 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd07c07afe0 0x7fd07c07d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:30.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.790+0000 7fd097400640 1 -- 172.21.15.151:0/1907238985 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5963+0+0 (secure 0 0 0) 0x7fd08809a420 con 0x7fd0a006cb70 2024-09-16T15:56:30.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.790+0000 7fd09ea00640 1 --2- 172.21.15.151:0/1907238985 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd07c07afe0 0x7fd07c07d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:30.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.792+0000 7fd09ea00640 1 --2- 172.21.15.151:0/1907238985 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd07c07afe0 0x7fd07c07d4a0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fd090034000 tx=0x7fd090004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:30.797 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:30.795+0000 7fd097400640 1 -- 172.21.15.151:0/1907238985 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd088063f20 con 0x7fd0a006cb70 2024-09-16T15:56:31.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.053+0000 7fd0a6600640 1 -- 172.21.15.151:0/1907238985 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd0a00cfc10 con 0x7fd0a006cb70 2024-09-16T15:56:31.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.054+0000 7fd097400640 1 -- 172.21.15.151:0/1907238985 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v148) ==== 74+0+17171 (secure 0 0 0) 0x7fd088068950 con 0x7fd0a006cb70 2024-09-16T15:56:31.057 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:56:31.057 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":148,"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","created":"2024-09-16T15:38:56.348535+0000","modified":"2024-09-16T15:56:22.814921+0000","last_up_change":"2024-09-16T15:39:10.681384+0000","last_in_change":"2024-09-16T15:39:00.544477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:39:11.602252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:54:53.628710+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-16T15:54:54.832847+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":148,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5524076-a903-4d0a-af6a-caf474c4765d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6809","nonce":3111342741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6811","nonce":3111342741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6815","nonce":3111342741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6813","nonce":3111342741}]},"public_addr":"172.21.15.151:6809/3111342741","cluster_addr":"172.21.15.151:6811/3111342741","heartbeat_back_addr":"172.21.15.151:6815/3111342741","heartbeat_front_addr":"172.21.15.151:6813/3111342741","state":["exists","up"]},{"osd":1,"uuid":"dd25925f-0ae1-494e-8529-4dc205759db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6801","nonce":3368304087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6803","nonce":3368304087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6807","nonce":3368304087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6805","nonce":3368304087}]},"public_addr":"172.21.15.151:6801/3368304087","cluster_addr":"172.21.15.151:6803/3368304087","heartbeat_back_addr":"172.21.15.151:6807/3368304087","heartbeat_front_addr":"172.21.15.151:6805/3368304087","state":["exists","up"]},{"osd":2,"uuid":"c2f18115-013d-4da6-a0fe-cb978f4494b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6823","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6825","nonce":3079373271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6827","nonce":3079373271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6831","nonce":3079373271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6829","nonce":3079373271}]},"public_addr":"172.21.15.151:6825/3079373271","cluster_addr":"172.21.15.151:6827/3079373271","heartbeat_back_addr":"172.21.15.151:6831/3079373271","heartbeat_front_addr":"172.21.15.151:6829/3079373271","state":["exists","up"]},{"osd":3,"uuid":"36edb718-2fcb-4d88-8ddc-c321eed4279d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6817","nonce":1142509041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6819","nonce":1142509041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6824","nonce":1142509041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6821","nonce":1142509041}]},"public_addr":"172.21.15.151:6817/1142509041","cluster_addr":"172.21.15.151:6819/1142509041","heartbeat_back_addr":"172.21.15.151:6824/1142509041","heartbeat_front_addr":"172.21.15.151:6821/1142509041","state":["exists","up"]},{"osd":4,"uuid":"64d33708-cd0a-4236-abe3-1b646e82ab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6811","nonce":2735495271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6813","nonce":2735495271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6817","nonce":2735495271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6815","nonce":2735495271}]},"public_addr":"172.21.15.167:6811/2735495271","cluster_addr":"172.21.15.167:6813/2735495271","heartbeat_back_addr":"172.21.15.167:6817/2735495271","heartbeat_front_addr":"172.21.15.167:6815/2735495271","state":["exists","up"]},{"osd":5,"uuid":"f14b44a9-5910-458c-b518-09a8d0de94de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6827","nonce":1494916572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6829","nonce":1494916572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6833","nonce":1494916572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6831","nonce":1494916572}]},"public_addr":"172.21.15.167:6827/1494916572","cluster_addr":"172.21.15.167:6829/1494916572","heartbeat_back_addr":"172.21.15.167:6833/1494916572","heartbeat_front_addr":"172.21.15.167:6831/1494916572","state":["exists","up"]},{"osd":6,"uuid":"39335545-2654-4710-84b0-a9aa47cfcbba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6819","nonce":2538273273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6821","nonce":2538273273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6825","nonce":2538273273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6823","nonce":2538273273}]},"public_addr":"172.21.15.167:6819/2538273273","cluster_addr":"172.21.15.167:6821/2538273273","heartbeat_back_addr":"172.21.15.167:6825/2538273273","heartbeat_front_addr":"172.21.15.167:6823/2538273273","state":["exists","up"]},{"osd":7,"uuid":"4f8bfc81-53dd-4339-ac22-1a4007d9f963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6803","nonce":265072227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6805","nonce":265072227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6809","nonce":265072227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6807","nonce":265072227}]},"public_addr":"172.21.15.167:6803/265072227","cluster_addr":"172.21.15.167:6805/265072227","heartbeat_back_addr":"172.21.15.167:6809/265072227","heartbeat_front_addr":"172.21.15.167:6807/265072227","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.150359+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.084358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.722368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.332809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.320709+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.395153+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.935250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.801443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.2f","mappings":[{"from":2,"to":5}]},{"pgid":"21.3c","mappings":[{"from":2,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":2,"length":2}]},{"pool":21,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":21,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:56:31.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.057+0000 7fd0a6600640 1 -- 172.21.15.151:0/1907238985 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd07c07afe0 msgr2=0x7fd07c07d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:31.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.057+0000 7fd0a6600640 1 --2- 172.21.15.151:0/1907238985 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd07c07afe0 0x7fd07c07d4a0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fd090034000 tx=0x7fd090004730 comp rx=0 tx=0).stop 2024-09-16T15:56:31.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.057+0000 7fd0a6600640 1 -- 172.21.15.151:0/1907238985 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0a006cb70 msgr2=0x7fd0a0154810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:31.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.057+0000 7fd0a6600640 1 --2- 172.21.15.151:0/1907238985 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0a006cb70 0x7fd0a0154810 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fd08800d950 tx=0x7fd08800de10 comp rx=0 tx=0).stop 2024-09-16T15:56:31.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.057+0000 7fd0a6600640 1 -- 172.21.15.151:0/1907238985 shutdown_connections 2024-09-16T15:56:31.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.057+0000 7fd0a6600640 1 --2- 172.21.15.151:0/1907238985 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd07c07afe0 0x7fd07c07d4a0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:31.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.057+0000 7fd0a6600640 1 --2- 172.21.15.151:0/1907238985 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0a00ca4d0 0x7fd0a015a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:31.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.057+0000 7fd0a6600640 1 --2- 172.21.15.151:0/1907238985 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd0a00709e0 0x7fd0a0159b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:31.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.057+0000 7fd0a6600640 1 --2- 172.21.15.151:0/1907238985 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0a006cb70 0x7fd0a0154810 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:31.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.057+0000 7fd0a6600640 1 -- 172.21.15.151:0/1907238985 >> 172.21.15.151:0/1907238985 conn(0x7fd0a00b2b90 msgr2=0x7fd0a00b2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:31.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.057+0000 7fd0a6600640 1 -- 172.21.15.151:0/1907238985 shutdown_connections 2024-09-16T15:56:31.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.057+0000 7fd0a6600640 1 -- 172.21.15.151:0/1907238985 wait complete. 2024-09-16T15:56:31.075 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-16T15:56:31.076 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:56:31.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.231+0000 7fc012000640 1 -- 172.21.15.151:0/1832432702 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc00c073fb0 msgr2=0x7fc00c074440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:31.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.231+0000 7fc012000640 1 --2- 172.21.15.151:0/1832432702 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc00c073fb0 0x7fc00c074440 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc004005fd0 tx=0x7fc00402a820 comp rx=0 tx=0).stop 2024-09-16T15:56:31.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.232+0000 7fc012000640 1 -- 172.21.15.151:0/1832432702 shutdown_connections 2024-09-16T15:56:31.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.232+0000 7fc012000640 1 --2- 172.21.15.151:0/1832432702 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc00c073fb0 0x7fc00c074440 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:31.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.232+0000 7fc012000640 1 --2- 172.21.15.151:0/1832432702 >> v2:172.21.15.167:3300/0 conn(0x7fc00c0ccf00 0x7fc00c0b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:31.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.232+0000 7fc012000640 1 -- 172.21.15.151:0/1832432702 >> 172.21.15.151:0/1832432702 conn(0x7fc00c0b1950 msgr2=0x7fc00c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:31.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.232+0000 7fc012000640 1 -- 172.21.15.151:0/1832432702 shutdown_connections 2024-09-16T15:56:31.234 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.232+0000 7fc012000640 1 -- 172.21.15.151:0/1832432702 wait complete. 2024-09-16T15:56:31.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.233+0000 7fc012000640 1 Processor -- start 2024-09-16T15:56:31.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.233+0000 7fc012000640 1 -- start start 2024-09-16T15:56:31.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.234+0000 7fc012000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc00c073fb0 0x7fc00c0d01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:31.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.234+0000 7fc012000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc00c0ccf00 0x7fc00c0cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:31.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.234+0000 7fc012000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc00c15f5a0 0x7fc00c1f4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:31.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.234+0000 7fc012000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c06f2e0 con 0x7fc00c0ccf00 2024-09-16T15:56:31.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.234+0000 7fc012000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c06f160 con 0x7fc00c073fb0 2024-09-16T15:56:31.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.234+0000 7fc012000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fc00c06f460 con 0x7fc00c15f5a0 2024-09-16T15:56:31.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.235+0000 7fc00aa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc00c0ccf00 0x7fc00c0cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:31.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.235+0000 7fc00b400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc00c073fb0 0x7fc00c0d01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:31.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.235+0000 7fc00aa00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc00c0ccf00 0x7fc00c0cdae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43660/0 (socket says 172.21.15.151:43660) 2024-09-16T15:56:31.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.235+0000 7fc00aa00640 1 -- 172.21.15.151:0/848207526 learned_addr learned my addr 172.21.15.151:0/848207526 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:56:31.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.235+0000 7fc00be00640 1 --2- 172.21.15.151:0/848207526 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc00c15f5a0 0x7fc00c1f4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:31.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.235+0000 7fc00b400640 1 -- 172.21.15.151:0/848207526 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc00c15f5a0 msgr2=0x7fc00c1f4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:31.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.235+0000 7fc00b400640 1 --2- 172.21.15.151:0/848207526 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc00c15f5a0 0x7fc00c1f4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:31.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.235+0000 7fc00b400640 1 -- 172.21.15.151:0/848207526 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc00c0ccf00 msgr2=0x7fc00c0cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:31.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.235+0000 7fc00b400640 1 --2- 172.21.15.151:0/848207526 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc00c0ccf00 0x7fc00c0cdae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:31.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.235+0000 7fc00b400640 1 -- 172.21.15.151:0/848207526 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc00c15a370 con 0x7fc00c073fb0 2024-09-16T15:56:31.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.235+0000 7fc00b400640 1 --2- 172.21.15.151:0/848207526 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc00c073fb0 0x7fc00c0d01d0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fbffc00c3c0 tx=0x7fbffc00c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:31.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.236+0000 7fc003400640 1 -- 172.21.15.151:0/848207526 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbffc011790 con 0x7fc00c073fb0 2024-09-16T15:56:31.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.236+0000 7fc003400640 1 -- 172.21.15.151:0/848207526 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbffc017070 con 0x7fc00c073fb0 2024-09-16T15:56:31.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.236+0000 7fc003400640 1 -- 172.21.15.151:0/848207526 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbffc011d20 con 0x7fc00c073fb0 2024-09-16T15:56:31.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.236+0000 7fc012000640 1 -- 172.21.15.151:0/848207526 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc00c1f5350 con 0x7fc00c073fb0 2024-09-16T15:56:31.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.236+0000 7fc012000640 1 -- 172.21.15.151:0/848207526 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc00c1f5910 con 0x7fc00c073fb0 2024-09-16T15:56:31.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.237+0000 7fc012000640 1 -- 172.21.15.151:0/848207526 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc00c06f9f0 con 0x7fc00c073fb0 2024-09-16T15:56:31.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.238+0000 7fc003400640 1 -- 172.21.15.151:0/848207526 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 55) ==== 104857+0+0 (secure 0 0 0) 0x7fbffc01c460 con 0x7fc00c073fb0 2024-09-16T15:56:31.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.239+0000 7fc003400640 1 --2- 172.21.15.151:0/848207526 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbfec07afb0 0x7fbfec07d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:31.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.239+0000 7fc003400640 1 -- 172.21.15.151:0/848207526 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5963+0+0 (secure 0 0 0) 0x7fbffc0997d0 con 0x7fc00c073fb0 2024-09-16T15:56:31.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.239+0000 7fc00aa00640 1 --2- 172.21.15.151:0/848207526 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbfec07afb0 0x7fbfec07d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:31.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.240+0000 7fc00aa00640 1 --2- 172.21.15.151:0/848207526 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbfec07afb0 0x7fbfec07d470 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fbff80095f0 tx=0x7fbff8005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:31.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.244+0000 7fc003400640 1 -- 172.21.15.151:0/848207526 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc00c15a370 con 0x7fc00c073fb0 2024-09-16T15:56:31.468 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.466+0000 7fc012000640 1 -- 172.21.15.151:0/848207526 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc00c1552e0 con 0x7fc00c073fb0 2024-09-16T15:56:31.469 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.467+0000 7fc003400640 1 -- 172.21.15.151:0/848207526 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) ==== 94+0+5236 (secure 0 0 0) 0x7fbffc063250 con 0x7fc00c073fb0 2024-09-16T15:56:31.469 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:56:31.469 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":97,"btime":"2024-09-16T15:55:09:523485+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7739,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2795044069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2795044069},{"type":"v1","addr":"172.21.15.167:6837","nonce":2795044069}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":8644,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/675530853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":675530853},{"type":"v1","addr":"172.21.15.151:6835","nonce":675530853}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":9468,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/1930858450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1930858450},{"type":"v1","addr":"172.21.15.151:6834","nonce":1930858450}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:54:55.864255+0000","modified":"2024-09-16T15:54:58.898875+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9474},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9474":{"gid":9474,"name":"b","rank":0,"incarnation":93,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6836/1231498178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1231498178},{"type":"v1","addr":"172.21.15.167:6836","nonce":1231498178}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9474,"qdb_cluster":[9474]},"id":10}]} 2024-09-16T15:56:31.469 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 97 2024-09-16T15:56:31.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.469+0000 7fc012000640 1 -- 172.21.15.151:0/848207526 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbfec07afb0 msgr2=0x7fbfec07d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:31.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.469+0000 7fc012000640 1 --2- 172.21.15.151:0/848207526 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbfec07afb0 0x7fbfec07d470 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fbff80095f0 tx=0x7fbff8005f90 comp rx=0 tx=0).stop 2024-09-16T15:56:31.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.469+0000 7fc012000640 1 -- 172.21.15.151:0/848207526 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc00c073fb0 msgr2=0x7fc00c0d01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:31.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.469+0000 7fc012000640 1 --2- 172.21.15.151:0/848207526 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc00c073fb0 0x7fc00c0d01d0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fbffc00c3c0 tx=0x7fbffc00c880 comp rx=0 tx=0).stop 2024-09-16T15:56:31.472 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.470+0000 7fc012000640 1 -- 172.21.15.151:0/848207526 shutdown_connections 2024-09-16T15:56:31.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.470+0000 7fc012000640 1 --2- 172.21.15.151:0/848207526 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbfec07afb0 0x7fbfec07d470 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:31.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.470+0000 7fc012000640 1 --2- 172.21.15.151:0/848207526 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc00c15f5a0 0x7fc00c1f4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:31.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.470+0000 7fc012000640 1 --2- 172.21.15.151:0/848207526 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc00c0ccf00 0x7fc00c0cdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:31.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.470+0000 7fc012000640 1 --2- 172.21.15.151:0/848207526 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc00c073fb0 0x7fc00c0d01d0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:31.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.470+0000 7fc012000640 1 -- 172.21.15.151:0/848207526 >> 172.21.15.151:0/848207526 conn(0x7fc00c0b1950 msgr2=0x7fc00c09cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:31.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.470+0000 7fc012000640 1 -- 172.21.15.151:0/848207526 shutdown_connections 2024-09-16T15:56:31.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.470+0000 7fc012000640 1 -- 172.21.15.151:0/848207526 wait complete. 2024-09-16T15:56:31.488 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:56:31.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.644+0000 7f2675800640 1 -- 172.21.15.151:0/1337407413 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f26700b28d0 msgr2=0x7f2670162730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:31.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.644+0000 7f2675800640 1 --2- 172.21.15.151:0/1337407413 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f26700b28d0 0x7f2670162730 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f2668005fd0 tx=0x7f266802a9f0 comp rx=0 tx=0).stop 2024-09-16T15:56:31.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.645+0000 7f2675800640 1 -- 172.21.15.151:0/1337407413 shutdown_connections 2024-09-16T15:56:31.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.645+0000 7f2675800640 1 --2- 172.21.15.151:0/1337407413 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f26700b28d0 0x7f2670162730 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:31.647 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.645+0000 7f2675800640 1 --2- 172.21.15.151:0/1337407413 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26700d1ec0 0x7f26700d22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:31.648 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.645+0000 7f2675800640 1 -- 172.21.15.151:0/1337407413 >> 172.21.15.151:0/1337407413 conn(0x7f26700d3de0 msgr2=0x7f26700d41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:31.648 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.645+0000 7f2675800640 1 -- 172.21.15.151:0/1337407413 shutdown_connections 2024-09-16T15:56:31.648 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.645+0000 7f2675800640 1 -- 172.21.15.151:0/1337407413 wait complete. 2024-09-16T15:56:31.648 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.646+0000 7f2675800640 1 Processor -- start 2024-09-16T15:56:31.649 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.646+0000 7f2675800640 1 -- start start 2024-09-16T15:56:31.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.647+0000 7f2675800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f26700b28d0 0x7f2670161580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:31.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.647+0000 7f2675800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26700b4d10 0x7f26701f7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:31.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.647+0000 7f2675800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26700d1ec0 0x7f267015b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:31.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.647+0000 7f2675800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f26700b3a80 con 0x7f26700b28d0 2024-09-16T15:56:31.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.647+0000 7f2675800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f26700b3900 con 0x7f26700d1ec0 2024-09-16T15:56:31.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.647+0000 7f2675800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f26700b3c00 con 0x7f26700b4d10 2024-09-16T15:56:31.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.648+0000 7f266fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f26700b28d0 0x7f2670161580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:31.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.648+0000 7f266fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f26700b28d0 0x7f2670161580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43674/0 (socket says 172.21.15.151:43674) 2024-09-16T15:56:31.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.648+0000 7f266fe00640 1 -- 172.21.15.151:0/3494850033 learned_addr learned my addr 172.21.15.151:0/3494850033 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:56:31.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.648+0000 7f266f400640 1 --2- 172.21.15.151:0/3494850033 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26700b4d10 0x7f26701f7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:31.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.648+0000 7f2674e00640 1 --2- 172.21.15.151:0/3494850033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26700d1ec0 0x7f267015b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:31.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.648+0000 7f266fe00640 1 -- 172.21.15.151:0/3494850033 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26700b4d10 msgr2=0x7f26701f7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:31.652 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.648+0000 7f266fe00640 1 --2- 172.21.15.151:0/3494850033 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26700b4d10 0x7f26701f7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:31.653 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.648+0000 7f266fe00640 1 -- 172.21.15.151:0/3494850033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26700d1ec0 msgr2=0x7f267015b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:31.653 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.648+0000 7f266fe00640 1 --2- 172.21.15.151:0/3494850033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26700d1ec0 0x7f267015b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:31.653 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.648+0000 7f266fe00640 1 -- 172.21.15.151:0/3494850033 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f267015be80 con 0x7f26700b28d0 2024-09-16T15:56:31.653 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.648+0000 7f266fe00640 1 --2- 172.21.15.151:0/3494850033 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f26700b28d0 0x7f2670161580 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f266000c9a0 tx=0x7f266000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:31.653 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.648+0000 7f2667e00640 1 -- 172.21.15.151:0/3494850033 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2660007bb0 con 0x7f26700b28d0 2024-09-16T15:56:31.653 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.649+0000 7f2667e00640 1 -- 172.21.15.151:0/3494850033 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2660007e40 con 0x7f26700b28d0 2024-09-16T15:56:31.653 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.649+0000 7f2667e00640 1 -- 172.21.15.151:0/3494850033 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f266000f690 con 0x7f26700b28d0 2024-09-16T15:56:31.653 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.649+0000 7f2675800640 1 -- 172.21.15.151:0/3494850033 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26701f8210 con 0x7f26700b28d0 2024-09-16T15:56:31.653 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.649+0000 7f2675800640 1 -- 172.21.15.151:0/3494850033 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26701f87d0 con 0x7f26700b28d0 2024-09-16T15:56:31.653 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.650+0000 7f2675800640 1 -- 172.21.15.151:0/3494850033 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26700d22a0 con 0x7f26700b28d0 2024-09-16T15:56:31.654 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.651+0000 7f2667e00640 1 -- 172.21.15.151:0/3494850033 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 55) ==== 104857+0+0 (secure 0 0 0) 0x7f266000f830 con 0x7f26700b28d0 2024-09-16T15:56:31.654 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.652+0000 7f2667e00640 1 --2- 172.21.15.151:0/3494850033 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f265007afb0 0x7f265007d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:31.654 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.652+0000 7f2667e00640 1 -- 172.21.15.151:0/3494850033 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5935+0+0 (secure 0 0 0) 0x7f266009a3b0 con 0x7f26700b28d0 2024-09-16T15:56:31.654 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.652+0000 7f266f400640 1 --2- 172.21.15.151:0/3494850033 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f265007afb0 0x7f265007d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:31.654 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.653+0000 7f266f400640 1 --2- 172.21.15.151:0/3494850033 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f265007afb0 0x7f265007d470 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f26700b9d80 tx=0x7f265c007e30 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:31.658 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.656+0000 7f2667e00640 1 -- 172.21.15.151:0/3494850033 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2660014030 con 0x7f26700b28d0 2024-09-16T15:56:31.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.875+0000 7f2675800640 1 -- 172.21.15.151:0/3494850033 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f26700b0b10 con 0x7f26700b28d0 2024-09-16T15:56:31.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.876+0000 7f2667e00640 1 -- 172.21.15.151:0/3494850033 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) ==== 94+0+5236 (secure 0 0 0) 0x7f2660063e30 con 0x7f26700b28d0 2024-09-16T15:56:31.878 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:56:31.878 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":97,"btime":"2024-09-16T15:55:09:523485+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7739,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2795044069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2795044069},{"type":"v1","addr":"172.21.15.167:6837","nonce":2795044069}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":8644,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/675530853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":675530853},{"type":"v1","addr":"172.21.15.151:6835","nonce":675530853}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":9468,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/1930858450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1930858450},{"type":"v1","addr":"172.21.15.151:6834","nonce":1930858450}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T15:54:55.864255+0000","modified":"2024-09-16T15:54:58.898875+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9474},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9474":{"gid":9474,"name":"b","rank":0,"incarnation":93,"state":"up:active","state_seq":3,"addr":"172.21.15.167:6836/1231498178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":1231498178},{"type":"v1","addr":"172.21.15.167:6836","nonce":1231498178}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9474,"qdb_cluster":[9474]},"id":10}]} 2024-09-16T15:56:31.878 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 97 2024-09-16T15:56:31.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.878+0000 7f2675800640 1 -- 172.21.15.151:0/3494850033 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f265007afb0 msgr2=0x7f265007d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:31.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.878+0000 7f2675800640 1 --2- 172.21.15.151:0/3494850033 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f265007afb0 0x7f265007d470 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f26700b9d80 tx=0x7f265c007e30 comp rx=0 tx=0).stop 2024-09-16T15:56:31.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.878+0000 7f2675800640 1 -- 172.21.15.151:0/3494850033 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f26700b28d0 msgr2=0x7f2670161580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:31.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.878+0000 7f2675800640 1 --2- 172.21.15.151:0/3494850033 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f26700b28d0 0x7f2670161580 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f266000c9a0 tx=0x7f266000ce60 comp rx=0 tx=0).stop 2024-09-16T15:56:31.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.878+0000 7f2675800640 1 -- 172.21.15.151:0/3494850033 shutdown_connections 2024-09-16T15:56:31.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.878+0000 7f2675800640 1 --2- 172.21.15.151:0/3494850033 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f265007afb0 0x7f265007d470 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:31.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.878+0000 7f2675800640 1 --2- 172.21.15.151:0/3494850033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f26700d1ec0 0x7f267015b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:31.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.878+0000 7f2675800640 1 --2- 172.21.15.151:0/3494850033 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f26700b4d10 0x7f26701f7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:31.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.878+0000 7f2675800640 1 --2- 172.21.15.151:0/3494850033 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f26700b28d0 0x7f2670161580 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:31.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.878+0000 7f2675800640 1 -- 172.21.15.151:0/3494850033 >> 172.21.15.151:0/3494850033 conn(0x7f26700d3de0 msgr2=0x7f2670161be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:31.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.879+0000 7f2675800640 1 -- 172.21.15.151:0/3494850033 shutdown_connections 2024-09-16T15:56:31.881 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:31.879+0000 7f2675800640 1 -- 172.21.15.151:0/3494850033 wait complete. 2024-09-16T15:56:31.897 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:56:32.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.053+0000 7fbcf5600640 1 -- 172.21.15.151:0/983707074 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbcf0073fb0 msgr2=0x7fbcf0074440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:32.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.053+0000 7fbcf5600640 1 --2- 172.21.15.151:0/983707074 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbcf0073fb0 0x7fbcf0074440 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fbce8005fd0 tx=0x7fbce802aa10 comp rx=0 tx=0).stop 2024-09-16T15:56:32.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.053+0000 7fbcf5600640 1 -- 172.21.15.151:0/983707074 shutdown_connections 2024-09-16T15:56:32.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.053+0000 7fbcf5600640 1 --2- 172.21.15.151:0/983707074 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbcf0073fb0 0x7fbcf0074440 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:32.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.053+0000 7fbcf5600640 1 --2- 172.21.15.151:0/983707074 >> v2:172.21.15.167:3300/0 conn(0x7fbcf015f5a0 0x7fbcf00b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:32.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.053+0000 7fbcf5600640 1 -- 172.21.15.151:0/983707074 >> 172.21.15.151:0/983707074 conn(0x7fbcf00b1950 msgr2=0x7fbcf00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:32.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.054+0000 7fbcf5600640 1 -- 172.21.15.151:0/983707074 shutdown_connections 2024-09-16T15:56:32.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.054+0000 7fbcf5600640 1 -- 172.21.15.151:0/983707074 wait complete. 2024-09-16T15:56:32.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.055+0000 7fbcf5600640 1 Processor -- start 2024-09-16T15:56:32.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.055+0000 7fbcf5600640 1 -- start start 2024-09-16T15:56:32.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.056+0000 7fbcf5600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbcf0073fb0 0x7fbcf0159fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:32.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.056+0000 7fbcf5600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbcf00ccf00 0x7fbcf014b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:32.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.056+0000 7fbcf5600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbcf015f5a0 0x7fbcf00ce530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:32.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.056+0000 7fbcf5600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf006f2e0 con 0x7fbcf0073fb0 2024-09-16T15:56:32.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.056+0000 7fbcf5600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf006f160 con 0x7fbcf015f5a0 2024-09-16T15:56:32.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.056+0000 7fbcf5600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fbcf006f460 con 0x7fbcf00ccf00 2024-09-16T15:56:32.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.056+0000 7fbceea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbcf0073fb0 0x7fbcf0159fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:32.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.057+0000 7fbceea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbcf0073fb0 0x7fbcf0159fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43686/0 (socket says 172.21.15.151:43686) 2024-09-16T15:56:32.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.057+0000 7fbceea00640 1 -- 172.21.15.151:0/2804755687 learned_addr learned my addr 172.21.15.151:0/2804755687 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:56:32.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.057+0000 7fbcee000640 1 --2- 172.21.15.151:0/2804755687 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbcf00ccf00 0x7fbcf014b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:32.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.057+0000 7fbcef400640 1 --2- 172.21.15.151:0/2804755687 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbcf015f5a0 0x7fbcf00ce530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:32.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.057+0000 7fbceea00640 1 -- 172.21.15.151:0/2804755687 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbcf00ccf00 msgr2=0x7fbcf014b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:32.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.057+0000 7fbceea00640 1 --2- 172.21.15.151:0/2804755687 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbcf00ccf00 0x7fbcf014b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:32.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.057+0000 7fbceea00640 1 -- 172.21.15.151:0/2804755687 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbcf015f5a0 msgr2=0x7fbcf00ce530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:32.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.057+0000 7fbceea00640 1 --2- 172.21.15.151:0/2804755687 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbcf015f5a0 0x7fbcf00ce530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:32.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.057+0000 7fbceea00640 1 -- 172.21.15.151:0/2804755687 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcf015a6a0 con 0x7fbcf0073fb0 2024-09-16T15:56:32.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.057+0000 7fbceea00640 1 --2- 172.21.15.151:0/2804755687 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbcf0073fb0 0x7fbcf0159fc0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7fbce000e9f0 tx=0x7fbce000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:32.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.057+0000 7fbce6a00640 1 -- 172.21.15.151:0/2804755687 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce000cd80 con 0x7fbcf0073fb0 2024-09-16T15:56:32.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.057+0000 7fbce6a00640 1 -- 172.21.15.151:0/2804755687 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbce0004020 con 0x7fbcf0073fb0 2024-09-16T15:56:32.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.058+0000 7fbcf5600640 1 -- 172.21.15.151:0/2804755687 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcf015a980 con 0x7fbcf0073fb0 2024-09-16T15:56:32.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.058+0000 7fbce6a00640 1 -- 172.21.15.151:0/2804755687 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce0004470 con 0x7fbcf0073fb0 2024-09-16T15:56:32.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.058+0000 7fbcf5600640 1 -- 172.21.15.151:0/2804755687 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcf014bb60 con 0x7fbcf0073fb0 2024-09-16T15:56:32.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.059+0000 7fbcf5600640 1 -- 172.21.15.151:0/2804755687 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcf006f980 con 0x7fbcf0073fb0 2024-09-16T15:56:32.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.060+0000 7fbce6a00640 1 -- 172.21.15.151:0/2804755687 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 55) ==== 104857+0+0 (secure 0 0 0) 0x7fbce00035f0 con 0x7fbcf0073fb0 2024-09-16T15:56:32.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.061+0000 7fbce6a00640 1 --2- 172.21.15.151:0/2804755687 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbcd007afb0 0x7fbcd007d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:32.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.061+0000 7fbce6a00640 1 -- 172.21.15.151:0/2804755687 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5935+0+0 (secure 0 0 0) 0x7fbce009a690 con 0x7fbcf0073fb0 2024-09-16T15:56:32.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.061+0000 7fbcee000640 1 --2- 172.21.15.151:0/2804755687 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbcd007afb0 0x7fbcd007d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:32.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.062+0000 7fbcee000640 1 --2- 172.21.15.151:0/2804755687 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbcd007afb0 0x7fbcd007d470 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fbcf0157c70 tx=0x7fbcdc009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:32.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.065+0000 7fbce6a00640 1 -- 172.21.15.151:0/2804755687 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fbce0064110 con 0x7fbcf0073fb0 2024-09-16T15:56:32.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.255+0000 7fbcf5600640 1 -- 172.21.15.151:0/2804755687 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbcf015ab50 con 0x7fbcf0073fb0 2024-09-16T15:56:32.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.256+0000 7fbce6a00640 1 -- 172.21.15.151:0/2804755687 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v149) ==== 74+0+17126 (secure 0 0 0) 0x7fbce0068b40 con 0x7fbcf0073fb0 2024-09-16T15:56:32.258 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:56:32.258 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":149,"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","created":"2024-09-16T15:38:56.348535+0000","modified":"2024-09-16T15:56:31.560681+0000","last_up_change":"2024-09-16T15:39:10.681384+0000","last_in_change":"2024-09-16T15:39:00.544477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:39:11.602252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-16T15:54:53.628710+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-16T15:54:54.832847+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":148,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5524076-a903-4d0a-af6a-caf474c4765d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6809","nonce":3111342741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6811","nonce":3111342741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6815","nonce":3111342741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6813","nonce":3111342741}]},"public_addr":"172.21.15.151:6809/3111342741","cluster_addr":"172.21.15.151:6811/3111342741","heartbeat_back_addr":"172.21.15.151:6815/3111342741","heartbeat_front_addr":"172.21.15.151:6813/3111342741","state":["exists","up"]},{"osd":1,"uuid":"dd25925f-0ae1-494e-8529-4dc205759db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6801","nonce":3368304087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6803","nonce":3368304087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6807","nonce":3368304087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6805","nonce":3368304087}]},"public_addr":"172.21.15.151:6801/3368304087","cluster_addr":"172.21.15.151:6803/3368304087","heartbeat_back_addr":"172.21.15.151:6807/3368304087","heartbeat_front_addr":"172.21.15.151:6805/3368304087","state":["exists","up"]},{"osd":2,"uuid":"c2f18115-013d-4da6-a0fe-cb978f4494b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6823","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6825","nonce":3079373271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6827","nonce":3079373271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6831","nonce":3079373271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6829","nonce":3079373271}]},"public_addr":"172.21.15.151:6825/3079373271","cluster_addr":"172.21.15.151:6827/3079373271","heartbeat_back_addr":"172.21.15.151:6831/3079373271","heartbeat_front_addr":"172.21.15.151:6829/3079373271","state":["exists","up"]},{"osd":3,"uuid":"36edb718-2fcb-4d88-8ddc-c321eed4279d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6817","nonce":1142509041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6819","nonce":1142509041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6824","nonce":1142509041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6821","nonce":1142509041}]},"public_addr":"172.21.15.151:6817/1142509041","cluster_addr":"172.21.15.151:6819/1142509041","heartbeat_back_addr":"172.21.15.151:6824/1142509041","heartbeat_front_addr":"172.21.15.151:6821/1142509041","state":["exists","up"]},{"osd":4,"uuid":"64d33708-cd0a-4236-abe3-1b646e82ab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6811","nonce":2735495271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6813","nonce":2735495271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6817","nonce":2735495271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6815","nonce":2735495271}]},"public_addr":"172.21.15.167:6811/2735495271","cluster_addr":"172.21.15.167:6813/2735495271","heartbeat_back_addr":"172.21.15.167:6817/2735495271","heartbeat_front_addr":"172.21.15.167:6815/2735495271","state":["exists","up"]},{"osd":5,"uuid":"f14b44a9-5910-458c-b518-09a8d0de94de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6827","nonce":1494916572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6829","nonce":1494916572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6833","nonce":1494916572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6831","nonce":1494916572}]},"public_addr":"172.21.15.167:6827/1494916572","cluster_addr":"172.21.15.167:6829/1494916572","heartbeat_back_addr":"172.21.15.167:6833/1494916572","heartbeat_front_addr":"172.21.15.167:6831/1494916572","state":["exists","up"]},{"osd":6,"uuid":"39335545-2654-4710-84b0-a9aa47cfcbba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6819","nonce":2538273273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6821","nonce":2538273273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6825","nonce":2538273273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6823","nonce":2538273273}]},"public_addr":"172.21.15.167:6819/2538273273","cluster_addr":"172.21.15.167:6821/2538273273","heartbeat_back_addr":"172.21.15.167:6825/2538273273","heartbeat_front_addr":"172.21.15.167:6823/2538273273","state":["exists","up"]},{"osd":7,"uuid":"4f8bfc81-53dd-4339-ac22-1a4007d9f963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6803","nonce":265072227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6805","nonce":265072227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6809","nonce":265072227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6807","nonce":265072227}]},"public_addr":"172.21.15.167:6803/265072227","cluster_addr":"172.21.15.167:6805/265072227","heartbeat_back_addr":"172.21.15.167:6809/265072227","heartbeat_front_addr":"172.21.15.167:6807/265072227","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.150359+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.084358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.722368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.332809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.320709+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.395153+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.935250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.801443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.2f","mappings":[{"from":2,"to":5}]},{"pgid":"21.3c","mappings":[{"from":2,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":21,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:56:32.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.258+0000 7fbcf5600640 1 -- 172.21.15.151:0/2804755687 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbcd007afb0 msgr2=0x7fbcd007d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:32.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.258+0000 7fbcf5600640 1 --2- 172.21.15.151:0/2804755687 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbcd007afb0 0x7fbcd007d470 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fbcf0157c70 tx=0x7fbcdc009040 comp rx=0 tx=0).stop 2024-09-16T15:56:32.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.258+0000 7fbcf5600640 1 -- 172.21.15.151:0/2804755687 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbcf0073fb0 msgr2=0x7fbcf0159fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:32.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.258+0000 7fbcf5600640 1 --2- 172.21.15.151:0/2804755687 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbcf0073fb0 0x7fbcf0159fc0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7fbce000e9f0 tx=0x7fbce000eeb0 comp rx=0 tx=0).stop 2024-09-16T15:56:32.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.258+0000 7fbcf5600640 1 -- 172.21.15.151:0/2804755687 shutdown_connections 2024-09-16T15:56:32.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.258+0000 7fbcf5600640 1 --2- 172.21.15.151:0/2804755687 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbcd007afb0 0x7fbcd007d470 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:32.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.258+0000 7fbcf5600640 1 --2- 172.21.15.151:0/2804755687 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbcf015f5a0 0x7fbcf00ce530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:32.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.258+0000 7fbcf5600640 1 --2- 172.21.15.151:0/2804755687 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbcf00ccf00 0x7fbcf014b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:32.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.258+0000 7fbcf5600640 1 --2- 172.21.15.151:0/2804755687 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbcf0073fb0 0x7fbcf0159fc0 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:32.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.258+0000 7fbcf5600640 1 -- 172.21.15.151:0/2804755687 >> 172.21.15.151:0/2804755687 conn(0x7fbcf00b1950 msgr2=0x7fbcf00afec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:32.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.259+0000 7fbcf5600640 1 -- 172.21.15.151:0/2804755687 shutdown_connections 2024-09-16T15:56:32.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.259+0000 7fbcf5600640 1 -- 172.21.15.151:0/2804755687 wait complete. 2024-09-16T15:56:32.277 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-16T15:56:32.439 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.436+0000 7f625a600640 1 -- 172.21.15.151:0/3359083647 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f62540725d0 msgr2=0x7f6254073d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:32.439 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.436+0000 7f625a600640 1 --2- 172.21.15.151:0/3359083647 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f62540725d0 0x7f6254073d40 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f623c005fd0 tx=0x7f623c02a5d0 comp rx=0 tx=0).stop 2024-09-16T15:56:32.439 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.437+0000 7f625a600640 1 -- 172.21.15.151:0/3359083647 shutdown_connections 2024-09-16T15:56:32.439 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.437+0000 7f625a600640 1 --2- 172.21.15.151:0/3359083647 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f62540725d0 0x7f6254073d40 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:32.439 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.437+0000 7f625a600640 1 --2- 172.21.15.151:0/3359083647 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f625415bc30 0x7f62540b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:32.439 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.437+0000 7f625a600640 1 -- 172.21.15.151:0/3359083647 >> 172.21.15.151:0/3359083647 conn(0x7f62540cfa20 msgr2=0x7f62540cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:32.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.437+0000 7f625a600640 1 -- 172.21.15.151:0/3359083647 shutdown_connections 2024-09-16T15:56:32.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.437+0000 7f625a600640 1 -- 172.21.15.151:0/3359083647 wait complete. 2024-09-16T15:56:32.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.438+0000 7f625a600640 1 Processor -- start 2024-09-16T15:56:32.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.438+0000 7f625a600640 1 -- start start 2024-09-16T15:56:32.443 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.439+0000 7f625a600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62540d0540 0x7f62541f4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:32.443 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.439+0000 7f625a600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f625415bc30 0x7f625415d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:32.443 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.439+0000 7f625a600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f625415e0e0 0x7f625415c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:32.443 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.439+0000 7f625a600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6254074610 con 0x7f625415bc30 2024-09-16T15:56:32.443 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.439+0000 7f625a600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6254074490 con 0x7f625415e0e0 2024-09-16T15:56:32.443 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.439+0000 7f625a600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f6254074790 con 0x7f62540d0540 2024-09-16T15:56:32.443 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.439+0000 7f6253e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f625415bc30 0x7f625415d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:32.443 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.440+0000 7f6253e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f625415bc30 0x7f625415d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:43698/0 (socket says 172.21.15.151:43698) 2024-09-16T15:56:32.443 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.440+0000 7f6253e00640 1 -- 172.21.15.151:0/2372295760 learned_addr learned my addr 172.21.15.151:0/2372295760 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:56:32.443 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.440+0000 7f6259200640 1 --2- 172.21.15.151:0/2372295760 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62540d0540 0x7f62541f4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:32.443 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.440+0000 7f6259c00640 1 --2- 172.21.15.151:0/2372295760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f625415e0e0 0x7f625415c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:32.443 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.440+0000 7f6253e00640 1 -- 172.21.15.151:0/2372295760 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62540d0540 msgr2=0x7f62541f4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:32.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.440+0000 7f6253e00640 1 --2- 172.21.15.151:0/2372295760 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62540d0540 0x7f62541f4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:32.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.440+0000 7f6253e00640 1 -- 172.21.15.151:0/2372295760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f625415e0e0 msgr2=0x7f625415c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:32.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.440+0000 7f6253e00640 1 --2- 172.21.15.151:0/2372295760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f625415e0e0 0x7f625415c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:32.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.440+0000 7f6253e00640 1 -- 172.21.15.151:0/2372295760 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f625415e5f0 con 0x7f625415bc30 2024-09-16T15:56:32.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.440+0000 7f6259200640 1 --2- 172.21.15.151:0/2372295760 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62540d0540 0x7f62541f4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:56:32.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.440+0000 7f6253e00640 1 --2- 172.21.15.151:0/2372295760 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f625415bc30 0x7f625415d150 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f623c009850 tx=0x7f623c0035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:32.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.440+0000 7f6250c00640 1 -- 172.21.15.151:0/2372295760 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f623c03f070 con 0x7f625415bc30 2024-09-16T15:56:32.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.441+0000 7f6250c00640 1 -- 172.21.15.151:0/2372295760 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f623c003ce0 con 0x7f625415bc30 2024-09-16T15:56:32.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.441+0000 7f6250c00640 1 -- 172.21.15.151:0/2372295760 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f623c035030 con 0x7f625415bc30 2024-09-16T15:56:32.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.441+0000 7f625a600640 1 -- 172.21.15.151:0/2372295760 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f625415e8d0 con 0x7f625415bc30 2024-09-16T15:56:32.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.441+0000 7f625a600640 1 -- 172.21.15.151:0/2372295760 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f625415ee90 con 0x7f625415bc30 2024-09-16T15:56:32.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.442+0000 7f625a600640 1 -- 172.21.15.151:0/2372295760 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f625406fc60 con 0x7f625415bc30 2024-09-16T15:56:32.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.443+0000 7f6250c00640 1 -- 172.21.15.151:0/2372295760 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 55) ==== 104857+0+0 (secure 0 0 0) 0x7f623c042070 con 0x7f625415bc30 2024-09-16T15:56:32.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.444+0000 7f6250c00640 1 --2- 172.21.15.151:0/2372295760 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f623007afb0 0x7f623007d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:32.446 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.445+0000 7f6250c00640 1 -- 172.21.15.151:0/2372295760 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5935+0+0 (secure 0 0 0) 0x7f623c0ba8b0 con 0x7f625415bc30 2024-09-16T15:56:32.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.445+0000 7f6259200640 1 --2- 172.21.15.151:0/2372295760 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f623007afb0 0x7f623007d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:32.447 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.446+0000 7f6259200640 1 --2- 172.21.15.151:0/2372295760 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f623007afb0 0x7f623007d470 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f6244005fd0 tx=0x7f6244009500 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:32.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.449+0000 7f6250c00640 1 -- 172.21.15.151:0/2372295760 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f623c084980 con 0x7f625415bc30 2024-09-16T15:56:32.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.678+0000 7f625a600640 1 -- 172.21.15.151:0/2372295760 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f62540b2a90 con 0x7f625415bc30 2024-09-16T15:56:32.680 INFO:tasks.ceph.mon.a.smithi151.stderr:2024-09-16T15:56:32.679+0000 7f62c3200640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T15:56:32.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.681+0000 7f6250c00640 1 -- 172.21.15.151:0/2372295760 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v98) ==== 161+0+0 (secure 0 0 0) 0x7f623c084280 con 0x7f625415bc30 2024-09-16T15:56:32.683 INFO:teuthology.orchestra.run.smithi151.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-16T15:56:32.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.683+0000 7f625a600640 1 -- 172.21.15.151:0/2372295760 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f623007afb0 msgr2=0x7f623007d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:32.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.683+0000 7f625a600640 1 --2- 172.21.15.151:0/2372295760 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f623007afb0 0x7f623007d470 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f6244005fd0 tx=0x7f6244009500 comp rx=0 tx=0).stop 2024-09-16T15:56:32.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.684+0000 7f625a600640 1 -- 172.21.15.151:0/2372295760 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f625415bc30 msgr2=0x7f625415d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:32.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.684+0000 7f625a600640 1 --2- 172.21.15.151:0/2372295760 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f625415bc30 0x7f625415d150 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f623c009850 tx=0x7f623c0035f0 comp rx=0 tx=0).stop 2024-09-16T15:56:32.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.684+0000 7f625a600640 1 -- 172.21.15.151:0/2372295760 shutdown_connections 2024-09-16T15:56:32.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.684+0000 7f625a600640 1 --2- 172.21.15.151:0/2372295760 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f623007afb0 0x7f623007d470 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:32.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.684+0000 7f625a600640 1 --2- 172.21.15.151:0/2372295760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f625415e0e0 0x7f625415c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:32.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.684+0000 7f625a600640 1 --2- 172.21.15.151:0/2372295760 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f625415bc30 0x7f625415d150 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:32.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.684+0000 7f625a600640 1 --2- 172.21.15.151:0/2372295760 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f62540d0540 0x7f62541f4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:32.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.684+0000 7f625a600640 1 -- 172.21.15.151:0/2372295760 >> 172.21.15.151:0/2372295760 conn(0x7f62540cfa20 msgr2=0x7f62540cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:32.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.684+0000 7f625a600640 1 -- 172.21.15.151:0/2372295760 shutdown_connections 2024-09-16T15:56:32.687 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.684+0000 7f625a600640 1 -- 172.21.15.151:0/2372295760 wait complete. 2024-09-16T15:56:32.701 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-16T15:56:32.758 INFO:tasks.ceph.mds.b.smithi167.stdout:starting mds.b at 2024-09-16T15:56:32.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.863+0000 7f3672400640 1 -- 172.21.15.151:0/1382131258 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f366c06efa0 msgr2=0x7f366c0cc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:32.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.863+0000 7f3672400640 1 --2- 172.21.15.151:0/1382131258 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f366c06efa0 0x7f366c0cc5c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3658009830 tx=0x7f365802a480 comp rx=0 tx=0).stop 2024-09-16T15:56:32.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.863+0000 7f3672400640 1 -- 172.21.15.151:0/1382131258 shutdown_connections 2024-09-16T15:56:32.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.863+0000 7f3672400640 1 --2- 172.21.15.151:0/1382131258 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f366c06efa0 0x7f366c0cc5c0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:32.865 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.863+0000 7f3672400640 1 --2- 172.21.15.151:0/1382131258 >> v2:172.21.15.167:3300/0 conn(0x7f366c145750 0x7f366c073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:32.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.863+0000 7f3672400640 1 -- 172.21.15.151:0/1382131258 >> 172.21.15.151:0/1382131258 conn(0x7f366c0b1840 msgr2=0x7f366c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:32.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.863+0000 7f3672400640 1 -- 172.21.15.151:0/1382131258 shutdown_connections 2024-09-16T15:56:32.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.863+0000 7f3672400640 1 -- 172.21.15.151:0/1382131258 wait complete. 2024-09-16T15:56:32.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.865+0000 7f3672400640 1 Processor -- start 2024-09-16T15:56:32.866 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.865+0000 7f3672400640 1 -- start start 2024-09-16T15:56:32.869 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.866+0000 7f3672400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f366c06efa0 0x7f366c1f87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:32.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.866+0000 7f3672400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f366c0ccb00 0x7f366c1e9650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:32.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.866+0000 7f3672400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f366c145750 0x7f366c14d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:32.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.866+0000 7f3672400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f366c0b26b0 con 0x7f366c06efa0 2024-09-16T15:56:32.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.866+0000 7f3672400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f366c0cedd0 con 0x7f366c145750 2024-09-16T15:56:32.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.866+0000 7f3672400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f366c160100 con 0x7f366c0ccb00 2024-09-16T15:56:32.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.866+0000 7f366b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f366c06efa0 0x7f366c1f87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:32.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.866+0000 7f366b400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f366c06efa0 0x7f366c1f87c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37220/0 (socket says 172.21.15.151:37220) 2024-09-16T15:56:32.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.866+0000 7f366b400640 1 -- 172.21.15.151:0/2684570661 learned_addr learned my addr 172.21.15.151:0/2684570661 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:56:32.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.866+0000 7f366be00640 1 --2- 172.21.15.151:0/2684570661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f366c145750 0x7f366c14d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:32.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.866+0000 7f366aa00640 1 --2- 172.21.15.151:0/2684570661 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f366c0ccb00 0x7f366c1e9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:32.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.866+0000 7f366b400640 1 -- 172.21.15.151:0/2684570661 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f366c0ccb00 msgr2=0x7f366c1e9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:32.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.866+0000 7f366b400640 1 --2- 172.21.15.151:0/2684570661 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f366c0ccb00 0x7f366c1e9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:32.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.866+0000 7f366b400640 1 -- 172.21.15.151:0/2684570661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f366c145750 msgr2=0x7f366c14d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:32.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.866+0000 7f366b400640 1 --2- 172.21.15.151:0/2684570661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f366c145750 0x7f366c14d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:32.870 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.866+0000 7f366b400640 1 -- 172.21.15.151:0/2684570661 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f366c14ded0 con 0x7f366c06efa0 2024-09-16T15:56:32.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.866+0000 7f366be00640 1 --2- 172.21.15.151:0/2684570661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f366c145750 0x7f366c14d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:56:32.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.867+0000 7f366b400640 1 --2- 172.21.15.151:0/2684570661 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f366c06efa0 0x7f366c1f87c0 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f365c00e790 tx=0x7f365c00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:32.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.867+0000 7f3663400640 1 -- 172.21.15.151:0/2684570661 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f365c00eea0 con 0x7f366c06efa0 2024-09-16T15:56:32.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.867+0000 7f3663400640 1 -- 172.21.15.151:0/2684570661 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f365c00cdb0 con 0x7f366c06efa0 2024-09-16T15:56:32.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.867+0000 7f3672400640 1 -- 172.21.15.151:0/2684570661 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f366c14e1b0 con 0x7f366c06efa0 2024-09-16T15:56:32.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.867+0000 7f3663400640 1 -- 172.21.15.151:0/2684570661 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f365c010690 con 0x7f366c06efa0 2024-09-16T15:56:32.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.867+0000 7f3672400640 1 -- 172.21.15.151:0/2684570661 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f366c1f8d00 con 0x7f366c06efa0 2024-09-16T15:56:32.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.868+0000 7f3672400640 1 -- 172.21.15.151:0/2684570661 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f366c071030 con 0x7f366c06efa0 2024-09-16T15:56:32.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.869+0000 7f3663400640 1 -- 172.21.15.151:0/2684570661 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 55) ==== 104857+0+0 (secure 0 0 0) 0x7f365c010830 con 0x7f366c06efa0 2024-09-16T15:56:32.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.871+0000 7f3663400640 1 --2- 172.21.15.151:0/2684570661 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f364c07afb0 0x7f364c07d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:32.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.871+0000 7f3663400640 1 -- 172.21.15.151:0/2684570661 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 5993+0+0 (secure 0 0 0) 0x7f365c09a620 con 0x7f366c06efa0 2024-09-16T15:56:32.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.871+0000 7f366aa00640 1 --2- 172.21.15.151:0/2684570661 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f364c07afb0 0x7f364c07d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:32.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.872+0000 7f366aa00640 1 --2- 172.21.15.151:0/2684570661 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f364c07afb0 0x7f364c07d470 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f3658034000 tx=0x7f3658004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:32.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:32.875+0000 7f3663400640 1 -- 172.21.15.151:0/2684570661 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f366c071030 con 0x7f366c06efa0 2024-09-16T15:56:33.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:33.100+0000 7f3672400640 1 -- 172.21.15.151:0/2684570661 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f366c0b0600 con 0x7f366c06efa0 2024-09-16T15:56:33.685 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:33.683+0000 7f3663400640 1 -- 172.21.15.151:0/2684570661 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v99) ==== 104+0+0 (secure 0 0 0) 0x7f365c064770 con 0x7f366c06efa0 2024-09-16T15:56:33.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:33.686+0000 7f3672400640 1 -- 172.21.15.151:0/2684570661 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f364c07afb0 msgr2=0x7f364c07d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:33.689 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:33.686+0000 7f3672400640 1 --2- 172.21.15.151:0/2684570661 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f364c07afb0 0x7f364c07d470 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f3658034000 tx=0x7f3658004730 comp rx=0 tx=0).stop 2024-09-16T15:56:33.689 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:33.686+0000 7f3672400640 1 -- 172.21.15.151:0/2684570661 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f366c06efa0 msgr2=0x7f366c1f87c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:33.689 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:33.686+0000 7f3672400640 1 --2- 172.21.15.151:0/2684570661 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f366c06efa0 0x7f366c1f87c0 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f365c00e790 tx=0x7f365c00ec50 comp rx=0 tx=0).stop 2024-09-16T15:56:33.689 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:33.686+0000 7f3672400640 1 -- 172.21.15.151:0/2684570661 shutdown_connections 2024-09-16T15:56:33.689 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:33.686+0000 7f3672400640 1 --2- 172.21.15.151:0/2684570661 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f364c07afb0 0x7f364c07d470 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:33.689 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:33.686+0000 7f3672400640 1 --2- 172.21.15.151:0/2684570661 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f366c145750 0x7f366c14d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:33.689 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:33.686+0000 7f3672400640 1 --2- 172.21.15.151:0/2684570661 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f366c0ccb00 0x7f366c1e9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:33.689 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:33.686+0000 7f3672400640 1 --2- 172.21.15.151:0/2684570661 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f366c06efa0 0x7f366c1f87c0 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:33.689 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:33.686+0000 7f3672400640 1 -- 172.21.15.151:0/2684570661 >> 172.21.15.151:0/2684570661 conn(0x7f366c0b1840 msgr2=0x7f366c0ad870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:33.689 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:33.686+0000 7f3672400640 1 -- 172.21.15.151:0/2684570661 shutdown_connections 2024-09-16T15:56:33.689 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:33.686+0000 7f3672400640 1 -- 172.21.15.151:0/2684570661 wait complete. 2024-09-16T15:56:33.705 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-16T15:56:33.871 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:33.869+0000 7fbc41400640 1 -- 172.21.15.151:0/1493918633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc3c06e910 msgr2=0x7fbc3c0cdad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:33.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:33.869+0000 7fbc41400640 1 --2- 172.21.15.151:0/1493918633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc3c06e910 0x7fbc3c0cdad0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fbc24009830 tx=0x7fbc2402a5c0 comp rx=0 tx=0).stop 2024-09-16T15:56:33.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:33.869+0000 7fbc41400640 1 -- 172.21.15.151:0/1493918633 shutdown_connections 2024-09-16T15:56:33.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:33.869+0000 7fbc41400640 1 --2- 172.21.15.151:0/1493918633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc3c06e910 0x7fbc3c0cdad0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:33.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:33.869+0000 7fbc41400640 1 -- 172.21.15.151:0/1493918633 >> 172.21.15.151:0/1493918633 conn(0x7fbc3c0b18e0 msgr2=0x7fbc3c0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:33.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:33.869+0000 7fbc41400640 1 -- 172.21.15.151:0/1493918633 shutdown_connections 2024-09-16T15:56:33.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:33.869+0000 7fbc41400640 1 -- 172.21.15.151:0/1493918633 wait complete. 2024-09-16T15:56:33.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:33.871+0000 7fbc41400640 1 Processor -- start 2024-09-16T15:56:33.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:33.871+0000 7fbc41400640 1 -- start start 2024-09-16T15:56:33.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:33.872+0000 7fbc41400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc3c06e910 0x7fbc3c1f4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:33.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:33.872+0000 7fbc41400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc3c074460 0x7fbc3c0b4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:33.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:33.872+0000 7fbc41400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbc3c15bd60 0x7fbc3c1eeba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:33.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:33.872+0000 7fbc41400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3c0cd050 con 0x7fbc3c06e910 2024-09-16T15:56:33.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:33.872+0000 7fbc41400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3c0cced0 con 0x7fbc3c074460 2024-09-16T15:56:33.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:33.872+0000 7fbc41400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fbc3c0cd1d0 con 0x7fbc3c15bd60 2024-09-16T15:56:33.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:33.872+0000 7fbc3a800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc3c06e910 0x7fbc3c1f4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:33.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:33.872+0000 7fbc3a800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc3c06e910 0x7fbc3c1f4db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37226/0 (socket says 172.21.15.151:37226) 2024-09-16T15:56:33.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:33.872+0000 7fbc3a800640 1 -- 172.21.15.151:0/1475426002 learned_addr learned my addr 172.21.15.151:0/1475426002 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:56:33.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:33.872+0000 7fbc33e00640 1 --2- 172.21.15.151:0/1475426002 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc3c074460 0x7fbc3c0b4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:33.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:33.872+0000 7fbc3b200640 1 --2- 172.21.15.151:0/1475426002 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbc3c15bd60 0x7fbc3c1eeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:33.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:33.873+0000 7fbc3a800640 1 -- 172.21.15.151:0/1475426002 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbc3c15bd60 msgr2=0x7fbc3c1eeba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:33.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:33.873+0000 7fbc3a800640 1 --2- 172.21.15.151:0/1475426002 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbc3c15bd60 0x7fbc3c1eeba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:33.876 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:33.873+0000 7fbc3a800640 1 -- 172.21.15.151:0/1475426002 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc3c074460 msgr2=0x7fbc3c0b4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:33.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:33.873+0000 7fbc3a800640 1 --2- 172.21.15.151:0/1475426002 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc3c074460 0x7fbc3c0b4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:33.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:33.873+0000 7fbc3a800640 1 -- 172.21.15.151:0/1475426002 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc3c0b3080 con 0x7fbc3c06e910 2024-09-16T15:56:33.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:33.873+0000 7fbc33e00640 1 --2- 172.21.15.151:0/1475426002 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc3c074460 0x7fbc3c0b4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:56:33.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:33.873+0000 7fbc3a800640 1 --2- 172.21.15.151:0/1475426002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc3c06e910 0x7fbc3c1f4db0 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c00c400 tx=0x7fbc2c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:33.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:33.873+0000 7fbc32a00640 1 -- 172.21.15.151:0/1475426002 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c011790 con 0x7fbc3c06e910 2024-09-16T15:56:33.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:33.873+0000 7fbc32a00640 1 -- 172.21.15.151:0/1475426002 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbc2c017070 con 0x7fbc3c06e910 2024-09-16T15:56:33.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:33.873+0000 7fbc32a00640 1 -- 172.21.15.151:0/1475426002 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c011ce0 con 0x7fbc3c06e910 2024-09-16T15:56:33.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:33.873+0000 7fbc41400640 1 -- 172.21.15.151:0/1475426002 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc3c0b3360 con 0x7fbc3c06e910 2024-09-16T15:56:33.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:33.873+0000 7fbc41400640 1 -- 172.21.15.151:0/1475426002 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc3c0b3920 con 0x7fbc3c06e910 2024-09-16T15:56:33.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:33.875+0000 7fbc41400640 1 -- 172.21.15.151:0/1475426002 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc3c069360 con 0x7fbc3c06e910 2024-09-16T15:56:33.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:33.876+0000 7fbc32a00640 1 -- 172.21.15.151:0/1475426002 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 55) ==== 104857+0+0 (secure 0 0 0) 0x7fbc2c01c460 con 0x7fbc3c06e910 2024-09-16T15:56:33.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:33.877+0000 7fbc32a00640 1 --2- 172.21.15.151:0/1475426002 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbc1807afb0 0x7fbc1807d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:33.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:33.877+0000 7fbc32a00640 1 -- 172.21.15.151:0/1475426002 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 5993+0+0 (secure 0 0 0) 0x7fbc2c0999b0 con 0x7fbc3c06e910 2024-09-16T15:56:33.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:33.877+0000 7fbc33e00640 1 --2- 172.21.15.151:0/1475426002 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbc1807afb0 0x7fbc1807d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:33.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:33.878+0000 7fbc33e00640 1 --2- 172.21.15.151:0/1475426002 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbc1807afb0 0x7fbc1807d470 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fbc24034000 tx=0x7fbc24004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:33.882 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:33.881+0000 7fbc32a00640 1 -- 172.21.15.151:0/1475426002 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fbc3c15c140 con 0x7fbc3c06e910 2024-09-16T15:56:34.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:34.064+0000 7fbc41400640 1 -- 172.21.15.151:0/1475426002 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fbc3c0cdad0 con 0x7fbc3c06e910 2024-09-16T15:56:34.873 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:34.871+0000 7fbc32a00640 1 -- 172.21.15.151:0/1475426002 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v151) ==== 181+0+0 (secure 0 0 0) 0x7fbc2c063b20 con 0x7fbc3c06e910 2024-09-16T15:56:34.873 INFO:teuthology.orchestra.run.smithi151.stderr:pool 'cephfs_metadata' removed 2024-09-16T15:56:34.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:34.875+0000 7fbc41400640 1 -- 172.21.15.151:0/1475426002 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbc1807afb0 msgr2=0x7fbc1807d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:34.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:34.875+0000 7fbc41400640 1 --2- 172.21.15.151:0/1475426002 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbc1807afb0 0x7fbc1807d470 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fbc24034000 tx=0x7fbc24004730 comp rx=0 tx=0).stop 2024-09-16T15:56:34.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:34.875+0000 7fbc41400640 1 -- 172.21.15.151:0/1475426002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc3c06e910 msgr2=0x7fbc3c1f4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:34.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:34.875+0000 7fbc41400640 1 --2- 172.21.15.151:0/1475426002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc3c06e910 0x7fbc3c1f4db0 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c00c400 tx=0x7fbc2c00c8c0 comp rx=0 tx=0).stop 2024-09-16T15:56:34.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:34.876+0000 7fbc41400640 1 -- 172.21.15.151:0/1475426002 shutdown_connections 2024-09-16T15:56:34.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:34.876+0000 7fbc41400640 1 --2- 172.21.15.151:0/1475426002 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbc1807afb0 0x7fbc1807d470 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:34.878 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:34.876+0000 7fbc41400640 1 --2- 172.21.15.151:0/1475426002 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc3c074460 0x7fbc3c0b4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:34.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:34.876+0000 7fbc41400640 1 --2- 172.21.15.151:0/1475426002 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbc3c15bd60 0x7fbc3c1eeba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:34.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:34.876+0000 7fbc41400640 1 --2- 172.21.15.151:0/1475426002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc3c06e910 0x7fbc3c1f4db0 secure :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c00c400 tx=0x7fbc2c00c8c0 comp rx=0 tx=0).stop 2024-09-16T15:56:34.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:34.876+0000 7fbc41400640 1 -- 172.21.15.151:0/1475426002 >> 172.21.15.151:0/1475426002 conn(0x7fbc3c0b18e0 msgr2=0x7fbc3c0ce520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:34.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:34.876+0000 7fbc41400640 1 -- 172.21.15.151:0/1475426002 shutdown_connections 2024-09-16T15:56:34.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:34.877+0000 7fbc41400640 1 -- 172.21.15.151:0/1475426002 wait complete. 2024-09-16T15:56:34.899 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-16T15:56:35.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:35.063+0000 7fde8d800640 1 -- 172.21.15.151:0/3381039356 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde88161270 msgr2=0x7fde881616f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:35.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:35.063+0000 7fde8d800640 1 --2- 172.21.15.151:0/3381039356 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde88161270 0x7fde881616f0 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7fde8808c8c0 tx=0x7fde8002a980 comp rx=0 tx=0).stop 2024-09-16T15:56:35.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:35.063+0000 7fde8d800640 1 -- 172.21.15.151:0/3381039356 shutdown_connections 2024-09-16T15:56:35.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:35.063+0000 7fde8d800640 1 --2- 172.21.15.151:0/3381039356 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde88161270 0x7fde881616f0 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:35.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:35.063+0000 7fde8d800640 1 --2- 172.21.15.151:0/3381039356 >> v2:172.21.15.167:3300/0 conn(0x7fde880b4de0 0x7fde880d0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:35.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:35.063+0000 7fde8d800640 1 --2- 172.21.15.151:0/3381039356 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde880d0ab0 0x7fde880d4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:35.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:35.063+0000 7fde8d800640 1 -- 172.21.15.151:0/3381039356 >> 172.21.15.151:0/3381039356 conn(0x7fde880d4000 msgr2=0x7fde880d4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:35.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:35.063+0000 7fde8d800640 1 -- 172.21.15.151:0/3381039356 shutdown_connections 2024-09-16T15:56:35.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:35.063+0000 7fde8d800640 1 -- 172.21.15.151:0/3381039356 wait complete. 2024-09-16T15:56:35.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:35.065+0000 7fde8d800640 1 Processor -- start 2024-09-16T15:56:35.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:35.065+0000 7fde8d800640 1 -- start start 2024-09-16T15:56:35.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:35.066+0000 7fde8d800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde880b4de0 0x7fde880ccb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:35.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:35.066+0000 7fde8d800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde880d0ab0 0x7fde880a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:35.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:35.066+0000 7fde8d800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde88161270 0x7fde880cb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:35.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:35.066+0000 7fde8d800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fde880b3da0 con 0x7fde880d0ab0 2024-09-16T15:56:35.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:35.066+0000 7fde8d800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fde880b3c20 con 0x7fde88161270 2024-09-16T15:56:35.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:35.066+0000 7fde8d800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fde880b3f20 con 0x7fde880b4de0 2024-09-16T15:56:35.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:35.066+0000 7fde86000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde880d0ab0 0x7fde880a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:35.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:35.066+0000 7fde86000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde880d0ab0 0x7fde880a0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37240/0 (socket says 172.21.15.151:37240) 2024-09-16T15:56:35.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:35.066+0000 7fde86000640 1 -- 172.21.15.151:0/3325526749 learned_addr learned my addr 172.21.15.151:0/3325526749 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:56:35.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:35.067+0000 7fde86a00640 1 --2- 172.21.15.151:0/3325526749 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde880b4de0 0x7fde880ccb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:35.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:35.067+0000 7fde87400640 1 --2- 172.21.15.151:0/3325526749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde88161270 0x7fde880cb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:35.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:35.067+0000 7fde86000640 1 -- 172.21.15.151:0/3325526749 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde880b4de0 msgr2=0x7fde880ccb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:35.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:35.067+0000 7fde86000640 1 --2- 172.21.15.151:0/3325526749 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde880b4de0 0x7fde880ccb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:35.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:35.067+0000 7fde86000640 1 -- 172.21.15.151:0/3325526749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde88161270 msgr2=0x7fde880cb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:35.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:35.067+0000 7fde86000640 1 --2- 172.21.15.151:0/3325526749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde88161270 0x7fde880cb0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:35.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:35.067+0000 7fde86000640 1 -- 172.21.15.151:0/3325526749 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde88152ca0 con 0x7fde880d0ab0 2024-09-16T15:56:35.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:35.067+0000 7fde86000640 1 --2- 172.21.15.151:0/3325526749 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde880d0ab0 0x7fde880a0ee0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7fde7400d770 tx=0x7fde7400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:35.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:35.067+0000 7fde7ea00640 1 -- 172.21.15.151:0/3325526749 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde74017070 con 0x7fde880d0ab0 2024-09-16T15:56:35.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:35.067+0000 7fde7ea00640 1 -- 172.21.15.151:0/3325526749 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fde74009d20 con 0x7fde880d0ab0 2024-09-16T15:56:35.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:35.067+0000 7fde8d800640 1 -- 172.21.15.151:0/3325526749 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde88152f40 con 0x7fde880d0ab0 2024-09-16T15:56:35.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:35.067+0000 7fde7ea00640 1 -- 172.21.15.151:0/3325526749 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde740106c0 con 0x7fde880d0ab0 2024-09-16T15:56:35.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:35.068+0000 7fde8d800640 1 -- 172.21.15.151:0/3325526749 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde881537e0 con 0x7fde880d0ab0 2024-09-16T15:56:35.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:35.069+0000 7fde8d800640 1 -- 172.21.15.151:0/3325526749 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde880b4440 con 0x7fde880d0ab0 2024-09-16T15:56:35.072 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:35.070+0000 7fde7ea00640 1 -- 172.21.15.151:0/3325526749 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 55) ==== 104857+0+0 (secure 0 0 0) 0x7fde7401a070 con 0x7fde880d0ab0 2024-09-16T15:56:35.073 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:35.071+0000 7fde7ea00640 1 --2- 172.21.15.151:0/3325526749 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fde6807afb0 0x7fde6807d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:35.073 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:35.071+0000 7fde7ea00640 1 -- 172.21.15.151:0/3325526749 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5555+0+0 (secure 0 0 0) 0x7fde74099e40 con 0x7fde880d0ab0 2024-09-16T15:56:35.073 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:35.071+0000 7fde86a00640 1 --2- 172.21.15.151:0/3325526749 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fde6807afb0 0x7fde6807d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:35.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:35.072+0000 7fde86a00640 1 --2- 172.21.15.151:0/3325526749 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fde6807afb0 0x7fde6807d470 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fde78005fd0 tx=0x7fde78005850 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:35.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:35.075+0000 7fde7ea00640 1 -- 172.21.15.151:0/3325526749 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fde74063f10 con 0x7fde880d0ab0 2024-09-16T15:56:35.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:35.271+0000 7fde8d800640 1 -- 172.21.15.151:0/3325526749 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fde88076280 con 0x7fde880d0ab0 2024-09-16T15:56:35.875 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:35.873+0000 7fde7ea00640 1 -- 172.21.15.151:0/3325526749 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v152) ==== 169+0+0 (secure 0 0 0) 0x7fde74063810 con 0x7fde880d0ab0 2024-09-16T15:56:35.875 INFO:teuthology.orchestra.run.smithi151.stderr:pool 'cephfs_data' removed 2024-09-16T15:56:35.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:35.876+0000 7fde8d800640 1 -- 172.21.15.151:0/3325526749 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fde6807afb0 msgr2=0x7fde6807d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:35.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:35.876+0000 7fde8d800640 1 --2- 172.21.15.151:0/3325526749 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fde6807afb0 0x7fde6807d470 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fde78005fd0 tx=0x7fde78005850 comp rx=0 tx=0).stop 2024-09-16T15:56:35.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:35.876+0000 7fde8d800640 1 -- 172.21.15.151:0/3325526749 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde880d0ab0 msgr2=0x7fde880a0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:35.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:35.876+0000 7fde8d800640 1 --2- 172.21.15.151:0/3325526749 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde880d0ab0 0x7fde880a0ee0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7fde7400d770 tx=0x7fde7400dc30 comp rx=0 tx=0).stop 2024-09-16T15:56:35.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:35.876+0000 7fde8d800640 1 -- 172.21.15.151:0/3325526749 shutdown_connections 2024-09-16T15:56:35.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:35.877+0000 7fde8d800640 1 --2- 172.21.15.151:0/3325526749 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fde6807afb0 0x7fde6807d470 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:35.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:35.877+0000 7fde8d800640 1 --2- 172.21.15.151:0/3325526749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde88161270 0x7fde880cb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:35.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:35.877+0000 7fde8d800640 1 --2- 172.21.15.151:0/3325526749 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde880d0ab0 0x7fde880a0ee0 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:35.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:35.877+0000 7fde8d800640 1 --2- 172.21.15.151:0/3325526749 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fde880b4de0 0x7fde880ccb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:35.879 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:35.877+0000 7fde8d800640 1 -- 172.21.15.151:0/3325526749 >> 172.21.15.151:0/3325526749 conn(0x7fde880d4000 msgr2=0x7fde88160710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:35.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:35.877+0000 7fde8d800640 1 -- 172.21.15.151:0/3325526749 shutdown_connections 2024-09-16T15:56:35.880 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:35.877+0000 7fde8d800640 1 -- 172.21.15.151:0/3325526749 wait complete. 2024-09-16T15:56:35.901 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-16T15:56:36.065 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.063+0000 7f449fc00640 1 -- 172.21.15.151:0/2841881103 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f447c004800 msgr2=0x7f447c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:36.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.063+0000 7f449fc00640 1 --2- 172.21.15.151:0/2841881103 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f447c004800 0x7f447c004c80 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f448400c310 tx=0x7f447c005830 comp rx=0 tx=0).stop 2024-09-16T15:56:36.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.063+0000 7f449fc00640 1 -- 172.21.15.151:0/2841881103 shutdown_connections 2024-09-16T15:56:36.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.063+0000 7f449fc00640 1 --2- 172.21.15.151:0/2841881103 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f447c004800 0x7f447c004c80 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:36.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.063+0000 7f449fc00640 1 -- 172.21.15.151:0/2841881103 >> 172.21.15.151:0/2841881103 conn(0x7f44980b1950 msgr2=0x7f44980b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:36.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.063+0000 7f449fc00640 1 -- 172.21.15.151:0/2841881103 shutdown_connections 2024-09-16T15:56:36.066 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.063+0000 7f449fc00640 1 -- 172.21.15.151:0/2841881103 wait complete. 2024-09-16T15:56:36.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.065+0000 7f449fc00640 1 Processor -- start 2024-09-16T15:56:36.067 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.065+0000 7f449fc00640 1 -- start start 2024-09-16T15:56:36.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.066+0000 7f449fc00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f447c004800 0x7f44981e7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:36.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.066+0000 7f449fc00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44981f2590 0x7f4498144510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:36.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.066+0000 7f449fc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f44981e4730 0x7f44981e4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:36.069 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.066+0000 7f449fc00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4498074420 con 0x7f44981e4730 2024-09-16T15:56:36.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.066+0000 7f449fc00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f44980b26a0 con 0x7f44981f2590 2024-09-16T15:56:36.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.066+0000 7f449fc00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f44980745a0 con 0x7f447c004800 2024-09-16T15:56:36.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.066+0000 7f449dc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f44981e4730 0x7f44981e4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:36.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.066+0000 7f449dc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f44981e4730 0x7f44981e4bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37258/0 (socket says 172.21.15.151:37258) 2024-09-16T15:56:36.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.066+0000 7f449dc00640 1 -- 172.21.15.151:0/2015755885 learned_addr learned my addr 172.21.15.151:0/2015755885 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:56:36.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.067+0000 7f4497e00640 1 --2- 172.21.15.151:0/2015755885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44981f2590 0x7f4498144510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:36.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.067+0000 7f449d200640 1 --2- 172.21.15.151:0/2015755885 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f447c004800 0x7f44981e7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:36.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.067+0000 7f449dc00640 1 -- 172.21.15.151:0/2015755885 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f447c004800 msgr2=0x7f44981e7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:36.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.067+0000 7f449dc00640 1 --2- 172.21.15.151:0/2015755885 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f447c004800 0x7f44981e7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:36.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.067+0000 7f449dc00640 1 -- 172.21.15.151:0/2015755885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44981f2590 msgr2=0x7f4498144510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:36.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.067+0000 7f449dc00640 1 --2- 172.21.15.151:0/2015755885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44981f2590 0x7f4498144510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:36.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.067+0000 7f449dc00640 1 -- 172.21.15.151:0/2015755885 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44981f2db0 con 0x7f44981e4730 2024-09-16T15:56:36.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.067+0000 7f449d200640 1 --2- 172.21.15.151:0/2015755885 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f447c004800 0x7f44981e7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:56:36.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.067+0000 7f4497e00640 1 --2- 172.21.15.151:0/2015755885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44981f2590 0x7f4498144510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:56:36.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.067+0000 7f449dc00640 1 --2- 172.21.15.151:0/2015755885 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f44981e4730 0x7f44981e4bc0 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f4498076720 tx=0x7f449000ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:36.070 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.067+0000 7f4494c00640 1 -- 172.21.15.151:0/2015755885 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449000c940 con 0x7f44981e4730 2024-09-16T15:56:36.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.067+0000 7f4494c00640 1 -- 172.21.15.151:0/2015755885 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f449001a070 con 0x7f44981e4730 2024-09-16T15:56:36.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.067+0000 7f4494c00640 1 -- 172.21.15.151:0/2015755885 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449000cd80 con 0x7f44981e4730 2024-09-16T15:56:36.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.067+0000 7f449fc00640 1 -- 172.21.15.151:0/2015755885 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44981f3090 con 0x7f44981e4730 2024-09-16T15:56:36.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.067+0000 7f449fc00640 1 -- 172.21.15.151:0/2015755885 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44980ce1e0 con 0x7f44981e4730 2024-09-16T15:56:36.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.069+0000 7f449fc00640 1 -- 172.21.15.151:0/2015755885 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44981f35b0 con 0x7f44981e4730 2024-09-16T15:56:36.071 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.070+0000 7f4494c00640 1 -- 172.21.15.151:0/2015755885 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 55) ==== 104857+0+0 (secure 0 0 0) 0x7f449001f460 con 0x7f44981e4730 2024-09-16T15:56:36.073 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.071+0000 7f4494c00640 1 --2- 172.21.15.151:0/2015755885 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f447807afb0 0x7f447807d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:36.073 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.071+0000 7f4494c00640 1 -- 172.21.15.151:0/2015755885 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5083+0+0 (secure 0 0 0) 0x7f4490099e10 con 0x7f44981e4730 2024-09-16T15:56:36.073 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.071+0000 7f449d200640 1 --2- 172.21.15.151:0/2015755885 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f447807afb0 0x7f447807d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:36.074 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.072+0000 7f449d200640 1 --2- 172.21.15.151:0/2015755885 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f447807afb0 0x7f447807d470 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f4498158d80 tx=0x7f448800b260 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:36.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.075+0000 7f4494c00640 1 -- 172.21.15.151:0/2015755885 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f44981f35b0 con 0x7f44981e4730 2024-09-16T15:56:36.272 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.270+0000 7f449fc00640 1 -- 172.21.15.151:0/2015755885 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f44980ce3e0 con 0x7f44981e4730 2024-09-16T15:56:36.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.275+0000 7f4494c00640 1 -- 172.21.15.151:0/2015755885 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v57) ==== 71+0+0 (secure 0 0 0) 0x7f44980ce3e0 con 0x7f44981e4730 2024-09-16T15:56:36.280 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.277+0000 7f449fc00640 1 -- 172.21.15.151:0/2015755885 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f447807afb0 msgr2=0x7f447807d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:36.280 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.277+0000 7f449fc00640 1 --2- 172.21.15.151:0/2015755885 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f447807afb0 0x7f447807d470 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f4498158d80 tx=0x7f448800b260 comp rx=0 tx=0).stop 2024-09-16T15:56:36.280 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.277+0000 7f449fc00640 1 -- 172.21.15.151:0/2015755885 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f44981e4730 msgr2=0x7f44981e4bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:36.280 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.277+0000 7f449fc00640 1 --2- 172.21.15.151:0/2015755885 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f44981e4730 0x7f44981e4bc0 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f4498076720 tx=0x7f449000ed10 comp rx=0 tx=0).stop 2024-09-16T15:56:36.280 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.278+0000 7f449fc00640 1 -- 172.21.15.151:0/2015755885 shutdown_connections 2024-09-16T15:56:36.280 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.278+0000 7f449fc00640 1 --2- 172.21.15.151:0/2015755885 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f447807afb0 0x7f447807d470 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:36.281 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.278+0000 7f449fc00640 1 --2- 172.21.15.151:0/2015755885 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44981f2590 0x7f4498144510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:36.281 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.278+0000 7f449fc00640 1 --2- 172.21.15.151:0/2015755885 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f44981e4730 0x7f44981e4bc0 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:36.281 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.278+0000 7f449fc00640 1 --2- 172.21.15.151:0/2015755885 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f447c004800 0x7f44981e7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:36.281 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.278+0000 7f449fc00640 1 -- 172.21.15.151:0/2015755885 >> 172.21.15.151:0/2015755885 conn(0x7f44980b1950 msgr2=0x7f4498155320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:36.281 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.278+0000 7f449fc00640 1 -- 172.21.15.151:0/2015755885 shutdown_connections 2024-09-16T15:56:36.281 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.278+0000 7f449fc00640 1 -- 172.21.15.151:0/2015755885 wait complete. 2024-09-16T15:56:36.296 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_more_than_cloner_threads' 2024-09-16T15:56:36.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.453+0000 7fa4cc000640 1 -- 172.21.15.151:0/2845602665 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa4c415f5a0 msgr2=0x7fa4c40b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:36.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.453+0000 7fa4cc000640 1 --2- 172.21.15.151:0/2845602665 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa4c415f5a0 0x7fa4c40b2440 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa4b8009830 tx=0x7fa4b802a480 comp rx=0 tx=0).stop 2024-09-16T15:56:36.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.453+0000 7fa4cc000640 1 -- 172.21.15.151:0/2845602665 shutdown_connections 2024-09-16T15:56:36.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.453+0000 7fa4cc000640 1 --2- 172.21.15.151:0/2845602665 >> v2:172.21.15.167:3300/0 conn(0x7fa4c40ccf00 0x7fa4c40acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:36.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.453+0000 7fa4cc000640 1 --2- 172.21.15.151:0/2845602665 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa4c415f5a0 0x7fa4c40b2440 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:36.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.453+0000 7fa4cc000640 1 -- 172.21.15.151:0/2845602665 >> 172.21.15.151:0/2845602665 conn(0x7fa4c40b1950 msgr2=0x7fa4c40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:36.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.453+0000 7fa4cc000640 1 -- 172.21.15.151:0/2845602665 shutdown_connections 2024-09-16T15:56:36.456 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.453+0000 7fa4cc000640 1 -- 172.21.15.151:0/2845602665 wait complete. 2024-09-16T15:56:36.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.455+0000 7fa4cc000640 1 Processor -- start 2024-09-16T15:56:36.457 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.455+0000 7fa4cc000640 1 -- start start 2024-09-16T15:56:36.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.456+0000 7fa4cc000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4c40ad270 0x7fa4c414cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:36.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.456+0000 7fa4cc000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa4c40ccf00 0x7fa4c414ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:36.459 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.456+0000 7fa4cc000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4c415f5a0 0x7fa4c41464c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:36.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.456+0000 7fa4cc000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c406f2e0 con 0x7fa4c415f5a0 2024-09-16T15:56:36.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.456+0000 7fa4cc000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c406f160 con 0x7fa4c40ad270 2024-09-16T15:56:36.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.456+0000 7fa4cc000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fa4c406f460 con 0x7fa4c40ccf00 2024-09-16T15:56:36.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.456+0000 7fa4ca000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4c415f5a0 0x7fa4c41464c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:36.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.456+0000 7fa4ca000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4c415f5a0 0x7fa4c41464c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37268/0 (socket says 172.21.15.151:37268) 2024-09-16T15:56:36.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.456+0000 7fa4ca000640 1 -- 172.21.15.151:0/3468657046 learned_addr learned my addr 172.21.15.151:0/3468657046 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:56:36.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.456+0000 7fa4c8c00640 1 --2- 172.21.15.151:0/3468657046 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa4c40ccf00 0x7fa4c414ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:36.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.456+0000 7fa4c9600640 1 --2- 172.21.15.151:0/3468657046 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4c40ad270 0x7fa4c414cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:36.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.457+0000 7fa4ca000640 1 -- 172.21.15.151:0/3468657046 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa4c40ccf00 msgr2=0x7fa4c414ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:36.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.457+0000 7fa4ca000640 1 --2- 172.21.15.151:0/3468657046 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa4c40ccf00 0x7fa4c414ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:36.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.457+0000 7fa4ca000640 1 -- 172.21.15.151:0/3468657046 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4c40ad270 msgr2=0x7fa4c414cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:36.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.457+0000 7fa4ca000640 1 --2- 172.21.15.151:0/3468657046 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4c40ad270 0x7fa4c414cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:36.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.457+0000 7fa4ca000640 1 -- 172.21.15.151:0/3468657046 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4c4153bd0 con 0x7fa4c415f5a0 2024-09-16T15:56:36.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.457+0000 7fa4ca000640 1 --2- 172.21.15.151:0/3468657046 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4c415f5a0 0x7fa4c41464c0 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc00cb30 tx=0x7fa4bc00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:36.460 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.457+0000 7fa4c1600640 1 -- 172.21.15.151:0/3468657046 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc016020 con 0x7fa4c415f5a0 2024-09-16T15:56:36.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.457+0000 7fa4c1600640 1 -- 172.21.15.151:0/3468657046 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa4bc00ad90 con 0x7fa4c415f5a0 2024-09-16T15:56:36.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.457+0000 7fa4cc000640 1 -- 172.21.15.151:0/3468657046 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4c41538f0 con 0x7fa4c415f5a0 2024-09-16T15:56:36.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.457+0000 7fa4c1600640 1 -- 172.21.15.151:0/3468657046 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc012680 con 0x7fa4c415f5a0 2024-09-16T15:56:36.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.457+0000 7fa4cc000640 1 -- 172.21.15.151:0/3468657046 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4c40cef20 con 0x7fa4c415f5a0 2024-09-16T15:56:36.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.459+0000 7fa4cc000640 1 -- 172.21.15.151:0/3468657046 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4c409d4e0 con 0x7fa4c415f5a0 2024-09-16T15:56:36.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.460+0000 7fa4c1600640 1 -- 172.21.15.151:0/3468657046 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 55) ==== 104857+0+0 (secure 0 0 0) 0x7fa4bc019020 con 0x7fa4c415f5a0 2024-09-16T15:56:36.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.460+0000 7fa4c1600640 1 --2- 172.21.15.151:0/3468657046 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa4a407afb0 0x7fa4a407d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:36.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.461+0000 7fa4c1600640 1 -- 172.21.15.151:0/3468657046 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5083+0+0 (secure 0 0 0) 0x7fa4bc09a1b0 con 0x7fa4c415f5a0 2024-09-16T15:56:36.463 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.461+0000 7fa4c9600640 1 --2- 172.21.15.151:0/3468657046 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa4a407afb0 0x7fa4a407d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:36.464 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.462+0000 7fa4c9600640 1 --2- 172.21.15.151:0/3468657046 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa4a407afb0 0x7fa4a407d470 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fa4b8034000 tx=0x7fa4b8004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:36.466 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.465+0000 7fa4c1600640 1 -- 172.21.15.151:0/3468657046 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fa4bc020020 con 0x7fa4c415f5a0 2024-09-16T15:56:36.719 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.717+0000 7fa4cc000640 1 -- 172.21.15.151:0/3468657046 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_more_than_cloner_threads"]} v 0) -- 0x7fa4c4155be0 con 0x7fa4c415f5a0 2024-09-16T15:56:36.890 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.888+0000 7fa4c1600640 1 -- 172.21.15.151:0/3468657046 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgressReporter.test_when_clones_cancelled_are_more_than_cloner_threads"]}]=0 v788) ==== 186+0+0 (secure 0 0 0) 0x7fa4bc063c30 con 0x7fa4c415f5a0 2024-09-16T15:56:36.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.891+0000 7fa4cc000640 1 -- 172.21.15.151:0/3468657046 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa4a407afb0 msgr2=0x7fa4a407d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:36.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.891+0000 7fa4cc000640 1 --2- 172.21.15.151:0/3468657046 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa4a407afb0 0x7fa4a407d470 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fa4b8034000 tx=0x7fa4b8004730 comp rx=0 tx=0).stop 2024-09-16T15:56:36.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.891+0000 7fa4cc000640 1 -- 172.21.15.151:0/3468657046 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4c415f5a0 msgr2=0x7fa4c41464c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:36.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.891+0000 7fa4cc000640 1 --2- 172.21.15.151:0/3468657046 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4c415f5a0 0x7fa4c41464c0 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc00cb30 tx=0x7fa4bc00a060 comp rx=0 tx=0).stop 2024-09-16T15:56:36.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.891+0000 7fa4cc000640 1 -- 172.21.15.151:0/3468657046 shutdown_connections 2024-09-16T15:56:36.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.891+0000 7fa4cc000640 1 --2- 172.21.15.151:0/3468657046 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa4a407afb0 0x7fa4a407d470 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:36.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.891+0000 7fa4cc000640 1 --2- 172.21.15.151:0/3468657046 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4c415f5a0 0x7fa4c41464c0 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:36.893 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.891+0000 7fa4cc000640 1 --2- 172.21.15.151:0/3468657046 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa4c40ccf00 0x7fa4c414ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:36.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.891+0000 7fa4cc000640 1 --2- 172.21.15.151:0/3468657046 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4c40ad270 0x7fa4c414cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:36.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.891+0000 7fa4cc000640 1 -- 172.21.15.151:0/3468657046 >> 172.21.15.151:0/3468657046 conn(0x7fa4c40b1950 msgr2=0x7fa4c40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:36.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.891+0000 7fa4cc000640 1 -- 172.21.15.151:0/3468657046 shutdown_connections 2024-09-16T15:56:36.894 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:36.891+0000 7fa4cc000640 1 -- 172.21.15.151:0/3468657046 wait complete. 2024-09-16T15:56:36.909 INFO:tasks.cephfs_test_runner:Test that both the progress bars, that are printed for all 7 clone ... ok 2024-09-16T15:56:36.910 INFO:tasks.cephfs_test_runner: 2024-09-16T15:56:36.910 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-09-16T15:56:36.910 INFO:tasks.cephfs_test_runner:Ran 9 tests in 1004.003s 2024-09-16T15:56:36.910 INFO:tasks.cephfs_test_runner: 2024-09-16T15:56:36.910 INFO:tasks.cephfs_test_runner:OK 2024-09-16T15:56:36.910 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-09-16T15:56:36.919 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-16T15:56:36.927 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-16T15:56:36.927 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:56:36.927 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:56:36.962 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T15:56:36.962 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-16T15:56:36.962 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:56:37.038 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-16T15:56:37.038 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-16T15:56:37.038 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:56:37.078 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:56:37.078 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-16T15:56:37.078 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T15:56:37.157 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052629,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497976k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-16T15:56:37.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:56:37.157 DEBUG:teuthology.orchestra.run.smithi151:> ip netns list 2024-09-16T15:56:37.192 INFO:teuthology.orchestra.run.smithi151.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-16T15:56:37.192 INFO:teuthology.orchestra.run.smithi151.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-16T15:56:37.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:56:37.193 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T15:56:37.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:56:37.290 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-16T15:56:37.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:56:37.549 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip link delete ceph-brx 2024-09-16T15:56:37.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:56:37.841 DEBUG:teuthology.orchestra.run.smithi151:> ip netns list 2024-09-16T15:56:37.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:56:37.877 DEBUG:teuthology.orchestra.run.smithi151:> sudo ip link delete ceph-brx 2024-09-16T15:56:37.979 INFO:teuthology.orchestra.run.smithi151.stderr:Cannot find device "ceph-brx" 2024-09-16T15:56:37.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T15:56:37.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:56:37.982 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list 2024-09-16T15:56:38.022 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-16T15:56:38.022 INFO:teuthology.orchestra.run.smithi167.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-16T15:56:38.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:56:38.023 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-16T15:56:38.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:56:38.125 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-16T15:56:38.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:56:38.366 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link delete ceph-brx 2024-09-16T15:56:38.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:56:38.653 DEBUG:teuthology.orchestra.run.smithi167:> ip netns list 2024-09-16T15:56:38.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T15:56:38.688 DEBUG:teuthology.orchestra.run.smithi167:> sudo ip link delete ceph-brx 2024-09-16T15:56:38.788 INFO:teuthology.orchestra.run.smithi167.stderr:Cannot find device "ceph-brx" 2024-09-16T15:56:38.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T15:56:38.791 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-16T15:56:38.800 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T15:56:38.800 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:56:38.972 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:38.969+0000 7fc9fba00640 1 -- 172.21.15.151:0/1153142924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc9d8003c10 msgr2=0x7fc9d8004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:38.972 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:38.969+0000 7fc9fba00640 1 --2- 172.21.15.151:0/1153142924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc9d8003c10 0x7fc9d8004010 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fc9e400c560 tx=0x7fc9d8005830 comp rx=0 tx=0).stop 2024-09-16T15:56:38.972 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:38.969+0000 7fc9f9a00640 1 -- 172.21.15.151:0/1153142924 reap_dead start 2024-09-16T15:56:38.972 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:38.969+0000 7fc9fba00640 1 -- 172.21.15.151:0/1153142924 shutdown_connections 2024-09-16T15:56:38.972 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:38.969+0000 7fc9fba00640 1 -- 172.21.15.151:0/1153142924 >> 172.21.15.151:0/1153142924 conn(0x7fc9f40b1950 msgr2=0x7fc9f40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:38.972 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:38.969+0000 7fc9fba00640 1 -- 172.21.15.151:0/1153142924 shutdown_connections 2024-09-16T15:56:38.972 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:38.969+0000 7fc9fba00640 1 -- 172.21.15.151:0/1153142924 wait complete. 2024-09-16T15:56:38.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:38.971+0000 7fc9fba00640 1 Processor -- start 2024-09-16T15:56:38.973 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:38.971+0000 7fc9fba00640 1 -- start start 2024-09-16T15:56:38.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:38.972+0000 7fc9fba00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc9d8003c10 0x7fc9f40b3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:38.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:38.972+0000 7fc9fba00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc9f4156300 0x7fc9f4155ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:38.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:38.972+0000 7fc9fba00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc9f40ca8d0 0x7fc9f41e4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:38.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:38.972+0000 7fc9fba00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f4074840 con 0x7fc9f4156300 2024-09-16T15:56:38.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:38.972+0000 7fc9fba00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f40746c0 con 0x7fc9d8003c10 2024-09-16T15:56:38.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:38.972+0000 7fc9fba00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fc9f406f160 con 0x7fc9f40ca8d0 2024-09-16T15:56:38.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:38.972+0000 7fc9f3e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc9f4156300 0x7fc9f4155ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:38.976 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:38.972+0000 7fc9f3e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc9f4156300 0x7fc9f4155ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37286/0 (socket says 172.21.15.151:37286) 2024-09-16T15:56:38.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:38.972+0000 7fc9f3e00640 1 -- 172.21.15.151:0/1359783722 learned_addr learned my addr 172.21.15.151:0/1359783722 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:56:38.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:38.972+0000 7fc9f9000640 1 --2- 172.21.15.151:0/1359783722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc9d8003c10 0x7fc9f40b3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:38.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:38.972+0000 7fc9f9a00640 1 --2- 172.21.15.151:0/1359783722 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc9f40ca8d0 0x7fc9f41e4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:38.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:38.973+0000 7fc9f3e00640 1 -- 172.21.15.151:0/1359783722 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc9f40ca8d0 msgr2=0x7fc9f41e4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:38.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:38.973+0000 7fc9f3e00640 1 --2- 172.21.15.151:0/1359783722 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc9f40ca8d0 0x7fc9f41e4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:38.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:38.973+0000 7fc9f3e00640 1 -- 172.21.15.151:0/1359783722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc9d8003c10 msgr2=0x7fc9f40b3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:38.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:38.973+0000 7fc9f3e00640 1 --2- 172.21.15.151:0/1359783722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc9d8003c10 0x7fc9f40b3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:38.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:38.973+0000 7fc9f3e00640 1 -- 172.21.15.151:0/1359783722 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9f40c9b10 con 0x7fc9f4156300 2024-09-16T15:56:38.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:38.973+0000 7fc9f3e00640 1 --2- 172.21.15.151:0/1359783722 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc9f4156300 0x7fc9f4155ca0 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0003e10 tx=0x7fc9e0001800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:38.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:38.973+0000 7fc9f0c00640 1 -- 172.21.15.151:0/1359783722 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e0011430 con 0x7fc9f4156300 2024-09-16T15:56:38.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:38.973+0000 7fc9f0c00640 1 -- 172.21.15.151:0/1359783722 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc9e0016020 con 0x7fc9f4156300 2024-09-16T15:56:38.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:38.973+0000 7fc9f0c00640 1 -- 172.21.15.151:0/1359783722 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e0011830 con 0x7fc9f4156300 2024-09-16T15:56:38.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:38.973+0000 7fc9fba00640 1 -- 172.21.15.151:0/1359783722 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9f40c9df0 con 0x7fc9f4156300 2024-09-16T15:56:38.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:38.974+0000 7fc9fba00640 1 -- 172.21.15.151:0/1359783722 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9f40ca3b0 con 0x7fc9f4156300 2024-09-16T15:56:38.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:38.975+0000 7fc9fba00640 1 -- 172.21.15.151:0/1359783722 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9f40ca5b0 con 0x7fc9f4156300 2024-09-16T15:56:38.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:38.976+0000 7fc9f0c00640 1 -- 172.21.15.151:0/1359783722 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 55) ==== 104857+0+0 (secure 0 0 0) 0x7fc9e00119d0 con 0x7fc9f4156300 2024-09-16T15:56:38.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:38.977+0000 7fc9f0c00640 1 --2- 172.21.15.151:0/1359783722 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc9d407afe0 0x7fc9d407d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:38.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:38.977+0000 7fc9f0c00640 1 -- 172.21.15.151:0/1359783722 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5083+0+0 (secure 0 0 0) 0x7fc9e009a5e0 con 0x7fc9f4156300 2024-09-16T15:56:38.979 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:38.977+0000 7fc9f9000640 1 --2- 172.21.15.151:0/1359783722 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc9d407afe0 0x7fc9d407d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:38.980 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:38.978+0000 7fc9f9000640 1 --2- 172.21.15.151:0/1359783722 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc9d407afe0 0x7fc9d407d4a0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fc9e4033000 tx=0x7fc9e4038040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:38.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:38.981+0000 7fc9f0c00640 1 -- 172.21.15.151:0/1359783722 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fc9f40ca5b0 con 0x7fc9f4156300 2024-09-16T15:56:39.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.163+0000 7fc9fba00640 1 -- 172.21.15.151:0/1359783722 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc9f406f6c0 con 0x7fc9d407afe0 2024-09-16T15:56:39.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.164+0000 7fc9f0c00640 1 -- 172.21.15.151:0/1359783722 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13546 (secure 0 0 0) 0x7fc9f406f6c0 con 0x7fc9d407afe0 2024-09-16T15:56:39.167 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:56:39.167 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-09-16T15:56:39.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.167+0000 7fc9fba00640 1 -- 172.21.15.151:0/1359783722 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc9d407afe0 msgr2=0x7fc9d407d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:39.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.167+0000 7fc9fba00640 1 --2- 172.21.15.151:0/1359783722 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc9d407afe0 0x7fc9d407d4a0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fc9e4033000 tx=0x7fc9e4038040 comp rx=0 tx=0).stop 2024-09-16T15:56:39.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.167+0000 7fc9fba00640 1 -- 172.21.15.151:0/1359783722 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc9f4156300 msgr2=0x7fc9f4155ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:39.169 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.167+0000 7fc9fba00640 1 --2- 172.21.15.151:0/1359783722 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc9f4156300 0x7fc9f4155ca0 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0003e10 tx=0x7fc9e0001800 comp rx=0 tx=0).stop 2024-09-16T15:56:39.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.167+0000 7fc9fba00640 1 -- 172.21.15.151:0/1359783722 shutdown_connections 2024-09-16T15:56:39.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.167+0000 7fc9fba00640 1 --2- 172.21.15.151:0/1359783722 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fc9d407afe0 0x7fc9d407d4a0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:39.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.167+0000 7fc9fba00640 1 --2- 172.21.15.151:0/1359783722 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fc9f40ca8d0 0x7fc9f41e4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:39.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.167+0000 7fc9fba00640 1 --2- 172.21.15.151:0/1359783722 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc9f4156300 0x7fc9f4155ca0 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:39.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.167+0000 7fc9fba00640 1 --2- 172.21.15.151:0/1359783722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc9d8003c10 0x7fc9f40b3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:39.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.167+0000 7fc9fba00640 1 -- 172.21.15.151:0/1359783722 >> 172.21.15.151:0/1359783722 conn(0x7fc9f40b1950 msgr2=0x7fc9f40af5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:39.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.167+0000 7fc9fba00640 1 -- 172.21.15.151:0/1359783722 shutdown_connections 2024-09-16T15:56:39.170 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.167+0000 7fc9fba00640 1 -- 172.21.15.151:0/1359783722 wait complete. 2024-09-16T15:56:39.185 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":669,"stamp":"2024-09-16T15:56:38.006507+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":170,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":1241392,"kb_used_data":15864,"kb_used_omap":12,"kb_used_meta":1223347,"kb_avail":753733328,"statfs":{"total":773094113280,"available":771822927872,"internally_reserved":0,"allocated":16244736,"data_stored":7881135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12738,"internal_metadata":1252707902},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-715052043,"num_objects":-194,"num_object_clones":0,"num_object_copies":-388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-194,"num_whiteouts":0,"num_read":-4259,"num_read_kb":-17055839,"num_write":-10430,"num_write_kb":-20130065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001160"},"pg_stats":[{"pgid":"1.0","version":"17'72","reported_seq":360,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T15:56:36.168763+0000","last_change":"2024-09-16T15:39:12.698813+0000","last_active":"2024-09-16T15:56:36.168763+0000","last_peered":"2024-09-16T15:56:36.168763+0000","last_clean":"2024-09-16T15:56:36.168763+0000","last_became_active":"2024-09-16T15:39:12.697695+0000","last_became_peered":"2024-09-16T15:39:12.697695+0000","last_unstale":"2024-09-16T15:56:36.168763+0000","last_undegraded":"2024-09-16T15:56:36.168763+0000","last_fullsized":"2024-09-16T15:56:36.168763+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_clean_scrub_stamp":"2024-09-16T15:39:11.684967+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:02.061081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509654,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":171780,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":169918,"kb_avail":94200060,"statfs":{"total":96636764160,"available":96460861440,"internally_reserved":0,"allocated":1884160,"data_stored":838325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":173996488},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509653,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":152152,"kb_used_data":2420,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94219688,"statfs":{"total":96636764160,"available":96480960512,"internally_reserved":0,"allocated":2478080,"data_stored":1428693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":151255495},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509652,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":167620,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":165758,"kb_avail":94204220,"statfs":{"total":96636764160,"available":96465121280,"internally_reserved":0,"allocated":1884160,"data_stored":838325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":169736645},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509652,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149572,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222268,"statfs":{"total":96636764160,"available":96483602432,"internally_reserved":0,"allocated":1884160,"data_stored":838325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":151255497},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509652,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149576,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222264,"statfs":{"total":96636764160,"available":96483598336,"internally_reserved":0,"allocated":1884160,"data_stored":838325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":151255495},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509653,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":141832,"kb_used_data":2420,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94230008,"statfs":{"total":96636764160,"available":96491528192,"internally_reserved":0,"allocated":2478080,"data_stored":1428693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":142735815},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509651,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":163460,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":161598,"kb_avail":94208380,"statfs":{"total":96636764160,"available":96469381120,"internally_reserved":0,"allocated":1884160,"data_stored":838325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":165476807},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542358,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145400,"kb_used_data":1824,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226440,"statfs":{"total":96636764160,"available":96487874560,"internally_reserved":0,"allocated":1867776,"data_stored":832124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":146995660},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:56:39.186 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:56:39.346 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.344+0000 7fd244a00640 1 -- 172.21.15.151:0/1774653461 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2400d0540 msgr2=0x7fd24006f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:39.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.344+0000 7fd244a00640 1 --2- 172.21.15.151:0/1774653461 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2400d0540 0x7fd24006f320 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fd238005fd0 tx=0x7fd23802aa90 comp rx=0 tx=0).stop 2024-09-16T15:56:39.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.344+0000 7fd244a00640 1 -- 172.21.15.151:0/1774653461 shutdown_connections 2024-09-16T15:56:39.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.344+0000 7fd244a00640 1 --2- 172.21.15.151:0/1774653461 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2400d0540 0x7fd24006f320 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:39.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.344+0000 7fd244a00640 1 --2- 172.21.15.151:0/1774653461 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd24015bc30 0x7fd2400b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:39.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.344+0000 7fd244a00640 1 -- 172.21.15.151:0/1774653461 >> 172.21.15.151:0/1774653461 conn(0x7fd2400cfa20 msgr2=0x7fd2400cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:39.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.344+0000 7fd244a00640 1 -- 172.21.15.151:0/1774653461 shutdown_connections 2024-09-16T15:56:39.347 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.344+0000 7fd244a00640 1 -- 172.21.15.151:0/1774653461 wait complete. 2024-09-16T15:56:39.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.345+0000 7fd244a00640 1 Processor -- start 2024-09-16T15:56:39.348 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.346+0000 7fd244a00640 1 -- start start 2024-09-16T15:56:39.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.346+0000 7fd244a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd2400d0540 0x7fd240158660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:39.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.346+0000 7fd244a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd24015bc30 0x7fd2400b0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:39.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.346+0000 7fd244a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2400b0fd0 0x7fd2400ce200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:39.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.346+0000 7fd244a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd240074660 con 0x7fd2400d0540 2024-09-16T15:56:39.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.346+0000 7fd244a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd24006f860 con 0x7fd2400b0fd0 2024-09-16T15:56:39.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.346+0000 7fd244a00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fd2400b1460 con 0x7fd24015bc30 2024-09-16T15:56:39.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.347+0000 7fd23f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd2400d0540 0x7fd240158660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:39.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.347+0000 7fd23f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd2400d0540 0x7fd240158660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37290/0 (socket says 172.21.15.151:37290) 2024-09-16T15:56:39.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.347+0000 7fd23f400640 1 -- 172.21.15.151:0/1299144554 learned_addr learned my addr 172.21.15.151:0/1299144554 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:56:39.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.347+0000 7fd23fe00640 1 --2- 172.21.15.151:0/1299144554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2400b0fd0 0x7fd2400ce200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:39.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.347+0000 7fd23ea00640 1 --2- 172.21.15.151:0/1299144554 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd24015bc30 0x7fd2400b0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:39.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.347+0000 7fd23f400640 1 -- 172.21.15.151:0/1299144554 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd24015bc30 msgr2=0x7fd2400b0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:39.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.347+0000 7fd23f400640 1 --2- 172.21.15.151:0/1299144554 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd24015bc30 0x7fd2400b0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:39.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.347+0000 7fd23f400640 1 -- 172.21.15.151:0/1299144554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2400b0fd0 msgr2=0x7fd2400ce200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:39.352 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.347+0000 7fd23f400640 1 --2- 172.21.15.151:0/1299144554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2400b0fd0 0x7fd2400ce200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:39.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.347+0000 7fd23f400640 1 -- 172.21.15.151:0/1299144554 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2400ae240 con 0x7fd2400d0540 2024-09-16T15:56:39.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.347+0000 7fd23fe00640 1 --2- 172.21.15.151:0/1299144554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2400b0fd0 0x7fd2400ce200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:56:39.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.347+0000 7fd23f400640 1 --2- 172.21.15.151:0/1299144554 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd2400d0540 0x7fd240158660 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7fd23000e790 tx=0x7fd23000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:39.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.348+0000 7fd237400640 1 -- 172.21.15.151:0/1299144554 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd23000eea0 con 0x7fd2400d0540 2024-09-16T15:56:39.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.348+0000 7fd237400640 1 -- 172.21.15.151:0/1299144554 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd23000cdb0 con 0x7fd2400d0540 2024-09-16T15:56:39.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.348+0000 7fd244a00640 1 -- 172.21.15.151:0/1299144554 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2400ae520 con 0x7fd2400d0540 2024-09-16T15:56:39.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.348+0000 7fd237400640 1 -- 172.21.15.151:0/1299144554 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd230010690 con 0x7fd2400d0540 2024-09-16T15:56:39.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.348+0000 7fd244a00640 1 -- 172.21.15.151:0/1299144554 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd240158ba0 con 0x7fd2400d0540 2024-09-16T15:56:39.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.349+0000 7fd244a00640 1 -- 172.21.15.151:0/1299144554 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd24006fdf0 con 0x7fd2400d0540 2024-09-16T15:56:39.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.350+0000 7fd237400640 1 -- 172.21.15.151:0/1299144554 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 55) ==== 104857+0+0 (secure 0 0 0) 0x7fd230010830 con 0x7fd2400d0540 2024-09-16T15:56:39.354 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.351+0000 7fd237400640 1 --2- 172.21.15.151:0/1299144554 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd22007afb0 0x7fd22007d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:39.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.351+0000 7fd237400640 1 -- 172.21.15.151:0/1299144554 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5083+0+0 (secure 0 0 0) 0x7fd23009a6a0 con 0x7fd2400d0540 2024-09-16T15:56:39.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.351+0000 7fd23ea00640 1 --2- 172.21.15.151:0/1299144554 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd22007afb0 0x7fd22007d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:39.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.352+0000 7fd23ea00640 1 --2- 172.21.15.151:0/1299144554 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd22007afb0 0x7fd22007d470 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fd2401f7d10 tx=0x7fd22c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:39.357 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.355+0000 7fd237400640 1 -- 172.21.15.151:0/1299144554 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd230064770 con 0x7fd2400d0540 2024-09-16T15:56:39.542 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.540+0000 7fd244a00640 1 -- 172.21.15.151:0/1299144554 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd240158e50 con 0x7fd22007afb0 2024-09-16T15:56:39.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.541+0000 7fd237400640 1 -- 172.21.15.151:0/1299144554 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13546 (secure 0 0 0) 0x7fd240158e50 con 0x7fd22007afb0 2024-09-16T15:56:39.543 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:56:39.544 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-09-16T15:56:39.546 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.543+0000 7fd244a00640 1 -- 172.21.15.151:0/1299144554 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd22007afb0 msgr2=0x7fd22007d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:39.546 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.544+0000 7fd244a00640 1 --2- 172.21.15.151:0/1299144554 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd22007afb0 0x7fd22007d470 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fd2401f7d10 tx=0x7fd22c009040 comp rx=0 tx=0).stop 2024-09-16T15:56:39.546 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.544+0000 7fd244a00640 1 -- 172.21.15.151:0/1299144554 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd2400d0540 msgr2=0x7fd240158660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:39.546 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.544+0000 7fd244a00640 1 --2- 172.21.15.151:0/1299144554 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd2400d0540 0x7fd240158660 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7fd23000e790 tx=0x7fd23000ec50 comp rx=0 tx=0).stop 2024-09-16T15:56:39.546 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.544+0000 7fd244a00640 1 -- 172.21.15.151:0/1299144554 shutdown_connections 2024-09-16T15:56:39.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.544+0000 7fd244a00640 1 --2- 172.21.15.151:0/1299144554 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd22007afb0 0x7fd22007d470 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:39.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.544+0000 7fd244a00640 1 --2- 172.21.15.151:0/1299144554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2400b0fd0 0x7fd2400ce200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:39.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.544+0000 7fd244a00640 1 --2- 172.21.15.151:0/1299144554 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd24015bc30 0x7fd2400b0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:39.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.544+0000 7fd244a00640 1 --2- 172.21.15.151:0/1299144554 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd2400d0540 0x7fd240158660 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:39.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.544+0000 7fd244a00640 1 -- 172.21.15.151:0/1299144554 >> 172.21.15.151:0/1299144554 conn(0x7fd2400cfa20 msgr2=0x7fd24015a230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:39.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.544+0000 7fd244a00640 1 -- 172.21.15.151:0/1299144554 shutdown_connections 2024-09-16T15:56:39.547 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.544+0000 7fd244a00640 1 -- 172.21.15.151:0/1299144554 wait complete. 2024-09-16T15:56:39.562 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":669,"stamp":"2024-09-16T15:56:38.006507+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":170,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":1241392,"kb_used_data":15864,"kb_used_omap":12,"kb_used_meta":1223347,"kb_avail":753733328,"statfs":{"total":773094113280,"available":771822927872,"internally_reserved":0,"allocated":16244736,"data_stored":7881135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12738,"internal_metadata":1252707902},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-715052043,"num_objects":-194,"num_object_clones":0,"num_object_copies":-388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-194,"num_whiteouts":0,"num_read":-4259,"num_read_kb":-17055839,"num_write":-10430,"num_write_kb":-20130065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001160"},"pg_stats":[{"pgid":"1.0","version":"17'72","reported_seq":360,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T15:56:36.168763+0000","last_change":"2024-09-16T15:39:12.698813+0000","last_active":"2024-09-16T15:56:36.168763+0000","last_peered":"2024-09-16T15:56:36.168763+0000","last_clean":"2024-09-16T15:56:36.168763+0000","last_became_active":"2024-09-16T15:39:12.697695+0000","last_became_peered":"2024-09-16T15:39:12.697695+0000","last_unstale":"2024-09-16T15:56:36.168763+0000","last_undegraded":"2024-09-16T15:56:36.168763+0000","last_fullsized":"2024-09-16T15:56:36.168763+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_clean_scrub_stamp":"2024-09-16T15:39:11.684967+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:02.061081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509654,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":171780,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":169918,"kb_avail":94200060,"statfs":{"total":96636764160,"available":96460861440,"internally_reserved":0,"allocated":1884160,"data_stored":838325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":173996488},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509653,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":152152,"kb_used_data":2420,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94219688,"statfs":{"total":96636764160,"available":96480960512,"internally_reserved":0,"allocated":2478080,"data_stored":1428693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":151255495},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509652,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":167620,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":165758,"kb_avail":94204220,"statfs":{"total":96636764160,"available":96465121280,"internally_reserved":0,"allocated":1884160,"data_stored":838325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":169736645},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509652,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149572,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222268,"statfs":{"total":96636764160,"available":96483602432,"internally_reserved":0,"allocated":1884160,"data_stored":838325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":151255497},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509652,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149576,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222264,"statfs":{"total":96636764160,"available":96483598336,"internally_reserved":0,"allocated":1884160,"data_stored":838325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":151255495},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509653,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":141832,"kb_used_data":2420,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94230008,"statfs":{"total":96636764160,"available":96491528192,"internally_reserved":0,"allocated":2478080,"data_stored":1428693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":142735815},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509651,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":163460,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":161598,"kb_avail":94208380,"statfs":{"total":96636764160,"available":96469381120,"internally_reserved":0,"allocated":1884160,"data_stored":838325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":165476807},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542358,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145400,"kb_used_data":1824,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226440,"statfs":{"total":96636764160,"available":96487874560,"internally_reserved":0,"allocated":1867776,"data_stored":832124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":146995660},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:56:39.563 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T15:56:39.563 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:56:39.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.719+0000 7f7faa400640 1 -- 172.21.15.151:0/3297319877 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7fa415f5a0 msgr2=0x7f7fa40b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:39.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.719+0000 7f7faa400640 1 --2- 172.21.15.151:0/3297319877 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7fa415f5a0 0x7f7fa40b2440 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f7f94005fd0 tx=0x7f7f9402a4b0 comp rx=0 tx=0).stop 2024-09-16T15:56:39.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.719+0000 7f7faa400640 1 -- 172.21.15.151:0/3297319877 shutdown_connections 2024-09-16T15:56:39.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.719+0000 7f7faa400640 1 --2- 172.21.15.151:0/3297319877 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7fa4073fb0 0x7f7fa4074440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:39.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.719+0000 7f7faa400640 1 --2- 172.21.15.151:0/3297319877 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7fa415f5a0 0x7f7fa40b2440 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:39.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.719+0000 7f7faa400640 1 -- 172.21.15.151:0/3297319877 >> 172.21.15.151:0/3297319877 conn(0x7f7fa40b1950 msgr2=0x7f7fa40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:39.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.719+0000 7f7faa400640 1 -- 172.21.15.151:0/3297319877 shutdown_connections 2024-09-16T15:56:39.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.719+0000 7f7faa400640 1 -- 172.21.15.151:0/3297319877 wait complete. 2024-09-16T15:56:39.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.721+0000 7f7faa400640 1 Processor -- start 2024-09-16T15:56:39.723 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.721+0000 7f7faa400640 1 -- start start 2024-09-16T15:56:39.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.722+0000 7f7faa400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7fa4073fb0 0x7f7fa41465f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:39.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.722+0000 7f7faa400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7fa40ccf00 0x7f7fa4153650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:39.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.722+0000 7f7faa400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7fa415f5a0 0x7f7fa414e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:39.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.722+0000 7f7faa400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa406f2e0 con 0x7f7fa40ccf00 2024-09-16T15:56:39.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.722+0000 7f7faa400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa406f160 con 0x7f7fa415f5a0 2024-09-16T15:56:39.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.722+0000 7f7faa400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f7fa406f460 con 0x7f7fa4073fb0 2024-09-16T15:56:39.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.722+0000 7f7fa2a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7fa40ccf00 0x7f7fa4153650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:39.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.722+0000 7f7fa2a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7fa40ccf00 0x7f7fa4153650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37316/0 (socket says 172.21.15.151:37316) 2024-09-16T15:56:39.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.722+0000 7f7fa2a00640 1 -- 172.21.15.151:0/2090378770 learned_addr learned my addr 172.21.15.151:0/2090378770 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:56:39.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.722+0000 7f7fa3400640 1 --2- 172.21.15.151:0/2090378770 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7fa4073fb0 0x7f7fa41465f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:39.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.722+0000 7f7fa3e00640 1 --2- 172.21.15.151:0/2090378770 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7fa415f5a0 0x7f7fa414e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:39.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.723+0000 7f7fa2a00640 1 -- 172.21.15.151:0/2090378770 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7fa4073fb0 msgr2=0x7f7fa41465f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:39.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.723+0000 7f7fa2a00640 1 --2- 172.21.15.151:0/2090378770 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7fa4073fb0 0x7f7fa41465f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:39.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.723+0000 7f7fa2a00640 1 -- 172.21.15.151:0/2090378770 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7fa415f5a0 msgr2=0x7f7fa414e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:39.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.723+0000 7f7fa2a00640 1 --2- 172.21.15.151:0/2090378770 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7fa415f5a0 0x7f7fa414e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:39.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.723+0000 7f7fa2a00640 1 -- 172.21.15.151:0/2090378770 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fa4153b90 con 0x7f7fa40ccf00 2024-09-16T15:56:39.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.723+0000 7f7fa2a00640 1 --2- 172.21.15.151:0/2090378770 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7fa40ccf00 0x7f7fa4153650 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f7f9000c430 tx=0x7f7f9000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:39.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.723+0000 7f7f9b400640 1 -- 172.21.15.151:0/2090378770 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f90011790 con 0x7f7fa40ccf00 2024-09-16T15:56:39.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.723+0000 7f7f9b400640 1 -- 172.21.15.151:0/2090378770 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7f90017070 con 0x7f7fa40ccf00 2024-09-16T15:56:39.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.723+0000 7f7faa400640 1 -- 172.21.15.151:0/2090378770 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fa4153e60 con 0x7f7fa40ccf00 2024-09-16T15:56:39.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.723+0000 7f7f9b400640 1 -- 172.21.15.151:0/2090378770 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f90011d00 con 0x7f7fa40ccf00 2024-09-16T15:56:39.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.723+0000 7f7faa400640 1 -- 172.21.15.151:0/2090378770 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fa40b30d0 con 0x7f7fa40ccf00 2024-09-16T15:56:39.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.724+0000 7f7faa400640 1 -- 172.21.15.151:0/2090378770 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fa414e640 con 0x7f7fa40ccf00 2024-09-16T15:56:39.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.726+0000 7f7f9b400640 1 -- 172.21.15.151:0/2090378770 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 55) ==== 104857+0+0 (secure 0 0 0) 0x7f7f9001c460 con 0x7f7fa40ccf00 2024-09-16T15:56:39.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.727+0000 7f7f9b400640 1 --2- 172.21.15.151:0/2090378770 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7f8407afb0 0x7f7f8407d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:39.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.727+0000 7f7f9b400640 1 -- 172.21.15.151:0/2090378770 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5083+0+0 (secure 0 0 0) 0x7f7f90099a80 con 0x7f7fa40ccf00 2024-09-16T15:56:39.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.727+0000 7f7fa3400640 1 --2- 172.21.15.151:0/2090378770 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7f8407afb0 0x7f7f8407d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:39.729 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.728+0000 7f7fa3400640 1 --2- 172.21.15.151:0/2090378770 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7f8407afb0 0x7f7f8407d470 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f7f94033000 tx=0x7f7f94004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:39.732 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.731+0000 7f7f9b400640 1 -- 172.21.15.151:0/2090378770 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f7f90063480 con 0x7f7fa40ccf00 2024-09-16T15:56:39.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.915+0000 7f7faa400640 1 -- 172.21.15.151:0/2090378770 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7fa406aaf0 con 0x7f7f8407afb0 2024-09-16T15:56:39.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.915+0000 7f7f9b400640 1 -- 172.21.15.151:0/2090378770 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13546 (secure 0 0 0) 0x7f7fa406aaf0 con 0x7f7f8407afb0 2024-09-16T15:56:39.918 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:56:39.918 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-09-16T15:56:39.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.918+0000 7f7faa400640 1 -- 172.21.15.151:0/2090378770 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7f8407afb0 msgr2=0x7f7f8407d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:39.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.918+0000 7f7faa400640 1 --2- 172.21.15.151:0/2090378770 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7f8407afb0 0x7f7f8407d470 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f7f94033000 tx=0x7f7f94004730 comp rx=0 tx=0).stop 2024-09-16T15:56:39.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.918+0000 7f7faa400640 1 -- 172.21.15.151:0/2090378770 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7fa40ccf00 msgr2=0x7f7fa4153650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:39.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.918+0000 7f7faa400640 1 --2- 172.21.15.151:0/2090378770 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7fa40ccf00 0x7f7fa4153650 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f7f9000c430 tx=0x7f7f9000c8f0 comp rx=0 tx=0).stop 2024-09-16T15:56:39.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.918+0000 7f7faa400640 1 -- 172.21.15.151:0/2090378770 shutdown_connections 2024-09-16T15:56:39.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.918+0000 7f7faa400640 1 --2- 172.21.15.151:0/2090378770 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f7f8407afb0 0x7f7f8407d470 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:39.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.918+0000 7f7faa400640 1 --2- 172.21.15.151:0/2090378770 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7fa415f5a0 0x7f7fa414e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:39.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.918+0000 7f7faa400640 1 --2- 172.21.15.151:0/2090378770 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7fa40ccf00 0x7f7fa4153650 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:39.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.918+0000 7f7faa400640 1 --2- 172.21.15.151:0/2090378770 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7fa4073fb0 0x7f7fa41465f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:39.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.918+0000 7f7faa400640 1 -- 172.21.15.151:0/2090378770 >> 172.21.15.151:0/2090378770 conn(0x7f7fa40b1950 msgr2=0x7f7fa41540c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:39.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.918+0000 7f7faa400640 1 -- 172.21.15.151:0/2090378770 shutdown_connections 2024-09-16T15:56:39.922 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:39.918+0000 7f7faa400640 1 -- 172.21.15.151:0/2090378770 wait complete. 2024-09-16T15:56:39.937 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":669,"stamp":"2024-09-16T15:56:38.006507+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":170,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":1241392,"kb_used_data":15864,"kb_used_omap":12,"kb_used_meta":1223347,"kb_avail":753733328,"statfs":{"total":773094113280,"available":771822927872,"internally_reserved":0,"allocated":16244736,"data_stored":7881135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12738,"internal_metadata":1252707902},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-715052043,"num_objects":-194,"num_object_clones":0,"num_object_copies":-388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-194,"num_whiteouts":0,"num_read":-4259,"num_read_kb":-17055839,"num_write":-10430,"num_write_kb":-20130065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001160"},"pg_stats":[{"pgid":"1.0","version":"17'72","reported_seq":360,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T15:56:36.168763+0000","last_change":"2024-09-16T15:39:12.698813+0000","last_active":"2024-09-16T15:56:36.168763+0000","last_peered":"2024-09-16T15:56:36.168763+0000","last_clean":"2024-09-16T15:56:36.168763+0000","last_became_active":"2024-09-16T15:39:12.697695+0000","last_became_peered":"2024-09-16T15:39:12.697695+0000","last_unstale":"2024-09-16T15:56:36.168763+0000","last_undegraded":"2024-09-16T15:56:36.168763+0000","last_fullsized":"2024-09-16T15:56:36.168763+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_clean_scrub_stamp":"2024-09-16T15:39:11.684967+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:02.061081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509654,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":171780,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":169918,"kb_avail":94200060,"statfs":{"total":96636764160,"available":96460861440,"internally_reserved":0,"allocated":1884160,"data_stored":838325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":173996488},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509653,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":152152,"kb_used_data":2420,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94219688,"statfs":{"total":96636764160,"available":96480960512,"internally_reserved":0,"allocated":2478080,"data_stored":1428693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":151255495},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509652,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":167620,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":165758,"kb_avail":94204220,"statfs":{"total":96636764160,"available":96465121280,"internally_reserved":0,"allocated":1884160,"data_stored":838325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":169736645},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509652,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149572,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222268,"statfs":{"total":96636764160,"available":96483602432,"internally_reserved":0,"allocated":1884160,"data_stored":838325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":151255497},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509652,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149576,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222264,"statfs":{"total":96636764160,"available":96483598336,"internally_reserved":0,"allocated":1884160,"data_stored":838325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":151255495},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509653,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":141832,"kb_used_data":2420,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94230008,"statfs":{"total":96636764160,"available":96491528192,"internally_reserved":0,"allocated":2478080,"data_stored":1428693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":142735815},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509651,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":163460,"kb_used_data":1840,"kb_used_omap":1,"kb_used_meta":161598,"kb_avail":94208380,"statfs":{"total":96636764160,"available":96469381120,"internally_reserved":0,"allocated":1884160,"data_stored":838325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":165476807},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542358,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145400,"kb_used_data":1824,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226440,"statfs":{"total":96636764160,"available":96487874560,"internally_reserved":0,"allocated":1867776,"data_stored":832124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":146995660},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:56:39.938 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T15:56:40.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:40.093+0000 7f571e800640 1 -- 172.21.15.151:0/55125603 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f571815f5a0 msgr2=0x7f57180b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:40.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:40.093+0000 7f571e800640 1 --2- 172.21.15.151:0/55125603 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f571815f5a0 0x7f57180b2440 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f5708005fd0 tx=0x7f570802a980 comp rx=0 tx=0).stop 2024-09-16T15:56:40.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:40.093+0000 7f571e800640 1 -- 172.21.15.151:0/55125603 shutdown_connections 2024-09-16T15:56:40.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:40.093+0000 7f571e800640 1 --2- 172.21.15.151:0/55125603 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f5718073fb0 0x7f5718074440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:40.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:40.093+0000 7f571e800640 1 --2- 172.21.15.151:0/55125603 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f571815f5a0 0x7f57180b2440 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:40.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:40.093+0000 7f571e800640 1 -- 172.21.15.151:0/55125603 >> 172.21.15.151:0/55125603 conn(0x7f57180b1950 msgr2=0x7f57180b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:40.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:40.093+0000 7f571e800640 1 -- 172.21.15.151:0/55125603 shutdown_connections 2024-09-16T15:56:40.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:40.093+0000 7f571e800640 1 -- 172.21.15.151:0/55125603 wait complete. 2024-09-16T15:56:40.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:40.095+0000 7f571e800640 1 Processor -- start 2024-09-16T15:56:40.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:40.095+0000 7f571e800640 1 -- start start 2024-09-16T15:56:40.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:40.096+0000 7f571e800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5718073fb0 0x7f5718154970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:40.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:40.096+0000 7f571e800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f57180ccf00 0x7f5718153cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:40.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:40.096+0000 7f571e800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f571815f5a0 0x7f571809cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:40.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:40.096+0000 7f571e800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f571806f2e0 con 0x7f5718073fb0 2024-09-16T15:56:40.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:40.096+0000 7f571e800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f571806f160 con 0x7f571815f5a0 2024-09-16T15:56:40.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:40.096+0000 7f571e800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f571806f460 con 0x7f57180ccf00 2024-09-16T15:56:40.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:40.096+0000 7f5717400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5718073fb0 0x7f5718154970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:40.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:40.096+0000 7f5717400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5718073fb0 0x7f5718154970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37334/0 (socket says 172.21.15.151:37334) 2024-09-16T15:56:40.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:40.096+0000 7f5717400640 1 -- 172.21.15.151:0/2562251436 learned_addr learned my addr 172.21.15.151:0/2562251436 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:56:40.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:40.096+0000 7f5717400640 1 -- 172.21.15.151:0/2562251436 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f57180ccf00 msgr2=0x7f5718153cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:40.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:40.096+0000 7f5717400640 1 --2- 172.21.15.151:0/2562251436 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f57180ccf00 0x7f5718153cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:40.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:40.096+0000 7f5717400640 1 -- 172.21.15.151:0/2562251436 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f571815f5a0 msgr2=0x7f571809cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:40.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:40.096+0000 7f5717400640 1 --2- 172.21.15.151:0/2562251436 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f571815f5a0 0x7f571809cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:40.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:40.096+0000 7f5717400640 1 -- 172.21.15.151:0/2562251436 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57180cbef0 con 0x7f5718073fb0 2024-09-16T15:56:40.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:40.097+0000 7f5717400640 1 --2- 172.21.15.151:0/2562251436 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5718073fb0 0x7f5718154970 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f5708006010 tx=0x7f5708003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:40.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:40.097+0000 7f570f400640 1 -- 172.21.15.151:0/2562251436 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5708034040 con 0x7f5718073fb0 2024-09-16T15:56:40.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:40.097+0000 7f570f400640 1 -- 172.21.15.151:0/2562251436 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5708004640 con 0x7f5718073fb0 2024-09-16T15:56:40.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:40.097+0000 7f571e800640 1 -- 172.21.15.151:0/2562251436 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5718154230 con 0x7f5718073fb0 2024-09-16T15:56:40.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:40.097+0000 7f570f400640 1 -- 172.21.15.151:0/2562251436 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5708045480 con 0x7f5718073fb0 2024-09-16T15:56:40.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:40.097+0000 7f571e800640 1 -- 172.21.15.151:0/2562251436 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57181562e0 con 0x7f5718073fb0 2024-09-16T15:56:40.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:40.098+0000 7f571e800640 1 -- 172.21.15.151:0/2562251436 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f571806f980 con 0x7f5718073fb0 2024-09-16T15:56:40.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:40.099+0000 7f570f400640 1 -- 172.21.15.151:0/2562251436 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 55) ==== 104857+0+0 (secure 0 0 0) 0x7f5708037020 con 0x7f5718073fb0 2024-09-16T15:56:40.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:40.100+0000 7f570f400640 1 --2- 172.21.15.151:0/2562251436 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f56f807afb0 0x7f56f807d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:40.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:40.100+0000 7f570f400640 1 -- 172.21.15.151:0/2562251436 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5083+0+0 (secure 0 0 0) 0x7f57080bb640 con 0x7f5718073fb0 2024-09-16T15:56:40.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:40.101+0000 7f5716a00640 1 --2- 172.21.15.151:0/2562251436 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f56f807afb0 0x7f56f807d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:40.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:40.101+0000 7f5716a00640 1 --2- 172.21.15.151:0/2562251436 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f56f807afb0 0x7f56f807d470 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f57181f80a0 tx=0x7f57040079b0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:40.106 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:40.104+0000 7f570f400640 1 -- 172.21.15.151:0/2562251436 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f5708085710 con 0x7f5718073fb0 2024-09-16T15:56:40.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:40.289+0000 7f571e800640 1 -- 172.21.15.151:0/2562251436 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f571815ebc0 con 0x7f5718073fb0 2024-09-16T15:56:40.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:40.290+0000 7f570f400640 1 -- 172.21.15.151:0/2562251436 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v152) ==== 74+0+13326 (secure 0 0 0) 0x7f5708085010 con 0x7f5718073fb0 2024-09-16T15:56:40.292 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:56:40.292 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":152,"fsid":"1757be18-e41a-4446-bb4f-ee4ac81c39df","created":"2024-09-16T15:38:56.348535+0000","modified":"2024-09-16T15:56:35.873138+0000","last_up_change":"2024-09-16T15:39:10.681384+0000","last_in_change":"2024-09-16T15:39:00.544477+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T15:39:11.602252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d5524076-a903-4d0a-af6a-caf474c4765d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6809","nonce":3111342741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6811","nonce":3111342741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6815","nonce":3111342741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3111342741},{"type":"v1","addr":"172.21.15.151:6813","nonce":3111342741}]},"public_addr":"172.21.15.151:6809/3111342741","cluster_addr":"172.21.15.151:6811/3111342741","heartbeat_back_addr":"172.21.15.151:6815/3111342741","heartbeat_front_addr":"172.21.15.151:6813/3111342741","state":["exists","up"]},{"osd":1,"uuid":"dd25925f-0ae1-494e-8529-4dc205759db3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6801","nonce":3368304087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6803","nonce":3368304087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6807","nonce":3368304087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":3368304087},{"type":"v1","addr":"172.21.15.151:6805","nonce":3368304087}]},"public_addr":"172.21.15.151:6801/3368304087","cluster_addr":"172.21.15.151:6803/3368304087","heartbeat_back_addr":"172.21.15.151:6807/3368304087","heartbeat_front_addr":"172.21.15.151:6805/3368304087","state":["exists","up"]},{"osd":2,"uuid":"c2f18115-013d-4da6-a0fe-cb978f4494b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6823","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6825","nonce":3079373271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6827","nonce":3079373271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6831","nonce":3079373271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3079373271},{"type":"v1","addr":"172.21.15.151:6829","nonce":3079373271}]},"public_addr":"172.21.15.151:6825/3079373271","cluster_addr":"172.21.15.151:6827/3079373271","heartbeat_back_addr":"172.21.15.151:6831/3079373271","heartbeat_front_addr":"172.21.15.151:6829/3079373271","state":["exists","up"]},{"osd":3,"uuid":"36edb718-2fcb-4d88-8ddc-c321eed4279d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6817","nonce":1142509041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6819","nonce":1142509041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6824","nonce":1142509041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":1142509041},{"type":"v1","addr":"172.21.15.151:6821","nonce":1142509041}]},"public_addr":"172.21.15.151:6817/1142509041","cluster_addr":"172.21.15.151:6819/1142509041","heartbeat_back_addr":"172.21.15.151:6824/1142509041","heartbeat_front_addr":"172.21.15.151:6821/1142509041","state":["exists","up"]},{"osd":4,"uuid":"64d33708-cd0a-4236-abe3-1b646e82ab54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6811","nonce":2735495271}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6813","nonce":2735495271}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6817","nonce":2735495271}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2735495271},{"type":"v1","addr":"172.21.15.167:6815","nonce":2735495271}]},"public_addr":"172.21.15.167:6811/2735495271","cluster_addr":"172.21.15.167:6813/2735495271","heartbeat_back_addr":"172.21.15.167:6817/2735495271","heartbeat_front_addr":"172.21.15.167:6815/2735495271","state":["exists","up"]},{"osd":5,"uuid":"f14b44a9-5910-458c-b518-09a8d0de94de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6827","nonce":1494916572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6829","nonce":1494916572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6833","nonce":1494916572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":1494916572},{"type":"v1","addr":"172.21.15.167:6831","nonce":1494916572}]},"public_addr":"172.21.15.167:6827/1494916572","cluster_addr":"172.21.15.167:6829/1494916572","heartbeat_back_addr":"172.21.15.167:6833/1494916572","heartbeat_front_addr":"172.21.15.167:6831/1494916572","state":["exists","up"]},{"osd":6,"uuid":"39335545-2654-4710-84b0-a9aa47cfcbba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6819","nonce":2538273273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6821","nonce":2538273273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6825","nonce":2538273273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":2538273273},{"type":"v1","addr":"172.21.15.167:6823","nonce":2538273273}]},"public_addr":"172.21.15.167:6819/2538273273","cluster_addr":"172.21.15.167:6821/2538273273","heartbeat_back_addr":"172.21.15.167:6825/2538273273","heartbeat_front_addr":"172.21.15.167:6823/2538273273","state":["exists","up"]},{"osd":7,"uuid":"4f8bfc81-53dd-4339-ac22-1a4007d9f963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6803","nonce":265072227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6805","nonce":265072227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6809","nonce":265072227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":265072227},{"type":"v1","addr":"172.21.15.167:6807","nonce":265072227}]},"public_addr":"172.21.15.167:6803/265072227","cluster_addr":"172.21.15.167:6805/265072227","heartbeat_back_addr":"172.21.15.167:6809/265072227","heartbeat_front_addr":"172.21.15.167:6807/265072227","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.150359+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.084358+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.722368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.332809+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.320709+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:07.395153+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.935250+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T15:39:06.801443+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.167:6836/1231498178":"2024-09-17T15:56:32.680388+0000","172.21.15.167:6834/1231498178":"2024-09-17T15:56:32.680388+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T15:56:40.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:40.293+0000 7f571e800640 1 -- 172.21.15.151:0/2562251436 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f56f807afb0 msgr2=0x7f56f807d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:40.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:40.293+0000 7f571e800640 1 --2- 172.21.15.151:0/2562251436 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f56f807afb0 0x7f56f807d470 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f57181f80a0 tx=0x7f57040079b0 comp rx=0 tx=0).stop 2024-09-16T15:56:40.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:40.293+0000 7f571e800640 1 -- 172.21.15.151:0/2562251436 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5718073fb0 msgr2=0x7f5718154970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:40.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:40.293+0000 7f571e800640 1 --2- 172.21.15.151:0/2562251436 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5718073fb0 0x7f5718154970 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f5708006010 tx=0x7f5708003f90 comp rx=0 tx=0).stop 2024-09-16T15:56:40.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:40.293+0000 7f571e800640 1 -- 172.21.15.151:0/2562251436 shutdown_connections 2024-09-16T15:56:40.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:40.293+0000 7f571e800640 1 --2- 172.21.15.151:0/2562251436 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f56f807afb0 0x7f56f807d470 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:40.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:40.293+0000 7f571e800640 1 --2- 172.21.15.151:0/2562251436 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f571815f5a0 0x7f571809cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:40.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:40.293+0000 7f571e800640 1 --2- 172.21.15.151:0/2562251436 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f57180ccf00 0x7f5718153cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:40.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:40.293+0000 7f571e800640 1 --2- 172.21.15.151:0/2562251436 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5718073fb0 0x7f5718154970 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:40.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:40.293+0000 7f571e800640 1 -- 172.21.15.151:0/2562251436 >> 172.21.15.151:0/2562251436 conn(0x7f57180b1950 msgr2=0x7f57180b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:40.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:40.293+0000 7f571e800640 1 -- 172.21.15.151:0/2562251436 shutdown_connections 2024-09-16T15:56:40.296 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:40.293+0000 7f571e800640 1 -- 172.21.15.151:0/2562251436 wait complete. 2024-09-16T15:56:41.312 INFO:tasks.ceph:Scrubbing osd.0 2024-09-16T15:56:41.312 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-16T15:56:41.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.476+0000 7efc46600640 1 -- 172.21.15.151:0/2423670760 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc24003a50 msgr2=0x7efc24003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:41.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.476+0000 7efc46600640 1 --2- 172.21.15.151:0/2423670760 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc24003a50 0x7efc24003e70 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7efc30009ed0 tx=0x7efc24005830 comp rx=0 tx=0).stop 2024-09-16T15:56:41.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.476+0000 7efc46600640 1 -- 172.21.15.151:0/2423670760 shutdown_connections 2024-09-16T15:56:41.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.476+0000 7efc46600640 1 --2- 172.21.15.151:0/2423670760 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc24003a50 0x7efc24003e70 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:41.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.476+0000 7efc46600640 1 -- 172.21.15.151:0/2423670760 >> 172.21.15.151:0/2423670760 conn(0x7efc400b18e0 msgr2=0x7efc400b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:41.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.477+0000 7efc46600640 1 -- 172.21.15.151:0/2423670760 shutdown_connections 2024-09-16T15:56:41.479 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.477+0000 7efc46600640 1 -- 172.21.15.151:0/2423670760 wait complete. 2024-09-16T15:56:41.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.478+0000 7efc46600640 1 Processor -- start 2024-09-16T15:56:41.480 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.478+0000 7efc46600640 1 -- start start 2024-09-16T15:56:41.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.479+0000 7efc46600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc24003a50 0x7efc4014e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:41.482 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.479+0000 7efc46600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc401567e0 0x7efc40156c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:41.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.479+0000 7efc46600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efc400acd30 0x7efc400ad1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:41.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.479+0000 7efc46600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efc4006f830 con 0x7efc24003a50 2024-09-16T15:56:41.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.479+0000 7efc46600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efc4006f6b0 con 0x7efc401567e0 2024-09-16T15:56:41.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.479+0000 7efc46600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7efc4006f9b0 con 0x7efc400acd30 2024-09-16T15:56:41.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.479+0000 7efc3f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc24003a50 0x7efc4014e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:41.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.479+0000 7efc3f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc24003a50 0x7efc4014e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37350/0 (socket says 172.21.15.151:37350) 2024-09-16T15:56:41.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.479+0000 7efc3f400640 1 -- 172.21.15.151:0/2657865533 learned_addr learned my addr 172.21.15.151:0/2657865533 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:56:41.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.480+0000 7efc3f400640 1 -- 172.21.15.151:0/2657865533 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efc400acd30 msgr2=0x7efc400ad1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:41.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.480+0000 7efc3f400640 1 --2- 172.21.15.151:0/2657865533 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efc400acd30 0x7efc400ad1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:41.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.480+0000 7efc3f400640 1 -- 172.21.15.151:0/2657865533 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc401567e0 msgr2=0x7efc40156c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:41.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.480+0000 7efc3f400640 1 --2- 172.21.15.151:0/2657865533 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc401567e0 0x7efc40156c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:41.483 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.480+0000 7efc3f400640 1 -- 172.21.15.151:0/2657865533 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc401572a0 con 0x7efc24003a50 2024-09-16T15:56:41.484 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.480+0000 7efc3f400640 1 --2- 172.21.15.151:0/2657865533 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc24003a50 0x7efc4014e7d0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7efc3002aec0 tx=0x7efc30033cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:41.484 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.480+0000 7efc37400640 1 -- 172.21.15.151:0/2657865533 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc30038040 con 0x7efc24003a50 2024-09-16T15:56:41.484 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.480+0000 7efc37400640 1 -- 172.21.15.151:0/2657865533 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc30041070 con 0x7efc24003a50 2024-09-16T15:56:41.484 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.480+0000 7efc46600640 1 -- 172.21.15.151:0/2657865533 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc401f86d0 con 0x7efc24003a50 2024-09-16T15:56:41.484 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.480+0000 7efc37400640 1 -- 172.21.15.151:0/2657865533 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3003c670 con 0x7efc24003a50 2024-09-16T15:56:41.484 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.481+0000 7efc46600640 1 -- 172.21.15.151:0/2657865533 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc401f8c90 con 0x7efc24003a50 2024-09-16T15:56:41.484 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.481+0000 7efc46600640 1 -- 172.21.15.151:0/2657865533 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7efc4014ed10 con 0x7efc24003a50 2024-09-16T15:56:41.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.483+0000 7efc37400640 1 -- 172.21.15.151:0/2657865533 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 55) ==== 104857+0+0 (secure 0 0 0) 0x7efc3003c810 con 0x7efc24003a50 2024-09-16T15:56:41.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.484+0000 7efc37400640 1 --2- 172.21.15.151:0/2657865533 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efc2007afb0 0x7efc2007d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:41.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.484+0000 7efc37400640 1 -- 172.21.15.151:0/2657865533 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5083+0+0 (secure 0 0 0) 0x7efc300baa80 con 0x7efc24003a50 2024-09-16T15:56:41.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.484+0000 7efc37400640 1 --2- 172.21.15.151:0/2657865533 >> [v2:172.21.15.151:6808/3111342741,v1:172.21.15.151:6809/3111342741] conn(0x7efc20080f30 0x7efc200833d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:41.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.484+0000 7efc37400640 1 -- 172.21.15.151:0/2657865533 --> [v2:172.21.15.151:6808/3111342741,v1:172.21.15.151:6809/3111342741] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efc3002be30 con 0x7efc20080f30 2024-09-16T15:56:41.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.484+0000 7efc37400640 1 -- 172.21.15.151:0/2657865533 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_get_version_reply(handle=1 version=152) ==== 24+0+0 (secure 0 0 0) 0x7efc300bb1d0 con 0x7efc24003a50 2024-09-16T15:56:41.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.484+0000 7efc3fe00640 1 --2- 172.21.15.151:0/2657865533 >> [v2:172.21.15.151:6808/3111342741,v1:172.21.15.151:6809/3111342741] conn(0x7efc20080f30 0x7efc200833d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:41.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.484+0000 7efc3ea00640 1 --2- 172.21.15.151:0/2657865533 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efc2007afb0 0x7efc2007d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:41.487 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.485+0000 7efc3fe00640 1 --2- 172.21.15.151:0/2657865533 >> [v2:172.21.15.151:6808/3111342741,v1:172.21.15.151:6809/3111342741] conn(0x7efc20080f30 0x7efc200833d0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:41.487 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.485+0000 7efc3ea00640 1 --2- 172.21.15.151:0/2657865533 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efc2007afb0 0x7efc2007d470 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7efc2c006fd0 tx=0x7efc2c006f40 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:41.487 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.486+0000 7efc37400640 1 -- 172.21.15.151:0/2657865533 <== osd.0 v2:172.21.15.151:6808/3111342741 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efc3002be30 con 0x7efc20080f30 2024-09-16T15:56:41.500 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.499+0000 7efc46600640 1 -- 172.21.15.151:0/2657865533 --> [v2:172.21.15.151:6808/3111342741,v1:172.21.15.151:6809/3111342741] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7efc400746a0 con 0x7efc20080f30 2024-09-16T15:56:41.501 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.499+0000 7efc37400640 1 -- 172.21.15.151:0/2657865533 <== osd.0 v2:172.21.15.151:6808/3111342741 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7efc400746a0 con 0x7efc20080f30 2024-09-16T15:56:41.501 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-16T15:56:41.501 INFO:teuthology.orchestra.run.smithi151.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T15:56:41.501 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-16T15:56:41.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.499+0000 7efc46600640 1 -- 172.21.15.151:0/2657865533 >> [v2:172.21.15.151:6808/3111342741,v1:172.21.15.151:6809/3111342741] conn(0x7efc20080f30 msgr2=0x7efc200833d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:41.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.499+0000 7efc46600640 1 --2- 172.21.15.151:0/2657865533 >> [v2:172.21.15.151:6808/3111342741,v1:172.21.15.151:6809/3111342741] conn(0x7efc20080f30 0x7efc200833d0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:41.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.499+0000 7efc46600640 1 -- 172.21.15.151:0/2657865533 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efc2007afb0 msgr2=0x7efc2007d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:41.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.499+0000 7efc46600640 1 --2- 172.21.15.151:0/2657865533 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efc2007afb0 0x7efc2007d470 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7efc2c006fd0 tx=0x7efc2c006f40 comp rx=0 tx=0).stop 2024-09-16T15:56:41.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.499+0000 7efc46600640 1 -- 172.21.15.151:0/2657865533 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc24003a50 msgr2=0x7efc4014e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:41.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.499+0000 7efc46600640 1 --2- 172.21.15.151:0/2657865533 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc24003a50 0x7efc4014e7d0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7efc3002aec0 tx=0x7efc30033cb0 comp rx=0 tx=0).stop 2024-09-16T15:56:41.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.499+0000 7efc3fe00640 1 -- 172.21.15.151:0/2657865533 reap_dead start 2024-09-16T15:56:41.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.500+0000 7efc46600640 1 -- 172.21.15.151:0/2657865533 shutdown_connections 2024-09-16T15:56:41.502 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.500+0000 7efc46600640 1 -- 172.21.15.151:0/2657865533 >> 172.21.15.151:0/2657865533 conn(0x7efc400b18e0 msgr2=0x7efc400b3bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:41.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.500+0000 7efc46600640 1 -- 172.21.15.151:0/2657865533 shutdown_connections 2024-09-16T15:56:41.503 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.500+0000 7efc46600640 1 -- 172.21.15.151:0/2657865533 wait complete. 2024-09-16T15:56:41.517 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-16T15:56:41.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.674+0000 7fe357400640 1 -- 172.21.15.151:0/190128711 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe350073fb0 msgr2=0x7fe350074440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:41.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.674+0000 7fe357400640 1 --2- 172.21.15.151:0/190128711 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe350073fb0 0x7fe350074440 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7fe348005fd0 tx=0x7fe34802a540 comp rx=0 tx=0).stop 2024-09-16T15:56:41.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.674+0000 7fe357400640 1 -- 172.21.15.151:0/190128711 shutdown_connections 2024-09-16T15:56:41.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.674+0000 7fe357400640 1 --2- 172.21.15.151:0/190128711 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe350073fb0 0x7fe350074440 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:41.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.674+0000 7fe357400640 1 --2- 172.21.15.151:0/190128711 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe3500ccf00 0x7fe3500b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:41.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.674+0000 7fe357400640 1 -- 172.21.15.151:0/190128711 >> 172.21.15.151:0/190128711 conn(0x7fe3500b1950 msgr2=0x7fe3500b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:41.677 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.674+0000 7fe357400640 1 -- 172.21.15.151:0/190128711 shutdown_connections 2024-09-16T15:56:41.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.674+0000 7fe357400640 1 -- 172.21.15.151:0/190128711 wait complete. 2024-09-16T15:56:41.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.676+0000 7fe357400640 1 Processor -- start 2024-09-16T15:56:41.678 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.676+0000 7fe357400640 1 -- start start 2024-09-16T15:56:41.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.677+0000 7fe357400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe350073fb0 0x7fe3501f3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:41.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.677+0000 7fe357400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe3500ccf00 0x7fe3501f5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:41.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.677+0000 7fe357400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe35015f5a0 0x7fe3500cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:41.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.677+0000 7fe357400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe35006f2e0 con 0x7fe3500ccf00 2024-09-16T15:56:41.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.677+0000 7fe357400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe35006f160 con 0x7fe35015f5a0 2024-09-16T15:56:41.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.677+0000 7fe357400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fe35006f460 con 0x7fe350073fb0 2024-09-16T15:56:41.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.677+0000 7fe34fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe3500ccf00 0x7fe3501f5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:41.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.677+0000 7fe34fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe3500ccf00 0x7fe3501f5f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37364/0 (socket says 172.21.15.151:37364) 2024-09-16T15:56:41.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.677+0000 7fe34fe00640 1 -- 172.21.15.151:0/859099613 learned_addr learned my addr 172.21.15.151:0/859099613 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:56:41.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.678+0000 7fe355400640 1 --2- 172.21.15.151:0/859099613 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe35015f5a0 0x7fe3500cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:41.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.678+0000 7fe354a00640 1 --2- 172.21.15.151:0/859099613 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe350073fb0 0x7fe3501f3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:41.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.678+0000 7fe34fe00640 1 -- 172.21.15.151:0/859099613 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe350073fb0 msgr2=0x7fe3501f3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:41.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.678+0000 7fe34fe00640 1 --2- 172.21.15.151:0/859099613 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe350073fb0 0x7fe3501f3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:41.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.678+0000 7fe34fe00640 1 -- 172.21.15.151:0/859099613 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe35015f5a0 msgr2=0x7fe3500cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:41.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.678+0000 7fe34fe00640 1 --2- 172.21.15.151:0/859099613 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe35015f5a0 0x7fe3500cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:41.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.678+0000 7fe34fe00640 1 -- 172.21.15.151:0/859099613 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3501590f0 con 0x7fe3500ccf00 2024-09-16T15:56:41.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.678+0000 7fe355400640 1 --2- 172.21.15.151:0/859099613 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe35015f5a0 0x7fe3500cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:56:41.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.678+0000 7fe34fe00640 1 --2- 172.21.15.151:0/859099613 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe3500ccf00 0x7fe3501f5f10 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7fe33c00d980 tx=0x7fe33c00de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:41.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.678+0000 7fe34cc00640 1 -- 172.21.15.151:0/859099613 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe33c017070 con 0x7fe3500ccf00 2024-09-16T15:56:41.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.678+0000 7fe34cc00640 1 -- 172.21.15.151:0/859099613 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe33c009d20 con 0x7fe3500ccf00 2024-09-16T15:56:41.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.678+0000 7fe357400640 1 -- 172.21.15.151:0/859099613 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3501f3670 con 0x7fe3500ccf00 2024-09-16T15:56:41.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.678+0000 7fe34cc00640 1 -- 172.21.15.151:0/859099613 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe33c0106c0 con 0x7fe3500ccf00 2024-09-16T15:56:41.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.679+0000 7fe357400640 1 -- 172.21.15.151:0/859099613 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe35015ba70 con 0x7fe3500ccf00 2024-09-16T15:56:41.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.680+0000 7fe357400640 1 -- 172.21.15.151:0/859099613 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe35006f9f0 con 0x7fe3500ccf00 2024-09-16T15:56:41.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.681+0000 7fe34cc00640 1 -- 172.21.15.151:0/859099613 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 55) ==== 104857+0+0 (secure 0 0 0) 0x7fe33c010860 con 0x7fe3500ccf00 2024-09-16T15:56:41.685 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.682+0000 7fe34cc00640 1 --2- 172.21.15.151:0/859099613 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe33007afb0 0x7fe33007d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:41.685 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.682+0000 7fe34cc00640 1 -- 172.21.15.151:0/859099613 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5083+0+0 (secure 0 0 0) 0x7fe33c09a100 con 0x7fe3500ccf00 2024-09-16T15:56:41.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.682+0000 7fe354a00640 1 --2- 172.21.15.151:0/859099613 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe33007afb0 0x7fe33007d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:41.686 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.683+0000 7fe354a00640 1 --2- 172.21.15.151:0/859099613 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe33007afb0 0x7fe33007d470 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fe3500cdf40 tx=0x7fe340009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:41.688 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.686+0000 7fe34cc00640 1 -- 172.21.15.151:0/859099613 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fe3501f3670 con 0x7fe3500ccf00 2024-09-16T15:56:41.872 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.870+0000 7fe357400640 1 -- 172.21.15.151:0/859099613 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7fe3500ae9f0 con 0x7fe33007afb0 2024-09-16T15:56:41.874 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.871+0000 7fe34cc00640 1 -- 172.21.15.151:0/859099613 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fe3500ae9f0 con 0x7fe33007afb0 2024-09-16T15:56:41.874 INFO:teuthology.orchestra.run.smithi151.stderr:instructed osd(s) 0 to deep-scrub 2024-09-16T15:56:41.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.874+0000 7fe357400640 1 -- 172.21.15.151:0/859099613 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe33007afb0 msgr2=0x7fe33007d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:41.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.874+0000 7fe357400640 1 --2- 172.21.15.151:0/859099613 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe33007afb0 0x7fe33007d470 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fe3500cdf40 tx=0x7fe340009040 comp rx=0 tx=0).stop 2024-09-16T15:56:41.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.874+0000 7fe357400640 1 -- 172.21.15.151:0/859099613 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe3500ccf00 msgr2=0x7fe3501f5f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:41.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.874+0000 7fe357400640 1 --2- 172.21.15.151:0/859099613 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe3500ccf00 0x7fe3501f5f10 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7fe33c00d980 tx=0x7fe33c00de40 comp rx=0 tx=0).stop 2024-09-16T15:56:41.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.874+0000 7fe357400640 1 -- 172.21.15.151:0/859099613 shutdown_connections 2024-09-16T15:56:41.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.874+0000 7fe357400640 1 --2- 172.21.15.151:0/859099613 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe33007afb0 0x7fe33007d470 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:41.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.874+0000 7fe357400640 1 --2- 172.21.15.151:0/859099613 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe35015f5a0 0x7fe3500cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:41.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.874+0000 7fe357400640 1 --2- 172.21.15.151:0/859099613 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe3500ccf00 0x7fe3501f5f10 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:41.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.874+0000 7fe357400640 1 --2- 172.21.15.151:0/859099613 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe350073fb0 0x7fe3501f3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:41.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.874+0000 7fe357400640 1 -- 172.21.15.151:0/859099613 >> 172.21.15.151:0/859099613 conn(0x7fe3500b1950 msgr2=0x7fe35009d300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:41.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.874+0000 7fe357400640 1 -- 172.21.15.151:0/859099613 shutdown_connections 2024-09-16T15:56:41.877 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:41.874+0000 7fe357400640 1 -- 172.21.15.151:0/859099613 wait complete. 2024-09-16T15:56:41.893 INFO:tasks.ceph:Scrubbing osd.1 2024-09-16T15:56:41.893 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-16T15:56:42.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.052+0000 7fbc1ec00640 1 -- 172.21.15.151:0/3386377578 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc1815dfc0 msgr2=0x7fbc1806f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:42.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.052+0000 7fbc1ec00640 1 --2- 172.21.15.151:0/3386377578 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc1815dfc0 0x7fbc1806f300 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7fbc08005fd0 tx=0x7fbc0802a9f0 comp rx=0 tx=0).stop 2024-09-16T15:56:42.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.053+0000 7fbc1ec00640 1 -- 172.21.15.151:0/3386377578 shutdown_connections 2024-09-16T15:56:42.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.053+0000 7fbc1ec00640 1 --2- 172.21.15.151:0/3386377578 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbc1806ee00 0x7fbc180ad6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:42.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.053+0000 7fbc1ec00640 1 --2- 172.21.15.151:0/3386377578 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc1815dfc0 0x7fbc1806f300 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:42.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.053+0000 7fbc1ec00640 1 -- 172.21.15.151:0/3386377578 >> 172.21.15.151:0/3386377578 conn(0x7fbc180b18e0 msgr2=0x7fbc180b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:42.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.053+0000 7fbc1ec00640 1 -- 172.21.15.151:0/3386377578 shutdown_connections 2024-09-16T15:56:42.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.053+0000 7fbc1ec00640 1 -- 172.21.15.151:0/3386377578 wait complete. 2024-09-16T15:56:42.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.054+0000 7fbc1ec00640 1 Processor -- start 2024-09-16T15:56:42.056 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.054+0000 7fbc1ec00640 1 -- start start 2024-09-16T15:56:42.058 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.055+0000 7fbc1ec00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbc1806ee00 0x7fbc1815fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:42.058 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.055+0000 7fbc1ec00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc180b4770 0x7fbc18072c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:42.058 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.055+0000 7fbc1ec00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc1815dfc0 0x7fbc18073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:42.058 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.055+0000 7fbc1ec00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbc180adfa0 con 0x7fbc1815dfc0 2024-09-16T15:56:42.058 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.055+0000 7fbc1ec00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbc180ade20 con 0x7fbc180b4770 2024-09-16T15:56:42.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.055+0000 7fbc1ec00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fbc180ae120 con 0x7fbc1806ee00 2024-09-16T15:56:42.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.056+0000 7fbc1cc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc1815dfc0 0x7fbc18073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:42.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.056+0000 7fbc1cc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc1815dfc0 0x7fbc18073180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37374/0 (socket says 172.21.15.151:37374) 2024-09-16T15:56:42.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.056+0000 7fbc1cc00640 1 -- 172.21.15.151:0/3360052626 learned_addr learned my addr 172.21.15.151:0/3360052626 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:56:42.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.056+0000 7fbc17e00640 1 --2- 172.21.15.151:0/3360052626 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbc1806ee00 0x7fbc1815fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:42.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.056+0000 7fbc17400640 1 --2- 172.21.15.151:0/3360052626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc180b4770 0x7fbc18072c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:42.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.056+0000 7fbc1cc00640 1 -- 172.21.15.151:0/3360052626 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbc1806ee00 msgr2=0x7fbc1815fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:42.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.056+0000 7fbc1cc00640 1 --2- 172.21.15.151:0/3360052626 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbc1806ee00 0x7fbc1815fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:42.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.056+0000 7fbc1cc00640 1 -- 172.21.15.151:0/3360052626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc180b4770 msgr2=0x7fbc18072c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:42.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.056+0000 7fbc1cc00640 1 --2- 172.21.15.151:0/3360052626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc180b4770 0x7fbc18072c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:42.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.056+0000 7fbc1cc00640 1 -- 172.21.15.151:0/3360052626 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc181f4bb0 con 0x7fbc1815dfc0 2024-09-16T15:56:42.059 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.056+0000 7fbc1cc00640 1 --2- 172.21.15.151:0/3360052626 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc1815dfc0 0x7fbc18073180 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7fbc1000d780 tx=0x7fbc1000dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:42.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.057+0000 7fbc0fe00640 1 -- 172.21.15.151:0/3360052626 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc10017070 con 0x7fbc1815dfc0 2024-09-16T15:56:42.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.057+0000 7fbc0fe00640 1 -- 172.21.15.151:0/3360052626 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbc1000de90 con 0x7fbc1815dfc0 2024-09-16T15:56:42.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.057+0000 7fbc0fe00640 1 -- 172.21.15.151:0/3360052626 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc10005020 con 0x7fbc1815dfc0 2024-09-16T15:56:42.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.057+0000 7fbc1ec00640 1 -- 172.21.15.151:0/3360052626 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc181f4e50 con 0x7fbc1815dfc0 2024-09-16T15:56:42.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.057+0000 7fbc1ec00640 1 -- 172.21.15.151:0/3360052626 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc181f5410 con 0x7fbc1815dfc0 2024-09-16T15:56:42.060 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.058+0000 7fbc1ec00640 1 -- 172.21.15.151:0/3360052626 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbc180ad6d0 con 0x7fbc1815dfc0 2024-09-16T15:56:42.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.059+0000 7fbc0fe00640 1 -- 172.21.15.151:0/3360052626 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 55) ==== 104857+0+0 (secure 0 0 0) 0x7fbc100051c0 con 0x7fbc1815dfc0 2024-09-16T15:56:42.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.060+0000 7fbc0fe00640 1 --2- 172.21.15.151:0/3360052626 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbbf807afb0 0x7fbbf807d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:42.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.060+0000 7fbc0fe00640 1 -- 172.21.15.151:0/3360052626 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5083+0+0 (secure 0 0 0) 0x7fbc10099ce0 con 0x7fbc1815dfc0 2024-09-16T15:56:42.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.060+0000 7fbc0fe00640 1 --2- 172.21.15.151:0/3360052626 >> [v2:172.21.15.151:6800/3368304087,v1:172.21.15.151:6801/3368304087] conn(0x7fbbf8080f30 0x7fbbf80833d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:42.062 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.060+0000 7fbc0fe00640 1 -- 172.21.15.151:0/3360052626 --> [v2:172.21.15.151:6800/3368304087,v1:172.21.15.151:6801/3368304087] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbbf8083aa0 con 0x7fbbf8080f30 2024-09-16T15:56:42.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.060+0000 7fbc0fe00640 1 -- 172.21.15.151:0/3360052626 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_get_version_reply(handle=1 version=152) ==== 24+0+0 (secure 0 0 0) 0x7fbc100a6050 con 0x7fbc1815dfc0 2024-09-16T15:56:42.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.060+0000 7fbc17e00640 1 --2- 172.21.15.151:0/3360052626 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbbf807afb0 0x7fbbf807d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:42.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.060+0000 7fbc17400640 1 --2- 172.21.15.151:0/3360052626 >> [v2:172.21.15.151:6800/3368304087,v1:172.21.15.151:6801/3368304087] conn(0x7fbbf8080f30 0x7fbbf80833d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:42.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.061+0000 7fbc17400640 1 --2- 172.21.15.151:0/3360052626 >> [v2:172.21.15.151:6800/3368304087,v1:172.21.15.151:6801/3368304087] conn(0x7fbbf8080f30 0x7fbbf80833d0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:42.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.061+0000 7fbc17e00640 1 --2- 172.21.15.151:0/3360052626 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbbf807afb0 0x7fbbf807d470 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fbc08035000 tx=0x7fbc0803b040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:42.063 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.062+0000 7fbc0fe00640 1 -- 172.21.15.151:0/3360052626 <== osd.1 v2:172.21.15.151:6800/3368304087 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbbf8083aa0 con 0x7fbbf8080f30 2024-09-16T15:56:42.076 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.074+0000 7fbc1ec00640 1 -- 172.21.15.151:0/3360052626 --> [v2:172.21.15.151:6800/3368304087,v1:172.21.15.151:6801/3368304087] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fbc180af530 con 0x7fbbf8080f30 2024-09-16T15:56:42.077 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.075+0000 7fbc0fe00640 1 -- 172.21.15.151:0/3360052626 <== osd.1 v2:172.21.15.151:6800/3368304087 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fbc180af530 con 0x7fbbf8080f30 2024-09-16T15:56:42.077 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-16T15:56:42.077 INFO:teuthology.orchestra.run.smithi151.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T15:56:42.077 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-16T15:56:42.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.075+0000 7fbc1ec00640 1 -- 172.21.15.151:0/3360052626 >> [v2:172.21.15.151:6800/3368304087,v1:172.21.15.151:6801/3368304087] conn(0x7fbbf8080f30 msgr2=0x7fbbf80833d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:42.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.075+0000 7fbc1ec00640 1 --2- 172.21.15.151:0/3360052626 >> [v2:172.21.15.151:6800/3368304087,v1:172.21.15.151:6801/3368304087] conn(0x7fbbf8080f30 0x7fbbf80833d0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:42.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.075+0000 7fbc1ec00640 1 -- 172.21.15.151:0/3360052626 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbbf807afb0 msgr2=0x7fbbf807d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:42.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.075+0000 7fbc1ec00640 1 --2- 172.21.15.151:0/3360052626 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fbbf807afb0 0x7fbbf807d470 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fbc08035000 tx=0x7fbc0803b040 comp rx=0 tx=0).stop 2024-09-16T15:56:42.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.075+0000 7fbc1ec00640 1 -- 172.21.15.151:0/3360052626 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc1815dfc0 msgr2=0x7fbc18073180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:42.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.075+0000 7fbc1ec00640 1 --2- 172.21.15.151:0/3360052626 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc1815dfc0 0x7fbc18073180 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7fbc1000d780 tx=0x7fbc1000dc40 comp rx=0 tx=0).stop 2024-09-16T15:56:42.078 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.075+0000 7fbc1cc00640 1 -- 172.21.15.151:0/3360052626 reap_dead start 2024-09-16T15:56:42.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.075+0000 7fbc1ec00640 1 -- 172.21.15.151:0/3360052626 shutdown_connections 2024-09-16T15:56:42.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.075+0000 7fbc1ec00640 1 -- 172.21.15.151:0/3360052626 >> 172.21.15.151:0/3360052626 conn(0x7fbc180b18e0 msgr2=0x7fbc180cdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:42.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.076+0000 7fbc1ec00640 1 -- 172.21.15.151:0/3360052626 shutdown_connections 2024-09-16T15:56:42.079 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.076+0000 7fbc1ec00640 1 -- 172.21.15.151:0/3360052626 wait complete. 2024-09-16T15:56:42.092 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-16T15:56:42.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.251+0000 7fd9f9c00640 1 -- 172.21.15.151:0/3125343254 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd9f40cf300 msgr2=0x7fd9f40729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:42.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.251+0000 7fd9f9c00640 1 --2- 172.21.15.151:0/3125343254 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd9f40cf300 0x7fd9f40729a0 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc009900 tx=0x7fd9dc02a6e0 comp rx=0 tx=0).stop 2024-09-16T15:56:42.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.251+0000 7fd9f9c00640 1 -- 172.21.15.151:0/3125343254 shutdown_connections 2024-09-16T15:56:42.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.251+0000 7fd9f9c00640 1 --2- 172.21.15.151:0/3125343254 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd9f40cf300 0x7fd9f40729a0 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:42.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.251+0000 7fd9f9c00640 1 -- 172.21.15.151:0/3125343254 >> 172.21.15.151:0/3125343254 conn(0x7fd9f40b1950 msgr2=0x7fd9f40b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:42.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.251+0000 7fd9f9c00640 1 -- 172.21.15.151:0/3125343254 shutdown_connections 2024-09-16T15:56:42.254 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.251+0000 7fd9f9c00640 1 -- 172.21.15.151:0/3125343254 wait complete. 2024-09-16T15:56:42.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.253+0000 7fd9f9c00640 1 Processor -- start 2024-09-16T15:56:42.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.253+0000 7fd9f9c00640 1 -- start start 2024-09-16T15:56:42.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.254+0000 7fd9f9c00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd9f40ccfd0 0x7fd9f41423b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:42.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.254+0000 7fd9f9c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd9f40cf300 0x7fd9f41428f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:42.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.254+0000 7fd9f9c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd9f41445d0 0x7fd9f41f6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:42.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.254+0000 7fd9f9c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f40735f0 con 0x7fd9f40cf300 2024-09-16T15:56:42.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.254+0000 7fd9f9c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f40b4f60 con 0x7fd9f41445d0 2024-09-16T15:56:42.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.254+0000 7fd9f9c00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fd9f4073770 con 0x7fd9f40ccfd0 2024-09-16T15:56:42.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.254+0000 7fd9ebe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd9f40cf300 0x7fd9f41428f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:42.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.254+0000 7fd9ebe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd9f40cf300 0x7fd9f41428f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37388/0 (socket says 172.21.15.151:37388) 2024-09-16T15:56:42.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.254+0000 7fd9ebe00640 1 -- 172.21.15.151:0/1718068062 learned_addr learned my addr 172.21.15.151:0/1718068062 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:56:42.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.254+0000 7fd9f3400640 1 --2- 172.21.15.151:0/1718068062 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd9f41445d0 0x7fd9f41f6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:42.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.254+0000 7fd9f2a00640 1 --2- 172.21.15.151:0/1718068062 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd9f40ccfd0 0x7fd9f41423b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:42.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.255+0000 7fd9ebe00640 1 -- 172.21.15.151:0/1718068062 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd9f40ccfd0 msgr2=0x7fd9f41423b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:42.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.255+0000 7fd9ebe00640 1 --2- 172.21.15.151:0/1718068062 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd9f40ccfd0 0x7fd9f41423b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:42.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.255+0000 7fd9ebe00640 1 -- 172.21.15.151:0/1718068062 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd9f41445d0 msgr2=0x7fd9f41f6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:42.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.255+0000 7fd9ebe00640 1 --2- 172.21.15.151:0/1718068062 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd9f41445d0 0x7fd9f41f6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:42.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.255+0000 7fd9ebe00640 1 -- 172.21.15.151:0/1718068062 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9f40c5920 con 0x7fd9f40cf300 2024-09-16T15:56:42.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.255+0000 7fd9f3400640 1 --2- 172.21.15.151:0/1718068062 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd9f41445d0 0x7fd9f41f6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:56:42.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.255+0000 7fd9ebe00640 1 --2- 172.21.15.151:0/1718068062 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd9f40cf300 0x7fd9f41428f0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc000c00 tx=0x7fd9dc0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:42.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.255+0000 7fd9eaa00640 1 -- 172.21.15.151:0/1718068062 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc03f070 con 0x7fd9f40cf300 2024-09-16T15:56:42.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.255+0000 7fd9eaa00640 1 -- 172.21.15.151:0/1718068062 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd9dc004890 con 0x7fd9f40cf300 2024-09-16T15:56:42.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.255+0000 7fd9eaa00640 1 -- 172.21.15.151:0/1718068062 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc035030 con 0x7fd9f40cf300 2024-09-16T15:56:42.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.255+0000 7fd9f9c00640 1 -- 172.21.15.151:0/1718068062 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9f41f2d80 con 0x7fd9f40cf300 2024-09-16T15:56:42.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.256+0000 7fd9f9c00640 1 -- 172.21.15.151:0/1718068062 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9f41f3340 con 0x7fd9f40cf300 2024-09-16T15:56:42.259 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.257+0000 7fd9f9c00640 1 -- 172.21.15.151:0/1718068062 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9f40b3f00 con 0x7fd9f40cf300 2024-09-16T15:56:42.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.258+0000 7fd9eaa00640 1 -- 172.21.15.151:0/1718068062 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 55) ==== 104857+0+0 (secure 0 0 0) 0x7fd9dc042070 con 0x7fd9f40cf300 2024-09-16T15:56:42.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.259+0000 7fd9eaa00640 1 --2- 172.21.15.151:0/1718068062 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd9d007afb0 0x7fd9d007d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:42.260 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.259+0000 7fd9eaa00640 1 -- 172.21.15.151:0/1718068062 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5083+0+0 (secure 0 0 0) 0x7fd9dc0bab30 con 0x7fd9f40cf300 2024-09-16T15:56:42.261 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.259+0000 7fd9f2a00640 1 --2- 172.21.15.151:0/1718068062 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd9d007afb0 0x7fd9d007d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:42.262 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.260+0000 7fd9f2a00640 1 --2- 172.21.15.151:0/1718068062 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd9d007afb0 0x7fd9d007d470 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4000f60 tx=0x7fd9e4009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:42.265 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.263+0000 7fd9eaa00640 1 -- 172.21.15.151:0/1718068062 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fd9f40b3f00 con 0x7fd9f40cf300 2024-09-16T15:56:42.451 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.449+0000 7fd9f9c00640 1 -- 172.21.15.151:0/1718068062 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7fd9f4073c20 con 0x7fd9d007afb0 2024-09-16T15:56:42.452 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.450+0000 7fd9eaa00640 1 -- 172.21.15.151:0/1718068062 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fd9f4073c20 con 0x7fd9d007afb0 2024-09-16T15:56:42.452 INFO:teuthology.orchestra.run.smithi151.stderr:instructed osd(s) 1 to deep-scrub 2024-09-16T15:56:42.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.452+0000 7fd9f9c00640 1 -- 172.21.15.151:0/1718068062 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd9d007afb0 msgr2=0x7fd9d007d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:42.454 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.452+0000 7fd9f9c00640 1 --2- 172.21.15.151:0/1718068062 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd9d007afb0 0x7fd9d007d470 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4000f60 tx=0x7fd9e4009040 comp rx=0 tx=0).stop 2024-09-16T15:56:42.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.452+0000 7fd9f9c00640 1 -- 172.21.15.151:0/1718068062 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd9f40cf300 msgr2=0x7fd9f41428f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:42.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.452+0000 7fd9f9c00640 1 --2- 172.21.15.151:0/1718068062 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd9f40cf300 0x7fd9f41428f0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc000c00 tx=0x7fd9dc0039e0 comp rx=0 tx=0).stop 2024-09-16T15:56:42.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.452+0000 7fd9f9c00640 1 -- 172.21.15.151:0/1718068062 shutdown_connections 2024-09-16T15:56:42.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.452+0000 7fd9f9c00640 1 --2- 172.21.15.151:0/1718068062 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fd9d007afb0 0x7fd9d007d470 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:42.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.452+0000 7fd9f9c00640 1 --2- 172.21.15.151:0/1718068062 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd9f41445d0 0x7fd9f41f6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:42.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.452+0000 7fd9f9c00640 1 --2- 172.21.15.151:0/1718068062 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd9f40cf300 0x7fd9f41428f0 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:42.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.452+0000 7fd9f9c00640 1 --2- 172.21.15.151:0/1718068062 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd9f40ccfd0 0x7fd9f41423b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:42.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.452+0000 7fd9f9c00640 1 -- 172.21.15.151:0/1718068062 >> 172.21.15.151:0/1718068062 conn(0x7fd9f40b1950 msgr2=0x7fd9f409ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:42.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.452+0000 7fd9f9c00640 1 -- 172.21.15.151:0/1718068062 shutdown_connections 2024-09-16T15:56:42.455 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.452+0000 7fd9f9c00640 1 -- 172.21.15.151:0/1718068062 wait complete. 2024-09-16T15:56:42.470 INFO:tasks.ceph:Scrubbing osd.2 2024-09-16T15:56:42.470 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-16T15:56:42.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.630+0000 7f3fad000640 1 -- 172.21.15.151:0/1575023891 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3fa80b4ab0 msgr2=0x7f3fa8072970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:42.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.630+0000 7f3fad000640 1 --2- 172.21.15.151:0/1575023891 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3fa80b4ab0 0x7f3fa8072970 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f3f94005fd0 tx=0x7f3f9402a4b0 comp rx=0 tx=0).stop 2024-09-16T15:56:42.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.630+0000 7f3fad000640 1 -- 172.21.15.151:0/1575023891 shutdown_connections 2024-09-16T15:56:42.633 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.630+0000 7f3fad000640 1 --2- 172.21.15.151:0/1575023891 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3fa80b4ab0 0x7f3fa8072970 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:42.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.630+0000 7f3fad000640 1 -- 172.21.15.151:0/1575023891 >> 172.21.15.151:0/1575023891 conn(0x7f3fa80b18e0 msgr2=0x7f3fa80b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:42.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.631+0000 7f3fad000640 1 -- 172.21.15.151:0/1575023891 shutdown_connections 2024-09-16T15:56:42.636 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.631+0000 7f3fad000640 1 -- 172.21.15.151:0/1575023891 wait complete. 2024-09-16T15:56:42.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.632+0000 7f3fad000640 1 Processor -- start 2024-09-16T15:56:42.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.632+0000 7f3fad000640 1 -- start start 2024-09-16T15:56:42.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.633+0000 7f3fad000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3fa8074760 0x7f3fa809d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:42.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.633+0000 7f3fad000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fa80b4ab0 0x7f3fa81f2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:42.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.633+0000 7f3fad000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3fa815db70 0x7f3fa80cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:42.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.633+0000 7f3fad000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa8073780 con 0x7f3fa815db70 2024-09-16T15:56:42.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.633+0000 7f3fad000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa8073600 con 0x7f3fa80b4ab0 2024-09-16T15:56:42.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.633+0000 7f3fad000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f3fa8073900 con 0x7f3fa8074760 2024-09-16T15:56:42.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.634+0000 7f3fa6800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3fa815db70 0x7f3fa80cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:42.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.634+0000 7f3fa6800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3fa815db70 0x7f3fa80cf9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:37410/0 (socket says 172.21.15.151:37410) 2024-09-16T15:56:42.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.634+0000 7f3fa6800640 1 -- 172.21.15.151:0/871440115 learned_addr learned my addr 172.21.15.151:0/871440115 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:56:42.638 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.634+0000 7f3fa5e00640 1 --2- 172.21.15.151:0/871440115 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3fa8074760 0x7f3fa809d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:42.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.634+0000 7f3fa5400640 1 --2- 172.21.15.151:0/871440115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fa80b4ab0 0x7f3fa81f2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:42.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.634+0000 7f3fa6800640 1 -- 172.21.15.151:0/871440115 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3fa8074760 msgr2=0x7f3fa809d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:42.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.634+0000 7f3fa6800640 1 --2- 172.21.15.151:0/871440115 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3fa8074760 0x7f3fa809d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:42.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.634+0000 7f3fa6800640 1 -- 172.21.15.151:0/871440115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fa80b4ab0 msgr2=0x7f3fa81f2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:42.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.634+0000 7f3fa6800640 1 --2- 172.21.15.151:0/871440115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fa80b4ab0 0x7f3fa81f2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:42.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.634+0000 7f3fa6800640 1 -- 172.21.15.151:0/871440115 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fa81504f0 con 0x7f3fa815db70 2024-09-16T15:56:42.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.634+0000 7f3fa6800640 1 --2- 172.21.15.151:0/871440115 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3fa815db70 0x7f3fa80cf9d0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f3fa000c780 tx=0x7f3fa000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:42.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.635+0000 7f3f9e000640 1 -- 172.21.15.151:0/871440115 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa0004b60 con 0x7f3fa815db70 2024-09-16T15:56:42.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.635+0000 7f3f9e000640 1 -- 172.21.15.151:0/871440115 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3fa0004df0 con 0x7f3fa815db70 2024-09-16T15:56:42.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.635+0000 7f3f9e000640 1 -- 172.21.15.151:0/871440115 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa000aa70 con 0x7f3fa815db70 2024-09-16T15:56:42.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.635+0000 7f3fad000640 1 -- 172.21.15.151:0/871440115 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fa809dd10 con 0x7f3fa815db70 2024-09-16T15:56:42.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.635+0000 7f3fad000640 1 -- 172.21.15.151:0/871440115 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fa81e5750 con 0x7f3fa815db70 2024-09-16T15:56:42.639 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.636+0000 7f3fad000640 1 -- 172.21.15.151:0/871440115 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3fa80b43a0 con 0x7f3fa815db70 2024-09-16T15:56:42.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.637+0000 7f3f9e000640 1 -- 172.21.15.151:0/871440115 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 55) ==== 104857+0+0 (secure 0 0 0) 0x7f3fa000ac10 con 0x7f3fa815db70 2024-09-16T15:56:42.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.638+0000 7f3f9e000640 1 --2- 172.21.15.151:0/871440115 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3f8807afb0 0x7f3f8807d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:42.640 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.638+0000 7f3f9e000640 1 -- 172.21.15.151:0/871440115 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5083+0+0 (secure 0 0 0) 0x7f3fa0099e00 con 0x7f3fa815db70 2024-09-16T15:56:42.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.638+0000 7f3f9e000640 1 --2- 172.21.15.151:0/871440115 >> [v2:172.21.15.151:6823/3079373271,v1:172.21.15.151:6825/3079373271] conn(0x7f3f88080f30 0x7f3f880833d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:42.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.638+0000 7f3f9e000640 1 -- 172.21.15.151:0/871440115 --> [v2:172.21.15.151:6823/3079373271,v1:172.21.15.151:6825/3079373271] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3f88083aa0 con 0x7f3f88080f30 2024-09-16T15:56:42.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.638+0000 7f3f9e000640 1 -- 172.21.15.151:0/871440115 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_get_version_reply(handle=1 version=152) ==== 24+0+0 (secure 0 0 0) 0x7f3fa00a6050 con 0x7f3fa815db70 2024-09-16T15:56:42.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.639+0000 7f3fa5e00640 1 --2- 172.21.15.151:0/871440115 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3f8807afb0 0x7f3f8807d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:42.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.639+0000 7f3fa5400640 1 --2- 172.21.15.151:0/871440115 >> [v2:172.21.15.151:6823/3079373271,v1:172.21.15.151:6825/3079373271] conn(0x7f3f88080f30 0x7f3f880833d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:42.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.639+0000 7f3fa5e00640 1 --2- 172.21.15.151:0/871440115 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3f8807afb0 0x7f3f8807d470 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f3f98001720 tx=0x7f3f98009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:42.641 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.639+0000 7f3fa5400640 1 --2- 172.21.15.151:0/871440115 >> [v2:172.21.15.151:6823/3079373271,v1:172.21.15.151:6825/3079373271] conn(0x7f3f88080f30 0x7f3f880833d0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:42.642 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.640+0000 7f3f9e000640 1 -- 172.21.15.151:0/871440115 <== osd.2 v2:172.21.15.151:6823/3079373271 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3f88083aa0 con 0x7f3f88080f30 2024-09-16T15:56:42.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.653+0000 7f3fad000640 1 -- 172.21.15.151:0/871440115 --> [v2:172.21.15.151:6823/3079373271,v1:172.21.15.151:6825/3079373271] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f3fa80af2a0 con 0x7f3f88080f30 2024-09-16T15:56:42.655 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.653+0000 7f3f9e000640 1 -- 172.21.15.151:0/871440115 <== osd.2 v2:172.21.15.151:6823/3079373271 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f3fa80af2a0 con 0x7f3f88080f30 2024-09-16T15:56:42.655 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-16T15:56:42.655 INFO:teuthology.orchestra.run.smithi151.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T15:56:42.655 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-16T15:56:42.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.654+0000 7f3fad000640 1 -- 172.21.15.151:0/871440115 >> [v2:172.21.15.151:6823/3079373271,v1:172.21.15.151:6825/3079373271] conn(0x7f3f88080f30 msgr2=0x7f3f880833d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:42.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.654+0000 7f3fad000640 1 --2- 172.21.15.151:0/871440115 >> [v2:172.21.15.151:6823/3079373271,v1:172.21.15.151:6825/3079373271] conn(0x7f3f88080f30 0x7f3f880833d0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:42.656 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.654+0000 7f3fad000640 1 -- 172.21.15.151:0/871440115 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3f8807afb0 msgr2=0x7f3f8807d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:42.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.654+0000 7f3fad000640 1 --2- 172.21.15.151:0/871440115 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3f8807afb0 0x7f3f8807d470 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f3f98001720 tx=0x7f3f98009040 comp rx=0 tx=0).stop 2024-09-16T15:56:42.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.654+0000 7f3fad000640 1 -- 172.21.15.151:0/871440115 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3fa815db70 msgr2=0x7f3fa80cf9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:42.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.654+0000 7f3fad000640 1 --2- 172.21.15.151:0/871440115 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3fa815db70 0x7f3fa80cf9d0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f3fa000c780 tx=0x7f3fa000cc40 comp rx=0 tx=0).stop 2024-09-16T15:56:42.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.654+0000 7f3fa6800640 1 -- 172.21.15.151:0/871440115 reap_dead start 2024-09-16T15:56:42.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.654+0000 7f3fad000640 1 -- 172.21.15.151:0/871440115 shutdown_connections 2024-09-16T15:56:42.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.654+0000 7f3fad000640 1 -- 172.21.15.151:0/871440115 >> 172.21.15.151:0/871440115 conn(0x7f3fa80b18e0 msgr2=0x7f3fa80ad520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:42.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.654+0000 7f3fad000640 1 -- 172.21.15.151:0/871440115 shutdown_connections 2024-09-16T15:56:42.657 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.654+0000 7f3fad000640 1 -- 172.21.15.151:0/871440115 wait complete. 2024-09-16T15:56:42.670 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-16T15:56:42.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.829+0000 7f9915600640 1 -- 172.21.15.151:0/2352472470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99100af3d0 msgr2=0x7f991006f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:42.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.829+0000 7f9915600640 1 --2- 172.21.15.151:0/2352472470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99100af3d0 0x7f991006f660 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f98fc005fd0 tx=0x7f98fc02a540 comp rx=0 tx=0).stop 2024-09-16T15:56:42.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.829+0000 7f9915600640 1 -- 172.21.15.151:0/2352472470 shutdown_connections 2024-09-16T15:56:42.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.829+0000 7f9915600640 1 --2- 172.21.15.151:0/2352472470 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9910072df0 0x7f9910073280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:42.831 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.829+0000 7f9915600640 1 --2- 172.21.15.151:0/2352472470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99100af3d0 0x7f991006f660 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:42.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.829+0000 7f9915600640 1 -- 172.21.15.151:0/2352472470 >> 172.21.15.151:0/2352472470 conn(0x7f99100b1950 msgr2=0x7f99100b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:42.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.829+0000 7f9915600640 1 -- 172.21.15.151:0/2352472470 shutdown_connections 2024-09-16T15:56:42.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.829+0000 7f9915600640 1 -- 172.21.15.151:0/2352472470 wait complete. 2024-09-16T15:56:42.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.830+0000 7f9915600640 1 Processor -- start 2024-09-16T15:56:42.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.831+0000 7f9915600640 1 -- start start 2024-09-16T15:56:42.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.831+0000 7f9915600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9910072df0 0x7f9910153e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:42.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.831+0000 7f9915600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99100af3d0 0x7f99100cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:42.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.831+0000 7f9915600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9910161550 0x7f99100d0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:42.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.831+0000 7f9915600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f99100737c0 con 0x7f99100af3d0 2024-09-16T15:56:42.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.831+0000 7f9915600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f991006fdb0 con 0x7f9910161550 2024-09-16T15:56:42.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.831+0000 7f9915600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f9910073940 con 0x7f9910072df0 2024-09-16T15:56:42.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.832+0000 7f990e000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99100af3d0 0x7f99100cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:42.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.832+0000 7f990e000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99100af3d0 0x7f99100cfb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:35890/0 (socket says 172.21.15.151:35890) 2024-09-16T15:56:42.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.832+0000 7f990e000640 1 -- 172.21.15.151:0/200236572 learned_addr learned my addr 172.21.15.151:0/200236572 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:56:42.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.832+0000 7f990f400640 1 --2- 172.21.15.151:0/200236572 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9910161550 0x7f99100d0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:42.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.832+0000 7f990ea00640 1 --2- 172.21.15.151:0/200236572 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9910072df0 0x7f9910153e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:42.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.832+0000 7f990e000640 1 -- 172.21.15.151:0/200236572 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9910072df0 msgr2=0x7f9910153e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:42.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.832+0000 7f990e000640 1 --2- 172.21.15.151:0/200236572 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9910072df0 0x7f9910153e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:42.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.832+0000 7f990e000640 1 -- 172.21.15.151:0/200236572 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9910161550 msgr2=0x7f99100d0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:42.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.832+0000 7f990e000640 1 --2- 172.21.15.151:0/200236572 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9910161550 0x7f99100d0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:42.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.832+0000 7f990e000640 1 -- 172.21.15.151:0/200236572 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99100d0700 con 0x7f99100af3d0 2024-09-16T15:56:42.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.832+0000 7f990f400640 1 --2- 172.21.15.151:0/200236572 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9910161550 0x7f99100d0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:56:42.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.833+0000 7f990e000640 1 --2- 172.21.15.151:0/200236572 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99100af3d0 0x7f99100cfb40 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f98fc009850 tx=0x7f98fc0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:42.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.833+0000 7f9906a00640 1 -- 172.21.15.151:0/200236572 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98fc03f070 con 0x7f99100af3d0 2024-09-16T15:56:42.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.833+0000 7f9906a00640 1 -- 172.21.15.151:0/200236572 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f98fc0047a0 con 0x7f99100af3d0 2024-09-16T15:56:42.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.833+0000 7f9915600640 1 -- 172.21.15.151:0/200236572 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99101f8090 con 0x7f99100af3d0 2024-09-16T15:56:42.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.833+0000 7f9906a00640 1 -- 172.21.15.151:0/200236572 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98fc035030 con 0x7f99100af3d0 2024-09-16T15:56:42.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.833+0000 7f9915600640 1 -- 172.21.15.151:0/200236572 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99101f8650 con 0x7f99100af3d0 2024-09-16T15:56:42.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.834+0000 7f9915600640 1 -- 172.21.15.151:0/200236572 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99101f7e80 con 0x7f99100af3d0 2024-09-16T15:56:42.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.835+0000 7f9906a00640 1 -- 172.21.15.151:0/200236572 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 55) ==== 104857+0+0 (secure 0 0 0) 0x7f98fc042070 con 0x7f99100af3d0 2024-09-16T15:56:42.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.836+0000 7f9906a00640 1 --2- 172.21.15.151:0/200236572 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f98f007afe0 0x7f98f007d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:42.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.836+0000 7f9906a00640 1 -- 172.21.15.151:0/200236572 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5083+0+0 (secure 0 0 0) 0x7f98fc0ba990 con 0x7f99100af3d0 2024-09-16T15:56:42.838 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.837+0000 7f990ea00640 1 --2- 172.21.15.151:0/200236572 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f98f007afe0 0x7f98f007d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:42.839 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.837+0000 7f990ea00640 1 --2- 172.21.15.151:0/200236572 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f98f007afe0 0x7f98f007d4a0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f9900001e20 tx=0x7f9900009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:42.842 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:42.840+0000 7f9906a00640 1 -- 172.21.15.151:0/200236572 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f98fc084410 con 0x7f99100af3d0 2024-09-16T15:56:43.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.029+0000 7f9915600640 1 -- 172.21.15.151:0/200236572 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f9910073ea0 con 0x7f98f007afe0 2024-09-16T15:56:43.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.031+0000 7f9906a00640 1 -- 172.21.15.151:0/200236572 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f9910073ea0 con 0x7f98f007afe0 2024-09-16T15:56:43.032 INFO:teuthology.orchestra.run.smithi151.stderr:instructed osd(s) 2 to deep-scrub 2024-09-16T15:56:43.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.033+0000 7f9915600640 1 -- 172.21.15.151:0/200236572 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f98f007afe0 msgr2=0x7f98f007d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:43.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.033+0000 7f9915600640 1 --2- 172.21.15.151:0/200236572 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f98f007afe0 0x7f98f007d4a0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f9900001e20 tx=0x7f9900009040 comp rx=0 tx=0).stop 2024-09-16T15:56:43.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.033+0000 7f9915600640 1 -- 172.21.15.151:0/200236572 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99100af3d0 msgr2=0x7f99100cfb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:43.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.033+0000 7f9915600640 1 --2- 172.21.15.151:0/200236572 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99100af3d0 0x7f99100cfb40 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f98fc009850 tx=0x7f98fc0038a0 comp rx=0 tx=0).stop 2024-09-16T15:56:43.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.033+0000 7f9915600640 1 -- 172.21.15.151:0/200236572 shutdown_connections 2024-09-16T15:56:43.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.033+0000 7f9915600640 1 --2- 172.21.15.151:0/200236572 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f98f007afe0 0x7f98f007d4a0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:43.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.033+0000 7f9915600640 1 --2- 172.21.15.151:0/200236572 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9910161550 0x7f99100d0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:43.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.033+0000 7f9915600640 1 --2- 172.21.15.151:0/200236572 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99100af3d0 0x7f99100cfb40 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:43.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.033+0000 7f9915600640 1 --2- 172.21.15.151:0/200236572 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9910072df0 0x7f9910153e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:43.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.033+0000 7f9915600640 1 -- 172.21.15.151:0/200236572 >> 172.21.15.151:0/200236572 conn(0x7f99100b1950 msgr2=0x7f991006ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:43.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.033+0000 7f9915600640 1 -- 172.21.15.151:0/200236572 shutdown_connections 2024-09-16T15:56:43.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.033+0000 7f9915600640 1 -- 172.21.15.151:0/200236572 wait complete. 2024-09-16T15:56:43.051 INFO:tasks.ceph:Scrubbing osd.3 2024-09-16T15:56:43.051 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-16T15:56:43.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.211+0000 7fa747800640 1 -- 172.21.15.151:0/2725868321 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa7400b4770 msgr2=0x7fa74006e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:43.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.211+0000 7fa747800640 1 --2- 172.21.15.151:0/2725868321 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa7400b4770 0x7fa74006e6f0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7fa728005fd0 tx=0x7fa72802a530 comp rx=0 tx=0).stop 2024-09-16T15:56:43.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.211+0000 7fa747800640 1 -- 172.21.15.151:0/2725868321 shutdown_connections 2024-09-16T15:56:43.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.211+0000 7fa747800640 1 --2- 172.21.15.151:0/2725868321 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa7400b4770 0x7fa74006e6f0 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:43.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.211+0000 7fa747800640 1 --2- 172.21.15.151:0/2725868321 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa74015dfc0 0x7fa74006f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:43.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.211+0000 7fa747800640 1 -- 172.21.15.151:0/2725868321 >> 172.21.15.151:0/2725868321 conn(0x7fa7400b18e0 msgr2=0x7fa7400b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:43.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.212+0000 7fa747800640 1 -- 172.21.15.151:0/2725868321 shutdown_connections 2024-09-16T15:56:43.214 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.212+0000 7fa747800640 1 -- 172.21.15.151:0/2725868321 wait complete. 2024-09-16T15:56:43.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.213+0000 7fa747800640 1 Processor -- start 2024-09-16T15:56:43.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.213+0000 7fa747800640 1 -- start start 2024-09-16T15:56:43.217 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.214+0000 7fa747800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa74006ec30 0x7fa740146f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:43.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.214+0000 7fa747800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7400b4770 0x7fa740143c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:43.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.214+0000 7fa747800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa74015dfc0 0x7fa7400afaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:43.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.214+0000 7fa747800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa7401622c0 con 0x7fa74006ec30 2024-09-16T15:56:43.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.214+0000 7fa747800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa7400ae8f0 con 0x7fa7400b4770 2024-09-16T15:56:43.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.214+0000 7fa747800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fa740162440 con 0x7fa74015dfc0 2024-09-16T15:56:43.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.215+0000 7fa744e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa74006ec30 0x7fa740146f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:43.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.215+0000 7fa744e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa74006ec30 0x7fa740146f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:35908/0 (socket says 172.21.15.151:35908) 2024-09-16T15:56:43.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.215+0000 7fa744e00640 1 -- 172.21.15.151:0/146197447 learned_addr learned my addr 172.21.15.151:0/146197447 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:56:43.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.215+0000 7fa745800640 1 --2- 172.21.15.151:0/146197447 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa74015dfc0 0x7fa7400afaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:43.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.215+0000 7fa737e00640 1 --2- 172.21.15.151:0/146197447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7400b4770 0x7fa740143c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:43.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.215+0000 7fa744e00640 1 -- 172.21.15.151:0/146197447 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa74015dfc0 msgr2=0x7fa7400afaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:43.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.215+0000 7fa744e00640 1 --2- 172.21.15.151:0/146197447 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fa74015dfc0 0x7fa7400afaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:43.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.215+0000 7fa744e00640 1 -- 172.21.15.151:0/146197447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7400b4770 msgr2=0x7fa740143c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:43.218 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.215+0000 7fa744e00640 1 --2- 172.21.15.151:0/146197447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa7400b4770 0x7fa740143c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:43.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.215+0000 7fa744e00640 1 -- 172.21.15.151:0/146197447 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7401474c0 con 0x7fa74006ec30 2024-09-16T15:56:43.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.215+0000 7fa744e00640 1 --2- 172.21.15.151:0/146197447 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa74006ec30 0x7fa740146f80 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7fa73000c9a0 tx=0x7fa73000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:43.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.216+0000 7fa73d600640 1 -- 172.21.15.151:0/146197447 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa730007bb0 con 0x7fa74006ec30 2024-09-16T15:56:43.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.216+0000 7fa73d600640 1 -- 172.21.15.151:0/146197447 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa730007e40 con 0x7fa74006ec30 2024-09-16T15:56:43.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.216+0000 7fa73d600640 1 -- 172.21.15.151:0/146197447 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa730005280 con 0x7fa74006ec30 2024-09-16T15:56:43.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.216+0000 7fa747800640 1 -- 172.21.15.151:0/146197447 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa74009bdd0 con 0x7fa74006ec30 2024-09-16T15:56:43.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.216+0000 7fa747800640 1 -- 172.21.15.151:0/146197447 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa74009c390 con 0x7fa74006ec30 2024-09-16T15:56:43.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.217+0000 7fa747800640 1 -- 172.21.15.151:0/146197447 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa740162960 con 0x7fa74006ec30 2024-09-16T15:56:43.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.218+0000 7fa73d600640 1 -- 172.21.15.151:0/146197447 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 55) ==== 104857+0+0 (secure 0 0 0) 0x7fa730005420 con 0x7fa74006ec30 2024-09-16T15:56:43.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.219+0000 7fa73d600640 1 --2- 172.21.15.151:0/146197447 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa71c07afb0 0x7fa71c07d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:43.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.219+0000 7fa73d600640 1 -- 172.21.15.151:0/146197447 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5083+0+0 (secure 0 0 0) 0x7fa73009a6c0 con 0x7fa74006ec30 2024-09-16T15:56:43.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.219+0000 7fa73d600640 1 --2- 172.21.15.151:0/146197447 >> [v2:172.21.15.151:6816/1142509041,v1:172.21.15.151:6817/1142509041] conn(0x7fa71c080f30 0x7fa71c0833d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:43.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.219+0000 7fa73d600640 1 -- 172.21.15.151:0/146197447 --> [v2:172.21.15.151:6816/1142509041,v1:172.21.15.151:6817/1142509041] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa73000a0a0 con 0x7fa71c080f30 2024-09-16T15:56:43.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.219+0000 7fa73d600640 1 -- 172.21.15.151:0/146197447 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_get_version_reply(handle=1 version=152) ==== 24+0+0 (secure 0 0 0) 0x7fa73009aa70 con 0x7fa74006ec30 2024-09-16T15:56:43.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.220+0000 7fa737e00640 1 --2- 172.21.15.151:0/146197447 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa71c07afb0 0x7fa71c07d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:43.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.220+0000 7fa745800640 1 --2- 172.21.15.151:0/146197447 >> [v2:172.21.15.151:6816/1142509041,v1:172.21.15.151:6817/1142509041] conn(0x7fa71c080f30 0x7fa71c0833d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:43.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.220+0000 7fa745800640 1 --2- 172.21.15.151:0/146197447 >> [v2:172.21.15.151:6816/1142509041,v1:172.21.15.151:6817/1142509041] conn(0x7fa71c080f30 0x7fa71c0833d0 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:43.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.220+0000 7fa737e00640 1 --2- 172.21.15.151:0/146197447 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa71c07afb0 0x7fa71c07d470 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fa728033000 tx=0x7fa728004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:43.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.221+0000 7fa73d600640 1 -- 172.21.15.151:0/146197447 <== osd.3 v2:172.21.15.151:6816/1142509041 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa73000a0a0 con 0x7fa71c080f30 2024-09-16T15:56:43.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.234+0000 7fa747800640 1 -- 172.21.15.151:0/146197447 --> [v2:172.21.15.151:6816/1142509041,v1:172.21.15.151:6817/1142509041] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fa7400af530 con 0x7fa71c080f30 2024-09-16T15:56:43.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.234+0000 7fa73d600640 1 -- 172.21.15.151:0/146197447 <== osd.3 v2:172.21.15.151:6816/1142509041 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fa7400af530 con 0x7fa71c080f30 2024-09-16T15:56:43.236 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-16T15:56:43.236 INFO:teuthology.orchestra.run.smithi151.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T15:56:43.236 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-16T15:56:43.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.235+0000 7fa747800640 1 -- 172.21.15.151:0/146197447 >> [v2:172.21.15.151:6816/1142509041,v1:172.21.15.151:6817/1142509041] conn(0x7fa71c080f30 msgr2=0x7fa71c0833d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:43.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.235+0000 7fa747800640 1 --2- 172.21.15.151:0/146197447 >> [v2:172.21.15.151:6816/1142509041,v1:172.21.15.151:6817/1142509041] conn(0x7fa71c080f30 0x7fa71c0833d0 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:43.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.235+0000 7fa747800640 1 -- 172.21.15.151:0/146197447 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa71c07afb0 msgr2=0x7fa71c07d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:43.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.235+0000 7fa747800640 1 --2- 172.21.15.151:0/146197447 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fa71c07afb0 0x7fa71c07d470 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fa728033000 tx=0x7fa728004730 comp rx=0 tx=0).stop 2024-09-16T15:56:43.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.235+0000 7fa747800640 1 -- 172.21.15.151:0/146197447 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa74006ec30 msgr2=0x7fa740146f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:43.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.235+0000 7fa747800640 1 --2- 172.21.15.151:0/146197447 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa74006ec30 0x7fa740146f80 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7fa73000c9a0 tx=0x7fa73000ce60 comp rx=0 tx=0).stop 2024-09-16T15:56:43.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.235+0000 7fa745800640 1 -- 172.21.15.151:0/146197447 reap_dead start 2024-09-16T15:56:43.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.235+0000 7fa747800640 1 -- 172.21.15.151:0/146197447 shutdown_connections 2024-09-16T15:56:43.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.235+0000 7fa747800640 1 -- 172.21.15.151:0/146197447 >> 172.21.15.151:0/146197447 conn(0x7fa7400b18e0 msgr2=0x7fa7400b39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:43.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.235+0000 7fa747800640 1 -- 172.21.15.151:0/146197447 shutdown_connections 2024-09-16T15:56:43.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.235+0000 7fa747800640 1 -- 172.21.15.151:0/146197447 wait complete. 2024-09-16T15:56:43.251 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-16T15:56:43.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.408+0000 7fb115a00640 1 -- 172.21.15.151:0/2818764978 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb0f4004d40 msgr2=0x7fb0f40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:43.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.408+0000 7fb115a00640 1 --2- 172.21.15.151:0/2818764978 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb0f4004d40 0x7fb0f40051c0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7fb10800c610 tx=0x7fb0f4005830 comp rx=0 tx=0).stop 2024-09-16T15:56:43.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.408+0000 7fb10f400640 1 -- 172.21.15.151:0/2818764978 reap_dead start 2024-09-16T15:56:43.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.408+0000 7fb115a00640 1 -- 172.21.15.151:0/2818764978 shutdown_connections 2024-09-16T15:56:43.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.408+0000 7fb115a00640 1 -- 172.21.15.151:0/2818764978 >> 172.21.15.151:0/2818764978 conn(0x7fb1100b1950 msgr2=0x7fb1100b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:43.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.408+0000 7fb115a00640 1 -- 172.21.15.151:0/2818764978 shutdown_connections 2024-09-16T15:56:43.411 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.408+0000 7fb115a00640 1 -- 172.21.15.151:0/2818764978 wait complete. 2024-09-16T15:56:43.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.410+0000 7fb115a00640 1 Processor -- start 2024-09-16T15:56:43.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.410+0000 7fb115a00640 1 -- start start 2024-09-16T15:56:43.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.411+0000 7fb115a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb0f4004d40 0x7fb1100ccfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:43.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.411+0000 7fb115a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb11015bf90 0x7fb11015b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:43.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.411+0000 7fb115a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb1100cbe70 0x7fb1100c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:43.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.411+0000 7fb115a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb11006ef50 con 0x7fb0f4004d40 2024-09-16T15:56:43.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.411+0000 7fb115a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb11006fdb0 con 0x7fb11015bf90 2024-09-16T15:56:43.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.411+0000 7fb115a00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fb11006f0d0 con 0x7fb1100cbe70 2024-09-16T15:56:43.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.411+0000 7fb10ea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb0f4004d40 0x7fb1100ccfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:43.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.411+0000 7fb10ea00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb0f4004d40 0x7fb1100ccfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:35924/0 (socket says 172.21.15.151:35924) 2024-09-16T15:56:43.414 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.411+0000 7fb10ea00640 1 -- 172.21.15.151:0/128174537 learned_addr learned my addr 172.21.15.151:0/128174537 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:56:43.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.411+0000 7fb10f400640 1 --2- 172.21.15.151:0/128174537 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb1100cbe70 0x7fb1100c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:43.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.411+0000 7fb10e000640 1 --2- 172.21.15.151:0/128174537 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb11015bf90 0x7fb11015b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:43.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.412+0000 7fb10ea00640 1 -- 172.21.15.151:0/128174537 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb1100cbe70 msgr2=0x7fb1100c8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:43.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.412+0000 7fb10ea00640 1 --2- 172.21.15.151:0/128174537 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb1100cbe70 0x7fb1100c8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:43.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.412+0000 7fb10ea00640 1 -- 172.21.15.151:0/128174537 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb11015bf90 msgr2=0x7fb11015b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:43.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.412+0000 7fb10ea00640 1 --2- 172.21.15.151:0/128174537 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb11015bf90 0x7fb11015b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:43.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.412+0000 7fb10ea00640 1 -- 172.21.15.151:0/128174537 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1100cb110 con 0x7fb0f4004d40 2024-09-16T15:56:43.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.412+0000 7fb10ea00640 1 --2- 172.21.15.151:0/128174537 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb0f4004d40 0x7fb1100ccfd0 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7fb100011a50 tx=0x7fb100011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:43.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.412+0000 7fb106a00640 1 -- 172.21.15.151:0/128174537 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb10001f070 con 0x7fb0f4004d40 2024-09-16T15:56:43.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.412+0000 7fb106a00640 1 -- 172.21.15.151:0/128174537 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb10000fe80 con 0x7fb0f4004d40 2024-09-16T15:56:43.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.412+0000 7fb115a00640 1 -- 172.21.15.151:0/128174537 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1101f5150 con 0x7fb0f4004d40 2024-09-16T15:56:43.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.412+0000 7fb106a00640 1 -- 172.21.15.151:0/128174537 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb100012690 con 0x7fb0f4004d40 2024-09-16T15:56:43.415 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.412+0000 7fb115a00640 1 -- 172.21.15.151:0/128174537 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1100cb670 con 0x7fb0f4004d40 2024-09-16T15:56:43.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.413+0000 7fb115a00640 1 -- 172.21.15.151:0/128174537 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1100732a0 con 0x7fb0f4004d40 2024-09-16T15:56:43.416 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.414+0000 7fb106a00640 1 -- 172.21.15.151:0/128174537 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 55) ==== 104857+0+0 (secure 0 0 0) 0x7fb100022070 con 0x7fb0f4004d40 2024-09-16T15:56:43.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.415+0000 7fb106a00640 1 --2- 172.21.15.151:0/128174537 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb0f007afb0 0x7fb0f007d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:43.417 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.415+0000 7fb106a00640 1 -- 172.21.15.151:0/128174537 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5083+0+0 (secure 0 0 0) 0x7fb10009a780 con 0x7fb0f4004d40 2024-09-16T15:56:43.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.416+0000 7fb10e000640 1 --2- 172.21.15.151:0/128174537 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb0f007afb0 0x7fb0f007d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:43.418 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.417+0000 7fb10e000640 1 --2- 172.21.15.151:0/128174537 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb0f007afb0 0x7fb0f007d470 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fb11015d7f0 tx=0x7fb0fc006b10 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:43.421 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.419+0000 7fb106a00640 1 -- 172.21.15.151:0/128174537 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fb1100cb110 con 0x7fb0f4004d40 2024-09-16T15:56:43.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.603+0000 7fb115a00640 1 -- 172.21.15.151:0/128174537 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7fb1101f4e40 con 0x7fb0f007afb0 2024-09-16T15:56:43.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.604+0000 7fb106a00640 1 -- 172.21.15.151:0/128174537 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fb1101f4e40 con 0x7fb0f007afb0 2024-09-16T15:56:43.606 INFO:teuthology.orchestra.run.smithi151.stderr:instructed osd(s) 3 to deep-scrub 2024-09-16T15:56:43.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.606+0000 7fb115a00640 1 -- 172.21.15.151:0/128174537 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb0f007afb0 msgr2=0x7fb0f007d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:43.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.606+0000 7fb115a00640 1 --2- 172.21.15.151:0/128174537 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb0f007afb0 0x7fb0f007d470 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fb11015d7f0 tx=0x7fb0fc006b10 comp rx=0 tx=0).stop 2024-09-16T15:56:43.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.606+0000 7fb115a00640 1 -- 172.21.15.151:0/128174537 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb0f4004d40 msgr2=0x7fb1100ccfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:43.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.606+0000 7fb115a00640 1 --2- 172.21.15.151:0/128174537 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb0f4004d40 0x7fb1100ccfd0 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7fb100011a50 tx=0x7fb100011f10 comp rx=0 tx=0).stop 2024-09-16T15:56:43.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.606+0000 7fb115a00640 1 -- 172.21.15.151:0/128174537 shutdown_connections 2024-09-16T15:56:43.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.606+0000 7fb115a00640 1 --2- 172.21.15.151:0/128174537 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fb0f007afb0 0x7fb0f007d470 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:43.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.606+0000 7fb115a00640 1 --2- 172.21.15.151:0/128174537 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb11015bf90 0x7fb11015b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:43.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.606+0000 7fb115a00640 1 --2- 172.21.15.151:0/128174537 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fb1100cbe70 0x7fb1100c8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:43.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.606+0000 7fb115a00640 1 --2- 172.21.15.151:0/128174537 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb0f4004d40 0x7fb1100ccfd0 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:43.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.606+0000 7fb115a00640 1 -- 172.21.15.151:0/128174537 >> 172.21.15.151:0/128174537 conn(0x7fb1100b1950 msgr2=0x7fb1100ae1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:43.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.606+0000 7fb115a00640 1 -- 172.21.15.151:0/128174537 shutdown_connections 2024-09-16T15:56:43.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.606+0000 7fb115a00640 1 -- 172.21.15.151:0/128174537 wait complete. 2024-09-16T15:56:43.624 INFO:tasks.ceph:Scrubbing osd.4 2024-09-16T15:56:43.624 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-16T15:56:43.785 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.783+0000 7f9f46e00640 1 -- 172.21.15.151:0/1801492163 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f4015cff0 msgr2=0x7f9f4006e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:43.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.783+0000 7f9f46e00640 1 --2- 172.21.15.151:0/1801492163 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f4015cff0 0x7f9f4006e6f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c009860 tx=0x7f9f2c02a540 comp rx=0 tx=0).stop 2024-09-16T15:56:43.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.783+0000 7f9f46e00640 1 -- 172.21.15.151:0/1801492163 shutdown_connections 2024-09-16T15:56:43.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.783+0000 7f9f46e00640 1 --2- 172.21.15.151:0/1801492163 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f4015cff0 0x7f9f4006e6f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:43.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.783+0000 7f9f46e00640 1 --2- 172.21.15.151:0/1801492163 >> v2:172.21.15.167:3300/0 conn(0x7f9f40159aa0 0x7f9f400741a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:43.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.783+0000 7f9f46e00640 1 -- 172.21.15.151:0/1801492163 >> 172.21.15.151:0/1801492163 conn(0x7f9f400b18e0 msgr2=0x7f9f400b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:43.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.783+0000 7f9f46e00640 1 -- 172.21.15.151:0/1801492163 shutdown_connections 2024-09-16T15:56:43.786 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.783+0000 7f9f46e00640 1 -- 172.21.15.151:0/1801492163 wait complete. 2024-09-16T15:56:43.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.785+0000 7f9f46e00640 1 Processor -- start 2024-09-16T15:56:43.787 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.785+0000 7f9f46e00640 1 -- start start 2024-09-16T15:56:43.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.786+0000 7f9f46e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9f4006f220 0x7f9f40152480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:43.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.786+0000 7f9f46e00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f40159aa0 0x7f9f400b4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:43.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.786+0000 7f9f46e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f4015cff0 0x7f9f40154c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:43.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.786+0000 7f9f46e00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4006fa60 con 0x7f9f4006f220 2024-09-16T15:56:43.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.786+0000 7f9f46e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4006f8e0 con 0x7f9f4015cff0 2024-09-16T15:56:43.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.786+0000 7f9f46e00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f9f400cc8a0 con 0x7f9f40159aa0 2024-09-16T15:56:43.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.786+0000 7f9f3fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9f4006f220 0x7f9f40152480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:43.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.786+0000 7f9f3fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9f4006f220 0x7f9f40152480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:35938/0 (socket says 172.21.15.151:35938) 2024-09-16T15:56:43.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.786+0000 7f9f3fe00640 1 -- 172.21.15.151:0/4270542578 learned_addr learned my addr 172.21.15.151:0/4270542578 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:56:43.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.787+0000 7f9f3f400640 1 --2- 172.21.15.151:0/4270542578 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f40159aa0 0x7f9f400b4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:43.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.787+0000 7f9f44e00640 1 --2- 172.21.15.151:0/4270542578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f4015cff0 0x7f9f40154c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:43.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.787+0000 7f9f3fe00640 1 -- 172.21.15.151:0/4270542578 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f40159aa0 msgr2=0x7f9f400b4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:43.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.787+0000 7f9f3fe00640 1 --2- 172.21.15.151:0/4270542578 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9f40159aa0 0x7f9f400b4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:43.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.787+0000 7f9f3fe00640 1 -- 172.21.15.151:0/4270542578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f4015cff0 msgr2=0x7f9f40154c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:43.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.787+0000 7f9f3fe00640 1 --2- 172.21.15.151:0/4270542578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f4015cff0 0x7f9f40154c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:43.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.787+0000 7f9f3fe00640 1 -- 172.21.15.151:0/4270542578 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f40155170 con 0x7f9f4006f220 2024-09-16T15:56:43.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.787+0000 7f9f44e00640 1 --2- 172.21.15.151:0/4270542578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f4015cff0 0x7f9f40154c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:56:43.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.787+0000 7f9f3fe00640 1 --2- 172.21.15.151:0/4270542578 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9f4006f220 0x7f9f40152480 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f9f3000ea90 tx=0x7f9f3000ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:43.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.787+0000 7f9f37e00640 1 -- 172.21.15.151:0/4270542578 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3000ce40 con 0x7f9f4006f220 2024-09-16T15:56:43.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.787+0000 7f9f37e00640 1 -- 172.21.15.151:0/4270542578 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9f30004020 con 0x7f9f4006f220 2024-09-16T15:56:43.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.787+0000 7f9f46e00640 1 -- 172.21.15.151:0/4270542578 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f40153b90 con 0x7f9f4006f220 2024-09-16T15:56:43.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.787+0000 7f9f37e00640 1 -- 172.21.15.151:0/4270542578 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f30004470 con 0x7f9f4006f220 2024-09-16T15:56:43.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.788+0000 7f9f46e00640 1 -- 172.21.15.151:0/4270542578 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f40154150 con 0x7f9f4006f220 2024-09-16T15:56:43.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.788+0000 7f9f46e00640 1 -- 172.21.15.151:0/4270542578 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9f4006e6f0 con 0x7f9f4006f220 2024-09-16T15:56:43.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.790+0000 7f9f37e00640 1 -- 172.21.15.151:0/4270542578 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 55) ==== 104857+0+0 (secure 0 0 0) 0x7f9f300035f0 con 0x7f9f4006f220 2024-09-16T15:56:43.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.791+0000 7f9f37e00640 1 --2- 172.21.15.151:0/4270542578 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9f2007afb0 0x7f9f2007d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:43.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.791+0000 7f9f37e00640 1 -- 172.21.15.151:0/4270542578 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5083+0+0 (secure 0 0 0) 0x7f9f3009a680 con 0x7f9f4006f220 2024-09-16T15:56:43.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.791+0000 7f9f37e00640 1 --2- 172.21.15.151:0/4270542578 >> [v2:172.21.15.167:6810/2735495271,v1:172.21.15.167:6811/2735495271] conn(0x7f9f20080f30 0x7f9f200833d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:43.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.791+0000 7f9f37e00640 1 -- 172.21.15.151:0/4270542578 --> [v2:172.21.15.167:6810/2735495271,v1:172.21.15.167:6811/2735495271] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9f30013e50 con 0x7f9f20080f30 2024-09-16T15:56:43.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.791+0000 7f9f37e00640 1 -- 172.21.15.151:0/4270542578 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_get_version_reply(handle=1 version=152) ==== 24+0+0 (secure 0 0 0) 0x7f9f300a6050 con 0x7f9f4006f220 2024-09-16T15:56:43.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.791+0000 7f9f3f400640 1 --2- 172.21.15.151:0/4270542578 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9f2007afb0 0x7f9f2007d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:43.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.791+0000 7f9f44e00640 1 --2- 172.21.15.151:0/4270542578 >> [v2:172.21.15.167:6810/2735495271,v1:172.21.15.167:6811/2735495271] conn(0x7f9f20080f30 0x7f9f200833d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:43.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.792+0000 7f9f3f400640 1 --2- 172.21.15.151:0/4270542578 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9f2007afb0 0x7f9f2007d470 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c034000 tx=0x7f9f2c004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:43.794 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.792+0000 7f9f44e00640 1 --2- 172.21.15.151:0/4270542578 >> [v2:172.21.15.167:6810/2735495271,v1:172.21.15.167:6811/2735495271] conn(0x7f9f20080f30 0x7f9f200833d0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:43.795 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.793+0000 7f9f37e00640 1 -- 172.21.15.151:0/4270542578 <== osd.4 v2:172.21.15.167:6810/2735495271 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9f30013e50 con 0x7f9f20080f30 2024-09-16T15:56:43.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.806+0000 7f9f46e00640 1 -- 172.21.15.151:0/4270542578 --> [v2:172.21.15.167:6810/2735495271,v1:172.21.15.167:6811/2735495271] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f9f401547a0 con 0x7f9f20080f30 2024-09-16T15:56:43.808 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.806+0000 7f9f37e00640 1 -- 172.21.15.151:0/4270542578 <== osd.4 v2:172.21.15.167:6810/2735495271 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f9f401547a0 con 0x7f9f20080f30 2024-09-16T15:56:43.808 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-16T15:56:43.808 INFO:teuthology.orchestra.run.smithi151.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T15:56:43.808 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-16T15:56:43.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.807+0000 7f9f46e00640 1 -- 172.21.15.151:0/4270542578 >> [v2:172.21.15.167:6810/2735495271,v1:172.21.15.167:6811/2735495271] conn(0x7f9f20080f30 msgr2=0x7f9f200833d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:43.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.807+0000 7f9f46e00640 1 --2- 172.21.15.151:0/4270542578 >> [v2:172.21.15.167:6810/2735495271,v1:172.21.15.167:6811/2735495271] conn(0x7f9f20080f30 0x7f9f200833d0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:43.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.807+0000 7f9f46e00640 1 -- 172.21.15.151:0/4270542578 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9f2007afb0 msgr2=0x7f9f2007d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:43.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.807+0000 7f9f46e00640 1 --2- 172.21.15.151:0/4270542578 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f9f2007afb0 0x7f9f2007d470 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c034000 tx=0x7f9f2c004730 comp rx=0 tx=0).stop 2024-09-16T15:56:43.809 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.807+0000 7f9f46e00640 1 -- 172.21.15.151:0/4270542578 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9f4006f220 msgr2=0x7f9f40152480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:43.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.807+0000 7f9f46e00640 1 --2- 172.21.15.151:0/4270542578 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9f4006f220 0x7f9f40152480 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f9f3000ea90 tx=0x7f9f3000ef50 comp rx=0 tx=0).stop 2024-09-16T15:56:43.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.807+0000 7f9f44e00640 1 -- 172.21.15.151:0/4270542578 reap_dead start 2024-09-16T15:56:43.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.807+0000 7f9f46e00640 1 -- 172.21.15.151:0/4270542578 shutdown_connections 2024-09-16T15:56:43.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.807+0000 7f9f46e00640 1 -- 172.21.15.151:0/4270542578 >> 172.21.15.151:0/4270542578 conn(0x7f9f400b18e0 msgr2=0x7f9f4015b650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:43.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.807+0000 7f9f46e00640 1 -- 172.21.15.151:0/4270542578 shutdown_connections 2024-09-16T15:56:43.810 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.807+0000 7f9f46e00640 1 -- 172.21.15.151:0/4270542578 wait complete. 2024-09-16T15:56:43.824 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-16T15:56:43.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.982+0000 7f8b9e400640 1 -- 172.21.15.151:0/4018637806 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b980af3d0 msgr2=0x7f8b9806f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:43.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.982+0000 7f8b9e400640 1 --2- 172.21.15.151:0/4018637806 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b980af3d0 0x7f8b9806f490 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f8b9806d9e0 tx=0x7f8b8402a4d0 comp rx=0 tx=0).stop 2024-09-16T15:56:43.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.983+0000 7f8b9e400640 1 -- 172.21.15.151:0/4018637806 shutdown_connections 2024-09-16T15:56:43.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.983+0000 7f8b9e400640 1 --2- 172.21.15.151:0/4018637806 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b9806f9d0 0x7f8b98161c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:43.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.983+0000 7f8b9e400640 1 --2- 172.21.15.151:0/4018637806 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b980af3d0 0x7f8b9806f490 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:43.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.983+0000 7f8b9e400640 1 --2- 172.21.15.151:0/4018637806 >> v2:172.21.15.167:3300/0 conn(0x7f8b98161550 0x7f8b9806ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:43.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.983+0000 7f8b9e400640 1 -- 172.21.15.151:0/4018637806 >> 172.21.15.151:0/4018637806 conn(0x7f8b980b1950 msgr2=0x7f8b980b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:43.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.983+0000 7f8b9e400640 1 -- 172.21.15.151:0/4018637806 shutdown_connections 2024-09-16T15:56:43.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.983+0000 7f8b9e400640 1 -- 172.21.15.151:0/4018637806 wait complete. 2024-09-16T15:56:43.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.984+0000 7f8b9e400640 1 Processor -- start 2024-09-16T15:56:43.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.984+0000 7f8b9e400640 1 -- start start 2024-09-16T15:56:43.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.985+0000 7f8b9e400640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b9806f9d0 0x7f8b980ae040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:43.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.985+0000 7f8b9e400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b980af3d0 0x7f8b9815a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:43.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.985+0000 7f8b9e400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b98161550 0x7f8b981f7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:43.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.985+0000 7f8b9e400640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8b98162560 con 0x7f8b98161550 2024-09-16T15:56:43.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.985+0000 7f8b9e400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8b981623e0 con 0x7f8b980af3d0 2024-09-16T15:56:43.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.985+0000 7f8b9e400640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f8b981626e0 con 0x7f8b9806f9d0 2024-09-16T15:56:43.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.986+0000 7f8b97e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b98161550 0x7f8b981f7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:43.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.986+0000 7f8b97e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b98161550 0x7f8b981f7420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:35964/0 (socket says 172.21.15.151:35964) 2024-09-16T15:56:43.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.986+0000 7f8b97e00640 1 -- 172.21.15.151:0/1437770337 learned_addr learned my addr 172.21.15.151:0/1437770337 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:56:43.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.986+0000 7f8b96a00640 1 --2- 172.21.15.151:0/1437770337 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b980af3d0 0x7f8b9815a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:43.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.986+0000 7f8b97400640 1 --2- 172.21.15.151:0/1437770337 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b9806f9d0 0x7f8b980ae040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:43.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.986+0000 7f8b97e00640 1 -- 172.21.15.151:0/1437770337 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b9806f9d0 msgr2=0x7f8b980ae040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:43.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.986+0000 7f8b97e00640 1 --2- 172.21.15.151:0/1437770337 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b9806f9d0 0x7f8b980ae040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:43.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.986+0000 7f8b97e00640 1 -- 172.21.15.151:0/1437770337 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b980af3d0 msgr2=0x7f8b9815a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:43.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.986+0000 7f8b97e00640 1 --2- 172.21.15.151:0/1437770337 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b980af3d0 0x7f8b9815a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:43.989 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.986+0000 7f8b97e00640 1 -- 172.21.15.151:0/1437770337 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b981f82d0 con 0x7f8b98161550 2024-09-16T15:56:43.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.986+0000 7f8b97e00640 1 --2- 172.21.15.151:0/1437770337 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b98161550 0x7f8b981f7420 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f8b9000e980 tx=0x7f8b9000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:43.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.987+0000 7f8b8f400640 1 -- 172.21.15.151:0/1437770337 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b90004d20 con 0x7f8b98161550 2024-09-16T15:56:43.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.987+0000 7f8b8f400640 1 -- 172.21.15.151:0/1437770337 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8b9000cb20 con 0x7f8b98161550 2024-09-16T15:56:43.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.987+0000 7f8b8f400640 1 -- 172.21.15.151:0/1437770337 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b90005020 con 0x7f8b98161550 2024-09-16T15:56:43.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.987+0000 7f8b9e400640 1 -- 172.21.15.151:0/1437770337 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b981f85b0 con 0x7f8b98161550 2024-09-16T15:56:43.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.987+0000 7f8b9e400640 1 -- 172.21.15.151:0/1437770337 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b981f8b70 con 0x7f8b98161550 2024-09-16T15:56:43.990 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.988+0000 7f8b9e400640 1 -- 172.21.15.151:0/1437770337 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b98161c90 con 0x7f8b98161550 2024-09-16T15:56:43.991 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.989+0000 7f8b8f400640 1 -- 172.21.15.151:0/1437770337 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 55) ==== 104857+0+0 (secure 0 0 0) 0x7f8b900051c0 con 0x7f8b98161550 2024-09-16T15:56:43.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.990+0000 7f8b8f400640 1 --2- 172.21.15.151:0/1437770337 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8b7807afb0 0x7f8b7807d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:43.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.990+0000 7f8b8f400640 1 -- 172.21.15.151:0/1437770337 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5083+0+0 (secure 0 0 0) 0x7f8b9009a530 con 0x7f8b98161550 2024-09-16T15:56:43.992 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.991+0000 7f8b97400640 1 --2- 172.21.15.151:0/1437770337 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8b7807afb0 0x7f8b7807d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:43.993 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.991+0000 7f8b97400640 1 --2- 172.21.15.151:0/1437770337 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8b7807afb0 0x7f8b7807d470 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f8b88005fd0 tx=0x7f8b8800b2e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:43.996 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:43.994+0000 7f8b8f400640 1 -- 172.21.15.151:0/1437770337 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f8b90016030 con 0x7f8b98161550 2024-09-16T15:56:44.180 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.178+0000 7f8b9e400640 1 -- 172.21.15.151:0/1437770337 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f8b980b3cf0 con 0x7f8b7807afb0 2024-09-16T15:56:44.181 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.179+0000 7f8b8f400640 1 -- 172.21.15.151:0/1437770337 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f8b980b3cf0 con 0x7f8b7807afb0 2024-09-16T15:56:44.181 INFO:teuthology.orchestra.run.smithi151.stderr:instructed osd(s) 4 to deep-scrub 2024-09-16T15:56:44.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.181+0000 7f8b9e400640 1 -- 172.21.15.151:0/1437770337 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8b7807afb0 msgr2=0x7f8b7807d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:44.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.181+0000 7f8b9e400640 1 --2- 172.21.15.151:0/1437770337 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8b7807afb0 0x7f8b7807d470 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f8b88005fd0 tx=0x7f8b8800b2e0 comp rx=0 tx=0).stop 2024-09-16T15:56:44.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.181+0000 7f8b9e400640 1 -- 172.21.15.151:0/1437770337 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b98161550 msgr2=0x7f8b981f7420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:44.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.181+0000 7f8b9e400640 1 --2- 172.21.15.151:0/1437770337 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b98161550 0x7f8b981f7420 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f8b9000e980 tx=0x7f8b9000ee40 comp rx=0 tx=0).stop 2024-09-16T15:56:44.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.181+0000 7f8b9e400640 1 -- 172.21.15.151:0/1437770337 shutdown_connections 2024-09-16T15:56:44.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.181+0000 7f8b9e400640 1 --2- 172.21.15.151:0/1437770337 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f8b7807afb0 0x7f8b7807d470 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:44.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.181+0000 7f8b9e400640 1 --2- 172.21.15.151:0/1437770337 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b980af3d0 0x7f8b9815a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:44.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.181+0000 7f8b9e400640 1 --2- 172.21.15.151:0/1437770337 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b98161550 0x7f8b981f7420 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:44.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.181+0000 7f8b9e400640 1 --2- 172.21.15.151:0/1437770337 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8b9806f9d0 0x7f8b980ae040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:44.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.181+0000 7f8b9e400640 1 -- 172.21.15.151:0/1437770337 >> 172.21.15.151:0/1437770337 conn(0x7f8b980b1950 msgr2=0x7f8b98073a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:44.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.182+0000 7f8b9e400640 1 -- 172.21.15.151:0/1437770337 shutdown_connections 2024-09-16T15:56:44.184 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.182+0000 7f8b9e400640 1 -- 172.21.15.151:0/1437770337 wait complete. 2024-09-16T15:56:44.199 INFO:tasks.ceph:Scrubbing osd.5 2024-09-16T15:56:44.200 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-16T15:56:44.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.360+0000 7f462ba00640 1 -- 172.21.15.151:0/2066602328 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f462415cff0 msgr2=0x7f462406e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:44.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.360+0000 7f462ba00640 1 --2- 172.21.15.151:0/2066602328 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f462415cff0 0x7f462406e6f0 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f4624075110 tx=0x7f461002a540 comp rx=0 tx=0).stop 2024-09-16T15:56:44.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.361+0000 7f462ba00640 1 -- 172.21.15.151:0/2066602328 shutdown_connections 2024-09-16T15:56:44.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.361+0000 7f462ba00640 1 --2- 172.21.15.151:0/2066602328 >> v2:172.21.15.167:3300/0 conn(0x7f462406f220 0x7f462406f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:44.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.361+0000 7f462ba00640 1 --2- 172.21.15.151:0/2066602328 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f462415cff0 0x7f462406e6f0 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:44.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.361+0000 7f462ba00640 1 -- 172.21.15.151:0/2066602328 >> 172.21.15.151:0/2066602328 conn(0x7f46240b18e0 msgr2=0x7f46240b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:44.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.361+0000 7f462ba00640 1 -- 172.21.15.151:0/2066602328 shutdown_connections 2024-09-16T15:56:44.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.361+0000 7f462ba00640 1 -- 172.21.15.151:0/2066602328 wait complete. 2024-09-16T15:56:44.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.362+0000 7f462ba00640 1 Processor -- start 2024-09-16T15:56:44.364 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.362+0000 7f462ba00640 1 -- start start 2024-09-16T15:56:44.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.363+0000 7f462ba00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f462406f220 0x7f4624154830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:44.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.363+0000 7f462ba00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4624159aa0 0x7f46241f53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:44.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.363+0000 7f462ba00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f462415cff0 0x7f46240d0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:44.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.363+0000 7f462ba00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f46240cc520 con 0x7f462415cff0 2024-09-16T15:56:44.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.363+0000 7f462ba00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f46240cc3a0 con 0x7f4624159aa0 2024-09-16T15:56:44.367 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.363+0000 7f462ba00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f46240cc6a0 con 0x7f462406f220 2024-09-16T15:56:44.368 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.364+0000 7f4629a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f462415cff0 0x7f46240d0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:44.368 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.364+0000 7f4629a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f462415cff0 0x7f46240d0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:35994/0 (socket says 172.21.15.151:35994) 2024-09-16T15:56:44.368 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.364+0000 7f4629a00640 1 -- 172.21.15.151:0/3405452079 learned_addr learned my addr 172.21.15.151:0/3405452079 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:56:44.368 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.364+0000 7f4623e00640 1 --2- 172.21.15.151:0/3405452079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4624159aa0 0x7f46241f53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:44.368 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.364+0000 7f4629000640 1 --2- 172.21.15.151:0/3405452079 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f462406f220 0x7f4624154830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:44.368 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.364+0000 7f4629a00640 1 -- 172.21.15.151:0/3405452079 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f462406f220 msgr2=0x7f4624154830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:44.368 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.364+0000 7f4629a00640 1 --2- 172.21.15.151:0/3405452079 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f462406f220 0x7f4624154830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:44.368 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.364+0000 7f4629a00640 1 -- 172.21.15.151:0/3405452079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4624159aa0 msgr2=0x7f46241f53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:44.368 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.364+0000 7f4629a00640 1 --2- 172.21.15.151:0/3405452079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4624159aa0 0x7f46241f53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:44.368 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.364+0000 7f4629a00640 1 -- 172.21.15.151:0/3405452079 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46240af760 con 0x7f462415cff0 2024-09-16T15:56:44.368 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.364+0000 7f4623e00640 1 --2- 172.21.15.151:0/3405452079 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4624159aa0 0x7f46241f53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:56:44.368 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.364+0000 7f4629000640 1 --2- 172.21.15.151:0/3405452079 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f462406f220 0x7f4624154830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:56:44.368 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.364+0000 7f4629a00640 1 --2- 172.21.15.151:0/3405452079 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f462415cff0 0x7f46240d0270 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f461c00d740 tx=0x7f461c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:44.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.365+0000 7f4620c00640 1 -- 172.21.15.151:0/3405452079 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461c017070 con 0x7f462415cff0 2024-09-16T15:56:44.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.365+0000 7f4620c00640 1 -- 172.21.15.151:0/3405452079 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f461c009d20 con 0x7f462415cff0 2024-09-16T15:56:44.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.365+0000 7f462ba00640 1 -- 172.21.15.151:0/3405452079 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46240afa40 con 0x7f462415cff0 2024-09-16T15:56:44.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.365+0000 7f4620c00640 1 -- 172.21.15.151:0/3405452079 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461c004280 con 0x7f462415cff0 2024-09-16T15:56:44.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.365+0000 7f462ba00640 1 -- 172.21.15.151:0/3405452079 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46240d07b0 con 0x7f462415cff0 2024-09-16T15:56:44.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.366+0000 7f462ba00640 1 -- 172.21.15.151:0/3405452079 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f46240ccb50 con 0x7f462415cff0 2024-09-16T15:56:44.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.367+0000 7f4620c00640 1 -- 172.21.15.151:0/3405452079 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 55) ==== 104857+0+0 (secure 0 0 0) 0x7f461c0035f0 con 0x7f462415cff0 2024-09-16T15:56:44.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.368+0000 7f4620c00640 1 --2- 172.21.15.151:0/3405452079 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f460407afb0 0x7f460407d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:44.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.368+0000 7f4620c00640 1 -- 172.21.15.151:0/3405452079 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5083+0+0 (secure 0 0 0) 0x7f461c09a330 con 0x7f462415cff0 2024-09-16T15:56:44.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.368+0000 7f4620c00640 1 --2- 172.21.15.151:0/3405452079 >> [v2:172.21.15.167:6826/1494916572,v1:172.21.15.167:6827/1494916572] conn(0x7f4604080f30 0x7f46040833d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:44.370 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.368+0000 7f4620c00640 1 -- 172.21.15.151:0/3405452079 --> [v2:172.21.15.167:6826/1494916572,v1:172.21.15.167:6827/1494916572] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4604083aa0 con 0x7f4604080f30 2024-09-16T15:56:44.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.368+0000 7f4620c00640 1 -- 172.21.15.151:0/3405452079 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_get_version_reply(handle=1 version=152) ==== 24+0+0 (secure 0 0 0) 0x7f461c09a6e0 con 0x7f462415cff0 2024-09-16T15:56:44.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.369+0000 7f4629000640 1 --2- 172.21.15.151:0/3405452079 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f460407afb0 0x7f460407d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:44.371 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.369+0000 7f4623e00640 1 --2- 172.21.15.151:0/3405452079 >> [v2:172.21.15.167:6826/1494916572,v1:172.21.15.167:6827/1494916572] conn(0x7f4604080f30 0x7f46040833d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:44.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.370+0000 7f4629000640 1 --2- 172.21.15.151:0/3405452079 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f460407afb0 0x7f460407d470 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f46240d0060 tx=0x7f4614009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:44.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.370+0000 7f4623e00640 1 --2- 172.21.15.151:0/3405452079 >> [v2:172.21.15.167:6826/1494916572,v1:172.21.15.167:6827/1494916572] conn(0x7f4604080f30 0x7f46040833d0 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:44.372 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.371+0000 7f4620c00640 1 -- 172.21.15.151:0/3405452079 <== osd.5 v2:172.21.15.167:6826/1494916572 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4604083aa0 con 0x7f4604080f30 2024-09-16T15:56:44.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.384+0000 7f462ba00640 1 -- 172.21.15.151:0/3405452079 --> [v2:172.21.15.167:6826/1494916572,v1:172.21.15.167:6827/1494916572] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f4624070980 con 0x7f4604080f30 2024-09-16T15:56:44.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.384+0000 7f4620c00640 1 -- 172.21.15.151:0/3405452079 <== osd.5 v2:172.21.15.167:6826/1494916572 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f4624070980 con 0x7f4604080f30 2024-09-16T15:56:44.386 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-16T15:56:44.386 INFO:teuthology.orchestra.run.smithi151.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T15:56:44.387 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-16T15:56:44.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.385+0000 7f462ba00640 1 -- 172.21.15.151:0/3405452079 >> [v2:172.21.15.167:6826/1494916572,v1:172.21.15.167:6827/1494916572] conn(0x7f4604080f30 msgr2=0x7f46040833d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:44.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.385+0000 7f462ba00640 1 --2- 172.21.15.151:0/3405452079 >> [v2:172.21.15.167:6826/1494916572,v1:172.21.15.167:6827/1494916572] conn(0x7f4604080f30 0x7f46040833d0 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:44.387 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.385+0000 7f462ba00640 1 -- 172.21.15.151:0/3405452079 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f460407afb0 msgr2=0x7f460407d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:44.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.385+0000 7f462ba00640 1 --2- 172.21.15.151:0/3405452079 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f460407afb0 0x7f460407d470 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f46240d0060 tx=0x7f4614009040 comp rx=0 tx=0).stop 2024-09-16T15:56:44.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.385+0000 7f462ba00640 1 -- 172.21.15.151:0/3405452079 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f462415cff0 msgr2=0x7f46240d0270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:44.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.385+0000 7f462ba00640 1 --2- 172.21.15.151:0/3405452079 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f462415cff0 0x7f46240d0270 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f461c00d740 tx=0x7f461c00dc00 comp rx=0 tx=0).stop 2024-09-16T15:56:44.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.385+0000 7f4629a00640 1 -- 172.21.15.151:0/3405452079 reap_dead start 2024-09-16T15:56:44.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.385+0000 7f462ba00640 1 -- 172.21.15.151:0/3405452079 shutdown_connections 2024-09-16T15:56:44.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.385+0000 7f462ba00640 1 -- 172.21.15.151:0/3405452079 >> 172.21.15.151:0/3405452079 conn(0x7f46240b18e0 msgr2=0x7f462415e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:44.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.385+0000 7f462ba00640 1 -- 172.21.15.151:0/3405452079 shutdown_connections 2024-09-16T15:56:44.388 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.385+0000 7f462ba00640 1 -- 172.21.15.151:0/3405452079 wait complete. 2024-09-16T15:56:44.401 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-16T15:56:44.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.559+0000 7f3a7ca00640 1 -- 172.21.15.151:0/3397686692 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3a78161440 msgr2=0x7f3a780b5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:44.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.559+0000 7f3a7ca00640 1 --2- 172.21.15.151:0/3397686692 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3a78161440 0x7f3a780b5370 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3a780d4ea0 tx=0x7f3a6802a480 comp rx=0 tx=0).stop 2024-09-16T15:56:44.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.559+0000 7f3a7ca00640 1 -- 172.21.15.151:0/3397686692 shutdown_connections 2024-09-16T15:56:44.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.559+0000 7f3a7ca00640 1 --2- 172.21.15.151:0/3397686692 >> v2:172.21.15.167:3300/0 conn(0x7f3a780b34b0 0x7f3a780b3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:44.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.559+0000 7f3a7ca00640 1 --2- 172.21.15.151:0/3397686692 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3a78161440 0x7f3a780b5370 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:44.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.559+0000 7f3a7ca00640 1 -- 172.21.15.151:0/3397686692 >> 172.21.15.151:0/3397686692 conn(0x7f3a780d3df0 msgr2=0x7f3a780d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:44.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.559+0000 7f3a7ca00640 1 -- 172.21.15.151:0/3397686692 shutdown_connections 2024-09-16T15:56:44.562 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.559+0000 7f3a7ca00640 1 -- 172.21.15.151:0/3397686692 wait complete. 2024-09-16T15:56:44.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.561+0000 7f3a7ca00640 1 Processor -- start 2024-09-16T15:56:44.563 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.561+0000 7f3a7ca00640 1 -- start start 2024-09-16T15:56:44.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.562+0000 7f3a7ca00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a780b34b0 0x7f3a780b6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:44.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.562+0000 7f3a7ca00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3a780b3e50 0x7f3a781f4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:44.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.562+0000 7f3a7ca00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3a78161440 0x7f3a780b8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:44.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.562+0000 7f3a7ca00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3a781619c0 con 0x7f3a78161440 2024-09-16T15:56:44.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.562+0000 7f3a7ca00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3a780b2990 con 0x7f3a780b34b0 2024-09-16T15:56:44.565 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.562+0000 7f3a7ca00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f3a780d2470 con 0x7f3a780b3e50 2024-09-16T15:56:44.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.562+0000 7f3a77e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3a78161440 0x7f3a780b8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:44.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.563+0000 7f3a77e00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3a78161440 0x7f3a780b8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36008/0 (socket says 172.21.15.151:36008) 2024-09-16T15:56:44.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.563+0000 7f3a77e00640 1 -- 172.21.15.151:0/1248963715 learned_addr learned my addr 172.21.15.151:0/1248963715 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:56:44.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.563+0000 7f3a77400640 1 --2- 172.21.15.151:0/1248963715 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a780b34b0 0x7f3a780b6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:44.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.563+0000 7f3a76a00640 1 --2- 172.21.15.151:0/1248963715 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3a780b3e50 0x7f3a781f4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:44.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.563+0000 7f3a77e00640 1 -- 172.21.15.151:0/1248963715 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3a780b3e50 msgr2=0x7f3a781f4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:44.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.563+0000 7f3a77e00640 1 --2- 172.21.15.151:0/1248963715 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3a780b3e50 0x7f3a781f4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:44.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.563+0000 7f3a77e00640 1 -- 172.21.15.151:0/1248963715 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a780b34b0 msgr2=0x7f3a780b6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:44.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.563+0000 7f3a77e00640 1 --2- 172.21.15.151:0/1248963715 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a780b34b0 0x7f3a780b6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:44.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.563+0000 7f3a77e00640 1 -- 172.21.15.151:0/1248963715 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a780b92f0 con 0x7f3a78161440 2024-09-16T15:56:44.566 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.563+0000 7f3a77e00640 1 --2- 172.21.15.151:0/1248963715 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3a78161440 0x7f3a780b8540 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f3a7000c7a0 tx=0x7f3a7000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:44.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.564+0000 7f3a6f400640 1 -- 172.21.15.151:0/1248963715 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a70003b70 con 0x7f3a78161440 2024-09-16T15:56:44.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.564+0000 7f3a6f400640 1 -- 172.21.15.151:0/1248963715 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3a70003d10 con 0x7f3a78161440 2024-09-16T15:56:44.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.564+0000 7f3a6f400640 1 -- 172.21.15.151:0/1248963715 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a7000aab0 con 0x7f3a78161440 2024-09-16T15:56:44.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.564+0000 7f3a7ca00640 1 -- 172.21.15.151:0/1248963715 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a780b95d0 con 0x7f3a78161440 2024-09-16T15:56:44.567 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.564+0000 7f3a7ca00640 1 -- 172.21.15.151:0/1248963715 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a780b9b90 con 0x7f3a78161440 2024-09-16T15:56:44.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.565+0000 7f3a7ca00640 1 -- 172.21.15.151:0/1248963715 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a780b3910 con 0x7f3a78161440 2024-09-16T15:56:44.568 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.566+0000 7f3a6f400640 1 -- 172.21.15.151:0/1248963715 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 55) ==== 104857+0+0 (secure 0 0 0) 0x7f3a7000ac50 con 0x7f3a78161440 2024-09-16T15:56:44.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.567+0000 7f3a6f400640 1 --2- 172.21.15.151:0/1248963715 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3a5807afb0 0x7f3a5807d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:44.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.567+0000 7f3a6f400640 1 -- 172.21.15.151:0/1248963715 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5083+0+0 (secure 0 0 0) 0x7f3a70099950 con 0x7f3a78161440 2024-09-16T15:56:44.569 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.568+0000 7f3a77400640 1 --2- 172.21.15.151:0/1248963715 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3a5807afb0 0x7f3a5807d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:44.570 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.569+0000 7f3a77400640 1 --2- 172.21.15.151:0/1248963715 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3a5807afb0 0x7f3a5807d470 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f3a68034000 tx=0x7f3a68004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:44.573 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.571+0000 7f3a6f400640 1 -- 172.21.15.151:0/1248963715 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f3a70063450 con 0x7f3a78161440 2024-09-16T15:56:44.757 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.755+0000 7f3a7ca00640 1 -- 172.21.15.151:0/1248963715 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f3a780b2430 con 0x7f3a5807afb0 2024-09-16T15:56:44.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.756+0000 7f3a6f400640 1 -- 172.21.15.151:0/1248963715 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f3a780b2430 con 0x7f3a5807afb0 2024-09-16T15:56:44.781 INFO:teuthology.orchestra.run.smithi151.stderr:instructed osd(s) 5 to deep-scrub 2024-09-16T15:56:44.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.758+0000 7f3a7ca00640 1 -- 172.21.15.151:0/1248963715 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3a5807afb0 msgr2=0x7f3a5807d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:44.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.758+0000 7f3a7ca00640 1 --2- 172.21.15.151:0/1248963715 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3a5807afb0 0x7f3a5807d470 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f3a68034000 tx=0x7f3a68004730 comp rx=0 tx=0).stop 2024-09-16T15:56:44.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.758+0000 7f3a7ca00640 1 -- 172.21.15.151:0/1248963715 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3a78161440 msgr2=0x7f3a780b8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:44.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.758+0000 7f3a7ca00640 1 --2- 172.21.15.151:0/1248963715 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3a78161440 0x7f3a780b8540 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f3a7000c7a0 tx=0x7f3a7000cc60 comp rx=0 tx=0).stop 2024-09-16T15:56:44.781 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.758+0000 7f3a7ca00640 1 -- 172.21.15.151:0/1248963715 shutdown_connections 2024-09-16T15:56:44.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.758+0000 7f3a7ca00640 1 --2- 172.21.15.151:0/1248963715 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f3a5807afb0 0x7f3a5807d470 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:44.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.758+0000 7f3a7ca00640 1 --2- 172.21.15.151:0/1248963715 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3a78161440 0x7f3a780b8540 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:44.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.758+0000 7f3a7ca00640 1 --2- 172.21.15.151:0/1248963715 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3a780b3e50 0x7f3a781f4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:44.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.758+0000 7f3a7ca00640 1 --2- 172.21.15.151:0/1248963715 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a780b34b0 0x7f3a780b6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:44.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.758+0000 7f3a7ca00640 1 -- 172.21.15.151:0/1248963715 >> 172.21.15.151:0/1248963715 conn(0x7f3a780d3df0 msgr2=0x7f3a780b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:44.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.758+0000 7f3a7ca00640 1 -- 172.21.15.151:0/1248963715 shutdown_connections 2024-09-16T15:56:44.782 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.758+0000 7f3a7ca00640 1 -- 172.21.15.151:0/1248963715 wait complete. 2024-09-16T15:56:44.782 INFO:tasks.ceph:Scrubbing osd.6 2024-09-16T15:56:44.782 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-16T15:56:44.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.944+0000 7fe23ca00640 1 -- 172.21.15.151:0/173572658 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe238159aa0 msgr2=0x7fe2380741a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:44.946 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.944+0000 7fe23ca00640 1 --2- 172.21.15.151:0/173572658 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe238159aa0 0x7fe2380741a0 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7fe228009900 tx=0x7fe22802a610 comp rx=0 tx=0).stop 2024-09-16T15:56:44.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.944+0000 7fe23ca00640 1 -- 172.21.15.151:0/173572658 shutdown_connections 2024-09-16T15:56:44.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.944+0000 7fe23ca00640 1 --2- 172.21.15.151:0/173572658 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe23815cff0 0x7fe23806e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:44.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.944+0000 7fe23ca00640 1 --2- 172.21.15.151:0/173572658 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe238159aa0 0x7fe2380741a0 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:44.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.944+0000 7fe23ca00640 1 -- 172.21.15.151:0/173572658 >> 172.21.15.151:0/173572658 conn(0x7fe2380b18e0 msgr2=0x7fe2380b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:44.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.944+0000 7fe23ca00640 1 -- 172.21.15.151:0/173572658 shutdown_connections 2024-09-16T15:56:44.947 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.944+0000 7fe23ca00640 1 -- 172.21.15.151:0/173572658 wait complete. 2024-09-16T15:56:44.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.946+0000 7fe23ca00640 1 Processor -- start 2024-09-16T15:56:44.948 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.946+0000 7fe23ca00640 1 -- start start 2024-09-16T15:56:44.949 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.947+0000 7fe23ca00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe23806f220 0x7fe2381549b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:44.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.947+0000 7fe23ca00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe238159aa0 0x7fe23809c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:44.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.947+0000 7fe23ca00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe23815cff0 0x7fe2381f8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:44.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.947+0000 7fe23ca00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe23806fa60 con 0x7fe23815cff0 2024-09-16T15:56:44.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.947+0000 7fe23ca00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe23806f8e0 con 0x7fe23806f220 2024-09-16T15:56:44.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.947+0000 7fe23ca00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fe2380cc8a0 con 0x7fe238159aa0 2024-09-16T15:56:44.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.947+0000 7fe236800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe23815cff0 0x7fe2381f8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:44.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.947+0000 7fe236800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe23815cff0 0x7fe2381f8290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36030/0 (socket says 172.21.15.151:36030) 2024-09-16T15:56:44.950 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.947+0000 7fe236800640 1 -- 172.21.15.151:0/1275312811 learned_addr learned my addr 172.21.15.151:0/1275312811 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:56:44.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.947+0000 7fe235400640 1 --2- 172.21.15.151:0/1275312811 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe238159aa0 0x7fe23809c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:44.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.947+0000 7fe235e00640 1 --2- 172.21.15.151:0/1275312811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe23806f220 0x7fe2381549b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:44.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.948+0000 7fe236800640 1 -- 172.21.15.151:0/1275312811 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe238159aa0 msgr2=0x7fe23809c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:44.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.948+0000 7fe236800640 1 --2- 172.21.15.151:0/1275312811 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe238159aa0 0x7fe23809c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:44.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.948+0000 7fe236800640 1 -- 172.21.15.151:0/1275312811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe23806f220 msgr2=0x7fe2381549b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:44.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.948+0000 7fe236800640 1 --2- 172.21.15.151:0/1275312811 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe23806f220 0x7fe2381549b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:44.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.948+0000 7fe236800640 1 -- 172.21.15.151:0/1275312811 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2380cdad0 con 0x7fe23815cff0 2024-09-16T15:56:44.951 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.948+0000 7fe236800640 1 --2- 172.21.15.151:0/1275312811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe23815cff0 0x7fe2381f8290 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7fe23000c7a0 tx=0x7fe23000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:44.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.948+0000 7fe22e000640 1 -- 172.21.15.151:0/1275312811 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe230003b70 con 0x7fe23815cff0 2024-09-16T15:56:44.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.948+0000 7fe22e000640 1 -- 172.21.15.151:0/1275312811 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe230003d30 con 0x7fe23815cff0 2024-09-16T15:56:44.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.948+0000 7fe22e000640 1 -- 172.21.15.151:0/1275312811 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe23000aab0 con 0x7fe23815cff0 2024-09-16T15:56:44.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.948+0000 7fe23ca00640 1 -- 172.21.15.151:0/1275312811 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe23809ccb0 con 0x7fe23815cff0 2024-09-16T15:56:44.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.948+0000 7fe23ca00640 1 -- 172.21.15.151:0/1275312811 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe23809d270 con 0x7fe23815cff0 2024-09-16T15:56:44.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.949+0000 7fe23ca00640 1 -- 172.21.15.151:0/1275312811 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe2380741a0 con 0x7fe23815cff0 2024-09-16T15:56:44.952 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.950+0000 7fe22e000640 1 -- 172.21.15.151:0/1275312811 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 55) ==== 104857+0+0 (secure 0 0 0) 0x7fe23000ac50 con 0x7fe23815cff0 2024-09-16T15:56:44.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.951+0000 7fe22e000640 1 --2- 172.21.15.151:0/1275312811 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe21807afb0 0x7fe21807d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:44.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.951+0000 7fe22e000640 1 -- 172.21.15.151:0/1275312811 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5083+0+0 (secure 0 0 0) 0x7fe2300999b0 con 0x7fe23815cff0 2024-09-16T15:56:44.953 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.952+0000 7fe22e000640 1 --2- 172.21.15.151:0/1275312811 >> [v2:172.21.15.167:6818/2538273273,v1:172.21.15.167:6819/2538273273] conn(0x7fe218080f30 0x7fe2180833d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:44.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.952+0000 7fe22e000640 1 -- 172.21.15.151:0/1275312811 --> [v2:172.21.15.167:6818/2538273273,v1:172.21.15.167:6819/2538273273] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe23001ae50 con 0x7fe218080f30 2024-09-16T15:56:44.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.952+0000 7fe22e000640 1 -- 172.21.15.151:0/1275312811 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_get_version_reply(handle=1 version=152) ==== 24+0+0 (secure 0 0 0) 0x7fe23009a100 con 0x7fe23815cff0 2024-09-16T15:56:44.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.952+0000 7fe235e00640 1 --2- 172.21.15.151:0/1275312811 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe21807afb0 0x7fe21807d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:44.954 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.952+0000 7fe235400640 1 --2- 172.21.15.151:0/1275312811 >> [v2:172.21.15.167:6818/2538273273,v1:172.21.15.167:6819/2538273273] conn(0x7fe218080f30 0x7fe2180833d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:44.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.953+0000 7fe235e00640 1 --2- 172.21.15.151:0/1275312811 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe21807afb0 0x7fe21807d470 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fe228033000 tx=0x7fe228004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:44.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.953+0000 7fe235400640 1 --2- 172.21.15.151:0/1275312811 >> [v2:172.21.15.167:6818/2538273273,v1:172.21.15.167:6819/2538273273] conn(0x7fe218080f30 0x7fe2180833d0 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:44.955 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.953+0000 7fe22e000640 1 -- 172.21.15.151:0/1275312811 <== osd.6 v2:172.21.15.167:6818/2538273273 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe23001ae50 con 0x7fe218080f30 2024-09-16T15:56:44.968 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.966+0000 7fe23ca00640 1 -- 172.21.15.151:0/1275312811 --> [v2:172.21.15.167:6818/2538273273,v1:172.21.15.167:6819/2538273273] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fe238070980 con 0x7fe218080f30 2024-09-16T15:56:44.969 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.968+0000 7fe22e000640 1 -- 172.21.15.151:0/1275312811 <== osd.6 v2:172.21.15.167:6818/2538273273 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fe238070980 con 0x7fe218080f30 2024-09-16T15:56:44.969 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-16T15:56:44.970 INFO:teuthology.orchestra.run.smithi151.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T15:56:44.970 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-16T15:56:44.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.968+0000 7fe23ca00640 1 -- 172.21.15.151:0/1275312811 >> [v2:172.21.15.167:6818/2538273273,v1:172.21.15.167:6819/2538273273] conn(0x7fe218080f30 msgr2=0x7fe2180833d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:44.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.968+0000 7fe23ca00640 1 --2- 172.21.15.151:0/1275312811 >> [v2:172.21.15.167:6818/2538273273,v1:172.21.15.167:6819/2538273273] conn(0x7fe218080f30 0x7fe2180833d0 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:44.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.968+0000 7fe23ca00640 1 -- 172.21.15.151:0/1275312811 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe21807afb0 msgr2=0x7fe21807d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:44.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.968+0000 7fe23ca00640 1 --2- 172.21.15.151:0/1275312811 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fe21807afb0 0x7fe21807d470 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fe228033000 tx=0x7fe228004730 comp rx=0 tx=0).stop 2024-09-16T15:56:44.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.968+0000 7fe23ca00640 1 -- 172.21.15.151:0/1275312811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe23815cff0 msgr2=0x7fe2381f8290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:44.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.968+0000 7fe23ca00640 1 --2- 172.21.15.151:0/1275312811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe23815cff0 0x7fe2381f8290 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7fe23000c7a0 tx=0x7fe23000cc60 comp rx=0 tx=0).stop 2024-09-16T15:56:44.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.968+0000 7fe236800640 1 -- 172.21.15.151:0/1275312811 reap_dead start 2024-09-16T15:56:44.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.968+0000 7fe23ca00640 1 -- 172.21.15.151:0/1275312811 shutdown_connections 2024-09-16T15:56:44.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.968+0000 7fe23ca00640 1 -- 172.21.15.151:0/1275312811 >> 172.21.15.151:0/1275312811 conn(0x7fe2380b18e0 msgr2=0x7fe2380acd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:44.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.968+0000 7fe23ca00640 1 -- 172.21.15.151:0/1275312811 shutdown_connections 2024-09-16T15:56:44.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:44.968+0000 7fe23ca00640 1 -- 172.21.15.151:0/1275312811 wait complete. 2024-09-16T15:56:44.985 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-16T15:56:45.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.141+0000 7fab72600640 1 -- 172.21.15.151:0/3885827268 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab6c0737d0 msgr2=0x7fab6c151750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:45.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.141+0000 7fab72600640 1 --2- 172.21.15.151:0/3885827268 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab6c0737d0 0x7fab6c151750 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7fab6c076530 tx=0x7fab6402a8b0 comp rx=0 tx=0).stop 2024-09-16T15:56:45.143 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.141+0000 7fab72600640 1 -- 172.21.15.151:0/3885827268 shutdown_connections 2024-09-16T15:56:45.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.141+0000 7fab72600640 1 --2- 172.21.15.151:0/3885827268 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab6c0737d0 0x7fab6c151750 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:45.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.141+0000 7fab72600640 1 --2- 172.21.15.151:0/3885827268 >> v2:172.21.15.167:3300/0 conn(0x7fab6c15bdc0 0x7fab6c0b3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:45.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.141+0000 7fab72600640 1 -- 172.21.15.151:0/3885827268 >> 172.21.15.151:0/3885827268 conn(0x7fab6c0b2230 msgr2=0x7fab6c0b2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:45.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.141+0000 7fab72600640 1 -- 172.21.15.151:0/3885827268 shutdown_connections 2024-09-16T15:56:45.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.141+0000 7fab72600640 1 -- 172.21.15.151:0/3885827268 wait complete. 2024-09-16T15:56:45.144 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.143+0000 7fab72600640 1 Processor -- start 2024-09-16T15:56:45.145 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.143+0000 7fab72600640 1 -- start start 2024-09-16T15:56:45.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.144+0000 7fab72600640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab6c0737d0 0x7fab6c154ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:45.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.144+0000 7fab72600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab6c0b4240 0x7fab6c154250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:45.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.144+0000 7fab72600640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab6c15bdc0 0x7fab6c0ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:45.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.144+0000 7fab72600640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fab6c151e10 con 0x7fab6c15bdc0 2024-09-16T15:56:45.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.144+0000 7fab72600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fab6c151c90 con 0x7fab6c0b4240 2024-09-16T15:56:45.147 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.144+0000 7fab72600640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fab6c151f90 con 0x7fab6c0737d0 2024-09-16T15:56:45.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.144+0000 7fab6be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab6c15bdc0 0x7fab6c0ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:45.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.144+0000 7fab6be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab6c15bdc0 0x7fab6c0ceb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36052/0 (socket says 172.21.15.151:36052) 2024-09-16T15:56:45.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.144+0000 7fab6be00640 1 -- 172.21.15.151:0/3884721558 learned_addr learned my addr 172.21.15.151:0/3884721558 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:56:45.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.144+0000 7fab6b400640 1 --2- 172.21.15.151:0/3884721558 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab6c0737d0 0x7fab6c154ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:45.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.144+0000 7fab6aa00640 1 --2- 172.21.15.151:0/3884721558 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab6c0b4240 0x7fab6c154250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:45.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.144+0000 7fab6be00640 1 -- 172.21.15.151:0/3884721558 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab6c0737d0 msgr2=0x7fab6c154ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:45.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.144+0000 7fab6be00640 1 --2- 172.21.15.151:0/3884721558 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab6c0737d0 0x7fab6c154ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:45.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.144+0000 7fab6be00640 1 -- 172.21.15.151:0/3884721558 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab6c0b4240 msgr2=0x7fab6c154250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:45.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.144+0000 7fab6be00640 1 --2- 172.21.15.151:0/3884721558 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab6c0b4240 0x7fab6c154250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:45.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.144+0000 7fab6be00640 1 -- 172.21.15.151:0/3884721558 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab6c153490 con 0x7fab6c15bdc0 2024-09-16T15:56:45.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.144+0000 7fab6aa00640 1 --2- 172.21.15.151:0/3884721558 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab6c0b4240 0x7fab6c154250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:56:45.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.144+0000 7fab6b400640 1 --2- 172.21.15.151:0/3884721558 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab6c0737d0 0x7fab6c154ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T15:56:45.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.145+0000 7fab6be00640 1 --2- 172.21.15.151:0/3884721558 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab6c15bdc0 0x7fab6c0ceb80 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7fab6402aea0 tx=0x7fab64031760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:45.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.145+0000 7fab63400640 1 -- 172.21.15.151:0/3884721558 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab64040070 con 0x7fab6c15bdc0 2024-09-16T15:56:45.148 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.145+0000 7fab63400640 1 -- 172.21.15.151:0/3884721558 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fab64004040 con 0x7fab6c15bdc0 2024-09-16T15:56:45.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.145+0000 7fab63400640 1 -- 172.21.15.151:0/3884721558 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab64004470 con 0x7fab6c15bdc0 2024-09-16T15:56:45.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.145+0000 7fab72600640 1 -- 172.21.15.151:0/3884721558 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab6c153770 con 0x7fab6c15bdc0 2024-09-16T15:56:45.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.145+0000 7fab72600640 1 -- 172.21.15.151:0/3884721558 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab6c155390 con 0x7fab6c15bdc0 2024-09-16T15:56:45.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.146+0000 7fab72600640 1 -- 172.21.15.151:0/3884721558 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab6c153f40 con 0x7fab6c15bdc0 2024-09-16T15:56:45.149 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.147+0000 7fab63400640 1 -- 172.21.15.151:0/3884721558 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 55) ==== 104857+0+0 (secure 0 0 0) 0x7fab64039040 con 0x7fab6c15bdc0 2024-09-16T15:56:45.150 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.148+0000 7fab63400640 1 --2- 172.21.15.151:0/3884721558 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fab4c07afb0 0x7fab4c07d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:45.150 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.148+0000 7fab63400640 1 -- 172.21.15.151:0/3884721558 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5083+0+0 (secure 0 0 0) 0x7fab640bb3d0 con 0x7fab6c15bdc0 2024-09-16T15:56:45.150 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.149+0000 7fab6b400640 1 --2- 172.21.15.151:0/3884721558 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fab4c07afb0 0x7fab4c07d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:45.151 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.149+0000 7fab6b400640 1 --2- 172.21.15.151:0/3884721558 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fab4c07afb0 0x7fab4c07d470 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fab5c00bbc0 tx=0x7fab5c005ef0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:45.154 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.152+0000 7fab63400640 1 -- 172.21.15.151:0/3884721558 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fab640854a0 con 0x7fab6c15bdc0 2024-09-16T15:56:45.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.336+0000 7fab72600640 1 -- 172.21.15.151:0/3884721558 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7fab6c151750 con 0x7fab4c07afb0 2024-09-16T15:56:45.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.337+0000 7fab63400640 1 -- 172.21.15.151:0/3884721558 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fab6c151750 con 0x7fab4c07afb0 2024-09-16T15:56:45.340 INFO:teuthology.orchestra.run.smithi151.stderr:instructed osd(s) 6 to deep-scrub 2024-09-16T15:56:45.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.339+0000 7fab72600640 1 -- 172.21.15.151:0/3884721558 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fab4c07afb0 msgr2=0x7fab4c07d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:45.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.339+0000 7fab72600640 1 --2- 172.21.15.151:0/3884721558 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fab4c07afb0 0x7fab4c07d470 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fab5c00bbc0 tx=0x7fab5c005ef0 comp rx=0 tx=0).stop 2024-09-16T15:56:45.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.340+0000 7fab72600640 1 -- 172.21.15.151:0/3884721558 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab6c15bdc0 msgr2=0x7fab6c0ceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:45.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.340+0000 7fab72600640 1 --2- 172.21.15.151:0/3884721558 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab6c15bdc0 0x7fab6c0ceb80 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7fab6402aea0 tx=0x7fab64031760 comp rx=0 tx=0).stop 2024-09-16T15:56:45.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.340+0000 7fab72600640 1 -- 172.21.15.151:0/3884721558 shutdown_connections 2024-09-16T15:56:45.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.340+0000 7fab72600640 1 --2- 172.21.15.151:0/3884721558 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fab4c07afb0 0x7fab4c07d470 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:45.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.340+0000 7fab72600640 1 --2- 172.21.15.151:0/3884721558 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fab6c0b4240 0x7fab6c154250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:45.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.340+0000 7fab72600640 1 --2- 172.21.15.151:0/3884721558 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fab6c15bdc0 0x7fab6c0ceb80 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:45.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.340+0000 7fab72600640 1 --2- 172.21.15.151:0/3884721558 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fab6c0737d0 0x7fab6c154ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:45.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.340+0000 7fab72600640 1 -- 172.21.15.151:0/3884721558 >> 172.21.15.151:0/3884721558 conn(0x7fab6c0b2230 msgr2=0x7fab6c0730c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:45.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.340+0000 7fab72600640 1 -- 172.21.15.151:0/3884721558 shutdown_connections 2024-09-16T15:56:45.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.340+0000 7fab72600640 1 -- 172.21.15.151:0/3884721558 wait complete. 2024-09-16T15:56:45.357 INFO:tasks.ceph:Scrubbing osd.7 2024-09-16T15:56:45.358 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-16T15:56:45.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.518+0000 7f6247000640 1 -- 172.21.15.151:0/3371946905 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f624015dfc0 msgr2=0x7f624006f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:45.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.518+0000 7f6247000640 1 --2- 172.21.15.151:0/3371946905 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f624015dfc0 0x7f624006f300 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f62400b3050 tx=0x7f623002a560 comp rx=0 tx=0).stop 2024-09-16T15:56:45.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.519+0000 7f6247000640 1 -- 172.21.15.151:0/3371946905 shutdown_connections 2024-09-16T15:56:45.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.519+0000 7f6247000640 1 --2- 172.21.15.151:0/3371946905 >> v2:172.21.15.167:3300/0 conn(0x7f62400b4770 0x7f624006e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:45.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.519+0000 7f6247000640 1 --2- 172.21.15.151:0/3371946905 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f624015dfc0 0x7f624006f300 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:45.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.519+0000 7f6247000640 1 -- 172.21.15.151:0/3371946905 >> 172.21.15.151:0/3371946905 conn(0x7f62400b18e0 msgr2=0x7f62400b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:45.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.519+0000 7f6247000640 1 -- 172.21.15.151:0/3371946905 shutdown_connections 2024-09-16T15:56:45.521 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.519+0000 7f6247000640 1 -- 172.21.15.151:0/3371946905 wait complete. 2024-09-16T15:56:45.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.520+0000 7f6247000640 1 Processor -- start 2024-09-16T15:56:45.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.520+0000 7f6247000640 1 -- start start 2024-09-16T15:56:45.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.521+0000 7f6247000640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f624006ec30 0x7f62401476f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:45.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.521+0000 7f6247000640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f62400b4770 0x7f6240160c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:45.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.521+0000 7f6247000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f624015dfc0 0x7f62401611c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:45.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.521+0000 7f6247000640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f624015e460 con 0x7f62400b4770 2024-09-16T15:56:45.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.521+0000 7f6247000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f62400ae8f0 con 0x7f624015dfc0 2024-09-16T15:56:45.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.521+0000 7f6247000640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f62401622c0 con 0x7f624006ec30 2024-09-16T15:56:45.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.521+0000 7f623f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f62400b4770 0x7f6240160c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:45.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.521+0000 7f623f400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f62400b4770 0x7f6240160c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36074/0 (socket says 172.21.15.151:36074) 2024-09-16T15:56:45.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.521+0000 7f623f400640 1 -- 172.21.15.151:0/895338773 learned_addr learned my addr 172.21.15.151:0/895338773 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:56:45.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.522+0000 7f623fe00640 1 --2- 172.21.15.151:0/895338773 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f624006ec30 0x7f62401476f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:45.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.522+0000 7f6245000640 1 --2- 172.21.15.151:0/895338773 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f624015dfc0 0x7f62401611c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:45.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.522+0000 7f623f400640 1 -- 172.21.15.151:0/895338773 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f624006ec30 msgr2=0x7f62401476f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:45.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.522+0000 7f623f400640 1 --2- 172.21.15.151:0/895338773 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f624006ec30 0x7f62401476f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:45.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.522+0000 7f623f400640 1 -- 172.21.15.151:0/895338773 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f624015dfc0 msgr2=0x7f62401611c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:45.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.522+0000 7f623f400640 1 --2- 172.21.15.151:0/895338773 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f624015dfc0 0x7f62401611c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:45.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.522+0000 7f623f400640 1 -- 172.21.15.151:0/895338773 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6240153e20 con 0x7f62400b4770 2024-09-16T15:56:45.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.522+0000 7f623f400640 1 --2- 172.21.15.151:0/895338773 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f62400b4770 0x7f6240160c80 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f622c00b4a0 tx=0x7f622c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:45.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.522+0000 7f6237e00640 1 -- 172.21.15.151:0/895338773 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f622c014070 con 0x7f62400b4770 2024-09-16T15:56:45.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.522+0000 7f6237e00640 1 -- 172.21.15.151:0/895338773 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f622c010700 con 0x7f62400b4770 2024-09-16T15:56:45.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.522+0000 7f6247000640 1 -- 172.21.15.151:0/895338773 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f624015c350 con 0x7f62400b4770 2024-09-16T15:56:45.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.522+0000 7f6237e00640 1 -- 172.21.15.151:0/895338773 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f622c010b50 con 0x7f62400b4770 2024-09-16T15:56:45.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.523+0000 7f6247000640 1 -- 172.21.15.151:0/895338773 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f624015c910 con 0x7f62400b4770 2024-09-16T15:56:45.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.523+0000 7f6247000640 1 -- 172.21.15.151:0/895338773 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6240162780 con 0x7f62400b4770 2024-09-16T15:56:45.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.525+0000 7f6237e00640 1 -- 172.21.15.151:0/895338773 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 55) ==== 104857+0+0 (secure 0 0 0) 0x7f622c017070 con 0x7f62400b4770 2024-09-16T15:56:45.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.526+0000 7f6237e00640 1 --2- 172.21.15.151:0/895338773 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f622007afb0 0x7f622007d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:45.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.526+0000 7f6237e00640 1 -- 172.21.15.151:0/895338773 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5083+0+0 (secure 0 0 0) 0x7f622c099600 con 0x7f62400b4770 2024-09-16T15:56:45.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.526+0000 7f6237e00640 1 --2- 172.21.15.151:0/895338773 >> [v2:172.21.15.167:6802/265072227,v1:172.21.15.167:6803/265072227] conn(0x7f6220080f30 0x7f62200833d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:45.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.526+0000 7f6237e00640 1 -- 172.21.15.151:0/895338773 --> [v2:172.21.15.167:6802/265072227,v1:172.21.15.167:6803/265072227] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6220083aa0 con 0x7f6220080f30 2024-09-16T15:56:45.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.526+0000 7f6237e00640 1 -- 172.21.15.151:0/895338773 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_get_version_reply(handle=1 version=152) ==== 24+0+0 (secure 0 0 0) 0x7f622c0a5050 con 0x7f62400b4770 2024-09-16T15:56:45.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.526+0000 7f623fe00640 1 --2- 172.21.15.151:0/895338773 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f622007afb0 0x7f622007d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:45.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.526+0000 7f6245000640 1 --2- 172.21.15.151:0/895338773 >> [v2:172.21.15.167:6802/265072227,v1:172.21.15.167:6803/265072227] conn(0x7f6220080f30 0x7f62200833d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:45.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.527+0000 7f623fe00640 1 --2- 172.21.15.151:0/895338773 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f622007afb0 0x7f622007d470 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f6230033000 tx=0x7f6230004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:45.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.527+0000 7f6245000640 1 --2- 172.21.15.151:0/895338773 >> [v2:172.21.15.167:6802/265072227,v1:172.21.15.167:6803/265072227] conn(0x7f6220080f30 0x7f62200833d0 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:45.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.528+0000 7f6237e00640 1 -- 172.21.15.151:0/895338773 <== osd.7 v2:172.21.15.167:6802/265072227 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6220083aa0 con 0x7f6220080f30 2024-09-16T15:56:45.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.541+0000 7f6247000640 1 -- 172.21.15.151:0/895338773 --> [v2:172.21.15.167:6802/265072227,v1:172.21.15.167:6803/265072227] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f624015cd30 con 0x7f6220080f30 2024-09-16T15:56:45.543 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.541+0000 7f6237e00640 1 -- 172.21.15.151:0/895338773 <== osd.7 v2:172.21.15.167:6802/265072227 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f624015cd30 con 0x7f6220080f30 2024-09-16T15:56:45.544 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-16T15:56:45.544 INFO:teuthology.orchestra.run.smithi151.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T15:56:45.544 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-16T15:56:45.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.542+0000 7f6247000640 1 -- 172.21.15.151:0/895338773 >> [v2:172.21.15.167:6802/265072227,v1:172.21.15.167:6803/265072227] conn(0x7f6220080f30 msgr2=0x7f62200833d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:45.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.542+0000 7f6247000640 1 --2- 172.21.15.151:0/895338773 >> [v2:172.21.15.167:6802/265072227,v1:172.21.15.167:6803/265072227] conn(0x7f6220080f30 0x7f62200833d0 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:45.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.542+0000 7f6247000640 1 -- 172.21.15.151:0/895338773 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f622007afb0 msgr2=0x7f622007d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:45.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.542+0000 7f6247000640 1 --2- 172.21.15.151:0/895338773 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f622007afb0 0x7f622007d470 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f6230033000 tx=0x7f6230004730 comp rx=0 tx=0).stop 2024-09-16T15:56:45.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.542+0000 7f6247000640 1 -- 172.21.15.151:0/895338773 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f62400b4770 msgr2=0x7f6240160c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:45.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.542+0000 7f6247000640 1 --2- 172.21.15.151:0/895338773 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f62400b4770 0x7f6240160c80 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f622c00b4a0 tx=0x7f622c00b960 comp rx=0 tx=0).stop 2024-09-16T15:56:45.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.542+0000 7f6245000640 1 -- 172.21.15.151:0/895338773 reap_dead start 2024-09-16T15:56:45.545 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.542+0000 7f6247000640 1 -- 172.21.15.151:0/895338773 shutdown_connections 2024-09-16T15:56:45.546 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.542+0000 7f6247000640 1 -- 172.21.15.151:0/895338773 >> 172.21.15.151:0/895338773 conn(0x7f62400b18e0 msgr2=0x7f62400b39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:45.546 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.542+0000 7f6247000640 1 -- 172.21.15.151:0/895338773 shutdown_connections 2024-09-16T15:56:45.546 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.542+0000 7f6247000640 1 -- 172.21.15.151:0/895338773 wait complete. 2024-09-16T15:56:45.559 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-16T15:56:45.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.718+0000 7fee15c00640 1 -- 172.21.15.151:0/150062843 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fedf40051c0 msgr2=0x7fedf400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:45.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.718+0000 7fee15c00640 1 --2- 172.21.15.151:0/150062843 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fedf40051c0 0x7fedf400b9c0 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7fee08001f00 tx=0x7fedf4005830 comp rx=0 tx=0).stop 2024-09-16T15:56:45.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.718+0000 7fee15200640 1 -- 172.21.15.151:0/150062843 reap_dead start 2024-09-16T15:56:45.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.718+0000 7fee15c00640 1 -- 172.21.15.151:0/150062843 shutdown_connections 2024-09-16T15:56:45.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.718+0000 7fee15c00640 1 -- 172.21.15.151:0/150062843 >> 172.21.15.151:0/150062843 conn(0x7fee100d3df0 msgr2=0x7fee100d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:45.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.718+0000 7fee15c00640 1 -- 172.21.15.151:0/150062843 shutdown_connections 2024-09-16T15:56:45.721 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.718+0000 7fee15c00640 1 -- 172.21.15.151:0/150062843 wait complete. 2024-09-16T15:56:45.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.720+0000 7fee15c00640 1 Processor -- start 2024-09-16T15:56:45.722 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.720+0000 7fee15c00640 1 -- start start 2024-09-16T15:56:45.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.721+0000 7fee15c00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fedf40051c0 0x7fee101f84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:45.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.721+0000 7fee15c00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee101f44c0 0x7fee100c88a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:45.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.721+0000 7fee15c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee100d0600 0x7fee10156d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:45.724 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.721+0000 7fee15c00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fee100d2780 con 0x7fedf40051c0 2024-09-16T15:56:45.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.721+0000 7fee15c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fee100d2600 con 0x7fee100d0600 2024-09-16T15:56:45.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.721+0000 7fee15c00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7fee100b1ad0 con 0x7fee101f44c0 2024-09-16T15:56:45.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.721+0000 7fee0fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fedf40051c0 0x7fee101f84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:45.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.721+0000 7fee0fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fedf40051c0 0x7fee101f84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36098/0 (socket says 172.21.15.151:36098) 2024-09-16T15:56:45.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.721+0000 7fee0fe00640 1 -- 172.21.15.151:0/2697332183 learned_addr learned my addr 172.21.15.151:0/2697332183 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:56:45.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.722+0000 7fee15200640 1 --2- 172.21.15.151:0/2697332183 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee100d0600 0x7fee10156d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:45.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.722+0000 7fee0f400640 1 --2- 172.21.15.151:0/2697332183 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee101f44c0 0x7fee100c88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:45.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.722+0000 7fee0fe00640 1 -- 172.21.15.151:0/2697332183 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee101f44c0 msgr2=0x7fee100c88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:45.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.722+0000 7fee0fe00640 1 --2- 172.21.15.151:0/2697332183 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee101f44c0 0x7fee100c88a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:45.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.722+0000 7fee0fe00640 1 -- 172.21.15.151:0/2697332183 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee100d0600 msgr2=0x7fee10156d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:45.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.722+0000 7fee0fe00640 1 --2- 172.21.15.151:0/2697332183 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee100d0600 0x7fee10156d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:45.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.722+0000 7fee0fe00640 1 -- 172.21.15.151:0/2697332183 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee101f4c30 con 0x7fedf40051c0 2024-09-16T15:56:45.725 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.722+0000 7fee0fe00640 1 --2- 172.21.15.151:0/2697332183 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fedf40051c0 0x7fee101f84c0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7fee00011900 tx=0x7fee00011dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:45.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.722+0000 7fee07e00640 1 -- 172.21.15.151:0/2697332183 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee0000fe10 con 0x7fedf40051c0 2024-09-16T15:56:45.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.722+0000 7fee07e00640 1 -- 172.21.15.151:0/2697332183 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fee00004030 con 0x7fedf40051c0 2024-09-16T15:56:45.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.722+0000 7fee07e00640 1 -- 172.21.15.151:0/2697332183 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee00004480 con 0x7fedf40051c0 2024-09-16T15:56:45.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.722+0000 7fee15c00640 1 -- 172.21.15.151:0/2697332183 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee101f4f10 con 0x7fedf40051c0 2024-09-16T15:56:45.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.722+0000 7fee15c00640 1 -- 172.21.15.151:0/2697332183 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee101588b0 con 0x7fedf40051c0 2024-09-16T15:56:45.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.723+0000 7fee15c00640 1 -- 172.21.15.151:0/2697332183 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee100d1c60 con 0x7fedf40051c0 2024-09-16T15:56:45.726 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.725+0000 7fee07e00640 1 -- 172.21.15.151:0/2697332183 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 55) ==== 104857+0+0 (secure 0 0 0) 0x7fee000035f0 con 0x7fedf40051c0 2024-09-16T15:56:45.727 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.725+0000 7fee07e00640 1 --2- 172.21.15.151:0/2697332183 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fedf007afe0 0x7fedf007d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:45.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.726+0000 7fee07e00640 1 -- 172.21.15.151:0/2697332183 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5083+0+0 (secure 0 0 0) 0x7fee0009aa80 con 0x7fedf40051c0 2024-09-16T15:56:45.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.726+0000 7fee0f400640 1 --2- 172.21.15.151:0/2697332183 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fedf007afe0 0x7fedf007d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:45.728 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.727+0000 7fee0f400640 1 --2- 172.21.15.151:0/2697332183 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fedf007afe0 0x7fedf007d4a0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fedfc001eb0 tx=0x7fedfc000f10 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:45.731 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.729+0000 7fee07e00640 1 -- 172.21.15.151:0/2697332183 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7fee00064b50 con 0x7fedf40051c0 2024-09-16T15:56:45.916 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.914+0000 7fee15c00640 1 -- 172.21.15.151:0/2697332183 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7fee1015f950 con 0x7fedf007afe0 2024-09-16T15:56:45.917 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.915+0000 7fee07e00640 1 -- 172.21.15.151:0/2697332183 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fee1015f950 con 0x7fedf007afe0 2024-09-16T15:56:45.917 INFO:teuthology.orchestra.run.smithi151.stderr:instructed osd(s) 7 to deep-scrub 2024-09-16T15:56:45.920 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.918+0000 7fee15c00640 1 -- 172.21.15.151:0/2697332183 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fedf007afe0 msgr2=0x7fedf007d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:45.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.918+0000 7fee15c00640 1 --2- 172.21.15.151:0/2697332183 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fedf007afe0 0x7fedf007d4a0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fedfc001eb0 tx=0x7fedfc000f10 comp rx=0 tx=0).stop 2024-09-16T15:56:45.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.918+0000 7fee15c00640 1 -- 172.21.15.151:0/2697332183 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fedf40051c0 msgr2=0x7fee101f84c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:45.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.918+0000 7fee15c00640 1 --2- 172.21.15.151:0/2697332183 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fedf40051c0 0x7fee101f84c0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7fee00011900 tx=0x7fee00011dc0 comp rx=0 tx=0).stop 2024-09-16T15:56:45.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.918+0000 7fee15c00640 1 -- 172.21.15.151:0/2697332183 shutdown_connections 2024-09-16T15:56:45.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.918+0000 7fee15c00640 1 --2- 172.21.15.151:0/2697332183 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7fedf007afe0 0x7fedf007d4a0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:45.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.918+0000 7fee15c00640 1 --2- 172.21.15.151:0/2697332183 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee100d0600 0x7fee10156d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:45.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.918+0000 7fee15c00640 1 --2- 172.21.15.151:0/2697332183 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee101f44c0 0x7fee100c88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:45.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.918+0000 7fee15c00640 1 --2- 172.21.15.151:0/2697332183 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fedf40051c0 0x7fee101f84c0 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:45.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.918+0000 7fee15c00640 1 -- 172.21.15.151:0/2697332183 >> 172.21.15.151:0/2697332183 conn(0x7fee100d3df0 msgr2=0x7fee100ceff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:45.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.918+0000 7fee15c00640 1 -- 172.21.15.151:0/2697332183 shutdown_connections 2024-09-16T15:56:45.921 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:45.918+0000 7fee15c00640 1 -- 172.21.15.151:0/2697332183 wait complete. 2024-09-16T15:56:45.937 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:56:46.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:46.093+0000 7f2c5fc00640 1 -- 172.21.15.151:0/307125998 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c58073fb0 msgr2=0x7f2c58074420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:46.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:46.093+0000 7f2c5fc00640 1 --2- 172.21.15.151:0/307125998 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c58073fb0 0x7f2c58074420 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f2c58075130 tx=0x7f2c5002aa30 comp rx=0 tx=0).stop 2024-09-16T15:56:46.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:46.093+0000 7f2c5fc00640 1 -- 172.21.15.151:0/307125998 shutdown_connections 2024-09-16T15:56:46.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:46.093+0000 7f2c5fc00640 1 --2- 172.21.15.151:0/307125998 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c58073fb0 0x7f2c58074420 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:46.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:46.093+0000 7f2c5fc00640 1 --2- 172.21.15.151:0/307125998 >> v2:172.21.15.167:3300/0 conn(0x7f2c5815f5a0 0x7f2c580b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:46.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:46.093+0000 7f2c5fc00640 1 -- 172.21.15.151:0/307125998 >> 172.21.15.151:0/307125998 conn(0x7f2c580b1950 msgr2=0x7f2c580b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:46.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:46.093+0000 7f2c5fc00640 1 -- 172.21.15.151:0/307125998 shutdown_connections 2024-09-16T15:56:46.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:46.093+0000 7f2c5fc00640 1 -- 172.21.15.151:0/307125998 wait complete. 2024-09-16T15:56:46.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:46.095+0000 7f2c5fc00640 1 Processor -- start 2024-09-16T15:56:46.097 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:46.095+0000 7f2c5fc00640 1 -- start start 2024-09-16T15:56:46.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:46.096+0000 7f2c5fc00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c58073fb0 0x7f2c5809d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:46.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:46.096+0000 7f2c5fc00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c580ccf00 0x7f2c581f6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:46.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:46.096+0000 7f2c5fc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c5815f5a0 0x7f2c5809c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:46.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:46.096+0000 7f2c5fc00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5806f2e0 con 0x7f2c5815f5a0 2024-09-16T15:56:46.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:46.096+0000 7f2c5fc00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5806f160 con 0x7f2c580ccf00 2024-09-16T15:56:46.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:46.096+0000 7f2c5fc00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f2c5806f460 con 0x7f2c58073fb0 2024-09-16T15:56:46.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:46.096+0000 7f2c5dc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c5815f5a0 0x7f2c5809c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:46.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:46.096+0000 7f2c5dc00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c5815f5a0 0x7f2c5809c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:36108/0 (socket says 172.21.15.151:36108) 2024-09-16T15:56:46.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:46.096+0000 7f2c5dc00640 1 -- 172.21.15.151:0/299851428 learned_addr learned my addr 172.21.15.151:0/299851428 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:56:46.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:46.097+0000 7f2c57e00640 1 --2- 172.21.15.151:0/299851428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c580ccf00 0x7f2c581f6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:46.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:46.097+0000 7f2c5dc00640 1 -- 172.21.15.151:0/299851428 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c58073fb0 msgr2=0x7f2c5809d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:46.099 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:46.097+0000 7f2c5dc00640 1 --2- 172.21.15.151:0/299851428 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c58073fb0 0x7f2c5809d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:46.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:46.097+0000 7f2c5dc00640 1 -- 172.21.15.151:0/299851428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c580ccf00 msgr2=0x7f2c581f6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:46.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:46.097+0000 7f2c5dc00640 1 --2- 172.21.15.151:0/299851428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c580ccf00 0x7f2c581f6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:46.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:46.097+0000 7f2c5dc00640 1 -- 172.21.15.151:0/299851428 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c58156090 con 0x7f2c5815f5a0 2024-09-16T15:56:46.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:46.097+0000 7f2c57e00640 1 --2- 172.21.15.151:0/299851428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c580ccf00 0x7f2c581f6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T15:56:46.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:46.097+0000 7f2c5dc00640 1 --2- 172.21.15.151:0/299851428 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c5815f5a0 0x7f2c5809c260 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f2c50009d60 tx=0x7f2c50003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:46.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:46.097+0000 7f2c54c00640 1 -- 172.21.15.151:0/299851428 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c50047070 con 0x7f2c5815f5a0 2024-09-16T15:56:46.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:46.097+0000 7f2c54c00640 1 -- 172.21.15.151:0/299851428 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2c50004250 con 0x7f2c5815f5a0 2024-09-16T15:56:46.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:46.097+0000 7f2c54c00640 1 -- 172.21.15.151:0/299851428 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c50033050 con 0x7f2c5815f5a0 2024-09-16T15:56:46.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:46.097+0000 7f2c5fc00640 1 -- 172.21.15.151:0/299851428 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c58156370 con 0x7f2c5815f5a0 2024-09-16T15:56:46.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:46.097+0000 7f2c5fc00640 1 -- 172.21.15.151:0/299851428 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c58158010 con 0x7f2c5815f5a0 2024-09-16T15:56:46.101 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:46.099+0000 7f2c5fc00640 1 -- 172.21.15.151:0/299851428 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c5806f9f0 con 0x7f2c5815f5a0 2024-09-16T15:56:46.102 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:46.100+0000 7f2c54c00640 1 -- 172.21.15.151:0/299851428 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 55) ==== 104857+0+0 (secure 0 0 0) 0x7f2c5004a070 con 0x7f2c5815f5a0 2024-09-16T15:56:46.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:46.101+0000 7f2c54c00640 1 --2- 172.21.15.151:0/299851428 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2c3807afb0 0x7f2c3807d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:56:46.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:46.101+0000 7f2c54c00640 1 -- 172.21.15.151:0/299851428 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5083+0+0 (secure 0 0 0) 0x7f2c500baef0 con 0x7f2c5815f5a0 2024-09-16T15:56:46.103 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:46.101+0000 7f2c5d200640 1 --2- 172.21.15.151:0/299851428 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2c3807afb0 0x7f2c3807d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:56:46.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:46.102+0000 7f2c5d200640 1 --2- 172.21.15.151:0/299851428 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2c3807afb0 0x7f2c3807d470 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f2c480058c0 tx=0x7f2c48005850 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:56:46.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:46.105+0000 7f2c54c00640 1 -- 172.21.15.151:0/299851428 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f2c50085060 con 0x7f2c5815f5a0 2024-09-16T15:56:46.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:46.289+0000 7f2c5fc00640 1 -- 172.21.15.151:0/299851428 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2c581581e0 con 0x7f2c3807afb0 2024-09-16T15:56:46.291 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:46.290+0000 7f2c54c00640 1 -- 172.21.15.151:0/299851428 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13384 (secure 0 0 0) 0x7f2c581581e0 con 0x7f2c3807afb0 2024-09-16T15:56:46.292 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:56:46.292 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-09-16T15:56:46.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:46.292+0000 7f2c5fc00640 1 -- 172.21.15.151:0/299851428 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2c3807afb0 msgr2=0x7f2c3807d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:46.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:46.292+0000 7f2c5fc00640 1 --2- 172.21.15.151:0/299851428 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2c3807afb0 0x7f2c3807d470 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f2c480058c0 tx=0x7f2c48005850 comp rx=0 tx=0).stop 2024-09-16T15:56:46.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:46.292+0000 7f2c5fc00640 1 -- 172.21.15.151:0/299851428 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c5815f5a0 msgr2=0x7f2c5809c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:56:46.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:46.292+0000 7f2c5fc00640 1 --2- 172.21.15.151:0/299851428 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c5815f5a0 0x7f2c5809c260 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f2c50009d60 tx=0x7f2c50003f10 comp rx=0 tx=0).stop 2024-09-16T15:56:46.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:46.292+0000 7f2c5fc00640 1 -- 172.21.15.151:0/299851428 shutdown_connections 2024-09-16T15:56:46.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:46.292+0000 7f2c5fc00640 1 --2- 172.21.15.151:0/299851428 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f2c3807afb0 0x7f2c3807d470 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:46.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:46.292+0000 7f2c5fc00640 1 --2- 172.21.15.151:0/299851428 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2c580ccf00 0x7f2c581f6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:46.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:46.292+0000 7f2c5fc00640 1 --2- 172.21.15.151:0/299851428 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2c5815f5a0 0x7f2c5809c260 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:46.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:46.292+0000 7f2c5fc00640 1 --2- 172.21.15.151:0/299851428 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2c58073fb0 0x7f2c5809d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:56:46.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:46.292+0000 7f2c5fc00640 1 -- 172.21.15.151:0/299851428 >> 172.21.15.151:0/299851428 conn(0x7f2c580b1950 msgr2=0x7f2c580b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:56:46.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:46.292+0000 7f2c5fc00640 1 -- 172.21.15.151:0/299851428 shutdown_connections 2024-09-16T15:56:46.295 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:56:46.292+0000 7f2c5fc00640 1 -- 172.21.15.151:0/299851428 wait complete. 2024-09-16T15:56:46.311 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":673,"stamp":"2024-09-16T15:56:46.008279+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":1239548,"kb_used_data":16008,"kb_used_omap":12,"kb_used_meta":1223347,"kb_avail":753735172,"statfs":{"total":773094113280,"available":771824816128,"internally_reserved":0,"allocated":16392192,"data_stored":7933248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12738,"internal_metadata":1252707902},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.132163"},"pg_stats":[{"pgid":"1.0","version":"17'72","reported_seq":360,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T15:56:36.168763+0000","last_change":"2024-09-16T15:39:12.698813+0000","last_active":"2024-09-16T15:56:36.168763+0000","last_peered":"2024-09-16T15:56:36.168763+0000","last_clean":"2024-09-16T15:56:36.168763+0000","last_became_active":"2024-09-16T15:39:12.697695+0000","last_became_peered":"2024-09-16T15:39:12.697695+0000","last_unstale":"2024-09-16T15:56:36.168763+0000","last_undegraded":"2024-09-16T15:56:36.168763+0000","last_fullsized":"2024-09-16T15:56:36.168763+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T15:39:11.684967+0000","last_clean_scrub_stamp":"2024-09-16T15:39:11.684967+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:12:02.061081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509656,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":171800,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":169918,"kb_avail":94200040,"statfs":{"total":96636764160,"available":96460840960,"internally_reserved":0,"allocated":1900544,"data_stored":844064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":173996488},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509655,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":150164,"kb_used_data":2436,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94221676,"statfs":{"total":96636764160,"available":96482996224,"internally_reserved":0,"allocated":2494464,"data_stored":1434432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":151255495},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509654,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":167640,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":165758,"kb_avail":94204200,"statfs":{"total":96636764160,"available":96465100800,"internally_reserved":0,"allocated":1900544,"data_stored":844064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":169736645},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509654,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":149592,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222248,"statfs":{"total":96636764160,"available":96483581952,"internally_reserved":0,"allocated":1900544,"data_stored":844064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":151255497},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509654,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":149588,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222252,"statfs":{"total":96636764160,"available":96483586048,"internally_reserved":0,"allocated":1900544,"data_stored":844064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":151255495},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509654,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":141852,"kb_used_data":2436,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94229988,"statfs":{"total":96636764160,"available":96491507712,"internally_reserved":0,"allocated":2494464,"data_stored":1434432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":142735815},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509652,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":163480,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":161598,"kb_avail":94208360,"statfs":{"total":96636764160,"available":96469360640,"internally_reserved":0,"allocated":1900544,"data_stored":844064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":165476807},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542360,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":145432,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226408,"statfs":{"total":96636764160,"available":96487841792,"internally_reserved":0,"allocated":1900544,"data_stored":844064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":146995660},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:56:46.314 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-16T15:39:11.684967+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=15, tm_min=39, tm_sec=11, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=15, tm_min=56, tm_sec=40, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T15:56:46.314 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-16T15:56:52.758 DEBUG:teuthology.orchestra.run.smithi151:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:56:52.763 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:57:06.315 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T15:57:06.485 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.484+0000 7efe37a00640 1 -- 172.21.15.151:0/981938740 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe300ccf00 msgr2=0x7efe300b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:57:06.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.484+0000 7efe37a00640 1 --2- 172.21.15.151:0/981938740 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe300ccf00 0x7efe300b2440 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7efe3006dac0 tx=0x7efe1c02a560 comp rx=0 tx=0).stop 2024-09-16T15:57:06.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.484+0000 7efe37a00640 1 -- 172.21.15.151:0/981938740 shutdown_connections 2024-09-16T15:57:06.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.484+0000 7efe37a00640 1 --2- 172.21.15.151:0/981938740 >> v2:172.21.15.167:3300/0 conn(0x7efe30073fb0 0x7efe30074420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:57:06.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.484+0000 7efe37a00640 1 --2- 172.21.15.151:0/981938740 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe300ccf00 0x7efe300b2440 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:57:06.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.484+0000 7efe37a00640 1 -- 172.21.15.151:0/981938740 >> 172.21.15.151:0/981938740 conn(0x7efe300b1950 msgr2=0x7efe300b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:57:06.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.484+0000 7efe37a00640 1 -- 172.21.15.151:0/981938740 shutdown_connections 2024-09-16T15:57:06.486 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.484+0000 7efe37a00640 1 -- 172.21.15.151:0/981938740 wait complete. 2024-09-16T15:57:06.487 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.486+0000 7efe37a00640 1 Processor -- start 2024-09-16T15:57:06.487 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.486+0000 7efe37a00640 1 -- start start 2024-09-16T15:57:06.488 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.487+0000 7efe37a00640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efe30073fb0 0x7efe3009d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:57:06.488 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.487+0000 7efe37a00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe300ccf00 0x7efe301f47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:57:06.488 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.487+0000 7efe37a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe3015f5a0 0x7efe301f6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:57:06.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.487+0000 7efe37a00640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efe3006f2e0 con 0x7efe300ccf00 2024-09-16T15:57:06.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.487+0000 7efe37a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efe3006f160 con 0x7efe3015f5a0 2024-09-16T15:57:06.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.487+0000 7efe37a00640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7efe3006f460 con 0x7efe30073fb0 2024-09-16T15:57:06.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.487+0000 7efe2fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe300ccf00 0x7efe301f47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:57:06.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.487+0000 7efe2fe00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe300ccf00 0x7efe301f47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:35274/0 (socket says 172.21.15.151:35274) 2024-09-16T15:57:06.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.487+0000 7efe2fe00640 1 -- 172.21.15.151:0/457493616 learned_addr learned my addr 172.21.15.151:0/457493616 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:57:06.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.487+0000 7efe35000640 1 --2- 172.21.15.151:0/457493616 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efe30073fb0 0x7efe3009d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:57:06.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.487+0000 7efe35a00640 1 --2- 172.21.15.151:0/457493616 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe3015f5a0 0x7efe301f6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:57:06.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.488+0000 7efe2fe00640 1 -- 172.21.15.151:0/457493616 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efe30073fb0 msgr2=0x7efe3009d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:57:06.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.488+0000 7efe2fe00640 1 --2- 172.21.15.151:0/457493616 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efe30073fb0 0x7efe3009d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:57:06.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.488+0000 7efe2fe00640 1 -- 172.21.15.151:0/457493616 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe3015f5a0 msgr2=0x7efe301f6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:57:06.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.488+0000 7efe2fe00640 1 --2- 172.21.15.151:0/457493616 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe3015f5a0 0x7efe301f6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:57:06.489 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.488+0000 7efe2fe00640 1 -- 172.21.15.151:0/457493616 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe301f51d0 con 0x7efe300ccf00 2024-09-16T15:57:06.490 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.488+0000 7efe2fe00640 1 --2- 172.21.15.151:0/457493616 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe300ccf00 0x7efe301f47f0 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7efe1c02ab50 tx=0x7efe1c0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:57:06.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.488+0000 7efe2cc00640 1 -- 172.21.15.151:0/457493616 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe1c03f070 con 0x7efe300ccf00 2024-09-16T15:57:06.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.488+0000 7efe2cc00640 1 -- 172.21.15.151:0/457493616 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efe1c004890 con 0x7efe300ccf00 2024-09-16T15:57:06.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.488+0000 7efe2cc00640 1 -- 172.21.15.151:0/457493616 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe1c035030 con 0x7efe300ccf00 2024-09-16T15:57:06.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.488+0000 7efe37a00640 1 -- 172.21.15.151:0/457493616 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe301f54b0 con 0x7efe300ccf00 2024-09-16T15:57:06.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.488+0000 7efe37a00640 1 -- 172.21.15.151:0/457493616 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe3009d710 con 0x7efe300ccf00 2024-09-16T15:57:06.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.489+0000 7efe37a00640 1 -- 172.21.15.151:0/457493616 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe3006f9f0 con 0x7efe300ccf00 2024-09-16T15:57:06.491 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.491+0000 7efe2cc00640 1 -- 172.21.15.151:0/457493616 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 55) ==== 104857+0+0 (secure 0 0 0) 0x7efe1c042070 con 0x7efe300ccf00 2024-09-16T15:57:06.492 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.492+0000 7efe2cc00640 1 --2- 172.21.15.151:0/457493616 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efe1007afb0 0x7efe1007d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:57:06.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.492+0000 7efe2cc00640 1 -- 172.21.15.151:0/457493616 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5083+0+0 (secure 0 0 0) 0x7efe1c0bab40 con 0x7efe300ccf00 2024-09-16T15:57:06.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.492+0000 7efe35000640 1 --2- 172.21.15.151:0/457493616 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efe1007afb0 0x7efe1007d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:57:06.493 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.493+0000 7efe35000640 1 --2- 172.21.15.151:0/457493616 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efe1007afb0 0x7efe1007d470 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7efe301f8e90 tx=0x7efe20009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:57:06.497 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.496+0000 7efe2cc00640 1 -- 172.21.15.151:0/457493616 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7efe1c0845c0 con 0x7efe300ccf00 2024-09-16T15:57:06.680 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.679+0000 7efe37a00640 1 -- 172.21.15.151:0/457493616 --> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efe30069ac0 con 0x7efe1007afb0 2024-09-16T15:57:06.681 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.680+0000 7efe2cc00640 1 -- 172.21.15.151:0/457493616 <== mgr.4123 v2:172.21.15.167:6800/501171453 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13391 (secure 0 0 0) 0x7efe30069ac0 con 0x7efe1007afb0 2024-09-16T15:57:06.681 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:57:06.681 INFO:teuthology.orchestra.run.smithi151.stderr:dumped all 2024-09-16T15:57:06.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.682+0000 7efe37a00640 1 -- 172.21.15.151:0/457493616 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efe1007afb0 msgr2=0x7efe1007d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:57:06.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.682+0000 7efe37a00640 1 --2- 172.21.15.151:0/457493616 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efe1007afb0 0x7efe1007d470 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7efe301f8e90 tx=0x7efe20009040 comp rx=0 tx=0).stop 2024-09-16T15:57:06.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.682+0000 7efe37a00640 1 -- 172.21.15.151:0/457493616 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe300ccf00 msgr2=0x7efe301f47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:57:06.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.682+0000 7efe37a00640 1 --2- 172.21.15.151:0/457493616 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe300ccf00 0x7efe301f47f0 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7efe1c02ab50 tx=0x7efe1c0039e0 comp rx=0 tx=0).stop 2024-09-16T15:57:06.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.682+0000 7efe37a00640 1 -- 172.21.15.151:0/457493616 shutdown_connections 2024-09-16T15:57:06.683 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.682+0000 7efe37a00640 1 --2- 172.21.15.151:0/457493616 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7efe1007afb0 0x7efe1007d470 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:57:06.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.682+0000 7efe37a00640 1 --2- 172.21.15.151:0/457493616 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe3015f5a0 0x7efe301f6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:57:06.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.682+0000 7efe37a00640 1 --2- 172.21.15.151:0/457493616 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe300ccf00 0x7efe301f47f0 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:57:06.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.682+0000 7efe37a00640 1 --2- 172.21.15.151:0/457493616 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7efe30073fb0 0x7efe3009d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:57:06.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.682+0000 7efe37a00640 1 -- 172.21.15.151:0/457493616 >> 172.21.15.151:0/457493616 conn(0x7efe300b1950 msgr2=0x7efe300b0770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:57:06.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.682+0000 7efe37a00640 1 -- 172.21.15.151:0/457493616 shutdown_connections 2024-09-16T15:57:06.684 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.682+0000 7efe37a00640 1 -- 172.21.15.151:0/457493616 wait complete. 2024-09-16T15:57:06.699 INFO:teuthology.orchestra.run.smithi151.stdout:{"pg_ready":true,"pg_map":{"version":683,"stamp":"2024-09-16T15:57:06.012668+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":1239540,"kb_used_data":16008,"kb_used_omap":12,"kb_used_meta":1223347,"kb_avail":753735180,"statfs":{"total":773094113280,"available":771824824320,"internally_reserved":0,"allocated":16392192,"data_stored":7933248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12738,"internal_metadata":1252707902},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002611"},"pg_stats":[{"pgid":"1.0","version":"152'74","reported_seq":370,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-16T15:56:42.711682+0000","last_change":"2024-09-16T15:56:42.711682+0000","last_active":"2024-09-16T15:56:42.711682+0000","last_peered":"2024-09-16T15:56:42.711682+0000","last_clean":"2024-09-16T15:56:42.711682+0000","last_became_active":"2024-09-16T15:39:12.697695+0000","last_became_peered":"2024-09-16T15:39:12.697695+0000","last_unstale":"2024-09-16T15:56:42.711682+0000","last_undegraded":"2024-09-16T15:56:42.711682+0000","last_fullsized":"2024-09-16T15:56:42.711682+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"152'74","last_scrub_stamp":"2024-09-16T15:56:42.711601+0000","last_deep_scrub":"152'74","last_deep_scrub_stamp":"2024-09-16T15:56:42.711601+0000","last_clean_scrub_stamp":"2024-09-16T15:56:42.711601+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:31:43.001406+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509660,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":171800,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":169918,"kb_avail":94200040,"statfs":{"total":96636764160,"available":96460840960,"internally_reserved":0,"allocated":1900544,"data_stored":844064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":173996488},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509659,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":150164,"kb_used_data":2436,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94221676,"statfs":{"total":96636764160,"available":96482996224,"internally_reserved":0,"allocated":2494464,"data_stored":1434432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":151255495},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509658,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":167640,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":165758,"kb_avail":94204200,"statfs":{"total":96636764160,"available":96465100800,"internally_reserved":0,"allocated":1900544,"data_stored":844064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":169736645},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509658,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":149592,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222248,"statfs":{"total":96636764160,"available":96483581952,"internally_reserved":0,"allocated":1900544,"data_stored":844064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":151255497},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509658,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":149588,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":94222252,"statfs":{"total":96636764160,"available":96483586048,"internally_reserved":0,"allocated":1900544,"data_stored":844064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":151255495},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509658,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":141844,"kb_used_data":2436,"kb_used_omap":1,"kb_used_meta":139390,"kb_avail":94229996,"statfs":{"total":96636764160,"available":96491515904,"internally_reserved":0,"allocated":2494464,"data_stored":1434432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":142735815},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509656,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":163480,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":161598,"kb_avail":94208360,"statfs":{"total":96636764160,"available":96469360640,"internally_reserved":0,"allocated":1900544,"data_stored":844064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":165476807},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542364,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":145432,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":143550,"kb_avail":94226408,"statfs":{"total":96636764160,"available":96487841792,"internally_reserved":0,"allocated":1900544,"data_stored":844064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1588,"internal_metadata":146995660},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T15:57:06.700 DEBUG:teuthology.orchestra.run.smithi151:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-16T15:57:06.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.854+0000 7f584e800640 1 -- 172.21.15.151:0/3805408707 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f58480cbc00 msgr2=0x7f5848073ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:57:06.855 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.854+0000 7f584e800640 1 --2- 172.21.15.151:0/3805408707 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f58480cbc00 0x7f5848073ef0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5840005fd0 tx=0x7f584002a8b0 comp rx=0 tx=0).stop 2024-09-16T15:57:06.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.854+0000 7f584e800640 1 -- 172.21.15.151:0/3805408707 shutdown_connections 2024-09-16T15:57:06.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.854+0000 7f584e800640 1 --2- 172.21.15.151:0/3805408707 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f58480cbc00 0x7f5848073ef0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:57:06.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.854+0000 7f584e800640 1 --2- 172.21.15.151:0/3805408707 >> v2:172.21.15.167:3300/0 conn(0x7f584815bec0 0x7f584806fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:57:06.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.854+0000 7f584e800640 1 -- 172.21.15.151:0/3805408707 >> 172.21.15.151:0/3805408707 conn(0x7f58480cf9f0 msgr2=0x7f58480cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:57:06.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.854+0000 7f584e800640 1 -- 172.21.15.151:0/3805408707 shutdown_connections 2024-09-16T15:57:06.856 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.854+0000 7f584e800640 1 -- 172.21.15.151:0/3805408707 wait complete. 2024-09-16T15:57:06.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.856+0000 7f584e800640 1 Processor -- start 2024-09-16T15:57:06.857 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.856+0000 7f584e800640 1 -- start start 2024-09-16T15:57:06.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.857+0000 7f584e800640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5848072560 0x7f58480b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:57:06.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.857+0000 7f584e800640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f58480cbc00 0x7f5848152f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:57:06.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.857+0000 7f584e800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f584815bec0 0x7f58481555a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:57:06.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.857+0000 7f584e800640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f58480ae930 con 0x7f5848072560 2024-09-16T15:57:06.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.857+0000 7f584e800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f58480ae7b0 con 0x7f584815bec0 2024-09-16T15:57:06.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.857+0000 7f584e800640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f58480aeab0 con 0x7f58480cbc00 2024-09-16T15:57:06.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.857+0000 7f584d400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5848072560 0x7f58480b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:57:06.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.857+0000 7f584d400640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5848072560 0x7f58480b3d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:35284/0 (socket says 172.21.15.151:35284) 2024-09-16T15:57:06.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.857+0000 7f584d400640 1 -- 172.21.15.151:0/2560155469 learned_addr learned my addr 172.21.15.151:0/2560155469 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:57:06.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.858+0000 7f584d400640 1 -- 172.21.15.151:0/2560155469 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f58480cbc00 msgr2=0x7f5848152f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:57:06.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.858+0000 7f584d400640 1 --2- 172.21.15.151:0/2560155469 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f58480cbc00 0x7f5848152f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:57:06.859 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.858+0000 7f584d400640 1 -- 172.21.15.151:0/2560155469 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f584815bec0 msgr2=0x7f58481555a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:57:06.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.858+0000 7f584d400640 1 --2- 172.21.15.151:0/2560155469 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f584815bec0 0x7f58481555a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:57:06.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.858+0000 7f584d400640 1 -- 172.21.15.151:0/2560155469 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5848153880 con 0x7f5848072560 2024-09-16T15:57:06.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.858+0000 7f584d400640 1 --2- 172.21.15.151:0/2560155469 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5848072560 0x7f58480b3d20 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f583c00e7c0 tx=0x7f583c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:57:06.860 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.858+0000 7f5845600640 1 -- 172.21.15.151:0/2560155469 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f583c00ccc0 con 0x7f5848072560 2024-09-16T15:57:06.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.858+0000 7f5845600640 1 -- 172.21.15.151:0/2560155469 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f583c00ce60 con 0x7f5848072560 2024-09-16T15:57:06.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.858+0000 7f5845600640 1 -- 172.21.15.151:0/2560155469 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f583c005280 con 0x7f5848072560 2024-09-16T15:57:06.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.858+0000 7f584e800640 1 -- 172.21.15.151:0/2560155469 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5848153b60 con 0x7f5848072560 2024-09-16T15:57:06.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.858+0000 7f584e800640 1 -- 172.21.15.151:0/2560155469 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5848154120 con 0x7f5848072560 2024-09-16T15:57:06.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.859+0000 7f584e800640 1 -- 172.21.15.151:0/2560155469 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58480aefd0 con 0x7f5848072560 2024-09-16T15:57:06.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.861+0000 7f5845600640 1 -- 172.21.15.151:0/2560155469 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 55) ==== 104857+0+0 (secure 0 0 0) 0x7f583c005420 con 0x7f5848072560 2024-09-16T15:57:06.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.862+0000 7f5845600640 1 --2- 172.21.15.151:0/2560155469 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f582807afb0 0x7f582807d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:57:06.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.862+0000 7f5845600640 1 -- 172.21.15.151:0/2560155469 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5083+0+0 (secure 0 0 0) 0x7f583c09a650 con 0x7f5848072560 2024-09-16T15:57:06.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.862+0000 7f584ca00640 1 --2- 172.21.15.151:0/2560155469 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f582807afb0 0x7f582807d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:57:06.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.863+0000 7f584ca00640 1 --2- 172.21.15.151:0/2560155469 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f582807afb0 0x7f582807d470 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f5834000f60 tx=0x7f58340079b0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:57:06.867 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:06.866+0000 7f5845600640 1 -- 172.21.15.151:0/2560155469 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f583c064720 con 0x7f5848072560 2024-09-16T15:57:07.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:07.052+0000 7f584e800640 1 -- 172.21.15.151:0/2560155469 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f58480b4260 con 0x7f5848072560 2024-09-16T15:57:07.057 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:07.056+0000 7f5845600640 1 -- 172.21.15.151:0/2560155469 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v58)=0 v58) ==== 123+0+0 (secure 0 0 0) 0x7f583c064020 con 0x7f5848072560 2024-09-16T15:57:07.058 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:07.057+0000 7f5845600640 1 -- 172.21.15.151:0/2560155469 <== mon.0 v2:172.21.15.151:3300/0 8 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7f583c063b30 con 0x7f5848072560 2024-09-16T15:57:07.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:07.059+0000 7f584e800640 1 -- 172.21.15.151:0/2560155469 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f582807afb0 msgr2=0x7f582807d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:57:07.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:07.059+0000 7f584e800640 1 --2- 172.21.15.151:0/2560155469 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f582807afb0 0x7f582807d470 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f5834000f60 tx=0x7f58340079b0 comp rx=0 tx=0).stop 2024-09-16T15:57:07.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:07.059+0000 7f584e800640 1 -- 172.21.15.151:0/2560155469 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5848072560 msgr2=0x7f58480b3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:57:07.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:07.059+0000 7f584e800640 1 --2- 172.21.15.151:0/2560155469 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5848072560 0x7f58480b3d20 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f583c00e7c0 tx=0x7f583c00ec80 comp rx=0 tx=0).stop 2024-09-16T15:57:07.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:07.059+0000 7f584e800640 1 -- 172.21.15.151:0/2560155469 shutdown_connections 2024-09-16T15:57:07.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:07.059+0000 7f584e800640 1 --2- 172.21.15.151:0/2560155469 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f582807afb0 0x7f582807d470 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:57:07.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:07.059+0000 7f584e800640 1 --2- 172.21.15.151:0/2560155469 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f584815bec0 0x7f58481555a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:57:07.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:07.059+0000 7f584e800640 1 --2- 172.21.15.151:0/2560155469 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f58480cbc00 0x7f5848152f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:57:07.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:07.059+0000 7f584e800640 1 --2- 172.21.15.151:0/2560155469 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5848072560 0x7f58480b3d20 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:57:07.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:07.059+0000 7f584e800640 1 -- 172.21.15.151:0/2560155469 >> 172.21.15.151:0/2560155469 conn(0x7f58480cf9f0 msgr2=0x7f58480b30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:57:07.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:07.060+0000 7f584e800640 1 -- 172.21.15.151:0/2560155469 shutdown_connections 2024-09-16T15:57:07.061 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:07.060+0000 7f584e800640 1 -- 172.21.15.151:0/2560155469 wait complete. 2024-09-16T15:57:07.076 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-16T15:57:07.076 DEBUG:teuthology.orchestra.run.smithi151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T15:57:07.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:07.234+0000 7f7953200640 1 -- 172.21.15.151:0/95049894 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79540cb740 msgr2=0x7f79540736e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:57:07.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:07.234+0000 7f7953200640 1 --2- 172.21.15.151:0/95049894 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79540cb740 0x7f79540736e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f793c005fd0 tx=0x7f793c02a5d0 comp rx=0 tx=0).stop 2024-09-16T15:57:07.235 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:07.234+0000 7f7953200640 1 -- 172.21.15.151:0/95049894 shutdown_connections 2024-09-16T15:57:07.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:07.234+0000 7f7953200640 1 --2- 172.21.15.151:0/95049894 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f79540cb740 0x7f79540736e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:57:07.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:07.234+0000 7f7953200640 1 --2- 172.21.15.151:0/95049894 >> v2:172.21.15.167:3300/0 conn(0x7f7954071130 0x7f795415bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:57:07.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:07.234+0000 7f7953200640 1 -- 172.21.15.151:0/95049894 >> 172.21.15.151:0/95049894 conn(0x7f79540d0b30 msgr2=0x7f79540d0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:57:07.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:07.234+0000 7f7953200640 1 -- 172.21.15.151:0/95049894 shutdown_connections 2024-09-16T15:57:07.236 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:07.234+0000 7f7953200640 1 -- 172.21.15.151:0/95049894 wait complete. 2024-09-16T15:57:07.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:07.236+0000 7f7953200640 1 Processor -- start 2024-09-16T15:57:07.237 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:07.236+0000 7f7953200640 1 -- start start 2024-09-16T15:57:07.238 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:07.237+0000 7f7953200640 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7954071130 0x7f79541f7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:57:07.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:07.237+0000 7f7953200640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7954073c20 0x7f79541527b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:57:07.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:07.237+0000 7f7953200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79540cb740 0x7f795414e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:57:07.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:07.237+0000 7f7953200640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7954154fb0 con 0x7f7954073c20 2024-09-16T15:57:07.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:07.237+0000 7f7953200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7954154e30 con 0x7f79540cb740 2024-09-16T15:57:07.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:07.237+0000 7f7953200640 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 -- 0x7f7954155130 con 0x7f7954071130 2024-09-16T15:57:07.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:07.237+0000 7f794be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7954073c20 0x7f79541527b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:57:07.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:07.237+0000 7f794be00640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7954073c20 0x7f79541527b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:35294/0 (socket says 172.21.15.151:35294) 2024-09-16T15:57:07.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:07.237+0000 7f794be00640 1 -- 172.21.15.151:0/433790076 learned_addr learned my addr 172.21.15.151:0/433790076 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-16T15:57:07.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:07.237+0000 7f7951e00640 1 --2- 172.21.15.151:0/433790076 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7954071130 0x7f79541f7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:57:07.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:07.237+0000 7f7952800640 1 --2- 172.21.15.151:0/433790076 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79540cb740 0x7f795414e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:57:07.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:07.237+0000 7f794be00640 1 -- 172.21.15.151:0/433790076 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7954071130 msgr2=0x7f79541f7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:57:07.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:07.237+0000 7f794be00640 1 --2- 172.21.15.151:0/433790076 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7954071130 0x7f79541f7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:57:07.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:07.237+0000 7f794be00640 1 -- 172.21.15.151:0/433790076 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79540cb740 msgr2=0x7f795414e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:57:07.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:07.237+0000 7f794be00640 1 --2- 172.21.15.151:0/433790076 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79540cb740 0x7f795414e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:57:07.239 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:07.237+0000 7f794be00640 1 -- 172.21.15.151:0/433790076 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79541f88e0 con 0x7f7954073c20 2024-09-16T15:57:07.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:07.238+0000 7f794be00640 1 --2- 172.21.15.151:0/433790076 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7954073c20 0x7f79541527b0 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f793c034040 tx=0x7f793c0035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:57:07.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:07.238+0000 7f794a000640 1 -- 172.21.15.151:0/433790076 <== mon.0 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f793c03e030 con 0x7f7954073c20 2024-09-16T15:57:07.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:07.238+0000 7f794a000640 1 -- 172.21.15.151:0/433790076 <== mon.0 v2:172.21.15.151:3300/0 2 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7f793c003b00 con 0x7f7954073c20 2024-09-16T15:57:07.240 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:07.238+0000 7f7953200640 1 -- 172.21.15.151:0/433790076 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79541f8040 con 0x7f7954073c20 2024-09-16T15:57:07.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:07.238+0000 7f794a000640 1 -- 172.21.15.151:0/433790076 <== mon.0 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f793c035040 con 0x7f7954073c20 2024-09-16T15:57:07.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:07.238+0000 7f7953200640 1 -- 172.21.15.151:0/433790076 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79541f4580 con 0x7f7954073c20 2024-09-16T15:57:07.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:07.239+0000 7f7953200640 1 -- 172.21.15.151:0/433790076 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f795415bb90 con 0x7f7954073c20 2024-09-16T15:57:07.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:07.241+0000 7f794a000640 1 -- 172.21.15.151:0/433790076 <== mon.0 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 55) ==== 104857+0+0 (secure 0 0 0) 0x7f793c041030 con 0x7f7954073c20 2024-09-16T15:57:07.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:07.242+0000 7f794a000640 1 --2- 172.21.15.151:0/433790076 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f793007afb0 0x7f793007d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T15:57:07.242 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:07.242+0000 7f794a000640 1 -- 172.21.15.151:0/433790076 <== mon.0 v2:172.21.15.151:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5083+0+0 (secure 0 0 0) 0x7f793c0b9fd0 con 0x7f7954073c20 2024-09-16T15:57:07.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:07.242+0000 7f7951e00640 1 --2- 172.21.15.151:0/433790076 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f793007afb0 0x7f793007d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T15:57:07.243 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:07.243+0000 7f7951e00640 1 --2- 172.21.15.151:0/433790076 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f793007afb0 0x7f793007d470 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f79440095f0 tx=0x7f7944005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T15:57:07.246 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:07.246+0000 7f794a000640 1 -- 172.21.15.151:0/433790076 <== mon.0 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196735 (secure 0 0 0) 0x7f793c0840a0 con 0x7f7954073c20 2024-09-16T15:57:07.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:07.469+0000 7f7953200640 1 -- 172.21.15.151:0/433790076 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f79540d1a10 con 0x7f7954073c20 2024-09-16T15:57:07.470 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:07.470+0000 7f794a000640 1 -- 172.21.15.151:0/433790076 <== mon.0 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) ==== 94+0+3920 (secure 0 0 0) 0x7f793c0839a0 con 0x7f7954073c20 2024-09-16T15:57:07.471 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-16T15:57:07.471 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":99,"btime":"2024-09-16T15:56:33:682808+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7739,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2795044069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6835","nonce":2795044069},{"type":"v1","addr":"172.21.15.167:6837","nonce":2795044069}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":8644,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/675530853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6833","nonce":675530853},{"type":"v1","addr":"172.21.15.151:6835","nonce":675530853}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92},{"gid":9082,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6836/3424667775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":3424667775},{"type":"v1","addr":"172.21.15.167:6836","nonce":3424667775}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":99},{"gid":9468,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6834/1930858450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":1930858450},{"type":"v1","addr":"172.21.15.151:6834","nonce":1930858450}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91}],"filesystems":[]} 2024-09-16T15:57:07.471 INFO:teuthology.orchestra.run.smithi151.stderr:dumped fsmap epoch 99 2024-09-16T15:57:07.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:07.472+0000 7f7953200640 1 -- 172.21.15.151:0/433790076 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f793007afb0 msgr2=0x7f793007d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:57:07.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:07.472+0000 7f7953200640 1 --2- 172.21.15.151:0/433790076 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f793007afb0 0x7f793007d470 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f79440095f0 tx=0x7f7944005f90 comp rx=0 tx=0).stop 2024-09-16T15:57:07.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:07.472+0000 7f7953200640 1 -- 172.21.15.151:0/433790076 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7954073c20 msgr2=0x7f79541527b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T15:57:07.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:07.472+0000 7f7953200640 1 --2- 172.21.15.151:0/433790076 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7954073c20 0x7f79541527b0 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f793c034040 tx=0x7f793c0035f0 comp rx=0 tx=0).stop 2024-09-16T15:57:07.473 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:07.472+0000 7f7953200640 1 -- 172.21.15.151:0/433790076 shutdown_connections 2024-09-16T15:57:07.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:07.472+0000 7f7953200640 1 --2- 172.21.15.151:0/433790076 >> [v2:172.21.15.167:6800/501171453,v1:172.21.15.167:6801/501171453] conn(0x7f793007afb0 0x7f793007d470 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:57:07.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:07.472+0000 7f7953200640 1 --2- 172.21.15.151:0/433790076 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f79540cb740 0x7f795414e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:57:07.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:07.472+0000 7f7953200640 1 --2- 172.21.15.151:0/433790076 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7954073c20 0x7f79541527b0 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:57:07.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:07.472+0000 7f7953200640 1 --2- 172.21.15.151:0/433790076 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f7954071130 0x7f79541f7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T15:57:07.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:07.472+0000 7f7953200640 1 -- 172.21.15.151:0/433790076 >> 172.21.15.151:0/433790076 conn(0x7f79540d0b30 msgr2=0x7f795414f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T15:57:07.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:07.472+0000 7f7953200640 1 -- 172.21.15.151:0/433790076 shutdown_connections 2024-09-16T15:57:07.474 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-16T15:57:07.472+0000 7f7953200640 1 -- 172.21.15.151:0/433790076 wait complete. 2024-09-16T15:57:07.490 DEBUG:tasks.cephfs.filesystem:already dead... 2024-09-16T15:57:07.490 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-16T15:57:07.490 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-16T15:57:07.490 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:57:07.505 INFO:tasks.ceph.mds.a:Stopped 2024-09-16T15:57:07.505 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-16T15:57:07.505 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:57:07.560 INFO:tasks.ceph.mds.c:Stopped 2024-09-16T15:57:07.560 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-16T15:57:07.560 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:57:07.577 INFO:tasks.ceph.mds.b:Stopped 2024-09-16T15:57:07.577 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-16T15:57:07.577 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:57:07.635 INFO:tasks.ceph.mds.d:Stopped 2024-09-16T15:57:07.635 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-16T15:57:07.635 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-16T15:57:07.635 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:57:07.710 INFO:tasks.ceph.osd.0:Stopped 2024-09-16T15:57:07.710 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-16T15:57:07.710 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:57:07.775 INFO:tasks.ceph.osd.1:Stopped 2024-09-16T15:57:07.775 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-16T15:57:07.775 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:57:07.846 INFO:tasks.ceph.osd.2:Stopped 2024-09-16T15:57:07.846 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-16T15:57:07.846 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:57:07.910 INFO:tasks.ceph.osd.3:Stopped 2024-09-16T15:57:07.910 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-16T15:57:07.910 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:57:07.985 INFO:tasks.ceph.osd.4:Stopped 2024-09-16T15:57:07.986 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-16T15:57:07.986 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:57:08.074 INFO:tasks.ceph.osd.5:Stopped 2024-09-16T15:57:08.074 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-16T15:57:08.074 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:57:08.148 INFO:tasks.ceph.osd.6:Stopped 2024-09-16T15:57:08.148 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-16T15:57:08.148 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:57:08.214 INFO:tasks.ceph.osd.7:Stopped 2024-09-16T15:57:08.214 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-16T15:57:08.215 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-16T15:57:08.215 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:57:08.279 INFO:tasks.ceph.mgr.y:Stopped 2024-09-16T15:57:08.280 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-16T15:57:08.280 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:57:12.471 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~0s 2024-09-16T15:57:14.381 INFO:tasks.ceph.mgr.x:Stopped 2024-09-16T15:57:14.381 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-16T15:57:14.382 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-16T15:57:14.382 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:57:14.409 INFO:tasks.ceph.mon.a:Stopped 2024-09-16T15:57:14.409 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-16T15:57:14.410 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:57:14.432 INFO:tasks.ceph.mon.b:Stopped 2024-09-16T15:57:14.433 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-16T15:57:14.433 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T15:57:14.497 INFO:tasks.ceph.mon.c:Stopped 2024-09-16T15:57:14.497 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-16T15:57:14.498 DEBUG:teuthology.orchestra.run.smithi151:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2024-09-16T15:57:14.589 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi151.front.sepia.ceph.com 2024-09-16T15:57:14.589 DEBUG:teuthology.orchestra.run.smithi151:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-16T15:57:14.808 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi151.front.sepia.ceph.com 2024-09-16T15:57:14.808 DEBUG:teuthology.orchestra.run.smithi151:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-16T15:57:14.925 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi151.front.sepia.ceph.com 2024-09-16T15:57:14.926 DEBUG:teuthology.orchestra.run.smithi151:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-16T15:57:15.045 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi151.front.sepia.ceph.com 2024-09-16T15:57:15.045 DEBUG:teuthology.orchestra.run.smithi151:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-16T15:57:15.171 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi167.front.sepia.ceph.com 2024-09-16T15:57:15.171 DEBUG:teuthology.orchestra.run.smithi167:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-16T15:57:15.420 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi167.front.sepia.ceph.com 2024-09-16T15:57:15.420 DEBUG:teuthology.orchestra.run.smithi167:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-16T15:57:15.543 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi167.front.sepia.ceph.com 2024-09-16T15:57:15.544 DEBUG:teuthology.orchestra.run.smithi167:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-16T15:57:15.658 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi167.front.sepia.ceph.com 2024-09-16T15:57:15.658 DEBUG:teuthology.orchestra.run.smithi167:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-16T15:57:15.828 INFO:tasks.ceph:Archiving mon data... 2024-09-16T15:57:15.829 DEBUG:teuthology.misc:Transferring archived files from smithi151:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/avan-2024-09-16_15:21:40-fs:volumes-wip-athakkar-testing-2024-09-16-1454-distro-default-smithi/7907354/data/mon.a.tgz 2024-09-16T15:57:15.830 DEBUG:teuthology.orchestra.run.smithi151:> mktemp 2024-09-16T15:57:15.866 INFO:teuthology.orchestra.run.smithi151.stdout:/tmp/tmp.wOG6bhMqaP 2024-09-16T15:57:15.867 DEBUG:teuthology.orchestra.run.smithi151:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.wOG6bhMqaP 2024-09-16T15:57:16.255 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod 0666 /tmp/tmp.wOG6bhMqaP 2024-09-16T15:57:16.351 DEBUG:teuthology.orchestra.remote:smithi151:/tmp/tmp.wOG6bhMqaP is 2MB 2024-09-16T15:57:16.470 DEBUG:teuthology.orchestra.run.smithi151:> rm -fr /tmp/tmp.wOG6bhMqaP 2024-09-16T15:57:16.506 DEBUG:teuthology.misc:Transferring archived files from smithi167:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/avan-2024-09-16_15:21:40-fs:volumes-wip-athakkar-testing-2024-09-16-1454-distro-default-smithi/7907354/data/mon.b.tgz 2024-09-16T15:57:16.506 DEBUG:teuthology.orchestra.run.smithi167:> mktemp 2024-09-16T15:57:16.545 INFO:teuthology.orchestra.run.smithi167.stdout:/tmp/tmp.7ayjF8OWga 2024-09-16T15:57:16.545 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.7ayjF8OWga 2024-09-16T15:57:16.936 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 0666 /tmp/tmp.7ayjF8OWga 2024-09-16T15:57:17.029 DEBUG:teuthology.orchestra.remote:smithi167:/tmp/tmp.7ayjF8OWga is 2MB 2024-09-16T15:57:17.134 DEBUG:teuthology.orchestra.run.smithi167:> rm -fr /tmp/tmp.7ayjF8OWga 2024-09-16T15:57:17.170 DEBUG:teuthology.misc:Transferring archived files from smithi167:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/avan-2024-09-16_15:21:40-fs:volumes-wip-athakkar-testing-2024-09-16-1454-distro-default-smithi/7907354/data/mon.c.tgz 2024-09-16T15:57:17.170 DEBUG:teuthology.orchestra.run.smithi167:> mktemp 2024-09-16T15:57:17.244 INFO:teuthology.orchestra.run.smithi167.stdout:/tmp/tmp.novbzlCRwu 2024-09-16T15:57:17.245 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.novbzlCRwu 2024-09-16T15:57:17.472 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x has been restored 2024-09-16T15:57:17.610 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 0666 /tmp/tmp.novbzlCRwu 2024-09-16T15:57:17.704 DEBUG:teuthology.orchestra.remote:smithi167:/tmp/tmp.novbzlCRwu is 2MB 2024-09-16T15:57:17.820 DEBUG:teuthology.orchestra.run.smithi167:> rm -fr /tmp/tmp.novbzlCRwu 2024-09-16T15:57:17.857 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-16T15:57:17.857 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-16T15:57:17.862 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-16T15:57:17.991 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-16T15:57:18.026 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-16T15:57:18.026 INFO:tasks.ceph:Archiving crash dumps... 2024-09-16T15:57:18.028 DEBUG:teuthology.misc:Transferring archived files from smithi151:/var/lib/ceph/crash to /home/teuthworker/archive/avan-2024-09-16_15:21:40-fs:volumes-wip-athakkar-testing-2024-09-16-1454-distro-default-smithi/7907354/remote/smithi151/crash 2024-09-16T15:57:18.029 DEBUG:teuthology.orchestra.run.smithi151:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-16T15:57:18.087 DEBUG:teuthology.misc:Transferring archived files from smithi167:/var/lib/ceph/crash to /home/teuthworker/archive/avan-2024-09-16_15:21:40-fs:volumes-wip-athakkar-testing-2024-09-16-1454-distro-default-smithi/7907354/remote/smithi167/crash 2024-09-16T15:57:18.088 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-16T15:57:18.143 INFO:tasks.ceph:Shutting down logrotate 2024-09-16T15:57:18.143 DEBUG:teuthology.orchestra.run.smithi151:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:57:18.147 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T15:57:18.245 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-16T15:57:18.301 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-16T15:57:18.357 INFO:tasks.ceph:Compressing logs... 2024-09-16T15:57:18.357 DEBUG:teuthology.orchestra.run.smithi151:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-16T15:57:18.362 DEBUG:teuthology.orchestra.run.smithi167:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-16T15:57:18.412 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-16T15:57:18.413 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62308.log 2024-09-16T15:57:18.414 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46501.log 2024-09-16T15:57:18.415 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.62308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54615.log 2024-09-16T15:57:18.415 INFO:teuthology.orchestra.run.smithi151.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.62308.log.gz 2024-09-16T15:57:18.415 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.46501.log: gzip -5 --verbose 87.6% -- /var/log/ceph/ceph-client.admin.52193.log -- replaced with /var/log/ceph/ceph-client.admin.46501.log.gz 2024-09-16T15:57:18.415 INFO:teuthology.orchestra.run.smithi151.stderr: 2024-09-16T15:57:18.416 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36043.log 2024-09-16T15:57:18.418 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.54615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77155.log 2024-09-16T15:57:18.418 INFO:teuthology.orchestra.run.smithi151.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.54615.log.gz 2024-09-16T15:57:18.419 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.52193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50582.log 2024-09-16T15:57:18.419 INFO:teuthology.orchestra.run.smithi151.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.52193.log.gz 2024-09-16T15:57:18.419 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.77155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.76317.log 2024-09-16T15:57:18.419 INFO:teuthology.orchestra.run.smithi151.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.77155.log.gz 2024-09-16T15:57:18.428 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-16T15:57:18.428 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.36043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36163.log 2024-09-16T15:57:18.428 INFO:teuthology.orchestra.run.smithi167.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.36043.log.gz 2024-09-16T15:57:18.428 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.41964.log 2024-09-16T15:57:18.429 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.36163.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-09-16T15:57:18.429 INFO:teuthology.orchestra.run.smithi167.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.36163.log.gz 2024-09-16T15:57:18.429 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.3.41964.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.41964.log.gz 2024-09-16T15:57:18.429 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36003.log 2024-09-16T15:57:18.429 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-16T15:57:18.429 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.36003.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-16T15:57:18.429 INFO:teuthology.orchestra.run.smithi167.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.36003.log.gz 2024-09-16T15:57:18.429 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36283.log 2024-09-16T15:57:18.429 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-16T15:57:18.430 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.36283.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-16T15:57:18.430 INFO:teuthology.orchestra.run.smithi167.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.36283.log.gz 2024-09-16T15:57:18.430 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-16T15:57:18.430 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36083.log 2024-09-16T15:57:18.436 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-osd.5.log: /var/log/ceph/ceph-client.admin.36083.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.36083.log.gz 2024-09-16T15:57:18.436 INFO:teuthology.orchestra.run.smithi167.stderr: 93.6%gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-16T15:57:18.436 INFO:teuthology.orchestra.run.smithi167.stderr: -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-16T15:57:18.436 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36243.log 2024-09-16T15:57:18.436 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.36243.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-09-16T15:57:18.445 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-mds.d.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.36243.log.gz 2024-09-16T15:57:18.445 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-16T15:57:18.445 INFO:teuthology.orchestra.run.smithi167.stderr: 91.9% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-16T15:57:18.445 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-16T15:57:18.445 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-mon.b.log: /var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.41960.log 2024-09-16T15:57:18.448 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.50582.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50582.log.gzgzip 2024-09-16T15:57:18.448 INFO:teuthology.orchestra.run.smithi151.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.54075.log 2024-09-16T15:57:18.449 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.0.76317.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.76317.log.gz 2024-09-16T15:57:18.449 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82433.log 2024-09-16T15:57:18.449 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.54075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81099.log 2024-09-16T15:57:18.449 INFO:teuthology.orchestra.run.smithi151.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54075.log.gz 2024-09-16T15:57:18.449 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.82433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50834.log 2024-09-16T15:57:18.449 INFO:teuthology.orchestra.run.smithi151.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.82433.log.gz 2024-09-16T15:57:18.449 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.81099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49408.log 2024-09-16T15:57:18.449 INFO:teuthology.orchestra.run.smithi151.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.81099.log.gz 2024-09-16T15:57:18.449 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.50834.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50834.log.gz 2024-09-16T15:57:18.449 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81578.log 2024-09-16T15:57:18.449 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.49408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67860.log 2024-09-16T15:57:18.450 INFO:teuthology.orchestra.run.smithi151.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49408.log.gz 2024-09-16T15:57:18.450 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.81578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52585.log 2024-09-16T15:57:18.450 INFO:teuthology.orchestra.run.smithi151.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.81578.log.gz 2024-09-16T15:57:18.450 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.67860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78752.log 2024-09-16T15:57:18.450 INFO:teuthology.orchestra.run.smithi151.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.67860.log.gz 2024-09-16T15:57:18.450 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.52585.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.52585.log.gz 2024-09-16T15:57:18.450 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55925.log 2024-09-16T15:57:18.450 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.78752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80723.log 2024-09-16T15:57:18.450 INFO:teuthology.orchestra.run.smithi151.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.78752.log.gz 2024-09-16T15:57:18.450 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.55925.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67186.log 2024-09-16T15:57:18.450 INFO:teuthology.orchestra.run.smithi151.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.55925.log.gz 2024-09-16T15:57:18.451 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.80723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79970.log 2024-09-16T15:57:18.451 INFO:teuthology.orchestra.run.smithi151.stderr: 83.3% -- replaced with /var/log/ceph/ceph-client.admin.80723.log.gz 2024-09-16T15:57:18.451 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.67186.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.67186.log.gz 2024-09-16T15:57:18.451 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74720.log 2024-09-16T15:57:18.451 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.79970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75811.log 2024-09-16T15:57:18.451 INFO:teuthology.orchestra.run.smithi151.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.79970.log.gz 2024-09-16T15:57:18.451 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.74720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.58558.log 2024-09-16T15:57:18.451 INFO:teuthology.orchestra.run.smithi151.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.74720.log.gz 2024-09-16T15:57:18.451 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.75811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70561.log 2024-09-16T15:57:18.451 INFO:teuthology.orchestra.run.smithi151.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.75811.log.gz 2024-09-16T15:57:18.451 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.0.58558.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.58558.log.gz 2024-09-16T15:57:18.451 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69892.log 2024-09-16T15:57:18.452 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.70561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74092.log 2024-09-16T15:57:18.452 INFO:teuthology.orchestra.run.smithi151.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.70561.log.gz 2024-09-16T15:57:18.452 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.69892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50666.log 2024-09-16T15:57:18.452 INFO:teuthology.orchestra.run.smithi151.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.69892.log.gz 2024-09-16T15:57:18.452 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.74092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75230.log 2024-09-16T15:57:18.452 INFO:teuthology.orchestra.run.smithi151.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.74092.log.gz 2024-09-16T15:57:18.452 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.50666.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.50666.log.gz 2024-09-16T15:57:18.452 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65929.log 2024-09-16T15:57:18.452 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.75230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61344.log 2024-09-16T15:57:18.452 INFO:teuthology.orchestra.run.smithi151.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.75230.log.gz 2024-09-16T15:57:18.452 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.65929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50410.log 2024-09-16T15:57:18.452 INFO:teuthology.orchestra.run.smithi151.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.65929.log.gz 2024-09-16T15:57:18.453 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.61344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65026.log 2024-09-16T15:57:18.453 INFO:teuthology.orchestra.run.smithi151.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.61344.log.gz 2024-09-16T15:57:18.453 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.50410.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50410.log.gz 2024-09-16T15:57:18.453 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65149.log 2024-09-16T15:57:18.453 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.65026.log: gzip 84.6% -- replaced with /var/log/ceph/ceph-client.admin.65026.log.gz -5 2024-09-16T15:57:18.453 INFO:teuthology.orchestra.run.smithi151.stderr: --verbose -- /var/log/ceph/ceph-client.admin.50144.log 2024-09-16T15:57:18.453 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.65149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82918.log 2024-09-16T15:57:18.453 INFO:teuthology.orchestra.run.smithi151.stderr: 84.8% -- replaced with /var/log/ceph/ceph-client.admin.65149.log.gz 2024-09-16T15:57:18.453 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.50144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50102.log 2024-09-16T15:57:18.453 INFO:teuthology.orchestra.run.smithi151.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50144.log.gz 2024-09-16T15:57:18.453 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.82918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.72600.log 2024-09-16T15:57:18.454 INFO:teuthology.orchestra.run.smithi151.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.82918.log.gz 2024-09-16T15:57:18.454 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.50102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75853.log 2024-09-16T15:57:18.454 INFO:teuthology.orchestra.run.smithi151.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50102.log.gz 2024-09-16T15:57:18.454 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.0.72600.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.72600.log.gz 2024-09-16T15:57:18.454 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79760.log 2024-09-16T15:57:18.454 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.75853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57126.log 2024-09-16T15:57:18.454 INFO:teuthology.orchestra.run.smithi151.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.75853.log.gz 2024-09-16T15:57:18.454 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.79760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67060.log 2024-09-16T15:57:18.454 INFO:teuthology.orchestra.run.smithi151.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.79760.log.gz 2024-09-16T15:57:18.454 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.57126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75168.log 2024-09-16T15:57:18.454 INFO:teuthology.orchestra.run.smithi151.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57126.log.gz 2024-09-16T15:57:18.458 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36203.log 2024-09-16T15:57:18.458 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.3.41960.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.41960.log.gz 2024-09-16T15:57:18.458 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.36203.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.36203.log.gz 2024-09-16T15:57:18.458 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.41546.log 2024-09-16T15:57:18.463 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36123.log 2024-09-16T15:57:18.463 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.1.41546.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.41546.log.gz 2024-09-16T15:57:18.466 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.67060.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.67060.log.gz 2024-09-16T15:57:18.466 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57781.log 2024-09-16T15:57:18.466 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.75168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42293.log 2024-09-16T15:57:18.466 INFO:teuthology.orchestra.run.smithi151.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.75168.log.gz 2024-09-16T15:57:18.467 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.57781.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35642.log 2024-09-16T15:57:18.467 INFO:teuthology.orchestra.run.smithi151.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.57781.log.gz 2024-09-16T15:57:18.467 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.42293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75272.log 2024-09-16T15:57:18.467 INFO:teuthology.orchestra.run.smithi151.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42293.log.gz 2024-09-16T15:57:18.467 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.35642.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35642.log.gz 2024-09-16T15:57:18.467 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65593.log 2024-09-16T15:57:18.467 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.75272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73888.log 2024-09-16T15:57:18.467 INFO:teuthology.orchestra.run.smithi151.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.75272.log.gz 2024-09-16T15:57:18.467 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.65593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52237.log 2024-09-16T15:57:18.467 INFO:teuthology.orchestra.run.smithi151.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.65593.log.gz 2024-09-16T15:57:18.467 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.73888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81225.log 2024-09-16T15:57:18.468 INFO:teuthology.orchestra.run.smithi151.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.73888.log.gz 2024-09-16T15:57:18.468 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.52237.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.52237.log.gz 2024-09-16T15:57:18.468 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57571.log 2024-09-16T15:57:18.468 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.81225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36403.log 2024-09-16T15:57:18.468 INFO:teuthology.orchestra.run.smithi151.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.81225.log.gz 2024-09-16T15:57:18.468 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.57571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53567.log 2024-09-16T15:57:18.468 INFO:teuthology.orchestra.run.smithi151.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57571.log.gz 2024-09-16T15:57:18.468 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.36403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63939.log 2024-09-16T15:57:18.468 INFO:teuthology.orchestra.run.smithi151.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.36403.log.gz 2024-09-16T15:57:18.468 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.53567.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.53567.log.gz 2024-09-16T15:57:18.468 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43519.log 2024-09-16T15:57:18.469 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.63939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66139.log 2024-09-16T15:57:18.469 INFO:teuthology.orchestra.run.smithi151.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.63939.log.gz 2024-09-16T15:57:18.469 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.43519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81961.log 2024-09-16T15:57:18.469 INFO:teuthology.orchestra.run.smithi151.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43519.log.gz 2024-09-16T15:57:18.469 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.66139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74407.log 2024-09-16T15:57:18.469 INFO:teuthology.orchestra.run.smithi151.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.66139.log.gz 2024-09-16T15:57:18.469 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.81961.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.81961.log.gz 2024-09-16T15:57:18.469 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47447.log 2024-09-16T15:57:18.469 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.74407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64545.log 2024-09-16T15:57:18.469 INFO:teuthology.orchestra.run.smithi151.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.74407.log.gz 2024-09-16T15:57:18.469 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.47447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66265.log 2024-09-16T15:57:18.469 INFO:teuthology.orchestra.run.smithi151.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.47447.log.gz 2024-09-16T15:57:18.470 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.64545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42461.log 2024-09-16T15:57:18.470 INFO:teuthology.orchestra.run.smithi151.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.64545.log.gz 2024-09-16T15:57:18.470 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.66265.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.66265.log.gz 2024-09-16T15:57:18.470 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66787.log 2024-09-16T15:57:18.472 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.admin.36123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.41565.log 2024-09-16T15:57:18.472 INFO:teuthology.orchestra.run.smithi167.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.36123.log.gz 2024-09-16T15:57:18.472 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/ceph-client.1.41565.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.41565.log.gz 2024-09-16T15:57:18.476 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.42461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66976.log 2024-09-16T15:57:18.476 INFO:teuthology.orchestra.run.smithi151.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42461.log.gz 2024-09-16T15:57:18.476 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.66787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54531.log 2024-09-16T15:57:18.476 INFO:teuthology.orchestra.run.smithi151.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.66787.log.gz 2024-09-16T15:57:18.477 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.66976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81015.log 2024-09-16T15:57:18.477 INFO:teuthology.orchestra.run.smithi151.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.66976.log.gz 2024-09-16T15:57:18.477 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.54531.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54531.log.gz 2024-09-16T15:57:18.477 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47513.log 2024-09-16T15:57:18.477 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.81015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76021.log 2024-09-16T15:57:18.477 INFO:teuthology.orchestra.run.smithi151.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.81015.log.gz 2024-09-16T15:57:18.477 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.47513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71210.log 2024-09-16T15:57:18.477 INFO:teuthology.orchestra.run.smithi151.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47513.log.gz 2024-09-16T15:57:18.477 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.76021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82792.log 2024-09-16T15:57:18.477 INFO:teuthology.orchestra.run.smithi151.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.76021.log.gz 2024-09-16T15:57:18.477 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.71210.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.71210.log.gz 2024-09-16T15:57:18.478 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58258.log 2024-09-16T15:57:18.478 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.82792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43847.log 2024-09-16T15:57:18.478 INFO:teuthology.orchestra.run.smithi151.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.82792.log.gz 2024-09-16T15:57:18.478 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.58258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.44945.log 2024-09-16T15:57:18.478 INFO:teuthology.orchestra.run.smithi151.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.58258.log.gz 2024-09-16T15:57:18.478 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.43847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50792.log 2024-09-16T15:57:18.478 INFO:teuthology.orchestra.run.smithi151.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43847.log.gz 2024-09-16T15:57:18.478 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.2.44945.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.44945.log.gz 2024-09-16T15:57:18.480 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78137.log 2024-09-16T15:57:18.480 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.50792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56855.log 2024-09-16T15:57:18.480 INFO:teuthology.orchestra.run.smithi151.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.50792.log.gz 2024-09-16T15:57:18.480 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.78137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75937.log 2024-09-16T15:57:18.480 INFO:teuthology.orchestra.run.smithi151.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.78137.log.gz 2024-09-16T15:57:18.480 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.56855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60152.log 2024-09-16T15:57:18.480 INFO:teuthology.orchestra.run.smithi151.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56855.log.gz 2024-09-16T15:57:18.481 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.75937.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.75937.log.gz 2024-09-16T15:57:18.481 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67144.log 2024-09-16T15:57:18.481 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.60152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48865.log 2024-09-16T15:57:18.481 INFO:teuthology.orchestra.run.smithi151.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.60152.log.gz 2024-09-16T15:57:18.482 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.67144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52753.log 2024-09-16T15:57:18.482 INFO:teuthology.orchestra.run.smithi151.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.67144.log.gz 2024-09-16T15:57:18.483 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.48865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77866.log 2024-09-16T15:57:18.483 INFO:teuthology.orchestra.run.smithi151.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.48865.log.gz 2024-09-16T15:57:18.484 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.52753.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.52753.log.gz 2024-09-16T15:57:18.484 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81267.log 2024-09-16T15:57:18.484 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.77866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73283.log 2024-09-16T15:57:18.484 INFO:teuthology.orchestra.run.smithi151.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.77866.log.gz 2024-09-16T15:57:18.485 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.81267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74593.log 2024-09-16T15:57:18.485 INFO:teuthology.orchestra.run.smithi151.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.81267.log.gz 2024-09-16T15:57:18.486 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.73283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58678.log 2024-09-16T15:57:18.486 INFO:teuthology.orchestra.run.smithi151.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.73283.log.gz 2024-09-16T15:57:18.487 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.74593.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.74593.log.gz 2024-09-16T15:57:18.487 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45961.log 2024-09-16T15:57:18.488 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.58678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65272.log 2024-09-16T15:57:18.488 INFO:teuthology.orchestra.run.smithi151.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.58678.log.gz 2024-09-16T15:57:18.489 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.45961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.51172.log 2024-09-16T15:57:18.489 INFO:teuthology.orchestra.run.smithi151.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45961.log.gz 2024-09-16T15:57:18.490 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.65272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69976.log 2024-09-16T15:57:18.490 INFO:teuthology.orchestra.run.smithi151.stderr: 83.6% -- replaced with /var/log/ceph/ceph-client.admin.65272.log.gz 2024-09-16T15:57:18.490 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.0.51172.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.51172.log.gz 2024-09-16T15:57:18.490 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67685.log 2024-09-16T15:57:18.491 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.69976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67391.log 2024-09-16T15:57:18.492 INFO:teuthology.orchestra.run.smithi151.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.69976.log.gz 2024-09-16T15:57:18.492 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.67685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81141.log 2024-09-16T15:57:18.492 INFO:teuthology.orchestra.run.smithi151.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.67685.log.gz 2024-09-16T15:57:18.493 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.67391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36606.log 2024-09-16T15:57:18.493 INFO:teuthology.orchestra.run.smithi151.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.67391.log.gz 2024-09-16T15:57:18.494 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.81141.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.81141.log.gz 2024-09-16T15:57:18.494 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-16T15:57:18.495 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.36606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62224.log 2024-09-16T15:57:18.495 INFO:teuthology.orchestra.run.smithi151.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.36606.log.gz 2024-09-16T15:57:18.495 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83254.log 2024-09-16T15:57:18.497 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.62224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68207.log 2024-09-16T15:57:18.497 INFO:teuthology.orchestra.run.smithi151.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.62224.log.gz 2024-09-16T15:57:18.497 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.83254.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.83254.log.gz 2024-09-16T15:57:18.498 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57736.log 2024-09-16T15:57:18.499 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.68207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57825.log 2024-09-16T15:57:18.499 INFO:teuthology.orchestra.run.smithi151.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.68207.log.gz 2024-09-16T15:57:18.500 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.57736.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57736.log.gz 2024-09-16T15:57:18.500 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66643.log 2024-09-16T15:57:18.501 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.57825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60779.log 2024-09-16T15:57:18.501 INFO:teuthology.orchestra.run.smithi151.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.57825.log.gz 2024-09-16T15:57:18.502 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.66643.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.66643.log.gz 2024-09-16T15:57:18.502 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73535.log 2024-09-16T15:57:18.503 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.60779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69135.log 2024-09-16T15:57:18.503 INFO:teuthology.orchestra.run.smithi151.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.60779.log.gz 2024-09-16T15:57:18.504 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.73535.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.73535.log.gz 2024-09-16T15:57:18.504 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66559.log 2024-09-16T15:57:18.505 INFO:teuthology.orchestra.run.smithi167.stderr: 94.1% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-09-16T15:57:18.505 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.69135.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.69135.log.gz 2024-09-16T15:57:18.505 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56593.log 2024-09-16T15:57:18.506 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.66559.log: gzip -5 87.1% --verbose -- replaced with /var/log/ceph/ceph-client.admin.66559.log.gz -- 2024-09-16T15:57:18.506 INFO:teuthology.orchestra.run.smithi151.stderr: /var/log/ceph/ceph-client.admin.60110.log 2024-09-16T15:57:18.507 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.56593.log: 87.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.56593.log.gz 2024-09-16T15:57:18.507 INFO:teuthology.orchestra.run.smithi151.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.43171.log 2024-09-16T15:57:18.508 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.60110.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.60110.log.gz 2024-09-16T15:57:18.508 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75709.log 2024-09-16T15:57:18.509 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.43171.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43171.log.gz 2024-09-16T15:57:18.509 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53651.log 2024-09-16T15:57:18.509 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.75709.log: gzip 87.4% -5 -- replaced with /var/log/ceph/ceph-client.admin.75709.log.gz --verbose 2024-09-16T15:57:18.510 INFO:teuthology.orchestra.run.smithi151.stderr: -- /var/log/ceph/ceph-client.admin.77520.log 2024-09-16T15:57:18.510 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.53651.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53651.log.gz 2024-09-16T15:57:18.510 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60320.log 2024-09-16T15:57:18.511 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.77520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65108.log 2024-09-16T15:57:18.511 INFO:teuthology.orchestra.run.smithi151.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.77520.log.gz 2024-09-16T15:57:18.512 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.60320.log: gzip 86.2% -- replaced with /var/log/ceph/ceph-client.admin.60320.log.gz 2024-09-16T15:57:18.512 INFO:teuthology.orchestra.run.smithi151.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.46129.log 2024-09-16T15:57:18.513 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.65108.log: 83.6% -- replaced with /var/log/ceph/ceph-client.admin.65108.log.gz 2024-09-16T15:57:18.513 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43183.log 2024-09-16T15:57:18.514 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.46129.log: gzip -5 --verbose -- 87.9% /var/log/ceph/ceph-client.admin.64482.log -- replaced with /var/log/ceph/ceph-client.admin.46129.log.gz 2024-09-16T15:57:18.514 INFO:teuthology.orchestra.run.smithi151.stderr: 2024-09-16T15:57:18.515 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.43183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47909.log 2024-09-16T15:57:18.515 INFO:teuthology.orchestra.run.smithi151.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43183.log.gz 2024-09-16T15:57:18.516 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.64482.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.64482.log.gz 2024-09-16T15:57:18.516 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68114.log 2024-09-16T15:57:18.517 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.47909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64985.log 2024-09-16T15:57:18.517 INFO:teuthology.orchestra.run.smithi151.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47909.log.gz 2024-09-16T15:57:18.518 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.68114.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.68114.log.gz 2024-09-16T15:57:18.518 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81452.log 2024-09-16T15:57:18.519 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.64985.log: gzip -5 84.2% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.64985.log.gz 2024-09-16T15:57:18.519 INFO:teuthology.orchestra.run.smithi151.stderr: /var/log/ceph/ceph-client.admin.83338.log 2024-09-16T15:57:18.519 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.81452.log: 87.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.81452.log.gz 2024-09-16T15:57:18.519 INFO:teuthology.orchestra.run.smithi151.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.60905.log 2024-09-16T15:57:18.520 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.83338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60026.log 2024-09-16T15:57:18.520 INFO:teuthology.orchestra.run.smithi151.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.83338.log.gz 2024-09-16T15:57:18.521 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.60905.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.60905.log.gz 2024-09-16T15:57:18.521 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63585.log 2024-09-16T15:57:18.522 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.60026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63417.log 2024-09-16T15:57:18.523 INFO:teuthology.orchestra.run.smithi151.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.60026.log.gz 2024-09-16T15:57:18.523 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.63585.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.63585.log.gz 2024-09-16T15:57:18.523 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42852.log 2024-09-16T15:57:18.524 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.63417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42419.log 2024-09-16T15:57:18.524 INFO:teuthology.orchestra.run.smithi151.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.63417.log.gz 2024-09-16T15:57:18.525 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.42852.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42852.log.gz 2024-09-16T15:57:18.525 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82708.log 2024-09-16T15:57:18.526 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.42419.log: gzip -5 --verbose -- 87.5% /var/log/ceph/ceph-client.admin.56060.log -- replaced with /var/log/ceph/ceph-client.admin.42419.log.gz 2024-09-16T15:57:18.526 INFO:teuthology.orchestra.run.smithi151.stderr: 2024-09-16T15:57:18.527 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.82708.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.82708.log.gz 2024-09-16T15:57:18.527 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51254.log 2024-09-16T15:57:18.528 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.56060.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56060.log.gz 2024-09-16T15:57:18.528 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67559.log 2024-09-16T15:57:18.529 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.51254.log: 87.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.51254.log.gz 2024-09-16T15:57:18.529 INFO:teuthology.orchestra.run.smithi151.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.59984.log 2024-09-16T15:57:18.530 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.67559.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.67559.log.gz 2024-09-16T15:57:18.530 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80681.log 2024-09-16T15:57:18.531 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.59984.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.59984.log.gz 2024-09-16T15:57:18.531 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56396.log 2024-09-16T15:57:18.532 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.80681.log: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.80681.log.gz 2024-09-16T15:57:18.532 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58720.log 2024-09-16T15:57:18.532 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.56396.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.56396.log.gzgzip 2024-09-16T15:57:18.533 INFO:teuthology.orchestra.run.smithi151.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.67228.log 2024-09-16T15:57:18.533 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.58720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62014.log 2024-09-16T15:57:18.534 INFO:teuthology.orchestra.run.smithi151.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.58720.log.gz 2024-09-16T15:57:18.534 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.67228.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.67228.log.gz 2024-09-16T15:57:18.534 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58366.log 2024-09-16T15:57:18.535 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.62014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43508.log 2024-09-16T15:57:18.535 INFO:teuthology.orchestra.run.smithi151.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.62014.log.gz 2024-09-16T15:57:18.536 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.58366.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.58366.log.gz 2024-09-16T15:57:18.536 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72048.log 2024-09-16T15:57:18.537 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.43508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42671.log 2024-09-16T15:57:18.537 INFO:teuthology.orchestra.run.smithi151.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.43508.log.gz 2024-09-16T15:57:18.538 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.72048.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.72048.log.gz 2024-09-16T15:57:18.538 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46213.log 2024-09-16T15:57:18.539 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.42671.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42671.log.gz 2024-09-16T15:57:18.539 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54033.log 2024-09-16T15:57:18.540 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.46213.log: gzip -5 --verbose 87.4% -- -- replaced with /var/log/ceph/ceph-client.admin.46213.log.gz 2024-09-16T15:57:18.540 INFO:teuthology.orchestra.run.smithi151.stderr: /var/log/ceph/ceph-client.admin.67601.log 2024-09-16T15:57:18.541 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.54033.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54033.log.gz 2024-09-16T15:57:18.541 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63897.log 2024-09-16T15:57:18.542 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.67601.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.67601.log.gz 2024-09-16T15:57:18.542 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59527.log 2024-09-16T15:57:18.543 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.63897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72006.log 2024-09-16T15:57:18.543 INFO:teuthology.orchestra.run.smithi151.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.63897.log.gz 2024-09-16T15:57:18.544 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.59527.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.59527.log.gz 2024-09-16T15:57:18.544 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77656.log 2024-09-16T15:57:18.544 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.72006.log: gzip -5 --verbose -- 87.2% /var/log/ceph/ceph-client.admin.51338.log 2024-09-16T15:57:18.545 INFO:teuthology.orchestra.run.smithi151.stderr: -- replaced with /var/log/ceph/ceph-client.admin.72006.log.gz 2024-09-16T15:57:18.545 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.77656.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.77656.log.gz 2024-09-16T15:57:18.545 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79928.log 2024-09-16T15:57:18.546 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.51338.log: gzip 87.6% -5 -- replaced with /var/log/ceph/ceph-client.admin.51338.log.gz --verbose 2024-09-16T15:57:18.546 INFO:teuthology.orchestra.run.smithi151.stderr: -- /var/log/ceph/ceph-client.admin.78920.log 2024-09-16T15:57:18.547 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.79928.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.79928.log.gz 2024-09-16T15:57:18.547 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83620.log 2024-09-16T15:57:18.548 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.78920.log: gzip -5 87.2% --verbose -- replaced with /var/log/ceph/ceph-client.admin.78920.log.gz -- 2024-09-16T15:57:18.548 INFO:teuthology.orchestra.run.smithi151.stderr: /var/log/ceph/ceph-client.admin.60068.log 2024-09-16T15:57:18.549 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.83620.log: gzip -5 87.1% --verbose -- replaced with /var/log/ceph/ceph-client.admin.83620.log.gz -- 2024-09-16T15:57:18.549 INFO:teuthology.orchestra.run.smithi151.stderr: /var/log/ceph/ceph-client.admin.71964.log 2024-09-16T15:57:18.550 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.60068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57473.log 87.0% 2024-09-16T15:57:18.550 INFO:teuthology.orchestra.run.smithi151.stderr: -- replaced with /var/log/ceph/ceph-client.admin.60068.log.gz 2024-09-16T15:57:18.551 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.71964.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.71964.log.gz 2024-09-16T15:57:18.551 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42853.log 2024-09-16T15:57:18.552 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.57473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49156.log 2024-09-16T15:57:18.552 INFO:teuthology.orchestra.run.smithi151.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57473.log.gz 2024-09-16T15:57:18.553 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.42853.log: 87.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.42853.log.gz 2024-09-16T15:57:18.553 INFO:teuthology.orchestra.run.smithi151.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.72762.log 2024-09-16T15:57:18.554 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.49156.log: gzip -5 --verbose -- 87.5% /var/log/ceph/ceph-client.admin.75979.log -- replaced with /var/log/ceph/ceph-client.admin.49156.log.gz 2024-09-16T15:57:18.554 INFO:teuthology.orchestra.run.smithi151.stderr: 2024-09-16T15:57:18.555 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.72762.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.72762.log.gz 2024-09-16T15:57:18.555 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79718.log 2024-09-16T15:57:18.556 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.75979.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.75979.log.gz 2024-09-16T15:57:18.556 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74888.log 2024-09-16T15:57:18.557 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.79718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49324.log 2024-09-16T15:57:18.557 INFO:teuthology.orchestra.run.smithi151.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.79718.log.gz 2024-09-16T15:57:18.558 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.74888.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.74888.log.gz 2024-09-16T15:57:18.558 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77782.log 2024-09-16T15:57:18.558 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.49324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48823.log 2024-09-16T15:57:18.559 INFO:teuthology.orchestra.run.smithi151.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49324.log.gz 2024-09-16T15:57:18.560 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.77782.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.77782.log.gz 2024-09-16T15:57:18.560 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36013.log 2024-09-16T15:57:18.560 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.48823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76441.log 2024-09-16T15:57:18.561 INFO:teuthology.orchestra.run.smithi151.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.48823.log.gz 2024-09-16T15:57:18.562 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.36013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42083.log 2024-09-16T15:57:18.562 INFO:teuthology.orchestra.run.smithi151.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.36013.log.gz 2024-09-16T15:57:18.563 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.76441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43165.log 2024-09-16T15:57:18.563 INFO:teuthology.orchestra.run.smithi151.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.76441.log.gz 2024-09-16T15:57:18.564 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.42083.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42083.log.gz 2024-09-16T15:57:18.564 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63501.log 2024-09-16T15:57:18.565 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.43165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74032.log 2024-09-16T15:57:18.565 INFO:teuthology.orchestra.run.smithi151.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.43165.log.gz 2024-09-16T15:57:18.566 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.63501.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.63501.log.gz 2024-09-16T15:57:18.566 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66601.log 2024-09-16T15:57:18.567 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.74032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35746.log 2024-09-16T15:57:18.567 INFO:teuthology.orchestra.run.smithi151.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.74032.log.gz 2024-09-16T15:57:18.568 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.66601.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.66601.log.gz 2024-09-16T15:57:18.568 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56018.log 2024-09-16T15:57:18.569 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.35746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66097.log 2024-09-16T15:57:18.569 INFO:teuthology.orchestra.run.smithi151.stderr: 75.6% -- replaced with /var/log/ceph/ceph-client.admin.35746.log.gz 2024-09-16T15:57:18.570 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.56018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56102.log 2024-09-16T15:57:18.570 INFO:teuthology.orchestra.run.smithi151.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.56018.log.gz 2024-09-16T15:57:18.570 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.66097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52109.log 2024-09-16T15:57:18.571 INFO:teuthology.orchestra.run.smithi151.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.66097.log.gz 2024-09-16T15:57:18.571 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.56102.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.56102.log.gz 2024-09-16T15:57:18.572 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60278.log 2024-09-16T15:57:18.573 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.52109.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83170.log 2024-09-16T15:57:18.573 INFO:teuthology.orchestra.run.smithi151.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.52109.log.gz 2024-09-16T15:57:18.574 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.60278.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.60278.log.gz 2024-09-16T15:57:18.574 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48035.log 2024-09-16T15:57:18.575 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.83170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68715.log 2024-09-16T15:57:18.575 INFO:teuthology.orchestra.run.smithi151.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.83170.log.gz 2024-09-16T15:57:18.576 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.48035.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.48035.log.gz 2024-09-16T15:57:18.576 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54162.log 2024-09-16T15:57:18.577 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.68715.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.68715.log.gz 2024-09-16T15:57:18.577 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74491.log 2024-09-16T15:57:18.578 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.54162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77012.log 87.1% 2024-09-16T15:57:18.578 INFO:teuthology.orchestra.run.smithi151.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54162.log.gz 2024-09-16T15:57:18.579 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.74491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44489.log 2024-09-16T15:57:18.579 INFO:teuthology.orchestra.run.smithi151.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.74491.log.gz 2024-09-16T15:57:18.579 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.77012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.58554.log 2024-09-16T15:57:18.579 INFO:teuthology.orchestra.run.smithi151.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.77012.log.gz 2024-09-16T15:57:18.580 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.0.44489.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.44489.log.gz 2024-09-16T15:57:18.580 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80537.log 2024-09-16T15:57:18.581 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.0.58554.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.58554.log.gz 2024-09-16T15:57:18.581 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56939.log 2024-09-16T15:57:18.582 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.80537.log: gzip -5 --verbose 87.1% -- -- replaced with /var/log/ceph/ceph-client.admin.80537.log.gz 2024-09-16T15:57:18.582 INFO:teuthology.orchestra.run.smithi151.stderr: /var/log/ceph/ceph-client.admin.41869.log 2024-09-16T15:57:18.583 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.56939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42847.log 2024-09-16T15:57:18.583 INFO:teuthology.orchestra.run.smithi151.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56939.log.gz 2024-09-16T15:57:18.584 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.41869.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41869.log.gz 2024-09-16T15:57:18.584 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72342.log 2024-09-16T15:57:18.585 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.42847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51932.log 2024-09-16T15:57:18.585 INFO:teuthology.orchestra.run.smithi151.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.42847.log.gz 2024-09-16T15:57:18.586 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.72342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50456.log 2024-09-16T15:57:18.586 INFO:teuthology.orchestra.run.smithi151.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.72342.log.gz 2024-09-16T15:57:18.587 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.51932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47951.log 2024-09-16T15:57:18.587 INFO:teuthology.orchestra.run.smithi151.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51932.log.gz 2024-09-16T15:57:18.588 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.50456.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50456.log.gz 2024-09-16T15:57:18.588 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68337.log 2024-09-16T15:57:18.589 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.47951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81806.log 2024-09-16T15:57:18.589 INFO:teuthology.orchestra.run.smithi151.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47951.log.gz 2024-09-16T15:57:18.590 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.68337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57694.log 2024-09-16T15:57:18.590 INFO:teuthology.orchestra.run.smithi151.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.68337.log.gz 2024-09-16T15:57:18.590 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.81806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42503.log 2024-09-16T15:57:18.590 INFO:teuthology.orchestra.run.smithi151.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.81806.log.gz 2024-09-16T15:57:18.591 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.57694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60404.log 2024-09-16T15:57:18.591 INFO:teuthology.orchestra.run.smithi151.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.57694.log.gz 2024-09-16T15:57:18.592 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.42503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81494.log 2024-09-16T15:57:18.592 INFO:teuthology.orchestra.run.smithi151.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.42503.log.gz 2024-09-16T15:57:18.593 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.60404.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.60404.log.gz 2024-09-16T15:57:18.593 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80495.log 2024-09-16T15:57:18.594 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.81494.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.81494.log.gz 2024-09-16T15:57:18.594 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66433.log 2024-09-16T15:57:18.595 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.80495.log: gzip 86.7% -- replaced with /var/log/ceph/ceph-client.admin.80495.log.gz 2024-09-16T15:57:18.595 INFO:teuthology.orchestra.run.smithi151.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.64031.log 2024-09-16T15:57:18.596 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.66433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43189.log 2024-09-16T15:57:18.596 INFO:teuthology.orchestra.run.smithi151.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.66433.log.gz 2024-09-16T15:57:18.597 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.64031.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.64031.log.gz 2024-09-16T15:57:18.597 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56312.log 2024-09-16T15:57:18.598 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.43189.log: gzip 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43189.log.gz 2024-09-16T15:57:18.598 INFO:teuthology.orchestra.run.smithi151.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.46627.log 2024-09-16T15:57:18.599 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.56312.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.56312.log.gz 2024-09-16T15:57:18.599 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78962.log 2024-09-16T15:57:18.600 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.46627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75751.log 2024-09-16T15:57:18.600 INFO:teuthology.orchestra.run.smithi151.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46627.log.gz 2024-09-16T15:57:18.601 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.78962.log: gzip 87.1% -- replaced with /var/log/ceph/ceph-client.admin.78962.log.gz -5 2024-09-16T15:57:18.601 INFO:teuthology.orchestra.run.smithi151.stderr: --verbose -- /var/log/ceph/ceph-client.admin.53296.log 2024-09-16T15:57:18.601 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.75751.log: gzip -5 --verbose -- 87.1% /var/log/ceph/ceph-client.admin.56228.log -- replaced with /var/log/ceph/ceph-client.admin.75751.log.gz 2024-09-16T15:57:18.602 INFO:teuthology.orchestra.run.smithi151.stderr: 2024-09-16T15:57:18.602 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.53296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46458.log 2024-09-16T15:57:18.603 INFO:teuthology.orchestra.run.smithi151.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53296.log.gz 2024-09-16T15:57:18.603 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.56228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52501.log 2024-09-16T15:57:18.603 INFO:teuthology.orchestra.run.smithi151.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.56228.log.gz 2024-09-16T15:57:18.604 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.46458.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.46458.log.gz 2024-09-16T15:57:18.604 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67433.log 2024-09-16T15:57:18.605 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.52501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81183.log 2024-09-16T15:57:18.605 INFO:teuthology.orchestra.run.smithi151.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.52501.log.gz 2024-09-16T15:57:18.606 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.67433.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.67433.log.gz 2024-09-16T15:57:18.606 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60965.log 2024-09-16T15:57:18.607 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.81183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43763.log 2024-09-16T15:57:18.607 INFO:teuthology.orchestra.run.smithi151.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.81183.log.gz 2024-09-16T15:57:18.608 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.60965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58300.log 2024-09-16T15:57:18.608 INFO:teuthology.orchestra.run.smithi151.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.60965.log.gz 2024-09-16T15:57:18.609 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.43763.log: gzip -5 --verbose 87.4% -- -- replaced with /var/log/ceph/ceph-client.admin.43763.log.gz 2024-09-16T15:57:18.609 INFO:teuthology.orchestra.run.smithi151.stderr: /var/log/ceph/ceph-client.admin.68823.log 2024-09-16T15:57:18.610 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.58300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53872.log 2024-09-16T15:57:18.610 INFO:teuthology.orchestra.run.smithi151.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.58300.log.gz 2024-09-16T15:57:18.611 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.68823.log: 87.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.68823.log.gz 2024-09-16T15:57:18.611 INFO:teuthology.orchestra.run.smithi151.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.77698.log 2024-09-16T15:57:18.612 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.53872.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53872.log.gz 2024-09-16T15:57:18.612 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54867.log 2024-09-16T15:57:18.612 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.77698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48545.log 2024-09-16T15:57:18.613 INFO:teuthology.orchestra.run.smithi151.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.77698.log.gz 2024-09-16T15:57:18.613 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.54867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52711.log 2024-09-16T15:57:18.614 INFO:teuthology.orchestra.run.smithi151.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.54867.log.gz 2024-09-16T15:57:18.614 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.48545.log: 87.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.48545.log.gz 2024-09-16T15:57:18.615 INFO:teuthology.orchestra.run.smithi151.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.72090.log 2024-09-16T15:57:18.615 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.52711.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.52711.log.gz 2024-09-16T15:57:18.616 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59275.log 2024-09-16T15:57:18.617 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.72090.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.72090.log.gz 2024-09-16T15:57:18.617 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41955.log 2024-09-16T15:57:18.617 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.59275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65508.log 2024-09-16T15:57:18.618 INFO:teuthology.orchestra.run.smithi151.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.59275.log.gz 2024-09-16T15:57:18.618 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.41955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58090.log 87.4% 2024-09-16T15:57:18.618 INFO:teuthology.orchestra.run.smithi151.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41955.log.gz 2024-09-16T15:57:18.619 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.65508.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.65508.log.gz 2024-09-16T15:57:18.619 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66685.log 2024-09-16T15:57:18.620 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.58090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42587.log 2024-09-16T15:57:18.620 INFO:teuthology.orchestra.run.smithi151.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.58090.log.gz 2024-09-16T15:57:18.621 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.66685.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.66685.log.gzgzip 2024-09-16T15:57:18.621 INFO:teuthology.orchestra.run.smithi151.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.50498.log 2024-09-16T15:57:18.622 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.42587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52627.log 2024-09-16T15:57:18.622 INFO:teuthology.orchestra.run.smithi151.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42587.log.gz 2024-09-16T15:57:18.623 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.50498.log: gzip -5 --verbose 87.4% -- -- replaced with /var/log/ceph/ceph-client.admin.50498.log.gz /var/log/ceph/ceph-client.admin.55287.log 2024-09-16T15:57:18.623 INFO:teuthology.orchestra.run.smithi151.stderr: 2024-09-16T15:57:18.624 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.52627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42850.log 2024-09-16T15:57:18.624 INFO:teuthology.orchestra.run.smithi151.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.52627.log.gz 2024-09-16T15:57:18.624 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.55287.log: gzip 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55287.log.gz -5 2024-09-16T15:57:18.625 INFO:teuthology.orchestra.run.smithi151.stderr: --verbose -- /var/log/ceph/ceph-client.admin.65887.log 2024-09-16T15:57:18.625 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.42850.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.42850.log.gz 2024-09-16T15:57:18.626 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61218.log 2024-09-16T15:57:18.626 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.65887.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.65887.log.gz 2024-09-16T15:57:18.627 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56999.log 2024-09-16T15:57:18.627 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.61218.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.61218.log.gzgzip 2024-09-16T15:57:18.627 INFO:teuthology.orchestra.run.smithi151.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.62938.log 2024-09-16T15:57:18.628 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.56999.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.56999.log.gz 2024-09-16T15:57:18.628 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66517.log 2024-09-16T15:57:18.629 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.62938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83506.log 2024-09-16T15:57:18.629 INFO:teuthology.orchestra.run.smithi151.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.62938.log.gz 2024-09-16T15:57:18.630 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.66517.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.66517.log.gz 2024-09-16T15:57:18.630 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55413.log 2024-09-16T15:57:18.631 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.83506.log: gzip -5 --verbose -- 86.9% /var/log/ceph/ceph-client.admin.57960.log 2024-09-16T15:57:18.631 INFO:teuthology.orchestra.run.smithi151.stderr: -- replaced with /var/log/ceph/ceph-client.admin.83506.log.gz 2024-09-16T15:57:18.632 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.55413.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.55413.log.gz 2024-09-16T15:57:18.632 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65845.log 2024-09-16T15:57:18.633 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.57960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59794.log 2024-09-16T15:57:18.633 INFO:teuthology.orchestra.run.smithi151.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.57960.log.gz 2024-09-16T15:57:18.634 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.65845.log: gzip -5 87.0% --verbose -- replaced with /var/log/ceph/ceph-client.admin.65845.log.gz -- 2024-09-16T15:57:18.634 INFO:teuthology.orchestra.run.smithi151.stderr: /var/log/ceph/ceph-client.admin.46585.log 2024-09-16T15:57:18.635 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.59794.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.59794.log.gz 2024-09-16T15:57:18.635 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82750.log 2024-09-16T15:57:18.636 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.46585.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46585.log.gz 2024-09-16T15:57:18.636 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47783.log 2024-09-16T15:57:18.636 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.82750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55883.log 2024-09-16T15:57:18.637 INFO:teuthology.orchestra.run.smithi151.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.82750.log.gz 2024-09-16T15:57:18.637 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.47783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50021.log 2024-09-16T15:57:18.637 INFO:teuthology.orchestra.run.smithi151.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47783.log.gz 2024-09-16T15:57:18.638 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.55883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70018.log 2024-09-16T15:57:18.639 INFO:teuthology.orchestra.run.smithi151.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.55883.log.gz 2024-09-16T15:57:18.639 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.50021.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50021.log.gz 2024-09-16T15:57:18.640 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46374.log 2024-09-16T15:57:18.640 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.70018.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.70018.log.gz 2024-09-16T15:57:18.640 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79340.log 2024-09-16T15:57:18.642 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.46374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68463.log 2024-09-16T15:57:18.642 INFO:teuthology.orchestra.run.smithi151.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.46374.log.gz 2024-09-16T15:57:18.643 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.79340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77197.log 2024-09-16T15:57:18.643 INFO:teuthology.orchestra.run.smithi151.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.79340.log.gz 2024-09-16T15:57:18.643 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.68463.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.68463.log.gz 2024-09-16T15:57:18.644 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46925.log 2024-09-16T15:57:18.644 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.77197.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.77197.log.gz 2024-09-16T15:57:18.644 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71427.log 2024-09-16T15:57:18.645 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.46925.log: gzip 87.7% -5 -- replaced with /var/log/ceph/ceph-client.admin.46925.log.gz --verbose 2024-09-16T15:57:18.645 INFO:teuthology.orchestra.run.smithi151.stderr: -- /var/log/ceph/ceph-client.admin.41999.log 2024-09-16T15:57:18.646 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.71427.log: gzip -5 --verbose -- 87.9% /var/log/ceph/ceph-client.admin.58636.log -- replaced with /var/log/ceph/ceph-client.admin.71427.log.gz 2024-09-16T15:57:18.646 INFO:teuthology.orchestra.run.smithi151.stderr: 2024-09-16T15:57:18.647 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.41999.log: gzip 87.3% -5 -- replaced with /var/log/ceph/ceph-client.admin.41999.log.gz 2024-09-16T15:57:18.647 INFO:teuthology.orchestra.run.smithi151.stderr: --verbose -- /var/log/ceph/ceph-client.admin.47195.log 2024-09-16T15:57:18.648 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.58636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49804.log 2024-09-16T15:57:18.648 INFO:teuthology.orchestra.run.smithi151.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.58636.log.gz 2024-09-16T15:57:18.649 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.47195.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47195.log.gz 2024-09-16T15:57:18.649 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-16T15:57:18.650 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.49804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65551.log 2024-09-16T15:57:18.650 INFO:teuthology.orchestra.run.smithi151.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49804.log.gz 2024-09-16T15:57:18.651 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43973.log 2024-09-16T15:57:18.651 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.65551.log: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.65551.log.gz 2024-09-16T15:57:18.652 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44015.log 2024-09-16T15:57:18.653 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.43973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46171.log 2024-09-16T15:57:18.653 INFO:teuthology.orchestra.run.smithi151.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43973.log.gz 2024-09-16T15:57:18.654 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.44015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41911.log 2024-09-16T15:57:18.654 INFO:teuthology.orchestra.run.smithi151.stderr: 90.0% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-16T15:57:18.654 INFO:teuthology.orchestra.run.smithi151.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.44015.log.gz 2024-09-16T15:57:18.655 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.46171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56771.log 2024-09-16T15:57:18.655 INFO:teuthology.orchestra.run.smithi151.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46171.log.gz 2024-09-16T15:57:18.656 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.41911.log: gzip -5 87.6% --verbose -- replaced with /var/log/ceph/ceph-client.admin.41911.log.gz -- 2024-09-16T15:57:18.656 INFO:teuthology.orchestra.run.smithi151.stderr: /var/log/ceph/ceph-client.admin.80453.log 2024-09-16T15:57:18.657 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.56771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72258.log 2024-09-16T15:57:18.657 INFO:teuthology.orchestra.run.smithi151.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56771.log.gz 2024-09-16T15:57:18.658 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.80453.log: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.80453.log.gz 2024-09-16T15:57:18.658 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65190.log 2024-09-16T15:57:18.659 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.72258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75402.log 2024-09-16T15:57:18.659 INFO:teuthology.orchestra.run.smithi151.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.72258.log.gz 2024-09-16T15:57:18.660 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.65190.log: 84.0% -- replaced with /var/log/ceph/ceph-client.admin.65190.log.gz 2024-09-16T15:57:18.660 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46841.log 2024-09-16T15:57:18.661 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.75402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45547.log 2024-09-16T15:57:18.661 INFO:teuthology.orchestra.run.smithi151.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.75402.log.gz 2024-09-16T15:57:18.661 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.46841.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46841.log.gz 2024-09-16T15:57:18.662 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82223.log 2024-09-16T15:57:18.662 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.45547.log: gzip -5 --verbose -- 87.7% /var/log/ceph/ceph-client.admin.77281.log 2024-09-16T15:57:18.663 INFO:teuthology.orchestra.run.smithi151.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45547.log.gz 2024-09-16T15:57:18.663 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.82223.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.82223.log.gz 2024-09-16T15:57:18.664 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50231.log 2024-09-16T15:57:18.664 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.77281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48503.log 2024-09-16T15:57:18.664 INFO:teuthology.orchestra.run.smithi151.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.77281.log.gz 2024-09-16T15:57:18.665 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.50231.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50231.log.gz 2024-09-16T15:57:18.665 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35722.log 2024-09-16T15:57:18.666 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.48503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61972.log 2024-09-16T15:57:18.666 INFO:teuthology.orchestra.run.smithi151.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.48503.log.gz 2024-09-16T15:57:18.667 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.35722.log: 77.0% -- replaced with /var/log/ceph/ceph-client.admin.35722.log.gz 2024-09-16T15:57:18.667 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45226.log 2024-09-16T15:57:18.668 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.61972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59942.log 2024-09-16T15:57:18.668 INFO:teuthology.orchestra.run.smithi151.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.61972.log.gz 2024-09-16T15:57:18.669 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.45226.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45226.log.gz 2024-09-16T15:57:18.669 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64155.log 2024-09-16T15:57:18.670 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.59942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42835.log 2024-09-16T15:57:18.670 INFO:teuthology.orchestra.run.smithi151.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.59942.log.gz 2024-09-16T15:57:18.671 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.64155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80217.log 2024-09-16T15:57:18.671 INFO:teuthology.orchestra.run.smithi151.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.64155.log.gz 2024-09-16T15:57:18.672 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.42835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82391.log 2024-09-16T15:57:18.672 INFO:teuthology.orchestra.run.smithi151.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.42835.log.gz 2024-09-16T15:57:18.672 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.80217.log: gzip 87.1% -- replaced with /var/log/ceph/ceph-client.admin.80217.log.gz 2024-09-16T15:57:18.673 INFO:teuthology.orchestra.run.smithi151.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.69546.log 2024-09-16T15:57:18.673 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.82391.log: gzip -5 --verbose -- 87.4% /var/log/ceph/ceph-client.admin.65231.log -- replaced with /var/log/ceph/ceph-client.admin.82391.log.gz 2024-09-16T15:57:18.673 INFO:teuthology.orchestra.run.smithi151.stderr: 2024-09-16T15:57:18.674 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.69546.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.69546.log.gz 2024-09-16T15:57:18.674 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62812.log 2024-09-16T15:57:18.675 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.65231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65803.log 2024-09-16T15:57:18.675 INFO:teuthology.orchestra.run.smithi151.stderr: 84.4% -- replaced with /var/log/ceph/ceph-client.admin.65231.log.gz 2024-09-16T15:57:18.676 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.62812.log: 87.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.62812.log.gz 2024-09-16T15:57:18.676 INFO:teuthology.orchestra.run.smithi151.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.62140.log 2024-09-16T15:57:18.677 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.65803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41827.log 2024-09-16T15:57:18.677 INFO:teuthology.orchestra.run.smithi151.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.65803.log.gz 2024-09-16T15:57:18.678 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.62140.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.62140.log.gz 2024-09-16T15:57:18.678 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74678.log 2024-09-16T15:57:18.679 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.41827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72930.log 87.7% 2024-09-16T15:57:18.679 INFO:teuthology.orchestra.run.smithi151.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41827.log.gz 2024-09-16T15:57:18.680 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.74678.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.74678.log.gz 2024-09-16T15:57:18.680 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76525.log 2024-09-16T15:57:18.681 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.72930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79844.log 2024-09-16T15:57:18.681 INFO:teuthology.orchestra.run.smithi151.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.72930.log.gz 2024-09-16T15:57:18.682 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.76525.log: 87.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.76525.log.gz 2024-09-16T15:57:18.682 INFO:teuthology.orchestra.run.smithi151.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.80369.log 2024-09-16T15:57:18.683 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.79844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49576.log 2024-09-16T15:57:18.683 INFO:teuthology.orchestra.run.smithi151.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.79844.log.gz 2024-09-16T15:57:18.684 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.80369.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.80369.log.gz 2024-09-16T15:57:18.684 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50984.log 2024-09-16T15:57:18.685 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.49576.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49576.log.gz 2024-09-16T15:57:18.685 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.76321.log 2024-09-16T15:57:18.686 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.50984.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50984.log.gz 2024-09-16T15:57:18.686 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52543.log 2024-09-16T15:57:18.686 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.0.76321.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.76321.log.gz 2024-09-16T15:57:18.686 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68295.log 2024-09-16T15:57:18.687 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.52543.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.52543.log.gz 2024-09-16T15:57:18.687 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82349.log 2024-09-16T15:57:18.688 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.68295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66013.log 2024-09-16T15:57:18.688 INFO:teuthology.orchestra.run.smithi151.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.68295.log.gz 2024-09-16T15:57:18.689 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.82349.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.82349.log.gz 2024-09-16T15:57:18.689 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59401.log 2024-09-16T15:57:18.690 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.66013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72132.log 2024-09-16T15:57:18.690 INFO:teuthology.orchestra.run.smithi151.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.66013.log.gz 2024-09-16T15:57:18.691 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.59401.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.59401.log.gz 2024-09-16T15:57:18.691 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78498.log 2024-09-16T15:57:18.692 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.72132.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.72132.log.gz 2024-09-16T15:57:18.692 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47867.log 2024-09-16T15:57:18.693 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.78498.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.78498.log.gz 2024-09-16T15:57:18.693 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47321.log 2024-09-16T15:57:18.694 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.47867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62182.log 2024-09-16T15:57:18.694 INFO:teuthology.orchestra.run.smithi151.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.47867.log.gz 2024-09-16T15:57:18.695 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.47321.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47321.log.gz 2024-09-16T15:57:18.695 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58174.log 2024-09-16T15:57:18.696 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.62182.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.62182.log.gz 2024-09-16T15:57:18.696 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73451.log 2024-09-16T15:57:18.696 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.58174.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.58174.log.gz 2024-09-16T15:57:18.697 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53170.log 2024-09-16T15:57:18.697 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.73451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70729.log 2024-09-16T15:57:18.698 INFO:teuthology.orchestra.run.smithi151.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.73451.log.gz 2024-09-16T15:57:18.698 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.53170.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53170.log.gz 2024-09-16T15:57:18.698 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53338.log 2024-09-16T15:57:18.699 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.70729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70144.log 2024-09-16T15:57:18.699 INFO:teuthology.orchestra.run.smithi151.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.70729.log.gz 2024-09-16T15:57:18.700 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.53338.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53338.log.gz 2024-09-16T15:57:18.700 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62434.log 2024-09-16T15:57:18.701 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.70144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83380.log 2024-09-16T15:57:18.701 INFO:teuthology.orchestra.run.smithi151.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.70144.log.gz 2024-09-16T15:57:18.702 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.62434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52327.log 2024-09-16T15:57:18.702 INFO:teuthology.orchestra.run.smithi151.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.62434.log.gz 2024-09-16T15:57:18.703 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.83380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49366.log 2024-09-16T15:57:18.703 INFO:teuthology.orchestra.run.smithi151.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.83380.log.gz 2024-09-16T15:57:18.704 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.52327.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.52327.log.gz 2024-09-16T15:57:18.704 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57294.log 2024-09-16T15:57:18.705 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.49366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43170.log 87.5% 2024-09-16T15:57:18.705 INFO:teuthology.orchestra.run.smithi151.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49366.log.gz 2024-09-16T15:57:18.706 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.57294.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.57294.log.gz 2024-09-16T15:57:18.706 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59443.log 2024-09-16T15:57:18.707 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.43170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49492.log 2024-09-16T15:57:18.707 INFO:teuthology.orchestra.run.smithi151.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43170.log.gz 2024-09-16T15:57:18.708 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.59443.log: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.59443.log.gzgzip 2024-09-16T15:57:18.708 INFO:teuthology.orchestra.run.smithi151.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.72888.log 2024-09-16T15:57:18.709 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.49492.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49492.log.gz 2024-09-16T15:57:18.709 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36078.log 2024-09-16T15:57:18.710 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.72888.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.72888.log.gz 2024-09-16T15:57:18.710 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60737.log 2024-09-16T15:57:18.711 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.36078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60695.log 2024-09-16T15:57:18.711 INFO:teuthology.orchestra.run.smithi151.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.36078.log.gz 2024-09-16T15:57:18.712 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.60737.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.60737.log.gz 2024-09-16T15:57:18.712 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69832.log 2024-09-16T15:57:18.712 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.60695.log: gzip -5 87.4% --verbose -- replaced with /var/log/ceph/ceph-client.admin.60695.log.gz -- 2024-09-16T15:57:18.713 INFO:teuthology.orchestra.run.smithi151.stderr: /var/log/ceph/ceph-client.admin.72678.log 2024-09-16T15:57:18.713 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.69832.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.69832.log.gz 2024-09-16T15:57:18.713 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51380.log 2024-09-16T15:57:18.714 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.72678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53525.log 2024-09-16T15:57:18.714 INFO:teuthology.orchestra.run.smithi151.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.72678.log.gz 2024-09-16T15:57:18.715 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.51380.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51380.log.gz 2024-09-16T15:57:18.715 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71681.log 2024-09-16T15:57:18.716 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.53525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49450.log 2024-09-16T15:57:18.716 INFO:teuthology.orchestra.run.smithi151.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53525.log.gz 2024-09-16T15:57:18.717 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.71681.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.71681.log.gz 2024-09-16T15:57:18.717 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46883.log 2024-09-16T15:57:18.718 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.49450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52669.log 2024-09-16T15:57:18.718 INFO:teuthology.orchestra.run.smithi151.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49450.log.gz 2024-09-16T15:57:18.719 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.46883.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46883.log.gz 2024-09-16T15:57:18.719 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64748.log 2024-09-16T15:57:18.720 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.52669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44508.log 2024-09-16T15:57:18.720 INFO:teuthology.orchestra.run.smithi151.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.52669.log.gz 2024-09-16T15:57:18.721 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.64748.log: 84.6% -- replaced with /var/log/ceph/ceph-client.admin.64748.log.gz 2024-09-16T15:57:18.721 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63729.log 2024-09-16T15:57:18.721 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.0.44508.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.44508.log.gz 2024-09-16T15:57:18.722 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70603.log 2024-09-16T15:57:18.722 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.63729.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.63729.log.gz 2024-09-16T15:57:18.722 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57652.log 2024-09-16T15:57:18.723 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.70603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66475.log 2024-09-16T15:57:18.723 INFO:teuthology.orchestra.run.smithi151.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.70603.log.gz 2024-09-16T15:57:18.724 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.57652.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.57652.log.gz 2024-09-16T15:57:18.724 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64830.log 2024-09-16T15:57:18.725 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.66475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74533.log 2024-09-16T15:57:18.725 INFO:teuthology.orchestra.run.smithi151.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.66475.log.gz 2024-09-16T15:57:18.726 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.64830.log: 84.4% -- replaced with /var/log/ceph/ceph-client.admin.64830.log.gz 2024-09-16T15:57:18.726 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81368.log 2024-09-16T15:57:18.727 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.74533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49618.log 2024-09-16T15:57:18.727 INFO:teuthology.orchestra.run.smithi151.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.74533.log.gz 2024-09-16T15:57:18.728 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.81368.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.81368.log.gz 2024-09-16T15:57:18.728 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62266.log 2024-09-16T15:57:18.729 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.49618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60821.log 2024-09-16T15:57:18.729 INFO:teuthology.orchestra.run.smithi151.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.49618.log.gz 2024-09-16T15:57:18.730 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.62266.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.62266.log.gz 2024-09-16T15:57:18.730 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71000.log 2024-09-16T15:57:18.731 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.60821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81704.log 2024-09-16T15:57:18.731 INFO:teuthology.orchestra.run.smithi151.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.60821.log.gz 2024-09-16T15:57:18.731 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.71000.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.71000.log.gz 2024-09-16T15:57:18.732 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81410.log 2024-09-16T15:57:18.732 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.81704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79886.log 2024-09-16T15:57:18.733 INFO:teuthology.orchestra.run.smithi151.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.81704.log.gz 2024-09-16T15:57:18.733 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.81410.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.81410.log.gz 2024-09-16T15:57:18.733 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73131.log 2024-09-16T15:57:18.734 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.79886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44641.log 2024-09-16T15:57:18.734 INFO:teuthology.orchestra.run.smithi151.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.79886.log.gz 2024-09-16T15:57:18.735 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.73131.log: gzip -5 87.1% --verbose -- replaced with /var/log/ceph/ceph-client.admin.73131.log.gz -- 2024-09-16T15:57:18.735 INFO:teuthology.orchestra.run.smithi151.stderr: /var/log/ceph/ceph-client.admin.73661.log 2024-09-16T15:57:18.736 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.44641.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44641.log.gz 2024-09-16T15:57:18.736 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75067.log 2024-09-16T15:57:18.737 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.73661.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.73661.log.gz 2024-09-16T15:57:18.737 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63687.log 2024-09-16T15:57:18.738 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.75067.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.75067.log.gz 2024-09-16T15:57:18.738 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69093.log 2024-09-16T15:57:18.739 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.63687.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.63687.log.gz 2024-09-16T15:57:18.739 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73577.log 2024-09-16T15:57:18.740 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.69093.log: gzip 87.0% -- replaced with /var/log/ceph/ceph-client.admin.69093.log.gz -5 2024-09-16T15:57:18.740 INFO:teuthology.orchestra.run.smithi151.stderr: --verbose -- /var/log/ceph/ceph-client.admin.69622.log 2024-09-16T15:57:18.741 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.73577.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.73577.log.gz 2024-09-16T15:57:18.741 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74972.log 2024-09-16T15:57:18.742 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.69622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45352.log 2024-09-16T15:57:18.742 INFO:teuthology.orchestra.run.smithi151.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.69622.log.gz 2024-09-16T15:57:18.743 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.74972.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.74972.log.gz 2024-09-16T15:57:18.743 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61134.log 2024-09-16T15:57:18.744 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.45352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58804.log 2024-09-16T15:57:18.744 INFO:teuthology.orchestra.run.smithi151.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45352.log.gz 2024-09-16T15:57:18.745 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.61134.log: 87.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.61134.log.gz 2024-09-16T15:57:18.745 INFO:teuthology.orchestra.run.smithi151.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.71732.log 2024-09-16T15:57:18.746 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.58804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54447.log 87.6% 2024-09-16T15:57:18.746 INFO:teuthology.orchestra.run.smithi151.stderr: -- replaced with /var/log/ceph/ceph-client.admin.58804.log.gz 2024-09-16T15:57:18.746 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.71732.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.71732.log.gz 2024-09-16T15:57:18.747 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73804.log 2024-09-16T15:57:18.747 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.54447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47705.log 2024-09-16T15:57:18.747 INFO:teuthology.orchestra.run.smithi151.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54447.log.gz 2024-09-16T15:57:18.748 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.73804.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.73804.log.gz 2024-09-16T15:57:18.748 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45268.log 2024-09-16T15:57:18.749 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.0.47705.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.47705.log.gz 2024-09-16T15:57:18.749 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72972.log 2024-09-16T15:57:18.750 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.45268.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45268.log.gz 2024-09-16T15:57:18.750 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47993.log 2024-09-16T15:57:18.751 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.72972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73173.log 2024-09-16T15:57:18.751 INFO:teuthology.orchestra.run.smithi151.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.72972.log.gz 2024-09-16T15:57:18.752 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.47993.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47993.log.gz 2024-09-16T15:57:18.752 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46967.log 2024-09-16T15:57:18.753 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.73173.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.73173.log.gz 2024-09-16T15:57:18.753 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81764.log 2024-09-16T15:57:18.754 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.46967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56897.log 2024-09-16T15:57:18.754 INFO:teuthology.orchestra.run.smithi151.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46967.log.gz 2024-09-16T15:57:18.755 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.81764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80639.log 2024-09-16T15:57:18.755 INFO:teuthology.orchestra.run.smithi151.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.81764.log.gz 2024-09-16T15:57:18.755 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.56897.log: gzip 87.7% -5 -- replaced with /var/log/ceph/ceph-client.admin.56897.log.gz --verbose 2024-09-16T15:57:18.756 INFO:teuthology.orchestra.run.smithi151.stderr: -- /var/log/ceph/ceph-client.admin.61642.log 2024-09-16T15:57:18.756 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.80639.log: gzip 86.4% -5 -- replaced with /var/log/ceph/ceph-client.admin.80639.log.gz --verbose 2024-09-16T15:57:18.757 INFO:teuthology.orchestra.run.smithi151.stderr: -- /var/log/ceph/ceph-client.admin.42839.log 2024-09-16T15:57:18.757 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.61642.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.61642.log.gz 2024-09-16T15:57:18.757 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61926.log 2024-09-16T15:57:18.758 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.42839.log: 87.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.42839.log.gz 2024-09-16T15:57:18.758 INFO:teuthology.orchestra.run.smithi151.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.63459.log 2024-09-16T15:57:18.759 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.61926.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.61926.log.gz 2024-09-16T15:57:18.759 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49198.log 2024-09-16T15:57:18.760 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.63459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47363.log 2024-09-16T15:57:18.760 INFO:teuthology.orchestra.run.smithi151.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.63459.log.gz 2024-09-16T15:57:18.761 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.49198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78095.log 2024-09-16T15:57:18.761 INFO:teuthology.orchestra.run.smithi151.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49198.log.gz 2024-09-16T15:57:18.762 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.47363.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.47363.log.gz 2024-09-16T15:57:18.762 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-09-16T15:57:18.763 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.78095.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.78095.log.gz 2024-09-16T15:57:18.763 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52837.log 2024-09-16T15:57:18.764 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74762.log 2024-09-16T15:57:18.765 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.52837.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.52837.log.gz 2024-09-16T15:57:18.765 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49941.log 2024-09-16T15:57:18.766 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.74762.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.74762.log.gz 2024-09-16T15:57:18.766 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50275.log 2024-09-16T15:57:18.767 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.49941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66882.log 2024-09-16T15:57:18.768 INFO:teuthology.orchestra.run.smithi151.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49941.log.gz 2024-09-16T15:57:18.769 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.50275.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50275.log.gz 2024-09-16T15:57:18.769 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50368.log 2024-09-16T15:57:18.770 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.66882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.69011.log 2024-09-16T15:57:18.770 INFO:teuthology.orchestra.run.smithi151.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.66882.log.gz 2024-09-16T15:57:18.771 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.50368.log: gzip 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50368.log.gz 2024-09-16T15:57:18.771 INFO:teuthology.orchestra.run.smithi151.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.80847.log 2024-09-16T15:57:18.772 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.0.69011.log: -- /var/log/ceph/ceph-client.admin.79172.log 2024-09-16T15:57:18.772 INFO:teuthology.orchestra.run.smithi151.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.69011.log.gz 2024-09-16T15:57:18.773 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.80847.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.80847.log.gz 2024-09-16T15:57:18.773 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71816.log 2024-09-16T15:57:18.774 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.79172.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.79172.log.gz 2024-09-16T15:57:18.774 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78836.log 2024-09-16T15:57:18.775 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.71816.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69345.log 2024-09-16T15:57:18.775 INFO:teuthology.orchestra.run.smithi151.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.71816.log.gz 2024-09-16T15:57:18.776 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.78836.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.78836.log.gz 2024-09-16T15:57:18.776 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42041.log 2024-09-16T15:57:18.777 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.69345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81057.log 2024-09-16T15:57:18.778 INFO:teuthology.orchestra.run.smithi151.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.69345.log.gz 2024-09-16T15:57:18.778 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.42041.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42041.log.gz 2024-09-16T15:57:18.778 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47405.log 2024-09-16T15:57:18.779 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.81057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78221.log 2024-09-16T15:57:18.780 INFO:teuthology.orchestra.run.smithi151.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.81057.log.gz 2024-09-16T15:57:18.781 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.47405.log: gzip 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47405.log.gz -5 2024-09-16T15:57:18.781 INFO:teuthology.orchestra.run.smithi151.stderr: --verbose -- /var/log/ceph/ceph-client.admin.67517.log 2024-09-16T15:57:18.782 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.78221.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.78221.log.gz 2024-09-16T15:57:18.782 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62056.log 2024-09-16T15:57:18.783 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.67517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45394.log 2024-09-16T15:57:18.783 INFO:teuthology.orchestra.run.smithi151.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.67517.log.gz 2024-09-16T15:57:18.784 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.62056.log: gzip -5 --verbose 87.2% -- /var/log/ceph/ceph-client.admin.46711.log -- replaced with /var/log/ceph/ceph-client.admin.62056.log.gz 2024-09-16T15:57:18.784 INFO:teuthology.orchestra.run.smithi151.stderr: 2024-09-16T15:57:18.785 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.45394.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45394.log.gz 2024-09-16T15:57:18.785 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49762.log 2024-09-16T15:57:18.786 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.46711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75358.log 2024-09-16T15:57:18.786 INFO:teuthology.orchestra.run.smithi151.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46711.log.gz 2024-09-16T15:57:18.787 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.49762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48697.log 2024-09-16T15:57:18.788 INFO:teuthology.orchestra.run.smithi151.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49762.log.gz 2024-09-16T15:57:18.789 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.75358.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.75358.log.gz 2024-09-16T15:57:18.789 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64440.log 2024-09-16T15:57:18.789 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.48697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45082.log 2024-09-16T15:57:18.790 INFO:teuthology.orchestra.run.smithi151.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48697.log.gz 2024-09-16T15:57:18.791 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.64440.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.64440.log.gz 2024-09-16T15:57:18.791 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53483.log 2024-09-16T15:57:18.792 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.45082.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45082.log.gz 2024-09-16T15:57:18.792 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50750.log 2024-09-16T15:57:18.793 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.53483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35620.log 2024-09-16T15:57:18.793 INFO:teuthology.orchestra.run.smithi151.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53483.log.gz 2024-09-16T15:57:18.794 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.50750.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50750.log.gz 2024-09-16T15:57:18.794 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43162.log 2024-09-16T15:57:18.795 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.35620.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35620.log.gz 2024-09-16T15:57:18.795 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.79636.log 2024-09-16T15:57:18.796 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.43162.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.43162.log.gz 2024-09-16T15:57:18.797 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48077.log 2024-09-16T15:57:18.797 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.0.79636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68673.log 2024-09-16T15:57:18.798 INFO:teuthology.orchestra.run.smithi151.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.79636.log.gz 2024-09-16T15:57:18.799 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.48077.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.48077.log.gz 2024-09-16T15:57:18.799 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62854.log 2024-09-16T15:57:18.800 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.68673.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.68673.log.gz 2024-09-16T15:57:18.800 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69706.log 2024-09-16T15:57:18.801 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.62854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66727.log 2024-09-16T15:57:18.801 INFO:teuthology.orchestra.run.smithi151.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.62854.log.gz 2024-09-16T15:57:18.802 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.69706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47279.log 2024-09-16T15:57:18.802 INFO:teuthology.orchestra.run.smithi151.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.69706.log.gz 2024-09-16T15:57:18.803 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.66727.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.66727.log.gz 2024-09-16T15:57:18.803 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72804.log 2024-09-16T15:57:18.804 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.47279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75895.log 2024-09-16T15:57:18.805 INFO:teuthology.orchestra.run.smithi151.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47279.log.gz 2024-09-16T15:57:18.806 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.72804.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.72804.log.gz 2024-09-16T15:57:18.806 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60464.log 2024-09-16T15:57:18.807 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.75895.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.75895.log.gz 2024-09-16T15:57:18.807 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58006.log 2024-09-16T15:57:18.808 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.60464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71347.log 2024-09-16T15:57:18.808 INFO:teuthology.orchestra.run.smithi151.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60464.log.gz 2024-09-16T15:57:18.809 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.58006.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.58006.log.gz 2024-09-16T15:57:18.809 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70687.log 2024-09-16T15:57:18.810 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.71347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71637.log 2024-09-16T15:57:18.810 INFO:teuthology.orchestra.run.smithi151.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.71347.log.gz 2024-09-16T15:57:18.811 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.70687.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.70687.log.gz 2024-09-16T15:57:18.811 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52067.log 2024-09-16T15:57:18.812 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.71637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46543.log 2024-09-16T15:57:18.813 INFO:teuthology.orchestra.run.smithi151.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.71637.log.gz 2024-09-16T15:57:18.814 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.52067.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.52067.log.gz 2024-09-16T15:57:18.814 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57168.log 2024-09-16T15:57:18.815 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.46543.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46543.log.gz 2024-09-16T15:57:18.815 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59359.log 2024-09-16T15:57:18.816 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.57168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61791.log 2024-09-16T15:57:18.816 INFO:teuthology.orchestra.run.smithi151.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.57168.log.gz 2024-09-16T15:57:18.817 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.59359.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.59359.log.gz 2024-09-16T15:57:18.817 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83464.log 2024-09-16T15:57:18.818 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.61791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83212.log 2024-09-16T15:57:18.818 INFO:teuthology.orchestra.run.smithi151.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.61791.log.gz 2024-09-16T15:57:18.819 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.83464.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.83464.log.gz 2024-09-16T15:57:18.819 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68165.log 2024-09-16T15:57:18.820 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.83212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73493.log 2024-09-16T15:57:18.821 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.68165.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.83212.log.gz 2024-09-16T15:57:18.821 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73409.log 2024-09-16T15:57:18.821 INFO:teuthology.orchestra.run.smithi151.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.68165.log.gz 2024-09-16T15:57:18.822 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.73493.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.73493.log.gz 2024-09-16T15:57:18.822 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76399.log 2024-09-16T15:57:18.823 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.73409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49660.log 2024-09-16T15:57:18.824 INFO:teuthology.orchestra.run.smithi151.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.73409.log.gz 2024-09-16T15:57:18.825 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.76399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74323.log 2024-09-16T15:57:18.825 INFO:teuthology.orchestra.run.smithi151.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.76399.log.gz 2024-09-16T15:57:18.826 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.49660.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49660.log.gz 2024-09-16T15:57:18.826 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63627.log 2024-09-16T15:57:18.827 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.74323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64654.log 2024-09-16T15:57:18.827 INFO:teuthology.orchestra.run.smithi151.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.74323.log.gz 2024-09-16T15:57:18.828 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.63627.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.63627.log.gz 2024-09-16T15:57:18.828 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77926.log 2024-09-16T15:57:18.829 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.64654.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.64654.log.gz 2024-09-16T15:57:18.829 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82307.log 2024-09-16T15:57:18.830 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.77926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79256.log 2024-09-16T15:57:18.831 INFO:teuthology.orchestra.run.smithi151.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.77926.log.gz 2024-09-16T15:57:18.831 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.82307.log: gzip -5 87.3% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.82307.log.gz /var/log/ceph/ceph-client.admin.63375.log 2024-09-16T15:57:18.832 INFO:teuthology.orchestra.run.smithi151.stderr: 2024-09-16T15:57:18.833 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.79256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48781.log 2024-09-16T15:57:18.833 INFO:teuthology.orchestra.run.smithi151.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.79256.log.gz 2024-09-16T15:57:18.834 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.63375.log: gzip 87.0% -- replaced with /var/log/ceph/ceph-client.admin.63375.log.gz 2024-09-16T15:57:18.834 INFO:teuthology.orchestra.run.smithi151.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.69790.log 2024-09-16T15:57:18.835 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.48781.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83044.log 2024-09-16T15:57:18.835 INFO:teuthology.orchestra.run.smithi151.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.48781.log.gz 2024-09-16T15:57:18.836 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.69790.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.69790.log.gz 2024-09-16T15:57:18.836 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42842.log 2024-09-16T15:57:18.837 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.83044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70102.log 2024-09-16T15:57:18.837 INFO:teuthology.orchestra.run.smithi151.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.83044.log.gz 2024-09-16T15:57:18.838 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.42842.log: gzip -5 87.3% --verbose -- replaced with /var/log/ceph/ceph-client.admin.42842.log.gz 2024-09-16T15:57:18.838 INFO:teuthology.orchestra.run.smithi151.stderr: -- /var/log/ceph/ceph-mon.a.log 2024-09-16T15:57:18.839 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.70102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59132.log 2024-09-16T15:57:18.840 INFO:teuthology.orchestra.run.smithi151.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.70102.log.gz 2024-09-16T15:57:18.840 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83128.log 2024-09-16T15:57:18.842 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.59132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65452.log 2024-09-16T15:57:18.842 INFO:teuthology.orchestra.run.smithi151.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.59132.log.gz 2024-09-16T15:57:18.843 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.83128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55203.log 2024-09-16T15:57:18.843 INFO:teuthology.orchestra.run.smithi151.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.83128.log.gz 2024-09-16T15:57:18.844 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.65452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67983.log 2024-09-16T15:57:18.844 INFO:teuthology.orchestra.run.smithi151.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.65452.log.gz 2024-09-16T15:57:18.845 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.55203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56144.log 2024-09-16T15:57:18.845 INFO:teuthology.orchestra.run.smithi151.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55203.log.gz 2024-09-16T15:57:18.846 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.67983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71126.log 2024-09-16T15:57:18.847 INFO:teuthology.orchestra.run.smithi151.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.67983.log.gz 2024-09-16T15:57:18.848 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.56144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51464.log 2024-09-16T15:57:18.848 INFO:teuthology.orchestra.run.smithi151.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.56144.log.gz 2024-09-16T15:57:18.849 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.71126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53254.log 2024-09-16T15:57:18.849 INFO:teuthology.orchestra.run.smithi151.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.71126.log.gz 2024-09-16T15:57:18.850 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.51464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72300.log 2024-09-16T15:57:18.850 INFO:teuthology.orchestra.run.smithi151.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51464.log.gz 2024-09-16T15:57:18.852 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.53254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79004.log 2024-09-16T15:57:18.852 INFO:teuthology.orchestra.run.smithi151.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53254.log.gz 2024-09-16T15:57:18.852 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.72300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77614.log 2024-09-16T15:57:18.853 INFO:teuthology.orchestra.run.smithi151.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.72300.log.gz 2024-09-16T15:57:18.854 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.79004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49062.log 2024-09-16T15:57:18.854 INFO:teuthology.orchestra.run.smithi151.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.79004.log.gz 2024-09-16T15:57:18.855 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.77614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80579.log 2024-09-16T15:57:18.855 INFO:teuthology.orchestra.run.smithi151.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.77614.log.gz 2024-09-16T15:57:18.856 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.49062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78263.log 2024-09-16T15:57:18.856 INFO:teuthology.orchestra.run.smithi151.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49062.log.gz 2024-09-16T15:57:18.858 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.80579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50624.log 2024-09-16T15:57:18.858 INFO:teuthology.orchestra.run.smithi151.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.80579.log.gz 2024-09-16T15:57:18.859 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.78263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83296.log 2024-09-16T15:57:18.859 INFO:teuthology.orchestra.run.smithi151.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.78263.log.gz 2024-09-16T15:57:18.860 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.50624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70813.log 2024-09-16T15:57:18.860 INFO:teuthology.orchestra.run.smithi151.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50624.log.gz 2024-09-16T15:57:18.861 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.83296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56687.log 2024-09-16T15:57:18.861 INFO:teuthology.orchestra.run.smithi151.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.83296.log.gz 2024-09-16T15:57:18.863 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.70813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48907.log 2024-09-16T15:57:18.863 INFO:teuthology.orchestra.run.smithi151.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.70813.log.gz 2024-09-16T15:57:18.863 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.56687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83660.log 2024-09-16T15:57:18.864 INFO:teuthology.orchestra.run.smithi151.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.56687.log.gz 2024-09-16T15:57:18.865 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.48907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56813.log 2024-09-16T15:57:18.865 INFO:teuthology.orchestra.run.smithi151.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.48907.log.gz 2024-09-16T15:57:18.866 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.83660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45919.log 2024-09-16T15:57:18.866 INFO:teuthology.orchestra.run.smithi151.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.83660.log.gz 2024-09-16T15:57:18.867 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.56813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35764.log 2024-09-16T15:57:18.867 INFO:teuthology.orchestra.run.smithi151.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56813.log.gz 2024-09-16T15:57:18.868 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.45919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83086.log 2024-09-16T15:57:18.869 INFO:teuthology.orchestra.run.smithi151.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45919.log.gz 2024-09-16T15:57:18.870 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.35764.log: gzip -5 78.0% -- replaced with /var/log/ceph/ceph-client.admin.35764.log.gz 2024-09-16T15:57:18.870 INFO:teuthology.orchestra.run.smithi151.stderr: --verbose -- /var/log/ceph/ceph-client.admin.48967.log 2024-09-16T15:57:18.871 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.83086.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.83086.log.gz 2024-09-16T15:57:18.871 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42629.log 2024-09-16T15:57:18.873 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.48967.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48967.log.gz 2024-09-16T15:57:18.873 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43511.log 2024-09-16T15:57:18.874 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.42629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.51176.log 2024-09-16T15:57:18.874 INFO:teuthology.orchestra.run.smithi151.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42629.log.gz 2024-09-16T15:57:18.875 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.43511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52399.log 2024-09-16T15:57:18.875 INFO:teuthology.orchestra.run.smithi151.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43511.log.gz 2024-09-16T15:57:18.876 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.0.51176.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.51176.log.gz 2024-09-16T15:57:18.876 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69504.log 2024-09-16T15:57:18.878 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.52399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51506.log 2024-09-16T15:57:18.878 INFO:teuthology.orchestra.run.smithi151.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.52399.log.gz 2024-09-16T15:57:18.879 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.69504.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.69504.log.gz 2024-09-16T15:57:18.879 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59317.log 2024-09-16T15:57:18.881 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.51506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54387.log 2024-09-16T15:57:18.881 INFO:teuthology.orchestra.run.smithi151.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.51506.log.gz 2024-09-16T15:57:18.882 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.59317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70372.log 2024-09-16T15:57:18.882 INFO:teuthology.orchestra.run.smithi151.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.59317.log.gz 2024-09-16T15:57:18.883 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.54387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53952.log 2024-09-16T15:57:18.883 INFO:teuthology.orchestra.run.smithi151.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54387.log.gz 2024-09-16T15:57:18.884 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.70372.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79298.log 2024-09-16T15:57:18.885 INFO:teuthology.orchestra.run.smithi151.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.70372.log.gz 2024-09-16T15:57:18.886 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.53952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77740.log 2024-09-16T15:57:18.886 INFO:teuthology.orchestra.run.smithi151.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53952.log.gz 2024-09-16T15:57:18.887 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.79298.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.29588.log 2024-09-16T15:57:18.887 INFO:teuthology.orchestra.run.smithi151.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.79298.log.gz 2024-09-16T15:57:18.888 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.77740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53398.log 2024-09-16T15:57:18.888 INFO:teuthology.orchestra.run.smithi151.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.77740.log.gz 2024-09-16T15:57:18.889 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph.tmp-client.admin.29588.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.29588.log.gz 2024-09-16T15:57:18.889 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71592.log 2024-09-16T15:57:18.890 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.53398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65067.log 2024-09-16T15:57:18.891 INFO:teuthology.orchestra.run.smithi151.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53398.log.gz 2024-09-16T15:57:18.892 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.71592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58048.log 2024-09-16T15:57:18.892 INFO:teuthology.orchestra.run.smithi151.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.71592.log.gz 2024-09-16T15:57:18.893 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.65067.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71862.log 2024-09-16T15:57:18.893 INFO:teuthology.orchestra.run.smithi151.stderr: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.65067.log.gz 2024-09-16T15:57:18.894 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.58048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46293.log 2024-09-16T15:57:18.894 INFO:teuthology.orchestra.run.smithi151.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.58048.log.gz 2024-09-16T15:57:18.896 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.71862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70270.log 2024-09-16T15:57:18.896 INFO:teuthology.orchestra.run.smithi151.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.71862.log.gz 2024-09-16T15:57:18.897 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.46293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70060.log 2024-09-16T15:57:18.897 INFO:teuthology.orchestra.run.smithi151.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46293.log.gz 2024-09-16T15:57:18.898 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.70270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47701.log 2024-09-16T15:57:18.898 INFO:teuthology.orchestra.run.smithi151.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.70270.log.gz 2024-09-16T15:57:18.899 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.70060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43520.log 2024-09-16T15:57:18.899 INFO:teuthology.orchestra.run.smithi151.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.70060.log.gz 2024-09-16T15:57:18.900 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.0.47701.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.47701.log.gz 2024-09-16T15:57:18.900 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.44949.log 2024-09-16T15:57:18.901 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.43520.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-16T15:57:18.901 INFO:teuthology.orchestra.run.smithi151.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43520.log.gz 2024-09-16T15:57:18.903 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.2.44949.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.2.44949.log.gz 2024-09-16T15:57:18.903 INFO:teuthology.orchestra.run.smithi151.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.48655.log 2024-09-16T15:57:18.904 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58846.log 2024-09-16T15:57:18.905 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.48655.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.48655.log.gz 2024-09-16T15:57:18.906 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70467.log 2024-09-16T15:57:18.907 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.58846.log: gzip 87.4% -5 -- replaced with /var/log/ceph/ceph-client.admin.58846.log.gz --verbose 2024-09-16T15:57:18.907 INFO:teuthology.orchestra.run.smithi151.stderr: -- /var/log/ceph/ceph-client.admin.62770.log 2024-09-16T15:57:18.908 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.70467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60653.log 2024-09-16T15:57:18.908 INFO:teuthology.orchestra.run.smithi151.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.70467.log.gz 2024-09-16T15:57:18.909 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.62770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50918.log 2024-09-16T15:57:18.909 INFO:teuthology.orchestra.run.smithi151.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.62770.log.gz 2024-09-16T15:57:18.910 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.60653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79130.log 2024-09-16T15:57:18.911 INFO:teuthology.orchestra.run.smithi151.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60653.log.gz 2024-09-16T15:57:18.912 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.50918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79448.log 2024-09-16T15:57:18.912 INFO:teuthology.orchestra.run.smithi151.stderr: 91.0%/var/log/ceph/ceph-client.admin.79130.log: -- replaced with /var/log/ceph/ceph-client.admin.50918.log.gz 2024-09-16T15:57:18.913 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 87.4% -- replaced with /var/log/ceph/ceph-client.admin.79130.log.gz 2024-09-16T15:57:18.913 INFO:teuthology.orchestra.run.smithi151.stderr: --verbose -- /var/log/ceph/ceph-client.admin.59485.log 2024-09-16T15:57:18.914 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.79448.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.79448.log.gz 2024-09-16T15:57:18.915 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46087.log 2024-09-16T15:57:18.916 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.59485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57210.log 2024-09-16T15:57:18.916 INFO:teuthology.orchestra.run.smithi151.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.59485.log.gz 2024-09-16T15:57:18.917 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.46087.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46087.log.gz 2024-09-16T15:57:18.918 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61747.log 2024-09-16T15:57:18.918 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.57210.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.57210.log.gz 2024-09-16T15:57:18.919 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54657.log 2024-09-16T15:57:18.920 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.61747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80806.log 2024-09-16T15:57:18.920 INFO:teuthology.orchestra.run.smithi151.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.61747.log.gz 2024-09-16T15:57:18.921 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.54657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61519.log 2024-09-16T15:57:18.922 INFO:teuthology.orchestra.run.smithi151.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54657.log.gz 2024-09-16T15:57:18.923 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.80806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60559.log 2024-09-16T15:57:18.923 INFO:teuthology.orchestra.run.smithi151.stderr: 84.6% -- replaced with /var/log/ceph/ceph-client.admin.80806.log.gz 2024-09-16T15:57:18.924 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.61519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61439.log 2024-09-16T15:57:18.924 INFO:teuthology.orchestra.run.smithi151.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.61519.log.gz 2024-09-16T15:57:18.925 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.60559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42167.log 2024-09-16T15:57:18.925 INFO:teuthology.orchestra.run.smithi151.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.60559.log.gz 2024-09-16T15:57:18.927 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.61439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71084.log 2024-09-16T15:57:18.927 INFO:teuthology.orchestra.run.smithi151.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.61439.log.gz 2024-09-16T15:57:18.927 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.42167.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42167.log.gz 2024-09-16T15:57:18.928 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73846.log 2024-09-16T15:57:18.929 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.71084.log: gzip 87.5% -- replaced with /var/log/ceph/ceph-client.admin.71084.log.gz -5 2024-09-16T15:57:18.929 INFO:teuthology.orchestra.run.smithi151.stderr: --verbose -- /var/log/ceph/ceph-client.admin.51296.log 2024-09-16T15:57:18.931 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.73846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35948.log 2024-09-16T15:57:18.931 INFO:teuthology.orchestra.run.smithi151.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.73846.log.gz 2024-09-16T15:57:18.932 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.51296.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51296.log.gz 2024-09-16T15:57:18.932 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64306.log 2024-09-16T15:57:18.933 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.35948.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.35948.log.gz 2024-09-16T15:57:18.934 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53086.log 2024-09-16T15:57:18.935 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.64306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64234.log 2024-09-16T15:57:18.935 INFO:teuthology.orchestra.run.smithi151.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.64306.log.gz 2024-09-16T15:57:18.936 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.53086.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.53086.log.gz 2024-09-16T15:57:18.936 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81866.log 2024-09-16T15:57:18.938 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.64234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57252.log 2024-09-16T15:57:18.938 INFO:teuthology.orchestra.run.smithi151.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.64234.log.gz 2024-09-16T15:57:18.940 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.81866.log: gzip -5 --verbose 87.7% -- replaced with /var/log/ceph/ceph-client.admin.81866.log.gz 2024-09-16T15:57:18.940 INFO:teuthology.orchestra.run.smithi151.stderr: -- /var/log/ceph/ceph-client.admin.47825.log 2024-09-16T15:57:18.940 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.57252.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57252.log.gz 2024-09-16T15:57:18.941 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71508.log 2024-09-16T15:57:18.942 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.47825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76852.log 2024-09-16T15:57:18.942 INFO:teuthology.orchestra.run.smithi151.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47825.log.gz 2024-09-16T15:57:18.943 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.71508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69303.log 2024-09-16T15:57:18.943 INFO:teuthology.orchestra.run.smithi151.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.71508.log.gz 2024-09-16T15:57:18.944 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.76852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70645.log 2024-09-16T15:57:18.945 INFO:teuthology.orchestra.run.smithi151.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.76852.log.gz 2024-09-16T15:57:18.946 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.69303.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.69303.log.gz 2024-09-16T15:57:18.946 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78878.log 2024-09-16T15:57:18.947 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.70645.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.70645.log.gz 2024-09-16T15:57:18.947 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55455.log 2024-09-16T15:57:18.949 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.78878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58132.log 2024-09-16T15:57:18.949 INFO:teuthology.orchestra.run.smithi151.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.78878.log.gz 2024-09-16T15:57:18.950 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.55455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59569.log 2024-09-16T15:57:18.950 INFO:teuthology.orchestra.run.smithi151.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55455.log.gz 2024-09-16T15:57:18.951 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.58132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82097.log 2024-09-16T15:57:18.951 INFO:teuthology.orchestra.run.smithi151.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.58132.log.gz 2024-09-16T15:57:18.953 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.59569.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.59569.log.gz 2024-09-16T15:57:18.954 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.82097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65635.log 2024-09-16T15:57:18.954 INFO:teuthology.orchestra.run.smithi151.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.82097.log.gz 2024-09-16T15:57:18.955 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-16T15:57:18.956 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.65635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71168.log 2024-09-16T15:57:18.956 INFO:teuthology.orchestra.run.smithi151.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.65635.log.gz 2024-09-16T15:57:18.957 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81662.log 2024-09-16T15:57:18.959 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.71168.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.71168.log.gz 2024-09-16T15:57:18.959 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5/var/log/ceph/ceph-client.admin.81662.log: --verbose -- /var/log/ceph/ceph-client.admin.78305.log 2024-09-16T15:57:18.960 INFO:teuthology.orchestra.run.smithi151.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.81662.log.gz 2024-09-16T15:57:18.961 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54206.log 2024-09-16T15:57:18.962 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.78305.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.78305.log.gz 2024-09-16T15:57:18.963 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50186.log 2024-09-16T15:57:18.964 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.54206.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54206.log.gz 2024-09-16T15:57:18.965 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.50186.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50186.log.gz 2024-09-16T15:57:18.966 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69748.log 2024-09-16T15:57:18.967 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46003.log 2024-09-16T15:57:18.969 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.69748.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.69748.log.gz 2024-09-16T15:57:18.969 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73762.log 2024-09-16T15:57:18.970 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.46003.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46003.log.gz 2024-09-16T15:57:18.972 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.73762.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.73762.log.gz 2024-09-16T15:57:18.972 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82517.log 2024-09-16T15:57:18.974 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78179.log 94.3% 2024-09-16T15:57:18.974 INFO:teuthology.orchestra.run.smithi151.stderr: -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-16T15:57:18.974 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.82517.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.82517.log.gz 2024-09-16T15:57:18.975 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63771.log 2024-09-16T15:57:18.976 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.78179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54489.log 2024-09-16T15:57:18.976 INFO:teuthology.orchestra.run.smithi151.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.78179.log.gz 2024-09-16T15:57:18.977 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.63771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67475.log 2024-09-16T15:57:18.978 INFO:teuthology.orchestra.run.smithi151.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.63771.log.gz 2024-09-16T15:57:18.979 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.54489.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54489.log.gz 2024-09-16T15:57:18.980 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.67475.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.67475.log.gz 2024-09-16T15:57:18.980 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49240.log 2024-09-16T15:57:18.981 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74281.log 2024-09-16T15:57:18.983 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.49240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78708.log 2024-09-16T15:57:18.983 INFO:teuthology.orchestra.run.smithi151.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49240.log.gz 2024-09-16T15:57:18.984 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.74281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59668.log 2024-09-16T15:57:18.984 INFO:teuthology.orchestra.run.smithi151.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.74281.log.gz 2024-09-16T15:57:18.985 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.78708.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.78708.log.gz 2024-09-16T15:57:18.986 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60863.log 2024-09-16T15:57:18.987 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.59668.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.59668.log.gz 2024-09-16T15:57:18.987 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64595.log 2024-09-16T15:57:18.988 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.60863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78621.log 2024-09-16T15:57:18.989 INFO:teuthology.orchestra.run.smithi151.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.60863.log.gz 2024-09-16T15:57:18.989 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.64595.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.64595.log.gz 2024-09-16T15:57:18.990 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68505.log 2024-09-16T15:57:18.991 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.78621.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.78621.log.gz 2024-09-16T15:57:18.991 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52795.log 2024-09-16T15:57:18.993 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.68505.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-16T15:57:18.993 INFO:teuthology.orchestra.run.smithi151.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.68505.log.gz 2024-09-16T15:57:18.994 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.52795.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52795.log.gz 2024-09-16T15:57:18.994 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42125.log 2024-09-16T15:57:18.995 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62500.log 2024-09-16T15:57:18.996 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.42125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43516.log 2024-09-16T15:57:18.997 INFO:teuthology.orchestra.run.smithi151.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42125.log.gz 2024-09-16T15:57:18.999 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.62500.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.62500.log.gz 2024-09-16T15:57:18.999 INFO:teuthology.orchestra.run.smithi151.stderr: 86.1% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-16T15:57:18.999 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80973.log 2024-09-16T15:57:18.999 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.43516.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43516.log.gz 2024-09-16T15:57:19.000 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59090.log 2024-09-16T15:57:19.002 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.80973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71252.log 2024-09-16T15:57:19.002 INFO:teuthology.orchestra.run.smithi151.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.80973.log.gz 2024-09-16T15:57:19.003 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.59090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74187.log 2024-09-16T15:57:19.003 INFO:teuthology.orchestra.run.smithi151.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.59090.log.gz 2024-09-16T15:57:19.004 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.71252.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.71252.log.gz 2024-09-16T15:57:19.005 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.74187.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.74187.log.gz 2024-09-16T15:57:19.006 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43502.log 2024-09-16T15:57:19.007 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68547.log 2024-09-16T15:57:19.008 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.43502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62980.log 2024-09-16T15:57:19.008 INFO:teuthology.orchestra.run.smithi151.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43502.log.gz 2024-09-16T15:57:19.009 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.68547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57918.log 2024-09-16T15:57:19.009 INFO:teuthology.orchestra.run.smithi151.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.68547.log.gz 2024-09-16T15:57:19.011 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.62980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67018.log 2024-09-16T15:57:19.011 INFO:teuthology.orchestra.run.smithi151.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.62980.log.gz 2024-09-16T15:57:19.012 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.57918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83002.log 2024-09-16T15:57:19.012 INFO:teuthology.orchestra.run.smithi151.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57918.log.gz 2024-09-16T15:57:19.013 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.67018.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.67018.log.gz 2024-09-16T15:57:19.014 INFO:teuthology.orchestra.run.smithi151.stderr:gzip/var/log/ceph/ceph-client.admin.83002.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.64944.log 2024-09-16T15:57:19.014 INFO:teuthology.orchestra.run.smithi151.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.83002.log.gz 2024-09-16T15:57:19.015 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70916.log 2024-09-16T15:57:19.017 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.64944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54825.log 2024-09-16T15:57:19.017 INFO:teuthology.orchestra.run.smithi151.stderr: 84.0% -- replaced with /var/log/ceph/ceph-client.admin.64944.log.gz 2024-09-16T15:57:19.018 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.70916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78579.log 2024-09-16T15:57:19.018 INFO:teuthology.orchestra.run.smithi151.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.70916.log.gz 2024-09-16T15:57:19.019 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.54825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56270.log 2024-09-16T15:57:19.019 INFO:teuthology.orchestra.run.smithi151.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54825.log.gz 2024-09-16T15:57:19.021 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.78579.log: 87.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.78579.log.gz 2024-09-16T15:57:19.021 INFO:teuthology.orchestra.run.smithi151.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.43181.log 2024-09-16T15:57:19.022 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.56270.log: 86.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71909.log 2024-09-16T15:57:19.022 INFO:teuthology.orchestra.run.smithi151.stderr: -- replaced with /var/log/ceph/ceph-client.admin.56270.log.gz 2024-09-16T15:57:19.023 INFO:teuthology.orchestra.run.smithi167.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-16T15:57:19.023 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.43181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53212.log 2024-09-16T15:57:19.023 INFO:teuthology.orchestra.run.smithi151.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43181.log.gz 2024-09-16T15:57:19.024 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.71909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80012.log 2024-09-16T15:57:19.025 INFO:teuthology.orchestra.run.smithi151.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.71909.log.gz 2024-09-16T15:57:19.026 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.53212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67349.log 2024-09-16T15:57:19.026 INFO:teuthology.orchestra.run.smithi151.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53212.log.gz 2024-09-16T15:57:19.027 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.80012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62392.log 2024-09-16T15:57:19.027 INFO:teuthology.orchestra.run.smithi151.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.80012.log.gz 2024-09-16T15:57:19.028 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.67349.log: 87.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79382.log 2024-09-16T15:57:19.028 INFO:teuthology.orchestra.run.smithi151.stderr: -- replaced with /var/log/ceph/ceph-client.admin.67349.log.gz 2024-09-16T15:57:19.030 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.62392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72720.log 2024-09-16T15:57:19.030 INFO:teuthology.orchestra.run.smithi151.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.62392.log.gz 2024-09-16T15:57:19.031 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.79382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77824.log 2024-09-16T15:57:19.031 INFO:teuthology.orchestra.run.smithi151.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.79382.log.gz 2024-09-16T15:57:19.032 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.72720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49846.log 2024-09-16T15:57:19.032 INFO:teuthology.orchestra.run.smithi151.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.72720.log.gz 2024-09-16T15:57:19.033 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.77824.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.77824.log.gz 2024-09-16T15:57:19.033 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36154.log 2024-09-16T15:57:19.035 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.49846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68070.log 2024-09-16T15:57:19.035 INFO:teuthology.orchestra.run.smithi151.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49846.log.gz 2024-09-16T15:57:19.036 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.36154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82055.log 2024-09-16T15:57:19.036 INFO:teuthology.orchestra.run.smithi151.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.36154.log.gz 2024-09-16T15:57:19.037 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.68070.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.68070.log.gz 2024-09-16T15:57:19.038 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62350.log 2024-09-16T15:57:19.038 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.82055.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.82055.log.gz 2024-09-16T15:57:19.039 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67941.log 2024-09-16T15:57:19.040 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.62350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65391.log 2024-09-16T15:57:19.040 INFO:teuthology.orchestra.run.smithi151.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.62350.log.gz 2024-09-16T15:57:19.041 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.67941.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.67941.log.gz 2024-09-16T15:57:19.042 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36699.log 2024-09-16T15:57:19.043 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.65391.log: 87.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45436.log 2024-09-16T15:57:19.043 INFO:teuthology.orchestra.run.smithi151.stderr: -- replaced with /var/log/ceph/ceph-client.admin.65391.log.gz 2024-09-16T15:57:19.044 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.36699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78011.log 2024-09-16T15:57:19.044 INFO:teuthology.orchestra.run.smithi151.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.36699.log.gz 2024-09-16T15:57:19.046 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.45436.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45436.log.gz 2024-09-16T15:57:19.046 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75625.log 2024-09-16T15:57:19.047 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.78011.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.78011.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52897.log 2024-09-16T15:57:19.047 INFO:teuthology.orchestra.run.smithi151.stderr: 2024-09-16T15:57:19.048 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.75625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72216.log 2024-09-16T15:57:19.048 INFO:teuthology.orchestra.run.smithi151.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.75625.log.gz 2024-09-16T15:57:19.050 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.52897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61008.log 2024-09-16T15:57:19.050 INFO:teuthology.orchestra.run.smithi151.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52897.log.gz 2024-09-16T15:57:19.050 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.72216.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.72216.log.gz 2024-09-16T15:57:19.051 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59233.log 2024-09-16T15:57:19.052 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.61008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71550.log 2024-09-16T15:57:19.052 INFO:teuthology.orchestra.run.smithi151.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.61008.log.gz 2024-09-16T15:57:19.054 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.59233.log: 86.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77425.log 2024-09-16T15:57:19.054 INFO:teuthology.orchestra.run.smithi151.stderr: -- replaced with /var/log/ceph/ceph-client.admin.59233.log.gz 2024-09-16T15:57:19.054 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.71550.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.71550.log.gz 2024-09-16T15:57:19.056 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.77425.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.77425.log.gz 2024-09-16T15:57:19.056 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75537.log 2024-09-16T15:57:19.057 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45715.log 2024-09-16T15:57:19.059 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.75537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69177.log 2024-09-16T15:57:19.059 INFO:teuthology.orchestra.run.smithi151.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.75537.log.gz 2024-09-16T15:57:19.060 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.45715.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45715.log.gz 2024-09-16T15:57:19.061 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57378.log 2024-09-16T15:57:19.062 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.69177.log: 87.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36130.log 2024-09-16T15:57:19.062 INFO:teuthology.orchestra.run.smithi151.stderr: -- replaced with /var/log/ceph/ceph-client.admin.69177.log.gz 2024-09-16T15:57:19.063 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.57378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73619.log 2024-09-16T15:57:19.063 INFO:teuthology.orchestra.run.smithi151.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.57378.log.gz 2024-09-16T15:57:19.064 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.36130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66349.log 2024-09-16T15:57:19.064 INFO:teuthology.orchestra.run.smithi151.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.36130.log.gz 2024-09-16T15:57:19.065 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.73619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36156.log 2024-09-16T15:57:19.066 INFO:teuthology.orchestra.run.smithi151.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.73619.log.gz 2024-09-16T15:57:19.067 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.66349.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.66349.log.gz 2024-09-16T15:57:19.068 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.36156.log: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.36156.log.gz 2024-09-16T15:57:19.068 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54699.log 2024-09-16T15:57:19.069 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35891.log 2024-09-16T15:57:19.071 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.54699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42251.log 2024-09-16T15:57:19.071 INFO:teuthology.orchestra.run.smithi151.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54699.log.gz 2024-09-16T15:57:19.072 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.35891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79214.log 2024-09-16T15:57:19.072 INFO:teuthology.orchestra.run.smithi151.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.35891.log.gz 2024-09-16T15:57:19.073 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.42251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80931.log 2024-09-16T15:57:19.074 INFO:teuthology.orchestra.run.smithi151.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42251.log.gz 2024-09-16T15:57:19.075 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.79214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83702.log 2024-09-16T15:57:19.075 INFO:teuthology.orchestra.run.smithi151.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.79214.log.gz 2024-09-16T15:57:19.076 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.80931.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.80931.log.gz 2024-09-16T15:57:19.077 INFO:teuthology.orchestra.run.smithi167.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-16T15:57:19.077 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50326.log 2024-09-16T15:57:19.077 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.83702.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.83702.log.gz 2024-09-16T15:57:19.078 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70873.log 2024-09-16T15:57:19.079 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.50326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76129.log 2024-09-16T15:57:19.079 INFO:teuthology.orchestra.run.smithi151.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50326.log.gz 2024-09-16T15:57:19.080 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.70873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43931.log 2024-09-16T15:57:19.080 INFO:teuthology.orchestra.run.smithi151.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.70873.log.gz 2024-09-16T15:57:19.081 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.76129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45310.log 2024-09-16T15:57:19.082 INFO:teuthology.orchestra.run.smithi151.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.76129.log.gz 2024-09-16T15:57:19.083 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.43931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59715.log 2024-09-16T15:57:19.083 INFO:teuthology.orchestra.run.smithi151.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43931.log.gz 2024-09-16T15:57:19.084 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.45310.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45310.log.gz 2024-09-16T15:57:19.084 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45184.log 2024-09-16T15:57:19.085 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.59715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42545.log 2024-09-16T15:57:19.085 INFO:teuthology.orchestra.run.smithi151.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.59715.log.gz 2024-09-16T15:57:19.087 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.45184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58216.log 2024-09-16T15:57:19.087 INFO:teuthology.orchestra.run.smithi151.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45184.log.gz 2024-09-16T15:57:19.088 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.42545.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42545.log.gz 2024-09-16T15:57:19.088 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65677.log 2024-09-16T15:57:19.090 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.58216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56438.log 2024-09-16T15:57:19.090 INFO:teuthology.orchestra.run.smithi151.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.58216.log.gz 2024-09-16T15:57:19.091 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.65677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63064.log 2024-09-16T15:57:19.091 INFO:teuthology.orchestra.run.smithi151.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.65677.log.gz 2024-09-16T15:57:19.092 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.56438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65761.log 2024-09-16T15:57:19.092 INFO:teuthology.orchestra.run.smithi151.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.56438.log.gz 2024-09-16T15:57:19.093 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.63064.log: gzip -5 87.0% --verbose -- /var/log/ceph/ceph-client.admin.57876.log 2024-09-16T15:57:19.093 INFO:teuthology.orchestra.run.smithi151.stderr: -- replaced with /var/log/ceph/ceph-client.admin.63064.log.gz 2024-09-16T15:57:19.094 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.65761.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.65761.log.gz 2024-09-16T15:57:19.095 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53693.log 2024-09-16T15:57:19.096 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.57876.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57876.log.gz 2024-09-16T15:57:19.096 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75453.log 2024-09-16T15:57:19.098 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.53693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65971.log 2024-09-16T15:57:19.098 INFO:teuthology.orchestra.run.smithi151.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.53693.log.gz 2024-09-16T15:57:19.099 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.75453.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.75453.log.gz 2024-09-16T15:57:19.099 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58930.log 2024-09-16T15:57:19.100 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.65971.log: gzip 87.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.65971.log.gz 2024-09-16T15:57:19.100 INFO:teuthology.orchestra.run.smithi151.stderr: --verbose -- /var/log/ceph/ceph-client.admin.64706.log 2024-09-16T15:57:19.101 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.58930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52992.log 2024-09-16T15:57:19.102 INFO:teuthology.orchestra.run.smithi151.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.58930.log.gz 2024-09-16T15:57:19.103 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.64706.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.64706.log.gz 2024-09-16T15:57:19.104 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5/var/log/ceph/ceph-client.admin.52992.log: --verbose -- /var/log/ceph/ceph-client.admin.54741.log 2024-09-16T15:57:19.104 INFO:teuthology.orchestra.run.smithi151.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52992.log.gz 2024-09-16T15:57:19.105 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61176.log 2024-09-16T15:57:19.106 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.54741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42849.log 2024-09-16T15:57:19.106 INFO:teuthology.orchestra.run.smithi151.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.54741.log.gz 2024-09-16T15:57:19.108 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.61176.log: gzip 87.8% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.61176.log.gz -- 2024-09-16T15:57:19.108 INFO:teuthology.orchestra.run.smithi151.stderr: /var/log/ceph/ceph-client.admin.68589.log 2024-09-16T15:57:19.109 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.42849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41715.log 2024-09-16T15:57:19.109 INFO:teuthology.orchestra.run.smithi151.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.42849.log.gz 2024-09-16T15:57:19.110 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.68589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63223.log 2024-09-16T15:57:19.110 INFO:teuthology.orchestra.run.smithi151.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.68589.log.gz 2024-09-16T15:57:19.112 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.41715.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41715.log.gz 2024-09-16T15:57:19.113 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.63223.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.63223.log.gz 2024-09-16T15:57:19.113 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64398.log 2024-09-16T15:57:19.114 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80411.log 2024-09-16T15:57:19.115 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.64398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56186.log 2024-09-16T15:57:19.116 INFO:teuthology.orchestra.run.smithi151.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.64398.log.gz 2024-09-16T15:57:19.117 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.80411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55245.log 2024-09-16T15:57:19.117 INFO:teuthology.orchestra.run.smithi151.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.80411.log.gz 2024-09-16T15:57:19.118 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.56186.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.56186.log.gz 2024-09-16T15:57:19.118 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74930.log 2024-09-16T15:57:19.119 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.55245.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55245.log.gz 2024-09-16T15:57:19.120 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77096.log 2024-09-16T15:57:19.121 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.74930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66055.log 2024-09-16T15:57:19.121 INFO:teuthology.orchestra.run.smithi151.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.74930.log.gz 2024-09-16T15:57:19.122 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.77096.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.77096.log.gz 2024-09-16T15:57:19.123 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.66055.log: /var/log/ceph/ceph-client.admin.80054.log 2024-09-16T15:57:19.123 INFO:teuthology.orchestra.run.smithi151.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.66055.log.gz 2024-09-16T15:57:19.124 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53735.log 2024-09-16T15:57:19.126 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.80054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47093.log 2024-09-16T15:57:19.126 INFO:teuthology.orchestra.run.smithi151.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.80054.log.gz 2024-09-16T15:57:19.127 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.53735.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53735.log.gz 2024-09-16T15:57:19.127 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59191.log 2024-09-16T15:57:19.128 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.47093.log: 87.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.0.69015.log 2024-09-16T15:57:19.128 INFO:teuthology.orchestra.run.smithi151.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47093.log.gz 2024-09-16T15:57:19.129 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.59191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77365.log 2024-09-16T15:57:19.130 INFO:teuthology.orchestra.run.smithi151.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.59191.log.gz 2024-09-16T15:57:19.130 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.0.69015.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.69015.log.gz 2024-09-16T15:57:19.131 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82876.log 2024-09-16T15:57:19.132 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.77365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83548.log 2024-09-16T15:57:19.132 INFO:teuthology.orchestra.run.smithi151.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.77365.log.gz 2024-09-16T15:57:19.134 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.82876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52151.log 2024-09-16T15:57:19.134 INFO:teuthology.orchestra.run.smithi151.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.82876.log.gz 2024-09-16T15:57:19.135 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.83548.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.83548.log.gz 2024-09-16T15:57:19.136 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.52151.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.52151.log.gz 2024-09-16T15:57:19.136 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45142.log 2024-09-16T15:57:19.138 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42335.log 2024-09-16T15:57:19.139 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.45142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54783.log 2024-09-16T15:57:19.139 INFO:teuthology.orchestra.run.smithi151.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45142.log.gz 2024-09-16T15:57:19.140 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.42335.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42335.log.gz 2024-09-16T15:57:19.140 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61600.log 2024-09-16T15:57:19.142 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.54783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80259.log 2024-09-16T15:57:19.142 INFO:teuthology.orchestra.run.smithi151.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54783.log.gz 2024-09-16T15:57:19.143 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.61600.log: gzip -5 --verbose -- 87.5% /var/log/ceph/ceph-client.admin.73325.log 2024-09-16T15:57:19.143 INFO:teuthology.orchestra.run.smithi151.stderr: -- replaced with /var/log/ceph/ceph-client.admin.61600.log.gz 2024-09-16T15:57:19.144 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.80259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.62688.log 2024-09-16T15:57:19.144 INFO:teuthology.orchestra.run.smithi151.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.80259.log.gz 2024-09-16T15:57:19.145 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.73325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68379.log 2024-09-16T15:57:19.146 INFO:teuthology.orchestra.run.smithi151.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.73325.log.gz 2024-09-16T15:57:19.146 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.0.62688.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.62688.log.gz 2024-09-16T15:57:19.147 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53609.log 2024-09-16T15:57:19.148 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.68379.log: 87.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54299.log 2024-09-16T15:57:19.148 INFO:teuthology.orchestra.run.smithi151.stderr: -- replaced with /var/log/ceph/ceph-client.admin.68379.log.gz 2024-09-16T15:57:19.149 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.53609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69934.log 2024-09-16T15:57:19.149 INFO:teuthology.orchestra.run.smithi151.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53609.log.gz 2024-09-16T15:57:19.151 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.54299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67780.log 2024-09-16T15:57:19.151 INFO:teuthology.orchestra.run.smithi151.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54299.log.gz 2024-09-16T15:57:19.152 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.69934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69261.log 2024-09-16T15:57:19.152 INFO:teuthology.orchestra.run.smithi151.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.69934.log.gz 2024-09-16T15:57:19.153 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.67780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69664.log 2024-09-16T15:57:19.153 INFO:teuthology.orchestra.run.smithi151.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.67780.log.gz 2024-09-16T15:57:19.155 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.69261.log: gzip -5 87.1% -- replaced with /var/log/ceph/ceph-client.admin.69261.log.gz 2024-09-16T15:57:19.155 INFO:teuthology.orchestra.run.smithi151.stderr: --verbose -- /var/log/ceph/ceph-client.admin.73948.log 2024-09-16T15:57:19.156 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.69664.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.69664.log.gz 2024-09-16T15:57:19.156 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67102.log 2024-09-16T15:57:19.158 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.73948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68025.log 2024-09-16T15:57:19.158 INFO:teuthology.orchestra.run.smithi151.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.73948.log.gz 2024-09-16T15:57:19.159 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.67102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46045.log 2024-09-16T15:57:19.159 INFO:teuthology.orchestra.run.smithi151.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.67102.log.gz 2024-09-16T15:57:19.160 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.68025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61092.log 2024-09-16T15:57:19.160 INFO:teuthology.orchestra.run.smithi151.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.68025.log.gz 2024-09-16T15:57:19.161 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.46045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76651.log 2024-09-16T15:57:19.161 INFO:teuthology.orchestra.run.smithi151.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46045.log.gz 2024-09-16T15:57:19.162 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.61092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46755.log 2024-09-16T15:57:19.163 INFO:teuthology.orchestra.run.smithi151.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.61092.log.gz 2024-09-16T15:57:19.164 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.76651.log: gzip -5 --verbose -- 87.0% /var/log/ceph/ceph-client.admin.81536.log 2024-09-16T15:57:19.164 INFO:teuthology.orchestra.run.smithi151.stderr: -- replaced with /var/log/ceph/ceph-client.admin.76651.log.gz 2024-09-16T15:57:19.165 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.46755.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.46755.log.gz 2024-09-16T15:57:19.165 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52279.log 2024-09-16T15:57:19.166 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.81536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81326.log 2024-09-16T15:57:19.167 INFO:teuthology.orchestra.run.smithi151.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.81536.log.gz 2024-09-16T15:57:19.168 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.52279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43805.log 2024-09-16T15:57:19.168 INFO:teuthology.orchestra.run.smithi151.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.52279.log.gz 2024-09-16T15:57:19.169 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.81326.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.81326.log.gz 2024-09-16T15:57:19.169 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.55121.log 2024-09-16T15:57:19.171 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.43805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83422.log 2024-09-16T15:57:19.171 INFO:teuthology.orchestra.run.smithi151.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43805.log.gz 2024-09-16T15:57:19.172 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.0.55121.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54257.log 2024-09-16T15:57:19.172 INFO:teuthology.orchestra.run.smithi151.stderr: -- replaced with /var/log/ceph/ceph-client.0.55121.log.gz 2024-09-16T15:57:19.173 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.83422.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.83422.log.gz 2024-09-16T15:57:19.174 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73720.log 2024-09-16T15:57:19.174 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.54257.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54257.log.gz 2024-09-16T15:57:19.175 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81620.log 2024-09-16T15:57:19.176 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.73720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80889.log 2024-09-16T15:57:19.176 INFO:teuthology.orchestra.run.smithi151.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.73720.log.gz 2024-09-16T15:57:19.178 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.81620.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.81620.log.gz 2024-09-16T15:57:19.178 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36202.log 2024-09-16T15:57:19.179 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.80889.log: gzip -5 86.6% -- replaced with /var/log/ceph/ceph-client.admin.80889.log.gz 2024-09-16T15:57:19.179 INFO:teuthology.orchestra.run.smithi151.stderr: --verbose -- /var/log/ceph/ceph-client.admin.54117.log 2024-09-16T15:57:19.180 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.36202.log: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.36202.log.gz 2024-09-16T15:57:19.180 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45625.log 2024-09-16T15:57:19.182 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.54117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70771.log 2024-09-16T15:57:19.182 INFO:teuthology.orchestra.run.smithi151.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54117.log.gz 2024-09-16T15:57:19.183 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.45625.log: gzip -5 --verbose 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45625.log.gz 2024-09-16T15:57:19.183 INFO:teuthology.orchestra.run.smithi151.stderr: -- /var/log/ceph/ceph-client.admin.61684.log 2024-09-16T15:57:19.184 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.70771.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.70771.log.gz 2024-09-16T15:57:19.185 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50708.log 2024-09-16T15:57:19.186 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.61684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.62692.log 2024-09-16T15:57:19.186 INFO:teuthology.orchestra.run.smithi151.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.61684.log.gz 2024-09-16T15:57:19.187 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.50708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64073.log 2024-09-16T15:57:19.187 INFO:teuthology.orchestra.run.smithi151.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50708.log.gz 2024-09-16T15:57:19.188 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.0.62692.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.62692.log.gz 2024-09-16T15:57:19.188 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63543.log 2024-09-16T15:57:19.190 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.64073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75314.log 2024-09-16T15:57:19.190 INFO:teuthology.orchestra.run.smithi151.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.64073.log.gz 2024-09-16T15:57:19.191 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.63543.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.63543.log.gz 2024-09-16T15:57:19.191 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77969.log 2024-09-16T15:57:19.192 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.75314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61302.log 2024-09-16T15:57:19.192 INFO:teuthology.orchestra.run.smithi151.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.75314.log.gz 2024-09-16T15:57:19.194 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.77969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45792.log 2024-09-16T15:57:19.194 INFO:teuthology.orchestra.run.smithi151.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.77969.log.gz 2024-09-16T15:57:19.195 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.61302.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.61302.log.gz 2024-09-16T15:57:19.195 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67288.log 2024-09-16T15:57:19.197 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.45792.log: 87.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73990.log 2024-09-16T15:57:19.197 INFO:teuthology.orchestra.run.smithi151.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45792.log.gz 2024-09-16T15:57:19.198 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.67288.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60362.log 2024-09-16T15:57:19.198 INFO:teuthology.orchestra.run.smithi151.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.67288.log.gz 2024-09-16T15:57:19.199 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.73990.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.73990.log.gz 2024-09-16T15:57:19.199 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60236.log 2024-09-16T15:57:19.200 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.60362.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.60362.log.gz 2024-09-16T15:57:19.201 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71042.log 2024-09-16T15:57:19.202 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.60236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52459.log 2024-09-16T15:57:19.202 INFO:teuthology.orchestra.run.smithi151.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.60236.log.gz 2024-09-16T15:57:19.203 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.71042.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.71042.log.gz 2024-09-16T15:57:19.203 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66223.log 2024-09-16T15:57:19.204 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.52459.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.52459.log.gz 2024-09-16T15:57:19.204 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63813.log 2024-09-16T15:57:19.206 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.66223.log: 87.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.66223.log.gz 2024-09-16T15:57:19.206 INFO:teuthology.orchestra.run.smithi151.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.78794.log 2024-09-16T15:57:19.207 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.63813.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.63813.log.gz 2024-09-16T15:57:19.208 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42377.log 2024-09-16T15:57:19.209 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.78794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53777.log 2024-09-16T15:57:19.209 INFO:teuthology.orchestra.run.smithi151.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.78794.log.gz 2024-09-16T15:57:19.210 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.42377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63265.log 2024-09-16T15:57:19.210 INFO:teuthology.orchestra.run.smithi151.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.42377.log.gz 2024-09-16T15:57:19.211 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.53777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82475.log 2024-09-16T15:57:19.211 INFO:teuthology.orchestra.run.smithi151.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53777.log.gz 2024-09-16T15:57:19.213 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.63265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75667.log 2024-09-16T15:57:19.213 INFO:teuthology.orchestra.run.smithi151.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.63265.log.gz 2024-09-16T15:57:19.214 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.82475.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.82475.log.gz 2024-09-16T15:57:19.214 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35936.log 2024-09-16T15:57:19.215 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.75667.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.75667.log.gz 2024-09-16T15:57:19.215 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76609.log 2024-09-16T15:57:19.217 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.35936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82834.log 2024-09-16T15:57:19.217 INFO:teuthology.orchestra.run.smithi151.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.35936.log.gz 2024-09-16T15:57:19.218 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.76609.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.76609.log.gz 2024-09-16T15:57:19.218 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56498.log 2024-09-16T15:57:19.219 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.82834.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.82834.log.gz 2024-09-16T15:57:19.220 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74449.log 2024-09-16T15:57:19.221 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.56498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36123.log 2024-09-16T15:57:19.221 INFO:teuthology.orchestra.run.smithi151.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56498.log.gz 2024-09-16T15:57:19.223 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.74449.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.74449.log.gz 2024-09-16T15:57:19.223 INFO:teuthology.orchestra.run.smithi151.stderr:gzip/var/log/ceph/ceph-client.admin.36123.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.49534.log 2024-09-16T15:57:19.224 INFO:teuthology.orchestra.run.smithi151.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.36123.log.gz 2024-09-16T15:57:19.224 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51422.log 2024-09-16T15:57:19.226 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.49534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76483.log 2024-09-16T15:57:19.226 INFO:teuthology.orchestra.run.smithi151.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49534.log.gz 2024-09-16T15:57:19.227 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.51422.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51422.log.gz 2024-09-16T15:57:19.227 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56729.log 2024-09-16T15:57:19.228 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.76483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66391.log 2024-09-16T15:57:19.228 INFO:teuthology.orchestra.run.smithi151.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.76483.log.gz 2024-09-16T15:57:19.230 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.56729.log: gzip -5 --verbose 87.6% -- /var/log/ceph/ceph-client.admin.43527.log 2024-09-16T15:57:19.230 INFO:teuthology.orchestra.run.smithi151.stderr: -- replaced with /var/log/ceph/ceph-client.admin.56729.log.gz 2024-09-16T15:57:19.231 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.66391.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.66391.log.gz 2024-09-16T15:57:19.231 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43889.log 2024-09-16T15:57:19.232 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.43527.log: 87.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74636.log 2024-09-16T15:57:19.232 INFO:teuthology.orchestra.run.smithi151.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43527.log.gz 2024-09-16T15:57:19.234 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.43889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74804.log 2024-09-16T15:57:19.234 INFO:teuthology.orchestra.run.smithi151.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43889.log.gz 2024-09-16T15:57:19.235 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.74636.log: gzip -5 --verbose -- 87.1% -- replaced with /var/log/ceph/ceph-client.admin.74636.log.gz 2024-09-16T15:57:19.235 INFO:teuthology.orchestra.run.smithi151.stderr: /var/log/ceph/ceph-client.admin.77054.log 2024-09-16T15:57:19.236 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.74804.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.74804.log.gz 2024-09-16T15:57:19.236 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50876.log 2024-09-16T15:57:19.238 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.77054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43411.log 2024-09-16T15:57:19.238 INFO:teuthology.orchestra.run.smithi151.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.77054.log.gz 2024-09-16T15:57:19.239 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.50876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70228.log 2024-09-16T15:57:19.239 INFO:teuthology.orchestra.run.smithi151.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50876.log.gz 2024-09-16T15:57:19.240 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.43411.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43411.log.gz 2024-09-16T15:57:19.241 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47051.log 2024-09-16T15:57:19.241 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.70228.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.70228.log.gz 2024-09-16T15:57:19.242 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54933.log 2024-09-16T15:57:19.243 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.47051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47237.log 2024-09-16T15:57:19.243 INFO:teuthology.orchestra.run.smithi151.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47051.log.gz 2024-09-16T15:57:19.245 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.54933.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54933.log.gz 2024-09-16T15:57:19.245 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64350.log 2024-09-16T15:57:19.245 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.47237.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47237.log.gz 2024-09-16T15:57:19.246 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61050.log 2024-09-16T15:57:19.248 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.64350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61260.log 2024-09-16T15:57:19.248 INFO:teuthology.orchestra.run.smithi151.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.64350.log.gz 2024-09-16T15:57:19.249 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.61050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78663.log 2024-09-16T15:57:19.249 INFO:teuthology.orchestra.run.smithi151.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.61050.log.gz 2024-09-16T15:57:19.250 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.61260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82265.log 2024-09-16T15:57:19.250 INFO:teuthology.orchestra.run.smithi151.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.61260.log.gz 2024-09-16T15:57:19.251 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.78663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79088.log 2024-09-16T15:57:19.251 INFO:teuthology.orchestra.run.smithi151.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.78663.log.gz 2024-09-16T15:57:19.253 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.82265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82139.log 2024-09-16T15:57:19.253 INFO:teuthology.orchestra.run.smithi151.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.82265.log.gz 2024-09-16T15:57:19.254 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.79088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76567.log 2024-09-16T15:57:19.254 INFO:teuthology.orchestra.run.smithi151.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.79088.log.gz 2024-09-16T15:57:19.255 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.82139.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.82139.log.gz 2024-09-16T15:57:19.256 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73367.log/var/log/ceph/ceph-client.admin.76567.log: 2024-09-16T15:57:19.256 INFO:teuthology.orchestra.run.smithi151.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.76567.log.gz 2024-09-16T15:57:19.257 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41619.log 2024-09-16T15:57:19.259 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.73367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62098.log 2024-09-16T15:57:19.259 INFO:teuthology.orchestra.run.smithi151.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.73367.log.gz 2024-09-16T15:57:19.260 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.41619.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41619.log.gz 2024-09-16T15:57:19.260 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57042.log 2024-09-16T15:57:19.261 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.62098.log: 87.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68249.log 2024-09-16T15:57:19.261 INFO:teuthology.orchestra.run.smithi151.stderr: -- replaced with /var/log/ceph/ceph-client.admin.62098.log.gz 2024-09-16T15:57:19.263 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.57042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49282.log 2024-09-16T15:57:19.263 INFO:teuthology.orchestra.run.smithi151.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57042.log.gz 2024-09-16T15:57:19.264 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.68249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36198.log 2024-09-16T15:57:19.264 INFO:teuthology.orchestra.run.smithi151.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.68249.log.gz 2024-09-16T15:57:19.265 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.49282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74846.log 2024-09-16T15:57:19.265 INFO:teuthology.orchestra.run.smithi151.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49282.log.gz 2024-09-16T15:57:19.267 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.36198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62896.log 2024-09-16T15:57:19.267 INFO:teuthology.orchestra.run.smithi151.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.36198.log.gz 2024-09-16T15:57:19.268 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.74846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59848.log 2024-09-16T15:57:19.268 INFO:teuthology.orchestra.run.smithi151.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.74846.log.gz 2024-09-16T15:57:19.269 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.62896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68631.log 2024-09-16T15:57:19.269 INFO:teuthology.orchestra.run.smithi151.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.62896.log.gz 2024-09-16T15:57:19.270 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.59848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65719.log 2024-09-16T15:57:19.271 INFO:teuthology.orchestra.run.smithi151.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.59848.log.gz 2024-09-16T15:57:19.272 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.68631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68757.log 2024-09-16T15:57:19.272 INFO:teuthology.orchestra.run.smithi151.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.68631.log.gz 2024-09-16T15:57:19.273 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.65719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69219.log 2024-09-16T15:57:19.273 INFO:teuthology.orchestra.run.smithi151.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.65719.log.gz 2024-09-16T15:57:19.275 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.68757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.79640.log 2024-09-16T15:57:19.275 INFO:teuthology.orchestra.run.smithi151.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.68757.log.gz 2024-09-16T15:57:19.276 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.69219.log: gzip 87.5% -- replaced with /var/log/ceph/ceph-client.admin.69219.log.gz 2024-09-16T15:57:19.276 INFO:teuthology.orchestra.run.smithi151.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.47009.log 2024-09-16T15:57:19.277 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.0.79640.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.79640.log.gz 2024-09-16T15:57:19.277 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79046.log 2024-09-16T15:57:19.278 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.47009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58762.log 2024-09-16T15:57:19.278 INFO:teuthology.orchestra.run.smithi151.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47009.log.gz 2024-09-16T15:57:19.280 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.79046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76063.log 2024-09-16T15:57:19.280 INFO:teuthology.orchestra.run.smithi151.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.79046.log.gz 2024-09-16T15:57:19.281 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.58762.log: gzip -5 --verbose -- 87.5% /var/log/ceph/ceph-client.admin.48739.log 2024-09-16T15:57:19.281 INFO:teuthology.orchestra.run.smithi151.stderr: -- replaced with /var/log/ceph/ceph-client.admin.58762.log.gz 2024-09-16T15:57:19.283 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.76063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46669.log 2024-09-16T15:57:19.283 INFO:teuthology.orchestra.run.smithi151.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.76063.log.gz 2024-09-16T15:57:19.283 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.48739.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48739.log.gz 2024-09-16T15:57:19.284 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71774.log 2024-09-16T15:57:19.285 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.46669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77239.log 2024-09-16T15:57:19.285 INFO:teuthology.orchestra.run.smithi151.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46669.log.gz 2024-09-16T15:57:19.286 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.71774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56354.log 2024-09-16T15:57:19.286 INFO:teuthology.orchestra.run.smithi151.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.71774.log.gz 2024-09-16T15:57:19.288 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.77239.log: 87.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57336.log 2024-09-16T15:57:19.288 INFO:teuthology.orchestra.run.smithi151.stderr: -- replaced with /var/log/ceph/ceph-client.admin.77239.log.gz 2024-09-16T15:57:19.289 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.56354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53441.log 2024-09-16T15:57:19.289 INFO:teuthology.orchestra.run.smithi151.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.56354.log.gz 2024-09-16T15:57:19.290 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.57336.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.57336.log.gz 2024-09-16T15:57:19.291 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70186.log 2024-09-16T15:57:19.291 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.53441.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.53441.log.gz 2024-09-16T15:57:19.292 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55371.log 2024-09-16T15:57:19.293 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.70186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45834.log 2024-09-16T15:57:19.293 INFO:teuthology.orchestra.run.smithi151.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.70186.log.gz 2024-09-16T15:57:19.294 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.55371.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55371.log.gz 2024-09-16T15:57:19.295 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64789.log 2024-09-16T15:57:19.296 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.45834.log: 87.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63022.log 2024-09-16T15:57:19.296 INFO:teuthology.orchestra.run.smithi151.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45834.log.gz 2024-09-16T15:57:19.297 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.64789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67643.log 2024-09-16T15:57:19.297 INFO:teuthology.orchestra.run.smithi151.stderr: 83.8% -- replaced with /var/log/ceph/ceph-client.admin.64789.log.gz 2024-09-16T15:57:19.298 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.63022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54573.log 2024-09-16T15:57:19.298 INFO:teuthology.orchestra.run.smithi151.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.63022.log.gz 2024-09-16T15:57:19.300 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.67643.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.67643.log.gz 2024-09-16T15:57:19.300 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.54573.log: -- /var/log/ceph/ceph-client.admin.53128.log 2024-09-16T15:57:19.301 INFO:teuthology.orchestra.run.smithi151.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.54573.log.gz 2024-09-16T15:57:19.301 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57084.log 2024-09-16T15:57:19.303 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.53128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59900.log 2024-09-16T15:57:19.303 INFO:teuthology.orchestra.run.smithi151.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.53128.log.gz 2024-09-16T15:57:19.304 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.57084.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.57084.log.gz 2024-09-16T15:57:19.305 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54345.log 2024-09-16T15:57:19.305 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.59900.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.59900.log.gz 2024-09-16T15:57:19.306 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75583.log 2024-09-16T15:57:19.307 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.54345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58888.log 2024-09-16T15:57:19.307 INFO:teuthology.orchestra.run.smithi151.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54345.log.gz 2024-09-16T15:57:19.309 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.75583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35812.log 2024-09-16T15:57:19.309 INFO:teuthology.orchestra.run.smithi151.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.75583.log.gz 2024-09-16T15:57:19.310 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.58888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78400.log 2024-09-16T15:57:19.310 INFO:teuthology.orchestra.run.smithi151.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.58888.log.gz 2024-09-16T15:57:19.311 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.35812.log: 79.5% -- replaced with /var/log/ceph/ceph-client.admin.35812.log.gz 2024-09-16T15:57:19.311 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49702.log 2024-09-16T15:57:19.312 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.78400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46799.log 2024-09-16T15:57:19.313 INFO:teuthology.orchestra.run.smithi151.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.78400.log.gz 2024-09-16T15:57:19.314 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.49702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68421.log 2024-09-16T15:57:19.314 INFO:teuthology.orchestra.run.smithi151.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49702.log.gz 2024-09-16T15:57:19.315 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.46799.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.46799.log.gz 2024-09-16T15:57:19.316 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-16T15:57:19.316 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.68421.log: 87.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76894.log 2024-09-16T15:57:19.317 INFO:teuthology.orchestra.run.smithi151.stderr: -- replaced with /var/log/ceph/ceph-client.admin.68421.log.gz 2024-09-16T15:57:19.318 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61842.log 2024-09-16T15:57:19.319 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.76894.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.76894.log.gz 2024-09-16T15:57:19.320 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.72596.log 2024-09-16T15:57:19.320 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.61842.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.61842.log.gz 2024-09-16T15:57:19.321 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76970.log 2024-09-16T15:57:19.322 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.0.72596.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.72596.log.gz 2024-09-16T15:57:19.323 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72846.log 2024-09-16T15:57:19.324 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.76970.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.76970.log.gz 2024-09-16T15:57:19.324 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55329.log 2024-09-16T15:57:19.325 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.72846.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.72846.log.gz 2024-09-16T15:57:19.326 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79802.log 2024-09-16T15:57:19.327 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.55329.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55329.log.gz 2024-09-16T15:57:19.328 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-09-16T15:57:19.329 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.79802.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.79802.log.gz 2024-09-16T15:57:19.331 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74365.log 2024-09-16T15:57:19.334 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70312.log 2024-09-16T15:57:19.334 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.74365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70958.log 2024-09-16T15:57:19.334 INFO:teuthology.orchestra.run.smithi151.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.74365.log.gz 2024-09-16T15:57:19.336 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.70312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47135.log 2024-09-16T15:57:19.337 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.70958.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.70958.log.gz 2024-09-16T15:57:19.338 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42209.log 2024-09-16T15:57:19.339 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.47135.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.70312.log.gz 2024-09-16T15:57:19.339 INFO:teuthology.orchestra.run.smithi151.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47135.log.gz 2024-09-16T15:57:19.341 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.42209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41332.log 2024-09-16T15:57:19.343 INFO:teuthology.orchestra.run.smithi151.stderr: 87.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82960.log 2024-09-16T15:57:19.343 INFO:teuthology.orchestra.run.smithi151.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42209.log.gz 2024-09-16T15:57:19.345 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.41332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46416.log 2024-09-16T15:57:19.346 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.82960.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41332.log.gz 2024-09-16T15:57:19.349 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.46416.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.82960.log.gz 2024-09-16T15:57:19.350 INFO:teuthology.orchestra.run.smithi151.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46416.log.gz 2024-09-16T15:57:19.350 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63855.log 2024-09-16T15:57:19.352 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72174.log 2024-09-16T15:57:19.354 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.63855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51974.log 2024-09-16T15:57:19.355 INFO:teuthology.orchestra.run.smithi151.stderr: 86.8%/var/log/ceph/ceph-client.admin.72174.log: -- replaced with /var/log/ceph/ceph-client.admin.63855.log.gz 2024-09-16T15:57:19.356 INFO:teuthology.orchestra.run.smithi151.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.72174.log.gz 2024-09-16T15:57:19.358 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72408.log 2024-09-16T15:57:19.359 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.51974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75495.log 2024-09-16T15:57:19.361 INFO:teuthology.orchestra.run.smithi151.stderr: 87.1%/var/log/ceph/ceph-client.admin.72408.log: -- replaced with /var/log/ceph/ceph-client.admin.51974.log.gz 2024-09-16T15:57:19.363 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82181.log 2024-09-16T15:57:19.363 INFO:teuthology.orchestra.run.smithi151.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.72408.log.gz 2024-09-16T15:57:19.364 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.75495.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.75495.log.gz 2024-09-16T15:57:19.367 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.82181.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.82181.log.gz 2024-09-16T15:57:19.368 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77323.log 2024-09-16T15:57:19.369 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78053.log 2024-09-16T15:57:19.371 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.77323.log: 87.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66181.log 2024-09-16T15:57:19.372 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.78053.log: -- replaced with /var/log/ceph/ceph-client.admin.77323.log.gz 2024-09-16T15:57:19.373 INFO:teuthology.orchestra.run.smithi151.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.78053.log.gz 2024-09-16T15:57:19.374 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60194.log 2024-09-16T15:57:19.376 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.66181.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.66181.log.gz 2024-09-16T15:57:19.379 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.60194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50540.log 2024-09-16T15:57:19.380 INFO:teuthology.orchestra.run.smithi151.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.60194.log.gz 2024-09-16T15:57:19.381 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.55125.log 2024-09-16T15:57:19.386 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.50540.log: /var/log/ceph/ceph-client.0.55125.log: 87.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59611.log 2024-09-16T15:57:19.386 INFO:teuthology.orchestra.run.smithi151.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50540.log.gz 2024-09-16T15:57:19.388 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65313.log 2024-09-16T15:57:19.390 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.59611.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.55125.log.gz 2024-09-16T15:57:19.392 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.65313.log: 84.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61884.log 2024-09-16T15:57:19.392 INFO:teuthology.orchestra.run.smithi151.stderr: -- replaced with /var/log/ceph/ceph-client.admin.59611.log.gz 2024-09-16T15:57:19.394 INFO:teuthology.orchestra.run.smithi151.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.65313.log.gz 2024-09-16T15:57:19.394 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.61884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43721.log 2024-09-16T15:57:19.396 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45876.log 2024-09-16T15:57:19.400 INFO:teuthology.orchestra.run.smithi151.stderr: 87.7%/var/log/ceph/ceph-client.admin.43721.log: -- replaced with /var/log/ceph/ceph-client.admin.61884.log.gz/var/log/ceph/ceph-client.admin.45876.log: 2024-09-16T15:57:19.401 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose 87.2% -- /var/log/ceph/ceph-client.admin.80764.log 2024-09-16T15:57:19.402 INFO:teuthology.orchestra.run.smithi151.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43721.log.gz 2024-09-16T15:57:19.402 INFO:teuthology.orchestra.run.smithi151.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45876.log.gz 2024-09-16T15:57:19.404 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43177.log 2024-09-16T15:57:19.408 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.43177.log: /var/log/ceph/ceph-client.admin.80764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66307.log 2024-09-16T15:57:19.409 INFO:teuthology.orchestra.run.smithi151.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.80764.log.gz 2024-09-16T15:57:19.410 INFO:teuthology.orchestra.run.smithi151.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43177.log.gz 2024-09-16T15:57:19.410 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44153.log 2024-09-16T15:57:19.412 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.44153.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44153.log.gz 2024-09-16T15:57:19.413 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.66307.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.66307.log.gz 2024-09-16T15:57:19.414 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76693.log 2024-09-16T15:57:19.416 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.76693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36009.log 2024-09-16T15:57:19.418 INFO:teuthology.orchestra.run.smithi151.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.76693.log.gz 2024-09-16T15:57:19.420 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/ceph-client.admin.36009.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.36009.log.gz 2024-09-16T15:57:19.996 INFO:teuthology.orchestra.run.smithi167.stderr: 93.3% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-09-16T15:57:20.143 INFO:teuthology.orchestra.run.smithi151.stderr: 90.3% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-16T15:57:21.473 INFO:teuthology.orchestra.run.smithi151.stderr: 93.5% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-09-16T15:57:22.717 INFO:teuthology.orchestra.run.smithi167.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-16T15:57:22.844 INFO:teuthology.orchestra.run.smithi167.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-16T15:57:22.875 INFO:teuthology.orchestra.run.smithi167.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-16T15:57:23.453 INFO:teuthology.orchestra.run.smithi151.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-16T15:57:23.464 INFO:teuthology.orchestra.run.smithi151.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-16T15:57:23.530 INFO:teuthology.orchestra.run.smithi167.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-16T15:57:24.121 INFO:teuthology.orchestra.run.smithi151.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-16T15:57:24.185 INFO:teuthology.orchestra.run.smithi151.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-16T15:57:24.348 INFO:teuthology.orchestra.run.smithi151.stderr: 93.4% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-09-16T15:57:24.352 INFO:teuthology.orchestra.run.smithi151.stderr: 2024-09-16T15:57:24.352 INFO:teuthology.orchestra.run.smithi151.stderr:real 0m5.961s 2024-09-16T15:57:24.352 INFO:teuthology.orchestra.run.smithi151.stderr:user 0m27.020s 2024-09-16T15:57:24.352 INFO:teuthology.orchestra.run.smithi151.stderr:sys 0m4.280s 2024-09-16T15:57:25.073 INFO:teuthology.orchestra.run.smithi167.stderr: 92.8% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-16T15:57:25.077 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-16T15:57:25.078 INFO:teuthology.orchestra.run.smithi167.stderr:real 0m6.682s 2024-09-16T15:57:25.078 INFO:teuthology.orchestra.run.smithi167.stderr:user 0m25.587s 2024-09-16T15:57:25.078 INFO:teuthology.orchestra.run.smithi167.stderr:sys 0m2.213s 2024-09-16T15:57:25.078 INFO:tasks.ceph:Archiving logs... 2024-09-16T15:57:25.078 DEBUG:teuthology.misc:Transferring archived files from smithi151:/var/log/ceph to /home/teuthworker/archive/avan-2024-09-16_15:21:40-fs:volumes-wip-athakkar-testing-2024-09-16-1454-distro-default-smithi/7907354/remote/smithi151/log 2024-09-16T15:57:25.080 DEBUG:teuthology.orchestra.run.smithi151:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T15:57:28.829 DEBUG:teuthology.misc:Transferring archived files from smithi167:/var/log/ceph to /home/teuthworker/archive/avan-2024-09-16_15:21:40-fs:volumes-wip-athakkar-testing-2024-09-16-1454-distro-default-smithi/7907354/remote/smithi167/log 2024-09-16T15:57:28.829 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T15:57:31.925 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-16T15:57:31.932 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-16T15:57:31.933 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-16T15:57:31.938 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-16T15:57:32.050 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-16T15:57:32.058 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-16T15:57:32.059 DEBUG:teuthology.orchestra.run.smithi151:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T15:57:32.063 DEBUG:teuthology.orchestra.run.smithi167:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T15:57:32.094 INFO:teuthology.orchestra.run.smithi151.stderr:bash: line 1: ntpq: command not found 2024-09-16T15:57:32.096 INFO:teuthology.orchestra.run.smithi167.stderr:bash: line 1: ntpq: command not found 2024-09-16T15:57:32.101 INFO:teuthology.orchestra.run.smithi151.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T15:57:32.102 INFO:teuthology.orchestra.run.smithi151.stdout:=============================================================================== 2024-09-16T15:57:32.102 INFO:teuthology.orchestra.run.smithi151.stdout:^* hv01.front.sepia.ceph.com 3 6 377 30 -134us[ -129us] +/- 43ms 2024-09-16T15:57:32.102 INFO:teuthology.orchestra.run.smithi151.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 28 -252us[ -252us] +/- 92ms 2024-09-16T15:57:32.102 INFO:teuthology.orchestra.run.smithi151.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 32 +426us[ +430us] +/- 64ms 2024-09-16T15:57:32.102 INFO:teuthology.orchestra.run.smithi151.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T15:57:32.103 INFO:teuthology.orchestra.run.smithi167.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T15:57:32.103 INFO:teuthology.orchestra.run.smithi167.stdout:=============================================================================== 2024-09-16T15:57:32.103 INFO:teuthology.orchestra.run.smithi167.stdout:^* hv01.front.sepia.ceph.com 3 6 377 31 -104us[ -82us] +/- 43ms 2024-09-16T15:57:32.103 INFO:teuthology.orchestra.run.smithi167.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 32 -229us[ -207us] +/- 92ms 2024-09-16T15:57:32.103 INFO:teuthology.orchestra.run.smithi167.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 32 +449us[ +471us] +/- 64ms 2024-09-16T15:57:32.104 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T15:57:32.104 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-16T15:57:32.113 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-16T15:57:32.113 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-16T15:57:32.122 DEBUG:teuthology.orchestra.run.smithi151:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-16T15:57:32.285 DEBUG:teuthology.orchestra.run.smithi167:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-16T15:57:32.399 DEBUG:teuthology.orchestra.run.smithi151:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-16T15:57:32.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T15:57:32.461 DEBUG:teuthology.orchestra.run.smithi167:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-16T15:57:32.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T15:57:32.522 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-16T15:57:32.531 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-16T15:57:32.539 INFO:teuthology.task.internal:Duration was 1545.622111 seconds 2024-09-16T15:57:32.539 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-16T15:57:32.548 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-16T15:57:32.548 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T15:57:32.552 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T15:57:32.636 INFO:teuthology.orchestra.run.smithi151.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T15:57:32.639 INFO:teuthology.orchestra.run.smithi167.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T15:57:32.962 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-16T15:57:32.962 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi151.front.sepia.ceph.com 2024-09-16T15:57:32.963 DEBUG:teuthology.orchestra.run.smithi151:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T15:57:33.011 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi167.front.sepia.ceph.com 2024-09-16T15:57:33.011 DEBUG:teuthology.orchestra.run.smithi167:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T15:57:33.061 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-16T15:57:33.061 DEBUG:teuthology.orchestra.run.smithi151:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T15:57:33.065 DEBUG:teuthology.orchestra.run.smithi167:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T15:57:33.123 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-16T15:57:33.123 DEBUG:teuthology.orchestra.run.smithi151:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T15:57:33.127 DEBUG:teuthology.orchestra.run.smithi167:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T15:57:33.302 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-16T15:57:33.312 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-16T15:57:33.312 DEBUG:teuthology.orchestra.run.smithi151:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T15:57:33.368 DEBUG:teuthology.orchestra.run.smithi167:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T15:57:33.424 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-16T15:57:33.434 DEBUG:teuthology.orchestra.run.smithi151:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T15:57:33.438 DEBUG:teuthology.orchestra.run.smithi167:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T15:57:33.504 INFO:teuthology.orchestra.run.smithi151.stdout:kernel.core_pattern = core 2024-09-16T15:57:33.508 INFO:teuthology.orchestra.run.smithi167.stdout:kernel.core_pattern = core 2024-09-16T15:57:33.538 DEBUG:teuthology.orchestra.run.smithi151:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T15:57:33.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T15:57:33.571 DEBUG:teuthology.orchestra.run.smithi167:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T15:57:33.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T15:57:33.604 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-16T15:57:33.614 INFO:teuthology.task.internal:Transferring archived files... 2024-09-16T15:57:33.614 DEBUG:teuthology.misc:Transferring archived files from smithi151:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/avan-2024-09-16_15:21:40-fs:volumes-wip-athakkar-testing-2024-09-16-1454-distro-default-smithi/7907354/remote/smithi151 2024-09-16T15:57:33.615 DEBUG:teuthology.orchestra.run.smithi151:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T15:57:33.683 DEBUG:teuthology.misc:Transferring archived files from smithi167:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/avan-2024-09-16_15:21:40-fs:volumes-wip-athakkar-testing-2024-09-16-1454-distro-default-smithi/7907354/remote/smithi167 2024-09-16T15:57:33.683 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T15:57:33.746 INFO:teuthology.task.internal:Removing archive directory... 2024-09-16T15:57:33.746 DEBUG:teuthology.orchestra.run.smithi151:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T15:57:33.750 DEBUG:teuthology.orchestra.run.smithi167:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T15:57:33.785 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-16T15:57:33.794 INFO:teuthology.task.internal:Not uploading archives. 2024-09-16T15:57:33.795 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-16T15:57:33.804 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-16T15:57:33.804 DEBUG:teuthology.orchestra.run.smithi151:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T15:57:33.826 DEBUG:teuthology.orchestra.run.smithi167:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T15:57:33.856 INFO:teuthology.orchestra.run.smithi151.stdout: 393255 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 15:57 /home/ubuntu/cephtest 2024-09-16T15:57:33.860 INFO:teuthology.orchestra.run.smithi167.stdout: 393245 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 15:57 /home/ubuntu/cephtest 2024-09-16T15:57:33.863 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-16T15:57:33.872 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-16T15:57:33.895 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a client.1-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a description: fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/clone-progress}} duration: 1545.6221106052399 flavor: default owner: scheduled_avan@teuthology success: true 2024-09-16T15:57:33.895 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-16T15:57:33.995 INFO:teuthology.run:pass